2014-08-13 16:24:01.625505 7fd951da67a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 25220 2014-08-13 16:24:01.705860 7fd951da67a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 16:24:01.705890 7fd951da67a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-08-13 16:24:01.708680 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 16:24:01.708702 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 16:24:01.729696 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 16:24:01.730200 7fd951da67a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 16:24:01.845139 7fd951da67a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-13 16:24:01.864873 7fd951da67a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 16:24:01.907647 7fd951da67a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 16:24:01.917440 7fd951da67a0 1 journal close /var/lib/ceph/osd/ceph-4/journal 2014-08-13 16:24:01.948473 7fd951da67a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 16:24:01.950626 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 16:24:01.950651 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 16:24:01.966804 7fd951da67a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 16:24:01.966899 7fd951da67a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 16:24:01.971470 7fd951da67a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-13 16:24:01.976686 7fd951da67a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 16:24:01.981274 7fd951da67a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 16:24:02.117881 7fd951da67a0 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-13 16:24:02.169214 7fd951da67a0 0 osd.4 194338 crush map has features 2303210029056, adjusting msgr requires for clients 2014-08-13 16:24:02.169240 7fd951da67a0 0 osd.4 194338 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:02.169247 7fd951da67a0 0 osd.4 194338 crush map has features 2578087936000, adjusting msgr requires for osds 2014-08-13 16:24:02.169272 7fd951da67a0 0 osd.4 194338 load_pgs 2014-08-13 16:24:02.851452 7fd951da67a0 0 osd.4 194338 load_pgs opened 72 pgs 2014-08-13 16:24:02.860183 7fd93f3f2700 0 osd.4 194338 ignoring osdmap until we have initialized 2014-08-13 16:24:02.861162 7fd93f3f2700 0 osd.4 194338 ignoring osdmap until we have initialized 2014-08-13 16:24:02.935867 7fd92bd8d700 0 -- 192.168.1.115:6813/25220 >> 192.168.1.107:6802/9255 pipe(0x4245780 sd=73 :0 s=1 pgs=0 cs=0 l=0 c=0x4fdc620).fault with nothing to send, going to standby 2014-08-13 16:24:02.949300 7fd951da67a0 0 osd.4 194338 done with init, starting boot process 2014-08-13 16:24:03.302990 7fd93f3f2700 0 osd.4 194439 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:03.496632 7fd93f3f2700 0 osd.4 194446 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:03.605430 7fd93f3f2700 0 osd.4 194447 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:04.395673 7fd93dff0700 0 osd.4 194448 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:04.428685 7fd919771700 0 -- 192.168.1.115:6813/25220 >> 192.168.1.110:6800/41204 pipe(0x7bc6680 sd=280 :6813 s=0 pgs=0 cs=0 l=0 c=0x8300580).accept connect_seq 0 vs existing 0 state connecting 2014-08-13 16:24:05.396948 7fd93dff0700 0 osd.4 194449 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:07.081384 7fd9371e5700 0 log [WRN] : pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 16:24:07.083922 7fd9367e4700 0 log [WRN] : pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 16:24:07.393935 7fd9208e2700 0 -- 192.168.1.115:6813/25220 >> 192.168.1.110:6804/25590 pipe(0x8e88a00 sd=227 :43002 s=2 pgs=126 cs=1 l=0 c=0x78b0580).fault with nothing to send, going to standby 2014-08-13 16:24:24.688397 7fd93dff0700 0 osd.4 194450 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:25.188134 7fd9353e2700 0 log [INF] : 22.1b8 deep-scrub ok 2014-08-13 16:24:25.698262 7fd93dff0700 0 osd.4 194451 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:27.659382 7fd93dff0700 0 osd.4 194452 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:27.926371 7fd93f3f2700 0 osd.4 194453 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:28.928535 7fd93f3f2700 0 osd.4 194454 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:29.639835 7fd9353e2700 0 log [INF] : 22.6bd deep-scrub ok 2014-08-13 16:24:29.937470 7fd93dff0700 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 16:24:30.304280 7fd9371e5700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 16:24:30.309373 7fd9371e5700 -1 *** Caught signal (Floating point exception) ** in thread 7fd9371e5700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fd950d53710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fd950d4b9d1] 16: (clone()+0x6d) [0x7fd94fcdbb6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -10000> 2014-08-13 16:24:05.619697 7fd93dff0700 3 osd.4 194449 handle_osd_map epochs [194449,194449], i have 194449, src has [185139,194449] -9999> 2014-08-13 16:24:05.619717 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 5 ==== pg_notify(22.f3(15) epoch 194449) v5 ==== 1551+0+0 (390793866 0 0) 0x775d400 con 0x8da5120 -9998> 2014-08-13 16:24:05.619735 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619591, event: header_read, request: pg_notify(22.f3(15) epoch 194449) v5 -9997> 2014-08-13 16:24:05.619751 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619592, event: throttled, request: pg_notify(22.f3(15) epoch 194449) v5 -9996> 2014-08-13 16:24:05.619761 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619684, event: all_read, request: pg_notify(22.f3(15) epoch 194449) v5 -9995> 2014-08-13 16:24:05.619770 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619732, event: dispatched, request: pg_notify(22.f3(15) epoch 194449) v5 -9994> 2014-08-13 16:24:05.619781 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619781, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194449) v5 -9993> 2014-08-13 16:24:05.619794 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619794, event: started, request: pg_notify(22.f3(15) epoch 194449) v5 -9992> 2014-08-13 16:24:05.619829 7fd93dff0700 5 -- op tracker -- , seq: 626, time: 2014-08-13 16:24:05.619829, event: done, request: pg_notify(22.f3(15) epoch 194449) v5 -9991> 2014-08-13 16:24:05.621605 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 4 ==== osd_map(194449..194449 src has 185139..194449) v3 ==== 454+0+0 (3409739995 0 0) 0x7e7b600 con 0x7379ce0 -9990> 2014-08-13 16:24:05.621655 7fd93dff0700 3 osd.4 194449 handle_osd_map epochs [194449,194449], i have 194449, src has [185139,194449] -9989> 2014-08-13 16:24:05.621673 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 5 ==== pg_notify(24.181(25) epoch 194449) v5 ==== 2159+0+0 (3528692128 0 0) 0x775e3c0 con 0x7379ce0 -9988> 2014-08-13 16:24:05.621694 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621548, event: header_read, request: pg_notify(24.181(25) epoch 194449) v5 -9987> 2014-08-13 16:24:05.621709 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621549, event: throttled, request: pg_notify(24.181(25) epoch 194449) v5 -9986> 2014-08-13 16:24:05.621719 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621657, event: all_read, request: pg_notify(24.181(25) epoch 194449) v5 -9985> 2014-08-13 16:24:05.621729 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621691, event: dispatched, request: pg_notify(24.181(25) epoch 194449) v5 -9984> 2014-08-13 16:24:05.621740 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621739, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194449) v5 -9983> 2014-08-13 16:24:05.621752 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621752, event: started, request: pg_notify(24.181(25) epoch 194449) v5 -9982> 2014-08-13 16:24:05.621786 7fd93dff0700 5 -- op tracker -- , seq: 627, time: 2014-08-13 16:24:05.621785, event: done, request: pg_notify(24.181(25) epoch 194449) v5 -9981> 2014-08-13 16:24:05.652947 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.652672) v2 ==== 47+0+0 (1353673141 0 0) 0x7d03100 con 0x7d2f0c0 -9980> 2014-08-13 16:24:05.653001 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.652672) v2 -- ?+0 0x7d06900 con 0x7d2f0c0 -9979> 2014-08-13 16:24:05.653086 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.652672) v2 ==== 47+0+0 (1353673141 0 0) 0x7d01180 con 0x7d2dc20 -9978> 2014-08-13 16:24:05.653131 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.652672) v2 -- ?+0 0x7d05b00 con 0x7d2dc20 -9977> 2014-08-13 16:24:05.682630 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.682217) v2 ==== 47+0+0 (458340757 0 0) 0x7d06900 con 0x7ebf380 -9976> 2014-08-13 16:24:05.682684 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.682217) v2 -- ?+0 0x7d01180 con 0x7ebf380 -9975> 2014-08-13 16:24:05.682661 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.682217) v2 ==== 47+0+0 (458340757 0 0) 0x7d05b00 con 0x7ebfe80 -9974> 2014-08-13 16:24:05.682755 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.682217) v2 -- ?+0 0x7d03100 con 0x7ebfe80 -9973> 2014-08-13 16:24:05.833144 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.833030) v2 ==== 47+0+0 (3803373985 0 0) 0x7d04600 con 0x7ab8420 -9972> 2014-08-13 16:24:05.833212 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.833030) v2 -- ?+0 0x7d06900 con 0x7ab8420 -9971> 2014-08-13 16:24:05.833205 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.833030) v2 ==== 47+0+0 (3803373985 0 0) 0x7d01f80 con 0x7ab8000 -9970> 2014-08-13 16:24:05.833262 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.833030) v2 -- ?+0 0x7d05b00 con 0x7ab8000 -9969> 2014-08-13 16:24:05.852732 7fd9385e7700 10 monclient: tick -9968> 2014-08-13 16:24:05.852805 7fd9385e7700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 16:23:35.852803) -9967> 2014-08-13 16:24:05.852819 7fd9385e7700 10 monclient: renew subs? (now: 2014-08-13 16:24:05.852819; renew after: 2014-08-13 16:26:33.460427) -- no -9966> 2014-08-13 16:24:05.882349 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.882269) v2 ==== 47+0+0 (1575149205 0 0) 0x7d055c0 con 0x7d2cba0 -9965> 2014-08-13 16:24:05.882374 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.882269) v2 ==== 47+0+0 (1575149205 0 0) 0x7d00700 con 0x7d28420 -9964> 2014-08-13 16:24:05.882403 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.882269) v2 -- ?+0 0x7d01f80 con 0x7d2cba0 -9963> 2014-08-13 16:24:05.882471 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.882269) v2 -- ?+0 0x7d04600 con 0x7d28420 -9962> 2014-08-13 16:24:05.933171 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.933250) v2 ==== 47+0+0 (1648375419 0 0) 0x7d02140 con 0x7d29fa0 -9961> 2014-08-13 16:24:05.933223 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.933250) v2 -- ?+0 0x7d055c0 con 0x7d29fa0 -9960> 2014-08-13 16:24:05.933230 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:05.933250) v2 ==== 47+0+0 (1648375419 0 0) 0x7d03f00 con 0x7d2b700 -9959> 2014-08-13 16:24:05.933282 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:05.933250) v2 -- ?+0 0x7d00700 con 0x7d2b700 -9958> 2014-08-13 16:24:06.065439 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.064858) v2 ==== 47+0+0 (2842064475 0 0) 0x7d07380 con 0x7ebce60 -9957> 2014-08-13 16:24:06.065465 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.064858) v2 ==== 47+0+0 (2842064475 0 0) 0x7d01500 con 0x7ebb860 -9956> 2014-08-13 16:24:06.065493 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.064858) v2 -- ?+0 0x7d02140 con 0x7ebce60 -9955> 2014-08-13 16:24:06.065542 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.064858) v2 -- ?+0 0x7d03f00 con 0x7ebb860 -9954> 2014-08-13 16:24:06.183717 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.183112) v2 ==== 47+0+0 (820336254 0 0) 0x7d03f00 con 0x7d2c360 -9953> 2014-08-13 16:24:06.183736 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.183112) v2 ==== 47+0+0 (820336254 0 0) 0x7d02140 con 0x7d2ac00 -9952> 2014-08-13 16:24:06.183767 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.183112) v2 -- ?+0 0x7d07380 con 0x7d2c360 -9951> 2014-08-13 16:24:06.183842 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.183112) v2 -- ?+0 0x7d01500 con 0x7d2ac00 -9950> 2014-08-13 16:24:06.335048 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.334945) v2 ==== 47+0+0 (1504620102 0 0) 0x7d05940 con 0x7ab8420 -9949> 2014-08-13 16:24:06.335105 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.334945) v2 -- ?+0 0x7d02140 con 0x7ab8420 -9948> 2014-08-13 16:24:06.335106 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.334945) v2 ==== 47+0+0 (1504620102 0 0) 0x7d06580 con 0x7ab8000 -9947> 2014-08-13 16:24:06.335151 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.334945) v2 -- ?+0 0x7d03f00 con 0x7ab8000 -9946> 2014-08-13 16:24:06.342180 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.342294) v2 ==== 47+0+0 (3287714931 0 0) 0x8c8a680 con 0x7ebd120 -9945> 2014-08-13 16:24:06.342201 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.342294) v2 ==== 47+0+0 (3287714931 0 0) 0x8c8f000 con 0x7ebbb20 -9944> 2014-08-13 16:24:06.342230 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.342294) v2 -- ?+0 0x7d06580 con 0x7ebd120 -9943> 2014-08-13 16:24:06.342290 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.342294) v2 -- ?+0 0x7d05940 con 0x7ebbb20 -9942> 2014-08-13 16:24:06.383978 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.383769) v2 ==== 47+0+0 (3706363139 0 0) 0x7d05940 con 0x7eba3c0 -9941> 2014-08-13 16:24:06.384027 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.383769) v2 -- ?+0 0x8c8a680 con 0x7eba3c0 -9940> 2014-08-13 16:24:06.384062 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.383769) v2 ==== 47+0+0 (3706363139 0 0) 0x7d06580 con 0x7eb8f20 -9939> 2014-08-13 16:24:06.384115 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.383769) v2 -- ?+0 0x8c8f000 con 0x7eb8f20 -9938> 2014-08-13 16:24:06.435482 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.435418) v2 ==== 47+0+0 (1888727501 0 0) 0x8c89180 con 0x7d29fa0 -9937> 2014-08-13 16:24:06.435539 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.435418) v2 -- ?+0 0x7d05940 con 0x7d29fa0 -9936> 2014-08-13 16:24:06.435573 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.435418) v2 ==== 47+0+0 (1888727501 0 0) 0x8c88e00 con 0x7d2b700 -9935> 2014-08-13 16:24:06.435624 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.435418) v2 -- ?+0 0x7d06580 con 0x7d2b700 -9934> 2014-08-13 16:24:06.454506 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.454308) v2 ==== 47+0+0 (146228703 0 0) 0x7d00700 con 0x7cae880 -9933> 2014-08-13 16:24:06.454557 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.454308) v2 -- ?+0 0x8c88e00 con 0x7cae880 -9932> 2014-08-13 16:24:06.454548 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.454308) v2 ==== 47+0+0 (146228703 0 0) 0x7d06580 con 0x7cae040 -9931> 2014-08-13 16:24:06.454602 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.454308) v2 -- ?+0 0x8c89180 con 0x7cae040 -9930> 2014-08-13 16:24:06.505210 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.505224) v2 ==== 47+0+0 (2934476285 0 0) 0x7d05940 con 0x7d2a260 -9929> 2014-08-13 16:24:06.505280 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.505224) v2 -- ?+0 0x7d06580 con 0x7d2a260 -9928> 2014-08-13 16:24:06.505216 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.505224) v2 ==== 47+0+0 (2934476285 0 0) 0x8c8af40 con 0x7d28160 -9927> 2014-08-13 16:24:06.505441 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.505224) v2 -- ?+0 0x7d00700 con 0x7d28160 -9926> 2014-08-13 16:24:06.530190 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.530109) v2 ==== 47+0+0 (3716035947 0 0) 0x7d07540 con 0x7ffe880 -9925> 2014-08-13 16:24:06.530240 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.530109) v2 -- ?+0 0x8c8af40 con 0x7ffe880 -9924> 2014-08-13 16:24:06.530376 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.530109) v2 ==== 47+0+0 (3716035947 0 0) 0x7d00700 con 0x7edbb20 -9923> 2014-08-13 16:24:06.530415 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.530109) v2 -- ?+0 0x7d05940 con 0x7edbb20 -9922> 2014-08-13 16:24:06.534254 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.533939) v2 ==== 47+0+0 (1085930152 0 0) 0x8c8cd00 con 0x7eb8c60 -9921> 2014-08-13 16:24:06.534301 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.533939) v2 -- ?+0 0x7d07540 con 0x7eb8c60 -9920> 2014-08-13 16:24:06.534326 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.533939) v2 ==== 47+0+0 (1085930152 0 0) 0x8c8d080 con 0x7eba100 -9919> 2014-08-13 16:24:06.534353 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.533939) v2 -- ?+0 0x7d00700 con 0x7eba100 -9918> 2014-08-13 16:24:06.700180 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.699785) v2 ==== 47+0+0 (197525998 0 0) 0x8c8abc0 con 0x7eda940 -9917> 2014-08-13 16:24:06.700192 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.699785) v2 ==== 47+0+0 (197525998 0 0) 0x7d06580 con 0x7edc8e0 -9916> 2014-08-13 16:24:06.700229 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.699785) v2 -- ?+0 0x8c8cd00 con 0x7eda940 -9915> 2014-08-13 16:24:06.700323 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.699785) v2 -- ?+0 0x8c8d080 con 0x7edc8e0 -9914> 2014-08-13 16:24:06.785474 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.784868) v2 ==== 47+0+0 (3672594669 0 0) 0x8c8c600 con 0x7ebfe80 -9913> 2014-08-13 16:24:06.785527 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.784868) v2 -- ?+0 0x7d06580 con 0x7ebfe80 -9912> 2014-08-13 16:24:06.785524 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:06.784868) v2 ==== 47+0+0 (3672594669 0 0) 0x8c896c0 con 0x7ebf380 -9911> 2014-08-13 16:24:06.785573 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:06.784868) v2 -- ?+0 0x8c8abc0 con 0x7ebf380 -9910> 2014-08-13 16:24:07.014955 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194448:24.2a1) v4 -- ?+0 0x4f89500 -9909> 2014-08-13 16:24:07.016039 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194448:24.33a) v4 -- ?+0 0x7d03100 -9908> 2014-08-13 16:24:07.016387 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.17 192.168.1.103:6811/37658 13 ==== pg_info(1 pgs e194449:24.2a1) v4 ==== 748+0+0 (1821417065 0 0) 0x4f89500 con 0x4fde1a0 -9907> 2014-08-13 16:24:07.016414 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.325 192.168.1.110:6804/25590 -- pg_info(1 pgs e194448:24.34) v4 -- ?+0 0x7d01180 -9906> 2014-08-13 16:24:07.016428 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016233, event: header_read, request: pg_info(1 pgs e194449:24.2a1) v4 -9905> 2014-08-13 16:24:07.016446 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016236, event: throttled, request: pg_info(1 pgs e194449:24.2a1) v4 -9904> 2014-08-13 16:24:07.016458 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016347, event: all_read, request: pg_info(1 pgs e194449:24.2a1) v4 -9903> 2014-08-13 16:24:07.016465 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016420, event: dispatched, request: pg_info(1 pgs e194449:24.2a1) v4 -9902> 2014-08-13 16:24:07.016472 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016472, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.2a1) v4 -9901> 2014-08-13 16:24:07.016484 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016483, event: started, request: pg_info(1 pgs e194449:24.2a1) v4 -9900> 2014-08-13 16:24:07.016520 7fd93dff0700 5 -- op tracker -- , seq: 628, time: 2014-08-13 16:24:07.016520, event: done, request: pg_info(1 pgs e194449:24.2a1) v4 -9899> 2014-08-13 16:24:07.017336 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.304 192.168.1.101:6808/22782 11 ==== pg_info(1 pgs e194449:24.33a) v4 ==== 748+0+0 (2974224327 0 0) 0x7d03100 con 0x4e4d800 -9898> 2014-08-13 16:24:07.017374 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017208, event: header_read, request: pg_info(1 pgs e194449:24.33a) v4 -9897> 2014-08-13 16:24:07.017391 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017211, event: throttled, request: pg_info(1 pgs e194449:24.33a) v4 -9896> 2014-08-13 16:24:07.017400 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017311, event: all_read, request: pg_info(1 pgs e194449:24.33a) v4 -9895> 2014-08-13 16:24:07.017409 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017369, event: dispatched, request: pg_info(1 pgs e194449:24.33a) v4 -9894> 2014-08-13 16:24:07.017420 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017420, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.33a) v4 -9893> 2014-08-13 16:24:07.017432 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017432, event: started, request: pg_info(1 pgs e194449:24.33a) v4 -9892> 2014-08-13 16:24:07.017466 7fd93dff0700 5 -- op tracker -- , seq: 629, time: 2014-08-13 16:24:07.017466, event: done, request: pg_info(1 pgs e194449:24.33a) v4 -9891> 2014-08-13 16:24:07.017659 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.325 192.168.1.110:6804/25590 7 ==== pg_info(1 pgs e194449:24.34) v4 ==== 740+0+0 (1773261430 0 0) 0x8c8abc0 con 0x78b0580 -9890> 2014-08-13 16:24:07.017692 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017532, event: header_read, request: pg_info(1 pgs e194449:24.34) v4 -9889> 2014-08-13 16:24:07.017704 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017534, event: throttled, request: pg_info(1 pgs e194449:24.34) v4 -9888> 2014-08-13 16:24:07.017720 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017632, event: all_read, request: pg_info(1 pgs e194449:24.34) v4 -9887> 2014-08-13 16:24:07.017727 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017687, event: dispatched, request: pg_info(1 pgs e194449:24.34) v4 -9886> 2014-08-13 16:24:07.017775 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017775, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.34) v4 -9885> 2014-08-13 16:24:07.017809 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017808, event: started, request: pg_info(1 pgs e194449:24.34) v4 -9884> 2014-08-13 16:24:07.017840 7fd93dff0700 5 -- op tracker -- , seq: 630, time: 2014-08-13 16:24:07.017840, event: done, request: pg_info(1 pgs e194449:24.34) v4 -9883> 2014-08-13 16:24:07.018633 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194448:24.350) v4 -- ?+0 0x8c8e3c0 -9882> 2014-08-13 16:24:07.018656 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.304 192.168.1.101:6808/22782 12 ==== MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 ==== 26+0+0 (2630862356 0 0) 0x8c888c0 con 0x4e4d800 -9881> 2014-08-13 16:24:07.018693 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018559, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9880> 2014-08-13 16:24:07.018705 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018561, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9879> 2014-08-13 16:24:07.018711 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018631, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9878> 2014-08-13 16:24:07.018717 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018688, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9877> 2014-08-13 16:24:07.018724 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018723, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9876> 2014-08-13 16:24:07.018756 7fd93dff0700 5 -- op tracker -- , seq: 631, time: 2014-08-13 16:24:07.018756, event: done, request: MRecoveryReserve REQUEST pgid: 24.33a, query_epoch: 194449 v2 -9875> 2014-08-13 16:24:07.018778 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194447 pi=193250-194446/19 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] exit Started/ReplicaActive/RepNotRecovering 2.592382 7 0.010391 -9874> 2014-08-13 16:24:07.018832 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194447 pi=193250-194446/19 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] enter Started/ReplicaActive/RepWaitRecoveryReserved -9873> 2014-08-13 16:24:07.019017 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.304 192.168.1.101:6808/22782 -- MRecoveryReserve GRANT pgid: 24.33a, query_epoch: 194449 v2 -- ?+0 0x4f8b2c0 -9872> 2014-08-13 16:24:07.019052 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194447 pi=193250-194446/19 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.000220 1 0.000097 -9871> 2014-08-13 16:24:07.019083 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194447 pi=193250-194446/19 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] enter Started/ReplicaActive/RepRecovering -9870> 2014-08-13 16:24:07.019675 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194448:24.369) v4 -- ?+0 0x8c8f700 -9869> 2014-08-13 16:24:07.019721 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 19 ==== pg_info(1 pgs e194449:24.350) v4 ==== 748+0+0 (3792698413 0 0) 0x8c8e3c0 con 0x4fd8000 -9868> 2014-08-13 16:24:07.019744 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019602, event: header_read, request: pg_info(1 pgs e194449:24.350) v4 -9867> 2014-08-13 16:24:07.019752 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019604, event: throttled, request: pg_info(1 pgs e194449:24.350) v4 -9866> 2014-08-13 16:24:07.019758 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019698, event: all_read, request: pg_info(1 pgs e194449:24.350) v4 -9865> 2014-08-13 16:24:07.019763 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019740, event: dispatched, request: pg_info(1 pgs e194449:24.350) v4 -9864> 2014-08-13 16:24:07.019770 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019770, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.350) v4 -9863> 2014-08-13 16:24:07.019778 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019778, event: started, request: pg_info(1 pgs e194449:24.350) v4 -9862> 2014-08-13 16:24:07.019806 7fd93dff0700 5 -- op tracker -- , seq: 632, time: 2014-08-13 16:24:07.019806, event: done, request: pg_info(1 pgs e194449:24.350) v4 -9861> 2014-08-13 16:24:07.020836 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 15 ==== pg_info(1 pgs e194449:24.369) v4 ==== 748+0+0 (3327703055 0 0) 0x8c8f700 con 0x4fda940 -9860> 2014-08-13 16:24:07.020866 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020731, event: header_read, request: pg_info(1 pgs e194449:24.369) v4 -9859> 2014-08-13 16:24:07.020879 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020733, event: throttled, request: pg_info(1 pgs e194449:24.369) v4 -9858> 2014-08-13 16:24:07.020888 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020810, event: all_read, request: pg_info(1 pgs e194449:24.369) v4 -9857> 2014-08-13 16:24:07.020898 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020862, event: dispatched, request: pg_info(1 pgs e194449:24.369) v4 -9856> 2014-08-13 16:24:07.020908 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020908, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.369) v4 -9855> 2014-08-13 16:24:07.020920 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020920, event: started, request: pg_info(1 pgs e194449:24.369) v4 -9854> 2014-08-13 16:24:07.020951 7fd93dff0700 5 -- op tracker -- , seq: 633, time: 2014-08-13 16:24:07.020951, event: done, request: pg_info(1 pgs e194449:24.369) v4 -9853> 2014-08-13 16:24:07.021294 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 20 ==== MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 ==== 26+0+0 (3024219852 0 0) 0x81c7540 con 0x4fd8000 -9852> 2014-08-13 16:24:07.021338 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021113, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9851> 2014-08-13 16:24:07.021353 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021115, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9850> 2014-08-13 16:24:07.021363 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021210, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9849> 2014-08-13 16:24:07.021380 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021332, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9848> 2014-08-13 16:24:07.021391 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021390, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9847> 2014-08-13 16:24:07.021417 7fd93dff0700 5 -- op tracker -- , seq: 634, time: 2014-08-13 16:24:07.021417, event: done, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194449 v2 -9846> 2014-08-13 16:24:07.021429 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194447 pi=193151-194446/19 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] exit Started/ReplicaActive/RepNotRecovering 2.586485 7 0.002328 -9845> 2014-08-13 16:24:07.021468 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194447 pi=193151-194446/19 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] enter Started/ReplicaActive/RepWaitRecoveryReserved -9844> 2014-08-13 16:24:07.022424 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 16 ==== MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 ==== 26+0+0 (780661303 0 0) 0x8c8f380 con 0x4fda940 -9843> 2014-08-13 16:24:07.022468 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022314, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9842> 2014-08-13 16:24:07.022480 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022317, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9841> 2014-08-13 16:24:07.022486 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022379, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9840> 2014-08-13 16:24:07.022491 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022462, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9839> 2014-08-13 16:24:07.022498 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022498, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9838> 2014-08-13 16:24:07.022521 7fd93dff0700 5 -- op tracker -- , seq: 635, time: 2014-08-13 16:24:07.022521, event: done, request: MRecoveryReserve REQUEST pgid: 24.369, query_epoch: 194449 v2 -9837> 2014-08-13 16:24:07.022557 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194447 pi=193186-194446/19 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] exit Started/ReplicaActive/RepNotRecovering 2.581397 7 0.002593 -9836> 2014-08-13 16:24:07.022595 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194447 pi=193186-194446/19 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] enter Started/ReplicaActive/RepWaitRecoveryReserved -9835> 2014-08-13 16:24:07.023579 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.6 192.168.1.107:6809/53141 -- pg_info(1 pgs e194448:24.cd) v4 -- ?+0 0x8c8a4c0 -9834> 2014-08-13 16:24:07.024803 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194448:24.5e) v4 -- ?+0 0x77588c0 -9833> 2014-08-13 16:24:07.025050 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.6 192.168.1.107:6809/53141 6 ==== pg_info(1 pgs e194449:24.cd) v4 ==== 748+0+0 (190395080 0 0) 0x8c8a4c0 con 0x5109340 -9832> 2014-08-13 16:24:07.025095 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.024875, event: header_read, request: pg_info(1 pgs e194449:24.cd) v4 -9831> 2014-08-13 16:24:07.025108 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.024878, event: throttled, request: pg_info(1 pgs e194449:24.cd) v4 -9830> 2014-08-13 16:24:07.025114 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.024983, event: all_read, request: pg_info(1 pgs e194449:24.cd) v4 -9829> 2014-08-13 16:24:07.025120 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.025089, event: dispatched, request: pg_info(1 pgs e194449:24.cd) v4 -9828> 2014-08-13 16:24:07.025128 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.025128, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.cd) v4 -9827> 2014-08-13 16:24:07.025138 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.025138, event: started, request: pg_info(1 pgs e194449:24.cd) v4 -9826> 2014-08-13 16:24:07.025169 7fd93dff0700 5 -- op tracker -- , seq: 636, time: 2014-08-13 16:24:07.025169, event: done, request: pg_info(1 pgs e194449:24.cd) v4 -9825> 2014-08-13 16:24:07.026067 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 16 ==== pg_info(1 pgs e194449:24.5e) v4 ==== 748+0+0 (948432771 0 0) 0x77588c0 con 0x4fdb2e0 -9824> 2014-08-13 16:24:07.026111 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.025896, event: header_read, request: pg_info(1 pgs e194449:24.5e) v4 -9823> 2014-08-13 16:24:07.026123 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.025899, event: throttled, request: pg_info(1 pgs e194449:24.5e) v4 -9822> 2014-08-13 16:24:07.026129 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.025996, event: all_read, request: pg_info(1 pgs e194449:24.5e) v4 -9821> 2014-08-13 16:24:07.026135 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.026104, event: dispatched, request: pg_info(1 pgs e194449:24.5e) v4 -9820> 2014-08-13 16:24:07.026142 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.026142, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.5e) v4 -9819> 2014-08-13 16:24:07.026151 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.026151, event: started, request: pg_info(1 pgs e194449:24.5e) v4 -9818> 2014-08-13 16:24:07.026182 7fd93dff0700 5 -- op tracker -- , seq: 637, time: 2014-08-13 16:24:07.026182, event: done, request: pg_info(1 pgs e194449:24.5e) v4 -9817> 2014-08-13 16:24:07.027083 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194448:24.206) v4 -- ?+0 0x775c280 -9816> 2014-08-13 16:24:07.027579 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 17 ==== MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 ==== 26+0+0 (2532392470 0 0) 0x80d96c0 con 0x4fdb2e0 -9815> 2014-08-13 16:24:07.027624 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027431, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9814> 2014-08-13 16:24:07.027640 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027435, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9813> 2014-08-13 16:24:07.027650 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027513, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9812> 2014-08-13 16:24:07.027659 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027618, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9811> 2014-08-13 16:24:07.027670 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027670, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9810> 2014-08-13 16:24:07.027697 7fd93dff0700 5 -- op tracker -- , seq: 638, time: 2014-08-13 16:24:07.027696, event: done, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194449 v2 -9809> 2014-08-13 16:24:07.027709 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194447 pi=193125-194446/32 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] exit Started/ReplicaActive/RepNotRecovering 2.582832 7 0.003073 -9808> 2014-08-13 16:24:07.027744 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194447 pi=193125-194446/32 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] enter Started/ReplicaActive/RepWaitRecoveryReserved -9807> 2014-08-13 16:24:07.028112 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194448:24.1e6) v4 -- ?+0 0x7758380 -9806> 2014-08-13 16:24:07.028230 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.343 192.168.1.104:6802/61960 17 ==== pg_info(1 pgs e194449:24.206) v4 ==== 748+0+0 (4026595344 0 0) 0x775c280 con 0x508dd80 -9805> 2014-08-13 16:24:07.028274 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028082, event: header_read, request: pg_info(1 pgs e194449:24.206) v4 -9804> 2014-08-13 16:24:07.028286 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028083, event: throttled, request: pg_info(1 pgs e194449:24.206) v4 -9803> 2014-08-13 16:24:07.028293 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028170, event: all_read, request: pg_info(1 pgs e194449:24.206) v4 -9802> 2014-08-13 16:24:07.028303 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028268, event: dispatched, request: pg_info(1 pgs e194449:24.206) v4 -9801> 2014-08-13 16:24:07.028314 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028314, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.206) v4 -9800> 2014-08-13 16:24:07.028327 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028327, event: started, request: pg_info(1 pgs e194449:24.206) v4 -9799> 2014-08-13 16:24:07.028361 7fd93dff0700 5 -- op tracker -- , seq: 639, time: 2014-08-13 16:24:07.028361, event: done, request: pg_info(1 pgs e194449:24.206) v4 -9798> 2014-08-13 16:24:07.029239 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.285 192.168.1.101:6822/26295 12 ==== pg_info(1 pgs e194449:24.1e6) v4 ==== 748+0+0 (2576055129 0 0) 0x7758380 con 0x4fe0840 -9797> 2014-08-13 16:24:07.029282 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029090, event: header_read, request: pg_info(1 pgs e194449:24.1e6) v4 -9796> 2014-08-13 16:24:07.029295 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029092, event: throttled, request: pg_info(1 pgs e194449:24.1e6) v4 -9795> 2014-08-13 16:24:07.029304 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029176, event: all_read, request: pg_info(1 pgs e194449:24.1e6) v4 -9794> 2014-08-13 16:24:07.029314 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029276, event: dispatched, request: pg_info(1 pgs e194449:24.1e6) v4 -9793> 2014-08-13 16:24:07.029321 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029321, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.1e6) v4 -9792> 2014-08-13 16:24:07.029331 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029331, event: started, request: pg_info(1 pgs e194449:24.1e6) v4 -9791> 2014-08-13 16:24:07.029365 7fd93dff0700 5 -- op tracker -- , seq: 640, time: 2014-08-13 16:24:07.029365, event: done, request: pg_info(1 pgs e194449:24.1e6) v4 -9790> 2014-08-13 16:24:07.069927 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.10 192.168.1.115:6815/60573 -- pg_info(1 pgs e194449:24.30b) v4 -- ?+0 0x7758700 -9789> 2014-08-13 16:24:07.069986 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194449:24.30b) v4 -- ?+0 0x775cb40 -9788> 2014-08-13 16:24:07.070015 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.375 192.168.1.114:6830/52879 -- pg_info(1 pgs e194449:24.30b) v4 -- ?+0 0x775d080 -9787> 2014-08-13 16:24:07.070153 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194449:22.362) v4 -- ?+0 0x775e040 -9786> 2014-08-13 16:24:07.070221 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194449:22.362) v4 -- ?+0 0x775bf00 -9785> 2014-08-13 16:24:07.070446 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194449:22.84) v4 -- ?+0 0x775b9c0 -9784> 2014-08-13 16:24:07.070508 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194449:22.84) v4 -- ?+0 0x7758a80 -9783> 2014-08-13 16:24:07.070589 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.301 192.168.1.113:6814/16727 -- pg_info(1 pgs e194449:22.22c) v4 -- ?+0 0x775e200 -9782> 2014-08-13 16:24:07.070619 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.324 192.168.1.104:6814/65209 -- pg_info(1 pgs e194449:22.22c) v4 -- ?+0 0x775de80 -9781> 2014-08-13 16:24:07.070690 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.673133 12 0.000595 -9780> 2014-08-13 16:24:07.070426 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194448 pi=193640-194447/20 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] on activate: bft=286 from 0//0//-1 -9779> 2014-08-13 16:24:07.070746 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9778> 2014-08-13 16:24:07.070745 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194448 pi=193640-194447/20 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] target shard 286 from 0//0//-1 -9777> 2014-08-13 16:24:07.071078 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194340 194447/194448/194448) [4,28,304] r=0 lpr=194448 pi=194331-194447/10 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.654474 10 0.000251 -9776> 2014-08-13 16:24:07.071110 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194340 194447/194448/194448) [4,28,304] r=0 lpr=194448 pi=194331-194447/10 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9775> 2014-08-13 16:24:07.071143 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194340 194447/194448/194448) [4,28,304] r=0 lpr=194448 pi=194331-194447/10 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000034 0 0.000000 -9774> 2014-08-13 16:24:07.071168 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194340 194447/194448/194448) [4,28,304] r=0 lpr=194448 pi=194331-194447/10 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9773> 2014-08-13 16:24:07.071221 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194449:22.84) v4 -- ?+0 0x81c3b80 -9772> 2014-08-13 16:24:07.071256 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194449:22.84) v4 -- ?+0 0x81c0700 -9771> 2014-08-13 16:24:07.071479 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194449/194389 194447/194448/194448) [4,301,324] r=0 lpr=194448 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.654024 11 0.000636 -9770> 2014-08-13 16:24:07.071526 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194449/194389 194447/194448/194448) [4,301,324] r=0 lpr=194448 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9769> 2014-08-13 16:24:07.071563 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.000817 1 0.000102 -9768> 2014-08-13 16:24:07.071589 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved -9767> 2014-08-13 16:24:07.071617 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.111:6834/14316 -- MRecoveryReserve REQUEST pgid: 22.362, query_epoch: 194449 v2 -- ?+0 0x81c1dc0 con 0x4e4dd80 -9766> 2014-08-13 16:24:07.071752 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194449; pg22.84; ) v2 -- ?+0 0x80d9180 -9765> 2014-08-13 16:24:07.071500 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194448 pi=193640-194447/20 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] exit Started/Primary/Active/Activating 1.671457 12 0.004637 -9764> 2014-08-13 16:24:07.071809 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194448 pi=193640-194447/20 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] enter Started/Primary/Active/WaitLocalRecoveryReserved -9763> 2014-08-13 16:24:07.072082 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.62 192.168.1.112:6824/5505 -- pg_info(1 pgs e194449:22.619) v4 -- ?+0 0x7759f80 -9762> 2014-08-13 16:24:07.072136 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.65 192.168.1.111:6842/17345 -- pg_info(1 pgs e194449:22.619) v4 -- ?+0 0x80d8e00 -9761> 2014-08-13 16:24:07.072370 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.291 192.168.1.101:6820/25633 -- pg_info(1 pgs e194449:24.9b) v4 -- ?+0 0x80dc0c0 -9760> 2014-08-13 16:24:07.072408 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.350 192.168.1.114:6832/53185 -- pg_info(1 pgs e194449:24.9b) v4 -- ?+0 0x80daf40 -9759> 2014-08-13 16:24:07.072432 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194449:24.9b) v4 -- ?+0 0x80da4c0 -9758> 2014-08-13 16:24:07.072457 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.378 192.168.1.102:6840/19195 -- pg_info(1 pgs e194449:24.9b) v4 -- ?+0 0x80db100 -9757> 2014-08-13 16:24:07.072558 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194449:22.f3) v4 -- ?+0 0x80dde80 -9756> 2014-08-13 16:24:07.072608 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.312 192.168.1.113:6816/17521 -- pg_info(1 pgs e194449:22.f3) v4 -- ?+0 0x775cd00 -9755> 2014-08-13 16:24:07.072781 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 14 ==== MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 ==== 26+0+0 (2461361463 0 0) 0x81c1dc0 con 0x4e4dd80 -9754> 2014-08-13 16:24:07.072817 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072679, event: header_read, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9753> 2014-08-13 16:24:07.072796 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,65,62] r=0 lpr=194448 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.654851 12 0.000560 -9752> 2014-08-13 16:24:07.072832 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072681, event: throttled, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9751> 2014-08-13 16:24:07.072842 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072742, event: all_read, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9750> 2014-08-13 16:24:07.072838 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,65,62] r=0 lpr=194448 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9749> 2014-08-13 16:24:07.072851 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072812, event: dispatched, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9748> 2014-08-13 16:24:07.072859 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072859, event: waiting_for_osdmap, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9747> 2014-08-13 16:24:07.072882 7fd93dff0700 5 -- op tracker -- , seq: 641, time: 2014-08-13 16:24:07.072882, event: done, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9746> 2014-08-13 16:24:07.072867 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,312,285] r=0 lpr=194448 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.652289 11 0.000517 -9745> 2014-08-13 16:24:07.072904 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,312,285] r=0 lpr=194448 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9744> 2014-08-13 16:24:07.072929 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194449:22.4df) v4 -- ?+0 0x80d9f80 -9743> 2014-08-13 16:24:07.072981 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.357 192.168.1.102:6834/17224 -- pg_info(1 pgs e194449:22.4df) v4 -- ?+0 0x80dd400 -9742> 2014-08-13 16:24:07.073078 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194449:24.3d) v4 -- ?+0 0x80dad80 -9741> 2014-08-13 16:24:07.073119 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.25 192.168.1.103:6800/36920 -- pg_info(1 pgs e194449:24.3d) v4 -- ?+0 0x80d8700 -9740> 2014-08-13 16:24:07.073149 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.303 192.168.1.109:6802/30768 -- pg_info(1 pgs e194449:24.3d) v4 -- ?+0 0x80df540 -9739> 2014-08-13 16:24:07.073202 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194449:24.3d) v4 -- ?+0 0x80dc280 -9738> 2014-08-13 16:24:07.073306 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194449:22.39d) v4 -- ?+0 0x80db800 -9737> 2014-08-13 16:24:07.073345 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194449:22.39d) v4 -- ?+0 0x80df1c0 -9736> 2014-08-13 16:24:07.073390 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6802/61960 -- MRecoveryReserve REQUEST pgid: 22.362, query_epoch: 194449 v2 -- ?+0 0x77596c0 con 0x508dd80 -9735> 2014-08-13 16:24:07.073214 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194449/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194448 pi=193637-194447/19 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 -9734> 2014-08-13 16:24:07.073509 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.56 192.168.1.111:6826/11446 -- pg_info(1 pgs e194449:22.775) v4 -- ?+0 0x80df700 -9733> 2014-08-13 16:24:07.073575 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.353 192.168.1.108:6836/6227 -- pg_info(1 pgs e194449:22.775) v4 -- ?+0 0x80de740 -9732> 2014-08-13 16:24:07.073780 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194449:22.1d2) v4 -- ?+0 0x80dc600 -9731> 2014-08-13 16:24:07.073827 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194449:22.1d2) v4 -- ?+0 0x80dcec0 -9730> 2014-08-13 16:24:07.073545 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194449/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194448 pi=193637-194447/19 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 -9729> 2014-08-13 16:24:07.074011 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:2.2a) v4 -- ?+0 0x80d9500 -9728> 2014-08-13 16:24:07.073859 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194449/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194448 pi=193637-194447/19 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 -9727> 2014-08-13 16:24:07.074230 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.313 192.168.1.109:6820/35445 -- pg_info(1 pgs e194449:22.3e4) v4 -- ?+0 0x80db9c0 -9726> 2014-08-13 16:24:07.074278 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.318 192.168.1.110:6814/43857 -- pg_info(1 pgs e194449:22.3e4) v4 -- ?+0 0x80df380 -9725> 2014-08-13 16:24:07.073981 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194449/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194448 pi=193635-194447/19 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 -9724> 2014-08-13 16:24:07.074356 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.21 192.168.1.107:6821/54786 -- pg_info(1 pgs e194449:22.707) v4 -- ?+0 0x80dc980 -9723> 2014-08-13 16:24:07.074388 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.310 192.168.1.109:6812/32413 -- pg_info(1 pgs e194449:22.707) v4 -- ?+0 0x81c63c0 -9722> 2014-08-13 16:24:07.074607 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.343 192.168.1.104:6802/61960 18 ==== MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 ==== 26+0+0 (2461361463 0 0) 0x77596c0 con 0x508dd80 -9721> 2014-08-13 16:24:07.074651 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074460, event: header_read, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9720> 2014-08-13 16:24:07.074667 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074462, event: throttled, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9719> 2014-08-13 16:24:07.074676 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074522, event: all_read, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9718> 2014-08-13 16:24:07.074686 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074645, event: dispatched, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9717> 2014-08-13 16:24:07.074696 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074696, event: waiting_for_osdmap, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9716> 2014-08-13 16:24:07.074721 7fd93dff0700 5 -- op tracker -- , seq: 642, time: 2014-08-13 16:24:07.074721, event: done, request: MRecoveryReserve GRANT pgid: 22.362, query_epoch: 194449 v2 -9715> 2014-08-13 16:24:07.074306 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194449/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194448 pi=193635-194447/19 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 -9714> 2014-08-13 16:24:07.074767 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194449:24.b6) v4 -- ?+0 0x81c08c0 -9713> 2014-08-13 16:24:07.074801 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194449:24.b6) v4 -- ?+0 0x80dd940 -9712> 2014-08-13 16:24:07.074832 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.315 192.168.1.113:6800/14161 -- pg_info(1 pgs e194449:24.b6) v4 -- ?+0 0x80dc7c0 -9711> 2014-08-13 16:24:07.074859 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194449:24.b6) v4 -- ?+0 0x80d8c40 -9710> 2014-08-13 16:24:07.074553 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194449/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194448 pi=193637-194447/19 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] exit Started/Primary/Active/Activating 1.654807 14 0.003608 -9709> 2014-08-13 16:24:07.074918 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194449:22.113) v4 -- ?+0 0x80de580 -9708> 2014-08-13 16:24:07.074944 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.339 192.168.1.110:6818/45268 -- pg_info(1 pgs e194449:22.113) v4 -- ?+0 0x80d88c0 -9707> 2014-08-13 16:24:07.074998 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.5 192.168.1.103:6827/40192 -- pg_info(1 pgs e194449:22.1f1) v4 -- ?+0 0x80df8c0 -9706> 2014-08-13 16:24:07.075026 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.47 192.168.1.111:6828/12102 -- pg_info(1 pgs e194449:22.1f1) v4 -- ?+0 0x80de3c0 -9705> 2014-08-13 16:24:07.074749 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194449/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194448 pi=193635-194447/19 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 -9704> 2014-08-13 16:24:07.075218 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:1.2b) v4 -- ?+0 0x80dbf00 -9703> 2014-08-13 16:24:07.074995 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194449/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194448 pi=193637-194447/19 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] enter Started/Primary/Active/WaitLocalRecoveryReserved -9702> 2014-08-13 16:24:07.075395 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194449:22.99) v4 -- ?+0 0x81c3800 -9701> 2014-08-13 16:24:07.075436 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194449:22.99) v4 -- ?+0 0x81c2140 -9700> 2014-08-13 16:24:07.075555 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.55 192.168.1.112:6834/6966 -- pg_info(1 pgs e194449:24.1f7) v4 -- ?+0 0x81c4440 -9699> 2014-08-13 16:24:07.075589 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.59 192.168.1.112:6822/5265 -- pg_info(1 pgs e194449:24.1f7) v4 -- ?+0 0x81c5080 -9698> 2014-08-13 16:24:07.075619 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.351 192.168.1.102:6830/16444 -- pg_info(1 pgs e194449:24.1f7) v4 -- ?+0 0x81c5240 -9697> 2014-08-13 16:24:07.075815 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x81c0380 -9696> 2014-08-13 16:24:07.075844 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x81c6580 -9695> 2014-08-13 16:24:07.076004 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.338 192.168.1.106:6814/25893 -- pg_info(1 pgs e194449:22.3d9) v4 -- ?+0 0x81c0540 -9694> 2014-08-13 16:24:07.076008 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194449/194389 194447/194448/194448) [4,356,357] r=0 lpr=194448 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.654106 12 0.005397 -9693> 2014-08-13 16:24:07.076035 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194449:22.3d9) v4 -- ?+0 0x81c6ac0 -9692> 2014-08-13 16:24:07.076057 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194449/194389 194447/194448/194448) [4,356,357] r=0 lpr=194448 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9691> 2014-08-13 16:24:07.076308 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194449:22.29a) v4 -- ?+0 0x81c5400 -9690> 2014-08-13 16:24:07.076092 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194449/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194448 pi=193635-194447/19 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] exit Started/Primary/Active/Activating 1.651234 14 0.006166 -9689> 2014-08-13 16:24:07.076457 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.321 192.168.1.110:6812/43236 -- pg_info(1 pgs e194449:22.4aa) v4 -- ?+0 0x81c3100 -9688> 2014-08-13 16:24:07.076514 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,373,344] r=0 lpr=194448 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.651333 12 0.000561 -9687> 2014-08-13 16:24:07.076546 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,373,344] r=0 lpr=194448 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9686> 2014-08-13 16:24:07.076574 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194449/194389 194447/194448/194448) [4,353,56] r=0 lpr=194448 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.622169 12 0.000722 -9685> 2014-08-13 16:24:07.076600 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194449/194389 194447/194448/194448) [4,353,56] r=0 lpr=194448 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9684> 2014-08-13 16:24:07.076629 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194449/194389 194447/194448/194448) [4,349,61] r=0 lpr=194448 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.620760 12 0.000608 -9683> 2014-08-13 16:24:07.076652 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194449/194389 194447/194448/194448) [4,349,61] r=0 lpr=194448 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9682> 2014-08-13 16:24:07.076663 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194449:22.2b9) v4 -- ?+0 0x81c1a40 -9681> 2014-08-13 16:24:07.076422 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194449/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194448 pi=193635-194447/19 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] enter Started/Primary/Active/WaitLocalRecoveryReserved -9680> 2014-08-13 16:24:07.076744 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.005154 2 0.000216 -9679> 2014-08-13 16:24:07.076771 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/194448) [4,58,343] r=0 lpr=194448 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/Recovering -9678> 2014-08-13 16:24:07.076824 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194449:24.136) v4 -- ?+0 0x81c5e80 -9677> 2014-08-13 16:24:07.076855 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.55 192.168.1.112:6834/6966 -- pg_info(1 pgs e194449:24.136) v4 -- ?+0 0x81c1500 -9676> 2014-08-13 16:24:07.076889 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194449:24.136) v4 -- ?+0 0x81c39c0 -9675> 2014-08-13 16:24:07.076926 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.65 192.168.1.111:6842/17345 -- pg_info(1 pgs e194449:24.136) v4 -- ?+0 0x81c0c40 -9674> 2014-08-13 16:24:07.076928 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.618821 8 0.000284 -9673> 2014-08-13 16:24:07.076955 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9672> 2014-08-13 16:24:07.076973 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194449:22.6bd) v4 -- ?+0 0x81c1340 -9671> 2014-08-13 16:24:07.076979 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000024 0 0.000000 -9670> 2014-08-13 16:24:07.076998 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194449:22.6bd) v4 -- ?+0 0x81c7c40 -9669> 2014-08-13 16:24:07.076999 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9668> 2014-08-13 16:24:07.077032 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:2.2a) v4 -- ?+0 0x775f700 -9667> 2014-08-13 16:24:07.077140 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.44 192.168.1.111:6822/10782 -- pg_info(1 pgs e194449:22.3af) v4 -- ?+0 0x81c6e40 -9666> 2014-08-13 16:24:07.077231 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194449/194389 194447/194448/194448) [4,313,318] r=0 lpr=194448 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.618686 12 0.000633 -9665> 2014-08-13 16:24:07.077274 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194449/194389 194447/194448/194448) [4,313,318] r=0 lpr=194448 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9664> 2014-08-13 16:24:07.077319 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194449/194389 194447/194448/194448) [4,310,21] r=0 lpr=194448 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.617987 12 0.000638 -9663> 2014-08-13 16:24:07.077328 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194449:22.3af) v4 -- ?+0 0x81c7000 -9662> 2014-08-13 16:24:07.077347 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194449/194389 194447/194448/194448) [4,310,21] r=0 lpr=194448 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9661> 2014-08-13 16:24:07.077386 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 10 ==== pg_info(1 pgs e194449:22.29a) v4 ==== 714+0+0 (3136828842 0 0) 0x81c5400 con 0x4fe0000 -9660> 2014-08-13 16:24:07.077381 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194449; pg2.2a; ) v2 -- ?+0 0x81c16c0 -9659> 2014-08-13 16:24:07.077412 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077208, event: header_read, request: pg_info(1 pgs e194449:22.29a) v4 -9658> 2014-08-13 16:24:07.077426 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077211, event: throttled, request: pg_info(1 pgs e194449:22.29a) v4 -9657> 2014-08-13 16:24:07.077433 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077294, event: all_read, request: pg_info(1 pgs e194449:22.29a) v4 -9656> 2014-08-13 16:24:07.077129 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194449/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194448 pi=193748-194447/19 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=28,330 from 0//0//-1 -9655> 2014-08-13 16:24:07.077444 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077408, event: dispatched, request: pg_info(1 pgs e194449:22.29a) v4 -9654> 2014-08-13 16:24:07.077456 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077455, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.29a) v4 -9653> 2014-08-13 16:24:07.077465 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077465, event: started, request: pg_info(1 pgs e194449:22.29a) v4 -9652> 2014-08-13 16:24:07.077496 7fd93dff0700 5 -- op tracker -- , seq: 643, time: 2014-08-13 16:24:07.077495, event: done, request: pg_info(1 pgs e194449:22.29a) v4 -9651> 2014-08-13 16:24:07.077494 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194449/194389 194447/194448/194448) [4,338,376] r=0 lpr=194448 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.600002 14 0.000595 -9650> 2014-08-13 16:24:07.077517 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 7 ==== pg_info(1 pgs e194449:22.4aa) v4 ==== 714+0+0 (3752181727 0 0) 0x81c3100 con 0x4fe4ba0 -9649> 2014-08-13 16:24:07.077533 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077221, event: header_read, request: pg_info(1 pgs e194449:22.4aa) v4 -9648> 2014-08-13 16:24:07.077544 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077223, event: throttled, request: pg_info(1 pgs e194449:22.4aa) v4 -9647> 2014-08-13 16:24:07.077534 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194449/194389 194447/194448/194448) [4,338,376] r=0 lpr=194448 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9646> 2014-08-13 16:24:07.077555 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077303, event: all_read, request: pg_info(1 pgs e194449:22.4aa) v4 -9645> 2014-08-13 16:24:07.077565 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077531, event: dispatched, request: pg_info(1 pgs e194449:22.4aa) v4 -9644> 2014-08-13 16:24:07.077575 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077574, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.4aa) v4 -9643> 2014-08-13 16:24:07.077588 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077588, event: started, request: pg_info(1 pgs e194449:22.4aa) v4 -9642> 2014-08-13 16:24:07.077611 7fd93dff0700 5 -- op tracker -- , seq: 644, time: 2014-08-13 16:24:07.077611, event: done, request: pg_info(1 pgs e194449:22.4aa) v4 -9641> 2014-08-13 16:24:07.077615 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.296 192.168.1.109:6806/31286 -- pg_info(1 pgs e194449:24.181) v4 -- ?+0 0x81c0e00 -9640> 2014-08-13 16:24:07.077656 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.320 192.168.1.106:6820/10050 -- pg_info(1 pgs e194449:24.181) v4 -- ?+0 0x81c78c0 -9639> 2014-08-13 16:24:07.077683 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194449:24.181) v4 -- ?+0 0x81c47c0 -9638> 2014-08-13 16:24:07.077449 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194449/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194448 pi=193748-194447/19 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 28 from 0//0//-1 -9637> 2014-08-13 16:24:07.077755 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 10 ==== pg_info(1 pgs e194449:22.2b9) v4 ==== 714+0+0 (2322893492 0 0) 0x81c1a40 con 0x4fe3700 -9636> 2014-08-13 16:24:07.077778 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077650, event: header_read, request: pg_info(1 pgs e194449:22.2b9) v4 -9635> 2014-08-13 16:24:07.077789 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077652, event: throttled, request: pg_info(1 pgs e194449:22.2b9) v4 -9634> 2014-08-13 16:24:07.077804 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077730, event: all_read, request: pg_info(1 pgs e194449:22.2b9) v4 -9633> 2014-08-13 16:24:07.077813 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077775, event: dispatched, request: pg_info(1 pgs e194449:22.2b9) v4 -9632> 2014-08-13 16:24:07.077820 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077820, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.2b9) v4 -9631> 2014-08-13 16:24:07.077831 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077831, event: started, request: pg_info(1 pgs e194449:22.2b9) v4 -9630> 2014-08-13 16:24:07.077855 7fd93dff0700 5 -- op tracker -- , seq: 645, time: 2014-08-13 16:24:07.077854, event: done, request: pg_info(1 pgs e194449:22.2b9) v4 -9629> 2014-08-13 16:24:07.077873 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 8 ==== pg_info(1 pgs e194449:22.4aa) v4 ==== 714+0+0 (1090961190 0 0) 0x7d06040 con 0x4fe4ba0 -9628> 2014-08-13 16:24:07.077891 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077706, event: header_read, request: pg_info(1 pgs e194449:22.4aa) v4 -9627> 2014-08-13 16:24:07.077901 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077708, event: throttled, request: pg_info(1 pgs e194449:22.4aa) v4 -9626> 2014-08-13 16:24:07.077910 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077794, event: all_read, request: pg_info(1 pgs e194449:22.4aa) v4 -9625> 2014-08-13 16:24:07.077919 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077889, event: dispatched, request: pg_info(1 pgs e194449:22.4aa) v4 -9624> 2014-08-13 16:24:07.077928 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077928, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.4aa) v4 -9623> 2014-08-13 16:24:07.077939 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077939, event: started, request: pg_info(1 pgs e194449:22.4aa) v4 -9622> 2014-08-13 16:24:07.077960 7fd93dff0700 5 -- op tracker -- , seq: 646, time: 2014-08-13 16:24:07.077960, event: done, request: pg_info(1 pgs e194449:22.4aa) v4 -9621> 2014-08-13 16:24:07.077984 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 11 ==== pg_info(1 pgs e194449:22.29a) v4 ==== 714+0+0 (3517546341 0 0) 0x81c5b00 con 0x4fe0000 -9620> 2014-08-13 16:24:07.078003 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.077848, event: header_read, request: pg_info(1 pgs e194449:22.29a) v4 -9619> 2014-08-13 16:24:07.078014 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.077850, event: throttled, request: pg_info(1 pgs e194449:22.29a) v4 -9618> 2014-08-13 16:24:07.078023 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.077938, event: all_read, request: pg_info(1 pgs e194449:22.29a) v4 -9617> 2014-08-13 16:24:07.078032 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.078001, event: dispatched, request: pg_info(1 pgs e194449:22.29a) v4 -9616> 2014-08-13 16:24:07.078041 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.078041, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.29a) v4 -9615> 2014-08-13 16:24:07.078052 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.078052, event: started, request: pg_info(1 pgs e194449:22.29a) v4 -9614> 2014-08-13 16:24:07.077762 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194449/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194448 pi=193748-194447/19 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 330 from 0//0//-1 -9613> 2014-08-13 16:24:07.078074 7fd93dff0700 5 -- op tracker -- , seq: 647, time: 2014-08-13 16:24:07.078074, event: done, request: pg_info(1 pgs e194449:22.29a) v4 -9612> 2014-08-13 16:24:07.078143 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.287 192.168.1.109:6814/33435 -- pg_info(1 pgs e194449:22.481) v4 -- ?+0 0x8ee81c0 -9611> 2014-08-13 16:24:07.078188 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.309 192.168.1.101:6814/23703 -- pg_info(1 pgs e194449:22.481) v4 -- ?+0 0x8eea840 -9610> 2014-08-13 16:24:07.077960 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194449/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194448 pi=193747-194447/18 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 active+remapped m=51] on activate: bft=55,61 from 0//0//-1 -9609> 2014-08-13 16:24:07.078261 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194449/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194448 pi=193747-194447/18 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 active+remapped m=51] target shard 55 from 0//0//-1 -9608> 2014-08-13 16:24:07.078536 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.294 192.168.1.101:6812/23402 -- pg_info(1 pgs e194449:22.630) v4 -- ?+0 0x8ee8000 -9607> 2014-08-13 16:24:07.078537 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194449/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194448 pi=193747-194447/18 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 active+remapped m=51] target shard 61 from 0//0//-1 -9606> 2014-08-13 16:24:07.079001 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 11 ==== pg_info(1 pgs e194449:22.2b9) v4 ==== 714+0+0 (1029592817 0 0) 0x8c89a40 con 0x4fe3700 -9605> 2014-08-13 16:24:07.079030 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.078896, event: header_read, request: pg_info(1 pgs e194449:22.2b9) v4 -9604> 2014-08-13 16:24:07.079022 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.358 192.168.1.114:6824/52078 -- pg_info(1 pgs e194449:22.2d1) v4 -- ?+0 0x8ee88c0 -9603> 2014-08-13 16:24:07.079042 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.078899, event: throttled, request: pg_info(1 pgs e194449:22.2b9) v4 -9602> 2014-08-13 16:24:07.079052 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.078977, event: all_read, request: pg_info(1 pgs e194449:22.2b9) v4 -9601> 2014-08-13 16:24:07.079057 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.079027, event: dispatched, request: pg_info(1 pgs e194449:22.2b9) v4 -9600> 2014-08-13 16:24:07.079064 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.079064, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.2b9) v4 -9599> 2014-08-13 16:24:07.079073 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.079072, event: started, request: pg_info(1 pgs e194449:22.2b9) v4 -9598> 2014-08-13 16:24:07.079093 7fd93dff0700 5 -- op tracker -- , seq: 648, time: 2014-08-13 16:24:07.079093, event: done, request: pg_info(1 pgs e194449:22.2b9) v4 -9597> 2014-08-13 16:24:07.079204 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.51 192.168.1.112:6832/6655 -- pg_info(1 pgs e194449:22.537) v4 -- ?+0 0x81c1180 -9596> 2014-08-13 16:24:07.079497 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194449:22.48e) v4 -- ?+0 0x81c5940 -9595> 2014-08-13 16:24:07.079544 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.341 192.168.1.106:6818/9142 -- pg_info(1 pgs e194449:22.48e) v4 -- ?+0 0x81c5780 -9594> 2014-08-13 16:24:07.079318 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194449/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194448 pi=193678-194447/19 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 active+remapped m=62] on activate: bft=59 from 0//0//-1 -9593> 2014-08-13 16:24:07.079688 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.331 192.168.1.104:6816/837 -- pg_info(1 pgs e194449:24.1bb) v4 -- ?+0 0x81c6c80 -9592> 2014-08-13 16:24:07.079720 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.332 192.168.1.110:6806/42024 -- pg_info(1 pgs e194449:24.1bb) v4 -- ?+0 0x81c4d00 -9591> 2014-08-13 16:24:07.079762 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.294 192.168.1.101:6812/23402 7 ==== pg_info(1 pgs e194449:22.630) v4 ==== 714+0+0 (2465600705 0 0) 0x8ee8000 con 0x4fd8b00 -9590> 2014-08-13 16:24:07.079805 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079471, event: header_read, request: pg_info(1 pgs e194449:22.630) v4 -9589> 2014-08-13 16:24:07.079821 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079473, event: throttled, request: pg_info(1 pgs e194449:22.630) v4 -9588> 2014-08-13 16:24:07.079828 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079718, event: all_read, request: pg_info(1 pgs e194449:22.630) v4 -9587> 2014-08-13 16:24:07.079837 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079800, event: dispatched, request: pg_info(1 pgs e194449:22.630) v4 -9586> 2014-08-13 16:24:07.079848 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079848, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.630) v4 -9585> 2014-08-13 16:24:07.079861 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079860, event: started, request: pg_info(1 pgs e194449:22.630) v4 -9584> 2014-08-13 16:24:07.079582 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194449/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194448 pi=193747-194447/18 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 active+remapped m=51] exit Started/Primary/Active/Activating 1.601479 14 0.003821 -9583> 2014-08-13 16:24:07.079851 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194449:24.1bb) v4 -- ?+0 0x81c24c0 -9582> 2014-08-13 16:24:07.079894 7fd93dff0700 5 -- op tracker -- , seq: 649, time: 2014-08-13 16:24:07.079894, event: done, request: pg_info(1 pgs e194449:22.630) v4 -9581> 2014-08-13 16:24:07.079897 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.348 192.168.1.110:6800/41204 -- pg_info(1 pgs e194449:24.1bb) v4 -- ?+0 0x81c6040 -9580> 2014-08-13 16:24:07.079918 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 7 ==== pg_info(1 pgs e194449:22.537) v4 ==== 714+0+0 (3156073260 0 0) 0x81c1180 con 0x50bc4c0 -9579> 2014-08-13 16:24:07.079942 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079716, event: header_read, request: pg_info(1 pgs e194449:22.537) v4 -9578> 2014-08-13 16:24:07.079949 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079718, event: throttled, request: pg_info(1 pgs e194449:22.537) v4 -9577> 2014-08-13 16:24:07.079955 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079790, event: all_read, request: pg_info(1 pgs e194449:22.537) v4 -9576> 2014-08-13 16:24:07.079965 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079939, event: dispatched, request: pg_info(1 pgs e194449:22.537) v4 -9575> 2014-08-13 16:24:07.079976 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079976, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.537) v4 -9574> 2014-08-13 16:24:07.079988 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.079988, event: started, request: pg_info(1 pgs e194449:22.537) v4 -9573> 2014-08-13 16:24:07.080011 7fd93dff0700 5 -- op tracker -- , seq: 650, time: 2014-08-13 16:24:07.080011, event: done, request: pg_info(1 pgs e194449:22.537) v4 -9572> 2014-08-13 16:24:07.080032 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 7 ==== pg_info(1 pgs e194449:22.2d1) v4 ==== 714+0+0 (2770177 0 0) 0x8ee88c0 con 0x50b9a20 -9571> 2014-08-13 16:24:07.080050 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.079789, event: header_read, request: pg_info(1 pgs e194449:22.2d1) v4 -9570> 2014-08-13 16:24:07.080061 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.079791, event: throttled, request: pg_info(1 pgs e194449:22.2d1) v4 -9569> 2014-08-13 16:24:07.080070 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.079920, event: all_read, request: pg_info(1 pgs e194449:22.2d1) v4 -9568> 2014-08-13 16:24:07.080079 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.080048, event: dispatched, request: pg_info(1 pgs e194449:22.2d1) v4 -9567> 2014-08-13 16:24:07.079650 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194449/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194448 pi=193678-194447/19 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 active+remapped m=62] target shard 59 from 0//0//-1 -9566> 2014-08-13 16:24:07.080090 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.080090, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.2d1) v4 -9565> 2014-08-13 16:24:07.080102 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.080102, event: started, request: pg_info(1 pgs e194449:22.2d1) v4 -9564> 2014-08-13 16:24:07.080124 7fd93dff0700 5 -- op tracker -- , seq: 651, time: 2014-08-13 16:24:07.080124, event: done, request: pg_info(1 pgs e194449:22.2d1) v4 -9563> 2014-08-13 16:24:07.079875 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194449/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194448 pi=193747-194447/18 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 active+remapped m=51] enter Started/Primary/Active/WaitLocalRecoveryReserved -9562> 2014-08-13 16:24:07.080162 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.294 192.168.1.101:6812/23402 8 ==== pg_info(1 pgs e194449:22.630) v4 ==== 714+0+0 (256035148 0 0) 0x775c7c0 con 0x4fd8b00 -9561> 2014-08-13 16:24:07.080184 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.079952, event: header_read, request: pg_info(1 pgs e194449:22.630) v4 -9560> 2014-08-13 16:24:07.080195 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.079954, event: throttled, request: pg_info(1 pgs e194449:22.630) v4 -9559> 2014-08-13 16:24:07.080204 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.080063, event: all_read, request: pg_info(1 pgs e194449:22.630) v4 -9558> 2014-08-13 16:24:07.080210 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.080181, event: dispatched, request: pg_info(1 pgs e194449:22.630) v4 -9557> 2014-08-13 16:24:07.080220 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.080219, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.630) v4 -9556> 2014-08-13 16:24:07.080230 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.080230, event: started, request: pg_info(1 pgs e194449:22.630) v4 -9555> 2014-08-13 16:24:07.080253 7fd93dff0700 5 -- op tracker -- , seq: 652, time: 2014-08-13 16:24:07.080253, event: done, request: pg_info(1 pgs e194449:22.630) v4 -9554> 2014-08-13 16:24:07.080277 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 8 ==== pg_info(1 pgs e194449:22.537) v4 ==== 714+0+0 (3748921824 0 0) 0x81c7380 con 0x50bc4c0 -9553> 2014-08-13 16:24:07.080296 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080050, event: header_read, request: pg_info(1 pgs e194449:22.537) v4 -9552> 2014-08-13 16:24:07.080306 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080052, event: throttled, request: pg_info(1 pgs e194449:22.537) v4 -9551> 2014-08-13 16:24:07.080315 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080129, event: all_read, request: pg_info(1 pgs e194449:22.537) v4 -9550> 2014-08-13 16:24:07.080324 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080294, event: dispatched, request: pg_info(1 pgs e194449:22.537) v4 -9549> 2014-08-13 16:24:07.080334 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080334, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.537) v4 -9548> 2014-08-13 16:24:07.080345 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080344, event: started, request: pg_info(1 pgs e194449:22.537) v4 -9547> 2014-08-13 16:24:07.080367 7fd93dff0700 5 -- op tracker -- , seq: 653, time: 2014-08-13 16:24:07.080366, event: done, request: pg_info(1 pgs e194449:22.537) v4 -9546> 2014-08-13 16:24:07.080383 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 8 ==== pg_info(1 pgs e194449:22.2d1) v4 ==== 714+0+0 (1629865229 0 0) 0x8c8f8c0 con 0x50b9a20 -9545> 2014-08-13 16:24:07.080401 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080140, event: header_read, request: pg_info(1 pgs e194449:22.2d1) v4 -9544> 2014-08-13 16:24:07.080411 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080142, event: throttled, request: pg_info(1 pgs e194449:22.2d1) v4 -9543> 2014-08-13 16:24:07.080420 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080225, event: all_read, request: pg_info(1 pgs e194449:22.2d1) v4 -9542> 2014-08-13 16:24:07.080429 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080399, event: dispatched, request: pg_info(1 pgs e194449:22.2d1) v4 -9541> 2014-08-13 16:24:07.080439 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080439, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.2d1) v4 -9540> 2014-08-13 16:24:07.080449 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080449, event: started, request: pg_info(1 pgs e194449:22.2d1) v4 -9539> 2014-08-13 16:24:07.080472 7fd93dff0700 5 -- op tracker -- , seq: 654, time: 2014-08-13 16:24:07.080472, event: done, request: pg_info(1 pgs e194449:22.2d1) v4 -9538> 2014-08-13 16:24:07.080500 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194449/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194448 pi=193679-194447/22 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] on activate: bft=344 from 0//0//-1 -9537> 2014-08-13 16:24:07.080782 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.364 192.168.1.114:6826/52317 -- pg_info(1 pgs e194449:24.1de) v4 -- ?+0 0x81c2f40 -9536> 2014-08-13 16:24:07.080510 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194449/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194448 pi=193748-194447/19 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 active+remapped m=59] exit Started/Primary/Active/Activating 1.618476 14 0.003704 -9535> 2014-08-13 16:24:07.080755 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194449/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194448 pi=193679-194447/22 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] target shard 344 from 0//0//-1 -9534> 2014-08-13 16:24:07.081015 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194449:22.3d5) v4 -- ?+0 0x81c1c00 -9533> 2014-08-13 16:24:07.081057 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194449:22.3d5) v4 -- ?+0 0x8c89a40 -9532> 2014-08-13 16:24:07.080835 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194449/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194448 pi=193748-194447/19 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 active+remapped m=59] enter Started/Primary/Active/WaitLocalRecoveryReserved -9531> 2014-08-13 16:24:07.081159 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194449/194389 194447/194448/194448) [4,339,45] r=0 lpr=194448 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.609276 12 0.013819 -9530> 2014-08-13 16:24:07.081186 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194449/194389 194447/194448/194448) [4,339,45] r=0 lpr=194448 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9529> 2014-08-13 16:24:07.081221 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194449/194389 194447/194448/194448) [4,5,47] r=0 lpr=194448 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.621131 12 0.000740 -9528> 2014-08-13 16:24:07.081246 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194449/194389 194447/194448/194448) [4,5,47] r=0 lpr=194448 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9527> 2014-08-13 16:24:07.081275 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.608262 8 0.000462 -9526> 2014-08-13 16:24:07.081300 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9525> 2014-08-13 16:24:07.081324 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000025 0 0.000000 -9524> 2014-08-13 16:24:07.081321 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.334 192.168.1.104:6808/62751 -- pg_info(1 pgs e194449:22.69f) v4 -- ?+0 0x7d06040 -9523> 2014-08-13 16:24:07.081351 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194443 194447/194448/194448) [4,340] r=0 lpr=194448 pi=194331-194447/10 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9522> 2014-08-13 16:24:07.081384 7fd9371e5700 0 log [WRN] : pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate -9521> 2014-08-13 16:24:07.081388 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:1.2b) v4 -- ?+0 0x775a300 -9520> 2014-08-13 16:24:07.081532 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.364 192.168.1.114:6826/52317 8 ==== pg_info(1 pgs e194449:24.1de) v4 ==== 748+0+0 (437095772 0 0) 0x8c8e3c0 con 0x4fe6ca0 -9519> 2014-08-13 16:24:07.081543 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,374,31] r=0 lpr=194448 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.607687 12 0.000623 -9518> 2014-08-13 16:24:07.081556 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.351 192.168.1.102:6830/16444 -- pg_info(1 pgs e194449:22.220) v4 -- ?+0 0x7d03100 -9517> 2014-08-13 16:24:07.081567 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081416, event: header_read, request: pg_info(1 pgs e194449:24.1de) v4 -9516> 2014-08-13 16:24:07.081580 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081418, event: throttled, request: pg_info(1 pgs e194449:24.1de) v4 -9515> 2014-08-13 16:24:07.081578 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/194448) [4,374,31] r=0 lpr=194448 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9514> 2014-08-13 16:24:07.081591 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081505, event: all_read, request: pg_info(1 pgs e194449:24.1de) v4 -9513> 2014-08-13 16:24:07.081598 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081563, event: dispatched, request: pg_info(1 pgs e194449:24.1de) v4 -9512> 2014-08-13 16:24:07.081617 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081617, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.1de) v4 -9511> 2014-08-13 16:24:07.081627 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081626, event: started, request: pg_info(1 pgs e194449:24.1de) v4 -9510> 2014-08-13 16:24:07.081654 7fd93dff0700 5 -- op tracker -- , seq: 655, time: 2014-08-13 16:24:07.081654, event: done, request: pg_info(1 pgs e194449:24.1de) v4 -9509> 2014-08-13 16:24:07.081657 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194449; pg1.2b; ) v2 -- ?+0 0x81c55c0 -9508> 2014-08-13 16:24:07.081843 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194449:22.791) v4 -- ?+0 0x7d01340 -9507> 2014-08-13 16:24:07.081609 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194449/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194448 pi=193678-194447/19 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 active+remapped m=62] exit Started/Primary/Active/Activating 1.605956 12 0.003856 -9506> 2014-08-13 16:24:07.081965 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.323 192.168.1.106:6816/8118 -- pg_info(1 pgs e194449:22.12a) v4 -- ?+0 0x7d00e00 -9505> 2014-08-13 16:24:07.082100 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194340 194447/194448/194448) [4,50,289] r=0 lpr=194448 pi=194331-194447/10 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.602934 10 0.000266 -9504> 2014-08-13 16:24:07.082135 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194340 194447/194448/194448) [4,50,289] r=0 lpr=194448 pi=194331-194447/10 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9503> 2014-08-13 16:24:07.082171 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194340 194447/194448/194448) [4,50,289] r=0 lpr=194448 pi=194331-194447/10 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000035 0 0.000000 -9502> 2014-08-13 16:24:07.082177 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.292 192.168.1.113:6812/16056 -- pg_info(1 pgs e194449:22.3ad) v4 -- ?+0 0x80d96c0 -9501> 2014-08-13 16:24:07.082197 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194340 194447/194448/194448) [4,50,289] r=0 lpr=194448 pi=194331-194447/10 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9500> 2014-08-13 16:24:07.081940 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194449/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194448 pi=193678-194447/19 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 active+remapped m=62] enter Started/Primary/Active/WaitLocalRecoveryReserved -9499> 2014-08-13 16:24:07.082269 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 7 ==== pg_info(1 pgs e194449:22.69f) v4 ==== 714+0+0 (1154808186 0 0) 0x7d06040 con 0x5112aa0 -9498> 2014-08-13 16:24:07.082320 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082123, event: header_read, request: pg_info(1 pgs e194449:22.69f) v4 -9497> 2014-08-13 16:24:07.082332 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082126, event: throttled, request: pg_info(1 pgs e194449:22.69f) v4 -9496> 2014-08-13 16:24:07.082338 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082243, event: all_read, request: pg_info(1 pgs e194449:22.69f) v4 -9495> 2014-08-13 16:24:07.082344 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082315, event: dispatched, request: pg_info(1 pgs e194449:22.69f) v4 -9494> 2014-08-13 16:24:07.082329 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194449:22.6bd) v4 -- ?+0 0x775d240 -9493> 2014-08-13 16:24:07.082352 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082351, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.69f) v4 -9492> 2014-08-13 16:24:07.082361 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082361, event: started, request: pg_info(1 pgs e194449:22.69f) v4 -9491> 2014-08-13 16:24:07.082375 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194449:22.6bd) v4 -- ?+0 0x77581c0 -9490> 2014-08-13 16:24:07.082388 7fd93dff0700 5 -- op tracker -- , seq: 656, time: 2014-08-13 16:24:07.082388, event: done, request: pg_info(1 pgs e194449:22.69f) v4 -9489> 2014-08-13 16:24:07.082408 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.351 192.168.1.102:6830/16444 11 ==== pg_info(1 pgs e194449:22.220) v4 ==== 714+0+0 (2998134269 0 0) 0x8c8f380 con 0x51148e0 -9488> 2014-08-13 16:24:07.082424 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082242, event: header_read, request: pg_info(1 pgs e194449:22.220) v4 -9487> 2014-08-13 16:24:07.082431 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082244, event: throttled, request: pg_info(1 pgs e194449:22.220) v4 -9486> 2014-08-13 16:24:07.082437 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082339, event: all_read, request: pg_info(1 pgs e194449:22.220) v4 -9485> 2014-08-13 16:24:07.082442 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082422, event: dispatched, request: pg_info(1 pgs e194449:22.220) v4 -9484> 2014-08-13 16:24:07.082448 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082448, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.220) v4 -9483> 2014-08-13 16:24:07.082455 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082455, event: started, request: pg_info(1 pgs e194449:22.220) v4 -9482> 2014-08-13 16:24:07.082474 7fd93dff0700 5 -- op tracker -- , seq: 657, time: 2014-08-13 16:24:07.082474, event: done, request: pg_info(1 pgs e194449:22.220) v4 -9481> 2014-08-13 16:24:07.082547 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194449/194389 194447/194448/194448) [4,44,362] r=0 lpr=194448 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.600551 12 0.000568 -9480> 2014-08-13 16:24:07.082579 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194449/194389 194447/194448/194448) [4,44,362] r=0 lpr=194448 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9479> 2014-08-13 16:24:07.082672 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.32 192.168.1.115:6812/59915 -- osd pg remove(epoch 194449; pg22.6bd; ) v2 -- ?+0 0x81c2300 -9478> 2014-08-13 16:24:07.082896 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 7 ==== pg_info(1 pgs e194449:22.3ad) v4 ==== 714+0+0 (146777198 0 0) 0x81c5240 con 0x510fd20 -9477> 2014-08-13 16:24:07.082949 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082768, event: header_read, request: pg_info(1 pgs e194449:22.3ad) v4 -9476> 2014-08-13 16:24:07.082959 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082771, event: throttled, request: pg_info(1 pgs e194449:22.3ad) v4 -9475> 2014-08-13 16:24:07.082965 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082868, event: all_read, request: pg_info(1 pgs e194449:22.3ad) v4 -9474> 2014-08-13 16:24:07.082976 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082944, event: dispatched, request: pg_info(1 pgs e194449:22.3ad) v4 -9473> 2014-08-13 16:24:07.082983 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082983, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.3ad) v4 -9472> 2014-08-13 16:24:07.082992 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.082992, event: started, request: pg_info(1 pgs e194449:22.3ad) v4 -9471> 2014-08-13 16:24:07.083017 7fd93dff0700 5 -- op tracker -- , seq: 658, time: 2014-08-13 16:24:07.083017, event: done, request: pg_info(1 pgs e194449:22.3ad) v4 -9470> 2014-08-13 16:24:07.082792 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194449/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194448 pi=193644-194447/18 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] on activate: bft=331,348 from 0//0//-1 -9469> 2014-08-13 16:24:07.083036 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 7 ==== pg_info(1 pgs e194449:22.791) v4 ==== 714+0+0 (2830330724 0 0) 0x7d01340 con 0x4e4f900 -9468> 2014-08-13 16:24:07.083052 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.082824, event: header_read, request: pg_info(1 pgs e194449:22.791) v4 -9467> 2014-08-13 16:24:07.083059 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.082826, event: throttled, request: pg_info(1 pgs e194449:22.791) v4 -9466> 2014-08-13 16:24:07.083065 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.082899, event: all_read, request: pg_info(1 pgs e194449:22.791) v4 -9465> 2014-08-13 16:24:07.083070 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.083050, event: dispatched, request: pg_info(1 pgs e194449:22.791) v4 -9464> 2014-08-13 16:24:07.083076 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.083076, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.791) v4 -9463> 2014-08-13 16:24:07.083084 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.083084, event: started, request: pg_info(1 pgs e194449:22.791) v4 -9462> 2014-08-13 16:24:07.083105 7fd93dff0700 5 -- op tracker -- , seq: 659, time: 2014-08-13 16:24:07.083105, event: done, request: pg_info(1 pgs e194449:22.791) v4 -9461> 2014-08-13 16:24:07.082862 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194449/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194448 pi=193679-194447/22 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] exit Started/Primary/Active/Activating 1.601883 13 0.003122 -9460> 2014-08-13 16:24:07.083119 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 8 ==== pg_info(1 pgs e194449:22.3ad) v4 ==== 714+0+0 (1497045211 0 0) 0x80d96c0 con 0x510fd20 -9459> 2014-08-13 16:24:07.083133 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.082903, event: header_read, request: pg_info(1 pgs e194449:22.3ad) v4 -9458> 2014-08-13 16:24:07.083140 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.082905, event: throttled, request: pg_info(1 pgs e194449:22.3ad) v4 -9457> 2014-08-13 16:24:07.083145 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.083082, event: all_read, request: pg_info(1 pgs e194449:22.3ad) v4 -9456> 2014-08-13 16:24:07.083150 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.083131, event: dispatched, request: pg_info(1 pgs e194449:22.3ad) v4 -9455> 2014-08-13 16:24:07.083156 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.083156, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.3ad) v4 -9454> 2014-08-13 16:24:07.083163 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.083163, event: started, request: pg_info(1 pgs e194449:22.3ad) v4 -9453> 2014-08-13 16:24:07.083192 7fd93dff0700 5 -- op tracker -- , seq: 660, time: 2014-08-13 16:24:07.083192, event: done, request: pg_info(1 pgs e194449:22.3ad) v4 -9452> 2014-08-13 16:24:07.083206 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.351 192.168.1.102:6830/16444 12 ==== MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 ==== 26+0+0 (1494517724 0 0) 0x7d03100 con 0x51148e0 -9451> 2014-08-13 16:24:07.083223 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083105, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9450> 2014-08-13 16:24:07.083229 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083107, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9449> 2014-08-13 16:24:07.083235 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083169, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9448> 2014-08-13 16:24:07.083240 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083221, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9447> 2014-08-13 16:24:07.083246 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083245, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9446> 2014-08-13 16:24:07.083261 7fd93dff0700 5 -- op tracker -- , seq: 661, time: 2014-08-13 16:24:07.083261, event: done, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194449 v2 -9445> 2014-08-13 16:24:07.083273 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 8 ==== pg_info(1 pgs e194449:22.69f) v4 ==== 714+0+0 (833546700 0 0) 0x8c8fc40 con 0x5112aa0 -9444> 2014-08-13 16:24:07.083287 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083107, event: header_read, request: pg_info(1 pgs e194449:22.69f) v4 -9443> 2014-08-13 16:24:07.083294 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083108, event: throttled, request: pg_info(1 pgs e194449:22.69f) v4 -9442> 2014-08-13 16:24:07.083299 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083179, event: all_read, request: pg_info(1 pgs e194449:22.69f) v4 -9441> 2014-08-13 16:24:07.083304 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083286, event: dispatched, request: pg_info(1 pgs e194449:22.69f) v4 -9440> 2014-08-13 16:24:07.083310 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083310, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.69f) v4 -9439> 2014-08-13 16:24:07.083317 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083317, event: started, request: pg_info(1 pgs e194449:22.69f) v4 -9438> 2014-08-13 16:24:07.083047 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194449/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194448 pi=193644-194447/18 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 331 from 0//0//-1 -9437> 2014-08-13 16:24:07.083335 7fd93dff0700 5 -- op tracker -- , seq: 662, time: 2014-08-13 16:24:07.083335, event: done, request: pg_info(1 pgs e194449:22.69f) v4 -9436> 2014-08-13 16:24:07.083124 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194449/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194448 pi=193679-194447/22 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] enter Started/Primary/Active/WaitLocalRecoveryReserved -9435> 2014-08-13 16:24:07.083385 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 8 ==== pg_info(1 pgs e194449:22.791) v4 ==== 714+0+0 (2553122374 0 0) 0x8eea4c0 con 0x4e4f900 -9434> 2014-08-13 16:24:07.083401 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083224, event: header_read, request: pg_info(1 pgs e194449:22.791) v4 -9433> 2014-08-13 16:24:07.083407 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083226, event: throttled, request: pg_info(1 pgs e194449:22.791) v4 -9432> 2014-08-13 16:24:07.083413 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083300, event: all_read, request: pg_info(1 pgs e194449:22.791) v4 -9431> 2014-08-13 16:24:07.083418 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083399, event: dispatched, request: pg_info(1 pgs e194449:22.791) v4 -9430> 2014-08-13 16:24:07.083424 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083424, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.791) v4 -9429> 2014-08-13 16:24:07.083431 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083431, event: started, request: pg_info(1 pgs e194449:22.791) v4 -9428> 2014-08-13 16:24:07.083449 7fd93dff0700 5 -- op tracker -- , seq: 663, time: 2014-08-13 16:24:07.083449, event: done, request: pg_info(1 pgs e194449:22.791) v4 -9427> 2014-08-13 16:24:07.083481 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 7 ==== pg_info(1 pgs e194449:22.12a) v4 ==== 714+0+0 (1261918296 0 0) 0x7d00e00 con 0x4fdd3e0 -9426> 2014-08-13 16:24:07.083495 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083165, event: header_read, request: pg_info(1 pgs e194449:22.12a) v4 -9425> 2014-08-13 16:24:07.083502 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083168, event: throttled, request: pg_info(1 pgs e194449:22.12a) v4 -9424> 2014-08-13 16:24:07.083507 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083300, event: all_read, request: pg_info(1 pgs e194449:22.12a) v4 -9423> 2014-08-13 16:24:07.083328 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194449/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194448 pi=193644-194447/18 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 348 from 0//0//-1 -9422> 2014-08-13 16:24:07.083512 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083493, event: dispatched, request: pg_info(1 pgs e194449:22.12a) v4 -9421> 2014-08-13 16:24:07.083626 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083626, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.12a) v4 -9420> 2014-08-13 16:24:07.083644 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083644, event: started, request: pg_info(1 pgs e194449:22.12a) v4 -9419> 2014-08-13 16:24:07.083666 7fd93dff0700 5 -- op tracker -- , seq: 664, time: 2014-08-13 16:24:07.083666, event: done, request: pg_info(1 pgs e194449:22.12a) v4 -9418> 2014-08-13 16:24:07.083701 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 8 ==== pg_info(1 pgs e194449:22.12a) v4 ==== 714+0+0 (1863186006 0 0) 0x8ee9dc0 con 0x4fdd3e0 -9417> 2014-08-13 16:24:07.083722 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083350, event: header_read, request: pg_info(1 pgs e194449:22.12a) v4 -9416> 2014-08-13 16:24:07.083733 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083353, event: throttled, request: pg_info(1 pgs e194449:22.12a) v4 -9415> 2014-08-13 16:24:07.083742 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083420, event: all_read, request: pg_info(1 pgs e194449:22.12a) v4 -9414> 2014-08-13 16:24:07.083751 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083719, event: dispatched, request: pg_info(1 pgs e194449:22.12a) v4 -9413> 2014-08-13 16:24:07.083760 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083760, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.12a) v4 -9412> 2014-08-13 16:24:07.083771 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083771, event: started, request: pg_info(1 pgs e194449:22.12a) v4 -9411> 2014-08-13 16:24:07.083806 7fd93dff0700 5 -- op tracker -- , seq: 665, time: 2014-08-13 16:24:07.083806, event: done, request: pg_info(1 pgs e194449:22.12a) v4 -9410> 2014-08-13 16:24:07.083922 7fd9367e4700 0 log [WRN] : pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate -9409> 2014-08-13 16:24:07.083959 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194340 194447/194448/194448) [4,340,347] r=0 lpr=194448 pi=194331-194447/10 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.607187 10 0.000236 -9408> 2014-08-13 16:24:07.083988 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194340 194447/194448/194448) [4,340,347] r=0 lpr=194448 pi=194331-194447/10 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9407> 2014-08-13 16:24:07.084014 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194340 194447/194448/194448) [4,340,347] r=0 lpr=194448 pi=194331-194447/10 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000026 0 0.000000 -9406> 2014-08-13 16:24:07.084036 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194340 194447/194448/194448) [4,340,347] r=0 lpr=194448 pi=194331-194447/10 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9405> 2014-08-13 16:24:07.084073 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x775ec80 -9404> 2014-08-13 16:24:07.084103 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x775f380 -9403> 2014-08-13 16:24:07.084255 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/194448) [4,289,341] r=0 lpr=194448 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.601183 11 0.000630 -9402> 2014-08-13 16:24:07.084288 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/194448) [4,289,341] r=0 lpr=194448 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9401> 2014-08-13 16:24:07.084438 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194449; pg22.1b8; ) v2 -- ?+0 0x8c8a4c0 -9400> 2014-08-13 16:24:07.084315 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194449/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194448 pi=193644-194447/18 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] exit Started/Primary/Active/Activating 1.598829 14 0.003823 -9399> 2014-08-13 16:24:07.084697 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.325 192.168.1.110:6804/25590 -- pg_info(1 pgs e194449:24.39a) v4 -- ?+0 0x80dcb40 -9398> 2014-08-13 16:24:07.084575 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194449/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194448 pi=193644-194447/18 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] enter Started/Primary/Active/WaitLocalRecoveryReserved -9397> 2014-08-13 16:24:07.085690 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194449/194304 194447/194448/194448) [4,287,309] r=0 lpr=194448 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.603167 13 0.000579 -9396> 2014-08-13 16:24:07.085725 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194449/194304 194447/194448/194448) [4,287,309] r=0 lpr=194448 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -9395> 2014-08-13 16:24:07.086022 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194448 pi=192970-194447/19 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.597836 4 0.000120 -9394> 2014-08-13 16:24:07.086054 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194448 pi=192970-194447/19 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -9393> 2014-08-13 16:24:07.086263 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194340 194447/194448/194448) [4,343,286] r=0 lpr=194448 pi=194331-194447/10 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 1.600268 10 0.000167 -9392> 2014-08-13 16:24:07.086294 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194340 194447/194448/194448) [4,343,286] r=0 lpr=194448 pi=194331-194447/10 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -9391> 2014-08-13 16:24:07.086320 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194340 194447/194448/194448) [4,343,286] r=0 lpr=194448 pi=194331-194447/10 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000027 0 0.000000 -9390> 2014-08-13 16:24:07.086343 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194340 194447/194448/194448) [4,343,286] r=0 lpr=194448 pi=194331-194447/10 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -9389> 2014-08-13 16:24:07.086379 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194449:22.3d5) v4 -- ?+0 0x8c8b2c0 -9388> 2014-08-13 16:24:07.086414 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194449:22.3d5) v4 -- ?+0 0x8c88540 -9387> 2014-08-13 16:24:07.086821 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.325 192.168.1.110:6804/25590 8 ==== pg_info(1 pgs e194449:24.39a) v4 ==== 748+0+0 (1725253058 0 0) 0x80dcb40 con 0x78b0580 -9386> 2014-08-13 16:24:07.086852 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086687, event: header_read, request: pg_info(1 pgs e194449:24.39a) v4 -9385> 2014-08-13 16:24:07.086867 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086689, event: throttled, request: pg_info(1 pgs e194449:24.39a) v4 -9384> 2014-08-13 16:24:07.086876 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086773, event: all_read, request: pg_info(1 pgs e194449:24.39a) v4 -9383> 2014-08-13 16:24:07.086886 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086847, event: dispatched, request: pg_info(1 pgs e194449:24.39a) v4 -9382> 2014-08-13 16:24:07.086897 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086896, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.39a) v4 -9381> 2014-08-13 16:24:07.086943 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086943, event: started, request: pg_info(1 pgs e194449:24.39a) v4 -9380> 2014-08-13 16:24:07.086981 7fd93dff0700 5 -- op tracker -- , seq: 666, time: 2014-08-13 16:24:07.086981, event: done, request: pg_info(1 pgs e194449:24.39a) v4 -9379> 2014-08-13 16:24:07.087087 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.346 192.168.1.104:6820/2308 -- pg_info(1 pgs e194449:24.e2) v4 -- ?+0 0x81c3100 -9378> 2014-08-13 16:24:07.087164 7fd942628700 1 -- 192.168.1.115:6813/25220 --> osd.10 192.168.1.115:6815/60573 -- osd pg remove(epoch 194449; pg22.3d5; ) v2 -- ?+0 0x8c8f700 -9377> 2014-08-13 16:24:07.087523 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.9 192.168.1.107:6817/53997 -- pg_info(1 pgs e194449:22.28f) v4 -- ?+0 0x8c8ad80 -9376> 2014-08-13 16:24:07.087942 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194449:22.14a) v4 -- ?+0 0x8c8e040 -9375> 2014-08-13 16:24:07.088339 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.54 192.168.1.111:6836/14923 -- pg_info(1 pgs e194449:22.30f) v4 -- ?+0 0x8c8d780 -9374> 2014-08-13 16:24:07.088530 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.346 192.168.1.104:6820/2308 8 ==== pg_info(1 pgs e194449:24.e2) v4 ==== 748+0+0 (2857988545 0 0) 0x81c3100 con 0x5112d60 -9373> 2014-08-13 16:24:07.088570 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088174, event: header_read, request: pg_info(1 pgs e194449:24.e2) v4 -9372> 2014-08-13 16:24:07.088593 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088184, event: throttled, request: pg_info(1 pgs e194449:24.e2) v4 -9371> 2014-08-13 16:24:07.088603 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088257, event: all_read, request: pg_info(1 pgs e194449:24.e2) v4 -9370> 2014-08-13 16:24:07.088613 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088565, event: dispatched, request: pg_info(1 pgs e194449:24.e2) v4 -9369> 2014-08-13 16:24:07.088605 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194449:22.116) v4 -- ?+0 0x4f89500 -9368> 2014-08-13 16:24:07.088625 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088625, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.e2) v4 -9367> 2014-08-13 16:24:07.088639 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088638, event: started, request: pg_info(1 pgs e194449:24.e2) v4 -9366> 2014-08-13 16:24:07.088675 7fd93dff0700 5 -- op tracker -- , seq: 667, time: 2014-08-13 16:24:07.088675, event: done, request: pg_info(1 pgs e194449:24.e2) v4 -9365> 2014-08-13 16:24:07.088719 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 7 ==== pg_info(1 pgs e194449:22.28f) v4 ==== 714+0+0 (1447514413 0 0) 0x8c8ad80 con 0x4e498c0 -9364> 2014-08-13 16:24:07.088741 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088399, event: header_read, request: pg_info(1 pgs e194449:22.28f) v4 -9363> 2014-08-13 16:24:07.088752 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088401, event: throttled, request: pg_info(1 pgs e194449:22.28f) v4 -9362> 2014-08-13 16:24:07.088762 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088487, event: all_read, request: pg_info(1 pgs e194449:22.28f) v4 -9361> 2014-08-13 16:24:07.088771 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088738, event: dispatched, request: pg_info(1 pgs e194449:22.28f) v4 -9360> 2014-08-13 16:24:07.088781 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088780, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.28f) v4 -9359> 2014-08-13 16:24:07.088795 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088795, event: started, request: pg_info(1 pgs e194449:22.28f) v4 -9358> 2014-08-13 16:24:07.088815 7fd93dff0700 5 -- op tracker -- , seq: 668, time: 2014-08-13 16:24:07.088815, event: done, request: pg_info(1 pgs e194449:22.28f) v4 -9357> 2014-08-13 16:24:07.088849 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 15 ==== pg_info(1 pgs e194449:22.14a) v4 ==== 714+0+0 (2368288056 0 0) 0x8c8e040 con 0x4e4dd80 -9356> 2014-08-13 16:24:07.088869 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088716, event: header_read, request: pg_info(1 pgs e194449:22.14a) v4 -9355> 2014-08-13 16:24:07.088880 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088717, event: throttled, request: pg_info(1 pgs e194449:22.14a) v4 -9354> 2014-08-13 16:24:07.088890 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088784, event: all_read, request: pg_info(1 pgs e194449:22.14a) v4 -9353> 2014-08-13 16:24:07.088899 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088867, event: dispatched, request: pg_info(1 pgs e194449:22.14a) v4 -9352> 2014-08-13 16:24:07.088908 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088908, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.14a) v4 -9351> 2014-08-13 16:24:07.088919 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088919, event: started, request: pg_info(1 pgs e194449:22.14a) v4 -9350> 2014-08-13 16:24:07.088944 7fd93dff0700 5 -- op tracker -- , seq: 669, time: 2014-08-13 16:24:07.088944, event: done, request: pg_info(1 pgs e194449:22.14a) v4 -9349> 2014-08-13 16:24:07.088987 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 8 ==== pg_info(1 pgs e194449:22.28f) v4 ==== 714+0+0 (1238022160 0 0) 0x7d01c00 con 0x4e498c0 -9348> 2014-08-13 16:24:07.089008 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.088854, event: header_read, request: pg_info(1 pgs e194449:22.28f) v4 -9347> 2014-08-13 16:24:07.089019 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.088856, event: throttled, request: pg_info(1 pgs e194449:22.28f) v4 -9346> 2014-08-13 16:24:07.089028 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.088925, event: all_read, request: pg_info(1 pgs e194449:22.28f) v4 -9345> 2014-08-13 16:24:07.089034 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.089005, event: dispatched, request: pg_info(1 pgs e194449:22.28f) v4 -9344> 2014-08-13 16:24:07.089044 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.089043, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.28f) v4 -9343> 2014-08-13 16:24:07.089055 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.089055, event: started, request: pg_info(1 pgs e194449:22.28f) v4 -9342> 2014-08-13 16:24:07.089083 7fd93dff0700 5 -- op tracker -- , seq: 670, time: 2014-08-13 16:24:07.089083, event: done, request: pg_info(1 pgs e194449:22.28f) v4 -9341> 2014-08-13 16:24:07.089271 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 9 ==== pg_info(1 pgs e194449:22.30f) v4 ==== 714+0+0 (1084439927 0 0) 0x8c8d780 con 0x4e4ce60 -9340> 2014-08-13 16:24:07.089316 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089086, event: header_read, request: pg_info(1 pgs e194449:22.30f) v4 -9339> 2014-08-13 16:24:07.089328 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089088, event: throttled, request: pg_info(1 pgs e194449:22.30f) v4 -9338> 2014-08-13 16:24:07.089335 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089178, event: all_read, request: pg_info(1 pgs e194449:22.30f) v4 -9337> 2014-08-13 16:24:07.089340 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089309, event: dispatched, request: pg_info(1 pgs e194449:22.30f) v4 -9336> 2014-08-13 16:24:07.089352 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089351, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.30f) v4 -9335> 2014-08-13 16:24:07.089361 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089361, event: started, request: pg_info(1 pgs e194449:22.30f) v4 -9334> 2014-08-13 16:24:07.089391 7fd93dff0700 5 -- op tracker -- , seq: 671, time: 2014-08-13 16:24:07.089391, event: done, request: pg_info(1 pgs e194449:22.30f) v4 -9333> 2014-08-13 16:24:07.089433 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 12 ==== pg_info(1 pgs e194449:22.116) v4 ==== 714+0+0 (1335434653 0 0) 0x4f89500 con 0x4fe0000 -9332> 2014-08-13 16:24:07.089457 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089232, event: header_read, request: pg_info(1 pgs e194449:22.116) v4 -9331> 2014-08-13 16:24:07.089464 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089233, event: throttled, request: pg_info(1 pgs e194449:22.116) v4 -9330> 2014-08-13 16:24:07.089470 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089307, event: all_read, request: pg_info(1 pgs e194449:22.116) v4 -9329> 2014-08-13 16:24:07.089479 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089454, event: dispatched, request: pg_info(1 pgs e194449:22.116) v4 -9328> 2014-08-13 16:24:07.089489 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089489, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.116) v4 -9327> 2014-08-13 16:24:07.089500 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089500, event: started, request: pg_info(1 pgs e194449:22.116) v4 -9326> 2014-08-13 16:24:07.089527 7fd93dff0700 5 -- op tracker -- , seq: 672, time: 2014-08-13 16:24:07.089527, event: done, request: pg_info(1 pgs e194449:22.116) v4 -9325> 2014-08-13 16:24:07.089581 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 16 ==== pg_info(1 pgs e194449:22.14a) v4 ==== 714+0+0 (3306083192 0 0) 0x775e040 con 0x4e4dd80 -9324> 2014-08-13 16:24:07.089603 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089364, event: header_read, request: pg_info(1 pgs e194449:22.14a) v4 -9323> 2014-08-13 16:24:07.089615 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089366, event: throttled, request: pg_info(1 pgs e194449:22.14a) v4 -9322> 2014-08-13 16:24:07.089624 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089437, event: all_read, request: pg_info(1 pgs e194449:22.14a) v4 -9321> 2014-08-13 16:24:07.089633 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089601, event: dispatched, request: pg_info(1 pgs e194449:22.14a) v4 -9320> 2014-08-13 16:24:07.089644 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089644, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.14a) v4 -9319> 2014-08-13 16:24:07.089652 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089651, event: started, request: pg_info(1 pgs e194449:22.14a) v4 -9318> 2014-08-13 16:24:07.089680 7fd93dff0700 5 -- op tracker -- , seq: 673, time: 2014-08-13 16:24:07.089680, event: done, request: pg_info(1 pgs e194449:22.14a) v4 -9317> 2014-08-13 16:24:07.089736 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 10 ==== pg_info(1 pgs e194449:22.30f) v4 ==== 714+0+0 (2031657330 0 0) 0x7d078c0 con 0x4e4ce60 -9316> 2014-08-13 16:24:07.089757 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089473, event: header_read, request: pg_info(1 pgs e194449:22.30f) v4 -9315> 2014-08-13 16:24:07.089768 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089474, event: throttled, request: pg_info(1 pgs e194449:22.30f) v4 -9314> 2014-08-13 16:24:07.089777 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089532, event: all_read, request: pg_info(1 pgs e194449:22.30f) v4 -9313> 2014-08-13 16:24:07.089786 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089754, event: dispatched, request: pg_info(1 pgs e194449:22.30f) v4 -9312> 2014-08-13 16:24:07.089796 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089796, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.30f) v4 -9311> 2014-08-13 16:24:07.089807 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089807, event: started, request: pg_info(1 pgs e194449:22.30f) v4 -9310> 2014-08-13 16:24:07.089833 7fd93dff0700 5 -- op tracker -- , seq: 674, time: 2014-08-13 16:24:07.089833, event: done, request: pg_info(1 pgs e194449:22.30f) v4 -9309> 2014-08-13 16:24:07.089867 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.346 192.168.1.104:6820/2308 9 ==== MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 ==== 26+0+0 (1353383872 0 0) 0x81c3f00 con 0x5112d60 -9308> 2014-08-13 16:24:07.089884 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089700, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9307> 2014-08-13 16:24:07.089892 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089703, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9306> 2014-08-13 16:24:07.089897 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089769, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9305> 2014-08-13 16:24:07.089902 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089882, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9304> 2014-08-13 16:24:07.089915 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089915, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9303> 2014-08-13 16:24:07.089934 7fd93dff0700 5 -- op tracker -- , seq: 675, time: 2014-08-13 16:24:07.089933, event: done, request: MRecoveryReserve REQUEST pgid: 24.e2, query_epoch: 194449 v2 -9302> 2014-08-13 16:24:07.089947 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194448 pi=193125-194447/21 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] exit Started/ReplicaActive/RepNotRecovering 1.605155 4 0.002820 -9301> 2014-08-13 16:24:07.089964 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 13 ==== MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 ==== 26+0+0 (3196491682 0 0) 0x81c4600 con 0x4fe0000 -9300> 2014-08-13 16:24:07.090003 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.089853, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9299> 2014-08-13 16:24:07.089993 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194448 pi=193125-194447/21 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] enter Started/ReplicaActive/RepWaitRecoveryReserved -9298> 2014-08-13 16:24:07.090018 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.089856, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9297> 2014-08-13 16:24:07.090025 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.089916, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9296> 2014-08-13 16:24:07.090034 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.090000, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9295> 2014-08-13 16:24:07.090043 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.090043, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9294> 2014-08-13 16:24:07.090065 7fd93dff0700 5 -- op tracker -- , seq: 676, time: 2014-08-13 16:24:07.090065, event: done, request: MRecoveryReserve REQUEST pgid: 22.116, query_epoch: 194449 v2 -9293> 2014-08-13 16:24:07.090077 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194448 pi=192981-194447/17 luod=0'0 crt=192888'464 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.596874 4 0.000121 -9292> 2014-08-13 16:24:07.090097 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194449:24.3f7) v4 -- ?+0 0x7d06ac0 -9291> 2014-08-13 16:24:07.090118 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194448 pi=192981-194447/17 luod=0'0 crt=192888'464 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -9290> 2014-08-13 16:24:07.091171 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 14 ==== pg_info(1 pgs e194449:24.3f7) v4 ==== 748+0+0 (3003290813 0 0) 0x7d00a80 con 0x5113b20 -9289> 2014-08-13 16:24:07.091225 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.090993, event: header_read, request: pg_info(1 pgs e194449:24.3f7) v4 -9288> 2014-08-13 16:24:07.091241 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.090994, event: throttled, request: pg_info(1 pgs e194449:24.3f7) v4 -9287> 2014-08-13 16:24:07.091252 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.091085, event: all_read, request: pg_info(1 pgs e194449:24.3f7) v4 -9286> 2014-08-13 16:24:07.091261 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.091218, event: dispatched, request: pg_info(1 pgs e194449:24.3f7) v4 -9285> 2014-08-13 16:24:07.091272 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.091272, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.3f7) v4 -9284> 2014-08-13 16:24:07.091282 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.091282, event: started, request: pg_info(1 pgs e194449:24.3f7) v4 -9283> 2014-08-13 16:24:07.091316 7fd93dff0700 5 -- op tracker -- , seq: 677, time: 2014-08-13 16:24:07.091316, event: done, request: pg_info(1 pgs e194449:24.3f7) v4 -9282> 2014-08-13 16:24:07.092274 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194449:24.3b7) v4 -- ?+0 0x7758e00 -9281> 2014-08-13 16:24:07.092821 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194449:22.639) v4 -- ?+0 0x81c1dc0 -9280> 2014-08-13 16:24:07.093521 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 15 ==== MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 ==== 26+0+0 (2329488420 0 0) 0x7d06ac0 con 0x5113b20 -9279> 2014-08-13 16:24:07.093565 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093355, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9278> 2014-08-13 16:24:07.093578 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093358, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9277> 2014-08-13 16:24:07.093587 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093423, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9276> 2014-08-13 16:24:07.093597 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093559, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9275> 2014-08-13 16:24:07.093608 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093608, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9274> 2014-08-13 16:24:07.093631 7fd93dff0700 5 -- op tracker -- , seq: 678, time: 2014-08-13 16:24:07.093631, event: done, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194449 v2 -9273> 2014-08-13 16:24:07.093674 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 21 ==== pg_info(1 pgs e194449:24.3b7) v4 ==== 748+0+0 (2111446660 0 0) 0x7758e00 con 0x4fd8000 -9272> 2014-08-13 16:24:07.093695 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093386, event: header_read, request: pg_info(1 pgs e194449:24.3b7) v4 -9271> 2014-08-13 16:24:07.093703 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093388, event: throttled, request: pg_info(1 pgs e194449:24.3b7) v4 -9270> 2014-08-13 16:24:07.093709 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093469, event: all_read, request: pg_info(1 pgs e194449:24.3b7) v4 -9269> 2014-08-13 16:24:07.093714 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093692, event: dispatched, request: pg_info(1 pgs e194449:24.3b7) v4 -9268> 2014-08-13 16:24:07.093721 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093721, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.3b7) v4 -9267> 2014-08-13 16:24:07.093694 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194448 pi=193227-194447/23 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] exit Started/ReplicaActive/RepNotRecovering 1.603835 4 0.002720 -9266> 2014-08-13 16:24:07.093729 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093729, event: started, request: pg_info(1 pgs e194449:24.3b7) v4 -9265> 2014-08-13 16:24:07.093739 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194448 pi=193227-194447/23 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] enter Started/ReplicaActive/RepWaitRecoveryReserved -9264> 2014-08-13 16:24:07.093757 7fd93dff0700 5 -- op tracker -- , seq: 679, time: 2014-08-13 16:24:07.093757, event: done, request: pg_info(1 pgs e194449:24.3b7) v4 -9263> 2014-08-13 16:24:07.094041 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 11 ==== pg_info(1 pgs e194449:22.639) v4 ==== 714+0+0 (3115355863 0 0) 0x81c5400 con 0x5112520 -9262> 2014-08-13 16:24:07.094084 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.093817, event: header_read, request: pg_info(1 pgs e194449:22.639) v4 -9261> 2014-08-13 16:24:07.094097 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.093820, event: throttled, request: pg_info(1 pgs e194449:22.639) v4 -9260> 2014-08-13 16:24:07.094103 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.093944, event: all_read, request: pg_info(1 pgs e194449:22.639) v4 -9259> 2014-08-13 16:24:07.094109 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.094078, event: dispatched, request: pg_info(1 pgs e194449:22.639) v4 -9258> 2014-08-13 16:24:07.094116 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.094115, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.639) v4 -9257> 2014-08-13 16:24:07.094125 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.094125, event: started, request: pg_info(1 pgs e194449:22.639) v4 -9256> 2014-08-13 16:24:07.094121 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.15 192.168.1.107:6812/53408 -- pg_info(1 pgs e194449:24.20e) v4 -- ?+0 0x81c7540 -9255> 2014-08-13 16:24:07.094295 7fd93dff0700 5 -- op tracker -- , seq: 680, time: 2014-08-13 16:24:07.094294, event: done, request: pg_info(1 pgs e194449:22.639) v4 -9254> 2014-08-13 16:24:07.094357 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 12 ==== pg_info(1 pgs e194449:22.639) v4 ==== 714+0+0 (2481835892 0 0) 0x81c1dc0 con 0x5112520 -9253> 2014-08-13 16:24:07.094380 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094136, event: header_read, request: pg_info(1 pgs e194449:22.639) v4 -9252> 2014-08-13 16:24:07.094388 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094138, event: throttled, request: pg_info(1 pgs e194449:22.639) v4 -9251> 2014-08-13 16:24:07.094380 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.371 192.168.1.102:6832/16753 -- pg_info(1 pgs e194449:22.653) v4 -- ?+0 0x81c4b40 -9250> 2014-08-13 16:24:07.094394 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094226, event: all_read, request: pg_info(1 pgs e194449:22.639) v4 -9249> 2014-08-13 16:24:07.094406 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094377, event: dispatched, request: pg_info(1 pgs e194449:22.639) v4 -9248> 2014-08-13 16:24:07.094413 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094412, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.639) v4 -9247> 2014-08-13 16:24:07.094421 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094421, event: started, request: pg_info(1 pgs e194449:22.639) v4 -9246> 2014-08-13 16:24:07.094511 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.320 192.168.1.106:6820/10050 -- pg_info(1 pgs e194449:22.581) v4 -- ?+0 0x775c0c0 -9245> 2014-08-13 16:24:07.094527 7fd93dff0700 5 -- op tracker -- , seq: 681, time: 2014-08-13 16:24:07.094526, event: done, request: pg_info(1 pgs e194449:22.639) v4 -9244> 2014-08-13 16:24:07.094713 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194449:22.1a0) v4 -- ?+0 0x775b480 -9243> 2014-08-13 16:24:07.095269 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 11 ==== pg_info(1 pgs e194449:24.20e) v4 ==== 748+0+0 (1452068703 0 0) 0x775f540 con 0x50ba520 -9242> 2014-08-13 16:24:07.095298 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095074, event: header_read, request: pg_info(1 pgs e194449:24.20e) v4 -9241> 2014-08-13 16:24:07.095310 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095077, event: throttled, request: pg_info(1 pgs e194449:24.20e) v4 -9240> 2014-08-13 16:24:07.095319 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095221, event: all_read, request: pg_info(1 pgs e194449:24.20e) v4 -9239> 2014-08-13 16:24:07.095328 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095294, event: dispatched, request: pg_info(1 pgs e194449:24.20e) v4 -9238> 2014-08-13 16:24:07.095338 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095338, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.20e) v4 -9237> 2014-08-13 16:24:07.095350 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095349, event: started, request: pg_info(1 pgs e194449:24.20e) v4 -9236> 2014-08-13 16:24:07.095375 7fd93dff0700 5 -- op tracker -- , seq: 682, time: 2014-08-13 16:24:07.095375, event: done, request: pg_info(1 pgs e194449:24.20e) v4 -9235> 2014-08-13 16:24:07.095412 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 9 ==== pg_info(1 pgs e194449:22.653) v4 ==== 714+0+0 (4069773092 0 0) 0x81c4b40 con 0x5112260 -9234> 2014-08-13 16:24:07.095431 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095167, event: header_read, request: pg_info(1 pgs e194449:22.653) v4 -9233> 2014-08-13 16:24:07.095439 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095177, event: throttled, request: pg_info(1 pgs e194449:22.653) v4 -9232> 2014-08-13 16:24:07.095444 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095259, event: all_read, request: pg_info(1 pgs e194449:22.653) v4 -9231> 2014-08-13 16:24:07.095449 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095429, event: dispatched, request: pg_info(1 pgs e194449:22.653) v4 -9230> 2014-08-13 16:24:07.095456 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095456, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.653) v4 -9229> 2014-08-13 16:24:07.095467 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095467, event: started, request: pg_info(1 pgs e194449:22.653) v4 -9228> 2014-08-13 16:24:07.095492 7fd93dff0700 5 -- op tracker -- , seq: 683, time: 2014-08-13 16:24:07.095492, event: done, request: pg_info(1 pgs e194449:22.653) v4 -9227> 2014-08-13 16:24:07.095636 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 12 ==== pg_info(1 pgs e194449:22.1a0) v4 ==== 714+0+0 (3808893691 0 0) 0x775b480 con 0x4fe3700 -9226> 2014-08-13 16:24:07.095659 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095508, event: header_read, request: pg_info(1 pgs e194449:22.1a0) v4 -9225> 2014-08-13 16:24:07.095670 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095512, event: throttled, request: pg_info(1 pgs e194449:22.1a0) v4 -9224> 2014-08-13 16:24:07.095679 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095599, event: all_read, request: pg_info(1 pgs e194449:22.1a0) v4 -9223> 2014-08-13 16:24:07.095688 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095656, event: dispatched, request: pg_info(1 pgs e194449:22.1a0) v4 -9222> 2014-08-13 16:24:07.095698 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095698, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.1a0) v4 -9221> 2014-08-13 16:24:07.095709 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095709, event: started, request: pg_info(1 pgs e194449:22.1a0) v4 -9220> 2014-08-13 16:24:07.095736 7fd93dff0700 5 -- op tracker -- , seq: 684, time: 2014-08-13 16:24:07.095736, event: done, request: pg_info(1 pgs e194449:22.1a0) v4 -9219> 2014-08-13 16:24:07.095773 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 14 ==== pg_info(1 pgs e194449:22.581) v4 ==== 714+0+0 (1135940708 0 0) 0x775c0c0 con 0x5114360 -9218> 2014-08-13 16:24:07.095789 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095632, event: header_read, request: pg_info(1 pgs e194449:22.581) v4 -9217> 2014-08-13 16:24:07.095796 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095634, event: throttled, request: pg_info(1 pgs e194449:22.581) v4 -9216> 2014-08-13 16:24:07.095801 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095717, event: all_read, request: pg_info(1 pgs e194449:22.581) v4 -9215> 2014-08-13 16:24:07.095807 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095787, event: dispatched, request: pg_info(1 pgs e194449:22.581) v4 -9214> 2014-08-13 16:24:07.095813 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095813, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.581) v4 -9213> 2014-08-13 16:24:07.095820 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095820, event: started, request: pg_info(1 pgs e194449:22.581) v4 -9212> 2014-08-13 16:24:07.095843 7fd93dff0700 5 -- op tracker -- , seq: 685, time: 2014-08-13 16:24:07.095843, event: done, request: pg_info(1 pgs e194449:22.581) v4 -9211> 2014-08-13 16:24:07.095918 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 10 ==== MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 ==== 26+0+0 (4036278383 0 0) 0x7d04440 con 0x5112260 -9210> 2014-08-13 16:24:07.095943 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095816, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9209> 2014-08-13 16:24:07.095951 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095817, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9208> 2014-08-13 16:24:07.095956 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095884, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9207> 2014-08-13 16:24:07.095961 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095940, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9206> 2014-08-13 16:24:07.095967 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095967, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9205> 2014-08-13 16:24:07.095991 7fd93dff0700 5 -- op tracker -- , seq: 686, time: 2014-08-13 16:24:07.095991, event: done, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194449 v2 -9204> 2014-08-13 16:24:07.096004 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194448 pi=192977-194447/17 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.597469 4 0.000091 -9203> 2014-08-13 16:24:07.096044 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194448 pi=192977-194447/17 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -9202> 2014-08-13 16:24:07.096534 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 12 ==== MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 ==== 26+0+0 (627049433 0 0) 0x81c7540 con 0x50ba520 -9201> 2014-08-13 16:24:07.096557 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096416, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9200> 2014-08-13 16:24:07.096564 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096418, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9199> 2014-08-13 16:24:07.096570 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096502, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9198> 2014-08-13 16:24:07.096575 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096554, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9197> 2014-08-13 16:24:07.096580 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096580, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9196> 2014-08-13 16:24:07.096598 7fd93dff0700 5 -- op tracker -- , seq: 687, time: 2014-08-13 16:24:07.096598, event: done, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194449 v2 -9195> 2014-08-13 16:24:07.096611 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194448 pi=193264-194447/21 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 1.601870 4 0.004004 -9194> 2014-08-13 16:24:07.096634 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 15 ==== MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 ==== 26+0+0 (2009478687 0 0) 0x81c78c0 con 0x5114360 -9193> 2014-08-13 16:24:07.096650 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096470, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9192> 2014-08-13 16:24:07.096657 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096472, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9191> 2014-08-13 16:24:07.096645 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194448 pi=193264-194447/21 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved -9190> 2014-08-13 16:24:07.096663 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096532, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9189> 2014-08-13 16:24:07.096668 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096648, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9188> 2014-08-13 16:24:07.096674 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096674, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9187> 2014-08-13 16:24:07.096693 7fd93dff0700 5 -- op tracker -- , seq: 688, time: 2014-08-13 16:24:07.096693, event: done, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194449 v2 -9186> 2014-08-13 16:24:07.096705 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194448 pi=192979-194447/19 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.598009 4 0.000089 -9185> 2014-08-13 16:24:07.096722 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 13 ==== pg_info(1 pgs e194449:22.1a0) v4 ==== 714+0+0 (375615943 0 0) 0x8db01c0 con 0x4fe3700 -9184> 2014-08-13 16:24:07.096738 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096481, event: header_read, request: pg_info(1 pgs e194449:22.1a0) v4 -9183> 2014-08-13 16:24:07.096733 7fd9367e4700 5 osd.4 pg_epoch: 194449 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194448 pi=192979-194447/19 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -9182> 2014-08-13 16:24:07.096745 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096484, event: throttled, request: pg_info(1 pgs e194449:22.1a0) v4 -9181> 2014-08-13 16:24:07.096751 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096585, event: all_read, request: pg_info(1 pgs e194449:22.1a0) v4 -9180> 2014-08-13 16:24:07.096756 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096736, event: dispatched, request: pg_info(1 pgs e194449:22.1a0) v4 -9179> 2014-08-13 16:24:07.096762 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096762, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.1a0) v4 -9178> 2014-08-13 16:24:07.096769 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096769, event: started, request: pg_info(1 pgs e194449:22.1a0) v4 -9177> 2014-08-13 16:24:07.096801 7fd93dff0700 5 -- op tracker -- , seq: 689, time: 2014-08-13 16:24:07.096801, event: done, request: pg_info(1 pgs e194449:22.1a0) v4 -9176> 2014-08-13 16:24:07.097554 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194449:24.2c) v4 -- ?+0 0x7759180 -9175> 2014-08-13 16:24:07.097845 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.397 192.168.1.105:6808/41085 -- pg_info(1 pgs e194449:22.3b9) v4 -- ?+0 0x7758540 -9174> 2014-08-13 16:24:07.098054 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194449:22.4fe) v4 -- ?+0 0x775a4c0 -9173> 2014-08-13 16:24:07.098914 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 14 ==== pg_info(1 pgs e194449:24.2c) v4 ==== 744+0+0 (332906907 0 0) 0x80dcb40 con 0x4fe1600 -9172> 2014-08-13 16:24:07.098938 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.098766, event: header_read, request: pg_info(1 pgs e194449:24.2c) v4 -9171> 2014-08-13 16:24:07.098949 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.098768, event: throttled, request: pg_info(1 pgs e194449:24.2c) v4 -9170> 2014-08-13 16:24:07.098970 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.098856, event: all_read, request: pg_info(1 pgs e194449:24.2c) v4 -9169> 2014-08-13 16:24:07.098980 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.098935, event: dispatched, request: pg_info(1 pgs e194449:24.2c) v4 -9168> 2014-08-13 16:24:07.098990 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.098990, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.2c) v4 -9167> 2014-08-13 16:24:07.099001 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.099001, event: started, request: pg_info(1 pgs e194449:24.2c) v4 -9166> 2014-08-13 16:24:07.099029 7fd93dff0700 5 -- op tracker -- , seq: 690, time: 2014-08-13 16:24:07.099029, event: done, request: pg_info(1 pgs e194449:24.2c) v4 -9165> 2014-08-13 16:24:07.099069 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 13 ==== pg_info(1 pgs e194449:22.4fe) v4 ==== 714+0+0 (3068091734 0 0) 0x775a4c0 con 0x508bde0 -9164> 2014-08-13 16:24:07.099089 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.098905, event: header_read, request: pg_info(1 pgs e194449:22.4fe) v4 -9163> 2014-08-13 16:24:07.099100 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.098908, event: throttled, request: pg_info(1 pgs e194449:22.4fe) v4 -9162> 2014-08-13 16:24:07.099109 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.098982, event: all_read, request: pg_info(1 pgs e194449:22.4fe) v4 -9161> 2014-08-13 16:24:07.099118 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.099087, event: dispatched, request: pg_info(1 pgs e194449:22.4fe) v4 -9160> 2014-08-13 16:24:07.099128 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.099128, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.4fe) v4 -9159> 2014-08-13 16:24:07.099139 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.099139, event: started, request: pg_info(1 pgs e194449:22.4fe) v4 -9158> 2014-08-13 16:24:07.099161 7fd93dff0700 5 -- op tracker -- , seq: 691, time: 2014-08-13 16:24:07.099161, event: done, request: pg_info(1 pgs e194449:22.4fe) v4 -9157> 2014-08-13 16:24:07.099337 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194449:24.2ec) v4 -- ?+0 0x81c1a40 -9156> 2014-08-13 16:24:07.099367 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 9 ==== pg_info(1 pgs e194449:22.3b9) v4 ==== 714+0+0 (1175407972 0 0) 0x7758540 con 0x5111fa0 -9155> 2014-08-13 16:24:07.099397 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099188, event: header_read, request: pg_info(1 pgs e194449:22.3b9) v4 -9154> 2014-08-13 16:24:07.099405 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099190, event: throttled, request: pg_info(1 pgs e194449:22.3b9) v4 -9153> 2014-08-13 16:24:07.099411 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099307, event: all_read, request: pg_info(1 pgs e194449:22.3b9) v4 -9152> 2014-08-13 16:24:07.099416 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099393, event: dispatched, request: pg_info(1 pgs e194449:22.3b9) v4 -9151> 2014-08-13 16:24:07.099422 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099422, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.3b9) v4 -9150> 2014-08-13 16:24:07.099430 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099430, event: started, request: pg_info(1 pgs e194449:22.3b9) v4 -9149> 2014-08-13 16:24:07.099457 7fd93dff0700 5 -- op tracker -- , seq: 692, time: 2014-08-13 16:24:07.099456, event: done, request: pg_info(1 pgs e194449:22.3b9) v4 -9148> 2014-08-13 16:24:07.099492 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 10 ==== pg_info(1 pgs e194449:22.3b9) v4 ==== 714+0+0 (2032259396 0 0) 0x7d07000 con 0x5111fa0 -9147> 2014-08-13 16:24:07.099508 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099333, event: header_read, request: pg_info(1 pgs e194449:22.3b9) v4 -9146> 2014-08-13 16:24:07.099516 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099334, event: throttled, request: pg_info(1 pgs e194449:22.3b9) v4 -9145> 2014-08-13 16:24:07.099526 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099434, event: all_read, request: pg_info(1 pgs e194449:22.3b9) v4 -9144> 2014-08-13 16:24:07.099532 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099506, event: dispatched, request: pg_info(1 pgs e194449:22.3b9) v4 -9143> 2014-08-13 16:24:07.099542 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099541, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.3b9) v4 -9142> 2014-08-13 16:24:07.099553 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099552, event: started, request: pg_info(1 pgs e194449:22.3b9) v4 -9141> 2014-08-13 16:24:07.099592 7fd93dff0700 5 -- op tracker -- , seq: 693, time: 2014-08-13 16:24:07.099592, event: done, request: pg_info(1 pgs e194449:22.3b9) v4 -9140> 2014-08-13 16:24:07.099612 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194449:22.1fb) v4 -- ?+0 0x81c4980 -9139> 2014-08-13 16:24:07.099638 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 14 ==== pg_info(1 pgs e194449:22.4fe) v4 ==== 714+0+0 (3560701349 0 0) 0x81c24c0 con 0x508bde0 -9138> 2014-08-13 16:24:07.099658 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099414, event: header_read, request: pg_info(1 pgs e194449:22.4fe) v4 -9137> 2014-08-13 16:24:07.099665 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099415, event: throttled, request: pg_info(1 pgs e194449:22.4fe) v4 -9136> 2014-08-13 16:24:07.099674 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099489, event: all_read, request: pg_info(1 pgs e194449:22.4fe) v4 -9135> 2014-08-13 16:24:07.099685 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099655, event: dispatched, request: pg_info(1 pgs e194449:22.4fe) v4 -9134> 2014-08-13 16:24:07.099699 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099698, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.4fe) v4 -9133> 2014-08-13 16:24:07.099707 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099706, event: started, request: pg_info(1 pgs e194449:22.4fe) v4 -9132> 2014-08-13 16:24:07.099725 7fd93dff0700 5 -- op tracker -- , seq: 694, time: 2014-08-13 16:24:07.099725, event: done, request: pg_info(1 pgs e194449:22.4fe) v4 -9131> 2014-08-13 16:24:07.099731 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194449:22.226) v4 -- ?+0 0x7d04280 -9130> 2014-08-13 16:24:07.100494 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 18 ==== pg_info(1 pgs e194449:22.1fb) v4 ==== 714+0+0 (1731226786 0 0) 0x81c4980 con 0x51135a0 -9129> 2014-08-13 16:24:07.100526 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100343, event: header_read, request: pg_info(1 pgs e194449:22.1fb) v4 -9128> 2014-08-13 16:24:07.100539 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100345, event: throttled, request: pg_info(1 pgs e194449:22.1fb) v4 -9127> 2014-08-13 16:24:07.100549 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100452, event: all_read, request: pg_info(1 pgs e194449:22.1fb) v4 -9126> 2014-08-13 16:24:07.100559 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100522, event: dispatched, request: pg_info(1 pgs e194449:22.1fb) v4 -9125> 2014-08-13 16:24:07.100569 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100569, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.1fb) v4 -9124> 2014-08-13 16:24:07.100587 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100587, event: started, request: pg_info(1 pgs e194449:22.1fb) v4 -9123> 2014-08-13 16:24:07.100617 7fd93dff0700 5 -- op tracker -- , seq: 695, time: 2014-08-13 16:24:07.100617, event: done, request: pg_info(1 pgs e194449:22.1fb) v4 -9122> 2014-08-13 16:24:07.100657 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.374 192.168.1.102:6828/16129 14 ==== pg_info(1 pgs e194449:24.2ec) v4 ==== 748+0+0 (2838233279 0 0) 0x8c8f380 con 0x51140a0 -9121> 2014-08-13 16:24:07.100678 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100464, event: header_read, request: pg_info(1 pgs e194449:24.2ec) v4 -9120> 2014-08-13 16:24:07.100689 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100466, event: throttled, request: pg_info(1 pgs e194449:24.2ec) v4 -9119> 2014-08-13 16:24:07.100696 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100545, event: all_read, request: pg_info(1 pgs e194449:24.2ec) v4 -9118> 2014-08-13 16:24:07.100704 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100675, event: dispatched, request: pg_info(1 pgs e194449:24.2ec) v4 -9117> 2014-08-13 16:24:07.100714 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100714, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.2ec) v4 -9116> 2014-08-13 16:24:07.100725 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100724, event: started, request: pg_info(1 pgs e194449:24.2ec) v4 -9115> 2014-08-13 16:24:07.100751 7fd93dff0700 5 -- op tracker -- , seq: 696, time: 2014-08-13 16:24:07.100751, event: done, request: pg_info(1 pgs e194449:24.2ec) v4 -9114> 2014-08-13 16:24:07.100788 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 17 ==== pg_info(1 pgs e194449:22.226) v4 ==== 714+0+0 (4115188846 0 0) 0x7d04280 con 0x5111a20 -9113> 2014-08-13 16:24:07.100808 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100575, event: header_read, request: pg_info(1 pgs e194449:22.226) v4 -9112> 2014-08-13 16:24:07.100818 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100577, event: throttled, request: pg_info(1 pgs e194449:22.226) v4 -9111> 2014-08-13 16:24:07.100827 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100661, event: all_read, request: pg_info(1 pgs e194449:22.226) v4 -9110> 2014-08-13 16:24:07.100836 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100805, event: dispatched, request: pg_info(1 pgs e194449:22.226) v4 -9109> 2014-08-13 16:24:07.100846 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100846, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.226) v4 -9108> 2014-08-13 16:24:07.100857 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100857, event: started, request: pg_info(1 pgs e194449:22.226) v4 -9107> 2014-08-13 16:24:07.100888 7fd93dff0700 5 -- op tracker -- , seq: 697, time: 2014-08-13 16:24:07.100888, event: done, request: pg_info(1 pgs e194449:22.226) v4 -9106> 2014-08-13 16:24:07.101228 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 19 ==== pg_info(1 pgs e194449:22.1fb) v4 ==== 714+0+0 (3303721106 0 0) 0x80df1c0 con 0x51135a0 -9105> 2014-08-13 16:24:07.101251 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101100, event: header_read, request: pg_info(1 pgs e194449:22.1fb) v4 -9104> 2014-08-13 16:24:07.101262 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101101, event: throttled, request: pg_info(1 pgs e194449:22.1fb) v4 -9103> 2014-08-13 16:24:07.101271 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101193, event: all_read, request: pg_info(1 pgs e194449:22.1fb) v4 -9102> 2014-08-13 16:24:07.101280 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101248, event: dispatched, request: pg_info(1 pgs e194449:22.1fb) v4 -9101> 2014-08-13 16:24:07.101290 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101289, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.1fb) v4 -9100> 2014-08-13 16:24:07.101300 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101300, event: started, request: pg_info(1 pgs e194449:22.1fb) v4 -9099> 2014-08-13 16:24:07.101327 7fd93dff0700 5 -- op tracker -- , seq: 698, time: 2014-08-13 16:24:07.101327, event: done, request: pg_info(1 pgs e194449:22.1fb) v4 -9098> 2014-08-13 16:24:07.101692 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 18 ==== MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 ==== 26+0+0 (2627877103 0 0) 0x81c6ac0 con 0x5111a20 -9097> 2014-08-13 16:24:07.101715 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101585, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9096> 2014-08-13 16:24:07.101726 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101587, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9095> 2014-08-13 16:24:07.101735 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101655, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9094> 2014-08-13 16:24:07.101743 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101712, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9093> 2014-08-13 16:24:07.101752 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101752, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9092> 2014-08-13 16:24:07.101774 7fd93dff0700 5 -- op tracker -- , seq: 699, time: 2014-08-13 16:24:07.101774, event: done, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194449 v2 -9091> 2014-08-13 16:24:07.101792 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194448 pi=192975-194447/17 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.597954 4 0.000261 -9090> 2014-08-13 16:24:07.101835 7fd9371e5700 5 osd.4 pg_epoch: 194449 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194448 pi=192975-194447/17 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -9089> 2014-08-13 16:24:07.102875 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194449:24.1e4) v4 -- ?+0 0x81c5cc0 -9088> 2014-08-13 16:24:07.103822 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.31 192.168.1.103:6831/41157 13 ==== pg_info(1 pgs e194449:24.1e4) v4 ==== 748+0+0 (369154831 0 0) 0x7d078c0 con 0x5089760 -9087> 2014-08-13 16:24:07.103845 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103708, event: header_read, request: pg_info(1 pgs e194449:24.1e4) v4 -9086> 2014-08-13 16:24:07.103853 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103710, event: throttled, request: pg_info(1 pgs e194449:24.1e4) v4 -9085> 2014-08-13 16:24:07.103858 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103787, event: all_read, request: pg_info(1 pgs e194449:24.1e4) v4 -9084> 2014-08-13 16:24:07.103863 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103842, event: dispatched, request: pg_info(1 pgs e194449:24.1e4) v4 -9083> 2014-08-13 16:24:07.103869 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103869, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:24.1e4) v4 -9082> 2014-08-13 16:24:07.103877 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103877, event: started, request: pg_info(1 pgs e194449:24.1e4) v4 -9081> 2014-08-13 16:24:07.103901 7fd93dff0700 5 -- op tracker -- , seq: 700, time: 2014-08-13 16:24:07.103901, event: done, request: pg_info(1 pgs e194449:24.1e4) v4 -9080> 2014-08-13 16:24:07.124801 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.124669) v2 ==== 47+0+0 (838167227 0 0) 0x8c8c7c0 con 0x7eda100 -9079> 2014-08-13 16:24:07.124824 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.124669) v2 ==== 47+0+0 (838167227 0 0) 0x4f8b2c0 con 0x7edc0a0 -9078> 2014-08-13 16:24:07.124849 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.124669) v2 -- ?+0 0x8c896c0 con 0x7eda100 -9077> 2014-08-13 16:24:07.124951 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.124669) v2 -- ?+0 0x8c8c600 con 0x7edc0a0 -9076> 2014-08-13 16:24:07.231949 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.231592) v2 ==== 47+0+0 (1106640913 0 0) 0x7d06e40 con 0x7caeb40 -9075> 2014-08-13 16:24:07.231985 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.231592) v2 ==== 47+0+0 (1106640913 0 0) 0x7d04980 con 0x7cae300 -9074> 2014-08-13 16:24:07.231997 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.231592) v2 -- ?+0 0x8c8c7c0 con 0x7caeb40 -9073> 2014-08-13 16:24:07.232128 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.231592) v2 -- ?+0 0x4f8b2c0 con 0x7cae300 -9072> 2014-08-13 16:24:07.393859 7fd9208e2700 2 -- 192.168.1.115:6813/25220 >> 192.168.1.110:6804/25590 pipe(0x8e88a00 sd=227 :43002 s=2 pgs=126 cs=1 l=0 c=0x78b0580).reader couldn't read tag, (0) Success -9071> 2014-08-13 16:24:07.393918 7fd9208e2700 2 -- 192.168.1.115:6813/25220 >> 192.168.1.110:6804/25590 pipe(0x8e88a00 sd=227 :43002 s=2 pgs=126 cs=1 l=0 c=0x78b0580).fault (0) Success -9070> 2014-08-13 16:24:07.393935 7fd9208e2700 0 -- 192.168.1.115:6813/25220 >> 192.168.1.110:6804/25590 pipe(0x8e88a00 sd=227 :43002 s=2 pgs=126 cs=1 l=0 c=0x78b0580).fault with nothing to send, going to standby -9069> 2014-08-13 16:24:07.393906 7fd917953700 2 -- 192.168.100.115:6806/25220 >> 192.168.100.110:0/25590 pipe(0x7bc7300 sd=302 :6806 s=2 pgs=263 cs=1 l=1 c=0x7d2c0a0).reader couldn't read tag, (0) Success -9068> 2014-08-13 16:24:07.393963 7fd917953700 2 -- 192.168.100.115:6806/25220 >> 192.168.100.110:0/25590 pipe(0x7bc7300 sd=302 :6806 s=2 pgs=263 cs=1 l=1 c=0x7d2c0a0).fault (0) Success -9067> 2014-08-13 16:24:07.395622 7fd91754f700 2 -- 192.168.1.115:6817/25220 >> 192.168.100.110:0/25590 pipe(0x7bc5f00 sd=303 :6817 s=2 pgs=264 cs=1 l=1 c=0x7d2e720).reader couldn't read tag, (0) Success -9066> 2014-08-13 16:24:07.395676 7fd91754f700 2 -- 192.168.1.115:6817/25220 >> 192.168.100.110:0/25590 pipe(0x7bc5f00 sd=303 :6817 s=2 pgs=264 cs=1 l=1 c=0x7d2e720).fault (0) Success -9065> 2014-08-13 16:24:07.424190 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.423950) v2 ==== 47+0+0 (351653690 0 0) 0x80dd780 con 0x7ffeca0 -9064> 2014-08-13 16:24:07.424212 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.423950) v2 ==== 47+0+0 (351653690 0 0) 0x80dc440 con 0x7edbf40 -9063> 2014-08-13 16:24:07.424242 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.423950) v2 -- ?+0 0x7d06e40 con 0x7ffeca0 -9062> 2014-08-13 16:24:07.424291 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.423950) v2 -- ?+0 0x7d04980 con 0x7edbf40 -9061> 2014-08-13 16:24:07.542243 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 9 ==== osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3570962850 0 0) 0x7ecac00 con 0x4fdd3e0 -9060> 2014-08-13 16:24:07.542303 7fd93dff0700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.541961, event: header_read, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9059> 2014-08-13 16:24:07.542319 7fd93dff0700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.541967, event: throttled, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9058> 2014-08-13 16:24:07.542330 7fd93dff0700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542127, event: all_read, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9057> 2014-08-13 16:24:07.542339 7fd93dff0700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542295, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9056> 2014-08-13 16:24:07.542355 7fd93dff0700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542355, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9055> 2014-08-13 16:24:07.542475 7fd9367e4700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542475, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9054> 2014-08-13 16:24:07.542523 7fd9367e4700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542522, event: started, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9053> 2014-08-13 16:24:07.542545 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.106:6816/8118 -- osd_sub_op_reply(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003480 con 0x4fdd3e0 -9052> 2014-08-13 16:24:07.542584 7fd9367e4700 5 -- op tracker -- , seq: 701, time: 2014-08-13 16:24:07.542584, event: done, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9051> 2014-08-13 16:24:07.542926 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 13 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x7ec8b00 con 0x5112520 -9050> 2014-08-13 16:24:07.542978 7fd93dff0700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.542703, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9049> 2014-08-13 16:24:07.542995 7fd93dff0700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.542708, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9048> 2014-08-13 16:24:07.543005 7fd93dff0700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.542827, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9047> 2014-08-13 16:24:07.543015 7fd93dff0700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.542972, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9046> 2014-08-13 16:24:07.543031 7fd93dff0700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.543031, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9045> 2014-08-13 16:24:07.543136 7fd9371e5700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.543135, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9044> 2014-08-13 16:24:07.543192 7fd9371e5700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.543192, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9043> 2014-08-13 16:24:07.543216 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8e180 con 0x5112520 -9042> 2014-08-13 16:24:07.543249 7fd9371e5700 5 -- op tracker -- , seq: 702, time: 2014-08-13 16:24:07.543248, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9041> 2014-08-13 16:24:07.544493 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 14 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x7ec9600 con 0x5112520 -9040> 2014-08-13 16:24:07.544543 7fd93dff0700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544280, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9039> 2014-08-13 16:24:07.544560 7fd93dff0700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544283, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9038> 2014-08-13 16:24:07.544570 7fd93dff0700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544413, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9037> 2014-08-13 16:24:07.544579 7fd93dff0700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544537, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9036> 2014-08-13 16:24:07.544601 7fd93dff0700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544601, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9035> 2014-08-13 16:24:07.544700 7fd9367e4700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544699, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9034> 2014-08-13 16:24:07.544746 7fd9367e4700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544745, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9033> 2014-08-13 16:24:07.544762 7fd9367e4700 5 -- op tracker -- , seq: 703, time: 2014-08-13 16:24:07.544762, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -9032> 2014-08-13 16:24:07.549999 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 10 ==== replica scrub(pg: 22.12a,from:0'0,to:194215'491,epoch:194449,start:0//0//-1,end:825e692a//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2966439974 0 0) 0x8bd0480 con 0x4fdd3e0 -9031> 2014-08-13 16:24:07.553246 7fd933fe0700 5 osd.4 194449 heartbeat: osd_stat(8099 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,7,10,11,15,17,20,21,24,25,28,31,32,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,396,397]/[] op hist []) -9030> 2014-08-13 16:24:07.553294 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x80da4c0 con 0x8c391e0 -9029> 2014-08-13 16:24:07.553336 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x80d9500 con 0x8c38f20 -9028> 2014-08-13 16:24:07.553358 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x80d8a80 con 0x78844c0 -9027> 2014-08-13 16:24:07.553381 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7d063c0 con 0x7884200 -9026> 2014-08-13 16:24:07.553399 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7d039c0 con 0x78848e0 -9025> 2014-08-13 16:24:07.553421 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x80dbf00 con 0x7884620 -9024> 2014-08-13 16:24:07.553440 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6839/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c0700 con 0x8ba06e0 -9023> 2014-08-13 16:24:07.553459 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6846/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c0380 con 0x8c698c0 -9022> 2014-08-13 16:24:07.553475 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7d06ac0 con 0x794c360 -9021> 2014-08-13 16:24:07.553495 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7d00a80 con 0x794e300 -9020> 2014-08-13 16:24:07.553518 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c1a40 con 0x794e040 -9019> 2014-08-13 16:24:07.553535 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c2140 con 0x794dd80 -9018> 2014-08-13 16:24:07.553554 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c7700 con 0x8c3a3c0 -9017> 2014-08-13 16:24:07.553575 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c5cc0 con 0x8c3a100 -9016> 2014-08-13 16:24:07.553594 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c3800 con 0x8c3c0a0 -9015> 2014-08-13 16:24:07.553619 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c5240 con 0x8c3bde0 -9014> 2014-08-13 16:24:07.553646 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c7380 con 0x8babde0 -9013> 2014-08-13 16:24:07.553664 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c1180 con 0x8babb20 -9012> 2014-08-13 16:24:07.553681 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c5b00 con 0x8c3d6a0 -9011> 2014-08-13 16:24:07.553697 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c2f40 con 0x8c3d3e0 -9010> 2014-08-13 16:24:07.553719 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c2d80 con 0x8badac0 -9009> 2014-08-13 16:24:07.553742 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7758a80 con 0x8bad800 -9008> 2014-08-13 16:24:07.553760 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c4b40 con 0x8bab860 -9007> 2014-08-13 16:24:07.553777 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c1dc0 con 0x8bab5a0 -9006> 2014-08-13 16:24:07.553793 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c5400 con 0x8c69600 -9005> 2014-08-13 16:24:07.553814 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c4600 con 0x8c69340 -9004> 2014-08-13 16:24:07.553836 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c3f00 con 0x8baee00 -9003> 2014-08-13 16:24:07.553855 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x81c3100 con 0x8ba1a20 -9002> 2014-08-13 16:24:07.553872 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6814/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7758e00 con 0x8d21e40 -9001> 2014-08-13 16:24:07.553891 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6831/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775e040 con 0x8d21b80 -9000> 2014-08-13 16:24:07.553914 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775db00 con 0x7fba3c0 -8999> 2014-08-13 16:24:07.553931 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7759c00 con 0x7fba260 -8998> 2014-08-13 16:24:07.553948 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775bd40 con 0x8bad540 -8997> 2014-08-13 16:24:07.553964 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775d5c0 con 0x8bad280 -8996> 2014-08-13 16:24:07.553982 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7759180 con 0x8c3cba0 -8995> 2014-08-13 16:24:07.554001 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775ee40 con 0x8c3c8e0 -8994> 2014-08-13 16:24:07.554022 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7758380 con 0x7fba100 -8993> 2014-08-13 16:24:07.554046 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775c280 con 0x7fb9fa0 -8992> 2014-08-13 16:24:07.553988 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775f540 con 0x8c391e0 -8991> 2014-08-13 16:24:07.554063 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x77588c0 con 0x8d223c0 -8990> 2014-08-13 16:24:07.554080 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775e3c0 con 0x8d22100 -8989> 2014-08-13 16:24:07.554098 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775d400 con 0x8c3c620 -8988> 2014-08-13 16:24:07.554115 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775a840 con 0x8c3c360 -8987> 2014-08-13 16:24:07.554132 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x7759dc0 con 0x8bafe80 -8986> 2014-08-13 16:24:07.554150 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775d780 con 0x8bafbc0 -8985> 2014-08-13 16:24:07.554179 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x775c0c0 con 0x8c38c60 -8984> 2014-08-13 16:24:07.554200 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee88c0 con 0x8c389a0 -8983> 2014-08-13 16:24:07.554219 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee8000 con 0x8b68c60 -8982> 2014-08-13 16:24:07.554241 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee8c40 con 0x8b68b00 -8981> 2014-08-13 16:24:07.554257 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eea140 con 0x8c6fe80 -8980> 2014-08-13 16:24:07.554275 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeaa00 con 0x8c6fbc0 -8979> 2014-08-13 16:24:07.554292 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeabc0 con 0x8b689a0 -8978> 2014-08-13 16:24:07.554309 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9f80 con 0x8b68840 -8977> 2014-08-13 16:24:07.554326 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eea680 con 0x7fb9e40 -8976> 2014-08-13 16:24:07.554343 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9dc0 con 0x7fb9ce0 -8975> 2014-08-13 16:24:07.554361 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eea4c0 con 0x8c3b9c0 -8974> 2014-08-13 16:24:07.554383 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeb480 con 0x8c3b700 -8973> 2014-08-13 16:24:07.554402 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeb2c0 con 0x7fbb020 -8972> 2014-08-13 16:24:07.554418 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeb100 con 0x7fbaec0 -8971> 2014-08-13 16:24:07.554435 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeaf40 con 0x7fbb440 -8970> 2014-08-13 16:24:07.554452 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eead80 con 0x7fbb2e0 -8969> 2014-08-13 16:24:07.554469 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9880 con 0x7ffd960 -8968> 2014-08-13 16:24:07.554486 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9a40 con 0x7ffdc20 -8967> 2014-08-13 16:24:07.554508 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee8e00 con 0x8d20840 -8966> 2014-08-13 16:24:07.554528 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9180 con 0x8d25960 -8965> 2014-08-13 16:24:07.554544 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee8fc0 con 0x8d218c0 -8964> 2014-08-13 16:24:07.554562 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8ee9c00 con 0x8d21600 -8963> 2014-08-13 16:24:07.554582 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eed240 con 0x7fbaaa0 -8962> 2014-08-13 16:24:07.554601 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eed080 con 0x7fba940 -8961> 2014-08-13 16:24:07.554620 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eecec0 con 0x8d26e00 -8960> 2014-08-13 16:24:07.554637 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eecd00 con 0x8d26b40 -8959> 2014-08-13 16:24:07.554654 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eecb40 con 0x8d21340 -8958> 2014-08-13 16:24:07.554671 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec980 con 0x8d21080 -8957> 2014-08-13 16:24:07.554688 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec7c0 con 0x7ffdee0 -8956> 2014-08-13 16:24:07.554706 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec600 con 0x7ffdd80 -8955> 2014-08-13 16:24:07.554723 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec440 con 0x7fb9a20 -8954> 2014-08-13 16:24:07.554741 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec280 con 0x7fb98c0 -8953> 2014-08-13 16:24:07.554759 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eec0c0 con 0x8bab2e0 -8952> 2014-08-13 16:24:07.554777 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eebf00 con 0x8bab020 -8951> 2014-08-13 16:24:07.554796 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eef380 con 0x8c69080 -8950> 2014-08-13 16:24:07.554817 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eef1c0 con 0x8c68dc0 -8949> 2014-08-13 16:24:07.554836 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eef000 con 0x8d256a0 -8948> 2014-08-13 16:24:07.554855 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeee40 con 0x8d253e0 -8947> 2014-08-13 16:24:07.554872 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeec80 con 0x8d20dc0 -8946> 2014-08-13 16:24:07.554890 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eeeac0 con 0x8d20b00 -8945> 2014-08-13 16:24:07.554906 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee900 con 0x8d25120 -8944> 2014-08-13 16:24:07.554924 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee740 con 0x8d24e60 -8943> 2014-08-13 16:24:07.554940 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee580 con 0x8c3d120 -8942> 2014-08-13 16:24:07.554958 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee3c0 con 0x8c3ce60 -8941> 2014-08-13 16:24:07.554976 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee200 con 0x7fbb180 -8940> 2014-08-13 16:24:07.555000 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eee040 con 0x7fbca40 -8939> 2014-08-13 16:24:07.555020 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8eede80 con 0x7fbc780 -8938> 2014-08-13 16:24:07.555923 7fd91754f700 1 -- 192.168.1.115:6813/25220 >> :/0 pipe(0x8d46400 sd=24 :6813 s=0 pgs=0 cs=0 l=0 c=0x8300420).accept sd=24 192.168.1.107:45589/0 -8937> 2014-08-13 16:24:07.556114 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x92816c0 con 0x7fbc4c0 -8936> 2014-08-13 16:24:07.556152 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9281500 con 0x8c3aec0 -8935> 2014-08-13 16:24:07.556191 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9281340 con 0x8c3ac00 -8934> 2014-08-13 16:24:07.556213 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9281180 con 0x7fbad60 -8933> 2014-08-13 16:24:07.556234 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280fc0 con 0x7fbac00 -8932> 2014-08-13 16:24:07.556255 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280e00 con 0x8c3e880 -8931> 2014-08-13 16:24:07.556276 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280c40 con 0x8c3e5c0 -8930> 2014-08-13 16:24:07.556296 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280a80 con 0x8c3a940 -8929> 2014-08-13 16:24:07.556316 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x92808c0 con 0x8c3a680 -8928> 2014-08-13 16:24:07.556335 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280700 con 0x8c69fa0 -8927> 2014-08-13 16:24:07.556355 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280540 con 0x8c39fa0 -8926> 2014-08-13 16:24:07.556376 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280380 con 0x7fb9760 -8925> 2014-08-13 16:24:07.556396 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x92801c0 con 0x7fb9600 -8924> 2014-08-13 16:24:07.556419 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9280000 con 0x8c3e300 -8923> 2014-08-13 16:24:07.556441 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101f80 con 0x8c618c0 -8922> 2014-08-13 16:24:07.556464 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101dc0 con 0x8bac780 -8921> 2014-08-13 16:24:07.556488 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101c00 con 0x8bac4c0 -8920> 2014-08-13 16:24:07.556508 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101a40 con 0x8bac200 -8919> 2014-08-13 16:24:07.556525 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101880 con 0x8babf40 -8918> 2014-08-13 16:24:07.556542 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91016c0 con 0x8d240a0 -8917> 2014-08-13 16:24:07.556559 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101500 con 0x8d23de0 -8916> 2014-08-13 16:24:07.556575 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101340 con 0x8ba1760 -8915> 2014-08-13 16:24:07.556592 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9101180 con 0x8ba14a0 -8914> 2014-08-13 16:24:07.556614 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9100fc0 con 0x8d26880 -8913> 2014-08-13 16:24:07.556635 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9100e00 con 0x8d265c0 -8912> 2014-08-13 16:24:07.556658 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9100c40 con 0x8baf900 -8911> 2014-08-13 16:24:07.556677 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9100a80 con 0x8baf640 -8910> 2014-08-13 16:24:07.556701 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91008c0 con 0x8bae040 -8909> 2014-08-13 16:24:07.556722 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9100700 con 0x8badd80 -8908> 2014-08-13 16:24:07.556746 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91047c0 con 0x8d24620 -8907> 2014-08-13 16:24:07.556770 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9104600 con 0x8d24360 -8906> 2014-08-13 16:24:07.556787 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9104440 con 0x8baf380 -8905> 2014-08-13 16:24:07.556806 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9104280 con 0x8baf0c0 -8904> 2014-08-13 16:24:07.556824 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91040c0 con 0x8b6e1a0 -8903> 2014-08-13 16:24:07.556841 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103f00 con 0x8b6e040 -8902> 2014-08-13 16:24:07.556857 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103d40 con 0x8c39ce0 -8901> 2014-08-13 16:24:07.556879 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103b80 con 0x8c39a20 -8900> 2014-08-13 16:24:07.556899 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91039c0 con 0x8d27380 -8899> 2014-08-13 16:24:07.556920 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103800 con 0x8d270c0 -8898> 2014-08-13 16:24:07.556941 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103640 con 0x8d23b20 -8897> 2014-08-13 16:24:07.556962 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103480 con 0x8d27bc0 -8896> 2014-08-13 16:24:07.556979 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91032c0 con 0x8b6dee0 -8895> 2014-08-13 16:24:07.556995 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9103100 con 0x8b6dd80 -8894> 2014-08-13 16:24:07.557017 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9102f40 con 0x7fba7e0 -8893> 2014-08-13 16:24:07.557036 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9102d80 con 0x7fba680 -8892> 2014-08-13 16:24:07.557058 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9107380 con 0x8c6f900 -8891> 2014-08-13 16:24:07.557083 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91071c0 con 0x8c6f640 -8890> 2014-08-13 16:24:07.557101 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9107000 con 0x8c68b00 -8889> 2014-08-13 16:24:07.557123 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106e40 con 0x8c68840 -8888> 2014-08-13 16:24:07.557141 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106c80 con 0x8c3b440 -8887> 2014-08-13 16:24:07.557165 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106ac0 con 0x8c3b180 -8886> 2014-08-13 16:24:07.557189 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106900 con 0x8c68580 -8885> 2014-08-13 16:24:07.557207 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106740 con 0x8c6b2e0 -8884> 2014-08-13 16:24:07.557226 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106580 con 0x8c6b020 -8883> 2014-08-13 16:24:07.557243 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x91063c0 con 0x8c6ad60 -8882> 2014-08-13 16:24:07.557263 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106200 con 0x8b6dc20 -8881> 2014-08-13 16:24:07.557284 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9106040 con 0x7fb9b80 -8880> 2014-08-13 16:24:07.557302 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9105e80 con 0x8ba11e0 -8879> 2014-08-13 16:24:07.557325 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9105cc0 con 0x8ba0f20 -8878> 2014-08-13 16:24:07.557345 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9105b00 con 0x7fbbc80 -8877> 2014-08-13 16:24:07.557367 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9105940 con 0x7fbbb20 -8876> 2014-08-13 16:24:07.557390 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x9105780 con 0x8c39760 -8875> 2014-08-13 16:24:07.557413 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899a680 con 0x8c394a0 -8874> 2014-08-13 16:24:07.557438 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899a4c0 con 0x7fbb9c0 -8873> 2014-08-13 16:24:07.557459 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899a300 con 0x7fbb860 -8872> 2014-08-13 16:24:07.557480 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899a140 con 0x7fb94a0 -8871> 2014-08-13 16:24:07.557500 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999f80 con 0x7fba520 -8870> 2014-08-13 16:24:07.557522 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999dc0 con 0x7fbc200 -8869> 2014-08-13 16:24:07.557540 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999c00 con 0x7fbbf40 -8868> 2014-08-13 16:24:07.557561 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999a40 con 0x8c6aaa0 -8867> 2014-08-13 16:24:07.557581 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999880 con 0x8c6a7e0 -8866> 2014-08-13 16:24:07.557601 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x89996c0 con 0x8bacd00 -8865> 2014-08-13 16:24:07.557621 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999500 con 0x8baca40 -8864> 2014-08-13 16:24:07.557641 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999340 con 0x8ba0c60 -8863> 2014-08-13 16:24:07.557662 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8999180 con 0x8ba09a0 -8862> 2014-08-13 16:24:07.557682 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8998fc0 con 0x7ffdac0 -8861> 2014-08-13 16:24:07.557702 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8998e00 con 0x7ffe040 -8860> 2014-08-13 16:24:07.557721 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8998c40 con 0x8c6f380 -8859> 2014-08-13 16:24:07.557742 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x8998a80 con 0x8c6f0c0 -8858> 2014-08-13 16:24:07.557763 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x89988c0 con 0x7fbb700 -8857> 2014-08-13 16:24:07.557786 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899d240 con 0x7fbb5a0 -8856> 2014-08-13 16:24:07.557806 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899d080 con 0x8d27900 -8855> 2014-08-13 16:24:07.557826 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899cec0 con 0x8d27640 -8854> 2014-08-13 16:24:07.557847 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899cd00 con 0x8c6a520 -8853> 2014-08-13 16:24:07.557869 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899cb40 con 0x8c6a260 -8852> 2014-08-13 16:24:07.557886 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c980 con 0x8d24ba0 -8851> 2014-08-13 16:24:07.557903 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c7c0 con 0x8d248e0 -8850> 2014-08-13 16:24:07.557922 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c600 con 0x8d26300 -8849> 2014-08-13 16:24:07.557942 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c440 con 0x8d26040 -8848> 2014-08-13 16:24:07.557958 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c280 con 0x8d25d80 -8847> 2014-08-13 16:24:07.557976 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899c0c0 con 0x8d140a0 -8846> 2014-08-13 16:24:07.557997 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899bf00 con 0x8d13de0 -8845> 2014-08-13 16:24:07.558017 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:07.553291) v2 -- ?+0 0x899bd40 con 0x8d13b20 -8844> 2014-08-13 16:24:07.558112 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775b480 con 0x794e300 -8843> 2014-08-13 16:24:07.558180 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefc40 con 0x794e040 -8842> 2014-08-13 16:24:07.558252 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130c40 con 0x7ffd960 -8841> 2014-08-13 16:24:07.558291 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x89981c0 con 0x7ffdc20 -8840> 2014-08-13 16:24:07.558320 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102a00 con 0x7ffdd80 -8839> 2014-08-13 16:24:07.558348 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91001c0 con 0x8b68c60 -8838> 2014-08-13 16:24:07.558373 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedb00 con 0x8b68840 -8837> 2014-08-13 16:24:07.558399 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107e00 con 0x8b6e040 -8836> 2014-08-13 16:24:07.558426 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ad80 con 0x8b6dee0 -8835> 2014-08-13 16:24:07.558453 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899aa00 con 0x8b6dd80 -8834> 2014-08-13 16:24:07.558481 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d5c0 con 0x8b6dc20 -8833> 2014-08-13 16:24:07.558508 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102840 con 0x7fb9a20 -8832> 2014-08-13 16:24:07.558535 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91301c0 con 0x7fb98c0 -8831> 2014-08-13 16:24:07.558561 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105240 con 0x7fb9760 -8830> 2014-08-13 16:24:07.558599 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e3c0 con 0x7fba520 -8829> 2014-08-13 16:24:07.558635 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131500 con 0x7fba3c0 -8828> 2014-08-13 16:24:07.558663 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eebd40 con 0x7fba260 -8827> 2014-08-13 16:24:07.558701 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e580 con 0x7fba100 -8826> 2014-08-13 16:24:07.558732 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131c00 con 0x7fb9fa0 -8825> 2014-08-13 16:24:07.558757 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131880 con 0x7fb9e40 -8824> 2014-08-13 16:24:07.558786 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9132140 con 0x7fb9ce0 -8823> 2014-08-13 16:24:07.558813 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x92481c0 con 0x7fbaec0 -8822> 2014-08-13 16:24:07.558837 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e200 con 0x7fbad60 -8821> 2014-08-13 16:24:07.558861 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d780 con 0x7fbac00 -8820> 2014-08-13 16:24:07.558886 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9281880 con 0x7fba940 -8819> 2014-08-13 16:24:07.558918 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899af40 con 0x7fba7e0 -8818> 2014-08-13 16:24:07.558952 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9132300 con 0x7fbb440 -8817> 2014-08-13 16:24:07.558979 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130380 con 0x7fbb2e0 -8816> 2014-08-13 16:24:07.559005 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104980 con 0x7fbb180 -8815> 2014-08-13 16:24:07.559032 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105080 con 0x7fbc780 -8814> 2014-08-13 16:24:07.559056 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899b100 con 0x7fbc4c0 -8813> 2014-08-13 16:24:07.559082 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899dcc0 con 0x7fbc200 -8812> 2014-08-13 16:24:07.559109 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed780 con 0x8babde0 -8811> 2014-08-13 16:24:07.559134 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef8c0 con 0x8babb20 -8810> 2014-08-13 16:24:07.559163 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee9340 con 0x8bab860 -8809> 2014-08-13 16:24:07.559199 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e3c0 con 0x8bab5a0 -8808> 2014-08-13 16:24:07.559226 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131f80 con 0x8bab2e0 -8807> 2014-08-13 16:24:07.559247 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130700 con 0x8bab020 -8806> 2014-08-13 16:24:07.559267 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107e00 con 0x8bac200 -8805> 2014-08-13 16:24:07.559293 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107700 con 0x8babf40 -8804> 2014-08-13 16:24:07.559330 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998540 con 0x8bae040 -8803> 2014-08-13 16:24:07.559362 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedb00 con 0x8badd80 -8802> 2014-08-13 16:24:07.559391 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9248000 con 0x8badac0 -8801> 2014-08-13 16:24:07.559423 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998000 con 0x8bad800 -8800> 2014-08-13 16:24:07.559454 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedcc0 con 0x8bad540 -8799> 2014-08-13 16:24:07.559482 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefa80 con 0x8bad280 -8798> 2014-08-13 16:24:07.559508 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed940 con 0x8bafe80 -8797> 2014-08-13 16:24:07.559534 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102bc0 con 0x8bafbc0 -8796> 2014-08-13 16:24:07.559558 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105080 con 0x8baf640 -8795> 2014-08-13 16:24:07.559584 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107700 con 0x8baf0c0 -8794> 2014-08-13 16:24:07.559623 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105240 con 0x8baee00 -8793> 2014-08-13 16:24:07.559662 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131180 con 0x8ba1a20 -8792> 2014-08-13 16:24:07.559689 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91078c0 con 0x8ba14a0 -8791> 2014-08-13 16:24:07.559721 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.1.115:6839/62713 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8a80 con 0x8ba06e0 -8790> 2014-08-13 16:24:07.559749 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.100.115:6846/62713 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9100000 con 0x8c698c0 -8789> 2014-08-13 16:24:07.559777 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eeb640 con 0x8c69600 -8788> 2014-08-13 16:24:07.559802 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9208000 con 0x8c69340 -8787> 2014-08-13 16:24:07.559828 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102140 con 0x8c69080 -8786> 2014-08-13 16:24:07.559852 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102680 con 0x8c68dc0 -8785> 2014-08-13 16:24:07.559879 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ad80 con 0x8c68b00 -8784> 2014-08-13 16:24:07.559905 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ec80 con 0x8c68840 -8783> 2014-08-13 16:24:07.559932 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775f540 con 0x8c68580 -8782> 2014-08-13 16:24:07.559959 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e040 con 0x8c6b2e0 -8781> 2014-08-13 16:24:07.559983 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedcc0 con 0x8c6b020 -8780> 2014-08-13 16:24:07.560014 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d400 con 0x8c6ad60 -8779> 2014-08-13 16:24:07.560043 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899de80 con 0x8c69fa0 -8778> 2014-08-13 16:24:07.560071 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899a840 con 0x8c39ce0 -8777> 2014-08-13 16:24:07.560098 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eebd40 con 0x8c39a20 -8776> 2014-08-13 16:24:07.560123 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105240 con 0x8c39760 -8775> 2014-08-13 16:24:07.560152 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8380 con 0x8c38f20 -8774> 2014-08-13 16:24:07.560186 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899eac0 con 0x8c38c60 -8773> 2014-08-13 16:24:07.560213 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91316c0 con 0x8c3b9c0 -8772> 2014-08-13 16:24:07.560238 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef540 con 0x8c3b700 -8771> 2014-08-13 16:24:07.560261 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998540 con 0x8c3b440 -8770> 2014-08-13 16:24:07.560286 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ee40 con 0x8c3b180 -8769> 2014-08-13 16:24:07.560311 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899db00 con 0x8c3aec0 -8768> 2014-08-13 16:24:07.560349 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104ec0 con 0x8c3ac00 -8767> 2014-08-13 16:24:07.560377 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899af40 con 0x8c3a940 -8766> 2014-08-13 16:24:07.560406 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105400 con 0x8c3a680 -8765> 2014-08-13 16:24:07.560430 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x88c81c0 con 0x8c3a3c0 -8764> 2014-08-13 16:24:07.560455 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefe00 con 0x8c3a100 -8763> 2014-08-13 16:24:07.560474 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131340 con 0x8c3d6a0 -8762> 2014-08-13 16:24:07.560494 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eea300 con 0x8c3d3e0 -8761> 2014-08-13 16:24:07.560513 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130e00 con 0x8c3d120 -8760> 2014-08-13 16:24:07.560535 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998700 con 0x8c3ce60 -8759> 2014-08-13 16:24:07.560555 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed400 con 0x8c3cba0 -8758> 2014-08-13 16:24:07.560680 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eeb9c0 con 0x8c3c8e0 -8757> 2014-08-13 16:24:07.560717 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed5c0 con 0x8c3c620 -8756> 2014-08-13 16:24:07.560745 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131a40 con 0x8c3c360 -8755> 2014-08-13 16:24:07.560773 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee9500 con 0x8c3c0a0 -8754> 2014-08-13 16:24:07.560808 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eebb80 con 0x8c3bde0 -8753> 2014-08-13 16:24:07.560841 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed940 con 0x8c3e880 -8752> 2014-08-13 16:24:07.560871 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104d00 con 0x8c3e5c0 -8751> 2014-08-13 16:24:07.560896 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e740 con 0x8c3e300 -8750> 2014-08-13 16:24:07.560924 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107540 con 0x8c618c0 -8749> 2014-08-13 16:24:07.560948 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x88c8000 con 0x8c6fe80 -8748> 2014-08-13 16:24:07.560971 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102300 con 0x8c6fbc0 -8747> 2014-08-13 16:24:07.560991 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedb00 con 0x8c6f900 -8746> 2014-08-13 16:24:07.561015 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefc40 con 0x8c6f640 -8745> 2014-08-13 16:24:07.561039 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899de80 con 0x8c6f380 -8744> 2014-08-13 16:24:07.561065 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775b480 con 0x8c6f0c0 -8743> 2014-08-13 16:24:07.561092 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9131dc0 con 0x8d223c0 -8742> 2014-08-13 16:24:07.561118 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef700 con 0x8d22100 -8741> 2014-08-13 16:24:07.561412 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.1.115:6814/59915 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eeb800 con 0x8d21e40 -8740> 2014-08-13 16:24:07.561449 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130540 con 0x8d218c0 -8739> 2014-08-13 16:24:07.561528 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130000 con 0x8d21600 -8738> 2014-08-13 16:24:07.561583 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91024c0 con 0x8d21340 -8737> 2014-08-13 16:24:07.561612 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9100380 con 0x8d21080 -8736> 2014-08-13 16:24:07.561640 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104b40 con 0x8d20dc0 -8735> 2014-08-13 16:24:07.561667 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ad80 con 0x8d20b00 -8734> 2014-08-13 16:24:07.561691 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9100540 con 0x8d20840 -8733> 2014-08-13 16:24:07.561716 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91055c0 con 0x8d25960 -8732> 2014-08-13 16:24:07.561739 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130a80 con 0x8d256a0 -8731> 2014-08-13 16:24:07.561764 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107e00 con 0x8d253e0 -8730> 2014-08-13 16:24:07.561788 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91308c0 con 0x8d25120 -8729> 2014-08-13 16:24:07.561812 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9130fc0 con 0x8d24e60 -8728> 2014-08-13 16:24:07.561836 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998540 con 0x8d24ba0 -8727> 2014-08-13 16:24:07.561862 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102a00 con 0x8d248e0 -8726> 2014-08-13 16:24:07.561890 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d5c0 con 0x8d24620 -8725> 2014-08-13 16:24:07.561916 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107c40 con 0x8d24360 -8724> 2014-08-13 16:24:07.561943 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998380 con 0x8d240a0 -8723> 2014-08-13 16:24:07.561968 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e580 con 0x8d23de0 -8722> 2014-08-13 16:24:07.561991 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefc40 con 0x8d23b20 -8721> 2014-08-13 16:24:07.562015 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e900 con 0x8d27bc0 -8720> 2014-08-13 16:24:07.562039 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef540 con 0x8d27900 -8719> 2014-08-13 16:24:07.562064 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed400 con 0x8d27640 -8718> 2014-08-13 16:24:07.562089 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102140 con 0x8d27380 -8717> 2014-08-13 16:24:07.562114 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899abc0 con 0x8d270c0 -8716> 2014-08-13 16:24:07.562138 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d940 con 0x8d26e00 -8715> 2014-08-13 16:24:07.562176 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e3c0 con 0x8d26b40 -8714> 2014-08-13 16:24:07.562201 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107a80 con 0x8d26880 -8713> 2014-08-13 16:24:07.562225 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102840 con 0x8d265c0 -8712> 2014-08-13 16:24:07.562249 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91001c0 con 0x8d26300 -8711> 2014-08-13 16:24:07.562273 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x89981c0 con 0x8d26040 -8710> 2014-08-13 16:24:07.562298 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899aa00 con 0x8d25d80 -8709> 2014-08-13 16:24:07.562324 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e200 con 0x8d140a0 -8708> 2014-08-13 16:24:07.562351 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedb00 con 0x8d13de0 -8707> 2014-08-13 16:24:07.562376 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102300 con 0x8d13b20 -8706> 2014-08-13 16:24:07.562400 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8700 con 0x78844c0 -8705> 2014-08-13 16:24:07.562426 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775c7c0 con 0x7884200 -8704> 2014-08-13 16:24:07.562449 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x77596c0 con 0x78848e0 -8703> 2014-08-13 16:24:07.562472 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee96c0 con 0x7884620 -8702> 2014-08-13 16:24:07.562495 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8540 con 0x794c360 -8701> 2014-08-13 16:24:07.562518 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d780 con 0x794dd80 -8700> 2014-08-13 16:24:07.562543 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ee40 con 0x7ffdac0 -8699> 2014-08-13 16:24:07.562568 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899eac0 con 0x7ffe040 -8698> 2014-08-13 16:24:07.562591 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef8c0 con 0x7ffdee0 -8697> 2014-08-13 16:24:07.562614 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed5c0 con 0x8b68b00 -8696> 2014-08-13 16:24:07.562638 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104980 con 0x8b689a0 -8695> 2014-08-13 16:24:07.562662 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eed780 con 0x8b6e1a0 -8694> 2014-08-13 16:24:07.562686 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefa80 con 0x7fb9b80 -8693> 2014-08-13 16:24:07.562711 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee9500 con 0x7fb9600 -8692> 2014-08-13 16:24:07.562736 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eef700 con 0x7fb94a0 -8691> 2014-08-13 16:24:07.562765 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9104ec0 con 0x7fbb020 -8690> 2014-08-13 16:24:07.562788 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9100000 con 0x7fbaaa0 -8689> 2014-08-13 16:24:07.562811 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee9340 con 0x7fba680 -8688> 2014-08-13 16:24:07.562834 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775f540 con 0x7fbbc80 -8687> 2014-08-13 16:24:07.562858 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102bc0 con 0x7fbbb20 -8686> 2014-08-13 16:24:07.562880 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x91078c0 con 0x7fbb9c0 -8685> 2014-08-13 16:24:07.562903 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eeb640 con 0x7fbb860 -8684> 2014-08-13 16:24:07.562925 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eebd40 con 0x7fbb700 -8683> 2014-08-13 16:24:07.562950 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899af40 con 0x7fbb5a0 -8682> 2014-08-13 16:24:07.562974 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899b100 con 0x7fbca40 -8681> 2014-08-13 16:24:07.562998 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899ec80 con 0x7fbbf40 -8680> 2014-08-13 16:24:07.563022 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899a840 con 0x8bacd00 -8679> 2014-08-13 16:24:07.563045 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8a80 con 0x8baca40 -8678> 2014-08-13 16:24:07.563067 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8ee8380 con 0x8bac780 -8677> 2014-08-13 16:24:07.563091 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899de80 con 0x8bac4c0 -8676> 2014-08-13 16:24:07.563113 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899dcc0 con 0x8baf900 -8675> 2014-08-13 16:24:07.563139 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105080 con 0x8baf380 -8674> 2014-08-13 16:24:07.563175 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8998700 con 0x8ba1760 -8673> 2014-08-13 16:24:07.563203 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899db00 con 0x8ba11e0 -8672> 2014-08-13 16:24:07.563229 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9107700 con 0x8ba0f20 -8671> 2014-08-13 16:24:07.563253 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9102680 con 0x8ba0c60 -8670> 2014-08-13 16:24:07.563277 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899e040 con 0x8ba09a0 -8669> 2014-08-13 16:24:07.563300 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eea300 con 0x8c6aaa0 -8668> 2014-08-13 16:24:07.563323 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x899d400 con 0x8c6a7e0 -8667> 2014-08-13 16:24:07.563346 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105400 con 0x8c6a520 -8666> 2014-08-13 16:24:07.563369 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eefe00 con 0x8c6a260 -8665> 2014-08-13 16:24:07.563391 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x9105240 con 0x8c39fa0 -8664> 2014-08-13 16:24:07.563414 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eedcc0 con 0x8c394a0 -8663> 2014-08-13 16:24:07.563436 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x8eeb9c0 con 0x8c389a0 -8662> 2014-08-13 16:24:07.563462 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.100.115:6831/59915 1 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.553291) v2 ==== 47+0+0 (444498177 0 0) 0x775b480 con 0x8d21b80 -8661> 2014-08-13 16:24:07.584491 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.584375) v2 ==== 47+0+0 (3029692297 0 0) 0x775c7c0 con 0x7d28420 -8660> 2014-08-13 16:24:07.584546 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.584375) v2 -- ?+0 0x80dd780 con 0x7d28420 -8659> 2014-08-13 16:24:07.584522 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.584375) v2 ==== 47+0+0 (3029692297 0 0) 0x8ee9500 con 0x7d2cba0 -8658> 2014-08-13 16:24:07.584587 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.584375) v2 -- ?+0 0x80dc440 con 0x7d2cba0 -8657> 2014-08-13 16:24:07.631398 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 9 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3946485199 0 0) 0x7ec9600 con 0x4e4f900 -8656> 2014-08-13 16:24:07.631453 7fd93dff0700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631176, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8655> 2014-08-13 16:24:07.631494 7fd93dff0700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631203, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8654> 2014-08-13 16:24:07.631508 7fd93dff0700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631316, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8653> 2014-08-13 16:24:07.631519 7fd93dff0700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631446, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8652> 2014-08-13 16:24:07.631530 7fd93dff0700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631530, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8651> 2014-08-13 16:24:07.631596 7fd9371e5700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631595, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8650> 2014-08-13 16:24:07.631639 7fd9371e5700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631639, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8649> 2014-08-13 16:24:07.631662 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8e400 con 0x4e4f900 -8648> 2014-08-13 16:24:07.631691 7fd9371e5700 5 -- op tracker -- , seq: 704, time: 2014-08-13 16:24:07.631690, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8647> 2014-08-13 16:24:07.632659 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 10 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2053850730 0 0) 0x7ec9600 con 0x4e4f900 -8646> 2014-08-13 16:24:07.632740 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632468, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8645> 2014-08-13 16:24:07.632764 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632470, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8644> 2014-08-13 16:24:07.632775 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632562, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8643> 2014-08-13 16:24:07.632784 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632735, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8642> 2014-08-13 16:24:07.632796 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632795, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8641> 2014-08-13 16:24:07.632851 7fd9367e4700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632850, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8640> 2014-08-13 16:24:07.632876 7fd9367e4700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632876, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8639> 2014-08-13 16:24:07.632902 7fd93dff0700 5 -- op tracker -- , seq: 705, time: 2014-08-13 16:24:07.632901, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8638> 2014-08-13 16:24:07.672539 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.672234) v2 ==== 47+0+0 (3819633550 0 0) 0x7d04600 con 0x7caf380 -8637> 2014-08-13 16:24:07.672588 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.672234) v2 -- ?+0 0x8ee9500 con 0x7caf380 -8636> 2014-08-13 16:24:07.672632 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.672234) v2 ==== 47+0+0 (3819633550 0 0) 0x80dd780 con 0x7d289a0 -8635> 2014-08-13 16:24:07.672711 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.672234) v2 -- ?+0 0x775c7c0 con 0x7d289a0 -8634> 2014-08-13 16:24:07.693900 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 9 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x7ece300 con 0x4e498c0 -8633> 2014-08-13 16:24:07.693945 7fd93dff0700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.693744, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8632> 2014-08-13 16:24:07.693961 7fd93dff0700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.693748, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8631> 2014-08-13 16:24:07.693971 7fd93dff0700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.693859, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8630> 2014-08-13 16:24:07.693980 7fd93dff0700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.693939, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8629> 2014-08-13 16:24:07.694000 7fd93dff0700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.694000, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8628> 2014-08-13 16:24:07.694102 7fd9371e5700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.694101, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8627> 2014-08-13 16:24:07.694142 7fd9371e5700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.694142, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8626> 2014-08-13 16:24:07.694161 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8da00 con 0x4e498c0 -8625> 2014-08-13 16:24:07.694193 7fd9371e5700 5 -- op tracker -- , seq: 706, time: 2014-08-13 16:24:07.694193, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8624> 2014-08-13 16:24:07.695488 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 10 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x863d800 con 0x4e498c0 -8623> 2014-08-13 16:24:07.695539 7fd93dff0700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695303, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8622> 2014-08-13 16:24:07.695555 7fd93dff0700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695306, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8621> 2014-08-13 16:24:07.695565 7fd93dff0700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695407, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8620> 2014-08-13 16:24:07.695574 7fd93dff0700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695533, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8619> 2014-08-13 16:24:07.695585 7fd93dff0700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695585, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8618> 2014-08-13 16:24:07.695687 7fd9367e4700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695686, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8617> 2014-08-13 16:24:07.695727 7fd9367e4700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695727, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8616> 2014-08-13 16:24:07.695742 7fd9367e4700 5 -- op tracker -- , seq: 707, time: 2014-08-13 16:24:07.695742, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8615> 2014-08-13 16:24:07.739612 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.739069) v2 ==== 47+0+0 (1018872513 0 0) 0x4f8aa00 con 0x7ebc620 -8614> 2014-08-13 16:24:07.739627 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.739069) v2 ==== 47+0+0 (1018872513 0 0) 0x775c7c0 con 0x7eb9b80 -8613> 2014-08-13 16:24:07.739663 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.739069) v2 -- ?+0 0x80dd780 con 0x7ebc620 -8612> 2014-08-13 16:24:07.739767 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.739069) v2 -- ?+0 0x7d04600 con 0x7eb9b80 -8611> 2014-08-13 16:24:07.864585 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 14 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8638000 con 0x4fe0000 -8610> 2014-08-13 16:24:07.864638 7fd93dff0700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864393, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8609> 2014-08-13 16:24:07.864654 7fd93dff0700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864397, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8608> 2014-08-13 16:24:07.864664 7fd93dff0700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864504, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8607> 2014-08-13 16:24:07.864673 7fd93dff0700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864632, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8606> 2014-08-13 16:24:07.864684 7fd93dff0700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864684, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8605> 2014-08-13 16:24:07.864788 7fd9371e5700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864788, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8604> 2014-08-13 16:24:07.864829 7fd9371e5700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864829, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8603> 2014-08-13 16:24:07.864848 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8df00 con 0x4fe0000 -8602> 2014-08-13 16:24:07.864877 7fd9371e5700 5 -- op tracker -- , seq: 708, time: 2014-08-13 16:24:07.864877, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8601> 2014-08-13 16:24:07.866010 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 15 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x8639600 con 0x4fe0000 -8600> 2014-08-13 16:24:07.866061 7fd93dff0700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.865817, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8599> 2014-08-13 16:24:07.866077 7fd93dff0700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.865820, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8598> 2014-08-13 16:24:07.866088 7fd93dff0700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.865930, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8597> 2014-08-13 16:24:07.866108 7fd93dff0700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.866055, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8596> 2014-08-13 16:24:07.866124 7fd93dff0700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.866124, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8595> 2014-08-13 16:24:07.866238 7fd9367e4700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.866237, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8594> 2014-08-13 16:24:07.866278 7fd9367e4700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.866278, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8593> 2014-08-13 16:24:07.866293 7fd9367e4700 5 -- op tracker -- , seq: 709, time: 2014-08-13 16:24:07.866293, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8592> 2014-08-13 16:24:07.943873 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 15 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x863c200 con 0x508bde0 -8591> 2014-08-13 16:24:07.943929 7fd93dff0700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.943678, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8590> 2014-08-13 16:24:07.943946 7fd93dff0700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.943682, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8589> 2014-08-13 16:24:07.943956 7fd93dff0700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.943793, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8588> 2014-08-13 16:24:07.943965 7fd93dff0700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.943922, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8587> 2014-08-13 16:24:07.943976 7fd93dff0700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.943976, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8586> 2014-08-13 16:24:07.944080 7fd9371e5700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.944080, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8585> 2014-08-13 16:24:07.944121 7fd9371e5700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.944121, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8584> 2014-08-13 16:24:07.944140 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8d780 con 0x508bde0 -8583> 2014-08-13 16:24:07.944173 7fd9371e5700 5 -- op tracker -- , seq: 710, time: 2014-08-13 16:24:07.944173, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8582> 2014-08-13 16:24:07.945739 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 16 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x8638000 con 0x508bde0 -8581> 2014-08-13 16:24:07.945793 7fd93dff0700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945573, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8580> 2014-08-13 16:24:07.945810 7fd93dff0700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945576, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8579> 2014-08-13 16:24:07.945820 7fd93dff0700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945674, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8578> 2014-08-13 16:24:07.945829 7fd93dff0700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945784, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8577> 2014-08-13 16:24:07.945854 7fd93dff0700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945854, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8576> 2014-08-13 16:24:07.945952 7fd9367e4700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945951, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8575> 2014-08-13 16:24:07.945992 7fd9367e4700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.945992, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8574> 2014-08-13 16:24:07.946007 7fd9367e4700 5 -- op tracker -- , seq: 711, time: 2014-08-13 16:24:07.946007, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8573> 2014-08-13 16:24:07.958851 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.958567) v2 ==== 47+0+0 (3179482034 0 0) 0x7d04b40 con 0x7d2f0c0 -8572> 2014-08-13 16:24:07.958905 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.958567) v2 -- ?+0 0x775c7c0 con 0x7d2f0c0 -8571> 2014-08-13 16:24:07.958933 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:07.958567) v2 ==== 47+0+0 (3179482034 0 0) 0x7d03480 con 0x7d2dc20 -8570> 2014-08-13 16:24:07.958986 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:07.958567) v2 -- ?+0 0x4f8aa00 con 0x7d2dc20 -8569> 2014-08-13 16:24:08.093908 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.093868) v2 ==== 47+0+0 (3692436223 0 0) 0x8c8e740 con 0x7edaec0 -8568> 2014-08-13 16:24:08.093930 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.093868) v2 ==== 47+0+0 (3692436223 0 0) 0x775fc40 con 0x7ed8000 -8567> 2014-08-13 16:24:08.093962 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.093868) v2 -- ?+0 0x7d03480 con 0x7edaec0 -8566> 2014-08-13 16:24:08.094017 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.093868) v2 -- ?+0 0x7d04b40 con 0x7ed8000 -8565> 2014-08-13 16:24:08.127570 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.106:6816/8118 -- osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11507 0x7ecac00 con 0x4fdd3e0 -8564> 2014-08-13 16:24:08.135544 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 11 ==== replica scrub(pg: 22.12a,from:0'0,to:194215'486,epoch:194449,start:825e692a//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (1688399103 0 0) 0x8bd0240 con 0x4fdd3e0 -8563> 2014-08-13 16:24:08.177362 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.177241) v2 ==== 47+0+0 (1305213926 0 0) 0x8ee9500 con 0x7ffeb40 -8562> 2014-08-13 16:24:08.177413 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.177241) v2 ==== 47+0+0 (1305213926 0 0) 0x8eef700 con 0x7edbde0 -8561> 2014-08-13 16:24:08.177412 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.177241) v2 -- ?+0 0x8c8e740 con 0x7ffeb40 -8560> 2014-08-13 16:24:08.177550 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.177241) v2 -- ?+0 0x775fc40 con 0x7edbde0 -8559> 2014-08-13 16:24:08.207362 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.207260) v2 ==== 47+0+0 (1214645650 0 0) 0x7d055c0 con 0x7d2a260 -8558> 2014-08-13 16:24:08.207393 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.207260) v2 ==== 47+0+0 (1214645650 0 0) 0x8c8bb80 con 0x7d28160 -8557> 2014-08-13 16:24:08.207417 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.207260) v2 -- ?+0 0x8eef700 con 0x7d2a260 -8556> 2014-08-13 16:24:08.207485 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.207260) v2 -- ?+0 0x8ee9500 con 0x7d28160 -8555> 2014-08-13 16:24:08.229009 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.106:6816/8118 -- osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+7404 0x863d800 con 0x4fdd3e0 -8554> 2014-08-13 16:24:08.233040 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 12 ==== osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1166623239 0 0) 0x863d800 con 0x4fdd3e0 -8553> 2014-08-13 16:24:08.233096 7fd93dff0700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.232844, event: header_read, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8552> 2014-08-13 16:24:08.233113 7fd93dff0700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.232848, event: throttled, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8551> 2014-08-13 16:24:08.233124 7fd93dff0700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.232956, event: all_read, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8550> 2014-08-13 16:24:08.233136 7fd93dff0700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.233089, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8549> 2014-08-13 16:24:08.233152 7fd93dff0700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.233151, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8548> 2014-08-13 16:24:08.233202 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 13 ==== pg_info(1 pgs e194449:22.12a) v4 ==== 714+0+0 (3954340606 0 0) 0x8eeb640 con 0x4fdd3e0 -8547> 2014-08-13 16:24:08.233223 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.232982, event: header_read, request: pg_info(1 pgs e194449:22.12a) v4 -8546> 2014-08-13 16:24:08.233231 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.232984, event: throttled, request: pg_info(1 pgs e194449:22.12a) v4 -8545> 2014-08-13 16:24:08.233240 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.233049, event: all_read, request: pg_info(1 pgs e194449:22.12a) v4 -8544> 2014-08-13 16:24:08.233249 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.233221, event: dispatched, request: pg_info(1 pgs e194449:22.12a) v4 -8543> 2014-08-13 16:24:08.233259 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.233259, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.12a) v4 -8542> 2014-08-13 16:24:08.233271 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.233271, event: started, request: pg_info(1 pgs e194449:22.12a) v4 -8541> 2014-08-13 16:24:08.233290 7fd9371e5700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.233275, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8540> 2014-08-13 16:24:08.233338 7fd9371e5700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.233338, event: started, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8539> 2014-08-13 16:24:08.233364 7fd9371e5700 5 -- op tracker -- , seq: 712, time: 2014-08-13 16:24:08.233364, event: done, request: osd_sub_op(unknown.0.0:0 22.12a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8538> 2014-08-13 16:24:08.233385 7fd93dff0700 5 -- op tracker -- , seq: 713, time: 2014-08-13 16:24:08.233377, event: done, request: pg_info(1 pgs e194449:22.12a) v4 -8537> 2014-08-13 16:24:08.263850 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 9 ==== osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (95788370 0 0) 0x8639600 con 0x50bc4c0 -8536> 2014-08-13 16:24:08.263903 7fd93dff0700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.263661, event: header_read, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8535> 2014-08-13 16:24:08.263922 7fd93dff0700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.263664, event: throttled, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8534> 2014-08-13 16:24:08.263936 7fd93dff0700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.263769, event: all_read, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8533> 2014-08-13 16:24:08.263949 7fd93dff0700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.263897, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8532> 2014-08-13 16:24:08.263963 7fd93dff0700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.263963, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8531> 2014-08-13 16:24:08.264064 7fd9371e5700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.264064, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8530> 2014-08-13 16:24:08.264108 7fd9371e5700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.264108, event: started, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8529> 2014-08-13 16:24:08.264131 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6832/6655 -- osd_sub_op_reply(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8004100 con 0x50bc4c0 -8528> 2014-08-13 16:24:08.264166 7fd9371e5700 5 -- op tracker -- , seq: 714, time: 2014-08-13 16:24:08.264166, event: done, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8527> 2014-08-13 16:24:08.278636 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 10 ==== replica scrub(pg: 22.537,from:0'0,to:194215'501,epoch:194449,start:0//0//-1,end:321ecd37//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (4095679556 0 0) 0x859fbc0 con 0x50bc4c0 -8526> 2014-08-13 16:24:08.334967 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 9 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4028734706 0 0) 0x8638000 con 0x5112aa0 -8525> 2014-08-13 16:24:08.335017 7fd93dff0700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.334813, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8524> 2014-08-13 16:24:08.335036 7fd93dff0700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.334817, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8523> 2014-08-13 16:24:08.335047 7fd93dff0700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.334927, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8522> 2014-08-13 16:24:08.335056 7fd93dff0700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.335011, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8521> 2014-08-13 16:24:08.335067 7fd93dff0700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.335067, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8520> 2014-08-13 16:24:08.335170 7fd9367e4700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.335169, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8519> 2014-08-13 16:24:08.335224 7fd9367e4700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.335223, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8518> 2014-08-13 16:24:08.335243 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6808/62751 -- osd_sub_op_reply(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003200 con 0x5112aa0 -8517> 2014-08-13 16:24:08.335274 7fd9367e4700 5 -- op tracker -- , seq: 715, time: 2014-08-13 16:24:08.335274, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8516> 2014-08-13 16:24:08.336749 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 10 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1634750295 0 0) 0x863b700 con 0x5112aa0 -8515> 2014-08-13 16:24:08.336797 7fd93dff0700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336609, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8514> 2014-08-13 16:24:08.336816 7fd93dff0700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336612, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8513> 2014-08-13 16:24:08.336827 7fd93dff0700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336712, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8512> 2014-08-13 16:24:08.336837 7fd93dff0700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336792, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8511> 2014-08-13 16:24:08.336851 7fd93dff0700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336850, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8510> 2014-08-13 16:24:08.336950 7fd9371e5700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336949, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8509> 2014-08-13 16:24:08.336990 7fd9371e5700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.336990, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8508> 2014-08-13 16:24:08.337005 7fd9371e5700 5 -- op tracker -- , seq: 716, time: 2014-08-13 16:24:08.337005, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8507> 2014-08-13 16:24:08.366661 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.366100) v2 ==== 47+0+0 (2211113507 0 0) 0x7d001c0 con 0x7ebce60 -8506> 2014-08-13 16:24:08.366711 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.366100) v2 -- ?+0 0x7d055c0 con 0x7ebce60 -8505> 2014-08-13 16:24:08.366716 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.366100) v2 ==== 47+0+0 (2211113507 0 0) 0x7d05080 con 0x7ebb860 -8504> 2014-08-13 16:24:08.366765 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.366100) v2 -- ?+0 0x8c8bb80 con 0x7ebb860 -8503> 2014-08-13 16:24:08.423306 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.422919) v2 ==== 47+0+0 (2542247066 0 0) 0x8eedb00 con 0x7d2b440 -8502> 2014-08-13 16:24:08.423314 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.422919) v2 ==== 47+0+0 (2542247066 0 0) 0x8ee8a80 con 0x7d29ce0 -8501> 2014-08-13 16:24:08.423352 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.422919) v2 -- ?+0 0x7d05080 con 0x7d2b440 -8500> 2014-08-13 16:24:08.423424 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.422919) v2 -- ?+0 0x7d001c0 con 0x7d29ce0 -8499> 2014-08-13 16:24:08.424986 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6832/6655 -- osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11547 0x863ee00 con 0x50bc4c0 -8498> 2014-08-13 16:24:08.430303 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 11 ==== replica scrub(pg: 22.537,from:0'0,to:192888'461,epoch:194449,start:321ecd37//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (315550372 0 0) 0x859f980 con 0x50bc4c0 -8497> 2014-08-13 16:24:08.431854 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6832/6655 -- osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+1425 0x7ecac00 con 0x50bc4c0 -8496> 2014-08-13 16:24:08.460857 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 12 ==== osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2497990391 0 0) 0x7ecac00 con 0x50bc4c0 -8495> 2014-08-13 16:24:08.460913 7fd93dff0700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.460689, event: header_read, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8494> 2014-08-13 16:24:08.460930 7fd93dff0700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.460691, event: throttled, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8493> 2014-08-13 16:24:08.460940 7fd93dff0700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.460789, event: all_read, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8492> 2014-08-13 16:24:08.460949 7fd93dff0700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.460907, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8491> 2014-08-13 16:24:08.460960 7fd93dff0700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.460960, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8490> 2014-08-13 16:24:08.460999 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 13 ==== pg_info(1 pgs e194449:22.537) v4 ==== 714+0+0 (3708548455 0 0) 0x8c8cec0 con 0x50bc4c0 -8489> 2014-08-13 16:24:08.461019 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.460833, event: header_read, request: pg_info(1 pgs e194449:22.537) v4 -8488> 2014-08-13 16:24:08.461027 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.460835, event: throttled, request: pg_info(1 pgs e194449:22.537) v4 -8487> 2014-08-13 16:24:08.461033 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.460910, event: all_read, request: pg_info(1 pgs e194449:22.537) v4 -8486> 2014-08-13 16:24:08.461038 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.461017, event: dispatched, request: pg_info(1 pgs e194449:22.537) v4 -8485> 2014-08-13 16:24:08.461044 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.461044, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.537) v4 -8484> 2014-08-13 16:24:08.461052 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.461052, event: started, request: pg_info(1 pgs e194449:22.537) v4 -8483> 2014-08-13 16:24:08.461062 7fd9367e4700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.461061, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8482> 2014-08-13 16:24:08.461105 7fd9367e4700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.461105, event: started, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8481> 2014-08-13 16:24:08.461129 7fd9367e4700 5 -- op tracker -- , seq: 717, time: 2014-08-13 16:24:08.461129, event: done, request: osd_sub_op(unknown.0.0:0 22.537 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8480> 2014-08-13 16:24:08.461194 7fd93dff0700 5 -- op tracker -- , seq: 718, time: 2014-08-13 16:24:08.461193, event: done, request: pg_info(1 pgs e194449:22.537) v4 -8479> 2014-08-13 16:24:08.485404 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.484717) v2 ==== 47+0+0 (1178585415 0 0) 0x7d05080 con 0x7d2ac00 -8478> 2014-08-13 16:24:08.485454 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.484717) v2 -- ?+0 0x8eedb00 con 0x7d2ac00 -8477> 2014-08-13 16:24:08.485513 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.484717) v2 ==== 47+0+0 (1178585415 0 0) 0x8ee96c0 con 0x7d2c360 -8476> 2014-08-13 16:24:08.485620 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.484717) v2 -- ?+0 0x8ee8a80 con 0x7d2c360 -8475> 2014-08-13 16:24:08.509353 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.509298) v2 ==== 47+0+0 (2116936334 0 0) 0x8eedb00 con 0x7edcba0 -8474> 2014-08-13 16:24:08.509361 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.509298) v2 ==== 47+0+0 (2116936334 0 0) 0x7d01500 con 0x7edac00 -8473> 2014-08-13 16:24:08.509405 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.509298) v2 -- ?+0 0x8ee96c0 con 0x7edcba0 -8472> 2014-08-13 16:24:08.509614 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.509298) v2 -- ?+0 0x7d05080 con 0x7edac00 -8471> 2014-08-13 16:24:08.595572 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.595547) v2 ==== 47+0+0 (1083546050 0 0) 0x8ee96c0 con 0x7edaec0 -8470> 2014-08-13 16:24:08.595599 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.595547) v2 ==== 47+0+0 (1083546050 0 0) 0x7d05080 con 0x7ed8000 -8469> 2014-08-13 16:24:08.595627 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.595547) v2 -- ?+0 0x7d01500 con 0x7edaec0 -8468> 2014-08-13 16:24:08.595686 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.595547) v2 -- ?+0 0x8eedb00 con 0x7ed8000 -8467> 2014-08-13 16:24:08.637134 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.637077) v2 ==== 47+0+0 (3985106055 0 0) 0x7d01a40 con 0x7ab8000 -8466> 2014-08-13 16:24:08.637232 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.637077) v2 -- ?+0 0x7d05080 con 0x7ab8000 -8465> 2014-08-13 16:24:08.637142 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.637077) v2 ==== 47+0+0 (3985106055 0 0) 0x7d04d00 con 0x7ab8420 -8464> 2014-08-13 16:24:08.637345 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.637077) v2 -- ?+0 0x8ee96c0 con 0x7ab8420 -8463> 2014-08-13 16:24:08.714455 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.714176) v2 ==== 47+0+0 (249256892 0 0) 0x7d02140 con 0x7ed8580 -8462> 2014-08-13 16:24:08.714510 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.714176) v2 -- ?+0 0x7d01a40 con 0x7ed8580 -8461> 2014-08-13 16:24:08.714502 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.714176) v2 ==== 47+0+0 (249256892 0 0) 0x7d03f00 con 0x7ebe5c0 -8460> 2014-08-13 16:24:08.714552 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.714176) v2 -- ?+0 0x7d04d00 con 0x7ebe5c0 -8459> 2014-08-13 16:24:08.737842 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.737774) v2 ==== 47+0+0 (484210976 0 0) 0x775f000 con 0x7d286e0 -8458> 2014-08-13 16:24:08.737859 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.737774) v2 ==== 47+0+0 (484210976 0 0) 0x7d01a40 con 0x7caf0c0 -8457> 2014-08-13 16:24:08.737892 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.737774) v2 -- ?+0 0x7d03f00 con 0x7d286e0 -8456> 2014-08-13 16:24:08.738005 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.737774) v2 -- ?+0 0x7d02140 con 0x7caf0c0 -8455> 2014-08-13 16:24:08.780414 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.780430) v2 ==== 47+0+0 (291524437 0 0) 0x7759a40 con 0x7cae5c0 -8454> 2014-08-13 16:24:08.780467 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.780430) v2 -- ?+0 0x775f000 con 0x7cae5c0 -8453> 2014-08-13 16:24:08.780475 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.780430) v2 ==== 47+0+0 (291524437 0 0) 0x7d03f00 con 0x7d2a520 -8452> 2014-08-13 16:24:08.780518 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.780430) v2 -- ?+0 0x7d01a40 con 0x7d2a520 -8451> 2014-08-13 16:24:08.780905 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.780599) v2 ==== 47+0+0 (3652844002 0 0) 0x775f000 con 0x7ffe9e0 -8450> 2014-08-13 16:24:08.780953 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.780599) v2 -- ?+0 0x7759a40 con 0x7ffe9e0 -8449> 2014-08-13 16:24:08.781000 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 1 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.780599) v2 ==== 47+0+0 (3652844002 0 0) 0x775f8c0 con 0x7edbc80 -8448> 2014-08-13 16:24:08.781043 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.780599) v2 -- ?+0 0x7d03f00 con 0x7edbc80 -8447> 2014-08-13 16:24:08.782784 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.782777) v2 ==== 47+0+0 (361898427 0 0) 0x7d01a40 con 0x7eb91e0 -8446> 2014-08-13 16:24:08.782800 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.782777) v2 ==== 47+0+0 (361898427 0 0) 0x7d04b40 con 0x7eba680 -8445> 2014-08-13 16:24:08.782823 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.782777) v2 -- ?+0 0x775f000 con 0x7eb91e0 -8444> 2014-08-13 16:24:08.782908 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.782777) v2 -- ?+0 0x775f8c0 con 0x7eba680 -8443> 2014-08-13 16:24:08.799622 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.799452) v2 ==== 47+0+0 (1118068155 0 0) 0x775f000 con 0x7eda3c0 -8442> 2014-08-13 16:24:08.799634 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.799452) v2 ==== 47+0+0 (1118068155 0 0) 0x775f8c0 con 0x7edc360 -8441> 2014-08-13 16:24:08.799672 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.799452) v2 -- ?+0 0x7d01a40 con 0x7eda3c0 -8440> 2014-08-13 16:24:08.799739 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.799452) v2 -- ?+0 0x7d04b40 con 0x7edc360 -8439> 2014-08-13 16:24:08.829085 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 9 ==== osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1100188727 0 0) 0x863e300 con 0x4fe4ba0 -8438> 2014-08-13 16:24:08.829130 7fd93dff0700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.828931, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8437> 2014-08-13 16:24:08.829159 7fd93dff0700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.828933, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8436> 2014-08-13 16:24:08.829170 7fd93dff0700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829049, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8435> 2014-08-13 16:24:08.829184 7fd93dff0700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829125, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8434> 2014-08-13 16:24:08.829196 7fd93dff0700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829195, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8433> 2014-08-13 16:24:08.829304 7fd9367e4700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829304, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8432> 2014-08-13 16:24:08.829349 7fd9367e4700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829349, event: started, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8431> 2014-08-13 16:24:08.829369 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6812/43236 -- osd_sub_op_reply(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002f80 con 0x4fe4ba0 -8430> 2014-08-13 16:24:08.829410 7fd9367e4700 5 -- op tracker -- , seq: 719, time: 2014-08-13 16:24:08.829410, event: done, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8429> 2014-08-13 16:24:08.832450 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.832517) v2 ==== 47+0+0 (734640390 0 0) 0x7d04b40 con 0x7ffe880 -8428> 2014-08-13 16:24:08.832461 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.832517) v2 ==== 47+0+0 (734640390 0 0) 0x7d01a40 con 0x7edbb20 -8427> 2014-08-13 16:24:08.832502 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.832517) v2 -- ?+0 0x775f000 con 0x7ffe880 -8426> 2014-08-13 16:24:08.832641 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.832517) v2 -- ?+0 0x775f8c0 con 0x7edbb20 -8425> 2014-08-13 16:24:08.835986 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 10 ==== replica scrub(pg: 22.4aa,from:0'0,to:194215'473,epoch:194449,start:0//0//-1,end:f33d4caa//0//-1,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (1088076081 0 0) 0x7cf5a00 con 0x4fe4ba0 -8424> 2014-08-13 16:24:08.838773 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.838431) v2 ==== 47+0+0 (4048840676 0 0) 0x7d01500 con 0x7eb8c60 -8423> 2014-08-13 16:24:08.838808 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.838431) v2 -- ?+0 0x7d04b40 con 0x7eb8c60 -8422> 2014-08-13 16:24:08.838810 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.838431) v2 ==== 47+0+0 (4048840676 0 0) 0x775f000 con 0x7eba100 -8421> 2014-08-13 16:24:08.838862 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.838431) v2 -- ?+0 0x7d01a40 con 0x7eba100 -8420> 2014-08-13 16:24:08.849041 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.849113) v2 ==== 47+0+0 (3390280044 0 0) 0x8eedb00 con 0x7ebd6a0 -8419> 2014-08-13 16:24:08.849092 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.849113) v2 -- ?+0 0x775f000 con 0x7ebd6a0 -8418> 2014-08-13 16:24:08.849100 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.849113) v2 ==== 47+0+0 (3390280044 0 0) 0x8eef8c0 con 0x7ebc0a0 -8417> 2014-08-13 16:24:08.849210 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.849113) v2 -- ?+0 0x7d01500 con 0x7ebc0a0 -8416> 2014-08-13 16:24:08.853053 7fd94ac49700 5 osd.4 194449 tick -8415> 2014-08-13 16:24:08.853311 7fd94ac49700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -8414> 2014-08-13 16:24:08.853334 7fd94ac49700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- pg_stats(31 pgs tid 1 v 0) v1 -- ?+0 0x8003480 con 0x4e3dd80 -8413> 2014-08-13 16:24:08.865248 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 16 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8bc1600 con 0x4fe0000 -8412> 2014-08-13 16:24:08.865299 7fd93dff0700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865037, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8411> 2014-08-13 16:24:08.865316 7fd93dff0700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865041, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8410> 2014-08-13 16:24:08.865326 7fd93dff0700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865164, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8409> 2014-08-13 16:24:08.865335 7fd93dff0700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865293, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8408> 2014-08-13 16:24:08.865351 7fd93dff0700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865351, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8407> 2014-08-13 16:24:08.865454 7fd9371e5700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865453, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8406> 2014-08-13 16:24:08.865495 7fd9371e5700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865494, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8405> 2014-08-13 16:24:08.865515 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003e80 con 0x4fe0000 -8404> 2014-08-13 16:24:08.865546 7fd9371e5700 5 -- op tracker -- , seq: 720, time: 2014-08-13 16:24:08.865546, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8403> 2014-08-13 16:24:08.866627 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 17 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x8bc0b00 con 0x4fe0000 -8402> 2014-08-13 16:24:08.866680 7fd93dff0700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866458, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8401> 2014-08-13 16:24:08.866699 7fd93dff0700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866460, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8400> 2014-08-13 16:24:08.866713 7fd93dff0700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866560, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8399> 2014-08-13 16:24:08.866726 7fd93dff0700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866673, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8398> 2014-08-13 16:24:08.866741 7fd93dff0700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866741, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8397> 2014-08-13 16:24:08.866844 7fd9367e4700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866843, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8396> 2014-08-13 16:24:08.866886 7fd9367e4700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866886, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8395> 2014-08-13 16:24:08.866906 7fd9367e4700 5 -- op tracker -- , seq: 721, time: 2014-08-13 16:24:08.866905, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8394> 2014-08-13 16:24:08.907824 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.907583) v2 ==== 47+0+0 (2898530981 0 0) 0x7d01500 con 0x7edb180 -8393> 2014-08-13 16:24:08.907939 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.907583) v2 -- ?+0 0x8eef8c0 con 0x7edb180 -8392> 2014-08-13 16:24:08.907868 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.907583) v2 ==== 47+0+0 (2898530981 0 0) 0x775f000 con 0x7ed82c0 -8391> 2014-08-13 16:24:08.908060 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.907583) v2 -- ?+0 0x8eedb00 con 0x7ed82c0 -8390> 2014-08-13 16:24:08.987335 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.986715) v2 ==== 47+0+0 (2351765182 0 0) 0x7d001c0 con 0x7d2ac00 -8389> 2014-08-13 16:24:08.987351 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:08.986715) v2 ==== 47+0+0 (2351765182 0 0) 0x8ee8700 con 0x7d2c360 -8388> 2014-08-13 16:24:08.987395 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.986715) v2 -- ?+0 0x7d01500 con 0x7d2ac00 -8387> 2014-08-13 16:24:08.987542 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:08.986715) v2 -- ?+0 0x775f000 con 0x7d2c360 -8386> 2014-08-13 16:24:09.013689 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 11 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x7ecac00 con 0x5111fa0 -8385> 2014-08-13 16:24:09.013746 7fd93dff0700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013492, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8384> 2014-08-13 16:24:09.013763 7fd93dff0700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013496, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8383> 2014-08-13 16:24:09.013773 7fd93dff0700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013607, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8382> 2014-08-13 16:24:09.013782 7fd93dff0700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013740, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8381> 2014-08-13 16:24:09.013793 7fd93dff0700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013792, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8380> 2014-08-13 16:24:09.013897 7fd9371e5700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013897, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8379> 2014-08-13 16:24:09.013942 7fd9371e5700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013942, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8378> 2014-08-13 16:24:09.013961 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003c00 con 0x5111fa0 -8377> 2014-08-13 16:24:09.013990 7fd9371e5700 5 -- op tracker -- , seq: 722, time: 2014-08-13 16:24:09.013990, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8376> 2014-08-13 16:24:09.015194 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 12 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x863e300 con 0x5111fa0 -8375> 2014-08-13 16:24:09.015245 7fd93dff0700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.014986, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8374> 2014-08-13 16:24:09.015261 7fd93dff0700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.014989, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8373> 2014-08-13 16:24:09.015271 7fd93dff0700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015087, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8372> 2014-08-13 16:24:09.015280 7fd93dff0700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015239, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8371> 2014-08-13 16:24:09.015292 7fd93dff0700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015292, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8370> 2014-08-13 16:24:09.015368 7fd9367e4700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015367, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8369> 2014-08-13 16:24:09.015408 7fd9367e4700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015408, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8368> 2014-08-13 16:24:09.015424 7fd9367e4700 5 -- op tracker -- , seq: 723, time: 2014-08-13 16:24:09.015423, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8367> 2014-08-13 16:24:09.064810 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.064559) v2 ==== 47+0+0 (3003876450 0 0) 0x8eedb00 con 0x7d2f0c0 -8366> 2014-08-13 16:24:09.064863 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.064559) v2 -- ?+0 0x8ee8700 con 0x7d2f0c0 -8365> 2014-08-13 16:24:09.064887 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.064559) v2 ==== 47+0+0 (3003876450 0 0) 0x8eed780 con 0x7d2dc20 -8364> 2014-08-13 16:24:09.064938 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.064559) v2 -- ?+0 0x7d001c0 con 0x7d2dc20 -8363> 2014-08-13 16:24:09.126535 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.126202) v2 ==== 47+0+0 (2399800644 0 0) 0x775abc0 con 0x7ebbde0 -8362> 2014-08-13 16:24:09.126587 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.126202) v2 -- ?+0 0x8eed780 con 0x7ebbde0 -8361> 2014-08-13 16:24:09.126624 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.126202) v2 ==== 47+0+0 (2399800644 0 0) 0x7759500 con 0x7ebd3e0 -8360> 2014-08-13 16:24:09.126663 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.126202) v2 -- ?+0 0x8eedb00 con 0x7ebd3e0 -8359> 2014-08-13 16:24:09.213505 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.212753) v2 ==== 47+0+0 (734011026 0 0) 0x8eefc40 con 0x7ab9ce0 -8358> 2014-08-13 16:24:09.213513 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.212753) v2 ==== 47+0+0 (734011026 0 0) 0x8eed400 con 0x7ab9a20 -8357> 2014-08-13 16:24:09.213544 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.212753) v2 -- ?+0 0x7759500 con 0x7ab9ce0 -8356> 2014-08-13 16:24:09.213689 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.212753) v2 -- ?+0 0x775abc0 con 0x7ab9a20 -8355> 2014-08-13 16:24:09.301554 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.301418) v2 ==== 47+0+0 (2136770438 0 0) 0x775d940 con 0x7eda3c0 -8354> 2014-08-13 16:24:09.301566 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.301418) v2 ==== 47+0+0 (2136770438 0 0) 0x775b2c0 con 0x7edc360 -8353> 2014-08-13 16:24:09.301609 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.301418) v2 -- ?+0 0x8eed400 con 0x7eda3c0 -8352> 2014-08-13 16:24:09.301667 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.301418) v2 -- ?+0 0x8eefc40 con 0x7edc360 -8351> 2014-08-13 16:24:09.351352 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 22 ==== pg_stats_ack(31 pgs tid 1) v1 ==== 903+0+0 (93875842 0 0) 0x4f8bb80 con 0x4e3dd80 -8350> 2014-08-13 16:24:09.411286 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.411053) v2 ==== 47+0+0 (18052839 0 0) 0x7d03640 con 0x7ed82c0 -8349> 2014-08-13 16:24:09.411334 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.411053) v2 -- ?+0 0x775b2c0 con 0x7ed82c0 -8348> 2014-08-13 16:24:09.411314 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.411053) v2 ==== 47+0+0 (18052839 0 0) 0x7d00c40 con 0x7edb180 -8347> 2014-08-13 16:24:09.411416 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.411053) v2 -- ?+0 0x775d940 con 0x7edb180 -8346> 2014-08-13 16:24:09.543490 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 15 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x863b700 con 0x5112520 -8345> 2014-08-13 16:24:09.543544 7fd93dff0700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543272, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8344> 2014-08-13 16:24:09.543561 7fd93dff0700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543276, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8343> 2014-08-13 16:24:09.543571 7fd93dff0700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543409, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8342> 2014-08-13 16:24:09.543580 7fd93dff0700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543538, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8341> 2014-08-13 16:24:09.543591 7fd93dff0700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543591, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8340> 2014-08-13 16:24:09.543702 7fd9371e5700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543702, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8339> 2014-08-13 16:24:09.543744 7fd9371e5700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543744, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8338> 2014-08-13 16:24:09.543764 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003980 con 0x5112520 -8337> 2014-08-13 16:24:09.543797 7fd9371e5700 5 -- op tracker -- , seq: 724, time: 2014-08-13 16:24:09.543797, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8336> 2014-08-13 16:24:09.545310 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 16 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x8639600 con 0x5112520 -8335> 2014-08-13 16:24:09.545361 7fd93dff0700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545119, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8334> 2014-08-13 16:24:09.545377 7fd93dff0700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545122, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8333> 2014-08-13 16:24:09.545388 7fd93dff0700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545232, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8332> 2014-08-13 16:24:09.545397 7fd93dff0700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545355, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8331> 2014-08-13 16:24:09.545408 7fd93dff0700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545408, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8330> 2014-08-13 16:24:09.545508 7fd9367e4700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545508, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8329> 2014-08-13 16:24:09.545549 7fd9367e4700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545549, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8328> 2014-08-13 16:24:09.545565 7fd9367e4700 5 -- op tracker -- , seq: 725, time: 2014-08-13 16:24:09.545565, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8327> 2014-08-13 16:24:09.632548 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 11 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3946485199 0 0) 0x7ec8b00 con 0x4e4f900 -8326> 2014-08-13 16:24:09.632602 7fd93dff0700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632367, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8325> 2014-08-13 16:24:09.632618 7fd93dff0700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632371, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8324> 2014-08-13 16:24:09.632628 7fd93dff0700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632481, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8323> 2014-08-13 16:24:09.632638 7fd93dff0700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632596, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8322> 2014-08-13 16:24:09.632649 7fd93dff0700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632649, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8321> 2014-08-13 16:24:09.632752 7fd9371e5700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632752, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8320> 2014-08-13 16:24:09.632793 7fd9371e5700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632793, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8319> 2014-08-13 16:24:09.632812 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003700 con 0x4e4f900 -8318> 2014-08-13 16:24:09.632841 7fd9371e5700 5 -- op tracker -- , seq: 726, time: 2014-08-13 16:24:09.632841, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8317> 2014-08-13 16:24:09.634046 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 12 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2053850730 0 0) 0x8bc2100 con 0x4e4f900 -8316> 2014-08-13 16:24:09.634099 7fd93dff0700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.633849, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8315> 2014-08-13 16:24:09.634115 7fd93dff0700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.633852, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8314> 2014-08-13 16:24:09.634125 7fd93dff0700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.633964, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8313> 2014-08-13 16:24:09.634135 7fd93dff0700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.634091, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8312> 2014-08-13 16:24:09.634150 7fd93dff0700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.634150, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8311> 2014-08-13 16:24:09.634268 7fd9367e4700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.634267, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8310> 2014-08-13 16:24:09.634309 7fd9367e4700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.634308, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8309> 2014-08-13 16:24:09.634324 7fd9367e4700 5 -- op tracker -- , seq: 727, time: 2014-08-13 16:24:09.634324, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8308> 2014-08-13 16:24:09.698930 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.698867) v2 ==== 47+0+0 (221192815 0 0) 0x775abc0 con 0x7ebf0c0 -8307> 2014-08-13 16:24:09.698963 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.698867) v2 ==== 47+0+0 (221192815 0 0) 0x7759500 con 0x7ed9080 -8306> 2014-08-13 16:24:09.698983 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.698867) v2 -- ?+0 0x7d00c40 con 0x7ebf0c0 -8305> 2014-08-13 16:24:09.699061 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.698867) v2 -- ?+0 0x7d03640 con 0x7ed9080 -8304> 2014-08-13 16:24:09.713089 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.712673) v2 ==== 47+0+0 (2424832756 0 0) 0x775b100 con 0x7eda680 -8303> 2014-08-13 16:24:09.713144 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.712673) v2 -- ?+0 0x775abc0 con 0x7eda680 -8302> 2014-08-13 16:24:09.713114 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.712673) v2 ==== 47+0+0 (2424832756 0 0) 0x7d00c40 con 0x7edc620 -8301> 2014-08-13 16:24:09.713257 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.712673) v2 -- ?+0 0x7759500 con 0x7edc620 -8300> 2014-08-13 16:24:09.714922 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.714382) v2 ==== 47+0+0 (2108106216 0 0) 0x8eef540 con 0x7ab9a20 -8299> 2014-08-13 16:24:09.714963 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.714382) v2 -- ?+0 0x775b100 con 0x7ab9a20 -8298> 2014-08-13 16:24:09.714975 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.714382) v2 ==== 47+0+0 (2108106216 0 0) 0x8eefa80 con 0x7ab9ce0 -8297> 2014-08-13 16:24:09.715057 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.714382) v2 -- ?+0 0x7d00c40 con 0x7ab9ce0 -8296> 2014-08-13 16:24:09.742254 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.742140) v2 ==== 47+0+0 (127310966 0 0) 0x775abc0 con 0x7ebee00 -8295> 2014-08-13 16:24:09.742262 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.742140) v2 ==== 47+0+0 (127310966 0 0) 0x7d00000 con 0x7ed8dc0 -8294> 2014-08-13 16:24:09.742300 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.742140) v2 -- ?+0 0x8eef540 con 0x7ebee00 -8293> 2014-08-13 16:24:09.742347 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.742140) v2 -- ?+0 0x8eefa80 con 0x7ed8dc0 -8292> 2014-08-13 16:24:09.853431 7fd94ac49700 5 osd.4 194449 tick -8291> 2014-08-13 16:24:09.865827 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 18 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8bc0000 con 0x4fe0000 -8290> 2014-08-13 16:24:09.865878 7fd93dff0700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.865624, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8289> 2014-08-13 16:24:09.865894 7fd93dff0700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.865628, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8288> 2014-08-13 16:24:09.865904 7fd93dff0700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.865744, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8287> 2014-08-13 16:24:09.865914 7fd93dff0700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.865872, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8286> 2014-08-13 16:24:09.865925 7fd93dff0700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.865925, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8285> 2014-08-13 16:24:09.866028 7fd9371e5700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.866028, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8284> 2014-08-13 16:24:09.866070 7fd9371e5700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.866069, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8283> 2014-08-13 16:24:09.866089 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x7bc7300 con 0x4fe0000 -8282> 2014-08-13 16:24:09.866118 7fd9371e5700 5 -- op tracker -- , seq: 728, time: 2014-08-13 16:24:09.866117, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8281> 2014-08-13 16:24:09.867507 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 19 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x8bc4200 con 0x4fe0000 -8280> 2014-08-13 16:24:09.867558 7fd93dff0700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867317, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8279> 2014-08-13 16:24:09.867574 7fd93dff0700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867320, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8278> 2014-08-13 16:24:09.867585 7fd93dff0700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867426, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8277> 2014-08-13 16:24:09.867603 7fd93dff0700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867552, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8276> 2014-08-13 16:24:09.867621 7fd93dff0700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867621, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8275> 2014-08-13 16:24:09.867722 7fd9367e4700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867722, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8274> 2014-08-13 16:24:09.867763 7fd9367e4700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867763, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8273> 2014-08-13 16:24:09.867778 7fd9367e4700 5 -- op tracker -- , seq: 729, time: 2014-08-13 16:24:09.867778, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8272> 2014-08-13 16:24:09.882519 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.882584) v2 ==== 47+0+0 (1139842210 0 0) 0x775a680 con 0x7cae5c0 -8271> 2014-08-13 16:24:09.882573 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.882584) v2 -- ?+0 0x775abc0 con 0x7cae5c0 -8270> 2014-08-13 16:24:09.882511 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.882584) v2 ==== 47+0+0 (1139842210 0 0) 0x7d02140 con 0x7d2a520 -8269> 2014-08-13 16:24:09.882623 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.882584) v2 -- ?+0 0x7d00000 con 0x7d2a520 -8268> 2014-08-13 16:24:09.913518 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.913298) v2 ==== 47+0+0 (482628679 0 0) 0x7758c40 con 0x7edb180 -8267> 2014-08-13 16:24:09.913570 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.913298) v2 -- ?+0 0x775a680 con 0x7edb180 -8266> 2014-08-13 16:24:09.913554 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.913298) v2 ==== 47+0+0 (482628679 0 0) 0x8eeb800 con 0x7ed82c0 -8265> 2014-08-13 16:24:09.913815 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.913298) v2 -- ?+0 0x7d02140 con 0x7ed82c0 -8264> 2014-08-13 16:24:09.940281 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.939992) v2 ==== 47+0+0 (1846908582 0 0) 0x8eed940 con 0x7d2aec0 -8263> 2014-08-13 16:24:09.940332 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.939992) v2 -- ?+0 0x7758c40 con 0x7d2aec0 -8262> 2014-08-13 16:24:09.940342 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:09.939992) v2 ==== 47+0+0 (1846908582 0 0) 0x9100000 con 0x7d2c620 -8261> 2014-08-13 16:24:09.940385 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:09.939992) v2 -- ?+0 0x8eeb800 con 0x7d2c620 -8260> 2014-08-13 16:24:09.991055 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 9 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1487161271 0 0) 0x8bc4d00 con 0x50b9a20 -8259> 2014-08-13 16:24:09.991100 7fd93dff0700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.990897, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8258> 2014-08-13 16:24:09.991116 7fd93dff0700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.990900, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8257> 2014-08-13 16:24:09.991126 7fd93dff0700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991012, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8256> 2014-08-13 16:24:09.991142 7fd93dff0700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991094, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8255> 2014-08-13 16:24:09.991169 7fd93dff0700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991169, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8254> 2014-08-13 16:24:09.991657 7fd9371e5700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991656, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8253> 2014-08-13 16:24:09.991701 7fd9371e5700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991701, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8252> 2014-08-13 16:24:09.991719 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op_reply(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x7bc5f00 con 0x50b9a20 -8251> 2014-08-13 16:24:09.991751 7fd9371e5700 5 -- op tracker -- , seq: 730, time: 2014-08-13 16:24:09.991751, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8250> 2014-08-13 16:24:09.993424 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 10 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3388314642 0 0) 0x7ecac00 con 0x50b9a20 -8249> 2014-08-13 16:24:09.993474 7fd93dff0700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993289, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8248> 2014-08-13 16:24:09.993494 7fd93dff0700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993291, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8247> 2014-08-13 16:24:09.993508 7fd93dff0700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993387, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8246> 2014-08-13 16:24:09.993518 7fd93dff0700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993468, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8245> 2014-08-13 16:24:09.993533 7fd93dff0700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993533, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8244> 2014-08-13 16:24:09.993590 7fd9367e4700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993589, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8243> 2014-08-13 16:24:09.993627 7fd9367e4700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993627, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8242> 2014-08-13 16:24:09.993648 7fd9367e4700 5 -- op tracker -- , seq: 731, time: 2014-08-13 16:24:09.993648, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8241> 2014-08-13 16:24:10.029273 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.028803) v2 ==== 47+0+0 (879138291 0 0) 0x775aa00 con 0x7eda100 -8240> 2014-08-13 16:24:10.029325 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.028803) v2 -- ?+0 0x8eed940 con 0x7eda100 -8239> 2014-08-13 16:24:10.029395 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.028803) v2 ==== 47+0+0 (879138291 0 0) 0x7758c40 con 0x7edc0a0 -8238> 2014-08-13 16:24:10.029431 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.028803) v2 -- ?+0 0x9100000 con 0x7edc0a0 -8237> 2014-08-13 16:24:10.210752 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.210764) v2 ==== 47+0+0 (537666224 0 0) 0x8ee8a80 con 0x7edcba0 -8236> 2014-08-13 16:24:10.210788 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.210764) v2 ==== 47+0+0 (537666224 0 0) 0x7d07380 con 0x7edac00 -8235> 2014-08-13 16:24:10.210808 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.210764) v2 -- ?+0 0x7758c40 con 0x7edcba0 -8234> 2014-08-13 16:24:10.210910 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.210764) v2 -- ?+0 0x775aa00 con 0x7edac00 -8233> 2014-08-13 16:24:10.243202 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.243092) v2 ==== 47+0+0 (856280418 0 0) 0x7d07700 con 0x7ed8dc0 -8232> 2014-08-13 16:24:10.243255 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.243092) v2 -- ?+0 0x8ee8a80 con 0x7ed8dc0 -8231> 2014-08-13 16:24:10.243245 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.243092) v2 ==== 47+0+0 (856280418 0 0) 0x7759500 con 0x7ebee00 -8230> 2014-08-13 16:24:10.243322 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.243092) v2 -- ?+0 0x7d07380 con 0x7ebee00 -8229> 2014-08-13 16:24:10.335402 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 11 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4028734706 0 0) 0x863c200 con 0x5112aa0 -8228> 2014-08-13 16:24:10.335436 7fd93dff0700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335259, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8227> 2014-08-13 16:24:10.335446 7fd93dff0700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335261, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8226> 2014-08-13 16:24:10.335451 7fd93dff0700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335353, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8225> 2014-08-13 16:24:10.335457 7fd93dff0700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335430, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8224> 2014-08-13 16:24:10.335466 7fd93dff0700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335466, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8223> 2014-08-13 16:24:10.335536 7fd9371e5700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335536, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8222> 2014-08-13 16:24:10.335563 7fd9371e5700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335563, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8221> 2014-08-13 16:24:10.335577 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6808/62751 -- osd_sub_op_reply(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47800 con 0x5112aa0 -8220> 2014-08-13 16:24:10.335595 7fd9371e5700 5 -- op tracker -- , seq: 732, time: 2014-08-13 16:24:10.335595, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8219> 2014-08-13 16:24:10.336692 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 12 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1634750295 0 0) 0x7ecac00 con 0x5112aa0 -8218> 2014-08-13 16:24:10.336722 7fd93dff0700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336585, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8217> 2014-08-13 16:24:10.336732 7fd93dff0700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336587, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8216> 2014-08-13 16:24:10.336738 7fd93dff0700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336648, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8215> 2014-08-13 16:24:10.336744 7fd93dff0700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336719, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8214> 2014-08-13 16:24:10.336750 7fd93dff0700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336750, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8213> 2014-08-13 16:24:10.336814 7fd9367e4700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336814, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8212> 2014-08-13 16:24:10.336839 7fd9367e4700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336839, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8211> 2014-08-13 16:24:10.336848 7fd9367e4700 5 -- op tracker -- , seq: 733, time: 2014-08-13 16:24:10.336848, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8210> 2014-08-13 16:24:10.441983 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.441922) v2 ==== 47+0+0 (1655457497 0 0) 0x9104ec0 con 0x7d2c620 -8209> 2014-08-13 16:24:10.442008 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.441922) v2 ==== 47+0+0 (1655457497 0 0) 0x8eebb80 con 0x7d2aec0 -8208> 2014-08-13 16:24:10.442031 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.441922) v2 -- ?+0 0x7d07700 con 0x7d2c620 -8207> 2014-08-13 16:24:10.442098 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.441922) v2 -- ?+0 0x7759500 con 0x7d2aec0 -8206> 2014-08-13 16:24:10.486785 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.486755) v2 ==== 47+0+0 (906339344 0 0) 0x775f540 con 0x7d28420 -8205> 2014-08-13 16:24:10.486799 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.486755) v2 ==== 47+0+0 (906339344 0 0) 0x77596c0 con 0x7d2cba0 -8204> 2014-08-13 16:24:10.486835 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.486755) v2 -- ?+0 0x8eebb80 con 0x7d28420 -8203> 2014-08-13 16:24:10.486878 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.486755) v2 -- ?+0 0x9104ec0 con 0x7d2cba0 -8202> 2014-08-13 16:24:10.543688 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 17 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x863d800 con 0x5112520 -8201> 2014-08-13 16:24:10.543739 7fd93dff0700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543522, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8200> 2014-08-13 16:24:10.543755 7fd93dff0700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543526, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8199> 2014-08-13 16:24:10.543765 7fd93dff0700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543657, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8198> 2014-08-13 16:24:10.543778 7fd93dff0700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543732, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8197> 2014-08-13 16:24:10.543794 7fd93dff0700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543793, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8196> 2014-08-13 16:24:10.543879 7fd9371e5700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543878, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8195> 2014-08-13 16:24:10.543920 7fd9371e5700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543920, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8194> 2014-08-13 16:24:10.543938 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47580 con 0x5112520 -8193> 2014-08-13 16:24:10.543967 7fd9371e5700 5 -- op tracker -- , seq: 734, time: 2014-08-13 16:24:10.543967, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8192> 2014-08-13 16:24:10.545306 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 18 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x8bc0000 con 0x5112520 -8191> 2014-08-13 16:24:10.545333 7fd93dff0700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545189, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8190> 2014-08-13 16:24:10.545345 7fd93dff0700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545190, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8189> 2014-08-13 16:24:10.545354 7fd93dff0700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545283, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8188> 2014-08-13 16:24:10.545363 7fd93dff0700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545330, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8187> 2014-08-13 16:24:10.545375 7fd93dff0700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545374, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8186> 2014-08-13 16:24:10.545502 7fd9367e4700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545501, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8185> 2014-08-13 16:24:10.545550 7fd9367e4700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545549, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8184> 2014-08-13 16:24:10.545566 7fd9367e4700 5 -- op tracker -- , seq: 735, time: 2014-08-13 16:24:10.545566, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8183> 2014-08-13 16:24:10.802350 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.801979) v2 ==== 47+0+0 (3092751294 0 0) 0x8c896c0 con 0x7eda940 -8182> 2014-08-13 16:24:10.802415 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.801979) v2 -- ?+0 0x775f540 con 0x7eda940 -8181> 2014-08-13 16:24:10.802359 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.801979) v2 ==== 47+0+0 (3092751294 0 0) 0x9100000 con 0x7edc8e0 -8180> 2014-08-13 16:24:10.802507 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.801979) v2 -- ?+0 0x77596c0 con 0x7edc8e0 -8179> 2014-08-13 16:24:10.853648 7fd94ac49700 5 osd.4 194449 tick -8178> 2014-08-13 16:24:10.866141 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 20 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8bc3700 con 0x4fe0000 -8177> 2014-08-13 16:24:10.866198 7fd93dff0700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.865976, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8176> 2014-08-13 16:24:10.866214 7fd93dff0700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.865980, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8175> 2014-08-13 16:24:10.866224 7fd93dff0700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866100, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8174> 2014-08-13 16:24:10.866234 7fd93dff0700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866192, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8173> 2014-08-13 16:24:10.866245 7fd93dff0700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866244, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8172> 2014-08-13 16:24:10.866359 7fd9371e5700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866358, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8171> 2014-08-13 16:24:10.866406 7fd9371e5700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866406, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8170> 2014-08-13 16:24:10.866426 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47300 con 0x4fe0000 -8169> 2014-08-13 16:24:10.866456 7fd9371e5700 5 -- op tracker -- , seq: 736, time: 2014-08-13 16:24:10.866456, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8168> 2014-08-13 16:24:10.867704 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 21 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x8bc2c00 con 0x4fe0000 -8167> 2014-08-13 16:24:10.867754 7fd93dff0700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867524, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8166> 2014-08-13 16:24:10.867770 7fd93dff0700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867527, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8165> 2014-08-13 16:24:10.867781 7fd93dff0700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867623, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8164> 2014-08-13 16:24:10.867790 7fd93dff0700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867748, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8163> 2014-08-13 16:24:10.867801 7fd93dff0700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867801, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8162> 2014-08-13 16:24:10.867879 7fd9367e4700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867878, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8161> 2014-08-13 16:24:10.867917 7fd9367e4700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867917, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8160> 2014-08-13 16:24:10.867932 7fd9367e4700 5 -- op tracker -- , seq: 737, time: 2014-08-13 16:24:10.867932, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8159> 2014-08-13 16:24:10.943520 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.943476) v2 ==== 47+0+0 (3171834145 0 0) 0x77596c0 con 0x7d2aec0 -8158> 2014-08-13 16:24:10.943591 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.943476) v2 -- ?+0 0x8c896c0 con 0x7d2aec0 -8157> 2014-08-13 16:24:10.943520 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:10.943476) v2 ==== 47+0+0 (3171834145 0 0) 0x91078c0 con 0x7d2c620 -8156> 2014-08-13 16:24:10.943704 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:10.943476) v2 -- ?+0 0x9100000 con 0x7d2c620 -8155> 2014-08-13 16:24:10.944531 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 17 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x7ece300 con 0x508bde0 -8154> 2014-08-13 16:24:10.944583 7fd93dff0700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944356, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8153> 2014-08-13 16:24:10.944600 7fd93dff0700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944358, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8152> 2014-08-13 16:24:10.944610 7fd93dff0700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944464, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8151> 2014-08-13 16:24:10.944619 7fd93dff0700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944577, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8150> 2014-08-13 16:24:10.944630 7fd93dff0700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944629, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8149> 2014-08-13 16:24:10.944717 7fd9371e5700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944717, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8148> 2014-08-13 16:24:10.944841 7fd9371e5700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944841, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8147> 2014-08-13 16:24:10.944867 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47080 con 0x508bde0 -8146> 2014-08-13 16:24:10.944898 7fd9371e5700 5 -- op tracker -- , seq: 738, time: 2014-08-13 16:24:10.944897, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8145> 2014-08-13 16:24:10.946288 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 18 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x8bc6300 con 0x508bde0 -8144> 2014-08-13 16:24:10.946343 7fd93dff0700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946100, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8143> 2014-08-13 16:24:10.946362 7fd93dff0700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946102, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8142> 2014-08-13 16:24:10.946373 7fd93dff0700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946209, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8141> 2014-08-13 16:24:10.946382 7fd93dff0700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946336, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8140> 2014-08-13 16:24:10.946397 7fd93dff0700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946397, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8139> 2014-08-13 16:24:10.946479 7fd9367e4700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946479, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8138> 2014-08-13 16:24:10.946514 7fd9367e4700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946514, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8137> 2014-08-13 16:24:10.946528 7fd9367e4700 5 -- op tracker -- , seq: 739, time: 2014-08-13 16:24:10.946528, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8136> 2014-08-13 16:24:10.991718 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 11 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1487161271 0 0) 0x863b700 con 0x50b9a20 -8135> 2014-08-13 16:24:10.991763 7fd93dff0700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991564, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8134> 2014-08-13 16:24:10.991779 7fd93dff0700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991568, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8133> 2014-08-13 16:24:10.991789 7fd93dff0700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991678, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8132> 2014-08-13 16:24:10.991798 7fd93dff0700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991758, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8131> 2014-08-13 16:24:10.991809 7fd93dff0700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991809, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8130> 2014-08-13 16:24:10.991910 7fd9371e5700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991910, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8129> 2014-08-13 16:24:10.991951 7fd9371e5700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.991951, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8128> 2014-08-13 16:24:10.991970 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op_reply(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46e00 con 0x50b9a20 -8127> 2014-08-13 16:24:10.992000 7fd9371e5700 5 -- op tracker -- , seq: 740, time: 2014-08-13 16:24:10.992000, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8126> 2014-08-13 16:24:10.993591 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 12 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3388314642 0 0) 0x8bc6e00 con 0x50b9a20 -8125> 2014-08-13 16:24:10.993640 7fd93dff0700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993450, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8124> 2014-08-13 16:24:10.993657 7fd93dff0700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993452, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8123> 2014-08-13 16:24:10.993667 7fd93dff0700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993554, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8122> 2014-08-13 16:24:10.993684 7fd93dff0700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993635, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8121> 2014-08-13 16:24:10.993696 7fd93dff0700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993696, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8120> 2014-08-13 16:24:10.993793 7fd9367e4700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993792, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8119> 2014-08-13 16:24:10.993833 7fd9367e4700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993833, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8118> 2014-08-13 16:24:10.993849 7fd9367e4700 5 -- op tracker -- , seq: 741, time: 2014-08-13 16:24:10.993848, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8117> 2014-08-13 16:24:11.014365 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 13 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x8638000 con 0x5111fa0 -8116> 2014-08-13 16:24:11.014421 7fd93dff0700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014194, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8115> 2014-08-13 16:24:11.014442 7fd93dff0700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014199, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8114> 2014-08-13 16:24:11.014456 7fd93dff0700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014313, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8113> 2014-08-13 16:24:11.014470 7fd93dff0700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014411, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8112> 2014-08-13 16:24:11.014484 7fd93dff0700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014484, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8111> 2014-08-13 16:24:11.014559 7fd9371e5700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014559, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8110> 2014-08-13 16:24:11.014597 7fd9371e5700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014596, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8109> 2014-08-13 16:24:11.014619 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46b80 con 0x5111fa0 -8108> 2014-08-13 16:24:11.014650 7fd9371e5700 5 -- op tracker -- , seq: 742, time: 2014-08-13 16:24:11.014650, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8107> 2014-08-13 16:24:11.016023 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 14 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x863c200 con 0x5111fa0 -8106> 2014-08-13 16:24:11.016075 7fd93dff0700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.015845, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8105> 2014-08-13 16:24:11.016095 7fd93dff0700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.015847, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8104> 2014-08-13 16:24:11.016109 7fd93dff0700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.015957, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8103> 2014-08-13 16:24:11.016122 7fd93dff0700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.016068, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8102> 2014-08-13 16:24:11.016136 7fd93dff0700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.016136, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8101> 2014-08-13 16:24:11.016221 7fd9367e4700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.016220, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8100> 2014-08-13 16:24:11.016256 7fd9367e4700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.016256, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8099> 2014-08-13 16:24:11.016273 7fd9367e4700 5 -- op tracker -- , seq: 743, time: 2014-08-13 16:24:11.016272, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -8098> 2014-08-13 16:24:11.017132 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.016975) v2 ==== 47+0+0 (3682897160 0 0) 0x8eef540 con 0x7ed82c0 -8097> 2014-08-13 16:24:11.017191 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.016975) v2 -- ?+0 0x91078c0 con 0x7ed82c0 -8096> 2014-08-13 16:24:11.017127 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.016975) v2 ==== 47+0+0 (3682897160 0 0) 0x775e900 con 0x7edb180 -8095> 2014-08-13 16:24:11.017258 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.016975) v2 -- ?+0 0x77596c0 con 0x7edb180 -8094> 2014-08-13 16:24:11.058211 7fd933fe0700 5 osd.4 194449 heartbeat: osd_stat(8087 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,7,10,11,15,17,20,21,24,25,28,31,32,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,396,397]/[] op hist []) -8093> 2014-08-13 16:24:11.058260 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899bb80 con 0x8c391e0 -8092> 2014-08-13 16:24:11.058291 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b9c0 con 0x8c38f20 -8091> 2014-08-13 16:24:11.058311 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b800 con 0x78844c0 -8090> 2014-08-13 16:24:11.058332 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b640 con 0x7884200 -8089> 2014-08-13 16:24:11.058354 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b480 con 0x78848e0 -8088> 2014-08-13 16:24:11.058374 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b2c0 con 0x7884620 -8087> 2014-08-13 16:24:11.058401 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6839/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102a00 con 0x8ba06e0 -8086> 2014-08-13 16:24:11.058422 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6846/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107e00 con 0x8c698c0 -8085> 2014-08-13 16:24:11.058438 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91055c0 con 0x794c360 -8084> 2014-08-13 16:24:11.058456 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9100540 con 0x794e300 -8083> 2014-08-13 16:24:11.058473 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9104b40 con 0x794e040 -8082> 2014-08-13 16:24:11.058499 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9100380 con 0x794dd80 -8081> 2014-08-13 16:24:11.058516 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91024c0 con 0x8c3a3c0 -8080> 2014-08-13 16:24:11.058533 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91001c0 con 0x8c3a100 -8079> 2014-08-13 16:24:11.058549 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102840 con 0x8c3c0a0 -8078> 2014-08-13 16:24:11.058572 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107a80 con 0x8c3bde0 -8077> 2014-08-13 16:24:11.058589 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102140 con 0x8babde0 -8076> 2014-08-13 16:24:11.058606 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107c40 con 0x8babb20 -8075> 2014-08-13 16:24:11.058623 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9104980 con 0x8c3d6a0 -8074> 2014-08-13 16:24:11.058644 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102300 con 0x8c3d3e0 -8073> 2014-08-13 16:24:11.058660 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91078c0 con 0x8badac0 -8072> 2014-08-13 16:24:11.058683 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d02140 con 0x8bad800 -8071> 2014-08-13 16:24:11.058700 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x775b2c0 con 0x8bab860 -8070> 2014-08-13 16:24:11.058720 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x77596c0 con 0x8bab5a0 -8069> 2014-08-13 16:24:11.058740 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x775a680 con 0x8c69600 -8068> 2014-08-13 16:24:11.058757 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x775d940 con 0x8c69340 -8067> 2014-08-13 16:24:11.058777 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x80d9500 con 0x8baee00 -8066> 2014-08-13 16:24:11.058803 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x80dbf00 con 0x8ba1a20 -8065> 2014-08-13 16:24:11.058826 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6814/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d063c0 con 0x8d21e40 -8064> 2014-08-13 16:24:11.058822 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c2d80 con 0x8c38f20 -8063> 2014-08-13 16:24:11.058845 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6831/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d039c0 con 0x8d21b80 -8062> 2014-08-13 16:24:11.058864 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d06ac0 con 0x7fba3c0 -8061> 2014-08-13 16:24:11.058883 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d00a80 con 0x7fba260 -8060> 2014-08-13 16:24:11.058904 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x7d02140 con 0x8bad540 -8059> 2014-08-13 16:24:11.058920 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c0700 con 0x8bad280 -8058> 2014-08-13 16:24:11.058938 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c0380 con 0x8c3cba0 -8057> 2014-08-13 16:24:11.058960 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c2140 con 0x8c3c8e0 -8056> 2014-08-13 16:24:11.058979 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c1a40 con 0x7fba100 -8055> 2014-08-13 16:24:11.058996 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c7700 con 0x7fb9fa0 -8054> 2014-08-13 16:24:11.059013 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c3800 con 0x8d223c0 -8053> 2014-08-13 16:24:11.059031 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c5240 con 0x8d22100 -8052> 2014-08-13 16:24:11.059050 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c7380 con 0x8c3c620 -8051> 2014-08-13 16:24:11.059065 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c1180 con 0x8c3c360 -8050> 2014-08-13 16:24:11.059085 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c5b00 con 0x8bafe80 -8049> 2014-08-13 16:24:11.059104 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c2f40 con 0x8bafbc0 -8048> 2014-08-13 16:24:11.059129 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c5cc0 con 0x8c38c60 -8047> 2014-08-13 16:24:11.059149 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c4b40 con 0x8c389a0 -8046> 2014-08-13 16:24:11.059178 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x775b2c0 con 0x8b68c60 -8045> 2014-08-13 16:24:11.059201 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899af40 con 0x8b68b00 -8044> 2014-08-13 16:24:11.059223 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899bb80 con 0x8c6fe80 -8043> 2014-08-13 16:24:11.059247 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b800 con 0x8c6fbc0 -8042> 2014-08-13 16:24:11.059265 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b640 con 0x8b689a0 -8041> 2014-08-13 16:24:11.059282 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b9c0 con 0x8b68840 -8040> 2014-08-13 16:24:11.059300 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b2c0 con 0x7fb9e40 -8039> 2014-08-13 16:24:11.059319 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b480 con 0x7fb9ce0 -8038> 2014-08-13 16:24:11.059336 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107540 con 0x8c3b9c0 -8037> 2014-08-13 16:24:11.059358 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9104d00 con 0x8c3b700 -8036> 2014-08-13 16:24:11.059376 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91055c0 con 0x7fbb020 -8035> 2014-08-13 16:24:11.059392 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9100540 con 0x7fbaec0 -8034> 2014-08-13 16:24:11.059410 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102a00 con 0x7fbb440 -8033> 2014-08-13 16:24:11.059427 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107e00 con 0x7fbb2e0 -8032> 2014-08-13 16:24:11.059443 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9100380 con 0x7ffd960 -8031> 2014-08-13 16:24:11.059461 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9104b40 con 0x7ffdc20 -8030> 2014-08-13 16:24:11.059477 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x91024c0 con 0x8d20840 -8029> 2014-08-13 16:24:11.059496 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102840 con 0x8d25960 -8028> 2014-08-13 16:24:11.059512 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107a80 con 0x8d218c0 -8027> 2014-08-13 16:24:11.059528 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9102140 con 0x8d21600 -8026> 2014-08-13 16:24:11.059546 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107c40 con 0x7fbaaa0 -8025> 2014-08-13 16:24:11.059565 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x81c5cc0 con 0x7fba940 -8024> 2014-08-13 16:24:11.059583 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x775c0c0 con 0x8d26e00 -8023> 2014-08-13 16:24:11.059605 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899dcc0 con 0x8d26b40 -8022> 2014-08-13 16:24:11.059628 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899de80 con 0x8d21340 -8021> 2014-08-13 16:24:11.059647 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899a840 con 0x8d21080 -8020> 2014-08-13 16:24:11.059669 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899bb80 con 0x7ffdee0 -8019> 2014-08-13 16:24:11.059687 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b800 con 0x7ffdd80 -8018> 2014-08-13 16:24:11.059703 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b640 con 0x7fb9a20 -8017> 2014-08-13 16:24:11.059720 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b9c0 con 0x7fb98c0 -8016> 2014-08-13 16:24:11.059739 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b480 con 0x8bab2e0 -8015> 2014-08-13 16:24:11.059768 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899b2c0 con 0x8bab020 -8014> 2014-08-13 16:24:11.059784 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8ee9dc0 con 0x8c69080 -8013> 2014-08-13 16:24:11.059801 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eea4c0 con 0x8c68dc0 -8012> 2014-08-13 16:24:11.059820 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eea680 con 0x8d256a0 -8011> 2014-08-13 16:24:11.059838 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeb480 con 0x8d253e0 -8010> 2014-08-13 16:24:11.059854 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeb2c0 con 0x8d20dc0 -8009> 2014-08-13 16:24:11.059877 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeb100 con 0x8d20b00 -8008> 2014-08-13 16:24:11.059892 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeaf40 con 0x8d25120 -8007> 2014-08-13 16:24:11.059919 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eead80 con 0x8d24e60 -8006> 2014-08-13 16:24:11.059934 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8ee9a40 con 0x8c3d120 -8005> 2014-08-13 16:24:11.059956 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8ee9c00 con 0x8c3ce60 -8004> 2014-08-13 16:24:11.059978 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eed240 con 0x7fbb180 -8003> 2014-08-13 16:24:11.059996 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8ee8fc0 con 0x7fbca40 -8002> 2014-08-13 16:24:11.060014 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107c40 con 0x7fbc780 -8001> 2014-08-13 16:24:11.060037 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9107a80 con 0x7fbc4c0 -8000> 2014-08-13 16:24:11.060068 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899f700 con 0x8c3aec0 -7999> 2014-08-13 16:24:11.060095 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899f540 con 0x8c3ac00 -7998> 2014-08-13 16:24:11.060117 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899f380 con 0x7fbad60 -7997> 2014-08-13 16:24:11.060138 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899f1c0 con 0x7fbac00 -7996> 2014-08-13 16:24:11.060166 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899f000 con 0x8c3e880 -7995> 2014-08-13 16:24:11.060186 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8998000 con 0x8c3e5c0 -7994> 2014-08-13 16:24:11.060203 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899e740 con 0x8c3a940 -7993> 2014-08-13 16:24:11.060226 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8998540 con 0x8c3a680 -7992> 2014-08-13 16:24:11.060242 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899ad80 con 0x8c69fa0 -7991> 2014-08-13 16:24:11.060257 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x89981c0 con 0x8c39fa0 -7990> 2014-08-13 16:24:11.060275 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899e3c0 con 0x7fb9760 -7989> 2014-08-13 16:24:11.060297 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899d940 con 0x7fb9600 -7988> 2014-08-13 16:24:11.060314 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899abc0 con 0x8c3e300 -7987> 2014-08-13 16:24:11.060329 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899e900 con 0x8c618c0 -7986> 2014-08-13 16:24:11.060348 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899e580 con 0x8bac780 -7985> 2014-08-13 16:24:11.060369 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8998380 con 0x8bac4c0 -7984> 2014-08-13 16:24:11.060385 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899d5c0 con 0x8bac200 -7983> 2014-08-13 16:24:11.060400 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899eac0 con 0x8babf40 -7982> 2014-08-13 16:24:11.060417 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899ee40 con 0x8d240a0 -7981> 2014-08-13 16:24:11.060432 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899d780 con 0x8d23de0 -7980> 2014-08-13 16:24:11.060457 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eead80 con 0x8ba1760 -7979> 2014-08-13 16:24:11.060476 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eee740 con 0x8ba14a0 -7978> 2014-08-13 16:24:11.060494 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeb2c0 con 0x8d26880 -7977> 2014-08-13 16:24:11.060510 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eeec80 con 0x8d265c0 -7976> 2014-08-13 16:24:11.060536 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283480 con 0x8baf900 -7975> 2014-08-13 16:24:11.060557 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92832c0 con 0x8baf640 -7974> 2014-08-13 16:24:11.060574 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283100 con 0x8bae040 -7973> 2014-08-13 16:24:11.060589 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282f40 con 0x8badd80 -7972> 2014-08-13 16:24:11.060607 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282d80 con 0x8d24620 -7971> 2014-08-13 16:24:11.060623 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282bc0 con 0x8d24360 -7970> 2014-08-13 16:24:11.060639 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282a00 con 0x8baf380 -7969> 2014-08-13 16:24:11.060656 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282840 con 0x8baf0c0 -7968> 2014-08-13 16:24:11.060674 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282680 con 0x8b6e1a0 -7967> 2014-08-13 16:24:11.060695 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92824c0 con 0x8b6e040 -7966> 2014-08-13 16:24:11.060715 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282300 con 0x8c39ce0 -7965> 2014-08-13 16:24:11.060731 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282140 con 0x8c39a20 -7964> 2014-08-13 16:24:11.060748 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281f80 con 0x8d27380 -7963> 2014-08-13 16:24:11.060766 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281dc0 con 0x8d270c0 -7962> 2014-08-13 16:24:11.060783 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281c00 con 0x8d23b20 -7961> 2014-08-13 16:24:11.060798 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281500 con 0x8d27bc0 -7960> 2014-08-13 16:24:11.060813 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281340 con 0x8b6dee0 -7959> 2014-08-13 16:24:11.060831 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281180 con 0x8b6dd80 -7958> 2014-08-13 16:24:11.060846 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9280fc0 con 0x7fba7e0 -7957> 2014-08-13 16:24:11.060865 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899eac0 con 0x7fba680 -7956> 2014-08-13 16:24:11.060880 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899d5c0 con 0x8c6f900 -7955> 2014-08-13 16:24:11.060898 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eee740 con 0x8c6f640 -7954> 2014-08-13 16:24:11.060919 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8eead80 con 0x8c68b00 -7953> 2014-08-13 16:24:11.060939 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9101180 con 0x8c68840 -7952> 2014-08-13 16:24:11.060959 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9101340 con 0x8c3b440 -7951> 2014-08-13 16:24:11.060980 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9285400 con 0x8c3b180 -7950> 2014-08-13 16:24:11.061001 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9285240 con 0x8c68580 -7949> 2014-08-13 16:24:11.061020 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9285080 con 0x8c6b2e0 -7948> 2014-08-13 16:24:11.061036 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284ec0 con 0x8c6b020 -7947> 2014-08-13 16:24:11.061055 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284d00 con 0x8c6ad60 -7946> 2014-08-13 16:24:11.061072 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284b40 con 0x8b6dc20 -7945> 2014-08-13 16:24:11.061089 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284980 con 0x7fb9b80 -7944> 2014-08-13 16:24:11.061107 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92847c0 con 0x8ba11e0 -7943> 2014-08-13 16:24:11.061127 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284600 con 0x8ba0f20 -7942> 2014-08-13 16:24:11.061143 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284440 con 0x7fbbc80 -7941> 2014-08-13 16:24:11.061166 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9284280 con 0x7fbbb20 -7940> 2014-08-13 16:24:11.061185 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92840c0 con 0x8c39760 -7939> 2014-08-13 16:24:11.061209 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283f00 con 0x8c394a0 -7938> 2014-08-13 16:24:11.061228 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283d40 con 0x7fbb9c0 -7937> 2014-08-13 16:24:11.061254 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283100 con 0x7fbb860 -7936> 2014-08-13 16:24:11.061273 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282bc0 con 0x7fb94a0 -7935> 2014-08-13 16:24:11.061290 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282d80 con 0x7fba520 -7934> 2014-08-13 16:24:11.061312 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282840 con 0x7fbc200 -7933> 2014-08-13 16:24:11.061329 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9282300 con 0x7fbbf40 -7932> 2014-08-13 16:24:11.061347 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281dc0 con 0x8c6aaa0 -7931> 2014-08-13 16:24:11.061366 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92824c0 con 0x8c6a7e0 -7930> 2014-08-13 16:24:11.061383 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281c00 con 0x8bacd00 -7929> 2014-08-13 16:24:11.061404 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281f80 con 0x8baca40 -7928> 2014-08-13 16:24:11.061421 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281500 con 0x8ba0c60 -7927> 2014-08-13 16:24:11.061444 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9281340 con 0x8ba09a0 -7926> 2014-08-13 16:24:11.061461 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9280fc0 con 0x7ffdac0 -7925> 2014-08-13 16:24:11.061478 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287e00 con 0x7ffe040 -7924> 2014-08-13 16:24:11.061500 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287c40 con 0x8c6f380 -7923> 2014-08-13 16:24:11.061519 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287a80 con 0x8c6f0c0 -7922> 2014-08-13 16:24:11.061540 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92878c0 con 0x7fbb700 -7921> 2014-08-13 16:24:11.061556 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287700 con 0x7fbb5a0 -7920> 2014-08-13 16:24:11.061573 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287540 con 0x8d27900 -7919> 2014-08-13 16:24:11.061604 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287380 con 0x8d27640 -7918> 2014-08-13 16:24:11.061621 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92871c0 con 0x8c6a520 -7917> 2014-08-13 16:24:11.061638 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9287000 con 0x8c6a260 -7916> 2014-08-13 16:24:11.061655 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9286e40 con 0x8d24ba0 -7915> 2014-08-13 16:24:11.061673 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9286c80 con 0x8d248e0 -7914> 2014-08-13 16:24:11.061691 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9286ac0 con 0x8d26300 -7913> 2014-08-13 16:24:11.061709 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x9283100 con 0x8d26040 -7912> 2014-08-13 16:24:11.061730 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x92824c0 con 0x8d25d80 -7911> 2014-08-13 16:24:11.061747 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x899a300 con 0x8d140a0 -7910> 2014-08-13 16:24:11.061764 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x8999880 con 0x8d13de0 -7909> 2014-08-13 16:24:11.061784 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:11.058257) v2 -- ?+0 0x87e2f40 con 0x8d13b20 -7908> 2014-08-13 16:24:11.062076 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775a680 con 0x794c360 -7907> 2014-08-13 16:24:11.062136 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c3f00 con 0x794e300 -7906> 2014-08-13 16:24:11.062165 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeaa00 con 0x794e040 -7905> 2014-08-13 16:24:11.062213 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899ec80 con 0x794dd80 -7904> 2014-08-13 16:24:11.062251 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899f700 con 0x7ffd960 -7903> 2014-08-13 16:24:11.062275 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9280e00 con 0x7ffdc20 -7902> 2014-08-13 16:24:11.062606 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e5cc0 con 0x7ffdac0 -7901> 2014-08-13 16:24:11.062642 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e71c0 con 0x7ffe040 -7900> 2014-08-13 16:24:11.062763 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92816c0 con 0x7ffdee0 -7899> 2014-08-13 16:24:11.062805 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9101c00 con 0x7ffdd80 -7898> 2014-08-13 16:24:11.062827 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eea4c0 con 0x8b68c60 -7897> 2014-08-13 16:24:11.062848 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee9180 con 0x8b68b00 -7896> 2014-08-13 16:24:11.062871 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c5cc0 con 0x8b689a0 -7895> 2014-08-13 16:24:11.062893 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899f540 con 0x8b68840 -7894> 2014-08-13 16:24:11.062916 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9284ec0 con 0x8b6e1a0 -7893> 2014-08-13 16:24:11.062941 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8999a40 con 0x8b6e040 -7892> 2014-08-13 16:24:11.062963 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286580 con 0x8b6dee0 -7891> 2014-08-13 16:24:11.062988 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9287a80 con 0x8b6dd80 -7890> 2014-08-13 16:24:11.063014 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e32c0 con 0x8b6dc20 -7889> 2014-08-13 16:24:11.063038 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92871c0 con 0x7fb9b80 -7888> 2014-08-13 16:24:11.063058 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281a40 con 0x7fb9a20 -7887> 2014-08-13 16:24:11.063080 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9280a80 con 0x7fb98c0 -7886> 2014-08-13 16:24:11.063104 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eee740 con 0x7fb9760 -7885> 2014-08-13 16:24:11.063128 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x91039c0 con 0x7fb9600 -7884> 2014-08-13 16:24:11.063155 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92878c0 con 0x7fb94a0 -7883> 2014-08-13 16:24:11.063190 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e6040 con 0x7fba520 -7882> 2014-08-13 16:24:11.063219 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x77596c0 con 0x7fba3c0 -7881> 2014-08-13 16:24:11.063244 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9107540 con 0x7fba260 -7880> 2014-08-13 16:24:11.063269 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eec0c0 con 0x7fba100 -7879> 2014-08-13 16:24:11.063294 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eec980 con 0x7fb9fa0 -7878> 2014-08-13 16:24:11.063317 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899fe00 con 0x7fb9e40 -7877> 2014-08-13 16:24:11.063340 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeb100 con 0x7fb9ce0 -7876> 2014-08-13 16:24:11.063364 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeaf40 con 0x7fbb020 -7875> 2014-08-13 16:24:11.063390 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899b480 con 0x7fbaec0 -7874> 2014-08-13 16:24:11.063413 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9103d40 con 0x7fbad60 -7873> 2014-08-13 16:24:11.063436 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9103480 con 0x7fbac00 -7872> 2014-08-13 16:24:11.063459 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9280000 con 0x7fbaaa0 -7871> 2014-08-13 16:24:11.063483 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eed240 con 0x7fba940 -7870> 2014-08-13 16:24:11.063508 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e0540 con 0x7fba7e0 -7869> 2014-08-13 16:24:11.063533 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281340 con 0x7fba680 -7868> 2014-08-13 16:24:11.063556 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e2f40 con 0x7fbbc80 -7867> 2014-08-13 16:24:11.063581 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e6ac0 con 0x7fbbb20 -7866> 2014-08-13 16:24:11.063604 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e3640 con 0x7fbb9c0 -7865> 2014-08-13 16:24:11.063627 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e47c0 con 0x7fbb860 -7864> 2014-08-13 16:24:11.063649 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e6740 con 0x7fbb700 -7863> 2014-08-13 16:24:11.063676 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e7c40 con 0x7fbb5a0 -7862> 2014-08-13 16:24:11.063700 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee9c00 con 0x7fbb440 -7861> 2014-08-13 16:24:11.063728 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899b640 con 0x7fbb2e0 -7860> 2014-08-13 16:24:11.063752 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282a00 con 0x7fbb180 -7859> 2014-08-13 16:24:11.063777 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9104440 con 0x7fbca40 -7858> 2014-08-13 16:24:11.063800 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92832c0 con 0x7fbc780 -7857> 2014-08-13 16:24:11.063824 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9283480 con 0x7fbc4c0 -7856> 2014-08-13 16:24:11.063846 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e39c0 con 0x7fbc200 -7855> 2014-08-13 16:24:11.063870 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e78c0 con 0x7fbbf40 -7854> 2014-08-13 16:24:11.063892 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775e3c0 con 0x8babde0 -7853> 2014-08-13 16:24:11.063917 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c2f40 con 0x8babb20 -7852> 2014-08-13 16:24:11.063939 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x80d9500 con 0x8bab860 -7851> 2014-08-13 16:24:11.063962 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9104b40 con 0x8bab5a0 -7850> 2014-08-13 16:24:11.063985 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899d780 con 0x8bab2e0 -7849> 2014-08-13 16:24:11.064009 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee8fc0 con 0x8bab020 -7848> 2014-08-13 16:24:11.064033 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e55c0 con 0x8bacd00 -7847> 2014-08-13 16:24:11.064057 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e3d40 con 0x8baca40 -7846> 2014-08-13 16:24:11.064080 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9284d00 con 0x8bac780 -7845> 2014-08-13 16:24:11.064105 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282d80 con 0x8bac4c0 -7844> 2014-08-13 16:24:11.064128 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9285e80 con 0x8bac200 -7843> 2014-08-13 16:24:11.064153 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9285080 con 0x8babf40 -7842> 2014-08-13 16:24:11.064186 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92847c0 con 0x8bae040 -7841> 2014-08-13 16:24:11.064215 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9285780 con 0x8badd80 -7840> 2014-08-13 16:24:11.064239 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c5b00 con 0x8badac0 -7839> 2014-08-13 16:24:11.064266 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c2140 con 0x8bad800 -7838> 2014-08-13 16:24:11.064290 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee9880 con 0x8bad540 -7837> 2014-08-13 16:24:11.064315 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899e200 con 0x8bad280 -7836> 2014-08-13 16:24:11.064339 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee8e00 con 0x8bafe80 -7835> 2014-08-13 16:24:11.064363 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eea680 con 0x8bafbc0 -7834> 2014-08-13 16:24:11.064386 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eead80 con 0x8baf900 -7833> 2014-08-13 16:24:11.064410 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e01c0 con 0x8baf640 -7832> 2014-08-13 16:24:11.064433 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9284440 con 0x8baf380 -7831> 2014-08-13 16:24:11.064457 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9106040 con 0x8baf0c0 -7830> 2014-08-13 16:24:11.064479 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775d780 con 0x8baee00 -7829> 2014-08-13 16:24:11.064505 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c4b40 con 0x8ba1a20 -7828> 2014-08-13 16:24:11.064528 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9101340 con 0x8ba1760 -7827> 2014-08-13 16:24:11.064551 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9102d80 con 0x8ba14a0 -7826> 2014-08-13 16:24:11.064574 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e4ec0 con 0x8ba11e0 -7825> 2014-08-13 16:24:11.064600 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e40c0 con 0x8ba0f20 -7824> 2014-08-13 16:24:11.064622 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e4440 con 0x8ba0c60 -7823> 2014-08-13 16:24:11.064646 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899a300 con 0x8ba09a0 -7822> 2014-08-13 16:24:11.064668 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.1.115:6839/62713 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x91078c0 con 0x8ba06e0 -7821> 2014-08-13 16:24:11.064692 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.100.115:6846/62713 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x80da4c0 con 0x8c698c0 -7820> 2014-08-13 16:24:11.064715 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9104980 con 0x8c69600 -7819> 2014-08-13 16:24:11.064738 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899af40 con 0x8c69340 -7818> 2014-08-13 16:24:11.064763 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeb2c0 con 0x8c69080 -7817> 2014-08-13 16:24:11.064787 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899d940 con 0x8c68dc0 -7816> 2014-08-13 16:24:11.064814 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281f80 con 0x8c68b00 -7815> 2014-08-13 16:24:11.064839 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x89988c0 con 0x8c68840 -7814> 2014-08-13 16:24:11.064862 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9287c40 con 0x8c68580 -7813> 2014-08-13 16:24:11.064888 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9287380 con 0x8c6b2e0 -7812> 2014-08-13 16:24:11.064912 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286ac0 con 0x8c6b020 -7811> 2014-08-13 16:24:11.064935 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899d240 con 0x8c6ad60 -7810> 2014-08-13 16:24:11.064958 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e5940 con 0x8c6aaa0 -7809> 2014-08-13 16:24:11.064982 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e4b40 con 0x8c6a7e0 -7808> 2014-08-13 16:24:11.065004 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8910e00 con 0x8c6a520 -7807> 2014-08-13 16:24:11.065027 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8910a80 con 0x8c6a260 -7806> 2014-08-13 16:24:11.065050 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9285400 con 0x8c69fa0 -7805> 2014-08-13 16:24:11.065074 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899d5c0 con 0x8c39fa0 -7804> 2014-08-13 16:24:11.065097 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282300 con 0x8c39ce0 -7803> 2014-08-13 16:24:11.065120 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281c00 con 0x8c39a20 -7802> 2014-08-13 16:24:11.065142 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286e40 con 0x8c39760 -7801> 2014-08-13 16:24:11.065167 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281dc0 con 0x8c394a0 -7800> 2014-08-13 16:24:11.065200 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775d940 con 0x8c391e0 -7799> 2014-08-13 16:24:11.065225 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9102a00 con 0x8c38c60 -7798> 2014-08-13 16:24:11.065248 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eed080 con 0x8c389a0 -7797> 2014-08-13 16:24:11.065270 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899b9c0 con 0x8c3b9c0 -7796> 2014-08-13 16:24:11.065295 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee9dc0 con 0x8c3b700 -7795> 2014-08-13 16:24:11.065318 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9287700 con 0x8c3b440 -7794> 2014-08-13 16:24:11.065345 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286c80 con 0x8c3b180 -7793> 2014-08-13 16:24:11.065368 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x91047c0 con 0x8c3aec0 -7792> 2014-08-13 16:24:11.065396 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899ad80 con 0x8c3ac00 -7791> 2014-08-13 16:24:11.065419 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282140 con 0x8c3a940 -7790> 2014-08-13 16:24:11.065442 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9101180 con 0x8c3a680 -7789> 2014-08-13 16:24:11.065467 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x7d063c0 con 0x8c3a3c0 -7788> 2014-08-13 16:24:11.065492 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x7759c00 con 0x8c3a100 -7787> 2014-08-13 16:24:11.065514 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775bd40 con 0x8c3d6a0 -7786> 2014-08-13 16:24:11.065537 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c1a40 con 0x8c3d3e0 -7785> 2014-08-13 16:24:11.065560 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x91008c0 con 0x8c3d120 -7784> 2014-08-13 16:24:11.065583 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9283b80 con 0x8c3ce60 -7783> 2014-08-13 16:24:11.065605 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775b2c0 con 0x8c3cba0 -7782> 2014-08-13 16:24:11.065628 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeb480 con 0x8c3c8e0 -7781> 2014-08-13 16:24:11.065650 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899de80 con 0x8c3c620 -7780> 2014-08-13 16:24:11.065674 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9100540 con 0x8c3c360 -7779> 2014-08-13 16:24:11.065696 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775e040 con 0x8c3c0a0 -7778> 2014-08-13 16:24:11.065721 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c7700 con 0x8c3bde0 -7777> 2014-08-13 16:24:11.065746 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeec80 con 0x8c3e880 -7776> 2014-08-13 16:24:11.065769 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9283800 con 0x8c3e5c0 -7775> 2014-08-13 16:24:11.065792 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9103640 con 0x8c3e300 -7774> 2014-08-13 16:24:11.065816 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8999dc0 con 0x8c618c0 -7773> 2014-08-13 16:24:11.065838 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eecec0 con 0x8c6fe80 -7772> 2014-08-13 16:24:11.065861 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899fa80 con 0x8c6fbc0 -7771> 2014-08-13 16:24:11.065884 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x89101c0 con 0x8c6f900 -7770> 2014-08-13 16:24:11.065908 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9283100 con 0x8c6f640 -7769> 2014-08-13 16:24:11.065934 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8910380 con 0x8c6f380 -7768> 2014-08-13 16:24:11.065959 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e6e40 con 0x8c6f0c0 -7767> 2014-08-13 16:24:11.065985 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eec440 con 0x8d223c0 -7766> 2014-08-13 16:24:11.066009 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899a840 con 0x8d22100 -7765> 2014-08-13 16:24:11.066032 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.1.115:6814/59915 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eeabc0 con 0x8d21e40 -7764> 2014-08-13 16:24:11.066057 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.100.115:6831/59915 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c1180 con 0x8d21b80 -7763> 2014-08-13 16:24:11.066081 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9280380 con 0x8d218c0 -7762> 2014-08-13 16:24:11.066104 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92808c0 con 0x8d21600 -7761> 2014-08-13 16:24:11.066127 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899ee40 con 0x8d21340 -7760> 2014-08-13 16:24:11.066157 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899e740 con 0x8d21080 -7759> 2014-08-13 16:24:11.066188 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8998380 con 0x8d20dc0 -7758> 2014-08-13 16:24:11.066213 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9101a40 con 0x8d20b00 -7757> 2014-08-13 16:24:11.066236 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee9a40 con 0x8d20840 -7756> 2014-08-13 16:24:11.066259 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8998000 con 0x8d25960 -7755> 2014-08-13 16:24:11.066281 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9106ac0 con 0x8d256a0 -7754> 2014-08-13 16:24:11.066304 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eede80 con 0x8d253e0 -7753> 2014-08-13 16:24:11.066326 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282840 con 0x8d25120 -7752> 2014-08-13 16:24:11.066349 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92840c0 con 0x8d24e60 -7751> 2014-08-13 16:24:11.066374 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8910700 con 0x8d24ba0 -7750> 2014-08-13 16:24:11.066399 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e7540 con 0x8d248e0 -7749> 2014-08-13 16:24:11.066422 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286900 con 0x8d24620 -7748> 2014-08-13 16:24:11.066445 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9284280 con 0x8d24360 -7747> 2014-08-13 16:24:11.066468 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9105940 con 0x8d240a0 -7746> 2014-08-13 16:24:11.066492 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9285b00 con 0x8d23de0 -7745> 2014-08-13 16:24:11.066514 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8999340 con 0x8d23b20 -7744> 2014-08-13 16:24:11.066537 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282bc0 con 0x8d27bc0 -7743> 2014-08-13 16:24:11.066563 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e5240 con 0x8d27900 -7742> 2014-08-13 16:24:11.066588 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8999880 con 0x8d27640 -7741> 2014-08-13 16:24:11.066614 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x899a680 con 0x8d27380 -7740> 2014-08-13 16:24:11.066637 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9286200 con 0x8d270c0 -7739> 2014-08-13 16:24:11.066660 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9282f40 con 0x8d26e00 -7738> 2014-08-13 16:24:11.066685 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x92801c0 con 0x8d26b40 -7737> 2014-08-13 16:24:11.066708 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9281180 con 0x8d26880 -7736> 2014-08-13 16:24:11.066732 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9284b40 con 0x8d265c0 -7735> 2014-08-13 16:24:11.066754 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x87e63c0 con 0x8d26300 -7734> 2014-08-13 16:24:11.066777 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9107700 con 0x8d26040 -7733> 2014-08-13 16:24:11.066801 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8eea300 con 0x8d25d80 -7732> 2014-08-13 16:24:11.066826 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c2d80 con 0x8d140a0 -7731> 2014-08-13 16:24:11.066848 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x775b480 con 0x8d13de0 -7730> 2014-08-13 16:24:11.066871 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x9105400 con 0x8d13b20 -7729> 2014-08-13 16:24:11.066894 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x7d06ac0 con 0x78844c0 -7728> 2014-08-13 16:24:11.066918 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x7d02140 con 0x7884200 -7727> 2014-08-13 16:24:11.066940 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x81c0700 con 0x78848e0 -7726> 2014-08-13 16:24:11.066964 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 2 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.058257) v2 ==== 47+0+0 (1153032521 0 0) 0x8ee8000 con 0x7884620 -7725> 2014-08-13 16:24:11.088026 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.087770) v2 ==== 47+0+0 (2224376954 0 0) 0x7759c00 con 0x7eb8f20 -7724> 2014-08-13 16:24:11.088037 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.087770) v2 ==== 47+0+0 (2224376954 0 0) 0x80da4c0 con 0x7eba3c0 -7723> 2014-08-13 16:24:11.088068 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.087770) v2 -- ?+0 0x775e900 con 0x7eb8f20 -7722> 2014-08-13 16:24:11.088125 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.087770) v2 -- ?+0 0x8eef540 con 0x7eba3c0 -7721> 2014-08-13 16:24:11.141865 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.141398) v2 ==== 47+0+0 (82722871 0 0) 0x775f8c0 con 0x7eba100 -7720> 2014-08-13 16:24:11.141868 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.141398) v2 ==== 47+0+0 (82722871 0 0) 0x7d03480 con 0x7eb8c60 -7719> 2014-08-13 16:24:11.141926 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.141398) v2 -- ?+0 0x80da4c0 con 0x7eba100 -7718> 2014-08-13 16:24:11.142020 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.141398) v2 -- ?+0 0x7759c00 con 0x7eb8c60 -7717> 2014-08-13 16:24:11.156135 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.155788) v2 ==== 47+0+0 (1196258337 0 0) 0x7d04b40 con 0x7cae880 -7716> 2014-08-13 16:24:11.156172 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.155788) v2 ==== 47+0+0 (1196258337 0 0) 0x7d01a40 con 0x7cae040 -7715> 2014-08-13 16:24:11.156199 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.155788) v2 -- ?+0 0x7d03480 con 0x7cae880 -7714> 2014-08-13 16:24:11.156274 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.155788) v2 -- ?+0 0x775f8c0 con 0x7cae040 -7713> 2014-08-13 16:24:11.288847 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.288256) v2 ==== 47+0+0 (4054854351 0 0) 0x8c89180 con 0x7d2ac00 -7712> 2014-08-13 16:24:11.288875 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.288256) v2 ==== 47+0+0 (4054854351 0 0) 0x8c8f000 con 0x7d2c360 -7711> 2014-08-13 16:24:11.288899 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.288256) v2 -- ?+0 0x7d04b40 con 0x7d2ac00 -7710> 2014-08-13 16:24:11.288948 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.288256) v2 -- ?+0 0x7d01a40 con 0x7d2c360 -7709> 2014-08-13 16:24:11.644149 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.644160) v2 ==== 47+0+0 (308460981 0 0) 0x7d00fc0 con 0x7ebd120 -7708> 2014-08-13 16:24:11.644188 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.644160) v2 ==== 47+0+0 (308460981 0 0) 0x8c88a80 con 0x7ebbb20 -7707> 2014-08-13 16:24:11.644215 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.644160) v2 -- ?+0 0x8c8f000 con 0x7ebd120 -7706> 2014-08-13 16:24:11.644277 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.644160) v2 -- ?+0 0x8c89180 con 0x7ebbb20 -7705> 2014-08-13 16:24:11.734774 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.734610) v2 ==== 47+0+0 (935880629 0 0) 0x7d07a80 con 0x7edbb20 -7704> 2014-08-13 16:24:11.734821 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.734610) v2 -- ?+0 0x7d00fc0 con 0x7edbb20 -7703> 2014-08-13 16:24:11.734821 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.734610) v2 ==== 47+0+0 (935880629 0 0) 0x7d06c80 con 0x7ffe880 -7702> 2014-08-13 16:24:11.734892 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.734610) v2 -- ?+0 0x8c88a80 con 0x7ffe880 -7701> 2014-08-13 16:24:11.853799 7fd94ac49700 5 osd.4 194449 tick -7700> 2014-08-13 16:24:11.866486 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 22 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8bc1600 con 0x4fe0000 -7699> 2014-08-13 16:24:11.866539 7fd93dff0700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866259, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7698> 2014-08-13 16:24:11.866556 7fd93dff0700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866263, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7697> 2014-08-13 16:24:11.866566 7fd93dff0700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866402, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7696> 2014-08-13 16:24:11.866580 7fd93dff0700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866532, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7695> 2014-08-13 16:24:11.866596 7fd93dff0700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866596, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7694> 2014-08-13 16:24:11.866708 7fd9371e5700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866708, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7693> 2014-08-13 16:24:11.866751 7fd9371e5700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866751, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7692> 2014-08-13 16:24:11.866771 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46900 con 0x4fe0000 -7691> 2014-08-13 16:24:11.866800 7fd9371e5700 5 -- op tracker -- , seq: 744, time: 2014-08-13 16:24:11.866799, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7690> 2014-08-13 16:24:11.868411 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 23 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x8bc6e00 con 0x4fe0000 -7689> 2014-08-13 16:24:11.868461 7fd93dff0700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868210, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7688> 2014-08-13 16:24:11.868477 7fd93dff0700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868213, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7687> 2014-08-13 16:24:11.868487 7fd93dff0700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868327, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7686> 2014-08-13 16:24:11.868511 7fd93dff0700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868455, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7685> 2014-08-13 16:24:11.868527 7fd93dff0700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868526, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7684> 2014-08-13 16:24:11.868628 7fd9367e4700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868627, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7683> 2014-08-13 16:24:11.868669 7fd9367e4700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868668, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7682> 2014-08-13 16:24:11.868694 7fd9367e4700 5 -- op tracker -- , seq: 745, time: 2014-08-13 16:24:11.868694, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7681> 2014-08-13 16:24:11.911992 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.911861) v2 ==== 47+0+0 (4138566259 0 0) 0x775e900 con 0x7edac00 -7680> 2014-08-13 16:24:11.912037 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.911861) v2 -- ?+0 0x7d06c80 con 0x7edac00 -7679> 2014-08-13 16:24:11.912010 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.911861) v2 ==== 47+0+0 (4138566259 0 0) 0x775e040 con 0x7edcba0 -7678> 2014-08-13 16:24:11.912075 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.911861) v2 -- ?+0 0x7d07a80 con 0x7edcba0 -7677> 2014-08-13 16:24:11.924885 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.924501) v2 ==== 47+0+0 (2335377031 0 0) 0x8eebd40 con 0x7d29ce0 -7676> 2014-08-13 16:24:11.924933 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.924501) v2 -- ?+0 0x775e040 con 0x7d29ce0 -7675> 2014-08-13 16:24:11.924937 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.924501) v2 ==== 47+0+0 (2335377031 0 0) 0x8ee8380 con 0x7d2b440 -7674> 2014-08-13 16:24:11.924988 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.924501) v2 -- ?+0 0x775e900 con 0x7d2b440 -7673> 2014-08-13 16:24:11.933965 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.933645) v2 ==== 47+0+0 (2028037584 0 0) 0x775e3c0 con 0x7caeb40 -7672> 2014-08-13 16:24:11.933991 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.933645) v2 ==== 47+0+0 (2028037584 0 0) 0x81c1a40 con 0x7cae300 -7671> 2014-08-13 16:24:11.934018 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.933645) v2 -- ?+0 0x8ee8380 con 0x7caeb40 -7670> 2014-08-13 16:24:11.934119 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.933645) v2 -- ?+0 0x8eebd40 con 0x7cae300 -7669> 2014-08-13 16:24:11.944211 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.944074) v2 ==== 47+0+0 (450426348 0 0) 0x81c2140 con 0x7ed8dc0 -7668> 2014-08-13 16:24:11.944262 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.944074) v2 -- ?+0 0x81c1a40 con 0x7ed8dc0 -7667> 2014-08-13 16:24:11.944254 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:11.944074) v2 ==== 47+0+0 (450426348 0 0) 0x81c5cc0 con 0x7ebee00 -7666> 2014-08-13 16:24:11.944306 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:11.944074) v2 -- ?+0 0x775e3c0 con 0x7ebee00 -7665> 2014-08-13 16:24:12.003831 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.003824) v2 ==== 47+0+0 (3060229482 0 0) 0x775e740 con 0x7ebf0c0 -7664> 2014-08-13 16:24:12.003882 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.003824) v2 -- ?+0 0x81c5cc0 con 0x7ebf0c0 -7663> 2014-08-13 16:24:12.003864 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.003824) v2 ==== 47+0+0 (3060229482 0 0) 0x775af40 con 0x7ed9080 -7662> 2014-08-13 16:24:12.003936 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.003824) v2 -- ?+0 0x81c2140 con 0x7ed9080 -7661> 2014-08-13 16:24:12.013721 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 9 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x8bc2c00 con 0x510fd20 -7660> 2014-08-13 16:24:12.013773 7fd93dff0700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013523, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7659> 2014-08-13 16:24:12.013789 7fd93dff0700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013527, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7658> 2014-08-13 16:24:12.013799 7fd93dff0700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013638, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7657> 2014-08-13 16:24:12.013819 7fd93dff0700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013767, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7656> 2014-08-13 16:24:12.013846 7fd93dff0700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013846, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7655> 2014-08-13 16:24:12.013949 7fd9371e5700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013949, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7654> 2014-08-13 16:24:12.013990 7fd9371e5700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.013990, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7653> 2014-08-13 16:24:12.014009 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46680 con 0x510fd20 -7652> 2014-08-13 16:24:12.014038 7fd9371e5700 5 -- op tracker -- , seq: 746, time: 2014-08-13 16:24:12.014038, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7651> 2014-08-13 16:24:12.014567 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.014212) v2 ==== 47+0+0 (4178623422 0 0) 0x7758000 con 0x7eda680 -7650> 2014-08-13 16:24:12.014623 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.014212) v2 -- ?+0 0x775e740 con 0x7eda680 -7649> 2014-08-13 16:24:12.014610 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.014212) v2 ==== 47+0+0 (4178623422 0 0) 0x7d03640 con 0x7edc620 -7648> 2014-08-13 16:24:12.014721 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.014212) v2 -- ?+0 0x775af40 con 0x7edc620 -7647> 2014-08-13 16:24:12.015386 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 10 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x863c200 con 0x510fd20 -7646> 2014-08-13 16:24:12.015437 7fd93dff0700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015199, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7645> 2014-08-13 16:24:12.015453 7fd93dff0700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015202, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7644> 2014-08-13 16:24:12.015463 7fd93dff0700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015303, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7643> 2014-08-13 16:24:12.015473 7fd93dff0700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015431, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7642> 2014-08-13 16:24:12.015484 7fd93dff0700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015484, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7641> 2014-08-13 16:24:12.015617 7fd9367e4700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015617, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7640> 2014-08-13 16:24:12.015663 7fd9367e4700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015663, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7639> 2014-08-13 16:24:12.015679 7fd9367e4700 5 -- op tracker -- , seq: 747, time: 2014-08-13 16:24:12.015679, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7638> 2014-08-13 16:24:12.087689 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.087103) v2 ==== 47+0+0 (3961641926 0 0) 0x8c8db00 con 0x7ebf380 -7637> 2014-08-13 16:24:12.087745 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.087103) v2 -- ?+0 0x7758000 con 0x7ebf380 -7636> 2014-08-13 16:24:12.087770 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.087103) v2 ==== 47+0+0 (3961641926 0 0) 0x8c881c0 con 0x7ebfe80 -7635> 2014-08-13 16:24:12.087813 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.087103) v2 -- ?+0 0x7d03640 con 0x7ebfe80 -7634> 2014-08-13 16:24:12.125646 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.125479) v2 ==== 47+0+0 (58286332 0 0) 0x87e4b40 con 0x7edbf40 -7633> 2014-08-13 16:24:12.125672 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.125479) v2 ==== 47+0+0 (58286332 0 0) 0x87e7540 con 0x7ffeca0 -7632> 2014-08-13 16:24:12.125698 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.125479) v2 -- ?+0 0x8c881c0 con 0x7edbf40 -7631> 2014-08-13 16:24:12.125805 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.125479) v2 -- ?+0 0x8c8db00 con 0x7ffeca0 -7630> 2014-08-13 16:24:12.188934 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.188789) v2 ==== 47+0+0 (458584338 0 0) 0x8c8db00 con 0x7d2cba0 -7629> 2014-08-13 16:24:12.188965 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.188789) v2 ==== 47+0+0 (458584338 0 0) 0x87e4ec0 con 0x7d28420 -7628> 2014-08-13 16:24:12.188987 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.188789) v2 -- ?+0 0x87e4b40 con 0x7d2cba0 -7627> 2014-08-13 16:24:12.189030 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.188789) v2 -- ?+0 0x87e7540 con 0x7d28420 -7626> 2014-08-13 16:24:12.239719 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.239790) v2 ==== 47+0+0 (1747358323 0 0) 0x775c440 con 0x7d286e0 -7625> 2014-08-13 16:24:12.239774 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.239790) v2 -- ?+0 0x87e4ec0 con 0x7d286e0 -7624> 2014-08-13 16:24:12.239754 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.239790) v2 ==== 47+0+0 (1747358323 0 0) 0x7d04d00 con 0x7caf0c0 -7623> 2014-08-13 16:24:12.239848 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.239790) v2 -- ?+0 0x8c8db00 con 0x7caf0c0 -7622> 2014-08-13 16:24:12.283033 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.282983) v2 ==== 47+0+0 (3244342424 0 0) 0x87e4440 con 0x7ffeb40 -7621> 2014-08-13 16:24:12.283060 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.282983) v2 ==== 47+0+0 (3244342424 0 0) 0x87e39c0 con 0x7edbde0 -7620> 2014-08-13 16:24:12.283087 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.282983) v2 -- ?+0 0x775c440 con 0x7ffeb40 -7619> 2014-08-13 16:24:12.283128 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.282983) v2 -- ?+0 0x7d04d00 con 0x7edbde0 -7618> 2014-08-13 16:24:12.308767 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.308772) v2 ==== 47+0+0 (1242695327 0 0) 0x775fc40 con 0x7d2a260 -7617> 2014-08-13 16:24:12.308815 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.308772) v2 -- ?+0 0x87e39c0 con 0x7d2a260 -7616> 2014-08-13 16:24:12.308800 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.308772) v2 ==== 47+0+0 (1242695327 0 0) 0x87e55c0 con 0x7d28160 -7615> 2014-08-13 16:24:12.308992 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.308772) v2 -- ?+0 0x87e4440 con 0x7d28160 -7614> 2014-08-13 16:24:12.337188 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.337374) v2 ==== 47+0+0 (885358902 0 0) 0x8c8f1c0 con 0x7d2b700 -7613> 2014-08-13 16:24:12.337237 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.337374) v2 -- ?+0 0x87e55c0 con 0x7d2b700 -7612> 2014-08-13 16:24:12.337315 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.337374) v2 ==== 47+0+0 (885358902 0 0) 0x8c8c280 con 0x7d29fa0 -7611> 2014-08-13 16:24:12.337360 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.337374) v2 -- ?+0 0x775fc40 con 0x7d29fa0 -7610> 2014-08-13 16:24:12.441419 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.440852) v2 ==== 47+0+0 (3679334976 0 0) 0x87e6040 con 0x7ebc620 -7609> 2014-08-13 16:24:12.441433 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.440852) v2 ==== 47+0+0 (3679334976 0 0) 0x87e2f40 con 0x7eb9b80 -7608> 2014-08-13 16:24:12.441482 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.440852) v2 -- ?+0 0x8c8f1c0 con 0x7ebc620 -7607> 2014-08-13 16:24:12.441577 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.440852) v2 -- ?+0 0x8c8c280 con 0x7eb9b80 -7606> 2014-08-13 16:24:12.544246 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 19 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x8bc4200 con 0x5112520 -7605> 2014-08-13 16:24:12.544300 7fd93dff0700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544018, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7604> 2014-08-13 16:24:12.544316 7fd93dff0700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544022, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7603> 2014-08-13 16:24:12.544326 7fd93dff0700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544163, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7602> 2014-08-13 16:24:12.544336 7fd93dff0700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544292, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7601> 2014-08-13 16:24:12.544352 7fd93dff0700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544351, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7600> 2014-08-13 16:24:12.544471 7fd9371e5700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544471, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7599> 2014-08-13 16:24:12.544518 7fd9371e5700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544518, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7598> 2014-08-13 16:24:12.544538 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8df00 con 0x5112520 -7597> 2014-08-13 16:24:12.544569 7fd9371e5700 5 -- op tracker -- , seq: 748, time: 2014-08-13 16:24:12.544568, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7596> 2014-08-13 16:24:12.545987 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 20 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x8bc2100 con 0x5112520 -7595> 2014-08-13 16:24:12.546038 7fd93dff0700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.545807, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7594> 2014-08-13 16:24:12.546054 7fd93dff0700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.545810, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7593> 2014-08-13 16:24:12.546064 7fd93dff0700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.545905, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7592> 2014-08-13 16:24:12.546074 7fd93dff0700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.546032, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7591> 2014-08-13 16:24:12.546095 7fd93dff0700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.546095, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7590> 2014-08-13 16:24:12.546201 7fd9367e4700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.546200, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7589> 2014-08-13 16:24:12.546242 7fd9367e4700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.546241, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7588> 2014-08-13 16:24:12.546258 7fd9367e4700 5 -- op tracker -- , seq: 749, time: 2014-08-13 16:24:12.546258, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7587> 2014-08-13 16:24:12.645377 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.645372) v2 ==== 47+0+0 (1215305724 0 0) 0x8c8cd00 con 0x7d2aec0 -7586> 2014-08-13 16:24:12.645438 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.645372) v2 -- ?+0 0x87e6040 con 0x7d2aec0 -7585> 2014-08-13 16:24:12.645468 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.645372) v2 ==== 47+0+0 (1215305724 0 0) 0x9102bc0 con 0x7d2c620 -7584> 2014-08-13 16:24:12.645512 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.645372) v2 -- ?+0 0x87e2f40 con 0x7d2c620 -7583> 2014-08-13 16:24:12.696052 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 11 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x863ac00 con 0x4e498c0 -7582> 2014-08-13 16:24:12.696103 7fd93dff0700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.695854, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7581> 2014-08-13 16:24:12.696119 7fd93dff0700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.695858, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7580> 2014-08-13 16:24:12.696129 7fd93dff0700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.695968, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7579> 2014-08-13 16:24:12.696138 7fd93dff0700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.696096, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7578> 2014-08-13 16:24:12.696149 7fd93dff0700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.696148, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7577> 2014-08-13 16:24:12.696263 7fd9371e5700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.696263, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7576> 2014-08-13 16:24:12.696304 7fd9371e5700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.696304, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7575> 2014-08-13 16:24:12.696323 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8e400 con 0x4e498c0 -7574> 2014-08-13 16:24:12.696355 7fd9371e5700 5 -- op tracker -- , seq: 750, time: 2014-08-13 16:24:12.696355, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7573> 2014-08-13 16:24:12.697755 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 12 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x8638000 con 0x4e498c0 -7572> 2014-08-13 16:24:12.697812 7fd93dff0700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.697557, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7571> 2014-08-13 16:24:12.697829 7fd93dff0700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.697560, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7570> 2014-08-13 16:24:12.697839 7fd93dff0700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.697674, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7569> 2014-08-13 16:24:12.697849 7fd93dff0700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.697805, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7568> 2014-08-13 16:24:12.697864 7fd93dff0700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.697864, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7567> 2014-08-13 16:24:12.698001 7fd9367e4700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.698001, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7566> 2014-08-13 16:24:12.698055 7fd9367e4700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.698055, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7565> 2014-08-13 16:24:12.698071 7fd9367e4700 5 -- op tracker -- , seq: 751, time: 2014-08-13 16:24:12.698071, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7564> 2014-08-13 16:24:12.742451 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.742339) v2 ==== 47+0+0 (1355046551 0 0) 0x8c8c280 con 0x7d286e0 -7563> 2014-08-13 16:24:12.742497 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.742339) v2 -- ?+0 0x8c8cd00 con 0x7d286e0 -7562> 2014-08-13 16:24:12.742474 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.742339) v2 ==== 47+0+0 (1355046551 0 0) 0x7d04600 con 0x7caf0c0 -7561> 2014-08-13 16:24:12.742540 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.742339) v2 -- ?+0 0x9102bc0 con 0x7caf0c0 -7560> 2014-08-13 16:24:12.839647 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.839790) v2 ==== 47+0+0 (2674337944 0 0) 0x87e6040 con 0x7d2b700 -7559> 2014-08-13 16:24:12.839693 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.839790) v2 -- ?+0 0x8c8c280 con 0x7d2b700 -7558> 2014-08-13 16:24:12.839739 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.839790) v2 ==== 47+0+0 (2674337944 0 0) 0x87e47c0 con 0x7d29fa0 -7557> 2014-08-13 16:24:12.839788 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.839790) v2 -- ?+0 0x7d04600 con 0x7d29fa0 -7556> 2014-08-13 16:24:12.853952 7fd94ac49700 5 osd.4 194449 tick -7555> 2014-08-13 16:24:12.953964 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.953916) v2 ==== 47+0+0 (3975120422 0 0) 0x8ee96c0 con 0x7ebd6a0 -7554> 2014-08-13 16:24:12.954019 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.953916) v2 -- ?+0 0x87e47c0 con 0x7ebd6a0 -7553> 2014-08-13 16:24:12.953975 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:12.953916) v2 ==== 47+0+0 (3975120422 0 0) 0x8ee8540 con 0x7ebc0a0 -7552> 2014-08-13 16:24:12.954067 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:12.953916) v2 -- ?+0 0x87e6040 con 0x7ebc0a0 -7551> 2014-08-13 16:24:13.014133 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 11 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x8bc6300 con 0x510fd20 -7550> 2014-08-13 16:24:13.014201 7fd93dff0700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.013919, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7549> 2014-08-13 16:24:13.014218 7fd93dff0700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.013924, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7548> 2014-08-13 16:24:13.014229 7fd93dff0700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014047, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7547> 2014-08-13 16:24:13.014242 7fd93dff0700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014193, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7546> 2014-08-13 16:24:13.014258 7fd93dff0700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014258, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7545> 2014-08-13 16:24:13.014339 7fd9371e5700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014338, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7544> 2014-08-13 16:24:13.014384 7fd9371e5700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014384, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7543> 2014-08-13 16:24:13.014403 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003700 con 0x510fd20 -7542> 2014-08-13 16:24:13.014433 7fd9371e5700 5 -- op tracker -- , seq: 752, time: 2014-08-13 16:24:13.014433, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7541> 2014-08-13 16:24:13.015651 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 12 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x863b700 con 0x510fd20 -7540> 2014-08-13 16:24:13.015706 7fd93dff0700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015469, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7539> 2014-08-13 16:24:13.015723 7fd93dff0700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015472, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7538> 2014-08-13 16:24:13.015734 7fd93dff0700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015569, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7537> 2014-08-13 16:24:13.015743 7fd93dff0700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015699, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7536> 2014-08-13 16:24:13.015759 7fd93dff0700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015758, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7535> 2014-08-13 16:24:13.015882 7fd9367e4700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015882, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7534> 2014-08-13 16:24:13.015927 7fd9367e4700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015926, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7533> 2014-08-13 16:24:13.015943 7fd9367e4700 5 -- op tracker -- , seq: 753, time: 2014-08-13 16:24:13.015943, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7532> 2014-08-13 16:24:13.067746 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.067165) v2 ==== 47+0+0 (164937463 0 0) 0x87e71c0 con 0x7ebce60 -7531> 2014-08-13 16:24:13.067802 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.067165) v2 -- ?+0 0x8ee96c0 con 0x7ebce60 -7530> 2014-08-13 16:24:13.067844 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.067165) v2 ==== 47+0+0 (164937463 0 0) 0x92808c0 con 0x7ebb860 -7529> 2014-08-13 16:24:13.067894 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.067165) v2 -- ?+0 0x8ee8540 con 0x7ebb860 -7528> 2014-08-13 16:24:13.170310 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.170015) v2 ==== 47+0+0 (1573479013 0 0) 0x80db640 con 0x7d2f0c0 -7527> 2014-08-13 16:24:13.170333 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.170015) v2 ==== 47+0+0 (1573479013 0 0) 0x8eed5c0 con 0x7d2dc20 -7526> 2014-08-13 16:24:13.170365 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.170015) v2 -- ?+0 0x87e71c0 con 0x7d2f0c0 -7525> 2014-08-13 16:24:13.170444 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.170015) v2 -- ?+0 0x92808c0 con 0x7d2dc20 -7524> 2014-08-13 16:24:13.339186 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.339072) v2 ==== 47+0+0 (363852829 0 0) 0x7d03d40 con 0x7ab8420 -7523> 2014-08-13 16:24:13.339241 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.339072) v2 -- ?+0 0x8eed5c0 con 0x7ab8420 -7522> 2014-08-13 16:24:13.339232 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.339072) v2 ==== 47+0+0 (363852829 0 0) 0x7d032c0 con 0x7ab8000 -7521> 2014-08-13 16:24:13.339274 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.339072) v2 -- ?+0 0x80db640 con 0x7ab8000 -7520> 2014-08-13 16:24:13.416405 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.416021) v2 ==== 47+0+0 (374200089 0 0) 0x7d06900 con 0x7ed8580 -7519> 2014-08-13 16:24:13.416425 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.416021) v2 ==== 47+0+0 (374200089 0 0) 0x7d05b00 con 0x7ebe5c0 -7518> 2014-08-13 16:24:13.416483 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.416021) v2 -- ?+0 0x7d03d40 con 0x7ebe5c0 -7517> 2014-08-13 16:24:13.416578 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.416021) v2 -- ?+0 0x7d032c0 con 0x7ed8580 -7516> 2014-08-13 16:24:13.544427 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 21 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x7ec8b00 con 0x5112520 -7515> 2014-08-13 16:24:13.544483 7fd93dff0700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544224, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7514> 2014-08-13 16:24:13.544499 7fd93dff0700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544228, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7513> 2014-08-13 16:24:13.544509 7fd93dff0700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544348, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7512> 2014-08-13 16:24:13.544519 7fd93dff0700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544475, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7511> 2014-08-13 16:24:13.544534 7fd93dff0700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544533, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7510> 2014-08-13 16:24:13.544643 7fd9371e5700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544643, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7509> 2014-08-13 16:24:13.544690 7fd9371e5700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544690, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7508> 2014-08-13 16:24:13.544711 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003e80 con 0x5112520 -7507> 2014-08-13 16:24:13.544744 7fd9371e5700 5 -- op tracker -- , seq: 754, time: 2014-08-13 16:24:13.544744, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7506> 2014-08-13 16:24:13.545920 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 22 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x86dac00 con 0x5112520 -7505> 2014-08-13 16:24:13.545976 7fd93dff0700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.545770, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7504> 2014-08-13 16:24:13.545996 7fd93dff0700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.545771, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7503> 2014-08-13 16:24:13.546010 7fd93dff0700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.545855, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7502> 2014-08-13 16:24:13.546023 7fd93dff0700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.545969, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7501> 2014-08-13 16:24:13.546038 7fd93dff0700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.546038, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7500> 2014-08-13 16:24:13.546201 7fd9367e4700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.546201, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7499> 2014-08-13 16:24:13.546248 7fd9367e4700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.546248, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7498> 2014-08-13 16:24:13.546264 7fd9367e4700 5 -- op tracker -- , seq: 755, time: 2014-08-13 16:24:13.546264, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7497> 2014-08-13 16:24:13.574250 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.574419) v2 ==== 47+0+0 (985962709 0 0) 0x80dc440 con 0x7d289a0 -7496> 2014-08-13 16:24:13.574298 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.574419) v2 -- ?+0 0x7d05b00 con 0x7d289a0 -7495> 2014-08-13 16:24:13.574287 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.574419) v2 ==== 47+0+0 (985962709 0 0) 0x7d01f80 con 0x7caf380 -7494> 2014-08-13 16:24:13.574330 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.574419) v2 -- ?+0 0x7d06900 con 0x7caf380 -7493> 2014-08-13 16:24:13.645129 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.645068) v2 ==== 47+0+0 (2389359453 0 0) 0x81c2f40 con 0x7ebee00 -7492> 2014-08-13 16:24:13.645152 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.645068) v2 ==== 47+0+0 (2389359453 0 0) 0x81c5b00 con 0x7ed8dc0 -7491> 2014-08-13 16:24:13.645192 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.645068) v2 -- ?+0 0x80dc440 con 0x7ebee00 -7490> 2014-08-13 16:24:13.645253 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.645068) v2 -- ?+0 0x7d01f80 con 0x7ed8dc0 -7489> 2014-08-13 16:24:13.675984 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.675746) v2 ==== 47+0+0 (854450800 0 0) 0x80dd240 con 0x7d2f0c0 -7488> 2014-08-13 16:24:13.676037 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.675746) v2 -- ?+0 0x81c5b00 con 0x7d2f0c0 -7487> 2014-08-13 16:24:13.676056 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.675746) v2 ==== 47+0+0 (854450800 0 0) 0x8eef8c0 con 0x7d2dc20 -7486> 2014-08-13 16:24:13.676108 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.675746) v2 -- ?+0 0x81c2f40 con 0x7d2dc20 -7485> 2014-08-13 16:24:13.696793 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 13 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x863d800 con 0x4e498c0 -7484> 2014-08-13 16:24:13.696844 7fd93dff0700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696597, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7483> 2014-08-13 16:24:13.696860 7fd93dff0700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696601, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7482> 2014-08-13 16:24:13.696870 7fd93dff0700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696710, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7481> 2014-08-13 16:24:13.696879 7fd93dff0700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696838, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7480> 2014-08-13 16:24:13.696890 7fd93dff0700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696890, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7479> 2014-08-13 16:24:13.696993 7fd9371e5700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.696993, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7478> 2014-08-13 16:24:13.697034 7fd9371e5700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.697034, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7477> 2014-08-13 16:24:13.697053 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x7bc7300 con 0x4e498c0 -7476> 2014-08-13 16:24:13.697089 7fd9371e5700 5 -- op tracker -- , seq: 756, time: 2014-08-13 16:24:13.697088, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7475> 2014-08-13 16:24:13.698280 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 14 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x8bc6300 con 0x4e498c0 -7474> 2014-08-13 16:24:13.698335 7fd93dff0700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698094, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7473> 2014-08-13 16:24:13.698352 7fd93dff0700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698097, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7472> 2014-08-13 16:24:13.698362 7fd93dff0700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698197, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7471> 2014-08-13 16:24:13.698376 7fd93dff0700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698329, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7470> 2014-08-13 16:24:13.698393 7fd93dff0700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698392, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7469> 2014-08-13 16:24:13.698537 7fd9367e4700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698537, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7468> 2014-08-13 16:24:13.698579 7fd9367e4700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698579, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7467> 2014-08-13 16:24:13.698595 7fd9367e4700 5 -- op tracker -- , seq: 757, time: 2014-08-13 16:24:13.698595, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7466> 2014-08-13 16:24:13.789666 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.789174) v2 ==== 47+0+0 (2805843731 0 0) 0x4f8aa00 con 0x7ebf380 -7465> 2014-08-13 16:24:13.789684 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.789174) v2 ==== 47+0+0 (2805843731 0 0) 0x7d001c0 con 0x7ebfe80 -7464> 2014-08-13 16:24:13.789723 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.789174) v2 -- ?+0 0x8eef8c0 con 0x7ebf380 -7463> 2014-08-13 16:24:13.789793 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.789174) v2 -- ?+0 0x80dd240 con 0x7ebfe80 -7462> 2014-08-13 16:24:13.827040 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.826905) v2 ==== 47+0+0 (3810463955 0 0) 0x87e5940 con 0x7edbf40 -7461> 2014-08-13 16:24:13.827090 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.826905) v2 -- ?+0 0x7d001c0 con 0x7edbf40 -7460> 2014-08-13 16:24:13.827121 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.826905) v2 ==== 47+0+0 (3810463955 0 0) 0x87e6e40 con 0x7ffeca0 -7459> 2014-08-13 16:24:13.827185 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.826905) v2 -- ?+0 0x4f8aa00 con 0x7ffeca0 -7458> 2014-08-13 16:24:13.854067 7fd94ac49700 5 osd.4 194449 tick -7457> 2014-08-13 16:24:13.897647 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.897630) v2 ==== 47+0+0 (3955476700 0 0) 0x81c01c0 con 0x7edaec0 -7456> 2014-08-13 16:24:13.897704 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.897630) v2 -- ?+0 0x87e6e40 con 0x7edaec0 -7455> 2014-08-13 16:24:13.897725 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.897630) v2 ==== 47+0+0 (3955476700 0 0) 0x7d05cc0 con 0x7ed8000 -7454> 2014-08-13 16:24:13.897835 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.897630) v2 -- ?+0 0x87e5940 con 0x7ed8000 -7453> 2014-08-13 16:24:13.945537 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.945694) v2 ==== 47+0+0 (396017144 0 0) 0x87e71c0 con 0x7ebd120 -7452> 2014-08-13 16:24:13.945586 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.945694) v2 -- ?+0 0x7d05cc0 con 0x7ebd120 -7451> 2014-08-13 16:24:13.945578 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.945694) v2 ==== 47+0+0 (396017144 0 0) 0x92840c0 con 0x7ebbb20 -7450> 2014-08-13 16:24:13.945628 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.945694) v2 -- ?+0 0x81c01c0 con 0x7ebbb20 -7449> 2014-08-13 16:24:13.985298 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.985152) v2 ==== 47+0+0 (3866345810 0 0) 0x9282140 con 0x7cae5c0 -7448> 2014-08-13 16:24:13.985306 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.985152) v2 ==== 47+0+0 (3866345810 0 0) 0x9285b00 con 0x7d2a520 -7447> 2014-08-13 16:24:13.985355 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.985152) v2 -- ?+0 0x92840c0 con 0x7cae5c0 -7446> 2014-08-13 16:24:13.985417 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.985152) v2 -- ?+0 0x87e71c0 con 0x7d2a520 -7445> 2014-08-13 16:24:13.992073 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.991755) v2 ==== 47+0+0 (2744884580 0 0) 0x7d063c0 con 0x7eba3c0 -7444> 2014-08-13 16:24:13.992118 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.991755) v2 -- ?+0 0x9285b00 con 0x7eba3c0 -7443> 2014-08-13 16:24:13.992107 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:13.991755) v2 ==== 47+0+0 (2744884580 0 0) 0x80d9500 con 0x7eb8f20 -7442> 2014-08-13 16:24:13.992155 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:13.991755) v2 -- ?+0 0x9282140 con 0x7eb8f20 -7441> 2014-08-13 16:24:14.037271 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.037158) v2 ==== 47+0+0 (2147599594 0 0) 0x9287380 con 0x7edbb20 -7440> 2014-08-13 16:24:14.037296 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.037158) v2 ==== 47+0+0 (2147599594 0 0) 0x9283100 con 0x7ffe880 -7439> 2014-08-13 16:24:14.037325 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.037158) v2 -- ?+0 0x7d063c0 con 0x7edbb20 -7438> 2014-08-13 16:24:14.037443 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.037158) v2 -- ?+0 0x80d9500 con 0x7ffe880 -7437> 2014-08-13 16:24:14.083188 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.082772) v2 ==== 47+0+0 (720285872 0 0) 0x9285400 con 0x7ffe9e0 -7436> 2014-08-13 16:24:14.083220 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 2 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.082772) v2 ==== 47+0+0 (720285872 0 0) 0x9281c00 con 0x7edbc80 -7435> 2014-08-13 16:24:14.083235 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.082772) v2 -- ?+0 0x9283100 con 0x7ffe9e0 -7434> 2014-08-13 16:24:14.083327 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.082772) v2 -- ?+0 0x9287380 con 0x7edbc80 -7433> 2014-08-13 16:24:14.143131 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.142547) v2 ==== 47+0+0 (4101563788 0 0) 0x87e0540 con 0x7eb9b80 -7432> 2014-08-13 16:24:14.143122 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.142547) v2 ==== 47+0+0 (4101563788 0 0) 0x87e3d40 con 0x7ebc620 -7431> 2014-08-13 16:24:14.143187 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.142547) v2 -- ?+0 0x9281c00 con 0x7eb9b80 -7430> 2014-08-13 16:24:14.143259 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.142547) v2 -- ?+0 0x9285400 con 0x7ebc620 -7429> 2014-08-13 16:24:14.217264 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6812/43236 -- osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11495 0x7ec8b00 con 0x4fe4ba0 -7428> 2014-08-13 16:24:14.225170 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 11 ==== replica scrub(pg: 22.4aa,from:0'0,to:194215'472,epoch:194449,start:f33d4caa//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (2942263805 0 0) 0x859fbc0 con 0x4fe4ba0 -7427> 2014-08-13 16:24:14.226336 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.226111) v2 ==== 47+0+0 (198405177 0 0) 0x9283100 con 0x7d29ce0 -7426> 2014-08-13 16:24:14.226384 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.226111) v2 -- ?+0 0x87e0540 con 0x7d29ce0 -7425> 2014-08-13 16:24:14.226377 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.226111) v2 ==== 47+0+0 (198405177 0 0) 0x9287700 con 0x7d2b440 -7424> 2014-08-13 16:24:14.226423 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.226111) v2 -- ?+0 0x87e3d40 con 0x7d2b440 -7423> 2014-08-13 16:24:14.487553 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.487603) v2 ==== 47+0+0 (3176669795 0 0) 0x9284280 con 0x7d2a520 -7422> 2014-08-13 16:24:14.487571 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.487603) v2 ==== 47+0+0 (3176669795 0 0) 0x9286c80 con 0x7cae5c0 -7421> 2014-08-13 16:24:14.487606 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.487603) v2 -- ?+0 0x9283100 con 0x7d2a520 -7420> 2014-08-13 16:24:14.487732 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.487603) v2 -- ?+0 0x9287700 con 0x7cae5c0 -7419> 2014-08-13 16:24:14.517751 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.517711) v2 ==== 47+0+0 (831392666 0 0) 0x87e71c0 con 0x7ebe5c0 -7418> 2014-08-13 16:24:14.517804 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.517711) v2 -- ?+0 0x9286c80 con 0x7ebe5c0 -7417> 2014-08-13 16:24:14.517821 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.517711) v2 ==== 47+0+0 (831392666 0 0) 0x9283100 con 0x7ed8580 -7416> 2014-08-13 16:24:14.517864 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.517711) v2 -- ?+0 0x9284280 con 0x7ed8580 -7415> 2014-08-13 16:24:14.544636 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 23 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x86da100 con 0x5112520 -7414> 2014-08-13 16:24:14.544695 7fd93dff0700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544433, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7413> 2014-08-13 16:24:14.544712 7fd93dff0700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544435, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7412> 2014-08-13 16:24:14.544722 7fd93dff0700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544552, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7411> 2014-08-13 16:24:14.544732 7fd93dff0700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544684, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7410> 2014-08-13 16:24:14.544749 7fd93dff0700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544749, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7409> 2014-08-13 16:24:14.544860 7fd9371e5700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544859, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7408> 2014-08-13 16:24:14.544905 7fd9371e5700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544905, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7407> 2014-08-13 16:24:14.544930 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47580 con 0x5112520 -7406> 2014-08-13 16:24:14.544965 7fd9371e5700 5 -- op tracker -- , seq: 758, time: 2014-08-13 16:24:14.544965, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7405> 2014-08-13 16:24:14.546092 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 24 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x86d8b00 con 0x5112520 -7404> 2014-08-13 16:24:14.546145 7fd93dff0700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.545948, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7403> 2014-08-13 16:24:14.546162 7fd93dff0700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.545949, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7402> 2014-08-13 16:24:14.546172 7fd93dff0700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546027, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7401> 2014-08-13 16:24:14.546199 7fd93dff0700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546139, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7400> 2014-08-13 16:24:14.546211 7fd93dff0700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546211, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7399> 2014-08-13 16:24:14.546380 7fd9367e4700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546380, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7398> 2014-08-13 16:24:14.546429 7fd9367e4700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546428, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7397> 2014-08-13 16:24:14.546445 7fd9367e4700 5 -- op tracker -- , seq: 759, time: 2014-08-13 16:24:14.546445, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7396> 2014-08-13 16:24:14.603628 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.603492) v2 ==== 47+0+0 (2874545391 0 0) 0x9282a00 con 0x7edc360 -7395> 2014-08-13 16:24:14.603682 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.603492) v2 -- ?+0 0x9283100 con 0x7edc360 -7394> 2014-08-13 16:24:14.603759 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.603492) v2 ==== 47+0+0 (2874545391 0 0) 0x9284d00 con 0x7eda3c0 -7393> 2014-08-13 16:24:14.603803 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.603492) v2 -- ?+0 0x87e71c0 con 0x7eda3c0 -7392> 2014-08-13 16:24:14.633567 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 13 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3946485199 0 0) 0x8bc0b00 con 0x4e4f900 -7391> 2014-08-13 16:24:14.633622 7fd93dff0700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633373, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7390> 2014-08-13 16:24:14.633638 7fd93dff0700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633376, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7389> 2014-08-13 16:24:14.633649 7fd93dff0700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633486, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7388> 2014-08-13 16:24:14.633658 7fd93dff0700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633615, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7387> 2014-08-13 16:24:14.633673 7fd93dff0700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633672, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7386> 2014-08-13 16:24:14.633750 7fd9371e5700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633749, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7385> 2014-08-13 16:24:14.633793 7fd9371e5700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633792, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7384> 2014-08-13 16:24:14.633816 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47080 con 0x4e4f900 -7383> 2014-08-13 16:24:14.633849 7fd9371e5700 5 -- op tracker -- , seq: 760, time: 2014-08-13 16:24:14.633849, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7382> 2014-08-13 16:24:14.635153 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 14 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2053850730 0 0) 0x863c200 con 0x4e4f900 -7381> 2014-08-13 16:24:14.635224 7fd93dff0700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.634981, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7380> 2014-08-13 16:24:14.635245 7fd93dff0700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.634982, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7379> 2014-08-13 16:24:14.635259 7fd93dff0700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635060, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7378> 2014-08-13 16:24:14.635272 7fd93dff0700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635211, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7377> 2014-08-13 16:24:14.635287 7fd93dff0700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635287, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7376> 2014-08-13 16:24:14.635429 7fd9367e4700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635429, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7375> 2014-08-13 16:24:14.635473 7fd9367e4700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635473, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7374> 2014-08-13 16:24:14.635492 7fd9367e4700 5 -- op tracker -- , seq: 761, time: 2014-08-13 16:24:14.635492, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7373> 2014-08-13 16:24:14.685027 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.684747) v2 ==== 47+0+0 (410118347 0 0) 0x7d05080 con 0x7eba680 -7372> 2014-08-13 16:24:14.685058 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.684747) v2 ==== 47+0+0 (410118347 0 0) 0x7d01880 con 0x7eb91e0 -7371> 2014-08-13 16:24:14.685087 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.684747) v2 -- ?+0 0x9282a00 con 0x7eba680 -7370> 2014-08-13 16:24:14.685128 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.684747) v2 -- ?+0 0x9284d00 con 0x7eb91e0 -7369> 2014-08-13 16:24:14.760598 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 20 ==== osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1493162489 0 0) 0x8638000 con 0x51135a0 -7368> 2014-08-13 16:24:14.760653 7fd93dff0700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760401, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7367> 2014-08-13 16:24:14.760673 7fd93dff0700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760405, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7366> 2014-08-13 16:24:14.760687 7fd93dff0700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760512, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7365> 2014-08-13 16:24:14.760700 7fd93dff0700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760646, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7364> 2014-08-13 16:24:14.760715 7fd93dff0700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760715, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7363> 2014-08-13 16:24:14.760821 7fd9371e5700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760820, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7362> 2014-08-13 16:24:14.760865 7fd9371e5700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760865, event: started, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7361> 2014-08-13 16:24:14.760888 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.108:6840/7435 -- osd_sub_op_reply(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46e00 con 0x51135a0 -7360> 2014-08-13 16:24:14.760922 7fd9371e5700 5 -- op tracker -- , seq: 762, time: 2014-08-13 16:24:14.760922, event: done, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7359> 2014-08-13 16:24:14.762469 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 21 ==== osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3383626332 0 0) 0x863ac00 con 0x51135a0 -7358> 2014-08-13 16:24:14.762532 7fd93dff0700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762270, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7357> 2014-08-13 16:24:14.762549 7fd93dff0700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762272, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7356> 2014-08-13 16:24:14.762559 7fd93dff0700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762371, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7355> 2014-08-13 16:24:14.762569 7fd93dff0700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762522, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7354> 2014-08-13 16:24:14.762584 7fd93dff0700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762584, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7353> 2014-08-13 16:24:14.762720 7fd9367e4700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762720, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7352> 2014-08-13 16:24:14.762768 7fd9367e4700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762767, event: started, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7351> 2014-08-13 16:24:14.762784 7fd9367e4700 5 -- op tracker -- , seq: 763, time: 2014-08-13 16:24:14.762784, event: done, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7350> 2014-08-13 16:24:14.781338 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.781287) v2 ==== 47+0+0 (4143122297 0 0) 0x9285080 con 0x7d2dc20 -7349> 2014-08-13 16:24:14.781392 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.781287) v2 -- ?+0 0x7d01880 con 0x7d2dc20 -7348> 2014-08-13 16:24:14.781403 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.781287) v2 ==== 47+0+0 (4143122297 0 0) 0x9281340 con 0x7d2f0c0 -7347> 2014-08-13 16:24:14.781447 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.781287) v2 -- ?+0 0x7d05080 con 0x7d2f0c0 -7346> 2014-08-13 16:24:14.813336 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.813347) v2 ==== 47+0+0 (335050484 0 0) 0x7759c00 con 0x7edac00 -7345> 2014-08-13 16:24:14.813393 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.813347) v2 -- ?+0 0x9285080 con 0x7edac00 -7344> 2014-08-13 16:24:14.813392 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.813347) v2 ==== 47+0+0 (335050484 0 0) 0x775b2c0 con 0x7edcba0 -7343> 2014-08-13 16:24:14.813436 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.813347) v2 -- ?+0 0x9281340 con 0x7edcba0 -7342> 2014-08-13 16:24:14.835784 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.835604) v2 ==== 47+0+0 (1834016355 0 0) 0x775d780 con 0x7caeb40 -7341> 2014-08-13 16:24:14.835835 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.835604) v2 -- ?+0 0x7759c00 con 0x7caeb40 -7340> 2014-08-13 16:24:14.835845 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.835604) v2 ==== 47+0+0 (1834016355 0 0) 0x81c1180 con 0x7cae300 -7339> 2014-08-13 16:24:14.835888 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.835604) v2 -- ?+0 0x775b2c0 con 0x7cae300 -7338> 2014-08-13 16:24:14.854185 7fd94ac49700 5 osd.4 194449 tick -7337> 2014-08-13 16:24:14.892276 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.891620) v2 ==== 47+0+0 (1937576030 0 0) 0x7d07380 con 0x7ebfe80 -7336> 2014-08-13 16:24:14.892286 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:14.891620) v2 ==== 47+0+0 (1937576030 0 0) 0x80dc440 con 0x7ebf380 -7335> 2014-08-13 16:24:14.892327 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.891620) v2 -- ?+0 0x81c1180 con 0x7ebfe80 -7334> 2014-08-13 16:24:14.892446 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:14.891620) v2 -- ?+0 0x775d780 con 0x7ebf380 -7333> 2014-08-13 16:24:14.992635 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 13 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1487161271 0 0) 0x86db700 con 0x50b9a20 -7332> 2014-08-13 16:24:14.992722 7fd93dff0700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992465, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7331> 2014-08-13 16:24:14.992744 7fd93dff0700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992469, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7330> 2014-08-13 16:24:14.992788 7fd93dff0700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992600, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7329> 2014-08-13 16:24:14.992800 7fd93dff0700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992716, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7328> 2014-08-13 16:24:14.992811 7fd93dff0700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992811, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7327> 2014-08-13 16:24:14.992887 7fd9371e5700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992887, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7326> 2014-08-13 16:24:14.992927 7fd9371e5700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992927, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7325> 2014-08-13 16:24:14.992945 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op_reply(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46b80 con 0x50b9a20 -7324> 2014-08-13 16:24:14.992975 7fd9371e5700 5 -- op tracker -- , seq: 764, time: 2014-08-13 16:24:14.992974, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7323> 2014-08-13 16:24:14.994184 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 14 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3388314642 0 0) 0x86d9600 con 0x50b9a20 -7322> 2014-08-13 16:24:14.994236 7fd93dff0700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994016, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7321> 2014-08-13 16:24:14.994252 7fd93dff0700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994019, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7320> 2014-08-13 16:24:14.994262 7fd93dff0700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994120, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7319> 2014-08-13 16:24:14.994272 7fd93dff0700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994225, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7318> 2014-08-13 16:24:14.994287 7fd93dff0700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994287, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7317> 2014-08-13 16:24:14.994433 7fd9367e4700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994432, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7316> 2014-08-13 16:24:14.994474 7fd9367e4700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994474, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7315> 2014-08-13 16:24:14.994490 7fd9367e4700 5 -- op tracker -- , seq: 765, time: 2014-08-13 16:24:14.994490, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7314> 2014-08-13 16:24:15.014895 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 13 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x86dd800 con 0x510fd20 -7313> 2014-08-13 16:24:15.014947 7fd93dff0700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.014699, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7312> 2014-08-13 16:24:15.014963 7fd93dff0700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.014702, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7311> 2014-08-13 16:24:15.014973 7fd93dff0700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.014813, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7310> 2014-08-13 16:24:15.014982 7fd93dff0700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.014941, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7309> 2014-08-13 16:24:15.014993 7fd93dff0700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.014993, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7308> 2014-08-13 16:24:15.015096 7fd9371e5700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.015096, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7307> 2014-08-13 16:24:15.015138 7fd9371e5700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.015138, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7306> 2014-08-13 16:24:15.015157 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46900 con 0x510fd20 -7305> 2014-08-13 16:24:15.015206 7fd9371e5700 5 -- op tracker -- , seq: 766, time: 2014-08-13 16:24:15.015206, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7304> 2014-08-13 16:24:15.016765 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 14 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x86dcd00 con 0x510fd20 -7303> 2014-08-13 16:24:15.016820 7fd93dff0700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.016568, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7302> 2014-08-13 16:24:15.016837 7fd93dff0700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.016571, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7301> 2014-08-13 16:24:15.016847 7fd93dff0700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.016669, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7300> 2014-08-13 16:24:15.016857 7fd93dff0700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.016809, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7299> 2014-08-13 16:24:15.016872 7fd93dff0700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.016872, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7298> 2014-08-13 16:24:15.017012 7fd9367e4700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.017011, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7297> 2014-08-13 16:24:15.017058 7fd9367e4700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.017058, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7296> 2014-08-13 16:24:15.017075 7fd9367e4700 5 -- op tracker -- , seq: 767, time: 2014-08-13 16:24:15.017075, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7295> 2014-08-13 16:24:15.028073 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.027563) v2 ==== 47+0+0 (4148886039 0 0) 0x775d780 con 0x7ebbde0 -7294> 2014-08-13 16:24:15.028088 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.027563) v2 ==== 47+0+0 (4148886039 0 0) 0x4f8b2c0 con 0x7ebd3e0 -7293> 2014-08-13 16:24:15.028115 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.027563) v2 -- ?+0 0x80dc440 con 0x7ebbde0 -7292> 2014-08-13 16:24:15.028168 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.027563) v2 -- ?+0 0x7d07380 con 0x7ebd3e0 -7291> 2014-08-13 16:24:15.041050 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.040976) v2 ==== 47+0+0 (2284658805 0 0) 0x775aa00 con 0x7ab8420 -7290> 2014-08-13 16:24:15.041083 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.040976) v2 ==== 47+0+0 (2284658805 0 0) 0x9281340 con 0x7ab8000 -7289> 2014-08-13 16:24:15.041101 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.040976) v2 -- ?+0 0x775d780 con 0x7ab8420 -7288> 2014-08-13 16:24:15.041150 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.040976) v2 -- ?+0 0x4f8b2c0 con 0x7ab8000 -7287> 2014-08-13 16:24:15.076407 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6812/43236 -- osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+4181 0x86dee00 con 0x4fe4ba0 -7286> 2014-08-13 16:24:15.080878 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 12 ==== osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3502399378 0 0) 0x86dee00 con 0x4fe4ba0 -7285> 2014-08-13 16:24:15.080933 7fd93dff0700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.080688, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7284> 2014-08-13 16:24:15.080950 7fd93dff0700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.080691, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7283> 2014-08-13 16:24:15.080960 7fd93dff0700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.080794, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7282> 2014-08-13 16:24:15.080969 7fd93dff0700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.080927, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7281> 2014-08-13 16:24:15.080984 7fd93dff0700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.080984, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7280> 2014-08-13 16:24:15.081038 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.321 192.168.1.110:6812/43236 13 ==== pg_info(1 pgs e194449:22.4aa) v4 ==== 714+0+0 (1721136034 0 0) 0x9280e00 con 0x4fe4ba0 -7279> 2014-08-13 16:24:15.081060 7fd9371e5700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.081059, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7278> 2014-08-13 16:24:15.081105 7fd9371e5700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.081105, event: started, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7277> 2014-08-13 16:24:15.081125 7fd9371e5700 5 -- op tracker -- , seq: 768, time: 2014-08-13 16:24:15.081125, event: done, request: osd_sub_op(unknown.0.0:0 22.4aa 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7276> 2014-08-13 16:24:15.081181 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.080818, event: header_read, request: pg_info(1 pgs e194449:22.4aa) v4 -7275> 2014-08-13 16:24:15.081218 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.080819, event: throttled, request: pg_info(1 pgs e194449:22.4aa) v4 -7274> 2014-08-13 16:24:15.081226 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.080893, event: all_read, request: pg_info(1 pgs e194449:22.4aa) v4 -7273> 2014-08-13 16:24:15.081232 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.081057, event: dispatched, request: pg_info(1 pgs e194449:22.4aa) v4 -7272> 2014-08-13 16:24:15.081239 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.081239, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.4aa) v4 -7271> 2014-08-13 16:24:15.081250 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.081250, event: started, request: pg_info(1 pgs e194449:22.4aa) v4 -7270> 2014-08-13 16:24:15.081284 7fd93dff0700 5 -- op tracker -- , seq: 769, time: 2014-08-13 16:24:15.081284, event: done, request: pg_info(1 pgs e194449:22.4aa) v4 -7269> 2014-08-13 16:24:15.315161 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.315096) v2 ==== 47+0+0 (2521481291 0 0) 0x775bd40 con 0x7edcba0 -7268> 2014-08-13 16:24:15.315185 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.315096) v2 ==== 47+0+0 (2521481291 0 0) 0x775f8c0 con 0x7edac00 -7267> 2014-08-13 16:24:15.315240 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.315096) v2 -- ?+0 0x9281340 con 0x7edcba0 -7266> 2014-08-13 16:24:15.315340 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.315096) v2 -- ?+0 0x775aa00 con 0x7edac00 -7265> 2014-08-13 16:24:15.333329 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.332837) v2 ==== 47+0+0 (12478672 0 0) 0x8eeb800 con 0x7eda100 -7264> 2014-08-13 16:24:15.333369 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.332837) v2 -- ?+0 0x775f8c0 con 0x7eda100 -7263> 2014-08-13 16:24:15.333359 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.332837) v2 ==== 47+0+0 (12478672 0 0) 0x775e580 con 0x7edc0a0 -7262> 2014-08-13 16:24:15.333401 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.332837) v2 -- ?+0 0x775bd40 con 0x7edc0a0 -7261> 2014-08-13 16:24:15.346266 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.346144) v2 ==== 47+0+0 (3060446081 0 0) 0x81c7700 con 0x7ed8dc0 -7260> 2014-08-13 16:24:15.346297 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.346144) v2 ==== 47+0+0 (3060446081 0 0) 0x81c4b40 con 0x7ebee00 -7259> 2014-08-13 16:24:15.346321 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.346144) v2 -- ?+0 0x775e580 con 0x7ed8dc0 -7258> 2014-08-13 16:24:15.346442 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.346144) v2 -- ?+0 0x8eeb800 con 0x7ebee00 -7257> 2014-08-13 16:24:15.616312 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.615898) v2 ==== 47+0+0 (3146948533 0 0) 0x92871c0 con 0x7ab9ce0 -7256> 2014-08-13 16:24:15.616348 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.615898) v2 -- ?+0 0x81c7700 con 0x7ab9ce0 -7255> 2014-08-13 16:24:15.616359 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.615898) v2 ==== 47+0+0 (3146948533 0 0) 0x9284ec0 con 0x7ab9a20 -7254> 2014-08-13 16:24:15.616387 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.615898) v2 -- ?+0 0x81c4b40 con 0x7ab9a20 -7253> 2014-08-13 16:24:15.643911 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.643973) v2 ==== 47+0+0 (925733172 0 0) 0x8c896c0 con 0x7d286e0 -7252> 2014-08-13 16:24:15.643932 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.643973) v2 ==== 47+0+0 (925733172 0 0) 0x7d07700 con 0x7caf0c0 -7251> 2014-08-13 16:24:15.643941 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.643973) v2 -- ?+0 0x9284ec0 con 0x7d286e0 -7250> 2014-08-13 16:24:15.644029 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.643973) v2 -- ?+0 0x92871c0 con 0x7caf0c0 -7249> 2014-08-13 16:24:15.720918 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.720752) v2 ==== 47+0+0 (2693454902 0 0) 0x81c2680 con 0x7edb180 -7248> 2014-08-13 16:24:15.720962 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.720752) v2 ==== 47+0+0 (2693454902 0 0) 0x8eefa80 con 0x7ed82c0 -7247> 2014-08-13 16:24:15.720971 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.720752) v2 -- ?+0 0x8c896c0 con 0x7edb180 -7246> 2014-08-13 16:24:15.721004 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.720752) v2 -- ?+0 0x7d07700 con 0x7ed82c0 -7245> 2014-08-13 16:24:15.761325 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 22 ==== osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1493162489 0 0) 0x863b700 con 0x51135a0 -7244> 2014-08-13 16:24:15.761380 7fd93dff0700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761043, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7243> 2014-08-13 16:24:15.761396 7fd93dff0700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761048, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7242> 2014-08-13 16:24:15.761406 7fd93dff0700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761210, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7241> 2014-08-13 16:24:15.761416 7fd93dff0700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761371, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7240> 2014-08-13 16:24:15.761431 7fd93dff0700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761431, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7239> 2014-08-13 16:24:15.761546 7fd9371e5700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761545, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7238> 2014-08-13 16:24:15.761589 7fd9371e5700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761589, event: started, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7237> 2014-08-13 16:24:15.761609 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.108:6840/7435 -- osd_sub_op_reply(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46680 con 0x51135a0 -7236> 2014-08-13 16:24:15.761639 7fd9371e5700 5 -- op tracker -- , seq: 770, time: 2014-08-13 16:24:15.761638, event: done, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7235> 2014-08-13 16:24:15.765564 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 23 ==== replica scrub(pg: 22.1fb,from:0'0,to:194215'522,epoch:194449,start:0//0//-1,end:66e8b9fb//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2471566580 0 0) 0x859dc40 con 0x51135a0 -7234> 2014-08-13 16:24:15.810384 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.810333) v2 ==== 47+0+0 (1579989887 0 0) 0x8c8e740 con 0x7d2a260 -7233> 2014-08-13 16:24:15.810436 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.810333) v2 -- ?+0 0x8eefa80 con 0x7d2a260 -7232> 2014-08-13 16:24:15.810491 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.810333) v2 ==== 47+0+0 (1579989887 0 0) 0x87e78c0 con 0x7d28160 -7231> 2014-08-13 16:24:15.810543 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.810333) v2 -- ?+0 0x81c2680 con 0x7d28160 -7230> 2014-08-13 16:24:15.840139 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.108:6840/7435 -- osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11497 0x7ec8b00 con 0x51135a0 -7229> 2014-08-13 16:24:15.852947 7fd9385e7700 10 monclient: tick -7228> 2014-08-13 16:24:15.852972 7fd9385e7700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 16:23:45.852969) -7227> 2014-08-13 16:24:15.852985 7fd9385e7700 10 monclient: renew subs? (now: 2014-08-13 16:24:15.852985; renew after: 2014-08-13 16:26:33.460427) -- no -7226> 2014-08-13 16:24:15.853001 7fd9385e7700 10 log_queue is 2 last_log 2 sent 0 num 2 unsent 2 sending 2 -7225> 2014-08-13 16:24:15.853008 7fd9385e7700 10 will send 2014-08-13 16:24:07.081392 osd.4 192.168.100.115:6805/25220 1 : [WRN] pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate -7224> 2014-08-13 16:24:15.853047 7fd9385e7700 10 will send 2014-08-13 16:24:07.083929 osd.4 192.168.100.115:6805/25220 2 : [WRN] pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate -7223> 2014-08-13 16:24:15.853059 7fd9385e7700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -7222> 2014-08-13 16:24:15.853064 7fd9385e7700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- log(2 entries) v1 -- ?+0 0x7cf72c0 con 0x4e3dd80 -7221> 2014-08-13 16:24:15.854343 7fd94ac49700 5 osd.4 194449 tick -7220> 2014-08-13 16:24:15.858353 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.858360) v2 ==== 47+0+0 (2008368855 0 0) 0x80dbb80 con 0x7ebd6a0 -7219> 2014-08-13 16:24:15.858406 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.858360) v2 -- ?+0 0x8c8e740 con 0x7ebd6a0 -7218> 2014-08-13 16:24:15.858403 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.858360) v2 ==== 47+0+0 (2008368855 0 0) 0x87e4ec0 con 0x7ebc0a0 -7217> 2014-08-13 16:24:15.858457 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.858360) v2 -- ?+0 0x87e78c0 con 0x7ebc0a0 -7216> 2014-08-13 16:24:15.877899 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 24 ==== replica scrub(pg: 22.1fb,from:0'0,to:192888'489,epoch:194449,start:66e8b9fb//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2710380830 0 0) 0x7cf6e40 con 0x51135a0 -7215> 2014-08-13 16:24:15.884268 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.108:6840/7435 -- osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+1884 0x86de300 con 0x51135a0 -7214> 2014-08-13 16:24:15.912050 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 25 ==== osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3383626332 0 0) 0x86de300 con 0x51135a0 -7213> 2014-08-13 16:24:15.912106 7fd93dff0700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.911837, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7212> 2014-08-13 16:24:15.912122 7fd93dff0700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.911840, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7211> 2014-08-13 16:24:15.912132 7fd93dff0700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.911965, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7210> 2014-08-13 16:24:15.912142 7fd93dff0700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.912099, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7209> 2014-08-13 16:24:15.912174 7fd93dff0700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.912173, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7208> 2014-08-13 16:24:15.912210 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 26 ==== pg_info(1 pgs e194449:22.1fb) v4 ==== 714+0+0 (4174713179 0 0) 0x9281a40 con 0x51135a0 -7207> 2014-08-13 16:24:15.912231 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.911999, event: header_read, request: pg_info(1 pgs e194449:22.1fb) v4 -7206> 2014-08-13 16:24:15.912238 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912000, event: throttled, request: pg_info(1 pgs e194449:22.1fb) v4 -7205> 2014-08-13 16:24:15.912244 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912079, event: all_read, request: pg_info(1 pgs e194449:22.1fb) v4 -7204> 2014-08-13 16:24:15.912249 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912229, event: dispatched, request: pg_info(1 pgs e194449:22.1fb) v4 -7203> 2014-08-13 16:24:15.912260 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912260, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.1fb) v4 -7202> 2014-08-13 16:24:15.912272 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912272, event: started, request: pg_info(1 pgs e194449:22.1fb) v4 -7201> 2014-08-13 16:24:15.912288 7fd9367e4700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.912253, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7200> 2014-08-13 16:24:15.912333 7fd9367e4700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.912333, event: started, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7199> 2014-08-13 16:24:15.912353 7fd9367e4700 5 -- op tracker -- , seq: 771, time: 2014-08-13 16:24:15.912352, event: done, request: osd_sub_op(unknown.0.0:0 22.1fb 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7198> 2014-08-13 16:24:15.912422 7fd93dff0700 5 -- op tracker -- , seq: 772, time: 2014-08-13 16:24:15.912421, event: done, request: pg_info(1 pgs e194449:22.1fb) v4 -7197> 2014-08-13 16:24:15.914144 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 23 ==== log(last 2) v1 ==== 24+0+0 (3957342332 0 0) 0x8db0380 con 0x4e3dd80 -7196> 2014-08-13 16:24:15.914188 7fd93f3f2700 10 handle_log_ack log(last 2) v1 -7195> 2014-08-13 16:24:15.914197 7fd93f3f2700 10 logged 2014-08-13 16:24:07.081392 osd.4 192.168.100.115:6805/25220 1 : [WRN] pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate -7194> 2014-08-13 16:24:15.914217 7fd93f3f2700 10 logged 2014-08-13 16:24:07.083929 osd.4 192.168.100.115:6805/25220 2 : [WRN] pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate -7193> 2014-08-13 16:24:15.990456 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.989844) v2 ==== 47+0+0 (3003323037 0 0) 0x80da4c0 con 0x7d2ac00 -7192> 2014-08-13 16:24:15.990481 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:15.989844) v2 ==== 47+0+0 (3003323037 0 0) 0x8c8a680 con 0x7d2c360 -7191> 2014-08-13 16:24:15.990512 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.989844) v2 -- ?+0 0x87e4ec0 con 0x7d2ac00 -7190> 2014-08-13 16:24:15.990564 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:15.989844) v2 -- ?+0 0x80dbb80 con 0x7d2c360 -7189> 2014-08-13 16:24:15.993241 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 15 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1487161271 0 0) 0x863d800 con 0x50b9a20 -7188> 2014-08-13 16:24:15.993292 7fd93dff0700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993032, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7187> 2014-08-13 16:24:15.993308 7fd93dff0700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993036, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7186> 2014-08-13 16:24:15.993318 7fd93dff0700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993159, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7185> 2014-08-13 16:24:15.993327 7fd93dff0700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993286, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7184> 2014-08-13 16:24:15.993345 7fd93dff0700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993345, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7183> 2014-08-13 16:24:15.993447 7fd9371e5700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993447, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7182> 2014-08-13 16:24:15.993488 7fd9371e5700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993488, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7181> 2014-08-13 16:24:15.993507 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op_reply(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003200 con 0x50b9a20 -7180> 2014-08-13 16:24:15.993538 7fd9371e5700 5 -- op tracker -- , seq: 773, time: 2014-08-13 16:24:15.993538, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7179> 2014-08-13 16:24:16.000968 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 16 ==== replica scrub(pg: 22.2d1,from:0'0,to:194215'526,epoch:194449,start:0//0//-1,end:ad8a5ad1//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (3367477282 0 0) 0x859ec00 con 0x50b9a20 -7178> 2014-08-13 16:24:16.015327 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 15 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x86dc200 con 0x510fd20 -7177> 2014-08-13 16:24:16.015384 7fd93dff0700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015117, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7176> 2014-08-13 16:24:16.015400 7fd93dff0700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015120, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7175> 2014-08-13 16:24:16.015410 7fd93dff0700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015244, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7174> 2014-08-13 16:24:16.015420 7fd93dff0700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015377, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7173> 2014-08-13 16:24:16.015431 7fd93dff0700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015430, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7172> 2014-08-13 16:24:16.015571 7fd9367e4700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015571, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7171> 2014-08-13 16:24:16.015625 7fd9367e4700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015625, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7170> 2014-08-13 16:24:16.015645 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002d00 con 0x510fd20 -7169> 2014-08-13 16:24:16.015679 7fd9367e4700 5 -- op tracker -- , seq: 774, time: 2014-08-13 16:24:16.015679, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7168> 2014-08-13 16:24:16.017026 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 16 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x86dac00 con 0x510fd20 -7167> 2014-08-13 16:24:16.017078 7fd93dff0700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.016816, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7166> 2014-08-13 16:24:16.017094 7fd93dff0700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.016819, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7165> 2014-08-13 16:24:16.017104 7fd93dff0700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.016945, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7164> 2014-08-13 16:24:16.017118 7fd93dff0700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.017071, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7163> 2014-08-13 16:24:16.017134 7fd93dff0700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.017133, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7162> 2014-08-13 16:24:16.017257 7fd9371e5700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.017257, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7161> 2014-08-13 16:24:16.017298 7fd9371e5700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.017297, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7160> 2014-08-13 16:24:16.017313 7fd9371e5700 5 -- op tracker -- , seq: 775, time: 2014-08-13 16:24:16.017313, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7159> 2014-08-13 16:24:16.048662 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11513 0x863d800 con 0x50b9a20 -7158> 2014-08-13 16:24:16.053390 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 14 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4136492510 0 0) 0x86dee00 con 0x4fe3700 -7157> 2014-08-13 16:24:16.053441 7fd93dff0700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053199, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7156> 2014-08-13 16:24:16.053457 7fd93dff0700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053203, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7155> 2014-08-13 16:24:16.053467 7fd93dff0700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053306, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7154> 2014-08-13 16:24:16.053480 7fd93dff0700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053435, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7153> 2014-08-13 16:24:16.053491 7fd93dff0700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053491, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7152> 2014-08-13 16:24:16.053611 7fd9367e4700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053611, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7151> 2014-08-13 16:24:16.053653 7fd9367e4700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053652, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7150> 2014-08-13 16:24:16.053671 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_sub_op_reply(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002a80 con 0x4fe3700 -7149> 2014-08-13 16:24:16.053700 7fd9367e4700 5 -- op tracker -- , seq: 776, time: 2014-08-13 16:24:16.053700, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7148> 2014-08-13 16:24:16.055297 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 15 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1742474875 0 0) 0x86d9600 con 0x4fe3700 -7147> 2014-08-13 16:24:16.055358 7fd93dff0700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055079, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7146> 2014-08-13 16:24:16.055375 7fd93dff0700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055082, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7145> 2014-08-13 16:24:16.055386 7fd93dff0700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055214, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7144> 2014-08-13 16:24:16.055396 7fd93dff0700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055346, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7143> 2014-08-13 16:24:16.055411 7fd93dff0700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055411, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7142> 2014-08-13 16:24:16.055470 7fd9371e5700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055469, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7141> 2014-08-13 16:24:16.055507 7fd9371e5700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055507, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7140> 2014-08-13 16:24:16.055522 7fd9371e5700 5 -- op tracker -- , seq: 777, time: 2014-08-13 16:24:16.055522, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7139> 2014-08-13 16:24:16.088503 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 17 ==== replica scrub(pg: 22.2d1,from:0'0,to:194215'525,epoch:194449,start:ad8a5ad1//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (898857157 0 0) 0x859e9c0 con 0x50b9a20 -7138> 2014-08-13 16:24:16.092955 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6824/52078 -- osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+5476 0x863d800 con 0x50b9a20 -7137> 2014-08-13 16:24:16.103954 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.103492) v2 ==== 47+0+0 (2438292156 0 0) 0x8eed940 con 0x7edc8e0 -7136> 2014-08-13 16:24:16.104009 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.103492) v2 -- ?+0 0x8c8a680 con 0x7edc8e0 -7135> 2014-08-13 16:24:16.104047 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.103492) v2 ==== 47+0+0 (2438292156 0 0) 0x8c8c600 con 0x7eda940 -7134> 2014-08-13 16:24:16.104088 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.103492) v2 -- ?+0 0x80da4c0 con 0x7eda940 -7133> 2014-08-13 16:24:16.108536 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.108546) v2 ==== 47+0+0 (1061598446 0 0) 0x87e4ec0 con 0x7ed9080 -7132> 2014-08-13 16:24:16.108544 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.108546) v2 ==== 47+0+0 (1061598446 0 0) 0x7d01a40 con 0x7ebf0c0 -7131> 2014-08-13 16:24:16.108572 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.108546) v2 -- ?+0 0x8eed940 con 0x7ed9080 -7130> 2014-08-13 16:24:16.108766 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.108546) v2 -- ?+0 0x8c8c600 con 0x7ebf0c0 -7129> 2014-08-13 16:24:16.129253 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 18 ==== osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3388314642 0 0) 0x863d800 con 0x50b9a20 -7128> 2014-08-13 16:24:16.129303 7fd93dff0700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129084, event: header_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7127> 2014-08-13 16:24:16.129320 7fd93dff0700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129086, event: throttled, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7126> 2014-08-13 16:24:16.129330 7fd93dff0700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129194, event: all_read, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7125> 2014-08-13 16:24:16.129344 7fd93dff0700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129297, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7124> 2014-08-13 16:24:16.129359 7fd93dff0700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129359, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7123> 2014-08-13 16:24:16.129479 7fd9367e4700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129479, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7122> 2014-08-13 16:24:16.129513 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 19 ==== pg_info(1 pgs e194449:22.2d1) v4 ==== 714+0+0 (487866147 0 0) 0x9102bc0 con 0x50b9a20 -7121> 2014-08-13 16:24:16.129527 7fd9367e4700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129527, event: started, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7120> 2014-08-13 16:24:16.129547 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129273, event: header_read, request: pg_info(1 pgs e194449:22.2d1) v4 -7119> 2014-08-13 16:24:16.129560 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129281, event: throttled, request: pg_info(1 pgs e194449:22.2d1) v4 -7118> 2014-08-13 16:24:16.129571 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129398, event: all_read, request: pg_info(1 pgs e194449:22.2d1) v4 -7117> 2014-08-13 16:24:16.129577 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129536, event: dispatched, request: pg_info(1 pgs e194449:22.2d1) v4 -7116> 2014-08-13 16:24:16.129584 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129584, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.2d1) v4 -7115> 2014-08-13 16:24:16.129608 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129608, event: started, request: pg_info(1 pgs e194449:22.2d1) v4 -7114> 2014-08-13 16:24:16.129641 7fd93dff0700 5 -- op tracker -- , seq: 779, time: 2014-08-13 16:24:16.129641, event: done, request: pg_info(1 pgs e194449:22.2d1) v4 -7113> 2014-08-13 16:24:16.129674 7fd9367e4700 5 -- op tracker -- , seq: 778, time: 2014-08-13 16:24:16.129548, event: done, request: osd_sub_op(unknown.0.0:0 22.2d1 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7112> 2014-08-13 16:24:16.146788 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.146717) v2 ==== 47+0+0 (2279719988 0 0) 0x8c8d080 con 0x7d2aec0 -7111> 2014-08-13 16:24:16.146839 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.146717) v2 -- ?+0 0x7d01a40 con 0x7d2aec0 -7110> 2014-08-13 16:24:16.146889 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.146717) v2 ==== 47+0+0 (2279719988 0 0) 0x775f540 con 0x7d2c620 -7109> 2014-08-13 16:24:16.146940 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.146717) v2 -- ?+0 0x87e4ec0 con 0x7d2c620 -7108> 2014-08-13 16:24:16.336419 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 13 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4028734706 0 0) 0x8639600 con 0x5112aa0 -7107> 2014-08-13 16:24:16.336475 7fd93dff0700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336209, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7106> 2014-08-13 16:24:16.336491 7fd93dff0700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336213, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7105> 2014-08-13 16:24:16.336501 7fd93dff0700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336336, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7104> 2014-08-13 16:24:16.336514 7fd93dff0700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336468, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7103> 2014-08-13 16:24:16.336530 7fd93dff0700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336530, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7102> 2014-08-13 16:24:16.336663 7fd9367e4700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336662, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7101> 2014-08-13 16:24:16.336704 7fd9367e4700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336704, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7100> 2014-08-13 16:24:16.336725 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6808/62751 -- osd_sub_op_reply(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47d00 con 0x5112aa0 -7099> 2014-08-13 16:24:16.336756 7fd9367e4700 5 -- op tracker -- , seq: 780, time: 2014-08-13 16:24:16.336756, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7098> 2014-08-13 16:24:16.342028 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 14 ==== replica scrub(pg: 22.69f,from:0'0,to:194215'527,epoch:194449,start:0//0//-1,end:13261e9f//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (42899418 0 0) 0x7b11440 con 0x5112aa0 -7097> 2014-08-13 16:24:16.346891 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6808/62751 -- osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11555 0x863c200 con 0x5112aa0 -7096> 2014-08-13 16:24:16.361948 7fd933fe0700 5 osd.4 194449 heartbeat: osd_stat(8079 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,7,10,11,15,17,20,21,24,25,28,31,32,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,396,397]/[] op hist []) -7095> 2014-08-13 16:24:16.361995 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2d80 con 0x8c391e0 -7094> 2014-08-13 16:24:16.362028 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2bc0 con 0x8c38f20 -7093> 2014-08-13 16:24:16.362051 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2a00 con 0x78844c0 -7092> 2014-08-13 16:24:16.362073 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2840 con 0x7884200 -7091> 2014-08-13 16:24:16.362090 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2680 con 0x78848e0 -7090> 2014-08-13 16:24:16.362107 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e24c0 con 0x7884620 -7089> 2014-08-13 16:24:16.362124 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6839/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2300 con 0x8ba06e0 -7088> 2014-08-13 16:24:16.362145 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6846/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e2140 con 0x8c698c0 -7087> 2014-08-13 16:24:16.362173 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1f80 con 0x794c360 -7086> 2014-08-13 16:24:16.362197 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1dc0 con 0x794e300 -7085> 2014-08-13 16:24:16.362220 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1c00 con 0x794e040 -7084> 2014-08-13 16:24:16.362240 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1a40 con 0x794dd80 -7083> 2014-08-13 16:24:16.362258 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1880 con 0x8c3a3c0 -7082> 2014-08-13 16:24:16.362281 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e16c0 con 0x8c3a100 -7081> 2014-08-13 16:24:16.362299 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1500 con 0x8c3c0a0 -7080> 2014-08-13 16:24:16.362315 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1340 con 0x8c3bde0 -7079> 2014-08-13 16:24:16.362333 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e1180 con 0x8babde0 -7078> 2014-08-13 16:24:16.362356 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e0fc0 con 0x8babb20 -7077> 2014-08-13 16:24:16.362375 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e0e00 con 0x8c3d6a0 -7076> 2014-08-13 16:24:16.362396 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e0c40 con 0x8c3d3e0 -7075> 2014-08-13 16:24:16.362417 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e0a80 con 0x8badac0 -7074> 2014-08-13 16:24:16.362439 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e08c0 con 0x8bad800 -7073> 2014-08-13 16:24:16.362459 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x87e0700 con 0x8bab860 -7072> 2014-08-13 16:24:16.362479 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9287000 con 0x8bab5a0 -7071> 2014-08-13 16:24:16.362469 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7758a80 con 0x8c38f20 -7070> 2014-08-13 16:24:16.362499 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x92824c0 con 0x8c69600 -7069> 2014-08-13 16:24:16.362518 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899cb40 con 0x8c69340 -7068> 2014-08-13 16:24:16.362535 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899c280 con 0x8baee00 -7067> 2014-08-13 16:24:16.362557 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee8a80 con 0x8ba1a20 -7066> 2014-08-13 16:24:16.362574 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6814/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee8380 con 0x8d21e40 -7065> 2014-08-13 16:24:16.362591 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6831/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eebd40 con 0x8d21b80 -7064> 2014-08-13 16:24:16.362610 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eef8c0 con 0x7fba3c0 -7063> 2014-08-13 16:24:16.362627 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eefa80 con 0x7fba260 -7062> 2014-08-13 16:24:16.362645 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eef700 con 0x8bad540 -7061> 2014-08-13 16:24:16.362661 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9500 con 0x8bad280 -7060> 2014-08-13 16:24:16.362680 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x91039c0 con 0x8c3cba0 -7059> 2014-08-13 16:24:16.362699 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9101c00 con 0x8c3c8e0 -7058> 2014-08-13 16:24:16.362717 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9103480 con 0x7fba100 -7057> 2014-08-13 16:24:16.362733 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9103d40 con 0x7fb9fa0 -7056> 2014-08-13 16:24:16.362750 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9107540 con 0x8d223c0 -7055> 2014-08-13 16:24:16.362770 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9104b40 con 0x8d22100 -7054> 2014-08-13 16:24:16.362789 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9104440 con 0x8c3c620 -7053> 2014-08-13 16:24:16.362805 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9104980 con 0x8c3c360 -7052> 2014-08-13 16:24:16.362821 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x91078c0 con 0x8bafe80 -7051> 2014-08-13 16:24:16.362841 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9102d80 con 0x8bafbc0 -7050> 2014-08-13 16:24:16.362857 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9101340 con 0x8c38c60 -7049> 2014-08-13 16:24:16.362875 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9106040 con 0x8c389a0 -7048> 2014-08-13 16:24:16.362892 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9102a00 con 0x8b68c60 -7047> 2014-08-13 16:24:16.362909 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9103640 con 0x8b68b00 -7046> 2014-08-13 16:24:16.362925 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9100540 con 0x8c6fe80 -7045> 2014-08-13 16:24:16.362943 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x91008c0 con 0x8c6fbc0 -7044> 2014-08-13 16:24:16.362960 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9101180 con 0x8b689a0 -7043> 2014-08-13 16:24:16.362977 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x91047c0 con 0x8b68840 -7042> 2014-08-13 16:24:16.362994 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9105940 con 0x7fb9e40 -7041> 2014-08-13 16:24:16.363014 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9106ac0 con 0x7fb9ce0 -7040> 2014-08-13 16:24:16.363030 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9101a40 con 0x8c3b9c0 -7039> 2014-08-13 16:24:16.363048 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x9100000 con 0x8c3b700 -7038> 2014-08-13 16:24:16.363068 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeaa00 con 0x7fbb020 -7037> 2014-08-13 16:24:16.363083 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeb9c0 con 0x7fbaec0 -7036> 2014-08-13 16:24:16.363099 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eedcc0 con 0x7fbb440 -7035> 2014-08-13 16:24:16.363115 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eefe00 con 0x7fbb2e0 -7034> 2014-08-13 16:24:16.363130 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eee740 con 0x7ffd960 -7033> 2014-08-13 16:24:16.363147 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9180 con 0x7ffdc20 -7032> 2014-08-13 16:24:16.363177 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eea4c0 con 0x8d20840 -7031> 2014-08-13 16:24:16.363198 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eed240 con 0x8d25960 -7030> 2014-08-13 16:24:16.363221 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeaf40 con 0x8d218c0 -7029> 2014-08-13 16:24:16.363242 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeb100 con 0x8d21600 -7028> 2014-08-13 16:24:16.363262 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eec980 con 0x7fbaaa0 -7027> 2014-08-13 16:24:16.363278 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eec0c0 con 0x7fba940 -7026> 2014-08-13 16:24:16.363305 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee8fc0 con 0x8d26e00 -7025> 2014-08-13 16:24:16.363323 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9c00 con 0x8d26b40 -7024> 2014-08-13 16:24:16.363343 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eead80 con 0x8d21340 -7023> 2014-08-13 16:24:16.363361 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eea680 con 0x8d21080 -7022> 2014-08-13 16:24:16.363377 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee8e00 con 0x7ffdee0 -7021> 2014-08-13 16:24:16.363401 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9880 con 0x7ffdd80 -7020> 2014-08-13 16:24:16.363423 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9dc0 con 0x7fb9a20 -7019> 2014-08-13 16:24:16.363441 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eed080 con 0x7fb98c0 -7018> 2014-08-13 16:24:16.363460 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeb2c0 con 0x8bab2e0 -7017> 2014-08-13 16:24:16.363481 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eecec0 con 0x8bab020 -7016> 2014-08-13 16:24:16.363506 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeec80 con 0x8c69080 -7015> 2014-08-13 16:24:16.363522 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeb480 con 0x8c68dc0 -7014> 2014-08-13 16:24:16.363540 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eede80 con 0x8d256a0 -7013> 2014-08-13 16:24:16.363562 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee9a40 con 0x8d253e0 -7012> 2014-08-13 16:24:16.363585 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eeabc0 con 0x8d20dc0 -7011> 2014-08-13 16:24:16.363610 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eec440 con 0x8d20b00 -7010> 2014-08-13 16:24:16.363630 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8eef540 con 0x8d25120 -7009> 2014-08-13 16:24:16.363651 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8ee8700 con 0x8d24e60 -7008> 2014-08-13 16:24:16.363675 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8910380 con 0x8c3d120 -7007> 2014-08-13 16:24:16.363695 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899ec80 con 0x8c3ce60 -7006> 2014-08-13 16:24:16.363717 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899d400 con 0x7fbb180 -7005> 2014-08-13 16:24:16.363736 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899e040 con 0x7fbca40 -7004> 2014-08-13 16:24:16.363752 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899db00 con 0x7fbc780 -7003> 2014-08-13 16:24:16.363771 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8998700 con 0x7fbc4c0 -7002> 2014-08-13 16:24:16.363787 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8999a40 con 0x8c3aec0 -7001> 2014-08-13 16:24:16.363804 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899f540 con 0x8c3ac00 -7000> 2014-08-13 16:24:16.363833 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899f700 con 0x7fbad60 -6999> 2014-08-13 16:24:16.363849 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899b480 con 0x7fbac00 -6998> 2014-08-13 16:24:16.363865 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899fe00 con 0x8c3e880 -6997> 2014-08-13 16:24:16.363885 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899d780 con 0x8c3e5c0 -6996> 2014-08-13 16:24:16.363906 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899b640 con 0x8c3a940 -6995> 2014-08-13 16:24:16.363928 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899af40 con 0x8c3a680 -6994> 2014-08-13 16:24:16.363945 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899a300 con 0x8c69fa0 -6993> 2014-08-13 16:24:16.363964 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899e200 con 0x8c39fa0 -6992> 2014-08-13 16:24:16.363985 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899b9c0 con 0x7fb9760 -6991> 2014-08-13 16:24:16.364001 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899d5c0 con 0x7fb9600 -6990> 2014-08-13 16:24:16.364018 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899d240 con 0x8c3e300 -6989> 2014-08-13 16:24:16.364034 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89988c0 con 0x8c618c0 -6988> 2014-08-13 16:24:16.364056 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899d940 con 0x8bac780 -6987> 2014-08-13 16:24:16.364076 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899fa80 con 0x8bac4c0 -6986> 2014-08-13 16:24:16.364097 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8999dc0 con 0x8bac200 -6985> 2014-08-13 16:24:16.364117 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899de80 con 0x8babf40 -6984> 2014-08-13 16:24:16.364139 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899ad80 con 0x8d240a0 -6983> 2014-08-13 16:24:16.364163 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8999340 con 0x8d23de0 -6982> 2014-08-13 16:24:16.364185 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8998000 con 0x8ba1760 -6981> 2014-08-13 16:24:16.364202 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8998380 con 0x8ba14a0 -6980> 2014-08-13 16:24:16.364220 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899e740 con 0x8d26880 -6979> 2014-08-13 16:24:16.364238 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899ee40 con 0x8d265c0 -6978> 2014-08-13 16:24:16.364254 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x899a840 con 0x8baf900 -6977> 2014-08-13 16:24:16.364273 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913f00 con 0x8baf640 -6976> 2014-08-13 16:24:16.364293 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913d40 con 0x8bae040 -6975> 2014-08-13 16:24:16.364310 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913b80 con 0x8badd80 -6974> 2014-08-13 16:24:16.364327 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89139c0 con 0x8d24620 -6973> 2014-08-13 16:24:16.364343 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913800 con 0x8d24360 -6972> 2014-08-13 16:24:16.364359 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913640 con 0x8baf380 -6971> 2014-08-13 16:24:16.364377 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913480 con 0x8baf0c0 -6970> 2014-08-13 16:24:16.364396 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89132c0 con 0x8b6e1a0 -6969> 2014-08-13 16:24:16.364414 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8913100 con 0x8b6e040 -6968> 2014-08-13 16:24:16.364430 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912f40 con 0x8c39ce0 -6967> 2014-08-13 16:24:16.364446 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912d80 con 0x8c39a20 -6966> 2014-08-13 16:24:16.364462 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912bc0 con 0x8d27380 -6965> 2014-08-13 16:24:16.364483 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912a00 con 0x8d270c0 -6964> 2014-08-13 16:24:16.364502 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912840 con 0x8d23b20 -6963> 2014-08-13 16:24:16.364518 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912680 con 0x8d27bc0 -6962> 2014-08-13 16:24:16.364537 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89124c0 con 0x8b6dee0 -6961> 2014-08-13 16:24:16.364554 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912300 con 0x8b6dd80 -6960> 2014-08-13 16:24:16.364575 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8912140 con 0x7fba7e0 -6959> 2014-08-13 16:24:16.364591 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911f80 con 0x7fba680 -6958> 2014-08-13 16:24:16.364607 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911dc0 con 0x8c6f900 -6957> 2014-08-13 16:24:16.364624 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911c00 con 0x8c6f640 -6956> 2014-08-13 16:24:16.364642 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911a40 con 0x8c68b00 -6955> 2014-08-13 16:24:16.364658 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911880 con 0x8c68840 -6954> 2014-08-13 16:24:16.364677 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89116c0 con 0x8c3b440 -6953> 2014-08-13 16:24:16.364694 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911500 con 0x8c3b180 -6952> 2014-08-13 16:24:16.364720 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911340 con 0x8c68580 -6951> 2014-08-13 16:24:16.364742 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8911180 con 0x8c6b2e0 -6950> 2014-08-13 16:24:16.364762 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8910fc0 con 0x8c6b020 -6949> 2014-08-13 16:24:16.364784 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8910a80 con 0x8c6ad60 -6948> 2014-08-13 16:24:16.364804 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8910e00 con 0x8b6dc20 -6947> 2014-08-13 16:24:16.364823 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89101c0 con 0x7fb9b80 -6946> 2014-08-13 16:24:16.364845 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8910700 con 0x8ba11e0 -6945> 2014-08-13 16:24:16.364866 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89178c0 con 0x8ba0f20 -6944> 2014-08-13 16:24:16.364884 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917700 con 0x7fbbc80 -6943> 2014-08-13 16:24:16.364901 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917540 con 0x7fbbb20 -6942> 2014-08-13 16:24:16.364919 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917380 con 0x8c39760 -6941> 2014-08-13 16:24:16.364936 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89171c0 con 0x8c394a0 -6940> 2014-08-13 16:24:16.364954 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917000 con 0x7fbb9c0 -6939> 2014-08-13 16:24:16.364975 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916e40 con 0x7fbb860 -6938> 2014-08-13 16:24:16.364997 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916c80 con 0x7fb94a0 -6937> 2014-08-13 16:24:16.365019 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916ac0 con 0x7fba520 -6936> 2014-08-13 16:24:16.365036 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916900 con 0x7fbc200 -6935> 2014-08-13 16:24:16.365055 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916740 con 0x7fbbf40 -6934> 2014-08-13 16:24:16.365075 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916580 con 0x8c6aaa0 -6933> 2014-08-13 16:24:16.365101 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89163c0 con 0x8c6a7e0 -6932> 2014-08-13 16:24:16.365121 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916200 con 0x8bacd00 -6931> 2014-08-13 16:24:16.365142 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8916040 con 0x8baca40 -6930> 2014-08-13 16:24:16.365177 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915e80 con 0x8ba0c60 -6929> 2014-08-13 16:24:16.365195 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915cc0 con 0x8ba09a0 -6928> 2014-08-13 16:24:16.365212 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915b00 con 0x7ffdac0 -6927> 2014-08-13 16:24:16.365233 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915940 con 0x7ffe040 -6926> 2014-08-13 16:24:16.365252 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915780 con 0x8c6f380 -6925> 2014-08-13 16:24:16.365274 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89155c0 con 0x8c6f0c0 -6924> 2014-08-13 16:24:16.365294 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915400 con 0x7fbb700 -6923> 2014-08-13 16:24:16.365314 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915240 con 0x7fbb5a0 -6922> 2014-08-13 16:24:16.365334 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8915080 con 0x8d27900 -6921> 2014-08-13 16:24:16.365351 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914ec0 con 0x8d27640 -6920> 2014-08-13 16:24:16.365368 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914d00 con 0x8c6a520 -6919> 2014-08-13 16:24:16.365390 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914b40 con 0x8c6a260 -6918> 2014-08-13 16:24:16.365408 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914980 con 0x8d24ba0 -6917> 2014-08-13 16:24:16.365426 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89147c0 con 0x8d248e0 -6916> 2014-08-13 16:24:16.365447 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914600 con 0x8d26300 -6915> 2014-08-13 16:24:16.365474 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914440 con 0x8d26040 -6914> 2014-08-13 16:24:16.365492 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8914280 con 0x8d25d80 -6913> 2014-08-13 16:24:16.365512 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x89140c0 con 0x8d140a0 -6912> 2014-08-13 16:24:16.365531 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917e00 con 0x8d13de0 -6911> 2014-08-13 16:24:16.365547 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:16.361992) v2 -- ?+0 0x8917c40 con 0x8d13b20 -6910> 2014-08-13 16:24:16.365611 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.1.115:6839/62713 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91001c0 con 0x8ba06e0 -6909> 2014-08-13 16:24:16.365677 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.100.115:6846/62713 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x80d8a80 con 0x8c698c0 -6908> 2014-08-13 16:24:16.365705 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9102300 con 0x8c69600 -6907> 2014-08-13 16:24:16.365739 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8ee8c40 con 0x8c69340 -6906> 2014-08-13 16:24:16.365785 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100fc0 con 0x8c69080 -6905> 2014-08-13 16:24:16.365842 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899e580 con 0x8c68dc0 -6904> 2014-08-13 16:24:16.365867 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8999500 con 0x8c68b00 -6903> 2014-08-13 16:24:16.365900 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899d080 con 0x8c68840 -6902> 2014-08-13 16:24:16.365940 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9280fc0 con 0x8c68580 -6901> 2014-08-13 16:24:16.365967 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899cec0 con 0x8c6b2e0 -6900> 2014-08-13 16:24:16.365993 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899c600 con 0x8c6b020 -6899> 2014-08-13 16:24:16.366017 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8998e00 con 0x8c6ad60 -6898> 2014-08-13 16:24:16.366041 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e5780 con 0x8c6aaa0 -6897> 2014-08-13 16:24:16.366066 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e4980 con 0x8c6a7e0 -6896> 2014-08-13 16:24:16.366092 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8910c40 con 0x8c6a520 -6895> 2014-08-13 16:24:16.366118 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x89108c0 con 0x8c6a260 -6894> 2014-08-13 16:24:16.366141 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9285240 con 0x8c69fa0 -6893> 2014-08-13 16:24:16.366174 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9107380 con 0x8c39fa0 -6892> 2014-08-13 16:24:16.366200 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8999c00 con 0x8c39ce0 -6891> 2014-08-13 16:24:16.366227 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x89996c0 con 0x8c39a20 -6890> 2014-08-13 16:24:16.366258 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899c980 con 0x8c39760 -6889> 2014-08-13 16:24:16.366287 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9281500 con 0x8c394a0 -6888> 2014-08-13 16:24:16.366312 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c4600 con 0x8c391e0 -6887> 2014-08-13 16:24:16.366337 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9107e00 con 0x8c38c60 -6886> 2014-08-13 16:24:16.366362 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eecb40 con 0x8c389a0 -6885> 2014-08-13 16:24:16.366385 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eec280 con 0x8c3b9c0 -6884> 2014-08-13 16:24:16.366410 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eef380 con 0x8c3b700 -6883> 2014-08-13 16:24:16.366432 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9287e00 con 0x8c3b440 -6882> 2014-08-13 16:24:16.366459 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899c7c0 con 0x8c3b180 -6881> 2014-08-13 16:24:16.366483 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9104280 con 0x8c3aec0 -6880> 2014-08-13 16:24:16.366507 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9101dc0 con 0x8c3ac00 -6879> 2014-08-13 16:24:16.366530 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9103b80 con 0x8c3a940 -6878> 2014-08-13 16:24:16.366553 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106e40 con 0x8c3a680 -6877> 2014-08-13 16:24:16.366575 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775d5c0 con 0x8c3a3c0 -6876> 2014-08-13 16:24:16.366599 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775ee40 con 0x8c3a100 -6875> 2014-08-13 16:24:16.366622 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7759180 con 0x8c3d6a0 -6874> 2014-08-13 16:24:16.366650 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c5240 con 0x8c3d3e0 -6873> 2014-08-13 16:24:16.366674 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9104600 con 0x8c3d120 -6872> 2014-08-13 16:24:16.366699 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x92839c0 con 0x8c3ce60 -6871> 2014-08-13 16:24:16.366722 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775d400 con 0x8c3cba0 -6870> 2014-08-13 16:24:16.366746 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eeee40 con 0x8c3c8e0 -6869> 2014-08-13 16:24:16.366770 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899dcc0 con 0x8c3c620 -6868> 2014-08-13 16:24:16.366796 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100380 con 0x8c3c360 -6867> 2014-08-13 16:24:16.366820 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7758e00 con 0x8c3c0a0 -6866> 2014-08-13 16:24:16.366844 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7758380 con 0x8c3bde0 -6865> 2014-08-13 16:24:16.366867 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100e00 con 0x8c3e880 -6864> 2014-08-13 16:24:16.366890 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9283640 con 0x8c3e5c0 -6863> 2014-08-13 16:24:16.366913 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9103100 con 0x8c3e300 -6862> 2014-08-13 16:24:16.366936 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105cc0 con 0x8c618c0 -6861> 2014-08-13 16:24:16.366958 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775c0c0 con 0x8c6fe80 -6860> 2014-08-13 16:24:16.366985 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899f8c0 con 0x8c6fbc0 -6859> 2014-08-13 16:24:16.367008 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8910000 con 0x8c6f900 -6858> 2014-08-13 16:24:16.367032 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899c440 con 0x8c6f640 -6857> 2014-08-13 16:24:16.367055 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e7e00 con 0x8c6f380 -6856> 2014-08-13 16:24:16.367079 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e6c80 con 0x8c6f0c0 -6855> 2014-08-13 16:24:16.367102 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eebf00 con 0x8d223c0 -6854> 2014-08-13 16:24:16.367129 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899bb80 con 0x8d22100 -6853> 2014-08-13 16:24:16.367152 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.1.115:6814/59915 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8ee9f80 con 0x8d21e40 -6852> 2014-08-13 16:24:16.367185 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.100.115:6831/59915 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775a840 con 0x8d21b80 -6851> 2014-08-13 16:24:16.367209 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8998540 con 0x8d218c0 -6850> 2014-08-13 16:24:16.367234 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9280c40 con 0x8d21600 -6849> 2014-08-13 16:24:16.367259 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91016c0 con 0x8d21340 -6848> 2014-08-13 16:24:16.367284 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x89981c0 con 0x8d21080 -6847> 2014-08-13 16:24:16.367307 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899e900 con 0x8d20dc0 -6846> 2014-08-13 16:24:16.367331 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9101500 con 0x8d20b00 -6845> 2014-08-13 16:24:16.367358 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eee580 con 0x8d20840 -6844> 2014-08-13 16:24:16.367383 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899e3c0 con 0x8d25960 -6843> 2014-08-13 16:24:16.367408 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106900 con 0x8d256a0 -6842> 2014-08-13 16:24:16.367431 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899f000 con 0x8d253e0 -6841> 2014-08-13 16:24:16.367454 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9284600 con 0x8d25120 -6840> 2014-08-13 16:24:16.367477 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105780 con 0x8d24e60 -6839> 2014-08-13 16:24:16.367502 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8910540 con 0x8d24ba0 -6838> 2014-08-13 16:24:16.367526 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e7380 con 0x8d248e0 -6837> 2014-08-13 16:24:16.367549 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9286740 con 0x8d24620 -6836> 2014-08-13 16:24:16.367573 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9283f00 con 0x8d24360 -6835> 2014-08-13 16:24:16.367595 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105b00 con 0x8d240a0 -6834> 2014-08-13 16:24:16.367618 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9285940 con 0x8d23de0 -6833> 2014-08-13 16:24:16.367641 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8998c40 con 0x8d23b20 -6832> 2014-08-13 16:24:16.367665 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899a140 con 0x8d27bc0 -6831> 2014-08-13 16:24:16.367688 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e5080 con 0x8d27900 -6830> 2014-08-13 16:24:16.367712 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899bf00 con 0x8d27640 -6829> 2014-08-13 16:24:16.367736 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899a4c0 con 0x8d27380 -6828> 2014-08-13 16:24:16.367762 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9286040 con 0x8d270c0 -6827> 2014-08-13 16:24:16.367785 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100700 con 0x8d26e00 -6826> 2014-08-13 16:24:16.367809 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9280700 con 0x8d26b40 -6825> 2014-08-13 16:24:16.367832 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9103f00 con 0x8d26880 -6824> 2014-08-13 16:24:16.367858 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106200 con 0x8d265c0 -6823> 2014-08-13 16:24:16.367881 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e6200 con 0x8d26300 -6822> 2014-08-13 16:24:16.367905 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105080 con 0x8d26040 -6821> 2014-08-13 16:24:16.367928 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c3f00 con 0x8d25d80 -6820> 2014-08-13 16:24:16.367951 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775a680 con 0x8d140a0 -6819> 2014-08-13 16:24:16.367973 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105240 con 0x8d13de0 -6818> 2014-08-13 16:24:16.367998 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9102680 con 0x8d13b20 -6817> 2014-08-13 16:24:16.368021 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775db00 con 0x78844c0 -6816> 2014-08-13 16:24:16.368045 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7d00a80 con 0x7884200 -6815> 2014-08-13 16:24:16.368073 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c0380 con 0x78848e0 -6814> 2014-08-13 16:24:16.368098 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eea140 con 0x7884620 -6813> 2014-08-13 16:24:16.368125 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c5400 con 0x794c360 -6812> 2014-08-13 16:24:16.368148 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c3100 con 0x794e300 -6811> 2014-08-13 16:24:16.368178 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8ee88c0 con 0x794e040 -6810> 2014-08-13 16:24:16.368203 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899b100 con 0x794dd80 -6809> 2014-08-13 16:24:16.368228 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899f1c0 con 0x7ffd960 -6808> 2014-08-13 16:24:16.368252 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9107c40 con 0x7ffdc20 -6807> 2014-08-13 16:24:16.368276 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e5b00 con 0x7ffdac0 -6806> 2014-08-13 16:24:16.368301 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e7000 con 0x7ffe040 -6805> 2014-08-13 16:24:16.368324 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9107a80 con 0x7ffdee0 -6804> 2014-08-13 16:24:16.368348 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9101f80 con 0x7ffdd80 -6803> 2014-08-13 16:24:16.368370 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eef1c0 con 0x8b68c60 -6802> 2014-08-13 16:24:16.368394 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9102840 con 0x8b68b00 -6801> 2014-08-13 16:24:16.368417 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eecd00 con 0x8b689a0 -6800> 2014-08-13 16:24:16.368440 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899f380 con 0x8b68840 -6799> 2014-08-13 16:24:16.368462 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106580 con 0x8b6e1a0 -6798> 2014-08-13 16:24:16.368486 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8998fc0 con 0x8b6e040 -6797> 2014-08-13 16:24:16.368511 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x92863c0 con 0x8b6dee0 -6796> 2014-08-13 16:24:16.368535 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8998a80 con 0x8b6dd80 -6795> 2014-08-13 16:24:16.368558 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e3100 con 0x8b6dc20 -6794> 2014-08-13 16:24:16.368581 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899cd00 con 0x7fb9b80 -6793> 2014-08-13 16:24:16.368606 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9281880 con 0x7fb9a20 -6792> 2014-08-13 16:24:16.368630 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9280540 con 0x7fb98c0 -6791> 2014-08-13 16:24:16.368653 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91071c0 con 0x7fb9760 -6790> 2014-08-13 16:24:16.368676 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9102f40 con 0x7fb9600 -6789> 2014-08-13 16:24:16.368698 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9287540 con 0x7fb94a0 -6788> 2014-08-13 16:24:16.368722 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e5e80 con 0x7fba520 -6787> 2014-08-13 16:24:16.368744 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c1dc0 con 0x7fba3c0 -6786> 2014-08-13 16:24:16.368768 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9104d00 con 0x7fba260 -6785> 2014-08-13 16:24:16.368791 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eec600 con 0x7fba100 -6784> 2014-08-13 16:24:16.368821 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eec7c0 con 0x7fb9fa0 -6783> 2014-08-13 16:24:16.368845 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899fc40 con 0x7fb9e40 -6782> 2014-08-13 16:24:16.368871 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eeeac0 con 0x7fb9ce0 -6781> 2014-08-13 16:24:16.368895 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eee900 con 0x7fbb020 -6780> 2014-08-13 16:24:16.368918 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899b800 con 0x7fbaec0 -6779> 2014-08-13 16:24:16.368940 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9103800 con 0x7fbad60 -6778> 2014-08-13 16:24:16.368964 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91032c0 con 0x7fbac00 -6777> 2014-08-13 16:24:16.368989 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899abc0 con 0x7fbaaa0 -6776> 2014-08-13 16:24:16.369013 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eee200 con 0x7fba940 -6775> 2014-08-13 16:24:16.369036 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e0380 con 0x7fba7e0 -6774> 2014-08-13 16:24:16.369059 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8999180 con 0x7fba680 -6773> 2014-08-13 16:24:16.369081 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899bd40 con 0x7fbbc80 -6772> 2014-08-13 16:24:16.369104 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e6900 con 0x7fbbb20 -6771> 2014-08-13 16:24:16.369126 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e3480 con 0x7fbb9c0 -6770> 2014-08-13 16:24:16.369149 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e4600 con 0x7fbb860 -6769> 2014-08-13 16:24:16.369178 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e6580 con 0x7fbb700 -6768> 2014-08-13 16:24:16.369203 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e7a80 con 0x7fbb5a0 -6767> 2014-08-13 16:24:16.369226 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eee3c0 con 0x7fbb440 -6766> 2014-08-13 16:24:16.369250 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899b2c0 con 0x7fbb2e0 -6765> 2014-08-13 16:24:16.369273 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9282680 con 0x7fbb180 -6764> 2014-08-13 16:24:16.369299 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91040c0 con 0x7fbca40 -6763> 2014-08-13 16:24:16.369322 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100a80 con 0x7fbc780 -6762> 2014-08-13 16:24:16.369347 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9100c40 con 0x7fbc4c0 -6761> 2014-08-13 16:24:16.369372 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e3800 con 0x7fbc200 -6760> 2014-08-13 16:24:16.369396 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e7700 con 0x7fbbf40 -6759> 2014-08-13 16:24:16.369419 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x775c280 con 0x8babde0 -6758> 2014-08-13 16:24:16.369442 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c3800 con 0x8babb20 -6757> 2014-08-13 16:24:16.369464 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x80dbf00 con 0x8bab860 -6756> 2014-08-13 16:24:16.369487 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9102140 con 0x8bab5a0 -6755> 2014-08-13 16:24:16.369509 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9101880 con 0x8bab2e0 -6754> 2014-08-13 16:24:16.369532 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eee040 con 0x8bab020 -6753> 2014-08-13 16:24:16.369554 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e5400 con 0x8bacd00 -6752> 2014-08-13 16:24:16.369582 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e3b80 con 0x8baca40 -6751> 2014-08-13 16:24:16.369606 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91063c0 con 0x8bac780 -6750> 2014-08-13 16:24:16.369634 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8999f80 con 0x8bac4c0 -6749> 2014-08-13 16:24:16.369657 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9285cc0 con 0x8bac200 -6748> 2014-08-13 16:24:16.369680 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9284980 con 0x8babf40 -6747> 2014-08-13 16:24:16.369703 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106740 con 0x8bae040 -6746> 2014-08-13 16:24:16.369726 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x92855c0 con 0x8badd80 -6745> 2014-08-13 16:24:16.369751 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7759dc0 con 0x8badac0 -6744> 2014-08-13 16:24:16.369775 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x7d039c0 con 0x8bad800 -6743> 2014-08-13 16:24:16.369797 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91055c0 con 0x8bad540 -6742> 2014-08-13 16:24:16.369821 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899aa00 con 0x8bad280 -6741> 2014-08-13 16:24:16.369844 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x91024c0 con 0x8bafe80 -6740> 2014-08-13 16:24:16.369867 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x8eef000 con 0x8bafbc0 -6739> 2014-08-13 16:24:16.369892 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9107000 con 0x8baf900 -6738> 2014-08-13 16:24:16.369916 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e0000 con 0x8baf640 -6737> 2014-08-13 16:24:16.369939 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9283d40 con 0x8baf380 -6736> 2014-08-13 16:24:16.369962 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9105e80 con 0x8baf0c0 -6735> 2014-08-13 16:24:16.369984 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x77588c0 con 0x8baee00 -6734> 2014-08-13 16:24:16.370008 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x81c7380 con 0x8ba1a20 -6733> 2014-08-13 16:24:16.370030 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x9106c80 con 0x8ba1760 -6732> 2014-08-13 16:24:16.370053 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899eac0 con 0x8ba14a0 -6731> 2014-08-13 16:24:16.370076 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e4d00 con 0x8ba11e0 -6730> 2014-08-13 16:24:16.370101 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e3f00 con 0x8ba0f20 -6729> 2014-08-13 16:24:16.370126 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x87e4280 con 0x8ba0c60 -6728> 2014-08-13 16:24:16.370152 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 3 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.361992) v2 ==== 47+0+0 (1423996366 0 0) 0x899c0c0 con 0x8ba09a0 -6727> 2014-08-13 16:24:16.457480 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.457255) v2 ==== 47+0+0 (2427256230 0 0) 0x7d07540 con 0x7cae880 -6726> 2014-08-13 16:24:16.457526 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.457255) v2 -- ?+0 0x8c8d080 con 0x7cae880 -6725> 2014-08-13 16:24:16.457587 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.457255) v2 ==== 47+0+0 (2427256230 0 0) 0x7d00700 con 0x7cae040 -6724> 2014-08-13 16:24:16.457654 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.457255) v2 -- ?+0 0x775f540 con 0x7cae040 -6723> 2014-08-13 16:24:16.499894 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 15 ==== replica scrub(pg: 22.69f,from:0'0,to:194215'528,epoch:194449,start:13261e9f//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (3715557826 0 0) 0x7cf3a80 con 0x5112aa0 -6722> 2014-08-13 16:24:16.502716 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6808/62751 -- osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+5564 0x8638000 con 0x5112aa0 -6721> 2014-08-13 16:24:16.554685 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 16 ==== osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1634750295 0 0) 0x8638000 con 0x5112aa0 -6720> 2014-08-13 16:24:16.554742 7fd93dff0700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554510, event: header_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6719> 2014-08-13 16:24:16.554758 7fd93dff0700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554513, event: throttled, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6718> 2014-08-13 16:24:16.554769 7fd93dff0700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554613, event: all_read, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6717> 2014-08-13 16:24:16.554778 7fd93dff0700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554735, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6716> 2014-08-13 16:24:16.554789 7fd93dff0700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554789, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6715> 2014-08-13 16:24:16.554830 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.334 192.168.1.104:6808/62751 17 ==== pg_info(1 pgs e194449:22.69f) v4 ==== 714+0+0 (1023858175 0 0) 0x8910000 con 0x5112aa0 -6714> 2014-08-13 16:24:16.554850 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554653, event: header_read, request: pg_info(1 pgs e194449:22.69f) v4 -6713> 2014-08-13 16:24:16.554858 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554654, event: throttled, request: pg_info(1 pgs e194449:22.69f) v4 -6712> 2014-08-13 16:24:16.554864 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554731, event: all_read, request: pg_info(1 pgs e194449:22.69f) v4 -6711> 2014-08-13 16:24:16.554869 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554848, event: dispatched, request: pg_info(1 pgs e194449:22.69f) v4 -6710> 2014-08-13 16:24:16.554875 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554875, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.69f) v4 -6709> 2014-08-13 16:24:16.554884 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.554883, event: started, request: pg_info(1 pgs e194449:22.69f) v4 -6708> 2014-08-13 16:24:16.554893 7fd9371e5700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554892, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6707> 2014-08-13 16:24:16.554937 7fd9371e5700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554937, event: started, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6706> 2014-08-13 16:24:16.554957 7fd9371e5700 5 -- op tracker -- , seq: 781, time: 2014-08-13 16:24:16.554957, event: done, request: osd_sub_op(unknown.0.0:0 22.69f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6705> 2014-08-13 16:24:16.555022 7fd93dff0700 5 -- op tracker -- , seq: 782, time: 2014-08-13 16:24:16.555021, event: done, request: pg_info(1 pgs e194449:22.69f) v4 -6704> 2014-08-13 16:24:16.854492 7fd94ac49700 5 osd.4 194449 tick -6703> 2014-08-13 16:24:16.867275 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 24 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1182254683 0 0) 0x8bc0000 con 0x4fe0000 -6702> 2014-08-13 16:24:16.867328 7fd93dff0700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867065, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6701> 2014-08-13 16:24:16.867344 7fd93dff0700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867069, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6700> 2014-08-13 16:24:16.867354 7fd93dff0700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867192, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6699> 2014-08-13 16:24:16.867363 7fd93dff0700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867321, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6698> 2014-08-13 16:24:16.867374 7fd93dff0700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867374, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6697> 2014-08-13 16:24:16.867435 7fd9371e5700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867435, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6696> 2014-08-13 16:24:16.867475 7fd9371e5700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867475, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6695> 2014-08-13 16:24:16.867493 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op_reply(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003980 con 0x4fe0000 -6694> 2014-08-13 16:24:16.867524 7fd9371e5700 5 -- op tracker -- , seq: 783, time: 2014-08-13 16:24:16.867524, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6693> 2014-08-13 16:24:16.883813 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 25 ==== replica scrub(pg: 22.29a,from:0'0,to:194215'488,epoch:194449,start:0//0//-1,end:f36fba9a//0//-1,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (3738665850 0 0) 0x7e7d100 con 0x4fe0000 -6692> 2014-08-13 16:24:16.911887 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.911820) v2 ==== 47+0+0 (3085165551 0 0) 0x775c440 con 0x7d28160 -6691> 2014-08-13 16:24:16.911938 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.911820) v2 -- ?+0 0x7d07540 con 0x7d28160 -6690> 2014-08-13 16:24:16.911967 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.911820) v2 ==== 47+0+0 (3085165551 0 0) 0x7d04d00 con 0x7d2a260 -6689> 2014-08-13 16:24:16.912021 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.911820) v2 -- ?+0 0x7d00700 con 0x7d2a260 -6688> 2014-08-13 16:24:16.986827 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.986567) v2 ==== 47+0+0 (714306905 0 0) 0x9282680 con 0x7eba680 -6687> 2014-08-13 16:24:16.986883 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.986567) v2 -- ?+0 0x7d04d00 con 0x7eba680 -6686> 2014-08-13 16:24:16.987029 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:16.986567) v2 ==== 47+0+0 (714306905 0 0) 0x9280540 con 0x7eb91e0 -6685> 2014-08-13 16:24:16.987069 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:16.986567) v2 -- ?+0 0x775c440 con 0x7eb91e0 -6684> 2014-08-13 16:24:17.015824 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 17 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x86dcd00 con 0x510fd20 -6683> 2014-08-13 16:24:17.015879 7fd93dff0700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.015614, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6682> 2014-08-13 16:24:17.015895 7fd93dff0700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.015617, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6681> 2014-08-13 16:24:17.015906 7fd93dff0700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.015743, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6680> 2014-08-13 16:24:17.015915 7fd93dff0700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.015872, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6679> 2014-08-13 16:24:17.015931 7fd93dff0700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.015931, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6678> 2014-08-13 16:24:17.016077 7fd9367e4700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.016076, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6677> 2014-08-13 16:24:17.016130 7fd9367e4700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.016129, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6676> 2014-08-13 16:24:17.016150 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47a80 con 0x510fd20 -6675> 2014-08-13 16:24:17.016198 7fd9367e4700 5 -- op tracker -- , seq: 784, time: 2014-08-13 16:24:17.016198, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6674> 2014-08-13 16:24:17.017310 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 18 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x86da100 con 0x510fd20 -6673> 2014-08-13 16:24:17.017364 7fd93dff0700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017137, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6672> 2014-08-13 16:24:17.017381 7fd93dff0700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017138, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6671> 2014-08-13 16:24:17.017391 7fd93dff0700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017229, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6670> 2014-08-13 16:24:17.017404 7fd93dff0700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017358, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6669> 2014-08-13 16:24:17.017415 7fd93dff0700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017415, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6668> 2014-08-13 16:24:17.017475 7fd9371e5700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017475, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6667> 2014-08-13 16:24:17.017514 7fd9371e5700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017514, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6666> 2014-08-13 16:24:17.017529 7fd9371e5700 5 -- op tracker -- , seq: 785, time: 2014-08-13 16:24:17.017529, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6665> 2014-08-13 16:24:17.043785 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.043407) v2 ==== 47+0+0 (3060121686 0 0) 0x7d05940 con 0x7eb8c60 -6664> 2014-08-13 16:24:17.043794 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.043407) v2 ==== 47+0+0 (3060121686 0 0) 0x8c8af40 con 0x7eba100 -6663> 2014-08-13 16:24:17.043844 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.043407) v2 -- ?+0 0x9280540 con 0x7eb8c60 -6662> 2014-08-13 16:24:17.043904 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.043407) v2 -- ?+0 0x9282680 con 0x7eba100 -6661> 2014-08-13 16:24:17.545180 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 25 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x86d8000 con 0x5112520 -6660> 2014-08-13 16:24:17.545236 7fd93dff0700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.544969, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6659> 2014-08-13 16:24:17.545252 7fd93dff0700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.544972, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6658> 2014-08-13 16:24:17.545262 7fd93dff0700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545087, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6657> 2014-08-13 16:24:17.545271 7fd93dff0700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545229, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6656> 2014-08-13 16:24:17.545292 7fd93dff0700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545292, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6655> 2014-08-13 16:24:17.545401 7fd9367e4700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545401, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6654> 2014-08-13 16:24:17.545458 7fd9367e4700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545457, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6653> 2014-08-13 16:24:17.545478 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47800 con 0x5112520 -6652> 2014-08-13 16:24:17.545513 7fd9367e4700 5 -- op tracker -- , seq: 786, time: 2014-08-13 16:24:17.545513, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6651> 2014-08-13 16:24:17.546598 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 26 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1209768036 0 0) 0x863e300 con 0x5112520 -6650> 2014-08-13 16:24:17.546649 7fd93dff0700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546409, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6649> 2014-08-13 16:24:17.546665 7fd93dff0700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546412, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6648> 2014-08-13 16:24:17.546675 7fd93dff0700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546515, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6647> 2014-08-13 16:24:17.546685 7fd93dff0700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546643, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6646> 2014-08-13 16:24:17.546699 7fd93dff0700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546699, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6645> 2014-08-13 16:24:17.546778 7fd9371e5700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546778, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6644> 2014-08-13 16:24:17.546816 7fd9371e5700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546816, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6643> 2014-08-13 16:24:17.546831 7fd9371e5700 5 -- op tracker -- , seq: 787, time: 2014-08-13 16:24:17.546831, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6642> 2014-08-13 16:24:17.559193 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.558824) v2 ==== 47+0+0 (155705651 0 0) 0x9286740 con 0x7cae880 -6641> 2014-08-13 16:24:17.559249 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.558824) v2 -- ?+0 0x7d05940 con 0x7cae880 -6640> 2014-08-13 16:24:17.559237 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.558824) v2 ==== 47+0+0 (155705651 0 0) 0x9286040 con 0x7cae040 -6639> 2014-08-13 16:24:17.559326 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.558824) v2 -- ?+0 0x8c8af40 con 0x7cae040 -6638> 2014-08-13 16:24:17.585483 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.585193) v2 ==== 47+0+0 (1057832899 0 0) 0x9282300 con 0x7edbc80 -6637> 2014-08-13 16:24:17.585511 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.585193) v2 ==== 47+0+0 (1057832899 0 0) 0x9286ac0 con 0x7ffe9e0 -6636> 2014-08-13 16:24:17.585536 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.585193) v2 -- ?+0 0x9286040 con 0x7edbc80 -6635> 2014-08-13 16:24:17.585578 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.585193) v2 -- ?+0 0x9286740 con 0x7ffe9e0 -6634> 2014-08-13 16:24:17.588120 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.587904) v2 ==== 47+0+0 (3703710192 0 0) 0x87e40c0 con 0x7ffeb40 -6633> 2014-08-13 16:24:17.588185 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.587904) v2 -- ?+0 0x9286ac0 con 0x7ffeb40 -6632> 2014-08-13 16:24:17.588200 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 3 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.587904) v2 ==== 47+0+0 (3703710192 0 0) 0x87e7c40 con 0x7edbde0 -6631> 2014-08-13 16:24:17.588243 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.587904) v2 -- ?+0 0x9282300 con 0x7edbde0 -6630> 2014-08-13 16:24:17.636999 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.636627) v2 ==== 47+0+0 (3068296600 0 0) 0x9283640 con 0x7eda100 -6629> 2014-08-13 16:24:17.637022 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.636627) v2 ==== 47+0+0 (3068296600 0 0) 0x9285240 con 0x7edc0a0 -6628> 2014-08-13 16:24:17.637053 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.636627) v2 -- ?+0 0x87e40c0 con 0x7eda100 -6627> 2014-08-13 16:24:17.637136 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.636627) v2 -- ?+0 0x87e7c40 con 0x7edc0a0 -6626> 2014-08-13 16:24:17.698136 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 15 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x8bc4200 con 0x4e498c0 -6625> 2014-08-13 16:24:17.698202 7fd93dff0700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.697936, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6624> 2014-08-13 16:24:17.698219 7fd93dff0700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.697941, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6623> 2014-08-13 16:24:17.698229 7fd93dff0700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698051, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6622> 2014-08-13 16:24:17.698238 7fd93dff0700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698195, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6621> 2014-08-13 16:24:17.698253 7fd93dff0700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698253, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6620> 2014-08-13 16:24:17.698388 7fd9367e4700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698388, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6619> 2014-08-13 16:24:17.698453 7fd9367e4700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698453, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6618> 2014-08-13 16:24:17.698473 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46680 con 0x4e498c0 -6617> 2014-08-13 16:24:17.698507 7fd9367e4700 5 -- op tracker -- , seq: 788, time: 2014-08-13 16:24:17.698507, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6616> 2014-08-13 16:24:17.699694 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 16 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x8bc2c00 con 0x4e498c0 -6615> 2014-08-13 16:24:17.699750 7fd93dff0700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699517, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6614> 2014-08-13 16:24:17.699766 7fd93dff0700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699520, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6613> 2014-08-13 16:24:17.699776 7fd93dff0700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699614, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6612> 2014-08-13 16:24:17.699790 7fd93dff0700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699743, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6611> 2014-08-13 16:24:17.699807 7fd93dff0700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699807, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6610> 2014-08-13 16:24:17.699865 7fd9371e5700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699864, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6609> 2014-08-13 16:24:17.699903 7fd9371e5700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699903, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6608> 2014-08-13 16:24:17.699918 7fd9371e5700 5 -- op tracker -- , seq: 789, time: 2014-08-13 16:24:17.699918, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6607> 2014-08-13 16:24:17.768581 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.768165) v2 ==== 47+0+0 (969527662 0 0) 0x9280380 con 0x7ebb860 -6606> 2014-08-13 16:24:17.768634 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.768165) v2 -- ?+0 0x9283640 con 0x7ebb860 -6605> 2014-08-13 16:24:17.768625 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.768165) v2 ==== 47+0+0 (969527662 0 0) 0x87e5cc0 con 0x7ebce60 -6604> 2014-08-13 16:24:17.768678 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.768165) v2 -- ?+0 0x9285240 con 0x7ebce60 -6603> 2014-08-13 16:24:17.854637 7fd94ac49700 5 osd.4 194449 tick -6602> 2014-08-13 16:24:17.915932 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.915562) v2 ==== 47+0+0 (2468604728 0 0) 0x9283640 con 0x7eda680 -6601> 2014-08-13 16:24:17.915987 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.915562) v2 -- ?+0 0x9280380 con 0x7eda680 -6600> 2014-08-13 16:24:17.915993 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.915562) v2 ==== 47+0+0 (2468604728 0 0) 0x9286200 con 0x7edc620 -6599> 2014-08-13 16:24:17.916047 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.915562) v2 -- ?+0 0x87e5cc0 con 0x7edc620 -6598> 2014-08-13 16:24:17.918063 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.917453) v2 ==== 47+0+0 (48695921 0 0) 0x92816c0 con 0x7ab9a20 -6597> 2014-08-13 16:24:17.918179 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.917453) v2 ==== 47+0+0 (48695921 0 0) 0x9287a80 con 0x7ab9ce0 -6596> 2014-08-13 16:24:17.918111 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.917453) v2 -- ?+0 0x9283640 con 0x7ab9a20 -6595> 2014-08-13 16:24:17.918272 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.917453) v2 -- ?+0 0x9286200 con 0x7ab9ce0 -6594> 2014-08-13 16:24:17.945751 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 19 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x8bc5800 con 0x508bde0 -6593> 2014-08-13 16:24:17.945806 7fd93dff0700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945550, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6592> 2014-08-13 16:24:17.945826 7fd93dff0700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945555, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6591> 2014-08-13 16:24:17.945840 7fd93dff0700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945668, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6590> 2014-08-13 16:24:17.945852 7fd93dff0700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945799, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6589> 2014-08-13 16:24:17.945867 7fd93dff0700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945867, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6588> 2014-08-13 16:24:17.945973 7fd9367e4700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.945972, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6587> 2014-08-13 16:24:17.946033 7fd9367e4700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.946033, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6586> 2014-08-13 16:24:17.946057 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46e00 con 0x508bde0 -6585> 2014-08-13 16:24:17.946090 7fd9367e4700 5 -- op tracker -- , seq: 790, time: 2014-08-13 16:24:17.946089, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6584> 2014-08-13 16:24:17.947088 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 20 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x8bc4d00 con 0x508bde0 -6583> 2014-08-13 16:24:17.947143 7fd93dff0700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.946910, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6582> 2014-08-13 16:24:17.947164 7fd93dff0700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.946912, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6581> 2014-08-13 16:24:17.947191 7fd93dff0700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947009, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6580> 2014-08-13 16:24:17.947204 7fd93dff0700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947137, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6579> 2014-08-13 16:24:17.947219 7fd93dff0700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947219, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6578> 2014-08-13 16:24:17.947297 7fd9371e5700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947297, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6577> 2014-08-13 16:24:17.947335 7fd9371e5700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947335, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6576> 2014-08-13 16:24:17.947351 7fd9371e5700 5 -- op tracker -- , seq: 791, time: 2014-08-13 16:24:17.947350, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6575> 2014-08-13 16:24:17.990601 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.990717) v2 ==== 47+0+0 (2588885469 0 0) 0x9286900 con 0x7d2a520 -6574> 2014-08-13 16:24:17.990656 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.990717) v2 -- ?+0 0x9287a80 con 0x7d2a520 -6573> 2014-08-13 16:24:17.990649 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:17.990717) v2 ==== 47+0+0 (2588885469 0 0) 0x9282bc0 con 0x7cae5c0 -6572> 2014-08-13 16:24:17.990690 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:17.990717) v2 -- ?+0 0x92816c0 con 0x7cae5c0 -6571> 2014-08-13 16:24:18.015523 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 15 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x7ece300 con 0x5111fa0 -6570> 2014-08-13 16:24:18.015567 7fd93dff0700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015369, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6569> 2014-08-13 16:24:18.015584 7fd93dff0700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015372, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6568> 2014-08-13 16:24:18.015594 7fd93dff0700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015483, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6567> 2014-08-13 16:24:18.015603 7fd93dff0700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015562, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6566> 2014-08-13 16:24:18.015614 7fd93dff0700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015614, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6565> 2014-08-13 16:24:18.015739 7fd9367e4700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015738, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6564> 2014-08-13 16:24:18.015797 7fd9367e4700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015797, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6563> 2014-08-13 16:24:18.015817 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003e80 con 0x5111fa0 -6562> 2014-08-13 16:24:18.015850 7fd9367e4700 5 -- op tracker -- , seq: 792, time: 2014-08-13 16:24:18.015849, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6561> 2014-08-13 16:24:18.016199 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 19 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x8bc0b00 con 0x510fd20 -6560> 2014-08-13 16:24:18.016242 7fd93dff0700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.015986, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6559> 2014-08-13 16:24:18.016258 7fd93dff0700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.015988, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6558> 2014-08-13 16:24:18.016268 7fd93dff0700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016086, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6557> 2014-08-13 16:24:18.016278 7fd93dff0700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016237, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6556> 2014-08-13 16:24:18.016294 7fd93dff0700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016294, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6555> 2014-08-13 16:24:18.016367 7fd9371e5700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016367, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6554> 2014-08-13 16:24:18.016413 7fd9371e5700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016413, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6553> 2014-08-13 16:24:18.016433 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003c00 con 0x510fd20 -6552> 2014-08-13 16:24:18.016473 7fd9371e5700 5 -- op tracker -- , seq: 793, time: 2014-08-13 16:24:18.016473, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6551> 2014-08-13 16:24:18.016948 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 16 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x8bc3700 con 0x5111fa0 -6550> 2014-08-13 16:24:18.016996 7fd93dff0700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.016811, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6549> 2014-08-13 16:24:18.017013 7fd93dff0700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.016814, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6548> 2014-08-13 16:24:18.017023 7fd93dff0700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.016911, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6547> 2014-08-13 16:24:18.017032 7fd93dff0700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.016990, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6546> 2014-08-13 16:24:18.017049 7fd93dff0700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.017049, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6545> 2014-08-13 16:24:18.017149 7fd9367e4700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.017149, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6544> 2014-08-13 16:24:18.017214 7fd9367e4700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.017214, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6543> 2014-08-13 16:24:18.017231 7fd9367e4700 5 -- op tracker -- , seq: 794, time: 2014-08-13 16:24:18.017230, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6542> 2014-08-13 16:24:18.017583 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 20 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x8bc4d00 con 0x510fd20 -6541> 2014-08-13 16:24:18.017634 7fd93dff0700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017435, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6540> 2014-08-13 16:24:18.017650 7fd93dff0700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017436, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6539> 2014-08-13 16:24:18.017661 7fd93dff0700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017522, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6538> 2014-08-13 16:24:18.017670 7fd93dff0700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017628, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6537> 2014-08-13 16:24:18.017687 7fd93dff0700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017687, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6536> 2014-08-13 16:24:18.017788 7fd9371e5700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017787, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6535> 2014-08-13 16:24:18.017828 7fd9371e5700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017828, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6534> 2014-08-13 16:24:18.017844 7fd9371e5700 5 -- op tracker -- , seq: 795, time: 2014-08-13 16:24:18.017844, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6533> 2014-08-13 16:24:18.060452 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.060088) v2 ==== 47+0+0 (3221172328 0 0) 0x9284600 con 0x7cae880 -6532> 2014-08-13 16:24:18.060458 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.060088) v2 ==== 47+0+0 (3221172328 0 0) 0x9285940 con 0x7cae040 -6531> 2014-08-13 16:24:18.060520 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.060088) v2 -- ?+0 0x9282bc0 con 0x7cae880 -6530> 2014-08-13 16:24:18.060614 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.060088) v2 -- ?+0 0x9286900 con 0x7cae040 -6529> 2014-08-13 16:24:18.091171 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.091029) v2 ==== 47+0+0 (2933273708 0 0) 0x7d04980 con 0x7d2cba0 -6528> 2014-08-13 16:24:18.091219 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.091029) v2 -- ?+0 0x9285940 con 0x7d2cba0 -6527> 2014-08-13 16:24:18.091215 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.091029) v2 ==== 47+0+0 (2933273708 0 0) 0x87e01c0 con 0x7d28420 -6526> 2014-08-13 16:24:18.091262 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.091029) v2 -- ?+0 0x9284600 con 0x7d28420 -6525> 2014-08-13 16:24:18.244640 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.244084) v2 ==== 47+0+0 (2093677951 0 0) 0x9284600 con 0x7ebc620 -6524> 2014-08-13 16:24:18.244656 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.244084) v2 ==== 47+0+0 (2093677951 0 0) 0x9282bc0 con 0x7eb9b80 -6523> 2014-08-13 16:24:18.244696 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.244084) v2 -- ?+0 0x87e01c0 con 0x7ebc620 -6522> 2014-08-13 16:24:18.244807 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.244084) v2 -- ?+0 0x7d04980 con 0x7eb9b80 -6521> 2014-08-13 16:24:18.394928 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.394311) v2 ==== 47+0+0 (2695524058 0 0) 0x7d01f80 con 0x7ebfe80 -6520> 2014-08-13 16:24:18.394939 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.394311) v2 ==== 47+0+0 (2695524058 0 0) 0x81c1a40 con 0x7ebf380 -6519> 2014-08-13 16:24:18.394983 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.394311) v2 -- ?+0 0x9282bc0 con 0x7ebfe80 -6518> 2014-08-13 16:24:18.395135 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.394311) v2 -- ?+0 0x9284600 con 0x7ebf380 -6517> 2014-08-13 16:24:18.528624 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.528390) v2 ==== 47+0+0 (360207509 0 0) 0x9282f40 con 0x7edbf40 -6516> 2014-08-13 16:24:18.528630 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.528390) v2 ==== 47+0+0 (360207509 0 0) 0x9284b40 con 0x7ffeca0 -6515> 2014-08-13 16:24:18.528683 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.528390) v2 -- ?+0 0x7d01f80 con 0x7edbf40 -6514> 2014-08-13 16:24:18.528798 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.528390) v2 -- ?+0 0x81c1a40 con 0x7ffeca0 -6513> 2014-08-13 16:24:18.599487 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.599473) v2 ==== 47+0+0 (3806909425 0 0) 0x87e4600 con 0x7ed8000 -6512> 2014-08-13 16:24:18.599510 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.599473) v2 ==== 47+0+0 (3806909425 0 0) 0x87e0380 con 0x7edaec0 -6511> 2014-08-13 16:24:18.599540 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.599473) v2 -- ?+0 0x9282f40 con 0x7ed8000 -6510> 2014-08-13 16:24:18.599638 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.599473) v2 -- ?+0 0x9284b40 con 0x7edaec0 -6509> 2014-08-13 16:24:18.634391 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 15 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3946485199 0 0) 0x8bc6300 con 0x4e4f900 -6508> 2014-08-13 16:24:18.634446 7fd93dff0700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634168, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6507> 2014-08-13 16:24:18.634463 7fd93dff0700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634172, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6506> 2014-08-13 16:24:18.634473 7fd93dff0700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634307, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6505> 2014-08-13 16:24:18.634482 7fd93dff0700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634438, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6504> 2014-08-13 16:24:18.634498 7fd93dff0700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634498, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6503> 2014-08-13 16:24:18.634658 7fd9367e4700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634658, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6502> 2014-08-13 16:24:18.634729 7fd9367e4700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634729, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6501> 2014-08-13 16:24:18.634751 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002f80 con 0x4e4f900 -6500> 2014-08-13 16:24:18.634785 7fd9367e4700 5 -- op tracker -- , seq: 796, time: 2014-08-13 16:24:18.634785, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6499> 2014-08-13 16:24:18.635874 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 16 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2053850730 0 0) 0x8bc2100 con 0x4e4f900 -6498> 2014-08-13 16:24:18.635922 7fd93dff0700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.635738, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6497> 2014-08-13 16:24:18.635938 7fd93dff0700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.635741, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6496> 2014-08-13 16:24:18.635948 7fd93dff0700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.635838, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6495> 2014-08-13 16:24:18.635958 7fd93dff0700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.635913, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6494> 2014-08-13 16:24:18.635969 7fd93dff0700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.635968, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6493> 2014-08-13 16:24:18.636068 7fd9371e5700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.636068, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6492> 2014-08-13 16:24:18.636109 7fd9371e5700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.636109, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6491> 2014-08-13 16:24:18.636125 7fd9371e5700 5 -- op tracker -- , seq: 797, time: 2014-08-13 16:24:18.636125, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6490> 2014-08-13 16:24:18.647314 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.647542) v2 ==== 47+0+0 (376683133 0 0) 0x9282840 con 0x7ebbb20 -6489> 2014-08-13 16:24:18.647355 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.647542) v2 -- ?+0 0x87e0380 con 0x7ebbb20 -6488> 2014-08-13 16:24:18.647447 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.647542) v2 ==== 47+0+0 (376683133 0 0) 0x775c7c0 con 0x7ebd120 -6487> 2014-08-13 16:24:18.647492 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.647542) v2 -- ?+0 0x87e4600 con 0x7ebd120 -6486> 2014-08-13 16:24:18.705927 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.705866) v2 ==== 47+0+0 (1206750074 0 0) 0x9284440 con 0x7edc360 -6485> 2014-08-13 16:24:18.705933 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.705866) v2 ==== 47+0+0 (1206750074 0 0) 0x9283480 con 0x7eda3c0 -6484> 2014-08-13 16:24:18.705975 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.705866) v2 -- ?+0 0x775c7c0 con 0x7edc360 -6483> 2014-08-13 16:24:18.706095 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.705866) v2 -- ?+0 0x9282840 con 0x7eda3c0 -6482> 2014-08-13 16:24:18.742261 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.742277) v2 ==== 47+0+0 (2867710670 0 0) 0x87e2f40 con 0x7d2b700 -6481> 2014-08-13 16:24:18.742311 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.742277) v2 -- ?+0 0x9283480 con 0x7d2b700 -6480> 2014-08-13 16:24:18.742294 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.742277) v2 ==== 47+0+0 (2867710670 0 0) 0x87e3640 con 0x7d29fa0 -6479> 2014-08-13 16:24:18.742499 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.742277) v2 -- ?+0 0x9284440 con 0x7d29fa0 -6478> 2014-08-13 16:24:18.745501 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.745161) v2 ==== 47+0+0 (508792758 0 0) 0x9283100 con 0x7eb8c60 -6477> 2014-08-13 16:24:18.745537 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.745161) v2 -- ?+0 0x87e2f40 con 0x7eb8c60 -6476> 2014-08-13 16:24:18.745534 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.745161) v2 ==== 47+0+0 (508792758 0 0) 0x87e55c0 con 0x7eba100 -6475> 2014-08-13 16:24:18.745580 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.745161) v2 -- ?+0 0x87e3640 con 0x7eba100 -6474> 2014-08-13 16:24:18.746174 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.745809) v2 ==== 47+0+0 (1666580211 0 0) 0x8c8d080 con 0x7ebc620 -6473> 2014-08-13 16:24:18.746208 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.745809) v2 -- ?+0 0x9283100 con 0x7ebc620 -6472> 2014-08-13 16:24:18.746294 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.745809) v2 ==== 47+0+0 (1666580211 0 0) 0x9286900 con 0x7eb9b80 -6471> 2014-08-13 16:24:18.746344 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.745809) v2 -- ?+0 0x87e55c0 con 0x7eb9b80 -6470> 2014-08-13 16:24:18.854779 7fd94ac49700 5 osd.4 194449 tick -6469> 2014-08-13 16:24:18.876483 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.876442) v2 ==== 47+0+0 (891840295 0 0) 0x87e71c0 con 0x7d289a0 -6468> 2014-08-13 16:24:18.876488 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:18.876442) v2 ==== 47+0+0 (891840295 0 0) 0x9283100 con 0x7caf380 -6467> 2014-08-13 16:24:18.876531 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.876442) v2 -- ?+0 0x8c8d080 con 0x7d289a0 -6466> 2014-08-13 16:24:18.876660 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:18.876442) v2 -- ?+0 0x9286900 con 0x7caf380 -6465> 2014-08-13 16:24:18.946088 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 21 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x8bc5800 con 0x508bde0 -6464> 2014-08-13 16:24:18.946143 7fd93dff0700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.945860, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6463> 2014-08-13 16:24:18.946171 7fd93dff0700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.945863, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6462> 2014-08-13 16:24:18.946182 7fd93dff0700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946002, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6461> 2014-08-13 16:24:18.946191 7fd93dff0700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946136, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6460> 2014-08-13 16:24:18.946207 7fd93dff0700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946206, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6459> 2014-08-13 16:24:18.946311 7fd9367e4700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946310, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6458> 2014-08-13 16:24:18.946369 7fd9367e4700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946368, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6457> 2014-08-13 16:24:18.946388 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003200 con 0x508bde0 -6456> 2014-08-13 16:24:18.946421 7fd9367e4700 5 -- op tracker -- , seq: 798, time: 2014-08-13 16:24:18.946421, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6455> 2014-08-13 16:24:18.947537 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 22 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x86d8000 con 0x508bde0 -6454> 2014-08-13 16:24:18.947587 7fd93dff0700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947367, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6453> 2014-08-13 16:24:18.947603 7fd93dff0700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947370, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6452> 2014-08-13 16:24:18.947613 7fd93dff0700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947470, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6451> 2014-08-13 16:24:18.947623 7fd93dff0700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947581, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6450> 2014-08-13 16:24:18.947634 7fd93dff0700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947633, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6449> 2014-08-13 16:24:18.947762 7fd9371e5700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947761, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6448> 2014-08-13 16:24:18.947802 7fd9371e5700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947802, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6447> 2014-08-13 16:24:18.947818 7fd9371e5700 5 -- op tracker -- , seq: 799, time: 2014-08-13 16:24:18.947818, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6446> 2014-08-13 16:24:19.015845 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 17 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x8bc0b00 con 0x5111fa0 -6445> 2014-08-13 16:24:19.015898 7fd93dff0700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.015647, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6444> 2014-08-13 16:24:19.015914 7fd93dff0700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.015651, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6443> 2014-08-13 16:24:19.015924 7fd93dff0700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.015763, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6442> 2014-08-13 16:24:19.015933 7fd93dff0700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.015891, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6441> 2014-08-13 16:24:19.015944 7fd93dff0700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.015944, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6440> 2014-08-13 16:24:19.016077 7fd9367e4700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.016076, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6439> 2014-08-13 16:24:19.016137 7fd9367e4700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.016137, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6438> 2014-08-13 16:24:19.016160 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47d00 con 0x5111fa0 -6437> 2014-08-13 16:24:19.016207 7fd9367e4700 5 -- op tracker -- , seq: 800, time: 2014-08-13 16:24:19.016207, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6436> 2014-08-13 16:24:19.017661 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 18 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x8bc6300 con 0x5111fa0 -6435> 2014-08-13 16:24:19.017714 7fd93dff0700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017467, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6434> 2014-08-13 16:24:19.017730 7fd93dff0700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017470, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6433> 2014-08-13 16:24:19.017740 7fd93dff0700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017577, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6432> 2014-08-13 16:24:19.017754 7fd93dff0700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017707, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6431> 2014-08-13 16:24:19.017770 7fd93dff0700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017770, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6430> 2014-08-13 16:24:19.017874 7fd9371e5700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017873, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6429> 2014-08-13 16:24:19.017916 7fd9371e5700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017915, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6428> 2014-08-13 16:24:19.017931 7fd9371e5700 5 -- op tracker -- , seq: 801, time: 2014-08-13 16:24:19.017931, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6427> 2014-08-13 16:24:19.020834 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11555 0x7ec8b00 con 0x4fe0000 -6426> 2014-08-13 16:24:19.054102 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 16 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4136492510 0 0) 0x86d8b00 con 0x4fe3700 -6425> 2014-08-13 16:24:19.054153 7fd93dff0700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.053912, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6424> 2014-08-13 16:24:19.054182 7fd93dff0700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.053915, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6423> 2014-08-13 16:24:19.054193 7fd93dff0700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054017, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6422> 2014-08-13 16:24:19.054206 7fd93dff0700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054147, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6421> 2014-08-13 16:24:19.054222 7fd93dff0700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054221, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6420> 2014-08-13 16:24:19.054324 7fd9367e4700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054323, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6419> 2014-08-13 16:24:19.054377 7fd9367e4700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054377, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6418> 2014-08-13 16:24:19.054396 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_sub_op_reply(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47800 con 0x4fe3700 -6417> 2014-08-13 16:24:19.054429 7fd9367e4700 5 -- op tracker -- , seq: 802, time: 2014-08-13 16:24:19.054429, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6416> 2014-08-13 16:24:19.055612 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 17 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1742474875 0 0) 0x7ece300 con 0x4fe3700 -6415> 2014-08-13 16:24:19.055662 7fd93dff0700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055426, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6414> 2014-08-13 16:24:19.055679 7fd93dff0700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055429, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6413> 2014-08-13 16:24:19.055689 7fd93dff0700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055530, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6412> 2014-08-13 16:24:19.055698 7fd93dff0700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055657, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6411> 2014-08-13 16:24:19.055709 7fd93dff0700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055709, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6410> 2014-08-13 16:24:19.055839 7fd9371e5700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055838, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6409> 2014-08-13 16:24:19.055880 7fd9371e5700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055880, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6408> 2014-08-13 16:24:19.055895 7fd9371e5700 5 -- op tracker -- , seq: 803, time: 2014-08-13 16:24:19.055895, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6407> 2014-08-13 16:24:19.217048 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 26 ==== replica scrub(pg: 22.29a,from:0'0,to:194215'477,epoch:194449,start:f36fba9a//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (2825391861 0 0) 0x7cf1d40 con 0x4fe0000 -6406> 2014-08-13 16:24:19.224256 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.224177) v2 ==== 47+0+0 (3718677340 0 0) 0x87e7a80 con 0x7ed82c0 -6405> 2014-08-13 16:24:19.224309 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.224177) v2 -- ?+0 0x9283100 con 0x7ed82c0 -6404> 2014-08-13 16:24:19.224305 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.224177) v2 ==== 47+0+0 (3718677340 0 0) 0x87e5b00 con 0x7edb180 -6403> 2014-08-13 16:24:19.224367 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.224177) v2 -- ?+0 0x87e71c0 con 0x7edb180 -6402> 2014-08-13 16:24:19.287944 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.287471) v2 ==== 47+0+0 (3648961602 0 0) 0x87e6200 con 0x7ffe9e0 -6401> 2014-08-13 16:24:19.287943 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.287471) v2 ==== 47+0+0 (3648961602 0 0) 0x87e5e80 con 0x7edbc80 -6400> 2014-08-13 16:24:19.288001 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.287471) v2 -- ?+0 0x87e5b00 con 0x7ffe9e0 -6399> 2014-08-13 16:24:19.288050 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.287471) v2 -- ?+0 0x87e7a80 con 0x7edbc80 -6398> 2014-08-13 16:24:19.293088 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.292881) v2 ==== 47+0+0 (2264208322 0 0) 0x87e5780 con 0x7ffeb40 -6397> 2014-08-13 16:24:19.293139 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.292881) v2 -- ?+0 0x87e6200 con 0x7ffeb40 -6396> 2014-08-13 16:24:19.293151 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.292881) v2 ==== 47+0+0 (2264208322 0 0) 0x87e63c0 con 0x7edbde0 -6395> 2014-08-13 16:24:19.293216 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.292881) v2 -- ?+0 0x87e5e80 con 0x7edbde0 -6394> 2014-08-13 16:24:19.528300 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.527922) v2 ==== 47+0+0 (907005538 0 0) 0x9287380 con 0x7d29ce0 -6393> 2014-08-13 16:24:19.528302 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.527922) v2 ==== 47+0+0 (907005538 0 0) 0x9281dc0 con 0x7d2b440 -6392> 2014-08-13 16:24:19.528357 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.527922) v2 -- ?+0 0x87e63c0 con 0x7d29ce0 -6391> 2014-08-13 16:24:19.528402 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.527922) v2 -- ?+0 0x87e5780 con 0x7d2b440 -6390> 2014-08-13 16:24:19.537908 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.537568) v2 ==== 47+0+0 (1505243561 0 0) 0x775d940 con 0x7caeb40 -6389> 2014-08-13 16:24:19.537933 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.537568) v2 ==== 47+0+0 (1505243561 0 0) 0x77596c0 con 0x7cae300 -6388> 2014-08-13 16:24:19.537958 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.537568) v2 -- ?+0 0x9281dc0 con 0x7caeb40 -6387> 2014-08-13 16:24:19.538102 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.537568) v2 -- ?+0 0x9287380 con 0x7cae300 -6386> 2014-08-13 16:24:19.617238 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.617011) v2 ==== 47+0+0 (1227906082 0 0) 0x9285240 con 0x7eda680 -6385> 2014-08-13 16:24:19.617312 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.617011) v2 -- ?+0 0x775d940 con 0x7eda680 -6384> 2014-08-13 16:24:19.617352 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.617011) v2 ==== 47+0+0 (1227906082 0 0) 0x9287e00 con 0x7edc620 -6383> 2014-08-13 16:24:19.617400 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.617011) v2 -- ?+0 0x77596c0 con 0x7edc620 -6382> 2014-08-13 16:24:19.691232 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.110:6820/46337 -- osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+2804 0x7ec8b00 con 0x4fe0000 -6381> 2014-08-13 16:24:19.694002 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 27 ==== osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3609597438 0 0) 0x7ec8b00 con 0x4fe0000 -6380> 2014-08-13 16:24:19.694061 7fd93dff0700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.693813, event: header_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6379> 2014-08-13 16:24:19.694079 7fd93dff0700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.693816, event: throttled, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6378> 2014-08-13 16:24:19.694089 7fd93dff0700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.693920, event: all_read, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6377> 2014-08-13 16:24:19.694098 7fd93dff0700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.694054, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6376> 2014-08-13 16:24:19.694110 7fd93dff0700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.694109, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6375> 2014-08-13 16:24:19.694154 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 28 ==== pg_info(1 pgs e194449:22.29a) v4 ==== 714+0+0 (1339832052 0 0) 0x87e63c0 con 0x4fe0000 -6374> 2014-08-13 16:24:19.694189 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.693945, event: header_read, request: pg_info(1 pgs e194449:22.29a) v4 -6373> 2014-08-13 16:24:19.694197 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.693947, event: throttled, request: pg_info(1 pgs e194449:22.29a) v4 -6372> 2014-08-13 16:24:19.694203 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.694024, event: all_read, request: pg_info(1 pgs e194449:22.29a) v4 -6371> 2014-08-13 16:24:19.694208 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.694186, event: dispatched, request: pg_info(1 pgs e194449:22.29a) v4 -6370> 2014-08-13 16:24:19.694224 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.694224, event: waiting_for_osdmap, request: pg_info(1 pgs e194449:22.29a) v4 -6369> 2014-08-13 16:24:19.694240 7fd9367e4700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.694215, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6368> 2014-08-13 16:24:19.694306 7fd9367e4700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.694305, event: started, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6367> 2014-08-13 16:24:19.694327 7fd9367e4700 5 -- op tracker -- , seq: 804, time: 2014-08-13 16:24:19.694327, event: done, request: osd_sub_op(unknown.0.0:0 22.29a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6366> 2014-08-13 16:24:19.694378 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.694240, event: started, request: pg_info(1 pgs e194449:22.29a) v4 -6365> 2014-08-13 16:24:19.694444 7fd93dff0700 5 -- op tracker -- , seq: 805, time: 2014-08-13 16:24:19.694444, event: done, request: pg_info(1 pgs e194449:22.29a) v4 -6364> 2014-08-13 16:24:19.729230 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.728894) v2 ==== 47+0+0 (726534391 0 0) 0x7758000 con 0x7ebbde0 -6363> 2014-08-13 16:24:19.729284 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.728894) v2 -- ?+0 0x9285240 con 0x7ebbde0 -6362> 2014-08-13 16:24:19.729317 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.728894) v2 ==== 47+0+0 (726534391 0 0) 0x81c1180 con 0x7ebd3e0 -6361> 2014-08-13 16:24:19.729367 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.728894) v2 -- ?+0 0x9287e00 con 0x7ebd3e0 -6360> 2014-08-13 16:24:19.819480 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.819429) v2 ==== 47+0+0 (687942081 0 0) 0x9287700 con 0x7ed8580 -6359> 2014-08-13 16:24:19.819536 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.819429) v2 -- ?+0 0x81c1180 con 0x7ed8580 -6358> 2014-08-13 16:24:19.819626 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.819429) v2 ==== 47+0+0 (687942081 0 0) 0x7d00000 con 0x7ebe5c0 -6357> 2014-08-13 16:24:19.819672 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.819429) v2 -- ?+0 0x7758000 con 0x7ebe5c0 -6356> 2014-08-13 16:24:19.854933 7fd94ac49700 5 osd.4 194449 tick -6355> 2014-08-13 16:24:19.855042 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x7ecac00 -6354> 2014-08-13 16:24:19.855092 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x76f8b00 -6353> 2014-08-13 16:24:19.856563 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 15 ==== osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (2324805707 0 4067132163) 0x8003980 con 0x4fe69e0 -6352> 2014-08-13 16:24:19.856612 7fd93dff0700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856395, event: header_read, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6351> 2014-08-13 16:24:19.856627 7fd93dff0700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856400, event: throttled, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6350> 2014-08-13 16:24:19.856635 7fd93dff0700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856518, event: all_read, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6349> 2014-08-13 16:24:19.856643 7fd93dff0700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856606, event: dispatched, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6348> 2014-08-13 16:24:19.856652 7fd93dff0700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856652, event: waiting_for_osdmap, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6347> 2014-08-13 16:24:19.856697 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.347 192.168.1.106:6800/4640 7 ==== osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (1826512121 0 4067132163) 0x8002a80 con 0x50bbf40 -6346> 2014-08-13 16:24:19.856719 7fd93dff0700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856520, event: header_read, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6345> 2014-08-13 16:24:19.856729 7fd93dff0700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856523, event: throttled, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6344> 2014-08-13 16:24:19.856737 7fd93dff0700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856617, event: all_read, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6343> 2014-08-13 16:24:19.856744 7fd93dff0700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856717, event: dispatched, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6342> 2014-08-13 16:24:19.856753 7fd93dff0700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856753, event: waiting_for_osdmap, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6341> 2014-08-13 16:24:19.856768 7fd9367e4700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856757, event: reached_pg, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6340> 2014-08-13 16:24:19.856824 7fd9367e4700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856824, event: started, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6339> 2014-08-13 16:24:19.856844 7fd9367e4700 5 -- op tracker -- , seq: 806, time: 2014-08-13 16:24:19.856844, event: done, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6338> 2014-08-13 16:24:19.856859 7fd9371e5700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856849, event: reached_pg, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6337> 2014-08-13 16:24:19.856919 7fd9371e5700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856906, event: started, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6336> 2014-08-13 16:24:19.856944 7fd9371e5700 5 -- op tracker -- , seq: 807, time: 2014-08-13 16:24:19.856944, event: done, request: osd_sub_op_reply(unknown.0.0:0 22.1b8 0//0//22 [scrub-reserve] ack, result = 0) v2 -6335> 2014-08-13 16:24:19.876131 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- replica scrub(pg: 22.1b8,from:0'0,to:194215'478,epoch:194449,start:0//0//-1,end:2b4699b8//0//-1,chunky:1,deep:1,version:5) v5 -- ?+0 0x7cf1d40 -6334> 2014-08-13 16:24:19.876197 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- replica scrub(pg: 22.1b8,from:0'0,to:194215'478,epoch:194449,start:0//0//-1,end:2b4699b8//0//-1,chunky:1,deep:1,version:5) v5 -- ?+0 0x7b11440 -6333> 2014-08-13 16:24:19.895950 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.895614) v2 ==== 47+0+0 (2422189748 0 0) 0x9287e00 con 0x7eba3c0 -6332> 2014-08-13 16:24:19.896005 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.895614) v2 -- ?+0 0x9287700 con 0x7eba3c0 -6331> 2014-08-13 16:24:19.895980 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.895614) v2 ==== 47+0+0 (2422189748 0 0) 0x7d032c0 con 0x7eb8f20 -6330> 2014-08-13 16:24:19.896074 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.895614) v2 -- ?+0 0x7d00000 con 0x7eb8f20 -6329> 2014-08-13 16:24:19.940416 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.940344) v2 ==== 47+0+0 (401801353 0 0) 0x9283800 con 0x7ffe880 -6328> 2014-08-13 16:24:19.940471 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.940344) v2 -- ?+0 0x7d032c0 con 0x7ffe880 -6327> 2014-08-13 16:24:19.940488 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.940344) v2 ==== 47+0+0 (401801353 0 0) 0x9287c40 con 0x7edbb20 -6326> 2014-08-13 16:24:19.940537 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.940344) v2 -- ?+0 0x9287e00 con 0x7edbb20 -6325> 2014-08-13 16:24:19.978451 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.978494) v2 ==== 47+0+0 (3837926667 0 0) 0x87e01c0 con 0x7d289a0 -6324> 2014-08-13 16:24:19.978442 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:19.978494) v2 ==== 47+0+0 (3837926667 0 0) 0x9285400 con 0x7caf380 -6323> 2014-08-13 16:24:19.978496 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.978494) v2 -- ?+0 0x9283800 con 0x7d289a0 -6322> 2014-08-13 16:24:19.978578 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:19.978494) v2 -- ?+0 0x9287c40 con 0x7caf380 -6321> 2014-08-13 16:24:20.016683 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 21 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x8bc2c00 con 0x510fd20 -6320> 2014-08-13 16:24:20.016733 7fd93dff0700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016522, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6319> 2014-08-13 16:24:20.016755 7fd93dff0700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016526, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6318> 2014-08-13 16:24:20.016765 7fd93dff0700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016642, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6317> 2014-08-13 16:24:20.016774 7fd93dff0700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016727, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6316> 2014-08-13 16:24:20.016786 7fd93dff0700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016785, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6315> 2014-08-13 16:24:20.016941 7fd9367e4700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016941, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6314> 2014-08-13 16:24:20.016993 7fd9367e4700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.016993, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6313> 2014-08-13 16:24:20.017016 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003980 con 0x510fd20 -6312> 2014-08-13 16:24:20.017051 7fd9367e4700 5 -- op tracker -- , seq: 808, time: 2014-08-13 16:24:20.017051, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6311> 2014-08-13 16:24:20.018071 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 22 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x8bc0000 con 0x510fd20 -6310> 2014-08-13 16:24:20.018118 7fd93dff0700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.017930, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6309> 2014-08-13 16:24:20.018134 7fd93dff0700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.017932, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6308> 2014-08-13 16:24:20.018143 7fd93dff0700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018029, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6307> 2014-08-13 16:24:20.018164 7fd93dff0700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018112, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6306> 2014-08-13 16:24:20.018181 7fd93dff0700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018181, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6305> 2014-08-13 16:24:20.018242 7fd9371e5700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018241, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6304> 2014-08-13 16:24:20.018281 7fd9371e5700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018281, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6303> 2014-08-13 16:24:20.018297 7fd9371e5700 5 -- op tracker -- , seq: 809, time: 2014-08-13 16:24:20.018297, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6302> 2014-08-13 16:24:20.047596 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.047224) v2 ==== 47+0+0 (4281827457 0 0) 0x9287e00 con 0x7ed8dc0 -6301> 2014-08-13 16:24:20.047614 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.047224) v2 ==== 47+0+0 (4281827457 0 0) 0x7d05b00 con 0x7ebee00 -6300> 2014-08-13 16:24:20.047646 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.047224) v2 -- ?+0 0x9285400 con 0x7ed8dc0 -6299> 2014-08-13 16:24:20.047734 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.047224) v2 -- ?+0 0x87e01c0 con 0x7ebee00 -6298> 2014-08-13 16:24:20.054478 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 18 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4136492510 0 0) 0x863b700 con 0x4fe3700 -6297> 2014-08-13 16:24:20.054533 7fd93dff0700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054258, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6296> 2014-08-13 16:24:20.054549 7fd93dff0700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054262, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6295> 2014-08-13 16:24:20.054559 7fd93dff0700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054407, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6294> 2014-08-13 16:24:20.054569 7fd93dff0700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054526, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6293> 2014-08-13 16:24:20.054584 7fd93dff0700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054584, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6292> 2014-08-13 16:24:20.054707 7fd9367e4700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054706, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6291> 2014-08-13 16:24:20.054750 7fd9367e4700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054750, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6290> 2014-08-13 16:24:20.054769 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_sub_op_reply(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47580 con 0x4fe3700 -6289> 2014-08-13 16:24:20.054801 7fd9367e4700 5 -- op tracker -- , seq: 810, time: 2014-08-13 16:24:20.054801, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6288> 2014-08-13 16:24:20.055837 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 19 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1742474875 0 0) 0x863ac00 con 0x4fe3700 -6287> 2014-08-13 16:24:20.055891 7fd93dff0700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055692, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6286> 2014-08-13 16:24:20.055907 7fd93dff0700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055694, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6285> 2014-08-13 16:24:20.055917 7fd93dff0700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055771, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6284> 2014-08-13 16:24:20.055927 7fd93dff0700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055884, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6283> 2014-08-13 16:24:20.055938 7fd93dff0700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055938, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6282> 2014-08-13 16:24:20.055999 7fd9371e5700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.055998, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6281> 2014-08-13 16:24:20.056035 7fd9371e5700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.056035, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6280> 2014-08-13 16:24:20.056050 7fd9371e5700 5 -- op tracker -- , seq: 811, time: 2014-08-13 16:24:20.056050, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6279> 2014-08-13 16:24:20.092181 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.091495) v2 ==== 47+0+0 (3463985208 0 0) 0x7d03480 con 0x7d2ac00 -6278> 2014-08-13 16:24:20.092231 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.091495) v2 -- ?+0 0x7d05b00 con 0x7d2ac00 -6277> 2014-08-13 16:24:20.092245 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.091495) v2 ==== 47+0+0 (3463985208 0 0) 0x8c88e00 con 0x7d2c360 -6276> 2014-08-13 16:24:20.092307 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.091495) v2 -- ?+0 0x9287e00 con 0x7d2c360 -6275> 2014-08-13 16:24:20.213785 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.213748) v2 ==== 47+0+0 (1319163851 0 0) 0x80dbb80 con 0x7ebf0c0 -6274> 2014-08-13 16:24:20.213833 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.213748) v2 -- ?+0 0x7d03480 con 0x7ebf0c0 -6273> 2014-08-13 16:24:20.213833 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.213748) v2 ==== 47+0+0 (1319163851 0 0) 0x7d04b40 con 0x7ed9080 -6272> 2014-08-13 16:24:20.213912 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.213748) v2 -- ?+0 0x8c88e00 con 0x7ed9080 -6271> 2014-08-13 16:24:20.219779 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.219072) v2 ==== 47+0+0 (1866521301 0 0) 0x9286580 con 0x7ab9ce0 -6270> 2014-08-13 16:24:20.219826 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.219072) v2 -- ?+0 0x7d04b40 con 0x7ab9ce0 -6269> 2014-08-13 16:24:20.219920 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.219072) v2 ==== 47+0+0 (1866521301 0 0) 0x9285080 con 0x7ab9a20 -6268> 2014-08-13 16:24:20.219980 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.219072) v2 -- ?+0 0x80dbb80 con 0x7ab9a20 -6267> 2014-08-13 16:24:20.301982 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.301881) v2 ==== 47+0+0 (4270016655 0 0) 0x87e3480 con 0x7ed8000 -6266> 2014-08-13 16:24:20.301965 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.301881) v2 ==== 47+0+0 (4270016655 0 0) 0x87e4b40 con 0x7edaec0 -6265> 2014-08-13 16:24:20.302041 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.301881) v2 -- ?+0 0x9286580 con 0x7ed8000 -6264> 2014-08-13 16:24:20.302106 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.301881) v2 -- ?+0 0x9285080 con 0x7edaec0 -6263> 2014-08-13 16:24:20.399907 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.399599) v2 ==== 47+0+0 (89761219 0 0) 0x9284280 con 0x7eba3c0 -6262> 2014-08-13 16:24:20.399958 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.399599) v2 ==== 47+0+0 (89761219 0 0) 0x7d03d40 con 0x7eb8f20 -6261> 2014-08-13 16:24:20.399952 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.399599) v2 -- ?+0 0x87e3480 con 0x7eba3c0 -6260> 2014-08-13 16:24:20.400104 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.399599) v2 -- ?+0 0x87e4b40 con 0x7eb8f20 -6259> 2014-08-13 16:24:20.465730 7fd933fe0700 5 osd.4 194449 heartbeat: osd_stat(8079 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,7,10,11,15,17,20,21,24,25,28,31,32,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,396,397]/[] op hist []) -6258> 2014-08-13 16:24:20.465779 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8917a80 con 0x8c391e0 -6257> 2014-08-13 16:24:20.465815 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c4b40 con 0x8c38f20 -6256> 2014-08-13 16:24:20.465837 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c7700 con 0x78844c0 -6255> 2014-08-13 16:24:20.465858 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c01c0 con 0x7884200 -6254> 2014-08-13 16:24:20.465877 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c1180 con 0x78848e0 -6253> 2014-08-13 16:24:20.465899 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c2140 con 0x7884620 -6252> 2014-08-13 16:24:20.465918 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6839/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x81c5cc0 con 0x8ba06e0 -6251> 2014-08-13 16:24:20.465934 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6846/62713 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d02140 con 0x8c698c0 -6250> 2014-08-13 16:24:20.465951 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d06ac0 con 0x794c360 -6249> 2014-08-13 16:24:20.465968 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d00a80 con 0x794e300 -6248> 2014-08-13 16:24:20.465997 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d055c0 con 0x794e040 -6247> 2014-08-13 16:24:20.466015 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d00c40 con 0x794dd80 -6246> 2014-08-13 16:24:20.466040 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d05940 con 0x8c3a3c0 -6245> 2014-08-13 16:24:20.466057 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d05cc0 con 0x8c3a100 -6244> 2014-08-13 16:24:20.466074 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d04600 con 0x8c3c0a0 -6243> 2014-08-13 16:24:20.466091 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d04980 con 0x8c3bde0 -6242> 2014-08-13 16:24:20.466109 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d00540 con 0x8babde0 -6241> 2014-08-13 16:24:20.466132 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d07380 con 0x8babb20 -6240> 2014-08-13 16:24:20.466152 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d032c0 con 0x8c3d6a0 -6239> 2014-08-13 16:24:20.466184 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d063c0 con 0x8c3d3e0 -6238> 2014-08-13 16:24:20.466206 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d00fc0 con 0x8badac0 -6237> 2014-08-13 16:24:20.466224 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d03480 con 0x8bad800 -6236> 2014-08-13 16:24:20.466240 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7d00000 con 0x8bab860 -6235> 2014-08-13 16:24:20.466257 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8c88e00 con 0x8bab5a0 -6234> 2014-08-13 16:24:20.466273 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8c8c600 con 0x8c69600 -6233> 2014-08-13 16:24:20.466292 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9286580 con 0x8c69340 -6232> 2014-08-13 16:24:20.466310 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9282f40 con 0x8baee00 -6231> 2014-08-13 16:24:20.466306 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x81c5400 con 0x8c391e0 -6230> 2014-08-13 16:24:20.466327 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9285080 con 0x8ba1a20 -6229> 2014-08-13 16:24:20.466447 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6814/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9284b40 con 0x8d21e40 -6228> 2014-08-13 16:24:20.466483 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6831/59915 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9287700 con 0x8d21b80 -6227> 2014-08-13 16:24:20.466503 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9285b00 con 0x7fba3c0 -6226> 2014-08-13 16:24:20.466524 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9282140 con 0x7fba260 -6225> 2014-08-13 16:24:20.466541 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x87e5940 con 0x8bad540 -6224> 2014-08-13 16:24:20.466557 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x87e6e40 con 0x8bad280 -6223> 2014-08-13 16:24:20.466575 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x87e3480 con 0x8c3cba0 -6222> 2014-08-13 16:24:20.466592 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x87e4b40 con 0x8c3c8e0 -6221> 2014-08-13 16:24:20.466617 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eea300 con 0x7fba100 -6220> 2014-08-13 16:24:20.466636 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eef380 con 0x7fb9fa0 -6219> 2014-08-13 16:24:20.466653 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eec280 con 0x8d223c0 -6218> 2014-08-13 16:24:20.466674 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eecb40 con 0x8d22100 -6217> 2014-08-13 16:24:20.466699 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8ee9f80 con 0x8c3c620 -6216> 2014-08-13 16:24:20.466714 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eebf00 con 0x8c3c360 -6215> 2014-08-13 16:24:20.466730 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eeee40 con 0x8bafe80 -6214> 2014-08-13 16:24:20.466749 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eee580 con 0x8bafbc0 -6213> 2014-08-13 16:24:20.466779 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eec600 con 0x8c38c60 -6212> 2014-08-13 16:24:20.466796 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eecd00 con 0x8c389a0 -6211> 2014-08-13 16:24:20.466812 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eef1c0 con 0x8b68c60 -6210> 2014-08-13 16:24:20.466831 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8ee88c0 con 0x8b68b00 -6209> 2014-08-13 16:24:20.466847 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eea140 con 0x8c6fe80 -6208> 2014-08-13 16:24:20.466868 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eee040 con 0x8c6fbc0 -6207> 2014-08-13 16:24:20.466885 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eee3c0 con 0x8b689a0 -6206> 2014-08-13 16:24:20.466904 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eee200 con 0x8b68840 -6205> 2014-08-13 16:24:20.466925 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eee900 con 0x7fb9e40 -6204> 2014-08-13 16:24:20.466948 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eeeac0 con 0x7fb9ce0 -6203> 2014-08-13 16:24:20.466968 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eec7c0 con 0x8c3b9c0 -6202> 2014-08-13 16:24:20.466988 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8ee8540 con 0x8c3b700 -6201> 2014-08-13 16:24:20.467004 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8ee96c0 con 0x7fbb020 -6200> 2014-08-13 16:24:20.467019 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eebb80 con 0x7fbaec0 -6199> 2014-08-13 16:24:20.467036 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eefc40 con 0x7fbb440 -6198> 2014-08-13 16:24:20.467055 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eed400 con 0x7fbb2e0 -6197> 2014-08-13 16:24:20.467070 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eed780 con 0x7ffd960 -6196> 2014-08-13 16:24:20.467096 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eedb00 con 0x7ffdc20 -6195> 2014-08-13 16:24:20.467114 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8eed940 con 0x8d20840 -6194> 2014-08-13 16:24:20.467132 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80d8a80 con 0x8d25960 -6193> 2014-08-13 16:24:20.467152 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80dbf00 con 0x8d218c0 -6192> 2014-08-13 16:24:20.467168 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80db480 con 0x8d21600 -6191> 2014-08-13 16:24:20.467204 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80dfe00 con 0x7fbaaa0 -6190> 2014-08-13 16:24:20.467221 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80d9340 con 0x7fba940 -6189> 2014-08-13 16:24:20.467239 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80da300 con 0x8d26e00 -6188> 2014-08-13 16:24:20.467257 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80dc440 con 0x8d26b40 -6187> 2014-08-13 16:24:20.467285 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x80d9500 con 0x8d21340 -6186> 2014-08-13 16:24:20.467302 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7758a80 con 0x8d21080 -6185> 2014-08-13 16:24:20.467322 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8575400 con 0x7ffdee0 -6184> 2014-08-13 16:24:20.467340 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8575240 con 0x7ffdd80 -6183> 2014-08-13 16:24:20.467364 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8575080 con 0x7fb9a20 -6182> 2014-08-13 16:24:20.467382 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574ec0 con 0x7fb98c0 -6181> 2014-08-13 16:24:20.467401 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574d00 con 0x8bab2e0 -6180> 2014-08-13 16:24:20.467417 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574b40 con 0x8bab020 -6179> 2014-08-13 16:24:20.467435 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574980 con 0x8c69080 -6178> 2014-08-13 16:24:20.467462 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85747c0 con 0x8c68dc0 -6177> 2014-08-13 16:24:20.467480 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574600 con 0x8d256a0 -6176> 2014-08-13 16:24:20.467504 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574440 con 0x8d253e0 -6175> 2014-08-13 16:24:20.467522 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8574280 con 0x8d20dc0 -6174> 2014-08-13 16:24:20.467541 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85740c0 con 0x8d20b00 -6173> 2014-08-13 16:24:20.467557 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573f00 con 0x8d25120 -6172> 2014-08-13 16:24:20.467575 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573d40 con 0x8d24e60 -6171> 2014-08-13 16:24:20.467595 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573b80 con 0x8c3d120 -6170> 2014-08-13 16:24:20.467613 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85739c0 con 0x8c3ce60 -6169> 2014-08-13 16:24:20.467633 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573800 con 0x7fbb180 -6168> 2014-08-13 16:24:20.467655 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573640 con 0x7fbca40 -6167> 2014-08-13 16:24:20.467671 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573480 con 0x7fbc780 -6166> 2014-08-13 16:24:20.467689 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85732c0 con 0x7fbc4c0 -6165> 2014-08-13 16:24:20.467705 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8573100 con 0x8c3aec0 -6164> 2014-08-13 16:24:20.467722 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572f40 con 0x8c3ac00 -6163> 2014-08-13 16:24:20.467739 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572d80 con 0x7fbad60 -6162> 2014-08-13 16:24:20.467760 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572bc0 con 0x7fbac00 -6161> 2014-08-13 16:24:20.467776 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572a00 con 0x8c3e880 -6160> 2014-08-13 16:24:20.467795 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572840 con 0x8c3e5c0 -6159> 2014-08-13 16:24:20.467813 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572680 con 0x8c3a940 -6158> 2014-08-13 16:24:20.467829 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85724c0 con 0x8c3a680 -6157> 2014-08-13 16:24:20.467859 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572300 con 0x8c69fa0 -6156> 2014-08-13 16:24:20.467884 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8572140 con 0x8c39fa0 -6155> 2014-08-13 16:24:20.467904 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571f80 con 0x7fb9760 -6154> 2014-08-13 16:24:20.467920 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571dc0 con 0x7fb9600 -6153> 2014-08-13 16:24:20.467938 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571c00 con 0x8c3e300 -6152> 2014-08-13 16:24:20.467954 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571a40 con 0x8c618c0 -6151> 2014-08-13 16:24:20.467971 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571880 con 0x8bac780 -6150> 2014-08-13 16:24:20.467994 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x85716c0 con 0x8bac4c0 -6149> 2014-08-13 16:24:20.468010 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x8571500 con 0x8bac200 -6148> 2014-08-13 16:24:20.468032 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9116040 con 0x8babf40 -6147> 2014-08-13 16:24:20.468056 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115e80 con 0x8d240a0 -6146> 2014-08-13 16:24:20.468072 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115cc0 con 0x8d23de0 -6145> 2014-08-13 16:24:20.468089 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115b00 con 0x8ba1760 -6144> 2014-08-13 16:24:20.468111 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115940 con 0x8ba14a0 -6143> 2014-08-13 16:24:20.468128 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115780 con 0x8d26880 -6142> 2014-08-13 16:24:20.468154 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91155c0 con 0x8d265c0 -6141> 2014-08-13 16:24:20.468182 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115400 con 0x8baf900 -6140> 2014-08-13 16:24:20.468200 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115240 con 0x8baf640 -6139> 2014-08-13 16:24:20.468222 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9115080 con 0x8bae040 -6138> 2014-08-13 16:24:20.468241 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114ec0 con 0x8badd80 -6137> 2014-08-13 16:24:20.468258 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114d00 con 0x8d24620 -6136> 2014-08-13 16:24:20.468278 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114b40 con 0x8d24360 -6135> 2014-08-13 16:24:20.468299 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114980 con 0x8baf380 -6134> 2014-08-13 16:24:20.468320 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91147c0 con 0x8baf0c0 -6133> 2014-08-13 16:24:20.468343 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114600 con 0x8b6e1a0 -6132> 2014-08-13 16:24:20.468362 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114440 con 0x8b6e040 -6131> 2014-08-13 16:24:20.468383 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9114280 con 0x8c39ce0 -6130> 2014-08-13 16:24:20.468399 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91140c0 con 0x8c39a20 -6129> 2014-08-13 16:24:20.468416 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113f00 con 0x8d27380 -6128> 2014-08-13 16:24:20.468440 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113d40 con 0x8d270c0 -6127> 2014-08-13 16:24:20.468457 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113b80 con 0x8d23b20 -6126> 2014-08-13 16:24:20.468472 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91139c0 con 0x8d27bc0 -6125> 2014-08-13 16:24:20.468499 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113800 con 0x8b6dee0 -6124> 2014-08-13 16:24:20.468519 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113640 con 0x8b6dd80 -6123> 2014-08-13 16:24:20.468537 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113480 con 0x7fba7e0 -6122> 2014-08-13 16:24:20.468554 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91132c0 con 0x7fba680 -6121> 2014-08-13 16:24:20.468569 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9113100 con 0x8c6f900 -6120> 2014-08-13 16:24:20.468590 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112f40 con 0x8c6f640 -6119> 2014-08-13 16:24:20.468607 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112d80 con 0x8c68b00 -6118> 2014-08-13 16:24:20.468623 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112bc0 con 0x8c68840 -6117> 2014-08-13 16:24:20.468641 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112a00 con 0x8c3b440 -6116> 2014-08-13 16:24:20.468658 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112840 con 0x8c3b180 -6115> 2014-08-13 16:24:20.468675 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112680 con 0x8c68580 -6114> 2014-08-13 16:24:20.468693 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x91124c0 con 0x8c6b2e0 -6113> 2014-08-13 16:24:20.468709 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112300 con 0x8c6b020 -6112> 2014-08-13 16:24:20.468728 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9112140 con 0x8c6ad60 -6111> 2014-08-13 16:24:20.468745 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x9111f80 con 0x8b6dc20 -6110> 2014-08-13 16:24:20.468767 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed71c0 con 0x7fb9b80 -6109> 2014-08-13 16:24:20.468786 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed7000 con 0x8ba11e0 -6108> 2014-08-13 16:24:20.468804 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6e40 con 0x8ba0f20 -6107> 2014-08-13 16:24:20.468824 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6c80 con 0x7fbbc80 -6106> 2014-08-13 16:24:20.468841 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6ac0 con 0x7fbbb20 -6105> 2014-08-13 16:24:20.468869 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6900 con 0x8c39760 -6104> 2014-08-13 16:24:20.468886 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6740 con 0x8c394a0 -6103> 2014-08-13 16:24:20.468906 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6580 con 0x7fbb9c0 -6102> 2014-08-13 16:24:20.468923 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed63c0 con 0x7fbb860 -6101> 2014-08-13 16:24:20.468939 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6200 con 0x7fb94a0 -6100> 2014-08-13 16:24:20.468955 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed6040 con 0x7fba520 -6099> 2014-08-13 16:24:20.468972 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5e80 con 0x7fbc200 -6098> 2014-08-13 16:24:20.468989 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5cc0 con 0x7fbbf40 -6097> 2014-08-13 16:24:20.469007 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5b00 con 0x8c6aaa0 -6096> 2014-08-13 16:24:20.469024 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5940 con 0x8c6a7e0 -6095> 2014-08-13 16:24:20.469040 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5780 con 0x8bacd00 -6094> 2014-08-13 16:24:20.469057 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed55c0 con 0x8baca40 -6093> 2014-08-13 16:24:20.469075 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5400 con 0x8ba0c60 -6092> 2014-08-13 16:24:20.469091 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5240 con 0x8ba09a0 -6091> 2014-08-13 16:24:20.469107 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed5080 con 0x7ffdac0 -6090> 2014-08-13 16:24:20.469122 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4ec0 con 0x7ffe040 -6089> 2014-08-13 16:24:20.469138 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4d00 con 0x8c6f380 -6088> 2014-08-13 16:24:20.469155 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4b40 con 0x8c6f0c0 -6087> 2014-08-13 16:24:20.469178 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4980 con 0x7fbb700 -6086> 2014-08-13 16:24:20.469197 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed47c0 con 0x7fbb5a0 -6085> 2014-08-13 16:24:20.469220 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4600 con 0x8d27900 -6084> 2014-08-13 16:24:20.469238 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4440 con 0x8d27640 -6083> 2014-08-13 16:24:20.469258 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed4280 con 0x8c6a520 -6082> 2014-08-13 16:24:20.469283 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed40c0 con 0x8c6a260 -6081> 2014-08-13 16:24:20.469307 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3f00 con 0x8d24ba0 -6080> 2014-08-13 16:24:20.469325 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3d40 con 0x8d248e0 -6079> 2014-08-13 16:24:20.469343 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3b80 con 0x8d26300 -6078> 2014-08-13 16:24:20.469361 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed39c0 con 0x8d26040 -6077> 2014-08-13 16:24:20.469383 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3800 con 0x8d25d80 -6076> 2014-08-13 16:24:20.469400 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3640 con 0x8d140a0 -6075> 2014-08-13 16:24:20.469418 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed3480 con 0x8d13de0 -6074> 2014-08-13 16:24:20.469438 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194449 stamp 2014-08-13 16:24:20.465777) v2 -- ?+0 0x7ed32c0 con 0x8d13b20 -6073> 2014-08-13 16:24:20.469631 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.1.115:6839/62713 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x81c2d80 con 0x8ba06e0 -6072> 2014-08-13 16:24:20.469687 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.100.115:6846/62713 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9104d00 con 0x8c698c0 -6071> 2014-08-13 16:24:20.469780 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899b2c0 con 0x8c69600 -6070> 2014-08-13 16:24:20.469814 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9106200 con 0x8c69340 -6069> 2014-08-13 16:24:20.469839 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9117e00 con 0x8c69080 -6068> 2014-08-13 16:24:20.469865 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9110380 con 0x8c68dc0 -6067> 2014-08-13 16:24:20.469889 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79be200 con 0x8c68b00 -6066> 2014-08-13 16:24:20.469932 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8409dc0 con 0x8c68840 -6065> 2014-08-13 16:24:20.469958 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bb800 con 0x8c68580 -6064> 2014-08-13 16:24:20.469986 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840c7c0 con 0x8c6b2e0 -6063> 2014-08-13 16:24:20.470009 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bc7c0 con 0x8c6b020 -6062> 2014-08-13 16:24:20.470038 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bc280 con 0x8c6ad60 -6061> 2014-08-13 16:24:20.470063 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8408e00 con 0x8c6aaa0 -6060> 2014-08-13 16:24:20.470086 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8409340 con 0x8c6a7e0 -6059> 2014-08-13 16:24:20.470116 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83c8380 con 0x8c6a520 -6058> 2014-08-13 16:24:20.470142 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840e200 con 0x8c6a260 -6057> 2014-08-13 16:24:20.470179 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834b800 con 0x8c69fa0 -6056> 2014-08-13 16:24:20.470207 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834d240 con 0x8c39fa0 -6055> 2014-08-13 16:24:20.470232 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b88c0 con 0x8c39ce0 -6054> 2014-08-13 16:24:20.470257 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83c9880 con 0x8c39a20 -6053> 2014-08-13 16:24:20.470280 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840a840 con 0x8c39760 -6052> 2014-08-13 16:24:20.470304 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840b2c0 con 0x8c394a0 -6051> 2014-08-13 16:24:20.470328 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x81c3f00 con 0x8c38f20 -6050> 2014-08-13 16:24:20.470353 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8571340 con 0x8c38c60 -6049> 2014-08-13 16:24:20.470377 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8428380 con 0x8c389a0 -6048> 2014-08-13 16:24:20.470400 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x84288c0 con 0x8c3b9c0 -6047> 2014-08-13 16:24:20.470425 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8576e40 con 0x8c3b700 -6046> 2014-08-13 16:24:20.470447 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79be740 con 0x8c3b440 -6045> 2014-08-13 16:24:20.470471 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bbd40 con 0x8c3b180 -6044> 2014-08-13 16:24:20.470495 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8348e00 con 0x8c3aec0 -6043> 2014-08-13 16:24:20.470519 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8349880 con 0x8c3ac00 -6042> 2014-08-13 16:24:20.470548 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834bd40 con 0x8c3a940 -6041> 2014-08-13 16:24:20.470574 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834b2c0 con 0x8c3a680 -6040> 2014-08-13 16:24:20.470597 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9105b00 con 0x8c3a3c0 -6039> 2014-08-13 16:24:20.470622 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775d5c0 con 0x8c3a100 -6038> 2014-08-13 16:24:20.470647 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9104ec0 con 0x8c3d6a0 -6037> 2014-08-13 16:24:20.470672 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9107e00 con 0x8c3d3e0 -6036> 2014-08-13 16:24:20.470695 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9111340 con 0x8c3d120 -6035> 2014-08-13 16:24:20.470719 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8348380 con 0x8c3ce60 -6034> 2014-08-13 16:24:20.470743 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899a4c0 con 0x8c3cba0 -6033> 2014-08-13 16:24:20.470767 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899c600 con 0x8c3c8e0 -6032> 2014-08-13 16:24:20.470796 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899c440 con 0x8c3c620 -6031> 2014-08-13 16:24:20.470821 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8428e00 con 0x8c3c360 -6030> 2014-08-13 16:24:20.470849 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9107a80 con 0x8c3c0a0 -6029> 2014-08-13 16:24:20.470872 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775b100 con 0x8c3bde0 -6028> 2014-08-13 16:24:20.470895 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834ad80 con 0x8c3e880 -6027> 2014-08-13 16:24:20.470918 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834a840 con 0x8c3e5c0 -6026> 2014-08-13 16:24:20.470942 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed7e00 con 0x8c3e300 -6025> 2014-08-13 16:24:20.470969 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834cd00 con 0x8c618c0 -6024> 2014-08-13 16:24:20.470993 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x85763c0 con 0x8c6fe80 -6023> 2014-08-13 16:24:20.471017 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8577e00 con 0x8c6fbc0 -6022> 2014-08-13 16:24:20.471040 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8408380 con 0x8c6f900 -6021> 2014-08-13 16:24:20.471064 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bf1c0 con 0x8c6f640 -6020> 2014-08-13 16:24:20.471088 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840e740 con 0x8c6f380 -6019> 2014-08-13 16:24:20.471112 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840cd00 con 0x8c6f0c0 -6018> 2014-08-13 16:24:20.471135 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899d080 con 0x8d223c0 -6017> 2014-08-13 16:24:20.471158 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8999500 con 0x8d22100 -6016> 2014-08-13 16:24:20.471188 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.1.115:6814/59915 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899abc0 con 0x8d21e40 -6015> 2014-08-13 16:24:20.471214 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.100.115:6831/59915 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8998c40 con 0x8d21b80 -6014> 2014-08-13 16:24:20.471239 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842f1c0 con 0x8d218c0 -6013> 2014-08-13 16:24:20.471264 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842b2c0 con 0x8d21600 -6012> 2014-08-13 16:24:20.471287 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842e740 con 0x8d21340 -6011> 2014-08-13 16:24:20.471312 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842dcc0 con 0x8d21080 -6010> 2014-08-13 16:24:20.471335 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9116ac0 con 0x8d20dc0 -6009> 2014-08-13 16:24:20.471359 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842fc40 con 0x8d20b00 -6008> 2014-08-13 16:24:20.471382 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842b800 con 0x8d20840 -6007> 2014-08-13 16:24:20.471409 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x91108c0 con 0x8d25960 -6006> 2014-08-13 16:24:20.471435 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9111dc0 con 0x8d256a0 -6005> 2014-08-13 16:24:20.471459 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9117000 con 0x8d253e0 -6004> 2014-08-13 16:24:20.471482 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9116580 con 0x8d25120 -6003> 2014-08-13 16:24:20.471505 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9111880 con 0x8d24e60 -6002> 2014-08-13 16:24:20.471528 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83c9340 con 0x8d24ba0 -6001> 2014-08-13 16:24:20.471552 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840fe00 con 0x8d248e0 -6000> 2014-08-13 16:24:20.471576 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed1340 con 0x8d24620 -5999> 2014-08-13 16:24:20.471600 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b8e00 con 0x8d24360 -5998> 2014-08-13 16:24:20.471623 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed0380 con 0x8d240a0 -5997> 2014-08-13 16:24:20.471653 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834e200 con 0x8d23de0 -5996> 2014-08-13 16:24:20.471677 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79ba840 con 0x8d23b20 -5995> 2014-08-13 16:24:20.471704 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b9dc0 con 0x8d27bc0 -5994> 2014-08-13 16:24:20.471727 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840dcc0 con 0x8d27900 -5993> 2014-08-13 16:24:20.471750 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840f1c0 con 0x8d27640 -5992> 2014-08-13 16:24:20.471774 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed7c40 con 0x8d27380 -5991> 2014-08-13 16:24:20.471797 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b9880 con 0x8d270c0 -5990> 2014-08-13 16:24:20.471824 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842e200 con 0x8d26e00 -5989> 2014-08-13 16:24:20.471853 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842c7c0 con 0x8d26b40 -5988> 2014-08-13 16:24:20.471877 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834ec80 con 0x8d26880 -5987> 2014-08-13 16:24:20.471901 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834e740 con 0x8d265c0 -5986> 2014-08-13 16:24:20.471924 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840fc40 con 0x8d26300 -5985> 2014-08-13 16:24:20.471948 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83c8e00 con 0x8d26040 -5984> 2014-08-13 16:24:20.471971 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x87e3b80 con 0x8d25d80 -5983> 2014-08-13 16:24:20.471994 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x87e3f00 con 0x8d140a0 -5982> 2014-08-13 16:24:20.472018 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9285cc0 con 0x8d13de0 -5981> 2014-08-13 16:24:20.472042 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9283d40 con 0x8d13b20 -5980> 2014-08-13 16:24:20.472066 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7758380 con 0x78844c0 -5979> 2014-08-13 16:24:20.472089 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775d940 con 0x7884200 -5978> 2014-08-13 16:24:20.472113 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775db00 con 0x78848e0 -5977> 2014-08-13 16:24:20.472136 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9105400 con 0x7884620 -5976> 2014-08-13 16:24:20.472159 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9102140 con 0x794c360 -5975> 2014-08-13 16:24:20.472190 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775af40 con 0x794e300 -5974> 2014-08-13 16:24:20.472213 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x91040c0 con 0x794e040 -5973> 2014-08-13 16:24:20.472237 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775fc40 con 0x794dd80 -5972> 2014-08-13 16:24:20.472260 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842bd40 con 0x7ffd960 -5971> 2014-08-13 16:24:20.472286 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842d240 con 0x7ffdc20 -5970> 2014-08-13 16:24:20.472310 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8409880 con 0x7ffdac0 -5969> 2014-08-13 16:24:20.472333 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840d240 con 0x7ffe040 -5968> 2014-08-13 16:24:20.472356 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842c280 con 0x7ffdee0 -5967> 2014-08-13 16:24:20.472380 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842d780 con 0x7ffdd80 -5966> 2014-08-13 16:24:20.472402 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x89981c0 con 0x8b68c60 -5965> 2014-08-13 16:24:20.472426 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8429dc0 con 0x8b68b00 -5964> 2014-08-13 16:24:20.472449 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8570380 con 0x8b689a0 -5963> 2014-08-13 16:24:20.472475 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8575940 con 0x8b68840 -5962> 2014-08-13 16:24:20.472498 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed2840 con 0x8b6e1a0 -5961> 2014-08-13 16:24:20.472529 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b8380 con 0x8b6e040 -5960> 2014-08-13 16:24:20.472552 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79b9340 con 0x8b6dee0 -5959> 2014-08-13 16:24:20.472579 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bb2c0 con 0x8b6dd80 -5958> 2014-08-13 16:24:20.472607 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bcd00 con 0x8b6dc20 -5957> 2014-08-13 16:24:20.472632 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bf700 con 0x7fb9b80 -5956> 2014-08-13 16:24:20.472654 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842f700 con 0x7fb9a20 -5955> 2014-08-13 16:24:20.472677 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842ec80 con 0x7fb98c0 -5954> 2014-08-13 16:24:20.472700 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834dcc0 con 0x7fb9760 -5953> 2014-08-13 16:24:20.472726 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834d780 con 0x7fb9600 -5952> 2014-08-13 16:24:20.472750 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bfc40 con 0x7fb94a0 -5951> 2014-08-13 16:24:20.472774 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bfe00 con 0x7fba520 -5950> 2014-08-13 16:24:20.472797 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899cd00 con 0x7fba3c0 -5949> 2014-08-13 16:24:20.472820 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8570e00 con 0x7fba260 -5948> 2014-08-13 16:24:20.472843 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8575e80 con 0x7fba100 -5947> 2014-08-13 16:24:20.472866 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8999880 con 0x7fb9fa0 -5946> 2014-08-13 16:24:20.472889 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8429340 con 0x7fb9e40 -5945> 2014-08-13 16:24:20.472912 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x85778c0 con 0x7fb9ce0 -5944> 2014-08-13 16:24:20.472934 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842a840 con 0x7fbb020 -5943> 2014-08-13 16:24:20.472957 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8577380 con 0x7fbaec0 -5942> 2014-08-13 16:24:20.472979 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9117a80 con 0x7fbad60 -5941> 2014-08-13 16:24:20.473003 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834c280 con 0x7fbac00 -5940> 2014-08-13 16:24:20.473026 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842fe00 con 0x7fbaaa0 -5939> 2014-08-13 16:24:20.473050 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9110e00 con 0x7fba940 -5938> 2014-08-13 16:24:20.473072 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79ba300 con 0x7fba7e0 -5937> 2014-08-13 16:24:20.473098 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bad80 con 0x7fba680 -5936> 2014-08-13 16:24:20.473121 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840b800 con 0x7fbbc80 -5935> 2014-08-13 16:24:20.473147 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bec80 con 0x7fbbb20 -5934> 2014-08-13 16:24:20.473177 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bd780 con 0x7fbb9c0 -5933> 2014-08-13 16:24:20.473201 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bd240 con 0x7fbb860 -5932> 2014-08-13 16:24:20.473225 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840f700 con 0x7fbb700 -5931> 2014-08-13 16:24:20.473248 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840d780 con 0x7fbb5a0 -5930> 2014-08-13 16:24:20.473271 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842a300 con 0x7fbb440 -5929> 2014-08-13 16:24:20.473295 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842ad80 con 0x7fbb2e0 -5928> 2014-08-13 16:24:20.473317 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9117540 con 0x7fbb180 -5927> 2014-08-13 16:24:20.473340 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8349340 con 0x7fbca40 -5926> 2014-08-13 16:24:20.473363 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83488c0 con 0x7fbc780 -5925> 2014-08-13 16:24:20.473386 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8349dc0 con 0x7fbc4c0 -5924> 2014-08-13 16:24:20.473414 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840bd40 con 0x7fbc200 -5923> 2014-08-13 16:24:20.473438 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x83c88c0 con 0x7fbbf40 -5922> 2014-08-13 16:24:20.473464 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9100e00 con 0x8babde0 -5921> 2014-08-13 16:24:20.473487 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x775b480 con 0x8babb20 -5920> 2014-08-13 16:24:20.473510 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9103b80 con 0x8bab860 -5919> 2014-08-13 16:24:20.473535 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9106580 con 0x8bab5a0 -5918> 2014-08-13 16:24:20.473558 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x842cd00 con 0x8bab2e0 -5917> 2014-08-13 16:24:20.473581 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834a300 con 0x8bab020 -5916> 2014-08-13 16:24:20.473607 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840ad80 con 0x8bacd00 -5915> 2014-08-13 16:24:20.473631 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840ec80 con 0x8baca40 -5914> 2014-08-13 16:24:20.473655 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834c7c0 con 0x8bac780 -5913> 2014-08-13 16:24:20.473679 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834f1c0 con 0x8bac4c0 -5912> 2014-08-13 16:24:20.473702 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834fe00 con 0x8bac200 -5911> 2014-08-13 16:24:20.473727 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed08c0 con 0x8babf40 -5910> 2014-08-13 16:24:20.473751 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed1880 con 0x8bae040 -5909> 2014-08-13 16:24:20.473774 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed0e00 con 0x8badd80 -5908> 2014-08-13 16:24:20.473797 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x899f380 con 0x8badac0 -5907> 2014-08-13 16:24:20.473820 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9102680 con 0x8bad800 -5906> 2014-08-13 16:24:20.473843 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x85708c0 con 0x8bad540 -5905> 2014-08-13 16:24:20.473866 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8429880 con 0x8bad280 -5904> 2014-08-13 16:24:20.473888 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x89996c0 con 0x8bafe80 -5903> 2014-08-13 16:24:20.473912 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x8576900 con 0x8bafbc0 -5902> 2014-08-13 16:24:20.473934 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed7700 con 0x8baf900 -5901> 2014-08-13 16:24:20.473958 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed1dc0 con 0x8baf640 -5900> 2014-08-13 16:24:20.473981 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed2300 con 0x8baf380 -5899> 2014-08-13 16:24:20.474004 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x7ed2d80 con 0x8baf0c0 -5898> 2014-08-13 16:24:20.474027 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x91016c0 con 0x8baee00 -5897> 2014-08-13 16:24:20.474054 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x9100fc0 con 0x8ba1a20 -5896> 2014-08-13 16:24:20.474077 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834f700 con 0x8ba1760 -5895> 2014-08-13 16:24:20.474101 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x834fc40 con 0x8ba14a0 -5894> 2014-08-13 16:24:20.474123 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x79bdcc0 con 0x8ba11e0 -5893> 2014-08-13 16:24:20.474146 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840c280 con 0x8ba0f20 -5892> 2014-08-13 16:24:20.474176 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x84088c0 con 0x8ba0c60 -5891> 2014-08-13 16:24:20.474203 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 4 ==== osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.465777) v2 ==== 47+0+0 (3407294819 0 0) 0x840a300 con 0x8ba09a0 -5890> 2014-08-13 16:24:20.538919 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.538547) v2 ==== 47+0+0 (4045417851 0 0) 0x92839c0 con 0x7eda100 -5889> 2014-08-13 16:24:20.538970 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.538547) v2 -- ?+0 0x7d03d40 con 0x7eda100 -5888> 2014-08-13 16:24:20.538989 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.538547) v2 ==== 47+0+0 (4045417851 0 0) 0x9280fc0 con 0x7edc0a0 -5887> 2014-08-13 16:24:20.539056 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.538547) v2 -- ?+0 0x9284280 con 0x7edc0a0 -5886> 2014-08-13 16:24:20.617279 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.617190) v2 ==== 47+0+0 (4110997863 0 0) 0x9283d40 con 0x7edac00 -5885> 2014-08-13 16:24:20.617334 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.617190) v2 -- ?+0 0x92839c0 con 0x7edac00 -5884> 2014-08-13 16:24:20.617365 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.617190) v2 ==== 47+0+0 (4110997863 0 0) 0x7ed1340 con 0x7edcba0 -5883> 2014-08-13 16:24:20.617414 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.617190) v2 -- ?+0 0x9280fc0 con 0x7edcba0 -5882> 2014-08-13 16:24:20.669693 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.669203) v2 ==== 47+0+0 (743617620 0 0) 0x87e6740 con 0x7ebce60 -5881> 2014-08-13 16:24:20.669745 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.669203) v2 -- ?+0 0x7ed1340 con 0x7ebce60 -5880> 2014-08-13 16:24:20.669775 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.669203) v2 ==== 47+0+0 (743617620 0 0) 0x87e32c0 con 0x7ebb860 -5879> 2014-08-13 16:24:20.669808 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.669203) v2 -- ?+0 0x9283d40 con 0x7ebb860 -5878> 2014-08-13 16:24:20.684497 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.684286) v2 ==== 47+0+0 (2166778457 0 0) 0x9285e80 con 0x7d2dc20 -5877> 2014-08-13 16:24:20.684550 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.684286) v2 -- ?+0 0x87e32c0 con 0x7d2dc20 -5876> 2014-08-13 16:24:20.684551 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.684286) v2 ==== 47+0+0 (2166778457 0 0) 0x9280000 con 0x7d2f0c0 -5875> 2014-08-13 16:24:20.684611 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.684286) v2 -- ?+0 0x87e6740 con 0x7d2f0c0 -5874> 2014-08-13 16:24:20.699434 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 17 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x8bc1600 con 0x4e498c0 -5873> 2014-08-13 16:24:20.699490 7fd93dff0700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699218, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5872> 2014-08-13 16:24:20.699507 7fd93dff0700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699222, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5871> 2014-08-13 16:24:20.699517 7fd93dff0700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699352, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5870> 2014-08-13 16:24:20.699526 7fd93dff0700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699482, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5869> 2014-08-13 16:24:20.699542 7fd93dff0700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699541, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5868> 2014-08-13 16:24:20.699690 7fd9367e4700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699689, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5867> 2014-08-13 16:24:20.699742 7fd9367e4700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699742, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5866> 2014-08-13 16:24:20.699767 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47a80 con 0x4e498c0 -5865> 2014-08-13 16:24:20.699801 7fd9367e4700 5 -- op tracker -- , seq: 812, time: 2014-08-13 16:24:20.699801, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5864> 2014-08-13 16:24:20.700880 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 18 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x76f9600 con 0x4e498c0 -5863> 2014-08-13 16:24:20.700925 7fd93dff0700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.700733, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5862> 2014-08-13 16:24:20.700941 7fd93dff0700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.700736, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5861> 2014-08-13 16:24:20.700951 7fd93dff0700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.700842, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5860> 2014-08-13 16:24:20.700961 7fd93dff0700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.700920, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5859> 2014-08-13 16:24:20.700972 7fd93dff0700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.700972, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5858> 2014-08-13 16:24:20.701050 7fd9371e5700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.701049, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5857> 2014-08-13 16:24:20.701090 7fd9371e5700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.701090, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5856> 2014-08-13 16:24:20.701106 7fd9371e5700 5 -- op tracker -- , seq: 813, time: 2014-08-13 16:24:20.701106, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5855> 2014-08-13 16:24:20.718934 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.718535) v2 ==== 47+0+0 (165385857 0 0) 0x8c8bb80 con 0x7eda680 -5854> 2014-08-13 16:24:20.718994 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.718535) v2 -- ?+0 0x9285e80 con 0x7eda680 -5853> 2014-08-13 16:24:20.718999 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.718535) v2 ==== 47+0+0 (165385857 0 0) 0x9281500 con 0x7edc620 -5852> 2014-08-13 16:24:20.719042 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.718535) v2 -- ?+0 0x9280000 con 0x7edc620 -5851> 2014-08-13 16:24:20.721446 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.720940) v2 ==== 47+0+0 (3182345875 0 0) 0x840d780 con 0x7ab9ce0 -5850> 2014-08-13 16:24:20.721496 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.720940) v2 -- ?+0 0x9281500 con 0x7ab9ce0 -5849> 2014-08-13 16:24:20.721478 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.720940) v2 ==== 47+0+0 (3182345875 0 0) 0x8409880 con 0x7ab9a20 -5848> 2014-08-13 16:24:20.721566 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.720940) v2 -- ?+0 0x8c8bb80 con 0x7ab9a20 -5847> 2014-08-13 16:24:20.806168 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.805843) v2 ==== 47+0+0 (1297403846 0 0) 0x8409dc0 con 0x7edc8e0 -5846> 2014-08-13 16:24:20.806226 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.805843) v2 -- ?+0 0x840d780 con 0x7edc8e0 -5845> 2014-08-13 16:24:20.806156 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 4 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.805843) v2 ==== 47+0+0 (1297403846 0 0) 0x840e740 con 0x7eda940 -5844> 2014-08-13 16:24:20.806271 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.805843) v2 -- ?+0 0x8409880 con 0x7eda940 -5843> 2014-08-13 16:24:20.855187 7fd94ac49700 5 osd.4 194449 tick -5842> 2014-08-13 16:24:20.942967 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.942988) v2 ==== 47+0+0 (1749222315 0 0) 0x7758c40 con 0x7ab8420 -5841> 2014-08-13 16:24:20.942999 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.942988) v2 -- ?+0 0x840e740 con 0x7ab8420 -5840> 2014-08-13 16:24:20.942989 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.942988) v2 ==== 47+0+0 (1749222315 0 0) 0x7d06c80 con 0x7ab8000 -5839> 2014-08-13 16:24:20.943086 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.942988) v2 -- ?+0 0x8409dc0 con 0x7ab8000 -5838> 2014-08-13 16:24:20.946176 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.946274) v2 ==== 47+0+0 (3457154680 0 0) 0x7759500 con 0x7caf0c0 -5837> 2014-08-13 16:24:20.946177 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:20.946274) v2 ==== 47+0+0 (3457154680 0 0) 0x80dd780 con 0x7d286e0 -5836> 2014-08-13 16:24:20.946205 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.946274) v2 -- ?+0 0x7d06c80 con 0x7caf0c0 -5835> 2014-08-13 16:24:20.946237 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:20.946274) v2 -- ?+0 0x7758c40 con 0x7d286e0 -5834> 2014-08-13 16:24:21.013370 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.013498) v2 ==== 47+0+0 (2202370521 0 0) 0x840e200 con 0x7d2a260 -5833> 2014-08-13 16:24:21.013397 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.013498) v2 -- ?+0 0x7759500 con 0x7d2a260 -5832> 2014-08-13 16:24:21.013395 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.013498) v2 ==== 47+0+0 (2202370521 0 0) 0x842ec80 con 0x7d28160 -5831> 2014-08-13 16:24:21.013513 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.013498) v2 -- ?+0 0x80dd780 con 0x7d28160 -5830> 2014-08-13 16:24:21.016045 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 19 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x8bc3700 con 0x5111fa0 -5829> 2014-08-13 16:24:21.016078 7fd93dff0700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.015892, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5828> 2014-08-13 16:24:21.016088 7fd93dff0700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.015894, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5827> 2014-08-13 16:24:21.016093 7fd93dff0700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.015984, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5826> 2014-08-13 16:24:21.016106 7fd93dff0700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.016072, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5825> 2014-08-13 16:24:21.016113 7fd93dff0700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.016112, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5824> 2014-08-13 16:24:21.016192 7fd9367e4700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.016192, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5823> 2014-08-13 16:24:21.016216 7fd9367e4700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.016215, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5822> 2014-08-13 16:24:21.016228 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46900 con 0x5111fa0 -5821> 2014-08-13 16:24:21.016245 7fd9367e4700 5 -- op tracker -- , seq: 814, time: 2014-08-13 16:24:21.016245, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5820> 2014-08-13 16:24:21.016745 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 23 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x86da100 con 0x510fd20 -5819> 2014-08-13 16:24:21.016766 7fd93dff0700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016674, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5818> 2014-08-13 16:24:21.016774 7fd93dff0700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016675, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5817> 2014-08-13 16:24:21.016779 7fd93dff0700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016725, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5816> 2014-08-13 16:24:21.016784 7fd93dff0700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016763, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5815> 2014-08-13 16:24:21.016790 7fd93dff0700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016790, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5814> 2014-08-13 16:24:21.016839 7fd9371e5700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016839, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5813> 2014-08-13 16:24:21.016866 7fd9371e5700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016866, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5812> 2014-08-13 16:24:21.016877 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002a80 con 0x510fd20 -5811> 2014-08-13 16:24:21.016895 7fd9371e5700 5 -- op tracker -- , seq: 815, time: 2014-08-13 16:24:21.016894, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5810> 2014-08-13 16:24:21.017273 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 20 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x8bc4200 con 0x5111fa0 -5809> 2014-08-13 16:24:21.017303 7fd93dff0700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017179, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5808> 2014-08-13 16:24:21.017312 7fd93dff0700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017180, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5807> 2014-08-13 16:24:21.017318 7fd93dff0700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017230, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5806> 2014-08-13 16:24:21.017323 7fd93dff0700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017299, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5805> 2014-08-13 16:24:21.017330 7fd93dff0700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017329, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5804> 2014-08-13 16:24:21.017396 7fd9367e4700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017396, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5803> 2014-08-13 16:24:21.017418 7fd9367e4700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017417, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5802> 2014-08-13 16:24:21.017426 7fd9367e4700 5 -- op tracker -- , seq: 816, time: 2014-08-13 16:24:21.017426, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5801> 2014-08-13 16:24:21.017747 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 24 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x86d9600 con 0x510fd20 -5800> 2014-08-13 16:24:21.017781 7fd93dff0700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017658, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5799> 2014-08-13 16:24:21.017791 7fd93dff0700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017659, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5798> 2014-08-13 16:24:21.017797 7fd93dff0700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017709, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5797> 2014-08-13 16:24:21.017802 7fd93dff0700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017775, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5796> 2014-08-13 16:24:21.017808 7fd93dff0700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017808, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5795> 2014-08-13 16:24:21.017840 7fd9371e5700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017840, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5794> 2014-08-13 16:24:21.017856 7fd9371e5700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017856, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5793> 2014-08-13 16:24:21.017864 7fd9371e5700 5 -- op tracker -- , seq: 817, time: 2014-08-13 16:24:21.017864, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5792> 2014-08-13 16:24:21.088412 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.088240) v2 ==== 47+0+0 (637497998 0 0) 0x9281880 con 0x7eb91e0 -5791> 2014-08-13 16:24:21.088414 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.088240) v2 ==== 47+0+0 (637497998 0 0) 0x89108c0 con 0x7eba680 -5790> 2014-08-13 16:24:21.088444 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.088240) v2 -- ?+0 0x842ec80 con 0x7eb91e0 -5789> 2014-08-13 16:24:21.088492 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.088240) v2 -- ?+0 0x840e200 con 0x7eba680 -5788> 2014-08-13 16:24:21.430441 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.430104) v2 ==== 47+0+0 (4256520697 0 0) 0x9283640 con 0x7edbf40 -5787> 2014-08-13 16:24:21.430461 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.430104) v2 ==== 47+0+0 (4256520697 0 0) 0x9281180 con 0x7ffeca0 -5786> 2014-08-13 16:24:21.430486 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.430104) v2 -- ?+0 0x89108c0 con 0x7edbf40 -5785> 2014-08-13 16:24:21.430566 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.430104) v2 -- ?+0 0x9281880 con 0x7ffeca0 -5784> 2014-08-13 16:24:21.527111 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.347 192.168.1.106:6800/4640 8 ==== osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+11549 (1701821610 0 542558679) 0x76f8b00 con 0x50bbf40 -5783> 2014-08-13 16:24:21.527174 7fd93dff0700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:21.526877, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5782> 2014-08-13 16:24:21.527196 7fd93dff0700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:21.526882, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5781> 2014-08-13 16:24:21.527207 7fd93dff0700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:21.527045, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5780> 2014-08-13 16:24:21.527220 7fd93dff0700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:21.527156, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5779> 2014-08-13 16:24:21.527242 7fd93dff0700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:21.527241, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5778> 2014-08-13 16:24:21.546495 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 16 ==== osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+11549 (916975984 0 542558679) 0x7ecac00 con 0x4fe69e0 -5777> 2014-08-13 16:24:21.546541 7fd93dff0700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:21.546318, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5776> 2014-08-13 16:24:21.546557 7fd93dff0700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:21.546323, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5775> 2014-08-13 16:24:21.546566 7fd93dff0700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:21.546455, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5774> 2014-08-13 16:24:21.546575 7fd93dff0700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:21.546535, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5773> 2014-08-13 16:24:21.546586 7fd93dff0700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:21.546586, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5772> 2014-08-13 16:24:21.642614 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.642519) v2 ==== 47+0+0 (1886284767 0 0) 0x9283b80 con 0x7ffe880 -5771> 2014-08-13 16:24:21.642675 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.642519) v2 -- ?+0 0x9281180 con 0x7ffe880 -5770> 2014-08-13 16:24:21.642741 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.642519) v2 ==== 47+0+0 (1886284767 0 0) 0x9281f80 con 0x7edbb20 -5769> 2014-08-13 16:24:21.642778 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.642519) v2 -- ?+0 0x9283640 con 0x7edbb20 -5768> 2014-08-13 16:24:21.647787 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.647364) v2 ==== 47+0+0 (882984914 0 0) 0x8c89180 con 0x7eba100 -5767> 2014-08-13 16:24:21.647808 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.647364) v2 ==== 47+0+0 (882984914 0 0) 0x9282840 con 0x7eb8c60 -5766> 2014-08-13 16:24:21.647835 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.647364) v2 -- ?+0 0x9281f80 con 0x7eba100 -5765> 2014-08-13 16:24:21.647940 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.647364) v2 -- ?+0 0x9283b80 con 0x7eb8c60 -5764> 2014-08-13 16:24:21.762718 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.762888) v2 ==== 47+0+0 (2959235458 0 0) 0x87e39c0 con 0x7ebc0a0 -5763> 2014-08-13 16:24:21.762771 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.762888) v2 -- ?+0 0x9282840 con 0x7ebc0a0 -5762> 2014-08-13 16:24:21.762804 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:21.762888) v2 ==== 47+0+0 (2959235458 0 0) 0x7d03f00 con 0x7ebd6a0 -5761> 2014-08-13 16:24:21.762858 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:21.762888) v2 -- ?+0 0x8c89180 con 0x7ebd6a0 -5760> 2014-08-13 16:24:21.855315 7fd94ac49700 5 osd.4 194449 tick -5759> 2014-08-13 16:24:22.016369 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 21 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4270054503 0 0) 0x8bc0000 con 0x5111fa0 -5758> 2014-08-13 16:24:22.016424 7fd93dff0700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:22.016164, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5757> 2014-08-13 16:24:22.016444 7fd93dff0700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:22.016168, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5756> 2014-08-13 16:24:22.016455 7fd93dff0700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:22.016286, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5755> 2014-08-13 16:24:22.016468 7fd93dff0700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:22.016417, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5754> 2014-08-13 16:24:22.016482 7fd93dff0700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:22.016482, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5753> 2014-08-13 16:24:22.017552 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 25 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2879098172 0 0) 0x76fa100 con 0x510fd20 -5752> 2014-08-13 16:24:22.017598 7fd93dff0700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:22.017413, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5751> 2014-08-13 16:24:22.017614 7fd93dff0700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:22.017416, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5750> 2014-08-13 16:24:22.017624 7fd93dff0700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:22.017519, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5749> 2014-08-13 16:24:22.017634 7fd93dff0700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:22.017593, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5748> 2014-08-13 16:24:22.017648 7fd93dff0700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:22.017648, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5747> 2014-08-13 16:24:22.045297 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.045171) v2 ==== 47+0+0 (3356306242 0 0) 0x7759c00 con 0x7ab8420 -5746> 2014-08-13 16:24:22.045352 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.045171) v2 -- ?+0 0x87e39c0 con 0x7ab8420 -5745> 2014-08-13 16:24:22.045340 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.045171) v2 ==== 47+0+0 (3356306242 0 0) 0x7d07a80 con 0x7ab8000 -5744> 2014-08-13 16:24:22.045486 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.045171) v2 -- ?+0 0x7d03f00 con 0x7ab8000 -5743> 2014-08-13 16:24:22.048712 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.048630) v2 ==== 47+0+0 (663031006 0 0) 0x8c89180 con 0x7d2aec0 -5742> 2014-08-13 16:24:22.048766 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.048630) v2 -- ?+0 0x7759c00 con 0x7d2aec0 -5741> 2014-08-13 16:24:22.048760 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.048630) v2 ==== 47+0+0 (663031006 0 0) 0x7d03f00 con 0x7d2c620 -5740> 2014-08-13 16:24:22.048804 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.048630) v2 -- ?+0 0x7d07a80 con 0x7d2c620 -5739> 2014-08-13 16:24:22.149464 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.149573) v2 ==== 47+0+0 (10529160 0 0) 0x775e3c0 con 0x7ebd120 -5738> 2014-08-13 16:24:22.149474 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.149573) v2 ==== 47+0+0 (10529160 0 0) 0x92808c0 con 0x7ebbb20 -5737> 2014-08-13 16:24:22.149516 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.149573) v2 -- ?+0 0x7d03f00 con 0x7ebd120 -5736> 2014-08-13 16:24:22.149601 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.149573) v2 -- ?+0 0x8c89180 con 0x7ebbb20 -5735> 2014-08-13 16:24:22.161964 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.161795) v2 ==== 47+0+0 (2647677671 0 0) 0x9287540 con 0x7cae880 -5734> 2014-08-13 16:24:22.161972 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.161795) v2 ==== 47+0+0 (2647677671 0 0) 0x9283f00 con 0x7cae040 -5733> 2014-08-13 16:24:22.162020 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.161795) v2 -- ?+0 0x92808c0 con 0x7cae880 -5732> 2014-08-13 16:24:22.162140 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.161795) v2 -- ?+0 0x775e3c0 con 0x7cae040 -5731> 2014-08-13 16:24:22.192457 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.192141) v2 ==== 47+0+0 (3676086611 0 0) 0x87e5080 con 0x7ffe9e0 -5730> 2014-08-13 16:24:22.192510 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.192141) v2 -- ?+0 0x9287540 con 0x7ffe9e0 -5729> 2014-08-13 16:24:22.192522 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.192141) v2 ==== 47+0+0 (3676086611 0 0) 0x87e3100 con 0x7edbc80 -5728> 2014-08-13 16:24:22.192575 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.192141) v2 -- ?+0 0x9283f00 con 0x7edbc80 -5727> 2014-08-13 16:24:22.198635 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.198394) v2 ==== 47+0+0 (3253127498 0 0) 0x87e5240 con 0x7edbde0 -5726> 2014-08-13 16:24:22.198636 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 5 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.198394) v2 ==== 47+0+0 (3253127498 0 0) 0x87e6c80 con 0x7ffeb40 -5725> 2014-08-13 16:24:22.198707 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.198394) v2 -- ?+0 0x87e3100 con 0x7edbde0 -5724> 2014-08-13 16:24:22.198867 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.198394) v2 -- ?+0 0x87e5080 con 0x7ffeb40 -5723> 2014-08-13 16:24:22.244313 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.244247) v2 ==== 47+0+0 (1583926317 0 0) 0x87e6ac0 con 0x7d29fa0 -5722> 2014-08-13 16:24:22.244341 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.244247) v2 ==== 47+0+0 (1583926317 0 0) 0x8c8f1c0 con 0x7d2b700 -5721> 2014-08-13 16:24:22.244367 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.244247) v2 -- ?+0 0x87e5240 con 0x7d29fa0 -5720> 2014-08-13 16:24:22.244658 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.244247) v2 -- ?+0 0x87e6c80 con 0x7d2b700 -5719> 2014-08-13 16:24:22.389069 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.388845) v2 ==== 47+0+0 (2616391836 0 0) 0x92878c0 con 0x7d2f0c0 -5718> 2014-08-13 16:24:22.389069 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.388845) v2 ==== 47+0+0 (2616391836 0 0) 0x9282d80 con 0x7d2dc20 -5717> 2014-08-13 16:24:22.389120 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.388845) v2 -- ?+0 0x87e6ac0 con 0x7d2f0c0 -5716> 2014-08-13 16:24:22.389183 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.388845) v2 -- ?+0 0x8c8f1c0 con 0x7d2dc20 -5715> 2014-08-13 16:24:22.420366 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.420042) v2 ==== 47+0+0 (2824782377 0 0) 0x840e740 con 0x7eda680 -5714> 2014-08-13 16:24:22.420381 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.420042) v2 ==== 47+0+0 (2824782377 0 0) 0x7d001c0 con 0x7edc620 -5713> 2014-08-13 16:24:22.420420 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.420042) v2 -- ?+0 0x9282d80 con 0x7eda680 -5712> 2014-08-13 16:24:22.420530 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.420042) v2 -- ?+0 0x92878c0 con 0x7edc620 -5711> 2014-08-13 16:24:22.439808 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.439442) v2 ==== 47+0+0 (2169759149 0 0) 0x9282840 con 0x7caeb40 -5710> 2014-08-13 16:24:22.439862 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.439442) v2 -- ?+0 0x840e740 con 0x7caeb40 -5709> 2014-08-13 16:24:22.439815 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.439442) v2 ==== 47+0+0 (2169759149 0 0) 0x9286040 con 0x7cae300 -5708> 2014-08-13 16:24:22.439986 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.439442) v2 -- ?+0 0x7d001c0 con 0x7cae300 -5707> 2014-08-13 16:24:22.692490 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.692641) v2 ==== 47+0+0 (1396465307 0 0) 0x842ec80 con 0x7cae5c0 -5706> 2014-08-13 16:24:22.692537 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.692641) v2 -- ?+0 0x9282840 con 0x7cae5c0 -5705> 2014-08-13 16:24:22.692579 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:22.692641) v2 ==== 47+0+0 (1396465307 0 0) 0x842a300 con 0x7d2a520 -5704> 2014-08-13 16:24:22.692631 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:22.692641) v2 -- ?+0 0x9286040 con 0x7d2a520 -5703> 2014-08-13 16:24:22.700182 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 19 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1085755931 0 0) 0x863e300 con 0x4e498c0 -5702> 2014-08-13 16:24:22.700235 7fd93dff0700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:22.699967, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5701> 2014-08-13 16:24:22.700252 7fd93dff0700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:22.699970, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5700> 2014-08-13 16:24:22.700266 7fd93dff0700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:22.700097, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5699> 2014-08-13 16:24:22.700279 7fd93dff0700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:22.700228, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5698> 2014-08-13 16:24:22.700294 7fd93dff0700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:22.700294, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5697> 2014-08-13 16:24:22.702147 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 20 ==== osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3521544638 0 0) 0x8638000 con 0x4e498c0 -5696> 2014-08-13 16:24:22.702212 7fd93dff0700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:22.701978, event: header_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5695> 2014-08-13 16:24:22.702232 7fd93dff0700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:22.701980, event: throttled, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5694> 2014-08-13 16:24:22.702246 7fd93dff0700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:22.702069, event: all_read, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5693> 2014-08-13 16:24:22.702259 7fd93dff0700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:22.702206, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5692> 2014-08-13 16:24:22.702274 7fd93dff0700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:22.702274, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5691> 2014-08-13 16:24:22.855462 7fd94ac49700 5 osd.4 194449 tick -5690> 2014-08-13 16:24:22.946884 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 23 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x86dcd00 con 0x508bde0 -5689> 2014-08-13 16:24:22.946941 7fd93dff0700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:22.946684, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5688> 2014-08-13 16:24:22.946957 7fd93dff0700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:22.946688, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5687> 2014-08-13 16:24:22.946967 7fd93dff0700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:22.946802, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5686> 2014-08-13 16:24:22.946976 7fd93dff0700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:22.946933, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5685> 2014-08-13 16:24:22.946996 7fd93dff0700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:22.946995, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5684> 2014-08-13 16:24:23.097537 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.096897) v2 ==== 47+0+0 (492701988 0 0) 0x842bd40 con 0x7ebfe80 -5683> 2014-08-13 16:24:23.097592 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.096897) v2 -- ?+0 0x842a300 con 0x7ebfe80 -5682> 2014-08-13 16:24:23.097589 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.096897) v2 ==== 47+0+0 (492701988 0 0) 0x8429dc0 con 0x7ebf380 -5681> 2014-08-13 16:24:23.097628 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.096897) v2 -- ?+0 0x842ec80 con 0x7ebf380 -5680> 2014-08-13 16:24:23.230627 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.230314) v2 ==== 47+0+0 (2186516985 0 0) 0x80dd240 con 0x7ebd3e0 -5679> 2014-08-13 16:24:23.230675 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.230314) v2 -- ?+0 0x842bd40 con 0x7ebd3e0 -5678> 2014-08-13 16:24:23.230710 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.230314) v2 ==== 47+0+0 (2186516985 0 0) 0x8c8c7c0 con 0x7ebbde0 -5677> 2014-08-13 16:24:23.230752 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.230314) v2 -- ?+0 0x8429dc0 con 0x7ebbde0 -5676> 2014-08-13 16:24:23.297499 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.296484) v2 ==== 47+0+0 (1394901681 0 0) 0x87e7000 con 0x7edbc80 -5675> 2014-08-13 16:24:23.297549 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.296484) v2 -- ?+0 0x80dd240 con 0x7edbc80 -5674> 2014-08-13 16:24:23.297607 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.296484) v2 ==== 47+0+0 (1394901681 0 0) 0x87e7380 con 0x7ffe9e0 -5673> 2014-08-13 16:24:23.297649 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.296484) v2 -- ?+0 0x8c8c7c0 con 0x7ffe9e0 -5672> 2014-08-13 16:24:23.303654 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.303302) v2 ==== 47+0+0 (1280731544 0 0) 0x9286c80 con 0x7eba3c0 -5671> 2014-08-13 16:24:23.303704 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.303302) v2 -- ?+0 0x87e7000 con 0x7eba3c0 -5670> 2014-08-13 16:24:23.303689 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.303302) v2 ==== 47+0+0 (1280731544 0 0) 0x9285240 con 0x7eb8f20 -5669> 2014-08-13 16:24:23.303851 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.303302) v2 -- ?+0 0x87e7380 con 0x7eb8f20 -5668> 2014-08-13 16:24:23.448038 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.447572) v2 ==== 47+0+0 (2493062996 0 0) 0x8c8af40 con 0x7ebc620 -5667> 2014-08-13 16:24:23.448038 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.447572) v2 ==== 47+0+0 (2493062996 0 0) 0x9285940 con 0x7eb9b80 -5666> 2014-08-13 16:24:23.448096 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.447572) v2 -- ?+0 0x9285240 con 0x7ebc620 -5665> 2014-08-13 16:24:23.448141 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.447572) v2 -- ?+0 0x9286c80 con 0x7eb9b80 -5664> 2014-08-13 16:24:23.480762 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.480638) v2 ==== 47+0+0 (2604800559 0 0) 0x87e55c0 con 0x7d289a0 -5663> 2014-08-13 16:24:23.480809 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.480638) v2 -- ?+0 0x8c8af40 con 0x7d289a0 -5662> 2014-08-13 16:24:23.480822 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.480638) v2 ==== 47+0+0 (2604800559 0 0) 0x9281c00 con 0x7caf380 -5661> 2014-08-13 16:24:23.480911 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.480638) v2 -- ?+0 0x9285940 con 0x7caf380 -5660> 2014-08-13 16:24:23.492108 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 10 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.491829) v2 ==== 47+0+0 (2513815555 0 0) 0x92847c0 con 0x7d2f0c0 -5659> 2014-08-13 16:24:23.492167 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.491829) v2 -- ?+0 0x9281c00 con 0x7d2f0c0 -5658> 2014-08-13 16:24:23.492179 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 10 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.491829) v2 ==== 47+0+0 (2513815555 0 0) 0x80d9c00 con 0x7d2dc20 -5657> 2014-08-13 16:24:23.492219 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.491829) v2 -- ?+0 0x87e55c0 con 0x7d2dc20 -5656> 2014-08-13 16:24:23.548644 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.548420) v2 ==== 47+0+0 (1883706205 0 0) 0x9283800 con 0x7ed8dc0 -5655> 2014-08-13 16:24:23.548674 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.548420) v2 ==== 47+0+0 (1883706205 0 0) 0x7d06900 con 0x7ebee00 -5654> 2014-08-13 16:24:23.548696 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.548420) v2 -- ?+0 0x92847c0 con 0x7ed8dc0 -5653> 2014-08-13 16:24:23.548846 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.548420) v2 -- ?+0 0x80d9c00 con 0x7ebee00 -5652> 2014-08-13 16:24:23.600013 7fd9367e4700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:23.600012, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5651> 2014-08-13 16:24:23.600063 7fd9367e4700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:23.600062, event: started, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5650> 2014-08-13 16:24:23.600209 7fd9367e4700 5 -- op tracker -- , seq: 818, time: 2014-08-13 16:24:23.600209, event: done, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5649> 2014-08-13 16:24:23.600307 7fd9371e5700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:23.600264, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5648> 2014-08-13 16:24:23.600368 7fd9371e5700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:23.600368, event: started, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5647> 2014-08-13 16:24:23.600389 7fd9367e4700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:23.600329, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5646> 2014-08-13 16:24:23.600425 7fd9367e4700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:23.600424, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5645> 2014-08-13 16:24:23.600446 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d46e00 con 0x5111fa0 -5644> 2014-08-13 16:24:23.600480 7fd9367e4700 5 -- op tracker -- , seq: 820, time: 2014-08-13 16:24:23.600480, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5643> 2014-08-13 16:24:23.600513 7fd9371e5700 5 -- op tracker -- , seq: 819, time: 2014-08-13 16:24:23.600500, event: done, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5642> 2014-08-13 16:24:23.600571 7fd9371e5700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:23.600570, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5641> 2014-08-13 16:24:23.600594 7fd9371e5700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:23.600593, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5640> 2014-08-13 16:24:23.600608 7fd9367e4700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:23.600523, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5639> 2014-08-13 16:24:23.600611 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003700 con 0x510fd20 -5638> 2014-08-13 16:24:23.600595 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub osd.4 has 25 items -5637> 2014-08-13 16:24:23.600645 7fd9367e4700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:23.600644, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5636> 2014-08-13 16:24:23.600653 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 340 has 25 items -5635> 2014-08-13 16:24:23.600671 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6817/53997 -- osd_sub_op_reply(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002d00 con 0x4e498c0 -5634> 2014-08-13 16:24:23.600673 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 347 has 25 items -5633> 2014-08-13 16:24:23.600686 7fd9371e5700 5 -- op tracker -- , seq: 821, time: 2014-08-13 16:24:23.600649, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5632> 2014-08-13 16:24:23.600746 7fd9371e5700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:23.600746, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5631> 2014-08-13 16:24:23.600769 7fd9371e5700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:23.600769, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5630> 2014-08-13 16:24:23.600791 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8d780 con 0x508bde0 -5629> 2014-08-13 16:24:23.600790 7fd9367e4700 5 -- op tracker -- , seq: 822, time: 2014-08-13 16:24:23.600704, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5628> 2014-08-13 16:24:23.600827 7fd9371e5700 5 -- op tracker -- , seq: 824, time: 2014-08-13 16:24:23.600821, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5627> 2014-08-13 16:24:23.600887 7fd9371e5700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:23.600869, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5626> 2014-08-13 16:24:23.600913 7fd9371e5700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:23.600913, event: started, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5625> 2014-08-13 16:24:23.600931 7fd9371e5700 5 -- op tracker -- , seq: 823, time: 2014-08-13 16:24:23.600931, event: done, request: osd_sub_op(unknown.0.0:0 22.28f 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5624> 2014-08-13 16:24:23.601713 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] -5623> 2014-08-13 16:24:23.601891 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 26 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (986341017 0 0) 0x76f9600 con 0x510fd20 -5622> 2014-08-13 16:24:23.601941 7fd93dff0700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.601714, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5621> 2014-08-13 16:24:23.601957 7fd93dff0700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.601717, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5620> 2014-08-13 16:24:23.601967 7fd93dff0700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.601828, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5619> 2014-08-13 16:24:23.601977 7fd93dff0700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.601934, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5618> 2014-08-13 16:24:23.601992 7fd93dff0700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.601992, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5617> 2014-08-13 16:24:23.602034 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 22 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1876037570 0 0) 0x8bc6300 con 0x5111fa0 -5616> 2014-08-13 16:24:23.602047 7fd9367e4700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.602046, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5615> 2014-08-13 16:24:23.602073 7fd9367e4700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.602071, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5614> 2014-08-13 16:24:23.602089 7fd9367e4700 5 -- op tracker -- , seq: 825, time: 2014-08-13 16:24:23.602089, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5613> 2014-08-13 16:24:23.602119 7fd93dff0700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.601765, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5612> 2014-08-13 16:24:23.602147 7fd93dff0700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.601769, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5611> 2014-08-13 16:24:23.602157 7fd93dff0700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.601870, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5610> 2014-08-13 16:24:23.602166 7fd93dff0700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.602054, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5609> 2014-08-13 16:24:23.602188 7fd93dff0700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.602187, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5608> 2014-08-13 16:24:23.602220 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 24 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x76fd800 con 0x508bde0 -5607> 2014-08-13 16:24:23.602245 7fd93dff0700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.601866, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5606> 2014-08-13 16:24:23.602259 7fd93dff0700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.601868, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5605> 2014-08-13 16:24:23.602272 7fd93dff0700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.601952, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5604> 2014-08-13 16:24:23.602291 7fd93dff0700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.602242, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5603> 2014-08-13 16:24:23.602308 7fd93dff0700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.602308, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5602> 2014-08-13 16:24:23.602326 7fd9371e5700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.602283, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5601> 2014-08-13 16:24:23.602374 7fd9371e5700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.602373, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5600> 2014-08-13 16:24:23.602393 7fd9371e5700 5 -- op tracker -- , seq: 826, time: 2014-08-13 16:24:23.602392, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5599> 2014-08-13 16:24:23.602425 7fd9367e4700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.602425, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5598> 2014-08-13 16:24:23.602467 7fd9367e4700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.602466, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5597> 2014-08-13 16:24:23.602486 7fd9367e4700 5 -- op tracker -- , seq: 827, time: 2014-08-13 16:24:23.602485, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5596> 2014-08-13 16:24:23.602754 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- replica scrub(pg: 22.1b8,from:0'0,to:194215'479,epoch:194449,start:2b4699b8//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -- ?+0 0x7cf0fc0 -5595> 2014-08-13 16:24:23.602804 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- replica scrub(pg: 22.1b8,from:0'0,to:194215'479,epoch:194449,start:2b4699b8//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -- ?+0 0x7cf2400 -5594> 2014-08-13 16:24:23.851758 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.851840) v2 ==== 47+0+0 (2018004030 0 0) 0x842dcc0 con 0x7ebbb20 -5593> 2014-08-13 16:24:23.851818 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.851840) v2 -- ?+0 0x7d06900 con 0x7ebbb20 -5592> 2014-08-13 16:24:23.851836 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.851840) v2 ==== 47+0+0 (2018004030 0 0) 0x84288c0 con 0x7ebd120 -5591> 2014-08-13 16:24:23.851903 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.851840) v2 -- ?+0 0x9283800 con 0x7ebd120 -5590> 2014-08-13 16:24:23.855594 7fd94ac49700 5 osd.4 194449 tick -5589> 2014-08-13 16:24:23.855645 7fd94ac49700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5588> 2014-08-13 16:24:23.855658 7fd94ac49700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- pg_stats(1 pgs tid 2 v 0) v1 -- ?+0 0x8003e80 con 0x4e3dd80 -5587> 2014-08-13 16:24:23.915449 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.915342) v2 ==== 47+0+0 (1265820337 0 0) 0x8409340 con 0x7d2a260 -5586> 2014-08-13 16:24:23.915502 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.915342) v2 -- ?+0 0x84288c0 con 0x7d2a260 -5585> 2014-08-13 16:24:23.915523 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.915342) v2 ==== 47+0+0 (1265820337 0 0) 0x842f700 con 0x7d28160 -5584> 2014-08-13 16:24:23.915576 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.915342) v2 -- ?+0 0x842dcc0 con 0x7d28160 -5583> 2014-08-13 16:24:23.927806 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.927616) v2 ==== 47+0+0 (2726396006 0 0) 0x87e6580 con 0x7ed82c0 -5582> 2014-08-13 16:24:23.927823 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 8 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.927616) v2 ==== 47+0+0 (2726396006 0 0) 0x87e5400 con 0x7edb180 -5581> 2014-08-13 16:24:23.927851 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.927616) v2 -- ?+0 0x8409340 con 0x7ed82c0 -5580> 2014-08-13 16:24:23.928070 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.927616) v2 -- ?+0 0x842f700 con 0x7edb180 -5579> 2014-08-13 16:24:23.947326 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 25 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3786964101 0 0) 0x76fcd00 con 0x508bde0 -5578> 2014-08-13 16:24:23.947382 7fd93dff0700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947111, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5577> 2014-08-13 16:24:23.947399 7fd93dff0700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947115, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5576> 2014-08-13 16:24:23.947409 7fd93dff0700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947243, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5575> 2014-08-13 16:24:23.947419 7fd93dff0700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947375, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5574> 2014-08-13 16:24:23.947434 7fd93dff0700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947434, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5573> 2014-08-13 16:24:23.947546 7fd9371e5700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947545, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5572> 2014-08-13 16:24:23.947592 7fd9371e5700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947592, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5571> 2014-08-13 16:24:23.947612 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8e400 con 0x508bde0 -5570> 2014-08-13 16:24:23.947645 7fd9371e5700 5 -- op tracker -- , seq: 828, time: 2014-08-13 16:24:23.947645, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5569> 2014-08-13 16:24:23.948704 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 26 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1894362912 0 0) 0x76fc200 con 0x508bde0 -5568> 2014-08-13 16:24:23.948756 7fd93dff0700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948520, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5567> 2014-08-13 16:24:23.948773 7fd93dff0700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948523, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5566> 2014-08-13 16:24:23.948789 7fd93dff0700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948623, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5565> 2014-08-13 16:24:23.948798 7fd93dff0700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948750, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5564> 2014-08-13 16:24:23.948814 7fd93dff0700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948813, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5563> 2014-08-13 16:24:23.948939 7fd9367e4700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948939, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5562> 2014-08-13 16:24:23.948980 7fd9367e4700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948980, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5561> 2014-08-13 16:24:23.948995 7fd9367e4700 5 -- op tracker -- , seq: 829, time: 2014-08-13 16:24:23.948995, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5560> 2014-08-13 16:24:23.949916 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.949449) v2 ==== 47+0+0 (3036066431 0 0) 0x8c8f000 con 0x7eba100 -5559> 2014-08-13 16:24:23.949937 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 9 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.949449) v2 ==== 47+0+0 (3036066431 0 0) 0x9283480 con 0x7eb8c60 -5558> 2014-08-13 16:24:23.949970 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.949449) v2 -- ?+0 0x87e6580 con 0x7eba100 -5557> 2014-08-13 16:24:23.950043 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.949449) v2 -- ?+0 0x87e5400 con 0x7eb8c60 -5556> 2014-08-13 16:24:23.993819 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.993646) v2 ==== 47+0+0 (1846081875 0 0) 0x7d06e40 con 0x7d2cba0 -5555> 2014-08-13 16:24:23.993867 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.993646) v2 -- ?+0 0x8c8f000 con 0x7d2cba0 -5554> 2014-08-13 16:24:23.993868 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:23.993646) v2 ==== 47+0+0 (1846081875 0 0) 0x8c881c0 con 0x7d28420 -5553> 2014-08-13 16:24:23.993915 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:23.993646) v2 -- ?+0 0x9283480 con 0x7d28420 -5552> 2014-08-13 16:24:24.331758 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.331575) v2 ==== 47+0+0 (3233600597 0 0) 0x9283b80 con 0x7ebd3e0 -5551> 2014-08-13 16:24:24.331808 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.331575) v2 -- ?+0 0x7d06e40 con 0x7ebd3e0 -5550> 2014-08-13 16:24:24.331903 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.331575) v2 ==== 47+0+0 (3233600597 0 0) 0x8577380 con 0x7ebbde0 -5549> 2014-08-13 16:24:24.331951 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.331575) v2 -- ?+0 0x8c881c0 con 0x7ebbde0 -5548> 2014-08-13 16:24:24.370015 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 17 ==== osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+4179 (916975984 0 3025586153) 0x86d8b00 con 0x4fe69e0 -5547> 2014-08-13 16:24:24.370067 7fd93dff0700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:24.369794, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5546> 2014-08-13 16:24:24.370083 7fd93dff0700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:24.369799, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5545> 2014-08-13 16:24:24.370093 7fd93dff0700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:24.369933, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5544> 2014-08-13 16:24:24.370113 7fd93dff0700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:24.370060, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5543> 2014-08-13 16:24:24.370139 7fd93dff0700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:24.370139, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5542> 2014-08-13 16:24:24.378902 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.347 192.168.1.106:6800/4640 9 ==== osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+4179 (1701821610 0 3025586153) 0x7ec8b00 con 0x50bbf40 -5541> 2014-08-13 16:24:24.378954 7fd93dff0700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:24.378692, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5540> 2014-08-13 16:24:24.378970 7fd93dff0700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:24.378696, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5539> 2014-08-13 16:24:24.378980 7fd93dff0700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:24.378819, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5538> 2014-08-13 16:24:24.378989 7fd93dff0700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:24.378948, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5537> 2014-08-13 16:24:24.379005 7fd93dff0700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:24.379005, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5536> 2014-08-13 16:24:24.463394 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.463253) v2 ==== 47+0+0 (837706141 0 0) 0x8577e00 con 0x7cae880 -5535> 2014-08-13 16:24:24.463449 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.463253) v2 -- ?+0 0x8577380 con 0x7cae880 -5534> 2014-08-13 16:24:24.463439 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.463253) v2 ==== 47+0+0 (837706141 0 0) 0x775c440 con 0x7cae040 -5533> 2014-08-13 16:24:24.463562 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.463253) v2 -- ?+0 0x9283b80 con 0x7cae040 -5532> 2014-08-13 16:24:24.521580 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.521343) v2 ==== 47+0+0 (4035677876 0 0) 0x92840c0 con 0x7ed8580 -5531> 2014-08-13 16:24:24.521633 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.521343) v2 -- ?+0 0x775c440 con 0x7ed8580 -5530> 2014-08-13 16:24:24.521636 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.521343) v2 ==== 47+0+0 (4035677876 0 0) 0x775abc0 con 0x7ebe5c0 -5529> 2014-08-13 16:24:24.521690 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.521343) v2 -- ?+0 0x8577e00 con 0x7ebe5c0 -5528> 2014-08-13 16:24:24.546299 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 27 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3645524929 0 0) 0x86dee00 con 0x5112520 -5527> 2014-08-13 16:24:24.546354 7fd93dff0700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:24.546091, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5526> 2014-08-13 16:24:24.546370 7fd93dff0700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:24.546095, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5525> 2014-08-13 16:24:24.546380 7fd93dff0700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:24.546218, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5524> 2014-08-13 16:24:24.546395 7fd93dff0700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:24.546347, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5523> 2014-08-13 16:24:24.546411 7fd93dff0700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:24.546411, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5522> 2014-08-13 16:24:24.549433 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.549050) v2 ==== 47+0+0 (2574629592 0 0) 0x775d940 con 0x7eb9b80 -5521> 2014-08-13 16:24:24.549485 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.549050) v2 -- ?+0 0x92840c0 con 0x7eb9b80 -5520> 2014-08-13 16:24:24.549478 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 7 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.549050) v2 ==== 47+0+0 (2574629592 0 0) 0x775b100 con 0x7ebc620 -5519> 2014-08-13 16:24:24.549524 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.549050) v2 -- ?+0 0x775abc0 con 0x7ebc620 -5518> 2014-08-13 16:24:24.608601 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.608413) v2 ==== 47+0+0 (4205802200 0 0) 0x9285780 con 0x7edc360 -5517> 2014-08-13 16:24:24.608617 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.608413) v2 ==== 47+0+0 (4205802200 0 0) 0x92832c0 con 0x7eda3c0 -5516> 2014-08-13 16:24:24.608654 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.608413) v2 -- ?+0 0x775d940 con 0x7edc360 -5515> 2014-08-13 16:24:24.608755 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.608413) v2 -- ?+0 0x775b100 con 0x7eda3c0 -5514> 2014-08-13 16:24:24.635265 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 17 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3946485199 0 0) 0x8bc6e00 con 0x4e4f900 -5513> 2014-08-13 16:24:24.635317 7fd93dff0700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:24.635060, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5512> 2014-08-13 16:24:24.635333 7fd93dff0700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:24.635063, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5511> 2014-08-13 16:24:24.635343 7fd93dff0700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:24.635184, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5510> 2014-08-13 16:24:24.635353 7fd93dff0700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:24.635311, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5509> 2014-08-13 16:24:24.635364 7fd93dff0700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:24.635363, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5508> 2014-08-13 16:24:24.640296 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.30 192.168.100.107:0/55491 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.640106) v2 ==== 47+0+0 (1224622006 0 0) 0x9280700 con 0x7eda100 -5507> 2014-08-13 16:24:24.640348 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.640106) v2 -- ?+0 0x92832c0 con 0x7eda100 -5506> 2014-08-13 16:24:24.640321 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.30 192.168.100.107:0/55491 6 ==== osd_ping(ping e194449 stamp 2014-08-13 16:24:24.640106) v2 ==== 47+0+0 (1224622006 0 0) 0x9280c40 con 0x7edc0a0 -5505> 2014-08-13 16:24:24.640458 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.640106) v2 -- ?+0 0x9285780 con 0x7edc0a0 -5504> 2014-08-13 16:24:24.665141 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 24 ==== pg_stats_ack(1 pgs tid 2) v1 ==== 33+0+0 (3016011495 0 0) 0x8db0540 con 0x4e3dd80 -5503> 2014-08-13 16:24:24.682752 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 10 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf2640 con 0x7315ee0 -5502> 2014-08-13 16:24:24.682818 7fd93dff0700 3 osd.4 194449 handle_osd_map epochs [194450,194450], i have 194449, src has [185139,194450] -5501> 2014-08-13 16:24:24.688229 7fd93dff0700 1 -- 192.168.1.115:6813/25220 mark_down 192.168.1.110:6804/25590 -- 0x8e88a00 -5500> 2014-08-13 16:24:24.688397 7fd93dff0700 0 osd.4 194450 crush map has features 2578087936000, adjusting msgr requires for mons -5499> 2014-08-13 16:24:24.830012 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.829568) v2 ==== 47+0+0 (102968958 0 0) 0x7759a40 con 0x7d29ce0 -5498> 2014-08-13 16:24:24.830029 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.829568) v2 ==== 47+0+0 (102968958 0 0) 0x9286e40 con 0x7d2b440 -5497> 2014-08-13 16:24:24.830081 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.829568) v2 -- ?+0 0x9280c40 con 0x7d29ce0 -5496> 2014-08-13 16:24:24.830225 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.829568) v2 -- ?+0 0x9280700 con 0x7d2b440 -5495> 2014-08-13 16:24:24.833793 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.833290) v2 ==== 47+0+0 (3615463595 0 0) 0x85778c0 con 0x7ebbde0 -5494> 2014-08-13 16:24:24.833837 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.833290) v2 -- ?+0 0x9286e40 con 0x7ebbde0 -5493> 2014-08-13 16:24:24.833754 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.833290) v2 ==== 47+0+0 (3615463595 0 0) 0x9280540 con 0x7ebd3e0 -5492> 2014-08-13 16:24:24.833889 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.833290) v2 -- ?+0 0x7759a40 con 0x7ebd3e0 -5491> 2014-08-13 16:24:24.908302 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 5 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.907915) v2 ==== 47+0+0 (146035687 0 0) 0x840fe00 con 0x7edc8e0 -5490> 2014-08-13 16:24:24.908356 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.907915) v2 -- ?+0 0x9280540 con 0x7edc8e0 -5489> 2014-08-13 16:24:24.908355 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 5 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.907915) v2 ==== 47+0+0 (146035687 0 0) 0x8408380 con 0x7eda940 -5488> 2014-08-13 16:24:24.908407 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.907915) v2 -- ?+0 0x85778c0 con 0x7eda940 -5487> 2014-08-13 16:24:24.931664 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.931618) v2 ==== 47+0+0 (925007022 0 0) 0x92801c0 con 0x7ffeca0 -5486> 2014-08-13 16:24:24.931717 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.931618) v2 -- ?+0 0x8408380 con 0x7ffeca0 -5485> 2014-08-13 16:24:24.931792 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:24.931618) v2 ==== 47+0+0 (925007022 0 0) 0x9286200 con 0x7edbf40 -5484> 2014-08-13 16:24:24.931865 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:24.931618) v2 -- ?+0 0x840fe00 con 0x7edbf40 -5483> 2014-08-13 16:24:25.003668 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.003626) v2 ==== 47+0+0 (3773656244 0 0) 0x87e7540 con 0x7edaec0 -5482> 2014-08-13 16:24:25.003722 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:25.003626) v2 -- ?+0 0x92801c0 con 0x7edaec0 -5481> 2014-08-13 16:24:25.003721 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.003626) v2 ==== 47+0+0 (3773656244 0 0) 0x87e6900 con 0x7ed8000 -5480> 2014-08-13 16:24:25.003793 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:25.003626) v2 -- ?+0 0x9286200 con 0x7ed8000 -5479> 2014-08-13 16:24:25.146015 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.146042) v2 ==== 47+0+0 (1577523071 0 0) 0x9117000 con 0x7d29fa0 -5478> 2014-08-13 16:24:25.146073 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:25.146042) v2 -- ?+0 0x87e6900 con 0x7d29fa0 -5477> 2014-08-13 16:24:25.146146 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.146042) v2 ==== 47+0+0 (1577523071 0 0) 0x9117e00 con 0x7d2b700 -5476> 2014-08-13 16:24:25.146207 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194449 stamp 2014-08-13 16:24:25.146042) v2 -- ?+0 0x87e7540 con 0x7d2b700 -5475> 2014-08-13 16:24:25.187139 7fd9371e5700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:25.187138, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5474> 2014-08-13 16:24:25.187199 7fd9371e5700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:25.187198, event: started, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5473> 2014-08-13 16:24:25.187274 7fd9371e5700 5 -- op tracker -- , seq: 830, time: 2014-08-13 16:24:25.187274, event: done, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5472> 2014-08-13 16:24:25.187346 7fd9371e5700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:25.187346, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5471> 2014-08-13 16:24:25.187367 7fd9371e5700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:25.187367, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5470> 2014-08-13 16:24:25.187390 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8da00 con 0x5112520 -5469> 2014-08-13 16:24:25.187406 7fd9367e4700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:25.187367, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5468> 2014-08-13 16:24:25.187455 7fd9367e4700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:25.187454, event: started, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5467> 2014-08-13 16:24:25.187471 7fd9371e5700 5 -- op tracker -- , seq: 832, time: 2014-08-13 16:24:25.187428, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5466> 2014-08-13 16:24:25.187519 7fd9371e5700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:25.187519, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5465> 2014-08-13 16:24:25.187542 7fd9367e4700 5 -- op tracker -- , seq: 831, time: 2014-08-13 16:24:25.187529, event: done, request: osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5464> 2014-08-13 16:24:25.187580 7fd9371e5700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:25.187543, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5463> 2014-08-13 16:24:25.187614 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x7bc5f00 con 0x4e4f900 -5462> 2014-08-13 16:24:25.187598 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub osd.4 has 9 items -5461> 2014-08-13 16:24:25.187641 7fd9371e5700 5 -- op tracker -- , seq: 833, time: 2014-08-13 16:24:25.187641, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5460> 2014-08-13 16:24:25.187643 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 340 has 9 items -5459> 2014-08-13 16:24:25.187670 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 347 has 9 items -5458> 2014-08-13 16:24:25.188018 7fd9353e2700 2 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194448 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] -5457> 2014-08-13 16:24:25.188134 7fd9353e2700 0 log [INF] : 22.1b8 deep-scrub ok -5456> 2014-08-13 16:24:25.188304 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6806/62463 -- osd_map(194450..194450 src has 185139..194450) v3 -- ?+0 0x8d0a640 con 0x4fe69e0 -5455> 2014-08-13 16:24:25.188339 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x76fee00 -5454> 2014-08-13 16:24:25.188558 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.106:6800/4640 -- osd_map(194450..194450 src has 185139..194450) v3 -- ?+0 0x8d0a1c0 con 0x50bbf40 -5453> 2014-08-13 16:24:25.188605 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- osd_sub_op(unknown.0.0:0 22.1b8 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x76fe300 -5452> 2014-08-13 16:24:25.188763 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x79b8380 -5451> 2014-08-13 16:24:25.188843 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194449:22.1b8) v4 -- ?+0 0x9110380 -5450> 2014-08-13 16:24:25.189376 7fd93dff0700 1 osd.4 194450 ms_handle_reset con 0x78b0580 session 0 -5449> 2014-08-13 16:24:25.189397 7fd94ac49700 5 osd.4 194450 tick -5448> 2014-08-13 16:24:25.189394 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 11 ==== pg_notify(24.39a(27) epoch 194450) v5 ==== 2245+0+0 (2134286940 0 0) 0x81c55c0 con 0x7315ee0 -5447> 2014-08-13 16:24:25.189526 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6842/9607 -- osd_map(194450..194450 src has 185139..194450) v3 -- ?+0 0x7cf4ec0 con 0x508c620 -5446> 2014-08-13 16:24:25.189562 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x76f8000 -5445> 2014-08-13 16:24:25.189609 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6808/15284 -- osd_map(194450..194450 src has 185139..194450) v3 -- ?+0 0x7cf69c0 con 0x4fdb2e0 -5444> 2014-08-13 16:24:25.189633 7fd94ac49700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x86de300 -5443> 2014-08-13 16:24:25.189818 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:24.682697, event: header_read, request: pg_notify(24.39a(27) epoch 194450) v5 -5442> 2014-08-13 16:24:25.189849 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:24.682698, event: throttled, request: pg_notify(24.39a(27) epoch 194450) v5 -5441> 2014-08-13 16:24:25.189856 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:24.682831, event: all_read, request: pg_notify(24.39a(27) epoch 194450) v5 -5440> 2014-08-13 16:24:25.189862 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:25.189416, event: dispatched, request: pg_notify(24.39a(27) epoch 194450) v5 -5439> 2014-08-13 16:24:25.189870 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:25.189870, event: waiting_for_osdmap, request: pg_notify(24.39a(27) epoch 194450) v5 -5438> 2014-08-13 16:24:25.189880 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:25.189880, event: started, request: pg_notify(24.39a(27) epoch 194450) v5 -5437> 2014-08-13 16:24:25.189910 7fd93dff0700 5 -- op tracker -- , seq: 834, time: 2014-08-13 16:24:25.189910, event: done, request: pg_notify(24.39a(27) epoch 194450) v5 -5436> 2014-08-13 16:24:25.189945 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 7 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf5100 con 0x7314fc0 -5435> 2014-08-13 16:24:25.189977 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5434> 2014-08-13 16:24:25.189989 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x506e0c0 con 0x7316f60 -5433> 2014-08-13 16:24:25.190005 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5432> 2014-08-13 16:24:25.189971 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194447/194448/194445) [325,349,4]/[325,4,395] r=1 lpr=194448 pi=193304-194447/50 luod=0'0 crt=194422'13151 lcod 0'0 active+remapped m=49] exit Started/ReplicaActive/RepNotRecovering 19.699923 4 0.002320 -5431> 2014-08-13 16:24:25.190014 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b145c0 con 0x8da1760 -5430> 2014-08-13 16:24:25.190033 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5429> 2014-08-13 16:24:25.190017 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194447/194448/194445) [325,349,4]/[325,4,395] r=1 lpr=194448 pi=193304-194447/50 luod=0'0 crt=194422'13151 lcod 0'0 active+remapped m=49] exit Started/ReplicaActive 19.699996 0 0.000000 -5428> 2014-08-13 16:24:25.190049 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf6540 con 0x8da5120 -5427> 2014-08-13 16:24:25.190066 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5426> 2014-08-13 16:24:25.190054 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194447/194448/194445) [325,349,4]/[325,4,395] r=1 lpr=194448 pi=193304-194447/50 luod=0'0 crt=194422'13151 lcod 0'0 active+remapped m=49] exit Started 20.784828 0 0.000000 -5425> 2014-08-13 16:24:25.190074 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 7 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7cc80 con 0x72d9b80 -5424> 2014-08-13 16:24:25.190078 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194447/194448/194445) [325,349,4]/[325,4,395] r=1 lpr=194448 pi=193304-194447/50 luod=0'0 crt=194422'13151 lcod 0'0 active+remapped m=49] enter Reset -5423> 2014-08-13 16:24:25.190091 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5422> 2014-08-13 16:24:25.190100 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7d340 con 0x72dfa60 -5421> 2014-08-13 16:24:25.190118 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5420> 2014-08-13 16:24:25.190126 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7ee40 con 0x72de9e0 -5419> 2014-08-13 16:24:25.190143 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5418> 2014-08-13 16:24:25.190164 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Reset 0.000086 1 0.000192 -5417> 2014-08-13 16:24:25.190191 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8598fc0 con 0x8e29fa0 -5416> 2014-08-13 16:24:25.190208 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5415> 2014-08-13 16:24:25.190203 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started -5414> 2014-08-13 16:24:25.190217 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 9 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7d7c0 con 0x8e286e0 -5413> 2014-08-13 16:24:25.190231 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5412> 2014-08-13 16:24:25.190227 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Start -5411> 2014-08-13 16:24:25.190244 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b14a40 con 0x8e2c360 -5410> 2014-08-13 16:24:25.190248 7fd9367e4700 1 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] state: transitioning to Primary -5409> 2014-08-13 16:24:25.190276 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Start 0.000050 0 0.000000 -5408> 2014-08-13 16:24:25.190301 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary -5407> 2014-08-13 16:24:25.190323 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary/Peering -5406> 2014-08-13 16:24:25.190261 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5405> 2014-08-13 16:24:25.190346 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/GetInfo -5404> 2014-08-13 16:24:25.190356 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7b3c0 con 0x7379ce0 -5403> 2014-08-13 16:24:25.190382 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5402> 2014-08-13 16:24:25.190396 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7f2c0 con 0x73ca680 -5401> 2014-08-13 16:24:25.190415 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5400> 2014-08-13 16:24:25.190387 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.190276) v2 ==== 47+0+0 (1063272186 0 0) 0x8910c40 con 0x7eba680 -5399> 2014-08-13 16:24:25.190426 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 12 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd5e80 con 0x73cdc20 -5398> 2014-08-13 16:24:25.190440 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5397> 2014-08-13 16:24:25.190430 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.190276) v2 -- ?+0 0x9117000 con 0x7eba680 -5396> 2014-08-13 16:24:25.190449 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 3 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8599440 con 0x73ccba0 -5395> 2014-08-13 16:24:25.190462 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5394> 2014-08-13 16:24:25.190471 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 3 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b169c0 con 0x7462100 -5393> 2014-08-13 16:24:25.190484 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5392> 2014-08-13 16:24:25.190492 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 8 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf60c0 con 0x7723440 -5391> 2014-08-13 16:24:25.190505 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5390> 2014-08-13 16:24:25.190513 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 1 ==== osd_map(194351..194450 src has 185139..194450) v3 ==== 85393+0+0 (127312512 0 0) 0x7cf0b40 con 0x7d2d120 -5389> 2014-08-13 16:24:25.190527 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194351,194450], i have 194450, src has [185139,194450] -5388> 2014-08-13 16:24:25.190578 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7e300 con 0x7d2d280 -5387> 2014-08-13 16:24:25.190597 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5386> 2014-08-13 16:24:25.190577 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.190276) v2 ==== 47+0+0 (1063272186 0 0) 0x92863c0 con 0x7eb91e0 -5385> 2014-08-13 16:24:25.190606 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf0d80 con 0x821f900 -5384> 2014-08-13 16:24:25.190608 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.190276) v2 -- ?+0 0x9117e00 con 0x7eb91e0 -5383> 2014-08-13 16:24:25.190620 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5382> 2014-08-13 16:24:25.190628 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 3 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x33e3cc0 con 0x80c7e80 -5381> 2014-08-13 16:24:25.190642 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5380> 2014-08-13 16:24:25.190626 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194447/194447/194444) [325,4] r=1 lpr=194447 pi=193125-194446/41 luod=0'0 crt=193305'14233 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 20.751386 7 0.000207 -5379> 2014-08-13 16:24:25.190650 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 5 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf0000 con 0x80c7d20 -5378> 2014-08-13 16:24:25.190664 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5377> 2014-08-13 16:24:25.190657 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194447/194447/194444) [325,4] r=1 lpr=194447 pi=193125-194446/41 luod=0'0 crt=193305'14233 lcod 0'0 active] exit Started/ReplicaActive 20.751445 0 0.000000 -5376> 2014-08-13 16:24:25.190673 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 13 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b16780 con 0x4e4b860 -5375> 2014-08-13 16:24:25.190687 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5374> 2014-08-13 16:24:25.190678 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194447/194447/194444) [325,4] r=1 lpr=194447 pi=193125-194446/41 luod=0'0 crt=193305'14233 lcod 0'0 active] exit Started 21.577301 0 0.000000 -5373> 2014-08-13 16:24:25.190699 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 17 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859bcc0 con 0x4e4dd80 -5372> 2014-08-13 16:24:25.190697 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194447/194447/194444) [325,4] r=1 lpr=194447 pi=193125-194446/41 luod=0'0 crt=193305'14233 lcod 0'0 active] enter Reset -5371> 2014-08-13 16:24:25.190714 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5370> 2014-08-13 16:24:25.190723 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 11 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd0b40 con 0x4e4ce60 -5369> 2014-08-13 16:24:25.190741 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5368> 2014-08-13 16:24:25.190750 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 18 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd3180 con 0x4e4f900 -5367> 2014-08-13 16:24:25.190759 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000061 1 0.000131 -5366> 2014-08-13 16:24:25.190764 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5365> 2014-08-13 16:24:25.190784 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started -5364> 2014-08-13 16:24:25.190788 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 22 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859b180 con 0x4fd8000 -5363> 2014-08-13 16:24:25.190810 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Start -5362> 2014-08-13 16:24:25.190825 7fd9367e4700 1 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5361> 2014-08-13 16:24:25.190834 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5360> 2014-08-13 16:24:25.190847 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -5359> 2014-08-13 16:24:25.190868 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5358> 2014-08-13 16:24:25.190863 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 15 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd4ec0 con 0x4fe1600 -5357> 2014-08-13 16:24:25.190899 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5356> 2014-08-13 16:24:25.190924 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 14 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7b840 con 0x4fe40a0 -5355> 2014-08-13 16:24:25.190947 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5354> 2014-08-13 16:24:25.190957 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 18 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf0fc0 con 0x4fe69e0 -5353> 2014-08-13 16:24:25.190971 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5352> 2014-08-13 16:24:25.190980 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 17 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b15a00 con 0x4fda940 -5351> 2014-08-13 16:24:25.190998 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5350> 2014-08-13 16:24:25.190892 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5349> 2014-08-13 16:24:25.191012 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 1 ==== osd_map(194351..194450 src has 185139..194450) v3 ==== 85393+0+0 (127312512 0 0) 0x7cf18c0 con 0x4fd9b80 -5348> 2014-08-13 16:24:25.191032 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194351,194450], i have 194450, src has [185139,194450] -5347> 2014-08-13 16:24:25.191020 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5346> 2014-08-13 16:24:25.191061 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000041 0 0.000000 -5345> 2014-08-13 16:24:25.191074 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 18 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf69c0 con 0x4fdb2e0 -5344> 2014-08-13 16:24:25.191107 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5343> 2014-08-13 16:24:25.191099 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5342> 2014-08-13 16:24:25.191124 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.31 192.168.1.103:6831/41157 14 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf1680 con 0x5089760 -5341> 2014-08-13 16:24:25.191140 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5340> 2014-08-13 16:24:25.191149 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 17 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd2d00 con 0x50889a0 -5339> 2014-08-13 16:24:25.191168 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5338> 2014-08-13 16:24:25.191163 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000065 0 0.000000 -5337> 2014-08-13 16:24:25.191199 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 13 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd3a80 con 0x508a680 -5336> 2014-08-13 16:24:25.191200 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/Incomplete -5335> 2014-08-13 16:24:25.191221 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5334> 2014-08-13 16:24:25.191230 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.50 192.168.1.112:6842/9607 14 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf4ec0 con 0x508c620 -5333> 2014-08-13 16:24:25.191248 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5332> 2014-08-13 16:24:25.191256 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 27 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd1440 con 0x508bde0 -5331> 2014-08-13 16:24:25.191274 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5330> 2014-08-13 16:24:25.191286 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 20 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf5a00 con 0x50b9a20 -5329> 2014-08-13 16:24:25.191303 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5328> 2014-08-13 16:24:25.191316 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 13 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b17980 con 0x50ba520 -5327> 2014-08-13 16:24:25.191334 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5326> 2014-08-13 16:24:25.191346 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 14 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859f740 con 0x50bc4c0 -5325> 2014-08-13 16:24:25.191364 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5324> 2014-08-13 16:24:25.191375 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.347 192.168.1.106:6800/4640 10 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8d0a1c0 con 0x50bbf40 -5323> 2014-08-13 16:24:25.191392 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5322> 2014-08-13 16:24:25.191403 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7cf6780 con 0x50bfa60 -5321> 2014-08-13 16:24:25.191420 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5320> 2014-08-13 16:24:25.191431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 14 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b11b00 con 0x510b9c0 -5319> 2014-08-13 16:24:25.191448 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5318> 2014-08-13 16:24:25.191460 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 12 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e7c5c0 con 0x5108b00 -5317> 2014-08-13 16:24:25.191487 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5316> 2014-08-13 16:24:25.191501 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b14380 con 0x510dac0 -5315> 2014-08-13 16:24:25.191519 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5314> 2014-08-13 16:24:25.191534 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 1 ==== osd_map(194351..194450 src has 185139..194450) v3 ==== 85393+0+0 (127312512 0 0) 0x859dc40 con 0x51127e0 -5313> 2014-08-13 16:24:25.191551 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194351,194450], i have 194450, src has [185139,194450] -5312> 2014-08-13 16:24:25.191605 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 28 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859e300 con 0x5112520 -5311> 2014-08-13 16:24:25.191624 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5310> 2014-08-13 16:24:25.191637 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 11 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b16c00 con 0x5112260 -5309> 2014-08-13 16:24:25.191654 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5308> 2014-08-13 16:24:25.191666 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 9 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e79440 con 0x5114620 -5307> 2014-08-13 16:24:25.191683 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5306> 2014-08-13 16:24:25.191694 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 16 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e79b00 con 0x5114360 -5305> 2014-08-13 16:24:25.191711 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5304> 2014-08-13 16:24:25.191719 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 6 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b172c0 con 0x5113de0 -5303> 2014-08-13 16:24:25.191736 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5302> 2014-08-13 16:24:25.191748 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 16 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859ad00 con 0x5113b20 -5301> 2014-08-13 16:24:25.191765 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5300> 2014-08-13 16:24:25.191776 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 9 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7e78fc0 con 0x5113860 -5299> 2014-08-13 16:24:25.191793 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5298> 2014-08-13 16:24:25.191804 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 23 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b14800 con 0x5111fa0 -5297> 2014-08-13 16:24:25.191821 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5296> 2014-08-13 16:24:25.191833 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 27 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x8bd33c0 con 0x51135a0 -5295> 2014-08-13 16:24:25.191851 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5294> 2014-08-13 16:24:25.191865 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 19 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x859d100 con 0x5111a20 -5293> 2014-08-13 16:24:25.191883 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5292> 2014-08-13 16:24:25.191894 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 12 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b14c80 con 0x7313860 -5291> 2014-08-13 16:24:25.191911 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5290> 2014-08-13 16:24:25.191922 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 7 ==== osd_map(194450..194450 src has 185139..194450) v3 ==== 211+0+0 (2479179274 0 0) 0x7b15340 con 0x7316720 -5289> 2014-08-13 16:24:25.191940 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194450,194450], i have 194450, src has [185139,194450] -5288> 2014-08-13 16:24:25.191952 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 8 ==== pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 ==== 2595+0+0 (2320701836 0 0) 0x8c8d5c0 con 0x7314fc0 -5287> 2014-08-13 16:24:25.191972 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:24.718509, event: header_read, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5286> 2014-08-13 16:24:25.191987 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:24.718511, event: throttled, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5285> 2014-08-13 16:24:25.191998 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:24.718662, event: all_read, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5284> 2014-08-13 16:24:25.192008 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:25.191969, event: dispatched, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5283> 2014-08-13 16:24:25.192019 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:25.192019, event: waiting_for_osdmap, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5282> 2014-08-13 16:24:25.192033 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:25.192033, event: started, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5281> 2014-08-13 16:24:25.192077 7fd93dff0700 5 -- op tracker -- , seq: 835, time: 2014-08-13 16:24:25.192077, event: done, request: pg_notify(22.113(4),22.3e4(17) epoch 194450) v5 -5280> 2014-08-13 16:24:25.192108 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 7 ==== pg_notify(22.22c(4) epoch 194450) v5 ==== 942+0+0 (3820857498 0 0) 0x9105400 con 0x7316f60 -5279> 2014-08-13 16:24:25.192128 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:24.715017, event: header_read, request: pg_notify(22.22c(4) epoch 194450) v5 -5278> 2014-08-13 16:24:25.192139 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:24.715019, event: throttled, request: pg_notify(22.22c(4) epoch 194450) v5 -5277> 2014-08-13 16:24:25.192149 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:24.715087, event: all_read, request: pg_notify(22.22c(4) epoch 194450) v5 -5276> 2014-08-13 16:24:25.192158 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:25.192126, event: dispatched, request: pg_notify(22.22c(4) epoch 194450) v5 -5275> 2014-08-13 16:24:25.192168 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:25.192168, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194450) v5 -5274> 2014-08-13 16:24:25.192176 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:25.192176, event: started, request: pg_notify(22.22c(4) epoch 194450) v5 -5273> 2014-08-13 16:24:25.192212 7fd93dff0700 5 -- op tracker -- , seq: 836, time: 2014-08-13 16:24:25.192212, event: done, request: pg_notify(22.22c(4) epoch 194450) v5 -5272> 2014-08-13 16:24:25.192269 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 6 ==== pg_notify(22.619(17) epoch 194450) v5 ==== 1665+0+0 (2086399844 0 0) 0x80d9a40 con 0x8da1760 -5271> 2014-08-13 16:24:25.192291 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:24.734926, event: header_read, request: pg_notify(22.619(17) epoch 194450) v5 -5270> 2014-08-13 16:24:25.192302 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:24.734927, event: throttled, request: pg_notify(22.619(17) epoch 194450) v5 -5269> 2014-08-13 16:24:25.192318 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:24.735030, event: all_read, request: pg_notify(22.619(17) epoch 194450) v5 -5268> 2014-08-13 16:24:25.192328 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:25.192288, event: dispatched, request: pg_notify(22.619(17) epoch 194450) v5 -5267> 2014-08-13 16:24:25.192338 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:25.192337, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194450) v5 -5266> 2014-08-13 16:24:25.192349 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:25.192349, event: started, request: pg_notify(22.619(17) epoch 194450) v5 -5265> 2014-08-13 16:24:25.192380 7fd93dff0700 5 -- op tracker -- , seq: 837, time: 2014-08-13 16:24:25.192379, event: done, request: pg_notify(22.619(17) epoch 194450) v5 -5264> 2014-08-13 16:24:25.192424 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 7 ==== pg_notify(22.f3(15) epoch 194450) v5 ==== 1551+0+0 (273510896 0 0) 0x9280540 con 0x8da5120 -5263> 2014-08-13 16:24:25.192445 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:24.919087, event: header_read, request: pg_notify(22.f3(15) epoch 194450) v5 -5262> 2014-08-13 16:24:25.192456 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:24.919088, event: throttled, request: pg_notify(22.f3(15) epoch 194450) v5 -5261> 2014-08-13 16:24:25.192462 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:24.919190, event: all_read, request: pg_notify(22.f3(15) epoch 194450) v5 -5260> 2014-08-13 16:24:25.192470 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:25.192442, event: dispatched, request: pg_notify(22.f3(15) epoch 194450) v5 -5259> 2014-08-13 16:24:25.192480 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:25.192480, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194450) v5 -5258> 2014-08-13 16:24:25.192491 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:25.192491, event: started, request: pg_notify(22.f3(15) epoch 194450) v5 -5257> 2014-08-13 16:24:25.192515 7fd93dff0700 5 -- op tracker -- , seq: 838, time: 2014-08-13 16:24:25.192515, event: done, request: pg_notify(22.f3(15) epoch 194450) v5 -5256> 2014-08-13 16:24:25.192540 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 8 ==== pg_notify(22.99(4) epoch 194450) v5 ==== 942+0+0 (2531270626 0 0) 0x8c8de80 con 0x72d9b80 -5255> 2014-08-13 16:24:25.192559 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:24.729735, event: header_read, request: pg_notify(22.99(4) epoch 194450) v5 -5254> 2014-08-13 16:24:25.192570 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:24.729736, event: throttled, request: pg_notify(22.99(4) epoch 194450) v5 -5253> 2014-08-13 16:24:25.192579 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:24.729810, event: all_read, request: pg_notify(22.99(4) epoch 194450) v5 -5252> 2014-08-13 16:24:25.192588 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:25.192557, event: dispatched, request: pg_notify(22.99(4) epoch 194450) v5 -5251> 2014-08-13 16:24:25.192597 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:25.192597, event: waiting_for_osdmap, request: pg_notify(22.99(4) epoch 194450) v5 -5250> 2014-08-13 16:24:25.192608 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:25.192608, event: started, request: pg_notify(22.99(4) epoch 194450) v5 -5249> 2014-08-13 16:24:25.192631 7fd93dff0700 5 -- op tracker -- , seq: 839, time: 2014-08-13 16:24:25.192630, event: done, request: pg_notify(22.99(4) epoch 194450) v5 -5248> 2014-08-13 16:24:25.192654 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 7 ==== pg_notify(22.4df(4) epoch 194450) v5 ==== 942+0+0 (4239931283 0 0) 0x87e39c0 con 0x72dfa60 -5247> 2014-08-13 16:24:25.192674 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:24.693087, event: header_read, request: pg_notify(22.4df(4) epoch 194450) v5 -5246> 2014-08-13 16:24:25.192687 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:24.693088, event: throttled, request: pg_notify(22.4df(4) epoch 194450) v5 -5245> 2014-08-13 16:24:25.192696 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:24.693194, event: all_read, request: pg_notify(22.4df(4) epoch 194450) v5 -5244> 2014-08-13 16:24:25.192705 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:25.192672, event: dispatched, request: pg_notify(22.4df(4) epoch 194450) v5 -5243> 2014-08-13 16:24:25.192715 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:25.192715, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194450) v5 -5242> 2014-08-13 16:24:25.192726 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:25.192726, event: started, request: pg_notify(22.4df(4) epoch 194450) v5 -5241> 2014-08-13 16:24:25.192749 7fd93dff0700 5 -- op tracker -- , seq: 840, time: 2014-08-13 16:24:25.192748, event: done, request: pg_notify(22.4df(4) epoch 194450) v5 -5240> 2014-08-13 16:24:25.192770 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 6 ==== pg_notify(22.39d(4) epoch 194450) v5 ==== 942+0+0 (2190464647 0 0) 0x8c8b9c0 con 0x72de9e0 -5239> 2014-08-13 16:24:25.192789 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:24.855019, event: header_read, request: pg_notify(22.39d(4) epoch 194450) v5 -5238> 2014-08-13 16:24:25.192800 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:24.855021, event: throttled, request: pg_notify(22.39d(4) epoch 194450) v5 -5237> 2014-08-13 16:24:25.192809 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:24.855092, event: all_read, request: pg_notify(22.39d(4) epoch 194450) v5 -5236> 2014-08-13 16:24:25.192818 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:25.192787, event: dispatched, request: pg_notify(22.39d(4) epoch 194450) v5 -5235> 2014-08-13 16:24:25.192824 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:25.192824, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194450) v5 -5234> 2014-08-13 16:24:25.192835 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:25.192835, event: started, request: pg_notify(22.39d(4) epoch 194450) v5 -5233> 2014-08-13 16:24:25.192857 7fd93dff0700 5 -- op tracker -- , seq: 841, time: 2014-08-13 16:24:25.192857, event: done, request: pg_notify(22.39d(4) epoch 194450) v5 -5232> 2014-08-13 16:24:25.192877 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 7 ==== pg_notify(22.775(17) epoch 194450) v5 ==== 1665+0+0 (1032054793 0 0) 0x9106200 con 0x8e29fa0 -5231> 2014-08-13 16:24:25.192896 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:24.842368, event: header_read, request: pg_notify(22.775(17) epoch 194450) v5 -5230> 2014-08-13 16:24:25.192907 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:24.842370, event: throttled, request: pg_notify(22.775(17) epoch 194450) v5 -5229> 2014-08-13 16:24:25.192916 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:24.842464, event: all_read, request: pg_notify(22.775(17) epoch 194450) v5 -5228> 2014-08-13 16:24:25.192925 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:25.192894, event: dispatched, request: pg_notify(22.775(17) epoch 194450) v5 -5227> 2014-08-13 16:24:25.192935 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:25.192935, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194450) v5 -5226> 2014-08-13 16:24:25.192946 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:25.192946, event: started, request: pg_notify(22.775(17) epoch 194450) v5 -5225> 2014-08-13 16:24:25.193421 7fd93dff0700 5 -- op tracker -- , seq: 842, time: 2014-08-13 16:24:25.193421, event: done, request: pg_notify(22.775(17) epoch 194450) v5 -5224> 2014-08-13 16:24:25.193460 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 10 ==== pg_notify(22.3af(17) epoch 194450) v5 ==== 1665+0+0 (2560541676 0 0) 0x91063c0 con 0x8e286e0 -5223> 2014-08-13 16:24:25.193482 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:24.717198, event: header_read, request: pg_notify(22.3af(17) epoch 194450) v5 -5222> 2014-08-13 16:24:25.193492 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:24.717199, event: throttled, request: pg_notify(22.3af(17) epoch 194450) v5 -5221> 2014-08-13 16:24:25.193498 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:24.717278, event: all_read, request: pg_notify(22.3af(17) epoch 194450) v5 -5220> 2014-08-13 16:24:25.193507 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:25.193479, event: dispatched, request: pg_notify(22.3af(17) epoch 194450) v5 -5219> 2014-08-13 16:24:25.193517 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:25.193516, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194450) v5 -5218> 2014-08-13 16:24:25.193528 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:25.193528, event: started, request: pg_notify(22.3af(17) epoch 194450) v5 -5217> 2014-08-13 16:24:25.193552 7fd93dff0700 5 -- op tracker -- , seq: 843, time: 2014-08-13 16:24:25.193551, event: done, request: pg_notify(22.3af(17) epoch 194450) v5 -5216> 2014-08-13 16:24:25.193580 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 6 ==== pg_notify(22.4df(15) epoch 194450) v5 ==== 1551+0+0 (1013148116 0 0) 0x8c88000 con 0x8e2c360 -5215> 2014-08-13 16:24:25.193600 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:24.845280, event: header_read, request: pg_notify(22.4df(15) epoch 194450) v5 -5214> 2014-08-13 16:24:25.193611 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:24.845281, event: throttled, request: pg_notify(22.4df(15) epoch 194450) v5 -5213> 2014-08-13 16:24:25.193620 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:24.845384, event: all_read, request: pg_notify(22.4df(15) epoch 194450) v5 -5212> 2014-08-13 16:24:25.193629 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:25.193598, event: dispatched, request: pg_notify(22.4df(15) epoch 194450) v5 -5211> 2014-08-13 16:24:25.193638 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:25.193638, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194450) v5 -5210> 2014-08-13 16:24:25.193649 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:25.193649, event: started, request: pg_notify(22.4df(15) epoch 194450) v5 -5209> 2014-08-13 16:24:25.193677 7fd93dff0700 5 -- op tracker -- , seq: 844, time: 2014-08-13 16:24:25.193676, event: done, request: pg_notify(22.4df(15) epoch 194450) v5 -5208> 2014-08-13 16:24:25.193701 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 7 ==== pg_notify(24.181(25) epoch 194450) v5 ==== 2159+0+0 (2285503668 0 0) 0x9117540 con 0x7379ce0 -5207> 2014-08-13 16:24:25.193720 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:24.924831, event: header_read, request: pg_notify(24.181(25) epoch 194450) v5 -5206> 2014-08-13 16:24:25.193730 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:24.924832, event: throttled, request: pg_notify(24.181(25) epoch 194450) v5 -5205> 2014-08-13 16:24:25.193740 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:24.924938, event: all_read, request: pg_notify(24.181(25) epoch 194450) v5 -5204> 2014-08-13 16:24:25.193749 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:25.193718, event: dispatched, request: pg_notify(24.181(25) epoch 194450) v5 -5203> 2014-08-13 16:24:25.193758 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:25.193758, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194450) v5 -5202> 2014-08-13 16:24:25.193769 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:25.193769, event: started, request: pg_notify(24.181(25) epoch 194450) v5 -5201> 2014-08-13 16:24:25.193792 7fd93dff0700 5 -- op tracker -- , seq: 845, time: 2014-08-13 16:24:25.193792, event: done, request: pg_notify(24.181(25) epoch 194450) v5 -5200> 2014-08-13 16:24:25.193814 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 7 ==== pg_notify(22.1f1(4) epoch 194450) v5 ==== 942+0+0 (3856736104 0 0) 0x87e7c40 con 0x73ca680 -5199> 2014-08-13 16:24:25.193833 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:24.699831, event: header_read, request: pg_notify(22.1f1(4) epoch 194450) v5 -5198> 2014-08-13 16:24:25.193844 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:24.699832, event: throttled, request: pg_notify(22.1f1(4) epoch 194450) v5 -5197> 2014-08-13 16:24:25.193853 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:24.699904, event: all_read, request: pg_notify(22.1f1(4) epoch 194450) v5 -5196> 2014-08-13 16:24:25.193862 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:25.193831, event: dispatched, request: pg_notify(22.1f1(4) epoch 194450) v5 -5195> 2014-08-13 16:24:25.193872 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:25.193872, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194450) v5 -5194> 2014-08-13 16:24:25.193883 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:25.193883, event: started, request: pg_notify(22.1f1(4) epoch 194450) v5 -5193> 2014-08-13 16:24:25.193905 7fd93dff0700 5 -- op tracker -- , seq: 846, time: 2014-08-13 16:24:25.193905, event: done, request: pg_notify(22.1f1(4) epoch 194450) v5 -5192> 2014-08-13 16:24:25.193945 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 13 ==== pg_notify(22.3af(17) epoch 194450) v5 ==== 1665+0+0 (2560541676 0 0) 0x80d88c0 con 0x73cdc20 -5191> 2014-08-13 16:24:25.193973 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:24.715251, event: header_read, request: pg_notify(22.3af(17) epoch 194450) v5 -5190> 2014-08-13 16:24:25.193984 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:24.715252, event: throttled, request: pg_notify(22.3af(17) epoch 194450) v5 -5189> 2014-08-13 16:24:25.193993 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:24.715340, event: all_read, request: pg_notify(22.3af(17) epoch 194450) v5 -5188> 2014-08-13 16:24:25.194002 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:25.193963, event: dispatched, request: pg_notify(22.3af(17) epoch 194450) v5 -5187> 2014-08-13 16:24:25.194012 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:25.194012, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194450) v5 -5186> 2014-08-13 16:24:25.194023 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:25.194023, event: started, request: pg_notify(22.3af(17) epoch 194450) v5 -5185> 2014-08-13 16:24:25.194045 7fd93dff0700 5 -- op tracker -- , seq: 847, time: 2014-08-13 16:24:25.194045, event: done, request: pg_notify(22.3af(17) epoch 194450) v5 -5184> 2014-08-13 16:24:25.194077 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 4 ==== pg_notify(22.481(32) epoch 194450) v5 ==== 2518+0+0 (993786580 0 0) 0x8c8e900 con 0x73ccba0 -5183> 2014-08-13 16:24:25.194097 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:24.718007, event: header_read, request: pg_notify(22.481(32) epoch 194450) v5 -5182> 2014-08-13 16:24:25.194107 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:24.718009, event: throttled, request: pg_notify(22.481(32) epoch 194450) v5 -5181> 2014-08-13 16:24:25.194117 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:24.718130, event: all_read, request: pg_notify(22.481(32) epoch 194450) v5 -5180> 2014-08-13 16:24:25.194126 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:25.194095, event: dispatched, request: pg_notify(22.481(32) epoch 194450) v5 -5179> 2014-08-13 16:24:25.194135 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:25.194135, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194450) v5 -5178> 2014-08-13 16:24:25.194147 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:25.194146, event: started, request: pg_notify(22.481(32) epoch 194450) v5 -5177> 2014-08-13 16:24:25.194186 7fd93dff0700 5 -- op tracker -- , seq: 848, time: 2014-08-13 16:24:25.194186, event: done, request: pg_notify(22.481(32) epoch 194450) v5 -5176> 2014-08-13 16:24:25.194217 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 4 ==== pg_notify(22.3d9(35) epoch 194450) v5 ==== 2603+0+0 (578253407 0 0) 0x775a140 con 0x7462100 -5175> 2014-08-13 16:24:25.194238 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:24.940213, event: header_read, request: pg_notify(22.3d9(35) epoch 194450) v5 -5174> 2014-08-13 16:24:25.194248 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:24.940215, event: throttled, request: pg_notify(22.3d9(35) epoch 194450) v5 -5173> 2014-08-13 16:24:25.194257 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:24.940346, event: all_read, request: pg_notify(22.3d9(35) epoch 194450) v5 -5172> 2014-08-13 16:24:25.194267 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:25.194235, event: dispatched, request: pg_notify(22.3d9(35) epoch 194450) v5 -5171> 2014-08-13 16:24:25.194277 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:25.194276, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194450) v5 -5170> 2014-08-13 16:24:25.194287 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:25.194287, event: started, request: pg_notify(22.3d9(35) epoch 194450) v5 -5169> 2014-08-13 16:24:25.194362 7fd93dff0700 5 -- op tracker -- , seq: 849, time: 2014-08-13 16:24:25.194361, event: done, request: pg_notify(22.3d9(35) epoch 194450) v5 -5168> 2014-08-13 16:24:25.194397 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 9 ==== pg_notify(24.9b(4) epoch 194450) v5 ==== 976+0+0 (2373045065 0 0) 0x80deac0 con 0x7723440 -5167> 2014-08-13 16:24:25.194408 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5166> 2014-08-13 16:24:25.194419 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:24.707169, event: header_read, request: pg_notify(24.9b(4) epoch 194450) v5 -5165> 2014-08-13 16:24:25.194432 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_alive(want up_thru 194450 have 194450) v1 -- ?+0 0x7d04ec0 con 0x4e3dd80 -5164> 2014-08-13 16:24:25.194439 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:24.707171, event: throttled, request: pg_notify(24.9b(4) epoch 194450) v5 -5163> 2014-08-13 16:24:25.194459 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:24.707258, event: all_read, request: pg_notify(24.9b(4) epoch 194450) v5 -5162> 2014-08-13 16:24:25.194470 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:25.194417, event: dispatched, request: pg_notify(24.9b(4) epoch 194450) v5 -5161> 2014-08-13 16:24:25.194467 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6800/61726 -- osd_map(194450..194450 src has 185139..194450) v3 -- ?+0 0x7b12400 con 0x7313860 -5160> 2014-08-13 16:24:25.194480 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:25.194480, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194450) v5 -5159> 2014-08-13 16:24:25.194489 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:25.194489, event: started, request: pg_notify(24.9b(4) epoch 194450) v5 -5158> 2014-08-13 16:24:25.194489 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6800/61726 -- pg_query(24.39a epoch 194450) v3 -- ?+0 0x801d460 con 0x7313860 -5157> 2014-08-13 16:24:25.194504 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6810/41287 -- pg_query(24.39a epoch 194450) v3 -- ?+0 0x8018000 con 0x51127e0 -5156> 2014-08-13 16:24:25.194521 7fd93dff0700 5 -- op tracker -- , seq: 850, time: 2014-08-13 16:24:25.194520, event: done, request: pg_notify(24.9b(4) epoch 194450) v5 -5155> 2014-08-13 16:24:25.194552 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 2 ==== pg_notify(24.34(29) epoch 194450) v5 ==== 2199+0+0 (2298229182 0 0) 0x87e3100 con 0x7d2d120 -5154> 2014-08-13 16:24:25.194581 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:24.692719, event: header_read, request: pg_notify(24.34(29) epoch 194450) v5 -5153> 2014-08-13 16:24:25.194593 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:24.692721, event: throttled, request: pg_notify(24.34(29) epoch 194450) v5 -5152> 2014-08-13 16:24:25.194602 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:24.692830, event: all_read, request: pg_notify(24.34(29) epoch 194450) v5 -5151> 2014-08-13 16:24:25.194611 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:25.194570, event: dispatched, request: pg_notify(24.34(29) epoch 194450) v5 -5150> 2014-08-13 16:24:25.194621 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:25.194621, event: waiting_for_osdmap, request: pg_notify(24.34(29) epoch 194450) v5 -5149> 2014-08-13 16:24:25.194632 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:25.194632, event: started, request: pg_notify(24.34(29) epoch 194450) v5 -5148> 2014-08-13 16:24:25.194658 7fd93dff0700 5 -- op tracker -- , seq: 851, time: 2014-08-13 16:24:25.194658, event: done, request: pg_notify(24.34(29) epoch 194450) v5 -5147> 2014-08-13 16:24:25.194697 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 6 ==== pg_notify(22.619(17) epoch 194450) v5 ==== 1665+0+0 (2086399844 0 0) 0x80d8fc0 con 0x7d2d280 -5146> 2014-08-13 16:24:25.194719 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:24.714503, event: header_read, request: pg_notify(22.619(17) epoch 194450) v5 -5145> 2014-08-13 16:24:25.194729 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:24.714504, event: throttled, request: pg_notify(22.619(17) epoch 194450) v5 -5144> 2014-08-13 16:24:25.194739 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:24.714623, event: all_read, request: pg_notify(22.619(17) epoch 194450) v5 -5143> 2014-08-13 16:24:25.194754 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:25.194716, event: dispatched, request: pg_notify(22.619(17) epoch 194450) v5 -5142> 2014-08-13 16:24:25.194764 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:25.194763, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194450) v5 -5141> 2014-08-13 16:24:25.194775 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:25.194775, event: started, request: pg_notify(22.619(17) epoch 194450) v5 -5140> 2014-08-13 16:24:25.194798 7fd93dff0700 5 -- op tracker -- , seq: 852, time: 2014-08-13 16:24:25.194798, event: done, request: pg_notify(22.619(17) epoch 194450) v5 -5139> 2014-08-13 16:24:25.194834 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 6 ==== pg_notify(22.775(17) epoch 194450) v5 ==== 1665+0+0 (5291565 0 0) 0x8c88fc0 con 0x821f900 -5138> 2014-08-13 16:24:25.194854 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:24.713765, event: header_read, request: pg_notify(22.775(17) epoch 194450) v5 -5137> 2014-08-13 16:24:25.194865 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:24.713765, event: throttled, request: pg_notify(22.775(17) epoch 194450) v5 -5136> 2014-08-13 16:24:25.194874 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:24.713847, event: all_read, request: pg_notify(22.775(17) epoch 194450) v5 -5135> 2014-08-13 16:24:25.194882 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:25.194852, event: dispatched, request: pg_notify(22.775(17) epoch 194450) v5 -5134> 2014-08-13 16:24:25.194892 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:25.194892, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194450) v5 -5133> 2014-08-13 16:24:25.194903 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:25.194903, event: started, request: pg_notify(22.775(17) epoch 194450) v5 -5132> 2014-08-13 16:24:25.194926 7fd93dff0700 5 -- op tracker -- , seq: 853, time: 2014-08-13 16:24:25.194926, event: done, request: pg_notify(22.775(17) epoch 194450) v5 -5131> 2014-08-13 16:24:25.194949 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 4 ==== pg_notify(22.3d9(39) epoch 194450) v5 ==== 2815+0+0 (3149102051 0 0) 0x8c8c980 con 0x80c7e80 -5130> 2014-08-13 16:24:25.194968 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:24.697930, event: header_read, request: pg_notify(22.3d9(39) epoch 194450) v5 -5129> 2014-08-13 16:24:25.194978 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:24.697932, event: throttled, request: pg_notify(22.3d9(39) epoch 194450) v5 -5128> 2014-08-13 16:24:25.194988 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:24.698081, event: all_read, request: pg_notify(22.3d9(39) epoch 194450) v5 -5127> 2014-08-13 16:24:25.194997 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:25.194966, event: dispatched, request: pg_notify(22.3d9(39) epoch 194450) v5 -5126> 2014-08-13 16:24:25.195006 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:25.195006, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194450) v5 -5125> 2014-08-13 16:24:25.195020 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:25.195020, event: started, request: pg_notify(22.3d9(39) epoch 194450) v5 -5124> 2014-08-13 16:24:25.195043 7fd93dff0700 5 -- op tracker -- , seq: 854, time: 2014-08-13 16:24:25.195043, event: done, request: pg_notify(22.3d9(39) epoch 194450) v5 -5123> 2014-08-13 16:24:25.195062 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 6 ==== pg_notify(22.481(34) epoch 194450) v5 ==== 2620+0+0 (1321096168 0 0) 0x8c8d940 con 0x80c7d20 -5122> 2014-08-13 16:24:25.195081 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:24.726795, event: header_read, request: pg_notify(22.481(34) epoch 194450) v5 -5121> 2014-08-13 16:24:25.195091 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:24.726796, event: throttled, request: pg_notify(22.481(34) epoch 194450) v5 -5120> 2014-08-13 16:24:25.195100 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:24.726933, event: all_read, request: pg_notify(22.481(34) epoch 194450) v5 -5119> 2014-08-13 16:24:25.195109 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:25.195079, event: dispatched, request: pg_notify(22.481(34) epoch 194450) v5 -5118> 2014-08-13 16:24:25.195119 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:25.195118, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194450) v5 -5117> 2014-08-13 16:24:25.195129 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:25.195129, event: started, request: pg_notify(22.481(34) epoch 194450) v5 -5116> 2014-08-13 16:24:25.195151 7fd93dff0700 5 -- op tracker -- , seq: 855, time: 2014-08-13 16:24:25.195151, event: done, request: pg_notify(22.481(34) epoch 194450) v5 -5115> 2014-08-13 16:24:25.195175 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 14 ==== pg_notify(24.30b(4) epoch 194450) v5 ==== 976+0+0 (3315474031 0 0) 0x8c88700 con 0x4e4b860 -5114> 2014-08-13 16:24:25.195205 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:24.688165, event: header_read, request: pg_notify(24.30b(4) epoch 194450) v5 -5113> 2014-08-13 16:24:25.195216 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:24.688166, event: throttled, request: pg_notify(24.30b(4) epoch 194450) v5 -5112> 2014-08-13 16:24:25.195226 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:24.688333, event: all_read, request: pg_notify(24.30b(4) epoch 194450) v5 -5111> 2014-08-13 16:24:25.195235 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:25.195203, event: dispatched, request: pg_notify(24.30b(4) epoch 194450) v5 -5110> 2014-08-13 16:24:25.195245 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:25.195245, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194450) v5 -5109> 2014-08-13 16:24:25.195256 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:25.195256, event: started, request: pg_notify(24.30b(4) epoch 194450) v5 -5108> 2014-08-13 16:24:25.195280 7fd93dff0700 5 -- op tracker -- , seq: 856, time: 2014-08-13 16:24:25.195279, event: done, request: pg_notify(24.30b(4) epoch 194450) v5 -5107> 2014-08-13 16:24:25.195308 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 18 ==== pg_notify(22.1d2(17) epoch 194450) v5 ==== 1669+0+0 (403066675 0 0) 0x7d06740 con 0x4e4dd80 -5106> 2014-08-13 16:24:25.195328 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:24.713426, event: header_read, request: pg_notify(22.1d2(17) epoch 194450) v5 -5105> 2014-08-13 16:24:25.195338 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:24.713427, event: throttled, request: pg_notify(22.1d2(17) epoch 194450) v5 -5104> 2014-08-13 16:24:25.195348 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:24.713530, event: all_read, request: pg_notify(22.1d2(17) epoch 194450) v5 -5103> 2014-08-13 16:24:25.195357 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:25.195326, event: dispatched, request: pg_notify(22.1d2(17) epoch 194450) v5 -5102> 2014-08-13 16:24:25.195366 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:25.195366, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194450) v5 -5101> 2014-08-13 16:24:25.195377 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:25.195377, event: started, request: pg_notify(22.1d2(17) epoch 194450) v5 -5100> 2014-08-13 16:24:25.195400 7fd93dff0700 5 -- op tracker -- , seq: 857, time: 2014-08-13 16:24:25.195400, event: done, request: pg_notify(22.1d2(17) epoch 194450) v5 -5099> 2014-08-13 16:24:25.195422 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 12 ==== pg_notify(22.1f1(17) epoch 194450) v5 ==== 1665+0+0 (1698113794 0 0) 0x899aa00 con 0x4e4ce60 -5098> 2014-08-13 16:24:25.195441 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:24.711926, event: header_read, request: pg_notify(22.1f1(17) epoch 194450) v5 -5097> 2014-08-13 16:24:25.195451 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:24.711927, event: throttled, request: pg_notify(22.1f1(17) epoch 194450) v5 -5096> 2014-08-13 16:24:25.195461 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:24.712012, event: all_read, request: pg_notify(22.1f1(17) epoch 194450) v5 -5095> 2014-08-13 16:24:25.195469 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:25.195439, event: dispatched, request: pg_notify(22.1f1(17) epoch 194450) v5 -5094> 2014-08-13 16:24:25.195479 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:25.195479, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194450) v5 -5093> 2014-08-13 16:24:25.195490 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:25.195490, event: started, request: pg_notify(22.1f1(17) epoch 194450) v5 -5092> 2014-08-13 16:24:25.195513 7fd93dff0700 5 -- op tracker -- , seq: 858, time: 2014-08-13 16:24:25.195512, event: done, request: pg_notify(22.1f1(17) epoch 194450) v5 -5091> 2014-08-13 16:24:25.195538 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 19 ==== pg_notify(24.34(42) epoch 194450) v5 ==== 2882+0+0 (3113999263 0 0) 0x9105e80 con 0x4e4f900 -5090> 2014-08-13 16:24:25.195559 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:24.726020, event: header_read, request: pg_notify(24.34(42) epoch 194450) v5 -5089> 2014-08-13 16:24:25.195569 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:24.726022, event: throttled, request: pg_notify(24.34(42) epoch 194450) v5 -5088> 2014-08-13 16:24:25.195587 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:24.726190, event: all_read, request: pg_notify(24.34(42) epoch 194450) v5 -5087> 2014-08-13 16:24:25.195597 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:25.195557, event: dispatched, request: pg_notify(24.34(42) epoch 194450) v5 -5086> 2014-08-13 16:24:25.195606 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:25.195606, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194450) v5 -5085> 2014-08-13 16:24:25.195617 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:25.195617, event: started, request: pg_notify(24.34(42) epoch 194450) v5 -5084> 2014-08-13 16:24:25.195642 7fd93dff0700 5 -- op tracker -- , seq: 859, time: 2014-08-13 16:24:25.195642, event: done, request: pg_notify(24.34(42) epoch 194450) v5 -5083> 2014-08-13 16:24:25.195670 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 23 ==== pg_notify(24.136(17) epoch 194450) v5 ==== 1695+0+0 (3474366168 0 0) 0x80d9f80 con 0x4fd8000 -5082> 2014-08-13 16:24:25.195690 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:24.714655, event: header_read, request: pg_notify(24.136(17) epoch 194450) v5 -5081> 2014-08-13 16:24:25.195700 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:24.714656, event: throttled, request: pg_notify(24.136(17) epoch 194450) v5 -5080> 2014-08-13 16:24:25.195709 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:24.714762, event: all_read, request: pg_notify(24.136(17) epoch 194450) v5 -5079> 2014-08-13 16:24:25.195718 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:25.195688, event: dispatched, request: pg_notify(24.136(17) epoch 194450) v5 -5078> 2014-08-13 16:24:25.195728 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:25.195728, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194450) v5 -5077> 2014-08-13 16:24:25.195739 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:25.195739, event: started, request: pg_notify(24.136(17) epoch 194450) v5 -5076> 2014-08-13 16:24:25.195762 7fd93dff0700 5 -- op tracker -- , seq: 860, time: 2014-08-13 16:24:25.195762, event: done, request: pg_notify(24.136(17) epoch 194450) v5 -5075> 2014-08-13 16:24:25.195784 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 16 ==== pg_notify(22.99(15) epoch 194450) v5 ==== 1551+0+0 (1417088600 0 0) 0x80d96c0 con 0x4fe1600 -5074> 2014-08-13 16:24:25.195808 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:24.888175, event: header_read, request: pg_notify(22.99(15) epoch 194450) v5 -5073> 2014-08-13 16:24:25.195818 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:24.888176, event: throttled, request: pg_notify(22.99(15) epoch 194450) v5 -5072> 2014-08-13 16:24:25.195827 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:24.888272, event: all_read, request: pg_notify(22.99(15) epoch 194450) v5 -5071> 2014-08-13 16:24:25.195836 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:25.195806, event: dispatched, request: pg_notify(22.99(15) epoch 194450) v5 -5070> 2014-08-13 16:24:25.195846 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:25.195845, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194450) v5 -5069> 2014-08-13 16:24:25.195856 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:25.195856, event: started, request: pg_notify(22.99(15) epoch 194450) v5 -5068> 2014-08-13 16:24:25.195879 7fd93dff0700 5 -- op tracker -- , seq: 861, time: 2014-08-13 16:24:25.195879, event: done, request: pg_notify(22.99(15) epoch 194450) v5 -5067> 2014-08-13 16:24:25.195903 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 15 ==== pg_notify(22.3af(4) epoch 194450) v5 ==== 942+0+0 (3777779298 0 0) 0x81c39c0 con 0x4fe40a0 -5066> 2014-08-13 16:24:25.195922 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:24.687102, event: header_read, request: pg_notify(22.3af(4) epoch 194450) v5 -5065> 2014-08-13 16:24:25.195932 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:24.687103, event: throttled, request: pg_notify(22.3af(4) epoch 194450) v5 -5064> 2014-08-13 16:24:25.195942 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:24.687185, event: all_read, request: pg_notify(22.3af(4) epoch 194450) v5 -5063> 2014-08-13 16:24:25.195950 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:25.195920, event: dispatched, request: pg_notify(22.3af(4) epoch 194450) v5 -5062> 2014-08-13 16:24:25.195960 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:25.195960, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194450) v5 -5061> 2014-08-13 16:24:25.195971 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:25.195971, event: started, request: pg_notify(22.3af(4) epoch 194450) v5 -5060> 2014-08-13 16:24:25.195993 7fd93dff0700 5 -- op tracker -- , seq: 862, time: 2014-08-13 16:24:25.195993, event: done, request: pg_notify(22.3af(4) epoch 194450) v5 -5059> 2014-08-13 16:24:25.196015 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 19 ==== pg_notify(24.34(42) epoch 194450) v5 ==== 2882+0+0 (1268677266 0 0) 0x775ec80 con 0x4fe69e0 -5058> 2014-08-13 16:24:25.196034 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:24.912071, event: header_read, request: pg_notify(24.34(42) epoch 194450) v5 -5057> 2014-08-13 16:24:25.196044 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:24.912073, event: throttled, request: pg_notify(24.34(42) epoch 194450) v5 -5056> 2014-08-13 16:24:25.196051 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:24.912253, event: all_read, request: pg_notify(24.34(42) epoch 194450) v5 -5055> 2014-08-13 16:24:25.196059 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:25.196032, event: dispatched, request: pg_notify(24.34(42) epoch 194450) v5 -5054> 2014-08-13 16:24:25.196069 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:25.196069, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194450) v5 -5053> 2014-08-13 16:24:25.196080 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:25.196080, event: started, request: pg_notify(24.34(42) epoch 194450) v5 -5052> 2014-08-13 16:24:25.196104 7fd93dff0700 5 -- op tracker -- , seq: 863, time: 2014-08-13 16:24:25.196103, event: done, request: pg_notify(24.34(42) epoch 194450) v5 -5051> 2014-08-13 16:24:25.196149 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 18 ==== pg_notify(22.3e4(17) epoch 194450) v5 ==== 1665+0+0 (2557289884 0 0) 0x80de580 con 0x4fda940 -5050> 2014-08-13 16:24:25.196170 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:24.705754, event: header_read, request: pg_notify(22.3e4(17) epoch 194450) v5 -5049> 2014-08-13 16:24:25.196189 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:24.705755, event: throttled, request: pg_notify(22.3e4(17) epoch 194450) v5 -5048> 2014-08-13 16:24:25.196198 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:24.705848, event: all_read, request: pg_notify(22.3e4(17) epoch 194450) v5 -5047> 2014-08-13 16:24:25.196204 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:25.196168, event: dispatched, request: pg_notify(22.3e4(17) epoch 194450) v5 -5046> 2014-08-13 16:24:25.196214 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:25.196214, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194450) v5 -5045> 2014-08-13 16:24:25.196225 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:25.196225, event: started, request: pg_notify(22.3e4(17) epoch 194450) v5 -5044> 2014-08-13 16:24:25.196248 7fd93dff0700 5 -- op tracker -- , seq: 864, time: 2014-08-13 16:24:25.196248, event: done, request: pg_notify(22.3e4(17) epoch 194450) v5 -5043> 2014-08-13 16:24:25.196270 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 2 ==== pg_notify(24.39a(41) epoch 194450) v5 ==== 2987+0+0 (4104395866 0 0) 0x87e47c0 con 0x4fd9b80 -5042> 2014-08-13 16:24:25.196290 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:24.693904, event: header_read, request: pg_notify(24.39a(41) epoch 194450) v5 -5041> 2014-08-13 16:24:25.196300 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:24.693906, event: throttled, request: pg_notify(24.39a(41) epoch 194450) v5 -5040> 2014-08-13 16:24:25.196309 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:24.694037, event: all_read, request: pg_notify(24.39a(41) epoch 194450) v5 -5039> 2014-08-13 16:24:25.196318 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:25.196287, event: dispatched, request: pg_notify(24.39a(41) epoch 194450) v5 -5038> 2014-08-13 16:24:25.196327 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:25.196327, event: waiting_for_osdmap, request: pg_notify(24.39a(41) epoch 194450) v5 -5037> 2014-08-13 16:24:25.196338 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:25.196338, event: started, request: pg_notify(24.39a(41) epoch 194450) v5 -5036> 2014-08-13 16:24:25.196361 7fd93dff0700 5 -- op tracker -- , seq: 865, time: 2014-08-13 16:24:25.196360, event: done, request: pg_notify(24.39a(41) epoch 194450) v5 -5035> 2014-08-13 16:24:25.196381 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 19 ==== osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (942085385 0 4067132163) 0x8e8d780 con 0x4fdb2e0 -5034> 2014-08-13 16:24:25.196404 7fd93dff0700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.190961, event: header_read, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5033> 2014-08-13 16:24:25.196418 7fd93dff0700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.190963, event: throttled, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5032> 2014-08-13 16:24:25.196430 7fd93dff0700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.191092, event: all_read, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5031> 2014-08-13 16:24:25.196441 7fd93dff0700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.196402, event: dispatched, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5030> 2014-08-13 16:24:25.196453 7fd93dff0700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.196453, event: waiting_for_osdmap, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5029> 2014-08-13 16:24:25.196475 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 18 ==== pg_notify(22.113(17) epoch 194450) v5 ==== 1669+0+0 (1824872029 0 0) 0x81c4440 con 0x50889a0 -5028> 2014-08-13 16:24:25.196494 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:24.698112, event: header_read, request: pg_notify(22.113(17) epoch 194450) v5 -5027> 2014-08-13 16:24:25.196504 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:24.698114, event: throttled, request: pg_notify(22.113(17) epoch 194450) v5 -5026> 2014-08-13 16:24:25.196513 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:24.698239, event: all_read, request: pg_notify(22.113(17) epoch 194450) v5 -5025> 2014-08-13 16:24:25.196522 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:25.196492, event: dispatched, request: pg_notify(22.113(17) epoch 194450) v5 -5024> 2014-08-13 16:24:25.196532 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:25.196532, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194450) v5 -5023> 2014-08-13 16:24:25.196543 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:25.196543, event: started, request: pg_notify(22.113(17) epoch 194450) v5 -5022> 2014-08-13 16:24:25.196566 7fd93dff0700 5 -- op tracker -- , seq: 867, time: 2014-08-13 16:24:25.196565, event: done, request: pg_notify(22.113(17) epoch 194450) v5 -5021> 2014-08-13 16:24:25.196588 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 14 ==== pg_notify(22.619(4) epoch 194450) v5 ==== 942+0+0 (3481651328 0 0) 0x81c6e40 con 0x508a680 -5020> 2014-08-13 16:24:25.196607 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:24.698889, event: header_read, request: pg_notify(22.619(4) epoch 194450) v5 -5019> 2014-08-13 16:24:25.196618 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:24.698890, event: throttled, request: pg_notify(22.619(4) epoch 194450) v5 -5018> 2014-08-13 16:24:25.196627 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:24.698961, event: all_read, request: pg_notify(22.619(4) epoch 194450) v5 -5017> 2014-08-13 16:24:25.196636 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:25.196605, event: dispatched, request: pg_notify(22.619(4) epoch 194450) v5 -5016> 2014-08-13 16:24:25.196646 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:25.196646, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194450) v5 -5015> 2014-08-13 16:24:25.196657 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:25.196657, event: started, request: pg_notify(22.619(4) epoch 194450) v5 -5014> 2014-08-13 16:24:25.196683 7fd93dff0700 5 -- op tracker -- , seq: 868, time: 2014-08-13 16:24:25.196682, event: done, request: pg_notify(22.619(4) epoch 194450) v5 -5013> 2014-08-13 16:24:25.196702 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.50 192.168.1.112:6842/9607 15 ==== osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (4102455617 0 4067132163) 0x8e88a00 con 0x508c620 -5012> 2014-08-13 16:24:25.196723 7fd93dff0700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.190648, event: header_read, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5011> 2014-08-13 16:24:25.196736 7fd93dff0700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.190650, event: throttled, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5010> 2014-08-13 16:24:25.196748 7fd93dff0700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.190719, event: all_read, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5009> 2014-08-13 16:24:25.196759 7fd93dff0700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.196721, event: dispatched, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5008> 2014-08-13 16:24:25.196771 7fd93dff0700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.196771, event: waiting_for_osdmap, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -5007> 2014-08-13 16:24:25.196790 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 28 ==== pg_notify(22.99(15) epoch 194450) v5 ==== 1551+0+0 (276714767 0 0) 0x91024c0 con 0x508bde0 -5006> 2014-08-13 16:24:25.196808 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:24.721121, event: header_read, request: pg_notify(22.99(15) epoch 194450) v5 -5005> 2014-08-13 16:24:25.196818 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:24.721121, event: throttled, request: pg_notify(22.99(15) epoch 194450) v5 -5004> 2014-08-13 16:24:25.196827 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:24.721239, event: all_read, request: pg_notify(22.99(15) epoch 194450) v5 -5003> 2014-08-13 16:24:25.196836 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:25.196806, event: dispatched, request: pg_notify(22.99(15) epoch 194450) v5 -5002> 2014-08-13 16:24:25.196846 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:25.196845, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194450) v5 -5001> 2014-08-13 16:24:25.196857 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:25.196857, event: started, request: pg_notify(22.99(15) epoch 194450) v5 -5000> 2014-08-13 16:24:25.196880 7fd93dff0700 5 -- op tracker -- , seq: 870, time: 2014-08-13 16:24:25.196880, event: done, request: pg_notify(22.99(15) epoch 194450) v5 -4999> 2014-08-13 16:24:25.196901 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 14 ==== pg_notify(22.707(19) epoch 194450) v5 ==== 1783+0+0 (156432357 0 0) 0x81c7a80 con 0x50ba520 -4998> 2014-08-13 16:24:25.196920 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:24.720941, event: header_read, request: pg_notify(22.707(19) epoch 194450) v5 -4997> 2014-08-13 16:24:25.196933 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:24.720942, event: throttled, request: pg_notify(22.707(19) epoch 194450) v5 -4996> 2014-08-13 16:24:25.196939 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:24.721065, event: all_read, request: pg_notify(22.707(19) epoch 194450) v5 -4995> 2014-08-13 16:24:25.196945 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:25.196918, event: dispatched, request: pg_notify(22.707(19) epoch 194450) v5 -4994> 2014-08-13 16:24:25.196920 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 incomplete] exit Started/Primary/Peering/Incomplete 0.005720 2 0.000471 -4993> 2014-08-13 16:24:25.196951 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:25.196951, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194450) v5 -4992> 2014-08-13 16:24:25.196959 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:25.196958, event: started, request: pg_notify(22.707(19) epoch 194450) v5 -4991> 2014-08-13 16:24:25.196955 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/GetLog -4990> 2014-08-13 16:24:25.196978 7fd93dff0700 5 -- op tracker -- , seq: 871, time: 2014-08-13 16:24:25.196978, event: done, request: pg_notify(22.707(19) epoch 194450) v5 -4989> 2014-08-13 16:24:25.197005 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 7 ==== pg_notify(22.113(17) epoch 194450) v5 ==== 1669+0+0 (646519014 0 0) 0x8c8a680 con 0x50bfa60 -4988> 2014-08-13 16:24:25.197007 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary/Peering/GetLog 0.000053 0 0.000000 -4987> 2014-08-13 16:24:25.197025 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:24.910381, event: header_read, request: pg_notify(22.113(17) epoch 194450) v5 -4986> 2014-08-13 16:24:25.197032 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:24.910382, event: throttled, request: pg_notify(22.113(17) epoch 194450) v5 -4985> 2014-08-13 16:24:25.197027 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/Incomplete -4984> 2014-08-13 16:24:25.197038 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:24.910488, event: all_read, request: pg_notify(22.113(17) epoch 194450) v5 -4983> 2014-08-13 16:24:25.197044 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:25.197022, event: dispatched, request: pg_notify(22.113(17) epoch 194450) v5 -4982> 2014-08-13 16:24:25.197050 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:25.197050, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194450) v5 -4981> 2014-08-13 16:24:25.197058 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:25.197057, event: started, request: pg_notify(22.113(17) epoch 194450) v5 -4980> 2014-08-13 16:24:25.197076 7fd93dff0700 5 -- op tracker -- , seq: 872, time: 2014-08-13 16:24:25.197075, event: done, request: pg_notify(22.113(17) epoch 194450) v5 -4979> 2014-08-13 16:24:25.197090 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 incomplete] exit Started/Primary/Peering/Incomplete 0.000063 1 0.000136 -4978> 2014-08-13 16:24:25.197099 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 15 ==== pg_notify(22.362(17) epoch 194450) v5 ==== 1665+0+0 (3573334538 0 0) 0x81c7000 con 0x510b9c0 -4977> 2014-08-13 16:24:25.197130 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:24.716934, event: header_read, request: pg_notify(22.362(17) epoch 194450) v5 -4976> 2014-08-13 16:24:25.197123 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/GetLog -4975> 2014-08-13 16:24:25.197138 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:24.716935, event: throttled, request: pg_notify(22.362(17) epoch 194450) v5 -4974> 2014-08-13 16:24:25.197148 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:24.717013, event: all_read, request: pg_notify(22.362(17) epoch 194450) v5 -4973> 2014-08-13 16:24:25.197160 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:25.197127, event: dispatched, request: pg_notify(22.362(17) epoch 194450) v5 -4972> 2014-08-13 16:24:25.197166 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:25.197166, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194450) v5 -4971> 2014-08-13 16:24:25.197174 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:25.197174, event: started, request: pg_notify(22.362(17) epoch 194450) v5 -4970> 2014-08-13 16:24:25.197180 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary/Peering/GetLog 0.000057 0 0.000000 -4969> 2014-08-13 16:24:25.197201 7fd93dff0700 5 -- op tracker -- , seq: 873, time: 2014-08-13 16:24:25.197200, event: done, request: pg_notify(22.362(17) epoch 194450) v5 -4968> 2014-08-13 16:24:25.197208 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/Incomplete -4967> 2014-08-13 16:24:25.197233 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 13 ==== pg_notify(22.707(4) epoch 194450) v5 ==== 942+0+0 (1878822056 0 0) 0x80df540 con 0x5108b00 -4966> 2014-08-13 16:24:25.197255 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:24.841052, event: header_read, request: pg_notify(22.707(4) epoch 194450) v5 -4965> 2014-08-13 16:24:25.197262 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:24.841053, event: throttled, request: pg_notify(22.707(4) epoch 194450) v5 -4964> 2014-08-13 16:24:25.197251 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 incomplete] exit Started/Primary/Peering/Incomplete 0.000042 1 0.000143 -4963> 2014-08-13 16:24:25.197268 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:24.841135, event: all_read, request: pg_notify(22.707(4) epoch 194450) v5 -4962> 2014-08-13 16:24:25.197274 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:25.197252, event: dispatched, request: pg_notify(22.707(4) epoch 194450) v5 -4961> 2014-08-13 16:24:25.197281 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:25.197281, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194450) v5 -4960> 2014-08-13 16:24:25.197272 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/GetLog -4959> 2014-08-13 16:24:25.197289 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:25.197288, event: started, request: pg_notify(22.707(4) epoch 194450) v5 -4958> 2014-08-13 16:24:25.197307 7fd93dff0700 5 -- op tracker -- , seq: 874, time: 2014-08-13 16:24:25.197307, event: done, request: pg_notify(22.707(4) epoch 194450) v5 -4957> 2014-08-13 16:24:25.197327 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 7 ==== pg_notify(22.3d9(39) epoch 194450) v5 ==== 2815+0+0 (2515339276 0 0) 0x8999880 con 0x510dac0 -4956> 2014-08-13 16:24:25.197327 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary/Peering/GetLog 0.000055 0 0.000000 -4955> 2014-08-13 16:24:25.197343 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:24.707244, event: header_read, request: pg_notify(22.3d9(39) epoch 194450) v5 -4954> 2014-08-13 16:24:25.197351 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:24.707246, event: throttled, request: pg_notify(22.3d9(39) epoch 194450) v5 -4953> 2014-08-13 16:24:25.197356 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:24.707376, event: all_read, request: pg_notify(22.3d9(39) epoch 194450) v5 -4952> 2014-08-13 16:24:25.197347 7fd9367e4700 5 osd.4 pg_epoch: 194450 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering/Incomplete -4951> 2014-08-13 16:24:25.197362 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:25.197341, event: dispatched, request: pg_notify(22.3d9(39) epoch 194450) v5 -4950> 2014-08-13 16:24:25.197369 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:25.197368, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194450) v5 -4949> 2014-08-13 16:24:25.197376 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:25.197375, event: started, request: pg_notify(22.3d9(39) epoch 194450) v5 -4948> 2014-08-13 16:24:25.197393 7fd93dff0700 5 -- op tracker -- , seq: 875, time: 2014-08-13 16:24:25.197393, event: done, request: pg_notify(22.3d9(39) epoch 194450) v5 -4947> 2014-08-13 16:24:25.197415 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 2 ==== pg_notify(24.39a(47) epoch 194450) v5 ==== 3273+0+0 (2819770588 0 0) 0x899eac0 con 0x51127e0 -4946> 2014-08-13 16:24:25.197438 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:24.714234, event: header_read, request: pg_notify(24.39a(47) epoch 194450) v5 -4945> 2014-08-13 16:24:25.197445 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:24.714237, event: throttled, request: pg_notify(24.39a(47) epoch 194450) v5 -4944> 2014-08-13 16:24:25.197464 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:24.714377, event: all_read, request: pg_notify(24.39a(47) epoch 194450) v5 -4943> 2014-08-13 16:24:25.197470 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:25.197436, event: dispatched, request: pg_notify(24.39a(47) epoch 194450) v5 -4942> 2014-08-13 16:24:25.197476 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:25.197476, event: waiting_for_osdmap, request: pg_notify(24.39a(47) epoch 194450) v5 -4941> 2014-08-13 16:24:25.197483 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:25.197483, event: started, request: pg_notify(24.39a(47) epoch 194450) v5 -4940> 2014-08-13 16:24:25.197508 7fd93dff0700 5 -- op tracker -- , seq: 876, time: 2014-08-13 16:24:25.197508, event: done, request: pg_notify(24.39a(47) epoch 194450) v5 -4939> 2014-08-13 16:24:25.197549 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 29 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (514409132 0 0) 0x86dd800 con 0x5112520 -4938> 2014-08-13 16:24:25.197576 7fd93dff0700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.188483, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4937> 2014-08-13 16:24:25.197592 7fd93dff0700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.188484, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4936> 2014-08-13 16:24:25.197605 7fd93dff0700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.188577, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4935> 2014-08-13 16:24:25.197618 7fd93dff0700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.197574, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4934> 2014-08-13 16:24:25.197629 7fd93dff0700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.197629, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4933> 2014-08-13 16:24:25.197663 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 12 ==== pg_notify(22.4df(15) epoch 194450) v5 ==== 1551+0+0 (1013148116 0 0) 0x81c5b00 con 0x5112260 -4932> 2014-08-13 16:24:25.197682 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:24.703366, event: header_read, request: pg_notify(22.4df(15) epoch 194450) v5 -4931> 2014-08-13 16:24:25.197689 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:24.703367, event: throttled, request: pg_notify(22.4df(15) epoch 194450) v5 -4930> 2014-08-13 16:24:25.197695 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:24.703484, event: all_read, request: pg_notify(22.4df(15) epoch 194450) v5 -4929> 2014-08-13 16:24:25.197704 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:25.197680, event: dispatched, request: pg_notify(22.4df(15) epoch 194450) v5 -4928> 2014-08-13 16:24:25.197721 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:25.197721, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194450) v5 -4927> 2014-08-13 16:24:25.197727 7fd9367e4700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.197727, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4926> 2014-08-13 16:24:25.197770 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:25.197729, event: started, request: pg_notify(22.4df(15) epoch 194450) v5 -4925> 2014-08-13 16:24:25.197798 7fd9367e4700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.197772, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4924> 2014-08-13 16:24:25.197829 7fd9367e4700 5 -- op tracker -- , seq: 877, time: 2014-08-13 16:24:25.197829, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4923> 2014-08-13 16:24:25.197871 7fd93dff0700 5 -- op tracker -- , seq: 878, time: 2014-08-13 16:24:25.197803, event: done, request: pg_notify(22.4df(15) epoch 194450) v5 -4922> 2014-08-13 16:24:25.197953 7fd9367e4700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.197909, event: reached_pg, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4921> 2014-08-13 16:24:25.197986 7fd9367e4700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.197986, event: started, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4920> 2014-08-13 16:24:25.197985 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 10 ==== pg_notify(22.1d2(4) epoch 194450) v5 ==== 942+0+0 (511978276 0 0) 0x81c6c80 con 0x5114620 -4919> 2014-08-13 16:24:25.198004 7fd9367e4700 5 -- op tracker -- , seq: 866, time: 2014-08-13 16:24:25.198004, event: done, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4918> 2014-08-13 16:24:25.198037 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:24.943896, event: header_read, request: pg_notify(22.1d2(4) epoch 194450) v5 -4917> 2014-08-13 16:24:25.198055 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:24.943897, event: throttled, request: pg_notify(22.1d2(4) epoch 194450) v5 -4916> 2014-08-13 16:24:25.198066 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:24.943966, event: all_read, request: pg_notify(22.1d2(4) epoch 194450) v5 -4915> 2014-08-13 16:24:25.198079 7fd9367e4700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.198044, event: reached_pg, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4914> 2014-08-13 16:24:25.198109 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:25.198010, event: dispatched, request: pg_notify(22.1d2(4) epoch 194450) v5 -4913> 2014-08-13 16:24:25.198127 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:25.198127, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194450) v5 -4912> 2014-08-13 16:24:25.198141 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:25.198140, event: started, request: pg_notify(22.1d2(4) epoch 194450) v5 -4911> 2014-08-13 16:24:25.198177 7fd9367e4700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.198111, event: started, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4910> 2014-08-13 16:24:25.198215 7fd93dff0700 5 -- op tracker -- , seq: 879, time: 2014-08-13 16:24:25.198187, event: done, request: pg_notify(22.1d2(4) epoch 194450) v5 -4909> 2014-08-13 16:24:25.198247 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 17 ==== pg_notify(22.39d(15) epoch 194450) v5 ==== 1551+0+0 (2956017696 0 0) 0x7d05400 con 0x5114360 -4908> 2014-08-13 16:24:25.198258 7fd9367e4700 5 -- op tracker -- , seq: 869, time: 2014-08-13 16:24:25.198225, event: done, request: osd_sub_op_reply(unknown.0.0:0 22.6bd 0//0//22 [scrub-reserve] ack, result = 0) v2 -4907> 2014-08-13 16:24:25.198300 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:24.719801, event: header_read, request: pg_notify(22.39d(15) epoch 194450) v5 -4906> 2014-08-13 16:24:25.198314 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:24.719802, event: throttled, request: pg_notify(22.39d(15) epoch 194450) v5 -4905> 2014-08-13 16:24:25.198324 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:24.719888, event: all_read, request: pg_notify(22.39d(15) epoch 194450) v5 -4904> 2014-08-13 16:24:25.198334 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:25.198268, event: dispatched, request: pg_notify(22.39d(15) epoch 194450) v5 -4903> 2014-08-13 16:24:25.198344 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:25.198344, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194450) v5 -4902> 2014-08-13 16:24:25.198356 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:25.198355, event: started, request: pg_notify(22.39d(15) epoch 194450) v5 -4901> 2014-08-13 16:24:25.198398 7fd93dff0700 5 -- op tracker -- , seq: 880, time: 2014-08-13 16:24:25.198398, event: done, request: pg_notify(22.39d(15) epoch 194450) v5 -4900> 2014-08-13 16:24:25.198437 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 7 ==== pg_notify(24.1bb(21) epoch 194450) v5 ==== 1931+0+0 (3131560890 0 0) 0x899a4c0 con 0x5113de0 -4899> 2014-08-13 16:24:25.198460 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:24.710895, event: header_read, request: pg_notify(24.1bb(21) epoch 194450) v5 -4898> 2014-08-13 16:24:25.198471 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:24.710896, event: throttled, request: pg_notify(24.1bb(21) epoch 194450) v5 -4897> 2014-08-13 16:24:25.198480 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:24.711000, event: all_read, request: pg_notify(24.1bb(21) epoch 194450) v5 -4896> 2014-08-13 16:24:25.198489 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:25.198457, event: dispatched, request: pg_notify(24.1bb(21) epoch 194450) v5 -4895> 2014-08-13 16:24:25.198498 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:25.198498, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194450) v5 -4894> 2014-08-13 16:24:25.198509 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:25.198509, event: started, request: pg_notify(24.1bb(21) epoch 194450) v5 -4893> 2014-08-13 16:24:25.198542 7fd93dff0700 5 -- op tracker -- , seq: 881, time: 2014-08-13 16:24:25.198541, event: done, request: pg_notify(24.1bb(21) epoch 194450) v5 -4892> 2014-08-13 16:24:25.198571 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 17 ==== pg_notify(22.1d2(17) epoch 194450) v5 ==== 1669+0+0 (1380581768 0 0) 0x80d8c40 con 0x5113b20 -4891> 2014-08-13 16:24:25.198591 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:24.718483, event: header_read, request: pg_notify(22.1d2(17) epoch 194450) v5 -4890> 2014-08-13 16:24:25.198601 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:24.718484, event: throttled, request: pg_notify(22.1d2(17) epoch 194450) v5 -4889> 2014-08-13 16:24:25.198611 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:24.718568, event: all_read, request: pg_notify(22.1d2(17) epoch 194450) v5 -4888> 2014-08-13 16:24:25.198620 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:25.198588, event: dispatched, request: pg_notify(22.1d2(17) epoch 194450) v5 -4887> 2014-08-13 16:24:25.198629 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:25.198629, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194450) v5 -4886> 2014-08-13 16:24:25.198640 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:25.198640, event: started, request: pg_notify(22.1d2(17) epoch 194450) v5 -4885> 2014-08-13 16:24:25.198666 7fd93dff0700 5 -- op tracker -- , seq: 882, time: 2014-08-13 16:24:25.198666, event: done, request: pg_notify(22.1d2(17) epoch 194450) v5 -4884> 2014-08-13 16:24:25.198689 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 10 ==== pg_notify(24.b6(4) epoch 194450) v5 ==== 976+0+0 (394948466 0 0) 0x81c5400 con 0x5113860 -4883> 2014-08-13 16:24:25.198707 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:24.704628, event: header_read, request: pg_notify(24.b6(4) epoch 194450) v5 -4882> 2014-08-13 16:24:25.198718 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:24.704629, event: throttled, request: pg_notify(24.b6(4) epoch 194450) v5 -4881> 2014-08-13 16:24:25.198727 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:24.704719, event: all_read, request: pg_notify(24.b6(4) epoch 194450) v5 -4880> 2014-08-13 16:24:25.198736 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:25.198705, event: dispatched, request: pg_notify(24.b6(4) epoch 194450) v5 -4879> 2014-08-13 16:24:25.198746 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:25.198746, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194450) v5 -4878> 2014-08-13 16:24:25.198757 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:25.198756, event: started, request: pg_notify(24.b6(4) epoch 194450) v5 -4877> 2014-08-13 16:24:25.198782 7fd93dff0700 5 -- op tracker -- , seq: 883, time: 2014-08-13 16:24:25.198782, event: done, request: pg_notify(24.b6(4) epoch 194450) v5 -4876> 2014-08-13 16:24:25.198802 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 24 ==== pg_notify(22.3d9(6) epoch 194450) v5 ==== 1060+0+0 (3633041941 0 0) 0x9107a80 con 0x5111fa0 -4875> 2014-08-13 16:24:25.198820 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:24.715144, event: header_read, request: pg_notify(22.3d9(6) epoch 194450) v5 -4874> 2014-08-13 16:24:25.198831 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:24.715146, event: throttled, request: pg_notify(22.3d9(6) epoch 194450) v5 -4873> 2014-08-13 16:24:25.198840 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:24.715238, event: all_read, request: pg_notify(22.3d9(6) epoch 194450) v5 -4872> 2014-08-13 16:24:25.198848 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:25.198818, event: dispatched, request: pg_notify(22.3d9(6) epoch 194450) v5 -4871> 2014-08-13 16:24:25.198858 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:25.198858, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194450) v5 -4870> 2014-08-13 16:24:25.198869 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:25.198869, event: started, request: pg_notify(22.3d9(6) epoch 194450) v5 -4869> 2014-08-13 16:24:25.198894 7fd93dff0700 5 -- op tracker -- , seq: 884, time: 2014-08-13 16:24:25.198894, event: done, request: pg_notify(22.3d9(6) epoch 194450) v5 -4868> 2014-08-13 16:24:25.198914 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 28 ==== pg_notify(24.3d(18) epoch 194450) v5 ==== 1754+0+0 (1764117957 0 0) 0x9284ec0 con 0x51135a0 -4867> 2014-08-13 16:24:25.198932 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:24.695875, event: header_read, request: pg_notify(24.3d(18) epoch 194450) v5 -4866> 2014-08-13 16:24:25.198943 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:24.695876, event: throttled, request: pg_notify(24.3d(18) epoch 194450) v5 -4865> 2014-08-13 16:24:25.198952 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:24.696000, event: all_read, request: pg_notify(24.3d(18) epoch 194450) v5 -4864> 2014-08-13 16:24:25.198961 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:25.198930, event: dispatched, request: pg_notify(24.3d(18) epoch 194450) v5 -4863> 2014-08-13 16:24:25.198971 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:25.198971, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194450) v5 -4862> 2014-08-13 16:24:25.198982 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:25.198982, event: started, request: pg_notify(24.3d(18) epoch 194450) v5 -4861> 2014-08-13 16:24:25.199008 7fd93dff0700 5 -- op tracker -- , seq: 885, time: 2014-08-13 16:24:25.199008, event: done, request: pg_notify(24.3d(18) epoch 194450) v5 -4860> 2014-08-13 16:24:25.199041 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 20 ==== pg_notify(22.48e(15) epoch 194450) v5 ==== 1551+0+0 (1756728601 0 0) 0x87e4280 con 0x5111a20 -4859> 2014-08-13 16:24:25.199062 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:24.702590, event: header_read, request: pg_notify(22.48e(15) epoch 194450) v5 -4858> 2014-08-13 16:24:25.199072 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:24.702591, event: throttled, request: pg_notify(22.48e(15) epoch 194450) v5 -4857> 2014-08-13 16:24:25.199082 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:24.702692, event: all_read, request: pg_notify(22.48e(15) epoch 194450) v5 -4856> 2014-08-13 16:24:25.199090 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:25.199059, event: dispatched, request: pg_notify(22.48e(15) epoch 194450) v5 -4855> 2014-08-13 16:24:25.199100 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:25.199100, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194450) v5 -4854> 2014-08-13 16:24:25.199111 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:25.199111, event: started, request: pg_notify(22.48e(15) epoch 194450) v5 -4853> 2014-08-13 16:24:25.199101 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- replica scrub(pg: 22.6bd,from:0'0,to:194215'503,epoch:194450,start:0//0//-1,end:338abebd//0//-1,chunky:1,deep:1,version:5) v5 -- ?+0 0x8598b40 -4852> 2014-08-13 16:24:25.199187 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- replica scrub(pg: 22.6bd,from:0'0,to:194215'503,epoch:194450,start:0//0//-1,end:338abebd//0//-1,chunky:1,deep:1,version:5) v5 -- ?+0 0x859ad00 -4851> 2014-08-13 16:24:25.199137 7fd93dff0700 5 -- op tracker -- , seq: 886, time: 2014-08-13 16:24:25.199137, event: done, request: pg_notify(22.48e(15) epoch 194450) v5 -4850> 2014-08-13 16:24:25.199311 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 13 ==== pg_notify(22.362(17) epoch 194450) v5 ==== 1665+0+0 (3573334538 0 0) 0x80dcec0 con 0x7313860 -4849> 2014-08-13 16:24:25.199339 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:24.697030, event: header_read, request: pg_notify(22.362(17) epoch 194450) v5 -4848> 2014-08-13 16:24:25.199355 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:24.697032, event: throttled, request: pg_notify(22.362(17) epoch 194450) v5 -4847> 2014-08-13 16:24:25.199361 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:24.697137, event: all_read, request: pg_notify(22.362(17) epoch 194450) v5 -4846> 2014-08-13 16:24:25.199367 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:25.199336, event: dispatched, request: pg_notify(22.362(17) epoch 194450) v5 -4845> 2014-08-13 16:24:25.199373 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:25.199373, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194450) v5 -4844> 2014-08-13 16:24:25.199382 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:25.199382, event: started, request: pg_notify(22.362(17) epoch 194450) v5 -4843> 2014-08-13 16:24:25.199409 7fd93dff0700 5 -- op tracker -- , seq: 887, time: 2014-08-13 16:24:25.199409, event: done, request: pg_notify(22.362(17) epoch 194450) v5 -4842> 2014-08-13 16:24:25.199435 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 8 ==== pg_notify(22.22c(17) epoch 194450) v5 ==== 1665+0+0 (1647684192 0 0) 0x8c8bf00 con 0x7316720 -4841> 2014-08-13 16:24:25.199456 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:24.845518, event: header_read, request: pg_notify(22.22c(17) epoch 194450) v5 -4840> 2014-08-13 16:24:25.199463 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:24.845519, event: throttled, request: pg_notify(22.22c(17) epoch 194450) v5 -4839> 2014-08-13 16:24:25.199469 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:24.845608, event: all_read, request: pg_notify(22.22c(17) epoch 194450) v5 -4838> 2014-08-13 16:24:25.199475 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:25.199454, event: dispatched, request: pg_notify(22.22c(17) epoch 194450) v5 -4837> 2014-08-13 16:24:25.199484 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:25.199484, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194450) v5 -4836> 2014-08-13 16:24:25.199493 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:25.199492, event: started, request: pg_notify(22.22c(17) epoch 194450) v5 -4835> 2014-08-13 16:24:25.199519 7fd93dff0700 5 -- op tracker -- , seq: 888, time: 2014-08-13 16:24:25.199518, event: done, request: pg_notify(22.22c(17) epoch 194450) v5 -4834> 2014-08-13 16:24:25.199542 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 9 ==== pg_notify(22.707(19) epoch 194450) v5 ==== 1783+0+0 (3042054600 0 0) 0x8c8d400 con 0x7314fc0 -4833> 2014-08-13 16:24:25.199561 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:24.718828, event: header_read, request: pg_notify(22.707(19) epoch 194450) v5 -4832> 2014-08-13 16:24:25.199568 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:24.718830, event: throttled, request: pg_notify(22.707(19) epoch 194450) v5 -4831> 2014-08-13 16:24:25.199573 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:24.718921, event: all_read, request: pg_notify(22.707(19) epoch 194450) v5 -4830> 2014-08-13 16:24:25.199578 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:25.199559, event: dispatched, request: pg_notify(22.707(19) epoch 194450) v5 -4829> 2014-08-13 16:24:25.199584 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:25.199584, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194450) v5 -4828> 2014-08-13 16:24:25.199591 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:25.199591, event: started, request: pg_notify(22.707(19) epoch 194450) v5 -4827> 2014-08-13 16:24:25.199613 7fd93dff0700 5 -- op tracker -- , seq: 889, time: 2014-08-13 16:24:25.199613, event: done, request: pg_notify(22.707(19) epoch 194450) v5 -4826> 2014-08-13 16:24:25.199644 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 9 ==== pg_notify(22.39d(15) epoch 194450) v5 ==== 1551+0+0 (4097317751 0 0) 0x8c8dcc0 con 0x72d9b80 -4825> 2014-08-13 16:24:25.199663 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:24.730909, event: header_read, request: pg_notify(22.39d(15) epoch 194450) v5 -4824> 2014-08-13 16:24:25.199670 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:24.730911, event: throttled, request: pg_notify(22.39d(15) epoch 194450) v5 -4823> 2014-08-13 16:24:25.199675 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:24.731012, event: all_read, request: pg_notify(22.39d(15) epoch 194450) v5 -4822> 2014-08-13 16:24:25.199681 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:25.199660, event: dispatched, request: pg_notify(22.39d(15) epoch 194450) v5 -4821> 2014-08-13 16:24:25.199687 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:25.199686, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194450) v5 -4820> 2014-08-13 16:24:25.199694 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:25.199694, event: started, request: pg_notify(22.39d(15) epoch 194450) v5 -4819> 2014-08-13 16:24:25.199716 7fd93dff0700 5 -- op tracker -- , seq: 890, time: 2014-08-13 16:24:25.199715, event: done, request: pg_notify(22.39d(15) epoch 194450) v5 -4818> 2014-08-13 16:24:25.199737 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 11 ==== pg_notify(22.f3(15) epoch 194450) v5 ==== 1551+0+0 (1413753511 0 0) 0x9107e00 con 0x8e286e0 -4817> 2014-08-13 16:24:25.199755 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:24.717293, event: header_read, request: pg_notify(22.f3(15) epoch 194450) v5 -4816> 2014-08-13 16:24:25.199762 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:24.717294, event: throttled, request: pg_notify(22.f3(15) epoch 194450) v5 -4815> 2014-08-13 16:24:25.199768 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:24.717372, event: all_read, request: pg_notify(22.f3(15) epoch 194450) v5 -4814> 2014-08-13 16:24:25.199777 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:25.199753, event: dispatched, request: pg_notify(22.f3(15) epoch 194450) v5 -4813> 2014-08-13 16:24:25.199783 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:25.199783, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194450) v5 -4812> 2014-08-13 16:24:25.199791 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:25.199790, event: started, request: pg_notify(22.f3(15) epoch 194450) v5 -4811> 2014-08-13 16:24:25.199828 7fd93dff0700 5 -- op tracker -- , seq: 891, time: 2014-08-13 16:24:25.199828, event: done, request: pg_notify(22.f3(15) epoch 194450) v5 -4810> 2014-08-13 16:24:25.199850 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 10 ==== pg_notify(22.481(27) epoch 194450) v5 ==== 2235+0+0 (3122486035 0 0) 0x8999500 con 0x7723440 -4809> 2014-08-13 16:24:25.199866 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:24.708881, event: header_read, request: pg_notify(22.481(27) epoch 194450) v5 -4808> 2014-08-13 16:24:25.199873 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:24.708883, event: throttled, request: pg_notify(22.481(27) epoch 194450) v5 -4807> 2014-08-13 16:24:25.199878 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:24.708996, event: all_read, request: pg_notify(22.481(27) epoch 194450) v5 -4806> 2014-08-13 16:24:25.199884 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:25.199864, event: dispatched, request: pg_notify(22.481(27) epoch 194450) v5 -4805> 2014-08-13 16:24:25.199898 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:25.199898, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194450) v5 -4804> 2014-08-13 16:24:25.199910 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:25.199909, event: started, request: pg_notify(22.481(27) epoch 194450) v5 -4803> 2014-08-13 16:24:25.199936 7fd93dff0700 5 -- op tracker -- , seq: 892, time: 2014-08-13 16:24:25.199936, event: done, request: pg_notify(22.481(27) epoch 194450) v5 -4802> 2014-08-13 16:24:25.199965 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 20 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (752458914 0 0) 0x86dc200 con 0x4e4f900 -4801> 2014-08-13 16:24:25.200000 7fd93dff0700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.189272, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4800> 2014-08-13 16:24:25.200015 7fd93dff0700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.189275, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4799> 2014-08-13 16:24:25.200028 7fd93dff0700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.189390, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4798> 2014-08-13 16:24:25.200041 7fd93dff0700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.199997, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4797> 2014-08-13 16:24:25.200055 7fd93dff0700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.200054, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4796> 2014-08-13 16:24:25.200088 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 16 ==== pg_notify(22.362(4) epoch 194450) v5 ==== 942+0+0 (348953318 0 0) 0x80dc600 con 0x4fe40a0 -4795> 2014-08-13 16:24:25.200107 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:24.689317, event: header_read, request: pg_notify(22.362(4) epoch 194450) v5 -4794> 2014-08-13 16:24:25.200118 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:24.689321, event: throttled, request: pg_notify(22.362(4) epoch 194450) v5 -4793> 2014-08-13 16:24:25.200127 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:24.689423, event: all_read, request: pg_notify(22.362(4) epoch 194450) v5 -4792> 2014-08-13 16:24:25.200135 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:25.200105, event: dispatched, request: pg_notify(22.362(4) epoch 194450) v5 -4791> 2014-08-13 16:24:25.200159 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:25.200145, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194450) v5 -4790> 2014-08-13 16:24:25.200176 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:25.200176, event: started, request: pg_notify(22.362(4) epoch 194450) v5 -4789> 2014-08-13 16:24:25.200205 7fd9371e5700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.200151, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4788> 2014-08-13 16:24:25.200250 7fd93dff0700 5 -- op tracker -- , seq: 894, time: 2014-08-13 16:24:25.200208, event: done, request: pg_notify(22.362(4) epoch 194450) v5 -4787> 2014-08-13 16:24:25.200284 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 19 ==== pg_notify(24.181(9) epoch 194450) v5 ==== 1263+0+0 (3261147600 0 0) 0x80ddb00 con 0x50889a0 -4786> 2014-08-13 16:24:25.200282 7fd9371e5700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.200254, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4785> 2014-08-13 16:24:25.200324 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:24.705524, event: header_read, request: pg_notify(24.181(9) epoch 194450) v5 -4784> 2014-08-13 16:24:25.200340 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:24.705529, event: throttled, request: pg_notify(24.181(9) epoch 194450) v5 -4783> 2014-08-13 16:24:25.200353 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:24.705655, event: all_read, request: pg_notify(24.181(9) epoch 194450) v5 -4782> 2014-08-13 16:24:25.200366 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:25.200307, event: dispatched, request: pg_notify(24.181(9) epoch 194450) v5 -4781> 2014-08-13 16:24:25.200379 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:25.200379, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194450) v5 -4780> 2014-08-13 16:24:25.200394 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:25.200394, event: started, request: pg_notify(24.181(9) epoch 194450) v5 -4779> 2014-08-13 16:24:25.200410 7fd9371e5700 5 -- op tracker -- , seq: 893, time: 2014-08-13 16:24:25.200324, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4778> 2014-08-13 16:24:25.200427 7fd93dff0700 5 -- op tracker -- , seq: 895, time: 2014-08-13 16:24:25.200427, event: done, request: pg_notify(24.181(9) epoch 194450) v5 -4777> 2014-08-13 16:24:25.200456 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 14 ==== pg_notify(22.3e4(4) epoch 194450) v5 ==== 942+0+0 (1914361578 0 0) 0x80df000 con 0x5108b00 -4776> 2014-08-13 16:24:25.200477 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:24.841394, event: header_read, request: pg_notify(22.3e4(4) epoch 194450) v5 -4775> 2014-08-13 16:24:25.200487 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:24.841397, event: throttled, request: pg_notify(22.3e4(4) epoch 194450) v5 -4774> 2014-08-13 16:24:25.200496 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:24.841503, event: all_read, request: pg_notify(22.3e4(4) epoch 194450) v5 -4773> 2014-08-13 16:24:25.200505 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:25.200474, event: dispatched, request: pg_notify(22.3e4(4) epoch 194450) v5 -4772> 2014-08-13 16:24:25.200515 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:25.200515, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194450) v5 -4771> 2014-08-13 16:24:25.200526 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:25.200526, event: started, request: pg_notify(22.3e4(4) epoch 194450) v5 -4770> 2014-08-13 16:24:25.200552 7fd93dff0700 5 -- op tracker -- , seq: 896, time: 2014-08-13 16:24:25.200551, event: done, request: pg_notify(22.3e4(4) epoch 194450) v5 -4769> 2014-08-13 16:24:25.200571 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 3 ==== pg_notify(24.39a(47) epoch 194450) v5 ==== 3273+0+0 (2819770588 0 0) 0x7d06580 con 0x51127e0 -4768> 2014-08-13 16:24:25.200590 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.195717, event: header_read, request: pg_notify(24.39a(47) epoch 194450) v5 -4767> 2014-08-13 16:24:25.200600 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.195720, event: throttled, request: pg_notify(24.39a(47) epoch 194450) v5 -4766> 2014-08-13 16:24:25.200609 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.195886, event: all_read, request: pg_notify(24.39a(47) epoch 194450) v5 -4765> 2014-08-13 16:24:25.200618 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.200588, event: dispatched, request: pg_notify(24.39a(47) epoch 194450) v5 -4764> 2014-08-13 16:24:25.200627 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.200627, event: waiting_for_osdmap, request: pg_notify(24.39a(47) epoch 194450) v5 -4763> 2014-08-13 16:24:25.200639 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.200638, event: started, request: pg_notify(24.39a(47) epoch 194450) v5 -4762> 2014-08-13 16:24:25.200665 7fd93dff0700 5 -- op tracker -- , seq: 897, time: 2014-08-13 16:24:25.200665, event: done, request: pg_notify(24.39a(47) epoch 194450) v5 -4761> 2014-08-13 16:24:25.200706 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 11 ==== pg_notify(22.48e(15) epoch 194450) v5 ==== 1551+0+0 (750573134 0 0) 0x81c7380 con 0x5113860 -4760> 2014-08-13 16:24:25.200727 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:24.718211, event: header_read, request: pg_notify(22.48e(15) epoch 194450) v5 -4759> 2014-08-13 16:24:25.200737 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:24.718213, event: throttled, request: pg_notify(22.48e(15) epoch 194450) v5 -4758> 2014-08-13 16:24:25.200746 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:24.718310, event: all_read, request: pg_notify(22.48e(15) epoch 194450) v5 -4757> 2014-08-13 16:24:25.200755 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:25.200724, event: dispatched, request: pg_notify(22.48e(15) epoch 194450) v5 -4756> 2014-08-13 16:24:25.200765 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:25.200765, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194450) v5 -4755> 2014-08-13 16:24:25.200776 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:25.200775, event: started, request: pg_notify(22.48e(15) epoch 194450) v5 -4754> 2014-08-13 16:24:25.200804 7fd93dff0700 5 -- op tracker -- , seq: 898, time: 2014-08-13 16:24:25.200803, event: done, request: pg_notify(22.48e(15) epoch 194450) v5 -4753> 2014-08-13 16:24:25.200826 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 29 ==== pg_notify(22.3d9(32) epoch 194450) v5 ==== 2458+0+0 (1667047773 0 0) 0x92871c0 con 0x51135a0 -4752> 2014-08-13 16:24:25.200846 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:24.696385, event: header_read, request: pg_notify(22.3d9(32) epoch 194450) v5 -4751> 2014-08-13 16:24:25.200856 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:24.696388, event: throttled, request: pg_notify(22.3d9(32) epoch 194450) v5 -4750> 2014-08-13 16:24:25.200865 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:24.696527, event: all_read, request: pg_notify(22.3d9(32) epoch 194450) v5 -4749> 2014-08-13 16:24:25.200873 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:25.200843, event: dispatched, request: pg_notify(22.3d9(32) epoch 194450) v5 -4748> 2014-08-13 16:24:25.200883 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:25.200883, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194450) v5 -4747> 2014-08-13 16:24:25.200894 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:25.200894, event: started, request: pg_notify(22.3d9(32) epoch 194450) v5 -4746> 2014-08-13 16:24:25.200919 7fd93dff0700 5 -- op tracker -- , seq: 899, time: 2014-08-13 16:24:25.200919, event: done, request: pg_notify(22.3d9(32) epoch 194450) v5 -4745> 2014-08-13 16:24:25.200940 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 21 ==== pg_notify(24.1f7(18) epoch 194450) v5 ==== 1754+0+0 (3998581369 0 0) 0x87e3f00 con 0x5111a20 -4744> 2014-08-13 16:24:25.200959 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:24.703425, event: header_read, request: pg_notify(24.1f7(18) epoch 194450) v5 -4743> 2014-08-13 16:24:25.200968 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:24.703429, event: throttled, request: pg_notify(24.1f7(18) epoch 194450) v5 -4742> 2014-08-13 16:24:25.200977 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:24.703563, event: all_read, request: pg_notify(24.1f7(18) epoch 194450) v5 -4741> 2014-08-13 16:24:25.200984 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:25.200956, event: dispatched, request: pg_notify(24.1f7(18) epoch 194450) v5 -4740> 2014-08-13 16:24:25.200992 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:25.200992, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194450) v5 -4739> 2014-08-13 16:24:25.201003 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:25.201003, event: started, request: pg_notify(24.1f7(18) epoch 194450) v5 -4738> 2014-08-13 16:24:25.201029 7fd93dff0700 5 -- op tracker -- , seq: 900, time: 2014-08-13 16:24:25.201029, event: done, request: pg_notify(24.1f7(18) epoch 194450) v5 -4737> 2014-08-13 16:24:25.201056 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 14 ==== pg_notify(24.39a(41) epoch 194450) v5 ==== 2987+0+0 (2021156241 0 0) 0x8c89dc0 con 0x7313860 -4736> 2014-08-13 16:24:25.201075 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:24.698583, event: header_read, request: pg_notify(24.39a(41) epoch 194450) v5 -4735> 2014-08-13 16:24:25.201085 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:24.698585, event: throttled, request: pg_notify(24.39a(41) epoch 194450) v5 -4734> 2014-08-13 16:24:25.201095 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:24.698762, event: all_read, request: pg_notify(24.39a(41) epoch 194450) v5 -4733> 2014-08-13 16:24:25.201103 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:25.201073, event: dispatched, request: pg_notify(24.39a(41) epoch 194450) v5 -4732> 2014-08-13 16:24:25.201113 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:25.201113, event: waiting_for_osdmap, request: pg_notify(24.39a(41) epoch 194450) v5 -4731> 2014-08-13 16:24:25.201124 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:25.201123, event: started, request: pg_notify(24.39a(41) epoch 194450) v5 -4730> 2014-08-13 16:24:25.201160 7fd93dff0700 5 -- op tracker -- , seq: 901, time: 2014-08-13 16:24:25.201160, event: done, request: pg_notify(24.39a(41) epoch 194450) v5 -4729> 2014-08-13 16:24:25.201186 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 9 ==== pg_notify(22.1f1(17) epoch 194450) v5 ==== 1665+0+0 (1698113794 0 0) 0x8c8bd40 con 0x7316720 -4728> 2014-08-13 16:24:25.201206 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:24.846265, event: header_read, request: pg_notify(22.1f1(17) epoch 194450) v5 -4727> 2014-08-13 16:24:25.201188 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/GetInfo 0.010843 6 0.000273 -4726> 2014-08-13 16:24:25.201217 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:24.846269, event: throttled, request: pg_notify(22.1f1(17) epoch 194450) v5 -4725> 2014-08-13 16:24:25.201226 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:24.846385, event: all_read, request: pg_notify(22.1f1(17) epoch 194450) v5 -4724> 2014-08-13 16:24:25.201235 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:25.201204, event: dispatched, request: pg_notify(22.1f1(17) epoch 194450) v5 -4723> 2014-08-13 16:24:25.201225 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/GetLog -4722> 2014-08-13 16:24:25.201245 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:25.201244, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194450) v5 -4721> 2014-08-13 16:24:25.201253 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:25.201252, event: started, request: pg_notify(22.1f1(17) epoch 194450) v5 -4720> 2014-08-13 16:24:25.201279 7fd93dff0700 5 -- op tracker -- , seq: 902, time: 2014-08-13 16:24:25.201278, event: done, request: pg_notify(22.1f1(17) epoch 194450) v5 -4719> 2014-08-13 16:24:25.201299 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4718> 2014-08-13 16:24:25.201299 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 12 ==== pg_notify(22.775(4) epoch 194450) v5 ==== 942+0+0 (2314511153 0 0) 0x9104ec0 con 0x8e286e0 -4717> 2014-08-13 16:24:25.201304 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/GetLog 0.000079 0 0.000000 -4716> 2014-08-13 16:24:25.201321 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194450 {24.39a=[4,395,13]} v194450) v1 -- ?+0 0x8019680 con 0x4e3dd80 -4715> 2014-08-13 16:24:25.201334 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:24.717940, event: header_read, request: pg_notify(22.775(4) epoch 194450) v5 -4714> 2014-08-13 16:24:25.201343 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:24.717941, event: throttled, request: pg_notify(22.775(4) epoch 194450) v5 -4713> 2014-08-13 16:24:25.201338 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering 0.011016 0 0.000000 -4712> 2014-08-13 16:24:25.201352 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:24.718012, event: all_read, request: pg_notify(22.775(4) epoch 194450) v5 -4711> 2014-08-13 16:24:25.201374 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:25.201330, event: dispatched, request: pg_notify(22.775(4) epoch 194450) v5 -4710> 2014-08-13 16:24:25.201370 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Started/Primary 0.011069 0 0.000000 -4709> 2014-08-13 16:24:25.201386 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:25.201385, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194450) v5 -4708> 2014-08-13 16:24:25.201397 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:25.201397, event: started, request: pg_notify(22.775(4) epoch 194450) v5 -4707> 2014-08-13 16:24:25.201392 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary -4706> 2014-08-13 16:24:25.201412 7fd9371e5700 5 osd.4 pg_epoch: 194450 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary/Peering/WaitActingChange -4705> 2014-08-13 16:24:25.201423 7fd93dff0700 5 -- op tracker -- , seq: 903, time: 2014-08-13 16:24:25.201423, event: done, request: pg_notify(22.775(4) epoch 194450) v5 -4704> 2014-08-13 16:24:25.201448 7fd9371e5700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4703> 2014-08-13 16:24:25.201454 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 15 ==== pg_notify(22.481(27) epoch 194450) v5 ==== 2235+0+0 (4130580896 0 0) 0x80dee40 con 0x5108b00 -4702> 2014-08-13 16:24:25.201463 7fd9371e5700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194450 {24.39a=[4,395,13]} v194450) v1 -- ?+0 0x8019c20 con 0x4e3dd80 -4701> 2014-08-13 16:24:25.201475 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:24.842713, event: header_read, request: pg_notify(22.481(27) epoch 194450) v5 -4700> 2014-08-13 16:24:25.201503 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4699> 2014-08-13 16:24:25.201513 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:24.842716, event: throttled, request: pg_notify(22.481(27) epoch 194450) v5 -4698> 2014-08-13 16:24:25.201540 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:24.842873, event: all_read, request: pg_notify(22.481(27) epoch 194450) v5 -4697> 2014-08-13 16:24:25.201532 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194450 {24.39a=[4,395,13]} v194450) v1 -- ?+0 0x801f620 con 0x4e3dd80 -4696> 2014-08-13 16:24:25.201555 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:25.201473, event: dispatched, request: pg_notify(22.481(27) epoch 194450) v5 -4695> 2014-08-13 16:24:25.201585 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:25.201585, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194450) v5 -4694> 2014-08-13 16:24:25.201600 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:25.201600, event: started, request: pg_notify(22.481(27) epoch 194450) v5 -4693> 2014-08-13 16:24:25.201650 7fd93dff0700 5 -- op tracker -- , seq: 904, time: 2014-08-13 16:24:25.201649, event: done, request: pg_notify(22.481(27) epoch 194450) v5 -4692> 2014-08-13 16:24:25.201670 7fd9371e5700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4691> 2014-08-13 16:24:25.201676 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 15 ==== pg_notify(24.39a(41) epoch 194450) v5 ==== 2987+0+0 (2021156241 0 0) 0x8c89c00 con 0x7313860 -4690> 2014-08-13 16:24:25.201683 7fd9371e5700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194450 {24.39a=[4,395,13]} v194450) v1 -- ?+0 0x801a1c0 con 0x4e3dd80 -4689> 2014-08-13 16:24:25.201701 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.195693, event: header_read, request: pg_notify(24.39a(41) epoch 194450) v5 -4688> 2014-08-13 16:24:25.201712 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.195696, event: throttled, request: pg_notify(24.39a(41) epoch 194450) v5 -4687> 2014-08-13 16:24:25.201722 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.195853, event: all_read, request: pg_notify(24.39a(41) epoch 194450) v5 -4686> 2014-08-13 16:24:25.201731 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.201698, event: dispatched, request: pg_notify(24.39a(41) epoch 194450) v5 -4685> 2014-08-13 16:24:25.201740 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.201740, event: waiting_for_osdmap, request: pg_notify(24.39a(41) epoch 194450) v5 -4684> 2014-08-13 16:24:25.201752 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.201751, event: started, request: pg_notify(24.39a(41) epoch 194450) v5 -4683> 2014-08-13 16:24:25.201778 7fd93dff0700 5 -- op tracker -- , seq: 905, time: 2014-08-13 16:24:25.201778, event: done, request: pg_notify(24.39a(41) epoch 194450) v5 -4682> 2014-08-13 16:24:25.201855 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4681> 2014-08-13 16:24:25.201880 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194450 {24.39a=[4,395,13]} v194450) v1 -- ?+0 0x801a940 con 0x4e3dd80 -4680> 2014-08-13 16:24:25.267766 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.267702) v2 ==== 47+0+0 (743776422 0 0) 0x81c2680 con 0x7ebd6a0 -4679> 2014-08-13 16:24:25.267776 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 6 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.267702) v2 ==== 47+0+0 (743776422 0 0) 0x87e4440 con 0x7ebc0a0 -4678> 2014-08-13 16:24:25.267817 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.267702) v2 -- ?+0 0x8910c40 con 0x7ebd6a0 -4677> 2014-08-13 16:24:25.267868 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.267702) v2 -- ?+0 0x92863c0 con 0x7ebc0a0 -4676> 2014-08-13 16:24:25.335110 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.15 192.168.1.107:0/53408 9 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.334747) v2 ==== 47+0+0 (219990201 0 0) 0x9281f80 con 0x7ebd3e0 -4675> 2014-08-13 16:24:25.335165 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.15 192.168.1.107:0/53408 9 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.334747) v2 ==== 47+0+0 (219990201 0 0) 0x8575e80 con 0x7ebbde0 -4674> 2014-08-13 16:24:25.335206 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.334747) v2 -- ?+0 0x81c2680 con 0x7ebd3e0 -4673> 2014-08-13 16:24:25.335363 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.334747) v2 -- ?+0 0x87e4440 con 0x7ebbde0 -4672> 2014-08-13 16:24:25.394304 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.393529) v2 ==== 47+0+0 (3251041798 0 0) 0x9286e40 con 0x7d2c360 -4671> 2014-08-13 16:24:25.394357 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.393529) v2 -- ?+0 0x9281f80 con 0x7d2c360 -4670> 2014-08-13 16:24:25.394345 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.393529) v2 ==== 47+0+0 (3251041798 0 0) 0x7d06740 con 0x7d2ac00 -4669> 2014-08-13 16:24:25.394388 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.393529) v2 -- ?+0 0x8575e80 con 0x7d2ac00 -4668> 2014-08-13 16:24:25.423352 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.422774) v2 ==== 47+0+0 (1601138379 0 0) 0x840f700 con 0x7ab9ce0 -4667> 2014-08-13 16:24:25.423402 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.422774) v2 -- ?+0 0x9286e40 con 0x7ab9ce0 -4666> 2014-08-13 16:24:25.423433 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.422774) v2 ==== 47+0+0 (1601138379 0 0) 0x840fc40 con 0x7ab9a20 -4665> 2014-08-13 16:24:25.423513 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.422774) v2 -- ?+0 0x7d06740 con 0x7ab9a20 -4664> 2014-08-13 16:24:25.546570 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 30 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2415431945 0 0) 0x86db700 con 0x5112520 -4663> 2014-08-13 16:24:25.546623 7fd93dff0700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546361, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4662> 2014-08-13 16:24:25.546640 7fd93dff0700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546365, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4661> 2014-08-13 16:24:25.546649 7fd93dff0700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546486, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4660> 2014-08-13 16:24:25.546659 7fd93dff0700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546617, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4659> 2014-08-13 16:24:25.546680 7fd93dff0700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546680, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4658> 2014-08-13 16:24:25.546787 7fd9371e5700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546787, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4657> 2014-08-13 16:24:25.546828 7fd9371e5700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546828, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4656> 2014-08-13 16:24:25.546855 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d45780 con 0x5112520 -4655> 2014-08-13 16:24:25.546887 7fd9371e5700 5 -- op tracker -- , seq: 906, time: 2014-08-13 16:24:25.546887, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4654> 2014-08-13 16:24:25.548141 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 31 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (514409132 0 0) 0x8bc3700 con 0x5112520 -4653> 2014-08-13 16:24:25.548204 7fd93dff0700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.547952, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4652> 2014-08-13 16:24:25.548220 7fd93dff0700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.547955, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4651> 2014-08-13 16:24:25.548230 7fd93dff0700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548061, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4650> 2014-08-13 16:24:25.548245 7fd93dff0700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548197, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4649> 2014-08-13 16:24:25.548262 7fd93dff0700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548262, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4648> 2014-08-13 16:24:25.548331 7fd9367e4700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548331, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4647> 2014-08-13 16:24:25.548377 7fd9367e4700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548376, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4646> 2014-08-13 16:24:25.548392 7fd9367e4700 5 -- op tracker -- , seq: 907, time: 2014-08-13 16:24:25.548392, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4645> 2014-08-13 16:24:25.565025 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.564799) v2 ==== 47+0+0 (3651757998 0 0) 0x85763c0 con 0x7cae880 -4644> 2014-08-13 16:24:25.565050 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 8 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.564799) v2 ==== 47+0+0 (3651757998 0 0) 0x775d780 con 0x7cae040 -4643> 2014-08-13 16:24:25.565072 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.564799) v2 -- ?+0 0x840fc40 con 0x7cae880 -4642> 2014-08-13 16:24:25.565130 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.564799) v2 -- ?+0 0x840f700 con 0x7cae040 -4641> 2014-08-13 16:24:25.635471 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 21 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3179826951 0 0) 0x76fb700 con 0x4e4f900 -4640> 2014-08-13 16:24:25.635523 7fd93dff0700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635273, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4639> 2014-08-13 16:24:25.635539 7fd93dff0700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635277, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4638> 2014-08-13 16:24:25.635549 7fd93dff0700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635387, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4637> 2014-08-13 16:24:25.635558 7fd93dff0700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635516, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4636> 2014-08-13 16:24:25.635574 7fd93dff0700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635574, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4635> 2014-08-13 16:24:25.635674 7fd9371e5700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635673, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4634> 2014-08-13 16:24:25.635714 7fd9371e5700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635714, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4633> 2014-08-13 16:24:25.635733 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d45500 con 0x4e4f900 -4632> 2014-08-13 16:24:25.635762 7fd9371e5700 5 -- op tracker -- , seq: 908, time: 2014-08-13 16:24:25.635761, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4631> 2014-08-13 16:24:25.637120 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 22 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (752458914 0 0) 0x8bc4200 con 0x4e4f900 -4630> 2014-08-13 16:24:25.637184 7fd93dff0700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.636939, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4629> 2014-08-13 16:24:25.637200 7fd93dff0700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.636942, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4628> 2014-08-13 16:24:25.637210 7fd93dff0700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637038, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4627> 2014-08-13 16:24:25.637223 7fd93dff0700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637177, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4626> 2014-08-13 16:24:25.637238 7fd93dff0700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637238, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4625> 2014-08-13 16:24:25.637337 7fd9367e4700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637337, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4624> 2014-08-13 16:24:25.637378 7fd9367e4700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637378, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4623> 2014-08-13 16:24:25.637393 7fd9367e4700 5 -- op tracker -- , seq: 909, time: 2014-08-13 16:24:25.637393, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -4622> 2014-08-13 16:24:25.648746 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.334 192.168.1.104:0/62751 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.648770) v2 ==== 47+0+0 (3535075126 0 0) 0x8c88000 con 0x7caf0c0 -4621> 2014-08-13 16:24:25.648762 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.334 192.168.1.104:0/62751 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:25.648770) v2 ==== 47+0+0 (3535075126 0 0) 0x8c8d940 con 0x7d286e0 -4620> 2014-08-13 16:24:25.648798 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.648770) v2 -- ?+0 0x775d780 con 0x7caf0c0 -4619> 2014-08-13 16:24:25.648852 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.648770) v2 -- ?+0 0x85763c0 con 0x7d286e0 -4618> 2014-08-13 16:24:25.690080 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 3 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf4140 con 0x7d2d120 -4617> 2014-08-13 16:24:25.690767 7fd93dff0700 3 osd.4 194450 handle_osd_map epochs [194451,194451], i have 194450, src has [185139,194451] -4616> 2014-08-13 16:24:25.696819 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 25 ==== osd_map(194450..194451 src has 185139..194451) v3 ==== 1774+0+0 (2937653276 0 0) 0x5077980 con 0x4e3dd80 -4615> 2014-08-13 16:24:25.698262 7fd93dff0700 0 osd.4 194451 crush map has features 2578087936000, adjusting msgr requires for mons -4614> 2014-08-13 16:24:25.769615 7fd933fe0700 5 osd.4 194451 heartbeat: osd_stat(8081 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,7,10,11,15,17,20,21,24,25,28,31,32,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,396,397]/[] op hist []) -4613> 2014-08-13 16:24:25.769672 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x7ed3100 con 0x8c391e0 -4612> 2014-08-13 16:24:25.769706 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x7ed2f40 con 0x8c38f20 -4611> 2014-08-13 16:24:25.769732 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83ca140 con 0x78844c0 -4610> 2014-08-13 16:24:25.769750 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9f80 con 0x7884200 -4609> 2014-08-13 16:24:25.769767 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9dc0 con 0x78848e0 -4608> 2014-08-13 16:24:25.769783 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9c00 con 0x7884620 -4607> 2014-08-13 16:24:25.769807 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6839/62713 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9a40 con 0x8ba06e0 -4606> 2014-08-13 16:24:25.769829 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6846/62713 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9880 con 0x8c698c0 -4605> 2014-08-13 16:24:25.769845 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c8380 con 0x794c360 -4604> 2014-08-13 16:24:25.769864 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c9340 con 0x794e300 -4603> 2014-08-13 16:24:25.769885 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x83c8e00 con 0x794e040 -4602> 2014-08-13 16:24:25.769904 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79be200 con 0x794dd80 -4601> 2014-08-13 16:24:25.769930 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bbd40 con 0x8c3a3c0 -4600> 2014-08-13 16:24:25.769947 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79be740 con 0x8c3a100 -4599> 2014-08-13 16:24:25.769964 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b88c0 con 0x8c3c0a0 -4598> 2014-08-13 16:24:25.769992 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bc280 con 0x8c3bde0 -4597> 2014-08-13 16:24:25.770017 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bc7c0 con 0x8babde0 -4596> 2014-08-13 16:24:25.770035 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bb800 con 0x8babb20 -4595> 2014-08-13 16:24:25.770052 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b8e00 con 0x8c3d6a0 -4594> 2014-08-13 16:24:25.770068 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bf1c0 con 0x8c3d3e0 -4593> 2014-08-13 16:24:25.770086 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9880 con 0x8badac0 -4592> 2014-08-13 16:24:25.770105 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9dc0 con 0x8bad800 -4591> 2014-08-13 16:24:25.770121 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79ba840 con 0x8bab860 -4590> 2014-08-13 16:24:25.770137 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bd240 con 0x8bab5a0 -4589> 2014-08-13 16:24:25.770154 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bd780 con 0x8c69600 -4588> 2014-08-13 16:24:25.770181 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bec80 con 0x8c69340 -4587> 2014-08-13 16:24:25.770205 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bad80 con 0x8baee00 -4586> 2014-08-13 16:24:25.770225 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79ba300 con 0x8ba1a20 -4585> 2014-08-13 16:24:25.770242 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6814/59915 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bfe00 con 0x8d21e40 -4584> 2014-08-13 16:24:25.770259 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6831/59915 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bfc40 con 0x8d21b80 -4583> 2014-08-13 16:24:25.770281 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bf700 con 0x7fba3c0 -4582> 2014-08-13 16:24:25.770299 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bcd00 con 0x7fba260 -4581> 2014-08-13 16:24:25.770315 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bb2c0 con 0x8bad540 -4580> 2014-08-13 16:24:25.770331 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9340 con 0x8bad280 -4579> 2014-08-13 16:24:25.770349 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9107000 con 0x8c3cba0 -4578> 2014-08-13 16:24:25.770368 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91024c0 con 0x8c3c8e0 -4577> 2014-08-13 16:24:25.770387 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9105e80 con 0x7fba100 -4576> 2014-08-13 16:24:25.770410 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9105400 con 0x7fb9fa0 -4575> 2014-08-13 16:24:25.770427 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91063c0 con 0x8d223c0 -4574> 2014-08-13 16:24:25.770449 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9106200 con 0x8d22100 -4573> 2014-08-13 16:24:25.770467 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x81c3d40 con 0x8c3c620 -4572> 2014-08-13 16:24:25.770482 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x81c7000 con 0x8c3c360 -4571> 2014-08-13 16:24:25.770504 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9880 con 0x8bafe80 -4570> 2014-08-13 16:24:25.770523 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9dc0 con 0x8bafbc0 -4569> 2014-08-13 16:24:25.770539 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79ba840 con 0x8c38c60 -4568> 2014-08-13 16:24:25.770561 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bd240 con 0x8c389a0 -4567> 2014-08-13 16:24:25.770579 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bec80 con 0x8b68c60 -4566> 2014-08-13 16:24:25.770598 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bad80 con 0x8b68b00 -4565> 2014-08-13 16:24:25.770614 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bfc40 con 0x8c6fe80 -4564> 2014-08-13 16:24:25.770632 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bf700 con 0x8c6fbc0 -4563> 2014-08-13 16:24:25.770655 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bd780 con 0x8b689a0 -4562> 2014-08-13 16:24:25.770674 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bcd00 con 0x8b68840 -4561> 2014-08-13 16:24:25.770653 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x775e740 con 0x7884200 -4560> 2014-08-13 16:24:25.770691 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79ba300 con 0x7fb9e40 -4559> 2014-08-13 16:24:25.770709 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79b9340 con 0x7fb9ce0 -4558> 2014-08-13 16:24:25.770727 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x79bb2c0 con 0x8c3b9c0 -4557> 2014-08-13 16:24:25.770745 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8c88e00 con 0x8c3b700 -4556> 2014-08-13 16:24:25.770768 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8c8c600 con 0x7fbb020 -4555> 2014-08-13 16:24:25.770784 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9287000 con 0x7fbaec0 -4554> 2014-08-13 16:24:25.770801 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9286580 con 0x7fbb440 -4553> 2014-08-13 16:24:25.770825 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9282f40 con 0x7fbb2e0 -4552> 2014-08-13 16:24:25.770843 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9287700 con 0x7ffd960 -4551> 2014-08-13 16:24:25.770861 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9285b00 con 0x7ffdc20 -4550> 2014-08-13 16:24:25.770883 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x92824c0 con 0x8d20840 -4549> 2014-08-13 16:24:25.770903 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9282140 con 0x8d25960 -4548> 2014-08-13 16:24:25.770919 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9285080 con 0x8d218c0 -4547> 2014-08-13 16:24:25.770936 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899cb40 con 0x8d21600 -4546> 2014-08-13 16:24:25.770953 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899c280 con 0x7fbaaa0 -4545> 2014-08-13 16:24:25.770969 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eebd40 con 0x7fba940 -4544> 2014-08-13 16:24:25.770988 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eef8c0 con 0x8d26e00 -4543> 2014-08-13 16:24:25.771005 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eefa80 con 0x8d26b40 -4542> 2014-08-13 16:24:25.771026 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee8a80 con 0x8d21340 -4541> 2014-08-13 16:24:25.771042 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee9500 con 0x8d21080 -4540> 2014-08-13 16:24:25.771059 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eef700 con 0x7ffdee0 -4539> 2014-08-13 16:24:25.771081 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eea300 con 0x7ffdd80 -4538> 2014-08-13 16:24:25.771098 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eec280 con 0x7fb9a20 -4537> 2014-08-13 16:24:25.771116 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9105e80 con 0x7fb98c0 -4536> 2014-08-13 16:24:25.771134 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9103480 con 0x8bab2e0 -4535> 2014-08-13 16:24:25.771152 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9107000 con 0x8bab020 -4534> 2014-08-13 16:24:25.771187 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91039c0 con 0x8c69080 -4533> 2014-08-13 16:24:25.771214 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91024c0 con 0x8c68dc0 -4532> 2014-08-13 16:24:25.771233 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9101c00 con 0x8d256a0 -4531> 2014-08-13 16:24:25.771252 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91063c0 con 0x8d253e0 -4530> 2014-08-13 16:24:25.771276 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9107540 con 0x8d20dc0 -4529> 2014-08-13 16:24:25.771301 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eec980 con 0x8d20b00 -4528> 2014-08-13 16:24:25.771318 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eefa80 con 0x8d25120 -4527> 2014-08-13 16:24:25.771348 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee9c00 con 0x8d24e60 -4526> 2014-08-13 16:24:25.771366 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee9500 con 0x8c3d120 -4525> 2014-08-13 16:24:25.771384 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eea680 con 0x8c3ce60 -4524> 2014-08-13 16:24:25.771401 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eebd40 con 0x7fbb180 -4523> 2014-08-13 16:24:25.771422 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eec0c0 con 0x7fbca40 -4522> 2014-08-13 16:24:25.771438 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eea300 con 0x7fbc780 -4521> 2014-08-13 16:24:25.771459 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee9880 con 0x7fbc4c0 -4520> 2014-08-13 16:24:25.771475 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eeb100 con 0x8c3aec0 -4519> 2014-08-13 16:24:25.771495 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eec280 con 0x8c3ac00 -4518> 2014-08-13 16:24:25.771517 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee9dc0 con 0x7fbad60 -4517> 2014-08-13 16:24:25.771534 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eed240 con 0x7fbac00 -4516> 2014-08-13 16:24:25.771553 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eed080 con 0x8c3e880 -4515> 2014-08-13 16:24:25.771572 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eeb2c0 con 0x8c3e5c0 -4514> 2014-08-13 16:24:25.771589 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eef700 con 0x8c3a940 -4513> 2014-08-13 16:24:25.771606 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8ee8e00 con 0x8c3a680 -4512> 2014-08-13 16:24:25.771635 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8eeb480 con 0x8c69fa0 -4511> 2014-08-13 16:24:25.771651 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8c88e00 con 0x8c39fa0 -4510> 2014-08-13 16:24:25.771669 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x92824c0 con 0x7fb9760 -4509> 2014-08-13 16:24:25.771686 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9282f40 con 0x7fb9600 -4508> 2014-08-13 16:24:25.771703 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9285b00 con 0x8c3e300 -4507> 2014-08-13 16:24:25.771720 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9285080 con 0x8c618c0 -4506> 2014-08-13 16:24:25.771738 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9286580 con 0x8bac780 -4505> 2014-08-13 16:24:25.771756 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9282140 con 0x8bac4c0 -4504> 2014-08-13 16:24:25.771773 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dbf00 con 0x8bac200 -4503> 2014-08-13 16:24:25.771794 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80da300 con 0x8babf40 -4502> 2014-08-13 16:24:25.771811 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9500 con 0x8d240a0 -4501> 2014-08-13 16:24:25.771834 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dfe00 con 0x8d23de0 -4500> 2014-08-13 16:24:25.771851 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dc440 con 0x8ba1760 -4499> 2014-08-13 16:24:25.771868 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9340 con 0x8ba14a0 -4498> 2014-08-13 16:24:25.771885 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80db480 con 0x8d26880 -4497> 2014-08-13 16:24:25.771901 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d8a80 con 0x8d265c0 -4496> 2014-08-13 16:24:25.771918 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899c280 con 0x8baf900 -4495> 2014-08-13 16:24:25.771934 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899cb40 con 0x8baf640 -4494> 2014-08-13 16:24:25.771952 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x7758a80 con 0x8bae040 -4493> 2014-08-13 16:24:25.771968 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8575240 con 0x8badd80 -4492> 2014-08-13 16:24:25.771989 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8575080 con 0x8d24620 -4491> 2014-08-13 16:24:25.772008 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8574ec0 con 0x8d24360 -4490> 2014-08-13 16:24:25.772025 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8574d00 con 0x8baf380 -4489> 2014-08-13 16:24:25.772043 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8575400 con 0x8baf0c0 -4488> 2014-08-13 16:24:25.772061 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x85747c0 con 0x8b6e1a0 -4487> 2014-08-13 16:24:25.772084 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8999a40 con 0x8b6e040 -4486> 2014-08-13 16:24:25.772107 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899b480 con 0x8c39ce0 -4485> 2014-08-13 16:24:25.772124 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899d780 con 0x8c39a20 -4484> 2014-08-13 16:24:25.772142 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899f700 con 0x8d27380 -4483> 2014-08-13 16:24:25.772158 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899b640 con 0x8d270c0 -4482> 2014-08-13 16:24:25.772183 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899fe00 con 0x8d23b20 -4481> 2014-08-13 16:24:25.772200 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8998700 con 0x8d27bc0 -4480> 2014-08-13 16:24:25.772216 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899f540 con 0x8b6dee0 -4479> 2014-08-13 16:24:25.772233 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899a300 con 0x8b6dd80 -4478> 2014-08-13 16:24:25.772250 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899af40 con 0x7fba7e0 -4477> 2014-08-13 16:24:25.772269 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899d5c0 con 0x7fba680 -4476> 2014-08-13 16:24:25.772286 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899e200 con 0x8c6f900 -4475> 2014-08-13 16:24:25.772303 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8999dc0 con 0x8c6f640 -4474> 2014-08-13 16:24:25.772321 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899fa80 con 0x8c68b00 -4473> 2014-08-13 16:24:25.772338 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x89988c0 con 0x8c68840 -4472> 2014-08-13 16:24:25.772355 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8998000 con 0x8c3b440 -4471> 2014-08-13 16:24:25.772372 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8998380 con 0x8c3b180 -4470> 2014-08-13 16:24:25.772389 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899ad80 con 0x8c68580 -4469> 2014-08-13 16:24:25.772406 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x8999340 con 0x8c6b2e0 -4468> 2014-08-13 16:24:25.772422 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899c280 con 0x8c6b020 -4467> 2014-08-13 16:24:25.772441 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899a840 con 0x8c6ad60 -4466> 2014-08-13 16:24:25.772458 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899e740 con 0x8b6dc20 -4465> 2014-08-13 16:24:25.772476 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899cb40 con 0x7fb9b80 -4464> 2014-08-13 16:24:25.772493 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899de80 con 0x8ba11e0 -4463> 2014-08-13 16:24:25.772516 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dbf00 con 0x8ba0f20 -4462> 2014-08-13 16:24:25.772540 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dc440 con 0x7fbbc80 -4461> 2014-08-13 16:24:25.772561 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9340 con 0x7fbbb20 -4460> 2014-08-13 16:24:25.772589 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9500 con 0x8c39760 -4459> 2014-08-13 16:24:25.772610 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dfe00 con 0x8c394a0 -4458> 2014-08-13 16:24:25.772629 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80db480 con 0x7fbb9c0 -4457> 2014-08-13 16:24:25.772647 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80da300 con 0x7fbb860 -4456> 2014-08-13 16:24:25.772664 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9282140 con 0x7fb94a0 -4455> 2014-08-13 16:24:25.772681 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9285080 con 0x7fba520 -4454> 2014-08-13 16:24:25.772699 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115b00 con 0x7fbc200 -4453> 2014-08-13 16:24:25.772715 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115940 con 0x7fbbf40 -4452> 2014-08-13 16:24:25.772731 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115e80 con 0x8c6aaa0 -4451> 2014-08-13 16:24:25.772749 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115cc0 con 0x8c6a7e0 -4450> 2014-08-13 16:24:25.772767 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115400 con 0x8bacd00 -4449> 2014-08-13 16:24:25.772785 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115780 con 0x8baca40 -4448> 2014-08-13 16:24:25.772803 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115240 con 0x8ba0c60 -4447> 2014-08-13 16:24:25.772825 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9116040 con 0x8ba09a0 -4446> 2014-08-13 16:24:25.772843 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9115080 con 0x7ffdac0 -4445> 2014-08-13 16:24:25.772859 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x7758a80 con 0x7ffe040 -4444> 2014-08-13 16:24:25.772889 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899de80 con 0x8c6f380 -4443> 2014-08-13 16:24:25.772909 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899cb40 con 0x8c6f0c0 -4442> 2014-08-13 16:24:25.772926 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899fa80 con 0x7fbb700 -4441> 2014-08-13 16:24:25.772946 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899f700 con 0x7fbb5a0 -4440> 2014-08-13 16:24:25.772962 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x899d940 con 0x8d27900 -4439> 2014-08-13 16:24:25.772979 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d8a80 con 0x8d27640 -4438> 2014-08-13 16:24:25.772996 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dbf00 con 0x8c6a520 -4437> 2014-08-13 16:24:25.773014 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dc440 con 0x8c6a260 -4436> 2014-08-13 16:24:25.773031 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80dfe00 con 0x8d24ba0 -4435> 2014-08-13 16:24:25.773053 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80da300 con 0x8d248e0 -4434> 2014-08-13 16:24:25.773071 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80db480 con 0x8d26300 -4433> 2014-08-13 16:24:25.773090 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9340 con 0x8d26040 -4432> 2014-08-13 16:24:25.773108 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x80d9500 con 0x8d25d80 -4431> 2014-08-13 16:24:25.773134 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x91155c0 con 0x8d140a0 -4430> 2014-08-13 16:24:25.773153 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9114ec0 con 0x8d13de0 -4429> 2014-08-13 16:24:25.773175 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194450 stamp 2014-08-13 16:24:25.769669) v2 -- ?+0 0x9114d00 con 0x8d13b20 -4428> 2014-08-13 16:24:25.773901 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x775a680 con 0x78848e0 -4427> 2014-08-13 16:24:25.773946 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9107700 con 0x7884620 -4426> 2014-08-13 16:24:25.773972 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9100c40 con 0x794c360 -4425> 2014-08-13 16:24:25.774017 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7758000 con 0x794e300 -4424> 2014-08-13 16:24:25.774041 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91032c0 con 0x794e040 -4423> 2014-08-13 16:24:25.774076 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x775e040 con 0x794dd80 -4422> 2014-08-13 16:24:25.774100 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842bb80 con 0x7ffd960 -4421> 2014-08-13 16:24:25.774125 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842d080 con 0x7ffdc20 -4420> 2014-08-13 16:24:25.774149 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x84096c0 con 0x7ffdac0 -4419> 2014-08-13 16:24:25.774184 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840d080 con 0x7ffe040 -4418> 2014-08-13 16:24:25.774212 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842c0c0 con 0x7ffdee0 -4417> 2014-08-13 16:24:25.774245 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842d5c0 con 0x7ffdd80 -4416> 2014-08-13 16:24:25.774271 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8998540 con 0x8b68c60 -4415> 2014-08-13 16:24:25.774297 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8429c00 con 0x8b68b00 -4414> 2014-08-13 16:24:25.774321 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x85701c0 con 0x8b689a0 -4413> 2014-08-13 16:24:25.774345 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8575780 con 0x8b68840 -4412> 2014-08-13 16:24:25.774369 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed2680 con 0x8b6e1a0 -4411> 2014-08-13 16:24:25.774393 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b81c0 con 0x8b6e040 -4410> 2014-08-13 16:24:25.774416 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b9180 con 0x8b6dee0 -4409> 2014-08-13 16:24:25.774437 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bb100 con 0x8b6dd80 -4408> 2014-08-13 16:24:25.774456 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bcb40 con 0x8b6dc20 -4407> 2014-08-13 16:24:25.774476 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bf540 con 0x7fb9b80 -4406> 2014-08-13 16:24:25.774494 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842f540 con 0x7fb9a20 -4405> 2014-08-13 16:24:25.774513 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842eac0 con 0x7fb98c0 -4404> 2014-08-13 16:24:25.774532 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834db00 con 0x7fb9760 -4403> 2014-08-13 16:24:25.774551 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834d5c0 con 0x7fb9600 -4402> 2014-08-13 16:24:25.774570 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x79bfa80 con 0x7fb94a0 -4401> 2014-08-13 16:24:25.774590 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x80da300 con 0x7fba520 -4400> 2014-08-13 16:24:25.774608 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8998a80 con 0x7fba3c0 -4399> 2014-08-13 16:24:25.774628 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8570c40 con 0x7fba260 -4398> 2014-08-13 16:24:25.774647 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8575cc0 con 0x7fba100 -4397> 2014-08-13 16:24:25.774669 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899c7c0 con 0x7fb9fa0 -4396> 2014-08-13 16:24:25.774688 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8429180 con 0x7fb9e40 -4395> 2014-08-13 16:24:25.774710 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8577700 con 0x7fb9ce0 -4394> 2014-08-13 16:24:25.774730 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842a680 con 0x7fbb020 -4393> 2014-08-13 16:24:25.774748 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x85771c0 con 0x7fbaec0 -4392> 2014-08-13 16:24:25.774767 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91178c0 con 0x7fbad60 -4391> 2014-08-13 16:24:25.774787 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834c0c0 con 0x7fbac00 -4390> 2014-08-13 16:24:25.774805 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899b9c0 con 0x7fbaaa0 -4389> 2014-08-13 16:24:25.774825 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9110c40 con 0x7fba940 -4388> 2014-08-13 16:24:25.774843 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79ba140 con 0x7fba7e0 -4387> 2014-08-13 16:24:25.774862 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79babc0 con 0x7fba680 -4386> 2014-08-13 16:24:25.774887 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840b640 con 0x7fbbc80 -4385> 2014-08-13 16:24:25.774908 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79beac0 con 0x7fbbb20 -4384> 2014-08-13 16:24:25.774929 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bd5c0 con 0x7fbb9c0 -4383> 2014-08-13 16:24:25.774955 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bd080 con 0x7fbb860 -4382> 2014-08-13 16:24:25.774978 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840f540 con 0x7fbb700 -4381> 2014-08-13 16:24:25.775001 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840d5c0 con 0x7fbb5a0 -4380> 2014-08-13 16:24:25.775024 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842a140 con 0x7fbb440 -4379> 2014-08-13 16:24:25.775047 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842abc0 con 0x7fbb2e0 -4378> 2014-08-13 16:24:25.775070 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9117380 con 0x7fbb180 -4377> 2014-08-13 16:24:25.775101 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8349180 con 0x7fbca40 -4376> 2014-08-13 16:24:25.775125 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x8348700 con 0x7fbc780 -4375> 2014-08-13 16:24:25.775148 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x8349c00 con 0x7fbc4c0 -4374> 2014-08-13 16:24:25.775178 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840bb80 con 0x7fbc200 -4373> 2014-08-13 16:24:25.775204 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83c8700 con 0x7fbbf40 -4372> 2014-08-13 16:24:25.775227 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x9100380 con 0x8babde0 -4371> 2014-08-13 16:24:25.775248 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x7759180 con 0x8babb20 -4370> 2014-08-13 16:24:25.775267 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9101dc0 con 0x8bab860 -4369> 2014-08-13 16:24:25.775286 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9102840 con 0x8bab5a0 -4368> 2014-08-13 16:24:25.775309 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x842cb40 con 0x8bab2e0 -4367> 2014-08-13 16:24:25.775336 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x834a140 con 0x8bab020 -4366> 2014-08-13 16:24:25.775359 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840abc0 con 0x8bacd00 -4365> 2014-08-13 16:24:25.775383 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840eac0 con 0x8baca40 -4364> 2014-08-13 16:24:25.775402 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834c600 con 0x8bac780 -4363> 2014-08-13 16:24:25.775426 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834f000 con 0x8bac4c0 -4362> 2014-08-13 16:24:25.775449 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91140c0 con 0x8bac200 -4361> 2014-08-13 16:24:25.775472 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed0700 con 0x8babf40 -4360> 2014-08-13 16:24:25.775495 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x7ed16c0 con 0x8bae040 -4359> 2014-08-13 16:24:25.775518 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x7ed0c40 con 0x8badd80 -4358> 2014-08-13 16:24:25.775541 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899f1c0 con 0x8badac0 -4357> 2014-08-13 16:24:25.775567 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9105240 con 0x8bad800 -4356> 2014-08-13 16:24:25.775591 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8570700 con 0x8bad540 -4355> 2014-08-13 16:24:25.775614 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x84296c0 con 0x8bad280 -4354> 2014-08-13 16:24:25.775637 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8999c00 con 0x8bafe80 -4353> 2014-08-13 16:24:25.775661 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8576740 con 0x8bafbc0 -4352> 2014-08-13 16:24:25.775684 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed7540 con 0x8baf900 -4351> 2014-08-13 16:24:25.775707 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed1c00 con 0x8baf640 -4350> 2014-08-13 16:24:25.775730 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed2140 con 0x8baf380 -4349> 2014-08-13 16:24:25.775753 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed2bc0 con 0x8baf0c0 -4348> 2014-08-13 16:24:25.775776 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9105cc0 con 0x8baee00 -4347> 2014-08-13 16:24:25.775799 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9102300 con 0x8ba1a20 -4346> 2014-08-13 16:24:25.775828 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834f540 con 0x8ba1760 -4345> 2014-08-13 16:24:25.775852 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834fa80 con 0x8ba14a0 -4344> 2014-08-13 16:24:25.775877 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bdb00 con 0x8ba11e0 -4343> 2014-08-13 16:24:25.775900 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840c0c0 con 0x8ba0f20 -4342> 2014-08-13 16:24:25.775923 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8408700 con 0x8ba0c60 -4341> 2014-08-13 16:24:25.775945 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840a140 con 0x8ba09a0 -4340> 2014-08-13 16:24:25.775968 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.1.115:6839/62713 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x81c4600 con 0x8ba06e0 -4339> 2014-08-13 16:24:25.775993 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.7 192.168.100.115:6846/62713 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9102f40 con 0x8c698c0 -4338> 2014-08-13 16:24:25.776017 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899bd40 con 0x8c69600 -4337> 2014-08-13 16:24:25.776044 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9103f00 con 0x8c69340 -4336> 2014-08-13 16:24:25.776067 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9117c40 con 0x8c69080 -4335> 2014-08-13 16:24:25.776092 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91101c0 con 0x8c68dc0 -4334> 2014-08-13 16:24:25.776115 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79be040 con 0x8c68b00 -4333> 2014-08-13 16:24:25.776140 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8409c00 con 0x8c68840 -4332> 2014-08-13 16:24:25.776163 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bb640 con 0x8c68580 -4331> 2014-08-13 16:24:25.776209 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840c600 con 0x8c6b2e0 -4330> 2014-08-13 16:24:25.776234 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bc600 con 0x8c6b020 -4329> 2014-08-13 16:24:25.776256 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bc0c0 con 0x8c6ad60 -4328> 2014-08-13 16:24:25.776279 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8408c40 con 0x8c6aaa0 -4327> 2014-08-13 16:24:25.776302 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8409180 con 0x8c6a7e0 -4326> 2014-08-13 16:24:25.776324 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x83c81c0 con 0x8c6a520 -4325> 2014-08-13 16:24:25.776348 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x840e040 con 0x8c6a260 -4324> 2014-08-13 16:24:25.776371 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834b640 con 0x8c69fa0 -4323> 2014-08-13 16:24:25.776395 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834d080 con 0x8c39fa0 -4322> 2014-08-13 16:24:25.776418 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b8700 con 0x8c39ce0 -4321> 2014-08-13 16:24:25.776441 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83c96c0 con 0x8c39a20 -4320> 2014-08-13 16:24:25.776463 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840a680 con 0x8c39760 -4319> 2014-08-13 16:24:25.776486 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840b100 con 0x8c394a0 -4318> 2014-08-13 16:24:25.776508 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x81c0380 con 0x8c391e0 -4317> 2014-08-13 16:24:25.776536 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x81c1dc0 con 0x8c38f20 -4316> 2014-08-13 16:24:25.776566 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8571180 con 0x8c38c60 -4315> 2014-08-13 16:24:25.776591 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x84281c0 con 0x8c389a0 -4314> 2014-08-13 16:24:25.776614 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8428700 con 0x8c3b9c0 -4313> 2014-08-13 16:24:25.776637 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8576c80 con 0x8c3b700 -4312> 2014-08-13 16:24:25.776659 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79be580 con 0x8c3b440 -4311> 2014-08-13 16:24:25.776682 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bbb80 con 0x8c3b180 -4310> 2014-08-13 16:24:25.776708 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8348c40 con 0x8c3aec0 -4309> 2014-08-13 16:24:25.776731 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83496c0 con 0x8c3ac00 -4308> 2014-08-13 16:24:25.776754 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834bb80 con 0x8c3a940 -4307> 2014-08-13 16:24:25.776777 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834b100 con 0x8c3a680 -4306> 2014-08-13 16:24:25.776799 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9105780 con 0x8c3a3c0 -4305> 2014-08-13 16:24:25.776829 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x775a840 con 0x8c3a100 -4304> 2014-08-13 16:24:25.776853 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8ee8c40 con 0x8c3d6a0 -4303> 2014-08-13 16:24:25.776881 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9107380 con 0x8c3d3e0 -4302> 2014-08-13 16:24:25.776900 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9111180 con 0x8c3d120 -4301> 2014-08-13 16:24:25.776919 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83481c0 con 0x8c3ce60 -4300> 2014-08-13 16:24:25.776938 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x899bf00 con 0x8c3cba0 -4299> 2014-08-13 16:24:25.776957 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x899cec0 con 0x8c3c8e0 -4298> 2014-08-13 16:24:25.776976 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899f8c0 con 0x8c3c620 -4297> 2014-08-13 16:24:25.776995 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8428c40 con 0x8c3c360 -4296> 2014-08-13 16:24:25.777017 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9107c40 con 0x8c3c0a0 -4295> 2014-08-13 16:24:25.777037 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x775f540 con 0x8c3bde0 -4294> 2014-08-13 16:24:25.777055 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834abc0 con 0x8c3e880 -4293> 2014-08-13 16:24:25.777075 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834a680 con 0x8c3e5c0 -4292> 2014-08-13 16:24:25.777093 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8c88540 con 0x8c3e300 -4291> 2014-08-13 16:24:25.777112 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834cb40 con 0x8c618c0 -4290> 2014-08-13 16:24:25.777131 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8576200 con 0x8c6fe80 -4289> 2014-08-13 16:24:25.777150 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8577c40 con 0x8c6fbc0 -4288> 2014-08-13 16:24:25.777205 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x84081c0 con 0x8c6f900 -4287> 2014-08-13 16:24:25.777229 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79bf000 con 0x8c6f640 -4286> 2014-08-13 16:24:25.777248 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840e580 con 0x8c6f380 -4285> 2014-08-13 16:24:25.777268 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840cb40 con 0x8c6f0c0 -4284> 2014-08-13 16:24:25.777292 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x8eee740 con 0x8d223c0 -4283> 2014-08-13 16:24:25.777316 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x899e580 con 0x8d22100 -4282> 2014-08-13 16:24:25.777339 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.1.115:6814/59915 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899b800 con 0x8d21e40 -4281> 2014-08-13 16:24:25.777362 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.32 192.168.100.115:6831/59915 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x899f000 con 0x8d21b80 -4280> 2014-08-13 16:24:25.777388 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x842f000 con 0x8d218c0 -4279> 2014-08-13 16:24:25.777411 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x842b100 con 0x8d21600 -4278> 2014-08-13 16:24:25.777434 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842e580 con 0x8d21340 -4277> 2014-08-13 16:24:25.777457 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842db00 con 0x8d21080 -4276> 2014-08-13 16:24:25.777479 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9116900 con 0x8d20dc0 -4275> 2014-08-13 16:24:25.777506 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842fa80 con 0x8d20b00 -4274> 2014-08-13 16:24:25.777529 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842b640 con 0x8d20840 -4273> 2014-08-13 16:24:25.777552 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9110700 con 0x8d25960 -4272> 2014-08-13 16:24:25.777577 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9111c00 con 0x8d256a0 -4271> 2014-08-13 16:24:25.777600 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x9116e40 con 0x8d253e0 -4270> 2014-08-13 16:24:25.777622 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91163c0 con 0x8d25120 -4269> 2014-08-13 16:24:25.777645 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x91116c0 con 0x8d24e60 -4268> 2014-08-13 16:24:25.777668 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83c9180 con 0x8d24ba0 -4267> 2014-08-13 16:24:25.777691 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x80d9340 con 0x8d248e0 -4266> 2014-08-13 16:24:25.777713 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed1180 con 0x8d24620 -4265> 2014-08-13 16:24:25.777736 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b8c40 con 0x8d24360 -4264> 2014-08-13 16:24:25.777758 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed01c0 con 0x8d240a0 -4263> 2014-08-13 16:24:25.777789 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834e040 con 0x8d23de0 -4262> 2014-08-13 16:24:25.777813 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79ba680 con 0x8d23b20 -4261> 2014-08-13 16:24:25.777838 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b9c00 con 0x8d27bc0 -4260> 2014-08-13 16:24:25.777861 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840db00 con 0x8d27900 -4259> 2014-08-13 16:24:25.777884 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840f000 con 0x8d27640 -4258> 2014-08-13 16:24:25.777907 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x7ed7a80 con 0x8d27380 -4257> 2014-08-13 16:24:25.777930 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x79b96c0 con 0x8d270c0 -4256> 2014-08-13 16:24:25.777952 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842e040 con 0x8d26e00 -4255> 2014-08-13 16:24:25.777977 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x842c600 con 0x8d26b40 -4254> 2014-08-13 16:24:25.777999 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834eac0 con 0x8d26880 -4253> 2014-08-13 16:24:25.778019 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x834e580 con 0x8d265c0 -4252> 2014-08-13 16:24:25.778038 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x840fa80 con 0x8d26300 -4251> 2014-08-13 16:24:25.778057 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x83c8c40 con 0x8d26040 -4250> 2014-08-13 16:24:25.778078 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x77588c0 con 0x8d25d80 -4249> 2014-08-13 16:24:25.778097 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x87e4d00 con 0x8d140a0 -4248> 2014-08-13 16:24:25.778115 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x8eef000 con 0x8d13de0 -4247> 2014-08-13 16:24:25.778135 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 5 ==== osd_ping(ping_reply e194451 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3003095663 0 0) 0x92855c0 con 0x8d13b20 -4246> 2014-08-13 16:24:25.778154 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 5 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.769669) v2 ==== 47+0+0 (3394829026 0 0) 0x7758e00 con 0x78844c0 -4245> 2014-08-13 16:24:25.794748 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.9 192.168.100.107:0/53997 11 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.794509) v2 ==== 47+0+0 (48196263 0 0) 0x842b640 con 0x7d2f0c0 -4244> 2014-08-13 16:24:25.794806 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.794509) v2 -- ?+0 0x8c88000 con 0x7d2f0c0 -4243> 2014-08-13 16:24:25.794826 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.9 192.168.100.107:0/53997 11 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.794509) v2 ==== 47+0+0 (48196263 0 0) 0x842cb40 con 0x7d2dc20 -4242> 2014-08-13 16:24:25.794872 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.794509) v2 -- ?+0 0x8c8d940 con 0x7d2dc20 -4241> 2014-08-13 16:24:25.853183 7fd9385e7700 10 monclient: tick -4240> 2014-08-13 16:24:25.853207 7fd9385e7700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 16:23:55.853205) -4239> 2014-08-13 16:24:25.853221 7fd9385e7700 10 monclient: renew subs? (now: 2014-08-13 16:24:25.853220; renew after: 2014-08-13 16:26:33.460427) -- no -4238> 2014-08-13 16:24:25.853239 7fd9385e7700 10 log_queue is 1 last_log 3 sent 2 num 1 unsent 1 sending 1 -4237> 2014-08-13 16:24:25.853251 7fd9385e7700 10 will send 2014-08-13 16:24:25.188141 osd.4 192.168.100.115:6805/25220 3 : [INF] 22.1b8 deep-scrub ok -4236> 2014-08-13 16:24:25.853276 7fd9385e7700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4235> 2014-08-13 16:24:25.853282 7fd9385e7700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- log(1 entries) v1 -- ?+0 0x8d01680 con 0x4e3dd80 -4234> 2014-08-13 16:24:25.895821 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.895144) v2 ==== 47+0+0 (2328064770 0 0) 0x7d06e40 con 0x7d2ac00 -4233> 2014-08-13 16:24:25.895873 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.895144) v2 -- ?+0 0x842cb40 con 0x7d2ac00 -4232> 2014-08-13 16:24:25.895890 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.895144) v2 ==== 47+0+0 (2328064770 0 0) 0x842f1c0 con 0x7d2c360 -4231> 2014-08-13 16:24:25.895939 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.895144) v2 -- ?+0 0x842b640 con 0x7d2c360 -4230> 2014-08-13 16:24:25.921805 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.921510) v2 ==== 47+0+0 (3147199867 0 0) 0x8409dc0 con 0x7eda680 -4229> 2014-08-13 16:24:25.921856 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.921510) v2 -- ?+0 0x7d06e40 con 0x7eda680 -4228> 2014-08-13 16:24:25.921854 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.921510) v2 ==== 47+0+0 (3147199867 0 0) 0x7d05080 con 0x7edc620 -4227> 2014-08-13 16:24:25.921899 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.921510) v2 -- ?+0 0x842f1c0 con 0x7edc620 -4226> 2014-08-13 16:24:25.970872 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.397 192.168.1.105:0/41085 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.970407) v2 ==== 47+0+0 (3906353330 0 0) 0x842cd00 con 0x7ebce60 -4225> 2014-08-13 16:24:25.970880 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.397 192.168.1.105:0/41085 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:25.970407) v2 ==== 47+0+0 (3906353330 0 0) 0x834a680 con 0x7ebb860 -4224> 2014-08-13 16:24:25.970935 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.970407) v2 -- ?+0 0x7d05080 con 0x7ebce60 -4223> 2014-08-13 16:24:25.971098 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:25.970407) v2 -- ?+0 0x8409dc0 con 0x7ebb860 -4222> 2014-08-13 16:24:26.066569 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 9 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.066338) v2 ==== 47+0+0 (2266709776 0 0) 0x8576e40 con 0x7cae040 -4221> 2014-08-13 16:24:26.066621 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.066338) v2 -- ?+0 0x842cd00 con 0x7cae040 -4220> 2014-08-13 16:24:26.066652 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 9 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.066338) v2 ==== 47+0+0 (2266709776 0 0) 0x8575940 con 0x7cae880 -4219> 2014-08-13 16:24:26.066705 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.066338) v2 -- ?+0 0x834a680 con 0x7cae880 -4218> 2014-08-13 16:24:26.118637 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 6 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.118691) v2 ==== 47+0+0 (308368366 0 0) 0x775f000 con 0x7ebf0c0 -4217> 2014-08-13 16:24:26.118690 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.118691) v2 -- ?+0 0x8575940 con 0x7ebf0c0 -4216> 2014-08-13 16:24:26.118676 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 6 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.118691) v2 ==== 47+0+0 (308368366 0 0) 0x7d01500 con 0x7ed9080 -4215> 2014-08-13 16:24:26.118754 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.118691) v2 -- ?+0 0x8576e40 con 0x7ed9080 -4214> 2014-08-13 16:24:26.150809 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.150829) v2 ==== 47+0+0 (2585682768 0 0) 0x8c8e740 con 0x7d2aec0 -4213> 2014-08-13 16:24:26.150845 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.150829) v2 ==== 47+0+0 (2585682768 0 0) 0x89108c0 con 0x7d2c620 -4212> 2014-08-13 16:24:26.150867 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.150829) v2 -- ?+0 0x775f000 con 0x7d2aec0 -4211> 2014-08-13 16:24:26.150989 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.150829) v2 -- ?+0 0x7d01500 con 0x7d2c620 -4210> 2014-08-13 16:24:26.292568 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.292497) v2 ==== 47+0+0 (2259875580 0 0) 0x83496c0 con 0x7eba680 -4209> 2014-08-13 16:24:26.292603 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.292497) v2 ==== 47+0+0 (2259875580 0 0) 0x834a680 con 0x7eb91e0 -4208> 2014-08-13 16:24:26.292619 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.292497) v2 -- ?+0 0x89108c0 con 0x7eba680 -4207> 2014-08-13 16:24:26.292768 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.292497) v2 -- ?+0 0x8c8e740 con 0x7eb91e0 -4206> 2014-08-13 16:24:26.310467 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.310496) v2 ==== 47+0+0 (1047094488 0 0) 0x834f000 con 0x7edc360 -4205> 2014-08-13 16:24:26.310475 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 7 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.310496) v2 ==== 47+0+0 (1047094488 0 0) 0x8348700 con 0x7eda3c0 -4204> 2014-08-13 16:24:26.310495 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.310496) v2 -- ?+0 0x83496c0 con 0x7edc360 -4203> 2014-08-13 16:24:26.310535 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.310496) v2 -- ?+0 0x834a680 con 0x7eda3c0 -4202> 2014-08-13 16:24:26.518933 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.518964) v2 ==== 47+0+0 (4223381181 0 0) 0x9285cc0 con 0x7edac00 -4201> 2014-08-13 16:24:26.519000 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.518964) v2 -- ?+0 0x8348700 con 0x7edac00 -4200> 2014-08-13 16:24:26.519016 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.518964) v2 ==== 47+0+0 (4223381181 0 0) 0x7ed7e00 con 0x7edcba0 -4199> 2014-08-13 16:24:26.519043 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.518964) v2 -- ?+0 0x834f000 con 0x7edcba0 -4198> 2014-08-13 16:24:26.709052 7fd9208e2700 1 -- 192.168.1.115:6813/25220 >> :/0 pipe(0x8d46180 sd=118 :6813 s=0 pgs=0 cs=0 l=0 c=0x83002c0).accept sd=118 192.168.1.107:45709/0 -4197> 2014-08-13 16:24:26.747091 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 10 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:26.747077) v2 ==== 47+0+0 (3392224061 0 0) 0x7d03640 con 0x7ab8000 -4196> 2014-08-13 16:24:26.747093 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 10 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:26.747077) v2 ==== 47+0+0 (3392224061 0 0) 0x775b2c0 con 0x7ab8420 -4195> 2014-08-13 16:24:26.747147 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.747077) v2 -- ?+0 0x7ed7e00 con 0x7ab8000 -4194> 2014-08-13 16:24:26.747239 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.747077) v2 -- ?+0 0x9285cc0 con 0x7ab8420 -4193> 2014-08-13 16:24:26.753624 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.292 192.168.1.113:0/16056 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.753740) v2 ==== 47+0+0 (2566609794 0 0) 0x8428380 con 0x7ebd120 -4192> 2014-08-13 16:24:26.753631 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.292 192.168.1.113:0/16056 8 ==== osd_ping(ping e194451 stamp 2014-08-13 16:24:26.753740) v2 ==== 47+0+0 (2566609794 0 0) 0x842e740 con 0x7ebbb20 -4191> 2014-08-13 16:24:26.753669 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.753740) v2 -- ?+0 0x7d03640 con 0x7ebd120 -4190> 2014-08-13 16:24:26.753760 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.753740) v2 -- ?+0 0x775b2c0 con 0x7ebbb20 -4189> 2014-08-13 16:24:26.896041 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.374 192.168.100.102:0/16129 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:26.895927) v2 ==== 47+0+0 (3733071293 0 0) 0x9115940 con 0x7d28420 -4188> 2014-08-13 16:24:26.896065 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.374 192.168.100.102:0/16129 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:26.895927) v2 ==== 47+0+0 (3733071293 0 0) 0x9116040 con 0x7d2cba0 -4187> 2014-08-13 16:24:26.896094 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.895927) v2 -- ?+0 0x842e740 con 0x7d28420 -4186> 2014-08-13 16:24:26.896158 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:26.895927) v2 -- ?+0 0x8428380 con 0x7d2cba0 -4185> 2014-08-13 16:24:27.141498 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.141155) v2 ==== 47+0+0 (3413925670 0 0) 0x9287540 con 0x7caeb40 -4184> 2014-08-13 16:24:27.141550 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.141155) v2 ==== 47+0+0 (3413925670 0 0) 0x9282300 con 0x7cae300 -4183> 2014-08-13 16:24:27.141547 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.141155) v2 -- ?+0 0x9115940 con 0x7caeb40 -4182> 2014-08-13 16:24:27.141681 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.141155) v2 -- ?+0 0x9116040 con 0x7cae300 -4181> 2014-08-13 16:24:27.401562 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.401213) v2 ==== 47+0+0 (389928364 0 0) 0x899f1c0 con 0x7ffe9e0 -4180> 2014-08-13 16:24:27.401621 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.401213) v2 -- ?+0 0x9287540 con 0x7ffe9e0 -4179> 2014-08-13 16:24:27.401614 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.401213) v2 ==== 47+0+0 (389928364 0 0) 0x899c7c0 con 0x7edbc80 -4178> 2014-08-13 16:24:27.401659 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.401213) v2 -- ?+0 0x9282300 con 0x7edbc80 -4177> 2014-08-13 16:24:27.413273 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.412808) v2 ==== 47+0+0 (3859290505 0 0) 0x8349180 con 0x7eda3c0 -4176> 2014-08-13 16:24:27.413292 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.412808) v2 ==== 47+0+0 (3859290505 0 0) 0x834c600 con 0x7edc360 -4175> 2014-08-13 16:24:27.413326 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.412808) v2 -- ?+0 0x899f1c0 con 0x7eda3c0 -4174> 2014-08-13 16:24:27.413392 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.412808) v2 -- ?+0 0x899c7c0 con 0x7edc360 -4173> 2014-08-13 16:24:27.416817 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 9 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.416864) v2 ==== 47+0+0 (3932614538 0 0) 0x840b2c0 con 0x7d28160 -4172> 2014-08-13 16:24:27.416869 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.416864) v2 -- ?+0 0x8349180 con 0x7d28160 -4171> 2014-08-13 16:24:27.416842 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 9 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.416864) v2 ==== 47+0+0 (3932614538 0 0) 0x8408e00 con 0x7d2a260 -4170> 2014-08-13 16:24:27.416919 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.416864) v2 -- ?+0 0x834c600 con 0x7d2a260 -4169> 2014-08-13 16:24:27.448384 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.448351) v2 ==== 47+0+0 (3662241985 0 0) 0x9111340 con 0x7d2b700 -4168> 2014-08-13 16:24:27.448435 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.448351) v2 -- ?+0 0x840b2c0 con 0x7d2b700 -4167> 2014-08-13 16:24:27.448508 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.448351) v2 ==== 47+0+0 (3662241985 0 0) 0x9111dc0 con 0x7d29fa0 -4166> 2014-08-13 16:24:27.448550 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.448351) v2 -- ?+0 0x8408e00 con 0x7d29fa0 -4165> 2014-08-13 16:24:27.451143 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.345 192.168.1.110:0/46337 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.450615) v2 ==== 47+0+0 (2094915993 0 0) 0x775fc40 con 0x7ebc620 -4164> 2014-08-13 16:24:27.451202 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.450615) v2 -- ?+0 0x9111340 con 0x7ebc620 -4163> 2014-08-13 16:24:27.451178 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.345 192.168.1.110:0/46337 8 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.450615) v2 ==== 47+0+0 (2094915993 0 0) 0x7758380 con 0x7eb9b80 -4162> 2014-08-13 16:24:27.451245 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.450615) v2 -- ?+0 0x9111dc0 con 0x7eb9b80 -4161> 2014-08-13 16:24:27.503664 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 6 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.503553) v2 ==== 47+0+0 (3886568844 0 0) 0x87e7e00 con 0x7ffeb40 -4160> 2014-08-13 16:24:27.503677 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 6 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.503553) v2 ==== 47+0+0 (3886568844 0 0) 0x87e4980 con 0x7edbde0 -4159> 2014-08-13 16:24:27.503720 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.503553) v2 -- ?+0 0x775fc40 con 0x7ffeb40 -4158> 2014-08-13 16:24:27.503861 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.503553) v2 -- ?+0 0x7758380 con 0x7edbde0 -4157> 2014-08-13 16:24:27.544588 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.544710) v2 ==== 47+0+0 (382860210 0 0) 0x842dcc0 con 0x7ffe880 -4156> 2014-08-13 16:24:27.544699 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 7 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.544710) v2 ==== 47+0+0 (382860210 0 0) 0x899f380 con 0x7edbb20 -4155> 2014-08-13 16:24:27.544684 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.544710) v2 -- ?+0 0x87e7e00 con 0x7ffe880 -4154> 2014-08-13 16:24:27.544893 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:27.544710) v2 -- ?+0 0x87e4980 con 0x7edbb20 -4153> 2014-08-13 16:24:27.652089 7fd93dff0700 1 -- 192.168.1.115:0/25220 mark_down 0x8ba06e0 -- 0x8bfd780 -4152> 2014-08-13 16:24:27.652173 7fd93dff0700 1 -- 192.168.1.115:0/25220 mark_down 0x8c698c0 -- 0x8bfd500 -4151> 2014-08-13 16:24:27.652210 7fd93dff0700 1 -- 192.168.1.115:0/25220 mark_down 0x8d21e40 -- 0x8caf800 -4150> 2014-08-13 16:24:27.652240 7fd93dff0700 1 -- 192.168.1.115:0/25220 mark_down 0x8d21b80 -- 0x8caf580 -4149> 2014-08-13 16:24:27.652292 7fd9018d4700 2 -- 192.168.1.115:0/25220 >> 192.168.100.115:6831/59915 pipe(0x8caf580 sd=519 :44121 s=4 pgs=120 cs=1 l=1 c=0x8d21b80).reader couldn't read tag, (0) Success -4148> 2014-08-13 16:24:27.652308 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 12 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf3a80 con 0x7315ee0 -4147> 2014-08-13 16:24:27.652327 7fd9018d4700 2 -- 192.168.1.115:0/25220 >> 192.168.100.115:6831/59915 pipe(0x8caf580 sd=519 :44121 s=4 pgs=120 cs=1 l=1 c=0x8d21b80).fault (0) Success -4146> 2014-08-13 16:24:27.652337 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4145> 2014-08-13 16:24:27.652352 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7d580 con 0x7314fc0 -4144> 2014-08-13 16:24:27.652367 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4143> 2014-08-13 16:24:27.652376 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859d340 con 0x7316f60 -4142> 2014-08-13 16:24:27.652391 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4141> 2014-08-13 16:24:27.652363 7fd908a45700 2 -- 192.168.1.115:0/25220 >> 192.168.1.115:6839/62713 pipe(0x8bfd780 sd=459 :49080 s=4 pgs=131 cs=1 l=1 c=0x8ba06e0).reader couldn't read tag, (0) Success -4140> 2014-08-13 16:24:27.652400 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e79200 con 0x8da1760 -4139> 2014-08-13 16:24:27.652400 7fd908a45700 2 -- 192.168.1.115:0/25220 >> 192.168.1.115:6839/62713 pipe(0x8bfd780 sd=459 :49080 s=4 pgs=131 cs=1 l=1 c=0x8ba06e0).fault (0) Success -4138> 2014-08-13 16:24:27.652414 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4137> 2014-08-13 16:24:27.652423 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.310 192.168.1.109:6812/32413 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x33e3600 con 0x8da0b00 -4136> 2014-08-13 16:24:27.652437 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4135> 2014-08-13 16:24:27.652464 7fd93f3f2700 3 osd.4 194451 handle_osd_map epochs [194450,194451], i have 194451, src has [185139,194451] -4134> 2014-08-13 16:24:27.652472 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.350 192.168.1.114:6832/53185 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8598900 con 0x8da6b40 -4133> 2014-08-13 16:24:27.652489 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4132> 2014-08-13 16:24:27.652500 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d08480 con 0x8da5120 -4131> 2014-08-13 16:24:27.652498 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 26 ==== log(last 3) v1 ==== 24+0+0 (2732342619 0 0) 0x899e580 con 0x4e3dd80 -4130> 2014-08-13 16:24:27.652514 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4129> 2014-08-13 16:24:27.652519 7fd93f3f2700 10 handle_log_ack log(last 3) v1 -4128> 2014-08-13 16:24:27.652522 7fd93f3f2700 10 logged 2014-08-13 16:24:25.188141 osd.4 192.168.100.115:6805/25220 3 : [INF] 22.1b8 deep-scrub ok -4127> 2014-08-13 16:24:27.652525 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7e0c0 con 0x72d9b80 -4126> 2014-08-13 16:24:27.652539 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4125> 2014-08-13 16:24:27.652550 7fd94ac49700 5 osd.4 194451 tick -4124> 2014-08-13 16:24:27.652548 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.378 192.168.1.102:6840/19195 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b11680 con 0x72dc620 -4123> 2014-08-13 16:24:27.652641 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4122> 2014-08-13 16:24:27.652640 7fd901dd9700 2 -- 192.168.1.115:0/25220 >> 192.168.1.115:6814/59915 pipe(0x8caf800 sd=518 :35343 s=4 pgs=120 cs=1 l=1 c=0x8d21e40).reader couldn't read tag, (0) Success -4121> 2014-08-13 16:24:27.652661 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.7 192.168.1.115:6838/62713 9 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7a400 con 0x72d8dc0 -4120> 2014-08-13 16:24:27.652670 7fd901dd9700 2 -- 192.168.1.115:0/25220 >> 192.168.1.115:6814/59915 pipe(0x8caf800 sd=518 :35343 s=4 pgs=120 cs=1 l=1 c=0x8d21e40).fault (0) Success -4119> 2014-08-13 16:24:27.652681 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4118> 2014-08-13 16:24:27.652692 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf5c40 con 0x72dfa60 -4117> 2014-08-13 16:24:27.652706 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4116> 2014-08-13 16:24:27.652715 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x506ee40 con 0x72de9e0 -4115> 2014-08-13 16:24:27.652728 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4114> 2014-08-13 16:24:27.652738 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e78b40 con 0x8e29fa0 -4113> 2014-08-13 16:24:27.652752 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4112> 2014-08-13 16:24:27.652760 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e79d40 con 0x8e286e0 -4111> 2014-08-13 16:24:27.652774 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4110> 2014-08-13 16:24:27.652784 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x506e540 con 0x8e2c360 -4109> 2014-08-13 16:24:27.652798 7fd9371e5700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4108> 2014-08-13 16:24:27.652821 7fd9371e5700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194451 {24.39a=[4,395,13]} v194451) v1 -- ?+0 0x801b480 con 0x4e3dd80 -4107> 2014-08-13 16:24:27.652835 7fd907f3a700 2 -- 192.168.1.115:0/25220 >> 192.168.100.115:6846/62713 pipe(0x8bfd500 sd=460 :51090 s=4 pgs=131 cs=1 l=1 c=0x8c698c0).reader couldn't read tag, (0) Success -4106> 2014-08-13 16:24:27.652867 7fd907f3a700 2 -- 192.168.1.115:0/25220 >> 192.168.100.115:6846/62713 pipe(0x8bfd500 sd=460 :51090 s=4 pgs=131 cs=1 l=1 c=0x8c698c0).fault (0) Success -4105> 2014-08-13 16:24:27.652934 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4104> 2014-08-13 16:24:27.652959 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.65 192.168.1.111:6842/17345 14 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b14140 con 0x8e2f900 -4103> 2014-08-13 16:24:27.652979 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4102> 2014-08-13 16:24:27.652987 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.25 192.168.1.103:6800/36920 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b121c0 con 0x8e2b860 -4101> 2014-08-13 16:24:27.653001 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4100> 2014-08-13 16:24:27.653013 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d08b40 con 0x7379ce0 -4099> 2014-08-13 16:24:27.653027 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4098> 2014-08-13 16:24:27.653036 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.338 192.168.1.106:6814/25893 6 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf3f00 con 0x737fe80 -4097> 2014-08-13 16:24:27.653049 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4096> 2014-08-13 16:24:27.653057 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf7740 con 0x73ca680 -4095> 2014-08-13 16:24:27.653070 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4094> 2014-08-13 16:24:27.653079 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 14 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd5c40 con 0x73cdc20 -4093> 2014-08-13 16:24:27.653092 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4092> 2014-08-13 16:24:27.653101 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 5 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7c140 con 0x73ccba0 -4091> 2014-08-13 16:24:27.653114 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4090> 2014-08-13 16:24:27.653122 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 5 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d08fc0 con 0x7462100 -4089> 2014-08-13 16:24:27.653135 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4088> 2014-08-13 16:24:27.653143 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 11 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x33e2640 con 0x7723440 -4087> 2014-08-13 16:24:27.653159 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4086> 2014-08-13 16:24:27.653167 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.357 192.168.1.102:6834/17224 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b13cc0 con 0x7724780 -4085> 2014-08-13 16:24:27.653194 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4084> 2014-08-13 16:24:27.653203 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.353 192.168.1.108:6836/6227 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd6c00 con 0x77809a0 -4083> 2014-08-13 16:24:27.653217 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4082> 2014-08-13 16:24:27.653225 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.318 192.168.1.110:6814/43857 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7a880 con 0x77c2260 -4081> 2014-08-13 16:24:27.653238 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4080> 2014-08-13 16:24:27.653247 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.309 192.168.1.101:6814/23703 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b10900 con 0x77fe720 -4079> 2014-08-13 16:24:27.653260 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4078> 2014-08-13 16:24:27.653270 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 4 ==== pg_notify(24.34(29) epoch 194451) v5 ==== 2199+0+0 (1147011204 0 0) 0x8c8a680 con 0x7d2d120 -4077> 2014-08-13 16:24:27.653287 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:25.689055, event: header_read, request: pg_notify(24.34(29) epoch 194451) v5 -4076> 2014-08-13 16:24:27.653297 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:25.689056, event: throttled, request: pg_notify(24.34(29) epoch 194451) v5 -4075> 2014-08-13 16:24:27.653303 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:25.689310, event: all_read, request: pg_notify(24.34(29) epoch 194451) v5 -4074> 2014-08-13 16:24:27.653309 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:27.653283, event: dispatched, request: pg_notify(24.34(29) epoch 194451) v5 -4073> 2014-08-13 16:24:27.653316 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:27.653316, event: waiting_for_osdmap, request: pg_notify(24.34(29) epoch 194451) v5 -4072> 2014-08-13 16:24:27.653327 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:27.653327, event: started, request: pg_notify(24.34(29) epoch 194451) v5 -4071> 2014-08-13 16:24:27.653364 7fd93dff0700 5 -- op tracker -- , seq: 910, time: 2014-08-13 16:24:27.653364, event: done, request: pg_notify(24.34(29) epoch 194451) v5 -4070> 2014-08-13 16:24:27.653394 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.59 192.168.1.112:6822/5265 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b12640 con 0x81aad60 -4069> 2014-08-13 16:24:27.653412 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4068> 2014-08-13 16:24:27.653421 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859f500 con 0x7d2d280 -4067> 2014-08-13 16:24:27.653435 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4066> 2014-08-13 16:24:27.653443 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859ec00 con 0x821f900 -4065> 2014-08-13 16:24:27.653457 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4064> 2014-08-13 16:24:27.653469 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 5 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf3600 con 0x80c7e80 -4063> 2014-08-13 16:24:27.653483 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4062> 2014-08-13 16:24:27.653491 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e79f80 con 0x80c7d20 -4061> 2014-08-13 16:24:27.653505 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4060> 2014-08-13 16:24:27.653513 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.348 192.168.1.110:6800/41204 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7e780 con 0x8238c60 -4059> 2014-08-13 16:24:27.653526 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4058> 2014-08-13 16:24:27.653535 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.10 192.168.1.115:6815/60573 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd7500 con 0x4e49080 -4057> 2014-08-13 16:24:27.653548 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4056> 2014-08-13 16:24:27.653556 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.9 192.168.1.107:6817/53997 21 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859f080 con 0x4e498c0 -4055> 2014-08-13 16:24:27.653570 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4054> 2014-08-13 16:24:27.653578 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.28 192.168.1.103:6805/37391 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8599200 con 0x4e4ad60 -4053> 2014-08-13 16:24:27.653591 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4052> 2014-08-13 16:24:27.653600 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 15 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf21c0 con 0x4e4b860 -4051> 2014-08-13 16:24:27.653613 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4050> 2014-08-13 16:24:27.653620 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.62 192.168.1.112:6824/5505 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8598240 con 0x4e4c200 -4049> 2014-08-13 16:24:27.653634 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4048> 2014-08-13 16:24:27.653642 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 19 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859ba80 con 0x4e4dd80 -4047> 2014-08-13 16:24:27.653655 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4046> 2014-08-13 16:24:27.653664 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd0900 con 0x4e4ce60 -4045> 2014-08-13 16:24:27.653678 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4044> 2014-08-13 16:24:27.653686 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.291 192.168.1.101:6820/25633 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b10000 con 0x4e4e5c0 -4043> 2014-08-13 16:24:27.653699 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4042> 2014-08-13 16:24:27.653707 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.304 192.168.1.101:6808/22782 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf3840 con 0x4e4d800 -4041> 2014-08-13 16:24:27.653720 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4040> 2014-08-13 16:24:27.653737 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 23 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d0b3c0 con 0x4e4f900 -4039> 2014-08-13 16:24:27.653757 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4038> 2014-08-13 16:24:27.653766 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.285 192.168.1.101:6822/26295 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d08240 con 0x4fe0840 -4037> 2014-08-13 16:24:27.653779 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194451], i have 194451, src has [185139,194451] -4036> 2014-08-13 16:24:27.653786 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.345 192.168.1.110:6820/46337 29 ==== osd_map(194451..194452 src has 185139..194452) v3 ==== 1977+0+0 (1444941786 0 0) 0x7cf3cc0 con 0x4fe0000 -4035> 2014-08-13 16:24:27.653832 7fd93dff0700 3 osd.4 194451 handle_osd_map epochs [194451,194452], i have 194451, src has [185139,194452] -4034> 2014-08-13 16:24:27.653832 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 27 ==== osd_map(194451..194452 src has 185139..194452) v3 ==== 1977+0+0 (1444941786 0 0) 0x5070240 con 0x4e3dd80 -4033> 2014-08-13 16:24:27.654741 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4032> 2014-08-13 16:24:27.654763 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194451 {24.39a=[4,395,13]} v194451) v1 -- ?+0 0x801e180 con 0x4e3dd80 -4031> 2014-08-13 16:24:27.655236 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Started/Primary/Peering/WaitActingChange 2.453823 2 0.000273 -4030> 2014-08-13 16:24:27.655284 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Started/Primary 2.453892 0 0.000000 -4029> 2014-08-13 16:24:27.655307 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Started 2.465104 0 0.000000 -4028> 2014-08-13 16:24:27.655329 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194450/194450) [349,4]/[4,395] r=0 lpr=194450 pi=193304-194449/51 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Reset -4027> 2014-08-13 16:24:27.655429 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Reset 0.000101 1 0.000191 -4026> 2014-08-13 16:24:27.655459 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started -4025> 2014-08-13 16:24:27.655478 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Start -4024> 2014-08-13 16:24:27.655500 7fd9371e5700 1 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] state: transitioning to Primary -4023> 2014-08-13 16:24:27.655525 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] exit Start 0.000047 0 0.000000 -4022> 2014-08-13 16:24:27.655549 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary -4021> 2014-08-13 16:24:27.655570 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary/Peering -4020> 2014-08-13 16:24:27.655588 7fd9371e5700 5 osd.4 pg_epoch: 194451 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/GetInfo -4019> 2014-08-13 16:24:27.659021 7fd9371e5700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4018> 2014-08-13 16:24:27.659040 7fd9371e5700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_alive(want up_thru 194451 have 194451) v1 -- ?+0 0x81c0000 con 0x4e3dd80 -4017> 2014-08-13 16:24:27.659062 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.115:6804/58748 -- pg_query(24.39a epoch 194451) v3 -- ?+0 0x8019a40 con 0x4fd9b80 -4016> 2014-08-13 16:24:27.659082 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6800/61726 -- pg_query(24.39a epoch 194451) v3 -- ?+0 0x801aee0 con 0x7313860 -4015> 2014-08-13 16:24:27.659103 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6810/41287 -- pg_query(24.39a epoch 194451) v3 -- ?+0 0x8018960 con 0x51127e0 -4014> 2014-08-13 16:24:27.659382 7fd93dff0700 0 osd.4 194452 crush map has features 2578087936000, adjusting msgr requires for mons -4013> 2014-08-13 16:24:27.663801 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 24 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859af40 con 0x4fd8000 -4012> 2014-08-13 16:24:27.663835 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -4011> 2014-08-13 16:24:27.663851 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 17 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8599d40 con 0x4fe1600 -4010> 2014-08-13 16:24:27.663871 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -4009> 2014-08-13 16:24:27.663881 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.344 192.168.1.106:6812/6757 12 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf4380 con 0x4fe23c0 -4008> 2014-08-13 16:24:27.663903 7fd93f3f2700 3 osd.4 194452 handle_osd_map epochs [194451,194452], i have 194452, src has [185139,194452] -4007> 2014-08-13 16:24:27.663941 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -4006> 2014-08-13 16:24:27.663957 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 20 ==== osd_map(194450..194451 src has 185139..194451) v3 ==== 1774+0+0 (2937653276 0 0) 0x8bd4a40 con 0x4fe3700 -4005> 2014-08-13 16:24:27.663977 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194450,194451], i have 194452, src has [185139,194451] -4004> 2014-08-13 16:24:27.663990 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 17 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf7500 con 0x4fe40a0 -4003> 2014-08-13 16:24:27.664007 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -4002> 2014-08-13 16:24:27.664019 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.364 192.168.1.114:6826/52317 9 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b157c0 con 0x4fe6ca0 -4001> 2014-08-13 16:24:27.664037 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -4000> 2014-08-13 16:24:27.664049 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.56 192.168.1.111:6826/11446 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7ca40 con 0x4fd9340 -3999> 2014-08-13 16:24:27.664067 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3998> 2014-08-13 16:24:27.664086 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 20 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf1d40 con 0x4fe69e0 -3997> 2014-08-13 16:24:27.664103 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3996> 2014-08-13 16:24:27.664116 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.294 192.168.1.101:6812/23402 9 ==== osd_map(194451..194452 src has 185139..194452) v3 ==== 1977+0+0 (1444941786 0 0) 0x8bd4c80 con 0x4fd8b00 -3995> 2014-08-13 16:24:27.664133 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194452], i have 194452, src has [185139,194452] -3994> 2014-08-13 16:24:27.664145 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 19 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x33e3840 con 0x4fda940 -3993> 2014-08-13 16:24:27.664186 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3992> 2014-08-13 16:24:27.664199 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 3 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf2880 con 0x4fd9b80 -3991> 2014-08-13 16:24:27.664217 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3990> 2014-08-13 16:24:27.664231 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.323 192.168.1.106:6816/8118 14 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859b3c0 con 0x4fdd3e0 -3989> 2014-08-13 16:24:27.664248 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3988> 2014-08-13 16:24:27.664260 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.313 192.168.1.109:6820/35445 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d00480 con 0x4fdeca0 -3987> 2014-08-13 16:24:27.664278 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3986> 2014-08-13 16:24:27.664290 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.17 192.168.1.103:6811/37658 14 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd0fc0 con 0x4fde1a0 -3985> 2014-08-13 16:24:27.664307 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3984> 2014-08-13 16:24:27.664319 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.31 192.168.1.103:6831/41157 15 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf4a40 con 0x5089760 -3983> 2014-08-13 16:24:27.664337 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3982> 2014-08-13 16:24:27.664356 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 20 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd2880 con 0x50889a0 -3981> 2014-08-13 16:24:27.664373 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3980> 2014-08-13 16:24:27.664385 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 15 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd3840 con 0x508a680 -3979> 2014-08-13 16:24:27.664402 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3978> 2014-08-13 16:24:27.664414 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.50 192.168.1.112:6842/9607 16 ==== osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+11541 (2969197595 0 1473524713) 0x76f8000 con 0x508c620 -3977> 2014-08-13 16:24:27.664443 7fd93dff0700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.321937, event: header_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3976> 2014-08-13 16:24:27.664461 7fd93dff0700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.321940, event: throttled, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3975> 2014-08-13 16:24:27.664474 7fd93dff0700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.322079, event: all_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3974> 2014-08-13 16:24:27.664486 7fd93dff0700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.664438, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3973> 2014-08-13 16:24:27.664500 7fd93dff0700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.664500, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3972> 2014-08-13 16:24:27.664526 7fd93dff0700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6842/9607 -- osd_map(194451..194452 src has 185139..194452) v3 -- ?+0 0x8bd3840 con 0x508c620 -3971> 2014-08-13 16:24:27.664566 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 29 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e78480 con 0x508bde0 -3970> 2014-08-13 16:24:27.664586 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3969> 2014-08-13 16:24:27.664599 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.21 192.168.1.107:6821/54786 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf45c0 con 0x508b5a0 -3968> 2014-08-13 16:24:27.664617 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3967> 2014-08-13 16:24:27.664628 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.32 192.168.1.115:6812/59915 6 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7af40 con 0x508e5c0 -3966> 2014-08-13 16:24:27.664645 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3965> 2014-08-13 16:24:27.664657 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.343 192.168.1.104:6802/61960 19 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b160c0 con 0x508dd80 -3964> 2014-08-13 16:24:27.664674 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3963> 2014-08-13 16:24:27.664673 7fd9367e4700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.664673, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3962> 2014-08-13 16:24:27.664691 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.5 192.168.1.103:6827/40192 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8598480 con 0x508f220 -3961> 2014-08-13 16:24:27.664722 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3960> 2014-08-13 16:24:27.664717 7fd9367e4700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.664717, event: started, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3959> 2014-08-13 16:24:27.664737 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.358 192.168.1.114:6824/52078 21 ==== osd_map(194451..194452 src has 185139..194452) v3 ==== 1977+0+0 (1444941786 0 0) 0x7cf6e40 con 0x50b9a20 -3958> 2014-08-13 16:24:27.664755 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194452], i have 194452, src has [185139,194452] -3957> 2014-08-13 16:24:27.664768 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.315 192.168.1.113:6800/14161 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7f740 con 0x50b8c60 -3956> 2014-08-13 16:24:27.664785 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3955> 2014-08-13 16:24:27.664794 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.296 192.168.1.109:6806/31286 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7ec00 con 0x50bb020 -3954> 2014-08-13 16:24:27.664807 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3953> 2014-08-13 16:24:27.664815 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 15 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd1f80 con 0x50ba520 -3952> 2014-08-13 16:24:27.664829 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3951> 2014-08-13 16:24:27.664839 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.347 192.168.1.106:6800/4640 11 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf2400 con 0x50bbf40 -3950> 2014-08-13 16:24:27.664853 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3949> 2014-08-13 16:24:27.664850 7fd9367e4700 5 -- op tracker -- , seq: 911, time: 2014-08-13 16:24:27.664849, event: done, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3948> 2014-08-13 16:24:27.664866 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x506f500 con 0x50bfa60 -3947> 2014-08-13 16:24:27.664883 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3946> 2014-08-13 16:24:27.664892 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.6 192.168.1.107:6809/53141 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b17bc0 con 0x5109340 -3945> 2014-08-13 16:24:27.664909 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3944> 2014-08-13 16:24:27.664921 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 16 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7e9c0 con 0x510b9c0 -3943> 2014-08-13 16:24:27.664938 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3942> 2014-08-13 16:24:27.664946 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 16 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7dc40 con 0x5108b00 -3941> 2014-08-13 16:24:27.664963 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3940> 2014-08-13 16:24:27.664975 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.287 192.168.1.109:6814/33435 9 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd72c0 con 0x510ad60 -3939> 2014-08-13 16:24:27.664992 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3938> 2014-08-13 16:24:27.665001 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859b840 con 0x510dac0 -3937> 2014-08-13 16:24:27.665018 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3936> 2014-08-13 16:24:27.665029 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.341 192.168.1.106:6818/9142 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7e7f980 con 0x510a100 -3935> 2014-08-13 16:24:27.665043 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3934> 2014-08-13 16:24:27.665056 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 27 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf4800 con 0x510fd20 -3933> 2014-08-13 16:24:27.665072 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3932> 2014-08-13 16:24:27.665081 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.332 192.168.1.110:6806/42024 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd6540 con 0x510e720 -3931> 2014-08-13 16:24:27.665098 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3930> 2014-08-13 16:24:27.665110 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.286 192.168.1.113:6820/19173 11 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf0480 con 0x510e1a0 -3929> 2014-08-13 16:24:27.665127 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3928> 2014-08-13 16:24:27.665138 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.346 192.168.1.104:6820/2308 10 ==== osd_map(194451..194452 src has 185139..194452) v3 ==== 1977+0+0 (1444941786 0 0) 0x8bd4140 con 0x5112d60 -3927> 2014-08-13 16:24:27.665168 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194452], i have 194452, src has [185139,194452] -3926> 2014-08-13 16:24:27.665179 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 4 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859ee40 con 0x51127e0 -3925> 2014-08-13 16:24:27.665193 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3924> 2014-08-13 16:24:27.665201 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 32 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d0ad00 con 0x5112520 -3923> 2014-08-13 16:24:27.665214 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3922> 2014-08-13 16:24:27.665222 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859c5c0 con 0x5112260 -3921> 2014-08-13 16:24:27.665235 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3920> 2014-08-13 16:24:27.665243 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.351 192.168.1.102:6830/16444 13 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x506f980 con 0x51148e0 -3919> 2014-08-13 16:24:27.665256 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3918> 2014-08-13 16:24:27.665266 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 11 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8d09680 con 0x5114620 -3917> 2014-08-13 16:24:27.665280 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3916> 2014-08-13 16:24:27.665288 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 18 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b16e40 con 0x5114360 -3915> 2014-08-13 16:24:27.665312 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3914> 2014-08-13 16:24:27.665321 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.374 192.168.1.102:6828/16129 15 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859a400 con 0x51140a0 -3913> 2014-08-13 16:24:27.665334 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3912> 2014-08-13 16:24:27.665342 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 8 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859ca40 con 0x5113de0 -3911> 2014-08-13 16:24:27.665355 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3910> 2014-08-13 16:24:27.665363 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 18 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859aac0 con 0x5113b20 -3909> 2014-08-13 16:24:27.665376 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3908> 2014-08-13 16:24:27.665384 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 12 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7cf2d00 con 0x5113860 -3907> 2014-08-13 16:24:27.665396 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3906> 2014-08-13 16:24:27.665404 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 25 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8599f80 con 0x5111fa0 -3905> 2014-08-13 16:24:27.665417 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3904> 2014-08-13 16:24:27.665425 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 30 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x8bd0240 con 0x51135a0 -3903> 2014-08-13 16:24:27.665438 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3902> 2014-08-13 16:24:27.665451 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 22 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x859cec0 con 0x5111a20 -3901> 2014-08-13 16:24:27.665469 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3900> 2014-08-13 16:24:27.665477 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.375 192.168.1.114:6830/52879 7 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b15100 con 0x5117220 -3899> 2014-08-13 16:24:27.665491 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3898> 2014-08-13 16:24:27.665503 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 16 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x7b12400 con 0x7313860 -3897> 2014-08-13 16:24:27.665519 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3896> 2014-08-13 16:24:27.665528 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 10 ==== osd_map(194451..194451 src has 185139..194451) v3 ==== 1595+0+0 (2946913855 0 0) 0x506e780 con 0x7316720 -3895> 2014-08-13 16:24:27.665544 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194451,194451], i have 194452, src has [185139,194451] -3894> 2014-08-13 16:24:27.665556 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 13 ==== pg_notify(24.39a(28) epoch 194451) v5 ==== 2300+0+0 (11361555 0 0) 0x81c16c0 con 0x7315ee0 -3893> 2014-08-13 16:24:27.665574 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:25.689702, event: header_read, request: pg_notify(24.39a(28) epoch 194451) v5 -3892> 2014-08-13 16:24:27.665586 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:25.689704, event: throttled, request: pg_notify(24.39a(28) epoch 194451) v5 -3891> 2014-08-13 16:24:27.665595 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:25.689804, event: all_read, request: pg_notify(24.39a(28) epoch 194451) v5 -3890> 2014-08-13 16:24:27.665604 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:27.665572, event: dispatched, request: pg_notify(24.39a(28) epoch 194451) v5 -3889> 2014-08-13 16:24:27.665614 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:27.665614, event: waiting_for_osdmap, request: pg_notify(24.39a(28) epoch 194451) v5 -3888> 2014-08-13 16:24:27.665626 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:27.665625, event: started, request: pg_notify(24.39a(28) epoch 194451) v5 -3887> 2014-08-13 16:24:27.665657 7fd93dff0700 5 -- op tracker -- , seq: 912, time: 2014-08-13 16:24:27.665657, event: done, request: pg_notify(24.39a(28) epoch 194451) v5 -3886> 2014-08-13 16:24:27.665691 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 11 ==== pg_notify(22.113(4) epoch 194451) v5 ==== 942+0+0 (452618792 0 0) 0x80d88c0 con 0x7314fc0 -3885> 2014-08-13 16:24:27.665711 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:25.715649, event: header_read, request: pg_notify(22.113(4) epoch 194451) v5 -3884> 2014-08-13 16:24:27.665721 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:25.715650, event: throttled, request: pg_notify(22.113(4) epoch 194451) v5 -3883> 2014-08-13 16:24:27.665730 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:25.715718, event: all_read, request: pg_notify(22.113(4) epoch 194451) v5 -3882> 2014-08-13 16:24:27.665739 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:27.665709, event: dispatched, request: pg_notify(22.113(4) epoch 194451) v5 -3881> 2014-08-13 16:24:27.665749 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:27.665749, event: waiting_for_osdmap, request: pg_notify(22.113(4) epoch 194451) v5 -3880> 2014-08-13 16:24:27.665760 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:27.665760, event: started, request: pg_notify(22.113(4) epoch 194451) v5 -3879> 2014-08-13 16:24:27.665784 7fd93dff0700 5 -- op tracker -- , seq: 913, time: 2014-08-13 16:24:27.665784, event: done, request: pg_notify(22.113(4) epoch 194451) v5 -3878> 2014-08-13 16:24:27.665824 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 9 ==== pg_notify(22.22c(4) epoch 194451) v5 ==== 942+0+0 (2122108286 0 0) 0x899b2c0 con 0x7316f60 -3877> 2014-08-13 16:24:27.665844 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:25.709643, event: header_read, request: pg_notify(22.22c(4) epoch 194451) v5 -3876> 2014-08-13 16:24:27.665854 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:25.709644, event: throttled, request: pg_notify(22.22c(4) epoch 194451) v5 -3875> 2014-08-13 16:24:27.665863 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:25.709714, event: all_read, request: pg_notify(22.22c(4) epoch 194451) v5 -3874> 2014-08-13 16:24:27.665872 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:27.665842, event: dispatched, request: pg_notify(22.22c(4) epoch 194451) v5 -3873> 2014-08-13 16:24:27.665881 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:27.665881, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194451) v5 -3872> 2014-08-13 16:24:27.665893 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:27.665892, event: started, request: pg_notify(22.22c(4) epoch 194451) v5 -3871> 2014-08-13 16:24:27.665919 7fd93dff0700 5 -- op tracker -- , seq: 914, time: 2014-08-13 16:24:27.665918, event: done, request: pg_notify(22.22c(4) epoch 194451) v5 -3870> 2014-08-13 16:24:27.665950 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 8 ==== pg_notify(22.619(17) epoch 194451) v5 ==== 1665+0+0 (3643341703 0 0) 0x834e740 con 0x8da1760 -3869> 2014-08-13 16:24:27.665970 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:25.704514, event: header_read, request: pg_notify(22.619(17) epoch 194451) v5 -3868> 2014-08-13 16:24:27.665981 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:25.704515, event: throttled, request: pg_notify(22.619(17) epoch 194451) v5 -3867> 2014-08-13 16:24:27.665990 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:25.704634, event: all_read, request: pg_notify(22.619(17) epoch 194451) v5 -3866> 2014-08-13 16:24:27.665999 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:27.665968, event: dispatched, request: pg_notify(22.619(17) epoch 194451) v5 -3865> 2014-08-13 16:24:27.666009 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:27.666008, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194451) v5 -3864> 2014-08-13 16:24:27.666020 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:27.666020, event: started, request: pg_notify(22.619(17) epoch 194451) v5 -3863> 2014-08-13 16:24:27.666043 7fd93dff0700 5 -- op tracker -- , seq: 915, time: 2014-08-13 16:24:27.666042, event: done, request: pg_notify(22.619(17) epoch 194451) v5 -3862> 2014-08-13 16:24:27.666073 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 9 ==== pg_notify(22.f3(15) epoch 194451) v5 ==== 1551+0+0 (313567014 0 0) 0x840d780 con 0x8da5120 -3861> 2014-08-13 16:24:27.666093 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:25.714938, event: header_read, request: pg_notify(22.f3(15) epoch 194451) v5 -3860> 2014-08-13 16:24:27.666104 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:25.714939, event: throttled, request: pg_notify(22.f3(15) epoch 194451) v5 -3859> 2014-08-13 16:24:27.666113 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:25.715026, event: all_read, request: pg_notify(22.f3(15) epoch 194451) v5 -3858> 2014-08-13 16:24:27.666122 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:27.666091, event: dispatched, request: pg_notify(22.f3(15) epoch 194451) v5 -3857> 2014-08-13 16:24:27.666128 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:27.666128, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194451) v5 -3856> 2014-08-13 16:24:27.666139 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:27.666139, event: started, request: pg_notify(22.f3(15) epoch 194451) v5 -3855> 2014-08-13 16:24:27.666414 7fd93dff0700 5 -- op tracker -- , seq: 916, time: 2014-08-13 16:24:27.666414, event: done, request: pg_notify(22.f3(15) epoch 194451) v5 -3854> 2014-08-13 16:24:27.666440 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 11 ==== pg_notify(22.99(4),22.39d(15) epoch 194451) v5 ==== 2481+0+0 (584719010 0 0) 0x80deac0 con 0x72d9b80 -3853> 2014-08-13 16:24:27.666462 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:25.716120, event: header_read, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3852> 2014-08-13 16:24:27.666472 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:25.716121, event: throttled, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3851> 2014-08-13 16:24:27.666482 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:25.716270, event: all_read, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3850> 2014-08-13 16:24:27.666491 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:27.666459, event: dispatched, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3849> 2014-08-13 16:24:27.666501 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:27.666501, event: waiting_for_osdmap, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3848> 2014-08-13 16:24:27.666513 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:27.666513, event: started, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3847> 2014-08-13 16:24:27.666551 7fd93dff0700 5 -- op tracker -- , seq: 917, time: 2014-08-13 16:24:27.666550, event: done, request: pg_notify(22.99(4),22.39d(15) epoch 194451) v5 -3846> 2014-08-13 16:24:27.666578 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 9 ==== pg_notify(22.4df(4) epoch 194451) v5 ==== 942+0+0 (1635340919 0 0) 0x87e5b00 con 0x72dfa60 -3845> 2014-08-13 16:24:27.666602 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:25.698050, event: header_read, request: pg_notify(22.4df(4) epoch 194451) v5 -3844> 2014-08-13 16:24:27.666621 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:25.698051, event: throttled, request: pg_notify(22.4df(4) epoch 194451) v5 -3843> 2014-08-13 16:24:27.666631 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:25.698114, event: all_read, request: pg_notify(22.4df(4) epoch 194451) v5 -3842> 2014-08-13 16:24:27.666640 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:27.666599, event: dispatched, request: pg_notify(22.4df(4) epoch 194451) v5 -3841> 2014-08-13 16:24:27.666650 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:27.666649, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194451) v5 -3840> 2014-08-13 16:24:27.666658 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:27.666658, event: started, request: pg_notify(22.4df(4) epoch 194451) v5 -3839> 2014-08-13 16:24:27.666681 7fd93dff0700 5 -- op tracker -- , seq: 918, time: 2014-08-13 16:24:27.666681, event: done, request: pg_notify(22.4df(4) epoch 194451) v5 -3838> 2014-08-13 16:24:27.666702 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 8 ==== pg_notify(22.39d(4) epoch 194451) v5 ==== 942+0+0 (525251427 0 0) 0x80dabc0 con 0x72de9e0 -3837> 2014-08-13 16:24:27.666721 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:25.719891, event: header_read, request: pg_notify(22.39d(4) epoch 194451) v5 -3836> 2014-08-13 16:24:27.666731 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:25.719893, event: throttled, request: pg_notify(22.39d(4) epoch 194451) v5 -3835> 2014-08-13 16:24:27.666741 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:25.719993, event: all_read, request: pg_notify(22.39d(4) epoch 194451) v5 -3834> 2014-08-13 16:24:27.666749 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:27.666719, event: dispatched, request: pg_notify(22.39d(4) epoch 194451) v5 -3833> 2014-08-13 16:24:27.666759 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:27.666759, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194451) v5 -3832> 2014-08-13 16:24:27.666770 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:27.666769, event: started, request: pg_notify(22.39d(4) epoch 194451) v5 -3831> 2014-08-13 16:24:27.666792 7fd93dff0700 5 -- op tracker -- , seq: 919, time: 2014-08-13 16:24:27.666792, event: done, request: pg_notify(22.39d(4) epoch 194451) v5 -3830> 2014-08-13 16:24:27.666812 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 9 ==== pg_notify(22.775(17) epoch 194451) v5 ==== 1665+0+0 (2565930218 0 0) 0x9104d00 con 0x8e29fa0 -3829> 2014-08-13 16:24:27.666831 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:25.727567, event: header_read, request: pg_notify(22.775(17) epoch 194451) v5 -3828> 2014-08-13 16:24:27.666841 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:25.727568, event: throttled, request: pg_notify(22.775(17) epoch 194451) v5 -3827> 2014-08-13 16:24:27.666850 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:25.727652, event: all_read, request: pg_notify(22.775(17) epoch 194451) v5 -3826> 2014-08-13 16:24:27.666859 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:27.666829, event: dispatched, request: pg_notify(22.775(17) epoch 194451) v5 -3825> 2014-08-13 16:24:27.666869 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:27.666869, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194451) v5 -3824> 2014-08-13 16:24:27.666880 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:27.666880, event: started, request: pg_notify(22.775(17) epoch 194451) v5 -3823> 2014-08-13 16:24:27.666902 7fd93dff0700 5 -- op tracker -- , seq: 920, time: 2014-08-13 16:24:27.666902, event: done, request: pg_notify(22.775(17) epoch 194451) v5 -3822> 2014-08-13 16:24:27.666941 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 14 ==== pg_notify(22.3af(17) epoch 194451) v5 ==== 1665+0+0 (1038997263 0 0) 0x9105b00 con 0x8e286e0 -3821> 2014-08-13 16:24:27.666961 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:25.704948, event: header_read, request: pg_notify(22.3af(17) epoch 194451) v5 -3820> 2014-08-13 16:24:27.666971 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:25.704950, event: throttled, request: pg_notify(22.3af(17) epoch 194451) v5 -3819> 2014-08-13 16:24:27.666980 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:25.705058, event: all_read, request: pg_notify(22.3af(17) epoch 194451) v5 -3818> 2014-08-13 16:24:27.666989 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:27.666958, event: dispatched, request: pg_notify(22.3af(17) epoch 194451) v5 -3817> 2014-08-13 16:24:27.666999 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:27.666998, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194451) v5 -3816> 2014-08-13 16:24:27.667010 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:27.667009, event: started, request: pg_notify(22.3af(17) epoch 194451) v5 -3815> 2014-08-13 16:24:27.667032 7fd93dff0700 5 -- op tracker -- , seq: 921, time: 2014-08-13 16:24:27.667032, event: done, request: pg_notify(22.3af(17) epoch 194451) v5 -3814> 2014-08-13 16:24:27.667053 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 8 ==== pg_notify(22.4df(15) epoch 194451) v5 ==== 1551+0+0 (1050590978 0 0) 0x842f540 con 0x8e2c360 -3813> 2014-08-13 16:24:27.667072 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:25.901707, event: header_read, request: pg_notify(22.4df(15) epoch 194451) v5 -3812> 2014-08-13 16:24:27.667082 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:25.901708, event: throttled, request: pg_notify(22.4df(15) epoch 194451) v5 -3811> 2014-08-13 16:24:27.667091 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:25.901812, event: all_read, request: pg_notify(22.4df(15) epoch 194451) v5 -3810> 2014-08-13 16:24:27.667097 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:27.667070, event: dispatched, request: pg_notify(22.4df(15) epoch 194451) v5 -3809> 2014-08-13 16:24:27.667108 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:27.667107, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194451) v5 -3808> 2014-08-13 16:24:27.667118 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:27.667118, event: started, request: pg_notify(22.4df(15) epoch 194451) v5 -3807> 2014-08-13 16:24:27.667141 7fd93dff0700 5 -- op tracker -- , seq: 922, time: 2014-08-13 16:24:27.667140, event: done, request: pg_notify(22.4df(15) epoch 194451) v5 -3806> 2014-08-13 16:24:27.667182 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 9 ==== pg_notify(24.181(25) epoch 194451) v5 ==== 2159+0+0 (1119443479 0 0) 0x9110e00 con 0x7379ce0 -3805> 2014-08-13 16:24:27.667205 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:25.712909, event: header_read, request: pg_notify(24.181(25) epoch 194451) v5 -3804> 2014-08-13 16:24:27.667216 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:25.712910, event: throttled, request: pg_notify(24.181(25) epoch 194451) v5 -3803> 2014-08-13 16:24:27.667225 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:25.713008, event: all_read, request: pg_notify(24.181(25) epoch 194451) v5 -3802> 2014-08-13 16:24:27.667234 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:27.667203, event: dispatched, request: pg_notify(24.181(25) epoch 194451) v5 -3801> 2014-08-13 16:24:27.667244 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:27.667244, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194451) v5 -3800> 2014-08-13 16:24:27.667255 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:27.667255, event: started, request: pg_notify(24.181(25) epoch 194451) v5 -3799> 2014-08-13 16:24:27.667280 7fd93dff0700 5 -- op tracker -- , seq: 923, time: 2014-08-13 16:24:27.667279, event: done, request: pg_notify(24.181(25) epoch 194451) v5 -3798> 2014-08-13 16:24:27.667301 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 9 ==== pg_notify(22.1f1(4) epoch 194451) v5 ==== 942+0+0 (2015394444 0 0) 0x87e78c0 con 0x73ca680 -3797> 2014-08-13 16:24:27.667320 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:25.712590, event: header_read, request: pg_notify(22.1f1(4) epoch 194451) v5 -3796> 2014-08-13 16:24:27.667331 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:25.712591, event: throttled, request: pg_notify(22.1f1(4) epoch 194451) v5 -3795> 2014-08-13 16:24:27.667340 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:25.712667, event: all_read, request: pg_notify(22.1f1(4) epoch 194451) v5 -3794> 2014-08-13 16:24:27.667348 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:27.667318, event: dispatched, request: pg_notify(22.1f1(4) epoch 194451) v5 -3793> 2014-08-13 16:24:27.667358 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:27.667358, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194451) v5 -3792> 2014-08-13 16:24:27.667369 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:27.667369, event: started, request: pg_notify(22.1f1(4) epoch 194451) v5 -3791> 2014-08-13 16:24:27.667392 7fd93dff0700 5 -- op tracker -- , seq: 924, time: 2014-08-13 16:24:27.667392, event: done, request: pg_notify(22.1f1(4) epoch 194451) v5 -3790> 2014-08-13 16:24:27.667412 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 15 ==== pg_notify(22.3af(17) epoch 194451) v5 ==== 1665+0+0 (1038997263 0 0) 0x8c89500 con 0x73cdc20 -3789> 2014-08-13 16:24:27.667432 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:25.717471, event: header_read, request: pg_notify(22.3af(17) epoch 194451) v5 -3788> 2014-08-13 16:24:27.667442 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:25.717474, event: throttled, request: pg_notify(22.3af(17) epoch 194451) v5 -3787> 2014-08-13 16:24:27.667451 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:25.717583, event: all_read, request: pg_notify(22.3af(17) epoch 194451) v5 -3786> 2014-08-13 16:24:27.667460 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:27.667430, event: dispatched, request: pg_notify(22.3af(17) epoch 194451) v5 -3785> 2014-08-13 16:24:27.667469 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:27.667469, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194451) v5 -3784> 2014-08-13 16:24:27.667480 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:27.667480, event: started, request: pg_notify(22.3af(17) epoch 194451) v5 -3783> 2014-08-13 16:24:27.667503 7fd93dff0700 5 -- op tracker -- , seq: 925, time: 2014-08-13 16:24:27.667502, event: done, request: pg_notify(22.3af(17) epoch 194451) v5 -3782> 2014-08-13 16:24:27.667525 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 6 ==== pg_notify(22.481(32) epoch 194451) v5 ==== 2518+0+0 (3481752237 0 0) 0x87e6200 con 0x73ccba0 -3781> 2014-08-13 16:24:27.667544 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:25.712541, event: header_read, request: pg_notify(22.481(32) epoch 194451) v5 -3780> 2014-08-13 16:24:27.667554 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:25.712542, event: throttled, request: pg_notify(22.481(32) epoch 194451) v5 -3779> 2014-08-13 16:24:27.667563 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:25.712662, event: all_read, request: pg_notify(22.481(32) epoch 194451) v5 -3778> 2014-08-13 16:24:27.667572 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:27.667542, event: dispatched, request: pg_notify(22.481(32) epoch 194451) v5 -3777> 2014-08-13 16:24:27.667581 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:27.667581, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194451) v5 -3776> 2014-08-13 16:24:27.667592 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:27.667592, event: started, request: pg_notify(22.481(32) epoch 194451) v5 -3775> 2014-08-13 16:24:27.667615 7fd93dff0700 5 -- op tracker -- , seq: 926, time: 2014-08-13 16:24:27.667615, event: done, request: pg_notify(22.481(32) epoch 194451) v5 -3774> 2014-08-13 16:24:27.667635 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 6 ==== pg_notify(22.3d9(35) epoch 194451) v5 ==== 2603+0+0 (3615099036 0 0) 0x834fa80 con 0x7462100 -3773> 2014-08-13 16:24:27.667654 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:25.977762, event: header_read, request: pg_notify(22.3d9(35) epoch 194451) v5 -3772> 2014-08-13 16:24:27.667664 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:25.977763, event: throttled, request: pg_notify(22.3d9(35) epoch 194451) v5 -3771> 2014-08-13 16:24:27.667673 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:25.977882, event: all_read, request: pg_notify(22.3d9(35) epoch 194451) v5 -3770> 2014-08-13 16:24:27.667682 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:27.667652, event: dispatched, request: pg_notify(22.3d9(35) epoch 194451) v5 -3769> 2014-08-13 16:24:27.667691 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:27.667691, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194451) v5 -3768> 2014-08-13 16:24:27.667702 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:27.667702, event: started, request: pg_notify(22.3d9(35) epoch 194451) v5 -3767> 2014-08-13 16:24:27.667725 7fd93dff0700 5 -- op tracker -- , seq: 927, time: 2014-08-13 16:24:27.667725, event: done, request: pg_notify(22.3d9(35) epoch 194451) v5 -3766> 2014-08-13 16:24:27.667745 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 12 ==== pg_notify(24.9b(4) epoch 194451) v5 ==== 976+0+0 (4227211403 0 0) 0x899d080 con 0x7723440 -3765> 2014-08-13 16:24:27.667764 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:25.705151, event: header_read, request: pg_notify(24.9b(4) epoch 194451) v5 -3764> 2014-08-13 16:24:27.667774 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:25.705153, event: throttled, request: pg_notify(24.9b(4) epoch 194451) v5 -3763> 2014-08-13 16:24:27.667783 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:25.705233, event: all_read, request: pg_notify(24.9b(4) epoch 194451) v5 -3762> 2014-08-13 16:24:27.667793 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:27.667762, event: dispatched, request: pg_notify(24.9b(4) epoch 194451) v5 -3761> 2014-08-13 16:24:27.667803 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:27.667803, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194451) v5 -3760> 2014-08-13 16:24:27.667814 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:27.667813, event: started, request: pg_notify(24.9b(4) epoch 194451) v5 -3759> 2014-08-13 16:24:27.667837 7fd93dff0700 5 -- op tracker -- , seq: 928, time: 2014-08-13 16:24:27.667837, event: done, request: pg_notify(24.9b(4) epoch 194451) v5 -3758> 2014-08-13 16:24:27.667857 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 5 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7b16540 con 0x7d2d120 -3757> 2014-08-13 16:24:27.667878 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3756> 2014-08-13 16:24:27.667891 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 8 ==== pg_notify(22.619(17) epoch 194451) v5 ==== 1665+0+0 (3643341703 0 0) 0x834c280 con 0x7d2d280 -3755> 2014-08-13 16:24:27.667908 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:25.699837, event: header_read, request: pg_notify(22.619(17) epoch 194451) v5 -3754> 2014-08-13 16:24:27.667918 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:25.699839, event: throttled, request: pg_notify(22.619(17) epoch 194451) v5 -3753> 2014-08-13 16:24:27.667939 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:25.699947, event: all_read, request: pg_notify(22.619(17) epoch 194451) v5 -3752> 2014-08-13 16:24:27.667948 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:27.667906, event: dispatched, request: pg_notify(22.619(17) epoch 194451) v5 -3751> 2014-08-13 16:24:27.667957 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:27.667957, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194451) v5 -3750> 2014-08-13 16:24:27.667968 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:27.667968, event: started, request: pg_notify(22.619(17) epoch 194451) v5 -3749> 2014-08-13 16:24:27.667997 7fd93dff0700 5 -- op tracker -- , seq: 929, time: 2014-08-13 16:24:27.667997, event: done, request: pg_notify(22.619(17) epoch 194451) v5 -3748> 2014-08-13 16:24:27.668032 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 8 ==== pg_notify(22.775(17) epoch 194451) v5 ==== 1665+0+0 (2770578126 0 0) 0x80da840 con 0x821f900 -3747> 2014-08-13 16:24:27.668053 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:25.715216, event: header_read, request: pg_notify(22.775(17) epoch 194451) v5 -3746> 2014-08-13 16:24:27.668060 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:25.715217, event: throttled, request: pg_notify(22.775(17) epoch 194451) v5 -3745> 2014-08-13 16:24:27.668069 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:25.715332, event: all_read, request: pg_notify(22.775(17) epoch 194451) v5 -3744> 2014-08-13 16:24:27.668078 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:27.668050, event: dispatched, request: pg_notify(22.775(17) epoch 194451) v5 -3743> 2014-08-13 16:24:27.668088 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:27.668088, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194451) v5 -3742> 2014-08-13 16:24:27.668096 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:27.668096, event: started, request: pg_notify(22.775(17) epoch 194451) v5 -3741> 2014-08-13 16:24:27.668118 7fd93dff0700 5 -- op tracker -- , seq: 930, time: 2014-08-13 16:24:27.668118, event: done, request: pg_notify(22.775(17) epoch 194451) v5 -3740> 2014-08-13 16:24:27.668140 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 6 ==== pg_notify(22.3d9(39) epoch 194451) v5 ==== 2815+0+0 (3525312804 0 0) 0x81c7a80 con 0x80c7e80 -3739> 2014-08-13 16:24:27.668166 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:25.724497, event: header_read, request: pg_notify(22.3d9(39) epoch 194451) v5 -3738> 2014-08-13 16:24:27.668178 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:25.724498, event: throttled, request: pg_notify(22.3d9(39) epoch 194451) v5 -3737> 2014-08-13 16:24:27.668187 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:25.724619, event: all_read, request: pg_notify(22.3d9(39) epoch 194451) v5 -3736> 2014-08-13 16:24:27.668196 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:27.668164, event: dispatched, request: pg_notify(22.3d9(39) epoch 194451) v5 -3735> 2014-08-13 16:24:27.668206 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:27.668206, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194451) v5 -3734> 2014-08-13 16:24:27.668217 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:27.668217, event: started, request: pg_notify(22.3d9(39) epoch 194451) v5 -3733> 2014-08-13 16:24:27.668240 7fd93dff0700 5 -- op tracker -- , seq: 931, time: 2014-08-13 16:24:27.668240, event: done, request: pg_notify(22.3d9(39) epoch 194451) v5 -3732> 2014-08-13 16:24:27.668268 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 8 ==== pg_notify(22.481(34) epoch 194451) v5 ==== 2620+0+0 (2194483090 0 0) 0x8349dc0 con 0x80c7d20 -3731> 2014-08-13 16:24:27.668287 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:25.702103, event: header_read, request: pg_notify(22.481(34) epoch 194451) v5 -3730> 2014-08-13 16:24:27.668298 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:25.702104, event: throttled, request: pg_notify(22.481(34) epoch 194451) v5 -3729> 2014-08-13 16:24:27.668307 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:25.702285, event: all_read, request: pg_notify(22.481(34) epoch 194451) v5 -3728> 2014-08-13 16:24:27.668316 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:27.668285, event: dispatched, request: pg_notify(22.481(34) epoch 194451) v5 -3727> 2014-08-13 16:24:27.668325 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:27.668325, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194451) v5 -3726> 2014-08-13 16:24:27.668336 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:27.668336, event: started, request: pg_notify(22.481(34) epoch 194451) v5 -3725> 2014-08-13 16:24:27.668359 7fd93dff0700 5 -- op tracker -- , seq: 932, time: 2014-08-13 16:24:27.668358, event: done, request: pg_notify(22.481(34) epoch 194451) v5 -3724> 2014-08-13 16:24:27.668396 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 16 ==== pg_notify(24.30b(4) epoch 194451) v5 ==== 976+0+0 (3004822957 0 0) 0x834d780 con 0x4e4b860 -3723> 2014-08-13 16:24:27.668416 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:25.696662, event: header_read, request: pg_notify(24.30b(4) epoch 194451) v5 -3722> 2014-08-13 16:24:27.668427 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:25.696666, event: throttled, request: pg_notify(24.30b(4) epoch 194451) v5 -3721> 2014-08-13 16:24:27.668436 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:25.696768, event: all_read, request: pg_notify(24.30b(4) epoch 194451) v5 -3720> 2014-08-13 16:24:27.668445 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:27.668414, event: dispatched, request: pg_notify(24.30b(4) epoch 194451) v5 -3719> 2014-08-13 16:24:27.668454 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:27.668454, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194451) v5 -3718> 2014-08-13 16:24:27.668465 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:27.668465, event: started, request: pg_notify(24.30b(4) epoch 194451) v5 -3717> 2014-08-13 16:24:27.668488 7fd93dff0700 5 -- op tracker -- , seq: 933, time: 2014-08-13 16:24:27.668488, event: done, request: pg_notify(24.30b(4) epoch 194451) v5 -3716> 2014-08-13 16:24:27.668508 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 20 ==== pg_notify(22.1d2(17) epoch 194451) v5 ==== 1669+0+0 (491290162 0 0) 0x81c2840 con 0x4e4dd80 -3715> 2014-08-13 16:24:27.668527 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:25.718913, event: header_read, request: pg_notify(22.1d2(17) epoch 194451) v5 -3714> 2014-08-13 16:24:27.668537 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:25.718915, event: throttled, request: pg_notify(22.1d2(17) epoch 194451) v5 -3713> 2014-08-13 16:24:27.668545 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:25.719011, event: all_read, request: pg_notify(22.1d2(17) epoch 194451) v5 -3712> 2014-08-13 16:24:27.668554 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:27.668525, event: dispatched, request: pg_notify(22.1d2(17) epoch 194451) v5 -3711> 2014-08-13 16:24:27.668564 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:27.668563, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194451) v5 -3710> 2014-08-13 16:24:27.668574 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:27.668574, event: started, request: pg_notify(22.1d2(17) epoch 194451) v5 -3709> 2014-08-13 16:24:27.668597 7fd93dff0700 5 -- op tracker -- , seq: 934, time: 2014-08-13 16:24:27.668596, event: done, request: pg_notify(22.1d2(17) epoch 194451) v5 -3708> 2014-08-13 16:24:27.668627 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 14 ==== pg_notify(22.1f1(17) epoch 194451) v5 ==== 1665+0+0 (3225697761 0 0) 0x8999f80 con 0x4e4ce60 -3707> 2014-08-13 16:24:27.668657 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:25.722217, event: header_read, request: pg_notify(22.1f1(17) epoch 194451) v5 -3706> 2014-08-13 16:24:27.668667 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:25.722219, event: throttled, request: pg_notify(22.1f1(17) epoch 194451) v5 -3705> 2014-08-13 16:24:27.668676 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:25.722344, event: all_read, request: pg_notify(22.1f1(17) epoch 194451) v5 -3704> 2014-08-13 16:24:27.668685 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:27.668654, event: dispatched, request: pg_notify(22.1f1(17) epoch 194451) v5 -3703> 2014-08-13 16:24:27.668695 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:27.668695, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194451) v5 -3702> 2014-08-13 16:24:27.668706 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:27.668706, event: started, request: pg_notify(22.1f1(17) epoch 194451) v5 -3701> 2014-08-13 16:24:27.668728 7fd93dff0700 5 -- op tracker -- , seq: 935, time: 2014-08-13 16:24:27.668728, event: done, request: pg_notify(22.1f1(17) epoch 194451) v5 -3700> 2014-08-13 16:24:27.668750 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 24 ==== pg_notify(24.34(42) epoch 194451) v5 ==== 2882+0+0 (2490577655 0 0) 0x80de900 con 0x4e4f900 -3699> 2014-08-13 16:24:27.668769 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:25.714806, event: header_read, request: pg_notify(24.34(42) epoch 194451) v5 -3698> 2014-08-13 16:24:27.668780 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:25.714807, event: throttled, request: pg_notify(24.34(42) epoch 194451) v5 -3697> 2014-08-13 16:24:27.668789 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:25.714932, event: all_read, request: pg_notify(24.34(42) epoch 194451) v5 -3696> 2014-08-13 16:24:27.668798 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:27.668767, event: dispatched, request: pg_notify(24.34(42) epoch 194451) v5 -3695> 2014-08-13 16:24:27.668808 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:27.668808, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194451) v5 -3694> 2014-08-13 16:24:27.668819 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:27.668819, event: started, request: pg_notify(24.34(42) epoch 194451) v5 -3693> 2014-08-13 16:24:27.668843 7fd93dff0700 5 -- op tracker -- , seq: 936, time: 2014-08-13 16:24:27.668843, event: done, request: pg_notify(24.34(42) epoch 194451) v5 -3692> 2014-08-13 16:24:27.668866 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.285 192.168.1.101:6822/26295 14 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d08000 con 0x4fe0840 -3691> 2014-08-13 16:24:27.668887 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3690> 2014-08-13 16:24:27.668899 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 25 ==== pg_notify(24.136(17) epoch 194451) v5 ==== 1695+0+0 (451314554 0 0) 0x81c3480 con 0x4fd8000 -3689> 2014-08-13 16:24:27.668917 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:25.722889, event: header_read, request: pg_notify(24.136(17) epoch 194451) v5 -3688> 2014-08-13 16:24:27.668927 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:25.722891, event: throttled, request: pg_notify(24.136(17) epoch 194451) v5 -3687> 2014-08-13 16:24:27.668936 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:25.722999, event: all_read, request: pg_notify(24.136(17) epoch 194451) v5 -3686> 2014-08-13 16:24:27.668945 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:27.668915, event: dispatched, request: pg_notify(24.136(17) epoch 194451) v5 -3685> 2014-08-13 16:24:27.668954 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:27.668954, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194451) v5 -3684> 2014-08-13 16:24:27.668965 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:27.668965, event: started, request: pg_notify(24.136(17) epoch 194451) v5 -3683> 2014-08-13 16:24:27.669766 7fd93dff0700 5 -- op tracker -- , seq: 937, time: 2014-08-13 16:24:27.669766, event: done, request: pg_notify(24.136(17) epoch 194451) v5 -3682> 2014-08-13 16:24:27.669792 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 18 ==== pg_notify(22.99(15) epoch 194451) v5 ==== 1551+0+0 (1451939982 0 0) 0x80dad80 con 0x4fe1600 -3681> 2014-08-13 16:24:27.669812 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:25.781774, event: header_read, request: pg_notify(22.99(15) epoch 194451) v5 -3680> 2014-08-13 16:24:27.669822 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:25.781775, event: throttled, request: pg_notify(22.99(15) epoch 194451) v5 -3679> 2014-08-13 16:24:27.669832 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:25.781864, event: all_read, request: pg_notify(22.99(15) epoch 194451) v5 -3678> 2014-08-13 16:24:27.669840 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:27.669810, event: dispatched, request: pg_notify(22.99(15) epoch 194451) v5 -3677> 2014-08-13 16:24:27.669850 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:27.669850, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194451) v5 -3676> 2014-08-13 16:24:27.669861 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:27.669861, event: started, request: pg_notify(22.99(15) epoch 194451) v5 -3675> 2014-08-13 16:24:27.669884 7fd93dff0700 5 -- op tracker -- , seq: 938, time: 2014-08-13 16:24:27.669884, event: done, request: pg_notify(22.99(15) epoch 194451) v5 -3674> 2014-08-13 16:24:27.669910 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 21 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2454802094 0 0) 0x7976300 con 0x4fe3700 -3673> 2014-08-13 16:24:27.669936 7fd93dff0700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:26.055689, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3672> 2014-08-13 16:24:27.669950 7fd93dff0700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:26.055691, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3671> 2014-08-13 16:24:27.669963 7fd93dff0700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:26.055785, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3670> 2014-08-13 16:24:27.669976 7fd93dff0700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:27.669933, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3669> 2014-08-13 16:24:27.669990 7fd93dff0700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:27.669990, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3668> 2014-08-13 16:24:27.670010 7fd93dff0700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_map(194452..194452 src has 185139..194452) v3 -- ?+0 0x8d08000 con 0x4fe3700 -3667> 2014-08-13 16:24:27.670068 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 18 ==== pg_notify(22.3af(4) epoch 194451) v5 ==== 942+0+0 (2095920006 0 0) 0x8c8a300 con 0x4fe40a0 -3666> 2014-08-13 16:24:27.670089 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:25.708279, event: header_read, request: pg_notify(22.3af(4) epoch 194451) v5 -3665> 2014-08-13 16:24:27.670100 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:25.708280, event: throttled, request: pg_notify(22.3af(4) epoch 194451) v5 -3664> 2014-08-13 16:24:27.670109 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:25.708365, event: all_read, request: pg_notify(22.3af(4) epoch 194451) v5 -3663> 2014-08-13 16:24:27.670118 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:27.670087, event: dispatched, request: pg_notify(22.3af(4) epoch 194451) v5 -3662> 2014-08-13 16:24:27.670128 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:27.670128, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194451) v5 -3661> 2014-08-13 16:24:27.670139 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:27.670139, event: started, request: pg_notify(22.3af(4) epoch 194451) v5 -3660> 2014-08-13 16:24:27.670175 7fd93dff0700 5 -- op tracker -- , seq: 940, time: 2014-08-13 16:24:27.670174, event: done, request: pg_notify(22.3af(4) epoch 194451) v5 -3659> 2014-08-13 16:24:27.670197 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 21 ==== pg_notify(24.34(42) epoch 194451) v5 ==== 2882+0+0 (1719047162 0 0) 0x79b8380 con 0x4fe69e0 -3658> 2014-08-13 16:24:27.670217 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:26.127221, event: header_read, request: pg_notify(24.34(42) epoch 194451) v5 -3657> 2014-08-13 16:24:27.670228 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:26.127223, event: throttled, request: pg_notify(24.34(42) epoch 194451) v5 -3656> 2014-08-13 16:24:27.670237 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:26.127379, event: all_read, request: pg_notify(24.34(42) epoch 194451) v5 -3655> 2014-08-13 16:24:27.670245 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:27.670215, event: dispatched, request: pg_notify(24.34(42) epoch 194451) v5 -3654> 2014-08-13 16:24:27.670255 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:27.670255, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194451) v5 -3653> 2014-08-13 16:24:27.670266 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:27.670265, event: started, request: pg_notify(24.34(42) epoch 194451) v5 -3652> 2014-08-13 16:24:27.670290 7fd93dff0700 5 -- op tracker -- , seq: 941, time: 2014-08-13 16:24:27.670289, event: done, request: pg_notify(24.34(42) epoch 194451) v5 -3651> 2014-08-13 16:24:27.670330 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 20 ==== pg_notify(22.3e4(17) epoch 194451) v5 ==== 1665+0+0 (1025388927 0 0) 0x834b800 con 0x4fda940 -3650> 2014-08-13 16:24:27.670350 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:25.704822, event: header_read, request: pg_notify(22.3e4(17) epoch 194451) v5 -3649> 2014-08-13 16:24:27.670361 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:25.704823, event: throttled, request: pg_notify(22.3e4(17) epoch 194451) v5 -3648> 2014-08-13 16:24:27.670370 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:25.704929, event: all_read, request: pg_notify(22.3e4(17) epoch 194451) v5 -3647> 2014-08-13 16:24:27.670379 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:27.670348, event: dispatched, request: pg_notify(22.3e4(17) epoch 194451) v5 -3646> 2014-08-13 16:24:27.670388 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:27.670388, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194451) v5 -3645> 2014-08-13 16:24:27.670399 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:27.670399, event: started, request: pg_notify(22.3e4(17) epoch 194451) v5 -3644> 2014-08-13 16:24:27.670422 7fd93dff0700 5 -- op tracker -- , seq: 942, time: 2014-08-13 16:24:27.670422, event: done, request: pg_notify(22.3e4(17) epoch 194451) v5 -3643> 2014-08-13 16:24:27.670456 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 4 ==== pg_notify(24.39a(42) epoch 194451) v5 ==== 3042+0+0 (2308973820 0 0) 0x7759a40 con 0x4fd9b80 -3642> 2014-08-13 16:24:27.670486 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:25.690446, event: header_read, request: pg_notify(24.39a(42) epoch 194451) v5 -3641> 2014-08-13 16:24:27.670497 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:25.690449, event: throttled, request: pg_notify(24.39a(42) epoch 194451) v5 -3640> 2014-08-13 16:24:27.670506 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:25.690606, event: all_read, request: pg_notify(24.39a(42) epoch 194451) v5 -3639> 2014-08-13 16:24:27.670515 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:27.670483, event: dispatched, request: pg_notify(24.39a(42) epoch 194451) v5 -3638> 2014-08-13 16:24:27.670524 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:27.670524, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194451) v5 -3637> 2014-08-13 16:24:27.670535 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:27.670535, event: started, request: pg_notify(24.39a(42) epoch 194451) v5 -3636> 2014-08-13 16:24:27.670558 7fd93dff0700 5 -- op tracker -- , seq: 943, time: 2014-08-13 16:24:27.670558, event: done, request: pg_notify(24.39a(42) epoch 194451) v5 -3635> 2014-08-13 16:24:27.670587 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.17 192.168.1.103:6811/37658 15 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8bd0d80 con 0x4fde1a0 -3634> 2014-08-13 16:24:27.670610 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3633> 2014-08-13 16:24:27.670623 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 21 ==== pg_notify(24.181(9) epoch 194451) v5 ==== 1263+0+0 (4113897226 0 0) 0x87e32c0 con 0x50889a0 -3632> 2014-08-13 16:24:27.670640 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:25.710192, event: header_read, request: pg_notify(24.181(9) epoch 194451) v5 -3631> 2014-08-13 16:24:27.670651 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:25.710194, event: throttled, request: pg_notify(24.181(9) epoch 194451) v5 -3630> 2014-08-13 16:24:27.670660 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:25.710346, event: all_read, request: pg_notify(24.181(9) epoch 194451) v5 -3629> 2014-08-13 16:24:27.670669 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:27.670638, event: dispatched, request: pg_notify(24.181(9) epoch 194451) v5 -3628> 2014-08-13 16:24:27.670679 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:27.670679, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194451) v5 -3627> 2014-08-13 16:24:27.670690 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:27.670689, event: started, request: pg_notify(24.181(9) epoch 194451) v5 -3626> 2014-08-13 16:24:27.670720 7fd93dff0700 5 -- op tracker -- , seq: 944, time: 2014-08-13 16:24:27.670720, event: done, request: pg_notify(24.181(9) epoch 194451) v5 -3625> 2014-08-13 16:24:27.670750 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 16 ==== pg_notify(22.619(4) epoch 194451) v5 ==== 942+0+0 (1380232548 0 0) 0x80de200 con 0x508a680 -3624> 2014-08-13 16:24:27.670770 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:25.693078, event: header_read, request: pg_notify(22.619(4) epoch 194451) v5 -3623> 2014-08-13 16:24:27.670781 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:25.693080, event: throttled, request: pg_notify(22.619(4) epoch 194451) v5 -3622> 2014-08-13 16:24:27.670790 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:25.693157, event: all_read, request: pg_notify(22.619(4) epoch 194451) v5 -3621> 2014-08-13 16:24:27.670798 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:27.670768, event: dispatched, request: pg_notify(22.619(4) epoch 194451) v5 -3620> 2014-08-13 16:24:27.670812 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:27.670812, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194451) v5 -3619> 2014-08-13 16:24:27.670826 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:27.670826, event: started, request: pg_notify(22.619(4) epoch 194451) v5 -3618> 2014-08-13 16:24:27.670842 7fd9371e5700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:27.670802, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3617> 2014-08-13 16:24:27.670881 7fd93dff0700 5 -- op tracker -- , seq: 945, time: 2014-08-13 16:24:27.670853, event: done, request: pg_notify(22.619(4) epoch 194451) v5 -3616> 2014-08-13 16:24:27.670923 7fd9371e5700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:27.670883, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3615> 2014-08-13 16:24:27.670925 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 30 ==== pg_notify(22.99(15) epoch 194451) v5 ==== 1551+0+0 (310608857 0 0) 0x91055c0 con 0x508bde0 -3614> 2014-08-13 16:24:27.670947 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:25.711844, event: header_read, request: pg_notify(22.99(15) epoch 194451) v5 -3613> 2014-08-13 16:24:27.670958 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:25.711845, event: throttled, request: pg_notify(22.99(15) epoch 194451) v5 -3612> 2014-08-13 16:24:27.670968 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:25.711931, event: all_read, request: pg_notify(22.99(15) epoch 194451) v5 -3611> 2014-08-13 16:24:27.670976 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:27.670944, event: dispatched, request: pg_notify(22.99(15) epoch 194451) v5 -3610> 2014-08-13 16:24:27.670986 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:27.670986, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194451) v5 -3609> 2014-08-13 16:24:27.670997 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:27.670997, event: started, request: pg_notify(22.99(15) epoch 194451) v5 -3608> 2014-08-13 16:24:27.670971 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_sub_op_reply(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d45280 con 0x4fe3700 -3607> 2014-08-13 16:24:27.671021 7fd9371e5700 5 -- op tracker -- , seq: 939, time: 2014-08-13 16:24:27.671020, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3606> 2014-08-13 16:24:27.671058 7fd93dff0700 5 -- op tracker -- , seq: 946, time: 2014-08-13 16:24:27.671023, event: done, request: pg_notify(22.99(15) epoch 194451) v5 -3605> 2014-08-13 16:24:27.671092 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 16 ==== pg_notify(22.707(19) epoch 194451) v5 ==== 1783+0+0 (68754129 0 0) 0x87e7c40 con 0x50ba520 -3604> 2014-08-13 16:24:27.671114 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:25.709791, event: header_read, request: pg_notify(22.707(19) epoch 194451) v5 -3603> 2014-08-13 16:24:27.671125 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:25.709793, event: throttled, request: pg_notify(22.707(19) epoch 194451) v5 -3602> 2014-08-13 16:24:27.671134 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:25.709903, event: all_read, request: pg_notify(22.707(19) epoch 194451) v5 -3601> 2014-08-13 16:24:27.671142 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:27.671112, event: dispatched, request: pg_notify(22.707(19) epoch 194451) v5 -3600> 2014-08-13 16:24:27.671152 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:27.671152, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194451) v5 -3599> 2014-08-13 16:24:27.671171 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:27.671170, event: started, request: pg_notify(22.707(19) epoch 194451) v5 -3598> 2014-08-13 16:24:27.671195 7fd93dff0700 5 -- op tracker -- , seq: 947, time: 2014-08-13 16:24:27.671194, event: done, request: pg_notify(22.707(19) epoch 194451) v5 -3597> 2014-08-13 16:24:27.671217 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 9 ==== pg_notify(22.113(17) epoch 194451) v5 ==== 1669+0+0 (600254951 0 0) 0x85778c0 con 0x50bfa60 -3596> 2014-08-13 16:24:27.671237 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:25.706651, event: header_read, request: pg_notify(22.113(17) epoch 194451) v5 -3595> 2014-08-13 16:24:27.671247 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:25.706652, event: throttled, request: pg_notify(22.113(17) epoch 194451) v5 -3594> 2014-08-13 16:24:27.671256 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:25.706750, event: all_read, request: pg_notify(22.113(17) epoch 194451) v5 -3593> 2014-08-13 16:24:27.671265 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:27.671235, event: dispatched, request: pg_notify(22.113(17) epoch 194451) v5 -3592> 2014-08-13 16:24:27.671275 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:27.671275, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194451) v5 -3591> 2014-08-13 16:24:27.671285 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:27.671285, event: started, request: pg_notify(22.113(17) epoch 194451) v5 -3590> 2014-08-13 16:24:27.671308 7fd93dff0700 5 -- op tracker -- , seq: 948, time: 2014-08-13 16:24:27.671308, event: done, request: pg_notify(22.113(17) epoch 194451) v5 -3589> 2014-08-13 16:24:27.671339 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 17 ==== pg_notify(22.362(17) epoch 194451) v5 ==== 1665+0+0 (1905252073 0 0) 0x80dc280 con 0x510b9c0 -3588> 2014-08-13 16:24:27.671359 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:25.710194, event: header_read, request: pg_notify(22.362(17) epoch 194451) v5 -3587> 2014-08-13 16:24:27.671370 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:25.710196, event: throttled, request: pg_notify(22.362(17) epoch 194451) v5 -3586> 2014-08-13 16:24:27.671379 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:25.710327, event: all_read, request: pg_notify(22.362(17) epoch 194451) v5 -3585> 2014-08-13 16:24:27.671388 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:27.671357, event: dispatched, request: pg_notify(22.362(17) epoch 194451) v5 -3584> 2014-08-13 16:24:27.671398 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:27.671397, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194451) v5 -3583> 2014-08-13 16:24:27.671408 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:27.671408, event: started, request: pg_notify(22.362(17) epoch 194451) v5 -3582> 2014-08-13 16:24:27.671434 7fd93dff0700 5 -- op tracker -- , seq: 949, time: 2014-08-13 16:24:27.671434, event: done, request: pg_notify(22.362(17) epoch 194451) v5 -3581> 2014-08-13 16:24:27.671454 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 17 ==== pg_notify(22.707(4) epoch 194451) v5 ==== 942+0+0 (4064143692 0 0) 0x842f000 con 0x5108b00 -3580> 2014-08-13 16:24:27.671473 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:25.777982, event: header_read, request: pg_notify(22.707(4) epoch 194451) v5 -3579> 2014-08-13 16:24:27.671483 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:25.777983, event: throttled, request: pg_notify(22.707(4) epoch 194451) v5 -3578> 2014-08-13 16:24:27.671492 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:25.778055, event: all_read, request: pg_notify(22.707(4) epoch 194451) v5 -3577> 2014-08-13 16:24:27.671501 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:27.671471, event: dispatched, request: pg_notify(22.707(4) epoch 194451) v5 -3576> 2014-08-13 16:24:27.671511 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:27.671511, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194451) v5 -3575> 2014-08-13 16:24:27.671522 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:27.671522, event: started, request: pg_notify(22.707(4) epoch 194451) v5 -3574> 2014-08-13 16:24:27.671547 7fd93dff0700 5 -- op tracker -- , seq: 950, time: 2014-08-13 16:24:27.671547, event: done, request: pg_notify(22.707(4) epoch 194451) v5 -3573> 2014-08-13 16:24:27.671569 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 9 ==== pg_notify(22.3d9(39) epoch 194451) v5 ==== 2815+0+0 (4236145355 0 0) 0x899cd00 con 0x510dac0 -3572> 2014-08-13 16:24:27.671588 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:25.694223, event: header_read, request: pg_notify(22.3d9(39) epoch 194451) v5 -3571> 2014-08-13 16:24:27.671598 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:25.694225, event: throttled, request: pg_notify(22.3d9(39) epoch 194451) v5 -3570> 2014-08-13 16:24:27.671608 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:25.694356, event: all_read, request: pg_notify(22.3d9(39) epoch 194451) v5 -3569> 2014-08-13 16:24:27.671616 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:27.671586, event: dispatched, request: pg_notify(22.3d9(39) epoch 194451) v5 -3568> 2014-08-13 16:24:27.671626 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:27.671626, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194451) v5 -3567> 2014-08-13 16:24:27.671637 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:27.671637, event: started, request: pg_notify(22.3d9(39) epoch 194451) v5 -3566> 2014-08-13 16:24:27.671676 7fd93dff0700 5 -- op tracker -- , seq: 951, time: 2014-08-13 16:24:27.671676, event: done, request: pg_notify(22.3d9(39) epoch 194451) v5 -3565> 2014-08-13 16:24:27.671700 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 5 ==== pg_notify(24.39a(48) epoch 194451) v5 ==== 3328+0+0 (727983353 0 0) 0x7d01180 con 0x51127e0 -3564> 2014-08-13 16:24:27.671720 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:25.697536, event: header_read, request: pg_notify(24.39a(48) epoch 194451) v5 -3563> 2014-08-13 16:24:27.671730 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:25.697538, event: throttled, request: pg_notify(24.39a(48) epoch 194451) v5 -3562> 2014-08-13 16:24:27.671739 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:25.697675, event: all_read, request: pg_notify(24.39a(48) epoch 194451) v5 -3561> 2014-08-13 16:24:27.671748 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:27.671717, event: dispatched, request: pg_notify(24.39a(48) epoch 194451) v5 -3560> 2014-08-13 16:24:27.671757 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:27.671757, event: waiting_for_osdmap, request: pg_notify(24.39a(48) epoch 194451) v5 -3559> 2014-08-13 16:24:27.671768 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:27.671768, event: started, request: pg_notify(24.39a(48) epoch 194451) v5 -3558> 2014-08-13 16:24:27.671795 7fd93dff0700 5 -- op tracker -- , seq: 952, time: 2014-08-13 16:24:27.671794, event: done, request: pg_notify(24.39a(48) epoch 194451) v5 -3557> 2014-08-13 16:24:27.671816 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 33 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3180779697 0 0) 0x8bc1600 con 0x5112520 -3556> 2014-08-13 16:24:27.671840 7fd93dff0700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:26.546563, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3555> 2014-08-13 16:24:27.671854 7fd93dff0700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:26.546566, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3554> 2014-08-13 16:24:27.671866 7fd93dff0700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:26.546709, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3553> 2014-08-13 16:24:27.671879 7fd93dff0700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:27.671838, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3552> 2014-08-13 16:24:27.671891 7fd93dff0700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:27.671891, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3551> 2014-08-13 16:24:27.671922 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 14 ==== pg_notify(22.4df(15) epoch 194451) v5 ==== 1551+0+0 (1050590978 0 0) 0x81c2f40 con 0x5112260 -3550> 2014-08-13 16:24:27.671935 7fd9371e5700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:27.671935, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3549> 2014-08-13 16:24:27.671969 7fd9371e5700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:27.671969, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3548> 2014-08-13 16:24:27.671990 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:25.713674, event: header_read, request: pg_notify(22.4df(15) epoch 194451) v5 -3547> 2014-08-13 16:24:27.671992 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x7bc5f00 con 0x5112520 -3546> 2014-08-13 16:24:27.672005 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:25.713675, event: throttled, request: pg_notify(22.4df(15) epoch 194451) v5 -3545> 2014-08-13 16:24:27.672028 7fd9371e5700 5 -- op tracker -- , seq: 953, time: 2014-08-13 16:24:27.672025, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3544> 2014-08-13 16:24:27.672058 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:25.713764, event: all_read, request: pg_notify(22.4df(15) epoch 194451) v5 -3543> 2014-08-13 16:24:27.672068 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:27.671938, event: dispatched, request: pg_notify(22.4df(15) epoch 194451) v5 -3542> 2014-08-13 16:24:27.672075 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:27.672075, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194451) v5 -3541> 2014-08-13 16:24:27.672083 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:27.672083, event: started, request: pg_notify(22.4df(15) epoch 194451) v5 -3540> 2014-08-13 16:24:27.672108 7fd93dff0700 5 -- op tracker -- , seq: 954, time: 2014-08-13 16:24:27.672107, event: done, request: pg_notify(22.4df(15) epoch 194451) v5 -3539> 2014-08-13 16:24:27.672434 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.351 192.168.1.102:6830/16444 14 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d0e0c0 con 0x51148e0 -3538> 2014-08-13 16:24:27.672459 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3537> 2014-08-13 16:24:27.672477 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 12 ==== pg_notify(22.1d2(4) epoch 194451) v5 ==== 942+0+0 (2202357440 0 0) 0x8c8b100 con 0x5114620 -3536> 2014-08-13 16:24:27.672496 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:25.712301, event: header_read, request: pg_notify(22.1d2(4) epoch 194451) v5 -3535> 2014-08-13 16:24:27.672506 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:25.712302, event: throttled, request: pg_notify(22.1d2(4) epoch 194451) v5 -3534> 2014-08-13 16:24:27.672516 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:25.712363, event: all_read, request: pg_notify(22.1d2(4) epoch 194451) v5 -3533> 2014-08-13 16:24:27.672525 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:27.672493, event: dispatched, request: pg_notify(22.1d2(4) epoch 194451) v5 -3532> 2014-08-13 16:24:27.672534 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:27.672534, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194451) v5 -3531> 2014-08-13 16:24:27.672545 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:27.672545, event: started, request: pg_notify(22.1d2(4) epoch 194451) v5 -3530> 2014-08-13 16:24:27.672574 7fd93dff0700 5 -- op tracker -- , seq: 955, time: 2014-08-13 16:24:27.672574, event: done, request: pg_notify(22.1d2(4) epoch 194451) v5 -3529> 2014-08-13 16:24:27.672600 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 19 ==== pg_notify(22.39d(15) epoch 194451) v5 ==== 1551+0+0 (2999752438 0 0) 0x80daa00 con 0x5114360 -3528> 2014-08-13 16:24:27.672620 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:25.711058, event: header_read, request: pg_notify(22.39d(15) epoch 194451) v5 -3527> 2014-08-13 16:24:27.672630 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:25.711059, event: throttled, request: pg_notify(22.39d(15) epoch 194451) v5 -3526> 2014-08-13 16:24:27.672640 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:25.711168, event: all_read, request: pg_notify(22.39d(15) epoch 194451) v5 -3525> 2014-08-13 16:24:27.672649 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:27.672617, event: dispatched, request: pg_notify(22.39d(15) epoch 194451) v5 -3524> 2014-08-13 16:24:27.672659 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:27.672658, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194451) v5 -3523> 2014-08-13 16:24:27.672670 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:27.672669, event: started, request: pg_notify(22.39d(15) epoch 194451) v5 -3522> 2014-08-13 16:24:27.672695 7fd93dff0700 5 -- op tracker -- , seq: 956, time: 2014-08-13 16:24:27.672694, event: done, request: pg_notify(22.39d(15) epoch 194451) v5 -3521> 2014-08-13 16:24:27.672717 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.374 192.168.1.102:6828/16129 16 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859a1c0 con 0x51140a0 -3520> 2014-08-13 16:24:27.672737 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3519> 2014-08-13 16:24:27.672752 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 9 ==== pg_notify(24.1bb(21) epoch 194451) v5 ==== 1931+0+0 (1163313808 0 0) 0x8998c40 con 0x5113de0 -3518> 2014-08-13 16:24:27.672769 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:25.702255, event: header_read, request: pg_notify(24.1bb(21) epoch 194451) v5 -3517> 2014-08-13 16:24:27.672780 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:25.702256, event: throttled, request: pg_notify(24.1bb(21) epoch 194451) v5 -3516> 2014-08-13 16:24:27.672789 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:25.702354, event: all_read, request: pg_notify(24.1bb(21) epoch 194451) v5 -3515> 2014-08-13 16:24:27.672798 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:27.672767, event: dispatched, request: pg_notify(24.1bb(21) epoch 194451) v5 -3514> 2014-08-13 16:24:27.672808 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:27.672808, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194451) v5 -3513> 2014-08-13 16:24:27.672819 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:27.672819, event: started, request: pg_notify(24.1bb(21) epoch 194451) v5 -3512> 2014-08-13 16:24:27.673430 7fd93dff0700 5 -- op tracker -- , seq: 957, time: 2014-08-13 16:24:27.673430, event: done, request: pg_notify(24.1bb(21) epoch 194451) v5 -3511> 2014-08-13 16:24:27.673458 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 19 ==== pg_notify(22.1d2(17) epoch 194451) v5 ==== 1669+0+0 (1459892361 0 0) 0x8c8fe00 con 0x5113b20 -3510> 2014-08-13 16:24:27.673478 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:25.715491, event: header_read, request: pg_notify(22.1d2(17) epoch 194451) v5 -3509> 2014-08-13 16:24:27.673485 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:25.715494, event: throttled, request: pg_notify(22.1d2(17) epoch 194451) v5 -3508> 2014-08-13 16:24:27.673494 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:25.715668, event: all_read, request: pg_notify(22.1d2(17) epoch 194451) v5 -3507> 2014-08-13 16:24:27.673503 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:27.673476, event: dispatched, request: pg_notify(22.1d2(17) epoch 194451) v5 -3506> 2014-08-13 16:24:27.673512 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:27.673512, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194451) v5 -3505> 2014-08-13 16:24:27.673523 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:27.673523, event: started, request: pg_notify(22.1d2(17) epoch 194451) v5 -3504> 2014-08-13 16:24:27.673548 7fd93dff0700 5 -- op tracker -- , seq: 958, time: 2014-08-13 16:24:27.673548, event: done, request: pg_notify(22.1d2(17) epoch 194451) v5 -3503> 2014-08-13 16:24:27.673570 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 13 ==== pg_notify(24.b6(4) epoch 194451) v5 ==== 976+0+0 (1628275888 0 0) 0x81c3f00 con 0x5113860 -3502> 2014-08-13 16:24:27.673589 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:25.694778, event: header_read, request: pg_notify(24.b6(4) epoch 194451) v5 -3501> 2014-08-13 16:24:27.673600 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:25.694779, event: throttled, request: pg_notify(24.b6(4) epoch 194451) v5 -3500> 2014-08-13 16:24:27.673609 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:25.694851, event: all_read, request: pg_notify(24.b6(4) epoch 194451) v5 -3499> 2014-08-13 16:24:27.673618 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:27.673587, event: dispatched, request: pg_notify(24.b6(4) epoch 194451) v5 -3498> 2014-08-13 16:24:27.673628 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:27.673628, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194451) v5 -3497> 2014-08-13 16:24:27.673639 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:27.673639, event: started, request: pg_notify(24.b6(4) epoch 194451) v5 -3496> 2014-08-13 16:24:27.673664 7fd93dff0700 5 -- op tracker -- , seq: 959, time: 2014-08-13 16:24:27.673664, event: done, request: pg_notify(24.b6(4) epoch 194451) v5 -3495> 2014-08-13 16:24:27.673695 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 26 ==== pg_notify(22.3d9(6) epoch 194451) v5 ==== 1060+0+0 (112042954 0 0) 0x91040c0 con 0x5111fa0 -3494> 2014-08-13 16:24:27.673714 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:25.709706, event: header_read, request: pg_notify(22.3d9(6) epoch 194451) v5 -3493> 2014-08-13 16:24:27.673725 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:25.709707, event: throttled, request: pg_notify(22.3d9(6) epoch 194451) v5 -3492> 2014-08-13 16:24:27.673734 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:25.709780, event: all_read, request: pg_notify(22.3d9(6) epoch 194451) v5 -3491> 2014-08-13 16:24:27.673743 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:27.673712, event: dispatched, request: pg_notify(22.3d9(6) epoch 194451) v5 -3490> 2014-08-13 16:24:27.673752 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:27.673752, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194451) v5 -3489> 2014-08-13 16:24:27.673763 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:27.673763, event: started, request: pg_notify(22.3d9(6) epoch 194451) v5 -3488> 2014-08-13 16:24:27.673788 7fd93dff0700 5 -- op tracker -- , seq: 960, time: 2014-08-13 16:24:27.673788, event: done, request: pg_notify(22.3d9(6) epoch 194451) v5 -3487> 2014-08-13 16:24:27.673809 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 31 ==== pg_notify(24.3d(18) epoch 194451) v5 ==== 1754+0+0 (2393444723 0 0) 0x8c8cd00 con 0x51135a0 -3486> 2014-08-13 16:24:27.673828 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:25.708190, event: header_read, request: pg_notify(24.3d(18) epoch 194451) v5 -3485> 2014-08-13 16:24:27.673839 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:25.708191, event: throttled, request: pg_notify(24.3d(18) epoch 194451) v5 -3484> 2014-08-13 16:24:27.673848 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:25.708285, event: all_read, request: pg_notify(24.3d(18) epoch 194451) v5 -3483> 2014-08-13 16:24:27.673857 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:27.673826, event: dispatched, request: pg_notify(24.3d(18) epoch 194451) v5 -3482> 2014-08-13 16:24:27.673867 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:27.673867, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194451) v5 -3481> 2014-08-13 16:24:27.673877 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:27.673877, event: started, request: pg_notify(24.3d(18) epoch 194451) v5 -3480> 2014-08-13 16:24:27.673903 7fd93dff0700 5 -- op tracker -- , seq: 961, time: 2014-08-13 16:24:27.673903, event: done, request: pg_notify(24.3d(18) epoch 194451) v5 -3479> 2014-08-13 16:24:27.673923 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 23 ==== pg_notify(22.48e(15) epoch 194451) v5 ==== 1551+0+0 (1783126991 0 0) 0x87e3b80 con 0x5111a20 -3478> 2014-08-13 16:24:27.673942 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:25.695424, event: header_read, request: pg_notify(22.48e(15) epoch 194451) v5 -3477> 2014-08-13 16:24:27.673953 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:25.695425, event: throttled, request: pg_notify(22.48e(15) epoch 194451) v5 -3476> 2014-08-13 16:24:27.673962 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:25.695521, event: all_read, request: pg_notify(22.48e(15) epoch 194451) v5 -3475> 2014-08-13 16:24:27.673971 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:27.673940, event: dispatched, request: pg_notify(22.48e(15) epoch 194451) v5 -3474> 2014-08-13 16:24:27.673980 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:27.673980, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194451) v5 -3473> 2014-08-13 16:24:27.673991 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:27.673991, event: started, request: pg_notify(22.48e(15) epoch 194451) v5 -3472> 2014-08-13 16:24:27.674016 7fd93dff0700 5 -- op tracker -- , seq: 962, time: 2014-08-13 16:24:27.674016, event: done, request: pg_notify(22.48e(15) epoch 194451) v5 -3471> 2014-08-13 16:24:27.674039 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 17 ==== pg_notify(22.362(17) epoch 194451) v5 ==== 1665+0+0 (1905252073 0 0) 0x775d5c0 con 0x7313860 -3470> 2014-08-13 16:24:27.674058 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:25.693348, event: header_read, request: pg_notify(22.362(17) epoch 194451) v5 -3469> 2014-08-13 16:24:27.674068 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:25.693349, event: throttled, request: pg_notify(22.362(17) epoch 194451) v5 -3468> 2014-08-13 16:24:27.674077 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:25.693432, event: all_read, request: pg_notify(22.362(17) epoch 194451) v5 -3467> 2014-08-13 16:24:27.674086 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:27.674056, event: dispatched, request: pg_notify(22.362(17) epoch 194451) v5 -3466> 2014-08-13 16:24:27.674096 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:27.674096, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194451) v5 -3465> 2014-08-13 16:24:27.674107 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:27.674107, event: started, request: pg_notify(22.362(17) epoch 194451) v5 -3464> 2014-08-13 16:24:27.674132 7fd93dff0700 5 -- op tracker -- , seq: 963, time: 2014-08-13 16:24:27.674132, event: done, request: pg_notify(22.362(17) epoch 194451) v5 -3463> 2014-08-13 16:24:27.674152 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 11 ==== pg_notify(22.22c(17) epoch 194451) v5 ==== 1665+0+0 (3343302275 0 0) 0x842c0c0 con 0x7316720 -3462> 2014-08-13 16:24:27.674184 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:25.800453, event: header_read, request: pg_notify(22.22c(17) epoch 194451) v5 -3461> 2014-08-13 16:24:27.674194 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:25.800454, event: throttled, request: pg_notify(22.22c(17) epoch 194451) v5 -3460> 2014-08-13 16:24:27.674204 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:25.800547, event: all_read, request: pg_notify(22.22c(17) epoch 194451) v5 -3459> 2014-08-13 16:24:27.674213 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:27.674182, event: dispatched, request: pg_notify(22.22c(17) epoch 194451) v5 -3458> 2014-08-13 16:24:27.674222 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:27.674222, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194451) v5 -3457> 2014-08-13 16:24:27.674233 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:27.674233, event: started, request: pg_notify(22.22c(17) epoch 194451) v5 -3456> 2014-08-13 16:24:27.674258 7fd93dff0700 5 -- op tracker -- , seq: 964, time: 2014-08-13 16:24:27.674258, event: done, request: pg_notify(22.22c(17) epoch 194451) v5 -3455> 2014-08-13 16:24:27.674281 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 14 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf5e80 con 0x7315ee0 -3454> 2014-08-13 16:24:27.674302 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3453> 2014-08-13 16:24:27.674318 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 12 ==== pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 ==== 3436+0+0 (3377124003 0 0) 0x80d9a40 con 0x7314fc0 -3452> 2014-08-13 16:24:27.674337 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:25.716279, event: header_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3451> 2014-08-13 16:24:27.674348 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:25.716281, event: throttled, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3450> 2014-08-13 16:24:27.674358 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:25.716424, event: all_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3449> 2014-08-13 16:24:27.674368 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:27.674335, event: dispatched, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3448> 2014-08-13 16:24:27.674378 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:27.674378, event: waiting_for_osdmap, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3447> 2014-08-13 16:24:27.674390 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:27.674390, event: started, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3446> 2014-08-13 16:24:27.674431 7fd93dff0700 5 -- op tracker -- , seq: 965, time: 2014-08-13 16:24:27.674431, event: done, request: pg_notify(22.3e4(17),22.707(19) epoch 194451) v5 -3445> 2014-08-13 16:24:27.674474 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x85998c0 con 0x7316f60 -3444> 2014-08-13 16:24:27.674495 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3443> 2014-08-13 16:24:27.674509 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e79680 con 0x8da1760 -3442> 2014-08-13 16:24:27.674526 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3441> 2014-08-13 16:24:27.674540 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x5077740 con 0x8da5120 -3440> 2014-08-13 16:24:27.674558 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3439> 2014-08-13 16:24:27.674569 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 12 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e78240 con 0x72d9b80 -3438> 2014-08-13 16:24:27.674586 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3437> 2014-08-13 16:24:27.674598 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf1b00 con 0x72dfa60 -3436> 2014-08-13 16:24:27.674615 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3435> 2014-08-13 16:24:27.674627 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x506ec00 con 0x72de9e0 -3434> 2014-08-13 16:24:27.674644 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3433> 2014-08-13 16:24:27.674656 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7ba80 con 0x8e29fa0 -3432> 2014-08-13 16:24:27.674673 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3431> 2014-08-13 16:24:27.674685 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 15 ==== pg_notify(22.775(4) epoch 194451) v5 ==== 942+0+0 (339068629 0 0) 0x834b2c0 con 0x8e286e0 -3430> 2014-08-13 16:24:27.674702 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:25.705880, event: header_read, request: pg_notify(22.775(4) epoch 194451) v5 -3429> 2014-08-13 16:24:27.674712 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:25.705883, event: throttled, request: pg_notify(22.775(4) epoch 194451) v5 -3428> 2014-08-13 16:24:27.674722 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:25.706006, event: all_read, request: pg_notify(22.775(4) epoch 194451) v5 -3427> 2014-08-13 16:24:27.674730 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:27.674699, event: dispatched, request: pg_notify(22.775(4) epoch 194451) v5 -3426> 2014-08-13 16:24:27.674740 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:27.674740, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194451) v5 -3425> 2014-08-13 16:24:27.674751 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:27.674750, event: started, request: pg_notify(22.775(4) epoch 194451) v5 -3424> 2014-08-13 16:24:27.674787 7fd93dff0700 5 -- op tracker -- , seq: 966, time: 2014-08-13 16:24:27.674786, event: done, request: pg_notify(22.775(4) epoch 194451) v5 -3423> 2014-08-13 16:24:27.674808 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x506cec0 con 0x8e2c360 -3422> 2014-08-13 16:24:27.674835 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3421> 2014-08-13 16:24:27.674848 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d08900 con 0x7379ce0 -3420> 2014-08-13 16:24:27.674866 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3419> 2014-08-13 16:24:27.674877 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf3180 con 0x73ca680 -3418> 2014-08-13 16:24:27.674895 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3417> 2014-08-13 16:24:27.674906 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 16 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8bd5a00 con 0x73cdc20 -3416> 2014-08-13 16:24:27.674923 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3415> 2014-08-13 16:24:27.674935 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 7 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7ad00 con 0x73ccba0 -3414> 2014-08-13 16:24:27.674952 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3413> 2014-08-13 16:24:27.674963 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 7 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d08d80 con 0x7462100 -3412> 2014-08-13 16:24:27.674980 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3411> 2014-08-13 16:24:27.674992 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 13 ==== pg_notify(22.481(27) epoch 194451) v5 ==== 2235+0+0 (1904242912 0 0) 0x899c440 con 0x7723440 -3410> 2014-08-13 16:24:27.675009 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:25.708119, event: header_read, request: pg_notify(22.481(27) epoch 194451) v5 -3409> 2014-08-13 16:24:27.675019 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:25.708122, event: throttled, request: pg_notify(22.481(27) epoch 194451) v5 -3408> 2014-08-13 16:24:27.675029 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:25.708259, event: all_read, request: pg_notify(22.481(27) epoch 194451) v5 -3407> 2014-08-13 16:24:27.675037 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:27.675006, event: dispatched, request: pg_notify(22.481(27) epoch 194451) v5 -3406> 2014-08-13 16:24:27.675047 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:27.675047, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194451) v5 -3405> 2014-08-13 16:24:27.675068 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:27.675068, event: started, request: pg_notify(22.481(27) epoch 194451) v5 -3404> 2014-08-13 16:24:27.675095 7fd93dff0700 5 -- op tracker -- , seq: 967, time: 2014-08-13 16:24:27.675095, event: done, request: pg_notify(22.481(27) epoch 194451) v5 -3403> 2014-08-13 16:24:27.675130 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 6 ==== pg_notify(24.34(29) epoch 194452) v5 ==== 2199+0+0 (686324416 0 0) 0x8c88fc0 con 0x7d2d120 -3402> 2014-08-13 16:24:27.675152 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:26.698315, event: header_read, request: pg_notify(24.34(29) epoch 194452) v5 -3401> 2014-08-13 16:24:27.675172 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:26.698316, event: throttled, request: pg_notify(24.34(29) epoch 194452) v5 -3400> 2014-08-13 16:24:27.675179 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:26.698444, event: all_read, request: pg_notify(24.34(29) epoch 194452) v5 -3399> 2014-08-13 16:24:27.675188 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:27.675149, event: dispatched, request: pg_notify(24.34(29) epoch 194452) v5 -3398> 2014-08-13 16:24:27.675197 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:27.675197, event: waiting_for_osdmap, request: pg_notify(24.34(29) epoch 194452) v5 -3397> 2014-08-13 16:24:27.675208 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:27.675208, event: started, request: pg_notify(24.34(29) epoch 194452) v5 -3396> 2014-08-13 16:24:27.675236 7fd93dff0700 5 -- op tracker -- , seq: 968, time: 2014-08-13 16:24:27.675236, event: done, request: pg_notify(24.34(29) epoch 194452) v5 -3395> 2014-08-13 16:24:27.675268 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859f2c0 con 0x7d2d280 -3394> 2014-08-13 16:24:27.675290 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3393> 2014-08-13 16:24:27.675304 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859e9c0 con 0x821f900 -3392> 2014-08-13 16:24:27.675322 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3391> 2014-08-13 16:24:27.675331 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 7 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf2f40 con 0x80c7e80 -3390> 2014-08-13 16:24:27.675348 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3389> 2014-08-13 16:24:27.675373 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 9 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e78d80 con 0x80c7d20 -3388> 2014-08-13 16:24:27.675391 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3387> 2014-08-13 16:24:27.675402 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 17 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf0240 con 0x4e4b860 -3386> 2014-08-13 16:24:27.675420 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3385> 2014-08-13 16:24:27.675431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 21 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7d7c0 con 0x4e4dd80 -3384> 2014-08-13 16:24:27.675448 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3383> 2014-08-13 16:24:27.675471 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 15 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8bd06c0 con 0x4e4ce60 -3382> 2014-08-13 16:24:27.675488 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3381> 2014-08-13 16:24:27.675500 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 25 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d0b180 con 0x4e4f900 -3380> 2014-08-13 16:24:27.675517 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3379> 2014-08-13 16:24:27.675528 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 26 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x33e3cc0 con 0x4fd8000 -3378> 2014-08-13 16:24:27.675544 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3377> 2014-08-13 16:24:27.675556 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 19 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8599b00 con 0x4fe1600 -3376> 2014-08-13 16:24:27.675573 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3375> 2014-08-13 16:24:27.675585 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 22 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2617991459 0 0) 0x7974d00 con 0x4fe3700 -3374> 2014-08-13 16:24:27.675607 7fd93dff0700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.672441, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3373> 2014-08-13 16:24:27.675622 7fd93dff0700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.672443, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3372> 2014-08-13 16:24:27.675635 7fd93dff0700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.672566, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3371> 2014-08-13 16:24:27.675648 7fd93dff0700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.675604, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3370> 2014-08-13 16:24:27.675661 7fd93dff0700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.675661, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3369> 2014-08-13 16:24:27.675692 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 19 ==== pg_notify(22.362(4) epoch 194451) v5 ==== 942+0+0 (2299395842 0 0) 0x8c8a140 con 0x4fe40a0 -3368> 2014-08-13 16:24:27.675711 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:25.720669, event: header_read, request: pg_notify(22.362(4) epoch 194451) v5 -3367> 2014-08-13 16:24:27.675721 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:25.720672, event: throttled, request: pg_notify(22.362(4) epoch 194451) v5 -3366> 2014-08-13 16:24:27.675730 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:25.720766, event: all_read, request: pg_notify(22.362(4) epoch 194451) v5 -3365> 2014-08-13 16:24:27.675739 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:27.675709, event: dispatched, request: pg_notify(22.362(4) epoch 194451) v5 -3364> 2014-08-13 16:24:27.675749 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:27.675749, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194451) v5 -3363> 2014-08-13 16:24:27.675765 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:27.675765, event: started, request: pg_notify(22.362(4) epoch 194451) v5 -3362> 2014-08-13 16:24:27.675785 7fd9367e4700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.675756, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3361> 2014-08-13 16:24:27.675834 7fd9367e4700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.675833, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3360> 2014-08-13 16:24:27.675853 7fd9367e4700 5 -- op tracker -- , seq: 969, time: 2014-08-13 16:24:27.675853, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3359> 2014-08-13 16:24:27.675894 7fd93dff0700 5 -- op tracker -- , seq: 970, time: 2014-08-13 16:24:27.675797, event: done, request: pg_notify(22.362(4) epoch 194451) v5 -3358> 2014-08-13 16:24:27.675951 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 22 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7b17080 con 0x4fe69e0 -3357> 2014-08-13 16:24:27.675983 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3356> 2014-08-13 16:24:27.675998 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 21 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859e300 con 0x4fda940 -3355> 2014-08-13 16:24:27.676017 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3354> 2014-08-13 16:24:27.676028 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 5 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf0900 con 0x4fd9b80 -3353> 2014-08-13 16:24:27.676045 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3352> 2014-08-13 16:24:27.676057 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 22 ==== pg_notify(22.113(17) epoch 194451) v5 ==== 1669+0+0 (1770743644 0 0) 0x87e5400 con 0x50889a0 -3351> 2014-08-13 16:24:27.676075 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:25.710734, event: header_read, request: pg_notify(22.113(17) epoch 194451) v5 -3350> 2014-08-13 16:24:27.676087 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:25.710737, event: throttled, request: pg_notify(22.113(17) epoch 194451) v5 -3349> 2014-08-13 16:24:27.676096 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:25.710864, event: all_read, request: pg_notify(22.113(17) epoch 194451) v5 -3348> 2014-08-13 16:24:27.676105 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:27.676072, event: dispatched, request: pg_notify(22.113(17) epoch 194451) v5 -3347> 2014-08-13 16:24:27.676115 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:27.676115, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194451) v5 -3346> 2014-08-13 16:24:27.676128 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:27.676127, event: started, request: pg_notify(22.113(17) epoch 194451) v5 -3345> 2014-08-13 16:24:27.676172 7fd93dff0700 5 -- op tracker -- , seq: 971, time: 2014-08-13 16:24:27.676172, event: done, request: pg_notify(22.113(17) epoch 194451) v5 -3344> 2014-08-13 16:24:27.676199 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 17 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8bd3600 con 0x508a680 -3343> 2014-08-13 16:24:27.676221 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3342> 2014-08-13 16:24:27.676234 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 31 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7f500 con 0x508bde0 -3341> 2014-08-13 16:24:27.676252 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3340> 2014-08-13 16:24:27.676263 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 17 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8bd1d40 con 0x50ba520 -3339> 2014-08-13 16:24:27.676280 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3338> 2014-08-13 16:24:27.676292 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x506f2c0 con 0x50bfa60 -3337> 2014-08-13 16:24:27.676308 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3336> 2014-08-13 16:24:27.676320 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 18 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7fbc0 con 0x510b9c0 -3335> 2014-08-13 16:24:27.676337 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3334> 2014-08-13 16:24:27.676349 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 18 ==== pg_notify(22.3e4(4) epoch 194451) v5 ==== 942+0+0 (4024152846 0 0) 0x8428c40 con 0x5108b00 -3333> 2014-08-13 16:24:27.676366 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:25.778611, event: header_read, request: pg_notify(22.3e4(4) epoch 194451) v5 -3332> 2014-08-13 16:24:27.676377 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:25.778613, event: throttled, request: pg_notify(22.3e4(4) epoch 194451) v5 -3331> 2014-08-13 16:24:27.676386 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:25.778702, event: all_read, request: pg_notify(22.3e4(4) epoch 194451) v5 -3330> 2014-08-13 16:24:27.676395 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:27.676364, event: dispatched, request: pg_notify(22.3e4(4) epoch 194451) v5 -3329> 2014-08-13 16:24:27.676405 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:27.676405, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194451) v5 -3328> 2014-08-13 16:24:27.676417 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:27.676416, event: started, request: pg_notify(22.3e4(4) epoch 194451) v5 -3327> 2014-08-13 16:24:27.676444 7fd93dff0700 5 -- op tracker -- , seq: 972, time: 2014-08-13 16:24:27.676443, event: done, request: pg_notify(22.3e4(4) epoch 194451) v5 -3326> 2014-08-13 16:24:27.676470 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859b600 con 0x510dac0 -3325> 2014-08-13 16:24:27.676491 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3324> 2014-08-13 16:24:27.676510 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 6 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859fbc0 con 0x51127e0 -3323> 2014-08-13 16:24:27.676528 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3322> 2014-08-13 16:24:27.676541 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 34 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d0a880 con 0x5112520 -3321> 2014-08-13 16:24:27.676558 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3320> 2014-08-13 16:24:27.676569 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 15 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859c380 con 0x5112260 -3319> 2014-08-13 16:24:27.676586 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3318> 2014-08-13 16:24:27.676598 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 13 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x8d09440 con 0x5114620 -3317> 2014-08-13 16:24:27.676615 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3316> 2014-08-13 16:24:27.676627 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 20 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7b13180 con 0x5114360 -3315> 2014-08-13 16:24:27.676644 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3314> 2014-08-13 16:24:27.676655 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 10 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859c800 con 0x5113de0 -3313> 2014-08-13 16:24:27.676672 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3312> 2014-08-13 16:24:27.676684 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 20 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859a880 con 0x5113b20 -3311> 2014-08-13 16:24:27.676701 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3310> 2014-08-13 16:24:27.676713 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 14 ==== pg_notify(22.48e(15) epoch 194451) v5 ==== 1551+0+0 (776011928 0 0) 0x81c1a40 con 0x5113860 -3309> 2014-08-13 16:24:27.676730 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:25.696227, event: header_read, request: pg_notify(22.48e(15) epoch 194451) v5 -3308> 2014-08-13 16:24:27.676741 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:25.696229, event: throttled, request: pg_notify(22.48e(15) epoch 194451) v5 -3307> 2014-08-13 16:24:27.676750 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:25.696342, event: all_read, request: pg_notify(22.48e(15) epoch 194451) v5 -3306> 2014-08-13 16:24:27.676759 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:27.676728, event: dispatched, request: pg_notify(22.48e(15) epoch 194451) v5 -3305> 2014-08-13 16:24:27.676766 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:27.676765, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194451) v5 -3304> 2014-08-13 16:24:27.676777 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:27.676777, event: started, request: pg_notify(22.48e(15) epoch 194451) v5 -3303> 2014-08-13 16:24:27.676807 7fd93dff0700 5 -- op tracker -- , seq: 973, time: 2014-08-13 16:24:27.676807, event: done, request: pg_notify(22.48e(15) epoch 194451) v5 -3302> 2014-08-13 16:24:27.676837 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 27 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859bf00 con 0x5111fa0 -3301> 2014-08-13 16:24:27.676858 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3300> 2014-08-13 16:24:27.676870 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 32 ==== pg_notify(22.3d9(32) epoch 194451) v5 ==== 2458+0+0 (3590111613 0 0) 0x8c8db00 con 0x51135a0 -3299> 2014-08-13 16:24:27.676888 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:25.709896, event: header_read, request: pg_notify(22.3d9(32) epoch 194451) v5 -3298> 2014-08-13 16:24:27.676899 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:25.709898, event: throttled, request: pg_notify(22.3d9(32) epoch 194451) v5 -3297> 2014-08-13 16:24:27.676907 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:25.710013, event: all_read, request: pg_notify(22.3d9(32) epoch 194451) v5 -3296> 2014-08-13 16:24:27.676916 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:27.676886, event: dispatched, request: pg_notify(22.3d9(32) epoch 194451) v5 -3295> 2014-08-13 16:24:27.676926 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:27.676926, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194451) v5 -3294> 2014-08-13 16:24:27.676937 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:27.676937, event: started, request: pg_notify(22.3d9(32) epoch 194451) v5 -3293> 2014-08-13 16:24:27.676963 7fd93dff0700 5 -- op tracker -- , seq: 974, time: 2014-08-13 16:24:27.676963, event: done, request: pg_notify(22.3d9(32) epoch 194451) v5 -3292> 2014-08-13 16:24:27.676986 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 24 ==== pg_notify(24.1f7(18) epoch 194451) v5 ==== 1754+0+0 (165338831 0 0) 0x87e40c0 con 0x5111a20 -3291> 2014-08-13 16:24:27.677005 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:25.695544, event: header_read, request: pg_notify(24.1f7(18) epoch 194451) v5 -3290> 2014-08-13 16:24:27.677016 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:25.695545, event: throttled, request: pg_notify(24.1f7(18) epoch 194451) v5 -3289> 2014-08-13 16:24:27.677025 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:25.695627, event: all_read, request: pg_notify(24.1f7(18) epoch 194451) v5 -3288> 2014-08-13 16:24:27.677034 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:27.677003, event: dispatched, request: pg_notify(24.1f7(18) epoch 194451) v5 -3287> 2014-08-13 16:24:27.677043 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:27.677043, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194451) v5 -3286> 2014-08-13 16:24:27.677054 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:27.677054, event: started, request: pg_notify(24.1f7(18) epoch 194451) v5 -3285> 2014-08-13 16:24:27.677080 7fd93dff0700 5 -- op tracker -- , seq: 975, time: 2014-08-13 16:24:27.677080, event: done, request: pg_notify(24.1f7(18) epoch 194451) v5 -3284> 2014-08-13 16:24:27.677103 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 18 ==== pg_notify(24.39a(42) epoch 194451) v5 ==== 3042+0+0 (3743541503 0 0) 0x775e200 con 0x7313860 -3283> 2014-08-13 16:24:27.677122 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:25.693485, event: header_read, request: pg_notify(24.39a(42) epoch 194451) v5 -3282> 2014-08-13 16:24:27.677143 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:25.693486, event: throttled, request: pg_notify(24.39a(42) epoch 194451) v5 -3281> 2014-08-13 16:24:27.677152 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:25.693611, event: all_read, request: pg_notify(24.39a(42) epoch 194451) v5 -3280> 2014-08-13 16:24:27.677169 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:27.677120, event: dispatched, request: pg_notify(24.39a(42) epoch 194451) v5 -3279> 2014-08-13 16:24:27.677179 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:27.677179, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194451) v5 -3278> 2014-08-13 16:24:27.677190 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:27.677190, event: started, request: pg_notify(24.39a(42) epoch 194451) v5 -3277> 2014-08-13 16:24:27.677218 7fd93dff0700 5 -- op tracker -- , seq: 976, time: 2014-08-13 16:24:27.677218, event: done, request: pg_notify(24.39a(42) epoch 194451) v5 -3276> 2014-08-13 16:24:27.677246 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 12 ==== pg_notify(22.1f1(17) epoch 194451) v5 ==== 1665+0+0 (3225697761 0 0) 0x842d080 con 0x7316720 -3275> 2014-08-13 16:24:27.677267 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:25.800821, event: header_read, request: pg_notify(22.1f1(17) epoch 194451) v5 -3274> 2014-08-13 16:24:27.677277 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:25.800825, event: throttled, request: pg_notify(22.1f1(17) epoch 194451) v5 -3273> 2014-08-13 16:24:27.677287 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:25.800954, event: all_read, request: pg_notify(22.1f1(17) epoch 194451) v5 -3272> 2014-08-13 16:24:27.677293 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:27.677264, event: dispatched, request: pg_notify(22.1f1(17) epoch 194451) v5 -3271> 2014-08-13 16:24:27.677302 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:27.677302, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194451) v5 -3270> 2014-08-13 16:24:27.677271 7fd9367e4700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/GetInfo 0.021683 8 0.000376 -3269> 2014-08-13 16:24:27.677313 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:27.677313, event: started, request: pg_notify(22.1f1(17) epoch 194451) v5 -3268> 2014-08-13 16:24:27.677318 7fd9367e4700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/GetLog -3267> 2014-08-13 16:24:27.677339 7fd93dff0700 5 -- op tracker -- , seq: 977, time: 2014-08-13 16:24:27.677339, event: done, request: pg_notify(22.1f1(17) epoch 194451) v5 -3266> 2014-08-13 16:24:27.677362 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 15 ==== pg_notify(24.39a(28) epoch 194452) v5 ==== 2300+0+0 (1476893116 0 0) 0x775de80 con 0x7315ee0 -3265> 2014-08-13 16:24:27.677382 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:26.703993, event: header_read, request: pg_notify(24.39a(28) epoch 194452) v5 -3264> 2014-08-13 16:24:27.677392 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:26.703994, event: throttled, request: pg_notify(24.39a(28) epoch 194452) v5 -3263> 2014-08-13 16:24:27.677401 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:26.704100, event: all_read, request: pg_notify(24.39a(28) epoch 194452) v5 -3262> 2014-08-13 16:24:27.677407 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:27.677380, event: dispatched, request: pg_notify(24.39a(28) epoch 194452) v5 -3261> 2014-08-13 16:24:27.677417 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:27.677417, event: waiting_for_osdmap, request: pg_notify(24.39a(28) epoch 194452) v5 -3260> 2014-08-13 16:24:27.677428 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:27.677428, event: started, request: pg_notify(24.39a(28) epoch 194452) v5 -3259> 2014-08-13 16:24:27.677425 7fd9367e4700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/GetLog 0.000108 0 0.000000 -3258> 2014-08-13 16:24:27.677456 7fd9367e4700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/GetMissing -3257> 2014-08-13 16:24:27.677528 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6810/41287 -- pg_query(24.39a epoch 194452) v3 -- ?+0 0x8019860 con 0x51127e0 -3256> 2014-08-13 16:24:27.677661 7fd93dff0700 5 -- op tracker -- , seq: 978, time: 2014-08-13 16:24:27.677661, event: done, request: pg_notify(24.39a(28) epoch 194452) v5 -3255> 2014-08-13 16:24:27.677705 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 13 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7a640 con 0x7314fc0 -3254> 2014-08-13 16:24:27.677732 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3253> 2014-08-13 16:24:27.677743 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 11 ==== pg_notify(22.22c(4) epoch 194452) v5 ==== 942+0+0 (2702077905 0 0) 0x899c0c0 con 0x7316f60 -3252> 2014-08-13 16:24:27.677761 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:26.699767, event: header_read, request: pg_notify(22.22c(4) epoch 194452) v5 -3251> 2014-08-13 16:24:27.677772 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:26.699768, event: throttled, request: pg_notify(22.22c(4) epoch 194452) v5 -3250> 2014-08-13 16:24:27.677781 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:26.699871, event: all_read, request: pg_notify(22.22c(4) epoch 194452) v5 -3249> 2014-08-13 16:24:27.677790 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:27.677759, event: dispatched, request: pg_notify(22.22c(4) epoch 194452) v5 -3248> 2014-08-13 16:24:27.677800 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:27.677799, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194452) v5 -3247> 2014-08-13 16:24:27.677811 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:27.677811, event: started, request: pg_notify(22.22c(4) epoch 194452) v5 -3246> 2014-08-13 16:24:27.677839 7fd93dff0700 5 -- op tracker -- , seq: 979, time: 2014-08-13 16:24:27.677839, event: done, request: pg_notify(22.22c(4) epoch 194452) v5 -3245> 2014-08-13 16:24:27.677860 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 10 ==== pg_notify(22.619(17) epoch 194452) v5 ==== 1665+0+0 (2906983997 0 0) 0x834ec80 con 0x8da1760 -3244> 2014-08-13 16:24:27.677882 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:26.722438, event: header_read, request: pg_notify(22.619(17) epoch 194452) v5 -3243> 2014-08-13 16:24:27.677889 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:26.722440, event: throttled, request: pg_notify(22.619(17) epoch 194452) v5 -3242> 2014-08-13 16:24:27.677898 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:26.722544, event: all_read, request: pg_notify(22.619(17) epoch 194452) v5 -3241> 2014-08-13 16:24:27.677907 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:27.677880, event: dispatched, request: pg_notify(22.619(17) epoch 194452) v5 -3240> 2014-08-13 16:24:27.677913 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:27.677913, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194452) v5 -3239> 2014-08-13 16:24:27.677920 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:27.677920, event: started, request: pg_notify(22.619(17) epoch 194452) v5 -3238> 2014-08-13 16:24:27.677943 7fd93dff0700 5 -- op tracker -- , seq: 980, time: 2014-08-13 16:24:27.677943, event: done, request: pg_notify(22.619(17) epoch 194452) v5 -3237> 2014-08-13 16:24:27.677965 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 11 ==== pg_notify(22.f3(15) epoch 194452) v5 ==== 1551+0+0 (507610884 0 0) 0x8409880 con 0x8da5120 -3236> 2014-08-13 16:24:27.677984 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:26.761171, event: header_read, request: pg_notify(22.f3(15) epoch 194452) v5 -3235> 2014-08-13 16:24:27.677994 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:26.761172, event: throttled, request: pg_notify(22.f3(15) epoch 194452) v5 -3234> 2014-08-13 16:24:27.678003 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:26.761259, event: all_read, request: pg_notify(22.f3(15) epoch 194452) v5 -3233> 2014-08-13 16:24:27.678012 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:27.677982, event: dispatched, request: pg_notify(22.f3(15) epoch 194452) v5 -3232> 2014-08-13 16:24:27.678022 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:27.678021, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194452) v5 -3231> 2014-08-13 16:24:27.678032 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:27.678032, event: started, request: pg_notify(22.f3(15) epoch 194452) v5 -3230> 2014-08-13 16:24:27.678058 7fd93dff0700 5 -- op tracker -- , seq: 981, time: 2014-08-13 16:24:27.678058, event: done, request: pg_notify(22.f3(15) epoch 194452) v5 -3229> 2014-08-13 16:24:27.678087 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 13 ==== pg_notify(22.99(4),22.39d(15) epoch 194452) v5 ==== 2481+0+0 (2483536329 0 0) 0x80d96c0 con 0x72d9b80 -3228> 2014-08-13 16:24:27.678107 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:26.710498, event: header_read, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3227> 2014-08-13 16:24:27.678118 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:26.710500, event: throttled, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3226> 2014-08-13 16:24:27.678128 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:26.710619, event: all_read, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3225> 2014-08-13 16:24:27.678137 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:27.678105, event: dispatched, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3224> 2014-08-13 16:24:27.678148 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:27.678147, event: waiting_for_osdmap, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3223> 2014-08-13 16:24:27.678167 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:27.678167, event: started, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3222> 2014-08-13 16:24:27.678208 7fd93dff0700 5 -- op tracker -- , seq: 982, time: 2014-08-13 16:24:27.678207, event: done, request: pg_notify(22.99(4),22.39d(15) epoch 194452) v5 -3221> 2014-08-13 16:24:27.678237 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 11 ==== pg_notify(22.4df(4) epoch 194452) v5 ==== 942+0+0 (3188453592 0 0) 0x87e7a80 con 0x72dfa60 -3220> 2014-08-13 16:24:27.678257 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:26.705793, event: header_read, request: pg_notify(22.4df(4) epoch 194452) v5 -3219> 2014-08-13 16:24:27.678268 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:26.705794, event: throttled, request: pg_notify(22.4df(4) epoch 194452) v5 -3218> 2014-08-13 16:24:27.678277 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:26.705867, event: all_read, request: pg_notify(22.4df(4) epoch 194452) v5 -3217> 2014-08-13 16:24:27.678286 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:27.678255, event: dispatched, request: pg_notify(22.4df(4) epoch 194452) v5 -3216> 2014-08-13 16:24:27.678295 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:27.678295, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194452) v5 -3215> 2014-08-13 16:24:27.678306 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:27.678305, event: started, request: pg_notify(22.4df(4) epoch 194452) v5 -3214> 2014-08-13 16:24:27.678333 7fd93dff0700 5 -- op tracker -- , seq: 983, time: 2014-08-13 16:24:27.678333, event: done, request: pg_notify(22.4df(4) epoch 194452) v5 -3213> 2014-08-13 16:24:27.678353 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 10 ==== pg_notify(22.39d(4) epoch 194452) v5 ==== 942+0+0 (3225162188 0 0) 0x80df540 con 0x72de9e0 -3212> 2014-08-13 16:24:27.678372 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:26.801090, event: header_read, request: pg_notify(22.39d(4) epoch 194452) v5 -3211> 2014-08-13 16:24:27.678382 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:26.801092, event: throttled, request: pg_notify(22.39d(4) epoch 194452) v5 -3210> 2014-08-13 16:24:27.678391 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:26.801177, event: all_read, request: pg_notify(22.39d(4) epoch 194452) v5 -3209> 2014-08-13 16:24:27.678400 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:27.678370, event: dispatched, request: pg_notify(22.39d(4) epoch 194452) v5 -3208> 2014-08-13 16:24:27.678409 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:27.678408, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194452) v5 -3207> 2014-08-13 16:24:27.678419 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:27.678419, event: started, request: pg_notify(22.39d(4) epoch 194452) v5 -3206> 2014-08-13 16:24:27.678441 7fd93dff0700 5 -- op tracker -- , seq: 984, time: 2014-08-13 16:24:27.678441, event: done, request: pg_notify(22.39d(4) epoch 194452) v5 -3205> 2014-08-13 16:24:27.678456 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 11 ==== pg_notify(22.775(17) epoch 194452) v5 ==== 1665+0+0 (3969714512 0 0) 0x9106c80 con 0x8e29fa0 -3204> 2014-08-13 16:24:27.678475 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:26.838711, event: header_read, request: pg_notify(22.775(17) epoch 194452) v5 -3203> 2014-08-13 16:24:27.678485 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:26.838712, event: throttled, request: pg_notify(22.775(17) epoch 194452) v5 -3202> 2014-08-13 16:24:27.678494 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:26.838806, event: all_read, request: pg_notify(22.775(17) epoch 194452) v5 -3201> 2014-08-13 16:24:27.678503 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:27.678473, event: dispatched, request: pg_notify(22.775(17) epoch 194452) v5 -3200> 2014-08-13 16:24:27.678512 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:27.678512, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194452) v5 -3199> 2014-08-13 16:24:27.678523 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:27.678522, event: started, request: pg_notify(22.775(17) epoch 194452) v5 -3198> 2014-08-13 16:24:27.678548 7fd93dff0700 5 -- op tracker -- , seq: 985, time: 2014-08-13 16:24:27.678548, event: done, request: pg_notify(22.775(17) epoch 194452) v5 -3197> 2014-08-13 16:24:27.678570 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 16 ==== pg_notify(22.f3(15) epoch 194451) v5 ==== 1551+0+0 (1455029361 0 0) 0x834bd40 con 0x8e286e0 -3196> 2014-08-13 16:24:27.678588 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:25.706447, event: header_read, request: pg_notify(22.f3(15) epoch 194451) v5 -3195> 2014-08-13 16:24:27.678599 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:25.706449, event: throttled, request: pg_notify(22.f3(15) epoch 194451) v5 -3194> 2014-08-13 16:24:27.678608 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:25.706575, event: all_read, request: pg_notify(22.f3(15) epoch 194451) v5 -3193> 2014-08-13 16:24:27.678617 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:27.678586, event: dispatched, request: pg_notify(22.f3(15) epoch 194451) v5 -3192> 2014-08-13 16:24:27.678628 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:27.678627, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194451) v5 -3191> 2014-08-13 16:24:27.678638 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:27.678638, event: started, request: pg_notify(22.f3(15) epoch 194451) v5 -3190> 2014-08-13 16:24:27.678664 7fd93dff0700 5 -- op tracker -- , seq: 986, time: 2014-08-13 16:24:27.678663, event: done, request: pg_notify(22.f3(15) epoch 194451) v5 -3189> 2014-08-13 16:24:27.678689 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 10 ==== pg_notify(22.4df(15) epoch 194452) v5 ==== 1551+0+0 (846176032 0 0) 0x8429c00 con 0x8e2c360 -3188> 2014-08-13 16:24:27.678708 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:26.843483, event: header_read, request: pg_notify(22.4df(15) epoch 194452) v5 -3187> 2014-08-13 16:24:27.678718 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:26.843484, event: throttled, request: pg_notify(22.4df(15) epoch 194452) v5 -3186> 2014-08-13 16:24:27.678727 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:26.843581, event: all_read, request: pg_notify(22.4df(15) epoch 194452) v5 -3185> 2014-08-13 16:24:27.678733 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:27.678706, event: dispatched, request: pg_notify(22.4df(15) epoch 194452) v5 -3184> 2014-08-13 16:24:27.678739 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:27.678739, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194452) v5 -3183> 2014-08-13 16:24:27.678750 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:27.678750, event: started, request: pg_notify(22.4df(15) epoch 194452) v5 -3182> 2014-08-13 16:24:27.678775 7fd93dff0700 5 -- op tracker -- , seq: 987, time: 2014-08-13 16:24:27.678775, event: done, request: pg_notify(22.4df(15) epoch 194452) v5 -3181> 2014-08-13 16:24:27.678817 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 11 ==== pg_notify(24.181(25) epoch 194452) v5 ==== 2159+0+0 (1021914780 0 0) 0x9117a80 con 0x7379ce0 -3180> 2014-08-13 16:24:27.678838 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:26.785433, event: header_read, request: pg_notify(24.181(25) epoch 194452) v5 -3179> 2014-08-13 16:24:27.678849 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:26.785434, event: throttled, request: pg_notify(24.181(25) epoch 194452) v5 -3178> 2014-08-13 16:24:27.678858 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:26.785552, event: all_read, request: pg_notify(24.181(25) epoch 194452) v5 -3177> 2014-08-13 16:24:27.678866 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:27.678835, event: dispatched, request: pg_notify(24.181(25) epoch 194452) v5 -3176> 2014-08-13 16:24:27.678876 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:27.678875, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194452) v5 -3175> 2014-08-13 16:24:27.678886 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:27.678886, event: started, request: pg_notify(24.181(25) epoch 194452) v5 -3174> 2014-08-13 16:24:27.678912 7fd93dff0700 5 -- op tracker -- , seq: 988, time: 2014-08-13 16:24:27.678912, event: done, request: pg_notify(24.181(25) epoch 194452) v5 -3173> 2014-08-13 16:24:27.678947 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 11 ==== pg_notify(22.1f1(4) epoch 194452) v5 ==== 942+0+0 (2807224355 0 0) 0x87e6040 con 0x73ca680 -3172> 2014-08-13 16:24:27.678967 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:26.717911, event: header_read, request: pg_notify(22.1f1(4) epoch 194452) v5 -3171> 2014-08-13 16:24:27.678977 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:26.717912, event: throttled, request: pg_notify(22.1f1(4) epoch 194452) v5 -3170> 2014-08-13 16:24:27.678986 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:26.718012, event: all_read, request: pg_notify(22.1f1(4) epoch 194452) v5 -3169> 2014-08-13 16:24:27.678995 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:27.678965, event: dispatched, request: pg_notify(22.1f1(4) epoch 194452) v5 -3168> 2014-08-13 16:24:27.679005 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:27.679004, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194452) v5 -3167> 2014-08-13 16:24:27.679015 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:27.679015, event: started, request: pg_notify(22.1f1(4) epoch 194452) v5 -3166> 2014-08-13 16:24:27.679041 7fd93dff0700 5 -- op tracker -- , seq: 989, time: 2014-08-13 16:24:27.679041, event: done, request: pg_notify(22.1f1(4) epoch 194452) v5 -3165> 2014-08-13 16:24:27.679064 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 17 ==== pg_notify(22.3af(17) epoch 194452) v5 ==== 1665+0+0 (1233134261 0 0) 0x8c89340 con 0x73cdc20 -3164> 2014-08-13 16:24:27.679083 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:26.712761, event: header_read, request: pg_notify(22.3af(17) epoch 194452) v5 -3163> 2014-08-13 16:24:27.679093 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:26.712762, event: throttled, request: pg_notify(22.3af(17) epoch 194452) v5 -3162> 2014-08-13 16:24:27.679102 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:26.712846, event: all_read, request: pg_notify(22.3af(17) epoch 194452) v5 -3161> 2014-08-13 16:24:27.679111 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:27.679081, event: dispatched, request: pg_notify(22.3af(17) epoch 194452) v5 -3160> 2014-08-13 16:24:27.679120 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:27.679120, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194452) v5 -3159> 2014-08-13 16:24:27.679131 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:27.679131, event: started, request: pg_notify(22.3af(17) epoch 194452) v5 -3158> 2014-08-13 16:24:27.679168 7fd93dff0700 5 -- op tracker -- , seq: 990, time: 2014-08-13 16:24:27.679168, event: done, request: pg_notify(22.3af(17) epoch 194452) v5 -3157> 2014-08-13 16:24:27.679197 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 8 ==== pg_notify(22.481(32) epoch 194452) v5 ==== 2518+0+0 (191424032 0 0) 0x87e6ac0 con 0x73ccba0 -3156> 2014-08-13 16:24:27.679217 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:26.732044, event: header_read, request: pg_notify(22.481(32) epoch 194452) v5 -3155> 2014-08-13 16:24:27.679227 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:26.732045, event: throttled, request: pg_notify(22.481(32) epoch 194452) v5 -3154> 2014-08-13 16:24:27.679236 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:26.732175, event: all_read, request: pg_notify(22.481(32) epoch 194452) v5 -3153> 2014-08-13 16:24:27.679245 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:27.679215, event: dispatched, request: pg_notify(22.481(32) epoch 194452) v5 -3152> 2014-08-13 16:24:27.679252 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:27.679252, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194452) v5 -3151> 2014-08-13 16:24:27.679263 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:27.679262, event: started, request: pg_notify(22.481(32) epoch 194452) v5 -3150> 2014-08-13 16:24:27.679289 7fd93dff0700 5 -- op tracker -- , seq: 991, time: 2014-08-13 16:24:27.679289, event: done, request: pg_notify(22.481(32) epoch 194452) v5 -3149> 2014-08-13 16:24:27.679313 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 8 ==== pg_notify(22.3d9(35) epoch 194452) v5 ==== 2603+0+0 (394297655 0 0) 0x834f540 con 0x7462100 -3148> 2014-08-13 16:24:27.679332 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:26.759253, event: header_read, request: pg_notify(22.3d9(35) epoch 194452) v5 -3147> 2014-08-13 16:24:27.679342 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:26.759254, event: throttled, request: pg_notify(22.3d9(35) epoch 194452) v5 -3146> 2014-08-13 16:24:27.679351 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:26.759395, event: all_read, request: pg_notify(22.3d9(35) epoch 194452) v5 -3145> 2014-08-13 16:24:27.679359 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:27.679330, event: dispatched, request: pg_notify(22.3d9(35) epoch 194452) v5 -3144> 2014-08-13 16:24:27.679369 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:27.679369, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194452) v5 -3143> 2014-08-13 16:24:27.679380 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:27.679379, event: started, request: pg_notify(22.3d9(35) epoch 194452) v5 -3142> 2014-08-13 16:24:27.679406 7fd93dff0700 5 -- op tracker -- , seq: 992, time: 2014-08-13 16:24:27.679405, event: done, request: pg_notify(22.3d9(35) epoch 194452) v5 -3141> 2014-08-13 16:24:27.679431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 14 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x33e3a80 con 0x7723440 -3140> 2014-08-13 16:24:27.679452 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3139> 2014-08-13 16:24:27.679466 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 10 ==== pg_notify(22.619(17) epoch 194452) v5 ==== 1665+0+0 (2906983997 0 0) 0x7d040c0 con 0x7d2d280 -3138> 2014-08-13 16:24:27.679483 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:26.702297, event: header_read, request: pg_notify(22.619(17) epoch 194452) v5 -3137> 2014-08-13 16:24:27.679493 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:26.702298, event: throttled, request: pg_notify(22.619(17) epoch 194452) v5 -3136> 2014-08-13 16:24:27.679512 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:26.702406, event: all_read, request: pg_notify(22.619(17) epoch 194452) v5 -3135> 2014-08-13 16:24:27.679521 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:27.679481, event: dispatched, request: pg_notify(22.619(17) epoch 194452) v5 -3134> 2014-08-13 16:24:27.679530 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:27.679530, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194452) v5 -3133> 2014-08-13 16:24:27.679541 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:27.679541, event: started, request: pg_notify(22.619(17) epoch 194452) v5 -3132> 2014-08-13 16:24:27.679568 7fd93dff0700 5 -- op tracker -- , seq: 993, time: 2014-08-13 16:24:27.679568, event: done, request: pg_notify(22.619(17) epoch 194452) v5 -3131> 2014-08-13 16:24:27.679618 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 10 ==== pg_notify(22.775(17) epoch 194452) v5 ==== 1665+0+0 (3511574388 0 0) 0x80d8c40 con 0x821f900 -3130> 2014-08-13 16:24:27.679640 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:26.713649, event: header_read, request: pg_notify(22.775(17) epoch 194452) v5 -3129> 2014-08-13 16:24:27.679651 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:26.713650, event: throttled, request: pg_notify(22.775(17) epoch 194452) v5 -3128> 2014-08-13 16:24:27.679660 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:26.713739, event: all_read, request: pg_notify(22.775(17) epoch 194452) v5 -3127> 2014-08-13 16:24:27.679669 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:27.679638, event: dispatched, request: pg_notify(22.775(17) epoch 194452) v5 -3126> 2014-08-13 16:24:27.679679 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:27.679678, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194452) v5 -3125> 2014-08-13 16:24:27.679690 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:27.679690, event: started, request: pg_notify(22.775(17) epoch 194452) v5 -3124> 2014-08-13 16:24:27.679722 7fd93dff0700 5 -- op tracker -- , seq: 994, time: 2014-08-13 16:24:27.679721, event: done, request: pg_notify(22.775(17) epoch 194452) v5 -3123> 2014-08-13 16:24:27.679750 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 8 ==== pg_notify(22.3d9(39) epoch 194452) v5 ==== 2815+0+0 (3409187712 0 0) 0x81c6200 con 0x80c7e80 -3122> 2014-08-13 16:24:27.679770 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:26.722703, event: header_read, request: pg_notify(22.3d9(39) epoch 194452) v5 -3121> 2014-08-13 16:24:27.679781 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:26.722706, event: throttled, request: pg_notify(22.3d9(39) epoch 194452) v5 -3120> 2014-08-13 16:24:27.679790 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:26.722846, event: all_read, request: pg_notify(22.3d9(39) epoch 194452) v5 -3119> 2014-08-13 16:24:27.679799 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:27.679768, event: dispatched, request: pg_notify(22.3d9(39) epoch 194452) v5 -3118> 2014-08-13 16:24:27.679808 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:27.679808, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194452) v5 -3117> 2014-08-13 16:24:27.679819 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:27.679819, event: started, request: pg_notify(22.3d9(39) epoch 194452) v5 -3116> 2014-08-13 16:24:27.679848 7fd93dff0700 5 -- op tracker -- , seq: 995, time: 2014-08-13 16:24:27.679848, event: done, request: pg_notify(22.3d9(39) epoch 194452) v5 -3115> 2014-08-13 16:24:27.679878 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 10 ==== pg_notify(22.481(34) epoch 194452) v5 ==== 2620+0+0 (3964503830 0 0) 0x83488c0 con 0x80c7d20 -3114> 2014-08-13 16:24:27.679897 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:26.692745, event: header_read, request: pg_notify(22.481(34) epoch 194452) v5 -3113> 2014-08-13 16:24:27.679908 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:26.692746, event: throttled, request: pg_notify(22.481(34) epoch 194452) v5 -3112> 2014-08-13 16:24:27.679916 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:26.692879, event: all_read, request: pg_notify(22.481(34) epoch 194452) v5 -3111> 2014-08-13 16:24:27.679925 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:27.679895, event: dispatched, request: pg_notify(22.481(34) epoch 194452) v5 -3110> 2014-08-13 16:24:27.679935 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:27.679935, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194452) v5 -3109> 2014-08-13 16:24:27.679945 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:27.679945, event: started, request: pg_notify(22.481(34) epoch 194452) v5 -3108> 2014-08-13 16:24:27.679971 7fd93dff0700 5 -- op tracker -- , seq: 996, time: 2014-08-13 16:24:27.679971, event: done, request: pg_notify(22.481(34) epoch 194452) v5 -3107> 2014-08-13 16:24:27.679998 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 18 ==== pg_notify(24.30b(4) epoch 194452) v5 ==== 976+0+0 (4150398994 0 0) 0x834dcc0 con 0x4e4b860 -3106> 2014-08-13 16:24:27.680017 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:26.707375, event: header_read, request: pg_notify(24.30b(4) epoch 194452) v5 -3105> 2014-08-13 16:24:27.680027 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:26.707376, event: throttled, request: pg_notify(24.30b(4) epoch 194452) v5 -3104> 2014-08-13 16:24:27.680036 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:26.707464, event: all_read, request: pg_notify(24.30b(4) epoch 194452) v5 -3103> 2014-08-13 16:24:27.680045 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:27.680015, event: dispatched, request: pg_notify(24.30b(4) epoch 194452) v5 -3102> 2014-08-13 16:24:27.680054 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:27.680054, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194452) v5 -3101> 2014-08-13 16:24:27.680065 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:27.680065, event: started, request: pg_notify(24.30b(4) epoch 194452) v5 -3100> 2014-08-13 16:24:27.680092 7fd93dff0700 5 -- op tracker -- , seq: 997, time: 2014-08-13 16:24:27.680091, event: done, request: pg_notify(24.30b(4) epoch 194452) v5 -3099> 2014-08-13 16:24:27.680111 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 22 ==== pg_notify(22.1d2(17) epoch 194452) v5 ==== 1669+0+0 (128117045 0 0) 0x8408700 con 0x4e4dd80 -3098> 2014-08-13 16:24:27.680129 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:26.698108, event: header_read, request: pg_notify(22.1d2(17) epoch 194452) v5 -3097> 2014-08-13 16:24:27.680139 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:26.698109, event: throttled, request: pg_notify(22.1d2(17) epoch 194452) v5 -3096> 2014-08-13 16:24:27.680148 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:26.698436, event: all_read, request: pg_notify(22.1d2(17) epoch 194452) v5 -3095> 2014-08-13 16:24:27.680164 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:27.680127, event: dispatched, request: pg_notify(22.1d2(17) epoch 194452) v5 -3094> 2014-08-13 16:24:27.680175 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:27.680174, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194452) v5 -3093> 2014-08-13 16:24:27.680186 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:27.680185, event: started, request: pg_notify(22.1d2(17) epoch 194452) v5 -3092> 2014-08-13 16:24:27.680223 7fd93dff0700 5 -- op tracker -- , seq: 998, time: 2014-08-13 16:24:27.680222, event: done, request: pg_notify(22.1d2(17) epoch 194452) v5 -3091> 2014-08-13 16:24:27.680256 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 16 ==== pg_notify(22.1f1(17) epoch 194452) v5 ==== 1665+0+0 (3022637147 0 0) 0x899c600 con 0x4e4ce60 -3090> 2014-08-13 16:24:27.680282 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:26.714243, event: header_read, request: pg_notify(22.1f1(17) epoch 194452) v5 -3089> 2014-08-13 16:24:27.680293 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:26.714244, event: throttled, request: pg_notify(22.1f1(17) epoch 194452) v5 -3088> 2014-08-13 16:24:27.680302 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:26.714371, event: all_read, request: pg_notify(22.1f1(17) epoch 194452) v5 -3087> 2014-08-13 16:24:27.680311 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:27.680280, event: dispatched, request: pg_notify(22.1f1(17) epoch 194452) v5 -3086> 2014-08-13 16:24:27.680321 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:27.680320, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194452) v5 -3085> 2014-08-13 16:24:27.680331 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:27.680331, event: started, request: pg_notify(22.1f1(17) epoch 194452) v5 -3084> 2014-08-13 16:24:27.680364 7fd93dff0700 5 -- op tracker -- , seq: 999, time: 2014-08-13 16:24:27.680364, event: done, request: pg_notify(22.1f1(17) epoch 194452) v5 -3083> 2014-08-13 16:24:27.680399 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 26 ==== pg_notify(24.34(42) epoch 194452) v5 ==== 2882+0+0 (1441377519 0 0) 0x80dd5c0 con 0x4e4f900 -3082> 2014-08-13 16:24:27.680425 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:26.707529, event: header_read, request: pg_notify(24.34(42) epoch 194452) v5 -3081> 2014-08-13 16:24:27.680436 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:26.707530, event: throttled, request: pg_notify(24.34(42) epoch 194452) v5 -3080> 2014-08-13 16:24:27.680445 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:26.707697, event: all_read, request: pg_notify(24.34(42) epoch 194452) v5 -3079> 2014-08-13 16:24:27.680453 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:27.680416, event: dispatched, request: pg_notify(24.34(42) epoch 194452) v5 -3078> 2014-08-13 16:24:27.680463 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:27.680463, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194452) v5 -3077> 2014-08-13 16:24:27.680474 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:27.680474, event: started, request: pg_notify(24.34(42) epoch 194452) v5 -3076> 2014-08-13 16:24:27.680501 7fd93dff0700 5 -- op tracker -- , seq: 1000, time: 2014-08-13 16:24:27.680501, event: done, request: pg_notify(24.34(42) epoch 194452) v5 -3075> 2014-08-13 16:24:27.680549 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 27 ==== pg_notify(24.136(17) epoch 194452) v5 ==== 1695+0+0 (955137782 0 0) 0x81c2680 con 0x4fd8000 -3074> 2014-08-13 16:24:27.680570 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:26.707533, event: header_read, request: pg_notify(24.136(17) epoch 194452) v5 -3073> 2014-08-13 16:24:27.680581 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:26.707535, event: throttled, request: pg_notify(24.136(17) epoch 194452) v5 -3072> 2014-08-13 16:24:27.680590 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:26.707666, event: all_read, request: pg_notify(24.136(17) epoch 194452) v5 -3071> 2014-08-13 16:24:27.680599 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:27.680567, event: dispatched, request: pg_notify(24.136(17) epoch 194452) v5 -3070> 2014-08-13 16:24:27.680609 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:27.680608, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194452) v5 -3069> 2014-08-13 16:24:27.680619 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:27.680619, event: started, request: pg_notify(24.136(17) epoch 194452) v5 -3068> 2014-08-13 16:24:27.680645 7fd93dff0700 5 -- op tracker -- , seq: 1001, time: 2014-08-13 16:24:27.680645, event: done, request: pg_notify(24.136(17) epoch 194452) v5 -3067> 2014-08-13 16:24:27.680707 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 20 ==== pg_notify(22.99(15) epoch 194452) v5 ==== 1551+0+0 (1518074028 0 0) 0x9110700 con 0x4fe1600 -3066> 2014-08-13 16:24:27.680727 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:26.761145, event: header_read, request: pg_notify(22.99(15) epoch 194452) v5 -3065> 2014-08-13 16:24:27.680738 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:26.761149, event: throttled, request: pg_notify(22.99(15) epoch 194452) v5 -3064> 2014-08-13 16:24:27.680747 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:26.761280, event: all_read, request: pg_notify(22.99(15) epoch 194452) v5 -3063> 2014-08-13 16:24:27.680756 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:27.680725, event: dispatched, request: pg_notify(22.99(15) epoch 194452) v5 -3062> 2014-08-13 16:24:27.680766 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:27.680765, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194452) v5 -3061> 2014-08-13 16:24:27.680776 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:27.680776, event: started, request: pg_notify(22.99(15) epoch 194452) v5 -3060> 2014-08-13 16:24:27.680801 7fd93dff0700 5 -- op tracker -- , seq: 1002, time: 2014-08-13 16:24:27.680801, event: done, request: pg_notify(22.99(15) epoch 194452) v5 -3059> 2014-08-13 16:24:27.680834 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 20 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7b17740 con 0x4fe40a0 -3058> 2014-08-13 16:24:27.680856 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3057> 2014-08-13 16:24:27.680886 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 23 ==== pg_notify(24.34(42) epoch 194452) v5 ==== 2882+0+0 (2817265122 0 0) 0x775f700 con 0x4fe69e0 -3056> 2014-08-13 16:24:27.680905 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:26.763351, event: header_read, request: pg_notify(24.34(42) epoch 194452) v5 -3055> 2014-08-13 16:24:27.680915 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:26.763352, event: throttled, request: pg_notify(24.34(42) epoch 194452) v5 -3054> 2014-08-13 16:24:27.680925 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:26.763500, event: all_read, request: pg_notify(24.34(42) epoch 194452) v5 -3053> 2014-08-13 16:24:27.680934 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:27.680903, event: dispatched, request: pg_notify(24.34(42) epoch 194452) v5 -3052> 2014-08-13 16:24:27.680943 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:27.680943, event: waiting_for_osdmap, request: pg_notify(24.34(42) epoch 194452) v5 -3051> 2014-08-13 16:24:27.680954 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:27.680954, event: started, request: pg_notify(24.34(42) epoch 194452) v5 -3050> 2014-08-13 16:24:27.680984 7fd93dff0700 5 -- op tracker -- , seq: 1003, time: 2014-08-13 16:24:27.680983, event: done, request: pg_notify(24.34(42) epoch 194452) v5 -3049> 2014-08-13 16:24:27.681011 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 22 ==== pg_notify(22.3e4(17) epoch 194452) v5 ==== 1665+0+0 (1232323781 0 0) 0x834cd00 con 0x4fda940 -3048> 2014-08-13 16:24:27.681030 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:26.711383, event: header_read, request: pg_notify(22.3e4(17) epoch 194452) v5 -3047> 2014-08-13 16:24:27.681040 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:26.711384, event: throttled, request: pg_notify(22.3e4(17) epoch 194452) v5 -3046> 2014-08-13 16:24:27.681049 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:26.711468, event: all_read, request: pg_notify(22.3e4(17) epoch 194452) v5 -3045> 2014-08-13 16:24:27.681058 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:27.681028, event: dispatched, request: pg_notify(22.3e4(17) epoch 194452) v5 -3044> 2014-08-13 16:24:27.681067 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:27.681067, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194452) v5 -3043> 2014-08-13 16:24:27.681078 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:27.681078, event: started, request: pg_notify(22.3e4(17) epoch 194452) v5 -3042> 2014-08-13 16:24:27.681104 7fd93dff0700 5 -- op tracker -- , seq: 1004, time: 2014-08-13 16:24:27.681103, event: done, request: pg_notify(22.3e4(17) epoch 194452) v5 -3041> 2014-08-13 16:24:27.681125 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 6 ==== pg_notify(24.39a(42) epoch 194452) v5 ==== 3042+0+0 (2114632510 0 0) 0x775a300 con 0x4fd9b80 -3040> 2014-08-13 16:24:27.681144 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:26.707114, event: header_read, request: pg_notify(24.39a(42) epoch 194452) v5 -3039> 2014-08-13 16:24:27.681152 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:26.707115, event: throttled, request: pg_notify(24.39a(42) epoch 194452) v5 -3038> 2014-08-13 16:24:27.681170 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:26.707395, event: all_read, request: pg_notify(24.39a(42) epoch 194452) v5 -3037> 2014-08-13 16:24:27.681181 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:27.681142, event: dispatched, request: pg_notify(24.39a(42) epoch 194452) v5 -3036> 2014-08-13 16:24:27.681188 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:27.681188, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194452) v5 -3035> 2014-08-13 16:24:27.681195 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:27.681195, event: started, request: pg_notify(24.39a(42) epoch 194452) v5 -3034> 2014-08-13 16:24:27.681218 7fd93dff0700 5 -- op tracker -- , seq: 1005, time: 2014-08-13 16:24:27.681218, event: done, request: pg_notify(24.39a(42) epoch 194452) v5 -3033> 2014-08-13 16:24:27.681243 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 23 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7aac0 con 0x50889a0 -3032> 2014-08-13 16:24:27.681264 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -3031> 2014-08-13 16:24:27.681276 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 18 ==== pg_notify(22.619(4) epoch 194452) v5 ==== 942+0+0 (2369130443 0 0) 0x834fe00 con 0x508a680 -3030> 2014-08-13 16:24:27.681290 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:26.696957, event: header_read, request: pg_notify(22.619(4) epoch 194452) v5 -3029> 2014-08-13 16:24:27.681297 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:26.696958, event: throttled, request: pg_notify(22.619(4) epoch 194452) v5 -3028> 2014-08-13 16:24:27.681307 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:26.697032, event: all_read, request: pg_notify(22.619(4) epoch 194452) v5 -3027> 2014-08-13 16:24:27.681316 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:27.681288, event: dispatched, request: pg_notify(22.619(4) epoch 194452) v5 -3026> 2014-08-13 16:24:27.681326 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:27.681326, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194452) v5 -3025> 2014-08-13 16:24:27.681336 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:27.681336, event: started, request: pg_notify(22.619(4) epoch 194452) v5 -3024> 2014-08-13 16:24:27.681362 7fd93dff0700 5 -- op tracker -- , seq: 1006, time: 2014-08-13 16:24:27.681362, event: done, request: pg_notify(22.619(4) epoch 194452) v5 -3023> 2014-08-13 16:24:27.681387 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 32 ==== pg_notify(22.99(15) epoch 194452) v5 ==== 1551+0+0 (510829563 0 0) 0x9106740 con 0x508bde0 -3022> 2014-08-13 16:24:27.681406 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:26.712818, event: header_read, request: pg_notify(22.99(15) epoch 194452) v5 -3021> 2014-08-13 16:24:27.681416 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:26.712819, event: throttled, request: pg_notify(22.99(15) epoch 194452) v5 -3020> 2014-08-13 16:24:27.681425 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:26.712932, event: all_read, request: pg_notify(22.99(15) epoch 194452) v5 -3019> 2014-08-13 16:24:27.681434 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:27.681404, event: dispatched, request: pg_notify(22.99(15) epoch 194452) v5 -3018> 2014-08-13 16:24:27.681441 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:27.681440, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194452) v5 -3017> 2014-08-13 16:24:27.681451 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:27.681451, event: started, request: pg_notify(22.99(15) epoch 194452) v5 -3016> 2014-08-13 16:24:27.681476 7fd93dff0700 5 -- op tracker -- , seq: 1007, time: 2014-08-13 16:24:27.681476, event: done, request: pg_notify(22.99(15) epoch 194452) v5 -3015> 2014-08-13 16:24:27.681498 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 18 ==== pg_notify(22.707(19) epoch 194452) v5 ==== 1783+0+0 (652908893 0 0) 0x87e4440 con 0x50ba520 -3014> 2014-08-13 16:24:27.681516 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:26.714213, event: header_read, request: pg_notify(22.707(19) epoch 194452) v5 -3013> 2014-08-13 16:24:27.681527 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:26.714214, event: throttled, request: pg_notify(22.707(19) epoch 194452) v5 -3012> 2014-08-13 16:24:27.681536 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:26.714310, event: all_read, request: pg_notify(22.707(19) epoch 194452) v5 -3011> 2014-08-13 16:24:27.681544 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:27.681514, event: dispatched, request: pg_notify(22.707(19) epoch 194452) v5 -3010> 2014-08-13 16:24:27.681554 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:27.681553, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194452) v5 -3009> 2014-08-13 16:24:27.681564 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:27.681564, event: started, request: pg_notify(22.707(19) epoch 194452) v5 -3008> 2014-08-13 16:24:27.681589 7fd93dff0700 5 -- op tracker -- , seq: 1008, time: 2014-08-13 16:24:27.681589, event: done, request: pg_notify(22.707(19) epoch 194452) v5 -3007> 2014-08-13 16:24:27.681622 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 11 ==== pg_notify(22.113(17) epoch 194452) v5 ==== 1669+0+0 (959299296 0 0) 0x80da4c0 con 0x50bfa60 -3006> 2014-08-13 16:24:27.681641 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:26.786066, event: header_read, request: pg_notify(22.113(17) epoch 194452) v5 -3005> 2014-08-13 16:24:27.681651 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:26.786067, event: throttled, request: pg_notify(22.113(17) epoch 194452) v5 -3004> 2014-08-13 16:24:27.681660 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:26.786170, event: all_read, request: pg_notify(22.113(17) epoch 194452) v5 -3003> 2014-08-13 16:24:27.681668 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:27.681639, event: dispatched, request: pg_notify(22.113(17) epoch 194452) v5 -3002> 2014-08-13 16:24:27.681678 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:27.681678, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194452) v5 -3001> 2014-08-13 16:24:27.681688 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:27.681688, event: started, request: pg_notify(22.113(17) epoch 194452) v5 -3000> 2014-08-13 16:24:27.681714 7fd93dff0700 5 -- op tracker -- , seq: 1009, time: 2014-08-13 16:24:27.681713, event: done, request: pg_notify(22.113(17) epoch 194452) v5 -2999> 2014-08-13 16:24:27.681735 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 19 ==== pg_notify(22.362(17) epoch 194452) v5 ==== 1665+0+0 (98706259 0 0) 0x8c8ee40 con 0x510b9c0 -2998> 2014-08-13 16:24:27.681753 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:26.760847, event: header_read, request: pg_notify(22.362(17) epoch 194452) v5 -2997> 2014-08-13 16:24:27.681763 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:26.760849, event: throttled, request: pg_notify(22.362(17) epoch 194452) v5 -2996> 2014-08-13 16:24:27.681773 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:26.760960, event: all_read, request: pg_notify(22.362(17) epoch 194452) v5 -2995> 2014-08-13 16:24:27.681781 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:27.681751, event: dispatched, request: pg_notify(22.362(17) epoch 194452) v5 -2994> 2014-08-13 16:24:27.681801 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:27.681800, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194452) v5 -2993> 2014-08-13 16:24:27.681811 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:27.681811, event: started, request: pg_notify(22.362(17) epoch 194452) v5 -2992> 2014-08-13 16:24:27.681837 7fd93dff0700 5 -- op tracker -- , seq: 1010, time: 2014-08-13 16:24:27.681837, event: done, request: pg_notify(22.362(17) epoch 194452) v5 -2991> 2014-08-13 16:24:27.681859 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 19 ==== pg_notify(22.481(27) epoch 194451) v5 ==== 2235+0+0 (1034846803 0 0) 0x80da300 con 0x5108b00 -2990> 2014-08-13 16:24:27.681878 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:25.779273, event: header_read, request: pg_notify(22.481(27) epoch 194451) v5 -2989> 2014-08-13 16:24:27.681888 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:25.779277, event: throttled, request: pg_notify(22.481(27) epoch 194451) v5 -2988> 2014-08-13 16:24:27.681897 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:25.779423, event: all_read, request: pg_notify(22.481(27) epoch 194451) v5 -2987> 2014-08-13 16:24:27.681905 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:27.681876, event: dispatched, request: pg_notify(22.481(27) epoch 194451) v5 -2986> 2014-08-13 16:24:27.681915 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:27.681914, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194451) v5 -2985> 2014-08-13 16:24:27.681925 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:27.681925, event: started, request: pg_notify(22.481(27) epoch 194451) v5 -2984> 2014-08-13 16:24:27.681951 7fd93dff0700 5 -- op tracker -- , seq: 1011, time: 2014-08-13 16:24:27.681951, event: done, request: pg_notify(22.481(27) epoch 194451) v5 -2983> 2014-08-13 16:24:27.681985 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 11 ==== pg_notify(22.3d9(39) epoch 194452) v5 ==== 2815+0+0 (3848953967 0 0) 0x81c2d80 con 0x510dac0 -2982> 2014-08-13 16:24:27.682005 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:26.691622, event: header_read, request: pg_notify(22.3d9(39) epoch 194452) v5 -2981> 2014-08-13 16:24:27.682015 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:26.691624, event: throttled, request: pg_notify(22.3d9(39) epoch 194452) v5 -2980> 2014-08-13 16:24:27.682024 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:26.691757, event: all_read, request: pg_notify(22.3d9(39) epoch 194452) v5 -2979> 2014-08-13 16:24:27.682033 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:27.682003, event: dispatched, request: pg_notify(22.3d9(39) epoch 194452) v5 -2978> 2014-08-13 16:24:27.682043 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:27.682042, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194452) v5 -2977> 2014-08-13 16:24:27.682053 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:27.682053, event: started, request: pg_notify(22.3d9(39) epoch 194452) v5 -2976> 2014-08-13 16:24:27.682079 7fd93dff0700 5 -- op tracker -- , seq: 1012, time: 2014-08-13 16:24:27.682079, event: done, request: pg_notify(22.3d9(39) epoch 194452) v5 -2975> 2014-08-13 16:24:27.682106 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 7 ==== pg_notify(24.39a(48) epoch 194452) v5 ==== 3328+0+0 (1980776475 0 0) 0x834c0c0 con 0x51127e0 -2974> 2014-08-13 16:24:27.682124 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:26.696050, event: header_read, request: pg_notify(24.39a(48) epoch 194452) v5 -2973> 2014-08-13 16:24:27.682135 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:26.696051, event: throttled, request: pg_notify(24.39a(48) epoch 194452) v5 -2972> 2014-08-13 16:24:27.682144 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:26.696205, event: all_read, request: pg_notify(24.39a(48) epoch 194452) v5 -2971> 2014-08-13 16:24:27.682149 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:27.682122, event: dispatched, request: pg_notify(24.39a(48) epoch 194452) v5 -2970> 2014-08-13 16:24:27.682168 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:27.682167, event: waiting_for_osdmap, request: pg_notify(24.39a(48) epoch 194452) v5 -2969> 2014-08-13 16:24:27.682179 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:27.682179, event: started, request: pg_notify(24.39a(48) epoch 194452) v5 -2968> 2014-08-13 16:24:27.682206 7fd93dff0700 5 -- op tracker -- , seq: 1013, time: 2014-08-13 16:24:27.682206, event: done, request: pg_notify(24.39a(48) epoch 194452) v5 -2967> 2014-08-13 16:24:27.682245 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 35 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3016554300 0 0) 0x8bc2c00 con 0x5112520 -2966> 2014-08-13 16:24:27.682272 7fd93dff0700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.673272, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2965> 2014-08-13 16:24:27.682287 7fd93dff0700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.673274, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2964> 2014-08-13 16:24:27.682299 7fd93dff0700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.673359, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2963> 2014-08-13 16:24:27.682312 7fd93dff0700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.682269, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2962> 2014-08-13 16:24:27.682325 7fd93dff0700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.682325, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2961> 2014-08-13 16:24:27.682359 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 16 ==== pg_notify(22.4df(15) epoch 194452) v5 ==== 1551+0+0 (846176032 0 0) 0x81c1500 con 0x5112260 -2960> 2014-08-13 16:24:27.682374 7fd9371e5700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.682374, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2959> 2014-08-13 16:24:27.682420 7fd9371e5700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.682420, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2958> 2014-08-13 16:24:27.682443 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:26.714226, event: header_read, request: pg_notify(22.4df(15) epoch 194452) v5 -2957> 2014-08-13 16:24:27.682460 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:26.714227, event: throttled, request: pg_notify(22.4df(15) epoch 194452) v5 -2956> 2014-08-13 16:24:27.682473 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:26.714318, event: all_read, request: pg_notify(22.4df(15) epoch 194452) v5 -2955> 2014-08-13 16:24:27.682486 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:27.682375, event: dispatched, request: pg_notify(22.4df(15) epoch 194452) v5 -2954> 2014-08-13 16:24:27.682500 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:27.682499, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194452) v5 -2953> 2014-08-13 16:24:27.682515 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:27.682515, event: started, request: pg_notify(22.4df(15) epoch 194452) v5 -2952> 2014-08-13 16:24:27.682531 7fd9371e5700 5 -- op tracker -- , seq: 1014, time: 2014-08-13 16:24:27.682443, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2951> 2014-08-13 16:24:27.682569 7fd93dff0700 5 -- op tracker -- , seq: 1015, time: 2014-08-13 16:24:27.682546, event: done, request: pg_notify(22.4df(15) epoch 194452) v5 -2950> 2014-08-13 16:24:27.682612 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 14 ==== pg_notify(22.1d2(4) epoch 194452) v5 ==== 942+0+0 (1547168879 0 0) 0x9110c40 con 0x5114620 -2949> 2014-08-13 16:24:27.682644 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:26.793184, event: header_read, request: pg_notify(22.1d2(4) epoch 194452) v5 -2948> 2014-08-13 16:24:27.682655 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:26.793185, event: throttled, request: pg_notify(22.1d2(4) epoch 194452) v5 -2947> 2014-08-13 16:24:27.682664 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:26.793260, event: all_read, request: pg_notify(22.1d2(4) epoch 194452) v5 -2946> 2014-08-13 16:24:27.682673 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:27.682632, event: dispatched, request: pg_notify(22.1d2(4) epoch 194452) v5 -2945> 2014-08-13 16:24:27.682683 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:27.682683, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194452) v5 -2944> 2014-08-13 16:24:27.682694 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:27.682694, event: started, request: pg_notify(22.1d2(4) epoch 194452) v5 -2943> 2014-08-13 16:24:27.682721 7fd93dff0700 5 -- op tracker -- , seq: 1016, time: 2014-08-13 16:24:27.682721, event: done, request: pg_notify(22.1d2(4) epoch 194452) v5 -2942> 2014-08-13 16:24:27.682753 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 21 ==== pg_notify(22.39d(15) epoch 194452) v5 ==== 1551+0+0 (3191715540 0 0) 0x840ec80 con 0x5114360 -2941> 2014-08-13 16:24:27.682773 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:26.719319, event: header_read, request: pg_notify(22.39d(15) epoch 194452) v5 -2940> 2014-08-13 16:24:27.682783 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:26.719321, event: throttled, request: pg_notify(22.39d(15) epoch 194452) v5 -2939> 2014-08-13 16:24:27.682793 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:26.719436, event: all_read, request: pg_notify(22.39d(15) epoch 194452) v5 -2938> 2014-08-13 16:24:27.682801 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:27.682770, event: dispatched, request: pg_notify(22.39d(15) epoch 194452) v5 -2937> 2014-08-13 16:24:27.682811 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:27.682811, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194452) v5 -2936> 2014-08-13 16:24:27.682821 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:27.682821, event: started, request: pg_notify(22.39d(15) epoch 194452) v5 -2935> 2014-08-13 16:24:27.682856 7fd93dff0700 5 -- op tracker -- , seq: 1017, time: 2014-08-13 16:24:27.682856, event: done, request: pg_notify(22.39d(15) epoch 194452) v5 -2934> 2014-08-13 16:24:27.682887 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 11 ==== pg_notify(24.1bb(21) epoch 194452) v5 ==== 1931+0+0 (2975422628 0 0) 0x899abc0 con 0x5113de0 -2933> 2014-08-13 16:24:27.682906 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:26.704708, event: header_read, request: pg_notify(24.1bb(21) epoch 194452) v5 -2932> 2014-08-13 16:24:27.682917 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:26.704712, event: throttled, request: pg_notify(24.1bb(21) epoch 194452) v5 -2931> 2014-08-13 16:24:27.682926 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:26.704846, event: all_read, request: pg_notify(24.1bb(21) epoch 194452) v5 -2930> 2014-08-13 16:24:27.682934 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:27.682904, event: dispatched, request: pg_notify(24.1bb(21) epoch 194452) v5 -2929> 2014-08-13 16:24:27.682944 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:27.682944, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194452) v5 -2928> 2014-08-13 16:24:27.682955 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:27.682955, event: started, request: pg_notify(24.1bb(21) epoch 194452) v5 -2927> 2014-08-13 16:24:27.682987 7fd93dff0700 5 -- op tracker -- , seq: 1018, time: 2014-08-13 16:24:27.682987, event: done, request: pg_notify(24.1bb(21) epoch 194452) v5 -2926> 2014-08-13 16:24:27.683019 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 21 ==== pg_notify(22.1d2(17) epoch 194452) v5 ==== 1669+0+0 (1307486094 0 0) 0x840f540 con 0x5113b20 -2925> 2014-08-13 16:24:27.683039 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:26.712889, event: header_read, request: pg_notify(22.1d2(17) epoch 194452) v5 -2924> 2014-08-13 16:24:27.683049 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:26.712890, event: throttled, request: pg_notify(22.1d2(17) epoch 194452) v5 -2923> 2014-08-13 16:24:27.683058 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:26.713008, event: all_read, request: pg_notify(22.1d2(17) epoch 194452) v5 -2922> 2014-08-13 16:24:27.683067 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:27.683037, event: dispatched, request: pg_notify(22.1d2(17) epoch 194452) v5 -2921> 2014-08-13 16:24:27.683076 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:27.683076, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194452) v5 -2920> 2014-08-13 16:24:27.683087 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:27.683087, event: started, request: pg_notify(22.1d2(17) epoch 194452) v5 -2919> 2014-08-13 16:24:27.683115 7fd93dff0700 5 -- op tracker -- , seq: 1019, time: 2014-08-13 16:24:27.683115, event: done, request: pg_notify(22.1d2(17) epoch 194452) v5 -2918> 2014-08-13 16:24:27.683137 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 15 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf33c0 con 0x5113860 -2917> 2014-08-13 16:24:27.683170 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2916> 2014-08-13 16:24:27.683189 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 28 ==== pg_notify(22.3d9(6) epoch 194452) v5 ==== 1060+0+0 (394664757 0 0) 0x9102140 con 0x5111fa0 -2915> 2014-08-13 16:24:27.683207 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:26.717176, event: header_read, request: pg_notify(22.3d9(6) epoch 194452) v5 -2914> 2014-08-13 16:24:27.683218 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:26.717178, event: throttled, request: pg_notify(22.3d9(6) epoch 194452) v5 -2913> 2014-08-13 16:24:27.683227 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:26.717256, event: all_read, request: pg_notify(22.3d9(6) epoch 194452) v5 -2912> 2014-08-13 16:24:27.683235 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:27.683205, event: dispatched, request: pg_notify(22.3d9(6) epoch 194452) v5 -2911> 2014-08-13 16:24:27.683245 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:27.683245, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194452) v5 -2910> 2014-08-13 16:24:27.683256 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:27.683255, event: started, request: pg_notify(22.3d9(6) epoch 194452) v5 -2909> 2014-08-13 16:24:27.683281 7fd93dff0700 5 -- op tracker -- , seq: 1020, time: 2014-08-13 16:24:27.683280, event: done, request: pg_notify(22.3d9(6) epoch 194452) v5 -2908> 2014-08-13 16:24:27.683301 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 33 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e79440 con 0x51135a0 -2907> 2014-08-13 16:24:27.683321 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2906> 2014-08-13 16:24:27.683333 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 25 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x859bcc0 con 0x5111a20 -2905> 2014-08-13 16:24:27.683350 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2904> 2014-08-13 16:24:27.683364 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 19 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7cf5580 con 0x7313860 -2903> 2014-08-13 16:24:27.683382 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2902> 2014-08-13 16:24:27.683392 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 13 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x506fbc0 con 0x7316720 -2901> 2014-08-13 16:24:27.683410 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2900> 2014-08-13 16:24:27.683421 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 14 ==== pg_notify(22.113(4) epoch 194452) v5 ==== 942+0+0 (3314075783 0 0) 0x80d9880 con 0x7314fc0 -2899> 2014-08-13 16:24:27.683437 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:26.737406, event: header_read, request: pg_notify(22.113(4) epoch 194452) v5 -2898> 2014-08-13 16:24:27.683448 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:26.737408, event: throttled, request: pg_notify(22.113(4) epoch 194452) v5 -2897> 2014-08-13 16:24:27.683457 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:26.737501, event: all_read, request: pg_notify(22.113(4) epoch 194452) v5 -2896> 2014-08-13 16:24:27.683465 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:27.683435, event: dispatched, request: pg_notify(22.113(4) epoch 194452) v5 -2895> 2014-08-13 16:24:27.683475 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:27.683474, event: waiting_for_osdmap, request: pg_notify(22.113(4) epoch 194452) v5 -2894> 2014-08-13 16:24:27.683485 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:27.683485, event: started, request: pg_notify(22.113(4) epoch 194452) v5 -2893> 2014-08-13 16:24:27.683512 7fd93dff0700 5 -- op tracker -- , seq: 1021, time: 2014-08-13 16:24:27.683511, event: done, request: pg_notify(22.113(4) epoch 194452) v5 -2892> 2014-08-13 16:24:27.683531 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 17 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7d100 con 0x8e286e0 -2891> 2014-08-13 16:24:27.683550 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2890> 2014-08-13 16:24:27.683563 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 15 ==== pg_notify(24.9b(4) epoch 194452) v5 ==== 976+0+0 (3213739316 0 0) 0x89981c0 con 0x7723440 -2889> 2014-08-13 16:24:27.683579 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:26.727716, event: header_read, request: pg_notify(24.9b(4) epoch 194452) v5 -2888> 2014-08-13 16:24:27.683590 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:26.727717, event: throttled, request: pg_notify(24.9b(4) epoch 194452) v5 -2887> 2014-08-13 16:24:27.683599 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:26.727801, event: all_read, request: pg_notify(24.9b(4) epoch 194452) v5 -2886> 2014-08-13 16:24:27.683608 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:27.683577, event: dispatched, request: pg_notify(24.9b(4) epoch 194452) v5 -2885> 2014-08-13 16:24:27.683618 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:27.683617, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194452) v5 -2884> 2014-08-13 16:24:27.683628 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:27.683628, event: started, request: pg_notify(24.9b(4) epoch 194452) v5 -2883> 2014-08-13 16:24:27.683654 7fd93dff0700 5 -- op tracker -- , seq: 1022, time: 2014-08-13 16:24:27.683654, event: done, request: pg_notify(24.9b(4) epoch 194452) v5 -2882> 2014-08-13 16:24:27.683673 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 21 ==== pg_notify(22.3af(4) epoch 194452) v5 ==== 942+0+0 (2745143593 0 0) 0x84081c0 con 0x4fe40a0 -2881> 2014-08-13 16:24:27.683692 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:26.698124, event: header_read, request: pg_notify(22.3af(4) epoch 194452) v5 -2880> 2014-08-13 16:24:27.683702 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:26.698125, event: throttled, request: pg_notify(22.3af(4) epoch 194452) v5 -2879> 2014-08-13 16:24:27.683711 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:26.698212, event: all_read, request: pg_notify(22.3af(4) epoch 194452) v5 -2878> 2014-08-13 16:24:27.683720 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:27.683690, event: dispatched, request: pg_notify(22.3af(4) epoch 194452) v5 -2877> 2014-08-13 16:24:27.683729 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:27.683729, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194452) v5 -2876> 2014-08-13 16:24:27.683740 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:27.683740, event: started, request: pg_notify(22.3af(4) epoch 194452) v5 -2875> 2014-08-13 16:24:27.683765 7fd93dff0700 5 -- op tracker -- , seq: 1023, time: 2014-08-13 16:24:27.683765, event: done, request: pg_notify(22.3af(4) epoch 194452) v5 -2874> 2014-08-13 16:24:27.683791 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 7 ==== pg_notify(24.39a(42) epoch 194452) v5 ==== 3042+0+0 (1794217119 0 0) 0x8571180 con 0x4fd9b80 -2873> 2014-08-13 16:24:27.683810 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.660354, event: header_read, request: pg_notify(24.39a(42) epoch 194452) v5 -2872> 2014-08-13 16:24:27.683821 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.660357, event: throttled, request: pg_notify(24.39a(42) epoch 194452) v5 -2871> 2014-08-13 16:24:27.683830 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.660508, event: all_read, request: pg_notify(24.39a(42) epoch 194452) v5 -2870> 2014-08-13 16:24:27.683838 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.683808, event: dispatched, request: pg_notify(24.39a(42) epoch 194452) v5 -2869> 2014-08-13 16:24:27.683845 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.683845, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194452) v5 -2868> 2014-08-13 16:24:27.683855 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.683855, event: started, request: pg_notify(24.39a(42) epoch 194452) v5 -2867> 2014-08-13 16:24:27.683883 7fd93dff0700 5 -- op tracker -- , seq: 1024, time: 2014-08-13 16:24:27.683883, event: done, request: pg_notify(24.39a(42) epoch 194452) v5 -2866> 2014-08-13 16:24:27.683912 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 24 ==== pg_notify(22.113(17) epoch 194452) v5 ==== 1669+0+0 (1935798363 0 0) 0x87e2f40 con 0x50889a0 -2865> 2014-08-13 16:24:27.683931 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:26.709825, event: header_read, request: pg_notify(22.113(17) epoch 194452) v5 -2864> 2014-08-13 16:24:27.683941 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:26.709826, event: throttled, request: pg_notify(22.113(17) epoch 194452) v5 -2863> 2014-08-13 16:24:27.683951 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:26.709972, event: all_read, request: pg_notify(22.113(17) epoch 194452) v5 -2862> 2014-08-13 16:24:27.683960 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:27.683929, event: dispatched, request: pg_notify(22.113(17) epoch 194452) v5 -2861> 2014-08-13 16:24:27.683970 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:27.683969, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194452) v5 -2860> 2014-08-13 16:24:27.683980 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:27.683980, event: started, request: pg_notify(22.113(17) epoch 194452) v5 -2859> 2014-08-13 16:24:27.684006 7fd93dff0700 5 -- op tracker -- , seq: 1025, time: 2014-08-13 16:24:27.684006, event: done, request: pg_notify(22.113(17) epoch 194452) v5 -2858> 2014-08-13 16:24:27.684036 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 33 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (443487197 0 0) 0x76fac00 con 0x508bde0 -2857> 2014-08-13 16:24:27.684060 7fd93dff0700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:26.948005, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2856> 2014-08-13 16:24:27.684074 7fd93dff0700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:26.948009, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2855> 2014-08-13 16:24:27.684087 7fd93dff0700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:26.948124, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2854> 2014-08-13 16:24:27.684100 7fd93dff0700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:27.684058, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2853> 2014-08-13 16:24:27.684113 7fd93dff0700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:27.684113, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2852> 2014-08-13 16:24:27.684144 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 20 ==== osd_map(194452..194452 src has 185139..194452) v3 ==== 414+0+0 (2152727924 0 0) 0x7e7c380 con 0x5108b00 -2851> 2014-08-13 16:24:27.684159 7fd9367e4700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:27.684159, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2850> 2014-08-13 16:24:27.684174 7fd93dff0700 3 osd.4 194452 handle_osd_map epochs [194452,194452], i have 194452, src has [185139,194452] -2849> 2014-08-13 16:24:27.684197 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 8 ==== pg_notify(24.39a(48) epoch 194452) v5 ==== 3328+0+0 (1128002696 0 0) 0x834d5c0 con 0x51127e0 -2848> 2014-08-13 16:24:27.684203 7fd9367e4700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:27.684203, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2847> 2014-08-13 16:24:27.684227 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.660145, event: header_read, request: pg_notify(24.39a(48) epoch 194452) v5 -2846> 2014-08-13 16:24:27.684227 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e88a00 con 0x508bde0 -2845> 2014-08-13 16:24:27.684240 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.660147, event: throttled, request: pg_notify(24.39a(48) epoch 194452) v5 -2844> 2014-08-13 16:24:27.684260 7fd9367e4700 5 -- op tracker -- , seq: 1026, time: 2014-08-13 16:24:27.684260, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2843> 2014-08-13 16:24:27.684282 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.660315, event: all_read, request: pg_notify(24.39a(48) epoch 194452) v5 -2842> 2014-08-13 16:24:27.684297 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.684213, event: dispatched, request: pg_notify(24.39a(48) epoch 194452) v5 -2841> 2014-08-13 16:24:27.684311 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.684310, event: waiting_for_osdmap, request: pg_notify(24.39a(48) epoch 194452) v5 -2840> 2014-08-13 16:24:27.684343 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.684325, event: started, request: pg_notify(24.39a(48) epoch 194452) v5 -2839> 2014-08-13 16:24:27.684377 7fd93dff0700 5 -- op tracker -- , seq: 1027, time: 2014-08-13 16:24:27.684377, event: done, request: pg_notify(24.39a(48) epoch 194452) v5 -2838> 2014-08-13 16:24:27.684413 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 16 ==== pg_notify(24.b6(4) epoch 194452) v5 ==== 976+0+0 (628515087 0 0) 0x8409180 con 0x5113860 -2837> 2014-08-13 16:24:27.684435 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:26.698873, event: header_read, request: pg_notify(24.b6(4) epoch 194452) v5 -2836> 2014-08-13 16:24:27.684446 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:26.698874, event: throttled, request: pg_notify(24.b6(4) epoch 194452) v5 -2835> 2014-08-13 16:24:27.684455 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:26.698940, event: all_read, request: pg_notify(24.b6(4) epoch 194452) v5 -2834> 2014-08-13 16:24:27.684461 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:27.684433, event: dispatched, request: pg_notify(24.b6(4) epoch 194452) v5 -2833> 2014-08-13 16:24:27.684471 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:27.684470, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194452) v5 -2832> 2014-08-13 16:24:27.684497 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:27.684497, event: started, request: pg_notify(24.b6(4) epoch 194452) v5 -2831> 2014-08-13 16:24:27.684527 7fd93dff0700 5 -- op tracker -- , seq: 1028, time: 2014-08-13 16:24:27.684527, event: done, request: pg_notify(24.b6(4) epoch 194452) v5 -2830> 2014-08-13 16:24:27.684555 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 34 ==== pg_notify(24.3d(18) epoch 194452) v5 ==== 1754+0+0 (870149011 0 0) 0x775f540 con 0x51135a0 -2829> 2014-08-13 16:24:27.684575 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:26.719661, event: header_read, request: pg_notify(24.3d(18) epoch 194452) v5 -2828> 2014-08-13 16:24:27.684585 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:26.719662, event: throttled, request: pg_notify(24.3d(18) epoch 194452) v5 -2827> 2014-08-13 16:24:27.684595 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:26.719767, event: all_read, request: pg_notify(24.3d(18) epoch 194452) v5 -2826> 2014-08-13 16:24:27.684603 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:27.684573, event: dispatched, request: pg_notify(24.3d(18) epoch 194452) v5 -2825> 2014-08-13 16:24:27.684613 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:27.684613, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194452) v5 -2824> 2014-08-13 16:24:27.684623 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:27.684623, event: started, request: pg_notify(24.3d(18) epoch 194452) v5 -2823> 2014-08-13 16:24:27.684650 7fd93dff0700 5 -- op tracker -- , seq: 1029, time: 2014-08-13 16:24:27.684649, event: done, request: pg_notify(24.3d(18) epoch 194452) v5 -2822> 2014-08-13 16:24:27.684671 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 26 ==== pg_notify(22.48e(15) epoch 194452) v5 ==== 1551+0+0 (1723431917 0 0) 0x84096c0 con 0x5111a20 -2821> 2014-08-13 16:24:27.684690 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:26.715829, event: header_read, request: pg_notify(22.48e(15) epoch 194452) v5 -2820> 2014-08-13 16:24:27.684700 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:26.715830, event: throttled, request: pg_notify(22.48e(15) epoch 194452) v5 -2819> 2014-08-13 16:24:27.684709 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:26.715916, event: all_read, request: pg_notify(22.48e(15) epoch 194452) v5 -2818> 2014-08-13 16:24:27.684718 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:27.684688, event: dispatched, request: pg_notify(22.48e(15) epoch 194452) v5 -2817> 2014-08-13 16:24:27.684727 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:27.684727, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194452) v5 -2816> 2014-08-13 16:24:27.684738 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:27.684737, event: started, request: pg_notify(22.48e(15) epoch 194452) v5 -2815> 2014-08-13 16:24:27.684763 7fd93dff0700 5 -- op tracker -- , seq: 1030, time: 2014-08-13 16:24:27.684763, event: done, request: pg_notify(22.48e(15) epoch 194452) v5 -2814> 2014-08-13 16:24:27.684787 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 20 ==== pg_notify(22.362(17),24.39a(42) epoch 194452) v5 ==== 4695+0+0 (59410538 0 0) 0x775ec80 con 0x7313860 -2813> 2014-08-13 16:24:27.684807 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:26.696844, event: header_read, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2812> 2014-08-13 16:24:27.684818 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:26.696846, event: throttled, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2811> 2014-08-13 16:24:27.684828 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:26.697011, event: all_read, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2810> 2014-08-13 16:24:27.684837 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:27.684805, event: dispatched, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2809> 2014-08-13 16:24:27.684848 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:27.684848, event: waiting_for_osdmap, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2808> 2014-08-13 16:24:27.684859 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:27.684859, event: started, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2807> 2014-08-13 16:24:27.684901 7fd93dff0700 5 -- op tracker -- , seq: 1031, time: 2014-08-13 16:24:27.684900, event: done, request: pg_notify(22.362(17),24.39a(42) epoch 194452) v5 -2806> 2014-08-13 16:24:27.684937 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 14 ==== pg_notify(22.1f1(17) epoch 194452) v5 ==== 1665+0+0 (3022637147 0 0) 0x842bb80 con 0x7316720 -2805> 2014-08-13 16:24:27.684956 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:26.762145, event: header_read, request: pg_notify(22.1f1(17) epoch 194452) v5 -2804> 2014-08-13 16:24:27.684966 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:26.762146, event: throttled, request: pg_notify(22.1f1(17) epoch 194452) v5 -2803> 2014-08-13 16:24:27.684976 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:26.762257, event: all_read, request: pg_notify(22.1f1(17) epoch 194452) v5 -2802> 2014-08-13 16:24:27.684984 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:27.684954, event: dispatched, request: pg_notify(22.1f1(17) epoch 194452) v5 -2801> 2014-08-13 16:24:27.684994 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:27.684993, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194452) v5 -2800> 2014-08-13 16:24:27.685004 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:27.685004, event: started, request: pg_notify(22.1f1(17) epoch 194452) v5 -2799> 2014-08-13 16:24:27.685030 7fd93dff0700 5 -- op tracker -- , seq: 1032, time: 2014-08-13 16:24:27.685030, event: done, request: pg_notify(22.1f1(17) epoch 194452) v5 -2798> 2014-08-13 16:24:27.685052 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 15 ==== pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 ==== 3436+0+0 (547221251 0 0) 0x80d9dc0 con 0x7314fc0 -2797> 2014-08-13 16:24:27.685071 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:26.737619, event: header_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2796> 2014-08-13 16:24:27.685082 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:26.737620, event: throttled, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2795> 2014-08-13 16:24:27.685091 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:26.737739, event: all_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2794> 2014-08-13 16:24:27.685101 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:27.685069, event: dispatched, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2793> 2014-08-13 16:24:27.685111 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:27.685111, event: waiting_for_osdmap, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2792> 2014-08-13 16:24:27.685123 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:27.685122, event: started, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2791> 2014-08-13 16:24:27.685171 7fd93dff0700 5 -- op tracker -- , seq: 1033, time: 2014-08-13 16:24:27.685171, event: done, request: pg_notify(22.3e4(17),22.707(19) epoch 194452) v5 -2790> 2014-08-13 16:24:27.685200 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 18 ==== pg_notify(22.3af(17) epoch 194452) v5 ==== 1665+0+0 (1233134261 0 0) 0x8349880 con 0x8e286e0 -2789> 2014-08-13 16:24:27.685220 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:26.703355, event: header_read, request: pg_notify(22.3af(17) epoch 194452) v5 -2788> 2014-08-13 16:24:27.685231 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:26.703356, event: throttled, request: pg_notify(22.3af(17) epoch 194452) v5 -2787> 2014-08-13 16:24:27.685240 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:26.703438, event: all_read, request: pg_notify(22.3af(17) epoch 194452) v5 -2786> 2014-08-13 16:24:27.685246 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:27.685218, event: dispatched, request: pg_notify(22.3af(17) epoch 194452) v5 -2785> 2014-08-13 16:24:27.685255 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:27.685255, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194452) v5 -2784> 2014-08-13 16:24:27.685265 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:27.685265, event: started, request: pg_notify(22.3af(17) epoch 194452) v5 -2783> 2014-08-13 16:24:27.685291 7fd93dff0700 5 -- op tracker -- , seq: 1034, time: 2014-08-13 16:24:27.685291, event: done, request: pg_notify(22.3af(17) epoch 194452) v5 -2782> 2014-08-13 16:24:27.685312 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 16 ==== pg_notify(22.481(27) epoch 194452) v5 ==== 2235+0+0 (143347419 0 0) 0x775e740 con 0x7723440 -2781> 2014-08-13 16:24:27.685330 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:26.729272, event: header_read, request: pg_notify(22.481(27) epoch 194452) v5 -2780> 2014-08-13 16:24:27.685340 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:26.729275, event: throttled, request: pg_notify(22.481(27) epoch 194452) v5 -2779> 2014-08-13 16:24:27.685349 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:26.729407, event: all_read, request: pg_notify(22.481(27) epoch 194452) v5 -2778> 2014-08-13 16:24:27.685357 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:27.685328, event: dispatched, request: pg_notify(22.481(27) epoch 194452) v5 -2777> 2014-08-13 16:24:27.685367 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:27.685367, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194452) v5 -2776> 2014-08-13 16:24:27.685377 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:27.685377, event: started, request: pg_notify(22.481(27) epoch 194452) v5 -2775> 2014-08-13 16:24:27.685403 7fd93dff0700 5 -- op tracker -- , seq: 1035, time: 2014-08-13 16:24:27.685403, event: done, request: pg_notify(22.481(27) epoch 194452) v5 -2774> 2014-08-13 16:24:27.685432 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 22 ==== pg_notify(22.362(4) epoch 194452) v5 ==== 942+0+0 (1451187629 0 0) 0x840f700 con 0x4fe40a0 -2773> 2014-08-13 16:24:27.685451 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:26.712338, event: header_read, request: pg_notify(22.362(4) epoch 194452) v5 -2772> 2014-08-13 16:24:27.685461 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:26.712340, event: throttled, request: pg_notify(22.362(4) epoch 194452) v5 -2771> 2014-08-13 16:24:27.685470 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:26.712432, event: all_read, request: pg_notify(22.362(4) epoch 194452) v5 -2770> 2014-08-13 16:24:27.685479 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:27.685449, event: dispatched, request: pg_notify(22.362(4) epoch 194452) v5 -2769> 2014-08-13 16:24:27.685489 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:27.685489, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194452) v5 -2768> 2014-08-13 16:24:27.685500 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:27.685500, event: started, request: pg_notify(22.362(4) epoch 194452) v5 -2767> 2014-08-13 16:24:27.685525 7fd93dff0700 5 -- op tracker -- , seq: 1036, time: 2014-08-13 16:24:27.685525, event: done, request: pg_notify(22.362(4) epoch 194452) v5 -2766> 2014-08-13 16:24:27.685544 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 25 ==== pg_notify(24.181(9) epoch 194452) v5 ==== 1263+0+0 (1905897228 0 0) 0x87e6580 con 0x50889a0 -2765> 2014-08-13 16:24:27.685563 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:26.710089, event: header_read, request: pg_notify(24.181(9) epoch 194452) v5 -2764> 2014-08-13 16:24:27.685573 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:26.710091, event: throttled, request: pg_notify(24.181(9) epoch 194452) v5 -2763> 2014-08-13 16:24:27.685582 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:26.710210, event: all_read, request: pg_notify(24.181(9) epoch 194452) v5 -2762> 2014-08-13 16:24:27.685591 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:27.685561, event: dispatched, request: pg_notify(24.181(9) epoch 194452) v5 -2761> 2014-08-13 16:24:27.685600 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:27.685600, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194452) v5 -2760> 2014-08-13 16:24:27.685611 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:27.685611, event: started, request: pg_notify(24.181(9) epoch 194452) v5 -2759> 2014-08-13 16:24:27.685637 7fd93dff0700 5 -- op tracker -- , seq: 1037, time: 2014-08-13 16:24:27.685637, event: done, request: pg_notify(24.181(9) epoch 194452) v5 -2758> 2014-08-13 16:24:27.685657 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 34 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2336145528 0 0) 0x7976e00 con 0x508bde0 -2757> 2014-08-13 16:24:27.685680 7fd93dff0700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:26.949820, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2756> 2014-08-13 16:24:27.685694 7fd93dff0700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:26.949824, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2755> 2014-08-13 16:24:27.685707 7fd93dff0700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:26.949933, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2754> 2014-08-13 16:24:27.685719 7fd93dff0700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:27.685678, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2753> 2014-08-13 16:24:27.685732 7fd93dff0700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:27.685732, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2752> 2014-08-13 16:24:27.685762 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 21 ==== pg_notify(22.3e4(4) epoch 194452) v5 ==== 942+0+0 (816386465 0 0) 0x80d9340 con 0x5108b00 -2751> 2014-08-13 16:24:27.685775 7fd9371e5700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:27.685774, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2750> 2014-08-13 16:24:27.685803 7fd9371e5700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:27.685802, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2749> 2014-08-13 16:24:27.685823 7fd9371e5700 5 -- op tracker -- , seq: 1038, time: 2014-08-13 16:24:27.685823, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2748> 2014-08-13 16:24:27.685854 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:26.761105, event: header_read, request: pg_notify(22.3e4(4) epoch 194452) v5 -2747> 2014-08-13 16:24:27.685870 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:26.761106, event: throttled, request: pg_notify(22.3e4(4) epoch 194452) v5 -2746> 2014-08-13 16:24:27.685879 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:26.761191, event: all_read, request: pg_notify(22.3e4(4) epoch 194452) v5 -2745> 2014-08-13 16:24:27.685888 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:27.685778, event: dispatched, request: pg_notify(22.3e4(4) epoch 194452) v5 -2744> 2014-08-13 16:24:27.685898 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:27.685898, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194452) v5 -2743> 2014-08-13 16:24:27.685909 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:27.685909, event: started, request: pg_notify(22.3e4(4) epoch 194452) v5 -2742> 2014-08-13 16:24:27.685936 7fd93dff0700 5 -- op tracker -- , seq: 1039, time: 2014-08-13 16:24:27.685936, event: done, request: pg_notify(22.3e4(4) epoch 194452) v5 -2741> 2014-08-13 16:24:27.685957 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 9 ==== pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 ==== 1143+0+0 (2054634609 0 0) 0x8669200 con 0x51127e0 -2740> 2014-08-13 16:24:27.685980 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.678507, event: header_read, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2739> 2014-08-13 16:24:27.685992 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.678510, event: throttled, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2738> 2014-08-13 16:24:27.686003 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.678616, event: all_read, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2737> 2014-08-13 16:24:27.686014 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.685977, event: dispatched, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2736> 2014-08-13 16:24:27.686026 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.686025, event: waiting_for_osdmap, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2735> 2014-08-13 16:24:27.686039 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.686039, event: started, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2734> 2014-08-13 16:24:27.686062 7fd93dff0700 5 -- op tracker -- , seq: 1040, time: 2014-08-13 16:24:27.686062, event: done, request: pg_log(24.39a epoch 194452 log log((194422'13151,194422'13151], crt=194383'13093) query_epoch 194452) v4 -2733> 2014-08-13 16:24:27.686082 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 17 ==== pg_notify(22.48e(15) epoch 194452) v5 ==== 1551+0+0 (581968058 0 0) 0x8408c40 con 0x5113860 -2732> 2014-08-13 16:24:27.686101 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:26.700635, event: header_read, request: pg_notify(22.48e(15) epoch 194452) v5 -2731> 2014-08-13 16:24:27.686111 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:26.700639, event: throttled, request: pg_notify(22.48e(15) epoch 194452) v5 -2730> 2014-08-13 16:24:27.686120 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:26.700752, event: all_read, request: pg_notify(22.48e(15) epoch 194452) v5 -2729> 2014-08-13 16:24:27.686094 7fd9371e5700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/GetMissing 0.008638 8 0.000269 -2728> 2014-08-13 16:24:27.686129 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:27.686099, event: dispatched, request: pg_notify(22.48e(15) epoch 194452) v5 -2727> 2014-08-13 16:24:27.686139 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:27.686138, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194452) v5 -2726> 2014-08-13 16:24:27.686141 7fd9371e5700 5 osd.4 pg_epoch: 194452 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] enter Started/Primary/Peering/WaitUpThru -2725> 2014-08-13 16:24:27.686149 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:27.686149, event: started, request: pg_notify(22.48e(15) epoch 194452) v5 -2724> 2014-08-13 16:24:27.686180 7fd93dff0700 5 -- op tracker -- , seq: 1041, time: 2014-08-13 16:24:27.686179, event: done, request: pg_notify(22.48e(15) epoch 194452) v5 -2723> 2014-08-13 16:24:27.686201 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 35 ==== pg_notify(22.3d9(32) epoch 194452) v5 ==== 2458+0+0 (3668834958 0 0) 0x775a840 con 0x51135a0 -2722> 2014-08-13 16:24:27.686225 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:26.719966, event: header_read, request: pg_notify(22.3d9(32) epoch 194452) v5 -2721> 2014-08-13 16:24:27.686236 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:26.719968, event: throttled, request: pg_notify(22.3d9(32) epoch 194452) v5 -2720> 2014-08-13 16:24:27.686245 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:26.720092, event: all_read, request: pg_notify(22.3d9(32) epoch 194452) v5 -2719> 2014-08-13 16:24:27.686253 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:27.686223, event: dispatched, request: pg_notify(22.3d9(32) epoch 194452) v5 -2718> 2014-08-13 16:24:27.686263 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:27.686263, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194452) v5 -2717> 2014-08-13 16:24:27.686274 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:27.686273, event: started, request: pg_notify(22.3d9(32) epoch 194452) v5 -2716> 2014-08-13 16:24:27.686299 7fd93dff0700 5 -- op tracker -- , seq: 1042, time: 2014-08-13 16:24:27.686299, event: done, request: pg_notify(22.3d9(32) epoch 194452) v5 -2715> 2014-08-13 16:24:27.686339 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 27 ==== pg_notify(24.1f7(18) epoch 194452) v5 ==== 1754+0+0 (3031344175 0 0) 0x840eac0 con 0x5111a20 -2714> 2014-08-13 16:24:27.686359 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:26.717924, event: header_read, request: pg_notify(24.1f7(18) epoch 194452) v5 -2713> 2014-08-13 16:24:27.686370 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:26.717926, event: throttled, request: pg_notify(24.1f7(18) epoch 194452) v5 -2712> 2014-08-13 16:24:27.686379 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:26.718019, event: all_read, request: pg_notify(24.1f7(18) epoch 194452) v5 -2711> 2014-08-13 16:24:27.686388 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:27.686357, event: dispatched, request: pg_notify(24.1f7(18) epoch 194452) v5 -2710> 2014-08-13 16:24:27.686397 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:27.686397, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194452) v5 -2709> 2014-08-13 16:24:27.686408 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:27.686408, event: started, request: pg_notify(24.1f7(18) epoch 194452) v5 -2708> 2014-08-13 16:24:27.686434 7fd93dff0700 5 -- op tracker -- , seq: 1043, time: 2014-08-13 16:24:27.686434, event: done, request: pg_notify(24.1f7(18) epoch 194452) v5 -2707> 2014-08-13 16:24:27.686466 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 21 ==== pg_notify(24.39a(42) epoch 194452) v5 ==== 3042+0+0 (578406792 0 0) 0x775a140 con 0x7313860 -2706> 2014-08-13 16:24:27.686485 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.660339, event: header_read, request: pg_notify(24.39a(42) epoch 194452) v5 -2705> 2014-08-13 16:24:27.686496 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.660342, event: throttled, request: pg_notify(24.39a(42) epoch 194452) v5 -2704> 2014-08-13 16:24:27.686505 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.660500, event: all_read, request: pg_notify(24.39a(42) epoch 194452) v5 -2703> 2014-08-13 16:24:27.686513 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.686483, event: dispatched, request: pg_notify(24.39a(42) epoch 194452) v5 -2702> 2014-08-13 16:24:27.686522 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.686522, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194452) v5 -2701> 2014-08-13 16:24:27.686533 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.686533, event: started, request: pg_notify(24.39a(42) epoch 194452) v5 -2700> 2014-08-13 16:24:27.686560 7fd93dff0700 5 -- op tracker -- , seq: 1044, time: 2014-08-13 16:24:27.686559, event: done, request: pg_notify(24.39a(42) epoch 194452) v5 -2699> 2014-08-13 16:24:27.686583 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 15 ==== pg_notify(22.22c(17) epoch 194452) v5 ==== 1665+0+0 (3005958969 0 0) 0x84296c0 con 0x7316720 -2698> 2014-08-13 16:24:27.686613 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:26.762290, event: header_read, request: pg_notify(22.22c(17) epoch 194452) v5 -2697> 2014-08-13 16:24:27.686623 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:26.762292, event: throttled, request: pg_notify(22.22c(17) epoch 194452) v5 -2696> 2014-08-13 16:24:27.686632 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:26.762371, event: all_read, request: pg_notify(22.22c(17) epoch 194452) v5 -2695> 2014-08-13 16:24:27.686640 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:27.686610, event: dispatched, request: pg_notify(22.22c(17) epoch 194452) v5 -2694> 2014-08-13 16:24:27.686650 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:27.686650, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194452) v5 -2693> 2014-08-13 16:24:27.686661 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:27.686661, event: started, request: pg_notify(22.22c(17) epoch 194452) v5 -2692> 2014-08-13 16:24:27.686686 7fd93dff0700 5 -- op tracker -- , seq: 1045, time: 2014-08-13 16:24:27.686686, event: done, request: pg_notify(22.22c(17) epoch 194452) v5 -2691> 2014-08-13 16:24:27.686711 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 19 ==== pg_notify(22.f3(15) epoch 194452) v5 ==== 1551+0+0 (1514724435 0 0) 0x8348e00 con 0x8e286e0 -2690> 2014-08-13 16:24:27.686730 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:26.704878, event: header_read, request: pg_notify(22.f3(15) epoch 194452) v5 -2689> 2014-08-13 16:24:27.686741 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:26.704881, event: throttled, request: pg_notify(22.f3(15) epoch 194452) v5 -2688> 2014-08-13 16:24:27.686750 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:26.704983, event: all_read, request: pg_notify(22.f3(15) epoch 194452) v5 -2687> 2014-08-13 16:24:27.686758 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:27.686728, event: dispatched, request: pg_notify(22.f3(15) epoch 194452) v5 -2686> 2014-08-13 16:24:27.686768 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:27.686767, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194452) v5 -2685> 2014-08-13 16:24:27.686778 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:27.686778, event: started, request: pg_notify(22.f3(15) epoch 194452) v5 -2684> 2014-08-13 16:24:27.686803 7fd93dff0700 5 -- op tracker -- , seq: 1046, time: 2014-08-13 16:24:27.686803, event: done, request: pg_notify(22.f3(15) epoch 194452) v5 -2683> 2014-08-13 16:24:27.686823 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 22 ==== pg_notify(22.707(4) epoch 194452) v5 ==== 942+0+0 (760171491 0 0) 0x8c88540 con 0x5108b00 -2682> 2014-08-13 16:24:27.686841 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:26.761207, event: header_read, request: pg_notify(22.707(4) epoch 194452) v5 -2681> 2014-08-13 16:24:27.686852 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:26.761209, event: throttled, request: pg_notify(22.707(4) epoch 194452) v5 -2680> 2014-08-13 16:24:27.686860 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:26.761282, event: all_read, request: pg_notify(22.707(4) epoch 194452) v5 -2679> 2014-08-13 16:24:27.686869 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:27.686839, event: dispatched, request: pg_notify(22.707(4) epoch 194452) v5 -2678> 2014-08-13 16:24:27.686878 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:27.686878, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194452) v5 -2677> 2014-08-13 16:24:27.686889 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:27.686889, event: started, request: pg_notify(22.707(4) epoch 194452) v5 -2676> 2014-08-13 16:24:27.686914 7fd93dff0700 5 -- op tracker -- , seq: 1047, time: 2014-08-13 16:24:27.686914, event: done, request: pg_notify(22.707(4) epoch 194452) v5 -2675> 2014-08-13 16:24:27.686932 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 20 ==== pg_notify(22.775(4) epoch 194452) v5 ==== 942+0+0 (3410453626 0 0) 0x834d240 con 0x8e286e0 -2674> 2014-08-13 16:24:27.686951 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:26.707955, event: header_read, request: pg_notify(22.775(4) epoch 194452) v5 -2673> 2014-08-13 16:24:27.686961 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:26.707956, event: throttled, request: pg_notify(22.775(4) epoch 194452) v5 -2672> 2014-08-13 16:24:27.686970 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:26.708038, event: all_read, request: pg_notify(22.775(4) epoch 194452) v5 -2671> 2014-08-13 16:24:27.686978 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:27.686949, event: dispatched, request: pg_notify(22.775(4) epoch 194452) v5 -2670> 2014-08-13 16:24:27.686988 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:27.686987, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194452) v5 -2669> 2014-08-13 16:24:27.686998 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:27.686998, event: started, request: pg_notify(22.775(4) epoch 194452) v5 -2668> 2014-08-13 16:24:27.687023 7fd93dff0700 5 -- op tracker -- , seq: 1048, time: 2014-08-13 16:24:27.687022, event: done, request: pg_notify(22.775(4) epoch 194452) v5 -2667> 2014-08-13 16:24:27.687042 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 23 ==== pg_notify(22.481(27) epoch 194452) v5 ==== 2235+0+0 (1151712360 0 0) 0x91101c0 con 0x5108b00 -2666> 2014-08-13 16:24:27.687060 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:26.762692, event: header_read, request: pg_notify(22.481(27) epoch 194452) v5 -2665> 2014-08-13 16:24:27.687070 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:26.762694, event: throttled, request: pg_notify(22.481(27) epoch 194452) v5 -2664> 2014-08-13 16:24:27.687079 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:26.762811, event: all_read, request: pg_notify(22.481(27) epoch 194452) v5 -2663> 2014-08-13 16:24:27.687087 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:27.687058, event: dispatched, request: pg_notify(22.481(27) epoch 194452) v5 -2662> 2014-08-13 16:24:27.687094 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:27.687094, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194452) v5 -2661> 2014-08-13 16:24:27.687104 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:27.687104, event: started, request: pg_notify(22.481(27) epoch 194452) v5 -2660> 2014-08-13 16:24:27.687129 7fd93dff0700 5 -- op tracker -- , seq: 1049, time: 2014-08-13 16:24:27.687129, event: done, request: pg_notify(22.481(27) epoch 194452) v5 -2659> 2014-08-13 16:24:27.809537 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.58 192.168.1.111:0/14316 6 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.809160) v2 ==== 47+0+0 (2955057199 0 0) 0x840cd00 con 0x7edc8e0 -2658> 2014-08-13 16:24:27.809564 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.58 192.168.1.111:0/14316 6 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:27.809160) v2 ==== 47+0+0 (2955057199 0 0) 0x840d240 con 0x7eda940 -2657> 2014-08-13 16:24:27.809596 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194452 stamp 2014-08-13 16:24:27.809160) v2 -- ?+0 0x899f380 con 0x7edc8e0 -2656> 2014-08-13 16:24:27.809676 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194452 stamp 2014-08-13 16:24:27.809160) v2 -- ?+0 0x842dcc0 con 0x7eda940 -2655> 2014-08-13 16:24:27.920700 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 28 ==== osd_map(194452..194453 src has 185139..194453) v3 ==== 1202+0+0 (3320519090 0 0) 0x8d0a880 con 0x4e3dd80 -2654> 2014-08-13 16:24:27.920779 7fd93f3f2700 3 osd.4 194452 handle_osd_map epochs [194452,194453], i have 194452, src has [185139,194453] -2653> 2014-08-13 16:24:27.926371 7fd93f3f2700 0 osd.4 194453 crush map has features 2578087936000, adjusting msgr requires for mons -2652> 2014-08-13 16:24:27.926522 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 9 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d07080 con 0x7462100 -2651> 2014-08-13 16:24:27.927097 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2650> 2014-08-13 16:24:27.927133 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 10 ==== pg_notify(22.3d9(35) epoch 194453) v5 ==== 2603+0+0 (3800955892 0 0) 0x834cb40 con 0x7462100 -2649> 2014-08-13 16:24:27.927172 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.926574, event: header_read, request: pg_notify(22.3d9(35) epoch 194453) v5 -2648> 2014-08-13 16:24:27.927188 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.926578, event: throttled, request: pg_notify(22.3d9(35) epoch 194453) v5 -2647> 2014-08-13 16:24:27.927195 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.926729, event: all_read, request: pg_notify(22.3d9(35) epoch 194453) v5 -2646> 2014-08-13 16:24:27.927200 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.927158, event: dispatched, request: pg_notify(22.3d9(35) epoch 194453) v5 -2645> 2014-08-13 16:24:27.927207 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.927207, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194453) v5 -2644> 2014-08-13 16:24:27.927217 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.927216, event: started, request: pg_notify(22.3d9(35) epoch 194453) v5 -2643> 2014-08-13 16:24:27.927244 7fd93dff0700 5 -- op tracker -- , seq: 1050, time: 2014-08-13 16:24:27.927244, event: done, request: pg_notify(22.3d9(35) epoch 194453) v5 -2642> 2014-08-13 16:24:27.931078 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering/WaitUpThru 0.244938 3 0.000111 -2641> 2014-08-13 16:24:27.931127 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped+peering m=49] exit Started/Primary/Peering 0.275557 0 0.000000 -2640> 2014-08-13 16:24:27.931159 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194449 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary/Active -2639> 2014-08-13 16:24:27.932840 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 remapped m=49] enter Started/Primary/Active/Activating -2638> 2014-08-13 16:24:27.933351 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 incomplete] exit Started/Primary/Peering/Incomplete 2.736004 13 0.000159 -2637> 2014-08-13 16:24:27.933386 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary/Peering 2.742493 0 0.000000 -2636> 2014-08-13 16:24:27.933410 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 2.742542 0 0.000000 -2635> 2014-08-13 16:24:27.933429 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started 2.742645 0 0.000000 -2634> 2014-08-13 16:24:27.933446 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194450/194450/194450) [4] r=0 lpr=194450 pi=193125-194449/42 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset -2633> 2014-08-13 16:24:27.933519 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000073 1 0.000168 -2632> 2014-08-13 16:24:27.933547 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started -2631> 2014-08-13 16:24:27.933564 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Start -2630> 2014-08-13 16:24:27.933578 7fd9367e4700 1 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -2629> 2014-08-13 16:24:27.933596 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 -2628> 2014-08-13 16:24:27.933618 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -2627> 2014-08-13 16:24:27.933635 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -2626> 2014-08-13 16:24:27.933653 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -2625> 2014-08-13 16:24:27.935276 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194447/194448/192971) [20,4]/[20,4,374] r=1 lpr=194448 pi=193125-194447/25 luod=0'0 crt=194422'15445 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 22.436596 13 0.003139 -2624> 2014-08-13 16:24:27.935313 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194447/194448/192971) [20,4]/[20,4,374] r=1 lpr=194448 pi=193125-194447/25 luod=0'0 crt=194422'15445 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive 22.436683 0 0.000000 -2623> 2014-08-13 16:24:27.935337 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194447/194448/192971) [20,4]/[20,4,374] r=1 lpr=194448 pi=193125-194447/25 luod=0'0 crt=194422'15445 lcod 0'0 active+remapped m=56] exit Started 23.535753 0 0.000000 -2622> 2014-08-13 16:24:27.935357 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194447/194448/192971) [20,4]/[20,4,374] r=1 lpr=194448 pi=193125-194447/25 luod=0'0 crt=194422'15445 lcod 0'0 active+remapped m=56] enter Reset -2621> 2014-08-13 16:24:27.935449 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000092 1 0.000141 -2620> 2014-08-13 16:24:27.935478 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Started -2619> 2014-08-13 16:24:27.935496 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Start -2618> 2014-08-13 16:24:27.935512 7fd9367e4700 1 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -2617> 2014-08-13 16:24:27.935535 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000040 0 0.000000 -2616> 2014-08-13 16:24:27.935559 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -2615> 2014-08-13 16:24:27.935675 7fd9367e4700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -2614> 2014-08-13 16:24:27.935692 7fd9367e4700 1 -- 192.168.100.115:6805/25220 --> 192.168.100.113:6789/0 -- osd_alive(want up_thru 194453 have 194453) v1 -- ?+0 0x80d8540 con 0x4e3dd80 -2613> 2014-08-13 16:24:27.935719 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- osd_map(194451..194453 src has 185139..194453) v3 -- ?+0 0x7b10480 con 0x4fe1600 -2612> 2014-08-13 16:24:27.935739 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(26) epoch 194453) v5 -- ?+0 0x8c8b480 con 0x4fe1600 -2611> 2014-08-13 16:24:27.935883 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6802/9548 -- pg_query(24.34 epoch 194453) v3 -- ?+0 0x80181e0 con 0x83002c0 -2610> 2014-08-13 16:24:27.935940 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.115:6804/58748 -- pg_info(1 pgs e194453:24.39a(52)) v4 -- ?+0 0x8c8aa00 con 0x4fd9b80 -2609> 2014-08-13 16:24:27.935969 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6800/61726 -- osd_map(194452..194453 src has 185139..194453) v3 -- ?+0 0x8bd3a80 con 0x7313860 -2608> 2014-08-13 16:24:27.936020 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6800/61726 -- pg_info(1 pgs e194453:24.39a(52)) v4 -- ?+0 0x8db0000 con 0x7313860 -2607> 2014-08-13 16:24:27.936062 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6810/41287 -- pg_info(1 pgs e194453:24.39a(52)) v4 -- ?+0 0x834fe00 con 0x51127e0 -2606> 2014-08-13 16:24:27.937218 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 14 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e786c0 con 0x72d9b80 -2605> 2014-08-13 16:24:27.937255 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2604> 2014-08-13 16:24:27.937303 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 15 ==== pg_notify(22.99(4) epoch 194453) v5 ==== 942+0+0 (1234353485 0 0) 0x80d9f80 con 0x72d9b80 -2603> 2014-08-13 16:24:27.937326 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937214, event: header_read, request: pg_notify(22.99(4) epoch 194453) v5 -2602> 2014-08-13 16:24:27.937338 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937215, event: throttled, request: pg_notify(22.99(4) epoch 194453) v5 -2601> 2014-08-13 16:24:27.937344 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937285, event: all_read, request: pg_notify(22.99(4) epoch 194453) v5 -2600> 2014-08-13 16:24:27.937353 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937323, event: dispatched, request: pg_notify(22.99(4) epoch 194453) v5 -2599> 2014-08-13 16:24:27.937361 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937361, event: waiting_for_osdmap, request: pg_notify(22.99(4) epoch 194453) v5 -2598> 2014-08-13 16:24:27.937370 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937370, event: started, request: pg_notify(22.99(4) epoch 194453) v5 -2597> 2014-08-13 16:24:27.937399 7fd93dff0700 5 -- op tracker -- , seq: 1051, time: 2014-08-13 16:24:27.937399, event: done, request: pg_notify(22.99(4) epoch 194453) v5 -2596> 2014-08-13 16:24:27.938263 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 16 ==== pg_notify(22.39d(15) epoch 194453) v5 ==== 1551+0+0 (4173937493 0 0) 0x80d8fc0 con 0x72d9b80 -2595> 2014-08-13 16:24:27.938287 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938092, event: header_read, request: pg_notify(22.39d(15) epoch 194453) v5 -2594> 2014-08-13 16:24:27.938295 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938094, event: throttled, request: pg_notify(22.39d(15) epoch 194453) v5 -2593> 2014-08-13 16:24:27.938300 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938221, event: all_read, request: pg_notify(22.39d(15) epoch 194453) v5 -2592> 2014-08-13 16:24:27.938306 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938284, event: dispatched, request: pg_notify(22.39d(15) epoch 194453) v5 -2591> 2014-08-13 16:24:27.938316 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938316, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194453) v5 -2590> 2014-08-13 16:24:27.938327 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938327, event: started, request: pg_notify(22.39d(15) epoch 194453) v5 -2589> 2014-08-13 16:24:27.938351 7fd93dff0700 5 -- op tracker -- , seq: 1052, time: 2014-08-13 16:24:27.938351, event: done, request: pg_notify(22.39d(15) epoch 194453) v5 -2588> 2014-08-13 16:24:27.944772 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 22 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8bd3a80 con 0x7313860 -2587> 2014-08-13 16:24:27.944809 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2586> 2014-08-13 16:24:27.944934 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 10 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d021c0 con 0x51127e0 -2585> 2014-08-13 16:24:27.944978 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2584> 2014-08-13 16:24:27.944991 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 23 ==== pg_notify(22.362(17) epoch 194453) v5 ==== 1665+0+0 (2693871536 0 0) 0x8349dc0 con 0x7313860 -2583> 2014-08-13 16:24:27.945008 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.944769, event: header_read, request: pg_notify(22.362(17) epoch 194453) v5 -2582> 2014-08-13 16:24:27.945023 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.944770, event: throttled, request: pg_notify(22.362(17) epoch 194453) v5 -2581> 2014-08-13 16:24:27.945033 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.944900, event: all_read, request: pg_notify(22.362(17) epoch 194453) v5 -2580> 2014-08-13 16:24:27.945043 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.945005, event: dispatched, request: pg_notify(22.362(17) epoch 194453) v5 -2579> 2014-08-13 16:24:27.945054 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.945053, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194453) v5 -2578> 2014-08-13 16:24:27.945067 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.945067, event: started, request: pg_notify(22.362(17) epoch 194453) v5 -2577> 2014-08-13 16:24:27.945101 7fd93dff0700 5 -- op tracker -- , seq: 1053, time: 2014-08-13 16:24:27.945101, event: done, request: pg_notify(22.362(17) epoch 194453) v5 -2576> 2014-08-13 16:24:27.945126 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 11 ==== pg_notify(24.39a(48) epoch 194453) v5 ==== 3328+0+0 (1761730957 0 0) 0x834fe00 con 0x51127e0 -2575> 2014-08-13 16:24:27.945143 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.944918, event: header_read, request: pg_notify(24.39a(48) epoch 194453) v5 -2574> 2014-08-13 16:24:27.945150 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.944920, event: throttled, request: pg_notify(24.39a(48) epoch 194453) v5 -2573> 2014-08-13 16:24:27.945155 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.945048, event: all_read, request: pg_notify(24.39a(48) epoch 194453) v5 -2572> 2014-08-13 16:24:27.945176 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.945140, event: dispatched, request: pg_notify(24.39a(48) epoch 194453) v5 -2571> 2014-08-13 16:24:27.945185 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.945185, event: waiting_for_osdmap, request: pg_notify(24.39a(48) epoch 194453) v5 -2570> 2014-08-13 16:24:27.945197 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.945197, event: started, request: pg_notify(24.39a(48) epoch 194453) v5 -2569> 2014-08-13 16:24:27.945224 7fd93dff0700 5 -- op tracker -- , seq: 1054, time: 2014-08-13 16:24:27.945224, event: done, request: pg_notify(24.39a(48) epoch 194453) v5 -2568> 2014-08-13 16:24:27.945805 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 24 ==== pg_notify(24.39a(42) epoch 194453) v5 ==== 3042+0+0 (3623225949 0 0) 0x834b800 con 0x7313860 -2567> 2014-08-13 16:24:27.945829 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945631, event: header_read, request: pg_notify(24.39a(42) epoch 194453) v5 -2566> 2014-08-13 16:24:27.945837 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945633, event: throttled, request: pg_notify(24.39a(42) epoch 194453) v5 -2565> 2014-08-13 16:24:27.945843 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945784, event: all_read, request: pg_notify(24.39a(42) epoch 194453) v5 -2564> 2014-08-13 16:24:27.945852 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945826, event: dispatched, request: pg_notify(24.39a(42) epoch 194453) v5 -2563> 2014-08-13 16:24:27.945862 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945862, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194453) v5 -2562> 2014-08-13 16:24:27.945871 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945871, event: started, request: pg_notify(24.39a(42) epoch 194453) v5 -2561> 2014-08-13 16:24:27.945899 7fd93dff0700 5 -- op tracker -- , seq: 1055, time: 2014-08-13 16:24:27.945899, event: done, request: pg_notify(24.39a(42) epoch 194453) v5 -2560> 2014-08-13 16:24:27.946528 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 21 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7b10480 con 0x4fe1600 -2559> 2014-08-13 16:24:27.946557 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2558> 2014-08-13 16:24:27.946657 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 22 ==== pg_notify(22.99(15) epoch 194453) v5 ==== 1551+0+0 (1485188730 0 0) 0x8c8b480 con 0x4fe1600 -2557> 2014-08-13 16:24:27.946678 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946532, event: header_read, request: pg_notify(22.99(15) epoch 194453) v5 -2556> 2014-08-13 16:24:27.946690 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946533, event: throttled, request: pg_notify(22.99(15) epoch 194453) v5 -2555> 2014-08-13 16:24:27.946699 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946636, event: all_read, request: pg_notify(22.99(15) epoch 194453) v5 -2554> 2014-08-13 16:24:27.946708 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946675, event: dispatched, request: pg_notify(22.99(15) epoch 194453) v5 -2553> 2014-08-13 16:24:27.946718 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946717, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194453) v5 -2552> 2014-08-13 16:24:27.946729 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946729, event: started, request: pg_notify(22.99(15) epoch 194453) v5 -2551> 2014-08-13 16:24:27.946757 7fd93dff0700 5 -- op tracker -- , seq: 1056, time: 2014-08-13 16:24:27.946756, event: done, request: pg_notify(22.99(15) epoch 194453) v5 -2550> 2014-08-13 16:24:27.946787 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 23 ==== pg_query(24.2c epoch 194453) v3 ==== 144+0+0 (3094323019 0 0) 0x349eea0 con 0x4fe1600 -2549> 2014-08-13 16:24:27.946807 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946669, event: header_read, request: pg_query(24.2c epoch 194453) v3 -2548> 2014-08-13 16:24:27.946817 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946671, event: throttled, request: pg_query(24.2c epoch 194453) v3 -2547> 2014-08-13 16:24:27.946826 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946716, event: all_read, request: pg_query(24.2c epoch 194453) v3 -2546> 2014-08-13 16:24:27.946835 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946804, event: dispatched, request: pg_query(24.2c epoch 194453) v3 -2545> 2014-08-13 16:24:27.946842 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946842, event: waiting_for_osdmap, request: pg_query(24.2c epoch 194453) v3 -2544> 2014-08-13 16:24:27.946853 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946853, event: started, request: pg_query(24.2c epoch 194453) v3 -2543> 2014-08-13 16:24:27.946876 7fd93dff0700 5 -- op tracker -- , seq: 1057, time: 2014-08-13 16:24:27.946876, event: done, request: pg_query(24.2c epoch 194453) v3 -2542> 2014-08-13 16:24:27.947012 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(26) epoch 194453) v5 -- ?+0 0x834cd00 con 0x4fe1600 -2541> 2014-08-13 16:24:27.947796 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.395 192.168.1.105:6810/41287 12 ==== pg_info(1 pgs e194453:24.39a) v4 ==== 744+0+0 (3999645444 0 0) 0x834db00 con 0x51127e0 -2540> 2014-08-13 16:24:27.947819 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947697, event: header_read, request: pg_info(1 pgs e194453:24.39a) v4 -2539> 2014-08-13 16:24:27.947830 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947699, event: throttled, request: pg_info(1 pgs e194453:24.39a) v4 -2538> 2014-08-13 16:24:27.947839 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947775, event: all_read, request: pg_info(1 pgs e194453:24.39a) v4 -2537> 2014-08-13 16:24:27.947845 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947817, event: dispatched, request: pg_info(1 pgs e194453:24.39a) v4 -2536> 2014-08-13 16:24:27.947851 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947851, event: waiting_for_osdmap, request: pg_info(1 pgs e194453:24.39a) v4 -2535> 2014-08-13 16:24:27.947862 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947862, event: started, request: pg_info(1 pgs e194453:24.39a) v4 -2534> 2014-08-13 16:24:27.947892 7fd93dff0700 5 -- op tracker -- , seq: 1058, time: 2014-08-13 16:24:27.947891, event: done, request: pg_info(1 pgs e194453:24.39a) v4 -2533> 2014-08-13 16:24:27.949425 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 25 ==== pg_info(1 pgs e194453:24.39a) v4 ==== 744+0+0 (3180650381 0 0) 0x834e740 con 0x7313860 -2532> 2014-08-13 16:24:27.949448 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949323, event: header_read, request: pg_info(1 pgs e194453:24.39a) v4 -2531> 2014-08-13 16:24:27.949455 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949325, event: throttled, request: pg_info(1 pgs e194453:24.39a) v4 -2530> 2014-08-13 16:24:27.949461 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949405, event: all_read, request: pg_info(1 pgs e194453:24.39a) v4 -2529> 2014-08-13 16:24:27.949466 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949445, event: dispatched, request: pg_info(1 pgs e194453:24.39a) v4 -2528> 2014-08-13 16:24:27.949476 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949476, event: waiting_for_osdmap, request: pg_info(1 pgs e194453:24.39a) v4 -2527> 2014-08-13 16:24:27.949487 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949487, event: started, request: pg_info(1 pgs e194453:24.39a) v4 -2526> 2014-08-13 16:24:27.949519 7fd93dff0700 5 -- op tracker -- , seq: 1059, time: 2014-08-13 16:24:27.949519, event: done, request: pg_info(1 pgs e194453:24.39a) v4 -2525> 2014-08-13 16:24:27.952523 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859e540 con 0x7316f60 -2524> 2014-08-13 16:24:27.952573 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2523> 2014-08-13 16:24:27.952587 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 13 ==== pg_notify(22.22c(4) epoch 194453) v5 ==== 942+0+0 (1020201525 0 0) 0x842f000 con 0x7316f60 -2522> 2014-08-13 16:24:27.952607 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952466, event: header_read, request: pg_notify(22.22c(4) epoch 194453) v5 -2521> 2014-08-13 16:24:27.952618 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952467, event: throttled, request: pg_notify(22.22c(4) epoch 194453) v5 -2520> 2014-08-13 16:24:27.952624 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952542, event: all_read, request: pg_notify(22.22c(4) epoch 194453) v5 -2519> 2014-08-13 16:24:27.952634 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952604, event: dispatched, request: pg_notify(22.22c(4) epoch 194453) v5 -2518> 2014-08-13 16:24:27.952646 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952645, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194453) v5 -2517> 2014-08-13 16:24:27.952656 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952655, event: started, request: pg_notify(22.22c(4) epoch 194453) v5 -2516> 2014-08-13 16:24:27.952690 7fd93dff0700 5 -- op tracker -- , seq: 1060, time: 2014-08-13 16:24:27.952690, event: done, request: pg_notify(22.22c(4) epoch 194453) v5 -2515> 2014-08-13 16:24:27.954340 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 8 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d03f00 con 0x4fd9b80 -2514> 2014-08-13 16:24:27.954380 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2513> 2014-08-13 16:24:27.954487 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 9 ==== pg_notify(24.39a(42) epoch 194453) v5 ==== 3042+0+0 (2675857226 0 0) 0x8c8aa00 con 0x4fd9b80 -2512> 2014-08-13 16:24:27.954511 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954342, event: header_read, request: pg_notify(24.39a(42) epoch 194453) v5 -2511> 2014-08-13 16:24:27.954526 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954343, event: throttled, request: pg_notify(24.39a(42) epoch 194453) v5 -2510> 2014-08-13 16:24:27.954536 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954465, event: all_read, request: pg_notify(24.39a(42) epoch 194453) v5 -2509> 2014-08-13 16:24:27.954553 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954508, event: dispatched, request: pg_notify(24.39a(42) epoch 194453) v5 -2508> 2014-08-13 16:24:27.954564 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954563, event: waiting_for_osdmap, request: pg_notify(24.39a(42) epoch 194453) v5 -2507> 2014-08-13 16:24:27.954576 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954576, event: started, request: pg_notify(24.39a(42) epoch 194453) v5 -2506> 2014-08-13 16:24:27.954611 7fd93dff0700 5 -- op tracker -- , seq: 1061, time: 2014-08-13 16:24:27.954611, event: done, request: pg_notify(24.39a(42) epoch 194453) v5 -2505> 2014-08-13 16:24:27.954773 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7bcc0 con 0x8e29fa0 -2504> 2014-08-13 16:24:27.954812 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2503> 2014-08-13 16:24:27.954922 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 13 ==== pg_notify(22.775(17) epoch 194453) v5 ==== 1665+0+0 (1240341939 0 0) 0x842f540 con 0x8e29fa0 -2502> 2014-08-13 16:24:27.954956 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.954775, event: header_read, request: pg_notify(22.775(17) epoch 194453) v5 -2501> 2014-08-13 16:24:27.954975 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.954777, event: throttled, request: pg_notify(22.775(17) epoch 194453) v5 -2500> 2014-08-13 16:24:27.954987 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.954897, event: all_read, request: pg_notify(22.775(17) epoch 194453) v5 -2499> 2014-08-13 16:24:27.954996 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.954951, event: dispatched, request: pg_notify(22.775(17) epoch 194453) v5 -2498> 2014-08-13 16:24:27.955007 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.955007, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194453) v5 -2497> 2014-08-13 16:24:27.955020 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.955020, event: started, request: pg_notify(22.775(17) epoch 194453) v5 -2496> 2014-08-13 16:24:27.955062 7fd93dff0700 5 -- op tracker -- , seq: 1062, time: 2014-08-13 16:24:27.955061, event: done, request: pg_notify(22.775(17) epoch 194453) v5 -2495> 2014-08-13 16:24:27.956120 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 16 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7cec0 con 0x7314fc0 -2494> 2014-08-13 16:24:27.956156 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2493> 2014-08-13 16:24:27.956254 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 17 ==== pg_notify(22.113(4) epoch 194453) v5 ==== 942+0+0 (1481223523 0 0) 0x899b2c0 con 0x7314fc0 -2492> 2014-08-13 16:24:27.956275 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956154, event: header_read, request: pg_notify(22.113(4) epoch 194453) v5 -2491> 2014-08-13 16:24:27.956284 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956155, event: throttled, request: pg_notify(22.113(4) epoch 194453) v5 -2490> 2014-08-13 16:24:27.956290 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956237, event: all_read, request: pg_notify(22.113(4) epoch 194453) v5 -2489> 2014-08-13 16:24:27.956295 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956272, event: dispatched, request: pg_notify(22.113(4) epoch 194453) v5 -2488> 2014-08-13 16:24:27.956302 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956302, event: waiting_for_osdmap, request: pg_notify(22.113(4) epoch 194453) v5 -2487> 2014-08-13 16:24:27.956311 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956311, event: started, request: pg_notify(22.113(4) epoch 194453) v5 -2486> 2014-08-13 16:24:27.956341 7fd93dff0700 5 -- op tracker -- , seq: 1063, time: 2014-08-13 16:24:27.956341, event: done, request: pg_notify(22.113(4) epoch 194453) v5 -2485> 2014-08-13 16:24:27.956363 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 35 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7f080 con 0x508bde0 -2484> 2014-08-13 16:24:27.956383 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2483> 2014-08-13 16:24:27.956430 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 36 ==== pg_notify(22.99(15) epoch 194453) v5 ==== 1551+0+0 (479163693 0 0) 0x899e3c0 con 0x508bde0 -2482> 2014-08-13 16:24:27.956451 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956290, event: header_read, request: pg_notify(22.99(15) epoch 194453) v5 -2481> 2014-08-13 16:24:27.956461 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956291, event: throttled, request: pg_notify(22.99(15) epoch 194453) v5 -2480> 2014-08-13 16:24:27.956471 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956411, event: all_read, request: pg_notify(22.99(15) epoch 194453) v5 -2479> 2014-08-13 16:24:27.956481 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956448, event: dispatched, request: pg_notify(22.99(15) epoch 194453) v5 -2478> 2014-08-13 16:24:27.956492 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956492, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194453) v5 -2477> 2014-08-13 16:24:27.956505 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956505, event: started, request: pg_notify(22.99(15) epoch 194453) v5 -2476> 2014-08-13 16:24:27.956534 7fd93dff0700 5 -- op tracker -- , seq: 1064, time: 2014-08-13 16:24:27.956534, event: done, request: pg_notify(22.99(15) epoch 194453) v5 -2475> 2014-08-13 16:24:27.956569 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 18 ==== pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 ==== 3436+0+0 (1878587304 0 0) 0x89981c0 con 0x7314fc0 -2474> 2014-08-13 16:24:27.956590 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956356, event: header_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2473> 2014-08-13 16:24:27.956601 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956357, event: throttled, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2472> 2014-08-13 16:24:27.956611 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956476, event: all_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2471> 2014-08-13 16:24:27.956621 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956588, event: dispatched, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2470> 2014-08-13 16:24:27.956631 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956631, event: waiting_for_osdmap, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2469> 2014-08-13 16:24:27.956643 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956643, event: started, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2468> 2014-08-13 16:24:27.956685 7fd93dff0700 5 -- op tracker -- , seq: 1065, time: 2014-08-13 16:24:27.956685, event: done, request: pg_notify(22.3e4(17),22.707(19) epoch 194453) v5 -2467> 2014-08-13 16:24:27.956712 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 9 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d06300 con 0x80c7e80 -2466> 2014-08-13 16:24:27.956733 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2465> 2014-08-13 16:24:27.956870 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 10 ==== pg_notify(22.3d9(39) epoch 194453) v5 ==== 2815+0+0 (2728880455 0 0) 0x81c6740 con 0x80c7e80 -2464> 2014-08-13 16:24:27.956891 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956713, event: header_read, request: pg_notify(22.3d9(39) epoch 194453) v5 -2463> 2014-08-13 16:24:27.956901 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956715, event: throttled, request: pg_notify(22.3d9(39) epoch 194453) v5 -2462> 2014-08-13 16:24:27.956910 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956850, event: all_read, request: pg_notify(22.3d9(39) epoch 194453) v5 -2461> 2014-08-13 16:24:27.956919 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956888, event: dispatched, request: pg_notify(22.3d9(39) epoch 194453) v5 -2460> 2014-08-13 16:24:27.956929 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956929, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194453) v5 -2459> 2014-08-13 16:24:27.956941 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956940, event: started, request: pg_notify(22.3d9(39) epoch 194453) v5 -2458> 2014-08-13 16:24:27.956966 7fd93dff0700 5 -- op tracker -- , seq: 1066, time: 2014-08-13 16:24:27.956966, event: done, request: pg_notify(22.3d9(39) epoch 194453) v5 -2457> 2014-08-13 16:24:27.957488 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.343 192.168.1.104:6802/61960 20 ==== MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 ==== 26+0+0 (257019632 0 0) 0x8c89a40 con 0x508dd80 -2456> 2014-08-13 16:24:27.957518 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957390, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2455> 2014-08-13 16:24:27.957530 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957392, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2454> 2014-08-13 16:24:27.957538 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957450, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2453> 2014-08-13 16:24:27.957547 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957514, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2452> 2014-08-13 16:24:27.957557 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957556, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2451> 2014-08-13 16:24:27.957581 7fd93dff0700 5 -- op tracker -- , seq: 1067, time: 2014-08-13 16:24:27.957581, event: done, request: MRecoveryReserve REQUEST pgid: 24.206, query_epoch: 194453 v2 -2450> 2014-08-13 16:24:27.957599 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194447 pi=193262-194446/19 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] exit Started/ReplicaActive/RepNotRecovering 23.504308 19 0.002665 -2449> 2014-08-13 16:24:27.957646 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194447 pi=193262-194446/19 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] enter Started/ReplicaActive/RepWaitRecoveryReserved -2448> 2014-08-13 16:24:27.957859 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8598000 con 0x5113de0 -2447> 2014-08-13 16:24:27.957884 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2446> 2014-08-13 16:24:27.958000 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 13 ==== pg_notify(24.1bb(21) epoch 194453) v5 ==== 1931+0+0 (1319651726 0 0) 0x899d940 con 0x5113de0 -2445> 2014-08-13 16:24:27.958020 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.957865, event: header_read, request: pg_notify(24.1bb(21) epoch 194453) v5 -2444> 2014-08-13 16:24:27.958031 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.957866, event: throttled, request: pg_notify(24.1bb(21) epoch 194453) v5 -2443> 2014-08-13 16:24:27.958040 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.957980, event: all_read, request: pg_notify(24.1bb(21) epoch 194453) v5 -2442> 2014-08-13 16:24:27.958049 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.958018, event: dispatched, request: pg_notify(24.1bb(21) epoch 194453) v5 -2441> 2014-08-13 16:24:27.958058 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.958058, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194453) v5 -2440> 2014-08-13 16:24:27.958070 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.958070, event: started, request: pg_notify(24.1bb(21) epoch 194453) v5 -2439> 2014-08-13 16:24:27.958099 7fd93dff0700 5 -- op tracker -- , seq: 1068, time: 2014-08-13 16:24:27.958098, event: done, request: pg_notify(24.1bb(21) epoch 194453) v5 -2438> 2014-08-13 16:24:27.962611 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 23 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859dc40 con 0x4fda940 -2437> 2014-08-13 16:24:27.962660 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2436> 2014-08-13 16:24:27.962675 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 24 ==== pg_notify(22.3e4(17) epoch 194453) v5 ==== 1665+0+0 (3959476262 0 0) 0x834a840 con 0x4fda940 -2435> 2014-08-13 16:24:27.962696 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962559, event: header_read, request: pg_notify(22.3e4(17) epoch 194453) v5 -2434> 2014-08-13 16:24:27.962711 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962560, event: throttled, request: pg_notify(22.3e4(17) epoch 194453) v5 -2433> 2014-08-13 16:24:27.962721 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962649, event: all_read, request: pg_notify(22.3e4(17) epoch 194453) v5 -2432> 2014-08-13 16:24:27.962730 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962693, event: dispatched, request: pg_notify(22.3e4(17) epoch 194453) v5 -2431> 2014-08-13 16:24:27.962741 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962740, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194453) v5 -2430> 2014-08-13 16:24:27.962753 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962753, event: started, request: pg_notify(22.3e4(17) epoch 194453) v5 -2429> 2014-08-13 16:24:27.962787 7fd93dff0700 5 -- op tracker -- , seq: 1069, time: 2014-08-13 16:24:27.962786, event: done, request: pg_notify(22.3e4(17) epoch 194453) v5 -2428> 2014-08-13 16:24:27.963692 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 23 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7b10b40 con 0x4fe40a0 -2427> 2014-08-13 16:24:27.963724 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2426> 2014-08-13 16:24:27.963807 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 24 ==== pg_notify(22.3af(4) epoch 194453) v5 ==== 942+0+0 (1046410445 0 0) 0x840fc40 con 0x4fe40a0 -2425> 2014-08-13 16:24:27.963853 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963674, event: header_read, request: pg_notify(22.3af(4) epoch 194453) v5 -2424> 2014-08-13 16:24:27.963866 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963675, event: throttled, request: pg_notify(22.3af(4) epoch 194453) v5 -2423> 2014-08-13 16:24:27.963872 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963782, event: all_read, request: pg_notify(22.3af(4) epoch 194453) v5 -2422> 2014-08-13 16:24:27.963883 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963847, event: dispatched, request: pg_notify(22.3af(4) epoch 194453) v5 -2421> 2014-08-13 16:24:27.963895 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963895, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194453) v5 -2420> 2014-08-13 16:24:27.963907 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963907, event: started, request: pg_notify(22.3af(4) epoch 194453) v5 -2419> 2014-08-13 16:24:27.963944 7fd93dff0700 5 -- op tracker -- , seq: 1070, time: 2014-08-13 16:24:27.963944, event: done, request: pg_notify(22.3af(4) epoch 194453) v5 -2418> 2014-08-13 16:24:27.965080 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 22 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e78900 con 0x5114360 -2417> 2014-08-13 16:24:27.965127 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2416> 2014-08-13 16:24:27.965193 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 23 ==== pg_notify(22.39d(15) epoch 194453) v5 ==== 1551+0+0 (3166724098 0 0) 0x840ad80 con 0x5114360 -2415> 2014-08-13 16:24:27.965220 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965041, event: header_read, request: pg_notify(22.39d(15) epoch 194453) v5 -2414> 2014-08-13 16:24:27.965232 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965043, event: throttled, request: pg_notify(22.39d(15) epoch 194453) v5 -2413> 2014-08-13 16:24:27.965238 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965134, event: all_read, request: pg_notify(22.39d(15) epoch 194453) v5 -2412> 2014-08-13 16:24:27.965250 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965217, event: dispatched, request: pg_notify(22.39d(15) epoch 194453) v5 -2411> 2014-08-13 16:24:27.965257 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965257, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194453) v5 -2410> 2014-08-13 16:24:27.965267 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965267, event: started, request: pg_notify(22.39d(15) epoch 194453) v5 -2409> 2014-08-13 16:24:27.965298 7fd93dff0700 5 -- op tracker -- , seq: 1071, time: 2014-08-13 16:24:27.965298, event: done, request: pg_notify(22.39d(15) epoch 194453) v5 -2408> 2014-08-13 16:24:27.965334 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 15 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d09200 con 0x5114620 -2407> 2014-08-13 16:24:27.965353 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2406> 2014-08-13 16:24:27.965459 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 16 ==== pg_notify(22.1d2(4) epoch 194453) v5 ==== 942+0+0 (3254159755 0 0) 0x91178c0 con 0x5114620 -2405> 2014-08-13 16:24:27.965495 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965309, event: header_read, request: pg_notify(22.1d2(4) epoch 194453) v5 -2404> 2014-08-13 16:24:27.965507 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965310, event: throttled, request: pg_notify(22.1d2(4) epoch 194453) v5 -2403> 2014-08-13 16:24:27.965513 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965382, event: all_read, request: pg_notify(22.1d2(4) epoch 194453) v5 -2402> 2014-08-13 16:24:27.965528 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965489, event: dispatched, request: pg_notify(22.1d2(4) epoch 194453) v5 -2401> 2014-08-13 16:24:27.965540 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965540, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194453) v5 -2400> 2014-08-13 16:24:27.965553 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965553, event: started, request: pg_notify(22.1d2(4) epoch 194453) v5 -2399> 2014-08-13 16:24:27.965584 7fd93dff0700 5 -- op tracker -- , seq: 1072, time: 2014-08-13 16:24:27.965584, event: done, request: pg_notify(22.1d2(4) epoch 194453) v5 -2398> 2014-08-13 16:24:27.965636 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 25 ==== pg_notify(22.362(4) epoch 194453) v5 ==== 942+0+0 (3418241097 0 0) 0x834fc40 con 0x4fe40a0 -2397> 2014-08-13 16:24:27.965658 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965423, event: header_read, request: pg_notify(22.362(4) epoch 194453) v5 -2396> 2014-08-13 16:24:27.965669 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965426, event: throttled, request: pg_notify(22.362(4) epoch 194453) v5 -2395> 2014-08-13 16:24:27.965675 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965550, event: all_read, request: pg_notify(22.362(4) epoch 194453) v5 -2394> 2014-08-13 16:24:27.965683 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965655, event: dispatched, request: pg_notify(22.362(4) epoch 194453) v5 -2393> 2014-08-13 16:24:27.965694 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965694, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194453) v5 -2392> 2014-08-13 16:24:27.965706 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965706, event: started, request: pg_notify(22.362(4) epoch 194453) v5 -2391> 2014-08-13 16:24:27.965744 7fd93dff0700 5 -- op tracker -- , seq: 1073, time: 2014-08-13 16:24:27.965744, event: done, request: pg_notify(22.362(4) epoch 194453) v5 -2390> 2014-08-13 16:24:27.966872 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7cf2ac0 con 0x72dfa60 -2389> 2014-08-13 16:24:27.966919 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2388> 2014-08-13 16:24:27.967025 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 13 ==== pg_notify(22.4df(4) epoch 194453) v5 ==== 942+0+0 (600479036 0 0) 0x8408c40 con 0x72dfa60 -2387> 2014-08-13 16:24:27.967055 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.966857, event: header_read, request: pg_notify(22.4df(4) epoch 194453) v5 -2386> 2014-08-13 16:24:27.967065 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.966858, event: throttled, request: pg_notify(22.4df(4) epoch 194453) v5 -2385> 2014-08-13 16:24:27.967071 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.966997, event: all_read, request: pg_notify(22.4df(4) epoch 194453) v5 -2384> 2014-08-13 16:24:27.967077 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.967050, event: dispatched, request: pg_notify(22.4df(4) epoch 194453) v5 -2383> 2014-08-13 16:24:27.967084 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.967084, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194453) v5 -2382> 2014-08-13 16:24:27.967093 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.967092, event: started, request: pg_notify(22.4df(4) epoch 194453) v5 -2381> 2014-08-13 16:24:27.967129 7fd93dff0700 5 -- op tracker -- , seq: 1074, time: 2014-08-13 16:24:27.967128, event: done, request: pg_notify(22.4df(4) epoch 194453) v5 -2380> 2014-08-13 16:24:27.967151 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 19 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7b12d00 con 0x4e4b860 -2379> 2014-08-13 16:24:27.967175 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2378> 2014-08-13 16:24:27.967243 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 20 ==== pg_notify(24.30b(4) epoch 194453) v5 ==== 976+0+0 (2179333072 0 0) 0x899a4c0 con 0x4e4b860 -2377> 2014-08-13 16:24:27.967285 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967126, event: header_read, request: pg_notify(24.30b(4) epoch 194453) v5 -2376> 2014-08-13 16:24:27.967298 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967127, event: throttled, request: pg_notify(24.30b(4) epoch 194453) v5 -2375> 2014-08-13 16:24:27.967304 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967219, event: all_read, request: pg_notify(24.30b(4) epoch 194453) v5 -2374> 2014-08-13 16:24:27.967310 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967279, event: dispatched, request: pg_notify(24.30b(4) epoch 194453) v5 -2373> 2014-08-13 16:24:27.967317 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967317, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194453) v5 -2372> 2014-08-13 16:24:27.967331 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967331, event: started, request: pg_notify(24.30b(4) epoch 194453) v5 -2371> 2014-08-13 16:24:27.967368 7fd93dff0700 5 -- op tracker -- , seq: 1075, time: 2014-08-13 16:24:27.967368, event: done, request: pg_notify(24.30b(4) epoch 194453) v5 -2370> 2014-08-13 16:24:27.967398 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 16 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x506e300 con 0x7316720 -2369> 2014-08-13 16:24:27.967423 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2368> 2014-08-13 16:24:27.967437 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 17 ==== pg_notify(22.1f1(17) epoch 194453) v5 ==== 1665+0+0 (291167416 0 0) 0x775a840 con 0x7316720 -2367> 2014-08-13 16:24:27.967455 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967288, event: header_read, request: pg_notify(22.1f1(17) epoch 194453) v5 -2366> 2014-08-13 16:24:27.967466 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967289, event: throttled, request: pg_notify(22.1f1(17) epoch 194453) v5 -2365> 2014-08-13 16:24:27.967476 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967392, event: all_read, request: pg_notify(22.1f1(17) epoch 194453) v5 -2364> 2014-08-13 16:24:27.967485 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967453, event: dispatched, request: pg_notify(22.1f1(17) epoch 194453) v5 -2363> 2014-08-13 16:24:27.967494 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967494, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194453) v5 -2362> 2014-08-13 16:24:27.967506 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967506, event: started, request: pg_notify(22.1f1(17) epoch 194453) v5 -2361> 2014-08-13 16:24:27.967542 7fd93dff0700 5 -- op tracker -- , seq: 1076, time: 2014-08-13 16:24:27.967542, event: done, request: pg_notify(22.1f1(17) epoch 194453) v5 -2360> 2014-08-13 16:24:27.967618 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 18 ==== pg_notify(22.22c(17) epoch 194453) v5 ==== 1665+0+0 (374890458 0 0) 0x775e740 con 0x7316720 -2359> 2014-08-13 16:24:27.967644 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967494, event: header_read, request: pg_notify(22.22c(17) epoch 194453) v5 -2358> 2014-08-13 16:24:27.967655 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967496, event: throttled, request: pg_notify(22.22c(17) epoch 194453) v5 -2357> 2014-08-13 16:24:27.967665 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967597, event: all_read, request: pg_notify(22.22c(17) epoch 194453) v5 -2356> 2014-08-13 16:24:27.967674 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967641, event: dispatched, request: pg_notify(22.22c(17) epoch 194453) v5 -2355> 2014-08-13 16:24:27.967683 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967683, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194453) v5 -2354> 2014-08-13 16:24:27.967695 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967695, event: started, request: pg_notify(22.22c(17) epoch 194453) v5 -2353> 2014-08-13 16:24:27.967722 7fd93dff0700 5 -- op tracker -- , seq: 1077, time: 2014-08-13 16:24:27.967722, event: done, request: pg_notify(22.22c(17) epoch 194453) v5 -2352> 2014-08-13 16:24:27.968181 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 21 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7b600 con 0x8e286e0 -2351> 2014-08-13 16:24:27.968218 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2350> 2014-08-13 16:24:27.968303 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 22 ==== pg_notify(22.3af(17) epoch 194453) v5 ==== 1665+0+0 (3975362134 0 0) 0x834d240 con 0x8e286e0 -2349> 2014-08-13 16:24:27.968329 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968180, event: header_read, request: pg_notify(22.3af(17) epoch 194453) v5 -2348> 2014-08-13 16:24:27.968344 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968181, event: throttled, request: pg_notify(22.3af(17) epoch 194453) v5 -2347> 2014-08-13 16:24:27.968354 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968270, event: all_read, request: pg_notify(22.3af(17) epoch 194453) v5 -2346> 2014-08-13 16:24:27.968364 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968325, event: dispatched, request: pg_notify(22.3af(17) epoch 194453) v5 -2345> 2014-08-13 16:24:27.968375 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968374, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194453) v5 -2344> 2014-08-13 16:24:27.968388 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968387, event: started, request: pg_notify(22.3af(17) epoch 194453) v5 -2343> 2014-08-13 16:24:27.968421 7fd93dff0700 5 -- op tracker -- , seq: 1078, time: 2014-08-13 16:24:27.968421, event: done, request: pg_notify(22.3af(17) epoch 194453) v5 -2342> 2014-08-13 16:24:27.968625 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d086c0 con 0x7379ce0 -2341> 2014-08-13 16:24:27.968659 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2340> 2014-08-13 16:24:27.968674 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 18 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7cf4c80 con 0x5113860 -2339> 2014-08-13 16:24:27.968692 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2338> 2014-08-13 16:24:27.968701 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 13 ==== pg_notify(24.181(25) epoch 194453) v5 ==== 2159+0+0 (4134170687 0 0) 0x834b800 con 0x7379ce0 -2337> 2014-08-13 16:24:27.968720 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968332, event: header_read, request: pg_notify(24.181(25) epoch 194453) v5 -2336> 2014-08-13 16:24:27.968732 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968333, event: throttled, request: pg_notify(24.181(25) epoch 194453) v5 -2335> 2014-08-13 16:24:27.968742 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968435, event: all_read, request: pg_notify(24.181(25) epoch 194453) v5 -2334> 2014-08-13 16:24:27.968748 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968717, event: dispatched, request: pg_notify(24.181(25) epoch 194453) v5 -2333> 2014-08-13 16:24:27.968758 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968758, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194453) v5 -2332> 2014-08-13 16:24:27.968776 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968776, event: started, request: pg_notify(24.181(25) epoch 194453) v5 -2331> 2014-08-13 16:24:27.968812 7fd93dff0700 5 -- op tracker -- , seq: 1079, time: 2014-08-13 16:24:27.968812, event: done, request: pg_notify(24.181(25) epoch 194453) v5 -2330> 2014-08-13 16:24:27.968853 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 19 ==== pg_notify(22.48e(15) epoch 194453) v5 ==== 1551+0+0 (541980268 0 0) 0x840c600 con 0x5113860 -2329> 2014-08-13 16:24:27.968874 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968567, event: header_read, request: pg_notify(22.48e(15) epoch 194453) v5 -2328> 2014-08-13 16:24:27.968885 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968568, event: throttled, request: pg_notify(22.48e(15) epoch 194453) v5 -2327> 2014-08-13 16:24:27.968894 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968682, event: all_read, request: pg_notify(22.48e(15) epoch 194453) v5 -2326> 2014-08-13 16:24:27.968903 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968872, event: dispatched, request: pg_notify(22.48e(15) epoch 194453) v5 -2325> 2014-08-13 16:24:27.968913 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968913, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194453) v5 -2324> 2014-08-13 16:24:27.968924 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968924, event: started, request: pg_notify(22.48e(15) epoch 194453) v5 -2323> 2014-08-13 16:24:27.968951 7fd93dff0700 5 -- op tracker -- , seq: 1080, time: 2014-08-13 16:24:27.968951, event: done, request: pg_notify(22.48e(15) epoch 194453) v5 -2322> 2014-08-13 16:24:27.969165 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 11 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859cc80 con 0x821f900 -2321> 2014-08-13 16:24:27.969199 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2320> 2014-08-13 16:24:27.969271 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 12 ==== pg_notify(22.775(17) epoch 194453) v5 ==== 1665+0+0 (1950174103 0 0) 0x80dec80 con 0x821f900 -2319> 2014-08-13 16:24:27.969291 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969165, event: header_read, request: pg_notify(22.775(17) epoch 194453) v5 -2318> 2014-08-13 16:24:27.969300 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969167, event: throttled, request: pg_notify(22.775(17) epoch 194453) v5 -2317> 2014-08-13 16:24:27.969305 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969251, event: all_read, request: pg_notify(22.775(17) epoch 194453) v5 -2316> 2014-08-13 16:24:27.969310 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969289, event: dispatched, request: pg_notify(22.775(17) epoch 194453) v5 -2315> 2014-08-13 16:24:27.969317 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969316, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194453) v5 -2314> 2014-08-13 16:24:27.969325 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969324, event: started, request: pg_notify(22.775(17) epoch 194453) v5 -2313> 2014-08-13 16:24:27.969353 7fd93dff0700 5 -- op tracker -- , seq: 1081, time: 2014-08-13 16:24:27.969353, event: done, request: pg_notify(22.775(17) epoch 194453) v5 -2312> 2014-08-13 16:24:27.969376 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 23 ==== pg_notify(22.775(4) epoch 194453) v5 ==== 942+0+0 (1451622814 0 0) 0x8348e00 con 0x8e286e0 -2311> 2014-08-13 16:24:27.969392 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969234, event: header_read, request: pg_notify(22.775(4) epoch 194453) v5 -2310> 2014-08-13 16:24:27.969399 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969237, event: throttled, request: pg_notify(22.775(4) epoch 194453) v5 -2309> 2014-08-13 16:24:27.969405 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969339, event: all_read, request: pg_notify(22.775(4) epoch 194453) v5 -2308> 2014-08-13 16:24:27.969410 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969390, event: dispatched, request: pg_notify(22.775(4) epoch 194453) v5 -2307> 2014-08-13 16:24:27.969416 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969416, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194453) v5 -2306> 2014-08-13 16:24:27.969424 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969424, event: started, request: pg_notify(22.775(4) epoch 194453) v5 -2305> 2014-08-13 16:24:27.969450 7fd93dff0700 5 -- op tracker -- , seq: 1082, time: 2014-08-13 16:24:27.969450, event: done, request: pg_notify(22.775(4) epoch 194453) v5 -2304> 2014-08-13 16:24:27.970441 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 19 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8bd1b00 con 0x50ba520 -2303> 2014-08-13 16:24:27.970465 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2302> 2014-08-13 16:24:27.970604 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 20 ==== pg_notify(22.707(19) epoch 194453) v5 ==== 1783+0+0 (732001385 0 0) 0x87e5e80 con 0x50ba520 -2301> 2014-08-13 16:24:27.970635 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970461, event: header_read, request: pg_notify(22.707(19) epoch 194453) v5 -2300> 2014-08-13 16:24:27.970647 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970462, event: throttled, request: pg_notify(22.707(19) epoch 194453) v5 -2299> 2014-08-13 16:24:27.970657 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970581, event: all_read, request: pg_notify(22.707(19) epoch 194453) v5 -2298> 2014-08-13 16:24:27.970665 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970631, event: dispatched, request: pg_notify(22.707(19) epoch 194453) v5 -2297> 2014-08-13 16:24:27.970675 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970675, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194453) v5 -2296> 2014-08-13 16:24:27.970687 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970687, event: started, request: pg_notify(22.707(19) epoch 194453) v5 -2295> 2014-08-13 16:24:27.970717 7fd93dff0700 5 -- op tracker -- , seq: 1083, time: 2014-08-13 16:24:27.970716, event: done, request: pg_notify(22.707(19) epoch 194453) v5 -2294> 2014-08-13 16:24:27.970865 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 24 ==== pg_notify(22.f3(15) epoch 194453) v5 ==== 1551+0+0 (1488259717 0 0) 0x8349880 con 0x8e286e0 -2293> 2014-08-13 16:24:27.970888 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970752, event: header_read, request: pg_notify(22.f3(15) epoch 194453) v5 -2292> 2014-08-13 16:24:27.970899 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970754, event: throttled, request: pg_notify(22.f3(15) epoch 194453) v5 -2291> 2014-08-13 16:24:27.970908 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970845, event: all_read, request: pg_notify(22.f3(15) epoch 194453) v5 -2290> 2014-08-13 16:24:27.970917 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970885, event: dispatched, request: pg_notify(22.f3(15) epoch 194453) v5 -2289> 2014-08-13 16:24:27.970927 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970926, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194453) v5 -2288> 2014-08-13 16:24:27.970938 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970938, event: started, request: pg_notify(22.f3(15) epoch 194453) v5 -2287> 2014-08-13 16:24:27.970966 7fd93dff0700 5 -- op tracker -- , seq: 1084, time: 2014-08-13 16:24:27.970965, event: done, request: pg_notify(22.f3(15) epoch 194453) v5 -2286> 2014-08-13 16:24:27.971086 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 24 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d07740 con 0x4fe69e0 -2285> 2014-08-13 16:24:27.971111 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2284> 2014-08-13 16:24:27.971476 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 25 ==== pg_notify(24.34(43) epoch 194453) v5 ==== 2933+0+0 (2833778363 0 0) 0x842f540 con 0x4fe69e0 -2283> 2014-08-13 16:24:27.971522 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971279, event: header_read, request: pg_notify(24.34(43) epoch 194453) v5 -2282> 2014-08-13 16:24:27.971537 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971282, event: throttled, request: pg_notify(24.34(43) epoch 194453) v5 -2281> 2014-08-13 16:24:27.971547 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971410, event: all_read, request: pg_notify(24.34(43) epoch 194453) v5 -2280> 2014-08-13 16:24:27.971557 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971516, event: dispatched, request: pg_notify(24.34(43) epoch 194453) v5 -2279> 2014-08-13 16:24:27.971567 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971567, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194453) v5 -2278> 2014-08-13 16:24:27.971580 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971580, event: started, request: pg_notify(24.34(43) epoch 194453) v5 -2277> 2014-08-13 16:24:27.971617 7fd93dff0700 5 -- op tracker -- , seq: 1085, time: 2014-08-13 16:24:27.971617, event: done, request: pg_notify(24.34(43) epoch 194453) v5 -2276> 2014-08-13 16:24:27.972178 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 20 ==== pg_notify(24.b6(4) epoch 194453) v5 ==== 976+0+0 (1408338637 0 0) 0x8409c00 con 0x5113860 -2275> 2014-08-13 16:24:27.972233 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972020, event: header_read, request: pg_notify(24.b6(4) epoch 194453) v5 -2274> 2014-08-13 16:24:27.972249 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972023, event: throttled, request: pg_notify(24.b6(4) epoch 194453) v5 -2273> 2014-08-13 16:24:27.972259 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972113, event: all_read, request: pg_notify(24.b6(4) epoch 194453) v5 -2272> 2014-08-13 16:24:27.972268 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972227, event: dispatched, request: pg_notify(24.b6(4) epoch 194453) v5 -2271> 2014-08-13 16:24:27.972278 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972278, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194453) v5 -2270> 2014-08-13 16:24:27.972291 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972291, event: started, request: pg_notify(24.b6(4) epoch 194453) v5 -2269> 2014-08-13 16:24:27.972325 7fd93dff0700 5 -- op tracker -- , seq: 1086, time: 2014-08-13 16:24:27.972325, event: done, request: pg_notify(24.b6(4) epoch 194453) v5 -2268> 2014-08-13 16:24:27.972886 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 29 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859a640 con 0x5111fa0 -2267> 2014-08-13 16:24:27.972934 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2266> 2014-08-13 16:24:27.973082 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 30 ==== pg_notify(22.3d9(6) epoch 194453) v5 ==== 1060+0+0 (3382733546 0 0) 0x899d940 con 0x5111fa0 -2265> 2014-08-13 16:24:27.973128 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.972897, event: header_read, request: pg_notify(22.3d9(6) epoch 194453) v5 -2264> 2014-08-13 16:24:27.973141 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.972899, event: throttled, request: pg_notify(22.3d9(6) epoch 194453) v5 -2263> 2014-08-13 16:24:27.973147 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.973006, event: all_read, request: pg_notify(22.3d9(6) epoch 194453) v5 -2262> 2014-08-13 16:24:27.973153 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.973123, event: dispatched, request: pg_notify(22.3d9(6) epoch 194453) v5 -2261> 2014-08-13 16:24:27.973160 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.973160, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194453) v5 -2260> 2014-08-13 16:24:27.973169 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.973169, event: started, request: pg_notify(22.3d9(6) epoch 194453) v5 -2259> 2014-08-13 16:24:27.973213 7fd93dff0700 5 -- op tracker -- , seq: 1087, time: 2014-08-13 16:24:27.973212, event: done, request: pg_notify(22.3d9(6) epoch 194453) v5 -2258> 2014-08-13 16:24:27.974235 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 27 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d0af40 con 0x4e4f900 -2257> 2014-08-13 16:24:27.974284 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2256> 2014-08-13 16:24:27.974403 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 28 ==== pg_notify(24.34(43) epoch 194453) v5 ==== 2933+0+0 (2367058643 0 0) 0x87e5080 con 0x4e4f900 -2255> 2014-08-13 16:24:27.974448 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974212, event: header_read, request: pg_notify(24.34(43) epoch 194453) v5 -2254> 2014-08-13 16:24:27.974464 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974214, event: throttled, request: pg_notify(24.34(43) epoch 194453) v5 -2253> 2014-08-13 16:24:27.974474 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974338, event: all_read, request: pg_notify(24.34(43) epoch 194453) v5 -2252> 2014-08-13 16:24:27.974483 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974442, event: dispatched, request: pg_notify(24.34(43) epoch 194453) v5 -2251> 2014-08-13 16:24:27.974494 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974494, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194453) v5 -2250> 2014-08-13 16:24:27.974507 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974507, event: started, request: pg_notify(24.34(43) epoch 194453) v5 -2249> 2014-08-13 16:24:27.974540 7fd93dff0700 5 -- op tracker -- , seq: 1088, time: 2014-08-13 16:24:27.974540, event: done, request: pg_notify(24.34(43) epoch 194453) v5 -2248> 2014-08-13 16:24:27.974572 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 9 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d069c0 con 0x73ccba0 -2247> 2014-08-13 16:24:27.974598 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2246> 2014-08-13 16:24:27.974802 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 23 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7ee40 con 0x4e4dd80 -2245> 2014-08-13 16:24:27.974849 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2244> 2014-08-13 16:24:27.974864 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 10 ==== pg_notify(22.481(32) epoch 194453) v5 ==== 2518+0+0 (4292103769 0 0) 0x87e6740 con 0x73ccba0 -2243> 2014-08-13 16:24:27.974891 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974582, event: header_read, request: pg_notify(22.481(32) epoch 194453) v5 -2242> 2014-08-13 16:24:27.974906 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974584, event: throttled, request: pg_notify(22.481(32) epoch 194453) v5 -2241> 2014-08-13 16:24:27.974916 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974720, event: all_read, request: pg_notify(22.481(32) epoch 194453) v5 -2240> 2014-08-13 16:24:27.974925 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974888, event: dispatched, request: pg_notify(22.481(32) epoch 194453) v5 -2239> 2014-08-13 16:24:27.974936 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974935, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194453) v5 -2238> 2014-08-13 16:24:27.974949 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974948, event: started, request: pg_notify(22.481(32) epoch 194453) v5 -2237> 2014-08-13 16:24:27.974980 7fd93dff0700 5 -- op tracker -- , seq: 1089, time: 2014-08-13 16:24:27.974980, event: done, request: pg_notify(22.481(32) epoch 194453) v5 -2236> 2014-08-13 16:24:27.975028 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 24 ==== pg_notify(22.1d2(17) epoch 194453) v5 ==== 1669+0+0 (49076276 0 0) 0x840c0c0 con 0x4e4dd80 -2235> 2014-08-13 16:24:27.975050 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.974731, event: header_read, request: pg_notify(22.1d2(17) epoch 194453) v5 -2234> 2014-08-13 16:24:27.975060 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.974732, event: throttled, request: pg_notify(22.1d2(17) epoch 194453) v5 -2233> 2014-08-13 16:24:27.975071 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.974826, event: all_read, request: pg_notify(22.1d2(17) epoch 194453) v5 -2232> 2014-08-13 16:24:27.975080 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.975047, event: dispatched, request: pg_notify(22.1d2(17) epoch 194453) v5 -2231> 2014-08-13 16:24:27.975089 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.975089, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194453) v5 -2230> 2014-08-13 16:24:27.975100 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.975100, event: started, request: pg_notify(22.1d2(17) epoch 194453) v5 -2229> 2014-08-13 16:24:27.975128 7fd93dff0700 5 -- op tracker -- , seq: 1090, time: 2014-08-13 16:24:27.975128, event: done, request: pg_notify(22.1d2(17) epoch 194453) v5 -2228> 2014-08-13 16:24:27.977262 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 26 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e798c0 con 0x50889a0 -2227> 2014-08-13 16:24:27.977308 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2226> 2014-08-13 16:24:27.977397 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 27 ==== pg_notify(22.113(17) epoch 194453) v5 ==== 1669+0+0 (1982841178 0 0) 0x87e3640 con 0x50889a0 -2225> 2014-08-13 16:24:27.977436 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977262, event: header_read, request: pg_notify(22.113(17) epoch 194453) v5 -2224> 2014-08-13 16:24:27.977452 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977263, event: throttled, request: pg_notify(22.113(17) epoch 194453) v5 -2223> 2014-08-13 16:24:27.977461 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977370, event: all_read, request: pg_notify(22.113(17) epoch 194453) v5 -2222> 2014-08-13 16:24:27.977472 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977431, event: dispatched, request: pg_notify(22.113(17) epoch 194453) v5 -2221> 2014-08-13 16:24:27.977480 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977480, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194453) v5 -2220> 2014-08-13 16:24:27.977493 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977493, event: started, request: pg_notify(22.113(17) epoch 194453) v5 -2219> 2014-08-13 16:24:27.977526 7fd93dff0700 5 -- op tracker -- , seq: 1091, time: 2014-08-13 16:24:27.977526, event: done, request: pg_notify(22.113(17) epoch 194453) v5 -2218> 2014-08-13 16:24:27.978355 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 28 ==== pg_notify(24.181(9) epoch 194453) v5 ==== 1263+0+0 (1187877334 0 0) 0x8998380 con 0x50889a0 -2217> 2014-08-13 16:24:27.978395 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978173, event: header_read, request: pg_notify(24.181(9) epoch 194453) v5 -2216> 2014-08-13 16:24:27.978411 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978175, event: throttled, request: pg_notify(24.181(9) epoch 194453) v5 -2215> 2014-08-13 16:24:27.978779 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978268, event: all_read, request: pg_notify(24.181(9) epoch 194453) v5 -2214> 2014-08-13 16:24:27.978807 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978390, event: dispatched, request: pg_notify(24.181(9) epoch 194453) v5 -2213> 2014-08-13 16:24:27.978815 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978815, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194453) v5 -2212> 2014-08-13 16:24:27.978828 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978828, event: started, request: pg_notify(24.181(9) epoch 194453) v5 -2211> 2014-08-13 16:24:27.978865 7fd93dff0700 5 -- op tracker -- , seq: 1092, time: 2014-08-13 16:24:27.978865, event: done, request: pg_notify(24.181(9) epoch 194453) v5 -2210> 2014-08-13 16:24:27.978888 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 1 ==== pg_notify(24.34(43) epoch 194453) v5 ==== 2933+0+0 (3191460802 0 0) 0x899fa80 con 0x83002c0 -2209> 2014-08-13 16:24:27.978908 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978145, event: header_read, request: pg_notify(24.34(43) epoch 194453) v5 -2208> 2014-08-13 16:24:27.978915 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978149, event: throttled, request: pg_notify(24.34(43) epoch 194453) v5 -2207> 2014-08-13 16:24:27.978921 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978320, event: all_read, request: pg_notify(24.34(43) epoch 194453) v5 -2206> 2014-08-13 16:24:27.978926 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978905, event: dispatched, request: pg_notify(24.34(43) epoch 194453) v5 -2205> 2014-08-13 16:24:27.978933 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978932, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194453) v5 -2204> 2014-08-13 16:24:27.978940 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978940, event: started, request: pg_notify(24.34(43) epoch 194453) v5 -2203> 2014-08-13 16:24:27.978964 7fd93dff0700 5 -- op tracker -- , seq: 1093, time: 2014-08-13 16:24:27.978964, event: done, request: pg_notify(24.34(43) epoch 194453) v5 -2202> 2014-08-13 16:24:27.978999 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 36 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7c5c0 con 0x51135a0 -2201> 2014-08-13 16:24:27.979020 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2200> 2014-08-13 16:24:27.979031 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 18 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d04a40 con 0x73cdc20 -2199> 2014-08-13 16:24:27.979003 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.045350 4 0.000201 -2198> 2014-08-13 16:24:27.979045 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2197> 2014-08-13 16:24:27.979052 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 37 ==== pg_notify(24.3d(18) epoch 194453) v5 ==== 1754+0+0 (3562157861 0 0) 0x775e3c0 con 0x51135a0 -2196> 2014-08-13 16:24:27.979053 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2195> 2014-08-13 16:24:27.979066 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.978458, event: header_read, request: pg_notify(24.3d(18) epoch 194453) v5 -2194> 2014-08-13 16:24:27.979073 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.978459, event: throttled, request: pg_notify(24.3d(18) epoch 194453) v5 -2193> 2014-08-13 16:24:27.979079 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.978635, event: all_read, request: pg_notify(24.3d(18) epoch 194453) v5 -2192> 2014-08-13 16:24:27.979084 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.979064, event: dispatched, request: pg_notify(24.3d(18) epoch 194453) v5 -2191> 2014-08-13 16:24:27.979091 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.979090, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194453) v5 -2190> 2014-08-13 16:24:27.979098 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.979098, event: started, request: pg_notify(24.3d(18) epoch 194453) v5 -2189> 2014-08-13 16:24:27.979123 7fd93dff0700 5 -- op tracker -- , seq: 1094, time: 2014-08-13 16:24:27.979123, event: done, request: pg_notify(24.3d(18) epoch 194453) v5 -2188> 2014-08-13 16:24:27.979140 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 19 ==== pg_notify(22.3af(17) epoch 194453) v5 ==== 1665+0+0 (3975362134 0 0) 0x899af40 con 0x73cdc20 -2187> 2014-08-13 16:24:27.979155 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.978716, event: header_read, request: pg_notify(22.3af(17) epoch 194453) v5 -2186> 2014-08-13 16:24:27.979156 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000103 0 0.000000 -2185> 2014-08-13 16:24:27.979173 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.978719, event: throttled, request: pg_notify(22.3af(17) epoch 194453) v5 -2184> 2014-08-13 16:24:27.979179 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.978829, event: all_read, request: pg_notify(22.3af(17) epoch 194453) v5 -2183> 2014-08-13 16:24:27.979185 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.979154, event: dispatched, request: pg_notify(22.3af(17) epoch 194453) v5 -2182> 2014-08-13 16:24:27.979191 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.979191, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194453) v5 -2181> 2014-08-13 16:24:27.979182 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2180> 2014-08-13 16:24:27.979199 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.979199, event: started, request: pg_notify(22.3af(17) epoch 194453) v5 -2179> 2014-08-13 16:24:27.979219 7fd93dff0700 5 -- op tracker -- , seq: 1095, time: 2014-08-13 16:24:27.979219, event: done, request: pg_notify(22.3af(17) epoch 194453) v5 -2178> 2014-08-13 16:24:27.979217 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000035 0 0.000000 -2177> 2014-08-13 16:24:27.979238 7fd9371e5700 5 osd.4 pg_epoch: 194453 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -2176> 2014-08-13 16:24:27.979694 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 17 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7cf1200 con 0x7723440 -2175> 2014-08-13 16:24:27.979733 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2174> 2014-08-13 16:24:27.979879 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 18 ==== pg_notify(24.9b(4) epoch 194453) v5 ==== 976+0+0 (3372887798 0 0) 0x775b480 con 0x7723440 -2173> 2014-08-13 16:24:27.979901 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979766, event: header_read, request: pg_notify(24.9b(4) epoch 194453) v5 -2172> 2014-08-13 16:24:27.979912 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979768, event: throttled, request: pg_notify(24.9b(4) epoch 194453) v5 -2171> 2014-08-13 16:24:27.979918 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979858, event: all_read, request: pg_notify(24.9b(4) epoch 194453) v5 -2170> 2014-08-13 16:24:27.979924 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979898, event: dispatched, request: pg_notify(24.9b(4) epoch 194453) v5 -2169> 2014-08-13 16:24:27.979932 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979932, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194453) v5 -2168> 2014-08-13 16:24:27.979944 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979944, event: started, request: pg_notify(24.9b(4) epoch 194453) v5 -2167> 2014-08-13 16:24:27.979978 7fd93dff0700 5 -- op tracker -- , seq: 1096, time: 2014-08-13 16:24:27.979978, event: done, request: pg_notify(24.9b(4) epoch 194453) v5 -2166> 2014-08-13 16:24:27.982258 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 38 ==== pg_notify(22.3d9(32) epoch 194453) v5 ==== 2458+0+0 (1812732590 0 0) 0x840a300 con 0x51135a0 -2165> 2014-08-13 16:24:27.982294 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982073, event: header_read, request: pg_notify(22.3d9(32) epoch 194453) v5 -2164> 2014-08-13 16:24:27.982304 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982076, event: throttled, request: pg_notify(22.3d9(32) epoch 194453) v5 -2163> 2014-08-13 16:24:27.982310 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982222, event: all_read, request: pg_notify(22.3d9(32) epoch 194453) v5 -2162> 2014-08-13 16:24:27.982316 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982289, event: dispatched, request: pg_notify(22.3d9(32) epoch 194453) v5 -2161> 2014-08-13 16:24:27.982323 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982322, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194453) v5 -2160> 2014-08-13 16:24:27.982331 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982331, event: started, request: pg_notify(22.3d9(32) epoch 194453) v5 -2159> 2014-08-13 16:24:27.982374 7fd93dff0700 5 -- op tracker -- , seq: 1097, time: 2014-08-13 16:24:27.982373, event: done, request: pg_notify(22.3d9(32) epoch 194453) v5 -2158> 2014-08-13 16:24:27.982495 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 16 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8bd1680 con 0x7315ee0 -2157> 2014-08-13 16:24:27.982534 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2156> 2014-08-13 16:24:27.982545 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 17 ==== pg_notify(24.39a(28) epoch 194453) v5 ==== 2300+0+0 (4063594072 0 0) 0x8c8b480 con 0x7315ee0 -2155> 2014-08-13 16:24:27.982561 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982325, event: header_read, request: pg_notify(24.39a(28) epoch 194453) v5 -2154> 2014-08-13 16:24:27.982569 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982326, event: throttled, request: pg_notify(24.39a(28) epoch 194453) v5 -2153> 2014-08-13 16:24:27.982575 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982427, event: all_read, request: pg_notify(24.39a(28) epoch 194453) v5 -2152> 2014-08-13 16:24:27.982580 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982559, event: dispatched, request: pg_notify(24.39a(28) epoch 194453) v5 -2151> 2014-08-13 16:24:27.982586 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982585, event: waiting_for_osdmap, request: pg_notify(24.39a(28) epoch 194453) v5 -2150> 2014-08-13 16:24:27.982593 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982593, event: started, request: pg_notify(24.39a(28) epoch 194453) v5 -2149> 2014-08-13 16:24:27.982640 7fd93dff0700 5 -- op tracker -- , seq: 1098, time: 2014-08-13 16:24:27.982640, event: done, request: pg_notify(24.39a(28) epoch 194453) v5 -2148> 2014-08-13 16:24:27.983839 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 24 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7a1c0 con 0x5108b00 -2147> 2014-08-13 16:24:27.983881 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2146> 2014-08-13 16:24:27.983970 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 25 ==== pg_notify(22.707(4) epoch 194453) v5 ==== 942+0+0 (2962107911 0 0) 0x9117c40 con 0x5108b00 -2145> 2014-08-13 16:24:27.983993 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.983818, event: header_read, request: pg_notify(22.707(4) epoch 194453) v5 -2144> 2014-08-13 16:24:27.984004 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.983819, event: throttled, request: pg_notify(22.707(4) epoch 194453) v5 -2143> 2014-08-13 16:24:27.984014 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.983950, event: all_read, request: pg_notify(22.707(4) epoch 194453) v5 -2142> 2014-08-13 16:24:27.984021 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.983990, event: dispatched, request: pg_notify(22.707(4) epoch 194453) v5 -2141> 2014-08-13 16:24:27.984032 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.984031, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194453) v5 -2140> 2014-08-13 16:24:27.984047 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.984047, event: started, request: pg_notify(22.707(4) epoch 194453) v5 -2139> 2014-08-13 16:24:27.984082 7fd93dff0700 5 -- op tracker -- , seq: 1099, time: 2014-08-13 16:24:27.984082, event: done, request: pg_notify(22.707(4) epoch 194453) v5 -2138> 2014-08-13 16:24:27.984118 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 26 ==== pg_notify(22.3e4(4) epoch 194453) v5 ==== 942+0+0 (2909303877 0 0) 0x91116c0 con 0x5108b00 -2137> 2014-08-13 16:24:27.984136 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984007, event: header_read, request: pg_notify(22.3e4(4) epoch 194453) v5 -2136> 2014-08-13 16:24:27.984144 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984009, event: throttled, request: pg_notify(22.3e4(4) epoch 194453) v5 -2135> 2014-08-13 16:24:27.984149 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984086, event: all_read, request: pg_notify(22.3e4(4) epoch 194453) v5 -2134> 2014-08-13 16:24:27.984155 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984134, event: dispatched, request: pg_notify(22.3e4(4) epoch 194453) v5 -2133> 2014-08-13 16:24:27.984170 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984170, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194453) v5 -2132> 2014-08-13 16:24:27.984179 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984179, event: started, request: pg_notify(22.3e4(4) epoch 194453) v5 -2131> 2014-08-13 16:24:27.984204 7fd93dff0700 5 -- op tracker -- , seq: 1100, time: 2014-08-13 16:24:27.984204, event: done, request: pg_notify(22.3e4(4) epoch 194453) v5 -2130> 2014-08-13 16:24:27.984406 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 27 ==== pg_notify(22.481(27) epoch 194453) v5 ==== 2235+0+0 (2402850715 0 0) 0x91163c0 con 0x5108b00 -2129> 2014-08-13 16:24:27.984440 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984119, event: header_read, request: pg_notify(22.481(27) epoch 194453) v5 -2128> 2014-08-13 16:24:27.984454 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984120, event: throttled, request: pg_notify(22.481(27) epoch 194453) v5 -2127> 2014-08-13 16:24:27.984460 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984357, event: all_read, request: pg_notify(22.481(27) epoch 194453) v5 -2126> 2014-08-13 16:24:27.984466 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984436, event: dispatched, request: pg_notify(22.481(27) epoch 194453) v5 -2125> 2014-08-13 16:24:27.984477 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984477, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194453) v5 -2124> 2014-08-13 16:24:27.984490 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984490, event: started, request: pg_notify(22.481(27) epoch 194453) v5 -2123> 2014-08-13 16:24:27.984527 7fd93dff0700 5 -- op tracker -- , seq: 1101, time: 2014-08-13 16:24:27.984527, event: done, request: pg_notify(22.481(27) epoch 194453) v5 -2122> 2014-08-13 16:24:27.984555 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 19 ==== pg_notify(22.481(27) epoch 194453) v5 ==== 2235+0+0 (3273024808 0 0) 0x775e040 con 0x7723440 -2121> 2014-08-13 16:24:27.984575 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984312, event: header_read, request: pg_notify(22.481(27) epoch 194453) v5 -2120> 2014-08-13 16:24:27.984586 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984314, event: throttled, request: pg_notify(22.481(27) epoch 194453) v5 -2119> 2014-08-13 16:24:27.984595 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984443, event: all_read, request: pg_notify(22.481(27) epoch 194453) v5 -2118> 2014-08-13 16:24:27.984604 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984573, event: dispatched, request: pg_notify(22.481(27) epoch 194453) v5 -2117> 2014-08-13 16:24:27.984614 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984614, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194453) v5 -2116> 2014-08-13 16:24:27.984625 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984625, event: started, request: pg_notify(22.481(27) epoch 194453) v5 -2115> 2014-08-13 16:24:27.984654 7fd93dff0700 5 -- op tracker -- , seq: 1102, time: 2014-08-13 16:24:27.984654, event: done, request: pg_notify(22.481(27) epoch 194453) v5 -2114> 2014-08-13 16:24:27.984672 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 11 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7e540 con 0x80c7d20 -2113> 2014-08-13 16:24:27.984696 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2112> 2014-08-13 16:24:27.984842 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 12 ==== pg_notify(22.481(34) epoch 194453) v5 ==== 2620+0+0 (540996460 0 0) 0x8349340 con 0x80c7d20 -2111> 2014-08-13 16:24:27.984882 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984543, event: header_read, request: pg_notify(22.481(34) epoch 194453) v5 -2110> 2014-08-13 16:24:27.984894 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984544, event: throttled, request: pg_notify(22.481(34) epoch 194453) v5 -2109> 2014-08-13 16:24:27.984901 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984744, event: all_read, request: pg_notify(22.481(34) epoch 194453) v5 -2108> 2014-08-13 16:24:27.984906 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984877, event: dispatched, request: pg_notify(22.481(34) epoch 194453) v5 -2107> 2014-08-13 16:24:27.984913 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984913, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194453) v5 -2106> 2014-08-13 16:24:27.984922 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984922, event: started, request: pg_notify(22.481(34) epoch 194453) v5 -2105> 2014-08-13 16:24:27.984953 7fd93dff0700 5 -- op tracker -- , seq: 1103, time: 2014-08-13 16:24:27.984952, event: done, request: pg_notify(22.481(34) epoch 194453) v5 -2104> 2014-08-13 16:24:27.984989 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 22 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d05340 con 0x5113b20 -2103> 2014-08-13 16:24:27.985011 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2102> 2014-08-13 16:24:27.985049 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 23 ==== pg_notify(22.1d2(17) epoch 194453) v5 ==== 1669+0+0 (1218483855 0 0) 0x840b640 con 0x5113b20 -2101> 2014-08-13 16:24:27.985082 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.984934, event: header_read, request: pg_notify(22.1d2(17) epoch 194453) v5 -2100> 2014-08-13 16:24:27.985094 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.984935, event: throttled, request: pg_notify(22.1d2(17) epoch 194453) v5 -2099> 2014-08-13 16:24:27.985104 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.985015, event: all_read, request: pg_notify(22.1d2(17) epoch 194453) v5 -2098> 2014-08-13 16:24:27.985113 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.985077, event: dispatched, request: pg_notify(22.1d2(17) epoch 194453) v5 -2097> 2014-08-13 16:24:27.985123 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.985122, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194453) v5 -2096> 2014-08-13 16:24:27.985135 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.985135, event: started, request: pg_notify(22.1d2(17) epoch 194453) v5 -2095> 2014-08-13 16:24:27.985175 7fd93dff0700 5 -- op tracker -- , seq: 1104, time: 2014-08-13 16:24:27.985175, event: done, request: pg_notify(22.1d2(17) epoch 194453) v5 -2094> 2014-08-13 16:24:27.987345 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 2 ==== pg_notify(24.181(25) epoch 194453) v5 ==== 2159+0+0 (3947621597 0 0) 0x8c89a40 con 0x83002c0 -2093> 2014-08-13 16:24:27.987382 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987118, event: header_read, request: pg_notify(24.181(25) epoch 194453) v5 -2092> 2014-08-13 16:24:27.987392 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987121, event: throttled, request: pg_notify(24.181(25) epoch 194453) v5 -2091> 2014-08-13 16:24:27.987402 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987262, event: all_read, request: pg_notify(24.181(25) epoch 194453) v5 -2090> 2014-08-13 16:24:27.987411 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987377, event: dispatched, request: pg_notify(24.181(25) epoch 194453) v5 -2089> 2014-08-13 16:24:27.987422 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987422, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194453) v5 -2088> 2014-08-13 16:24:27.987432 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987432, event: started, request: pg_notify(24.181(25) epoch 194453) v5 -2087> 2014-08-13 16:24:27.987462 7fd93dff0700 5 -- op tracker -- , seq: 1105, time: 2014-08-13 16:24:27.987461, event: done, request: pg_notify(24.181(25) epoch 194453) v5 -2086> 2014-08-13 16:24:27.987698 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 28 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8bd5e80 con 0x5111a20 -2085> 2014-08-13 16:24:27.987734 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2084> 2014-08-13 16:24:27.987850 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 29 ==== pg_notify(22.48e(15) epoch 194453) v5 ==== 1551+0+0 (1682222395 0 0) 0x840abc0 con 0x5111a20 -2083> 2014-08-13 16:24:27.987890 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987731, event: header_read, request: pg_notify(22.48e(15) epoch 194453) v5 -2082> 2014-08-13 16:24:27.987907 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987733, event: throttled, request: pg_notify(22.48e(15) epoch 194453) v5 -2081> 2014-08-13 16:24:27.987919 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987826, event: all_read, request: pg_notify(22.48e(15) epoch 194453) v5 -2080> 2014-08-13 16:24:27.987928 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987885, event: dispatched, request: pg_notify(22.48e(15) epoch 194453) v5 -2079> 2014-08-13 16:24:27.987939 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987939, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194453) v5 -2078> 2014-08-13 16:24:27.987952 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987952, event: started, request: pg_notify(22.48e(15) epoch 194453) v5 -2077> 2014-08-13 16:24:27.987986 7fd93dff0700 5 -- op tracker -- , seq: 1106, time: 2014-08-13 16:24:27.987986, event: done, request: pg_notify(22.48e(15) epoch 194453) v5 -2076> 2014-08-13 16:24:27.988734 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 3 ==== pg_notify(24.34(43) epoch 194453) v5 ==== 2933+0+0 (1121570235 0 0) 0x8c8aa00 con 0x83002c0 -2075> 2014-08-13 16:24:27.988776 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988560, event: header_read, request: pg_notify(24.34(43) epoch 194453) v5 -2074> 2014-08-13 16:24:27.988822 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988564, event: throttled, request: pg_notify(24.34(43) epoch 194453) v5 -2073> 2014-08-13 16:24:27.988833 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988703, event: all_read, request: pg_notify(24.34(43) epoch 194453) v5 -2072> 2014-08-13 16:24:27.988878 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988771, event: dispatched, request: pg_notify(24.34(43) epoch 194453) v5 -2071> 2014-08-13 16:24:27.988892 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988892, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194453) v5 -2070> 2014-08-13 16:24:27.988905 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988905, event: started, request: pg_notify(24.34(43) epoch 194453) v5 -2069> 2014-08-13 16:24:27.988946 7fd93dff0700 5 -- op tracker -- , seq: 1107, time: 2014-08-13 16:24:27.988945, event: done, request: pg_notify(24.34(43) epoch 194453) v5 -2068> 2014-08-13 16:24:27.989006 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 30 ==== pg_notify(24.1f7(18) epoch 194453) v5 ==== 1754+0+0 (1394735257 0 0) 0x840bb80 con 0x5111a20 -2067> 2014-08-13 16:24:27.989031 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.988881, event: header_read, request: pg_notify(24.1f7(18) epoch 194453) v5 -2066> 2014-08-13 16:24:27.989043 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.988883, event: throttled, request: pg_notify(24.1f7(18) epoch 194453) v5 -2065> 2014-08-13 16:24:27.989052 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.988980, event: all_read, request: pg_notify(24.1f7(18) epoch 194453) v5 -2064> 2014-08-13 16:24:27.989061 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.989028, event: dispatched, request: pg_notify(24.1f7(18) epoch 194453) v5 -2063> 2014-08-13 16:24:27.989071 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.989071, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194453) v5 -2062> 2014-08-13 16:24:27.989082 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.989082, event: started, request: pg_notify(24.1f7(18) epoch 194453) v5 -2061> 2014-08-13 16:24:27.989112 7fd93dff0700 5 -- op tracker -- , seq: 1108, time: 2014-08-13 16:24:27.989112, event: done, request: pg_notify(24.1f7(18) epoch 194453) v5 -2060> 2014-08-13 16:24:27.991191 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.17 192.168.1.103:6811/37658 16 ==== MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 ==== 26+0+0 (835259416 0 0) 0x81c08c0 con 0x4fde1a0 -2059> 2014-08-13 16:24:27.991224 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991092, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2058> 2014-08-13 16:24:27.991238 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991095, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2057> 2014-08-13 16:24:27.991244 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991167, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2056> 2014-08-13 16:24:27.991250 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991220, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2055> 2014-08-13 16:24:27.991260 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991260, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2054> 2014-08-13 16:24:27.991286 7fd93dff0700 5 -- op tracker -- , seq: 1109, time: 2014-08-13 16:24:27.991286, event: done, request: MRecoveryReserve REQUEST pgid: 24.2a1, query_epoch: 194453 v2 -2053> 2014-08-13 16:24:27.991306 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194447 pi=193188-194446/25 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] exit Started/ReplicaActive/RepNotRecovering 23.561535 19 0.002633 -2052> 2014-08-13 16:24:27.991352 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194447 pi=193188-194446/25 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] enter Started/ReplicaActive/RepWaitRecoveryReserved -2051> 2014-08-13 16:24:27.991508 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 28 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7e300 con 0x4fd8000 -2050> 2014-08-13 16:24:27.991545 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2049> 2014-08-13 16:24:27.991966 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 7 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d03840 con 0x7d2d120 -2048> 2014-08-13 16:24:27.992003 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2047> 2014-08-13 16:24:27.992015 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 29 ==== pg_notify(24.136(17) epoch 194453) v5 ==== 1695+0+0 (3978175316 0 0) 0x80dcd00 con 0x4fd8000 -2046> 2014-08-13 16:24:27.992031 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.991532, event: header_read, request: pg_notify(24.136(17) epoch 194453) v5 -2045> 2014-08-13 16:24:27.992040 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.991534, event: throttled, request: pg_notify(24.136(17) epoch 194453) v5 -2044> 2014-08-13 16:24:27.992046 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.991633, event: all_read, request: pg_notify(24.136(17) epoch 194453) v5 -2043> 2014-08-13 16:24:27.992051 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.992028, event: dispatched, request: pg_notify(24.136(17) epoch 194453) v5 -2042> 2014-08-13 16:24:27.992058 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.992058, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194453) v5 -2041> 2014-08-13 16:24:27.992066 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.992066, event: started, request: pg_notify(24.136(17) epoch 194453) v5 -2040> 2014-08-13 16:24:27.992096 7fd93dff0700 5 -- op tracker -- , seq: 1110, time: 2014-08-13 16:24:27.992096, event: done, request: pg_notify(24.136(17) epoch 194453) v5 -2039> 2014-08-13 16:24:27.992126 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 17 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859c140 con 0x5112260 -2038> 2014-08-13 16:24:27.992143 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2037> 2014-08-13 16:24:27.992152 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 8 ==== pg_notify(24.34(30) epoch 194453) v5 ==== 2250+0+0 (3844313431 0 0) 0x8c88fc0 con 0x7d2d120 -2036> 2014-08-13 16:24:27.992166 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.991943, event: header_read, request: pg_notify(24.34(30) epoch 194453) v5 -2035> 2014-08-13 16:24:27.992173 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.991945, event: throttled, request: pg_notify(24.34(30) epoch 194453) v5 -2034> 2014-08-13 16:24:27.992189 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.992077, event: all_read, request: pg_notify(24.34(30) epoch 194453) v5 -2033> 2014-08-13 16:24:27.992195 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.992164, event: dispatched, request: pg_notify(24.34(30) epoch 194453) v5 -2032> 2014-08-13 16:24:27.992202 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.992201, event: waiting_for_osdmap, request: pg_notify(24.34(30) epoch 194453) v5 -2031> 2014-08-13 16:24:27.992210 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.992209, event: started, request: pg_notify(24.34(30) epoch 194453) v5 -2030> 2014-08-13 16:24:27.992236 7fd93dff0700 5 -- op tracker -- , seq: 1111, time: 2014-08-13 16:24:27.992236, event: done, request: pg_notify(24.34(30) epoch 194453) v5 -2029> 2014-08-13 16:24:27.992263 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 18 ==== pg_notify(22.4df(15) epoch 194453) v5 ==== 1551+0+0 (814893558 0 0) 0x8c8fe00 con 0x5112260 -2028> 2014-08-13 16:24:27.992279 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.991880, event: header_read, request: pg_notify(22.4df(15) epoch 194453) v5 -2027> 2014-08-13 16:24:27.992286 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.991882, event: throttled, request: pg_notify(22.4df(15) epoch 194453) v5 -2026> 2014-08-13 16:24:27.992291 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.991993, event: all_read, request: pg_notify(22.4df(15) epoch 194453) v5 -2025> 2014-08-13 16:24:27.992296 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.992277, event: dispatched, request: pg_notify(22.4df(15) epoch 194453) v5 -2024> 2014-08-13 16:24:27.992302 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.992302, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194453) v5 -2023> 2014-08-13 16:24:27.992310 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.992309, event: started, request: pg_notify(22.4df(15) epoch 194453) v5 -2022> 2014-08-13 16:24:27.992333 7fd93dff0700 5 -- op tracker -- , seq: 1112, time: 2014-08-13 16:24:27.992333, event: done, request: pg_notify(22.4df(15) epoch 194453) v5 -2021> 2014-08-13 16:24:27.992501 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 11 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7c800 con 0x8da1760 -2020> 2014-08-13 16:24:27.992529 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2019> 2014-08-13 16:24:27.992620 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 12 ==== pg_notify(22.619(17) epoch 194453) v5 ==== 1665+0+0 (137763550 0 0) 0x834e200 con 0x8da1760 -2018> 2014-08-13 16:24:27.992641 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992505, event: header_read, request: pg_notify(22.619(17) epoch 194453) v5 -2017> 2014-08-13 16:24:27.992649 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992506, event: throttled, request: pg_notify(22.619(17) epoch 194453) v5 -2016> 2014-08-13 16:24:27.992654 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992598, event: all_read, request: pg_notify(22.619(17) epoch 194453) v5 -2015> 2014-08-13 16:24:27.992659 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992637, event: dispatched, request: pg_notify(22.619(17) epoch 194453) v5 -2014> 2014-08-13 16:24:27.992665 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992665, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194453) v5 -2013> 2014-08-13 16:24:27.992673 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992673, event: started, request: pg_notify(22.619(17) epoch 194453) v5 -2012> 2014-08-13 16:24:27.992704 7fd93dff0700 5 -- op tracker -- , seq: 1113, time: 2014-08-13 16:24:27.992704, event: done, request: pg_notify(22.619(17) epoch 194453) v5 -2011> 2014-08-13 16:24:27.993259 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x506f080 con 0x50bfa60 -2010> 2014-08-13 16:24:27.993299 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -2009> 2014-08-13 16:24:27.993411 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 13 ==== pg_notify(22.113(17) epoch 194453) v5 ==== 1669+0+0 (1013157857 0 0) 0x8998380 con 0x50bfa60 -2008> 2014-08-13 16:24:27.993443 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993261, event: header_read, request: pg_notify(22.113(17) epoch 194453) v5 -2007> 2014-08-13 16:24:27.993458 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993283, event: throttled, request: pg_notify(22.113(17) epoch 194453) v5 -2006> 2014-08-13 16:24:27.993465 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993385, event: all_read, request: pg_notify(22.113(17) epoch 194453) v5 -2005> 2014-08-13 16:24:27.993470 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993438, event: dispatched, request: pg_notify(22.113(17) epoch 194453) v5 -2004> 2014-08-13 16:24:27.993481 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993481, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194453) v5 -2003> 2014-08-13 16:24:27.993494 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993493, event: started, request: pg_notify(22.113(17) epoch 194453) v5 -2002> 2014-08-13 16:24:27.993528 7fd93dff0700 5 -- op tracker -- , seq: 1114, time: 2014-08-13 16:24:27.993528, event: done, request: pg_notify(22.113(17) epoch 194453) v5 -2001> 2014-08-13 16:24:27.993554 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.13 192.168.1.115:6804/58748 10 ==== pg_info(1 pgs e194453:24.39a) v4 ==== 744+0+0 (1310246748 0 0) 0x8577c40 con 0x4fd9b80 -2000> 2014-08-13 16:24:27.993575 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993442, event: header_read, request: pg_info(1 pgs e194453:24.39a) v4 -1999> 2014-08-13 16:24:27.993586 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993443, event: throttled, request: pg_info(1 pgs e194453:24.39a) v4 -1998> 2014-08-13 16:24:27.993595 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993514, event: all_read, request: pg_info(1 pgs e194453:24.39a) v4 -1997> 2014-08-13 16:24:27.993604 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993572, event: dispatched, request: pg_info(1 pgs e194453:24.39a) v4 -1996> 2014-08-13 16:24:27.993614 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993614, event: waiting_for_osdmap, request: pg_info(1 pgs e194453:24.39a) v4 -1995> 2014-08-13 16:24:27.993626 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993626, event: started, request: pg_info(1 pgs e194453:24.39a) v4 -1994> 2014-08-13 16:24:27.993654 7fd93dff0700 5 -- op tracker -- , seq: 1115, time: 2014-08-13 16:24:27.993654, event: done, request: pg_info(1 pgs e194453:24.39a) v4 -1993> 2014-08-13 16:24:27.993711 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.13 192.168.1.115:6804/58748 -- pg_info(1 pgs e194453:24.39a) v4 -- ?+0 0x834dcc0 -1992> 2014-08-13 16:24:27.993760 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194453:24.39a) v4 -- ?+0 0x83488c0 -1991> 2014-08-13 16:24:27.993808 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> osd.395 192.168.1.105:6810/41287 -- pg_info(1 pgs e194453:24.39a) v4 -- ?+0 0x834f540 -1990> 2014-08-13 16:24:27.994334 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x33e2400 con 0x510dac0 -1989> 2014-08-13 16:24:27.994368 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1988> 2014-08-13 16:24:27.994231 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194453/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 active+remapped m=49] on activate: bft=349 from 0//0//-1 -1987> 2014-08-13 16:24:27.994537 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.5 192.168.1.103:0/40192 8 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:27.994551) v2 ==== 47+0+0 (3638063519 0 0) 0x842cd00 con 0x7eb91e0 -1986> 2014-08-13 16:24:27.994547 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.5 192.168.1.103:0/40192 8 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:27.994551) v2 ==== 47+0+0 (3638063519 0 0) 0x8348c40 con 0x7eba680 -1985> 2014-08-13 16:24:27.994579 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:27.994551) v2 -- ?+0 0x840d240 con 0x7eb91e0 -1984> 2014-08-13 16:24:27.994579 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 13 ==== pg_notify(22.3d9(39) epoch 194453) v5 ==== 2815+0+0 (2365135528 0 0) 0x9117c40 con 0x510dac0 -1983> 2014-08-13 16:24:27.994605 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994337, event: header_read, request: pg_notify(22.3d9(39) epoch 194453) v5 -1982> 2014-08-13 16:24:27.994615 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994338, event: throttled, request: pg_notify(22.3d9(39) epoch 194453) v5 -1981> 2014-08-13 16:24:27.994622 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994557, event: all_read, request: pg_notify(22.3d9(39) epoch 194453) v5 -1980> 2014-08-13 16:24:27.994612 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:27.994551) v2 -- ?+0 0x840cd00 con 0x7eba680 -1979> 2014-08-13 16:24:27.994627 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994601, event: dispatched, request: pg_notify(22.3d9(39) epoch 194453) v5 -1978> 2014-08-13 16:24:27.994634 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994634, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194453) v5 -1977> 2014-08-13 16:24:27.994643 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994643, event: started, request: pg_notify(22.3d9(39) epoch 194453) v5 -1976> 2014-08-13 16:24:27.994678 7fd93dff0700 5 -- op tracker -- , seq: 1116, time: 2014-08-13 16:24:27.994678, event: done, request: pg_notify(22.3d9(39) epoch 194453) v5 -1975> 2014-08-13 16:24:27.994506 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194453/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 active+remapped m=49] target shard 349 from 0//0//-1 -1974> 2014-08-13 16:24:27.994769 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8bd2ac0 con 0x73ca680 -1973> 2014-08-13 16:24:27.994796 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1972> 2014-08-13 16:24:27.994953 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 13 ==== pg_notify(22.1f1(4) epoch 194453) v5 ==== 942+0+0 (982760903 0 0) 0x775e740 con 0x73ca680 -1971> 2014-08-13 16:24:27.994948 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.328 192.168.100.104:0/64303 8 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:27.995056) v2 ==== 47+0+0 (790644414 0 0) 0x842fe00 con 0x7d2a520 -1970> 2014-08-13 16:24:27.994978 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.994776, event: header_read, request: pg_notify(22.1f1(4) epoch 194453) v5 -1969> 2014-08-13 16:24:27.994987 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.994777, event: throttled, request: pg_notify(22.1f1(4) epoch 194453) v5 -1968> 2014-08-13 16:24:27.994981 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:27.995056) v2 -- ?+0 0x8348c40 con 0x7d2a520 -1967> 2014-08-13 16:24:27.994993 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.994849, event: all_read, request: pg_notify(22.1f1(4) epoch 194453) v5 -1966> 2014-08-13 16:24:27.994999 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.994974, event: dispatched, request: pg_notify(22.1f1(4) epoch 194453) v5 -1965> 2014-08-13 16:24:27.995036 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.995036, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194453) v5 -1964> 2014-08-13 16:24:27.995047 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.995047, event: started, request: pg_notify(22.1f1(4) epoch 194453) v5 -1963> 2014-08-13 16:24:27.995078 7fd93dff0700 5 -- op tracker -- , seq: 1117, time: 2014-08-13 16:24:27.995078, event: done, request: pg_notify(22.1f1(4) epoch 194453) v5 -1962> 2014-08-13 16:24:27.995347 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.328 192.168.100.104:0/64303 8 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:27.995056) v2 ==== 47+0+0 (790644414 0 0) 0x9284d00 con 0x7cae5c0 -1961> 2014-08-13 16:24:27.995387 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:27.995056) v2 -- ?+0 0x842cd00 con 0x7cae5c0 -1960> 2014-08-13 16:24:27.995227 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194453/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 active+remapped m=49] exit Started/Primary/Active/Activating 0.062387 11 0.003497 -1959> 2014-08-13 16:24:27.995498 7fd9367e4700 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194453/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194451 pi=193304-194450/52 bft=349 crt=194422'13151 lcod 0'0 mlcod 0'0 active+remapped m=49] enter Started/Primary/Active/WaitLocalRecoveryReserved -1958> 2014-08-13 16:24:27.997378 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 19 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d03180 con 0x508a680 -1957> 2014-08-13 16:24:27.997425 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1956> 2014-08-13 16:24:27.997457 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 20 ==== pg_notify(22.619(4) epoch 194453) v5 ==== 942+0+0 (284650031 0 0) 0x834f1c0 con 0x508a680 -1955> 2014-08-13 16:24:27.997480 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997340, event: header_read, request: pg_notify(22.619(4) epoch 194453) v5 -1954> 2014-08-13 16:24:27.997494 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997341, event: throttled, request: pg_notify(22.619(4) epoch 194453) v5 -1953> 2014-08-13 16:24:27.997501 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997438, event: all_read, request: pg_notify(22.619(4) epoch 194453) v5 -1952> 2014-08-13 16:24:27.997507 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997477, event: dispatched, request: pg_notify(22.619(4) epoch 194453) v5 -1951> 2014-08-13 16:24:27.997514 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997514, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194453) v5 -1950> 2014-08-13 16:24:27.997527 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997527, event: started, request: pg_notify(22.619(4) epoch 194453) v5 -1949> 2014-08-13 16:24:27.997564 7fd93dff0700 5 -- op tracker -- , seq: 1118, time: 2014-08-13 16:24:27.997564, event: done, request: pg_notify(22.619(4) epoch 194453) v5 -1948> 2014-08-13 16:24:27.998303 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 20 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x859f980 con 0x510b9c0 -1947> 2014-08-13 16:24:27.998339 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1946> 2014-08-13 16:24:27.998351 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 21 ==== pg_notify(22.362(17) epoch 194453) v5 ==== 1665+0+0 (2693871536 0 0) 0x8c8ec80 con 0x510b9c0 -1945> 2014-08-13 16:24:27.998368 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.997868, event: header_read, request: pg_notify(22.362(17) epoch 194453) v5 -1944> 2014-08-13 16:24:27.998378 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.997869, event: throttled, request: pg_notify(22.362(17) epoch 194453) v5 -1943> 2014-08-13 16:24:27.998391 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.997975, event: all_read, request: pg_notify(22.362(17) epoch 194453) v5 -1942> 2014-08-13 16:24:27.998400 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.998365, event: dispatched, request: pg_notify(22.362(17) epoch 194453) v5 -1941> 2014-08-13 16:24:27.998411 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.998410, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194453) v5 -1940> 2014-08-13 16:24:27.998424 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.998423, event: started, request: pg_notify(22.362(17) epoch 194453) v5 -1939> 2014-08-13 16:24:27.998457 7fd93dff0700 5 -- op tracker -- , seq: 1119, time: 2014-08-13 16:24:27.998457, event: done, request: pg_notify(22.362(17) epoch 194453) v5 -1938> 2014-08-13 16:24:28.004106 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 11 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x7e7b180 con 0x8e2c360 -1937> 2014-08-13 16:24:28.004155 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1936> 2014-08-13 16:24:28.004179 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 12 ==== pg_notify(22.4df(15) epoch 194453) v5 ==== 1551+0+0 (814893558 0 0) 0x842d5c0 con 0x8e2c360 -1935> 2014-08-13 16:24:28.004220 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004026, event: header_read, request: pg_notify(22.4df(15) epoch 194453) v5 -1934> 2014-08-13 16:24:28.004232 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004028, event: throttled, request: pg_notify(22.4df(15) epoch 194453) v5 -1933> 2014-08-13 16:24:28.004239 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004124, event: all_read, request: pg_notify(22.4df(15) epoch 194453) v5 -1932> 2014-08-13 16:24:28.004244 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004216, event: dispatched, request: pg_notify(22.4df(15) epoch 194453) v5 -1931> 2014-08-13 16:24:28.004251 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004251, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194453) v5 -1930> 2014-08-13 16:24:28.004261 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004261, event: started, request: pg_notify(22.4df(15) epoch 194453) v5 -1929> 2014-08-13 16:24:28.004297 7fd93dff0700 5 -- op tracker -- , seq: 1120, time: 2014-08-13 16:24:28.004297, event: done, request: pg_notify(22.4df(15) epoch 194453) v5 -1928> 2014-08-13 16:24:28.008090 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 12 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x5077500 con 0x8da5120 -1927> 2014-08-13 16:24:28.008126 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1926> 2014-08-13 16:24:28.008221 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 13 ==== pg_notify(22.f3(15) epoch 194453) v5 ==== 1551+0+0 (482103762 0 0) 0x834a840 con 0x8da5120 -1925> 2014-08-13 16:24:28.008244 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008091, event: header_read, request: pg_notify(22.f3(15) epoch 194453) v5 -1924> 2014-08-13 16:24:28.008257 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008092, event: throttled, request: pg_notify(22.f3(15) epoch 194453) v5 -1923> 2014-08-13 16:24:28.008267 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008202, event: all_read, request: pg_notify(22.f3(15) epoch 194453) v5 -1922> 2014-08-13 16:24:28.008276 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008241, event: dispatched, request: pg_notify(22.f3(15) epoch 194453) v5 -1921> 2014-08-13 16:24:28.008286 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008286, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194453) v5 -1920> 2014-08-13 16:24:28.008299 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008298, event: started, request: pg_notify(22.f3(15) epoch 194453) v5 -1919> 2014-08-13 16:24:28.008332 7fd93dff0700 5 -- op tracker -- , seq: 1121, time: 2014-08-13 16:24:28.008331, event: done, request: pg_notify(22.f3(15) epoch 194453) v5 -1918> 2014-08-13 16:24:28.026334 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 11 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x506e9c0 con 0x72de9e0 -1917> 2014-08-13 16:24:28.026384 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1916> 2014-08-13 16:24:28.026411 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 12 ==== pg_notify(22.39d(4) epoch 194453) v5 ==== 942+0+0 (1576891432 0 0) 0x80de580 con 0x72de9e0 -1915> 2014-08-13 16:24:28.026444 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026280, event: header_read, request: pg_notify(22.39d(4) epoch 194453) v5 -1914> 2014-08-13 16:24:28.026459 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026281, event: throttled, request: pg_notify(22.39d(4) epoch 194453) v5 -1913> 2014-08-13 16:24:28.026468 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026352, event: all_read, request: pg_notify(22.39d(4) epoch 194453) v5 -1912> 2014-08-13 16:24:28.026477 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026440, event: dispatched, request: pg_notify(22.39d(4) epoch 194453) v5 -1911> 2014-08-13 16:24:28.026488 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026488, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194453) v5 -1910> 2014-08-13 16:24:28.026501 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026501, event: started, request: pg_notify(22.39d(4) epoch 194453) v5 -1909> 2014-08-13 16:24:28.026538 7fd93dff0700 5 -- op tracker -- , seq: 1122, time: 2014-08-13 16:24:28.026538, event: done, request: pg_notify(22.39d(4) epoch 194453) v5 -1908> 2014-08-13 16:24:28.031278 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 9 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:28.031171) v2 ==== 47+0+0 (1369678219 0 0) 0x87e0380 con 0x7ed82c0 -1907> 2014-08-13 16:24:28.031327 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.031171) v2 -- ?+0 0x842fe00 con 0x7ed82c0 -1906> 2014-08-13 16:24:28.031328 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 9 ==== osd_ping(ping e194452 stamp 2014-08-13 16:24:28.031171) v2 ==== 47+0+0 (1369678219 0 0) 0x87e7700 con 0x7edb180 -1905> 2014-08-13 16:24:28.031365 7fd93c1ed700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.101:6808/22782 -- osd_map(194453..194453 src has 185139..194453) v3 -- ?+0 0x85998c0 con 0x4e4d800 -1904> 2014-08-13 16:24:28.031422 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.031171) v2 -- ?+0 0x9284d00 con 0x7edb180 -1903> 2014-08-13 16:24:28.060357 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 17 ==== osd_map(194453..194453 src has 185139..194453) v3 ==== 820+0+0 (2950693608 0 0) 0x8d05c40 con 0x4e4ce60 -1902> 2014-08-13 16:24:28.060404 7fd93dff0700 3 osd.4 194453 handle_osd_map epochs [194453,194453], i have 194453, src has [185139,194453] -1901> 2014-08-13 16:24:28.060447 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 18 ==== pg_notify(22.1f1(17) epoch 194453) v5 ==== 1665+0+0 (291167416 0 0) 0x842fe00 con 0x4e4ce60 -1900> 2014-08-13 16:24:28.060527 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060322, event: header_read, request: pg_notify(22.1f1(17) epoch 194453) v5 -1899> 2014-08-13 16:24:28.060539 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060323, event: throttled, request: pg_notify(22.1f1(17) epoch 194453) v5 -1898> 2014-08-13 16:24:28.060546 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060420, event: all_read, request: pg_notify(22.1f1(17) epoch 194453) v5 -1897> 2014-08-13 16:24:28.060551 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060523, event: dispatched, request: pg_notify(22.1f1(17) epoch 194453) v5 -1896> 2014-08-13 16:24:28.060558 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060558, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194453) v5 -1895> 2014-08-13 16:24:28.060568 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060568, event: started, request: pg_notify(22.1f1(17) epoch 194453) v5 -1894> 2014-08-13 16:24:28.060605 7fd93dff0700 5 -- op tracker -- , seq: 1123, time: 2014-08-13 16:24:28.060605, event: done, request: pg_notify(22.1f1(17) epoch 194453) v5 -1893> 2014-08-13 16:24:28.074012 7fd933fe0700 5 osd.4 194453 heartbeat: osd_stat(8082 MB used, 2783 GB avail, 2791 GB total, peers [0,3,5,10,11,13,15,17,20,21,24,25,26,28,30,31,44,45,47,48,50,54,55,56,58,59,61,62,63,65,285,286,287,289,291,293,296,301,302,303,304,306,307,309,310,311,312,313,314,315,318,320,324,327,330,331,332,337,338,339,340,341,342,343,344,347,348,349,350,351,352,353,356,357,362,363,365,366,367,368,369,371,373,374,375,376,378,379,380,381,382,395,396,397]/[] op hist []) -1892> 2014-08-13 16:24:28.074060 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6811/59615 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91147c0 con 0x8c391e0 -1891> 2014-08-13 16:24:28.074094 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6825/59615 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9114b40 con 0x8c38f20 -1890> 2014-08-13 16:24:28.074117 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6841/56287 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9114440 con 0x78844c0 -1889> 2014-08-13 16:24:28.074139 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6845/56287 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9114280 con 0x7884200 -1888> 2014-08-13 16:24:28.074162 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6830/40192 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9114600 con 0x78848e0 -1887> 2014-08-13 16:24:28.074198 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6847/40192 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9114980 con 0x7884620 -1886> 2014-08-13 16:24:28.074220 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6816/60573 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9113800 con 0x794c360 -1885> 2014-08-13 16:24:28.074238 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6834/60573 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9113640 con 0x794e300 -1884> 2014-08-13 16:24:28.074255 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6814/37929 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91132c0 con 0x794e040 -1883> 2014-08-13 16:24:28.074277 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6821/37929 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112f40 con 0x794dd80 -1882> 2014-08-13 16:24:28.074298 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6805/58748 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9113100 con 0x8d12100 -1881> 2014-08-13 16:24:28.074315 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6810/58748 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112bc0 con 0x823c4c0 -1880> 2014-08-13 16:24:28.074335 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6814/53408 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9113480 con 0x8c3a3c0 -1879> 2014-08-13 16:24:28.074353 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6819/53408 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112680 con 0x8c3a100 -1878> 2014-08-13 16:24:28.074370 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6812/37658 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112a00 con 0x8c3c0a0 -1877> 2014-08-13 16:24:28.074386 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6818/37658 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112140 con 0x8c3bde0 -1876> 2014-08-13 16:24:28.074404 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6816/38269 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112840 con 0x8babde0 -1875> 2014-08-13 16:24:28.074422 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6827/38269 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91124c0 con 0x8babb20 -1874> 2014-08-13 16:24:28.074441 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6822/54786 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112300 con 0x8c3d6a0 -1873> 2014-08-13 16:24:28.074459 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6838/54786 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9111f80 con 0x8c3d3e0 -1872> 2014-08-13 16:24:28.074479 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6801/52669 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9112d80 con 0x8badac0 -1871> 2014-08-13 16:24:28.074497 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6803/52669 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9113f00 con 0x8bad800 -1870> 2014-08-13 16:24:28.074514 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6801/36920 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9115e80 con 0x8bab860 -1869> 2014-08-13 16:24:28.074532 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6802/36920 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9115b00 con 0x8bab5a0 -1868> 2014-08-13 16:24:28.074555 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.115:6803/58522 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9115780 con 0x823c8e0 -1867> 2014-08-13 16:24:28.074539 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.1.115:6811/59615 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x81c3800 con 0x8c391e0 -1866> 2014-08-13 16:24:28.074579 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.115:6804/58522 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9286580 con 0x823ca40 -1865> 2014-08-13 16:24:28.074598 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6808/37391 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9285080 con 0x8c69600 -1864> 2014-08-13 16:24:28.074634 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6813/37391 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9282140 con 0x8c69340 -1863> 2014-08-13 16:24:28.074658 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.107:6824/55491 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d9a40 con 0x823f4e0 -1862> 2014-08-13 16:24:28.074675 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.107:6841/55491 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80daa00 con 0x823f640 -1861> 2014-08-13 16:24:28.074691 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.103:6832/41157 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80dc280 con 0x8baee00 -1860> 2014-08-13 16:24:28.074709 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.103:6850/41157 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80de900 con 0x8ba1a20 -1859> 2014-08-13 16:24:28.074726 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6823/10782 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80da840 con 0x7fba3c0 -1858> 2014-08-13 16:24:28.074744 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6802/10782 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80deac0 con 0x7fba260 -1857> 2014-08-13 16:24:28.074759 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6829/6040 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d9880 con 0x8bad540 -1856> 2014-08-13 16:24:28.074777 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6823/6040 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80da300 con 0x8bad280 -1855> 2014-08-13 16:24:28.074795 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6829/12102 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80da4c0 con 0x8c3cba0 -1854> 2014-08-13 16:24:28.074812 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6815/12102 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80dd5c0 con 0x8c3c8e0 -1853> 2014-08-13 16:24:28.074830 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6839/8300 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d8c40 con 0x7fba100 -1852> 2014-08-13 16:24:28.074848 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6842/8300 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80df540 con 0x7fb9fa0 -1851> 2014-08-13 16:24:28.074865 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6843/9607 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80da140 con 0x8d223c0 -1850> 2014-08-13 16:24:28.074882 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6849/9607 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d8fc0 con 0x8d22100 -1849> 2014-08-13 16:24:28.074900 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6837/14923 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d9f80 con 0x8c3c620 -1848> 2014-08-13 16:24:28.074919 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6833/14923 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d9340 con 0x8c3c360 -1847> 2014-08-13 16:24:28.074935 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6835/6966 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80d9dc0 con 0x8bafe80 -1846> 2014-08-13 16:24:28.074954 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6836/6966 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80dcd00 con 0x8bafbc0 -1845> 2014-08-13 16:24:28.074970 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6827/11446 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x80dec80 con 0x8c38c60 -1844> 2014-08-13 16:24:28.074986 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6811/11446 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c89500 con 0x8c389a0 -1843> 2014-08-13 16:24:28.075003 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6835/14316 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c88700 con 0x8b68c60 -1842> 2014-08-13 16:24:28.075022 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6829/14316 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c89dc0 con 0x8b68b00 -1841> 2014-08-13 16:24:28.075038 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6823/5265 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c8cd00 con 0x8c6fe80 -1840> 2014-08-13 16:24:28.075055 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6804/5265 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c8a140 con 0x8c6fbc0 -1839> 2014-08-13 16:24:28.075072 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6841/16566 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c8aa00 con 0x8b689a0 -1838> 2014-08-13 16:24:28.075091 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6839/16566 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c89a40 con 0x8b68840 -1837> 2014-08-13 16:24:28.075108 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.112:6825/5505 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c8b480 con 0x7fb9e40 -1836> 2014-08-13 16:24:28.075125 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.112:6810/5505 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8c896c0 con 0x7fb9ce0 -1835> 2014-08-13 16:24:28.075146 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6831/12680 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840bb80 con 0x8c3b9c0 -1834> 2014-08-13 16:24:28.075165 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6820/12680 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840abc0 con 0x8c3b700 -1833> 2014-08-13 16:24:28.075192 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.111:6843/17345 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840b640 con 0x7fbb020 -1832> 2014-08-13 16:24:28.075208 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.111:6842/17345 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840a300 con 0x7fbaec0 -1831> 2014-08-13 16:24:28.075225 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6823/26295 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840c0c0 con 0x7fbb440 -1830> 2014-08-13 16:24:28.075242 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6845/26295 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8409c00 con 0x7fbb2e0 -1829> 2014-08-13 16:24:28.075259 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6821/19173 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840c600 con 0x7ffd960 -1828> 2014-08-13 16:24:28.075276 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6849/19173 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8408c40 con 0x7ffdc20 -1827> 2014-08-13 16:24:28.075296 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6815/33435 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840ad80 con 0x8d20840 -1826> 2014-08-13 16:24:28.075315 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6839/33435 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x840fc40 con 0x8d25960 -1825> 2014-08-13 16:24:28.075334 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6809/15284 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8409340 con 0x8d218c0 -1824> 2014-08-13 16:24:28.075354 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6828/15284 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8349340 con 0x8d21600 -1823> 2014-08-13 16:24:28.075371 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6821/25633 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8349880 con 0x7fbaaa0 -1822> 2014-08-13 16:24:28.075389 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6841/25633 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8348e00 con 0x7fba940 -1821> 2014-08-13 16:24:28.075408 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6817/33855 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x834b800 con 0x8d26e00 -1820> 2014-08-13 16:24:28.075436 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6844/33855 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x834d240 con 0x8d26b40 -1819> 2014-08-13 16:24:28.075456 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6807/31286 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x834fc40 con 0x8d21340 -1818> 2014-08-13 16:24:28.075484 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6815/31286 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8574980 con 0x8d21080 -1817> 2014-08-13 16:24:28.075505 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6815/16727 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573d40 con 0x7ffdee0 -1816> 2014-08-13 16:24:28.075523 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6840/16727 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573b80 con 0x7ffdd80 -1815> 2014-08-13 16:24:28.075543 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6811/15731 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573f00 con 0x7fb9a20 -1814> 2014-08-13 16:24:28.075562 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6834/15731 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573800 con 0x7fb98c0 -1813> 2014-08-13 16:24:28.075582 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6803/30768 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573640 con 0x8bab2e0 -1812> 2014-08-13 16:24:28.075602 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6807/30768 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573480 con 0x8bab020 -1811> 2014-08-13 16:24:28.075624 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6809/22782 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85739c0 con 0x8c69080 -1810> 2014-08-13 16:24:28.075644 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6812/22782 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8573100 con 0x8c68dc0 -1809> 2014-08-13 16:24:28.075663 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6807/6812 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8574440 con 0x8d256a0 -1808> 2014-08-13 16:24:28.075681 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6805/6812 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572bc0 con 0x8d253e0 -1807> 2014-08-13 16:24:28.075698 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6805/31016 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572840 con 0x8d20dc0 -1806> 2014-08-13 16:24:28.075718 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6811/31016 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572d80 con 0x8d20b00 -1805> 2014-08-13 16:24:28.075734 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6815/23703 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572680 con 0x8d25120 -1804> 2014-08-13 16:24:28.075756 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6825/23703 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572a00 con 0x8d24e60 -1803> 2014-08-13 16:24:28.075772 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6813/32413 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85732c0 con 0x8c3d120 -1802> 2014-08-13 16:24:28.075790 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6833/32413 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572f40 con 0x8c3ce60 -1801> 2014-08-13 16:24:28.075806 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6817/24270 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572300 con 0x7fbb180 -1800> 2014-08-13 16:24:28.075825 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6831/24270 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85724c0 con 0x7fbca40 -1799> 2014-08-13 16:24:28.075841 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6817/17521 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571dc0 con 0x7fbc780 -1798> 2014-08-13 16:24:28.075859 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6843/17521 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8572140 con 0x7fbc4c0 -1797> 2014-08-13 16:24:28.075878 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.109:6821/35445 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571f80 con 0x8c3aec0 -1796> 2014-08-13 16:24:28.075896 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.109:6850/35445 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571500 con 0x8c3ac00 -1795> 2014-08-13 16:24:28.075913 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.101:6803/22303 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85716c0 con 0x7fbad60 -1794> 2014-08-13 16:24:28.075929 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.101:6803/22303 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571a40 con 0x7fbac00 -1793> 2014-08-13 16:24:28.075944 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.113:6801/14161 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8575240 con 0x8c3e880 -1792> 2014-08-13 16:24:28.075963 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.113:6801/14161 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8575080 con 0x8c3e5c0 -1791> 2014-08-13 16:24:28.075980 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6815/43857 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8575400 con 0x8c3a940 -1790> 2014-08-13 16:24:28.076000 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6840/43857 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8574ec0 con 0x8c3a680 -1789> 2014-08-13 16:24:28.076017 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6821/10050 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85747c0 con 0x8c69fa0 -1788> 2014-08-13 16:24:28.076033 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6849/10050 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8574d00 con 0x8c39fa0 -1787> 2014-08-13 16:24:28.076050 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6815/65209 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571880 con 0x7fb9760 -1786> 2014-08-13 16:24:28.076068 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6840/65209 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571c00 con 0x7fb9600 -1785> 2014-08-13 16:24:28.076086 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6805/5112 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8576900 con 0x8c3e300 -1784> 2014-08-13 16:24:28.076101 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6813/5112 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85708c0 con 0x8c618c0 -1783> 2014-08-13 16:24:28.076118 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6803/4876 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85771c0 con 0x8bac780 -1782> 2014-08-13 16:24:28.076141 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6807/4876 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8577700 con 0x8bac4c0 -1781> 2014-08-13 16:24:28.076157 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6817/837 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8575cc0 con 0x8bac200 -1780> 2014-08-13 16:24:28.076182 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6843/837 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8570c40 con 0x8babf40 -1779> 2014-08-13 16:24:28.076201 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6807/42024 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8575780 con 0x8d240a0 -1778> 2014-08-13 16:24:28.076219 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6825/42024 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85701c0 con 0x8d23de0 -1777> 2014-08-13 16:24:28.076235 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6805/62197 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8576740 con 0x8ba1760 -1776> 2014-08-13 16:24:28.076251 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6816/62197 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8570700 con 0x8ba14a0 -1775> 2014-08-13 16:24:28.076278 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6815/25893 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8576c80 con 0x8d26880 -1774> 2014-08-13 16:24:28.076296 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6802/25893 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x85778c0 con 0x8d265c0 -1773> 2014-08-13 16:24:28.076313 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6819/45268 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8571180 con 0x8baf900 -1772> 2014-08-13 16:24:28.076335 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6846/45268 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eed780 con 0x8baf640 -1771> 2014-08-13 16:24:28.076353 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6807/62463 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eee580 con 0x8bae040 -1770> 2014-08-13 16:24:28.076374 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6822/62463 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8540 con 0x8badd80 -1769> 2014-08-13 16:24:28.076394 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6819/9142 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eed940 con 0x8d24620 -1768> 2014-08-13 16:24:28.076409 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6846/9142 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eea4c0 con 0x8d24360 -1767> 2014-08-13 16:24:28.076426 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6803/41438 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeeac0 con 0x8baf380 -1766> 2014-08-13 16:24:28.076447 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6810/41438 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eed400 con 0x8baf0c0 -1765> 2014-08-13 16:24:28.076471 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6803/61960 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eefe00 con 0x8b6e1a0 -1764> 2014-08-13 16:24:28.076488 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6810/61960 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eedb00 con 0x8b6e040 -1763> 2014-08-13 16:24:28.076505 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6813/6757 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9180 con 0x8c39ce0 -1762> 2014-08-13 16:24:28.076526 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6837/6757 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeaf40 con 0x8c39a20 -1761> 2014-08-13 16:24:28.076542 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.106:6801/4640 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eef8c0 con 0x8d27380 -1760> 2014-08-13 16:24:28.076559 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.106:6803/4640 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8fc0 con 0x8d270c0 -1759> 2014-08-13 16:24:28.076577 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.110:6801/41204 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eefc40 con 0x8d23b20 -1758> 2014-08-13 16:24:28.076593 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.110:6804/41204 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eedcc0 con 0x8d27bc0 -1757> 2014-08-13 16:24:28.076610 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.104:6801/61726 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8a80 con 0x8b6dee0 -1756> 2014-08-13 16:24:28.076625 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.104:6803/61726 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eead80 con 0x8b6dd80 -1755> 2014-08-13 16:24:28.076642 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6833/53185 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eede80 con 0x7fba7e0 -1754> 2014-08-13 16:24:28.076659 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6835/53185 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeabc0 con 0x7fba680 -1753> 2014-08-13 16:24:28.076675 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6831/16444 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eec980 con 0x8c6f900 -1752> 2014-08-13 16:24:28.076692 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6826/16444 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eec440 con 0x8c6f640 -1751> 2014-08-13 16:24:28.076709 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6827/4273 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eecec0 con 0x8c68b00 -1750> 2014-08-13 16:24:28.076726 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6823/4273 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeec80 con 0x8c68840 -1749> 2014-08-13 16:24:28.076745 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6837/6227 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9c00 con 0x8c3b440 -1748> 2014-08-13 16:24:28.076762 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6844/6227 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8700 con 0x8c3b180 -1747> 2014-08-13 16:24:28.076778 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6833/5235 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9500 con 0x8c68580 -1746> 2014-08-13 16:24:28.076795 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6838/5235 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eefa80 con 0x8c6b2e0 -1745> 2014-08-13 16:24:28.076811 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6835/17224 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eef540 con 0x8c6b020 -1744> 2014-08-13 16:24:28.076832 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6835/17224 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eebd40 con 0x8c6ad60 -1743> 2014-08-13 16:24:28.076849 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6843/8170 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eec0c0 con 0x8b6dc20 -1742> 2014-08-13 16:24:28.076866 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6853/8170 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eea300 con 0x7fb9b80 -1741> 2014-08-13 16:24:28.076888 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6843/20017 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eea680 con 0x8ba11e0 -1740> 2014-08-13 16:24:28.076906 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6847/20017 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeb100 con 0x8ba0f20 -1739> 2014-08-13 16:24:28.076923 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6825/15623 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9a40 con 0x7fbbc80 -1738> 2014-08-13 16:24:28.076941 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6806/15623 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eed240 con 0x7fbbb20 -1737> 2014-08-13 16:24:28.076959 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6829/4534 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeb2c0 con 0x8c39760 -1736> 2014-08-13 16:24:28.076976 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6832/4534 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9dc0 con 0x8c394a0 -1735> 2014-08-13 16:24:28.076995 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6841/55513 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eef700 con 0x7fbb9c0 -1734> 2014-08-13 16:24:28.077012 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6847/55513 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eed080 con 0x7fbb860 -1733> 2014-08-13 16:24:28.077030 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6827/15861 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee9880 con 0x7fb94a0 -1732> 2014-08-13 16:24:28.077046 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6811/15861 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eec280 con 0x7fba520 -1731> 2014-08-13 16:24:28.077064 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6825/4037 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eeb480 con 0x7fbc200 -1730> 2014-08-13 16:24:28.077079 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6814/4037 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8e00 con 0x7fbbf40 -1729> 2014-08-13 16:24:28.077104 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6833/16753 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8380 con 0x8c6aaa0 -1728> 2014-08-13 16:24:28.077122 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6832/16753 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee88c0 con 0x8c6a7e0 -1727> 2014-08-13 16:24:28.077138 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6841/7435 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8eee740 con 0x8bacd00 -1726> 2014-08-13 16:24:28.077155 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6850/7435 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x8ee8c40 con 0x8baca40 -1725> 2014-08-13 16:24:28.077184 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6829/16129 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91063c0 con 0x8ba0c60 -1724> 2014-08-13 16:24:28.077206 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6817/16129 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9103640 con 0x8ba09a0 -1723> 2014-08-13 16:24:28.077227 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6831/52879 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9100c40 con 0x7ffdac0 -1722> 2014-08-13 16:24:28.077248 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6832/52879 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9107700 con 0x7ffe040 -1721> 2014-08-13 16:24:28.077268 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6839/6893 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9100fc0 con 0x8c6f380 -1720> 2014-08-13 16:24:28.077289 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6847/6893 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91016c0 con 0x8c6f0c0 -1719> 2014-08-13 16:24:28.077311 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6841/19195 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9102680 con 0x7fbb700 -1718> 2014-08-13 16:24:28.077332 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6844/19195 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9106580 con 0x7fbb5a0 -1717> 2014-08-13 16:24:28.077352 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6831/4861 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9103b80 con 0x8d27900 -1716> 2014-08-13 16:24:28.077368 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6835/4861 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9100e00 con 0x8d27640 -1715> 2014-08-13 16:24:28.077385 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.114:6823/51844 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x91032c0 con 0x8c6a520 -1714> 2014-08-13 16:24:28.077408 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.114:6803/51844 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9102300 con 0x8c6a260 -1713> 2014-08-13 16:24:28.077425 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.102:6823/15390 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9105cc0 con 0x8d24ba0 -1712> 2014-08-13 16:24:28.077443 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.102:6802/15390 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9105240 con 0x8d248e0 -1711> 2014-08-13 16:24:28.077462 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.108:6801/30527 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9102840 con 0x8d26300 -1710> 2014-08-13 16:24:28.077485 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.108:6801/30527 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9101dc0 con 0x8d26040 -1709> 2014-08-13 16:24:28.077502 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6811/41287 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9100380 con 0x823c620 -1708> 2014-08-13 16:24:28.077524 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6811/41287 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9105780 con 0x823c780 -1707> 2014-08-13 16:24:28.077548 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6815/41806 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9103f00 con 0x8d25d80 -1706> 2014-08-13 16:24:28.077569 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6817/41806 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9102f40 con 0x8d140a0 -1705> 2014-08-13 16:24:28.077590 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.1.105:6809/41085 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9107c40 con 0x8d13de0 -1704> 2014-08-13 16:24:28.077610 7fd933fe0700 1 -- 192.168.1.115:0/25220 --> 192.168.100.105:6802/41085 -- osd_ping(ping e194453 stamp 2014-08-13 16:24:28.074057) v2 -- ?+0 0x9107380 con 0x8d13b20 -1703> 2014-08-13 16:24:28.077670 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.0 192.168.100.115:6825/59615 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x81c3100 con 0x8c38f20 -1702> 2014-08-13 16:24:28.077722 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.1.111:6827/11446 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8570fc0 con 0x8c38c60 -1701> 2014-08-13 16:24:28.077758 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.56 192.168.100.111:6811/11446 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8428000 con 0x8c389a0 -1700> 2014-08-13 16:24:28.077784 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.1.111:6831/12680 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8428540 con 0x8c3b9c0 -1699> 2014-08-13 16:24:28.077817 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.63 192.168.100.111:6820/12680 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8576ac0 con 0x8c3b700 -1698> 2014-08-13 16:24:28.077849 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.1.108:6837/6227 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79be3c0 con 0x8c3b440 -1697> 2014-08-13 16:24:28.077879 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.353 192.168.100.108:6844/6227 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bb9c0 con 0x8c3b180 -1696> 2014-08-13 16:24:28.077909 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.1.109:6821/35445 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8348a80 con 0x8c3aec0 -1695> 2014-08-13 16:24:28.077939 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.313 192.168.100.109:6850/35445 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8349500 con 0x8c3ac00 -1694> 2014-08-13 16:24:28.077965 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.1.110:6815/43857 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834b9c0 con 0x8c3a940 -1693> 2014-08-13 16:24:28.078004 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.318 192.168.100.110:6840/43857 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834af40 con 0x8c3a680 -1692> 2014-08-13 16:24:28.078029 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.1.107:6814/53408 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9106900 con 0x8c3a3c0 -1691> 2014-08-13 16:24:28.078057 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.15 192.168.100.107:6819/53408 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775c0c0 con 0x8c3a100 -1690> 2014-08-13 16:24:28.078086 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.1.107:6822/54786 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8ee8000 con 0x8c3d6a0 -1689> 2014-08-13 16:24:28.078112 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.21 192.168.100.107:6838/54786 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9101500 con 0x8c3d3e0 -1688> 2014-08-13 16:24:28.078132 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.1.109:6813/32413 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9110fc0 con 0x8c3d120 -1687> 2014-08-13 16:24:28.078178 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.310 192.168.100.109:6833/32413 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8348000 con 0x8c3ce60 -1686> 2014-08-13 16:24:28.078230 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.1.111:6829/12102 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899a140 con 0x8c3cba0 -1685> 2014-08-13 16:24:28.078255 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.47 192.168.100.111:6815/12102 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899e900 con 0x8c3c8e0 -1684> 2014-08-13 16:24:28.078279 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.1.111:6837/14923 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899dcc0 con 0x8c3c620 -1683> 2014-08-13 16:24:28.078310 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.54 192.168.100.111:6833/14923 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8428a80 con 0x8c3c360 -1682> 2014-08-13 16:24:28.078334 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.1.103:6812/37658 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9101880 con 0x8c3c0a0 -1681> 2014-08-13 16:24:28.078356 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.17 192.168.100.103:6818/37658 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775c7c0 con 0x8c3bde0 -1680> 2014-08-13 16:24:28.078376 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.1.113:6801/14161 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834aa00 con 0x8c3e880 -1679> 2014-08-13 16:24:28.078416 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.315 192.168.100.113:6801/14161 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834a4c0 con 0x8c3e5c0 -1678> 2014-08-13 16:24:28.078441 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.1.106:6805/5112 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7758a80 con 0x8c3e300 -1677> 2014-08-13 16:24:28.078466 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.327 192.168.100.106:6813/5112 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834c980 con 0x8c618c0 -1676> 2014-08-13 16:24:28.078485 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.1.112:6823/5265 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8576040 con 0x8c6fe80 -1675> 2014-08-13 16:24:28.078509 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.59 192.168.100.112:6804/5265 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8577a80 con 0x8c6fbc0 -1674> 2014-08-13 16:24:28.078532 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.1.102:6831/16444 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8408000 con 0x8c6f900 -1673> 2014-08-13 16:24:28.078558 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.351 192.168.100.102:6826/16444 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bee40 con 0x8c6f640 -1672> 2014-08-13 16:24:28.078581 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.1.108:6839/6893 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840e3c0 con 0x8c6f380 -1671> 2014-08-13 16:24:28.078613 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.376 192.168.100.108:6847/6893 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840c980 con 0x8c6f0c0 -1670> 2014-08-13 16:24:28.078637 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.1.112:6843/9607 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9287000 con 0x8d223c0 -1669> 2014-08-13 16:24:28.078662 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.50 192.168.100.112:6849/9607 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899a680 con 0x8d22100 -1668> 2014-08-13 16:24:28.078686 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.1.113:6809/15284 6 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (3773321266 0 0) 0x842ee40 con 0x8d218c0 -1667> 2014-08-13 16:24:28.078717 7fd93cbee700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6808/15284 -- osd_map(194451..194453 src has 185139..194453) v3 -- ?+0 0x859a1c0 con 0x4fdb2e0 -1666> 2014-08-13 16:24:28.078754 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.289 192.168.100.113:6828/15284 6 ==== osd_ping(ping_reply e194450 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (3773321266 0 0) 0x842af40 con 0x8d21600 -1665> 2014-08-13 16:24:28.078860 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.1.109:6807/31286 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842e3c0 con 0x8d21340 -1664> 2014-08-13 16:24:28.078889 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.296 192.168.100.109:6815/31286 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842d940 con 0x8d21080 -1663> 2014-08-13 16:24:28.078913 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.1.109:6805/31016 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9116740 con 0x8d20dc0 -1662> 2014-08-13 16:24:28.078937 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.307 192.168.100.109:6811/31016 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842f8c0 con 0x8d20b00 -1661> 2014-08-13 16:24:28.078961 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.1.109:6815/33435 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842b480 con 0x8d20840 -1660> 2014-08-13 16:24:28.078984 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.287 192.168.100.109:6839/33435 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9110540 con 0x8d25960 -1659> 2014-08-13 16:24:28.079007 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.1.101:6807/6812 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9111a40 con 0x8d256a0 -1658> 2014-08-13 16:24:28.079031 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.306 192.168.100.101:6805/6812 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9116c80 con 0x8d253e0 -1657> 2014-08-13 16:24:28.079055 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.1.101:6815/23703 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9116200 con 0x8d25120 -1656> 2014-08-13 16:24:28.079078 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.309 192.168.100.101:6825/23703 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9111500 con 0x8d24e60 -1655> 2014-08-13 16:24:28.079100 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.1.102:6823/15390 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x83c8fc0 con 0x8d24ba0 -1654> 2014-08-13 16:24:28.079125 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.381 192.168.100.102:6802/15390 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80d9500 con 0x8d248e0 -1653> 2014-08-13 16:24:28.079148 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.1.106:6819/9142 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed0fc0 con 0x8d24620 -1652> 2014-08-13 16:24:28.079180 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.341 192.168.100.106:6846/9142 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b8a80 con 0x8d24360 -1651> 2014-08-13 16:24:28.079204 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.1.110:6807/42024 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed0000 con 0x8d240a0 -1650> 2014-08-13 16:24:28.079234 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.332 192.168.100.110:6825/42024 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834de80 con 0x8d23de0 -1649> 2014-08-13 16:24:28.079258 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.1.110:6801/41204 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79ba4c0 con 0x8d23b20 -1648> 2014-08-13 16:24:28.079287 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.348 192.168.100.110:6804/41204 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b9a40 con 0x8d27bc0 -1647> 2014-08-13 16:24:28.079311 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.1.108:6831/4861 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840d940 con 0x8d27900 -1646> 2014-08-13 16:24:28.079335 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.379 192.168.100.108:6835/4861 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840ee40 con 0x8d27640 -1645> 2014-08-13 16:24:28.079358 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.1.106:6801/4640 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed78c0 con 0x8d27380 -1644> 2014-08-13 16:24:28.079381 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.347 192.168.100.106:6803/4640 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b9500 con 0x8d270c0 -1643> 2014-08-13 16:24:28.079404 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.1.109:6817/33855 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842de80 con 0x8d26e00 -1642> 2014-08-13 16:24:28.079428 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.293 192.168.100.109:6844/33855 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842c440 con 0x8d26b40 -1641> 2014-08-13 16:24:28.079451 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.1.106:6815/25893 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834e900 con 0x8d26880 -1640> 2014-08-13 16:24:28.079475 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.338 192.168.100.106:6802/25893 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834e3c0 con 0x8d265c0 -1639> 2014-08-13 16:24:28.079504 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.1.108:6801/30527 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840f8c0 con 0x8d26300 -1638> 2014-08-13 16:24:28.079528 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.382 192.168.100.108:6801/30527 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x83c8a80 con 0x8d26040 -1637> 2014-08-13 16:24:28.079551 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.1.105:6815/41806 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7759dc0 con 0x8d25d80 -1636> 2014-08-13 16:24:28.079575 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.396 192.168.100.105:6817/41806 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x87e0000 con 0x8d140a0 -1635> 2014-08-13 16:24:28.079599 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.1.105:6809/41085 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7d039c0 con 0x8d13de0 -1634> 2014-08-13 16:24:28.079629 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.397 192.168.100.105:6802/41085 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9284980 con 0x8d13b20 -1633> 2014-08-13 16:24:28.079658 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.13 192.168.1.115:6805/58748 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80d96c0 con 0x8d12100 -1632> 2014-08-13 16:24:28.079684 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.13 192.168.100.115:6810/58748 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80de040 con 0x823c4c0 -1631> 2014-08-13 16:24:28.079707 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.395 192.168.1.105:6811/41287 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x91039c0 con 0x823c620 -1630> 2014-08-13 16:24:28.079735 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.395 192.168.100.105:6811/41287 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9107000 con 0x823c780 -1629> 2014-08-13 16:24:28.079760 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.26 192.168.1.115:6803/58522 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80dc600 con 0x823c8e0 -1628> 2014-08-13 16:24:28.079785 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.26 192.168.100.115:6804/58522 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80dabc0 con 0x823ca40 -1627> 2014-08-13 16:24:28.079808 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.30 192.168.1.107:6824/55491 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80dad80 con 0x823f4e0 -1626> 2014-08-13 16:24:28.079832 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.30 192.168.100.107:6841/55491 1 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80d88c0 con 0x823f640 -1625> 2014-08-13 16:24:28.079860 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.1.107:6841/56287 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775d400 con 0x78844c0 -1624> 2014-08-13 16:24:28.079885 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.3 192.168.100.107:6845/56287 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x77596c0 con 0x7884200 -1623> 2014-08-13 16:24:28.079909 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.1.103:6830/40192 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775c280 con 0x78848e0 -1622> 2014-08-13 16:24:28.079932 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.5 192.168.100.103:6847/40192 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9106e40 con 0x7884620 -1621> 2014-08-13 16:24:28.079955 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.1.115:6816/60573 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9100a80 con 0x794c360 -1620> 2014-08-13 16:24:28.079979 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.10 192.168.100.115:6834/60573 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x81c0700 con 0x794e300 -1619> 2014-08-13 16:24:28.080005 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.1.103:6814/37929 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9103800 con 0x794e040 -1618> 2014-08-13 16:24:28.080030 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.11 192.168.100.103:6821/37929 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775e900 con 0x794dd80 -1617> 2014-08-13 16:24:28.080053 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.1.113:6821/19173 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842b9c0 con 0x7ffd960 -1616> 2014-08-13 16:24:28.080077 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.286 192.168.100.113:6849/19173 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842cec0 con 0x7ffdc20 -1615> 2014-08-13 16:24:28.080100 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.1.114:6831/52879 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8409500 con 0x7ffdac0 -1614> 2014-08-13 16:24:28.080123 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.375 192.168.100.114:6832/52879 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840cec0 con 0x7ffe040 -1613> 2014-08-13 16:24:28.080147 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.1.113:6815/16727 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842bf00 con 0x7ffdee0 -1612> 2014-08-13 16:24:28.080178 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.301 192.168.100.113:6840/16727 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842d400 con 0x7ffdd80 -1611> 2014-08-13 16:24:28.080203 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.1.111:6835/14316 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899bb80 con 0x8b68c60 -1610> 2014-08-13 16:24:28.080227 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.58 192.168.100.111:6829/14316 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8429a40 con 0x8b68b00 -1609> 2014-08-13 16:24:28.080250 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.1.111:6841/16566 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8570000 con 0x8b689a0 -1608> 2014-08-13 16:24:28.080273 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.61 192.168.100.111:6839/16566 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x85755c0 con 0x8b68840 -1607> 2014-08-13 16:24:28.080296 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.1.104:6803/61960 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed24c0 con 0x8b6e1a0 -1606> 2014-08-13 16:24:28.080320 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.343 192.168.100.104:6810/61960 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b8000 con 0x8b6e040 -1605> 2014-08-13 16:24:28.080343 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.1.104:6801/61726 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b8fc0 con 0x8b6dee0 -1604> 2014-08-13 16:24:28.080366 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.349 192.168.100.104:6803/61726 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79baf40 con 0x8b6dd80 -1603> 2014-08-13 16:24:28.080394 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.1.108:6843/8170 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bc980 con 0x8b6dc20 -1602> 2014-08-13 16:24:28.080419 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.362 192.168.100.108:6853/8170 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bf380 con 0x7fb9b80 -1601> 2014-08-13 16:24:28.080441 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.1.113:6811/15731 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842f380 con 0x7fb9a20 -1600> 2014-08-13 16:24:28.080464 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.302 192.168.100.113:6834/15731 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842e900 con 0x7fb98c0 -1599> 2014-08-13 16:24:28.080488 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.1.104:6815/65209 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834d940 con 0x7fb9760 -1598> 2014-08-13 16:24:28.080511 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.324 192.168.100.104:6840/65209 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834d400 con 0x7fb9600 -1597> 2014-08-13 16:24:28.080533 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.1.102:6827/15861 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bf8c0 con 0x7fb94a0 -1596> 2014-08-13 16:24:28.080557 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.368 192.168.100.102:6811/15861 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x80dc440 con 0x7fba520 -1595> 2014-08-13 16:24:28.080579 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.1.111:6823/10782 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8998fc0 con 0x7fba3c0 -1594> 2014-08-13 16:24:28.080603 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.44 192.168.100.111:6802/10782 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8570a80 con 0x7fba260 -1593> 2014-08-13 16:24:28.080626 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.1.112:6839/8300 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8575b00 con 0x7fba100 -1592> 2014-08-13 16:24:28.080654 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.48 192.168.100.112:6842/8300 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899c980 con 0x7fb9fa0 -1591> 2014-08-13 16:24:28.080677 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.1.112:6825/5505 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8428fc0 con 0x7fb9e40 -1590> 2014-08-13 16:24:28.080700 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.62 192.168.100.112:6810/5505 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8577540 con 0x7fb9ce0 -1589> 2014-08-13 16:24:28.080727 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.1.111:6843/17345 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842a4c0 con 0x7fbb020 -1588> 2014-08-13 16:24:28.080755 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.65 192.168.100.111:6842/17345 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8577000 con 0x7fbaec0 -1587> 2014-08-13 16:24:28.080780 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.1.101:6803/22303 6 ==== osd_ping(ping_reply e194452 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (4141311469 0 0) 0x9117700 con 0x7fbad60 -1586> 2014-08-13 16:24:28.080806 7fd93cbee700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.101:6802/22303 -- osd_map(194453..194453 src has 185139..194453) v3 -- ?+0 0x859b840 con 0x7d2d280 -1585> 2014-08-13 16:24:28.080834 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.314 192.168.100.101:6803/22303 6 ==== osd_ping(ping_reply e194452 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (4141311469 0 0) 0x834bf00 con 0x7fbac00 -1584> 2014-08-13 16:24:28.080989 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.1.101:6821/25633 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9282f40 con 0x7fbaaa0 -1583> 2014-08-13 16:24:28.081018 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.291 192.168.100.101:6841/25633 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9110a80 con 0x7fba940 -1582> 2014-08-13 16:24:28.081044 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.1.114:6833/53185 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b9f80 con 0x7fba7e0 -1581> 2014-08-13 16:24:28.081068 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.350 192.168.100.114:6835/53185 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79baa00 con 0x7fba680 -1580> 2014-08-13 16:24:28.081091 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.1.102:6825/15623 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840b480 con 0x7fbbc80 -1579> 2014-08-13 16:24:28.081120 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.365 192.168.100.102:6806/15623 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79be900 con 0x7fbbb20 -1578> 2014-08-13 16:24:28.081143 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.1.114:6841/55513 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bd400 con 0x7fbb9c0 -1577> 2014-08-13 16:24:28.081173 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.367 192.168.100.114:6847/55513 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bcec0 con 0x7fbb860 -1576> 2014-08-13 16:24:28.081198 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.1.102:6841/19195 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840f380 con 0x7fbb700 -1575> 2014-08-13 16:24:28.081221 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.378 192.168.100.102:6844/19195 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840d400 con 0x7fbb5a0 -1574> 2014-08-13 16:24:28.081244 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.1.101:6823/26295 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8429f80 con 0x7fbb440 -1573> 2014-08-13 16:24:28.081267 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.285 192.168.100.101:6845/26295 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842aa00 con 0x7fbb2e0 -1572> 2014-08-13 16:24:28.081290 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.1.101:6817/24270 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x91171c0 con 0x7fbb180 -1571> 2014-08-13 16:24:28.081313 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.311 192.168.100.101:6831/24270 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8348fc0 con 0x7fbca40 -1570> 2014-08-13 16:24:28.081336 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.1.113:6817/17521 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8348540 con 0x7fbc780 -1569> 2014-08-13 16:24:28.081359 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.312 192.168.100.113:6843/17521 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8349a40 con 0x7fbc4c0 -1568> 2014-08-13 16:24:28.081381 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.1.108:6825/4037 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840b9c0 con 0x7fbc200 -1567> 2014-08-13 16:24:28.081404 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.369 192.168.100.108:6814/4037 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x83c8540 con 0x7fbbf40 -1566> 2014-08-13 16:24:28.081427 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.1.103:6816/38269 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9104600 con 0x8babde0 -1565> 2014-08-13 16:24:28.081450 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.20 192.168.100.103:6827/38269 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x775ee40 con 0x8babb20 -1564> 2014-08-13 16:24:28.081473 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.1.103:6801/36920 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9104280 con 0x8bab860 -1563> 2014-08-13 16:24:28.081495 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.25 192.168.100.103:6802/36920 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9101f80 con 0x8bab5a0 -1562> 2014-08-13 16:24:28.081518 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.1.109:6803/30768 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x842c980 con 0x8bab2e0 -1561> 2014-08-13 16:24:28.081542 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.303 192.168.100.109:6807/30768 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8349f80 con 0x8bab020 -1560> 2014-08-13 16:24:28.081565 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.1.108:6841/7435 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840aa00 con 0x8bacd00 -1559> 2014-08-13 16:24:28.081588 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.373 192.168.100.108:6850/7435 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840e900 con 0x8baca40 -1558> 2014-08-13 16:24:28.081611 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.1.106:6803/4876 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834c440 con 0x8bac780 -1557> 2014-08-13 16:24:28.081634 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.330 192.168.100.106:6807/4876 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834ee40 con 0x8bac4c0 -1556> 2014-08-13 16:24:28.081661 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.1.104:6817/837 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9113d40 con 0x8bac200 -1555> 2014-08-13 16:24:28.081684 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.331 192.168.100.104:6843/837 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed0540 con 0x8babf40 -1554> 2014-08-13 16:24:28.081707 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.1.104:6807/62463 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed1500 con 0x8bae040 -1553> 2014-08-13 16:24:28.081730 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.340 192.168.100.104:6822/62463 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed0a80 con 0x8badd80 -1552> 2014-08-13 16:24:28.081753 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.1.107:6801/52669 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x899b100 con 0x8badac0 -1551> 2014-08-13 16:24:28.081779 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.24 192.168.100.107:6803/52669 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9105080 con 0x8bad800 -1550> 2014-08-13 16:24:28.081802 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.1.112:6829/6040 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8570540 con 0x8bad540 -1549> 2014-08-13 16:24:28.081826 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.45 192.168.100.112:6823/6040 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8429500 con 0x8bad280 -1548> 2014-08-13 16:24:28.081848 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.1.112:6835/6966 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8998e00 con 0x8bafe80 -1547> 2014-08-13 16:24:28.081872 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.55 192.168.100.112:6836/6966 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8576580 con 0x8bafbc0 -1546> 2014-08-13 16:24:28.081894 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.1.110:6819/45268 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed7380 con 0x8baf900 -1545> 2014-08-13 16:24:28.081917 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.339 192.168.100.110:6846/45268 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed1a40 con 0x8baf640 -1544> 2014-08-13 16:24:28.081939 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.1.110:6803/41438 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed1f80 con 0x8baf380 -1543> 2014-08-13 16:24:28.081967 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.342 192.168.100.110:6810/41438 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x7ed2a00 con 0x8baf0c0 -1542> 2014-08-13 16:24:28.081999 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.1.103:6832/41157 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9103100 con 0x8baee00 -1541> 2014-08-13 16:24:28.082024 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.31 192.168.100.103:6850/41157 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x91001c0 con 0x8ba1a20 -1540> 2014-08-13 16:24:28.082046 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.1.104:6805/62197 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834f380 con 0x8ba1760 -1539> 2014-08-13 16:24:28.082070 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.337 192.168.100.104:6816/62197 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834f8c0 con 0x8ba14a0 -1538> 2014-08-13 16:24:28.082092 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.1.102:6843/20017 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bd940 con 0x8ba11e0 -1537> 2014-08-13 16:24:28.082114 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.363 192.168.100.102:6847/20017 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840bf00 con 0x8ba0f20 -1536> 2014-08-13 16:24:28.082139 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.1.102:6829/16129 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8408540 con 0x8ba0c60 -1535> 2014-08-13 16:24:28.082168 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.374 192.168.100.102:6817/16129 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8409f80 con 0x8ba09a0 -1534> 2014-08-13 16:24:28.082192 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.1.103:6808/37391 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8999180 con 0x8c69600 -1533> 2014-08-13 16:24:28.082221 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.28 192.168.100.103:6813/37391 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9100700 con 0x8c69340 -1532> 2014-08-13 16:24:28.082245 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.1.101:6809/22782 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8574600 con 0x8c69080 -1531> 2014-08-13 16:24:28.082268 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.304 192.168.100.101:6812/22782 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x9110000 con 0x8c68dc0 -1530> 2014-08-13 16:24:28.082291 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.1.108:6827/4273 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bde80 con 0x8c68b00 -1529> 2014-08-13 16:24:28.082315 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.352 192.168.100.108:6823/4273 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8409a40 con 0x8c68840 -1528> 2014-08-13 16:24:28.082337 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.1.108:6833/5235 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bb480 con 0x8c68580 -1527> 2014-08-13 16:24:28.082361 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.356 192.168.100.108:6838/5235 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840c440 con 0x8c6b2e0 -1526> 2014-08-13 16:24:28.082383 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.1.102:6835/17224 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bc440 con 0x8c6b020 -1525> 2014-08-13 16:24:28.082406 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.357 192.168.100.102:6835/17224 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79bbf00 con 0x8c6ad60 -1524> 2014-08-13 16:24:28.082429 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.1.102:6833/16753 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8408a80 con 0x8c6aaa0 -1523> 2014-08-13 16:24:28.082452 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.371 192.168.100.102:6832/16753 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x8408fc0 con 0x8c6a7e0 -1522> 2014-08-13 16:24:28.082475 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.1.114:6823/51844 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x83c8000 con 0x8c6a520 -1521> 2014-08-13 16:24:28.082497 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.380 192.168.100.114:6803/51844 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840de80 con 0x8c6a260 -1520> 2014-08-13 16:24:28.082519 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.1.106:6821/10050 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834b480 con 0x8c69fa0 -1519> 2014-08-13 16:24:28.082542 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.320 192.168.100.106:6849/10050 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x834cec0 con 0x8c39fa0 -1518> 2014-08-13 16:24:28.082565 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.1.106:6813/6757 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x79b8540 con 0x8c39ce0 -1517> 2014-08-13 16:24:28.082591 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.344 192.168.100.106:6837/6757 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x83c9500 con 0x8c39a20 -1516> 2014-08-13 16:24:28.082614 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.1.108:6829/4534 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840a4c0 con 0x8c39760 -1515> 2014-08-13 16:24:28.082637 7fd93cbee700 1 -- 192.168.1.115:0/25220 <== osd.366 192.168.100.108:6832/4534 6 ==== osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.074057) v2 ==== 47+0+0 (2389706080 0 0) 0x840af40 con 0x8c394a0 -1514> 2014-08-13 16:24:28.089550 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 11 ==== pg_notify(22.619(17) epoch 194453) v5 ==== 1665+0+0 (137763550 0 0) 0x8c8f540 con 0x7d2d280 -1513> 2014-08-13 16:24:28.089599 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089344, event: header_read, request: pg_notify(22.619(17) epoch 194453) v5 -1512> 2014-08-13 16:24:28.089615 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089347, event: throttled, request: pg_notify(22.619(17) epoch 194453) v5 -1511> 2014-08-13 16:24:28.089622 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089468, event: all_read, request: pg_notify(22.619(17) epoch 194453) v5 -1510> 2014-08-13 16:24:28.089628 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089593, event: dispatched, request: pg_notify(22.619(17) epoch 194453) v5 -1509> 2014-08-13 16:24:28.089639 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089639, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194453) v5 -1508> 2014-08-13 16:24:28.089667 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089667, event: started, request: pg_notify(22.619(17) epoch 194453) v5 -1507> 2014-08-13 16:24:28.089705 7fd93dff0700 5 -- op tracker -- , seq: 1124, time: 2014-08-13 16:24:28.089705, event: done, request: pg_notify(22.619(17) epoch 194453) v5 -1506> 2014-08-13 16:24:28.182656 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.289 192.168.100.113:0/15284 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:28.182629) v2 ==== 47+0+0 (3022024300 0 0) 0x80dabc0 con 0x7caf380 -1505> 2014-08-13 16:24:28.182711 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.182629) v2 -- ?+0 0x87e0380 con 0x7caf380 -1504> 2014-08-13 16:24:28.182697 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.289 192.168.100.113:0/15284 7 ==== osd_ping(ping e194450 stamp 2014-08-13 16:24:28.182629) v2 ==== 47+0+0 (3022024300 0 0) 0x80d9500 con 0x7d289a0 -1503> 2014-08-13 16:24:28.182745 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.182629) v2 -- ?+0 0x87e7700 con 0x7d289a0 -1502> 2014-08-13 16:24:28.197268 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.323 192.168.100.106:0/8118 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.196690) v2 ==== 47+0+0 (891257747 0 0) 0x842bd40 con 0x7d2c360 -1501> 2014-08-13 16:24:28.197319 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.196690) v2 -- ?+0 0x80dabc0 con 0x7d2c360 -1500> 2014-08-13 16:24:28.197268 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.323 192.168.100.106:0/8118 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.196690) v2 ==== 47+0+0 (891257747 0 0) 0x9117540 con 0x7d2ac00 -1499> 2014-08-13 16:24:28.197390 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.196690) v2 -- ?+0 0x80d9500 con 0x7d2ac00 -1498> 2014-08-13 16:24:28.223365 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.222953) v2 ==== 47+0+0 (3349605053 0 0) 0x7d04d00 con 0x7eda680 -1497> 2014-08-13 16:24:28.223375 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.222953) v2 ==== 47+0+0 (3349605053 0 0) 0x7d01880 con 0x7edc620 -1496> 2014-08-13 16:24:28.223417 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.222953) v2 -- ?+0 0x9117540 con 0x7eda680 -1495> 2014-08-13 16:24:28.223485 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.222953) v2 -- ?+0 0x842bd40 con 0x7edc620 -1494> 2014-08-13 16:24:28.249660 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.371 192.168.100.102:0/16753 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.249532) v2 ==== 47+0+0 (1918794053 0 0) 0x9287c40 con 0x7ed8dc0 -1493> 2014-08-13 16:24:28.249710 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.249532) v2 -- ?+0 0x7d01880 con 0x7ed8dc0 -1492> 2014-08-13 16:24:28.249696 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.371 192.168.100.102:0/16753 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.249532) v2 ==== 47+0+0 (1918794053 0 0) 0x8c88a80 con 0x7ebee00 -1491> 2014-08-13 16:24:28.249838 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.249532) v2 -- ?+0 0x7d04d00 con 0x7ebee00 -1490> 2014-08-13 16:24:28.324949 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.320 192.168.1.106:0/10050 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.324457) v2 ==== 47+0+0 (3841394754 0 0) 0x840b800 con 0x7ab9ce0 -1489> 2014-08-13 16:24:28.324963 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.320 192.168.1.106:0/10050 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.324457) v2 ==== 47+0+0 (3841394754 0 0) 0x840f1c0 con 0x7ab9a20 -1488> 2014-08-13 16:24:28.324996 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.324457) v2 -- ?+0 0x9287c40 con 0x7ab9ce0 -1487> 2014-08-13 16:24:28.325049 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.324457) v2 -- ?+0 0x8c88a80 con 0x7ab9a20 -1486> 2014-08-13 16:24:28.368057 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.6 192.168.1.107:0/53141 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.367782) v2 ==== 47+0+0 (3954793129 0 0) 0x8570380 con 0x7cae880 -1485> 2014-08-13 16:24:28.368108 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.367782) v2 -- ?+0 0x840f1c0 con 0x7cae880 -1484> 2014-08-13 16:24:28.368140 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.6 192.168.1.107:0/53141 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.367782) v2 ==== 47+0+0 (3954793129 0 0) 0x8571340 con 0x7cae040 -1483> 2014-08-13 16:24:28.368227 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.107:0/53141 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.367782) v2 -- ?+0 0x840b800 con 0x7cae040 -1482> 2014-08-13 16:24:28.518256 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.346 192.168.100.104:0/2308 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.518230) v2 ==== 47+0+0 (4015383729 0 0) 0x840c7c0 con 0x7d2a260 -1481> 2014-08-13 16:24:28.518309 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.518230) v2 -- ?+0 0x8571340 con 0x7d2a260 -1480> 2014-08-13 16:24:28.518354 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.346 192.168.100.104:0/2308 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.518230) v2 ==== 47+0+0 (4015383729 0 0) 0x840a840 con 0x7d28160 -1479> 2014-08-13 16:24:28.518397 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.518230) v2 -- ?+0 0x8570380 con 0x7d28160 -1478> 2014-08-13 16:24:28.536129 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.304 192.168.100.101:0/22782 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.535822) v2 ==== 47+0+0 (2529431048 0 0) 0x87e3800 con 0x7edb180 -1477> 2014-08-13 16:24:28.536148 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.304 192.168.100.101:0/22782 10 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.535822) v2 ==== 47+0+0 (2529431048 0 0) 0x87e4600 con 0x7ed82c0 -1476> 2014-08-13 16:24:28.536188 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.535822) v2 -- ?+0 0x840a840 con 0x7edb180 -1475> 2014-08-13 16:24:28.536241 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.535822) v2 -- ?+0 0x840c7c0 con 0x7ed82c0 -1474> 2014-08-13 16:24:28.547523 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 36 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (284388641 0 0) 0x86d8b00 con 0x5112520 -1473> 2014-08-13 16:24:28.547578 7fd93dff0700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547038, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1472> 2014-08-13 16:24:28.547598 7fd93dff0700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547042, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1471> 2014-08-13 16:24:28.547612 7fd93dff0700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547490, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1470> 2014-08-13 16:24:28.547625 7fd93dff0700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547569, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1469> 2014-08-13 16:24:28.547640 7fd93dff0700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547640, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1468> 2014-08-13 16:24:28.547755 7fd9367e4700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547754, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1467> 2014-08-13 16:24:28.547802 7fd9367e4700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547801, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1466> 2014-08-13 16:24:28.547825 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8e8d780 con 0x5112520 -1465> 2014-08-13 16:24:28.547859 7fd9367e4700 5 -- op tracker -- , seq: 1125, time: 2014-08-13 16:24:28.547858, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1464> 2014-08-13 16:24:28.549126 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.367 192.168.1.114:6840/55513 37 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2174926468 0 0) 0x7eccd00 con 0x5112520 -1463> 2014-08-13 16:24:28.549154 7fd93dff0700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549027, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1462> 2014-08-13 16:24:28.549184 7fd93dff0700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549029, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1461> 2014-08-13 16:24:28.549198 7fd93dff0700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549105, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1460> 2014-08-13 16:24:28.549208 7fd93dff0700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549151, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1459> 2014-08-13 16:24:28.549223 7fd93dff0700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549222, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1458> 2014-08-13 16:24:28.549318 7fd9371e5700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549318, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1457> 2014-08-13 16:24:28.549361 7fd9371e5700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549360, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1456> 2014-08-13 16:24:28.549380 7fd9371e5700 5 -- op tracker -- , seq: 1126, time: 2014-08-13 16:24:28.549380, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1455> 2014-08-13 16:24:28.608912 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.285 192.168.100.101:0/26295 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.608717) v2 ==== 47+0+0 (1767416385 0 0) 0x840f380 con 0x7edbde0 -1454> 2014-08-13 16:24:28.608961 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.608717) v2 -- ?+0 0x87e4600 con 0x7edbde0 -1453> 2014-08-13 16:24:28.608971 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.285 192.168.100.101:0/26295 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.608717) v2 ==== 47+0+0 (1767416385 0 0) 0x80d9c00 con 0x7ffeb40 -1452> 2014-08-13 16:24:28.609014 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.608717) v2 -- ?+0 0x87e3800 con 0x7ffeb40 -1451> 2014-08-13 16:24:28.652636 7fd94ac49700 5 osd.4 194453 tick -1450> 2014-08-13 16:24:28.680079 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 20 ==== osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+11541 (2073344252 0 1473524713) 0x86de300 con 0x4fdb2e0 -1449> 2014-08-13 16:24:28.680132 7fd93dff0700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.679869, event: header_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1448> 2014-08-13 16:24:28.680149 7fd93dff0700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.679873, event: throttled, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1447> 2014-08-13 16:24:28.680159 7fd93dff0700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680010, event: all_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1446> 2014-08-13 16:24:28.680168 7fd93dff0700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680126, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1445> 2014-08-13 16:24:28.680196 7fd93dff0700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680196, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1444> 2014-08-13 16:24:28.680305 7fd9367e4700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680305, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1443> 2014-08-13 16:24:28.680346 7fd9367e4700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680346, event: started, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1442> 2014-08-13 16:24:28.680514 7fd9367e4700 5 -- op tracker -- , seq: 1127, time: 2014-08-13 16:24:28.680514, event: done, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1441> 2014-08-13 16:24:28.680633 7fd9353e2700 2 osd.4 pg_epoch: 194453 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub osd.4 has 25 items -1440> 2014-08-13 16:24:28.680698 7fd9353e2700 2 osd.4 pg_epoch: 194453 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 50 has 25 items -1439> 2014-08-13 16:24:28.680716 7fd9353e2700 2 osd.4 pg_epoch: 194453 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 289 has 25 items -1438> 2014-08-13 16:24:28.681602 7fd9353e2700 2 osd.4 pg_epoch: 194453 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] -1437> 2014-08-13 16:24:28.682706 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- replica scrub(pg: 22.6bd,from:0'0,to:194215'504,epoch:194453,start:338abebd//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -- ?+0 0x8599d40 -1436> 2014-08-13 16:24:28.682767 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- replica scrub(pg: 22.6bd,from:0'0,to:194215'504,epoch:194453,start:338abebd//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -- ?+0 0x859af40 -1435> 2014-08-13 16:24:28.771963 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.337 192.168.1.104:0/62197 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.771906) v2 ==== 47+0+0 (2369749508 0 0) 0x840e900 con 0x7ebd6a0 -1434> 2014-08-13 16:24:28.772021 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.771906) v2 -- ?+0 0x840f380 con 0x7ebd6a0 -1433> 2014-08-13 16:24:28.772006 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.337 192.168.1.104:0/62197 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.771906) v2 ==== 47+0+0 (2369749508 0 0) 0x840e3c0 con 0x7ebc0a0 -1432> 2014-08-13 16:24:28.772117 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.771906) v2 -- ?+0 0x80d9c00 con 0x7ebc0a0 -1431> 2014-08-13 16:24:28.820560 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.45 192.168.1.112:0/6040 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.820535) v2 ==== 47+0+0 (1923338602 0 0) 0x7ed2840 con 0x7edcba0 -1430> 2014-08-13 16:24:28.820613 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.820535) v2 -- ?+0 0x840e900 con 0x7edcba0 -1429> 2014-08-13 16:24:28.820612 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.45 192.168.1.112:0/6040 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.820535) v2 ==== 47+0+0 (1923338602 0 0) 0x9284280 con 0x7edac00 -1428> 2014-08-13 16:24:28.820655 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6040 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.820535) v2 -- ?+0 0x840e3c0 con 0x7edac00 -1427> 2014-08-13 16:24:28.842855 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.294 192.168.1.101:0/23402 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.842496) v2 ==== 47+0+0 (2569219868 0 0) 0x9286740 con 0x7caeb40 -1426> 2014-08-13 16:24:28.842910 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.842496) v2 -- ?+0 0x9284280 con 0x7caeb40 -1425> 2014-08-13 16:24:28.842938 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.294 192.168.1.101:0/23402 9 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.842496) v2 ==== 47+0+0 (2569219868 0 0) 0x9286ac0 con 0x7cae300 -1424> 2014-08-13 16:24:28.842991 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.842496) v2 -- ?+0 0x7ed2840 con 0x7cae300 -1423> 2014-08-13 16:24:28.922591 7fd93f3f2700 1 -- 192.168.100.115:6805/25220 <== mon.2 192.168.100.113:6789/0 29 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d0b180 con 0x4e3dd80 -1422> 2014-08-13 16:24:28.922723 7fd93f3f2700 3 osd.4 194453 handle_osd_map epochs [194454,194454], i have 194453, src has [185139,194454] -1421> 2014-08-13 16:24:28.928535 7fd93f3f2700 0 osd.4 194454 crush map has features 2578087936000, adjusting msgr requires for mons -1420> 2014-08-13 16:24:28.930008 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 19 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf6e40 con 0x7314fc0 -1419> 2014-08-13 16:24:28.931273 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.930983) v2 ==== 47+0+0 (4281118052 0 0) 0x9282d80 con 0x7d29ce0 -1418> 2014-08-13 16:24:28.931321 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.930983) v2 -- ?+0 0x9286ac0 con 0x7d29ce0 -1417> 2014-08-13 16:24:28.931369 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 7 ==== osd_ping(ping e194453 stamp 2014-08-13 16:24:28.930983) v2 ==== 47+0+0 (4281118052 0 0) 0x9284980 con 0x7d2b440 -1416> 2014-08-13 16:24:28.931416 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.930983) v2 -- ?+0 0x9286740 con 0x7d2b440 -1415> 2014-08-13 16:24:28.999653 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:28.999224) v2 ==== 47+0+0 (1482451873 0 0) 0x842d780 con 0x7ebf380 -1414> 2014-08-13 16:24:28.999711 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.999224) v2 -- ?+0 0x9284980 con 0x7ebf380 -1413> 2014-08-13 16:24:28.999703 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:28.999224) v2 ==== 47+0+0 (1482451873 0 0) 0x842c7c0 con 0x7ebfe80 -1412> 2014-08-13 16:24:28.999760 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:28.999224) v2 -- ?+0 0x9282d80 con 0x7ebfe80 -1411> 2014-08-13 16:24:29.033676 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.31 192.168.100.103:0/41157 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.033612) v2 ==== 47+0+0 (3200374660 0 0) 0x8348540 con 0x7ffeca0 -1410> 2014-08-13 16:24:29.033739 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.033612) v2 -- ?+0 0x842d780 con 0x7ffeca0 -1409> 2014-08-13 16:24:29.033686 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.31 192.168.100.103:0/41157 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.033612) v2 ==== 47+0+0 (3200374660 0 0) 0x834ee40 con 0x7edbf40 -1408> 2014-08-13 16:24:29.033807 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.033612) v2 -- ?+0 0x842c7c0 con 0x7edbf40 -1407> 2014-08-13 16:24:29.049066 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.351 192.168.100.102:0/16444 11 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.049128) v2 ==== 47+0+0 (137182472 0 0) 0x9104280 con 0x7ab8420 -1406> 2014-08-13 16:24:29.049085 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.351 192.168.100.102:0/16444 11 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.049128) v2 ==== 47+0+0 (137182472 0 0) 0x834b9c0 con 0x7ab8000 -1405> 2014-08-13 16:24:29.049113 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.049128) v2 -- ?+0 0x8348540 con 0x7ab8420 -1404> 2014-08-13 16:24:29.049151 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.049128) v2 -- ?+0 0x834ee40 con 0x7ab8000 -1403> 2014-08-13 16:24:29.105316 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.51 192.168.1.112:0/6655 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.105296) v2 ==== 47+0+0 (617046418 0 0) 0x9107000 con 0x7edaec0 -1402> 2014-08-13 16:24:29.105341 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.51 192.168.1.112:0/6655 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.105296) v2 ==== 47+0+0 (617046418 0 0) 0x9106900 con 0x7ed8000 -1401> 2014-08-13 16:24:29.105368 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.105296) v2 -- ?+0 0x9104280 con 0x7edaec0 -1400> 2014-08-13 16:24:29.105665 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194453 stamp 2014-08-13 16:24:29.105296) v2 -- ?+0 0x834b9c0 con 0x7ed8000 -1399> 2014-08-13 16:24:29.192080 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1398> 2014-08-13 16:24:29.192134 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 20 ==== pg_notify(22.3e4(17) epoch 194454) v5 ==== 1665+0+0 (1001049558 0 0) 0x899abc0 con 0x7314fc0 -1397> 2014-08-13 16:24:29.192166 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:28.929614, event: header_read, request: pg_notify(22.3e4(17) epoch 194454) v5 -1396> 2014-08-13 16:24:29.192178 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:28.929615, event: throttled, request: pg_notify(22.3e4(17) epoch 194454) v5 -1395> 2014-08-13 16:24:29.192185 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:28.929735, event: all_read, request: pg_notify(22.3e4(17) epoch 194454) v5 -1394> 2014-08-13 16:24:29.192194 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:29.192162, event: dispatched, request: pg_notify(22.3e4(17) epoch 194454) v5 -1393> 2014-08-13 16:24:29.192217 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:29.192216, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194454) v5 -1392> 2014-08-13 16:24:29.192232 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:29.192231, event: started, request: pg_notify(22.3e4(17) epoch 194454) v5 -1391> 2014-08-13 16:24:29.192268 7fd93dff0700 5 -- op tracker -- , seq: 1128, time: 2014-08-13 16:24:29.192268, event: done, request: pg_notify(22.3e4(17) epoch 194454) v5 -1390> 2014-08-13 16:24:29.192312 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d08480 con 0x7316f60 -1389> 2014-08-13 16:24:29.192336 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1388> 2014-08-13 16:24:29.192355 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 13 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7b17740 con 0x8da1760 -1387> 2014-08-13 16:24:29.192375 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1386> 2014-08-13 16:24:29.192390 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859fbc0 con 0x8da5120 -1385> 2014-08-13 16:24:29.192407 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1384> 2014-08-13 16:24:29.192420 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 17 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d0e0c0 con 0x72d9b80 -1383> 2014-08-13 16:24:29.192437 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1382> 2014-08-13 16:24:29.192450 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x33e3600 con 0x72dfa60 -1381> 2014-08-13 16:24:29.192467 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1380> 2014-08-13 16:24:29.192479 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 13 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859bf00 con 0x72de9e0 -1379> 2014-08-13 16:24:29.192496 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1378> 2014-08-13 16:24:29.192507 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf3180 con 0x8e29fa0 -1377> 2014-08-13 16:24:29.192524 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1376> 2014-08-13 16:24:29.192535 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 25 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd0d80 con 0x8e286e0 -1375> 2014-08-13 16:24:29.192553 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1374> 2014-08-13 16:24:29.192565 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 13 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859e9c0 con 0x8e2c360 -1373> 2014-08-13 16:24:29.192582 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1372> 2014-08-13 16:24:29.192593 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd4c80 con 0x7379ce0 -1371> 2014-08-13 16:24:29.192611 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1370> 2014-08-13 16:24:29.192622 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8598000 con 0x73ca680 -1369> 2014-08-13 16:24:29.192639 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1368> 2014-08-13 16:24:29.192650 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 20 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d04800 con 0x73cdc20 -1367> 2014-08-13 16:24:29.192667 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1366> 2014-08-13 16:24:29.192679 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 11 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d06780 con 0x73ccba0 -1365> 2014-08-13 16:24:29.192696 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1364> 2014-08-13 16:24:29.192708 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 11 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d06e40 con 0x7462100 -1363> 2014-08-13 16:24:29.192725 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1362> 2014-08-13 16:24:29.192737 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 20 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7dc40 con 0x7723440 -1361> 2014-08-13 16:24:29.192754 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1360> 2014-08-13 16:24:29.192766 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 9 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d03600 con 0x7d2d120 -1359> 2014-08-13 16:24:29.192783 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1358> 2014-08-13 16:24:29.192794 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 12 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859b840 con 0x7d2d280 -1357> 2014-08-13 16:24:29.192811 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1356> 2014-08-13 16:24:29.192822 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 13 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e78240 con 0x821f900 -1355> 2014-08-13 16:24:29.192839 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1354> 2014-08-13 16:24:29.192851 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 11 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d060c0 con 0x80c7e80 -1353> 2014-08-13 16:24:29.192868 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1352> 2014-08-13 16:24:29.192880 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 13 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e79d40 con 0x80c7d20 -1351> 2014-08-13 16:24:29.192897 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1350> 2014-08-13 16:24:29.192909 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 4 ==== pg_notify(24.34(43) epoch 194454) v5 ==== 2933+0+0 (3240343846 0 0) 0x8428000 con 0x83002c0 -1349> 2014-08-13 16:24:29.192927 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:28.955934, event: header_read, request: pg_notify(24.34(43) epoch 194454) v5 -1348> 2014-08-13 16:24:29.192938 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:28.955938, event: throttled, request: pg_notify(24.34(43) epoch 194454) v5 -1347> 2014-08-13 16:24:29.192957 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:28.956127, event: all_read, request: pg_notify(24.34(43) epoch 194454) v5 -1346> 2014-08-13 16:24:29.192963 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:29.192924, event: dispatched, request: pg_notify(24.34(43) epoch 194454) v5 -1345> 2014-08-13 16:24:29.192970 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:29.192970, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194454) v5 -1344> 2014-08-13 16:24:29.192978 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:29.192978, event: started, request: pg_notify(24.34(43) epoch 194454) v5 -1343> 2014-08-13 16:24:29.193001 7fd93dff0700 5 -- op tracker -- , seq: 1129, time: 2014-08-13 16:24:29.193000, event: done, request: pg_notify(24.34(43) epoch 194454) v5 -1342> 2014-08-13 16:24:29.193036 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 21 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd06c0 con 0x4e4b860 -1341> 2014-08-13 16:24:29.193057 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1340> 2014-08-13 16:24:29.193067 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 25 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7d340 con 0x4e4dd80 -1339> 2014-08-13 16:24:29.193081 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1338> 2014-08-13 16:24:29.193089 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 19 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d05a00 con 0x4e4ce60 -1337> 2014-08-13 16:24:29.193103 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1336> 2014-08-13 16:24:29.193114 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 29 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd2f40 con 0x4e4f900 -1335> 2014-08-13 16:24:29.193132 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1334> 2014-08-13 16:24:29.193140 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 30 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7f2c0 con 0x4fd8000 -1333> 2014-08-13 16:24:29.193154 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1332> 2014-08-13 16:24:29.193171 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 24 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8598d80 con 0x4fe1600 -1331> 2014-08-13 16:24:29.193185 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1330> 2014-08-13 16:24:29.193194 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 23 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d08000 con 0x4fe3700 -1329> 2014-08-13 16:24:29.193222 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1328> 2014-08-13 16:24:29.193231 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 26 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf4a40 con 0x4fe40a0 -1327> 2014-08-13 16:24:29.193249 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1326> 2014-08-13 16:24:29.193260 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 26 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d07500 con 0x4fe69e0 -1325> 2014-08-13 16:24:29.193277 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1324> 2014-08-13 16:24:29.193286 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 25 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859f740 con 0x4fda940 -1323> 2014-08-13 16:24:29.193303 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1322> 2014-08-13 16:24:29.193314 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.31 192.168.1.103:6831/41157 16 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf6780 con 0x5089760 -1321> 2014-08-13 16:24:29.193332 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1320> 2014-08-13 16:24:29.193341 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 29 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e78fc0 con 0x50889a0 -1319> 2014-08-13 16:24:29.193355 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1318> 2014-08-13 16:24:29.193366 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 21 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d02f40 con 0x508a680 -1317> 2014-08-13 16:24:29.193392 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1316> 2014-08-13 16:24:29.193401 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 37 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf3600 con 0x508bde0 -1315> 2014-08-13 16:24:29.193418 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1314> 2014-08-13 16:24:29.193426 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 21 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7d7c0 con 0x50ba520 -1313> 2014-08-13 16:24:29.193440 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1312> 2014-08-13 16:24:29.193452 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.51 192.168.1.112:6832/6655 15 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d09f80 con 0x50bc4c0 -1311> 2014-08-13 16:24:29.193466 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1310> 2014-08-13 16:24:29.193478 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7b14c80 con 0x50bfa60 -1309> 2014-08-13 16:24:29.193495 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1308> 2014-08-13 16:24:29.193507 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 22 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x859c140 con 0x510b9c0 -1307> 2014-08-13 16:24:29.193524 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1306> 2014-08-13 16:24:29.193535 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 28 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e79200 con 0x5108b00 -1305> 2014-08-13 16:24:29.193551 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1304> 2014-08-13 16:24:29.193562 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7b13cc0 con 0x510dac0 -1303> 2014-08-13 16:24:29.193579 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1302> 2014-08-13 16:24:29.193591 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 19 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7b12880 con 0x5112260 -1301> 2014-08-13 16:24:29.193607 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1300> 2014-08-13 16:24:29.193618 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.351 192.168.1.102:6830/16444 15 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d0de80 con 0x51148e0 -1299> 2014-08-13 16:24:29.193635 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1298> 2014-08-13 16:24:29.193647 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 17 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf5e80 con 0x5114620 -1297> 2014-08-13 16:24:29.193664 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1296> 2014-08-13 16:24:29.193676 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 24 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf3a80 con 0x5114360 -1295> 2014-08-13 16:24:29.193693 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1294> 2014-08-13 16:24:29.193704 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 14 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7cf4380 con 0x5113de0 -1293> 2014-08-13 16:24:29.193721 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1292> 2014-08-13 16:24:29.193732 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 24 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d05100 con 0x5113b20 -1291> 2014-08-13 16:24:29.193749 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1290> 2014-08-13 16:24:29.193760 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 21 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd6c00 con 0x5113860 -1289> 2014-08-13 16:24:29.193777 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1288> 2014-08-13 16:24:29.193789 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 31 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e79680 con 0x5111fa0 -1287> 2014-08-13 16:24:29.193805 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1286> 2014-08-13 16:24:29.193817 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 39 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7b840 con 0x51135a0 -1285> 2014-08-13 16:24:29.193834 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1284> 2014-08-13 16:24:29.193845 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 31 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd33c0 con 0x5111a20 -1283> 2014-08-13 16:24:29.193862 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1282> 2014-08-13 16:24:29.193873 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 26 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8d02640 con 0x7313860 -1281> 2014-08-13 16:24:29.193890 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1280> 2014-08-13 16:24:29.193901 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 19 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x8bd72c0 con 0x7316720 -1279> 2014-08-13 16:24:29.193918 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1278> 2014-08-13 16:24:29.193929 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 18 ==== osd_map(194454..194454 src has 185139..194454) v3 ==== 1369+0+0 (1813043236 0 0) 0x7e7ca40 con 0x7315ee0 -1277> 2014-08-13 16:24:29.193945 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194454,194454], i have 194454, src has [185139,194454] -1276> 2014-08-13 16:24:29.193959 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 21 ==== pg_notify(22.113(4) epoch 194454) v5 ==== 942+0+0 (4226184638 0 0) 0x899c600 con 0x7314fc0 -1275> 2014-08-13 16:24:29.193978 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:28.930437, event: header_read, request: pg_notify(22.113(4) epoch 194454) v5 -1274> 2014-08-13 16:24:29.193989 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:28.930441, event: throttled, request: pg_notify(22.113(4) epoch 194454) v5 -1273> 2014-08-13 16:24:29.193999 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:28.930541, event: all_read, request: pg_notify(22.113(4) epoch 194454) v5 -1272> 2014-08-13 16:24:29.194016 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:29.193975, event: dispatched, request: pg_notify(22.113(4) epoch 194454) v5 -1271> 2014-08-13 16:24:29.194026 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:29.194026, event: waiting_for_osdmap, request: pg_notify(22.113(4) epoch 194454) v5 -1270> 2014-08-13 16:24:29.194038 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:29.194038, event: started, request: pg_notify(22.113(4) epoch 194454) v5 -1269> 2014-08-13 16:24:29.194310 7fd93dff0700 5 -- op tracker -- , seq: 1130, time: 2014-08-13 16:24:29.194309, event: done, request: pg_notify(22.113(4) epoch 194454) v5 -1268> 2014-08-13 16:24:29.194359 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 15 ==== pg_notify(22.22c(4) epoch 194454) v5 ==== 942+0+0 (2673922792 0 0) 0x8429c00 con 0x7316f60 -1267> 2014-08-13 16:24:29.194386 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:28.935662, event: header_read, request: pg_notify(22.22c(4) epoch 194454) v5 -1266> 2014-08-13 16:24:29.194398 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:28.935664, event: throttled, request: pg_notify(22.22c(4) epoch 194454) v5 -1265> 2014-08-13 16:24:29.194408 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:28.935735, event: all_read, request: pg_notify(22.22c(4) epoch 194454) v5 -1264> 2014-08-13 16:24:29.194417 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:29.194383, event: dispatched, request: pg_notify(22.22c(4) epoch 194454) v5 -1263> 2014-08-13 16:24:29.194427 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:29.194427, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194454) v5 -1262> 2014-08-13 16:24:29.194439 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:29.194439, event: started, request: pg_notify(22.22c(4) epoch 194454) v5 -1261> 2014-08-13 16:24:29.194468 7fd93dff0700 5 -- op tracker -- , seq: 1131, time: 2014-08-13 16:24:29.194468, event: done, request: pg_notify(22.22c(4) epoch 194454) v5 -1260> 2014-08-13 16:24:29.194540 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 14 ==== pg_notify(22.619(17) epoch 194454) v5 ==== 1665+0+0 (3796848906 0 0) 0x8577000 con 0x8da1760 -1259> 2014-08-13 16:24:29.194561 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:28.936457, event: header_read, request: pg_notify(22.619(17) epoch 194454) v5 -1258> 2014-08-13 16:24:29.194572 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:28.936458, event: throttled, request: pg_notify(22.619(17) epoch 194454) v5 -1257> 2014-08-13 16:24:29.194581 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:28.936566, event: all_read, request: pg_notify(22.619(17) epoch 194454) v5 -1256> 2014-08-13 16:24:29.194590 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:29.194559, event: dispatched, request: pg_notify(22.619(17) epoch 194454) v5 -1255> 2014-08-13 16:24:29.194600 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:29.194600, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194454) v5 -1254> 2014-08-13 16:24:29.194611 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:29.194611, event: started, request: pg_notify(22.619(17) epoch 194454) v5 -1253> 2014-08-13 16:24:29.194635 7fd93dff0700 5 -- op tracker -- , seq: 1132, time: 2014-08-13 16:24:29.194635, event: done, request: pg_notify(22.619(17) epoch 194454) v5 -1252> 2014-08-13 16:24:29.194693 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.311 192.168.1.101:6816/24270 15 ==== pg_notify(22.f3(15) epoch 194454) v5 ==== 1551+0+0 (465185448 0 0) 0x775e040 con 0x8da5120 -1251> 2014-08-13 16:24:29.194713 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.060422, event: header_read, request: pg_notify(22.f3(15) epoch 194454) v5 -1250> 2014-08-13 16:24:29.194724 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.060423, event: throttled, request: pg_notify(22.f3(15) epoch 194454) v5 -1249> 2014-08-13 16:24:29.194733 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.060537, event: all_read, request: pg_notify(22.f3(15) epoch 194454) v5 -1248> 2014-08-13 16:24:29.194742 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.194711, event: dispatched, request: pg_notify(22.f3(15) epoch 194454) v5 -1247> 2014-08-13 16:24:29.194752 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.194751, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194454) v5 -1246> 2014-08-13 16:24:29.194768 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.194768, event: started, request: pg_notify(22.f3(15) epoch 194454) v5 -1245> 2014-08-13 16:24:29.194792 7fd93dff0700 5 -- op tracker -- , seq: 1133, time: 2014-08-13 16:24:29.194792, event: done, request: pg_notify(22.f3(15) epoch 194454) v5 -1244> 2014-08-13 16:24:29.194832 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 18 ==== pg_notify(22.99(4) epoch 194454) v5 ==== 942+0+0 (3929892240 0 0) 0x91171c0 con 0x72d9b80 -1243> 2014-08-13 16:24:29.194853 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:28.941089, event: header_read, request: pg_notify(22.99(4) epoch 194454) v5 -1242> 2014-08-13 16:24:29.194864 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:28.941090, event: throttled, request: pg_notify(22.99(4) epoch 194454) v5 -1241> 2014-08-13 16:24:29.194873 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:28.941172, event: all_read, request: pg_notify(22.99(4) epoch 194454) v5 -1240> 2014-08-13 16:24:29.194882 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:29.194851, event: dispatched, request: pg_notify(22.99(4) epoch 194454) v5 -1239> 2014-08-13 16:24:29.194892 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:29.194891, event: waiting_for_osdmap, request: pg_notify(22.99(4) epoch 194454) v5 -1238> 2014-08-13 16:24:29.194902 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:29.194902, event: started, request: pg_notify(22.99(4) epoch 194454) v5 -1237> 2014-08-13 16:24:29.194926 7fd93dff0700 5 -- op tracker -- , seq: 1134, time: 2014-08-13 16:24:29.194925, event: done, request: pg_notify(22.99(4) epoch 194454) v5 -1236> 2014-08-13 16:24:29.194966 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 15 ==== pg_notify(22.4df(4) epoch 194454) v5 ==== 942+0+0 (2154122721 0 0) 0x840f700 con 0x72dfa60 -1235> 2014-08-13 16:24:29.194986 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:28.935823, event: header_read, request: pg_notify(22.4df(4) epoch 194454) v5 -1234> 2014-08-13 16:24:29.194996 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:28.935824, event: throttled, request: pg_notify(22.4df(4) epoch 194454) v5 -1233> 2014-08-13 16:24:29.195005 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:28.935941, event: all_read, request: pg_notify(22.4df(4) epoch 194454) v5 -1232> 2014-08-13 16:24:29.195014 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:29.194984, event: dispatched, request: pg_notify(22.4df(4) epoch 194454) v5 -1231> 2014-08-13 16:24:29.195024 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:29.195024, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194454) v5 -1230> 2014-08-13 16:24:29.195035 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:29.195035, event: started, request: pg_notify(22.4df(4) epoch 194454) v5 -1229> 2014-08-13 16:24:29.195058 7fd93dff0700 5 -- op tracker -- , seq: 1135, time: 2014-08-13 16:24:29.195058, event: done, request: pg_notify(22.4df(4) epoch 194454) v5 -1228> 2014-08-13 16:24:29.195097 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 14 ==== pg_notify(22.39d(4) epoch 194454) v5 ==== 942+0+0 (4266831093 0 0) 0x834c980 con 0x72de9e0 -1227> 2014-08-13 16:24:29.195116 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.048242, event: header_read, request: pg_notify(22.39d(4) epoch 194454) v5 -1226> 2014-08-13 16:24:29.195127 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.048243, event: throttled, request: pg_notify(22.39d(4) epoch 194454) v5 -1225> 2014-08-13 16:24:29.195136 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.048332, event: all_read, request: pg_notify(22.39d(4) epoch 194454) v5 -1224> 2014-08-13 16:24:29.195144 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.195114, event: dispatched, request: pg_notify(22.39d(4) epoch 194454) v5 -1223> 2014-08-13 16:24:29.195154 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.195154, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194454) v5 -1222> 2014-08-13 16:24:29.195172 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.195172, event: started, request: pg_notify(22.39d(4) epoch 194454) v5 -1221> 2014-08-13 16:24:29.195158 7fd9367e4700 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.215920 5 0.000314 -1220> 2014-08-13 16:24:29.195196 7fd93dff0700 5 -- op tracker -- , seq: 1136, time: 2014-08-13 16:24:29.195196, event: done, request: pg_notify(22.39d(4) epoch 194454) v5 -1219> 2014-08-13 16:24:29.195214 7fd9367e4700 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.261580 0 0.000000 -1218> 2014-08-13 16:24:29.195244 7fd9367e4700 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194448 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1217> 2014-08-13 16:24:29.195251 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.0 192.168.1.115:6810/59615 15 ==== pg_notify(22.775(17) epoch 194454) v5 ==== 1665+0+0 (2744604263 0 0) 0x842d080 con 0x8e29fa0 -1216> 2014-08-13 16:24:29.195272 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:28.941272, event: header_read, request: pg_notify(22.775(17) epoch 194454) v5 -1215> 2014-08-13 16:24:29.195283 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:28.941273, event: throttled, request: pg_notify(22.775(17) epoch 194454) v5 -1214> 2014-08-13 16:24:29.195293 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:28.941374, event: all_read, request: pg_notify(22.775(17) epoch 194454) v5 -1213> 2014-08-13 16:24:29.195302 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:29.195269, event: dispatched, request: pg_notify(22.775(17) epoch 194454) v5 -1212> 2014-08-13 16:24:29.195311 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:29.195311, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194454) v5 -1211> 2014-08-13 16:24:29.195323 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:29.195323, event: started, request: pg_notify(22.775(17) epoch 194454) v5 -1210> 2014-08-13 16:24:29.195347 7fd93dff0700 5 -- op tracker -- , seq: 1137, time: 2014-08-13 16:24:29.195347, event: done, request: pg_notify(22.775(17) epoch 194454) v5 -1209> 2014-08-13 16:24:29.195410 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 26 ==== pg_notify(22.3af(17) epoch 194454) v5 ==== 1665+0+0 (109739394 0 0) 0x834d5c0 con 0x8e286e0 -1208> 2014-08-13 16:24:29.195430 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.045861, event: header_read, request: pg_notify(22.3af(17) epoch 194454) v5 -1207> 2014-08-13 16:24:29.195441 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.045862, event: throttled, request: pg_notify(22.3af(17) epoch 194454) v5 -1206> 2014-08-13 16:24:29.195451 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.045954, event: all_read, request: pg_notify(22.3af(17) epoch 194454) v5 -1205> 2014-08-13 16:24:29.195438 7fd9367e4700 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194453 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded] enter Started/Primary/Active/Activating -1204> 2014-08-13 16:24:29.195460 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.195428, event: dispatched, request: pg_notify(22.3af(17) epoch 194454) v5 -1203> 2014-08-13 16:24:29.195470 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.195470, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194454) v5 -1202> 2014-08-13 16:24:29.195481 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.195481, event: started, request: pg_notify(22.3af(17) epoch 194454) v5 -1201> 2014-08-13 16:24:29.195504 7fd93dff0700 5 -- op tracker -- , seq: 1138, time: 2014-08-13 16:24:29.195504, event: done, request: pg_notify(22.3af(17) epoch 194454) v5 -1200> 2014-08-13 16:24:29.195541 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 14 ==== pg_notify(22.4df(15) epoch 194454) v5 ==== 1551+0+0 (932459148 0 0) 0x8348700 con 0x8e2c360 -1199> 2014-08-13 16:24:29.195562 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:28.977944, event: header_read, request: pg_notify(22.4df(15) epoch 194454) v5 -1198> 2014-08-13 16:24:29.195572 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:28.977945, event: throttled, request: pg_notify(22.4df(15) epoch 194454) v5 -1197> 2014-08-13 16:24:29.195581 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:28.978056, event: all_read, request: pg_notify(22.4df(15) epoch 194454) v5 -1196> 2014-08-13 16:24:29.195590 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:29.195560, event: dispatched, request: pg_notify(22.4df(15) epoch 194454) v5 -1195> 2014-08-13 16:24:29.195600 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:29.195600, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194454) v5 -1194> 2014-08-13 16:24:29.195611 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:29.195611, event: started, request: pg_notify(22.4df(15) epoch 194454) v5 -1193> 2014-08-13 16:24:29.195847 7fd93dff0700 5 -- op tracker -- , seq: 1139, time: 2014-08-13 16:24:29.195846, event: done, request: pg_notify(22.4df(15) epoch 194454) v5 -1192> 2014-08-13 16:24:29.195929 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 15 ==== pg_notify(24.181(25) epoch 194454) v5 ==== 2159+0+0 (2885936427 0 0) 0x834fe00 con 0x7379ce0 -1191> 2014-08-13 16:24:29.195958 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:28.941104, event: header_read, request: pg_notify(24.181(25) epoch 194454) v5 -1190> 2014-08-13 16:24:29.195970 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:28.941105, event: throttled, request: pg_notify(24.181(25) epoch 194454) v5 -1189> 2014-08-13 16:24:29.195980 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:28.941219, event: all_read, request: pg_notify(24.181(25) epoch 194454) v5 -1188> 2014-08-13 16:24:29.195989 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:29.195955, event: dispatched, request: pg_notify(24.181(25) epoch 194454) v5 -1187> 2014-08-13 16:24:29.195999 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:29.195999, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194454) v5 -1186> 2014-08-13 16:24:29.196012 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:29.196011, event: started, request: pg_notify(24.181(25) epoch 194454) v5 -1185> 2014-08-13 16:24:29.196041 7fd93dff0700 5 -- op tracker -- , seq: 1140, time: 2014-08-13 16:24:29.196041, event: done, request: pg_notify(24.181(25) epoch 194454) v5 -1184> 2014-08-13 16:24:29.196120 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 15 ==== pg_notify(22.1f1(4) epoch 194454) v5 ==== 942+0+0 (2570872090 0 0) 0x775a840 con 0x73ca680 -1183> 2014-08-13 16:24:29.196144 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.033405, event: header_read, request: pg_notify(22.1f1(4) epoch 194454) v5 -1182> 2014-08-13 16:24:29.196159 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.033406, event: throttled, request: pg_notify(22.1f1(4) epoch 194454) v5 -1181> 2014-08-13 16:24:29.196169 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.033598, event: all_read, request: pg_notify(22.1f1(4) epoch 194454) v5 -1180> 2014-08-13 16:24:29.196178 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.196141, event: dispatched, request: pg_notify(22.1f1(4) epoch 194454) v5 -1179> 2014-08-13 16:24:29.196131 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.107:6802/9548 -- pg_info(1 pgs e194454:24.34(43)) v4 -- ?+0 0x92871c0 con 0x83002c0 -1178> 2014-08-13 16:24:29.196191 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.196191, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194454) v5 -1177> 2014-08-13 16:24:29.196217 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.196217, event: started, request: pg_notify(22.1f1(4) epoch 194454) v5 -1176> 2014-08-13 16:24:29.196247 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.260689 4 0.000237 -1175> 2014-08-13 16:24:29.196301 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Started 1.260823 0 0.000000 -1174> 2014-08-13 16:24:29.196310 7fd93dff0700 5 -- op tracker -- , seq: 1141, time: 2014-08-13 16:24:29.196309, event: done, request: pg_notify(22.1f1(4) epoch 194454) v5 -1173> 2014-08-13 16:24:29.196332 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194453/192971) [20,27,4]/[20,4,374] r=1 lpr=194453 pi=193125-194452/26 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Reset -1172> 2014-08-13 16:24:29.196388 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 21 ==== pg_notify(22.3af(17) epoch 194454) v5 ==== 1665+0+0 (109739394 0 0) 0x899ad80 con 0x73cdc20 -1171> 2014-08-13 16:24:29.196414 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:28.933299, event: header_read, request: pg_notify(22.3af(17) epoch 194454) v5 -1170> 2014-08-13 16:24:29.196425 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:28.933300, event: throttled, request: pg_notify(22.3af(17) epoch 194454) v5 -1169> 2014-08-13 16:24:29.196434 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:28.933423, event: all_read, request: pg_notify(22.3af(17) epoch 194454) v5 -1168> 2014-08-13 16:24:29.196443 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:29.196411, event: dispatched, request: pg_notify(22.3af(17) epoch 194454) v5 -1167> 2014-08-13 16:24:29.196453 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:29.196452, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194454) v5 -1166> 2014-08-13 16:24:29.196464 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:29.196464, event: started, request: pg_notify(22.3af(17) epoch 194454) v5 -1165> 2014-08-13 16:24:29.196485 7fd93dff0700 5 -- op tracker -- , seq: 1142, time: 2014-08-13 16:24:29.196485, event: done, request: pg_notify(22.3af(17) epoch 194454) v5 -1164> 2014-08-13 16:24:29.196495 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.000163 1 0.000165 -1163> 2014-08-13 16:24:29.196528 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started -1162> 2014-08-13 16:24:29.196529 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 12 ==== pg_notify(22.481(32) epoch 194454) v5 ==== 2518+0+0 (3892166691 0 0) 0x842c440 con 0x73ccba0 -1161> 2014-08-13 16:24:29.196554 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Start -1160> 2014-08-13 16:24:29.196564 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:28.952963, event: header_read, request: pg_notify(22.481(32) epoch 194454) v5 -1159> 2014-08-13 16:24:29.196579 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:28.952965, event: throttled, request: pg_notify(22.481(32) epoch 194454) v5 -1158> 2014-08-13 16:24:29.196577 7fd9371e5700 1 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -1157> 2014-08-13 16:24:29.196589 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:28.953132, event: all_read, request: pg_notify(22.481(32) epoch 194454) v5 -1156> 2014-08-13 16:24:29.196596 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:29.196561, event: dispatched, request: pg_notify(22.481(32) epoch 194454) v5 -1155> 2014-08-13 16:24:29.196602 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000049 0 0.000000 -1154> 2014-08-13 16:24:29.196605 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:29.196605, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194454) v5 -1153> 2014-08-13 16:24:29.196636 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:29.196636, event: started, request: pg_notify(22.481(32) epoch 194454) v5 -1152> 2014-08-13 16:24:29.196630 7fd9371e5700 5 osd.4 pg_epoch: 194454 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -1151> 2014-08-13 16:24:29.196665 7fd93dff0700 5 -- op tracker -- , seq: 1143, time: 2014-08-13 16:24:29.196665, event: done, request: pg_notify(22.481(32) epoch 194454) v5 -1150> 2014-08-13 16:24:29.196716 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 12 ==== pg_notify(22.3d9(35) epoch 194454) v5 ==== 2603+0+0 (4168483392 0 0) 0x92847c0 con 0x7462100 -1149> 2014-08-13 16:24:29.196738 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:28.930358, event: header_read, request: pg_notify(22.3d9(35) epoch 194454) v5 -1148> 2014-08-13 16:24:29.196749 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:28.930359, event: throttled, request: pg_notify(22.3d9(35) epoch 194454) v5 -1147> 2014-08-13 16:24:29.196758 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:28.930475, event: all_read, request: pg_notify(22.3d9(35) epoch 194454) v5 -1146> 2014-08-13 16:24:29.196767 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:29.196735, event: dispatched, request: pg_notify(22.3d9(35) epoch 194454) v5 -1145> 2014-08-13 16:24:29.196776 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:29.196776, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194454) v5 -1144> 2014-08-13 16:24:29.196787 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:29.196787, event: started, request: pg_notify(22.3d9(35) epoch 194454) v5 -1143> 2014-08-13 16:24:29.196811 7fd93dff0700 5 -- op tracker -- , seq: 1144, time: 2014-08-13 16:24:29.196811, event: done, request: pg_notify(22.3d9(35) epoch 194454) v5 -1142> 2014-08-13 16:24:29.196848 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 21 ==== pg_notify(24.9b(4) epoch 194454) v5 ==== 976+0+0 (1384256176 0 0) 0x7758000 con 0x7723440 -1141> 2014-08-13 16:24:29.196868 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:28.934552, event: header_read, request: pg_notify(24.9b(4) epoch 194454) v5 -1140> 2014-08-13 16:24:29.196879 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:28.934553, event: throttled, request: pg_notify(24.9b(4) epoch 194454) v5 -1139> 2014-08-13 16:24:29.196888 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:28.934659, event: all_read, request: pg_notify(24.9b(4) epoch 194454) v5 -1138> 2014-08-13 16:24:29.196897 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:29.196866, event: dispatched, request: pg_notify(24.9b(4) epoch 194454) v5 -1137> 2014-08-13 16:24:29.196907 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:29.196907, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194454) v5 -1136> 2014-08-13 16:24:29.196918 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:29.196917, event: started, request: pg_notify(24.9b(4) epoch 194454) v5 -1135> 2014-08-13 16:24:29.196941 7fd93dff0700 5 -- op tracker -- , seq: 1145, time: 2014-08-13 16:24:29.196940, event: done, request: pg_notify(24.9b(4) epoch 194454) v5 -1134> 2014-08-13 16:24:29.196981 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 10 ==== pg_notify(24.34(30) epoch 194454) v5 ==== 2250+0+0 (2062795877 0 0) 0x8c8a680 con 0x7d2d120 -1133> 2014-08-13 16:24:29.197001 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:28.948046, event: header_read, request: pg_notify(24.34(30) epoch 194454) v5 -1132> 2014-08-13 16:24:29.197012 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:28.948047, event: throttled, request: pg_notify(24.34(30) epoch 194454) v5 -1131> 2014-08-13 16:24:29.197022 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:28.948201, event: all_read, request: pg_notify(24.34(30) epoch 194454) v5 -1130> 2014-08-13 16:24:29.197031 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:29.196999, event: dispatched, request: pg_notify(24.34(30) epoch 194454) v5 -1129> 2014-08-13 16:24:29.197041 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:29.197040, event: waiting_for_osdmap, request: pg_notify(24.34(30) epoch 194454) v5 -1128> 2014-08-13 16:24:29.197052 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:29.197052, event: started, request: pg_notify(24.34(30) epoch 194454) v5 -1127> 2014-08-13 16:24:29.197077 7fd93dff0700 5 -- op tracker -- , seq: 1146, time: 2014-08-13 16:24:29.197077, event: done, request: pg_notify(24.34(30) epoch 194454) v5 -1126> 2014-08-13 16:24:29.197131 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 13 ==== pg_notify(22.619(17) epoch 194454) v5 ==== 1665+0+0 (3796848906 0 0) 0x834e040 con 0x7d2d280 -1125> 2014-08-13 16:24:29.197151 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.048563, event: header_read, request: pg_notify(22.619(17) epoch 194454) v5 -1124> 2014-08-13 16:24:29.197162 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.048564, event: throttled, request: pg_notify(22.619(17) epoch 194454) v5 -1123> 2014-08-13 16:24:29.197168 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.048670, event: all_read, request: pg_notify(22.619(17) epoch 194454) v5 -1122> 2014-08-13 16:24:29.197177 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.197149, event: dispatched, request: pg_notify(22.619(17) epoch 194454) v5 -1121> 2014-08-13 16:24:29.197187 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.197186, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194454) v5 -1120> 2014-08-13 16:24:29.197198 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.197197, event: started, request: pg_notify(22.619(17) epoch 194454) v5 -1119> 2014-08-13 16:24:29.197230 7fd93dff0700 5 -- op tracker -- , seq: 1147, time: 2014-08-13 16:24:29.197229, event: done, request: pg_notify(22.619(17) epoch 194454) v5 -1118> 2014-08-13 16:24:29.197274 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 14 ==== pg_notify(22.775(17) epoch 194454) v5 ==== 1665+0+0 (2655256643 0 0) 0x85755c0 con 0x821f900 -1117> 2014-08-13 16:24:29.197296 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:28.938952, event: header_read, request: pg_notify(22.775(17) epoch 194454) v5 -1116> 2014-08-13 16:24:29.197307 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:28.938953, event: throttled, request: pg_notify(22.775(17) epoch 194454) v5 -1115> 2014-08-13 16:24:29.197316 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:28.939052, event: all_read, request: pg_notify(22.775(17) epoch 194454) v5 -1114> 2014-08-13 16:24:29.197325 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:29.197293, event: dispatched, request: pg_notify(22.775(17) epoch 194454) v5 -1113> 2014-08-13 16:24:29.197334 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:29.197334, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194454) v5 -1112> 2014-08-13 16:24:29.197345 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:29.197345, event: started, request: pg_notify(22.775(17) epoch 194454) v5 -1111> 2014-08-13 16:24:29.197369 7fd93dff0700 5 -- op tracker -- , seq: 1148, time: 2014-08-13 16:24:29.197368, event: done, request: pg_notify(22.775(17) epoch 194454) v5 -1110> 2014-08-13 16:24:29.197422 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 12 ==== pg_notify(22.3d9(39) epoch 194454) v5 ==== 2815+0+0 (403891726 0 0) 0x775e580 con 0x80c7e80 -1109> 2014-08-13 16:24:29.197447 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:28.957441, event: header_read, request: pg_notify(22.3d9(39) epoch 194454) v5 -1108> 2014-08-13 16:24:29.197458 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:28.957443, event: throttled, request: pg_notify(22.3d9(39) epoch 194454) v5 -1107> 2014-08-13 16:24:29.197467 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:28.957588, event: all_read, request: pg_notify(22.3d9(39) epoch 194454) v5 -1106> 2014-08-13 16:24:29.197476 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:29.197444, event: dispatched, request: pg_notify(22.3d9(39) epoch 194454) v5 -1105> 2014-08-13 16:24:29.197485 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:29.197485, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194454) v5 -1104> 2014-08-13 16:24:29.197496 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:29.197496, event: started, request: pg_notify(22.3d9(39) epoch 194454) v5 -1103> 2014-08-13 16:24:29.197520 7fd93dff0700 5 -- op tracker -- , seq: 1149, time: 2014-08-13 16:24:29.197520, event: done, request: pg_notify(22.3d9(39) epoch 194454) v5 -1102> 2014-08-13 16:24:29.197569 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 14 ==== pg_notify(22.481(34) epoch 194454) v5 ==== 2620+0+0 (1900484883 0 0) 0x77588c0 con 0x80c7d20 -1101> 2014-08-13 16:24:29.197589 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:28.962295, event: header_read, request: pg_notify(22.481(34) epoch 194454) v5 -1100> 2014-08-13 16:24:29.197600 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:28.962297, event: throttled, request: pg_notify(22.481(34) epoch 194454) v5 -1099> 2014-08-13 16:24:29.197609 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:28.962478, event: all_read, request: pg_notify(22.481(34) epoch 194454) v5 -1098> 2014-08-13 16:24:29.197618 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:29.197587, event: dispatched, request: pg_notify(22.481(34) epoch 194454) v5 -1097> 2014-08-13 16:24:29.197628 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:29.197628, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194454) v5 -1096> 2014-08-13 16:24:29.197639 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:29.197639, event: started, request: pg_notify(22.481(34) epoch 194454) v5 -1095> 2014-08-13 16:24:29.197661 7fd93dff0700 5 -- op tracker -- , seq: 1150, time: 2014-08-13 16:24:29.197661, event: done, request: pg_notify(22.481(34) epoch 194454) v5 -1094> 2014-08-13 16:24:29.197722 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 5 ==== pg_notify(24.181(25) epoch 194454) v5 ==== 2159+0+0 (2971826633 0 0) 0x842c600 con 0x83002c0 -1093> 2014-08-13 16:24:29.197743 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:28.960279, event: header_read, request: pg_notify(24.181(25) epoch 194454) v5 -1092> 2014-08-13 16:24:29.197754 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:28.960281, event: throttled, request: pg_notify(24.181(25) epoch 194454) v5 -1091> 2014-08-13 16:24:29.197763 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:28.960433, event: all_read, request: pg_notify(24.181(25) epoch 194454) v5 -1090> 2014-08-13 16:24:29.197771 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:29.197740, event: dispatched, request: pg_notify(24.181(25) epoch 194454) v5 -1089> 2014-08-13 16:24:29.197781 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:29.197781, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194454) v5 -1088> 2014-08-13 16:24:29.197792 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:29.197792, event: started, request: pg_notify(24.181(25) epoch 194454) v5 -1087> 2014-08-13 16:24:29.198067 7fd93dff0700 5 -- op tracker -- , seq: 1151, time: 2014-08-13 16:24:29.198066, event: done, request: pg_notify(24.181(25) epoch 194454) v5 -1086> 2014-08-13 16:24:29.198145 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 22 ==== pg_notify(24.30b(4) epoch 194454) v5 ==== 976+0+0 (443402134 0 0) 0x7d05400 con 0x4e4b860 -1085> 2014-08-13 16:24:29.198168 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:28.950695, event: header_read, request: pg_notify(24.30b(4) epoch 194454) v5 -1084> 2014-08-13 16:24:29.198179 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:28.950696, event: throttled, request: pg_notify(24.30b(4) epoch 194454) v5 -1083> 2014-08-13 16:24:29.198188 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:28.950764, event: all_read, request: pg_notify(24.30b(4) epoch 194454) v5 -1082> 2014-08-13 16:24:29.198197 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:29.198165, event: dispatched, request: pg_notify(24.30b(4) epoch 194454) v5 -1081> 2014-08-13 16:24:29.198215 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:29.198215, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194454) v5 -1080> 2014-08-13 16:24:29.198227 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:29.198227, event: started, request: pg_notify(24.30b(4) epoch 194454) v5 -1079> 2014-08-13 16:24:29.198252 7fd93dff0700 5 -- op tracker -- , seq: 1152, time: 2014-08-13 16:24:29.198252, event: done, request: pg_notify(24.30b(4) epoch 194454) v5 -1078> 2014-08-13 16:24:29.198295 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 26 ==== pg_notify(22.1d2(17) epoch 194454) v5 ==== 1669+0+0 (222202679 0 0) 0x840cb40 con 0x4e4dd80 -1077> 2014-08-13 16:24:29.198316 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:28.951179, event: header_read, request: pg_notify(22.1d2(17) epoch 194454) v5 -1076> 2014-08-13 16:24:29.198326 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:28.951180, event: throttled, request: pg_notify(22.1d2(17) epoch 194454) v5 -1075> 2014-08-13 16:24:29.198336 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:28.951279, event: all_read, request: pg_notify(22.1d2(17) epoch 194454) v5 -1074> 2014-08-13 16:24:29.198344 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:29.198313, event: dispatched, request: pg_notify(22.1d2(17) epoch 194454) v5 -1073> 2014-08-13 16:24:29.198354 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:29.198354, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194454) v5 -1072> 2014-08-13 16:24:29.198365 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:29.198365, event: started, request: pg_notify(22.1d2(17) epoch 194454) v5 -1071> 2014-08-13 16:24:29.198388 7fd93dff0700 5 -- op tracker -- , seq: 1153, time: 2014-08-13 16:24:29.198388, event: done, request: pg_notify(22.1d2(17) epoch 194454) v5 -1070> 2014-08-13 16:24:29.198431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 20 ==== pg_notify(22.1f1(17) epoch 194454) v5 ==== 1665+0+0 (4213413740 0 0) 0x842f700 con 0x4e4ce60 -1069> 2014-08-13 16:24:29.198452 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.044540, event: header_read, request: pg_notify(22.1f1(17) epoch 194454) v5 -1068> 2014-08-13 16:24:29.198462 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.044542, event: throttled, request: pg_notify(22.1f1(17) epoch 194454) v5 -1067> 2014-08-13 16:24:29.198472 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.044643, event: all_read, request: pg_notify(22.1f1(17) epoch 194454) v5 -1066> 2014-08-13 16:24:29.198481 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.198449, event: dispatched, request: pg_notify(22.1f1(17) epoch 194454) v5 -1065> 2014-08-13 16:24:29.198490 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.198490, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194454) v5 -1064> 2014-08-13 16:24:29.198501 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.198501, event: started, request: pg_notify(22.1f1(17) epoch 194454) v5 -1063> 2014-08-13 16:24:29.198904 7fd93dff0700 5 -- op tracker -- , seq: 1154, time: 2014-08-13 16:24:29.198903, event: done, request: pg_notify(22.1f1(17) epoch 194454) v5 -1062> 2014-08-13 16:24:29.198951 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 30 ==== pg_notify(24.34(43) epoch 194454) v5 ==== 2933+0+0 (250349134 0 0) 0x8998e00 con 0x4e4f900 -1061> 2014-08-13 16:24:29.198973 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:28.945549, event: header_read, request: pg_notify(24.34(43) epoch 194454) v5 -1060> 2014-08-13 16:24:29.198983 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:28.945551, event: throttled, request: pg_notify(24.34(43) epoch 194454) v5 -1059> 2014-08-13 16:24:29.198992 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:28.945706, event: all_read, request: pg_notify(24.34(43) epoch 194454) v5 -1058> 2014-08-13 16:24:29.199001 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:29.198970, event: dispatched, request: pg_notify(24.34(43) epoch 194454) v5 -1057> 2014-08-13 16:24:29.199010 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:29.199010, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194454) v5 -1056> 2014-08-13 16:24:29.199021 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:29.199021, event: started, request: pg_notify(24.34(43) epoch 194454) v5 -1055> 2014-08-13 16:24:29.199046 7fd93dff0700 5 -- op tracker -- , seq: 1155, time: 2014-08-13 16:24:29.199046, event: done, request: pg_notify(24.34(43) epoch 194454) v5 -1054> 2014-08-13 16:24:29.199104 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 31 ==== pg_notify(24.136(17) epoch 194454) v5 ==== 1695+0+0 (2531423043 0 0) 0x775c280 con 0x4fd8000 -1053> 2014-08-13 16:24:29.199126 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:28.961319, event: header_read, request: pg_notify(24.136(17) epoch 194454) v5 -1052> 2014-08-13 16:24:29.199137 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:28.961320, event: throttled, request: pg_notify(24.136(17) epoch 194454) v5 -1051> 2014-08-13 16:24:29.199146 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:28.961424, event: all_read, request: pg_notify(24.136(17) epoch 194454) v5 -1050> 2014-08-13 16:24:29.199152 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:29.199123, event: dispatched, request: pg_notify(24.136(17) epoch 194454) v5 -1049> 2014-08-13 16:24:29.199161 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:29.199161, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194454) v5 -1048> 2014-08-13 16:24:29.199172 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:29.199172, event: started, request: pg_notify(24.136(17) epoch 194454) v5 -1047> 2014-08-13 16:24:29.199198 7fd93dff0700 5 -- op tracker -- , seq: 1156, time: 2014-08-13 16:24:29.199198, event: done, request: pg_notify(24.136(17) epoch 194454) v5 -1046> 2014-08-13 16:24:29.199257 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 25 ==== pg_notify(22.99(15) epoch 194454) v5 ==== 1551+0+0 (1602245888 0 0) 0x834cd00 con 0x4fe1600 -1045> 2014-08-13 16:24:29.199278 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:28.931969, event: header_read, request: pg_notify(22.99(15) epoch 194454) v5 -1044> 2014-08-13 16:24:29.199289 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:28.931970, event: throttled, request: pg_notify(22.99(15) epoch 194454) v5 -1043> 2014-08-13 16:24:29.199295 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:28.932065, event: all_read, request: pg_notify(22.99(15) epoch 194454) v5 -1042> 2014-08-13 16:24:29.199300 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:29.199276, event: dispatched, request: pg_notify(22.99(15) epoch 194454) v5 -1041> 2014-08-13 16:24:29.199307 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:29.199306, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194454) v5 -1040> 2014-08-13 16:24:29.199315 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:29.199314, event: started, request: pg_notify(22.99(15) epoch 194454) v5 -1039> 2014-08-13 16:24:29.199337 7fd93dff0700 5 -- op tracker -- , seq: 1157, time: 2014-08-13 16:24:29.199337, event: done, request: pg_notify(22.99(15) epoch 194454) v5 -1038> 2014-08-13 16:24:29.199375 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 24 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1770398198 0 0) 0x7974200 con 0x4fe3700 -1037> 2014-08-13 16:24:29.199403 7fd93dff0700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.056521, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1036> 2014-08-13 16:24:29.199418 7fd93dff0700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.056523, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1035> 2014-08-13 16:24:29.199428 7fd93dff0700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.056597, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1034> 2014-08-13 16:24:29.199440 7fd93dff0700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.199400, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1033> 2014-08-13 16:24:29.199453 7fd93dff0700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.199453, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1032> 2014-08-13 16:24:29.199478 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 27 ==== pg_notify(22.3af(4) epoch 194454) v5 ==== 942+0+0 (2649828368 0 0) 0x834f700 con 0x4fe40a0 -1031> 2014-08-13 16:24:29.199496 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:28.939816, event: header_read, request: pg_notify(22.3af(4) epoch 194454) v5 -1030> 2014-08-13 16:24:29.199506 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:28.939818, event: throttled, request: pg_notify(22.3af(4) epoch 194454) v5 -1029> 2014-08-13 16:24:29.199515 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:28.939902, event: all_read, request: pg_notify(22.3af(4) epoch 194454) v5 -1028> 2014-08-13 16:24:29.199524 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:29.199494, event: dispatched, request: pg_notify(22.3af(4) epoch 194454) v5 -1027> 2014-08-13 16:24:29.199534 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:29.199533, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194454) v5 -1026> 2014-08-13 16:24:29.199544 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:29.199544, event: started, request: pg_notify(22.3af(4) epoch 194454) v5 -1025> 2014-08-13 16:24:29.199567 7fd93dff0700 5 -- op tracker -- , seq: 1159, time: 2014-08-13 16:24:29.199567, event: done, request: pg_notify(22.3af(4) epoch 194454) v5 -1024> 2014-08-13 16:24:29.199610 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 27 ==== pg_notify(24.34(43) epoch 194454) v5 ==== 2933+0+0 (722629158 0 0) 0x842f000 con 0x4fe69e0 -1023> 2014-08-13 16:24:29.199631 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:28.940119, event: header_read, request: pg_notify(24.34(43) epoch 194454) v5 -1022> 2014-08-13 16:24:29.199641 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:28.940121, event: throttled, request: pg_notify(24.34(43) epoch 194454) v5 -1021> 2014-08-13 16:24:29.199650 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:28.940261, event: all_read, request: pg_notify(24.34(43) epoch 194454) v5 -1020> 2014-08-13 16:24:29.199659 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:29.199628, event: dispatched, request: pg_notify(24.34(43) epoch 194454) v5 -1019> 2014-08-13 16:24:29.199668 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:29.199668, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194454) v5 -1018> 2014-08-13 16:24:29.199679 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:29.199679, event: started, request: pg_notify(24.34(43) epoch 194454) v5 -1017> 2014-08-13 16:24:29.199703 7fd93dff0700 5 -- op tracker -- , seq: 1160, time: 2014-08-13 16:24:29.199703, event: done, request: pg_notify(24.34(43) epoch 194454) v5 -1016> 2014-08-13 16:24:29.199759 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 26 ==== pg_notify(22.3e4(17) epoch 194454) v5 ==== 1665+0+0 (108633074 0 0) 0x834ad80 con 0x4fda940 -1015> 2014-08-13 16:24:29.199781 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:28.947378, event: header_read, request: pg_notify(22.3e4(17) epoch 194454) v5 -1014> 2014-08-13 16:24:29.199792 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:28.947382, event: throttled, request: pg_notify(22.3e4(17) epoch 194454) v5 -1013> 2014-08-13 16:24:29.199801 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:28.947505, event: all_read, request: pg_notify(22.3e4(17) epoch 194454) v5 -1012> 2014-08-13 16:24:29.199810 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:29.199779, event: dispatched, request: pg_notify(22.3e4(17) epoch 194454) v5 -1011> 2014-08-13 16:24:29.199819 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:29.199819, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194454) v5 -1010> 2014-08-13 16:24:29.199830 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:29.199830, event: started, request: pg_notify(22.3e4(17) epoch 194454) v5 -1009> 2014-08-13 16:24:29.199853 7fd93dff0700 5 -- op tracker -- , seq: 1161, time: 2014-08-13 16:24:29.199853, event: done, request: pg_notify(22.3e4(17) epoch 194454) v5 -1008> 2014-08-13 16:24:29.199884 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 30 ==== pg_notify(22.113(17) epoch 194454) v5 ==== 1669+0+0 (2046661209 0 0) 0x8999340 con 0x50889a0 -1007> 2014-08-13 16:24:29.199904 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:28.943523, event: header_read, request: pg_notify(22.113(17) epoch 194454) v5 -1006> 2014-08-13 16:24:29.199914 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:28.943524, event: throttled, request: pg_notify(22.113(17) epoch 194454) v5 -1005> 2014-08-13 16:24:29.199924 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:28.943608, event: all_read, request: pg_notify(22.113(17) epoch 194454) v5 -1004> 2014-08-13 16:24:29.199933 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:29.199902, event: dispatched, request: pg_notify(22.113(17) epoch 194454) v5 -1003> 2014-08-13 16:24:29.199942 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:29.199942, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194454) v5 -1002> 2014-08-13 16:24:29.199953 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:29.199953, event: started, request: pg_notify(22.113(17) epoch 194454) v5 -1001> 2014-08-13 16:24:29.199975 7fd93dff0700 5 -- op tracker -- , seq: 1162, time: 2014-08-13 16:24:29.199975, event: done, request: pg_notify(22.113(17) epoch 194454) v5 -1000> 2014-08-13 16:24:29.200031 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 22 ==== pg_notify(22.619(4) epoch 194454) v5 ==== 942+0+0 (3008934642 0 0) 0x834c7c0 con 0x508a680 -999> 2014-08-13 16:24:29.200052 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:28.958597, event: header_read, request: pg_notify(22.619(4) epoch 194454) v5 -998> 2014-08-13 16:24:29.200063 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:28.958598, event: throttled, request: pg_notify(22.619(4) epoch 194454) v5 -997> 2014-08-13 16:24:29.200072 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:28.958662, event: all_read, request: pg_notify(22.619(4) epoch 194454) v5 -996> 2014-08-13 16:24:29.200081 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:29.200050, event: dispatched, request: pg_notify(22.619(4) epoch 194454) v5 -995> 2014-08-13 16:24:29.200090 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:29.200090, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194454) v5 -994> 2014-08-13 16:24:29.200101 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:29.200101, event: started, request: pg_notify(22.619(4) epoch 194454) v5 -993> 2014-08-13 16:24:29.200124 7fd93dff0700 5 -- op tracker -- , seq: 1163, time: 2014-08-13 16:24:29.200124, event: done, request: pg_notify(22.619(4) epoch 194454) v5 -992> 2014-08-13 16:24:29.200165 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 38 ==== pg_notify(22.99(15) epoch 194454) v5 ==== 1551+0+0 (461966935 0 0) 0x899fc40 con 0x508bde0 -991> 2014-08-13 16:24:29.200185 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:28.941865, event: header_read, request: pg_notify(22.99(15) epoch 194454) v5 -990> 2014-08-13 16:24:29.200196 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:28.941866, event: throttled, request: pg_notify(22.99(15) epoch 194454) v5 -989> 2014-08-13 16:24:29.200213 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:28.941961, event: all_read, request: pg_notify(22.99(15) epoch 194454) v5 -988> 2014-08-13 16:24:29.200222 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:29.200183, event: dispatched, request: pg_notify(22.99(15) epoch 194454) v5 -987> 2014-08-13 16:24:29.200234 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:29.200234, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194454) v5 -986> 2014-08-13 16:24:29.200246 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:29.200245, event: started, request: pg_notify(22.99(15) epoch 194454) v5 -985> 2014-08-13 16:24:29.200268 7fd93dff0700 5 -- op tracker -- , seq: 1164, time: 2014-08-13 16:24:29.200268, event: done, request: pg_notify(22.99(15) epoch 194454) v5 -984> 2014-08-13 16:24:29.200314 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 22 ==== pg_notify(22.707(19) epoch 194454) v5 ==== 1783+0+0 (1014842165 0 0) 0x8348380 con 0x50ba520 -983> 2014-08-13 16:24:29.200335 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:28.944115, event: header_read, request: pg_notify(22.707(19) epoch 194454) v5 -982> 2014-08-13 16:24:29.200345 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:28.944117, event: throttled, request: pg_notify(22.707(19) epoch 194454) v5 -981> 2014-08-13 16:24:29.200354 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:28.944275, event: all_read, request: pg_notify(22.707(19) epoch 194454) v5 -980> 2014-08-13 16:24:29.200363 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:29.200332, event: dispatched, request: pg_notify(22.707(19) epoch 194454) v5 -979> 2014-08-13 16:24:29.200373 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:29.200372, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194454) v5 -978> 2014-08-13 16:24:29.200383 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:29.200383, event: started, request: pg_notify(22.707(19) epoch 194454) v5 -977> 2014-08-13 16:24:29.200406 7fd93dff0700 5 -- op tracker -- , seq: 1165, time: 2014-08-13 16:24:29.200406, event: done, request: pg_notify(22.707(19) epoch 194454) v5 -976> 2014-08-13 16:24:29.200450 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.24 192.168.1.107:6800/52669 15 ==== pg_notify(22.113(17) epoch 194454) v5 ==== 1669+0+0 (867294434 0 0) 0x899d940 con 0x50bfa60 -975> 2014-08-13 16:24:29.200469 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:28.949024, event: header_read, request: pg_notify(22.113(17) epoch 194454) v5 -974> 2014-08-13 16:24:29.200480 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:28.949025, event: throttled, request: pg_notify(22.113(17) epoch 194454) v5 -973> 2014-08-13 16:24:29.200489 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:28.949120, event: all_read, request: pg_notify(22.113(17) epoch 194454) v5 -972> 2014-08-13 16:24:29.200498 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:29.200467, event: dispatched, request: pg_notify(22.113(17) epoch 194454) v5 -971> 2014-08-13 16:24:29.200507 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:29.200507, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194454) v5 -970> 2014-08-13 16:24:29.200518 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:29.200518, event: started, request: pg_notify(22.113(17) epoch 194454) v5 -969> 2014-08-13 16:24:29.200540 7fd93dff0700 5 -- op tracker -- , seq: 1166, time: 2014-08-13 16:24:29.200540, event: done, request: pg_notify(22.113(17) epoch 194454) v5 -968> 2014-08-13 16:24:29.200594 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 23 ==== pg_notify(22.362(17) epoch 194454) v5 ==== 1665+0+0 (1256750180 0 0) 0x8577a80 con 0x510b9c0 -967> 2014-08-13 16:24:29.200615 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:28.939211, event: header_read, request: pg_notify(22.362(17) epoch 194454) v5 -966> 2014-08-13 16:24:29.200625 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:28.939213, event: throttled, request: pg_notify(22.362(17) epoch 194454) v5 -965> 2014-08-13 16:24:29.200634 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:28.939324, event: all_read, request: pg_notify(22.362(17) epoch 194454) v5 -964> 2014-08-13 16:24:29.200643 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:29.200612, event: dispatched, request: pg_notify(22.362(17) epoch 194454) v5 -963> 2014-08-13 16:24:29.200653 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:29.200653, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194454) v5 -962> 2014-08-13 16:24:29.200664 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:29.200664, event: started, request: pg_notify(22.362(17) epoch 194454) v5 -961> 2014-08-13 16:24:29.200687 7fd93dff0700 5 -- op tracker -- , seq: 1167, time: 2014-08-13 16:24:29.200686, event: done, request: pg_notify(22.362(17) epoch 194454) v5 -960> 2014-08-13 16:24:29.200732 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 29 ==== pg_notify(22.707(4) epoch 194454) v5 ==== 942+0+0 (320992986 0 0) 0x9116e40 con 0x5108b00 -959> 2014-08-13 16:24:29.200752 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:28.946847, event: header_read, request: pg_notify(22.707(4) epoch 194454) v5 -958> 2014-08-13 16:24:29.200762 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:28.946849, event: throttled, request: pg_notify(22.707(4) epoch 194454) v5 -957> 2014-08-13 16:24:29.200772 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:28.946939, event: all_read, request: pg_notify(22.707(4) epoch 194454) v5 -956> 2014-08-13 16:24:29.200781 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:29.200750, event: dispatched, request: pg_notify(22.707(4) epoch 194454) v5 -955> 2014-08-13 16:24:29.200790 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:29.200790, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194454) v5 -954> 2014-08-13 16:24:29.200801 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:29.200801, event: started, request: pg_notify(22.707(4) epoch 194454) v5 -953> 2014-08-13 16:24:29.200823 7fd93dff0700 5 -- op tracker -- , seq: 1168, time: 2014-08-13 16:24:29.200823, event: done, request: pg_notify(22.707(4) epoch 194454) v5 -952> 2014-08-13 16:24:29.200858 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 15 ==== pg_notify(22.3d9(39) epoch 194454) v5 ==== 2815+0+0 (910988769 0 0) 0x91178c0 con 0x510dac0 -951> 2014-08-13 16:24:29.200878 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:28.950141, event: header_read, request: pg_notify(22.3d9(39) epoch 194454) v5 -950> 2014-08-13 16:24:29.200888 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:28.950143, event: throttled, request: pg_notify(22.3d9(39) epoch 194454) v5 -949> 2014-08-13 16:24:29.200898 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:28.950289, event: all_read, request: pg_notify(22.3d9(39) epoch 194454) v5 -948> 2014-08-13 16:24:29.200887 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- osd_map(194454..194454 src has 185139..194454) v3 -- ?+0 0x859e0c0 con 0x4fe1600 -947> 2014-08-13 16:24:29.200907 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:29.200876, event: dispatched, request: pg_notify(22.3d9(39) epoch 194454) v5 -946> 2014-08-13 16:24:29.200913 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:29.200913, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194454) v5 -945> 2014-08-13 16:24:29.200921 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:29.200921, event: started, request: pg_notify(22.3d9(39) epoch 194454) v5 -944> 2014-08-13 16:24:29.200926 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(27) epoch 194454) v5 -- ?+0 0x842fe00 con 0x4fe1600 -943> 2014-08-13 16:24:29.201306 7fd93dff0700 5 -- op tracker -- , seq: 1169, time: 2014-08-13 16:24:29.201306, event: done, request: pg_notify(22.3d9(39) epoch 194454) v5 -942> 2014-08-13 16:24:29.201376 7fd9371e5700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.201351, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -941> 2014-08-13 16:24:29.201415 7fd9371e5700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.201414, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -940> 2014-08-13 16:24:29.201427 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 20 ==== pg_notify(22.4df(15) epoch 194454) v5 ==== 1551+0+0 (932459148 0 0) 0x8c8b100 con 0x5112260 -939> 2014-08-13 16:24:29.201437 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.104:6812/64303 -- osd_sub_op_reply(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8002f80 con 0x4fe3700 -938> 2014-08-13 16:24:29.201457 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:28.957524, event: header_read, request: pg_notify(22.4df(15) epoch 194454) v5 -937> 2014-08-13 16:24:29.201473 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:28.957525, event: throttled, request: pg_notify(22.4df(15) epoch 194454) v5 -936> 2014-08-13 16:24:29.201486 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:28.957616, event: all_read, request: pg_notify(22.4df(15) epoch 194454) v5 -935> 2014-08-13 16:24:29.201499 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:29.201453, event: dispatched, request: pg_notify(22.4df(15) epoch 194454) v5 -934> 2014-08-13 16:24:29.201513 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:29.201512, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194454) v5 -933> 2014-08-13 16:24:29.201528 7fd9371e5700 5 -- op tracker -- , seq: 1158, time: 2014-08-13 16:24:29.201465, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -932> 2014-08-13 16:24:29.201568 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:29.201529, event: started, request: pg_notify(22.4df(15) epoch 194454) v5 -931> 2014-08-13 16:24:29.201617 7fd93dff0700 5 -- op tracker -- , seq: 1170, time: 2014-08-13 16:24:29.201617, event: done, request: pg_notify(22.4df(15) epoch 194454) v5 -930> 2014-08-13 16:24:29.201676 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 18 ==== pg_notify(22.1d2(4) epoch 194454) v5 ==== 942+0+0 (1650020694 0 0) 0x91140c0 con 0x5114620 -929> 2014-08-13 16:24:29.201700 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:28.937133, event: header_read, request: pg_notify(22.1d2(4) epoch 194454) v5 -928> 2014-08-13 16:24:29.201711 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:28.937134, event: throttled, request: pg_notify(22.1d2(4) epoch 194454) v5 -927> 2014-08-13 16:24:29.201720 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:28.937218, event: all_read, request: pg_notify(22.1d2(4) epoch 194454) v5 -926> 2014-08-13 16:24:29.201726 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:29.201697, event: dispatched, request: pg_notify(22.1d2(4) epoch 194454) v5 -925> 2014-08-13 16:24:29.201736 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:29.201736, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194454) v5 -924> 2014-08-13 16:24:29.201748 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:29.201748, event: started, request: pg_notify(22.1d2(4) epoch 194454) v5 -923> 2014-08-13 16:24:29.201775 7fd93dff0700 5 -- op tracker -- , seq: 1171, time: 2014-08-13 16:24:29.201775, event: done, request: pg_notify(22.1d2(4) epoch 194454) v5 -922> 2014-08-13 16:24:29.201819 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 25 ==== pg_notify(22.39d(15) epoch 194454) v5 ==== 1551+0+0 (3150322552 0 0) 0x840bd40 con 0x5114360 -921> 2014-08-13 16:24:29.201839 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:28.950670, event: header_read, request: pg_notify(22.39d(15) epoch 194454) v5 -920> 2014-08-13 16:24:29.201850 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:28.950672, event: throttled, request: pg_notify(22.39d(15) epoch 194454) v5 -919> 2014-08-13 16:24:29.201859 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:28.950781, event: all_read, request: pg_notify(22.39d(15) epoch 194454) v5 -918> 2014-08-13 16:24:29.201868 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:29.201837, event: dispatched, request: pg_notify(22.39d(15) epoch 194454) v5 -917> 2014-08-13 16:24:29.201878 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:29.201878, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194454) v5 -916> 2014-08-13 16:24:29.201890 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:29.201890, event: started, request: pg_notify(22.39d(15) epoch 194454) v5 -915> 2014-08-13 16:24:29.201916 7fd93dff0700 5 -- op tracker -- , seq: 1172, time: 2014-08-13 16:24:29.201916, event: done, request: pg_notify(22.39d(15) epoch 194454) v5 -914> 2014-08-13 16:24:29.201961 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 15 ==== pg_notify(24.1bb(21) epoch 194454) v5 ==== 1931+0+0 (1263995905 0 0) 0x899cb40 con 0x5113de0 -913> 2014-08-13 16:24:29.201982 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:28.947735, event: header_read, request: pg_notify(24.1bb(21) epoch 194454) v5 -912> 2014-08-13 16:24:29.201993 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:28.947736, event: throttled, request: pg_notify(24.1bb(21) epoch 194454) v5 -911> 2014-08-13 16:24:29.202002 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:28.947832, event: all_read, request: pg_notify(24.1bb(21) epoch 194454) v5 -910> 2014-08-13 16:24:29.202010 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:29.201980, event: dispatched, request: pg_notify(24.1bb(21) epoch 194454) v5 -909> 2014-08-13 16:24:29.202020 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:29.202020, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194454) v5 -908> 2014-08-13 16:24:29.202031 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:29.202030, event: started, request: pg_notify(24.1bb(21) epoch 194454) v5 -907> 2014-08-13 16:24:29.202057 7fd93dff0700 5 -- op tracker -- , seq: 1173, time: 2014-08-13 16:24:29.202057, event: done, request: pg_notify(24.1bb(21) epoch 194454) v5 -906> 2014-08-13 16:24:29.202105 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 25 ==== pg_notify(22.1d2(17) epoch 194454) v5 ==== 1669+0+0 (1198704012 0 0) 0x840b100 con 0x5113b20 -905> 2014-08-13 16:24:29.202126 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:28.944621, event: header_read, request: pg_notify(22.1d2(17) epoch 194454) v5 -904> 2014-08-13 16:24:29.202136 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:28.944624, event: throttled, request: pg_notify(22.1d2(17) epoch 194454) v5 -903> 2014-08-13 16:24:29.202146 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:28.944756, event: all_read, request: pg_notify(22.1d2(17) epoch 194454) v5 -902> 2014-08-13 16:24:29.202158 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:29.202123, event: dispatched, request: pg_notify(22.1d2(17) epoch 194454) v5 -901> 2014-08-13 16:24:29.202165 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:29.202165, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194454) v5 -900> 2014-08-13 16:24:29.202172 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:29.202172, event: started, request: pg_notify(22.1d2(17) epoch 194454) v5 -899> 2014-08-13 16:24:29.202199 7fd93dff0700 5 -- op tracker -- , seq: 1174, time: 2014-08-13 16:24:29.202199, event: done, request: pg_notify(22.1d2(17) epoch 194454) v5 -898> 2014-08-13 16:24:29.202247 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 22 ==== pg_notify(22.48e(15) epoch 194454) v5 ==== 1551+0+0 (659262742 0 0) 0x840a140 con 0x5113860 -897> 2014-08-13 16:24:29.202269 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:28.942021, event: header_read, request: pg_notify(22.48e(15) epoch 194454) v5 -896> 2014-08-13 16:24:29.202279 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:28.942022, event: throttled, request: pg_notify(22.48e(15) epoch 194454) v5 -895> 2014-08-13 16:24:29.202288 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:28.942109, event: all_read, request: pg_notify(22.48e(15) epoch 194454) v5 -894> 2014-08-13 16:24:29.202296 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:29.202266, event: dispatched, request: pg_notify(22.48e(15) epoch 194454) v5 -893> 2014-08-13 16:24:29.202306 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:29.202306, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194454) v5 -892> 2014-08-13 16:24:29.202317 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:29.202317, event: started, request: pg_notify(22.48e(15) epoch 194454) v5 -891> 2014-08-13 16:24:29.202343 7fd93dff0700 5 -- op tracker -- , seq: 1175, time: 2014-08-13 16:24:29.202343, event: done, request: pg_notify(22.48e(15) epoch 194454) v5 -890> 2014-08-13 16:24:29.202390 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 32 ==== pg_notify(22.3d9(6) epoch 194454) v5 ==== 1060+0+0 (2921762426 0 0) 0x89981c0 con 0x5111fa0 -889> 2014-08-13 16:24:29.202411 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:28.942180, event: header_read, request: pg_notify(22.3d9(6) epoch 194454) v5 -888> 2014-08-13 16:24:29.202421 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:28.942182, event: throttled, request: pg_notify(22.3d9(6) epoch 194454) v5 -887> 2014-08-13 16:24:29.202430 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:28.942269, event: all_read, request: pg_notify(22.3d9(6) epoch 194454) v5 -886> 2014-08-13 16:24:29.202439 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:29.202408, event: dispatched, request: pg_notify(22.3d9(6) epoch 194454) v5 -885> 2014-08-13 16:24:29.202448 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:29.202448, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194454) v5 -884> 2014-08-13 16:24:29.202459 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:29.202459, event: started, request: pg_notify(22.3d9(6) epoch 194454) v5 -883> 2014-08-13 16:24:29.202485 7fd93dff0700 5 -- op tracker -- , seq: 1176, time: 2014-08-13 16:24:29.202485, event: done, request: pg_notify(22.3d9(6) epoch 194454) v5 -882> 2014-08-13 16:24:29.202539 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 40 ==== pg_notify(24.3d(18) epoch 194454) v5 ==== 1754+0+0 (4180648974 0 0) 0x84088c0 con 0x51135a0 -881> 2014-08-13 16:24:29.202559 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:28.952934, event: header_read, request: pg_notify(24.3d(18) epoch 194454) v5 -880> 2014-08-13 16:24:29.202570 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:28.952936, event: throttled, request: pg_notify(24.3d(18) epoch 194454) v5 -879> 2014-08-13 16:24:29.202579 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:28.953035, event: all_read, request: pg_notify(24.3d(18) epoch 194454) v5 -878> 2014-08-13 16:24:29.202588 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:29.202557, event: dispatched, request: pg_notify(24.3d(18) epoch 194454) v5 -877> 2014-08-13 16:24:29.202597 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:29.202597, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194454) v5 -876> 2014-08-13 16:24:29.202608 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:29.202608, event: started, request: pg_notify(24.3d(18) epoch 194454) v5 -875> 2014-08-13 16:24:29.202634 7fd93dff0700 5 -- op tracker -- , seq: 1177, time: 2014-08-13 16:24:29.202634, event: done, request: pg_notify(24.3d(18) epoch 194454) v5 -874> 2014-08-13 16:24:29.202657 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 32 ==== pg_notify(22.48e(15) epoch 194454) v5 ==== 1551+0+0 (1665324609 0 0) 0x840d5c0 con 0x5111a20 -873> 2014-08-13 16:24:29.202676 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:28.950116, event: header_read, request: pg_notify(22.48e(15) epoch 194454) v5 -872> 2014-08-13 16:24:29.202686 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:28.950117, event: throttled, request: pg_notify(22.48e(15) epoch 194454) v5 -871> 2014-08-13 16:24:29.202695 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:28.950225, event: all_read, request: pg_notify(22.48e(15) epoch 194454) v5 -870> 2014-08-13 16:24:29.202704 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:29.202674, event: dispatched, request: pg_notify(22.48e(15) epoch 194454) v5 -869> 2014-08-13 16:24:29.202713 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:29.202713, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194454) v5 -868> 2014-08-13 16:24:29.202724 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:29.202724, event: started, request: pg_notify(22.48e(15) epoch 194454) v5 -867> 2014-08-13 16:24:29.202750 7fd93dff0700 5 -- op tracker -- , seq: 1178, time: 2014-08-13 16:24:29.202750, event: done, request: pg_notify(22.48e(15) epoch 194454) v5 -866> 2014-08-13 16:24:29.202799 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 27 ==== pg_notify(22.362(17) epoch 194454) v5 ==== 1665+0+0 (1256750180 0 0) 0x83488c0 con 0x7313860 -865> 2014-08-13 16:24:29.202820 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:28.931222, event: header_read, request: pg_notify(22.362(17) epoch 194454) v5 -864> 2014-08-13 16:24:29.202830 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:28.931223, event: throttled, request: pg_notify(22.362(17) epoch 194454) v5 -863> 2014-08-13 16:24:29.202839 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:28.931311, event: all_read, request: pg_notify(22.362(17) epoch 194454) v5 -862> 2014-08-13 16:24:29.202848 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:29.202817, event: dispatched, request: pg_notify(22.362(17) epoch 194454) v5 -861> 2014-08-13 16:24:29.202858 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:29.202858, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194454) v5 -860> 2014-08-13 16:24:29.202868 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:29.202868, event: started, request: pg_notify(22.362(17) epoch 194454) v5 -859> 2014-08-13 16:24:29.202907 7fd93dff0700 5 -- op tracker -- , seq: 1179, time: 2014-08-13 16:24:29.202907, event: done, request: pg_notify(22.362(17) epoch 194454) v5 -858> 2014-08-13 16:24:29.202966 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 20 ==== pg_notify(22.22c(17) epoch 194454) v5 ==== 1665+0+0 (4230026254 0 0) 0x775ec80 con 0x7316720 -857> 2014-08-13 16:24:29.202987 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:28.954678, event: header_read, request: pg_notify(22.22c(17) epoch 194454) v5 -856> 2014-08-13 16:24:29.202997 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:28.954679, event: throttled, request: pg_notify(22.22c(17) epoch 194454) v5 -855> 2014-08-13 16:24:29.203006 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:28.954770, event: all_read, request: pg_notify(22.22c(17) epoch 194454) v5 -854> 2014-08-13 16:24:29.203015 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:29.202984, event: dispatched, request: pg_notify(22.22c(17) epoch 194454) v5 -853> 2014-08-13 16:24:29.203025 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:29.203025, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194454) v5 -852> 2014-08-13 16:24:29.203036 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:29.203036, event: started, request: pg_notify(22.22c(17) epoch 194454) v5 -851> 2014-08-13 16:24:29.203069 7fd93dff0700 5 -- op tracker -- , seq: 1180, time: 2014-08-13 16:24:29.203069, event: done, request: pg_notify(22.22c(17) epoch 194454) v5 -850> 2014-08-13 16:24:29.203127 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 19 ==== pg_notify(24.39a(28) epoch 194454) v5 ==== 2300+0+0 (160419973 0 0) 0x8c88540 con 0x7315ee0 -849> 2014-08-13 16:24:29.203156 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:28.966460, event: header_read, request: pg_notify(24.39a(28) epoch 194454) v5 -848> 2014-08-13 16:24:29.203167 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:28.966461, event: throttled, request: pg_notify(24.39a(28) epoch 194454) v5 -847> 2014-08-13 16:24:29.203176 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:28.966579, event: all_read, request: pg_notify(24.39a(28) epoch 194454) v5 -846> 2014-08-13 16:24:29.203181 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:29.203146, event: dispatched, request: pg_notify(24.39a(28) epoch 194454) v5 -845> 2014-08-13 16:24:29.203191 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:29.203190, event: waiting_for_osdmap, request: pg_notify(24.39a(28) epoch 194454) v5 -844> 2014-08-13 16:24:29.203209 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:29.203209, event: started, request: pg_notify(24.39a(28) epoch 194454) v5 -843> 2014-08-13 16:24:29.203238 7fd93dff0700 5 -- op tracker -- , seq: 1181, time: 2014-08-13 16:24:29.203237, event: done, request: pg_notify(24.39a(28) epoch 194454) v5 -842> 2014-08-13 16:24:29.203290 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 22 ==== pg_notify(22.707(19) epoch 194454) v5 ==== 1783+0+0 (2155717912 0 0) 0x899c440 con 0x7314fc0 -841> 2014-08-13 16:24:29.203322 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:28.931491, event: header_read, request: pg_notify(22.707(19) epoch 194454) v5 -840> 2014-08-13 16:24:29.203333 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:28.931493, event: throttled, request: pg_notify(22.707(19) epoch 194454) v5 -839> 2014-08-13 16:24:29.203339 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:28.931603, event: all_read, request: pg_notify(22.707(19) epoch 194454) v5 -838> 2014-08-13 16:24:29.203345 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:29.203319, event: dispatched, request: pg_notify(22.707(19) epoch 194454) v5 -837> 2014-08-13 16:24:29.203351 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:29.203351, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194454) v5 -836> 2014-08-13 16:24:29.203359 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:29.203359, event: started, request: pg_notify(22.707(19) epoch 194454) v5 -835> 2014-08-13 16:24:29.203384 7fd93dff0700 5 -- op tracker -- , seq: 1182, time: 2014-08-13 16:24:29.203384, event: done, request: pg_notify(22.707(19) epoch 194454) v5 -834> 2014-08-13 16:24:29.203431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 19 ==== pg_notify(22.39d(15) epoch 194454) v5 ==== 1551+0+0 (4291772463 0 0) 0x9110a80 con 0x72d9b80 -833> 2014-08-13 16:24:29.203452 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:28.944013, event: header_read, request: pg_notify(22.39d(15) epoch 194454) v5 -832> 2014-08-13 16:24:29.203462 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:28.944015, event: throttled, request: pg_notify(22.39d(15) epoch 194454) v5 -831> 2014-08-13 16:24:29.203468 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:28.944118, event: all_read, request: pg_notify(22.39d(15) epoch 194454) v5 -830> 2014-08-13 16:24:29.203473 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:29.203449, event: dispatched, request: pg_notify(22.39d(15) epoch 194454) v5 -829> 2014-08-13 16:24:29.203479 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:29.203479, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194454) v5 -828> 2014-08-13 16:24:29.203487 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:29.203486, event: started, request: pg_notify(22.39d(15) epoch 194454) v5 -827> 2014-08-13 16:24:29.203513 7fd93dff0700 5 -- op tracker -- , seq: 1183, time: 2014-08-13 16:24:29.203513, event: done, request: pg_notify(22.39d(15) epoch 194454) v5 -826> 2014-08-13 16:24:29.203545 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 27 ==== pg_notify(22.775(4) epoch 194454) v5 ==== 942+0+0 (4113160515 0 0) 0x91101c0 con 0x8e286e0 -825> 2014-08-13 16:24:29.203564 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.046810, event: header_read, request: pg_notify(22.775(4) epoch 194454) v5 -824> 2014-08-13 16:24:29.203575 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.046813, event: throttled, request: pg_notify(22.775(4) epoch 194454) v5 -823> 2014-08-13 16:24:29.203584 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.046934, event: all_read, request: pg_notify(22.775(4) epoch 194454) v5 -822> 2014-08-13 16:24:29.203593 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.203562, event: dispatched, request: pg_notify(22.775(4) epoch 194454) v5 -821> 2014-08-13 16:24:29.203603 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.203602, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194454) v5 -820> 2014-08-13 16:24:29.203613 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.203613, event: started, request: pg_notify(22.775(4) epoch 194454) v5 -819> 2014-08-13 16:24:29.203639 7fd93dff0700 5 -- op tracker -- , seq: 1184, time: 2014-08-13 16:24:29.203638, event: done, request: pg_notify(22.775(4) epoch 194454) v5 -818> 2014-08-13 16:24:29.203682 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 22 ==== pg_notify(22.481(27) epoch 194454) v5 ==== 2235+0+0 (2589814732 0 0) 0x7759180 con 0x7723440 -817> 2014-08-13 16:24:29.203702 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:28.937867, event: header_read, request: pg_notify(22.481(27) epoch 194454) v5 -816> 2014-08-13 16:24:29.203713 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:28.937870, event: throttled, request: pg_notify(22.481(27) epoch 194454) v5 -815> 2014-08-13 16:24:29.203722 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:28.937990, event: all_read, request: pg_notify(22.481(27) epoch 194454) v5 -814> 2014-08-13 16:24:29.203731 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:29.203700, event: dispatched, request: pg_notify(22.481(27) epoch 194454) v5 -813> 2014-08-13 16:24:29.203740 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:29.203740, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194454) v5 -812> 2014-08-13 16:24:29.203751 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:29.203751, event: started, request: pg_notify(22.481(27) epoch 194454) v5 -811> 2014-08-13 16:24:29.203778 7fd93dff0700 5 -- op tracker -- , seq: 1185, time: 2014-08-13 16:24:29.203777, event: done, request: pg_notify(22.481(27) epoch 194454) v5 -810> 2014-08-13 16:24:29.203836 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 26 ==== pg_query(24.2c epoch 194454) v3 ==== 144+0+0 (2839370746 0 0) 0x801aee0 con 0x4fe1600 -809> 2014-08-13 16:24:29.203857 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:28.932082, event: header_read, request: pg_query(24.2c epoch 194454) v3 -808> 2014-08-13 16:24:29.203868 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:28.932083, event: throttled, request: pg_query(24.2c epoch 194454) v3 -807> 2014-08-13 16:24:29.203877 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:28.932128, event: all_read, request: pg_query(24.2c epoch 194454) v3 -806> 2014-08-13 16:24:29.203885 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:29.203855, event: dispatched, request: pg_query(24.2c epoch 194454) v3 -805> 2014-08-13 16:24:29.203895 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:29.203895, event: waiting_for_osdmap, request: pg_query(24.2c epoch 194454) v3 -804> 2014-08-13 16:24:29.203906 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:29.203906, event: started, request: pg_query(24.2c epoch 194454) v3 -803> 2014-08-13 16:24:29.203930 7fd93dff0700 5 -- op tracker -- , seq: 1186, time: 2014-08-13 16:24:29.203930, event: done, request: pg_query(24.2c epoch 194454) v3 -802> 2014-08-13 16:24:29.203967 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.328 192.168.1.104:6812/64303 25 ==== osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (4174901843 0 0) 0x8bc0000 con 0x4fe3700 -801> 2014-08-13 16:24:29.203993 7fd93dff0700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.202510, event: header_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -800> 2014-08-13 16:24:29.204008 7fd93dff0700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.202512, event: throttled, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -799> 2014-08-13 16:24:29.204021 7fd93dff0700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.202612, event: all_read, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -798> 2014-08-13 16:24:29.204033 7fd93dff0700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.203991, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -797> 2014-08-13 16:24:29.204047 7fd93dff0700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.204047, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -796> 2014-08-13 16:24:29.204042 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(27) epoch 194454) v5 -- ?+0 0x9284ec0 con 0x4fe1600 -795> 2014-08-13 16:24:29.204080 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 28 ==== pg_notify(22.362(4) epoch 194454) v5 ==== 942+0+0 (1746002068 0 0) 0x9116040 con 0x4fe40a0 -794> 2014-08-13 16:24:29.204100 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:28.940555, event: header_read, request: pg_notify(22.362(4) epoch 194454) v5 -793> 2014-08-13 16:24:29.204113 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:28.940559, event: throttled, request: pg_notify(22.362(4) epoch 194454) v5 -792> 2014-08-13 16:24:29.204126 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:28.940705, event: all_read, request: pg_notify(22.362(4) epoch 194454) v5 -791> 2014-08-13 16:24:29.204139 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:29.204097, event: dispatched, request: pg_notify(22.362(4) epoch 194454) v5 -790> 2014-08-13 16:24:29.204152 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:29.204152, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194454) v5 -789> 2014-08-13 16:24:29.204164 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:29.204164, event: started, request: pg_notify(22.362(4) epoch 194454) v5 -788> 2014-08-13 16:24:29.204180 7fd9371e5700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.204103, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -787> 2014-08-13 16:24:29.204225 7fd9371e5700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.204225, event: started, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -786> 2014-08-13 16:24:29.204248 7fd9371e5700 5 -- op tracker -- , seq: 1187, time: 2014-08-13 16:24:29.204248, event: done, request: osd_sub_op(unknown.0.0:0 22.1a0 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -785> 2014-08-13 16:24:29.204264 7fd93dff0700 5 -- op tracker -- , seq: 1188, time: 2014-08-13 16:24:29.204193, event: done, request: pg_notify(22.362(4) epoch 194454) v5 -784> 2014-08-13 16:24:29.204305 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 31 ==== pg_notify(24.181(9) epoch 194454) v5 ==== 1263+0+0 (523358904 0 0) 0x899c280 con 0x50889a0 -783> 2014-08-13 16:24:29.204332 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:28.944821, event: header_read, request: pg_notify(24.181(9) epoch 194454) v5 -782> 2014-08-13 16:24:29.204342 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:28.944823, event: throttled, request: pg_notify(24.181(9) epoch 194454) v5 -781> 2014-08-13 16:24:29.204351 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:28.944929, event: all_read, request: pg_notify(24.181(9) epoch 194454) v5 -780> 2014-08-13 16:24:29.204360 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:29.204328, event: dispatched, request: pg_notify(24.181(9) epoch 194454) v5 -779> 2014-08-13 16:24:29.204369 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:29.204369, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194454) v5 -778> 2014-08-13 16:24:29.204381 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:29.204380, event: started, request: pg_notify(24.181(9) epoch 194454) v5 -777> 2014-08-13 16:24:29.204410 7fd93dff0700 5 -- op tracker -- , seq: 1189, time: 2014-08-13 16:24:29.204410, event: done, request: pg_notify(24.181(9) epoch 194454) v5 -776> 2014-08-13 16:24:29.204431 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 30 ==== pg_notify(22.3e4(4) epoch 194454) v5 ==== 942+0+0 (247966872 0 0) 0x9111c00 con 0x5108b00 -775> 2014-08-13 16:24:29.204450 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:28.948433, event: header_read, request: pg_notify(22.3e4(4) epoch 194454) v5 -774> 2014-08-13 16:24:29.204460 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:28.948437, event: throttled, request: pg_notify(22.3e4(4) epoch 194454) v5 -773> 2014-08-13 16:24:29.204469 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:28.948533, event: all_read, request: pg_notify(22.3e4(4) epoch 194454) v5 -772> 2014-08-13 16:24:29.204478 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:29.204448, event: dispatched, request: pg_notify(22.3e4(4) epoch 194454) v5 -771> 2014-08-13 16:24:29.204488 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:29.204488, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194454) v5 -770> 2014-08-13 16:24:29.204499 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:29.204499, event: started, request: pg_notify(22.3e4(4) epoch 194454) v5 -769> 2014-08-13 16:24:29.204524 7fd93dff0700 5 -- op tracker -- , seq: 1190, time: 2014-08-13 16:24:29.204524, event: done, request: pg_notify(22.3e4(4) epoch 194454) v5 -768> 2014-08-13 16:24:29.204543 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 23 ==== pg_notify(24.b6(4) epoch 194454) v5 ==== 976+0+0 (3363419787 0 0) 0x9116200 con 0x5113860 -767> 2014-08-13 16:24:29.204561 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:28.943034, event: header_read, request: pg_notify(24.b6(4) epoch 194454) v5 -766> 2014-08-13 16:24:29.204571 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:28.943037, event: throttled, request: pg_notify(24.b6(4) epoch 194454) v5 -765> 2014-08-13 16:24:29.204580 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:28.943185, event: all_read, request: pg_notify(24.b6(4) epoch 194454) v5 -764> 2014-08-13 16:24:29.204589 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:29.204559, event: dispatched, request: pg_notify(24.b6(4) epoch 194454) v5 -763> 2014-08-13 16:24:29.204599 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:29.204599, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194454) v5 -762> 2014-08-13 16:24:29.204610 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:29.204609, event: started, request: pg_notify(24.b6(4) epoch 194454) v5 -761> 2014-08-13 16:24:29.204635 7fd93dff0700 5 -- op tracker -- , seq: 1191, time: 2014-08-13 16:24:29.204635, event: done, request: pg_notify(24.b6(4) epoch 194454) v5 -760> 2014-08-13 16:24:29.204654 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 41 ==== pg_notify(22.3d9(32) epoch 194454) v5 ==== 2458+0+0 (2986495039 0 0) 0x840c280 con 0x51135a0 -759> 2014-08-13 16:24:29.204672 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:28.954916, event: header_read, request: pg_notify(22.3d9(32) epoch 194454) v5 -758> 2014-08-13 16:24:29.204682 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:28.954918, event: throttled, request: pg_notify(22.3d9(32) epoch 194454) v5 -757> 2014-08-13 16:24:29.204692 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:28.955037, event: all_read, request: pg_notify(22.3d9(32) epoch 194454) v5 -756> 2014-08-13 16:24:29.204701 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:29.204670, event: dispatched, request: pg_notify(22.3d9(32) epoch 194454) v5 -755> 2014-08-13 16:24:29.204710 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:29.204710, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194454) v5 -754> 2014-08-13 16:24:29.204721 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:29.204721, event: started, request: pg_notify(22.3d9(32) epoch 194454) v5 -753> 2014-08-13 16:24:29.204746 7fd93dff0700 5 -- op tracker -- , seq: 1192, time: 2014-08-13 16:24:29.204746, event: done, request: pg_notify(22.3d9(32) epoch 194454) v5 -752> 2014-08-13 16:24:29.204771 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 33 ==== pg_notify(24.1f7(18) epoch 194454) v5 ==== 1754+0+0 (2119986098 0 0) 0x842d940 con 0x5111a20 -751> 2014-08-13 16:24:29.204789 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:28.950244, event: header_read, request: pg_notify(24.1f7(18) epoch 194454) v5 -750> 2014-08-13 16:24:29.204799 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:28.950245, event: throttled, request: pg_notify(24.1f7(18) epoch 194454) v5 -749> 2014-08-13 16:24:29.204809 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:28.950344, event: all_read, request: pg_notify(24.1f7(18) epoch 194454) v5 -748> 2014-08-13 16:24:29.204817 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:29.204787, event: dispatched, request: pg_notify(24.1f7(18) epoch 194454) v5 -747> 2014-08-13 16:24:29.204827 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:29.204827, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194454) v5 -746> 2014-08-13 16:24:29.204838 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:29.204838, event: started, request: pg_notify(24.1f7(18) epoch 194454) v5 -745> 2014-08-13 16:24:29.204864 7fd93dff0700 5 -- op tracker -- , seq: 1193, time: 2014-08-13 16:24:29.204864, event: done, request: pg_notify(24.1f7(18) epoch 194454) v5 -744> 2014-08-13 16:24:29.204885 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 21 ==== pg_notify(22.1f1(17) epoch 194454) v5 ==== 1665+0+0 (4213413740 0 0) 0x775f540 con 0x7316720 -743> 2014-08-13 16:24:29.204904 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:28.956066, event: header_read, request: pg_notify(22.1f1(17) epoch 194454) v5 -742> 2014-08-13 16:24:29.204914 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:28.956070, event: throttled, request: pg_notify(22.1f1(17) epoch 194454) v5 -741> 2014-08-13 16:24:29.204923 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:28.956204, event: all_read, request: pg_notify(22.1f1(17) epoch 194454) v5 -740> 2014-08-13 16:24:29.204932 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:29.204902, event: dispatched, request: pg_notify(22.1f1(17) epoch 194454) v5 -739> 2014-08-13 16:24:29.204942 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:29.204942, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194454) v5 -738> 2014-08-13 16:24:29.204953 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:29.204952, event: started, request: pg_notify(22.1f1(17) epoch 194454) v5 -737> 2014-08-13 16:24:29.204978 7fd93dff0700 5 -- op tracker -- , seq: 1194, time: 2014-08-13 16:24:29.204977, event: done, request: pg_notify(22.1f1(17) epoch 194454) v5 -736> 2014-08-13 16:24:29.205007 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 28 ==== pg_notify(22.f3(15) epoch 194454) v5 ==== 1551+0+0 (1605595647 0 0) 0x775a140 con 0x8e286e0 -735> 2014-08-13 16:24:29.205027 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.047666, event: header_read, request: pg_notify(22.f3(15) epoch 194454) v5 -734> 2014-08-13 16:24:29.205037 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.047669, event: throttled, request: pg_notify(22.f3(15) epoch 194454) v5 -733> 2014-08-13 16:24:29.205047 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.047837, event: all_read, request: pg_notify(22.f3(15) epoch 194454) v5 -732> 2014-08-13 16:24:29.205056 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.205025, event: dispatched, request: pg_notify(22.f3(15) epoch 194454) v5 -731> 2014-08-13 16:24:29.205065 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.205065, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194454) v5 -730> 2014-08-13 16:24:29.205076 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.205076, event: started, request: pg_notify(22.f3(15) epoch 194454) v5 -729> 2014-08-13 16:24:29.205102 7fd93dff0700 5 -- op tracker -- , seq: 1195, time: 2014-08-13 16:24:29.205101, event: done, request: pg_notify(22.f3(15) epoch 194454) v5 -728> 2014-08-13 16:24:29.205134 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 27 ==== pg_query(24.2c epoch 194454) v3 ==== 144+0+0 (1802397237 0 0) 0x801ecc0 con 0x4fe1600 -727> 2014-08-13 16:24:29.205154 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.202180, event: header_read, request: pg_query(24.2c epoch 194454) v3 -726> 2014-08-13 16:24:29.205164 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.202182, event: throttled, request: pg_query(24.2c epoch 194454) v3 -725> 2014-08-13 16:24:29.205173 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.202261, event: all_read, request: pg_query(24.2c epoch 194454) v3 -724> 2014-08-13 16:24:29.205190 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.205152, event: dispatched, request: pg_query(24.2c epoch 194454) v3 -723> 2014-08-13 16:24:29.205208 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.205208, event: waiting_for_osdmap, request: pg_query(24.2c epoch 194454) v3 -722> 2014-08-13 16:24:29.205220 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.205220, event: started, request: pg_query(24.2c epoch 194454) v3 -721> 2014-08-13 16:24:29.205245 7fd93dff0700 5 -- op tracker -- , seq: 1196, time: 2014-08-13 16:24:29.205245, event: done, request: pg_query(24.2c epoch 194454) v3 -720> 2014-08-13 16:24:29.205263 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 31 ==== pg_notify(22.481(27) epoch 194454) v5 ==== 2235+0+0 (3597901183 0 0) 0x842f380 con 0x5108b00 -719> 2014-08-13 16:24:29.205283 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:28.950088, event: header_read, request: pg_notify(22.481(27) epoch 194454) v5 -718> 2014-08-13 16:24:29.205292 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:28.950092, event: throttled, request: pg_notify(22.481(27) epoch 194454) v5 -717> 2014-08-13 16:24:29.205302 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:28.950242, event: all_read, request: pg_notify(22.481(27) epoch 194454) v5 -716> 2014-08-13 16:24:29.205310 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:29.205280, event: dispatched, request: pg_notify(22.481(27) epoch 194454) v5 -715> 2014-08-13 16:24:29.205320 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:29.205320, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194454) v5 -714> 2014-08-13 16:24:29.205308 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_log(24.2c epoch 194454 log log((194422'15445,194422'15445], crt=194422'15445) query_epoch 194454) v4 -- ?+0 0x866bc00 con 0x4fe1600 -713> 2014-08-13 16:24:29.205328 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:29.205328, event: started, request: pg_notify(22.481(27) epoch 194454) v5 -712> 2014-08-13 16:24:29.205367 7fd93dff0700 5 -- op tracker -- , seq: 1197, time: 2014-08-13 16:24:29.205367, event: done, request: pg_notify(22.481(27) epoch 194454) v5 -711> 2014-08-13 16:24:29.207625 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.373 192.168.1.108:0/7435 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.207393) v2 ==== 47+0+0 (2704447715 0 0) 0x842c440 con 0x7eb8f20 -710> 2014-08-13 16:24:29.207673 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.207393) v2 -- ?+0 0x9107000 con 0x7eb8f20 -709> 2014-08-13 16:24:29.207715 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.373 192.168.1.108:0/7435 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.207393) v2 ==== 47+0+0 (2704447715 0 0) 0x899abc0 con 0x7eba3c0 -708> 2014-08-13 16:24:29.207762 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.207393) v2 -- ?+0 0x9106900 con 0x7eba3c0 -707> 2014-08-13 16:24:29.223350 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.20 192.168.1.103:0/38269 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.222902) v2 ==== 47+0+0 (2504440657 0 0) 0x775e580 con 0x7ebe5c0 -706> 2014-08-13 16:24:29.223381 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.20 192.168.1.103:0/38269 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.222902) v2 ==== 47+0+0 (2504440657 0 0) 0x775e740 con 0x7ed8580 -705> 2014-08-13 16:24:29.223397 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.222902) v2 -- ?+0 0x842c440 con 0x7ebe5c0 -704> 2014-08-13 16:24:29.223511 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.222902) v2 -- ?+0 0x899abc0 con 0x7ed8580 -703> 2014-08-13 16:24:29.247394 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.367 192.168.100.114:0/55513 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.247244) v2 ==== 47+0+0 (3257562761 0 0) 0x84288c0 con 0x7ffe880 -702> 2014-08-13 16:24:29.247409 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.367 192.168.100.114:0/55513 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.247244) v2 ==== 47+0+0 (3257562761 0 0) 0x899b9c0 con 0x7edbb20 -701> 2014-08-13 16:24:29.247445 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.247244) v2 -- ?+0 0x775e580 con 0x7ffe880 -700> 2014-08-13 16:24:29.247601 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.247244) v2 -- ?+0 0x775e740 con 0x7edbb20 -699> 2014-08-13 16:24:29.251771 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.321 192.168.1.110:0/43236 10 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.251300) v2 ==== 47+0+0 (3097485203 0 0) 0x8c8c280 con 0x7eba100 -698> 2014-08-13 16:24:29.251821 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.251300) v2 -- ?+0 0x899b9c0 con 0x7eba100 -697> 2014-08-13 16:24:29.251804 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.321 192.168.1.110:0/43236 10 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.251300) v2 ==== 47+0+0 (3097485203 0 0) 0x9284440 con 0x7eb8c60 -696> 2014-08-13 16:24:29.251864 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.251300) v2 -- ?+0 0x84288c0 con 0x7eb8c60 -695> 2014-08-13 16:24:29.294756 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.50 192.168.1.112:6842/9607 17 ==== osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+4650 (804364339 0 3752130531) 0x8638b00 con 0x508c620 -694> 2014-08-13 16:24:29.294809 7fd93dff0700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294540, event: header_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -693> 2014-08-13 16:24:29.294825 7fd93dff0700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294544, event: throttled, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -692> 2014-08-13 16:24:29.294835 7fd93dff0700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294669, event: all_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -691> 2014-08-13 16:24:29.294844 7fd93dff0700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294803, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -690> 2014-08-13 16:24:29.294855 7fd93dff0700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294855, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -689> 2014-08-13 16:24:29.294880 7fd93dff0700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.112:6842/9607 -- osd_map(194454..194454 src has 185139..194454) v3 -- ?+0 0x7e7ca40 con 0x508c620 -688> 2014-08-13 16:24:29.294987 7fd9371e5700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.294986, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -687> 2014-08-13 16:24:29.295030 7fd9371e5700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.295029, event: started, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -686> 2014-08-13 16:24:29.295100 7fd9371e5700 5 -- op tracker -- , seq: 1198, time: 2014-08-13 16:24:29.295100, event: done, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -685> 2014-08-13 16:24:29.433200 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.376 192.168.1.108:0/6893 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.432842) v2 ==== 47+0+0 (3688997018 0 0) 0x9285e80 con 0x7d29ce0 -684> 2014-08-13 16:24:29.433248 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.432842) v2 -- ?+0 0x8c8c280 con 0x7d29ce0 -683> 2014-08-13 16:24:29.433303 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.376 192.168.1.108:0/6893 8 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.432842) v2 ==== 47+0+0 (3688997018 0 0) 0x9282f40 con 0x7d2b440 -682> 2014-08-13 16:24:29.433356 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.432842) v2 -- ?+0 0x9284440 con 0x7d2b440 -681> 2014-08-13 16:24:29.502024 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.330 192.168.1.106:0/4876 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.501448) v2 ==== 47+0+0 (3271197813 0 0) 0x842c280 con 0x7ebf380 -680> 2014-08-13 16:24:29.502075 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.501448) v2 -- ?+0 0x9282f40 con 0x7ebf380 -679> 2014-08-13 16:24:29.502149 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.330 192.168.1.106:0/4876 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.501448) v2 ==== 47+0+0 (3271197813 0 0) 0x842e200 con 0x7ebfe80 -678> 2014-08-13 16:24:29.502208 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.501448) v2 -- ?+0 0x9285e80 con 0x7ebfe80 -677> 2014-08-13 16:24:29.623459 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.364 192.168.100.114:0/52317 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.623429) v2 ==== 47+0+0 (4164286705 0 0) 0x9284600 con 0x7ebf0c0 -676> 2014-08-13 16:24:29.623513 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.623429) v2 -- ?+0 0x842c280 con 0x7ebf0c0 -675> 2014-08-13 16:24:29.623511 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.364 192.168.100.114:0/52317 7 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.623429) v2 ==== 47+0+0 (4164286705 0 0) 0x834d5c0 con 0x7ed9080 -674> 2014-08-13 16:24:29.623555 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.623429) v2 -- ?+0 0x842e200 con 0x7ed9080 -673> 2014-08-13 16:24:29.638786 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.289 192.168.1.113:6808/15284 21 ==== osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+4650 (3835643092 0 3752130531) 0x8638b00 con 0x4fdb2e0 -672> 2014-08-13 16:24:29.638844 7fd93dff0700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.638562, event: header_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -671> 2014-08-13 16:24:29.638864 7fd93dff0700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.638566, event: throttled, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -670> 2014-08-13 16:24:29.638878 7fd93dff0700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.638697, event: all_read, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -669> 2014-08-13 16:24:29.638891 7fd93dff0700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.638836, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -668> 2014-08-13 16:24:29.638905 7fd93dff0700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.638905, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -667> 2014-08-13 16:24:29.638931 7fd93dff0700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6808/15284 -- osd_map(194454..194454 src has 185139..194454) v3 -- ?+0 0x8bd72c0 con 0x4fdb2e0 -666> 2014-08-13 16:24:29.639044 7fd9367e4700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.639044, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -665> 2014-08-13 16:24:29.639087 7fd9367e4700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.639086, event: started, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -664> 2014-08-13 16:24:29.639158 7fd9367e4700 5 -- op tracker -- , seq: 1199, time: 2014-08-13 16:24:29.639158, event: done, request: osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -663> 2014-08-13 16:24:29.639215 7fd9353e2700 2 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub osd.4 has 10 items -662> 2014-08-13 16:24:29.639353 7fd9353e2700 2 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 50 has 10 items -661> 2014-08-13 16:24:29.639376 7fd9353e2700 2 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] scrub replica 289 has 10 items -660> 2014-08-13 16:24:29.639735 7fd9353e2700 2 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194448 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep] -659> 2014-08-13 16:24:29.639835 7fd9353e2700 0 log [INF] : 22.6bd deep-scrub ok -658> 2014-08-13 16:24:29.640031 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x7972100 -657> 2014-08-13 16:24:29.640106 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- osd_sub_op(unknown.0.0:0 22.6bd 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+0 0x86d8b00 -656> 2014-08-13 16:24:29.640318 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194454:22.6bd) v4 -- ?+0 0x85755c0 -655> 2014-08-13 16:24:29.640411 7fd9353e2700 1 -- 192.168.1.115:6813/25220 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194454:22.6bd) v4 -- ?+0 0x8577000 -654> 2014-08-13 16:24:29.652686 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.343 192.168.1.104:0/61960 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.652605) v2 ==== 47+0+0 (2906906078 0 0) 0x8eed5c0 con 0x7d2aec0 -653> 2014-08-13 16:24:29.652739 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.652605) v2 -- ?+0 0x9284600 con 0x7d2aec0 -652> 2014-08-13 16:24:29.652787 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.343 192.168.1.104:0/61960 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.652605) v2 ==== 47+0+0 (2906906078 0 0) 0x4f8aa00 con 0x7d2c620 -651> 2014-08-13 16:24:29.652830 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.652605) v2 -- ?+0 0x834d5c0 con 0x7d2c620 -650> 2014-08-13 16:24:29.652960 7fd94ac49700 5 osd.4 194454 tick -649> 2014-08-13 16:24:29.715327 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.17 192.168.100.103:0/37658 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.715090) v2 ==== 47+0+0 (3897369887 0 0) 0x834a140 con 0x7edc360 -648> 2014-08-13 16:24:29.715377 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.715090) v2 -- ?+0 0x4f8aa00 con 0x7edc360 -647> 2014-08-13 16:24:29.715410 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.17 192.168.100.103:0/37658 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.715090) v2 ==== 47+0+0 (3897369887 0 0) 0x834b100 con 0x7eda3c0 -646> 2014-08-13 16:24:29.715470 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.715090) v2 -- ?+0 0x8eed5c0 con 0x7eda3c0 -645> 2014-08-13 16:24:29.750794 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.358 192.168.100.114:0/52078 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.750678) v2 ==== 47+0+0 (1404043359 0 0) 0x9111880 con 0x7d2b700 -644> 2014-08-13 16:24:29.750797 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.358 192.168.100.114:0/52078 9 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.750678) v2 ==== 47+0+0 (1404043359 0 0) 0x91108c0 con 0x7d29fa0 -643> 2014-08-13 16:24:29.750852 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.750678) v2 -- ?+0 0x834b100 con 0x7d2b700 -642> 2014-08-13 16:24:29.750971 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.750678) v2 -- ?+0 0x834a140 con 0x7d29fa0 -641> 2014-08-13 16:24:29.924481 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.356 192.168.1.108:0/5235 10 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.924207) v2 ==== 47+0+0 (2431212307 0 0) 0x7d01f80 con 0x7eda680 -640> 2014-08-13 16:24:29.924532 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.924207) v2 -- ?+0 0x9111880 con 0x7eda680 -639> 2014-08-13 16:24:29.924543 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.356 192.168.1.108:0/5235 10 ==== osd_ping(ping e194454 stamp 2014-08-13 16:24:29.924207) v2 ==== 47+0+0 (2431212307 0 0) 0x8c8f1c0 con 0x7edc620 -638> 2014-08-13 16:24:29.924592 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194454 stamp 2014-08-13 16:24:29.924207) v2 -- ?+0 0x91108c0 con 0x7edc620 -637> 2014-08-13 16:24:29.931762 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x33e3840 con 0x72dfa60 -636> 2014-08-13 16:24:29.931837 7fd93dff0700 3 osd.4 194454 handle_osd_map epochs [194455,194455], i have 194454, src has [185139,194455] -635> 2014-08-13 16:24:29.937470 7fd93dff0700 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for mons -634> 2014-08-13 16:24:29.939066 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.352 192.168.1.108:6826/4273 17 ==== pg_notify(22.4df(4) epoch 194455) v5 ==== 942+0+0 (497297413 0 0) 0x8409180 con 0x72dfa60 -633> 2014-08-13 16:24:29.939099 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.931709, event: header_read, request: pg_notify(22.4df(4) epoch 194455) v5 -632> 2014-08-13 16:24:29.939111 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.931710, event: throttled, request: pg_notify(22.4df(4) epoch 194455) v5 -631> 2014-08-13 16:24:29.939118 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.931784, event: all_read, request: pg_notify(22.4df(4) epoch 194455) v5 -630> 2014-08-13 16:24:29.939124 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.939095, event: dispatched, request: pg_notify(22.4df(4) epoch 194455) v5 -629> 2014-08-13 16:24:29.939146 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.939146, event: waiting_for_osdmap, request: pg_notify(22.4df(4) epoch 194455) v5 -628> 2014-08-13 16:24:29.939155 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.939155, event: started, request: pg_notify(22.4df(4) epoch 194455) v5 -627> 2014-08-13 16:24:29.939220 7fd93dff0700 5 -- op tracker -- , seq: 1200, time: 2014-08-13 16:24:29.939220, event: done, request: pg_notify(22.4df(4) epoch 194455) v5 -626> 2014-08-13 16:24:29.939252 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 23 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7e9c0 con 0x7723440 -625> 2014-08-13 16:24:29.939284 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -624> 2014-08-13 16:24:29.939295 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 28 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859e0c0 con 0x4fe1600 -623> 2014-08-13 16:24:29.939314 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -622> 2014-08-13 16:24:29.939322 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 29 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf2880 con 0x4fe40a0 -621> 2014-08-13 16:24:29.939336 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -620> 2014-08-13 16:24:29.939344 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 19 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf7740 con 0x5114620 -619> 2014-08-13 16:24:29.939361 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -618> 2014-08-13 16:24:29.939369 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 28 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d02400 con 0x7313860 -617> 2014-08-13 16:24:29.939386 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -616> 2014-08-13 16:24:29.939397 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 23 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf0900 con 0x7314fc0 -615> 2014-08-13 16:24:29.939414 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -614> 2014-08-13 16:24:29.939425 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 24 ==== pg_notify(24.9b(4) epoch 194455) v5 ==== 976+0+0 (604371314 0 0) 0x9284600 con 0x7723440 -613> 2014-08-13 16:24:29.939442 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.934217, event: header_read, request: pg_notify(24.9b(4) epoch 194455) v5 -612> 2014-08-13 16:24:29.939450 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.934218, event: throttled, request: pg_notify(24.9b(4) epoch 194455) v5 -611> 2014-08-13 16:24:29.939456 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.934293, event: all_read, request: pg_notify(24.9b(4) epoch 194455) v5 -610> 2014-08-13 16:24:29.939461 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.939440, event: dispatched, request: pg_notify(24.9b(4) epoch 194455) v5 -609> 2014-08-13 16:24:29.939468 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.939468, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194455) v5 -608> 2014-08-13 16:24:29.939476 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.939476, event: started, request: pg_notify(24.9b(4) epoch 194455) v5 -607> 2014-08-13 16:24:29.939497 7fd93dff0700 5 -- op tracker -- , seq: 1201, time: 2014-08-13 16:24:29.939497, event: done, request: pg_notify(24.9b(4) epoch 194455) v5 -606> 2014-08-13 16:24:29.939513 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 29 ==== pg_notify(22.99(15) epoch 194455) v5 ==== 1551+0+0 (1568506838 0 0) 0x842fe00 con 0x4fe1600 -605> 2014-08-13 16:24:29.939529 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.935234, event: header_read, request: pg_notify(22.99(15) epoch 194455) v5 -604> 2014-08-13 16:24:29.939535 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.935235, event: throttled, request: pg_notify(22.99(15) epoch 194455) v5 -603> 2014-08-13 16:24:29.939541 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.935328, event: all_read, request: pg_notify(22.99(15) epoch 194455) v5 -602> 2014-08-13 16:24:29.939546 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.939527, event: dispatched, request: pg_notify(22.99(15) epoch 194455) v5 -601> 2014-08-13 16:24:29.939552 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.939552, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194455) v5 -600> 2014-08-13 16:24:29.939559 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.939559, event: started, request: pg_notify(22.99(15) epoch 194455) v5 -599> 2014-08-13 16:24:29.939577 7fd93dff0700 5 -- op tracker -- , seq: 1202, time: 2014-08-13 16:24:29.939577, event: done, request: pg_notify(22.99(15) epoch 194455) v5 -598> 2014-08-13 16:24:29.939594 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 30 ==== pg_notify(22.3af(4) epoch 194455) v5 ==== 942+0+0 (3164660 0 0) 0x8eef000 con 0x4fe40a0 -597> 2014-08-13 16:24:29.939607 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.938025, event: header_read, request: pg_notify(22.3af(4) epoch 194455) v5 -596> 2014-08-13 16:24:29.939614 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.938026, event: throttled, request: pg_notify(22.3af(4) epoch 194455) v5 -595> 2014-08-13 16:24:29.939619 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.938120, event: all_read, request: pg_notify(22.3af(4) epoch 194455) v5 -594> 2014-08-13 16:24:29.939625 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.939606, event: dispatched, request: pg_notify(22.3af(4) epoch 194455) v5 -593> 2014-08-13 16:24:29.939631 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.939631, event: waiting_for_osdmap, request: pg_notify(22.3af(4) epoch 194455) v5 -592> 2014-08-13 16:24:29.939638 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.939638, event: started, request: pg_notify(22.3af(4) epoch 194455) v5 -591> 2014-08-13 16:24:29.939656 7fd93dff0700 5 -- op tracker -- , seq: 1203, time: 2014-08-13 16:24:29.939656, event: done, request: pg_notify(22.3af(4) epoch 194455) v5 -590> 2014-08-13 16:24:29.939669 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 28 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d072c0 con 0x4fe69e0 -589> 2014-08-13 16:24:29.939685 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -588> 2014-08-13 16:24:29.939693 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.331 192.168.1.104:6816/837 20 ==== pg_notify(22.1d2(4) epoch 194455) v5 ==== 942+0+0 (4288165042 0 0) 0x842ec80 con 0x5114620 -587> 2014-08-13 16:24:29.939707 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.934409, event: header_read, request: pg_notify(22.1d2(4) epoch 194455) v5 -586> 2014-08-13 16:24:29.939713 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.934410, event: throttled, request: pg_notify(22.1d2(4) epoch 194455) v5 -585> 2014-08-13 16:24:29.939719 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.934475, event: all_read, request: pg_notify(22.1d2(4) epoch 194455) v5 -584> 2014-08-13 16:24:29.939724 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.939705, event: dispatched, request: pg_notify(22.1d2(4) epoch 194455) v5 -583> 2014-08-13 16:24:29.939730 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.939729, event: waiting_for_osdmap, request: pg_notify(22.1d2(4) epoch 194455) v5 -582> 2014-08-13 16:24:29.939737 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.939737, event: started, request: pg_notify(22.1d2(4) epoch 194455) v5 -581> 2014-08-13 16:24:29.939757 7fd93dff0700 5 -- op tracker -- , seq: 1204, time: 2014-08-13 16:24:29.939757, event: done, request: pg_notify(22.1d2(4) epoch 194455) v5 -580> 2014-08-13 16:24:29.939776 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 24 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd5c40 con 0x5113860 -579> 2014-08-13 16:24:29.939795 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -578> 2014-08-13 16:24:29.939808 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.349 192.168.1.104:6800/61726 29 ==== pg_notify(22.362(17) epoch 194455) v5 ==== 1665+0+0 (4019939463 0 0) 0x8db0000 con 0x7313860 -577> 2014-08-13 16:24:29.939825 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.933817, event: header_read, request: pg_notify(22.362(17) epoch 194455) v5 -576> 2014-08-13 16:24:29.939835 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.933818, event: throttled, request: pg_notify(22.362(17) epoch 194455) v5 -575> 2014-08-13 16:24:29.939844 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.933907, event: all_read, request: pg_notify(22.362(17) epoch 194455) v5 -574> 2014-08-13 16:24:29.939853 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.939823, event: dispatched, request: pg_notify(22.362(17) epoch 194455) v5 -573> 2014-08-13 16:24:29.939862 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.939862, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194455) v5 -572> 2014-08-13 16:24:29.939873 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.939873, event: started, request: pg_notify(22.362(17) epoch 194455) v5 -571> 2014-08-13 16:24:29.939896 7fd93dff0700 5 -- op tracker -- , seq: 1205, time: 2014-08-13 16:24:29.939896, event: done, request: pg_notify(22.362(17) epoch 194455) v5 -570> 2014-08-13 16:24:29.939916 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 24 ==== pg_notify(22.113(4) epoch 194455) v5 ==== 942+0+0 (1713836122 0 0) 0x4f8aa00 con 0x7314fc0 -569> 2014-08-13 16:24:29.939935 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.933719, event: header_read, request: pg_notify(22.113(4) epoch 194455) v5 -568> 2014-08-13 16:24:29.939946 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.933720, event: throttled, request: pg_notify(22.113(4) epoch 194455) v5 -567> 2014-08-13 16:24:29.939955 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.933793, event: all_read, request: pg_notify(22.113(4) epoch 194455) v5 -566> 2014-08-13 16:24:29.939964 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.939933, event: dispatched, request: pg_notify(22.113(4) epoch 194455) v5 -565> 2014-08-13 16:24:29.939973 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.939973, event: waiting_for_osdmap, request: pg_notify(22.113(4) epoch 194455) v5 -564> 2014-08-13 16:24:29.939984 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.939984, event: started, request: pg_notify(22.113(4) epoch 194455) v5 -563> 2014-08-13 16:24:29.940006 7fd93dff0700 5 -- op tracker -- , seq: 1206, time: 2014-08-13 16:24:29.940006, event: done, request: pg_notify(22.113(4) epoch 194455) v5 -562> 2014-08-13 16:24:29.940024 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.365 192.168.1.102:6824/15623 25 ==== pg_notify(22.481(27) epoch 194455) v5 ==== 2235+0+0 (1371554879 0 0) 0x834d5c0 con 0x7723440 -561> 2014-08-13 16:24:29.940042 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.935240, event: header_read, request: pg_notify(22.481(27) epoch 194455) v5 -560> 2014-08-13 16:24:29.940053 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.935242, event: throttled, request: pg_notify(22.481(27) epoch 194455) v5 -559> 2014-08-13 16:24:29.940062 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.935356, event: all_read, request: pg_notify(22.481(27) epoch 194455) v5 -558> 2014-08-13 16:24:29.940071 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.940040, event: dispatched, request: pg_notify(22.481(27) epoch 194455) v5 -557> 2014-08-13 16:24:29.940081 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.940081, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194455) v5 -556> 2014-08-13 16:24:29.940092 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.940091, event: started, request: pg_notify(22.481(27) epoch 194455) v5 -555> 2014-08-13 16:24:29.940114 7fd93dff0700 5 -- op tracker -- , seq: 1207, time: 2014-08-13 16:24:29.940114, event: done, request: pg_notify(22.481(27) epoch 194455) v5 -554> 2014-08-13 16:24:29.940161 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.20 192.168.1.103:6815/38269 30 ==== pg_info(1 pgs e194455:24.2c(4)) v4 ==== 960+0+0 (721421594 0 0) 0x9116900 con 0x4fe1600 -553> 2014-08-13 16:24:29.940192 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.935362, event: header_read, request: pg_info(1 pgs e194455:24.2c(4)) v4 -552> 2014-08-13 16:24:29.940201 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.935364, event: throttled, request: pg_info(1 pgs e194455:24.2c(4)) v4 -551> 2014-08-13 16:24:29.940207 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.935437, event: all_read, request: pg_info(1 pgs e194455:24.2c(4)) v4 -550> 2014-08-13 16:24:29.940212 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.940189, event: dispatched, request: pg_info(1 pgs e194455:24.2c(4)) v4 -549> 2014-08-13 16:24:29.940218 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.940218, event: waiting_for_osdmap, request: pg_info(1 pgs e194455:24.2c(4)) v4 -548> 2014-08-13 16:24:29.940227 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.940226, event: started, request: pg_info(1 pgs e194455:24.2c(4)) v4 -547> 2014-08-13 16:24:29.940248 7fd93dff0700 5 -- op tracker -- , seq: 1208, time: 2014-08-13 16:24:29.940248, event: done, request: pg_info(1 pgs e194455:24.2c(4)) v4 -546> 2014-08-13 16:24:29.940263 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.61 192.168.1.111:6840/16566 31 ==== pg_notify(22.362(4) epoch 194455) v5 ==== 942+0+0 (4124083568 0 0) 0x8eeb800 con 0x4fe40a0 -545> 2014-08-13 16:24:29.940279 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.938464, event: header_read, request: pg_notify(22.362(4) epoch 194455) v5 -544> 2014-08-13 16:24:29.940285 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.938467, event: throttled, request: pg_notify(22.362(4) epoch 194455) v5 -543> 2014-08-13 16:24:29.940290 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.938613, event: all_read, request: pg_notify(22.362(4) epoch 194455) v5 -542> 2014-08-13 16:24:29.940296 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.940277, event: dispatched, request: pg_notify(22.362(4) epoch 194455) v5 -541> 2014-08-13 16:24:29.940302 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.940301, event: waiting_for_osdmap, request: pg_notify(22.362(4) epoch 194455) v5 -540> 2014-08-13 16:24:29.940309 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.940308, event: started, request: pg_notify(22.362(4) epoch 194455) v5 -539> 2014-08-13 16:24:29.940327 7fd93dff0700 5 -- op tracker -- , seq: 1209, time: 2014-08-13 16:24:29.940327, event: done, request: pg_notify(22.362(4) epoch 194455) v5 -538> 2014-08-13 16:24:29.940342 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.340 192.168.1.104:6806/62463 29 ==== pg_notify(24.34(43) epoch 194455) v5 ==== 2933+0+0 (2837416962 0 0) 0x842bb80 con 0x4fe69e0 -537> 2014-08-13 16:24:29.940356 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.939567, event: header_read, request: pg_notify(24.34(43) epoch 194455) v5 -536> 2014-08-13 16:24:29.940362 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.939568, event: throttled, request: pg_notify(24.34(43) epoch 194455) v5 -535> 2014-08-13 16:24:29.940368 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.939695, event: all_read, request: pg_notify(24.34(43) epoch 194455) v5 -534> 2014-08-13 16:24:29.940374 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.940354, event: dispatched, request: pg_notify(24.34(43) epoch 194455) v5 -533> 2014-08-13 16:24:29.940386 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.940386, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194455) v5 -532> 2014-08-13 16:24:29.940397 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.940397, event: started, request: pg_notify(24.34(43) epoch 194455) v5 -531> 2014-08-13 16:24:29.940422 7fd93dff0700 5 -- op tracker -- , seq: 1210, time: 2014-08-13 16:24:29.940421, event: done, request: pg_notify(24.34(43) epoch 194455) v5 -530> 2014-08-13 16:24:29.940445 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 25 ==== pg_notify(24.b6(4) epoch 194455) v5 ==== 976+0+0 (3204341065 0 0) 0x9111a40 con 0x5113860 -529> 2014-08-13 16:24:29.940463 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.939468, event: header_read, request: pg_notify(24.b6(4) epoch 194455) v5 -528> 2014-08-13 16:24:29.940474 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.939470, event: throttled, request: pg_notify(24.b6(4) epoch 194455) v5 -527> 2014-08-13 16:24:29.940480 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.939566, event: all_read, request: pg_notify(24.b6(4) epoch 194455) v5 -526> 2014-08-13 16:24:29.940485 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.940461, event: dispatched, request: pg_notify(24.b6(4) epoch 194455) v5 -525> 2014-08-13 16:24:29.940491 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.940491, event: waiting_for_osdmap, request: pg_notify(24.b6(4) epoch 194455) v5 -524> 2014-08-13 16:24:29.940499 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.940499, event: started, request: pg_notify(24.b6(4) epoch 194455) v5 -523> 2014-08-13 16:24:29.940517 7fd93dff0700 5 -- op tracker -- , seq: 1211, time: 2014-08-13 16:24:29.940517, event: done, request: pg_notify(24.b6(4) epoch 194455) v5 -522> 2014-08-13 16:24:29.940531 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 33 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7a640 con 0x5111fa0 -521> 2014-08-13 16:24:29.940547 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -520> 2014-08-13 16:24:29.940556 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.342 192.168.1.110:6802/41438 25 ==== pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 ==== 3436+0+0 (4046636798 0 0) 0x899c7c0 con 0x7314fc0 -519> 2014-08-13 16:24:29.940570 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.934998, event: header_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -518> 2014-08-13 16:24:29.940577 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.935001, event: throttled, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -517> 2014-08-13 16:24:29.940583 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.935156, event: all_read, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -516> 2014-08-13 16:24:29.940589 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.940568, event: dispatched, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -515> 2014-08-13 16:24:29.940596 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.940595, event: waiting_for_osdmap, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -514> 2014-08-13 16:24:29.940603 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.940603, event: started, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -513> 2014-08-13 16:24:29.941050 7fd93dff0700 5 -- op tracker -- , seq: 1212, time: 2014-08-13 16:24:29.941049, event: done, request: pg_notify(22.3e4(17),22.707(19) epoch 194455) v5 -512> 2014-08-13 16:24:29.941125 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.327 192.168.1.106:6804/5112 26 ==== pg_notify(22.48e(15) epoch 194455) v5 ==== 1551+0+0 (632708032 0 0) 0x9110540 con 0x5113860 -511> 2014-08-13 16:24:29.941187 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.940243, event: header_read, request: pg_notify(22.48e(15) epoch 194455) v5 -510> 2014-08-13 16:24:29.941212 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.940246, event: throttled, request: pg_notify(22.48e(15) epoch 194455) v5 -509> 2014-08-13 16:24:29.941231 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.940356, event: all_read, request: pg_notify(22.48e(15) epoch 194455) v5 -508> 2014-08-13 16:24:29.941249 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.941175, event: dispatched, request: pg_notify(22.48e(15) epoch 194455) v5 -507> 2014-08-13 16:24:29.941269 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.941269, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194455) v5 -506> 2014-08-13 16:24:29.941291 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.941290, event: started, request: pg_notify(22.48e(15) epoch 194455) v5 -505> 2014-08-13 16:24:29.941328 7fd93dff0700 5 -- op tracker -- , seq: 1213, time: 2014-08-13 16:24:29.941327, event: done, request: pg_notify(22.48e(15) epoch 194455) v5 -504> 2014-08-13 16:24:29.941372 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 34 ==== pg_notify(22.3d9(6) epoch 194455) v5 ==== 1060+0+0 (1879113637 0 0) 0x899b2c0 con 0x5111fa0 -503> 2014-08-13 16:24:29.941405 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.940284, event: header_read, request: pg_notify(22.3d9(6) epoch 194455) v5 -502> 2014-08-13 16:24:29.941427 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.940285, event: throttled, request: pg_notify(22.3d9(6) epoch 194455) v5 -501> 2014-08-13 16:24:29.941445 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.940354, event: all_read, request: pg_notify(22.3d9(6) epoch 194455) v5 -500> 2014-08-13 16:24:29.941463 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.941402, event: dispatched, request: pg_notify(22.3d9(6) epoch 194455) v5 -499> 2014-08-13 16:24:29.941492 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.941492, event: waiting_for_osdmap, request: pg_notify(22.3d9(6) epoch 194455) v5 -498> 2014-08-13 16:24:29.941514 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.941514, event: started, request: pg_notify(22.3d9(6) epoch 194455) v5 -497> 2014-08-13 16:24:29.941560 7fd93dff0700 5 -- op tracker -- , seq: 1214, time: 2014-08-13 16:24:29.941560, event: done, request: pg_notify(22.3d9(6) epoch 194455) v5 -496> 2014-08-13 16:24:29.941561 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- osd_map(194455..194455 src has 185139..194455) v3 -- ?+0 0x8bd2d00 con 0x4fe1600 -495> 2014-08-13 16:24:29.941600 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(27) epoch 194455) v5 -- ?+0 0x8409880 con 0x4fe1600 -494> 2014-08-13 16:24:29.942066 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 29 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd0240 con 0x8e286e0 -493> 2014-08-13 16:24:29.942100 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -492> 2014-08-13 16:24:29.942115 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 30 ==== pg_notify(22.3af(17) epoch 194455) v5 ==== 1665+0+0 (2751031649 0 0) 0x91140c0 con 0x8e286e0 -491> 2014-08-13 16:24:29.942133 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.941943, event: header_read, request: pg_notify(22.3af(17) epoch 194455) v5 -490> 2014-08-13 16:24:29.942145 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.941945, event: throttled, request: pg_notify(22.3af(17) epoch 194455) v5 -489> 2014-08-13 16:24:29.942155 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.942048, event: all_read, request: pg_notify(22.3af(17) epoch 194455) v5 -488> 2014-08-13 16:24:29.942163 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.942131, event: dispatched, request: pg_notify(22.3af(17) epoch 194455) v5 -487> 2014-08-13 16:24:29.942173 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.942173, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194455) v5 -486> 2014-08-13 16:24:29.942195 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.942195, event: started, request: pg_notify(22.3af(17) epoch 194455) v5 -485> 2014-08-13 16:24:29.942227 7fd93dff0700 5 -- op tracker -- , seq: 1215, time: 2014-08-13 16:24:29.942227, event: done, request: pg_notify(22.3af(17) epoch 194455) v5 -484> 2014-08-13 16:24:29.942755 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 32 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e79b00 con 0x50889a0 -483> 2014-08-13 16:24:29.942808 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -482> 2014-08-13 16:24:29.942823 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 31 ==== pg_notify(22.775(4) epoch 194455) v5 ==== 942+0+0 (1760083111 0 0) 0x91178c0 con 0x8e286e0 -481> 2014-08-13 16:24:29.942847 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942623, event: header_read, request: pg_notify(22.775(4) epoch 194455) v5 -480> 2014-08-13 16:24:29.942877 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942626, event: throttled, request: pg_notify(22.775(4) epoch 194455) v5 -479> 2014-08-13 16:24:29.942897 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942727, event: all_read, request: pg_notify(22.775(4) epoch 194455) v5 -478> 2014-08-13 16:24:29.942916 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942844, event: dispatched, request: pg_notify(22.775(4) epoch 194455) v5 -477> 2014-08-13 16:24:29.942935 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942935, event: waiting_for_osdmap, request: pg_notify(22.775(4) epoch 194455) v5 -476> 2014-08-13 16:24:29.942958 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942958, event: started, request: pg_notify(22.775(4) epoch 194455) v5 -475> 2014-08-13 16:24:29.942999 7fd93dff0700 5 -- op tracker -- , seq: 1216, time: 2014-08-13 16:24:29.942999, event: done, request: pg_notify(22.775(4) epoch 194455) v5 -474> 2014-08-13 16:24:29.943053 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 33 ==== pg_notify(22.113(17) epoch 194455) v5 ==== 1669+0+0 (2092147544 0 0) 0x899de80 con 0x50889a0 -473> 2014-08-13 16:24:29.943078 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.942877, event: header_read, request: pg_notify(22.113(17) epoch 194455) v5 -472> 2014-08-13 16:24:29.943111 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.942879, event: throttled, request: pg_notify(22.113(17) epoch 194455) v5 -471> 2014-08-13 16:24:29.943118 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.943006, event: all_read, request: pg_notify(22.113(17) epoch 194455) v5 -470> 2014-08-13 16:24:29.943127 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.943075, event: dispatched, request: pg_notify(22.113(17) epoch 194455) v5 -469> 2014-08-13 16:24:29.943138 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.943138, event: waiting_for_osdmap, request: pg_notify(22.113(17) epoch 194455) v5 -468> 2014-08-13 16:24:29.943147 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.943147, event: started, request: pg_notify(22.113(17) epoch 194455) v5 -467> 2014-08-13 16:24:29.943168 7fd93dff0700 5 -- op tracker -- , seq: 1217, time: 2014-08-13 16:24:29.943167, event: done, request: pg_notify(22.113(17) epoch 194455) v5 -466> 2014-08-13 16:24:29.944134 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.55 192.168.1.112:6834/6966 34 ==== pg_notify(24.181(9) epoch 194455) v5 ==== 1263+0+0 (677768290 0 0) 0x899f700 con 0x50889a0 -465> 2014-08-13 16:24:29.944199 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.943959, event: header_read, request: pg_notify(24.181(9) epoch 194455) v5 -464> 2014-08-13 16:24:29.944218 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.943961, event: throttled, request: pg_notify(24.181(9) epoch 194455) v5 -463> 2014-08-13 16:24:29.944225 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.944092, event: all_read, request: pg_notify(24.181(9) epoch 194455) v5 -462> 2014-08-13 16:24:29.944231 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.944183, event: dispatched, request: pg_notify(24.181(9) epoch 194455) v5 -461> 2014-08-13 16:24:29.944242 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.944242, event: waiting_for_osdmap, request: pg_notify(24.181(9) epoch 194455) v5 -460> 2014-08-13 16:24:29.944255 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.944255, event: started, request: pg_notify(24.181(9) epoch 194455) v5 -459> 2014-08-13 16:24:29.944289 7fd93dff0700 5 -- op tracker -- , seq: 1218, time: 2014-08-13 16:24:29.944289, event: done, request: pg_notify(24.181(9) epoch 194455) v5 -458> 2014-08-13 16:24:29.944936 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 11 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d033c0 con 0x7d2d120 -457> 2014-08-13 16:24:29.944973 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -456> 2014-08-13 16:24:29.945374 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.26 192.168.1.115:6802/58522 12 ==== pg_notify(24.34(30) epoch 194455) v5 ==== 2250+0+0 (263981195 0 0) 0x775b100 con 0x7d2d120 -455> 2014-08-13 16:24:29.945407 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945208, event: header_read, request: pg_notify(24.34(30) epoch 194455) v5 -454> 2014-08-13 16:24:29.945417 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945211, event: throttled, request: pg_notify(24.34(30) epoch 194455) v5 -453> 2014-08-13 16:24:29.945428 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945350, event: all_read, request: pg_notify(24.34(30) epoch 194455) v5 -452> 2014-08-13 16:24:29.945438 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945403, event: dispatched, request: pg_notify(24.34(30) epoch 194455) v5 -451> 2014-08-13 16:24:29.945448 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945448, event: waiting_for_osdmap, request: pg_notify(24.34(30) epoch 194455) v5 -450> 2014-08-13 16:24:29.945461 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945461, event: started, request: pg_notify(24.34(30) epoch 194455) v5 -449> 2014-08-13 16:24:29.945491 7fd93dff0700 5 -- op tracker -- , seq: 1219, time: 2014-08-13 16:24:29.945491, event: done, request: pg_notify(24.34(30) epoch 194455) v5 -448> 2014-08-13 16:24:29.946361 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd4a40 con 0x7379ce0 -447> 2014-08-13 16:24:29.946371 7fd9371e5700 5 osd.4 pg_epoch: 194455 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.749741 8 0.000463 -446> 2014-08-13 16:24:29.946408 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -445> 2014-08-13 16:24:29.946420 7fd9371e5700 5 osd.4 pg_epoch: 194455 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/ReplicaActive -444> 2014-08-13 16:24:29.946446 7fd9371e5700 5 osd.4 pg_epoch: 194455 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194449 n=56 ec=192688 les/c 194449/194442 194453/194454/192971) [20,27,4] r=2 lpr=194454 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering -443> 2014-08-13 16:24:29.946766 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.369 192.168.1.108:6824/4037 32 ==== pg_notify(22.f3(15) epoch 194455) v5 ==== 1551+0+0 (1565435689 0 0) 0x9116e40 con 0x8e286e0 -442> 2014-08-13 16:24:29.946800 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946376, event: header_read, request: pg_notify(22.f3(15) epoch 194455) v5 -441> 2014-08-13 16:24:29.946815 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946379, event: throttled, request: pg_notify(22.f3(15) epoch 194455) v5 -440> 2014-08-13 16:24:29.946825 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946495, event: all_read, request: pg_notify(22.f3(15) epoch 194455) v5 -439> 2014-08-13 16:24:29.946834 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946795, event: dispatched, request: pg_notify(22.f3(15) epoch 194455) v5 -438> 2014-08-13 16:24:29.946845 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946845, event: waiting_for_osdmap, request: pg_notify(22.f3(15) epoch 194455) v5 -437> 2014-08-13 16:24:29.946858 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946858, event: started, request: pg_notify(22.f3(15) epoch 194455) v5 -436> 2014-08-13 16:24:29.946889 7fd93dff0700 5 -- op tracker -- , seq: 1220, time: 2014-08-13 16:24:29.946889, event: done, request: pg_notify(22.f3(15) epoch 194455) v5 -435> 2014-08-13 16:24:29.946916 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.307 192.168.1.109:6804/31016 17 ==== pg_notify(24.181(25) epoch 194455) v5 ==== 2159+0+0 (1719712648 0 0) 0x834cb40 con 0x7379ce0 -434> 2014-08-13 16:24:29.946936 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946320, event: header_read, request: pg_notify(24.181(25) epoch 194455) v5 -433> 2014-08-13 16:24:29.946951 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946321, event: throttled, request: pg_notify(24.181(25) epoch 194455) v5 -432> 2014-08-13 16:24:29.946957 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946515, event: all_read, request: pg_notify(24.181(25) epoch 194455) v5 -431> 2014-08-13 16:24:29.946966 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946934, event: dispatched, request: pg_notify(24.181(25) epoch 194455) v5 -430> 2014-08-13 16:24:29.946976 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946975, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194455) v5 -429> 2014-08-13 16:24:29.946986 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.946986, event: started, request: pg_notify(24.181(25) epoch 194455) v5 -428> 2014-08-13 16:24:29.947010 7fd93dff0700 5 -- op tracker -- , seq: 1221, time: 2014-08-13 16:24:29.947010, event: done, request: pg_notify(24.181(25) epoch 194455) v5 -427> 2014-08-13 16:24:29.947041 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 39 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf2d00 con 0x508bde0 -426> 2014-08-13 16:24:29.947064 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -425> 2014-08-13 16:24:29.947085 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.337 192.168.1.104:6804/62197 40 ==== pg_notify(22.99(15) epoch 194455) v5 ==== 1551+0+0 (427088001 0 0) 0x899d080 con 0x508bde0 -424> 2014-08-13 16:24:29.947103 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.946590, event: header_read, request: pg_notify(22.99(15) epoch 194455) v5 -423> 2014-08-13 16:24:29.947114 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.946590, event: throttled, request: pg_notify(22.99(15) epoch 194455) v5 -422> 2014-08-13 16:24:29.947123 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.946671, event: all_read, request: pg_notify(22.99(15) epoch 194455) v5 -421> 2014-08-13 16:24:29.947131 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.947101, event: dispatched, request: pg_notify(22.99(15) epoch 194455) v5 -420> 2014-08-13 16:24:29.947141 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.947141, event: waiting_for_osdmap, request: pg_notify(22.99(15) epoch 194455) v5 -419> 2014-08-13 16:24:29.947152 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.947152, event: started, request: pg_notify(22.99(15) epoch 194455) v5 -418> 2014-08-13 16:24:29.947175 7fd93dff0700 5 -- op tracker -- , seq: 1222, time: 2014-08-13 16:24:29.947175, event: done, request: pg_notify(22.99(15) epoch 194455) v5 -417> 2014-08-13 16:24:29.947225 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 13 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d05e80 con 0x80c7e80 -416> 2014-08-13 16:24:29.947250 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -415> 2014-08-13 16:24:29.947337 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 20 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d09680 con 0x72d9b80 -414> 2014-08-13 16:24:29.947354 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -413> 2014-08-13 16:24:29.947449 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.363 192.168.1.102:6842/20017 21 ==== pg_notify(22.99(4),22.39d(15) epoch 194455) v5 ==== 2481+0+0 (3177127869 0 0) 0x9117540 con 0x72d9b80 -412> 2014-08-13 16:24:29.947466 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947336, event: header_read, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -411> 2014-08-13 16:24:29.947475 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947338, event: throttled, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -410> 2014-08-13 16:24:29.947481 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947430, event: all_read, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -409> 2014-08-13 16:24:29.947487 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947463, event: dispatched, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -408> 2014-08-13 16:24:29.947495 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947495, event: waiting_for_osdmap, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -407> 2014-08-13 16:24:29.947507 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947507, event: started, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -406> 2014-08-13 16:24:29.947563 7fd93dff0700 5 -- op tracker -- , seq: 1223, time: 2014-08-13 16:24:29.947563, event: done, request: pg_notify(22.99(4),22.39d(15) epoch 194455) v5 -405> 2014-08-13 16:24:29.947587 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.382 192.168.1.108:6800/30527 14 ==== pg_notify(22.3d9(39) epoch 194455) v5 ==== 2815+0+0 (1904319689 0 0) 0x8428a80 con 0x80c7e80 -404> 2014-08-13 16:24:29.947607 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947340, event: header_read, request: pg_notify(22.3d9(39) epoch 194455) v5 -403> 2014-08-13 16:24:29.947617 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947342, event: throttled, request: pg_notify(22.3d9(39) epoch 194455) v5 -402> 2014-08-13 16:24:29.947626 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947468, event: all_read, request: pg_notify(22.3d9(39) epoch 194455) v5 -401> 2014-08-13 16:24:29.947635 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947605, event: dispatched, request: pg_notify(22.3d9(39) epoch 194455) v5 -400> 2014-08-13 16:24:29.947644 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947644, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194455) v5 -399> 2014-08-13 16:24:29.947656 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947656, event: started, request: pg_notify(22.3d9(39) epoch 194455) v5 -398> 2014-08-13 16:24:29.947683 7fd93dff0700 5 -- op tracker -- , seq: 1224, time: 2014-08-13 16:24:29.947683, event: done, request: pg_notify(22.3d9(39) epoch 194455) v5 -397> 2014-08-13 16:24:29.947709 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 26 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf4140 con 0x5114360 -396> 2014-08-13 16:24:29.947730 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -395> 2014-08-13 16:24:29.947743 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.320 192.168.1.106:6820/10050 27 ==== pg_notify(22.39d(15) epoch 194455) v5 ==== 1551+0+0 (3107704238 0 0) 0x8ee8000 con 0x5114360 -394> 2014-08-13 16:24:29.947760 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947530, event: header_read, request: pg_notify(22.39d(15) epoch 194455) v5 -393> 2014-08-13 16:24:29.947770 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947531, event: throttled, request: pg_notify(22.39d(15) epoch 194455) v5 -392> 2014-08-13 16:24:29.947779 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947617, event: all_read, request: pg_notify(22.39d(15) epoch 194455) v5 -391> 2014-08-13 16:24:29.947788 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947758, event: dispatched, request: pg_notify(22.39d(15) epoch 194455) v5 -390> 2014-08-13 16:24:29.947795 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947795, event: waiting_for_osdmap, request: pg_notify(22.39d(15) epoch 194455) v5 -389> 2014-08-13 16:24:29.947802 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947802, event: started, request: pg_notify(22.39d(15) epoch 194455) v5 -388> 2014-08-13 16:24:29.947828 7fd93dff0700 5 -- op tracker -- , seq: 1225, time: 2014-08-13 16:24:29.947828, event: done, request: pg_notify(22.39d(15) epoch 194455) v5 -387> 2014-08-13 16:24:29.947874 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7cf3cc0 con 0x5113de0 -386> 2014-08-13 16:24:29.947896 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -385> 2014-08-13 16:24:29.947987 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.379 192.168.1.108:6830/4861 17 ==== pg_notify(24.1bb(21) epoch 194455) v5 ==== 1931+0+0 (3030785323 0 0) 0x89996c0 con 0x5113de0 -384> 2014-08-13 16:24:29.948009 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.947791, event: header_read, request: pg_notify(24.1bb(21) epoch 194455) v5 -383> 2014-08-13 16:24:29.948020 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.947792, event: throttled, request: pg_notify(24.1bb(21) epoch 194455) v5 -382> 2014-08-13 16:24:29.948029 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.947885, event: all_read, request: pg_notify(24.1bb(21) epoch 194455) v5 -381> 2014-08-13 16:24:29.948038 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.948006, event: dispatched, request: pg_notify(24.1bb(21) epoch 194455) v5 -380> 2014-08-13 16:24:29.948048 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.948047, event: waiting_for_osdmap, request: pg_notify(24.1bb(21) epoch 194455) v5 -379> 2014-08-13 16:24:29.948059 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.948058, event: started, request: pg_notify(24.1bb(21) epoch 194455) v5 -378> 2014-08-13 16:24:29.948087 7fd93dff0700 5 -- op tracker -- , seq: 1226, time: 2014-08-13 16:24:29.948087, event: done, request: pg_notify(24.1bb(21) epoch 194455) v5 -377> 2014-08-13 16:24:29.948120 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 23 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd5a00 con 0x4e4b860 -376> 2014-08-13 16:24:29.948142 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -375> 2014-08-13 16:24:29.948155 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 27 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859b180 con 0x4fda940 -374> 2014-08-13 16:24:29.948173 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -373> 2014-08-13 16:24:29.948191 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.301 192.168.1.113:6814/16727 24 ==== pg_notify(24.30b(4) epoch 194455) v5 ==== 976+0+0 (1827286100 0 0) 0x842fe00 con 0x4e4b860 -372> 2014-08-13 16:24:29.948210 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948055, event: header_read, request: pg_notify(24.30b(4) epoch 194455) v5 -371> 2014-08-13 16:24:29.948221 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948056, event: throttled, request: pg_notify(24.30b(4) epoch 194455) v5 -370> 2014-08-13 16:24:29.948230 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948123, event: all_read, request: pg_notify(24.30b(4) epoch 194455) v5 -369> 2014-08-13 16:24:29.948239 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948207, event: dispatched, request: pg_notify(24.30b(4) epoch 194455) v5 -368> 2014-08-13 16:24:29.948249 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948249, event: waiting_for_osdmap, request: pg_notify(24.30b(4) epoch 194455) v5 -367> 2014-08-13 16:24:29.948260 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948260, event: started, request: pg_notify(24.30b(4) epoch 194455) v5 -366> 2014-08-13 16:24:29.948292 7fd93dff0700 5 -- op tracker -- , seq: 1227, time: 2014-08-13 16:24:29.948291, event: done, request: pg_notify(24.30b(4) epoch 194455) v5 -365> 2014-08-13 16:24:29.948307 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.45 192.168.1.112:6828/6040 28 ==== pg_notify(22.3e4(17) epoch 194455) v5 ==== 1665+0+0 (2735378193 0 0) 0x8408e00 con 0x4fda940 -364> 2014-08-13 16:24:29.948323 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948071, event: header_read, request: pg_notify(22.3e4(17) epoch 194455) v5 -363> 2014-08-13 16:24:29.948330 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948072, event: throttled, request: pg_notify(22.3e4(17) epoch 194455) v5 -362> 2014-08-13 16:24:29.948335 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948161, event: all_read, request: pg_notify(22.3e4(17) epoch 194455) v5 -361> 2014-08-13 16:24:29.948340 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948321, event: dispatched, request: pg_notify(22.3e4(17) epoch 194455) v5 -360> 2014-08-13 16:24:29.948346 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948346, event: waiting_for_osdmap, request: pg_notify(22.3e4(17) epoch 194455) v5 -359> 2014-08-13 16:24:29.948353 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948353, event: started, request: pg_notify(22.3e4(17) epoch 194455) v5 -358> 2014-08-13 16:24:29.948381 7fd93dff0700 5 -- op tracker -- , seq: 1228, time: 2014-08-13 16:24:29.948381, event: done, request: pg_notify(22.3e4(17) epoch 194455) v5 -357> 2014-08-13 16:24:29.948685 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 6 ==== pg_notify(24.181(25) epoch 194455) v5 ==== 2159+0+0 (2074158954 0 0) 0x92871c0 con 0x83002c0 -356> 2014-08-13 16:24:29.948710 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948534, event: header_read, request: pg_notify(24.181(25) epoch 194455) v5 -355> 2014-08-13 16:24:29.948721 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948536, event: throttled, request: pg_notify(24.181(25) epoch 194455) v5 -354> 2014-08-13 16:24:29.948730 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948660, event: all_read, request: pg_notify(24.181(25) epoch 194455) v5 -353> 2014-08-13 16:24:29.948739 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948707, event: dispatched, request: pg_notify(24.181(25) epoch 194455) v5 -352> 2014-08-13 16:24:29.948746 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948745, event: waiting_for_osdmap, request: pg_notify(24.181(25) epoch 194455) v5 -351> 2014-08-13 16:24:29.948753 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948753, event: started, request: pg_notify(24.181(25) epoch 194455) v5 -350> 2014-08-13 16:24:29.948779 7fd93dff0700 5 -- op tracker -- , seq: 1229, time: 2014-08-13 16:24:29.948779, event: done, request: pg_notify(24.181(25) epoch 194455) v5 -349> 2014-08-13 16:24:29.948833 7fd943029700 1 -- 192.168.1.115:6813/25220 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194455:24.2c) v4 -- ?+0 0x7d03100 -348> 2014-08-13 16:24:29.949091 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 24 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859a640 con 0x510b9c0 -347> 2014-08-13 16:24:29.949135 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -346> 2014-08-13 16:24:29.949378 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.362 192.168.1.108:6842/8170 25 ==== pg_notify(22.362(17) epoch 194455) v5 ==== 1665+0+0 (4019939463 0 0) 0x8577540 con 0x510b9c0 -345> 2014-08-13 16:24:29.949406 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949090, event: header_read, request: pg_notify(22.362(17) epoch 194455) v5 -344> 2014-08-13 16:24:29.949421 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949091, event: throttled, request: pg_notify(22.362(17) epoch 194455) v5 -343> 2014-08-13 16:24:29.949428 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949200, event: all_read, request: pg_notify(22.362(17) epoch 194455) v5 -342> 2014-08-13 16:24:29.949433 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949403, event: dispatched, request: pg_notify(22.362(17) epoch 194455) v5 -341> 2014-08-13 16:24:29.949441 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949440, event: waiting_for_osdmap, request: pg_notify(22.362(17) epoch 194455) v5 -340> 2014-08-13 16:24:29.949450 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949449, event: started, request: pg_notify(22.362(17) epoch 194455) v5 -339> 2014-08-13 16:24:29.949484 7fd93dff0700 5 -- op tracker -- , seq: 1230, time: 2014-08-13 16:24:29.949484, event: done, request: pg_notify(22.362(17) epoch 194455) v5 -338> 2014-08-13 16:24:29.950109 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 34 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd1440 con 0x5111a20 -337> 2014-08-13 16:24:29.950152 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -336> 2014-08-13 16:24:29.950342 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 35 ==== pg_notify(22.48e(15) epoch 194455) v5 ==== 1551+0+0 (1639907479 0 0) 0x842af40 con 0x5111a20 -335> 2014-08-13 16:24:29.950402 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950109, event: header_read, request: pg_notify(22.48e(15) epoch 194455) v5 -334> 2014-08-13 16:24:29.950418 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950110, event: throttled, request: pg_notify(22.48e(15) epoch 194455) v5 -333> 2014-08-13 16:24:29.950428 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950272, event: all_read, request: pg_notify(22.48e(15) epoch 194455) v5 -332> 2014-08-13 16:24:29.950438 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950396, event: dispatched, request: pg_notify(22.48e(15) epoch 194455) v5 -331> 2014-08-13 16:24:29.950449 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950448, event: waiting_for_osdmap, request: pg_notify(22.48e(15) epoch 194455) v5 -330> 2014-08-13 16:24:29.950459 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950459, event: started, request: pg_notify(22.48e(15) epoch 194455) v5 -329> 2014-08-13 16:24:29.950493 7fd93dff0700 5 -- op tracker -- , seq: 1231, time: 2014-08-13 16:24:29.950493, event: done, request: pg_notify(22.48e(15) epoch 194455) v5 -328> 2014-08-13 16:24:29.950521 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.376 192.168.1.108:6838/6893 36 ==== pg_notify(24.1f7(18) epoch 194455) v5 ==== 1754+0+0 (2580756228 0 0) 0x842ee40 con 0x5111a20 -327> 2014-08-13 16:24:29.950543 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950292, event: header_read, request: pg_notify(24.1f7(18) epoch 194455) v5 -326> 2014-08-13 16:24:29.950553 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950293, event: throttled, request: pg_notify(24.1f7(18) epoch 194455) v5 -325> 2014-08-13 16:24:29.950563 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950375, event: all_read, request: pg_notify(24.1f7(18) epoch 194455) v5 -324> 2014-08-13 16:24:29.950572 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950540, event: dispatched, request: pg_notify(24.1f7(18) epoch 194455) v5 -323> 2014-08-13 16:24:29.950581 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950581, event: waiting_for_osdmap, request: pg_notify(24.1f7(18) epoch 194455) v5 -322> 2014-08-13 16:24:29.950593 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950593, event: started, request: pg_notify(24.1f7(18) epoch 194455) v5 -321> 2014-08-13 16:24:29.950620 7fd93dff0700 5 -- op tracker -- , seq: 1232, time: 2014-08-13 16:24:29.950620, event: done, request: pg_notify(24.1f7(18) epoch 194455) v5 -320> 2014-08-13 16:24:29.951628 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 13 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d06540 con 0x73ccba0 -319> 2014-08-13 16:24:29.951676 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -318> 2014-08-13 16:24:29.951945 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.306 192.168.1.101:6806/6812 14 ==== pg_notify(22.481(32) epoch 194455) v5 ==== 2518+0+0 (323053658 0 0) 0x842b480 con 0x73ccba0 -317> 2014-08-13 16:24:29.951976 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.951763, event: header_read, request: pg_notify(22.481(32) epoch 194455) v5 -316> 2014-08-13 16:24:29.951987 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.951765, event: throttled, request: pg_notify(22.481(32) epoch 194455) v5 -315> 2014-08-13 16:24:29.951993 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.951911, event: all_read, request: pg_notify(22.481(32) epoch 194455) v5 -314> 2014-08-13 16:24:29.951999 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.951972, event: dispatched, request: pg_notify(22.481(32) epoch 194455) v5 -313> 2014-08-13 16:24:29.952006 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.952005, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194455) v5 -312> 2014-08-13 16:24:29.952022 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.952022, event: started, request: pg_notify(22.481(32) epoch 194455) v5 -311> 2014-08-13 16:24:29.952054 7fd93dff0700 5 -- op tracker -- , seq: 1233, time: 2014-08-13 16:24:29.952054, event: done, request: pg_notify(22.481(32) epoch 194455) v5 -310> 2014-08-13 16:24:29.952079 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 22 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d045c0 con 0x73cdc20 -309> 2014-08-13 16:24:29.952098 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -308> 2014-08-13 16:24:29.952112 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.339 192.168.1.110:6818/45268 23 ==== pg_notify(22.3af(17) epoch 194455) v5 ==== 1665+0+0 (2751031649 0 0) 0x899a840 con 0x73cdc20 -307> 2014-08-13 16:24:29.952130 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.951981, event: header_read, request: pg_notify(22.3af(17) epoch 194455) v5 -306> 2014-08-13 16:24:29.952141 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.951983, event: throttled, request: pg_notify(22.3af(17) epoch 194455) v5 -305> 2014-08-13 16:24:29.952150 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.952074, event: all_read, request: pg_notify(22.3af(17) epoch 194455) v5 -304> 2014-08-13 16:24:29.952156 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.952128, event: dispatched, request: pg_notify(22.3af(17) epoch 194455) v5 -303> 2014-08-13 16:24:29.952163 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.952162, event: waiting_for_osdmap, request: pg_notify(22.3af(17) epoch 194455) v5 -302> 2014-08-13 16:24:29.952173 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.952173, event: started, request: pg_notify(22.3af(17) epoch 194455) v5 -301> 2014-08-13 16:24:29.952214 7fd93dff0700 5 -- op tracker -- , seq: 1234, time: 2014-08-13 16:24:29.952213, event: done, request: pg_notify(22.3af(17) epoch 194455) v5 -300> 2014-08-13 16:24:29.953209 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 42 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8599440 con 0x51135a0 -299> 2014-08-13 16:24:29.953257 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -298> 2014-08-13 16:24:29.953318 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 43 ==== pg_notify(24.3d(18) epoch 194455) v5 ==== 1754+0+0 (513865912 0 0) 0x9110540 con 0x51135a0 -297> 2014-08-13 16:24:29.953350 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953188, event: header_read, request: pg_notify(24.3d(18) epoch 194455) v5 -296> 2014-08-13 16:24:29.953365 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953189, event: throttled, request: pg_notify(24.3d(18) epoch 194455) v5 -295> 2014-08-13 16:24:29.953375 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953278, event: all_read, request: pg_notify(24.3d(18) epoch 194455) v5 -294> 2014-08-13 16:24:29.953384 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953345, event: dispatched, request: pg_notify(24.3d(18) epoch 194455) v5 -293> 2014-08-13 16:24:29.953394 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953394, event: waiting_for_osdmap, request: pg_notify(24.3d(18) epoch 194455) v5 -292> 2014-08-13 16:24:29.953407 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953407, event: started, request: pg_notify(24.3d(18) epoch 194455) v5 -291> 2014-08-13 16:24:29.953445 7fd93dff0700 5 -- op tracker -- , seq: 1235, time: 2014-08-13 16:24:29.953445, event: done, request: pg_notify(24.3d(18) epoch 194455) v5 -290> 2014-08-13 16:24:29.953485 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.373 192.168.1.108:6840/7435 44 ==== pg_notify(22.3d9(32) epoch 194455) v5 ==== 2458+0+0 (77835295 0 0) 0x9111a40 con 0x51135a0 -289> 2014-08-13 16:24:29.953506 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953310, event: header_read, request: pg_notify(22.3d9(32) epoch 194455) v5 -288> 2014-08-13 16:24:29.953517 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953311, event: throttled, request: pg_notify(22.3d9(32) epoch 194455) v5 -287> 2014-08-13 16:24:29.953523 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953408, event: all_read, request: pg_notify(22.3d9(32) epoch 194455) v5 -286> 2014-08-13 16:24:29.953528 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953504, event: dispatched, request: pg_notify(22.3d9(32) epoch 194455) v5 -285> 2014-08-13 16:24:29.953538 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953537, event: waiting_for_osdmap, request: pg_notify(22.3d9(32) epoch 194455) v5 -284> 2014-08-13 16:24:29.953548 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953548, event: started, request: pg_notify(22.3d9(32) epoch 194455) v5 -283> 2014-08-13 16:24:29.953576 7fd93dff0700 5 -- op tracker -- , seq: 1236, time: 2014-08-13 16:24:29.953576, event: done, request: pg_notify(22.3d9(32) epoch 194455) v5 -282> 2014-08-13 16:24:29.955132 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7b121c0 con 0x510dac0 -281> 2014-08-13 16:24:29.955190 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -280> 2014-08-13 16:24:29.955336 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 15 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e78b40 con 0x80c7d20 -279> 2014-08-13 16:24:29.955384 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -278> 2014-08-13 16:24:29.955415 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.293 192.168.1.109:6816/33855 17 ==== pg_notify(22.3d9(39) epoch 194455) v5 ==== 2815+0+0 (1608511270 0 0) 0x899fa80 con 0x510dac0 -277> 2014-08-13 16:24:29.955437 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955078, event: header_read, request: pg_notify(22.3d9(39) epoch 194455) v5 -276> 2014-08-13 16:24:29.955452 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955079, event: throttled, request: pg_notify(22.3d9(39) epoch 194455) v5 -275> 2014-08-13 16:24:29.955461 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955235, event: all_read, request: pg_notify(22.3d9(39) epoch 194455) v5 -274> 2014-08-13 16:24:29.955470 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955434, event: dispatched, request: pg_notify(22.3d9(39) epoch 194455) v5 -273> 2014-08-13 16:24:29.955481 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955481, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194455) v5 -272> 2014-08-13 16:24:29.955494 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955493, event: started, request: pg_notify(22.3d9(39) epoch 194455) v5 -271> 2014-08-13 16:24:29.955527 7fd93dff0700 5 -- op tracker -- , seq: 1237, time: 2014-08-13 16:24:29.955527, event: done, request: pg_notify(22.3d9(39) epoch 194455) v5 -270> 2014-08-13 16:24:29.955566 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.381 192.168.1.102:6822/15390 16 ==== pg_notify(22.481(34) epoch 194455) v5 ==== 2620+0+0 (3174329705 0 0) 0x7758a80 con 0x80c7d20 -269> 2014-08-13 16:24:29.955587 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955265, event: header_read, request: pg_notify(22.481(34) epoch 194455) v5 -268> 2014-08-13 16:24:29.955598 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955266, event: throttled, request: pg_notify(22.481(34) epoch 194455) v5 -267> 2014-08-13 16:24:29.955608 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955387, event: all_read, request: pg_notify(22.481(34) epoch 194455) v5 -266> 2014-08-13 16:24:29.955617 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955584, event: dispatched, request: pg_notify(22.481(34) epoch 194455) v5 -265> 2014-08-13 16:24:29.955626 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955626, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194455) v5 -264> 2014-08-13 16:24:29.955637 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955637, event: started, request: pg_notify(22.481(34) epoch 194455) v5 -263> 2014-08-13 16:24:29.955664 7fd93dff0700 5 -- op tracker -- , seq: 1238, time: 2014-08-13 16:24:29.955663, event: done, request: pg_notify(22.481(34) epoch 194455) v5 -262> 2014-08-13 16:24:29.955703 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 22 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8bd1f80 con 0x7316720 -261> 2014-08-13 16:24:29.955725 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -260> 2014-08-13 16:24:29.955737 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 23 ==== pg_notify(22.22c(17) epoch 194455) v5 ==== 1665+0+0 (1498546413 0 0) 0x834c7c0 con 0x7316720 -259> 2014-08-13 16:24:29.955754 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955505, event: header_read, request: pg_notify(22.22c(17) epoch 194455) v5 -258> 2014-08-13 16:24:29.955765 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955507, event: throttled, request: pg_notify(22.22c(17) epoch 194455) v5 -257> 2014-08-13 16:24:29.955774 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955599, event: all_read, request: pg_notify(22.22c(17) epoch 194455) v5 -256> 2014-08-13 16:24:29.955782 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955752, event: dispatched, request: pg_notify(22.22c(17) epoch 194455) v5 -255> 2014-08-13 16:24:29.955792 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955792, event: waiting_for_osdmap, request: pg_notify(22.22c(17) epoch 194455) v5 -254> 2014-08-13 16:24:29.955803 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955803, event: started, request: pg_notify(22.22c(17) epoch 194455) v5 -253> 2014-08-13 16:24:29.955830 7fd93dff0700 5 -- op tracker -- , seq: 1239, time: 2014-08-13 16:24:29.955830, event: done, request: pg_notify(22.22c(17) epoch 194455) v5 -252> 2014-08-13 16:24:29.956640 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.368 192.168.1.102:6826/15861 24 ==== pg_notify(22.1f1(17) epoch 194455) v5 ==== 1665+0+0 (1582334863 0 0) 0x834ad80 con 0x7316720 -251> 2014-08-13 16:24:29.956685 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956482, event: header_read, request: pg_notify(22.1f1(17) epoch 194455) v5 -250> 2014-08-13 16:24:29.956701 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956484, event: throttled, request: pg_notify(22.1f1(17) epoch 194455) v5 -249> 2014-08-13 16:24:29.956711 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956577, event: all_read, request: pg_notify(22.1f1(17) epoch 194455) v5 -248> 2014-08-13 16:24:29.956720 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956679, event: dispatched, request: pg_notify(22.1f1(17) epoch 194455) v5 -247> 2014-08-13 16:24:29.956731 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956730, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194455) v5 -246> 2014-08-13 16:24:29.956744 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956743, event: started, request: pg_notify(22.1f1(17) epoch 194455) v5 -245> 2014-08-13 16:24:29.956778 7fd93dff0700 5 -- op tracker -- , seq: 1240, time: 2014-08-13 16:24:29.956777, event: done, request: pg_notify(22.1f1(17) epoch 194455) v5 -244> 2014-08-13 16:24:29.957563 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 23 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e78d80 con 0x50ba520 -243> 2014-08-13 16:24:29.957611 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -242> 2014-08-13 16:24:29.957643 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.15 192.168.1.107:6812/53408 24 ==== pg_notify(22.707(19) epoch 194455) v5 ==== 1783+0+0 (825667073 0 0) 0x834d5c0 con 0x50ba520 -241> 2014-08-13 16:24:29.957666 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957528, event: header_read, request: pg_notify(22.707(19) epoch 194455) v5 -240> 2014-08-13 16:24:29.957680 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957529, event: throttled, request: pg_notify(22.707(19) epoch 194455) v5 -239> 2014-08-13 16:24:29.957687 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957626, event: all_read, request: pg_notify(22.707(19) epoch 194455) v5 -238> 2014-08-13 16:24:29.957692 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957662, event: dispatched, request: pg_notify(22.707(19) epoch 194455) v5 -237> 2014-08-13 16:24:29.957700 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957699, event: waiting_for_osdmap, request: pg_notify(22.707(19) epoch 194455) v5 -236> 2014-08-13 16:24:29.957712 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957712, event: started, request: pg_notify(22.707(19) epoch 194455) v5 -235> 2014-08-13 16:24:29.957745 7fd93dff0700 5 -- op tracker -- , seq: 1241, time: 2014-08-13 16:24:29.957745, event: done, request: pg_notify(22.707(19) epoch 194455) v5 -234> 2014-08-13 16:24:29.958797 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 23 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d02d00 con 0x508a680 -233> 2014-08-13 16:24:29.958845 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -232> 2014-08-13 16:24:29.958863 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 32 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7b3c0 con 0x4fd8000 -231> 2014-08-13 16:24:29.958882 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -230> 2014-08-13 16:24:29.958891 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.44 192.168.1.111:6822/10782 24 ==== pg_notify(22.619(4) epoch 194455) v5 ==== 942+0+0 (781834006 0 0) 0x899c7c0 con 0x508a680 -229> 2014-08-13 16:24:29.958907 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958738, event: header_read, request: pg_notify(22.619(4) epoch 194455) v5 -228> 2014-08-13 16:24:29.958921 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958739, event: throttled, request: pg_notify(22.619(4) epoch 194455) v5 -227> 2014-08-13 16:24:29.958928 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958811, event: all_read, request: pg_notify(22.619(4) epoch 194455) v5 -226> 2014-08-13 16:24:29.958933 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958903, event: dispatched, request: pg_notify(22.619(4) epoch 194455) v5 -225> 2014-08-13 16:24:29.958940 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958940, event: waiting_for_osdmap, request: pg_notify(22.619(4) epoch 194455) v5 -224> 2014-08-13 16:24:29.958954 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958953, event: started, request: pg_notify(22.619(4) epoch 194455) v5 -223> 2014-08-13 16:24:29.958987 7fd93dff0700 5 -- op tracker -- , seq: 1242, time: 2014-08-13 16:24:29.958987, event: done, request: pg_notify(22.619(4) epoch 194455) v5 -222> 2014-08-13 16:24:29.959014 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.356 192.168.1.108:6832/5235 33 ==== pg_notify(24.136(17) epoch 194455) v5 ==== 1695+0+0 (1125281505 0 0) 0x775d400 con 0x4fd8000 -221> 2014-08-13 16:24:29.959034 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.958750, event: header_read, request: pg_notify(24.136(17) epoch 194455) v5 -220> 2014-08-13 16:24:29.959041 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.958752, event: throttled, request: pg_notify(24.136(17) epoch 194455) v5 -219> 2014-08-13 16:24:29.959047 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.958866, event: all_read, request: pg_notify(24.136(17) epoch 194455) v5 -218> 2014-08-13 16:24:29.959056 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.959032, event: dispatched, request: pg_notify(24.136(17) epoch 194455) v5 -217> 2014-08-13 16:24:29.959067 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.959066, event: waiting_for_osdmap, request: pg_notify(24.136(17) epoch 194455) v5 -216> 2014-08-13 16:24:29.959078 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.959078, event: started, request: pg_notify(24.136(17) epoch 194455) v5 -215> 2014-08-13 16:24:29.959105 7fd93dff0700 5 -- op tracker -- , seq: 1243, time: 2014-08-13 16:24:29.959105, event: done, request: pg_notify(24.136(17) epoch 194455) v5 -214> 2014-08-13 16:24:29.959125 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d08240 con 0x7316f60 -213> 2014-08-13 16:24:29.959146 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -212> 2014-08-13 16:24:29.959158 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.302 192.168.1.113:6810/15731 17 ==== pg_notify(22.22c(4) epoch 194455) v5 ==== 942+0+0 (44152588 0 0) 0x834ec80 con 0x7316f60 -211> 2014-08-13 16:24:29.959175 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.958928, event: header_read, request: pg_notify(22.22c(4) epoch 194455) v5 -210> 2014-08-13 16:24:29.959199 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.958930, event: throttled, request: pg_notify(22.22c(4) epoch 194455) v5 -209> 2014-08-13 16:24:29.959205 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.958999, event: all_read, request: pg_notify(22.22c(4) epoch 194455) v5 -208> 2014-08-13 16:24:29.959214 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.959173, event: dispatched, request: pg_notify(22.22c(4) epoch 194455) v5 -207> 2014-08-13 16:24:29.959223 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.959223, event: waiting_for_osdmap, request: pg_notify(22.22c(4) epoch 194455) v5 -206> 2014-08-13 16:24:29.959234 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.959234, event: started, request: pg_notify(22.22c(4) epoch 194455) v5 -205> 2014-08-13 16:24:29.959261 7fd93dff0700 5 -- op tracker -- , seq: 1244, time: 2014-08-13 16:24:29.959261, event: done, request: pg_notify(22.22c(4) epoch 194455) v5 -204> 2014-08-13 16:24:29.959441 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 27 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7cc80 con 0x4e4dd80 -203> 2014-08-13 16:24:29.959488 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -202> 2014-08-13 16:24:29.959682 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.58 192.168.1.111:6834/14316 28 ==== pg_notify(22.1d2(17) epoch 194455) v5 ==== 1669+0+0 (141605430 0 0) 0x899c0c0 con 0x4e4dd80 -201> 2014-08-13 16:24:29.959727 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959407, event: header_read, request: pg_notify(22.1d2(17) epoch 194455) v5 -200> 2014-08-13 16:24:29.959742 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959408, event: throttled, request: pg_notify(22.1d2(17) epoch 194455) v5 -199> 2014-08-13 16:24:29.959752 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959611, event: all_read, request: pg_notify(22.1d2(17) epoch 194455) v5 -198> 2014-08-13 16:24:29.959759 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959721, event: dispatched, request: pg_notify(22.1d2(17) epoch 194455) v5 -197> 2014-08-13 16:24:29.959766 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959766, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194455) v5 -196> 2014-08-13 16:24:29.959778 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959778, event: started, request: pg_notify(22.1d2(17) epoch 194455) v5 -195> 2014-08-13 16:24:29.959819 7fd93dff0700 5 -- op tracker -- , seq: 1245, time: 2014-08-13 16:24:29.959818, event: done, request: pg_notify(22.1d2(17) epoch 194455) v5 -194> 2014-08-13 16:24:29.960650 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 15 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859f2c0 con 0x8e2c360 -193> 2014-08-13 16:24:29.960697 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -192> 2014-08-13 16:24:29.960716 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.380 192.168.1.114:6822/51844 16 ==== pg_notify(22.4df(15) epoch 194455) v5 ==== 1551+0+0 (896132186 0 0) 0x775c0c0 con 0x8e2c360 -191> 2014-08-13 16:24:29.960736 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960593, event: header_read, request: pg_notify(22.4df(15) epoch 194455) v5 -190> 2014-08-13 16:24:29.960750 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960594, event: throttled, request: pg_notify(22.4df(15) epoch 194455) v5 -189> 2014-08-13 16:24:29.960760 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960700, event: all_read, request: pg_notify(22.4df(15) epoch 194455) v5 -188> 2014-08-13 16:24:29.960769 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960733, event: dispatched, request: pg_notify(22.4df(15) epoch 194455) v5 -187> 2014-08-13 16:24:29.960780 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960780, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194455) v5 -186> 2014-08-13 16:24:29.960792 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960792, event: started, request: pg_notify(22.4df(15) epoch 194455) v5 -185> 2014-08-13 16:24:29.960826 7fd93dff0700 5 -- op tracker -- , seq: 1246, time: 2014-08-13 16:24:29.960826, event: done, request: pg_notify(22.4df(15) epoch 194455) v5 -184> 2014-08-13 16:24:29.964598 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 15 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7aac0 con 0x821f900 -183> 2014-08-13 16:24:29.964645 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -182> 2014-08-13 16:24:29.964691 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.63 192.168.1.111:6830/12680 16 ==== pg_notify(22.775(17) epoch 194455) v5 ==== 1665+0+0 (993465504 0 0) 0x8576580 con 0x821f900 -181> 2014-08-13 16:24:29.964714 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964558, event: header_read, request: pg_notify(22.775(17) epoch 194455) v5 -180> 2014-08-13 16:24:29.964725 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964559, event: throttled, request: pg_notify(22.775(17) epoch 194455) v5 -179> 2014-08-13 16:24:29.964731 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964658, event: all_read, request: pg_notify(22.775(17) epoch 194455) v5 -178> 2014-08-13 16:24:29.964737 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964710, event: dispatched, request: pg_notify(22.775(17) epoch 194455) v5 -177> 2014-08-13 16:24:29.964748 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964748, event: waiting_for_osdmap, request: pg_notify(22.775(17) epoch 194455) v5 -176> 2014-08-13 16:24:29.964761 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964761, event: started, request: pg_notify(22.775(17) epoch 194455) v5 -175> 2014-08-13 16:24:29.964798 7fd93dff0700 5 -- op tracker -- , seq: 1247, time: 2014-08-13 16:24:29.964798, event: done, request: pg_notify(22.775(17) epoch 194455) v5 -174> 2014-08-13 16:24:29.966650 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 13 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d06c00 con 0x7462100 -173> 2014-08-13 16:24:29.966699 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -172> 2014-08-13 16:24:29.966860 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.396 192.168.1.105:6814/41806 14 ==== pg_notify(22.3d9(35) epoch 194455) v5 ==== 2603+0+0 (226195587 0 0) 0x9287c40 con 0x7462100 -171> 2014-08-13 16:24:29.966904 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966631, event: header_read, request: pg_notify(22.3d9(35) epoch 194455) v5 -170> 2014-08-13 16:24:29.966919 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966632, event: throttled, request: pg_notify(22.3d9(35) epoch 194455) v5 -169> 2014-08-13 16:24:29.966927 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966749, event: all_read, request: pg_notify(22.3d9(35) epoch 194455) v5 -168> 2014-08-13 16:24:29.966935 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966898, event: dispatched, request: pg_notify(22.3d9(35) epoch 194455) v5 -167> 2014-08-13 16:24:29.966946 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966946, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194455) v5 -166> 2014-08-13 16:24:29.966960 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966959, event: started, request: pg_notify(22.3d9(35) epoch 194455) v5 -165> 2014-08-13 16:24:29.966996 7fd93dff0700 5 -- op tracker -- , seq: 1248, time: 2014-08-13 16:24:29.966995, event: done, request: pg_notify(22.3d9(35) epoch 194455) v5 -164> 2014-08-13 16:24:30.008581 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 14 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859de80 con 0x7d2d280 -163> 2014-08-13 16:24:30.008635 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -162> 2014-08-13 16:24:30.008662 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.314 192.168.1.101:6802/22303 15 ==== pg_notify(22.619(17) epoch 194455) v5 ==== 1665+0+0 (1195132393 0 0) 0x834e900 con 0x7d2d280 -161> 2014-08-13 16:24:30.008680 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008516, event: header_read, request: pg_notify(22.619(17) epoch 194455) v5 -160> 2014-08-13 16:24:30.008696 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008517, event: throttled, request: pg_notify(22.619(17) epoch 194455) v5 -159> 2014-08-13 16:24:30.008707 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008611, event: all_read, request: pg_notify(22.619(17) epoch 194455) v5 -158> 2014-08-13 16:24:30.008713 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008677, event: dispatched, request: pg_notify(22.619(17) epoch 194455) v5 -157> 2014-08-13 16:24:30.008723 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008723, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194455) v5 -156> 2014-08-13 16:24:30.008736 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008736, event: started, request: pg_notify(22.619(17) epoch 194455) v5 -155> 2014-08-13 16:24:30.008790 7fd93dff0700 5 -- op tracker -- , seq: 1249, time: 2014-08-13 16:24:30.008790, event: done, request: pg_notify(22.619(17) epoch 194455) v5 -154> 2014-08-13 16:24:30.009833 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 20 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7e780 con 0x7315ee0 -153> 2014-08-13 16:24:30.009880 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -152> 2014-08-13 16:24:30.009912 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.324 192.168.1.104:6814/65209 21 ==== pg_notify(24.39a(28) epoch 194455) v5 ==== 2300+0+0 (2747125601 0 0) 0x89988c0 con 0x7315ee0 -151> 2014-08-13 16:24:30.009935 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009787, event: header_read, request: pg_notify(24.39a(28) epoch 194455) v5 -150> 2014-08-13 16:24:30.009950 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009788, event: throttled, request: pg_notify(24.39a(28) epoch 194455) v5 -149> 2014-08-13 16:24:30.009957 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009892, event: all_read, request: pg_notify(24.39a(28) epoch 194455) v5 -148> 2014-08-13 16:24:30.009962 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009932, event: dispatched, request: pg_notify(24.39a(28) epoch 194455) v5 -147> 2014-08-13 16:24:30.009969 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009969, event: waiting_for_osdmap, request: pg_notify(24.39a(28) epoch 194455) v5 -146> 2014-08-13 16:24:30.009979 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.009979, event: started, request: pg_notify(24.39a(28) epoch 194455) v5 -145> 2014-08-13 16:24:30.010014 7fd93dff0700 5 -- op tracker -- , seq: 1250, time: 2014-08-13 16:24:30.010014, event: done, request: pg_notify(24.39a(28) epoch 194455) v5 -144> 2014-08-13 16:24:30.010323 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 32 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7d580 con 0x5108b00 -143> 2014-08-13 16:24:30.010370 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -142> 2014-08-13 16:24:30.010393 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 33 ==== pg_notify(22.3e4(4) epoch 194455) v5 ==== 942+0+0 (2466695548 0 0) 0x842d400 con 0x5108b00 -141> 2014-08-13 16:24:30.010414 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010265, event: header_read, request: pg_notify(22.3e4(4) epoch 194455) v5 -140> 2014-08-13 16:24:30.010429 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010266, event: throttled, request: pg_notify(22.3e4(4) epoch 194455) v5 -139> 2014-08-13 16:24:30.010438 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010359, event: all_read, request: pg_notify(22.3e4(4) epoch 194455) v5 -138> 2014-08-13 16:24:30.010448 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010410, event: dispatched, request: pg_notify(22.3e4(4) epoch 194455) v5 -137> 2014-08-13 16:24:30.010458 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010458, event: waiting_for_osdmap, request: pg_notify(22.3e4(4) epoch 194455) v5 -136> 2014-08-13 16:24:30.010471 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010471, event: started, request: pg_notify(22.3e4(4) epoch 194455) v5 -135> 2014-08-13 16:24:30.010509 7fd93dff0700 5 -- op tracker -- , seq: 1251, time: 2014-08-13 16:24:30.010509, event: done, request: pg_notify(22.3e4(4) epoch 194455) v5 -134> 2014-08-13 16:24:30.010533 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 34 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7e7af40 con 0x5108b00 -133> 2014-08-13 16:24:30.010554 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -132> 2014-08-13 16:24:30.010566 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 35 ==== pg_notify(22.707(4) epoch 194455) v5 ==== 942+0+0 (2397082430 0 0) 0x899c980 con 0x5108b00 -131> 2014-08-13 16:24:30.010583 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010432, event: header_read, request: pg_notify(22.707(4) epoch 194455) v5 -130> 2014-08-13 16:24:30.010594 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010433, event: throttled, request: pg_notify(22.707(4) epoch 194455) v5 -129> 2014-08-13 16:24:30.010603 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010490, event: all_read, request: pg_notify(22.707(4) epoch 194455) v5 -128> 2014-08-13 16:24:30.010611 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010581, event: dispatched, request: pg_notify(22.707(4) epoch 194455) v5 -127> 2014-08-13 16:24:30.010621 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010621, event: waiting_for_osdmap, request: pg_notify(22.707(4) epoch 194455) v5 -126> 2014-08-13 16:24:30.010632 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010632, event: started, request: pg_notify(22.707(4) epoch 194455) v5 -125> 2014-08-13 16:24:30.010659 7fd93dff0700 5 -- op tracker -- , seq: 1252, time: 2014-08-13 16:24:30.010659, event: done, request: pg_notify(22.707(4) epoch 194455) v5 -124> 2014-08-13 16:24:30.011534 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.303 192.168.1.109:6802/30768 36 ==== pg_notify(22.481(27) epoch 194455) v5 ==== 2235+0+0 (502183564 0 0) 0x8998fc0 con 0x5108b00 -123> 2014-08-13 16:24:30.011578 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011324, event: header_read, request: pg_notify(22.481(27) epoch 194455) v5 -122> 2014-08-13 16:24:30.011593 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011326, event: throttled, request: pg_notify(22.481(27) epoch 194455) v5 -121> 2014-08-13 16:24:30.011603 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011457, event: all_read, request: pg_notify(22.481(27) epoch 194455) v5 -120> 2014-08-13 16:24:30.011613 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011572, event: dispatched, request: pg_notify(22.481(27) epoch 194455) v5 -119> 2014-08-13 16:24:30.011623 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011623, event: waiting_for_osdmap, request: pg_notify(22.481(27) epoch 194455) v5 -118> 2014-08-13 16:24:30.011636 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011636, event: started, request: pg_notify(22.481(27) epoch 194455) v5 -117> 2014-08-13 16:24:30.011670 7fd93dff0700 5 -- op tracker -- , seq: 1253, time: 2014-08-13 16:24:30.011670, event: done, request: pg_notify(22.481(27) epoch 194455) v5 -116> 2014-08-13 16:24:30.012695 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 26 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d04ec0 con 0x5113b20 -115> 2014-08-13 16:24:30.012742 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -114> 2014-08-13 16:24:30.012779 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.330 192.168.1.106:6802/4876 27 ==== pg_notify(22.1d2(17) epoch 194455) v5 ==== 1669+0+0 (1111291021 0 0) 0x840e040 con 0x5113b20 -113> 2014-08-13 16:24:30.012802 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012650, event: header_read, request: pg_notify(22.1d2(17) epoch 194455) v5 -112> 2014-08-13 16:24:30.012813 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012651, event: throttled, request: pg_notify(22.1d2(17) epoch 194455) v5 -111> 2014-08-13 16:24:30.012822 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012732, event: all_read, request: pg_notify(22.1d2(17) epoch 194455) v5 -110> 2014-08-13 16:24:30.012832 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012798, event: dispatched, request: pg_notify(22.1d2(17) epoch 194455) v5 -109> 2014-08-13 16:24:30.012843 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012843, event: waiting_for_osdmap, request: pg_notify(22.1d2(17) epoch 194455) v5 -108> 2014-08-13 16:24:30.012853 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012853, event: started, request: pg_notify(22.1d2(17) epoch 194455) v5 -107> 2014-08-13 16:24:30.012892 7fd93dff0700 5 -- op tracker -- , seq: 1254, time: 2014-08-13 16:24:30.012891, event: done, request: pg_notify(22.1d2(17) epoch 194455) v5 -106> 2014-08-13 16:24:30.017822 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 35 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1407609335 0 0) 0x8bc2100 con 0x5111fa0 -105> 2014-08-13 16:24:30.017867 7fd93dff0700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.017685, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -104> 2014-08-13 16:24:30.017883 7fd93dff0700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.017688, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -103> 2014-08-13 16:24:30.017900 7fd93dff0700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.017790, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -102> 2014-08-13 16:24:30.017910 7fd93dff0700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.017862, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -101> 2014-08-13 16:24:30.017926 7fd93dff0700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.017926, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -100> 2014-08-13 16:24:30.018014 7fd9367e4700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.018013, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -99> 2014-08-13 16:24:30.018057 7fd9367e4700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.018056, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -98> 2014-08-13 16:24:30.018076 7fd9367e4700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8003c00 con 0x5111fa0 -97> 2014-08-13 16:24:30.018106 7fd9367e4700 5 -- op tracker -- , seq: 1255, time: 2014-08-13 16:24:30.018106, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -96> 2014-08-13 16:24:30.019311 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 28 ==== osd_map(194452..194455 src has 185139..194455) v3 ==== 2929+0+0 (1143447685 0 0) 0x7b15e80 con 0x510fd20 -95> 2014-08-13 16:24:30.019345 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194452,194455], i have 194455, src has [185139,194455] -94> 2014-08-13 16:24:30.019550 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 29 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (117323948 0 0) 0x86dac00 con 0x510fd20 -93> 2014-08-13 16:24:30.019600 7fd93dff0700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019383, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -92> 2014-08-13 16:24:30.019616 7fd93dff0700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019385, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -91> 2014-08-13 16:24:30.019626 7fd93dff0700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019472, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -90> 2014-08-13 16:24:30.019639 7fd93dff0700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019594, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -89> 2014-08-13 16:24:30.019656 7fd93dff0700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019655, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -88> 2014-08-13 16:24:30.019733 7fd9371e5700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019732, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -87> 2014-08-13 16:24:30.019764 7fd9371e5700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019763, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -86> 2014-08-13 16:24:30.019804 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> 192.168.1.113:6812/16056 -- osd_sub_op_reply(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x8d47d00 con 0x510fd20 -85> 2014-08-13 16:24:30.019836 7fd9371e5700 5 -- op tracker -- , seq: 1256, time: 2014-08-13 16:24:30.019835, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -84> 2014-08-13 16:24:30.019964 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.397 192.168.1.105:6808/41085 36 ==== replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194455,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (3256394998 0 0) 0x7e7fbc0 con 0x5111fa0 -83> 2014-08-13 16:24:30.020911 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.292 192.168.1.113:6812/16056 30 ==== osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2544888585 0 0) 0x86d8000 con 0x510fd20 -82> 2014-08-13 16:24:30.020948 7fd93dff0700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.020790, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -81> 2014-08-13 16:24:30.020961 7fd93dff0700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.020792, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -80> 2014-08-13 16:24:30.020974 7fd93dff0700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.020881, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -79> 2014-08-13 16:24:30.020987 7fd93dff0700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.020943, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -78> 2014-08-13 16:24:30.020998 7fd93dff0700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.020997, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -77> 2014-08-13 16:24:30.021099 7fd9367e4700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.021099, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -76> 2014-08-13 16:24:30.021142 7fd9367e4700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.021141, event: started, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -75> 2014-08-13 16:24:30.021171 7fd9367e4700 5 -- op tracker -- , seq: 1257, time: 2014-08-13 16:24:30.021171, event: done, request: osd_sub_op(unknown.0.0:0 22.3ad 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -74> 2014-08-13 16:24:30.024576 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 15 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7b13180 con 0x8da1760 -73> 2014-08-13 16:24:30.024624 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -72> 2014-08-13 16:24:30.024854 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.48 192.168.1.112:6838/8300 16 ==== pg_notify(22.619(17) epoch 194455) v5 ==== 1665+0+0 (1195132393 0 0) 0x92855c0 con 0x8da1760 -71> 2014-08-13 16:24:30.024899 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024637, event: header_read, request: pg_notify(22.619(17) epoch 194455) v5 -70> 2014-08-13 16:24:30.024915 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024641, event: throttled, request: pg_notify(22.619(17) epoch 194455) v5 -69> 2014-08-13 16:24:30.024925 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024778, event: all_read, request: pg_notify(22.619(17) epoch 194455) v5 -68> 2014-08-13 16:24:30.024935 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024893, event: dispatched, request: pg_notify(22.619(17) epoch 194455) v5 -67> 2014-08-13 16:24:30.024945 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024945, event: waiting_for_osdmap, request: pg_notify(22.619(17) epoch 194455) v5 -66> 2014-08-13 16:24:30.024959 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024958, event: started, request: pg_notify(22.619(17) epoch 194455) v5 -65> 2014-08-13 16:24:30.025000 7fd93dff0700 5 -- op tracker -- , seq: 1258, time: 2014-08-13 16:24:30.024999, event: done, request: pg_notify(22.619(17) epoch 194455) v5 -64> 2014-08-13 16:24:30.080946 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 15 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859a880 con 0x72de9e0 -63> 2014-08-13 16:24:30.080994 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -62> 2014-08-13 16:24:30.081013 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 21 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x7b14ec0 con 0x5112260 -61> 2014-08-13 16:24:30.081036 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -60> 2014-08-13 16:24:30.081054 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.366 192.168.1.108:6828/4534 16 ==== pg_notify(22.39d(4) epoch 194455) v5 ==== 942+0+0 (1670629649 0 0) 0x834d400 con 0x72de9e0 -59> 2014-08-13 16:24:30.081073 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.080908, event: header_read, request: pg_notify(22.39d(4) epoch 194455) v5 -58> 2014-08-13 16:24:30.081085 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.080910, event: throttled, request: pg_notify(22.39d(4) epoch 194455) v5 -57> 2014-08-13 16:24:30.081095 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.080994, event: all_read, request: pg_notify(22.39d(4) epoch 194455) v5 -56> 2014-08-13 16:24:30.081103 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.081070, event: dispatched, request: pg_notify(22.39d(4) epoch 194455) v5 -55> 2014-08-13 16:24:30.081111 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.081110, event: waiting_for_osdmap, request: pg_notify(22.39d(4) epoch 194455) v5 -54> 2014-08-13 16:24:30.081120 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.081119, event: started, request: pg_notify(22.39d(4) epoch 194455) v5 -53> 2014-08-13 16:24:30.081158 7fd93dff0700 5 -- op tracker -- , seq: 1259, time: 2014-08-13 16:24:30.081157, event: done, request: pg_notify(22.39d(4) epoch 194455) v5 -52> 2014-08-13 16:24:30.081196 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.371 192.168.1.102:6832/16753 22 ==== pg_notify(22.4df(15) epoch 194455) v5 ==== 1551+0+0 (896132186 0 0) 0x8c89340 con 0x5112260 -51> 2014-08-13 16:24:30.081213 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.080972, event: header_read, request: pg_notify(22.4df(15) epoch 194455) v5 -50> 2014-08-13 16:24:30.081220 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.080974, event: throttled, request: pg_notify(22.4df(15) epoch 194455) v5 -49> 2014-08-13 16:24:30.081226 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.081062, event: all_read, request: pg_notify(22.4df(15) epoch 194455) v5 -48> 2014-08-13 16:24:30.081231 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.081211, event: dispatched, request: pg_notify(22.4df(15) epoch 194455) v5 -47> 2014-08-13 16:24:30.081237 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.081237, event: waiting_for_osdmap, request: pg_notify(22.4df(15) epoch 194455) v5 -46> 2014-08-13 16:24:30.081245 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.081245, event: started, request: pg_notify(22.4df(15) epoch 194455) v5 -45> 2014-08-13 16:24:30.081276 7fd93dff0700 5 -- op tracker -- , seq: 1260, time: 2014-08-13 16:24:30.081276, event: done, request: pg_notify(22.4df(15) epoch 194455) v5 -44> 2014-08-13 16:24:30.188275 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 16 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859e540 con 0x73ca680 -43> 2014-08-13 16:24:30.188327 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -42> 2014-08-13 16:24:30.188342 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.11 192.168.1.103:6813/37929 17 ==== pg_notify(22.1f1(4) epoch 194455) v5 ==== 942+0+0 (83689726 0 0) 0x91116c0 con 0x73ca680 -41> 2014-08-13 16:24:30.188358 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188222, event: header_read, request: pg_notify(22.1f1(4) epoch 194455) v5 -40> 2014-08-13 16:24:30.188370 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188223, event: throttled, request: pg_notify(22.1f1(4) epoch 194455) v5 -39> 2014-08-13 16:24:30.188376 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188308, event: all_read, request: pg_notify(22.1f1(4) epoch 194455) v5 -38> 2014-08-13 16:24:30.188382 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188355, event: dispatched, request: pg_notify(22.1f1(4) epoch 194455) v5 -37> 2014-08-13 16:24:30.188389 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188389, event: waiting_for_osdmap, request: pg_notify(22.1f1(4) epoch 194455) v5 -36> 2014-08-13 16:24:30.188399 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188398, event: started, request: pg_notify(22.1f1(4) epoch 194455) v5 -35> 2014-08-13 16:24:30.188436 7fd93dff0700 5 -- op tracker -- , seq: 1261, time: 2014-08-13 16:24:30.188435, event: done, request: pg_notify(22.1f1(4) epoch 194455) v5 -34> 2014-08-13 16:24:30.190792 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 31 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x859af40 con 0x4e4f900 -33> 2014-08-13 16:24:30.190839 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -32> 2014-08-13 16:24:30.190882 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.30 192.168.1.107:6823/55491 32 ==== pg_notify(24.34(43) epoch 194455) v5 ==== 2933+0+0 (2363619434 0 0) 0x9110fc0 con 0x4e4f900 -31> 2014-08-13 16:24:30.190905 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190732, event: header_read, request: pg_notify(24.34(43) epoch 194455) v5 -30> 2014-08-13 16:24:30.190916 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190733, event: throttled, request: pg_notify(24.34(43) epoch 194455) v5 -29> 2014-08-13 16:24:30.190923 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190862, event: all_read, request: pg_notify(24.34(43) epoch 194455) v5 -28> 2014-08-13 16:24:30.190932 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190901, event: dispatched, request: pg_notify(24.34(43) epoch 194455) v5 -27> 2014-08-13 16:24:30.190943 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190942, event: waiting_for_osdmap, request: pg_notify(24.34(43) epoch 194455) v5 -26> 2014-08-13 16:24:30.190955 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190955, event: started, request: pg_notify(24.34(43) epoch 194455) v5 -25> 2014-08-13 16:24:30.190992 7fd93dff0700 5 -- op tracker -- , seq: 1262, time: 2014-08-13 16:24:30.190991, event: done, request: pg_notify(24.34(43) epoch 194455) v5 -24> 2014-08-13 16:24:30.229040 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 21 ==== osd_map(194455..194455 src has 185139..194455) v3 ==== 422+0+0 (2361729950 0 0) 0x8d057c0 con 0x4e4ce60 -23> 2014-08-13 16:24:30.229090 7fd93dff0700 3 osd.4 194455 handle_osd_map epochs [194455,194455], i have 194455, src has [185139,194455] -22> 2014-08-13 16:24:30.229107 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.54 192.168.1.111:6836/14923 22 ==== pg_notify(22.1f1(17) epoch 194455) v5 ==== 1665+0+0 (1582334863 0 0) 0x9284d00 con 0x4e4ce60 -21> 2014-08-13 16:24:30.229125 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.228982, event: header_read, request: pg_notify(22.1f1(17) epoch 194455) v5 -20> 2014-08-13 16:24:30.229141 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.228984, event: throttled, request: pg_notify(22.1f1(17) epoch 194455) v5 -19> 2014-08-13 16:24:30.229151 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.229082, event: all_read, request: pg_notify(22.1f1(17) epoch 194455) v5 -18> 2014-08-13 16:24:30.229157 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.229121, event: dispatched, request: pg_notify(22.1f1(17) epoch 194455) v5 -17> 2014-08-13 16:24:30.229180 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.229180, event: waiting_for_osdmap, request: pg_notify(22.1f1(17) epoch 194455) v5 -16> 2014-08-13 16:24:30.229194 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.229194, event: started, request: pg_notify(22.1f1(17) epoch 194455) v5 -15> 2014-08-13 16:24:30.229228 7fd93dff0700 5 -- op tracker -- , seq: 1263, time: 2014-08-13 16:24:30.229228, event: done, request: pg_notify(22.1f1(17) epoch 194455) v5 -14> 2014-08-13 16:24:30.303603 7fd93dff0700 1 -- 192.168.1.115:6813/25220 <== osd.27 192.168.1.107:6802/9548 7 ==== pg_info(1 pgs e194454:24.34) v4 ==== 740+0+0 (3730227456 0 0) 0x842e040 con 0x83002c0 -13> 2014-08-13 16:24:30.303647 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303416, event: header_read, request: pg_info(1 pgs e194454:24.34) v4 -12> 2014-08-13 16:24:30.303660 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303420, event: throttled, request: pg_info(1 pgs e194454:24.34) v4 -11> 2014-08-13 16:24:30.303666 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303522, event: all_read, request: pg_info(1 pgs e194454:24.34) v4 -10> 2014-08-13 16:24:30.303671 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303641, event: dispatched, request: pg_info(1 pgs e194454:24.34) v4 -9> 2014-08-13 16:24:30.303678 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303678, event: waiting_for_osdmap, request: pg_info(1 pgs e194454:24.34) v4 -8> 2014-08-13 16:24:30.303688 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303688, event: started, request: pg_info(1 pgs e194454:24.34) v4 -7> 2014-08-13 16:24:30.303718 7fd93dff0700 5 -- op tracker -- , seq: 1264, time: 2014-08-13 16:24:30.303718, event: done, request: pg_info(1 pgs e194454:24.34) v4 -6> 2014-08-13 16:24:30.303803 7fd9371e5700 1 -- 192.168.1.115:6813/25220 --> osd.27 192.168.1.107:6802/9548 -- pg_info(1 pgs e194455:24.34) v4 -- ?+0 0x834a840 -5> 2014-08-13 16:24:30.304280 7fd9371e5700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate -4> 2014-08-13 16:24:30.304885 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 <== osd.54 192.168.100.111:0/14923 8 ==== osd_ping(ping e194455 stamp 2014-08-13 16:24:30.304457) v2 ==== 47+0+0 (3697782309 0 0) 0x8998a80 con 0x7edbc80 -3> 2014-08-13 16:24:30.304897 7fd93adeb700 1 -- 192.168.1.115:6817/25220 <== osd.54 192.168.100.111:0/14923 8 ==== osd_ping(ping e194455 stamp 2014-08-13 16:24:30.304457) v2 ==== 47+0+0 (3697782309 0 0) 0x899bd40 con 0x7ffe9e0 -2> 2014-08-13 16:24:30.304942 7fd93c1ed700 1 -- 192.168.100.115:6806/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194455 stamp 2014-08-13 16:24:30.304457) v2 -- ?+0 0x8c8f1c0 con 0x7edbc80 -1> 2014-08-13 16:24:30.305019 7fd93adeb700 1 -- 192.168.1.115:6817/25220 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194455 stamp 2014-08-13 16:24:30.304457) v2 -- ?+0 0x7d01f80 con 0x7ffe9e0 0> 2014-08-13 16:24:30.309373 7fd9371e5700 -1 *** Caught signal (Floating point exception) ** in thread 7fd9371e5700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fd950d53710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fd950d4b9d1] 16: (clone()+0x6d) [0x7fd94fcdbb6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.4.log --- end dump of recent events --- 2014-08-13 17:22:48.059016 7f74ed1707a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 30165 2014-08-13 17:22:48.068793 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:22:48.068825 7f74ed1707a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-08-13 17:22:48.071633 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:22:48.071658 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:22:48.086704 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:22:48.086819 7f74ed1707a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:22:48.131471 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-13 17:22:48.137291 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:22:48.222989 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:22:48.235395 7f74ed1707a0 1 journal close /var/lib/ceph/osd/ceph-4/journal 2014-08-13 17:22:48.237922 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:22:48.240649 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:22:48.240665 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:22:48.259661 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:22:48.259749 7f74ed1707a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:22:48.264866 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-13 17:22:48.270062 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:22:48.289003 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:22:48.301033 7f74ed1707a0 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-13 17:22:48.303654 7f74ed1707a0 0 osd.4 194455 crush map has features 2303210029056, adjusting msgr requires for clients 2014-08-13 17:22:48.303679 7f74ed1707a0 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:22:48.303688 7f74ed1707a0 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for osds 2014-08-13 17:22:48.303711 7f74ed1707a0 0 osd.4 194455 load_pgs 2014-08-13 17:22:48.681398 7f74ed1707a0 0 osd.4 194455 load_pgs opened 72 pgs 2014-08-13 17:22:48.689704 7f74d9b37700 0 osd.4 194455 ignoring osdmap until we have initialized 2014-08-13 17:22:48.690691 7f74d9b37700 0 osd.4 194455 ignoring osdmap until we have initialized 2014-08-13 17:22:48.708771 7f74c5bd9700 0 -- 192.168.1.115:6813/30165 >> 192.168.1.107:6802/9548 pipe(0x2ef5780 sd=91 :0 s=1 pgs=0 cs=0 l=0 c=0x47f5120).fault with nothing to send, going to standby 2014-08-13 17:22:48.721722 7f74ed1707a0 0 osd.4 194455 done with init, starting boot process 2014-08-13 17:22:48.891397 7f74d8735700 0 osd.4 194470 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:22:49.802122 7f74d9b37700 0 osd.4 194471 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:22:50.845711 7f74d9b37700 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:22:51.959378 7f74d192a700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 2014-08-13 17:22:51.960077 7f74d0f29700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 2014-08-13 17:22:51.960655 7f74d0f29700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 2014-08-13 17:22:51.970328 7f74d192a700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 2014-08-13 17:22:51.970739 7f74d192a700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 2014-08-13 17:22:51.975794 7f74d192a700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 2014-08-13 17:22:51.976195 7f74d192a700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 2014-08-13 17:22:51.980996 7f74d192a700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 2014-08-13 17:22:51.981011 7f74d0f29700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 2014-08-13 17:22:51.981693 7f74d0f29700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 2014-08-13 17:22:52.001535 7f74d192a700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 2014-08-13 17:22:52.003174 7f74d192a700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 2014-08-13 17:22:52.023203 7f74d0f29700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 17:22:52.028252 7f74d0f29700 -1 *** Caught signal (Floating point exception) ** in thread 7f74d0f29700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7f74ec11d710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7f74ec1159d1] 16: (clone()+0x6d) [0x7f74eb0a5b6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -9516> 2014-08-13 17:22:48.054987 7f74ed1707a0 5 asok(0x2700230) register_command perfcounters_dump hook 0x26b8010 -9515> 2014-08-13 17:22:48.055058 7f74ed1707a0 5 asok(0x2700230) register_command 1 hook 0x26b8010 -9514> 2014-08-13 17:22:48.055070 7f74ed1707a0 5 asok(0x2700230) register_command perf dump hook 0x26b8010 -9513> 2014-08-13 17:22:48.055080 7f74ed1707a0 5 asok(0x2700230) register_command perfcounters_schema hook 0x26b8010 -9512> 2014-08-13 17:22:48.055087 7f74ed1707a0 5 asok(0x2700230) register_command 2 hook 0x26b8010 -9511> 2014-08-13 17:22:48.055093 7f74ed1707a0 5 asok(0x2700230) register_command perf schema hook 0x26b8010 -9510> 2014-08-13 17:22:48.055098 7f74ed1707a0 5 asok(0x2700230) register_command config show hook 0x26b8010 -9509> 2014-08-13 17:22:48.055107 7f74ed1707a0 5 asok(0x2700230) register_command config set hook 0x26b8010 -9508> 2014-08-13 17:22:48.055113 7f74ed1707a0 5 asok(0x2700230) register_command config get hook 0x26b8010 -9507> 2014-08-13 17:22:48.055118 7f74ed1707a0 5 asok(0x2700230) register_command log flush hook 0x26b8010 -9506> 2014-08-13 17:22:48.055124 7f74ed1707a0 5 asok(0x2700230) register_command log dump hook 0x26b8010 -9505> 2014-08-13 17:22:48.055130 7f74ed1707a0 5 asok(0x2700230) register_command log reopen hook 0x26b8010 -9504> 2014-08-13 17:22:48.059016 7f74ed1707a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 30165 -9503> 2014-08-13 17:22:48.060970 7f74ed1707a0 1 -- 192.168.100.115:0/0 learned my addr 192.168.100.115:0/0 -9502> 2014-08-13 17:22:48.060991 7f74ed1707a0 1 accepter.accepter.bind my_inst.addr is 192.168.100.115:6805/30165 need_addr=0 -9501> 2014-08-13 17:22:48.061047 7f74ed1707a0 1 -- 192.168.1.115:0/0 learned my addr 192.168.1.115:0/0 -9500> 2014-08-13 17:22:48.061059 7f74ed1707a0 1 accepter.accepter.bind my_inst.addr is 192.168.1.115:6813/30165 need_addr=0 -9499> 2014-08-13 17:22:48.061104 7f74ed1707a0 1 -- 192.168.1.115:0/0 learned my addr 192.168.1.115:0/0 -9498> 2014-08-13 17:22:48.061115 7f74ed1707a0 1 accepter.accepter.bind my_inst.addr is 192.168.1.115:6817/30165 need_addr=0 -9497> 2014-08-13 17:22:48.061164 7f74ed1707a0 1 -- 192.168.100.115:0/0 learned my addr 192.168.100.115:0/0 -9496> 2014-08-13 17:22:48.061176 7f74ed1707a0 1 accepter.accepter.bind my_inst.addr is 192.168.100.115:6806/30165 need_addr=0 -9495> 2014-08-13 17:22:48.061214 7f74ed1707a0 1 -- 192.168.100.115:0/0 learned my addr 192.168.100.115:0/0 -9494> 2014-08-13 17:22:48.061225 7f74ed1707a0 1 accepter.accepter.bind my_inst.addr is 192.168.100.115:6808/30165 need_addr=0 -9493> 2014-08-13 17:22:48.061836 7f74ed1707a0 1 finished global_init_daemonize -9492> 2014-08-13 17:22:48.067490 7f74ed1707a0 5 asok(0x2700230) init /var/run/ceph/ceph-osd.4.asok -9491> 2014-08-13 17:22:48.067532 7f74ed1707a0 5 asok(0x2700230) bind_and_listen /var/run/ceph/ceph-osd.4.asok -9490> 2014-08-13 17:22:48.067766 7f74ed1707a0 5 asok(0x2700230) register_command 0 hook 0x26b00b0 -9489> 2014-08-13 17:22:48.067789 7f74ed1707a0 5 asok(0x2700230) register_command version hook 0x26b00b0 -9488> 2014-08-13 17:22:48.067797 7f74ed1707a0 5 asok(0x2700230) register_command git_version hook 0x26b00b0 -9487> 2014-08-13 17:22:48.067805 7f74ed1707a0 5 asok(0x2700230) register_command help hook 0x26b80b0 -9486> 2014-08-13 17:22:48.067817 7f74ed1707a0 5 asok(0x2700230) register_command get_command_descriptions hook 0x26b8150 -9485> 2014-08-13 17:22:48.067909 7f74e8817700 5 asok(0x2700230) entry start -9484> 2014-08-13 17:22:48.068793 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) -9483> 2014-08-13 17:22:48.068825 7f74ed1707a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs -9482> 2014-08-13 17:22:48.071633 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work -9481> 2014-08-13 17:22:48.071658 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -9480> 2014-08-13 17:22:48.086704 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported -9479> 2014-08-13 17:22:48.086819 7f74ed1707a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf -9478> 2014-08-13 17:22:48.131471 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled -9477> 2014-08-13 17:22:48.133252 7f74ed1707a0 2 journal open /var/lib/ceph/osd/ceph-4/journal fsid cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 fs_op_seq 16298032 -9476> 2014-08-13 17:22:48.137291 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -9475> 2014-08-13 17:22:48.137745 7f74ed1707a0 2 journal read_entry 800899072 : seq 16298033 3724 bytes -9474> 2014-08-13 17:22:48.137784 7f74ed1707a0 2 journal read_entry 800899072 : seq 16298033 3724 bytes -9473> 2014-08-13 17:22:48.137788 7f74ed1707a0 3 journal journal_replay: applying op seq 16298033 -9472> 2014-08-13 17:22:48.138378 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298033 -9471> 2014-08-13 17:22:48.140555 7f74ed1707a0 2 journal read_entry 801177600 : seq 16298034 271109 bytes -9470> 2014-08-13 17:22:48.140572 7f74ed1707a0 3 journal journal_replay: applying op seq 16298034 -9469> 2014-08-13 17:22:48.141329 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298034 -9468> 2014-08-13 17:22:48.141358 7f74ed1707a0 2 journal read_entry 801181696 : seq 16298035 43 bytes -9467> 2014-08-13 17:22:48.141366 7f74ed1707a0 3 journal journal_replay: applying op seq 16298035 -9466> 2014-08-13 17:22:48.141383 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298035 -9465> 2014-08-13 17:22:48.142406 7f74ed1707a0 2 journal read_entry 801460224 : seq 16298036 271515 bytes -9464> 2014-08-13 17:22:48.142421 7f74ed1707a0 3 journal journal_replay: applying op seq 16298036 -9463> 2014-08-13 17:22:48.143070 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298036 -9462> 2014-08-13 17:22:48.143101 7f74ed1707a0 2 journal read_entry 801464320 : seq 16298037 3728 bytes -9461> 2014-08-13 17:22:48.143109 7f74ed1707a0 3 journal journal_replay: applying op seq 16298037 -9460> 2014-08-13 17:22:48.143334 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298037 -9459> 2014-08-13 17:22:48.143391 7f74ed1707a0 2 journal read_entry 801468416 : seq 16298038 3090 bytes -9458> 2014-08-13 17:22:48.143400 7f74ed1707a0 3 journal journal_replay: applying op seq 16298038 -9457> 2014-08-13 17:22:48.143590 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298038 -9456> 2014-08-13 17:22:48.143616 7f74ed1707a0 2 journal read_entry 801472512 : seq 16298039 2323 bytes -9455> 2014-08-13 17:22:48.143624 7f74ed1707a0 3 journal journal_replay: applying op seq 16298039 -9454> 2014-08-13 17:22:48.143792 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298039 -9453> 2014-08-13 17:22:48.143818 7f74ed1707a0 2 journal read_entry 801476608 : seq 16298040 43 bytes -9452> 2014-08-13 17:22:48.143825 7f74ed1707a0 3 journal journal_replay: applying op seq 16298040 -9451> 2014-08-13 17:22:48.143832 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298040 -9450> 2014-08-13 17:22:48.144877 7f74ed1707a0 2 journal read_entry 801751040 : seq 16298041 270843 bytes -9449> 2014-08-13 17:22:48.144891 7f74ed1707a0 3 journal journal_replay: applying op seq 16298041 -9448> 2014-08-13 17:22:48.145557 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298041 -9447> 2014-08-13 17:22:48.145587 7f74ed1707a0 2 journal read_entry 801755136 : seq 16298042 3094 bytes -9446> 2014-08-13 17:22:48.145595 7f74ed1707a0 3 journal journal_replay: applying op seq 16298042 -9445> 2014-08-13 17:22:48.145785 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298042 -9444> 2014-08-13 17:22:48.145812 7f74ed1707a0 2 journal read_entry 801759232 : seq 16298043 2382 bytes -9443> 2014-08-13 17:22:48.145819 7f74ed1707a0 3 journal journal_replay: applying op seq 16298043 -9442> 2014-08-13 17:22:48.145975 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298043 -9441> 2014-08-13 17:22:48.146012 7f74ed1707a0 2 journal read_entry 801763328 : seq 16298044 844 bytes -9440> 2014-08-13 17:22:48.146020 7f74ed1707a0 3 journal journal_replay: applying op seq 16298044 -9439> 2014-08-13 17:22:48.146191 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298044 -9438> 2014-08-13 17:22:48.160122 7f74ed1707a0 2 journal read_entry 802037760 : seq 16298045 269896 bytes -9437> 2014-08-13 17:22:48.160141 7f74ed1707a0 3 journal journal_replay: applying op seq 16298045 -9436> 2014-08-13 17:22:48.160810 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298045 -9435> 2014-08-13 17:22:48.160840 7f74ed1707a0 2 journal read_entry 802041856 : seq 16298046 2382 bytes -9434> 2014-08-13 17:22:48.160845 7f74ed1707a0 3 journal journal_replay: applying op seq 16298046 -9433> 2014-08-13 17:22:48.161082 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298046 -9432> 2014-08-13 17:22:48.161115 7f74ed1707a0 2 journal read_entry 802045952 : seq 16298047 876 bytes -9431> 2014-08-13 17:22:48.161120 7f74ed1707a0 3 journal journal_replay: applying op seq 16298047 -9430> 2014-08-13 17:22:48.161283 7f74ed1707a0 3 journal journal_replay: r = 0, op_seq now 16298047 -9429> 2014-08-13 17:22:48.161305 7f74ed1707a0 2 journal No further valid entries found, journal is most likely valid -9428> 2014-08-13 17:22:48.161315 7f74ed1707a0 3 journal journal_replay: end of journal, done. -9427> 2014-08-13 17:22:48.222989 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -9426> 2014-08-13 17:22:48.235156 7f74e3bf3700 1 FileStore::op_tp worker finish -9425> 2014-08-13 17:22:48.235203 7f74e31f2700 1 FileStore::op_tp worker finish -9424> 2014-08-13 17:22:48.235395 7f74ed1707a0 1 journal close /var/lib/ceph/osd/ceph-4/journal -9423> 2014-08-13 17:22:48.237185 7f74ed1707a0 10 monclient(hunting): build_initial_monmap -9422> 2014-08-13 17:22:48.237364 7f74ed1707a0 5 adding auth protocol: cephx -9421> 2014-08-13 17:22:48.237374 7f74ed1707a0 5 adding auth protocol: cephx -9420> 2014-08-13 17:22:48.237532 7f74ed1707a0 1 -- 192.168.100.115:6805/30165 messenger.start -9419> 2014-08-13 17:22:48.237582 7f74ed1707a0 1 -- :/0 messenger.start -9418> 2014-08-13 17:22:48.237614 7f74ed1707a0 1 -- 192.168.100.115:6806/30165 messenger.start -9417> 2014-08-13 17:22:48.237637 7f74ed1707a0 1 -- 192.168.1.115:6817/30165 messenger.start -9416> 2014-08-13 17:22:48.237683 7f74ed1707a0 1 -- 192.168.1.115:6813/30165 messenger.start -9415> 2014-08-13 17:22:48.237728 7f74ed1707a0 1 -- 192.168.100.115:6808/30165 messenger.start -9414> 2014-08-13 17:22:48.237828 7f74ed1707a0 2 osd.4 0 mounting /var/lib/ceph/osd/ceph-4 /var/lib/ceph/osd/ceph-4/journal -9413> 2014-08-13 17:22:48.237922 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) -9412> 2014-08-13 17:22:48.240649 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work -9411> 2014-08-13 17:22:48.240665 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -9410> 2014-08-13 17:22:48.259661 7f74ed1707a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported -9409> 2014-08-13 17:22:48.259749 7f74ed1707a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf -9408> 2014-08-13 17:22:48.264866 7f74ed1707a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf -9407> 2014-08-13 17:22:48.266177 7f74ed1707a0 2 journal open /var/lib/ceph/osd/ceph-4/journal fsid cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 fs_op_seq 16298047 -9406> 2014-08-13 17:22:48.270062 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -9405> 2014-08-13 17:22:48.270238 7f74ed1707a0 2 journal read_entry 800899072 : seq 16298033 3724 bytes -9404> 2014-08-13 17:22:48.271951 7f74ed1707a0 2 journal read_entry 801177600 : seq 16298034 271109 bytes -9403> 2014-08-13 17:22:48.271980 7f74ed1707a0 2 journal read_entry 801181696 : seq 16298035 43 bytes -9402> 2014-08-13 17:22:48.272783 7f74ed1707a0 2 journal read_entry 801460224 : seq 16298036 271515 bytes -9401> 2014-08-13 17:22:48.272813 7f74ed1707a0 2 journal read_entry 801464320 : seq 16298037 3728 bytes -9400> 2014-08-13 17:22:48.272839 7f74ed1707a0 2 journal read_entry 801468416 : seq 16298038 3090 bytes -9399> 2014-08-13 17:22:48.272863 7f74ed1707a0 2 journal read_entry 801472512 : seq 16298039 2323 bytes -9398> 2014-08-13 17:22:48.272877 7f74ed1707a0 2 journal read_entry 801476608 : seq 16298040 43 bytes -9397> 2014-08-13 17:22:48.273706 7f74ed1707a0 2 journal read_entry 801751040 : seq 16298041 270843 bytes -9396> 2014-08-13 17:22:48.273734 7f74ed1707a0 2 journal read_entry 801755136 : seq 16298042 3094 bytes -9395> 2014-08-13 17:22:48.273746 7f74ed1707a0 2 journal read_entry 801759232 : seq 16298043 2382 bytes -9394> 2014-08-13 17:22:48.273766 7f74ed1707a0 2 journal read_entry 801763328 : seq 16298044 844 bytes -9393> 2014-08-13 17:22:48.284994 7f74ed1707a0 2 journal read_entry 802037760 : seq 16298045 269896 bytes -9392> 2014-08-13 17:22:48.285034 7f74ed1707a0 2 journal read_entry 802041856 : seq 16298046 2382 bytes -9391> 2014-08-13 17:22:48.285047 7f74ed1707a0 2 journal read_entry 802045952 : seq 16298047 876 bytes -9390> 2014-08-13 17:22:48.285057 7f74ed1707a0 2 journal No further valid entries found, journal is most likely valid -9389> 2014-08-13 17:22:48.285090 7f74ed1707a0 2 journal No further valid entries found, journal is most likely valid -9388> 2014-08-13 17:22:48.285096 7f74ed1707a0 3 journal journal_replay: end of journal, done. -9387> 2014-08-13 17:22:48.289003 7f74ed1707a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -9386> 2014-08-13 17:22:48.289435 7f74ed1707a0 2 osd.4 0 boot -9385> 2014-08-13 17:22:48.295463 7f74ed1707a0 1 cls/refcount/cls_refcount.cc:231: Loaded refcount class! -9384> 2014-08-13 17:22:48.295700 7f74ed1707a0 1 cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! -9383> 2014-08-13 17:22:48.295909 7f74ed1707a0 1 cls/statelog/cls_statelog.cc:306: Loaded log class! -9382> 2014-08-13 17:22:48.299983 7f74ed1707a0 1 cls/rgw/cls_rgw.cc:1599: Loaded rgw class! -9381> 2014-08-13 17:22:48.300281 7f74ed1707a0 1 cls/version/cls_version.cc:227: Loaded version class! -9380> 2014-08-13 17:22:48.301033 7f74ed1707a0 0 cls/hello/cls_hello.cc:271: loading cls_hello -9379> 2014-08-13 17:22:48.301322 7f74ed1707a0 1 cls/log/cls_log.cc:312: Loaded log class! -9378> 2014-08-13 17:22:48.301533 7f74ed1707a0 1 cls/user/cls_user.cc:367: Loaded user class! -9377> 2014-08-13 17:22:48.303654 7f74ed1707a0 0 osd.4 194455 crush map has features 2303210029056, adjusting msgr requires for clients -9376> 2014-08-13 17:22:48.303679 7f74ed1707a0 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for mons -9375> 2014-08-13 17:22:48.303688 7f74ed1707a0 0 osd.4 194455 crush map has features 2578087936000, adjusting msgr requires for osds -9374> 2014-08-13 17:22:48.303711 7f74ed1707a0 0 osd.4 194455 load_pgs -9373> 2014-08-13 17:22:48.308356 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[1.2b(unlocked)] enter Initial -9372> 2014-08-13 17:22:48.342836 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.034483 0 0.000000 -9371> 2014-08-13 17:22:48.342900 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -9370> 2014-08-13 17:22:48.343419 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[2.2a(unlocked)] enter Initial -9369> 2014-08-13 17:22:48.356721 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.013302 0 0.000000 -9368> 2014-08-13 17:22:48.356759 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -9367> 2014-08-13 17:22:48.357128 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.84(unlocked)] enter Initial -9366> 2014-08-13 17:22:48.359080 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001952 0 0.000000 -9365> 2014-08-13 17:22:48.359115 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset -9364> 2014-08-13 17:22:48.359479 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.99(unlocked)] enter Initial -9363> 2014-08-13 17:22:48.361296 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=0 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001816 0 0.000000 -9362> 2014-08-13 17:22:48.361333 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=0 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset -9361> 2014-08-13 17:22:48.361674 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.f3(unlocked)] enter Initial -9360> 2014-08-13 17:22:48.363863 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=0 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002188 0 0.000000 -9359> 2014-08-13 17:22:48.363899 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=0 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset -9358> 2014-08-13 17:22:48.364315 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.113(unlocked)] enter Initial -9357> 2014-08-13 17:22:48.366091 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=0 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001776 0 0.000000 -9356> 2014-08-13 17:22:48.366127 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=0 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset -9355> 2014-08-13 17:22:48.366504 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.116(unlocked)] enter Initial -9354> 2014-08-13 17:22:48.368312 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=0 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Initial 0.001808 0 0.000000 -9353> 2014-08-13 17:22:48.368348 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=0 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset -9352> 2014-08-13 17:22:48.368663 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.12a(unlocked)] enter Initial -9351> 2014-08-13 17:22:48.370843 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=0 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.002181 0 0.000000 -9350> 2014-08-13 17:22:48.370880 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=0 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -9349> 2014-08-13 17:22:48.371200 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.14a(unlocked)] enter Initial -9348> 2014-08-13 17:22:48.373379 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002180 0 0.000000 -9347> 2014-08-13 17:22:48.373417 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -9346> 2014-08-13 17:22:48.373719 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1a0(unlocked)] enter Initial -9345> 2014-08-13 17:22:48.375602 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=0 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.001884 0 0.000000 -9344> 2014-08-13 17:22:48.375639 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=0 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -9343> 2014-08-13 17:22:48.375983 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1b8(unlocked)] enter Initial -9342> 2014-08-13 17:22:48.377757 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001775 0 0.000000 -9341> 2014-08-13 17:22:48.377792 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset -9340> 2014-08-13 17:22:48.378121 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1d2(unlocked)] enter Initial -9339> 2014-08-13 17:22:48.379905 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=0 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001783 0 0.000000 -9338> 2014-08-13 17:22:48.379942 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=0 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset -9337> 2014-08-13 17:22:48.380290 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1f1(unlocked)] enter Initial -9336> 2014-08-13 17:22:48.382381 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=0 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002091 0 0.000000 -9335> 2014-08-13 17:22:48.382416 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=0 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset -9334> 2014-08-13 17:22:48.382756 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1fb(unlocked)] enter Initial -9333> 2014-08-13 17:22:48.385040 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=0 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.002284 0 0.000000 -9332> 2014-08-13 17:22:48.385076 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=0 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -9331> 2014-08-13 17:22:48.385398 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.220(unlocked)] enter Initial -9330> 2014-08-13 17:22:48.387523 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=0 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002124 0 0.000000 -9329> 2014-08-13 17:22:48.387559 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=0 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -9328> 2014-08-13 17:22:48.387864 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.226(unlocked)] enter Initial -9327> 2014-08-13 17:22:48.390059 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=0 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.002196 0 0.000000 -9326> 2014-08-13 17:22:48.390100 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=0 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -9325> 2014-08-13 17:22:48.390632 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.22c(unlocked)] enter Initial -9324> 2014-08-13 17:22:48.392692 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=0 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002060 0 0.000000 -9323> 2014-08-13 17:22:48.392729 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=0 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset -9322> 2014-08-13 17:22:48.393081 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.28f(unlocked)] enter Initial -9321> 2014-08-13 17:22:48.395073 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=0 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.001991 0 0.000000 -9320> 2014-08-13 17:22:48.395109 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=0 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -9319> 2014-08-13 17:22:48.395421 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.29a(unlocked)] enter Initial -9318> 2014-08-13 17:22:48.397080 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=0 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001659 0 0.000000 -9317> 2014-08-13 17:22:48.397117 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=0 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -9316> 2014-08-13 17:22:48.397442 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2b9(unlocked)] enter Initial -9315> 2014-08-13 17:22:48.399556 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Initial 0.002115 0 0.000000 -9314> 2014-08-13 17:22:48.399592 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset -9313> 2014-08-13 17:22:48.399928 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2d1(unlocked)] enter Initial -9312> 2014-08-13 17:22:48.402141 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=0 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Initial 0.002214 0 0.000000 -9311> 2014-08-13 17:22:48.402187 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=0 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset -9310> 2014-08-13 17:22:48.402506 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.30f(unlocked)] enter Initial -9309> 2014-08-13 17:22:48.404266 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=0 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Initial 0.001759 0 0.000000 -9308> 2014-08-13 17:22:48.404302 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=0 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset -9307> 2014-08-13 17:22:48.404631 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.362(unlocked)] enter Initial -9306> 2014-08-13 17:22:48.406392 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=0 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001761 0 0.000000 -9305> 2014-08-13 17:22:48.406428 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=0 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset -9304> 2014-08-13 17:22:48.406731 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.39d(unlocked)] enter Initial -9303> 2014-08-13 17:22:48.408499 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=0 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001769 0 0.000000 -9302> 2014-08-13 17:22:48.408536 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=0 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset -9301> 2014-08-13 17:22:48.408872 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3ad(unlocked)] enter Initial -9300> 2014-08-13 17:22:48.410780 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=0 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Initial 0.001908 0 0.000000 -9299> 2014-08-13 17:22:48.410816 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=0 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset -9298> 2014-08-13 17:22:48.411125 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3af(unlocked)] enter Initial -9297> 2014-08-13 17:22:48.413210 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=0 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002085 0 0.000000 -9296> 2014-08-13 17:22:48.413248 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=0 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset -9295> 2014-08-13 17:22:48.413558 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3b9(unlocked)] enter Initial -9294> 2014-08-13 17:22:48.415744 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=0 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.002186 0 0.000000 -9293> 2014-08-13 17:22:48.415781 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=0 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -9292> 2014-08-13 17:22:48.416102 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d5(unlocked)] enter Initial -9291> 2014-08-13 17:22:48.417575 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001473 0 0.000000 -9290> 2014-08-13 17:22:48.417609 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset -9289> 2014-08-13 17:22:48.417962 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d9(unlocked)] enter Initial -9288> 2014-08-13 17:22:48.419948 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=0 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001987 0 0.000000 -9287> 2014-08-13 17:22:48.419985 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=0 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset -9286> 2014-08-13 17:22:48.420297 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3e4(unlocked)] enter Initial -9285> 2014-08-13 17:22:48.422393 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=0 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002095 0 0.000000 -9284> 2014-08-13 17:22:48.422429 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=0 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -9283> 2014-08-13 17:22:48.422746 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.481(unlocked)] enter Initial -9282> 2014-08-13 17:22:48.424523 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=0 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001778 0 0.000000 -9281> 2014-08-13 17:22:48.424559 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=0 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset -9280> 2014-08-13 17:22:48.424908 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.48e(unlocked)] enter Initial -9279> 2014-08-13 17:22:48.426982 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=0 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002075 0 0.000000 -9278> 2014-08-13 17:22:48.427018 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=0 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset -9277> 2014-08-13 17:22:48.427344 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4aa(unlocked)] enter Initial -9276> 2014-08-13 17:22:48.429135 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=0 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Initial 0.001792 0 0.000000 -9275> 2014-08-13 17:22:48.429186 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=0 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset -9274> 2014-08-13 17:22:48.429567 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4df(unlocked)] enter Initial -9273> 2014-08-13 17:22:48.431475 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=0 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001908 0 0.000000 -9272> 2014-08-13 17:22:48.431512 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=0 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset -9271> 2014-08-13 17:22:48.431812 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4fe(unlocked)] enter Initial -9270> 2014-08-13 17:22:48.433738 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001927 0 0.000000 -9269> 2014-08-13 17:22:48.433774 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -9268> 2014-08-13 17:22:48.434149 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.537(unlocked)] enter Initial -9267> 2014-08-13 17:22:48.436008 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=0 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001860 0 0.000000 -9266> 2014-08-13 17:22:48.436044 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=0 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -9265> 2014-08-13 17:22:48.436362 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.581(unlocked)] enter Initial -9264> 2014-08-13 17:22:48.438281 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=0 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Initial 0.001920 0 0.000000 -9263> 2014-08-13 17:22:48.438302 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=0 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset -9262> 2014-08-13 17:22:48.438485 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.619(unlocked)] enter Initial -9261> 2014-08-13 17:22:48.439625 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=0 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001139 0 0.000000 -9260> 2014-08-13 17:22:48.439645 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=0 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset -9259> 2014-08-13 17:22:48.439834 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.630(unlocked)] enter Initial -9258> 2014-08-13 17:22:48.440861 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=0 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Initial 0.001028 0 0.000000 -9257> 2014-08-13 17:22:48.440882 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=0 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset -9256> 2014-08-13 17:22:48.441054 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.639(unlocked)] enter Initial -9255> 2014-08-13 17:22:48.442127 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001073 0 0.000000 -9254> 2014-08-13 17:22:48.442148 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -9253> 2014-08-13 17:22:48.442354 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.653(unlocked)] enter Initial -9252> 2014-08-13 17:22:48.443295 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=0 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.000942 0 0.000000 -9251> 2014-08-13 17:22:48.443316 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=0 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -9250> 2014-08-13 17:22:48.443493 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.69f(unlocked)] enter Initial -9249> 2014-08-13 17:22:48.444748 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=0 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Initial 0.001255 0 0.000000 -9248> 2014-08-13 17:22:48.444769 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=0 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset -9247> 2014-08-13 17:22:48.446877 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[22.6bd(unlocked)] enter Initial -9246> 2014-08-13 17:22:48.447945 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001068 0 0.000000 -9245> 2014-08-13 17:22:48.447967 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -9244> 2014-08-13 17:22:48.448146 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.707(unlocked)] enter Initial -9243> 2014-08-13 17:22:48.449077 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=0 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.000931 0 0.000000 -9242> 2014-08-13 17:22:48.449098 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=0 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset -9241> 2014-08-13 17:22:48.449245 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.775(unlocked)] enter Initial -9240> 2014-08-13 17:22:48.450216 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=0 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.000971 0 0.000000 -9239> 2014-08-13 17:22:48.450236 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=0 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset -9238> 2014-08-13 17:22:48.450383 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.791(unlocked)] enter Initial -9237> 2014-08-13 17:22:48.451354 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.000972 0 0.000000 -9236> 2014-08-13 17:22:48.451374 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=0 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -9235> 2014-08-13 17:22:48.451534 7f74ed1707a0 5 osd.4 pg_epoch: 194455 pg[24.2c(unlocked)] enter Initial -9234> 2014-08-13 17:22:48.459892 7f74ed1707a0 5 osd.4 pg_epoch: 194455 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194453/194454/192971) [20,27,4] r=2 lpr=0 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.008358 0 0.000000 -9233> 2014-08-13 17:22:48.459915 7f74ed1707a0 5 osd.4 pg_epoch: 194455 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194453/194454/192971) [20,27,4] r=2 lpr=0 pi=193125-194453/27 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Reset -9232> 2014-08-13 17:22:48.460084 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[24.34(unlocked)] enter Initial -9231> 2014-08-13 17:22:48.468043 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=0 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.007959 0 0.000000 -9230> 2014-08-13 17:22:48.468066 7f74ed1707a0 5 osd.4 pg_epoch: 194454 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=0 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset -9229> 2014-08-13 17:22:48.468244 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3d(unlocked)] enter Initial -9228> 2014-08-13 17:22:48.480450 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Initial 0.012206 0 0.000000 -9227> 2014-08-13 17:22:48.480474 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset -9226> 2014-08-13 17:22:48.480649 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.5e(unlocked)] enter Initial -9225> 2014-08-13 17:22:48.489392 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=0 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Initial 0.008743 0 0.000000 -9224> 2014-08-13 17:22:48.489414 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=0 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset -9223> 2014-08-13 17:22:48.489577 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.9b(unlocked)] enter Initial -9222> 2014-08-13 17:22:48.496913 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.007337 0 0.000000 -9221> 2014-08-13 17:22:48.496936 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -9220> 2014-08-13 17:22:48.497088 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.b6(unlocked)] enter Initial -9219> 2014-08-13 17:22:48.504301 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.007213 0 0.000000 -9218> 2014-08-13 17:22:48.504324 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -9217> 2014-08-13 17:22:48.504474 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.cd(unlocked)] enter Initial -9216> 2014-08-13 17:22:48.516442 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=0 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Initial 0.011968 0 0.000000 -9215> 2014-08-13 17:22:48.516465 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=0 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset -9214> 2014-08-13 17:22:48.516646 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.e2(unlocked)] enter Initial -9213> 2014-08-13 17:22:48.525316 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Initial 0.008671 0 0.000000 -9212> 2014-08-13 17:22:48.525339 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset -9211> 2014-08-13 17:22:48.525506 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.136(unlocked)] enter Initial -9210> 2014-08-13 17:22:48.533692 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Initial 0.008186 0 0.000000 -9209> 2014-08-13 17:22:48.533715 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset -9208> 2014-08-13 17:22:48.533867 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.181(unlocked)] enter Initial -9207> 2014-08-13 17:22:48.541601 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Initial 0.007735 0 0.000000 -9206> 2014-08-13 17:22:48.541624 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset -9205> 2014-08-13 17:22:48.541817 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1bb(unlocked)] enter Initial -9204> 2014-08-13 17:22:48.550896 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Initial 0.009079 0 0.000000 -9203> 2014-08-13 17:22:48.550920 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset -9202> 2014-08-13 17:22:48.551083 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1de(unlocked)] enter Initial -9201> 2014-08-13 17:22:48.558605 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Initial 0.007522 0 0.000000 -9200> 2014-08-13 17:22:48.558628 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset -9199> 2014-08-13 17:22:48.558799 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e4(unlocked)] enter Initial -9198> 2014-08-13 17:22:48.566264 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=0 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Initial 0.007465 0 0.000000 -9197> 2014-08-13 17:22:48.566287 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=0 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset -9196> 2014-08-13 17:22:48.566438 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e6(unlocked)] enter Initial -9195> 2014-08-13 17:22:48.573659 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=0 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.007222 0 0.000000 -9194> 2014-08-13 17:22:48.573699 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=0 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset -9193> 2014-08-13 17:22:48.573851 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1f7(unlocked)] enter Initial -9192> 2014-08-13 17:22:48.582789 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Initial 0.008938 0 0.000000 -9191> 2014-08-13 17:22:48.582812 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset -9190> 2014-08-13 17:22:48.582984 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.206(unlocked)] enter Initial -9189> 2014-08-13 17:22:48.590560 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=0 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Initial 0.007577 0 0.000000 -9188> 2014-08-13 17:22:48.590583 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=0 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset -9187> 2014-08-13 17:22:48.590732 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.20e(unlocked)] enter Initial -9186> 2014-08-13 17:22:48.602972 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.012240 0 0.000000 -9185> 2014-08-13 17:22:48.602996 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset -9184> 2014-08-13 17:22:48.603176 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2a1(unlocked)] enter Initial -9183> 2014-08-13 17:22:48.611279 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=0 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.008103 0 0.000000 -9182> 2014-08-13 17:22:48.611301 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=0 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset -9181> 2014-08-13 17:22:48.611463 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2ec(unlocked)] enter Initial -9180> 2014-08-13 17:22:48.619402 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Initial 0.007939 0 0.000000 -9179> 2014-08-13 17:22:48.619425 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset -9178> 2014-08-13 17:22:48.619584 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.30b(unlocked)] enter Initial -9177> 2014-08-13 17:22:48.631695 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Initial 0.012112 0 0.000000 -9176> 2014-08-13 17:22:48.631718 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset -9175> 2014-08-13 17:22:48.631870 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.33a(unlocked)] enter Initial -9174> 2014-08-13 17:22:48.639740 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=0 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Initial 0.007871 0 0.000000 -9173> 2014-08-13 17:22:48.639762 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=0 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset -9172> 2014-08-13 17:22:48.639913 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.350(unlocked)] enter Initial -9171> 2014-08-13 17:22:48.646919 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=0 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Initial 0.007007 0 0.000000 -9170> 2014-08-13 17:22:48.646942 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=0 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset -9169> 2014-08-13 17:22:48.647104 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.369(unlocked)] enter Initial -9168> 2014-08-13 17:22:48.655025 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=0 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.007921 0 0.000000 -9167> 2014-08-13 17:22:48.655048 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=0 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset -9166> 2014-08-13 17:22:48.657100 7f74ed1707a0 5 osd.4 pg_epoch: 194453 pg[24.39a(unlocked)] enter Initial -9165> 2014-08-13 17:22:48.664559 7f74ed1707a0 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=0 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] exit Initial 0.007459 0 0.000000 -9164> 2014-08-13 17:22:48.664583 7f74ed1707a0 5 osd.4 pg_epoch: 194453 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=0 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Reset -9163> 2014-08-13 17:22:48.664737 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3b7(unlocked)] enter Initial -9162> 2014-08-13 17:22:48.672728 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.007990 0 0.000000 -9161> 2014-08-13 17:22:48.672780 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset -9160> 2014-08-13 17:22:48.672937 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3f7(unlocked)] enter Initial -9159> 2014-08-13 17:22:48.681363 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Initial 0.008426 0 0.000000 -9158> 2014-08-13 17:22:48.681386 7f74ed1707a0 5 osd.4 pg_epoch: 194449 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset -9157> 2014-08-13 17:22:48.681398 7f74ed1707a0 0 osd.4 194455 load_pgs opened 72 pgs -9156> 2014-08-13 17:22:48.681442 7f74ed1707a0 2 osd.4 194455 superblock: i am osd.4 -9155> 2014-08-13 17:22:48.681518 7f74ed1707a0 1 accepter.accepter.start -9154> 2014-08-13 17:22:48.681572 7f74ed1707a0 1 accepter.accepter.start -9153> 2014-08-13 17:22:48.681632 7f74ed1707a0 1 accepter.accepter.start -9152> 2014-08-13 17:22:48.681669 7f74ed1707a0 1 accepter.accepter.start -9151> 2014-08-13 17:22:48.681717 7f74ed1707a0 1 accepter.accepter.start -9150> 2014-08-13 17:22:48.681740 7f74ed1707a0 10 monclient(hunting): init -9149> 2014-08-13 17:22:48.681749 7f74ed1707a0 5 adding auth protocol: cephx -9148> 2014-08-13 17:22:48.681752 7f74ed1707a0 10 monclient(hunting): auth_supported 2 method cephx -9147> 2014-08-13 17:22:48.681916 7f74ed1707a0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-4/keyring -9146> 2014-08-13 17:22:48.682291 7f74ed1707a0 5 asok(0x2700230) register_command objecter_requests hook 0x4290f90 -9145> 2014-08-13 17:22:48.682295 7f74ce725700 5 osd.4 194455 heartbeat: osd_stat(8082 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) -9144> 2014-08-13 17:22:48.682338 7f74ed1707a0 10 monclient(hunting): renew_subs -9143> 2014-08-13 17:22:48.682346 7f74ed1707a0 10 monclient(hunting): _reopen_session rank -1 name -9142> 2014-08-13 17:22:48.682428 7f74ed1707a0 10 monclient(hunting): picked mon.storage0113-ib con 0x45d48e0 addr 192.168.100.113:6789/0 -9141> 2014-08-13 17:22:48.682446 7f74ed1707a0 10 monclient(hunting): _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9140> 2014-08-13 17:22:48.682450 7f74ed1707a0 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 0x2710d80 con 0x45d48e0 -9139> 2014-08-13 17:22:48.682458 7f74ed1707a0 10 monclient(hunting): renew_subs -9138> 2014-08-13 17:22:48.683292 7f74d9b37700 10 monclient(hunting): renew_subs -9137> 2014-08-13 17:22:48.683915 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 1 ==== mon_map v1 ==== 512+0+0 (3258631466 0 0) 0x29c01e0 con 0x45d48e0 -9136> 2014-08-13 17:22:48.683963 7f74d9b37700 10 monclient(hunting): handle_monmap mon_map v1 -9135> 2014-08-13 17:22:48.683980 7f74d9b37700 10 monclient(hunting): got monmap 3, mon.storage0113-ib is now rank 2 -9134> 2014-08-13 17:22:48.683985 7f74d9b37700 10 monclient(hunting): dump: epoch 3 fsid 009d3518-e60d-4f74-a26d-c08c1976263c last_changed 2014-06-13 16:08:03.914184 created 2014-06-13 16:07:42.361285 0: 192.168.100.111:6789/0 mon.storage0111-ib 1: 192.168.100.112:6789/0 mon.storage0112-ib 2: 192.168.100.113:6789/0 mon.storage0113-ib -9133> 2014-08-13 17:22:48.684040 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1945782250 0 0) 0x272c400 con 0x45d48e0 -9132> 2014-08-13 17:22:48.684081 7f74d9b37700 10 monclient(hunting): my global_id is 1656707 -9131> 2014-08-13 17:22:48.684415 7f74d9b37700 10 monclient(hunting): _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9130> 2014-08-13 17:22:48.684436 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x2712400 con 0x45d48e0 -9129> 2014-08-13 17:22:48.685691 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (2194589536 0 0) 0x272c200 con 0x45d48e0 -9128> 2014-08-13 17:22:48.685830 7f74d9b37700 10 monclient(hunting): _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9127> 2014-08-13 17:22:48.685848 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x2712880 con 0x45d48e0 -9126> 2014-08-13 17:22:48.686731 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 393+0+0 (2144924215 0 0) 0x272c600 con 0x45d48e0 -9125> 2014-08-13 17:22:48.686860 7f74d9b37700 1 monclient(hunting): found mon.storage0113-ib -9124> 2014-08-13 17:22:48.686872 7f74d9b37700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9123> 2014-08-13 17:22:48.686883 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x33fa300 con 0x45d48e0 -9122> 2014-08-13 17:22:48.686919 7f74d9b37700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9121> 2014-08-13 17:22:48.686928 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- mon_subscribe({monmap=0+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x33fb100 con 0x45d48e0 -9120> 2014-08-13 17:22:48.686995 7f74d9b37700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-13 17:22:18.686994) -9119> 2014-08-13 17:22:48.687010 7f74d9b37700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -9118> 2014-08-13 17:22:48.687031 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- ?+0 0x2712640 con 0x45d48e0 -9117> 2014-08-13 17:22:48.687072 7f74ed1707a0 5 monclient: authenticate success, global_id 1656707 -9116> 2014-08-13 17:22:48.687108 7f74ed1707a0 10 monclient: wait_auth_rotating waiting (until 2014-08-13 17:23:18.687107) -9115> 2014-08-13 17:22:48.689669 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 5 ==== osd_map(194469..194469 src has 185139..194469) v3 ==== 268298+0+0 (2450856240 0 0) 0x2710d80 con 0x45d48e0 -9114> 2014-08-13 17:22:48.689704 7f74d9b37700 0 osd.4 194455 ignoring osdmap until we have initialized -9113> 2014-08-13 17:22:48.689714 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 6 ==== mon_map v1 ==== 512+0+0 (3258631466 0 0) 0x29c05a0 con 0x45d48e0 -9112> 2014-08-13 17:22:48.689736 7f74d9b37700 10 monclient: handle_monmap mon_map v1 -9111> 2014-08-13 17:22:48.689754 7f74d9b37700 10 monclient: got monmap 3, mon.storage0113-ib is now rank 2 -9110> 2014-08-13 17:22:48.689759 7f74d9b37700 10 monclient: dump: epoch 3 fsid 009d3518-e60d-4f74-a26d-c08c1976263c last_changed 2014-06-13 16:08:03.914184 created 2014-06-13 16:07:42.361285 0: 192.168.100.111:6789/0 mon.storage0111-ib 1: 192.168.100.112:6789/0 mon.storage0112-ib 2: 192.168.100.113:6789/0 mon.storage0113-ib -9109> 2014-08-13 17:22:48.690644 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 7 ==== osd_map(194469..194469 src has 185139..194469) v3 ==== 268298+0+0 (2450856240 0 0) 0x2712ac0 con 0x45d48e0 -9108> 2014-08-13 17:22:48.690691 7f74d9b37700 0 osd.4 194455 ignoring osdmap until we have initialized -9107> 2014-08-13 17:22:48.690704 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 8 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x33fa300 con 0x45d48e0 -9106> 2014-08-13 17:22:48.690728 7f74d9b37700 10 monclient: handle_subscribe_ack sent 2014-08-13 17:22:48.682460 renew after 2014-08-13 17:25:18.682460 -9105> 2014-08-13 17:22:48.690749 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 9 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (257085135 0 0) 0x272c000 con 0x45d48e0 -9104> 2014-08-13 17:22:48.690844 7f74d9b37700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 17:22:18.690843) -9103> 2014-08-13 17:22:48.690884 7f74ed1707a0 10 monclient: wait_auth_rotating done -9102> 2014-08-13 17:22:48.700103 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Reset 0.068385 7 0.000080 -9101> 2014-08-13 17:22:48.700105 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.293676 7 0.000082 -9100> 2014-08-13 17:22:48.700166 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started -9099> 2014-08-13 17:22:48.700181 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started -9098> 2014-08-13 17:22:48.700194 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Start -9097> 2014-08-13 17:22:48.700207 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start -9096> 2014-08-13 17:22:48.700219 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: transitioning to Primary -9095> 2014-08-13 17:22:48.700232 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9094> 2014-08-13 17:22:48.700253 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Start 0.000059 0 0.000000 -9093> 2014-08-13 17:22:48.700257 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000050 0 0.000000 -9092> 2014-08-13 17:22:48.700289 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9091> 2014-08-13 17:22:48.700287 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary -9090> 2014-08-13 17:22:48.700318 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9089> 2014-08-13 17:22:48.700325 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering -9088> 2014-08-13 17:22:48.700343 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9087> 2014-08-13 17:22:48.700347 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetInfo -9086> 2014-08-13 17:22:48.700915 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.305806 7 0.000074 -9085> 2014-08-13 17:22:48.700888 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.313329 7 0.000069 -9084> 2014-08-13 17:22:48.700973 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -9083> 2014-08-13 17:22:48.701013 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -9082> 2014-08-13 17:22:48.701046 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -9081> 2014-08-13 17:22:48.701092 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9080> 2014-08-13 17:22:48.701072 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -9079> 2014-08-13 17:22:48.701120 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9078> 2014-08-13 17:22:48.701122 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000077 0 0.000000 -9077> 2014-08-13 17:22:48.701161 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000089 0 0.000000 -9076> 2014-08-13 17:22:48.701178 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -9075> 2014-08-13 17:22:48.701196 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -9074> 2014-08-13 17:22:48.701932 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.328516 7 0.000071 -9073> 2014-08-13 17:22:48.701940 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.342825 7 0.000084 -9072> 2014-08-13 17:22:48.701961 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -9071> 2014-08-13 17:22:48.701973 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -9070> 2014-08-13 17:22:48.701995 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9069> 2014-08-13 17:22:48.702013 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -9068> 2014-08-13 17:22:48.702035 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -9067> 2014-08-13 17:22:48.701961 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started -9066> 2014-08-13 17:22:48.702288 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start -9065> 2014-08-13 17:22:48.702299 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9064> 2014-08-13 17:22:48.702314 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000025 0 0.000000 -9063> 2014-08-13 17:22:48.702344 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9062> 2014-08-13 17:22:48.702363 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9061> 2014-08-13 17:22:48.702382 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9060> 2014-08-13 17:22:48.702508 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.309779 7 0.000075 -9059> 2014-08-13 17:22:48.702531 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started -9058> 2014-08-13 17:22:48.702543 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start -9057> 2014-08-13 17:22:48.702565 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9056> 2014-08-13 17:22:48.702584 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000040 0 0.000000 -9055> 2014-08-13 17:22:48.702608 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9054> 2014-08-13 17:22:48.702601 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.271089 7 0.000084 -9053> 2014-08-13 17:22:48.702639 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9052> 2014-08-13 17:22:48.702640 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started -9051> 2014-08-13 17:22:48.702655 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9050> 2014-08-13 17:22:48.702661 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start -9049> 2014-08-13 17:22:48.702673 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9048> 2014-08-13 17:22:48.702685 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 -9047> 2014-08-13 17:22:48.702698 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9046> 2014-08-13 17:22:48.702729 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9045> 2014-08-13 17:22:48.702752 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9044> 2014-08-13 17:22:48.702943 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.263297 7 0.000039 -9043> 2014-08-13 17:22:48.702978 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started -9042> 2014-08-13 17:22:48.702997 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start -9041> 2014-08-13 17:22:48.703014 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9040> 2014-08-13 17:22:48.703035 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000038 0 0.000000 -9039> 2014-08-13 17:22:48.703062 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9038> 2014-08-13 17:22:48.703073 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9037> 2014-08-13 17:22:48.703085 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9036> 2014-08-13 17:22:48.703079 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.294544 7 0.000067 -9035> 2014-08-13 17:22:48.703107 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started -9034> 2014-08-13 17:22:48.703120 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start -9033> 2014-08-13 17:22:48.703132 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -9032> 2014-08-13 17:22:48.703180 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000060 0 0.000000 -9031> 2014-08-13 17:22:48.703193 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -9030> 2014-08-13 17:22:48.703206 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -9029> 2014-08-13 17:22:48.703219 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -9028> 2014-08-13 17:22:48.703415 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.299114 7 0.000067 -9027> 2014-08-13 17:22:48.703466 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started -9026> 2014-08-13 17:22:48.703488 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start -9025> 2014-08-13 17:22:48.703515 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9024> 2014-08-13 17:22:48.703527 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 -9023> 2014-08-13 17:22:48.703540 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray -9022> 2014-08-13 17:22:48.703534 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.303941 7 0.000066 -9021> 2014-08-13 17:22:48.703578 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started -9020> 2014-08-13 17:22:48.703600 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start -9019> 2014-08-13 17:22:48.703618 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9018> 2014-08-13 17:22:48.703646 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000045 0 0.000000 -9017> 2014-08-13 17:22:48.703660 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray -9016> 2014-08-13 17:22:48.703852 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.206916 7 0.000043 -9015> 2014-08-13 17:22:48.703877 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started -9014> 2014-08-13 17:22:48.703894 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Start -9013> 2014-08-13 17:22:48.703905 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary -9012> 2014-08-13 17:22:48.703918 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000024 0 0.000000 -9011> 2014-08-13 17:22:48.703946 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -9010> 2014-08-13 17:22:48.703971 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering -9009> 2014-08-13 17:22:48.703996 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo -9008> 2014-08-13 17:22:48.704298 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Reset 0.022912 7 0.000065 -9007> 2014-08-13 17:22:48.704308 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.266006 7 0.000048 -9006> 2014-08-13 17:22:48.704330 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started -9005> 2014-08-13 17:22:48.704335 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started -9004> 2014-08-13 17:22:48.704345 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start -9003> 2014-08-13 17:22:48.704345 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Start -9002> 2014-08-13 17:22:48.704359 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -9001> 2014-08-13 17:22:48.704364 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: transitioning to Stray -9000> 2014-08-13 17:22:48.704373 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000028 0 0.000000 -8999> 2014-08-13 17:22:48.704379 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Start 0.000035 0 0.000000 -8998> 2014-08-13 17:22:48.704387 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray -8997> 2014-08-13 17:22:48.704394 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started/Stray -8996> 2014-08-13 17:22:48.704575 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.340676 7 0.000066 -8995> 2014-08-13 17:22:48.704595 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started -8994> 2014-08-13 17:22:48.704608 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start -8993> 2014-08-13 17:22:48.704618 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8992> 2014-08-13 17:22:48.704620 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.254384 7 0.000038 -8991> 2014-08-13 17:22:48.704631 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000023 0 0.000000 -8990> 2014-08-13 17:22:48.704644 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started -8989> 2014-08-13 17:22:48.704658 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start -8988> 2014-08-13 17:22:48.704645 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8987> 2014-08-13 17:22:48.704672 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8986> 2014-08-13 17:22:48.704676 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8985> 2014-08-13 17:22:48.704685 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000026 0 0.000000 -8984> 2014-08-13 17:22:48.704689 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8983> 2014-08-13 17:22:48.704701 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8982> 2014-08-13 17:22:48.704716 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8981> 2014-08-13 17:22:48.704728 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8980> 2014-08-13 17:22:48.704893 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.336545 7 0.000064 -8979> 2014-08-13 17:22:48.704914 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started -8978> 2014-08-13 17:22:48.704925 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start -8977> 2014-08-13 17:22:48.704923 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.264041 7 0.000041 -8976> 2014-08-13 17:22:48.704935 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8975> 2014-08-13 17:22:48.704944 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started -8974> 2014-08-13 17:22:48.704950 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000 -8973> 2014-08-13 17:22:48.704957 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start -8972> 2014-08-13 17:22:48.704966 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8971> 2014-08-13 17:22:48.704965 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray -8970> 2014-08-13 17:22:48.704984 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000028 0 0.000000 -8969> 2014-08-13 17:22:48.704995 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray -8968> 2014-08-13 17:22:48.705144 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Reset 0.224670 7 0.000045 -8967> 2014-08-13 17:22:48.705165 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started -8966> 2014-08-13 17:22:48.705180 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Start -8965> 2014-08-13 17:22:48.705195 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: transitioning to Primary -8964> 2014-08-13 17:22:48.705210 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Start 0.000030 0 0.000000 -8963> 2014-08-13 17:22:48.705223 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary -8962> 2014-08-13 17:22:48.705235 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering -8961> 2014-08-13 17:22:48.705248 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetInfo -8960> 2014-08-13 17:22:48.705345 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Reset 0.180006 7 0.000045 -8959> 2014-08-13 17:22:48.705374 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started -8958> 2014-08-13 17:22:48.705387 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Start -8957> 2014-08-13 17:22:48.705418 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: transitioning to Stray -8956> 2014-08-13 17:22:48.705442 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Start 0.000055 0 0.000000 -8955> 2014-08-13 17:22:48.705472 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started/Stray -8954> 2014-08-13 17:22:48.705666 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.276480 7 0.000092 -8953> 2014-08-13 17:22:48.705687 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started -8952> 2014-08-13 17:22:48.705701 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start -8951> 2014-08-13 17:22:48.705714 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8950> 2014-08-13 17:22:48.705727 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000026 0 0.000000 -8949> 2014-08-13 17:22:48.705738 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray -8948> 2014-08-13 17:22:48.705978 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.326037 7 0.000069 -8947> 2014-08-13 17:22:48.706019 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started -8946> 2014-08-13 17:22:48.706040 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start -8945> 2014-08-13 17:22:48.706029 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.261260 7 0.000042 -8944> 2014-08-13 17:22:48.706064 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8943> 2014-08-13 17:22:48.706099 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started -8942> 2014-08-13 17:22:48.706105 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000065 0 0.000000 -8941> 2014-08-13 17:22:48.706112 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start -8940> 2014-08-13 17:22:48.706121 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8939> 2014-08-13 17:22:48.706121 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8938> 2014-08-13 17:22:48.706136 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8937> 2014-08-13 17:22:48.706137 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8936> 2014-08-13 17:22:48.706170 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray -8935> 2014-08-13 17:22:48.706174 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8934> 2014-08-13 17:22:48.706417 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.283988 7 0.000066 -8933> 2014-08-13 17:22:48.706457 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -8932> 2014-08-13 17:22:48.706478 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -8931> 2014-08-13 17:22:48.706496 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8930> 2014-08-13 17:22:48.706529 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000052 0 0.000000 -8929> 2014-08-13 17:22:48.706542 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8928> 2014-08-13 17:22:48.706541 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Reset 0.147913 7 0.000046 -8927> 2014-08-13 17:22:48.706556 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8926> 2014-08-13 17:22:48.706562 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started -8925> 2014-08-13 17:22:48.706567 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8924> 2014-08-13 17:22:48.706572 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Start -8923> 2014-08-13 17:22:48.706588 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: transitioning to Stray -8922> 2014-08-13 17:22:48.706615 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Start 0.000043 0 0.000000 -8921> 2014-08-13 17:22:48.706639 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started/Stray -8920> 2014-08-13 17:22:48.706883 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.095581 7 0.000046 -8919> 2014-08-13 17:22:48.706924 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started -8918> 2014-08-13 17:22:48.706924 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.350166 7 0.000084 -8917> 2014-08-13 17:22:48.706936 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start -8916> 2014-08-13 17:22:48.706949 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray -8915> 2014-08-13 17:22:48.706952 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8914> 2014-08-13 17:22:48.706962 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8913> 2014-08-13 17:22:48.706962 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000026 0 0.000000 -8912> 2014-08-13 17:22:48.706973 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8911> 2014-08-13 17:22:48.706974 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray -8910> 2014-08-13 17:22:48.706985 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000023 0 0.000000 -8909> 2014-08-13 17:22:48.706998 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8908> 2014-08-13 17:22:48.707009 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8907> 2014-08-13 17:22:48.707022 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8906> 2014-08-13 17:22:48.707136 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.264988 7 0.000048 -8905> 2014-08-13 17:22:48.707155 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -8904> 2014-08-13 17:22:48.707167 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -8903> 2014-08-13 17:22:48.707177 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8902> 2014-08-13 17:22:48.707189 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 -8901> 2014-08-13 17:22:48.707208 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -8900> 2014-08-13 17:22:48.707241 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.341115 7 0.000061 -8899> 2014-08-13 17:22:48.707265 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started -8898> 2014-08-13 17:22:48.707277 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start -8897> 2014-08-13 17:22:48.707286 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8896> 2014-08-13 17:22:48.707296 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 -8895> 2014-08-13 17:22:48.707306 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8894> 2014-08-13 17:22:48.707315 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8893> 2014-08-13 17:22:48.707323 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8892> 2014-08-13 17:22:48.707380 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.258283 7 0.000039 -8891> 2014-08-13 17:22:48.707407 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started -8890> 2014-08-13 17:22:48.707419 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start -8889> 2014-08-13 17:22:48.707429 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8888> 2014-08-13 17:22:48.707441 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000022 0 0.000000 -8887> 2014-08-13 17:22:48.707447 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] exit Reset 0.042865 3 0.000044 -8886> 2014-08-13 17:22:48.707456 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8885> 2014-08-13 17:22:48.707471 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Started -8884> 2014-08-13 17:22:48.707475 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8883> 2014-08-13 17:22:48.707484 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Start -8882> 2014-08-13 17:22:48.707485 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8881> 2014-08-13 17:22:48.707498 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] state: transitioning to Primary -8880> 2014-08-13 17:22:48.707511 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] exit Start 0.000027 0 0.000000 -8879> 2014-08-13 17:22:48.707527 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Started/Primary -8878> 2014-08-13 17:22:48.707542 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Started/Primary/Peering -8877> 2014-08-13 17:22:48.707555 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 peering m=49] enter Started/Primary/Peering/GetInfo -8876> 2014-08-13 17:22:48.707711 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.218297 7 0.000045 -8875> 2014-08-13 17:22:48.707733 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started -8874> 2014-08-13 17:22:48.707765 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start -8873> 2014-08-13 17:22:48.707784 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray -8872> 2014-08-13 17:22:48.707788 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.068025 7 0.000056 -8871> 2014-08-13 17:22:48.707815 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000050 0 0.000000 -8870> 2014-08-13 17:22:48.707831 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started -8869> 2014-08-13 17:22:48.707832 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray -8868> 2014-08-13 17:22:48.707847 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start -8867> 2014-08-13 17:22:48.707857 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray -8866> 2014-08-13 17:22:48.707869 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000023 0 0.000000 -8865> 2014-08-13 17:22:48.707885 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray -8864> 2014-08-13 17:22:48.707946 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.325530 7 0.000063 -8863> 2014-08-13 17:22:48.707967 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started -8862> 2014-08-13 17:22:48.707979 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start -8861> 2014-08-13 17:22:48.707996 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8860> 2014-08-13 17:22:48.708009 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000030 0 0.000000 -8859> 2014-08-13 17:22:48.708022 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8858> 2014-08-13 17:22:48.708033 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8857> 2014-08-13 17:22:48.708029 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.365130 7 0.000115 -8856> 2014-08-13 17:22:48.708045 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8855> 2014-08-13 17:22:48.708050 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8854> 2014-08-13 17:22:48.708078 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8853> 2014-08-13 17:22:48.708095 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8852> 2014-08-13 17:22:48.708118 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 -8851> 2014-08-13 17:22:48.708138 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8850> 2014-08-13 17:22:48.708162 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8849> 2014-08-13 17:22:48.708180 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8848> 2014-08-13 17:22:48.708333 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.240268 2 0.000039 -8847> 2014-08-13 17:22:48.708360 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started -8846> 2014-08-13 17:22:48.708360 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.306174 7 0.000075 -8845> 2014-08-13 17:22:48.708370 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Start -8844> 2014-08-13 17:22:48.708384 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started -8843> 2014-08-13 17:22:48.708398 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start -8842> 2014-08-13 17:22:48.708391 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8841> 2014-08-13 17:22:48.708412 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8840> 2014-08-13 17:22:48.708420 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000050 0 0.000000 -8839> 2014-08-13 17:22:48.708426 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000027 0 0.000000 -8838> 2014-08-13 17:22:48.708436 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8837> 2014-08-13 17:22:48.708437 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray -8836> 2014-08-13 17:22:48.708454 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8835> 2014-08-13 17:22:48.708467 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8834> 2014-08-13 17:22:48.708671 7f74c5bd9700 2 -- 192.168.1.115:6813/30165 >> 192.168.1.107:6802/9548 pipe(0x2ef5780 sd=91 :0 s=1 pgs=0 cs=0 l=0 c=0x47f5120).connect error 192.168.1.107:6802/9548, (111) Connection refused -8833> 2014-08-13 17:22:48.708754 7f74c5bd9700 2 -- 192.168.1.115:6813/30165 >> 192.168.1.107:6802/9548 pipe(0x2ef5780 sd=91 :0 s=1 pgs=0 cs=0 l=0 c=0x47f5120).fault (111) Connection refused -8832> 2014-08-13 17:22:48.708771 7f74c5bd9700 0 -- 192.168.1.115:6813/30165 >> 192.168.1.107:6802/9548 pipe(0x2ef5780 sd=91 :0 s=1 pgs=0 cs=0 l=0 c=0x47f5120).fault with nothing to send, going to standby -8831> 2014-08-13 17:22:48.708761 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.204438 7 0.000047 -8830> 2014-08-13 17:22:48.708778 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.347446 7 0.000069 -8829> 2014-08-13 17:22:48.708787 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started -8828> 2014-08-13 17:22:48.708797 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started -8827> 2014-08-13 17:22:48.708798 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Start -8826> 2014-08-13 17:22:48.708806 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start -8825> 2014-08-13 17:22:48.708808 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary -8824> 2014-08-13 17:22:48.708818 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8823> 2014-08-13 17:22:48.708819 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000021 0 0.000000 -8822> 2014-08-13 17:22:48.708846 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000040 0 0.000000 -8821> 2014-08-13 17:22:48.708848 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -8820> 2014-08-13 17:22:48.708873 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8819> 2014-08-13 17:22:48.708881 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering -8818> 2014-08-13 17:22:48.708895 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8817> 2014-08-13 17:22:48.708904 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo -8816> 2014-08-13 17:22:48.708927 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8815> 2014-08-13 17:22:48.709304 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Reset 0.126492 7 0.000047 -8814> 2014-08-13 17:22:48.709310 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.324234 7 0.000081 -8813> 2014-08-13 17:22:48.709326 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started -8812> 2014-08-13 17:22:48.709328 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -8811> 2014-08-13 17:22:48.709354 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -8810> 2014-08-13 17:22:48.709352 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Start -8809> 2014-08-13 17:22:48.709369 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8808> 2014-08-13 17:22:48.709370 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: transitioning to Primary -8807> 2014-08-13 17:22:48.709397 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000043 0 0.000000 -8806> 2014-08-13 17:22:48.709401 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Start 0.000049 0 0.000000 -8805> 2014-08-13 17:22:48.709422 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -8804> 2014-08-13 17:22:48.709423 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary -8803> 2014-08-13 17:22:48.709442 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering -8802> 2014-08-13 17:22:48.709464 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetInfo -8801> 2014-08-13 17:22:48.709748 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.062805 7 0.000045 -8800> 2014-08-13 17:22:48.709752 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Reset 0.176037 7 0.000048 -8799> 2014-08-13 17:22:48.709784 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started -8798> 2014-08-13 17:22:48.709785 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started -8797> 2014-08-13 17:22:48.709809 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start -8796> 2014-08-13 17:22:48.709829 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Start -8795> 2014-08-13 17:22:48.709834 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray -8794> 2014-08-13 17:22:48.709842 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: transitioning to Primary -8793> 2014-08-13 17:22:48.709847 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000039 0 0.000000 -8792> 2014-08-13 17:22:48.709855 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Start 0.000026 0 0.000000 -8791> 2014-08-13 17:22:48.709861 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray -8790> 2014-08-13 17:22:48.709870 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary -8789> 2014-08-13 17:22:48.709901 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering -8788> 2014-08-13 17:22:48.709925 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetInfo -8787> 2014-08-13 17:22:48.710214 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.274169 7 0.000073 -8786> 2014-08-13 17:22:48.710254 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -8785> 2014-08-13 17:22:48.710265 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -8784> 2014-08-13 17:22:48.710265 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.276490 7 0.000080 -8783> 2014-08-13 17:22:48.710275 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8782> 2014-08-13 17:22:48.710287 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -8781> 2014-08-13 17:22:48.710288 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8780> 2014-08-13 17:22:48.710324 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -8779> 2014-08-13 17:22:48.710333 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -8778> 2014-08-13 17:22:48.710345 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8777> 2014-08-13 17:22:48.710374 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000051 0 0.000000 -8776> 2014-08-13 17:22:48.710414 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -8775> 2014-08-13 17:22:48.710797 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.333005 7 0.000071 -8774> 2014-08-13 17:22:48.710830 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started -8773> 2014-08-13 17:22:48.710850 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start -8772> 2014-08-13 17:22:48.710867 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8771> 2014-08-13 17:22:48.710913 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000063 0 0.000000 -8770> 2014-08-13 17:22:48.710925 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8769> 2014-08-13 17:22:48.710935 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8768> 2014-08-13 17:22:48.710946 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8767> 2014-08-13 17:22:48.711018 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.267703 7 0.000065 -8766> 2014-08-13 17:22:48.711106 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -8765> 2014-08-13 17:22:48.711133 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -8764> 2014-08-13 17:22:48.711153 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8763> 2014-08-13 17:22:48.711234 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.314118 7 0.000067 -8762> 2014-08-13 17:22:48.711267 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -8761> 2014-08-13 17:22:48.711268 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000134 0 0.000000 -8760> 2014-08-13 17:22:48.711278 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -8759> 2014-08-13 17:22:48.711301 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8758> 2014-08-13 17:22:48.711302 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -8757> 2014-08-13 17:22:48.711319 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -8756> 2014-08-13 17:22:48.711331 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -8755> 2014-08-13 17:22:48.711769 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.195304 7 0.000051 -8754> 2014-08-13 17:22:48.711804 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started -8753> 2014-08-13 17:22:48.711815 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start -8752> 2014-08-13 17:22:48.711826 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray -8751> 2014-08-13 17:22:48.711851 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000036 0 0.000000 -8750> 2014-08-13 17:22:48.711873 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray -8749> 2014-08-13 17:22:48.711928 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.039149 7 0.000105 -8748> 2014-08-13 17:22:48.711951 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started -8747> 2014-08-13 17:22:48.711975 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Start -8746> 2014-08-13 17:22:48.712006 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -8745> 2014-08-13 17:22:48.712058 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000084 0 0.000000 -8744> 2014-08-13 17:22:48.712074 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -8743> 2014-08-13 17:22:48.712145 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.322045 7 0.000090 -8742> 2014-08-13 17:22:48.712180 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -8741> 2014-08-13 17:22:48.712214 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -8740> 2014-08-13 17:22:48.712235 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8739> 2014-08-13 17:22:48.712257 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000043 0 0.000000 -8738> 2014-08-13 17:22:48.712278 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -8737> 2014-08-13 17:22:48.712532 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.292548 7 0.000067 -8736> 2014-08-13 17:22:48.712554 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started -8735> 2014-08-13 17:22:48.712566 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start -8734> 2014-08-13 17:22:48.712576 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8733> 2014-08-13 17:22:48.712588 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000022 0 0.000000 -8732> 2014-08-13 17:22:48.712613 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8731> 2014-08-13 17:22:48.712634 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8730> 2014-08-13 17:22:48.712654 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8729> 2014-08-13 17:22:48.712618 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.296836 7 0.000067 -8728> 2014-08-13 17:22:48.712685 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -8727> 2014-08-13 17:22:48.712696 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -8726> 2014-08-13 17:22:48.712708 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8725> 2014-08-13 17:22:48.712742 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000045 0 0.000000 -8724> 2014-08-13 17:22:48.712756 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -8723> 2014-08-13 17:22:48.712876 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Reset 0.093451 7 0.000043 -8722> 2014-08-13 17:22:48.712917 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started -8721> 2014-08-13 17:22:48.712939 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Start -8720> 2014-08-13 17:22:48.712958 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: transitioning to Stray -8719> 2014-08-13 17:22:48.712986 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Start 0.000048 0 0.000000 -8718> 2014-08-13 17:22:48.713000 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started/Stray -8717> 2014-08-13 17:22:48.713307 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.337669 7 0.000065 -8716> 2014-08-13 17:22:48.713281 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.261907 7 0.000044 -8715> 2014-08-13 17:22:48.713329 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -8714> 2014-08-13 17:22:48.713340 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -8713> 2014-08-13 17:22:48.713331 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -8712> 2014-08-13 17:22:48.713350 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8711> 2014-08-13 17:22:48.713362 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 -8710> 2014-08-13 17:22:48.713375 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -8709> 2014-08-13 17:22:48.713360 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -8708> 2014-08-13 17:22:48.713413 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8707> 2014-08-13 17:22:48.713458 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000098 0 0.000000 -8706> 2014-08-13 17:22:48.713474 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -8705> 2014-08-13 17:22:48.713661 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.058614 7 0.000045 -8704> 2014-08-13 17:22:48.713684 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started -8703> 2014-08-13 17:22:48.713696 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start -8702> 2014-08-13 17:22:48.713706 7f74d0f29700 1 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -8701> 2014-08-13 17:22:48.713711 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.265744 2 0.000041 -8700> 2014-08-13 17:22:48.713719 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000024 0 0.000000 -8699> 2014-08-13 17:22:48.713737 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -8698> 2014-08-13 17:22:48.713741 7f74d0f29700 5 osd.4 pg_epoch: 194455 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -8697> 2014-08-13 17:22:48.713749 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -8696> 2014-08-13 17:22:48.713800 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8695> 2014-08-13 17:22:48.713827 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000078 0 0.000000 -8694> 2014-08-13 17:22:48.713858 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8693> 2014-08-13 17:22:48.713879 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8692> 2014-08-13 17:22:48.713924 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8691> 2014-08-13 17:22:48.715666 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Reset 0.174041 7 0.000098 -8690> 2014-08-13 17:22:48.715692 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started -8689> 2014-08-13 17:22:48.715710 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Start -8688> 2014-08-13 17:22:48.715739 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: transitioning to Primary -8687> 2014-08-13 17:22:48.715784 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Start 0.000073 0 0.000000 -8686> 2014-08-13 17:22:48.715808 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started/Primary -8685> 2014-08-13 17:22:48.715829 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started/Primary/Peering -8684> 2014-08-13 17:22:48.715860 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] enter Started/Primary/Peering/GetInfo -8683> 2014-08-13 17:22:48.716256 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.113261 7 0.000048 -8682> 2014-08-13 17:22:48.716305 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started -8681> 2014-08-13 17:22:48.716318 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Start -8680> 2014-08-13 17:22:48.716329 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -8679> 2014-08-13 17:22:48.716342 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000024 0 0.000000 -8678> 2014-08-13 17:22:48.716355 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -8677> 2014-08-13 17:22:48.716772 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.150485 7 0.000057 -8676> 2014-08-13 17:22:48.716794 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started -8675> 2014-08-13 17:22:48.716805 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start -8674> 2014-08-13 17:22:48.716821 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray -8673> 2014-08-13 17:22:48.716834 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000029 0 0.000000 -8672> 2014-08-13 17:22:48.716847 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray -8671> 2014-08-13 17:22:48.717229 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.143530 7 0.000065 -8670> 2014-08-13 17:22:48.717287 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started -8669> 2014-08-13 17:22:48.717319 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start -8668> 2014-08-13 17:22:48.717329 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -8667> 2014-08-13 17:22:48.717341 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000022 0 0.000000 -8666> 2014-08-13 17:22:48.717354 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -8665> 2014-08-13 17:22:48.717713 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.346833 7 0.000085 -8664> 2014-08-13 17:22:48.717754 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -8663> 2014-08-13 17:22:48.717791 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -8662> 2014-08-13 17:22:48.717807 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8661> 2014-08-13 17:22:48.717820 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 -8660> 2014-08-13 17:22:48.717864 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -8659> 2014-08-13 17:22:48.718246 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.304998 7 0.000074 -8658> 2014-08-13 17:22:48.718278 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started -8657> 2014-08-13 17:22:48.718296 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start -8656> 2014-08-13 17:22:48.718308 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8655> 2014-08-13 17:22:48.718343 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000047 0 0.000000 -8654> 2014-08-13 17:22:48.718371 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8653> 2014-08-13 17:22:48.718400 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8652> 2014-08-13 17:22:48.718420 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8651> 2014-08-13 17:22:48.718722 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.294163 7 0.000068 -8650> 2014-08-13 17:22:48.718777 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started -8649> 2014-08-13 17:22:48.718826 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start -8648> 2014-08-13 17:22:48.718850 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8647> 2014-08-13 17:22:48.718863 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -8646> 2014-08-13 17:22:48.718886 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8645> 2014-08-13 17:22:48.718897 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8644> 2014-08-13 17:22:48.718927 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8643> 2014-08-13 17:22:48.719282 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.292263 7 0.000147 -8642> 2014-08-13 17:22:48.719306 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started -8641> 2014-08-13 17:22:48.719353 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start -8640> 2014-08-13 17:22:48.719383 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8639> 2014-08-13 17:22:48.719394 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000041 0 0.000000 -8638> 2014-08-13 17:22:48.719407 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8637> 2014-08-13 17:22:48.719422 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8636> 2014-08-13 17:22:48.719445 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8635> 2014-08-13 17:22:48.719792 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.308975 7 0.000065 -8634> 2014-08-13 17:22:48.719830 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started -8633> 2014-08-13 17:22:48.719850 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start -8632> 2014-08-13 17:22:48.719860 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8631> 2014-08-13 17:22:48.719888 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000038 0 0.000000 -8630> 2014-08-13 17:22:48.719927 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray -8629> 2014-08-13 17:22:48.720178 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.129596 7 0.000055 -8628> 2014-08-13 17:22:48.720212 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started -8627> 2014-08-13 17:22:48.720252 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start -8626> 2014-08-13 17:22:48.720273 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray -8625> 2014-08-13 17:22:48.720303 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000051 0 0.000000 -8624> 2014-08-13 17:22:48.720315 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray -8623> 2014-08-13 17:22:48.720532 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Reset 0.169613 7 0.000045 -8622> 2014-08-13 17:22:48.720553 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started -8621> 2014-08-13 17:22:48.720584 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Start -8620> 2014-08-13 17:22:48.720603 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: transitioning to Primary -8619> 2014-08-13 17:22:48.720625 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Start 0.000041 0 0.000000 -8618> 2014-08-13 17:22:48.720647 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary -8617> 2014-08-13 17:22:48.720667 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering -8616> 2014-08-13 17:22:48.720686 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetInfo -8615> 2014-08-13 17:22:48.721037 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.303429 7 0.000082 -8614> 2014-08-13 17:22:48.721084 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started -8613> 2014-08-13 17:22:48.721130 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start -8612> 2014-08-13 17:22:48.721181 7f74d192a700 1 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8611> 2014-08-13 17:22:48.721218 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000088 0 0.000000 -8610> 2014-08-13 17:22:48.721231 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8609> 2014-08-13 17:22:48.721242 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8608> 2014-08-13 17:22:48.721285 7f74d192a700 5 osd.4 pg_epoch: 194455 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8607> 2014-08-13 17:22:48.721722 7f74ed1707a0 0 osd.4 194455 done with init, starting boot process -8606> 2014-08-13 17:22:48.721731 7f74ed1707a0 10 monclient: get_version osdmap req 0x33e9ea0 -8605> 2014-08-13 17:22:48.721736 7f74ed1707a0 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -8604> 2014-08-13 17:22:48.721743 7f74ed1707a0 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- ?+0 0x27d0820 con 0x45d48e0 -8603> 2014-08-13 17:22:48.721856 7f74ed1707a0 5 asok(0x2700230) register_command status hook 0x4290f40 -8602> 2014-08-13 17:22:48.721891 7f74ed1707a0 5 asok(0x2700230) register_command flush_journal hook 0x4290f40 -8601> 2014-08-13 17:22:48.721897 7f74ed1707a0 5 asok(0x2700230) register_command dump_ops_in_flight hook 0x4290f40 -8600> 2014-08-13 17:22:48.721904 7f74ed1707a0 5 asok(0x2700230) register_command dump_historic_ops hook 0x4290f40 -8599> 2014-08-13 17:22:48.721909 7f74ed1707a0 5 asok(0x2700230) register_command dump_op_pq_state hook 0x4290f40 -8598> 2014-08-13 17:22:48.721915 7f74ed1707a0 5 asok(0x2700230) register_command dump_blacklist hook 0x4290f40 -8597> 2014-08-13 17:22:48.721920 7f74ed1707a0 5 asok(0x2700230) register_command dump_watchers hook 0x4290f40 -8596> 2014-08-13 17:22:48.721929 7f74ed1707a0 5 asok(0x2700230) register_command setomapval hook 0x33ec440 -8595> 2014-08-13 17:22:48.721934 7f74ed1707a0 5 asok(0x2700230) register_command rmomapkey hook 0x33ec440 -8594> 2014-08-13 17:22:48.721940 7f74ed1707a0 5 asok(0x2700230) register_command setomapheader hook 0x33ec440 -8593> 2014-08-13 17:22:48.721945 7f74ed1707a0 5 asok(0x2700230) register_command getomap hook 0x33ec440 -8592> 2014-08-13 17:22:48.721951 7f74ed1707a0 5 asok(0x2700230) register_command truncobj hook 0x33ec440 -8591> 2014-08-13 17:22:48.721956 7f74ed1707a0 5 asok(0x2700230) register_command injectdataerr hook 0x33ec440 -8590> 2014-08-13 17:22:48.721974 7f74ed1707a0 5 asok(0x2700230) register_command injectmdataerr hook 0x33ec440 -8589> 2014-08-13 17:22:48.722249 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 10 ==== mon_check_map_ack(handle=1 version=194469) v2 ==== 24+0+0 (3124165440 0 0) 0x466b9c0 con 0x45d48e0 -8588> 2014-08-13 17:22:48.722267 7f74d9b37700 10 monclient: handle_get_version_reply finishing 0x33e9ea0 version 194469 -8587> 2014-08-13 17:22:48.722787 7f74d232b700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -8586> 2014-08-13 17:22:48.722808 7f74d232b700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_boot(osd.4 booted 0 v194455) v5 -- ?+0 0x26e4380 con 0x45d48e0 -8585> 2014-08-13 17:22:48.793187 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x480fbc0 con 0x45f1760 -8584> 2014-08-13 17:22:48.793242 7f74d8735700 10 monclient: renew_subs -8583> 2014-08-13 17:22:48.793247 7f74d8735700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -8582> 2014-08-13 17:22:48.793255 7f74d8735700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- mon_subscribe({monmap=4+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x466b640 con 0x45d48e0 -8581> 2014-08-13 17:22:48.793285 7f74d8735700 3 osd.4 194455 handle_osd_map epochs [194456,194470], i have 194455, src has [185139,194470] -8580> 2014-08-13 17:22:48.794580 7f74bf77e700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485e680 sd=190 :6813 s=0 pgs=0 cs=0 l=0 c=0x4900420).accept sd=190 192.168.1.115:40724/0 -8579> 2014-08-13 17:22:48.795474 7f74bf37a700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485eb80 sd=193 :6813 s=0 pgs=0 cs=0 l=0 c=0x4900f20).accept sd=193 192.168.1.110:39951/0 -8578> 2014-08-13 17:22:48.795499 7f74bf67d700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x485e900 sd=191 :6817 s=0 pgs=0 cs=0 l=0 c=0x4900840).accept sd=191 192.168.1.107:57289/0 -8577> 2014-08-13 17:22:48.798112 7f74bf279700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x485f300 sd=194 :6817 s=0 pgs=0 cs=0 l=0 c=0x49009a0).accept sd=194 192.168.1.115:58867/0 -8576> 2014-08-13 17:22:48.798659 7f74bf47b700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485ee00 sd=192 :6813 s=0 pgs=0 cs=0 l=0 c=0x4900580).accept sd=192 192.168.1.105:44219/0 -8575> 2014-08-13 17:22:48.798835 7f74bec73700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x485f080 sd=200 :6817 s=0 pgs=0 cs=0 l=0 c=0x4901a20).accept sd=200 192.168.1.101:40759/0 -8574> 2014-08-13 17:22:48.798897 7f74bf178700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485fa80 sd=195 :6813 s=0 pgs=0 cs=0 l=0 c=0x4901600).accept sd=195 192.168.1.110:39952/0 -8573> 2014-08-13 17:22:48.798965 7f74bf077700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x485fd00 sd=196 :6806 s=0 pgs=0 cs=0 l=0 c=0x4901760).accept sd=196 192.168.100.115:41569/0 -8572> 2014-08-13 17:22:48.799102 7f74beb72700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c8780 sd=201 :6813 s=0 pgs=0 cs=0 l=0 c=0x49023c0).accept sd=201 192.168.1.114:42022/0 -8571> 2014-08-13 17:22:48.799626 7f74bed74700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485f580 sd=199 :6813 s=0 pgs=0 cs=0 l=0 c=0x4902680).accept sd=199 192.168.1.102:42208/0 -8570> 2014-08-13 17:22:48.799718 7f74bea71700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49c8f00 sd=202 :6817 s=0 pgs=0 cs=0 l=0 c=0x4902c00).accept sd=202 192.168.1.110:50460/0 -8569> 2014-08-13 17:22:48.800041 7f74be46b700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c8000 sd=205 :6813 s=0 pgs=0 cs=0 l=0 c=0x4903440).accept sd=205 192.168.1.101:55147/0 -8568> 2014-08-13 17:22:48.800088 7f74be970700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d8280 sd=198 :6806 s=0 pgs=0 cs=0 l=0 c=0x4901ce0).accept sd=198 192.168.100.107:52413/0 -8567> 2014-08-13 17:22:48.801600 7f74be36a700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d8000 sd=206 :6806 s=0 pgs=0 cs=0 l=0 c=0x4902ec0).accept sd=206 192.168.100.110:59497/0 -8566> 2014-08-13 17:22:48.801616 7f74be76e700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c8500 sd=203 :6813 s=0 pgs=0 cs=0 l=0 c=0x4902100).accept sd=203 192.168.1.101:55146/0 -8565> 2014-08-13 17:22:48.801623 7f74be56c700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c8280 sd=204 :6813 s=0 pgs=0 cs=0 l=0 c=0x4903700).accept sd=204 192.168.1.109:60426/0 -8564> 2014-08-13 17:22:48.801698 7f74be168700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49d9680 sd=207 :6813 s=0 pgs=0 cs=0 l=0 c=0x4903180).accept sd=207 192.168.1.104:39647/0 -8563> 2014-08-13 17:22:48.801329 7f74bef76700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x485f800 sd=197 :6813 s=0 pgs=0 cs=0 l=0 c=0x4901340).accept sd=197 192.168.1.115:40736/0 -8562> 2014-08-13 17:22:48.803259 7f74bdb62700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49d9400 sd=210 :6813 s=0 pgs=0 cs=0 l=0 c=0x4904a40).accept sd=210 192.168.1.103:58751/0 -8561> 2014-08-13 17:22:48.801837 7f74bdc63700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d8780 sd=209 :6806 s=0 pgs=0 cs=0 l=0 c=0x4903de0).accept sd=209 192.168.100.103:47118/0 -8560> 2014-08-13 17:22:48.804324 7f74be269700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d8a00 sd=208 :6806 s=0 pgs=0 cs=0 l=0 c=0x49040a0).accept sd=208 192.168.100.101:53699/0 -8559> 2014-08-13 17:22:48.804603 7f74bd85f700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d8500 sd=212 :6806 s=0 pgs=0 cs=0 l=0 c=0x49056a0).accept sd=212 192.168.100.102:36395/0 -8558> 2014-08-13 17:22:48.805076 7f74bd960700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49c8c80 sd=211 :6817 s=0 pgs=0 cs=0 l=0 c=0x4902940).accept sd=211 192.168.1.103:34960/0 -8557> 2014-08-13 17:22:48.806818 7f74bd45b700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49d9180 sd=213 :6813 s=0 pgs=0 cs=0 l=0 c=0x4904780).accept sd=213 192.168.1.112:43732/0 -8556> 2014-08-13 17:22:48.807123 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 11 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x2712400 con 0x45d48e0 -8555> 2014-08-13 17:22:48.807162 7f74bd158700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49d8f00 sd=214 :6813 s=0 pgs=0 cs=0 l=0 c=0x49044c0).accept sd=214 192.168.1.113:45949/0 -8554> 2014-08-13 17:22:48.808701 7f74bd057700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49da080 sd=215 :6806 s=0 pgs=0 cs=0 l=0 c=0x49053e0).accept sd=215 192.168.100.111:34366/0 -8553> 2014-08-13 17:22:48.808769 7f74bcf56700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49c8a00 sd=216 :6817 s=0 pgs=0 cs=0 l=0 c=0x4906460).accept sd=216 192.168.1.111:35939/0 -8552> 2014-08-13 17:22:48.809162 7f74bcd54700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49d8c80 sd=218 :6813 s=0 pgs=0 cs=0 l=0 c=0x49070c0).accept sd=218 192.168.1.111:42045/0 -8551> 2014-08-13 17:22:48.809199 7f74bce55700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d9e00 sd=217 :6806 s=0 pgs=0 cs=0 l=0 c=0x4905120).accept sd=217 192.168.100.114:59212/0 -8550> 2014-08-13 17:22:48.809516 7f74bcc53700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49daa80 sd=219 :6817 s=0 pgs=0 cs=0 l=0 c=0x49061a0).accept sd=219 192.168.1.114:42639/0 -8549> 2014-08-13 17:22:48.810456 7f74bca51700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d9b80 sd=221 :6806 s=0 pgs=0 cs=0 l=0 c=0x49077a0).accept sd=221 192.168.100.112:58706/0 -8548> 2014-08-13 17:22:48.811546 7f74bc950700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49da800 sd=222 :6817 s=0 pgs=0 cs=0 l=0 c=0x4905ee0).accept sd=222 192.168.1.112:59456/0 -8547> 2014-08-13 17:22:48.811784 7f74bc84f700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49db700 sd=223 :6813 s=0 pgs=0 cs=0 l=0 c=0x4906b40).accept sd=223 192.168.1.114:42031/0 -8546> 2014-08-13 17:22:48.811927 7f74bc34a700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49db480 sd=225 :6813 s=0 pgs=0 cs=0 l=0 c=0x4906880).accept sd=225 192.168.1.113:45955/0 -8545> 2014-08-13 17:22:48.812086 7f74bc74e700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49da580 sd=224 :6817 s=0 pgs=0 cs=0 l=0 c=0x4907d20).accept sd=224 192.168.1.102:57864/0 -8544> 2014-08-13 17:22:48.812136 7f74bcb52700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49db980 sd=220 :6813 s=0 pgs=0 cs=0 l=0 c=0x4906e00).accept sd=220 192.168.1.103:58754/0 -8543> 2014-08-13 17:22:48.812138 7f74bbd44700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49db200 sd=226 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d58b00).accept sd=226 192.168.1.114:42033/0 -8542> 2014-08-13 17:22:48.812540 7f74bbc43700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49daf80 sd=227 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d58840).accept sd=227 192.168.1.108:32812/0 -8541> 2014-08-13 17:22:48.814454 7f74bb33a700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dcb00 sd=231 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d582c0).accept sd=231 192.168.1.102:42232/0 -8540> 2014-08-13 17:22:48.814509 7f74bbb42700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49d9900 sd=228 :6806 s=0 pgs=0 cs=0 l=0 c=0x49074e0).accept sd=228 192.168.100.108:58049/0 -8539> 2014-08-13 17:22:48.814646 7f74bb63d700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dad00 sd=230 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d58580).accept sd=230 192.168.1.115:40748/0 -8538> 2014-08-13 17:22:48.814825 7f74bb239700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dc880 sd=232 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d58000).accept sd=232 192.168.1.102:42233/0 -8537> 2014-08-13 17:22:48.815071 7f74bb940700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49da300 sd=229 :6817 s=0 pgs=0 cs=0 l=0 c=0x4907a60).accept sd=229 192.168.1.108:49285/0 -8536> 2014-08-13 17:22:48.815123 7f74bb138700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dc600 sd=233 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d51ce0).accept sd=233 192.168.1.102:42234/0 -8535> 2014-08-13 17:22:48.815240 7f74baf36700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dc380 sd=234 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d51a20).accept sd=234 192.168.1.111:42050/0 -8534> 2014-08-13 17:22:48.817865 7f74bae35700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dc100 sd=235 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d51760).accept sd=235 192.168.1.109:60447/0 -8533> 2014-08-13 17:22:48.819269 7f74bab32700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49de400 sd=237 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d509a0).accept sd=237 192.168.1.104:45642/0 -8532> 2014-08-13 17:22:48.818700 7f74bac33700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49dd780 sd=236 :6806 s=0 pgs=0 cs=0 l=0 c=0x4907220).accept sd=236 192.168.100.104:53311/0 -8531> 2014-08-13 17:22:48.819484 7f74ba52c700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dbe80 sd=238 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d514a0).accept sd=238 192.168.1.104:39661/0 -8530> 2014-08-13 17:22:48.819729 7f74ba229700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dbc00 sd=239 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d511e0).accept sd=239 192.168.1.102:42235/0 -8529> 2014-08-13 17:22:48.821999 7f74b9920700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49df800 sd=240 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d54200).accept sd=240 192.168.1.101:55159/0 -8528> 2014-08-13 17:22:48.822009 7f74b961d700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49df580 sd=241 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d53f40).accept sd=241 192.168.1.107:46301/0 -8527> 2014-08-13 17:22:48.823403 7f74b951c700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49df300 sd=242 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d53c80).accept sd=242 192.168.1.108:32821/0 -8526> 2014-08-13 17:22:48.824101 7f74b941b700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49df080 sd=243 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d539c0).accept sd=243 192.168.1.112:43747/0 -8525> 2014-08-13 17:22:48.824245 7f74b9118700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49deb80 sd=245 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d56300).accept sd=245 192.168.1.110:39974/0 -8524> 2014-08-13 17:22:48.824122 7f74b9219700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49dee00 sd=244 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d53700).accept sd=244 192.168.1.109:60449/0 -8523> 2014-08-13 17:22:48.824301 7f74b9017700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49de900 sd=246 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d56040).accept sd=246 192.168.1.114:42039/0 -8522> 2014-08-13 17:22:48.824407 7f74b8f16700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49de680 sd=247 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d55d80).accept sd=247 192.168.1.104:39668/0 -8521> 2014-08-13 17:22:48.824655 7f74b8d14700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c9e00 sd=249 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d55800).accept sd=249 192.168.1.111:42054/0 -8520> 2014-08-13 17:22:48.825123 7f74b8e15700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49ca080 sd=248 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d55ac0).accept sd=248 192.168.1.114:42041/0 -8519> 2014-08-13 17:22:48.825184 7f74b830a700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c9900 sd=251 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d57d20).accept sd=251 192.168.1.108:32826/0 -8518> 2014-08-13 17:22:48.827260 7f74b8c13700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c9b80 sd=250 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d55540).accept sd=250 192.168.1.101:55163/0 -8517> 2014-08-13 17:22:48.828166 7f74b7b02700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49dd500 sd=252 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d52940).accept sd=252 192.168.100.106:42361/0 -8516> 2014-08-13 17:22:48.828184 7f74b7a01700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49de180 sd=253 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d506e0).accept sd=253 192.168.1.106:51888/0 -8515> 2014-08-13 17:22:48.832415 7f74b76fe700 1 -- 192.168.1.115:6813/30165 >> :/0 pipe(0x49c9680 sd=254 :6813 s=0 pgs=0 cs=0 l=0 c=0x4d57a60).accept sd=254 192.168.1.108:32832/0 -8514> 2014-08-13 17:22:48.868227 7f74b74fc700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49ddf00 sd=255 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d50420).accept sd=255 192.168.1.102:57880/0 -8513> 2014-08-13 17:22:48.868264 7f74b73fb700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49dd280 sd=256 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d52680).accept sd=256 192.168.100.102:36422/0 -8512> 2014-08-13 17:22:48.872703 7f74b70f8700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49ddc80 sd=257 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d61ce0).accept sd=257 192.168.1.104:45649/0 -8511> 2014-08-13 17:22:48.872760 7f74b6ff7700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49dd000 sd=258 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d523c0).accept sd=258 192.168.100.104:53320/0 -8510> 2014-08-13 17:22:48.873940 7f74b6cf4700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x49dda00 sd=259 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d61a20).accept sd=259 192.168.1.112:59471/0 -8509> 2014-08-13 17:22:48.874092 7f74b6bf3700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x49dcd80 sd=260 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d62aa0).accept sd=260 192.168.100.112:58721/0 -8508> 2014-08-13 17:22:48.890291 7f74d8735700 1 -- 192.168.1.115:6813/30165 mark_down 192.168.1.107:6802/9548 -- 0x2ef5780 -8507> 2014-08-13 17:22:48.891087 7f74d8735700 1 osd.4 194470 state: booting -> active -8506> 2014-08-13 17:22:48.891397 7f74d8735700 0 osd.4 194470 crush map has features 2578087936000, adjusting msgr requires for mons -8505> 2014-08-13 17:22:48.891643 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: Peering, affected_by_map, going to Reset -8504> 2014-08-13 17:22:48.891697 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetInfo 0.191350 2 0.000340 -8503> 2014-08-13 17:22:48.891705 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.185967 3 0.000126 -8502> 2014-08-13 17:22:48.891729 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering 0.191404 0 0.000000 -8501> 2014-08-13 17:22:48.891751 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started 0.186064 0 0.000000 -8500> 2014-08-13 17:22:48.891759 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.191471 0 0.000000 -8499> 2014-08-13 17:22:48.891776 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192971) [321,379,4] r=2 lpr=194449 pi=192971-194447/16 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset -8498> 2014-08-13 17:22:48.891783 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started 0.191618 0 0.000000 -8497> 2014-08-13 17:22:48.891808 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194447/194448/194448) [4,286,375]/[4,375,10] r=0 lpr=194449 pi=193640-194447/20 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset -8496> 2014-08-13 17:22:48.892656 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.000880 15 0.000354 -8495> 2014-08-13 17:22:48.892671 7f74b67ef700 1 -- 192.168.1.115:0/30165 learned my addr 192.168.1.115:0/30165 -8494> 2014-08-13 17:22:48.892696 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started -8493> 2014-08-13 17:22:48.892719 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start -8492> 2014-08-13 17:22:48.892734 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8491> 2014-08-13 17:22:48.892753 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 -8490> 2014-08-13 17:22:48.892772 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray -8489> 2014-08-13 17:22:48.894303 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8488> 2014-08-13 17:22:48.894348 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.188175 2 0.000243 -8487> 2014-08-13 17:22:48.894381 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.188245 0 0.000000 -8486> 2014-08-13 17:22:48.894407 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.188285 0 0.000000 -8485> 2014-08-13 17:22:48.894441 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started 0.188423 0 0.000000 -8484> 2014-08-13 17:22:48.894464 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,349,61] r=0 lpr=194449 pi=193306-194447/16 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset -8483> 2014-08-13 17:22:48.894944 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000480 15 0.000355 -8482> 2014-08-13 17:22:48.894978 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started -8481> 2014-08-13 17:22:48.895000 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start -8480> 2014-08-13 17:22:48.895019 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8479> 2014-08-13 17:22:48.895041 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000041 0 0.000000 -8478> 2014-08-13 17:22:48.895064 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8477> 2014-08-13 17:22:48.895087 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8476> 2014-08-13 17:22:48.895108 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8475> 2014-08-13 17:22:48.895364 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started/Stray 0.188726 3 0.000175 -8474> 2014-08-13 17:22:48.895400 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started 0.188839 0 0.000000 -8473> 2014-08-13 17:22:48.895428 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194447/194448/192966) [364,4,360]/[364,4,301] r=1 lpr=194449 pi=193125-194447/22 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset -8472> 2014-08-13 17:22:48.895976 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Reset 0.000547 15 0.000212 -8471> 2014-08-13 17:22:48.896009 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started -8470> 2014-08-13 17:22:48.896029 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Start -8469> 2014-08-13 17:22:48.896898 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: transitioning to Stray -8468> 2014-08-13 17:22:48.896929 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Start 0.000899 0 0.000000 -8467> 2014-08-13 17:22:48.896955 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194449/194389 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started/Stray -8466> 2014-08-13 17:22:48.897495 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8465> 2014-08-13 17:22:48.897533 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.190510 2 0.000124 -8464> 2014-08-13 17:22:48.897557 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.190548 0 0.000000 -8463> 2014-08-13 17:22:48.897582 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.190584 0 0.000000 -8462> 2014-08-13 17:22:48.897608 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.190656 0 0.000000 -8461> 2014-08-13 17:22:48.897631 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -8460> 2014-08-13 17:22:48.897997 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000365 15 0.000278 -8459> 2014-08-13 17:22:48.898028 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8458> 2014-08-13 17:22:48.898048 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8457> 2014-08-13 17:22:48.898066 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8456> 2014-08-13 17:22:48.898086 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000038 0 0.000000 -8455> 2014-08-13 17:22:48.898108 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8454> 2014-08-13 17:22:48.898127 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8453> 2014-08-13 17:22:48.898147 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8452> 2014-08-13 17:22:48.899449 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8451> 2014-08-13 17:22:48.899495 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.192171 2 0.000104 -8450> 2014-08-13 17:22:48.899522 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.192207 0 0.000000 -8449> 2014-08-13 17:22:48.899547 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.192241 0 0.000000 -8448> 2014-08-13 17:22:48.899569 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started 0.192304 0 0.000000 -8447> 2014-08-13 17:22:48.899596 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194447/194448/194448) [4,339,45] r=0 lpr=194449 pi=193306-194447/16 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset -8446> 2014-08-13 17:22:48.900073 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000478 15 0.000345 -8445> 2014-08-13 17:22:48.900108 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started -8444> 2014-08-13 17:22:48.900127 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start -8443> 2014-08-13 17:22:48.900145 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8442> 2014-08-13 17:22:48.900178 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000050 0 0.000000 -8441> 2014-08-13 17:22:48.900201 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8440> 2014-08-13 17:22:48.900221 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8439> 2014-08-13 17:22:48.900240 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194443/194389 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8438> 2014-08-13 17:22:48.900725 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 peering m=49] state: Peering, affected_by_map, going to Reset -8437> 2014-08-13 17:22:48.900760 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 peering m=49] exit Started/Primary/Peering/GetInfo 0.193205 2 0.000134 -8436> 2014-08-13 17:22:48.900786 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 peering m=49] exit Started/Primary/Peering 0.193245 0 0.000000 -8435> 2014-08-13 17:22:48.900814 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] exit Started/Primary 0.193288 0 0.000000 -8434> 2014-08-13 17:22:48.900837 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] exit Started 0.193367 0 0.000000 -8433> 2014-08-13 17:22:48.900860 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194450/194451/194450) [349,4]/[4,395,13] r=0 lpr=194453 pi=193304-194450/52 crt=194422'13151 lcod 0'0 mlcod 0'0 inactive m=49] enter Reset -8432> 2014-08-13 17:22:48.914186 7f74d8735700 1 osd.4 194470 pausing recovery (NORECOVER flag set) -8431> 2014-08-13 17:22:48.914189 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Reset 0.022380 15 0.000736 -8430> 2014-08-13 17:22:48.914198 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.013338 15 0.000292 -8429> 2014-08-13 17:22:48.914245 7f74d9b37700 10 monclient: renew_subs -8428> 2014-08-13 17:22:48.914245 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started -8427> 2014-08-13 17:22:48.914264 7f74d8735700 1 osd.4 194470 ms_handle_reset con 0x47f5120 session 0 -8426> 2014-08-13 17:22:48.914249 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started -8425> 2014-08-13 17:22:48.914274 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Start -8424> 2014-08-13 17:22:48.914291 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Start -8423> 2014-08-13 17:22:48.914257 7f74d9b37700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -8422> 2014-08-13 17:22:48.914298 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: transitioning to Primary -8421> 2014-08-13 17:22:48.914314 7f74d9b37700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- mon_subscribe({monmap=4+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x466b9c0 con 0x45d48e0 -8420> 2014-08-13 17:22:48.914322 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Start 0.000048 0 0.000000 -8419> 2014-08-13 17:22:48.914355 7f74d9b37700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8418> 2014-08-13 17:22:48.914352 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary -8417> 2014-08-13 17:22:48.914312 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray -8416> 2014-08-13 17:22:48.914384 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 12 ==== osd_map(194470..194470 src has 185139..194470) v3 ==== 268298+0+0 (1909815769 0 0) 0x4c76300 con 0x45d48e0 -8415> 2014-08-13 17:22:48.914377 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering -8414> 2014-08-13 17:22:48.914408 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000118 0 0.000000 -8413> 2014-08-13 17:22:48.914475 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194460 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray -8412> 2014-08-13 17:22:48.914493 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 2 ==== pg_notify(22.3d5(2) epoch 194470) v5 ==== 828+0+0 (405522152 0 0) 0x466b800 con 0x45f1760 -8411> 2014-08-13 17:22:48.914472 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194443/194389 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetInfo -8410> 2014-08-13 17:22:48.914695 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 0.206810 3 0.000155 -8409> 2014-08-13 17:22:48.914739 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started 0.206908 0 0.000000 -8408> 2014-08-13 17:22:48.914764 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194447/194447/193250) [304,323,4] r=2 lpr=194449 pi=193250-194446/19 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset -8407> 2014-08-13 17:22:48.915284 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Reset 0.000519 15 0.000243 -8406> 2014-08-13 17:22:48.915318 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started -8405> 2014-08-13 17:22:48.915319 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.214123 3 0.000482 -8404> 2014-08-13 17:22:48.915344 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Start -8403> 2014-08-13 17:22:48.915368 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: transitioning to Stray -8402> 2014-08-13 17:22:48.915386 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.214373 0 0.000000 -8401> 2014-08-13 17:22:48.915397 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Start 0.000053 0 0.000000 -8400> 2014-08-13 17:22:48.915420 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194448/194442 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started/Stray -8399> 2014-08-13 17:22:48.915419 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194447/194448/192967) [351,4,18] r=1 lpr=194449 pi=192970-194447/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -8398> 2014-08-13 17:22:48.915583 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8397> 2014-08-13 17:22:48.915621 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.207440 2 0.000250 -8396> 2014-08-13 17:22:48.915649 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.207488 0 0.000000 -8395> 2014-08-13 17:22:48.915676 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.207538 0 0.000000 -8394> 2014-08-13 17:22:48.915700 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.207649 0 0.000000 -8393> 2014-08-13 17:22:48.915721 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194447/194448/194448) [4,340] r=0 lpr=194449 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -8392> 2014-08-13 17:22:48.916123 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.000704 15 0.000286 -8391> 2014-08-13 17:22:48.916162 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -8390> 2014-08-13 17:22:48.916183 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -8389> 2014-08-13 17:22:48.916216 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8388> 2014-08-13 17:22:48.916240 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000057 0 0.000000 -8387> 2014-08-13 17:22:48.916263 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194449/194389 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -8386> 2014-08-13 17:22:48.916260 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000540 15 0.000301 -8385> 2014-08-13 17:22:48.916292 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8384> 2014-08-13 17:22:48.916318 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8383> 2014-08-13 17:22:48.916338 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8382> 2014-08-13 17:22:48.916360 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 -8381> 2014-08-13 17:22:48.916382 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8380> 2014-08-13 17:22:48.916401 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8379> 2014-08-13 17:22:48.916420 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194449/194449 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8378> 2014-08-13 17:22:48.916435 7f74d9b37700 3 osd.4 194470 handle_osd_map epochs [194470,194470], i have 194470, src has [185139,194470] -8377> 2014-08-13 17:22:48.916431 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8376> 2014-08-13 17:22:48.916458 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.216114 2 0.000332 -8375> 2014-08-13 17:22:48.916469 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 13 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x466af40 con 0x45d48e0 -8374> 2014-08-13 17:22:48.916474 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.793117, event: header_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -8373> 2014-08-13 17:22:48.916482 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.216165 0 0.000000 -8372> 2014-08-13 17:22:48.916510 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.793118, event: throttled, request: pg_notify(22.3d5(2) epoch 194470) v5 -8371> 2014-08-13 17:22:48.916500 7f74d9b37700 10 monclient: handle_subscribe_ack sent 2014-08-13 17:22:48.793245 renew after 2014-08-13 17:25:18.793245 -8370> 2014-08-13 17:22:48.916524 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.793170, event: all_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -8369> 2014-08-13 17:22:48.916524 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.216235 0 0.000000 -8368> 2014-08-13 17:22:48.916536 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.914521, event: dispatched, request: pg_notify(22.3d5(2) epoch 194470) v5 -8367> 2014-08-13 17:22:48.916560 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.916560, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194470) v5 -8366> 2014-08-13 17:22:48.916552 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started 0.216372 0 0.000000 -8365> 2014-08-13 17:22:48.916588 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.916588, event: started, request: pg_notify(22.3d5(2) epoch 194470) v5 -8364> 2014-08-13 17:22:48.916585 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,58,343] r=0 lpr=194449 pi=193306-194447/18 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset -8363> 2014-08-13 17:22:48.916597 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8362> 2014-08-13 17:22:48.916626 7f74d8735700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:22:48.916626, event: done, request: pg_notify(22.3d5(2) epoch 194470) v5 -8361> 2014-08-13 17:22:48.916631 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.208164 2 0.000163 -8360> 2014-08-13 17:22:48.916656 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.208202 0 0.000000 -8359> 2014-08-13 17:22:48.916656 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4802400 con 0x45f1e40 -8358> 2014-08-13 17:22:48.916684 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8357> 2014-08-13 17:22:48.916683 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.208247 0 0.000000 -8356> 2014-08-13 17:22:48.916707 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4804c80 con 0x45f2940 -8355> 2014-08-13 17:22:48.916706 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Started 0.208347 0 0.000000 -8354> 2014-08-13 17:22:48.916733 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8353> 2014-08-13 17:22:48.916741 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194453/194453/194450) [4,27] r=0 lpr=194454 pi=193125-194452/43 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset -8352> 2014-08-13 17:22:48.916767 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a818c0 con 0x45f3b20 -8351> 2014-08-13 17:22:48.916787 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8350> 2014-08-13 17:22:48.916805 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4801680 con 0x45f5540 -8349> 2014-08-13 17:22:48.916824 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8348> 2014-08-13 17:22:48.916840 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4806c00 con 0x45f5960 -8347> 2014-08-13 17:22:48.916860 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8346> 2014-08-13 17:22:48.916878 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de0d80 con 0x45f6f60 -8345> 2014-08-13 17:22:48.916897 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8344> 2014-08-13 17:22:48.916915 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c75100 con 0x4758b00 -8343> 2014-08-13 17:22:48.916934 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8342> 2014-08-13 17:22:48.916950 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4802880 con 0x4758840 -8341> 2014-08-13 17:22:48.916969 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8340> 2014-08-13 17:22:48.916984 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4804800 con 0x47591e0 -8339> 2014-08-13 17:22:48.917002 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8338> 2014-08-13 17:22:48.917018 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a82f40 con 0x475a940 -8337> 2014-08-13 17:22:48.917039 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8336> 2014-08-13 17:22:48.917056 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e10000 con 0x475cd00 -8335> 2014-08-13 17:22:48.917076 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8334> 2014-08-13 17:22:48.917091 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 1 ==== osd_map(194453..194470 src has 185139..194470) v3 ==== 11782+0+0 (1086985419 0 0) 0x4802640 con 0x475c4c0 -8333> 2014-08-13 17:22:48.917110 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194453,194470], i have 194470, src has [185139,194470] -8332> 2014-08-13 17:22:48.917126 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 1 ==== osd_map(194454..194470 src has 185139..194470) v3 ==== 10994+0+0 (2346696470 0 0) 0x4c74140 con 0x475d960 -8331> 2014-08-13 17:22:48.917145 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194454,194470], i have 194470, src has [185139,194470] -8330> 2014-08-13 17:22:48.917146 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000560 15 0.000337 -8329> 2014-08-13 17:22:48.917178 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started -8328> 2014-08-13 17:22:48.917161 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c71680 con 0x475ee00 -8327> 2014-08-13 17:22:48.917199 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 14 ==== osd_map(194470..194470 src has 185139..194470) v3 ==== 268298+0+0 (1909815769 0 0) 0x4c760c0 con 0x45d48e0 -8326> 2014-08-13 17:22:48.917208 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start -8325> 2014-08-13 17:22:48.917219 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8324> 2014-08-13 17:22:48.917231 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8323> 2014-08-13 17:22:48.917257 7f74d9b37700 3 osd.4 194470 handle_osd_map epochs [194470,194470], i have 194470, src has [185139,194470] -8322> 2014-08-13 17:22:48.917256 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 1 ==== osd_map(194455..194470 src has 185139..194470) v3 ==== 9657+0+0 (2148734153 0 0) 0x4803840 con 0x475e5c0 -8321> 2014-08-13 17:22:48.917250 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 -8320> 2014-08-13 17:22:48.917283 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194455,194470], i have 194470, src has [185139,194470] -8319> 2014-08-13 17:22:48.917280 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8318> 2014-08-13 17:22:48.917285 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 15 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x57d0fc0 con 0x45d48e0 -8317> 2014-08-13 17:22:48.917300 7f74d9b37700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -8316> 2014-08-13 17:22:48.917302 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a84c80 con 0x47a27e0 -8315> 2014-08-13 17:22:48.917299 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8314> 2014-08-13 17:22:48.917326 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8313> 2014-08-13 17:22:48.917330 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8312> 2014-08-13 17:22:48.917345 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c74380 con 0x47a0b00 -8311> 2014-08-13 17:22:48.917363 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8310> 2014-08-13 17:22:48.917380 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 1 ==== osd_map(194455..194470 src has 185139..194470) v3 ==== 9657+0+0 (2148734153 0 0) 0x4dc8240 con 0x47a4200 -8309> 2014-08-13 17:22:48.917398 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194455,194470], i have 194470, src has [185139,194470] -8308> 2014-08-13 17:22:48.917412 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c74ec0 con 0x47a53e0 -8307> 2014-08-13 17:22:48.917407 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Reset 0.000666 15 0.000291 -8306> 2014-08-13 17:22:48.917435 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8305> 2014-08-13 17:22:48.917437 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started -8304> 2014-08-13 17:22:48.917451 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x48072c0 con 0x47a3f40 -8303> 2014-08-13 17:22:48.917466 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8302> 2014-08-13 17:22:48.917461 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Start -8301> 2014-08-13 17:22:48.917476 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8300> 2014-08-13 17:22:48.917481 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a83180 con 0x47a4ba0 -8299> 2014-08-13 17:22:48.917509 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8298> 2014-08-13 17:22:48.917501 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -8297> 2014-08-13 17:22:48.917521 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started/Stray -8296> 2014-08-13 17:22:48.917519 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.216341 3 0.000505 -8295> 2014-08-13 17:22:48.917526 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c70900 con 0x47a32e0 -8294> 2014-08-13 17:22:48.917551 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8293> 2014-08-13 17:22:48.917549 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.216577 0 0.000000 -8292> 2014-08-13 17:22:48.917567 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80240 con 0x47f0c60 -8291> 2014-08-13 17:22:48.917574 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194447/194448/192972) [9,292,4] r=2 lpr=194449 pi=192972-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -8290> 2014-08-13 17:22:48.917586 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8289> 2014-08-13 17:22:48.917612 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 1 ==== osd_map(194455..194470 src has 185139..194470) v3 ==== 9657+0+0 (2148734153 0 0) 0x4e32880 con 0x47f2680 -8288> 2014-08-13 17:22:48.917632 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194455,194470], i have 194470, src has [185139,194470] -8287> 2014-08-13 17:22:48.917653 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a83a80 con 0x47f4620 -8286> 2014-08-13 17:22:48.917671 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8285> 2014-08-13 17:22:48.917690 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80480 con 0x47f6880 -8284> 2014-08-13 17:22:48.917713 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8283> 2014-08-13 17:22:48.917746 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e186c0 con 0x47f7e80 -8282> 2014-08-13 17:22:48.917747 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6802/58522 -- pg_notify(24.34(47) epoch 194470) v5 -- ?+0 0x466aa00 con 0x4d2aaa0 -8281> 2014-08-13 17:22:48.917764 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8280> 2014-08-13 17:22:48.917782 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(23) epoch 194470) v5 -- ?+0 0x466abc0 con 0x47f0c60 -8279> 2014-08-13 17:22:48.917804 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(19) epoch 194470) v5 -- ?+0 0x57d24c0 con 0x475e5c0 -8278> 2014-08-13 17:22:48.917800 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4805e80 con 0x47f70c0 -8277> 2014-08-13 17:22:48.917831 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8276> 2014-08-13 17:22:48.917830 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194470) v5 -- ?+0 0x57d2300 con 0x4903180 -8275> 2014-08-13 17:22:48.917844 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 1 ==== osd_map(194455..194470 src has 185139..194470) v3 ==== 9657+0+0 (2148734153 0 0) 0x4c73180 con 0x4831b80 -8274> 2014-08-13 17:22:48.917848 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(26) epoch 194470) v5 -- ?+0 0x57d2140 con 0x47a4200 -8273> 2014-08-13 17:22:48.917858 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194455,194470], i have 194470, src has [185139,194470] -8272> 2014-08-13 17:22:48.917865 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6800/52669 -- pg_query(22.113 epoch 194470) v3 -- ?+0 0x29c0780 con 0x4830dc0 -8271> 2014-08-13 17:22:48.917872 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4803a80 con 0x4833440 -8270> 2014-08-13 17:22:48.917886 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8269> 2014-08-13 17:22:48.917893 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_query(22.113 epoch 194470) v3 -- ?+0 0x29c6360 con 0x47a32e0 -8268> 2014-08-13 17:22:48.917900 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a857c0 con 0x4831600 -8267> 2014-08-13 17:22:48.917914 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8266> 2014-08-13 17:22:48.917912 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6834/6966 -- pg_query(22.113 epoch 194470) v3 -- ?+0 0x29c6180 con 0x47f7e80 -8265> 2014-08-13 17:22:48.917929 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6834/14316 -- pg_query(22.1d2 epoch 194470) v3 -- ?+0 0x29c5fa0 con 0x45f1e40 -8264> 2014-08-13 17:22:48.917947 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6840/16566 -- pg_query(22.1d2 epoch 194470) v3 -- ?+0 0x29c6ae0 con 0x47a0b00 -8263> 2014-08-13 17:22:48.917962 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.000388 15 0.000175 -8262> 2014-08-13 17:22:48.917993 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -8261> 2014-08-13 17:22:48.918011 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -8260> 2014-08-13 17:22:48.918028 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8259> 2014-08-13 17:22:48.918049 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000038 0 0.000000 -8258> 2014-08-13 17:22:48.918071 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194449/194449 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -8257> 2014-08-13 17:22:48.918089 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_query(1.2b,2.2a epoch 194470) v3 -- ?+0 0x29c6900 con 0x4903180 -8256> 2014-08-13 17:22:48.918130 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_query(22.1d2 epoch 194470) v3 -- ?+0 0x29c6720 con 0x486e040 -8255> 2014-08-13 17:22:48.918139 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c72640 con 0x4832c00 -8254> 2014-08-13 17:22:48.918153 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6816/837 -- pg_query(22.1d2 epoch 194470) v3 -- ?+0 0x29c6540 con 0x486bf40 -8253> 2014-08-13 17:22:48.918173 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8252> 2014-08-13 17:22:48.918192 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6818/45268 -- pg_query(22.113 epoch 194470) v3 -- ?+0 0x29c7440 con 0x4900f20 -8251> 2014-08-13 17:22:48.918199 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c72f40 con 0x4830dc0 -8250> 2014-08-13 17:22:48.918219 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8249> 2014-08-13 17:22:48.918216 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6806/62463 -- pg_query(1.2b,2.2a epoch 194470) v3 -- ?+0 0x29c7260 con 0x47a3f40 -8248> 2014-08-13 17:22:48.918226 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.216190 3 0.000392 -8247> 2014-08-13 17:22:48.918240 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 1 ==== osd_map(194454..194470 src has 185139..194470) v3 ==== 10994+0+0 (2346696470 0 0) 0x4800d80 con 0x4835ac0 -8246> 2014-08-13 17:22:48.918246 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6802/41438 -- pg_query(22.113 epoch 194470) v3 -- ?+0 0x29c7080 con 0x4901600 -8245> 2014-08-13 17:22:48.918259 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194454,194470], i have 194470, src has [185139,194470] -8244> 2014-08-13 17:22:48.918256 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.216296 0 0.000000 -8243> 2014-08-13 17:22:48.918266 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6800/61726 -- pg_query(22.1d2 epoch 194470) v3 -- ?+0 0x29c6ea0 con 0x4d55d80 -8242> 2014-08-13 17:22:48.918275 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/192935) [58,28,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -8241> 2014-08-13 17:22:48.918278 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.32 192.168.1.115:6812/59915 1 ==== osd_map(194452..194470 src has 185139..194470) v3 ==== 12164+0+0 (339452127 0 0) 0x4804ec0 con 0x4832100 -8240> 2014-08-13 17:22:48.918283 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6842/56512 -- pg_query(1.2b,2.2a epoch 194470) v3 -- ?+0 0x29c6cc0 con 0x4906b40 -8239> 2014-08-13 17:22:48.918293 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194452,194470], i have 194470, src has [185139,194470] -8238> 2014-08-13 17:22:48.918306 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a84ec0 con 0x4835280 -8237> 2014-08-13 17:22:48.918320 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8236> 2014-08-13 17:22:48.918349 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de3cc0 con 0x4834620 -8235> 2014-08-13 17:22:48.918364 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8234> 2014-08-13 17:22:48.918376 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4803cc0 con 0x48377a0 -8233> 2014-08-13 17:22:48.918389 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8232> 2014-08-13 17:22:48.918400 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x48006c0 con 0x4868420 -8231> 2014-08-13 17:22:48.918418 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8230> 2014-08-13 17:22:48.918394 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8229> 2014-08-13 17:22:48.918438 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c76540 con 0x4868160 -8228> 2014-08-13 17:22:48.918452 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8227> 2014-08-13 17:22:48.918464 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x48060c0 con 0x486bf40 -8226> 2014-08-13 17:22:48.918481 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8225> 2014-08-13 17:22:48.918455 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.209528 2 0.000178 -8224> 2014-08-13 17:22:48.918496 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a82d00 con 0x486bc80 -8223> 2014-08-13 17:22:48.918513 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8222> 2014-08-13 17:22:48.918503 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.209608 0 0.000000 -8221> 2014-08-13 17:22:48.918563 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e30900 con 0x486b9c0 -8220> 2014-08-13 17:22:48.918568 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.209694 0 0.000000 -8219> 2014-08-13 17:22:48.918585 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8218> 2014-08-13 17:22:48.918599 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started 0.209803 0 0.000000 -8217> 2014-08-13 17:22:48.918612 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a806c0 con 0x486b700 -8216> 2014-08-13 17:22:48.918629 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8215> 2014-08-13 17:22:48.918629 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,374,31] r=0 lpr=194449 pi=193306-194447/16 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset -8214> 2014-08-13 17:22:48.918652 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c75340 con 0x486b440 -8213> 2014-08-13 17:22:48.918675 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8212> 2014-08-13 17:22:48.918689 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 1 ==== osd_map(194454..194470 src has 185139..194470) v3 ==== 10994+0+0 (2346696470 0 0) 0x4de0240 con 0x486b180 -8211> 2014-08-13 17:22:48.918707 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194454,194470], i have 194470, src has [185139,194470] -8210> 2014-08-13 17:22:48.918721 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 1 ==== osd_map(194451..194470 src has 185139..194470) v3 ==== 13727+0+0 (2212813540 0 0) 0x4a84800 con 0x486aec0 -8209> 2014-08-13 17:22:48.918739 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194451,194470], i have 194470, src has [185139,194470] -8208> 2014-08-13 17:22:48.918755 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c72d00 con 0x486a940 -8207> 2014-08-13 17:22:48.918774 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8206> 2014-08-13 17:22:48.918787 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 1 ==== osd_map(194454..194470 src has 185139..194470) v3 ==== 10994+0+0 (2346696470 0 0) 0x4de33c0 con 0x486a680 -8205> 2014-08-13 17:22:48.918805 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194454,194470], i have 194470, src has [185139,194470] -8204> 2014-08-13 17:22:48.918821 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a833c0 con 0x486a3c0 -8203> 2014-08-13 17:22:48.918840 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8202> 2014-08-13 17:22:48.918865 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80900 con 0x48d9ce0 -8201> 2014-08-13 17:22:48.918861 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.000586 15 0.000182 -8200> 2014-08-13 17:22:48.918887 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8199> 2014-08-13 17:22:48.918889 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -8198> 2014-08-13 17:22:48.918903 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a81440 con 0x486e040 -8197> 2014-08-13 17:22:48.918911 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -8196> 2014-08-13 17:22:48.918922 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8195> 2014-08-13 17:22:48.918934 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8194> 2014-08-13 17:22:48.918951 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e33840 con 0x48db9c0 -8193> 2014-08-13 17:22:48.918955 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000044 0 0.000000 -8192> 2014-08-13 17:22:48.918973 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8191> 2014-08-13 17:22:48.918975 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -8190> 2014-08-13 17:22:48.919004 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a83cc0 con 0x48dad60 -8189> 2014-08-13 17:22:48.919023 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8188> 2014-08-13 17:22:48.919038 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x48018c0 con 0x48dcd00 -8187> 2014-08-13 17:22:48.919053 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8186> 2014-08-13 17:22:48.919065 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4dc8480 con 0x48dfe80 -8185> 2014-08-13 17:22:48.919078 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8184> 2014-08-13 17:22:48.919088 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4804a40 con 0x48dfbc0 -8183> 2014-08-13 17:22:48.919102 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8182> 2014-08-13 17:22:48.919111 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4801440 con 0x4900420 -8181> 2014-08-13 17:22:48.919125 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8180> 2014-08-13 17:22:48.919136 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80000 con 0x4900580 -8179> 2014-08-13 17:22:48.919149 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8178> 2014-08-13 17:22:48.919160 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80b40 con 0x4900f20 -8177> 2014-08-13 17:22:48.919174 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8176> 2014-08-13 17:22:48.919193 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a81680 con 0x4901600 -8175> 2014-08-13 17:22:48.919213 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8174> 2014-08-13 17:22:48.919230 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.19 192.168.1.115:6818/61186 1 ==== osd_map(194371..194470 src has 185139..194470) v3 ==== 88939+0+0 (3309911880 0 0) 0x4c70b40 con 0x4901340 -8173> 2014-08-13 17:22:48.919250 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194371,194470], i have 194470, src has [185139,194470] -8172> 2014-08-13 17:22:48.919302 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4dc0000 con 0x4902680 -8171> 2014-08-13 17:22:48.919323 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8170> 2014-08-13 17:22:48.919335 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a80d80 con 0x49023c0 -8169> 2014-08-13 17:22:48.919357 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8168> 2014-08-13 17:22:48.919441 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8167> 2014-08-13 17:22:48.919481 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.216826 2 0.000176 -8166> 2014-08-13 17:22:48.919508 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.216868 0 0.000000 -8165> 2014-08-13 17:22:48.919533 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.216925 0 0.000000 -8164> 2014-08-13 17:22:48.919544 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a86300 con 0x4902100 -8163> 2014-08-13 17:22:48.919562 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started 0.217030 0 0.000000 -8162> 2014-08-13 17:22:48.919575 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8161> 2014-08-13 17:22:48.919589 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194447/194448/194448) [4,301,324] r=0 lpr=194449 pi=193306-194447/18 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset -8160> 2014-08-13 17:22:48.919565 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000936 15 0.000442 -8159> 2014-08-13 17:22:48.919602 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a84a40 con 0x4903700 -8158> 2014-08-13 17:22:48.919637 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8157> 2014-08-13 17:22:48.919629 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started -8156> 2014-08-13 17:22:48.919658 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c772c0 con 0x4903440 -8155> 2014-08-13 17:22:48.919664 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start -8154> 2014-08-13 17:22:48.919678 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8153> 2014-08-13 17:22:48.919681 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8152> 2014-08-13 17:22:48.919694 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4dc8000 con 0x4903180 -8151> 2014-08-13 17:22:48.919714 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8150> 2014-08-13 17:22:48.919704 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000040 0 0.000000 -8149> 2014-08-13 17:22:48.919733 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c77740 con 0x4904780 -8148> 2014-08-13 17:22:48.919737 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8147> 2014-08-13 17:22:48.919752 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8146> 2014-08-13 17:22:48.919754 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8145> 2014-08-13 17:22:48.919766 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c77080 con 0x49044c0 -8144> 2014-08-13 17:22:48.919784 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8143> 2014-08-13 17:22:48.919777 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8142> 2014-08-13 17:22:48.919801 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c77500 con 0x49070c0 -8141> 2014-08-13 17:22:48.919820 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8140> 2014-08-13 17:22:48.919833 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e18000 con 0x4906e00 -8139> 2014-08-13 17:22:48.919851 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8138> 2014-08-13 17:22:48.919865 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 1 ==== osd_map(194371..194470 src has 185139..194470) v3 ==== 88939+0+0 (3309911880 0 0) 0x4e10fc0 con 0x4906b40 -8137> 2014-08-13 17:22:48.919883 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194371,194470], i have 194470, src has [185139,194470] -8136> 2014-08-13 17:22:48.919930 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de0000 con 0x4906880 -8135> 2014-08-13 17:22:48.919950 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8134> 2014-08-13 17:22:48.919964 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e18480 con 0x4d58840 -8133> 2014-08-13 17:22:48.919983 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8132> 2014-08-13 17:22:48.919982 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8131> 2014-08-13 17:22:48.920002 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 1 ==== osd_map(194452..194470 src has 185139..194470) v3 ==== 12164+0+0 (339452127 0 0) 0x4e18900 con 0x4d58580 -8130> 2014-08-13 17:22:48.920018 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194452,194470], i have 194470, src has [185139,194470] -8129> 2014-08-13 17:22:48.920019 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.217636 2 0.000489 -8128> 2014-08-13 17:22:48.920033 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e18240 con 0x4d582c0 -8127> 2014-08-13 17:22:48.920051 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8126> 2014-08-13 17:22:48.920044 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.217681 0 0.000000 -8125> 2014-08-13 17:22:48.920065 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e11680 con 0x4d58000 -8124> 2014-08-13 17:22:48.920065 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.217721 0 0.000000 -8123> 2014-08-13 17:22:48.920058 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000469 15 0.000307 -8122> 2014-08-13 17:22:48.920083 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8121> 2014-08-13 17:22:48.920088 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started 0.218127 0 0.000000 -8120> 2014-08-13 17:22:48.920101 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started -8119> 2014-08-13 17:22:48.920119 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start -8118> 2014-08-13 17:22:48.920119 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e31f80 con 0x4d51ce0 -8117> 2014-08-13 17:22:48.920119 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/194448) [4,28,304] r=0 lpr=194449 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset -8116> 2014-08-13 17:22:48.920151 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8115> 2014-08-13 17:22:48.920143 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8114> 2014-08-13 17:22:48.920164 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e11440 con 0x4d51a20 -8113> 2014-08-13 17:22:48.920175 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000057 0 0.000000 -8112> 2014-08-13 17:22:48.920190 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8111> 2014-08-13 17:22:48.920202 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8110> 2014-08-13 17:22:48.920218 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e35e80 con 0x4d54200 -8109> 2014-08-13 17:22:48.920222 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8108> 2014-08-13 17:22:48.920241 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8107> 2014-08-13 17:22:48.920240 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194443/194389 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8106> 2014-08-13 17:22:48.920256 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e36e40 con 0x4d53c80 -8105> 2014-08-13 17:22:48.920274 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8104> 2014-08-13 17:22:48.920290 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e37500 con 0x4d56040 -8103> 2014-08-13 17:22:48.920308 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8102> 2014-08-13 17:22:48.920323 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e37740 con 0x4d55d80 -8101> 2014-08-13 17:22:48.920340 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8100> 2014-08-13 17:22:48.920356 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de1d40 con 0x4d55ac0 -8099> 2014-08-13 17:22:48.920374 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8098> 2014-08-13 17:22:48.920388 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de3a80 con 0x4d55540 -8097> 2014-08-13 17:22:48.920407 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8096> 2014-08-13 17:22:48.920425 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de1f80 con 0x4d57d20 -8095> 2014-08-13 17:22:48.920443 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8094> 2014-08-13 17:22:48.920427 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8093> 2014-08-13 17:22:48.920457 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de4800 con 0x4d57a60 -8092> 2014-08-13 17:22:48.920455 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.217369 2 0.000184 -8091> 2014-08-13 17:22:48.920479 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -8090> 2014-08-13 17:22:48.920493 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.217420 0 0.000000 -8089> 2014-08-13 17:22:48.920503 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 3 ==== pg_notify(24.30b(24) epoch 194470) v5 ==== 2096+0+0 (1338007029 0 0) 0x466eac0 con 0x45f1760 -8088> 2014-08-13 17:22:48.920523 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.795908, event: header_read, request: pg_notify(24.30b(24) epoch 194470) v5 -8087> 2014-08-13 17:22:48.920518 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.217456 0 0.000000 -8086> 2014-08-13 17:22:48.920538 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.795910, event: throttled, request: pg_notify(24.30b(24) epoch 194470) v5 -8085> 2014-08-13 17:22:48.920548 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.796346, event: all_read, request: pg_notify(24.30b(24) epoch 194470) v5 -8084> 2014-08-13 17:22:48.920540 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started 0.217563 0 0.000000 -8083> 2014-08-13 17:22:48.920561 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.920520, event: dispatched, request: pg_notify(24.30b(24) epoch 194470) v5 -8082> 2014-08-13 17:22:48.920571 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.920571, event: waiting_for_osdmap, request: pg_notify(24.30b(24) epoch 194470) v5 -8081> 2014-08-13 17:22:48.920572 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,65,62] r=0 lpr=194449 pi=193306-194447/18 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset -8080> 2014-08-13 17:22:48.920585 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.920584, event: started, request: pg_notify(24.30b(24) epoch 194470) v5 -8079> 2014-08-13 17:22:48.920613 7f74d8735700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:22:48.920613, event: done, request: pg_notify(24.30b(24) epoch 194470) v5 -8078> 2014-08-13 17:22:48.920632 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 2 ==== pg_query(22.14a epoch 194470) v3 ==== 144+0+0 (3179346447 0 0) 0x29c0f00 con 0x45f1e40 -8077> 2014-08-13 17:22:48.920631 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000511 15 0.000304 -8076> 2014-08-13 17:22:48.920651 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.799277, event: header_read, request: pg_query(22.14a epoch 194470) v3 -8075> 2014-08-13 17:22:48.920661 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.799278, event: throttled, request: pg_query(22.14a epoch 194470) v3 -8074> 2014-08-13 17:22:48.920670 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.799323, event: all_read, request: pg_query(22.14a epoch 194470) v3 -8073> 2014-08-13 17:22:48.920663 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started -8072> 2014-08-13 17:22:48.920684 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.920649, event: dispatched, request: pg_query(22.14a epoch 194470) v3 -8071> 2014-08-13 17:22:48.920703 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.920703, event: waiting_for_osdmap, request: pg_query(22.14a epoch 194470) v3 -8070> 2014-08-13 17:22:48.920697 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start -8069> 2014-08-13 17:22:48.920716 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.920716, event: started, request: pg_query(22.14a epoch 194470) v3 -8068> 2014-08-13 17:22:48.920715 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8067> 2014-08-13 17:22:48.920746 7f74d8735700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:22:48.920746, event: done, request: pg_query(22.14a epoch 194470) v3 -8066> 2014-08-13 17:22:48.920738 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000041 0 0.000000 -8065> 2014-08-13 17:22:48.920767 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 2 ==== pg_query(22.28f epoch 194470) v3 ==== 144+0+0 (706408228 0 0) 0x29c1680 con 0x45f2940 -8064> 2014-08-13 17:22:48.920771 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8063> 2014-08-13 17:22:48.920786 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.799637, event: header_read, request: pg_query(22.28f epoch 194470) v3 -8062> 2014-08-13 17:22:48.920797 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.799638, event: throttled, request: pg_query(22.28f epoch 194470) v3 -8061> 2014-08-13 17:22:48.920789 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8060> 2014-08-13 17:22:48.920809 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.799679, event: all_read, request: pg_query(22.28f epoch 194470) v3 -8059> 2014-08-13 17:22:48.920817 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.920783, event: dispatched, request: pg_query(22.28f epoch 194470) v3 -8058> 2014-08-13 17:22:48.920827 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.920827, event: waiting_for_osdmap, request: pg_query(22.28f epoch 194470) v3 -8057> 2014-08-13 17:22:48.920818 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8056> 2014-08-13 17:22:48.920841 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.920841, event: started, request: pg_query(22.28f epoch 194470) v3 -8055> 2014-08-13 17:22:48.920856 7f74d8735700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:22:48.920856, event: done, request: pg_query(22.28f epoch 194470) v3 -8054> 2014-08-13 17:22:48.920872 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 2 ==== pg_notify(22.84(2) epoch 194470) v5 ==== 828+0+0 (2765253268 0 0) 0x4ac3b80 con 0x45f3b20 -8053> 2014-08-13 17:22:48.920891 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.808416, event: header_read, request: pg_notify(22.84(2) epoch 194470) v5 -8052> 2014-08-13 17:22:48.920901 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.808417, event: throttled, request: pg_notify(22.84(2) epoch 194470) v5 -8051> 2014-08-13 17:22:48.920910 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.808457, event: all_read, request: pg_notify(22.84(2) epoch 194470) v5 -8050> 2014-08-13 17:22:48.920919 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.920889, event: dispatched, request: pg_notify(22.84(2) epoch 194470) v5 -8049> 2014-08-13 17:22:48.920928 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.920928, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194470) v5 -8048> 2014-08-13 17:22:48.920939 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.920939, event: started, request: pg_notify(22.84(2) epoch 194470) v5 -8047> 2014-08-13 17:22:48.920964 7f74d8735700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:22:48.920964, event: done, request: pg_notify(22.84(2) epoch 194470) v5 -8046> 2014-08-13 17:22:48.920980 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 2 ==== pg_notify(24.30b(1) epoch 194470) v5 ==== 811+0+0 (3328686156 0 0) 0x466ee40 con 0x45f5540 -8045> 2014-08-13 17:22:48.921001 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.797711, event: header_read, request: pg_notify(24.30b(1) epoch 194470) v5 -8044> 2014-08-13 17:22:48.920985 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -8043> 2014-08-13 17:22:48.921015 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.797712, event: throttled, request: pg_notify(24.30b(1) epoch 194470) v5 -8042> 2014-08-13 17:22:48.921029 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.797809, event: all_read, request: pg_notify(24.30b(1) epoch 194470) v5 -8041> 2014-08-13 17:22:48.921039 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.920998, event: dispatched, request: pg_notify(24.30b(1) epoch 194470) v5 -8040> 2014-08-13 17:22:48.921026 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.218274 2 0.000202 -8039> 2014-08-13 17:22:48.921052 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.921052, event: waiting_for_osdmap, request: pg_notify(24.30b(1) epoch 194470) v5 -8038> 2014-08-13 17:22:48.921035 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000464 15 0.000322 -8037> 2014-08-13 17:22:48.921061 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.921060, event: started, request: pg_notify(24.30b(1) epoch 194470) v5 -8036> 2014-08-13 17:22:48.921062 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.218333 0 0.000000 -8035> 2014-08-13 17:22:48.921065 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started -8034> 2014-08-13 17:22:48.921088 7f74d8735700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:22:48.921087, event: done, request: pg_notify(24.30b(1) epoch 194470) v5 -8033> 2014-08-13 17:22:48.921091 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.218393 0 0.000000 -8032> 2014-08-13 17:22:48.921103 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 2 ==== pg_query(24.350 epoch 194470) v3 ==== 144+0+0 (3622305021 0 0) 0x29c12c0 con 0x45f5960 -8031> 2014-08-13 17:22:48.921097 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start -8030> 2014-08-13 17:22:48.921120 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.799418, event: header_read, request: pg_query(24.350 epoch 194470) v3 -8029> 2014-08-13 17:22:48.921127 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.799419, event: throttled, request: pg_query(24.350 epoch 194470) v3 -8028> 2014-08-13 17:22:48.921120 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started 0.218479 0 0.000000 -8027> 2014-08-13 17:22:48.921136 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.799446, event: all_read, request: pg_query(24.350 epoch 194470) v3 -8026> 2014-08-13 17:22:48.921125 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8025> 2014-08-13 17:22:48.921148 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.921118, event: dispatched, request: pg_query(24.350 epoch 194470) v3 -8024> 2014-08-13 17:22:48.921170 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000074 0 0.000000 -8023> 2014-08-13 17:22:48.921185 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.921185, event: waiting_for_osdmap, request: pg_query(24.350 epoch 194470) v3 -8022> 2014-08-13 17:22:48.921195 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8021> 2014-08-13 17:22:48.921209 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.921209, event: started, request: pg_query(24.350 epoch 194470) v3 -8020> 2014-08-13 17:22:48.921215 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8019> 2014-08-13 17:22:48.921233 7f74d8735700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:22:48.921233, event: done, request: pg_query(24.350 epoch 194470) v3 -8018> 2014-08-13 17:22:48.921235 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8017> 2014-08-13 17:22:48.921257 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 2 ==== pg_query(22.30f epoch 194470) v3 ==== 144+0+0 (230757310 0 0) 0x29c5640 con 0x45f6f60 -8016> 2014-08-13 17:22:48.921283 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.818931, event: header_read, request: pg_query(22.30f epoch 194470) v3 -8015> 2014-08-13 17:22:48.921293 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.818933, event: throttled, request: pg_query(22.30f epoch 194470) v3 -8014> 2014-08-13 17:22:48.921303 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.818984, event: all_read, request: pg_query(22.30f epoch 194470) v3 -8013> 2014-08-13 17:22:48.921312 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.921279, event: dispatched, request: pg_query(22.30f epoch 194470) v3 -8012> 2014-08-13 17:22:48.921322 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.921322, event: waiting_for_osdmap, request: pg_query(22.30f epoch 194470) v3 -8011> 2014-08-13 17:22:48.921333 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.921333, event: started, request: pg_query(22.30f epoch 194470) v3 -8010> 2014-08-13 17:22:48.921358 7f74d8735700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:22:48.921357, event: done, request: pg_query(22.30f epoch 194470) v3 -8009> 2014-08-13 17:22:48.921375 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 2 ==== pg_query(22.2b9 epoch 194470) v3 ==== 144+0+0 (2875710442 0 0) 0x29c4b00 con 0x4758b00 -8008> 2014-08-13 17:22:48.921394 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.811875, event: header_read, request: pg_query(22.2b9 epoch 194470) v3 -8007> 2014-08-13 17:22:48.921403 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.811877, event: throttled, request: pg_query(22.2b9 epoch 194470) v3 -8006> 2014-08-13 17:22:48.921413 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.811934, event: all_read, request: pg_query(22.2b9 epoch 194470) v3 -8005> 2014-08-13 17:22:48.921404 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.217864 3 0.000199 -8004> 2014-08-13 17:22:48.921424 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.921391, event: dispatched, request: pg_query(22.2b9 epoch 194470) v3 -8003> 2014-08-13 17:22:48.921430 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.921430, event: waiting_for_osdmap, request: pg_query(22.2b9 epoch 194470) v3 -8002> 2014-08-13 17:22:48.921432 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started 0.217967 0 0.000000 -8001> 2014-08-13 17:22:48.921463 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.921462, event: started, request: pg_query(22.2b9 epoch 194470) v3 -8000> 2014-08-13 17:22:48.921482 7f74d8735700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:22:48.921482, event: done, request: pg_query(22.2b9 epoch 194470) v3 -7999> 2014-08-13 17:22:48.921472 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194447/194448/192936) [54,331,4] r=2 lpr=194449 pi=192980-194447/16 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset -7998> 2014-08-13 17:22:48.921503 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 2 ==== pg_notify(24.3d(1) epoch 194470) v5 ==== 811+0+0 (1680853822 0 0) 0x466f540 con 0x4758840 -7997> 2014-08-13 17:22:48.921523 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.799465, event: header_read, request: pg_notify(24.3d(1) epoch 194470) v5 -7996> 2014-08-13 17:22:48.921534 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.799466, event: throttled, request: pg_notify(24.3d(1) epoch 194470) v5 -7995> 2014-08-13 17:22:48.921543 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.799504, event: all_read, request: pg_notify(24.3d(1) epoch 194470) v5 -7994> 2014-08-13 17:22:48.921552 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.921521, event: dispatched, request: pg_notify(24.3d(1) epoch 194470) v5 -7993> 2014-08-13 17:22:48.921562 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.921562, event: waiting_for_osdmap, request: pg_notify(24.3d(1) epoch 194470) v5 -7992> 2014-08-13 17:22:48.921574 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.921574, event: started, request: pg_notify(24.3d(1) epoch 194470) v5 -7991> 2014-08-13 17:22:48.921603 7f74d8735700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:22:48.921603, event: done, request: pg_notify(24.3d(1) epoch 194470) v5 -7990> 2014-08-13 17:22:48.921623 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 2 ==== pg_notify(24.9b(26) epoch 194470) v5 ==== 2214+0+0 (1309885511 0 0) 0x466fe00 con 0x47591e0 -7989> 2014-08-13 17:22:48.921642 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.799608, event: header_read, request: pg_notify(24.9b(26) epoch 194470) v5 -7988> 2014-08-13 17:22:48.921653 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.799636, event: throttled, request: pg_notify(24.9b(26) epoch 194470) v5 -7987> 2014-08-13 17:22:48.921662 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.799830, event: all_read, request: pg_notify(24.9b(26) epoch 194470) v5 -7986> 2014-08-13 17:22:48.921671 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.921640, event: dispatched, request: pg_notify(24.9b(26) epoch 194470) v5 -7985> 2014-08-13 17:22:48.921682 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.921682, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194470) v5 -7984> 2014-08-13 17:22:48.921695 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.921695, event: started, request: pg_notify(24.9b(26) epoch 194470) v5 -7983> 2014-08-13 17:22:48.921721 7f74d8735700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:22:48.921721, event: done, request: pg_notify(24.9b(26) epoch 194470) v5 -7982> 2014-08-13 17:22:48.921739 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 2 ==== pg_query(22.791 epoch 194470) v3 ==== 144+0+0 (3126349129 0 0) 0x29c2580 con 0x475a940 -7981> 2014-08-13 17:22:48.921757 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.803595, event: header_read, request: pg_query(22.791 epoch 194470) v3 -7980> 2014-08-13 17:22:48.921767 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.803597, event: throttled, request: pg_query(22.791 epoch 194470) v3 -7979> 2014-08-13 17:22:48.921777 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.803659, event: all_read, request: pg_query(22.791 epoch 194470) v3 -7978> 2014-08-13 17:22:48.921786 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.921755, event: dispatched, request: pg_query(22.791 epoch 194470) v3 -7977> 2014-08-13 17:22:48.921795 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.921795, event: waiting_for_osdmap, request: pg_query(22.791 epoch 194470) v3 -7976> 2014-08-13 17:22:48.921805 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.921805, event: started, request: pg_query(22.791 epoch 194470) v3 -7975> 2014-08-13 17:22:48.921825 7f74d8735700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:22:48.921824, event: done, request: pg_query(22.791 epoch 194470) v3 -7974> 2014-08-13 17:22:48.921840 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 2 ==== pg_query(24.1e6 epoch 194470) v3 ==== 144+0+0 (2752075871 0 0) 0x29c50a0 con 0x475cd00 -7973> 2014-08-13 17:22:48.921857 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.815193, event: header_read, request: pg_query(24.1e6 epoch 194470) v3 -7972> 2014-08-13 17:22:48.921867 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.815196, event: throttled, request: pg_query(24.1e6 epoch 194470) v3 -7971> 2014-08-13 17:22:48.921876 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.815251, event: all_read, request: pg_query(24.1e6 epoch 194470) v3 -7970> 2014-08-13 17:22:48.921867 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.000395 15 0.000200 -7969> 2014-08-13 17:22:48.921140 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,356,357] r=0 lpr=194449 pi=193306-194447/16 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset -7968> 2014-08-13 17:22:48.921885 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.921855, event: dispatched, request: pg_query(24.1e6 epoch 194470) v3 -7967> 2014-08-13 17:22:48.921901 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started -7966> 2014-08-13 17:22:48.921912 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.921911, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194470) v3 -7965> 2014-08-13 17:22:48.921921 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start -7964> 2014-08-13 17:22:48.921933 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.921933, event: started, request: pg_query(24.1e6 epoch 194470) v3 -7963> 2014-08-13 17:22:48.921940 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7962> 2014-08-13 17:22:48.921955 7f74d8735700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:22:48.921955, event: done, request: pg_query(24.1e6 epoch 194470) v3 -7961> 2014-08-13 17:22:48.921962 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -7960> 2014-08-13 17:22:48.921974 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 2 ==== pg_query(22.29a epoch 194470) v3 ==== 144+0+0 (2386145314 0 0) 0x29c10e0 con 0x475c4c0 -7959> 2014-08-13 17:22:48.921984 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194449/194449 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray -7958> 2014-08-13 17:22:48.922000 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.799412, event: header_read, request: pg_query(22.29a epoch 194470) v3 -7957> 2014-08-13 17:22:48.922011 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.799412, event: throttled, request: pg_query(22.29a epoch 194470) v3 -7956> 2014-08-13 17:22:48.922020 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.799442, event: all_read, request: pg_query(22.29a epoch 194470) v3 -7955> 2014-08-13 17:22:48.922029 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.921997, event: dispatched, request: pg_query(22.29a epoch 194470) v3 -7954> 2014-08-13 17:22:48.922039 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.922038, event: waiting_for_osdmap, request: pg_query(22.29a epoch 194470) v3 -7953> 2014-08-13 17:22:48.922051 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.922051, event: started, request: pg_query(22.29a epoch 194470) v3 -7952> 2014-08-13 17:22:48.922071 7f74d8735700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:22:48.922070, event: done, request: pg_query(22.29a epoch 194470) v3 -7951> 2014-08-13 17:22:48.922088 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 2 ==== pg_query(22.630 epoch 194470) v3 ==== 144+0+0 (3826960337 0 0) 0x29c4380 con 0x475d960 -7950> 2014-08-13 17:22:48.922106 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.809768, event: header_read, request: pg_query(22.630 epoch 194470) v3 -7949> 2014-08-13 17:22:48.922119 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.809769, event: throttled, request: pg_query(22.630 epoch 194470) v3 -7948> 2014-08-13 17:22:48.922128 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.809832, event: all_read, request: pg_query(22.630 epoch 194470) v3 -7947> 2014-08-13 17:22:48.922137 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.922104, event: dispatched, request: pg_query(22.630 epoch 194470) v3 -7946> 2014-08-13 17:22:48.922147 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.922146, event: waiting_for_osdmap, request: pg_query(22.630 epoch 194470) v3 -7945> 2014-08-13 17:22:48.922157 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.922157, event: started, request: pg_query(22.630 epoch 194470) v3 -7944> 2014-08-13 17:22:48.922149 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Peering, affected_by_map, going to Reset -7943> 2014-08-13 17:22:48.922187 7f74d8735700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:22:48.922187, event: done, request: pg_query(22.630 epoch 194470) v3 -7942> 2014-08-13 17:22:48.922192 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.218196 2 0.000200 -7941> 2014-08-13 17:22:48.922207 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 2 ==== pg_notify(22.707(1) epoch 194470) v5 ==== 777+0+0 (4211101445 0 0) 0x4ac3d40 con 0x475ee00 -7940> 2014-08-13 17:22:48.922228 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.808510, event: header_read, request: pg_notify(22.707(1) epoch 194470) v5 -7939> 2014-08-13 17:22:48.922219 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.218248 0 0.000000 -7938> 2014-08-13 17:22:48.922238 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.808511, event: throttled, request: pg_notify(22.707(1) epoch 194470) v5 -7937> 2014-08-13 17:22:48.922247 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.808552, event: all_read, request: pg_notify(22.707(1) epoch 194470) v5 -7936> 2014-08-13 17:22:48.922242 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.218295 0 0.000000 -7935> 2014-08-13 17:22:48.922256 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.922225, event: dispatched, request: pg_notify(22.707(1) epoch 194470) v5 -7934> 2014-08-13 17:22:48.922266 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.922266, event: waiting_for_osdmap, request: pg_notify(22.707(1) epoch 194470) v5 -7933> 2014-08-13 17:22:48.922262 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.218384 0 0.000000 -7932> 2014-08-13 17:22:48.922277 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.922277, event: started, request: pg_notify(22.707(1) epoch 194470) v5 -7931> 2014-08-13 17:22:48.922286 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194447/194448/194448) [4,378,367]/[4,291,350] r=0 lpr=194449 pi=193637-194447/19 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -7930> 2014-08-13 17:22:48.922302 7f74d8735700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:22:48.922301, event: done, request: pg_notify(22.707(1) epoch 194470) v5 -7929> 2014-08-13 17:22:48.922316 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 2 ==== pg_query(22.4aa epoch 194470) v3 ==== 144+0+0 (51553995 0 0) 0x29c14a0 con 0x475e5c0 -7928> 2014-08-13 17:22:48.922334 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.799473, event: header_read, request: pg_query(22.4aa epoch 194470) v3 -7927> 2014-08-13 17:22:48.922344 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.799474, event: throttled, request: pg_query(22.4aa epoch 194470) v3 -7926> 2014-08-13 17:22:48.922353 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.799509, event: all_read, request: pg_query(22.4aa epoch 194470) v3 -7925> 2014-08-13 17:22:48.922362 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.922332, event: dispatched, request: pg_query(22.4aa epoch 194470) v3 -7924> 2014-08-13 17:22:48.922371 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.922371, event: waiting_for_osdmap, request: pg_query(22.4aa epoch 194470) v3 -7923> 2014-08-13 17:22:48.922361 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001221 15 0.001075 -7922> 2014-08-13 17:22:48.922382 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.922382, event: started, request: pg_query(22.4aa epoch 194470) v3 -7921> 2014-08-13 17:22:48.922398 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started -7920> 2014-08-13 17:22:48.922415 7f74d8735700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:22:48.922415, event: done, request: pg_query(22.4aa epoch 194470) v3 -7919> 2014-08-13 17:22:48.922419 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start -7918> 2014-08-13 17:22:48.922430 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 2 ==== pg_query(24.5e epoch 194470) v3 ==== 144+0+0 (4277202955 0 0) 0x29c2b20 con 0x47a27e0 -7917> 2014-08-13 17:22:48.922437 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7916> 2014-08-13 17:22:48.922449 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.805525, event: header_read, request: pg_query(24.5e epoch 194470) v3 -7915> 2014-08-13 17:22:48.922468 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.805526, event: throttled, request: pg_query(24.5e epoch 194470) v3 -7914> 2014-08-13 17:22:48.922464 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000045 0 0.000000 -7913> 2014-08-13 17:22:48.922479 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.806683, event: all_read, request: pg_query(24.5e epoch 194470) v3 -7912> 2014-08-13 17:22:48.922488 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.922447, event: dispatched, request: pg_query(24.5e epoch 194470) v3 -7911> 2014-08-13 17:22:48.922484 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7910> 2014-08-13 17:22:48.922498 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.922498, event: waiting_for_osdmap, request: pg_query(24.5e epoch 194470) v3 -7909> 2014-08-13 17:22:48.922500 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7908> 2014-08-13 17:22:48.922509 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.922509, event: started, request: pg_query(24.5e epoch 194470) v3 -7907> 2014-08-13 17:22:48.922522 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7906> 2014-08-13 17:22:48.922539 7f74d8735700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:22:48.922539, event: done, request: pg_query(24.5e epoch 194470) v3 -7905> 2014-08-13 17:22:48.922554 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 2 ==== pg_notify(22.3af(1) epoch 194470) v5 ==== 777+0+0 (1401344078 0 0) 0x4ac4600 con 0x47a0b00 -7904> 2014-08-13 17:22:48.922573 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.809593, event: header_read, request: pg_notify(22.3af(1) epoch 194470) v5 -7903> 2014-08-13 17:22:48.922583 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.809594, event: throttled, request: pg_notify(22.3af(1) epoch 194470) v5 -7902> 2014-08-13 17:22:48.922592 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.809631, event: all_read, request: pg_notify(22.3af(1) epoch 194470) v5 -7901> 2014-08-13 17:22:48.922601 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.922571, event: dispatched, request: pg_notify(22.3af(1) epoch 194470) v5 -7900> 2014-08-13 17:22:48.922611 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.922611, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194470) v5 -7899> 2014-08-13 17:22:48.922622 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.922621, event: started, request: pg_notify(22.3af(1) epoch 194470) v5 -7898> 2014-08-13 17:22:48.922646 7f74d8735700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:22:48.922646, event: done, request: pg_notify(22.3af(1) epoch 194470) v5 -7897> 2014-08-13 17:22:48.922676 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 2 ==== pg_query(24.1de epoch 194470) v3 ==== 144+0+0 (3852449925 0 0) 0x29c5280 con 0x47a4200 -7896> 2014-08-13 17:22:48.922698 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.815219, event: header_read, request: pg_query(24.1de epoch 194470) v3 -7895> 2014-08-13 17:22:48.922709 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.815221, event: throttled, request: pg_query(24.1de epoch 194470) v3 -7894> 2014-08-13 17:22:48.922717 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.815312, event: all_read, request: pg_query(24.1de epoch 194470) v3 -7893> 2014-08-13 17:22:48.922706 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6838/62713 -- pg_query(22.84 epoch 194470) v3 -- ?+0 0x29c7da0 con 0x4d58580 -7892> 2014-08-13 17:22:48.922726 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.922696, event: dispatched, request: pg_query(24.1de epoch 194470) v3 -7891> 2014-08-13 17:22:48.922736 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.922736, event: waiting_for_osdmap, request: pg_query(24.1de epoch 194470) v3 -7890> 2014-08-13 17:22:48.922739 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_query(22.99 epoch 194470) v3 -- ?+0 0x29c7bc0 con 0x4758840 -7889> 2014-08-13 17:22:48.922748 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.922747, event: started, request: pg_query(24.1de epoch 194470) v3 -7888> 2014-08-13 17:22:48.922755 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6805/37391 -- pg_query(22.84 epoch 194470) v3 -- ?+0 0x29c79e0 con 0x45f3b20 -7887> 2014-08-13 17:22:48.922769 7f74d8735700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:22:48.922768, event: done, request: pg_query(24.1de epoch 194470) v3 -7886> 2014-08-13 17:22:48.922773 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_query(22.99 epoch 194470) v3 -- ?+0 0x29c7800 con 0x47f4620 -7885> 2014-08-13 17:22:48.922761 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.000476 15 0.000293 -7884> 2014-08-13 17:22:48.922785 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 2 ==== pg_query(24.2a1 epoch 194470) v3 ==== 144+0+0 (991886390 0 0) 0x29c4740 con 0x47a53e0 -7883> 2014-08-13 17:22:48.922801 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.811516, event: header_read, request: pg_query(24.2a1 epoch 194470) v3 -7882> 2014-08-13 17:22:48.922791 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_query(22.84 epoch 194470) v3 -- ?+0 0x29c7620 con 0x47f0c60 -7881> 2014-08-13 17:22:48.922799 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started -7880> 2014-08-13 17:22:48.922812 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.811517, event: throttled, request: pg_query(24.2a1 epoch 194470) v3 -7879> 2014-08-13 17:22:48.922829 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Start -7878> 2014-08-13 17:22:48.922840 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.811550, event: all_read, request: pg_query(24.2a1 epoch 194470) v3 -7877> 2014-08-13 17:22:48.922853 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.922799, event: dispatched, request: pg_query(24.2a1 epoch 194470) v3 -7876> 2014-08-13 17:22:48.922846 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary -7875> 2014-08-13 17:22:48.922869 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.922869, event: waiting_for_osdmap, request: pg_query(24.2a1 epoch 194470) v3 -7874> 2014-08-13 17:22:48.922858 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_query(22.99 epoch 194470) v3 -- ?+0 0x4eb0b40 con 0x4833440 -7873> 2014-08-13 17:22:48.922877 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.922877, event: started, request: pg_query(24.2a1 epoch 194470) v3 -7872> 2014-08-13 17:22:48.922878 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000049 0 0.000000 -7871> 2014-08-13 17:22:48.922893 7f74d8735700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:22:48.922892, event: done, request: pg_query(24.2a1 epoch 194470) v3 -7870> 2014-08-13 17:22:48.922906 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -7869> 2014-08-13 17:22:48.922891 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6826/4273 -- pg_query(22.4df epoch 194470) v3 -- ?+0 0x4eb0960 con 0x4d57d20 -7868> 2014-08-13 17:22:48.922926 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 2 ==== pg_notify(22.1b8(2) epoch 194470) v5 ==== 828+0+0 (1631333252 0 0) 0x4ac01c0 con 0x47a3f40 -7867> 2014-08-13 17:22:48.922936 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_query(22.4df epoch 194470) v3 -- ?+0 0x4eb0780 con 0x45f5960 -7866> 2014-08-13 17:22:48.922928 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering -7865> 2014-08-13 17:22:48.922950 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.799848, event: header_read, request: pg_notify(22.1b8(2) epoch 194470) v5 -7864> 2014-08-13 17:22:48.922958 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.799849, event: throttled, request: pg_notify(22.1b8(2) epoch 194470) v5 -7863> 2014-08-13 17:22:48.922956 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6834/17224 -- pg_query(22.4df epoch 194470) v3 -- ?+0 0x4eb05a0 con 0x4d37a60 -7862> 2014-08-13 17:22:48.922970 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.801768, event: all_read, request: pg_notify(22.1b8(2) epoch 194470) v5 -7861> 2014-08-13 17:22:48.922958 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194443/194389 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo -7860> 2014-08-13 17:22:48.922986 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.922948, event: dispatched, request: pg_notify(22.1b8(2) epoch 194470) v5 -7859> 2014-08-13 17:22:48.922992 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6842/20017 -- pg_query(22.99 epoch 194470) v3 -- ?+0 0x4eb03c0 con 0x4d582c0 -7858> 2014-08-13 17:22:48.922996 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.922996, event: waiting_for_osdmap, request: pg_notify(22.1b8(2) epoch 194470) v5 -7857> 2014-08-13 17:22:48.923021 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.923021, event: started, request: pg_notify(22.1b8(2) epoch 194470) v5 -7856> 2014-08-13 17:22:48.923049 7f74d8735700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:22:48.923049, event: done, request: pg_notify(22.1b8(2) epoch 194470) v5 -7855> 2014-08-13 17:22:48.923043 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6832/16753 -- pg_query(22.4df epoch 194470) v3 -- ?+0 0x4eb01e0 con 0x4832c00 -7854> 2014-08-13 17:22:48.923071 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 2 ==== pg_notify(22.619(1) epoch 194470) v5 ==== 777+0+0 (37238582 0 0) 0x4ac2140 con 0x47a4ba0 -7853> 2014-08-13 17:22:48.923091 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.805687, event: header_read, request: pg_notify(22.619(1) epoch 194470) v5 -7852> 2014-08-13 17:22:48.923101 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.805688, event: throttled, request: pg_notify(22.619(1) epoch 194470) v5 -7851> 2014-08-13 17:22:48.923111 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.805727, event: all_read, request: pg_notify(22.619(1) epoch 194470) v5 -7850> 2014-08-13 17:22:48.923119 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.923089, event: dispatched, request: pg_notify(22.619(1) epoch 194470) v5 -7849> 2014-08-13 17:22:48.923129 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.923129, event: waiting_for_osdmap, request: pg_notify(22.619(1) epoch 194470) v5 -7848> 2014-08-13 17:22:48.923140 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.923140, event: started, request: pg_notify(22.619(1) epoch 194470) v5 -7847> 2014-08-13 17:22:48.923140 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.218752 3 0.000125 -7846> 2014-08-13 17:22:48.923171 7f74d8735700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:22:48.923171, event: done, request: pg_notify(22.619(1) epoch 194470) v5 -7845> 2014-08-13 17:22:48.923181 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started 0.218846 0 0.000000 -7844> 2014-08-13 17:22:48.923188 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 2 ==== pg_notify(22.3e4(22) epoch 194470) v5 ==== 1948+0+0 (2374870074 0 0) 0x4ac3800 con 0x47a32e0 -7843> 2014-08-13 17:22:48.923204 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.807158, event: header_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -7842> 2014-08-13 17:22:48.923204 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194447/194448/192979) [320,4,30] r=1 lpr=194449 pi=192979-194447/19 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset -7841> 2014-08-13 17:22:48.923215 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.807159, event: throttled, request: pg_notify(22.3e4(22) epoch 194470) v5 -7840> 2014-08-13 17:22:48.923225 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.807222, event: all_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -7839> 2014-08-13 17:22:48.923239 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.923202, event: dispatched, request: pg_notify(22.3e4(22) epoch 194470) v5 -7838> 2014-08-13 17:22:48.923249 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.923248, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194470) v5 -7837> 2014-08-13 17:22:48.923259 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.923259, event: started, request: pg_notify(22.3e4(22) epoch 194470) v5 -7836> 2014-08-13 17:22:48.923294 7f74d8735700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:22:48.923294, event: done, request: pg_notify(22.3e4(22) epoch 194470) v5 -7835> 2014-08-13 17:22:48.923293 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_query(22.99 epoch 194470) v3 -- ?+0 0x4eb0000 con 0x486b9c0 -7834> 2014-08-13 17:22:48.923314 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 2 ==== pg_query(24.33a epoch 194470) v3 ==== 144+0+0 (562130618 0 0) 0x29c2d00 con 0x47f0c60 -7833> 2014-08-13 17:22:48.923331 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6822/51844 -- pg_query(22.4df epoch 194470) v3 -- ?+0 0x4eb1680 con 0x4d56040 -7832> 2014-08-13 17:22:48.923335 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.805599, event: header_read, request: pg_query(24.33a epoch 194470) v3 -7831> 2014-08-13 17:22:48.923358 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.805600, event: throttled, request: pg_query(24.33a epoch 194470) v3 -7830> 2014-08-13 17:22:48.923368 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.805638, event: all_read, request: pg_query(24.33a epoch 194470) v3 -7829> 2014-08-13 17:22:48.923377 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.923333, event: dispatched, request: pg_query(24.33a epoch 194470) v3 -7828> 2014-08-13 17:22:48.923383 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.923383, event: waiting_for_osdmap, request: pg_query(24.33a epoch 194470) v3 -7827> 2014-08-13 17:22:48.923391 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.923390, event: started, request: pg_query(24.33a epoch 194470) v3 -7826> 2014-08-13 17:22:48.923407 7f74d8735700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:22:48.923407, event: done, request: pg_query(24.33a epoch 194470) v3 -7825> 2014-08-13 17:22:48.923409 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7824> 2014-08-13 17:22:48.923423 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 2 ==== pg_query(22.2d1 epoch 194470) v3 ==== 144+0+0 (3303696121 0 0) 0x29c5a00 con 0x47f2680 -7823> 2014-08-13 17:22:48.923437 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.821494, event: header_read, request: pg_query(22.2d1 epoch 194470) v3 -7822> 2014-08-13 17:22:48.923444 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.821495, event: throttled, request: pg_query(22.2d1 epoch 194470) v3 -7821> 2014-08-13 17:22:48.923449 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.821561, event: all_read, request: pg_query(22.2d1 epoch 194470) v3 -7820> 2014-08-13 17:22:48.923437 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.220217 2 0.000164 -7819> 2014-08-13 17:22:48.923454 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.923435, event: dispatched, request: pg_query(22.2d1 epoch 194470) v3 -7818> 2014-08-13 17:22:48.923460 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.923460, event: waiting_for_osdmap, request: pg_query(22.2d1 epoch 194470) v3 -7817> 2014-08-13 17:22:48.923467 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.923467, event: started, request: pg_query(22.2d1 epoch 194470) v3 -7816> 2014-08-13 17:22:48.923481 7f74d8735700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:22:48.923481, event: done, request: pg_query(22.2d1 epoch 194470) v3 -7815> 2014-08-13 17:22:48.923459 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.220253 0 0.000000 -7814> 2014-08-13 17:22:48.923499 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 2 ==== pg_query(24.1e4 epoch 194470) v3 ==== 144+0+0 (4230223656 0 0) 0x29c2760 con 0x47f4620 -7813> 2014-08-13 17:22:48.923518 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.804360, event: header_read, request: pg_query(24.1e4 epoch 194470) v3 -7812> 2014-08-13 17:22:48.923530 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.804362, event: throttled, request: pg_query(24.1e4 epoch 194470) v3 -7811> 2014-08-13 17:22:48.923539 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.804425, event: all_read, request: pg_query(24.1e4 epoch 194470) v3 -7810> 2014-08-13 17:22:48.923549 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.923516, event: dispatched, request: pg_query(24.1e4 epoch 194470) v3 -7809> 2014-08-13 17:22:48.923537 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.220344 0 0.000000 -7808> 2014-08-13 17:22:48.923559 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.923559, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194470) v3 -7807> 2014-08-13 17:22:48.923570 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.923570, event: started, request: pg_query(24.1e4 epoch 194470) v3 -7806> 2014-08-13 17:22:48.923560 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started 0.220453 0 0.000000 -7805> 2014-08-13 17:22:48.923588 7f74d8735700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:22:48.923588, event: done, request: pg_query(24.1e4 epoch 194470) v3 -7804> 2014-08-13 17:22:48.923583 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194447/194448/194448) [4,373,344] r=0 lpr=194449 pi=193306-194447/16 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset -7803> 2014-08-13 17:22:48.923591 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.000388 15 0.000176 -7802> 2014-08-13 17:22:48.923611 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 2 ==== pg_notify(24.b6(27) epoch 194470) v5 ==== 2273+0+0 (3499909850 0 0) 0x4ac0380 con 0x47f6880 -7801> 2014-08-13 17:22:48.923630 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.801749, event: header_read, request: pg_notify(24.b6(27) epoch 194470) v5 -7800> 2014-08-13 17:22:48.923622 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started -7799> 2014-08-13 17:22:48.923646 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.801750, event: throttled, request: pg_notify(24.b6(27) epoch 194470) v5 -7798> 2014-08-13 17:22:48.923657 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.801886, event: all_read, request: pg_notify(24.b6(27) epoch 194470) v5 -7797> 2014-08-13 17:22:48.923649 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start -7796> 2014-08-13 17:22:48.923670 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.923628, event: dispatched, request: pg_notify(24.b6(27) epoch 194470) v5 -7795> 2014-08-13 17:22:48.923681 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7794> 2014-08-13 17:22:48.923692 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.923692, event: waiting_for_osdmap, request: pg_notify(24.b6(27) epoch 194470) v5 -7793> 2014-08-13 17:22:48.923707 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.923707, event: started, request: pg_notify(24.b6(27) epoch 194470) v5 -7792> 2014-08-13 17:22:48.923703 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000055 0 0.000000 -7791> 2014-08-13 17:22:48.923722 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194449/194389 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray -7790> 2014-08-13 17:22:48.923734 7f74d8735700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:22:48.923733, event: done, request: pg_notify(24.b6(27) epoch 194470) v5 -7789> 2014-08-13 17:22:48.923751 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 2 ==== pg_notify(22.113(22) epoch 194470) v5 ==== 1952+0+0 (2442813993 0 0) 0x4ac7700 con 0x47f7e80 -7788> 2014-08-13 17:22:48.923768 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.818917, event: header_read, request: pg_notify(22.113(22) epoch 194470) v5 -7787> 2014-08-13 17:22:48.923774 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.818920, event: throttled, request: pg_notify(22.113(22) epoch 194470) v5 -7786> 2014-08-13 17:22:48.923780 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.819046, event: all_read, request: pg_notify(22.113(22) epoch 194470) v5 -7785> 2014-08-13 17:22:48.923785 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.923766, event: dispatched, request: pg_notify(22.113(22) epoch 194470) v5 -7784> 2014-08-13 17:22:48.923791 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.923791, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194470) v5 -7783> 2014-08-13 17:22:48.923798 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.923798, event: started, request: pg_notify(22.113(22) epoch 194470) v5 -7782> 2014-08-13 17:22:48.923816 7f74d8735700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:22:48.923816, event: done, request: pg_notify(22.113(22) epoch 194470) v5 -7781> 2014-08-13 17:22:48.923861 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 2 ==== pg_notify(24.136(1) epoch 194470) v5 ==== 811+0+0 (1105244236 0 0) 0x466f1c0 con 0x47f70c0 -7780> 2014-08-13 17:22:48.923880 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.799340, event: header_read, request: pg_notify(24.136(1) epoch 194470) v5 -7779> 2014-08-13 17:22:48.923912 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.799341, event: throttled, request: pg_notify(24.136(1) epoch 194470) v5 -7778> 2014-08-13 17:22:48.923921 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.799381, event: all_read, request: pg_notify(24.136(1) epoch 194470) v5 -7777> 2014-08-13 17:22:48.923932 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.923878, event: dispatched, request: pg_notify(24.136(1) epoch 194470) v5 -7776> 2014-08-13 17:22:48.923944 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.923944, event: waiting_for_osdmap, request: pg_notify(24.136(1) epoch 194470) v5 -7775> 2014-08-13 17:22:48.923952 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.923952, event: started, request: pg_notify(24.136(1) epoch 194470) v5 -7774> 2014-08-13 17:22:48.923973 7f74d8735700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:22:48.923973, event: done, request: pg_notify(24.136(1) epoch 194470) v5 -7773> 2014-08-13 17:22:48.923994 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 2 ==== pg_query(22.537 epoch 194470) v3 ==== 144+0+0 (3778232190 0 0) 0x29c3c00 con 0x4831b80 -7772> 2014-08-13 17:22:48.924014 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.809452, event: header_read, request: pg_query(22.537 epoch 194470) v3 -7771> 2014-08-13 17:22:48.924025 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.809453, event: throttled, request: pg_query(22.537 epoch 194470) v3 -7770> 2014-08-13 17:22:48.924031 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.809482, event: all_read, request: pg_query(22.537 epoch 194470) v3 -7769> 2014-08-13 17:22:48.924039 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.924011, event: dispatched, request: pg_query(22.537 epoch 194470) v3 -7768> 2014-08-13 17:22:48.924050 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.924050, event: waiting_for_osdmap, request: pg_query(22.537 epoch 194470) v3 -7767> 2014-08-13 17:22:48.924062 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.924062, event: started, request: pg_query(22.537 epoch 194470) v3 -7766> 2014-08-13 17:22:48.924081 7f74d8735700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:22:48.924081, event: done, request: pg_query(22.537 epoch 194470) v3 -7765> 2014-08-13 17:22:48.924123 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 2 ==== pg_notify(22.99(18) epoch 194470) v5 ==== 1720+0+0 (3779630166 0 0) 0x49e0000 con 0x4833440 -7764> 2014-08-13 17:22:48.924147 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.799928, event: header_read, request: pg_notify(22.99(18) epoch 194470) v5 -7763> 2014-08-13 17:22:48.924157 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.799929, event: throttled, request: pg_notify(22.99(18) epoch 194470) v5 -7762> 2014-08-13 17:22:48.924173 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.801218, event: all_read, request: pg_notify(22.99(18) epoch 194470) v5 -7761> 2014-08-13 17:22:48.924183 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.924144, event: dispatched, request: pg_notify(22.99(18) epoch 194470) v5 -7760> 2014-08-13 17:22:48.924190 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.924190, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194470) v5 -7759> 2014-08-13 17:22:48.924201 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.924201, event: started, request: pg_notify(22.99(18) epoch 194470) v5 -7758> 2014-08-13 17:22:48.924181 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7757> 2014-08-13 17:22:48.924231 7f74d8735700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:22:48.924230, event: done, request: pg_notify(22.99(18) epoch 194470) v5 -7756> 2014-08-13 17:22:48.924227 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.219538 2 0.000147 -7755> 2014-08-13 17:22:48.924250 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 2 ==== pg_notify(22.1b8(20) epoch 194470) v5 ==== 1838+0+0 (1499644251 0 0) 0x4ac1c00 con 0x4831600 -7754> 2014-08-13 17:22:48.924256 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.219581 0 0.000000 -7753> 2014-08-13 17:22:48.924270 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.805608, event: header_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -7752> 2014-08-13 17:22:48.924279 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.805609, event: throttled, request: pg_notify(22.1b8(20) epoch 194470) v5 -7751> 2014-08-13 17:22:48.924289 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.805777, event: all_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -7750> 2014-08-13 17:22:48.924282 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.219638 0 0.000000 -7749> 2014-08-13 17:22:48.924298 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.924268, event: dispatched, request: pg_notify(22.1b8(20) epoch 194470) v5 -7748> 2014-08-13 17:22:48.924307 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.924307, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194470) v5 -7747> 2014-08-13 17:22:48.924302 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started 0.219706 0 0.000000 -7746> 2014-08-13 17:22:48.924319 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.924319, event: started, request: pg_notify(22.1b8(20) epoch 194470) v5 -7745> 2014-08-13 17:22:48.924321 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194447/194448/194448) [4,312,285] r=0 lpr=194449 pi=193306-194447/16 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset -7744> 2014-08-13 17:22:48.924343 7f74d8735700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:22:48.924342, event: done, request: pg_notify(22.1b8(20) epoch 194470) v5 -7743> 2014-08-13 17:22:48.924369 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 2 ==== pg_notify(22.4df(20) epoch 194470) v5 ==== 1834+0+0 (819295757 0 0) 0x4ac4280 con 0x4832c00 -7742> 2014-08-13 17:22:48.924388 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.809374, event: header_read, request: pg_notify(22.4df(20) epoch 194470) v5 -7741> 2014-08-13 17:22:48.924398 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.809376, event: throttled, request: pg_notify(22.4df(20) epoch 194470) v5 -7740> 2014-08-13 17:22:48.924407 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.809444, event: all_read, request: pg_notify(22.4df(20) epoch 194470) v5 -7739> 2014-08-13 17:22:48.924416 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.924386, event: dispatched, request: pg_notify(22.4df(20) epoch 194470) v5 -7738> 2014-08-13 17:22:48.924426 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.924425, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194470) v5 -7737> 2014-08-13 17:22:48.924438 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.924438, event: started, request: pg_notify(22.4df(20) epoch 194470) v5 -7736> 2014-08-13 17:22:48.924462 7f74d8735700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:22:48.924462, event: done, request: pg_notify(22.4df(20) epoch 194470) v5 -7735> 2014-08-13 17:22:48.924488 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 2 ==== pg_notify(22.113(22) epoch 194470) v5 ==== 1952+0+0 (2442813993 0 0) 0x4ac4b40 con 0x4830dc0 -7734> 2014-08-13 17:22:48.924507 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.809449, event: header_read, request: pg_notify(22.113(22) epoch 194470) v5 -7733> 2014-08-13 17:22:48.924517 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.809450, event: throttled, request: pg_notify(22.113(22) epoch 194470) v5 -7732> 2014-08-13 17:22:48.924527 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.810394, event: all_read, request: pg_notify(22.113(22) epoch 194470) v5 -7731> 2014-08-13 17:22:48.924536 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.924505, event: dispatched, request: pg_notify(22.113(22) epoch 194470) v5 -7730> 2014-08-13 17:22:48.924546 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.924545, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194470) v5 -7729> 2014-08-13 17:22:48.924556 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.924556, event: started, request: pg_notify(22.113(22) epoch 194470) v5 -7728> 2014-08-13 17:22:48.924579 7f74d8735700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:22:48.924579, event: done, request: pg_notify(22.113(22) epoch 194470) v5 -7727> 2014-08-13 17:22:48.924601 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 2 ==== pg_query(24.cd epoch 194470) v3 ==== 144+0+0 (2367309051 0 0) 0x29c0d20 con 0x4835ac0 -7726> 2014-08-13 17:22:48.924619 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.794978, event: header_read, request: pg_query(24.cd epoch 194470) v3 -7725> 2014-08-13 17:22:48.924628 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.794979, event: throttled, request: pg_query(24.cd epoch 194470) v3 -7724> 2014-08-13 17:22:48.924613 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001030 15 0.000510 -7723> 2014-08-13 17:22:48.924638 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.795009, event: all_read, request: pg_query(24.cd epoch 194470) v3 -7722> 2014-08-13 17:22:48.924643 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started -7721> 2014-08-13 17:22:48.924647 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.924617, event: dispatched, request: pg_query(24.cd epoch 194470) v3 -7720> 2014-08-13 17:22:48.924667 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start -7719> 2014-08-13 17:22:48.924675 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.924675, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194470) v3 -7718> 2014-08-13 17:22:48.924688 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7717> 2014-08-13 17:22:48.924700 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.924699, event: started, request: pg_query(24.cd epoch 194470) v3 -7716> 2014-08-13 17:22:48.924719 7f74d8735700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:22:48.924719, event: done, request: pg_query(24.cd epoch 194470) v3 -7715> 2014-08-13 17:22:48.924710 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000043 0 0.000000 -7714> 2014-08-13 17:22:48.924745 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.32 192.168.1.115:6812/59915 2 ==== pg_notify(22.6bd(2) epoch 194470) v5 ==== 828+0+0 (8849289 0 0) 0x4ac0000 con 0x4832100 -7713> 2014-08-13 17:22:48.924744 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7712> 2014-08-13 17:22:48.924768 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.799704, event: header_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -7711> 2014-08-13 17:22:48.924775 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.799705, event: throttled, request: pg_notify(22.6bd(2) epoch 194470) v5 -7710> 2014-08-13 17:22:48.924785 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.801690, event: all_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -7709> 2014-08-13 17:22:48.924777 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7708> 2014-08-13 17:22:48.924794 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.924766, event: dispatched, request: pg_notify(22.6bd(2) epoch 194470) v5 -7707> 2014-08-13 17:22:48.924804 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.924803, event: waiting_for_osdmap, request: pg_notify(22.6bd(2) epoch 194470) v5 -7706> 2014-08-13 17:22:48.924796 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194443/194389 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7705> 2014-08-13 17:22:48.924819 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.924818, event: started, request: pg_notify(22.6bd(2) epoch 194470) v5 -7704> 2014-08-13 17:22:48.924840 7f74d8735700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:22:48.924840, event: done, request: pg_notify(22.6bd(2) epoch 194470) v5 -7703> 2014-08-13 17:22:48.924859 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 2 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (3345820745 0 0) 0x4ac2840 con 0x4835280 -7702> 2014-08-13 17:22:48.924878 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.806008, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -7701> 2014-08-13 17:22:48.924864 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000544 15 0.000293 -7700> 2014-08-13 17:22:48.924888 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.806010, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -7699> 2014-08-13 17:22:48.924897 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.806117, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -7698> 2014-08-13 17:22:48.924893 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started -7697> 2014-08-13 17:22:48.924906 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.924876, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -7696> 2014-08-13 17:22:48.924916 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.924916, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -7695> 2014-08-13 17:22:48.924912 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start -7694> 2014-08-13 17:22:48.924927 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.924927, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -7693> 2014-08-13 17:22:48.924926 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7692> 2014-08-13 17:22:48.924952 7f74d8735700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:22:48.924952, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -7691> 2014-08-13 17:22:48.924951 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -7690> 2014-08-13 17:22:48.924950 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 -7689> 2014-08-13 17:22:48.924963 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_alive(want up_thru 194470 have 194470) v1 -- ?+0 0x57d5080 con 0x45d48e0 -7688> 2014-08-13 17:22:48.924970 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 2 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (2543075558 0 0) 0x49e7e00 con 0x4834620 -7687> 2014-08-13 17:22:48.924975 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7686> 2014-08-13 17:22:48.924985 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_query(22.39d epoch 194470) v3 -- ?+0 0x4eb12c0 con 0x486bc80 -7685> 2014-08-13 17:22:48.924990 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.829047, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -7684> 2014-08-13 17:22:48.924994 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7683> 2014-08-13 17:22:48.925009 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6812/6757 -- pg_query(22.39d epoch 194470) v3 -- ?+0 0x4eb10e0 con 0x45f6300 -7682> 2014-08-13 17:22:48.925015 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.829050, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -7681> 2014-08-13 17:22:48.925023 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.829258, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -7680> 2014-08-13 17:22:48.925029 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.924988, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -7679> 2014-08-13 17:22:48.925022 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194443/194389 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7678> 2014-08-13 17:22:48.925036 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.925035, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -7677> 2014-08-13 17:22:48.925044 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.925043, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -7676> 2014-08-13 17:22:48.925054 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6842/20017 -- pg_query(22.39d epoch 194470) v3 -- ?+0 0x4eb0f00 con 0x4d582c0 -7675> 2014-08-13 17:22:48.925062 7f74d8735700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:22:48.925062, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -7674> 2014-08-13 17:22:48.925079 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6828/4534 -- pg_query(22.39d epoch 194470) v3 -- ?+0 0x4eb0d20 con 0x4d53c80 -7673> 2014-08-13 17:22:48.925087 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 2 ==== pg_notify(22.3d5(2) epoch 194470) v5 ==== 828+0+0 (2506349106 0 0) 0x49e01c0 con 0x48377a0 -7672> 2014-08-13 17:22:48.925098 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_query(22.39d epoch 194470) v3 -- ?+0 0x29c50a0 con 0x486a3c0 -7671> 2014-08-13 17:22:48.925108 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.801159, event: header_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -7670> 2014-08-13 17:22:48.925118 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.801160, event: throttled, request: pg_notify(22.3d5(2) epoch 194470) v5 -7669> 2014-08-13 17:22:48.925127 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.803263, event: all_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -7668> 2014-08-13 17:22:48.925138 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.925106, event: dispatched, request: pg_notify(22.3d5(2) epoch 194470) v5 -7667> 2014-08-13 17:22:48.925149 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.925149, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194470) v5 -7666> 2014-08-13 17:22:48.925169 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.925168, event: started, request: pg_notify(22.3d5(2) epoch 194470) v5 -7665> 2014-08-13 17:22:48.925193 7f74d8735700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:22:48.925193, event: done, request: pg_notify(22.3d5(2) epoch 194470) v5 -7664> 2014-08-13 17:22:48.925198 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.220233 3 0.000131 -7663> 2014-08-13 17:22:48.925212 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 2 ==== pg_query(22.220 epoch 194470) v3 ==== 144+0+0 (261674981 0 0) 0x29c0960 con 0x4868420 -7662> 2014-08-13 17:22:48.925241 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.794762, event: header_read, request: pg_query(22.220 epoch 194470) v3 -7661> 2014-08-13 17:22:48.925234 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started 0.220320 0 0.000000 -7660> 2014-08-13 17:22:48.925252 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.794763, event: throttled, request: pg_query(22.220 epoch 194470) v3 -7659> 2014-08-13 17:22:48.925261 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.794801, event: all_read, request: pg_query(22.220 epoch 194470) v3 -7658> 2014-08-13 17:22:48.925255 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194447/194448/192981) [345,4,374] r=1 lpr=194449 pi=192981-194447/17 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset -7657> 2014-08-13 17:22:48.925220 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.221560 3 0.000299 -7656> 2014-08-13 17:22:48.925274 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.925238, event: dispatched, request: pg_query(22.220 epoch 194470) v3 -7655> 2014-08-13 17:22:48.925292 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.925292, event: waiting_for_osdmap, request: pg_query(22.220 epoch 194470) v3 -7654> 2014-08-13 17:22:48.925304 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.925304, event: started, request: pg_query(22.220 epoch 194470) v3 -7653> 2014-08-13 17:22:48.925289 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started 0.221712 0 0.000000 -7652> 2014-08-13 17:22:48.925327 7f74d8735700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:22:48.925327, event: done, request: pg_query(22.220 epoch 194470) v3 -7651> 2014-08-13 17:22:48.925318 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192949) [328,364,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset -7650> 2014-08-13 17:22:48.925355 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 2 ==== pg_query(22.3ad epoch 194470) v3 ==== 144+0+0 (4236778832 0 0) 0x29c4920 con 0x4868160 -7649> 2014-08-13 17:22:48.925375 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.811668, event: header_read, request: pg_query(22.3ad epoch 194470) v3 -7648> 2014-08-13 17:22:48.925385 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.811668, event: throttled, request: pg_query(22.3ad epoch 194470) v3 -7647> 2014-08-13 17:22:48.925393 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.811696, event: all_read, request: pg_query(22.3ad epoch 194470) v3 -7646> 2014-08-13 17:22:48.925402 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.925373, event: dispatched, request: pg_query(22.3ad epoch 194470) v3 -7645> 2014-08-13 17:22:48.925411 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.925411, event: waiting_for_osdmap, request: pg_query(22.3ad epoch 194470) v3 -7644> 2014-08-13 17:22:48.925422 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.925422, event: started, request: pg_query(22.3ad epoch 194470) v3 -7643> 2014-08-13 17:22:48.925440 7f74d8735700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:22:48.925440, event: done, request: pg_query(22.3ad epoch 194470) v3 -7642> 2014-08-13 17:22:48.925458 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 2 ==== pg_notify(22.1d2(1) epoch 194470) v5 ==== 777+0+0 (167637636 0 0) 0x466f380 con 0x486bf40 -7641> 2014-08-13 17:22:48.925476 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.799346, event: header_read, request: pg_notify(22.1d2(1) epoch 194470) v5 -7640> 2014-08-13 17:22:48.925487 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.799347, event: throttled, request: pg_notify(22.1d2(1) epoch 194470) v5 -7639> 2014-08-13 17:22:48.925495 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.799391, event: all_read, request: pg_notify(22.1d2(1) epoch 194470) v5 -7638> 2014-08-13 17:22:48.925504 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.925474, event: dispatched, request: pg_notify(22.1d2(1) epoch 194470) v5 -7637> 2014-08-13 17:22:48.925514 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.925514, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194470) v5 -7636> 2014-08-13 17:22:48.925525 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.925524, event: started, request: pg_notify(22.1d2(1) epoch 194470) v5 -7635> 2014-08-13 17:22:48.925548 7f74d8735700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:22:48.925548, event: done, request: pg_notify(22.1d2(1) epoch 194470) v5 -7634> 2014-08-13 17:22:48.925591 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 2 ==== pg_query(24.181 epoch 194470) v3 ==== 144+0+0 (149944708 0 0) 0x29c3840 con 0x486bc80 -7633> 2014-08-13 17:22:48.925615 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.807272, event: header_read, request: pg_query(24.181 epoch 194470) v3 -7632> 2014-08-13 17:22:48.925626 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.807272, event: throttled, request: pg_query(24.181 epoch 194470) v3 -7631> 2014-08-13 17:22:48.925635 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.807309, event: all_read, request: pg_query(24.181 epoch 194470) v3 -7630> 2014-08-13 17:22:48.925644 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.925608, event: dispatched, request: pg_query(24.181 epoch 194470) v3 -7629> 2014-08-13 17:22:48.925653 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.925653, event: waiting_for_osdmap, request: pg_query(24.181 epoch 194470) v3 -7628> 2014-08-13 17:22:48.925665 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.925665, event: started, request: pg_query(24.181 epoch 194470) v3 -7627> 2014-08-13 17:22:48.925684 7f74d8735700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:22:48.925684, event: done, request: pg_query(24.181 epoch 194470) v3 -7626> 2014-08-13 17:22:48.925708 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 2 ==== pg_query(24.2ec epoch 194470) v3 ==== 144+0+0 (3912634502 0 0) 0x29c5820 con 0x486b9c0 -7625> 2014-08-13 17:22:48.925726 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.819507, event: header_read, request: pg_query(24.2ec epoch 194470) v3 -7624> 2014-08-13 17:22:48.925737 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.819509, event: throttled, request: pg_query(24.2ec epoch 194470) v3 -7623> 2014-08-13 17:22:48.925745 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.819585, event: all_read, request: pg_query(24.2ec epoch 194470) v3 -7622> 2014-08-13 17:22:48.925754 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.925724, event: dispatched, request: pg_query(24.2ec epoch 194470) v3 -7621> 2014-08-13 17:22:48.925764 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.925763, event: waiting_for_osdmap, request: pg_query(24.2ec epoch 194470) v3 -7620> 2014-08-13 17:22:48.925774 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.925774, event: started, request: pg_query(24.2ec epoch 194470) v3 -7619> 2014-08-13 17:22:48.925797 7f74d8735700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:22:48.925797, event: done, request: pg_query(24.2ec epoch 194470) v3 -7618> 2014-08-13 17:22:48.925821 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 2 ==== pg_notify(24.b6(1) epoch 194470) v5 ==== 811+0+0 (3923748831 0 0) 0x4ac0c40 con 0x486b700 -7617> 2014-08-13 17:22:48.925840 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.803527, event: header_read, request: pg_notify(24.b6(1) epoch 194470) v5 -7616> 2014-08-13 17:22:48.925819 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.000565 15 0.000191 -7615> 2014-08-13 17:22:48.925853 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.803528, event: throttled, request: pg_notify(24.b6(1) epoch 194470) v5 -7614> 2014-08-13 17:22:48.925873 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.805573, event: all_read, request: pg_notify(24.b6(1) epoch 194470) v5 -7613> 2014-08-13 17:22:48.925869 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started -7612> 2014-08-13 17:22:48.925883 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.925838, event: dispatched, request: pg_notify(24.b6(1) epoch 194470) v5 -7611> 2014-08-13 17:22:48.925894 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.925894, event: waiting_for_osdmap, request: pg_notify(24.b6(1) epoch 194470) v5 -7610> 2014-08-13 17:22:48.925886 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start -7609> 2014-08-13 17:22:48.925907 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.925907, event: started, request: pg_notify(24.b6(1) epoch 194470) v5 -7608> 2014-08-13 17:22:48.925914 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7607> 2014-08-13 17:22:48.925934 7f74d8735700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:22:48.925934, event: done, request: pg_notify(24.b6(1) epoch 194470) v5 -7606> 2014-08-13 17:22:48.925938 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000052 0 0.000000 -7605> 2014-08-13 17:22:48.925941 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.000623 15 0.000230 -7604> 2014-08-13 17:22:48.925965 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 2 ==== pg_notify(24.1bb(25) epoch 194470) v5 ==== 2155+0+0 (2005506023 0 0) 0x4ac6ac0 con 0x486b440 -7603> 2014-08-13 17:22:48.925989 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.811528, event: header_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -7602> 2014-08-13 17:22:48.926000 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.811530, event: throttled, request: pg_notify(24.1bb(25) epoch 194470) v5 -7601> 2014-08-13 17:22:48.926009 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.811612, event: all_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -7600> 2014-08-13 17:22:48.926018 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.925987, event: dispatched, request: pg_notify(24.1bb(25) epoch 194470) v5 -7599> 2014-08-13 17:22:48.926030 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.926030, event: waiting_for_osdmap, request: pg_notify(24.1bb(25) epoch 194470) v5 -7598> 2014-08-13 17:22:48.926020 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started -7597> 2014-08-13 17:22:48.926042 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.926042, event: started, request: pg_notify(24.1bb(25) epoch 194470) v5 -7596> 2014-08-13 17:22:48.926043 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start -7595> 2014-08-13 17:22:48.926063 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7594> 2014-08-13 17:22:48.926073 7f74d8735700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:22:48.926073, event: done, request: pg_notify(24.1bb(25) epoch 194470) v5 -7593> 2014-08-13 17:22:48.926095 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000052 0 0.000000 -7592> 2014-08-13 17:22:48.926119 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray -7591> 2014-08-13 17:22:48.926130 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 2 ==== pg_query(24.e2 epoch 194470) v3 ==== 144+0+0 (3471008233 0 0) 0x29c5460 con 0x486b180 -7590> 2014-08-13 17:22:48.925963 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194449/194389 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray -7589> 2014-08-13 17:22:48.926154 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.818629, event: header_read, request: pg_query(24.e2 epoch 194470) v3 -7588> 2014-08-13 17:22:48.926172 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.818632, event: throttled, request: pg_query(24.e2 epoch 194470) v3 -7587> 2014-08-13 17:22:48.926181 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.818793, event: all_read, request: pg_query(24.e2 epoch 194470) v3 -7586> 2014-08-13 17:22:48.926190 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.926152, event: dispatched, request: pg_query(24.e2 epoch 194470) v3 -7585> 2014-08-13 17:22:48.926201 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.926201, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194470) v3 -7584> 2014-08-13 17:22:48.926212 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.926212, event: started, request: pg_query(24.e2 epoch 194470) v3 -7583> 2014-08-13 17:22:48.926232 7f74d8735700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:22:48.926232, event: done, request: pg_query(24.e2 epoch 194470) v3 -7582> 2014-08-13 17:22:48.926252 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 2 ==== pg_query(22.69f epoch 194470) v3 ==== 144+0+0 (3217917722 0 0) 0x29c30c0 con 0x486aec0 -7581> 2014-08-13 17:22:48.926271 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.805700, event: header_read, request: pg_query(22.69f epoch 194470) v3 -7580> 2014-08-13 17:22:48.926280 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.805701, event: throttled, request: pg_query(22.69f epoch 194470) v3 -7579> 2014-08-13 17:22:48.926289 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.805735, event: all_read, request: pg_query(22.69f epoch 194470) v3 -7578> 2014-08-13 17:22:48.926297 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.926269, event: dispatched, request: pg_query(22.69f epoch 194470) v3 -7577> 2014-08-13 17:22:48.926307 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.926307, event: waiting_for_osdmap, request: pg_query(22.69f epoch 194470) v3 -7576> 2014-08-13 17:22:48.926317 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.926317, event: started, request: pg_query(22.69f epoch 194470) v3 -7575> 2014-08-13 17:22:48.926337 7f74d8735700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:22:48.926336, event: done, request: pg_query(22.69f epoch 194470) v3 -7574> 2014-08-13 17:22:48.926325 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Peering, affected_by_map, going to Reset -7573> 2014-08-13 17:22:48.926357 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 2 ==== pg_query(22.639 epoch 194470) v3 ==== 144+0+0 (780467502 0 0) 0x29c3a20 con 0x486a940 -7572> 2014-08-13 17:22:48.926374 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.809415, event: header_read, request: pg_query(22.639 epoch 194470) v3 -7571> 2014-08-13 17:22:48.926362 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetInfo 0.221113 2 0.000151 -7570> 2014-08-13 17:22:48.926387 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.809416, event: throttled, request: pg_query(22.639 epoch 194470) v3 -7569> 2014-08-13 17:22:48.926393 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.809445, event: all_read, request: pg_query(22.639 epoch 194470) v3 -7568> 2014-08-13 17:22:48.926405 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.926372, event: dispatched, request: pg_query(22.639 epoch 194470) v3 -7567> 2014-08-13 17:22:48.926398 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering 0.221163 0 0.000000 -7566> 2014-08-13 17:22:48.926414 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.926414, event: waiting_for_osdmap, request: pg_query(22.639 epoch 194470) v3 -7565> 2014-08-13 17:22:48.926436 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.926436, event: started, request: pg_query(22.639 epoch 194470) v3 -7564> 2014-08-13 17:22:48.926429 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.221206 0 0.000000 -7563> 2014-08-13 17:22:48.926456 7f74d8735700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:22:48.926456, event: done, request: pg_query(22.639 epoch 194470) v3 -7562> 2014-08-13 17:22:48.926457 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started 0.221292 0 0.000000 -7561> 2014-08-13 17:22:48.926478 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 2 ==== pg_query(22.12a epoch 194470) v3 ==== 144+0+0 (1407578389 0 0) 0x29c5dc0 con 0x486a680 -7560> 2014-08-13 17:22:48.926482 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194447/194448/194448) [4,25,362]/[4,20,303] r=0 lpr=194449 pi=193635-194447/19 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset -7559> 2014-08-13 17:22:48.926497 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.826575, event: header_read, request: pg_query(22.12a epoch 194470) v3 -7558> 2014-08-13 17:22:48.926503 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.826577, event: throttled, request: pg_query(22.12a epoch 194470) v3 -7557> 2014-08-13 17:22:48.926513 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.826641, event: all_read, request: pg_query(22.12a epoch 194470) v3 -7556> 2014-08-13 17:22:48.926522 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.926494, event: dispatched, request: pg_query(22.12a epoch 194470) v3 -7555> 2014-08-13 17:22:48.926531 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.926531, event: waiting_for_osdmap, request: pg_query(22.12a epoch 194470) v3 -7554> 2014-08-13 17:22:48.926541 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.926541, event: started, request: pg_query(22.12a epoch 194470) v3 -7553> 2014-08-13 17:22:48.926559 7f74d8735700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:22:48.926559, event: done, request: pg_query(22.12a epoch 194470) v3 -7552> 2014-08-13 17:22:48.926577 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 2 ==== pg_notify(24.3d(23) epoch 194470) v5 ==== 2037+0+0 (2598945574 0 0) 0x4ac2300 con 0x486a3c0 -7551> 2014-08-13 17:22:48.926595 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.805723, event: header_read, request: pg_notify(24.3d(23) epoch 194470) v5 -7550> 2014-08-13 17:22:48.926605 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.805724, event: throttled, request: pg_notify(24.3d(23) epoch 194470) v5 -7549> 2014-08-13 17:22:48.926619 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.805777, event: all_read, request: pg_notify(24.3d(23) epoch 194470) v5 -7548> 2014-08-13 17:22:48.926625 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.926593, event: dispatched, request: pg_notify(24.3d(23) epoch 194470) v5 -7547> 2014-08-13 17:22:48.926631 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.926631, event: waiting_for_osdmap, request: pg_notify(24.3d(23) epoch 194470) v5 -7546> 2014-08-13 17:22:48.926643 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.926643, event: started, request: pg_notify(24.3d(23) epoch 194470) v5 -7545> 2014-08-13 17:22:48.927001 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.467085 17 0.000224 -7544> 2014-08-13 17:22:48.927037 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Started -7543> 2014-08-13 17:22:48.927060 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Start -7542> 2014-08-13 17:22:48.927069 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Reset 0.000588 15 0.000341 -7541> 2014-08-13 17:22:48.927081 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -7540> 2014-08-13 17:22:48.927107 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started -7539> 2014-08-13 17:22:48.927115 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000055 0 0.000000 -7538> 2014-08-13 17:22:48.927128 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Start -7537> 2014-08-13 17:22:48.927148 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: transitioning to Primary -7536> 2014-08-13 17:22:48.927143 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194467 pi=193125-194466/32 crt=194422'15445 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -7535> 2014-08-13 17:22:48.927180 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Start 0.000053 0 0.000000 -7534> 2014-08-13 17:22:48.927206 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary -7533> 2014-08-13 17:22:48.927227 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering -7532> 2014-08-13 17:22:48.927246 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194443/194389 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetInfo -7531> 2014-08-13 17:22:48.927314 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started/Stray 0.222920 3 0.000141 -7530> 2014-08-13 17:22:48.927377 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started 0.223047 0 0.000000 -7529> 2014-08-13 17:22:48.927406 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194447/194448/193227) [330,4,288]/[330,4,57] r=1 lpr=194449 pi=193227-194447/23 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset -7528> 2014-08-13 17:22:48.927412 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(19) epoch 194470) v5 -- ?+0 0x57d1880 con 0x45f2940 -7527> 2014-08-13 17:22:48.927423 7f74d8735700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:22:48.927422, event: done, request: pg_notify(24.3d(23) epoch 194470) v5 -7526> 2014-08-13 17:22:48.927444 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(19) epoch 194470) v5 -- ?+0 0x57d16c0 con 0x45f6f60 -7525> 2014-08-13 17:22:48.927468 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6834/14316 -- pg_notify(22.14a(19) epoch 194470) v5 -- ?+0 0x57d1500 con 0x45f1e40 -7524> 2014-08-13 17:22:48.927490 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(22) epoch 194470) v5 -- ?+0 0x57d1340 con 0x486bc80 -7523> 2014-08-13 17:22:48.927506 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(20) epoch 194470) v5 -- ?+0 0x57d1180 con 0x475c4c0 -7522> 2014-08-13 17:22:48.927467 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 2 ==== pg_query(24.20e epoch 194470) v3 ==== 144+0+0 (1441797804 0 0) 0x29c1fe0 con 0x48d9ce0 -7521> 2014-08-13 17:22:48.927523 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.801748, event: header_read, request: pg_query(24.20e epoch 194470) v3 -7520> 2014-08-13 17:22:48.927525 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(22) epoch 194470) v5 -- ?+0 0x466b2c0 con 0x4868420 -7519> 2014-08-13 17:22:48.927531 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.801748, event: throttled, request: pg_query(24.20e epoch 194470) v3 -7518> 2014-08-13 17:22:48.927537 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.801833, event: all_read, request: pg_query(24.20e epoch 194470) v3 -7517> 2014-08-13 17:22:48.927542 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.927520, event: dispatched, request: pg_query(24.20e epoch 194470) v3 -7516> 2014-08-13 17:22:48.927548 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.927548, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194470) v3 -7515> 2014-08-13 17:22:48.927557 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.927556, event: started, request: pg_query(24.20e epoch 194470) v3 -7514> 2014-08-13 17:22:48.927573 7f74d8735700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:22:48.927573, event: done, request: pg_query(24.20e epoch 194470) v3 -7513> 2014-08-13 17:22:48.927589 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 2 ==== pg_notify(22.1d2(22) epoch 194470) v5 ==== 1952+0+0 (3049200412 0 0) 0x4ac1340 con 0x486e040 -7512> 2014-08-13 17:22:48.927604 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.805566, event: header_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -7511> 2014-08-13 17:22:48.927617 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.805567, event: throttled, request: pg_notify(22.1d2(22) epoch 194470) v5 -7510> 2014-08-13 17:22:48.927623 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.805625, event: all_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -7509> 2014-08-13 17:22:48.927632 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.927602, event: dispatched, request: pg_notify(22.1d2(22) epoch 194470) v5 -7508> 2014-08-13 17:22:48.927645 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.927645, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194470) v5 -7507> 2014-08-13 17:22:48.927653 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.927653, event: started, request: pg_notify(22.1d2(22) epoch 194470) v5 -7506> 2014-08-13 17:22:48.927677 7f74d8735700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:22:48.927677, event: done, request: pg_notify(22.1d2(22) epoch 194470) v5 -7505> 2014-08-13 17:22:48.927679 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6815/60573 -- pg_query(24.30b epoch 194470) v3 -- ?+0 0x29c0000 con 0x45f1760 -7504> 2014-08-13 17:22:48.927706 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_query(24.3d epoch 194470) v3 -- ?+0 0x29c3a20 con 0x4758840 -7503> 2014-08-13 17:22:48.927703 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 2 ==== pg_notify(22.362(22) epoch 194470) v5 ==== 1948+0+0 (1366328122 0 0) 0x49e1dc0 con 0x48db9c0 -7502> 2014-08-13 17:22:48.927720 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.822213, event: header_read, request: pg_notify(22.362(22) epoch 194470) v5 -7501> 2014-08-13 17:22:48.927722 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6800/36920 -- pg_query(24.3d epoch 194470) v3 -- ?+0 0x29c30c0 con 0x4904a40 -7500> 2014-08-13 17:22:48.927726 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.822216, event: throttled, request: pg_notify(22.362(22) epoch 194470) v5 -7499> 2014-08-13 17:22:48.927732 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.822318, event: all_read, request: pg_notify(22.362(22) epoch 194470) v5 -7498> 2014-08-13 17:22:48.927737 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.927718, event: dispatched, request: pg_notify(22.362(22) epoch 194470) v5 -7497> 2014-08-13 17:22:48.927743 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.927742, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194470) v5 -7496> 2014-08-13 17:22:48.927739 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6822/10782 -- pg_query(22.619 epoch 194470) v3 -- ?+0 0x29c5460 con 0x47a4ba0 -7495> 2014-08-13 17:22:48.927749 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.927749, event: started, request: pg_notify(22.362(22) epoch 194470) v5 -7494> 2014-08-13 17:22:48.927757 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6838/8300 -- pg_query(22.619 epoch 194470) v3 -- ?+0 0x29c2760 con 0x4904780 -7493> 2014-08-13 17:22:48.927770 7f74d8735700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:22:48.927770, event: done, request: pg_notify(22.362(22) epoch 194470) v5 -7492> 2014-08-13 17:22:48.927797 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 2 ==== pg_notify(22.481(39) epoch 194470) v5 ==== 2903+0+0 (479616867 0 0) 0x4ac1f80 con 0x48dad60 -7491> 2014-08-13 17:22:48.927811 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.805675, event: header_read, request: pg_notify(22.481(39) epoch 194470) v5 -7490> 2014-08-13 17:22:48.927817 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.805676, event: throttled, request: pg_notify(22.481(39) epoch 194470) v5 -7489> 2014-08-13 17:22:48.927824 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.805756, event: all_read, request: pg_notify(22.481(39) epoch 194470) v5 -7488> 2014-08-13 17:22:48.927829 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.927809, event: dispatched, request: pg_notify(22.481(39) epoch 194470) v5 -7487> 2014-08-13 17:22:48.927835 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.927835, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194470) v5 -7486> 2014-08-13 17:22:48.927842 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.927842, event: started, request: pg_notify(22.481(39) epoch 194470) v5 -7485> 2014-08-13 17:22:48.927860 7f74d8735700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:22:48.927860, event: done, request: pg_notify(22.481(39) epoch 194470) v5 -7484> 2014-08-13 17:22:48.927878 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 2 ==== pg_notify(24.1bb(1) epoch 194470) v5 ==== 811+0+0 (655629246 0 0) 0x466fc40 con 0x48dcd00 -7483> 2014-08-13 17:22:48.927893 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.799570, event: header_read, request: pg_notify(24.1bb(1) epoch 194470) v5 -7482> 2014-08-13 17:22:48.927901 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.799572, event: throttled, request: pg_notify(24.1bb(1) epoch 194470) v5 -7481> 2014-08-13 17:22:48.927906 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.799648, event: all_read, request: pg_notify(24.1bb(1) epoch 194470) v5 -7480> 2014-08-13 17:22:48.927912 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.927891, event: dispatched, request: pg_notify(24.1bb(1) epoch 194470) v5 -7479> 2014-08-13 17:22:48.927918 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.927917, event: waiting_for_osdmap, request: pg_notify(24.1bb(1) epoch 194470) v5 -7478> 2014-08-13 17:22:48.927925 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.927925, event: started, request: pg_notify(24.1bb(1) epoch 194470) v5 -7477> 2014-08-13 17:22:48.927914 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6834/14316 -- pg_query(22.362 epoch 194470) v3 -- ?+0 0x29c4920 con 0x45f1e40 -7476> 2014-08-13 17:22:48.927941 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6840/16566 -- pg_query(22.362 epoch 194470) v3 -- ?+0 0x29c2580 con 0x47a0b00 -7475> 2014-08-13 17:22:48.927945 7f74d8735700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:22:48.927945, event: done, request: pg_notify(24.1bb(1) epoch 194470) v5 -7474> 2014-08-13 17:22:48.927966 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6824/5505 -- pg_query(22.619 epoch 194470) v3 -- ?+0 0x29c2d00 con 0x45f77a0 -7473> 2014-08-13 17:22:48.927986 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6842/17345 -- pg_query(22.619 epoch 194470) v3 -- ?+0 0x29c1680 con 0x4d51a20 -7472> 2014-08-13 17:22:48.927992 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 2 ==== pg_notify(22.3d5(20) epoch 194470) v5 ==== 1838+0+0 (1108422747 0 0) 0x4ac7540 con 0x48dfe80 -7471> 2014-08-13 17:22:48.928003 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_query(22.f3 epoch 194470) v3 -- ?+0 0x29c5dc0 con 0x475cd00 -7470> 2014-08-13 17:22:48.928011 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.815259, event: header_read, request: pg_notify(22.3d5(20) epoch 194470) v5 -7469> 2014-08-13 17:22:48.928018 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.815261, event: throttled, request: pg_notify(22.3d5(20) epoch 194470) v5 -7468> 2014-08-13 17:22:48.928024 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.817407, event: all_read, request: pg_notify(22.3d5(20) epoch 194470) v5 -7467> 2014-08-13 17:22:48.928029 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.928008, event: dispatched, request: pg_notify(22.3d5(20) epoch 194470) v5 -7466> 2014-08-13 17:22:48.928035 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.928034, event: waiting_for_osdmap, request: pg_notify(22.3d5(20) epoch 194470) v5 -7465> 2014-08-13 17:22:48.928033 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6820/19173 -- pg_query(24.30b epoch 194470) v3 -- ?+0 0x29c2b20 con 0x48dfe80 -7464> 2014-08-13 17:22:48.928042 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.928042, event: started, request: pg_notify(22.3d5(20) epoch 194470) v5 -7463> 2014-08-13 17:22:48.928052 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6820/25633 -- pg_query(24.9b epoch 194470) v3 -- ?+0 0x29c3c00 con 0x47591e0 -7462> 2014-08-13 17:22:48.928070 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6814/16727 -- pg_query(22.22c,24.30b epoch 194470) v3 -- ?+0 0x29c4380 con 0x45f5540 -7461> 2014-08-13 17:22:48.928172 7f74d8735700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:22:48.928172, event: done, request: pg_notify(22.3d5(20) epoch 194470) v5 -7460> 2014-08-13 17:22:48.928189 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6810/15731 -- pg_query(22.22c epoch 194470) v3 -- ?+0 0x29c5a00 con 0x49044c0 -7459> 2014-08-13 17:22:48.928214 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_query(24.3d epoch 194470) v3 -- ?+0 0x29c5820 con 0x475ee00 -7458> 2014-08-13 17:22:48.928213 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 2 ==== pg_notify(22.3d9(9) epoch 194470) v5 ==== 1229+0+0 (432648880 0 0) 0x4ac0a80 con 0x48dfbc0 -7457> 2014-08-13 17:22:48.928232 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.801712, event: header_read, request: pg_notify(22.3d9(9) epoch 194470) v5 -7456> 2014-08-13 17:22:48.928232 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6816/24270 -- pg_query(22.f3 epoch 194470) v3 -- ?+0 0x29c3840 con 0x4d54200 -7455> 2014-08-13 17:22:48.928239 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.801713, event: throttled, request: pg_notify(22.3d9(9) epoch 194470) v5 -7454> 2014-08-13 17:22:48.928245 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.805114, event: all_read, request: pg_notify(22.3d9(9) epoch 194470) v5 -7453> 2014-08-13 17:22:48.927994 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Reset 0.000588 15 0.000276 -7452> 2014-08-13 17:22:48.928251 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.928229, event: dispatched, request: pg_notify(22.3d9(9) epoch 194470) v5 -7451> 2014-08-13 17:22:48.928251 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6816/17521 -- pg_query(22.f3 epoch 194470) v3 -- ?+0 0x4eb05a0 con 0x4906880 -7450> 2014-08-13 17:22:48.928258 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.928258, event: waiting_for_osdmap, request: pg_notify(22.3d9(9) epoch 194470) v5 -7449> 2014-08-13 17:22:48.928266 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.928266, event: started, request: pg_notify(22.3d9(9) epoch 194470) v5 -7448> 2014-08-13 17:22:48.928259 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started -7447> 2014-08-13 17:22:48.928269 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6802/22303 -- pg_query(22.619 epoch 194470) v3 -- ?+0 0x4eb10e0 con 0x4902100 -7446> 2014-08-13 17:22:48.928286 7f74d8735700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:22:48.928286, event: done, request: pg_notify(22.3d9(9) epoch 194470) v5 -7445> 2014-08-13 17:22:48.928288 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_query(22.22c epoch 194470) v3 -- ?+0 0x4eb0f00 con 0x4903180 -7444> 2014-08-13 17:22:48.928305 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_query(22.362 epoch 194470) v3 -- ?+0 0x4eb03c0 con 0x48377a0 -7443> 2014-08-13 17:22:48.928309 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 2 ==== pg_notify(22.775(22) epoch 194470) v5 ==== 1948+0+0 (899104279 0 0) 0x466ec80 con 0x4900420 -7442> 2014-08-13 17:22:48.928324 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.796596, event: header_read, request: pg_notify(22.775(22) epoch 194470) v5 -7441> 2014-08-13 17:22:48.928322 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6800/61726 -- pg_query(22.362 epoch 194470) v3 -- ?+0 0x29c4740 con 0x4d55d80 -7440> 2014-08-13 17:22:48.928333 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.796597, event: throttled, request: pg_notify(22.775(22) epoch 194470) v5 -7439> 2014-08-13 17:22:48.928338 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.797406, event: all_read, request: pg_notify(22.775(22) epoch 194470) v5 -7438> 2014-08-13 17:22:48.928343 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.928322, event: dispatched, request: pg_notify(22.775(22) epoch 194470) v5 -7437> 2014-08-13 17:22:48.928349 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.928349, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194470) v5 -7436> 2014-08-13 17:22:48.928357 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.928356, event: started, request: pg_notify(22.775(22) epoch 194470) v5 -7435> 2014-08-13 17:22:48.928375 7f74d8735700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:22:48.928374, event: done, request: pg_notify(22.775(22) epoch 194470) v5 -7434> 2014-08-13 17:22:48.928403 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 2 ==== pg_query(24.299 epoch 194470) v3 ==== 144+0+0 (1086028604 0 0) 0x29c21c0 con 0x4900580 -7433> 2014-08-13 17:22:48.928417 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.801533, event: header_read, request: pg_query(24.299 epoch 194470) v3 -7432> 2014-08-13 17:22:48.928424 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.801535, event: throttled, request: pg_query(24.299 epoch 194470) v3 -7431> 2014-08-13 17:22:48.928429 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.801859, event: all_read, request: pg_query(24.299 epoch 194470) v3 -7430> 2014-08-13 17:22:48.928434 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.928415, event: dispatched, request: pg_query(24.299 epoch 194470) v3 -7429> 2014-08-13 17:22:48.928427 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6832/53185 -- pg_query(24.9b epoch 194470) v3 -- ?+0 0x29c4b00 con 0x4d55ac0 -7428> 2014-08-13 17:22:48.928439 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.928439, event: waiting_for_osdmap, request: pg_query(24.299 epoch 194470) v3 -7427> 2014-08-13 17:22:48.928447 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.928447, event: started, request: pg_query(24.299 epoch 194470) v3 -7426> 2014-08-13 17:22:48.928452 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6842/8170 -- pg_query(22.362,24.3d epoch 194470) v3 -- ?+0 0x29c5280 con 0x48db9c0 -7425> 2014-08-13 17:22:48.928278 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Start -7424> 2014-08-13 17:22:48.928470 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6824/15623 -- pg_query(24.9b epoch 194470) v3 -- ?+0 0x4eb2580 con 0x4d51ce0 -7423> 2014-08-13 17:22:48.928486 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6840/55513 -- pg_query(24.9b epoch 194470) v3 -- ?+0 0x4eb23a0 con 0x486a940 -7422> 2014-08-13 17:22:48.928476 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: transitioning to Stray -7421> 2014-08-13 17:22:48.928495 7f74d8735700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6814/41806 -- pg_notify(24.299 epoch 194470) v5 -- ?+0 0x466fe00 con 0x4900580 -7420> 2014-08-13 17:22:48.928504 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6826/15861 -- pg_query(22.22c epoch 194470) v3 -- ?+0 0x4eb21c0 con 0x4902680 -7419> 2014-08-13 17:22:48.928528 7f74d8735700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:22:48.928528, event: done, request: pg_query(24.299 epoch 194470) v3 -7418> 2014-08-13 17:22:48.928546 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 2 ==== pg_notify(22.3af(22) epoch 194470) v5 ==== 1948+0+0 (2286998326 0 0) 0x4ac39c0 con 0x4900f20 -7417> 2014-08-13 17:22:48.928561 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.808411, event: header_read, request: pg_notify(22.3af(22) epoch 194470) v5 -7416> 2014-08-13 17:22:48.928568 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.808412, event: throttled, request: pg_notify(22.3af(22) epoch 194470) v5 -7415> 2014-08-13 17:22:48.928573 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.808478, event: all_read, request: pg_notify(22.3af(22) epoch 194470) v5 -7414> 2014-08-13 17:22:48.928578 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.928559, event: dispatched, request: pg_notify(22.3af(22) epoch 194470) v5 -7413> 2014-08-13 17:22:48.928584 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.928584, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194470) v5 -7412> 2014-08-13 17:22:48.928591 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.928591, event: started, request: pg_notify(22.3af(22) epoch 194470) v5 -7411> 2014-08-13 17:22:48.928626 7f74d8735700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:22:48.928626, event: done, request: pg_notify(22.3af(22) epoch 194470) v5 -7410> 2014-08-13 17:22:48.928651 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 2 ==== pg_notify(22.113(1) epoch 194470) v5 ==== 777+0+0 (2701808073 0 0) 0x4ac1180 con 0x4901600 -7409> 2014-08-13 17:22:48.928666 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.805506, event: header_read, request: pg_notify(22.113(1) epoch 194470) v5 -7408> 2014-08-13 17:22:48.928673 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.805507, event: throttled, request: pg_notify(22.113(1) epoch 194470) v5 -7407> 2014-08-13 17:22:48.928678 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.805586, event: all_read, request: pg_notify(22.113(1) epoch 194470) v5 -7406> 2014-08-13 17:22:48.928683 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.928664, event: dispatched, request: pg_notify(22.113(1) epoch 194470) v5 -7405> 2014-08-13 17:22:48.928689 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.928689, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194470) v5 -7404> 2014-08-13 17:22:48.928696 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.928696, event: started, request: pg_notify(22.113(1) epoch 194470) v5 -7403> 2014-08-13 17:22:48.928714 7f74d8735700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:22:48.928714, event: done, request: pg_notify(22.113(1) epoch 194470) v5 -7402> 2014-08-13 17:22:48.928743 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.19 192.168.1.115:6818/61186 2 ==== pg_query(24.3bf epoch 194470) v3 ==== 144+0+0 (2241491387 0 0) 0x29c5be0 con 0x4901340 -7401> 2014-08-13 17:22:48.928757 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.821528, event: header_read, request: pg_query(24.3bf epoch 194470) v3 -7400> 2014-08-13 17:22:48.928765 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.821529, event: throttled, request: pg_query(24.3bf epoch 194470) v3 -7399> 2014-08-13 17:22:48.928776 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.821579, event: all_read, request: pg_query(24.3bf epoch 194470) v3 -7398> 2014-08-13 17:22:48.928792 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6824/4037 -- pg_query(22.f3 epoch 194470) v3 -- ?+0 0x4eb1fe0 con 0x4d58840 -7397> 2014-08-13 17:22:48.928822 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_query(24.3d epoch 194470) v3 -- ?+0 0x4eb1e00 con 0x486a3c0 -7396> 2014-08-13 17:22:48.928923 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6830/52879 -- pg_query(24.30b epoch 194470) v3 -- ?+0 0x4eb1c20 con 0x49023c0 -7395> 2014-08-13 17:22:48.928958 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6840/19195 -- pg_query(24.9b epoch 194470) v3 -- ?+0 0x4eb1a40 con 0x4d511e0 -7394> 2014-08-13 17:22:48.928781 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.928755, event: dispatched, request: pg_query(24.3bf epoch 194470) v3 -7393> 2014-08-13 17:22:48.929051 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.217749 3 0.000351 -7392> 2014-08-13 17:22:48.929088 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.929088, event: waiting_for_osdmap, request: pg_query(24.3bf epoch 194470) v3 -7391> 2014-08-13 17:22:48.929107 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.929106, event: started, request: pg_query(24.3bf epoch 194470) v3 -7390> 2014-08-13 17:22:48.929096 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.217991 0 0.000000 -7389> 2014-08-13 17:22:48.929127 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194447/194448/192970) [371,4,21] r=1 lpr=194449 pi=192977-194447/17 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -7388> 2014-08-13 17:22:48.929295 7f74d8735700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6818/61186 -- pg_notify(24.3bf epoch 194470) v5 -- ?+0 0x4ac1f80 con 0x4901340 -7387> 2014-08-13 17:22:48.929708 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.000581 15 0.000264 -7386> 2014-08-13 17:22:48.929731 7f74d8735700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:22:48.929731, event: done, request: pg_query(24.3bf epoch 194470) v3 -7385> 2014-08-13 17:22:48.929739 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -7384> 2014-08-13 17:22:48.929755 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -7383> 2014-08-13 17:22:48.929757 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 2 ==== pg_notify(22.22c(20) epoch 194470) v5 ==== 1834+0+0 (2326237699 0 0) 0x49e2300 con 0x4902680 -7382> 2014-08-13 17:22:48.929777 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.822390, event: header_read, request: pg_notify(22.22c(20) epoch 194470) v5 -7381> 2014-08-13 17:22:48.929769 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7380> 2014-08-13 17:22:48.929784 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.822393, event: throttled, request: pg_notify(22.22c(20) epoch 194470) v5 -7379> 2014-08-13 17:22:48.929790 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.822525, event: all_read, request: pg_notify(22.22c(20) epoch 194470) v5 -7378> 2014-08-13 17:22:48.929796 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.929774, event: dispatched, request: pg_notify(22.22c(20) epoch 194470) v5 -7377> 2014-08-13 17:22:48.929789 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 -7376> 2014-08-13 17:22:48.929802 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.929802, event: waiting_for_osdmap, request: pg_notify(22.22c(20) epoch 194470) v5 -7375> 2014-08-13 17:22:48.929811 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.929811, event: started, request: pg_notify(22.22c(20) epoch 194470) v5 -7374> 2014-08-13 17:22:48.929809 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194449/194389 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -7373> 2014-08-13 17:22:48.929834 7f74d8735700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:22:48.929834, event: done, request: pg_notify(22.22c(20) epoch 194470) v5 -7372> 2014-08-13 17:22:48.929908 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 2 ==== pg_notify(24.30b(28) epoch 194470) v5 ==== 2332+0+0 (2687328995 0 0) 0x4ac2a00 con 0x49023c0 -7371> 2014-08-13 17:22:48.929987 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.806213, event: header_read, request: pg_notify(24.30b(28) epoch 194470) v5 -7370> 2014-08-13 17:22:48.930002 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.806214, event: throttled, request: pg_notify(24.30b(28) epoch 194470) v5 -7369> 2014-08-13 17:22:48.930011 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.806292, event: all_read, request: pg_notify(24.30b(28) epoch 194470) v5 -7368> 2014-08-13 17:22:48.930021 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.929983, event: dispatched, request: pg_notify(24.30b(28) epoch 194470) v5 -7367> 2014-08-13 17:22:48.930031 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.930031, event: waiting_for_osdmap, request: pg_notify(24.30b(28) epoch 194470) v5 -7366> 2014-08-13 17:22:48.930039 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.930039, event: started, request: pg_notify(24.30b(28) epoch 194470) v5 -7365> 2014-08-13 17:22:48.930061 7f74d8735700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:22:48.930061, event: done, request: pg_notify(24.30b(28) epoch 194470) v5 -7364> 2014-08-13 17:22:48.928505 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Start 0.000227 0 0.000000 -7363> 2014-08-13 17:22:48.930090 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 2 ==== pg_notify(22.619(22) epoch 194470) v5 ==== 1948+0+0 (737148708 0 0) 0x4ac7e00 con 0x4902100 -7362> 2014-08-13 17:22:48.930108 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.819436, event: header_read, request: pg_notify(22.619(22) epoch 194470) v5 -7361> 2014-08-13 17:22:48.930103 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194449/194389 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started/Stray -7360> 2014-08-13 17:22:48.930115 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.819438, event: throttled, request: pg_notify(22.619(22) epoch 194470) v5 -7359> 2014-08-13 17:22:48.930121 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.819537, event: all_read, request: pg_notify(22.619(22) epoch 194470) v5 -7358> 2014-08-13 17:22:48.930126 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.930106, event: dispatched, request: pg_notify(22.619(22) epoch 194470) v5 -7357> 2014-08-13 17:22:48.930132 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.930131, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194470) v5 -7356> 2014-08-13 17:22:48.930139 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.930138, event: started, request: pg_notify(22.619(22) epoch 194470) v5 -7355> 2014-08-13 17:22:48.930158 7f74d8735700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:22:48.930157, event: done, request: pg_notify(22.619(22) epoch 194470) v5 -7354> 2014-08-13 17:22:48.930194 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 2 ==== pg_notify(22.3d9(42) epoch 194470) v5 ==== 2984+0+0 (7658140 0 0) 0x4ac7380 con 0x4903700 -7353> 2014-08-13 17:22:48.930213 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.814280, event: header_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -7352> 2014-08-13 17:22:48.930219 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.814314, event: throttled, request: pg_notify(22.3d9(42) epoch 194470) v5 -7351> 2014-08-13 17:22:48.930225 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.814591, event: all_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -7350> 2014-08-13 17:22:48.930230 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.930210, event: dispatched, request: pg_notify(22.3d9(42) epoch 194470) v5 -7349> 2014-08-13 17:22:48.930236 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.930236, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194470) v5 -7348> 2014-08-13 17:22:48.930243 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.930243, event: started, request: pg_notify(22.3d9(42) epoch 194470) v5 -7347> 2014-08-13 17:22:48.930262 7f74d8735700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:22:48.930261, event: done, request: pg_notify(22.3d9(42) epoch 194470) v5 -7346> 2014-08-13 17:22:48.930278 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7345> 2014-08-13 17:22:48.930292 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 2 ==== pg_notify(22.481(32) epoch 194470) v5 ==== 2518+0+0 (2179313187 0 0) 0x49e0a80 con 0x4903440 -7344> 2014-08-13 17:22:48.930309 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.819533, event: header_read, request: pg_notify(22.481(32) epoch 194470) v5 -7343> 2014-08-13 17:22:48.930315 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.819535, event: throttled, request: pg_notify(22.481(32) epoch 194470) v5 -7342> 2014-08-13 17:22:48.930305 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.225576 2 0.000138 -7341> 2014-08-13 17:22:48.930321 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.820106, event: all_read, request: pg_notify(22.481(32) epoch 194470) v5 -7340> 2014-08-13 17:22:48.930326 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.930306, event: dispatched, request: pg_notify(22.481(32) epoch 194470) v5 -7339> 2014-08-13 17:22:48.930333 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.930333, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194470) v5 -7338> 2014-08-13 17:22:48.930341 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.930340, event: started, request: pg_notify(22.481(32) epoch 194470) v5 -7337> 2014-08-13 17:22:48.930327 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.218254 3 0.000341 -7336> 2014-08-13 17:22:48.930360 7f74d8735700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:22:48.930359, event: done, request: pg_notify(22.481(32) epoch 194470) v5 -7335> 2014-08-13 17:22:48.930359 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started 0.218408 0 0.000000 -7334> 2014-08-13 17:22:48.930326 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.225610 0 0.000000 -7333> 2014-08-13 17:22:48.930380 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194447/194448/192969) [356,4,334]/[356,4,32] r=1 lpr=194449 pi=193150-194447/23 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset -7332> 2014-08-13 17:22:48.930390 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 2 ==== pg_notify(22.22c(22) epoch 194470) v5 ==== 1952+0+0 (46188357 0 0) 0x49e0540 con 0x4903180 -7331> 2014-08-13 17:22:48.930385 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.225684 0 0.000000 -7330> 2014-08-13 17:22:48.930405 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.819750, event: header_read, request: pg_notify(22.22c(22) epoch 194470) v5 -7329> 2014-08-13 17:22:48.930412 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.819752, event: throttled, request: pg_notify(22.22c(22) epoch 194470) v5 -7328> 2014-08-13 17:22:48.930406 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started 0.225763 0 0.000000 -7327> 2014-08-13 17:22:48.930418 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.819842, event: all_read, request: pg_notify(22.22c(22) epoch 194470) v5 -7326> 2014-08-13 17:22:48.930427 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.930403, event: dispatched, request: pg_notify(22.22c(22) epoch 194470) v5 -7325> 2014-08-13 17:22:48.930433 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.930433, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194470) v5 -7324> 2014-08-13 17:22:48.930429 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,353,56] r=0 lpr=194449 pi=193306-194447/18 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset -7323> 2014-08-13 17:22:48.930440 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.930440, event: started, request: pg_notify(22.22c(22) epoch 194470) v5 -7322> 2014-08-13 17:22:48.930461 7f74d8735700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:22:48.930460, event: done, request: pg_notify(22.22c(22) epoch 194470) v5 -7321> 2014-08-13 17:22:48.930480 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a85580 con 0x4904a40 -7320> 2014-08-13 17:22:48.930498 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7319> 2014-08-13 17:22:48.930514 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 2 ==== pg_notify(22.619(22) epoch 194470) v5 ==== 1948+0+0 (737148708 0 0) 0x49e0c40 con 0x4904780 -7318> 2014-08-13 17:22:48.930529 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.820479, event: header_read, request: pg_notify(22.619(22) epoch 194470) v5 -7317> 2014-08-13 17:22:48.930535 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.820481, event: throttled, request: pg_notify(22.619(22) epoch 194470) v5 -7316> 2014-08-13 17:22:48.930541 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.820584, event: all_read, request: pg_notify(22.619(22) epoch 194470) v5 -7315> 2014-08-13 17:22:48.930546 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.930527, event: dispatched, request: pg_notify(22.619(22) epoch 194470) v5 -7314> 2014-08-13 17:22:48.930552 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.930552, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194470) v5 -7313> 2014-08-13 17:22:48.930559 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.930559, event: started, request: pg_notify(22.619(22) epoch 194470) v5 -7312> 2014-08-13 17:22:48.930577 7f74d8735700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:22:48.930577, event: done, request: pg_notify(22.619(22) epoch 194470) v5 -7311> 2014-08-13 17:22:48.930595 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 2 ==== pg_notify(22.22c(1) epoch 194470) v5 ==== 777+0+0 (3946482321 0 0) 0x4ac7c40 con 0x49044c0 -7310> 2014-08-13 17:22:48.930609 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.819401, event: header_read, request: pg_notify(22.22c(1) epoch 194470) v5 -7309> 2014-08-13 17:22:48.930615 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.819403, event: throttled, request: pg_notify(22.22c(1) epoch 194470) v5 -7308> 2014-08-13 17:22:48.930621 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.819489, event: all_read, request: pg_notify(22.22c(1) epoch 194470) v5 -7307> 2014-08-13 17:22:48.930626 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.930607, event: dispatched, request: pg_notify(22.22c(1) epoch 194470) v5 -7306> 2014-08-13 17:22:48.930631 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.930631, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194470) v5 -7305> 2014-08-13 17:22:48.930638 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.930638, event: started, request: pg_notify(22.22c(1) epoch 194470) v5 -7304> 2014-08-13 17:22:48.930657 7f74d8735700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:22:48.930657, event: done, request: pg_notify(22.22c(1) epoch 194470) v5 -7303> 2014-08-13 17:22:48.930674 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 2 ==== pg_notify(22.775(22) epoch 194470) v5 ==== 1948+0+0 (899104279 0 0) 0x49e0380 con 0x49070c0 -7302> 2014-08-13 17:22:48.930689 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.819643, event: header_read, request: pg_notify(22.775(22) epoch 194470) v5 -7301> 2014-08-13 17:22:48.930695 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.819645, event: throttled, request: pg_notify(22.775(22) epoch 194470) v5 -7300> 2014-08-13 17:22:48.930700 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.819742, event: all_read, request: pg_notify(22.775(22) epoch 194470) v5 -7299> 2014-08-13 17:22:48.930705 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.930687, event: dispatched, request: pg_notify(22.775(22) epoch 194470) v5 -7298> 2014-08-13 17:22:48.930711 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.930711, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194470) v5 -7297> 2014-08-13 17:22:48.930718 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.930718, event: started, request: pg_notify(22.775(22) epoch 194470) v5 -7296> 2014-08-13 17:22:48.931111 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000683 15 0.000392 -7295> 2014-08-13 17:22:48.931113 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.000733 15 0.000242 -7294> 2014-08-13 17:22:48.931144 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started -7293> 2014-08-13 17:22:48.931169 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Start -7292> 2014-08-13 17:22:48.931198 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -7291> 2014-08-13 17:22:48.931141 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started -7290> 2014-08-13 17:22:48.931221 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start -7289> 2014-08-13 17:22:48.931221 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000052 0 0.000000 -7288> 2014-08-13 17:22:48.931251 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7287> 2014-08-13 17:22:48.931259 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194449/194389 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -7286> 2014-08-13 17:22:48.931279 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000058 0 0.000000 -7285> 2014-08-13 17:22:48.931303 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7284> 2014-08-13 17:22:48.931322 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7283> 2014-08-13 17:22:48.931340 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7282> 2014-08-13 17:22:48.931417 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.218661 3 0.000195 -7281> 2014-08-13 17:22:48.931449 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.218765 0 0.000000 -7280> 2014-08-13 17:22:48.931472 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192958) [397,343,4] r=2 lpr=194449 pi=192965-194447/16 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -7279> 2014-08-13 17:22:48.931593 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194470) v5 -- ?+0 0x57d4d00 con 0x4758840 -7278> 2014-08-13 17:22:48.931628 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- pg_notify(22.2b9(19) epoch 194470) v5 -- ?+0 0x57d4b40 con 0x4758b00 -7277> 2014-08-13 17:22:48.931648 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(28) epoch 194470) v5 -- ?+0 0x57d4980 con 0x486e040 -7276> 2014-08-13 17:22:48.931669 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6810/59615 -- pg_query(22.775 epoch 194470) v3 -- ?+0 0x29c5640 con 0x4900420 -7275> 2014-08-13 17:22:48.931699 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6826/11446 -- pg_query(22.775 epoch 194470) v3 -- ?+0 0x4eb0f00 con 0x475aec0 -7274> 2014-08-13 17:22:48.931749 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6830/12680 -- pg_query(22.775 epoch 194470) v3 -- ?+0 0x29c3a20 con 0x49070c0 -7273> 2014-08-13 17:22:48.931777 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6836/6227 -- pg_query(22.775 epoch 194470) v3 -- ?+0 0x29c7bc0 con 0x532c360 -7272> 2014-08-13 17:22:48.931794 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6824/4037 -- pg_query(22.775 epoch 194470) v3 -- ?+0 0x29c2580 con 0x4d58840 -7271> 2014-08-13 17:22:48.931848 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.000375 15 0.000175 -7270> 2014-08-13 17:22:48.931877 7f74d8735700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:22:48.931876, event: done, request: pg_notify(22.775(22) epoch 194470) v5 -7269> 2014-08-13 17:22:48.931879 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -7268> 2014-08-13 17:22:48.931910 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -7267> 2014-08-13 17:22:48.931929 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7266> 2014-08-13 17:22:48.931930 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 2 ==== pg_notify(22.1f1(1) epoch 194470) v5 ==== 777+0+0 (884427129 0 0) 0x49e2140 con 0x4906e00 -7265> 2014-08-13 17:22:48.931952 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.822417, event: header_read, request: pg_notify(22.1f1(1) epoch 194470) v5 -7264> 2014-08-13 17:22:48.931951 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -7263> 2014-08-13 17:22:48.931964 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.822418, event: throttled, request: pg_notify(22.1f1(1) epoch 194470) v5 -7262> 2014-08-13 17:22:48.931983 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.822491, event: all_read, request: pg_notify(22.1f1(1) epoch 194470) v5 -7261> 2014-08-13 17:22:48.931979 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -7260> 2014-08-13 17:22:48.931902 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.226907 3 0.000111 -7259> 2014-08-13 17:22:48.931994 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.931949, event: dispatched, request: pg_notify(22.1f1(1) epoch 194470) v5 -7258> 2014-08-13 17:22:48.932015 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.932015, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194470) v5 -7257> 2014-08-13 17:22:48.932029 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.932029, event: started, request: pg_notify(22.1f1(1) epoch 194470) v5 -7256> 2014-08-13 17:22:48.932010 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started 0.227066 0 0.000000 -7255> 2014-08-13 17:22:48.932044 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192968) [294,327,4] r=2 lpr=194449 pi=192977-194447/16 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset -7254> 2014-08-13 17:22:48.932058 7f74d8735700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:22:48.932058, event: done, request: pg_notify(22.1f1(1) epoch 194470) v5 -7253> 2014-08-13 17:22:48.932095 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 2 ==== pg_notify(2.2a(2) epoch 194470) v5 ==== 804+0+0 (1954668336 0 0) 0x49e2d80 con 0x4906b40 -7252> 2014-08-13 17:22:48.932117 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.824018, event: header_read, request: pg_notify(2.2a(2) epoch 194470) v5 -7251> 2014-08-13 17:22:48.932125 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.824022, event: throttled, request: pg_notify(2.2a(2) epoch 194470) v5 -7250> 2014-08-13 17:22:48.932131 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.824350, event: all_read, request: pg_notify(2.2a(2) epoch 194470) v5 -7249> 2014-08-13 17:22:48.932116 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.218643 3 0.000297 -7248> 2014-08-13 17:22:48.932141 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.932114, event: dispatched, request: pg_notify(2.2a(2) epoch 194470) v5 -7247> 2014-08-13 17:22:48.932151 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.932151, event: waiting_for_osdmap, request: pg_notify(2.2a(2) epoch 194470) v5 -7246> 2014-08-13 17:22:48.932145 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.218813 0 0.000000 -7245> 2014-08-13 17:22:48.932163 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194447/194448/192976) [30,335,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -7244> 2014-08-13 17:22:48.932191 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.932191, event: started, request: pg_notify(2.2a(2) epoch 194470) v5 -7243> 2014-08-13 17:22:48.932221 7f74d8735700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:22:48.932221, event: done, request: pg_notify(2.2a(2) epoch 194470) v5 -7242> 2014-08-13 17:22:48.932241 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 2 ==== pg_notify(22.f3(1) epoch 194470) v5 ==== 777+0+0 (3929559114 0 0) 0x49e1f80 con 0x4906880 -7241> 2014-08-13 17:22:48.932261 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.822404, event: header_read, request: pg_notify(22.f3(1) epoch 194470) v5 -7240> 2014-08-13 17:22:48.932271 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.822407, event: throttled, request: pg_notify(22.f3(1) epoch 194470) v5 -7239> 2014-08-13 17:22:48.932280 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.822491, event: all_read, request: pg_notify(22.f3(1) epoch 194470) v5 -7238> 2014-08-13 17:22:48.932289 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.932259, event: dispatched, request: pg_notify(22.f3(1) epoch 194470) v5 -7237> 2014-08-13 17:22:48.932300 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.932300, event: waiting_for_osdmap, request: pg_notify(22.f3(1) epoch 194470) v5 -7236> 2014-08-13 17:22:48.932311 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.932311, event: started, request: pg_notify(22.f3(1) epoch 194470) v5 -7235> 2014-08-13 17:22:48.932334 7f74d8735700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:22:48.932333, event: done, request: pg_notify(22.f3(1) epoch 194470) v5 -7234> 2014-08-13 17:22:48.932358 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 2 ==== pg_notify(22.3af(22) epoch 194470) v5 ==== 1948+0+0 (2286998326 0 0) 0x49e1340 con 0x4d58840 -7233> 2014-08-13 17:22:48.932377 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.821788, event: header_read, request: pg_notify(22.3af(22) epoch 194470) v5 -7232> 2014-08-13 17:22:48.932384 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.821789, event: throttled, request: pg_notify(22.3af(22) epoch 194470) v5 -7231> 2014-08-13 17:22:48.932389 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.821876, event: all_read, request: pg_notify(22.3af(22) epoch 194470) v5 -7230> 2014-08-13 17:22:48.932395 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.932375, event: dispatched, request: pg_notify(22.3af(22) epoch 194470) v5 -7229> 2014-08-13 17:22:48.932405 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.932405, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194470) v5 -7228> 2014-08-13 17:22:48.932416 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.932416, event: started, request: pg_notify(22.3af(22) epoch 194470) v5 -7227> 2014-08-13 17:22:48.932461 7f74d8735700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:22:48.932461, event: done, request: pg_notify(22.3af(22) epoch 194470) v5 -7226> 2014-08-13 17:22:48.932502 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 2 ==== pg_notify(22.84(2) epoch 194470) v5 ==== 828+0+0 (950298263 0 0) 0x49e24c0 con 0x4d58580 -7225> 2014-08-13 17:22:48.932528 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.822453, event: header_read, request: pg_notify(22.84(2) epoch 194470) v5 -7224> 2014-08-13 17:22:48.932540 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.822456, event: throttled, request: pg_notify(22.84(2) epoch 194470) v5 -7223> 2014-08-13 17:22:48.932552 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.822533, event: all_read, request: pg_notify(22.84(2) epoch 194470) v5 -7222> 2014-08-13 17:22:48.932562 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.932525, event: dispatched, request: pg_notify(22.84(2) epoch 194470) v5 -7221> 2014-08-13 17:22:48.932571 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.932571, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194470) v5 -7220> 2014-08-13 17:22:48.932592 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.932592, event: started, request: pg_notify(22.84(2) epoch 194470) v5 -7219> 2014-08-13 17:22:48.932617 7f74d8735700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:22:48.932617, event: done, request: pg_notify(22.84(2) epoch 194470) v5 -7218> 2014-08-13 17:22:48.932641 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 2 ==== pg_notify(22.99(1) epoch 194470) v5 ==== 777+0+0 (309868313 0 0) 0x49e0e00 con 0x4d582c0 -7217> 2014-08-13 17:22:48.932661 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.821448, event: header_read, request: pg_notify(22.99(1) epoch 194470) v5 -7216> 2014-08-13 17:22:48.932671 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.821449, event: throttled, request: pg_notify(22.99(1) epoch 194470) v5 -7215> 2014-08-13 17:22:48.932680 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.821531, event: all_read, request: pg_notify(22.99(1) epoch 194470) v5 -7214> 2014-08-13 17:22:48.932689 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.932659, event: dispatched, request: pg_notify(22.99(1) epoch 194470) v5 -7213> 2014-08-13 17:22:48.932699 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.932699, event: waiting_for_osdmap, request: pg_notify(22.99(1) epoch 194470) v5 -7212> 2014-08-13 17:22:48.932710 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.932710, event: started, request: pg_notify(22.99(1) epoch 194470) v5 -7211> 2014-08-13 17:22:48.932733 7f74d8735700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:22:48.932733, event: done, request: pg_notify(22.99(1) epoch 194470) v5 -7210> 2014-08-13 17:22:48.932722 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.000679 15 0.000314 -7209> 2014-08-13 17:22:48.932757 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started -7208> 2014-08-13 17:22:48.932764 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 2 ==== pg_notify(22.481(39) epoch 194470) v5 ==== 2903+0+0 (4120949430 0 0) 0x49e1a40 con 0x4d58000 -7207> 2014-08-13 17:22:48.932782 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.821767, event: header_read, request: pg_notify(22.481(39) epoch 194470) v5 -7206> 2014-08-13 17:22:48.932777 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start -7205> 2014-08-13 17:22:48.932793 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.821772, event: throttled, request: pg_notify(22.481(39) epoch 194470) v5 -7204> 2014-08-13 17:22:48.932803 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.822258, event: all_read, request: pg_notify(22.481(39) epoch 194470) v5 -7203> 2014-08-13 17:22:48.932795 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7202> 2014-08-13 17:22:48.932816 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.932780, event: dispatched, request: pg_notify(22.481(39) epoch 194470) v5 -7201> 2014-08-13 17:22:48.932825 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.932825, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194470) v5 -7200> 2014-08-13 17:22:48.932837 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.932836, event: started, request: pg_notify(22.481(39) epoch 194470) v5 -7199> 2014-08-13 17:22:48.932827 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000051 0 0.000000 -7198> 2014-08-13 17:22:48.932850 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray -7197> 2014-08-13 17:22:48.932861 7f74d8735700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:22:48.932861, event: done, request: pg_notify(22.481(39) epoch 194470) v5 -7196> 2014-08-13 17:22:48.932908 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 2 ==== pg_notify(24.9b(1) epoch 194470) v5 ==== 811+0+0 (1423586113 0 0) 0x49e2f40 con 0x4d51ce0 -7195> 2014-08-13 17:22:48.932930 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.824195, event: header_read, request: pg_notify(24.9b(1) epoch 194470) v5 -7194> 2014-08-13 17:22:48.932940 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.824198, event: throttled, request: pg_notify(24.9b(1) epoch 194470) v5 -7193> 2014-08-13 17:22:48.932949 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.824296, event: all_read, request: pg_notify(24.9b(1) epoch 194470) v5 -7192> 2014-08-13 17:22:48.932959 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.932927, event: dispatched, request: pg_notify(24.9b(1) epoch 194470) v5 -7191> 2014-08-13 17:22:48.932969 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.932968, event: waiting_for_osdmap, request: pg_notify(24.9b(1) epoch 194470) v5 -7190> 2014-08-13 17:22:48.932980 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.932980, event: started, request: pg_notify(24.9b(1) epoch 194470) v5 -7189> 2014-08-13 17:22:48.933004 7f74d8735700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:22:48.933003, event: done, request: pg_notify(24.9b(1) epoch 194470) v5 -7188> 2014-08-13 17:22:48.932995 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.000832 15 0.000200 -7187> 2014-08-13 17:22:48.933022 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 2 ==== pg_notify(24.136(21) epoch 194470) v5 ==== 1919+0+0 (1355132185 0 0) 0x49e1c00 con 0x4d51a20 -7186> 2014-08-13 17:22:48.933031 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -7185> 2014-08-13 17:22:48.933008 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started/Stray 0.227536 3 0.000198 -7184> 2014-08-13 17:22:48.933045 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.822114, event: header_read, request: pg_notify(24.136(21) epoch 194470) v5 -7183> 2014-08-13 17:22:48.933051 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -7182> 2014-08-13 17:22:48.933072 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.822116, event: throttled, request: pg_notify(24.136(21) epoch 194470) v5 -7181> 2014-08-13 17:22:48.933064 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started 0.227690 0 0.000000 -7180> 2014-08-13 17:22:48.933080 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.822282, event: all_read, request: pg_notify(24.136(21) epoch 194470) v5 -7179> 2014-08-13 17:22:48.933080 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7178> 2014-08-13 17:22:48.933098 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.933043, event: dispatched, request: pg_notify(24.136(21) epoch 194470) v5 -7177> 2014-08-13 17:22:48.933094 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194447/194448/192955) [346,4,304]/[346,4,288] r=1 lpr=194449 pi=193125-194447/21 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset -7176> 2014-08-13 17:22:48.933111 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.933110, event: waiting_for_osdmap, request: pg_notify(24.136(21) epoch 194470) v5 -7175> 2014-08-13 17:22:48.933101 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000049 0 0.000000 -7174> 2014-08-13 17:22:48.933125 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.933125, event: started, request: pg_notify(24.136(21) epoch 194470) v5 -7173> 2014-08-13 17:22:48.933134 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194449/194449 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -7172> 2014-08-13 17:22:48.933147 7f74d8735700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:22:48.933147, event: done, request: pg_notify(24.136(21) epoch 194470) v5 -7171> 2014-08-13 17:22:48.933193 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e369c0 con 0x4d511e0 -7170> 2014-08-13 17:22:48.933220 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7169> 2014-08-13 17:22:48.933259 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 2 ==== pg_notify(22.f3(18) epoch 194470) v5 ==== 1720+0+0 (3451712932 0 0) 0x49e7380 con 0x4d54200 -7168> 2014-08-13 17:22:48.933279 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.825308, event: header_read, request: pg_notify(22.f3(18) epoch 194470) v5 -7167> 2014-08-13 17:22:48.933291 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.825310, event: throttled, request: pg_notify(22.f3(18) epoch 194470) v5 -7166> 2014-08-13 17:22:48.933300 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.825413, event: all_read, request: pg_notify(22.f3(18) epoch 194470) v5 -7165> 2014-08-13 17:22:48.933309 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.933276, event: dispatched, request: pg_notify(22.f3(18) epoch 194470) v5 -7164> 2014-08-13 17:22:48.933299 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7163> 2014-08-13 17:22:48.933319 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.933319, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194470) v5 -7162> 2014-08-13 17:22:48.933334 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.219410 2 0.000329 -7161> 2014-08-13 17:22:48.933353 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.933353, event: started, request: pg_notify(22.f3(18) epoch 194470) v5 -7160> 2014-08-13 17:22:48.933357 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.219478 0 0.000000 -7159> 2014-08-13 17:22:48.933392 7f74d8735700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:22:48.933392, event: done, request: pg_notify(22.f3(18) epoch 194470) v5 -7158> 2014-08-13 17:22:48.933386 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.219528 0 0.000000 -7157> 2014-08-13 17:22:48.933420 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.219682 0 0.000000 -7156> 2014-08-13 17:22:48.933432 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 2 ==== pg_notify(22.39d(1) epoch 194470) v5 ==== 777+0+0 (380101991 0 0) 0x49e71c0 con 0x4d53c80 -7155> 2014-08-13 17:22:48.933443 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/194448) [4,50,289] r=0 lpr=194454 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -7154> 2014-08-13 17:22:48.933453 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.825232, event: header_read, request: pg_notify(22.39d(1) epoch 194470) v5 -7153> 2014-08-13 17:22:48.933474 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.825233, event: throttled, request: pg_notify(22.39d(1) epoch 194470) v5 -7152> 2014-08-13 17:22:48.933484 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.825296, event: all_read, request: pg_notify(22.39d(1) epoch 194470) v5 -7151> 2014-08-13 17:22:48.933494 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.933450, event: dispatched, request: pg_notify(22.39d(1) epoch 194470) v5 -7150> 2014-08-13 17:22:48.933503 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.933503, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194470) v5 -7149> 2014-08-13 17:22:48.933514 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.933514, event: started, request: pg_notify(22.39d(1) epoch 194470) v5 -7148> 2014-08-13 17:22:48.933538 7f74d8735700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:22:48.933538, event: done, request: pg_notify(22.39d(1) epoch 194470) v5 -7147> 2014-08-13 17:22:48.933568 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 2 ==== pg_notify(22.4df(20) epoch 194470) v5 ==== 1834+0+0 (819295757 0 0) 0x49e7540 con 0x4d56040 -7146> 2014-08-13 17:22:48.933589 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.825816, event: header_read, request: pg_notify(22.4df(20) epoch 194470) v5 -7145> 2014-08-13 17:22:48.933600 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.825818, event: throttled, request: pg_notify(22.4df(20) epoch 194470) v5 -7144> 2014-08-13 17:22:48.933609 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.825908, event: all_read, request: pg_notify(22.4df(20) epoch 194470) v5 -7143> 2014-08-13 17:22:48.933618 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.933587, event: dispatched, request: pg_notify(22.4df(20) epoch 194470) v5 -7142> 2014-08-13 17:22:48.933628 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.933628, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194470) v5 -7141> 2014-08-13 17:22:48.933639 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.933638, event: started, request: pg_notify(22.4df(20) epoch 194470) v5 -7140> 2014-08-13 17:22:48.933661 7f74d8735700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:22:48.933661, event: done, request: pg_notify(22.4df(20) epoch 194470) v5 -7139> 2014-08-13 17:22:48.933672 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Reset 0.000578 15 0.000245 -7138> 2014-08-13 17:22:48.933688 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 2 ==== pg_notify(22.362(22) epoch 194470) v5 ==== 1948+0+0 (1366328122 0 0) 0x49e7700 con 0x4d55d80 -7137> 2014-08-13 17:22:48.933705 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.825849, event: header_read, request: pg_notify(22.362(22) epoch 194470) v5 -7136> 2014-08-13 17:22:48.933703 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started -7135> 2014-08-13 17:22:48.933717 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.825851, event: throttled, request: pg_notify(22.362(22) epoch 194470) v5 -7134> 2014-08-13 17:22:48.933727 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.826021, event: all_read, request: pg_notify(22.362(22) epoch 194470) v5 -7133> 2014-08-13 17:22:48.933721 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Start -7132> 2014-08-13 17:22:48.933736 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.933703, event: dispatched, request: pg_notify(22.362(22) epoch 194470) v5 -7131> 2014-08-13 17:22:48.933737 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: transitioning to Stray -7130> 2014-08-13 17:22:48.933746 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.933746, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194470) v5 -7129> 2014-08-13 17:22:48.933768 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.933768, event: started, request: pg_notify(22.362(22) epoch 194470) v5 -7128> 2014-08-13 17:22:48.933761 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Start 0.000041 0 0.000000 -7127> 2014-08-13 17:22:48.933781 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194449/194388 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started/Stray -7126> 2014-08-13 17:22:48.933793 7f74d8735700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:22:48.933793, event: done, request: pg_notify(22.362(22) epoch 194470) v5 -7125> 2014-08-13 17:22:48.933814 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 2 ==== pg_notify(24.9b(26) epoch 194470) v5 ==== 2214+0+0 (801795 0 0) 0x49e78c0 con 0x4d55ac0 -7124> 2014-08-13 17:22:48.933834 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.826014, event: header_read, request: pg_notify(24.9b(26) epoch 194470) v5 -7123> 2014-08-13 17:22:48.933844 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.826016, event: throttled, request: pg_notify(24.9b(26) epoch 194470) v5 -7122> 2014-08-13 17:22:48.933853 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.826114, event: all_read, request: pg_notify(24.9b(26) epoch 194470) v5 -7121> 2014-08-13 17:22:48.933862 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.933831, event: dispatched, request: pg_notify(24.9b(26) epoch 194470) v5 -7120> 2014-08-13 17:22:48.933872 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.933872, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194470) v5 -7119> 2014-08-13 17:22:48.933884 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.933884, event: started, request: pg_notify(24.9b(26) epoch 194470) v5 -7118> 2014-08-13 17:22:48.933907 7f74d8735700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:22:48.933906, event: done, request: pg_notify(24.9b(26) epoch 194470) v5 -7117> 2014-08-13 17:22:48.933905 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000463 15 0.000283 -7116> 2014-08-13 17:22:48.933928 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.227758 3 0.000226 -7115> 2014-08-13 17:22:48.933943 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -7114> 2014-08-13 17:22:48.933940 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 2 ==== pg_notify(22.481(37) epoch 194470) v5 ==== 2801+0+0 (203028685 0 0) 0x49e7c40 con 0x4d55540 -7113> 2014-08-13 17:22:48.933960 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -7112> 2014-08-13 17:22:48.933971 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.828267, event: header_read, request: pg_notify(22.481(37) epoch 194470) v5 -7111> 2014-08-13 17:22:48.933964 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started 0.227865 0 0.000000 -7110> 2014-08-13 17:22:48.933980 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.828268, event: throttled, request: pg_notify(22.481(37) epoch 194470) v5 -7109> 2014-08-13 17:22:48.933979 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7108> 2014-08-13 17:22:48.933984 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192946) [334,289,4] r=2 lpr=194449 pi=192974-194447/16 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset -7107> 2014-08-13 17:22:48.933991 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.828379, event: all_read, request: pg_notify(22.481(37) epoch 194470) v5 -7106> 2014-08-13 17:22:48.934009 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.933969, event: dispatched, request: pg_notify(22.481(37) epoch 194470) v5 -7105> 2014-08-13 17:22:48.934006 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000045 0 0.000000 -7104> 2014-08-13 17:22:48.934022 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.934021, event: waiting_for_osdmap, request: pg_notify(22.481(37) epoch 194470) v5 -7103> 2014-08-13 17:22:48.934030 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.934030, event: started, request: pg_notify(22.481(37) epoch 194470) v5 -7102> 2014-08-13 17:22:48.934032 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7101> 2014-08-13 17:22:48.934058 7f74d8735700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:22:48.934058, event: done, request: pg_notify(22.481(37) epoch 194470) v5 -7100> 2014-08-13 17:22:48.934053 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7099> 2014-08-13 17:22:48.934072 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7098> 2014-08-13 17:22:48.934091 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 2 ==== pg_notify(22.4df(1) epoch 194470) v5 ==== 777+0+0 (3297624547 0 0) 0x49e7a80 con 0x4d57d20 -7097> 2014-08-13 17:22:48.934112 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.826326, event: header_read, request: pg_notify(22.4df(1) epoch 194470) v5 -7096> 2014-08-13 17:22:48.934122 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.826328, event: throttled, request: pg_notify(22.4df(1) epoch 194470) v5 -7095> 2014-08-13 17:22:48.934133 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.826419, event: all_read, request: pg_notify(22.4df(1) epoch 194470) v5 -7094> 2014-08-13 17:22:48.934142 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.934110, event: dispatched, request: pg_notify(22.4df(1) epoch 194470) v5 -7093> 2014-08-13 17:22:48.934151 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.934151, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194470) v5 -7092> 2014-08-13 17:22:48.934170 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.934170, event: started, request: pg_notify(22.4df(1) epoch 194470) v5 -7091> 2014-08-13 17:22:48.934201 7f74d8735700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:22:48.934201, event: done, request: pg_notify(22.4df(1) epoch 194470) v5 -7090> 2014-08-13 17:22:48.934224 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 2 ==== pg_notify(22.3d9(42) epoch 194470) v5 ==== 2984+0+0 (569097740 0 0) 0x4f516c0 con 0x4d57a60 -7089> 2014-08-13 17:22:48.934219 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] state: Peering, affected_by_map, going to Reset -7088> 2014-08-13 17:22:48.934244 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.833484, event: header_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -7087> 2014-08-13 17:22:48.934254 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.833486, event: throttled, request: pg_notify(22.3d9(42) epoch 194470) v5 -7086> 2014-08-13 17:22:48.934246 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] exit Started/Primary/Peering/GetInfo 0.218387 2 0.000245 -7085> 2014-08-13 17:22:48.934264 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.833601, event: all_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -7084> 2014-08-13 17:22:48.934273 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.934241, event: dispatched, request: pg_notify(22.3d9(42) epoch 194470) v5 -7083> 2014-08-13 17:22:48.934269 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] exit Started/Primary/Peering 0.218440 0 0.000000 -7082> 2014-08-13 17:22:48.934283 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.934283, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194470) v5 -7081> 2014-08-13 17:22:48.934304 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.934304, event: started, request: pg_notify(22.3d9(42) epoch 194470) v5 -7080> 2014-08-13 17:22:48.934298 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Started/Primary 0.218491 0 0.000000 -7079> 2014-08-13 17:22:48.934330 7f74d8735700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:22:48.934330, event: done, request: pg_notify(22.3d9(42) epoch 194470) v5 -7078> 2014-08-13 17:22:48.934319 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Started 0.218628 0 0.000000 -7077> 2014-08-13 17:22:48.934355 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194447/194448/194448) [4,344,296]/[4,296,320] r=0 lpr=194449 pi=193679-194447/22 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset -7076> 2014-08-13 17:22:48.934367 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.357 192.168.1.102:6834/17224 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e33840 con 0x4d37a60 -7075> 2014-08-13 17:22:48.934386 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7074> 2014-08-13 17:22:48.934374 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.000390 15 0.000180 -7073> 2014-08-13 17:22:48.934399 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started -7072> 2014-08-13 17:22:48.934411 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.353 192.168.1.108:6836/6227 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4801680 con 0x532c360 -7071> 2014-08-13 17:22:48.934421 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start -7070> 2014-08-13 17:22:48.934431 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7069> 2014-08-13 17:22:48.934442 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7068> 2014-08-13 17:22:48.934463 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000043 0 0.000000 -7067> 2014-08-13 17:22:48.934478 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 4 ==== pg_notify(24.30b(24) epoch 194470) v5 ==== 2096+0+0 (1338007029 0 0) 0x466e900 con 0x45f1760 -7066> 2014-08-13 17:22:48.934485 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray -7065> 2014-08-13 17:22:48.934499 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.928810, event: header_read, request: pg_notify(24.30b(24) epoch 194470) v5 -7064> 2014-08-13 17:22:48.934507 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.928813, event: throttled, request: pg_notify(24.30b(24) epoch 194470) v5 -7063> 2014-08-13 17:22:48.934517 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.928923, event: all_read, request: pg_notify(24.30b(24) epoch 194470) v5 -7062> 2014-08-13 17:22:48.934526 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.934497, event: dispatched, request: pg_notify(24.30b(24) epoch 194470) v5 -7061> 2014-08-13 17:22:48.934536 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.934536, event: waiting_for_osdmap, request: pg_notify(24.30b(24) epoch 194470) v5 -7060> 2014-08-13 17:22:48.934548 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.934547, event: started, request: pg_notify(24.30b(24) epoch 194470) v5 -7059> 2014-08-13 17:22:48.934581 7f74d8735700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:22:48.934581, event: done, request: pg_notify(24.30b(24) epoch 194470) v5 -7058> 2014-08-13 17:22:48.934619 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 3 ==== pg_notify(22.1d2(22) epoch 194470) v5 ==== 1952+0+0 (3049200412 0 0) 0x4ac0e00 con 0x45f1e40 -7057> 2014-08-13 17:22:48.934640 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.803927, event: header_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -7056> 2014-08-13 17:22:48.934627 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7055> 2014-08-13 17:22:48.934655 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.803928, event: throttled, request: pg_notify(22.1d2(22) epoch 194470) v5 -7054> 2014-08-13 17:22:48.934675 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.805659, event: all_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -7053> 2014-08-13 17:22:48.934670 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.228103 2 0.000179 -7052> 2014-08-13 17:22:48.934685 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.934637, event: dispatched, request: pg_notify(22.1d2(22) epoch 194470) v5 -7051> 2014-08-13 17:22:48.934695 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.934695, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194470) v5 -7050> 2014-08-13 17:22:48.934692 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.228136 0 0.000000 -7049> 2014-08-13 17:22:48.934716 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.934716, event: started, request: pg_notify(22.1d2(22) epoch 194470) v5 -7048> 2014-08-13 17:22:48.934718 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.228177 0 0.000000 -7047> 2014-08-13 17:22:48.934742 7f74d8735700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:22:48.934742, event: done, request: pg_notify(22.1d2(22) epoch 194470) v5 -7046> 2014-08-13 17:22:48.934742 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.228286 0 0.000000 -7045> 2014-08-13 17:22:48.934764 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 3 ==== pg_notify(22.84(2) epoch 194470) v5 ==== 828+0+0 (2765253268 0 0) 0x57d3800 con 0x45f3b20 -7044> 2014-08-13 17:22:48.934765 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194447/194448/194448) [4,313,318] r=0 lpr=194449 pi=193306-194447/18 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -7043> 2014-08-13 17:22:48.934787 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.923845, event: header_read, request: pg_notify(22.84(2) epoch 194470) v5 -7042> 2014-08-13 17:22:48.934796 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.923847, event: throttled, request: pg_notify(22.84(2) epoch 194470) v5 -7041> 2014-08-13 17:22:48.934805 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.923924, event: all_read, request: pg_notify(22.84(2) epoch 194470) v5 -7040> 2014-08-13 17:22:48.934814 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.934785, event: dispatched, request: pg_notify(22.84(2) epoch 194470) v5 -7039> 2014-08-13 17:22:48.934824 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.934823, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194470) v5 -7038> 2014-08-13 17:22:48.934835 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.934834, event: started, request: pg_notify(22.84(2) epoch 194470) v5 -7037> 2014-08-13 17:22:48.934857 7f74d8735700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:22:48.934857, event: done, request: pg_notify(22.84(2) epoch 194470) v5 -7036> 2014-08-13 17:22:48.934875 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 3 ==== pg_notify(22.22c epoch 194470) v5 ==== 694+0+0 (2976030002 0 0) 0x57d7a80 con 0x45f5540 -7035> 2014-08-13 17:22:48.934892 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.929211, event: header_read, request: pg_notify(22.22c epoch 194470) v5 -7034> 2014-08-13 17:22:48.934902 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.929213, event: throttled, request: pg_notify(22.22c epoch 194470) v5 -7033> 2014-08-13 17:22:48.934912 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.929288, event: all_read, request: pg_notify(22.22c epoch 194470) v5 -7032> 2014-08-13 17:22:48.934920 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.934890, event: dispatched, request: pg_notify(22.22c epoch 194470) v5 -7031> 2014-08-13 17:22:48.934902 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Reset 0.000547 15 0.000313 -7030> 2014-08-13 17:22:48.934934 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.934934, event: waiting_for_osdmap, request: pg_notify(22.22c epoch 194470) v5 -7029> 2014-08-13 17:22:48.934955 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.934955, event: started, request: pg_notify(22.22c epoch 194470) v5 -7028> 2014-08-13 17:22:48.934948 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started -7027> 2014-08-13 17:22:48.934967 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Start -7026> 2014-08-13 17:22:48.934982 7f74d8735700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:22:48.934981, event: done, request: pg_notify(22.22c epoch 194470) v5 -7025> 2014-08-13 17:22:48.934988 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: transitioning to Stray -7024> 2014-08-13 17:22:48.934997 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 3 ==== pg_notify(24.136(21) epoch 194470) v5 ==== 1919+0+0 (1355132185 0 0) 0x466f700 con 0x45f5960 -7023> 2014-08-13 17:22:48.935012 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.799459, event: header_read, request: pg_notify(24.136(21) epoch 194470) v5 -7022> 2014-08-13 17:22:48.935023 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.799459, event: throttled, request: pg_notify(24.136(21) epoch 194470) v5 -7021> 2014-08-13 17:22:48.935012 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Start 0.000044 0 0.000000 -7020> 2014-08-13 17:22:48.935036 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.799528, event: all_read, request: pg_notify(24.136(21) epoch 194470) v5 -7019> 2014-08-13 17:22:48.935044 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.935010, event: dispatched, request: pg_notify(24.136(21) epoch 194470) v5 -7018> 2014-08-13 17:22:48.935054 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.935054, event: waiting_for_osdmap, request: pg_notify(24.136(21) epoch 194470) v5 -7017> 2014-08-13 17:22:48.935047 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194443/194357 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started/Stray -7016> 2014-08-13 17:22:48.935065 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.935065, event: started, request: pg_notify(24.136(21) epoch 194470) v5 -7015> 2014-08-13 17:22:48.935089 7f74d8735700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:22:48.935088, event: done, request: pg_notify(24.136(21) epoch 194470) v5 -7014> 2014-08-13 17:22:48.935107 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.62 192.168.1.112:6824/5505 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4804a40 con 0x45f77a0 -7013> 2014-08-13 17:22:48.935127 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7012> 2014-08-13 17:22:48.935171 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.344 192.168.1.106:6812/6757 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4a84a40 con 0x45f6300 -7011> 2014-08-13 17:22:48.935193 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -7010> 2014-08-13 17:22:48.935207 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 3 ==== pg_notify(22.1f1(22) epoch 194470) v5 ==== 1948+0+0 (597292973 0 0) 0x49e1500 con 0x45f6f60 -7009> 2014-08-13 17:22:48.935202 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000436 15 0.000320 -7008> 2014-08-13 17:22:48.935233 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.821802, event: header_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -7007> 2014-08-13 17:22:48.935235 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -7006> 2014-08-13 17:22:48.935245 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.821804, event: throttled, request: pg_notify(22.1f1(22) epoch 194470) v5 -7005> 2014-08-13 17:22:48.935259 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -7004> 2014-08-13 17:22:48.935266 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.821915, event: all_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -7003> 2014-08-13 17:22:48.935281 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7002> 2014-08-13 17:22:48.935288 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.935230, event: dispatched, request: pg_notify(22.1f1(22) epoch 194470) v5 -7001> 2014-08-13 17:22:48.935282 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(19) epoch 194470) v5 -- ?+0 0x466b480 con 0x475a940 -7000> 2014-08-13 17:22:48.935312 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.935312, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194470) v5 -6999> 2014-08-13 17:22:48.935306 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000046 0 0.000000 -6998> 2014-08-13 17:22:48.935322 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.935322, event: started, request: pg_notify(22.1f1(22) epoch 194470) v5 -6997> 2014-08-13 17:22:48.935328 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_notify(24.181(27) epoch 194470) v5 -- ?+0 0x466a840 con 0x486bc80 -6996> 2014-08-13 17:22:48.935329 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6995> 2014-08-13 17:22:48.935345 7f74d8735700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:22:48.935345, event: done, request: pg_notify(22.1f1(22) epoch 194470) v5 -6994> 2014-08-13 17:22:48.935350 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.3b7(28) epoch 194470) v5 -- ?+0 0x2940380 con 0x45f5960 -6993> 2014-08-13 17:22:48.935353 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6992> 2014-08-13 17:22:48.935372 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(22) epoch 194470) v5 -- ?+0 0x4ac4b40 con 0x4832c00 -6991> 2014-08-13 17:22:48.935372 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194443/194389 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6990> 2014-08-13 17:22:48.935368 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 3 ==== pg_query(22.1a0 epoch 194470) v3 ==== 144+0+0 (1392334373 0 0) 0x29c4ec0 con 0x4758b00 -6989> 2014-08-13 17:22:48.935396 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(19) epoch 194470) v5 -- ?+0 0x4ac4280 con 0x48dfbc0 -6988> 2014-08-13 17:22:48.935400 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.811955, event: header_read, request: pg_query(22.1a0 epoch 194470) v3 -6987> 2014-08-13 17:22:48.935411 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.811956, event: throttled, request: pg_query(22.1a0 epoch 194470) v3 -6986> 2014-08-13 17:22:48.935421 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.812014, event: all_read, request: pg_query(22.1a0 epoch 194470) v3 -6985> 2014-08-13 17:22:48.935420 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6812/59915 -- pg_query(22.6bd epoch 194470) v3 -- ?+0 0x4eb2760 con 0x4832100 -6984> 2014-08-13 17:22:48.935427 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.935397, event: dispatched, request: pg_query(22.1a0 epoch 194470) v3 -6983> 2014-08-13 17:22:48.935433 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.935433, event: waiting_for_osdmap, request: pg_query(22.1a0 epoch 194470) v3 -6982> 2014-08-13 17:22:48.935441 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.935441, event: started, request: pg_query(22.1a0 epoch 194470) v3 -6981> 2014-08-13 17:22:48.935452 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6842/9607 -- pg_query(22.6bd epoch 194470) v3 -- ?+0 0x4eb0f00 con 0x47f70c0 -6980> 2014-08-13 17:22:48.935461 7f74d8735700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:22:48.935460, event: done, request: pg_query(22.1a0 epoch 194470) v3 -6979> 2014-08-13 17:22:48.935471 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_query(22.6bd epoch 194470) v3 -- ?+0 0x4eb1fe0 con 0x47a27e0 -6978> 2014-08-13 17:22:48.935477 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 3 ==== pg_notify(22.99(18) epoch 194470) v5 ==== 1720+0+0 (3779630166 0 0) 0x466fa80 con 0x4758840 -6977> 2014-08-13 17:22:48.935493 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.799510, event: header_read, request: pg_notify(22.99(18) epoch 194470) v5 -6976> 2014-08-13 17:22:48.935499 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.799511, event: throttled, request: pg_notify(22.99(18) epoch 194470) v5 -6975> 2014-08-13 17:22:48.935504 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.799597, event: all_read, request: pg_notify(22.99(18) epoch 194470) v5 -6974> 2014-08-13 17:22:48.935514 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.935491, event: dispatched, request: pg_notify(22.99(18) epoch 194470) v5 -6973> 2014-08-13 17:22:48.935524 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.935524, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194470) v5 -6972> 2014-08-13 17:22:48.935540 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.935539, event: started, request: pg_notify(22.99(18) epoch 194470) v5 -6971> 2014-08-13 17:22:48.935565 7f74d8735700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:22:48.935565, event: done, request: pg_notify(22.99(18) epoch 194470) v5 -6970> 2014-08-13 17:22:48.935563 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(19) epoch 194470) v5 -- ?+0 0x57d47c0 con 0x475d960 -6969> 2014-08-13 17:22:48.935548 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.219193 3 0.000263 -6968> 2014-08-13 17:22:48.935601 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(19) epoch 194470) v5 -- ?+0 0x4ac7540 con 0x486aec0 -6967> 2014-08-13 17:22:48.935628 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(25) epoch 194470) v5 -- ?+0 0x4ac0380 con 0x486b180 -6966> 2014-08-13 17:22:48.935650 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_query(22.3e4 epoch 194470) v3 -- ?+0 0x29c6ae0 con 0x47a32e0 -6965> 2014-08-13 17:22:48.935673 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_query(22.3e4 epoch 194470) v3 -- ?+0 0x29c2d00 con 0x475ee00 -6964> 2014-08-13 17:22:48.935705 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6820/35445 -- pg_query(22.3e4 epoch 194470) v3 -- ?+0 0x29c4740 con 0x47a1b80 -6963> 2014-08-13 17:22:48.935730 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6814/43857 -- pg_query(22.3e4 epoch 194470) v3 -- ?+0 0x29c6ea0 con 0x541c4c0 -6962> 2014-08-13 17:22:48.935742 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6802/41438 -- pg_query(22.3e4 epoch 194470) v3 -- ?+0 0x29c6900 con 0x4901600 -6961> 2014-08-13 17:22:48.935730 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started 0.219424 0 0.000000 -6960> 2014-08-13 17:22:48.935605 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 3 ==== pg_notify(24.9b(26) epoch 194470) v5 ==== 2214+0+0 (1309885511 0 0) 0x53b88c0 con 0x47591e0 -6959> 2014-08-13 17:22:48.935763 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.929428, event: header_read, request: pg_notify(24.9b(26) epoch 194470) v5 -6958> 2014-08-13 17:22:48.935761 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194447/194448/193264) [15,4,52]/[15,4,61] r=1 lpr=194449 pi=193264-194447/21 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset -6957> 2014-08-13 17:22:48.935774 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.929430, event: throttled, request: pg_notify(24.9b(26) epoch 194470) v5 -6956> 2014-08-13 17:22:48.935780 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.929540, event: all_read, request: pg_notify(24.9b(26) epoch 194470) v5 -6955> 2014-08-13 17:22:48.935785 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.935759, event: dispatched, request: pg_notify(24.9b(26) epoch 194470) v5 -6954> 2014-08-13 17:22:48.935792 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.935792, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194470) v5 -6953> 2014-08-13 17:22:48.935804 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.935804, event: started, request: pg_notify(24.9b(26) epoch 194470) v5 -6952> 2014-08-13 17:22:48.935827 7f74d8735700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:22:48.935826, event: done, request: pg_notify(24.9b(26) epoch 194470) v5 -6951> 2014-08-13 17:22:48.935860 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 3 ==== pg_notify(22.f3 epoch 194470) v5 ==== 694+0+0 (2189735153 0 0) 0x57d7000 con 0x475cd00 -6950> 2014-08-13 17:22:48.935888 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.929019, event: header_read, request: pg_notify(22.f3 epoch 194470) v5 -6949> 2014-08-13 17:22:48.935895 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.929021, event: throttled, request: pg_notify(22.f3 epoch 194470) v5 -6948> 2014-08-13 17:22:48.935900 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.929259, event: all_read, request: pg_notify(22.f3 epoch 194470) v5 -6947> 2014-08-13 17:22:48.935905 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.935886, event: dispatched, request: pg_notify(22.f3 epoch 194470) v5 -6946> 2014-08-13 17:22:48.935911 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.935910, event: waiting_for_osdmap, request: pg_notify(22.f3 epoch 194470) v5 -6945> 2014-08-13 17:22:48.935918 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.935917, event: started, request: pg_notify(22.f3 epoch 194470) v5 -6944> 2014-08-13 17:22:48.935923 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 0.228949 3 0.000157 -6943> 2014-08-13 17:22:48.935954 7f74d8735700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:22:48.935954, event: done, request: pg_notify(22.f3 epoch 194470) v5 -6942> 2014-08-13 17:22:48.935956 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started 0.229031 0 0.000000 -6941> 2014-08-13 17:22:48.935980 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194447/194447/193188) [17,4,395] r=1 lpr=194449 pi=193188-194446/25 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset -6940> 2014-08-13 17:22:48.935985 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.56 192.168.1.111:6826/11446 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4802880 con 0x475aec0 -6939> 2014-08-13 17:22:48.936004 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6938> 2014-08-13 17:22:48.936023 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 3 ==== pg_query(22.116 epoch 194470) v3 ==== 144+0+0 (1966617784 0 0) 0x29c1a40 con 0x475c4c0 -6937> 2014-08-13 17:22:48.936042 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.799849, event: header_read, request: pg_query(22.116 epoch 194470) v3 -6936> 2014-08-13 17:22:48.936053 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.799850, event: throttled, request: pg_query(22.116 epoch 194470) v3 -6935> 2014-08-13 17:22:48.936062 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.800077, event: all_read, request: pg_query(22.116 epoch 194470) v3 -6934> 2014-08-13 17:22:48.936070 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.936040, event: dispatched, request: pg_query(22.116 epoch 194470) v3 -6933> 2014-08-13 17:22:48.936080 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.936080, event: waiting_for_osdmap, request: pg_query(22.116 epoch 194470) v3 -6932> 2014-08-13 17:22:48.936090 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.936090, event: started, request: pg_query(22.116 epoch 194470) v3 -6931> 2014-08-13 17:22:48.936110 7f74d8735700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:22:48.936110, event: done, request: pg_query(22.116 epoch 194470) v3 -6930> 2014-08-13 17:22:48.936133 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 3 ==== pg_notify(22.3e4(1) epoch 194470) v5 ==== 777+0+0 (3137208198 0 0) 0x4ac40c0 con 0x475ee00 -6929> 2014-08-13 17:22:48.936152 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.808957, event: header_read, request: pg_notify(22.3e4(1) epoch 194470) v5 -6928> 2014-08-13 17:22:48.936185 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.808959, event: throttled, request: pg_notify(22.3e4(1) epoch 194470) v5 -6927> 2014-08-13 17:22:48.936197 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.809098, event: all_read, request: pg_notify(22.3e4(1) epoch 194470) v5 -6926> 2014-08-13 17:22:48.936206 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.936150, event: dispatched, request: pg_notify(22.3e4(1) epoch 194470) v5 -6925> 2014-08-13 17:22:48.936216 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.936216, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194470) v5 -6924> 2014-08-13 17:22:48.936227 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.936227, event: started, request: pg_notify(22.3e4(1) epoch 194470) v5 -6923> 2014-08-13 17:22:48.936263 7f74d8735700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:22:48.936263, event: done, request: pg_notify(22.3e4(1) epoch 194470) v5 -6922> 2014-08-13 17:22:48.936289 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 3 ==== pg_notify(22.6bd(20) epoch 194470) v5 ==== 1838+0+0 (3391304317 0 0) 0x4ac2bc0 con 0x47a27e0 -6921> 2014-08-13 17:22:48.936310 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.806704, event: header_read, request: pg_notify(22.6bd(20) epoch 194470) v5 -6920> 2014-08-13 17:22:48.936321 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.806706, event: throttled, request: pg_notify(22.6bd(20) epoch 194470) v5 -6919> 2014-08-13 17:22:48.936330 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.806892, event: all_read, request: pg_notify(22.6bd(20) epoch 194470) v5 -6918> 2014-08-13 17:22:48.936339 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.936308, event: dispatched, request: pg_notify(22.6bd(20) epoch 194470) v5 -6917> 2014-08-13 17:22:48.936349 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.936348, event: waiting_for_osdmap, request: pg_notify(22.6bd(20) epoch 194470) v5 -6916> 2014-08-13 17:22:48.936338 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.000577 15 0.000367 -6915> 2014-08-13 17:22:48.936360 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.936360, event: started, request: pg_notify(22.6bd(20) epoch 194470) v5 -6914> 2014-08-13 17:22:48.936375 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started -6913> 2014-08-13 17:22:48.936398 7f74d8735700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:22:48.936398, event: done, request: pg_notify(22.6bd(20) epoch 194470) v5 -6912> 2014-08-13 17:22:48.936395 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Start -6911> 2014-08-13 17:22:48.936411 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -6910> 2014-08-13 17:22:48.936419 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 3 ==== pg_notify(22.362(1) epoch 194470) v5 ==== 777+0+0 (1550353488 0 0) 0x4ac4980 con 0x47a0b00 -6909> 2014-08-13 17:22:48.936440 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.809641, event: header_read, request: pg_notify(22.362(1) epoch 194470) v5 -6908> 2014-08-13 17:22:48.936432 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000037 0 0.000000 -6907> 2014-08-13 17:22:48.936454 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.809642, event: throttled, request: pg_notify(22.362(1) epoch 194470) v5 -6906> 2014-08-13 17:22:48.936473 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.809691, event: all_read, request: pg_notify(22.362(1) epoch 194470) v5 -6905> 2014-08-13 17:22:48.936468 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194449/194388 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -6904> 2014-08-13 17:22:48.936484 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.936438, event: dispatched, request: pg_notify(22.362(1) epoch 194470) v5 -6903> 2014-08-13 17:22:48.936494 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.936494, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194470) v5 -6902> 2014-08-13 17:22:48.936506 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.936505, event: started, request: pg_notify(22.362(1) epoch 194470) v5 -6901> 2014-08-13 17:22:48.936529 7f74d8735700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:22:48.936529, event: done, request: pg_notify(22.362(1) epoch 194470) v5 -6900> 2014-08-13 17:22:48.936557 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 3 ==== pg_notify(24.b6(27) epoch 194470) v5 ==== 2273+0+0 (585923396 0 0) 0x4ac6c80 con 0x47a53e0 -6899> 2014-08-13 17:22:48.936559 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Reset 0.000578 15 0.000213 -6898> 2014-08-13 17:22:48.936595 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started -6897> 2014-08-13 17:22:48.936617 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Start -6896> 2014-08-13 17:22:48.936635 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: transitioning to Stray -6895> 2014-08-13 17:22:48.936626 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started/Stray 0.219778 3 0.000189 -6894> 2014-08-13 17:22:48.936659 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Start 0.000041 0 0.000000 -6893> 2014-08-13 17:22:48.936661 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started 0.219867 0 0.000000 -6892> 2014-08-13 17:22:48.936684 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194447/194448/192978) [31,338,4] r=2 lpr=194449 pi=193125-194447/29 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset -6891> 2014-08-13 17:22:48.936683 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194448/194442 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started/Stray -6890> 2014-08-13 17:22:48.936577 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.811564, event: header_read, request: pg_notify(24.b6(27) epoch 194470) v5 -6889> 2014-08-13 17:22:48.936713 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.811565, event: throttled, request: pg_notify(24.b6(27) epoch 194470) v5 -6888> 2014-08-13 17:22:48.936727 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.811630, event: all_read, request: pg_notify(24.b6(27) epoch 194470) v5 -6887> 2014-08-13 17:22:48.936733 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.936575, event: dispatched, request: pg_notify(24.b6(27) epoch 194470) v5 -6886> 2014-08-13 17:22:48.936742 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.936742, event: waiting_for_osdmap, request: pg_notify(24.b6(27) epoch 194470) v5 -6885> 2014-08-13 17:22:48.936751 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.936750, event: started, request: pg_notify(24.b6(27) epoch 194470) v5 -6884> 2014-08-13 17:22:48.936788 7f74d8735700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:22:48.936788, event: done, request: pg_notify(24.b6(27) epoch 194470) v5 -6883> 2014-08-13 17:22:48.936823 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 3 ==== pg_notify(1.2b,2.2a epoch 194470) v5 ==== 1376+0+0 (546689739 0 0) 0x57d2840 con 0x47a3f40 -6882> 2014-08-13 17:22:48.936842 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.919125, event: header_read, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6881> 2014-08-13 17:22:48.936848 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.919127, event: throttled, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6880> 2014-08-13 17:22:48.936859 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.919357, event: all_read, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6879> 2014-08-13 17:22:48.936868 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.936839, event: dispatched, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6878> 2014-08-13 17:22:48.936879 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.936878, event: waiting_for_osdmap, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6877> 2014-08-13 17:22:48.936895 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.936895, event: started, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6876> 2014-08-13 17:22:48.936886 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.229677 3 0.000108 -6875> 2014-08-13 17:22:48.936912 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.229757 0 0.000000 -6874> 2014-08-13 17:22:48.936933 7f74d8735700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:22:48.936932, event: done, request: pg_notify(1.2b,2.2a epoch 194470) v5 -6873> 2014-08-13 17:22:48.936932 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194447/194448/192976) [367,44,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -6872> 2014-08-13 17:22:48.936950 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 3 ==== pg_notify(22.619(1) epoch 194470) v5 ==== 777+0+0 (37238582 0 0) 0x57d5cc0 con 0x47a4ba0 -6871> 2014-08-13 17:22:48.936965 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.928574, event: header_read, request: pg_notify(22.619(1) epoch 194470) v5 -6870> 2014-08-13 17:22:48.936972 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.928576, event: throttled, request: pg_notify(22.619(1) epoch 194470) v5 -6869> 2014-08-13 17:22:48.936980 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.928648, event: all_read, request: pg_notify(22.619(1) epoch 194470) v5 -6868> 2014-08-13 17:22:48.936985 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.936963, event: dispatched, request: pg_notify(22.619(1) epoch 194470) v5 -6867> 2014-08-13 17:22:48.936991 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.936991, event: waiting_for_osdmap, request: pg_notify(22.619(1) epoch 194470) v5 -6866> 2014-08-13 17:22:48.936998 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.936998, event: started, request: pg_notify(22.619(1) epoch 194470) v5 -6865> 2014-08-13 17:22:48.937016 7f74d8735700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:22:48.937016, event: done, request: pg_notify(22.619(1) epoch 194470) v5 -6864> 2014-08-13 17:22:48.937029 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 3 ==== pg_query(24.369 epoch 194470) v3 ==== 144+0+0 (1953033668 0 0) 0x29c4560 con 0x47a32e0 -6863> 2014-08-13 17:22:48.937043 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.810868, event: header_read, request: pg_query(24.369 epoch 194470) v3 -6862> 2014-08-13 17:22:48.937049 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.810870, event: throttled, request: pg_query(24.369 epoch 194470) v3 -6861> 2014-08-13 17:22:48.937054 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.810924, event: all_read, request: pg_query(24.369 epoch 194470) v3 -6860> 2014-08-13 17:22:48.937058 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.937041, event: dispatched, request: pg_query(24.369 epoch 194470) v3 -6859> 2014-08-13 17:22:48.937064 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.937064, event: waiting_for_osdmap, request: pg_query(24.369 epoch 194470) v3 -6858> 2014-08-13 17:22:48.937071 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.937070, event: started, request: pg_query(24.369 epoch 194470) v3 -6857> 2014-08-13 17:22:48.937092 7f74d8735700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:22:48.937091, event: done, request: pg_query(24.369 epoch 194470) v3 -6856> 2014-08-13 17:22:48.937122 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 3 ==== pg_notify(22.84(20) epoch 194470) v5 ==== 1838+0+0 (2113891409 0 0) 0x4ac1dc0 con 0x47f0c60 -6855> 2014-08-13 17:22:48.937142 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.805646, event: header_read, request: pg_notify(22.84(20) epoch 194470) v5 -6854> 2014-08-13 17:22:48.937152 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.805647, event: throttled, request: pg_notify(22.84(20) epoch 194470) v5 -6853> 2014-08-13 17:22:48.937174 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.805708, event: all_read, request: pg_notify(22.84(20) epoch 194470) v5 -6852> 2014-08-13 17:22:48.937184 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.937140, event: dispatched, request: pg_notify(22.84(20) epoch 194470) v5 -6851> 2014-08-13 17:22:48.937197 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.937197, event: waiting_for_osdmap, request: pg_notify(22.84(20) epoch 194470) v5 -6850> 2014-08-13 17:22:48.937223 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.937223, event: started, request: pg_notify(22.84(20) epoch 194470) v5 -6849> 2014-08-13 17:22:48.937216 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Reset 0.000531 15 0.000212 -6848> 2014-08-13 17:22:48.937250 7f74d8735700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:22:48.937250, event: done, request: pg_notify(22.84(20) epoch 194470) v5 -6847> 2014-08-13 17:22:48.937249 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started -6846> 2014-08-13 17:22:48.937271 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Start -6845> 2014-08-13 17:22:48.937273 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 3 ==== pg_notify(22.99 epoch 194470) v5 ==== 694+0+0 (1328439166 0 0) 0x5263640 con 0x47f4620 -6844> 2014-08-13 17:22:48.937294 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: transitioning to Stray -6843> 2014-08-13 17:22:48.937304 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.923729, event: header_read, request: pg_notify(22.99 epoch 194470) v5 -6842> 2014-08-13 17:22:48.937325 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.923731, event: throttled, request: pg_notify(22.99 epoch 194470) v5 -6841> 2014-08-13 17:22:48.937319 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Start 0.000048 0 0.000000 -6840> 2014-08-13 17:22:48.937336 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.923813, event: all_read, request: pg_notify(22.99 epoch 194470) v5 -6839> 2014-08-13 17:22:48.937345 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.937301, event: dispatched, request: pg_notify(22.99 epoch 194470) v5 -6838> 2014-08-13 17:22:48.937340 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194449/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started/Stray -6837> 2014-08-13 17:22:48.937355 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.937355, event: waiting_for_osdmap, request: pg_notify(22.99 epoch 194470) v5 -6836> 2014-08-13 17:22:48.937366 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.937366, event: started, request: pg_notify(22.99 epoch 194470) v5 -6835> 2014-08-13 17:22:48.937388 7f74d8735700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:22:48.937388, event: done, request: pg_notify(22.99 epoch 194470) v5 -6834> 2014-08-13 17:22:48.937406 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 3 ==== pg_notify(24.1f7(1) epoch 194470) v5 ==== 811+0+0 (763539561 0 0) 0x4ac7a80 con 0x47f7e80 -6833> 2014-08-13 17:22:48.937425 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.819084, event: header_read, request: pg_notify(24.1f7(1) epoch 194470) v5 -6832> 2014-08-13 17:22:48.937438 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.819088, event: throttled, request: pg_notify(24.1f7(1) epoch 194470) v5 -6831> 2014-08-13 17:22:48.937447 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.819186, event: all_read, request: pg_notify(24.1f7(1) epoch 194470) v5 -6830> 2014-08-13 17:22:48.937456 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.937423, event: dispatched, request: pg_notify(24.1f7(1) epoch 194470) v5 -6829> 2014-08-13 17:22:48.937466 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.937465, event: waiting_for_osdmap, request: pg_notify(24.1f7(1) epoch 194470) v5 -6828> 2014-08-13 17:22:48.937481 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.937480, event: started, request: pg_notify(24.1f7(1) epoch 194470) v5 -6827> 2014-08-13 17:22:48.937507 7f74d8735700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:22:48.937507, event: done, request: pg_notify(24.1f7(1) epoch 194470) v5 -6826> 2014-08-13 17:22:48.937506 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(25) epoch 194470) v5 -- ?+0 0x4ac3d40 con 0x48d9ce0 -6825> 2014-08-13 17:22:48.937528 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 3 ==== pg_notify(22.6bd(2) epoch 194470) v5 ==== 828+0+0 (1907705783 0 0) 0x4ac08c0 con 0x47f70c0 -6824> 2014-08-13 17:22:48.937523 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.000590 15 0.000173 -6823> 2014-08-13 17:22:48.937540 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(33) epoch 194470) v5 -- ?+0 0x4ac4600 con 0x47f4620 -6822> 2014-08-13 17:22:48.937547 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.802481, event: header_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6821> 2014-08-13 17:22:48.937555 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.802485, event: throttled, request: pg_notify(22.6bd(2) epoch 194470) v5 -6820> 2014-08-13 17:22:48.937551 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -6819> 2014-08-13 17:22:48.937564 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.805105, event: all_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6818> 2014-08-13 17:22:48.937574 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.937545, event: dispatched, request: pg_notify(22.6bd(2) epoch 194470) v5 -6817> 2014-08-13 17:22:48.937566 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -6816> 2014-08-13 17:22:48.937588 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.937588, event: waiting_for_osdmap, request: pg_notify(22.6bd(2) epoch 194470) v5 -6815> 2014-08-13 17:22:48.937596 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.937596, event: started, request: pg_notify(22.6bd(2) epoch 194470) v5 -6814> 2014-08-13 17:22:48.937592 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6813> 2014-08-13 17:22:48.937615 7f74d8735700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:22:48.937615, event: done, request: pg_notify(22.6bd(2) epoch 194470) v5 -6812> 2014-08-13 17:22:48.937613 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000046 0 0.000000 -6811> 2014-08-13 17:22:48.937632 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194449/194449 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -6810> 2014-08-13 17:22:48.937635 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 3 ==== pg_notify(24.1bb(25) epoch 194470) v5 ==== 2155+0+0 (2005506023 0 0) 0x4ac3640 con 0x4833440 -6809> 2014-08-13 17:22:48.937620 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.220266 3 0.000183 -6808> 2014-08-13 17:22:48.937651 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.806956, event: header_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -6807> 2014-08-13 17:22:48.937657 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.806958, event: throttled, request: pg_notify(24.1bb(25) epoch 194470) v5 -6806> 2014-08-13 17:22:48.937663 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.807177, event: all_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -6805> 2014-08-13 17:22:48.937668 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.937649, event: dispatched, request: pg_notify(24.1bb(25) epoch 194470) v5 -6804> 2014-08-13 17:22:48.937656 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started 0.220369 0 0.000000 -6803> 2014-08-13 17:22:48.937675 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.937675, event: waiting_for_osdmap, request: pg_notify(24.1bb(25) epoch 194470) v5 -6802> 2014-08-13 17:22:48.937678 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194447/194447/192975) [285,336,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset -6801> 2014-08-13 17:22:48.937689 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.937689, event: started, request: pg_notify(24.1bb(25) epoch 194470) v5 -6800> 2014-08-13 17:22:48.938011 7f74d8735700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:22:48.938011, event: done, request: pg_notify(24.1bb(25) epoch 194470) v5 -6799> 2014-08-13 17:22:48.938037 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 3 ==== pg_query(22.653 epoch 194470) v3 ==== 144+0+0 (3404995219 0 0) 0x29c3de0 con 0x4832c00 -6798> 2014-08-13 17:22:48.938058 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.809455, event: header_read, request: pg_query(22.653 epoch 194470) v3 -6797> 2014-08-13 17:22:48.938050 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6796> 2014-08-13 17:22:48.938070 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.809456, event: throttled, request: pg_query(22.653 epoch 194470) v3 -6795> 2014-08-13 17:22:48.938078 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.809494, event: all_read, request: pg_query(22.653 epoch 194470) v3 -6794> 2014-08-13 17:22:48.938087 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.938056, event: dispatched, request: pg_query(22.653 epoch 194470) v3 -6793> 2014-08-13 17:22:48.938079 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.230594 2 0.000140 -6792> 2014-08-13 17:22:48.938096 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.938096, event: waiting_for_osdmap, request: pg_query(22.653 epoch 194470) v3 -6791> 2014-08-13 17:22:48.938109 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.938109, event: started, request: pg_query(22.653 epoch 194470) v3 -6790> 2014-08-13 17:22:48.938100 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.230626 0 0.000000 -6789> 2014-08-13 17:22:48.938133 7f74d8735700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:22:48.938133, event: done, request: pg_query(22.653 epoch 194470) v3 -6788> 2014-08-13 17:22:48.938134 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.230678 0 0.000000 -6787> 2014-08-13 17:22:48.938180 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 3 ==== pg_notify(22.113(22) epoch 194470) v5 ==== 1952+0+0 (2442813993 0 0) 0x5264280 con 0x4830dc0 -6786> 2014-08-13 17:22:48.938163 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started 0.230756 0 0.000000 -6785> 2014-08-13 17:22:48.938207 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.919016, event: header_read, request: pg_notify(22.113(22) epoch 194470) v5 -6784> 2014-08-13 17:22:48.938216 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.919019, event: throttled, request: pg_notify(22.113(22) epoch 194470) v5 -6783> 2014-08-13 17:22:48.938225 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.919126, event: all_read, request: pg_notify(22.113(22) epoch 194470) v5 -6782> 2014-08-13 17:22:48.938217 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194447/194448/194448) [4,310,21] r=0 lpr=194449 pi=193306-194447/18 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset -6781> 2014-08-13 17:22:48.938238 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.938205, event: dispatched, request: pg_notify(22.113(22) epoch 194470) v5 -6780> 2014-08-13 17:22:48.938247 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.938247, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194470) v5 -6779> 2014-08-13 17:22:48.938259 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.938259, event: started, request: pg_notify(22.113(22) epoch 194470) v5 -6778> 2014-08-13 17:22:48.938283 7f74d8735700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:22:48.938283, event: done, request: pg_notify(22.113(22) epoch 194470) v5 -6777> 2014-08-13 17:22:48.938307 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.32 192.168.1.115:6812/59915 3 ==== pg_notify(22.6bd(2) epoch 194470) v5 ==== 828+0+0 (8849289 0 0) 0x57d7a80 con 0x4832100 -6776> 2014-08-13 17:22:48.938326 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.937077, event: header_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6775> 2014-08-13 17:22:48.938337 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.937079, event: throttled, request: pg_notify(22.6bd(2) epoch 194470) v5 -6774> 2014-08-13 17:22:48.938346 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.937166, event: all_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6773> 2014-08-13 17:22:48.938356 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.938324, event: dispatched, request: pg_notify(22.6bd(2) epoch 194470) v5 -6772> 2014-08-13 17:22:48.938365 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.938365, event: waiting_for_osdmap, request: pg_notify(22.6bd(2) epoch 194470) v5 -6771> 2014-08-13 17:22:48.938376 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.938376, event: started, request: pg_notify(22.6bd(2) epoch 194470) v5 -6770> 2014-08-13 17:22:48.938399 7f74d8735700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:22:48.938399, event: done, request: pg_notify(22.6bd(2) epoch 194470) v5 -6769> 2014-08-13 17:22:48.938417 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 3 ==== pg_notify(22.48e(20) epoch 194470) v5 ==== 1834+0+0 (1696045208 0 0) 0x4ac2f40 con 0x4835280 -6768> 2014-08-13 17:22:48.938435 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.806200, event: header_read, request: pg_notify(22.48e(20) epoch 194470) v5 -6767> 2014-08-13 17:22:48.938445 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.806201, event: throttled, request: pg_notify(22.48e(20) epoch 194470) v5 -6766> 2014-08-13 17:22:48.938456 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.807023, event: all_read, request: pg_notify(22.48e(20) epoch 194470) v5 -6765> 2014-08-13 17:22:48.938464 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.938433, event: dispatched, request: pg_notify(22.48e(20) epoch 194470) v5 -6764> 2014-08-13 17:22:48.938474 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.938474, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194470) v5 -6763> 2014-08-13 17:22:48.938485 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.938485, event: started, request: pg_notify(22.48e(20) epoch 194470) v5 -6762> 2014-08-13 17:22:48.938481 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.000802 15 0.000486 -6761> 2014-08-13 17:22:48.938509 7f74d8735700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:22:48.938509, event: done, request: pg_notify(22.48e(20) epoch 194470) v5 -6760> 2014-08-13 17:22:48.938515 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started -6759> 2014-08-13 17:22:48.938526 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 3 ==== pg_query(24.206 epoch 194470) v3 ==== 144+0+0 (3754893679 0 0) 0x29c23a0 con 0x48377a0 -6758> 2014-08-13 17:22:48.938541 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.803280, event: header_read, request: pg_query(24.206 epoch 194470) v3 -6757> 2014-08-13 17:22:48.938537 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Start -6756> 2014-08-13 17:22:48.938552 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.803281, event: throttled, request: pg_query(24.206 epoch 194470) v3 -6755> 2014-08-13 17:22:48.938561 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.803317, event: all_read, request: pg_query(24.206 epoch 194470) v3 -6754> 2014-08-13 17:22:48.938554 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray -6753> 2014-08-13 17:22:48.938570 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.938539, event: dispatched, request: pg_query(24.206 epoch 194470) v3 -6752> 2014-08-13 17:22:48.938580 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.938579, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194470) v3 -6751> 2014-08-13 17:22:48.938573 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000036 0 0.000000 -6750> 2014-08-13 17:22:48.938590 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.938590, event: started, request: pg_query(24.206 epoch 194470) v3 -6749> 2014-08-13 17:22:48.938607 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194448/194442 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray -6748> 2014-08-13 17:22:48.938625 7f74d8735700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:22:48.938625, event: done, request: pg_query(24.206 epoch 194470) v3 -6747> 2014-08-13 17:22:48.938653 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 3 ==== pg_notify(24.1f7(27) epoch 194470) v5 ==== 2273+0+0 (2131688865 0 0) 0x466f000 con 0x4868420 -6746> 2014-08-13 17:22:48.938674 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.797802, event: header_read, request: pg_notify(24.1f7(27) epoch 194470) v5 -6745> 2014-08-13 17:22:48.938684 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.797804, event: throttled, request: pg_notify(24.1f7(27) epoch 194470) v5 -6744> 2014-08-13 17:22:48.938693 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.798360, event: all_read, request: pg_notify(24.1f7(27) epoch 194470) v5 -6743> 2014-08-13 17:22:48.938702 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.938672, event: dispatched, request: pg_notify(24.1f7(27) epoch 194470) v5 -6742> 2014-08-13 17:22:48.938712 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.938712, event: waiting_for_osdmap, request: pg_notify(24.1f7(27) epoch 194470) v5 -6741> 2014-08-13 17:22:48.938725 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.938725, event: started, request: pg_notify(24.1f7(27) epoch 194470) v5 -6740> 2014-08-13 17:22:48.938720 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000503 15 0.000358 -6739> 2014-08-13 17:22:48.938747 7f74d8735700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:22:48.938746, event: done, request: pg_notify(24.1f7(27) epoch 194470) v5 -6738> 2014-08-13 17:22:48.938745 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started -6737> 2014-08-13 17:22:48.938751 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.220886 3 0.000239 -6736> 2014-08-13 17:22:48.938763 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start -6735> 2014-08-13 17:22:48.938768 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 3 ==== pg_notify(22.1d2(1) epoch 194470) v5 ==== 777+0+0 (167637636 0 0) 0x5263f00 con 0x486bf40 -6734> 2014-08-13 17:22:48.938784 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.918969, event: header_read, request: pg_notify(22.1d2(1) epoch 194470) v5 -6733> 2014-08-13 17:22:48.938781 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6732> 2014-08-13 17:22:48.938793 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.918971, event: throttled, request: pg_notify(22.1d2(1) epoch 194470) v5 -6731> 2014-08-13 17:22:48.938780 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.221026 0 0.000000 -6730> 2014-08-13 17:22:48.938813 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.919039, event: all_read, request: pg_notify(22.1d2(1) epoch 194470) v5 -6729> 2014-08-13 17:22:48.938821 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.938782, event: dispatched, request: pg_notify(22.1d2(1) epoch 194470) v5 -6728> 2014-08-13 17:22:48.938811 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000048 0 0.000000 -6727> 2014-08-13 17:22:48.938824 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194447/194448/192978) [323,359,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -6726> 2014-08-13 17:22:48.938835 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.938835, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194470) v5 -6725> 2014-08-13 17:22:48.938853 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.938853, event: started, request: pg_notify(22.1d2(1) epoch 194470) v5 -6724> 2014-08-13 17:22:48.938849 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6723> 2014-08-13 17:22:48.938866 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6722> 2014-08-13 17:22:48.938879 7f74d8735700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:22:48.938879, event: done, request: pg_notify(22.1d2(1) epoch 194470) v5 -6721> 2014-08-13 17:22:48.938886 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194443/194389 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6720> 2014-08-13 17:22:48.938894 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 3 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c71440 con 0x486bc80 -6719> 2014-08-13 17:22:48.938914 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6718> 2014-08-13 17:22:48.938933 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 3 ==== pg_notify(22.99 epoch 194470) v5 ==== 694+0+0 (1328439166 0 0) 0x57d39c0 con 0x486b9c0 -6717> 2014-08-13 17:22:48.938951 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.923945, event: header_read, request: pg_notify(22.99 epoch 194470) v5 -6716> 2014-08-13 17:22:48.938961 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.923947, event: throttled, request: pg_notify(22.99 epoch 194470) v5 -6715> 2014-08-13 17:22:48.938970 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.924012, event: all_read, request: pg_notify(22.99 epoch 194470) v5 -6714> 2014-08-13 17:22:48.938979 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.938948, event: dispatched, request: pg_notify(22.99 epoch 194470) v5 -6713> 2014-08-13 17:22:48.938988 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.938988, event: waiting_for_osdmap, request: pg_notify(22.99 epoch 194470) v5 -6712> 2014-08-13 17:22:48.938998 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.938998, event: started, request: pg_notify(22.99 epoch 194470) v5 -6711> 2014-08-13 17:22:48.939019 7f74d8735700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:22:48.939019, event: done, request: pg_notify(22.99 epoch 194470) v5 -6710> 2014-08-13 17:22:48.939037 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 3 ==== pg_notify(22.48e(20) epoch 194470) v5 ==== 1834+0+0 (1696045208 0 0) 0x4ac16c0 con 0x486b700 -6709> 2014-08-13 17:22:48.939056 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.805584, event: header_read, request: pg_notify(22.48e(20) epoch 194470) v5 -6708> 2014-08-13 17:22:48.939066 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.805585, event: throttled, request: pg_notify(22.48e(20) epoch 194470) v5 -6707> 2014-08-13 17:22:48.939074 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.805646, event: all_read, request: pg_notify(22.48e(20) epoch 194470) v5 -6706> 2014-08-13 17:22:48.939059 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 0.231226 3 0.000179 -6705> 2014-08-13 17:22:48.939085 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.939053, event: dispatched, request: pg_notify(22.48e(20) epoch 194470) v5 -6704> 2014-08-13 17:22:48.939089 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started 0.231356 0 0.000000 -6703> 2014-08-13 17:22:48.939114 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.939114, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194470) v5 -6702> 2014-08-13 17:22:48.939124 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.939124, event: started, request: pg_notify(22.48e(20) epoch 194470) v5 -6701> 2014-08-13 17:22:48.939124 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194447/194447/192968) [289,4,285] r=1 lpr=194449 pi=193125-194446/32 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset -6700> 2014-08-13 17:22:48.939150 7f74d8735700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:22:48.939149, event: done, request: pg_notify(22.48e(20) epoch 194470) v5 -6699> 2014-08-13 17:22:48.939182 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 3 ==== pg_notify(24.9b epoch 194470) v5 ==== 694+0+0 (4133067690 0 0) 0x53b8a80 con 0x486a940 -6698> 2014-08-13 17:22:48.939204 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.929517, event: header_read, request: pg_notify(24.9b epoch 194470) v5 -6697> 2014-08-13 17:22:48.939215 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.929519, event: throttled, request: pg_notify(24.9b epoch 194470) v5 -6696> 2014-08-13 17:22:48.939224 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.929585, event: all_read, request: pg_notify(24.9b epoch 194470) v5 -6695> 2014-08-13 17:22:48.939232 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.939202, event: dispatched, request: pg_notify(24.9b epoch 194470) v5 -6694> 2014-08-13 17:22:48.939239 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.939238, event: waiting_for_osdmap, request: pg_notify(24.9b epoch 194470) v5 -6693> 2014-08-13 17:22:48.939246 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.939246, event: started, request: pg_notify(24.9b epoch 194470) v5 -6692> 2014-08-13 17:22:48.939234 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.000410 15 0.000202 -6691> 2014-08-13 17:22:48.939267 7f74d8735700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:22:48.939267, event: done, request: pg_notify(24.9b epoch 194470) v5 -6690> 2014-08-13 17:22:48.939262 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -6689> 2014-08-13 17:22:48.939278 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -6688> 2014-08-13 17:22:48.939296 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6687> 2014-08-13 17:22:48.939301 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 3 ==== pg_notify(22.3d9(35) epoch 194470) v5 ==== 2627+0+0 (2695345450 0 0) 0x4ac2680 con 0x486a3c0 -6686> 2014-08-13 17:22:48.939323 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.805787, event: header_read, request: pg_notify(22.3d9(35) epoch 194470) v5 -6685> 2014-08-13 17:22:48.939317 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000040 0 0.000000 -6684> 2014-08-13 17:22:48.939334 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.805788, event: throttled, request: pg_notify(22.3d9(35) epoch 194470) v5 -6683> 2014-08-13 17:22:48.939343 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.805880, event: all_read, request: pg_notify(22.3d9(35) epoch 194470) v5 -6682> 2014-08-13 17:22:48.939337 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194449/194449 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -6681> 2014-08-13 17:22:48.939352 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.939321, event: dispatched, request: pg_notify(22.3d9(35) epoch 194470) v5 -6680> 2014-08-13 17:22:48.939362 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.939362, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194470) v5 -6679> 2014-08-13 17:22:48.939373 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.939373, event: started, request: pg_notify(22.3d9(35) epoch 194470) v5 -6678> 2014-08-13 17:22:48.939447 7f74d8735700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:22:48.939447, event: done, request: pg_notify(22.3d9(35) epoch 194470) v5 -6677> 2014-08-13 17:22:48.939476 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 3 ==== pg_notify(22.707(24) epoch 194470) v5 ==== 2066+0+0 (3384600017 0 0) 0x4ac0fc0 con 0x48d9ce0 -6676> 2014-08-13 17:22:48.939497 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.804461, event: header_read, request: pg_notify(22.707(24) epoch 194470) v5 -6675> 2014-08-13 17:22:48.939507 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.804462, event: throttled, request: pg_notify(22.707(24) epoch 194470) v5 -6674> 2014-08-13 17:22:48.939496 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6673> 2014-08-13 17:22:48.939538 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.805644, event: all_read, request: pg_notify(22.707(24) epoch 194470) v5 -6672> 2014-08-13 17:22:48.939549 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.939494, event: dispatched, request: pg_notify(22.707(24) epoch 194470) v5 -6671> 2014-08-13 17:22:48.939560 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.939560, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194470) v5 -6670> 2014-08-13 17:22:48.939547 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.221127 2 0.000222 -6669> 2014-08-13 17:22:48.939575 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.939575, event: started, request: pg_notify(22.707(24) epoch 194470) v5 -6668> 2014-08-13 17:22:48.939596 7f74d8735700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:22:48.939596, event: done, request: pg_notify(22.707(24) epoch 194470) v5 -6667> 2014-08-13 17:22:48.939584 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.221184 0 0.000000 -6666> 2014-08-13 17:22:48.939619 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 3 ==== pg_query(24.3f7 epoch 194470) v3 ==== 144+0+0 (1919007377 0 0) 0x29c2ee0 con 0x486e040 -6665> 2014-08-13 17:22:48.939621 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.221250 0 0.000000 -6664> 2014-08-13 17:22:48.939640 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.805632, event: header_read, request: pg_query(24.3f7 epoch 194470) v3 -6663> 2014-08-13 17:22:48.939647 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.805632, event: throttled, request: pg_query(24.3f7 epoch 194470) v3 -6662> 2014-08-13 17:22:48.939655 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.805669, event: all_read, request: pg_query(24.3f7 epoch 194470) v3 -6661> 2014-08-13 17:22:48.939649 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started 0.221370 0 0.000000 -6660> 2014-08-13 17:22:48.939664 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.939638, event: dispatched, request: pg_query(24.3f7 epoch 194470) v3 -6659> 2014-08-13 17:22:48.939674 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.939674, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194470) v3 -6658> 2014-08-13 17:22:48.939668 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194447/194448/194448) [4,44,362] r=0 lpr=194449 pi=193306-194447/18 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset -6657> 2014-08-13 17:22:48.939684 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.939684, event: started, request: pg_query(24.3f7 epoch 194470) v3 -6656> 2014-08-13 17:22:48.939705 7f74d8735700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:22:48.939704, event: done, request: pg_query(24.3f7 epoch 194470) v3 -6655> 2014-08-13 17:22:48.939694 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Reset 0.000570 15 0.000218 -6654> 2014-08-13 17:22:48.939723 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 3 ==== pg_notify(24.3d epoch 194470) v5 ==== 694+0+0 (3045703392 0 0) 0x53b8540 con 0x48db9c0 -6653> 2014-08-13 17:22:48.939729 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started -6652> 2014-08-13 17:22:48.939740 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.929380, event: header_read, request: pg_notify(24.3d epoch 194470) v5 -6651> 2014-08-13 17:22:48.939761 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.929381, event: throttled, request: pg_notify(24.3d epoch 194470) v5 -6650> 2014-08-13 17:22:48.939753 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Start -6649> 2014-08-13 17:22:48.939772 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.929479, event: all_read, request: pg_notify(24.3d epoch 194470) v5 -6648> 2014-08-13 17:22:48.939781 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.939738, event: dispatched, request: pg_notify(24.3d epoch 194470) v5 -6647> 2014-08-13 17:22:48.939772 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: transitioning to Stray -6646> 2014-08-13 17:22:48.939792 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.939791, event: waiting_for_osdmap, request: pg_notify(24.3d epoch 194470) v5 -6645> 2014-08-13 17:22:48.939802 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.939802, event: started, request: pg_notify(24.3d epoch 194470) v5 -6644> 2014-08-13 17:22:48.939794 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Start 0.000041 0 0.000000 -6643> 2014-08-13 17:22:48.939814 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194448/194442 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started/Stray -6642> 2014-08-13 17:22:48.939825 7f74d8735700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:22:48.939825, event: done, request: pg_notify(24.3d epoch 194470) v5 -6641> 2014-08-13 17:22:48.939853 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 3 ==== pg_notify(24.30b epoch 194470) v5 ==== 694+0+0 (24938660 0 0) 0x53b96c0 con 0x48dfe80 -6640> 2014-08-13 17:22:48.939872 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.929700, event: header_read, request: pg_notify(24.30b epoch 194470) v5 -6639> 2014-08-13 17:22:48.939882 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.929701, event: throttled, request: pg_notify(24.30b epoch 194470) v5 -6638> 2014-08-13 17:22:48.939891 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.929852, event: all_read, request: pg_notify(24.30b epoch 194470) v5 -6637> 2014-08-13 17:22:48.939900 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.939870, event: dispatched, request: pg_notify(24.30b epoch 194470) v5 -6636> 2014-08-13 17:22:48.939909 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.939909, event: waiting_for_osdmap, request: pg_notify(24.30b epoch 194470) v5 -6635> 2014-08-13 17:22:48.939920 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.939919, event: started, request: pg_notify(24.30b epoch 194470) v5 -6634> 2014-08-13 17:22:48.939942 7f74d8735700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:22:48.939941, event: done, request: pg_notify(24.30b epoch 194470) v5 -6633> 2014-08-13 17:22:48.939963 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 3 ==== pg_query(22.3b9 epoch 194470) v3 ==== 144+0+0 (1951113560 0 0) 0x29c2940 con 0x48dfbc0 -6632> 2014-08-13 17:22:48.939982 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.805131, event: header_read, request: pg_query(22.3b9 epoch 194470) v3 -6631> 2014-08-13 17:22:48.939973 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6630> 2014-08-13 17:22:48.939992 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.805131, event: throttled, request: pg_query(22.3b9 epoch 194470) v3 -6629> 2014-08-13 17:22:48.940010 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.805176, event: all_read, request: pg_query(22.3b9 epoch 194470) v3 -6628> 2014-08-13 17:22:48.940017 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.939980, event: dispatched, request: pg_query(22.3b9 epoch 194470) v3 -6627> 2014-08-13 17:22:48.940005 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.231960 2 0.000144 -6626> 2014-08-13 17:22:48.940030 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.940030, event: waiting_for_osdmap, request: pg_query(22.3b9 epoch 194470) v3 -6625> 2014-08-13 17:22:48.940049 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.940049, event: started, request: pg_query(22.3b9 epoch 194470) v3 -6624> 2014-08-13 17:22:48.940044 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.232011 0 0.000000 -6623> 2014-08-13 17:22:48.940070 7f74d8735700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:22:48.940070, event: done, request: pg_query(22.3b9 epoch 194470) v3 -6622> 2014-08-13 17:22:48.940067 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.232045 0 0.000000 -6621> 2014-08-13 17:22:48.940089 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 3 ==== pg_notify(22.775(22) epoch 194470) v5 ==== 1948+0+0 (899104279 0 0) 0x5263100 con 0x4900420 -6620> 2014-08-13 17:22:48.940089 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started 0.232123 0 0.000000 -6619> 2014-08-13 17:22:48.940112 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.932799, event: header_read, request: pg_notify(22.775(22) epoch 194470) v5 -6618> 2014-08-13 17:22:48.940120 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.932801, event: throttled, request: pg_notify(22.775(22) epoch 194470) v5 -6617> 2014-08-13 17:22:48.940120 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194447/194448/194448) [4,5,47] r=0 lpr=194449 pi=193306-194447/18 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset -6616> 2014-08-13 17:22:48.940120 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000452 15 0.000338 -6615> 2014-08-13 17:22:48.940134 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.932905, event: all_read, request: pg_notify(22.775(22) epoch 194470) v5 -6614> 2014-08-13 17:22:48.940169 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.940110, event: dispatched, request: pg_notify(22.775(22) epoch 194470) v5 -6613> 2014-08-13 17:22:48.940181 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.940181, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194470) v5 -6612> 2014-08-13 17:22:48.940192 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.940192, event: started, request: pg_notify(22.775(22) epoch 194470) v5 -6611> 2014-08-13 17:22:48.940169 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started -6610> 2014-08-13 17:22:48.940213 7f74d8735700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:22:48.940213, event: done, request: pg_notify(22.775(22) epoch 194470) v5 -6609> 2014-08-13 17:22:48.940211 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start -6608> 2014-08-13 17:22:48.940228 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6607> 2014-08-13 17:22:48.940242 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 3 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (3753113750 0 0) 0x4ac0540 con 0x4900580 -6606> 2014-08-13 17:22:48.940249 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -6605> 2014-08-13 17:22:48.940264 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.801876, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -6604> 2014-08-13 17:22:48.940275 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.801877, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -6603> 2014-08-13 17:22:48.940270 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6602> 2014-08-13 17:22:48.940284 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.805037, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -6601> 2014-08-13 17:22:48.940294 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.940261, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -6600> 2014-08-13 17:22:48.940287 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6599> 2014-08-13 17:22:48.940308 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.940308, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -6598> 2014-08-13 17:22:48.940328 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.940328, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -6597> 2014-08-13 17:22:48.940321 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194443/194389 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6596> 2014-08-13 17:22:48.940357 7f74d8735700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:22:48.940357, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -6595> 2014-08-13 17:22:48.940376 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 3 ==== pg_notify(22.113 epoch 194470) v5 ==== 694+0+0 (3360918536 0 0) 0x57d3640 con 0x4900f20 -6594> 2014-08-13 17:22:48.940396 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.919258, event: header_read, request: pg_notify(22.113 epoch 194470) v5 -6593> 2014-08-13 17:22:48.940406 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.919262, event: throttled, request: pg_notify(22.113 epoch 194470) v5 -6592> 2014-08-13 17:22:48.940415 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.919381, event: all_read, request: pg_notify(22.113 epoch 194470) v5 -6591> 2014-08-13 17:22:48.940423 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.940394, event: dispatched, request: pg_notify(22.113 epoch 194470) v5 -6590> 2014-08-13 17:22:48.940432 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.940432, event: waiting_for_osdmap, request: pg_notify(22.113 epoch 194470) v5 -6589> 2014-08-13 17:22:48.940442 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.940442, event: started, request: pg_notify(22.113 epoch 194470) v5 -6588> 2014-08-13 17:22:48.940463 7f74d8735700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:22:48.940462, event: done, request: pg_notify(22.113 epoch 194470) v5 -6587> 2014-08-13 17:22:48.940481 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 3 ==== pg_notify(22.3e4(22) epoch 194470) v5 ==== 1948+0+0 (2374870074 0 0) 0x4ac1a40 con 0x4901600 -6586> 2014-08-13 17:22:48.940500 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.805599, event: header_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -6585> 2014-08-13 17:22:48.940510 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.805603, event: throttled, request: pg_notify(22.3e4(22) epoch 194470) v5 -6584> 2014-08-13 17:22:48.940519 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.805662, event: all_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -6583> 2014-08-13 17:22:48.940526 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.940498, event: dispatched, request: pg_notify(22.3e4(22) epoch 194470) v5 -6582> 2014-08-13 17:22:48.940514 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6581> 2014-08-13 17:22:48.940532 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.940532, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194470) v5 -6580> 2014-08-13 17:22:48.940543 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.940543, event: started, request: pg_notify(22.3e4(22) epoch 194470) v5 -6579> 2014-08-13 17:22:48.940542 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.221616 2 0.000252 -6578> 2014-08-13 17:22:48.940567 7f74d8735700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:22:48.940566, event: done, request: pg_notify(22.3e4(22) epoch 194470) v5 -6577> 2014-08-13 17:22:48.940568 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.221671 0 0.000000 -6576> 2014-08-13 17:22:48.940584 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000464 15 0.000331 -6575> 2014-08-13 17:22:48.940594 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.221708 0 0.000000 -6574> 2014-08-13 17:22:48.940593 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 3 ==== pg_notify(22.1f1(22) epoch 194470) v5 ==== 1948+0+0 (597292973 0 0) 0x49e2bc0 con 0x4902680 -6573> 2014-08-13 17:22:48.940613 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started 0.221837 0 0.000000 -6572> 2014-08-13 17:22:48.940616 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started -6571> 2014-08-13 17:22:48.940624 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.822555, event: header_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -6570> 2014-08-13 17:22:48.940640 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.822557, event: throttled, request: pg_notify(22.1f1(22) epoch 194470) v5 -6569> 2014-08-13 17:22:48.940636 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start -6568> 2014-08-13 17:22:48.940650 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.822671, event: all_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -6567> 2014-08-13 17:22:48.940637 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194447/194448/194448) [4,287,309] r=0 lpr=194449 pi=193306-194447/35 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset -6566> 2014-08-13 17:22:48.940661 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.940622, event: dispatched, request: pg_notify(22.1f1(22) epoch 194470) v5 -6565> 2014-08-13 17:22:48.940668 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.940668, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194470) v5 -6564> 2014-08-13 17:22:48.940660 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6563> 2014-08-13 17:22:48.940679 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.940679, event: started, request: pg_notify(22.1f1(22) epoch 194470) v5 -6562> 2014-08-13 17:22:48.940687 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000051 0 0.000000 -6561> 2014-08-13 17:22:48.940712 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6560> 2014-08-13 17:22:48.940731 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6559> 2014-08-13 17:22:48.940749 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194443/194389 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6558> 2014-08-13 17:22:48.940951 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(29) epoch 194470) v5 -- ?+0 0x4ac2300 con 0x47a53e0 -6557> 2014-08-13 17:22:48.940987 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(36) epoch 194470) v5 -- ?+0 0x4ac6ac0 con 0x47a27e0 -6556> 2014-08-13 17:22:48.941007 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(19) epoch 194470) v5 -- ?+0 0x4ac2840 con 0x486a940 -6555> 2014-08-13 17:22:48.941029 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6827/40192 -- pg_query(22.1f1 epoch 194470) v3 -- ?+0 0x29c6900 con 0x47f2ec0 -6554> 2014-08-13 17:22:48.941051 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6813/37929 -- pg_query(22.1f1 epoch 194470) v3 -- ?+0 0x29c7080 con 0x4906e00 -6553> 2014-08-13 17:22:48.941071 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_query(22.707 epoch 194470) v3 -- ?+0 0x29c6ea0 con 0x48d9ce0 -6552> 2014-08-13 17:22:48.941072 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000435 15 0.000301 -6551> 2014-08-13 17:22:48.941087 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6821/54786 -- pg_query(22.707 epoch 194470) v3 -- ?+0 0x29c1fe0 con 0x47a6ca0 -6550> 2014-08-13 17:22:48.941096 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started -6549> 2014-08-13 17:22:48.941104 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6828/12102 -- pg_query(22.1f1 epoch 194470) v3 -- ?+0 0x29c4ec0 con 0x5490580 -6548> 2014-08-13 17:22:48.941111 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start -6547> 2014-08-13 17:22:48.941123 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6836/14923 -- pg_query(22.1f1 epoch 194470) v3 -- ?+0 0x29c14a0 con 0x45f6f60 -6546> 2014-08-13 17:22:48.941126 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6545> 2014-08-13 17:22:48.941141 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_query(22.707 epoch 194470) v3 -- ?+0 0x29c10e0 con 0x475ee00 -6544> 2014-08-13 17:22:48.941143 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 -6543> 2014-08-13 17:22:48.941159 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6812/32413 -- pg_query(22.707 epoch 194470) v3 -- ?+0 0x29c5be0 con 0x4d53700 -6542> 2014-08-13 17:22:48.941170 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6541> 2014-08-13 17:22:48.941195 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6540> 2014-08-13 17:22:48.941200 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6802/41438 -- pg_query(22.707 epoch 194470) v3 -- ?+0 0x29c12c0 con 0x4901600 -6539> 2014-08-13 17:22:48.941210 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194443/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6538> 2014-08-13 17:22:48.941217 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6826/15861 -- pg_query(22.1f1 epoch 194470) v3 -- ?+0 0x29c0f00 con 0x4902680 -6537> 2014-08-13 17:22:48.941287 7f74d8735700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:22:48.941287, event: done, request: pg_notify(22.1f1(22) epoch 194470) v5 -6536> 2014-08-13 17:22:48.941303 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.232866 3 0.000134 -6535> 2014-08-13 17:22:48.941320 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 3 ==== pg_notify(24.30b(28) epoch 194470) v5 ==== 2332+0+0 (2687328995 0 0) 0x53bad80 con 0x49023c0 -6534> 2014-08-13 17:22:48.941339 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.930077, event: header_read, request: pg_notify(24.30b(28) epoch 194470) v5 -6533> 2014-08-13 17:22:48.941332 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started 0.232948 0 0.000000 -6532> 2014-08-13 17:22:48.941346 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.930084, event: throttled, request: pg_notify(24.30b(28) epoch 194470) v5 -6531> 2014-08-13 17:22:48.941352 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.930224, event: all_read, request: pg_notify(24.30b(28) epoch 194470) v5 -6530> 2014-08-13 17:22:48.941359 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.941336, event: dispatched, request: pg_notify(24.30b(28) epoch 194470) v5 -6529> 2014-08-13 17:22:48.941350 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192966) [358,374,4] r=2 lpr=194449 pi=192969-194447/16 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset -6528> 2014-08-13 17:22:48.941366 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.941366, event: waiting_for_osdmap, request: pg_notify(24.30b(28) epoch 194470) v5 -6527> 2014-08-13 17:22:48.941374 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.941374, event: started, request: pg_notify(24.30b(28) epoch 194470) v5 -6526> 2014-08-13 17:22:48.941396 7f74d8735700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:22:48.941396, event: done, request: pg_notify(24.30b(28) epoch 194470) v5 -6525> 2014-08-13 17:22:48.941401 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6524> 2014-08-13 17:22:48.941415 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 3 ==== pg_notify(22.619(22) epoch 194470) v5 ==== 1948+0+0 (737148708 0 0) 0x53b9340 con 0x4902100 -6523> 2014-08-13 17:22:48.941431 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.929714, event: header_read, request: pg_notify(22.619(22) epoch 194470) v5 -6522> 2014-08-13 17:22:48.941425 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.221980 2 0.000216 -6521> 2014-08-13 17:22:48.941438 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.929717, event: throttled, request: pg_notify(22.619(22) epoch 194470) v5 -6520> 2014-08-13 17:22:48.941444 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.929883, event: all_read, request: pg_notify(22.619(22) epoch 194470) v5 -6519> 2014-08-13 17:22:48.941449 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.941429, event: dispatched, request: pg_notify(22.619(22) epoch 194470) v5 -6518> 2014-08-13 17:22:48.941455 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.941454, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194470) v5 -6517> 2014-08-13 17:22:48.941446 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.222024 0 0.000000 -6516> 2014-08-13 17:22:48.941462 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.941462, event: started, request: pg_notify(22.619(22) epoch 194470) v5 -6515> 2014-08-13 17:22:48.941466 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.222059 0 0.000000 -6514> 2014-08-13 17:22:48.941480 7f74d8735700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:22:48.941480, event: done, request: pg_notify(22.619(22) epoch 194470) v5 -6513> 2014-08-13 17:22:48.941484 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started 0.222179 0 0.000000 -6512> 2014-08-13 17:22:48.941498 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 3 ==== pg_notify(2.2a(1) epoch 194470) v5 ==== 761+0+0 (3728076559 0 0) 0x49e08c0 con 0x4903180 -6511> 2014-08-13 17:22:48.941502 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194447/194448/194448) [4,289,341] r=0 lpr=194449 pi=192978-194447/16 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset -6510> 2014-08-13 17:22:48.941513 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.819860, event: header_read, request: pg_notify(2.2a(1) epoch 194470) v5 -6509> 2014-08-13 17:22:48.941519 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.819861, event: throttled, request: pg_notify(2.2a(1) epoch 194470) v5 -6508> 2014-08-13 17:22:48.941524 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.819913, event: all_read, request: pg_notify(2.2a(1) epoch 194470) v5 -6507> 2014-08-13 17:22:48.941529 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.941511, event: dispatched, request: pg_notify(2.2a(1) epoch 194470) v5 -6506> 2014-08-13 17:22:48.941535 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.941535, event: waiting_for_osdmap, request: pg_notify(2.2a(1) epoch 194470) v5 -6505> 2014-08-13 17:22:48.941542 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.941542, event: started, request: pg_notify(2.2a(1) epoch 194470) v5 -6504> 2014-08-13 17:22:48.941560 7f74d8735700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:22:48.941560, event: done, request: pg_notify(2.2a(1) epoch 194470) v5 -6503> 2014-08-13 17:22:48.941583 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 2 ==== pg_notify(24.3d epoch 194470) v5 ==== 694+0+0 (3045703392 0 0) 0x57d6c80 con 0x4904a40 -6502> 2014-08-13 17:22:48.941602 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.928801, event: header_read, request: pg_notify(24.3d epoch 194470) v5 -6501> 2014-08-13 17:22:48.941611 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.928803, event: throttled, request: pg_notify(24.3d epoch 194470) v5 -6500> 2014-08-13 17:22:48.941620 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.928977, event: all_read, request: pg_notify(24.3d epoch 194470) v5 -6499> 2014-08-13 17:22:48.941628 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.941599, event: dispatched, request: pg_notify(24.3d epoch 194470) v5 -6498> 2014-08-13 17:22:48.941638 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.941638, event: waiting_for_osdmap, request: pg_notify(24.3d epoch 194470) v5 -6497> 2014-08-13 17:22:48.941648 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.941648, event: started, request: pg_notify(24.3d epoch 194470) v5 -6496> 2014-08-13 17:22:48.941669 7f74d8735700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:22:48.941669, event: done, request: pg_notify(24.3d epoch 194470) v5 -6495> 2014-08-13 17:22:48.941686 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 3 ==== pg_notify(22.619(22) epoch 194470) v5 ==== 1948+0+0 (737148708 0 0) 0x53ba4c0 con 0x4904780 -6494> 2014-08-13 17:22:48.941711 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.929880, event: header_read, request: pg_notify(22.619(22) epoch 194470) v5 -6493> 2014-08-13 17:22:48.941722 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.929883, event: throttled, request: pg_notify(22.619(22) epoch 194470) v5 -6492> 2014-08-13 17:22:48.941731 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.930009, event: all_read, request: pg_notify(22.619(22) epoch 194470) v5 -6491> 2014-08-13 17:22:48.941740 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.941709, event: dispatched, request: pg_notify(22.619(22) epoch 194470) v5 -6490> 2014-08-13 17:22:48.941749 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.941749, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194470) v5 -6489> 2014-08-13 17:22:48.941760 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.941760, event: started, request: pg_notify(22.619(22) epoch 194470) v5 -6488> 2014-08-13 17:22:48.941782 7f74d8735700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:22:48.941782, event: done, request: pg_notify(22.619(22) epoch 194470) v5 -6487> 2014-08-13 17:22:48.941805 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 3 ==== pg_notify(22.22c(1) epoch 194470) v5 ==== 777+0+0 (3946482321 0 0) 0x57d7700 con 0x49044c0 -6486> 2014-08-13 17:22:48.941824 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.929158, event: header_read, request: pg_notify(22.22c(1) epoch 194470) v5 -6485> 2014-08-13 17:22:48.941835 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.929161, event: throttled, request: pg_notify(22.22c(1) epoch 194470) v5 -6484> 2014-08-13 17:22:48.941843 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.929240, event: all_read, request: pg_notify(22.22c(1) epoch 194470) v5 -6483> 2014-08-13 17:22:48.941852 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.941822, event: dispatched, request: pg_notify(22.22c(1) epoch 194470) v5 -6482> 2014-08-13 17:22:48.941861 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.941861, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194470) v5 -6481> 2014-08-13 17:22:48.941872 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.941872, event: started, request: pg_notify(22.22c(1) epoch 194470) v5 -6480> 2014-08-13 17:22:48.941870 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.000519 15 0.000169 -6479> 2014-08-13 17:22:48.941900 7f74d8735700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:22:48.941900, event: done, request: pg_notify(22.22c(1) epoch 194470) v5 -6478> 2014-08-13 17:22:48.941901 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started -6477> 2014-08-13 17:22:48.941921 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 3 ==== pg_notify(22.775(22) epoch 194470) v5 ==== 1948+0+0 (899104279 0 0) 0x5263480 con 0x49070c0 -6476> 2014-08-13 17:22:48.941921 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start -6475> 2014-08-13 17:22:48.941943 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.933177, event: header_read, request: pg_notify(22.775(22) epoch 194470) v5 -6474> 2014-08-13 17:22:48.941950 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.933180, event: throttled, request: pg_notify(22.775(22) epoch 194470) v5 -6473> 2014-08-13 17:22:48.941960 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.933362, event: all_read, request: pg_notify(22.775(22) epoch 194470) v5 -6472> 2014-08-13 17:22:48.941972 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.941941, event: dispatched, request: pg_notify(22.775(22) epoch 194470) v5 -6471> 2014-08-13 17:22:48.941982 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.941982, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194470) v5 -6470> 2014-08-13 17:22:48.941993 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.941992, event: started, request: pg_notify(22.775(22) epoch 194470) v5 -6469> 2014-08-13 17:22:48.942016 7f74d8735700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:22:48.942016, event: done, request: pg_notify(22.775(22) epoch 194470) v5 -6468> 2014-08-13 17:22:48.942040 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 3 ==== pg_notify(1.2b(2) epoch 194470) v5 ==== 804+0+0 (935963835 0 0) 0x49e7000 con 0x4906b40 -6467> 2014-08-13 17:22:48.942058 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.824451, event: header_read, request: pg_notify(1.2b(2) epoch 194470) v5 -6466> 2014-08-13 17:22:48.942068 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.824453, event: throttled, request: pg_notify(1.2b(2) epoch 194470) v5 -6465> 2014-08-13 17:22:48.942062 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000560 15 0.000281 -6464> 2014-08-13 17:22:48.942078 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.824737, event: all_read, request: pg_notify(1.2b(2) epoch 194470) v5 -6463> 2014-08-13 17:22:48.942100 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.942056, event: dispatched, request: pg_notify(1.2b(2) epoch 194470) v5 -6462> 2014-08-13 17:22:48.942095 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started -6461> 2014-08-13 17:22:48.942112 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.942111, event: waiting_for_osdmap, request: pg_notify(1.2b(2) epoch 194470) v5 -6460> 2014-08-13 17:22:48.942112 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start -6459> 2014-08-13 17:22:48.942126 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.942125, event: started, request: pg_notify(1.2b(2) epoch 194470) v5 -6458> 2014-08-13 17:22:48.942134 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6457> 2014-08-13 17:22:48.942152 7f74d8735700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:22:48.942151, event: done, request: pg_notify(1.2b(2) epoch 194470) v5 -6456> 2014-08-13 17:22:48.941949 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6455> 2014-08-13 17:22:48.942161 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000049 0 0.000000 -6454> 2014-08-13 17:22:48.942195 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 3 ==== pg_notify(22.f3(1) epoch 194470) v5 ==== 777+0+0 (3929559114 0 0) 0x57d7e00 con 0x4906880 -6453> 2014-08-13 17:22:48.942185 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000265 0 0.000000 -6452> 2014-08-13 17:22:48.942215 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.929217, event: header_read, request: pg_notify(22.f3(1) epoch 194470) v5 -6451> 2014-08-13 17:22:48.942217 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray -6450> 2014-08-13 17:22:48.942230 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.929220, event: throttled, request: pg_notify(22.f3(1) epoch 194470) v5 -6449> 2014-08-13 17:22:48.942206 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6448> 2014-08-13 17:22:48.942241 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.929301, event: all_read, request: pg_notify(22.f3(1) epoch 194470) v5 -6447> 2014-08-13 17:22:48.942256 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.942212, event: dispatched, request: pg_notify(22.f3(1) epoch 194470) v5 -6446> 2014-08-13 17:22:48.942253 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6445> 2014-08-13 17:22:48.942267 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.942267, event: waiting_for_osdmap, request: pg_notify(22.f3(1) epoch 194470) v5 -6444> 2014-08-13 17:22:48.942286 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.942286, event: started, request: pg_notify(22.f3(1) epoch 194470) v5 -6443> 2014-08-13 17:22:48.942278 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194443/194389 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6442> 2014-08-13 17:22:48.942330 7f74d8735700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:22:48.942330, event: done, request: pg_notify(22.f3(1) epoch 194470) v5 -6441> 2014-08-13 17:22:48.942350 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 3 ==== pg_notify(22.775(1) epoch 194470) v5 ==== 777+0+0 (716904798 0 0) 0x49e1880 con 0x4d58840 -6440> 2014-08-13 17:22:48.942371 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.821908, event: header_read, request: pg_notify(22.775(1) epoch 194470) v5 -6439> 2014-08-13 17:22:48.942381 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.821912, event: throttled, request: pg_notify(22.775(1) epoch 194470) v5 -6438> 2014-08-13 17:22:48.942364 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: Peering, affected_by_map, going to Reset -6437> 2014-08-13 17:22:48.942391 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.822036, event: all_read, request: pg_notify(22.775(1) epoch 194470) v5 -6436> 2014-08-13 17:22:48.942397 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.942368, event: dispatched, request: pg_notify(22.775(1) epoch 194470) v5 -6435> 2014-08-13 17:22:48.942403 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.942403, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194470) v5 -6434> 2014-08-13 17:22:48.942395 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.233491 2 0.000191 -6433> 2014-08-13 17:22:48.942411 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.942410, event: started, request: pg_notify(22.775(1) epoch 194470) v5 -6432> 2014-08-13 17:22:48.942430 7f74d8735700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:22:48.942430, event: done, request: pg_notify(22.775(1) epoch 194470) v5 -6431> 2014-08-13 17:22:48.942419 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.233538 0 0.000000 -6430> 2014-08-13 17:22:48.942444 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 3 ==== pg_notify(22.1b8(20) epoch 194470) v5 ==== 1838+0+0 (1626369839 0 0) 0x49e2840 con 0x4d58580 -6429> 2014-08-13 17:22:48.942441 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.233593 0 0.000000 -6428> 2014-08-13 17:22:48.942459 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.822559, event: header_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -6427> 2014-08-13 17:22:48.942466 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.822560, event: throttled, request: pg_notify(22.1b8(20) epoch 194470) v5 -6426> 2014-08-13 17:22:48.942471 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.822665, event: all_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -6425> 2014-08-13 17:22:48.942461 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.233674 0 0.000000 -6424> 2014-08-13 17:22:48.942477 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.942457, event: dispatched, request: pg_notify(22.1b8(20) epoch 194470) v5 -6423> 2014-08-13 17:22:48.942483 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.942483, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194470) v5 -6422> 2014-08-13 17:22:48.942491 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.942490, event: started, request: pg_notify(22.1b8(20) epoch 194470) v5 -6421> 2014-08-13 17:22:48.942482 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194447/194448/194448) [4,330,28]/[4,17,315] r=0 lpr=194449 pi=193748-194447/19 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -6420> 2014-08-13 17:22:48.942510 7f74d8735700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:22:48.942509, event: done, request: pg_notify(22.1b8(20) epoch 194470) v5 -6419> 2014-08-13 17:22:48.942508 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.222581 3 0.000189 -6418> 2014-08-13 17:22:48.942531 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 3 ==== pg_notify(22.39d(20) epoch 194470) v5 ==== 1834+0+0 (3883755796 0 0) 0x49e1180 con 0x4d582c0 -6417> 2014-08-13 17:22:48.942546 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.821560, event: header_read, request: pg_notify(22.39d(20) epoch 194470) v5 -6416> 2014-08-13 17:22:48.942552 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.821563, event: throttled, request: pg_notify(22.39d(20) epoch 194470) v5 -6415> 2014-08-13 17:22:48.942542 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started 0.222712 0 0.000000 -6414> 2014-08-13 17:22:48.942557 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.821669, event: all_read, request: pg_notify(22.39d(20) epoch 194470) v5 -6413> 2014-08-13 17:22:48.942565 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.942544, event: dispatched, request: pg_notify(22.39d(20) epoch 194470) v5 -6412> 2014-08-13 17:22:48.942563 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194447/194448/192972) [292,345,4] r=2 lpr=194449 pi=192975-194447/16 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset -6411> 2014-08-13 17:22:48.942576 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.942576, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194470) v5 -6410> 2014-08-13 17:22:48.942598 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.942598, event: started, request: pg_notify(22.39d(20) epoch 194470) v5 -6409> 2014-08-13 17:22:48.942623 7f74d8735700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:22:48.942623, event: done, request: pg_notify(22.39d(20) epoch 194470) v5 -6408> 2014-08-13 17:22:48.942649 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 3 ==== pg_notify(22.481(32) epoch 194470) v5 ==== 2518+0+0 (1131474814 0 0) 0x49e6c80 con 0x4d51ce0 -6407> 2014-08-13 17:22:48.942669 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.824335, event: header_read, request: pg_notify(22.481(32) epoch 194470) v5 -6406> 2014-08-13 17:22:48.942679 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.824339, event: throttled, request: pg_notify(22.481(32) epoch 194470) v5 -6405> 2014-08-13 17:22:48.942688 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.824488, event: all_read, request: pg_notify(22.481(32) epoch 194470) v5 -6404> 2014-08-13 17:22:48.942698 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.942667, event: dispatched, request: pg_notify(22.481(32) epoch 194470) v5 -6403> 2014-08-13 17:22:48.942707 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.942707, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194470) v5 -6402> 2014-08-13 17:22:48.942719 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.942718, event: started, request: pg_notify(22.481(32) epoch 194470) v5 -6401> 2014-08-13 17:22:48.942745 7f74d8735700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:22:48.942745, event: done, request: pg_notify(22.481(32) epoch 194470) v5 -6400> 2014-08-13 17:22:48.942774 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 3 ==== pg_notify(22.619 epoch 194470) v5 ==== 694+0+0 (1863175669 0 0) 0x57d6580 con 0x4d51a20 -6399> 2014-08-13 17:22:48.942792 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.928761, event: header_read, request: pg_notify(22.619 epoch 194470) v5 -6398> 2014-08-13 17:22:48.942802 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.928764, event: throttled, request: pg_notify(22.619 epoch 194470) v5 -6397> 2014-08-13 17:22:48.942811 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.928830, event: all_read, request: pg_notify(22.619 epoch 194470) v5 -6396> 2014-08-13 17:22:48.942819 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.942789, event: dispatched, request: pg_notify(22.619 epoch 194470) v5 -6395> 2014-08-13 17:22:48.942829 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.942828, event: waiting_for_osdmap, request: pg_notify(22.619 epoch 194470) v5 -6394> 2014-08-13 17:22:48.942839 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.942839, event: started, request: pg_notify(22.619 epoch 194470) v5 -6393> 2014-08-13 17:22:48.942860 7f74d8735700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:22:48.942860, event: done, request: pg_notify(22.619 epoch 194470) v5 -6392> 2014-08-13 17:22:48.942877 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 2 ==== pg_notify(24.9b epoch 194470) v5 ==== 694+0+0 (4133067690 0 0) 0x53ba680 con 0x4d511e0 -6391> 2014-08-13 17:22:48.942894 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.929964, event: header_read, request: pg_notify(24.9b epoch 194470) v5 -6390> 2014-08-13 17:22:48.942904 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.929966, event: throttled, request: pg_notify(24.9b epoch 194470) v5 -6389> 2014-08-13 17:22:48.942913 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.930038, event: all_read, request: pg_notify(24.9b epoch 194470) v5 -6388> 2014-08-13 17:22:48.942922 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.942892, event: dispatched, request: pg_notify(24.9b epoch 194470) v5 -6387> 2014-08-13 17:22:48.942931 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.942931, event: waiting_for_osdmap, request: pg_notify(24.9b epoch 194470) v5 -6386> 2014-08-13 17:22:48.942941 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.942941, event: started, request: pg_notify(24.9b epoch 194470) v5 -6385> 2014-08-13 17:22:48.942961 7f74d8735700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:22:48.942961, event: done, request: pg_notify(24.9b epoch 194470) v5 -6384> 2014-08-13 17:22:48.942977 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 3 ==== pg_notify(22.f3(18) epoch 194470) v5 ==== 1720+0+0 (3451712932 0 0) 0x53b81c0 con 0x4d54200 -6383> 2014-08-13 17:22:48.942995 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.929313, event: header_read, request: pg_notify(22.f3(18) epoch 194470) v5 -6382> 2014-08-13 17:22:48.943004 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.929316, event: throttled, request: pg_notify(22.f3(18) epoch 194470) v5 -6381> 2014-08-13 17:22:48.943014 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.929419, event: all_read, request: pg_notify(22.f3(18) epoch 194470) v5 -6380> 2014-08-13 17:22:48.943022 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.942993, event: dispatched, request: pg_notify(22.f3(18) epoch 194470) v5 -6379> 2014-08-13 17:22:48.943032 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.943032, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194470) v5 -6378> 2014-08-13 17:22:48.943043 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.943043, event: started, request: pg_notify(22.f3(18) epoch 194470) v5 -6377> 2014-08-13 17:22:48.943065 7f74d8735700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:22:48.943065, event: done, request: pg_notify(22.f3(18) epoch 194470) v5 -6376> 2014-08-13 17:22:48.943095 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 3 ==== pg_notify(22.39d(1) epoch 194470) v5 ==== 777+0+0 (380101991 0 0) 0x57d5400 con 0x4d53c80 -6375> 2014-08-13 17:22:48.943114 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.926036, event: header_read, request: pg_notify(22.39d(1) epoch 194470) v5 -6374> 2014-08-13 17:22:48.943106 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.000624 15 0.000303 -6373> 2014-08-13 17:22:48.943125 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.926039, event: throttled, request: pg_notify(22.39d(1) epoch 194470) v5 -6372> 2014-08-13 17:22:48.943146 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.926115, event: all_read, request: pg_notify(22.39d(1) epoch 194470) v5 -6371> 2014-08-13 17:22:48.943141 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started -6370> 2014-08-13 17:22:48.943157 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.943112, event: dispatched, request: pg_notify(22.39d(1) epoch 194470) v5 -6369> 2014-08-13 17:22:48.943174 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.943174, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194470) v5 -6368> 2014-08-13 17:22:48.943159 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Start -6367> 2014-08-13 17:22:48.943183 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.943183, event: started, request: pg_notify(22.39d(1) epoch 194470) v5 -6366> 2014-08-13 17:22:48.943182 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary -6365> 2014-08-13 17:22:48.943208 7f74d8735700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:22:48.943208, event: done, request: pg_notify(22.39d(1) epoch 194470) v5 -6364> 2014-08-13 17:22:48.943197 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.000634 15 0.000206 -6363> 2014-08-13 17:22:48.943206 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000047 0 0.000000 -6362> 2014-08-13 17:22:48.943226 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.310 192.168.1.109:6812/32413 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4e37980 con 0x4d53700 -6361> 2014-08-13 17:22:48.943236 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started -6360> 2014-08-13 17:22:48.943250 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6359> 2014-08-13 17:22:48.943243 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -6358> 2014-08-13 17:22:48.943259 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start -6357> 2014-08-13 17:22:48.943267 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering -6356> 2014-08-13 17:22:48.943273 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 3 ==== pg_notify(22.4df(20) epoch 194470) v5 ==== 1834+0+0 (819295757 0 0) 0x57d4440 con 0x4d56040 -6355> 2014-08-13 17:22:48.943278 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6354> 2014-08-13 17:22:48.943292 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194443/194389 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo -6353> 2014-08-13 17:22:48.943304 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.924271, event: header_read, request: pg_notify(22.4df(20) epoch 194470) v5 -6352> 2014-08-13 17:22:48.943316 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.924274, event: throttled, request: pg_notify(22.4df(20) epoch 194470) v5 -6351> 2014-08-13 17:22:48.943312 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000054 0 0.000000 -6350> 2014-08-13 17:22:48.943326 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.924367, event: all_read, request: pg_notify(22.4df(20) epoch 194470) v5 -6349> 2014-08-13 17:22:48.943334 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.943302, event: dispatched, request: pg_notify(22.4df(20) epoch 194470) v5 -6348> 2014-08-13 17:22:48.943345 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.943344, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194470) v5 -6347> 2014-08-13 17:22:48.943337 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194449/194449 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray -6346> 2014-08-13 17:22:48.943360 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.943360, event: started, request: pg_notify(22.4df(20) epoch 194470) v5 -6345> 2014-08-13 17:22:48.943380 7f74d8735700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:22:48.943380, event: done, request: pg_notify(22.4df(20) epoch 194470) v5 -6344> 2014-08-13 17:22:48.943408 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 3 ==== pg_notify(22.1d2 epoch 194470) v5 ==== 694+0+0 (1888888175 0 0) 0x5264b40 con 0x4d55d80 -6343> 2014-08-13 17:22:48.943428 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.919106, event: header_read, request: pg_notify(22.1d2 epoch 194470) v5 -6342> 2014-08-13 17:22:48.943438 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.919109, event: throttled, request: pg_notify(22.1d2 epoch 194470) v5 -6341> 2014-08-13 17:22:48.943447 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.919194, event: all_read, request: pg_notify(22.1d2 epoch 194470) v5 -6340> 2014-08-13 17:22:48.943456 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.943426, event: dispatched, request: pg_notify(22.1d2 epoch 194470) v5 -6339> 2014-08-13 17:22:48.943465 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.943465, event: waiting_for_osdmap, request: pg_notify(22.1d2 epoch 194470) v5 -6338> 2014-08-13 17:22:48.943455 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.234033 3 0.000163 -6337> 2014-08-13 17:22:48.943476 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.943476, event: started, request: pg_notify(22.1d2 epoch 194470) v5 -6336> 2014-08-13 17:22:48.943480 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.234152 0 0.000000 -6335> 2014-08-13 17:22:48.943498 7f74d8735700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:22:48.943497, event: done, request: pg_notify(22.1d2 epoch 194470) v5 -6334> 2014-08-13 17:22:48.943487 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 0.223172 3 0.000203 -6333> 2014-08-13 17:22:48.943504 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194447/194448/192978) [373,360,4] r=2 lpr=194449 pi=192978-194447/16 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -6332> 2014-08-13 17:22:48.943519 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 3 ==== pg_notify(24.9b(26) epoch 194470) v5 ==== 2214+0+0 (801795 0 0) 0x53baa00 con 0x4d55ac0 -6331> 2014-08-13 17:22:48.943533 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.930073, event: header_read, request: pg_notify(24.9b(26) epoch 194470) v5 -6330> 2014-08-13 17:22:48.943522 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started 0.223310 0 0.000000 -6329> 2014-08-13 17:22:48.943541 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.930076, event: throttled, request: pg_notify(24.9b(26) epoch 194470) v5 -6328> 2014-08-13 17:22:48.943547 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.930233, event: all_read, request: pg_notify(24.9b(26) epoch 194470) v5 -6327> 2014-08-13 17:22:48.943552 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.943531, event: dispatched, request: pg_notify(24.9b(26) epoch 194470) v5 -6326> 2014-08-13 17:22:48.943544 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194447/194447/193262) [343,45,4] r=2 lpr=194449 pi=193262-194446/19 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset -6325> 2014-08-13 17:22:48.943558 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.943558, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194470) v5 -6324> 2014-08-13 17:22:48.943565 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.943565, event: started, request: pg_notify(24.9b(26) epoch 194470) v5 -6323> 2014-08-13 17:22:48.943586 7f74d8735700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:22:48.943586, event: done, request: pg_notify(24.9b(26) epoch 194470) v5 -6322> 2014-08-13 17:22:48.943611 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 3 ==== pg_notify(22.4df(1) epoch 194470) v5 ==== 777+0+0 (3297624547 0 0) 0x57d3d40 con 0x4d57d20 -6321> 2014-08-13 17:22:48.943631 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.924000, event: header_read, request: pg_notify(22.4df(1) epoch 194470) v5 -6320> 2014-08-13 17:22:48.943641 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.924003, event: throttled, request: pg_notify(22.4df(1) epoch 194470) v5 -6319> 2014-08-13 17:22:48.943650 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.924089, event: all_read, request: pg_notify(22.4df(1) epoch 194470) v5 -6318> 2014-08-13 17:22:48.943659 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.943629, event: dispatched, request: pg_notify(22.4df(1) epoch 194470) v5 -6317> 2014-08-13 17:22:48.943669 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.943669, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194470) v5 -6316> 2014-08-13 17:22:48.943680 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.943679, event: started, request: pg_notify(22.4df(1) epoch 194470) v5 -6315> 2014-08-13 17:22:48.943702 7f74d8735700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:22:48.943701, event: done, request: pg_notify(22.4df(1) epoch 194470) v5 -6314> 2014-08-13 17:22:48.943726 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.357 192.168.1.102:6834/17224 2 ==== pg_notify(22.4df epoch 194470) v5 ==== 694+0+0 (1928747819 0 0) 0x57d4600 con 0x4d37a60 -6313> 2014-08-13 17:22:48.943744 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.924603, event: header_read, request: pg_notify(22.4df epoch 194470) v5 -6312> 2014-08-13 17:22:48.943756 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.924604, event: throttled, request: pg_notify(22.4df epoch 194470) v5 -6311> 2014-08-13 17:22:48.943762 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.924666, event: all_read, request: pg_notify(22.4df epoch 194470) v5 -6310> 2014-08-13 17:22:48.943769 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.943742, event: dispatched, request: pg_notify(22.4df epoch 194470) v5 -6309> 2014-08-13 17:22:48.943778 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.943778, event: waiting_for_osdmap, request: pg_notify(22.4df epoch 194470) v5 -6308> 2014-08-13 17:22:48.943785 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.943785, event: started, request: pg_notify(22.4df epoch 194470) v5 -6307> 2014-08-13 17:22:48.943801 7f74d8735700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:22:48.943801, event: done, request: pg_notify(22.4df epoch 194470) v5 -6306> 2014-08-13 17:22:48.943821 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.353 192.168.1.108:6836/6227 2 ==== pg_notify(22.775 epoch 194470) v5 ==== 694+0+0 (720552278 0 0) 0x57d1c00 con 0x532c360 -6305> 2014-08-13 17:22:48.943835 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.933490, event: header_read, request: pg_notify(22.775 epoch 194470) v5 -6304> 2014-08-13 17:22:48.943841 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.933492, event: throttled, request: pg_notify(22.775 epoch 194470) v5 -6303> 2014-08-13 17:22:48.943846 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.933547, event: all_read, request: pg_notify(22.775 epoch 194470) v5 -6302> 2014-08-13 17:22:48.943850 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.943833, event: dispatched, request: pg_notify(22.775 epoch 194470) v5 -6301> 2014-08-13 17:22:48.943856 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.943856, event: waiting_for_osdmap, request: pg_notify(22.775 epoch 194470) v5 -6300> 2014-08-13 17:22:48.943863 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.943862, event: started, request: pg_notify(22.775 epoch 194470) v5 -6299> 2014-08-13 17:22:48.943884 7f74d8735700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:22:48.943884, event: done, request: pg_notify(22.775 epoch 194470) v5 -6298> 2014-08-13 17:22:48.943905 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.318 192.168.1.110:6814/43857 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c77740 con 0x541c4c0 -6297> 2014-08-13 17:22:48.943926 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6296> 2014-08-13 17:22:48.943938 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.47 192.168.1.111:6828/12102 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c76300 con 0x5490580 -6295> 2014-08-13 17:22:48.943952 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6294> 2014-08-13 17:22:48.943969 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 4 ==== pg_notify(22.1d2(22) epoch 194470) v5 ==== 1952+0+0 (3049200412 0 0) 0x57d2bc0 con 0x45f1e40 -6293> 2014-08-13 17:22:48.943987 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.919140, event: header_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -6292> 2014-08-13 17:22:48.943995 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.919142, event: throttled, request: pg_notify(22.1d2(22) epoch 194470) v5 -6291> 2014-08-13 17:22:48.944004 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.919275, event: all_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -6290> 2014-08-13 17:22:48.944013 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.943985, event: dispatched, request: pg_notify(22.1d2(22) epoch 194470) v5 -6289> 2014-08-13 17:22:48.944022 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.944022, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194470) v5 -6288> 2014-08-13 17:22:48.944033 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.944033, event: started, request: pg_notify(22.1d2(22) epoch 194470) v5 -6287> 2014-08-13 17:22:48.944056 7f74d8735700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:22:48.944056, event: done, request: pg_notify(22.1d2(22) epoch 194470) v5 -6286> 2014-08-13 17:22:48.944085 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 4 ==== pg_notify(24.30b(1) epoch 194470) v5 ==== 811+0+0 (3328686156 0 0) 0x57d78c0 con 0x45f5540 -6285> 2014-08-13 17:22:48.944105 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.929311, event: header_read, request: pg_notify(24.30b(1) epoch 194470) v5 -6284> 2014-08-13 17:22:48.944116 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.929312, event: throttled, request: pg_notify(24.30b(1) epoch 194470) v5 -6283> 2014-08-13 17:22:48.944125 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.929369, event: all_read, request: pg_notify(24.30b(1) epoch 194470) v5 -6282> 2014-08-13 17:22:48.944134 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.944103, event: dispatched, request: pg_notify(24.30b(1) epoch 194470) v5 -6281> 2014-08-13 17:22:48.944143 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.944143, event: waiting_for_osdmap, request: pg_notify(24.30b(1) epoch 194470) v5 -6280> 2014-08-13 17:22:48.944140 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.000635 15 0.000185 -6279> 2014-08-13 17:22:48.944183 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.944183, event: started, request: pg_notify(24.30b(1) epoch 194470) v5 -6278> 2014-08-13 17:22:48.944174 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Reset 0.000631 15 0.000213 -6277> 2014-08-13 17:22:48.944194 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -6276> 2014-08-13 17:22:48.944213 7f74d8735700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:22:48.944213, event: done, request: pg_notify(24.30b(1) epoch 194470) v5 -6275> 2014-08-13 17:22:48.944207 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started -6274> 2014-08-13 17:22:48.944216 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -6273> 2014-08-13 17:22:48.944227 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Start -6272> 2014-08-13 17:22:48.944236 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 4 ==== pg_query(24.3b7 epoch 194470) v3 ==== 144+0+0 (3334172972 0 0) 0x29c1e00 con 0x45f5960 -6271> 2014-08-13 17:22:48.944233 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6270> 2014-08-13 17:22:48.944252 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.800082, event: header_read, request: pg_query(24.3b7 epoch 194470) v3 -6269> 2014-08-13 17:22:48.944262 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.800084, event: throttled, request: pg_query(24.3b7 epoch 194470) v3 -6268> 2014-08-13 17:22:48.944253 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: transitioning to Stray -6267> 2014-08-13 17:22:48.944272 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.800122, event: all_read, request: pg_query(24.3b7 epoch 194470) v3 -6266> 2014-08-13 17:22:48.944264 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000048 0 0.000000 -6265> 2014-08-13 17:22:48.944281 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.944250, event: dispatched, request: pg_query(24.3b7 epoch 194470) v3 -6264> 2014-08-13 17:22:48.944291 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.944290, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194470) v3 -6263> 2014-08-13 17:22:48.944287 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194449/194449 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -6262> 2014-08-13 17:22:48.944276 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Start 0.000049 0 0.000000 -6261> 2014-08-13 17:22:48.944306 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.944306, event: started, request: pg_query(24.3b7 epoch 194470) v3 -6260> 2014-08-13 17:22:48.944312 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194448/194442 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started/Stray -6259> 2014-08-13 17:22:48.944327 7f74d8735700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:22:48.944326, event: done, request: pg_query(24.3b7 epoch 194470) v3 -6258> 2014-08-13 17:22:48.944364 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.62 192.168.1.112:6824/5505 2 ==== pg_notify(22.619 epoch 194470) v5 ==== 694+0+0 (1863175669 0 0) 0x57d6e40 con 0x45f77a0 -6257> 2014-08-13 17:22:48.944384 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.928993, event: header_read, request: pg_notify(22.619 epoch 194470) v5 -6256> 2014-08-13 17:22:48.944394 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.928995, event: throttled, request: pg_notify(22.619 epoch 194470) v5 -6255> 2014-08-13 17:22:48.944403 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.929058, event: all_read, request: pg_notify(22.619 epoch 194470) v5 -6254> 2014-08-13 17:22:48.944411 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.944382, event: dispatched, request: pg_notify(22.619 epoch 194470) v5 -6253> 2014-08-13 17:22:48.944420 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.944420, event: waiting_for_osdmap, request: pg_notify(22.619 epoch 194470) v5 -6252> 2014-08-13 17:22:48.944431 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.944430, event: started, request: pg_notify(22.619 epoch 194470) v5 -6251> 2014-08-13 17:22:48.944432 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 0.234571 3 0.000165 -6250> 2014-08-13 17:22:48.944453 7f74d8735700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:22:48.944452, event: done, request: pg_notify(22.619 epoch 194470) v5 -6249> 2014-08-13 17:22:48.944466 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started 0.234683 0 0.000000 -6248> 2014-08-13 17:22:48.944464 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: Peering, affected_by_map, going to Reset -6247> 2014-08-13 17:22:48.944486 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.344 192.168.1.106:6812/6757 2 ==== pg_notify(22.39d epoch 194470) v5 ==== 694+0+0 (2816481024 0 0) 0x57d55c0 con 0x45f6300 -6246> 2014-08-13 17:22:48.944502 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.926112, event: header_read, request: pg_notify(22.39d epoch 194470) v5 -6245> 2014-08-13 17:22:48.944491 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/193151) [356,364,4] r=2 lpr=194449 pi=193151-194446/19 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset -6244> 2014-08-13 17:22:48.944494 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetInfo 0.223807 2 0.000198 -6243> 2014-08-13 17:22:48.944510 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.926113, event: throttled, request: pg_notify(22.39d epoch 194470) v5 -6242> 2014-08-13 17:22:48.944515 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.926175, event: all_read, request: pg_notify(22.39d epoch 194470) v5 -6241> 2014-08-13 17:22:48.944526 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.944500, event: dispatched, request: pg_notify(22.39d epoch 194470) v5 -6240> 2014-08-13 17:22:48.944518 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering 0.223850 0 0.000000 -6239> 2014-08-13 17:22:48.944536 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.944536, event: waiting_for_osdmap, request: pg_notify(22.39d epoch 194470) v5 -6238> 2014-08-13 17:22:48.944547 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.944546, event: started, request: pg_notify(22.39d epoch 194470) v5 -6237> 2014-08-13 17:22:48.944541 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.223893 0 0.000000 -6236> 2014-08-13 17:22:48.944560 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started 0.224008 0 0.000000 -6235> 2014-08-13 17:22:48.944569 7f74d8735700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:22:48.944568, event: done, request: pg_notify(22.39d epoch 194470) v5 -6234> 2014-08-13 17:22:48.944589 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194447/194448/194448) [4,348,331]/[4,332,337] r=0 lpr=194449 pi=193644-194447/18 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset -6233> 2014-08-13 17:22:48.944612 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 4 ==== pg_notify(22.1f1(22) epoch 194470) v5 ==== 1948+0+0 (597292973 0 0) 0x57d16c0 con 0x45f6f60 -6232> 2014-08-13 17:22:48.944633 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.942340, event: header_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -6231> 2014-08-13 17:22:48.944644 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.942341, event: throttled, request: pg_notify(22.1f1(22) epoch 194470) v5 -6230> 2014-08-13 17:22:48.944653 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.942460, event: all_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -6229> 2014-08-13 17:22:48.944662 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.944631, event: dispatched, request: pg_notify(22.1f1(22) epoch 194470) v5 -6228> 2014-08-13 17:22:48.944672 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.944671, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194470) v5 -6227> 2014-08-13 17:22:48.944683 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.944682, event: started, request: pg_notify(22.1f1(22) epoch 194470) v5 -6226> 2014-08-13 17:22:48.944706 7f74d8735700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:22:48.944705, event: done, request: pg_notify(22.1f1(22) epoch 194470) v5 -6225> 2014-08-13 17:22:48.944740 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 4 ==== pg_notify(22.99(18) epoch 194470) v5 ==== 1720+0+0 (3779630166 0 0) 0x466f8c0 con 0x4758840 -6224> 2014-08-13 17:22:48.944760 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.923774, event: header_read, request: pg_notify(22.99(18) epoch 194470) v5 -6223> 2014-08-13 17:22:48.944770 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.923777, event: throttled, request: pg_notify(22.99(18) epoch 194470) v5 -6222> 2014-08-13 17:22:48.944781 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.923898, event: all_read, request: pg_notify(22.99(18) epoch 194470) v5 -6221> 2014-08-13 17:22:48.944790 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.944757, event: dispatched, request: pg_notify(22.99(18) epoch 194470) v5 -6220> 2014-08-13 17:22:48.944800 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.944800, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194470) v5 -6219> 2014-08-13 17:22:48.944811 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.944811, event: started, request: pg_notify(22.99(18) epoch 194470) v5 -6218> 2014-08-13 17:22:48.944833 7f74d8735700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:22:48.944833, event: done, request: pg_notify(22.99(18) epoch 194470) v5 -6217> 2014-08-13 17:22:48.944858 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.56 192.168.1.111:6826/11446 2 ==== pg_notify(22.775 epoch 194470) v5 ==== 694+0+0 (720552278 0 0) 0x5262d80 con 0x475aec0 -6216> 2014-08-13 17:22:48.944877 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.932791, event: header_read, request: pg_notify(22.775 epoch 194470) v5 -6215> 2014-08-13 17:22:48.944887 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.932792, event: throttled, request: pg_notify(22.775 epoch 194470) v5 -6214> 2014-08-13 17:22:48.944896 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.932852, event: all_read, request: pg_notify(22.775 epoch 194470) v5 -6213> 2014-08-13 17:22:48.944904 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.944875, event: dispatched, request: pg_notify(22.775 epoch 194470) v5 -6212> 2014-08-13 17:22:48.944914 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.944914, event: waiting_for_osdmap, request: pg_notify(22.775 epoch 194470) v5 -6211> 2014-08-13 17:22:48.944924 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.944924, event: started, request: pg_notify(22.775 epoch 194470) v5 -6210> 2014-08-13 17:22:48.944945 7f74d8735700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:22:48.944945, event: done, request: pg_notify(22.775 epoch 194470) v5 -6209> 2014-08-13 17:22:48.944964 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 4 ==== pg_notify(24.3d(23) epoch 194470) v5 ==== 2037+0+0 (2598945574 0 0) 0x4ac3f00 con 0x475ee00 -6208> 2014-08-13 17:22:48.944983 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.812097, event: header_read, request: pg_notify(24.3d(23) epoch 194470) v5 -6207> 2014-08-13 17:22:48.944993 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.812098, event: throttled, request: pg_notify(24.3d(23) epoch 194470) v5 -6206> 2014-08-13 17:22:48.945002 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.813974, event: all_read, request: pg_notify(24.3d(23) epoch 194470) v5 -6205> 2014-08-13 17:22:48.945011 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.944981, event: dispatched, request: pg_notify(24.3d(23) epoch 194470) v5 -6204> 2014-08-13 17:22:48.945020 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.945020, event: waiting_for_osdmap, request: pg_notify(24.3d(23) epoch 194470) v5 -6203> 2014-08-13 17:22:48.945031 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.945031, event: started, request: pg_notify(24.3d(23) epoch 194470) v5 -6202> 2014-08-13 17:22:48.945033 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Reset 0.000543 15 0.000209 -6201> 2014-08-13 17:22:48.945054 7f74d8735700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:22:48.945054, event: done, request: pg_notify(24.3d(23) epoch 194470) v5 -6200> 2014-08-13 17:22:48.945070 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started -6199> 2014-08-13 17:22:48.945091 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 4 ==== pg_notify(22.48e(1) epoch 194470) v5 ==== 777+0+0 (2821184580 0 0) 0x4ac32c0 con 0x47a27e0 -6198> 2014-08-13 17:22:48.945091 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Start -6197> 2014-08-13 17:22:48.945116 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.806924, event: header_read, request: pg_notify(22.48e(1) epoch 194470) v5 -6196> 2014-08-13 17:22:48.945126 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.806926, event: throttled, request: pg_notify(22.48e(1) epoch 194470) v5 -6195> 2014-08-13 17:22:48.945122 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: transitioning to Stray -6194> 2014-08-13 17:22:48.945135 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.806991, event: all_read, request: pg_notify(22.48e(1) epoch 194470) v5 -6193> 2014-08-13 17:22:48.945150 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Start 0.000059 0 0.000000 -6192> 2014-08-13 17:22:48.945157 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.945113, event: dispatched, request: pg_notify(22.48e(1) epoch 194470) v5 -6191> 2014-08-13 17:22:48.945172 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.945171, event: waiting_for_osdmap, request: pg_notify(22.48e(1) epoch 194470) v5 -6190> 2014-08-13 17:22:48.945181 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.945181, event: started, request: pg_notify(22.48e(1) epoch 194470) v5 -6189> 2014-08-13 17:22:48.945168 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Reset 0.000563 15 0.000292 -6188> 2014-08-13 17:22:48.945177 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started/Stray -6187> 2014-08-13 17:22:48.945207 7f74d8735700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:22:48.945207, event: done, request: pg_notify(22.48e(1) epoch 194470) v5 -6186> 2014-08-13 17:22:48.945199 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started -6185> 2014-08-13 17:22:48.945232 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Start -6184> 2014-08-13 17:22:48.945237 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 4 ==== pg_notify(22.1d2 epoch 194470) v5 ==== 694+0+0 (1888888175 0 0) 0x4ac47c0 con 0x47a0b00 -6183> 2014-08-13 17:22:48.945256 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: transitioning to Primary -6182> 2014-08-13 17:22:48.945265 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.918749, event: header_read, request: pg_notify(22.1d2 epoch 194470) v5 -6181> 2014-08-13 17:22:48.945287 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.918751, event: throttled, request: pg_notify(22.1d2 epoch 194470) v5 -6180> 2014-08-13 17:22:48.945282 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Start 0.000051 0 0.000000 -6179> 2014-08-13 17:22:48.945298 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.918833, event: all_read, request: pg_notify(22.1d2 epoch 194470) v5 -6178> 2014-08-13 17:22:48.945307 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.945263, event: dispatched, request: pg_notify(22.1d2 epoch 194470) v5 -6177> 2014-08-13 17:22:48.945302 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary -6176> 2014-08-13 17:22:48.945316 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.945316, event: waiting_for_osdmap, request: pg_notify(22.1d2 epoch 194470) v5 -6175> 2014-08-13 17:22:48.945327 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.945327, event: started, request: pg_notify(22.1d2 epoch 194470) v5 -6174> 2014-08-13 17:22:48.945320 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering -6173> 2014-08-13 17:22:48.945326 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: Peering, affected_by_map, going to Reset -6172> 2014-08-13 17:22:48.945354 7f74d8735700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:22:48.945354, event: done, request: pg_notify(22.1d2 epoch 194470) v5 -6171> 2014-08-13 17:22:48.945349 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194443/194388 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetInfo -6170> 2014-08-13 17:22:48.945359 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetInfo 0.235896 2 0.000210 -6169> 2014-08-13 17:22:48.945382 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.313 192.168.1.109:6820/35445 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x2712400 con 0x47a1b80 -6168> 2014-08-13 17:22:48.945386 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering 0.235944 0 0.000000 -6167> 2014-08-13 17:22:48.945405 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6166> 2014-08-13 17:22:48.945412 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.235989 0 0.000000 -6165> 2014-08-13 17:22:48.945419 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 4 ==== pg_notify(22.113 epoch 194470) v5 ==== 694+0+0 (3360918536 0 0) 0x57d2f40 con 0x47a32e0 -6164> 2014-08-13 17:22:48.945445 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.919209, event: header_read, request: pg_notify(22.113 epoch 194470) v5 -6163> 2014-08-13 17:22:48.945439 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started 0.236113 0 0.000000 -6162> 2014-08-13 17:22:48.945457 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.919212, event: throttled, request: pg_notify(22.113 epoch 194470) v5 -6161> 2014-08-13 17:22:48.945467 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.919360, event: all_read, request: pg_notify(22.113 epoch 194470) v5 -6160> 2014-08-13 17:22:48.945460 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194447/194448/194448) [4,59,351]/[4,351,55] r=0 lpr=194449 pi=193678-194447/19 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset -6159> 2014-08-13 17:22:48.945476 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.945442, event: dispatched, request: pg_notify(22.113 epoch 194470) v5 -6158> 2014-08-13 17:22:48.945485 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.945485, event: waiting_for_osdmap, request: pg_notify(22.113 epoch 194470) v5 -6157> 2014-08-13 17:22:48.945496 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.945495, event: started, request: pg_notify(22.113 epoch 194470) v5 -6156> 2014-08-13 17:22:48.945525 7f74d8735700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:22:48.945524, event: done, request: pg_notify(22.113 epoch 194470) v5 -6155> 2014-08-13 17:22:48.945552 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.21 192.168.1.107:6821/54786 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c76540 con 0x47a6ca0 -6154> 2014-08-13 17:22:48.945574 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6153> 2014-08-13 17:22:48.945567 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(23) epoch 194470) v5 -- ?+0 0x466a680 con 0x475cd00 -6152> 2014-08-13 17:22:48.945594 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 4 ==== pg_notify(22.84(20) epoch 194470) v5 ==== 1838+0+0 (2113891409 0 0) 0x466abc0 con 0x47f0c60 -6151> 2014-08-13 17:22:48.945603 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(19) epoch 194470) v5 -- ?+0 0x57d4ec0 con 0x4868160 -6150> 2014-08-13 17:22:48.945612 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.923754, event: header_read, request: pg_notify(22.84(20) epoch 194470) v5 -6149> 2014-08-13 17:22:48.945619 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.923755, event: throttled, request: pg_notify(22.84(20) epoch 194470) v5 -6148> 2014-08-13 17:22:48.945629 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.923860, event: all_read, request: pg_notify(22.84(20) epoch 194470) v5 -6147> 2014-08-13 17:22:48.945626 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(19) epoch 194470) v5 -- ?+0 0x57d1dc0 con 0x486a680 -6146> 2014-08-13 17:22:48.945642 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.945610, event: dispatched, request: pg_notify(22.84(20) epoch 194470) v5 -6145> 2014-08-13 17:22:48.945650 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.945650, event: waiting_for_osdmap, request: pg_notify(22.84(20) epoch 194470) v5 -6144> 2014-08-13 17:22:48.945656 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(23) epoch 194470) v5 -- ?+0 0x57d1a40 con 0x48377a0 -6143> 2014-08-13 17:22:48.945662 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.945662, event: started, request: pg_notify(22.84(20) epoch 194470) v5 -6142> 2014-08-13 17:22:48.945674 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6822/10782 -- pg_query(22.3af epoch 194470) v3 -- ?+0 0x29c2580 con 0x47a4ba0 -6141> 2014-08-13 17:22:48.945692 7f74d8735700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:22:48.945692, event: done, request: pg_notify(22.84(20) epoch 194470) v5 -6140> 2014-08-13 17:22:48.945695 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6840/16566 -- pg_query(22.3af epoch 194470) v3 -- ?+0 0x29c7bc0 con 0x47a0b00 -6139> 2014-08-13 17:22:48.945714 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6814/33435 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x29c01e0 con 0x48dad60 -6138> 2014-08-13 17:22:48.945739 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_query(22.48e epoch 194470) v3 -- ?+0 0x29c05a0 con 0x47a27e0 -6137> 2014-08-13 17:22:48.945755 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x29c0b40 con 0x475ee00 -6136> 2014-08-13 17:22:48.945773 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6806/6812 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x4eb2b20 con 0x4d55540 -6135> 2014-08-13 17:22:48.945848 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.5 192.168.1.103:6827/40192 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4c74140 con 0x47f2ec0 -6134> 2014-08-13 17:22:48.945878 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -6133> 2014-08-13 17:22:48.945902 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 4 ==== pg_notify(22.113(22) epoch 194470) v5 ==== 1952+0+0 (2442813993 0 0) 0x4ac78c0 con 0x47f7e80 -6132> 2014-08-13 17:22:48.945919 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.919614, event: header_read, request: pg_notify(22.113(22) epoch 194470) v5 -6131> 2014-08-13 17:22:48.945932 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.919616, event: throttled, request: pg_notify(22.113(22) epoch 194470) v5 -6130> 2014-08-13 17:22:48.945921 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Reset 0.000462 15 0.000300 -6129> 2014-08-13 17:22:48.945939 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.919713, event: all_read, request: pg_notify(22.113(22) epoch 194470) v5 -6128> 2014-08-13 17:22:48.945942 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6814/23703 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x4eb2940 con 0x4903440 -6127> 2014-08-13 17:22:48.945957 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.945916, event: dispatched, request: pg_notify(22.113(22) epoch 194470) v5 -6126> 2014-08-13 17:22:48.945955 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started -6125> 2014-08-13 17:22:48.945966 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.945966, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194470) v5 -6124> 2014-08-13 17:22:48.945971 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6804/5112 -- pg_query(22.48e epoch 194470) v3 -- ?+0 0x4eb21c0 con 0x486b700 -6123> 2014-08-13 17:22:48.945976 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.945976, event: started, request: pg_notify(22.113(22) epoch 194470) v5 -6122> 2014-08-13 17:22:48.945973 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Start -6121> 2014-08-13 17:22:48.945989 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6816/837 -- pg_query(24.1bb epoch 194470) v3 -- ?+0 0x29c4740 con 0x486bf40 -6120> 2014-08-13 17:22:48.945988 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: transitioning to Primary -6119> 2014-08-13 17:22:48.946008 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6806/42024 -- pg_query(24.1bb epoch 194470) v3 -- ?+0 0x29c2d00 con 0x48dcd00 -6118> 2014-08-13 17:22:48.946012 7f74d8735700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:22:48.946012, event: done, request: pg_notify(22.113(22) epoch 194470) v5 -6117> 2014-08-13 17:22:48.946008 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Start 0.000036 0 0.000000 -6116> 2014-08-13 17:22:48.946027 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary -6115> 2014-08-13 17:22:48.946034 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_query(24.1bb epoch 194470) v3 -- ?+0 0x29c5820 con 0x4833440 -6114> 2014-08-13 17:22:48.946040 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 4 ==== pg_notify(22.6bd(2) epoch 194470) v5 ==== 828+0+0 (1907705783 0 0) 0x4ac0700 con 0x47f70c0 -6113> 2014-08-13 17:22:48.946043 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering -6112> 2014-08-13 17:22:48.946056 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.936624, event: header_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6111> 2014-08-13 17:22:48.946063 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.936627, event: throttled, request: pg_notify(22.6bd(2) epoch 194470) v5 -6110> 2014-08-13 17:22:48.946068 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.936709, event: all_read, request: pg_notify(22.6bd(2) epoch 194470) v5 -6109> 2014-08-13 17:22:48.946059 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194443/194388 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetInfo -6108> 2014-08-13 17:22:48.946074 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.946054, event: dispatched, request: pg_notify(22.6bd(2) epoch 194470) v5 -6107> 2014-08-13 17:22:48.946080 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.946080, event: waiting_for_osdmap, request: pg_notify(22.6bd(2) epoch 194470) v5 -6106> 2014-08-13 17:22:48.946087 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.946087, event: started, request: pg_notify(22.6bd(2) epoch 194470) v5 -6105> 2014-08-13 17:22:48.946106 7f74d8735700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:22:48.946105, event: done, request: pg_notify(22.6bd(2) epoch 194470) v5 -6104> 2014-08-13 17:22:48.946119 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 4 ==== pg_query(22.4fe epoch 194470) v3 ==== 144+0+0 (4254964790 0 0) 0x29c3660 con 0x4833440 -6103> 2014-08-13 17:22:48.946133 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.807190, event: header_read, request: pg_query(22.4fe epoch 194470) v3 -6102> 2014-08-13 17:22:48.946139 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.807190, event: throttled, request: pg_query(22.4fe epoch 194470) v3 -6101> 2014-08-13 17:22:48.946144 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.807220, event: all_read, request: pg_query(22.4fe epoch 194470) v3 -6100> 2014-08-13 17:22:48.946149 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.946131, event: dispatched, request: pg_query(22.4fe epoch 194470) v3 -6099> 2014-08-13 17:22:48.946154 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.946154, event: waiting_for_osdmap, request: pg_query(22.4fe epoch 194470) v3 -6098> 2014-08-13 17:22:48.946170 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.946170, event: started, request: pg_query(22.4fe epoch 194470) v3 -6097> 2014-08-13 17:22:48.946192 7f74d8735700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:22:48.946192, event: done, request: pg_query(22.4fe epoch 194470) v3 -6096> 2014-08-13 17:22:48.946152 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6818/45268 -- pg_query(22.3af epoch 194470) v3 -- ?+0 0x29c1fe0 con 0x4900f20 -6095> 2014-08-13 17:22:48.946225 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 4 ==== pg_notify(22.4df(20) epoch 194470) v5 ==== 1834+0+0 (819295757 0 0) 0x57d40c0 con 0x4832c00 -6094> 2014-08-13 17:22:48.946249 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.924042, event: header_read, request: pg_notify(22.4df(20) epoch 194470) v5 -6093> 2014-08-13 17:22:48.946260 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.924043, event: throttled, request: pg_notify(22.4df(20) epoch 194470) v5 -6092> 2014-08-13 17:22:48.946266 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.924148, event: all_read, request: pg_notify(22.4df(20) epoch 194470) v5 -6091> 2014-08-13 17:22:48.946275 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.946247, event: dispatched, request: pg_notify(22.4df(20) epoch 194470) v5 -6090> 2014-08-13 17:22:48.946266 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(23) epoch 194470) v5 -- ?+0 0x4ac1c00 con 0x45f5960 -6089> 2014-08-13 17:22:48.946278 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6818/9142 -- pg_query(22.48e epoch 194470) v3 -- ?+0 0x29c6900 con 0x48de880 -6088> 2014-08-13 17:22:48.946284 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.946284, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194470) v5 -6087> 2014-08-13 17:22:48.946310 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(19) epoch 194470) v5 -- ?+0 0x4ac2140 con 0x47f2680 -6086> 2014-08-13 17:22:48.946331 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(19) epoch 194470) v5 -- ?+0 0x57d5400 con 0x486a3c0 -6085> 2014-08-13 17:22:48.946336 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.946336, event: started, request: pg_notify(22.4df(20) epoch 194470) v5 -6084> 2014-08-13 17:22:48.946356 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_query(24.b6 epoch 194470) v3 -- ?+0 0x4eb3c00 con 0x47a53e0 -6083> 2014-08-13 17:22:48.946372 7f74d8735700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:22:48.946371, event: done, request: pg_notify(22.4df(20) epoch 194470) v5 -6082> 2014-08-13 17:22:48.946374 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6805/37391 -- pg_query(24.b6 epoch 194470) v3 -- ?+0 0x4eb3a20 con 0x45f3b20 -6081> 2014-08-13 17:22:48.946391 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6834/6966 -- pg_query(24.1f7 epoch 194470) v3 -- ?+0 0x4eb3840 con 0x47f7e80 -6080> 2014-08-13 17:22:48.946391 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 4 ==== pg_query(22.226 epoch 194470) v3 ==== 144+0+0 (300676971 0 0) 0x29c3480 con 0x4835280 -6079> 2014-08-13 17:22:48.946406 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.807044, event: header_read, request: pg_query(22.226 epoch 194470) v3 -6078> 2014-08-13 17:22:48.946412 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.807045, event: throttled, request: pg_query(22.226 epoch 194470) v3 -6077> 2014-08-13 17:22:48.946409 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6822/5265 -- pg_query(24.1f7 epoch 194470) v3 -- ?+0 0x4eb3660 con 0x54e4360 -6076> 2014-08-13 17:22:48.946417 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.807090, event: all_read, request: pg_query(22.226 epoch 194470) v3 -6075> 2014-08-13 17:22:48.946423 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.946404, event: dispatched, request: pg_query(22.226 epoch 194470) v3 -6074> 2014-08-13 17:22:48.946421 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6800/14161 -- pg_query(24.b6 epoch 194470) v3 -- ?+0 0x4eb3480 con 0x47f6880 -6073> 2014-08-13 17:22:48.946428 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.946428, event: waiting_for_osdmap, request: pg_query(22.226 epoch 194470) v3 -6072> 2014-08-13 17:22:48.946436 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.946436, event: started, request: pg_query(22.226 epoch 194470) v3 -6071> 2014-08-13 17:22:48.946438 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6804/5112 -- pg_query(24.b6 epoch 194470) v3 -- ?+0 0x4eb32a0 con 0x486b700 -6070> 2014-08-13 17:22:48.946450 7f74d8735700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:22:48.946450, event: done, request: pg_query(22.226 epoch 194470) v3 -6069> 2014-08-13 17:22:48.946471 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_query(24.b6 epoch 194470) v3 -- ?+0 0x4eb30c0 con 0x486e040 -6068> 2014-08-13 17:22:48.946472 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 4 ==== pg_notify(22.362 epoch 194470) v5 ==== 694+0+0 (1266644936 0 0) 0x57d7380 con 0x48377a0 -6067> 2014-08-13 17:22:48.946486 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.929133, event: header_read, request: pg_notify(22.362 epoch 194470) v5 -6066> 2014-08-13 17:22:48.946492 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.929134, event: throttled, request: pg_notify(22.362 epoch 194470) v5 -6065> 2014-08-13 17:22:48.946491 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- pg_query(24.1f7 epoch 194470) v3 -- ?+0 0x4eb2ee0 con 0x4868420 -6064> 2014-08-13 17:22:48.946497 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.929220, event: all_read, request: pg_notify(22.362 epoch 194470) v5 -6063> 2014-08-13 17:22:48.946502 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.946484, event: dispatched, request: pg_notify(22.362 epoch 194470) v5 -6062> 2014-08-13 17:22:48.946508 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.946508, event: waiting_for_osdmap, request: pg_notify(22.362 epoch 194470) v5 -6061> 2014-08-13 17:22:48.946514 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.946514, event: started, request: pg_notify(22.362 epoch 194470) v5 -6060> 2014-08-13 17:22:48.946530 7f74d8735700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:22:48.946530, event: done, request: pg_notify(22.362 epoch 194470) v5 -6059> 2014-08-13 17:22:48.946544 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 4 ==== pg_notify(22.39d(20) epoch 194470) v5 ==== 1834+0+0 (3883755796 0 0) 0x4ac4440 con 0x486bc80 -6058> 2014-08-13 17:22:48.946558 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.809431, event: header_read, request: pg_notify(22.39d(20) epoch 194470) v5 -6057> 2014-08-13 17:22:48.946564 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.809432, event: throttled, request: pg_notify(22.39d(20) epoch 194470) v5 -6056> 2014-08-13 17:22:48.946569 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.809485, event: all_read, request: pg_notify(22.39d(20) epoch 194470) v5 -6055> 2014-08-13 17:22:48.946574 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.946556, event: dispatched, request: pg_notify(22.39d(20) epoch 194470) v5 -6054> 2014-08-13 17:22:48.946580 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.946580, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194470) v5 -6053> 2014-08-13 17:22:48.946587 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.946586, event: started, request: pg_notify(22.39d(20) epoch 194470) v5 -6052> 2014-08-13 17:22:48.946604 7f74d8735700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:22:48.946604, event: done, request: pg_notify(22.39d(20) epoch 194470) v5 -6051> 2014-08-13 17:22:48.946623 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_query(24.1f7 epoch 194470) v3 -- ?+0 0x4eb2d00 con 0x4835280 -6050> 2014-08-13 17:22:48.946631 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 4 ==== pg_query(22.1fb epoch 194470) v3 ==== 144+0+0 (1882721188 0 0) 0x29c32a0 con 0x486a3c0 -6049> 2014-08-13 17:22:48.946646 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.805904, event: header_read, request: pg_query(22.1fb epoch 194470) v3 -6048> 2014-08-13 17:22:48.946652 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.805905, event: throttled, request: pg_query(22.1fb epoch 194470) v3 -6047> 2014-08-13 17:22:48.946657 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.805956, event: all_read, request: pg_query(22.1fb epoch 194470) v3 -6046> 2014-08-13 17:22:48.946639 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6800/41204 -- pg_query(24.1bb epoch 194470) v3 -- ?+0 0x29c0f00 con 0x54e1600 -6045> 2014-08-13 17:22:48.946662 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.946644, event: dispatched, request: pg_query(22.1fb epoch 194470) v3 -6044> 2014-08-13 17:22:48.946668 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.946668, event: waiting_for_osdmap, request: pg_query(22.1fb epoch 194470) v3 -6043> 2014-08-13 17:22:48.946670 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6842/8170 -- pg_query(22.3af epoch 194470) v3 -- ?+0 0x29c4ec0 con 0x48db9c0 -6042> 2014-08-13 17:22:48.946675 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.946675, event: started, request: pg_query(22.1fb epoch 194470) v3 -6041> 2014-08-13 17:22:48.946691 7f74d8735700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:22:48.946691, event: done, request: pg_query(22.1fb epoch 194470) v3 -6040> 2014-08-13 17:22:48.946692 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6824/4037 -- pg_query(22.3af epoch 194470) v3 -- ?+0 0x4eb4ce0 con 0x4d58840 -6039> 2014-08-13 17:22:48.946708 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: Peering, affected_by_map, going to Reset -6038> 2014-08-13 17:22:48.946725 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 4 ==== pg_notify(22.707(24) epoch 194470) v5 ==== 2066+0+0 (3384600017 0 0) 0x4ac3d40 con 0x48d9ce0 -6037> 2014-08-13 17:22:48.946743 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.942586, event: header_read, request: pg_notify(22.707(24) epoch 194470) v5 -6036> 2014-08-13 17:22:48.946750 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.942588, event: throttled, request: pg_notify(22.707(24) epoch 194470) v5 -6035> 2014-08-13 17:22:48.946738 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetInfo 0.236812 2 0.000222 -6034> 2014-08-13 17:22:48.946756 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.942703, event: all_read, request: pg_notify(22.707(24) epoch 194470) v5 -6033> 2014-08-13 17:22:48.946762 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.946741, event: dispatched, request: pg_notify(22.707(24) epoch 194470) v5 -6032> 2014-08-13 17:22:48.946768 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.946768, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194470) v5 -6031> 2014-08-13 17:22:48.946761 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering 0.236861 0 0.000000 -6030> 2014-08-13 17:22:48.946776 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.946775, event: started, request: pg_notify(22.707(24) epoch 194470) v5 -6029> 2014-08-13 17:22:48.946784 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.236915 0 0.000000 -6028> 2014-08-13 17:22:48.946799 7f74d8735700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:22:48.946799, event: done, request: pg_notify(22.707(24) epoch 194470) v5 -6027> 2014-08-13 17:22:48.946803 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started 0.237018 0 0.000000 -6026> 2014-08-13 17:22:48.946815 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_query(22.48e epoch 194470) v3 -- ?+0 0x4eb4b00 con 0x4835280 -6025> 2014-08-13 17:22:48.946823 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194447/194448/194448) [4,55,61]/[4,50,65] r=0 lpr=194449 pi=193747-194447/18 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset -6024> 2014-08-13 17:22:48.946843 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 4 ==== pg_notify(22.1d2(22) epoch 194470) v5 ==== 1952+0+0 (3049200412 0 0) 0x57d32c0 con 0x486e040 -6023> 2014-08-13 17:22:48.946864 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.919263, event: header_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -6022> 2014-08-13 17:22:48.946871 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.919264, event: throttled, request: pg_notify(22.1d2(22) epoch 194470) v5 -6021> 2014-08-13 17:22:48.946877 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.919373, event: all_read, request: pg_notify(22.1d2(22) epoch 194470) v5 -6020> 2014-08-13 17:22:48.946882 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.946861, event: dispatched, request: pg_notify(22.1d2(22) epoch 194470) v5 -6019> 2014-08-13 17:22:48.946888 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.946888, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194470) v5 -6018> 2014-08-13 17:22:48.946895 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.946895, event: started, request: pg_notify(22.1d2(22) epoch 194470) v5 -6017> 2014-08-13 17:22:48.946915 7f74d8735700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:22:48.946915, event: done, request: pg_notify(22.1d2(22) epoch 194470) v5 -6016> 2014-08-13 17:22:48.946932 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 4 ==== pg_notify(22.362(22) epoch 194470) v5 ==== 1948+0+0 (1366328122 0 0) 0x53b8380 con 0x48db9c0 -6015> 2014-08-13 17:22:48.946947 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.929563, event: header_read, request: pg_notify(22.362(22) epoch 194470) v5 -6014> 2014-08-13 17:22:48.946953 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.929565, event: throttled, request: pg_notify(22.362(22) epoch 194470) v5 -6013> 2014-08-13 17:22:48.946958 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.929667, event: all_read, request: pg_notify(22.362(22) epoch 194470) v5 -6012> 2014-08-13 17:22:48.946963 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.946945, event: dispatched, request: pg_notify(22.362(22) epoch 194470) v5 -6011> 2014-08-13 17:22:48.946948 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6830/4861 -- pg_query(24.1bb epoch 194470) v3 -- ?+0 0x4eb4920 con 0x486b440 -6010> 2014-08-13 17:22:48.946969 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.946969, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194470) v5 -6009> 2014-08-13 17:22:48.946976 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.946976, event: started, request: pg_notify(22.362(22) epoch 194470) v5 -6008> 2014-08-13 17:22:48.946994 7f74d8735700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:22:48.946994, event: done, request: pg_notify(22.362(22) epoch 194470) v5 -6007> 2014-08-13 17:22:48.946999 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6822/15390 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x4eb4740 con 0x4d58000 -6006> 2014-08-13 17:22:48.947026 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 3 ==== pg_notify(22.481(39) epoch 194470) v5 ==== 2903+0+0 (479616867 0 0) 0x57d3d40 con 0x48dad60 -6005> 2014-08-13 17:22:48.947042 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.946695, event: header_read, request: pg_notify(22.481(39) epoch 194470) v5 -6004> 2014-08-13 17:22:48.947049 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.946697, event: throttled, request: pg_notify(22.481(39) epoch 194470) v5 -6003> 2014-08-13 17:22:48.947055 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.946822, event: all_read, request: pg_notify(22.481(39) epoch 194470) v5 -6002> 2014-08-13 17:22:48.947060 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.947040, event: dispatched, request: pg_notify(22.481(39) epoch 194470) v5 -6001> 2014-08-13 17:22:48.947066 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.947065, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194470) v5 -6000> 2014-08-13 17:22:48.947073 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.947073, event: started, request: pg_notify(22.481(39) epoch 194470) v5 -5999> 2014-08-13 17:22:48.947091 7f74d8735700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:22:48.947090, event: done, request: pg_notify(22.481(39) epoch 194470) v5 -5998> 2014-08-13 17:22:48.947191 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -5997> 2014-08-13 17:22:48.947229 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 4 ==== pg_notify(22.707(24) epoch 194470) v5 ==== 2066+0+0 (3384600017 0 0) 0x4ac1880 con 0x4901600 -5996> 2014-08-13 17:22:48.947245 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.225960 2 0.000287 -5995> 2014-08-13 17:22:48.947259 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.805672, event: header_read, request: pg_notify(22.707(24) epoch 194470) v5 -5994> 2014-08-13 17:22:48.947269 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.805673, event: throttled, request: pg_notify(22.707(24) epoch 194470) v5 -5993> 2014-08-13 17:22:48.947280 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.805737, event: all_read, request: pg_notify(22.707(24) epoch 194470) v5 -5992> 2014-08-13 17:22:48.947286 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.947254, event: dispatched, request: pg_notify(22.707(24) epoch 194470) v5 -5991> 2014-08-13 17:22:48.947293 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.947293, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194470) v5 -5990> 2014-08-13 17:22:48.947302 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.947302, event: started, request: pg_notify(22.707(24) epoch 194470) v5 -5989> 2014-08-13 17:22:48.947327 7f74d8735700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:22:48.947326, event: done, request: pg_notify(22.707(24) epoch 194470) v5 -5988> 2014-08-13 17:22:48.947361 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 4 ==== pg_notify(22.22c(20) epoch 194470) v5 ==== 1834+0+0 (2326237699 0 0) 0x49e2a00 con 0x4902680 -5987> 2014-08-13 17:22:48.947378 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.929887, event: header_read, request: pg_notify(22.22c(20) epoch 194470) v5 -5986> 2014-08-13 17:22:48.947384 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.929890, event: throttled, request: pg_notify(22.22c(20) epoch 194470) v5 -5985> 2014-08-13 17:22:48.947390 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.929992, event: all_read, request: pg_notify(22.22c(20) epoch 194470) v5 -5984> 2014-08-13 17:22:48.947395 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.947376, event: dispatched, request: pg_notify(22.22c(20) epoch 194470) v5 -5983> 2014-08-13 17:22:48.947400 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.947400, event: waiting_for_osdmap, request: pg_notify(22.22c(20) epoch 194470) v5 -5982> 2014-08-13 17:22:48.947408 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.947408, event: started, request: pg_notify(22.22c(20) epoch 194470) v5 -5981> 2014-08-13 17:22:48.947427 7f74d8735700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:22:48.947426, event: done, request: pg_notify(22.22c(20) epoch 194470) v5 -5980> 2014-08-13 17:22:48.947269 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.226026 0 0.000000 -5979> 2014-08-13 17:22:48.947463 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.226232 0 0.000000 -5978> 2014-08-13 17:22:48.947483 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started 0.226398 0 0.000000 -5977> 2014-08-13 17:22:48.947502 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194447/194448/194448) [4,343,286] r=0 lpr=194449 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset -5976> 2014-08-13 17:22:48.947531 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Reset 0.000707 15 0.000305 -5975> 2014-08-13 17:22:48.947571 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started -5974> 2014-08-13 17:22:48.947594 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Start -5973> 2014-08-13 17:22:48.947600 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 3 ==== pg_notify(22.481(32) epoch 194470) v5 ==== 2518+0+0 (2179313187 0 0) 0x57d55c0 con 0x4903440 -5972> 2014-08-13 17:22:48.947614 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: transitioning to Primary -5971> 2014-08-13 17:22:48.947630 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947158, event: header_read, request: pg_notify(22.481(32) epoch 194470) v5 -5970> 2014-08-13 17:22:48.947641 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947178, event: throttled, request: pg_notify(22.481(32) epoch 194470) v5 -5969> 2014-08-13 17:22:48.947633 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Start 0.000039 0 0.000000 -5968> 2014-08-13 17:22:48.947656 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947306, event: all_read, request: pg_notify(22.481(32) epoch 194470) v5 -5967> 2014-08-13 17:22:48.947662 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947626, event: dispatched, request: pg_notify(22.481(32) epoch 194470) v5 -5966> 2014-08-13 17:22:48.947669 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947669, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194470) v5 -5965> 2014-08-13 17:22:48.947665 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary -5964> 2014-08-13 17:22:48.947677 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947677, event: started, request: pg_notify(22.481(32) epoch 194470) v5 -5963> 2014-08-13 17:22:48.947685 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering -5962> 2014-08-13 17:22:48.947702 7f74d8735700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:22:48.947702, event: done, request: pg_notify(22.481(32) epoch 194470) v5 -5961> 2014-08-13 17:22:48.947704 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194443/194388 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetInfo -5960> 2014-08-13 17:22:48.947729 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 4 ==== pg_notify(1.2b(1) epoch 194470) v5 ==== 761+0+0 (3404807776 0 0) 0x49e0700 con 0x4903180 -5959> 2014-08-13 17:22:48.947745 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.819923, event: header_read, request: pg_notify(1.2b(1) epoch 194470) v5 -5958> 2014-08-13 17:22:48.947752 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.819924, event: throttled, request: pg_notify(1.2b(1) epoch 194470) v5 -5957> 2014-08-13 17:22:48.947758 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.819972, event: all_read, request: pg_notify(1.2b(1) epoch 194470) v5 -5956> 2014-08-13 17:22:48.947764 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.947743, event: dispatched, request: pg_notify(1.2b(1) epoch 194470) v5 -5955> 2014-08-13 17:22:48.947770 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.947769, event: waiting_for_osdmap, request: pg_notify(1.2b(1) epoch 194470) v5 -5954> 2014-08-13 17:22:48.947777 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.947776, event: started, request: pg_notify(1.2b(1) epoch 194470) v5 -5953> 2014-08-13 17:22:48.947796 7f74d8735700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:22:48.947796, event: done, request: pg_notify(1.2b(1) epoch 194470) v5 -5952> 2014-08-13 17:22:48.947810 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 3 ==== pg_notify(22.1f1(1) epoch 194470) v5 ==== 777+0+0 (884427129 0 0) 0x466eac0 con 0x4906e00 -5951> 2014-08-13 17:22:48.947829 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.942543, event: header_read, request: pg_notify(22.1f1(1) epoch 194470) v5 -5950> 2014-08-13 17:22:48.947839 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.942546, event: throttled, request: pg_notify(22.1f1(1) epoch 194470) v5 -5949> 2014-08-13 17:22:48.947848 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.942619, event: all_read, request: pg_notify(22.1f1(1) epoch 194470) v5 -5948> 2014-08-13 17:22:48.947857 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.947827, event: dispatched, request: pg_notify(22.1f1(1) epoch 194470) v5 -5947> 2014-08-13 17:22:48.947867 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.947866, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194470) v5 -5946> 2014-08-13 17:22:48.947877 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.947877, event: started, request: pg_notify(22.1f1(1) epoch 194470) v5 -5945> 2014-08-13 17:22:48.947900 7f74d8735700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:22:48.947900, event: done, request: pg_notify(22.1f1(1) epoch 194470) v5 -5944> 2014-08-13 17:22:48.947885 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.237471 3 0.000191 -5943> 2014-08-13 17:22:48.947916 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.237628 0 0.000000 -5942> 2014-08-13 17:22:48.947918 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 4 ==== pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 ==== 1596+0+0 (1600192850 0 0) 0x49e6e40 con 0x4906b40 -5941> 2014-08-13 17:22:48.947941 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194447/194448/192943) [337,20,4] r=2 lpr=194449 pi=192976-194447/16 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -5940> 2014-08-13 17:22:48.947950 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.919354, event: header_read, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5939> 2014-08-13 17:22:48.947970 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.919357, event: throttled, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5938> 2014-08-13 17:22:48.947964 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000462 15 0.000459 -5937> 2014-08-13 17:22:48.947982 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.919447, event: all_read, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5936> 2014-08-13 17:22:48.948004 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.947948, event: dispatched, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5935> 2014-08-13 17:22:48.947999 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started -5934> 2014-08-13 17:22:48.948022 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.948021, event: waiting_for_osdmap, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5933> 2014-08-13 17:22:48.948031 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.948031, event: started, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5932> 2014-08-13 17:22:48.948030 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start -5931> 2014-08-13 17:22:48.948050 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5930> 2014-08-13 17:22:48.948069 7f74d8735700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:22:48.948069, event: done, request: pg_notify(1.2b(2),2.2a(2) epoch 194470) v5 -5929> 2014-08-13 17:22:48.948073 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000043 0 0.000000 -5928> 2014-08-13 17:22:48.948095 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 4 ==== pg_notify(22.f3(18) epoch 194470) v5 ==== 1720+0+0 (3451712932 0 0) 0x49e16c0 con 0x4d58840 -5927> 2014-08-13 17:22:48.948097 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5926> 2014-08-13 17:22:48.948121 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.822062, event: header_read, request: pg_notify(22.f3(18) epoch 194470) v5 -5925> 2014-08-13 17:22:48.948132 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.822063, event: throttled, request: pg_notify(22.f3(18) epoch 194470) v5 -5924> 2014-08-13 17:22:48.948127 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5923> 2014-08-13 17:22:48.948141 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.822258, event: all_read, request: pg_notify(22.f3(18) epoch 194470) v5 -5922> 2014-08-13 17:22:48.948148 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.948119, event: dispatched, request: pg_notify(22.f3(18) epoch 194470) v5 -5921> 2014-08-13 17:22:48.948154 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.948154, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194470) v5 -5920> 2014-08-13 17:22:48.948151 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194449/194449 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5919> 2014-08-13 17:22:48.948165 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.948165, event: started, request: pg_notify(22.f3(18) epoch 194470) v5 -5918> 2014-08-13 17:22:48.948198 7f74d8735700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:22:48.948197, event: done, request: pg_notify(22.f3(18) epoch 194470) v5 -5917> 2014-08-13 17:22:48.948223 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 4 ==== pg_notify(22.84(2) epoch 194470) v5 ==== 828+0+0 (950298263 0 0) 0x49e2680 con 0x4d58580 -5916> 2014-08-13 17:22:48.948242 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.924028, event: header_read, request: pg_notify(22.84(2) epoch 194470) v5 -5915> 2014-08-13 17:22:48.948249 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.924033, event: throttled, request: pg_notify(22.84(2) epoch 194470) v5 -5914> 2014-08-13 17:22:48.948255 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.924132, event: all_read, request: pg_notify(22.84(2) epoch 194470) v5 -5913> 2014-08-13 17:22:48.948263 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.948240, event: dispatched, request: pg_notify(22.84(2) epoch 194470) v5 -5912> 2014-08-13 17:22:48.948273 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.948273, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194470) v5 -5911> 2014-08-13 17:22:48.948284 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.948283, event: started, request: pg_notify(22.84(2) epoch 194470) v5 -5910> 2014-08-13 17:22:48.948307 7f74d8735700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:22:48.948307, event: done, request: pg_notify(22.84(2) epoch 194470) v5 -5909> 2014-08-13 17:22:48.948310 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.237977 3 0.000245 -5908> 2014-08-13 17:22:48.948326 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 4 ==== pg_notify(22.99(1) epoch 194470) v5 ==== 777+0+0 (309868313 0 0) 0x49e0fc0 con 0x4d582c0 -5907> 2014-08-13 17:22:48.948342 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.923905, event: header_read, request: pg_notify(22.99(1) epoch 194470) v5 -5906> 2014-08-13 17:22:48.948353 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.923981, event: throttled, request: pg_notify(22.99(1) epoch 194470) v5 -5905> 2014-08-13 17:22:48.948341 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.000400 15 0.000179 -5904> 2014-08-13 17:22:48.948367 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.924098, event: all_read, request: pg_notify(22.99(1) epoch 194470) v5 -5903> 2014-08-13 17:22:48.948373 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.948340, event: dispatched, request: pg_notify(22.99(1) epoch 194470) v5 -5902> 2014-08-13 17:22:48.948341 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.238087 0 0.000000 -5901> 2014-08-13 17:22:48.948385 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.948385, event: waiting_for_osdmap, request: pg_notify(22.99(1) epoch 194470) v5 -5900> 2014-08-13 17:22:48.948379 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -5899> 2014-08-13 17:22:48.948393 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.948393, event: started, request: pg_notify(22.99(1) epoch 194470) v5 -5898> 2014-08-13 17:22:48.948397 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194447/194448/192971) [51,4,308] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -5897> 2014-08-13 17:22:48.948401 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -5896> 2014-08-13 17:22:48.948414 7f74d8735700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:22:48.948414, event: done, request: pg_notify(22.99(1) epoch 194470) v5 -5895> 2014-08-13 17:22:48.948418 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5894> 2014-08-13 17:22:48.948432 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 4 ==== pg_notify(24.9b(1) epoch 194470) v5 ==== 811+0+0 (1423586113 0 0) 0x49e6ac0 con 0x4d51ce0 -5893> 2014-08-13 17:22:48.948440 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000040 0 0.000000 -5892> 2014-08-13 17:22:48.948452 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.929504, event: header_read, request: pg_notify(24.9b(1) epoch 194470) v5 -5891> 2014-08-13 17:22:48.948473 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.929507, event: throttled, request: pg_notify(24.9b(1) epoch 194470) v5 -5890> 2014-08-13 17:22:48.948467 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194449/194449 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -5889> 2014-08-13 17:22:48.948488 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.929578, event: all_read, request: pg_notify(24.9b(1) epoch 194470) v5 -5888> 2014-08-13 17:22:48.948509 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.948449, event: dispatched, request: pg_notify(24.9b(1) epoch 194470) v5 -5887> 2014-08-13 17:22:48.948518 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.948517, event: waiting_for_osdmap, request: pg_notify(24.9b(1) epoch 194470) v5 -5886> 2014-08-13 17:22:48.948527 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.948527, event: started, request: pg_notify(24.9b(1) epoch 194470) v5 -5885> 2014-08-13 17:22:48.948551 7f74d8735700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:22:48.948551, event: done, request: pg_notify(24.9b(1) epoch 194470) v5 -5884> 2014-08-13 17:22:48.948570 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.310 192.168.1.109:6812/32413 2 ==== pg_notify(22.707 epoch 194470) v5 ==== 694+0+0 (4154148339 0 0) 0x57d5cc0 con 0x4d53700 -5883> 2014-08-13 17:22:48.948589 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.942246, event: header_read, request: pg_notify(22.707 epoch 194470) v5 -5882> 2014-08-13 17:22:48.948599 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.942247, event: throttled, request: pg_notify(22.707 epoch 194470) v5 -5881> 2014-08-13 17:22:48.948596 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(19) epoch 194470) v5 -- ?+0 0x57d7700 con 0x4833440 -5880> 2014-08-13 17:22:48.948607 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.942344, event: all_read, request: pg_notify(22.707 epoch 194470) v5 -5879> 2014-08-13 17:22:48.948627 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.948587, event: dispatched, request: pg_notify(22.707 epoch 194470) v5 -5878> 2014-08-13 17:22:48.948628 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6842/9607 -- pg_query(24.136 epoch 194470) v3 -- ?+0 0x29c5460 con 0x47f70c0 -5877> 2014-08-13 17:22:48.948638 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.948638, event: waiting_for_osdmap, request: pg_notify(22.707 epoch 194470) v5 -5876> 2014-08-13 17:22:48.948649 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.948649, event: started, request: pg_notify(22.707 epoch 194470) v5 -5875> 2014-08-13 17:22:48.948647 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6834/6966 -- pg_query(24.136 epoch 194470) v3 -- ?+0 0x4eb21c0 con 0x47f7e80 -5874> 2014-08-13 17:22:48.948675 7f74d8735700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:22:48.948675, event: done, request: pg_notify(22.707 epoch 194470) v5 -5873> 2014-08-13 17:22:48.948674 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6840/16566 -- pg_query(24.136 epoch 194470) v3 -- ?+0 0x4eb3840 con 0x47a0b00 -5872> 2014-08-13 17:22:48.948702 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6842/17345 -- pg_query(24.136 epoch 194470) v3 -- ?+0 0x4eb3a20 con 0x4d51a20 -5871> 2014-08-13 17:22:48.948699 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 4 ==== pg_notify(22.362(22) epoch 194470) v5 ==== 1948+0+0 (1366328122 0 0) 0x5264980 con 0x4d55d80 -5870> 2014-08-13 17:22:48.948814 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.929147, event: header_read, request: pg_notify(22.362(22) epoch 194470) v5 -5869> 2014-08-13 17:22:48.948803 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.000406 15 0.000201 -5868> 2014-08-13 17:22:48.948828 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.929150, event: throttled, request: pg_notify(22.362(22) epoch 194470) v5 -5867> 2014-08-13 17:22:48.948824 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_query(24.136 epoch 194470) v3 -- ?+0 0x4eb3660 con 0x45f5960 -5866> 2014-08-13 17:22:48.948838 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.929260, event: all_read, request: pg_notify(22.362(22) epoch 194470) v5 -5865> 2014-08-13 17:22:48.948832 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -5864> 2014-08-13 17:22:48.948848 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.948810, event: dispatched, request: pg_notify(22.362(22) epoch 194470) v5 -5863> 2014-08-13 17:22:48.948851 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -5862> 2014-08-13 17:22:48.948859 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.948859, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194470) v5 -5861> 2014-08-13 17:22:48.948874 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5860> 2014-08-13 17:22:48.948886 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.948886, event: started, request: pg_notify(22.362(22) epoch 194470) v5 -5859> 2014-08-13 17:22:48.948896 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000045 0 0.000000 -5858> 2014-08-13 17:22:48.948912 7f74d8735700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:22:48.948912, event: done, request: pg_notify(22.362(22) epoch 194470) v5 -5857> 2014-08-13 17:22:48.948920 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194449/194449 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -5856> 2014-08-13 17:22:48.948933 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 3 ==== pg_notify(22.481(37) epoch 194470) v5 ==== 2801+0+0 (203028685 0 0) 0x57d1c00 con 0x4d55540 -5855> 2014-08-13 17:22:48.948926 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 0.237052 3 0.000166 -5854> 2014-08-13 17:22:48.948954 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.946928, event: header_read, request: pg_notify(22.481(37) epoch 194470) v5 -5853> 2014-08-13 17:22:48.948965 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.946931, event: throttled, request: pg_notify(22.481(37) epoch 194470) v5 -5852> 2014-08-13 17:22:48.948956 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started 0.237151 0 0.000000 -5851> 2014-08-13 17:22:48.948975 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.947063, event: all_read, request: pg_notify(22.481(37) epoch 194470) v5 -5850> 2014-08-13 17:22:48.948985 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.948952, event: dispatched, request: pg_notify(22.481(37) epoch 194470) v5 -5849> 2014-08-13 17:22:48.948979 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194447/194447/192967) [6,374,4] r=2 lpr=194449 pi=193125-194446/19 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset -5848> 2014-08-13 17:22:48.948994 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.948994, event: waiting_for_osdmap, request: pg_notify(22.481(37) epoch 194470) v5 -5847> 2014-08-13 17:22:48.949005 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.949005, event: started, request: pg_notify(22.481(37) epoch 194470) v5 -5846> 2014-08-13 17:22:48.949028 7f74d8735700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:22:48.949028, event: done, request: pg_notify(22.481(37) epoch 194470) v5 -5845> 2014-08-13 17:22:48.949055 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.318 192.168.1.110:6814/43857 2 ==== pg_notify(22.3e4 epoch 194470) v5 ==== 694+0+0 (3404731573 0 0) 0x57d7000 con 0x541c4c0 -5844> 2014-08-13 17:22:48.949074 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.937719, event: header_read, request: pg_notify(22.3e4 epoch 194470) v5 -5843> 2014-08-13 17:22:48.949084 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.937981, event: throttled, request: pg_notify(22.3e4 epoch 194470) v5 -5842> 2014-08-13 17:22:48.949093 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.938050, event: all_read, request: pg_notify(22.3e4 epoch 194470) v5 -5841> 2014-08-13 17:22:48.949083 7f74d0f29700 1 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -5840> 2014-08-13 17:22:48.949102 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.949072, event: dispatched, request: pg_notify(22.3e4 epoch 194470) v5 -5839> 2014-08-13 17:22:48.949122 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.949121, event: waiting_for_osdmap, request: pg_notify(22.3e4 epoch 194470) v5 -5838> 2014-08-13 17:22:48.949116 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.238169 2 0.000199 -5837> 2014-08-13 17:22:48.949140 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.949140, event: started, request: pg_notify(22.3e4 epoch 194470) v5 -5836> 2014-08-13 17:22:48.949196 7f74d8735700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:22:48.949196, event: done, request: pg_notify(22.3e4 epoch 194470) v5 -5835> 2014-08-13 17:22:48.949211 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.47 192.168.1.111:6828/12102 2 ==== pg_notify(22.1f1 epoch 194470) v5 ==== 694+0+0 (4054455041 0 0) 0x466f380 con 0x5490580 -5834> 2014-08-13 17:22:48.949227 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.943389, event: header_read, request: pg_notify(22.1f1 epoch 194470) v5 -5833> 2014-08-13 17:22:48.949233 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.943390, event: throttled, request: pg_notify(22.1f1 epoch 194470) v5 -5832> 2014-08-13 17:22:48.949238 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.943446, event: all_read, request: pg_notify(22.1f1 epoch 194470) v5 -5831> 2014-08-13 17:22:48.949243 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.949224, event: dispatched, request: pg_notify(22.1f1 epoch 194470) v5 -5830> 2014-08-13 17:22:48.949248 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.949248, event: waiting_for_osdmap, request: pg_notify(22.1f1 epoch 194470) v5 -5829> 2014-08-13 17:22:48.949255 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.949255, event: started, request: pg_notify(22.1f1 epoch 194470) v5 -5828> 2014-08-13 17:22:48.949270 7f74d8735700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:22:48.949270, event: done, request: pg_notify(22.1f1 epoch 194470) v5 -5827> 2014-08-13 17:22:48.949283 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.348 192.168.1.110:6800/41204 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de5580 con 0x54e1600 -5826> 2014-08-13 17:22:48.949301 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -5825> 2014-08-13 17:22:48.949318 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.59 192.168.1.112:6822/5265 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de5c40 con 0x54e4360 -5824> 2014-08-13 17:22:48.949332 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -5823> 2014-08-13 17:22:48.949149 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.238213 0 0.000000 -5822> 2014-08-13 17:22:48.949351 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 5 ==== pg_notify(22.362 epoch 194470) v5 ==== 694+0+0 (1266644936 0 0) 0x57d1500 con 0x45f1e40 -5821> 2014-08-13 17:22:48.949370 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.928907, event: header_read, request: pg_notify(22.362 epoch 194470) v5 -5820> 2014-08-13 17:22:48.949360 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.238435 0 0.000000 -5819> 2014-08-13 17:22:48.949383 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.928909, event: throttled, request: pg_notify(22.362 epoch 194470) v5 -5818> 2014-08-13 17:22:48.949391 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.928987, event: all_read, request: pg_notify(22.362 epoch 194470) v5 -5817> 2014-08-13 17:22:48.949400 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.949368, event: dispatched, request: pg_notify(22.362 epoch 194470) v5 -5816> 2014-08-13 17:22:48.949393 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started 0.238564 0 0.000000 -5815> 2014-08-13 17:22:48.949410 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.949410, event: waiting_for_osdmap, request: pg_notify(22.362 epoch 194470) v5 -5814> 2014-08-13 17:22:48.949421 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.949421, event: started, request: pg_notify(22.362 epoch 194470) v5 -5813> 2014-08-13 17:22:48.949414 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194447/194448/194448) [4,340,347] r=0 lpr=194449 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset -5812> 2014-08-13 17:22:48.949448 7f74d8735700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:22:48.949447, event: done, request: pg_notify(22.362 epoch 194470) v5 -5811> 2014-08-13 17:22:48.949466 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 4 ==== pg_notify(24.b6 epoch 194470) v5 ==== 694+0+0 (45726271 0 0) 0x5263b80 con 0x45f3b20 -5810> 2014-08-13 17:22:48.949461 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Reset 0.000482 15 0.000201 -5809> 2014-08-13 17:22:48.949484 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.947472, event: header_read, request: pg_notify(24.b6 epoch 194470) v5 -5808> 2014-08-13 17:22:48.949504 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.947473, event: throttled, request: pg_notify(24.b6 epoch 194470) v5 -5807> 2014-08-13 17:22:48.949499 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started -5806> 2014-08-13 17:22:48.949514 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.947545, event: all_read, request: pg_notify(24.b6 epoch 194470) v5 -5805> 2014-08-13 17:22:48.949523 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.949482, event: dispatched, request: pg_notify(24.b6 epoch 194470) v5 -5804> 2014-08-13 17:22:48.949517 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Start -5803> 2014-08-13 17:22:48.949533 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.949533, event: waiting_for_osdmap, request: pg_notify(24.b6 epoch 194470) v5 -5802> 2014-08-13 17:22:48.949533 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: transitioning to Stray -5801> 2014-08-13 17:22:48.949544 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.949543, event: started, request: pg_notify(24.b6 epoch 194470) v5 -5800> 2014-08-13 17:22:48.949560 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Start 0.000042 0 0.000000 -5799> 2014-08-13 17:22:48.949580 7f74d8735700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:22:48.949580, event: done, request: pg_notify(24.b6 epoch 194470) v5 -5798> 2014-08-13 17:22:48.949584 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194448/194442 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started/Stray -5797> 2014-08-13 17:22:48.949595 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 5 ==== pg_notify(22.4df epoch 194470) v5 ==== 694+0+0 (1928747819 0 0) 0x52639c0 con 0x45f5960 -5796> 2014-08-13 17:22:48.949610 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.923757, event: header_read, request: pg_notify(22.4df epoch 194470) v5 -5795> 2014-08-13 17:22:48.949616 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.923759, event: throttled, request: pg_notify(22.4df epoch 194470) v5 -5794> 2014-08-13 17:22:48.949621 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.923831, event: all_read, request: pg_notify(22.4df epoch 194470) v5 -5793> 2014-08-13 17:22:48.949626 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.949608, event: dispatched, request: pg_notify(22.4df epoch 194470) v5 -5792> 2014-08-13 17:22:48.949631 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.949631, event: waiting_for_osdmap, request: pg_notify(22.4df epoch 194470) v5 -5791> 2014-08-13 17:22:48.949638 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.949638, event: started, request: pg_notify(22.4df epoch 194470) v5 -5790> 2014-08-13 17:22:48.949653 7f74d8735700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:22:48.949653, event: done, request: pg_notify(22.4df epoch 194470) v5 -5789> 2014-08-13 17:22:48.949666 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 5 ==== pg_notify(24.3d(1) epoch 194470) v5 ==== 811+0+0 (1680853822 0 0) 0x57d6200 con 0x4758840 -5788> 2014-08-13 17:22:48.949680 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.928623, event: header_read, request: pg_notify(24.3d(1) epoch 194470) v5 -5787> 2014-08-13 17:22:48.949686 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.928626, event: throttled, request: pg_notify(24.3d(1) epoch 194470) v5 -5786> 2014-08-13 17:22:48.949691 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.928700, event: all_read, request: pg_notify(24.3d(1) epoch 194470) v5 -5785> 2014-08-13 17:22:48.949696 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.949678, event: dispatched, request: pg_notify(24.3d(1) epoch 194470) v5 -5784> 2014-08-13 17:22:48.949705 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.949705, event: waiting_for_osdmap, request: pg_notify(24.3d(1) epoch 194470) v5 -5783> 2014-08-13 17:22:48.949712 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.949712, event: started, request: pg_notify(24.3d(1) epoch 194470) v5 -5782> 2014-08-13 17:22:48.949730 7f74d8735700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:22:48.949730, event: done, request: pg_notify(24.3d(1) epoch 194470) v5 -5781> 2014-08-13 17:22:48.949744 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.237465 3 0.000199 -5780> 2014-08-13 17:22:48.949754 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 5 ==== pg_notify(22.481(32) epoch 194470) v5 ==== 2518+0+0 (2237357479 0 0) 0x4ac71c0 con 0x475ee00 -5779> 2014-08-13 17:22:48.949790 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.237610 0 0.000000 -5778> 2014-08-13 17:22:48.949799 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.813997, event: header_read, request: pg_notify(22.481(32) epoch 194470) v5 -5777> 2014-08-13 17:22:48.949822 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.813998, event: throttled, request: pg_notify(22.481(32) epoch 194470) v5 -5776> 2014-08-13 17:22:48.949817 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194447/194448/192974) [376,4,303] r=1 lpr=194449 pi=192975-194447/17 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -5775> 2014-08-13 17:22:48.949833 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.814106, event: all_read, request: pg_notify(22.481(32) epoch 194470) v5 -5774> 2014-08-13 17:22:48.949842 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.949796, event: dispatched, request: pg_notify(22.481(32) epoch 194470) v5 -5773> 2014-08-13 17:22:48.949852 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.949852, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194470) v5 -5772> 2014-08-13 17:22:48.949864 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.949863, event: started, request: pg_notify(22.481(32) epoch 194470) v5 -5771> 2014-08-13 17:22:48.949887 7f74d8735700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:22:48.949887, event: done, request: pg_notify(22.481(32) epoch 194470) v5 -5770> 2014-08-13 17:22:48.949891 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000478 15 0.000480 -5769> 2014-08-13 17:22:48.949921 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started -5768> 2014-08-13 17:22:48.949941 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start -5767> 2014-08-13 17:22:48.949943 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 5 ==== pg_notify(22.6bd(20) epoch 194470) v5 ==== 1838+0+0 (3391304317 0 0) 0x4ac3100 con 0x47a27e0 -5766> 2014-08-13 17:22:48.949962 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.936947, event: header_read, request: pg_notify(22.6bd(20) epoch 194470) v5 -5765> 2014-08-13 17:22:48.949959 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5764> 2014-08-13 17:22:48.949974 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.936949, event: throttled, request: pg_notify(22.6bd(20) epoch 194470) v5 -5763> 2014-08-13 17:22:48.949983 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.937040, event: all_read, request: pg_notify(22.6bd(20) epoch 194470) v5 -5762> 2014-08-13 17:22:48.949992 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.949959, event: dispatched, request: pg_notify(22.6bd(20) epoch 194470) v5 -5761> 2014-08-13 17:22:48.949982 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000040 0 0.000000 -5760> 2014-08-13 17:22:48.950005 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.950005, event: waiting_for_osdmap, request: pg_notify(22.6bd(20) epoch 194470) v5 -5759> 2014-08-13 17:22:48.950025 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.950025, event: started, request: pg_notify(22.6bd(20) epoch 194470) v5 -5758> 2014-08-13 17:22:48.950018 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5757> 2014-08-13 17:22:48.950048 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5756> 2014-08-13 17:22:48.950061 7f74d8735700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:22:48.950060, event: done, request: pg_notify(22.6bd(20) epoch 194470) v5 -5755> 2014-08-13 17:22:48.950068 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194449/194449 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5754> 2014-08-13 17:22:48.950090 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 5 ==== pg_notify(22.362(1) epoch 194470) v5 ==== 777+0+0 (1550353488 0 0) 0x57d6ac0 con 0x47a0b00 -5753> 2014-08-13 17:22:48.950111 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.928779, event: header_read, request: pg_notify(22.362(1) epoch 194470) v5 -5752> 2014-08-13 17:22:48.950121 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.928781, event: throttled, request: pg_notify(22.362(1) epoch 194470) v5 -5751> 2014-08-13 17:22:48.950131 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.928850, event: all_read, request: pg_notify(22.362(1) epoch 194470) v5 -5750> 2014-08-13 17:22:48.950139 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.950108, event: dispatched, request: pg_notify(22.362(1) epoch 194470) v5 -5749> 2014-08-13 17:22:48.950149 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.950149, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194470) v5 -5748> 2014-08-13 17:22:48.950157 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.950157, event: started, request: pg_notify(22.362(1) epoch 194470) v5 -5747> 2014-08-13 17:22:48.950185 7f74d8735700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:22:48.950185, event: done, request: pg_notify(22.362(1) epoch 194470) v5 -5746> 2014-08-13 17:22:48.950209 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.313 192.168.1.109:6820/35445 2 ==== pg_notify(22.3e4 epoch 194470) v5 ==== 694+0+0 (3404731573 0 0) 0x4f516c0 con 0x47a1b80 -5745> 2014-08-13 17:22:48.950228 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.936715, event: header_read, request: pg_notify(22.3e4 epoch 194470) v5 -5744> 2014-08-13 17:22:48.950239 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.936717, event: throttled, request: pg_notify(22.3e4 epoch 194470) v5 -5743> 2014-08-13 17:22:48.950244 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.936791, event: all_read, request: pg_notify(22.3e4 epoch 194470) v5 -5742> 2014-08-13 17:22:48.950250 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.950226, event: dispatched, request: pg_notify(22.3e4 epoch 194470) v5 -5741> 2014-08-13 17:22:48.950233 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.238902 3 0.000177 -5740> 2014-08-13 17:22:48.950259 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.950259, event: waiting_for_osdmap, request: pg_notify(22.3e4 epoch 194470) v5 -5739> 2014-08-13 17:22:48.950267 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.950267, event: started, request: pg_notify(22.3e4 epoch 194470) v5 -5738> 2014-08-13 17:22:48.950262 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.238995 0 0.000000 -5737> 2014-08-13 17:22:48.950266 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.000449 15 0.000227 -5736> 2014-08-13 17:22:48.950290 7f74d8735700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:22:48.950289, event: done, request: pg_notify(22.3e4 epoch 194470) v5 -5735> 2014-08-13 17:22:48.950283 7f74d0f29700 5 osd.4 pg_epoch: 194456 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194447/194448/192981) [345,24,4] r=2 lpr=194449 pi=192984-194447/16 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -5734> 2014-08-13 17:22:48.950297 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -5733> 2014-08-13 17:22:48.950303 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 4 ==== pg_notify(24.b6(27) epoch 194470) v5 ==== 2273+0+0 (585923396 0 0) 0x4ac2300 con 0x47a53e0 -5732> 2014-08-13 17:22:48.950318 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.947682, event: header_read, request: pg_notify(24.b6(27) epoch 194470) v5 -5731> 2014-08-13 17:22:48.950317 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -5730> 2014-08-13 17:22:48.950329 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.947684, event: throttled, request: pg_notify(24.b6(27) epoch 194470) v5 -5729> 2014-08-13 17:22:48.950349 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.947790, event: all_read, request: pg_notify(24.b6(27) epoch 194470) v5 -5728> 2014-08-13 17:22:48.950341 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5727> 2014-08-13 17:22:48.950362 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.950316, event: dispatched, request: pg_notify(24.b6(27) epoch 194470) v5 -5726> 2014-08-13 17:22:48.950380 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.950380, event: waiting_for_osdmap, request: pg_notify(24.b6(27) epoch 194470) v5 -5725> 2014-08-13 17:22:48.950376 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000060 0 0.000000 -5724> 2014-08-13 17:22:48.950392 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.950391, event: started, request: pg_notify(24.b6(27) epoch 194470) v5 -5723> 2014-08-13 17:22:48.950405 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194449/194389 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -5722> 2014-08-13 17:22:48.950430 7f74d8735700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:22:48.950430, event: done, request: pg_notify(24.b6(27) epoch 194470) v5 -5721> 2014-08-13 17:22:48.950455 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 4 ==== pg_notify(22.3af epoch 194470) v5 ==== 694+0+0 (4225460538 0 0) 0x57d5b00 con 0x47a4ba0 -5720> 2014-08-13 17:22:48.950474 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.946333, event: header_read, request: pg_notify(22.3af epoch 194470) v5 -5719> 2014-08-13 17:22:48.950484 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.946336, event: throttled, request: pg_notify(22.3af epoch 194470) v5 -5718> 2014-08-13 17:22:48.950493 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.946418, event: all_read, request: pg_notify(22.3af epoch 194470) v5 -5717> 2014-08-13 17:22:48.950502 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.950472, event: dispatched, request: pg_notify(22.3af epoch 194470) v5 -5716> 2014-08-13 17:22:48.950511 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.950511, event: waiting_for_osdmap, request: pg_notify(22.3af epoch 194470) v5 -5715> 2014-08-13 17:22:48.950522 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.950521, event: started, request: pg_notify(22.3af epoch 194470) v5 -5714> 2014-08-13 17:22:48.950542 7f74d8735700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:22:48.950542, event: done, request: pg_notify(22.3af epoch 194470) v5 -5713> 2014-08-13 17:22:48.950559 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 5 ==== pg_notify(22.3e4(22) epoch 194470) v5 ==== 1948+0+0 (2374870074 0 0) 0x57d2d80 con 0x47a32e0 -5712> 2014-08-13 17:22:48.950577 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.936685, event: header_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -5711> 2014-08-13 17:22:48.950569 7f74d192a700 1 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -5710> 2014-08-13 17:22:48.950588 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.936688, event: throttled, request: pg_notify(22.3e4(22) epoch 194470) v5 -5709> 2014-08-13 17:22:48.950594 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.936816, event: all_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -5708> 2014-08-13 17:22:48.950602 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.950575, event: dispatched, request: pg_notify(22.3e4(22) epoch 194470) v5 -5707> 2014-08-13 17:22:48.950596 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.237942 2 0.000165 -5706> 2014-08-13 17:22:48.950613 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.950612, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194470) v5 -5705> 2014-08-13 17:22:48.950623 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.950623, event: started, request: pg_notify(22.3e4(22) epoch 194470) v5 -5704> 2014-08-13 17:22:48.950620 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.237986 0 0.000000 -5703> 2014-08-13 17:22:48.950646 7f74d8735700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:22:48.950645, event: done, request: pg_notify(22.3e4(22) epoch 194470) v5 -5702> 2014-08-13 17:22:48.950645 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.238045 0 0.000000 -5701> 2014-08-13 17:22:48.950665 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.21 192.168.1.107:6821/54786 2 ==== pg_notify(22.707 epoch 194470) v5 ==== 694+0+0 (4154148339 0 0) 0x57d2840 con 0x47a6ca0 -5700> 2014-08-13 17:22:48.950669 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started 0.238115 0 0.000000 -5699> 2014-08-13 17:22:48.950683 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.942221, event: header_read, request: pg_notify(22.707 epoch 194470) v5 -5698> 2014-08-13 17:22:48.950690 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.942223, event: throttled, request: pg_notify(22.707 epoch 194470) v5 -5697> 2014-08-13 17:22:48.950691 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194447/194448/194448) [4,338,376] r=0 lpr=194449 pi=193306-194447/38 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset -5696> 2014-08-13 17:22:48.950698 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.942288, event: all_read, request: pg_notify(22.707 epoch 194470) v5 -5695> 2014-08-13 17:22:48.950711 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.950681, event: dispatched, request: pg_notify(22.707 epoch 194470) v5 -5694> 2014-08-13 17:22:48.950698 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.000416 15 0.000187 -5693> 2014-08-13 17:22:48.950721 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.950721, event: waiting_for_osdmap, request: pg_notify(22.707 epoch 194470) v5 -5692> 2014-08-13 17:22:48.950724 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -5691> 2014-08-13 17:22:48.950732 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.950732, event: started, request: pg_notify(22.707 epoch 194470) v5 -5690> 2014-08-13 17:22:48.950748 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -5689> 2014-08-13 17:22:48.950771 7f74d8735700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:22:48.950771, event: done, request: pg_notify(22.707 epoch 194470) v5 -5688> 2014-08-13 17:22:48.950766 7f74d0f29700 1 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5687> 2014-08-13 17:22:48.950784 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000037 0 0.000000 -5686> 2014-08-13 17:22:48.950790 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.5 192.168.1.103:6827/40192 2 ==== pg_notify(22.1f1 epoch 194470) v5 ==== 694+0+0 (4054455041 0 0) 0x57d7a80 con 0x47f2ec0 -5685> 2014-08-13 17:22:48.950808 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.942386, event: header_read, request: pg_notify(22.1f1 epoch 194470) v5 -5684> 2014-08-13 17:22:48.950807 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194449/194449 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -5683> 2014-08-13 17:22:48.950818 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.942388, event: throttled, request: pg_notify(22.1f1 epoch 194470) v5 -5682> 2014-08-13 17:22:48.950837 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.942446, event: all_read, request: pg_notify(22.1f1 epoch 194470) v5 -5681> 2014-08-13 17:22:48.950847 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.950805, event: dispatched, request: pg_notify(22.1f1 epoch 194470) v5 -5680> 2014-08-13 17:22:48.950856 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.950856, event: waiting_for_osdmap, request: pg_notify(22.1f1 epoch 194470) v5 -5679> 2014-08-13 17:22:48.950867 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.950867, event: started, request: pg_notify(22.1f1 epoch 194470) v5 -5678> 2014-08-13 17:22:48.950888 7f74d8735700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:22:48.950888, event: done, request: pg_notify(22.1f1 epoch 194470) v5 -5677> 2014-08-13 17:22:48.950905 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 3 ==== pg_notify(24.b6(27) epoch 194470) v5 ==== 2273+0+0 (3499909850 0 0) 0x4ac0e00 con 0x47f6880 -5676> 2014-08-13 17:22:48.950924 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.947328, event: header_read, request: pg_notify(24.b6(27) epoch 194470) v5 -5675> 2014-08-13 17:22:48.950922 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(20) epoch 194470) v5 -- ?+0 0x57d1f80 con 0x4831b80 -5674> 2014-08-13 17:22:48.950934 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.947335, event: throttled, request: pg_notify(24.b6(27) epoch 194470) v5 -5673> 2014-08-13 17:22:48.950953 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.947445, event: all_read, request: pg_notify(24.b6(27) epoch 194470) v5 -5672> 2014-08-13 17:22:48.950960 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.950922, event: dispatched, request: pg_notify(24.b6(27) epoch 194470) v5 -5671> 2014-08-13 17:22:48.950958 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(19) epoch 194470) v5 -- ?+0 0x4ac1dc0 con 0x475c4c0 -5670> 2014-08-13 17:22:48.950967 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.950966, event: waiting_for_osdmap, request: pg_notify(24.b6(27) epoch 194470) v5 -5669> 2014-08-13 17:22:48.950974 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.950974, event: started, request: pg_notify(24.b6(27) epoch 194470) v5 -5668> 2014-08-13 17:22:48.950979 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6838/62713 -- pg_query(22.1b8 epoch 194470) v3 -- ?+0 0x4eb4560 con 0x4d58580 -5667> 2014-08-13 17:22:48.950993 7f74d8735700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:22:48.950993, event: done, request: pg_notify(24.b6(27) epoch 194470) v5 -5666> 2014-08-13 17:22:48.950997 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6815/60573 -- pg_query(22.3d5 epoch 194470) v3 -- ?+0 0x4eb4380 con 0x45f1760 -5665> 2014-08-13 17:22:48.951013 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6820/19173 -- pg_query(22.3d5 epoch 194470) v3 -- ?+0 0x4eb41a0 con 0x48dfe80 -5664> 2014-08-13 17:22:48.951017 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 5 ==== pg_notify(24.1f7(1) epoch 194470) v5 ==== 811+0+0 (763539561 0 0) 0x4ac6c80 con 0x47f7e80 -5663> 2014-08-13 17:22:48.951030 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6806/62463 -- pg_query(22.1b8 epoch 194470) v3 -- ?+0 0x4eb3fc0 con 0x47a3f40 -5662> 2014-08-13 17:22:48.951033 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.947354, event: header_read, request: pg_notify(24.1f7(1) epoch 194470) v5 -5661> 2014-08-13 17:22:48.951042 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.947360, event: throttled, request: pg_notify(24.1f7(1) epoch 194470) v5 -5660> 2014-08-13 17:22:48.951047 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.947431, event: all_read, request: pg_notify(24.1f7(1) epoch 194470) v5 -5659> 2014-08-13 17:22:48.951047 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_query(22.3d5 epoch 194470) v3 -- ?+0 0x4eb3de0 con 0x48377a0 -5658> 2014-08-13 17:22:48.951053 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.951031, event: dispatched, request: pg_notify(24.1f7(1) epoch 194470) v5 -5657> 2014-08-13 17:22:48.951059 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.951059, event: waiting_for_osdmap, request: pg_notify(24.1f7(1) epoch 194470) v5 -5656> 2014-08-13 17:22:48.951067 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.951067, event: started, request: pg_notify(24.1f7(1) epoch 194470) v5 -5655> 2014-08-13 17:22:48.951063 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6800/4640 -- pg_query(22.1b8 epoch 194470) v3 -- ?+0 0x29c4740 con 0x4831600 -5654> 2014-08-13 17:22:48.951058 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000367 15 0.000246 -5653> 2014-08-13 17:22:48.951089 7f74d8735700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:22:48.951089, event: done, request: pg_notify(24.1f7(1) epoch 194470) v5 -5652> 2014-08-13 17:22:48.951083 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started -5651> 2014-08-13 17:22:48.951115 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 5 ==== pg_notify(24.136(1) epoch 194470) v5 ==== 811+0+0 (1105244236 0 0) 0x49e7000 con 0x47f70c0 -5650> 2014-08-13 17:22:48.951117 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start -5649> 2014-08-13 17:22:48.951130 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.949258, event: header_read, request: pg_notify(24.136(1) epoch 194470) v5 -5648> 2014-08-13 17:22:48.951137 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.949261, event: throttled, request: pg_notify(24.136(1) epoch 194470) v5 -5647> 2014-08-13 17:22:48.951143 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.949335, event: all_read, request: pg_notify(24.136(1) epoch 194470) v5 -5646> 2014-08-13 17:22:48.951133 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5645> 2014-08-13 17:22:48.951148 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.951128, event: dispatched, request: pg_notify(24.136(1) epoch 194470) v5 -5644> 2014-08-13 17:22:48.951155 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.951155, event: waiting_for_osdmap, request: pg_notify(24.136(1) epoch 194470) v5 -5643> 2014-08-13 17:22:48.951162 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.951162, event: started, request: pg_notify(24.136(1) epoch 194470) v5 -5642> 2014-08-13 17:22:48.951157 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 -5641> 2014-08-13 17:22:48.951191 7f74d8735700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:22:48.951191, event: done, request: pg_notify(24.136(1) epoch 194470) v5 -5640> 2014-08-13 17:22:48.951176 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5639> 2014-08-13 17:22:48.951200 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5638> 2014-08-13 17:22:48.951208 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 5 ==== pg_notify(22.99(18) epoch 194470) v5 ==== 1720+0+0 (3779630166 0 0) 0x4ac3480 con 0x4833440 -5637> 2014-08-13 17:22:48.951224 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.924192, event: header_read, request: pg_notify(22.99(18) epoch 194470) v5 -5636> 2014-08-13 17:22:48.951216 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194443/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5635> 2014-08-13 17:22:48.951230 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.924194, event: throttled, request: pg_notify(22.99(18) epoch 194470) v5 -5634> 2014-08-13 17:22:48.951236 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.924298, event: all_read, request: pg_notify(22.99(18) epoch 194470) v5 -5633> 2014-08-13 17:22:48.951241 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.951222, event: dispatched, request: pg_notify(22.99(18) epoch 194470) v5 -5632> 2014-08-13 17:22:48.951247 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.951247, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194470) v5 -5631> 2014-08-13 17:22:48.951254 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.951254, event: started, request: pg_notify(22.99(18) epoch 194470) v5 -5630> 2014-08-13 17:22:48.951275 7f74d8735700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:22:48.951274, event: done, request: pg_notify(22.99(18) epoch 194470) v5 -5629> 2014-08-13 17:22:48.951293 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 5 ==== pg_notify(24.1f7(23) epoch 194470) v5 ==== 2037+0+0 (2318168837 0 0) 0x4ac2d80 con 0x4835280 -5628> 2014-08-13 17:22:48.951308 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.807103, event: header_read, request: pg_notify(24.1f7(23) epoch 194470) v5 -5627> 2014-08-13 17:22:48.951314 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.807103, event: throttled, request: pg_notify(24.1f7(23) epoch 194470) v5 -5626> 2014-08-13 17:22:48.951319 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.807261, event: all_read, request: pg_notify(24.1f7(23) epoch 194470) v5 -5625> 2014-08-13 17:22:48.951324 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.951306, event: dispatched, request: pg_notify(24.1f7(23) epoch 194470) v5 -5624> 2014-08-13 17:22:48.951330 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.951330, event: waiting_for_osdmap, request: pg_notify(24.1f7(23) epoch 194470) v5 -5623> 2014-08-13 17:22:48.951337 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.951337, event: started, request: pg_notify(24.1f7(23) epoch 194470) v5 -5622> 2014-08-13 17:22:48.951354 7f74d8735700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:22:48.951354, event: done, request: pg_notify(24.1f7(23) epoch 194470) v5 -5621> 2014-08-13 17:22:48.951379 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 4 ==== pg_notify(24.1f7(27) epoch 194470) v5 ==== 2273+0+0 (2131688865 0 0) 0x466b2c0 con 0x4868420 -5620> 2014-08-13 17:22:48.951399 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.947415, event: header_read, request: pg_notify(24.1f7(27) epoch 194470) v5 -5619> 2014-08-13 17:22:48.951406 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.947417, event: throttled, request: pg_notify(24.1f7(27) epoch 194470) v5 -5618> 2014-08-13 17:22:48.951412 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.947520, event: all_read, request: pg_notify(24.1f7(27) epoch 194470) v5 -5617> 2014-08-13 17:22:48.951417 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.951397, event: dispatched, request: pg_notify(24.1f7(27) epoch 194470) v5 -5616> 2014-08-13 17:22:48.951397 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started/Stray 0.238398 3 0.000188 -5615> 2014-08-13 17:22:48.951423 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.951423, event: waiting_for_osdmap, request: pg_notify(24.1f7(27) epoch 194470) v5 -5614> 2014-08-13 17:22:48.951431 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.951430, event: started, request: pg_notify(24.1f7(27) epoch 194470) v5 -5613> 2014-08-13 17:22:48.951426 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started 0.238509 0 0.000000 -5612> 2014-08-13 17:22:48.951451 7f74d8735700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:22:48.951450, event: done, request: pg_notify(24.1f7(27) epoch 194470) v5 -5611> 2014-08-13 17:22:48.951448 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194447/194448/193232) [374,4,314]/[374,4,12] r=1 lpr=194449 pi=192688-194447/34 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset -5610> 2014-08-13 17:22:48.951475 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 4 ==== pg_notify(24.1bb epoch 194470) v5 ==== 694+0+0 (982899203 0 0) 0x5263d40 con 0x486bf40 -5609> 2014-08-13 17:22:48.951489 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.946718, event: header_read, request: pg_notify(24.1bb epoch 194470) v5 -5608> 2014-08-13 17:22:48.951495 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.946720, event: throttled, request: pg_notify(24.1bb epoch 194470) v5 -5607> 2014-08-13 17:22:48.951500 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.946817, event: all_read, request: pg_notify(24.1bb epoch 194470) v5 -5606> 2014-08-13 17:22:48.951507 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.951487, event: dispatched, request: pg_notify(24.1bb epoch 194470) v5 -5605> 2014-08-13 17:22:48.951513 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.951513, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194470) v5 -5604> 2014-08-13 17:22:48.951520 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.951520, event: started, request: pg_notify(24.1bb epoch 194470) v5 -5603> 2014-08-13 17:22:48.951536 7f74d8735700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:22:48.951536, event: done, request: pg_notify(24.1bb epoch 194470) v5 -5602> 2014-08-13 17:22:48.951548 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 5 ==== pg_query(22.581 epoch 194470) v3 ==== 144+0+0 (185824306 0 0) 0x29c41a0 con 0x486bc80 -5601> 2014-08-13 17:22:48.951563 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.809495, event: header_read, request: pg_query(22.581 epoch 194470) v3 -5600> 2014-08-13 17:22:48.951569 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.809496, event: throttled, request: pg_query(22.581 epoch 194470) v3 -5599> 2014-08-13 17:22:48.951577 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.809570, event: all_read, request: pg_query(22.581 epoch 194470) v3 -5598> 2014-08-13 17:22:48.951587 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.951561, event: dispatched, request: pg_query(22.581 epoch 194470) v3 -5597> 2014-08-13 17:22:48.951598 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.951598, event: waiting_for_osdmap, request: pg_query(22.581 epoch 194470) v3 -5596> 2014-08-13 17:22:48.951609 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.951609, event: started, request: pg_query(22.581 epoch 194470) v3 -5595> 2014-08-13 17:22:48.951630 7f74d8735700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:22:48.951630, event: done, request: pg_query(22.581 epoch 194470) v3 -5594> 2014-08-13 17:22:48.951649 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 4 ==== pg_notify(22.48e(20) epoch 194470) v5 ==== 1834+0+0 (1696045208 0 0) 0x4ac1500 con 0x486b700 -5593> 2014-08-13 17:22:48.951664 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.946793, event: header_read, request: pg_notify(22.48e(20) epoch 194470) v5 -5592> 2014-08-13 17:22:48.951670 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.946795, event: throttled, request: pg_notify(22.48e(20) epoch 194470) v5 -5591> 2014-08-13 17:22:48.951676 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.946899, event: all_read, request: pg_notify(22.48e(20) epoch 194470) v5 -5590> 2014-08-13 17:22:48.951688 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.951662, event: dispatched, request: pg_notify(22.48e(20) epoch 194470) v5 -5589> 2014-08-13 17:22:48.951694 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.951694, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194470) v5 -5588> 2014-08-13 17:22:48.951701 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.951701, event: started, request: pg_notify(22.48e(20) epoch 194470) v5 -5587> 2014-08-13 17:22:48.951720 7f74d8735700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:22:48.951720, event: done, request: pg_notify(22.48e(20) epoch 194470) v5 -5586> 2014-08-13 17:22:48.951746 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 3 ==== pg_notify(24.1bb(25) epoch 194470) v5 ==== 2155+0+0 (2005506023 0 0) 0x4ac01c0 con 0x486b440 -5585> 2014-08-13 17:22:48.951766 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.948113, event: header_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -5584> 2014-08-13 17:22:48.951773 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.948115, event: throttled, request: pg_notify(24.1bb(25) epoch 194470) v5 -5583> 2014-08-13 17:22:48.951778 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.948261, event: all_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -5582> 2014-08-13 17:22:48.951783 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.951763, event: dispatched, request: pg_notify(24.1bb(25) epoch 194470) v5 -5581> 2014-08-13 17:22:48.951788 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.951788, event: waiting_for_osdmap, request: pg_notify(24.1bb(25) epoch 194470) v5 -5580> 2014-08-13 17:22:48.951795 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.951795, event: started, request: pg_notify(24.1bb(25) epoch 194470) v5 -5579> 2014-08-13 17:22:48.951814 7f74d8735700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:22:48.951814, event: done, request: pg_notify(24.1bb(25) epoch 194470) v5 -5578> 2014-08-13 17:22:48.951835 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 5 ==== pg_notify(22.39d epoch 194470) v5 ==== 694+0+0 (2816481024 0 0) 0x4ac24c0 con 0x486a3c0 -5577> 2014-08-13 17:22:48.951854 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.925930, event: header_read, request: pg_notify(22.39d epoch 194470) v5 -5576> 2014-08-13 17:22:48.951865 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.925932, event: throttled, request: pg_notify(22.39d epoch 194470) v5 -5575> 2014-08-13 17:22:48.951870 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.926047, event: all_read, request: pg_notify(22.39d epoch 194470) v5 -5574> 2014-08-13 17:22:48.951875 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.951852, event: dispatched, request: pg_notify(22.39d epoch 194470) v5 -5573> 2014-08-13 17:22:48.951881 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.951881, event: waiting_for_osdmap, request: pg_notify(22.39d epoch 194470) v5 -5572> 2014-08-13 17:22:48.951887 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.951887, event: started, request: pg_notify(22.39d epoch 194470) v5 -5571> 2014-08-13 17:22:48.951903 7f74d8735700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:22:48.951903, event: done, request: pg_notify(22.39d epoch 194470) v5 -5570> 2014-08-13 17:22:48.951916 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 5 ==== pg_notify(24.b6 epoch 194470) v5 ==== 694+0+0 (45726271 0 0) 0x57d4980 con 0x486e040 -5569> 2014-08-13 17:22:48.951933 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.947115, event: header_read, request: pg_notify(24.b6 epoch 194470) v5 -5568> 2014-08-13 17:22:48.951943 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.947117, event: throttled, request: pg_notify(24.b6 epoch 194470) v5 -5567> 2014-08-13 17:22:48.951952 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.947190, event: all_read, request: pg_notify(24.b6 epoch 194470) v5 -5566> 2014-08-13 17:22:48.951961 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.951932, event: dispatched, request: pg_notify(24.b6 epoch 194470) v5 -5565> 2014-08-13 17:22:48.951951 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Reset 0.000503 15 0.000210 -5564> 2014-08-13 17:22:48.951970 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.951970, event: waiting_for_osdmap, request: pg_notify(24.b6 epoch 194470) v5 -5563> 2014-08-13 17:22:48.951994 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.951994, event: started, request: pg_notify(24.b6 epoch 194470) v5 -5562> 2014-08-13 17:22:48.951987 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started -5561> 2014-08-13 17:22:48.952020 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Start -5560> 2014-08-13 17:22:48.952042 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: transitioning to Stray -5559> 2014-08-13 17:22:48.952064 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Start 0.000044 0 0.000000 -5558> 2014-08-13 17:22:48.952090 7f74d8735700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:22:48.952089, event: done, request: pg_notify(24.b6 epoch 194470) v5 -5557> 2014-08-13 17:22:48.952087 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194449/194389 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started/Stray -5556> 2014-08-13 17:22:48.952113 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 5 ==== pg_notify(22.3af epoch 194470) v5 ==== 694+0+0 (4225460538 0 0) 0x466e900 con 0x48db9c0 -5555> 2014-08-13 17:22:48.952138 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.947327, event: header_read, request: pg_notify(22.3af epoch 194470) v5 -5554> 2014-08-13 17:22:48.952149 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.947337, event: throttled, request: pg_notify(22.3af epoch 194470) v5 -5553> 2014-08-13 17:22:48.952155 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.947403, event: all_read, request: pg_notify(22.3af epoch 194470) v5 -5552> 2014-08-13 17:22:48.952160 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.952136, event: dispatched, request: pg_notify(22.3af epoch 194470) v5 -5551> 2014-08-13 17:22:48.952166 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.952165, event: waiting_for_osdmap, request: pg_notify(22.3af epoch 194470) v5 -5550> 2014-08-13 17:22:48.952183 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.952183, event: started, request: pg_notify(22.3af epoch 194470) v5 -5549> 2014-08-13 17:22:48.952205 7f74d8735700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:22:48.952205, event: done, request: pg_notify(22.3af epoch 194470) v5 -5548> 2014-08-13 17:22:48.952225 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.341 192.168.1.106:6818/9142 1 ==== osd_map(194456..194470 src has 185139..194470) v3 ==== 9267+0+0 (3513188942 0 0) 0x4de4ec0 con 0x48de880 -5547> 2014-08-13 17:22:48.952247 7f74d8735700 3 osd.4 194470 handle_osd_map epochs [194456,194470], i have 194470, src has [185139,194470] -5546> 2014-08-13 17:22:48.952262 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 3 ==== pg_notify(24.1bb(1) epoch 194470) v5 ==== 811+0+0 (655629246 0 0) 0x57d78c0 con 0x48dcd00 -5545> 2014-08-13 17:22:48.952280 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.947027, event: header_read, request: pg_notify(24.1bb(1) epoch 194470) v5 -5544> 2014-08-13 17:22:48.952290 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.947029, event: throttled, request: pg_notify(24.1bb(1) epoch 194470) v5 -5543> 2014-08-13 17:22:48.952273 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.238899 3 0.000164 -5542> 2014-08-13 17:22:48.952302 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.947112, event: all_read, request: pg_notify(24.1bb(1) epoch 194470) v5 -5541> 2014-08-13 17:22:48.952322 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.952277, event: dispatched, request: pg_notify(24.1bb(1) epoch 194470) v5 -5540> 2014-08-13 17:22:48.952317 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.238988 0 0.000000 -5539> 2014-08-13 17:22:48.952333 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.952332, event: waiting_for_osdmap, request: pg_notify(24.1bb(1) epoch 194470) v5 -5538> 2014-08-13 17:22:48.952345 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.952345, event: started, request: pg_notify(24.1bb(1) epoch 194470) v5 -5537> 2014-08-13 17:22:48.952338 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194447/194448/192949) [328,4,20] r=1 lpr=194449 pi=192975-194447/17 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -5536> 2014-08-13 17:22:48.952373 7f74d8735700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:22:48.952372, event: done, request: pg_notify(24.1bb(1) epoch 194470) v5 -5535> 2014-08-13 17:22:48.952409 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 4 ==== pg_notify(22.3af(22) epoch 194470) v5 ==== 1948+0+0 (2286998326 0 0) 0x57d3480 con 0x4900f20 -5534> 2014-08-13 17:22:48.952431 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.947146, event: header_read, request: pg_notify(22.3af(22) epoch 194470) v5 -5533> 2014-08-13 17:22:48.952421 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(29) epoch 194470) v5 -- ?+0 0x4ac7700 con 0x47a53e0 -5532> 2014-08-13 17:22:48.952438 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.947149, event: throttled, request: pg_notify(22.3af(22) epoch 194470) v5 -5531> 2014-08-13 17:22:48.952447 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.947278, event: all_read, request: pg_notify(22.3af(22) epoch 194470) v5 -5530> 2014-08-13 17:22:48.952447 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(19) epoch 194470) v5 -- ?+0 0x4ac3800 con 0x475a940 -5529> 2014-08-13 17:22:48.952456 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.952428, event: dispatched, request: pg_notify(22.3af(22) epoch 194470) v5 -5528> 2014-08-13 17:22:48.952463 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.952463, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194470) v5 -5527> 2014-08-13 17:22:48.952468 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(33) epoch 194470) v5 -- ?+0 0x52639c0 con 0x47f4620 -5526> 2014-08-13 17:22:48.952475 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.952475, event: started, request: pg_notify(22.3af(22) epoch 194470) v5 -5525> 2014-08-13 17:22:48.952486 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(20) epoch 194470) v5 -- ?+0 0x5263b80 con 0x4831b80 -5524> 2014-08-13 17:22:48.952496 7f74d8735700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:22:48.952496, event: done, request: pg_notify(22.3af(22) epoch 194470) v5 -5523> 2014-08-13 17:22:48.952507 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(23) epoch 194470) v5 -- ?+0 0x5264980 con 0x475cd00 -5522> 2014-08-13 17:22:48.952520 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 5 ==== pg_notify(22.113(1) epoch 194470) v5 ==== 777+0+0 (2701808073 0 0) 0x52647c0 con 0x4901600 -5521> 2014-08-13 17:22:48.952533 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(36) epoch 194470) v5 -- ?+0 0x466f380 con 0x47a27e0 -5520> 2014-08-13 17:22:48.952541 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.919088, event: header_read, request: pg_notify(22.113(1) epoch 194470) v5 -5519> 2014-08-13 17:22:48.952548 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.919091, event: throttled, request: pg_notify(22.113(1) epoch 194470) v5 -5518> 2014-08-13 17:22:48.952553 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.919165, event: all_read, request: pg_notify(22.113(1) epoch 194470) v5 -5517> 2014-08-13 17:22:48.952558 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.952538, event: dispatched, request: pg_notify(22.113(1) epoch 194470) v5 -5516> 2014-08-13 17:22:48.952554 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(23) epoch 194470) v5 -- ?+0 0x466eac0 con 0x47f0c60 -5515> 2014-08-13 17:22:48.952564 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.952564, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194470) v5 -5514> 2014-08-13 17:22:48.952572 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.952572, event: started, request: pg_notify(22.113(1) epoch 194470) v5 -5513> 2014-08-13 17:22:48.952590 7f74d8735700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:22:48.952590, event: done, request: pg_notify(22.113(1) epoch 194470) v5 -5512> 2014-08-13 17:22:48.952603 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 5 ==== pg_notify(22.1f1(22) epoch 194470) v5 ==== 1948+0+0 (597292973 0 0) 0x466b800 con 0x4902680 -5511> 2014-08-13 17:22:48.952619 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.942414, event: header_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -5510> 2014-08-13 17:22:48.952625 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.942418, event: throttled, request: pg_notify(22.1f1(22) epoch 194470) v5 -5509> 2014-08-13 17:22:48.952630 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.942528, event: all_read, request: pg_notify(22.1f1(22) epoch 194470) v5 -5508> 2014-08-13 17:22:48.952635 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.952617, event: dispatched, request: pg_notify(22.1f1(22) epoch 194470) v5 -5507> 2014-08-13 17:22:48.952641 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.952641, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194470) v5 -5506> 2014-08-13 17:22:48.952648 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.952647, event: started, request: pg_notify(22.1f1(22) epoch 194470) v5 -5505> 2014-08-13 17:22:48.952666 7f74d8735700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:22:48.952666, event: done, request: pg_notify(22.1f1(22) epoch 194470) v5 -5504> 2014-08-13 17:22:48.952694 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 5 ==== pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 ==== 1510+0+0 (2220136816 0 0) 0x57d2300 con 0x4903180 -5503> 2014-08-13 17:22:48.952714 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.919412, event: header_read, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5502> 2014-08-13 17:22:48.952725 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.919415, event: throttled, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5501> 2014-08-13 17:22:48.952715 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(19) epoch 194470) v5 -- ?+0 0x466ec80 con 0x475c4c0 -5500> 2014-08-13 17:22:48.952731 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.919506, event: all_read, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5499> 2014-08-13 17:22:48.952737 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.952711, event: dispatched, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5498> 2014-08-13 17:22:48.952742 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(19) epoch 194470) v5 -- ?+0 0x466b100 con 0x47f2680 -5497> 2014-08-13 17:22:48.952748 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.952748, event: waiting_for_osdmap, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5496> 2014-08-13 17:22:48.952771 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.952770, event: started, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5495> 2014-08-13 17:22:48.952936 7f74d8735700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:22:48.952935, event: done, request: pg_notify(1.2b(1),2.2a(1) epoch 194470) v5 -5494> 2014-08-13 17:22:48.952935 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.000597 15 0.000365 -5493> 2014-08-13 17:22:48.952961 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 5 ==== pg_notify(22.f3(18) epoch 194470) v5 ==== 1720+0+0 (3451712932 0 0) 0x53b9c00 con 0x4d58840 -5492> 2014-08-13 17:22:48.952968 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -5491> 2014-08-13 17:22:48.952984 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.929846, event: header_read, request: pg_notify(22.f3(18) epoch 194470) v5 -5490> 2014-08-13 17:22:48.952984 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -5489> 2014-08-13 17:22:48.952995 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.929849, event: throttled, request: pg_notify(22.f3(18) epoch 194470) v5 -5488> 2014-08-13 17:22:48.953007 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5487> 2014-08-13 17:22:48.953015 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.929959, event: all_read, request: pg_notify(22.f3(18) epoch 194470) v5 -5486> 2014-08-13 17:22:48.953036 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.952981, event: dispatched, request: pg_notify(22.f3(18) epoch 194470) v5 -5485> 2014-08-13 17:22:48.953031 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000047 0 0.000000 -5484> 2014-08-13 17:22:48.953044 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.953044, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194470) v5 -5483> 2014-08-13 17:22:48.953058 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194449/194449 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -5482> 2014-08-13 17:22:48.953067 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.953067, event: started, request: pg_notify(22.f3(18) epoch 194470) v5 -5481> 2014-08-13 17:22:48.953111 7f74d8735700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:22:48.953111, event: done, request: pg_notify(22.f3(18) epoch 194470) v5 -5480> 2014-08-13 17:22:48.953139 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 5 ==== pg_notify(22.1b8(20) epoch 194470) v5 ==== 1838+0+0 (1626369839 0 0) 0x57d6580 con 0x4d58580 -5479> 2014-08-13 17:22:48.953162 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.951704, event: header_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -5478> 2014-08-13 17:22:48.953180 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.951707, event: throttled, request: pg_notify(22.1b8(20) epoch 194470) v5 -5477> 2014-08-13 17:22:48.953190 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.951818, event: all_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -5476> 2014-08-13 17:22:48.953197 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.953159, event: dispatched, request: pg_notify(22.1b8(20) epoch 194470) v5 -5475> 2014-08-13 17:22:48.953206 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.953206, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194470) v5 -5474> 2014-08-13 17:22:48.953218 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.953218, event: started, request: pg_notify(22.1b8(20) epoch 194470) v5 -5473> 2014-08-13 17:22:48.953222 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.239481 3 0.000154 -5472> 2014-08-13 17:22:48.953242 7f74d8735700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:22:48.953242, event: done, request: pg_notify(22.1b8(20) epoch 194470) v5 -5471> 2014-08-13 17:22:48.953258 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started 0.239574 0 0.000000 -5470> 2014-08-13 17:22:48.953274 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 5 ==== pg_notify(22.39d(20) epoch 194470) v5 ==== 1834+0+0 (3883755796 0 0) 0x57d5080 con 0x4d582c0 -5469> 2014-08-13 17:22:48.953281 7f74d192a700 5 osd.4 pg_epoch: 194456 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194447/194447/193186) [45,319,4] r=2 lpr=194449 pi=193186-194446/19 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset -5468> 2014-08-13 17:22:48.953294 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.926009, event: header_read, request: pg_notify(22.39d(20) epoch 194470) v5 -5467> 2014-08-13 17:22:48.953307 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.926012, event: throttled, request: pg_notify(22.39d(20) epoch 194470) v5 -5466> 2014-08-13 17:22:48.953316 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.926121, event: all_read, request: pg_notify(22.39d(20) epoch 194470) v5 -5465> 2014-08-13 17:22:48.953325 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.953292, event: dispatched, request: pg_notify(22.39d(20) epoch 194470) v5 -5464> 2014-08-13 17:22:48.953334 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.953334, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194470) v5 -5463> 2014-08-13 17:22:48.953345 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.953345, event: started, request: pg_notify(22.39d(20) epoch 194470) v5 -5462> 2014-08-13 17:22:48.953369 7f74d8735700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:22:48.953368, event: done, request: pg_notify(22.39d(20) epoch 194470) v5 -5461> 2014-08-13 17:22:48.953393 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 3 ==== pg_notify(22.481(39) epoch 194470) v5 ==== 2903+0+0 (4120949430 0 0) 0x4ac0c40 con 0x4d58000 -5460> 2014-08-13 17:22:48.953413 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.948862, event: header_read, request: pg_notify(22.481(39) epoch 194470) v5 -5459> 2014-08-13 17:22:48.953423 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.948865, event: throttled, request: pg_notify(22.481(39) epoch 194470) v5 -5458> 2014-08-13 17:22:48.953432 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.949000, event: all_read, request: pg_notify(22.481(39) epoch 194470) v5 -5457> 2014-08-13 17:22:48.953441 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.953411, event: dispatched, request: pg_notify(22.481(39) epoch 194470) v5 -5456> 2014-08-13 17:22:48.953451 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.953451, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194470) v5 -5455> 2014-08-13 17:22:48.953462 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.953462, event: started, request: pg_notify(22.481(39) epoch 194470) v5 -5454> 2014-08-13 17:22:48.953484 7f74d8735700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:22:48.953484, event: done, request: pg_notify(22.481(39) epoch 194470) v5 -5453> 2014-08-13 17:22:48.953510 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 4 ==== pg_notify(24.136(21) epoch 194470) v5 ==== 1919+0+0 (1355132185 0 0) 0x57d63c0 con 0x4d51a20 -5452> 2014-08-13 17:22:48.953529 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.949607, event: header_read, request: pg_notify(24.136(21) epoch 194470) v5 -5451> 2014-08-13 17:22:48.953539 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.949610, event: throttled, request: pg_notify(24.136(21) epoch 194470) v5 -5450> 2014-08-13 17:22:48.953549 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.949730, event: all_read, request: pg_notify(24.136(21) epoch 194470) v5 -5449> 2014-08-13 17:22:48.953558 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.953527, event: dispatched, request: pg_notify(24.136(21) epoch 194470) v5 -5448> 2014-08-13 17:22:48.953567 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.953567, event: waiting_for_osdmap, request: pg_notify(24.136(21) epoch 194470) v5 -5447> 2014-08-13 17:22:48.953578 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.953578, event: started, request: pg_notify(24.136(21) epoch 194470) v5 -5446> 2014-08-13 17:22:48.953601 7f74d8735700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:22:48.953601, event: done, request: pg_notify(24.136(21) epoch 194470) v5 -5445> 2014-08-13 17:22:48.953626 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.348 192.168.1.110:6800/41204 2 ==== pg_notify(24.1bb epoch 194470) v5 ==== 694+0+0 (982899203 0 0) 0x4ac1180 con 0x54e1600 -5444> 2014-08-13 17:22:48.953648 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.948151, event: header_read, request: pg_notify(24.1bb epoch 194470) v5 -5443> 2014-08-13 17:22:48.953658 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.948153, event: throttled, request: pg_notify(24.1bb epoch 194470) v5 -5442> 2014-08-13 17:22:48.953667 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.948219, event: all_read, request: pg_notify(24.1bb epoch 194470) v5 -5441> 2014-08-13 17:22:48.953651 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.005500 4 0.000282 -5440> 2014-08-13 17:22:48.953677 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.953646, event: dispatched, request: pg_notify(24.1bb epoch 194470) v5 -5439> 2014-08-13 17:22:48.953686 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.953686, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194470) v5 -5438> 2014-08-13 17:22:48.953697 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.953697, event: started, request: pg_notify(24.1bb epoch 194470) v5 -5437> 2014-08-13 17:22:48.953690 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5436> 2014-08-13 17:22:48.953718 7f74d8735700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:22:48.953717, event: done, request: pg_notify(24.1bb epoch 194470) v5 -5435> 2014-08-13 17:22:48.953735 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.59 192.168.1.112:6822/5265 2 ==== pg_notify(24.1f7 epoch 194470) v5 ==== 694+0+0 (3014039305 0 0) 0x4ac1340 con 0x54e4360 -5434> 2014-08-13 17:22:48.953753 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.948702, event: header_read, request: pg_notify(24.1f7 epoch 194470) v5 -5433> 2014-08-13 17:22:48.953767 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.948703, event: throttled, request: pg_notify(24.1f7 epoch 194470) v5 -5432> 2014-08-13 17:22:48.953751 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.000471 15 0.000196 -5431> 2014-08-13 17:22:48.953776 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.948777, event: all_read, request: pg_notify(24.1f7 epoch 194470) v5 -5430> 2014-08-13 17:22:48.953766 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000077 0 0.000000 -5429> 2014-08-13 17:22:48.953785 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.953751, event: dispatched, request: pg_notify(24.1f7 epoch 194470) v5 -5428> 2014-08-13 17:22:48.953780 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started -5427> 2014-08-13 17:22:48.953794 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.953794, event: waiting_for_osdmap, request: pg_notify(24.1f7 epoch 194470) v5 -5426> 2014-08-13 17:22:48.953794 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5425> 2014-08-13 17:22:48.953807 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Start -5424> 2014-08-13 17:22:48.953820 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.953819, event: started, request: pg_notify(24.1f7 epoch 194470) v5 -5423> 2014-08-13 17:22:48.953840 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000047 0 0.000000 -5422> 2014-08-13 17:22:48.953842 7f74d192a700 1 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray -5421> 2014-08-13 17:22:48.953867 7f74d8735700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:22:48.953867, event: done, request: pg_notify(24.1f7 epoch 194470) v5 -5420> 2014-08-13 17:22:48.953870 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5419> 2014-08-13 17:22:48.953882 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 5 ==== pg_notify(22.3d5(2) epoch 194470) v5 ==== 828+0+0 (405522152 0 0) 0x4ac3f00 con 0x45f1760 -5418> 2014-08-13 17:22:48.953878 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000070 0 0.000000 -5417> 2014-08-13 17:22:48.953905 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.951695, event: header_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -5416> 2014-08-13 17:22:48.953912 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.951696, event: throttled, request: pg_notify(22.3d5(2) epoch 194470) v5 -5415> 2014-08-13 17:22:48.953921 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.951793, event: all_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -5414> 2014-08-13 17:22:48.953918 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(25) epoch 194470) v5 -- ?+0 0x466ad80 con 0x48d9ce0 -5413> 2014-08-13 17:22:48.953930 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.953903, event: dispatched, request: pg_notify(22.3d5(2) epoch 194470) v5 -5412> 2014-08-13 17:22:48.953916 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194448/194442 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray -5411> 2014-08-13 17:22:48.953952 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.953951, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194470) v5 -5410> 2014-08-13 17:22:48.953951 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(19) epoch 194470) v5 -- ?+0 0x49e0fc0 con 0x4868160 -5409> 2014-08-13 17:22:48.953961 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.953961, event: started, request: pg_notify(22.3d5(2) epoch 194470) v5 -5408> 2014-08-13 17:22:48.953972 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_notify(24.181(27) epoch 194470) v5 -- ?+0 0x49e2680 con 0x486bc80 -5407> 2014-08-13 17:22:48.953990 7f74d8735700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:22:48.953990, event: done, request: pg_notify(22.3d5(2) epoch 194470) v5 -5406> 2014-08-13 17:22:48.953993 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(19) epoch 194470) v5 -- ?+0 0x49e16c0 con 0x486a680 -5405> 2014-08-13 17:22:48.954010 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(19) epoch 194470) v5 -- ?+0 0x49e6e40 con 0x486aec0 -5404> 2014-08-13 17:22:48.954009 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 6 ==== pg_notify(24.136(21) epoch 194470) v5 ==== 1919+0+0 (1355132185 0 0) 0x4ac1c00 con 0x45f5960 -5403> 2014-08-13 17:22:48.954027 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.949772, event: header_read, request: pg_notify(24.136(21) epoch 194470) v5 -5402> 2014-08-13 17:22:48.954031 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(25) epoch 194470) v5 -- ?+0 0x49e0700 con 0x486b180 -5401> 2014-08-13 17:22:48.954046 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.949776, event: throttled, request: pg_notify(24.136(21) epoch 194470) v5 -5400> 2014-08-13 17:22:48.954062 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.949899, event: all_read, request: pg_notify(24.136(21) epoch 194470) v5 -5399> 2014-08-13 17:22:48.954068 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.954025, event: dispatched, request: pg_notify(24.136(21) epoch 194470) v5 -5398> 2014-08-13 17:22:48.954068 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(22) epoch 194470) v5 -- ?+0 0x49e2a00 con 0x4868420 -5397> 2014-08-13 17:22:48.954075 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.954075, event: waiting_for_osdmap, request: pg_notify(24.136(21) epoch 194470) v5 -5396> 2014-08-13 17:22:48.954082 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.954082, event: started, request: pg_notify(24.136(21) epoch 194470) v5 -5395> 2014-08-13 17:22:48.954101 7f74d8735700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:22:48.954101, event: done, request: pg_notify(24.136(21) epoch 194470) v5 -5394> 2014-08-13 17:22:48.954122 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 6 ==== pg_notify(24.3d(23) epoch 194470) v5 ==== 2037+0+0 (2598945574 0 0) 0x4ac7000 con 0x475ee00 -5393> 2014-08-13 17:22:48.954138 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.929339, event: header_read, request: pg_notify(24.3d(23) epoch 194470) v5 -5392> 2014-08-13 17:22:48.954149 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.929340, event: throttled, request: pg_notify(24.3d(23) epoch 194470) v5 -5391> 2014-08-13 17:22:48.954161 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.929438, event: all_read, request: pg_notify(24.3d(23) epoch 194470) v5 -5390> 2014-08-13 17:22:48.954166 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.954136, event: dispatched, request: pg_notify(24.3d(23) epoch 194470) v5 -5389> 2014-08-13 17:22:48.954179 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.954179, event: waiting_for_osdmap, request: pg_notify(24.3d(23) epoch 194470) v5 -5388> 2014-08-13 17:22:48.954191 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.954191, event: started, request: pg_notify(24.3d(23) epoch 194470) v5 -5387> 2014-08-13 17:22:48.954215 7f74d8735700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:22:48.954215, event: done, request: pg_notify(24.3d(23) epoch 194470) v5 -5386> 2014-08-13 17:22:48.954243 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 6 ==== pg_notify(22.48e(1) epoch 194470) v5 ==== 777+0+0 (2821184580 0 0) 0x4ac6ac0 con 0x47a27e0 -5385> 2014-08-13 17:22:48.954250 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(19) epoch 194470) v5 -- ?+0 0x57d6ac0 con 0x486a940 -5384> 2014-08-13 17:22:48.954263 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.946470, event: header_read, request: pg_notify(22.48e(1) epoch 194470) v5 -5383> 2014-08-13 17:22:48.954274 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.946472, event: throttled, request: pg_notify(22.48e(1) epoch 194470) v5 -5382> 2014-08-13 17:22:48.954281 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.946553, event: all_read, request: pg_notify(22.48e(1) epoch 194470) v5 -5381> 2014-08-13 17:22:48.954290 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.954261, event: dispatched, request: pg_notify(22.48e(1) epoch 194470) v5 -5380> 2014-08-13 17:22:48.954300 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.954300, event: waiting_for_osdmap, request: pg_notify(22.48e(1) epoch 194470) v5 -5379> 2014-08-13 17:22:48.954311 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.954310, event: started, request: pg_notify(22.48e(1) epoch 194470) v5 -5378> 2014-08-13 17:22:48.954333 7f74d8735700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:22:48.954333, event: done, request: pg_notify(22.48e(1) epoch 194470) v5 -5377> 2014-08-13 17:22:48.954354 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 6 ==== pg_notify(22.3af(1) epoch 194470) v5 ==== 777+0+0 (1401344078 0 0) 0x57d6900 con 0x47a0b00 -5376> 2014-08-13 17:22:48.954374 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.946936, event: header_read, request: pg_notify(22.3af(1) epoch 194470) v5 -5375> 2014-08-13 17:22:48.954384 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.946937, event: throttled, request: pg_notify(22.3af(1) epoch 194470) v5 -5374> 2014-08-13 17:22:48.954392 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.946997, event: all_read, request: pg_notify(22.3af(1) epoch 194470) v5 -5373> 2014-08-13 17:22:48.954401 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.954372, event: dispatched, request: pg_notify(22.3af(1) epoch 194470) v5 -5372> 2014-08-13 17:22:48.954411 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.954411, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194470) v5 -5371> 2014-08-13 17:22:48.954422 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.954421, event: started, request: pg_notify(22.3af(1) epoch 194470) v5 -5370> 2014-08-13 17:22:48.954445 7f74d8735700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:22:48.954445, event: done, request: pg_notify(22.3af(1) epoch 194470) v5 -5369> 2014-08-13 17:22:48.954462 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 4 ==== pg_notify(22.1b8(2) epoch 194470) v5 ==== 828+0+0 (1631333252 0 0) 0x57d2680 con 0x47a3f40 -5368> 2014-08-13 17:22:48.954480 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.952056, event: header_read, request: pg_notify(22.1b8(2) epoch 194470) v5 -5367> 2014-08-13 17:22:48.954490 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.952057, event: throttled, request: pg_notify(22.1b8(2) epoch 194470) v5 -5366> 2014-08-13 17:22:48.954496 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.952138, event: all_read, request: pg_notify(22.1b8(2) epoch 194470) v5 -5365> 2014-08-13 17:22:48.954501 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.954478, event: dispatched, request: pg_notify(22.1b8(2) epoch 194470) v5 -5364> 2014-08-13 17:22:48.954520 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.954520, event: waiting_for_osdmap, request: pg_notify(22.1b8(2) epoch 194470) v5 -5363> 2014-08-13 17:22:48.954531 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.954531, event: started, request: pg_notify(22.1b8(2) epoch 194470) v5 -5362> 2014-08-13 17:22:48.954553 7f74d8735700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:22:48.954553, event: done, request: pg_notify(22.1b8(2) epoch 194470) v5 -5361> 2014-08-13 17:22:48.954570 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 6 ==== pg_notify(24.136 epoch 194470) v5 ==== 694+0+0 (191053422 0 0) 0x4ac0000 con 0x47f7e80 -5360> 2014-08-13 17:22:48.954588 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.949240, event: header_read, request: pg_notify(24.136 epoch 194470) v5 -5359> 2014-08-13 17:22:48.954595 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.949241, event: throttled, request: pg_notify(24.136 epoch 194470) v5 -5358> 2014-08-13 17:22:48.954600 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.949324, event: all_read, request: pg_notify(24.136 epoch 194470) v5 -5357> 2014-08-13 17:22:48.954608 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.954586, event: dispatched, request: pg_notify(24.136 epoch 194470) v5 -5356> 2014-08-13 17:22:48.954617 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.954617, event: waiting_for_osdmap, request: pg_notify(24.136 epoch 194470) v5 -5355> 2014-08-13 17:22:48.954628 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.954628, event: started, request: pg_notify(24.136 epoch 194470) v5 -5354> 2014-08-13 17:22:48.954650 7f74d8735700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:22:48.954649, event: done, request: pg_notify(24.136 epoch 194470) v5 -5353> 2014-08-13 17:22:48.954666 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 6 ==== pg_notify(24.1bb(25) epoch 194470) v5 ==== 2155+0+0 (2005506023 0 0) 0x466f8c0 con 0x4833440 -5352> 2014-08-13 17:22:48.954685 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.947292, event: header_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -5351> 2014-08-13 17:22:48.954695 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.947294, event: throttled, request: pg_notify(24.1bb(25) epoch 194470) v5 -5350> 2014-08-13 17:22:48.954704 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.947412, event: all_read, request: pg_notify(24.1bb(25) epoch 194470) v5 -5349> 2014-08-13 17:22:48.954713 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.954683, event: dispatched, request: pg_notify(24.1bb(25) epoch 194470) v5 -5348> 2014-08-13 17:22:48.954722 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.954722, event: waiting_for_osdmap, request: pg_notify(24.1bb(25) epoch 194470) v5 -5347> 2014-08-13 17:22:48.954733 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.954733, event: started, request: pg_notify(24.1bb(25) epoch 194470) v5 -5346> 2014-08-13 17:22:48.954756 7f74d8735700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:22:48.954756, event: done, request: pg_notify(24.1bb(25) epoch 194470) v5 -5345> 2014-08-13 17:22:48.954788 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 3 ==== pg_notify(22.1b8(20) epoch 194470) v5 ==== 1838+0+0 (1499644251 0 0) 0x57d40c0 con 0x4831600 -5344> 2014-08-13 17:22:48.954808 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.952883, event: header_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -5343> 2014-08-13 17:22:48.954818 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.952884, event: throttled, request: pg_notify(22.1b8(20) epoch 194470) v5 -5342> 2014-08-13 17:22:48.954824 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.952989, event: all_read, request: pg_notify(22.1b8(20) epoch 194470) v5 -5341> 2014-08-13 17:22:48.954829 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.954805, event: dispatched, request: pg_notify(22.1b8(20) epoch 194470) v5 -5340> 2014-08-13 17:22:48.954838 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.954838, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194470) v5 -5339> 2014-08-13 17:22:48.954846 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.954846, event: started, request: pg_notify(22.1b8(20) epoch 194470) v5 -5338> 2014-08-13 17:22:48.954865 7f74d8735700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:22:48.954864, event: done, request: pg_notify(22.1b8(20) epoch 194470) v5 -5337> 2014-08-13 17:22:48.954891 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 6 ==== pg_notify(22.48e(20) epoch 194470) v5 ==== 1834+0+0 (1696045208 0 0) 0x4ac2bc0 con 0x4835280 -5336> 2014-08-13 17:22:48.954890 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(23) epoch 194470) v5 -- ?+0 0x5263480 con 0x4835ac0 -5335> 2014-08-13 17:22:48.954909 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.947911, event: header_read, request: pg_notify(22.48e(20) epoch 194470) v5 -5334> 2014-08-13 17:22:48.954917 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.947913, event: throttled, request: pg_notify(22.48e(20) epoch 194470) v5 -5333> 2014-08-13 17:22:48.954922 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.948081, event: all_read, request: pg_notify(22.48e(20) epoch 194470) v5 -5332> 2014-08-13 17:22:48.954927 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.954907, event: dispatched, request: pg_notify(22.48e(20) epoch 194470) v5 -5331> 2014-08-13 17:22:48.954934 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.954934, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194470) v5 -5330> 2014-08-13 17:22:48.954940 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(19) epoch 194470) v5 -- ?+0 0x57d2840 con 0x45f2940 -5329> 2014-08-13 17:22:48.954944 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.954944, event: started, request: pg_notify(22.48e(20) epoch 194470) v5 -5328> 2014-08-13 17:22:48.954965 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(23) epoch 194470) v5 -- ?+0 0x57d2d80 con 0x47a32e0 -5327> 2014-08-13 17:22:48.954984 7f74d8735700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:22:48.954984, event: done, request: pg_notify(22.48e(20) epoch 194470) v5 -5326> 2014-08-13 17:22:48.955005 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(19) epoch 194470) v5 -- ?+0 0x57d5b00 con 0x45f6f60 -5325> 2014-08-13 17:22:48.955008 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 5 ==== pg_notify(22.3d5(2) epoch 194470) v5 ==== 828+0+0 (2506349106 0 0) 0x57d1a40 con 0x48377a0 -5324> 2014-08-13 17:22:48.955026 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.952052, event: header_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -5323> 2014-08-13 17:22:48.955031 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6834/14316 -- pg_notify(22.14a(19) epoch 194470) v5 -- ?+0 0x466b800 con 0x45f1e40 -5322> 2014-08-13 17:22:48.955037 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.952054, event: throttled, request: pg_notify(22.3d5(2) epoch 194470) v5 -5321> 2014-08-13 17:22:48.955067 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.952139, event: all_read, request: pg_notify(22.3d5(2) epoch 194470) v5 -5320> 2014-08-13 17:22:48.955078 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.955024, event: dispatched, request: pg_notify(22.3d5(2) epoch 194470) v5 -5319> 2014-08-13 17:22:48.955073 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(19) epoch 194470) v5 -- ?+0 0x466e900 con 0x475d960 -5318> 2014-08-13 17:22:48.955090 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.955090, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194470) v5 -5317> 2014-08-13 17:22:48.955099 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.955098, event: started, request: pg_notify(22.3d5(2) epoch 194470) v5 -5316> 2014-08-13 17:22:48.955123 7f74d8735700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:22:48.955122, event: done, request: pg_notify(22.3d5(2) epoch 194470) v5 -5315> 2014-08-13 17:22:48.955122 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(20),22.2b9(19) epoch 194470) v5 -- ?+0 0x466b2c0 con 0x4758b00 -5314> 2014-08-13 17:22:48.955141 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 6 ==== pg_notify(22.39d(20) epoch 194470) v5 ==== 1834+0+0 (3883755796 0 0) 0x57d5940 con 0x486bc80 -5313> 2014-08-13 17:22:48.955160 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.926295, event: header_read, request: pg_notify(22.39d(20) epoch 194470) v5 -5312> 2014-08-13 17:22:48.955180 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.926297, event: throttled, request: pg_notify(22.39d(20) epoch 194470) v5 -5311> 2014-08-13 17:22:48.955172 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(23) epoch 194470) v5 -- ?+0 0x52647c0 con 0x45f5960 -5310> 2014-08-13 17:22:48.955196 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.926392, event: all_read, request: pg_notify(22.39d(20) epoch 194470) v5 -5309> 2014-08-13 17:22:48.955215 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.955158, event: dispatched, request: pg_notify(22.39d(20) epoch 194470) v5 -5308> 2014-08-13 17:22:48.955223 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.955223, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194470) v5 -5307> 2014-08-13 17:22:48.955231 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(39) epoch 194470) v5 -- ?+0 0x5263d40 con 0x486b9c0 -5306> 2014-08-13 17:22:48.955230 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.955230, event: started, request: pg_notify(22.39d(20) epoch 194470) v5 -5305> 2014-08-13 17:22:48.955253 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(22) epoch 194470) v5 -- ?+0 0x4f516c0 con 0x4835280 -5304> 2014-08-13 17:22:48.955240 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.035000 5 0.000257 -5303> 2014-08-13 17:22:48.955263 7f74d8735700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:22:48.955262, event: done, request: pg_notify(22.39d(20) epoch 194470) v5 -5302> 2014-08-13 17:22:48.955273 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6816/33855 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c7440 con 0x4903700 -5301> 2014-08-13 17:22:48.955272 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5300> 2014-08-13 17:22:48.955292 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6814/25893 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c4ec0 con 0x4834620 -5299> 2014-08-13 17:22:48.955315 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c5280 con 0x486a3c0 -5298> 2014-08-13 17:22:48.955332 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000060 0 0.000000 -5297> 2014-08-13 17:22:48.955360 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5296> 2014-08-13 17:22:48.955385 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000026 0 0.000000 -5295> 2014-08-13 17:22:48.955408 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5294> 2014-08-13 17:22:48.955281 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 5 ==== pg_notify(24.b6(1) epoch 194470) v5 ==== 811+0+0 (3923748831 0 0) 0x4ac2a00 con 0x486b700 -5293> 2014-08-13 17:22:48.955437 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c6180 con 0x4835280 -5292> 2014-08-13 17:22:48.955452 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.947225, event: header_read, request: pg_notify(24.b6(1) epoch 194470) v5 -5291> 2014-08-13 17:22:48.955467 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.947227, event: throttled, request: pg_notify(24.b6(1) epoch 194470) v5 -5290> 2014-08-13 17:22:48.955467 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6814/41806 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c79e0 con 0x4900580 -5289> 2014-08-13 17:22:48.955476 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.947480, event: all_read, request: pg_notify(24.b6(1) epoch 194470) v5 -5288> 2014-08-13 17:22:48.955485 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.955448, event: dispatched, request: pg_notify(24.b6(1) epoch 194470) v5 -5287> 2014-08-13 17:22:48.955495 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.955495, event: waiting_for_osdmap, request: pg_notify(24.b6(1) epoch 194470) v5 -5286> 2014-08-13 17:22:48.955507 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.955507, event: started, request: pg_notify(24.b6(1) epoch 194470) v5 -5285> 2014-08-13 17:22:48.955544 7f74d8735700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:22:48.955544, event: done, request: pg_notify(24.b6(1) epoch 194470) v5 -5284> 2014-08-13 17:22:48.955565 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 6 ==== pg_notify(24.3d(23) epoch 194470) v5 ==== 2037+0+0 (2598945574 0 0) 0x53ba140 con 0x486a3c0 -5283> 2014-08-13 17:22:48.955586 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.929852, event: header_read, request: pg_notify(24.3d(23) epoch 194470) v5 -5282> 2014-08-13 17:22:48.955596 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.929853, event: throttled, request: pg_notify(24.3d(23) epoch 194470) v5 -5281> 2014-08-13 17:22:48.955605 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.929960, event: all_read, request: pg_notify(24.3d(23) epoch 194470) v5 -5280> 2014-08-13 17:22:48.955614 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.955583, event: dispatched, request: pg_notify(24.3d(23) epoch 194470) v5 -5279> 2014-08-13 17:22:48.955624 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.955624, event: waiting_for_osdmap, request: pg_notify(24.3d(23) epoch 194470) v5 -5278> 2014-08-13 17:22:48.955635 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.955635, event: started, request: pg_notify(24.3d(23) epoch 194470) v5 -5277> 2014-08-13 17:22:48.955641 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(20) epoch 194470) v5 -- ?+0 0x57d7000 con 0x475c4c0 -5276> 2014-08-13 17:22:48.955658 7f74d8735700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:22:48.955658, event: done, request: pg_notify(24.3d(23) epoch 194470) v5 -5275> 2014-08-13 17:22:48.955683 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.341 192.168.1.106:6818/9142 2 ==== pg_notify(22.48e epoch 194470) v5 ==== 694+0+0 (782494176 0 0) 0x4ac0a80 con 0x48de880 -5274> 2014-08-13 17:22:48.955702 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.947652, event: header_read, request: pg_notify(22.48e epoch 194470) v5 -5273> 2014-08-13 17:22:48.955713 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.947654, event: throttled, request: pg_notify(22.48e epoch 194470) v5 -5272> 2014-08-13 17:22:48.955722 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.947723, event: all_read, request: pg_notify(22.48e epoch 194470) v5 -5271> 2014-08-13 17:22:48.955731 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.955700, event: dispatched, request: pg_notify(22.48e epoch 194470) v5 -5270> 2014-08-13 17:22:48.955741 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.955741, event: waiting_for_osdmap, request: pg_notify(22.48e epoch 194470) v5 -5269> 2014-08-13 17:22:48.955752 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.955751, event: started, request: pg_notify(22.48e epoch 194470) v5 -5268> 2014-08-13 17:22:48.955773 7f74d8735700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:22:48.955773, event: done, request: pg_notify(22.48e epoch 194470) v5 -5267> 2014-08-13 17:22:48.955801 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 4 ==== pg_notify(22.3d5(20) epoch 194470) v5 ==== 1838+0+0 (1108422747 0 0) 0x53b9500 con 0x48dfe80 -5266> 2014-08-13 17:22:48.955820 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.952416, event: header_read, request: pg_notify(22.3d5(20) epoch 194470) v5 -5265> 2014-08-13 17:22:48.955830 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.952418, event: throttled, request: pg_notify(22.3d5(20) epoch 194470) v5 -5264> 2014-08-13 17:22:48.955839 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.952541, event: all_read, request: pg_notify(22.3d5(20) epoch 194470) v5 -5263> 2014-08-13 17:22:48.955848 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.955818, event: dispatched, request: pg_notify(22.3d5(20) epoch 194470) v5 -5262> 2014-08-13 17:22:48.955858 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.955858, event: waiting_for_osdmap, request: pg_notify(22.3d5(20) epoch 194470) v5 -5261> 2014-08-13 17:22:48.955869 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.955869, event: started, request: pg_notify(22.3d5(20) epoch 194470) v5 -5260> 2014-08-13 17:22:48.955891 7f74d8735700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:22:48.955891, event: done, request: pg_notify(22.3d5(20) epoch 194470) v5 -5259> 2014-08-13 17:22:48.955888 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.021816 4 0.000229 -5258> 2014-08-13 17:22:48.955915 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 6 ==== pg_notify(22.3e4(22) epoch 194470) v5 ==== 1948+0+0 (2374870074 0 0) 0x5264600 con 0x4901600 -5257> 2014-08-13 17:22:48.955920 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5256> 2014-08-13 17:22:48.955934 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.937143, event: header_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -5255> 2014-08-13 17:22:48.955944 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.937173, event: throttled, request: pg_notify(22.3e4(22) epoch 194470) v5 -5254> 2014-08-13 17:22:48.955953 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.937278, event: all_read, request: pg_notify(22.3e4(22) epoch 194470) v5 -5253> 2014-08-13 17:22:48.955962 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.955932, event: dispatched, request: pg_notify(22.3e4(22) epoch 194470) v5 -5252> 2014-08-13 17:22:48.955972 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.955972, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194470) v5 -5251> 2014-08-13 17:22:48.955983 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.955983, event: started, request: pg_notify(22.3e4(22) epoch 194470) v5 -5250> 2014-08-13 17:22:48.955985 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000065 0 0.000000 -5249> 2014-08-13 17:22:48.956007 7f74d8735700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:22:48.956006, event: done, request: pg_notify(22.3e4(22) epoch 194470) v5 -5248> 2014-08-13 17:22:48.956011 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5247> 2014-08-13 17:22:48.956038 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 6 ==== pg_notify(22.22c(22) epoch 194470) v5 ==== 1952+0+0 (46188357 0 0) 0x53b8fc0 con 0x4903180 -5246> 2014-08-13 17:22:48.956036 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000025 0 0.000000 -5245> 2014-08-13 17:22:48.956061 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.929614, event: header_read, request: pg_notify(22.22c(22) epoch 194470) v5 -5244> 2014-08-13 17:22:48.956068 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.929617, event: throttled, request: pg_notify(22.22c(22) epoch 194470) v5 -5243> 2014-08-13 17:22:48.956078 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.929705, event: all_read, request: pg_notify(22.22c(22) epoch 194470) v5 -5242> 2014-08-13 17:22:48.956071 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5241> 2014-08-13 17:22:48.956087 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.956059, event: dispatched, request: pg_notify(22.22c(22) epoch 194470) v5 -5240> 2014-08-13 17:22:48.956097 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.956097, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194470) v5 -5239> 2014-08-13 17:22:48.956087 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(23) epoch 194470) v5 -- ?+0 0x4ac2300 con 0x4835ac0 -5238> 2014-08-13 17:22:48.956109 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.956109, event: started, request: pg_notify(22.22c(22) epoch 194470) v5 -5237> 2014-08-13 17:22:48.956116 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(19) epoch 194470) v5 -- ?+0 0x57d63c0 con 0x475e5c0 -5236> 2014-08-13 17:22:48.956132 7f74d8735700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:22:48.956131, event: done, request: pg_notify(22.22c(22) epoch 194470) v5 -5235> 2014-08-13 17:22:48.956140 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(20) epoch 194470) v5 -- ?+0 0x57d5080 con 0x4758b00 -5234> 2014-08-13 17:22:48.956161 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 6 ==== pg_notify(22.775(1) epoch 194470) v5 ==== 777+0+0 (716904798 0 0) 0x53b9a40 con 0x4d58840 -5233> 2014-08-13 17:22:48.956171 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(26) epoch 194470) v5 -- ?+0 0x57d6580 con 0x47a4200 -5232> 2014-08-13 17:22:48.956188 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.932643, event: header_read, request: pg_notify(22.775(1) epoch 194470) v5 -5231> 2014-08-13 17:22:48.956196 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.932645, event: throttled, request: pg_notify(22.775(1) epoch 194470) v5 -5230> 2014-08-13 17:22:48.956198 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(39) epoch 194470) v5 -- ?+0 0x57d2300 con 0x486b9c0 -5229> 2014-08-13 17:22:48.956204 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.932717, event: all_read, request: pg_notify(22.775(1) epoch 194470) v5 -5228> 2014-08-13 17:22:48.956223 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.956186, event: dispatched, request: pg_notify(22.775(1) epoch 194470) v5 -5227> 2014-08-13 17:22:48.956238 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.956238, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194470) v5 -5226> 2014-08-13 17:22:48.956250 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.956250, event: started, request: pg_notify(22.775(1) epoch 194470) v5 -5225> 2014-08-13 17:22:48.956274 7f74d8735700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:22:48.956274, event: done, request: pg_notify(22.775(1) epoch 194470) v5 -5224> 2014-08-13 17:22:48.956293 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 7 ==== pg_notify(22.3e4(1) epoch 194470) v5 ==== 777+0+0 (3137208198 0 0) 0x4ac6e40 con 0x475ee00 -5223> 2014-08-13 17:22:48.956312 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.936827, event: header_read, request: pg_notify(22.3e4(1) epoch 194470) v5 -5222> 2014-08-13 17:22:48.956265 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.035447 5 0.000266 -5221> 2014-08-13 17:22:48.956324 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.936829, event: throttled, request: pg_notify(22.3e4(1) epoch 194470) v5 -5220> 2014-08-13 17:22:48.956333 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.936897, event: all_read, request: pg_notify(22.3e4(1) epoch 194470) v5 -5219> 2014-08-13 17:22:48.956330 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5218> 2014-08-13 17:22:48.956342 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.956310, event: dispatched, request: pg_notify(22.3e4(1) epoch 194470) v5 -5217> 2014-08-13 17:22:48.956363 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.956363, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194470) v5 -5216> 2014-08-13 17:22:48.956373 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.956373, event: started, request: pg_notify(22.3e4(1) epoch 194470) v5 -5215> 2014-08-13 17:22:48.956394 7f74d8735700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:22:48.956393, event: done, request: pg_notify(22.3e4(1) epoch 194470) v5 -5214> 2014-08-13 17:22:48.956422 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 7 ==== pg_notify(24.136 epoch 194470) v5 ==== 694+0+0 (191053422 0 0) 0x57d6740 con 0x47a0b00 -5213> 2014-08-13 17:22:48.956418 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000089 0 0.000000 -5212> 2014-08-13 17:22:48.956437 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.949306, event: header_read, request: pg_notify(24.136 epoch 194470) v5 -5211> 2014-08-13 17:22:48.956443 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.949308, event: throttled, request: pg_notify(24.136 epoch 194470) v5 -5210> 2014-08-13 17:22:48.956448 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.949377, event: all_read, request: pg_notify(24.136 epoch 194470) v5 -5209> 2014-08-13 17:22:48.956441 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5208> 2014-08-13 17:22:48.956453 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.956435, event: dispatched, request: pg_notify(24.136 epoch 194470) v5 -5207> 2014-08-13 17:22:48.956460 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.956459, event: waiting_for_osdmap, request: pg_notify(24.136 epoch 194470) v5 -5206> 2014-08-13 17:22:48.956466 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.956466, event: started, request: pg_notify(24.136 epoch 194470) v5 -5205> 2014-08-13 17:22:48.956466 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000025 0 0.000000 -5204> 2014-08-13 17:22:48.956484 7f74d8735700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:22:48.956484, event: done, request: pg_notify(24.136 epoch 194470) v5 -5203> 2014-08-13 17:22:48.956487 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5202> 2014-08-13 17:22:48.956505 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 7 ==== pg_notify(24.1f7(23) epoch 194470) v5 ==== 2037+0+0 (2318168837 0 0) 0x4ac40c0 con 0x4835280 -5201> 2014-08-13 17:22:48.956520 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.948124, event: header_read, request: pg_notify(24.1f7(23) epoch 194470) v5 -5200> 2014-08-13 17:22:48.956526 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.948125, event: throttled, request: pg_notify(24.1f7(23) epoch 194470) v5 -5199> 2014-08-13 17:22:48.956531 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.948238, event: all_read, request: pg_notify(24.1f7(23) epoch 194470) v5 -5198> 2014-08-13 17:22:48.956536 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.956518, event: dispatched, request: pg_notify(24.1f7(23) epoch 194470) v5 -5197> 2014-08-13 17:22:48.956542 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.956542, event: waiting_for_osdmap, request: pg_notify(24.1f7(23) epoch 194470) v5 -5196> 2014-08-13 17:22:48.956549 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.956549, event: started, request: pg_notify(24.1f7(23) epoch 194470) v5 -5195> 2014-08-13 17:22:48.956538 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.036761 6 0.000331 -5194> 2014-08-13 17:22:48.956568 7f74d8735700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:22:48.956568, event: done, request: pg_notify(24.1f7(23) epoch 194470) v5 -5193> 2014-08-13 17:22:48.956570 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5192> 2014-08-13 17:22:48.956606 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 3 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (2543075558 0 0) 0x5262d80 con 0x4834620 -5191> 2014-08-13 17:22:48.956625 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956328, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5190> 2014-08-13 17:22:48.956636 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956329, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -5189> 2014-08-13 17:22:48.956633 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000064 0 0.000000 -5188> 2014-08-13 17:22:48.956645 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956454, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5187> 2014-08-13 17:22:48.956628 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetInfo 0.042156 6 0.021273 -5186> 2014-08-13 17:22:48.956666 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956623, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -5185> 2014-08-13 17:22:48.956663 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5184> 2014-08-13 17:22:48.956673 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956673, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -5183> 2014-08-13 17:22:48.956675 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetLog -5182> 2014-08-13 17:22:48.956696 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956696, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -5181> 2014-08-13 17:22:48.956689 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000026 0 0.000000 -5180> 2014-08-13 17:22:48.956729 7f74d8735700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:22:48.956729, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -5179> 2014-08-13 17:22:48.956725 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5178> 2014-08-13 17:22:48.956749 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetLog 0.000075 0 0.000000 -5177> 2014-08-13 17:22:48.956772 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 7 ==== pg_notify(22.3d9(35) epoch 194470) v5 ==== 2627+0+0 (2695345450 0 0) 0x57d5400 con 0x486a3c0 -5176> 2014-08-13 17:22:48.956775 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering 0.042398 0 0.000000 -5175> 2014-08-13 17:22:48.956793 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956382, event: header_read, request: pg_notify(22.3d9(35) epoch 194470) v5 -5174> 2014-08-13 17:22:48.956801 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956384, event: throttled, request: pg_notify(22.3d9(35) epoch 194470) v5 -5173> 2014-08-13 17:22:48.956810 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956554, event: all_read, request: pg_notify(22.3d9(35) epoch 194470) v5 -5172> 2014-08-13 17:22:48.956803 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.042451 0 0.000000 -5171> 2014-08-13 17:22:48.956819 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956791, event: dispatched, request: pg_notify(22.3d9(35) epoch 194470) v5 -5170> 2014-08-13 17:22:48.956829 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956828, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194470) v5 -5169> 2014-08-13 17:22:48.956824 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary -5168> 2014-08-13 17:22:48.956840 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956839, event: started, request: pg_notify(22.3d9(35) epoch 194470) v5 -5167> 2014-08-13 17:22:48.956841 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering/WaitActingChange -5166> 2014-08-13 17:22:48.956864 7f74d8735700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:22:48.956864, event: done, request: pg_notify(22.3d9(35) epoch 194470) v5 -5165> 2014-08-13 17:22:48.956885 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.005670 8 0.000267 -5164> 2014-08-13 17:22:48.956896 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 4 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (3753113750 0 0) 0x466fe00 con 0x4900580 -5163> 2014-08-13 17:22:48.956914 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956487, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5162> 2014-08-13 17:22:48.956911 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(23) epoch 194470) v5 -- ?+0 0x57d1c00 con 0x47a32e0 -5161> 2014-08-13 17:22:48.956913 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5160> 2014-08-13 17:22:48.956926 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956491, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -5159> 2014-08-13 17:22:48.956939 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(28) epoch 194470) v5 -- ?+0 0x57d5cc0 con 0x486e040 -5158> 2014-08-13 17:22:48.956942 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956706, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5157> 2014-08-13 17:22:48.956951 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956912, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -5156> 2014-08-13 17:22:48.956957 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956957, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -5155> 2014-08-13 17:22:48.956958 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(23) epoch 194470) v5 -- ?+0 0x57d55c0 con 0x48377a0 -5154> 2014-08-13 17:22:48.956965 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.956965, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -5153> 2014-08-13 17:22:48.956974 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(22) epoch 194470) v5 -- ?+0 0x57d3d40 con 0x4832c00 -5152> 2014-08-13 17:22:48.956991 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(19) epoch 194470) v5 -- ?+0 0x57d32c0 con 0x48dfbc0 -5151> 2014-08-13 17:22:48.957010 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5150> 2014-08-13 17:22:48.957016 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb3a20 con 0x45d48e0 -5149> 2014-08-13 17:22:48.957017 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000104 0 0.000000 -5148> 2014-08-13 17:22:48.957038 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5147> 2014-08-13 17:22:48.957107 7f74d8735700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:22:48.957106, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -5146> 2014-08-13 17:22:48.957146 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.058999 4 0.000231 -5145> 2014-08-13 17:22:48.957157 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 7 ==== pg_notify(22.707(24) epoch 194470) v5 ==== 2066+0+0 (3384600017 0 0) 0x5264440 con 0x4901600 -5144> 2014-08-13 17:22:48.957183 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.941998, event: header_read, request: pg_notify(22.707(24) epoch 194470) v5 -5143> 2014-08-13 17:22:48.957192 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.942001, event: throttled, request: pg_notify(22.707(24) epoch 194470) v5 -5142> 2014-08-13 17:22:48.957185 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5141> 2014-08-13 17:22:48.957203 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.942122, event: all_read, request: pg_notify(22.707(24) epoch 194470) v5 -5140> 2014-08-13 17:22:48.957212 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.957180, event: dispatched, request: pg_notify(22.707(24) epoch 194470) v5 -5139> 2014-08-13 17:22:48.957219 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.957219, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194470) v5 -5138> 2014-08-13 17:22:48.957230 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.957230, event: started, request: pg_notify(22.707(24) epoch 194470) v5 -5137> 2014-08-13 17:22:48.957241 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000057 0 0.000000 -5136> 2014-08-13 17:22:48.957242 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.025902 8 0.000340 -5135> 2014-08-13 17:22:48.957257 7f74d8735700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:22:48.957256, event: done, request: pg_notify(22.707(24) epoch 194470) v5 -5134> 2014-08-13 17:22:48.957271 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5133> 2014-08-13 17:22:48.957273 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5132> 2014-08-13 17:22:48.957292 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 -5131> 2014-08-13 17:22:48.957296 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 3 ==== pg_notify(22.3d9(42) epoch 194470) v5 ==== 2984+0+0 (7658140 0 0) 0x57d6c80 con 0x4903700 -5130> 2014-08-13 17:22:48.957314 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.956746, event: header_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -5129> 2014-08-13 17:22:48.957314 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5128> 2014-08-13 17:22:48.957326 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.956749, event: throttled, request: pg_notify(22.3d9(42) epoch 194470) v5 -5127> 2014-08-13 17:22:48.957336 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.956896, event: all_read, request: pg_notify(22.3d9(42) epoch 194470) v5 -5126> 2014-08-13 17:22:48.957345 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.957312, event: dispatched, request: pg_notify(22.3d9(42) epoch 194470) v5 -5125> 2014-08-13 17:22:48.957335 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000062 0 0.000000 -5124> 2014-08-13 17:22:48.957356 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.957356, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194470) v5 -5123> 2014-08-13 17:22:48.957367 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.957367, event: started, request: pg_notify(22.3d9(42) epoch 194470) v5 -5122> 2014-08-13 17:22:48.957360 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5121> 2014-08-13 17:22:48.957395 7f74d8735700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:22:48.957395, event: done, request: pg_notify(22.3d9(42) epoch 194470) v5 -5120> 2014-08-13 17:22:48.957396 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000037 0 0.000000 -5119> 2014-08-13 17:22:48.957424 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5118> 2014-08-13 17:22:48.957433 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 7 ==== pg_notify(22.3af(22) epoch 194470) v5 ==== 1948+0+0 (2286998326 0 0) 0x53b9880 con 0x4d58840 -5117> 2014-08-13 17:22:48.957453 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.947490, event: header_read, request: pg_notify(22.3af(22) epoch 194470) v5 -5116> 2014-08-13 17:22:48.957463 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.947492, event: throttled, request: pg_notify(22.3af(22) epoch 194470) v5 -5115> 2014-08-13 17:22:48.957471 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.007402 4 0.000245 -5114> 2014-08-13 17:22:48.957490 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.062383 8 0.000263 -5113> 2014-08-13 17:22:48.957501 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5112> 2014-08-13 17:22:48.957472 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.947582, event: all_read, request: pg_notify(22.3af(22) epoch 194470) v5 -5111> 2014-08-13 17:22:48.957527 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.957451, event: dispatched, request: pg_notify(22.3af(22) epoch 194470) v5 -5110> 2014-08-13 17:22:48.957524 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5109> 2014-08-13 17:22:48.957535 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.957535, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194470) v5 -5108> 2014-08-13 17:22:48.957545 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.957545, event: started, request: pg_notify(22.3af(22) epoch 194470) v5 -5107> 2014-08-13 17:22:48.957571 7f74d8735700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:22:48.957570, event: done, request: pg_notify(22.3af(22) epoch 194470) v5 -5106> 2014-08-13 17:22:48.957565 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000064 0 0.000000 -5105> 2014-08-13 17:22:48.957581 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000057 0 0.000000 -5104> 2014-08-13 17:22:48.957586 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5103> 2014-08-13 17:22:48.957611 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 8 ==== pg_notify(22.707(1) epoch 194470) v5 ==== 777+0+0 (4211101445 0 0) 0x5263100 con 0x475ee00 -5102> 2014-08-13 17:22:48.957611 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5101> 2014-08-13 17:22:48.957633 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.942134, event: header_read, request: pg_notify(22.707(1) epoch 194470) v5 -5100> 2014-08-13 17:22:48.957622 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000036 0 0.000000 -5099> 2014-08-13 17:22:48.957647 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.942136, event: throttled, request: pg_notify(22.707(1) epoch 194470) v5 -5098> 2014-08-13 17:22:48.957635 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000024 0 0.000000 -5097> 2014-08-13 17:22:48.957660 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.942232, event: all_read, request: pg_notify(22.707(1) epoch 194470) v5 -5096> 2014-08-13 17:22:48.957662 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5095> 2014-08-13 17:22:48.957679 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.957631, event: dispatched, request: pg_notify(22.707(1) epoch 194470) v5 -5094> 2014-08-13 17:22:48.957674 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5093> 2014-08-13 17:22:48.957690 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.957690, event: waiting_for_osdmap, request: pg_notify(22.707(1) epoch 194470) v5 -5092> 2014-08-13 17:22:48.957701 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.957701, event: started, request: pg_notify(22.707(1) epoch 194470) v5 -5091> 2014-08-13 17:22:48.957724 7f74d8735700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:22:48.957724, event: done, request: pg_notify(22.707(1) epoch 194470) v5 -5090> 2014-08-13 17:22:48.957727 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.057486 7 0.000283 -5089> 2014-08-13 17:22:48.957750 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 8 ==== pg_notify(22.3d9(38) epoch 194470) v5 ==== 2772+0+0 (3345820745 0 0) 0x4f516c0 con 0x4835280 -5088> 2014-08-13 17:22:48.957754 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5087> 2014-08-13 17:22:48.957769 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.956394, event: header_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5086> 2014-08-13 17:22:48.957756 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.034799 7 0.000307 -5085> 2014-08-13 17:22:48.957777 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.956396, event: throttled, request: pg_notify(22.3d9(38) epoch 194470) v5 -5084> 2014-08-13 17:22:48.957785 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.956556, event: all_read, request: pg_notify(22.3d9(38) epoch 194470) v5 -5083> 2014-08-13 17:22:48.957794 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.957767, event: dispatched, request: pg_notify(22.3d9(38) epoch 194470) v5 -5082> 2014-08-13 17:22:48.957784 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog -5081> 2014-08-13 17:22:48.957808 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.957808, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194470) v5 -5080> 2014-08-13 17:22:48.957807 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000052 0 0.000000 -5079> 2014-08-13 17:22:48.957828 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.957828, event: started, request: pg_notify(22.3d9(38) epoch 194470) v5 -5078> 2014-08-13 17:22:48.957830 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5077> 2014-08-13 17:22:48.957855 7f74d8735700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:22:48.957854, event: done, request: pg_notify(22.3d9(38) epoch 194470) v5 -5076> 2014-08-13 17:22:48.957853 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -5075> 2014-08-13 17:22:48.957876 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5074> 2014-08-13 17:22:48.957874 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000090 0 0.000000 -5073> 2014-08-13 17:22:48.957885 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 9 ==== pg_notify(22.481(32) epoch 194470) v5 ==== 2518+0+0 (2237357479 0 0) 0x5263f00 con 0x475ee00 -5072> 2014-08-13 17:22:48.957903 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.946581, event: header_read, request: pg_notify(22.481(32) epoch 194470) v5 -5071> 2014-08-13 17:22:48.957899 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.034971 0 0.000000 -5070> 2014-08-13 17:22:48.957914 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.946583, event: throttled, request: pg_notify(22.481(32) epoch 194470) v5 -5069> 2014-08-13 17:22:48.957923 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.946701, event: all_read, request: pg_notify(22.481(32) epoch 194470) v5 -5068> 2014-08-13 17:22:48.957932 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.957900, event: dispatched, request: pg_notify(22.481(32) epoch 194470) v5 -5067> 2014-08-13 17:22:48.957922 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.035015 0 0.000000 -5066> 2014-08-13 17:22:48.957946 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.957946, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194470) v5 -5065> 2014-08-13 17:22:48.957956 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.957956, event: started, request: pg_notify(22.481(32) epoch 194470) v5 -5064> 2014-08-13 17:22:48.957957 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -5063> 2014-08-13 17:22:48.957981 7f74d8735700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:22:48.957981, event: done, request: pg_notify(22.481(32) epoch 194470) v5 -5062> 2014-08-13 17:22:48.957980 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange -5061> 2014-08-13 17:22:48.958011 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_query(22.3d9 epoch 194470) v3 -- ?+0 0x29c0f00 con 0x4835280 -5060> 2014-08-13 17:22:48.958045 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5059> 2014-08-13 17:22:48.958056 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.9b=[4,291,350],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4ec0 con 0x45d48e0 -5058> 2014-08-13 17:22:48.958052 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(19) epoch 194470) v5 -- ?+0 0x4ac7000 con 0x4833440 -5057> 2014-08-13 17:22:48.958092 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.3b7(28) epoch 194470) v5 -- ?+0 0x4ac1c00 con 0x45f5960 -5056> 2014-08-13 17:22:48.958113 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(22) epoch 194470) v5 -- ?+0 0x4ac3f00 con 0x4835280 -5055> 2014-08-13 17:22:48.958134 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5054> 2014-08-13 17:22:48.958141 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.9b=[4,291,350],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4380 con 0x45d48e0 -5053> 2014-08-13 17:22:48.958157 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.036923 9 0.000305 -5052> 2014-08-13 17:22:48.958192 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5051> 2014-08-13 17:22:48.958244 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.040915 8 0.000279 -5050> 2014-08-13 17:22:48.958256 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000064 0 0.000000 -5049> 2014-08-13 17:22:48.958271 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5048> 2014-08-13 17:22:48.958277 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5047> 2014-08-13 17:22:48.958298 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 -5046> 2014-08-13 17:22:48.958325 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5045> 2014-08-13 17:22:48.958326 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000055 0 0.000000 -5044> 2014-08-13 17:22:48.958347 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5043> 2014-08-13 17:22:48.958371 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000024 0 0.000000 -5042> 2014-08-13 17:22:48.958393 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5041> 2014-08-13 17:22:48.958409 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetInfo 0.031163 6 0.000280 -5040> 2014-08-13 17:22:48.958441 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetLog -5039> 2014-08-13 17:22:48.958461 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.035939 9 0.000259 -5038> 2014-08-13 17:22:48.958490 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5037> 2014-08-13 17:22:48.958512 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetLog 0.000071 0 0.000000 -5036> 2014-08-13 17:22:48.958537 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering 0.031310 0 0.000000 -5035> 2014-08-13 17:22:48.958545 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000055 0 0.000000 -5034> 2014-08-13 17:22:48.958558 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.031352 0 0.000000 -5033> 2014-08-13 17:22:48.958569 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5032> 2014-08-13 17:22:48.958582 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary -5031> 2014-08-13 17:22:48.958592 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -5030> 2014-08-13 17:22:48.958603 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering/WaitActingChange -5029> 2014-08-13 17:22:48.958616 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5028> 2014-08-13 17:22:48.958687 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.023315 6 0.000263 -5027> 2014-08-13 17:22:48.958716 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5026> 2014-08-13 17:22:48.958770 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000054 0 0.000000 -5025> 2014-08-13 17:22:48.958768 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.042348 4 0.000226 -5024> 2014-08-13 17:22:48.958794 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5023> 2014-08-13 17:22:48.958806 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5022> 2014-08-13 17:22:48.958817 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -5021> 2014-08-13 17:22:48.958839 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5020> 2014-08-13 17:22:48.958859 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000053 0 0.000000 -5019> 2014-08-13 17:22:48.958882 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5018> 2014-08-13 17:22:48.958901 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000019 0 0.000000 -5017> 2014-08-13 17:22:48.958919 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5016> 2014-08-13 17:22:48.958917 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 9 ==== pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 ==== 4056+0+0 (1075323055 0 0) 0x278c800 con 0x4835280 -5015> 2014-08-13 17:22:48.958949 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.958743, event: header_read, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5014> 2014-08-13 17:22:48.958951 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5013> 2014-08-13 17:22:48.958963 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.958745, event: throttled, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5012> 2014-08-13 17:22:48.958965 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb3fc0 con 0x45d48e0 -5011> 2014-08-13 17:22:48.958975 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.958884, event: all_read, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5010> 2014-08-13 17:22:48.958984 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.958945, event: dispatched, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5009> 2014-08-13 17:22:48.958979 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(19) epoch 194470) v5 -- ?+0 0x57d4980 con 0x486a3c0 -5008> 2014-08-13 17:22:48.958992 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.958992, event: waiting_for_osdmap, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5007> 2014-08-13 17:22:48.959005 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -5006> 2014-08-13 17:22:48.959007 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.959006, event: started, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5005> 2014-08-13 17:22:48.959011 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb32a0 con 0x45d48e0 -5004> 2014-08-13 17:22:48.959028 7f74d8735700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:22:48.959027, event: done, request: pg_log(22.3d9 epoch 194470 log log((194215'511,194215'511], crt=194215'511) query_epoch 194470) v4 -5003> 2014-08-13 17:22:48.959077 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.034056 7 0.000246 -5002> 2014-08-13 17:22:48.959084 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.015791 8 0.000285 -5001> 2014-08-13 17:22:48.959114 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog -5000> 2014-08-13 17:22:48.959107 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4999> 2014-08-13 17:22:48.959212 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000105 0 0.000000 -4998> 2014-08-13 17:22:48.959238 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4997> 2014-08-13 17:22:48.959245 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000131 0 0.000000 -4996> 2014-08-13 17:22:48.959263 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000025 0 0.000000 -4995> 2014-08-13 17:22:48.959273 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.016006 0 0.000000 -4994> 2014-08-13 17:22:48.959287 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4993> 2014-08-13 17:22:48.959298 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.016055 0 0.000000 -4992> 2014-08-13 17:22:48.959322 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary -4991> 2014-08-13 17:22:48.959343 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange -4990> 2014-08-13 17:22:48.959388 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.019067 6 0.000306 -4989> 2014-08-13 17:22:48.959414 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4988> 2014-08-13 17:22:48.959411 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.018201 10 0.000253 -4987> 2014-08-13 17:22:48.959444 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4986> 2014-08-13 17:22:48.959472 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000059 0 0.000000 -4985> 2014-08-13 17:22:48.959497 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4984> 2014-08-13 17:22:48.959510 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000065 0 0.000000 -4983> 2014-08-13 17:22:48.959523 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000026 0 0.000000 -4982> 2014-08-13 17:22:48.959534 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4981> 2014-08-13 17:22:48.959546 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4980> 2014-08-13 17:22:48.959608 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.020721 7 0.000257 -4979> 2014-08-13 17:22:48.959636 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4978> 2014-08-13 17:22:48.959657 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetInfo 0.014309 8 0.000294 -4977> 2014-08-13 17:22:48.959689 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetLog -4976> 2014-08-13 17:22:48.959691 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000055 0 0.000000 -4975> 2014-08-13 17:22:48.959714 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4974> 2014-08-13 17:22:48.959737 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -4973> 2014-08-13 17:22:48.959759 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4972> 2014-08-13 17:22:48.959760 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetLog 0.000071 0 0.000000 -4971> 2014-08-13 17:22:48.959783 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering 0.014462 0 0.000000 -4970> 2014-08-13 17:22:48.959807 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.014505 0 0.000000 -4969> 2014-08-13 17:22:48.959814 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.019065 8 0.000253 -4968> 2014-08-13 17:22:48.959830 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary -4967> 2014-08-13 17:22:48.959842 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4966> 2014-08-13 17:22:48.959851 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering/WaitActingChange -4965> 2014-08-13 17:22:48.959888 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetInfo 0.013829 7 0.000211 -4964> 2014-08-13 17:22:48.959897 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000056 0 0.000000 -4963> 2014-08-13 17:22:48.959914 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetLog -4962> 2014-08-13 17:22:48.959919 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4961> 2014-08-13 17:22:48.959938 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 -4960> 2014-08-13 17:22:48.959960 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4959> 2014-08-13 17:22:48.959977 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetLog 0.000062 0 0.000000 -4958> 2014-08-13 17:22:48.960001 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering 0.013957 0 0.000000 -4957> 2014-08-13 17:22:48.960013 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.035217 7 0.000294 -4956> 2014-08-13 17:22:48.960024 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.013996 0 0.000000 -4955> 2014-08-13 17:22:48.960043 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary -4954> 2014-08-13 17:22:48.960043 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4953> 2014-08-13 17:22:48.960068 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering/WaitActingChange -4952> 2014-08-13 17:22:48.960117 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000074 0 0.000000 -4951> 2014-08-13 17:22:48.960143 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4950> 2014-08-13 17:22:48.960149 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(22) epoch 194470) v5 -- ?+0 0x4ac0c40 con 0x486bc80 -4949> 2014-08-13 17:22:48.960174 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4948> 2014-08-13 17:22:48.960165 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000022 0 0.000000 -4947> 2014-08-13 17:22:48.960184 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb41a0 con 0x45d48e0 -4946> 2014-08-13 17:22:48.960197 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4945> 2014-08-13 17:22:48.960314 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6814/23703 -- pg_query(22.481 epoch 194470) v3 -- ?+0 0x4eb14a0 con 0x4903440 -4944> 2014-08-13 17:22:48.960322 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.018043 8 0.000337 -4943> 2014-08-13 17:22:48.960348 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4942> 2014-08-13 17:22:48.960353 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4941> 2014-08-13 17:22:48.960355 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb1860 con 0x45d48e0 -4940> 2014-08-13 17:22:48.960417 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000064 0 0.000000 -4939> 2014-08-13 17:22:48.960429 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4938> 2014-08-13 17:22:48.960438 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4937> 2014-08-13 17:22:48.960438 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb1e00 con 0x45d48e0 -4936> 2014-08-13 17:22:48.960466 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 -4935> 2014-08-13 17:22:48.960488 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4934> 2014-08-13 17:22:48.960582 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetInfo 0.012879 9 0.000269 -4933> 2014-08-13 17:22:48.960613 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetLog -4932> 2014-08-13 17:22:48.960683 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetLog 0.000070 0 0.000000 -4931> 2014-08-13 17:22:48.960706 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering 0.013021 0 0.000000 -4930> 2014-08-13 17:22:48.960729 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.013064 0 0.000000 -4929> 2014-08-13 17:22:48.960752 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary -4928> 2014-08-13 17:22:48.960771 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering/WaitActingChange -4927> 2014-08-13 17:22:48.960872 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.003834 6 0.000199 -4926> 2014-08-13 17:22:48.960899 7f74d0f29700 5 osd.4 pg_epoch: 194470 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4925> 2014-08-13 17:22:48.960934 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4924> 2014-08-13 17:22:48.960947 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c5460 con 0x45d48e0 -4923> 2014-08-13 17:22:48.961785 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 4 ==== pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 ==== 4530+0+0 (261547847 0 0) 0x278ce00 con 0x4903440 -4922> 2014-08-13 17:22:48.961832 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961526, event: header_read, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4921> 2014-08-13 17:22:48.961849 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961528, event: throttled, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4920> 2014-08-13 17:22:48.961861 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961704, event: all_read, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4919> 2014-08-13 17:22:48.961872 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961826, event: dispatched, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4918> 2014-08-13 17:22:48.961885 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961884, event: waiting_for_osdmap, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4917> 2014-08-13 17:22:48.961899 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961899, event: started, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4916> 2014-08-13 17:22:48.961930 7f74d8735700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:22:48.961930, event: done, request: pg_log(22.481 epoch 194470 log log((194215'519,194215'519], crt=194215'519) query_epoch 194470) v4 -4915> 2014-08-13 17:22:48.961962 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.002428 3 0.000167 -4914> 2014-08-13 17:22:48.961993 7f74d192a700 5 osd.4 pg_epoch: 194470 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4913> 2014-08-13 17:22:48.962026 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4912> 2014-08-13 17:22:48.962037 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4b00 con 0x45d48e0 -4911> 2014-08-13 17:22:48.964535 7f74a26af700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c2f80 sd=432 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d61760).accept sd=432 192.168.1.106:51891/0 -4910> 2014-08-13 17:22:48.964618 7f74a25ae700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3e80 sd=433 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d627e0).accept sd=433 192.168.100.106:42366/0 -4909> 2014-08-13 17:22:48.967139 7f74a21aa700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c0f00 sd=435 :6817 s=0 pgs=0 cs=0 l=0 c=0x4d614a0).accept sd=435 192.168.1.111:35948/0 -4908> 2014-08-13 17:22:48.967194 7f74a22ab700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3c00 sd=434 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d62520).accept sd=434 192.168.100.111:34378/0 -4907> 2014-08-13 17:22:49.007427 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.328 192.168.100.104:0/64303 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.007541) v2 ==== 47+0+0 (2097905792 0 0) 0x4ac6ac0 con 0x4d509a0 -4906> 2014-08-13 17:22:49.007526 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.007541) v2 -- ?+0 0x49e6ac0 con 0x4d509a0 -4905> 2014-08-13 17:22:49.007579 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.328 192.168.100.104:0/64303 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.007541) v2 ==== 47+0+0 (2097905792 0 0) 0x49e7000 con 0x4907220 -4904> 2014-08-13 17:22:49.007634 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.104:0/64303 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.007541) v2 -- ?+0 0x49e1880 con 0x4907220 -4903> 2014-08-13 17:22:49.013737 7f74a1da6700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c0c80 sd=437 :6817 s=0 pgs=0 cs=0 l=0 c=0x58a7d20).accept sd=437 192.168.1.104:45651/0 -4902> 2014-08-13 17:22:49.013728 7f74a1ea7700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3980 sd=436 :6806 s=0 pgs=0 cs=0 l=0 c=0x4d62260).accept sd=436 192.168.100.104:53322/0 -4901> 2014-08-13 17:22:49.040926 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4900> 2014-08-13 17:22:49.040949 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c1680 con 0x45d48e0 -4899> 2014-08-13 17:22:49.064490 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4898> 2014-08-13 17:22:49.064519 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb2d00 con 0x45d48e0 -4897> 2014-08-13 17:22:49.065496 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4896> 2014-08-13 17:22:49.065523 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c0000 con 0x45d48e0 -4895> 2014-08-13 17:22:49.066078 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4894> 2014-08-13 17:22:49.066105 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c1fe0 con 0x45d48e0 -4893> 2014-08-13 17:22:49.066554 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4892> 2014-08-13 17:22:49.066583 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c7da0 con 0x45d48e0 -4891> 2014-08-13 17:22:49.067266 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4890> 2014-08-13 17:22:49.067295 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c6540 con 0x45d48e0 -4889> 2014-08-13 17:22:49.067753 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4888> 2014-08-13 17:22:49.067780 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c7260 con 0x45d48e0 -4887> 2014-08-13 17:22:49.068195 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4886> 2014-08-13 17:22:49.068221 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c2580 con 0x45d48e0 -4885> 2014-08-13 17:22:49.068780 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4884> 2014-08-13 17:22:49.068799 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c2b20 con 0x45d48e0 -4883> 2014-08-13 17:22:49.069196 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4882> 2014-08-13 17:22:49.069212 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c5280 con 0x45d48e0 -4881> 2014-08-13 17:22:49.069427 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4880> 2014-08-13 17:22:49.069445 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb2940 con 0x45d48e0 -4879> 2014-08-13 17:22:49.069886 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4878> 2014-08-13 17:22:49.069900 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c50a0 con 0x45d48e0 -4877> 2014-08-13 17:22:49.070522 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4876> 2014-08-13 17:22:49.070537 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb2d00 con 0x45d48e0 -4875> 2014-08-13 17:22:49.070951 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4874> 2014-08-13 17:22:49.070964 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c6180 con 0x45d48e0 -4873> 2014-08-13 17:22:49.071540 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4872> 2014-08-13 17:22:49.071555 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4b00 con 0x45d48e0 -4871> 2014-08-13 17:22:49.084742 7f74a19a2700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3700 sd=457 :6806 s=0 pgs=0 cs=0 l=0 c=0x58c4200).accept sd=457 192.168.100.101:53721/0 -4870> 2014-08-13 17:22:49.084763 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4869> 2014-08-13 17:22:49.084803 7f74a1aa3700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c0a00 sd=456 :6817 s=0 pgs=0 cs=0 l=0 c=0x58a7a60).accept sd=456 192.168.1.101:40779/0 -4868> 2014-08-13 17:22:49.084820 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb5820 con 0x45d48e0 -4867> 2014-08-13 17:22:49.084971 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4866> 2014-08-13 17:22:49.084997 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c5460 con 0x45d48e0 -4865> 2014-08-13 17:22:49.085540 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4864> 2014-08-13 17:22:49.085566 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb5640 con 0x45d48e0 -4863> 2014-08-13 17:22:49.085672 7f74a169f700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3480 sd=459 :6806 s=0 pgs=0 cs=0 l=0 c=0x58c3f40).accept sd=459 192.168.100.103:47130/0 -4862> 2014-08-13 17:22:49.085726 7f74a159e700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c0780 sd=460 :6817 s=0 pgs=0 cs=0 l=0 c=0x58a77a0).accept sd=460 192.168.1.103:34969/0 -4861> 2014-08-13 17:22:49.086415 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4860> 2014-08-13 17:22:49.086440 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c1fe0 con 0x45d48e0 -4859> 2014-08-13 17:22:49.087042 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4858> 2014-08-13 17:22:49.087068 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb5460 con 0x45d48e0 -4857> 2014-08-13 17:22:49.087632 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4856> 2014-08-13 17:22:49.087658 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c0000 con 0x45d48e0 -4855> 2014-08-13 17:22:49.087797 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4854> 2014-08-13 17:22:49.087824 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb5280 con 0x45d48e0 -4853> 2014-08-13 17:22:49.087876 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4852> 2014-08-13 17:22:49.087886 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb50a0 con 0x45d48e0 -4851> 2014-08-13 17:22:49.088162 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4850> 2014-08-13 17:22:49.088200 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c1680 con 0x45d48e0 -4849> 2014-08-13 17:22:49.088951 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4848> 2014-08-13 17:22:49.088972 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4ec0 con 0x45d48e0 -4847> 2014-08-13 17:22:49.089113 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4846> 2014-08-13 17:22:49.089129 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c5280 con 0x45d48e0 -4845> 2014-08-13 17:22:49.089328 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4844> 2014-08-13 17:22:49.089346 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb3a20 con 0x45d48e0 -4843> 2014-08-13 17:22:49.089685 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4842> 2014-08-13 17:22:49.089697 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c2b20 con 0x45d48e0 -4841> 2014-08-13 17:22:49.090084 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4840> 2014-08-13 17:22:49.090096 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb32a0 con 0x45d48e0 -4839> 2014-08-13 17:22:49.091491 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4838> 2014-08-13 17:22:49.091502 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c2580 con 0x45d48e0 -4837> 2014-08-13 17:22:49.092332 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4836> 2014-08-13 17:22:49.092356 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb3fc0 con 0x45d48e0 -4835> 2014-08-13 17:22:49.092557 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4834> 2014-08-13 17:22:49.092569 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c7260 con 0x45d48e0 -4833> 2014-08-13 17:22:49.092936 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4832> 2014-08-13 17:22:49.092961 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4380 con 0x45d48e0 -4831> 2014-08-13 17:22:49.094489 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4830> 2014-08-13 17:22:49.094514 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c6540 con 0x45d48e0 -4829> 2014-08-13 17:22:49.095140 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4828> 2014-08-13 17:22:49.095178 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb1e00 con 0x45d48e0 -4827> 2014-08-13 17:22:49.095297 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4826> 2014-08-13 17:22:49.095309 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb1860 con 0x45d48e0 -4825> 2014-08-13 17:22:49.095339 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4824> 2014-08-13 17:22:49.095363 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x29c7da0 con 0x45d48e0 -4823> 2014-08-13 17:22:49.095455 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4822> 2014-08-13 17:22:49.095466 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb41a0 con 0x45d48e0 -4821> 2014-08-13 17:22:49.095573 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4820> 2014-08-13 17:22:49.095585 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb14a0 con 0x45d48e0 -4819> 2014-08-13 17:22:49.095684 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4818> 2014-08-13 17:22:49.095697 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb4380 con 0x45d48e0 -4817> 2014-08-13 17:22:49.095762 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4816> 2014-08-13 17:22:49.095784 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb6180 con 0x45d48e0 -4815> 2014-08-13 17:22:49.095836 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4814> 2014-08-13 17:22:49.095849 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194470 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} v194470) v1 -- ?+0 0x4eb3fc0 con 0x45d48e0 -4813> 2014-08-13 17:22:49.172699 7f74a129b700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c3200 sd=461 :6806 s=0 pgs=0 cs=0 l=0 c=0x58c3c80).accept sd=461 192.168.100.104:53324/0 -4812> 2014-08-13 17:22:49.172758 7f74a119a700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c0500 sd=462 :6817 s=0 pgs=0 cs=0 l=0 c=0x58a74e0).accept sd=462 192.168.1.104:45653/0 -4811> 2014-08-13 17:22:49.184834 7f74a0e97700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c5500 sd=463 :6806 s=0 pgs=0 cs=0 l=0 c=0x58c39c0).accept sd=463 192.168.100.107:52438/0 -4810> 2014-08-13 17:22:49.184914 7f74a0d96700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6680 sd=464 :6817 s=0 pgs=0 cs=0 l=0 c=0x58a7220).accept sd=464 192.168.1.107:57316/0 -4809> 2014-08-13 17:22:49.230490 7f74a0a93700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c5280 sd=465 :6806 s=0 pgs=0 cs=0 l=0 c=0x58c3700).accept sd=465 192.168.100.113:56270/0 -4808> 2014-08-13 17:22:49.230535 7f74a0992700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6400 sd=466 :6817 s=0 pgs=0 cs=0 l=0 c=0x5698dc0).accept sd=466 192.168.1.113:49181/0 -4807> 2014-08-13 17:22:49.259781 7f74a068f700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c5000 sd=467 :6806 s=0 pgs=0 cs=0 l=0 c=0x5699e40).accept sd=467 192.168.100.107:52442/0 -4806> 2014-08-13 17:22:49.260016 7f74a058e700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6180 sd=468 :6817 s=0 pgs=0 cs=0 l=0 c=0x5698b00).accept sd=468 192.168.1.107:57320/0 -4805> 2014-08-13 17:22:49.285693 7f74a028b700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c5f00 sd=469 :6817 s=0 pgs=0 cs=0 l=0 c=0x5698840).accept sd=469 192.168.1.108:49302/0 -4804> 2014-08-13 17:22:49.285762 7f74a018a700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c4d80 sd=470 :6806 s=0 pgs=0 cs=0 l=0 c=0x5699b80).accept sd=470 192.168.100.108:58069/0 -4803> 2014-08-13 17:22:49.346845 7f749fe87700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c4b00 sd=471 :6806 s=0 pgs=0 cs=0 l=0 c=0x56998c0).accept sd=471 192.168.100.108:58071/0 -4802> 2014-08-13 17:22:49.346887 7f749fd86700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c5c80 sd=472 :6817 s=0 pgs=0 cs=0 l=0 c=0x5698580).accept sd=472 192.168.1.108:49304/0 -4801> 2014-08-13 17:22:49.373395 7f749fa83700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c4880 sd=473 :6806 s=0 pgs=0 cs=0 l=0 c=0x5699600).accept sd=473 192.168.100.110:59516/0 -4800> 2014-08-13 17:22:49.373443 7f749f982700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c5a00 sd=474 :6817 s=0 pgs=0 cs=0 l=0 c=0x56982c0).accept sd=474 192.168.1.110:50477/0 -4799> 2014-08-13 17:22:49.422136 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.324 192.168.100.104:0/65209 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.422217) v2 ==== 47+0+0 (3925690929 0 0) 0x2940380 con 0x58a74e0 -4798> 2014-08-13 17:22:49.422178 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.324 192.168.100.104:0/65209 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.422217) v2 ==== 47+0+0 (3925690929 0 0) 0x5263800 con 0x58c3c80 -4797> 2014-08-13 17:22:49.422205 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.104:0/65209 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.422217) v2 -- ?+0 0x4ac6ac0 con 0x58a74e0 -4796> 2014-08-13 17:22:49.422412 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.104:0/65209 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.422217) v2 -- ?+0 0x49e7000 con 0x58c3c80 -4795> 2014-08-13 17:22:49.424626 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.351 192.168.100.102:0/16444 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.424513) v2 ==== 47+0+0 (3529182831 0 0) 0x52647c0 con 0x4d52680 -4794> 2014-08-13 17:22:49.424651 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.351 192.168.100.102:0/16444 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.424513) v2 ==== 47+0+0 (3529182831 0 0) 0x5263480 con 0x4d50420 -4793> 2014-08-13 17:22:49.424675 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.424513) v2 -- ?+0 0x5263800 con 0x4d52680 -4792> 2014-08-13 17:22:49.424854 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.424513) v2 -- ?+0 0x2940380 con 0x4d50420 -4791> 2014-08-13 17:22:49.463422 7f749f67f700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c5780 sd=475 :6817 s=0 pgs=0 cs=0 l=0 c=0x569b860).accept sd=475 192.168.1.102:57884/0 -4790> 2014-08-13 17:22:49.463489 7f749f57e700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c4600 sd=476 :6806 s=0 pgs=0 cs=0 l=0 c=0x5699340).accept sd=476 192.168.100.102:36426/0 -4789> 2014-08-13 17:22:49.468914 7f749f27b700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c7a80 sd=477 :6817 s=0 pgs=0 cs=0 l=0 c=0x569b5a0).accept sd=477 192.168.1.107:57325/0 -4788> 2014-08-13 17:22:49.468944 7f749f17a700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x57c7d00 sd=478 :6806 s=0 pgs=0 cs=0 l=0 c=0x569ca40).accept sd=478 192.168.100.107:52447/0 -4787> 2014-08-13 17:22:49.495146 7f749ee77700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5679400 sd=479 :6806 s=0 pgs=0 cs=0 l=0 c=0x569c780).accept sd=479 192.168.100.104:53326/0 -4786> 2014-08-13 17:22:49.495196 7f749ed76700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c7800 sd=480 :6817 s=0 pgs=0 cs=0 l=0 c=0x569b2e0).accept sd=480 192.168.1.104:45655/0 -4785> 2014-08-13 17:22:49.519097 7f749ea73700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c7580 sd=481 :6817 s=0 pgs=0 cs=0 l=0 c=0x569b020).accept sd=481 192.168.1.105:48382/0 -4784> 2014-08-13 17:22:49.519214 7f749e972700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5679180 sd=482 :6806 s=0 pgs=0 cs=0 l=0 c=0x569c4c0).accept sd=482 192.168.100.105:39000/0 -4783> 2014-08-13 17:22:49.536003 7f749e66f700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c7300 sd=483 :6817 s=0 pgs=0 cs=0 l=0 c=0x569ad60).accept sd=483 192.168.1.114:42655/0 -4782> 2014-08-13 17:22:49.536035 7f749e56e700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678f00 sd=484 :6806 s=0 pgs=0 cs=0 l=0 c=0x569c200).accept sd=484 192.168.100.114:59231/0 -4781> 2014-08-13 17:22:49.536120 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.535702) v2 ==== 47+0+0 (2949161587 0 0) 0x5263b80 con 0x58a7a60 -4780> 2014-08-13 17:22:49.536166 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.535702) v2 -- ?+0 0x5263480 con 0x58a7a60 -4779> 2014-08-13 17:22:49.536154 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.535702) v2 ==== 47+0+0 (2949161587 0 0) 0x4ac1c00 con 0x58c4200 -4778> 2014-08-13 17:22:49.536213 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.535702) v2 -- ?+0 0x52647c0 con 0x58c4200 -4777> 2014-08-13 17:22:49.536698 7f749e36c700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678c80 sd=485 :6806 s=0 pgs=0 cs=0 l=0 c=0x569bf40).accept sd=485 192.168.100.105:39002/0 -4776> 2014-08-13 17:22:49.536783 7f749e26b700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c7080 sd=486 :6817 s=0 pgs=0 cs=0 l=0 c=0x569e880).accept sd=486 192.168.1.105:48384/0 -4775> 2014-08-13 17:22:49.551150 7f749de67700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678a00 sd=487 :6806 s=0 pgs=0 cs=0 l=0 c=0x569fd20).accept sd=487 192.168.100.113:56272/0 -4774> 2014-08-13 17:22:49.551203 7f749dd66700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6e00 sd=488 :6817 s=0 pgs=0 cs=0 l=0 c=0x569e5c0).accept sd=488 192.168.1.113:49183/0 -4773> 2014-08-13 17:22:49.560738 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.560528) v2 ==== 47+0+0 (3799404316 0 0) 0x4ac2300 con 0x569c200 -4772> 2014-08-13 17:22:49.560790 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.560528) v2 -- ?+0 0x4ac1c00 con 0x569c200 -4771> 2014-08-13 17:22:49.560791 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.560528) v2 ==== 47+0+0 (3799404316 0 0) 0x4ac2140 con 0x569ad60 -4770> 2014-08-13 17:22:49.560834 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.560528) v2 -- ?+0 0x5263b80 con 0x569ad60 -4769> 2014-08-13 17:22:49.589554 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.54 192.168.100.111:0/14923 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.589024) v2 ==== 47+0+0 (4009008682 0 0) 0x4ac7540 con 0x4d62520 -4768> 2014-08-13 17:22:49.589602 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.589024) v2 -- ?+0 0x4ac2300 con 0x4d62520 -4767> 2014-08-13 17:22:49.589578 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.54 192.168.100.111:0/14923 1 ==== osd_ping(ping e194470 stamp 2014-08-13 17:22:49.589024) v2 ==== 47+0+0 (4009008682 0 0) 0x4ac0380 con 0x4d614a0 -4766> 2014-08-13 17:22:49.589802 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.111:0/14923 -- osd_ping(ping_reply e194470 stamp 2014-08-13 17:22:49.589024) v2 -- ?+0 0x4ac2140 con 0x4d614a0 -4765> 2014-08-13 17:22:49.629469 7f749da63700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6b80 sd=490 :6817 s=0 pgs=0 cs=0 l=0 c=0x569e300).accept sd=490 192.168.1.101:40781/0 -4764> 2014-08-13 17:22:49.629522 7f749d962700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678780 sd=489 :6806 s=0 pgs=0 cs=0 l=0 c=0x569fa60).accept sd=489 192.168.100.101:53723/0 -4763> 2014-08-13 17:22:49.648699 7f749d65f700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x57c6900 sd=491 :6817 s=0 pgs=0 cs=0 l=0 c=0x569e040).accept sd=491 192.168.1.106:51896/0 -4762> 2014-08-13 17:22:49.648781 7f749d55e700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678500 sd=492 :6806 s=0 pgs=0 cs=0 l=0 c=0x569f7a0).accept sd=492 192.168.100.106:42369/0 -4761> 2014-08-13 17:22:49.722143 7f749d15a700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678280 sd=494 :6806 s=0 pgs=0 cs=0 l=0 c=0x569f4e0).accept sd=494 192.168.100.114:59233/0 -4760> 2014-08-13 17:22:49.722270 7f749d25b700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x567aa80 sd=493 :6817 s=0 pgs=0 cs=0 l=0 c=0x569dd80).accept sd=493 192.168.1.114:42657/0 -4759> 2014-08-13 17:22:49.785263 7f749ce57700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x567a800 sd=495 :6817 s=0 pgs=0 cs=0 l=0 c=0x569dac0).accept sd=495 192.168.1.103:34975/0 -4758> 2014-08-13 17:22:49.785346 7f749cd56700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x5678000 sd=496 :6806 s=0 pgs=0 cs=0 l=0 c=0x569f220).accept sd=496 192.168.100.103:47136/0 -4757> 2014-08-13 17:22:49.792873 7f749c952700 1 -- 192.168.100.115:6806/30165 >> :/0 pipe(0x567c380 sd=498 :6806 s=0 pgs=0 cs=0 l=0 c=0x569ef60).accept sd=498 192.168.100.104:53328/0 -4756> 2014-08-13 17:22:49.792930 7f749ca53700 1 -- 192.168.1.115:6817/30165 >> :/0 pipe(0x567a580 sd=497 :6817 s=0 pgs=0 cs=0 l=0 c=0x5641ce0).accept sd=497 192.168.1.104:45657/0 -4755> 2014-08-13 17:22:49.796245 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 16 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c75e80 con 0x45d48e0 -4754> 2014-08-13 17:22:49.796253 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a845c0 con 0x47f4620 -4753> 2014-08-13 17:22:49.796371 7f74d9b37700 3 osd.4 194470 handle_osd_map epochs [194471,194471], i have 194470, src has [185139,194471] -4752> 2014-08-13 17:22:49.802122 7f74d9b37700 0 osd.4 194471 crush map has features 2578087936000, adjusting msgr requires for mons -4751> 2014-08-13 17:22:49.802305 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary/Peering/WaitActingChange 0.845464 3 0.000248 -4750> 2014-08-13 17:22:49.802347 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.845523 0 0.000000 -4749> 2014-08-13 17:22:49.802370 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started 0.888125 0 0.000000 -4748> 2014-08-13 17:22:49.802390 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/194470) [4,286,375] r=0 lpr=194470 pi=193640-194469/25 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset -4747> 2014-08-13 17:22:49.802488 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Reset 0.000098 1 0.000182 -4746> 2014-08-13 17:22:49.802520 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started -4745> 2014-08-13 17:22:49.802542 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Start -4744> 2014-08-13 17:22:49.802563 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: transitioning to Primary -4743> 2014-08-13 17:22:49.802587 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Start 0.000045 0 0.000000 -4742> 2014-08-13 17:22:49.802612 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary -4741> 2014-08-13 17:22:49.802630 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Peering -4740> 2014-08-13 17:22:49.802647 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetInfo -4739> 2014-08-13 17:22:49.802799 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4738> 2014-08-13 17:22:49.802815 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_alive(want up_thru 194471 have 194471) v1 -- ?+0 0x4ac01c0 con 0x45d48e0 -4737> 2014-08-13 17:22:49.802840 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6815/60573 -- pg_query(24.30b epoch 194471) v3 -- ?+0 0x4eb5fa0 con 0x45f1760 -4736> 2014-08-13 17:22:49.802860 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6820/19173 -- pg_query(24.30b epoch 194471) v3 -- ?+0 0x4eb5dc0 con 0x48dfe80 -4735> 2014-08-13 17:22:49.802879 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6814/16727 -- pg_query(24.30b epoch 194471) v3 -- ?+0 0x4eb5be0 con 0x45f5540 -4734> 2014-08-13 17:22:49.802898 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6830/52879 -- pg_query(24.30b epoch 194471) v3 -- ?+0 0x4eb5a00 con 0x49023c0 -4733> 2014-08-13 17:22:49.802919 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4732> 2014-08-13 17:22:49.802925 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194471 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} v194471) v1 -- ?+0 0x4eb5820 con 0x45d48e0 -4731> 2014-08-13 17:22:49.803116 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x2713f00 con 0x4868420 -4730> 2014-08-13 17:22:49.803142 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(22) epoch 194471) v5 -- ?+0 0x57d7a80 con 0x4868420 -4729> 2014-08-13 17:22:49.803158 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -4728> 2014-08-13 17:22:49.803173 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194471 {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} v194471) v1 -- ?+0 0x4eb32a0 con 0x45d48e0 -4727> 2014-08-13 17:22:49.807613 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.849220 4 0.000194 -4726> 2014-08-13 17:22:49.807661 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.890362 0 0.000000 -4725> 2014-08-13 17:22:49.807736 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.851011 6 0.000234 -4724> 2014-08-13 17:22:49.807789 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.888033 0 0.000000 -4723> 2014-08-13 17:22:49.823837 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4722> 2014-08-13 17:22:49.823868 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4721> 2014-08-13 17:22:49.828402 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.58 192.168.1.111:6834/14316 -- pg_log(22.362 epoch 194471 log log((0'0,194215'479], crt=191727'415) query_epoch 194471) v4 -- ?+0 0x278ce00 -4720> 2014-08-13 17:22:49.828470 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.31 192.168.1.103:6831/41157 -- pg_log(22.99 epoch 194471 log log((0'0,194215'504], crt=191648'365) query_epoch 194471) v4 -- ?+0 0x278c800 -4719> 2014-08-13 17:22:49.828777 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.343 192.168.1.104:6802/61960 -- pg_log(22.362 epoch 194471 log log((0'0,194215'479], crt=191727'415) query_epoch 194471) v4 -- ?+0 0x278f200 -4718> 2014-08-13 17:22:49.828830 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.374 192.168.1.102:6828/16129 -- pg_log(22.99 epoch 194471 log log((0'0,194215'504], crt=191648'365) query_epoch 194471) v4 -- ?+0 0x278f800 -4717> 2014-08-13 17:22:49.829017 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4716> 2014-08-13 17:22:49.829102 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4715> 2014-08-13 17:22:49.829431 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.872944 5 0.000278 -4714> 2014-08-13 17:22:49.829446 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.874039 6 0.000222 -4713> 2014-08-13 17:22:49.829489 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.908700 0 0.000000 -4712> 2014-08-13 17:22:49.829491 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.909268 0 0.000000 -4711> 2014-08-13 17:22:49.829525 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4710> 2014-08-13 17:22:49.829540 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194449 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4709> 2014-08-13 17:22:49.829581 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194465/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4708> 2014-08-13 17:22:49.829728 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.871113 3 0.000194 -4707> 2014-08-13 17:22:49.829764 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.907265 0 0.000000 -4706> 2014-08-13 17:22:49.829791 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4705> 2014-08-13 17:22:49.830400 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d3e460 -- 0x577e680 -4704> 2014-08-13 17:22:49.830391 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.301 192.168.1.113:6814/16727 -- pg_log(22.22c epoch 194471 log log((0'0,194215'514], crt=191727'446) query_epoch 194471) v4 -- ?+0 0x5b18600 -4703> 2014-08-13 17:22:49.830462 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d32ec0 -- 0x577e400 -4702> 2014-08-13 17:22:49.830493 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d32c00 -- 0x577e180 -4701> 2014-08-13 17:22:49.830526 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d32940 -- 0x577df00 -4700> 2014-08-13 17:22:49.830459 7f74a63ec700 2 -- 192.168.1.115:0/30165 >> 192.168.1.109:6807/31286 pipe(0x577e680 sd=386 :36072 s=4 pgs=105 cs=1 l=1 c=0x4d3e460).reader couldn't read tag, (0) Success -4699> 2014-08-13 17:22:49.830540 7f74a63ec700 2 -- 192.168.1.115:0/30165 >> 192.168.1.109:6807/31286 pipe(0x577e680 sd=386 :36072 s=4 pgs=105 cs=1 l=1 c=0x4d3e460).fault (0) Success -4698> 2014-08-13 17:22:49.830596 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4697> 2014-08-13 17:22:49.830526 7f74a5ee7700 2 -- 192.168.1.115:0/30165 >> 192.168.1.109:6805/31016 pipe(0x577e180 sd=388 :56358 s=4 pgs=124 cs=1 l=1 c=0x4d32c00).reader couldn't read tag, (0) Success -4696> 2014-08-13 17:22:49.830611 7f74a5ee7700 2 -- 192.168.1.115:0/30165 >> 192.168.1.109:6805/31016 pipe(0x577e180 sd=388 :56358 s=4 pgs=124 cs=1 l=1 c=0x4d32c00).fault (0) Success -4695> 2014-08-13 17:22:49.830619 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.356 192.168.1.108:6832/5235 -- pg_log(22.4df epoch 194471 log log((0'0,194215'497], crt=191727'433) query_epoch 194471) v4 -- ?+0 0x5b19200 -4694> 2014-08-13 17:22:49.830623 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 5 ==== pg_query(24.1e4 epoch 194471) v3 ==== 144+0+0 (263292648 0 0) 0x29c7800 con 0x47f4620 -4693> 2014-08-13 17:22:49.830644 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.796227, event: header_read, request: pg_query(24.1e4 epoch 194471) v3 -4692> 2014-08-13 17:22:49.830656 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.796228, event: throttled, request: pg_query(24.1e4 epoch 194471) v3 -4691> 2014-08-13 17:22:49.830541 7f74a61ea700 2 -- 192.168.1.115:0/30165 >> 192.168.100.109:6815/31286 pipe(0x577e400 sd=387 :48155 s=4 pgs=105 cs=1 l=1 c=0x4d32ec0).reader couldn't read tag, (0) Success -4690> 2014-08-13 17:22:49.830662 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.796280, event: all_read, request: pg_query(24.1e4 epoch 194471) v3 -4689> 2014-08-13 17:22:49.830668 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.830641, event: dispatched, request: pg_query(24.1e4 epoch 194471) v3 -4688> 2014-08-13 17:22:49.830676 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.830676, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194471) v3 -4687> 2014-08-13 17:22:49.830667 7f74a61ea700 2 -- 192.168.1.115:0/30165 >> 192.168.100.109:6815/31286 pipe(0x577e400 sd=387 :48155 s=4 pgs=105 cs=1 l=1 c=0x4d32ec0).fault (0) Success -4686> 2014-08-13 17:22:49.830685 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.830685, event: started, request: pg_query(24.1e4 epoch 194471) v3 -4685> 2014-08-13 17:22:49.830580 7f74a5ae3700 2 -- 192.168.1.115:0/30165 >> 192.168.100.109:6811/31016 pipe(0x577df00 sd=389 :45779 s=4 pgs=124 cs=1 l=1 c=0x4d32940).reader couldn't read tag, (0) Success -4684> 2014-08-13 17:22:49.830700 7f74a5ae3700 2 -- 192.168.1.115:0/30165 >> 192.168.100.109:6811/31016 pipe(0x577df00 sd=389 :45779 s=4 pgs=124 cs=1 l=1 c=0x4d32940).fault (0) Success -4683> 2014-08-13 17:22:49.830709 7f74d8735700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:22:49.830709, event: done, request: pg_query(24.1e4 epoch 194471) v3 -4682> 2014-08-13 17:22:49.830706 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4681> 2014-08-13 17:22:49.830729 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de0b40 con 0x47f7e80 -4680> 2014-08-13 17:22:49.830748 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4679> 2014-08-13 17:22:49.830757 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c73f00 con 0x4831b80 -4678> 2014-08-13 17:22:49.830775 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4677> 2014-08-13 17:22:49.830785 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4807bc0 con 0x4833440 -4676> 2014-08-13 17:22:49.830800 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4675> 2014-08-13 17:22:49.830808 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c72ac0 con 0x4832c00 -4674> 2014-08-13 17:22:49.830822 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4673> 2014-08-13 17:22:49.830830 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c73a80 con 0x4830dc0 -4672> 2014-08-13 17:22:49.830843 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4671> 2014-08-13 17:22:49.830851 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4801200 con 0x4835ac0 -4670> 2014-08-13 17:22:49.830864 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4669> 2014-08-13 17:22:49.830871 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.32 192.168.1.115:6812/59915 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x48057c0 con 0x4832100 -4668> 2014-08-13 17:22:49.830884 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4667> 2014-08-13 17:22:49.830894 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 10 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a860c0 con 0x4835280 -4666> 2014-08-13 17:22:49.830908 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4665> 2014-08-13 17:22:49.830916 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de4140 con 0x4834620 -4664> 2014-08-13 17:22:49.830929 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4663> 2014-08-13 17:22:49.830936 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a81200 con 0x48377a0 -4662> 2014-08-13 17:22:49.830953 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4661> 2014-08-13 17:22:49.830960 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4800b40 con 0x4868420 -4660> 2014-08-13 17:22:49.830973 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4659> 2014-08-13 17:22:49.830981 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c769c0 con 0x4868160 -4658> 2014-08-13 17:22:49.830994 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4657> 2014-08-13 17:22:49.831003 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x48069c0 con 0x486bf40 -4656> 2014-08-13 17:22:49.831016 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4655> 2014-08-13 17:22:49.831024 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c71200 con 0x486bc80 -4654> 2014-08-13 17:22:49.831041 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4653> 2014-08-13 17:22:49.831048 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e30d80 con 0x486b9c0 -4652> 2014-08-13 17:22:49.831062 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4651> 2014-08-13 17:22:49.831069 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a83840 con 0x486b700 -4650> 2014-08-13 17:22:49.831082 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4649> 2014-08-13 17:22:49.831089 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c75c40 con 0x486b440 -4648> 2014-08-13 17:22:49.831102 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4647> 2014-08-13 17:22:49.831110 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de06c0 con 0x486b180 -4646> 2014-08-13 17:22:49.831123 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4645> 2014-08-13 17:22:49.831131 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a87bc0 con 0x486aec0 -4644> 2014-08-13 17:22:49.831144 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4643> 2014-08-13 17:22:49.831152 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c73600 con 0x486a940 -4642> 2014-08-13 17:22:49.831165 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4641> 2014-08-13 17:22:49.831173 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de3840 con 0x486a680 -4640> 2014-08-13 17:22:49.831197 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4639> 2014-08-13 17:22:49.831181 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.357 192.168.1.102:6834/17224 -- pg_log(22.4df epoch 194471 log log((0'0,194215'497], crt=191727'433) query_epoch 194471) v4 -- ?+0 0x5b18c00 -4638> 2014-08-13 17:22:49.831206 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 8 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c70240 con 0x486a3c0 -4637> 2014-08-13 17:22:49.831219 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4636> 2014-08-13 17:22:49.831253 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a82ac0 con 0x48d9ce0 -4635> 2014-08-13 17:22:49.831267 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4634> 2014-08-13 17:22:49.831247 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.872922 3 0.000214 -4633> 2014-08-13 17:22:49.831275 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a86780 con 0x486e040 -4632> 2014-08-13 17:22:49.831288 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4631> 2014-08-13 17:22:49.831280 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.910065 0 0.000000 -4630> 2014-08-13 17:22:49.831296 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a821c0 con 0x48dfbc0 -4629> 2014-08-13 17:22:49.831309 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4628> 2014-08-13 17:22:49.831311 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194449 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4627> 2014-08-13 17:22:49.831319 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a81d40 con 0x4900580 -4626> 2014-08-13 17:22:49.831334 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4625> 2014-08-13 17:22:49.831341 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c71b00 con 0x4900f20 -4624> 2014-08-13 17:22:49.831354 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4623> 2014-08-13 17:22:49.831361 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 8 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a85340 con 0x4901600 -4622> 2014-08-13 17:22:49.831374 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4621> 2014-08-13 17:22:49.831381 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.19 192.168.1.115:6818/61186 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e33600 con 0x4901340 -4620> 2014-08-13 17:22:49.831394 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4619> 2014-08-13 17:22:49.831401 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e34ec0 con 0x4902680 -4618> 2014-08-13 17:22:49.831414 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4617> 2014-08-13 17:22:49.831428 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c706c0 con 0x49023c0 -4616> 2014-08-13 17:22:49.831441 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4615> 2014-08-13 17:22:49.831449 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e30240 con 0x4902100 -4614> 2014-08-13 17:22:49.831462 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4613> 2014-08-13 17:22:49.831468 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c77bc0 con 0x4903700 -4612> 2014-08-13 17:22:49.831482 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4611> 2014-08-13 17:22:49.831497 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e31200 con 0x4903440 -4610> 2014-08-13 17:22:49.831489 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4609> 2014-08-13 17:22:49.831511 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4608> 2014-08-13 17:22:49.831518 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e31d40 con 0x4903180 -4607> 2014-08-13 17:22:49.831531 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4606> 2014-08-13 17:22:49.831539 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e306c0 con 0x49044c0 -4605> 2014-08-13 17:22:49.831552 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4604> 2014-08-13 17:22:49.831560 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e31680 con 0x49070c0 -4603> 2014-08-13 17:22:49.831574 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4602> 2014-08-13 17:22:49.831569 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.62 192.168.1.112:6824/5505 -- pg_log(22.619 epoch 194471 log log((0'0,194215'511], crt=191727'439) query_epoch 194471) v4 -- ?+0 0x5b18000 -4601> 2014-08-13 17:22:49.831581 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 8 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e34140 con 0x4d58840 -4600> 2014-08-13 17:22:49.831594 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4599> 2014-08-13 17:22:49.831601 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e32d00 con 0x4d582c0 -4598> 2014-08-13 17:22:49.831618 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4597> 2014-08-13 17:22:49.831626 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e36780 con 0x4d51ce0 -4596> 2014-08-13 17:22:49.831639 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4595> 2014-08-13 17:22:49.831657 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e345c0 con 0x4d51a20 -4594> 2014-08-13 17:22:49.831671 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4593> 2014-08-13 17:22:49.831678 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de2880 con 0x4d55d80 -4592> 2014-08-13 17:22:49.831691 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4591> 2014-08-13 17:22:49.831701 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de4c80 con 0x4d57a60 -4590> 2014-08-13 17:22:49.831715 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4589> 2014-08-13 17:22:49.831722 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 1 ==== osd_map(194456..194471 src has 185139..194471) v3 ==== 10429+0+0 (970627098 0 0) 0x4de7980 con 0x4d2aaa0 -4588> 2014-08-13 17:22:49.831734 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194456,194471], i have 194471, src has [185139,194471] -4587> 2014-08-13 17:22:49.831745 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4802d00 con 0x45f1e40 -4586> 2014-08-13 17:22:49.831759 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4585> 2014-08-13 17:22:49.831767 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4805c40 con 0x45f2940 -4584> 2014-08-13 17:22:49.831780 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4583> 2014-08-13 17:22:49.831788 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c71f80 con 0x45f3b20 -4582> 2014-08-13 17:22:49.831801 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4581> 2014-08-13 17:22:49.831775 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.871579 5 0.000242 -4580> 2014-08-13 17:22:49.831808 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x48021c0 con 0x45f5540 -4579> 2014-08-13 17:22:49.831822 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4578> 2014-08-13 17:22:49.831810 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.65 192.168.1.111:6842/17345 -- pg_log(22.619 epoch 194471 log log((0'0,194215'511], crt=191727'439) query_epoch 194471) v4 -- ?+0 0x5b1a400 -4577> 2014-08-13 17:22:49.831814 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.907037 0 0.000000 -4576> 2014-08-13 17:22:49.831829 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4807080 con 0x45f5960 -4575> 2014-08-13 17:22:49.831842 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4574> 2014-08-13 17:22:49.831851 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de1200 con 0x45f6f60 -4573> 2014-08-13 17:22:49.831864 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4572> 2014-08-13 17:22:49.831854 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4571> 2014-08-13 17:22:49.831872 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c76e40 con 0x4758b00 -4570> 2014-08-13 17:22:49.831885 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4569> 2014-08-13 17:22:49.831892 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4803600 con 0x4758840 -4568> 2014-08-13 17:22:49.831905 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4567> 2014-08-13 17:22:49.831912 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a84140 con 0x475a940 -4566> 2014-08-13 17:22:49.831925 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4565> 2014-08-13 17:22:49.831933 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e10480 con 0x475cd00 -4564> 2014-08-13 17:22:49.831946 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4563> 2014-08-13 17:22:49.831954 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4803180 con 0x475c4c0 -4562> 2014-08-13 17:22:49.831967 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4561> 2014-08-13 17:22:49.831974 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c74c80 con 0x475d960 -4560> 2014-08-13 17:22:49.831987 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4559> 2014-08-13 17:22:49.831994 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 10 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c72400 con 0x475ee00 -4558> 2014-08-13 17:22:49.832007 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4557> 2014-08-13 17:22:49.832014 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4804140 con 0x475e5c0 -4556> 2014-08-13 17:22:49.832027 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4555> 2014-08-13 17:22:49.832034 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 7 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a85c40 con 0x47a27e0 -4554> 2014-08-13 17:22:49.832047 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4553> 2014-08-13 17:22:49.832054 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 8 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c74800 con 0x47a0b00 -4552> 2014-08-13 17:22:49.832049 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4551> 2014-08-13 17:22:49.832066 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4550> 2014-08-13 17:22:49.832073 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e10900 con 0x47a4200 -4549> 2014-08-13 17:22:49.832087 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4548> 2014-08-13 17:22:49.832094 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c757c0 con 0x47a53e0 -4547> 2014-08-13 17:22:49.832107 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4546> 2014-08-13 17:22:49.832114 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a87980 con 0x47a4ba0 -4545> 2014-08-13 17:22:49.832127 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4544> 2014-08-13 17:22:49.832117 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.344 192.168.1.106:6812/6757 -- pg_log(22.39d epoch 194471 log log((0'0,194215'480], crt=191727'416) query_epoch 194471) v4 -- ?+0 0x5b1b600 -4543> 2014-08-13 17:22:49.832134 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c70fc0 con 0x47a32e0 -4542> 2014-08-13 17:22:49.832147 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4541> 2014-08-13 17:22:49.832156 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a86c00 con 0x47f0c60 -4540> 2014-08-13 17:22:49.832173 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4539> 2014-08-13 17:22:49.832189 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e33180 con 0x47f2680 -4538> 2014-08-13 17:22:49.832204 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4537> 2014-08-13 17:22:49.832211 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 8 ==== pg_notify(22.113(22) epoch 194471) v5 ==== 1952+0+0 (1311281533 0 0) 0x4ac7a80 con 0x47f7e80 -4536> 2014-08-13 17:22:49.832226 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.811335, event: header_read, request: pg_notify(22.113(22) epoch 194471) v5 -4535> 2014-08-13 17:22:49.832234 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.811336, event: throttled, request: pg_notify(22.113(22) epoch 194471) v5 -4534> 2014-08-13 17:22:49.832240 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.811435, event: all_read, request: pg_notify(22.113(22) epoch 194471) v5 -4533> 2014-08-13 17:22:49.832245 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.832223, event: dispatched, request: pg_notify(22.113(22) epoch 194471) v5 -4532> 2014-08-13 17:22:49.832252 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.832252, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194471) v5 -4531> 2014-08-13 17:22:49.832260 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.832260, event: started, request: pg_notify(22.113(22) epoch 194471) v5 -4530> 2014-08-13 17:22:49.832285 7f74d8735700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:22:49.832285, event: done, request: pg_notify(22.113(22) epoch 194471) v5 -4529> 2014-08-13 17:22:49.832314 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 4 ==== pg_info(1 pgs e194471:22.537(2)) v4 ==== 824+0+0 (3163201612 0 0) 0x49e7380 con 0x4831b80 -4528> 2014-08-13 17:22:49.832333 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.805008, event: header_read, request: pg_info(1 pgs e194471:22.537(2)) v4 -4527> 2014-08-13 17:22:49.832340 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.805009, event: throttled, request: pg_info(1 pgs e194471:22.537(2)) v4 -4526> 2014-08-13 17:22:49.832345 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.805069, event: all_read, request: pg_info(1 pgs e194471:22.537(2)) v4 -4525> 2014-08-13 17:22:49.832351 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.832331, event: dispatched, request: pg_info(1 pgs e194471:22.537(2)) v4 -4524> 2014-08-13 17:22:49.832356 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.832356, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.537(2)) v4 -4523> 2014-08-13 17:22:49.832370 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.832370, event: started, request: pg_info(1 pgs e194471:22.537(2)) v4 -4522> 2014-08-13 17:22:49.832392 7f74d8735700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:22:49.832392, event: done, request: pg_info(1 pgs e194471:22.537(2)) v4 -4521> 2014-08-13 17:22:49.832388 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.874408 3 0.000254 -4520> 2014-08-13 17:22:49.832406 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 8 ==== pg_notify(22.99(18) epoch 194471) v5 ==== 1720+0+0 (139481537 0 0) 0x4ac7000 con 0x4833440 -4519> 2014-08-13 17:22:49.832403 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.373 192.168.1.108:6840/7435 -- pg_log(22.39d epoch 194471 log log((0'0,194215'480], crt=191727'416) query_epoch 194471) v4 -- ?+0 0x5b1b000 -4518> 2014-08-13 17:22:49.832421 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.801966, event: header_read, request: pg_notify(22.99(18) epoch 194471) v5 -4517> 2014-08-13 17:22:49.832429 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.801968, event: throttled, request: pg_notify(22.99(18) epoch 194471) v5 -4516> 2014-08-13 17:22:49.832434 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.802059, event: all_read, request: pg_notify(22.99(18) epoch 194471) v5 -4515> 2014-08-13 17:22:49.832421 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.874465 0 0.000000 -4514> 2014-08-13 17:22:49.832439 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.832419, event: dispatched, request: pg_notify(22.99(18) epoch 194471) v5 -4513> 2014-08-13 17:22:49.832419 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 17 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e31200 con 0x45d48e0 -4512> 2014-08-13 17:22:49.832446 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.832446, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194471) v5 -4511> 2014-08-13 17:22:49.832454 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.832454, event: started, request: pg_notify(22.99(18) epoch 194471) v5 -4510> 2014-08-13 17:22:49.832443 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.909644 0 0.000000 -4509> 2014-08-13 17:22:49.832473 7f74d8735700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:22:49.832473, event: done, request: pg_notify(22.99(18) epoch 194471) v5 -4508> 2014-08-13 17:22:49.832464 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194470/194470) [4,378,367] r=0 lpr=194470 pi=193637-194469/23 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -4507> 2014-08-13 17:22:49.832504 7f74d9b37700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4506> 2014-08-13 17:22:49.832500 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 6 ==== pg_notify(22.4df(20) epoch 194471) v5 ==== 1834+0+0 (4045728566 0 0) 0x57d3d40 con 0x4832c00 -4505> 2014-08-13 17:22:49.832516 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.800639, event: header_read, request: pg_notify(22.4df(20) epoch 194471) v5 -4504> 2014-08-13 17:22:49.832523 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.800640, event: throttled, request: pg_notify(22.4df(20) epoch 194471) v5 -4503> 2014-08-13 17:22:49.832529 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.800732, event: all_read, request: pg_notify(22.4df(20) epoch 194471) v5 -4502> 2014-08-13 17:22:49.832534 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.832514, event: dispatched, request: pg_notify(22.4df(20) epoch 194471) v5 -4501> 2014-08-13 17:22:49.832539 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.832539, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194471) v5 -4500> 2014-08-13 17:22:49.832547 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.832546, event: started, request: pg_notify(22.4df(20) epoch 194471) v5 -4499> 2014-08-13 17:22:49.832566 7f74d8735700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:22:49.832565, event: done, request: pg_notify(22.4df(20) epoch 194471) v5 -4498> 2014-08-13 17:22:49.832557 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.000093 1 0.000166 -4497> 2014-08-13 17:22:49.832583 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 5 ==== pg_notify(22.113(22) epoch 194471) v5 ==== 1952+0+0 (1311281533 0 0) 0x52640c0 con 0x4830dc0 -4496> 2014-08-13 17:22:49.832583 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started -4495> 2014-08-13 17:22:49.832599 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.808789, event: header_read, request: pg_notify(22.113(22) epoch 194471) v5 -4494> 2014-08-13 17:22:49.832606 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.808790, event: throttled, request: pg_notify(22.113(22) epoch 194471) v5 -4493> 2014-08-13 17:22:49.832611 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.808888, event: all_read, request: pg_notify(22.113(22) epoch 194471) v5 -4492> 2014-08-13 17:22:49.832601 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Start -4491> 2014-08-13 17:22:49.832617 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.832597, event: dispatched, request: pg_notify(22.113(22) epoch 194471) v5 -4490> 2014-08-13 17:22:49.832623 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.832623, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194471) v5 -4489> 2014-08-13 17:22:49.832619 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary -4488> 2014-08-13 17:22:49.832630 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.832630, event: started, request: pg_notify(22.113(22) epoch 194471) v5 -4487> 2014-08-13 17:22:49.832652 7f74d8735700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:22:49.832652, event: done, request: pg_notify(22.113(22) epoch 194471) v5 -4486> 2014-08-13 17:22:49.832643 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000041 0 0.000000 -4485> 2014-08-13 17:22:49.832665 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary -4484> 2014-08-13 17:22:49.832671 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 4 ==== pg_query(24.cd epoch 194471) v3 ==== 144+0+0 (2123355451 0 0) 0x29c6900 con 0x4835ac0 -4483> 2014-08-13 17:22:49.832686 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.806014, event: header_read, request: pg_query(24.cd epoch 194471) v3 -4482> 2014-08-13 17:22:49.832692 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.806016, event: throttled, request: pg_query(24.cd epoch 194471) v3 -4481> 2014-08-13 17:22:49.832683 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering -4480> 2014-08-13 17:22:49.832698 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.806076, event: all_read, request: pg_query(24.cd epoch 194471) v3 -4479> 2014-08-13 17:22:49.832703 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.832684, event: dispatched, request: pg_query(24.cd epoch 194471) v3 -4478> 2014-08-13 17:22:49.832710 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.832709, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194471) v3 -4477> 2014-08-13 17:22:49.832690 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4476> 2014-08-13 17:22:49.832703 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo -4475> 2014-08-13 17:22:49.832717 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.832717, event: started, request: pg_query(24.cd epoch 194471) v3 -4474> 2014-08-13 17:22:49.832733 7f74d8735700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:22:49.832733, event: done, request: pg_query(24.cd epoch 194471) v3 -4473> 2014-08-13 17:22:49.832747 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.32 192.168.1.115:6812/59915 5 ==== pg_notify(22.6bd(2) epoch 194471) v5 ==== 828+0+0 (2496394698 0 0) 0x57d3800 con 0x4832100 -4472> 2014-08-13 17:22:49.832762 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.824693, event: header_read, request: pg_notify(22.6bd(2) epoch 194471) v5 -4471> 2014-08-13 17:22:49.832768 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.824695, event: throttled, request: pg_notify(22.6bd(2) epoch 194471) v5 -4470> 2014-08-13 17:22:49.832774 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.824797, event: all_read, request: pg_notify(22.6bd(2) epoch 194471) v5 -4469> 2014-08-13 17:22:49.832779 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.832760, event: dispatched, request: pg_notify(22.6bd(2) epoch 194471) v5 -4468> 2014-08-13 17:22:49.832785 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.832785, event: waiting_for_osdmap, request: pg_notify(22.6bd(2) epoch 194471) v5 -4467> 2014-08-13 17:22:49.832792 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.832792, event: started, request: pg_notify(22.6bd(2) epoch 194471) v5 -4466> 2014-08-13 17:22:49.832811 7f74d8735700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:22:49.832811, event: done, request: pg_notify(22.6bd(2) epoch 194471) v5 -4465> 2014-08-13 17:22:49.832823 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 11 ==== pg_notify(22.3d9(38) epoch 194471) v5 ==== 2772+0+0 (286268105 0 0) 0x4ac3f00 con 0x4835280 -4464> 2014-08-13 17:22:49.832838 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.800103, event: header_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4463> 2014-08-13 17:22:49.832844 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.800105, event: throttled, request: pg_notify(22.3d9(38) epoch 194471) v5 -4462> 2014-08-13 17:22:49.832849 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.800231, event: all_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4461> 2014-08-13 17:22:49.832854 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.832836, event: dispatched, request: pg_notify(22.3d9(38) epoch 194471) v5 -4460> 2014-08-13 17:22:49.832860 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.832860, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194471) v5 -4459> 2014-08-13 17:22:49.832867 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.832867, event: started, request: pg_notify(22.3d9(38) epoch 194471) v5 -4458> 2014-08-13 17:22:49.832885 7f74d8735700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:22:49.832885, event: done, request: pg_notify(22.3d9(38) epoch 194471) v5 -4457> 2014-08-13 17:22:49.832913 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 5 ==== pg_notify(22.3d9(38) epoch 194471) v5 ==== 2772+0+0 (1105819238 0 0) 0x466f540 con 0x4834620 -4456> 2014-08-13 17:22:49.832928 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.827815, event: header_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4455> 2014-08-13 17:22:49.832934 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.827816, event: throttled, request: pg_notify(22.3d9(38) epoch 194471) v5 -4454> 2014-08-13 17:22:49.832939 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.827966, event: all_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4453> 2014-08-13 17:22:49.832944 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.832926, event: dispatched, request: pg_notify(22.3d9(38) epoch 194471) v5 -4452> 2014-08-13 17:22:49.832950 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.832950, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194471) v5 -4451> 2014-08-13 17:22:49.832956 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.832956, event: started, request: pg_notify(22.3d9(38) epoch 194471) v5 -4450> 2014-08-13 17:22:49.832974 7f74d8735700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:22:49.832974, event: done, request: pg_notify(22.3d9(38) epoch 194471) v5 -4449> 2014-08-13 17:22:49.832984 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.873696 4 0.000256 -4448> 2014-08-13 17:22:49.833003 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 7 ==== pg_notify(22.3d5(2) epoch 194471) v5 ==== 828+0+0 (19867761 0 0) 0x57d55c0 con 0x48377a0 -4447> 2014-08-13 17:22:49.833018 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.808739, event: header_read, request: pg_notify(22.3d5(2) epoch 194471) v5 -4446> 2014-08-13 17:22:49.833024 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.808741, event: throttled, request: pg_notify(22.3d5(2) epoch 194471) v5 -4445> 2014-08-13 17:22:49.833014 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.908021 0 0.000000 -4444> 2014-08-13 17:22:49.833030 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.808797, event: all_read, request: pg_notify(22.3d5(2) epoch 194471) v5 -4443> 2014-08-13 17:22:49.833036 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.833016, event: dispatched, request: pg_notify(22.3d5(2) epoch 194471) v5 -4442> 2014-08-13 17:22:49.833042 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.833041, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194471) v5 -4441> 2014-08-13 17:22:49.833049 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.833048, event: started, request: pg_notify(22.3d5(2) epoch 194471) v5 -4440> 2014-08-13 17:22:49.833038 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4439> 2014-08-13 17:22:49.833067 7f74d8735700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:22:49.833067, event: done, request: pg_notify(22.3d5(2) epoch 194471) v5 -4438> 2014-08-13 17:22:49.833081 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 6 ==== pg_info(1 pgs e194471:22.220(1)) v4 ==== 773+0+0 (487036966 0 0) 0x49e2a00 con 0x4868420 -4437> 2014-08-13 17:22:49.833095 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.800408, event: header_read, request: pg_info(1 pgs e194471:22.220(1)) v4 -4436> 2014-08-13 17:22:49.833101 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.800409, event: throttled, request: pg_info(1 pgs e194471:22.220(1)) v4 -4435> 2014-08-13 17:22:49.833106 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.800467, event: all_read, request: pg_info(1 pgs e194471:22.220(1)) v4 -4434> 2014-08-13 17:22:49.833131 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.833093, event: dispatched, request: pg_info(1 pgs e194471:22.220(1)) v4 -4433> 2014-08-13 17:22:49.833109 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started/Stray 0.903006 4 0.002251 -4432> 2014-08-13 17:22:49.833137 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.833137, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.220(1)) v4 -4431> 2014-08-13 17:22:49.833146 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.833145, event: started, request: pg_info(1 pgs e194471:22.220(1)) v4 -4430> 2014-08-13 17:22:49.833142 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started 0.904884 0 0.000000 -4429> 2014-08-13 17:22:49.833166 7f74d8735700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:22:49.833166, event: done, request: pg_info(1 pgs e194471:22.220(1)) v4 -4428> 2014-08-13 17:22:49.833172 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194470/193227) [330,4,288] r=1 lpr=194470 pi=193227-194469/28 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset -4427> 2014-08-13 17:22:49.833195 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 4 ==== pg_info(1 pgs e194471:22.3ad(2)) v4 ==== 824+0+0 (2769111654 0 0) 0x49e1180 con 0x4868160 -4426> 2014-08-13 17:22:49.833212 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.802503, event: header_read, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4425> 2014-08-13 17:22:49.833219 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.802505, event: throttled, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4424> 2014-08-13 17:22:49.833225 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.802738, event: all_read, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4423> 2014-08-13 17:22:49.833230 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.833210, event: dispatched, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4422> 2014-08-13 17:22:49.833236 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.833236, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4421> 2014-08-13 17:22:49.833243 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.833243, event: started, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4420> 2014-08-13 17:22:49.833261 7f74d8735700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:22:49.833261, event: done, request: pg_info(1 pgs e194471:22.3ad(2)) v4 -4419> 2014-08-13 17:22:49.833279 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 6 ==== pg_notify(22.1d2(1) epoch 194471) v5 ==== 777+0+0 (1328840315 0 0) 0x53baf40 con 0x486bf40 -4418> 2014-08-13 17:22:49.833294 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.824637, event: header_read, request: pg_notify(22.1d2(1) epoch 194471) v5 -4417> 2014-08-13 17:22:49.833277 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Reset 0.000105 1 0.000124 -4416> 2014-08-13 17:22:49.833300 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.824639, event: throttled, request: pg_notify(22.1d2(1) epoch 194471) v5 -4415> 2014-08-13 17:22:49.833306 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.824749, event: all_read, request: pg_notify(22.1d2(1) epoch 194471) v5 -4414> 2014-08-13 17:22:49.833311 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.833292, event: dispatched, request: pg_notify(22.1d2(1) epoch 194471) v5 -4413> 2014-08-13 17:22:49.833304 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started -4412> 2014-08-13 17:22:49.833317 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.833317, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194471) v5 -4411> 2014-08-13 17:22:49.833324 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.833324, event: started, request: pg_notify(22.1d2(1) epoch 194471) v5 -4410> 2014-08-13 17:22:49.833323 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Start -4409> 2014-08-13 17:22:49.833344 7f74d8735700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:22:49.833344, event: done, request: pg_notify(22.1d2(1) epoch 194471) v5 -4408> 2014-08-13 17:22:49.833358 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 8 ==== pg_notify(22.39d(20) epoch 194471) v5 ==== 1834+0+0 (646771759 0 0) 0x4ac0c40 con 0x486bc80 -4407> 2014-08-13 17:22:49.833372 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.798747, event: header_read, request: pg_notify(22.39d(20) epoch 194471) v5 -4406> 2014-08-13 17:22:49.833340 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: transitioning to Stray -4405> 2014-08-13 17:22:49.833363 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x2713cc0 con 0x475cd00 -4404> 2014-08-13 17:22:49.833379 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.798748, event: throttled, request: pg_notify(22.39d(20) epoch 194471) v5 -4403> 2014-08-13 17:22:49.833384 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.798860, event: all_read, request: pg_notify(22.39d(20) epoch 194471) v5 -4402> 2014-08-13 17:22:49.833390 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.833370, event: dispatched, request: pg_notify(22.39d(20) epoch 194471) v5 -4401> 2014-08-13 17:22:49.833396 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.833396, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194471) v5 -4400> 2014-08-13 17:22:49.833386 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Start 0.000063 0 0.000000 -4399> 2014-08-13 17:22:49.833403 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.833403, event: started, request: pg_notify(22.39d(20) epoch 194471) v5 -4398> 2014-08-13 17:22:49.833394 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.285 192.168.1.101:6822/26295 -- pg_log(22.f3 epoch 194471 log log((0'0,194215'506], crt=191648'343) query_epoch 194471) v4 -- ?+0 0x5b19e00 -4397> 2014-08-13 17:22:49.833408 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/Stray -4396> 2014-08-13 17:22:49.833422 7f74d8735700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:22:49.833422, event: done, request: pg_notify(22.39d(20) epoch 194471) v5 -4395> 2014-08-13 17:22:49.833440 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 5 ==== pg_query(24.2ec epoch 194471) v3 ==== 144+0+0 (446923078 0 0) 0x4eb0000 con 0x486b9c0 -4394> 2014-08-13 17:22:49.833454 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.808409, event: header_read, request: pg_query(24.2ec epoch 194471) v3 -4393> 2014-08-13 17:22:49.833460 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.808410, event: throttled, request: pg_query(24.2ec epoch 194471) v3 -4392> 2014-08-13 17:22:49.833465 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.808451, event: all_read, request: pg_query(24.2ec epoch 194471) v3 -4391> 2014-08-13 17:22:49.833470 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.833452, event: dispatched, request: pg_query(24.2ec epoch 194471) v3 -4390> 2014-08-13 17:22:49.833475 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.833475, event: waiting_for_osdmap, request: pg_query(24.2ec epoch 194471) v3 -4389> 2014-08-13 17:22:49.833482 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.833482, event: started, request: pg_query(24.2ec epoch 194471) v3 -4388> 2014-08-13 17:22:49.833497 7f74d8735700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:22:49.833497, event: done, request: pg_query(24.2ec epoch 194471) v3 -4387> 2014-08-13 17:22:49.833509 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 7 ==== pg_notify(22.48e(20) epoch 194471) v5 ==== 1834+0+0 (2766538147 0 0) 0x4ac78c0 con 0x486b700 -4386> 2014-08-13 17:22:49.833523 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.825601, event: header_read, request: pg_notify(22.48e(20) epoch 194471) v5 -4385> 2014-08-13 17:22:49.833529 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.825603, event: throttled, request: pg_notify(22.48e(20) epoch 194471) v5 -4384> 2014-08-13 17:22:49.833534 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.825725, event: all_read, request: pg_notify(22.48e(20) epoch 194471) v5 -4383> 2014-08-13 17:22:49.833546 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.833521, event: dispatched, request: pg_notify(22.48e(20) epoch 194471) v5 -4382> 2014-08-13 17:22:49.833553 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.833553, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194471) v5 -4381> 2014-08-13 17:22:49.833560 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.833560, event: started, request: pg_notify(22.48e(20) epoch 194471) v5 -4380> 2014-08-13 17:22:49.833581 7f74d8735700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:22:49.833581, event: done, request: pg_notify(22.48e(20) epoch 194471) v5 -4379> 2014-08-13 17:22:49.833604 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 5 ==== pg_notify(24.1bb(26) epoch 194471) v5 ==== 2214+0+0 (1123115219 0 0) 0x4ac7c40 con 0x486b440 -4378> 2014-08-13 17:22:49.833624 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.809411, event: header_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -4377> 2014-08-13 17:22:49.833594 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.876170 4 0.000227 -4376> 2014-08-13 17:22:49.833638 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.809413, event: throttled, request: pg_notify(24.1bb(26) epoch 194471) v5 -4375> 2014-08-13 17:22:49.833644 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.809512, event: all_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -4374> 2014-08-13 17:22:49.833650 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.833622, event: dispatched, request: pg_notify(24.1bb(26) epoch 194471) v5 -4373> 2014-08-13 17:22:49.833643 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.902322 0 0.000000 -4372> 2014-08-13 17:22:49.833608 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4371> 2014-08-13 17:22:49.833663 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.833662, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194471) v5 -4370> 2014-08-13 17:22:49.833675 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.833675, event: started, request: pg_notify(24.1bb(26) epoch 194471) v5 -4369> 2014-08-13 17:22:49.833668 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4368> 2014-08-13 17:22:49.833699 7f74d8735700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:22:49.833699, event: done, request: pg_notify(24.1bb(26) epoch 194471) v5 -4367> 2014-08-13 17:22:49.833731 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 4 ==== pg_query(24.e2 epoch 194471) v3 ==== 144+0+0 (1031204905 0 0) 0x4eb1860 con 0x486b180 -4366> 2014-08-13 17:22:49.833750 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.808277, event: header_read, request: pg_query(24.e2 epoch 194471) v3 -4365> 2014-08-13 17:22:49.833760 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.808278, event: throttled, request: pg_query(24.e2 epoch 194471) v3 -4364> 2014-08-13 17:22:49.833769 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.808324, event: all_read, request: pg_query(24.e2 epoch 194471) v3 -4363> 2014-08-13 17:22:49.833778 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.833748, event: dispatched, request: pg_query(24.e2 epoch 194471) v3 -4362> 2014-08-13 17:22:49.833787 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.833787, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194471) v3 -4361> 2014-08-13 17:22:49.833798 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.833798, event: started, request: pg_query(24.e2 epoch 194471) v3 -4360> 2014-08-13 17:22:49.833818 7f74d8735700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:22:49.833817, event: done, request: pg_query(24.e2 epoch 194471) v3 -4359> 2014-08-13 17:22:49.833835 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 4 ==== pg_info(1 pgs e194471:22.69f(2)) v4 ==== 824+0+0 (332504503 0 0) 0x53be900 con 0x486aec0 -4358> 2014-08-13 17:22:49.833853 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.827864, event: header_read, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4357> 2014-08-13 17:22:49.833863 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.827867, event: throttled, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4356> 2014-08-13 17:22:49.833872 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.827964, event: all_read, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4355> 2014-08-13 17:22:49.833881 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.833851, event: dispatched, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4354> 2014-08-13 17:22:49.833891 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.833891, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4353> 2014-08-13 17:22:49.833903 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.833903, event: started, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4352> 2014-08-13 17:22:49.833927 7f74d8735700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:22:49.833927, event: done, request: pg_info(1 pgs e194471:22.69f(2)) v4 -4351> 2014-08-13 17:22:49.833944 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 5 ==== pg_info(1 pgs e194471:22.639(2)) v4 ==== 824+0+0 (99892275 0 0) 0x57d6ac0 con 0x486a940 -4350> 2014-08-13 17:22:49.833963 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.803909, event: header_read, request: pg_info(1 pgs e194471:22.639(2)) v4 -4349> 2014-08-13 17:22:49.833974 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.803911, event: throttled, request: pg_info(1 pgs e194471:22.639(2)) v4 -4348> 2014-08-13 17:22:49.833922 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.56 192.168.1.111:6826/11446 -- pg_log(22.775 epoch 194471 log log((0'0,194215'513], crt=191727'453) query_epoch 194471) v4 -- ?+0 0x5b1aa00 -4347> 2014-08-13 17:22:49.833983 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.804009, event: all_read, request: pg_info(1 pgs e194471:22.639(2)) v4 -4346> 2014-08-13 17:22:49.833992 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.833961, event: dispatched, request: pg_info(1 pgs e194471:22.639(2)) v4 -4345> 2014-08-13 17:22:49.834002 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.834002, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.639(2)) v4 -4344> 2014-08-13 17:22:49.834016 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.834016, event: started, request: pg_info(1 pgs e194471:22.639(2)) v4 -4343> 2014-08-13 17:22:49.834038 7f74d8735700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:22:49.834038, event: done, request: pg_info(1 pgs e194471:22.639(2)) v4 -4342> 2014-08-13 17:22:49.834063 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 4 ==== pg_info(1 pgs e194471:22.12a(2)) v4 ==== 824+0+0 (2941264644 0 0) 0x49e2f40 con 0x486a680 -4341> 2014-08-13 17:22:49.834081 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.804847, event: header_read, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4340> 2014-08-13 17:22:49.834063 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary/Peering/WaitActingChange 0.875460 4 0.000228 -4339> 2014-08-13 17:22:49.834096 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.804849, event: throttled, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4338> 2014-08-13 17:22:49.834105 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.804913, event: all_read, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4337> 2014-08-13 17:22:49.834114 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.834079, event: dispatched, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4336> 2014-08-13 17:22:49.834108 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.875525 0 0.000000 -4335> 2014-08-13 17:22:49.834124 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.834124, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4334> 2014-08-13 17:22:49.834136 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.834136, event: started, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4333> 2014-08-13 17:22:49.834131 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started 0.907024 0 0.000000 -4332> 2014-08-13 17:22:49.834159 7f74d8735700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:22:49.834159, event: done, request: pg_info(1 pgs e194471:22.12a(2)) v4 -4331> 2014-08-13 17:22:49.834151 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/194470) [4,25,362] r=0 lpr=194470 pi=193635-194469/24 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset -4330> 2014-08-13 17:22:49.834183 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 9 ==== pg_notify(24.3d(24) epoch 194471) v5 ==== 2096+0+0 (486193461 0 0) 0x57d4980 con 0x486a3c0 -4329> 2014-08-13 17:22:49.834203 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.802531, event: header_read, request: pg_notify(24.3d(24) epoch 194471) v5 -4328> 2014-08-13 17:22:49.834211 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.802532, event: throttled, request: pg_notify(24.3d(24) epoch 194471) v5 -4327> 2014-08-13 17:22:49.834217 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.802632, event: all_read, request: pg_notify(24.3d(24) epoch 194471) v5 -4326> 2014-08-13 17:22:49.834226 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.834201, event: dispatched, request: pg_notify(24.3d(24) epoch 194471) v5 -4325> 2014-08-13 17:22:49.834235 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.834235, event: waiting_for_osdmap, request: pg_notify(24.3d(24) epoch 194471) v5 -4324> 2014-08-13 17:22:49.834247 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.834246, event: started, request: pg_notify(24.3d(24) epoch 194471) v5 -4323> 2014-08-13 17:22:49.834254 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.353 192.168.1.108:6836/6227 -- pg_log(22.775 epoch 194471 log log((0'0,194215'513], crt=191727'453) query_epoch 194471) v4 -- ?+0 0x5b1ce00 -4322> 2014-08-13 17:22:49.834258 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Reset 0.000107 1 0.000193 -4321> 2014-08-13 17:22:49.834284 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started -4320> 2014-08-13 17:22:49.834307 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Start -4319> 2014-08-13 17:22:49.834327 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: transitioning to Primary -4318> 2014-08-13 17:22:49.834350 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Start 0.000042 0 0.000000 -4317> 2014-08-13 17:22:49.834373 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary -4316> 2014-08-13 17:22:49.834394 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Peering -4315> 2014-08-13 17:22:49.834415 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetInfo -4314> 2014-08-13 17:22:49.834547 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4313> 2014-08-13 17:22:49.834574 7f74d8735700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:22:49.834573, event: done, request: pg_notify(24.3d(24) epoch 194471) v5 -4312> 2014-08-13 17:22:49.834610 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 6 ==== pg_notify(22.707(24) epoch 194471) v5 ==== 2066+0+0 (2293908023 0 0) 0x466ad80 con 0x48d9ce0 -4311> 2014-08-13 17:22:49.834632 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.806318, event: header_read, request: pg_notify(22.707(24) epoch 194471) v5 -4310> 2014-08-13 17:22:49.834642 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.806319, event: throttled, request: pg_notify(22.707(24) epoch 194471) v5 -4309> 2014-08-13 17:22:49.834651 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.806431, event: all_read, request: pg_notify(22.707(24) epoch 194471) v5 -4308> 2014-08-13 17:22:49.834660 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.834629, event: dispatched, request: pg_notify(22.707(24) epoch 194471) v5 -4307> 2014-08-13 17:22:49.834670 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.834670, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194471) v5 -4306> 2014-08-13 17:22:49.834682 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.834682, event: started, request: pg_notify(22.707(24) epoch 194471) v5 -4305> 2014-08-13 17:22:49.834674 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.877000 4 0.000221 -4304> 2014-08-13 17:22:49.834706 7f74d8735700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:22:49.834706, event: done, request: pg_notify(22.707(24) epoch 194471) v5 -4303> 2014-08-13 17:22:49.834707 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.939620 0 0.000000 -4302> 2014-08-13 17:22:49.834729 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 7 ==== pg_query(24.3f7 epoch 194471) v3 ==== 144+0+0 (2180262737 0 0) 0x4eb30c0 con 0x486e040 -4301> 2014-08-13 17:22:49.834734 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194449 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4300> 2014-08-13 17:22:49.834747 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.799399, event: header_read, request: pg_query(24.3f7 epoch 194471) v3 -4299> 2014-08-13 17:22:49.834754 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.799401, event: throttled, request: pg_query(24.3f7 epoch 194471) v3 -4298> 2014-08-13 17:22:49.834763 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.799449, event: all_read, request: pg_query(24.3f7 epoch 194471) v3 -4297> 2014-08-13 17:22:49.834772 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.834745, event: dispatched, request: pg_query(24.3f7 epoch 194471) v3 -4296> 2014-08-13 17:22:49.834782 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.834781, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194471) v3 -4295> 2014-08-13 17:22:49.834793 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.834792, event: started, request: pg_query(24.3f7 epoch 194471) v3 -4294> 2014-08-13 17:22:49.834812 7f74d8735700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:22:49.834812, event: done, request: pg_query(24.3f7 epoch 194471) v3 -4293> 2014-08-13 17:22:49.834828 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 5 ==== pg_notify(22.3d9(9) epoch 194471) v5 ==== 1229+0+0 (2524317327 0 0) 0x57d32c0 con 0x48dfbc0 -4292> 2014-08-13 17:22:49.834851 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.803908, event: header_read, request: pg_notify(22.3d9(9) epoch 194471) v5 -4291> 2014-08-13 17:22:49.834834 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started/Stray 0.901053 4 0.000223 -4290> 2014-08-13 17:22:49.834862 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.803909, event: throttled, request: pg_notify(22.3d9(9) epoch 194471) v5 -4289> 2014-08-13 17:22:49.834871 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.804184, event: all_read, request: pg_notify(22.3d9(9) epoch 194471) v5 -4288> 2014-08-13 17:22:49.834866 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started 0.901162 0 0.000000 -4287> 2014-08-13 17:22:49.834880 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.834849, event: dispatched, request: pg_notify(22.3d9(9) epoch 194471) v5 -4286> 2014-08-13 17:22:49.834890 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.834890, event: waiting_for_osdmap, request: pg_notify(22.3d9(9) epoch 194471) v5 -4285> 2014-08-13 17:22:49.834885 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194470/192955) [346,4,304] r=1 lpr=194470 pi=193125-194469/25 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset -4284> 2014-08-13 17:22:49.834901 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.834901, event: started, request: pg_notify(22.3d9(9) epoch 194471) v5 -4283> 2014-08-13 17:22:49.834927 7f74d8735700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:22:49.834927, event: done, request: pg_notify(22.3d9(9) epoch 194471) v5 -4282> 2014-08-13 17:22:49.834946 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 6 ==== pg_notify(22.3d9(38) epoch 194471) v5 ==== 2772+0+0 (164548118 0 0) 0x466a840 con 0x4900580 -4281> 2014-08-13 17:22:49.834965 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.799265, event: header_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4280> 2014-08-13 17:22:49.834975 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.799267, event: throttled, request: pg_notify(22.3d9(38) epoch 194471) v5 -4279> 2014-08-13 17:22:49.834962 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.61 192.168.1.111:6840/16566 -- pg_log(22.1d2 epoch 194471 log log((0'0,194215'510], crt=191648'372) query_epoch 194471) v4 -- ?+0 0x5b19800 -4278> 2014-08-13 17:22:49.834979 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Reset 0.000093 1 0.000106 -4277> 2014-08-13 17:22:49.834990 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.799382, event: all_read, request: pg_notify(22.3d9(38) epoch 194471) v5 -4276> 2014-08-13 17:22:49.835010 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.834963, event: dispatched, request: pg_notify(22.3d9(38) epoch 194471) v5 -4275> 2014-08-13 17:22:49.835004 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started -4274> 2014-08-13 17:22:49.835022 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.835022, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194471) v5 -4273> 2014-08-13 17:22:49.835022 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Start -4272> 2014-08-13 17:22:49.835034 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.835033, event: started, request: pg_notify(22.3d9(38) epoch 194471) v5 -4271> 2014-08-13 17:22:49.835076 7f74d8735700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:22:49.835075, event: done, request: pg_notify(22.3d9(38) epoch 194471) v5 -4270> 2014-08-13 17:22:49.835048 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: transitioning to Stray -4269> 2014-08-13 17:22:49.835105 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Start 0.000084 0 0.000000 -4268> 2014-08-13 17:22:49.835109 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 6 ==== pg_notify(22.3af(22) epoch 194471) v5 ==== 1948+0+0 (3248752187 0 0) 0x53bbb80 con 0x4900f20 -4267> 2014-08-13 17:22:49.835140 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.825777, event: header_read, request: pg_notify(22.3af(22) epoch 194471) v5 -4266> 2014-08-13 17:22:49.835152 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.825779, event: throttled, request: pg_notify(22.3af(22) epoch 194471) v5 -4265> 2014-08-13 17:22:49.835166 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.825885, event: all_read, request: pg_notify(22.3af(22) epoch 194471) v5 -4264> 2014-08-13 17:22:49.835133 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/Stray -4263> 2014-08-13 17:22:49.835185 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.835137, event: dispatched, request: pg_notify(22.3af(22) epoch 194471) v5 -4262> 2014-08-13 17:22:49.835195 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.835195, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194471) v5 -4261> 2014-08-13 17:22:49.835207 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.835207, event: started, request: pg_notify(22.3af(22) epoch 194471) v5 -4260> 2014-08-13 17:22:49.835231 7f74d8735700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:22:49.835231, event: done, request: pg_notify(22.3af(22) epoch 194471) v5 -4259> 2014-08-13 17:22:49.835228 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.349 192.168.1.104:6800/61726 -- pg_log(22.1d2 epoch 194471 log log((0'0,194215'510], crt=191648'372) query_epoch 194471) v4 -- ?+0 0x5b1e600 -4258> 2014-08-13 17:22:49.835255 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 9 ==== pg_notify(22.113(1) epoch 194471) v5 ==== 777+0+0 (3888373046 0 0) 0x53bcec0 con 0x4901600 -4257> 2014-08-13 17:22:49.835274 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.826953, event: header_read, request: pg_notify(22.113(1) epoch 194471) v5 -4256> 2014-08-13 17:22:49.835285 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.826955, event: throttled, request: pg_notify(22.113(1) epoch 194471) v5 -4255> 2014-08-13 17:22:49.835294 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.827053, event: all_read, request: pg_notify(22.113(1) epoch 194471) v5 -4254> 2014-08-13 17:22:49.835303 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.835272, event: dispatched, request: pg_notify(22.113(1) epoch 194471) v5 -4253> 2014-08-13 17:22:49.835313 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.835313, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194471) v5 -4252> 2014-08-13 17:22:49.835324 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.835324, event: started, request: pg_notify(22.113(1) epoch 194471) v5 -4251> 2014-08-13 17:22:49.835347 7f74d8735700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:22:49.835346, event: done, request: pg_notify(22.113(1) epoch 194471) v5 -4250> 2014-08-13 17:22:49.835363 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.19 192.168.1.115:6818/61186 4 ==== pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 ==== 139390+0+0 (998449024 0 0) 0x278ec00 con 0x4901340 -4249> 2014-08-13 17:22:49.835384 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.804686, event: header_read, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4248> 2014-08-13 17:22:49.835396 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.804688, event: throttled, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4247> 2014-08-13 17:22:49.835407 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.830589, event: all_read, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4246> 2014-08-13 17:22:49.835418 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.835382, event: dispatched, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4245> 2014-08-13 17:22:49.835430 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.835430, event: waiting_for_osdmap, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4244> 2014-08-13 17:22:49.835444 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.835443, event: started, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4243> 2014-08-13 17:22:49.835453 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.876614 6 0.000199 -4242> 2014-08-13 17:22:49.835463 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4241> 2014-08-13 17:22:49.835494 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.900141 0 0.000000 -4240> 2014-08-13 17:22:49.835523 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194449 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4239> 2014-08-13 17:22:49.835590 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf(unlocked)] enter Initial -4238> 2014-08-13 17:22:49.835679 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 inactive] exit Initial 0.000090 0 0.000000 -4237> 2014-08-13 17:22:49.835711 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 inactive] enter Reset -4236> 2014-08-13 17:22:49.835717 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started/Stray 0.938762 4 0.001169 -4235> 2014-08-13 17:22:49.835734 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] exit Reset 0.000024 1 0.000054 -4234> 2014-08-13 17:22:49.835755 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Started -4233> 2014-08-13 17:22:49.835751 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started 0.939742 0 0.000000 -4232> 2014-08-13 17:22:49.835776 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Start -4231> 2014-08-13 17:22:49.835787 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194470/192966) [364,4,360] r=1 lpr=194470 pi=193125-194469/26 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset -4230> 2014-08-13 17:22:49.835793 7f74d8735700 1 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] state: transitioning to Stray -4229> 2014-08-13 17:22:49.835826 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] exit Start 0.000050 0 0.000000 -4228> 2014-08-13 17:22:49.835848 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.3bf( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Started/Stray -4227> 2014-08-13 17:22:49.835884 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.313 192.168.1.109:6820/35445 -- pg_log(22.3e4 epoch 194471 log log((0'0,194215'518], crt=191727'442) query_epoch 194471) v4 -- ?+0 0x5b1c800 -4226> 2014-08-13 17:22:49.835898 7f74d8735700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:22:49.835898, event: done, request: pg_log(24.3bf epoch 194471 log log((192834'12014,194422'13811], crt=193305'13459) query_epoch 194471) v4 -4225> 2014-08-13 17:22:49.835932 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Reset 0.000145 1 0.000139 -4224> 2014-08-13 17:22:49.835949 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 7 ==== pg_notify(22.1f1(22) epoch 194471) v5 ==== 1948+0+0 (1785537184 0 0) 0x57d3640 con 0x4902680 -4223> 2014-08-13 17:22:49.835963 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started -4222> 2014-08-13 17:22:49.835975 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.827879, event: header_read, request: pg_notify(22.1f1(22) epoch 194471) v5 -4221> 2014-08-13 17:22:49.835996 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.827881, event: throttled, request: pg_notify(22.1f1(22) epoch 194471) v5 -4220> 2014-08-13 17:22:49.835989 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Start -4219> 2014-08-13 17:22:49.836007 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.828068, event: all_read, request: pg_notify(22.1f1(22) epoch 194471) v5 -4218> 2014-08-13 17:22:49.836016 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.835972, event: dispatched, request: pg_notify(22.1f1(22) epoch 194471) v5 -4217> 2014-08-13 17:22:49.836008 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: transitioning to Stray -4216> 2014-08-13 17:22:49.836031 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.836031, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194471) v5 -4215> 2014-08-13 17:22:49.836041 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.836040, event: started, request: pg_notify(22.1f1(22) epoch 194471) v5 -4214> 2014-08-13 17:22:49.836041 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Start 0.000052 0 0.000000 -4213> 2014-08-13 17:22:49.836068 7f74d8735700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:22:49.836068, event: done, request: pg_notify(22.1f1(22) epoch 194471) v5 -4212> 2014-08-13 17:22:49.836067 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/Stray -4211> 2014-08-13 17:22:49.836096 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 5 ==== pg_notify(24.30b(29) epoch 194471) v5 ==== 2391+0+0 (2687185915 0 0) 0x53babc0 con 0x49023c0 -4210> 2014-08-13 17:22:49.836116 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.825396, event: header_read, request: pg_notify(24.30b(29) epoch 194471) v5 -4209> 2014-08-13 17:22:49.836127 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.825398, event: throttled, request: pg_notify(24.30b(29) epoch 194471) v5 -4208> 2014-08-13 17:22:49.836136 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.825498, event: all_read, request: pg_notify(24.30b(29) epoch 194471) v5 -4207> 2014-08-13 17:22:49.836145 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.836114, event: dispatched, request: pg_notify(24.30b(29) epoch 194471) v5 -4206> 2014-08-13 17:22:49.836155 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.836155, event: waiting_for_osdmap, request: pg_notify(24.30b(29) epoch 194471) v5 -4205> 2014-08-13 17:22:49.836166 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.836166, event: started, request: pg_notify(24.30b(29) epoch 194471) v5 -4204> 2014-08-13 17:22:49.836198 7f74d8735700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:22:49.836198, event: done, request: pg_notify(24.30b(29) epoch 194471) v5 -4203> 2014-08-13 17:22:49.836200 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.318 192.168.1.110:6814/43857 -- pg_log(22.3e4 epoch 194471 log log((0'0,194215'518], crt=191727'442) query_epoch 194471) v4 -- ?+0 0x5b1c200 -4202> 2014-08-13 17:22:49.836221 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 5 ==== pg_notify(22.619(22) epoch 194471) v5 ==== 1948+0+0 (1645947433 0 0) 0x53b9180 con 0x4902100 -4201> 2014-08-13 17:22:49.836241 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.824919, event: header_read, request: pg_notify(22.619(22) epoch 194471) v5 -4200> 2014-08-13 17:22:49.836229 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.878916 5 0.000209 -4199> 2014-08-13 17:22:49.836252 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.824921, event: throttled, request: pg_notify(22.619(22) epoch 194471) v5 -4198> 2014-08-13 17:22:49.836261 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.825020, event: all_read, request: pg_notify(22.619(22) epoch 194471) v5 -4197> 2014-08-13 17:22:49.836256 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.938129 0 0.000000 -4196> 2014-08-13 17:22:49.836270 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.836239, event: dispatched, request: pg_notify(22.619(22) epoch 194471) v5 -4195> 2014-08-13 17:22:49.836290 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.836290, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194471) v5 -4194> 2014-08-13 17:22:49.836287 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4193> 2014-08-13 17:22:49.836303 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.836302, event: started, request: pg_notify(22.619(22) epoch 194471) v5 -4192> 2014-08-13 17:22:49.836326 7f74d8735700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:22:49.836326, event: done, request: pg_notify(22.619(22) epoch 194471) v5 -4191> 2014-08-13 17:22:49.836361 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 5 ==== pg_notify(22.3d9(42) epoch 194471) v5 ==== 2984+0+0 (2691819881 0 0) 0x5264b40 con 0x4903700 -4190> 2014-08-13 17:22:49.836382 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.806608, event: header_read, request: pg_notify(22.3d9(42) epoch 194471) v5 -4189> 2014-08-13 17:22:49.836393 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.806610, event: throttled, request: pg_notify(22.3d9(42) epoch 194471) v5 -4188> 2014-08-13 17:22:49.836403 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.806726, event: all_read, request: pg_notify(22.3d9(42) epoch 194471) v5 -4187> 2014-08-13 17:22:49.836412 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.836379, event: dispatched, request: pg_notify(22.3d9(42) epoch 194471) v5 -4186> 2014-08-13 17:22:49.836422 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.836422, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194471) v5 -4185> 2014-08-13 17:22:49.836433 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.836433, event: started, request: pg_notify(22.3d9(42) epoch 194471) v5 -4184> 2014-08-13 17:22:49.836458 7f74d8735700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:22:49.836458, event: done, request: pg_notify(22.3d9(42) epoch 194471) v5 -4183> 2014-08-13 17:22:49.836457 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4182> 2014-08-13 17:22:49.836495 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 6 ==== pg_notify(22.481(32) epoch 194471) v5 ==== 2518+0+0 (1968768602 0 0) 0x57d6e40 con 0x4903440 -4181> 2014-08-13 17:22:49.836514 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.826932, event: header_read, request: pg_notify(22.481(32) epoch 194471) v5 -4180> 2014-08-13 17:22:49.836525 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.826934, event: throttled, request: pg_notify(22.481(32) epoch 194471) v5 -4179> 2014-08-13 17:22:49.836534 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.827049, event: all_read, request: pg_notify(22.481(32) epoch 194471) v5 -4178> 2014-08-13 17:22:49.836543 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.836512, event: dispatched, request: pg_notify(22.481(32) epoch 194471) v5 -4177> 2014-08-13 17:22:49.836553 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.836552, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194471) v5 -4176> 2014-08-13 17:22:49.836564 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.836564, event: started, request: pg_notify(22.481(32) epoch 194471) v5 -4175> 2014-08-13 17:22:49.836587 7f74d8735700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:22:49.836587, event: done, request: pg_notify(22.481(32) epoch 194471) v5 -4174> 2014-08-13 17:22:49.836609 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 8 ==== pg_notify(22.22c(22) epoch 194471) v5 ==== 1952+0+0 (3715249169 0 0) 0x53b8e00 con 0x4903180 -4173> 2014-08-13 17:22:49.836628 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.825474, event: header_read, request: pg_notify(22.22c(22) epoch 194471) v5 -4172> 2014-08-13 17:22:49.836638 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.825476, event: throttled, request: pg_notify(22.22c(22) epoch 194471) v5 -4171> 2014-08-13 17:22:49.836648 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.825624, event: all_read, request: pg_notify(22.22c(22) epoch 194471) v5 -4170> 2014-08-13 17:22:49.836657 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.836626, event: dispatched, request: pg_notify(22.22c(22) epoch 194471) v5 -4169> 2014-08-13 17:22:49.836667 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.836667, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194471) v5 -4168> 2014-08-13 17:22:49.836678 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.836678, event: started, request: pg_notify(22.22c(22) epoch 194471) v5 -4167> 2014-08-13 17:22:49.836701 7f74d8735700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:22:49.836701, event: done, request: pg_notify(22.22c(22) epoch 194471) v5 -4166> 2014-08-13 17:22:49.836726 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 5 ==== pg_notify(22.22c(1) epoch 194471) v5 ==== 777+0+0 (2918385262 0 0) 0x57d7540 con 0x49044c0 -4165> 2014-08-13 17:22:49.836745 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.829068, event: header_read, request: pg_notify(22.22c(1) epoch 194471) v5 -4164> 2014-08-13 17:22:49.836755 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.829069, event: throttled, request: pg_notify(22.22c(1) epoch 194471) v5 -4163> 2014-08-13 17:22:49.836764 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.829136, event: all_read, request: pg_notify(22.22c(1) epoch 194471) v5 -4162> 2014-08-13 17:22:49.836774 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.836743, event: dispatched, request: pg_notify(22.22c(1) epoch 194471) v5 -4161> 2014-08-13 17:22:49.836783 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.836783, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194471) v5 -4160> 2014-08-13 17:22:49.836794 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.836794, event: started, request: pg_notify(22.22c(1) epoch 194471) v5 -4159> 2014-08-13 17:22:49.836816 7f74d8735700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:22:49.836816, event: done, request: pg_notify(22.22c(1) epoch 194471) v5 -4158> 2014-08-13 17:22:49.836834 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 5 ==== pg_notify(22.775(22) epoch 194471) v5 ==== 1948+0+0 (2086923034 0 0) 0x52632c0 con 0x49070c0 -4157> 2014-08-13 17:22:49.836815 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started/Stray 0.900132 4 0.000267 -4156> 2014-08-13 17:22:49.836852 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.825867, event: header_read, request: pg_notify(22.775(22) epoch 194471) v5 -4155> 2014-08-13 17:22:49.836914 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.825870, event: throttled, request: pg_notify(22.775(22) epoch 194471) v5 -4154> 2014-08-13 17:22:49.836926 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.825968, event: all_read, request: pg_notify(22.775(22) epoch 194471) v5 -4153> 2014-08-13 17:22:49.836939 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.836850, event: dispatched, request: pg_notify(22.775(22) epoch 194471) v5 -4152> 2014-08-13 17:22:49.836909 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started 0.900314 0 0.000000 -4151> 2014-08-13 17:22:49.836950 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.836949, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194471) v5 -4150> 2014-08-13 17:22:49.836962 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.836962, event: started, request: pg_notify(22.775(22) epoch 194471) v5 -4149> 2014-08-13 17:22:49.836955 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194470/193188) [17,4,395]/[17,395] r=-1 lpr=194470 pi=193188-194469/29 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Reset -4148> 2014-08-13 17:22:49.836987 7f74d8735700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:22:49.836987, event: done, request: pg_notify(22.775(22) epoch 194471) v5 -4147> 2014-08-13 17:22:49.837008 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e357c0 con 0x4906e00 -4146> 2014-08-13 17:22:49.837032 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -4145> 2014-08-13 17:22:49.837046 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 9 ==== pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 ==== 3656+0+0 (1194758598 0 0) 0x53be040 con 0x4d58840 -4144> 2014-08-13 17:22:49.837065 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.827331, event: header_read, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4143> 2014-08-13 17:22:49.837057 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.000102 1 0.000200 -4142> 2014-08-13 17:22:49.837076 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.827333, event: throttled, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4141> 2014-08-13 17:22:49.837083 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.827493, event: all_read, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4140> 2014-08-13 17:22:49.837092 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.837062, event: dispatched, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4139> 2014-08-13 17:22:49.837089 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started -4138> 2014-08-13 17:22:49.837103 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.837103, event: waiting_for_osdmap, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4137> 2014-08-13 17:22:49.837106 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start -4136> 2014-08-13 17:22:49.837115 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.837115, event: started, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4135> 2014-08-13 17:22:49.837128 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray -4134> 2014-08-13 17:22:49.837162 7f74d8735700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:22:49.837162, event: done, request: pg_notify(22.3af(22),22.f3(18) epoch 194471) v5 -4133> 2014-08-13 17:22:49.837159 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000053 0 0.000000 -4132> 2014-08-13 17:22:49.837181 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray -4131> 2014-08-13 17:22:49.837207 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 7 ==== pg_notify(22.99(1) epoch 194471) v5 ==== 777+0+0 (1420934118 0 0) 0x466b9c0 con 0x4d582c0 -4130> 2014-08-13 17:22:49.837230 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.825487, event: header_read, request: pg_notify(22.99(1) epoch 194471) v5 -4129> 2014-08-13 17:22:49.837241 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.825490, event: throttled, request: pg_notify(22.99(1) epoch 194471) v5 -4128> 2014-08-13 17:22:49.837250 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.825559, event: all_read, request: pg_notify(22.99(1) epoch 194471) v5 -4127> 2014-08-13 17:22:49.837260 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.837227, event: dispatched, request: pg_notify(22.99(1) epoch 194471) v5 -4126> 2014-08-13 17:22:49.837270 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.837270, event: waiting_for_osdmap, request: pg_notify(22.99(1) epoch 194471) v5 -4125> 2014-08-13 17:22:49.837281 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.837281, event: started, request: pg_notify(22.99(1) epoch 194471) v5 -4124> 2014-08-13 17:22:49.837305 7f74d8735700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:22:49.837305, event: done, request: pg_notify(22.99(1) epoch 194471) v5 -4123> 2014-08-13 17:22:49.837323 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 6 ==== pg_notify(24.9b(2) epoch 194471) v5 ==== 870+0+0 (3792926027 0 0) 0x53bd940 con 0x4d51ce0 -4122> 2014-08-13 17:22:49.837341 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.826996, event: header_read, request: pg_notify(24.9b(2) epoch 194471) v5 -4121> 2014-08-13 17:22:49.837351 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.826997, event: throttled, request: pg_notify(24.9b(2) epoch 194471) v5 -4120> 2014-08-13 17:22:49.837361 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.827062, event: all_read, request: pg_notify(24.9b(2) epoch 194471) v5 -4119> 2014-08-13 17:22:49.837371 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.837339, event: dispatched, request: pg_notify(24.9b(2) epoch 194471) v5 -4118> 2014-08-13 17:22:49.837381 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.837380, event: waiting_for_osdmap, request: pg_notify(24.9b(2) epoch 194471) v5 -4117> 2014-08-13 17:22:49.837425 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.837425, event: started, request: pg_notify(24.9b(2) epoch 194471) v5 -4116> 2014-08-13 17:22:49.837412 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.877652 5 0.000200 -4115> 2014-08-13 17:22:49.837440 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.898574 0 0.000000 -4114> 2014-08-13 17:22:49.837450 7f74d8735700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:22:49.837450, event: done, request: pg_notify(24.9b(2) epoch 194471) v5 -4113> 2014-08-13 17:22:49.837472 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194449 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4112> 2014-08-13 17:22:49.837482 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 6 ==== pg_notify(24.136(22) epoch 194471) v5 ==== 1978+0+0 (2287980691 0 0) 0x53bc0c0 con 0x4d51a20 -4111> 2014-08-13 17:22:49.837503 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.826004, event: header_read, request: pg_notify(24.136(22) epoch 194471) v5 -4110> 2014-08-13 17:22:49.837513 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.826005, event: throttled, request: pg_notify(24.136(22) epoch 194471) v5 -4109> 2014-08-13 17:22:49.837522 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.826096, event: all_read, request: pg_notify(24.136(22) epoch 194471) v5 -4108> 2014-08-13 17:22:49.837532 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.837500, event: dispatched, request: pg_notify(24.136(22) epoch 194471) v5 -4107> 2014-08-13 17:22:49.837542 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.837541, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194471) v5 -4106> 2014-08-13 17:22:49.837553 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.837553, event: started, request: pg_notify(24.136(22) epoch 194471) v5 -4105> 2014-08-13 17:22:49.837577 7f74d8735700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:22:49.837576, event: done, request: pg_notify(24.136(22) epoch 194471) v5 -4104> 2014-08-13 17:22:49.837607 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 6 ==== pg_notify(22.362(22) epoch 194471) v5 ==== 1948+0+0 (411326007 0 0) 0x53be580 con 0x4d55d80 -4103> 2014-08-13 17:22:49.837626 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.827508, event: header_read, request: pg_notify(22.362(22) epoch 194471) v5 -4102> 2014-08-13 17:22:49.837637 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.827510, event: throttled, request: pg_notify(22.362(22) epoch 194471) v5 -4101> 2014-08-13 17:22:49.837646 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.827623, event: all_read, request: pg_notify(22.362(22) epoch 194471) v5 -4100> 2014-08-13 17:22:49.837655 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.837624, event: dispatched, request: pg_notify(22.362(22) epoch 194471) v5 -4099> 2014-08-13 17:22:49.837665 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.837664, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194471) v5 -4098> 2014-08-13 17:22:49.837676 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.837675, event: started, request: pg_notify(22.362(22) epoch 194471) v5 -4097> 2014-08-13 17:22:49.837699 7f74d8735700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:22:49.837699, event: done, request: pg_notify(22.362(22) epoch 194471) v5 -4096> 2014-08-13 17:22:49.837701 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.21 192.168.1.107:6821/54786 -- pg_log(22.707 epoch 194471 log log((0'0,194215'483], crt=191727'417) query_epoch 194471) v4 -- ?+0 0x5b1bc00 -4095> 2014-08-13 17:22:49.837719 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 4 ==== pg_notify(22.3d9(42) epoch 194471) v5 ==== 2984+0+0 (2179894777 0 0) 0x53bec80 con 0x4d57a60 -4094> 2014-08-13 17:22:49.837737 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.828156, event: header_read, request: pg_notify(22.3d9(42) epoch 194471) v5 -4093> 2014-08-13 17:22:49.837748 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.828157, event: throttled, request: pg_notify(22.3d9(42) epoch 194471) v5 -4092> 2014-08-13 17:22:49.837757 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.828292, event: all_read, request: pg_notify(22.3d9(42) epoch 194471) v5 -4091> 2014-08-13 17:22:49.837766 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.837735, event: dispatched, request: pg_notify(22.3d9(42) epoch 194471) v5 -4090> 2014-08-13 17:22:49.837776 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.837776, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194471) v5 -4089> 2014-08-13 17:22:49.837787 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.837787, event: started, request: pg_notify(22.3d9(42) epoch 194471) v5 -4088> 2014-08-13 17:22:49.837810 7f74d8735700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:22:49.837810, event: done, request: pg_notify(22.3d9(42) epoch 194471) v5 -4087> 2014-08-13 17:22:49.837837 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 2 ==== pg_notify(24.34(35) epoch 194471) v5 ==== 2513+0+0 (2259962378 0 0) 0x53bb100 con 0x4d2aaa0 -4086> 2014-08-13 17:22:49.837856 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.825102, event: header_read, request: pg_notify(24.34(35) epoch 194471) v5 -4085> 2014-08-13 17:22:49.837867 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.825105, event: throttled, request: pg_notify(24.34(35) epoch 194471) v5 -4084> 2014-08-13 17:22:49.837876 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.825232, event: all_read, request: pg_notify(24.34(35) epoch 194471) v5 -4083> 2014-08-13 17:22:49.837885 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.837854, event: dispatched, request: pg_notify(24.34(35) epoch 194471) v5 -4082> 2014-08-13 17:22:49.837895 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.837895, event: waiting_for_osdmap, request: pg_notify(24.34(35) epoch 194471) v5 -4081> 2014-08-13 17:22:49.837906 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.837906, event: started, request: pg_notify(24.34(35) epoch 194471) v5 -4080> 2014-08-13 17:22:49.837931 7f74d8735700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:22:49.837931, event: done, request: pg_notify(24.34(35) epoch 194471) v5 -4079> 2014-08-13 17:22:49.837933 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.310 192.168.1.109:6812/32413 -- pg_log(22.707 epoch 194471 log log((0'0,194215'483], crt=191727'417) query_epoch 194471) v4 -- ?+0 0x5b1f800 -4078> 2014-08-13 17:22:49.837953 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 7 ==== pg_notify(22.1d2(22) epoch 194471) v5 ==== 1952+0+0 (1779296328 0 0) 0x466b800 con 0x45f1e40 -4077> 2014-08-13 17:22:49.837971 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.800559, event: header_read, request: pg_notify(22.1d2(22) epoch 194471) v5 -4076> 2014-08-13 17:22:49.837982 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.800561, event: throttled, request: pg_notify(22.1d2(22) epoch 194471) v5 -4075> 2014-08-13 17:22:49.837991 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.800714, event: all_read, request: pg_notify(22.1d2(22) epoch 194471) v5 -4074> 2014-08-13 17:22:49.838000 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.837969, event: dispatched, request: pg_notify(22.1d2(22) epoch 194471) v5 -4073> 2014-08-13 17:22:49.838010 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.838010, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194471) v5 -4072> 2014-08-13 17:22:49.838021 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.838021, event: started, request: pg_notify(22.1d2(22) epoch 194471) v5 -4071> 2014-08-13 17:22:49.838043 7f74d8735700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:22:49.838043, event: done, request: pg_notify(22.1d2(22) epoch 194471) v5 -4070> 2014-08-13 17:22:49.838069 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 4 ==== pg_info(1 pgs e194471:22.28f(2)) v4 ==== 824+0+0 (520946053 0 0) 0x49e0540 con 0x45f2940 -4069> 2014-08-13 17:22:49.838088 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.803105, event: header_read, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4068> 2014-08-13 17:22:49.838099 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.803106, event: throttled, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4067> 2014-08-13 17:22:49.838109 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.803260, event: all_read, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4066> 2014-08-13 17:22:49.838118 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.838086, event: dispatched, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4065> 2014-08-13 17:22:49.838103 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4064> 2014-08-13 17:22:49.838132 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.838132, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4063> 2014-08-13 17:22:49.838155 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.838154, event: started, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4062> 2014-08-13 17:22:49.838187 7f74d8735700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:22:49.838187, event: done, request: pg_info(1 pgs e194471:22.28f(2)) v4 -4061> 2014-08-13 17:22:49.838210 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 6 ==== pg_notify(22.84(2) epoch 194471) v5 ==== 828+0+0 (815700183 0 0) 0x49e1500 con 0x45f3b20 -4060> 2014-08-13 17:22:49.838230 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.809428, event: header_read, request: pg_notify(22.84(2) epoch 194471) v5 -4059> 2014-08-13 17:22:49.838240 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.809429, event: throttled, request: pg_notify(22.84(2) epoch 194471) v5 -4058> 2014-08-13 17:22:49.838250 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.809494, event: all_read, request: pg_notify(22.84(2) epoch 194471) v5 -4057> 2014-08-13 17:22:49.838258 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.838228, event: dispatched, request: pg_notify(22.84(2) epoch 194471) v5 -4056> 2014-08-13 17:22:49.838268 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.838268, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194471) v5 -4055> 2014-08-13 17:22:49.838279 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.838279, event: started, request: pg_notify(22.84(2) epoch 194471) v5 -4054> 2014-08-13 17:22:49.838303 7f74d8735700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:22:49.838302, event: done, request: pg_notify(22.84(2) epoch 194471) v5 -4053> 2014-08-13 17:22:49.838320 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 6 ==== pg_notify(24.30b(2) epoch 194471) v5 ==== 870+0+0 (552981272 0 0) 0x53b9c00 con 0x45f5540 -4052> 2014-08-13 17:22:49.838339 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.811867, event: header_read, request: pg_notify(24.30b(2) epoch 194471) v5 -4051> 2014-08-13 17:22:49.838349 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.811868, event: throttled, request: pg_notify(24.30b(2) epoch 194471) v5 -4050> 2014-08-13 17:22:49.838358 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.811930, event: all_read, request: pg_notify(24.30b(2) epoch 194471) v5 -4049> 2014-08-13 17:22:49.838367 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.838337, event: dispatched, request: pg_notify(24.30b(2) epoch 194471) v5 -4048> 2014-08-13 17:22:49.838377 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.838377, event: waiting_for_osdmap, request: pg_notify(24.30b(2) epoch 194471) v5 -4047> 2014-08-13 17:22:49.838388 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.838388, event: started, request: pg_notify(24.30b(2) epoch 194471) v5 -4046> 2014-08-13 17:22:49.838411 7f74d8735700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:22:49.838411, event: done, request: pg_notify(24.30b(2) epoch 194471) v5 -4045> 2014-08-13 17:22:49.838429 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 8 ==== pg_notify(24.136(22) epoch 194471) v5 ==== 1978+0+0 (2287980691 0 0) 0x49e2d80 con 0x45f5960 -4044> 2014-08-13 17:22:49.838447 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.803200, event: header_read, request: pg_notify(24.136(22) epoch 194471) v5 -4043> 2014-08-13 17:22:49.838458 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.803201, event: throttled, request: pg_notify(24.136(22) epoch 194471) v5 -4042> 2014-08-13 17:22:49.838467 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.803294, event: all_read, request: pg_notify(24.136(22) epoch 194471) v5 -4041> 2014-08-13 17:22:49.838475 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.838445, event: dispatched, request: pg_notify(24.136(22) epoch 194471) v5 -4040> 2014-08-13 17:22:49.838485 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.838485, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194471) v5 -4039> 2014-08-13 17:22:49.838496 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.838496, event: started, request: pg_notify(24.136(22) epoch 194471) v5 -4038> 2014-08-13 17:22:49.838519 7f74d8735700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:22:49.838519, event: done, request: pg_notify(24.136(22) epoch 194471) v5 -4037> 2014-08-13 17:22:49.838495 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started/Stray 0.898681 4 0.000314 -4036> 2014-08-13 17:22:49.838548 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 6 ==== pg_info(1 pgs e194471:22.30f(2)) v4 ==== 824+0+0 (1337021764 0 0) 0x57d5b00 con 0x45f6f60 -4035> 2014-08-13 17:22:49.838578 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.805381, event: header_read, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4034> 2014-08-13 17:22:49.838571 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started 0.898843 0 0.000000 -4033> 2014-08-13 17:22:49.838590 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.805384, event: throttled, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4032> 2014-08-13 17:22:49.838603 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194470/192968) [289,4,285]/[289,285] r=-1 lpr=194470 pi=193125-194469/36 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Reset -4031> 2014-08-13 17:22:49.838617 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.805491, event: all_read, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4030> 2014-08-13 17:22:49.838631 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.838576, event: dispatched, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4029> 2014-08-13 17:22:49.838642 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.838642, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4028> 2014-08-13 17:22:49.838654 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.838654, event: started, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4027> 2014-08-13 17:22:49.838656 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_log(2.2a epoch 194471 log log((0'0,194215'930], crt=0'0) query_epoch 194471) v4 -- ?+0 0x5b1e000 -4026> 2014-08-13 17:22:49.838679 7f74d8735700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:22:49.838679, event: done, request: pg_info(1 pgs e194471:22.30f(2)) v4 -4025> 2014-08-13 17:22:49.838696 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4024> 2014-08-13 17:22:49.838695 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 5 ==== pg_info(1 pgs e194471:22.2b9(2)) v4 ==== 824+0+0 (1930642137 0 0) 0x57d5080 con 0x4758b00 -4023> 2014-08-13 17:22:49.838751 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.808575, event: header_read, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4022> 2014-08-13 17:22:49.838762 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.808576, event: throttled, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4021> 2014-08-13 17:22:49.838744 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.000140 1 0.000191 -4020> 2014-08-13 17:22:49.838775 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.808636, event: all_read, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4019> 2014-08-13 17:22:49.838794 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.838748, event: dispatched, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4018> 2014-08-13 17:22:49.838790 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started -4017> 2014-08-13 17:22:49.838806 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.838806, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4016> 2014-08-13 17:22:49.838808 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start -4015> 2014-08-13 17:22:49.838817 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.838817, event: started, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4014> 2014-08-13 17:22:49.838830 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray -4013> 2014-08-13 17:22:49.838856 7f74d8735700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:22:49.838856, event: done, request: pg_info(1 pgs e194471:22.2b9(2)) v4 -4012> 2014-08-13 17:22:49.838852 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000043 0 0.000000 -4011> 2014-08-13 17:22:49.838896 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray -4010> 2014-08-13 17:22:49.838899 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 7 ==== pg_notify(24.3d(2),22.99(18) epoch 194471) v5 ==== 2578+0+0 (2416780821 0 0) 0x57d4d00 con 0x4758840 -4009> 2014-08-13 17:22:49.838930 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.824908, event: header_read, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4008> 2014-08-13 17:22:49.838943 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.824910, event: throttled, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4007> 2014-08-13 17:22:49.838953 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.825640, event: all_read, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4006> 2014-08-13 17:22:49.838963 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.838928, event: dispatched, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4005> 2014-08-13 17:22:49.838974 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.838974, event: waiting_for_osdmap, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4004> 2014-08-13 17:22:49.838986 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.838986, event: started, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4003> 2014-08-13 17:22:49.839028 7f74d8735700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:22:49.839028, event: done, request: pg_notify(24.3d(2),22.99(18) epoch 194471) v5 -4002> 2014-08-13 17:22:49.839064 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 4 ==== pg_notify(24.34(48) epoch 194471) v5 ==== 3196+0+0 (3690207742 0 0) 0x49e2bc0 con 0x475a940 -4001> 2014-08-13 17:22:49.839084 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.803097, event: header_read, request: pg_notify(24.34(48) epoch 194471) v5 -4000> 2014-08-13 17:22:49.839095 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.803098, event: throttled, request: pg_notify(24.34(48) epoch 194471) v5 -3999> 2014-08-13 17:22:49.839105 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.803244, event: all_read, request: pg_notify(24.34(48) epoch 194471) v5 -3998> 2014-08-13 17:22:49.839114 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.839082, event: dispatched, request: pg_notify(24.34(48) epoch 194471) v5 -3997> 2014-08-13 17:22:49.839124 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.839123, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194471) v5 -3996> 2014-08-13 17:22:49.839135 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.839135, event: started, request: pg_notify(24.34(48) epoch 194471) v5 -3995> 2014-08-13 17:22:49.839166 7f74d8735700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:22:49.839166, event: done, request: pg_notify(24.34(48) epoch 194471) v5 -3994> 2014-08-13 17:22:49.839219 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 5 ==== pg_query(24.1e6 epoch 194471) v3 ==== 144+0+0 (1469893023 0 0) 0x29c5dc0 con 0x475cd00 -3993> 2014-08-13 17:22:49.839240 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.806001, event: header_read, request: pg_query(24.1e6 epoch 194471) v3 -3992> 2014-08-13 17:22:49.839251 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.806002, event: throttled, request: pg_query(24.1e6 epoch 194471) v3 -3991> 2014-08-13 17:22:49.839260 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.806046, event: all_read, request: pg_query(24.1e6 epoch 194471) v3 -3990> 2014-08-13 17:22:49.839269 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.839238, event: dispatched, request: pg_query(24.1e6 epoch 194471) v3 -3989> 2014-08-13 17:22:49.839279 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.839279, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194471) v3 -3988> 2014-08-13 17:22:49.839291 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.839290, event: started, request: pg_query(24.1e6 epoch 194471) v3 -3987> 2014-08-13 17:22:49.839311 7f74d8735700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:22:49.839311, event: done, request: pg_query(24.1e6 epoch 194471) v3 -3986> 2014-08-13 17:22:49.839328 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 5 ==== pg_info(1 pgs e194471:22.29a(2)) v4 ==== 824+0+0 (2601723123 0 0) 0x57d7000 con 0x475c4c0 -3985> 2014-08-13 17:22:49.839348 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.797924, event: header_read, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3984> 2014-08-13 17:22:49.839358 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.797925, event: throttled, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3983> 2014-08-13 17:22:49.839367 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.798007, event: all_read, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3982> 2014-08-13 17:22:49.839377 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.839346, event: dispatched, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3981> 2014-08-13 17:22:49.839386 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.839386, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3980> 2014-08-13 17:22:49.839398 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.839398, event: started, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3979> 2014-08-13 17:22:49.839421 7f74d8735700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:22:49.839421, event: done, request: pg_info(1 pgs e194471:22.29a(2)) v4 -3978> 2014-08-13 17:22:49.839439 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 4 ==== pg_info(1 pgs e194471:22.630(2)) v4 ==== 824+0+0 (3896040755 0 0) 0x49e01c0 con 0x475d960 -3977> 2014-08-13 17:22:49.839469 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.803038, event: header_read, request: pg_info(1 pgs e194471:22.630(2)) v4 -3976> 2014-08-13 17:22:49.839479 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.803038, event: throttled, request: pg_info(1 pgs e194471:22.630(2)) v4 -3975> 2014-08-13 17:22:49.839489 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.803102, event: all_read, request: pg_info(1 pgs e194471:22.630(2)) v4 -3974> 2014-08-13 17:22:49.839498 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.839466, event: dispatched, request: pg_info(1 pgs e194471:22.630(2)) v4 -3973> 2014-08-13 17:22:49.839508 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.839508, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.630(2)) v4 -3972> 2014-08-13 17:22:49.839519 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.839519, event: started, request: pg_info(1 pgs e194471:22.630(2)) v4 -3971> 2014-08-13 17:22:49.839542 7f74d8735700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:22:49.839541, event: done, request: pg_info(1 pgs e194471:22.630(2)) v4 -3970> 2014-08-13 17:22:49.839564 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 11 ==== pg_notify(22.707(1),24.3d(24) epoch 194471) v5 ==== 2861+0+0 (119202557 0 0) 0x5264280 con 0x475ee00 -3969> 2014-08-13 17:22:49.839584 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.810764, event: header_read, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3968> 2014-08-13 17:22:49.839595 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.810765, event: throttled, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3967> 2014-08-13 17:22:49.839605 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.810881, event: all_read, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3966> 2014-08-13 17:22:49.839614 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.839582, event: dispatched, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3965> 2014-08-13 17:22:49.839625 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.839625, event: waiting_for_osdmap, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3964> 2014-08-13 17:22:49.839637 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.839637, event: started, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3963> 2014-08-13 17:22:49.839697 7f74d8735700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:22:49.839696, event: done, request: pg_notify(22.707(1),24.3d(24) epoch 194471) v5 -3962> 2014-08-13 17:22:49.839739 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 4 ==== pg_info(1 pgs e194471:22.4aa(2)) v4 ==== 824+0+0 (2209767621 0 0) 0x466e900 con 0x475e5c0 -3961> 2014-08-13 17:22:49.839769 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.797923, event: header_read, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3960> 2014-08-13 17:22:49.839780 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.797926, event: throttled, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3959> 2014-08-13 17:22:49.839789 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.798008, event: all_read, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3958> 2014-08-13 17:22:49.839799 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.839757, event: dispatched, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3957> 2014-08-13 17:22:49.839809 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.839808, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3956> 2014-08-13 17:22:49.839820 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.839820, event: started, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3955> 2014-08-13 17:22:49.839844 7f74d8735700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:22:49.839843, event: done, request: pg_info(1 pgs e194471:22.4aa(2)) v4 -3954> 2014-08-13 17:22:49.839862 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 8 ==== pg_notify(22.6bd(20) epoch 194471) v5 ==== 1838+0+0 (1168528109 0 0) 0x466f380 con 0x47a27e0 -3953> 2014-08-13 17:22:49.839881 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.798987, event: header_read, request: pg_notify(22.6bd(20) epoch 194471) v5 -3952> 2014-08-13 17:22:49.839892 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.798989, event: throttled, request: pg_notify(22.6bd(20) epoch 194471) v5 -3951> 2014-08-13 17:22:49.839901 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.799084, event: all_read, request: pg_notify(22.6bd(20) epoch 194471) v5 -3950> 2014-08-13 17:22:49.839910 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.839879, event: dispatched, request: pg_notify(22.6bd(20) epoch 194471) v5 -3949> 2014-08-13 17:22:49.839920 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.839920, event: waiting_for_osdmap, request: pg_notify(22.6bd(20) epoch 194471) v5 -3948> 2014-08-13 17:22:49.839931 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.839931, event: started, request: pg_notify(22.6bd(20) epoch 194471) v5 -3947> 2014-08-13 17:22:49.839954 7f74d8735700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:22:49.839954, event: done, request: pg_notify(22.6bd(20) epoch 194471) v5 -3946> 2014-08-13 17:22:49.839976 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 9 ==== pg_notify(22.3af(1) epoch 194471) v5 ==== 777+0+0 (357522609 0 0) 0x53bc7c0 con 0x47a0b00 -3945> 2014-08-13 17:22:49.839994 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.826901, event: header_read, request: pg_notify(22.3af(1) epoch 194471) v5 -3944> 2014-08-13 17:22:49.840005 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.826903, event: throttled, request: pg_notify(22.3af(1) epoch 194471) v5 -3943> 2014-08-13 17:22:49.840014 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.826963, event: all_read, request: pg_notify(22.3af(1) epoch 194471) v5 -3942> 2014-08-13 17:22:49.840023 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.839992, event: dispatched, request: pg_notify(22.3af(1) epoch 194471) v5 -3941> 2014-08-13 17:22:49.840033 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.840033, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194471) v5 -3940> 2014-08-13 17:22:49.840044 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.840044, event: started, request: pg_notify(22.3af(1) epoch 194471) v5 -3939> 2014-08-13 17:22:49.840069 7f74d8735700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:22:49.840069, event: done, request: pg_notify(22.3af(1) epoch 194471) v5 -3938> 2014-08-13 17:22:49.840087 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 4 ==== pg_query(24.1de epoch 194471) v3 ==== 144+0+0 (369764677 0 0) 0x4eb14a0 con 0x47a4200 -3937> 2014-08-13 17:22:49.840105 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.809133, event: header_read, request: pg_query(24.1de epoch 194471) v3 -3936> 2014-08-13 17:22:49.840115 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.809134, event: throttled, request: pg_query(24.1de epoch 194471) v3 -3935> 2014-08-13 17:22:49.840124 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.809221, event: all_read, request: pg_query(24.1de epoch 194471) v3 -3934> 2014-08-13 17:22:49.840133 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.840103, event: dispatched, request: pg_query(24.1de epoch 194471) v3 -3933> 2014-08-13 17:22:49.840143 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.840142, event: waiting_for_osdmap, request: pg_query(24.1de epoch 194471) v3 -3932> 2014-08-13 17:22:49.840154 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.840153, event: started, request: pg_query(24.1de epoch 194471) v3 -3931> 2014-08-13 17:22:49.840172 7f74d8735700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:22:49.840172, event: done, request: pg_query(24.1de epoch 194471) v3 -3930> 2014-08-13 17:22:49.840197 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 6 ==== pg_query(24.2a1 epoch 194471) v3 ==== 144+0+0 (3364580854 0 0) 0x4eb3c00 con 0x47a53e0 -3929> 2014-08-13 17:22:49.840216 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.797592, event: header_read, request: pg_query(24.2a1 epoch 194471) v3 -3928> 2014-08-13 17:22:49.840226 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.797593, event: throttled, request: pg_query(24.2a1 epoch 194471) v3 -3927> 2014-08-13 17:22:49.840235 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.797641, event: all_read, request: pg_query(24.2a1 epoch 194471) v3 -3926> 2014-08-13 17:22:49.840244 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.840214, event: dispatched, request: pg_query(24.2a1 epoch 194471) v3 -3925> 2014-08-13 17:22:49.840254 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.840254, event: waiting_for_osdmap, request: pg_query(24.2a1 epoch 194471) v3 -3924> 2014-08-13 17:22:49.840264 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.840264, event: started, request: pg_query(24.2a1 epoch 194471) v3 -3923> 2014-08-13 17:22:49.840283 7f74d8735700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:22:49.840283, event: done, request: pg_query(24.2a1 epoch 194471) v3 -3922> 2014-08-13 17:22:49.840309 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 6 ==== pg_notify(22.619(1) epoch 194471) v5 ==== 777+0+0 (1156692937 0 0) 0x53bd400 con 0x47a4ba0 -3921> 2014-08-13 17:22:49.840327 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.826983, event: header_read, request: pg_notify(22.619(1) epoch 194471) v5 -3920> 2014-08-13 17:22:49.840338 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.826985, event: throttled, request: pg_notify(22.619(1) epoch 194471) v5 -3919> 2014-08-13 17:22:49.840347 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.827103, event: all_read, request: pg_notify(22.619(1) epoch 194471) v5 -3918> 2014-08-13 17:22:49.840356 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.840325, event: dispatched, request: pg_notify(22.619(1) epoch 194471) v5 -3917> 2014-08-13 17:22:49.840366 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.840366, event: waiting_for_osdmap, request: pg_notify(22.619(1) epoch 194471) v5 -3916> 2014-08-13 17:22:49.840377 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.840377, event: started, request: pg_notify(22.619(1) epoch 194471) v5 -3915> 2014-08-13 17:22:49.840400 7f74d8735700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:22:49.840400, event: done, request: pg_notify(22.619(1) epoch 194471) v5 -3914> 2014-08-13 17:22:49.840417 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 7 ==== pg_notify(22.3e4(22) epoch 194471) v5 ==== 1948+0+0 (3296293175 0 0) 0x57d1c00 con 0x47a32e0 -3913> 2014-08-13 17:22:49.840435 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.798932, event: header_read, request: pg_notify(22.3e4(22) epoch 194471) v5 -3912> 2014-08-13 17:22:49.840446 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.798934, event: throttled, request: pg_notify(22.3e4(22) epoch 194471) v5 -3911> 2014-08-13 17:22:49.840455 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.799033, event: all_read, request: pg_notify(22.3e4(22) epoch 194471) v5 -3910> 2014-08-13 17:22:49.840464 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.840433, event: dispatched, request: pg_notify(22.3e4(22) epoch 194471) v5 -3909> 2014-08-13 17:22:49.840474 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.840474, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194471) v5 -3908> 2014-08-13 17:22:49.840485 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.840485, event: started, request: pg_notify(22.3e4(22) epoch 194471) v5 -3907> 2014-08-13 17:22:49.840507 7f74d8735700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:22:49.840507, event: done, request: pg_notify(22.3e4(22) epoch 194471) v5 -3906> 2014-08-13 17:22:49.840529 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 6 ==== pg_notify(22.84(20) epoch 194471) v5 ==== 1838+0+0 (4068084929 0 0) 0x466eac0 con 0x47f0c60 -3905> 2014-08-13 17:22:49.840547 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.798776, event: header_read, request: pg_notify(22.84(20) epoch 194471) v5 -3904> 2014-08-13 17:22:49.840558 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.798778, event: throttled, request: pg_notify(22.84(20) epoch 194471) v5 -3903> 2014-08-13 17:22:49.840567 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.798876, event: all_read, request: pg_notify(22.84(20) epoch 194471) v5 -3902> 2014-08-13 17:22:49.840576 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.840545, event: dispatched, request: pg_notify(22.84(20) epoch 194471) v5 -3901> 2014-08-13 17:22:49.840586 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.840585, event: waiting_for_osdmap, request: pg_notify(22.84(20) epoch 194471) v5 -3900> 2014-08-13 17:22:49.840597 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.840596, event: started, request: pg_notify(22.84(20) epoch 194471) v5 -3899> 2014-08-13 17:22:49.840619 7f74d8735700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:22:49.840619, event: done, request: pg_notify(22.84(20) epoch 194471) v5 -3898> 2014-08-13 17:22:49.840641 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 4 ==== pg_info(1 pgs e194471:22.2d1(2)) v4 ==== 824+0+0 (2058479497 0 0) 0x49e7540 con 0x47f2680 -3897> 2014-08-13 17:22:49.840660 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.806533, event: header_read, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3896> 2014-08-13 17:22:49.840673 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.806534, event: throttled, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3895> 2014-08-13 17:22:49.840683 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.806600, event: all_read, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3894> 2014-08-13 17:22:49.840692 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.840657, event: dispatched, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3893> 2014-08-13 17:22:49.840702 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.840702, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3892> 2014-08-13 17:22:49.840714 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.840714, event: started, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3891> 2014-08-13 17:22:49.840738 7f74d8735700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:22:49.840737, event: done, request: pg_info(1 pgs e194471:22.2d1(2)) v4 -3890> 2014-08-13 17:22:49.840755 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 9 ==== pg_notify(24.181(3) epoch 194471) v5 ==== 925+0+0 (3893936117 0 0) 0x53bad80 con 0x47f7e80 -3889> 2014-08-13 17:22:49.840774 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.812620, event: header_read, request: pg_notify(24.181(3) epoch 194471) v5 -3888> 2014-08-13 17:22:49.840784 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.812624, event: throttled, request: pg_notify(24.181(3) epoch 194471) v5 -3887> 2014-08-13 17:22:49.840794 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.813019, event: all_read, request: pg_notify(24.181(3) epoch 194471) v5 -3886> 2014-08-13 17:22:49.840803 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.840772, event: dispatched, request: pg_notify(24.181(3) epoch 194471) v5 -3885> 2014-08-13 17:22:49.840813 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.840813, event: waiting_for_osdmap, request: pg_notify(24.181(3) epoch 194471) v5 -3884> 2014-08-13 17:22:49.840824 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.840824, event: started, request: pg_notify(24.181(3) epoch 194471) v5 -3883> 2014-08-13 17:22:49.840849 7f74d8735700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:22:49.840848, event: done, request: pg_notify(24.181(3) epoch 194471) v5 -3882> 2014-08-13 17:22:49.840866 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 9 ==== pg_info(1 pgs e194471:22.4fe(2)) v4 ==== 824+0+0 (679681911 0 0) 0x57d7700 con 0x4833440 -3881> 2014-08-13 17:22:49.840885 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.803380, event: header_read, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3880> 2014-08-13 17:22:49.840896 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.803382, event: throttled, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3879> 2014-08-13 17:22:49.840905 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.803442, event: all_read, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3878> 2014-08-13 17:22:49.840914 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.840883, event: dispatched, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3877> 2014-08-13 17:22:49.840924 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.840924, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3876> 2014-08-13 17:22:49.840935 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.840935, event: started, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3875> 2014-08-13 17:22:49.840958 7f74d8735700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:22:49.840957, event: done, request: pg_info(1 pgs e194471:22.4fe(2)) v4 -3874> 2014-08-13 17:22:49.840977 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 7 ==== pg_info(1 pgs e194471:22.653(1)) v4 ==== 773+0+0 (641672664 0 0) 0x4ac4b40 con 0x4832c00 -3873> 2014-08-13 17:22:49.840995 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.803568, event: header_read, request: pg_info(1 pgs e194471:22.653(1)) v4 -3872> 2014-08-13 17:22:49.841006 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.803569, event: throttled, request: pg_info(1 pgs e194471:22.653(1)) v4 -3871> 2014-08-13 17:22:49.841016 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.803643, event: all_read, request: pg_info(1 pgs e194471:22.653(1)) v4 -3870> 2014-08-13 17:22:49.841025 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.840993, event: dispatched, request: pg_info(1 pgs e194471:22.653(1)) v4 -3869> 2014-08-13 17:22:49.841034 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.841034, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.653(1)) v4 -3868> 2014-08-13 17:22:49.841046 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.841045, event: started, request: pg_info(1 pgs e194471:22.653(1)) v4 -3867> 2014-08-13 17:22:49.841068 7f74d8735700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:22:49.841068, event: done, request: pg_info(1 pgs e194471:22.653(1)) v4 -3866> 2014-08-13 17:22:49.841086 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 12 ==== pg_notify(22.48e(20) epoch 194471) v5 ==== 1834+0+0 (2766538147 0 0) 0x4ac4980 con 0x4835280 -3865> 2014-08-13 17:22:49.841104 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.802050, event: header_read, request: pg_notify(22.48e(20) epoch 194471) v5 -3864> 2014-08-13 17:22:49.841115 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.802053, event: throttled, request: pg_notify(22.48e(20) epoch 194471) v5 -3863> 2014-08-13 17:22:49.841124 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.802185, event: all_read, request: pg_notify(22.48e(20) epoch 194471) v5 -3862> 2014-08-13 17:22:49.841134 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.841102, event: dispatched, request: pg_notify(22.48e(20) epoch 194471) v5 -3861> 2014-08-13 17:22:49.841143 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.841143, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194471) v5 -3860> 2014-08-13 17:22:49.841154 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.841154, event: started, request: pg_notify(22.48e(20) epoch 194471) v5 -3859> 2014-08-13 17:22:49.841184 7f74d8735700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:22:49.841183, event: done, request: pg_notify(22.48e(20) epoch 194471) v5 -3858> 2014-08-13 17:22:49.841216 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 8 ==== pg_query(24.206 epoch 194471) v3 ==== 144+0+0 (744140975 0 0) 0x4eb3de0 con 0x48377a0 -3857> 2014-08-13 17:22:49.841235 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.810500, event: header_read, request: pg_query(24.206 epoch 194471) v3 -3856> 2014-08-13 17:22:49.841246 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.810501, event: throttled, request: pg_query(24.206 epoch 194471) v3 -3855> 2014-08-13 17:22:49.841255 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.810577, event: all_read, request: pg_query(24.206 epoch 194471) v3 -3854> 2014-08-13 17:22:49.841264 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.841233, event: dispatched, request: pg_query(24.206 epoch 194471) v3 -3853> 2014-08-13 17:22:49.841273 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.841273, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194471) v3 -3852> 2014-08-13 17:22:49.841284 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.841284, event: started, request: pg_query(24.206 epoch 194471) v3 -3851> 2014-08-13 17:22:49.841303 7f74d8735700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:22:49.841303, event: done, request: pg_query(24.206 epoch 194471) v3 -3850> 2014-08-13 17:22:49.841332 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 7 ==== pg_notify(24.1f7(28) epoch 194471) v5 ==== 2332+0+0 (1351492979 0 0) 0x49e2680 con 0x4868420 -3849> 2014-08-13 17:22:49.841351 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.801038, event: header_read, request: pg_notify(24.1f7(28) epoch 194471) v5 -3848> 2014-08-13 17:22:49.841362 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.801040, event: throttled, request: pg_notify(24.1f7(28) epoch 194471) v5 -3847> 2014-08-13 17:22:49.841371 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.801155, event: all_read, request: pg_notify(24.1f7(28) epoch 194471) v5 -3846> 2014-08-13 17:22:49.841380 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.841349, event: dispatched, request: pg_notify(24.1f7(28) epoch 194471) v5 -3845> 2014-08-13 17:22:49.841390 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.841390, event: waiting_for_osdmap, request: pg_notify(24.1f7(28) epoch 194471) v5 -3844> 2014-08-13 17:22:49.841401 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.841401, event: started, request: pg_notify(24.1f7(28) epoch 194471) v5 -3843> 2014-08-13 17:22:49.841426 7f74d8735700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:22:49.841425, event: done, request: pg_notify(24.1f7(28) epoch 194471) v5 -3842> 2014-08-13 17:22:49.841453 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 9 ==== pg_info(1 pgs e194471:22.581(1)) v4 ==== 773+0+0 (4053632776 0 0) 0x49e0700 con 0x486bc80 -3841> 2014-08-13 17:22:49.841472 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.799821, event: header_read, request: pg_info(1 pgs e194471:22.581(1)) v4 -3840> 2014-08-13 17:22:49.841483 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.799823, event: throttled, request: pg_info(1 pgs e194471:22.581(1)) v4 -3839> 2014-08-13 17:22:49.841492 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.799888, event: all_read, request: pg_info(1 pgs e194471:22.581(1)) v4 -3838> 2014-08-13 17:22:49.841501 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.841470, event: dispatched, request: pg_info(1 pgs e194471:22.581(1)) v4 -3837> 2014-08-13 17:22:49.841511 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.841511, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.581(1)) v4 -3836> 2014-08-13 17:22:49.841522 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.841522, event: started, request: pg_info(1 pgs e194471:22.581(1)) v4 -3835> 2014-08-13 17:22:49.841545 7f74d8735700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:22:49.841545, event: done, request: pg_info(1 pgs e194471:22.581(1)) v4 -3834> 2014-08-13 17:22:49.841573 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 6 ==== pg_info(1 pgs e194471:22.99) v4 ==== 714+0+0 (1465675444 0 0) 0x57d2300 con 0x486b9c0 -3833> 2014-08-13 17:22:49.841592 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.838751, event: header_read, request: pg_info(1 pgs e194471:22.99) v4 -3832> 2014-08-13 17:22:49.841603 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.838754, event: throttled, request: pg_info(1 pgs e194471:22.99) v4 -3831> 2014-08-13 17:22:49.841612 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.838832, event: all_read, request: pg_info(1 pgs e194471:22.99) v4 -3830> 2014-08-13 17:22:49.841621 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.841590, event: dispatched, request: pg_info(1 pgs e194471:22.99) v4 -3829> 2014-08-13 17:22:49.841630 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.841630, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.99) v4 -3828> 2014-08-13 17:22:49.841641 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.841641, event: started, request: pg_info(1 pgs e194471:22.99) v4 -3827> 2014-08-13 17:22:49.841695 7f74d8735700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:22:49.841694, event: done, request: pg_info(1 pgs e194471:22.99) v4 -3826> 2014-08-13 17:22:49.841719 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.883844 5 0.000200 -3825> 2014-08-13 17:22:49.841750 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 8 ==== pg_notify(24.b6(2) epoch 194471) v5 ==== 870+0+0 (926475702 0 0) 0x4ac47c0 con 0x486b700 -3824> 2014-08-13 17:22:49.841757 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.941536 0 0.000000 -3823> 2014-08-13 17:22:49.841770 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.825769, event: header_read, request: pg_notify(24.b6(2) epoch 194471) v5 -3822> 2014-08-13 17:22:49.841793 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.825771, event: throttled, request: pg_notify(24.b6(2) epoch 194471) v5 -3821> 2014-08-13 17:22:49.841803 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.825875, event: all_read, request: pg_notify(24.b6(2) epoch 194471) v5 -3820> 2014-08-13 17:22:49.841790 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194449 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3819> 2014-08-13 17:22:49.841816 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.841767, event: dispatched, request: pg_notify(24.b6(2) epoch 194471) v5 -3818> 2014-08-13 17:22:49.841823 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.841823, event: waiting_for_osdmap, request: pg_notify(24.b6(2) epoch 194471) v5 -3817> 2014-08-13 17:22:49.841834 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.841834, event: started, request: pg_notify(24.b6(2) epoch 194471) v5 -3816> 2014-08-13 17:22:49.841794 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.881834 4 0.000251 -3815> 2014-08-13 17:22:49.841859 7f74d8735700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:22:49.841859, event: done, request: pg_notify(24.b6(2) epoch 194471) v5 -3814> 2014-08-13 17:22:49.841866 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.901135 0 0.000000 -3813> 2014-08-13 17:22:49.841889 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 10 ==== pg_notify(22.3d9(35) epoch 194471) v5 ==== 2627+0+0 (3370785586 0 0) 0x53b9f80 con 0x486a3c0 -3812> 2014-08-13 17:22:49.841910 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.804832, event: header_read, request: pg_notify(22.3d9(35) epoch 194471) v5 -3811> 2014-08-13 17:22:49.841921 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.804834, event: throttled, request: pg_notify(22.3d9(35) epoch 194471) v5 -3810> 2014-08-13 17:22:49.841901 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194449 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3809> 2014-08-13 17:22:49.841930 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.804970, event: all_read, request: pg_notify(22.3d9(35) epoch 194471) v5 -3808> 2014-08-13 17:22:49.841939 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.841908, event: dispatched, request: pg_notify(22.3d9(35) epoch 194471) v5 -3807> 2014-08-13 17:22:49.841949 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.841949, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194471) v5 -3806> 2014-08-13 17:22:49.841960 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.841960, event: started, request: pg_notify(22.3d9(35) epoch 194471) v5 -3805> 2014-08-13 17:22:49.841983 7f74d8735700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:22:49.841983, event: done, request: pg_notify(22.3d9(35) epoch 194471) v5 -3804> 2014-08-13 17:22:49.842016 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 7 ==== pg_query(24.20e epoch 194471) v3 ==== 144+0+0 (2791671660 0 0) 0x29c6ea0 con 0x48d9ce0 -3803> 2014-08-13 17:22:49.842035 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.807827, event: header_read, request: pg_query(24.20e epoch 194471) v3 -3802> 2014-08-13 17:22:49.842046 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.807829, event: throttled, request: pg_query(24.20e epoch 194471) v3 -3801> 2014-08-13 17:22:49.842055 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.807893, event: all_read, request: pg_query(24.20e epoch 194471) v3 -3800> 2014-08-13 17:22:49.842064 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.842033, event: dispatched, request: pg_query(24.20e epoch 194471) v3 -3799> 2014-08-13 17:22:49.842055 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.45 192.168.1.112:6828/6040 -- pg_log(22.113 epoch 194471 log log((0'0,194215'528], crt=191648'383) query_epoch 194471) v4 -- ?+0 0x5b1da00 -3798> 2014-08-13 17:22:49.842074 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.842074, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194471) v3 -3797> 2014-08-13 17:22:49.842085 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.842085, event: started, request: pg_query(24.20e epoch 194471) v3 -3796> 2014-08-13 17:22:49.842104 7f74d8735700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:22:49.842104, event: done, request: pg_query(24.20e epoch 194471) v3 -3795> 2014-08-13 17:22:49.842131 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 8 ==== pg_notify(22.1d2(22) epoch 194471) v5 ==== 1952+0+0 (1779296328 0 0) 0x57d5cc0 con 0x486e040 -3794> 2014-08-13 17:22:49.842150 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.800517, event: header_read, request: pg_notify(22.1d2(22) epoch 194471) v5 -3793> 2014-08-13 17:22:49.842160 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.800519, event: throttled, request: pg_notify(22.1d2(22) epoch 194471) v5 -3792> 2014-08-13 17:22:49.842169 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.800630, event: all_read, request: pg_notify(22.1d2(22) epoch 194471) v5 -3791> 2014-08-13 17:22:49.842186 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.842147, event: dispatched, request: pg_notify(22.1d2(22) epoch 194471) v5 -3790> 2014-08-13 17:22:49.842196 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.842196, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194471) v5 -3789> 2014-08-13 17:22:49.842206 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.842206, event: started, request: pg_notify(22.1d2(22) epoch 194471) v5 -3788> 2014-08-13 17:22:49.842229 7f74d8735700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:22:49.842229, event: done, request: pg_notify(22.1d2(22) epoch 194471) v5 -3787> 2014-08-13 17:22:49.842271 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 6 ==== pg_info(1 pgs e194471:22.3b9(2)) v4 ==== 824+0+0 (2009382427 0 0) 0x4ac4280 con 0x48dfbc0 -3786> 2014-08-13 17:22:49.842291 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.804228, event: header_read, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3785> 2014-08-13 17:22:49.842302 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.804229, event: throttled, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3784> 2014-08-13 17:22:49.842311 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.804473, event: all_read, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3783> 2014-08-13 17:22:49.842308 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.339 192.168.1.110:6818/45268 -- pg_log(22.113 epoch 194471 log log((0'0,194215'528], crt=191648'383) query_epoch 194471) v4 -- ?+0 0x5b1d400 -3782> 2014-08-13 17:22:49.842321 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.842289, event: dispatched, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3781> 2014-08-13 17:22:49.842342 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.842342, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3780> 2014-08-13 17:22:49.842355 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.842355, event: started, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3779> 2014-08-13 17:22:49.842354 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.5 192.168.1.103:6827/40192 -- pg_log(22.1f1 epoch 194471 log log((0'0,194215'493], crt=191648'346) query_epoch 194471) v4 -- ?+0 0x5b1f200 -3778> 2014-08-13 17:22:49.842380 7f74d8735700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:22:49.842379, event: done, request: pg_info(1 pgs e194471:22.3b9(2)) v4 -3777> 2014-08-13 17:22:49.842409 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 7 ==== pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 ==== 154289+0+0 (262411742 0 0) 0x278da00 con 0x4900580 -3776> 2014-08-13 17:22:49.842432 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.802320, event: header_read, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3775> 2014-08-13 17:22:49.842445 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.802323, event: throttled, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3774> 2014-08-13 17:22:49.842457 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.824196, event: all_read, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3773> 2014-08-13 17:22:49.842470 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.842429, event: dispatched, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3772> 2014-08-13 17:22:49.842478 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.842478, event: waiting_for_osdmap, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3771> 2014-08-13 17:22:49.842493 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.842493, event: started, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3770> 2014-08-13 17:22:49.842531 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3769> 2014-08-13 17:22:49.842613 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299(unlocked)] enter Initial -3768> 2014-08-13 17:22:49.842689 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 inactive] exit Initial 0.000077 0 0.000000 -3767> 2014-08-13 17:22:49.842711 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 inactive] enter Reset -3766> 2014-08-13 17:22:49.842729 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] exit Reset 0.000017 1 0.000039 -3765> 2014-08-13 17:22:49.842745 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Started -3764> 2014-08-13 17:22:49.842767 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Start -3763> 2014-08-13 17:22:49.842764 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.47 192.168.1.111:6828/12102 -- pg_log(22.1f1 epoch 194471 log log((0'0,194215'493], crt=191648'346) query_epoch 194471) v4 -- ?+0 0x5b1ec00 -3762> 2014-08-13 17:22:49.842784 7f74d8735700 1 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] state: transitioning to Stray -3761> 2014-08-13 17:22:49.842817 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] exit Start 0.000050 0 0.000000 -3760> 2014-08-13 17:22:49.842839 7f74d8735700 5 osd.4 pg_epoch: 194471 pg[24.299( empty local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 inactive] enter Started/Stray -3759> 2014-08-13 17:22:49.842881 7f74d8735700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:22:49.842881, event: done, request: pg_log(24.299 epoch 194471 log log((192834'13573,194422'15411], crt=192888'14914) query_epoch 194471) v4 -3758> 2014-08-13 17:22:49.842911 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 10 ==== pg_notify(22.3e4(22) epoch 194471) v5 ==== 1948+0+0 (3296293175 0 0) 0x53bcd00 con 0x4901600 -3757> 2014-08-13 17:22:49.842934 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.827084, event: header_read, request: pg_notify(22.3e4(22) epoch 194471) v5 -3756> 2014-08-13 17:22:49.842945 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.827086, event: throttled, request: pg_notify(22.3e4(22) epoch 194471) v5 -3755> 2014-08-13 17:22:49.842954 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.827187, event: all_read, request: pg_notify(22.3e4(22) epoch 194471) v5 -3754> 2014-08-13 17:22:49.842968 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.842931, event: dispatched, request: pg_notify(22.3e4(22) epoch 194471) v5 -3753> 2014-08-13 17:22:49.842947 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started/Stray 0.927527 4 0.000260 -3752> 2014-08-13 17:22:49.842975 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.842975, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194471) v5 -3751> 2014-08-13 17:22:49.842983 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.842983, event: started, request: pg_notify(22.3e4(22) epoch 194471) v5 -3750> 2014-08-13 17:22:49.842979 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started 0.927662 0 0.000000 -3749> 2014-08-13 17:22:49.843005 7f74d8735700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:22:49.843004, event: done, request: pg_notify(22.3e4(22) epoch 194471) v5 -3748> 2014-08-13 17:22:49.843001 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194470/193250) [304,323,4]/[304,323] r=-1 lpr=194470 pi=193250-194469/23 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Reset -3747> 2014-08-13 17:22:49.843043 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 8 ==== pg_notify(22.22c(20) epoch 194471) v5 ==== 1834+0+0 (1263931192 0 0) 0x57d39c0 con 0x4902680 -3746> 2014-08-13 17:22:49.843059 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.828094, event: header_read, request: pg_notify(22.22c(20) epoch 194471) v5 -3745> 2014-08-13 17:22:49.843066 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.828096, event: throttled, request: pg_notify(22.22c(20) epoch 194471) v5 -3744> 2014-08-13 17:22:49.843074 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.828197, event: all_read, request: pg_notify(22.22c(20) epoch 194471) v5 -3743> 2014-08-13 17:22:49.843084 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.843057, event: dispatched, request: pg_notify(22.22c(20) epoch 194471) v5 -3742> 2014-08-13 17:22:49.843094 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.843094, event: waiting_for_osdmap, request: pg_notify(22.22c(20) epoch 194471) v5 -3741> 2014-08-13 17:22:49.843105 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.843105, event: started, request: pg_notify(22.22c(20) epoch 194471) v5 -3740> 2014-08-13 17:22:49.843129 7f74d8735700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:22:49.843129, event: done, request: pg_notify(22.22c(20) epoch 194471) v5 -3739> 2014-08-13 17:22:49.843115 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.000113 1 0.000109 -3738> 2014-08-13 17:22:49.843144 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started -3737> 2014-08-13 17:22:49.843162 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 6 ==== pg_notify(24.30b(29) epoch 194471) v5 ==== 2391+0+0 (2687185915 0 0) 0x53bb640 con 0x49023c0 -3736> 2014-08-13 17:22:49.843176 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start -3735> 2014-08-13 17:22:49.843187 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.825510, event: header_read, request: pg_notify(24.30b(29) epoch 194471) v5 -3734> 2014-08-13 17:22:49.843183 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3733> 2014-08-13 17:22:49.843202 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray -3732> 2014-08-13 17:22:49.843209 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.825511, event: throttled, request: pg_notify(24.30b(29) epoch 194471) v5 -3731> 2014-08-13 17:22:49.843228 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.825612, event: all_read, request: pg_notify(24.30b(29) epoch 194471) v5 -3730> 2014-08-13 17:22:49.843222 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000046 0 0.000000 -3729> 2014-08-13 17:22:49.843238 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.843176, event: dispatched, request: pg_notify(24.30b(29) epoch 194471) v5 -3728> 2014-08-13 17:22:49.843249 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.843249, event: waiting_for_osdmap, request: pg_notify(24.30b(29) epoch 194471) v5 -3727> 2014-08-13 17:22:49.843243 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray -3726> 2014-08-13 17:22:49.843262 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.843262, event: started, request: pg_notify(24.30b(29) epoch 194471) v5 -3725> 2014-08-13 17:22:49.843286 7f74d8735700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:22:49.843286, event: done, request: pg_notify(24.30b(29) epoch 194471) v5 -3724> 2014-08-13 17:22:49.843322 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 9 ==== pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 ==== 1510+0+0 (4244965681 0 0) 0x53b8c40 con 0x4903180 -3723> 2014-08-13 17:22:49.843344 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.825649, event: header_read, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3722> 2014-08-13 17:22:49.843356 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.825651, event: throttled, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3721> 2014-08-13 17:22:49.843367 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.826542, event: all_read, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3720> 2014-08-13 17:22:49.843376 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.843341, event: dispatched, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3719> 2014-08-13 17:22:49.843386 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.843386, event: waiting_for_osdmap, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3718> 2014-08-13 17:22:49.843398 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.843398, event: started, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3717> 2014-08-13 17:22:49.843547 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.884205 2 0.000288 -3716> 2014-08-13 17:22:49.843585 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.884263 0 0.000000 -3715> 2014-08-13 17:22:49.843578 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.884659 5 0.000197 -3714> 2014-08-13 17:22:49.843609 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.900468 0 0.000000 -3713> 2014-08-13 17:22:49.843623 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.927222 0 0.000000 -3712> 2014-08-13 17:22:49.843633 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194470/194470) [4,330,28] r=0 lpr=194470 pi=193748-194469/24 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -3711> 2014-08-13 17:22:49.843653 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194449 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3710> 2014-08-13 17:22:49.843716 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.000083 1 0.000182 -3709> 2014-08-13 17:22:49.843746 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started -3708> 2014-08-13 17:22:49.843767 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Start -3707> 2014-08-13 17:22:49.843787 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary -3706> 2014-08-13 17:22:49.843810 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000042 0 0.000000 -3705> 2014-08-13 17:22:49.843833 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary -3704> 2014-08-13 17:22:49.843855 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering -3703> 2014-08-13 17:22:49.843876 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo -3702> 2014-08-13 17:22:49.844140 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(30) epoch 194471) v5 -- ?+0 0x4ac6c80 con 0x47a53e0 -3701> 2014-08-13 17:22:49.844211 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194471) v5 -- ?+0 0x4ac3b80 con 0x4758840 -3700> 2014-08-13 17:22:49.844238 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(37) epoch 194471) v5 -- ?+0 0x4ac0700 con 0x47a27e0 -3699> 2014-08-13 17:22:49.844258 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(19) epoch 194471) v5 -- ?+0 0x4ac3640 con 0x475d960 -3698> 2014-08-13 17:22:49.844283 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x2713a80 con 0x4758b00 -3697> 2014-08-13 17:22:49.844304 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- pg_notify(22.2b9(19) epoch 194471) v5 -- ?+0 0x57d1500 con 0x4758b00 -3696> 2014-08-13 17:22:49.844408 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(29) epoch 194471) v5 -- ?+0 0x57d6200 con 0x486e040 -3695> 2014-08-13 17:22:49.844455 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(19) epoch 194471) v5 -- ?+0 0x4f516c0 con 0x486aec0 -3694> 2014-08-13 17:22:49.844537 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(26) epoch 194471) v5 -- ?+0 0x53b9880 con 0x486b180 -3693> 2014-08-13 17:22:49.844562 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6824/52078 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x2713840 con 0x47f2680 -3692> 2014-08-13 17:22:49.844589 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(19) epoch 194471) v5 -- ?+0 0x53b9a40 con 0x47f2680 -3691> 2014-08-13 17:22:49.844678 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(19) epoch 194471) v5 -- ?+0 0x53b8fc0 con 0x486a940 -3690> 2014-08-13 17:22:49.844723 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(19) epoch 194471) v5 -- ?+0 0x53b9500 con 0x486a3c0 -3689> 2014-08-13 17:22:49.844745 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_query(24.b6 epoch 194471) v3 -- ?+0 0x4eb2940 con 0x47a53e0 -3688> 2014-08-13 17:22:49.844781 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6805/37391 -- pg_query(24.b6 epoch 194471) v3 -- ?+0 0x4eb4ec0 con 0x45f3b20 -3687> 2014-08-13 17:22:49.844804 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6800/14161 -- pg_query(24.b6 epoch 194471) v3 -- ?+0 0x4eb50a0 con 0x47f6880 -3686> 2014-08-13 17:22:49.844840 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6804/5112 -- pg_query(24.b6 epoch 194471) v3 -- ?+0 0x4eb5280 con 0x486b700 -3685> 2014-08-13 17:22:49.844874 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_query(24.b6 epoch 194471) v3 -- ?+0 0x4eb5460 con 0x486e040 -3684> 2014-08-13 17:22:49.844916 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6805/37391 -- pg_info(1 pgs e194471:22.84(3)) v4 -- ?+0 0x53ba140 con 0x45f3b20 -3683> 2014-08-13 17:22:49.845010 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_info(1 pgs e194471:22.84(3)) v4 -- ?+0 0x53bfc40 con 0x47f0c60 -3682> 2014-08-13 17:22:49.845056 7f74d0f29700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -3681> 2014-08-13 17:22:49.845067 7f74d0f29700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194471 {24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} v194471) v1 -- ?+0 0x4eb5640 con 0x45d48e0 -3680> 2014-08-13 17:22:49.845295 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started/Stray 0.900118 4 0.000271 -3679> 2014-08-13 17:22:49.845333 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started 0.900264 0 0.000000 -3678> 2014-08-13 17:22:49.845362 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/193151) [356,364,4]/[356,364] r=-1 lpr=194470 pi=193151-194469/23 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Reset -3677> 2014-08-13 17:22:49.845477 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.000115 1 0.000144 -3676> 2014-08-13 17:22:49.845508 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started -3675> 2014-08-13 17:22:49.845598 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start -3674> 2014-08-13 17:22:49.845618 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray -3673> 2014-08-13 17:22:49.845659 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000061 0 0.000000 -3672> 2014-08-13 17:22:49.845685 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray -3671> 2014-08-13 17:22:49.845862 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary/Peering/WaitActingChange 0.885794 2 0.000212 -3670> 2014-08-13 17:22:49.845930 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.885886 0 0.000000 -3669> 2014-08-13 17:22:49.845962 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started 0.900007 0 0.000000 -3668> 2014-08-13 17:22:49.845985 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194470/194470) [4,59,351] r=0 lpr=194470 pi=193678-194469/24 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset -3667> 2014-08-13 17:22:49.846069 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Reset 0.000084 1 0.000195 -3666> 2014-08-13 17:22:49.846098 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started -3665> 2014-08-13 17:22:49.846119 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Start -3664> 2014-08-13 17:22:49.846139 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: transitioning to Primary -3663> 2014-08-13 17:22:49.846176 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Start 0.000057 0 0.000000 -3662> 2014-08-13 17:22:49.846200 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary -3661> 2014-08-13 17:22:49.846222 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Peering -3660> 2014-08-13 17:22:49.846242 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetInfo -3659> 2014-08-13 17:22:49.846410 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary/Peering/WaitActingChange 0.885639 1 0.000221 -3658> 2014-08-13 17:22:49.846440 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.885689 0 0.000000 -3657> 2014-08-13 17:22:49.846484 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started 0.898913 0 0.000000 -3656> 2014-08-13 17:22:49.846507 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194470/194470) [4,55,61] r=0 lpr=194470 pi=193747-194469/22 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset -3655> 2014-08-13 17:22:49.846618 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Reset 0.000110 1 0.000190 -3654> 2014-08-13 17:22:49.846657 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started -3653> 2014-08-13 17:22:49.846679 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Start -3652> 2014-08-13 17:22:49.846699 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: transitioning to Primary -3651> 2014-08-13 17:22:49.846721 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Start 0.000042 0 0.000000 -3650> 2014-08-13 17:22:49.846745 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary -3649> 2014-08-13 17:22:49.846765 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Peering -3648> 2014-08-13 17:22:49.846786 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetInfo -3647> 2014-08-13 17:22:49.847098 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.915838 4 0.000310 -3646> 2014-08-13 17:22:49.847132 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started 0.915987 0 0.000000 -3645> 2014-08-13 17:22:49.847174 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194470/192969) [356,4,334] r=1 lpr=194470 pi=193150-194469/28 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset -3644> 2014-08-13 17:22:49.847307 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000132 1 0.000141 -3643> 2014-08-13 17:22:49.847338 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started -3642> 2014-08-13 17:22:49.847358 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Start -3641> 2014-08-13 17:22:49.847378 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -3640> 2014-08-13 17:22:49.847400 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000042 0 0.000000 -3639> 2014-08-13 17:22:49.847451 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -3638> 2014-08-13 17:22:49.847786 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.891715 6 0.000395 -3637> 2014-08-13 17:22:49.847800 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_log(1.2b epoch 194471 log log((0'0,194215'1672], crt=0'0) query_epoch 194471) v4 -- ?+0 0x5c71800 -3636> 2014-08-13 17:22:49.847821 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.913768 0 0.000000 -3635> 2014-08-13 17:22:49.847848 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3634> 2014-08-13 17:22:49.847844 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194465/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3633> 2014-08-13 17:22:49.847919 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194465/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3632> 2014-08-13 17:22:49.852955 7f74d8735700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:22:49.852954, event: done, request: pg_notify(2.2a(1),1.2b(1) epoch 194471) v5 -3631> 2014-08-13 17:22:49.853024 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 5 ==== pg_notify(22.1f1(1) epoch 194471) v5 ==== 777+0+0 (1920910726 0 0) 0x466f1c0 con 0x4906e00 -3630> 2014-08-13 17:22:49.853054 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.832608, event: header_read, request: pg_notify(22.1f1(1) epoch 194471) v5 -3629> 2014-08-13 17:22:49.853063 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.832609, event: throttled, request: pg_notify(22.1f1(1) epoch 194471) v5 -3628> 2014-08-13 17:22:49.853068 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.832673, event: all_read, request: pg_notify(22.1f1(1) epoch 194471) v5 -3627> 2014-08-13 17:22:49.853074 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.853050, event: dispatched, request: pg_notify(22.1f1(1) epoch 194471) v5 -3626> 2014-08-13 17:22:49.853080 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.853080, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194471) v5 -3625> 2014-08-13 17:22:49.853094 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.853094, event: started, request: pg_notify(22.1f1(1) epoch 194471) v5 -3624> 2014-08-13 17:22:49.853126 7f74d8735700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:22:49.853126, event: done, request: pg_notify(22.1f1(1) epoch 194471) v5 -3623> 2014-08-13 17:22:49.853156 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 10 ==== pg_notify(22.775(1) epoch 194471) v5 ==== 777+0+0 (1819448737 0 0) 0x53bde80 con 0x4d58840 -3622> 2014-08-13 17:22:49.853178 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.827530, event: header_read, request: pg_notify(22.775(1) epoch 194471) v5 -3621> 2014-08-13 17:22:49.853185 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.827532, event: throttled, request: pg_notify(22.775(1) epoch 194471) v5 -3620> 2014-08-13 17:22:49.853191 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.827618, event: all_read, request: pg_notify(22.775(1) epoch 194471) v5 -3619> 2014-08-13 17:22:49.853131 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started/Stray 0.918085 4 0.000260 -3618> 2014-08-13 17:22:49.853198 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.853175, event: dispatched, request: pg_notify(22.775(1) epoch 194471) v5 -3617> 2014-08-13 17:22:49.853241 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.853241, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194471) v5 -3616> 2014-08-13 17:22:49.853256 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.853256, event: started, request: pg_notify(22.775(1) epoch 194471) v5 -3615> 2014-08-13 17:22:49.853229 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started 0.918281 0 0.000000 -3614> 2014-08-13 17:22:49.853271 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194470/194465) [4,344,296]/[320,307,296] r=-1 lpr=194470 pi=193679-194469/27 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Reset -3613> 2014-08-13 17:22:49.853282 7f74d8735700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:22:49.853282, event: done, request: pg_notify(22.775(1) epoch 194471) v5 -3612> 2014-08-13 17:22:49.853277 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.935756 3 0.000223 -3611> 2014-08-13 17:22:49.853322 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started 0.935885 0 0.000000 -3610> 2014-08-13 17:22:49.853326 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 8 ==== pg_notify(22.39d(20) epoch 194471) v5 ==== 1834+0+0 (646771759 0 0) 0x466b640 con 0x4d582c0 -3609> 2014-08-13 17:22:49.853354 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.825577, event: header_read, request: pg_notify(22.39d(20) epoch 194471) v5 -3608> 2014-08-13 17:22:49.853361 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.825579, event: throttled, request: pg_notify(22.39d(20) epoch 194471) v5 -3607> 2014-08-13 17:22:49.853368 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.825664, event: all_read, request: pg_notify(22.39d(20) epoch 194471) v5 -3606> 2014-08-13 17:22:49.853373 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.853351, event: dispatched, request: pg_notify(22.39d(20) epoch 194471) v5 -3605> 2014-08-13 17:22:49.853350 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194462/194462) [26,30,340] r=-1 lpr=194462 pi=193125-194461/47 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Reset -3604> 2014-08-13 17:22:49.853358 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Reset 0.000087 1 0.000225 -3603> 2014-08-13 17:22:49.853386 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.853386, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194471) v5 -3602> 2014-08-13 17:22:49.853395 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.853394, event: started, request: pg_notify(22.39d(20) epoch 194471) v5 -3601> 2014-08-13 17:22:49.853389 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started -3600> 2014-08-13 17:22:49.853408 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Start -3599> 2014-08-13 17:22:49.853421 7f74d8735700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:22:49.853421, event: done, request: pg_notify(22.39d(20) epoch 194471) v5 -3598> 2014-08-13 17:22:49.853425 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: transitioning to Primary -3597> 2014-08-13 17:22:49.853445 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Start 0.000037 0 0.000000 -3596> 2014-08-13 17:22:49.853451 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 7 ==== pg_notify(22.481(32) epoch 194471) v5 ==== 2518+0+0 (3083618055 0 0) 0x53bd780 con 0x4d51ce0 -3595> 2014-08-13 17:22:49.853467 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.829408, event: header_read, request: pg_notify(22.481(32) epoch 194471) v5 -3594> 2014-08-13 17:22:49.853474 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.829411, event: throttled, request: pg_notify(22.481(32) epoch 194471) v5 -3593> 2014-08-13 17:22:49.853465 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary -3592> 2014-08-13 17:22:49.853480 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.829553, event: all_read, request: pg_notify(22.481(32) epoch 194471) v5 -3591> 2014-08-13 17:22:49.853485 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.853465, event: dispatched, request: pg_notify(22.481(32) epoch 194471) v5 -3590> 2014-08-13 17:22:49.853471 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Reset 0.000120 1 0.000193 -3589> 2014-08-13 17:22:49.853492 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.853492, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194471) v5 -3588> 2014-08-13 17:22:49.853483 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Peering -3587> 2014-08-13 17:22:49.853499 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.853499, event: started, request: pg_notify(22.481(32) epoch 194471) v5 -3586> 2014-08-13 17:22:49.853499 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started -3585> 2014-08-13 17:22:49.853503 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetInfo -3584> 2014-08-13 17:22:49.853520 7f74d8735700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:22:49.853519, event: done, request: pg_notify(22.481(32) epoch 194471) v5 -3583> 2014-08-13 17:22:49.853518 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Start -3582> 2014-08-13 17:22:49.853535 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: transitioning to Primary -3581> 2014-08-13 17:22:49.853554 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Start 0.000036 0 0.000000 -3580> 2014-08-13 17:22:49.853570 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 7 ==== pg_info(1 pgs e194471:22.1d2) v4 ==== 714+0+0 (2506920389 0 0) 0x53be3c0 con 0x4d55d80 -3579> 2014-08-13 17:22:49.853574 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary -3578> 2014-08-13 17:22:49.853586 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.844756, event: header_read, request: pg_info(1 pgs e194471:22.1d2) v4 -3577> 2014-08-13 17:22:49.853593 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.844758, event: throttled, request: pg_info(1 pgs e194471:22.1d2) v4 -3576> 2014-08-13 17:22:49.853598 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.844836, event: all_read, request: pg_info(1 pgs e194471:22.1d2) v4 -3575> 2014-08-13 17:22:49.853591 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Peering -3574> 2014-08-13 17:22:49.853603 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.853584, event: dispatched, request: pg_info(1 pgs e194471:22.1d2) v4 -3573> 2014-08-13 17:22:49.853609 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.853609, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1d2) v4 -3572> 2014-08-13 17:22:49.853617 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.853617, event: started, request: pg_info(1 pgs e194471:22.1d2) v4 -3571> 2014-08-13 17:22:49.853608 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194448/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetInfo -3570> 2014-08-13 17:22:49.853639 7f74d8735700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:22:49.853638, event: done, request: pg_info(1 pgs e194471:22.1d2) v4 -3569> 2014-08-13 17:22:49.853659 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4800480 con 0x45f1760 -3568> 2014-08-13 17:22:49.853678 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3567> 2014-08-13 17:22:49.853696 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 8 ==== pg_info(1 pgs e194471:22.14a(2)) v4 ==== 824+0+0 (1845381796 0 0) 0x57d2a00 con 0x45f1e40 -3566> 2014-08-13 17:22:49.853685 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.917217 4 0.000240 -3565> 2014-08-13 17:22:49.853711 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.803470, event: header_read, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3564> 2014-08-13 17:22:49.853713 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started 0.917337 0 0.000000 -3563> 2014-08-13 17:22:49.853726 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.803471, event: throttled, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3562> 2014-08-13 17:22:49.853732 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.803527, event: all_read, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3561> 2014-08-13 17:22:49.853738 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.853709, event: dispatched, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3560> 2014-08-13 17:22:49.853732 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194470/193264) [15,4,52] r=1 lpr=194470 pi=193264-194469/25 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset -3559> 2014-08-13 17:22:49.853744 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.853744, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3558> 2014-08-13 17:22:49.853752 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.853752, event: started, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3557> 2014-08-13 17:22:49.853772 7f74d8735700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:22:49.853771, event: done, request: pg_info(1 pgs e194471:22.14a(2)) v4 -3556> 2014-08-13 17:22:49.853764 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(19) epoch 194471) v5 -- ?+0 0x53bf540 con 0x45f2940 -3555> 2014-08-13 17:22:49.853788 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 7 ==== pg_notify(24.b6 epoch 194471) v5 ==== 694+0+0 (2092223685 0 0) 0x49e7a80 con 0x45f3b20 -3554> 2014-08-13 17:22:49.853803 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.845284, event: header_read, request: pg_notify(24.b6 epoch 194471) v5 -3553> 2014-08-13 17:22:49.853809 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.845286, event: throttled, request: pg_notify(24.b6 epoch 194471) v5 -3552> 2014-08-13 17:22:49.853820 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.845359, event: all_read, request: pg_notify(24.b6 epoch 194471) v5 -3551> 2014-08-13 17:22:49.853828 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.853801, event: dispatched, request: pg_notify(24.b6 epoch 194471) v5 -3550> 2014-08-13 17:22:49.853838 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.853837, event: waiting_for_osdmap, request: pg_notify(24.b6 epoch 194471) v5 -3549> 2014-08-13 17:22:49.853849 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.853848, event: started, request: pg_notify(24.b6 epoch 194471) v5 -3548> 2014-08-13 17:22:49.853866 7f74d8735700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:22:49.853866, event: done, request: pg_notify(24.b6 epoch 194471) v5 -3547> 2014-08-13 17:22:49.853836 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000104 1 0.000102 -3546> 2014-08-13 17:22:49.853878 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started -3545> 2014-08-13 17:22:49.853883 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 7 ==== pg_notify(24.30b(2) epoch 194471) v5 ==== 870+0+0 (552981272 0 0) 0x49e0a80 con 0x45f5540 -3544> 2014-08-13 17:22:49.853902 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Start -3543> 2014-08-13 17:22:49.853917 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.813862, event: header_read, request: pg_notify(24.30b(2) epoch 194471) v5 -3542> 2014-08-13 17:22:49.853928 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.813864, event: throttled, request: pg_notify(24.30b(2) epoch 194471) v5 -3541> 2014-08-13 17:22:49.853942 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.813950, event: all_read, request: pg_notify(24.30b(2) epoch 194471) v5 -3540> 2014-08-13 17:22:49.853952 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.853915, event: dispatched, request: pg_notify(24.30b(2) epoch 194471) v5 -3539> 2014-08-13 17:22:49.853944 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6836/14923 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x4de7740 con 0x45f6f60 -3538> 2014-08-13 17:22:49.853958 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.853958, event: waiting_for_osdmap, request: pg_notify(24.30b(2) epoch 194471) v5 -3537> 2014-08-13 17:22:49.853966 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.853966, event: started, request: pg_notify(24.30b(2) epoch 194471) v5 -3536> 2014-08-13 17:22:49.854002 7f74d8735700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:22:49.854002, event: done, request: pg_notify(24.30b(2) epoch 194471) v5 -3535> 2014-08-13 17:22:49.853920 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -3534> 2014-08-13 17:22:49.854025 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 9 ==== pg_query(24.350,24.3b7 epoch 194471) v3 ==== 272+0+0 (1273921699 0 0) 0x4eb3660 con 0x45f5960 -3533> 2014-08-13 17:22:49.854045 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.803305, event: header_read, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3532> 2014-08-13 17:22:49.854056 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.803306, event: throttled, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3531> 2014-08-13 17:22:49.854039 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000136 0 0.000000 -3530> 2014-08-13 17:22:49.854066 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.803349, event: all_read, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3529> 2014-08-13 17:22:49.854075 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.854043, event: dispatched, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3528> 2014-08-13 17:22:49.854069 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -3527> 2014-08-13 17:22:49.854085 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.854085, event: waiting_for_osdmap, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3526> 2014-08-13 17:22:49.854096 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.854096, event: started, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3525> 2014-08-13 17:22:49.854123 7f74d8735700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:22:49.854122, event: done, request: pg_query(24.350,24.3b7 epoch 194471) v3 -3524> 2014-08-13 17:22:49.853972 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(19) epoch 194471) v5 -- ?+0 0x53bf380 con 0x45f6f60 -3523> 2014-08-13 17:22:49.854142 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 7 ==== pg_notify(22.1f1(22) epoch 194471) v5 ==== 1948+0+0 (1785537184 0 0) 0x49e78c0 con 0x45f6f60 -3522> 2014-08-13 17:22:49.854170 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.806792, event: header_read, request: pg_notify(22.1f1(22) epoch 194471) v5 -3521> 2014-08-13 17:22:49.854181 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.806794, event: throttled, request: pg_notify(22.1f1(22) epoch 194471) v5 -3520> 2014-08-13 17:22:49.854190 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.806887, event: all_read, request: pg_notify(22.1f1(22) epoch 194471) v5 -3519> 2014-08-13 17:22:49.854208 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.854167, event: dispatched, request: pg_notify(22.1f1(22) epoch 194471) v5 -3518> 2014-08-13 17:22:49.854219 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.854219, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194471) v5 -3517> 2014-08-13 17:22:49.854227 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.854227, event: started, request: pg_notify(22.1f1(22) epoch 194471) v5 -3516> 2014-08-13 17:22:49.854212 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6834/14316 -- pg_notify(22.14a(19) epoch 194471) v5 -- ?+0 0x53bf1c0 con 0x45f1e40 -3515> 2014-08-13 17:22:49.854246 7f74d8735700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:22:49.854246, event: done, request: pg_notify(22.1f1(22) epoch 194471) v5 -3514> 2014-08-13 17:22:49.854248 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(24) epoch 194471) v5 -- ?+0 0x53bf000 con 0x47f0c60 -3513> 2014-08-13 17:22:49.854249 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started/Stray 0.916910 4 0.000274 -3512> 2014-08-13 17:22:49.854271 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 6 ==== pg_info(1 pgs e194471:22.1a0(2)) v4 ==== 824+0+0 (2402927798 0 0) 0x49e1dc0 con 0x4758b00 -3511> 2014-08-13 17:22:49.854288 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.811184, event: header_read, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3510> 2014-08-13 17:22:49.854278 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started 0.917028 0 0.000000 -3509> 2014-08-13 17:22:49.854294 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.811186, event: throttled, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3508> 2014-08-13 17:22:49.854300 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.811268, event: all_read, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3507> 2014-08-13 17:22:49.854306 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.854285, event: dispatched, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3506> 2014-08-13 17:22:49.854298 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194470/192978) [31,338,4]/[31,338,16] r=-1 lpr=194470 pi=193125-194469/33 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Reset -3505> 2014-08-13 17:22:49.854315 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.854315, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3504> 2014-08-13 17:22:49.854323 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.854323, event: started, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3503> 2014-08-13 17:22:49.854348 7f74d8735700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:22:49.854348, event: done, request: pg_info(1 pgs e194471:22.1a0(2)) v4 -3502> 2014-08-13 17:22:49.854365 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 5 ==== pg_info(1 pgs e194471:22.791(2)) v4 ==== 824+0+0 (1689100820 0 0) 0x49e0000 con 0x475a940 -3501> 2014-08-13 17:22:49.854380 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.806731, event: header_read, request: pg_info(1 pgs e194471:22.791(2)) v4 -3500> 2014-08-13 17:22:49.854386 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.806732, event: throttled, request: pg_info(1 pgs e194471:22.791(2)) v4 -3499> 2014-08-13 17:22:49.854396 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.806798, event: all_read, request: pg_info(1 pgs e194471:22.791(2)) v4 -3498> 2014-08-13 17:22:49.854402 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.854378, event: dispatched, request: pg_info(1 pgs e194471:22.791(2)) v4 -3497> 2014-08-13 17:22:49.854408 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.854408, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.791(2)) v4 -3496> 2014-08-13 17:22:49.854397 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.000099 1 0.000106 -3495> 2014-08-13 17:22:49.854416 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.854415, event: started, request: pg_info(1 pgs e194471:22.791(2)) v4 -3494> 2014-08-13 17:22:49.854407 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(22) epoch 194471) v5 -- ?+0 0x53bee40 con 0x486bc80 -3493> 2014-08-13 17:22:49.854434 7f74d8735700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:22:49.854434, event: done, request: pg_info(1 pgs e194471:22.791(2)) v4 -3492> 2014-08-13 17:22:49.854437 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(19) epoch 194471) v5 -- ?+0 0x49e01c0 con 0x475e5c0 -3491> 2014-08-13 17:22:49.854448 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 6 ==== pg_info(1 pgs e194471:22.116(1)) v4 ==== 773+0+0 (1843894647 0 0) 0x466ec80 con 0x475c4c0 -3490> 2014-08-13 17:22:49.854462 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.800062, event: header_read, request: pg_info(1 pgs e194471:22.116(1)) v4 -3489> 2014-08-13 17:22:49.854462 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- osd_map(194471..194471 src has 185139..194471) v3 -- ?+0 0x4de7500 con 0x4903180 -3488> 2014-08-13 17:22:49.854469 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.800066, event: throttled, request: pg_info(1 pgs e194471:22.116(1)) v4 -3487> 2014-08-13 17:22:49.854474 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.800161, event: all_read, request: pg_info(1 pgs e194471:22.116(1)) v4 -3486> 2014-08-13 17:22:49.854480 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.854461, event: dispatched, request: pg_info(1 pgs e194471:22.116(1)) v4 -3485> 2014-08-13 17:22:49.854486 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.854485, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.116(1)) v4 -3484> 2014-08-13 17:22:49.854493 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.854492, event: started, request: pg_info(1 pgs e194471:22.116(1)) v4 -3483> 2014-08-13 17:22:49.854515 7f74d8735700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:22:49.854515, event: done, request: pg_info(1 pgs e194471:22.116(1)) v4 -3482> 2014-08-13 17:22:49.854529 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 12 ==== pg_notify(22.3e4(1) epoch 194471) v5 ==== 777+0+0 (4231295865 0 0) 0x5263640 con 0x475ee00 -3481> 2014-08-13 17:22:49.854544 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.810963, event: header_read, request: pg_notify(22.3e4(1) epoch 194471) v5 -3480> 2014-08-13 17:22:49.854550 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.810965, event: throttled, request: pg_notify(22.3e4(1) epoch 194471) v5 -3479> 2014-08-13 17:22:49.854555 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.811024, event: all_read, request: pg_notify(22.3e4(1) epoch 194471) v5 -3478> 2014-08-13 17:22:49.854560 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.854542, event: dispatched, request: pg_notify(22.3e4(1) epoch 194471) v5 -3477> 2014-08-13 17:22:49.854566 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.854566, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194471) v5 -3476> 2014-08-13 17:22:49.854573 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.854572, event: started, request: pg_notify(22.3e4(1) epoch 194471) v5 -3475> 2014-08-13 17:22:49.854591 7f74d8735700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:22:49.854591, event: done, request: pg_notify(22.3e4(1) epoch 194471) v5 -3474> 2014-08-13 17:22:49.854589 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194471) v5 -- ?+0 0x49e2bc0 con 0x4903180 -3473> 2014-08-13 17:22:49.854617 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 9 ==== pg_notify(22.48e(1) epoch 194471) v5 ==== 777+0+0 (4008595643 0 0) 0x57d1340 con 0x47a27e0 -3472> 2014-08-13 17:22:49.854633 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.799666, event: header_read, request: pg_notify(22.48e(1) epoch 194471) v5 -3471> 2014-08-13 17:22:49.854625 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(20) epoch 194471) v5 -- ?+0 0x53be3c0 con 0x475c4c0 -3470> 2014-08-13 17:22:49.854640 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.799668, event: throttled, request: pg_notify(22.48e(1) epoch 194471) v5 -3469> 2014-08-13 17:22:49.854645 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.799731, event: all_read, request: pg_notify(22.48e(1) epoch 194471) v5 -3468> 2014-08-13 17:22:49.854650 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.854631, event: dispatched, request: pg_notify(22.48e(1) epoch 194471) v5 -3467> 2014-08-13 17:22:49.854656 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.854656, event: waiting_for_osdmap, request: pg_notify(22.48e(1) epoch 194471) v5 -3466> 2014-08-13 17:22:49.854663 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.854663, event: started, request: pg_notify(22.48e(1) epoch 194471) v5 -3465> 2014-08-13 17:22:49.854681 7f74d8735700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:22:49.854681, event: done, request: pg_notify(22.48e(1) epoch 194471) v5 -3464> 2014-08-13 17:22:49.854694 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 10 ==== pg_notify(22.362(1) epoch 194471) v5 ==== 777+0+0 (446761135 0 0) 0x53bc600 con 0x47a0b00 -3463> 2014-08-13 17:22:49.854708 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.826973, event: header_read, request: pg_notify(22.362(1) epoch 194471) v5 -3462> 2014-08-13 17:22:49.854714 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.826975, event: throttled, request: pg_notify(22.362(1) epoch 194471) v5 -3461> 2014-08-13 17:22:49.854719 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.827027, event: all_read, request: pg_notify(22.362(1) epoch 194471) v5 -3460> 2014-08-13 17:22:49.854724 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.854706, event: dispatched, request: pg_notify(22.362(1) epoch 194471) v5 -3459> 2014-08-13 17:22:49.854730 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.854730, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194471) v5 -3458> 2014-08-13 17:22:49.854737 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.854737, event: started, request: pg_notify(22.362(1) epoch 194471) v5 -3457> 2014-08-13 17:22:49.854755 7f74d8735700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:22:49.854755, event: done, request: pg_notify(22.362(1) epoch 194471) v5 -3456> 2014-08-13 17:22:49.854767 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 7 ==== pg_notify(24.b6(28) epoch 194471) v5 ==== 2332+0+0 (380497260 0 0) 0x4ac7700 con 0x47a53e0 -3455> 2014-08-13 17:22:49.854771 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(27) epoch 194471) v5 -- ?+0 0x53bd780 con 0x47a4200 -3454> 2014-08-13 17:22:49.854787 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.798117, event: header_read, request: pg_notify(24.b6(28) epoch 194471) v5 -3453> 2014-08-13 17:22:49.854421 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started -3452> 2014-08-13 17:22:49.854800 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_query(24.3d epoch 194471) v3 -- ?+0 0x29c5460 con 0x4758840 -3451> 2014-08-13 17:22:49.854800 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.798119, event: throttled, request: pg_notify(24.b6(28) epoch 194471) v5 -3450> 2014-08-13 17:22:49.854825 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.798235, event: all_read, request: pg_notify(24.b6(28) epoch 194471) v5 -3449> 2014-08-13 17:22:49.854819 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start -3448> 2014-08-13 17:22:49.854824 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6800/36920 -- pg_query(24.3d epoch 194471) v3 -- ?+0 0x29c6180 con 0x4904a40 -3447> 2014-08-13 17:22:49.854838 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.854785, event: dispatched, request: pg_notify(24.b6(28) epoch 194471) v5 -3446> 2014-08-13 17:22:49.854840 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray -3445> 2014-08-13 17:22:49.854970 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.854969, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194471) v5 -3444> 2014-08-13 17:22:49.854986 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.854986, event: started, request: pg_notify(24.b6(28) epoch 194471) v5 -3443> 2014-08-13 17:22:49.854861 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6802/58522 -- pg_query(24.34 epoch 194471) v3 -- ?+0 0x29c50a0 con 0x4d2aaa0 -3442> 2014-08-13 17:22:49.854994 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000175 0 0.000000 -3441> 2014-08-13 17:22:49.855023 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray -3440> 2014-08-13 17:22:49.855033 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- pg_query(24.34 epoch 194471) v3 -- ?+0 0x29c1680 con 0x475a940 -3439> 2014-08-13 17:22:49.855059 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6820/25633 -- pg_query(24.9b epoch 194471) v3 -- ?+0 0x29c0000 con 0x47591e0 -3438> 2014-08-13 17:22:49.855076 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_query(24.3d epoch 194471) v3 -- ?+0 0x29c1fe0 con 0x475ee00 -3437> 2014-08-13 17:22:49.855119 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6806/62463 -- pg_query(24.34 epoch 194471) v3 -- ?+0 0x29c6540 con 0x47a3f40 -3436> 2014-08-13 17:22:49.855164 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6832/53185 -- pg_query(24.9b epoch 194471) v3 -- ?+0 0x29c7260 con 0x4d55ac0 -3435> 2014-08-13 17:22:49.855183 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6842/8170 -- pg_query(24.3d epoch 194471) v3 -- ?+0 0x29c2580 con 0x48db9c0 -3434> 2014-08-13 17:22:49.855203 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.916595 4 0.000254 -3433> 2014-08-13 17:22:49.855230 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started 0.916715 0 0.000000 -3432> 2014-08-13 17:22:49.855250 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194470/192975) [285,336,4]/[285,336] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Reset -3431> 2014-08-13 17:22:49.855009 7f74d8735700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:22:49.855009, event: done, request: pg_notify(24.b6(28) epoch 194471) v5 -3430> 2014-08-13 17:22:49.855298 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6824/15623 -- pg_query(24.9b epoch 194471) v3 -- ?+0 0x29c2b20 con 0x4d51ce0 -3429> 2014-08-13 17:22:49.855335 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000085 1 0.000102 -3428> 2014-08-13 17:22:49.855379 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started -3427> 2014-08-13 17:22:49.855374 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4807740 con 0x47a3f40 -3426> 2014-08-13 17:22:49.855413 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3425> 2014-08-13 17:22:49.855410 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start -3424> 2014-08-13 17:22:49.855431 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 8 ==== pg_query(24.369 epoch 194471) v3 ==== 144+0+0 (2281536516 0 0) 0x29c6ae0 con 0x47a32e0 -3423> 2014-08-13 17:22:49.855444 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -3422> 2014-08-13 17:22:49.855452 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.799802, event: header_read, request: pg_query(24.369 epoch 194471) v3 -3421> 2014-08-13 17:22:49.855463 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.799805, event: throttled, request: pg_query(24.369 epoch 194471) v3 -3420> 2014-08-13 17:22:49.855455 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6840/55513 -- pg_query(24.9b epoch 194471) v3 -- ?+0 0x29c5280 con 0x486a940 -3419> 2014-08-13 17:22:49.855469 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.799889, event: all_read, request: pg_query(24.369 epoch 194471) v3 -3418> 2014-08-13 17:22:49.855475 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.855449, event: dispatched, request: pg_query(24.369 epoch 194471) v3 -3417> 2014-08-13 17:22:49.855463 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000054 0 0.000000 -3416> 2014-08-13 17:22:49.855482 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.855482, event: waiting_for_osdmap, request: pg_query(24.369 epoch 194471) v3 -3415> 2014-08-13 17:22:49.855487 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6840/7435 -- pg_query(24.3d epoch 194471) v3 -- ?+0 0x4eb5a00 con 0x486a3c0 -3414> 2014-08-13 17:22:49.855491 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.855491, event: started, request: pg_query(24.369 epoch 194471) v3 -3413> 2014-08-13 17:22:49.855506 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6840/19195 -- pg_query(24.9b epoch 194471) v3 -- ?+0 0x4eb1c20 con 0x4d511e0 -3412> 2014-08-13 17:22:49.855549 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6816/17521 -- pg_info(1 pgs e194471:22.f3(19)) v4 -- ?+0 0x53bde80 con 0x4906880 -3411> 2014-08-13 17:22:49.855580 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -3410> 2014-08-13 17:22:49.855605 7f74d8735700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:22:49.855604, event: done, request: pg_query(24.369 epoch 194471) v3 -3409> 2014-08-13 17:22:49.855627 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 7 ==== pg_query(24.33a epoch 194471) v3 ==== 144+0+0 (3524591482 0 0) 0x29c7620 con 0x47f0c60 -3408> 2014-08-13 17:22:49.855645 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.799328, event: header_read, request: pg_query(24.33a epoch 194471) v3 -3407> 2014-08-13 17:22:49.855652 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.799332, event: throttled, request: pg_query(24.33a epoch 194471) v3 -3406> 2014-08-13 17:22:49.855657 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.799413, event: all_read, request: pg_query(24.33a epoch 194471) v3 -3405> 2014-08-13 17:22:49.855662 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.855643, event: dispatched, request: pg_query(24.33a epoch 194471) v3 -3404> 2014-08-13 17:22:49.855668 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.855668, event: waiting_for_osdmap, request: pg_query(24.33a epoch 194471) v3 -3403> 2014-08-13 17:22:49.855675 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.855675, event: started, request: pg_query(24.33a epoch 194471) v3 -3402> 2014-08-13 17:22:49.855696 7f74d8735700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:22:49.855696, event: done, request: pg_query(24.33a epoch 194471) v3 -3401> 2014-08-13 17:22:49.855715 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 6 ==== pg_info(1 pgs e194471:22.99) v4 ==== 714+0+0 (1465675444 0 0) 0x52639c0 con 0x47f4620 -3400> 2014-08-13 17:22:49.855735 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.854453, event: header_read, request: pg_info(1 pgs e194471:22.99) v4 -3399> 2014-08-13 17:22:49.855745 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.854455, event: throttled, request: pg_info(1 pgs e194471:22.99) v4 -3398> 2014-08-13 17:22:49.855755 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.854550, event: all_read, request: pg_info(1 pgs e194471:22.99) v4 -3397> 2014-08-13 17:22:49.855761 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.855733, event: dispatched, request: pg_info(1 pgs e194471:22.99) v4 -3396> 2014-08-13 17:22:49.855770 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.855770, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.99) v4 -3395> 2014-08-13 17:22:49.855781 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.855781, event: started, request: pg_info(1 pgs e194471:22.99) v4 -3394> 2014-08-13 17:22:49.855783 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_info(1 pgs e194471:22.22c(21)) v4 -- ?+0 0x53b8c40 con 0x4903180 -3393> 2014-08-13 17:22:49.855839 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -3392> 2014-08-13 17:22:49.855846 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194471 {24.1bb=[4,332,337]} v194471) v1 -- ?+0 0x4eb3660 con 0x45d48e0 -3391> 2014-08-13 17:22:49.855808 7f74d8735700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:22:49.855808, event: done, request: pg_info(1 pgs e194471:22.99) v4 -3390> 2014-08-13 17:22:49.855990 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 10 ==== pg_notify(24.1f7(2) epoch 194471) v5 ==== 870+0+0 (4122280054 0 0) 0x53b96c0 con 0x47f7e80 -3389> 2014-08-13 17:22:49.856014 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.813062, event: header_read, request: pg_notify(24.1f7(2) epoch 194471) v5 -3388> 2014-08-13 17:22:49.856022 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.813064, event: throttled, request: pg_notify(24.1f7(2) epoch 194471) v5 -3387> 2014-08-13 17:22:49.856028 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.813137, event: all_read, request: pg_notify(24.1f7(2) epoch 194471) v5 -3386> 2014-08-13 17:22:49.856034 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.856011, event: dispatched, request: pg_notify(24.1f7(2) epoch 194471) v5 -3385> 2014-08-13 17:22:49.856012 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.902143 6 0.000285 -3384> 2014-08-13 17:22:49.856040 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.856040, event: waiting_for_osdmap, request: pg_notify(24.1f7(2) epoch 194471) v5 -3383> 2014-08-13 17:22:49.856049 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.856049, event: started, request: pg_notify(24.1f7(2) epoch 194471) v5 -3382> 2014-08-13 17:22:49.856048 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.907922 0 0.000000 -3381> 2014-08-13 17:22:49.856074 7f74d8735700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:22:49.856074, event: done, request: pg_notify(24.1f7(2) epoch 194471) v5 -3380> 2014-08-13 17:22:49.856073 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194449 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3379> 2014-08-13 17:22:49.856090 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 10 ==== pg_notify(24.1bb(26) epoch 194471) v5 ==== 2214+0+0 (1123115219 0 0) 0x57d2f40 con 0x4833440 -3378> 2014-08-13 17:22:49.856105 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.805511, event: header_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -3377> 2014-08-13 17:22:49.856111 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.805514, event: throttled, request: pg_notify(24.1bb(26) epoch 194471) v5 -3376> 2014-08-13 17:22:49.856117 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.805630, event: all_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -3375> 2014-08-13 17:22:49.856122 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.856103, event: dispatched, request: pg_notify(24.1bb(26) epoch 194471) v5 -3374> 2014-08-13 17:22:49.856128 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.856128, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194471) v5 -3373> 2014-08-13 17:22:49.856135 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.856135, event: started, request: pg_notify(24.1bb(26) epoch 194471) v5 -3372> 2014-08-13 17:22:49.856140 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194465/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3371> 2014-08-13 17:22:49.856176 7f74d8735700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:22:49.856175, event: done, request: pg_notify(24.1bb(26) epoch 194471) v5 -3370> 2014-08-13 17:22:49.856227 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 13 ==== pg_notify(24.1f7(24) epoch 194471) v5 ==== 2096+0+0 (3771264069 0 0) 0x49e0e00 con 0x4835280 -3369> 2014-08-13 17:22:49.856245 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.804609, event: header_read, request: pg_notify(24.1f7(24) epoch 194471) v5 -3368> 2014-08-13 17:22:49.856252 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.804612, event: throttled, request: pg_notify(24.1f7(24) epoch 194471) v5 -3367> 2014-08-13 17:22:49.856258 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.804730, event: all_read, request: pg_notify(24.1f7(24) epoch 194471) v5 -3366> 2014-08-13 17:22:49.856263 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.856243, event: dispatched, request: pg_notify(24.1f7(24) epoch 194471) v5 -3365> 2014-08-13 17:22:49.856268 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.856268, event: waiting_for_osdmap, request: pg_notify(24.1f7(24) epoch 194471) v5 -3364> 2014-08-13 17:22:49.856276 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.856276, event: started, request: pg_notify(24.1f7(24) epoch 194471) v5 -3363> 2014-08-13 17:22:49.856295 7f74d8735700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:22:49.856295, event: done, request: pg_notify(24.1f7(24) epoch 194471) v5 -3362> 2014-08-13 17:22:49.856314 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 9 ==== pg_info(1 pgs e194471:22.362) v4 ==== 714+0+0 (2151035818 0 0) 0x57d71c0 con 0x48377a0 -3361> 2014-08-13 17:22:49.856340 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.838932, event: header_read, request: pg_info(1 pgs e194471:22.362) v4 -3360> 2014-08-13 17:22:49.856347 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.838934, event: throttled, request: pg_info(1 pgs e194471:22.362) v4 -3359> 2014-08-13 17:22:49.856456 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.898795 6 0.000235 -3358> 2014-08-13 17:22:49.856493 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.906446 0 0.000000 -3357> 2014-08-13 17:22:49.856521 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3356> 2014-08-13 17:22:49.856371 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.839016, event: all_read, request: pg_info(1 pgs e194471:22.362) v4 -3355> 2014-08-13 17:22:49.856568 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.856338, event: dispatched, request: pg_info(1 pgs e194471:22.362) v4 -3354> 2014-08-13 17:22:49.856572 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194465/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3353> 2014-08-13 17:22:49.856588 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.856588, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.362) v4 -3352> 2014-08-13 17:22:49.856601 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.856601, event: started, request: pg_info(1 pgs e194471:22.362) v4 -3351> 2014-08-13 17:22:49.856629 7f74d8735700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:22:49.856629, event: done, request: pg_info(1 pgs e194471:22.362) v4 -3350> 2014-08-13 17:22:49.856650 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 10 ==== pg_notify(24.181(3) epoch 194471) v5 ==== 925+0+0 (2958737920 0 0) 0x49e16c0 con 0x486bc80 -3349> 2014-08-13 17:22:49.856674 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.801488, event: header_read, request: pg_notify(24.181(3) epoch 194471) v5 -3348> 2014-08-13 17:22:49.856681 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.801492, event: throttled, request: pg_notify(24.181(3) epoch 194471) v5 -3347> 2014-08-13 17:22:49.856686 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.801593, event: all_read, request: pg_notify(24.181(3) epoch 194471) v5 -3346> 2014-08-13 17:22:49.856691 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.856671, event: dispatched, request: pg_notify(24.181(3) epoch 194471) v5 -3345> 2014-08-13 17:22:49.856697 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.856697, event: waiting_for_osdmap, request: pg_notify(24.181(3) epoch 194471) v5 -3344> 2014-08-13 17:22:49.856704 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.856704, event: started, request: pg_notify(24.181(3) epoch 194471) v5 -3343> 2014-08-13 17:22:49.856724 7f74d8735700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:22:49.856723, event: done, request: pg_notify(24.181(3) epoch 194471) v5 -3342> 2014-08-13 17:22:49.856739 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 9 ==== pg_notify(24.b6(2) epoch 194471) v5 ==== 870+0+0 (926475702 0 0) 0x49e0540 con 0x486b700 -3341> 2014-08-13 17:22:49.856765 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.845590, event: header_read, request: pg_notify(24.b6(2) epoch 194471) v5 -3340> 2014-08-13 17:22:49.856778 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.845592, event: throttled, request: pg_notify(24.b6(2) epoch 194471) v5 -3339> 2014-08-13 17:22:49.856757 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started/Stray 0.907173 4 0.000219 -3338> 2014-08-13 17:22:49.856784 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.845677, event: all_read, request: pg_notify(24.b6(2) epoch 194471) v5 -3337> 2014-08-13 17:22:49.856593 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.897047 6 0.000221 -3336> 2014-08-13 17:22:49.856888 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.916602 0 0.000000 -3335> 2014-08-13 17:22:49.856804 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.856751, event: dispatched, request: pg_notify(24.b6(2) epoch 194471) v5 -3334> 2014-08-13 17:22:49.856925 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.856925, event: waiting_for_osdmap, request: pg_notify(24.b6(2) epoch 194471) v5 -3333> 2014-08-13 17:22:49.856939 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.856939, event: started, request: pg_notify(24.b6(2) epoch 194471) v5 -3332> 2014-08-13 17:22:49.856917 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194449 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3331> 2014-08-13 17:22:49.856789 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started 0.907291 0 0.000000 -3330> 2014-08-13 17:22:49.856963 7f74d8735700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:22:49.856962, event: done, request: pg_notify(24.b6(2) epoch 194471) v5 -3329> 2014-08-13 17:22:49.856981 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194470/192967) [6,374,4]/[6,374] r=-1 lpr=194470 pi=193125-194469/23 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Reset -3328> 2014-08-13 17:22:49.856994 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 5 ==== pg_query(24.e2 epoch 194471) v3 ==== 144+0+0 (3465705300 0 0) 0x4eb1e00 con 0x486b180 -3327> 2014-08-13 17:22:49.857014 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.845612, event: header_read, request: pg_query(24.e2 epoch 194471) v3 -3326> 2014-08-13 17:22:49.857021 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.845614, event: throttled, request: pg_query(24.e2 epoch 194471) v3 -3325> 2014-08-13 17:22:49.857026 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.845673, event: all_read, request: pg_query(24.e2 epoch 194471) v3 -3324> 2014-08-13 17:22:49.857036 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.857011, event: dispatched, request: pg_query(24.e2 epoch 194471) v3 -3323> 2014-08-13 17:22:49.857046 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.857046, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194471) v3 -3322> 2014-08-13 17:22:49.857057 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.857057, event: started, request: pg_query(24.e2 epoch 194471) v3 -3321> 2014-08-13 17:22:49.857080 7f74d8735700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:22:49.857080, event: done, request: pg_query(24.e2 epoch 194471) v3 -3320> 2014-08-13 17:22:49.857098 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 6 ==== pg_notify(24.9b epoch 194471) v5 ==== 694+0+0 (2287267152 0 0) 0x53b8fc0 con 0x486a940 -3319> 2014-08-13 17:22:49.857113 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.856269, event: header_read, request: pg_notify(24.9b epoch 194471) v5 -3318> 2014-08-13 17:22:49.857119 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.856271, event: throttled, request: pg_notify(24.9b epoch 194471) v5 -3317> 2014-08-13 17:22:49.857124 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.856419, event: all_read, request: pg_notify(24.9b epoch 194471) v5 -3316> 2014-08-13 17:22:49.857106 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.000125 1 0.000286 -3315> 2014-08-13 17:22:49.857129 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.857111, event: dispatched, request: pg_notify(24.9b epoch 194471) v5 -3314> 2014-08-13 17:22:49.857135 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.857135, event: waiting_for_osdmap, request: pg_notify(24.9b epoch 194471) v5 -3313> 2014-08-13 17:22:49.857142 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.857142, event: started, request: pg_notify(24.9b epoch 194471) v5 -3312> 2014-08-13 17:22:49.857133 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started -3311> 2014-08-13 17:22:49.857161 7f74d8735700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:22:49.857160, event: done, request: pg_notify(24.9b epoch 194471) v5 -3310> 2014-08-13 17:22:49.857174 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 11 ==== pg_info(1 pgs e194471:22.1fb(2)) v4 ==== 824+0+0 (749739194 0 0) 0x53b9dc0 con 0x486a3c0 -3309> 2014-08-13 17:22:49.857150 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start -3308> 2014-08-13 17:22:49.857201 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.804994, event: header_read, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3307> 2014-08-13 17:22:49.857196 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray -3306> 2014-08-13 17:22:49.857217 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.804996, event: throttled, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3305> 2014-08-13 17:22:49.857232 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000083 0 0.000000 -3304> 2014-08-13 17:22:49.857245 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.805058, event: all_read, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3303> 2014-08-13 17:22:49.857253 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.857191, event: dispatched, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3302> 2014-08-13 17:22:49.857260 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.857260, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3301> 2014-08-13 17:22:49.857253 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray -3300> 2014-08-13 17:22:49.857273 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.857273, event: started, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3299> 2014-08-13 17:22:49.857299 7f74d8735700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:22:49.857298, event: done, request: pg_info(1 pgs e194471:22.1fb(2)) v4 -3298> 2014-08-13 17:22:49.857320 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 9 ==== pg_notify(24.b6 epoch 194471) v5 ==== 694+0+0 (2092223685 0 0) 0x57d6200 con 0x486e040 -3297> 2014-08-13 17:22:49.857340 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.845688, event: header_read, request: pg_notify(24.b6 epoch 194471) v5 -3296> 2014-08-13 17:22:49.857350 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.845690, event: throttled, request: pg_notify(24.b6 epoch 194471) v5 -3295> 2014-08-13 17:22:49.857359 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.845753, event: all_read, request: pg_notify(24.b6 epoch 194471) v5 -3294> 2014-08-13 17:22:49.857369 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.857338, event: dispatched, request: pg_notify(24.b6 epoch 194471) v5 -3293> 2014-08-13 17:22:49.857380 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.857380, event: waiting_for_osdmap, request: pg_notify(24.b6 epoch 194471) v5 -3292> 2014-08-13 17:22:49.857387 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.857387, event: started, request: pg_notify(24.b6 epoch 194471) v5 -3291> 2014-08-13 17:22:49.857409 7f74d8735700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:22:49.857409, event: done, request: pg_notify(24.b6 epoch 194471) v5 -3290> 2014-08-13 17:22:49.857427 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e10d80 con 0x48dfe80 -3289> 2014-08-13 17:22:49.857453 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3288> 2014-08-13 17:22:49.857464 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 11 ==== pg_notify(22.707(24) epoch 194471) v5 ==== 2066+0+0 (2293908023 0 0) 0x53bcb40 con 0x4901600 -3287> 2014-08-13 17:22:49.857478 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.827208, event: header_read, request: pg_notify(22.707(24) epoch 194471) v5 -3286> 2014-08-13 17:22:49.857485 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.827209, event: throttled, request: pg_notify(22.707(24) epoch 194471) v5 -3285> 2014-08-13 17:22:49.857491 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.827334, event: all_read, request: pg_notify(22.707(24) epoch 194471) v5 -3284> 2014-08-13 17:22:49.857496 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.857476, event: dispatched, request: pg_notify(22.707(24) epoch 194471) v5 -3283> 2014-08-13 17:22:49.857508 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.857508, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194471) v5 -3282> 2014-08-13 17:22:49.857519 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.857519, event: started, request: pg_notify(22.707(24) epoch 194471) v5 -3281> 2014-08-13 17:22:49.857539 7f74d8735700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:22:49.857539, event: done, request: pg_notify(22.707(24) epoch 194471) v5 -3280> 2014-08-13 17:22:49.857568 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.324 192.168.1.104:6814/65209 10 ==== pg_info(1 pgs e194471:22.22c) v4 ==== 714+0+0 (1337988637 0 0) 0x53b8c40 con 0x4903180 -3279> 2014-08-13 17:22:49.857587 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857265, event: header_read, request: pg_info(1 pgs e194471:22.22c) v4 -3278> 2014-08-13 17:22:49.857597 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857268, event: throttled, request: pg_info(1 pgs e194471:22.22c) v4 -3277> 2014-08-13 17:22:49.857583 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.896684 3 0.000092 -3276> 2014-08-13 17:22:49.857606 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857354, event: all_read, request: pg_info(1 pgs e194471:22.22c) v4 -3275> 2014-08-13 17:22:49.857631 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857585, event: dispatched, request: pg_info(1 pgs e194471:22.22c) v4 -3274> 2014-08-13 17:22:49.857624 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.906424 0 0.000000 -3273> 2014-08-13 17:22:49.857648 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857648, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.22c) v4 -3272> 2014-08-13 17:22:49.857656 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857656, event: started, request: pg_info(1 pgs e194471:22.22c) v4 -3271> 2014-08-13 17:22:49.857661 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194449 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3270> 2014-08-13 17:22:49.857683 7f74d8735700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:22:49.857683, event: done, request: pg_info(1 pgs e194471:22.22c) v4 -3269> 2014-08-13 17:22:49.857701 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e35340 con 0x4906880 -3268> 2014-08-13 17:22:49.857724 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3267> 2014-08-13 17:22:49.857738 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 8 ==== pg_notify(24.9b(2) epoch 194471) v5 ==== 870+0+0 (3792926027 0 0) 0x53bd5c0 con 0x4d51ce0 -3266> 2014-08-13 17:22:49.857756 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.856163, event: header_read, request: pg_notify(24.9b(2) epoch 194471) v5 -3265> 2014-08-13 17:22:49.857741 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.44 192.168.1.111:6822/10782 -- pg_log(22.3af epoch 194471 log log((0'0,194215'539], crt=191727'457) query_epoch 194471) v4 -- ?+0 0x5c73000 -3264> 2014-08-13 17:22:49.857767 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.856165, event: throttled, request: pg_notify(24.9b(2) epoch 194471) v5 -3263> 2014-08-13 17:22:49.857776 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.856251, event: all_read, request: pg_notify(24.9b(2) epoch 194471) v5 -3262> 2014-08-13 17:22:49.857785 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.857754, event: dispatched, request: pg_notify(24.9b(2) epoch 194471) v5 -3261> 2014-08-13 17:22:49.857795 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.857794, event: waiting_for_osdmap, request: pg_notify(24.9b(2) epoch 194471) v5 -3260> 2014-08-13 17:22:49.857806 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.857806, event: started, request: pg_notify(24.9b(2) epoch 194471) v5 -3259> 2014-08-13 17:22:49.857831 7f74d8735700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:22:49.857831, event: done, request: pg_notify(24.9b(2) epoch 194471) v5 -3258> 2014-08-13 17:22:49.857848 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4803a80 con 0x4d511e0 -3257> 2014-08-13 17:22:49.857867 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3256> 2014-08-13 17:22:49.857880 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 3 ==== pg_notify(24.34(35) epoch 194471) v5 ==== 2513+0+0 (2259962378 0 0) 0x5263f00 con 0x4d2aaa0 -3255> 2014-08-13 17:22:49.857897 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.856243, event: header_read, request: pg_notify(24.34(35) epoch 194471) v5 -3254> 2014-08-13 17:22:49.857908 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.856246, event: throttled, request: pg_notify(24.34(35) epoch 194471) v5 -3253> 2014-08-13 17:22:49.857917 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.856378, event: all_read, request: pg_notify(24.34(35) epoch 194471) v5 -3252> 2014-08-13 17:22:49.857926 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.857895, event: dispatched, request: pg_notify(24.34(35) epoch 194471) v5 -3251> 2014-08-13 17:22:49.857939 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.857938, event: waiting_for_osdmap, request: pg_notify(24.34(35) epoch 194471) v5 -3250> 2014-08-13 17:22:49.857950 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.857949, event: started, request: pg_notify(24.34(35) epoch 194471) v5 -3249> 2014-08-13 17:22:49.857975 7f74d8735700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:22:49.857974, event: done, request: pg_notify(24.34(35) epoch 194471) v5 -3248> 2014-08-13 17:22:49.857967 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3247> 2014-08-13 17:22:49.858006 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.357 192.168.1.102:6834/17224 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a80d80 con 0x4d37a60 -3246> 2014-08-13 17:22:49.858028 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3245> 2014-08-13 17:22:49.858041 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.307 192.168.1.109:6804/31016 1 ==== osd_map(194456..194471 src has 185139..194471) v3 ==== 10429+0+0 (970627098 0 0) 0x4e31680 con 0x5419fa0 -3244> 2014-08-13 17:22:49.858059 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194456,194471], i have 194471, src has [185139,194471] -3243> 2014-08-13 17:22:49.858044 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.362 192.168.1.108:6842/8170 -- pg_log(22.3af epoch 194471 log log((0'0,194215'539], crt=191727'457) query_epoch 194471) v4 -- ?+0 0x5c72a00 -3242> 2014-08-13 17:22:49.858076 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 7 ==== pg_notify(22.3d5(2) epoch 194471) v5 ==== 828+0+0 (2355573931 0 0) 0x4ac3d40 con 0x45f1760 -3241> 2014-08-13 17:22:49.858095 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.853417, event: header_read, request: pg_notify(22.3d5(2) epoch 194471) v5 -3240> 2014-08-13 17:22:49.858103 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.853418, event: throttled, request: pg_notify(22.3d5(2) epoch 194471) v5 -3239> 2014-08-13 17:22:49.858108 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.853505, event: all_read, request: pg_notify(22.3d5(2) epoch 194471) v5 -3238> 2014-08-13 17:22:49.858114 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.858093, event: dispatched, request: pg_notify(22.3d5(2) epoch 194471) v5 -3237> 2014-08-13 17:22:49.858143 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started/Stray 0.906057 4 0.000308 -3236> 2014-08-13 17:22:49.858183 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started 0.906196 0 0.000000 -3235> 2014-08-13 17:22:49.858203 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194470/193232) [374,4,314] r=1 lpr=194470 pi=192688-194469/39 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset -3234> 2014-08-13 17:22:49.858119 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.858119, event: waiting_for_osdmap, request: pg_notify(22.3d5(2) epoch 194471) v5 -3233> 2014-08-13 17:22:49.858243 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.858242, event: started, request: pg_notify(22.3d5(2) epoch 194471) v5 -3232> 2014-08-13 17:22:49.858273 7f74d8735700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:22:49.858272, event: done, request: pg_notify(22.3d5(2) epoch 194471) v5 -3231> 2014-08-13 17:22:49.858290 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 8 ==== pg_info(1 pgs e194471:22.22c) v4 ==== 714+0+0 (2587452909 0 0) 0x49e2300 con 0x45f5540 -3230> 2014-08-13 17:22:49.858307 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.851895, event: header_read, request: pg_info(1 pgs e194471:22.22c) v4 -3229> 2014-08-13 17:22:49.858313 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.851896, event: throttled, request: pg_info(1 pgs e194471:22.22c) v4 -3228> 2014-08-13 17:22:49.858318 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.851964, event: all_read, request: pg_info(1 pgs e194471:22.22c) v4 -3227> 2014-08-13 17:22:49.858323 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.858304, event: dispatched, request: pg_info(1 pgs e194471:22.22c) v4 -3226> 2014-08-13 17:22:49.858329 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.858329, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.22c) v4 -3225> 2014-08-13 17:22:49.858288 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3224> 2014-08-13 17:22:49.858337 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.858336, event: started, request: pg_info(1 pgs e194471:22.22c) v4 -3223> 2014-08-13 17:22:49.858357 7f74d8735700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:22:49.858357, event: done, request: pg_info(1 pgs e194471:22.22c) v4 -3222> 2014-08-13 17:22:49.858353 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Reset 0.000150 1 0.000116 -3221> 2014-08-13 17:22:49.858372 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 10 ==== pg_info(1 pgs e194471:22.4df) v4 ==== 714+0+0 (355172088 0 0) 0x49e2140 con 0x45f5960 -3220> 2014-08-13 17:22:49.858387 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.842343, event: header_read, request: pg_info(1 pgs e194471:22.4df) v4 -3219> 2014-08-13 17:22:49.858380 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started -3218> 2014-08-13 17:22:49.858393 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.842346, event: throttled, request: pg_info(1 pgs e194471:22.4df) v4 -3217> 2014-08-13 17:22:49.858399 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.842438, event: all_read, request: pg_info(1 pgs e194471:22.4df) v4 -3216> 2014-08-13 17:22:49.858405 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.858385, event: dispatched, request: pg_info(1 pgs e194471:22.4df) v4 -3215> 2014-08-13 17:22:49.858399 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Start -3214> 2014-08-13 17:22:49.858410 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.858410, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4df) v4 -3213> 2014-08-13 17:22:49.858418 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.858418, event: started, request: pg_info(1 pgs e194471:22.4df) v4 -3212> 2014-08-13 17:22:49.858416 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: transitioning to Stray -3211> 2014-08-13 17:22:49.858437 7f74d8735700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:22:49.858436, event: done, request: pg_info(1 pgs e194471:22.4df) v4 -3210> 2014-08-13 17:22:49.858435 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Start 0.000037 0 0.000000 -3209> 2014-08-13 17:22:49.858450 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.344 192.168.1.106:6812/6757 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a81440 con 0x45f6300 -3208> 2014-08-13 17:22:49.858455 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/Stray -3207> 2014-08-13 17:22:49.858467 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3206> 2014-08-13 17:22:49.858476 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 8 ==== pg_notify(24.3d(2) epoch 194471) v5 ==== 870+0+0 (1912751027 0 0) 0x4ac3b80 con 0x4758840 -3205> 2014-08-13 17:22:49.858490 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.855835, event: header_read, request: pg_notify(24.3d(2) epoch 194471) v5 -3204> 2014-08-13 17:22:49.858496 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.855837, event: throttled, request: pg_notify(24.3d(2) epoch 194471) v5 -3203> 2014-08-13 17:22:49.858502 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.855930, event: all_read, request: pg_notify(24.3d(2) epoch 194471) v5 -3202> 2014-08-13 17:22:49.858507 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.858488, event: dispatched, request: pg_notify(24.3d(2) epoch 194471) v5 -3201> 2014-08-13 17:22:49.858513 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.858513, event: waiting_for_osdmap, request: pg_notify(24.3d(2) epoch 194471) v5 -3200> 2014-08-13 17:22:49.858520 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.858520, event: started, request: pg_notify(24.3d(2) epoch 194471) v5 -3199> 2014-08-13 17:22:49.858539 7f74d8735700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:22:49.858539, event: done, request: pg_notify(24.3d(2) epoch 194471) v5 -3198> 2014-08-13 17:22:49.858552 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 6 ==== pg_notify(24.34(48) epoch 194471) v5 ==== 3196+0+0 (3690207742 0 0) 0x5264440 con 0x475a940 -3197> 2014-08-13 17:22:49.858566 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.856213, event: header_read, request: pg_notify(24.34(48) epoch 194471) v5 -3196> 2014-08-13 17:22:49.858572 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.856215, event: throttled, request: pg_notify(24.34(48) epoch 194471) v5 -3195> 2014-08-13 17:22:49.858584 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.856355, event: all_read, request: pg_notify(24.34(48) epoch 194471) v5 -3194> 2014-08-13 17:22:49.858593 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.858564, event: dispatched, request: pg_notify(24.34(48) epoch 194471) v5 -3193> 2014-08-13 17:22:49.858602 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.858602, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194471) v5 -3192> 2014-08-13 17:22:49.858613 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.858613, event: started, request: pg_notify(24.34(48) epoch 194471) v5 -3191> 2014-08-13 17:22:49.858637 7f74d8735700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:22:49.858637, event: done, request: pg_notify(24.34(48) epoch 194471) v5 -3190> 2014-08-13 17:22:49.858667 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 13 ==== pg_notify(22.481(32) epoch 194471) v5 ==== 2518+0+0 (1911027166 0 0) 0x53baa00 con 0x475ee00 -3189> 2014-08-13 17:22:49.858686 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.812395, event: header_read, request: pg_notify(22.481(32) epoch 194471) v5 -3188> 2014-08-13 17:22:49.858697 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.812398, event: throttled, request: pg_notify(22.481(32) epoch 194471) v5 -3187> 2014-08-13 17:22:49.858678 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.896685 3 0.000102 -3186> 2014-08-13 17:22:49.858689 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.904773 4 0.000324 -3185> 2014-08-13 17:22:49.858706 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.812534, event: all_read, request: pg_notify(22.481(32) epoch 194471) v5 -3184> 2014-08-13 17:22:49.858729 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.858684, event: dispatched, request: pg_notify(22.481(32) epoch 194471) v5 -3183> 2014-08-13 17:22:49.858722 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.917528 0 0.000000 -3182> 2014-08-13 17:22:49.858740 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.858740, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194471) v5 -3181> 2014-08-13 17:22:49.858752 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.858752, event: started, request: pg_notify(22.481(32) epoch 194471) v5 -3180> 2014-08-13 17:22:49.858721 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started 0.904942 0 0.000000 -3179> 2014-08-13 17:22:49.858754 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194449 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3178> 2014-08-13 17:22:49.858780 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194470/193186) [45,319,4]/[45,319] r=-1 lpr=194470 pi=193186-194469/23 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Reset -3177> 2014-08-13 17:22:49.858869 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000089 1 0.000148 -3176> 2014-08-13 17:22:49.858897 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started -3175> 2014-08-13 17:22:49.858917 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start -3174> 2014-08-13 17:22:49.858935 7f74d192a700 1 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -3173> 2014-08-13 17:22:49.858956 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000039 0 0.000000 -3172> 2014-08-13 17:22:49.858978 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -3171> 2014-08-13 17:22:49.859112 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3170> 2014-08-13 17:22:49.859125 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194465/194457 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.910206 7 0.000263 -3169> 2014-08-13 17:22:49.859164 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194465/194457 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3168> 2014-08-13 17:22:49.859193 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194449 n=28 ec=183096 les/c 194465/194457 194470/194470/192971) [51,4,308] r=1 lpr=194470 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3167> 2014-08-13 17:22:49.859307 7f74d8735700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:22:49.859307, event: done, request: pg_notify(22.481(32) epoch 194471) v5 -3166> 2014-08-13 17:22:49.859334 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 10 ==== pg_query(24.5e epoch 194471) v3 ==== 144+0+0 (224747979 0 0) 0x29c05a0 con 0x47a27e0 -3165> 2014-08-13 17:22:49.859354 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.800398, event: header_read, request: pg_query(24.5e epoch 194471) v3 -3164> 2014-08-13 17:22:49.859365 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.800401, event: throttled, request: pg_query(24.5e epoch 194471) v3 -3163> 2014-08-13 17:22:49.859374 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.800478, event: all_read, request: pg_query(24.5e epoch 194471) v3 -3162> 2014-08-13 17:22:49.859337 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.898849 3 0.000218 -3161> 2014-08-13 17:22:49.859385 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.859352, event: dispatched, request: pg_query(24.5e epoch 194471) v3 -3160> 2014-08-13 17:22:49.859404 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.859404, event: waiting_for_osdmap, request: pg_query(24.5e epoch 194471) v3 -3159> 2014-08-13 17:22:49.859399 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.917145 0 0.000000 -3158> 2014-08-13 17:22:49.859417 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.859416, event: started, request: pg_query(24.5e epoch 194471) v3 -3157> 2014-08-13 17:22:49.859448 7f74d8735700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:22:49.859447, event: done, request: pg_query(24.5e epoch 194471) v3 -3156> 2014-08-13 17:22:49.859447 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194449 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3155> 2014-08-13 17:22:49.859441 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194465/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.943178 7 0.000316 -3154> 2014-08-13 17:22:49.859465 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 5 ==== pg_query(24.1de epoch 194471) v3 ==== 144+0+0 (3562607754 0 0) 0x4eb41a0 con 0x47a4200 -3153> 2014-08-13 17:22:49.859488 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194465/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3152> 2014-08-13 17:22:49.859495 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.857143, event: header_read, request: pg_query(24.1de epoch 194471) v3 -3151> 2014-08-13 17:22:49.859519 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.857145, event: throttled, request: pg_query(24.1de epoch 194471) v3 -3150> 2014-08-13 17:22:49.859516 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194449 n=36 ec=183096 les/c 194465/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3149> 2014-08-13 17:22:49.859529 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.857223, event: all_read, request: pg_query(24.1de epoch 194471) v3 -3148> 2014-08-13 17:22:49.859538 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.859492, event: dispatched, request: pg_query(24.1de epoch 194471) v3 -3147> 2014-08-13 17:22:49.859547 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.859547, event: waiting_for_osdmap, request: pg_query(24.1de epoch 194471) v3 -3146> 2014-08-13 17:22:49.859558 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.859558, event: started, request: pg_query(24.1de epoch 194471) v3 -3145> 2014-08-13 17:22:49.859580 7f74d8735700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:22:49.859579, event: done, request: pg_query(24.1de epoch 194471) v3 -3144> 2014-08-13 17:22:49.859596 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 8 ==== pg_query(24.2a1 epoch 194471) v3 ==== 144+0+0 (3205511423 0 0) 0x29c21c0 con 0x47a53e0 -3143> 2014-08-13 17:22:49.859615 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.844993, event: header_read, request: pg_query(24.2a1 epoch 194471) v3 -3142> 2014-08-13 17:22:49.859625 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.844995, event: throttled, request: pg_query(24.2a1 epoch 194471) v3 -3141> 2014-08-13 17:22:49.859634 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.845052, event: all_read, request: pg_query(24.2a1 epoch 194471) v3 -3140> 2014-08-13 17:22:49.859642 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.859613, event: dispatched, request: pg_query(24.2a1 epoch 194471) v3 -3139> 2014-08-13 17:22:49.859635 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(24),24.cd(24) epoch 194471) v5 -- ?+0 0x53b9f80 con 0x4835ac0 -3138> 2014-08-13 17:22:49.859652 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.859652, event: waiting_for_osdmap, request: pg_query(24.2a1 epoch 194471) v3 -3137> 2014-08-13 17:22:49.859660 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.859660, event: started, request: pg_query(24.2a1 epoch 194471) v3 -3136> 2014-08-13 17:22:49.859668 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(24) epoch 194471) v5 -- ?+0 0x53bad80 con 0x47a32e0 -3135> 2014-08-13 17:22:49.859680 7f74d8735700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:22:49.859679, event: done, request: pg_query(24.2a1 epoch 194471) v3 -3134> 2014-08-13 17:22:49.859692 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(20) epoch 194471) v5 -- ?+0 0x53bd400 con 0x4831b80 -3133> 2014-08-13 17:22:49.859692 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 6 ==== pg_notify(24.34(48) epoch 194471) v5 ==== 3196+0+0 (2475150164 0 0) 0x5264600 con 0x47a3f40 -3132> 2014-08-13 17:22:49.859719 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(20) epoch 194471) v5 -- ?+0 0x53bc7c0 con 0x4758b00 -3131> 2014-08-13 17:22:49.859722 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.853633, event: header_read, request: pg_notify(24.34(48) epoch 194471) v5 -3130> 2014-08-13 17:22:49.859730 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.853634, event: throttled, request: pg_notify(24.34(48) epoch 194471) v5 -3129> 2014-08-13 17:22:49.859735 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.853790, event: all_read, request: pg_notify(24.34(48) epoch 194471) v5 -3128> 2014-08-13 17:22:49.859741 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.859719, event: dispatched, request: pg_notify(24.34(48) epoch 194471) v5 -3127> 2014-08-13 17:22:49.859738 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(19) epoch 194471) v5 -- ?+0 0x57d6900 con 0x475c4c0 -3126> 2014-08-13 17:22:49.859746 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.859746, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194471) v5 -3125> 2014-08-13 17:22:49.859754 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.859754, event: started, request: pg_notify(24.34(48) epoch 194471) v5 -3124> 2014-08-13 17:22:49.859756 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(40) epoch 194471) v5 -- ?+0 0x57d7380 con 0x486b9c0 -3123> 2014-08-13 17:22:49.859775 7f74d8735700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:22:49.859775, event: done, request: pg_notify(24.34(48) epoch 194471) v5 -3122> 2014-08-13 17:22:49.859812 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 9 ==== pg_info(1 pgs e194471:22.113) v4 ==== 714+0+0 (2469436721 0 0) 0x57d2d80 con 0x47a32e0 -3121> 2014-08-13 17:22:49.859828 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.848406, event: header_read, request: pg_info(1 pgs e194471:22.113) v4 -3120> 2014-08-13 17:22:49.859834 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.848408, event: throttled, request: pg_info(1 pgs e194471:22.113) v4 -3119> 2014-08-13 17:22:49.859839 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.848492, event: all_read, request: pg_info(1 pgs e194471:22.113) v4 -3118> 2014-08-13 17:22:49.859844 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.859825, event: dispatched, request: pg_info(1 pgs e194471:22.113) v4 -3117> 2014-08-13 17:22:49.859850 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.859850, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.113) v4 -3116> 2014-08-13 17:22:49.859857 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.859857, event: started, request: pg_info(1 pgs e194471:22.113) v4 -3115> 2014-08-13 17:22:49.859881 7f74d8735700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:22:49.859881, event: done, request: pg_info(1 pgs e194471:22.113) v4 -3114> 2014-08-13 17:22:49.859898 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 8 ==== pg_info(1 pgs e194471:22.84) v4 ==== 714+0+0 (736589484 0 0) 0x53bfc40 con 0x47f0c60 -3113> 2014-08-13 17:22:49.859916 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.846606, event: header_read, request: pg_info(1 pgs e194471:22.84) v4 -3112> 2014-08-13 17:22:49.859926 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.846608, event: throttled, request: pg_info(1 pgs e194471:22.84) v4 -3111> 2014-08-13 17:22:49.859935 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.846700, event: all_read, request: pg_info(1 pgs e194471:22.84) v4 -3110> 2014-08-13 17:22:49.859944 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.859914, event: dispatched, request: pg_info(1 pgs e194471:22.84) v4 -3109> 2014-08-13 17:22:49.859953 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.859953, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.84) v4 -3108> 2014-08-13 17:22:49.859964 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.859964, event: started, request: pg_info(1 pgs e194471:22.84) v4 -3107> 2014-08-13 17:22:49.859959 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.341 192.168.1.106:6818/9142 -- pg_log(22.48e epoch 194471 log log((0'0,194215'543], crt=191727'473) query_epoch 194471) v4 -- ?+0 0x5c72400 -3106> 2014-08-13 17:22:49.859990 7f74d8735700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:22:49.859990, event: done, request: pg_info(1 pgs e194471:22.84) v4 -3105> 2014-08-13 17:22:49.860004 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a82640 con 0x47f6880 -3104> 2014-08-13 17:22:49.860024 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -3103> 2014-08-13 17:22:49.860017 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(22) epoch 194471) v5 -- ?+0 0x57d78c0 con 0x4835280 -3102> 2014-08-13 17:22:49.860041 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 14 ==== pg_info(1 pgs e194471:22.226(1)) v4 ==== 773+0+0 (2813132905 0 0) 0x49e24c0 con 0x4835280 -3101> 2014-08-13 17:22:49.860051 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6820/19173 -- pg_info(1 pgs e194471:22.3d5(3)) v4 -- ?+0 0x57d3480 con 0x48dfe80 -3100> 2014-08-13 17:22:49.860060 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.804752, event: header_read, request: pg_info(1 pgs e194471:22.226(1)) v4 -3099> 2014-08-13 17:22:49.860071 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.804753, event: throttled, request: pg_info(1 pgs e194471:22.226(1)) v4 -3098> 2014-08-13 17:22:49.860080 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.804809, event: all_read, request: pg_info(1 pgs e194471:22.226(1)) v4 -3097> 2014-08-13 17:22:49.860089 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.860058, event: dispatched, request: pg_info(1 pgs e194471:22.226(1)) v4 -3096> 2014-08-13 17:22:49.860091 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6814/25893 -- pg_info(1 pgs e194471:22.3d9(41)) v4 -- ?+0 0x57d7540 con 0x4834620 -3095> 2014-08-13 17:22:49.860099 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.860099, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.226(1)) v4 -3094> 2014-08-13 17:22:49.860112 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.860112, event: started, request: pg_info(1 pgs e194471:22.226(1)) v4 -3093> 2014-08-13 17:22:49.860114 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:22.1b8(3)) v4 -- ?+0 0x57d6e40 con 0x47a3f40 -3092> 2014-08-13 17:22:49.860132 7f74d8735700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:22:49.860132, event: done, request: pg_info(1 pgs e194471:22.226(1)) v4 -3091> 2014-08-13 17:22:49.860134 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_info(1 pgs e194471:22.3d5(3)) v4 -- ?+0 0x57d3640 con 0x48377a0 -3090> 2014-08-13 17:22:49.860146 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 12 ==== pg_info(1 pgs e194471:22.39d) v4 ==== 714+0+0 (2951161645 0 0) 0x53bcec0 con 0x486a3c0 -3089> 2014-08-13 17:22:49.860160 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.842158, event: header_read, request: pg_info(1 pgs e194471:22.39d) v4 -3088> 2014-08-13 17:22:49.860167 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.842161, event: throttled, request: pg_info(1 pgs e194471:22.39d) v4 -3087> 2014-08-13 17:22:49.860164 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6800/4640 -- pg_info(1 pgs e194471:22.1b8(3)) v4 -- ?+0 0x57d32c0 con 0x4831600 -3086> 2014-08-13 17:22:49.860172 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.842265, event: all_read, request: pg_info(1 pgs e194471:22.39d) v4 -3085> 2014-08-13 17:22:49.860177 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.860159, event: dispatched, request: pg_info(1 pgs e194471:22.39d) v4 -3084> 2014-08-13 17:22:49.860183 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.860182, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.39d) v4 -3083> 2014-08-13 17:22:49.860189 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.860189, event: started, request: pg_info(1 pgs e194471:22.39d) v4 -3082> 2014-08-13 17:22:49.860197 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_info(1 pgs e194471:22.3d9(41)) v4 -- ?+0 0x57d4980 con 0x4835280 -3081> 2014-08-13 17:22:49.860217 7f74d8735700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:22:49.860217, event: done, request: pg_info(1 pgs e194471:22.39d) v4 -3080> 2014-08-13 17:22:49.860235 7f74d192a700 10 monclient: _send_mon_message to mon.storage0113-ib at 192.168.100.113:6789/0 -3079> 2014-08-13 17:22:49.860242 7f74d192a700 1 -- 192.168.100.115:6805/30165 --> 192.168.100.113:6789/0 -- osd_pgtemp(e194471 {24.1bb=[4,332,337]} v194471) v1 -- ?+0 0x4eb3c00 con 0x45d48e0 -3078> 2014-08-13 17:22:49.860231 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 10 ==== pg_query(24.3f7 epoch 194471) v3 ==== 144+0+0 (1135023774 0 0) 0x4eb5460 con 0x486e040 -3077> 2014-08-13 17:22:49.860283 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.847437, event: header_read, request: pg_query(24.3f7 epoch 194471) v3 -3076> 2014-08-13 17:22:49.860268 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3075> 2014-08-13 17:22:49.860290 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.847438, event: throttled, request: pg_query(24.3f7 epoch 194471) v3 -3074> 2014-08-13 17:22:49.860296 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.847496, event: all_read, request: pg_query(24.3f7 epoch 194471) v3 -3073> 2014-08-13 17:22:49.860300 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.860281, event: dispatched, request: pg_query(24.3f7 epoch 194471) v3 -3072> 2014-08-13 17:22:49.860307 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.860307, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194471) v3 -3071> 2014-08-13 17:22:49.860319 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.860319, event: started, request: pg_query(24.3f7 epoch 194471) v3 -3070> 2014-08-13 17:22:49.860343 7f74d8735700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:22:49.860343, event: done, request: pg_query(24.3f7 epoch 194471) v3 -3069> 2014-08-13 17:22:49.860360 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 6 ==== pg_notify(24.30b epoch 194471) v5 ==== 694+0+0 (2138177118 0 0) 0x49e7380 con 0x48dfe80 -3068> 2014-08-13 17:22:49.860378 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.850739, event: header_read, request: pg_notify(24.30b epoch 194471) v5 -3067> 2014-08-13 17:22:49.860388 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.850740, event: throttled, request: pg_notify(24.30b epoch 194471) v5 -3066> 2014-08-13 17:22:49.860397 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.850802, event: all_read, request: pg_notify(24.30b epoch 194471) v5 -3065> 2014-08-13 17:22:49.860406 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.860376, event: dispatched, request: pg_notify(24.30b epoch 194471) v5 -3064> 2014-08-13 17:22:49.860415 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.860415, event: waiting_for_osdmap, request: pg_notify(24.30b epoch 194471) v5 -3063> 2014-08-13 17:22:49.860426 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.860426, event: started, request: pg_notify(24.30b epoch 194471) v5 -3062> 2014-08-13 17:22:49.860448 7f74d8735700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:22:49.860447, event: done, request: pg_notify(24.30b epoch 194471) v5 -3061> 2014-08-13 17:22:49.860450 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.925965 7 0.000244 -3060> 2014-08-13 17:22:49.860464 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 5 ==== pg_notify(22.f3(1) epoch 194471) v5 ==== 777+0+0 (2901531829 0 0) 0x53bde80 con 0x4906880 -3059> 2014-08-13 17:22:49.860496 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.856829, event: header_read, request: pg_notify(22.f3(1) epoch 194471) v5 -3058> 2014-08-13 17:22:49.860489 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3057> 2014-08-13 17:22:49.860511 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.856831, event: throttled, request: pg_notify(22.f3(1) epoch 194471) v5 -3056> 2014-08-13 17:22:49.860518 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.856916, event: all_read, request: pg_notify(22.f3(1) epoch 194471) v5 -3055> 2014-08-13 17:22:49.860527 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.860493, event: dispatched, request: pg_notify(22.f3(1) epoch 194471) v5 -3054> 2014-08-13 17:22:49.860523 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192946) [334,289,4] r=2 lpr=194470 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3053> 2014-08-13 17:22:49.860537 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.860537, event: waiting_for_osdmap, request: pg_notify(22.f3(1) epoch 194471) v5 -3052> 2014-08-13 17:22:49.860548 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.860548, event: started, request: pg_notify(22.f3(1) epoch 194471) v5 -3051> 2014-08-13 17:22:49.860572 7f74d8735700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:22:49.860572, event: done, request: pg_notify(22.f3(1) epoch 194471) v5 -3050> 2014-08-13 17:22:49.860589 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 4 ==== pg_notify(24.9b epoch 194471) v5 ==== 694+0+0 (2287267152 0 0) 0x57d5b00 con 0x4d511e0 -3049> 2014-08-13 17:22:49.860607 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.856539, event: header_read, request: pg_notify(24.9b epoch 194471) v5 -3048> 2014-08-13 17:22:49.860617 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.856542, event: throttled, request: pg_notify(24.9b epoch 194471) v5 -3047> 2014-08-13 17:22:49.860626 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.856607, event: all_read, request: pg_notify(24.9b epoch 194471) v5 -3046> 2014-08-13 17:22:49.860635 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.860605, event: dispatched, request: pg_notify(24.9b epoch 194471) v5 -3045> 2014-08-13 17:22:49.860644 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.860644, event: waiting_for_osdmap, request: pg_notify(24.9b epoch 194471) v5 -3044> 2014-08-13 17:22:49.860656 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.860655, event: started, request: pg_notify(24.9b epoch 194471) v5 -3043> 2014-08-13 17:22:49.860645 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.923013 7 0.000256 -3042> 2014-08-13 17:22:49.860678 7f74d8735700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:22:49.860677, event: done, request: pg_notify(24.9b epoch 194471) v5 -3041> 2014-08-13 17:22:49.860670 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3040> 2014-08-13 17:22:49.860699 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.357 192.168.1.102:6834/17224 4 ==== pg_info(1 pgs e194471:22.4df) v4 ==== 714+0+0 (355172088 0 0) 0x49e7a80 con 0x4d37a60 -3039> 2014-08-13 17:22:49.860652 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started/Stray 0.916340 4 0.000269 -3038> 2014-08-13 17:22:49.860703 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192976) [367,44,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3037> 2014-08-13 17:22:49.860719 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.855039, event: header_read, request: pg_info(1 pgs e194471:22.4df) v4 -3036> 2014-08-13 17:22:49.860729 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.855041, event: throttled, request: pg_info(1 pgs e194471:22.4df) v4 -3035> 2014-08-13 17:22:49.860738 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.855118, event: all_read, request: pg_info(1 pgs e194471:22.4df) v4 -3034> 2014-08-13 17:22:49.860724 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started 0.916516 0 0.000000 -3033> 2014-08-13 17:22:49.860753 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.860717, event: dispatched, request: pg_info(1 pgs e194471:22.4df) v4 -3032> 2014-08-13 17:22:49.860773 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.860773, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4df) v4 -3031> 2014-08-13 17:22:49.860786 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.860786, event: started, request: pg_info(1 pgs e194471:22.4df) v4 -3030> 2014-08-13 17:22:49.860773 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194470/193262) [343,45,4]/[343,45] r=-1 lpr=194470 pi=193262-194469/23 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Reset -3029> 2014-08-13 17:22:49.860810 7f74d8735700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:22:49.860809, event: done, request: pg_info(1 pgs e194471:22.4df) v4 -3028> 2014-08-13 17:22:49.860838 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.307 192.168.1.109:6804/31016 2 ==== pg_notify(24.181(31) epoch 194471) v5 ==== 2493+0+0 (1820897233 0 0) 0x49e0a80 con 0x5419fa0 -3027> 2014-08-13 17:22:49.860858 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.855987, event: header_read, request: pg_notify(24.181(31) epoch 194471) v5 -3026> 2014-08-13 17:22:49.860868 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.855989, event: throttled, request: pg_notify(24.181(31) epoch 194471) v5 -3025> 2014-08-13 17:22:49.860878 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.856104, event: all_read, request: pg_notify(24.181(31) epoch 194471) v5 -3024> 2014-08-13 17:22:49.860887 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.860856, event: dispatched, request: pg_notify(24.181(31) epoch 194471) v5 -3023> 2014-08-13 17:22:49.860897 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.860896, event: waiting_for_osdmap, request: pg_notify(24.181(31) epoch 194471) v5 -3022> 2014-08-13 17:22:49.860908 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.860908, event: started, request: pg_notify(24.181(31) epoch 194471) v5 -3021> 2014-08-13 17:22:49.860932 7f74d8735700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:22:49.860931, event: done, request: pg_notify(24.181(31) epoch 194471) v5 -3020> 2014-08-13 17:22:49.860965 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 8 ==== pg_notify(24.30b(25) epoch 194471) v5 ==== 2155+0+0 (1498764271 0 0) 0x4ac2f40 con 0x45f1760 -3019> 2014-08-13 17:22:49.860895 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.000123 1 0.000206 -3018> 2014-08-13 17:22:49.860986 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.856970, event: header_read, request: pg_notify(24.30b(25) epoch 194471) v5 -3017> 2014-08-13 17:22:49.860996 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.856972, event: throttled, request: pg_notify(24.30b(25) epoch 194471) v5 -3016> 2014-08-13 17:22:49.860990 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started -3015> 2014-08-13 17:22:49.861006 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.857144, event: all_read, request: pg_notify(24.30b(25) epoch 194471) v5 -3014> 2014-08-13 17:22:49.861015 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.860984, event: dispatched, request: pg_notify(24.30b(25) epoch 194471) v5 -3013> 2014-08-13 17:22:49.861030 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.861030, event: waiting_for_osdmap, request: pg_notify(24.30b(25) epoch 194471) v5 -3012> 2014-08-13 17:22:49.861008 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start -3011> 2014-08-13 17:22:49.861044 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.861044, event: started, request: pg_notify(24.30b(25) epoch 194471) v5 -3010> 2014-08-13 17:22:49.861064 7f74d8735700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:22:49.861064, event: done, request: pg_notify(24.30b(25) epoch 194471) v5 -3009> 2014-08-13 17:22:49.861052 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray -3008> 2014-08-13 17:22:49.861091 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.344 192.168.1.106:6812/6757 4 ==== pg_info(1 pgs e194471:22.39d) v4 ==== 714+0+0 (2951161645 0 0) 0x49e2680 con 0x45f6300 -3007> 2014-08-13 17:22:49.861089 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000081 0 0.000000 -3006> 2014-08-13 17:22:49.861110 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.854482, event: header_read, request: pg_info(1 pgs e194471:22.39d) v4 -3005> 2014-08-13 17:22:49.861121 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.854484, event: throttled, request: pg_info(1 pgs e194471:22.39d) v4 -3004> 2014-08-13 17:22:49.861112 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray -3003> 2014-08-13 17:22:49.861133 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.854568, event: all_read, request: pg_info(1 pgs e194471:22.39d) v4 -3002> 2014-08-13 17:22:49.861141 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.861108, event: dispatched, request: pg_info(1 pgs e194471:22.39d) v4 -3001> 2014-08-13 17:22:49.861151 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.861151, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.39d) v4 -3000> 2014-08-13 17:22:49.861162 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.861162, event: started, request: pg_info(1 pgs e194471:22.39d) v4 -2999> 2014-08-13 17:22:49.861186 7f74d8735700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:22:49.861186, event: done, request: pg_info(1 pgs e194471:22.39d) v4 -2998> 2014-08-13 17:22:49.861212 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 14 ==== pg_notify(24.3d(24) epoch 194471) v5 ==== 2096+0+0 (486193461 0 0) 0x53b81c0 con 0x475ee00 -2997> 2014-08-13 17:22:49.861232 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.856026, event: header_read, request: pg_notify(24.3d(24) epoch 194471) v5 -2996> 2014-08-13 17:22:49.861242 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.856028, event: throttled, request: pg_notify(24.3d(24) epoch 194471) v5 -2995> 2014-08-13 17:22:49.861252 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.856139, event: all_read, request: pg_notify(24.3d(24) epoch 194471) v5 -2994> 2014-08-13 17:22:49.861261 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.861229, event: dispatched, request: pg_notify(24.3d(24) epoch 194471) v5 -2993> 2014-08-13 17:22:49.861271 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.861271, event: waiting_for_osdmap, request: pg_notify(24.3d(24) epoch 194471) v5 -2992> 2014-08-13 17:22:49.861282 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.861282, event: started, request: pg_notify(24.3d(24) epoch 194471) v5 -2991> 2014-08-13 17:22:49.861307 7f74d8735700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:22:49.861307, event: done, request: pg_notify(24.3d(24) epoch 194471) v5 -2990> 2014-08-13 17:22:49.861292 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary/Peering/WaitActingChange 0.901441 2 0.000225 -2989> 2014-08-13 17:22:49.861321 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.901491 0 0.000000 -2988> 2014-08-13 17:22:49.861348 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 11 ==== pg_query(24.5e epoch 194471) v3 ==== 144+0+0 (2063480002 0 0) 0x4eb1fe0 con 0x47a27e0 -2987> 2014-08-13 17:22:49.861345 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started 0.916147 0 0.000000 -2986> 2014-08-13 17:22:49.861371 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.845863, event: header_read, request: pg_query(24.5e epoch 194471) v3 -2985> 2014-08-13 17:22:49.861380 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.845864, event: throttled, request: pg_query(24.5e epoch 194471) v3 -2984> 2014-08-13 17:22:49.861390 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.845923, event: all_read, request: pg_query(24.5e epoch 194471) v3 -2983> 2014-08-13 17:22:49.861399 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.861369, event: dispatched, request: pg_query(24.5e epoch 194471) v3 -2982> 2014-08-13 17:22:49.861408 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.861408, event: waiting_for_osdmap, request: pg_query(24.5e epoch 194471) v3 -2981> 2014-08-13 17:22:49.861419 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.861419, event: started, request: pg_query(24.5e epoch 194471) v3 -2980> 2014-08-13 17:22:49.861440 7f74d8735700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:22:49.861440, event: done, request: pg_query(24.5e epoch 194471) v3 -2979> 2014-08-13 17:22:49.861388 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194470/194470) [4,348,331] r=0 lpr=194470 pi=193644-194469/22 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset -2978> 2014-08-13 17:22:49.861456 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 9 ==== pg_notify(24.b6(28) epoch 194471) v5 ==== 2332+0+0 (380497260 0 0) 0x4ac6c80 con 0x47a53e0 -2977> 2014-08-13 17:22:49.861475 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.845301, event: header_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2976> 2014-08-13 17:22:49.861485 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.845303, event: throttled, request: pg_notify(24.b6(28) epoch 194471) v5 -2975> 2014-08-13 17:22:49.861494 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.845454, event: all_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2974> 2014-08-13 17:22:49.861503 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.861473, event: dispatched, request: pg_notify(24.b6(28) epoch 194471) v5 -2973> 2014-08-13 17:22:49.861513 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.861512, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194471) v5 -2972> 2014-08-13 17:22:49.861524 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.861523, event: started, request: pg_notify(24.b6(28) epoch 194471) v5 -2971> 2014-08-13 17:22:49.861547 7f74d8735700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:22:49.861546, event: done, request: pg_notify(24.b6(28) epoch 194471) v5 -2970> 2014-08-13 17:22:49.861537 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Reset 0.000149 1 0.000244 -2969> 2014-08-13 17:22:49.861582 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 7 ==== pg_notify(22.1b8(2) epoch 194471) v5 ==== 828+0+0 (4117813703 0 0) 0x52632c0 con 0x47a3f40 -2968> 2014-08-13 17:22:49.861570 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started -2967> 2014-08-13 17:22:49.861601 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.854942, event: header_read, request: pg_notify(22.1b8(2) epoch 194471) v5 -2966> 2014-08-13 17:22:49.861623 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.854944, event: throttled, request: pg_notify(22.1b8(2) epoch 194471) v5 -2965> 2014-08-13 17:22:49.861635 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.855025, event: all_read, request: pg_notify(22.1b8(2) epoch 194471) v5 -2964> 2014-08-13 17:22:49.861643 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.861599, event: dispatched, request: pg_notify(22.1b8(2) epoch 194471) v5 -2963> 2014-08-13 17:22:49.861649 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.861649, event: waiting_for_osdmap, request: pg_notify(22.1b8(2) epoch 194471) v5 -2962> 2014-08-13 17:22:49.861657 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.861657, event: started, request: pg_notify(22.1b8(2) epoch 194471) v5 -2961> 2014-08-13 17:22:49.861676 7f74d8735700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:22:49.861675, event: done, request: pg_notify(22.1b8(2) epoch 194471) v5 -2960> 2014-08-13 17:22:49.861617 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Start -2959> 2014-08-13 17:22:49.861690 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 10 ==== pg_query(24.369 epoch 194471) v3 ==== 144+0+0 (4033227021 0 0) 0x29c6360 con 0x47a32e0 -2958> 2014-08-13 17:22:49.861705 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.860743, event: header_read, request: pg_query(24.369 epoch 194471) v3 -2957> 2014-08-13 17:22:49.861696 7f74d0f29700 1 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: transitioning to Primary -2956> 2014-08-13 17:22:49.861712 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.860745, event: throttled, request: pg_query(24.369 epoch 194471) v3 -2955> 2014-08-13 17:22:49.861723 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.860801, event: all_read, request: pg_query(24.369 epoch 194471) v3 -2954> 2014-08-13 17:22:49.861728 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.861703, event: dispatched, request: pg_query(24.369 epoch 194471) v3 -2953> 2014-08-13 17:22:49.861717 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Start 0.000100 0 0.000000 -2952> 2014-08-13 17:22:49.861733 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.861733, event: waiting_for_osdmap, request: pg_query(24.369 epoch 194471) v3 -2951> 2014-08-13 17:22:49.861741 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.861740, event: started, request: pg_query(24.369 epoch 194471) v3 -2950> 2014-08-13 17:22:49.861738 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary -2949> 2014-08-13 17:22:49.861757 7f74d8735700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:22:49.861757, event: done, request: pg_query(24.369 epoch 194471) v3 -2948> 2014-08-13 17:22:49.861757 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Peering -2947> 2014-08-13 17:22:49.861770 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 9 ==== pg_query(24.33a epoch 194471) v3 ==== 144+0+0 (2777655923 0 0) 0x29c6ea0 con 0x47f0c60 -2946> 2014-08-13 17:22:49.861787 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.855354, event: header_read, request: pg_query(24.33a epoch 194471) v3 -2945> 2014-08-13 17:22:49.861780 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetInfo -2944> 2014-08-13 17:22:49.861797 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.855355, event: throttled, request: pg_query(24.33a epoch 194471) v3 -2943> 2014-08-13 17:22:49.861803 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.855426, event: all_read, request: pg_query(24.33a epoch 194471) v3 -2942> 2014-08-13 17:22:49.861811 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.861785, event: dispatched, request: pg_query(24.33a epoch 194471) v3 -2941> 2014-08-13 17:22:49.861820 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.861820, event: waiting_for_osdmap, request: pg_query(24.33a epoch 194471) v3 -2940> 2014-08-13 17:22:49.861831 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.861831, event: started, request: pg_query(24.33a epoch 194471) v3 -2939> 2014-08-13 17:22:49.861851 7f74d8735700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:22:49.861851, event: done, request: pg_query(24.33a epoch 194471) v3 -2938> 2014-08-13 17:22:49.861869 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 5 ==== pg_notify(24.b6(28) epoch 194471) v5 ==== 2332+0+0 (3903713426 0 0) 0x466ee40 con 0x47f6880 -2937> 2014-08-13 17:22:49.861887 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.856714, event: header_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2936> 2014-08-13 17:22:49.861897 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.856715, event: throttled, request: pg_notify(24.b6(28) epoch 194471) v5 -2935> 2014-08-13 17:22:49.861906 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.856815, event: all_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2934> 2014-08-13 17:22:49.861915 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.861885, event: dispatched, request: pg_notify(24.b6(28) epoch 194471) v5 -2933> 2014-08-13 17:22:49.861925 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.861925, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194471) v5 -2932> 2014-08-13 17:22:49.861936 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.861936, event: started, request: pg_notify(24.b6(28) epoch 194471) v5 -2931> 2014-08-13 17:22:49.861942 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(26) epoch 194471) v5 -- ?+0 0x53bb100 con 0x48d9ce0 -2930> 2014-08-13 17:22:49.861959 7f74d8735700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:22:49.861958, event: done, request: pg_notify(24.b6(28) epoch 194471) v5 -2929> 2014-08-13 17:22:49.861976 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(19) epoch 194471) v5 -- ?+0 0x53bec80 con 0x475a940 -2928> 2014-08-13 17:22:49.861986 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 5 ==== pg_query(24.cd epoch 194471) v3 ==== 144+0+0 (984478667 0 0) 0x29c7da0 con 0x4835ac0 -2927> 2014-08-13 17:22:49.861998 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(34) epoch 194471) v5 -- ?+0 0x53be580 con 0x47f4620 -2926> 2014-08-13 17:22:49.862006 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.861054, event: header_read, request: pg_query(24.cd epoch 194471) v3 -2925> 2014-08-13 17:22:49.862013 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.861057, event: throttled, request: pg_query(24.cd epoch 194471) v3 -2924> 2014-08-13 17:22:49.862024 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.861139, event: all_read, request: pg_query(24.cd epoch 194471) v3 -2923> 2014-08-13 17:22:49.862004 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] exit Started/Stray 0.026155 1 0.000138 -2922> 2014-08-13 17:22:49.862038 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.862003, event: dispatched, request: pg_query(24.cd epoch 194471) v3 -2921> 2014-08-13 17:22:49.862038 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] enter Started/ReplicaActive -2920> 2014-08-13 17:22:49.862048 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.862048, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194471) v3 -2919> 2014-08-13 17:22:49.862021 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(24) epoch 194471) v5 -- ?+0 0x53bc0c0 con 0x475cd00 -2918> 2014-08-13 17:22:49.862069 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -2917> 2014-08-13 17:22:49.862074 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.862073, event: started, request: pg_query(24.cd epoch 194471) v3 -2916> 2014-08-13 17:22:49.862096 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(19) epoch 194471) v5 -- ?+0 0x53bd940 con 0x4868160 -2915> 2014-08-13 17:22:49.862110 7f74d8735700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:22:49.862110, event: done, request: pg_query(24.cd epoch 194471) v3 -2914> 2014-08-13 17:22:49.862149 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 10 ==== pg_info(1 pgs e194471:22.3d5) v4 ==== 714+0+0 (3798181664 0 0) 0x57d3640 con 0x48377a0 -2913> 2014-08-13 17:22:49.862141 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(19) epoch 194471) v5 -- ?+0 0x53be040 con 0x486a680 -2912> 2014-08-13 17:22:49.862170 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.861886, event: header_read, request: pg_info(1 pgs e194471:22.3d5) v4 -2911> 2014-08-13 17:22:49.862180 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.861888, event: throttled, request: pg_info(1 pgs e194471:22.3d5) v4 -2910> 2014-08-13 17:22:49.862187 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(19) epoch 194471) v5 -- ?+0 0x53b8e00 con 0x4833440 -2909> 2014-08-13 17:22:49.862189 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.861950, event: all_read, request: pg_info(1 pgs e194471:22.3d5) v4 -2908> 2014-08-13 17:22:49.862202 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.862168, event: dispatched, request: pg_info(1 pgs e194471:22.3d5) v4 -2907> 2014-08-13 17:22:49.862209 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.862209, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3d5) v4 -2906> 2014-08-13 17:22:49.862210 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(24) epoch 194471) v5 -- ?+0 0x53b9180 con 0x48377a0 -2905> 2014-08-13 17:22:49.862220 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.862219, event: started, request: pg_info(1 pgs e194471:22.3d5) v4 -2904> 2014-08-13 17:22:49.862245 7f74d8735700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:22:49.862245, event: done, request: pg_info(1 pgs e194471:22.3d5) v4 -2903> 2014-08-13 17:22:49.862231 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(24),24.3b7(29) epoch 194471) v5 -- ?+0 0x53babc0 con 0x45f5960 -2902> 2014-08-13 17:22:49.862264 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 7 ==== pg_query(24.2ec epoch 194471) v3 ==== 144+0+0 (3639655561 0 0) 0x29c10e0 con 0x486b9c0 -2901> 2014-08-13 17:22:49.862279 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.860959, event: header_read, request: pg_query(24.2ec epoch 194471) v3 -2900> 2014-08-13 17:22:49.862275 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(22) epoch 194471) v5 -- ?+0 0x49e2300 con 0x4832c00 -2899> 2014-08-13 17:22:49.862285 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.860961, event: throttled, request: pg_query(24.2ec epoch 194471) v3 -2898> 2014-08-13 17:22:49.862290 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.861013, event: all_read, request: pg_query(24.2ec epoch 194471) v3 -2897> 2014-08-13 17:22:49.862295 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.862277, event: dispatched, request: pg_query(24.2ec epoch 194471) v3 -2896> 2014-08-13 17:22:49.862294 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(19) epoch 194471) v5 -- ?+0 0x49e0540 con 0x48dfbc0 -2895> 2014-08-13 17:22:49.862301 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.862301, event: waiting_for_osdmap, request: pg_query(24.2ec epoch 194471) v3 -2894> 2014-08-13 17:22:49.862309 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.862308, event: started, request: pg_query(24.2ec epoch 194471) v3 -2893> 2014-08-13 17:22:49.862326 7f74d8735700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:22:49.862326, event: done, request: pg_query(24.2ec epoch 194471) v3 -2892> 2014-08-13 17:22:49.862339 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 13 ==== pg_notify(24.3d(24) epoch 194471) v5 ==== 2096+0+0 (486193461 0 0) 0x53b9500 con 0x486a3c0 -2891> 2014-08-13 17:22:49.862354 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.856422, event: header_read, request: pg_notify(24.3d(24) epoch 194471) v5 -2890> 2014-08-13 17:22:49.862360 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.856424, event: throttled, request: pg_notify(24.3d(24) epoch 194471) v5 -2889> 2014-08-13 17:22:49.862365 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.856529, event: all_read, request: pg_notify(24.3d(24) epoch 194471) v5 -2888> 2014-08-13 17:22:49.862370 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.862352, event: dispatched, request: pg_notify(24.3d(24) epoch 194471) v5 -2887> 2014-08-13 17:22:49.862376 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.862376, event: waiting_for_osdmap, request: pg_notify(24.3d(24) epoch 194471) v5 -2886> 2014-08-13 17:22:49.862383 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.862383, event: started, request: pg_notify(24.3d(24) epoch 194471) v5 -2885> 2014-08-13 17:22:49.862402 7f74d8735700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:22:49.862401, event: done, request: pg_notify(24.3d(24) epoch 194471) v5 -2884> 2014-08-13 17:22:49.862418 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 7 ==== pg_notify(22.3d5(20) epoch 194471) v5 ==== 1838+0+0 (3449049291 0 0) 0x49e2d80 con 0x48dfe80 -2883> 2014-08-13 17:22:49.862435 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.851499, event: header_read, request: pg_notify(22.3d5(20) epoch 194471) v5 -2882> 2014-08-13 17:22:49.862441 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.851501, event: throttled, request: pg_notify(22.3d5(20) epoch 194471) v5 -2881> 2014-08-13 17:22:49.862485 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.851604, event: all_read, request: pg_notify(22.3d5(20) epoch 194471) v5 -2880> 2014-08-13 17:22:49.862494 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.862433, event: dispatched, request: pg_notify(22.3d5(20) epoch 194471) v5 -2879> 2014-08-13 17:22:49.862504 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.862504, event: waiting_for_osdmap, request: pg_notify(22.3d5(20) epoch 194471) v5 -2878> 2014-08-13 17:22:49.862516 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.862515, event: started, request: pg_notify(22.3d5(20) epoch 194471) v5 -2877> 2014-08-13 17:22:49.862539 7f74d8735700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:22:49.862538, event: done, request: pg_notify(22.3d5(20) epoch 194471) v5 -2876> 2014-08-13 17:22:49.862533 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6842/9607 -- pg_query(24.136 epoch 194471) v3 -- ?+0 0x4eb1fe0 con 0x47f70c0 -2875> 2014-08-13 17:22:49.862565 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6834/6966 -- pg_query(24.136,24.1f7 epoch 194471) v3 -- ?+0 0x4eb5460 con 0x47f7e80 -2874> 2014-08-13 17:22:49.862582 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x480fbc0 con 0x4904a40 -2873> 2014-08-13 17:22:49.862593 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6822/5265 -- pg_query(24.1f7 epoch 194471) v3 -- ?+0 0x4eb41a0 con 0x54e4360 -2872> 2014-08-13 17:22:49.862607 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2871> 2014-08-13 17:22:49.862611 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6840/16566 -- pg_query(24.136 epoch 194471) v3 -- ?+0 0x4eb1e00 con 0x47a0b00 -2870> 2014-08-13 17:22:49.862632 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.111:6842/17345 -- pg_query(24.136 epoch 194471) v3 -- ?+0 0x29c1fe0 con 0x4d51a20 -2869> 2014-08-13 17:22:49.862629 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.312 192.168.1.113:6816/17521 6 ==== pg_info(1 pgs e194471:22.f3) v4 ==== 714+0+0 (1721699773 0 0) 0x57d7c40 con 0x4906880 -2868> 2014-08-13 17:22:49.862658 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.857999, event: header_read, request: pg_info(1 pgs e194471:22.f3) v4 -2867> 2014-08-13 17:22:49.862670 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.858001, event: throttled, request: pg_info(1 pgs e194471:22.f3) v4 -2866> 2014-08-13 17:22:49.862671 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6806/31286 -- pg_query(24.181 epoch 194471) v3 -- ?+0 0x29c0b40 con 0x486fe80 -2865> 2014-08-13 17:22:49.862679 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.858071, event: all_read, request: pg_info(1 pgs e194471:22.f3) v4 -2864> 2014-08-13 17:22:49.862688 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.862655, event: dispatched, request: pg_info(1 pgs e194471:22.f3) v4 -2863> 2014-08-13 17:22:49.862698 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.862698, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.f3) v4 -2862> 2014-08-13 17:22:49.862709 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.862709, event: started, request: pg_info(1 pgs e194471:22.f3) v4 -2861> 2014-08-13 17:22:49.862733 7f74d8735700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:22:49.862732, event: done, request: pg_info(1 pgs e194471:22.f3) v4 -2860> 2014-08-13 17:22:49.862691 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6804/31016 -- pg_query(24.181 epoch 194471) v3 -- ?+0 0x29c6180 con 0x5419fa0 -2859> 2014-08-13 17:22:49.862750 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.47 192.168.1.111:6828/12102 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de1d40 con 0x5490580 -2858> 2014-08-13 17:22:49.862785 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6820/10050 -- pg_query(24.181 epoch 194471) v3 -- ?+0 0x29c30c0 con 0x486bc80 -2857> 2014-08-13 17:22:49.862790 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2856> 2014-08-13 17:22:49.862812 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6816/837 -- pg_query(24.1bb epoch 194471) v3 -- ?+0 0x29c6ea0 con 0x486bf40 -2855> 2014-08-13 17:22:49.862823 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 9 ==== pg_notify(24.30b(25) epoch 194471) v5 ==== 2155+0+0 (1498764271 0 0) 0x57d5cc0 con 0x45f1760 -2854> 2014-08-13 17:22:49.862839 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6806/42024 -- pg_query(24.1bb epoch 194471) v3 -- ?+0 0x29c6360 con 0x48dcd00 -2853> 2014-08-13 17:22:49.862845 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.857565, event: header_read, request: pg_notify(24.30b(25) epoch 194471) v5 -2852> 2014-08-13 17:22:49.862853 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.857568, event: throttled, request: pg_notify(24.30b(25) epoch 194471) v5 -2851> 2014-08-13 17:22:49.862862 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.857698, event: all_read, request: pg_notify(24.30b(25) epoch 194471) v5 -2850> 2014-08-13 17:22:49.862860 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_query(24.1bb epoch 194471) v3 -- ?+0 0x29c21c0 con 0x4833440 -2849> 2014-08-13 17:22:49.862875 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.862842, event: dispatched, request: pg_notify(24.30b(25) epoch 194471) v5 -2848> 2014-08-13 17:22:49.862884 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.862884, event: waiting_for_osdmap, request: pg_notify(24.30b(25) epoch 194471) v5 -2847> 2014-08-13 17:22:49.862895 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.862895, event: started, request: pg_notify(24.30b(25) epoch 194471) v5 -2846> 2014-08-13 17:22:49.862889 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6812/6757 -- pg_query(24.181 epoch 194471) v3 -- ?+0 0x29c05a0 con 0x45f6300 -2845> 2014-08-13 17:22:49.862924 7f74d8735700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:22:49.862924, event: done, request: pg_notify(24.30b(25) epoch 194471) v5 -2844> 2014-08-13 17:22:49.862959 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 6 ==== pg_notify(24.b6(28) epoch 194471) v5 ==== 2332+0+0 (3903713426 0 0) 0x57d7000 con 0x47f6880 -2843> 2014-08-13 17:22:49.862979 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.856826, event: header_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2842> 2014-08-13 17:22:49.862989 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.856827, event: throttled, request: pg_notify(24.b6(28) epoch 194471) v5 -2841> 2014-08-13 17:22:49.863002 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.856921, event: all_read, request: pg_notify(24.b6(28) epoch 194471) v5 -2840> 2014-08-13 17:22:49.863011 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.862977, event: dispatched, request: pg_notify(24.b6(28) epoch 194471) v5 -2839> 2014-08-13 17:22:49.862967 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6800/41204 -- pg_query(24.1bb epoch 194471) v3 -- ?+0 0x29c7620 con 0x54e1600 -2838> 2014-08-13 17:22:49.863023 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.863023, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194471) v5 -2837> 2014-08-13 17:22:49.863032 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.863031, event: started, request: pg_notify(24.b6(28) epoch 194471) v5 -2836> 2014-08-13 17:22:49.863055 7f74d8735700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:22:49.863055, event: done, request: pg_notify(24.b6(28) epoch 194471) v5 -2835> 2014-08-13 17:22:49.863043 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6830/16444 -- pg_query(24.1f7 epoch 194471) v3 -- ?+0 0x29c6ae0 con 0x4868420 -2834> 2014-08-13 17:22:49.863075 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_query(24.136 epoch 194471) v3 -- ?+0 0x29c0f00 con 0x45f5960 -2833> 2014-08-13 17:22:49.863077 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 15 ==== pg_info(1 pgs e194471:22.3d9) v4 ==== 714+0+0 (2290861376 0 0) 0x57d4980 con 0x4835280 -2832> 2014-08-13 17:22:49.863099 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.862306, event: header_read, request: pg_info(1 pgs e194471:22.3d9) v4 -2831> 2014-08-13 17:22:49.863097 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6838/6893 -- pg_query(24.1f7 epoch 194471) v3 -- ?+0 0x4eb6720 con 0x4835280 -2830> 2014-08-13 17:22:49.863112 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.862308, event: throttled, request: pg_info(1 pgs e194471:22.3d9) v4 -2829> 2014-08-13 17:22:49.863120 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.862417, event: all_read, request: pg_info(1 pgs e194471:22.3d9) v4 -2828> 2014-08-13 17:22:49.863128 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.863097, event: dispatched, request: pg_info(1 pgs e194471:22.3d9) v4 -2827> 2014-08-13 17:22:49.863138 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.863138, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3d9) v4 -2826> 2014-08-13 17:22:49.863149 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.863149, event: started, request: pg_info(1 pgs e194471:22.3d9) v4 -2825> 2014-08-13 17:22:49.863172 7f74d8735700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:22:49.863172, event: done, request: pg_info(1 pgs e194471:22.3d9) v4 -2824> 2014-08-13 17:22:49.863189 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.338 192.168.1.106:6814/25893 6 ==== pg_info(1 pgs e194471:22.3d9) v4 ==== 714+0+0 (2167147397 0 0) 0x57d7540 con 0x4834620 -2823> 2014-08-13 17:22:49.863216 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.862478, event: header_read, request: pg_info(1 pgs e194471:22.3d9) v4 -2822> 2014-08-13 17:22:49.863127 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6830/4861 -- pg_query(24.1bb epoch 194471) v3 -- ?+0 0x4eb6540 con 0x486b440 -2821> 2014-08-13 17:22:49.863229 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.862480, event: throttled, request: pg_info(1 pgs e194471:22.3d9) v4 -2820> 2014-08-13 17:22:49.863237 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.862547, event: all_read, request: pg_info(1 pgs e194471:22.3d9) v4 -2819> 2014-08-13 17:22:49.863246 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.863213, event: dispatched, request: pg_info(1 pgs e194471:22.3d9) v4 -2818> 2014-08-13 17:22:49.863256 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.863256, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3d9) v4 -2817> 2014-08-13 17:22:49.863267 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.863267, event: started, request: pg_info(1 pgs e194471:22.3d9) v4 -2816> 2014-08-13 17:22:49.863289 7f74d8735700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:22:49.863289, event: done, request: pg_info(1 pgs e194471:22.3d9) v4 -2815> 2014-08-13 17:22:49.863251 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6842/9607 -- pg_info(1 pgs e194471:22.6bd(3)) v4 -- ?+0 0x49e16c0 con 0x47f70c0 -2814> 2014-08-13 17:22:49.863306 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 8 ==== pg_query(24.20e epoch 194471) v3 ==== 144+0+0 (1681308323 0 0) 0x4eb3a20 con 0x48d9ce0 -2813> 2014-08-13 17:22:49.863325 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863023, event: header_read, request: pg_query(24.20e epoch 194471) v3 -2812> 2014-08-13 17:22:49.863335 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863025, event: throttled, request: pg_query(24.20e epoch 194471) v3 -2811> 2014-08-13 17:22:49.863344 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863079, event: all_read, request: pg_query(24.20e epoch 194471) v3 -2810> 2014-08-13 17:22:49.863352 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863323, event: dispatched, request: pg_query(24.20e epoch 194471) v3 -2809> 2014-08-13 17:22:49.863361 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863361, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194471) v3 -2808> 2014-08-13 17:22:49.863362 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6814/33435 -- pg_info(1 pgs e194471:22.481(40)) v4 -- ?+0 0x49e0e00 con 0x48dad60 -2807> 2014-08-13 17:22:49.863372 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863372, event: started, request: pg_query(24.20e epoch 194471) v3 -2806> 2014-08-13 17:22:49.863405 7f74d8735700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:22:49.863404, event: done, request: pg_query(24.20e epoch 194471) v3 -2805> 2014-08-13 17:22:49.863415 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_info(2 pgs e194471:22.6bd(3),22.48e(21)) v4 -- ?+0 0x49e0000 con 0x47a27e0 -2804> 2014-08-13 17:22:49.863422 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 8 ==== pg_info(1 pgs e194471:22.3d5) v4 ==== 714+0+0 (1553610224 0 0) 0x57d3480 con 0x48dfe80 -2803> 2014-08-13 17:22:49.863437 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.861678, event: header_read, request: pg_info(1 pgs e194471:22.3d5) v4 -2802> 2014-08-13 17:22:49.863447 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.861679, event: throttled, request: pg_info(1 pgs e194471:22.3d5) v4 -2801> 2014-08-13 17:22:49.863456 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.861739, event: all_read, request: pg_info(1 pgs e194471:22.3d5) v4 -2800> 2014-08-13 17:22:49.863465 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.863435, event: dispatched, request: pg_info(1 pgs e194471:22.3d5) v4 -2799> 2014-08-13 17:22:49.863474 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.863474, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3d5) v4 -2798> 2014-08-13 17:22:49.863485 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.863485, event: started, request: pg_info(1 pgs e194471:22.3d5) v4 -2797> 2014-08-13 17:22:49.863507 7f74d8735700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:22:49.863507, event: done, request: pg_info(1 pgs e194471:22.3d5) v4 -2796> 2014-08-13 17:22:49.863489 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6814/23703 -- pg_info(1 pgs e194471:22.481(40)) v4 -- ?+0 0x49e1dc0 con 0x4903440 -2795> 2014-08-13 17:22:49.863528 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 4 ==== pg_notify(24.3d epoch 194471) v5 ==== 694+0+0 (3414476826 0 0) 0x52639c0 con 0x4904a40 -2794> 2014-08-13 17:22:49.863719 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.861525, event: header_read, request: pg_notify(24.3d epoch 194471) v5 -2793> 2014-08-13 17:22:49.863733 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.861526, event: throttled, request: pg_notify(24.3d epoch 194471) v5 -2792> 2014-08-13 17:22:49.863742 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.861582, event: all_read, request: pg_notify(24.3d epoch 194471) v5 -2791> 2014-08-13 17:22:49.863750 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.863715, event: dispatched, request: pg_notify(24.3d epoch 194471) v5 -2790> 2014-08-13 17:22:49.863774 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.863774, event: waiting_for_osdmap, request: pg_notify(24.3d epoch 194471) v5 -2789> 2014-08-13 17:22:49.863786 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.863786, event: started, request: pg_notify(24.3d epoch 194471) v5 -2788> 2014-08-13 17:22:49.863811 7f74d8735700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:22:49.863810, event: done, request: pg_notify(24.3d epoch 194471) v5 -2787> 2014-08-13 17:22:49.863832 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.47 192.168.1.111:6828/12102 4 ==== pg_info(1 pgs e194471:22.1f1) v4 ==== 714+0+0 (4000969918 0 0) 0x5264440 con 0x5490580 -2786> 2014-08-13 17:22:49.863851 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.861822, event: header_read, request: pg_info(1 pgs e194471:22.1f1) v4 -2785> 2014-08-13 17:22:49.863858 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.861823, event: throttled, request: pg_info(1 pgs e194471:22.1f1) v4 -2784> 2014-08-13 17:22:49.863863 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.861902, event: all_read, request: pg_info(1 pgs e194471:22.1f1) v4 -2783> 2014-08-13 17:22:49.863868 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.863848, event: dispatched, request: pg_info(1 pgs e194471:22.1f1) v4 -2782> 2014-08-13 17:22:49.863877 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.863877, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1f1) v4 -2781> 2014-08-13 17:22:49.863888 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.863888, event: started, request: pg_info(1 pgs e194471:22.1f1) v4 -2780> 2014-08-13 17:22:49.863912 7f74d8735700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:22:49.863912, event: done, request: pg_info(1 pgs e194471:22.1f1) v4 -2779> 2014-08-13 17:22:49.863850 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.920513 7 0.000297 -2778> 2014-08-13 17:22:49.863928 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 11 ==== pg_query(24.350 epoch 194471) v3 ==== 144+0+0 (1407793204 0 0) 0x4eb0780 con 0x45f5960 -2777> 2014-08-13 17:22:49.863926 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2776> 2014-08-13 17:22:49.863942 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863341, event: header_read, request: pg_query(24.350 epoch 194471) v3 -2775> 2014-08-13 17:22:49.863948 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863343, event: throttled, request: pg_query(24.350 epoch 194471) v3 -2774> 2014-08-13 17:22:49.863953 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863398, event: all_read, request: pg_query(24.350 epoch 194471) v3 -2773> 2014-08-13 17:22:49.863959 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863940, event: dispatched, request: pg_query(24.350 epoch 194471) v3 -2772> 2014-08-13 17:22:49.863953 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192972) [292,345,4] r=2 lpr=194470 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2771> 2014-08-13 17:22:49.863964 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863964, event: waiting_for_osdmap, request: pg_query(24.350 epoch 194471) v3 -2770> 2014-08-13 17:22:49.863973 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863973, event: started, request: pg_query(24.350 epoch 194471) v3 -2769> 2014-08-13 17:22:49.863989 7f74d8735700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:22:49.863989, event: done, request: pg_query(24.350 epoch 194471) v3 -2768> 2014-08-13 17:22:49.864002 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.344 192.168.1.106:6812/6757 5 ==== pg_notify(24.181 epoch 194471) v5 ==== 694+0+0 (3954779530 0 0) 0x49e7540 con 0x45f6300 -2767> 2014-08-13 17:22:49.864015 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.863500, event: header_read, request: pg_notify(24.181 epoch 194471) v5 -2766> 2014-08-13 17:22:49.864023 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.863502, event: throttled, request: pg_notify(24.181 epoch 194471) v5 -2765> 2014-08-13 17:22:49.864028 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.863598, event: all_read, request: pg_notify(24.181 epoch 194471) v5 -2764> 2014-08-13 17:22:49.864033 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.864013, event: dispatched, request: pg_notify(24.181 epoch 194471) v5 -2763> 2014-08-13 17:22:49.864038 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.864038, event: waiting_for_osdmap, request: pg_notify(24.181 epoch 194471) v5 -2762> 2014-08-13 17:22:49.864045 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.864044, event: started, request: pg_notify(24.181 epoch 194471) v5 -2761> 2014-08-13 17:22:49.864061 7f74d8735700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:22:49.864061, event: done, request: pg_notify(24.181 epoch 194471) v5 -2760> 2014-08-13 17:22:49.864074 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 6 ==== pg_query(24.1e6 epoch 194471) v3 ==== 144+0+0 (537369750 0 0) 0x4eb6ae0 con 0x475cd00 -2759> 2014-08-13 17:22:49.864092 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.863194, event: header_read, request: pg_query(24.1e6 epoch 194471) v3 -2758> 2014-08-13 17:22:49.864101 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.863195, event: throttled, request: pg_query(24.1e6 epoch 194471) v3 -2757> 2014-08-13 17:22:49.864110 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.863254, event: all_read, request: pg_query(24.1e6 epoch 194471) v3 -2756> 2014-08-13 17:22:49.864119 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.864090, event: dispatched, request: pg_query(24.1e6 epoch 194471) v3 -2755> 2014-08-13 17:22:49.864128 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.864128, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194471) v3 -2754> 2014-08-13 17:22:49.864139 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.864139, event: started, request: pg_query(24.1e6 epoch 194471) v3 -2753> 2014-08-13 17:22:49.864204 7f74d8735700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:22:49.864203, event: done, request: pg_query(24.1e6 epoch 194471) v3 -2752> 2014-08-13 17:22:49.864226 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 11 ==== pg_notify(24.136 epoch 194471) v5 ==== 694+0+0 (1970161812 0 0) 0x53bc440 con 0x47a0b00 -2751> 2014-08-13 17:22:49.864249 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.863763, event: header_read, request: pg_notify(24.136 epoch 194471) v5 -2750> 2014-08-13 17:22:49.864256 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.863765, event: throttled, request: pg_notify(24.136 epoch 194471) v5 -2749> 2014-08-13 17:22:49.864261 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.863831, event: all_read, request: pg_notify(24.136 epoch 194471) v5 -2748> 2014-08-13 17:22:49.864265 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.864247, event: dispatched, request: pg_notify(24.136 epoch 194471) v5 -2747> 2014-08-13 17:22:49.864271 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.864271, event: waiting_for_osdmap, request: pg_notify(24.136 epoch 194471) v5 -2746> 2014-08-13 17:22:49.864278 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.864278, event: started, request: pg_notify(24.136 epoch 194471) v5 -2745> 2014-08-13 17:22:49.864295 7f74d8735700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:22:49.864295, event: done, request: pg_notify(24.136 epoch 194471) v5 -2744> 2014-08-13 17:22:49.864308 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.313 192.168.1.109:6820/35445 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c772c0 con 0x47a1b80 -2743> 2014-08-13 17:22:49.864325 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2742> 2014-08-13 17:22:49.864336 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 7 ==== pg_query(24.1e4 epoch 194471) v3 ==== 144+0+0 (3454188327 0 0) 0x4eb4560 con 0x47f4620 -2741> 2014-08-13 17:22:49.864349 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.862750, event: header_read, request: pg_query(24.1e4 epoch 194471) v3 -2740> 2014-08-13 17:22:49.864356 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.862752, event: throttled, request: pg_query(24.1e4 epoch 194471) v3 -2739> 2014-08-13 17:22:49.864361 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.862805, event: all_read, request: pg_query(24.1e4 epoch 194471) v3 -2738> 2014-08-13 17:22:49.864366 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.864347, event: dispatched, request: pg_query(24.1e4 epoch 194471) v3 -2737> 2014-08-13 17:22:49.864371 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.864371, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194471) v3 -2736> 2014-08-13 17:22:49.864378 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.864377, event: started, request: pg_query(24.1e4 epoch 194471) v3 -2735> 2014-08-13 17:22:49.864393 7f74d8735700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:22:49.864393, event: done, request: pg_query(24.1e4 epoch 194471) v3 -2734> 2014-08-13 17:22:49.864405 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 11 ==== pg_notify(24.136 epoch 194471) v5 ==== 694+0+0 (1970161812 0 0) 0x53b8540 con 0x47f7e80 -2733> 2014-08-13 17:22:49.864424 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.863200, event: header_read, request: pg_notify(24.136 epoch 194471) v5 -2732> 2014-08-13 17:22:49.864433 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.863202, event: throttled, request: pg_notify(24.136 epoch 194471) v5 -2731> 2014-08-13 17:22:49.864442 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.863297, event: all_read, request: pg_notify(24.136 epoch 194471) v5 -2730> 2014-08-13 17:22:49.864451 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.864422, event: dispatched, request: pg_notify(24.136 epoch 194471) v5 -2729> 2014-08-13 17:22:49.864461 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.864461, event: waiting_for_osdmap, request: pg_notify(24.136 epoch 194471) v5 -2728> 2014-08-13 17:22:49.864470 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.864470, event: started, request: pg_notify(24.136 epoch 194471) v5 -2727> 2014-08-13 17:22:49.864488 7f74d8735700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:22:49.864487, event: done, request: pg_notify(24.136 epoch 194471) v5 -2726> 2014-08-13 17:22:49.864476 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194465/194457 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.925139 7 0.000272 -2725> 2014-08-13 17:22:49.864505 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 11 ==== pg_notify(24.1bb(26) epoch 194471) v5 ==== 2214+0+0 (1123115219 0 0) 0x53b8e00 con 0x4833440 -2724> 2014-08-13 17:22:49.864524 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.863676, event: header_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -2723> 2014-08-13 17:22:49.864525 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194465/194457 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2722> 2014-08-13 17:22:49.864536 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.863678, event: throttled, request: pg_notify(24.1bb(26) epoch 194471) v5 -2721> 2014-08-13 17:22:49.864548 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.863794, event: all_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -2720> 2014-08-13 17:22:49.864558 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.864522, event: dispatched, request: pg_notify(24.1bb(26) epoch 194471) v5 -2719> 2014-08-13 17:22:49.864567 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.864567, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194471) v5 -2718> 2014-08-13 17:22:49.864548 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194449 n=41 ec=183096 les/c 194465/194457 194470/194470/192978) [323,359,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2717> 2014-08-13 17:22:49.864579 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.864579, event: started, request: pg_notify(24.1bb(26) epoch 194471) v5 -2716> 2014-08-13 17:22:49.864608 7f74d8735700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:22:49.864608, event: done, request: pg_notify(24.1bb(26) epoch 194471) v5 -2715> 2014-08-13 17:22:49.864631 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 11 ==== pg_query(24.206 epoch 194471) v3 ==== 144+0+0 (1539433894 0 0) 0x4eb03c0 con 0x48377a0 -2714> 2014-08-13 17:22:49.864650 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.863532, event: header_read, request: pg_query(24.206 epoch 194471) v3 -2713> 2014-08-13 17:22:49.864660 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.863534, event: throttled, request: pg_query(24.206 epoch 194471) v3 -2712> 2014-08-13 17:22:49.864668 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.863592, event: all_read, request: pg_query(24.206 epoch 194471) v3 -2711> 2014-08-13 17:22:49.864677 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.864648, event: dispatched, request: pg_query(24.206 epoch 194471) v3 -2710> 2014-08-13 17:22:49.864686 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.864686, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194471) v3 -2709> 2014-08-13 17:22:49.864698 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.864697, event: started, request: pg_query(24.206 epoch 194471) v3 -2708> 2014-08-13 17:22:49.864718 7f74d8735700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:22:49.864718, event: done, request: pg_query(24.206 epoch 194471) v3 -2707> 2014-08-13 17:22:49.864731 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 8 ==== pg_notify(24.1f7(28) epoch 194471) v5 ==== 2332+0+0 (1351492979 0 0) 0x57d7a80 con 0x4868420 -2706> 2014-08-13 17:22:49.864746 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.863941, event: header_read, request: pg_notify(24.1f7(28) epoch 194471) v5 -2705> 2014-08-13 17:22:49.864752 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.863942, event: throttled, request: pg_notify(24.1f7(28) epoch 194471) v5 -2704> 2014-08-13 17:22:49.864757 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.864051, event: all_read, request: pg_notify(24.1f7(28) epoch 194471) v5 -2703> 2014-08-13 17:22:49.864762 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.864744, event: dispatched, request: pg_notify(24.1f7(28) epoch 194471) v5 -2702> 2014-08-13 17:22:49.864768 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.864768, event: waiting_for_osdmap, request: pg_notify(24.1f7(28) epoch 194471) v5 -2701> 2014-08-13 17:22:49.864775 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.864775, event: started, request: pg_notify(24.1f7(28) epoch 194471) v5 -2700> 2014-08-13 17:22:49.864793 7f74d8735700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:22:49.864792, event: done, request: pg_notify(24.1f7(28) epoch 194471) v5 -2699> 2014-08-13 17:22:49.864808 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 7 ==== pg_notify(24.1bb epoch 194471) v5 ==== 694+0+0 (1150856441 0 0) 0x53b88c0 con 0x486bf40 -2698> 2014-08-13 17:22:49.864822 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.863598, event: header_read, request: pg_notify(24.1bb epoch 194471) v5 -2697> 2014-08-13 17:22:49.864827 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.863600, event: throttled, request: pg_notify(24.1bb epoch 194471) v5 -2696> 2014-08-13 17:22:49.864832 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.863661, event: all_read, request: pg_notify(24.1bb epoch 194471) v5 -2695> 2014-08-13 17:22:49.864837 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.864820, event: dispatched, request: pg_notify(24.1bb epoch 194471) v5 -2694> 2014-08-13 17:22:49.864842 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.864842, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194471) v5 -2693> 2014-08-13 17:22:49.864849 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.864849, event: started, request: pg_notify(24.1bb epoch 194471) v5 -2692> 2014-08-13 17:22:49.864864 7f74d8735700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:22:49.864864, event: done, request: pg_notify(24.1bb epoch 194471) v5 -2691> 2014-08-13 17:22:49.864876 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 11 ==== pg_notify(24.181(3) epoch 194471) v5 ==== 925+0+0 (2958737920 0 0) 0x53bee40 con 0x486bc80 -2690> 2014-08-13 17:22:49.864890 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.863754, event: header_read, request: pg_notify(24.181(3) epoch 194471) v5 -2689> 2014-08-13 17:22:49.864896 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.863756, event: throttled, request: pg_notify(24.181(3) epoch 194471) v5 -2688> 2014-08-13 17:22:49.864901 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.863841, event: all_read, request: pg_notify(24.181(3) epoch 194471) v5 -2687> 2014-08-13 17:22:49.864906 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.864888, event: dispatched, request: pg_notify(24.181(3) epoch 194471) v5 -2686> 2014-08-13 17:22:49.864912 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.864912, event: waiting_for_osdmap, request: pg_notify(24.181(3) epoch 194471) v5 -2685> 2014-08-13 17:22:49.864918 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.864918, event: started, request: pg_notify(24.181(3) epoch 194471) v5 -2684> 2014-08-13 17:22:49.864936 7f74d8735700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:22:49.864936, event: done, request: pg_notify(24.181(3) epoch 194471) v5 -2683> 2014-08-13 17:22:49.864952 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 6 ==== pg_notify(24.1bb(26) epoch 194471) v5 ==== 2214+0+0 (1123115219 0 0) 0x57d1340 con 0x486b440 -2682> 2014-08-13 17:22:49.864966 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864393, event: header_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -2681> 2014-08-13 17:22:49.864972 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864395, event: throttled, request: pg_notify(24.1bb(26) epoch 194471) v5 -2680> 2014-08-13 17:22:49.864977 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864513, event: all_read, request: pg_notify(24.1bb(26) epoch 194471) v5 -2679> 2014-08-13 17:22:49.864982 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864964, event: dispatched, request: pg_notify(24.1bb(26) epoch 194471) v5 -2678> 2014-08-13 17:22:49.864988 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864988, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194471) v5 -2677> 2014-08-13 17:22:49.864995 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.864994, event: started, request: pg_notify(24.1bb(26) epoch 194471) v5 -2676> 2014-08-13 17:22:49.865014 7f74d8735700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:22:49.865013, event: done, request: pg_notify(24.1bb(26) epoch 194471) v5 -2675> 2014-08-13 17:22:49.865036 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e34a40 con 0x48db9c0 -2674> 2014-08-13 17:22:49.865025 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194465/194457 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.946955 7 0.000266 -2673> 2014-08-13 17:22:49.865052 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2672> 2014-08-13 17:22:49.865056 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194465/194457 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2671> 2014-08-13 17:22:49.865064 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 7 ==== pg_notify(24.136(22) epoch 194471) v5 ==== 1978+0+0 (2287980691 0 0) 0x53bbf00 con 0x4d51a20 -2670> 2014-08-13 17:22:49.865078 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.864411, event: header_read, request: pg_notify(24.136(22) epoch 194471) v5 -2669> 2014-08-13 17:22:49.865085 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.864413, event: throttled, request: pg_notify(24.136(22) epoch 194471) v5 -2668> 2014-08-13 17:22:49.865090 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.864518, event: all_read, request: pg_notify(24.136(22) epoch 194471) v5 -2667> 2014-08-13 17:22:49.865095 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.865076, event: dispatched, request: pg_notify(24.136(22) epoch 194471) v5 -2666> 2014-08-13 17:22:49.865101 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.865101, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194471) v5 -2665> 2014-08-13 17:22:49.865109 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.865109, event: started, request: pg_notify(24.136(22) epoch 194471) v5 -2664> 2014-08-13 17:22:49.865128 7f74d8735700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:22:49.865128, event: done, request: pg_notify(24.136(22) epoch 194471) v5 -2663> 2014-08-13 17:22:49.865100 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194449 n=42 ec=183096 les/c 194465/194457 194470/194470/192972) [9,292,4] r=2 lpr=194470 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2662> 2014-08-13 17:22:49.865144 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de2d00 con 0x4d55ac0 -2661> 2014-08-13 17:22:49.865160 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2660> 2014-08-13 17:22:49.865168 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.307 192.168.1.109:6804/31016 3 ==== pg_notify(24.181(31) epoch 194471) v5 ==== 2493+0+0 (1820897233 0 0) 0x49e7a80 con 0x5419fa0 -2659> 2014-08-13 17:22:49.865181 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.864034, event: header_read, request: pg_notify(24.181(31) epoch 194471) v5 -2658> 2014-08-13 17:22:49.865188 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.864035, event: throttled, request: pg_notify(24.181(31) epoch 194471) v5 -2657> 2014-08-13 17:22:49.865201 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.864144, event: all_read, request: pg_notify(24.181(31) epoch 194471) v5 -2656> 2014-08-13 17:22:49.865207 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.865180, event: dispatched, request: pg_notify(24.181(31) epoch 194471) v5 -2655> 2014-08-13 17:22:49.865213 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.865213, event: waiting_for_osdmap, request: pg_notify(24.181(31) epoch 194471) v5 -2654> 2014-08-13 17:22:49.865221 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.865221, event: started, request: pg_notify(24.181(31) epoch 194471) v5 -2653> 2014-08-13 17:22:49.865241 7f74d8735700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:22:49.865241, event: done, request: pg_notify(24.181(31) epoch 194471) v5 -2652> 2014-08-13 17:22:49.865267 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.348 192.168.1.110:6800/41204 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de5a00 con 0x54e1600 -2651> 2014-08-13 17:22:49.865284 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2650> 2014-08-13 17:22:49.865294 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 12 ==== pg_query(24.3b7 epoch 194471) v3 ==== 144+0+0 (4046601719 0 0) 0x29c0f00 con 0x45f5960 -2649> 2014-08-13 17:22:49.865308 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.863487, event: header_read, request: pg_query(24.3b7 epoch 194471) v3 -2648> 2014-08-13 17:22:49.865315 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.863488, event: throttled, request: pg_query(24.3b7 epoch 194471) v3 -2647> 2014-08-13 17:22:49.865326 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.863548, event: all_read, request: pg_query(24.3b7 epoch 194471) v3 -2646> 2014-08-13 17:22:49.865332 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.865305, event: dispatched, request: pg_query(24.3b7 epoch 194471) v3 -2645> 2014-08-13 17:22:49.865337 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.865337, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194471) v3 -2644> 2014-08-13 17:22:49.865345 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.865344, event: started, request: pg_query(24.3b7 epoch 194471) v3 -2643> 2014-08-13 17:22:49.865362 7f74d8735700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:22:49.865362, event: done, request: pg_query(24.3b7 epoch 194471) v3 -2642> 2014-08-13 17:22:49.865375 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4805340 con 0x47591e0 -2641> 2014-08-13 17:22:49.865363 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194465/194457 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.943379 7 0.000252 -2640> 2014-08-13 17:22:49.865391 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2639> 2014-08-13 17:22:49.865399 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.313 192.168.1.109:6820/35445 4 ==== pg_info(1 pgs e194471:22.3e4) v4 ==== 714+0+0 (3273464784 0 0) 0x53bc0c0 con 0x47a1b80 -2638> 2014-08-13 17:22:49.865396 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194465/194457 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2637> 2014-08-13 17:22:49.865412 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.863977, event: header_read, request: pg_info(1 pgs e194471:22.3e4) v4 -2636> 2014-08-13 17:22:49.865420 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.863979, event: throttled, request: pg_info(1 pgs e194471:22.3e4) v4 -2635> 2014-08-13 17:22:49.865430 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.864039, event: all_read, request: pg_info(1 pgs e194471:22.3e4) v4 -2634> 2014-08-13 17:22:49.865445 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.865410, event: dispatched, request: pg_info(1 pgs e194471:22.3e4) v4 -2633> 2014-08-13 17:22:49.865452 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.865452, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3e4) v4 -2632> 2014-08-13 17:22:49.865460 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.865460, event: started, request: pg_info(1 pgs e194471:22.3e4) v4 -2631> 2014-08-13 17:22:49.865450 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194449 n=27 ec=183096 les/c 194465/194457 194470/194470/192936) [54,331,4] r=2 lpr=194470 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2630> 2014-08-13 17:22:49.865480 7f74d8735700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:22:49.865480, event: done, request: pg_info(1 pgs e194471:22.3e4) v4 -2629> 2014-08-13 17:22:49.865494 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 12 ==== pg_notify(24.1f7(2) epoch 194471) v5 ==== 870+0+0 (4122280054 0 0) 0x53b8a80 con 0x47f7e80 -2628> 2014-08-13 17:22:49.865509 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.863326, event: header_read, request: pg_notify(24.1f7(2) epoch 194471) v5 -2627> 2014-08-13 17:22:49.865515 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.863327, event: throttled, request: pg_notify(24.1f7(2) epoch 194471) v5 -2626> 2014-08-13 17:22:49.865520 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.863390, event: all_read, request: pg_notify(24.1f7(2) epoch 194471) v5 -2625> 2014-08-13 17:22:49.865526 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.865507, event: dispatched, request: pg_notify(24.1f7(2) epoch 194471) v5 -2624> 2014-08-13 17:22:49.865531 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.865531, event: waiting_for_osdmap, request: pg_notify(24.1f7(2) epoch 194471) v5 -2623> 2014-08-13 17:22:49.865538 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.865538, event: started, request: pg_notify(24.1f7(2) epoch 194471) v5 -2622> 2014-08-13 17:22:49.865556 7f74d8735700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:22:49.865556, event: done, request: pg_notify(24.1f7(2) epoch 194471) v5 -2621> 2014-08-13 17:22:49.865569 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 16 ==== pg_notify(24.1f7(24) epoch 194471) v5 ==== 2096+0+0 (3771264069 0 0) 0x49e1340 con 0x4835280 -2620> 2014-08-13 17:22:49.865587 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.864462, event: header_read, request: pg_notify(24.1f7(24) epoch 194471) v5 -2619> 2014-08-13 17:22:49.865581 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.939462 7 0.000407 -2618> 2014-08-13 17:22:49.865598 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.864464, event: throttled, request: pg_notify(24.1f7(24) epoch 194471) v5 -2617> 2014-08-13 17:22:49.865621 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.864571, event: all_read, request: pg_notify(24.1f7(24) epoch 194471) v5 -2616> 2014-08-13 17:22:49.865614 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2615> 2014-08-13 17:22:49.865636 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.865585, event: dispatched, request: pg_notify(24.1f7(24) epoch 194471) v5 -2614> 2014-08-13 17:22:49.865644 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.865644, event: waiting_for_osdmap, request: pg_notify(24.1f7(24) epoch 194471) v5 -2613> 2014-08-13 17:22:49.865656 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.865655, event: started, request: pg_notify(24.1f7(24) epoch 194471) v5 -2612> 2014-08-13 17:22:49.865648 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192949) [328,364,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2611> 2014-08-13 17:22:49.865682 7f74d8735700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:22:49.865682, event: done, request: pg_notify(24.1f7(24) epoch 194471) v5 -2610> 2014-08-13 17:22:49.865718 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 7 ==== pg_notify(22.362(22) epoch 194471) v5 ==== 1948+0+0 (411326007 0 0) 0x466fc40 con 0x48db9c0 -2609> 2014-08-13 17:22:49.865739 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.863731, event: header_read, request: pg_notify(22.362(22) epoch 194471) v5 -2608> 2014-08-13 17:22:49.865750 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.863732, event: throttled, request: pg_notify(22.362(22) epoch 194471) v5 -2607> 2014-08-13 17:22:49.865758 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.863841, event: all_read, request: pg_notify(22.362(22) epoch 194471) v5 -2606> 2014-08-13 17:22:49.865767 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.865737, event: dispatched, request: pg_notify(22.362(22) epoch 194471) v5 -2605> 2014-08-13 17:22:49.865776 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.865776, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194471) v5 -2604> 2014-08-13 17:22:49.865787 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.865787, event: started, request: pg_notify(22.362(22) epoch 194471) v5 -2603> 2014-08-13 17:22:49.865787 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(34) epoch 194471) v5 -- ?+0 0x53bcec0 con 0x47f4620 -2602> 2014-08-13 17:22:49.865817 7f74d8735700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:22:49.865816, event: done, request: pg_notify(22.362(22) epoch 194471) v5 -2601> 2014-08-13 17:22:49.865846 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 5 ==== pg_notify(24.9b(27) epoch 194471) v5 ==== 2273+0+0 (2371126972 0 0) 0x53ba840 con 0x4d55ac0 -2600> 2014-08-13 17:22:49.865866 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.864476, event: header_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2599> 2014-08-13 17:22:49.865876 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.864477, event: throttled, request: pg_notify(24.9b(27) epoch 194471) v5 -2598> 2014-08-13 17:22:49.865885 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.864582, event: all_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2597> 2014-08-13 17:22:49.865894 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.865864, event: dispatched, request: pg_notify(24.9b(27) epoch 194471) v5 -2596> 2014-08-13 17:22:49.865904 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.865904, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194471) v5 -2595> 2014-08-13 17:22:49.865915 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.865914, event: started, request: pg_notify(24.9b(27) epoch 194471) v5 -2594> 2014-08-13 17:22:49.865937 7f74d8735700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:22:49.865937, event: done, request: pg_notify(24.9b(27) epoch 194471) v5 -2593> 2014-08-13 17:22:49.865958 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.348 192.168.1.110:6800/41204 4 ==== pg_notify(24.1bb epoch 194471) v5 ==== 694+0+0 (1150856441 0 0) 0x49e24c0 con 0x54e1600 -2592> 2014-08-13 17:22:49.865976 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.864014, event: header_read, request: pg_notify(24.1bb epoch 194471) v5 -2591> 2014-08-13 17:22:49.865986 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.864016, event: throttled, request: pg_notify(24.1bb epoch 194471) v5 -2590> 2014-08-13 17:22:49.865994 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.864074, event: all_read, request: pg_notify(24.1bb epoch 194471) v5 -2589> 2014-08-13 17:22:49.866003 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.865974, event: dispatched, request: pg_notify(24.1bb epoch 194471) v5 -2588> 2014-08-13 17:22:49.866041 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.866041, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194471) v5 -2587> 2014-08-13 17:22:49.866024 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.915217 7 0.000240 -2586> 2014-08-13 17:22:49.866053 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.866053, event: started, request: pg_notify(24.1bb epoch 194471) v5 -2585> 2014-08-13 17:22:49.866055 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2584> 2014-08-13 17:22:49.866075 7f74d8735700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:22:49.866075, event: done, request: pg_notify(24.1bb epoch 194471) v5 -2583> 2014-08-13 17:22:49.866087 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192981) [345,24,4] r=2 lpr=194470 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2582> 2014-08-13 17:22:49.866092 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 13 ==== pg_notify(24.136(22) epoch 194471) v5 ==== 1978+0+0 (2287980691 0 0) 0x53babc0 con 0x45f5960 -2581> 2014-08-13 17:22:49.866109 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.864219, event: header_read, request: pg_notify(24.136(22) epoch 194471) v5 -2580> 2014-08-13 17:22:49.866119 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.864226, event: throttled, request: pg_notify(24.136(22) epoch 194471) v5 -2579> 2014-08-13 17:22:49.866129 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.864326, event: all_read, request: pg_notify(24.136(22) epoch 194471) v5 -2578> 2014-08-13 17:22:49.866138 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.866107, event: dispatched, request: pg_notify(24.136(22) epoch 194471) v5 -2577> 2014-08-13 17:22:49.866148 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.866148, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194471) v5 -2576> 2014-08-13 17:22:49.866159 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.866158, event: started, request: pg_notify(24.136(22) epoch 194471) v5 -2575> 2014-08-13 17:22:49.866182 7f74d8735700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:22:49.866181, event: done, request: pg_notify(24.136(22) epoch 194471) v5 -2574> 2014-08-13 17:22:49.866208 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 5 ==== pg_notify(24.9b(27) epoch 194471) v5 ==== 2273+0+0 (833274124 0 0) 0x53b8700 con 0x47591e0 -2573> 2014-08-13 17:22:49.866228 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.865101, event: header_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2572> 2014-08-13 17:22:49.866239 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.865102, event: throttled, request: pg_notify(24.9b(27) epoch 194471) v5 -2571> 2014-08-13 17:22:49.866248 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.865218, event: all_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2570> 2014-08-13 17:22:49.866240 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.933389 7 0.000305 -2569> 2014-08-13 17:22:49.866257 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.866226, event: dispatched, request: pg_notify(24.9b(27) epoch 194471) v5 -2568> 2014-08-13 17:22:49.866280 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.866280, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194471) v5 -2567> 2014-08-13 17:22:49.866293 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.866293, event: started, request: pg_notify(24.9b(27) epoch 194471) v5 -2566> 2014-08-13 17:22:49.866276 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2565> 2014-08-13 17:22:49.866317 7f74d8735700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:22:49.866317, event: done, request: pg_notify(24.9b(27) epoch 194471) v5 -2564> 2014-08-13 17:22:49.866319 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194449 n=31 ec=183096 les/c 194465/194457 194470/194470/192968) [294,327,4] r=2 lpr=194470 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2563> 2014-08-13 17:22:49.866340 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 12 ==== pg_info(1 pgs e194471:22.6bd) v4 ==== 714+0+0 (1288475728 0 0) 0x49e0000 con 0x47a27e0 -2562> 2014-08-13 17:22:49.866359 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.865345, event: header_read, request: pg_info(1 pgs e194471:22.6bd) v4 -2561> 2014-08-13 17:22:49.866370 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.865346, event: throttled, request: pg_info(1 pgs e194471:22.6bd) v4 -2560> 2014-08-13 17:22:49.866379 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.865410, event: all_read, request: pg_info(1 pgs e194471:22.6bd) v4 -2559> 2014-08-13 17:22:49.866388 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.866357, event: dispatched, request: pg_info(1 pgs e194471:22.6bd) v4 -2558> 2014-08-13 17:22:49.866397 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.866397, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.6bd) v4 -2557> 2014-08-13 17:22:49.866408 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.866408, event: started, request: pg_info(1 pgs e194471:22.6bd) v4 -2556> 2014-08-13 17:22:49.866431 7f74d8735700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:22:49.866431, event: done, request: pg_info(1 pgs e194471:22.6bd) v4 -2555> 2014-08-13 17:22:49.866450 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 8 ==== pg_notify(24.3d epoch 194471) v5 ==== 694+0+0 (3414476826 0 0) 0x466abc0 con 0x48db9c0 -2554> 2014-08-13 17:22:49.866468 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.864927, event: header_read, request: pg_notify(24.3d epoch 194471) v5 -2553> 2014-08-13 17:22:49.866478 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.864928, event: throttled, request: pg_notify(24.3d epoch 194471) v5 -2552> 2014-08-13 17:22:49.866484 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.864998, event: all_read, request: pg_notify(24.3d epoch 194471) v5 -2551> 2014-08-13 17:22:49.866489 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.866466, event: dispatched, request: pg_notify(24.3d epoch 194471) v5 -2550> 2014-08-13 17:22:49.866495 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.866495, event: waiting_for_osdmap, request: pg_notify(24.3d epoch 194471) v5 -2549> 2014-08-13 17:22:49.866458 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.973686 7 0.000290 -2548> 2014-08-13 17:22:49.866502 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.866502, event: started, request: pg_notify(24.3d epoch 194471) v5 -2547> 2014-08-13 17:22:49.866520 7f74d8735700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:22:49.866520, event: done, request: pg_notify(24.3d epoch 194471) v5 -2546> 2014-08-13 17:22:49.866506 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2545> 2014-08-13 17:22:49.866533 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 6 ==== pg_notify(24.9b(27) epoch 194471) v5 ==== 2273+0+0 (2371126972 0 0) 0x57d6200 con 0x4d55ac0 -2544> 2014-08-13 17:22:49.866552 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.864615, event: header_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2543> 2014-08-13 17:22:49.866546 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192971) [321,379,4] r=2 lpr=194470 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2542> 2014-08-13 17:22:49.866566 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.864617, event: throttled, request: pg_notify(24.9b(27) epoch 194471) v5 -2541> 2014-08-13 17:22:49.866574 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.864749, event: all_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2540> 2014-08-13 17:22:49.866583 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.866550, event: dispatched, request: pg_notify(24.9b(27) epoch 194471) v5 -2539> 2014-08-13 17:22:49.866593 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.866593, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194471) v5 -2538> 2014-08-13 17:22:49.866604 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.866604, event: started, request: pg_notify(24.9b(27) epoch 194471) v5 -2537> 2014-08-13 17:22:49.866626 7f74d8735700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:22:49.866626, event: done, request: pg_notify(24.9b(27) epoch 194471) v5 -2536> 2014-08-13 17:22:49.866658 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 13 ==== pg_info(1 pgs e194471:22.48e) v4 ==== 714+0+0 (1101492374 0 0) 0x4ac0700 con 0x47a27e0 -2535> 2014-08-13 17:22:49.866677 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.865458, event: header_read, request: pg_info(1 pgs e194471:22.48e) v4 -2534> 2014-08-13 17:22:49.866687 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.865460, event: throttled, request: pg_info(1 pgs e194471:22.48e) v4 -2533> 2014-08-13 17:22:49.866696 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.865547, event: all_read, request: pg_info(1 pgs e194471:22.48e) v4 -2532> 2014-08-13 17:22:49.866705 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.866675, event: dispatched, request: pg_info(1 pgs e194471:22.48e) v4 -2531> 2014-08-13 17:22:49.866714 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.866714, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.48e) v4 -2530> 2014-08-13 17:22:49.866725 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.866725, event: started, request: pg_info(1 pgs e194471:22.48e) v4 -2529> 2014-08-13 17:22:49.866748 7f74d8735700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:22:49.866747, event: done, request: pg_info(1 pgs e194471:22.48e) v4 -2528> 2014-08-13 17:22:49.866870 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.924653 7 0.000506 -2527> 2014-08-13 17:22:49.866910 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2526> 2014-08-13 17:22:49.866934 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194449 n=37 ec=183096 les/c 194465/194457 194470/194470/192966) [358,374,4] r=2 lpr=194470 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2525> 2014-08-13 17:22:49.866983 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 6 ==== pg_notify(24.9b(27) epoch 194471) v5 ==== 2273+0+0 (833274124 0 0) 0x5264440 con 0x47591e0 -2524> 2014-08-13 17:22:49.867008 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.866841, event: header_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2523> 2014-08-13 17:22:49.867020 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.866843, event: throttled, request: pg_notify(24.9b(27) epoch 194471) v5 -2522> 2014-08-13 17:22:49.867029 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.866959, event: all_read, request: pg_notify(24.9b(27) epoch 194471) v5 -2521> 2014-08-13 17:22:49.867038 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.867005, event: dispatched, request: pg_notify(24.9b(27) epoch 194471) v5 -2520> 2014-08-13 17:22:49.867048 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.867047, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194471) v5 -2519> 2014-08-13 17:22:49.867059 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.867059, event: started, request: pg_notify(24.9b(27) epoch 194471) v5 -2518> 2014-08-13 17:22:49.867083 7f74d8735700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:22:49.867083, event: done, request: pg_notify(24.9b(27) epoch 194471) v5 -2517> 2014-08-13 17:22:49.867109 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.918642 7 0.000292 -2516> 2014-08-13 17:22:49.867140 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2515> 2014-08-13 17:22:49.867216 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194449 n=35 ec=183096 les/c 194465/194457 194470/194470/192943) [337,20,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2514> 2014-08-13 17:22:49.867378 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.937570 7 0.000272 -2513> 2014-08-13 17:22:49.867410 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2512> 2014-08-13 17:22:49.867446 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194449 n=23 ec=183096 les/c 194466/194466 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2511> 2014-08-13 17:22:49.867684 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194465/194465 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.943963 7 0.000278 -2510> 2014-08-13 17:22:49.867721 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194465/194465 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2509> 2014-08-13 17:22:49.867744 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194449 n=40 ec=183096 les/c 194465/194465 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2508> 2014-08-13 17:22:49.867897 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(30) epoch 194471) v5 -- ?+0 0x53baa00 con 0x47a53e0 -2507> 2014-08-13 17:22:49.867929 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(24) epoch 194471) v5 -- ?+0 0x53bd5c0 con 0x475cd00 -2506> 2014-08-13 17:22:49.867966 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(24) epoch 194471) v5 -- ?+0 0x53b8c40 con 0x48377a0 -2505> 2014-08-13 17:22:49.867988 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(27) epoch 194471) v5 -- ?+0 0x53bcb40 con 0x47a4200 -2504> 2014-08-13 17:22:49.868045 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.309 192.168.1.101:6814/23703 7 ==== pg_info(1 pgs e194471:22.481) v4 ==== 714+0+0 (3508577336 0 0) 0x49e1dc0 con 0x4903440 -2503> 2014-08-13 17:22:49.868086 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.867856, event: header_read, request: pg_info(1 pgs e194471:22.481) v4 -2502> 2014-08-13 17:22:49.868097 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.867858, event: throttled, request: pg_info(1 pgs e194471:22.481) v4 -2501> 2014-08-13 17:22:49.868106 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.868025, event: all_read, request: pg_info(1 pgs e194471:22.481) v4 -2500> 2014-08-13 17:22:49.868115 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.868083, event: dispatched, request: pg_info(1 pgs e194471:22.481) v4 -2499> 2014-08-13 17:22:49.868125 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.868125, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.481) v4 -2498> 2014-08-13 17:22:49.868136 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.868136, event: started, request: pg_info(1 pgs e194471:22.481) v4 -2497> 2014-08-13 17:22:49.868162 7f74d8735700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:22:49.868162, event: done, request: pg_info(1 pgs e194471:22.481) v4 -2496> 2014-08-13 17:22:49.868145 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.936167 7 0.000265 -2495> 2014-08-13 17:22:49.868209 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2494> 2014-08-13 17:22:49.868238 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194449 n=33 ec=183096 les/c 194465/194457 194470/194470/192958) [397,343,4] r=2 lpr=194470 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2493> 2014-08-13 17:22:49.868442 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.59 192.168.1.112:6822/5265 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de60c0 con 0x54e4360 -2492> 2014-08-13 17:22:49.868491 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2491> 2014-08-13 17:22:49.868533 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.59 192.168.1.112:6822/5265 4 ==== pg_notify(24.1f7 epoch 194471) v5 ==== 694+0+0 (3450593779 0 0) 0x53b9500 con 0x54e4360 -2490> 2014-08-13 17:22:49.868557 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868411, event: header_read, request: pg_notify(24.1f7 epoch 194471) v5 -2489> 2014-08-13 17:22:49.868572 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868412, event: throttled, request: pg_notify(24.1f7 epoch 194471) v5 -2488> 2014-08-13 17:22:49.868581 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868504, event: all_read, request: pg_notify(24.1f7 epoch 194471) v5 -2487> 2014-08-13 17:22:49.868590 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868553, event: dispatched, request: pg_notify(24.1f7 epoch 194471) v5 -2486> 2014-08-13 17:22:49.868600 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868600, event: waiting_for_osdmap, request: pg_notify(24.1f7 epoch 194471) v5 -2485> 2014-08-13 17:22:49.868613 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868613, event: started, request: pg_notify(24.1f7 epoch 194471) v5 -2484> 2014-08-13 17:22:49.868645 7f74d8735700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:22:49.868645, event: done, request: pg_notify(24.1f7 epoch 194471) v5 -2483> 2014-08-13 17:22:49.869458 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4806540 con 0x47f70c0 -2482> 2014-08-13 17:22:49.869496 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2481> 2014-08-13 17:22:49.869508 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de3180 con 0x4d57d20 -2480> 2014-08-13 17:22:49.869524 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2479> 2014-08-13 17:22:49.869570 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 7 ==== pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 ==== 1686+0+0 (2259181811 0 0) 0x49e16c0 con 0x47f70c0 -2478> 2014-08-13 17:22:49.869634 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869444, event: header_read, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2477> 2014-08-13 17:22:49.869647 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869445, event: throttled, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2476> 2014-08-13 17:22:49.869657 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869549, event: all_read, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2475> 2014-08-13 17:22:49.869666 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869631, event: dispatched, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2474> 2014-08-13 17:22:49.869677 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869677, event: waiting_for_osdmap, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2473> 2014-08-13 17:22:49.869690 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869690, event: started, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2472> 2014-08-13 17:22:49.869680 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] exit Started/Stray 0.026841 1 0.000133 -2471> 2014-08-13 17:22:49.869731 7f74d8735700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:22:49.869731, event: done, request: pg_notify(24.136(2),22.6bd(2) epoch 194471) v5 -2470> 2014-08-13 17:22:49.869723 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] enter Started/ReplicaActive -2469> 2014-08-13 17:22:49.869758 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 5 ==== pg_notify(22.4df(1) epoch 194471) v5 ==== 777+0+0 (2185514268 0 0) 0x57d3b80 con 0x4d57d20 -2468> 2014-08-13 17:22:49.869778 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869475, event: header_read, request: pg_notify(22.4df(1) epoch 194471) v5 -2467> 2014-08-13 17:22:49.869788 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869476, event: throttled, request: pg_notify(22.4df(1) epoch 194471) v5 -2466> 2014-08-13 17:22:49.869798 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869551, event: all_read, request: pg_notify(22.4df(1) epoch 194471) v5 -2465> 2014-08-13 17:22:49.869790 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194469/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -2464> 2014-08-13 17:22:49.869806 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869776, event: dispatched, request: pg_notify(22.4df(1) epoch 194471) v5 -2463> 2014-08-13 17:22:49.869817 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869816, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194471) v5 -2462> 2014-08-13 17:22:49.869830 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869830, event: started, request: pg_notify(22.4df(1) epoch 194471) v5 -2461> 2014-08-13 17:22:49.869854 7f74d8735700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:22:49.869853, event: done, request: pg_notify(22.4df(1) epoch 194471) v5 -2460> 2014-08-13 17:22:49.869871 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 8 ==== pg_notify(24.136(2) epoch 194471) v5 ==== 870+0+0 (2953840395 0 0) 0x4ac7380 con 0x47f70c0 -2459> 2014-08-13 17:22:49.869889 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869580, event: header_read, request: pg_notify(24.136(2) epoch 194471) v5 -2458> 2014-08-13 17:22:49.869900 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869582, event: throttled, request: pg_notify(24.136(2) epoch 194471) v5 -2457> 2014-08-13 17:22:49.869909 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869638, event: all_read, request: pg_notify(24.136(2) epoch 194471) v5 -2456> 2014-08-13 17:22:49.869918 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869887, event: dispatched, request: pg_notify(24.136(2) epoch 194471) v5 -2455> 2014-08-13 17:22:49.869927 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869927, event: waiting_for_osdmap, request: pg_notify(24.136(2) epoch 194471) v5 -2454> 2014-08-13 17:22:49.869939 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869939, event: started, request: pg_notify(24.136(2) epoch 194471) v5 -2453> 2014-08-13 17:22:49.869962 7f74d8735700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:22:49.869962, event: done, request: pg_notify(24.136(2) epoch 194471) v5 -2452> 2014-08-13 17:22:49.870283 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de45c0 con 0x4d55540 -2451> 2014-08-13 17:22:49.870308 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2450> 2014-08-13 17:22:49.870481 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 5 ==== pg_notify(22.481(37) epoch 194471) v5 ==== 2801+0+0 (1363036900 0 0) 0x57d4600 con 0x4d55540 -2449> 2014-08-13 17:22:49.870502 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870349, event: header_read, request: pg_notify(22.481(37) epoch 194471) v5 -2448> 2014-08-13 17:22:49.870513 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870351, event: throttled, request: pg_notify(22.481(37) epoch 194471) v5 -2447> 2014-08-13 17:22:49.870523 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870461, event: all_read, request: pg_notify(22.481(37) epoch 194471) v5 -2446> 2014-08-13 17:22:49.870532 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870499, event: dispatched, request: pg_notify(22.481(37) epoch 194471) v5 -2445> 2014-08-13 17:22:49.870542 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870542, event: waiting_for_osdmap, request: pg_notify(22.481(37) epoch 194471) v5 -2444> 2014-08-13 17:22:49.870554 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870553, event: started, request: pg_notify(22.481(37) epoch 194471) v5 -2443> 2014-08-13 17:22:49.870590 7f74d8735700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:22:49.870590, event: done, request: pg_notify(22.481(37) epoch 194471) v5 -2442> 2014-08-13 17:22:49.870589 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(29) epoch 194471) v5 -- ?+0 0x57d55c0 con 0x486e040 -2441> 2014-08-13 17:22:49.870624 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(26) epoch 194471) v5 -- ?+0 0x57d2a00 con 0x486b180 -2440> 2014-08-13 17:22:49.870646 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(40) epoch 194471) v5 -- ?+0 0x57d39c0 con 0x486b9c0 -2439> 2014-08-13 17:22:49.870643 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a87500 con 0x4831600 -2438> 2014-08-13 17:22:49.870678 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2437> 2014-08-13 17:22:49.870777 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 5 ==== pg_notify(22.1b8(20) epoch 194471) v5 ==== 1838+0+0 (3605504459 0 0) 0x57d32c0 con 0x4831600 -2436> 2014-08-13 17:22:49.870798 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870648, event: header_read, request: pg_notify(22.1b8(20) epoch 194471) v5 -2435> 2014-08-13 17:22:49.870806 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870650, event: throttled, request: pg_notify(22.1b8(20) epoch 194471) v5 -2434> 2014-08-13 17:22:49.870814 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870736, event: all_read, request: pg_notify(22.1b8(20) epoch 194471) v5 -2433> 2014-08-13 17:22:49.870824 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870795, event: dispatched, request: pg_notify(22.1b8(20) epoch 194471) v5 -2432> 2014-08-13 17:22:49.870834 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870834, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194471) v5 -2431> 2014-08-13 17:22:49.870846 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870846, event: started, request: pg_notify(22.1b8(20) epoch 194471) v5 -2430> 2014-08-13 17:22:49.870881 7f74d8735700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:22:49.870880, event: done, request: pg_notify(22.1b8(20) epoch 194471) v5 -2429> 2014-08-13 17:22:49.870945 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(24) epoch 194471) v5 -- ?+0 0x53bfe00 con 0x47a32e0 -2428> 2014-08-13 17:22:49.870974 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(24) epoch 194471) v5 -- ?+0 0x53bf8c0 con 0x47f0c60 -2427> 2014-08-13 17:22:49.871045 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.318 192.168.1.110:6814/43857 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c77500 con 0x541c4c0 -2426> 2014-08-13 17:22:49.871072 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2425> 2014-08-13 17:22:49.871111 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194471 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194471) v4 -- ?+0 0x278ec00 con 0x486b180 -2424> 2014-08-13 17:22:49.871129 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.318 192.168.1.110:6814/43857 4 ==== pg_info(1 pgs e194471:22.3e4) v4 ==== 714+0+0 (3273464784 0 0) 0x53bb640 con 0x541c4c0 -2423> 2014-08-13 17:22:49.871151 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871047, event: header_read, request: pg_info(1 pgs e194471:22.3e4) v4 -2422> 2014-08-13 17:22:49.871162 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871048, event: throttled, request: pg_info(1 pgs e194471:22.3e4) v4 -2421> 2014-08-13 17:22:49.871171 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871110, event: all_read, request: pg_info(1 pgs e194471:22.3e4) v4 -2420> 2014-08-13 17:22:49.871180 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871148, event: dispatched, request: pg_info(1 pgs e194471:22.3e4) v4 -2419> 2014-08-13 17:22:49.871190 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871189, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3e4) v4 -2418> 2014-08-13 17:22:49.871212 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871212, event: started, request: pg_info(1 pgs e194471:22.3e4) v4 -2417> 2014-08-13 17:22:49.871246 7f74d8735700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:22:49.871245, event: done, request: pg_info(1 pgs e194471:22.3e4) v4 -2416> 2014-08-13 17:22:49.871333 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.927047 7 0.000317 -2415> 2014-08-13 17:22:49.871372 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2414> 2014-08-13 17:22:49.871428 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194449 n=29 ec=183096 les/c 194465/194457 194470/194470/192978) [373,360,4] r=2 lpr=194470 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2413> 2014-08-13 17:22:49.871704 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_log(24.1de epoch 194471 log log((194422'17074,194422'17074], crt=194422'17074) query_epoch 194471) v4 -- ?+0 0x5c71200 con 0x47a4200 -2412> 2014-08-13 17:22:49.871771 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_log(24.2a1 epoch 194471 log log((194422'14353,194422'14353], crt=194422'14353) query_epoch 194471) v4 -- ?+0 0x5c70c00 con 0x47a53e0 -2411> 2014-08-13 17:22:49.871816 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.018207 6 0.000213 -2410> 2014-08-13 17:22:49.871849 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog -2409> 2014-08-13 17:22:49.871928 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000079 0 0.000000 -2408> 2014-08-13 17:22:49.871956 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing -2407> 2014-08-13 17:22:49.871988 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000031 0 0.000000 -2406> 2014-08-13 17:22:49.872013 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru -2405> 2014-08-13 17:22:49.872066 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(37) epoch 194471) v5 -- ?+0 0x53bfa80 con 0x47a27e0 -2404> 2014-08-13 17:22:49.872134 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194466/194466 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.921729 7 0.000305 -2403> 2014-08-13 17:22:49.872173 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194466/194466 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2402> 2014-08-13 17:22:49.872199 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194449 n=33 ec=183096 les/c 194466/194466 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2401> 2014-08-13 17:22:49.872384 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_log(24.3f7 epoch 194471 log log((194422'15537,194422'15537], crt=194422'15537) query_epoch 194471) v4 -- ?+0 0x5c70600 con 0x486e040 -2400> 2014-08-13 17:22:49.872447 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 6 ==== pg_query(24.e2 epoch 194471) v3 ==== 144+0+0 (4280667622 0 0) 0x29c4b00 con 0x486b180 -2399> 2014-08-13 17:22:49.872467 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872338, event: header_read, request: pg_query(24.e2 epoch 194471) v3 -2398> 2014-08-13 17:22:49.872474 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872340, event: throttled, request: pg_query(24.e2 epoch 194471) v3 -2397> 2014-08-13 17:22:49.872479 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872425, event: all_read, request: pg_query(24.e2 epoch 194471) v3 -2396> 2014-08-13 17:22:49.872484 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872464, event: dispatched, request: pg_query(24.e2 epoch 194471) v3 -2395> 2014-08-13 17:22:49.872489 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872489, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194471) v3 -2394> 2014-08-13 17:22:49.872497 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872497, event: started, request: pg_query(24.e2 epoch 194471) v3 -2393> 2014-08-13 17:22:49.872515 7f74d8735700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:22:49.872515, event: done, request: pg_query(24.e2 epoch 194471) v3 -2392> 2014-08-13 17:22:49.872511 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetInfo 0.069864 7 0.000250 -2391> 2014-08-13 17:22:49.872542 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetLog -2390> 2014-08-13 17:22:49.872618 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetLog 0.000076 0 0.000000 -2389> 2014-08-13 17:22:49.872648 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetMissing -2388> 2014-08-13 17:22:49.872671 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -2387> 2014-08-13 17:22:49.872697 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/WaitUpThru -2386> 2014-08-13 17:22:49.872788 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.113:6808/15284 -- pg_log(24.5e epoch 194471 log log((194422'17034,194422'17034], crt=194422'17034) query_epoch 194471) v4 -- ?+0 0x5c70000 con 0x47a27e0 -2385> 2014-08-13 17:22:49.873024 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 8 ==== pg_notify(24.34(48) epoch 194471) v5 ==== 3196+0+0 (2475150164 0 0) 0x57d6e40 con 0x47a3f40 -2384> 2014-08-13 17:22:49.873024 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_log(24.369 epoch 194471 log log((194422'13576,194422'13576], crt=194422'13576) query_epoch 194471) v4 -- ?+0 0x5c74e00 con 0x47a32e0 -2383> 2014-08-13 17:22:49.873052 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.872855, event: header_read, request: pg_notify(24.34(48) epoch 194471) v5 -2382> 2014-08-13 17:22:49.873060 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.872857, event: throttled, request: pg_notify(24.34(48) epoch 194471) v5 -2381> 2014-08-13 17:22:49.873069 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.873001, event: all_read, request: pg_notify(24.34(48) epoch 194471) v5 -2380> 2014-08-13 17:22:49.873078 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.873049, event: dispatched, request: pg_notify(24.34(48) epoch 194471) v5 -2379> 2014-08-13 17:22:49.873088 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.873088, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194471) v5 -2378> 2014-08-13 17:22:49.873100 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.873099, event: started, request: pg_notify(24.34(48) epoch 194471) v5 -2377> 2014-08-13 17:22:49.873126 7f74d8735700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:22:49.873126, event: done, request: pg_notify(24.34(48) epoch 194471) v5 -2376> 2014-08-13 17:22:49.873245 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_log(24.33a epoch 194471 log log((194422'16034,194422'16034], crt=194422'16034) query_epoch 194471) v4 -- ?+0 0x5c74800 con 0x47f0c60 -2375> 2014-08-13 17:22:49.873301 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194471 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194471) v4 -- ?+0 0x5c74200 con 0x4835ac0 -2374> 2014-08-13 17:22:49.873368 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_log(24.2ec epoch 194471 log log((194422'16449,194422'16449], crt=194422'16449) query_epoch 194471) v4 -- ?+0 0x5c73c00 con 0x486b9c0 -2373> 2014-08-13 17:22:49.873684 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6822/26295 -- pg_log(24.1e6 epoch 194471 log log((194422'17642,194422'17642], crt=194422'17642) query_epoch 194471) v4 -- ?+0 0x5c73600 con 0x475cd00 -2372> 2014-08-13 17:22:49.873766 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- pg_log(24.1e4 epoch 194471 log log((194422'18567,194422'18567], crt=194422'18567) query_epoch 194471) v4 -- ?+0 0x5b1e000 con 0x47f4620 -2371> 2014-08-13 17:22:49.873837 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_log(24.206 epoch 194471 log log((194422'14476,194422'14476], crt=194422'14476) query_epoch 194471) v4 -- ?+0 0x5c77200 con 0x48377a0 -2370> 2014-08-13 17:22:49.874117 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.041414 7 0.000240 -2369> 2014-08-13 17:22:49.874163 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -2368> 2014-08-13 17:22:49.874241 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000079 0 0.000000 -2367> 2014-08-13 17:22:49.874268 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -2366> 2014-08-13 17:22:49.874296 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000027 0 0.000000 -2365> 2014-08-13 17:22:49.874320 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -2364> 2014-08-13 17:22:49.874396 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.039980 9 0.000259 -2363> 2014-08-13 17:22:49.874427 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog -2362> 2014-08-13 17:22:49.874497 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000071 0 0.000000 -2361> 2014-08-13 17:22:49.874524 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing -2360> 2014-08-13 17:22:49.874551 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.000028 0 0.000000 -2359> 2014-08-13 17:22:49.874575 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru -2358> 2014-08-13 17:22:49.874642 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 6 ==== pg_query(24.cd epoch 194471) v3 ==== 144+0+0 (152338482 0 0) 0x29c7bc0 con 0x4835ac0 -2357> 2014-08-13 17:22:49.874651 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.028409 8 0.000250 -2356> 2014-08-13 17:22:49.874671 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874532, event: header_read, request: pg_query(24.cd epoch 194471) v3 -2355> 2014-08-13 17:22:49.874681 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874534, event: throttled, request: pg_query(24.cd epoch 194471) v3 -2354> 2014-08-13 17:22:49.874690 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874617, event: all_read, request: pg_query(24.cd epoch 194471) v3 -2353> 2014-08-13 17:22:49.874682 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog -2352> 2014-08-13 17:22:49.874699 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874667, event: dispatched, request: pg_query(24.cd epoch 194471) v3 -2351> 2014-08-13 17:22:49.874709 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874709, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194471) v3 -2350> 2014-08-13 17:22:49.874720 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874720, event: started, request: pg_query(24.cd epoch 194471) v3 -2349> 2014-08-13 17:22:49.874742 7f74d8735700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:22:49.874742, event: done, request: pg_query(24.cd epoch 194471) v3 -2348> 2014-08-13 17:22:49.874746 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000063 0 0.000000 -2347> 2014-08-13 17:22:49.874772 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing -2346> 2014-08-13 17:22:49.874797 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000025 0 0.000000 -2345> 2014-08-13 17:22:49.874820 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru -2344> 2014-08-13 17:22:49.874878 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.028093 8 0.000255 -2343> 2014-08-13 17:22:49.874907 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog -2342> 2014-08-13 17:22:49.874975 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000069 0 0.000000 -2341> 2014-08-13 17:22:49.875001 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing -2340> 2014-08-13 17:22:49.875026 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000026 0 0.000000 -2339> 2014-08-13 17:22:49.875050 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru -2338> 2014-08-13 17:22:49.875147 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194471 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194471) v4 -- ?+0 0x5c76c00 con 0x486b180 -2337> 2014-08-13 17:22:49.875389 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194471 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194471) v4 -- ?+0 0x5c76600 con 0x4835ac0 -2336> 2014-08-13 17:22:49.875451 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 9 ==== pg_info(1 pgs e194471:22.3af) v4 ==== 714+0+0 (3931229169 0 0) 0x49e2680 con 0x48db9c0 -2335> 2014-08-13 17:22:49.875473 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875350, event: header_read, request: pg_info(1 pgs e194471:22.3af) v4 -2334> 2014-08-13 17:22:49.875483 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875352, event: throttled, request: pg_info(1 pgs e194471:22.3af) v4 -2333> 2014-08-13 17:22:49.875492 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875429, event: all_read, request: pg_info(1 pgs e194471:22.3af) v4 -2332> 2014-08-13 17:22:49.875501 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875470, event: dispatched, request: pg_info(1 pgs e194471:22.3af) v4 -2331> 2014-08-13 17:22:49.875511 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875511, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3af) v4 -2330> 2014-08-13 17:22:49.875523 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875522, event: started, request: pg_info(1 pgs e194471:22.3af) v4 -2329> 2014-08-13 17:22:49.875553 7f74d8735700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:22:49.875552, event: done, request: pg_info(1 pgs e194471:22.3af) v4 -2328> 2014-08-13 17:22:49.876467 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 7 ==== pg_info(1 pgs e194471:22.f3) v4 ==== 714+0+0 (1650754937 0 0) 0x53bd5c0 con 0x475cd00 -2327> 2014-08-13 17:22:49.876511 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876317, event: header_read, request: pg_info(1 pgs e194471:22.f3) v4 -2326> 2014-08-13 17:22:49.876526 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876319, event: throttled, request: pg_info(1 pgs e194471:22.f3) v4 -2325> 2014-08-13 17:22:49.876536 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876400, event: all_read, request: pg_info(1 pgs e194471:22.f3) v4 -2324> 2014-08-13 17:22:49.876546 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876505, event: dispatched, request: pg_info(1 pgs e194471:22.f3) v4 -2323> 2014-08-13 17:22:49.876557 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876557, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.f3) v4 -2322> 2014-08-13 17:22:49.876570 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876570, event: started, request: pg_info(1 pgs e194471:22.f3) v4 -2321> 2014-08-13 17:22:49.876606 7f74d8735700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:22:49.876605, event: done, request: pg_info(1 pgs e194471:22.f3) v4 -2320> 2014-08-13 17:22:49.878034 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 7 ==== pg_info(1 pgs e194471:22.113) v4 ==== 714+0+0 (2469436721 0 0) 0x53bb9c0 con 0x4900f20 -2319> 2014-08-13 17:22:49.878073 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.877925, event: header_read, request: pg_info(1 pgs e194471:22.113) v4 -2318> 2014-08-13 17:22:49.878085 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.877927, event: throttled, request: pg_info(1 pgs e194471:22.113) v4 -2317> 2014-08-13 17:22:49.878092 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.878003, event: all_read, request: pg_info(1 pgs e194471:22.113) v4 -2316> 2014-08-13 17:22:49.878098 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.878068, event: dispatched, request: pg_info(1 pgs e194471:22.113) v4 -2315> 2014-08-13 17:22:49.878105 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.878104, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.113) v4 -2314> 2014-08-13 17:22:49.878114 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.878113, event: started, request: pg_info(1 pgs e194471:22.113) v4 -2313> 2014-08-13 17:22:49.878149 7f74d8735700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:22:49.878148, event: done, request: pg_info(1 pgs e194471:22.113) v4 -2312> 2014-08-13 17:22:49.879590 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.960615 7 0.000254 -2311> 2014-08-13 17:22:49.879658 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2310> 2014-08-13 17:22:49.879701 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194449 n=34 ec=183096 les/c 194465/194457 194470/194470/192935) [58,28,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2309> 2014-08-13 17:22:49.879763 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.341 192.168.1.106:6818/9142 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de5340 con 0x48de880 -2308> 2014-08-13 17:22:49.879810 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2307> 2014-08-13 17:22:49.879831 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.341 192.168.1.106:6818/9142 4 ==== pg_info(1 pgs e194471:22.48e) v4 ==== 714+0+0 (2947332500 0 0) 0x49e7540 con 0x48de880 -2306> 2014-08-13 17:22:49.879852 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879750, event: header_read, request: pg_info(1 pgs e194471:22.48e) v4 -2305> 2014-08-13 17:22:49.879867 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879751, event: throttled, request: pg_info(1 pgs e194471:22.48e) v4 -2304> 2014-08-13 17:22:49.879877 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879808, event: all_read, request: pg_info(1 pgs e194471:22.48e) v4 -2303> 2014-08-13 17:22:49.879886 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879849, event: dispatched, request: pg_info(1 pgs e194471:22.48e) v4 -2302> 2014-08-13 17:22:49.879897 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879897, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.48e) v4 -2301> 2014-08-13 17:22:49.879910 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879910, event: started, request: pg_info(1 pgs e194471:22.48e) v4 -2300> 2014-08-13 17:22:49.879947 7f74d8735700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:22:49.879947, event: done, request: pg_info(1 pgs e194471:22.48e) v4 -2299> 2014-08-13 17:22:49.880110 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194465/194457 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.927052 7 0.000292 -2298> 2014-08-13 17:22:49.880266 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194465/194457 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2297> 2014-08-13 17:22:49.880308 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194449 n=30 ec=183096 les/c 194465/194457 194470/194470/192949) [328,4,20] r=1 lpr=194470 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2296> 2014-08-13 17:22:49.880454 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194465/194457 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.947321 7 0.000306 -2295> 2014-08-13 17:22:49.880485 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194465/194457 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2294> 2014-08-13 17:22:49.880544 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194449 n=26 ec=183096 les/c 194465/194457 194470/194470/192976) [30,335,4] r=2 lpr=194470 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2293> 2014-08-13 17:22:49.880722 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de1680 con 0x4d53c80 -2292> 2014-08-13 17:22:49.880767 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2291> 2014-08-13 17:22:49.880754 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.954791 7 0.000469 -2290> 2014-08-13 17:22:49.880805 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2289> 2014-08-13 17:22:49.880836 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 5 ==== pg_notify(22.39d(1) epoch 194471) v5 ==== 777+0+0 (1349413272 0 0) 0x57d5240 con 0x4d53c80 -2288> 2014-08-13 17:22:49.880827 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194449 n=29 ec=183096 les/c 194465/194465 194470/194470/192981) [345,4,374] r=1 lpr=194470 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2287> 2014-08-13 17:22:49.880861 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880717, event: header_read, request: pg_notify(22.39d(1) epoch 194471) v5 -2286> 2014-08-13 17:22:49.880875 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880718, event: throttled, request: pg_notify(22.39d(1) epoch 194471) v5 -2285> 2014-08-13 17:22:49.880885 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880815, event: all_read, request: pg_notify(22.39d(1) epoch 194471) v5 -2284> 2014-08-13 17:22:49.880895 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880857, event: dispatched, request: pg_notify(22.39d(1) epoch 194471) v5 -2283> 2014-08-13 17:22:49.880905 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880905, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194471) v5 -2282> 2014-08-13 17:22:49.880919 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880919, event: started, request: pg_notify(22.39d(1) epoch 194471) v5 -2281> 2014-08-13 17:22:49.880954 7f74d8735700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:22:49.880953, event: done, request: pg_notify(22.39d(1) epoch 194471) v5 -2280> 2014-08-13 17:22:49.880997 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(26) epoch 194471) v5 -- ?+0 0x53b96c0 con 0x48d9ce0 -2279> 2014-08-13 17:22:49.881030 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(24),24.3b7(29) epoch 194471) v5 -- ?+0 0x53bc600 con 0x45f5960 -2278> 2014-08-13 17:22:49.881136 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.037261 8 0.000260 -2277> 2014-08-13 17:22:49.881193 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -2276> 2014-08-13 17:22:49.881276 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000084 0 0.000000 -2275> 2014-08-13 17:22:49.881319 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -2274> 2014-08-13 17:22:49.881347 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 -2273> 2014-08-13 17:22:49.881372 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -2272> 2014-08-13 17:22:49.881493 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6812/53408 -- pg_log(24.20e epoch 194471 log log((194422'11806,194422'11806], crt=194422'11806) query_epoch 194471) v4 -- ?+0 0x278da00 con 0x48d9ce0 -2271> 2014-08-13 17:22:49.881736 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_log(24.350 epoch 194471 log log((194422'17433,194422'17433], crt=194422'17433) query_epoch 194471) v4 -- ?+0 0x5c71e00 con 0x45f5960 -2270> 2014-08-13 17:22:49.881998 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194471 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194471) v4 -- ?+0 0x278e600 con 0x45f5960 -2269> 2014-08-13 17:22:49.882504 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.62 192.168.1.112:6824/5505 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x48060c0 con 0x45f77a0 -2268> 2014-08-13 17:22:49.882553 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2267> 2014-08-13 17:22:49.882603 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.62 192.168.1.112:6824/5505 4 ==== pg_info(1 pgs e194471:22.619) v4 ==== 714+0+0 (4247008137 0 0) 0x49e1340 con 0x45f77a0 -2266> 2014-08-13 17:22:49.882627 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882508, event: header_read, request: pg_info(1 pgs e194471:22.619) v4 -2265> 2014-08-13 17:22:49.882641 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882509, event: throttled, request: pg_info(1 pgs e194471:22.619) v4 -2264> 2014-08-13 17:22:49.882651 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882582, event: all_read, request: pg_info(1 pgs e194471:22.619) v4 -2263> 2014-08-13 17:22:49.882660 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882624, event: dispatched, request: pg_info(1 pgs e194471:22.619) v4 -2262> 2014-08-13 17:22:49.882670 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882670, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.619) v4 -2261> 2014-08-13 17:22:49.882682 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882682, event: started, request: pg_info(1 pgs e194471:22.619) v4 -2260> 2014-08-13 17:22:49.882716 7f74d8735700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:22:49.882716, event: done, request: pg_info(1 pgs e194471:22.619) v4 -2259> 2014-08-13 17:22:49.883368 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 14 ==== pg_query(24.3b7 epoch 194471) v3 ==== 144+0+0 (4152220963 0 0) 0x29c1c20 con 0x45f5960 -2258> 2014-08-13 17:22:49.883403 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883244, event: header_read, request: pg_query(24.3b7 epoch 194471) v3 -2257> 2014-08-13 17:22:49.883418 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883246, event: throttled, request: pg_query(24.3b7 epoch 194471) v3 -2256> 2014-08-13 17:22:49.883427 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883319, event: all_read, request: pg_query(24.3b7 epoch 194471) v3 -2255> 2014-08-13 17:22:49.883437 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883399, event: dispatched, request: pg_query(24.3b7 epoch 194471) v3 -2254> 2014-08-13 17:22:49.883447 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883447, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194471) v3 -2253> 2014-08-13 17:22:49.883459 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883459, event: started, request: pg_query(24.3b7 epoch 194471) v3 -2252> 2014-08-13 17:22:49.883488 7f74d8735700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:22:49.883488, event: done, request: pg_query(24.3b7 epoch 194471) v3 -2251> 2014-08-13 17:22:49.883531 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194471 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194471) v4 -- ?+0 0x278c200 con 0x45f5960 -2250> 2014-08-13 17:22:49.887269 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e32400 con 0x4904780 -2249> 2014-08-13 17:22:49.887316 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2248> 2014-08-13 17:22:49.887343 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 5 ==== pg_notify(22.619(22) epoch 194471) v5 ==== 1948+0+0 (1645947433 0 0) 0x53ba300 con 0x4904780 -2247> 2014-08-13 17:22:49.887365 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887226, event: header_read, request: pg_notify(22.619(22) epoch 194471) v5 -2246> 2014-08-13 17:22:49.887380 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887228, event: throttled, request: pg_notify(22.619(22) epoch 194471) v5 -2245> 2014-08-13 17:22:49.887390 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887322, event: all_read, request: pg_notify(22.619(22) epoch 194471) v5 -2244> 2014-08-13 17:22:49.887399 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887362, event: dispatched, request: pg_notify(22.619(22) epoch 194471) v5 -2243> 2014-08-13 17:22:49.887410 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887409, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194471) v5 -2242> 2014-08-13 17:22:49.887423 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887422, event: started, request: pg_notify(22.619(22) epoch 194471) v5 -2241> 2014-08-13 17:22:49.887457 7f74d8735700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:22:49.887457, event: done, request: pg_notify(22.619(22) epoch 194471) v5 -2240> 2014-08-13 17:22:49.888211 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.310 192.168.1.109:6812/32413 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c74380 con 0x4d53700 -2239> 2014-08-13 17:22:49.888258 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -2238> 2014-08-13 17:22:49.888282 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.310 192.168.1.109:6812/32413 4 ==== pg_info(1 pgs e194471:22.707) v4 ==== 714+0+0 (149088067 0 0) 0x49e0000 con 0x4d53700 -2237> 2014-08-13 17:22:49.888303 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888170, event: header_read, request: pg_info(1 pgs e194471:22.707) v4 -2236> 2014-08-13 17:22:49.888318 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888171, event: throttled, request: pg_info(1 pgs e194471:22.707) v4 -2235> 2014-08-13 17:22:49.888327 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888243, event: all_read, request: pg_info(1 pgs e194471:22.707) v4 -2234> 2014-08-13 17:22:49.888337 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888300, event: dispatched, request: pg_info(1 pgs e194471:22.707) v4 -2233> 2014-08-13 17:22:49.888347 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888347, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.707) v4 -2232> 2014-08-13 17:22:49.888360 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888360, event: started, request: pg_info(1 pgs e194471:22.707) v4 -2231> 2014-08-13 17:22:49.888396 7f74d8735700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:22:49.888395, event: done, request: pg_info(1 pgs e194471:22.707) v4 -2230> 2014-08-13 17:22:49.890849 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 9 ==== pg_info(1 pgs e194471:22.1b8) v4 ==== 714+0+0 (1218778660 0 0) 0x57d5cc0 con 0x47a3f40 -2229> 2014-08-13 17:22:49.890878 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890681, event: header_read, request: pg_info(1 pgs e194471:22.1b8) v4 -2228> 2014-08-13 17:22:49.890889 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890684, event: throttled, request: pg_info(1 pgs e194471:22.1b8) v4 -2227> 2014-08-13 17:22:49.890898 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890793, event: all_read, request: pg_info(1 pgs e194471:22.1b8) v4 -2226> 2014-08-13 17:22:49.890907 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890874, event: dispatched, request: pg_info(1 pgs e194471:22.1b8) v4 -2225> 2014-08-13 17:22:49.890917 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890917, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1b8) v4 -2224> 2014-08-13 17:22:49.890929 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890928, event: started, request: pg_info(1 pgs e194471:22.1b8) v4 -2223> 2014-08-13 17:22:49.890957 7f74d8735700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:22:49.890957, event: done, request: pg_info(1 pgs e194471:22.1b8) v4 -2222> 2014-08-13 17:22:49.893246 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 8 ==== pg_info(1 pgs e194471:22.619) v4 ==== 714+0+0 (4247008137 0 0) 0x53bbd40 con 0x4d51a20 -2221> 2014-08-13 17:22:49.893287 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893081, event: header_read, request: pg_info(1 pgs e194471:22.619) v4 -2220> 2014-08-13 17:22:49.893302 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893085, event: throttled, request: pg_info(1 pgs e194471:22.619) v4 -2219> 2014-08-13 17:22:49.893312 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893206, event: all_read, request: pg_info(1 pgs e194471:22.619) v4 -2218> 2014-08-13 17:22:49.893325 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893281, event: dispatched, request: pg_info(1 pgs e194471:22.619) v4 -2217> 2014-08-13 17:22:49.893336 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893336, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.619) v4 -2216> 2014-08-13 17:22:49.893348 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893348, event: started, request: pg_info(1 pgs e194471:22.619) v4 -2215> 2014-08-13 17:22:49.893403 7f74d8735700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:22:49.893403, event: done, request: pg_info(1 pgs e194471:22.619) v4 -2214> 2014-08-13 17:22:49.897583 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 8 ==== pg_info(1 pgs e194471:22.84) v4 ==== 714+0+0 (555736679 0 0) 0x53ba140 con 0x45f3b20 -2213> 2014-08-13 17:22:49.897620 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897456, event: header_read, request: pg_info(1 pgs e194471:22.84) v4 -2212> 2014-08-13 17:22:49.897635 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897459, event: throttled, request: pg_info(1 pgs e194471:22.84) v4 -2211> 2014-08-13 17:22:49.897645 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897547, event: all_read, request: pg_info(1 pgs e194471:22.84) v4 -2210> 2014-08-13 17:22:49.897654 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897615, event: dispatched, request: pg_info(1 pgs e194471:22.84) v4 -2209> 2014-08-13 17:22:49.897666 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897665, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.84) v4 -2208> 2014-08-13 17:22:49.897679 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897678, event: started, request: pg_info(1 pgs e194471:22.84) v4 -2207> 2014-08-13 17:22:49.897715 7f74d8735700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:22:49.897715, event: done, request: pg_info(1 pgs e194471:22.84) v4 -2206> 2014-08-13 17:22:49.905000 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.347 192.168.1.106:6800/4640 6 ==== pg_info(1 pgs e194471:22.1b8) v4 ==== 714+0+0 (3801716547 0 0) 0x57d7e00 con 0x4831600 -2205> 2014-08-13 17:22:49.905045 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.904837, event: header_read, request: pg_info(1 pgs e194471:22.1b8) v4 -2204> 2014-08-13 17:22:49.905056 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.904839, event: throttled, request: pg_info(1 pgs e194471:22.1b8) v4 -2203> 2014-08-13 17:22:49.905063 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.904935, event: all_read, request: pg_info(1 pgs e194471:22.1b8) v4 -2202> 2014-08-13 17:22:49.905068 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.905038, event: dispatched, request: pg_info(1 pgs e194471:22.1b8) v4 -2201> 2014-08-13 17:22:49.905075 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.905074, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1b8) v4 -2200> 2014-08-13 17:22:49.905084 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.905083, event: started, request: pg_info(1 pgs e194471:22.1b8) v4 -2199> 2014-08-13 17:22:49.905118 7f74d8735700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:22:49.905117, event: done, request: pg_info(1 pgs e194471:22.1b8) v4 -2198> 2014-08-13 17:22:49.908698 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 9 ==== pg_info(1 pgs e194471:22.6bd) v4 ==== 714+0+0 (2887009621 0 0) 0x4ac7e00 con 0x47f70c0 -2197> 2014-08-13 17:22:49.908743 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908535, event: header_read, request: pg_info(1 pgs e194471:22.6bd) v4 -2196> 2014-08-13 17:22:49.908754 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908539, event: throttled, request: pg_info(1 pgs e194471:22.6bd) v4 -2195> 2014-08-13 17:22:49.908760 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908618, event: all_read, request: pg_info(1 pgs e194471:22.6bd) v4 -2194> 2014-08-13 17:22:49.908766 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908736, event: dispatched, request: pg_info(1 pgs e194471:22.6bd) v4 -2193> 2014-08-13 17:22:49.908773 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908773, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.6bd) v4 -2192> 2014-08-13 17:22:49.908782 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908782, event: started, request: pg_info(1 pgs e194471:22.6bd) v4 -2191> 2014-08-13 17:22:49.908816 7f74d8735700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:22:49.908816, event: done, request: pg_info(1 pgs e194471:22.6bd) v4 -2190> 2014-08-13 17:22:49.917632 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 9 ==== pg_info(1 pgs e194471:22.362) v4 ==== 714+0+0 (2151035818 0 0) 0x53bf1c0 con 0x45f1e40 -2189> 2014-08-13 17:22:49.917679 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917426, event: header_read, request: pg_info(1 pgs e194471:22.362) v4 -2188> 2014-08-13 17:22:49.917695 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917429, event: throttled, request: pg_info(1 pgs e194471:22.362) v4 -2187> 2014-08-13 17:22:49.917701 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917521, event: all_read, request: pg_info(1 pgs e194471:22.362) v4 -2186> 2014-08-13 17:22:49.917707 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917673, event: dispatched, request: pg_info(1 pgs e194471:22.362) v4 -2185> 2014-08-13 17:22:49.917714 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917713, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.362) v4 -2184> 2014-08-13 17:22:49.917726 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917726, event: started, request: pg_info(1 pgs e194471:22.362) v4 -2183> 2014-08-13 17:22:49.917769 7f74d8735700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:22:49.917769, event: done, request: pg_info(1 pgs e194471:22.362) v4 -2182> 2014-08-13 17:22:49.924767 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194471:22.362) v4 -- ?+0 0x57d7000 -2181> 2014-08-13 17:22:49.924827 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194471:22.362) v4 -- ?+0 0x57d7540 -2180> 2014-08-13 17:22:49.924891 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194471:22.99) v4 -- ?+0 0x57d4980 -2179> 2014-08-13 17:22:49.924917 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194471:22.99) v4 -- ?+0 0x57d1340 -2178> 2014-08-13 17:22:49.925070 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194471:22.84) v4 -- ?+0 0x57d7a80 -2177> 2014-08-13 17:22:49.925128 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194471:22.84) v4 -- ?+0 0x57d3480 -2176> 2014-08-13 17:22:49.925244 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.096142 9 0.021611 -2175> 2014-08-13 17:22:49.925253 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.096237 9 0.021649 -2174> 2014-08-13 17:22:49.925289 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2173> 2014-08-13 17:22:49.925301 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/194470) [4,58,343] r=0 lpr=194470 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2172> 2014-08-13 17:22:49.925311 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.301 192.168.1.113:6814/16727 -- pg_info(1 pgs e194471:22.22c) v4 -- ?+0 0x57d7380 -2171> 2014-08-13 17:22:49.925367 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.324 192.168.1.104:6814/65209 -- pg_info(1 pgs e194471:22.22c) v4 -- ?+0 0x5263d40 -2170> 2014-08-13 17:22:49.925453 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.095871 8 0.000231 -2169> 2014-08-13 17:22:49.925478 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -2168> 2014-08-13 17:22:49.925499 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194471:22.4df) v4 -- ?+0 0x53bcec0 -2167> 2014-08-13 17:22:49.925507 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000030 0 0.000000 -2166> 2014-08-13 17:22:49.925526 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.357 192.168.1.102:6834/17224 -- pg_info(1 pgs e194471:22.4df) v4 -- ?+0 0x53be580 -2165> 2014-08-13 17:22:49.925532 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194457 194470/194470/194470) [4,28,304] r=0 lpr=194470 pi=194448-194469/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -2164> 2014-08-13 17:22:49.925565 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194471:22.84) v4 -- ?+0 0x5264600 -2163> 2014-08-13 17:22:49.925595 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194471:22.84) v4 -- ?+0 0x4ac3480 -2162> 2014-08-13 17:22:49.925619 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.62 192.168.1.112:6824/5505 -- pg_info(1 pgs e194471:22.619) v4 -- ?+0 0x4ac4980 -2161> 2014-08-13 17:22:49.925651 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.65 192.168.1.111:6842/17345 -- pg_info(1 pgs e194471:22.619) v4 -- ?+0 0x4ac4b40 -2160> 2014-08-13 17:22:49.925749 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194471/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.095043 9 0.001548 -2159> 2014-08-13 17:22:49.925794 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194471/194465 194470/194470/194470) [4,301,324] r=0 lpr=194470 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2158> 2014-08-13 17:22:49.925834 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.000546 1 0.000086 -2157> 2014-08-13 17:22:49.925862 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved -2156> 2014-08-13 17:22:49.925889 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6831/41157 -- MRecoveryReserve REQUEST pgid: 22.99, query_epoch: 194471 v2 -- ?+0 0x53ba840 con 0x47f4620 -2155> 2014-08-13 17:22:49.926006 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194471; pg22.84; ) v2 -- ?+0 0x57d3480 -2154> 2014-08-13 17:22:49.926071 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194471:22.39d) v4 -- ?+0 0x4ac3b80 -2153> 2014-08-13 17:22:49.926104 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194471:22.39d) v4 -- ?+0 0x4ac3d40 -2152> 2014-08-13 17:22:49.926355 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.093665 9 0.001123 -2151> 2014-08-13 17:22:49.926399 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/194470) [4,373,344] r=0 lpr=194470 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2150> 2014-08-13 17:22:49.926454 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194471:22.f3) v4 -- ?+0 0x4ac7700 -2149> 2014-08-13 17:22:49.926458 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194471/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094969 8 0.002021 -2148> 2014-08-13 17:22:49.926482 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.312 192.168.1.113:6816/17521 -- pg_info(1 pgs e194471:22.f3) v4 -- ?+0 0x4ac0c40 -2147> 2014-08-13 17:22:49.926486 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194471/194466 194470/194470/194470) [4,356,357] r=0 lpr=194470 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2146> 2014-08-13 17:22:49.926512 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194471/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094464 9 0.001012 -2145> 2014-08-13 17:22:49.926531 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194471/194466 194470/194470/194470) [4,65,62] r=0 lpr=194470 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2144> 2014-08-13 17:22:49.926782 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.093174 8 0.000911 -2143> 2014-08-13 17:22:49.926811 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 8 ==== MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 ==== 26+0+0 (3480433482 0 0) 0x53ba840 con 0x47f4620 -2142> 2014-08-13 17:22:49.926820 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,312,285] r=0 lpr=194470 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2141> 2014-08-13 17:22:49.926842 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926671, event: header_read, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2140> 2014-08-13 17:22:49.926853 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926677, event: throttled, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2139> 2014-08-13 17:22:49.926865 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926767, event: all_read, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2138> 2014-08-13 17:22:49.926871 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926837, event: dispatched, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2137> 2014-08-13 17:22:49.926882 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926881, event: waiting_for_osdmap, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2136> 2014-08-13 17:22:49.926918 7f74d8735700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:22:49.926918, event: done, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2135> 2014-08-13 17:22:49.926964 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- MRecoveryReserve REQUEST pgid: 22.99, query_epoch: 194471 v2 -- ?+0 0x53b8a80 con 0x486b9c0 -2134> 2014-08-13 17:22:49.927855 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.313 192.168.1.109:6820/35445 -- pg_info(1 pgs e194471:22.3e4) v4 -- ?+0 0x4ac4280 -2133> 2014-08-13 17:22:49.927906 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.318 192.168.1.110:6814/43857 -- pg_info(1 pgs e194471:22.3e4) v4 -- ?+0 0x4ac47c0 -2132> 2014-08-13 17:22:49.927992 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 8 ==== MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 ==== 26+0+0 (3480433482 0 0) 0x53b8a80 con 0x486b9c0 -2131> 2014-08-13 17:22:49.928047 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.927857, event: header_read, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2130> 2014-08-13 17:22:49.928062 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.927858, event: throttled, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2129> 2014-08-13 17:22:49.928072 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.927922, event: all_read, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2128> 2014-08-13 17:22:49.928081 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.928041, event: dispatched, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2127> 2014-08-13 17:22:49.928092 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.928092, event: waiting_for_osdmap, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2126> 2014-08-13 17:22:49.928120 7f74d8735700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:22:49.928120, event: done, request: MRecoveryReserve GRANT pgid: 22.99, query_epoch: 194471 v2 -2125> 2014-08-13 17:22:49.928139 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.002277 2 0.000137 -2124> 2014-08-13 17:22:49.928187 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/194470) [4,374,31] r=0 lpr=194470 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/Recovering -2123> 2014-08-13 17:22:49.928299 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194471/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.091841 9 0.001370 -2122> 2014-08-13 17:22:49.928356 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194471/194466 194470/194470/194470) [4,313,318] r=0 lpr=194470 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2121> 2014-08-13 17:22:49.929876 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194471:22.113) v4 -- ?+0 0x53bc600 -2120> 2014-08-13 17:22:49.929925 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.339 192.168.1.110:6818/45268 -- pg_info(1 pgs e194471:22.113) v4 -- ?+0 0x53bf8c0 -2119> 2014-08-13 17:22:49.930266 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194471/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.087734 9 0.001056 -2118> 2014-08-13 17:22:49.930307 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194471/194466 194470/194470/194470) [4,339,45] r=0 lpr=194470 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2117> 2014-08-13 17:22:49.932419 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194471:22.6bd) v4 -- ?+0 0x53bf000 -2116> 2014-08-13 17:22:49.932463 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194471:22.6bd) v4 -- ?+0 0x4ac1500 -2115> 2014-08-13 17:22:49.932592 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.084673 9 0.000242 -2114> 2014-08-13 17:22:49.932627 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -2113> 2014-08-13 17:22:49.932654 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000027 0 0.000000 -2112> 2014-08-13 17:22:49.932677 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194457 194470/194470/194470) [4,50,289] r=0 lpr=194470 pi=194448-194469/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -2111> 2014-08-13 17:22:49.932712 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194471:22.6bd) v4 -- ?+0 0x53bc0c0 -2110> 2014-08-13 17:22:49.932743 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194471:22.6bd) v4 -- ?+0 0x53bbf00 -2109> 2014-08-13 17:22:49.933056 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.32 192.168.1.115:6812/59915 -- osd pg remove(epoch 194471; pg22.6bd; ) v2 -- ?+0 0x4ac2bc0 -2108> 2014-08-13 17:22:49.933643 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194471:22.3d5) v4 -- ?+0 0x4ac24c0 -2107> 2014-08-13 17:22:49.933674 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194471:22.3d5) v4 -- ?+0 0x4ac1180 -2106> 2014-08-13 17:22:49.933826 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.077686 9 0.000230 -2105> 2014-08-13 17:22:49.933866 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -2104> 2014-08-13 17:22:49.933890 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000025 0 0.000000 -2103> 2014-08-13 17:22:49.933910 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194457 194470/194470/194470) [4,343,286] r=0 lpr=194470 pi=194448-194469/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -2102> 2014-08-13 17:22:49.933940 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194471:22.3d5) v4 -- ?+0 0x53bee40 -2101> 2014-08-13 17:22:49.933984 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194471:22.3d5) v4 -- ?+0 0x53b88c0 -2100> 2014-08-13 17:22:49.934126 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:22.1b8) v4 -- ?+0 0x4ac1340 -2099> 2014-08-13 17:22:49.934153 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194471:22.1b8) v4 -- ?+0 0x4ac0e00 -2098> 2014-08-13 17:22:49.934294 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.077722 8 0.000190 -2097> 2014-08-13 17:22:49.934321 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -2096> 2014-08-13 17:22:49.934341 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000020 0 0.000000 -2095> 2014-08-13 17:22:49.934365 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194457 194470/194470/194470) [4,340,347] r=0 lpr=194470 pi=194448-194469/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -2094> 2014-08-13 17:22:49.934360 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.10 192.168.1.115:6815/60573 -- osd pg remove(epoch 194471; pg22.3d5; ) v2 -- ?+0 0x4ac0000 -2093> 2014-08-13 17:22:49.934426 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:22.1b8) v4 -- ?+0 0x53b8e00 -2092> 2014-08-13 17:22:49.934463 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194471:22.1b8) v4 -- ?+0 0x53b8540 -2091> 2014-08-13 17:22:49.934638 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.338 192.168.1.106:6814/25893 -- pg_info(1 pgs e194471:22.3d9) v4 -- ?+0 0x4ac7c40 -2090> 2014-08-13 17:22:49.934678 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194471:22.3d9) v4 -- ?+0 0x4ac78c0 -2089> 2014-08-13 17:22:49.934805 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194471; pg22.1b8; ) v2 -- ?+0 0x57d7540 -2088> 2014-08-13 17:22:49.934994 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194471/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.077027 13 0.000655 -2087> 2014-08-13 17:22:49.935033 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194471/194389 194470/194470/194470) [4,338,376] r=0 lpr=194470 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2086> 2014-08-13 17:22:49.935365 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.51 192.168.1.112:6832/6655 -- pg_info(1 pgs e194471:22.537) v4 -- ?+0 0x4ac3f00 -2085> 2014-08-13 17:22:49.935533 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.351 192.168.1.102:6830/16444 -- pg_info(1 pgs e194471:22.220) v4 -- ?+0 0x4ac7000 -2084> 2014-08-13 17:22:49.936058 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194471:22.48e) v4 -- ?+0 0x4ac7a80 -2083> 2014-08-13 17:22:49.936096 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.341 192.168.1.106:6818/9142 -- pg_info(1 pgs e194471:22.48e) v4 -- ?+0 0x4ac40c0 -2082> 2014-08-13 17:22:49.936238 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.334 192.168.1.104:6808/62751 -- pg_info(1 pgs e194471:22.69f) v4 -- ?+0 0x4ac6e40 -2081> 2014-08-13 17:22:49.936418 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194471:22.639) v4 -- ?+0 0x4ac0a80 -2080> 2014-08-13 17:22:49.936483 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194471/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.076216 9 0.001227 -2079> 2014-08-13 17:22:49.936522 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194471/194466 194470/194470/194470) [4,289,341] r=0 lpr=194470 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -2078> 2014-08-13 17:22:49.937189 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.292 192.168.1.113:6812/16056 -- pg_info(1 pgs e194471:22.3ad) v4 -- ?+0 0x4ac2a00 -2077> 2014-08-13 17:22:49.937327 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.323 192.168.1.106:6816/8118 -- pg_info(1 pgs e194471:22.12a) v4 -- ?+0 0x4ac1180 -2076> 2014-08-13 17:22:49.937477 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 7 ==== pg_info(1 pgs e194471:22.639) v4 ==== 714+0+0 (2105292953 0 0) 0x4ac0a80 con 0x486a940 -2075> 2014-08-13 17:22:49.937516 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937011, event: header_read, request: pg_info(1 pgs e194471:22.639) v4 -2074> 2014-08-13 17:22:49.937527 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937013, event: throttled, request: pg_info(1 pgs e194471:22.639) v4 -2073> 2014-08-13 17:22:49.937541 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937093, event: all_read, request: pg_info(1 pgs e194471:22.639) v4 -2072> 2014-08-13 17:22:49.937552 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937510, event: dispatched, request: pg_info(1 pgs e194471:22.639) v4 -2071> 2014-08-13 17:22:49.937560 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937560, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.639) v4 -2070> 2014-08-13 17:22:49.937573 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937572, event: started, request: pg_info(1 pgs e194471:22.639) v4 -2069> 2014-08-13 17:22:49.937607 7f74d8735700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:22:49.937607, event: done, request: pg_info(1 pgs e194471:22.639) v4 -2068> 2014-08-13 17:22:49.937630 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 5 ==== pg_info(1 pgs e194471:22.537) v4 ==== 714+0+0 (676658711 0 0) 0x4ac3f00 con 0x4831b80 -2067> 2014-08-13 17:22:49.937650 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937133, event: header_read, request: pg_info(1 pgs e194471:22.537) v4 -2066> 2014-08-13 17:22:49.937660 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937135, event: throttled, request: pg_info(1 pgs e194471:22.537) v4 -2065> 2014-08-13 17:22:49.937669 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937217, event: all_read, request: pg_info(1 pgs e194471:22.537) v4 -2064> 2014-08-13 17:22:49.937678 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937647, event: dispatched, request: pg_info(1 pgs e194471:22.537) v4 -2063> 2014-08-13 17:22:49.937688 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937688, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.537) v4 -2062> 2014-08-13 17:22:49.937699 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937699, event: started, request: pg_info(1 pgs e194471:22.537) v4 -2061> 2014-08-13 17:22:49.937989 7f74d8735700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:22:49.937988, event: done, request: pg_info(1 pgs e194471:22.537) v4 -2060> 2014-08-13 17:22:49.938006 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 5 ==== pg_info(1 pgs e194471:22.69f) v4 ==== 714+0+0 (442245028 0 0) 0x4ac6e40 con 0x486aec0 -2059> 2014-08-13 17:22:49.938021 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.937027, event: header_read, request: pg_info(1 pgs e194471:22.69f) v4 -2058> 2014-08-13 17:22:49.938027 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.937029, event: throttled, request: pg_info(1 pgs e194471:22.69f) v4 -2057> 2014-08-13 17:22:49.938033 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.937137, event: all_read, request: pg_info(1 pgs e194471:22.69f) v4 -2056> 2014-08-13 17:22:49.938038 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.938019, event: dispatched, request: pg_info(1 pgs e194471:22.69f) v4 -2055> 2014-08-13 17:22:49.938043 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.938043, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.69f) v4 -2054> 2014-08-13 17:22:49.938051 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.938051, event: started, request: pg_info(1 pgs e194471:22.69f) v4 -2053> 2014-08-13 17:22:49.938070 7f74d8735700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:22:49.938070, event: done, request: pg_info(1 pgs e194471:22.69f) v4 -2052> 2014-08-13 17:22:49.938083 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 8 ==== pg_info(1 pgs e194471:22.639) v4 ==== 714+0+0 (3830226410 0 0) 0x4ac2840 con 0x486a940 -2051> 2014-08-13 17:22:49.938096 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.937341, event: header_read, request: pg_info(1 pgs e194471:22.639) v4 -2050> 2014-08-13 17:22:49.938102 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.937343, event: throttled, request: pg_info(1 pgs e194471:22.639) v4 -2049> 2014-08-13 17:22:49.938107 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.937430, event: all_read, request: pg_info(1 pgs e194471:22.639) v4 -2048> 2014-08-13 17:22:49.938112 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.938094, event: dispatched, request: pg_info(1 pgs e194471:22.639) v4 -2047> 2014-08-13 17:22:49.938118 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.938118, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.639) v4 -2046> 2014-08-13 17:22:49.938125 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.938125, event: started, request: pg_info(1 pgs e194471:22.639) v4 -2045> 2014-08-13 17:22:49.938142 7f74d8735700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:22:49.938142, event: done, request: pg_info(1 pgs e194471:22.639) v4 -2044> 2014-08-13 17:22:49.938157 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 6 ==== pg_info(1 pgs e194471:22.537) v4 ==== 714+0+0 (933045133 0 0) 0x53bd400 con 0x4831b80 -2043> 2014-08-13 17:22:49.938200 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.937243, event: header_read, request: pg_info(1 pgs e194471:22.537) v4 -2042> 2014-08-13 17:22:49.938208 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.937244, event: throttled, request: pg_info(1 pgs e194471:22.537) v4 -2041> 2014-08-13 17:22:49.938213 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.937300, event: all_read, request: pg_info(1 pgs e194471:22.537) v4 -2040> 2014-08-13 17:22:49.938218 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.938198, event: dispatched, request: pg_info(1 pgs e194471:22.537) v4 -2039> 2014-08-13 17:22:49.938224 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.938224, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.537) v4 -2038> 2014-08-13 17:22:49.938232 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.938231, event: started, request: pg_info(1 pgs e194471:22.537) v4 -2037> 2014-08-13 17:22:49.938251 7f74d8735700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:22:49.938250, event: done, request: pg_info(1 pgs e194471:22.537) v4 -2036> 2014-08-13 17:22:49.938265 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 5 ==== pg_info(1 pgs e194471:22.3ad) v4 ==== 714+0+0 (213971359 0 0) 0x4ac2a00 con 0x4868160 -2035> 2014-08-13 17:22:49.938280 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938029, event: header_read, request: pg_info(1 pgs e194471:22.3ad) v4 -2034> 2014-08-13 17:22:49.938286 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938030, event: throttled, request: pg_info(1 pgs e194471:22.3ad) v4 -2033> 2014-08-13 17:22:49.938291 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938133, event: all_read, request: pg_info(1 pgs e194471:22.3ad) v4 -2032> 2014-08-13 17:22:49.938296 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938278, event: dispatched, request: pg_info(1 pgs e194471:22.3ad) v4 -2031> 2014-08-13 17:22:49.938302 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938301, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3ad) v4 -2030> 2014-08-13 17:22:49.938309 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938309, event: started, request: pg_info(1 pgs e194471:22.3ad) v4 -2029> 2014-08-13 17:22:49.938327 7f74d8735700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:22:49.938327, event: done, request: pg_info(1 pgs e194471:22.3ad) v4 -2028> 2014-08-13 17:22:49.938340 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 5 ==== pg_info(1 pgs e194471:22.12a) v4 ==== 714+0+0 (1097306860 0 0) 0x4ac1180 con 0x486a680 -2027> 2014-08-13 17:22:49.938353 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938175, event: header_read, request: pg_info(1 pgs e194471:22.12a) v4 -2026> 2014-08-13 17:22:49.938360 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938177, event: throttled, request: pg_info(1 pgs e194471:22.12a) v4 -2025> 2014-08-13 17:22:49.938365 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938257, event: all_read, request: pg_info(1 pgs e194471:22.12a) v4 -2024> 2014-08-13 17:22:49.938370 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938352, event: dispatched, request: pg_info(1 pgs e194471:22.12a) v4 -2023> 2014-08-13 17:22:49.938376 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938375, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.12a) v4 -2022> 2014-08-13 17:22:49.938382 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938382, event: started, request: pg_info(1 pgs e194471:22.12a) v4 -2021> 2014-08-13 17:22:49.938372 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.54 192.168.1.111:6836/14923 -- pg_info(1 pgs e194471:22.30f) v4 -- ?+0 0x466a680 -2020> 2014-08-13 17:22:49.938402 7f74d8735700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:22:49.938402, event: done, request: pg_info(1 pgs e194471:22.12a) v4 -2019> 2014-08-13 17:22:49.938417 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.334 192.168.1.104:6808/62751 6 ==== pg_info(1 pgs e194471:22.69f) v4 ==== 714+0+0 (3323591065 0 0) 0x4f516c0 con 0x486aec0 -2018> 2014-08-13 17:22:49.938430 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938209, event: header_read, request: pg_info(1 pgs e194471:22.69f) v4 -2017> 2014-08-13 17:22:49.938436 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938211, event: throttled, request: pg_info(1 pgs e194471:22.69f) v4 -2016> 2014-08-13 17:22:49.938442 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938289, event: all_read, request: pg_info(1 pgs e194471:22.69f) v4 -2015> 2014-08-13 17:22:49.938446 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938428, event: dispatched, request: pg_info(1 pgs e194471:22.69f) v4 -2014> 2014-08-13 17:22:49.938453 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938452, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.69f) v4 -2013> 2014-08-13 17:22:49.938460 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938459, event: started, request: pg_info(1 pgs e194471:22.69f) v4 -2012> 2014-08-13 17:22:49.938478 7f74d8735700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:22:49.938477, event: done, request: pg_info(1 pgs e194471:22.69f) v4 -2011> 2014-08-13 17:22:49.938563 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194471:22.2b9) v4 -- ?+0 0x466abc0 -2010> 2014-08-13 17:22:49.938690 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194471:22.29a) v4 -- ?+0 0x466fc40 -2009> 2014-08-13 17:22:49.938781 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.323 192.168.1.106:6816/8118 6 ==== pg_info(1 pgs e194471:22.12a) v4 ==== 714+0+0 (2814808911 0 0) 0x53be040 con 0x486a680 -2008> 2014-08-13 17:22:49.938739 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.294 192.168.1.101:6812/23402 -- pg_info(1 pgs e194471:22.630) v4 -- ?+0 0x466ee40 -2007> 2014-08-13 17:22:49.938808 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938595, event: header_read, request: pg_info(1 pgs e194471:22.12a) v4 -2006> 2014-08-13 17:22:49.938817 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938597, event: throttled, request: pg_info(1 pgs e194471:22.12a) v4 -2005> 2014-08-13 17:22:49.938822 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938680, event: all_read, request: pg_info(1 pgs e194471:22.12a) v4 -2004> 2014-08-13 17:22:49.938827 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938804, event: dispatched, request: pg_info(1 pgs e194471:22.12a) v4 -2003> 2014-08-13 17:22:49.938833 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938832, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.12a) v4 -2002> 2014-08-13 17:22:49.938840 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938840, event: started, request: pg_info(1 pgs e194471:22.12a) v4 -2001> 2014-08-13 17:22:49.938862 7f74d8735700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:22:49.938862, event: done, request: pg_info(1 pgs e194471:22.12a) v4 -2000> 2014-08-13 17:22:49.938867 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.321 192.168.1.110:6812/43236 -- pg_info(1 pgs e194471:22.4aa) v4 -- ?+0 0x53bbf00 -1999> 2014-08-13 17:22:49.938973 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.358 192.168.1.114:6824/52078 -- pg_info(1 pgs e194471:22.2d1) v4 -- ?+0 0x53bfa80 -1998> 2014-08-13 17:22:49.939054 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 6 ==== pg_info(1 pgs e194471:22.3ad) v4 ==== 714+0+0 (1052399924 0 0) 0x53bd940 con 0x4868160 -1997> 2014-08-13 17:22:49.939079 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.938912, event: header_read, request: pg_info(1 pgs e194471:22.3ad) v4 -1996> 2014-08-13 17:22:49.939088 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.938913, event: throttled, request: pg_info(1 pgs e194471:22.3ad) v4 -1995> 2014-08-13 17:22:49.939093 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.938981, event: all_read, request: pg_info(1 pgs e194471:22.3ad) v4 -1994> 2014-08-13 17:22:49.939098 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.939076, event: dispatched, request: pg_info(1 pgs e194471:22.3ad) v4 -1993> 2014-08-13 17:22:49.939016 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.9 192.168.1.107:6817/53997 -- pg_info(1 pgs e194471:22.28f) v4 -- ?+0 0x4ac1500 -1992> 2014-08-13 17:22:49.939104 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.939104, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3ad) v4 -1991> 2014-08-13 17:22:49.939113 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.939113, event: started, request: pg_info(1 pgs e194471:22.3ad) v4 -1990> 2014-08-13 17:22:49.939135 7f74d8735700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:22:49.939134, event: done, request: pg_info(1 pgs e194471:22.3ad) v4 -1989> 2014-08-13 17:22:49.939149 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 8 ==== pg_info(1 pgs e194471:22.30f) v4 ==== 714+0+0 (3656851526 0 0) 0x466a680 con 0x45f6f60 -1988> 2014-08-13 17:22:49.939163 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.938935, event: header_read, request: pg_info(1 pgs e194471:22.30f) v4 -1987> 2014-08-13 17:22:49.939179 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.938937, event: throttled, request: pg_info(1 pgs e194471:22.30f) v4 -1986> 2014-08-13 17:22:49.939185 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.939024, event: all_read, request: pg_info(1 pgs e194471:22.30f) v4 -1985> 2014-08-13 17:22:49.939190 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.939161, event: dispatched, request: pg_info(1 pgs e194471:22.30f) v4 -1984> 2014-08-13 17:22:49.939196 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.939196, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.30f) v4 -1983> 2014-08-13 17:22:49.939203 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.939203, event: started, request: pg_info(1 pgs e194471:22.30f) v4 -1982> 2014-08-13 17:22:49.939224 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.371 192.168.1.102:6832/16753 -- pg_info(1 pgs e194471:22.653) v4 -- ?+0 0x57d5780 -1981> 2014-08-13 17:22:49.939322 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.320 192.168.1.106:6820/10050 -- pg_info(1 pgs e194471:22.581) v4 -- ?+0 0x57d4b40 -1980> 2014-08-13 17:22:49.939483 7f74d8735700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:22:49.939483, event: done, request: pg_info(1 pgs e194471:22.30f) v4 -1979> 2014-08-13 17:22:49.939512 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 7 ==== pg_info(1 pgs e194471:22.29a) v4 ==== 714+0+0 (1099454954 0 0) 0x466fc40 con 0x475c4c0 -1978> 2014-08-13 17:22:49.939530 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939162, event: header_read, request: pg_info(1 pgs e194471:22.29a) v4 -1977> 2014-08-13 17:22:49.939536 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939164, event: throttled, request: pg_info(1 pgs e194471:22.29a) v4 -1976> 2014-08-13 17:22:49.939541 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939243, event: all_read, request: pg_info(1 pgs e194471:22.29a) v4 -1975> 2014-08-13 17:22:49.939546 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939527, event: dispatched, request: pg_info(1 pgs e194471:22.29a) v4 -1974> 2014-08-13 17:22:49.939552 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939552, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.29a) v4 -1973> 2014-08-13 17:22:49.939560 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939560, event: started, request: pg_info(1 pgs e194471:22.29a) v4 -1972> 2014-08-13 17:22:49.939581 7f74d8735700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:22:49.939581, event: done, request: pg_info(1 pgs e194471:22.29a) v4 -1971> 2014-08-13 17:22:49.939598 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 8 ==== pg_info(1 pgs e194471:22.29a) v4 ==== 714+0+0 (1567671877 0 0) 0x57d6900 con 0x475c4c0 -1970> 2014-08-13 17:22:49.939612 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939458, event: header_read, request: pg_info(1 pgs e194471:22.29a) v4 -1969> 2014-08-13 17:22:49.939618 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939460, event: throttled, request: pg_info(1 pgs e194471:22.29a) v4 -1968> 2014-08-13 17:22:49.939625 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939547, event: all_read, request: pg_info(1 pgs e194471:22.29a) v4 -1967> 2014-08-13 17:22:49.939630 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939609, event: dispatched, request: pg_info(1 pgs e194471:22.29a) v4 -1966> 2014-08-13 17:22:49.939636 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939636, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.29a) v4 -1965> 2014-08-13 17:22:49.939643 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939643, event: started, request: pg_info(1 pgs e194471:22.29a) v4 -1964> 2014-08-13 17:22:49.939660 7f74d8735700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:22:49.939660, event: done, request: pg_info(1 pgs e194471:22.29a) v4 -1963> 2014-08-13 17:22:49.939673 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 7 ==== pg_info(1 pgs e194471:22.2b9) v4 ==== 714+0+0 (759561095 0 0) 0x466abc0 con 0x4758b00 -1962> 2014-08-13 17:22:49.939686 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939469, event: header_read, request: pg_info(1 pgs e194471:22.2b9) v4 -1961> 2014-08-13 17:22:49.939692 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939471, event: throttled, request: pg_info(1 pgs e194471:22.2b9) v4 -1960> 2014-08-13 17:22:49.939698 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939617, event: all_read, request: pg_info(1 pgs e194471:22.2b9) v4 -1959> 2014-08-13 17:22:49.939702 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939684, event: dispatched, request: pg_info(1 pgs e194471:22.2b9) v4 -1958> 2014-08-13 17:22:49.939708 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939708, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2b9) v4 -1957> 2014-08-13 17:22:49.939715 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939715, event: started, request: pg_info(1 pgs e194471:22.2b9) v4 -1956> 2014-08-13 17:22:49.939885 7f74d8735700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:22:49.939884, event: done, request: pg_info(1 pgs e194471:22.2b9) v4 -1955> 2014-08-13 17:22:49.939934 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 9 ==== pg_info(1 pgs e194471:22.30f) v4 ==== 714+0+0 (1250162245 0 0) 0x53bf380 con 0x45f6f60 -1954> 2014-08-13 17:22:49.939962 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.939607, event: header_read, request: pg_info(1 pgs e194471:22.30f) v4 -1953> 2014-08-13 17:22:49.939975 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.939609, event: throttled, request: pg_info(1 pgs e194471:22.30f) v4 -1952> 2014-08-13 17:22:49.939984 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.939688, event: all_read, request: pg_info(1 pgs e194471:22.30f) v4 -1951> 2014-08-13 17:22:49.939994 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.939959, event: dispatched, request: pg_info(1 pgs e194471:22.30f) v4 -1950> 2014-08-13 17:22:49.940004 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.940004, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.30f) v4 -1949> 2014-08-13 17:22:49.940016 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.940016, event: started, request: pg_info(1 pgs e194471:22.30f) v4 -1948> 2014-08-13 17:22:49.939964 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.397 192.168.1.105:6808/41085 -- pg_info(1 pgs e194471:22.3b9) v4 -- ?+0 0x4ac0a80 -1947> 2014-08-13 17:22:49.940044 7f74d8735700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:22:49.940044, event: done, request: pg_info(1 pgs e194471:22.30f) v4 -1946> 2014-08-13 17:22:49.940063 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 5 ==== pg_info(1 pgs e194471:22.630) v4 ==== 714+0+0 (2396647812 0 0) 0x466ee40 con 0x475d960 -1945> 2014-08-13 17:22:49.940079 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.939754, event: header_read, request: pg_info(1 pgs e194471:22.630) v4 -1944> 2014-08-13 17:22:49.940086 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.939756, event: throttled, request: pg_info(1 pgs e194471:22.630) v4 -1943> 2014-08-13 17:22:49.940091 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.939830, event: all_read, request: pg_info(1 pgs e194471:22.630) v4 -1942> 2014-08-13 17:22:49.940097 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.940077, event: dispatched, request: pg_info(1 pgs e194471:22.630) v4 -1941> 2014-08-13 17:22:49.940102 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.940102, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.630) v4 -1940> 2014-08-13 17:22:49.940110 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.940110, event: started, request: pg_info(1 pgs e194471:22.630) v4 -1939> 2014-08-13 17:22:49.940128 7f74d8735700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:22:49.940128, event: done, request: pg_info(1 pgs e194471:22.630) v4 -1938> 2014-08-13 17:22:49.940140 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 5 ==== pg_info(1 pgs e194471:22.2d1) v4 ==== 714+0+0 (3429787707 0 0) 0x53bfa80 con 0x47f2680 -1937> 2014-08-13 17:22:49.940161 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.939684, event: header_read, request: pg_info(1 pgs e194471:22.2d1) v4 -1936> 2014-08-13 17:22:49.940168 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.939686, event: throttled, request: pg_info(1 pgs e194471:22.2d1) v4 -1935> 2014-08-13 17:22:49.940173 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.939830, event: all_read, request: pg_info(1 pgs e194471:22.2d1) v4 -1934> 2014-08-13 17:22:49.940178 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.940159, event: dispatched, request: pg_info(1 pgs e194471:22.2d1) v4 -1933> 2014-08-13 17:22:49.940188 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.940188, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2d1) v4 -1932> 2014-08-13 17:22:49.940199 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.940199, event: started, request: pg_info(1 pgs e194471:22.2d1) v4 -1931> 2014-08-13 17:22:49.940233 7f74d8735700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:22:49.940233, event: done, request: pg_info(1 pgs e194471:22.2d1) v4 -1930> 2014-08-13 17:22:49.940255 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.358 192.168.1.114:6824/52078 6 ==== pg_info(1 pgs e194471:22.2d1) v4 ==== 714+0+0 (1076493249 0 0) 0x53b9a40 con 0x47f2680 -1929> 2014-08-13 17:22:49.940274 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.939857, event: header_read, request: pg_info(1 pgs e194471:22.2d1) v4 -1928> 2014-08-13 17:22:49.940285 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.939859, event: throttled, request: pg_info(1 pgs e194471:22.2d1) v4 -1927> 2014-08-13 17:22:49.940294 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.939916, event: all_read, request: pg_info(1 pgs e194471:22.2d1) v4 -1926> 2014-08-13 17:22:49.940299 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.940272, event: dispatched, request: pg_info(1 pgs e194471:22.2d1) v4 -1925> 2014-08-13 17:22:49.940305 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.940305, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2d1) v4 -1924> 2014-08-13 17:22:49.940312 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.940312, event: started, request: pg_info(1 pgs e194471:22.2d1) v4 -1923> 2014-08-13 17:22:49.940336 7f74d8735700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:22:49.940336, event: done, request: pg_info(1 pgs e194471:22.2d1) v4 -1922> 2014-08-13 17:22:49.940355 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 5 ==== pg_info(1 pgs e194471:22.28f) v4 ==== 714+0+0 (3655388288 0 0) 0x4ac1500 con 0x45f2940 -1921> 2014-08-13 17:22:49.940374 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940129, event: header_read, request: pg_info(1 pgs e194471:22.28f) v4 -1920> 2014-08-13 17:22:49.940383 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940131, event: throttled, request: pg_info(1 pgs e194471:22.28f) v4 -1919> 2014-08-13 17:22:49.940392 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940222, event: all_read, request: pg_info(1 pgs e194471:22.28f) v4 -1918> 2014-08-13 17:22:49.940401 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940372, event: dispatched, request: pg_info(1 pgs e194471:22.28f) v4 -1917> 2014-08-13 17:22:49.940411 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940411, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.28f) v4 -1916> 2014-08-13 17:22:49.940422 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940421, event: started, request: pg_info(1 pgs e194471:22.28f) v4 -1915> 2014-08-13 17:22:49.940448 7f74d8735700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:22:49.940448, event: done, request: pg_info(1 pgs e194471:22.28f) v4 -1914> 2014-08-13 17:22:49.940466 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 6 ==== pg_info(1 pgs e194471:22.630) v4 ==== 714+0+0 (3519055889 0 0) 0x4ac3640 con 0x475d960 -1913> 2014-08-13 17:22:49.940484 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940250, event: header_read, request: pg_info(1 pgs e194471:22.630) v4 -1912> 2014-08-13 17:22:49.940491 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940251, event: throttled, request: pg_info(1 pgs e194471:22.630) v4 -1911> 2014-08-13 17:22:49.940496 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940338, event: all_read, request: pg_info(1 pgs e194471:22.630) v4 -1910> 2014-08-13 17:22:49.940501 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940482, event: dispatched, request: pg_info(1 pgs e194471:22.630) v4 -1909> 2014-08-13 17:22:49.940508 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940508, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.630) v4 -1908> 2014-08-13 17:22:49.940518 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940518, event: started, request: pg_info(1 pgs e194471:22.630) v4 -1907> 2014-08-13 17:22:49.940535 7f74d8735700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:22:49.940535, event: done, request: pg_info(1 pgs e194471:22.630) v4 -1906> 2014-08-13 17:22:49.940548 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 5 ==== pg_info(1 pgs e194471:22.4aa) v4 ==== 714+0+0 (4219531917 0 0) 0x53bbf00 con 0x475e5c0 -1905> 2014-08-13 17:22:49.940566 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940431, event: header_read, request: pg_info(1 pgs e194471:22.4aa) v4 -1904> 2014-08-13 17:22:49.940576 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940434, event: throttled, request: pg_info(1 pgs e194471:22.4aa) v4 -1903> 2014-08-13 17:22:49.940585 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940511, event: all_read, request: pg_info(1 pgs e194471:22.4aa) v4 -1902> 2014-08-13 17:22:49.940594 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940564, event: dispatched, request: pg_info(1 pgs e194471:22.4aa) v4 -1901> 2014-08-13 17:22:49.940601 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940600, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4aa) v4 -1900> 2014-08-13 17:22:49.940608 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940607, event: started, request: pg_info(1 pgs e194471:22.4aa) v4 -1899> 2014-08-13 17:22:49.940626 7f74d8735700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:22:49.940626, event: done, request: pg_info(1 pgs e194471:22.4aa) v4 -1898> 2014-08-13 17:22:49.940639 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 6 ==== pg_info(1 pgs e194471:22.4aa) v4 ==== 714+0+0 (1850842752 0 0) 0x49e01c0 con 0x475e5c0 -1897> 2014-08-13 17:22:49.940652 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940530, event: header_read, request: pg_info(1 pgs e194471:22.4aa) v4 -1896> 2014-08-13 17:22:49.940658 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940532, event: throttled, request: pg_info(1 pgs e194471:22.4aa) v4 -1895> 2014-08-13 17:22:49.940664 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940585, event: all_read, request: pg_info(1 pgs e194471:22.4aa) v4 -1894> 2014-08-13 17:22:49.940676 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940651, event: dispatched, request: pg_info(1 pgs e194471:22.4aa) v4 -1893> 2014-08-13 17:22:49.940682 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940682, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4aa) v4 -1892> 2014-08-13 17:22:49.940688 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940688, event: started, request: pg_info(1 pgs e194471:22.4aa) v4 -1891> 2014-08-13 17:22:49.940705 7f74d8735700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:22:49.940705, event: done, request: pg_info(1 pgs e194471:22.4aa) v4 -1890> 2014-08-13 17:22:49.940718 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 8 ==== pg_info(1 pgs e194471:22.2b9) v4 ==== 714+0+0 (1328834096 0 0) 0x53bc7c0 con 0x4758b00 -1889> 2014-08-13 17:22:49.940733 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940467, event: header_read, request: pg_info(1 pgs e194471:22.2b9) v4 -1888> 2014-08-13 17:22:49.940739 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940469, event: throttled, request: pg_info(1 pgs e194471:22.2b9) v4 -1887> 2014-08-13 17:22:49.940745 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940565, event: all_read, request: pg_info(1 pgs e194471:22.2b9) v4 -1886> 2014-08-13 17:22:49.940750 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940731, event: dispatched, request: pg_info(1 pgs e194471:22.2b9) v4 -1885> 2014-08-13 17:22:49.940761 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940761, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.2b9) v4 -1884> 2014-08-13 17:22:49.940772 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940772, event: started, request: pg_info(1 pgs e194471:22.2b9) v4 -1883> 2014-08-13 17:22:49.940793 7f74d8735700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:22:49.940793, event: done, request: pg_info(1 pgs e194471:22.2b9) v4 -1882> 2014-08-13 17:22:49.940810 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 12 ==== pg_info(1 pgs e194471:22.581) v4 ==== 714+0+0 (1402861230 0 0) 0x57d4b40 con 0x486bc80 -1881> 2014-08-13 17:22:49.940828 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940622, event: header_read, request: pg_info(1 pgs e194471:22.581) v4 -1880> 2014-08-13 17:22:49.940838 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940624, event: throttled, request: pg_info(1 pgs e194471:22.581) v4 -1879> 2014-08-13 17:22:49.940847 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940713, event: all_read, request: pg_info(1 pgs e194471:22.581) v4 -1878> 2014-08-13 17:22:49.940856 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940826, event: dispatched, request: pg_info(1 pgs e194471:22.581) v4 -1877> 2014-08-13 17:22:49.940866 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940866, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.581) v4 -1876> 2014-08-13 17:22:49.940876 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940876, event: started, request: pg_info(1 pgs e194471:22.581) v4 -1875> 2014-08-13 17:22:49.940898 7f74d8735700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:22:49.940898, event: done, request: pg_info(1 pgs e194471:22.581) v4 -1874> 2014-08-13 17:22:49.940914 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.9 192.168.1.107:6817/53997 6 ==== pg_info(1 pgs e194471:22.28f) v4 ==== 714+0+0 (111514552 0 0) 0x53bf540 con 0x45f2940 -1873> 2014-08-13 17:22:49.940932 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940653, event: header_read, request: pg_info(1 pgs e194471:22.28f) v4 -1872> 2014-08-13 17:22:49.940942 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940655, event: throttled, request: pg_info(1 pgs e194471:22.28f) v4 -1871> 2014-08-13 17:22:49.940951 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940718, event: all_read, request: pg_info(1 pgs e194471:22.28f) v4 -1870> 2014-08-13 17:22:49.940960 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940930, event: dispatched, request: pg_info(1 pgs e194471:22.28f) v4 -1869> 2014-08-13 17:22:49.940970 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940970, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.28f) v4 -1868> 2014-08-13 17:22:49.940980 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.940980, event: started, request: pg_info(1 pgs e194471:22.28f) v4 -1867> 2014-08-13 17:22:49.941002 7f74d8735700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:22:49.941002, event: done, request: pg_info(1 pgs e194471:22.28f) v4 -1866> 2014-08-13 17:22:49.941019 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 9 ==== pg_info(1 pgs e194471:22.220) v4 ==== 714+0+0 (1429371500 0 0) 0x4ac7000 con 0x4868420 -1865> 2014-08-13 17:22:49.941037 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.940719, event: header_read, request: pg_info(1 pgs e194471:22.220) v4 -1864> 2014-08-13 17:22:49.941046 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.940721, event: throttled, request: pg_info(1 pgs e194471:22.220) v4 -1863> 2014-08-13 17:22:49.941056 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.940804, event: all_read, request: pg_info(1 pgs e194471:22.220) v4 -1862> 2014-08-13 17:22:49.941064 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.941035, event: dispatched, request: pg_info(1 pgs e194471:22.220) v4 -1861> 2014-08-13 17:22:49.941074 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.941074, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.220) v4 -1860> 2014-08-13 17:22:49.941084 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.941084, event: started, request: pg_info(1 pgs e194471:22.220) v4 -1859> 2014-08-13 17:22:49.941106 7f74d8735700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:22:49.941105, event: done, request: pg_info(1 pgs e194471:22.220) v4 -1858> 2014-08-13 17:22:49.941122 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 7 ==== pg_info(1 pgs e194471:22.3b9) v4 ==== 714+0+0 (3560947847 0 0) 0x4ac0a80 con 0x48dfbc0 -1857> 2014-08-13 17:22:49.941139 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.940926, event: header_read, request: pg_info(1 pgs e194471:22.3b9) v4 -1856> 2014-08-13 17:22:49.941149 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.940928, event: throttled, request: pg_info(1 pgs e194471:22.3b9) v4 -1855> 2014-08-13 17:22:49.941158 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.940993, event: all_read, request: pg_info(1 pgs e194471:22.3b9) v4 -1854> 2014-08-13 17:22:49.941167 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.941137, event: dispatched, request: pg_info(1 pgs e194471:22.3b9) v4 -1853> 2014-08-13 17:22:49.941176 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.941176, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3b9) v4 -1852> 2014-08-13 17:22:49.941187 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.941187, event: started, request: pg_info(1 pgs e194471:22.3b9) v4 -1851> 2014-08-13 17:22:49.941213 7f74d8735700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:22:49.941212, event: done, request: pg_info(1 pgs e194471:22.3b9) v4 -1850> 2014-08-13 17:22:49.941453 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 8 ==== pg_info(1 pgs e194471:22.3b9) v4 ==== 714+0+0 (2751433786 0 0) 0x49e0540 con 0x48dfbc0 -1849> 2014-08-13 17:22:49.941490 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941327, event: header_read, request: pg_info(1 pgs e194471:22.3b9) v4 -1848> 2014-08-13 17:22:49.941501 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941331, event: throttled, request: pg_info(1 pgs e194471:22.3b9) v4 -1847> 2014-08-13 17:22:49.941507 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941424, event: all_read, request: pg_info(1 pgs e194471:22.3b9) v4 -1846> 2014-08-13 17:22:49.941512 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941485, event: dispatched, request: pg_info(1 pgs e194471:22.3b9) v4 -1845> 2014-08-13 17:22:49.941519 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941518, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3b9) v4 -1844> 2014-08-13 17:22:49.941527 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.941527, event: started, request: pg_info(1 pgs e194471:22.3b9) v4 -1843> 2014-08-13 17:22:49.941618 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194471:22.4fe) v4 -- ?+0 0x4ac7e00 -1842> 2014-08-13 17:22:49.941697 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194471:22.1fb) v4 -- ?+0 0x4ac7380 -1841> 2014-08-13 17:22:49.941860 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194471:22.226) v4 -- ?+0 0x49e0000 -1840> 2014-08-13 17:22:49.942040 7f74d8735700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:22:49.942040, event: done, request: pg_info(1 pgs e194471:22.3b9) v4 -1839> 2014-08-13 17:22:49.942075 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 10 ==== MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 ==== 26+0+0 (4116065263 0 0) 0x49e6e40 con 0x4868420 -1838> 2014-08-13 17:22:49.942096 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.941516, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1837> 2014-08-13 17:22:49.942103 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.941518, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1836> 2014-08-13 17:22:49.942108 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.941582, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1835> 2014-08-13 17:22:49.942114 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.942093, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1834> 2014-08-13 17:22:49.942120 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.942119, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1833> 2014-08-13 17:22:49.942144 7f74d8735700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:22:49.942143, event: done, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194471 v2 -1832> 2014-08-13 17:22:49.942162 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 13 ==== MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 ==== 26+0+0 (3682900524 0 0) 0x49e0fc0 con 0x486bc80 -1831> 2014-08-13 17:22:49.942181 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.941739, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1830> 2014-08-13 17:22:49.942157 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.082641 3 0.000121 -1829> 2014-08-13 17:22:49.942202 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.941740, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1828> 2014-08-13 17:22:49.942221 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.941801, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1827> 2014-08-13 17:22:49.942231 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.942179, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1826> 2014-08-13 17:22:49.942220 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -1825> 2014-08-13 17:22:49.942245 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.942244, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1824> 2014-08-13 17:22:49.942266 7f74d8735700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:22:49.942266, event: done, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194471 v2 -1823> 2014-08-13 17:22:49.942279 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 7 ==== pg_info(1 pgs e194471:22.3af) v4 ==== 714+0+0 (3931229169 0 0) 0x53bd240 con 0x47a4ba0 -1822> 2014-08-13 17:22:49.942294 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.941524, event: header_read, request: pg_info(1 pgs e194471:22.3af) v4 -1821> 2014-08-13 17:22:49.942301 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.941526, event: throttled, request: pg_info(1 pgs e194471:22.3af) v4 -1820> 2014-08-13 17:22:49.942285 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.074541 3 0.000099 -1819> 2014-08-13 17:22:49.942308 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.941599, event: all_read, request: pg_info(1 pgs e194471:22.3af) v4 -1818> 2014-08-13 17:22:49.942314 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.942292, event: dispatched, request: pg_info(1 pgs e194471:22.3af) v4 -1817> 2014-08-13 17:22:49.942320 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.942320, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.3af) v4 -1816> 2014-08-13 17:22:49.942311 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194470 pi=192979-194469/22 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -1815> 2014-08-13 17:22:49.942327 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.942327, event: started, request: pg_info(1 pgs e194471:22.3af) v4 -1814> 2014-08-13 17:22:49.942353 7f74d8735700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:22:49.942352, event: done, request: pg_info(1 pgs e194471:22.3af) v4 -1813> 2014-08-13 17:22:49.942553 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 12 ==== pg_info(1 pgs e194471:22.4fe) v4 ==== 714+0+0 (3255067699 0 0) 0x4ac7e00 con 0x4833440 -1812> 2014-08-13 17:22:49.942581 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942434, event: header_read, request: pg_info(1 pgs e194471:22.4fe) v4 -1811> 2014-08-13 17:22:49.942591 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942436, event: throttled, request: pg_info(1 pgs e194471:22.4fe) v4 -1810> 2014-08-13 17:22:49.942596 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942522, event: all_read, request: pg_info(1 pgs e194471:22.4fe) v4 -1809> 2014-08-13 17:22:49.942601 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942577, event: dispatched, request: pg_info(1 pgs e194471:22.4fe) v4 -1808> 2014-08-13 17:22:49.942608 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942608, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4fe) v4 -1807> 2014-08-13 17:22:49.942616 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942616, event: started, request: pg_info(1 pgs e194471:22.4fe) v4 -1806> 2014-08-13 17:22:49.942584 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.44 192.168.1.111:6822/10782 -- pg_info(1 pgs e194471:22.3af) v4 -- ?+0 0x466f8c0 -1805> 2014-08-13 17:22:49.942648 7f74d8735700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:22:49.942648, event: done, request: pg_info(1 pgs e194471:22.4fe) v4 -1804> 2014-08-13 17:22:49.942666 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 12 ==== pg_info(1 pgs e194471:22.1d2) v4 ==== 714+0+0 (2506920389 0 0) 0x53bc280 con 0x47a0b00 -1803> 2014-08-13 17:22:49.942724 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942497, event: header_read, request: pg_info(1 pgs e194471:22.1d2) v4 -1802> 2014-08-13 17:22:49.942732 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942499, event: throttled, request: pg_info(1 pgs e194471:22.1d2) v4 -1801> 2014-08-13 17:22:49.942738 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942581, event: all_read, request: pg_info(1 pgs e194471:22.1d2) v4 -1800> 2014-08-13 17:22:49.942743 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942720, event: dispatched, request: pg_info(1 pgs e194471:22.1d2) v4 -1799> 2014-08-13 17:22:49.942749 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942749, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1d2) v4 -1798> 2014-08-13 17:22:49.942756 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942756, event: started, request: pg_info(1 pgs e194471:22.1d2) v4 -1797> 2014-08-13 17:22:49.942798 7f74d8735700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:22:49.942798, event: done, request: pg_info(1 pgs e194471:22.1d2) v4 -1796> 2014-08-13 17:22:49.942814 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 17 ==== pg_info(1 pgs e194471:22.226) v4 ==== 714+0+0 (1757110741 0 0) 0x49e0000 con 0x4835280 -1795> 2014-08-13 17:22:49.942832 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942652, event: header_read, request: pg_info(1 pgs e194471:22.226) v4 -1794> 2014-08-13 17:22:49.942838 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942654, event: throttled, request: pg_info(1 pgs e194471:22.226) v4 -1793> 2014-08-13 17:22:49.942843 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942745, event: all_read, request: pg_info(1 pgs e194471:22.226) v4 -1792> 2014-08-13 17:22:49.942848 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942830, event: dispatched, request: pg_info(1 pgs e194471:22.226) v4 -1791> 2014-08-13 17:22:49.942854 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942854, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.226) v4 -1790> 2014-08-13 17:22:49.942869 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942868, event: started, request: pg_info(1 pgs e194471:22.226) v4 -1789> 2014-08-13 17:22:49.942846 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194471:22.3af) v4 -- ?+0 0x466b800 -1788> 2014-08-13 17:22:49.942887 7f74d8735700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:22:49.942887, event: done, request: pg_info(1 pgs e194471:22.226) v4 -1787> 2014-08-13 17:22:49.942901 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 14 ==== pg_info(1 pgs e194471:22.1fb) v4 ==== 714+0+0 (1211191517 0 0) 0x4ac7380 con 0x486a3c0 -1786> 2014-08-13 17:22:49.942915 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942674, event: header_read, request: pg_info(1 pgs e194471:22.1fb) v4 -1785> 2014-08-13 17:22:49.942921 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942676, event: throttled, request: pg_info(1 pgs e194471:22.1fb) v4 -1784> 2014-08-13 17:22:49.942925 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942759, event: all_read, request: pg_info(1 pgs e194471:22.1fb) v4 -1783> 2014-08-13 17:22:49.942934 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942913, event: dispatched, request: pg_info(1 pgs e194471:22.1fb) v4 -1782> 2014-08-13 17:22:49.942946 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942946, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1fb) v4 -1781> 2014-08-13 17:22:49.942957 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942957, event: started, request: pg_info(1 pgs e194471:22.1fb) v4 -1780> 2014-08-13 17:22:49.942981 7f74d8735700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:22:49.942980, event: done, request: pg_info(1 pgs e194471:22.1fb) v4 -1779> 2014-08-13 17:22:49.942981 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194471:22.1d2) v4 -- ?+0 0x57d3d40 -1778> 2014-08-13 17:22:49.942993 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 13 ==== pg_info(1 pgs e194471:22.4fe) v4 ==== 714+0+0 (3205895034 0 0) 0x57d16c0 con 0x4833440 -1777> 2014-08-13 17:22:49.943007 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.942723, event: header_read, request: pg_info(1 pgs e194471:22.4fe) v4 -1776> 2014-08-13 17:22:49.943019 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.942725, event: throttled, request: pg_info(1 pgs e194471:22.4fe) v4 -1775> 2014-08-13 17:22:49.943024 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.942820, event: all_read, request: pg_info(1 pgs e194471:22.4fe) v4 -1774> 2014-08-13 17:22:49.943029 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.943006, event: dispatched, request: pg_info(1 pgs e194471:22.4fe) v4 -1773> 2014-08-13 17:22:49.943035 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.943035, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.4fe) v4 -1772> 2014-08-13 17:22:49.943042 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.943042, event: started, request: pg_info(1 pgs e194471:22.4fe) v4 -1771> 2014-08-13 17:22:49.943026 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.351 192.168.1.102:6830/16444 -- MRecoveryReserve GRANT pgid: 22.220, query_epoch: 194471 v2 -- ?+0 0x466b9c0 -1770> 2014-08-13 17:22:49.943059 7f74d8735700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:22:49.943059, event: done, request: pg_info(1 pgs e194471:22.4fe) v4 -1769> 2014-08-13 17:22:49.943068 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194471:22.1d2) v4 -- ?+0 0x57d6c80 -1768> 2014-08-13 17:22:49.943067 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.000847 1 0.000115 -1767> 2014-08-13 17:22:49.943099 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194470 pi=192970-194469/22 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepRecovering -1766> 2014-08-13 17:22:49.943498 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194471/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.108035 9 0.001336 -1765> 2014-08-13 17:22:49.943534 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194471/194466 194470/194470/194470) [4,349,61] r=0 lpr=194470 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1764> 2014-08-13 17:22:49.943529 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194471/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.085241 9 0.002268 -1763> 2014-08-13 17:22:49.943558 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194471/194466 194470/194470/194470) [4,44,362] r=0 lpr=194470 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1762> 2014-08-13 17:22:49.943784 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 18 ==== MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 ==== 26+0+0 (820024540 0 0) 0x4ac78c0 con 0x4835280 -1761> 2014-08-13 17:22:49.943818 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943690, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1760> 2014-08-13 17:22:49.943833 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943692, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1759> 2014-08-13 17:22:49.943839 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943758, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1758> 2014-08-13 17:22:49.943848 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943814, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1757> 2014-08-13 17:22:49.943860 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943860, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1756> 2014-08-13 17:22:49.943883 7f74d8735700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:22:49.943883, event: done, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194471 v2 -1755> 2014-08-13 17:22:49.943900 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 15 ==== pg_info(1 pgs e194471:22.1fb) v4 ==== 714+0+0 (575473733 0 0) 0x4ac3d40 con 0x486a3c0 -1754> 2014-08-13 17:22:49.943915 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943742, event: header_read, request: pg_info(1 pgs e194471:22.1fb) v4 -1753> 2014-08-13 17:22:49.943922 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943743, event: throttled, request: pg_info(1 pgs e194471:22.1fb) v4 -1752> 2014-08-13 17:22:49.943899 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.071699 3 0.000105 -1751> 2014-08-13 17:22:49.943927 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943816, event: all_read, request: pg_info(1 pgs e194471:22.1fb) v4 -1750> 2014-08-13 17:22:49.943932 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943913, event: dispatched, request: pg_info(1 pgs e194471:22.1fb) v4 -1749> 2014-08-13 17:22:49.943938 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943938, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1fb) v4 -1748> 2014-08-13 17:22:49.943932 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194470 pi=192975-194469/22 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -1747> 2014-08-13 17:22:49.943946 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943945, event: started, request: pg_info(1 pgs e194471:22.1fb) v4 -1746> 2014-08-13 17:22:49.943985 7f74d8735700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:22:49.943984, event: done, request: pg_info(1 pgs e194471:22.1fb) v4 -1745> 2014-08-13 17:22:49.947180 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 6 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e35c40 con 0x4d58580 -1744> 2014-08-13 17:22:49.947231 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1743> 2014-08-13 17:22:49.947393 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.7 192.168.1.115:6838/62713 7 ==== pg_notify(22.1b8(20) epoch 194471) v5 ==== 1838+0+0 (4017426367 0 0) 0x57d7540 con 0x4d58580 -1742> 2014-08-13 17:22:49.947438 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947182, event: header_read, request: pg_notify(22.1b8(20) epoch 194471) v5 -1741> 2014-08-13 17:22:49.947453 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947184, event: throttled, request: pg_notify(22.1b8(20) epoch 194471) v5 -1740> 2014-08-13 17:22:49.947463 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947325, event: all_read, request: pg_notify(22.1b8(20) epoch 194471) v5 -1739> 2014-08-13 17:22:49.947473 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947432, event: dispatched, request: pg_notify(22.1b8(20) epoch 194471) v5 -1738> 2014-08-13 17:22:49.947484 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947483, event: waiting_for_osdmap, request: pg_notify(22.1b8(20) epoch 194471) v5 -1737> 2014-08-13 17:22:49.947497 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947497, event: started, request: pg_notify(22.1b8(20) epoch 194471) v5 -1736> 2014-08-13 17:22:49.947533 7f74d8735700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:22:49.947532, event: done, request: pg_notify(22.1b8(20) epoch 194471) v5 -1735> 2014-08-13 17:22:49.948424 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de2400 con 0x4d56040 -1734> 2014-08-13 17:22:49.948472 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1733> 2014-08-13 17:22:49.948500 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 5 ==== pg_notify(22.4df(20) epoch 194471) v5 ==== 1834+0+0 (4045728566 0 0) 0x57d4280 con 0x4d56040 -1732> 2014-08-13 17:22:49.948522 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948387, event: header_read, request: pg_notify(22.4df(20) epoch 194471) v5 -1731> 2014-08-13 17:22:49.948537 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948388, event: throttled, request: pg_notify(22.4df(20) epoch 194471) v5 -1730> 2014-08-13 17:22:49.948547 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948477, event: all_read, request: pg_notify(22.4df(20) epoch 194471) v5 -1729> 2014-08-13 17:22:49.948556 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948519, event: dispatched, request: pg_notify(22.4df(20) epoch 194471) v5 -1728> 2014-08-13 17:22:49.948567 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948567, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194471) v5 -1727> 2014-08-13 17:22:49.948580 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948580, event: started, request: pg_notify(22.4df(20) epoch 194471) v5 -1726> 2014-08-13 17:22:49.948616 7f74d8735700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:22:49.948616, event: done, request: pg_notify(22.4df(20) epoch 194471) v5 -1725> 2014-08-13 17:22:49.949076 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.19 192.168.1.115:6818/61186 -- pg_info(1 pgs e194471:24.3bf) v4 -- ?+0 0x49e1340 -1724> 2014-08-13 17:22:49.949936 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194471:22.14a) v4 -- ?+0 0x49e7540 -1723> 2014-08-13 17:22:49.950111 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194471:22.1a0) v4 -- ?+0 0x49e2680 -1722> 2014-08-13 17:22:49.950165 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194471:22.791) v4 -- ?+0 0x49e16c0 -1721> 2014-08-13 17:22:49.950222 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.19 192.168.1.115:6818/61186 5 ==== pg_info(1 pgs e194471:24.3bf) v4 ==== 744+0+0 (3389438644 0 0) 0x4ac1f80 con 0x4901340 -1720> 2014-08-13 17:22:49.950246 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950103, event: header_read, request: pg_info(1 pgs e194471:24.3bf) v4 -1719> 2014-08-13 17:22:49.950260 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950106, event: throttled, request: pg_info(1 pgs e194471:24.3bf) v4 -1718> 2014-08-13 17:22:49.950270 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950197, event: all_read, request: pg_info(1 pgs e194471:24.3bf) v4 -1717> 2014-08-13 17:22:49.950280 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950241, event: dispatched, request: pg_info(1 pgs e194471:24.3bf) v4 -1716> 2014-08-13 17:22:49.950287 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950287, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:24.3bf) v4 -1715> 2014-08-13 17:22:49.950296 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950296, event: started, request: pg_info(1 pgs e194471:24.3bf) v4 -1714> 2014-08-13 17:22:49.950328 7f74d8735700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:22:49.950327, event: done, request: pg_info(1 pgs e194471:24.3bf) v4 -1713> 2014-08-13 17:22:49.950403 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.353 192.168.1.108:6836/6227 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4802400 con 0x532c360 -1712> 2014-08-13 17:22:49.950436 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1711> 2014-08-13 17:22:49.950559 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.353 192.168.1.108:6836/6227 4 ==== pg_info(1 pgs e194471:22.775) v4 ==== 714+0+0 (2810754400 0 0) 0x4ac0700 con 0x532c360 -1710> 2014-08-13 17:22:49.950566 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194471:22.116) v4 -- ?+0 0x49e1dc0 -1709> 2014-08-13 17:22:49.950594 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950409, event: header_read, request: pg_info(1 pgs e194471:22.775) v4 -1708> 2014-08-13 17:22:49.950625 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950411, event: throttled, request: pg_info(1 pgs e194471:22.775) v4 -1707> 2014-08-13 17:22:49.950639 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950528, event: all_read, request: pg_info(1 pgs e194471:22.775) v4 -1706> 2014-08-13 17:22:49.950645 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950589, event: dispatched, request: pg_info(1 pgs e194471:22.775) v4 -1705> 2014-08-13 17:22:49.950656 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950656, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.775) v4 -1704> 2014-08-13 17:22:49.950671 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950671, event: started, request: pg_info(1 pgs e194471:22.775) v4 -1703> 2014-08-13 17:22:49.950697 7f74d8735700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:22:49.950697, event: done, request: pg_info(1 pgs e194471:22.775) v4 -1702> 2014-08-13 17:22:49.950720 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 10 ==== pg_info(1 pgs e194471:22.14a) v4 ==== 714+0+0 (1705430726 0 0) 0x49e7540 con 0x45f1e40 -1701> 2014-08-13 17:22:49.950737 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950525, event: header_read, request: pg_info(1 pgs e194471:22.14a) v4 -1700> 2014-08-13 17:22:49.950743 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950527, event: throttled, request: pg_info(1 pgs e194471:22.14a) v4 -1699> 2014-08-13 17:22:49.950748 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950601, event: all_read, request: pg_info(1 pgs e194471:22.14a) v4 -1698> 2014-08-13 17:22:49.950753 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950734, event: dispatched, request: pg_info(1 pgs e194471:22.14a) v4 -1697> 2014-08-13 17:22:49.950759 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950759, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.14a) v4 -1696> 2014-08-13 17:22:49.950766 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950766, event: started, request: pg_info(1 pgs e194471:22.14a) v4 -1695> 2014-08-13 17:22:49.950789 7f74d8735700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:22:49.950789, event: done, request: pg_info(1 pgs e194471:22.14a) v4 -1694> 2014-08-13 17:22:49.950852 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 10 ==== pg_info(1 pgs e194471:2.2a) v4 ==== 706+0+0 (67553669 0 0) 0x53b8e00 con 0x47a3f40 -1693> 2014-08-13 17:22:49.950875 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950704, event: header_read, request: pg_info(1 pgs e194471:2.2a) v4 -1692> 2014-08-13 17:22:49.950886 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950706, event: throttled, request: pg_info(1 pgs e194471:2.2a) v4 -1691> 2014-08-13 17:22:49.950897 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950829, event: all_read, request: pg_info(1 pgs e194471:2.2a) v4 -1690> 2014-08-13 17:22:49.950906 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950872, event: dispatched, request: pg_info(1 pgs e194471:2.2a) v4 -1689> 2014-08-13 17:22:49.950915 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950915, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:2.2a) v4 -1688> 2014-08-13 17:22:49.950927 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950927, event: started, request: pg_info(1 pgs e194471:2.2a) v4 -1687> 2014-08-13 17:22:49.950955 7f74d8735700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:22:49.950955, event: done, request: pg_info(1 pgs e194471:2.2a) v4 -1686> 2014-08-13 17:22:49.951080 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 9 ==== pg_info(1 pgs e194471:22.1a0) v4 ==== 714+0+0 (1475145267 0 0) 0x49e2680 con 0x4758b00 -1685> 2014-08-13 17:22:49.951113 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.950976, event: header_read, request: pg_info(1 pgs e194471:22.1a0) v4 -1684> 2014-08-13 17:22:49.951128 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.950978, event: throttled, request: pg_info(1 pgs e194471:22.1a0) v4 -1683> 2014-08-13 17:22:49.951119 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:2.2a) v4 -- ?+0 0x57d5400 -1682> 2014-08-13 17:22:49.951139 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.951055, event: all_read, request: pg_info(1 pgs e194471:22.1a0) v4 -1681> 2014-08-13 17:22:49.951145 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.951108, event: dispatched, request: pg_info(1 pgs e194471:22.1a0) v4 -1680> 2014-08-13 17:22:49.951152 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.951152, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1a0) v4 -1679> 2014-08-13 17:22:49.951170 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.951170, event: started, request: pg_info(1 pgs e194471:22.1a0) v4 -1678> 2014-08-13 17:22:49.951205 7f74d8735700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:22:49.951205, event: done, request: pg_info(1 pgs e194471:22.1a0) v4 -1677> 2014-08-13 17:22:49.951227 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 7 ==== pg_info(1 pgs e194471:22.791) v4 ==== 714+0+0 (546369825 0 0) 0x49e16c0 con 0x475a940 -1676> 2014-08-13 17:22:49.951244 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.950988, event: header_read, request: pg_info(1 pgs e194471:22.791) v4 -1675> 2014-08-13 17:22:49.951250 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.950989, event: throttled, request: pg_info(1 pgs e194471:22.791) v4 -1674> 2014-08-13 17:22:49.951256 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.951073, event: all_read, request: pg_info(1 pgs e194471:22.791) v4 -1673> 2014-08-13 17:22:49.951261 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.951241, event: dispatched, request: pg_info(1 pgs e194471:22.791) v4 -1672> 2014-08-13 17:22:49.951267 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.951266, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.791) v4 -1671> 2014-08-13 17:22:49.951274 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.951274, event: started, request: pg_info(1 pgs e194471:22.791) v4 -1670> 2014-08-13 17:22:49.951306 7f74d8735700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:22:49.951306, event: done, request: pg_info(1 pgs e194471:22.791) v4 -1669> 2014-08-13 17:22:49.951321 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 9 ==== pg_info(1 pgs e194471:22.116) v4 ==== 714+0+0 (1962186479 0 0) 0x49e1dc0 con 0x475c4c0 -1668> 2014-08-13 17:22:49.951323 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.112628 6 0.002596 -1667> 2014-08-13 17:22:49.951336 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951125, event: header_read, request: pg_info(1 pgs e194471:22.116) v4 -1666> 2014-08-13 17:22:49.951353 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951127, event: throttled, request: pg_info(1 pgs e194471:22.116) v4 -1665> 2014-08-13 17:22:49.951359 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951210, event: all_read, request: pg_info(1 pgs e194471:22.116) v4 -1664> 2014-08-13 17:22:49.951364 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951334, event: dispatched, request: pg_info(1 pgs e194471:22.116) v4 -1663> 2014-08-13 17:22:49.951354 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1662> 2014-08-13 17:22:49.951370 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951370, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.116) v4 -1661> 2014-08-13 17:22:49.951377 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951377, event: started, request: pg_info(1 pgs e194471:22.116) v4 -1660> 2014-08-13 17:22:49.951378 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000024 0 0.000000 -1659> 2014-08-13 17:22:49.951402 7f74d8735700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:22:49.951402, event: done, request: pg_info(1 pgs e194471:22.116) v4 -1658> 2014-08-13 17:22:49.951399 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1657> 2014-08-13 17:22:49.951416 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 11 ==== pg_info(1 pgs e194471:22.14a) v4 ==== 714+0+0 (3419529701 0 0) 0x57d7000 con 0x45f1e40 -1656> 2014-08-13 17:22:49.951431 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951110, event: header_read, request: pg_info(1 pgs e194471:22.14a) v4 -1655> 2014-08-13 17:22:49.951437 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951112, event: throttled, request: pg_info(1 pgs e194471:22.14a) v4 -1654> 2014-08-13 17:22:49.951428 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:2.2a) v4 -- ?+0 0x57d6740 -1653> 2014-08-13 17:22:49.951443 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951187, event: all_read, request: pg_info(1 pgs e194471:22.14a) v4 -1652> 2014-08-13 17:22:49.951448 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951429, event: dispatched, request: pg_info(1 pgs e194471:22.14a) v4 -1651> 2014-08-13 17:22:49.951454 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951453, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.14a) v4 -1650> 2014-08-13 17:22:49.951460 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951460, event: started, request: pg_info(1 pgs e194471:22.14a) v4 -1649> 2014-08-13 17:22:49.951480 7f74d8735700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:22:49.951480, event: done, request: pg_info(1 pgs e194471:22.14a) v4 -1648> 2014-08-13 17:22:49.951601 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.328 192.168.1.104:6812/64303 10 ==== pg_info(1 pgs e194471:22.1a0) v4 ==== 714+0+0 (842927879 0 0) 0x57d1500 con 0x4758b00 -1647> 2014-08-13 17:22:49.951632 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951488, event: header_read, request: pg_info(1 pgs e194471:22.1a0) v4 -1646> 2014-08-13 17:22:49.951644 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951495, event: throttled, request: pg_info(1 pgs e194471:22.1a0) v4 -1645> 2014-08-13 17:22:49.951651 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951564, event: all_read, request: pg_info(1 pgs e194471:22.1a0) v4 -1644> 2014-08-13 17:22:49.951656 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951628, event: dispatched, request: pg_info(1 pgs e194471:22.1a0) v4 -1643> 2014-08-13 17:22:49.951663 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951663, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1a0) v4 -1642> 2014-08-13 17:22:49.951672 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951672, event: started, request: pg_info(1 pgs e194471:22.1a0) v4 -1641> 2014-08-13 17:22:49.951702 7f74d8735700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:22:49.951702, event: done, request: pg_info(1 pgs e194471:22.1a0) v4 -1640> 2014-08-13 17:22:49.951723 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 8 ==== pg_info(1 pgs e194471:22.791) v4 ==== 714+0+0 (272487733 0 0) 0x53bec80 con 0x475a940 -1639> 2014-08-13 17:22:49.951739 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951549, event: header_read, request: pg_info(1 pgs e194471:22.791) v4 -1638> 2014-08-13 17:22:49.951745 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951551, event: throttled, request: pg_info(1 pgs e194471:22.791) v4 -1637> 2014-08-13 17:22:49.951751 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951622, event: all_read, request: pg_info(1 pgs e194471:22.791) v4 -1636> 2014-08-13 17:22:49.951756 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951737, event: dispatched, request: pg_info(1 pgs e194471:22.791) v4 -1635> 2014-08-13 17:22:49.951762 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951762, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.791) v4 -1634> 2014-08-13 17:22:49.951769 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951769, event: started, request: pg_info(1 pgs e194471:22.791) v4 -1633> 2014-08-13 17:22:49.951792 7f74d8735700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:22:49.951791, event: done, request: pg_info(1 pgs e194471:22.791) v4 -1632> 2014-08-13 17:22:49.951876 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194471; pg2.2a; ) v2 -- ?+0 0x53b8c40 -1631> 2014-08-13 17:22:49.951916 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194471; pg2.2a; ) v2 -- ?+0 0x53b9180 -1630> 2014-08-13 17:22:49.953503 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x48045c0 con 0x48dcd00 -1629> 2014-08-13 17:22:49.953542 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1628> 2014-08-13 17:22:49.953632 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 5 ==== pg_notify(24.1bb(2) epoch 194471) v5 ==== 870+0+0 (4005233633 0 0) 0x57d2bc0 con 0x48dcd00 -1627> 2014-08-13 17:22:49.953656 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953506, event: header_read, request: pg_notify(24.1bb(2) epoch 194471) v5 -1626> 2014-08-13 17:22:49.953670 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953507, event: throttled, request: pg_notify(24.1bb(2) epoch 194471) v5 -1625> 2014-08-13 17:22:49.953681 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953609, event: all_read, request: pg_notify(24.1bb(2) epoch 194471) v5 -1624> 2014-08-13 17:22:49.953688 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953653, event: dispatched, request: pg_notify(24.1bb(2) epoch 194471) v5 -1623> 2014-08-13 17:22:49.953695 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953694, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194471) v5 -1622> 2014-08-13 17:22:49.953704 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953704, event: started, request: pg_notify(24.1bb(2) epoch 194471) v5 -1621> 2014-08-13 17:22:49.953737 7f74d8735700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:22:49.953737, event: done, request: pg_notify(24.1bb(2) epoch 194471) v5 -1620> 2014-08-13 17:22:49.953756 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 6 ==== pg_notify(24.1bb(2) epoch 194471) v5 ==== 870+0+0 (4005233633 0 0) 0x4ac6e40 con 0x48dcd00 -1619> 2014-08-13 17:22:49.953772 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953643, event: header_read, request: pg_notify(24.1bb(2) epoch 194471) v5 -1618> 2014-08-13 17:22:49.953778 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953644, event: throttled, request: pg_notify(24.1bb(2) epoch 194471) v5 -1617> 2014-08-13 17:22:49.953784 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953705, event: all_read, request: pg_notify(24.1bb(2) epoch 194471) v5 -1616> 2014-08-13 17:22:49.953790 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953770, event: dispatched, request: pg_notify(24.1bb(2) epoch 194471) v5 -1615> 2014-08-13 17:22:49.953796 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953795, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194471) v5 -1614> 2014-08-13 17:22:49.953803 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.953803, event: started, request: pg_notify(24.1bb(2) epoch 194471) v5 -1613> 2014-08-13 17:22:49.953779 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.092000 8 0.000445 -1612> 2014-08-13 17:22:49.953822 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog -1611> 2014-08-13 17:22:49.953909 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000087 0 0.000000 -1610> 2014-08-13 17:22:49.953934 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing -1609> 2014-08-13 17:22:49.953961 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000027 0 0.000000 -1608> 2014-08-13 17:22:49.953982 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru -1607> 2014-08-13 17:22:49.954110 7f74d8735700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:22:49.954109, event: done, request: pg_notify(24.1bb(2) epoch 194471) v5 -1606> 2014-08-13 17:22:49.956893 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.296 192.168.1.109:6806/31286 1 ==== osd_map(194456..194471 src has 185139..194471) v3 ==== 10429+0+0 (970627098 0 0) 0x4a82f40 con 0x486fe80 -1605> 2014-08-13 17:22:49.956942 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194456,194471], i have 194471, src has [185139,194471] -1604> 2014-08-13 17:22:49.957006 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.296 192.168.1.109:6806/31286 2 ==== pg_notify(24.181(24) epoch 194471) v5 ==== 2092+0+0 (2020063715 0 0) 0x4ac2840 con 0x486fe80 -1603> 2014-08-13 17:22:49.957039 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.956868, event: header_read, request: pg_notify(24.181(24) epoch 194471) v5 -1602> 2014-08-13 17:22:49.957054 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.956869, event: throttled, request: pg_notify(24.181(24) epoch 194471) v5 -1601> 2014-08-13 17:22:49.957064 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.956970, event: all_read, request: pg_notify(24.181(24) epoch 194471) v5 -1600> 2014-08-13 17:22:49.957073 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.957035, event: dispatched, request: pg_notify(24.181(24) epoch 194471) v5 -1599> 2014-08-13 17:22:49.957084 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.957083, event: waiting_for_osdmap, request: pg_notify(24.181(24) epoch 194471) v5 -1598> 2014-08-13 17:22:49.957097 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.957097, event: started, request: pg_notify(24.181(24) epoch 194471) v5 -1597> 2014-08-13 17:22:49.957132 7f74d8735700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:22:49.957131, event: done, request: pg_notify(24.181(24) epoch 194471) v5 -1596> 2014-08-13 17:22:49.957173 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.296 192.168.1.109:6806/31286 3 ==== pg_notify(24.181(24) epoch 194471) v5 ==== 2092+0+0 (2020063715 0 0) 0x4ac1180 con 0x486fe80 -1595> 2014-08-13 17:22:49.957204 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.956995, event: header_read, request: pg_notify(24.181(24) epoch 194471) v5 -1594> 2014-08-13 17:22:49.957216 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.956996, event: throttled, request: pg_notify(24.181(24) epoch 194471) v5 -1593> 2014-08-13 17:22:49.957225 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.957084, event: all_read, request: pg_notify(24.181(24) epoch 194471) v5 -1592> 2014-08-13 17:22:49.957238 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.957191, event: dispatched, request: pg_notify(24.181(24) epoch 194471) v5 -1591> 2014-08-13 17:22:49.957199 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.103697 8 0.000229 -1590> 2014-08-13 17:22:49.957253 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.957253, event: waiting_for_osdmap, request: pg_notify(24.181(24) epoch 194471) v5 -1589> 2014-08-13 17:22:49.957274 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.957274, event: started, request: pg_notify(24.181(24) epoch 194471) v5 -1588> 2014-08-13 17:22:49.957269 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog -1587> 2014-08-13 17:22:49.957376 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000107 0 0.000000 -1586> 2014-08-13 17:22:49.957405 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing -1585> 2014-08-13 17:22:49.957437 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000032 0 0.000000 -1584> 2014-08-13 17:22:49.957462 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru -1583> 2014-08-13 17:22:49.957580 7f74d8735700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:22:49.957580, event: done, request: pg_notify(24.181(24) epoch 194471) v5 -1582> 2014-08-13 17:22:49.957635 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4a87080 con 0x48dad60 -1581> 2014-08-13 17:22:49.957667 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1580> 2014-08-13 17:22:49.957687 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 5 ==== pg_notify(22.481(39) epoch 194471) v5 ==== 2903+0+0 (2579444395 0 0) 0x49e0e00 con 0x48dad60 -1579> 2014-08-13 17:22:49.957707 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957300, event: header_read, request: pg_notify(22.481(39) epoch 194471) v5 -1578> 2014-08-13 17:22:49.957719 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957301, event: throttled, request: pg_notify(22.481(39) epoch 194471) v5 -1577> 2014-08-13 17:22:49.957729 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957421, event: all_read, request: pg_notify(22.481(39) epoch 194471) v5 -1576> 2014-08-13 17:22:49.957738 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957704, event: dispatched, request: pg_notify(22.481(39) epoch 194471) v5 -1575> 2014-08-13 17:22:49.957748 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957748, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194471) v5 -1574> 2014-08-13 17:22:49.957761 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957761, event: started, request: pg_notify(22.481(39) epoch 194471) v5 -1573> 2014-08-13 17:22:49.957798 7f74d8735700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:22:49.957798, event: done, request: pg_notify(22.481(39) epoch 194471) v5 -1572> 2014-08-13 17:22:49.957835 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 5 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e36300 con 0x4906b40 -1571> 2014-08-13 17:22:49.957857 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1570> 2014-08-13 17:22:49.957912 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.370 192.168.1.114:6842/56512 6 ==== pg_notify(1.2b(2) epoch 194471) v5 ==== 804+0+0 (2607591658 0 0) 0x53b9180 con 0x4906b40 -1569> 2014-08-13 17:22:49.957941 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957797, event: header_read, request: pg_notify(1.2b(2) epoch 194471) v5 -1568> 2014-08-13 17:22:49.957953 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957798, event: throttled, request: pg_notify(1.2b(2) epoch 194471) v5 -1567> 2014-08-13 17:22:49.957963 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957881, event: all_read, request: pg_notify(1.2b(2) epoch 194471) v5 -1566> 2014-08-13 17:22:49.957972 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957937, event: dispatched, request: pg_notify(1.2b(2) epoch 194471) v5 -1565> 2014-08-13 17:22:49.957982 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957982, event: waiting_for_osdmap, request: pg_notify(1.2b(2) epoch 194471) v5 -1564> 2014-08-13 17:22:49.957994 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.957993, event: started, request: pg_notify(1.2b(2) epoch 194471) v5 -1563> 2014-08-13 17:22:49.958024 7f74d8735700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:22:49.958024, event: done, request: pg_notify(1.2b(2) epoch 194471) v5 -1562> 2014-08-13 17:22:49.959360 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.287 192.168.1.109:6814/33435 6 ==== pg_info(1 pgs e194471:22.481) v4 ==== 714+0+0 (253316036 0 0) 0x57d4440 con 0x48dad60 -1561> 2014-08-13 17:22:49.959391 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959252, event: header_read, request: pg_info(1 pgs e194471:22.481) v4 -1560> 2014-08-13 17:22:49.959401 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959254, event: throttled, request: pg_info(1 pgs e194471:22.481) v4 -1559> 2014-08-13 17:22:49.959406 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959336, event: all_read, request: pg_info(1 pgs e194471:22.481) v4 -1558> 2014-08-13 17:22:49.959415 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959387, event: dispatched, request: pg_info(1 pgs e194471:22.481) v4 -1557> 2014-08-13 17:22:49.959425 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959425, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.481) v4 -1556> 2014-08-13 17:22:49.959437 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959437, event: started, request: pg_info(1 pgs e194471:22.481) v4 -1555> 2014-08-13 17:22:49.959468 7f74d8735700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:22:49.959468, event: done, request: pg_info(1 pgs e194471:22.481) v4 -1554> 2014-08-13 17:22:49.959497 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.287 192.168.1.109:6814/33435 -- pg_info(1 pgs e194471:22.481) v4 -- ?+0 0x466f1c0 -1553> 2014-08-13 17:22:49.959536 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.309 192.168.1.101:6814/23703 -- pg_info(1 pgs e194471:22.481) v4 -- ?+0 0x466eac0 -1552> 2014-08-13 17:22:49.959986 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194471/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.100874 11 0.000893 -1551> 2014-08-13 17:22:49.960022 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194471/194304 194470/194470/194470) [4,287,309] r=0 lpr=194470 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1550> 2014-08-13 17:22:49.960284 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 8 ==== pg_info(1 pgs e194471:22.653) v4 ==== 714+0+0 (2461397334 0 0) 0x57d5780 con 0x4832c00 -1549> 2014-08-13 17:22:49.960317 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960148, event: header_read, request: pg_info(1 pgs e194471:22.653) v4 -1548> 2014-08-13 17:22:49.960331 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960151, event: throttled, request: pg_info(1 pgs e194471:22.653) v4 -1547> 2014-08-13 17:22:49.960341 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960253, event: all_read, request: pg_info(1 pgs e194471:22.653) v4 -1546> 2014-08-13 17:22:49.960349 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960312, event: dispatched, request: pg_info(1 pgs e194471:22.653) v4 -1545> 2014-08-13 17:22:49.960360 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960359, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.653) v4 -1544> 2014-08-13 17:22:49.960372 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960372, event: started, request: pg_info(1 pgs e194471:22.653) v4 -1543> 2014-08-13 17:22:49.960408 7f74d8735700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:22:49.960408, event: done, request: pg_info(1 pgs e194471:22.653) v4 -1542> 2014-08-13 17:22:49.960399 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.396 192.168.1.105:6814/41806 -- pg_info(1 pgs e194471:24.299) v4 -- ?+0 0x5264440 -1541> 2014-08-13 17:22:49.960797 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 9 ==== MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 ==== 26+0+0 (1557533788 0 0) 0x49e2300 con 0x4832c00 -1540> 2014-08-13 17:22:49.960834 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960713, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1539> 2014-08-13 17:22:49.960848 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960714, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1538> 2014-08-13 17:22:49.960857 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960775, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1537> 2014-08-13 17:22:49.960866 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960829, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1536> 2014-08-13 17:22:49.960877 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960877, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1535> 2014-08-13 17:22:49.960904 7f74d8735700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:22:49.960904, event: done, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194471 v2 -1534> 2014-08-13 17:22:49.960920 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.093474 3 0.000111 -1533> 2014-08-13 17:22:49.960961 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194470 pi=192977-194469/22 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -1532> 2014-08-13 17:22:49.961436 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.21 192.168.1.107:6821/54786 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c70900 con 0x47a6ca0 -1531> 2014-08-13 17:22:49.961477 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1530> 2014-08-13 17:22:49.961573 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 8 ==== pg_info(1 pgs e194471:24.299) v4 ==== 744+0+0 (676577684 0 0) 0x5264440 con 0x4900580 -1529> 2014-08-13 17:22:49.961598 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961487, event: header_read, request: pg_info(1 pgs e194471:24.299) v4 -1528> 2014-08-13 17:22:49.961613 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961488, event: throttled, request: pg_info(1 pgs e194471:24.299) v4 -1527> 2014-08-13 17:22:49.961622 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961552, event: all_read, request: pg_info(1 pgs e194471:24.299) v4 -1526> 2014-08-13 17:22:49.961632 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961594, event: dispatched, request: pg_info(1 pgs e194471:24.299) v4 -1525> 2014-08-13 17:22:49.961642 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961642, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:24.299) v4 -1524> 2014-08-13 17:22:49.961655 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961655, event: started, request: pg_info(1 pgs e194471:24.299) v4 -1523> 2014-08-13 17:22:49.961691 7f74d8735700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:22:49.961690, event: done, request: pg_info(1 pgs e194471:24.299) v4 -1522> 2014-08-13 17:22:49.961714 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.21 192.168.1.107:6821/54786 4 ==== pg_info(1 pgs e194471:22.707) v4 ==== 714+0+0 (149088067 0 0) 0x4ac3640 con 0x47a6ca0 -1521> 2014-08-13 17:22:49.961733 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961437, event: header_read, request: pg_info(1 pgs e194471:22.707) v4 -1520> 2014-08-13 17:22:49.961751 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961438, event: throttled, request: pg_info(1 pgs e194471:22.707) v4 -1519> 2014-08-13 17:22:49.961760 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961629, event: all_read, request: pg_info(1 pgs e194471:22.707) v4 -1518> 2014-08-13 17:22:49.961769 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961731, event: dispatched, request: pg_info(1 pgs e194471:22.707) v4 -1517> 2014-08-13 17:22:49.961779 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961779, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.707) v4 -1516> 2014-08-13 17:22:49.961790 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961790, event: started, request: pg_info(1 pgs e194471:22.707) v4 -1515> 2014-08-13 17:22:49.961818 7f74d8735700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:22:49.961818, event: done, request: pg_info(1 pgs e194471:22.707) v4 -1514> 2014-08-13 17:22:49.961849 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.21 192.168.1.107:6821/54786 -- pg_info(1 pgs e194471:22.707) v4 -- ?+0 0x466f380 -1513> 2014-08-13 17:22:49.961886 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.310 192.168.1.109:6812/32413 -- pg_info(1 pgs e194471:22.707) v4 -- ?+0 0x466e900 -1512> 2014-08-13 17:22:49.962140 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194471/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.124038 9 0.000990 -1511> 2014-08-13 17:22:49.962173 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194471/194466 194470/194470/194470) [4,310,21] r=0 lpr=194470 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1510> 2014-08-13 17:22:49.962873 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 9 ==== MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 ==== 30+0+0 (3163232828 0 0) 0x466b2c0 con 0x4900580 -1509> 2014-08-13 17:22:49.962909 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962763, event: header_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1508> 2014-08-13 17:22:49.962923 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962766, event: throttled, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1507> 2014-08-13 17:22:49.962933 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962847, event: all_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1506> 2014-08-13 17:22:49.962943 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962904, event: dispatched, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1505> 2014-08-13 17:22:49.962953 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962953, event: waiting_for_osdmap, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1504> 2014-08-13 17:22:49.962988 7f74d8735700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:22:49.962988, event: done, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194471, prio: 1 v3 -1503> 2014-08-13 17:22:49.963009 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 luod=0'0 crt=0'0 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.093219 4 0.001455 -1502> 2014-08-13 17:22:49.963055 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 luod=0'0 crt=0'0 lcod 0'0 active] enter Started/ReplicaActive/RepWaitBackfillReserved -1501> 2014-08-13 17:22:49.974844 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.5 192.168.1.103:6827/40192 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4c75100 con 0x47f2ec0 -1500> 2014-08-13 17:22:49.974893 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1499> 2014-08-13 17:22:49.974907 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.5 192.168.1.103:6827/40192 4 ==== pg_info(1 pgs e194471:22.1f1) v4 ==== 714+0+0 (4000969918 0 0) 0x4ac0a80 con 0x47f2ec0 -1498> 2014-08-13 17:22:49.974923 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974782, event: header_read, request: pg_info(1 pgs e194471:22.1f1) v4 -1497> 2014-08-13 17:22:49.974936 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974784, event: throttled, request: pg_info(1 pgs e194471:22.1f1) v4 -1496> 2014-08-13 17:22:49.974946 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974845, event: all_read, request: pg_info(1 pgs e194471:22.1f1) v4 -1495> 2014-08-13 17:22:49.974955 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974920, event: dispatched, request: pg_info(1 pgs e194471:22.1f1) v4 -1494> 2014-08-13 17:22:49.974966 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974966, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.1f1) v4 -1493> 2014-08-13 17:22:49.974979 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.974979, event: started, request: pg_info(1 pgs e194471:22.1f1) v4 -1492> 2014-08-13 17:22:49.975013 7f74d8735700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:22:49.975013, event: done, request: pg_info(1 pgs e194471:22.1f1) v4 -1491> 2014-08-13 17:22:49.975052 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.5 192.168.1.103:6827/40192 -- pg_info(1 pgs e194471:22.1f1) v4 -- ?+0 0x466ec80 -1490> 2014-08-13 17:22:49.975102 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.47 192.168.1.111:6828/12102 -- pg_info(1 pgs e194471:22.1f1) v4 -- ?+0 0x49e2680 -1489> 2014-08-13 17:22:49.975515 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194471/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.132333 9 0.001767 -1488> 2014-08-13 17:22:49.975551 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194471/194466 194470/194470/194470) [4,5,47] r=0 lpr=194470 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1487> 2014-08-13 17:22:49.985939 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.304 192.168.100.101:0/22782 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:49.985617) v2 ==== 47+0+0 (3626964187 0 0) 0x49e16c0 con 0x49040a0 -1486> 2014-08-13 17:22:49.985992 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:49.985617) v2 -- ?+0 0x4ac7540 con 0x49040a0 -1485> 2014-08-13 17:22:49.986014 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.304 192.168.100.101:0/22782 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:49.985617) v2 ==== 47+0+0 (3626964187 0 0) 0x49e1dc0 con 0x4901a20 -1484> 2014-08-13 17:22:49.986059 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:49.985617) v2 -- ?+0 0x4ac0380 con 0x4901a20 -1483> 2014-08-13 17:22:49.998536 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4801d40 con 0x4900420 -1482> 2014-08-13 17:22:49.998582 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1481> 2014-08-13 17:22:49.998640 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 5 ==== pg_notify(22.775(22) epoch 194471) v5 ==== 1948+0+0 (2086923034 0 0) 0x5262f40 con 0x4900420 -1480> 2014-08-13 17:22:49.998666 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998518, event: header_read, request: pg_notify(22.775(22) epoch 194471) v5 -1479> 2014-08-13 17:22:49.998678 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998519, event: throttled, request: pg_notify(22.775(22) epoch 194471) v5 -1478> 2014-08-13 17:22:49.998684 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998620, event: all_read, request: pg_notify(22.775(22) epoch 194471) v5 -1477> 2014-08-13 17:22:49.998690 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998662, event: dispatched, request: pg_notify(22.775(22) epoch 194471) v5 -1476> 2014-08-13 17:22:49.998697 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998697, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194471) v5 -1475> 2014-08-13 17:22:49.998707 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998706, event: started, request: pg_notify(22.775(22) epoch 194471) v5 -1474> 2014-08-13 17:22:49.998749 7f74d8735700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:22:49.998748, event: done, request: pg_notify(22.775(22) epoch 194471) v5 -1473> 2014-08-13 17:22:50.011092 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 11 ==== pg_info(1 pgs e194471:1.2b) v4 ==== 706+0+0 (1221177821 0 0) 0x53bfa80 con 0x47a3f40 -1472> 2014-08-13 17:22:50.011136 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.010926, event: header_read, request: pg_info(1 pgs e194471:1.2b) v4 -1471> 2014-08-13 17:22:50.011148 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.010928, event: throttled, request: pg_info(1 pgs e194471:1.2b) v4 -1470> 2014-08-13 17:22:50.011166 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.011021, event: all_read, request: pg_info(1 pgs e194471:1.2b) v4 -1469> 2014-08-13 17:22:50.011172 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.011130, event: dispatched, request: pg_info(1 pgs e194471:1.2b) v4 -1468> 2014-08-13 17:22:50.011179 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.011179, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:1.2b) v4 -1467> 2014-08-13 17:22:50.011188 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.011188, event: started, request: pg_info(1 pgs e194471:1.2b) v4 -1466> 2014-08-13 17:22:50.011226 7f74d8735700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:22:50.011225, event: done, request: pg_info(1 pgs e194471:1.2b) v4 -1465> 2014-08-13 17:22:50.011257 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:1.2b) v4 -- ?+0 0x49e7540 -1464> 2014-08-13 17:22:50.011435 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.163591 7 0.004441 -1463> 2014-08-13 17:22:50.011456 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.56 192.168.1.111:6826/11446 3 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4804c80 con 0x475aec0 -1462> 2014-08-13 17:22:50.011481 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1461> 2014-08-13 17:22:50.011471 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1460> 2014-08-13 17:22:50.011496 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000025 0 0.000000 -1459> 2014-08-13 17:22:50.011520 7f74d192a700 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194465 194470/194470/194470) [4,340] r=0 lpr=194470 pi=194448-194469/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1458> 2014-08-13 17:22:50.011541 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.56 192.168.1.111:6826/11446 4 ==== pg_info(1 pgs e194471:22.775) v4 ==== 714+0+0 (2810754400 0 0) 0x53bbf00 con 0x475aec0 -1457> 2014-08-13 17:22:50.011554 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194471:1.2b) v4 -- ?+0 0x49e0000 -1456> 2014-08-13 17:22:50.011562 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011455, event: header_read, request: pg_info(1 pgs e194471:22.775) v4 -1455> 2014-08-13 17:22:50.011580 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011456, event: throttled, request: pg_info(1 pgs e194471:22.775) v4 -1454> 2014-08-13 17:22:50.011591 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011522, event: all_read, request: pg_info(1 pgs e194471:22.775) v4 -1453> 2014-08-13 17:22:50.011597 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011559, event: dispatched, request: pg_info(1 pgs e194471:22.775) v4 -1452> 2014-08-13 17:22:50.011603 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011603, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.775) v4 -1451> 2014-08-13 17:22:50.011614 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011614, event: started, request: pg_info(1 pgs e194471:22.775) v4 -1450> 2014-08-13 17:22:50.011646 7f74d8735700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:22:50.011645, event: done, request: pg_info(1 pgs e194471:22.775) v4 -1449> 2014-08-13 17:22:50.011763 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.56 192.168.1.111:6826/11446 -- pg_info(1 pgs e194471:22.775) v4 -- ?+0 0x57d1a40 -1448> 2014-08-13 17:22:50.011811 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.353 192.168.1.108:6836/6227 -- pg_info(1 pgs e194471:22.775) v4 -- ?+0 0x57d40c0 -1447> 2014-08-13 17:22:50.011885 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194471; pg1.2b; ) v2 -- ?+0 0x53bc280 -1446> 2014-08-13 17:22:50.011931 7f74dcde8700 1 -- 192.168.1.115:6813/30165 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194471; pg1.2b; ) v2 -- ?+0 0x53bd240 -1445> 2014-08-13 17:22:50.012257 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194471/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.177710 9 0.001406 -1444> 2014-08-13 17:22:50.012297 7f74d0f29700 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194471/194466 194470/194470/194470) [4,353,56] r=0 lpr=194470 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1443> 2014-08-13 17:22:50.013463 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4de1b00 con 0x4d54200 -1442> 2014-08-13 17:22:50.013509 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1441> 2014-08-13 17:22:50.013562 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 5 ==== pg_notify(22.f3(18) epoch 194471) v5 ==== 1720+0+0 (614740019 0 0) 0x53b8000 con 0x4d54200 -1440> 2014-08-13 17:22:50.013584 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013403, event: header_read, request: pg_notify(22.f3(18) epoch 194471) v5 -1439> 2014-08-13 17:22:50.013599 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013404, event: throttled, request: pg_notify(22.f3(18) epoch 194471) v5 -1438> 2014-08-13 17:22:50.013609 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013494, event: all_read, request: pg_notify(22.f3(18) epoch 194471) v5 -1437> 2014-08-13 17:22:50.013618 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013580, event: dispatched, request: pg_notify(22.f3(18) epoch 194471) v5 -1436> 2014-08-13 17:22:50.013629 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013628, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194471) v5 -1435> 2014-08-13 17:22:50.013642 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013641, event: started, request: pg_notify(22.f3(18) epoch 194471) v5 -1434> 2014-08-13 17:22:50.013679 7f74d8735700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:22:50.013679, event: done, request: pg_notify(22.f3(18) epoch 194471) v5 -1433> 2014-08-13 17:22:50.018521 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 4 ==== osd_map(194471..194471 src has 185139..194471) v3 ==== 1194+0+0 (1052920059 0 0) 0x4e33cc0 con 0x4d58000 -1432> 2014-08-13 17:22:50.018569 7f74d8735700 3 osd.4 194471 handle_osd_map epochs [194471,194471], i have 194471, src has [185139,194471] -1431> 2014-08-13 17:22:50.018651 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 5 ==== pg_notify(22.481(39) epoch 194471) v5 ==== 2903+0+0 (1888083838 0 0) 0x4ac39c0 con 0x4d58000 -1430> 2014-08-13 17:22:50.018682 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018480, event: header_read, request: pg_notify(22.481(39) epoch 194471) v5 -1429> 2014-08-13 17:22:50.018694 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018481, event: throttled, request: pg_notify(22.481(39) epoch 194471) v5 -1428> 2014-08-13 17:22:50.018701 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018603, event: all_read, request: pg_notify(22.481(39) epoch 194471) v5 -1427> 2014-08-13 17:22:50.018706 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018678, event: dispatched, request: pg_notify(22.481(39) epoch 194471) v5 -1426> 2014-08-13 17:22:50.018713 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018713, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194471) v5 -1425> 2014-08-13 17:22:50.018726 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018726, event: started, request: pg_notify(22.481(39) epoch 194471) v5 -1424> 2014-08-13 17:22:50.018763 7f74d8735700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:22:50.018763, event: done, request: pg_notify(22.481(39) epoch 194471) v5 -1423> 2014-08-13 17:22:50.141601 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.141095) v2 ==== 47+0+0 (2290275617 0 0) 0x53b8e00 con 0x4d627e0 -1422> 2014-08-13 17:22:50.141655 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.141095) v2 -- ?+0 0x49e16c0 con 0x4d627e0 -1421> 2014-08-13 17:22:50.141666 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.141095) v2 ==== 47+0+0 (2290275617 0 0) 0x53bec80 con 0x4d61760 -1420> 2014-08-13 17:22:50.141699 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.141095) v2 -- ?+0 0x49e1dc0 con 0x4d61760 -1419> 2014-08-13 17:22:50.158135 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.294 192.168.1.101:0/23402 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.157837) v2 ==== 47+0+0 (3944338174 0 0) 0x53bde80 con 0x569fa60 -1418> 2014-08-13 17:22:50.158149 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.294 192.168.1.101:0/23402 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.157837) v2 ==== 47+0+0 (3944338174 0 0) 0x53bfc40 con 0x569e300 -1417> 2014-08-13 17:22:50.158185 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.157837) v2 -- ?+0 0x53b8e00 con 0x569fa60 -1416> 2014-08-13 17:22:50.158295 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.157837) v2 -- ?+0 0x53bec80 con 0x569e300 -1415> 2014-08-13 17:22:50.185450 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 9 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1060053482 0 0) 0x33f5800 con 0x475a940 -1414> 2014-08-13 17:22:50.185503 7f74d8735700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185248, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1413> 2014-08-13 17:22:50.185519 7f74d8735700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185252, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1412> 2014-08-13 17:22:50.185529 7f74d8735700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185403, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1411> 2014-08-13 17:22:50.185537 7f74d8735700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185497, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1410> 2014-08-13 17:22:50.185548 7f74d8735700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185548, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1409> 2014-08-13 17:22:50.185678 7f74d192a700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185678, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1408> 2014-08-13 17:22:50.185725 7f74d192a700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185725, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1407> 2014-08-13 17:22:50.185751 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- osd_sub_op_reply(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x485d000 con 0x475a940 -1406> 2014-08-13 17:22:50.185788 7f74d192a700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:22:50.185788, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1405> 2014-08-13 17:22:50.201979 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 10 ==== replica scrub(pg: 22.791,from:0'0,to:194215'517,epoch:194471,start:0//0//-1,end:285adf91//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (796631499 0 0) 0x4a83f00 con 0x475a940 -1404> 2014-08-13 17:22:50.229485 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.376 192.168.1.108:0/6893 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.228875) v2 ==== 47+0+0 (4074857103 0 0) 0x53b9dc0 con 0x56998c0 -1403> 2014-08-13 17:22:50.229496 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.376 192.168.1.108:0/6893 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.228875) v2 ==== 47+0+0 (4074857103 0 0) 0x53b8700 con 0x5698580 -1402> 2014-08-13 17:22:50.229544 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.228875) v2 -- ?+0 0x53bde80 con 0x56998c0 -1401> 2014-08-13 17:22:50.229621 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.228875) v2 -- ?+0 0x53bfc40 con 0x5698580 -1400> 2014-08-13 17:22:50.272001 7f74cfb27700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11499 0x33f2c00 con 0x475a940 -1399> 2014-08-13 17:22:50.324639 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 11 ==== replica scrub(pg: 22.791,from:0'0,to:194215'518,epoch:194471,start:285adf91//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2222445434 0 0) 0x4c74380 con 0x475a940 -1398> 2014-08-13 17:22:50.325609 7f74cfb27700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+503 0x33f6e00 con 0x475a940 -1397> 2014-08-13 17:22:50.337258 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 12 ==== osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2927555151 0 0) 0x33f6e00 con 0x475a940 -1396> 2014-08-13 17:22:50.337311 7f74d8735700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337044, event: header_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1395> 2014-08-13 17:22:50.337327 7f74d8735700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337047, event: throttled, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1394> 2014-08-13 17:22:50.337337 7f74d8735700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337176, event: all_read, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1393> 2014-08-13 17:22:50.337346 7f74d8735700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337305, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1392> 2014-08-13 17:22:50.337356 7f74d8735700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337356, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1391> 2014-08-13 17:22:50.337395 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 13 ==== pg_info(1 pgs e194471:22.791) v4 ==== 714+0+0 (4205193750 0 0) 0x5264280 con 0x475a940 -1390> 2014-08-13 17:22:50.337425 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337200, event: header_read, request: pg_info(1 pgs e194471:22.791) v4 -1389> 2014-08-13 17:22:50.337433 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337201, event: throttled, request: pg_info(1 pgs e194471:22.791) v4 -1388> 2014-08-13 17:22:50.337438 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337276, event: all_read, request: pg_info(1 pgs e194471:22.791) v4 -1387> 2014-08-13 17:22:50.337443 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337423, event: dispatched, request: pg_info(1 pgs e194471:22.791) v4 -1386> 2014-08-13 17:22:50.337449 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337449, event: waiting_for_osdmap, request: pg_info(1 pgs e194471:22.791) v4 -1385> 2014-08-13 17:22:50.337457 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337457, event: started, request: pg_info(1 pgs e194471:22.791) v4 -1384> 2014-08-13 17:22:50.337465 7f74d0f29700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337464, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1383> 2014-08-13 17:22:50.337512 7f74d0f29700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337512, event: started, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1382> 2014-08-13 17:22:50.337533 7f74d0f29700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:22:50.337532, event: done, request: osd_sub_op(unknown.0.0:0 22.791 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1381> 2014-08-13 17:22:50.337600 7f74d8735700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:22:50.337600, event: done, request: pg_info(1 pgs e194471:22.791) v4 -1380> 2014-08-13 17:22:50.373318 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.346 192.168.100.104:0/2308 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.373235) v2 ==== 47+0+0 (1548754154 0 0) 0x5262d80 con 0x5641ce0 -1379> 2014-08-13 17:22:50.373367 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.373235) v2 -- ?+0 0x53b8700 con 0x5641ce0 -1378> 2014-08-13 17:22:50.373366 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.346 192.168.100.104:0/2308 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.373235) v2 ==== 47+0+0 (1548754154 0 0) 0x53babc0 con 0x569ef60 -1377> 2014-08-13 17:22:50.373502 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.373235) v2 -- ?+0 0x53b9dc0 con 0x569ef60 -1376> 2014-08-13 17:22:50.534220 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.364 192.168.100.114:0/52317 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.534081) v2 ==== 47+0+0 (2735589933 0 0) 0x53b8fc0 con 0x569dd80 -1375> 2014-08-13 17:22:50.534266 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.534081) v2 -- ?+0 0x5262d80 con 0x569dd80 -1374> 2014-08-13 17:22:50.534273 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.364 192.168.100.114:0/52317 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.534081) v2 ==== 47+0+0 (2735589933 0 0) 0x53bc440 con 0x569f4e0 -1373> 2014-08-13 17:22:50.534323 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.534081) v2 -- ?+0 0x53babc0 con 0x569f4e0 -1372> 2014-08-13 17:22:50.537297 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 9 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (714500162 0 0) 0x33f2100 con 0x48dfbc0 -1371> 2014-08-13 17:22:50.537347 7f74d8735700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537087, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1370> 2014-08-13 17:22:50.537368 7f74d8735700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537090, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1369> 2014-08-13 17:22:50.537377 7f74d8735700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537211, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1368> 2014-08-13 17:22:50.537386 7f74d8735700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537341, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1367> 2014-08-13 17:22:50.537401 7f74d8735700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537400, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1366> 2014-08-13 17:22:50.537503 7f74d0f29700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537503, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1365> 2014-08-13 17:22:50.537544 7f74d0f29700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537544, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1364> 2014-08-13 17:22:50.537566 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x5acb700 con 0x48dfbc0 -1363> 2014-08-13 17:22:50.537595 7f74d0f29700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:22:50.537595, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1362> 2014-08-13 17:22:50.540029 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 10 ==== replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194471,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (2440104137 0 0) 0x4a81f80 con 0x48dfbc0 -1361> 2014-08-13 17:22:50.597685 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.396 192.168.1.105:0/41806 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.597258) v2 ==== 47+0+0 (2353539753 0 0) 0x53bb640 con 0x569c4c0 -1360> 2014-08-13 17:22:50.597695 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.396 192.168.1.105:0/41806 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.597258) v2 ==== 47+0+0 (2353539753 0 0) 0x53b9500 con 0x569b020 -1359> 2014-08-13 17:22:50.597736 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.105:0/41806 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.597258) v2 -- ?+0 0x53bc440 con 0x569c4c0 -1358> 2014-08-13 17:22:50.597787 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.105:0/41806 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.597258) v2 -- ?+0 0x53b8fc0 con 0x569b020 -1357> 2014-08-13 17:22:50.715696 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.337 192.168.1.104:0/62197 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.715899) v2 ==== 47+0+0 (1072967033 0 0) 0x53bb9c0 con 0x569b2e0 -1356> 2014-08-13 17:22:50.715748 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.715899) v2 -- ?+0 0x53b9500 con 0x569b2e0 -1355> 2014-08-13 17:22:50.715735 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.337 192.168.1.104:0/62197 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.715899) v2 ==== 47+0+0 (1072967033 0 0) 0x53bd5c0 con 0x569c780 -1354> 2014-08-13 17:22:50.715808 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.715899) v2 -- ?+0 0x53bb640 con 0x569c780 -1353> 2014-08-13 17:22:50.840102 7f74d9b37700 1 -- 192.168.100.115:6805/30165 <== mon.2 192.168.100.113:6789/0 18 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e30240 con 0x45d48e0 -1352> 2014-08-13 17:22:50.840205 7f74d9b37700 3 osd.4 194471 handle_osd_map epochs [194472,194472], i have 194471, src has [185139,194472] -1351> 2014-08-13 17:22:50.845711 7f74d9b37700 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for mons -1350> 2014-08-13 17:22:50.848328 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4800fc0 con 0x4835ac0 -1349> 2014-08-13 17:22:50.862963 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.397 192.168.1.105:0/41085 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.862621) v2 ==== 47+0+0 (3129260190 0 0) 0x5c41dc0 con 0x569e880 -1348> 2014-08-13 17:22:50.863016 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.862621) v2 -- ?+0 0x53bb9c0 con 0x569e880 -1347> 2014-08-13 17:22:50.863251 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.397 192.168.1.105:0/41085 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:50.862621) v2 ==== 47+0+0 (3129260190 0 0) 0x5c41f80 con 0x569bf40 -1346> 2014-08-13 17:22:50.863302 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.862621) v2 -- ?+0 0x53bd5c0 con 0x569bf40 -1345> 2014-08-13 17:22:50.876709 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.346 192.168.100.104:0/2308 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:50.876748) v2 ==== 47+0+0 (71503875 0 0) 0x5c44280 con 0x5641ce0 -1344> 2014-08-13 17:22:50.876761 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.876748) v2 -- ?+0 0x5c41dc0 con 0x5641ce0 -1343> 2014-08-13 17:22:50.876842 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.346 192.168.100.104:0/2308 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:50.876748) v2 ==== 47+0+0 (71503875 0 0) 0x5c44600 con 0x569ef60 -1342> 2014-08-13 17:22:50.876896 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.876748) v2 -- ?+0 0x5c41f80 con 0x569ef60 -1341> 2014-08-13 17:22:50.894550 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:50.893821) v2 ==== 47+0+0 (1999447948 0 0) 0x5c46580 con 0x4902c00 -1340> 2014-08-13 17:22:50.894603 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.893821) v2 -- ?+0 0x5c44280 con 0x4902c00 -1339> 2014-08-13 17:22:50.894631 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:50.893821) v2 ==== 47+0+0 (1999447948 0 0) 0x5c46740 con 0x4902ec0 -1338> 2014-08-13 17:22:50.894686 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:50.893821) v2 -- ?+0 0x5c44600 con 0x4902ec0 -1337> 2014-08-13 17:22:51.112318 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:51.112144) v2 ==== 47+0+0 (542762072 0 0) 0x5deabc0 con 0x58a7220 -1336> 2014-08-13 17:22:51.112348 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194471 stamp 2014-08-13 17:22:51.112144) v2 ==== 47+0+0 (542762072 0 0) 0x5dead80 con 0x58c39c0 -1335> 2014-08-13 17:22:51.112385 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.112144) v2 -- ?+0 0x5c46580 con 0x58a7220 -1334> 2014-08-13 17:22:51.112747 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.112144) v2 -- ?+0 0x5c46740 con 0x58c39c0 -1333> 2014-08-13 17:22:51.124360 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.124182) v2 ==== 47+0+0 (1605009766 0 0) 0x5deb640 con 0x49077a0 -1332> 2014-08-13 17:22:51.124390 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.124182) v2 ==== 47+0+0 (1605009766 0 0) 0x5deb2c0 con 0x4905ee0 -1331> 2014-08-13 17:22:51.124417 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.124182) v2 -- ?+0 0x5dead80 con 0x49077a0 -1330> 2014-08-13 17:22:51.124472 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.124182) v2 -- ?+0 0x5deabc0 con 0x4905ee0 -1329> 2014-08-13 17:22:51.252317 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.251786) v2 ==== 47+0+0 (151356543 0 0) 0x5debf00 con 0x4906460 -1328> 2014-08-13 17:22:51.252333 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.251786) v2 ==== 47+0+0 (151356543 0 0) 0x5debd40 con 0x49053e0 -1327> 2014-08-13 17:22:51.252369 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.251786) v2 -- ?+0 0x5deb2c0 con 0x4906460 -1326> 2014-08-13 17:22:51.252584 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.251786) v2 -- ?+0 0x5deb640 con 0x49053e0 -1325> 2014-08-13 17:22:51.260296 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.294 192.168.1.101:0/23402 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.259942) v2 ==== 47+0+0 (3988532676 0 0) 0x5dec280 con 0x569e300 -1324> 2014-08-13 17:22:51.260327 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.294 192.168.1.101:0/23402 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.259942) v2 ==== 47+0+0 (3988532676 0 0) 0x5dec600 con 0x569fa60 -1323> 2014-08-13 17:22:51.260353 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.259942) v2 -- ?+0 0x5debf00 con 0x569e300 -1322> 2014-08-13 17:22:51.260481 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.259942) v2 -- ?+0 0x5debd40 con 0x569fa60 -1321> 2014-08-13 17:22:51.266460 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.321 192.168.1.110:0/43236 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.265870) v2 ==== 47+0+0 (1165203671 0 0) 0x53b8e00 con 0x56982c0 -1320> 2014-08-13 17:22:51.266513 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.265870) v2 -- ?+0 0x5dec280 con 0x56982c0 -1319> 2014-08-13 17:22:51.266535 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.321 192.168.1.110:0/43236 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.265870) v2 ==== 47+0+0 (1165203671 0 0) 0x53bec80 con 0x5699600 -1318> 2014-08-13 17:22:51.266589 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.265870) v2 -- ?+0 0x5dec600 con 0x5699600 -1317> 2014-08-13 17:22:51.423439 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.423401) v2 ==== 47+0+0 (1781729169 0 0) 0x5dec980 con 0x5698dc0 -1316> 2014-08-13 17:22:51.423496 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.423401) v2 -- ?+0 0x53b8e00 con 0x5698dc0 -1315> 2014-08-13 17:22:51.423527 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.423401) v2 ==== 47+0+0 (1781729169 0 0) 0x5decb40 con 0x58c3700 -1314> 2014-08-13 17:22:51.423569 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.423401) v2 -- ?+0 0x53bec80 con 0x58c3700 -1313> 2014-08-13 17:22:51.553616 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.356 192.168.1.108:0/5235 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.552921) v2 ==== 47+0+0 (448804783 0 0) 0x5decd00 con 0x49074e0 -1312> 2014-08-13 17:22:51.553629 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.356 192.168.1.108:0/5235 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.552921) v2 ==== 47+0+0 (448804783 0 0) 0x5decec0 con 0x4907a60 -1311> 2014-08-13 17:22:51.553671 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.552921) v2 -- ?+0 0x5decb40 con 0x49074e0 -1310> 2014-08-13 17:22:51.553758 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.552921) v2 -- ?+0 0x5dec980 con 0x4907a60 -1309> 2014-08-13 17:22:51.651780 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.651612) v2 ==== 47+0+0 (3002610500 0 0) 0x5ded080 con 0x569b5a0 -1308> 2014-08-13 17:22:51.651836 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.651612) v2 -- ?+0 0x5decec0 con 0x569b5a0 -1307> 2014-08-13 17:22:51.651888 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.651612) v2 ==== 47+0+0 (3002610500 0 0) 0x5ded240 con 0x569ca40 -1306> 2014-08-13 17:22:51.651929 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.651612) v2 -- ?+0 0x5decd00 con 0x569ca40 -1305> 2014-08-13 17:22:51.682049 7f74d2d2c700 10 monclient: tick -1304> 2014-08-13 17:22:51.682110 7f74d2d2c700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 17:22:21.682108) -1303> 2014-08-13 17:22:51.682124 7f74d2d2c700 10 monclient: renew subs? (now: 2014-08-13 17:22:51.682124; renew after: 2014-08-13 17:25:18.793245) -- no -1302> 2014-08-13 17:22:51.754890 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.58 192.168.1.111:0/14316 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.754117) v2 ==== 47+0+0 (9008404 0 0) 0x5ded5c0 con 0x4906460 -1301> 2014-08-13 17:22:51.754946 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.754117) v2 -- ?+0 0x5ded080 con 0x4906460 -1300> 2014-08-13 17:22:51.754964 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.58 192.168.1.111:0/14316 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.754117) v2 ==== 47+0+0 (9008404 0 0) 0x5ded940 con 0x49053e0 -1299> 2014-08-13 17:22:51.755006 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.754117) v2 -- ?+0 0x5ded240 con 0x49053e0 -1298> 2014-08-13 17:22:51.843412 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.320 192.168.1.106:0/10050 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.842786) v2 ==== 47+0+0 (2173038579 0 0) 0x5ded080 con 0x4d627e0 -1297> 2014-08-13 17:22:51.843460 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.842786) v2 -- ?+0 0x5ded940 con 0x4d627e0 -1296> 2014-08-13 17:22:51.843547 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.320 192.168.1.106:0/10050 2 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.842786) v2 ==== 47+0+0 (2173038579 0 0) 0x5ded240 con 0x4d61760 -1295> 2014-08-13 17:22:51.843658 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.842786) v2 -- ?+0 0x5ded5c0 con 0x4d61760 -1294> 2014-08-13 17:22:51.904765 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.292 192.168.1.113:0/16056 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.904715) v2 ==== 47+0+0 (1297397248 0 0) 0x49e1dc0 con 0x569e5c0 -1293> 2014-08-13 17:22:51.904814 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.904715) v2 -- ?+0 0x5ded240 con 0x569e5c0 -1292> 2014-08-13 17:22:51.904814 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.292 192.168.1.113:0/16056 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.904715) v2 ==== 47+0+0 (1297397248 0 0) 0x49e16c0 con 0x569fd20 -1291> 2014-08-13 17:22:51.904862 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194471 stamp 2014-08-13 17:22:51.904715) v2 -- ?+0 0x5ded080 con 0x569fd20 -1290> 2014-08-13 17:22:51.956456 7f74cfb27700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.105:6808/41085 -- osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11549 0x61b5800 con 0x48dfbc0 -1289> 2014-08-13 17:22:51.956805 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 2.084108 4 0.000241 -1288> 2014-08-13 17:22:51.956862 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 2.154232 0 0.000000 -1287> 2014-08-13 17:22:51.956896 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active -1286> 2014-08-13 17:22:51.957560 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x59b0580 -- 0x567e900 -1285> 2014-08-13 17:22:51.957640 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x59b02c0 -- 0x567e680 -1284> 2014-08-13 17:22:51.957681 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x5a10b00 -- 0x567fa80 -1283> 2014-08-13 17:22:51.957681 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 2.083106 3 0.000225 -1282> 2014-08-13 17:22:51.957740 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 2.123346 0 0.000000 -1281> 2014-08-13 17:22:51.957775 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active -1280> 2014-08-13 17:22:51.957731 7f749a52e700 2 -- 192.168.1.115:0/30165 >> 192.168.1.115:6839/62713 pipe(0x567e900 sd=517 :50002 s=4 pgs=137 cs=1 l=1 c=0x59b0580).reader couldn't read tag, (0) Success -1279> 2014-08-13 17:22:51.957975 7f749a52e700 2 -- 192.168.1.115:0/30165 >> 192.168.1.115:6839/62713 pipe(0x567e900 sd=517 :50002 s=4 pgs=137 cs=1 l=1 c=0x59b0580).fault (0) Success -1278> 2014-08-13 17:22:51.957759 7f749941d700 2 -- 192.168.1.115:0/30165 >> 192.168.1.115:6814/59915 pipe(0x567fa80 sd=522 :36211 s=4 pgs=123 cs=1 l=1 c=0x5a10b00).reader couldn't read tag, (0) Success -1277> 2014-08-13 17:22:51.957867 7f7499d26700 2 -- 192.168.1.115:0/30165 >> 192.168.100.115:6846/62713 pipe(0x567e680 sd=513 :52007 s=4 pgs=137 cs=1 l=1 c=0x59b02c0).reader couldn't read tag, (0) Success -1276> 2014-08-13 17:22:51.958158 7f749941d700 2 -- 192.168.1.115:0/30165 >> 192.168.1.115:6814/59915 pipe(0x567fa80 sd=522 :36211 s=4 pgs=123 cs=1 l=1 c=0x5a10b00).fault (0) Success -1275> 2014-08-13 17:22:51.958168 7f7499d26700 2 -- 192.168.1.115:0/30165 >> 192.168.100.115:6846/62713 pipe(0x567e680 sd=513 :52007 s=4 pgs=137 cs=1 l=1 c=0x59b02c0).fault (0) Success -1274> 2014-08-13 17:22:51.957715 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x5a10840 -- 0x5ac9180 -1273> 2014-08-13 17:22:51.959378 7f74d192a700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 -1272> 2014-08-13 17:22:51.959364 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d4bb20 -- 0x5701900 -1271> 2014-08-13 17:22:51.959426 7f74d9b37700 1 -- 192.168.1.115:0/30165 mark_down 0x4d4b860 -- 0x5705000 -1270> 2014-08-13 17:22:51.959382 7f749931c700 2 -- 192.168.1.115:0/30165 >> 192.168.100.115:6831/59915 pipe(0x5ac9180 sd=523 :44989 s=4 pgs=123 cs=1 l=1 c=0x5a10840).reader couldn't read tag, (0) Success -1269> 2014-08-13 17:22:51.959453 7f749931c700 2 -- 192.168.1.115:0/30165 >> 192.168.100.115:6831/59915 pipe(0x5ac9180 sd=523 :44989 s=4 pgs=123 cs=1 l=1 c=0x5a10840).fault (0) Success -1268> 2014-08-13 17:22:51.959513 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1267> 2014-08-13 17:22:51.959465 7f74aef77700 2 -- 192.168.1.115:0/30165 >> 192.168.1.114:6843/56512 pipe(0x5701900 sd=318 :39656 s=4 pgs=98 cs=1 l=1 c=0x4d4bb20).reader couldn't read tag, (0) Success -1266> 2014-08-13 17:22:51.959561 7f74aef77700 2 -- 192.168.1.115:0/30165 >> 192.168.1.114:6843/56512 pipe(0x5701900 sd=318 :39656 s=4 pgs=98 cs=1 l=1 c=0x4d4bb20).fault (0) Success -1265> 2014-08-13 17:22:51.959558 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a869c0 con 0x47f0c60 -1264> 2014-08-13 17:22:51.959595 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1263> 2014-08-13 17:22:51.959610 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a84380 con 0x47f4620 -1262> 2014-08-13 17:22:51.959630 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1261> 2014-08-13 17:22:51.959643 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a82400 con 0x47f6880 -1260> 2014-08-13 17:22:51.959661 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1259> 2014-08-13 17:22:51.959674 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 13 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de0900 con 0x47f7e80 -1258> 2014-08-13 17:22:51.959692 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1257> 2014-08-13 17:22:51.959738 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4806300 con 0x47f70c0 -1256> 2014-08-13 17:22:51.959757 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1255> 2014-08-13 17:22:51.959770 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.51 192.168.1.112:6832/6655 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c73cc0 con 0x4831b80 -1254> 2014-08-13 17:22:51.959475 7f74ae971700 2 -- 192.168.1.115:0/30165 >> 192.168.100.114:6850/56512 pipe(0x5705000 sd=319 :60817 s=4 pgs=98 cs=1 l=1 c=0x4d4b860).reader couldn't read tag, (0) Success -1253> 2014-08-13 17:22:51.959788 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1252> 2014-08-13 17:22:51.959801 7f74ae971700 2 -- 192.168.1.115:0/30165 >> 192.168.100.114:6850/56512 pipe(0x5705000 sd=319 :60817 s=4 pgs=98 cs=1 l=1 c=0x4d4b860).fault (0) Success -1251> 2014-08-13 17:22:51.959812 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 14 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4807980 con 0x4833440 -1250> 2014-08-13 17:22:51.959828 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1249> 2014-08-13 17:22:51.959841 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c72880 con 0x4832c00 -1248> 2014-08-13 17:22:51.959859 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1247> 2014-08-13 17:22:51.959871 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c73840 con 0x4830dc0 -1246> 2014-08-13 17:22:51.959888 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1245> 2014-08-13 17:22:51.959901 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 8 ==== pg_info(1 pgs e194472:24.cd(4)) v4 ==== 968+0+0 (3333960549 0 0) 0x53b9f80 con 0x4835ac0 -1244> 2014-08-13 17:22:51.959921 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:50.848259, event: header_read, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1243> 2014-08-13 17:22:51.959935 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:50.848265, event: throttled, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1242> 2014-08-13 17:22:51.959945 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:50.848370, event: all_read, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1241> 2014-08-13 17:22:51.959928 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.286 192.168.1.113:6820/19173 -- pg_log(24.30b epoch 194472 log log((192718'15573,194422'18660], crt=194422'18660) query_epoch 194472) v4 -- ?+0 0x5c76000 -1240> 2014-08-13 17:22:51.959959 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:51.959917, event: dispatched, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1239> 2014-08-13 17:22:51.959967 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:51.959967, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1238> 2014-08-13 17:22:51.959978 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:51.959978, event: started, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1237> 2014-08-13 17:22:51.960006 7f74d8735700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:22:51.960006, event: done, request: pg_info(1 pgs e194472:24.cd(4)) v4 -1236> 2014-08-13 17:22:51.960029 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 19 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de6e40 con 0x4835280 -1235> 2014-08-13 17:22:51.960045 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1234> 2014-08-13 17:22:51.960054 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 12 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a80fc0 con 0x48377a0 -1233> 2014-08-13 17:22:51.960068 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1232> 2014-08-13 17:22:51.960077 7f74d0f29700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 -1231> 2014-08-13 17:22:51.960077 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 11 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x2713f00 con 0x4868420 -1230> 2014-08-13 17:22:51.960091 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1229> 2014-08-13 17:22:51.960098 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.292 192.168.1.113:6812/16056 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c76780 con 0x4868160 -1228> 2014-08-13 17:22:51.960111 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1227> 2014-08-13 17:22:51.960118 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 8 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4806780 con 0x486bf40 -1226> 2014-08-13 17:22:51.960132 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1225> 2014-08-13 17:22:51.960140 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 14 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de69c0 con 0x486bc80 -1224> 2014-08-13 17:22:51.960153 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1223> 2014-08-13 17:22:51.960161 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e30b40 con 0x486b9c0 -1222> 2014-08-13 17:22:51.960184 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1221> 2014-08-13 17:22:51.960201 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a83600 con 0x486b700 -1220> 2014-08-13 17:22:51.960216 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1219> 2014-08-13 17:22:51.960224 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c75a00 con 0x486b440 -1218> 2014-08-13 17:22:51.960246 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1217> 2014-08-13 17:22:51.960255 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de0480 con 0x486b180 -1216> 2014-08-13 17:22:51.960268 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1215> 2014-08-13 17:22:51.960276 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 16 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c70000 con 0x486a3c0 -1214> 2014-08-13 17:22:51.960289 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1213> 2014-08-13 17:22:51.960296 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.296 192.168.1.109:6806/31286 4 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de5c40 con 0x486fe80 -1212> 2014-08-13 17:22:51.960309 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1211> 2014-08-13 17:22:51.960317 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a82880 con 0x48d9ce0 -1210> 2014-08-13 17:22:51.960330 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1209> 2014-08-13 17:22:51.960338 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 11 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4804c80 con 0x486e040 -1208> 2014-08-13 17:22:51.960351 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1207> 2014-08-13 17:22:51.960359 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e34800 con 0x48db9c0 -1206> 2014-08-13 17:22:51.960372 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1205> 2014-08-13 17:22:51.960379 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4804380 con 0x48dcd00 -1204> 2014-08-13 17:22:51.960392 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1203> 2014-08-13 17:22:51.960400 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 11 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c76e40 con 0x48dfbc0 -1202> 2014-08-13 17:22:51.960413 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1201> 2014-08-13 17:22:51.960423 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4801b00 con 0x4900420 -1200> 2014-08-13 17:22:51.960437 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1199> 2014-08-13 17:22:51.960445 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a81b00 con 0x4900580 -1198> 2014-08-13 17:22:51.960458 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1197> 2014-08-13 17:22:51.960466 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 8 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c718c0 con 0x4900f20 -1196> 2014-08-13 17:22:51.960479 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1195> 2014-08-13 17:22:51.960486 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 12 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a85100 con 0x4901600 -1194> 2014-08-13 17:22:51.960499 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1193> 2014-08-13 17:22:51.960507 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e34c80 con 0x4902680 -1192> 2014-08-13 17:22:51.960523 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1191> 2014-08-13 17:22:51.960548 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c70480 con 0x49023c0 -1190> 2014-08-13 17:22:51.960565 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1189> 2014-08-13 17:22:51.960573 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e30000 con 0x4902100 -1188> 2014-08-13 17:22:51.960565 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.25 192.168.1.103:6800/36920 -- pg_log(24.3d epoch 194472 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194472) v4 -- ?+0 0x5b19e00 -1187> 2014-08-13 17:22:51.960587 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1186> 2014-08-13 17:22:51.960596 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c77980 con 0x4903700 -1185> 2014-08-13 17:22:51.960609 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1184> 2014-08-13 17:22:51.960616 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e321c0 con 0x4904780 -1183> 2014-08-13 17:22:51.960629 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1182> 2014-08-13 17:22:51.960637 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e30480 con 0x49044c0 -1181> 2014-08-13 17:22:51.960649 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1180> 2014-08-13 17:22:51.960655 7f74d0f29700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 -1179> 2014-08-13 17:22:51.960354 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=194471 pi=193640-194470/26 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating -1178> 2014-08-13 17:22:51.960657 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e31440 con 0x49070c0 -1177> 2014-08-13 17:22:51.960670 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1176> 2014-08-13 17:22:51.960678 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e35580 con 0x4906e00 -1175> 2014-08-13 17:22:51.960691 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1174> 2014-08-13 17:22:51.960698 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 11 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e33f00 con 0x4d58840 -1173> 2014-08-13 17:22:51.960711 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1172> 2014-08-13 17:22:51.960719 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e32ac0 con 0x4d582c0 -1171> 2014-08-13 17:22:51.960732 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1170> 2014-08-13 17:22:51.960740 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e33a80 con 0x4d58000 -1169> 2014-08-13 17:22:51.960753 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1168> 2014-08-13 17:22:51.960760 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e36540 con 0x4d51ce0 -1167> 2014-08-13 17:22:51.960772 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1166> 2014-08-13 17:22:51.960780 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e34380 con 0x4d51a20 -1165> 2014-08-13 17:22:51.960792 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1164> 2014-08-13 17:22:51.960802 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de18c0 con 0x4d54200 -1163> 2014-08-13 17:22:51.960815 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1162> 2014-08-13 17:22:51.960823 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de1440 con 0x4d53c80 -1161> 2014-08-13 17:22:51.960836 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1160> 2014-08-13 17:22:51.960843 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de21c0 con 0x4d56040 -1159> 2014-08-13 17:22:51.960856 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1158> 2014-08-13 17:22:51.960863 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 8 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de2640 con 0x4d55d80 -1157> 2014-08-13 17:22:51.960875 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1156> 2014-08-13 17:22:51.960882 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de2ac0 con 0x4d55ac0 -1155> 2014-08-13 17:22:51.960895 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1154> 2014-08-13 17:22:51.960903 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de4380 con 0x4d55540 -1153> 2014-08-13 17:22:51.960915 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1152> 2014-08-13 17:22:51.960922 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de2f40 con 0x4d57d20 -1151> 2014-08-13 17:22:51.960935 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1150> 2014-08-13 17:22:51.960942 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 5 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de4a40 con 0x4d57a60 -1149> 2014-08-13 17:22:51.960955 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1148> 2014-08-13 17:22:51.960963 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 4 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e11680 con 0x4d2aaa0 -1147> 2014-08-13 17:22:51.960976 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1146> 2014-08-13 17:22:51.960990 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.307 192.168.1.109:6804/31016 4 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e32d00 con 0x5419fa0 -1145> 2014-08-13 17:22:51.961003 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1144> 2014-08-13 17:22:51.961011 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4800240 con 0x45f1760 -1143> 2014-08-13 17:22:51.961024 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1142> 2014-08-13 17:22:51.961034 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 12 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4802ac0 con 0x45f1e40 -1141> 2014-08-13 17:22:51.961047 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1140> 2014-08-13 17:22:51.961054 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4801f80 con 0x45f5540 -1139> 2014-08-13 17:22:51.961067 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1138> 2014-08-13 17:22:51.961075 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 15 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x48045c0 con 0x45f5960 -1137> 2014-08-13 17:22:51.961088 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1136> 2014-08-13 17:22:51.961095 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de7740 con 0x45f6f60 -1135> 2014-08-13 17:22:51.961108 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1134> 2014-08-13 17:22:51.961115 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x48033c0 con 0x4758840 -1133> 2014-08-13 17:22:51.961128 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1132> 2014-08-13 17:22:51.961136 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4805100 con 0x47591e0 -1131> 2014-08-13 17:22:51.961149 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1130> 2014-08-13 17:22:51.961160 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 14 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a86540 con 0x475a940 -1129> 2014-08-13 17:22:51.961183 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1128> 2014-08-13 17:22:51.961191 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 8 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x2713cc0 con 0x475cd00 -1127> 2014-08-13 17:22:51.961213 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1126> 2014-08-13 17:22:51.961222 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.345 192.168.1.110:6820/46337 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4802f40 con 0x475c4c0 -1125> 2014-08-13 17:22:51.961235 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1124> 2014-08-13 17:22:51.961243 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.294 192.168.1.101:6812/23402 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c74a40 con 0x475d960 -1123> 2014-08-13 17:22:51.961256 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1122> 2014-08-13 17:22:51.961263 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 15 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c721c0 con 0x475ee00 -1121> 2014-08-13 17:22:51.961276 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1120> 2014-08-13 17:22:51.961284 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.321 192.168.1.110:6812/43236 7 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4803f00 con 0x475e5c0 -1119> 2014-08-13 17:22:51.961301 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1118> 2014-08-13 17:22:51.961313 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 14 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a85a00 con 0x47a27e0 -1117> 2014-08-13 17:22:51.961330 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1116> 2014-08-13 17:22:51.961341 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 13 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c745c0 con 0x47a0b00 -1115> 2014-08-13 17:22:51.961361 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1114> 2014-08-13 17:22:51.961373 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 6 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4e106c0 con 0x47a4200 -1113> 2014-08-13 17:22:51.961390 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1112> 2014-08-13 17:22:51.961401 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 10 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c75580 con 0x47a53e0 -1111> 2014-08-13 17:22:51.961418 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1110> 2014-08-13 17:22:51.961429 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 12 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4de4140 con 0x47a3f40 -1109> 2014-08-13 17:22:51.961446 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1108> 2014-08-13 17:22:51.961457 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 8 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a87740 con 0x47a4ba0 -1107> 2014-08-13 17:22:51.961474 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1106> 2014-08-13 17:22:51.961485 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 11 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c70d80 con 0x47a32e0 -1105> 2014-08-13 17:22:51.961503 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -1104> 2014-08-13 17:22:51.961515 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 11 ==== pg_info(1 pgs e194472:24.33a(4)) v4 ==== 968+0+0 (2664219780 0 0) 0x4ac3480 con 0x47f0c60 -1103> 2014-08-13 17:22:51.961533 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:50.848346, event: header_read, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1102> 2014-08-13 17:22:51.961545 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:50.848348, event: throttled, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1101> 2014-08-13 17:22:51.961555 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:50.848424, event: all_read, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1100> 2014-08-13 17:22:51.961564 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:51.961530, event: dispatched, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1099> 2014-08-13 17:22:51.961574 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:51.961574, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1098> 2014-08-13 17:22:51.961586 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:51.961586, event: started, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1097> 2014-08-13 17:22:51.961614 7f74d8735700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:22:51.961613, event: done, request: pg_info(1 pgs e194472:24.33a(4)) v4 -1096> 2014-08-13 17:22:51.961643 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.31 192.168.1.103:6831/41157 10 ==== pg_info(1 pgs e194472:24.1e4(11)) v4 ==== 1361+0+0 (3059879230 0 0) 0x57d4980 con 0x47f4620 -1095> 2014-08-13 17:22:51.961663 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:50.850936, event: header_read, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1094> 2014-08-13 17:22:51.961674 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:50.850937, event: throttled, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1093> 2014-08-13 17:22:51.961683 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:50.851026, event: all_read, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1092> 2014-08-13 17:22:51.961692 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:51.961660, event: dispatched, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1091> 2014-08-13 17:22:51.961702 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:51.961702, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1090> 2014-08-13 17:22:51.961713 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:51.961713, event: started, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1089> 2014-08-13 17:22:51.961736 7f74d8735700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:22:51.961736, event: done, request: pg_info(1 pgs e194472:24.1e4(11)) v4 -1088> 2014-08-13 17:22:51.961759 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.315 192.168.1.113:6800/14161 8 ==== pg_notify(24.b6(28) epoch 194472) v5 ==== 2332+0+0 (3142814829 0 0) 0x57d4d00 con 0x47f6880 -1087> 2014-08-13 17:22:51.961778 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:50.910000, event: header_read, request: pg_notify(24.b6(28) epoch 194472) v5 -1086> 2014-08-13 17:22:51.961789 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:50.910001, event: throttled, request: pg_notify(24.b6(28) epoch 194472) v5 -1085> 2014-08-13 17:22:51.961798 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:50.910117, event: all_read, request: pg_notify(24.b6(28) epoch 194472) v5 -1084> 2014-08-13 17:22:51.961807 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:51.961776, event: dispatched, request: pg_notify(24.b6(28) epoch 194472) v5 -1083> 2014-08-13 17:22:51.961817 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:51.961817, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194472) v5 -1082> 2014-08-13 17:22:51.961829 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:51.961829, event: started, request: pg_notify(24.b6(28) epoch 194472) v5 -1081> 2014-08-13 17:22:51.961855 7f74d8735700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:22:51.961854, event: done, request: pg_notify(24.b6(28) epoch 194472) v5 -1080> 2014-08-13 17:22:51.961889 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 14 ==== pg_notify(22.113(22) epoch 194472) v5 ==== 1952+0+0 (3121744053 0 0) 0x5c447c0 con 0x47f7e80 -1079> 2014-08-13 17:22:51.961909 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:50.877682, event: header_read, request: pg_notify(22.113(22) epoch 194472) v5 -1078> 2014-08-13 17:22:51.961919 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:50.877683, event: throttled, request: pg_notify(22.113(22) epoch 194472) v5 -1077> 2014-08-13 17:22:51.961928 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:50.877795, event: all_read, request: pg_notify(22.113(22) epoch 194472) v5 -1076> 2014-08-13 17:22:51.961937 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:51.961906, event: dispatched, request: pg_notify(22.113(22) epoch 194472) v5 -1075> 2014-08-13 17:22:51.961947 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:51.961947, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194472) v5 -1074> 2014-08-13 17:22:51.961958 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:51.961958, event: started, request: pg_notify(22.113(22) epoch 194472) v5 -1073> 2014-08-13 17:22:51.961983 7f74d8735700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:22:51.961982, event: done, request: pg_notify(22.113(22) epoch 194472) v5 -1072> 2014-08-13 17:22:51.962003 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.50 192.168.1.112:6842/9607 11 ==== pg_notify(24.136(2) epoch 194472) v5 ==== 870+0+0 (807308281 0 0) 0x53bc0c0 con 0x47f70c0 -1071> 2014-08-13 17:22:51.962021 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:50.998329, event: header_read, request: pg_notify(24.136(2) epoch 194472) v5 -1070> 2014-08-13 17:22:51.962032 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:50.998330, event: throttled, request: pg_notify(24.136(2) epoch 194472) v5 -1069> 2014-08-13 17:22:51.962041 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:50.998373, event: all_read, request: pg_notify(24.136(2) epoch 194472) v5 -1068> 2014-08-13 17:22:51.962050 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:51.962019, event: dispatched, request: pg_notify(24.136(2) epoch 194472) v5 -1067> 2014-08-13 17:22:51.962060 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:51.962059, event: waiting_for_osdmap, request: pg_notify(24.136(2) epoch 194472) v5 -1066> 2014-08-13 17:22:51.962070 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:51.962070, event: started, request: pg_notify(24.136(2) epoch 194472) v5 -1065> 2014-08-13 17:22:51.962094 7f74d8735700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:22:51.962093, event: done, request: pg_notify(24.136(2) epoch 194472) v5 -1064> 2014-08-13 17:22:51.962125 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 15 ==== pg_notify(22.99(18) epoch 194472) v5 ==== 1720+0+0 (3447818409 0 0) 0x5c46e40 con 0x4833440 -1063> 2014-08-13 17:22:51.962145 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:50.903694, event: header_read, request: pg_notify(22.99(18) epoch 194472) v5 -1062> 2014-08-13 17:22:51.962155 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:50.903695, event: throttled, request: pg_notify(22.99(18) epoch 194472) v5 -1061> 2014-08-13 17:22:51.962164 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:50.903809, event: all_read, request: pg_notify(22.99(18) epoch 194472) v5 -1060> 2014-08-13 17:22:51.962181 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:51.962142, event: dispatched, request: pg_notify(22.99(18) epoch 194472) v5 -1059> 2014-08-13 17:22:51.962191 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:51.962191, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194472) v5 -1058> 2014-08-13 17:22:51.962240 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:51.962239, event: started, request: pg_notify(22.99(18) epoch 194472) v5 -1057> 2014-08-13 17:22:51.962274 7f74d8735700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:22:51.962274, event: done, request: pg_notify(22.99(18) epoch 194472) v5 -1056> 2014-08-13 17:22:51.962314 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.371 192.168.1.102:6832/16753 11 ==== pg_notify(22.4df(20) epoch 194472) v5 ==== 1834+0+0 (2899118731 0 0) 0x57d3f00 con 0x4832c00 -1055> 2014-08-13 17:22:51.962339 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:50.902350, event: header_read, request: pg_notify(22.4df(20) epoch 194472) v5 -1054> 2014-08-13 17:22:51.962350 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:50.902351, event: throttled, request: pg_notify(22.4df(20) epoch 194472) v5 -1053> 2014-08-13 17:22:51.962359 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:50.902447, event: all_read, request: pg_notify(22.4df(20) epoch 194472) v5 -1052> 2014-08-13 17:22:51.962368 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:51.962336, event: dispatched, request: pg_notify(22.4df(20) epoch 194472) v5 -1051> 2014-08-13 17:22:51.962378 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:51.962377, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194472) v5 -1050> 2014-08-13 17:22:51.962389 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:51.962389, event: started, request: pg_notify(22.4df(20) epoch 194472) v5 -1049> 2014-08-13 17:22:51.962412 7f74d8735700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:22:51.962412, event: done, request: pg_notify(22.4df(20) epoch 194472) v5 -1048> 2014-08-13 17:22:51.962445 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.24 192.168.1.107:6800/52669 7 ==== pg_notify(22.113(22) epoch 194472) v5 ==== 1952+0+0 (3121744053 0 0) 0x5c47700 con 0x4830dc0 -1047> 2014-08-13 17:22:51.962465 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:50.907733, event: header_read, request: pg_notify(22.113(22) epoch 194472) v5 -1046> 2014-08-13 17:22:51.962475 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:50.907735, event: throttled, request: pg_notify(22.113(22) epoch 194472) v5 -1045> 2014-08-13 17:22:51.962484 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:50.907838, event: all_read, request: pg_notify(22.113(22) epoch 194472) v5 -1044> 2014-08-13 17:22:51.962493 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:51.962462, event: dispatched, request: pg_notify(22.113(22) epoch 194472) v5 -1043> 2014-08-13 17:22:51.962502 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:51.962502, event: waiting_for_osdmap, request: pg_notify(22.113(22) epoch 194472) v5 -1042> 2014-08-13 17:22:51.962513 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:51.962513, event: started, request: pg_notify(22.113(22) epoch 194472) v5 -1041> 2014-08-13 17:22:51.962536 7f74d8735700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:22:51.962536, event: done, request: pg_notify(22.113(22) epoch 194472) v5 -1040> 2014-08-13 17:22:51.962555 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 20 ==== pg_notify(24.1f7(24) epoch 194472) v5 ==== 2096+0+0 (1145537959 0 0) 0x49e1f80 con 0x4835280 -1039> 2014-08-13 17:22:51.962573 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:50.868936, event: header_read, request: pg_notify(24.1f7(24) epoch 194472) v5 -1038> 2014-08-13 17:22:51.962583 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:50.868938, event: throttled, request: pg_notify(24.1f7(24) epoch 194472) v5 -1037> 2014-08-13 17:22:51.962592 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:50.869051, event: all_read, request: pg_notify(24.1f7(24) epoch 194472) v5 -1036> 2014-08-13 17:22:51.962605 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:51.962571, event: dispatched, request: pg_notify(24.1f7(24) epoch 194472) v5 -1035> 2014-08-13 17:22:51.962615 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:51.962615, event: waiting_for_osdmap, request: pg_notify(24.1f7(24) epoch 194472) v5 -1034> 2014-08-13 17:22:51.962626 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:51.962626, event: started, request: pg_notify(24.1f7(24) epoch 194472) v5 -1033> 2014-08-13 17:22:51.962649 7f74d8735700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:22:51.962649, event: done, request: pg_notify(24.1f7(24) epoch 194472) v5 -1032> 2014-08-13 17:22:51.962680 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 13 ==== pg_info(1 pgs e194472:24.206(4)) v4 ==== 968+0+0 (819059747 0 0) 0x53b88c0 con 0x48377a0 -1031> 2014-08-13 17:22:51.962700 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:50.852443, event: header_read, request: pg_info(1 pgs e194472:24.206(4)) v4 -1030> 2014-08-13 17:22:51.962710 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:50.852444, event: throttled, request: pg_info(1 pgs e194472:24.206(4)) v4 -1029> 2014-08-13 17:22:51.962719 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:50.852507, event: all_read, request: pg_info(1 pgs e194472:24.206(4)) v4 -1028> 2014-08-13 17:22:51.962728 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:51.962698, event: dispatched, request: pg_info(1 pgs e194472:24.206(4)) v4 -1027> 2014-08-13 17:22:51.962738 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:51.962738, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.206(4)) v4 -1026> 2014-08-13 17:22:51.962749 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:51.962749, event: started, request: pg_info(1 pgs e194472:24.206(4)) v4 -1025> 2014-08-13 17:22:51.962773 7f74d8735700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:22:51.962772, event: done, request: pg_info(1 pgs e194472:24.206(4)) v4 -1024> 2014-08-13 17:22:51.962793 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.351 192.168.1.102:6830/16444 12 ==== pg_notify(24.1f7(28) epoch 194472) v5 ==== 2332+0+0 (57694604 0 0) 0x466b9c0 con 0x4868420 -1023> 2014-08-13 17:22:51.962812 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:50.870076, event: header_read, request: pg_notify(24.1f7(28) epoch 194472) v5 -1022> 2014-08-13 17:22:51.962822 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:50.870080, event: throttled, request: pg_notify(24.1f7(28) epoch 194472) v5 -1021> 2014-08-13 17:22:51.962831 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:50.870226, event: all_read, request: pg_notify(24.1f7(28) epoch 194472) v5 -1020> 2014-08-13 17:22:51.962840 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:51.962810, event: dispatched, request: pg_notify(24.1f7(28) epoch 194472) v5 -1019> 2014-08-13 17:22:51.962849 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:51.962849, event: waiting_for_osdmap, request: pg_notify(24.1f7(28) epoch 194472) v5 -1018> 2014-08-13 17:22:51.962860 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:51.962860, event: started, request: pg_notify(24.1f7(28) epoch 194472) v5 -1017> 2014-08-13 17:22:51.962884 7f74d8735700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:22:51.962883, event: done, request: pg_notify(24.1f7(28) epoch 194472) v5 -1016> 2014-08-13 17:22:51.962909 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.331 192.168.1.104:6816/837 9 ==== pg_notify(22.1d2(1) epoch 194472) v5 ==== 777+0+0 (2804950125 0 0) 0x53b9340 con 0x486bf40 -1015> 2014-08-13 17:22:51.962928 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:50.964878, event: header_read, request: pg_notify(22.1d2(1) epoch 194472) v5 -1014> 2014-08-13 17:22:51.962938 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:50.964879, event: throttled, request: pg_notify(22.1d2(1) epoch 194472) v5 -1013> 2014-08-13 17:22:51.962947 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:50.964921, event: all_read, request: pg_notify(22.1d2(1) epoch 194472) v5 -1012> 2014-08-13 17:22:51.962956 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:51.962926, event: dispatched, request: pg_notify(22.1d2(1) epoch 194472) v5 -1011> 2014-08-13 17:22:51.962965 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:51.962965, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194472) v5 -1010> 2014-08-13 17:22:51.962976 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:51.962976, event: started, request: pg_notify(22.1d2(1) epoch 194472) v5 -1009> 2014-08-13 17:22:51.963002 7f74d8735700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:22:51.963002, event: done, request: pg_notify(22.1d2(1) epoch 194472) v5 -1008> 2014-08-13 17:22:51.963022 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 15 ==== pg_notify(22.39d(20) epoch 194472) v5 ==== 1834+0+0 (2070222226 0 0) 0x466aa00 con 0x486bc80 -1007> 2014-08-13 17:22:51.963040 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:50.861373, event: header_read, request: pg_notify(22.39d(20) epoch 194472) v5 -1006> 2014-08-13 17:22:51.963050 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:50.861374, event: throttled, request: pg_notify(22.39d(20) epoch 194472) v5 -1005> 2014-08-13 17:22:51.963060 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:50.861471, event: all_read, request: pg_notify(22.39d(20) epoch 194472) v5 -1004> 2014-08-13 17:22:51.963068 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:51.963038, event: dispatched, request: pg_notify(22.39d(20) epoch 194472) v5 -1003> 2014-08-13 17:22:51.963078 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:51.963078, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194472) v5 -1002> 2014-08-13 17:22:51.963089 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:51.963088, event: started, request: pg_notify(22.39d(20) epoch 194472) v5 -1001> 2014-08-13 17:22:51.963110 7f74d8735700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:22:51.963110, event: done, request: pg_notify(22.39d(20) epoch 194472) v5 -1000> 2014-08-13 17:22:51.963128 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 10 ==== pg_info(1 pgs e194472:24.2ec(2)) v4 ==== 866+0+0 (1455582846 0 0) 0x57d1340 con 0x486b9c0 -999> 2014-08-13 17:22:51.963147 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:50.854865, event: header_read, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -998> 2014-08-13 17:22:51.963157 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:50.854866, event: throttled, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -997> 2014-08-13 17:22:51.963166 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:50.854950, event: all_read, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -996> 2014-08-13 17:22:51.963191 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:51.963145, event: dispatched, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -995> 2014-08-13 17:22:51.963216 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:51.963216, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -994> 2014-08-13 17:22:51.963229 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:51.963229, event: started, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -993> 2014-08-13 17:22:51.963254 7f74d8735700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:22:51.963254, event: done, request: pg_info(1 pgs e194472:24.2ec(2)) v4 -992> 2014-08-13 17:22:51.963274 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 11 ==== pg_notify(22.48e(20) epoch 194472) v5 ==== 1834+0+0 (4178568222 0 0) 0x49e2f40 con 0x486b700 -991> 2014-08-13 17:22:51.963293 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:50.880875, event: header_read, request: pg_notify(22.48e(20) epoch 194472) v5 -990> 2014-08-13 17:22:51.963303 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:50.880876, event: throttled, request: pg_notify(22.48e(20) epoch 194472) v5 -989> 2014-08-13 17:22:51.963318 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:50.880973, event: all_read, request: pg_notify(22.48e(20) epoch 194472) v5 -988> 2014-08-13 17:22:51.963327 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:51.963291, event: dispatched, request: pg_notify(22.48e(20) epoch 194472) v5 -987> 2014-08-13 17:22:51.963337 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:51.963336, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194472) v5 -986> 2014-08-13 17:22:51.963348 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:51.963348, event: started, request: pg_notify(22.48e(20) epoch 194472) v5 -985> 2014-08-13 17:22:51.963371 7f74d8735700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:22:51.963371, event: done, request: pg_notify(22.48e(20) epoch 194472) v5 -984> 2014-08-13 17:22:51.963400 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.379 192.168.1.108:6830/4861 8 ==== pg_notify(24.1bb(26) epoch 194472) v5 ==== 2214+0+0 (1542312338 0 0) 0x57d78c0 con 0x486b440 -983> 2014-08-13 17:22:51.963420 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:50.961981, event: header_read, request: pg_notify(24.1bb(26) epoch 194472) v5 -982> 2014-08-13 17:22:51.963430 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:50.961982, event: throttled, request: pg_notify(24.1bb(26) epoch 194472) v5 -981> 2014-08-13 17:22:51.963439 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:50.962093, event: all_read, request: pg_notify(24.1bb(26) epoch 194472) v5 -980> 2014-08-13 17:22:51.963448 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:51.963417, event: dispatched, request: pg_notify(24.1bb(26) epoch 194472) v5 -979> 2014-08-13 17:22:51.963457 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:51.963457, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194472) v5 -978> 2014-08-13 17:22:51.963468 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:51.963468, event: started, request: pg_notify(24.1bb(26) epoch 194472) v5 -977> 2014-08-13 17:22:51.963491 7f74d8735700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:22:51.963491, event: done, request: pg_notify(24.1bb(26) epoch 194472) v5 -976> 2014-08-13 17:22:51.963517 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 8 ==== pg_info(1 pgs e194472:24.e2(2)) v4 ==== 866+0+0 (3140649956 0 0) 0x57d2a00 con 0x486b180 -975> 2014-08-13 17:22:51.963536 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:50.852069, event: header_read, request: pg_info(1 pgs e194472:24.e2(2)) v4 -974> 2014-08-13 17:22:51.963546 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:50.852070, event: throttled, request: pg_info(1 pgs e194472:24.e2(2)) v4 -973> 2014-08-13 17:22:51.963556 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:50.852139, event: all_read, request: pg_info(1 pgs e194472:24.e2(2)) v4 -972> 2014-08-13 17:22:51.963564 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:51.963534, event: dispatched, request: pg_info(1 pgs e194472:24.e2(2)) v4 -971> 2014-08-13 17:22:51.963574 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:51.963574, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.e2(2)) v4 -970> 2014-08-13 17:22:51.963585 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:51.963585, event: started, request: pg_info(1 pgs e194472:24.e2(2)) v4 -969> 2014-08-13 17:22:51.963608 7f74d8735700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:22:51.963608, event: done, request: pg_info(1 pgs e194472:24.e2(2)) v4 -968> 2014-08-13 17:22:51.963627 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 17 ==== pg_notify(24.3d(24) epoch 194472) v5 ==== 2096+0+0 (3094728919 0 0) 0x53bbb80 con 0x486a3c0 -967> 2014-08-13 17:22:51.963645 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:50.896031, event: header_read, request: pg_notify(24.3d(24) epoch 194472) v5 -966> 2014-08-13 17:22:51.963656 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:50.896033, event: throttled, request: pg_notify(24.3d(24) epoch 194472) v5 -965> 2014-08-13 17:22:51.963665 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:50.896141, event: all_read, request: pg_notify(24.3d(24) epoch 194472) v5 -964> 2014-08-13 17:22:51.963674 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:51.963643, event: dispatched, request: pg_notify(24.3d(24) epoch 194472) v5 -963> 2014-08-13 17:22:51.963683 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:51.963683, event: waiting_for_osdmap, request: pg_notify(24.3d(24) epoch 194472) v5 -962> 2014-08-13 17:22:51.963694 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:51.963694, event: started, request: pg_notify(24.3d(24) epoch 194472) v5 -961> 2014-08-13 17:22:51.964371 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.362 192.168.1.108:6842/8170 -- pg_log(24.3d epoch 194472 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194472) v4 -- ?+0 0x5b19200 -960> 2014-08-13 17:22:51.964776 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=194471 pi=193635-194470/25 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating -959> 2014-08-13 17:22:51.968638 7f74d8735700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:22:51.968637, event: done, request: pg_notify(24.3d(24) epoch 194472) v5 -958> 2014-08-13 17:22:51.968681 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.296 192.168.1.109:6806/31286 5 ==== pg_notify(24.181(24) epoch 194472) v5 ==== 2092+0+0 (2569495000 0 0) 0x4ac2a00 con 0x486fe80 -957> 2014-08-13 17:22:51.968705 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.098763, event: header_read, request: pg_notify(24.181(24) epoch 194472) v5 -956> 2014-08-13 17:22:51.968717 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.098764, event: throttled, request: pg_notify(24.181(24) epoch 194472) v5 -955> 2014-08-13 17:22:51.968727 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.098894, event: all_read, request: pg_notify(24.181(24) epoch 194472) v5 -954> 2014-08-13 17:22:51.968741 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.968702, event: dispatched, request: pg_notify(24.181(24) epoch 194472) v5 -953> 2014-08-13 17:22:51.968752 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.968751, event: waiting_for_osdmap, request: pg_notify(24.181(24) epoch 194472) v5 -952> 2014-08-13 17:22:51.968761 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.968761, event: started, request: pg_notify(24.181(24) epoch 194472) v5 -951> 2014-08-13 17:22:51.968791 7f74d8735700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:22:51.968791, event: done, request: pg_notify(24.181(24) epoch 194472) v5 -950> 2014-08-13 17:22:51.968813 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 10 ==== pg_notify(22.707(24) epoch 194472) v5 ==== 2066+0+0 (1235915174 0 0) 0x53b96c0 con 0x48d9ce0 -949> 2014-08-13 17:22:51.968832 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:50.855721, event: header_read, request: pg_notify(22.707(24) epoch 194472) v5 -948> 2014-08-13 17:22:51.968839 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:50.855722, event: throttled, request: pg_notify(22.707(24) epoch 194472) v5 -947> 2014-08-13 17:22:51.968845 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:50.855826, event: all_read, request: pg_notify(22.707(24) epoch 194472) v5 -946> 2014-08-13 17:22:51.968850 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:51.968830, event: dispatched, request: pg_notify(22.707(24) epoch 194472) v5 -945> 2014-08-13 17:22:51.968856 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:51.968856, event: waiting_for_osdmap, request: pg_notify(22.707(24) epoch 194472) v5 -944> 2014-08-13 17:22:51.968863 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:51.968863, event: started, request: pg_notify(22.707(24) epoch 194472) v5 -943> 2014-08-13 17:22:51.968883 7f74d8735700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:22:51.968882, event: done, request: pg_notify(22.707(24) epoch 194472) v5 -942> 2014-08-13 17:22:51.968905 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 12 ==== pg_info(1 pgs e194472:24.3f7(2)) v4 ==== 866+0+0 (389951884 0 0) 0x57d55c0 con 0x486e040 -941> 2014-08-13 17:22:51.968925 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:50.853121, event: header_read, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -940> 2014-08-13 17:22:51.968932 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:50.853122, event: throttled, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -939> 2014-08-13 17:22:51.968941 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:50.853211, event: all_read, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -938> 2014-08-13 17:22:51.968949 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:51.968923, event: dispatched, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -937> 2014-08-13 17:22:51.968958 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:51.968958, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -936> 2014-08-13 17:22:51.968967 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:51.968966, event: started, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -935> 2014-08-13 17:22:51.968987 7f74d8735700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:22:51.968987, event: done, request: pg_info(1 pgs e194472:24.3f7(2)) v4 -934> 2014-08-13 17:22:51.969007 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 11 ==== pg_notify(22.362(22) epoch 194472) v5 ==== 1948+0+0 (2765639279 0 0) 0x466b800 con 0x48db9c0 -933> 2014-08-13 17:22:51.969026 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.005845, event: header_read, request: pg_notify(22.362(22) epoch 194472) v5 -932> 2014-08-13 17:22:51.969000 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 2.094679 5 0.000248 -931> 2014-08-13 17:22:51.969037 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.005845, event: throttled, request: pg_notify(22.362(22) epoch 194472) v5 -930> 2014-08-13 17:22:51.969043 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.005921, event: all_read, request: pg_notify(22.362(22) epoch 194472) v5 -929> 2014-08-13 17:22:51.969051 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.969024, event: dispatched, request: pg_notify(22.362(22) epoch 194472) v5 -928> 2014-08-13 17:22:51.969060 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.969060, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194472) v5 -927> 2014-08-13 17:22:51.969048 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 2.136365 0 0.000000 -926> 2014-08-13 17:22:51.969073 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.969073, event: started, request: pg_notify(22.362(22) epoch 194472) v5 -925> 2014-08-13 17:22:51.969097 7f74d8735700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:22:51.969097, event: done, request: pg_notify(22.362(22) epoch 194472) v5 -924> 2014-08-13 17:22:51.969089 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194449 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -923> 2014-08-13 17:22:51.969120 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 8 ==== pg_notify(24.1bb(2) epoch 194472) v5 ==== 870+0+0 (1857392403 0 0) 0x4ac3f00 con 0x48dcd00 -922> 2014-08-13 17:22:51.969116 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- osd_map(194472..194472 src has 185139..194472) v3 -- ?+0 0x4e31d40 con 0x47a4200 -921> 2014-08-13 17:22:51.969139 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:50.892326, event: header_read, request: pg_notify(24.1bb(2) epoch 194472) v5 -920> 2014-08-13 17:22:51.969146 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:50.892327, event: throttled, request: pg_notify(24.1bb(2) epoch 194472) v5 -919> 2014-08-13 17:22:51.969154 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:50.892402, event: all_read, request: pg_notify(24.1bb(2) epoch 194472) v5 -918> 2014-08-13 17:22:51.969155 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(27) epoch 194472) v5 -- ?+0 0x57d2680 con 0x47a4200 -917> 2014-08-13 17:22:51.969163 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:51.969137, event: dispatched, request: pg_notify(24.1bb(2) epoch 194472) v5 -916> 2014-08-13 17:22:51.969186 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:51.969186, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194472) v5 -915> 2014-08-13 17:22:51.969199 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:51.969199, event: started, request: pg_notify(24.1bb(2) epoch 194472) v5 -914> 2014-08-13 17:22:51.969223 7f74d8735700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:22:51.969223, event: done, request: pg_notify(24.1bb(2) epoch 194472) v5 -913> 2014-08-13 17:22:51.969243 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 12 ==== pg_notify(22.3d9(9) epoch 194472) v5 ==== 1229+0+0 (1546709789 0 0) 0x5c43480 con 0x48dfbc0 -912> 2014-08-13 17:22:51.969262 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:50.869452, event: header_read, request: pg_notify(22.3d9(9) epoch 194472) v5 -911> 2014-08-13 17:22:51.969273 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:50.869454, event: throttled, request: pg_notify(22.3d9(9) epoch 194472) v5 -910> 2014-08-13 17:22:51.969282 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:50.869527, event: all_read, request: pg_notify(22.3d9(9) epoch 194472) v5 -909> 2014-08-13 17:22:51.969291 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:51.969260, event: dispatched, request: pg_notify(22.3d9(9) epoch 194472) v5 -908> 2014-08-13 17:22:51.969300 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:51.969300, event: waiting_for_osdmap, request: pg_notify(22.3d9(9) epoch 194472) v5 -907> 2014-08-13 17:22:51.969311 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:51.969311, event: started, request: pg_notify(22.3d9(9) epoch 194472) v5 -906> 2014-08-13 17:22:51.969335 7f74d8735700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:22:51.969335, event: done, request: pg_notify(22.3d9(9) epoch 194472) v5 -905> 2014-08-13 17:22:51.969322 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_info(1 pgs e194472:24.3d(25)) v4 -- ?+0 0x57d5b00 con 0x4758840 -904> 2014-08-13 17:22:51.969365 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.0 192.168.1.115:6810/59615 7 ==== pg_notify(22.775(22) epoch 194472) v5 ==== 1948+0+0 (3225416514 0 0) 0x5de9340 con 0x4900420 -903> 2014-08-13 17:22:51.969370 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.109:6802/30768 -- pg_info(1 pgs e194472:24.3d(25)) v4 -- ?+0 0x57d2d80 con 0x475ee00 -902> 2014-08-13 17:22:51.969388 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:50.956346, event: header_read, request: pg_notify(22.775(22) epoch 194472) v5 -901> 2014-08-13 17:22:51.969400 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:50.956347, event: throttled, request: pg_notify(22.775(22) epoch 194472) v5 -900> 2014-08-13 17:22:51.969410 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:50.956414, event: all_read, request: pg_notify(22.775(22) epoch 194472) v5 -899> 2014-08-13 17:22:51.969419 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:51.969386, event: dispatched, request: pg_notify(22.775(22) epoch 194472) v5 -898> 2014-08-13 17:22:51.969429 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:51.969429, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194472) v5 -897> 2014-08-13 17:22:51.969440 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:51.969440, event: started, request: pg_notify(22.775(22) epoch 194472) v5 -896> 2014-08-13 17:22:51.969463 7f74d8735700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:22:51.969463, event: done, request: pg_notify(22.775(22) epoch 194472) v5 -895> 2014-08-13 17:22:51.969505 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.396 192.168.1.105:6814/41806 11 ==== pg_notify(22.3d9(38) epoch 194472) v5 ==== 2772+0+0 (2397380690 0 0) 0x5de9dc0 con 0x4900580 -894> 2014-08-13 17:22:51.969534 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:50.967482, event: header_read, request: pg_notify(22.3d9(38) epoch 194472) v5 -893> 2014-08-13 17:22:51.969545 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:50.967483, event: throttled, request: pg_notify(22.3d9(38) epoch 194472) v5 -892> 2014-08-13 17:22:51.969554 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:50.967565, event: all_read, request: pg_notify(22.3d9(38) epoch 194472) v5 -891> 2014-08-13 17:22:51.969563 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:51.969522, event: dispatched, request: pg_notify(22.3d9(38) epoch 194472) v5 -890> 2014-08-13 17:22:51.969573 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:51.969573, event: waiting_for_osdmap, request: pg_notify(22.3d9(38) epoch 194472) v5 -889> 2014-08-13 17:22:51.969584 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:51.969584, event: started, request: pg_notify(22.3d9(38) epoch 194472) v5 -888> 2014-08-13 17:22:51.969607 7f74d8735700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:22:51.969607, event: done, request: pg_notify(22.3d9(38) epoch 194472) v5 -887> 2014-08-13 17:22:51.969647 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.339 192.168.1.110:6818/45268 9 ==== pg_notify(22.3af(22) epoch 194472) v5 ==== 1948+0+0 (2113412707 0 0) 0x53bf8c0 con 0x4900f20 -886> 2014-08-13 17:22:51.969667 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:50.868062, event: header_read, request: pg_notify(22.3af(22) epoch 194472) v5 -885> 2014-08-13 17:22:51.969678 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:50.868063, event: throttled, request: pg_notify(22.3af(22) epoch 194472) v5 -884> 2014-08-13 17:22:51.969687 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:50.868182, event: all_read, request: pg_notify(22.3af(22) epoch 194472) v5 -883> 2014-08-13 17:22:51.969696 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:51.969665, event: dispatched, request: pg_notify(22.3af(22) epoch 194472) v5 -882> 2014-08-13 17:22:51.969705 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:51.969705, event: waiting_for_osdmap, request: pg_notify(22.3af(22) epoch 194472) v5 -881> 2014-08-13 17:22:51.969716 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:51.969716, event: started, request: pg_notify(22.3af(22) epoch 194472) v5 -880> 2014-08-13 17:22:51.969746 7f74d8735700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:22:51.969746, event: done, request: pg_notify(22.3af(22) epoch 194472) v5 -879> 2014-08-13 17:22:51.969792 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 13 ==== pg_notify(22.113(1) epoch 194472) v5 ==== 777+0+0 (264734496 0 0) 0x53bc980 con 0x4901600 -878> 2014-08-13 17:22:51.969813 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:50.892185, event: header_read, request: pg_notify(22.113(1) epoch 194472) v5 -877> 2014-08-13 17:22:51.969823 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:50.892187, event: throttled, request: pg_notify(22.113(1) epoch 194472) v5 -876> 2014-08-13 17:22:51.969833 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:50.892256, event: all_read, request: pg_notify(22.113(1) epoch 194472) v5 -875> 2014-08-13 17:22:51.969842 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:51.969810, event: dispatched, request: pg_notify(22.113(1) epoch 194472) v5 -874> 2014-08-13 17:22:51.969851 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:51.969851, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194472) v5 -873> 2014-08-13 17:22:51.969862 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:51.969862, event: started, request: pg_notify(22.113(1) epoch 194472) v5 -872> 2014-08-13 17:22:51.969937 7f74d8735700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:22:51.969937, event: done, request: pg_notify(22.113(1) epoch 194472) v5 -871> 2014-08-13 17:22:51.969982 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 10 ==== pg_notify(22.22c(20) epoch 194472) v5 ==== 1834+0+0 (381677189 0 0) 0x5c45b00 con 0x4902680 -870> 2014-08-13 17:22:51.970004 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:50.892713, event: header_read, request: pg_notify(22.22c(20) epoch 194472) v5 -869> 2014-08-13 17:22:51.970014 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:50.892715, event: throttled, request: pg_notify(22.22c(20) epoch 194472) v5 -868> 2014-08-13 17:22:51.970023 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:50.892820, event: all_read, request: pg_notify(22.22c(20) epoch 194472) v5 -867> 2014-08-13 17:22:51.970032 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:51.970001, event: dispatched, request: pg_notify(22.22c(20) epoch 194472) v5 -866> 2014-08-13 17:22:51.970042 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:51.970042, event: waiting_for_osdmap, request: pg_notify(22.22c(20) epoch 194472) v5 -865> 2014-08-13 17:22:51.970053 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:51.970053, event: started, request: pg_notify(22.22c(20) epoch 194472) v5 -864> 2014-08-13 17:22:51.970081 7f74d8735700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:22:51.970080, event: done, request: pg_notify(22.22c(20) epoch 194472) v5 -863> 2014-08-13 17:22:51.970104 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 8 ==== pg_notify(24.30b(29) epoch 194472) v5 ==== 2391+0+0 (346520738 0 0) 0x53bb480 con 0x49023c0 -862> 2014-08-13 17:22:51.970123 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:50.962052, event: header_read, request: pg_notify(24.30b(29) epoch 194472) v5 -861> 2014-08-13 17:22:51.970133 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:50.962054, event: throttled, request: pg_notify(24.30b(29) epoch 194472) v5 -860> 2014-08-13 17:22:51.970142 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:50.962136, event: all_read, request: pg_notify(24.30b(29) epoch 194472) v5 -859> 2014-08-13 17:22:51.970151 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:51.970121, event: dispatched, request: pg_notify(24.30b(29) epoch 194472) v5 -858> 2014-08-13 17:22:51.970161 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:51.970160, event: waiting_for_osdmap, request: pg_notify(24.30b(29) epoch 194472) v5 -857> 2014-08-13 17:22:51.970184 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:51.970184, event: started, request: pg_notify(24.30b(29) epoch 194472) v5 -856> 2014-08-13 17:22:51.970213 7f74d8735700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:22:51.970212, event: done, request: pg_notify(24.30b(29) epoch 194472) v5 -855> 2014-08-13 17:22:51.970328 7f74d192a700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 -854> 2014-08-13 17:22:51.970233 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.314 192.168.1.101:6802/22303 7 ==== pg_notify(22.619(22) epoch 194472) v5 ==== 1948+0+0 (3729207921 0 0) 0x5c41340 con 0x4902100 -853> 2014-08-13 17:22:51.970403 7f74d6932700 1 -- 192.168.100.115:6806/30165 <== osd.17 192.168.100.103:0/37658 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.970145) v2 ==== 47+0+0 (854796810 0 0) 0x5ded940 con 0x58c3f40 -852> 2014-08-13 17:22:51.970224 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 2.098212 4 0.000253 -851> 2014-08-13 17:22:51.970452 7f74d6932700 1 -- 192.168.100.115:6806/30165 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194472 stamp 2014-08-13 17:22:51.970145) v2 -- ?+0 0x49e16c0 con 0x58c3f40 -850> 2014-08-13 17:22:51.970466 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 2.116875 0 0.000000 -849> 2014-08-13 17:22:51.970418 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:50.860083, event: header_read, request: pg_notify(22.619(22) epoch 194472) v5 -848> 2014-08-13 17:22:51.970505 7f74d5530700 1 -- 192.168.1.115:6817/30165 <== osd.17 192.168.100.103:0/37658 1 ==== osd_ping(ping e194472 stamp 2014-08-13 17:22:51.970145) v2 ==== 47+0+0 (854796810 0 0) 0x5ded5c0 con 0x58a77a0 -847> 2014-08-13 17:22:51.970566 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194454 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active -846> 2014-08-13 17:22:51.970580 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:50.860084, event: throttled, request: pg_notify(22.619(22) epoch 194472) v5 -845> 2014-08-13 17:22:51.970590 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:50.860189, event: all_read, request: pg_notify(22.619(22) epoch 194472) v5 -844> 2014-08-13 17:22:51.970596 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:51.970410, event: dispatched, request: pg_notify(22.619(22) epoch 194472) v5 -843> 2014-08-13 17:22:51.970588 7f74d5530700 1 -- 192.168.1.115:6817/30165 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194472 stamp 2014-08-13 17:22:51.970145) v2 -- ?+0 0x49e1dc0 con 0x58a77a0 -842> 2014-08-13 17:22:51.970604 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:51.970603, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194472) v5 -841> 2014-08-13 17:22:51.970614 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:51.970614, event: started, request: pg_notify(22.619(22) epoch 194472) v5 -840> 2014-08-13 17:22:51.970690 7f74d8735700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:22:51.970690, event: done, request: pg_notify(22.619(22) epoch 194472) v5 -839> 2014-08-13 17:22:51.970691 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.367 192.168.1.114:6840/55513 -- pg_log(24.9b epoch 194472 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194472) v4 -- ?+0 0x5c75a00 -838> 2014-08-13 17:22:51.970739 7f74d192a700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 -837> 2014-08-13 17:22:51.970738 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.293 192.168.1.109:6816/33855 7 ==== pg_notify(22.3d9(42) epoch 194472) v5 ==== 2984+0+0 (3693796140 0 0) 0x5c40e00 con 0x4903700 -836> 2014-08-13 17:22:51.970761 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:50.856082, event: header_read, request: pg_notify(22.3d9(42) epoch 194472) v5 -835> 2014-08-13 17:22:51.970768 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:50.856083, event: throttled, request: pg_notify(22.3d9(42) epoch 194472) v5 -834> 2014-08-13 17:22:51.970773 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:50.856221, event: all_read, request: pg_notify(22.3d9(42) epoch 194472) v5 -833> 2014-08-13 17:22:51.970779 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:51.970758, event: dispatched, request: pg_notify(22.3d9(42) epoch 194472) v5 -832> 2014-08-13 17:22:51.970785 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:51.970785, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194472) v5 -831> 2014-08-13 17:22:51.970793 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:51.970792, event: started, request: pg_notify(22.3d9(42) epoch 194472) v5 -830> 2014-08-13 17:22:51.970818 7f74d8735700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:22:51.970818, event: done, request: pg_notify(22.3d9(42) epoch 194472) v5 -829> 2014-08-13 17:22:51.970842 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.48 192.168.1.112:6838/8300 7 ==== pg_notify(22.619(22) epoch 194472) v5 ==== 1948+0+0 (3729207921 0 0) 0x5c471c0 con 0x4904780 -828> 2014-08-13 17:22:51.970862 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:50.906660, event: header_read, request: pg_notify(22.619(22) epoch 194472) v5 -827> 2014-08-13 17:22:51.970869 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:50.906661, event: throttled, request: pg_notify(22.619(22) epoch 194472) v5 -826> 2014-08-13 17:22:51.970874 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:50.906768, event: all_read, request: pg_notify(22.619(22) epoch 194472) v5 -825> 2014-08-13 17:22:51.970860 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating -824> 2014-08-13 17:22:51.970880 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:51.970860, event: dispatched, request: pg_notify(22.619(22) epoch 194472) v5 -823> 2014-08-13 17:22:51.970908 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:51.970908, event: waiting_for_osdmap, request: pg_notify(22.619(22) epoch 194472) v5 -822> 2014-08-13 17:22:51.970918 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:51.970918, event: started, request: pg_notify(22.619(22) epoch 194472) v5 -821> 2014-08-13 17:22:51.970937 7f74d8735700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:22:51.970937, event: done, request: pg_notify(22.619(22) epoch 194472) v5 -820> 2014-08-13 17:22:51.970970 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.302 192.168.1.113:6810/15731 7 ==== pg_notify(22.22c(1) epoch 194472) v5 ==== 777+0+0 (1173838968 0 0) 0x5c43f00 con 0x49044c0 -819> 2014-08-13 17:22:51.970991 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:50.875697, event: header_read, request: pg_notify(22.22c(1) epoch 194472) v5 -818> 2014-08-13 17:22:51.970998 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:50.875698, event: throttled, request: pg_notify(22.22c(1) epoch 194472) v5 -817> 2014-08-13 17:22:51.970987 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.378 192.168.1.102:6840/19195 -- pg_log(24.9b epoch 194472 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194472) v4 -- ?+0 0x5c75400 -816> 2014-08-13 17:22:51.971004 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:50.875766, event: all_read, request: pg_notify(22.22c(1) epoch 194472) v5 -815> 2014-08-13 17:22:51.971009 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:51.970989, event: dispatched, request: pg_notify(22.22c(1) epoch 194472) v5 -814> 2014-08-13 17:22:51.971015 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:51.971015, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194472) v5 -813> 2014-08-13 17:22:51.971022 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:51.971022, event: started, request: pg_notify(22.22c(1) epoch 194472) v5 -812> 2014-08-13 17:22:51.971040 7f74d8735700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:22:51.971040, event: done, request: pg_notify(22.22c(1) epoch 194472) v5 -811> 2014-08-13 17:22:51.971060 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.63 192.168.1.111:6830/12680 7 ==== pg_notify(22.775(22) epoch 194472) v5 ==== 1948+0+0 (3225416514 0 0) 0x5c439c0 con 0x49070c0 -810> 2014-08-13 17:22:51.971079 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:50.870749, event: header_read, request: pg_notify(22.775(22) epoch 194472) v5 -809> 2014-08-13 17:22:51.971086 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:50.870751, event: throttled, request: pg_notify(22.775(22) epoch 194472) v5 -808> 2014-08-13 17:22:51.971094 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:50.870865, event: all_read, request: pg_notify(22.775(22) epoch 194472) v5 -807> 2014-08-13 17:22:51.971103 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:51.971077, event: dispatched, request: pg_notify(22.775(22) epoch 194472) v5 -806> 2014-08-13 17:22:51.971113 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:51.971112, event: waiting_for_osdmap, request: pg_notify(22.775(22) epoch 194472) v5 -805> 2014-08-13 17:22:51.971128 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:51.971128, event: started, request: pg_notify(22.775(22) epoch 194472) v5 -804> 2014-08-13 17:22:51.971152 7f74d8735700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:22:51.971151, event: done, request: pg_notify(22.775(22) epoch 194472) v5 -803> 2014-08-13 17:22:51.971193 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.11 192.168.1.103:6813/37929 7 ==== pg_notify(22.1f1(1) epoch 194472) v5 ==== 777+0+0 (2591727504 0 0) 0x5c44d00 con 0x4906e00 -802> 2014-08-13 17:22:51.971216 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:50.878029, event: header_read, request: pg_notify(22.1f1(1) epoch 194472) v5 -801> 2014-08-13 17:22:51.971223 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:50.878029, event: throttled, request: pg_notify(22.1f1(1) epoch 194472) v5 -800> 2014-08-13 17:22:51.971235 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:50.878097, event: all_read, request: pg_notify(22.1f1(1) epoch 194472) v5 -799> 2014-08-13 17:22:51.971244 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:51.971213, event: dispatched, request: pg_notify(22.1f1(1) epoch 194472) v5 -798> 2014-08-13 17:22:51.971254 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:51.971254, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194472) v5 -797> 2014-08-13 17:22:51.971262 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:51.971262, event: started, request: pg_notify(22.1f1(1) epoch 194472) v5 -796> 2014-08-13 17:22:51.971281 7f74d8735700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:22:51.971281, event: done, request: pg_notify(22.1f1(1) epoch 194472) v5 -795> 2014-08-13 17:22:51.971300 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 12 ==== pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 ==== 3656+0+0 (3418555898 0 0) 0x53bdcc0 con 0x4d58840 -794> 2014-08-13 17:22:51.971322 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:50.889110, event: header_read, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -793> 2014-08-13 17:22:51.971333 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:50.889111, event: throttled, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -792> 2014-08-13 17:22:51.971343 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:50.889278, event: all_read, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -791> 2014-08-13 17:22:51.971352 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:51.971320, event: dispatched, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -790> 2014-08-13 17:22:51.971362 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:51.971362, event: waiting_for_osdmap, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -789> 2014-08-13 17:22:51.971374 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:51.971373, event: started, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -788> 2014-08-13 17:22:51.971408 7f74d8735700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:22:51.971408, event: done, request: pg_notify(22.3af(22),22.f3(18) epoch 194472) v5 -787> 2014-08-13 17:22:51.971445 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 10 ==== pg_notify(22.99(1) epoch 194472) v5 ==== 777+0+0 (3166036464 0 0) 0x5deaa00 con 0x4d582c0 -786> 2014-08-13 17:22:51.971466 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.026189, event: header_read, request: pg_notify(22.99(1) epoch 194472) v5 -785> 2014-08-13 17:22:51.971473 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.026190, event: throttled, request: pg_notify(22.99(1) epoch 194472) v5 -784> 2014-08-13 17:22:51.971485 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.026226, event: all_read, request: pg_notify(22.99(1) epoch 194472) v5 -783> 2014-08-13 17:22:51.971494 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.971463, event: dispatched, request: pg_notify(22.99(1) epoch 194472) v5 -782> 2014-08-13 17:22:51.971504 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.971504, event: waiting_for_osdmap, request: pg_notify(22.99(1) epoch 194472) v5 -781> 2014-08-13 17:22:51.971515 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.971515, event: started, request: pg_notify(22.99(1) epoch 194472) v5 -780> 2014-08-13 17:22:51.971682 7f74d8735700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:22:51.971682, event: done, request: pg_notify(22.99(1) epoch 194472) v5 -779> 2014-08-13 17:22:51.971390 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=194471 pi=193637-194470/24 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -778> 2014-08-13 17:22:51.971730 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.381 192.168.1.102:6822/15390 7 ==== pg_notify(22.481(39) epoch 194472) v5 ==== 2903+0+0 (3684298897 0 0) 0x5dec7c0 con 0x4d58000 -777> 2014-08-13 17:22:51.971758 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.267032, event: header_read, request: pg_notify(22.481(39) epoch 194472) v5 -776> 2014-08-13 17:22:51.971770 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.267035, event: throttled, request: pg_notify(22.481(39) epoch 194472) v5 -775> 2014-08-13 17:22:51.971780 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.267190, event: all_read, request: pg_notify(22.481(39) epoch 194472) v5 -774> 2014-08-13 17:22:51.971789 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.971755, event: dispatched, request: pg_notify(22.481(39) epoch 194472) v5 -773> 2014-08-13 17:22:51.971799 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.971799, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194472) v5 -772> 2014-08-13 17:22:51.971812 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.971812, event: started, request: pg_notify(22.481(39) epoch 194472) v5 -771> 2014-08-13 17:22:51.971841 7f74d8735700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:22:51.971841, event: done, request: pg_notify(22.481(39) epoch 194472) v5 -770> 2014-08-13 17:22:51.971883 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 10 ==== pg_notify(24.9b(2) epoch 194472) v5 ==== 870+0+0 (1646131641 0 0) 0x5c45400 con 0x4d51ce0 -769> 2014-08-13 17:22:51.971904 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:50.891982, event: header_read, request: pg_notify(24.9b(2) epoch 194472) v5 -768> 2014-08-13 17:22:51.971915 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:50.891984, event: throttled, request: pg_notify(24.9b(2) epoch 194472) v5 -767> 2014-08-13 17:22:51.971924 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:50.892068, event: all_read, request: pg_notify(24.9b(2) epoch 194472) v5 -766> 2014-08-13 17:22:51.971933 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:51.971902, event: dispatched, request: pg_notify(24.9b(2) epoch 194472) v5 -765> 2014-08-13 17:22:51.971942 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:51.971942, event: waiting_for_osdmap, request: pg_notify(24.9b(2) epoch 194472) v5 -764> 2014-08-13 17:22:51.971954 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:51.971954, event: started, request: pg_notify(24.9b(2) epoch 194472) v5 -763> 2014-08-13 17:22:51.973532 7f74d8735700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:22:51.973531, event: done, request: pg_notify(24.9b(2) epoch 194472) v5 -762> 2014-08-13 17:22:51.973532 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6815/60573 -- pg_info(1 pgs e194472:24.30b(26)) v4 -- ?+0 0x49e6e40 con 0x45f1760 -761> 2014-08-13 17:22:51.973590 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6820/25633 -- pg_info(1 pgs e194472:24.9b(24)) v4 -- ?+0 0x49e0540 con 0x47591e0 -760> 2014-08-13 17:22:51.973584 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.65 192.168.1.111:6842/17345 10 ==== pg_notify(24.136(22) epoch 194472) v5 ==== 1978+0+0 (2332821113 0 0) 0x4ac4b40 con 0x4d51a20 -759> 2014-08-13 17:22:51.973608 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:50.874732, event: header_read, request: pg_notify(24.136(22) epoch 194472) v5 -758> 2014-08-13 17:22:51.973616 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:50.874733, event: throttled, request: pg_notify(24.136(22) epoch 194472) v5 -757> 2014-08-13 17:22:51.973622 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:50.874830, event: all_read, request: pg_notify(24.136(22) epoch 194472) v5 -756> 2014-08-13 17:22:51.973627 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:51.973605, event: dispatched, request: pg_notify(24.136(22) epoch 194472) v5 -755> 2014-08-13 17:22:51.973629 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6832/53185 -- pg_info(1 pgs e194472:24.9b(24)) v4 -- ?+0 0x49e01c0 con 0x4d55ac0 -754> 2014-08-13 17:22:51.973637 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:51.973637, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194472) v5 -753> 2014-08-13 17:22:51.973651 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:51.973651, event: started, request: pg_notify(24.136(22) epoch 194472) v5 -752> 2014-08-13 17:22:51.973661 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.114:6830/52879 -- pg_info(1 pgs e194472:24.30b(26)) v4 -- ?+0 0x4ac1180 con 0x49023c0 -751> 2014-08-13 17:22:51.973677 7f74d8735700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:22:51.973677, event: done, request: pg_notify(24.136(22) epoch 194472) v5 -750> 2014-08-13 17:22:51.973712 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.311 192.168.1.101:6816/24270 7 ==== pg_notify(22.f3(18) epoch 194472) v5 ==== 1720+0+0 (3782601051 0 0) 0x5dea300 con 0x4d54200 -749> 2014-08-13 17:22:51.973729 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.007979, event: header_read, request: pg_notify(22.f3(18) epoch 194472) v5 -748> 2014-08-13 17:22:51.973736 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.007980, event: throttled, request: pg_notify(22.f3(18) epoch 194472) v5 -747> 2014-08-13 17:22:51.973742 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.008050, event: all_read, request: pg_notify(22.f3(18) epoch 194472) v5 -746> 2014-08-13 17:22:51.973747 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.973727, event: dispatched, request: pg_notify(22.f3(18) epoch 194472) v5 -745> 2014-08-13 17:22:51.973753 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.973753, event: waiting_for_osdmap, request: pg_notify(22.f3(18) epoch 194472) v5 -744> 2014-08-13 17:22:51.973760 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.973760, event: started, request: pg_notify(22.f3(18) epoch 194472) v5 -743> 2014-08-13 17:22:51.973780 7f74d8735700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:22:51.973780, event: done, request: pg_notify(22.f3(18) epoch 194472) v5 -742> 2014-08-13 17:22:51.973799 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.366 192.168.1.108:6828/4534 7 ==== pg_notify(22.39d(1) epoch 194472) v5 ==== 777+0+0 (3093967758 0 0) 0x5de8e00 con 0x4d53c80 -741> 2014-08-13 17:22:51.973814 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:50.928443, event: header_read, request: pg_notify(22.39d(1) epoch 194472) v5 -740> 2014-08-13 17:22:51.973820 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:50.928445, event: throttled, request: pg_notify(22.39d(1) epoch 194472) v5 -739> 2014-08-13 17:22:51.973825 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:50.928509, event: all_read, request: pg_notify(22.39d(1) epoch 194472) v5 -738> 2014-08-13 17:22:51.973831 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:51.973812, event: dispatched, request: pg_notify(22.39d(1) epoch 194472) v5 -737> 2014-08-13 17:22:51.973836 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:51.973836, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194472) v5 -736> 2014-08-13 17:22:51.973844 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:51.973843, event: started, request: pg_notify(22.39d(1) epoch 194472) v5 -735> 2014-08-13 17:22:51.973980 7f74d8735700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:22:51.973979, event: done, request: pg_notify(22.39d(1) epoch 194472) v5 -734> 2014-08-13 17:22:51.974027 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.380 192.168.1.114:6822/51844 7 ==== pg_notify(22.4df(20) epoch 194472) v5 ==== 1834+0+0 (2899118731 0 0) 0x5debb80 con 0x4d56040 -733> 2014-08-13 17:22:51.974051 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.136493, event: header_read, request: pg_notify(22.4df(20) epoch 194472) v5 -732> 2014-08-13 17:22:51.974064 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.136493, event: throttled, request: pg_notify(22.4df(20) epoch 194472) v5 -731> 2014-08-13 17:22:51.974074 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.136555, event: all_read, request: pg_notify(22.4df(20) epoch 194472) v5 -730> 2014-08-13 17:22:51.974083 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.974048, event: dispatched, request: pg_notify(22.4df(20) epoch 194472) v5 -729> 2014-08-13 17:22:51.974093 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.974093, event: waiting_for_osdmap, request: pg_notify(22.4df(20) epoch 194472) v5 -728> 2014-08-13 17:22:51.974106 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.974106, event: started, request: pg_notify(22.4df(20) epoch 194472) v5 -727> 2014-08-13 17:22:51.974135 7f74d8735700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:22:51.974135, event: done, request: pg_notify(22.4df(20) epoch 194472) v5 -726> 2014-08-13 17:22:51.974161 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.349 192.168.1.104:6800/61726 9 ==== pg_notify(22.362(22) epoch 194472) v5 ==== 1948+0+0 (2765639279 0 0) 0x57d6c80 con 0x4d55d80 -725> 2014-08-13 17:22:51.974189 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:50.900581, event: header_read, request: pg_notify(22.362(22) epoch 194472) v5 -724> 2014-08-13 17:22:51.974196 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:50.900583, event: throttled, request: pg_notify(22.362(22) epoch 194472) v5 -723> 2014-08-13 17:22:51.974202 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:50.900697, event: all_read, request: pg_notify(22.362(22) epoch 194472) v5 -722> 2014-08-13 17:22:51.974207 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:51.974176, event: dispatched, request: pg_notify(22.362(22) epoch 194472) v5 -721> 2014-08-13 17:22:51.974214 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:51.974214, event: waiting_for_osdmap, request: pg_notify(22.362(22) epoch 194472) v5 -720> 2014-08-13 17:22:51.974226 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:51.974225, event: started, request: pg_notify(22.362(22) epoch 194472) v5 -719> 2014-08-13 17:22:51.974246 7f74d8735700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:22:51.974246, event: done, request: pg_notify(22.362(22) epoch 194472) v5 -718> 2014-08-13 17:22:51.974269 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 8 ==== pg_notify(24.9b(27) epoch 194472) v5 ==== 2273+0+0 (1901158279 0 0) 0x57d71c0 con 0x4d55ac0 -717> 2014-08-13 17:22:51.974288 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:50.971882, event: header_read, request: pg_notify(24.9b(27) epoch 194472) v5 -716> 2014-08-13 17:22:51.974295 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:50.971882, event: throttled, request: pg_notify(24.9b(27) epoch 194472) v5 -715> 2014-08-13 17:22:51.974300 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:50.971951, event: all_read, request: pg_notify(24.9b(27) epoch 194472) v5 -714> 2014-08-13 17:22:51.974309 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:51.974286, event: dispatched, request: pg_notify(24.9b(27) epoch 194472) v5 -713> 2014-08-13 17:22:51.974318 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:51.974318, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194472) v5 -712> 2014-08-13 17:22:51.974329 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:51.974329, event: started, request: pg_notify(24.9b(27) epoch 194472) v5 -711> 2014-08-13 17:22:51.974352 7f74d8735700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:22:51.974352, event: done, request: pg_notify(24.9b(27) epoch 194472) v5 -710> 2014-08-13 17:22:51.974383 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.306 192.168.1.101:6806/6812 7 ==== pg_notify(22.481(37) epoch 194472) v5 ==== 2801+0+0 (633429600 0 0) 0x5de88c0 con 0x4d55540 -709> 2014-08-13 17:22:51.974403 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:50.919201, event: header_read, request: pg_notify(22.481(37) epoch 194472) v5 -708> 2014-08-13 17:22:51.974413 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:50.919202, event: throttled, request: pg_notify(22.481(37) epoch 194472) v5 -707> 2014-08-13 17:22:51.974422 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:50.919327, event: all_read, request: pg_notify(22.481(37) epoch 194472) v5 -706> 2014-08-13 17:22:51.974431 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:51.974401, event: dispatched, request: pg_notify(22.481(37) epoch 194472) v5 -705> 2014-08-13 17:22:51.974441 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:51.974440, event: waiting_for_osdmap, request: pg_notify(22.481(37) epoch 194472) v5 -704> 2014-08-13 17:22:51.974452 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:51.974451, event: started, request: pg_notify(22.481(37) epoch 194472) v5 -703> 2014-08-13 17:22:51.974474 7f74d8735700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:22:51.974474, event: done, request: pg_notify(22.481(37) epoch 194472) v5 -702> 2014-08-13 17:22:51.974505 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.352 192.168.1.108:6826/4273 7 ==== pg_notify(22.4df(1) epoch 194472) v5 ==== 777+0+0 (1782585098 0 0) 0x5de8380 con 0x4d57d20 -701> 2014-08-13 17:22:51.974525 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:50.916530, event: header_read, request: pg_notify(22.4df(1) epoch 194472) v5 -700> 2014-08-13 17:22:51.974536 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:50.916531, event: throttled, request: pg_notify(22.4df(1) epoch 194472) v5 -699> 2014-08-13 17:22:51.974518 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 2.093147 4 0.000320 -698> 2014-08-13 17:22:51.974555 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:50.916601, event: all_read, request: pg_notify(22.4df(1) epoch 194472) v5 -697> 2014-08-13 17:22:51.974576 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:51.974523, event: dispatched, request: pg_notify(22.4df(1) epoch 194472) v5 -696> 2014-08-13 17:22:51.974570 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 2.130716 0 0.000000 -695> 2014-08-13 17:22:51.974587 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:51.974587, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194472) v5 -694> 2014-08-13 17:22:51.974606 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:51.974606, event: started, request: pg_notify(22.4df(1) epoch 194472) v5 -693> 2014-08-13 17:22:51.974605 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194449 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -692> 2014-08-13 17:22:51.974632 7f74d8735700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:22:51.974632, event: done, request: pg_notify(22.4df(1) epoch 194472) v5 -691> 2014-08-13 17:22:51.974651 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.382 192.168.1.108:6800/30527 6 ==== pg_notify(22.3d9(42) epoch 194472) v5 ==== 2984+0+0 (4256546748 0 0) 0x53beac0 con 0x4d57a60 -690> 2014-08-13 17:22:51.974670 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:50.865912, event: header_read, request: pg_notify(22.3d9(42) epoch 194472) v5 -689> 2014-08-13 17:22:51.974680 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:50.865914, event: throttled, request: pg_notify(22.3d9(42) epoch 194472) v5 -688> 2014-08-13 17:22:51.974689 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:50.866071, event: all_read, request: pg_notify(22.3d9(42) epoch 194472) v5 -687> 2014-08-13 17:22:51.974698 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:51.974668, event: dispatched, request: pg_notify(22.3d9(42) epoch 194472) v5 -686> 2014-08-13 17:22:51.974708 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:51.974708, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194472) v5 -685> 2014-08-13 17:22:51.974719 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:51.974718, event: started, request: pg_notify(22.3d9(42) epoch 194472) v5 -684> 2014-08-13 17:22:51.974742 7f74d8735700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:22:51.974741, event: done, request: pg_notify(22.3d9(42) epoch 194472) v5 -683> 2014-08-13 17:22:51.974766 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 5 ==== pg_notify(24.34(35) epoch 194472) v5 ==== 2513+0+0 (52585376 0 0) 0x5263100 con 0x4d2aaa0 -682> 2014-08-13 17:22:51.974785 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:50.886908, event: header_read, request: pg_notify(24.34(35) epoch 194472) v5 -681> 2014-08-13 17:22:51.974796 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:50.886910, event: throttled, request: pg_notify(24.34(35) epoch 194472) v5 -680> 2014-08-13 17:22:51.974805 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:50.887041, event: all_read, request: pg_notify(24.34(35) epoch 194472) v5 -679> 2014-08-13 17:22:51.974814 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:51.974783, event: dispatched, request: pg_notify(24.34(35) epoch 194472) v5 -678> 2014-08-13 17:22:51.974824 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:51.974824, event: waiting_for_osdmap, request: pg_notify(24.34(35) epoch 194472) v5 -677> 2014-08-13 17:22:51.974835 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:51.974835, event: started, request: pg_notify(24.34(35) epoch 194472) v5 -676> 2014-08-13 17:22:51.974861 7f74d8735700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:22:51.974860, event: done, request: pg_notify(24.34(35) epoch 194472) v5 -675> 2014-08-13 17:22:51.974894 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.307 192.168.1.109:6804/31016 5 ==== pg_notify(24.181(31) epoch 194472) v5 ==== 2493+0+0 (923173668 0 0) 0x49e7380 con 0x5419fa0 -674> 2014-08-13 17:22:51.974914 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:50.951913, event: header_read, request: pg_notify(24.181(31) epoch 194472) v5 -673> 2014-08-13 17:22:51.974924 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:50.951914, event: throttled, request: pg_notify(24.181(31) epoch 194472) v5 -672> 2014-08-13 17:22:51.974934 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:50.951988, event: all_read, request: pg_notify(24.181(31) epoch 194472) v5 -671> 2014-08-13 17:22:51.974943 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:51.974912, event: dispatched, request: pg_notify(24.181(31) epoch 194472) v5 -670> 2014-08-13 17:22:51.974952 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:51.974952, event: waiting_for_osdmap, request: pg_notify(24.181(31) epoch 194472) v5 -669> 2014-08-13 17:22:51.974963 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:51.974963, event: started, request: pg_notify(24.181(31) epoch 194472) v5 -668> 2014-08-13 17:22:51.974993 7f74d8735700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:22:51.974992, event: done, request: pg_notify(24.181(31) epoch 194472) v5 -667> 2014-08-13 17:22:51.975027 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 11 ==== pg_notify(24.30b(25) epoch 194472) v5 ==== 2155+0+0 (3889008340 0 0) 0x4ac0000 con 0x45f1760 -666> 2014-08-13 17:22:51.975048 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:50.958393, event: header_read, request: pg_notify(24.30b(25) epoch 194472) v5 -665> 2014-08-13 17:22:51.975058 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:50.958394, event: throttled, request: pg_notify(24.30b(25) epoch 194472) v5 -664> 2014-08-13 17:22:51.975067 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:50.958500, event: all_read, request: pg_notify(24.30b(25) epoch 194472) v5 -663> 2014-08-13 17:22:51.975076 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:51.975046, event: dispatched, request: pg_notify(24.30b(25) epoch 194472) v5 -662> 2014-08-13 17:22:51.975086 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:51.975086, event: waiting_for_osdmap, request: pg_notify(24.30b(25) epoch 194472) v5 -661> 2014-08-13 17:22:51.975097 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:51.975097, event: started, request: pg_notify(24.30b(25) epoch 194472) v5 -660> 2014-08-13 17:22:51.975120 7f74d8735700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:22:51.975120, event: done, request: pg_notify(24.30b(25) epoch 194472) v5 -659> 2014-08-13 17:22:51.975149 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.58 192.168.1.111:6834/14316 13 ==== pg_notify(22.1d2(22) epoch 194472) v5 ==== 1952+0+0 (2654433664 0 0) 0x5de9880 con 0x45f1e40 -658> 2014-08-13 17:22:51.975170 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:50.963521, event: header_read, request: pg_notify(22.1d2(22) epoch 194472) v5 -657> 2014-08-13 17:22:51.975185 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:50.963522, event: throttled, request: pg_notify(22.1d2(22) epoch 194472) v5 -656> 2014-08-13 17:22:51.975191 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:50.963590, event: all_read, request: pg_notify(22.1d2(22) epoch 194472) v5 -655> 2014-08-13 17:22:51.975196 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:51.975167, event: dispatched, request: pg_notify(22.1d2(22) epoch 194472) v5 -654> 2014-08-13 17:22:51.975202 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:51.975202, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194472) v5 -653> 2014-08-13 17:22:51.975209 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:51.975209, event: started, request: pg_notify(22.1d2(22) epoch 194472) v5 -652> 2014-08-13 17:22:51.975229 7f74d8735700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:22:51.975229, event: done, request: pg_notify(22.1d2(22) epoch 194472) v5 -651> 2014-08-13 17:22:51.975251 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.301 192.168.1.113:6814/16727 10 ==== pg_notify(24.30b(2) epoch 194472) v5 ==== 870+0+0 (2700828650 0 0) 0x57d7380 con 0x45f5540 -650> 2014-08-13 17:22:51.975270 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:50.863853, event: header_read, request: pg_notify(24.30b(2) epoch 194472) v5 -649> 2014-08-13 17:22:51.975277 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:50.863854, event: throttled, request: pg_notify(24.30b(2) epoch 194472) v5 -648> 2014-08-13 17:22:51.975282 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:50.863950, event: all_read, request: pg_notify(24.30b(2) epoch 194472) v5 -647> 2014-08-13 17:22:51.975288 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:51.975268, event: dispatched, request: pg_notify(24.30b(2) epoch 194472) v5 -646> 2014-08-13 17:22:51.975298 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:51.975297, event: waiting_for_osdmap, request: pg_notify(24.30b(2) epoch 194472) v5 -645> 2014-08-13 17:22:51.975308 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:51.975308, event: started, request: pg_notify(24.30b(2) epoch 194472) v5 -644> 2014-08-13 17:22:51.975331 7f74d8735700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:22:51.975330, event: done, request: pg_notify(24.30b(2) epoch 194472) v5 -643> 2014-08-13 17:22:51.975356 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 16 ==== pg_notify(24.136(22) epoch 194472) v5 ==== 1978+0+0 (2332821113 0 0) 0x53bcec0 con 0x45f5960 -642> 2014-08-13 17:22:51.975374 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:50.853621, event: header_read, request: pg_notify(24.136(22) epoch 194472) v5 -641> 2014-08-13 17:22:51.975381 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:50.853623, event: throttled, request: pg_notify(24.136(22) epoch 194472) v5 -640> 2014-08-13 17:22:51.975386 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:50.853726, event: all_read, request: pg_notify(24.136(22) epoch 194472) v5 -639> 2014-08-13 17:22:51.975395 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:51.975372, event: dispatched, request: pg_notify(24.136(22) epoch 194472) v5 -638> 2014-08-13 17:22:51.975405 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:51.975405, event: waiting_for_osdmap, request: pg_notify(24.136(22) epoch 194472) v5 -637> 2014-08-13 17:22:51.975416 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:51.975416, event: started, request: pg_notify(24.136(22) epoch 194472) v5 -636> 2014-08-13 17:22:51.975439 7f74d8735700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:22:51.975438, event: done, request: pg_notify(24.136(22) epoch 194472) v5 -635> 2014-08-13 17:22:51.975457 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.54 192.168.1.111:6836/14923 11 ==== pg_notify(22.1f1(22) epoch 194472) v5 ==== 1948+0+0 (3593550584 0 0) 0x49e7700 con 0x45f6f60 -634> 2014-08-13 17:22:51.975476 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:50.973077, event: header_read, request: pg_notify(22.1f1(22) epoch 194472) v5 -633> 2014-08-13 17:22:51.975485 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:50.973078, event: throttled, request: pg_notify(22.1f1(22) epoch 194472) v5 -632> 2014-08-13 17:22:51.975494 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:50.973137, event: all_read, request: pg_notify(22.1f1(22) epoch 194472) v5 -631> 2014-08-13 17:22:51.975503 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:51.975474, event: dispatched, request: pg_notify(22.1f1(22) epoch 194472) v5 -630> 2014-08-13 17:22:51.975513 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:51.975513, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194472) v5 -629> 2014-08-13 17:22:51.975524 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:51.975524, event: started, request: pg_notify(22.1f1(22) epoch 194472) v5 -628> 2014-08-13 17:22:51.975546 7f74d8735700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:22:51.975546, event: done, request: pg_notify(22.1f1(22) epoch 194472) v5 -627> 2014-08-13 17:22:51.975568 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 10 ==== pg_notify(22.99(18) epoch 194472) v5 ==== 1720+0+0 (3447818409 0 0) 0x57d6040 con 0x4758840 -626> 2014-08-13 17:22:51.975587 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.024956, event: header_read, request: pg_notify(22.99(18) epoch 194472) v5 -625> 2014-08-13 17:22:51.975597 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.024957, event: throttled, request: pg_notify(22.99(18) epoch 194472) v5 -624> 2014-08-13 17:22:51.975606 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.025028, event: all_read, request: pg_notify(22.99(18) epoch 194472) v5 -623> 2014-08-13 17:22:51.975615 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.975584, event: dispatched, request: pg_notify(22.99(18) epoch 194472) v5 -622> 2014-08-13 17:22:51.975625 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.975625, event: waiting_for_osdmap, request: pg_notify(22.99(18) epoch 194472) v5 -621> 2014-08-13 17:22:51.975635 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.975635, event: started, request: pg_notify(22.99(18) epoch 194472) v5 -620> 2014-08-13 17:22:51.975658 7f74d8735700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:22:51.975657, event: done, request: pg_notify(22.99(18) epoch 194472) v5 -619> 2014-08-13 17:22:51.975680 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 8 ==== pg_notify(24.9b(27) epoch 194472) v5 ==== 2273+0+0 (3450804279 0 0) 0x52639c0 con 0x47591e0 -618> 2014-08-13 17:22:51.975698 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:50.960514, event: header_read, request: pg_notify(24.9b(27) epoch 194472) v5 -617> 2014-08-13 17:22:51.975708 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:50.960515, event: throttled, request: pg_notify(24.9b(27) epoch 194472) v5 -616> 2014-08-13 17:22:51.975717 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:50.960581, event: all_read, request: pg_notify(24.9b(27) epoch 194472) v5 -615> 2014-08-13 17:22:51.975726 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:51.975696, event: dispatched, request: pg_notify(24.9b(27) epoch 194472) v5 -614> 2014-08-13 17:22:51.975736 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:51.975735, event: waiting_for_osdmap, request: pg_notify(24.9b(27) epoch 194472) v5 -613> 2014-08-13 17:22:51.975746 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:51.975746, event: started, request: pg_notify(24.9b(27) epoch 194472) v5 -612> 2014-08-13 17:22:51.975769 7f74d8735700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:22:51.975769, event: done, request: pg_notify(24.9b(27) epoch 194472) v5 -611> 2014-08-13 17:22:51.975755 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(30) epoch 194472) v5 -- ?+0 0x57d6ac0 con 0x47a53e0 -610> 2014-08-13 17:22:51.975794 7f74d192a700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 -609> 2014-08-13 17:22:51.975792 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194472) v5 -- ?+0 0x57d6200 con 0x4758840 -608> 2014-08-13 17:22:51.975806 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 15 ==== pg_notify(24.34(48) epoch 194472) v5 ==== 3196+0+0 (3985201095 0 0) 0x5deb480 con 0x475a940 -607> 2014-08-13 17:22:51.975815 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- osd_map(194472..194472 src has 185139..194472) v3 -- ?+0 0x4e31200 con 0x47f0c60 -606> 2014-08-13 17:22:51.975822 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.122978, event: header_read, request: pg_notify(24.34(48) epoch 194472) v5 -605> 2014-08-13 17:22:51.975829 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.122979, event: throttled, request: pg_notify(24.34(48) epoch 194472) v5 -604> 2014-08-13 17:22:51.975835 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.123135, event: all_read, request: pg_notify(24.34(48) epoch 194472) v5 -603> 2014-08-13 17:22:51.975832 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(24) epoch 194472) v5 -- ?+0 0x57d39c0 con 0x47f0c60 -602> 2014-08-13 17:22:51.975840 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.975820, event: dispatched, request: pg_notify(24.34(48) epoch 194472) v5 -601> 2014-08-13 17:22:51.975847 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.975846, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194472) v5 -600> 2014-08-13 17:22:51.975858 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.975858, event: started, request: pg_notify(24.34(48) epoch 194472) v5 -599> 2014-08-13 17:22:51.975882 7f74d8735700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:22:51.975882, event: done, request: pg_notify(24.34(48) epoch 194472) v5 -598> 2014-08-13 17:22:51.975923 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.285 192.168.1.101:6822/26295 9 ==== pg_info(1 pgs e194472:24.1e6(4)) v4 ==== 968+0+0 (2293602054 0 0) 0x4ac7700 con 0x475cd00 -597> 2014-08-13 17:22:51.975944 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:50.851309, event: header_read, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -596> 2014-08-13 17:22:51.975957 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:50.851310, event: throttled, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -595> 2014-08-13 17:22:51.975964 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:50.851385, event: all_read, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -594> 2014-08-13 17:22:51.975969 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:51.975942, event: dispatched, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -593> 2014-08-13 17:22:51.975975 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:51.975975, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -592> 2014-08-13 17:22:51.975983 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:51.975983, event: started, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -591> 2014-08-13 17:22:51.976005 7f74d8735700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:22:51.976005, event: done, request: pg_info(1 pgs e194472:24.1e6(4)) v4 -590> 2014-08-13 17:22:51.976028 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 16 ==== pg_notify(22.707(1) epoch 194472) v5 ==== 777+0+0 (1439537644 0 0) 0x53ba680 con 0x475ee00 -589> 2014-08-13 17:22:51.976043 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:50.906674, event: header_read, request: pg_notify(22.707(1) epoch 194472) v5 -588> 2014-08-13 17:22:51.976049 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:50.906679, event: throttled, request: pg_notify(22.707(1) epoch 194472) v5 -587> 2014-08-13 17:22:51.976006 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- osd_map(194472..194472 src has 185139..194472) v3 -- ?+0 0x4a869c0 con 0x4903180 -586> 2014-08-13 17:22:51.976055 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:50.906782, event: all_read, request: pg_notify(22.707(1) epoch 194472) v5 -585> 2014-08-13 17:22:51.976060 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:51.976041, event: dispatched, request: pg_notify(22.707(1) epoch 194472) v5 -584> 2014-08-13 17:22:51.976066 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:51.976066, event: waiting_for_osdmap, request: pg_notify(22.707(1) epoch 194472) v5 -583> 2014-08-13 17:22:51.976073 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:51.976073, event: started, request: pg_notify(22.707(1) epoch 194472) v5 -582> 2014-08-13 17:22:51.976076 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194472) v5 -- ?+0 0x53bb480 con 0x4903180 -581> 2014-08-13 17:22:51.976092 7f74d8735700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:22:51.976091, event: done, request: pg_notify(22.707(1) epoch 194472) v5 -580> 2014-08-13 17:22:51.976092 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(29) epoch 194472) v5 -- ?+0 0x53b9340 con 0x486e040 -579> 2014-08-13 17:22:51.976104 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.28 192.168.1.103:6805/37391 -- pg_log(24.b6 epoch 194472 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194472) v4 -- ?+0 0x5c71800 -578> 2014-08-13 17:22:51.976112 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.289 192.168.1.113:6808/15284 15 ==== pg_info(1 pgs e194472:24.5e(4)) v4 ==== 968+0+0 (1897705976 0 0) 0x4ac7a80 con 0x47a27e0 -577> 2014-08-13 17:22:51.976146 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:50.849999, event: header_read, request: pg_info(1 pgs e194472:24.5e(4)) v4 -576> 2014-08-13 17:22:51.976155 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:50.850001, event: throttled, request: pg_info(1 pgs e194472:24.5e(4)) v4 -575> 2014-08-13 17:22:51.976160 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:50.850078, event: all_read, request: pg_info(1 pgs e194472:24.5e(4)) v4 -574> 2014-08-13 17:22:51.976165 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:51.976143, event: dispatched, request: pg_info(1 pgs e194472:24.5e(4)) v4 -573> 2014-08-13 17:22:51.976171 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:51.976171, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.5e(4)) v4 -572> 2014-08-13 17:22:51.976186 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:51.976186, event: started, request: pg_info(1 pgs e194472:24.5e(4)) v4 -571> 2014-08-13 17:22:51.976182 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(26) epoch 194472) v5 -- ?+0 0x53bc0c0 con 0x486b180 -570> 2014-08-13 17:22:51.976195 7f74d192a700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 -569> 2014-08-13 17:22:51.976208 7f74d8735700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:22:51.976208, event: done, request: pg_info(1 pgs e194472:24.5e(4)) v4 -568> 2014-08-13 17:22:51.976225 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 14 ==== pg_notify(22.3af(1) epoch 194472) v5 ==== 777+0+0 (4249552551 0 0) 0x57d3d40 con 0x47a0b00 -567> 2014-08-13 17:22:51.976241 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:50.864662, event: header_read, request: pg_notify(22.3af(1) epoch 194472) v5 -566> 2014-08-13 17:22:51.976253 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:50.864664, event: throttled, request: pg_notify(22.3af(1) epoch 194472) v5 -565> 2014-08-13 17:22:51.976258 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:50.864783, event: all_read, request: pg_notify(22.3af(1) epoch 194472) v5 -564> 2014-08-13 17:22:51.976254 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.115:6802/58522 -- pg_info(1 pgs e194472:24.34(48)) v4 -- ?+0 0x53b8000 con 0x4d2aaa0 -563> 2014-08-13 17:22:51.976263 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:51.976239, event: dispatched, request: pg_notify(22.3af(1) epoch 194472) v5 -562> 2014-08-13 17:22:51.976269 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:51.976269, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194472) v5 -561> 2014-08-13 17:22:51.976277 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:51.976277, event: started, request: pg_notify(22.3af(1) epoch 194472) v5 -560> 2014-08-13 17:22:51.976295 7f74d8735700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:22:51.976295, event: done, request: pg_notify(22.3af(1) epoch 194472) v5 -559> 2014-08-13 17:22:51.976315 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 7 ==== pg_info(1 pgs e194472:24.1de(2)) v4 ==== 866+0+0 (3572973384 0 0) 0x53bcb40 con 0x47a4200 -558> 2014-08-13 17:22:51.976331 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:50.854378, event: header_read, request: pg_info(1 pgs e194472:24.1de(2)) v4 -557> 2014-08-13 17:22:51.976338 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:50.854379, event: throttled, request: pg_info(1 pgs e194472:24.1de(2)) v4 -556> 2014-08-13 17:22:51.976343 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:50.854450, event: all_read, request: pg_info(1 pgs e194472:24.1de(2)) v4 -555> 2014-08-13 17:22:51.976348 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:51.976329, event: dispatched, request: pg_info(1 pgs e194472:24.1de(2)) v4 -554> 2014-08-13 17:22:51.976354 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:51.976354, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1de(2)) v4 -553> 2014-08-13 17:22:51.976361 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:51.976361, event: started, request: pg_info(1 pgs e194472:24.1de(2)) v4 -552> 2014-08-13 17:22:51.976379 7f74d8735700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:22:51.976379, event: done, request: pg_info(1 pgs e194472:24.1de(2)) v4 -551> 2014-08-13 17:22:51.976396 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 11 ==== pg_info(1 pgs e194472:24.2a1(4)) v4 ==== 968+0+0 (2600915054 0 0) 0x53baa00 con 0x47a53e0 -550> 2014-08-13 17:22:51.976411 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:50.848703, event: header_read, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -549> 2014-08-13 17:22:51.976417 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:50.848704, event: throttled, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -548> 2014-08-13 17:22:51.976422 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:50.848768, event: all_read, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -547> 2014-08-13 17:22:51.976427 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:51.976409, event: dispatched, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -546> 2014-08-13 17:22:51.976433 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:51.976433, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -545> 2014-08-13 17:22:51.976440 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:51.976440, event: started, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -544> 2014-08-13 17:22:51.976458 7f74d8735700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:22:51.976458, event: done, request: pg_info(1 pgs e194472:24.2a1(4)) v4 -543> 2014-08-13 17:22:51.976485 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 13 ==== pg_notify(24.34(48) epoch 194472) v5 ==== 3196+0+0 (2784658797 0 0) 0x49e0000 con 0x47a3f40 -542> 2014-08-13 17:22:51.976499 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:50.951956, event: header_read, request: pg_notify(24.34(48) epoch 194472) v5 -541> 2014-08-13 17:22:51.976505 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:50.951958, event: throttled, request: pg_notify(24.34(48) epoch 194472) v5 -540> 2014-08-13 17:22:51.976511 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:50.952040, event: all_read, request: pg_notify(24.34(48) epoch 194472) v5 -539> 2014-08-13 17:22:51.976516 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:51.976497, event: dispatched, request: pg_notify(24.34(48) epoch 194472) v5 -538> 2014-08-13 17:22:51.976522 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:51.976521, event: waiting_for_osdmap, request: pg_notify(24.34(48) epoch 194472) v5 -537> 2014-08-13 17:22:51.976528 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:51.976528, event: started, request: pg_notify(24.34(48) epoch 194472) v5 -536> 2014-08-13 17:22:51.976548 7f74d8735700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:22:51.976548, event: done, request: pg_notify(24.34(48) epoch 194472) v5 -535> 2014-08-13 17:22:51.976531 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.330 192.168.1.106:6802/4876 -- pg_log(24.b6 epoch 194472 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194472) v4 -- ?+0 0x5c77800 -534> 2014-08-13 17:22:51.976595 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- osd_map(194472..194472 src has 185139..194472) v3 -- ?+0 0x480fbc0 con 0x475a940 -533> 2014-08-13 17:22:51.976606 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.44 192.168.1.111:6822/10782 9 ==== pg_notify(22.619(1) epoch 194472) v5 ==== 777+0+0 (2901795295 0 0) 0x466f8c0 con 0x47a4ba0 -532> 2014-08-13 17:22:51.976624 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:50.962118, event: header_read, request: pg_notify(22.619(1) epoch 194472) v5 -531> 2014-08-13 17:22:51.976630 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:50.962119, event: throttled, request: pg_notify(22.619(1) epoch 194472) v5 -530> 2014-08-13 17:22:51.976635 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:50.962170, event: all_read, request: pg_notify(22.619(1) epoch 194472) v5 -529> 2014-08-13 17:22:51.976641 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:51.976621, event: dispatched, request: pg_notify(22.619(1) epoch 194472) v5 -528> 2014-08-13 17:22:51.976646 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:51.976646, event: waiting_for_osdmap, request: pg_notify(22.619(1) epoch 194472) v5 -527> 2014-08-13 17:22:51.976654 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:51.976653, event: started, request: pg_notify(22.619(1) epoch 194472) v5 -526> 2014-08-13 17:22:51.976652 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6823/55491 -- pg_info(1 pgs e194472:24.34(48)) v4 -- ?+0 0x53bbf00 con 0x475a940 -525> 2014-08-13 17:22:51.976672 7f74d8735700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:22:51.976672, event: done, request: pg_notify(22.619(1) epoch 194472) v5 -524> 2014-08-13 17:22:51.976707 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 12 ==== pg_notify(22.3e4(22) epoch 194472) v5 ==== 1948+0+0 (2015717743 0 0) 0x53bc600 con 0x47a32e0 -523> 2014-08-13 17:22:51.976723 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:50.848486, event: header_read, request: pg_notify(22.3e4(22) epoch 194472) v5 -522> 2014-08-13 17:22:51.976730 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:50.848487, event: throttled, request: pg_notify(22.3e4(22) epoch 194472) v5 -521> 2014-08-13 17:22:51.976735 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:50.848587, event: all_read, request: pg_notify(22.3e4(22) epoch 194472) v5 -520> 2014-08-13 17:22:51.976740 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:51.976721, event: dispatched, request: pg_notify(22.3e4(22) epoch 194472) v5 -519> 2014-08-13 17:22:51.976746 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:51.976746, event: waiting_for_osdmap, request: pg_notify(22.3e4(22) epoch 194472) v5 -518> 2014-08-13 17:22:51.976739 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194472:24.34(48)) v4 -- ?+0 0x53bfa80 con 0x47a3f40 -517> 2014-08-13 17:22:51.976753 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:51.976753, event: started, request: pg_notify(22.3e4(22) epoch 194472) v5 -516> 2014-08-13 17:22:51.976771 7f74d8735700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:22:51.976771, event: done, request: pg_notify(22.3e4(22) epoch 194472) v5 -515> 2014-08-13 17:22:51.976803 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 15 ==== pg_notify(24.1f7(2) epoch 194472) v5 ==== 870+0+0 (1975225476 0 0) 0x5c41f80 con 0x47f7e80 -514> 2014-08-13 17:22:51.976818 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:50.878531, event: header_read, request: pg_notify(24.1f7(2) epoch 194472) v5 -513> 2014-08-13 17:22:51.976825 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:50.878533, event: throttled, request: pg_notify(24.1f7(2) epoch 194472) v5 -512> 2014-08-13 17:22:51.976830 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:50.878642, event: all_read, request: pg_notify(24.1f7(2) epoch 194472) v5 -511> 2014-08-13 17:22:51.976835 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:51.976816, event: dispatched, request: pg_notify(24.1f7(2) epoch 194472) v5 -510> 2014-08-13 17:22:51.976840 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:51.976840, event: waiting_for_osdmap, request: pg_notify(24.1f7(2) epoch 194472) v5 -509> 2014-08-13 17:22:51.976847 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:51.976847, event: started, request: pg_notify(24.1f7(2) epoch 194472) v5 -508> 2014-08-13 17:22:51.976867 7f74d8735700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:22:51.976866, event: done, request: pg_notify(24.1f7(2) epoch 194472) v5 -507> 2014-08-13 17:22:51.976881 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 16 ==== pg_notify(24.1bb(26) epoch 194472) v5 ==== 2214+0+0 (1542312338 0 0) 0x5c46ac0 con 0x4833440 -506> 2014-08-13 17:22:51.976895 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:50.903866, event: header_read, request: pg_notify(24.1bb(26) epoch 194472) v5 -505> 2014-08-13 17:22:51.976901 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:50.903868, event: throttled, request: pg_notify(24.1bb(26) epoch 194472) v5 -504> 2014-08-13 17:22:51.976906 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:50.903985, event: all_read, request: pg_notify(24.1bb(26) epoch 194472) v5 -503> 2014-08-13 17:22:51.976911 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:51.976893, event: dispatched, request: pg_notify(24.1bb(26) epoch 194472) v5 -502> 2014-08-13 17:22:51.976916 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:51.976916, event: waiting_for_osdmap, request: pg_notify(24.1bb(26) epoch 194472) v5 -501> 2014-08-13 17:22:51.976923 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:51.976923, event: started, request: pg_notify(24.1bb(26) epoch 194472) v5 -500> 2014-08-13 17:22:51.976941 7f74d8735700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:22:51.976941, event: done, request: pg_notify(24.1bb(26) epoch 194472) v5 -499> 2014-08-13 17:22:51.976972 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.376 192.168.1.108:6838/6893 21 ==== pg_notify(22.48e(20) epoch 194472) v5 ==== 1834+0+0 (4178568222 0 0) 0x5c43100 con 0x4835280 -498> 2014-08-13 17:22:51.976987 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:50.869329, event: header_read, request: pg_notify(22.48e(20) epoch 194472) v5 -497> 2014-08-13 17:22:51.976993 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:50.869331, event: throttled, request: pg_notify(22.48e(20) epoch 194472) v5 -496> 2014-08-13 17:22:51.976998 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:50.869468, event: all_read, request: pg_notify(22.48e(20) epoch 194472) v5 -495> 2014-08-13 17:22:51.977003 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:51.976985, event: dispatched, request: pg_notify(22.48e(20) epoch 194472) v5 -494> 2014-08-13 17:22:51.977009 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:51.977009, event: waiting_for_osdmap, request: pg_notify(22.48e(20) epoch 194472) v5 -493> 2014-08-13 17:22:51.977016 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:51.977016, event: started, request: pg_notify(22.48e(20) epoch 194472) v5 -492> 2014-08-13 17:22:51.977034 7f74d8735700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:22:51.977034, event: done, request: pg_notify(22.48e(20) epoch 194472) v5 -491> 2014-08-13 17:22:51.977070 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.320 192.168.1.106:6820/10050 16 ==== pg_notify(24.181(3) epoch 194472) v5 ==== 925+0+0 (3566896431 0 0) 0x5c41c00 con 0x486bc80 -490> 2014-08-13 17:22:51.977085 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:50.862736, event: header_read, request: pg_notify(24.181(3) epoch 194472) v5 -489> 2014-08-13 17:22:51.977092 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:50.862738, event: throttled, request: pg_notify(24.181(3) epoch 194472) v5 -488> 2014-08-13 17:22:51.977097 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:50.862862, event: all_read, request: pg_notify(24.181(3) epoch 194472) v5 -487> 2014-08-13 17:22:51.977102 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:51.977083, event: dispatched, request: pg_notify(24.181(3) epoch 194472) v5 -486> 2014-08-13 17:22:51.977108 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:51.977108, event: waiting_for_osdmap, request: pg_notify(24.181(3) epoch 194472) v5 -485> 2014-08-13 17:22:51.977115 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:51.977115, event: started, request: pg_notify(24.181(3) epoch 194472) v5 -484> 2014-08-13 17:22:51.977133 7f74d8735700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:22:51.977133, event: done, request: pg_notify(24.181(3) epoch 194472) v5 -483> 2014-08-13 17:22:51.977162 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.327 192.168.1.106:6804/5112 12 ==== pg_notify(24.b6(2) epoch 194472) v5 ==== 870+0+0 (3073796420 0 0) 0x49e1180 con 0x486b700 -482> 2014-08-13 17:22:51.977184 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:50.881810, event: header_read, request: pg_notify(24.b6(2) epoch 194472) v5 -481> 2014-08-13 17:22:51.977192 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:50.881812, event: throttled, request: pg_notify(24.b6(2) epoch 194472) v5 -480> 2014-08-13 17:22:51.977197 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:50.881913, event: all_read, request: pg_notify(24.b6(2) epoch 194472) v5 -479> 2014-08-13 17:22:51.977202 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:51.977175, event: dispatched, request: pg_notify(24.b6(2) epoch 194472) v5 -478> 2014-08-13 17:22:51.977208 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:51.977208, event: waiting_for_osdmap, request: pg_notify(24.b6(2) epoch 194472) v5 -477> 2014-08-13 17:22:51.977215 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:51.977215, event: started, request: pg_notify(24.b6(2) epoch 194472) v5 -476> 2014-08-13 17:22:51.977609 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=194471 pi=193748-194470/25 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -475> 2014-08-13 17:22:51.979103 7f74d8735700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:22:51.979102, event: done, request: pg_notify(24.b6(2) epoch 194472) v5 -474> 2014-08-13 17:22:51.979174 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.373 192.168.1.108:6840/7435 18 ==== pg_notify(22.3d9(35) epoch 194472) v5 ==== 2627+0+0 (2562706520 0 0) 0x53be900 con 0x486a3c0 -473> 2014-08-13 17:22:51.979210 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:50.896613, event: header_read, request: pg_notify(22.3d9(35) epoch 194472) v5 -472> 2014-08-13 17:22:51.979220 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:50.896615, event: throttled, request: pg_notify(22.3d9(35) epoch 194472) v5 -471> 2014-08-13 17:22:51.979226 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:50.896732, event: all_read, request: pg_notify(22.3d9(35) epoch 194472) v5 -470> 2014-08-13 17:22:51.979232 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:51.979207, event: dispatched, request: pg_notify(22.3d9(35) epoch 194472) v5 -469> 2014-08-13 17:22:51.979238 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:51.979238, event: waiting_for_osdmap, request: pg_notify(22.3d9(35) epoch 194472) v5 -468> 2014-08-13 17:22:51.979247 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:51.979247, event: started, request: pg_notify(22.3d9(35) epoch 194472) v5 -467> 2014-08-13 17:22:51.979281 7f74d8735700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:22:51.979281, event: done, request: pg_notify(22.3d9(35) epoch 194472) v5 -466> 2014-08-13 17:22:51.979254 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 2.025272 4 0.000270 -465> 2014-08-13 17:22:51.979294 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 2.117538 0 0.000000 -464> 2014-08-13 17:22:51.979328 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194449 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active -463> 2014-08-13 17:22:51.979334 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.15 192.168.1.107:6812/53408 11 ==== pg_info(1 pgs e194472:24.20e(2)) v4 ==== 866+0+0 (2544613393 0 0) 0x53bb100 con 0x48d9ce0 -462> 2014-08-13 17:22:51.979323 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 2.104503 3 0.000214 -461> 2014-08-13 17:22:51.979353 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:50.855842, event: header_read, request: pg_info(1 pgs e194472:24.20e(2)) v4 -460> 2014-08-13 17:22:51.979376 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:50.855843, event: throttled, request: pg_info(1 pgs e194472:24.20e(2)) v4 -459> 2014-08-13 17:22:51.979371 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 2.133150 0 0.000000 -458> 2014-08-13 17:22:51.979387 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:50.855900, event: all_read, request: pg_info(1 pgs e194472:24.20e(2)) v4 -457> 2014-08-13 17:22:51.979406 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:51.979351, event: dispatched, request: pg_info(1 pgs e194472:24.20e(2)) v4 -456> 2014-08-13 17:22:51.979405 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194449 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active -455> 2014-08-13 17:22:51.979419 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:51.979419, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.20e(2)) v4 -454> 2014-08-13 17:22:51.979440 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:51.979440, event: started, request: pg_info(1 pgs e194472:24.20e(2)) v4 -453> 2014-08-13 17:22:51.979468 7f74d8735700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:22:51.979468, event: done, request: pg_info(1 pgs e194472:24.20e(2)) v4 -452> 2014-08-13 17:22:51.979505 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 13 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c73f00 con 0x486e040 -451> 2014-08-13 17:22:51.979532 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -450> 2014-08-13 17:22:51.979556 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.397 192.168.1.105:6808/41085 13 ==== replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194472,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (828391270 0 0) 0x4c772c0 con 0x48dfbc0 -449> 2014-08-13 17:22:51.979589 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.342 192.168.1.110:6802/41438 14 ==== pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 ==== 4002+0+0 (2478534352 0 0) 0x5c463c0 con 0x4901600 -448> 2014-08-13 17:22:51.979609 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:50.893454, event: header_read, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -447> 2014-08-13 17:22:51.979621 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:50.893456, event: throttled, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -446> 2014-08-13 17:22:51.979631 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:50.893599, event: all_read, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -445> 2014-08-13 17:22:51.979641 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:51.979607, event: dispatched, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -444> 2014-08-13 17:22:51.979651 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:51.979651, event: waiting_for_osdmap, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -443> 2014-08-13 17:22:51.979663 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:51.979662, event: started, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -442> 2014-08-13 17:22:51.979707 7f74d8735700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:22:51.979707, event: done, request: pg_notify(22.3e4(22),22.707(24) epoch 194472) v5 -441> 2014-08-13 17:22:51.979743 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.368 192.168.1.102:6826/15861 11 ==== pg_notify(22.1f1(22) epoch 194472) v5 ==== 1948+0+0 (3593550584 0 0) 0x5c45940 con 0x4902680 -440> 2014-08-13 17:22:51.979763 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:50.893936, event: header_read, request: pg_notify(22.1f1(22) epoch 194472) v5 -439> 2014-08-13 17:22:51.979774 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:50.893939, event: throttled, request: pg_notify(22.1f1(22) epoch 194472) v5 -438> 2014-08-13 17:22:51.979783 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:50.894046, event: all_read, request: pg_notify(22.1f1(22) epoch 194472) v5 -437> 2014-08-13 17:22:51.979792 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:51.979761, event: dispatched, request: pg_notify(22.1f1(22) epoch 194472) v5 -436> 2014-08-13 17:22:51.979802 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:51.979801, event: waiting_for_osdmap, request: pg_notify(22.1f1(22) epoch 194472) v5 -435> 2014-08-13 17:22:51.979813 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:51.979813, event: started, request: pg_notify(22.1f1(22) epoch 194472) v5 -434> 2014-08-13 17:22:51.979838 7f74d8735700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:22:51.979837, event: done, request: pg_notify(22.1f1(22) epoch 194472) v5 -433> 2014-08-13 17:22:51.979864 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.375 192.168.1.114:6830/52879 9 ==== pg_info(1 pgs e194472:24.30b) v4 ==== 748+0+0 (366107680 0 0) 0x4ac1180 con 0x49023c0 -432> 2014-08-13 17:22:51.979883 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.975798, event: header_read, request: pg_info(1 pgs e194472:24.30b) v4 -431> 2014-08-13 17:22:51.979893 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.975802, event: throttled, request: pg_info(1 pgs e194472:24.30b) v4 -430> 2014-08-13 17:22:51.979902 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.975899, event: all_read, request: pg_info(1 pgs e194472:24.30b) v4 -429> 2014-08-13 17:22:51.979911 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.979881, event: dispatched, request: pg_info(1 pgs e194472:24.30b) v4 -428> 2014-08-13 17:22:51.979920 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.979920, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.30b) v4 -427> 2014-08-13 17:22:51.979931 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.979931, event: started, request: pg_info(1 pgs e194472:24.30b) v4 -426> 2014-08-13 17:22:51.979954 7f74d8735700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:22:51.979954, event: done, request: pg_info(1 pgs e194472:24.30b) v4 -425> 2014-08-13 17:22:51.979976 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.369 192.168.1.108:6824/4037 13 ==== pg_notify(22.775(1) epoch 194472) v5 ==== 777+0+0 (2222378935 0 0) 0x53bdb00 con 0x4d58840 -424> 2014-08-13 17:22:51.979995 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:50.890581, event: header_read, request: pg_notify(22.775(1) epoch 194472) v5 -423> 2014-08-13 17:22:51.980005 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:50.890585, event: throttled, request: pg_notify(22.775(1) epoch 194472) v5 -422> 2014-08-13 17:22:51.980014 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:50.890681, event: all_read, request: pg_notify(22.775(1) epoch 194472) v5 -421> 2014-08-13 17:22:51.980023 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:51.979993, event: dispatched, request: pg_notify(22.775(1) epoch 194472) v5 -420> 2014-08-13 17:22:51.980033 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:51.980033, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194472) v5 -419> 2014-08-13 17:22:51.980044 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:51.980044, event: started, request: pg_notify(22.775(1) epoch 194472) v5 -418> 2014-08-13 17:22:51.980066 7f74d8735700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:22:51.980066, event: done, request: pg_notify(22.775(1) epoch 194472) v5 -417> 2014-08-13 17:22:51.980083 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.363 192.168.1.102:6842/20017 11 ==== pg_notify(22.39d(20) epoch 194472) v5 ==== 1834+0+0 (2070222226 0 0) 0x5dea840 con 0x4d582c0 -416> 2014-08-13 17:22:51.980101 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.026261, event: header_read, request: pg_notify(22.39d(20) epoch 194472) v5 -415> 2014-08-13 17:22:51.980112 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.026262, event: throttled, request: pg_notify(22.39d(20) epoch 194472) v5 -414> 2014-08-13 17:22:51.980121 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.026317, event: all_read, request: pg_notify(22.39d(20) epoch 194472) v5 -413> 2014-08-13 17:22:51.980130 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.980099, event: dispatched, request: pg_notify(22.39d(20) epoch 194472) v5 -412> 2014-08-13 17:22:51.980140 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.980139, event: waiting_for_osdmap, request: pg_notify(22.39d(20) epoch 194472) v5 -411> 2014-08-13 17:22:51.980151 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.980150, event: started, request: pg_notify(22.39d(20) epoch 194472) v5 -410> 2014-08-13 17:22:51.980173 7f74d8735700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:22:51.980173, event: done, request: pg_notify(22.39d(20) epoch 194472) v5 -409> 2014-08-13 17:22:51.980209 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.365 192.168.1.102:6824/15623 11 ==== pg_notify(22.481(32) epoch 194472) v5 ==== 2518+0+0 (3477336213 0 0) 0x5c45240 con 0x4d51ce0 -408> 2014-08-13 17:22:51.980228 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:50.892091, event: header_read, request: pg_notify(22.481(32) epoch 194472) v5 -407> 2014-08-13 17:22:51.980239 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:50.892093, event: throttled, request: pg_notify(22.481(32) epoch 194472) v5 -406> 2014-08-13 17:22:51.980248 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:50.892219, event: all_read, request: pg_notify(22.481(32) epoch 194472) v5 -405> 2014-08-13 17:22:51.980257 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:51.980226, event: dispatched, request: pg_notify(22.481(32) epoch 194472) v5 -404> 2014-08-13 17:22:51.980267 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:51.980266, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194472) v5 -403> 2014-08-13 17:22:51.980277 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:51.980277, event: started, request: pg_notify(22.481(32) epoch 194472) v5 -402> 2014-08-13 17:22:51.980307 7f74d8735700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:22:51.980306, event: done, request: pg_notify(22.481(32) epoch 194472) v5 -401> 2014-08-13 17:22:51.980334 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.350 192.168.1.114:6832/53185 9 ==== pg_info(1 pgs e194472:24.9b) v4 ==== 748+0+0 (1158007667 0 0) 0x49e01c0 con 0x4d55ac0 -400> 2014-08-13 17:22:51.980353 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.975991, event: header_read, request: pg_info(1 pgs e194472:24.9b) v4 -399> 2014-08-13 17:22:51.980363 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.975994, event: throttled, request: pg_info(1 pgs e194472:24.9b) v4 -398> 2014-08-13 17:22:51.980372 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.976088, event: all_read, request: pg_info(1 pgs e194472:24.9b) v4 -397> 2014-08-13 17:22:51.980415 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.980351, event: dispatched, request: pg_info(1 pgs e194472:24.9b) v4 -396> 2014-08-13 17:22:51.980425 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.980425, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.9b) v4 -395> 2014-08-13 17:22:51.980436 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.980436, event: started, request: pg_info(1 pgs e194472:24.9b) v4 -394> 2014-08-13 17:22:51.980460 7f74d8735700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:22:51.980459, event: done, request: pg_info(1 pgs e194472:24.9b) v4 -393> 2014-08-13 17:22:51.980477 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.26 192.168.1.115:6802/58522 6 ==== pg_info(1 pgs e194472:24.34) v4 ==== 740+0+0 (521832053 0 0) 0x53b8000 con 0x4d2aaa0 -392> 2014-08-13 17:22:51.980496 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.978838, event: header_read, request: pg_info(1 pgs e194472:24.34) v4 -391> 2014-08-13 17:22:51.980506 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.978840, event: throttled, request: pg_info(1 pgs e194472:24.34) v4 -390> 2014-08-13 17:22:51.980515 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.978909, event: all_read, request: pg_info(1 pgs e194472:24.34) v4 -389> 2014-08-13 17:22:51.980523 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.980494, event: dispatched, request: pg_info(1 pgs e194472:24.34) v4 -388> 2014-08-13 17:22:51.980533 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.980533, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.34) v4 -387> 2014-08-13 17:22:51.980544 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.980544, event: started, request: pg_info(1 pgs e194472:24.34) v4 -386> 2014-08-13 17:22:51.980570 7f74d8735700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:22:51.980570, event: done, request: pg_info(1 pgs e194472:24.34) v4 -385> 2014-08-13 17:22:51.980592 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.10 192.168.1.115:6815/60573 12 ==== pg_info(1 pgs e194472:24.30b) v4 ==== 748+0+0 (1199983632 0 0) 0x49e6e40 con 0x45f1760 -384> 2014-08-13 17:22:51.980610 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.976504, event: header_read, request: pg_info(1 pgs e194472:24.30b) v4 -383> 2014-08-13 17:22:51.980620 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.976507, event: throttled, request: pg_info(1 pgs e194472:24.30b) v4 -382> 2014-08-13 17:22:51.980629 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.976584, event: all_read, request: pg_info(1 pgs e194472:24.30b) v4 -381> 2014-08-13 17:22:51.980638 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.980608, event: dispatched, request: pg_info(1 pgs e194472:24.30b) v4 -380> 2014-08-13 17:22:51.980648 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.980647, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.30b) v4 -379> 2014-08-13 17:22:51.980658 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.980658, event: started, request: pg_info(1 pgs e194472:24.30b) v4 -378> 2014-08-13 17:22:51.980680 7f74d8735700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:22:51.980680, event: done, request: pg_info(1 pgs e194472:24.30b) v4 -377> 2014-08-13 17:22:51.980696 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.356 192.168.1.108:6832/5235 17 ==== pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 ==== 1826+0+0 (1429521818 0 0) 0x49e0380 con 0x45f5960 -376> 2014-08-13 17:22:51.980716 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:50.853810, event: header_read, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -375> 2014-08-13 17:22:51.980727 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:50.853812, event: throttled, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -374> 2014-08-13 17:22:51.980737 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:50.853917, event: all_read, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -373> 2014-08-13 17:22:51.980746 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:51.980714, event: dispatched, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -372> 2014-08-13 17:22:51.980756 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:51.980756, event: waiting_for_osdmap, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -371> 2014-08-13 17:22:51.980768 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:51.980767, event: started, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -370> 2014-08-13 17:22:51.980804 7f74d8735700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:22:51.980804, event: done, request: pg_info(2 pgs e194472:24.350(4),24.3b7(2)) v4 -369> 2014-08-13 17:22:51.980834 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 11 ==== pg_notify(24.3d(2) epoch 194472) v5 ==== 870+0+0 (4060925761 0 0) 0x57d5e80 con 0x4758840 -368> 2014-08-13 17:22:51.980854 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.025039, event: header_read, request: pg_notify(24.3d(2) epoch 194472) v5 -367> 2014-08-13 17:22:51.980864 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.025039, event: throttled, request: pg_notify(24.3d(2) epoch 194472) v5 -366> 2014-08-13 17:22:51.980873 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.025077, event: all_read, request: pg_notify(24.3d(2) epoch 194472) v5 -365> 2014-08-13 17:22:51.980882 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.980851, event: dispatched, request: pg_notify(24.3d(2) epoch 194472) v5 -364> 2014-08-13 17:22:51.980892 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.980892, event: waiting_for_osdmap, request: pg_notify(24.3d(2) epoch 194472) v5 -363> 2014-08-13 17:22:51.980903 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.980903, event: started, request: pg_notify(24.3d(2) epoch 194472) v5 -362> 2014-08-13 17:22:51.980927 7f74d8735700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:22:51.980926, event: done, request: pg_notify(24.3d(2) epoch 194472) v5 -361> 2014-08-13 17:22:51.980944 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.291 192.168.1.101:6820/25633 9 ==== pg_info(1 pgs e194472:24.9b) v4 ==== 748+0+0 (4218985522 0 0) 0x49e0540 con 0x47591e0 -360> 2014-08-13 17:22:51.980962 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.975972, event: header_read, request: pg_info(1 pgs e194472:24.9b) v4 -359> 2014-08-13 17:22:51.980972 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.975974, event: throttled, request: pg_info(1 pgs e194472:24.9b) v4 -358> 2014-08-13 17:22:51.980981 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.976047, event: all_read, request: pg_info(1 pgs e194472:24.9b) v4 -357> 2014-08-13 17:22:51.980990 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.980960, event: dispatched, request: pg_info(1 pgs e194472:24.9b) v4 -356> 2014-08-13 17:22:51.980996 7f74d192a700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 -355> 2014-08-13 17:22:51.980999 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.980999, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.9b) v4 -354> 2014-08-13 17:22:51.981007 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.981007, event: started, request: pg_info(1 pgs e194472:24.9b) v4 -353> 2014-08-13 17:22:51.981011 7f74d0f29700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 -352> 2014-08-13 17:22:51.981025 7f74d8735700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:22:51.981025, event: done, request: pg_info(1 pgs e194472:24.9b) v4 -351> 2014-08-13 17:22:51.981038 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.30 192.168.1.107:6823/55491 16 ==== pg_info(1 pgs e194472:24.34) v4 ==== 740+0+0 (3434788435 0 0) 0x53bbf00 con 0x475a940 -350> 2014-08-13 17:22:51.981051 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.978981, event: header_read, request: pg_info(1 pgs e194472:24.34) v4 -349> 2014-08-13 17:22:51.981062 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.978984, event: throttled, request: pg_info(1 pgs e194472:24.34) v4 -348> 2014-08-13 17:22:51.981071 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.979073, event: all_read, request: pg_info(1 pgs e194472:24.34) v4 -347> 2014-08-13 17:22:51.981079 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.981050, event: dispatched, request: pg_info(1 pgs e194472:24.34) v4 -346> 2014-08-13 17:22:51.981088 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.981088, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.34) v4 -345> 2014-08-13 17:22:51.981099 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.981099, event: started, request: pg_info(1 pgs e194472:24.34) v4 -344> 2014-08-13 17:22:51.981122 7f74d8735700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:22:51.981121, event: done, request: pg_info(1 pgs e194472:24.34) v4 -343> 2014-08-13 17:22:51.981148 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 17 ==== pg_notify(24.3d(24) epoch 194472) v5 ==== 2096+0+0 (3094728919 0 0) 0x53ba4c0 con 0x475ee00 -342> 2014-08-13 17:22:51.981167 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:50.906977, event: header_read, request: pg_notify(24.3d(24) epoch 194472) v5 -341> 2014-08-13 17:22:51.981189 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:50.906978, event: throttled, request: pg_notify(24.3d(24) epoch 194472) v5 -340> 2014-08-13 17:22:51.981199 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:50.907090, event: all_read, request: pg_notify(24.3d(24) epoch 194472) v5 -339> 2014-08-13 17:22:51.981208 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:51.981165, event: dispatched, request: pg_notify(24.3d(24) epoch 194472) v5 -338> 2014-08-13 17:22:51.981218 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:51.981217, event: waiting_for_osdmap, request: pg_notify(24.3d(24) epoch 194472) v5 -337> 2014-08-13 17:22:51.981229 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:51.981228, event: started, request: pg_notify(24.3d(24) epoch 194472) v5 -336> 2014-08-13 17:22:51.981252 7f74d8735700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:22:51.981252, event: done, request: pg_notify(24.3d(24) epoch 194472) v5 -335> 2014-08-13 17:22:51.981275 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.61 192.168.1.111:6840/16566 15 ==== pg_notify(22.362(1) epoch 194472) v5 ==== 777+0+0 (4070920889 0 0) 0x5c42840 con 0x47a0b00 -334> 2014-08-13 17:22:51.981294 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:50.867497, event: header_read, request: pg_notify(22.362(1) epoch 194472) v5 -333> 2014-08-13 17:22:51.981304 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:50.867500, event: throttled, request: pg_notify(22.362(1) epoch 194472) v5 -332> 2014-08-13 17:22:51.981313 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:50.867578, event: all_read, request: pg_notify(22.362(1) epoch 194472) v5 -331> 2014-08-13 17:22:51.981322 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:51.981292, event: dispatched, request: pg_notify(22.362(1) epoch 194472) v5 -330> 2014-08-13 17:22:51.981331 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:51.981331, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194472) v5 -329> 2014-08-13 17:22:51.981342 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:51.981342, event: started, request: pg_notify(22.362(1) epoch 194472) v5 -328> 2014-08-13 17:22:51.981365 7f74d8735700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:22:51.981364, event: done, request: pg_notify(22.362(1) epoch 194472) v5 -327> 2014-08-13 17:22:51.981385 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 12 ==== pg_notify(24.b6(28) epoch 194472) v5 ==== 2332+0+0 (1163104659 0 0) 0x5c40380 con 0x47a53e0 -326> 2014-08-13 17:22:51.981404 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:50.849404, event: header_read, request: pg_notify(24.b6(28) epoch 194472) v5 -325> 2014-08-13 17:22:51.981414 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:50.849406, event: throttled, request: pg_notify(24.b6(28) epoch 194472) v5 -324> 2014-08-13 17:22:51.981424 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:50.849537, event: all_read, request: pg_notify(24.b6(28) epoch 194472) v5 -323> 2014-08-13 17:22:51.981433 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:51.981402, event: dispatched, request: pg_notify(24.b6(28) epoch 194472) v5 -322> 2014-08-13 17:22:51.981443 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:51.981442, event: waiting_for_osdmap, request: pg_notify(24.b6(28) epoch 194472) v5 -321> 2014-08-13 17:22:51.981454 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:51.981454, event: started, request: pg_notify(24.b6(28) epoch 194472) v5 -320> 2014-08-13 17:22:51.981477 7f74d8735700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:22:51.981477, event: done, request: pg_notify(24.b6(28) epoch 194472) v5 -319> 2014-08-13 17:22:51.981506 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.340 192.168.1.104:6806/62463 14 ==== pg_info(1 pgs e194472:24.34) v4 ==== 740+0+0 (3841002161 0 0) 0x53bfa80 con 0x47a3f40 -318> 2014-08-13 17:22:51.981525 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.979714, event: header_read, request: pg_info(1 pgs e194472:24.34) v4 -317> 2014-08-13 17:22:51.981535 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.979717, event: throttled, request: pg_info(1 pgs e194472:24.34) v4 -316> 2014-08-13 17:22:51.981524 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.59 192.168.1.112:6822/5265 -- pg_log(24.1f7 epoch 194472 log log((192718'12872,194422'15456], crt=194422'15456) query_epoch 194472) v4 -- ?+0 0x5c73600 -315> 2014-08-13 17:22:51.981544 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.979807, event: all_read, request: pg_info(1 pgs e194472:24.34) v4 -314> 2014-08-13 17:22:51.981571 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.981523, event: dispatched, request: pg_info(1 pgs e194472:24.34) v4 -313> 2014-08-13 17:22:51.981583 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.981583, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.34) v4 -312> 2014-08-13 17:22:51.981575 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.331 192.168.1.104:6816/837 -- pg_log(24.1bb epoch 194472 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194472) v4 -- ?+0 0x5b1e000 -311> 2014-08-13 17:22:51.981595 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.981595, event: started, request: pg_info(1 pgs e194472:24.34) v4 -310> 2014-08-13 17:22:51.981632 7f74d8735700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:22:51.981632, event: done, request: pg_info(1 pgs e194472:24.34) v4 -309> 2014-08-13 17:22:51.981651 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 13 ==== pg_info(1 pgs e194472:24.369(4)) v4 ==== 968+0+0 (2130782724 0 0) 0x53bfe00 con 0x47a32e0 -308> 2014-08-13 17:22:51.981671 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:50.849822, event: header_read, request: pg_info(1 pgs e194472:24.369(4)) v4 -307> 2014-08-13 17:22:51.981678 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:50.849824, event: throttled, request: pg_info(1 pgs e194472:24.369(4)) v4 -306> 2014-08-13 17:22:51.981688 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:50.849908, event: all_read, request: pg_info(1 pgs e194472:24.369(4)) v4 -305> 2014-08-13 17:22:51.981693 7f74d0f29700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 -304> 2014-08-13 17:22:51.981694 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:51.981669, event: dispatched, request: pg_info(1 pgs e194472:24.369(4)) v4 -303> 2014-08-13 17:22:51.981701 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:51.981700, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.369(4)) v4 -302> 2014-08-13 17:22:51.981708 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:51.981708, event: started, request: pg_info(1 pgs e194472:24.369(4)) v4 -301> 2014-08-13 17:22:51.981726 7f74d8735700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:22:51.981726, event: done, request: pg_info(1 pgs e194472:24.369(4)) v4 -300> 2014-08-13 17:22:51.981756 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.55 192.168.1.112:6834/6966 16 ==== pg_notify(24.181(3) epoch 194472) v5 ==== 925+0+0 (2363029722 0 0) 0x5c41dc0 con 0x47f7e80 -299> 2014-08-13 17:22:51.981772 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:50.878663, event: header_read, request: pg_notify(24.181(3) epoch 194472) v5 -298> 2014-08-13 17:22:51.981804 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:50.878664, event: throttled, request: pg_notify(24.181(3) epoch 194472) v5 -297> 2014-08-13 17:22:51.981810 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:50.878733, event: all_read, request: pg_notify(24.181(3) epoch 194472) v5 -296> 2014-08-13 17:22:51.981815 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:51.981770, event: dispatched, request: pg_notify(24.181(3) epoch 194472) v5 -295> 2014-08-13 17:22:51.981849 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:51.981849, event: waiting_for_osdmap, request: pg_notify(24.181(3) epoch 194472) v5 -294> 2014-08-13 17:22:51.981859 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:51.981859, event: started, request: pg_notify(24.181(3) epoch 194472) v5 -293> 2014-08-13 17:22:51.981893 7f74d8735700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:22:51.981893, event: done, request: pg_notify(24.181(3) epoch 194472) v5 -292> 2014-08-13 17:22:51.981911 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 17 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1010307001 0 0) 0x33f2c00 con 0x4833440 -291> 2014-08-13 17:22:51.981935 7f74d8735700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:51.495723, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -290> 2014-08-13 17:22:51.981945 7f74d8735700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:51.495726, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -289> 2014-08-13 17:22:51.981953 7f74d8735700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:51.495858, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -288> 2014-08-13 17:22:51.981962 7f74d8735700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:51.981932, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -287> 2014-08-13 17:22:51.981971 7f74d8735700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:51.981971, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -286> 2014-08-13 17:22:51.981990 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 14 ==== pg_notify(22.1d2(22) epoch 194472) v5 ==== 1952+0+0 (2654433664 0 0) 0x5c408c0 con 0x486e040 -285> 2014-08-13 17:22:51.982102 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:50.853286, event: header_read, request: pg_notify(22.1d2(22) epoch 194472) v5 -284> 2014-08-13 17:22:51.982111 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:50.853287, event: throttled, request: pg_notify(22.1d2(22) epoch 194472) v5 -283> 2014-08-13 17:22:51.982121 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:50.853395, event: all_read, request: pg_notify(22.1d2(22) epoch 194472) v5 -282> 2014-08-13 17:22:51.982130 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:51.982098, event: dispatched, request: pg_notify(22.1d2(22) epoch 194472) v5 -281> 2014-08-13 17:22:51.982140 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:51.982140, event: waiting_for_osdmap, request: pg_notify(22.1d2(22) epoch 194472) v5 -280> 2014-08-13 17:22:51.982152 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:51.982152, event: started, request: pg_notify(22.1d2(22) epoch 194472) v5 -279> 2014-08-13 17:22:51.982186 7f74d8735700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:22:51.982185, event: done, request: pg_notify(22.1d2(22) epoch 194472) v5 -278> 2014-08-13 17:22:51.982209 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.20 192.168.1.103:6815/38269 12 ==== pg_info(1 pgs e194472:24.3d) v4 ==== 748+0+0 (555486435 0 0) 0x57d5b00 con 0x4758840 -277> 2014-08-13 17:22:51.982228 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.971979, event: header_read, request: pg_info(1 pgs e194472:24.3d) v4 -276> 2014-08-13 17:22:51.982238 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.971982, event: throttled, request: pg_info(1 pgs e194472:24.3d) v4 -275> 2014-08-13 17:22:51.982247 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.972072, event: all_read, request: pg_info(1 pgs e194472:24.3d) v4 -274> 2014-08-13 17:22:51.982256 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.982226, event: dispatched, request: pg_info(1 pgs e194472:24.3d) v4 -273> 2014-08-13 17:22:51.982265 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.982265, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3d) v4 -272> 2014-08-13 17:22:51.982276 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.982276, event: started, request: pg_info(1 pgs e194472:24.3d) v4 -271> 2014-08-13 17:22:51.981955 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=194471 pi=193678-194470/25 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating -270> 2014-08-13 17:22:51.982299 7f74d8735700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:22:51.982298, event: done, request: pg_info(1 pgs e194472:24.3d) v4 -269> 2014-08-13 17:22:51.982318 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 18 ==== pg_notify(22.3e4(1) epoch 194472) v5 ==== 777+0+0 (338700655 0 0) 0x53b8380 con 0x475ee00 -268> 2014-08-13 17:22:51.982338 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:50.907710, event: header_read, request: pg_notify(22.3e4(1) epoch 194472) v5 -267> 2014-08-13 17:22:51.982349 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:50.907715, event: throttled, request: pg_notify(22.3e4(1) epoch 194472) v5 -266> 2014-08-13 17:22:51.982358 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:50.907819, event: all_read, request: pg_notify(22.3e4(1) epoch 194472) v5 -265> 2014-08-13 17:22:51.982368 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:51.982336, event: dispatched, request: pg_notify(22.3e4(1) epoch 194472) v5 -264> 2014-08-13 17:22:51.982377 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:51.982377, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194472) v5 -263> 2014-08-13 17:22:51.982388 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:51.982388, event: started, request: pg_notify(22.3e4(1) epoch 194472) v5 -262> 2014-08-13 17:22:51.982411 7f74d8735700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:22:51.982410, event: done, request: pg_notify(22.3e4(1) epoch 194472) v5 -261> 2014-08-13 17:22:51.982427 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 19 ==== pg_notify(22.481(32) epoch 194472) v5 ==== 2518+0+0 (157827660 0 0) 0x5c47e00 con 0x475ee00 -260> 2014-08-13 17:22:51.982445 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:50.908858, event: header_read, request: pg_notify(22.481(32) epoch 194472) v5 -259> 2014-08-13 17:22:51.982456 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:50.908862, event: throttled, request: pg_notify(22.481(32) epoch 194472) v5 -258> 2014-08-13 17:22:51.982464 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:50.909011, event: all_read, request: pg_notify(22.481(32) epoch 194472) v5 -257> 2014-08-13 17:22:51.982473 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:51.982443, event: dispatched, request: pg_notify(22.481(32) epoch 194472) v5 -256> 2014-08-13 17:22:51.982483 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:51.982483, event: waiting_for_osdmap, request: pg_notify(22.481(32) epoch 194472) v5 -255> 2014-08-13 17:22:51.982494 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:51.982493, event: started, request: pg_notify(22.481(32) epoch 194472) v5 -254> 2014-08-13 17:22:51.982516 7f74d8735700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:22:51.982516, event: done, request: pg_notify(22.481(32) epoch 194472) v5 -253> 2014-08-13 17:22:51.982539 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.303 192.168.1.109:6802/30768 20 ==== pg_info(1 pgs e194472:24.3d) v4 ==== 748+0+0 (668410751 0 0) 0x57d2d80 con 0x475ee00 -252> 2014-08-13 17:22:51.982557 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.972420, event: header_read, request: pg_info(1 pgs e194472:24.3d) v4 -251> 2014-08-13 17:22:51.982567 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.972421, event: throttled, request: pg_info(1 pgs e194472:24.3d) v4 -250> 2014-08-13 17:22:51.982576 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.972486, event: all_read, request: pg_info(1 pgs e194472:24.3d) v4 -249> 2014-08-13 17:22:51.982584 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.982555, event: dispatched, request: pg_info(1 pgs e194472:24.3d) v4 -248> 2014-08-13 17:22:51.982594 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.982594, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3d) v4 -247> 2014-08-13 17:22:51.982604 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.982604, event: started, request: pg_info(1 pgs e194472:24.3d) v4 -246> 2014-08-13 17:22:51.982626 7f74d8735700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:22:51.982626, event: done, request: pg_info(1 pgs e194472:24.3d) v4 -245> 2014-08-13 17:22:51.999930 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.348 192.168.1.110:6800/41204 -- pg_log(24.1bb epoch 194472 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194472) v4 -- ?+0 0x5b1da00 -244> 2014-08-13 17:22:51.999985 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 2.124936 4 0.000223 -243> 2014-08-13 17:22:52.000033 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 2.153268 0 0.000000 -242> 2014-08-13 17:22:52.000065 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194449 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active -241> 2014-08-13 17:22:52.000286 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=194471 pi=193644-194470/23 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating -240> 2014-08-13 17:22:52.001535 7f74d192a700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 -239> 2014-08-13 17:22:52.002063 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 5 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4805e80 con 0x4d511e0 -238> 2014-08-13 17:22:52.002111 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -237> 2014-08-13 17:22:52.002130 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.378 192.168.1.102:6840/19195 6 ==== pg_info(1 pgs e194472:24.9b) v4 ==== 724+0+0 (1439310649 0 0) 0x52640c0 con 0x4d511e0 -236> 2014-08-13 17:22:52.002150 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002010, event: header_read, request: pg_info(1 pgs e194472:24.9b) v4 -235> 2014-08-13 17:22:52.002165 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002011, event: throttled, request: pg_info(1 pgs e194472:24.9b) v4 -234> 2014-08-13 17:22:52.002190 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002074, event: all_read, request: pg_info(1 pgs e194472:24.9b) v4 -233> 2014-08-13 17:22:52.002201 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002147, event: dispatched, request: pg_info(1 pgs e194472:24.9b) v4 -232> 2014-08-13 17:22:52.002212 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002212, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.9b) v4 -231> 2014-08-13 17:22:52.002225 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002225, event: started, request: pg_info(1 pgs e194472:24.9b) v4 -230> 2014-08-13 17:22:52.002258 7f74d8735700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:22:52.002258, event: done, request: pg_info(1 pgs e194472:24.9b) v4 -229> 2014-08-13 17:22:52.002886 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a81b00 con 0x486a940 -228> 2014-08-13 17:22:52.002919 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -227> 2014-08-13 17:22:52.002978 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.367 192.168.1.114:6840/55513 10 ==== pg_info(1 pgs e194472:24.9b) v4 ==== 724+0+0 (1439310649 0 0) 0x52639c0 con 0x486a940 -226> 2014-08-13 17:22:52.003000 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.002890, event: header_read, request: pg_info(1 pgs e194472:24.9b) v4 -225> 2014-08-13 17:22:52.003012 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.002891, event: throttled, request: pg_info(1 pgs e194472:24.9b) v4 -224> 2014-08-13 17:22:52.003021 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.002952, event: all_read, request: pg_info(1 pgs e194472:24.9b) v4 -223> 2014-08-13 17:22:52.003030 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.002997, event: dispatched, request: pg_info(1 pgs e194472:24.9b) v4 -222> 2014-08-13 17:22:52.003040 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.003040, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.9b) v4 -221> 2014-08-13 17:22:52.003051 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.003051, event: started, request: pg_info(1 pgs e194472:24.9b) v4 -220> 2014-08-13 17:22:52.003076 7f74d8735700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:22:52.003076, event: done, request: pg_info(1 pgs e194472:24.9b) v4 -219> 2014-08-13 17:22:52.003106 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.55 192.168.1.112:6834/6966 -- pg_log(24.136 epoch 194472 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194472) v4 -- ?+0 0x5c71e00 -218> 2014-08-13 17:22:52.003174 7f74d192a700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 -217> 2014-08-13 17:22:52.003509 7f74d192a700 1 -- 192.168.1.115:6813/30165 --> osd.61 192.168.1.111:6840/16566 -- pg_log(24.136 epoch 194472 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194472) v4 -- ?+0 0x5c77200 -216> 2014-08-13 17:22:52.004110 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 2.146856 9 0.000457 -215> 2014-08-13 17:22:52.004156 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive -214> 2014-08-13 17:22:52.003902 7f74d192a700 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=194471 pi=193747-194470/23 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating -213> 2014-08-13 17:22:52.004192 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/192967) [6,374,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering -212> 2014-08-13 17:22:52.004759 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4c71d40 con 0x45f3b20 -211> 2014-08-13 17:22:52.004806 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -210> 2014-08-13 17:22:52.004824 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.28 192.168.1.103:6805/37391 10 ==== pg_info(1 pgs e194472:24.b6) v4 ==== 724+0+0 (1823871710 0 0) 0x5264600 con 0x45f3b20 -209> 2014-08-13 17:22:52.004844 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004708, event: header_read, request: pg_info(1 pgs e194472:24.b6) v4 -208> 2014-08-13 17:22:52.004859 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004709, event: throttled, request: pg_info(1 pgs e194472:24.b6) v4 -207> 2014-08-13 17:22:52.004869 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004771, event: all_read, request: pg_info(1 pgs e194472:24.b6) v4 -206> 2014-08-13 17:22:52.004878 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004841, event: dispatched, request: pg_info(1 pgs e194472:24.b6) v4 -205> 2014-08-13 17:22:52.004889 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004888, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.b6) v4 -204> 2014-08-13 17:22:52.004901 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004901, event: started, request: pg_info(1 pgs e194472:24.b6) v4 -203> 2014-08-13 17:22:52.004933 7f74d8735700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:22:52.004932, event: done, request: pg_info(1 pgs e194472:24.b6) v4 -202> 2014-08-13 17:22:52.007294 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 2.164052 8 0.000361 -201> 2014-08-13 17:22:52.007326 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive -200> 2014-08-13 17:22:52.007346 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194448 n=60 ec=192688 les/c 194466/194457 194470/194471/193250) [304,323,4] r=2 lpr=194471 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering -199> 2014-08-13 17:22:52.007843 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.6 192.168.1.107:6809/53141 -- pg_info(1 pgs e194472:24.cd) v4 -- ?+0 0x57d71c0 -198> 2014-08-13 17:22:52.008956 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(24) epoch 194472) v5 -- ?+0 0x53b9180 con 0x4835ac0 -197> 2014-08-13 17:22:52.008982 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(24) epoch 194472) v5 -- ?+0 0x57d3f00 con 0x47a32e0 -196> 2014-08-13 17:22:52.009004 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(24) epoch 194472) v5 -- ?+0 0x57d4d00 con 0x48377a0 -195> 2014-08-13 17:22:52.008995 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.6 192.168.1.107:6809/53141 9 ==== pg_info(1 pgs e194472:24.cd) v4 ==== 748+0+0 (268415919 0 0) 0x57d71c0 con 0x4835ac0 -194> 2014-08-13 17:22:52.009025 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(40) epoch 194472) v5 -- ?+0 0x57d78c0 con 0x486b9c0 -193> 2014-08-13 17:22:52.009028 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.008819, event: header_read, request: pg_info(1 pgs e194472:24.cd) v4 -192> 2014-08-13 17:22:52.009039 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.008821, event: throttled, request: pg_info(1 pgs e194472:24.cd) v4 -191> 2014-08-13 17:22:52.009046 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.008913, event: all_read, request: pg_info(1 pgs e194472:24.cd) v4 -190> 2014-08-13 17:22:52.009051 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.009023, event: dispatched, request: pg_info(1 pgs e194472:24.cd) v4 -189> 2014-08-13 17:22:52.009058 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.009058, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.cd) v4 -188> 2014-08-13 17:22:52.009056 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.110:6806/42024 -- pg_info(1 pgs e194472:24.1bb(23)) v4 -- ?+0 0x57d5780 con 0x48dcd00 -187> 2014-08-13 17:22:52.009068 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.009067, event: started, request: pg_info(1 pgs e194472:24.cd) v4 -186> 2014-08-13 17:22:52.009264 7f74d8735700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:22:52.009264, event: done, request: pg_info(1 pgs e194472:24.cd) v4 -185> 2014-08-13 17:22:52.009085 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- pg_info(1 pgs e194472:24.1bb(23)) v4 -- ?+0 0x57d4440 con 0x4833440 -184> 2014-08-13 17:22:52.009370 7f74d0f29700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:52.009369, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -183> 2014-08-13 17:22:52.009404 7f74d0f29700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:52.009404, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -182> 2014-08-13 17:22:52.009424 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x5acb480 con 0x4833440 -181> 2014-08-13 17:22:52.009462 7f74d0f29700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:22:52.009462, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -180> 2014-08-13 17:22:52.009496 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194472:24.33a) v4 -- ?+0 0x49e7380 -179> 2014-08-13 17:22:52.009598 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 2.148485 8 0.000473 -178> 2014-08-13 17:22:52.009640 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive -177> 2014-08-13 17:22:52.009686 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194448 n=43 ec=192688 les/c 194466/194457 194470/194471/193262) [343,45,4] r=2 lpr=194471 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering -176> 2014-08-13 17:22:52.010081 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 9 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a80d80 con 0x48dfe80 -175> 2014-08-13 17:22:52.010114 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -174> 2014-08-13 17:22:52.010181 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.286 192.168.1.113:6820/19173 10 ==== pg_info(1 pgs e194472:24.30b) v4 ==== 724+0+0 (3195371910 0 0) 0x4ac24c0 con 0x48dfe80 -173> 2014-08-13 17:22:52.010211 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010078, event: header_read, request: pg_info(1 pgs e194472:24.30b) v4 -172> 2014-08-13 17:22:52.010224 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010079, event: throttled, request: pg_info(1 pgs e194472:24.30b) v4 -171> 2014-08-13 17:22:52.010233 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010143, event: all_read, request: pg_info(1 pgs e194472:24.30b) v4 -170> 2014-08-13 17:22:52.010242 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010207, event: dispatched, request: pg_info(1 pgs e194472:24.30b) v4 -169> 2014-08-13 17:22:52.010252 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010252, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.30b) v4 -168> 2014-08-13 17:22:52.010265 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010265, event: started, request: pg_info(1 pgs e194472:24.30b) v4 -167> 2014-08-13 17:22:52.010294 7f74d8735700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:22:52.010293, event: done, request: pg_info(1 pgs e194472:24.30b) v4 -166> 2014-08-13 17:22:52.010326 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 5 ==== osd_map(194472..194472 src has 185139..194472) v3 ==== 326+0+0 (3889789184 0 0) 0x4a85340 con 0x4904a40 -165> 2014-08-13 17:22:52.010349 7f74d8735700 3 osd.4 194472 handle_osd_map epochs [194472,194472], i have 194472, src has [185139,194472] -164> 2014-08-13 17:22:52.010372 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.25 192.168.1.103:6800/36920 6 ==== pg_info(1 pgs e194472:24.3d) v4 ==== 724+0+0 (2631848690 0 0) 0x49e1f80 con 0x4904a40 -163> 2014-08-13 17:22:52.010390 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010242, event: header_read, request: pg_info(1 pgs e194472:24.3d) v4 -162> 2014-08-13 17:22:52.010401 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010243, event: throttled, request: pg_info(1 pgs e194472:24.3d) v4 -161> 2014-08-13 17:22:52.010410 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010307, event: all_read, request: pg_info(1 pgs e194472:24.3d) v4 -160> 2014-08-13 17:22:52.010420 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010388, event: dispatched, request: pg_info(1 pgs e194472:24.3d) v4 -159> 2014-08-13 17:22:52.010430 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010429, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3d) v4 -158> 2014-08-13 17:22:52.010441 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010441, event: started, request: pg_info(1 pgs e194472:24.3d) v4 -157> 2014-08-13 17:22:52.010467 7f74d8735700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:22:52.010467, event: done, request: pg_info(1 pgs e194472:24.3d) v4 -156> 2014-08-13 17:22:52.010484 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.304 192.168.1.101:6808/22782 12 ==== pg_info(1 pgs e194472:24.33a) v4 ==== 748+0+0 (3412018228 0 0) 0x49e7380 con 0x47f0c60 -155> 2014-08-13 17:22:52.010502 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010334, event: header_read, request: pg_info(1 pgs e194472:24.33a) v4 -154> 2014-08-13 17:22:52.010512 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010335, event: throttled, request: pg_info(1 pgs e194472:24.33a) v4 -153> 2014-08-13 17:22:52.010521 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010415, event: all_read, request: pg_info(1 pgs e194472:24.33a) v4 -152> 2014-08-13 17:22:52.010530 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010500, event: dispatched, request: pg_info(1 pgs e194472:24.33a) v4 -151> 2014-08-13 17:22:52.010540 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010539, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.33a) v4 -150> 2014-08-13 17:22:52.010551 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010550, event: started, request: pg_info(1 pgs e194472:24.33a) v4 -149> 2014-08-13 17:22:52.010574 7f74d8735700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:22:52.010574, event: done, request: pg_info(1 pgs e194472:24.33a) v4 -148> 2014-08-13 17:22:52.010884 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 18 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (2909395996 0 0) 0x61b4d00 con 0x4833440 -147> 2014-08-13 17:22:52.010913 7f74d8735700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.010768, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -146> 2014-08-13 17:22:52.010928 7f74d8735700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.010770, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -145> 2014-08-13 17:22:52.010941 7f74d8735700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.010861, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -144> 2014-08-13 17:22:52.010953 7f74d8735700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.010911, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -143> 2014-08-13 17:22:52.010966 7f74d8735700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.010966, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -142> 2014-08-13 17:22:52.011159 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 2.152704 8 0.000429 -141> 2014-08-13 17:22:52.011201 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive -140> 2014-08-13 17:22:52.011226 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194449 n=52 ec=192688 les/c 194466/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=194471 pi=192688-194470/40 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering -139> 2014-08-13 17:22:52.011569 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194472:24.206) v4 -- ?+0 0x49e2f40 -138> 2014-08-13 17:22:52.011637 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.332 192.168.1.110:6806/42024 9 ==== pg_info(1 pgs e194472:24.1bb) v4 ==== 748+0+0 (430425223 0 0) 0x57d5780 con 0x48dcd00 -137> 2014-08-13 17:22:52.011663 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011426, event: header_read, request: pg_info(1 pgs e194472:24.1bb) v4 -136> 2014-08-13 17:22:52.011674 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011427, event: throttled, request: pg_info(1 pgs e194472:24.1bb) v4 -135> 2014-08-13 17:22:52.011683 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011615, event: all_read, request: pg_info(1 pgs e194472:24.1bb) v4 -134> 2014-08-13 17:22:52.011692 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011660, event: dispatched, request: pg_info(1 pgs e194472:24.1bb) v4 -133> 2014-08-13 17:22:52.011702 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011702, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1bb) v4 -132> 2014-08-13 17:22:52.011714 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011713, event: started, request: pg_info(1 pgs e194472:24.1bb) v4 -131> 2014-08-13 17:22:52.011739 7f74d8735700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:22:52.011739, event: done, request: pg_info(1 pgs e194472:24.1bb) v4 -130> 2014-08-13 17:22:52.012435 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.343 192.168.1.104:6802/61960 14 ==== pg_info(1 pgs e194472:24.206) v4 ==== 748+0+0 (1922881076 0 0) 0x49e2f40 con 0x48377a0 -129> 2014-08-13 17:22:52.012456 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012335, event: header_read, request: pg_info(1 pgs e194472:24.206) v4 -128> 2014-08-13 17:22:52.012467 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012337, event: throttled, request: pg_info(1 pgs e194472:24.206) v4 -127> 2014-08-13 17:22:52.012476 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012412, event: all_read, request: pg_info(1 pgs e194472:24.206) v4 -126> 2014-08-13 17:22:52.012485 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012454, event: dispatched, request: pg_info(1 pgs e194472:24.206) v4 -125> 2014-08-13 17:22:52.012495 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012495, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.206) v4 -124> 2014-08-13 17:22:52.012506 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012505, event: started, request: pg_info(1 pgs e194472:24.206) v4 -123> 2014-08-13 17:22:52.012532 7f74d8735700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:22:52.012532, event: done, request: pg_info(1 pgs e194472:24.206) v4 -122> 2014-08-13 17:22:52.012688 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 2.177556 9 0.000490 -121> 2014-08-13 17:22:52.012720 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive -120> 2014-08-13 17:22:52.012744 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194449 n=69 ec=192688 les/c 194466/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=194471 pi=193125-194470/26 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering -119> 2014-08-13 17:22:52.013049 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194472:24.2ec) v4 -- ?+0 0x466aa00 -118> 2014-08-13 17:22:52.013399 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.337 192.168.1.104:6804/62197 19 ==== pg_info(1 pgs e194472:24.1bb) v4 ==== 748+0+0 (1147753898 0 0) 0x57d4440 con 0x4833440 -117> 2014-08-13 17:22:52.013421 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013282, event: header_read, request: pg_info(1 pgs e194472:24.1bb) v4 -116> 2014-08-13 17:22:52.013432 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013284, event: throttled, request: pg_info(1 pgs e194472:24.1bb) v4 -115> 2014-08-13 17:22:52.013441 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013377, event: all_read, request: pg_info(1 pgs e194472:24.1bb) v4 -114> 2014-08-13 17:22:52.013449 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013418, event: dispatched, request: pg_info(1 pgs e194472:24.1bb) v4 -113> 2014-08-13 17:22:52.013459 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013459, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1bb) v4 -112> 2014-08-13 17:22:52.013470 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013470, event: started, request: pg_info(1 pgs e194472:24.1bb) v4 -111> 2014-08-13 17:22:52.013493 7f74d8735700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:22:52.013493, event: done, request: pg_info(1 pgs e194472:24.1bb) v4 -110> 2014-08-13 17:22:52.014047 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.374 192.168.1.102:6828/16129 11 ==== pg_info(1 pgs e194472:24.2ec) v4 ==== 748+0+0 (3096730527 0 0) 0x466aa00 con 0x486b9c0 -109> 2014-08-13 17:22:52.014068 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.013956, event: header_read, request: pg_info(1 pgs e194472:24.2ec) v4 -108> 2014-08-13 17:22:52.014079 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.013958, event: throttled, request: pg_info(1 pgs e194472:24.2ec) v4 -107> 2014-08-13 17:22:52.014088 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.014025, event: all_read, request: pg_info(1 pgs e194472:24.2ec) v4 -106> 2014-08-13 17:22:52.014097 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.014066, event: dispatched, request: pg_info(1 pgs e194472:24.2ec) v4 -105> 2014-08-13 17:22:52.014107 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.014107, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.2ec) v4 -104> 2014-08-13 17:22:52.014119 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.014118, event: started, request: pg_info(1 pgs e194472:24.2ec) v4 -103> 2014-08-13 17:22:52.014162 7f74d8735700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:22:52.014161, event: done, request: pg_info(1 pgs e194472:24.2ec) v4 -102> 2014-08-13 17:22:52.014303 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 2.180895 8 0.000394 -101> 2014-08-13 17:22:52.014341 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive -100> 2014-08-13 17:22:52.014372 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194449 n=66 ec=192688 les/c 194466/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=194471 pi=193227-194470/29 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering -99> 2014-08-13 17:22:52.014506 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 15 ==== pg_info(1 pgs e194472:24.b6) v4 ==== 724+0+0 (1823871710 0 0) 0x466f000 con 0x486e040 -98> 2014-08-13 17:22:52.014530 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014405, event: header_read, request: pg_info(1 pgs e194472:24.b6) v4 -97> 2014-08-13 17:22:52.014541 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014406, event: throttled, request: pg_info(1 pgs e194472:24.b6) v4 -96> 2014-08-13 17:22:52.014550 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014482, event: all_read, request: pg_info(1 pgs e194472:24.b6) v4 -95> 2014-08-13 17:22:52.014559 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014527, event: dispatched, request: pg_info(1 pgs e194472:24.b6) v4 -94> 2014-08-13 17:22:52.014569 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014569, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.b6) v4 -93> 2014-08-13 17:22:52.014580 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014580, event: started, request: pg_info(1 pgs e194472:24.b6) v4 -92> 2014-08-13 17:22:52.014621 7f74d8735700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:22:52.014621, event: done, request: pg_info(1 pgs e194472:24.b6) v4 -91> 2014-08-13 17:22:52.014883 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.346 192.168.1.104:6820/2308 -- pg_info(1 pgs e194472:24.e2) v4 -- ?+0 0x466f8c0 -90> 2014-08-13 17:22:52.015736 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.346 192.168.1.104:6820/2308 9 ==== pg_info(1 pgs e194472:24.e2) v4 ==== 748+0+0 (3173859465 0 0) 0x49e0000 con 0x486b180 -89> 2014-08-13 17:22:52.015767 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015638, event: header_read, request: pg_info(1 pgs e194472:24.e2) v4 -88> 2014-08-13 17:22:52.015778 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015640, event: throttled, request: pg_info(1 pgs e194472:24.e2) v4 -87> 2014-08-13 17:22:52.015788 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015714, event: all_read, request: pg_info(1 pgs e194472:24.e2) v4 -86> 2014-08-13 17:22:52.015797 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015756, event: dispatched, request: pg_info(1 pgs e194472:24.e2) v4 -85> 2014-08-13 17:22:52.015806 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015806, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.e2) v4 -84> 2014-08-13 17:22:52.015817 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015817, event: started, request: pg_info(1 pgs e194472:24.e2) v4 -83> 2014-08-13 17:22:52.015879 7f74d0f29700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.015879, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -82> 2014-08-13 17:22:52.015911 7f74d8735700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:22:52.015883, event: done, request: pg_info(1 pgs e194472:24.e2) v4 -81> 2014-08-13 17:22:52.015938 7f74d0f29700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.015911, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -80> 2014-08-13 17:22:52.015965 7f74d0f29700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:22:52.015965, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -79> 2014-08-13 17:22:52.016148 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194472:24.3f7) v4 -- ?+0 0x466b9c0 -78> 2014-08-13 17:22:52.016334 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 2.180267 8 0.000413 -77> 2014-08-13 17:22:52.016373 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive -76> 2014-08-13 17:22:52.016398 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194449 n=49 ec=192688 les/c 194466/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=194471 pi=193125-194470/27 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering -75> 2014-08-13 17:22:52.017031 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 16 ==== pg_info(1 pgs e194472:24.3f7) v4 ==== 748+0+0 (528176652 0 0) 0x466b9c0 con 0x486e040 -74> 2014-08-13 17:22:52.017060 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.016932, event: header_read, request: pg_info(1 pgs e194472:24.3f7) v4 -73> 2014-08-13 17:22:52.017068 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.016933, event: throttled, request: pg_info(1 pgs e194472:24.3f7) v4 -72> 2014-08-13 17:22:52.017074 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.017006, event: all_read, request: pg_info(1 pgs e194472:24.3f7) v4 -71> 2014-08-13 17:22:52.017081 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.017056, event: dispatched, request: pg_info(1 pgs e194472:24.3f7) v4 -70> 2014-08-13 17:22:52.017091 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.017091, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3f7) v4 -69> 2014-08-13 17:22:52.017103 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.017102, event: started, request: pg_info(1 pgs e194472:24.3f7) v4 -68> 2014-08-13 17:22:52.017129 7f74d8735700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:22:52.017129, event: done, request: pg_info(1 pgs e194472:24.3f7) v4 -67> 2014-08-13 17:22:52.017149 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.362 192.168.1.108:6842/8170 12 ==== pg_info(1 pgs e194472:24.3d) v4 ==== 724+0+0 (2631848690 0 0) 0x53b9c00 con 0x48db9c0 -66> 2014-08-13 17:22:52.017167 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.016972, event: header_read, request: pg_info(1 pgs e194472:24.3d) v4 -65> 2014-08-13 17:22:52.017177 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.016974, event: throttled, request: pg_info(1 pgs e194472:24.3d) v4 -64> 2014-08-13 17:22:52.017186 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.017049, event: all_read, request: pg_info(1 pgs e194472:24.3d) v4 -63> 2014-08-13 17:22:52.017203 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.017165, event: dispatched, request: pg_info(1 pgs e194472:24.3d) v4 -62> 2014-08-13 17:22:52.017213 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.017213, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.3d) v4 -61> 2014-08-13 17:22:52.017228 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.017228, event: started, request: pg_info(1 pgs e194472:24.3d) v4 -60> 2014-08-13 17:22:52.017252 7f74d8735700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:22:52.017252, event: done, request: pg_info(1 pgs e194472:24.3d) v4 -59> 2014-08-13 17:22:52.017754 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 2.180572 8 0.000398 -58> 2014-08-13 17:22:52.017785 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive -57> 2014-08-13 17:22:52.017807 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194448 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=194471 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering -56> 2014-08-13 17:22:52.018118 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.364 192.168.1.114:6826/52317 -- pg_info(1 pgs e194472:24.1de) v4 -- ?+0 0x53ba680 -55> 2014-08-13 17:22:52.019230 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.364 192.168.1.114:6826/52317 8 ==== pg_info(1 pgs e194472:24.1de) v4 ==== 748+0+0 (2492037758 0 0) 0x53ba680 con 0x47a4200 -54> 2014-08-13 17:22:52.019255 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019086, event: header_read, request: pg_info(1 pgs e194472:24.1de) v4 -53> 2014-08-13 17:22:52.019266 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019087, event: throttled, request: pg_info(1 pgs e194472:24.1de) v4 -52> 2014-08-13 17:22:52.019275 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019172, event: all_read, request: pg_info(1 pgs e194472:24.1de) v4 -51> 2014-08-13 17:22:52.019284 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019251, event: dispatched, request: pg_info(1 pgs e194472:24.1de) v4 -50> 2014-08-13 17:22:52.019294 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019294, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.1de) v4 -49> 2014-08-13 17:22:52.019305 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019305, event: started, request: pg_info(1 pgs e194472:24.1de) v4 -48> 2014-08-13 17:22:52.019365 7f74d8735700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:22:52.019365, event: done, request: pg_info(1 pgs e194472:24.1de) v4 -47> 2014-08-13 17:22:52.019475 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.26 192.168.1.115:6802/58522 -- pg_info(1 pgs e194472:24.34) v4 -- ?+0 0x5262f40 -46> 2014-08-13 17:22:52.019513 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194472:24.34) v4 -- ?+0 0x5264440 -45> 2014-08-13 17:22:52.019537 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194472:24.34) v4 -- ?+0 0x49e2300 -44> 2014-08-13 17:22:52.019751 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.330 192.168.1.106:6802/4876 17 ==== MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 ==== 26+0+0 (1502133402 0 0) 0x466f700 con 0x486e040 -43> 2014-08-13 17:22:52.019738 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 2.160761 8 0.000309 -42> 2014-08-13 17:22:52.019775 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019626, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -41> 2014-08-13 17:22:52.019767 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194472:24.2a1) v4 -- ?+0 0x53bdcc0 -40> 2014-08-13 17:22:52.019783 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019629, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -39> 2014-08-13 17:22:52.019789 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019715, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -38> 2014-08-13 17:22:52.019798 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019773, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -37> 2014-08-13 17:22:52.019792 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -36> 2014-08-13 17:22:52.019808 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019808, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -35> 2014-08-13 17:22:52.019818 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194448 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=194471 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -34> 2014-08-13 17:22:52.019829 7f74d8735700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:22:52.019829, event: done, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194472 v2 -33> 2014-08-13 17:22:52.020704 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.17 192.168.1.103:6811/37658 13 ==== pg_info(1 pgs e194472:24.2a1) v4 ==== 748+0+0 (2725185502 0 0) 0x53bdcc0 con 0x47a53e0 -32> 2014-08-13 17:22:52.020726 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020602, event: header_read, request: pg_info(1 pgs e194472:24.2a1) v4 -31> 2014-08-13 17:22:52.020737 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020603, event: throttled, request: pg_info(1 pgs e194472:24.2a1) v4 -30> 2014-08-13 17:22:52.020746 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020681, event: all_read, request: pg_info(1 pgs e194472:24.2a1) v4 -29> 2014-08-13 17:22:52.020754 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020723, event: dispatched, request: pg_info(1 pgs e194472:24.2a1) v4 -28> 2014-08-13 17:22:52.020764 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020764, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.2a1) v4 -27> 2014-08-13 17:22:52.020775 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020775, event: started, request: pg_info(1 pgs e194472:24.2a1) v4 -26> 2014-08-13 17:22:52.020801 7f74d8735700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:22:52.020801, event: done, request: pg_info(1 pgs e194472:24.2a1) v4 -25> 2014-08-13 17:22:52.021392 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.291 192.168.1.101:6820/25633 -- pg_info(1 pgs e194472:24.9b) v4 -- ?+0 0x5c40700 -24> 2014-08-13 17:22:52.021431 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.350 192.168.1.114:6832/53185 -- pg_info(1 pgs e194472:24.9b) v4 -- ?+0 0x5c40540 -23> 2014-08-13 17:22:52.021457 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194472:24.9b) v4 -- ?+0 0x4ac3f00 -22> 2014-08-13 17:22:52.021481 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.378 192.168.1.102:6840/19195 -- pg_info(1 pgs e194472:24.9b) v4 -- ?+0 0x4ac2a00 -21> 2014-08-13 17:22:52.021750 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.10 192.168.1.115:6815/60573 -- pg_info(1 pgs e194472:24.30b) v4 -- ?+0 0x4ac2d80 -20> 2014-08-13 17:22:52.021792 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194472:24.30b) v4 -- ?+0 0x4ac1880 -19> 2014-08-13 17:22:52.021824 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.375 192.168.1.114:6830/52879 -- pg_info(1 pgs e194472:24.30b) v4 -- ?+0 0x4ac0540 -18> 2014-08-13 17:22:52.021856 7f74dd7e9700 1 -- 192.168.1.115:6813/30165 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194472:24.369) v4 -- ?+0 0x53bc980 -17> 2014-08-13 17:22:52.022388 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194472:24.3d) v4 -- ?+0 0x4ac4440 -16> 2014-08-13 17:22:52.022422 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.25 192.168.1.103:6800/36920 -- pg_info(1 pgs e194472:24.3d) v4 -- ?+0 0x4ac71c0 -15> 2014-08-13 17:22:52.022452 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.303 192.168.1.109:6802/30768 -- pg_info(1 pgs e194472:24.3d) v4 -- ?+0 0x4ac0fc0 -14> 2014-08-13 17:22:52.022473 7f74d0f29700 1 -- 192.168.1.115:6813/30165 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194472:24.3d) v4 -- ?+0 0x4ac08c0 -13> 2014-08-13 17:22:52.022702 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 -12> 2014-08-13 17:22:52.022736 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 -11> 2014-08-13 17:22:52.022759 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 -10> 2014-08-13 17:22:52.022779 7f74d0f29700 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=194471 pi=193125-194470/48 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 -9> 2014-08-13 17:22:52.022841 7f74d8735700 1 -- 192.168.1.115:6813/30165 <== osd.45 192.168.1.112:6828/6040 14 ==== pg_info(1 pgs e194472:24.369) v4 ==== 748+0+0 (3562404593 0 0) 0x53bc980 con 0x47a32e0 -8> 2014-08-13 17:22:52.022872 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022721, event: header_read, request: pg_info(1 pgs e194472:24.369) v4 -7> 2014-08-13 17:22:52.022886 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022723, event: throttled, request: pg_info(1 pgs e194472:24.369) v4 -6> 2014-08-13 17:22:52.022895 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022815, event: all_read, request: pg_info(1 pgs e194472:24.369) v4 -5> 2014-08-13 17:22:52.022904 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022868, event: dispatched, request: pg_info(1 pgs e194472:24.369) v4 -4> 2014-08-13 17:22:52.022914 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022914, event: waiting_for_osdmap, request: pg_info(1 pgs e194472:24.369) v4 -3> 2014-08-13 17:22:52.022926 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022926, event: started, request: pg_info(1 pgs e194472:24.369) v4 -2> 2014-08-13 17:22:52.022952 7f74d8735700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:22:52.022951, event: done, request: pg_info(1 pgs e194472:24.369) v4 -1> 2014-08-13 17:22:52.023203 7f74d0f29700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 0> 2014-08-13 17:22:52.028252 7f74d0f29700 -1 *** Caught signal (Floating point exception) ** in thread 7f74d0f29700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7f74ec11d710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7f74ec1159d1] 16: (clone()+0x6d) [0x7f74eb0a5b6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.4.log --- end dump of recent events --- 2014-08-13 17:23:19.383401 7fcd1d4027a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 31924 2014-08-13 17:23:19.390950 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:23:19.390981 7fcd1d4027a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-08-13 17:23:19.393101 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:23:19.393120 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:23:19.411128 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:23:19.411259 7fcd1d4027a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:23:19.427854 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-13 17:23:19.432463 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:23:19.633027 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:23:19.645438 7fcd1d4027a0 1 journal close /var/lib/ceph/osd/ceph-4/journal 2014-08-13 17:23:19.647765 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:23:19.649933 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:23:19.649949 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:23:19.690135 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:23:19.690233 7fcd1d4027a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:23:19.719527 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-13 17:23:19.725047 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:23:19.775089 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:23:19.787316 7fcd1d4027a0 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-13 17:23:19.789649 7fcd1d4027a0 0 osd.4 194472 crush map has features 2303210029056, adjusting msgr requires for clients 2014-08-13 17:23:19.789672 7fcd1d4027a0 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:23:19.789680 7fcd1d4027a0 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for osds 2014-08-13 17:23:19.789706 7fcd1d4027a0 0 osd.4 194472 load_pgs 2014-08-13 17:23:20.194952 7fcd1d4027a0 0 osd.4 194472 load_pgs opened 74 pgs 2014-08-13 17:23:20.202018 7fcd0aa4e700 0 osd.4 194472 ignoring osdmap until we have initialized 2014-08-13 17:23:20.203331 7fcd0aa4e700 0 osd.4 194472 ignoring osdmap until we have initialized 2014-08-13 17:23:20.214924 7fcd1d4027a0 0 osd.4 194472 done with init, starting boot process 2014-08-13 17:23:20.875699 7fcd0aa4e700 0 osd.4 194476 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:23:20.911934 7fccddc6c700 0 -- 192.168.1.115:6819/31924 >> 192.168.1.102:6840/19195 pipe(0x5308000 sd=344 :6819 s=0 pgs=0 cs=0 l=0 c=0x5300840).accept connect_seq 0 vs existing 0 state connecting 2014-08-13 17:23:21.638564 7fcd0aa4e700 0 osd.4 194477 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:23:22.636410 7fcd0aa4e700 0 osd.4 194478 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:23:22.705745 7fcd01e40700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 17:23:22.710795 7fcd01e40700 -1 *** Caught signal (Floating point exception) ** in thread 7fcd01e40700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fcd1c3af710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fcd1c3a79d1] 16: (clone()+0x6d) [0x7fcd1b337b6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -10000> 2014-08-13 17:23:19.473410 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298081 -9999> 2014-08-13 17:23:19.473784 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298081 -9998> 2014-08-13 17:23:19.473806 7fcd1d4027a0 2 journal read_entry 802189312 : seq 16298082 305 bytes -9997> 2014-08-13 17:23:19.473810 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298082 -9996> 2014-08-13 17:23:19.474119 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298082 -9995> 2014-08-13 17:23:19.474141 7fcd1d4027a0 2 journal read_entry 802193408 : seq 16298083 301 bytes -9994> 2014-08-13 17:23:19.474145 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298083 -9993> 2014-08-13 17:23:19.474599 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298083 -9992> 2014-08-13 17:23:19.474959 7fcd1d4027a0 2 journal read_entry 802197504 : seq 16298084 295 bytes -9991> 2014-08-13 17:23:19.474971 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298084 -9990> 2014-08-13 17:23:19.475509 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298084 -9989> 2014-08-13 17:23:19.475534 7fcd1d4027a0 2 journal read_entry 802201600 : seq 16298085 305 bytes -9988> 2014-08-13 17:23:19.475538 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298085 -9987> 2014-08-13 17:23:19.476062 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298085 -9986> 2014-08-13 17:23:19.476087 7fcd1d4027a0 2 journal read_entry 802205696 : seq 16298086 298 bytes -9985> 2014-08-13 17:23:19.476094 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298086 -9984> 2014-08-13 17:23:19.476465 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298086 -9983> 2014-08-13 17:23:19.476488 7fcd1d4027a0 2 journal read_entry 802209792 : seq 16298087 298 bytes -9982> 2014-08-13 17:23:19.476495 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298087 -9981> 2014-08-13 17:23:19.476834 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298087 -9980> 2014-08-13 17:23:19.476856 7fcd1d4027a0 2 journal read_entry 802213888 : seq 16298088 298 bytes -9979> 2014-08-13 17:23:19.476863 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298088 -9978> 2014-08-13 17:23:19.477181 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298088 -9977> 2014-08-13 17:23:19.477203 7fcd1d4027a0 2 journal read_entry 802217984 : seq 16298089 305 bytes -9976> 2014-08-13 17:23:19.477210 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298089 -9975> 2014-08-13 17:23:19.477558 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298089 -9974> 2014-08-13 17:23:19.477581 7fcd1d4027a0 2 journal read_entry 802222080 : seq 16298090 301 bytes -9973> 2014-08-13 17:23:19.477589 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298090 -9972> 2014-08-13 17:23:19.478098 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298090 -9971> 2014-08-13 17:23:19.478122 7fcd1d4027a0 2 journal read_entry 802226176 : seq 16298091 305 bytes -9970> 2014-08-13 17:23:19.478130 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298091 -9969> 2014-08-13 17:23:19.478517 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298091 -9968> 2014-08-13 17:23:19.478540 7fcd1d4027a0 2 journal read_entry 802230272 : seq 16298092 301 bytes -9967> 2014-08-13 17:23:19.478544 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298092 -9966> 2014-08-13 17:23:19.478917 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298092 -9965> 2014-08-13 17:23:19.478939 7fcd1d4027a0 2 journal read_entry 802234368 : seq 16298093 305 bytes -9964> 2014-08-13 17:23:19.478943 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298093 -9963> 2014-08-13 17:23:19.479331 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298093 -9962> 2014-08-13 17:23:19.479354 7fcd1d4027a0 2 journal read_entry 802238464 : seq 16298094 305 bytes -9961> 2014-08-13 17:23:19.479358 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298094 -9960> 2014-08-13 17:23:19.479701 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298094 -9959> 2014-08-13 17:23:19.479723 7fcd1d4027a0 2 journal read_entry 802242560 : seq 16298095 301 bytes -9958> 2014-08-13 17:23:19.479728 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298095 -9957> 2014-08-13 17:23:19.480170 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298095 -9956> 2014-08-13 17:23:19.480192 7fcd1d4027a0 2 journal read_entry 802246656 : seq 16298096 305 bytes -9955> 2014-08-13 17:23:19.480197 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298096 -9954> 2014-08-13 17:23:19.480571 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298096 -9953> 2014-08-13 17:23:19.480594 7fcd1d4027a0 2 journal read_entry 802250752 : seq 16298097 305 bytes -9952> 2014-08-13 17:23:19.480598 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298097 -9951> 2014-08-13 17:23:19.480930 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298097 -9950> 2014-08-13 17:23:19.480953 7fcd1d4027a0 2 journal read_entry 802254848 : seq 16298098 302 bytes -9949> 2014-08-13 17:23:19.480958 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298098 -9948> 2014-08-13 17:23:19.481385 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298098 -9947> 2014-08-13 17:23:19.481407 7fcd1d4027a0 2 journal read_entry 802258944 : seq 16298099 305 bytes -9946> 2014-08-13 17:23:19.481411 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298099 -9945> 2014-08-13 17:23:19.481825 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298099 -9944> 2014-08-13 17:23:19.481847 7fcd1d4027a0 2 journal read_entry 802263040 : seq 16298100 305 bytes -9943> 2014-08-13 17:23:19.481851 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298100 -9942> 2014-08-13 17:23:19.482248 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298100 -9941> 2014-08-13 17:23:19.482270 7fcd1d4027a0 2 journal read_entry 802267136 : seq 16298101 301 bytes -9940> 2014-08-13 17:23:19.482274 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298101 -9939> 2014-08-13 17:23:19.482686 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298101 -9938> 2014-08-13 17:23:19.482708 7fcd1d4027a0 2 journal read_entry 802271232 : seq 16298102 305 bytes -9937> 2014-08-13 17:23:19.482712 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298102 -9936> 2014-08-13 17:23:19.483122 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298102 -9935> 2014-08-13 17:23:19.483144 7fcd1d4027a0 2 journal read_entry 802275328 : seq 16298103 305 bytes -9934> 2014-08-13 17:23:19.483161 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298103 -9933> 2014-08-13 17:23:19.483505 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298103 -9932> 2014-08-13 17:23:19.483528 7fcd1d4027a0 2 journal read_entry 802279424 : seq 16298104 305 bytes -9931> 2014-08-13 17:23:19.483532 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298104 -9930> 2014-08-13 17:23:19.483958 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298104 -9929> 2014-08-13 17:23:19.483983 7fcd1d4027a0 2 journal read_entry 802283520 : seq 16298105 305 bytes -9928> 2014-08-13 17:23:19.483987 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298105 -9927> 2014-08-13 17:23:19.484440 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298105 -9926> 2014-08-13 17:23:19.484465 7fcd1d4027a0 2 journal read_entry 802287616 : seq 16298106 305 bytes -9925> 2014-08-13 17:23:19.484469 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298106 -9924> 2014-08-13 17:23:19.484855 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298106 -9923> 2014-08-13 17:23:19.484877 7fcd1d4027a0 2 journal read_entry 802291712 : seq 16298107 301 bytes -9922> 2014-08-13 17:23:19.484881 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298107 -9921> 2014-08-13 17:23:19.485233 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298107 -9920> 2014-08-13 17:23:19.485255 7fcd1d4027a0 2 journal read_entry 802295808 : seq 16298108 301 bytes -9919> 2014-08-13 17:23:19.485259 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298108 -9918> 2014-08-13 17:23:19.485633 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298108 -9917> 2014-08-13 17:23:19.485655 7fcd1d4027a0 2 journal read_entry 802299904 : seq 16298109 305 bytes -9916> 2014-08-13 17:23:19.485659 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298109 -9915> 2014-08-13 17:23:19.485967 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298109 -9914> 2014-08-13 17:23:19.485988 7fcd1d4027a0 2 journal read_entry 802304000 : seq 16298110 305 bytes -9913> 2014-08-13 17:23:19.485997 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298110 -9912> 2014-08-13 17:23:19.486423 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298110 -9911> 2014-08-13 17:23:19.486445 7fcd1d4027a0 2 journal read_entry 802308096 : seq 16298111 305 bytes -9910> 2014-08-13 17:23:19.486449 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298111 -9909> 2014-08-13 17:23:19.486862 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298111 -9908> 2014-08-13 17:23:19.486884 7fcd1d4027a0 2 journal read_entry 802312192 : seq 16298112 305 bytes -9907> 2014-08-13 17:23:19.486888 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298112 -9906> 2014-08-13 17:23:19.487232 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298112 -9905> 2014-08-13 17:23:19.487254 7fcd1d4027a0 2 journal read_entry 802316288 : seq 16298113 301 bytes -9904> 2014-08-13 17:23:19.487258 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298113 -9903> 2014-08-13 17:23:19.487616 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298113 -9902> 2014-08-13 17:23:19.487637 7fcd1d4027a0 2 journal read_entry 802320384 : seq 16298114 301 bytes -9901> 2014-08-13 17:23:19.487642 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298114 -9900> 2014-08-13 17:23:19.488013 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298114 -9899> 2014-08-13 17:23:19.488035 7fcd1d4027a0 2 journal read_entry 802324480 : seq 16298115 301 bytes -9898> 2014-08-13 17:23:19.488042 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298115 -9897> 2014-08-13 17:23:19.488370 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298115 -9896> 2014-08-13 17:23:19.488727 7fcd1d4027a0 2 journal read_entry 802328576 : seq 16298116 305 bytes -9895> 2014-08-13 17:23:19.488739 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298116 -9894> 2014-08-13 17:23:19.489246 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298116 -9893> 2014-08-13 17:23:19.489270 7fcd1d4027a0 2 journal read_entry 802332672 : seq 16298117 305 bytes -9892> 2014-08-13 17:23:19.489277 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298117 -9891> 2014-08-13 17:23:19.489652 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298117 -9890> 2014-08-13 17:23:19.489675 7fcd1d4027a0 2 journal read_entry 802336768 : seq 16298118 301 bytes -9889> 2014-08-13 17:23:19.489682 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298118 -9888> 2014-08-13 17:23:19.490024 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298118 -9887> 2014-08-13 17:23:19.490047 7fcd1d4027a0 2 journal read_entry 802340864 : seq 16298119 301 bytes -9886> 2014-08-13 17:23:19.490054 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298119 -9885> 2014-08-13 17:23:19.490479 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298119 -9884> 2014-08-13 17:23:19.508107 7fcd1d4027a0 2 journal read_entry 806387712 : seq 16298120 4040766 bytes -9883> 2014-08-13 17:23:19.508127 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298120 -9882> 2014-08-13 17:23:19.516012 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298120 -9881> 2014-08-13 17:23:19.516421 7fcd1d4027a0 2 journal read_entry 806391808 : seq 16298121 1943 bytes -9880> 2014-08-13 17:23:19.516564 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298121 -9879> 2014-08-13 17:23:19.516794 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298121 -9878> 2014-08-13 17:23:19.516821 7fcd1d4027a0 2 journal read_entry 806395904 : seq 16298122 2053 bytes -9877> 2014-08-13 17:23:19.516829 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298122 -9876> 2014-08-13 17:23:19.516984 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298122 -9875> 2014-08-13 17:23:19.517008 7fcd1d4027a0 2 journal read_entry 806400000 : seq 16298123 2378 bytes -9874> 2014-08-13 17:23:19.517015 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298123 -9873> 2014-08-13 17:23:19.517258 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298123 -9872> 2014-08-13 17:23:19.517291 7fcd1d4027a0 2 journal read_entry 806404096 : seq 16298124 1009 bytes -9871> 2014-08-13 17:23:19.517299 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298124 -9870> 2014-08-13 17:23:19.517445 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298124 -9869> 2014-08-13 17:23:19.517469 7fcd1d4027a0 2 journal read_entry 806408192 : seq 16298125 2053 bytes -9868> 2014-08-13 17:23:19.517476 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298125 -9867> 2014-08-13 17:23:19.517619 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298125 -9866> 2014-08-13 17:23:19.517648 7fcd1d4027a0 2 journal read_entry 806412288 : seq 16298126 3944 bytes -9865> 2014-08-13 17:23:19.517655 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298126 -9864> 2014-08-13 17:23:19.517828 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298126 -9863> 2014-08-13 17:23:19.517856 7fcd1d4027a0 2 journal read_entry 806416384 : seq 16298127 2315 bytes -9862> 2014-08-13 17:23:19.517864 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298127 -9861> 2014-08-13 17:23:19.518019 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298127 -9860> 2014-08-13 17:23:19.518043 7fcd1d4027a0 2 journal read_entry 806420480 : seq 16298128 2201 bytes -9859> 2014-08-13 17:23:19.518050 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298128 -9858> 2014-08-13 17:23:19.518298 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298128 -9857> 2014-08-13 17:23:19.518326 7fcd1d4027a0 2 journal read_entry 806424576 : seq 16298129 2116 bytes -9856> 2014-08-13 17:23:19.518333 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298129 -9855> 2014-08-13 17:23:19.518479 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298129 -9854> 2014-08-13 17:23:19.518506 7fcd1d4027a0 2 journal read_entry 806428672 : seq 16298130 1009 bytes -9853> 2014-08-13 17:23:19.518514 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298130 -9852> 2014-08-13 17:23:19.518654 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298130 -9851> 2014-08-13 17:23:19.518677 7fcd1d4027a0 2 journal read_entry 806432768 : seq 16298131 2167 bytes -9850> 2014-08-13 17:23:19.518684 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298131 -9849> 2014-08-13 17:23:19.518835 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298131 -9848> 2014-08-13 17:23:19.518859 7fcd1d4027a0 2 journal read_entry 806436864 : seq 16298132 1943 bytes -9847> 2014-08-13 17:23:19.518866 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298132 -9846> 2014-08-13 17:23:19.519020 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298132 -9845> 2014-08-13 17:23:19.519044 7fcd1d4027a0 2 journal read_entry 806440960 : seq 16298133 3306 bytes -9844> 2014-08-13 17:23:19.519051 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298133 -9843> 2014-08-13 17:23:19.519292 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298133 -9842> 2014-08-13 17:23:19.519319 7fcd1d4027a0 2 journal read_entry 806445056 : seq 16298134 1943 bytes -9841> 2014-08-13 17:23:19.519326 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298134 -9840> 2014-08-13 17:23:19.519488 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298134 -9839> 2014-08-13 17:23:19.519512 7fcd1d4027a0 2 journal read_entry 806449152 : seq 16298135 1936 bytes -9838> 2014-08-13 17:23:19.519519 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298135 -9837> 2014-08-13 17:23:19.519668 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298135 -9836> 2014-08-13 17:23:19.519690 7fcd1d4027a0 2 journal read_entry 806453248 : seq 16298136 2053 bytes -9835> 2014-08-13 17:23:19.519698 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298136 -9834> 2014-08-13 17:23:19.519854 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298136 -9833> 2014-08-13 17:23:19.519878 7fcd1d4027a0 2 journal read_entry 806457344 : seq 16298137 1044 bytes -9832> 2014-08-13 17:23:19.519885 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298137 -9831> 2014-08-13 17:23:19.520028 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298137 -9830> 2014-08-13 17:23:19.520052 7fcd1d4027a0 2 journal read_entry 806461440 : seq 16298138 2167 bytes -9829> 2014-08-13 17:23:19.520059 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298138 -9828> 2014-08-13 17:23:19.520278 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298138 -9827> 2014-08-13 17:23:19.520305 7fcd1d4027a0 2 journal read_entry 806465536 : seq 16298139 1943 bytes -9826> 2014-08-13 17:23:19.520310 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298139 -9825> 2014-08-13 17:23:19.520456 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298139 -9824> 2014-08-13 17:23:19.520479 7fcd1d4027a0 2 journal read_entry 806469632 : seq 16298140 2194 bytes -9823> 2014-08-13 17:23:19.520483 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298140 -9822> 2014-08-13 17:23:19.520640 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298140 -9821> 2014-08-13 17:23:19.520664 7fcd1d4027a0 2 journal read_entry 806473728 : seq 16298141 2053 bytes -9820> 2014-08-13 17:23:19.520668 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298141 -9819> 2014-08-13 17:23:19.520829 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298141 -9818> 2014-08-13 17:23:19.520851 7fcd1d4027a0 2 journal read_entry 806477824 : seq 16298142 2116 bytes -9817> 2014-08-13 17:23:19.520856 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298142 -9816> 2014-08-13 17:23:19.521014 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298142 -9815> 2014-08-13 17:23:19.521037 7fcd1d4027a0 2 journal read_entry 806481920 : seq 16298143 2053 bytes -9814> 2014-08-13 17:23:19.521045 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298143 -9813> 2014-08-13 17:23:19.521202 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298143 -9812> 2014-08-13 17:23:19.521225 7fcd1d4027a0 2 journal read_entry 806486016 : seq 16298144 1936 bytes -9811> 2014-08-13 17:23:19.521233 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298144 -9810> 2014-08-13 17:23:19.521402 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298144 -9809> 2014-08-13 17:23:19.521428 7fcd1d4027a0 2 journal read_entry 806490112 : seq 16298145 1943 bytes -9808> 2014-08-13 17:23:19.521435 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298145 -9807> 2014-08-13 17:23:19.521579 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298145 -9806> 2014-08-13 17:23:19.521602 7fcd1d4027a0 2 journal read_entry 806494208 : seq 16298146 2002 bytes -9805> 2014-08-13 17:23:19.521609 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298146 -9804> 2014-08-13 17:23:19.521886 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298146 -9803> 2014-08-13 17:23:19.521915 7fcd1d4027a0 2 journal read_entry 806498304 : seq 16298147 2641 bytes -9802> 2014-08-13 17:23:19.521922 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298147 -9801> 2014-08-13 17:23:19.522103 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298147 -9800> 2014-08-13 17:23:19.522128 7fcd1d4027a0 2 journal read_entry 806502400 : seq 16298148 2253 bytes -9799> 2014-08-13 17:23:19.522138 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298148 -9798> 2014-08-13 17:23:19.522318 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298148 -9797> 2014-08-13 17:23:19.522342 7fcd1d4027a0 2 journal read_entry 806506496 : seq 16298149 2496 bytes -9796> 2014-08-13 17:23:19.522349 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298149 -9795> 2014-08-13 17:23:19.522521 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298149 -9794> 2014-08-13 17:23:19.522547 7fcd1d4027a0 2 journal read_entry 806510592 : seq 16298150 2116 bytes -9793> 2014-08-13 17:23:19.522554 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298150 -9792> 2014-08-13 17:23:19.522703 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298150 -9791> 2014-08-13 17:23:19.522726 7fcd1d4027a0 2 journal read_entry 806514688 : seq 16298151 2496 bytes -9790> 2014-08-13 17:23:19.522733 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298151 -9789> 2014-08-13 17:23:19.522901 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298151 -9788> 2014-08-13 17:23:19.522925 7fcd1d4027a0 2 journal read_entry 806518784 : seq 16298152 2167 bytes -9787> 2014-08-13 17:23:19.522932 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298152 -9786> 2014-08-13 17:23:19.523087 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298152 -9785> 2014-08-13 17:23:19.523451 7fcd1d4027a0 2 journal read_entry 806522880 : seq 16298153 1943 bytes -9784> 2014-08-13 17:23:19.523464 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298153 -9783> 2014-08-13 17:23:19.523778 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298153 -9782> 2014-08-13 17:23:19.523804 7fcd1d4027a0 2 journal read_entry 806526976 : seq 16298154 1943 bytes -9781> 2014-08-13 17:23:19.523811 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298154 -9780> 2014-08-13 17:23:19.523955 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298154 -9779> 2014-08-13 17:23:19.523978 7fcd1d4027a0 2 journal read_entry 806531072 : seq 16298155 1943 bytes -9778> 2014-08-13 17:23:19.523985 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298155 -9777> 2014-08-13 17:23:19.524156 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298155 -9776> 2014-08-13 17:23:19.524182 7fcd1d4027a0 2 journal read_entry 806535168 : seq 16298156 2316 bytes -9775> 2014-08-13 17:23:19.524189 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298156 -9774> 2014-08-13 17:23:19.524362 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298156 -9773> 2014-08-13 17:23:19.524387 7fcd1d4027a0 2 journal read_entry 806539264 : seq 16298157 1047 bytes -9772> 2014-08-13 17:23:19.524394 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298157 -9771> 2014-08-13 17:23:19.524536 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298157 -9770> 2014-08-13 17:23:19.524562 7fcd1d4027a0 2 journal read_entry 806543360 : seq 16298158 1943 bytes -9769> 2014-08-13 17:23:19.524569 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298158 -9768> 2014-08-13 17:23:19.524714 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298158 -9767> 2014-08-13 17:23:19.524737 7fcd1d4027a0 2 journal read_entry 806547456 : seq 16298159 2421 bytes -9766> 2014-08-13 17:23:19.524745 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298159 -9765> 2014-08-13 17:23:19.524898 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298159 -9764> 2014-08-13 17:23:19.524922 7fcd1d4027a0 2 journal read_entry 806551552 : seq 16298160 2167 bytes -9763> 2014-08-13 17:23:19.524929 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298160 -9762> 2014-08-13 17:23:19.525091 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298160 -9761> 2014-08-13 17:23:19.525115 7fcd1d4027a0 2 journal read_entry 806555648 : seq 16298161 2319 bytes -9760> 2014-08-13 17:23:19.525123 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298161 -9759> 2014-08-13 17:23:19.525277 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298161 -9758> 2014-08-13 17:23:19.525301 7fcd1d4027a0 2 journal read_entry 806559744 : seq 16298162 2527 bytes -9757> 2014-08-13 17:23:19.525308 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298162 -9756> 2014-08-13 17:23:19.525476 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298162 -9755> 2014-08-13 17:23:19.525502 7fcd1d4027a0 2 journal read_entry 806563840 : seq 16298163 2675 bytes -9754> 2014-08-13 17:23:19.525510 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298163 -9753> 2014-08-13 17:23:19.525663 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298163 -9752> 2014-08-13 17:23:19.525686 7fcd1d4027a0 2 journal read_entry 806567936 : seq 16298164 1943 bytes -9751> 2014-08-13 17:23:19.525693 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298164 -9750> 2014-08-13 17:23:19.525845 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298164 -9749> 2014-08-13 17:23:19.525869 7fcd1d4027a0 2 journal read_entry 806572032 : seq 16298165 2201 bytes -9748> 2014-08-13 17:23:19.525877 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298165 -9747> 2014-08-13 17:23:19.526032 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298165 -9746> 2014-08-13 17:23:19.526056 7fcd1d4027a0 2 journal read_entry 806576128 : seq 16298166 2167 bytes -9745> 2014-08-13 17:23:19.526063 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298166 -9744> 2014-08-13 17:23:19.526223 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298166 -9743> 2014-08-13 17:23:19.526247 7fcd1d4027a0 2 journal read_entry 806580224 : seq 16298167 1943 bytes -9742> 2014-08-13 17:23:19.526255 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298167 -9741> 2014-08-13 17:23:19.526416 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298167 -9740> 2014-08-13 17:23:19.526443 7fcd1d4027a0 2 journal read_entry 806584320 : seq 16298168 2909 bytes -9739> 2014-08-13 17:23:19.526450 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298168 -9738> 2014-08-13 17:23:19.526610 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298168 -9737> 2014-08-13 17:23:19.526634 7fcd1d4027a0 2 journal read_entry 806588416 : seq 16298169 2167 bytes -9736> 2014-08-13 17:23:19.526641 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298169 -9735> 2014-08-13 17:23:19.526797 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298169 -9734> 2014-08-13 17:23:19.526821 7fcd1d4027a0 2 journal read_entry 806592512 : seq 16298170 2167 bytes -9733> 2014-08-13 17:23:19.526828 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298170 -9732> 2014-08-13 17:23:19.526972 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298170 -9731> 2014-08-13 17:23:19.526995 7fcd1d4027a0 2 journal read_entry 806596608 : seq 16298171 3122 bytes -9730> 2014-08-13 17:23:19.527003 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298171 -9729> 2014-08-13 17:23:19.527179 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298171 -9728> 2014-08-13 17:23:19.527204 7fcd1d4027a0 2 journal read_entry 806600704 : seq 16298172 1943 bytes -9727> 2014-08-13 17:23:19.527211 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298172 -9726> 2014-08-13 17:23:19.527373 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298172 -9725> 2014-08-13 17:23:19.527398 7fcd1d4027a0 2 journal read_entry 806604800 : seq 16298173 2053 bytes -9724> 2014-08-13 17:23:19.527406 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298173 -9723> 2014-08-13 17:23:19.527547 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298173 -9722> 2014-08-13 17:23:19.527570 7fcd1d4027a0 2 journal read_entry 806608896 : seq 16298174 2253 bytes -9721> 2014-08-13 17:23:19.527577 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298174 -9720> 2014-08-13 17:23:19.527732 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298174 -9719> 2014-08-13 17:23:19.527756 7fcd1d4027a0 2 journal read_entry 806612992 : seq 16298175 1943 bytes -9718> 2014-08-13 17:23:19.527764 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298175 -9717> 2014-08-13 17:23:19.527915 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298175 -9716> 2014-08-13 17:23:19.527938 7fcd1d4027a0 2 journal read_entry 806617088 : seq 16298176 1943 bytes -9715> 2014-08-13 17:23:19.527946 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298176 -9714> 2014-08-13 17:23:19.528095 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298176 -9713> 2014-08-13 17:23:19.528119 7fcd1d4027a0 2 journal read_entry 806621184 : seq 16298177 2201 bytes -9712> 2014-08-13 17:23:19.528127 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298177 -9711> 2014-08-13 17:23:19.528282 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298177 -9710> 2014-08-13 17:23:19.528304 7fcd1d4027a0 2 journal read_entry 806625280 : seq 16298178 2201 bytes -9709> 2014-08-13 17:23:19.528310 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298178 -9708> 2014-08-13 17:23:19.528497 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298178 -9707> 2014-08-13 17:23:19.528521 7fcd1d4027a0 2 journal read_entry 806629376 : seq 16298179 2256 bytes -9706> 2014-08-13 17:23:19.528526 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298179 -9705> 2014-08-13 17:23:19.528703 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298179 -9704> 2014-08-13 17:23:19.528727 7fcd1d4027a0 2 journal read_entry 806633472 : seq 16298180 2138 bytes -9703> 2014-08-13 17:23:19.528731 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298180 -9702> 2014-08-13 17:23:19.528871 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298180 -9701> 2014-08-13 17:23:19.528893 7fcd1d4027a0 2 journal read_entry 806637568 : seq 16298181 2138 bytes -9700> 2014-08-13 17:23:19.528900 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298181 -9699> 2014-08-13 17:23:19.529055 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298181 -9698> 2014-08-13 17:23:19.529079 7fcd1d4027a0 2 journal read_entry 806641664 : seq 16298182 1047 bytes -9697> 2014-08-13 17:23:19.529091 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298182 -9696> 2014-08-13 17:23:19.529243 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298182 -9695> 2014-08-13 17:23:19.529266 7fcd1d4027a0 2 journal read_entry 806645760 : seq 16298183 1943 bytes -9694> 2014-08-13 17:23:19.529274 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298183 -9693> 2014-08-13 17:23:19.529416 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298183 -9692> 2014-08-13 17:23:19.529439 7fcd1d4027a0 2 journal read_entry 806649856 : seq 16298184 2002 bytes -9691> 2014-08-13 17:23:19.529446 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298184 -9690> 2014-08-13 17:23:19.529605 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298184 -9689> 2014-08-13 17:23:19.529960 7fcd1d4027a0 2 journal read_entry 806653952 : seq 16298185 2198 bytes -9688> 2014-08-13 17:23:19.529973 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298185 -9687> 2014-08-13 17:23:19.530297 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298185 -9686> 2014-08-13 17:23:19.530323 7fcd1d4027a0 2 journal read_entry 806658048 : seq 16298186 1047 bytes -9685> 2014-08-13 17:23:19.530331 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298186 -9684> 2014-08-13 17:23:19.530469 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298186 -9683> 2014-08-13 17:23:19.530492 7fcd1d4027a0 2 journal read_entry 806662144 : seq 16298187 2116 bytes -9682> 2014-08-13 17:23:19.530499 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298187 -9681> 2014-08-13 17:23:19.530644 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298187 -9680> 2014-08-13 17:23:19.530666 7fcd1d4027a0 2 journal read_entry 806666240 : seq 16298188 1943 bytes -9679> 2014-08-13 17:23:19.530674 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298188 -9678> 2014-08-13 17:23:19.530827 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298188 -9677> 2014-08-13 17:23:19.530851 7fcd1d4027a0 2 journal read_entry 806670336 : seq 16298189 844 bytes -9676> 2014-08-13 17:23:19.530858 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298189 -9675> 2014-08-13 17:23:19.530997 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298189 -9674> 2014-08-13 17:23:19.531021 7fcd1d4027a0 2 journal read_entry 806674432 : seq 16298190 3085 bytes -9673> 2014-08-13 17:23:19.531028 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298190 -9672> 2014-08-13 17:23:19.531208 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298190 -9671> 2014-08-13 17:23:19.531233 7fcd1d4027a0 2 journal read_entry 806678528 : seq 16298191 844 bytes -9670> 2014-08-13 17:23:19.531245 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298191 -9669> 2014-08-13 17:23:19.531385 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298191 -9668> 2014-08-13 17:23:19.531407 7fcd1d4027a0 2 journal read_entry 806682624 : seq 16298192 844 bytes -9667> 2014-08-13 17:23:19.531414 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298192 -9666> 2014-08-13 17:23:19.531539 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298192 -9665> 2014-08-13 17:23:19.531560 7fcd1d4027a0 2 journal read_entry 806686720 : seq 16298193 844 bytes -9664> 2014-08-13 17:23:19.531569 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298193 -9663> 2014-08-13 17:23:19.531694 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298193 -9662> 2014-08-13 17:23:19.531716 7fcd1d4027a0 2 journal read_entry 806690816 : seq 16298194 844 bytes -9661> 2014-08-13 17:23:19.531723 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298194 -9660> 2014-08-13 17:23:19.531870 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298194 -9659> 2014-08-13 17:23:19.531894 7fcd1d4027a0 2 journal read_entry 806694912 : seq 16298195 876 bytes -9658> 2014-08-13 17:23:19.531901 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298195 -9657> 2014-08-13 17:23:19.532031 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298195 -9656> 2014-08-13 17:23:19.532053 7fcd1d4027a0 2 journal read_entry 806699008 : seq 16298196 844 bytes -9655> 2014-08-13 17:23:19.532061 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298196 -9654> 2014-08-13 17:23:19.532197 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298196 -9653> 2014-08-13 17:23:19.532219 7fcd1d4027a0 2 journal read_entry 806703104 : seq 16298197 878 bytes -9652> 2014-08-13 17:23:19.532226 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298197 -9651> 2014-08-13 17:23:19.532353 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298197 -9650> 2014-08-13 17:23:19.532376 7fcd1d4027a0 2 journal read_entry 806707200 : seq 16298198 3121 bytes -9649> 2014-08-13 17:23:19.532384 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298198 -9648> 2014-08-13 17:23:19.532558 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298198 -9647> 2014-08-13 17:23:19.532582 7fcd1d4027a0 2 journal read_entry 806711296 : seq 16298199 842 bytes -9646> 2014-08-13 17:23:19.532590 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298199 -9645> 2014-08-13 17:23:19.532726 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298199 -9644> 2014-08-13 17:23:19.532748 7fcd1d4027a0 2 journal read_entry 806715392 : seq 16298200 844 bytes -9643> 2014-08-13 17:23:19.532756 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298200 -9642> 2014-08-13 17:23:19.532894 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298200 -9641> 2014-08-13 17:23:19.532917 7fcd1d4027a0 2 journal read_entry 806719488 : seq 16298201 844 bytes -9640> 2014-08-13 17:23:19.532924 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298201 -9639> 2014-08-13 17:23:19.533053 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298201 -9638> 2014-08-13 17:23:19.533075 7fcd1d4027a0 2 journal read_entry 806723584 : seq 16298202 844 bytes -9637> 2014-08-13 17:23:19.533082 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298202 -9636> 2014-08-13 17:23:19.533227 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298202 -9635> 2014-08-13 17:23:19.533250 7fcd1d4027a0 2 journal read_entry 806727680 : seq 16298203 2002 bytes -9634> 2014-08-13 17:23:19.533258 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298203 -9633> 2014-08-13 17:23:19.533416 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298203 -9632> 2014-08-13 17:23:19.533440 7fcd1d4027a0 2 journal read_entry 806731776 : seq 16298204 878 bytes -9631> 2014-08-13 17:23:19.533447 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298204 -9630> 2014-08-13 17:23:19.533589 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298204 -9629> 2014-08-13 17:23:19.533612 7fcd1d4027a0 2 journal read_entry 806735872 : seq 16298205 844 bytes -9628> 2014-08-13 17:23:19.533619 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298205 -9627> 2014-08-13 17:23:19.533746 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298205 -9626> 2014-08-13 17:23:19.533767 7fcd1d4027a0 2 journal read_entry 806739968 : seq 16298206 844 bytes -9625> 2014-08-13 17:23:19.533775 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298206 -9624> 2014-08-13 17:23:19.533899 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298206 -9623> 2014-08-13 17:23:19.533919 7fcd1d4027a0 2 journal read_entry 806744064 : seq 16298207 43 bytes -9622> 2014-08-13 17:23:19.533926 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298207 -9621> 2014-08-13 17:23:19.533931 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298207 -9620> 2014-08-13 17:23:19.533945 7fcd1d4027a0 2 journal read_entry 806748160 : seq 16298208 2201 bytes -9619> 2014-08-13 17:23:19.533952 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298208 -9618> 2014-08-13 17:23:19.534121 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298208 -9617> 2014-08-13 17:23:19.534144 7fcd1d4027a0 2 journal read_entry 806752256 : seq 16298209 878 bytes -9616> 2014-08-13 17:23:19.534161 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298209 -9615> 2014-08-13 17:23:19.534297 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298209 -9614> 2014-08-13 17:23:19.534319 7fcd1d4027a0 2 journal read_entry 806756352 : seq 16298210 844 bytes -9613> 2014-08-13 17:23:19.534326 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298210 -9612> 2014-08-13 17:23:19.534452 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298210 -9611> 2014-08-13 17:23:19.534473 7fcd1d4027a0 2 journal read_entry 806760448 : seq 16298211 842 bytes -9610> 2014-08-13 17:23:19.534480 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298211 -9609> 2014-08-13 17:23:19.534606 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298211 -9608> 2014-08-13 17:23:19.534627 7fcd1d4027a0 2 journal read_entry 806764544 : seq 16298212 844 bytes -9607> 2014-08-13 17:23:19.534634 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298212 -9606> 2014-08-13 17:23:19.534786 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298212 -9605> 2014-08-13 17:23:19.534810 7fcd1d4027a0 2 journal read_entry 806768640 : seq 16298213 844 bytes -9604> 2014-08-13 17:23:19.534817 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298213 -9603> 2014-08-13 17:23:19.534946 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298213 -9602> 2014-08-13 17:23:19.534968 7fcd1d4027a0 2 journal read_entry 806772736 : seq 16298214 876 bytes -9601> 2014-08-13 17:23:19.534975 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298214 -9600> 2014-08-13 17:23:19.535109 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298214 -9599> 2014-08-13 17:23:19.535134 7fcd1d4027a0 2 journal read_entry 806776832 : seq 16298215 832 bytes -9598> 2014-08-13 17:23:19.535142 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298215 -9597> 2014-08-13 17:23:19.535280 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298215 -9596> 2014-08-13 17:23:19.535302 7fcd1d4027a0 2 journal read_entry 806780928 : seq 16298216 876 bytes -9595> 2014-08-13 17:23:19.535312 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298216 -9594> 2014-08-13 17:23:19.535465 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298216 -9593> 2014-08-13 17:23:19.535827 7fcd1d4027a0 2 journal read_entry 806785024 : seq 16298217 842 bytes -9592> 2014-08-13 17:23:19.535839 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298217 -9591> 2014-08-13 17:23:19.536113 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298217 -9590> 2014-08-13 17:23:19.536136 7fcd1d4027a0 2 journal read_entry 806789120 : seq 16298218 844 bytes -9589> 2014-08-13 17:23:19.536144 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298218 -9588> 2014-08-13 17:23:19.536287 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298218 -9587> 2014-08-13 17:23:19.536308 7fcd1d4027a0 2 journal read_entry 806793216 : seq 16298219 844 bytes -9586> 2014-08-13 17:23:19.536313 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298219 -9585> 2014-08-13 17:23:19.536437 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298219 -9584> 2014-08-13 17:23:19.536458 7fcd1d4027a0 2 journal read_entry 806797312 : seq 16298220 844 bytes -9583> 2014-08-13 17:23:19.536463 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298220 -9582> 2014-08-13 17:23:19.536630 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298220 -9581> 2014-08-13 17:23:19.536651 7fcd1d4027a0 2 journal read_entry 806801408 : seq 16298221 43 bytes -9580> 2014-08-13 17:23:19.536656 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298221 -9579> 2014-08-13 17:23:19.536664 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298221 -9578> 2014-08-13 17:23:19.536679 7fcd1d4027a0 2 journal read_entry 806805504 : seq 16298222 844 bytes -9577> 2014-08-13 17:23:19.536685 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298222 -9576> 2014-08-13 17:23:19.536985 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298222 -9575> 2014-08-13 17:23:19.537011 7fcd1d4027a0 2 journal read_entry 806809600 : seq 16298223 844 bytes -9574> 2014-08-13 17:23:19.537016 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298223 -9573> 2014-08-13 17:23:19.537165 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298223 -9572> 2014-08-13 17:23:19.537187 7fcd1d4027a0 2 journal read_entry 806813696 : seq 16298224 878 bytes -9571> 2014-08-13 17:23:19.537192 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298224 -9570> 2014-08-13 17:23:19.537320 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298224 -9569> 2014-08-13 17:23:19.537340 7fcd1d4027a0 2 journal read_entry 806817792 : seq 16298225 43 bytes -9568> 2014-08-13 17:23:19.537344 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298225 -9567> 2014-08-13 17:23:19.537349 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298225 -9566> 2014-08-13 17:23:19.537359 7fcd1d4027a0 2 journal read_entry 806821888 : seq 16298226 844 bytes -9565> 2014-08-13 17:23:19.537362 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298226 -9564> 2014-08-13 17:23:19.537525 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298226 -9563> 2014-08-13 17:23:19.537546 7fcd1d4027a0 2 journal read_entry 806825984 : seq 16298227 43 bytes -9562> 2014-08-13 17:23:19.537550 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298227 -9561> 2014-08-13 17:23:19.537555 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298227 -9560> 2014-08-13 17:23:19.537564 7fcd1d4027a0 2 journal read_entry 806830080 : seq 16298228 43 bytes -9559> 2014-08-13 17:23:19.537570 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298228 -9558> 2014-08-13 17:23:19.537573 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298228 -9557> 2014-08-13 17:23:19.537584 7fcd1d4027a0 2 journal read_entry 806834176 : seq 16298229 43 bytes -9556> 2014-08-13 17:23:19.537591 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298229 -9555> 2014-08-13 17:23:19.537595 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298229 -9554> 2014-08-13 17:23:19.537607 7fcd1d4027a0 2 journal read_entry 806838272 : seq 16298230 43 bytes -9553> 2014-08-13 17:23:19.537610 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298230 -9552> 2014-08-13 17:23:19.537613 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298230 -9551> 2014-08-13 17:23:19.537622 7fcd1d4027a0 2 journal read_entry 806842368 : seq 16298231 832 bytes -9550> 2014-08-13 17:23:19.537629 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298231 -9549> 2014-08-13 17:23:19.537755 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298231 -9548> 2014-08-13 17:23:19.537775 7fcd1d4027a0 2 journal read_entry 806846464 : seq 16298232 43 bytes -9547> 2014-08-13 17:23:19.537779 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298232 -9546> 2014-08-13 17:23:19.537785 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298232 -9545> 2014-08-13 17:23:19.537798 7fcd1d4027a0 2 journal read_entry 806850560 : seq 16298233 43 bytes -9544> 2014-08-13 17:23:19.537804 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298233 -9543> 2014-08-13 17:23:19.537807 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298233 -9542> 2014-08-13 17:23:19.537816 7fcd1d4027a0 2 journal read_entry 806854656 : seq 16298234 43 bytes -9541> 2014-08-13 17:23:19.537819 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298234 -9540> 2014-08-13 17:23:19.537856 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298234 -9539> 2014-08-13 17:23:19.537872 7fcd1d4027a0 2 journal read_entry 806858752 : seq 16298235 43 bytes -9538> 2014-08-13 17:23:19.537876 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298235 -9537> 2014-08-13 17:23:19.537880 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298235 -9536> 2014-08-13 17:23:19.537892 7fcd1d4027a0 2 journal read_entry 806862848 : seq 16298236 43 bytes -9535> 2014-08-13 17:23:19.537899 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298236 -9534> 2014-08-13 17:23:19.537902 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298236 -9533> 2014-08-13 17:23:19.537914 7fcd1d4027a0 2 journal read_entry 806866944 : seq 16298237 43 bytes -9532> 2014-08-13 17:23:19.537920 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298237 -9531> 2014-08-13 17:23:19.537924 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298237 -9530> 2014-08-13 17:23:19.537936 7fcd1d4027a0 2 journal read_entry 806871040 : seq 16298238 43 bytes -9529> 2014-08-13 17:23:19.537942 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298238 -9528> 2014-08-13 17:23:19.537945 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298238 -9527> 2014-08-13 17:23:19.537957 7fcd1d4027a0 2 journal read_entry 806875136 : seq 16298239 43 bytes -9526> 2014-08-13 17:23:19.537963 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298239 -9525> 2014-08-13 17:23:19.537967 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298239 -9524> 2014-08-13 17:23:19.537979 7fcd1d4027a0 2 journal read_entry 806879232 : seq 16298240 43 bytes -9523> 2014-08-13 17:23:19.537985 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298240 -9522> 2014-08-13 17:23:19.537988 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298240 -9521> 2014-08-13 17:23:19.538000 7fcd1d4027a0 2 journal read_entry 806883328 : seq 16298241 43 bytes -9520> 2014-08-13 17:23:19.538006 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298241 -9519> 2014-08-13 17:23:19.538009 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298241 -9518> 2014-08-13 17:23:19.538022 7fcd1d4027a0 2 journal read_entry 806887424 : seq 16298242 43 bytes -9517> 2014-08-13 17:23:19.538028 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298242 -9516> 2014-08-13 17:23:19.538031 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298242 -9515> 2014-08-13 17:23:19.538043 7fcd1d4027a0 2 journal read_entry 806891520 : seq 16298243 43 bytes -9514> 2014-08-13 17:23:19.538049 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298243 -9513> 2014-08-13 17:23:19.538052 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298243 -9512> 2014-08-13 17:23:19.538065 7fcd1d4027a0 2 journal read_entry 806895616 : seq 16298244 43 bytes -9511> 2014-08-13 17:23:19.538071 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298244 -9510> 2014-08-13 17:23:19.538074 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298244 -9509> 2014-08-13 17:23:19.538086 7fcd1d4027a0 2 journal read_entry 806899712 : seq 16298245 43 bytes -9508> 2014-08-13 17:23:19.538092 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298245 -9507> 2014-08-13 17:23:19.538096 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298245 -9506> 2014-08-13 17:23:19.538108 7fcd1d4027a0 2 journal read_entry 806903808 : seq 16298246 43 bytes -9505> 2014-08-13 17:23:19.538114 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298246 -9504> 2014-08-13 17:23:19.538117 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298246 -9503> 2014-08-13 17:23:19.538129 7fcd1d4027a0 2 journal read_entry 806907904 : seq 16298247 43 bytes -9502> 2014-08-13 17:23:19.538137 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298247 -9501> 2014-08-13 17:23:19.538140 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298247 -9500> 2014-08-13 17:23:19.539674 7fcd1d4027a0 2 journal read_entry 807182336 : seq 16298248 270312 bytes -9499> 2014-08-13 17:23:19.539689 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298248 -9498> 2014-08-13 17:23:19.540406 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298248 -9497> 2014-08-13 17:23:19.540437 7fcd1d4027a0 2 journal read_entry 807186432 : seq 16298249 2374 bytes -9496> 2014-08-13 17:23:19.540444 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298249 -9495> 2014-08-13 17:23:19.540708 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298249 -9494> 2014-08-13 17:23:19.540734 7fcd1d4027a0 2 journal read_entry 807190528 : seq 16298250 2171 bytes -9493> 2014-08-13 17:23:19.540742 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298250 -9492> 2014-08-13 17:23:19.540936 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298250 -9491> 2014-08-13 17:23:19.540961 7fcd1d4027a0 2 journal read_entry 807194624 : seq 16298251 1940 bytes -9490> 2014-08-13 17:23:19.540968 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298251 -9489> 2014-08-13 17:23:19.541120 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298251 -9488> 2014-08-13 17:23:19.541143 7fcd1d4027a0 2 journal read_entry 807198720 : seq 16298252 1048 bytes -9487> 2014-08-13 17:23:19.541160 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298252 -9486> 2014-08-13 17:23:19.541302 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298252 -9485> 2014-08-13 17:23:19.541325 7fcd1d4027a0 2 journal read_entry 807202816 : seq 16298253 2057 bytes -9484> 2014-08-13 17:23:19.541332 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298253 -9483> 2014-08-13 17:23:19.541497 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298253 -9482> 2014-08-13 17:23:19.541522 7fcd1d4027a0 2 journal read_entry 807206912 : seq 16298254 2057 bytes -9481> 2014-08-13 17:23:19.541529 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298254 -9480> 2014-08-13 17:23:19.541670 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298254 -9479> 2014-08-13 17:23:19.541693 7fcd1d4027a0 2 journal read_entry 807211008 : seq 16298255 2171 bytes -9478> 2014-08-13 17:23:19.541700 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298255 -9477> 2014-08-13 17:23:19.541864 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298255 -9476> 2014-08-13 17:23:19.541888 7fcd1d4027a0 2 journal read_entry 807215104 : seq 16298256 2057 bytes -9475> 2014-08-13 17:23:19.541895 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298256 -9474> 2014-08-13 17:23:19.542055 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298256 -9473> 2014-08-13 17:23:19.542079 7fcd1d4027a0 2 journal read_entry 807219200 : seq 16298257 2253 bytes -9472> 2014-08-13 17:23:19.542086 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298257 -9471> 2014-08-13 17:23:19.542245 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298257 -9470> 2014-08-13 17:23:19.542269 7fcd1d4027a0 2 journal read_entry 807223296 : seq 16298258 2555 bytes -9469> 2014-08-13 17:23:19.542276 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298258 -9468> 2014-08-13 17:23:19.542446 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298258 -9467> 2014-08-13 17:23:19.542471 7fcd1d4027a0 2 journal read_entry 807227392 : seq 16298259 1940 bytes -9466> 2014-08-13 17:23:19.542479 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298259 -9465> 2014-08-13 17:23:19.542629 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298259 -9464> 2014-08-13 17:23:19.542652 7fcd1d4027a0 2 journal read_entry 807231488 : seq 16298260 2312 bytes -9463> 2014-08-13 17:23:19.542662 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298260 -9462> 2014-08-13 17:23:19.542836 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298260 -9461> 2014-08-13 17:23:19.542861 7fcd1d4027a0 2 journal read_entry 807235584 : seq 16298261 2171 bytes -9460> 2014-08-13 17:23:19.542868 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298261 -9459> 2014-08-13 17:23:19.543016 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298261 -9458> 2014-08-13 17:23:19.543039 7fcd1d4027a0 2 journal read_entry 807239680 : seq 16298262 2375 bytes -9457> 2014-08-13 17:23:19.543047 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298262 -9456> 2014-08-13 17:23:19.543225 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298262 -9455> 2014-08-13 17:23:19.543250 7fcd1d4027a0 2 journal read_entry 807243776 : seq 16298263 2057 bytes -9454> 2014-08-13 17:23:19.543258 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298263 -9453> 2014-08-13 17:23:19.543423 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298263 -9452> 2014-08-13 17:23:19.543447 7fcd1d4027a0 2 journal read_entry 807247872 : seq 16298264 1358 bytes -9451> 2014-08-13 17:23:19.543454 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298264 -9450> 2014-08-13 17:23:19.544037 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298264 -9449> 2014-08-13 17:23:19.544063 7fcd1d4027a0 2 journal read_entry 807251968 : seq 16298265 2437 bytes -9448> 2014-08-13 17:23:19.544070 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298265 -9447> 2014-08-13 17:23:19.544245 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298265 -9446> 2014-08-13 17:23:19.544270 7fcd1d4027a0 2 journal read_entry 807256064 : seq 16298266 2171 bytes -9445> 2014-08-13 17:23:19.544277 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298266 -9444> 2014-08-13 17:23:19.544424 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298266 -9443> 2014-08-13 17:23:19.544448 7fcd1d4027a0 2 journal read_entry 807260160 : seq 16298267 2586 bytes -9442> 2014-08-13 17:23:19.544455 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298267 -9441> 2014-08-13 17:23:19.544620 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298267 -9440> 2014-08-13 17:23:19.544644 7fcd1d4027a0 2 journal read_entry 807264256 : seq 16298268 2171 bytes -9439> 2014-08-13 17:23:19.544652 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298268 -9438> 2014-08-13 17:23:19.544793 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298268 -9437> 2014-08-13 17:23:19.544822 7fcd1d4027a0 2 journal read_entry 807268352 : seq 16298269 1013 bytes -9436> 2014-08-13 17:23:19.544830 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298269 -9435> 2014-08-13 17:23:19.544979 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298269 -9434> 2014-08-13 17:23:19.545003 7fcd1d4027a0 2 journal read_entry 807272448 : seq 16298270 2968 bytes -9433> 2014-08-13 17:23:19.545011 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298270 -9432> 2014-08-13 17:23:19.545166 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298270 -9431> 2014-08-13 17:23:19.545189 7fcd1d4027a0 2 journal read_entry 807276544 : seq 16298271 2057 bytes -9430> 2014-08-13 17:23:19.545197 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298271 -9429> 2014-08-13 17:23:19.545362 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298271 -9428> 2014-08-13 17:23:19.545386 7fcd1d4027a0 2 journal read_entry 807280640 : seq 16298272 1358 bytes -9427> 2014-08-13 17:23:19.545394 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298272 -9426> 2014-08-13 17:23:19.546090 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298272 -9425> 2014-08-13 17:23:19.546116 7fcd1d4027a0 2 journal read_entry 807284736 : seq 16298273 2171 bytes -9424> 2014-08-13 17:23:19.546124 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298273 -9423> 2014-08-13 17:23:19.546307 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298273 -9422> 2014-08-13 17:23:19.546332 7fcd1d4027a0 2 journal read_entry 807288832 : seq 16298274 2260 bytes -9421> 2014-08-13 17:23:19.546339 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298274 -9420> 2014-08-13 17:23:19.546484 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298274 -9419> 2014-08-13 17:23:19.546506 7fcd1d4027a0 2 journal read_entry 807292928 : seq 16298275 2312 bytes -9418> 2014-08-13 17:23:19.546514 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298275 -9417> 2014-08-13 17:23:19.546673 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298275 -9416> 2014-08-13 17:23:19.546697 7fcd1d4027a0 2 journal read_entry 807297024 : seq 16298276 2260 bytes -9415> 2014-08-13 17:23:19.546705 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298276 -9414> 2014-08-13 17:23:19.546848 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298276 -9413> 2014-08-13 17:23:19.546871 7fcd1d4027a0 2 journal read_entry 807301120 : seq 16298277 2315 bytes -9412> 2014-08-13 17:23:19.546878 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298277 -9411> 2014-08-13 17:23:19.547078 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298277 -9410> 2014-08-13 17:23:19.547103 7fcd1d4027a0 2 journal read_entry 807305216 : seq 16298278 2197 bytes -9409> 2014-08-13 17:23:19.547111 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298278 -9408> 2014-08-13 17:23:19.547298 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298278 -9407> 2014-08-13 17:23:19.547661 7fcd1d4027a0 2 journal read_entry 807309312 : seq 16298279 2555 bytes -9406> 2014-08-13 17:23:19.547674 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298279 -9405> 2014-08-13 17:23:19.547975 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298279 -9404> 2014-08-13 17:23:19.547999 7fcd1d4027a0 2 journal read_entry 807313408 : seq 16298280 1013 bytes -9403> 2014-08-13 17:23:19.548007 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298280 -9402> 2014-08-13 17:23:19.548188 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298280 -9401> 2014-08-13 17:23:19.548214 7fcd1d4027a0 2 journal read_entry 807317504 : seq 16298281 1051 bytes -9400> 2014-08-13 17:23:19.548221 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298281 -9399> 2014-08-13 17:23:19.548383 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298281 -9398> 2014-08-13 17:23:19.548406 7fcd1d4027a0 2 journal read_entry 807321600 : seq 16298282 2476 bytes -9397> 2014-08-13 17:23:19.548413 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298282 -9396> 2014-08-13 17:23:19.548560 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298282 -9395> 2014-08-13 17:23:19.548583 7fcd1d4027a0 2 journal read_entry 807325696 : seq 16298283 3353 bytes -9394> 2014-08-13 17:23:19.548590 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298283 -9393> 2014-08-13 17:23:19.548764 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298283 -9392> 2014-08-13 17:23:19.548789 7fcd1d4027a0 2 journal read_entry 807329792 : seq 16298284 2378 bytes -9391> 2014-08-13 17:23:19.548796 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298284 -9390> 2014-08-13 17:23:19.548955 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298284 -9389> 2014-08-13 17:23:19.548979 7fcd1d4027a0 2 journal read_entry 807333888 : seq 16298285 2734 bytes -9388> 2014-08-13 17:23:19.548986 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298285 -9387> 2014-08-13 17:23:19.549161 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298285 -9386> 2014-08-13 17:23:19.549186 7fcd1d4027a0 2 journal read_entry 807337984 : seq 16298286 2260 bytes -9385> 2014-08-13 17:23:19.549194 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298286 -9384> 2014-08-13 17:23:19.549336 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298286 -9383> 2014-08-13 17:23:19.549364 7fcd1d4027a0 2 journal read_entry 807342080 : seq 16298287 1051 bytes -9382> 2014-08-13 17:23:19.549371 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298287 -9381> 2014-08-13 17:23:19.549512 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298287 -9380> 2014-08-13 17:23:19.549534 7fcd1d4027a0 2 journal read_entry 807346176 : seq 16298288 1051 bytes -9379> 2014-08-13 17:23:19.549542 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298288 -9378> 2014-08-13 17:23:19.549694 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298288 -9377> 2014-08-13 17:23:19.549719 7fcd1d4027a0 2 journal read_entry 807350272 : seq 16298289 2257 bytes -9376> 2014-08-13 17:23:19.549727 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298289 -9375> 2014-08-13 17:23:19.549878 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298289 -9374> 2014-08-13 17:23:19.549902 7fcd1d4027a0 2 journal read_entry 807354368 : seq 16298290 3089 bytes -9373> 2014-08-13 17:23:19.549909 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298290 -9372> 2014-08-13 17:23:19.550077 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298290 -9371> 2014-08-13 17:23:19.550101 7fcd1d4027a0 2 journal read_entry 807358464 : seq 16298291 2171 bytes -9370> 2014-08-13 17:23:19.550109 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298291 -9369> 2014-08-13 17:23:19.550287 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298291 -9368> 2014-08-13 17:23:19.550313 7fcd1d4027a0 2 journal read_entry 807362560 : seq 16298292 3180 bytes -9367> 2014-08-13 17:23:19.550320 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298292 -9366> 2014-08-13 17:23:19.550472 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298292 -9365> 2014-08-13 17:23:19.550496 7fcd1d4027a0 2 journal read_entry 807366656 : seq 16298293 2260 bytes -9364> 2014-08-13 17:23:19.550503 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298293 -9363> 2014-08-13 17:23:19.550656 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298293 -9362> 2014-08-13 17:23:19.550681 7fcd1d4027a0 2 journal read_entry 807370752 : seq 16298294 3126 bytes -9361> 2014-08-13 17:23:19.550688 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298294 -9360> 2014-08-13 17:23:19.550850 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298294 -9359> 2014-08-13 17:23:19.550874 7fcd1d4027a0 2 journal read_entry 807374848 : seq 16298295 2002 bytes -9358> 2014-08-13 17:23:19.550881 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298295 -9357> 2014-08-13 17:23:19.551024 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298295 -9356> 2014-08-13 17:23:19.551047 7fcd1d4027a0 2 journal read_entry 807378944 : seq 16298296 2116 bytes -9355> 2014-08-13 17:23:19.551054 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298296 -9354> 2014-08-13 17:23:19.551223 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298296 -9353> 2014-08-13 17:23:19.551247 7fcd1d4027a0 2 journal read_entry 807383040 : seq 16298297 2057 bytes -9352> 2014-08-13 17:23:19.551255 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298297 -9351> 2014-08-13 17:23:19.551412 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298297 -9350> 2014-08-13 17:23:19.551435 7fcd1d4027a0 2 journal read_entry 807387136 : seq 16298298 1943 bytes -9349> 2014-08-13 17:23:19.551442 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298298 -9348> 2014-08-13 17:23:19.551601 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298298 -9347> 2014-08-13 17:23:19.551625 7fcd1d4027a0 2 journal read_entry 807391232 : seq 16298299 1943 bytes -9346> 2014-08-13 17:23:19.551632 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298299 -9345> 2014-08-13 17:23:19.551773 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298299 -9344> 2014-08-13 17:23:19.551795 7fcd1d4027a0 2 journal read_entry 807395328 : seq 16298300 2260 bytes -9343> 2014-08-13 17:23:19.551802 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298300 -9342> 2014-08-13 17:23:19.551954 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298300 -9341> 2014-08-13 17:23:19.551980 7fcd1d4027a0 2 journal read_entry 807399424 : seq 16298301 2197 bytes -9340> 2014-08-13 17:23:19.551988 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298301 -9339> 2014-08-13 17:23:19.552163 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298301 -9338> 2014-08-13 17:23:19.552189 7fcd1d4027a0 2 journal read_entry 807403520 : seq 16298302 1943 bytes -9337> 2014-08-13 17:23:19.552194 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298302 -9336> 2014-08-13 17:23:19.552376 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298302 -9335> 2014-08-13 17:23:19.552398 7fcd1d4027a0 2 journal read_entry 807407616 : seq 16298303 1943 bytes -9334> 2014-08-13 17:23:19.552402 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298303 -9333> 2014-08-13 17:23:19.552566 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298303 -9332> 2014-08-13 17:23:19.552590 7fcd1d4027a0 2 journal read_entry 807411712 : seq 16298304 868 bytes -9331> 2014-08-13 17:23:19.552594 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298304 -9330> 2014-08-13 17:23:19.552726 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298304 -9329> 2014-08-13 17:23:19.552748 7fcd1d4027a0 2 journal read_entry 807415808 : seq 16298305 1943 bytes -9328> 2014-08-13 17:23:19.552752 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298305 -9327> 2014-08-13 17:23:19.552934 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298305 -9326> 2014-08-13 17:23:19.552957 7fcd1d4027a0 2 journal read_entry 807419904 : seq 16298306 1943 bytes -9325> 2014-08-13 17:23:19.552961 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298306 -9324> 2014-08-13 17:23:19.553116 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298306 -9323> 2014-08-13 17:23:19.553140 7fcd1d4027a0 2 journal read_entry 807424000 : seq 16298307 1943 bytes -9322> 2014-08-13 17:23:19.553144 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298307 -9321> 2014-08-13 17:23:19.553335 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298307 -9320> 2014-08-13 17:23:19.553358 7fcd1d4027a0 2 journal read_entry 807428096 : seq 16298308 1943 bytes -9319> 2014-08-13 17:23:19.553362 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298308 -9318> 2014-08-13 17:23:19.553528 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298308 -9317> 2014-08-13 17:23:19.553552 7fcd1d4027a0 2 journal read_entry 807432192 : seq 16298309 1943 bytes -9316> 2014-08-13 17:23:19.553557 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298309 -9315> 2014-08-13 17:23:19.553698 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298309 -9314> 2014-08-13 17:23:19.553720 7fcd1d4027a0 2 journal read_entry 807436288 : seq 16298310 1943 bytes -9313> 2014-08-13 17:23:19.553724 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298310 -9312> 2014-08-13 17:23:19.553894 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298310 -9311> 2014-08-13 17:23:19.554357 7fcd1d4027a0 2 journal read_entry 807440384 : seq 16298311 1943 bytes -9310> 2014-08-13 17:23:19.554371 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298311 -9309> 2014-08-13 17:23:19.554664 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298311 -9308> 2014-08-13 17:23:19.554689 7fcd1d4027a0 2 journal read_entry 807444480 : seq 16298312 1943 bytes -9307> 2014-08-13 17:23:19.554696 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298312 -9306> 2014-08-13 17:23:19.554860 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298312 -9305> 2014-08-13 17:23:19.554885 7fcd1d4027a0 2 journal read_entry 807448576 : seq 16298313 2116 bytes -9304> 2014-08-13 17:23:19.554892 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298313 -9303> 2014-08-13 17:23:19.555036 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298313 -9302> 2014-08-13 17:23:19.555058 7fcd1d4027a0 2 journal read_entry 807452672 : seq 16298314 2116 bytes -9301> 2014-08-13 17:23:19.555066 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298314 -9300> 2014-08-13 17:23:19.555268 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298314 -9299> 2014-08-13 17:23:19.555294 7fcd1d4027a0 2 journal read_entry 807456768 : seq 16298315 1943 bytes -9298> 2014-08-13 17:23:19.555303 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298315 -9297> 2014-08-13 17:23:19.555461 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298315 -9296> 2014-08-13 17:23:19.556017 7fcd1d4027a0 2 journal read_entry 807632896 : seq 16298316 174777 bytes -9295> 2014-08-13 17:23:19.556032 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298316 -9294> 2014-08-13 17:23:19.574462 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298316 -9293> 2014-08-13 17:23:19.574534 7fcd1d4027a0 2 journal read_entry 807636992 : seq 16298317 1943 bytes -9292> 2014-08-13 17:23:19.574543 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298317 -9291> 2014-08-13 17:23:19.574763 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298317 -9290> 2014-08-13 17:23:19.574789 7fcd1d4027a0 2 journal read_entry 807641088 : seq 16298318 43 bytes -9289> 2014-08-13 17:23:19.574797 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298318 -9288> 2014-08-13 17:23:19.574803 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298318 -9287> 2014-08-13 17:23:19.574818 7fcd1d4027a0 2 journal read_entry 807645184 : seq 16298319 2116 bytes -9286> 2014-08-13 17:23:19.574825 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298319 -9285> 2014-08-13 17:23:19.575005 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298319 -9284> 2014-08-13 17:23:19.575029 7fcd1d4027a0 2 journal read_entry 807649280 : seq 16298320 43 bytes -9283> 2014-08-13 17:23:19.575039 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298320 -9282> 2014-08-13 17:23:19.575044 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298320 -9281> 2014-08-13 17:23:19.575057 7fcd1d4027a0 2 journal read_entry 807653376 : seq 16298321 43 bytes -9280> 2014-08-13 17:23:19.575063 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298321 -9279> 2014-08-13 17:23:19.575066 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298321 -9278> 2014-08-13 17:23:19.575078 7fcd1d4027a0 2 journal read_entry 807657472 : seq 16298322 43 bytes -9277> 2014-08-13 17:23:19.575084 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298322 -9276> 2014-08-13 17:23:19.575087 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298322 -9275> 2014-08-13 17:23:19.575100 7fcd1d4027a0 2 journal read_entry 807661568 : seq 16298323 43 bytes -9274> 2014-08-13 17:23:19.575106 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298323 -9273> 2014-08-13 17:23:19.575109 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298323 -9272> 2014-08-13 17:23:19.575121 7fcd1d4027a0 2 journal read_entry 807665664 : seq 16298324 43 bytes -9271> 2014-08-13 17:23:19.575127 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298324 -9270> 2014-08-13 17:23:19.575130 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298324 -9269> 2014-08-13 17:23:19.576171 7fcd1d4027a0 2 journal read_entry 807862272 : seq 16298325 193436 bytes -9268> 2014-08-13 17:23:19.576185 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298325 -9267> 2014-08-13 17:23:19.593774 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298325 -9266> 2014-08-13 17:23:19.593807 7fcd1d4027a0 2 journal read_entry 807866368 : seq 16298326 1943 bytes -9265> 2014-08-13 17:23:19.593813 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298326 -9264> 2014-08-13 17:23:19.594006 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298326 -9263> 2014-08-13 17:23:19.594033 7fcd1d4027a0 2 journal read_entry 807870464 : seq 16298327 2002 bytes -9262> 2014-08-13 17:23:19.594041 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298327 -9261> 2014-08-13 17:23:19.594253 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298327 -9260> 2014-08-13 17:23:19.594279 7fcd1d4027a0 2 journal read_entry 807874560 : seq 16298328 1943 bytes -9259> 2014-08-13 17:23:19.594283 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298328 -9258> 2014-08-13 17:23:19.594458 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298328 -9257> 2014-08-13 17:23:19.594482 7fcd1d4027a0 2 journal read_entry 807878656 : seq 16298329 2002 bytes -9256> 2014-08-13 17:23:19.594487 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298329 -9255> 2014-08-13 17:23:19.594631 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298329 -9254> 2014-08-13 17:23:19.594652 7fcd1d4027a0 2 journal read_entry 807882752 : seq 16298330 43 bytes -9253> 2014-08-13 17:23:19.594656 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298330 -9252> 2014-08-13 17:23:19.594661 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298330 -9251> 2014-08-13 17:23:19.594671 7fcd1d4027a0 2 journal read_entry 807886848 : seq 16298331 875 bytes -9250> 2014-08-13 17:23:19.594674 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298331 -9249> 2014-08-13 17:23:19.594867 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298331 -9248> 2014-08-13 17:23:19.594891 7fcd1d4027a0 2 journal read_entry 807890944 : seq 16298332 878 bytes -9247> 2014-08-13 17:23:19.594895 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298332 -9246> 2014-08-13 17:23:19.595038 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298332 -9245> 2014-08-13 17:23:19.595060 7fcd1d4027a0 2 journal read_entry 807895040 : seq 16298333 878 bytes -9244> 2014-08-13 17:23:19.595064 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298333 -9243> 2014-08-13 17:23:19.595243 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298333 -9242> 2014-08-13 17:23:19.595270 7fcd1d4027a0 2 journal read_entry 807899136 : seq 16298334 878 bytes -9241> 2014-08-13 17:23:19.595274 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298334 -9240> 2014-08-13 17:23:19.595410 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298334 -9239> 2014-08-13 17:23:19.595432 7fcd1d4027a0 2 journal read_entry 807903232 : seq 16298335 844 bytes -9238> 2014-08-13 17:23:19.595436 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298335 -9237> 2014-08-13 17:23:19.595566 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298335 -9236> 2014-08-13 17:23:19.595588 7fcd1d4027a0 2 journal read_entry 807907328 : seq 16298336 844 bytes -9235> 2014-08-13 17:23:19.595596 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298336 -9234> 2014-08-13 17:23:19.595747 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298336 -9233> 2014-08-13 17:23:19.595771 7fcd1d4027a0 2 journal read_entry 807911424 : seq 16298337 844 bytes -9232> 2014-08-13 17:23:19.595779 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298337 -9231> 2014-08-13 17:23:19.595913 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298337 -9230> 2014-08-13 17:23:19.595935 7fcd1d4027a0 2 journal read_entry 807915520 : seq 16298338 844 bytes -9229> 2014-08-13 17:23:19.595943 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298338 -9228> 2014-08-13 17:23:19.596071 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298338 -9227> 2014-08-13 17:23:19.596093 7fcd1d4027a0 2 journal read_entry 807919616 : seq 16298339 844 bytes -9226> 2014-08-13 17:23:19.596100 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298339 -9225> 2014-08-13 17:23:19.596240 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298339 -9224> 2014-08-13 17:23:19.596262 7fcd1d4027a0 2 journal read_entry 807923712 : seq 16298340 844 bytes -9223> 2014-08-13 17:23:19.596269 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298340 -9222> 2014-08-13 17:23:19.596421 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298340 -9221> 2014-08-13 17:23:19.596444 7fcd1d4027a0 2 journal read_entry 807927808 : seq 16298341 844 bytes -9220> 2014-08-13 17:23:19.596449 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298341 -9219> 2014-08-13 17:23:19.596579 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298341 -9218> 2014-08-13 17:23:19.596600 7fcd1d4027a0 2 journal read_entry 807931904 : seq 16298342 844 bytes -9217> 2014-08-13 17:23:19.596605 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298342 -9216> 2014-08-13 17:23:19.596739 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298342 -9215> 2014-08-13 17:23:19.596760 7fcd1d4027a0 2 journal read_entry 807936000 : seq 16298343 844 bytes -9214> 2014-08-13 17:23:19.596764 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298343 -9213> 2014-08-13 17:23:19.596892 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298343 -9212> 2014-08-13 17:23:19.596913 7fcd1d4027a0 2 journal read_entry 807940096 : seq 16298344 844 bytes -9211> 2014-08-13 17:23:19.596920 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298344 -9210> 2014-08-13 17:23:19.597049 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298344 -9209> 2014-08-13 17:23:19.597070 7fcd1d4027a0 2 journal read_entry 807944192 : seq 16298345 844 bytes -9208> 2014-08-13 17:23:19.597077 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298345 -9207> 2014-08-13 17:23:19.597240 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298345 -9206> 2014-08-13 17:23:19.597264 7fcd1d4027a0 2 journal read_entry 807948288 : seq 16298346 844 bytes -9205> 2014-08-13 17:23:19.597272 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298346 -9204> 2014-08-13 17:23:19.597407 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298346 -9203> 2014-08-13 17:23:19.597429 7fcd1d4027a0 2 journal read_entry 807952384 : seq 16298347 844 bytes -9202> 2014-08-13 17:23:19.597433 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298347 -9201> 2014-08-13 17:23:19.597566 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298347 -9200> 2014-08-13 17:23:19.597592 7fcd1d4027a0 2 journal read_entry 807956480 : seq 16298348 844 bytes -9199> 2014-08-13 17:23:19.597597 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298348 -9198> 2014-08-13 17:23:19.597725 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298348 -9197> 2014-08-13 17:23:19.597746 7fcd1d4027a0 2 journal read_entry 807960576 : seq 16298349 844 bytes -9196> 2014-08-13 17:23:19.597750 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298349 -9195> 2014-08-13 17:23:19.597884 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298349 -9194> 2014-08-13 17:23:19.598252 7fcd1d4027a0 2 journal read_entry 807964672 : seq 16298350 844 bytes -9193> 2014-08-13 17:23:19.598265 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298350 -9192> 2014-08-13 17:23:19.598569 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298350 -9191> 2014-08-13 17:23:19.598593 7fcd1d4027a0 2 journal read_entry 807968768 : seq 16298351 844 bytes -9190> 2014-08-13 17:23:19.598601 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298351 -9189> 2014-08-13 17:23:19.598734 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298351 -9188> 2014-08-13 17:23:19.598757 7fcd1d4027a0 2 journal read_entry 807972864 : seq 16298352 844 bytes -9187> 2014-08-13 17:23:19.598764 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298352 -9186> 2014-08-13 17:23:19.598891 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298352 -9185> 2014-08-13 17:23:19.598913 7fcd1d4027a0 2 journal read_entry 807976960 : seq 16298353 844 bytes -9184> 2014-08-13 17:23:19.598920 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298353 -9183> 2014-08-13 17:23:19.599047 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298353 -9182> 2014-08-13 17:23:19.599069 7fcd1d4027a0 2 journal read_entry 807981056 : seq 16298354 844 bytes -9181> 2014-08-13 17:23:19.599076 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298354 -9180> 2014-08-13 17:23:19.599463 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298354 -9179> 2014-08-13 17:23:19.599491 7fcd1d4027a0 2 journal read_entry 807985152 : seq 16298355 844 bytes -9178> 2014-08-13 17:23:19.599498 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298355 -9177> 2014-08-13 17:23:19.599636 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298355 -9176> 2014-08-13 17:23:19.599659 7fcd1d4027a0 2 journal read_entry 807989248 : seq 16298356 844 bytes -9175> 2014-08-13 17:23:19.599666 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298356 -9174> 2014-08-13 17:23:19.599793 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298356 -9173> 2014-08-13 17:23:19.599814 7fcd1d4027a0 2 journal read_entry 807993344 : seq 16298357 844 bytes -9172> 2014-08-13 17:23:19.599821 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298357 -9171> 2014-08-13 17:23:19.599950 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298357 -9170> 2014-08-13 17:23:19.599971 7fcd1d4027a0 2 journal read_entry 807997440 : seq 16298358 844 bytes -9169> 2014-08-13 17:23:19.599979 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298358 -9168> 2014-08-13 17:23:19.600107 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298358 -9167> 2014-08-13 17:23:19.600128 7fcd1d4027a0 2 journal read_entry 808001536 : seq 16298359 844 bytes -9166> 2014-08-13 17:23:19.600136 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298359 -9165> 2014-08-13 17:23:19.600298 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298359 -9164> 2014-08-13 17:23:19.600322 7fcd1d4027a0 2 journal read_entry 808005632 : seq 16298360 844 bytes -9163> 2014-08-13 17:23:19.600330 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298360 -9162> 2014-08-13 17:23:19.600465 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298360 -9161> 2014-08-13 17:23:19.600488 7fcd1d4027a0 2 journal read_entry 808009728 : seq 16298361 844 bytes -9160> 2014-08-13 17:23:19.600495 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298361 -9159> 2014-08-13 17:23:19.600624 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298361 -9158> 2014-08-13 17:23:19.600646 7fcd1d4027a0 2 journal read_entry 808013824 : seq 16298362 844 bytes -9157> 2014-08-13 17:23:19.600653 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298362 -9156> 2014-08-13 17:23:19.600780 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298362 -9155> 2014-08-13 17:23:19.600802 7fcd1d4027a0 2 journal read_entry 808017920 : seq 16298363 844 bytes -9154> 2014-08-13 17:23:19.600809 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298363 -9153> 2014-08-13 17:23:19.600950 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298363 -9152> 2014-08-13 17:23:19.600973 7fcd1d4027a0 2 journal read_entry 808022016 : seq 16298364 844 bytes -9151> 2014-08-13 17:23:19.600980 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298364 -9150> 2014-08-13 17:23:19.601110 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298364 -9149> 2014-08-13 17:23:19.601137 7fcd1d4027a0 2 journal read_entry 808026112 : seq 16298365 844 bytes -9148> 2014-08-13 17:23:19.601144 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298365 -9147> 2014-08-13 17:23:19.601286 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298365 -9146> 2014-08-13 17:23:19.601308 7fcd1d4027a0 2 journal read_entry 808030208 : seq 16298366 844 bytes -9145> 2014-08-13 17:23:19.601316 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298366 -9144> 2014-08-13 17:23:19.601445 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298366 -9143> 2014-08-13 17:23:19.601466 7fcd1d4027a0 2 journal read_entry 808034304 : seq 16298367 844 bytes -9142> 2014-08-13 17:23:19.601473 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298367 -9141> 2014-08-13 17:23:19.601597 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298367 -9140> 2014-08-13 17:23:19.601618 7fcd1d4027a0 2 journal read_entry 808038400 : seq 16298368 1086 bytes -9139> 2014-08-13 17:23:19.601625 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298368 -9138> 2014-08-13 17:23:19.601846 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298368 -9137> 2014-08-13 17:23:19.601871 7fcd1d4027a0 2 journal read_entry 808042496 : seq 16298369 844 bytes -9136> 2014-08-13 17:23:19.601879 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298369 -9135> 2014-08-13 17:23:19.602011 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298369 -9134> 2014-08-13 17:23:19.602033 7fcd1d4027a0 2 journal read_entry 808046592 : seq 16298370 844 bytes -9133> 2014-08-13 17:23:19.602040 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298370 -9132> 2014-08-13 17:23:19.602176 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298370 -9131> 2014-08-13 17:23:19.602198 7fcd1d4027a0 2 journal read_entry 808050688 : seq 16298371 844 bytes -9130> 2014-08-13 17:23:19.602206 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298371 -9129> 2014-08-13 17:23:19.602336 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298371 -9128> 2014-08-13 17:23:19.602358 7fcd1d4027a0 2 journal read_entry 808054784 : seq 16298372 844 bytes -9127> 2014-08-13 17:23:19.602365 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298372 -9126> 2014-08-13 17:23:19.602511 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298372 -9125> 2014-08-13 17:23:19.602534 7fcd1d4027a0 2 journal read_entry 808058880 : seq 16298373 844 bytes -9124> 2014-08-13 17:23:19.602542 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298373 -9123> 2014-08-13 17:23:19.602672 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298373 -9122> 2014-08-13 17:23:19.602694 7fcd1d4027a0 2 journal read_entry 808062976 : seq 16298374 864 bytes -9121> 2014-08-13 17:23:19.602701 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298374 -9120> 2014-08-13 17:23:19.602836 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298374 -9119> 2014-08-13 17:23:19.602858 7fcd1d4027a0 2 journal read_entry 808067072 : seq 16298375 844 bytes -9118> 2014-08-13 17:23:19.602865 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298375 -9117> 2014-08-13 17:23:19.602989 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298375 -9116> 2014-08-13 17:23:19.603011 7fcd1d4027a0 2 journal read_entry 808071168 : seq 16298376 844 bytes -9115> 2014-08-13 17:23:19.603018 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298376 -9114> 2014-08-13 17:23:19.603156 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298376 -9113> 2014-08-13 17:23:19.603178 7fcd1d4027a0 2 journal read_entry 808075264 : seq 16298377 43 bytes -9112> 2014-08-13 17:23:19.603185 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298377 -9111> 2014-08-13 17:23:19.603190 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298377 -9110> 2014-08-13 17:23:19.603202 7fcd1d4027a0 2 journal read_entry 808079360 : seq 16298378 43 bytes -9109> 2014-08-13 17:23:19.603208 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298378 -9108> 2014-08-13 17:23:19.603212 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298378 -9107> 2014-08-13 17:23:19.603225 7fcd1d4027a0 2 journal read_entry 808083456 : seq 16298379 844 bytes -9106> 2014-08-13 17:23:19.603231 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298379 -9105> 2014-08-13 17:23:19.603384 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298379 -9104> 2014-08-13 17:23:19.603408 7fcd1d4027a0 2 journal read_entry 808087552 : seq 16298380 1086 bytes -9103> 2014-08-13 17:23:19.603416 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298380 -9102> 2014-08-13 17:23:19.603665 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298380 -9101> 2014-08-13 17:23:19.603689 7fcd1d4027a0 2 journal read_entry 808091648 : seq 16298381 864 bytes -9100> 2014-08-13 17:23:19.603696 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298381 -9099> 2014-08-13 17:23:19.603948 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298381 -9098> 2014-08-13 17:23:19.604379 7fcd1d4027a0 2 journal read_entry 808095744 : seq 16298382 844 bytes -9097> 2014-08-13 17:23:19.604393 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298382 -9096> 2014-08-13 17:23:19.604671 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298382 -9095> 2014-08-13 17:23:19.616428 7fcd1d4027a0 2 journal read_entry 808370176 : seq 16298383 269444 bytes -9094> 2014-08-13 17:23:19.616479 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298383 -9093> 2014-08-13 17:23:19.617201 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298383 -9092> 2014-08-13 17:23:19.617230 7fcd1d4027a0 2 journal read_entry 808374272 : seq 16298384 2378 bytes -9091> 2014-08-13 17:23:19.617234 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298384 -9090> 2014-08-13 17:23:19.617416 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298384 -9089> 2014-08-13 17:23:19.617441 7fcd1d4027a0 2 journal read_entry 808378368 : seq 16298385 2316 bytes -9088> 2014-08-13 17:23:19.617445 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298385 -9087> 2014-08-13 17:23:19.617612 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298385 -9086> 2014-08-13 17:23:19.617637 7fcd1d4027a0 2 journal read_entry 808382464 : seq 16298386 3357 bytes -9085> 2014-08-13 17:23:19.617645 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298386 -9084> 2014-08-13 17:23:19.617800 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298386 -9083> 2014-08-13 17:23:19.617823 7fcd1d4027a0 2 journal read_entry 808386560 : seq 16298387 2257 bytes -9082> 2014-08-13 17:23:19.617827 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298387 -9081> 2014-08-13 17:23:19.617979 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298387 -9080> 2014-08-13 17:23:19.618003 7fcd1d4027a0 2 journal read_entry 808390656 : seq 16298388 2316 bytes -9079> 2014-08-13 17:23:19.618007 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298388 -9078> 2014-08-13 17:23:19.618176 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298388 -9077> 2014-08-13 17:23:19.618200 7fcd1d4027a0 2 journal read_entry 808394752 : seq 16298389 2319 bytes -9076> 2014-08-13 17:23:19.618205 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298389 -9075> 2014-08-13 17:23:19.618350 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298389 -9074> 2014-08-13 17:23:19.618373 7fcd1d4027a0 2 journal read_entry 808398848 : seq 16298390 2201 bytes -9073> 2014-08-13 17:23:19.618381 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298390 -9072> 2014-08-13 17:23:19.618546 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298390 -9071> 2014-08-13 17:23:19.618570 7fcd1d4027a0 2 journal read_entry 808402944 : seq 16298391 2201 bytes -9070> 2014-08-13 17:23:19.618578 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298391 -9069> 2014-08-13 17:23:19.618727 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298391 -9068> 2014-08-13 17:23:19.618750 7fcd1d4027a0 2 journal read_entry 808407040 : seq 16298392 2257 bytes -9067> 2014-08-13 17:23:19.618754 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298392 -9066> 2014-08-13 17:23:19.618909 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298392 -9065> 2014-08-13 17:23:19.618931 7fcd1d4027a0 2 journal read_entry 808411136 : seq 16298393 2260 bytes -9064> 2014-08-13 17:23:19.618936 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298393 -9063> 2014-08-13 17:23:19.619122 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298393 -9062> 2014-08-13 17:23:19.619154 7fcd1d4027a0 2 journal read_entry 808415232 : seq 16298394 2260 bytes -9061> 2014-08-13 17:23:19.619159 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298394 -9060> 2014-08-13 17:23:19.619326 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298394 -9059> 2014-08-13 17:23:19.619352 7fcd1d4027a0 2 journal read_entry 808419328 : seq 16298395 3180 bytes -9058> 2014-08-13 17:23:19.619356 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298395 -9057> 2014-08-13 17:23:19.619516 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298395 -9056> 2014-08-13 17:23:19.619540 7fcd1d4027a0 2 journal read_entry 808423424 : seq 16298396 2375 bytes -9055> 2014-08-13 17:23:19.619549 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298396 -9054> 2014-08-13 17:23:19.619691 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298396 -9053> 2014-08-13 17:23:19.619714 7fcd1d4027a0 2 journal read_entry 808427520 : seq 16298397 2555 bytes -9052> 2014-08-13 17:23:19.619718 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298397 -9051> 2014-08-13 17:23:19.619872 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298397 -9050> 2014-08-13 17:23:19.619895 7fcd1d4027a0 2 journal read_entry 808431616 : seq 16298398 2437 bytes -9049> 2014-08-13 17:23:19.619900 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298398 -9048> 2014-08-13 17:23:19.620055 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298398 -9047> 2014-08-13 17:23:19.620078 7fcd1d4027a0 2 journal read_entry 808435712 : seq 16298399 2586 bytes -9046> 2014-08-13 17:23:19.620083 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298399 -9045> 2014-08-13 17:23:19.620285 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298399 -9044> 2014-08-13 17:23:19.620308 7fcd1d4027a0 2 journal read_entry 808439808 : seq 16298400 2260 bytes -9043> 2014-08-13 17:23:19.620312 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298400 -9042> 2014-08-13 17:23:19.620475 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298400 -9041> 2014-08-13 17:23:19.620499 7fcd1d4027a0 2 journal read_entry 808443904 : seq 16298401 876 bytes -9040> 2014-08-13 17:23:19.620506 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298401 -9039> 2014-08-13 17:23:19.620643 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298401 -9038> 2014-08-13 17:23:19.620665 7fcd1d4027a0 2 journal read_entry 808448000 : seq 16298402 878 bytes -9037> 2014-08-13 17:23:19.620673 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298402 -9036> 2014-08-13 17:23:19.620802 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298402 -9035> 2014-08-13 17:23:19.620824 7fcd1d4027a0 2 journal read_entry 808452096 : seq 16298403 878 bytes -9034> 2014-08-13 17:23:19.620831 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298403 -9033> 2014-08-13 17:23:19.620970 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298403 -9032> 2014-08-13 17:23:19.620994 7fcd1d4027a0 2 journal read_entry 808456192 : seq 16298404 878 bytes -9031> 2014-08-13 17:23:19.621001 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298404 -9030> 2014-08-13 17:23:19.621130 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298404 -9029> 2014-08-13 17:23:19.621162 7fcd1d4027a0 2 journal read_entry 808460288 : seq 16298405 876 bytes -9028> 2014-08-13 17:23:19.621171 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298405 -9027> 2014-08-13 17:23:19.621304 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298405 -9026> 2014-08-13 17:23:19.621326 7fcd1d4027a0 2 journal read_entry 808464384 : seq 16298406 878 bytes -9025> 2014-08-13 17:23:19.621333 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298406 -9024> 2014-08-13 17:23:19.621462 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298406 -9023> 2014-08-13 17:23:19.621484 7fcd1d4027a0 2 journal read_entry 808468480 : seq 16298407 878 bytes -9022> 2014-08-13 17:23:19.621491 7fcd1d4027a0 3 journal journal_replay: applying op seq 16298407 -9021> 2014-08-13 17:23:19.621639 7fcd1d4027a0 3 journal journal_replay: r = 0, op_seq now 16298407 -9020> 2014-08-13 17:23:19.621663 7fcd1d4027a0 2 journal No further valid entries found, journal is most likely valid -9019> 2014-08-13 17:23:19.621672 7fcd1d4027a0 3 journal journal_replay: end of journal, done. -9018> 2014-08-13 17:23:19.633027 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -9017> 2014-08-13 17:23:19.645264 7fcd1408e700 1 FileStore::op_tp worker finish -9016> 2014-08-13 17:23:19.645305 7fcd14a8f700 1 FileStore::op_tp worker finish -9015> 2014-08-13 17:23:19.645438 7fcd1d4027a0 1 journal close /var/lib/ceph/osd/ceph-4/journal -9014> 2014-08-13 17:23:19.646988 7fcd1d4027a0 10 monclient(hunting): build_initial_monmap -9013> 2014-08-13 17:23:19.647152 7fcd1d4027a0 5 adding auth protocol: cephx -9012> 2014-08-13 17:23:19.647171 7fcd1d4027a0 5 adding auth protocol: cephx -9011> 2014-08-13 17:23:19.647349 7fcd1d4027a0 1 -- 192.168.100.115:6805/31924 messenger.start -9010> 2014-08-13 17:23:19.647401 7fcd1d4027a0 1 -- :/0 messenger.start -9009> 2014-08-13 17:23:19.647432 7fcd1d4027a0 1 -- 192.168.100.115:6808/31924 messenger.start -9008> 2014-08-13 17:23:19.647454 7fcd1d4027a0 1 -- 192.168.1.115:6820/31924 messenger.start -9007> 2014-08-13 17:23:19.647534 7fcd1d4027a0 1 -- 192.168.1.115:6819/31924 messenger.start -9006> 2014-08-13 17:23:19.647577 7fcd1d4027a0 1 -- 192.168.100.115:6814/31924 messenger.start -9005> 2014-08-13 17:23:19.647671 7fcd1d4027a0 2 osd.4 0 mounting /var/lib/ceph/osd/ceph-4 /var/lib/ceph/osd/ceph-4/journal -9004> 2014-08-13 17:23:19.647765 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) -9003> 2014-08-13 17:23:19.649933 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work -9002> 2014-08-13 17:23:19.649949 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -9001> 2014-08-13 17:23:19.690135 7fcd1d4027a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported -9000> 2014-08-13 17:23:19.690233 7fcd1d4027a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf -8999> 2014-08-13 17:23:19.719527 7fcd1d4027a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf -8998> 2014-08-13 17:23:19.720978 7fcd1d4027a0 2 journal open /var/lib/ceph/osd/ceph-4/journal fsid cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 fs_op_seq 16298407 -8997> 2014-08-13 17:23:19.725047 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -8996> 2014-08-13 17:23:19.725357 7fcd1d4027a0 2 journal read_entry 800899072 : seq 16298033 3724 bytes -8995> 2014-08-13 17:23:19.727576 7fcd1d4027a0 2 journal read_entry 801177600 : seq 16298034 271109 bytes -8994> 2014-08-13 17:23:19.727606 7fcd1d4027a0 2 journal read_entry 801181696 : seq 16298035 43 bytes -8993> 2014-08-13 17:23:19.728680 7fcd1d4027a0 2 journal read_entry 801460224 : seq 16298036 271515 bytes -8992> 2014-08-13 17:23:19.728708 7fcd1d4027a0 2 journal read_entry 801464320 : seq 16298037 3728 bytes -8991> 2014-08-13 17:23:19.728730 7fcd1d4027a0 2 journal read_entry 801468416 : seq 16298038 3090 bytes -8990> 2014-08-13 17:23:19.728754 7fcd1d4027a0 2 journal read_entry 801472512 : seq 16298039 2323 bytes -8989> 2014-08-13 17:23:19.728769 7fcd1d4027a0 2 journal read_entry 801476608 : seq 16298040 43 bytes -8988> 2014-08-13 17:23:19.729843 7fcd1d4027a0 2 journal read_entry 801751040 : seq 16298041 270843 bytes -8987> 2014-08-13 17:23:19.729916 7fcd1d4027a0 2 journal read_entry 801755136 : seq 16298042 3094 bytes -8986> 2014-08-13 17:23:19.729932 7fcd1d4027a0 2 journal read_entry 801759232 : seq 16298043 2382 bytes -8985> 2014-08-13 17:23:19.729954 7fcd1d4027a0 2 journal read_entry 801763328 : seq 16298044 844 bytes -8984> 2014-08-13 17:23:19.731025 7fcd1d4027a0 2 journal read_entry 802037760 : seq 16298045 269896 bytes -8983> 2014-08-13 17:23:19.731052 7fcd1d4027a0 2 journal read_entry 802041856 : seq 16298046 2382 bytes -8982> 2014-08-13 17:23:19.731063 7fcd1d4027a0 2 journal read_entry 802045952 : seq 16298047 876 bytes -8981> 2014-08-13 17:23:19.731080 7fcd1d4027a0 2 journal read_entry 802050048 : seq 16298048 301 bytes -8980> 2014-08-13 17:23:19.731094 7fcd1d4027a0 2 journal read_entry 802054144 : seq 16298049 301 bytes -8979> 2014-08-13 17:23:19.731107 7fcd1d4027a0 2 journal read_entry 802058240 : seq 16298050 305 bytes -8978> 2014-08-13 17:23:19.731120 7fcd1d4027a0 2 journal read_entry 802062336 : seq 16298051 305 bytes -8977> 2014-08-13 17:23:19.731488 7fcd1d4027a0 2 journal read_entry 802066432 : seq 16298052 305 bytes -8976> 2014-08-13 17:23:19.731507 7fcd1d4027a0 2 journal read_entry 802070528 : seq 16298053 298 bytes -8975> 2014-08-13 17:23:19.731649 7fcd1d4027a0 2 journal read_entry 802074624 : seq 16298054 301 bytes -8974> 2014-08-13 17:23:19.731668 7fcd1d4027a0 2 journal read_entry 802078720 : seq 16298055 301 bytes -8973> 2014-08-13 17:23:19.731678 7fcd1d4027a0 2 journal read_entry 802082816 : seq 16298056 301 bytes -8972> 2014-08-13 17:23:19.731692 7fcd1d4027a0 2 journal read_entry 802086912 : seq 16298057 301 bytes -8971> 2014-08-13 17:23:19.731705 7fcd1d4027a0 2 journal read_entry 802091008 : seq 16298058 305 bytes -8970> 2014-08-13 17:23:19.731717 7fcd1d4027a0 2 journal read_entry 802095104 : seq 16298059 305 bytes -8969> 2014-08-13 17:23:19.731730 7fcd1d4027a0 2 journal read_entry 802099200 : seq 16298060 298 bytes -8968> 2014-08-13 17:23:19.731743 7fcd1d4027a0 2 journal read_entry 802103296 : seq 16298061 298 bytes -8967> 2014-08-13 17:23:19.731756 7fcd1d4027a0 2 journal read_entry 802107392 : seq 16298062 305 bytes -8966> 2014-08-13 17:23:19.731769 7fcd1d4027a0 2 journal read_entry 802111488 : seq 16298063 305 bytes -8965> 2014-08-13 17:23:19.731782 7fcd1d4027a0 2 journal read_entry 802115584 : seq 16298064 298 bytes -8964> 2014-08-13 17:23:19.731795 7fcd1d4027a0 2 journal read_entry 802119680 : seq 16298065 301 bytes -8963> 2014-08-13 17:23:19.731814 7fcd1d4027a0 2 journal read_entry 802123776 : seq 16298066 305 bytes -8962> 2014-08-13 17:23:19.731827 7fcd1d4027a0 2 journal read_entry 802127872 : seq 16298067 305 bytes -8961> 2014-08-13 17:23:19.731840 7fcd1d4027a0 2 journal read_entry 802131968 : seq 16298068 298 bytes -8960> 2014-08-13 17:23:19.731852 7fcd1d4027a0 2 journal read_entry 802136064 : seq 16298069 305 bytes -8959> 2014-08-13 17:23:19.731865 7fcd1d4027a0 2 journal read_entry 802140160 : seq 16298070 302 bytes -8958> 2014-08-13 17:23:19.731878 7fcd1d4027a0 2 journal read_entry 802144256 : seq 16298071 305 bytes -8957> 2014-08-13 17:23:19.731891 7fcd1d4027a0 2 journal read_entry 802148352 : seq 16298072 301 bytes -8956> 2014-08-13 17:23:19.731903 7fcd1d4027a0 2 journal read_entry 802152448 : seq 16298073 301 bytes -8955> 2014-08-13 17:23:19.731916 7fcd1d4027a0 2 journal read_entry 802156544 : seq 16298074 305 bytes -8954> 2014-08-13 17:23:19.731928 7fcd1d4027a0 2 journal read_entry 802160640 : seq 16298075 305 bytes -8953> 2014-08-13 17:23:19.731941 7fcd1d4027a0 2 journal read_entry 802164736 : seq 16298076 295 bytes -8952> 2014-08-13 17:23:19.731953 7fcd1d4027a0 2 journal read_entry 802168832 : seq 16298077 305 bytes -8951> 2014-08-13 17:23:19.731966 7fcd1d4027a0 2 journal read_entry 802172928 : seq 16298078 301 bytes -8950> 2014-08-13 17:23:19.731979 7fcd1d4027a0 2 journal read_entry 802177024 : seq 16298079 301 bytes -8949> 2014-08-13 17:23:19.731992 7fcd1d4027a0 2 journal read_entry 802181120 : seq 16298080 301 bytes -8948> 2014-08-13 17:23:19.732005 7fcd1d4027a0 2 journal read_entry 802185216 : seq 16298081 302 bytes -8947> 2014-08-13 17:23:19.732018 7fcd1d4027a0 2 journal read_entry 802189312 : seq 16298082 305 bytes -8946> 2014-08-13 17:23:19.732030 7fcd1d4027a0 2 journal read_entry 802193408 : seq 16298083 301 bytes -8945> 2014-08-13 17:23:19.732410 7fcd1d4027a0 2 journal read_entry 802197504 : seq 16298084 295 bytes -8944> 2014-08-13 17:23:19.732429 7fcd1d4027a0 2 journal read_entry 802201600 : seq 16298085 305 bytes -8943> 2014-08-13 17:23:19.732566 7fcd1d4027a0 2 journal read_entry 802205696 : seq 16298086 298 bytes -8942> 2014-08-13 17:23:19.732586 7fcd1d4027a0 2 journal read_entry 802209792 : seq 16298087 298 bytes -8941> 2014-08-13 17:23:19.732597 7fcd1d4027a0 2 journal read_entry 802213888 : seq 16298088 298 bytes -8940> 2014-08-13 17:23:19.732607 7fcd1d4027a0 2 journal read_entry 802217984 : seq 16298089 305 bytes -8939> 2014-08-13 17:23:19.732617 7fcd1d4027a0 2 journal read_entry 802222080 : seq 16298090 301 bytes -8938> 2014-08-13 17:23:19.732627 7fcd1d4027a0 2 journal read_entry 802226176 : seq 16298091 305 bytes -8937> 2014-08-13 17:23:19.732640 7fcd1d4027a0 2 journal read_entry 802230272 : seq 16298092 301 bytes -8936> 2014-08-13 17:23:19.732653 7fcd1d4027a0 2 journal read_entry 802234368 : seq 16298093 305 bytes -8935> 2014-08-13 17:23:19.732666 7fcd1d4027a0 2 journal read_entry 802238464 : seq 16298094 305 bytes -8934> 2014-08-13 17:23:19.732679 7fcd1d4027a0 2 journal read_entry 802242560 : seq 16298095 301 bytes -8933> 2014-08-13 17:23:19.732691 7fcd1d4027a0 2 journal read_entry 802246656 : seq 16298096 305 bytes -8932> 2014-08-13 17:23:19.732711 7fcd1d4027a0 2 journal read_entry 802250752 : seq 16298097 305 bytes -8931> 2014-08-13 17:23:19.732724 7fcd1d4027a0 2 journal read_entry 802254848 : seq 16298098 302 bytes -8930> 2014-08-13 17:23:19.732734 7fcd1d4027a0 2 journal read_entry 802258944 : seq 16298099 305 bytes -8929> 2014-08-13 17:23:19.732743 7fcd1d4027a0 2 journal read_entry 802263040 : seq 16298100 305 bytes -8928> 2014-08-13 17:23:19.732756 7fcd1d4027a0 2 journal read_entry 802267136 : seq 16298101 301 bytes -8927> 2014-08-13 17:23:19.732766 7fcd1d4027a0 2 journal read_entry 802271232 : seq 16298102 305 bytes -8926> 2014-08-13 17:23:19.732775 7fcd1d4027a0 2 journal read_entry 802275328 : seq 16298103 305 bytes -8925> 2014-08-13 17:23:19.732787 7fcd1d4027a0 2 journal read_entry 802279424 : seq 16298104 305 bytes -8924> 2014-08-13 17:23:19.732800 7fcd1d4027a0 2 journal read_entry 802283520 : seq 16298105 305 bytes -8923> 2014-08-13 17:23:19.732813 7fcd1d4027a0 2 journal read_entry 802287616 : seq 16298106 305 bytes -8922> 2014-08-13 17:23:19.732826 7fcd1d4027a0 2 journal read_entry 802291712 : seq 16298107 301 bytes -8921> 2014-08-13 17:23:19.732839 7fcd1d4027a0 2 journal read_entry 802295808 : seq 16298108 301 bytes -8920> 2014-08-13 17:23:19.732851 7fcd1d4027a0 2 journal read_entry 802299904 : seq 16298109 305 bytes -8919> 2014-08-13 17:23:19.732864 7fcd1d4027a0 2 journal read_entry 802304000 : seq 16298110 305 bytes -8918> 2014-08-13 17:23:19.732876 7fcd1d4027a0 2 journal read_entry 802308096 : seq 16298111 305 bytes -8917> 2014-08-13 17:23:19.732889 7fcd1d4027a0 2 journal read_entry 802312192 : seq 16298112 305 bytes -8916> 2014-08-13 17:23:19.732902 7fcd1d4027a0 2 journal read_entry 802316288 : seq 16298113 301 bytes -8915> 2014-08-13 17:23:19.732915 7fcd1d4027a0 2 journal read_entry 802320384 : seq 16298114 301 bytes -8914> 2014-08-13 17:23:19.732927 7fcd1d4027a0 2 journal read_entry 802324480 : seq 16298115 301 bytes -8913> 2014-08-13 17:23:19.733292 7fcd1d4027a0 2 journal read_entry 802328576 : seq 16298116 305 bytes -8912> 2014-08-13 17:23:19.733472 7fcd1d4027a0 2 journal read_entry 802332672 : seq 16298117 305 bytes -8911> 2014-08-13 17:23:19.733487 7fcd1d4027a0 2 journal read_entry 802336768 : seq 16298118 301 bytes -8910> 2014-08-13 17:23:19.733500 7fcd1d4027a0 2 journal read_entry 802340864 : seq 16298119 301 bytes -8909> 2014-08-13 17:23:19.751642 7fcd1d4027a0 2 journal read_entry 806387712 : seq 16298120 4040766 bytes -8908> 2014-08-13 17:23:19.752200 7fcd1d4027a0 2 journal read_entry 806391808 : seq 16298121 1943 bytes -8907> 2014-08-13 17:23:19.752227 7fcd1d4027a0 2 journal read_entry 806395904 : seq 16298122 2053 bytes -8906> 2014-08-13 17:23:19.752239 7fcd1d4027a0 2 journal read_entry 806400000 : seq 16298123 2378 bytes -8905> 2014-08-13 17:23:19.752260 7fcd1d4027a0 2 journal read_entry 806404096 : seq 16298124 1009 bytes -8904> 2014-08-13 17:23:19.752275 7fcd1d4027a0 2 journal read_entry 806408192 : seq 16298125 2053 bytes -8903> 2014-08-13 17:23:19.752286 7fcd1d4027a0 2 journal read_entry 806412288 : seq 16298126 3944 bytes -8902> 2014-08-13 17:23:19.752301 7fcd1d4027a0 2 journal read_entry 806416384 : seq 16298127 2315 bytes -8901> 2014-08-13 17:23:19.752315 7fcd1d4027a0 2 journal read_entry 806420480 : seq 16298128 2201 bytes -8900> 2014-08-13 17:23:19.752325 7fcd1d4027a0 2 journal read_entry 806424576 : seq 16298129 2116 bytes -8899> 2014-08-13 17:23:19.752340 7fcd1d4027a0 2 journal read_entry 806428672 : seq 16298130 1009 bytes -8898> 2014-08-13 17:23:19.752355 7fcd1d4027a0 2 journal read_entry 806432768 : seq 16298131 2167 bytes -8897> 2014-08-13 17:23:19.752366 7fcd1d4027a0 2 journal read_entry 806436864 : seq 16298132 1943 bytes -8896> 2014-08-13 17:23:19.752381 7fcd1d4027a0 2 journal read_entry 806440960 : seq 16298133 3306 bytes -8895> 2014-08-13 17:23:19.752396 7fcd1d4027a0 2 journal read_entry 806445056 : seq 16298134 1943 bytes -8894> 2014-08-13 17:23:19.752406 7fcd1d4027a0 2 journal read_entry 806449152 : seq 16298135 1936 bytes -8893> 2014-08-13 17:23:19.752417 7fcd1d4027a0 2 journal read_entry 806453248 : seq 16298136 2053 bytes -8892> 2014-08-13 17:23:19.752443 7fcd1d4027a0 2 journal read_entry 806457344 : seq 16298137 1044 bytes -8891> 2014-08-13 17:23:19.752454 7fcd1d4027a0 2 journal read_entry 806461440 : seq 16298138 2167 bytes -8890> 2014-08-13 17:23:19.752468 7fcd1d4027a0 2 journal read_entry 806465536 : seq 16298139 1943 bytes -8889> 2014-08-13 17:23:19.752479 7fcd1d4027a0 2 journal read_entry 806469632 : seq 16298140 2194 bytes -8888> 2014-08-13 17:23:19.752493 7fcd1d4027a0 2 journal read_entry 806473728 : seq 16298141 2053 bytes -8887> 2014-08-13 17:23:19.752506 7fcd1d4027a0 2 journal read_entry 806477824 : seq 16298142 2116 bytes -8886> 2014-08-13 17:23:19.752517 7fcd1d4027a0 2 journal read_entry 806481920 : seq 16298143 2053 bytes -8885> 2014-08-13 17:23:19.752527 7fcd1d4027a0 2 journal read_entry 806486016 : seq 16298144 1936 bytes -8884> 2014-08-13 17:23:19.752541 7fcd1d4027a0 2 journal read_entry 806490112 : seq 16298145 1943 bytes -8883> 2014-08-13 17:23:19.752551 7fcd1d4027a0 2 journal read_entry 806494208 : seq 16298146 2002 bytes -8882> 2014-08-13 17:23:19.752571 7fcd1d4027a0 2 journal read_entry 806498304 : seq 16298147 2641 bytes -8881> 2014-08-13 17:23:19.752586 7fcd1d4027a0 2 journal read_entry 806502400 : seq 16298148 2253 bytes -8880> 2014-08-13 17:23:19.752600 7fcd1d4027a0 2 journal read_entry 806506496 : seq 16298149 2496 bytes -8879> 2014-08-13 17:23:19.752613 7fcd1d4027a0 2 journal read_entry 806510592 : seq 16298150 2116 bytes -8878> 2014-08-13 17:23:19.752627 7fcd1d4027a0 2 journal read_entry 806514688 : seq 16298151 2496 bytes -8877> 2014-08-13 17:23:19.752641 7fcd1d4027a0 2 journal read_entry 806518784 : seq 16298152 2167 bytes -8876> 2014-08-13 17:23:19.752995 7fcd1d4027a0 2 journal read_entry 806522880 : seq 16298153 1943 bytes -8875> 2014-08-13 17:23:19.753015 7fcd1d4027a0 2 journal read_entry 806526976 : seq 16298154 1943 bytes -8874> 2014-08-13 17:23:19.753168 7fcd1d4027a0 2 journal read_entry 806531072 : seq 16298155 1943 bytes -8873> 2014-08-13 17:23:19.753187 7fcd1d4027a0 2 journal read_entry 806535168 : seq 16298156 2316 bytes -8872> 2014-08-13 17:23:19.753198 7fcd1d4027a0 2 journal read_entry 806539264 : seq 16298157 1047 bytes -8871> 2014-08-13 17:23:19.753208 7fcd1d4027a0 2 journal read_entry 806543360 : seq 16298158 1943 bytes -8870> 2014-08-13 17:23:19.753222 7fcd1d4027a0 2 journal read_entry 806547456 : seq 16298159 2421 bytes -8869> 2014-08-13 17:23:19.753236 7fcd1d4027a0 2 journal read_entry 806551552 : seq 16298160 2167 bytes -8868> 2014-08-13 17:23:19.753250 7fcd1d4027a0 2 journal read_entry 806555648 : seq 16298161 2319 bytes -8867> 2014-08-13 17:23:19.753264 7fcd1d4027a0 2 journal read_entry 806559744 : seq 16298162 2527 bytes -8866> 2014-08-13 17:23:19.753278 7fcd1d4027a0 2 journal read_entry 806563840 : seq 16298163 2675 bytes -8865> 2014-08-13 17:23:19.753292 7fcd1d4027a0 2 journal read_entry 806567936 : seq 16298164 1943 bytes -8864> 2014-08-13 17:23:19.753305 7fcd1d4027a0 2 journal read_entry 806572032 : seq 16298165 2201 bytes -8863> 2014-08-13 17:23:19.753319 7fcd1d4027a0 2 journal read_entry 806576128 : seq 16298166 2167 bytes -8862> 2014-08-13 17:23:19.753333 7fcd1d4027a0 2 journal read_entry 806580224 : seq 16298167 1943 bytes -8861> 2014-08-13 17:23:19.753354 7fcd1d4027a0 2 journal read_entry 806584320 : seq 16298168 2909 bytes -8860> 2014-08-13 17:23:19.753368 7fcd1d4027a0 2 journal read_entry 806588416 : seq 16298169 2167 bytes -8859> 2014-08-13 17:23:19.753382 7fcd1d4027a0 2 journal read_entry 806592512 : seq 16298170 2167 bytes -8858> 2014-08-13 17:23:19.753397 7fcd1d4027a0 2 journal read_entry 806596608 : seq 16298171 3122 bytes -8857> 2014-08-13 17:23:19.753411 7fcd1d4027a0 2 journal read_entry 806600704 : seq 16298172 1943 bytes -8856> 2014-08-13 17:23:19.753424 7fcd1d4027a0 2 journal read_entry 806604800 : seq 16298173 2053 bytes -8855> 2014-08-13 17:23:19.753438 7fcd1d4027a0 2 journal read_entry 806608896 : seq 16298174 2253 bytes -8854> 2014-08-13 17:23:19.753452 7fcd1d4027a0 2 journal read_entry 806612992 : seq 16298175 1943 bytes -8853> 2014-08-13 17:23:19.753466 7fcd1d4027a0 2 journal read_entry 806617088 : seq 16298176 1943 bytes -8852> 2014-08-13 17:23:19.753480 7fcd1d4027a0 2 journal read_entry 806621184 : seq 16298177 2201 bytes -8851> 2014-08-13 17:23:19.753494 7fcd1d4027a0 2 journal read_entry 806625280 : seq 16298178 2201 bytes -8850> 2014-08-13 17:23:19.753507 7fcd1d4027a0 2 journal read_entry 806629376 : seq 16298179 2256 bytes -8849> 2014-08-13 17:23:19.753521 7fcd1d4027a0 2 journal read_entry 806633472 : seq 16298180 2138 bytes -8848> 2014-08-13 17:23:19.753535 7fcd1d4027a0 2 journal read_entry 806637568 : seq 16298181 2138 bytes -8847> 2014-08-13 17:23:19.753548 7fcd1d4027a0 2 journal read_entry 806641664 : seq 16298182 1047 bytes -8846> 2014-08-13 17:23:19.753562 7fcd1d4027a0 2 journal read_entry 806645760 : seq 16298183 1943 bytes -8845> 2014-08-13 17:23:19.753576 7fcd1d4027a0 2 journal read_entry 806649856 : seq 16298184 2002 bytes -8844> 2014-08-13 17:23:19.753954 7fcd1d4027a0 2 journal read_entry 806653952 : seq 16298185 2198 bytes -8843> 2014-08-13 17:23:19.753974 7fcd1d4027a0 2 journal read_entry 806658048 : seq 16298186 1047 bytes -8842> 2014-08-13 17:23:19.754119 7fcd1d4027a0 2 journal read_entry 806662144 : seq 16298187 2116 bytes -8841> 2014-08-13 17:23:19.754138 7fcd1d4027a0 2 journal read_entry 806666240 : seq 16298188 1943 bytes -8840> 2014-08-13 17:23:19.754161 7fcd1d4027a0 2 journal read_entry 806670336 : seq 16298189 844 bytes -8839> 2014-08-13 17:23:19.754176 7fcd1d4027a0 2 journal read_entry 806674432 : seq 16298190 3085 bytes -8838> 2014-08-13 17:23:19.754190 7fcd1d4027a0 2 journal read_entry 806678528 : seq 16298191 844 bytes -8837> 2014-08-13 17:23:19.754204 7fcd1d4027a0 2 journal read_entry 806682624 : seq 16298192 844 bytes -8836> 2014-08-13 17:23:19.754224 7fcd1d4027a0 2 journal read_entry 806686720 : seq 16298193 844 bytes -8835> 2014-08-13 17:23:19.754238 7fcd1d4027a0 2 journal read_entry 806690816 : seq 16298194 844 bytes -8834> 2014-08-13 17:23:19.754251 7fcd1d4027a0 2 journal read_entry 806694912 : seq 16298195 876 bytes -8833> 2014-08-13 17:23:19.754264 7fcd1d4027a0 2 journal read_entry 806699008 : seq 16298196 844 bytes -8832> 2014-08-13 17:23:19.754278 7fcd1d4027a0 2 journal read_entry 806703104 : seq 16298197 878 bytes -8831> 2014-08-13 17:23:19.754292 7fcd1d4027a0 2 journal read_entry 806707200 : seq 16298198 3121 bytes -8830> 2014-08-13 17:23:19.754306 7fcd1d4027a0 2 journal read_entry 806711296 : seq 16298199 842 bytes -8829> 2014-08-13 17:23:19.754319 7fcd1d4027a0 2 journal read_entry 806715392 : seq 16298200 844 bytes -8828> 2014-08-13 17:23:19.754332 7fcd1d4027a0 2 journal read_entry 806719488 : seq 16298201 844 bytes -8827> 2014-08-13 17:23:19.754345 7fcd1d4027a0 2 journal read_entry 806723584 : seq 16298202 844 bytes -8826> 2014-08-13 17:23:19.754359 7fcd1d4027a0 2 journal read_entry 806727680 : seq 16298203 2002 bytes -8825> 2014-08-13 17:23:19.754372 7fcd1d4027a0 2 journal read_entry 806731776 : seq 16298204 878 bytes -8824> 2014-08-13 17:23:19.754385 7fcd1d4027a0 2 journal read_entry 806735872 : seq 16298205 844 bytes -8823> 2014-08-13 17:23:19.754398 7fcd1d4027a0 2 journal read_entry 806739968 : seq 16298206 844 bytes -8822> 2014-08-13 17:23:19.754411 7fcd1d4027a0 2 journal read_entry 806744064 : seq 16298207 43 bytes -8821> 2014-08-13 17:23:19.754424 7fcd1d4027a0 2 journal read_entry 806748160 : seq 16298208 2201 bytes -8820> 2014-08-13 17:23:19.754438 7fcd1d4027a0 2 journal read_entry 806752256 : seq 16298209 878 bytes -8819> 2014-08-13 17:23:19.754451 7fcd1d4027a0 2 journal read_entry 806756352 : seq 16298210 844 bytes -8818> 2014-08-13 17:23:19.754464 7fcd1d4027a0 2 journal read_entry 806760448 : seq 16298211 842 bytes -8817> 2014-08-13 17:23:19.754477 7fcd1d4027a0 2 journal read_entry 806764544 : seq 16298212 844 bytes -8816> 2014-08-13 17:23:19.754490 7fcd1d4027a0 2 journal read_entry 806768640 : seq 16298213 844 bytes -8815> 2014-08-13 17:23:19.754503 7fcd1d4027a0 2 journal read_entry 806772736 : seq 16298214 876 bytes -8814> 2014-08-13 17:23:19.754538 7fcd1d4027a0 2 journal read_entry 806776832 : seq 16298215 832 bytes -8813> 2014-08-13 17:23:19.754552 7fcd1d4027a0 2 journal read_entry 806780928 : seq 16298216 876 bytes -8812> 2014-08-13 17:23:19.754975 7fcd1d4027a0 2 journal read_entry 806785024 : seq 16298217 842 bytes -8811> 2014-08-13 17:23:19.755178 7fcd1d4027a0 2 journal read_entry 806789120 : seq 16298218 844 bytes -8810> 2014-08-13 17:23:19.755200 7fcd1d4027a0 2 journal read_entry 806793216 : seq 16298219 844 bytes -8809> 2014-08-13 17:23:19.755210 7fcd1d4027a0 2 journal read_entry 806797312 : seq 16298220 844 bytes -8808> 2014-08-13 17:23:19.755220 7fcd1d4027a0 2 journal read_entry 806801408 : seq 16298221 43 bytes -8807> 2014-08-13 17:23:19.755230 7fcd1d4027a0 2 journal read_entry 806805504 : seq 16298222 844 bytes -8806> 2014-08-13 17:23:19.755240 7fcd1d4027a0 2 journal read_entry 806809600 : seq 16298223 844 bytes -8805> 2014-08-13 17:23:19.755255 7fcd1d4027a0 2 journal read_entry 806813696 : seq 16298224 878 bytes -8804> 2014-08-13 17:23:19.755269 7fcd1d4027a0 2 journal read_entry 806817792 : seq 16298225 43 bytes -8803> 2014-08-13 17:23:19.755282 7fcd1d4027a0 2 journal read_entry 806821888 : seq 16298226 844 bytes -8802> 2014-08-13 17:23:19.755295 7fcd1d4027a0 2 journal read_entry 806825984 : seq 16298227 43 bytes -8801> 2014-08-13 17:23:19.755304 7fcd1d4027a0 2 journal read_entry 806830080 : seq 16298228 43 bytes -8800> 2014-08-13 17:23:19.755314 7fcd1d4027a0 2 journal read_entry 806834176 : seq 16298229 43 bytes -8799> 2014-08-13 17:23:19.755326 7fcd1d4027a0 2 journal read_entry 806838272 : seq 16298230 43 bytes -8798> 2014-08-13 17:23:19.755339 7fcd1d4027a0 2 journal read_entry 806842368 : seq 16298231 832 bytes -8797> 2014-08-13 17:23:19.755352 7fcd1d4027a0 2 journal read_entry 806846464 : seq 16298232 43 bytes -8796> 2014-08-13 17:23:19.755364 7fcd1d4027a0 2 journal read_entry 806850560 : seq 16298233 43 bytes -8795> 2014-08-13 17:23:19.755377 7fcd1d4027a0 2 journal read_entry 806854656 : seq 16298234 43 bytes -8794> 2014-08-13 17:23:19.755389 7fcd1d4027a0 2 journal read_entry 806858752 : seq 16298235 43 bytes -8793> 2014-08-13 17:23:19.755402 7fcd1d4027a0 2 journal read_entry 806862848 : seq 16298236 43 bytes -8792> 2014-08-13 17:23:19.755414 7fcd1d4027a0 2 journal read_entry 806866944 : seq 16298237 43 bytes -8791> 2014-08-13 17:23:19.755427 7fcd1d4027a0 2 journal read_entry 806871040 : seq 16298238 43 bytes -8790> 2014-08-13 17:23:19.755439 7fcd1d4027a0 2 journal read_entry 806875136 : seq 16298239 43 bytes -8789> 2014-08-13 17:23:19.755452 7fcd1d4027a0 2 journal read_entry 806879232 : seq 16298240 43 bytes -8788> 2014-08-13 17:23:19.755464 7fcd1d4027a0 2 journal read_entry 806883328 : seq 16298241 43 bytes -8787> 2014-08-13 17:23:19.755477 7fcd1d4027a0 2 journal read_entry 806887424 : seq 16298242 43 bytes -8786> 2014-08-13 17:23:19.755490 7fcd1d4027a0 2 journal read_entry 806891520 : seq 16298243 43 bytes -8785> 2014-08-13 17:23:19.755502 7fcd1d4027a0 2 journal read_entry 806895616 : seq 16298244 43 bytes -8784> 2014-08-13 17:23:19.755515 7fcd1d4027a0 2 journal read_entry 806899712 : seq 16298245 43 bytes -8783> 2014-08-13 17:23:19.755528 7fcd1d4027a0 2 journal read_entry 806903808 : seq 16298246 43 bytes -8782> 2014-08-13 17:23:19.755540 7fcd1d4027a0 2 journal read_entry 806907904 : seq 16298247 43 bytes -8781> 2014-08-13 17:23:19.757086 7fcd1d4027a0 2 journal read_entry 807182336 : seq 16298248 270312 bytes -8780> 2014-08-13 17:23:19.757113 7fcd1d4027a0 2 journal read_entry 807186432 : seq 16298249 2374 bytes -8779> 2014-08-13 17:23:19.757127 7fcd1d4027a0 2 journal read_entry 807190528 : seq 16298250 2171 bytes -8778> 2014-08-13 17:23:19.757141 7fcd1d4027a0 2 journal read_entry 807194624 : seq 16298251 1940 bytes -8777> 2014-08-13 17:23:19.757164 7fcd1d4027a0 2 journal read_entry 807198720 : seq 16298252 1048 bytes -8776> 2014-08-13 17:23:19.757176 7fcd1d4027a0 2 journal read_entry 807202816 : seq 16298253 2057 bytes -8775> 2014-08-13 17:23:19.757186 7fcd1d4027a0 2 journal read_entry 807206912 : seq 16298254 2057 bytes -8774> 2014-08-13 17:23:19.757196 7fcd1d4027a0 2 journal read_entry 807211008 : seq 16298255 2171 bytes -8773> 2014-08-13 17:23:19.757210 7fcd1d4027a0 2 journal read_entry 807215104 : seq 16298256 2057 bytes -8772> 2014-08-13 17:23:19.757221 7fcd1d4027a0 2 journal read_entry 807219200 : seq 16298257 2253 bytes -8771> 2014-08-13 17:23:19.757235 7fcd1d4027a0 2 journal read_entry 807223296 : seq 16298258 2555 bytes -8770> 2014-08-13 17:23:19.757249 7fcd1d4027a0 2 journal read_entry 807227392 : seq 16298259 1940 bytes -8769> 2014-08-13 17:23:19.757263 7fcd1d4027a0 2 journal read_entry 807231488 : seq 16298260 2312 bytes -8768> 2014-08-13 17:23:19.757276 7fcd1d4027a0 2 journal read_entry 807235584 : seq 16298261 2171 bytes -8767> 2014-08-13 17:23:19.757290 7fcd1d4027a0 2 journal read_entry 807239680 : seq 16298262 2375 bytes -8766> 2014-08-13 17:23:19.757304 7fcd1d4027a0 2 journal read_entry 807243776 : seq 16298263 2057 bytes -8765> 2014-08-13 17:23:19.757324 7fcd1d4027a0 2 journal read_entry 807247872 : seq 16298264 1358 bytes -8764> 2014-08-13 17:23:19.757339 7fcd1d4027a0 2 journal read_entry 807251968 : seq 16298265 2437 bytes -8763> 2014-08-13 17:23:19.757353 7fcd1d4027a0 2 journal read_entry 807256064 : seq 16298266 2171 bytes -8762> 2014-08-13 17:23:19.757367 7fcd1d4027a0 2 journal read_entry 807260160 : seq 16298267 2586 bytes -8761> 2014-08-13 17:23:19.757381 7fcd1d4027a0 2 journal read_entry 807264256 : seq 16298268 2171 bytes -8760> 2014-08-13 17:23:19.757395 7fcd1d4027a0 2 journal read_entry 807268352 : seq 16298269 1013 bytes -8759> 2014-08-13 17:23:19.757409 7fcd1d4027a0 2 journal read_entry 807272448 : seq 16298270 2968 bytes -8758> 2014-08-13 17:23:19.757423 7fcd1d4027a0 2 journal read_entry 807276544 : seq 16298271 2057 bytes -8757> 2014-08-13 17:23:19.757437 7fcd1d4027a0 2 journal read_entry 807280640 : seq 16298272 1358 bytes -8756> 2014-08-13 17:23:19.757451 7fcd1d4027a0 2 journal read_entry 807284736 : seq 16298273 2171 bytes -8755> 2014-08-13 17:23:19.757489 7fcd1d4027a0 2 journal read_entry 807288832 : seq 16298274 2260 bytes -8754> 2014-08-13 17:23:19.757503 7fcd1d4027a0 2 journal read_entry 807292928 : seq 16298275 2312 bytes -8753> 2014-08-13 17:23:19.757517 7fcd1d4027a0 2 journal read_entry 807297024 : seq 16298276 2260 bytes -8752> 2014-08-13 17:23:19.757531 7fcd1d4027a0 2 journal read_entry 807301120 : seq 16298277 2315 bytes -8751> 2014-08-13 17:23:19.757545 7fcd1d4027a0 2 journal read_entry 807305216 : seq 16298278 2197 bytes -8750> 2014-08-13 17:23:19.757906 7fcd1d4027a0 2 journal read_entry 807309312 : seq 16298279 2555 bytes -8749> 2014-08-13 17:23:19.757926 7fcd1d4027a0 2 journal read_entry 807313408 : seq 16298280 1013 bytes -8748> 2014-08-13 17:23:19.758071 7fcd1d4027a0 2 journal read_entry 807317504 : seq 16298281 1051 bytes -8747> 2014-08-13 17:23:19.758089 7fcd1d4027a0 2 journal read_entry 807321600 : seq 16298282 2476 bytes -8746> 2014-08-13 17:23:19.758104 7fcd1d4027a0 2 journal read_entry 807325696 : seq 16298283 3353 bytes -8745> 2014-08-13 17:23:19.758118 7fcd1d4027a0 2 journal read_entry 807329792 : seq 16298284 2378 bytes -8744> 2014-08-13 17:23:19.758133 7fcd1d4027a0 2 journal read_entry 807333888 : seq 16298285 2734 bytes -8743> 2014-08-13 17:23:19.758155 7fcd1d4027a0 2 journal read_entry 807337984 : seq 16298286 2260 bytes -8742> 2014-08-13 17:23:19.758170 7fcd1d4027a0 2 journal read_entry 807342080 : seq 16298287 1051 bytes -8741> 2014-08-13 17:23:19.758184 7fcd1d4027a0 2 journal read_entry 807346176 : seq 16298288 1051 bytes -8740> 2014-08-13 17:23:19.758198 7fcd1d4027a0 2 journal read_entry 807350272 : seq 16298289 2257 bytes -8739> 2014-08-13 17:23:19.758213 7fcd1d4027a0 2 journal read_entry 807354368 : seq 16298290 3089 bytes -8738> 2014-08-13 17:23:19.758227 7fcd1d4027a0 2 journal read_entry 807358464 : seq 16298291 2171 bytes -8737> 2014-08-13 17:23:19.758242 7fcd1d4027a0 2 journal read_entry 807362560 : seq 16298292 3180 bytes -8736> 2014-08-13 17:23:19.758256 7fcd1d4027a0 2 journal read_entry 807366656 : seq 16298293 2260 bytes -8735> 2014-08-13 17:23:19.758271 7fcd1d4027a0 2 journal read_entry 807370752 : seq 16298294 3126 bytes -8734> 2014-08-13 17:23:19.758285 7fcd1d4027a0 2 journal read_entry 807374848 : seq 16298295 2002 bytes -8733> 2014-08-13 17:23:19.758299 7fcd1d4027a0 2 journal read_entry 807378944 : seq 16298296 2116 bytes -8732> 2014-08-13 17:23:19.758313 7fcd1d4027a0 2 journal read_entry 807383040 : seq 16298297 2057 bytes -8731> 2014-08-13 17:23:19.758327 7fcd1d4027a0 2 journal read_entry 807387136 : seq 16298298 1943 bytes -8730> 2014-08-13 17:23:19.758341 7fcd1d4027a0 2 journal read_entry 807391232 : seq 16298299 1943 bytes -8729> 2014-08-13 17:23:19.758355 7fcd1d4027a0 2 journal read_entry 807395328 : seq 16298300 2260 bytes -8728> 2014-08-13 17:23:19.758369 7fcd1d4027a0 2 journal read_entry 807399424 : seq 16298301 2197 bytes -8727> 2014-08-13 17:23:19.758383 7fcd1d4027a0 2 journal read_entry 807403520 : seq 16298302 1943 bytes -8726> 2014-08-13 17:23:19.758397 7fcd1d4027a0 2 journal read_entry 807407616 : seq 16298303 1943 bytes -8725> 2014-08-13 17:23:19.758411 7fcd1d4027a0 2 journal read_entry 807411712 : seq 16298304 868 bytes -8724> 2014-08-13 17:23:19.758425 7fcd1d4027a0 2 journal read_entry 807415808 : seq 16298305 1943 bytes -8723> 2014-08-13 17:23:19.758440 7fcd1d4027a0 2 journal read_entry 807419904 : seq 16298306 1943 bytes -8722> 2014-08-13 17:23:19.758453 7fcd1d4027a0 2 journal read_entry 807424000 : seq 16298307 1943 bytes -8721> 2014-08-13 17:23:19.758467 7fcd1d4027a0 2 journal read_entry 807428096 : seq 16298308 1943 bytes -8720> 2014-08-13 17:23:19.758481 7fcd1d4027a0 2 journal read_entry 807432192 : seq 16298309 1943 bytes -8719> 2014-08-13 17:23:19.758495 7fcd1d4027a0 2 journal read_entry 807436288 : seq 16298310 1943 bytes -8718> 2014-08-13 17:23:19.758859 7fcd1d4027a0 2 journal read_entry 807440384 : seq 16298311 1943 bytes -8717> 2014-08-13 17:23:19.758879 7fcd1d4027a0 2 journal read_entry 807444480 : seq 16298312 1943 bytes -8716> 2014-08-13 17:23:19.759024 7fcd1d4027a0 2 journal read_entry 807448576 : seq 16298313 2116 bytes -8715> 2014-08-13 17:23:19.759041 7fcd1d4027a0 2 journal read_entry 807452672 : seq 16298314 2116 bytes -8714> 2014-08-13 17:23:19.759055 7fcd1d4027a0 2 journal read_entry 807456768 : seq 16298315 1943 bytes -8713> 2014-08-13 17:23:19.759632 7fcd1d4027a0 2 journal read_entry 807632896 : seq 16298316 174777 bytes -8712> 2014-08-13 17:23:19.759657 7fcd1d4027a0 2 journal read_entry 807636992 : seq 16298317 1943 bytes -8711> 2014-08-13 17:23:19.759671 7fcd1d4027a0 2 journal read_entry 807641088 : seq 16298318 43 bytes -8710> 2014-08-13 17:23:19.759685 7fcd1d4027a0 2 journal read_entry 807645184 : seq 16298319 2116 bytes -8709> 2014-08-13 17:23:19.759698 7fcd1d4027a0 2 journal read_entry 807649280 : seq 16298320 43 bytes -8708> 2014-08-13 17:23:19.759712 7fcd1d4027a0 2 journal read_entry 807653376 : seq 16298321 43 bytes -8707> 2014-08-13 17:23:19.759725 7fcd1d4027a0 2 journal read_entry 807657472 : seq 16298322 43 bytes -8706> 2014-08-13 17:23:19.759738 7fcd1d4027a0 2 journal read_entry 807661568 : seq 16298323 43 bytes -8705> 2014-08-13 17:23:19.759751 7fcd1d4027a0 2 journal read_entry 807665664 : seq 16298324 43 bytes -8704> 2014-08-13 17:23:19.760814 7fcd1d4027a0 2 journal read_entry 807862272 : seq 16298325 193436 bytes -8703> 2014-08-13 17:23:19.760838 7fcd1d4027a0 2 journal read_entry 807866368 : seq 16298326 1943 bytes -8702> 2014-08-13 17:23:19.760853 7fcd1d4027a0 2 journal read_entry 807870464 : seq 16298327 2002 bytes -8701> 2014-08-13 17:23:19.760868 7fcd1d4027a0 2 journal read_entry 807874560 : seq 16298328 1943 bytes -8700> 2014-08-13 17:23:19.760882 7fcd1d4027a0 2 journal read_entry 807878656 : seq 16298329 2002 bytes -8699> 2014-08-13 17:23:19.760895 7fcd1d4027a0 2 journal read_entry 807882752 : seq 16298330 43 bytes -8698> 2014-08-13 17:23:19.760909 7fcd1d4027a0 2 journal read_entry 807886848 : seq 16298331 875 bytes -8697> 2014-08-13 17:23:19.760923 7fcd1d4027a0 2 journal read_entry 807890944 : seq 16298332 878 bytes -8696> 2014-08-13 17:23:19.760936 7fcd1d4027a0 2 journal read_entry 807895040 : seq 16298333 878 bytes -8695> 2014-08-13 17:23:19.760949 7fcd1d4027a0 2 journal read_entry 807899136 : seq 16298334 878 bytes -8694> 2014-08-13 17:23:19.760963 7fcd1d4027a0 2 journal read_entry 807903232 : seq 16298335 844 bytes -8693> 2014-08-13 17:23:19.760976 7fcd1d4027a0 2 journal read_entry 807907328 : seq 16298336 844 bytes -8692> 2014-08-13 17:23:19.760989 7fcd1d4027a0 2 journal read_entry 807911424 : seq 16298337 844 bytes -8691> 2014-08-13 17:23:19.761002 7fcd1d4027a0 2 journal read_entry 807915520 : seq 16298338 844 bytes -8690> 2014-08-13 17:23:19.761016 7fcd1d4027a0 2 journal read_entry 807919616 : seq 16298339 844 bytes -8689> 2014-08-13 17:23:19.761029 7fcd1d4027a0 2 journal read_entry 807923712 : seq 16298340 844 bytes -8688> 2014-08-13 17:23:19.761042 7fcd1d4027a0 2 journal read_entry 807927808 : seq 16298341 844 bytes -8687> 2014-08-13 17:23:19.761055 7fcd1d4027a0 2 journal read_entry 807931904 : seq 16298342 844 bytes -8686> 2014-08-13 17:23:19.761068 7fcd1d4027a0 2 journal read_entry 807936000 : seq 16298343 844 bytes -8685> 2014-08-13 17:23:19.761081 7fcd1d4027a0 2 journal read_entry 807940096 : seq 16298344 844 bytes -8684> 2014-08-13 17:23:19.761094 7fcd1d4027a0 2 journal read_entry 807944192 : seq 16298345 844 bytes -8683> 2014-08-13 17:23:19.761108 7fcd1d4027a0 2 journal read_entry 807948288 : seq 16298346 844 bytes -8682> 2014-08-13 17:23:19.761121 7fcd1d4027a0 2 journal read_entry 807952384 : seq 16298347 844 bytes -8681> 2014-08-13 17:23:19.761135 7fcd1d4027a0 2 journal read_entry 807956480 : seq 16298348 844 bytes -8680> 2014-08-13 17:23:19.761157 7fcd1d4027a0 2 journal read_entry 807960576 : seq 16298349 844 bytes -8679> 2014-08-13 17:23:19.761525 7fcd1d4027a0 2 journal read_entry 807964672 : seq 16298350 844 bytes -8678> 2014-08-13 17:23:19.761544 7fcd1d4027a0 2 journal read_entry 807968768 : seq 16298351 844 bytes -8677> 2014-08-13 17:23:19.761690 7fcd1d4027a0 2 journal read_entry 807972864 : seq 16298352 844 bytes -8676> 2014-08-13 17:23:19.761709 7fcd1d4027a0 2 journal read_entry 807976960 : seq 16298353 844 bytes -8675> 2014-08-13 17:23:19.761723 7fcd1d4027a0 2 journal read_entry 807981056 : seq 16298354 844 bytes -8674> 2014-08-13 17:23:19.761737 7fcd1d4027a0 2 journal read_entry 807985152 : seq 16298355 844 bytes -8673> 2014-08-13 17:23:19.761750 7fcd1d4027a0 2 journal read_entry 807989248 : seq 16298356 844 bytes -8672> 2014-08-13 17:23:19.761763 7fcd1d4027a0 2 journal read_entry 807993344 : seq 16298357 844 bytes -8671> 2014-08-13 17:23:19.761777 7fcd1d4027a0 2 journal read_entry 807997440 : seq 16298358 844 bytes -8670> 2014-08-13 17:23:19.761790 7fcd1d4027a0 2 journal read_entry 808001536 : seq 16298359 844 bytes -8669> 2014-08-13 17:23:19.761803 7fcd1d4027a0 2 journal read_entry 808005632 : seq 16298360 844 bytes -8668> 2014-08-13 17:23:19.761817 7fcd1d4027a0 2 journal read_entry 808009728 : seq 16298361 844 bytes -8667> 2014-08-13 17:23:19.761830 7fcd1d4027a0 2 journal read_entry 808013824 : seq 16298362 844 bytes -8666> 2014-08-13 17:23:19.761844 7fcd1d4027a0 2 journal read_entry 808017920 : seq 16298363 844 bytes -8665> 2014-08-13 17:23:19.761857 7fcd1d4027a0 2 journal read_entry 808022016 : seq 16298364 844 bytes -8664> 2014-08-13 17:23:19.761870 7fcd1d4027a0 2 journal read_entry 808026112 : seq 16298365 844 bytes -8663> 2014-08-13 17:23:19.761883 7fcd1d4027a0 2 journal read_entry 808030208 : seq 16298366 844 bytes -8662> 2014-08-13 17:23:19.761897 7fcd1d4027a0 2 journal read_entry 808034304 : seq 16298367 844 bytes -8661> 2014-08-13 17:23:19.761910 7fcd1d4027a0 2 journal read_entry 808038400 : seq 16298368 1086 bytes -8660> 2014-08-13 17:23:19.761924 7fcd1d4027a0 2 journal read_entry 808042496 : seq 16298369 844 bytes -8659> 2014-08-13 17:23:19.761937 7fcd1d4027a0 2 journal read_entry 808046592 : seq 16298370 844 bytes -8658> 2014-08-13 17:23:19.761950 7fcd1d4027a0 2 journal read_entry 808050688 : seq 16298371 844 bytes -8657> 2014-08-13 17:23:19.761964 7fcd1d4027a0 2 journal read_entry 808054784 : seq 16298372 844 bytes -8656> 2014-08-13 17:23:19.761978 7fcd1d4027a0 2 journal read_entry 808058880 : seq 16298373 844 bytes -8655> 2014-08-13 17:23:19.761993 7fcd1d4027a0 2 journal read_entry 808062976 : seq 16298374 864 bytes -8654> 2014-08-13 17:23:19.762007 7fcd1d4027a0 2 journal read_entry 808067072 : seq 16298375 844 bytes -8653> 2014-08-13 17:23:19.762020 7fcd1d4027a0 2 journal read_entry 808071168 : seq 16298376 844 bytes -8652> 2014-08-13 17:23:19.762033 7fcd1d4027a0 2 journal read_entry 808075264 : seq 16298377 43 bytes -8651> 2014-08-13 17:23:19.762046 7fcd1d4027a0 2 journal read_entry 808079360 : seq 16298378 43 bytes -8650> 2014-08-13 17:23:19.762060 7fcd1d4027a0 2 journal read_entry 808083456 : seq 16298379 844 bytes -8649> 2014-08-13 17:23:19.762073 7fcd1d4027a0 2 journal read_entry 808087552 : seq 16298380 1086 bytes -8648> 2014-08-13 17:23:19.762087 7fcd1d4027a0 2 journal read_entry 808091648 : seq 16298381 864 bytes -8647> 2014-08-13 17:23:19.762451 7fcd1d4027a0 2 journal read_entry 808095744 : seq 16298382 844 bytes -8646> 2014-08-13 17:23:19.763796 7fcd1d4027a0 2 journal read_entry 808370176 : seq 16298383 269444 bytes -8645> 2014-08-13 17:23:19.763825 7fcd1d4027a0 2 journal read_entry 808374272 : seq 16298384 2378 bytes -8644> 2014-08-13 17:23:19.763840 7fcd1d4027a0 2 journal read_entry 808378368 : seq 16298385 2316 bytes -8643> 2014-08-13 17:23:19.763856 7fcd1d4027a0 2 journal read_entry 808382464 : seq 16298386 3357 bytes -8642> 2014-08-13 17:23:19.763870 7fcd1d4027a0 2 journal read_entry 808386560 : seq 16298387 2257 bytes -8641> 2014-08-13 17:23:19.763884 7fcd1d4027a0 2 journal read_entry 808390656 : seq 16298388 2316 bytes -8640> 2014-08-13 17:23:19.763898 7fcd1d4027a0 2 journal read_entry 808394752 : seq 16298389 2319 bytes -8639> 2014-08-13 17:23:19.763912 7fcd1d4027a0 2 journal read_entry 808398848 : seq 16298390 2201 bytes -8638> 2014-08-13 17:23:19.763926 7fcd1d4027a0 2 journal read_entry 808402944 : seq 16298391 2201 bytes -8637> 2014-08-13 17:23:19.763940 7fcd1d4027a0 2 journal read_entry 808407040 : seq 16298392 2257 bytes -8636> 2014-08-13 17:23:19.763953 7fcd1d4027a0 2 journal read_entry 808411136 : seq 16298393 2260 bytes -8635> 2014-08-13 17:23:19.763967 7fcd1d4027a0 2 journal read_entry 808415232 : seq 16298394 2260 bytes -8634> 2014-08-13 17:23:19.763981 7fcd1d4027a0 2 journal read_entry 808419328 : seq 16298395 3180 bytes -8633> 2014-08-13 17:23:19.763995 7fcd1d4027a0 2 journal read_entry 808423424 : seq 16298396 2375 bytes -8632> 2014-08-13 17:23:19.764009 7fcd1d4027a0 2 journal read_entry 808427520 : seq 16298397 2555 bytes -8631> 2014-08-13 17:23:19.764024 7fcd1d4027a0 2 journal read_entry 808431616 : seq 16298398 2437 bytes -8630> 2014-08-13 17:23:19.764038 7fcd1d4027a0 2 journal read_entry 808435712 : seq 16298399 2586 bytes -8629> 2014-08-13 17:23:19.764052 7fcd1d4027a0 2 journal read_entry 808439808 : seq 16298400 2260 bytes -8628> 2014-08-13 17:23:19.764066 7fcd1d4027a0 2 journal read_entry 808443904 : seq 16298401 876 bytes -8627> 2014-08-13 17:23:19.764079 7fcd1d4027a0 2 journal read_entry 808448000 : seq 16298402 878 bytes -8626> 2014-08-13 17:23:19.764093 7fcd1d4027a0 2 journal read_entry 808452096 : seq 16298403 878 bytes -8625> 2014-08-13 17:23:19.764106 7fcd1d4027a0 2 journal read_entry 808456192 : seq 16298404 878 bytes -8624> 2014-08-13 17:23:19.764119 7fcd1d4027a0 2 journal read_entry 808460288 : seq 16298405 876 bytes -8623> 2014-08-13 17:23:19.764133 7fcd1d4027a0 2 journal read_entry 808464384 : seq 16298406 878 bytes -8622> 2014-08-13 17:23:19.764155 7fcd1d4027a0 2 journal read_entry 808468480 : seq 16298407 878 bytes -8621> 2014-08-13 17:23:19.764169 7fcd1d4027a0 2 journal No further valid entries found, journal is most likely valid -8620> 2014-08-13 17:23:19.764192 7fcd1d4027a0 2 journal No further valid entries found, journal is most likely valid -8619> 2014-08-13 17:23:19.764197 7fcd1d4027a0 3 journal journal_replay: end of journal, done. -8618> 2014-08-13 17:23:19.775089 7fcd1d4027a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 -8617> 2014-08-13 17:23:19.775502 7fcd1d4027a0 2 osd.4 0 boot -8616> 2014-08-13 17:23:19.781627 7fcd1d4027a0 1 cls/refcount/cls_refcount.cc:231: Loaded refcount class! -8615> 2014-08-13 17:23:19.781864 7fcd1d4027a0 1 cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! -8614> 2014-08-13 17:23:19.782067 7fcd1d4027a0 1 cls/statelog/cls_statelog.cc:306: Loaded log class! -8613> 2014-08-13 17:23:19.786323 7fcd1d4027a0 1 cls/rgw/cls_rgw.cc:1599: Loaded rgw class! -8612> 2014-08-13 17:23:19.786588 7fcd1d4027a0 1 cls/version/cls_version.cc:227: Loaded version class! -8611> 2014-08-13 17:23:19.787316 7fcd1d4027a0 0 cls/hello/cls_hello.cc:271: loading cls_hello -8610> 2014-08-13 17:23:19.787522 7fcd1d4027a0 1 cls/log/cls_log.cc:312: Loaded log class! -8609> 2014-08-13 17:23:19.787726 7fcd1d4027a0 1 cls/user/cls_user.cc:367: Loaded user class! -8608> 2014-08-13 17:23:19.789649 7fcd1d4027a0 0 osd.4 194472 crush map has features 2303210029056, adjusting msgr requires for clients -8607> 2014-08-13 17:23:19.789672 7fcd1d4027a0 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for mons -8606> 2014-08-13 17:23:19.789680 7fcd1d4027a0 0 osd.4 194472 crush map has features 2578087936000, adjusting msgr requires for osds -8605> 2014-08-13 17:23:19.789706 7fcd1d4027a0 0 osd.4 194472 load_pgs -8604> 2014-08-13 17:23:19.794167 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[1.2b(unlocked)] enter Initial -8603> 2014-08-13 17:23:19.827006 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.032842 0 0.000000 -8602> 2014-08-13 17:23:19.827070 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -8601> 2014-08-13 17:23:19.827538 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[2.2a(unlocked)] enter Initial -8600> 2014-08-13 17:23:19.840696 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.013157 0 0.000000 -8599> 2014-08-13 17:23:19.840734 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -8598> 2014-08-13 17:23:19.841141 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.84(unlocked)] enter Initial -8597> 2014-08-13 17:23:19.843037 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001896 0 0.000000 -8596> 2014-08-13 17:23:19.843071 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset -8595> 2014-08-13 17:23:19.843391 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.99(unlocked)] enter Initial -8594> 2014-08-13 17:23:19.845170 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=0 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001778 0 0.000000 -8593> 2014-08-13 17:23:19.845208 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=0 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset -8592> 2014-08-13 17:23:19.845580 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.f3(unlocked)] enter Initial -8591> 2014-08-13 17:23:19.847705 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=0 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002126 0 0.000000 -8590> 2014-08-13 17:23:19.847742 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=0 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset -8589> 2014-08-13 17:23:19.848068 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.113(unlocked)] enter Initial -8588> 2014-08-13 17:23:19.849902 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=0 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001835 0 0.000000 -8587> 2014-08-13 17:23:19.849938 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=0 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset -8586> 2014-08-13 17:23:19.850247 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.116(unlocked)] enter Initial -8585> 2014-08-13 17:23:19.851938 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=0 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Initial 0.001691 0 0.000000 -8584> 2014-08-13 17:23:19.851975 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=0 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset -8583> 2014-08-13 17:23:19.852308 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.12a(unlocked)] enter Initial -8582> 2014-08-13 17:23:19.854275 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=0 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.001967 0 0.000000 -8581> 2014-08-13 17:23:19.854310 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=0 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -8580> 2014-08-13 17:23:19.854634 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.14a(unlocked)] enter Initial -8579> 2014-08-13 17:23:19.856718 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002084 0 0.000000 -8578> 2014-08-13 17:23:19.856753 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -8577> 2014-08-13 17:23:19.857103 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1a0(unlocked)] enter Initial -8576> 2014-08-13 17:23:19.858954 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=0 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.001852 0 0.000000 -8575> 2014-08-13 17:23:19.858990 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=0 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -8574> 2014-08-13 17:23:19.859340 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1b8(unlocked)] enter Initial -8573> 2014-08-13 17:23:19.861004 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001663 0 0.000000 -8572> 2014-08-13 17:23:19.861038 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset -8571> 2014-08-13 17:23:19.861344 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1d2(unlocked)] enter Initial -8570> 2014-08-13 17:23:19.863077 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=0 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001733 0 0.000000 -8569> 2014-08-13 17:23:19.863112 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=0 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset -8568> 2014-08-13 17:23:19.863454 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1f1(unlocked)] enter Initial -8567> 2014-08-13 17:23:19.865398 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=0 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001945 0 0.000000 -8566> 2014-08-13 17:23:19.865433 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=0 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset -8565> 2014-08-13 17:23:19.865795 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1fb(unlocked)] enter Initial -8564> 2014-08-13 17:23:19.867983 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=0 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.002188 0 0.000000 -8563> 2014-08-13 17:23:19.868018 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=0 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -8562> 2014-08-13 17:23:19.868392 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.220(unlocked)] enter Initial -8561> 2014-08-13 17:23:19.870469 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=0 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002078 0 0.000000 -8560> 2014-08-13 17:23:19.870505 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=0 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -8559> 2014-08-13 17:23:19.870871 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.226(unlocked)] enter Initial -8558> 2014-08-13 17:23:19.872732 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=0 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001862 0 0.000000 -8557> 2014-08-13 17:23:19.872768 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=0 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -8556> 2014-08-13 17:23:19.873067 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.22c(unlocked)] enter Initial -8555> 2014-08-13 17:23:19.874957 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=0 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001890 0 0.000000 -8554> 2014-08-13 17:23:19.875023 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=0 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset -8553> 2014-08-13 17:23:19.875325 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.28f(unlocked)] enter Initial -8552> 2014-08-13 17:23:19.877498 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=0 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.002173 0 0.000000 -8551> 2014-08-13 17:23:19.877568 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=0 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -8550> 2014-08-13 17:23:19.877981 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.29a(unlocked)] enter Initial -8549> 2014-08-13 17:23:19.879595 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=0 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001614 0 0.000000 -8548> 2014-08-13 17:23:19.879630 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=0 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -8547> 2014-08-13 17:23:19.879942 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2b9(unlocked)] enter Initial -8546> 2014-08-13 17:23:19.881938 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Initial 0.001997 0 0.000000 -8545> 2014-08-13 17:23:19.881973 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset -8544> 2014-08-13 17:23:19.882313 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2d1(unlocked)] enter Initial -8543> 2014-08-13 17:23:19.884508 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=0 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Initial 0.002195 0 0.000000 -8542> 2014-08-13 17:23:19.884544 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=0 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset -8541> 2014-08-13 17:23:19.884860 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.30f(unlocked)] enter Initial -8540> 2014-08-13 17:23:19.886598 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=0 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Initial 0.001738 0 0.000000 -8539> 2014-08-13 17:23:19.886633 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=0 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset -8538> 2014-08-13 17:23:19.886919 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.362(unlocked)] enter Initial -8537> 2014-08-13 17:23:19.888725 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=0 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001807 0 0.000000 -8536> 2014-08-13 17:23:19.888761 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=0 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset -8535> 2014-08-13 17:23:19.889091 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.39d(unlocked)] enter Initial -8534> 2014-08-13 17:23:19.890994 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=0 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001904 0 0.000000 -8533> 2014-08-13 17:23:19.891031 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=0 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset -8532> 2014-08-13 17:23:19.891419 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3ad(unlocked)] enter Initial -8531> 2014-08-13 17:23:19.893366 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=0 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Initial 0.001948 0 0.000000 -8530> 2014-08-13 17:23:19.893402 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=0 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset -8529> 2014-08-13 17:23:19.893754 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3af(unlocked)] enter Initial -8528> 2014-08-13 17:23:19.895669 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=0 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001916 0 0.000000 -8527> 2014-08-13 17:23:19.895706 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=0 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset -8526> 2014-08-13 17:23:19.896018 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3b9(unlocked)] enter Initial -8525> 2014-08-13 17:23:19.898167 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=0 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.002149 0 0.000000 -8524> 2014-08-13 17:23:19.898203 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=0 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -8523> 2014-08-13 17:23:19.898559 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d5(unlocked)] enter Initial -8522> 2014-08-13 17:23:19.899926 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001367 0 0.000000 -8521> 2014-08-13 17:23:19.899960 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset -8520> 2014-08-13 17:23:19.900326 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d9(unlocked)] enter Initial -8519> 2014-08-13 17:23:19.902234 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=0 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001908 0 0.000000 -8518> 2014-08-13 17:23:19.902270 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=0 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset -8517> 2014-08-13 17:23:19.902584 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3e4(unlocked)] enter Initial -8516> 2014-08-13 17:23:19.904750 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=0 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002166 0 0.000000 -8515> 2014-08-13 17:23:19.904786 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=0 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -8514> 2014-08-13 17:23:19.905140 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.481(unlocked)] enter Initial -8513> 2014-08-13 17:23:19.906936 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=0 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001797 0 0.000000 -8512> 2014-08-13 17:23:19.906972 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=0 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset -8511> 2014-08-13 17:23:19.907304 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.48e(unlocked)] enter Initial -8510> 2014-08-13 17:23:19.909292 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=0 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001989 0 0.000000 -8509> 2014-08-13 17:23:19.909328 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=0 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset -8508> 2014-08-13 17:23:19.909673 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4aa(unlocked)] enter Initial -8507> 2014-08-13 17:23:19.911301 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=0 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Initial 0.001629 0 0.000000 -8506> 2014-08-13 17:23:19.911366 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=0 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset -8505> 2014-08-13 17:23:19.911700 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4df(unlocked)] enter Initial -8504> 2014-08-13 17:23:19.913537 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=0 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001838 0 0.000000 -8503> 2014-08-13 17:23:19.913573 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=0 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset -8502> 2014-08-13 17:23:19.913859 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4fe(unlocked)] enter Initial -8501> 2014-08-13 17:23:19.915717 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001859 0 0.000000 -8500> 2014-08-13 17:23:19.915753 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -8499> 2014-08-13 17:23:19.916055 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.537(unlocked)] enter Initial -8498> 2014-08-13 17:23:19.917821 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=0 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001767 0 0.000000 -8497> 2014-08-13 17:23:19.917857 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=0 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -8496> 2014-08-13 17:23:19.918179 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.581(unlocked)] enter Initial -8495> 2014-08-13 17:23:19.919886 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=0 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Initial 0.001707 0 0.000000 -8494> 2014-08-13 17:23:19.919921 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=0 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset -8493> 2014-08-13 17:23:19.920252 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.619(unlocked)] enter Initial -8492> 2014-08-13 17:23:19.922232 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=0 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001981 0 0.000000 -8491> 2014-08-13 17:23:19.922272 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=0 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset -8490> 2014-08-13 17:23:19.922609 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.630(unlocked)] enter Initial -8489> 2014-08-13 17:23:19.924393 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=0 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Initial 0.001784 0 0.000000 -8488> 2014-08-13 17:23:19.924429 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=0 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset -8487> 2014-08-13 17:23:19.924928 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.639(unlocked)] enter Initial -8486> 2014-08-13 17:23:19.926779 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001851 0 0.000000 -8485> 2014-08-13 17:23:19.926814 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -8484> 2014-08-13 17:23:19.927336 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.653(unlocked)] enter Initial -8483> 2014-08-13 17:23:19.928970 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=0 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001634 0 0.000000 -8482> 2014-08-13 17:23:19.929006 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=0 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -8481> 2014-08-13 17:23:19.929367 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.69f(unlocked)] enter Initial -8480> 2014-08-13 17:23:19.931618 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=0 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Initial 0.002252 0 0.000000 -8479> 2014-08-13 17:23:19.931655 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=0 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset -8478> 2014-08-13 17:23:19.931944 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.6bd(unlocked)] enter Initial -8477> 2014-08-13 17:23:19.933735 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001791 0 0.000000 -8476> 2014-08-13 17:23:19.933769 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -8475> 2014-08-13 17:23:19.934089 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.707(unlocked)] enter Initial -8474> 2014-08-13 17:23:19.935723 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=0 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001634 0 0.000000 -8473> 2014-08-13 17:23:19.935759 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=0 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset -8472> 2014-08-13 17:23:19.936057 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.775(unlocked)] enter Initial -8471> 2014-08-13 17:23:19.938179 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=0 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002123 0 0.000000 -8470> 2014-08-13 17:23:19.938200 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=0 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset -8469> 2014-08-13 17:23:19.938349 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.791(unlocked)] enter Initial -8468> 2014-08-13 17:23:19.939374 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.001024 0 0.000000 -8467> 2014-08-13 17:23:19.939394 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=0 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -8466> 2014-08-13 17:23:19.941454 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] enter Initial -8465> 2014-08-13 17:23:19.949949 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.008495 0 0.000000 -8464> 2014-08-13 17:23:19.949972 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Reset -8463> 2014-08-13 17:23:19.950183 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.34(unlocked)] enter Initial -8462> 2014-08-13 17:23:19.958451 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=0 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.008268 0 0.000000 -8461> 2014-08-13 17:23:19.958474 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194471/194471) [26,30,340]/[4] r=0 lpr=0 pi=193125-194470/48 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset -8460> 2014-08-13 17:23:19.958683 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3d(unlocked)] enter Initial -8459> 2014-08-13 17:23:19.971646 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Initial 0.012963 0 0.000000 -8458> 2014-08-13 17:23:19.971671 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194470/194471/194470) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194470/25 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset -8457> 2014-08-13 17:23:19.971885 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.5e(unlocked)] enter Initial -8456> 2014-08-13 17:23:19.981024 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=0 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Initial 0.009139 0 0.000000 -8455> 2014-08-13 17:23:19.981047 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=0 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset -8454> 2014-08-13 17:23:19.981257 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.9b(unlocked)] enter Initial -8453> 2014-08-13 17:23:19.988880 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.007623 0 0.000000 -8452> 2014-08-13 17:23:19.988903 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194470/194471/194470) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194470/24 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -8451> 2014-08-13 17:23:19.989084 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.b6(unlocked)] enter Initial -8450> 2014-08-13 17:23:19.996544 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.007460 0 0.000000 -8449> 2014-08-13 17:23:19.996568 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194470/194471/194470) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194470/25 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset -8448> 2014-08-13 17:23:19.996758 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.cd(unlocked)] enter Initial -8447> 2014-08-13 17:23:20.009209 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194470/194471/192967) [6,374,4] r=2 lpr=0 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Initial 0.012452 0 0.000000 -8446> 2014-08-13 17:23:20.009233 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194470/194471/192967) [6,374,4] r=2 lpr=0 pi=193125-194470/24 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset -8445> 2014-08-13 17:23:20.009459 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.e2(unlocked)] enter Initial -8444> 2014-08-13 17:23:20.018390 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194470/26 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Initial 0.008931 0 0.000000 -8443> 2014-08-13 17:23:20.018413 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194470/194471/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194470/26 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset -8442> 2014-08-13 17:23:20.018616 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.136(unlocked)] enter Initial -8441> 2014-08-13 17:23:20.026986 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Initial 0.008371 0 0.000000 -8440> 2014-08-13 17:23:20.027031 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194470/194471/194470) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194470/23 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset -8439> 2014-08-13 17:23:20.027221 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.181(unlocked)] enter Initial -8438> 2014-08-13 17:23:20.035084 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Initial 0.007864 0 0.000000 -8437> 2014-08-13 17:23:20.035107 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset -8436> 2014-08-13 17:23:20.035343 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1bb(unlocked)] enter Initial -8435> 2014-08-13 17:23:20.044606 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Initial 0.009263 0 0.000000 -8434> 2014-08-13 17:23:20.044629 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194470/194471/194470) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194470/23 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset -8433> 2014-08-13 17:23:20.044826 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1de(unlocked)] enter Initial -8432> 2014-08-13 17:23:20.052479 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194470/27 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Initial 0.007654 0 0.000000 -8431> 2014-08-13 17:23:20.052502 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194470/194471/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194470/27 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset -8430> 2014-08-13 17:23:20.052699 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e4(unlocked)] enter Initial -8429> 2014-08-13 17:23:20.060347 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=0 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Initial 0.007647 0 0.000000 -8428> 2014-08-13 17:23:20.060370 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=0 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset -8427> 2014-08-13 17:23:20.060551 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e6(unlocked)] enter Initial -8426> 2014-08-13 17:23:20.068359 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=0 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.007809 0 0.000000 -8425> 2014-08-13 17:23:20.068382 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=0 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset -8424> 2014-08-13 17:23:20.068577 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1f7(unlocked)] enter Initial -8423> 2014-08-13 17:23:20.079235 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Initial 0.010658 0 0.000000 -8422> 2014-08-13 17:23:20.079260 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194470/194471/194470) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194470/25 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset -8421> 2014-08-13 17:23:20.079460 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.206(unlocked)] enter Initial -8420> 2014-08-13 17:23:20.087260 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194470/194471/193262) [343,45,4] r=2 lpr=0 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Initial 0.007801 0 0.000000 -8419> 2014-08-13 17:23:20.087282 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194470/194471/193262) [343,45,4] r=2 lpr=0 pi=193262-194470/24 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset -8418> 2014-08-13 17:23:20.087475 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.20e(unlocked)] enter Initial -8417> 2014-08-13 17:23:20.100211 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.012735 0 0.000000 -8416> 2014-08-13 17:23:20.100234 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset -8415> 2014-08-13 17:23:20.100432 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.299(unlocked)] enter Initial -8414> 2014-08-13 17:23:20.107969 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.007538 0 0.000000 -8413> 2014-08-13 17:23:20.107993 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset -8412> 2014-08-13 17:23:20.108200 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2a1(unlocked)] enter Initial -8411> 2014-08-13 17:23:20.116707 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=0 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.008508 0 0.000000 -8410> 2014-08-13 17:23:20.116731 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194470/194471/193188) [17,4,395] r=1 lpr=0 pi=193188-194470/30 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset -8409> 2014-08-13 17:23:20.116945 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2ec(unlocked)] enter Initial -8408> 2014-08-13 17:23:20.125265 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194470/40 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Initial 0.008320 0 0.000000 -8407> 2014-08-13 17:23:20.125289 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194470/194471/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194470/40 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset -8406> 2014-08-13 17:23:20.125496 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.30b(unlocked)] enter Initial -8405> 2014-08-13 17:23:20.138059 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Initial 0.012563 0 0.000000 -8404> 2014-08-13 17:23:20.138112 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194470/194471/194470) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194470/26 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset -8403> 2014-08-13 17:23:20.138296 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.33a(unlocked)] enter Initial -8402> 2014-08-13 17:23:20.146225 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194470/194471/193250) [304,323,4] r=2 lpr=0 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Initial 0.007929 0 0.000000 -8401> 2014-08-13 17:23:20.146248 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194470/194471/193250) [304,323,4] r=2 lpr=0 pi=193250-194470/24 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset -8400> 2014-08-13 17:23:20.146439 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.350(unlocked)] enter Initial -8399> 2014-08-13 17:23:20.153635 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=0 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Initial 0.007196 0 0.000000 -8398> 2014-08-13 17:23:20.153657 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=0 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset -8397> 2014-08-13 17:23:20.153852 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.369(unlocked)] enter Initial -8396> 2014-08-13 17:23:20.162026 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=0 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.008174 0 0.000000 -8395> 2014-08-13 17:23:20.162049 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194470/194471/193186) [45,319,4] r=2 lpr=0 pi=193186-194470/24 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset -8394> 2014-08-13 17:23:20.162252 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] enter Initial -8393> 2014-08-13 17:23:20.170120 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.007868 0 0.000000 -8392> 2014-08-13 17:23:20.170143 7fcd1d4027a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Reset -8391> 2014-08-13 17:23:20.170358 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3b7(unlocked)] enter Initial -8390> 2014-08-13 17:23:20.178506 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.008149 0 0.000000 -8389> 2014-08-13 17:23:20.178529 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset -8388> 2014-08-13 17:23:20.178720 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3bf(unlocked)] enter Initial -8387> 2014-08-13 17:23:20.185834 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.007114 0 0.000000 -8386> 2014-08-13 17:23:20.185858 7fcd1d4027a0 5 osd.4 pg_epoch: 194471 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset -8385> 2014-08-13 17:23:20.186037 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3f7(unlocked)] enter Initial -8384> 2014-08-13 17:23:20.194896 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194470/29 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Initial 0.008859 0 0.000000 -8383> 2014-08-13 17:23:20.194939 7fcd1d4027a0 5 osd.4 pg_epoch: 194472 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194470/194471/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194470/29 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset -8382> 2014-08-13 17:23:20.194952 7fcd1d4027a0 0 osd.4 194472 load_pgs opened 74 pgs -8381> 2014-08-13 17:23:20.194988 7fcd1d4027a0 2 osd.4 194472 superblock: i am osd.4 -8380> 2014-08-13 17:23:20.195089 7fcd1d4027a0 1 accepter.accepter.start -8379> 2014-08-13 17:23:20.195134 7fcd1d4027a0 1 accepter.accepter.start -8378> 2014-08-13 17:23:20.195231 7fcd1d4027a0 1 accepter.accepter.start -8377> 2014-08-13 17:23:20.195273 7fcd1d4027a0 1 accepter.accepter.start -8376> 2014-08-13 17:23:20.195315 7fcd1d4027a0 1 accepter.accepter.start -8375> 2014-08-13 17:23:20.195338 7fcd1d4027a0 10 monclient(hunting): init -8374> 2014-08-13 17:23:20.195344 7fcd1d4027a0 5 adding auth protocol: cephx -8373> 2014-08-13 17:23:20.195345 7fcd1d4027a0 10 monclient(hunting): auth_supported 2 method cephx -8372> 2014-08-13 17:23:20.195525 7fcd1d4027a0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-4/keyring -8371> 2014-08-13 17:23:20.195834 7fcd1d4027a0 5 asok(0x2e78230) register_command objecter_requests hook 0x4529d00 -8370> 2014-08-13 17:23:20.195852 7fccff63c700 5 osd.4 194472 heartbeat: osd_stat(8084 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) -8369> 2014-08-13 17:23:20.195906 7fcd1d4027a0 10 monclient(hunting): renew_subs -8368> 2014-08-13 17:23:20.195912 7fcd1d4027a0 10 monclient(hunting): _reopen_session rank -1 name -8367> 2014-08-13 17:23:20.196025 7fcd1d4027a0 10 monclient(hunting): picked mon.storage0111-ib con 0x4e25d80 addr 192.168.100.111:6789/0 -8366> 2014-08-13 17:23:20.196044 7fcd1d4027a0 10 monclient(hunting): _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8365> 2014-08-13 17:23:20.196050 7fcd1d4027a0 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 0x2e88fc0 con 0x4e25d80 -8364> 2014-08-13 17:23:20.196061 7fcd1d4027a0 10 monclient(hunting): renew_subs -8363> 2014-08-13 17:23:20.196609 7fcd0aa4e700 10 monclient(hunting): renew_subs -8362> 2014-08-13 17:23:20.197232 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 1 ==== mon_map v1 ==== 512+0+0 (3258631466 0 0) 0x2f905a0 con 0x4e25d80 -8361> 2014-08-13 17:23:20.197294 7fcd0aa4e700 10 monclient(hunting): handle_monmap mon_map v1 -8360> 2014-08-13 17:23:20.197323 7fcd0aa4e700 10 monclient(hunting): got monmap 3, mon.storage0111-ib is now rank 0 -8359> 2014-08-13 17:23:20.197331 7fcd0aa4e700 10 monclient(hunting): dump: epoch 3 fsid 009d3518-e60d-4f74-a26d-c08c1976263c last_changed 2014-06-13 16:08:03.914184 created 2014-06-13 16:07:42.361285 0: 192.168.100.111:6789/0 mon.storage0111-ib 1: 192.168.100.112:6789/0 mon.storage0112-ib 2: 192.168.100.113:6789/0 mon.storage0113-ib -8358> 2014-08-13 17:23:20.197383 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (3557713976 0 0) 0x2ea4600 con 0x4e25d80 -8357> 2014-08-13 17:23:20.197425 7fcd0aa4e700 10 monclient(hunting): my global_id is 2462667 -8356> 2014-08-13 17:23:20.197743 7fcd0aa4e700 10 monclient(hunting): _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8355> 2014-08-13 17:23:20.197763 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x2e8a400 con 0x4e25d80 -8354> 2014-08-13 17:23:20.198463 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (3198296724 0 0) 0x2ea4400 con 0x4e25d80 -8353> 2014-08-13 17:23:20.198597 7fcd0aa4e700 10 monclient(hunting): _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8352> 2014-08-13 17:23:20.198614 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x2e8a880 con 0x4e25d80 -8351> 2014-08-13 17:23:20.199395 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 393+0+0 (2966221554 0 0) 0x2ea1c00 con 0x4e25d80 -8350> 2014-08-13 17:23:20.199512 7fcd0aa4e700 1 monclient(hunting): found mon.storage0111-ib -8349> 2014-08-13 17:23:20.199521 7fcd0aa4e700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8348> 2014-08-13 17:23:20.199531 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x32afa80 con 0x4e25d80 -8347> 2014-08-13 17:23:20.199561 7fcd0aa4e700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8346> 2014-08-13 17:23:20.199570 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- mon_subscribe({monmap=0+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x4e38000 con 0x4e25d80 -8345> 2014-08-13 17:23:20.199595 7fcd0aa4e700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-13 17:22:50.199594) -8344> 2014-08-13 17:23:20.199612 7fcd0aa4e700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -8343> 2014-08-13 17:23:20.199621 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- ?+0 0x2e8a640 con 0x4e25d80 -8342> 2014-08-13 17:23:20.199644 7fcd1d4027a0 5 monclient: authenticate success, global_id 2462667 -8341> 2014-08-13 17:23:20.199663 7fcd1d4027a0 10 monclient: wait_auth_rotating waiting (until 2014-08-13 17:23:50.199661) -8340> 2014-08-13 17:23:20.201953 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 5 ==== osd_map(194475..194475 src has 185139..194475) v3 ==== 268990+0+0 (1004398410 0 0) 0x2e88fc0 con 0x4e25d80 -8339> 2014-08-13 17:23:20.202018 7fcd0aa4e700 0 osd.4 194472 ignoring osdmap until we have initialized -8338> 2014-08-13 17:23:20.202037 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 6 ==== mon_map v1 ==== 512+0+0 (3258631466 0 0) 0x2f90960 con 0x4e25d80 -8337> 2014-08-13 17:23:20.202050 7fcd0aa4e700 10 monclient: handle_monmap mon_map v1 -8336> 2014-08-13 17:23:20.202068 7fcd0aa4e700 10 monclient: got monmap 3, mon.storage0111-ib is now rank 0 -8335> 2014-08-13 17:23:20.202078 7fcd0aa4e700 10 monclient: dump: epoch 3 fsid 009d3518-e60d-4f74-a26d-c08c1976263c last_changed 2014-06-13 16:08:03.914184 created 2014-06-13 16:07:42.361285 0: 192.168.100.111:6789/0 mon.storage0111-ib 1: 192.168.100.112:6789/0 mon.storage0112-ib 2: 192.168.100.113:6789/0 mon.storage0113-ib -8334> 2014-08-13 17:23:20.203306 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 7 ==== osd_map(194475..194475 src has 185139..194475) v3 ==== 268990+0+0 (1004398410 0 0) 0x2e8aac0 con 0x4e25d80 -8333> 2014-08-13 17:23:20.203331 7fcd0aa4e700 0 osd.4 194472 ignoring osdmap until we have initialized -8332> 2014-08-13 17:23:20.203356 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 8 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x32afa80 con 0x4e25d80 -8331> 2014-08-13 17:23:20.203379 7fcd0aa4e700 10 monclient: handle_subscribe_ack sent 2014-08-13 17:23:20.196064 renew after 2014-08-13 17:25:50.196064 -8330> 2014-08-13 17:23:20.203402 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 9 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (937497080 0 0) 0x2ea4200 con 0x4e25d80 -8329> 2014-08-13 17:23:20.203483 7fcd0aa4e700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-08-13 17:22:50.203482) -8328> 2014-08-13 17:23:20.203558 7fcd1d4027a0 10 monclient: wait_auth_rotating done -8327> 2014-08-13 17:23:20.203813 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.356072 2 0.000087 -8326> 2014-08-13 17:23:20.203857 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started -8325> 2014-08-13 17:23:20.203874 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start -8324> 2014-08-13 17:23:20.203886 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8323> 2014-08-13 17:23:20.203903 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000029 0 0.000000 -8322> 2014-08-13 17:23:20.203904 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.333398 2 0.000059 -8321> 2014-08-13 17:23:20.203942 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8320> 2014-08-13 17:23:20.203954 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -8319> 2014-08-13 17:23:20.203960 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8318> 2014-08-13 17:23:20.203973 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -8317> 2014-08-13 17:23:20.203982 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8316> 2014-08-13 17:23:20.203990 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8315> 2014-08-13 17:23:20.204020 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000047 0 0.000000 -8314> 2014-08-13 17:23:20.204042 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -8313> 2014-08-13 17:23:20.204226 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.315465 2 0.000062 -8312> 2014-08-13 17:23:20.204264 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started -8311> 2014-08-13 17:23:20.204280 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start -8310> 2014-08-13 17:23:20.204305 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8309> 2014-08-13 17:23:20.204316 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000036 0 0.000000 -8308> 2014-08-13 17:23:20.204327 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8307> 2014-08-13 17:23:20.204336 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8306> 2014-08-13 17:23:20.204345 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8305> 2014-08-13 17:23:20.204370 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.352396 2 0.000062 -8304> 2014-08-13 17:23:20.204393 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started -8303> 2014-08-13 17:23:20.204404 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start -8302> 2014-08-13 17:23:20.204414 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8301> 2014-08-13 17:23:20.204427 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8300> 2014-08-13 17:23:20.204442 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray -8299> 2014-08-13 17:23:20.204509 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.326941 2 0.000127 -8298> 2014-08-13 17:23:20.204529 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -8297> 2014-08-13 17:23:20.204541 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -8296> 2014-08-13 17:23:20.204550 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8295> 2014-08-13 17:23:20.204563 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8294> 2014-08-13 17:23:20.204577 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -8293> 2014-08-13 17:23:20.204660 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.293294 2 0.000089 -8292> 2014-08-13 17:23:20.204682 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started -8291> 2014-08-13 17:23:20.204691 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start -8290> 2014-08-13 17:23:20.204702 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8289> 2014-08-13 17:23:20.204712 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 -8288> 2014-08-13 17:23:20.204722 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray -8287> 2014-08-13 17:23:20.204904 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.348151 2 0.000060 -8286> 2014-08-13 17:23:20.204951 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -8285> 2014-08-13 17:23:20.204974 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -8284> 2014-08-13 17:23:20.204983 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8283> 2014-08-13 17:23:20.204997 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8282> 2014-08-13 17:23:20.205010 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -8281> 2014-08-13 17:23:20.205120 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.342008 2 0.000064 -8280> 2014-08-13 17:23:20.205248 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started -8279> 2014-08-13 17:23:20.205260 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start -8278> 2014-08-13 17:23:20.205268 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8277> 2014-08-13 17:23:20.205279 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 -8276> 2014-08-13 17:23:20.205290 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8275> 2014-08-13 17:23:20.205302 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8274> 2014-08-13 17:23:20.205313 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8273> 2014-08-13 17:23:20.205839 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.330816 2 0.000093 -8272> 2014-08-13 17:23:20.205865 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started -8271> 2014-08-13 17:23:20.205878 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start -8270> 2014-08-13 17:23:20.205912 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8269> 2014-08-13 17:23:20.205942 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000064 0 0.000000 -8268> 2014-08-13 17:23:20.205958 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8267> 2014-08-13 17:23:20.205976 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8266> 2014-08-13 17:23:20.206003 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8265> 2014-08-13 17:23:20.206060 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.365326 2 0.000086 -8264> 2014-08-13 17:23:20.206082 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8263> 2014-08-13 17:23:20.206093 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8262> 2014-08-13 17:23:20.206103 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8261> 2014-08-13 17:23:20.206114 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 -8260> 2014-08-13 17:23:20.206126 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8259> 2014-08-13 17:23:20.206137 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8258> 2014-08-13 17:23:20.206147 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8257> 2014-08-13 17:23:20.206161 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.283889 2 0.000070 -8256> 2014-08-13 17:23:20.206269 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started -8255> 2014-08-13 17:23:20.206272 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.356334 2 0.000063 -8254> 2014-08-13 17:23:20.206316 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start -8253> 2014-08-13 17:23:20.206323 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started -8252> 2014-08-13 17:23:20.206332 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8251> 2014-08-13 17:23:20.206342 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start -8250> 2014-08-13 17:23:20.206351 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8249> 2014-08-13 17:23:20.206344 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000028 0 0.000000 -8248> 2014-08-13 17:23:20.206365 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000023 0 0.000000 -8247> 2014-08-13 17:23:20.206373 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8246> 2014-08-13 17:23:20.206377 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8245> 2014-08-13 17:23:20.206388 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8244> 2014-08-13 17:23:20.206397 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8243> 2014-08-13 17:23:20.206384 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8242> 2014-08-13 17:23:20.206434 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8241> 2014-08-13 17:23:20.206751 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.036607 3 0.000054 -8240> 2014-08-13 17:23:20.206778 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started -8239> 2014-08-13 17:23:20.206790 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Start -8238> 2014-08-13 17:23:20.206785 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.320151 2 0.000059 -8237> 2014-08-13 17:23:20.206802 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray -8236> 2014-08-13 17:23:20.206807 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started -8235> 2014-08-13 17:23:20.206814 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000024 0 0.000000 -8234> 2014-08-13 17:23:20.206817 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start -8233> 2014-08-13 17:23:20.206826 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray -8232> 2014-08-13 17:23:20.206826 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8231> 2014-08-13 17:23:20.206839 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 -8230> 2014-08-13 17:23:20.206850 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray -8229> 2014-08-13 17:23:20.207336 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.380267 2 0.000107 -8228> 2014-08-13 17:23:20.207357 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -8227> 2014-08-13 17:23:20.207418 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -8226> 2014-08-13 17:23:20.207436 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8225> 2014-08-13 17:23:20.207449 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000031 0 0.000000 -8224> 2014-08-13 17:23:20.207462 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8223> 2014-08-13 17:23:20.207473 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8222> 2014-08-13 17:23:20.207483 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8221> 2014-08-13 17:23:20.207501 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.287580 2 0.000060 -8220> 2014-08-13 17:23:20.207523 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started -8219> 2014-08-13 17:23:20.207534 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start -8218> 2014-08-13 17:23:20.207544 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8217> 2014-08-13 17:23:20.207557 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -8216> 2014-08-13 17:23:20.207562 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.362355 2 0.000098 -8215> 2014-08-13 17:23:20.207568 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray -8214> 2014-08-13 17:23:20.207579 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started -8213> 2014-08-13 17:23:20.207589 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start -8212> 2014-08-13 17:23:20.207597 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8211> 2014-08-13 17:23:20.207610 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 -8210> 2014-08-13 17:23:20.207626 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8209> 2014-08-13 17:23:20.207638 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8208> 2014-08-13 17:23:20.207649 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8207> 2014-08-13 17:23:20.207901 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.276247 2 0.000061 -8206> 2014-08-13 17:23:20.207922 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started -8205> 2014-08-13 17:23:20.207934 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start -8204> 2014-08-13 17:23:20.207925 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.364854 2 0.000060 -8203> 2014-08-13 17:23:20.207947 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8202> 2014-08-13 17:23:20.207955 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started -8201> 2014-08-13 17:23:20.207959 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000 -8200> 2014-08-13 17:23:20.207967 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start -8199> 2014-08-13 17:23:20.207973 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray -8198> 2014-08-13 17:23:20.207976 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8197> 2014-08-13 17:23:20.207989 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 -8196> 2014-08-13 17:23:20.208001 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8195> 2014-08-13 17:23:20.208011 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8194> 2014-08-13 17:23:20.208043 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8193> 2014-08-13 17:23:20.208079 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.303293 2 0.000060 -8192> 2014-08-13 17:23:20.208100 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -8191> 2014-08-13 17:23:20.208106 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.294533 2 0.000059 -8190> 2014-08-13 17:23:20.208112 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -8189> 2014-08-13 17:23:20.208128 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started -8188> 2014-08-13 17:23:20.208138 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start -8187> 2014-08-13 17:23:20.208138 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8186> 2014-08-13 17:23:20.208153 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8185> 2014-08-13 17:23:20.208159 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000047 0 0.000000 -8184> 2014-08-13 17:23:20.208176 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000038 0 0.000000 -8183> 2014-08-13 17:23:20.208180 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8182> 2014-08-13 17:23:20.208191 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8181> 2014-08-13 17:23:20.208191 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8180> 2014-08-13 17:23:20.208202 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8179> 2014-08-13 17:23:20.208202 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8178> 2014-08-13 17:23:20.208213 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8177> 2014-08-13 17:23:20.208323 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.317291 2 0.000062 -8176> 2014-08-13 17:23:20.208345 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started -8175> 2014-08-13 17:23:20.208356 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start -8174> 2014-08-13 17:23:20.208366 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8173> 2014-08-13 17:23:20.208379 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000023 0 0.000000 -8172> 2014-08-13 17:23:20.208392 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8171> 2014-08-13 17:23:20.208403 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8170> 2014-08-13 17:23:20.208413 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8169> 2014-08-13 17:23:20.208433 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.281619 2 0.000059 -8168> 2014-08-13 17:23:20.208456 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -8167> 2014-08-13 17:23:20.208470 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -8166> 2014-08-13 17:23:20.208482 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8165> 2014-08-13 17:23:20.208496 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000026 0 0.000000 -8164> 2014-08-13 17:23:20.208509 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -8163> 2014-08-13 17:23:20.208536 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.326563 2 0.000059 -8162> 2014-08-13 17:23:20.208560 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started -8161> 2014-08-13 17:23:20.208571 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start -8160> 2014-08-13 17:23:20.208579 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8159> 2014-08-13 17:23:20.208591 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 -8158> 2014-08-13 17:23:20.208604 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray -8157> 2014-08-13 17:23:20.208599 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.272839 2 0.000060 -8156> 2014-08-13 17:23:20.208620 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started -8155> 2014-08-13 17:23:20.208632 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start -8154> 2014-08-13 17:23:20.208643 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8153> 2014-08-13 17:23:20.208656 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 -8152> 2014-08-13 17:23:20.208668 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8151> 2014-08-13 17:23:20.208680 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8150> 2014-08-13 17:23:20.208692 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8149> 2014-08-13 17:23:20.208750 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.258778 3 0.000041 -8148> 2014-08-13 17:23:20.208772 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started -8147> 2014-08-13 17:23:20.208783 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Start -8146> 2014-08-13 17:23:20.208794 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -8145> 2014-08-13 17:23:20.208806 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000023 0 0.000000 -8144> 2014-08-13 17:23:20.208821 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -8143> 2014-08-13 17:23:20.209092 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.228045 2 0.000040 -8142> 2014-08-13 17:23:20.209123 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started -8141> 2014-08-13 17:23:20.209139 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start -8140> 2014-08-13 17:23:20.209218 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.271019 2 0.000036 -8139> 2014-08-13 17:23:20.209212 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray -8138> 2014-08-13 17:23:20.209240 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started -8137> 2014-08-13 17:23:20.209289 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000150 0 0.000000 -8136> 2014-08-13 17:23:20.209292 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start -8135> 2014-08-13 17:23:20.209324 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray -8134> 2014-08-13 17:23:20.209329 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8133> 2014-08-13 17:23:20.209353 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000061 0 0.000000 -8132> 2014-08-13 17:23:20.209368 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8131> 2014-08-13 17:23:20.209380 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8130> 2014-08-13 17:23:20.209392 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8129> 2014-08-13 17:23:20.209482 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.344049 2 0.000059 -8128> 2014-08-13 17:23:20.209529 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started -8127> 2014-08-13 17:23:20.209555 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start -8126> 2014-08-13 17:23:20.209573 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8125> 2014-08-13 17:23:20.209586 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.285158 2 0.000059 -8124> 2014-08-13 17:23:20.209607 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started -8123> 2014-08-13 17:23:20.209610 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000056 0 0.000000 -8122> 2014-08-13 17:23:20.209618 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start -8121> 2014-08-13 17:23:20.209627 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8120> 2014-08-13 17:23:20.209627 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8119> 2014-08-13 17:23:20.209642 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8118> 2014-08-13 17:23:20.209652 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8117> 2014-08-13 17:23:20.209651 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000033 0 0.000000 -8116> 2014-08-13 17:23:20.209671 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray -8115> 2014-08-13 17:23:20.209809 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.276041 2 0.000059 -8114> 2014-08-13 17:23:20.209828 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -8113> 2014-08-13 17:23:20.209839 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -8112> 2014-08-13 17:23:20.209850 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8111> 2014-08-13 17:23:20.209863 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 -8110> 2014-08-13 17:23:20.209875 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8109> 2014-08-13 17:23:20.209885 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8108> 2014-08-13 17:23:20.209899 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8107> 2014-08-13 17:23:20.209966 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.325422 2 0.000061 -8106> 2014-08-13 17:23:20.209992 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started -8105> 2014-08-13 17:23:20.210009 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Reset 0.174902 2 0.000039 -8104> 2014-08-13 17:23:20.210033 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started -8103> 2014-08-13 17:23:20.210049 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Start -8102> 2014-08-13 17:23:20.210017 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start -8101> 2014-08-13 17:23:20.210060 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: transitioning to Primary -8100> 2014-08-13 17:23:20.210073 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Start 0.000024 0 0.000000 -8099> 2014-08-13 17:23:20.210086 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started/Primary -8098> 2014-08-13 17:23:20.210067 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8097> 2014-08-13 17:23:20.210098 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Started/Primary/Peering -8096> 2014-08-13 17:23:20.210118 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000116 0 0.000000 -8095> 2014-08-13 17:23:20.210120 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] enter Started/Primary/Peering/GetInfo -8094> 2014-08-13 17:23:20.210133 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray -8093> 2014-08-13 17:23:20.210313 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.110078 2 0.000054 -8092> 2014-08-13 17:23:20.210333 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started -8091> 2014-08-13 17:23:20.210345 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Start -8090> 2014-08-13 17:23:20.210356 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -8089> 2014-08-13 17:23:20.210370 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000025 0 0.000000 -8088> 2014-08-13 17:23:20.210383 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -8087> 2014-08-13 17:23:20.210540 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.150171 2 0.000039 -8086> 2014-08-13 17:23:20.210561 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started -8085> 2014-08-13 17:23:20.210572 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start -8084> 2014-08-13 17:23:20.210585 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray -8083> 2014-08-13 17:23:20.210598 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000026 0 0.000000 -8082> 2014-08-13 17:23:20.210611 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray -8081> 2014-08-13 17:23:20.210728 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.142345 2 0.000040 -8080> 2014-08-13 17:23:20.210750 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started -8079> 2014-08-13 17:23:20.210760 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start -8078> 2014-08-13 17:23:20.210768 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -8077> 2014-08-13 17:23:20.210778 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000019 0 0.000000 -8076> 2014-08-13 17:23:20.210788 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -8075> 2014-08-13 17:23:20.210705 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.024848 2 0.000041 -8074> 2014-08-13 17:23:20.210801 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started -8073> 2014-08-13 17:23:20.210835 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start -8072> 2014-08-13 17:23:20.210858 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8071> 2014-08-13 17:23:20.210894 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.356584 2 0.000059 -8070> 2014-08-13 17:23:20.210870 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 -8069> 2014-08-13 17:23:20.210913 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -8068> 2014-08-13 17:23:20.210931 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray -8067> 2014-08-13 17:23:20.210936 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -8066> 2014-08-13 17:23:20.210956 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8065> 2014-08-13 17:23:20.210970 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000033 0 0.000000 -8064> 2014-08-13 17:23:20.210984 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -8063> 2014-08-13 17:23:20.211141 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.315436 2 0.000060 -8062> 2014-08-13 17:23:20.211167 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started -8061> 2014-08-13 17:23:20.211180 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start -8060> 2014-08-13 17:23:20.211191 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8059> 2014-08-13 17:23:20.211204 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 -8058> 2014-08-13 17:23:20.211217 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8057> 2014-08-13 17:23:20.211217 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.343198 2 0.000059 -8056> 2014-08-13 17:23:20.211229 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8055> 2014-08-13 17:23:20.211242 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8054> 2014-08-13 17:23:20.211242 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -8053> 2014-08-13 17:23:20.211264 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -8052> 2014-08-13 17:23:20.211281 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8051> 2014-08-13 17:23:20.211293 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000028 0 0.000000 -8050> 2014-08-13 17:23:20.211309 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -8049> 2014-08-13 17:23:20.211355 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.304383 2 0.000061 -8048> 2014-08-13 17:23:20.211375 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started -8047> 2014-08-13 17:23:20.211386 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start -8046> 2014-08-13 17:23:20.211395 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8045> 2014-08-13 17:23:20.211409 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000022 0 0.000000 -8044> 2014-08-13 17:23:20.211421 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8043> 2014-08-13 17:23:20.211432 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8042> 2014-08-13 17:23:20.211444 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8041> 2014-08-13 17:23:20.211543 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.057885 2 0.000039 -8040> 2014-08-13 17:23:20.211552 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.302224 2 0.000059 -8039> 2014-08-13 17:23:20.211572 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started -8038> 2014-08-13 17:23:20.211587 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start -8037> 2014-08-13 17:23:20.211570 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started -8036> 2014-08-13 17:23:20.211597 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -8035> 2014-08-13 17:23:20.211611 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 -8034> 2014-08-13 17:23:20.211624 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -8033> 2014-08-13 17:23:20.211635 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -8032> 2014-08-13 17:23:20.211605 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start -8031> 2014-08-13 17:23:20.211646 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -8030> 2014-08-13 17:23:20.211657 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray -8029> 2014-08-13 17:23:20.211719 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000114 0 0.000000 -8028> 2014-08-13 17:23:20.211735 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray -8027> 2014-08-13 17:23:20.211831 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.318429 2 0.000060 -8026> 2014-08-13 17:23:20.211854 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started -8025> 2014-08-13 17:23:20.211867 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start -8024> 2014-08-13 17:23:20.211877 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8023> 2014-08-13 17:23:20.211890 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000024 0 0.000000 -8022> 2014-08-13 17:23:20.211903 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray -8021> 2014-08-13 17:23:20.212327 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.296574 2 0.000060 -8020> 2014-08-13 17:23:20.212352 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -8019> 2014-08-13 17:23:20.212364 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -8018> 2014-08-13 17:23:20.212374 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8017> 2014-08-13 17:23:20.212385 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 -8016> 2014-08-13 17:23:20.212397 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -8015> 2014-08-13 17:23:20.212527 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.283521 2 0.000059 -8014> 2014-08-13 17:23:20.212547 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -8013> 2014-08-13 17:23:20.212558 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -8012> 2014-08-13 17:23:20.212571 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -8011> 2014-08-13 17:23:20.212585 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000027 0 0.000000 -8010> 2014-08-13 17:23:20.212598 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -8009> 2014-08-13 17:23:20.212753 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.034223 2 0.000039 -8008> 2014-08-13 17:23:20.212787 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started -8007> 2014-08-13 17:23:20.212802 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Start -8006> 2014-08-13 17:23:20.212814 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray -8005> 2014-08-13 17:23:20.212827 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000025 0 0.000000 -8004> 2014-08-13 17:23:20.212840 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray -8003> 2014-08-13 17:23:20.212844 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.104851 2 0.000041 -8002> 2014-08-13 17:23:20.212871 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started -8001> 2014-08-13 17:23:20.212885 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start -8000> 2014-08-13 17:23:20.212898 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7999> 2014-08-13 17:23:20.212962 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.314759 2 0.000059 -7998> 2014-08-13 17:23:20.212914 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 -7997> 2014-08-13 17:23:20.212982 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -7996> 2014-08-13 17:23:20.212987 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray -7995> 2014-08-13 17:23:20.212993 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -7994> 2014-08-13 17:23:20.213010 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7993> 2014-08-13 17:23:20.213025 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000032 0 0.000000 -7992> 2014-08-13 17:23:20.213035 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -7991> 2014-08-13 17:23:20.213225 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.313264 2 0.000058 -7990> 2014-08-13 17:23:20.213245 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started -7989> 2014-08-13 17:23:20.213256 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start -7988> 2014-08-13 17:23:20.213269 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7987> 2014-08-13 17:23:20.213282 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000026 0 0.000000 -7986> 2014-08-13 17:23:20.213294 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7985> 2014-08-13 17:23:20.213306 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7984> 2014-08-13 17:23:20.213251 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.273857 2 0.000034 -7983> 2014-08-13 17:23:20.213316 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7982> 2014-08-13 17:23:20.213325 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -7981> 2014-08-13 17:23:20.213340 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -7980> 2014-08-13 17:23:20.213352 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7979> 2014-08-13 17:23:20.213364 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000024 0 0.000000 -7978> 2014-08-13 17:23:20.213376 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -7977> 2014-08-13 17:23:20.213606 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.295750 2 0.000080 -7976> 2014-08-13 17:23:20.213626 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -7975> 2014-08-13 17:23:20.213638 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -7974> 2014-08-13 17:23:20.213649 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7973> 2014-08-13 17:23:20.213663 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000 -7972> 2014-08-13 17:23:20.213675 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -7971> 2014-08-13 17:23:20.213788 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.354799 2 0.000059 -7970> 2014-08-13 17:23:20.213811 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -7969> 2014-08-13 17:23:20.213822 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -7968> 2014-08-13 17:23:20.213834 7fcd02841700 1 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7967> 2014-08-13 17:23:20.213848 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000026 0 0.000000 -7966> 2014-08-13 17:23:20.213862 7fcd02841700 5 osd.4 pg_epoch: 194472 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -7965> 2014-08-13 17:23:20.213873 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.352835 2 0.000066 -7964> 2014-08-13 17:23:20.213901 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started -7963> 2014-08-13 17:23:20.213912 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start -7962> 2014-08-13 17:23:20.213921 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7961> 2014-08-13 17:23:20.213934 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000022 0 0.000000 -7960> 2014-08-13 17:23:20.213946 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7959> 2014-08-13 17:23:20.213956 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7958> 2014-08-13 17:23:20.213965 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7957> 2014-08-13 17:23:20.214153 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.334523 2 0.000060 -7956> 2014-08-13 17:23:20.214183 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -7955> 2014-08-13 17:23:20.214192 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -7954> 2014-08-13 17:23:20.214204 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7953> 2014-08-13 17:23:20.214215 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 -7952> 2014-08-13 17:23:20.214228 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -7951> 2014-08-13 17:23:20.214504 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.341737 2 0.000059 -7950> 2014-08-13 17:23:20.214526 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -7949> 2014-08-13 17:23:20.214535 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -7948> 2014-08-13 17:23:20.214544 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7947> 2014-08-13 17:23:20.214557 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 -7946> 2014-08-13 17:23:20.214567 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -7945> 2014-08-13 17:23:20.214686 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.312416 2 0.000060 -7944> 2014-08-13 17:23:20.214703 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started -7943> 2014-08-13 17:23:20.214713 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start -7942> 2014-08-13 17:23:20.214721 7fcd01e40700 1 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7941> 2014-08-13 17:23:20.214731 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 -7940> 2014-08-13 17:23:20.214745 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7939> 2014-08-13 17:23:20.214755 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7938> 2014-08-13 17:23:20.214765 7fcd01e40700 5 osd.4 pg_epoch: 194472 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7937> 2014-08-13 17:23:20.214924 7fcd1d4027a0 0 osd.4 194472 done with init, starting boot process -7936> 2014-08-13 17:23:20.214934 7fcd1d4027a0 10 monclient: get_version osdmap req 0x3addc00 -7935> 2014-08-13 17:23:20.214937 7fcd1d4027a0 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -7934> 2014-08-13 17:23:20.214943 7fcd1d4027a0 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- ?+0 0x2fa0d00 con 0x4e25d80 -7933> 2014-08-13 17:23:20.215321 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 10 ==== mon_check_map_ack(handle=1 version=194475) v2 ==== 24+0+0 (337843585 0 0) 0x4e51500 con 0x4e25d80 -7932> 2014-08-13 17:23:20.215340 7fcd0aa4e700 10 monclient: handle_get_version_reply finishing 0x3addc00 version 194475 -7931> 2014-08-13 17:23:20.215400 7fcd1d4027a0 5 asok(0x2e78230) register_command status hook 0x4529cb0 -7930> 2014-08-13 17:23:20.215517 7fcd1d4027a0 5 asok(0x2e78230) register_command flush_journal hook 0x4529cb0 -7929> 2014-08-13 17:23:20.215526 7fcd1d4027a0 5 asok(0x2e78230) register_command dump_ops_in_flight hook 0x4529cb0 -7928> 2014-08-13 17:23:20.215534 7fcd1d4027a0 5 asok(0x2e78230) register_command dump_historic_ops hook 0x4529cb0 -7927> 2014-08-13 17:23:20.215540 7fcd1d4027a0 5 asok(0x2e78230) register_command dump_op_pq_state hook 0x4529cb0 -7926> 2014-08-13 17:23:20.215548 7fcd1d4027a0 5 asok(0x2e78230) register_command dump_blacklist hook 0x4529cb0 -7925> 2014-08-13 17:23:20.215555 7fcd1d4027a0 5 asok(0x2e78230) register_command dump_watchers hook 0x4529cb0 -7924> 2014-08-13 17:23:20.215559 7fcd1d4027a0 5 asok(0x2e78230) register_command setomapval hook 0x3addba0 -7923> 2014-08-13 17:23:20.215562 7fcd1d4027a0 5 asok(0x2e78230) register_command rmomapkey hook 0x3addba0 -7922> 2014-08-13 17:23:20.215566 7fcd1d4027a0 5 asok(0x2e78230) register_command setomapheader hook 0x3addba0 -7921> 2014-08-13 17:23:20.215570 7fcd1d4027a0 5 asok(0x2e78230) register_command getomap hook 0x3addba0 -7920> 2014-08-13 17:23:20.215573 7fcd1d4027a0 5 asok(0x2e78230) register_command truncobj hook 0x3addba0 -7919> 2014-08-13 17:23:20.215576 7fcd1d4027a0 5 asok(0x2e78230) register_command injectdataerr hook 0x3addba0 -7918> 2014-08-13 17:23:20.215579 7fcd1d4027a0 5 asok(0x2e78230) register_command injectmdataerr hook 0x3addba0 -7917> 2014-08-13 17:23:20.215919 7fcd03242700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -7916> 2014-08-13 17:23:20.215931 7fcd03242700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- osd_boot(osd.4 booted 0 v194472) v5 -- ?+0 0x2e5ad00 con 0x4e25d80 -7915> 2014-08-13 17:23:20.854586 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 11 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x2e8a400 con 0x4e25d80 -7914> 2014-08-13 17:23:20.854657 7fcd0aa4e700 10 monclient: renew_subs -7913> 2014-08-13 17:23:20.854663 7fcd0aa4e700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -7912> 2014-08-13 17:23:20.854687 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- mon_subscribe({monmap=4+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x4e51500 con 0x4e25d80 -7911> 2014-08-13 17:23:20.854733 7fcd0aa4e700 3 osd.4 194472 handle_osd_map epochs [194473,194476], i have 194472, src has [185139,194476] -7910> 2014-08-13 17:23:20.864000 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x2e88fc0 con 0x4f86040 -7909> 2014-08-13 17:23:20.875565 7fcd0aa4e700 1 osd.4 194476 state: booting -> active -7908> 2014-08-13 17:23:20.875699 7fcd0aa4e700 0 osd.4 194476 crush map has features 2578087936000, adjusting msgr requires for mons -7907> 2014-08-13 17:23:20.875965 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.669115 3 0.000102 -7906> 2014-08-13 17:23:20.876017 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started 0.669211 0 0.000000 -7905> 2014-08-13 17:23:20.876046 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194470/194470/192936) [54,331,4] r=2 lpr=194471 pi=192980-194469/19 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset -7904> 2014-08-13 17:23:20.876240 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.738127 6 0.000275 -7903> 2014-08-13 17:23:20.876281 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter Started -7902> 2014-08-13 17:23:20.876280 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.000235 4 0.000199 -7901> 2014-08-13 17:23:20.876305 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter Start -7900> 2014-08-13 17:23:20.876308 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started -7899> 2014-08-13 17:23:20.876335 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -7898> 2014-08-13 17:23:20.876340 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start -7897> 2014-08-13 17:23:20.876357 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7896> 2014-08-13 17:23:20.876362 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000057 0 0.000000 -7895> 2014-08-13 17:23:20.876382 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 -7894> 2014-08-13 17:23:20.876385 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194466/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -7893> 2014-08-13 17:23:20.876407 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194471/194471 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray -7892> 2014-08-13 17:23:20.876656 7fccf0291700 1 -- 192.168.100.115:0/31924 learned my addr 192.168.100.115:0/31924 -7891> 2014-08-13 17:23:20.881140 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6830/52879 -- pg_notify(24.30b(29) epoch 194476) v5 -- ?+0 0x4e508c0 con 0x4fd2520 -7890> 2014-08-13 17:23:20.881297 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.677255 3 0.000231 -7889> 2014-08-13 17:23:20.881337 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.677383 0 0.000000 -7888> 2014-08-13 17:23:20.881367 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194470/194470/192967) [351,4,18] r=1 lpr=194471 pi=192970-194469/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -7887> 2014-08-13 17:23:20.881587 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] exit Reset 0.000220 4 0.000194 -7886> 2014-08-13 17:23:20.881619 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] enter Started -7885> 2014-08-13 17:23:20.881641 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] enter Start -7884> 2014-08-13 17:23:20.881667 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7883> 2014-08-13 17:23:20.881690 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] exit Start 0.000050 0 0.000000 -7882> 2014-08-13 17:23:20.881713 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194471/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] enter Started/Stray -7881> 2014-08-13 17:23:20.882745 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(25) epoch 194476) v5 -- ?+0 0x4e501c0 con 0x4fdee00 -7880> 2014-08-13 17:23:20.882837 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7879> 2014-08-13 17:23:20.882885 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.678539 2 0.000150 -7878> 2014-08-13 17:23:20.882916 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.678580 0 0.000000 -7877> 2014-08-13 17:23:20.882950 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.678623 0 0.000000 -7876> 2014-08-13 17:23:20.882975 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started 0.678711 0 0.000000 -7875> 2014-08-13 17:23:20.882998 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,58,343] r=0 lpr=194471 pi=193306-194469/23 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset -7874> 2014-08-13 17:23:20.907466 7fcd0aa4e700 1 osd.4 194476 pausing recovery (NORECOVER flag set) -7873> 2014-08-13 17:23:20.907478 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.918574 6 0.000136 -7872> 2014-08-13 17:23:20.907543 7fcd0964c700 10 monclient: renew_subs -7871> 2014-08-13 17:23:20.907532 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 12 ==== osd_map(194476..194476 src has 185139..194476) v3 ==== 268990+0+0 (1546163167 0 0) 0x4f7f980 con 0x4e25d80 -7870> 2014-08-13 17:23:20.907536 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started -7869> 2014-08-13 17:23:20.907487 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] exit Reset 0.024489 4 0.000290 -7868> 2014-08-13 17:23:20.907554 7fcd0964c700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -7867> 2014-08-13 17:23:20.907597 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] enter Started -7866> 2014-08-13 17:23:20.907622 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] enter Start -7865> 2014-08-13 17:23:20.907618 7fcd0964c700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- mon_subscribe({monmap=4+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x5185240 con 0x4e25d80 -7864> 2014-08-13 17:23:20.907639 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7863> 2014-08-13 17:23:20.907656 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7862> 2014-08-13 17:23:20.907665 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] exit Start 0.000043 0 0.000000 -7861> 2014-08-13 17:23:20.907687 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 2 ==== pg_notify(24.3d(27) epoch 194476) v5 ==== 2265+0+0 (1442779177 0 0) 0x4e55400 con 0x4f86040 -7860> 2014-08-13 17:23:20.907691 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] enter Started/Stray -7859> 2014-08-13 17:23:20.907561 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Start -7858> 2014-08-13 17:23:20.907736 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary -7857> 2014-08-13 17:23:20.907776 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000215 0 0.000000 -7856> 2014-08-13 17:23:20.907805 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary -7855> 2014-08-13 17:23:20.907828 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering -7854> 2014-08-13 17:23:20.907850 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194466/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo -7853> 2014-08-13 17:23:20.908241 7fccdd363700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308a00 sd=355 :6819 s=0 pgs=0 cs=0 l=0 c=0x5303440).accept sd=355 192.168.1.109:45396/0 -7852> 2014-08-13 17:23:20.908365 7fccdd767700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308500 sd=350 :6819 s=0 pgs=0 cs=0 l=0 c=0x5300b00).accept sd=350 192.168.1.105:33540/0 -7851> 2014-08-13 17:23:20.908374 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.703797 3 0.000107 -7850> 2014-08-13 17:23:20.908409 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.703879 0 0.000000 -7849> 2014-08-13 17:23:20.908434 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194470/194470/192972) [9,292,4] r=2 lpr=194471 pi=192972-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -7848> 2014-08-13 17:23:20.908291 7fccdd262700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308780 sd=356 :6819 s=0 pgs=0 cs=0 l=0 c=0x5305ee0).accept sd=356 192.168.1.106:43522/0 -7847> 2014-08-13 17:23:20.908921 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(21) epoch 194476) v5 -- ?+0 0x4e50e00 con 0x4ec2aa0 -7846> 2014-08-13 17:23:20.908962 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6820/25633 -- pg_query(24.9b epoch 194476) v3 -- ?+0 0x2f905a0 con 0x4ec27e0 -7845> 2014-08-13 17:23:20.908988 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6832/53185 -- pg_query(24.9b epoch 194476) v3 -- ?+0 0x2f90960 con 0x53056a0 -7844> 2014-08-13 17:23:20.909586 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.001152 4 0.000158 -7843> 2014-08-13 17:23:20.909700 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -7842> 2014-08-13 17:23:20.909724 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -7841> 2014-08-13 17:23:20.909753 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7840> 2014-08-13 17:23:20.909776 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000052 0 0.000000 -7839> 2014-08-13 17:23:20.909799 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194471/194471 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -7838> 2014-08-13 17:23:20.908711 7fccddc6c700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308000 sd=344 :6819 s=0 pgs=0 cs=0 l=0 c=0x5300840).accept sd=344 192.168.1.102:57723/0 -7837> 2014-08-13 17:23:20.909369 7fccdd464700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308c80 sd=354 :6819 s=0 pgs=0 cs=0 l=0 c=0x5303700).accept sd=354 192.168.1.101:48258/0 -7836> 2014-08-13 17:23:20.910480 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6824/15623 -- pg_query(24.9b epoch 194476) v3 -- ?+0 0x2f95640 con 0x53053e0 -7835> 2014-08-13 17:23:20.910508 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6840/55513 -- pg_query(24.9b epoch 194476) v3 -- ?+0 0x2f95460 con 0x4fd3860 -7834> 2014-08-13 17:23:20.908092 7fccdd565700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5308280 sd=353 :6819 s=0 pgs=0 cs=0 l=0 c=0x53011e0).accept sd=353 192.168.1.108:44632/0 -7833> 2014-08-13 17:23:20.910947 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6840/19195 -- pg_query(24.9b epoch 194476) v3 -- ?+0 0x2f95280 con 0x5305120 -7832> 2014-08-13 17:23:20.911003 7fcd01e40700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -7831> 2014-08-13 17:23:20.911012 7fcd01e40700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- osd_alive(want up_thru 194476 have 194476) v1 -- ?+0 0x4e50540 con 0x4e25d80 -7830> 2014-08-13 17:23:20.911099 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.703530 3 0.000108 -7829> 2014-08-13 17:23:20.911139 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started 0.703616 0 0.000000 -7828> 2014-08-13 17:23:20.911160 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194470/194470/192979) [320,4,30] r=1 lpr=194471 pi=192979-194469/22 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset -7827> 2014-08-13 17:23:20.911377 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] exit Reset 0.000218 4 0.000182 -7826> 2014-08-13 17:23:20.911404 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] enter Started -7825> 2014-08-13 17:23:20.911420 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] enter Start -7824> 2014-08-13 17:23:20.911435 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7823> 2014-08-13 17:23:20.911454 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] exit Start 0.000033 0 0.000000 -7822> 2014-08-13 17:23:20.911472 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194471/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] enter Started/Stray -7821> 2014-08-13 17:23:20.911621 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7820> 2014-08-13 17:23:20.911649 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.707667 2 0.000278 -7819> 2014-08-13 17:23:20.911676 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.707716 0 0.000000 -7818> 2014-08-13 17:23:20.911695 7fcd0aa4e700 3 osd.4 194476 handle_osd_map epochs [194476,194476], i have 194476, src has [185139,194476] -7817> 2014-08-13 17:23:20.911700 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.707759 0 0.000000 -7816> 2014-08-13 17:23:20.911725 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started 0.707867 0 0.000000 -7815> 2014-08-13 17:23:20.911729 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 13 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x4e51340 con 0x4e25d80 -7814> 2014-08-13 17:23:20.911747 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,312,285] r=0 lpr=194471 pi=193306-194469/19 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset -7813> 2014-08-13 17:23:20.911758 7fcd0aa4e700 10 monclient: handle_subscribe_ack sent 2014-08-13 17:23:20.854661 renew after 2014-08-13 17:25:50.854661 -7812> 2014-08-13 17:23:20.911896 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000148 4 0.000223 -7811> 2014-08-13 17:23:20.911918 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started -7810> 2014-08-13 17:23:20.911923 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.863943, event: header_read, request: pg_notify(24.3d(27) epoch 194476) v5 -7809> 2014-08-13 17:23:20.911944 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.863944, event: throttled, request: pg_notify(24.3d(27) epoch 194476) v5 -7808> 2014-08-13 17:23:20.911936 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start -7807> 2014-08-13 17:23:20.911952 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.864077, event: all_read, request: pg_notify(24.3d(27) epoch 194476) v5 -7806> 2014-08-13 17:23:20.911934 7fccddc6c700 0 -- 192.168.1.115:6819/31924 >> 192.168.1.102:6840/19195 pipe(0x5308000 sd=344 :6819 s=0 pgs=0 cs=0 l=0 c=0x5300840).accept connect_seq 0 vs existing 0 state connecting -7805> 2014-08-13 17:23:20.911959 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.907713, event: dispatched, request: pg_notify(24.3d(27) epoch 194476) v5 -7804> 2014-08-13 17:23:20.911955 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7803> 2014-08-13 17:23:20.911968 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.911968, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194476) v5 -7802> 2014-08-13 17:23:20.911974 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -7801> 2014-08-13 17:23:20.911986 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.911986, event: started, request: pg_notify(24.3d(27) epoch 194476) v5 -7800> 2014-08-13 17:23:20.911992 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7799> 2014-08-13 17:23:20.911986 7fccdc959700 2 -- 192.168.1.115:6819/31924 >> 192.168.1.102:6840/19195 pipe(0x4feb980 sd=359 :54811 s=1 pgs=0 cs=0 l=0 c=0x5305120). got newly_acked_seq 0 vs out_seq 0 -7798> 2014-08-13 17:23:20.912011 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7797> 2014-08-13 17:23:20.912003 7fccddc6c700 2 -- 192.168.1.115:6819/31924 >> 192.168.1.102:6840/19195 pipe(0x5308000 sd=344 :6819 s=4 pgs=0 cs=0 l=0 c=0x5300840).fault (0) Success -7796> 2014-08-13 17:23:20.912030 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7795> 2014-08-13 17:23:20.912211 7fcd0964c700 5 -- op tracker -- , seq: 0, time: 2014-08-13 17:23:20.912211, event: done, request: pg_notify(24.3d(27) epoch 194476) v5 -7794> 2014-08-13 17:23:20.912254 7fccdc454700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5308f00 sd=360 :6808 s=0 pgs=0 cs=0 l=0 c=0x5306e00).accept sd=360 192.168.100.110:39374/0 -7793> 2014-08-13 17:23:20.912321 7fcd0964c700 1 osd.4 194476 ms_handle_reset con 0x5300840 session 0x30210a0 -7792> 2014-08-13 17:23:20.912394 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(25) epoch 194476) v5 -- ?+0 0x5187e00 con 0x4fde300 -7791> 2014-08-13 17:23:20.912429 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_query(22.f3 epoch 194476) v3 -- ?+0 0x2f950a0 con 0x4e314a0 -7790> 2014-08-13 17:23:20.912460 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.707450 3 0.000157 -7789> 2014-08-13 17:23:20.912505 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.707554 0 0.000000 -7788> 2014-08-13 17:23:20.912529 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192935) [58,28,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset -7787> 2014-08-13 17:23:20.912631 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e4140 con 0x4f83860 -7786> 2014-08-13 17:23:20.912669 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7785> 2014-08-13 17:23:20.912692 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bb3c0 con 0x4fd9a20 -7784> 2014-08-13 17:23:20.912712 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7783> 2014-08-13 17:23:20.912727 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e1440 con 0x4fd9760 -7782> 2014-08-13 17:23:20.912746 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7781> 2014-08-13 17:23:20.912736 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.000208 4 0.000164 -7780> 2014-08-13 17:23:20.912759 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e2f40 con 0x4f86e00 -7779> 2014-08-13 17:23:20.912766 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -7778> 2014-08-13 17:23:20.912778 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7777> 2014-08-13 17:23:20.912788 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -7776> 2014-08-13 17:23:20.912795 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bb600 con 0x4fdad60 -7775> 2014-08-13 17:23:20.912811 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7774> 2014-08-13 17:23:20.912806 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7773> 2014-08-13 17:23:20.912825 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e4380 con 0x4fdee00 -7772> 2014-08-13 17:23:20.912826 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000038 0 0.000000 -7771> 2014-08-13 17:23:20.912844 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7770> 2014-08-13 17:23:20.912845 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -7769> 2014-08-13 17:23:20.912857 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b9200 con 0x4fde300 -7768> 2014-08-13 17:23:20.912874 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7767> 2014-08-13 17:23:20.912888 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e2d00 con 0x4fd3860 -7766> 2014-08-13 17:23:20.912906 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7765> 2014-08-13 17:23:20.912919 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bbf00 con 0x4fd5960 -7764> 2014-08-13 17:23:20.912938 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7763> 2014-08-13 17:23:20.912952 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e45c0 con 0x4fd32e0 -7762> 2014-08-13 17:23:20.912969 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7761> 2014-08-13 17:23:20.912981 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e5c40 con 0x4fd53e0 -7760> 2014-08-13 17:23:20.912999 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7759> 2014-08-13 17:23:20.913012 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e69c0 con 0x4fd2520 -7758> 2014-08-13 17:23:20.913030 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7757> 2014-08-13 17:23:20.913043 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x52bd580 con 0x53011e0 -7756> 2014-08-13 17:23:20.913060 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7755> 2014-08-13 17:23:20.913073 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x52bcec0 con 0x5303440 -7754> 2014-08-13 17:23:20.913091 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7753> 2014-08-13 17:23:20.913104 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bcc80 con 0x5305ee0 -7752> 2014-08-13 17:23:20.913122 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7751> 2014-08-13 17:23:20.913134 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e0240 con 0x4e335a0 -7750> 2014-08-13 17:23:20.913152 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7749> 2014-08-13 17:23:20.913176 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bca40 con 0x4e32940 -7748> 2014-08-13 17:23:20.913194 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7747> 2014-08-13 17:23:20.913206 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52be540 con 0x4e33de0 -7746> 2014-08-13 17:23:20.913224 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7745> 2014-08-13 17:23:20.913237 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e2640 con 0x4e33f40 -7744> 2014-08-13 17:23:20.913255 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7743> 2014-08-13 17:23:20.913268 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e1d40 con 0x4e35ac0 -7742> 2014-08-13 17:23:20.913287 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7741> 2014-08-13 17:23:20.913299 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b9f80 con 0x4e35800 -7740> 2014-08-13 17:23:20.913316 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7739> 2014-08-13 17:23:20.913330 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b8480 con 0x4ec09a0 -7738> 2014-08-13 17:23:20.913349 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7737> 2014-08-13 17:23:20.913361 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bad00 con 0x4ec3f40 -7736> 2014-08-13 17:23:20.913380 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7735> 2014-08-13 17:23:20.913392 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e6300 con 0x4ec56a0 -7734> 2014-08-13 17:23:20.913410 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7733> 2014-08-13 17:23:20.913421 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b98c0 con 0x4ec4780 -7732> 2014-08-13 17:23:20.913438 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7731> 2014-08-13 17:23:20.913451 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x52bd7c0 con 0x4ec5d80 -7730> 2014-08-13 17:23:20.913469 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7729> 2014-08-13 17:23:20.913482 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e7080 con 0x4ec5ac0 -7728> 2014-08-13 17:23:20.913502 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7727> 2014-08-13 17:23:20.913515 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e0000 con 0x4eb0c60 -7726> 2014-08-13 17:23:20.913533 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7725> 2014-08-13 17:23:20.913546 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b86c0 con 0x4eb23c0 -7724> 2014-08-13 17:23:20.913563 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7723> 2014-08-13 17:23:20.913576 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52ba1c0 con 0x4eb18c0 -7722> 2014-08-13 17:23:20.913594 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7721> 2014-08-13 17:23:20.913605 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e72c0 con 0x4eb7d20 -7720> 2014-08-13 17:23:20.913623 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7719> 2014-08-13 17:23:20.913635 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.296 192.168.1.109:6806/31286 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bb840 con 0x4f81600 -7718> 2014-08-13 17:23:20.913652 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7717> 2014-08-13 17:23:20.913664 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e7500 con 0x4f80b00 -7716> 2014-08-13 17:23:20.913683 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7715> 2014-08-13 17:23:20.913694 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x51e3a80 con 0x4f832e0 -7714> 2014-08-13 17:23:20.913712 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7713> 2014-08-13 17:23:20.913724 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.19 192.168.1.115:6818/61186 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x51e4800 con 0x4f84ba0 -7712> 2014-08-13 17:23:20.913742 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7711> 2014-08-13 17:23:20.913917 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 2 ==== pg_query(24.1de epoch 194476) v3 ==== 144+0+0 (2822274681 0 0) 0x2f91e00 con 0x4f83860 -7710> 2014-08-13 17:23:20.913957 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.892405, event: header_read, request: pg_query(24.1de epoch 194476) v3 -7709> 2014-08-13 17:23:20.913968 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.892408, event: throttled, request: pg_query(24.1de epoch 194476) v3 -7708> 2014-08-13 17:23:20.913974 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.892467, event: all_read, request: pg_query(24.1de epoch 194476) v3 -7707> 2014-08-13 17:23:20.913981 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.913949, event: dispatched, request: pg_query(24.1de epoch 194476) v3 -7706> 2014-08-13 17:23:20.913988 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.913988, event: waiting_for_osdmap, request: pg_query(24.1de epoch 194476) v3 -7705> 2014-08-13 17:23:20.913997 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.913997, event: started, request: pg_query(24.1de epoch 194476) v3 -7704> 2014-08-13 17:23:20.913987 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6816/24270 -- pg_query(22.f3 epoch 194476) v3 -- ?+0 0x2f96ae0 con 0x53077a0 -7703> 2014-08-13 17:23:20.914317 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6816/17521 -- pg_query(22.f3 epoch 194476) v3 -- ?+0 0x2f96900 con 0x5307a60 -7702> 2014-08-13 17:23:20.914363 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7701> 2014-08-13 17:23:20.914402 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.708399 2 0.000226 -7700> 2014-08-13 17:23:20.914430 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.708455 0 0.000000 -7699> 2014-08-13 17:23:20.914296 7fccdbb4b700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530aa80 sd=344 :6808 s=0 pgs=0 cs=0 l=0 c=0x5306f60).accept sd=344 192.168.100.113:57984/0 -7698> 2014-08-13 17:23:20.914452 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.708493 0 0.000000 -7697> 2014-08-13 17:23:20.914474 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started 0.708609 0 0.000000 -7696> 2014-08-13 17:23:20.914496 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194470/194470/194470) [4,301,324] r=0 lpr=194471 pi=193306-194469/21 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset -7695> 2014-08-13 17:23:20.914616 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6824/4037 -- pg_query(22.f3 epoch 194476) v3 -- ?+0 0x2f96720 con 0x552a260 -7694> 2014-08-13 17:23:20.914657 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000161 4 0.000231 -7693> 2014-08-13 17:23:20.914685 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started -7692> 2014-08-13 17:23:20.914701 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start -7691> 2014-08-13 17:23:20.914702 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.710260 3 0.000120 -7690> 2014-08-13 17:23:20.914721 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7689> 2014-08-13 17:23:20.914732 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started 0.710339 0 0.000000 -7688> 2014-08-13 17:23:20.914742 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000041 0 0.000000 -7687> 2014-08-13 17:23:20.914753 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194470/194470/192981) [345,4,374] r=1 lpr=194471 pi=192981-194469/20 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset -7686> 2014-08-13 17:23:20.914738 7fccdb949700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x530b700 sd=362 :6819 s=0 pgs=0 cs=0 l=0 c=0x5305c20).accept sd=362 192.168.1.110:45631/0 -7685> 2014-08-13 17:23:20.914763 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7684> 2014-08-13 17:23:20.914782 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7683> 2014-08-13 17:23:20.914804 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194465/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7682> 2014-08-13 17:23:20.914822 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 14 ==== osd_map(194476..194476 src has 185139..194476) v3 ==== 268990+0+0 (1546163167 0 0) 0x4f7f740 con 0x4e25d80 -7681> 2014-08-13 17:23:20.914922 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.000168 4 0.000140 -7680> 2014-08-13 17:23:20.914955 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started -7679> 2014-08-13 17:23:20.914971 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start -7678> 2014-08-13 17:23:20.914967 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7677> 2014-08-13 17:23:20.914987 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7676> 2014-08-13 17:23:20.914992 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.708558 2 0.000436 -7675> 2014-08-13 17:23:20.915005 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 -7674> 2014-08-13 17:23:20.915013 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.708629 0 0.000000 -7673> 2014-08-13 17:23:20.915024 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194471/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray -7672> 2014-08-13 17:23:20.915035 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.708663 0 0.000000 -7671> 2014-08-13 17:23:20.915054 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started 0.708785 0 0.000000 -7670> 2014-08-13 17:23:20.915072 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194470/194470/194470) [4,65,62] r=0 lpr=194471 pi=193306-194469/23 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset -7669> 2014-08-13 17:23:20.915280 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] exit Reset 0.000208 4 0.000203 -7668> 2014-08-13 17:23:20.915295 7fcd0964c700 5 -- op tracker -- , seq: 1, time: 2014-08-13 17:23:20.915295, event: done, request: pg_query(24.1de epoch 194476) v3 -7667> 2014-08-13 17:23:20.915320 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] enter Started -7666> 2014-08-13 17:23:20.915324 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Reset 0.943654 6 0.000126 -7665> 2014-08-13 17:23:20.915338 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] enter Start -7664> 2014-08-13 17:23:20.915347 7fcd0aa4e700 3 osd.4 194476 handle_osd_map epochs [194476,194476], i have 194476, src has [185139,194476] -7663> 2014-08-13 17:23:20.915345 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 2 ==== pg_notify(22.113(24) epoch 194476) v5 ==== 2062+0+0 (804026585 0 0) 0x5186e40 con 0x4fd9a20 -7662> 2014-08-13 17:23:20.915353 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started -7661> 2014-08-13 17:23:20.915355 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7660> 2014-08-13 17:23:20.915368 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.908151, event: header_read, request: pg_notify(22.113(24) epoch 194476) v5 -7659> 2014-08-13 17:23:20.915377 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.908154, event: throttled, request: pg_notify(22.113(24) epoch 194476) v5 -7658> 2014-08-13 17:23:20.915370 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 15 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (96432707 0 0) 0x4e50540 con 0x4e25d80 -7657> 2014-08-13 17:23:20.915373 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Start -7656> 2014-08-13 17:23:20.915385 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.908296, event: all_read, request: pg_notify(22.113(24) epoch 194476) v5 -7655> 2014-08-13 17:23:20.915376 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] exit Start 0.000039 0 0.000000 -7654> 2014-08-13 17:23:20.915389 7fcd0aa4e700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -7653> 2014-08-13 17:23:20.915392 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.915365, event: dispatched, request: pg_notify(22.113(24) epoch 194476) v5 -7652> 2014-08-13 17:23:20.915400 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.915399, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194476) v5 -7651> 2014-08-13 17:23:20.915392 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: transitioning to Primary -7650> 2014-08-13 17:23:20.915410 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.915409, event: started, request: pg_notify(22.113(24) epoch 194476) v5 -7649> 2014-08-13 17:23:20.915399 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194466/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] enter Started/Stray -7648> 2014-08-13 17:23:20.915423 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Start 0.000050 0 0.000000 -7647> 2014-08-13 17:23:20.915438 7fcd0964c700 5 -- op tracker -- , seq: 2, time: 2014-08-13 17:23:20.915438, event: done, request: pg_notify(22.113(24) epoch 194476) v5 -7646> 2014-08-13 17:23:20.915445 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary -7645> 2014-08-13 17:23:20.915460 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 2 ==== pg_query(22.619 epoch 194476) v3 ==== 144+0+0 (3083030671 0 0) 0x2f91680 con 0x4fd9760 -7644> 2014-08-13 17:23:20.915463 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Peering -7643> 2014-08-13 17:23:20.915476 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.880342, event: header_read, request: pg_query(22.619 epoch 194476) v3 -7642> 2014-08-13 17:23:20.915484 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.880343, event: throttled, request: pg_query(22.619 epoch 194476) v3 -7641> 2014-08-13 17:23:20.915490 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.880440, event: all_read, request: pg_query(22.619 epoch 194476) v3 -7640> 2014-08-13 17:23:20.915481 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194466/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetInfo -7639> 2014-08-13 17:23:20.915495 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.915473, event: dispatched, request: pg_query(22.619 epoch 194476) v3 -7638> 2014-08-13 17:23:20.915502 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.915502, event: waiting_for_osdmap, request: pg_query(22.619 epoch 194476) v3 -7637> 2014-08-13 17:23:20.915513 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.915513, event: started, request: pg_query(22.619 epoch 194476) v3 -7636> 2014-08-13 17:23:20.915517 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(21) epoch 194476) v5 -- ?+0 0x4e50a80 con 0x4e34780 -7635> 2014-08-13 17:23:20.915531 7fcd0964c700 5 -- op tracker -- , seq: 3, time: 2014-08-13 17:23:20.915531, event: done, request: pg_query(22.619 epoch 194476) v3 -7634> 2014-08-13 17:23:20.915550 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_notify(22.362(26),22.14a(21) epoch 194476) v5 -- ?+0 0x54d0700 con 0x4e335a0 -7633> 2014-08-13 17:23:20.915547 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 2 ==== pg_notify(22.3d5(1) epoch 194476) v5 ==== 769+0+0 (767481504 0 0) 0x5184980 con 0x4f86e00 -7632> 2014-08-13 17:23:20.915563 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.886100, event: header_read, request: pg_notify(22.3d5(1) epoch 194476) v5 -7631> 2014-08-13 17:23:20.915570 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.886102, event: throttled, request: pg_notify(22.3d5(1) epoch 194476) v5 -7630> 2014-08-13 17:23:20.915571 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_notify(22.619(26) epoch 194476) v5 -- ?+0 0x54d0540 con 0x4fd9760 -7629> 2014-08-13 17:23:20.915576 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.886187, event: all_read, request: pg_notify(22.3d5(1) epoch 194476) v5 -7628> 2014-08-13 17:23:20.915583 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.915561, event: dispatched, request: pg_notify(22.3d5(1) epoch 194476) v5 -7627> 2014-08-13 17:23:20.915589 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.915589, event: waiting_for_osdmap, request: pg_notify(22.3d5(1) epoch 194476) v5 -7626> 2014-08-13 17:23:20.915589 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6814/16727 -- pg_query(22.22c epoch 194476) v3 -- ?+0 0x2f903c0 con 0x4e35800 -7625> 2014-08-13 17:23:20.915597 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.915596, event: started, request: pg_notify(22.3d5(1) epoch 194476) v5 -7624> 2014-08-13 17:23:20.915923 7fcd0964c700 5 -- op tracker -- , seq: 4, time: 2014-08-13 17:23:20.915923, event: done, request: pg_notify(22.3d5(1) epoch 194476) v5 -7623> 2014-08-13 17:23:20.915956 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e2640 con 0x4fd8f20 -7622> 2014-08-13 17:23:20.915979 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7621> 2014-08-13 17:23:20.915991 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 2 ==== pg_notify(22.3d5(22) epoch 194476) v5 ==== 1948+0+0 (1599452933 0 0) 0x5187000 con 0x4fdad60 -7620> 2014-08-13 17:23:20.916006 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.911028, event: header_read, request: pg_notify(22.3d5(22) epoch 194476) v5 -7619> 2014-08-13 17:23:20.916014 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.911030, event: throttled, request: pg_notify(22.3d5(22) epoch 194476) v5 -7618> 2014-08-13 17:23:20.916021 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.911149, event: all_read, request: pg_notify(22.3d5(22) epoch 194476) v5 -7617> 2014-08-13 17:23:20.916031 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.916003, event: dispatched, request: pg_notify(22.3d5(22) epoch 194476) v5 -7616> 2014-08-13 17:23:20.916038 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.916038, event: waiting_for_osdmap, request: pg_notify(22.3d5(22) epoch 194476) v5 -7615> 2014-08-13 17:23:20.916047 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.916047, event: started, request: pg_notify(22.3d5(22) epoch 194476) v5 -7614> 2014-08-13 17:23:20.916069 7fcd0964c700 5 -- op tracker -- , seq: 5, time: 2014-08-13 17:23:20.916069, event: done, request: pg_notify(22.3d5(22) epoch 194476) v5 -7613> 2014-08-13 17:23:20.916085 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 2 ==== pg_query(22.220 epoch 194476) v3 ==== 144+0+0 (195137953 0 0) 0x2f921c0 con 0x4fdee00 -7612> 2014-08-13 17:23:20.916101 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.892453, event: header_read, request: pg_query(22.220 epoch 194476) v3 -7611> 2014-08-13 17:23:20.916108 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.892454, event: throttled, request: pg_query(22.220 epoch 194476) v3 -7610> 2014-08-13 17:23:20.916114 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.892706, event: all_read, request: pg_query(22.220 epoch 194476) v3 -7609> 2014-08-13 17:23:20.916097 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.711374 3 0.000198 -7608> 2014-08-13 17:23:20.916124 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.916099, event: dispatched, request: pg_query(22.220 epoch 194476) v3 -7607> 2014-08-13 17:23:20.916131 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.916131, event: waiting_for_osdmap, request: pg_query(22.220 epoch 194476) v3 -7606> 2014-08-13 17:23:20.916143 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.916143, event: started, request: pg_query(22.220 epoch 194476) v3 -7605> 2014-08-13 17:23:20.916134 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started 0.711451 0 0.000000 -7604> 2014-08-13 17:23:20.916179 7fcd0964c700 5 -- op tracker -- , seq: 6, time: 2014-08-13 17:23:20.916178, event: done, request: pg_query(22.220 epoch 194476) v3 -7603> 2014-08-13 17:23:20.916198 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/192971) [321,379,4] r=2 lpr=194471 pi=192971-194469/19 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset -7602> 2014-08-13 17:23:20.916225 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 2 ==== pg_query(22.581 epoch 194476) v3 ==== 144+0+0 (254423670 0 0) 0x2f93fc0 con 0x4fde300 -7601> 2014-08-13 17:23:20.916257 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.900242, event: header_read, request: pg_query(22.581 epoch 194476) v3 -7600> 2014-08-13 17:23:20.916272 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.900243, event: throttled, request: pg_query(22.581 epoch 194476) v3 -7599> 2014-08-13 17:23:20.916285 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.900394, event: all_read, request: pg_query(22.581 epoch 194476) v3 -7598> 2014-08-13 17:23:20.916296 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.916254, event: dispatched, request: pg_query(22.581 epoch 194476) v3 -7597> 2014-08-13 17:23:20.916306 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.916306, event: waiting_for_osdmap, request: pg_query(22.581 epoch 194476) v3 -7596> 2014-08-13 17:23:20.916320 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.916320, event: started, request: pg_query(22.581 epoch 194476) v3 -7595> 2014-08-13 17:23:20.916350 7fcd0964c700 5 -- op tracker -- , seq: 7, time: 2014-08-13 17:23:20.916349, event: done, request: pg_query(22.581 epoch 194476) v3 -7594> 2014-08-13 17:23:20.916210 7fccdba4a700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530ad00 sd=363 :6820 s=0 pgs=0 cs=0 l=0 c=0x552a940).accept sd=363 192.168.1.110:45979/0 -7593> 2014-08-13 17:23:20.916370 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x51e4380 con 0x4fde040 -7592> 2014-08-13 17:23:20.916391 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7591> 2014-08-13 17:23:20.916374 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.000176 4 0.000193 -7590> 2014-08-13 17:23:20.916412 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 2 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e33c0 con 0x4fd3860 -7589> 2014-08-13 17:23:20.916423 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started -7588> 2014-08-13 17:23:20.916437 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7587> 2014-08-13 17:23:20.916450 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start -7586> 2014-08-13 17:23:20.916467 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e2f40 con 0x4fd0f20 -7585> 2014-08-13 17:23:20.916469 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7584> 2014-08-13 17:23:20.916493 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7583> 2014-08-13 17:23:20.916290 7fccdb848700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530bc00 sd=365 :6820 s=0 pgs=0 cs=0 l=0 c=0x552aaa0).accept sd=365 192.168.1.113:56163/0 -7582> 2014-08-13 17:23:20.916503 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000053 0 0.000000 -7581> 2014-08-13 17:23:20.916513 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e45c0 con 0x4fd06e0 -7580> 2014-08-13 17:23:20.916529 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7579> 2014-08-13 17:23:20.916523 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray -7578> 2014-08-13 17:23:20.916544 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 2 ==== pg_query(22.226 epoch 194476) v3 ==== 144+0+0 (2021880267 0 0) 0x2f94ec0 con 0x4fd5960 -7577> 2014-08-13 17:23:20.916564 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.908960, event: header_read, request: pg_query(22.226 epoch 194476) v3 -7576> 2014-08-13 17:23:20.916575 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.908962, event: throttled, request: pg_query(22.226 epoch 194476) v3 -7575> 2014-08-13 17:23:20.916584 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.909450, event: all_read, request: pg_query(22.226 epoch 194476) v3 -7574> 2014-08-13 17:23:20.916594 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.916560, event: dispatched, request: pg_query(22.226 epoch 194476) v3 -7573> 2014-08-13 17:23:20.916610 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.916610, event: waiting_for_osdmap, request: pg_query(22.226 epoch 194476) v3 -7572> 2014-08-13 17:23:20.916620 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.916619, event: started, request: pg_query(22.226 epoch 194476) v3 -7571> 2014-08-13 17:23:20.916637 7fcd0964c700 5 -- op tracker -- , seq: 8, time: 2014-08-13 17:23:20.916637, event: done, request: pg_query(22.226 epoch 194476) v3 -7570> 2014-08-13 17:23:20.916655 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 2 ==== pg_notify(24.3d(26) epoch 194476) v5 ==== 2206+0+0 (3953936911 0 0) 0x4e501c0 con 0x4fd32e0 -7569> 2014-08-13 17:23:20.916677 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.892426, event: header_read, request: pg_notify(24.3d(26) epoch 194476) v5 -7568> 2014-08-13 17:23:20.916687 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.892428, event: throttled, request: pg_notify(24.3d(26) epoch 194476) v5 -7567> 2014-08-13 17:23:20.916694 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.892545, event: all_read, request: pg_notify(24.3d(26) epoch 194476) v5 -7566> 2014-08-13 17:23:20.916703 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.916674, event: dispatched, request: pg_notify(24.3d(26) epoch 194476) v5 -7565> 2014-08-13 17:23:20.916714 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.916714, event: waiting_for_osdmap, request: pg_notify(24.3d(26) epoch 194476) v5 -7564> 2014-08-13 17:23:20.916723 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.916723, event: started, request: pg_notify(24.3d(26) epoch 194476) v5 -7563> 2014-08-13 17:23:20.916751 7fcd0964c700 5 -- op tracker -- , seq: 9, time: 2014-08-13 17:23:20.916751, event: done, request: pg_notify(24.3d(26) epoch 194476) v5 -7562> 2014-08-13 17:23:20.916769 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e7080 con 0x4fd3020 -7561> 2014-08-13 17:23:20.916325 7fccdb545700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x530b480 sd=367 :6819 s=0 pgs=0 cs=0 l=0 c=0x5305960).accept sd=367 192.168.1.110:45632/0 -7560> 2014-08-13 17:23:20.916790 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7559> 2014-08-13 17:23:20.916807 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6810/15731 -- pg_query(22.22c epoch 194476) v3 -- ?+0 0x54f83c0 con 0x552b9c0 -7558> 2014-08-13 17:23:20.916823 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 2 ==== pg_query(22.3b9 epoch 194476) v3 ==== 144+0+0 (2497714507 0 0) 0x2f92580 con 0x4fd53e0 -7557> 2014-08-13 17:23:20.916838 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6814/65209 -- pg_query(22.22c epoch 194476) v3 -- ?+0 0x54f81e0 con 0x4fde5c0 -7556> 2014-08-13 17:23:20.916844 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.892986, event: header_read, request: pg_query(22.3b9 epoch 194476) v3 -7555> 2014-08-13 17:23:20.916861 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.892988, event: throttled, request: pg_query(22.3b9 epoch 194476) v3 -7554> 2014-08-13 17:23:20.916871 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.893036, event: all_read, request: pg_query(22.3b9 epoch 194476) v3 -7553> 2014-08-13 17:23:20.916881 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.916838, event: dispatched, request: pg_query(22.3b9 epoch 194476) v3 -7552> 2014-08-13 17:23:20.916895 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.916895, event: waiting_for_osdmap, request: pg_query(22.3b9 epoch 194476) v3 -7551> 2014-08-13 17:23:20.916873 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7550> 2014-08-13 17:23:20.916908 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.916908, event: started, request: pg_query(22.3b9 epoch 194476) v3 -7549> 2014-08-13 17:23:20.916909 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.711596 2 0.000223 -7548> 2014-08-13 17:23:20.916932 7fcd0964c700 5 -- op tracker -- , seq: 10, time: 2014-08-13 17:23:20.916932, event: done, request: pg_query(22.3b9 epoch 194476) v3 -7547> 2014-08-13 17:23:20.916940 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.711638 0 0.000000 -7546> 2014-08-13 17:23:20.916952 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 2 ==== pg_query(24.30b epoch 194476) v3 ==== 144+0+0 (4294900141 0 0) 0x2f930c0 con 0x4fd2520 -7545> 2014-08-13 17:23:20.916978 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.894734, event: header_read, request: pg_query(24.30b epoch 194476) v3 -7544> 2014-08-13 17:23:20.916966 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.711676 0 0.000000 -7543> 2014-08-13 17:23:20.916993 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.894735, event: throttled, request: pg_query(24.30b epoch 194476) v3 -7542> 2014-08-13 17:23:20.916999 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.894781, event: all_read, request: pg_query(24.30b epoch 194476) v3 -7541> 2014-08-13 17:23:20.917009 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.916973, event: dispatched, request: pg_query(24.30b epoch 194476) v3 -7540> 2014-08-13 17:23:20.916999 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started 0.711750 0 0.000000 -7539> 2014-08-13 17:23:20.917016 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.917016, event: waiting_for_osdmap, request: pg_query(24.30b epoch 194476) v3 -7538> 2014-08-13 17:23:20.917022 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194470/194470/194470) [4,349,61] r=0 lpr=194471 pi=193306-194469/21 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset -7537> 2014-08-13 17:23:20.917031 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.917031, event: started, request: pg_query(24.30b epoch 194476) v3 -7536> 2014-08-13 17:23:20.917050 7fcd0964c700 5 -- op tracker -- , seq: 11, time: 2014-08-13 17:23:20.917049, event: done, request: pg_query(24.30b epoch 194476) v3 -7535> 2014-08-13 17:23:20.917070 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 2 ==== pg_query(22.775 epoch 194476) v3 ==== 144+0+0 (459439588 0 0) 0x2f95be0 con 0x53011e0 -7534> 2014-08-13 17:23:20.917092 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.911920, event: header_read, request: pg_query(22.775 epoch 194476) v3 -7533> 2014-08-13 17:23:20.917105 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.911921, event: throttled, request: pg_query(22.775 epoch 194476) v3 -7532> 2014-08-13 17:23:20.917116 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.911976, event: all_read, request: pg_query(22.775 epoch 194476) v3 -7531> 2014-08-13 17:23:20.917121 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.917089, event: dispatched, request: pg_query(22.775 epoch 194476) v3 -7530> 2014-08-13 17:23:20.917131 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.917130, event: waiting_for_osdmap, request: pg_query(22.775 epoch 194476) v3 -7529> 2014-08-13 17:23:20.917143 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.917142, event: started, request: pg_query(22.775 epoch 194476) v3 -7528> 2014-08-13 17:23:20.917164 7fcd0964c700 5 -- op tracker -- , seq: 12, time: 2014-08-13 17:23:20.917164, event: done, request: pg_query(22.775 epoch 194476) v3 -7527> 2014-08-13 17:23:20.917178 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x51e0fc0 con 0x5303700 -7526> 2014-08-13 17:23:20.917174 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6826/15861 -- pg_query(22.22c epoch 194476) v3 -- ?+0 0x54f8000 con 0x552b700 -7525> 2014-08-13 17:23:20.917205 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7524> 2014-08-13 17:23:20.917204 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000182 4 0.000273 -7523> 2014-08-13 17:23:20.917218 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 2 ==== pg_query(22.707 epoch 194476) v3 ==== 144+0+0 (1831758493 0 0) 0x2f95dc0 con 0x5303440 -7522> 2014-08-13 17:23:20.917237 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.912109, event: header_read, request: pg_query(22.707 epoch 194476) v3 -7521> 2014-08-13 17:23:20.917244 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.912110, event: throttled, request: pg_query(22.707 epoch 194476) v3 -7520> 2014-08-13 17:23:20.917234 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started -7519> 2014-08-13 17:23:20.917250 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.912158, event: all_read, request: pg_query(22.707 epoch 194476) v3 -7518> 2014-08-13 17:23:20.917256 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.917234, event: dispatched, request: pg_query(22.707 epoch 194476) v3 -7517> 2014-08-13 17:23:20.917252 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start -7516> 2014-08-13 17:23:20.917262 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.917262, event: waiting_for_osdmap, request: pg_query(22.707 epoch 194476) v3 -7515> 2014-08-13 17:23:20.917272 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.917272, event: started, request: pg_query(22.707 epoch 194476) v3 -7514> 2014-08-13 17:23:20.917268 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7513> 2014-08-13 17:23:20.917290 7fcd0964c700 5 -- op tracker -- , seq: 13, time: 2014-08-13 17:23:20.917290, event: done, request: pg_query(22.707 epoch 194476) v3 -7512> 2014-08-13 17:23:20.917271 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7511> 2014-08-13 17:23:20.917288 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -7510> 2014-08-13 17:23:20.917311 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 2 ==== pg_notify(22.1d2(24) epoch 194476) v5 ==== 2062+0+0 (2408723064 0 0) 0x5187700 con 0x5305ee0 -7509> 2014-08-13 17:23:20.917305 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.707913 2 0.000241 -7508> 2014-08-13 17:23:20.917308 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7507> 2014-08-13 17:23:20.917327 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.912168, event: header_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -7506> 2014-08-13 17:23:20.917334 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.912169, event: throttled, request: pg_notify(22.1d2(24) epoch 194476) v5 -7505> 2014-08-13 17:23:20.917309 7fccdab3b700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5308000 sd=370 :6808 s=0 pgs=0 cs=0 l=0 c=0x552a680).accept sd=370 192.168.100.111:49046/0 -7504> 2014-08-13 17:23:20.917341 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.912415, event: all_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -7503> 2014-08-13 17:23:20.917334 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7502> 2014-08-13 17:23:20.917348 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.917325, event: dispatched, request: pg_notify(22.1d2(24) epoch 194476) v5 -7501> 2014-08-13 17:23:20.917335 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.707955 0 0.000000 -7500> 2014-08-13 17:23:20.917356 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.917356, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194476) v5 -7499> 2014-08-13 17:23:20.917365 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.917365, event: started, request: pg_notify(22.1d2(24) epoch 194476) v5 -7498> 2014-08-13 17:23:20.917355 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194466/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7497> 2014-08-13 17:23:20.917361 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.707993 0 0.000000 -7496> 2014-08-13 17:23:20.917384 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started 0.708145 0 0.000000 -7495> 2014-08-13 17:23:20.917405 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,353,56] r=0 lpr=194471 pi=193306-194469/23 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset -7494> 2014-08-13 17:23:20.917509 7fccdac3c700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530b980 sd=369 :6820 s=0 pgs=0 cs=0 l=0 c=0x552aec0).accept sd=369 192.168.1.111:58729/0 -7493> 2014-08-13 17:23:20.917542 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(21) epoch 194476) v5 -- ?+0 0x51878c0 con 0x4e35120 -7492> 2014-08-13 17:23:20.917577 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(22) epoch 194476) v5 -- ?+0 0x54d2140 con 0x4e32940 -7491> 2014-08-13 17:23:20.917599 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_query(24.3d epoch 194476) v3 -- ?+0 0x2f96360 con 0x4e33de0 -7490> 2014-08-13 17:23:20.917638 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] exit Reset 0.000232 4 0.000260 -7489> 2014-08-13 17:23:20.917669 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] enter Started -7488> 2014-08-13 17:23:20.917683 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6800/36920 -- pg_query(24.3d epoch 194476) v3 -- ?+0 0x54f90e0 con 0x552f7a0 -7487> 2014-08-13 17:23:20.917686 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] enter Start -7486> 2014-08-13 17:23:20.917702 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x54f8f00 con 0x4e335a0 -7485> 2014-08-13 17:23:20.917702 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7484> 2014-08-13 17:23:20.917721 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x54f8d20 con 0x4e35ac0 -7483> 2014-08-13 17:23:20.917722 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] exit Start 0.000036 0 0.000000 -7482> 2014-08-13 17:23:20.917738 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6802/30768 -- pg_query(24.3d epoch 194476) v3 -- ?+0 0x54f8b40 con 0x4fd06e0 -7481> 2014-08-13 17:23:20.917742 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] enter Started/Stray -7480> 2014-08-13 17:23:20.917774 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x54f8960 con 0x5305ee0 -7479> 2014-08-13 17:23:20.917793 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6816/837 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x54f8780 con 0x4fde880 -7478> 2014-08-13 17:23:20.917810 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6800/61726 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x54f9e00 con 0x552fa60 -7477> 2014-08-13 17:23:20.917828 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6842/8170 -- pg_query(24.3d epoch 194476) v3 -- ?+0 0x54f9c20 con 0x4f86040 -7476> 2014-08-13 17:23:20.917816 7fccda939700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530c380 sd=371 :6820 s=0 pgs=0 cs=0 l=0 c=0x5522ec0).accept sd=371 192.168.1.113:56165/0 -7475> 2014-08-13 17:23:20.918123 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6836/6227 -- pg_notify(22.775(26) epoch 194476) v5 -- ?+0 0x54d0380 con 0x53011e0 -7474> 2014-08-13 17:23:20.918219 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.708548 3 0.000123 -7473> 2014-08-13 17:23:20.918257 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started 0.708651 0 0.000000 -7472> 2014-08-13 17:23:20.918282 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192968) [294,327,4] r=2 lpr=194471 pi=192977-194469/19 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset -7471> 2014-08-13 17:23:20.918453 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_query(24.3d epoch 194476) v3 -- ?+0 0x54f9a40 con 0x4fd32e0 -7470> 2014-08-13 17:23:20.918530 7fcd0964c700 5 -- op tracker -- , seq: 14, time: 2014-08-13 17:23:20.918529, event: done, request: pg_notify(22.1d2(24) epoch 194476) v5 -7469> 2014-08-13 17:23:20.918566 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52bf500 con 0x53056a0 -7468> 2014-08-13 17:23:20.918594 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7467> 2014-08-13 17:23:20.918611 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x2e88fc0 con 0x5305120 -7466> 2014-08-13 17:23:20.918631 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7465> 2014-08-13 17:23:20.918648 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e4140 con 0x5305c20 -7464> 2014-08-13 17:23:20.918667 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7463> 2014-08-13 17:23:20.918679 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x55086c0 con 0x5305960 -7462> 2014-08-13 17:23:20.918697 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7461> 2014-08-13 17:23:20.918713 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 2 ==== pg_query(22.14a epoch 194476) v3 ==== 144+0+0 (1563116060 0 0) 0x2f910e0 con 0x4e335a0 -7460> 2014-08-13 17:23:20.918854 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.000572 4 0.000149 -7459> 2014-08-13 17:23:20.918889 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started -7458> 2014-08-13 17:23:20.918908 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start -7457> 2014-08-13 17:23:20.918906 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Reset 0.866404 6 0.000098 -7456> 2014-08-13 17:23:20.918928 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7455> 2014-08-13 17:23:20.918949 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started -7454> 2014-08-13 17:23:20.918952 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000043 0 0.000000 -7453> 2014-08-13 17:23:20.918978 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Start -7452> 2014-08-13 17:23:20.918985 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray -7451> 2014-08-13 17:23:20.918999 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: transitioning to Stray -7450> 2014-08-13 17:23:20.919023 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Start 0.000045 0 0.000000 -7449> 2014-08-13 17:23:20.919048 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194472/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/Stray -7448> 2014-08-13 17:23:20.919074 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.874464, event: header_read, request: pg_query(22.14a epoch 194476) v3 -7447> 2014-08-13 17:23:20.919097 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.874467, event: throttled, request: pg_query(22.14a epoch 194476) v3 -7446> 2014-08-13 17:23:20.919105 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.874521, event: all_read, request: pg_query(22.14a epoch 194476) v3 -7445> 2014-08-13 17:23:20.919111 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.918728, event: dispatched, request: pg_query(22.14a epoch 194476) v3 -7444> 2014-08-13 17:23:20.919119 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.919119, event: waiting_for_osdmap, request: pg_query(22.14a epoch 194476) v3 -7443> 2014-08-13 17:23:20.919128 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.919128, event: started, request: pg_query(22.14a epoch 194476) v3 -7442> 2014-08-13 17:23:20.919149 7fcd0964c700 5 -- op tracker -- , seq: 15, time: 2014-08-13 17:23:20.919149, event: done, request: pg_query(22.14a epoch 194476) v3 -7441> 2014-08-13 17:23:20.919171 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 2 ==== pg_query(22.29a epoch 194476) v3 ==== 144+0+0 (1855064113 0 0) 0x2f95fa0 con 0x4e32940 -7440> 2014-08-13 17:23:20.919202 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.912305, event: header_read, request: pg_query(22.29a epoch 194476) v3 -7439> 2014-08-13 17:23:20.919210 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.912307, event: throttled, request: pg_query(22.29a epoch 194476) v3 -7438> 2014-08-13 17:23:20.919216 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.912372, event: all_read, request: pg_query(22.29a epoch 194476) v3 -7437> 2014-08-13 17:23:20.919222 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.919198, event: dispatched, request: pg_query(22.29a epoch 194476) v3 -7436> 2014-08-13 17:23:20.919228 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.919228, event: waiting_for_osdmap, request: pg_query(22.29a epoch 194476) v3 -7435> 2014-08-13 17:23:20.919208 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7434> 2014-08-13 17:23:20.919243 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.919242, event: started, request: pg_query(22.29a epoch 194476) v3 -7433> 2014-08-13 17:23:20.919242 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.713095 2 0.000113 -7432> 2014-08-13 17:23:20.919263 7fcd0964c700 5 -- op tracker -- , seq: 16, time: 2014-08-13 17:23:20.919263, event: done, request: pg_query(22.29a epoch 194476) v3 -7431> 2014-08-13 17:23:20.919264 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.713126 0 0.000000 -7430> 2014-08-13 17:23:20.919283 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.713157 0 0.000000 -7429> 2014-08-13 17:23:20.919306 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.713223 0 0.000000 -7428> 2014-08-13 17:23:20.919327 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -7427> 2014-08-13 17:23:20.919344 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Reset 0.900931 6 0.000129 -7426> 2014-08-13 17:23:20.919379 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started -7425> 2014-08-13 17:23:20.919380 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 2 ==== pg_notify(24.3d(4) epoch 194476) v5 ==== 980+0+0 (2174914663 0 0) 0x5187540 con 0x4e33de0 -7424> 2014-08-13 17:23:20.919400 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Start -7423> 2014-08-13 17:23:20.919421 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: transitioning to Stray -7422> 2014-08-13 17:23:20.919417 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.911854, event: header_read, request: pg_notify(24.3d(4) epoch 194476) v5 -7421> 2014-08-13 17:23:20.919453 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.911856, event: throttled, request: pg_notify(24.3d(4) epoch 194476) v5 -7420> 2014-08-13 17:23:20.919444 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Start 0.000045 0 0.000000 -7419> 2014-08-13 17:23:20.919467 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.911929, event: all_read, request: pg_notify(24.3d(4) epoch 194476) v5 -7418> 2014-08-13 17:23:20.919478 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.919412, event: dispatched, request: pg_notify(24.3d(4) epoch 194476) v5 -7417> 2014-08-13 17:23:20.919470 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194472/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/Stray -7416> 2014-08-13 17:23:20.919464 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000138 4 0.000214 -7415> 2014-08-13 17:23:20.919496 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.919495, event: waiting_for_osdmap, request: pg_notify(24.3d(4) epoch 194476) v5 -7414> 2014-08-13 17:23:20.919520 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.919520, event: started, request: pg_notify(24.3d(4) epoch 194476) v5 -7413> 2014-08-13 17:23:20.919514 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -7412> 2014-08-13 17:23:20.919545 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -7411> 2014-08-13 17:23:20.919564 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7410> 2014-08-13 17:23:20.919587 7fcd0964c700 5 -- op tracker -- , seq: 17, time: 2014-08-13 17:23:20.919587, event: done, request: pg_notify(24.3d(4) epoch 194476) v5 -7409> 2014-08-13 17:23:20.919588 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 -7408> 2014-08-13 17:23:20.919655 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7407> 2014-08-13 17:23:20.919677 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7406> 2014-08-13 17:23:20.919696 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7405> 2014-08-13 17:23:20.919276 7fccda434700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530cb00 sd=375 :6808 s=0 pgs=0 cs=0 l=0 c=0x5523700).accept sd=375 192.168.100.113:57986/0 -7404> 2014-08-13 17:23:20.920357 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(21) epoch 194476) v5 -- ?+0 0x54d0000 con 0x4eb40a0 -7403> 2014-08-13 17:23:20.920415 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(28) epoch 194476) v5 -- ?+0 0x54d32c0 con 0x5520b00 -7402> 2014-08-13 17:23:20.920415 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7401> 2014-08-13 17:23:20.920445 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.714048 2 0.000155 -7400> 2014-08-13 17:23:20.920466 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.714078 0 0.000000 -7399> 2014-08-13 17:23:20.920487 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.714111 0 0.000000 -7398> 2014-08-13 17:23:20.920505 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started 0.714182 0 0.000000 -7397> 2014-08-13 17:23:20.920498 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.712524 3 0.000110 -7396> 2014-08-13 17:23:20.920527 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194470/194470/194470) [4,339,45] r=0 lpr=194471 pi=193306-194469/21 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset -7395> 2014-08-13 17:23:20.920533 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started 0.712611 0 0.000000 -7394> 2014-08-13 17:23:20.920554 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192946) [334,289,4] r=2 lpr=194471 pi=192974-194469/19 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset -7393> 2014-08-13 17:23:20.920745 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000219 4 0.000234 -7392> 2014-08-13 17:23:20.920782 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started -7391> 2014-08-13 17:23:20.920782 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.000229 4 0.000163 -7390> 2014-08-13 17:23:20.920806 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start -7389> 2014-08-13 17:23:20.920811 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started -7388> 2014-08-13 17:23:20.920825 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7387> 2014-08-13 17:23:20.920831 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start -7386> 2014-08-13 17:23:20.920847 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7385> 2014-08-13 17:23:20.920849 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000043 0 0.000000 -7384> 2014-08-13 17:23:20.920884 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7383> 2014-08-13 17:23:20.920905 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7382> 2014-08-13 17:23:20.920924 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194466/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7381> 2014-08-13 17:23:20.919662 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x5509200 con 0x4e35120 -7380> 2014-08-13 17:23:20.921072 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7379> 2014-08-13 17:23:20.921091 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 2 ==== pg_notify(22.84(1) epoch 194476) v5 ==== 769+0+0 (3649449316 0 0) 0x51847c0 con 0x4e33f40 -7378> 2014-08-13 17:23:20.921107 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.885623, event: header_read, request: pg_notify(22.84(1) epoch 194476) v5 -7377> 2014-08-13 17:23:20.921117 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.885624, event: throttled, request: pg_notify(22.84(1) epoch 194476) v5 -7376> 2014-08-13 17:23:20.921123 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.885779, event: all_read, request: pg_notify(22.84(1) epoch 194476) v5 -7375> 2014-08-13 17:23:20.921128 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.921104, event: dispatched, request: pg_notify(22.84(1) epoch 194476) v5 -7374> 2014-08-13 17:23:20.921135 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.921135, event: waiting_for_osdmap, request: pg_notify(22.84(1) epoch 194476) v5 -7373> 2014-08-13 17:23:20.921144 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.921144, event: started, request: pg_notify(22.84(1) epoch 194476) v5 -7372> 2014-08-13 17:23:20.921176 7fcd0964c700 5 -- op tracker -- , seq: 18, time: 2014-08-13 17:23:20.921176, event: done, request: pg_notify(22.84(1) epoch 194476) v5 -7371> 2014-08-13 17:23:20.921203 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 2 ==== pg_notify(22.1d2(23) epoch 194476) v5 ==== 2003+0+0 (1337698436 0 0) 0x5184600 con 0x4e35ac0 -7370> 2014-08-13 17:23:20.921220 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.883307, event: header_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -7369> 2014-08-13 17:23:20.920876 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000045 0 0.000000 -7368> 2014-08-13 17:23:20.921228 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.883309, event: throttled, request: pg_notify(22.1d2(23) epoch 194476) v5 -7367> 2014-08-13 17:23:20.921235 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.883447, event: all_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -7366> 2014-08-13 17:23:20.921242 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.921218, event: dispatched, request: pg_notify(22.1d2(23) epoch 194476) v5 -7365> 2014-08-13 17:23:20.921248 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.921248, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194476) v5 -7364> 2014-08-13 17:23:20.921238 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray -7363> 2014-08-13 17:23:20.921257 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.921256, event: started, request: pg_notify(22.1d2(23) epoch 194476) v5 -7362> 2014-08-13 17:23:20.921279 7fcd0964c700 5 -- op tracker -- , seq: 19, time: 2014-08-13 17:23:20.921279, event: done, request: pg_notify(22.1d2(23) epoch 194476) v5 -7361> 2014-08-13 17:23:20.921294 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 2 ==== pg_notify(22.22c(23) epoch 194476) v5 ==== 2003+0+0 (2227283430 0 0) 0x5185080 con 0x4e35800 -7360> 2014-08-13 17:23:20.921310 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.907041, event: header_read, request: pg_notify(22.22c(23) epoch 194476) v5 -7359> 2014-08-13 17:23:20.921317 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.907043, event: throttled, request: pg_notify(22.22c(23) epoch 194476) v5 -7358> 2014-08-13 17:23:20.921322 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.907231, event: all_read, request: pg_notify(22.22c(23) epoch 194476) v5 -7357> 2014-08-13 17:23:20.921327 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.921308, event: dispatched, request: pg_notify(22.22c(23) epoch 194476) v5 -7356> 2014-08-13 17:23:20.921337 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.921336, event: waiting_for_osdmap, request: pg_notify(22.22c(23) epoch 194476) v5 -7355> 2014-08-13 17:23:20.921346 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.921346, event: started, request: pg_notify(22.22c(23) epoch 194476) v5 -7354> 2014-08-13 17:23:20.921368 7fcd0964c700 5 -- op tracker -- , seq: 20, time: 2014-08-13 17:23:20.921367, event: done, request: pg_notify(22.22c(23) epoch 194476) v5 -7353> 2014-08-13 17:23:20.921392 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 2 ==== pg_notify(22.113(23) epoch 194476) v5 ==== 2003+0+0 (3933187701 0 0) 0x5184d00 con 0x4ec09a0 -7352> 2014-08-13 17:23:20.921387 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7351> 2014-08-13 17:23:20.921408 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.899106, event: header_read, request: pg_notify(22.113(23) epoch 194476) v5 -7350> 2014-08-13 17:23:20.921415 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.899108, event: throttled, request: pg_notify(22.113(23) epoch 194476) v5 -7349> 2014-08-13 17:23:20.921421 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.899232, event: all_read, request: pg_notify(22.113(23) epoch 194476) v5 -7348> 2014-08-13 17:23:20.921427 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.921406, event: dispatched, request: pg_notify(22.113(23) epoch 194476) v5 -7347> 2014-08-13 17:23:20.921415 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.713212 2 0.000151 -7346> 2014-08-13 17:23:20.921434 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.921434, event: waiting_for_osdmap, request: pg_notify(22.113(23) epoch 194476) v5 -7345> 2014-08-13 17:23:20.921442 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.921442, event: started, request: pg_notify(22.113(23) epoch 194476) v5 -7344> 2014-08-13 17:23:20.921437 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.713246 0 0.000000 -7343> 2014-08-13 17:23:20.921462 7fcd0964c700 5 -- op tracker -- , seq: 21, time: 2014-08-13 17:23:20.921462, event: done, request: pg_notify(22.113(23) epoch 194476) v5 -7342> 2014-08-13 17:23:20.921460 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.713280 0 0.000000 -7341> 2014-08-13 17:23:20.921477 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e0fc0 con 0x4ec2aa0 -7340> 2014-08-13 17:23:20.921478 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.713378 0 0.000000 -7339> 2014-08-13 17:23:20.921494 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7338> 2014-08-13 17:23:20.921499 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194470/194470/194470) [4,313,318] r=0 lpr=194471 pi=193306-194469/23 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -7337> 2014-08-13 17:23:20.921507 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 2 ==== pg_query(22.791 epoch 194476) v3 ==== 144+0+0 (1629551235 0 0) 0x2f94ce0 con 0x4ec3f40 -7336> 2014-08-13 17:23:20.921523 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.907339, event: header_read, request: pg_query(22.791 epoch 194476) v3 -7335> 2014-08-13 17:23:20.921530 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.907341, event: throttled, request: pg_query(22.791 epoch 194476) v3 -7334> 2014-08-13 17:23:20.921536 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.907394, event: all_read, request: pg_query(22.791 epoch 194476) v3 -7333> 2014-08-13 17:23:20.921517 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.775269 6 0.000105 -7332> 2014-08-13 17:23:20.921542 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.921521, event: dispatched, request: pg_query(22.791 epoch 194476) v3 -7331> 2014-08-13 17:23:20.921548 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.921548, event: waiting_for_osdmap, request: pg_query(22.791 epoch 194476) v3 -7330> 2014-08-13 17:23:20.921556 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.921556, event: started, request: pg_query(22.791 epoch 194476) v3 -7329> 2014-08-13 17:23:20.921547 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started -7328> 2014-08-13 17:23:20.921573 7fcd0964c700 5 -- op tracker -- , seq: 22, time: 2014-08-13 17:23:20.921573, event: done, request: pg_query(22.791 epoch 194476) v3 -7327> 2014-08-13 17:23:20.921564 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start -7326> 2014-08-13 17:23:20.921581 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray -7325> 2014-08-13 17:23:20.921590 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 2 ==== pg_query(24.1e4 epoch 194476) v3 ==== 144+0+0 (4162670956 0 0) 0x2f92ee0 con 0x4ec56a0 -7324> 2014-08-13 17:23:20.921604 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.894452, event: header_read, request: pg_query(24.1e4 epoch 194476) v3 -7323> 2014-08-13 17:23:20.921611 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.894454, event: throttled, request: pg_query(24.1e4 epoch 194476) v3 -7322> 2014-08-13 17:23:20.921600 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000036 0 0.000000 -7321> 2014-08-13 17:23:20.921616 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.894622, event: all_read, request: pg_query(24.1e4 epoch 194476) v3 -7320> 2014-08-13 17:23:20.921624 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.921602, event: dispatched, request: pg_query(24.1e4 epoch 194476) v3 -7319> 2014-08-13 17:23:20.921630 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.921630, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194476) v3 -7318> 2014-08-13 17:23:20.921622 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194472/194457 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray -7317> 2014-08-13 17:23:20.921638 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.921638, event: started, request: pg_query(24.1e4 epoch 194476) v3 -7316> 2014-08-13 17:23:20.921653 7fcd0964c700 5 -- op tracker -- , seq: 23, time: 2014-08-13 17:23:20.921653, event: done, request: pg_query(24.1e4 epoch 194476) v3 -7315> 2014-08-13 17:23:20.921674 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e0000 con 0x4ec3440 -7314> 2014-08-13 17:23:20.921691 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7313> 2014-08-13 17:23:20.921700 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 2 ==== pg_query(24.350,24.3b7 epoch 194476) v3 ==== 272+0+0 (4115034236 0 0) 0x2f94560 con 0x4ec4780 -7312> 2014-08-13 17:23:20.921714 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.906725, event: header_read, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7311> 2014-08-13 17:23:20.921698 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] exit Reset 0.000199 4 0.000217 -7310> 2014-08-13 17:23:20.921721 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.906727, event: throttled, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7309> 2014-08-13 17:23:20.921728 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.906790, event: all_read, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7308> 2014-08-13 17:23:20.921733 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.921712, event: dispatched, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7307> 2014-08-13 17:23:20.921724 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] enter Started -7306> 2014-08-13 17:23:20.921740 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.921740, event: waiting_for_osdmap, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7305> 2014-08-13 17:23:20.921749 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.921749, event: started, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7304> 2014-08-13 17:23:20.921742 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] enter Start -7303> 2014-08-13 17:23:20.921753 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7302> 2014-08-13 17:23:20.921760 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7301> 2014-08-13 17:23:20.921773 7fcd0964c700 5 -- op tracker -- , seq: 24, time: 2014-08-13 17:23:20.921773, event: done, request: pg_query(24.350,24.3b7 epoch 194476) v3 -7300> 2014-08-13 17:23:20.921779 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.714296 2 0.000171 -7299> 2014-08-13 17:23:20.921782 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] exit Start 0.000039 0 0.000000 -7298> 2014-08-13 17:23:20.921791 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 2 ==== pg_query(22.3e4 epoch 194476) v3 ==== 144+0+0 (2245915826 0 0) 0x2f95820 con 0x4ec5d80 -7297> 2014-08-13 17:23:20.921806 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.911203, event: header_read, request: pg_query(22.3e4 epoch 194476) v3 -7296> 2014-08-13 17:23:20.921799 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.714326 0 0.000000 -7295> 2014-08-13 17:23:20.921813 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.911204, event: throttled, request: pg_query(22.3e4 epoch 194476) v3 -7294> 2014-08-13 17:23:20.921820 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.911254, event: all_read, request: pg_query(22.3e4 epoch 194476) v3 -7293> 2014-08-13 17:23:20.921826 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.921804, event: dispatched, request: pg_query(22.3e4 epoch 194476) v3 -7292> 2014-08-13 17:23:20.921819 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.714356 0 0.000000 -7291> 2014-08-13 17:23:20.921833 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.921832, event: waiting_for_osdmap, request: pg_query(22.3e4 epoch 194476) v3 -7290> 2014-08-13 17:23:20.921845 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.921845, event: started, request: pg_query(22.3e4 epoch 194476) v3 -7289> 2014-08-13 17:23:20.921838 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.714481 0 0.000000 -7288> 2014-08-13 17:23:20.922073 7fccd9a2a700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530c100 sd=377 :6820 s=0 pgs=0 cs=0 l=0 c=0x5522c00).accept sd=377 192.168.1.101:45752/0 -7287> 2014-08-13 17:23:20.921802 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194466/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] enter Started/Stray -7286> 2014-08-13 17:23:20.921879 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194470/194470/194470) [4,340] r=0 lpr=194471 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset -7285> 2014-08-13 17:23:20.922138 7fccd9929700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530c880 sd=378 :6808 s=0 pgs=0 cs=0 l=0 c=0x5523440).accept sd=378 192.168.100.101:51705/0 -7284> 2014-08-13 17:23:20.922323 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- pg_notify(22.3e4(26) epoch 194476) v5 -- ?+0 0x54d3100 con 0x4ec5d80 -7283> 2014-08-13 17:23:20.922340 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000461 4 0.000536 -7282> 2014-08-13 17:23:20.922377 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -7281> 2014-08-13 17:23:20.922402 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -7280> 2014-08-13 17:23:20.922420 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7279> 2014-08-13 17:23:20.922442 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 -7278> 2014-08-13 17:23:20.922467 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7277> 2014-08-13 17:23:20.922487 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7276> 2014-08-13 17:23:20.922506 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194471/194471 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7275> 2014-08-13 17:23:20.922357 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(21) epoch 194476) v5 -- ?+0 0x54d2f40 con 0x4fde040 -7274> 2014-08-13 17:23:20.922599 7fcd0964c700 5 -- op tracker -- , seq: 25, time: 2014-08-13 17:23:20.922599, event: done, request: pg_query(22.3e4 epoch 194476) v3 -7273> 2014-08-13 17:23:20.922644 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 2 ==== pg_query(24.2a1 epoch 194476) v3 ==== 144+0+0 (3068429505 0 0) 0x2f932a0 con 0x4ec5ac0 -7272> 2014-08-13 17:23:20.922670 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.894920, event: header_read, request: pg_query(24.2a1 epoch 194476) v3 -7271> 2014-08-13 17:23:20.922679 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.894921, event: throttled, request: pg_query(24.2a1 epoch 194476) v3 -7270> 2014-08-13 17:23:20.922688 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.894971, event: all_read, request: pg_query(24.2a1 epoch 194476) v3 -7269> 2014-08-13 17:23:20.922693 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.922667, event: dispatched, request: pg_query(24.2a1 epoch 194476) v3 -7268> 2014-08-13 17:23:20.922700 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.922700, event: waiting_for_osdmap, request: pg_query(24.2a1 epoch 194476) v3 -7267> 2014-08-13 17:23:20.922709 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.922709, event: started, request: pg_query(24.2a1 epoch 194476) v3 -7266> 2014-08-13 17:23:20.922889 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Reset 0.964415 6 0.000083 -7265> 2014-08-13 17:23:20.922927 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started -7264> 2014-08-13 17:23:20.922949 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Start -7263> 2014-08-13 17:23:20.922967 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7262> 2014-08-13 17:23:20.922988 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 -7261> 2014-08-13 17:23:20.923009 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started/Stray -7260> 2014-08-13 17:23:20.923245 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.806514 6 0.000132 -7259> 2014-08-13 17:23:20.923280 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started -7258> 2014-08-13 17:23:20.923301 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start -7257> 2014-08-13 17:23:20.923320 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray -7256> 2014-08-13 17:23:20.923342 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000040 0 0.000000 -7255> 2014-08-13 17:23:20.923364 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194466/194457 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray -7254> 2014-08-13 17:23:20.945737 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6802/58522 -- pg_notify(24.34(49) epoch 194476) v5 -- ?+0 0x54d1c00 con 0x55244c0 -7253> 2014-08-13 17:23:20.946039 7fccd9121700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530be80 sd=382 :6820 s=0 pgs=0 cs=0 l=0 c=0x55f2ec0).accept sd=382 192.168.1.101:45756/0 -7252> 2014-08-13 17:23:20.946078 7fccd8d1d700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530f800 sd=386 :6808 s=0 pgs=0 cs=0 l=0 c=0x55f32e0).accept sd=386 192.168.100.108:38576/0 -7251> 2014-08-13 17:23:20.946117 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(26) epoch 194476) v5 -- ?+0 0x54d1a40 con 0x4ec1ce0 -7250> 2014-08-13 17:23:20.946152 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194476) v5 -- ?+0 0x54d1880 con 0x4fde5c0 -7249> 2014-08-13 17:23:20.946172 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(29) epoch 194476) v5 -- ?+0 0x54d16c0 con 0x4f83860 -7248> 2014-08-13 17:23:20.946201 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6800/52669 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x54f9680 con 0x4fd11e0 -7247> 2014-08-13 17:23:20.946219 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x54f94a0 con 0x4ec09a0 -7246> 2014-08-13 17:23:20.946196 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.737686 3 0.000117 -7245> 2014-08-13 17:23:20.946238 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6834/6966 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x54f92c0 con 0x4fd9a20 -7244> 2014-08-13 17:23:20.946262 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6818/45268 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x54f81e0 con 0x5305c20 -7243> 2014-08-13 17:23:20.946254 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.737797 0 0.000000 -7242> 2014-08-13 17:23:20.946276 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192976) [367,44,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -7241> 2014-08-13 17:23:20.946286 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_query(1.2b,2.2a epoch 194476) v3 -- ?+0 0x54f8780 con 0x4e35540 -7240> 2014-08-13 17:23:20.946309 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6802/41438 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x54f90e0 con 0x5305960 -7239> 2014-08-13 17:23:20.946410 7fcd0964c700 5 -- op tracker -- , seq: 26, time: 2014-08-13 17:23:20.946409, event: done, request: pg_query(24.2a1 epoch 194476) v3 -7238> 2014-08-13 17:23:20.946414 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7237> 2014-08-13 17:23:20.946450 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.738800 2 0.000120 -7236> 2014-08-13 17:23:20.946456 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 2 ==== pg_query(22.3af epoch 194476) v3 ==== 144+0+0 (3428236819 0 0) 0x2f90f00 con 0x4eb0c60 -7235> 2014-08-13 17:23:20.946480 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.874410, event: header_read, request: pg_query(22.3af epoch 194476) v3 -7234> 2014-08-13 17:23:20.946474 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.738836 0 0.000000 -7233> 2014-08-13 17:23:20.946488 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.874411, event: throttled, request: pg_query(22.3af epoch 194476) v3 -7232> 2014-08-13 17:23:20.946494 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.874473, event: all_read, request: pg_query(22.3af epoch 194476) v3 -7231> 2014-08-13 17:23:20.946500 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.946476, event: dispatched, request: pg_query(22.3af epoch 194476) v3 -7230> 2014-08-13 17:23:20.946507 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.946507, event: waiting_for_osdmap, request: pg_query(22.3af epoch 194476) v3 -7229> 2014-08-13 17:23:20.946497 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.738870 0 0.000000 -7228> 2014-08-13 17:23:20.946518 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.946517, event: started, request: pg_query(22.3af epoch 194476) v3 -7227> 2014-08-13 17:23:20.946517 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started 0.738937 0 0.000000 -7226> 2014-08-13 17:23:20.946539 7fcd0964c700 5 -- op tracker -- , seq: 27, time: 2014-08-13 17:23:20.946539, event: done, request: pg_query(22.3af epoch 194476) v3 -7225> 2014-08-13 17:23:20.946536 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194470/194470/194470) [4,374,31] r=0 lpr=194471 pi=193306-194469/19 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset -7224> 2014-08-13 17:23:20.946563 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 2 ==== pg_query(22.2b9 epoch 194476) v3 ==== 144+0+0 (1271686137 0 0) 0x2f93c00 con 0x4eb23c0 -7223> 2014-08-13 17:23:20.946579 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.899106, event: header_read, request: pg_query(22.2b9 epoch 194476) v3 -7222> 2014-08-13 17:23:20.946586 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.899109, event: throttled, request: pg_query(22.2b9 epoch 194476) v3 -7221> 2014-08-13 17:23:20.946591 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.899550, event: all_read, request: pg_query(22.2b9 epoch 194476) v3 -7220> 2014-08-13 17:23:20.946596 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.946577, event: dispatched, request: pg_query(22.2b9 epoch 194476) v3 -7219> 2014-08-13 17:23:20.946602 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.946602, event: waiting_for_osdmap, request: pg_query(22.2b9 epoch 194476) v3 -7218> 2014-08-13 17:23:20.946610 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.946610, event: started, request: pg_query(22.2b9 epoch 194476) v3 -7217> 2014-08-13 17:23:20.946627 7fcd0964c700 5 -- op tracker -- , seq: 28, time: 2014-08-13 17:23:20.946627, event: done, request: pg_query(22.2b9 epoch 194476) v3 -7216> 2014-08-13 17:23:20.946644 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 2 ==== pg_query(22.99 epoch 194476) v3 ==== 144+0+0 (2568978070 0 0) 0x2f94740 con 0x4eb18c0 -7215> 2014-08-13 17:23:20.946664 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.907092, event: header_read, request: pg_query(22.99 epoch 194476) v3 -7214> 2014-08-13 17:23:20.946671 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.907095, event: throttled, request: pg_query(22.99 epoch 194476) v3 -7213> 2014-08-13 17:23:20.946677 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.907152, event: all_read, request: pg_query(22.99 epoch 194476) v3 -7212> 2014-08-13 17:23:20.946682 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.946662, event: dispatched, request: pg_query(22.99 epoch 194476) v3 -7211> 2014-08-13 17:23:20.946690 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.946690, event: waiting_for_osdmap, request: pg_query(22.99 epoch 194476) v3 -7210> 2014-08-13 17:23:20.946699 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.946698, event: started, request: pg_query(22.99 epoch 194476) v3 -7209> 2014-08-13 17:23:20.946760 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] exit Reset 0.000223 4 0.000239 -7208> 2014-08-13 17:23:20.946794 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] enter Started -7207> 2014-08-13 17:23:20.946812 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] enter Start -7206> 2014-08-13 17:23:20.946828 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7205> 2014-08-13 17:23:20.946850 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] exit Start 0.000038 0 0.000000 -7204> 2014-08-13 17:23:20.946843 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.000568 4 0.000203 -7203> 2014-08-13 17:23:20.946870 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194465/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] enter Started/Stray -7202> 2014-08-13 17:23:20.946872 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -7201> 2014-08-13 17:23:20.946890 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -7200> 2014-08-13 17:23:20.946905 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -7199> 2014-08-13 17:23:20.946924 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 -7198> 2014-08-13 17:23:20.946943 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -7197> 2014-08-13 17:23:20.947028 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7196> 2014-08-13 17:23:20.947058 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.739014 2 0.000144 -7195> 2014-08-13 17:23:20.947077 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.739066 0 0.000000 -7194> 2014-08-13 17:23:20.947096 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.739095 0 0.000000 -7193> 2014-08-13 17:23:20.947090 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7192> 2014-08-13 17:23:20.947115 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started 0.739161 0 0.000000 -7191> 2014-08-13 17:23:20.947119 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.738427 2 0.000120 -7190> 2014-08-13 17:23:20.947134 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/194470) [4,28,304] r=0 lpr=194471 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset -7189> 2014-08-13 17:23:20.947143 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.738463 0 0.000000 -7188> 2014-08-13 17:23:20.947171 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.738504 0 0.000000 -7187> 2014-08-13 17:23:20.947189 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started 0.738569 0 0.000000 -7186> 2014-08-13 17:23:20.947222 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194470/194470/194470) [4,310,21] r=0 lpr=194471 pi=193306-194469/23 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset -7185> 2014-08-13 17:23:20.947308 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000174 4 0.000201 -7184> 2014-08-13 17:23:20.947331 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started -7183> 2014-08-13 17:23:20.947346 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start -7182> 2014-08-13 17:23:20.947361 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7181> 2014-08-13 17:23:20.947379 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 -7180> 2014-08-13 17:23:20.947398 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7179> 2014-08-13 17:23:20.947414 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7178> 2014-08-13 17:23:20.947418 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] exit Reset 0.000196 4 0.000237 -7177> 2014-08-13 17:23:20.947430 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7176> 2014-08-13 17:23:20.947446 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] enter Started -7175> 2014-08-13 17:23:20.947464 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] enter Start -7174> 2014-08-13 17:23:20.947480 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7173> 2014-08-13 17:23:20.947459 7fccd8c1c700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x530af80 sd=387 :6819 s=0 pgs=0 cs=0 l=0 c=0x552d540).accept sd=387 192.168.1.101:48262/0 -7172> 2014-08-13 17:23:20.947499 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] exit Start 0.000035 0 0.000000 -7171> 2014-08-13 17:23:20.947518 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194466/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] enter Started/Stray -7170> 2014-08-13 17:23:20.947552 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7169> 2014-08-13 17:23:20.947576 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.739362 2 0.000130 -7168> 2014-08-13 17:23:20.947595 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.739393 0 0.000000 -7167> 2014-08-13 17:23:20.947615 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.739424 0 0.000000 -7166> 2014-08-13 17:23:20.947632 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started 0.739504 0 0.000000 -7165> 2014-08-13 17:23:20.947650 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,356,357] r=0 lpr=194471 pi=193306-194469/21 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset -7164> 2014-08-13 17:23:20.947661 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 0.738336 3 0.000371 -7163> 2014-08-13 17:23:20.947690 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started 0.738568 0 0.000000 -7162> 2014-08-13 17:23:20.947709 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194470/194471/192968) [289,4,285] r=1 lpr=194471 pi=193125-194470/37 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset -7161> 2014-08-13 17:23:20.947795 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000146 4 0.000189 -7160> 2014-08-13 17:23:20.947817 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started -7159> 2014-08-13 17:23:20.947834 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start -7158> 2014-08-13 17:23:20.947848 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -7157> 2014-08-13 17:23:20.947816 7fccd8f1f700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x530b200 sd=381 :6819 s=0 pgs=0 cs=0 l=0 c=0x552d800).accept sd=381 192.168.1.101:48261/0 -7156> 2014-08-13 17:23:20.947922 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.000214 4 0.000125 -7155> 2014-08-13 17:23:20.947947 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started -7154> 2014-08-13 17:23:20.947963 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start -7153> 2014-08-13 17:23:20.947961 7fccd9020700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530c600 sd=383 :6808 s=0 pgs=0 cs=0 l=0 c=0x55f35a0).accept sd=383 192.168.100.101:51709/0 -7152> 2014-08-13 17:23:20.947977 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray -7151> 2014-08-13 17:23:20.947995 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000033 0 0.000000 -7150> 2014-08-13 17:23:20.947974 7fccd8e1e700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530ee00 sd=385 :6820 s=0 pgs=0 cs=0 l=0 c=0x55f2c00).accept sd=385 192.168.1.108:41024/0 -7149> 2014-08-13 17:23:20.947866 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 -7148> 2014-08-13 17:23:20.948013 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194466/194457 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray -7147> 2014-08-13 17:23:20.948031 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -7146> 2014-08-13 17:23:20.948052 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -7145> 2014-08-13 17:23:20.948074 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -7144> 2014-08-13 17:23:20.948177 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7143> 2014-08-13 17:23:20.948211 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.738559 2 0.000243 -7142> 2014-08-13 17:23:20.948232 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.738590 0 0.000000 -7141> 2014-08-13 17:23:20.948242 7fcd0964c700 5 -- op tracker -- , seq: 29, time: 2014-08-13 17:23:20.948241, event: done, request: pg_query(22.99 epoch 194476) v3 -7140> 2014-08-13 17:23:20.948259 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.738631 0 0.000000 -7139> 2014-08-13 17:23:20.948276 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started 0.738747 0 0.000000 -7138> 2014-08-13 17:23:20.948278 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x52b86c0 con 0x4eb40a0 -7137> 2014-08-13 17:23:20.948284 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_notify(22.99(22) epoch 194476) v5 -- ?+0 0x56a9340 con 0x4eb18c0 -7136> 2014-08-13 17:23:20.948307 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7135> 2014-08-13 17:23:20.948300 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194470/194470/194470) [4,5,47] r=0 lpr=194471 pi=193306-194469/23 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset -7134> 2014-08-13 17:23:20.948313 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6805/37391 -- pg_query(22.84 epoch 194476) v3 -- ?+0 0x54f9e00 con 0x4e33f40 -7133> 2014-08-13 17:23:20.948324 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x52be540 con 0x4eb2680 -7132> 2014-08-13 17:23:20.948330 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_query(22.84 epoch 194476) v3 -- ?+0 0x2f95be0 con 0x4ec1ce0 -7131> 2014-08-13 17:23:20.948340 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7130> 2014-08-13 17:23:20.948350 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 2 ==== pg_notify(22.6bd(1) epoch 194476) v5 ==== 769+0+0 (69560420 0 0) 0x4e508c0 con 0x4eb7d20 -7129> 2014-08-13 17:23:20.948367 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.894949, event: header_read, request: pg_notify(22.6bd(1) epoch 194476) v5 -7128> 2014-08-13 17:23:20.948377 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.894951, event: throttled, request: pg_notify(22.6bd(1) epoch 194476) v5 -7127> 2014-08-13 17:23:20.948384 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.895026, event: all_read, request: pg_notify(22.6bd(1) epoch 194476) v5 -7126> 2014-08-13 17:23:20.948390 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.948362, event: dispatched, request: pg_notify(22.6bd(1) epoch 194476) v5 -7125> 2014-08-13 17:23:20.948397 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.948397, event: waiting_for_osdmap, request: pg_notify(22.6bd(1) epoch 194476) v5 -7124> 2014-08-13 17:23:20.948391 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6826/4273 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x2f950a0 con 0x56b1fa0 -7123> 2014-08-13 17:23:20.948407 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.948406, event: started, request: pg_notify(22.6bd(1) epoch 194476) v5 -7122> 2014-08-13 17:23:20.948409 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x2f95640 con 0x4ec4780 -7121> 2014-08-13 17:23:20.948427 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6834/17224 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x2f96720 con 0x56b2260 -7120> 2014-08-13 17:23:20.948434 7fcd0964c700 5 -- op tracker -- , seq: 30, time: 2014-08-13 17:23:20.948433, event: done, request: pg_notify(22.6bd(1) epoch 194476) v5 -7119> 2014-08-13 17:23:20.948441 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x54fc1a0 con 0x4fd86e0 -7118> 2014-08-13 17:23:20.948460 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.296 192.168.1.109:6806/31286 2 ==== pg_query(24.181 epoch 194476) v3 ==== 144+0+0 (2342532864 0 0) 0x2f95a00 con 0x4f81600 -7117> 2014-08-13 17:23:20.948481 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.911421, event: header_read, request: pg_query(24.181 epoch 194476) v3 -7116> 2014-08-13 17:23:20.948489 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.911423, event: throttled, request: pg_query(24.181 epoch 194476) v3 -7115> 2014-08-13 17:23:20.948494 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.911484, event: all_read, request: pg_query(24.181 epoch 194476) v3 -7114> 2014-08-13 17:23:20.948499 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.948478, event: dispatched, request: pg_query(24.181 epoch 194476) v3 -7113> 2014-08-13 17:23:20.948506 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.948506, event: waiting_for_osdmap, request: pg_query(24.181 epoch 194476) v3 -7112> 2014-08-13 17:23:20.948515 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.948514, event: started, request: pg_query(24.181 epoch 194476) v3 -7111> 2014-08-13 17:23:20.948503 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] exit Reset 0.000202 4 0.000225 -7110> 2014-08-13 17:23:20.948532 7fcd0964c700 5 -- op tracker -- , seq: 31, time: 2014-08-13 17:23:20.948532, event: done, request: pg_query(24.181 epoch 194476) v3 -7109> 2014-08-13 17:23:20.948529 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] enter Started -7108> 2014-08-13 17:23:20.948546 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] enter Start -7107> 2014-08-13 17:23:20.948552 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x52bfbc0 con 0x4eb6720 -7106> 2014-08-13 17:23:20.948570 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7105> 2014-08-13 17:23:20.948561 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7104> 2014-08-13 17:23:20.948583 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 2 ==== pg_query(24.20e epoch 194476) v3 ==== 144+0+0 (1016631308 0 0) 0x2f93480 con 0x4f80b00 -7103> 2014-08-13 17:23:20.948580 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] exit Start 0.000034 0 0.000000 -7102> 2014-08-13 17:23:20.948597 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.894955, event: header_read, request: pg_query(24.20e epoch 194476) v3 -7101> 2014-08-13 17:23:20.948607 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.894957, event: throttled, request: pg_query(24.20e epoch 194476) v3 -7100> 2014-08-13 17:23:20.948599 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194466/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] enter Started/Stray -7099> 2014-08-13 17:23:20.948613 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.895052, event: all_read, request: pg_query(24.20e epoch 194476) v3 -7098> 2014-08-13 17:23:20.948619 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.948595, event: dispatched, request: pg_query(24.20e epoch 194476) v3 -7097> 2014-08-13 17:23:20.948625 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.948625, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194476) v3 -7096> 2014-08-13 17:23:20.948634 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.948634, event: started, request: pg_query(24.20e epoch 194476) v3 -7095> 2014-08-13 17:23:20.948651 7fcd0964c700 5 -- op tracker -- , seq: 32, time: 2014-08-13 17:23:20.948651, event: done, request: pg_query(24.20e epoch 194476) v3 -7094> 2014-08-13 17:23:20.948675 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 2 ==== pg_query(22.3d9 epoch 194476) v3 ==== 144+0+0 (129567836 0 0) 0x2f91c20 con 0x4f832e0 -7093> 2014-08-13 17:23:20.948691 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.891499, event: header_read, request: pg_query(22.3d9 epoch 194476) v3 -7092> 2014-08-13 17:23:20.948698 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.891501, event: throttled, request: pg_query(22.3d9 epoch 194476) v3 -7091> 2014-08-13 17:23:20.948703 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.891649, event: all_read, request: pg_query(22.3d9 epoch 194476) v3 -7090> 2014-08-13 17:23:20.948709 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.948689, event: dispatched, request: pg_query(22.3d9 epoch 194476) v3 -7089> 2014-08-13 17:23:20.948727 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.948727, event: waiting_for_osdmap, request: pg_query(22.3d9 epoch 194476) v3 -7088> 2014-08-13 17:23:20.948736 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.948735, event: started, request: pg_query(22.3d9 epoch 194476) v3 -7087> 2014-08-13 17:23:20.948719 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6822/51844 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x54fbfc0 con 0x56b1ce0 -7086> 2014-08-13 17:23:20.948753 7fcd0964c700 5 -- op tracker -- , seq: 33, time: 2014-08-13 17:23:20.948753, event: done, request: pg_query(22.3d9 epoch 194476) v3 -7085> 2014-08-13 17:23:20.948774 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.19 192.168.1.115:6818/61186 2 ==== pg_query(24.3bf epoch 194476) v3 ==== 144+0+0 (911308499 0 0) 0x2f91fe0 con 0x4f84ba0 -7084> 2014-08-13 17:23:20.948805 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- pg_notify(22.1f1(26) epoch 194476) v5 -- ?+0 0x54d2bc0 con 0x4eb2680 -7083> 2014-08-13 17:23:20.948839 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.892417, event: header_read, request: pg_query(24.3bf epoch 194476) v3 -7082> 2014-08-13 17:23:20.948849 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.892420, event: throttled, request: pg_query(24.3bf epoch 194476) v3 -7081> 2014-08-13 17:23:20.948860 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.892715, event: all_read, request: pg_query(24.3bf epoch 194476) v3 -7080> 2014-08-13 17:23:20.948858 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(32) epoch 194476) v5 -- ?+0 0x4e50e00 con 0x4ec5ac0 -7079> 2014-08-13 17:23:20.948875 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.948836, event: dispatched, request: pg_query(24.3bf epoch 194476) v3 -7078> 2014-08-13 17:23:20.948883 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.948882, event: waiting_for_osdmap, request: pg_query(24.3bf epoch 194476) v3 -7077> 2014-08-13 17:23:20.948894 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.948893, event: started, request: pg_query(24.3bf epoch 194476) v3 -7076> 2014-08-13 17:23:20.948873 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -7075> 2014-08-13 17:23:20.948904 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(39) epoch 194476) v5 -- ?+0 0x54d2d80 con 0x4ec3440 -7074> 2014-08-13 17:23:20.948922 7fcd0964c700 5 -- op tracker -- , seq: 34, time: 2014-08-13 17:23:20.948921, event: done, request: pg_query(24.3bf epoch 194476) v3 -7073> 2014-08-13 17:23:20.948919 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.740505 2 0.000118 -7072> 2014-08-13 17:23:20.948938 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6812/32413 -- pg_notify(22.707(26) epoch 194476) v5 -- ?+0 0x54d1dc0 con 0x5303440 -7071> 2014-08-13 17:23:20.948942 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.740539 0 0.000000 -7070> 2014-08-13 17:23:20.948950 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x550cc80 con 0x4f81fa0 -7069> 2014-08-13 17:23:20.948957 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(21) epoch 194476) v5 -- ?+0 0x54d1f80 con 0x4fd3860 -7068> 2014-08-13 17:23:20.948969 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -7067> 2014-08-13 17:23:20.948963 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.740571 0 0.000000 -7066> 2014-08-13 17:23:20.948982 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 3 ==== pg_notify(24.3d(27) epoch 194476) v5 ==== 2265+0+0 (1442779177 0 0) 0x54d2a00 con 0x4f86040 -7065> 2014-08-13 17:23:20.948982 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started 0.740637 0 0.000000 -7064> 2014-08-13 17:23:20.948997 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.918804, event: header_read, request: pg_notify(24.3d(27) epoch 194476) v5 -7063> 2014-08-13 17:23:20.949005 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.918807, event: throttled, request: pg_notify(24.3d(27) epoch 194476) v5 -7062> 2014-08-13 17:23:20.949000 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194470/194470/194470) [4,373,344] r=0 lpr=194471 pi=193306-194469/21 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset -7061> 2014-08-13 17:23:20.949011 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.918994, event: all_read, request: pg_notify(24.3d(27) epoch 194476) v5 -7060> 2014-08-13 17:23:20.949018 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.948995, event: dispatched, request: pg_notify(24.3d(27) epoch 194476) v5 -7059> 2014-08-13 17:23:20.949024 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.949023, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194476) v5 -7058> 2014-08-13 17:23:20.949032 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.949032, event: started, request: pg_notify(24.3d(27) epoch 194476) v5 -7057> 2014-08-13 17:23:20.949060 7fcd0964c700 5 -- op tracker -- , seq: 35, time: 2014-08-13 17:23:20.949060, event: done, request: pg_notify(24.3d(27) epoch 194476) v5 -7056> 2014-08-13 17:23:20.949090 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 3 ==== pg_query(24.1de epoch 194476) v3 ==== 144+0+0 (3855556984 0 0) 0x2f95820 con 0x4f83860 -7055> 2014-08-13 17:23:20.949110 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.947869, event: header_read, request: pg_query(24.1de epoch 194476) v3 -7054> 2014-08-13 17:23:20.949118 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.947872, event: throttled, request: pg_query(24.1de epoch 194476) v3 -7053> 2014-08-13 17:23:20.949123 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.947922, event: all_read, request: pg_query(24.1de epoch 194476) v3 -7052> 2014-08-13 17:23:20.949128 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.949108, event: dispatched, request: pg_query(24.1de epoch 194476) v3 -7051> 2014-08-13 17:23:20.949134 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.949134, event: waiting_for_osdmap, request: pg_query(24.1de epoch 194476) v3 -7050> 2014-08-13 17:23:20.949141 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.949141, event: started, request: pg_query(24.1de epoch 194476) v3 -7049> 2014-08-13 17:23:20.949161 7fcd0964c700 5 -- op tracker -- , seq: 36, time: 2014-08-13 17:23:20.949161, event: done, request: pg_query(24.1de epoch 194476) v3 -7048> 2014-08-13 17:23:20.949184 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 3 ==== pg_notify(22.113(24) epoch 194476) v5 ==== 2062+0+0 (804026585 0 0) 0x54d7000 con 0x4fd9a20 -7047> 2014-08-13 17:23:20.949202 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.947370, event: header_read, request: pg_notify(22.113(24) epoch 194476) v5 -7046> 2014-08-13 17:23:20.949209 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.947373, event: throttled, request: pg_notify(22.113(24) epoch 194476) v5 -7045> 2014-08-13 17:23:20.949221 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.947508, event: all_read, request: pg_notify(22.113(24) epoch 194476) v5 -7044> 2014-08-13 17:23:20.949208 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] exit Reset 0.000208 4 0.000230 -7043> 2014-08-13 17:23:20.949232 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.949200, event: dispatched, request: pg_notify(22.113(24) epoch 194476) v5 -7042> 2014-08-13 17:23:20.949242 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.949242, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194476) v5 -7041> 2014-08-13 17:23:20.949234 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] enter Started -7040> 2014-08-13 17:23:20.949254 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.949253, event: started, request: pg_notify(22.113(24) epoch 194476) v5 -7039> 2014-08-13 17:23:20.949254 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] enter Start -7038> 2014-08-13 17:23:20.949281 7fcd0964c700 5 -- op tracker -- , seq: 37, time: 2014-08-13 17:23:20.949281, event: done, request: pg_notify(22.113(24) epoch 194476) v5 -7037> 2014-08-13 17:23:20.949277 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -7036> 2014-08-13 17:23:20.949297 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] exit Start 0.000042 0 0.000000 -7035> 2014-08-13 17:23:20.949316 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e7080 con 0x4f870c0 -7034> 2014-08-13 17:23:20.949326 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194466/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] enter Started/Stray -7033> 2014-08-13 17:23:20.949341 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7032> 2014-08-13 17:23:20.949039 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.738907 3 0.000264 -7031> 2014-08-13 17:23:20.949362 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 3 ==== pg_query(24.206 epoch 194476) v3 ==== 144+0+0 (1379228056 0 0) 0x2f91a40 con 0x4f86e00 -7030> 2014-08-13 17:23:20.949382 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.888426, event: header_read, request: pg_query(24.206 epoch 194476) v3 -7029> 2014-08-13 17:23:20.949393 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.888428, event: throttled, request: pg_query(24.206 epoch 194476) v3 -7028> 2014-08-13 17:23:20.949402 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.888503, event: all_read, request: pg_query(24.206 epoch 194476) v3 -7027> 2014-08-13 17:23:20.949376 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started 0.739385 0 0.000000 -7026> 2014-08-13 17:23:20.949414 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.949379, event: dispatched, request: pg_query(24.206 epoch 194476) v3 -7025> 2014-08-13 17:23:20.949424 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.949424, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194476) v3 -7024> 2014-08-13 17:23:20.949436 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.949435, event: started, request: pg_query(24.206 epoch 194476) v3 -7023> 2014-08-13 17:23:20.949424 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192966) [358,374,4] r=2 lpr=194471 pi=192969-194469/19 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset -7022> 2014-08-13 17:23:20.949456 7fcd0964c700 5 -- op tracker -- , seq: 38, time: 2014-08-13 17:23:20.949456, event: done, request: pg_query(24.206 epoch 194476) v3 -7021> 2014-08-13 17:23:20.949468 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.740863 3 0.000114 -7020> 2014-08-13 17:23:20.949487 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 2 ==== pg_query(22.4fe epoch 194476) v3 ==== 144+0+0 (490087461 0 0) 0x2f97620 con 0x4fd8f20 -7019> 2014-08-13 17:23:20.949507 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.915214, event: header_read, request: pg_query(22.4fe epoch 194476) v3 -7018> 2014-08-13 17:23:20.949499 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started 0.740940 0 0.000000 -7017> 2014-08-13 17:23:20.949525 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.915216, event: throttled, request: pg_query(22.4fe epoch 194476) v3 -7016> 2014-08-13 17:23:20.949549 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.915278, event: all_read, request: pg_query(22.4fe epoch 194476) v3 -7015> 2014-08-13 17:23:20.949561 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.949505, event: dispatched, request: pg_query(22.4fe epoch 194476) v3 -7014> 2014-08-13 17:23:20.949546 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192949) [328,364,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset -7013> 2014-08-13 17:23:20.949571 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.949571, event: waiting_for_osdmap, request: pg_query(22.4fe epoch 194476) v3 -7012> 2014-08-13 17:23:20.949580 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.949579, event: started, request: pg_query(22.4fe epoch 194476) v3 -7011> 2014-08-13 17:23:20.949603 7fcd0964c700 5 -- op tracker -- , seq: 39, time: 2014-08-13 17:23:20.949602, event: done, request: pg_query(22.4fe epoch 194476) v3 -7010> 2014-08-13 17:23:20.949647 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e4380 con 0x4fd86e0 -7009> 2014-08-13 17:23:20.949672 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7008> 2014-08-13 17:23:20.949689 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.332 192.168.1.110:6806/42024 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550a1c0 con 0x4fdbde0 -7007> 2014-08-13 17:23:20.949707 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -7006> 2014-08-13 17:23:20.949738 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 3 ==== pg_query(24.1f7 epoch 194476) v3 ==== 144+0+0 (1008181138 0 0) 0x2f92d00 con 0x4fdee00 -7005> 2014-08-13 17:23:20.949758 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.894094, event: header_read, request: pg_query(24.1f7 epoch 194476) v3 -7004> 2014-08-13 17:23:20.949741 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.000316 4 0.000518 -7003> 2014-08-13 17:23:20.949745 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.000199 4 0.000173 -7002> 2014-08-13 17:23:20.949773 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.894097, event: throttled, request: pg_query(24.1f7 epoch 194476) v3 -7001> 2014-08-13 17:23:20.949779 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started -7000> 2014-08-13 17:23:20.949796 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.894180, event: all_read, request: pg_query(24.1f7 epoch 194476) v3 -6999> 2014-08-13 17:23:20.949798 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start -6998> 2014-08-13 17:23:20.949789 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started -6997> 2014-08-13 17:23:20.949815 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.949756, event: dispatched, request: pg_query(24.1f7 epoch 194476) v3 -6996> 2014-08-13 17:23:20.949814 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6995> 2014-08-13 17:23:20.949839 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.949839, event: waiting_for_osdmap, request: pg_query(24.1f7 epoch 194476) v3 -6994> 2014-08-13 17:23:20.949829 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start -6993> 2014-08-13 17:23:20.949849 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.949849, event: started, request: pg_query(24.1f7 epoch 194476) v3 -6992> 2014-08-13 17:23:20.949854 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6991> 2014-08-13 17:23:20.949873 7fcd0964c700 5 -- op tracker -- , seq: 40, time: 2014-08-13 17:23:20.949873, event: done, request: pg_query(24.1f7 epoch 194476) v3 -6990> 2014-08-13 17:23:20.949858 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000060 0 0.000000 -6989> 2014-08-13 17:23:20.949878 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000051 0 0.000000 -6988> 2014-08-13 17:23:20.949900 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray -6987> 2014-08-13 17:23:20.949902 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550dc40 con 0x4fde880 -6986> 2014-08-13 17:23:20.949900 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray -6985> 2014-08-13 17:23:20.949939 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6984> 2014-08-13 17:23:20.949963 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x550de80 con 0x4fde5c0 -6983> 2014-08-13 17:23:20.949986 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -6982> 2014-08-13 17:23:20.950000 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 2 ==== pg_query(22.69f epoch 194476) v3 ==== 144+0+0 (1600536329 0 0) 0x2f97080 con 0x4fde040 -6981> 2014-08-13 17:23:20.950017 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.914918, event: header_read, request: pg_query(22.69f epoch 194476) v3 -6980> 2014-08-13 17:23:20.950028 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.914920, event: throttled, request: pg_query(22.69f epoch 194476) v3 -6979> 2014-08-13 17:23:20.950037 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.914997, event: all_read, request: pg_query(22.69f epoch 194476) v3 -6978> 2014-08-13 17:23:20.950046 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.950015, event: dispatched, request: pg_query(22.69f epoch 194476) v3 -6977> 2014-08-13 17:23:20.950055 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.950055, event: waiting_for_osdmap, request: pg_query(22.69f epoch 194476) v3 -6976> 2014-08-13 17:23:20.950068 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.950068, event: started, request: pg_query(22.69f epoch 194476) v3 -6975> 2014-08-13 17:23:20.950087 7fcd0964c700 5 -- op tracker -- , seq: 41, time: 2014-08-13 17:23:20.950086, event: done, request: pg_query(22.69f epoch 194476) v3 -6974> 2014-08-13 17:23:20.950107 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.24 192.168.1.107:6800/52669 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550a880 con 0x4fd11e0 -6973> 2014-08-13 17:23:20.950124 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6972> 2014-08-13 17:23:20.950133 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 3 ==== pg_query(22.639 epoch 194476) v3 ==== 144+0+0 (3459204413 0 0) 0x2f91860 con 0x4fd3860 -6971> 2014-08-13 17:23:20.950146 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.886200, event: header_read, request: pg_query(22.639 epoch 194476) v3 -6970> 2014-08-13 17:23:20.950158 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.886201, event: throttled, request: pg_query(22.639 epoch 194476) v3 -6969> 2014-08-13 17:23:20.950167 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.886286, event: all_read, request: pg_query(22.639 epoch 194476) v3 -6968> 2014-08-13 17:23:20.950187 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.950144, event: dispatched, request: pg_query(22.639 epoch 194476) v3 -6967> 2014-08-13 17:23:20.950204 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.950203, event: waiting_for_osdmap, request: pg_query(22.639 epoch 194476) v3 -6966> 2014-08-13 17:23:20.950215 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.950215, event: started, request: pg_query(22.639 epoch 194476) v3 -6965> 2014-08-13 17:23:20.950232 7fcd0964c700 5 -- op tracker -- , seq: 42, time: 2014-08-13 17:23:20.950232, event: done, request: pg_query(22.639 epoch 194476) v3 -6964> 2014-08-13 17:23:20.950252 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 2 ==== pg_query(24.cd epoch 194476) v3 ==== 144+0+0 (14695436 0 0) 0x2f97260 con 0x4fd0f20 -6963> 2014-08-13 17:23:20.950271 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.915034, event: header_read, request: pg_query(24.cd epoch 194476) v3 -6962> 2014-08-13 17:23:20.950281 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.915035, event: throttled, request: pg_query(24.cd epoch 194476) v3 -6961> 2014-08-13 17:23:20.950290 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.915084, event: all_read, request: pg_query(24.cd epoch 194476) v3 -6960> 2014-08-13 17:23:20.950299 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.950269, event: dispatched, request: pg_query(24.cd epoch 194476) v3 -6959> 2014-08-13 17:23:20.950310 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.950309, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194476) v3 -6958> 2014-08-13 17:23:20.950320 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.950320, event: started, request: pg_query(24.cd epoch 194476) v3 -6957> 2014-08-13 17:23:20.950338 7fcd0964c700 5 -- op tracker -- , seq: 43, time: 2014-08-13 17:23:20.950338, event: done, request: pg_query(24.cd epoch 194476) v3 -6956> 2014-08-13 17:23:20.950370 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 2 ==== pg_notify(24.3d(26) epoch 194476) v5 ==== 2206+0+0 (1292580142 0 0) 0x4e50fc0 con 0x4fd06e0 -6955> 2014-08-13 17:23:20.950391 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.915396, event: header_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6954> 2014-08-13 17:23:20.950398 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.915397, event: throttled, request: pg_notify(24.3d(26) epoch 194476) v5 -6953> 2014-08-13 17:23:20.950404 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.915509, event: all_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6952> 2014-08-13 17:23:20.950410 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.950389, event: dispatched, request: pg_notify(24.3d(26) epoch 194476) v5 -6951> 2014-08-13 17:23:20.950417 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.950417, event: waiting_for_osdmap, request: pg_notify(24.3d(26) epoch 194476) v5 -6950> 2014-08-13 17:23:20.950402 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit Reset 0.953834 6 0.000156 -6949> 2014-08-13 17:23:20.950429 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.950428, event: started, request: pg_notify(24.3d(26) epoch 194476) v5 -6948> 2014-08-13 17:23:20.950447 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter Started -6947> 2014-08-13 17:23:20.950474 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter Start -6946> 2014-08-13 17:23:20.950499 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: transitioning to Stray -6945> 2014-08-13 17:23:20.950494 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Reset 0.755555 6 0.000147 -6944> 2014-08-13 17:23:20.950529 7fcd0964c700 5 -- op tracker -- , seq: 44, time: 2014-08-13 17:23:20.950528, event: done, request: pg_notify(24.3d(26) epoch 194476) v5 -6943> 2014-08-13 17:23:20.950529 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started -6942> 2014-08-13 17:23:20.950551 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Start -6941> 2014-08-13 17:23:20.950554 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 3 ==== pg_query(22.39d epoch 194476) v3 ==== 144+0+0 (2270289633 0 0) 0x2f923a0 con 0x4fd32e0 -6940> 2014-08-13 17:23:20.950585 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.892773, event: header_read, request: pg_query(22.39d epoch 194476) v3 -6939> 2014-08-13 17:23:20.950575 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: transitioning to Stray -6938> 2014-08-13 17:23:20.950599 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.892775, event: throttled, request: pg_query(22.39d epoch 194476) v3 -6937> 2014-08-13 17:23:20.950608 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.892843, event: all_read, request: pg_query(22.39d epoch 194476) v3 -6936> 2014-08-13 17:23:20.950602 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Start 0.000051 0 0.000000 -6935> 2014-08-13 17:23:20.950617 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.950582, event: dispatched, request: pg_query(22.39d epoch 194476) v3 -6934> 2014-08-13 17:23:20.950638 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.950638, event: waiting_for_osdmap, request: pg_query(22.39d epoch 194476) v3 -6933> 2014-08-13 17:23:20.950631 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194472/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/Stray -6932> 2014-08-13 17:23:20.950656 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.950655, event: started, request: pg_query(22.39d epoch 194476) v3 -6931> 2014-08-13 17:23:20.950673 7fcd0964c700 5 -- op tracker -- , seq: 45, time: 2014-08-13 17:23:20.950673, event: done, request: pg_query(22.39d epoch 194476) v3 -6930> 2014-08-13 17:23:20.950522 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit Start 0.000047 0 0.000000 -6929> 2014-08-13 17:23:20.950708 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 2 ==== pg_query(24.299 epoch 194476) v3 ==== 144+0+0 (4084235348 0 0) 0x2f96cc0 con 0x4fd3020 -6928> 2014-08-13 17:23:20.950730 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.913943, event: header_read, request: pg_query(24.299 epoch 194476) v3 -6927> 2014-08-13 17:23:20.950722 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194466/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter Started/Stray -6926> 2014-08-13 17:23:20.950742 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.914021, event: throttled, request: pg_query(24.299 epoch 194476) v3 -6925> 2014-08-13 17:23:20.950752 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.914092, event: all_read, request: pg_query(24.299 epoch 194476) v3 -6924> 2014-08-13 17:23:20.950760 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.950728, event: dispatched, request: pg_query(24.299 epoch 194476) v3 -6923> 2014-08-13 17:23:20.950770 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.950769, event: waiting_for_osdmap, request: pg_query(24.299 epoch 194476) v3 -6922> 2014-08-13 17:23:20.950781 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.950781, event: started, request: pg_query(24.299 epoch 194476) v3 -6921> 2014-08-13 17:23:20.950782 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194476) v5 -- ?+0 0x56a8fc0 con 0x4e33de0 -6920> 2014-08-13 17:23:20.950799 7fcd0964c700 5 -- op tracker -- , seq: 46, time: 2014-08-13 17:23:20.950799, event: done, request: pg_query(24.299 epoch 194476) v3 -6919> 2014-08-13 17:23:20.950813 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.2b9(21) epoch 194476) v5 -- ?+0 0x56a8e00 con 0x4eb23c0 -6918> 2014-08-13 17:23:20.950822 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 2 ==== pg_notify(22.481(34) epoch 194476) v5 ==== 2628+0+0 (996605771 0 0) 0x32a8000 con 0x5303700 -6917> 2014-08-13 17:23:20.950838 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.913960, event: header_read, request: pg_notify(22.481(34) epoch 194476) v5 -6916> 2014-08-13 17:23:20.950844 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.914027, event: throttled, request: pg_notify(22.481(34) epoch 194476) v5 -6915> 2014-08-13 17:23:20.950850 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.914269, event: all_read, request: pg_notify(22.481(34) epoch 194476) v5 -6914> 2014-08-13 17:23:20.950855 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.950836, event: dispatched, request: pg_notify(22.481(34) epoch 194476) v5 -6913> 2014-08-13 17:23:20.950865 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.950865, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194476) v5 -6912> 2014-08-13 17:23:20.950873 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.950873, event: started, request: pg_notify(22.481(34) epoch 194476) v5 -6911> 2014-08-13 17:23:20.950883 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(32) epoch 194476) v5 -- ?+0 0x56a8c40 con 0x5305ee0 -6910> 2014-08-13 17:23:20.950894 7fcd0964c700 5 -- op tracker -- , seq: 47, time: 2014-08-13 17:23:20.950894, event: done, request: pg_notify(22.481(34) epoch 194476) v5 -6909> 2014-08-13 17:23:20.950906 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_notify(22.39d(24) epoch 194476) v5 -- ?+0 0x56a8a80 con 0x4fd32e0 -6908> 2014-08-13 17:23:20.950910 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 3 ==== pg_query(24.3f7 epoch 194476) v3 ==== 144+0+0 (453590065 0 0) 0x2f96180 con 0x5305ee0 -6907> 2014-08-13 17:23:20.950924 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.912455, event: header_read, request: pg_query(24.3f7 epoch 194476) v3 -6906> 2014-08-13 17:23:20.950931 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.912459, event: throttled, request: pg_query(24.3f7 epoch 194476) v3 -6905> 2014-08-13 17:23:20.950936 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.912533, event: all_read, request: pg_query(24.3f7 epoch 194476) v3 -6904> 2014-08-13 17:23:20.950941 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.950923, event: dispatched, request: pg_query(24.3f7 epoch 194476) v3 -6903> 2014-08-13 17:23:20.950947 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.950947, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194476) v3 -6902> 2014-08-13 17:23:20.950954 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.950954, event: started, request: pg_query(24.3f7 epoch 194476) v3 -6901> 2014-08-13 17:23:20.950942 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.740011 3 0.000266 -6900> 2014-08-13 17:23:20.950970 7fcd0964c700 5 -- op tracker -- , seq: 48, time: 2014-08-13 17:23:20.950970, event: done, request: pg_query(24.3f7 epoch 194476) v3 -6899> 2014-08-13 17:23:20.950978 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.740178 0 0.000000 -6898> 2014-08-13 17:23:20.950984 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 2 ==== pg_notify(24.9b(29) epoch 194476) v5 ==== 2383+0+0 (3603669103 0 0) 0x54d0a80 con 0x53056a0 -6897> 2014-08-13 17:23:20.950996 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.737961 3 0.000204 -6896> 2014-08-13 17:23:20.951012 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/194468) [4,47,378]/[19,65] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset -6895> 2014-08-13 17:23:20.951023 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.915767, event: header_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6894> 2014-08-13 17:23:20.951051 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.915769, event: throttled, request: pg_notify(24.9b(29) epoch 194476) v5 -6893> 2014-08-13 17:23:20.951065 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.915876, event: all_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6892> 2014-08-13 17:23:20.951072 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.951016, event: dispatched, request: pg_notify(24.9b(29) epoch 194476) v5 -6891> 2014-08-13 17:23:20.951079 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.951079, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194476) v5 -6890> 2014-08-13 17:23:20.951089 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.951088, event: started, request: pg_notify(24.9b(29) epoch 194476) v5 -6889> 2014-08-13 17:23:20.951121 7fcd0964c700 5 -- op tracker -- , seq: 49, time: 2014-08-13 17:23:20.951121, event: done, request: pg_notify(24.9b(29) epoch 194476) v5 -6888> 2014-08-13 17:23:20.951158 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550e780 con 0x53053e0 -6887> 2014-08-13 17:23:20.951023 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.738041 0 0.000000 -6886> 2014-08-13 17:23:20.951184 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6885> 2014-08-13 17:23:20.951207 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 2 ==== pg_notify(24.9b(26) epoch 194476) v5 ==== 2206+0+0 (2058123629 0 0) 0x4e50700 con 0x5305120 -6884> 2014-08-13 17:23:20.951224 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.914016, event: header_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6883> 2014-08-13 17:23:20.951209 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194470/194470/192958) [397,343,4] r=2 lpr=194471 pi=192965-194469/19 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -6882> 2014-08-13 17:23:20.951209 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.000198 4 0.000172 -6881> 2014-08-13 17:23:20.951232 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.914019, event: throttled, request: pg_notify(24.9b(26) epoch 194476) v5 -6880> 2014-08-13 17:23:20.951239 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.914325, event: all_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6879> 2014-08-13 17:23:20.951246 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.951221, event: dispatched, request: pg_notify(24.9b(26) epoch 194476) v5 -6878> 2014-08-13 17:23:20.951236 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started -6877> 2014-08-13 17:23:20.951252 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.951252, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194476) v5 -6876> 2014-08-13 17:23:20.951262 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.951262, event: started, request: pg_notify(24.9b(26) epoch 194476) v5 -6875> 2014-08-13 17:23:20.951255 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start -6874> 2014-08-13 17:23:20.951273 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6873> 2014-08-13 17:23:20.951288 7fcd0964c700 5 -- op tracker -- , seq: 50, time: 2014-08-13 17:23:20.951288, event: done, request: pg_notify(24.9b(26) epoch 194476) v5 -6872> 2014-08-13 17:23:20.951292 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000038 0 0.000000 -6871> 2014-08-13 17:23:20.951311 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x550cec0 con 0x5307a60 -6870> 2014-08-13 17:23:20.951312 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray -6869> 2014-08-13 17:23:20.951342 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -6868> 2014-08-13 17:23:20.951377 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 2 ==== pg_notify(22.113(23) epoch 194476) v5 ==== 2003+0+0 (3841010039 0 0) 0x54d08c0 con 0x5305c20 -6867> 2014-08-13 17:23:20.951386 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.000177 4 0.000306 -6866> 2014-08-13 17:23:20.951409 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.915612, event: header_read, request: pg_notify(22.113(23) epoch 194476) v5 -6865> 2014-08-13 17:23:20.951410 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -6864> 2014-08-13 17:23:20.951428 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.915613, event: throttled, request: pg_notify(22.113(23) epoch 194476) v5 -6863> 2014-08-13 17:23:20.951440 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.915721, event: all_read, request: pg_notify(22.113(23) epoch 194476) v5 -6862> 2014-08-13 17:23:20.951433 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -6861> 2014-08-13 17:23:20.951454 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.951405, event: dispatched, request: pg_notify(22.113(23) epoch 194476) v5 -6860> 2014-08-13 17:23:20.951468 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6859> 2014-08-13 17:23:20.951479 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.951479, event: waiting_for_osdmap, request: pg_notify(22.113(23) epoch 194476) v5 -6858> 2014-08-13 17:23:20.951496 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000064 0 0.000000 -6857> 2014-08-13 17:23:20.951490 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.740180 3 0.000199 -6856> 2014-08-13 17:23:20.951517 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.951517, event: started, request: pg_notify(22.113(23) epoch 194476) v5 -6855> 2014-08-13 17:23:20.951525 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194471/194471 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -6854> 2014-08-13 17:23:20.951527 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.740286 0 0.000000 -6853> 2014-08-13 17:23:20.951555 7fcd0964c700 5 -- op tracker -- , seq: 51, time: 2014-08-13 17:23:20.951554, event: done, request: pg_notify(22.113(23) epoch 194476) v5 -6852> 2014-08-13 17:23:20.951551 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194470/194470/192978) [373,360,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -6851> 2014-08-13 17:23:20.951595 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550d7c0 con 0x552a260 -6850> 2014-08-13 17:23:20.951621 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6849> 2014-08-13 17:23:20.951639 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 2 ==== pg_notify(22.113(3) epoch 194476) v5 ==== 887+0+0 (2650056438 0 0) 0x54d2300 con 0x5305960 -6848> 2014-08-13 17:23:20.951658 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.918133, event: header_read, request: pg_notify(22.113(3) epoch 194476) v5 -6847> 2014-08-13 17:23:20.951670 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.918134, event: throttled, request: pg_notify(22.113(3) epoch 194476) v5 -6846> 2014-08-13 17:23:20.951679 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.918209, event: all_read, request: pg_notify(22.113(3) epoch 194476) v5 -6845> 2014-08-13 17:23:20.951689 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.951656, event: dispatched, request: pg_notify(22.113(3) epoch 194476) v5 -6844> 2014-08-13 17:23:20.951677 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.738300 3 0.000215 -6843> 2014-08-13 17:23:20.951699 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.951699, event: waiting_for_osdmap, request: pg_notify(22.113(3) epoch 194476) v5 -6842> 2014-08-13 17:23:20.951711 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.951711, event: started, request: pg_notify(22.113(3) epoch 194476) v5 -6841> 2014-08-13 17:23:20.951703 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.738379 0 0.000000 -6840> 2014-08-13 17:23:20.951728 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194470/194470/192976) [30,335,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset -6839> 2014-08-13 17:23:20.951746 7fcd0964c700 5 -- op tracker -- , seq: 52, time: 2014-08-13 17:23:20.951745, event: done, request: pg_notify(22.113(3) epoch 194476) v5 -6838> 2014-08-13 17:23:20.951774 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.302 192.168.1.113:6810/15731 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550b180 con 0x552b9c0 -6837> 2014-08-13 17:23:20.951797 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6836> 2014-08-13 17:23:20.951794 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.000243 4 0.000166 -6835> 2014-08-13 17:23:20.951810 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550c380 con 0x552b700 -6834> 2014-08-13 17:23:20.951830 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6833> 2014-08-13 17:23:20.951829 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -6832> 2014-08-13 17:23:20.951846 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x5678480 con 0x552fa60 -6831> 2014-08-13 17:23:20.951850 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -6830> 2014-08-13 17:23:20.951864 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6829> 2014-08-13 17:23:20.951867 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6828> 2014-08-13 17:23:20.951878 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550e0c0 con 0x552f7a0 -6827> 2014-08-13 17:23:20.951897 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6826> 2014-08-13 17:23:20.951889 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 -6825> 2014-08-13 17:23:20.951909 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550d100 con 0x5520b00 -6824> 2014-08-13 17:23:20.951909 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194471/194471 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -6823> 2014-08-13 17:23:20.951910 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.000183 4 0.000132 -6822> 2014-08-13 17:23:20.951928 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6821> 2014-08-13 17:23:20.951942 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started -6820> 2014-08-13 17:23:20.951953 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x5679d40 con 0x552d540 -6819> 2014-08-13 17:23:20.951969 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6818> 2014-08-13 17:23:20.951964 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start -6817> 2014-08-13 17:23:20.951983 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550ca40 con 0x4e314a0 -6816> 2014-08-13 17:23:20.951988 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6815> 2014-08-13 17:23:20.952001 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6814> 2014-08-13 17:23:20.952013 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000049 0 0.000000 -6813> 2014-08-13 17:23:20.952022 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 3 ==== pg_notify(22.1d2(24) epoch 194476) v5 ==== 2062+0+0 (2408723064 0 0) 0x5184440 con 0x4e335a0 -6812> 2014-08-13 17:23:20.952039 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.875818, event: header_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -6811> 2014-08-13 17:23:20.952036 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194471/194471 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray -6810> 2014-08-13 17:23:20.952047 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.875822, event: throttled, request: pg_notify(22.1d2(24) epoch 194476) v5 -6809> 2014-08-13 17:23:20.952054 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.875978, event: all_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -6808> 2014-08-13 17:23:20.952063 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.952037, event: dispatched, request: pg_notify(22.1d2(24) epoch 194476) v5 -6807> 2014-08-13 17:23:20.952074 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.952074, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194476) v5 -6806> 2014-08-13 17:23:20.952064 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 0.740329 3 0.000309 -6805> 2014-08-13 17:23:20.952086 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.952085, event: started, request: pg_notify(22.1d2(24) epoch 194476) v5 -6804> 2014-08-13 17:23:20.952095 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started 0.740525 0 0.000000 -6803> 2014-08-13 17:23:20.952109 7fcd0964c700 5 -- op tracker -- , seq: 53, time: 2014-08-13 17:23:20.952109, event: done, request: pg_notify(22.1d2(24) epoch 194476) v5 -6802> 2014-08-13 17:23:20.952123 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194470/194471/193151) [356,364,4] r=2 lpr=194471 pi=193151-194470/24 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset -6801> 2014-08-13 17:23:20.952138 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 3 ==== pg_query(22.116 epoch 194476) v3 ==== 144+0+0 (1439576736 0 0) 0x2f979e0 con 0x4e32940 -6800> 2014-08-13 17:23:20.952169 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.915657, event: header_read, request: pg_query(22.116 epoch 194476) v3 -6799> 2014-08-13 17:23:20.952180 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.915660, event: throttled, request: pg_query(22.116 epoch 194476) v3 -6798> 2014-08-13 17:23:20.952189 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.915749, event: all_read, request: pg_query(22.116 epoch 194476) v3 -6797> 2014-08-13 17:23:20.952171 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6796> 2014-08-13 17:23:20.952196 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.952166, event: dispatched, request: pg_query(22.116 epoch 194476) v3 -6795> 2014-08-13 17:23:20.952206 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.952206, event: waiting_for_osdmap, request: pg_query(22.116 epoch 194476) v3 -6794> 2014-08-13 17:23:20.952215 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.952215, event: started, request: pg_query(22.116 epoch 194476) v3 -6793> 2014-08-13 17:23:20.952209 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.742311 2 0.000114 -6792> 2014-08-13 17:23:20.952237 7fcd0964c700 5 -- op tracker -- , seq: 54, time: 2014-08-13 17:23:20.952237, event: done, request: pg_query(22.116 epoch 194476) v3 -6791> 2014-08-13 17:23:20.952235 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.742350 0 0.000000 -6790> 2014-08-13 17:23:20.952257 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 3 ==== pg_notify(24.3d(4) epoch 194476) v5 ==== 980+0+0 (2174914663 0 0) 0x54d2680 con 0x4e33de0 -6789> 2014-08-13 17:23:20.952259 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.742385 0 0.000000 -6788> 2014-08-13 17:23:20.952277 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.918362, event: header_read, request: pg_notify(24.3d(4) epoch 194476) v5 -6787> 2014-08-13 17:23:20.952277 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.742449 0 0.000000 -6786> 2014-08-13 17:23:20.952293 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.918363, event: throttled, request: pg_notify(24.3d(4) epoch 194476) v5 -6785> 2014-08-13 17:23:20.952302 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.920987, event: all_read, request: pg_notify(24.3d(4) epoch 194476) v5 -6784> 2014-08-13 17:23:20.952299 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/194470) [4,50,289] r=0 lpr=194471 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset -6783> 2014-08-13 17:23:20.952311 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.952275, event: dispatched, request: pg_notify(24.3d(4) epoch 194476) v5 -6782> 2014-08-13 17:23:20.952321 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.952320, event: waiting_for_osdmap, request: pg_notify(24.3d(4) epoch 194476) v5 -6781> 2014-08-13 17:23:20.952332 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.952331, event: started, request: pg_notify(24.3d(4) epoch 194476) v5 -6780> 2014-08-13 17:23:20.952317 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.000195 4 0.000157 -6779> 2014-08-13 17:23:20.952363 7fcd0964c700 5 -- op tracker -- , seq: 55, time: 2014-08-13 17:23:20.952362, event: done, request: pg_notify(24.3d(4) epoch 194476) v5 -6778> 2014-08-13 17:23:20.952360 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started -6777> 2014-08-13 17:23:20.952383 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x5679680 con 0x4e34780 -6776> 2014-08-13 17:23:20.952382 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start -6775> 2014-08-13 17:23:20.952409 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6774> 2014-08-13 17:23:20.952413 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray -6773> 2014-08-13 17:23:20.952420 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 2 ==== pg_query(22.4aa epoch 194476) v3 ==== 144+0+0 (3820560088 0 0) 0x54f9fe0 con 0x4e35120 -6772> 2014-08-13 17:23:20.952436 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.918403, event: header_read, request: pg_query(22.4aa epoch 194476) v3 -6771> 2014-08-13 17:23:20.952447 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.918406, event: throttled, request: pg_query(22.4aa epoch 194476) v3 -6770> 2014-08-13 17:23:20.952437 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000055 0 0.000000 -6769> 2014-08-13 17:23:20.952459 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.918466, event: all_read, request: pg_query(22.4aa epoch 194476) v3 -6768> 2014-08-13 17:23:20.952466 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.952434, event: dispatched, request: pg_query(22.4aa epoch 194476) v3 -6767> 2014-08-13 17:23:20.952451 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000151 4 0.000217 -6766> 2014-08-13 17:23:20.952480 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.952480, event: waiting_for_osdmap, request: pg_query(22.4aa epoch 194476) v3 -6765> 2014-08-13 17:23:20.952469 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194466/194457 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray -6764> 2014-08-13 17:23:20.952498 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -6763> 2014-08-13 17:23:20.952489 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.952489, event: started, request: pg_query(22.4aa epoch 194476) v3 -6762> 2014-08-13 17:23:20.952520 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -6761> 2014-08-13 17:23:20.952541 7fcd0964c700 5 -- op tracker -- , seq: 56, time: 2014-08-13 17:23:20.952541, event: done, request: pg_query(22.4aa epoch 194476) v3 -6760> 2014-08-13 17:23:20.952538 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6759> 2014-08-13 17:23:20.952556 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 3 ==== pg_notify(22.84(1) epoch 194476) v5 ==== 769+0+0 (3649449316 0 0) 0x5185080 con 0x4e33f40 -6758> 2014-08-13 17:23:20.952557 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 -6757> 2014-08-13 17:23:20.952572 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.949464, event: header_read, request: pg_notify(22.84(1) epoch 194476) v5 -6756> 2014-08-13 17:23:20.952580 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.949465, event: throttled, request: pg_notify(22.84(1) epoch 194476) v5 -6755> 2014-08-13 17:23:20.952586 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.949547, event: all_read, request: pg_notify(22.84(1) epoch 194476) v5 -6754> 2014-08-13 17:23:20.952576 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6753> 2014-08-13 17:23:20.952594 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.952570, event: dispatched, request: pg_notify(22.84(1) epoch 194476) v5 -6752> 2014-08-13 17:23:20.952601 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.952601, event: waiting_for_osdmap, request: pg_notify(22.84(1) epoch 194476) v5 -6751> 2014-08-13 17:23:20.952597 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6750> 2014-08-13 17:23:20.952609 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.952609, event: started, request: pg_notify(22.84(1) epoch 194476) v5 -6749> 2014-08-13 17:23:20.952614 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6748> 2014-08-13 17:23:20.952630 7fcd0964c700 5 -- op tracker -- , seq: 57, time: 2014-08-13 17:23:20.952630, event: done, request: pg_notify(22.84(1) epoch 194476) v5 -6747> 2014-08-13 17:23:20.952649 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 3 ==== pg_notify(22.1d2(23) epoch 194476) v5 ==== 2003+0+0 (1337698436 0 0) 0x54d40c0 con 0x4e35ac0 -6746> 2014-08-13 17:23:20.952671 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.918648, event: header_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -6745> 2014-08-13 17:23:20.952681 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.918649, event: throttled, request: pg_notify(22.1d2(23) epoch 194476) v5 -6744> 2014-08-13 17:23:20.952690 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.922238, event: all_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -6743> 2014-08-13 17:23:20.952699 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.952669, event: dispatched, request: pg_notify(22.1d2(23) epoch 194476) v5 -6742> 2014-08-13 17:23:20.952709 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.952708, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194476) v5 -6741> 2014-08-13 17:23:20.952720 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.952720, event: started, request: pg_notify(22.1d2(23) epoch 194476) v5 -6740> 2014-08-13 17:23:20.952743 7fcd0964c700 5 -- op tracker -- , seq: 58, time: 2014-08-13 17:23:20.952743, event: done, request: pg_notify(22.1d2(23) epoch 194476) v5 -6739> 2014-08-13 17:23:20.952748 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] state: Peering, affected_by_map, going to Reset -6738> 2014-08-13 17:23:20.952771 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 3 ==== pg_notify(22.22c(23) epoch 194476) v5 ==== 2003+0+0 (2227283430 0 0) 0x54d1500 con 0x4e35800 -6737> 2014-08-13 17:23:20.952779 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] exit Started/Primary/Peering/GetInfo 0.742659 2 0.000139 -6736> 2014-08-13 17:23:20.952793 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.916738, event: header_read, request: pg_notify(22.22c(23) epoch 194476) v5 -6735> 2014-08-13 17:23:20.952816 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.916740, event: throttled, request: pg_notify(22.22c(23) epoch 194476) v5 -6734> 2014-08-13 17:23:20.952811 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 peering m=47] exit Started/Primary/Peering 0.742712 0 0.000000 -6733> 2014-08-13 17:23:20.952827 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.916865, event: all_read, request: pg_notify(22.22c(23) epoch 194476) v5 -6732> 2014-08-13 17:23:20.952836 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.952791, event: dispatched, request: pg_notify(22.22c(23) epoch 194476) v5 -6731> 2014-08-13 17:23:20.952833 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Started/Primary 0.742746 0 0.000000 -6730> 2014-08-13 17:23:20.952846 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.952846, event: waiting_for_osdmap, request: pg_notify(22.22c(23) epoch 194476) v5 -6729> 2014-08-13 17:23:20.952868 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.952868, event: started, request: pg_notify(22.22c(23) epoch 194476) v5 -6728> 2014-08-13 17:23:20.952860 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Started 0.742826 0 0.000000 -6727> 2014-08-13 17:23:20.952857 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit Reset 0.873597 6 0.000146 -6726> 2014-08-13 17:23:20.952884 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194470/194471/194471) [4,344,296]/[4,296,320] r=0 lpr=194471 pi=193679-194470/28 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset -6725> 2014-08-13 17:23:20.952897 7fcd0964c700 5 -- op tracker -- , seq: 59, time: 2014-08-13 17:23:20.952896, event: done, request: pg_notify(22.22c(23) epoch 194476) v5 -6724> 2014-08-13 17:23:20.952895 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter Started -6723> 2014-08-13 17:23:20.952931 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter Start -6722> 2014-08-13 17:23:20.952943 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 3 ==== pg_query(24.369 epoch 194476) v3 ==== 144+0+0 (4187133235 0 0) 0x2f93de0 con 0x4ec09a0 -6721> 2014-08-13 17:23:20.952953 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: transitioning to Stray -6720> 2014-08-13 17:23:20.952965 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.899902, event: header_read, request: pg_query(24.369 epoch 194476) v3 -6719> 2014-08-13 17:23:20.952977 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.899904, event: throttled, request: pg_query(24.369 epoch 194476) v3 -6718> 2014-08-13 17:23:20.952986 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.899988, event: all_read, request: pg_query(24.369 epoch 194476) v3 -6717> 2014-08-13 17:23:20.952979 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit Start 0.000047 0 0.000000 -6716> 2014-08-13 17:23:20.952995 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.952963, event: dispatched, request: pg_query(24.369 epoch 194476) v3 -6715> 2014-08-13 17:23:20.953005 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.953005, event: waiting_for_osdmap, request: pg_query(24.369 epoch 194476) v3 -6714> 2014-08-13 17:23:20.953000 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194466/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter Started/Stray -6713> 2014-08-13 17:23:20.953017 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.953017, event: started, request: pg_query(24.369 epoch 194476) v3 -6712> 2014-08-13 17:23:20.953039 7fcd0964c700 5 -- op tracker -- , seq: 60, time: 2014-08-13 17:23:20.953039, event: done, request: pg_query(24.369 epoch 194476) v3 -6711> 2014-08-13 17:23:20.953068 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 2 ==== pg_query(22.30f epoch 194476) v3 ==== 144+0+0 (3983190957 0 0) 0x54fa1c0 con 0x4ec2aa0 -6710> 2014-08-13 17:23:20.953087 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.920176, event: header_read, request: pg_query(22.30f epoch 194476) v3 -6709> 2014-08-13 17:23:20.953097 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.920178, event: throttled, request: pg_query(22.30f epoch 194476) v3 -6708> 2014-08-13 17:23:20.953108 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.920237, event: all_read, request: pg_query(22.30f epoch 194476) v3 -6707> 2014-08-13 17:23:20.953116 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.953085, event: dispatched, request: pg_query(22.30f epoch 194476) v3 -6706> 2014-08-13 17:23:20.953126 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.953125, event: waiting_for_osdmap, request: pg_query(22.30f epoch 194476) v3 -6705> 2014-08-13 17:23:20.953136 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.953136, event: started, request: pg_query(22.30f epoch 194476) v3 -6704> 2014-08-13 17:23:20.953119 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Reset 0.000236 4 0.000258 -6703> 2014-08-13 17:23:20.953151 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started -6702> 2014-08-13 17:23:20.953165 7fcd0964c700 5 -- op tracker -- , seq: 61, time: 2014-08-13 17:23:20.953165, event: done, request: pg_query(22.30f epoch 194476) v3 -6701> 2014-08-13 17:23:20.953180 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Start -6700> 2014-08-13 17:23:20.953196 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550ba80 con 0x4ec1ce0 -6699> 2014-08-13 17:23:20.953198 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: transitioning to Stray -6698> 2014-08-13 17:23:20.953221 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6697> 2014-08-13 17:23:20.953234 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Start 0.000054 0 0.000000 -6696> 2014-08-13 17:23:20.953246 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550da00 con 0x4ec27e0 -6695> 2014-08-13 17:23:20.953263 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6694> 2014-08-13 17:23:20.953259 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194468/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started/Stray -6693> 2014-08-13 17:23:20.953275 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 2 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x51e69c0 con 0x4ec3440 -6692> 2014-08-13 17:23:20.953293 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6691> 2014-08-13 17:23:20.953305 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 3 ==== pg_notify(22.4df(23) epoch 194476) v5 ==== 2003+0+0 (3530789080 0 0) 0x5184ec0 con 0x4ec4780 -6690> 2014-08-13 17:23:20.953323 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.906810, event: header_read, request: pg_notify(22.4df(23) epoch 194476) v5 -6689> 2014-08-13 17:23:20.953333 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.906811, event: throttled, request: pg_notify(22.4df(23) epoch 194476) v5 -6688> 2014-08-13 17:23:20.953342 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.906908, event: all_read, request: pg_notify(22.4df(23) epoch 194476) v5 -6687> 2014-08-13 17:23:20.953351 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.953320, event: dispatched, request: pg_notify(22.4df(23) epoch 194476) v5 -6686> 2014-08-13 17:23:20.953361 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.953360, event: waiting_for_osdmap, request: pg_notify(22.4df(23) epoch 194476) v5 -6685> 2014-08-13 17:23:20.953372 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.953372, event: started, request: pg_notify(22.4df(23) epoch 194476) v5 -6684> 2014-08-13 17:23:20.953397 7fcd0964c700 5 -- op tracker -- , seq: 62, time: 2014-08-13 17:23:20.953397, event: done, request: pg_notify(22.4df(23) epoch 194476) v5 -6683> 2014-08-13 17:23:20.953421 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 3 ==== pg_query(24.b6 epoch 194476) v3 ==== 144+0+0 (2501115399 0 0) 0x2f93a20 con 0x4ec5ac0 -6682> 2014-08-13 17:23:20.953442 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.895480, event: header_read, request: pg_query(24.b6 epoch 194476) v3 -6681> 2014-08-13 17:23:20.953453 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.895484, event: throttled, request: pg_query(24.b6 epoch 194476) v3 -6680> 2014-08-13 17:23:20.953445 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(21) epoch 194476) v5 -- ?+0 0x54d40c0 con 0x4ec3f40 -6679> 2014-08-13 17:23:20.953462 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.895770, event: all_read, request: pg_query(24.b6 epoch 194476) v3 -6678> 2014-08-13 17:23:20.953455 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit Reset 0.926424 6 0.000157 -6677> 2014-08-13 17:23:20.953488 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.953440, event: dispatched, request: pg_query(24.b6 epoch 194476) v3 -6676> 2014-08-13 17:23:20.953488 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6806/31286 -- pg_notify(24.181(31) epoch 194476) v5 -- ?+0 0x54d2680 con 0x4f81600 -6675> 2014-08-13 17:23:20.953502 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.953502, event: waiting_for_osdmap, request: pg_query(24.b6 epoch 194476) v3 -6674> 2014-08-13 17:23:20.953493 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter Started -6673> 2014-08-13 17:23:20.953518 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.953518, event: started, request: pg_query(24.b6 epoch 194476) v3 -6672> 2014-08-13 17:23:20.953510 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(21) epoch 194476) v5 -- ?+0 0x54d2300 con 0x4fd53e0 -6671> 2014-08-13 17:23:20.953537 7fcd0964c700 5 -- op tracker -- , seq: 63, time: 2014-08-13 17:23:20.953537, event: done, request: pg_query(24.b6 epoch 194476) v3 -6670> 2014-08-13 17:23:20.953528 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter Start -6669> 2014-08-13 17:23:20.953542 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_query(22.6bd epoch 194476) v3 -- ?+0 0x54fba20 con 0x4eb7d20 -6668> 2014-08-13 17:23:20.953548 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: transitioning to Stray -6667> 2014-08-13 17:23:20.953562 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_query(22.6bd epoch 194476) v3 -- ?+0 0x54fb840 con 0x4ec3440 -6666> 2014-08-13 17:23:20.953565 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 3 ==== pg_query(22.1a0 epoch 194476) v3 ==== 144+0+0 (2992041526 0 0) 0x2f94380 con 0x4eb23c0 -6665> 2014-08-13 17:23:20.953582 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.900716, event: header_read, request: pg_query(22.1a0 epoch 194476) v3 -6664> 2014-08-13 17:23:20.953573 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit Start 0.000044 0 0.000000 -6663> 2014-08-13 17:23:20.953589 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.900718, event: throttled, request: pg_query(22.1a0 epoch 194476) v3 -6662> 2014-08-13 17:23:20.953594 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.900776, event: all_read, request: pg_query(22.1a0 epoch 194476) v3 -6661> 2014-08-13 17:23:20.953600 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.953580, event: dispatched, request: pg_query(22.1a0 epoch 194476) v3 -6660> 2014-08-13 17:23:20.953593 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194465/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter Started/Stray -6659> 2014-08-13 17:23:20.953606 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.953606, event: waiting_for_osdmap, request: pg_query(22.1a0 epoch 194476) v3 -6658> 2014-08-13 17:23:20.953614 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.953614, event: started, request: pg_query(22.1a0 epoch 194476) v3 -6657> 2014-08-13 17:23:20.953631 7fcd0964c700 5 -- op tracker -- , seq: 64, time: 2014-08-13 17:23:20.953630, event: done, request: pg_query(22.1a0 epoch 194476) v3 -6656> 2014-08-13 17:23:20.953658 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 3 ==== pg_query(24.2ec epoch 194476) v3 ==== 144+0+0 (2761021050 0 0) 0x2f94b00 con 0x4eb18c0 -6655> 2014-08-13 17:23:20.953678 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.907181, event: header_read, request: pg_query(24.2ec epoch 194476) v3 -6654> 2014-08-13 17:23:20.953689 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.907182, event: throttled, request: pg_query(24.2ec epoch 194476) v3 -6653> 2014-08-13 17:23:20.953697 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.907812, event: all_read, request: pg_query(24.2ec epoch 194476) v3 -6652> 2014-08-13 17:23:20.953706 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.953675, event: dispatched, request: pg_query(24.2ec epoch 194476) v3 -6651> 2014-08-13 17:23:20.953715 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.953715, event: waiting_for_osdmap, request: pg_query(24.2ec epoch 194476) v3 -6650> 2014-08-13 17:23:20.953726 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.953726, event: started, request: pg_query(24.2ec epoch 194476) v3 -6649> 2014-08-13 17:23:20.953744 7fcd0964c700 5 -- op tracker -- , seq: 65, time: 2014-08-13 17:23:20.953744, event: done, request: pg_query(24.2ec epoch 194476) v3 -6648> 2014-08-13 17:23:20.953765 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 2 ==== pg_query(22.630 epoch 194476) v3 ==== 144+0+0 (78741442 0 0) 0x2f97440 con 0x4eb40a0 -6647> 2014-08-13 17:23:20.953756 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.743373 3 0.000120 -6646> 2014-08-13 17:23:20.953767 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.741370 3 0.000116 -6645> 2014-08-13 17:23:20.953785 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.915072, event: header_read, request: pg_query(22.630 epoch 194476) v3 -6644> 2014-08-13 17:23:20.953788 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started 0.743455 0 0.000000 -6643> 2014-08-13 17:23:20.953811 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.915074, event: throttled, request: pg_query(22.630 epoch 194476) v3 -6642> 2014-08-13 17:23:20.953804 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.741451 0 0.000000 -6641> 2014-08-13 17:23:20.953825 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.915124, event: all_read, request: pg_query(22.630 epoch 194476) v3 -6640> 2014-08-13 17:23:20.953813 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194470/194471/193264) [15,4,52]/[15,4,61] r=1 lpr=194471 pi=193264-194470/26 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset -6639> 2014-08-13 17:23:20.953828 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194470/194470/192943) [337,20,4] r=2 lpr=194471 pi=192976-194469/19 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -6638> 2014-08-13 17:23:20.953838 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.953783, event: dispatched, request: pg_query(22.630 epoch 194476) v3 -6637> 2014-08-13 17:23:20.953866 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.953866, event: waiting_for_osdmap, request: pg_query(22.630 epoch 194476) v3 -6636> 2014-08-13 17:23:20.953880 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.953879, event: started, request: pg_query(22.630 epoch 194476) v3 -6635> 2014-08-13 17:23:20.953899 7fcd0964c700 5 -- op tracker -- , seq: 66, time: 2014-08-13 17:23:20.953899, event: done, request: pg_query(22.630 epoch 194476) v3 -6634> 2014-08-13 17:23:20.953920 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 2 ==== pg_query(22.1f1 epoch 194476) v3 ==== 144+0+0 (700933093 0 0) 0x2f90960 con 0x4eb2680 -6633> 2014-08-13 17:23:20.953939 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.915652, event: header_read, request: pg_query(22.1f1 epoch 194476) v3 -6632> 2014-08-13 17:23:20.953949 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.915654, event: throttled, request: pg_query(22.1f1 epoch 194476) v3 -6631> 2014-08-13 17:23:20.953957 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.915721, event: all_read, request: pg_query(22.1f1 epoch 194476) v3 -6630> 2014-08-13 17:23:20.953966 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.953937, event: dispatched, request: pg_query(22.1f1 epoch 194476) v3 -6629> 2014-08-13 17:23:20.953972 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.953972, event: waiting_for_osdmap, request: pg_query(22.1f1 epoch 194476) v3 -6628> 2014-08-13 17:23:20.954013 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.954013, event: started, request: pg_query(22.1f1 epoch 194476) v3 -6627> 2014-08-13 17:23:20.954032 7fcd0964c700 5 -- op tracker -- , seq: 67, time: 2014-08-13 17:23:20.954032, event: done, request: pg_query(22.1f1 epoch 194476) v3 -6626> 2014-08-13 17:23:20.954053 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 3 ==== pg_query(24.136 epoch 194476) v3 ==== 144+0+0 (950190490 0 0) 0x2f93660 con 0x4eb7d20 -6625> 2014-08-13 17:23:20.954073 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.895047, event: header_read, request: pg_query(24.136 epoch 194476) v3 -6624> 2014-08-13 17:23:20.954065 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.000237 4 0.000155 -6623> 2014-08-13 17:23:20.954083 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.895048, event: throttled, request: pg_query(24.136 epoch 194476) v3 -6622> 2014-08-13 17:23:20.954104 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.895096, event: all_read, request: pg_query(24.136 epoch 194476) v3 -6621> 2014-08-13 17:23:20.954098 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -6620> 2014-08-13 17:23:20.954086 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000273 4 0.000186 -6619> 2014-08-13 17:23:20.954117 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.954071, event: dispatched, request: pg_query(24.136 epoch 194476) v3 -6618> 2014-08-13 17:23:20.954116 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -6617> 2014-08-13 17:23:20.954142 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.954141, event: waiting_for_osdmap, request: pg_query(24.136 epoch 194476) v3 -6616> 2014-08-13 17:23:20.954132 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started -6615> 2014-08-13 17:23:20.954152 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.954151, event: started, request: pg_query(24.136 epoch 194476) v3 -6614> 2014-08-13 17:23:20.954151 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Start -6613> 2014-08-13 17:23:20.954150 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6612> 2014-08-13 17:23:20.954181 7fcd0964c700 5 -- op tracker -- , seq: 68, time: 2014-08-13 17:23:20.954181, event: done, request: pg_query(24.136 epoch 194476) v3 -6611> 2014-08-13 17:23:20.954172 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -6610> 2014-08-13 17:23:20.954184 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000068 0 0.000000 -6609> 2014-08-13 17:23:20.954213 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 2 ==== pg_query(22.2d1 epoch 194476) v3 ==== 144+0+0 (608296682 0 0) 0x54f85a0 con 0x4eb6720 -6608> 2014-08-13 17:23:20.954215 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000063 0 0.000000 -6607> 2014-08-13 17:23:20.954222 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194471/194471 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -6606> 2014-08-13 17:23:20.954233 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.917214, event: header_read, request: pg_query(22.2d1 epoch 194476) v3 -6605> 2014-08-13 17:23:20.954237 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194466/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -6604> 2014-08-13 17:23:20.954255 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.917216, event: throttled, request: pg_query(22.2d1 epoch 194476) v3 -6603> 2014-08-13 17:23:20.954265 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.917280, event: all_read, request: pg_query(22.2d1 epoch 194476) v3 -6602> 2014-08-13 17:23:20.954275 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.954231, event: dispatched, request: pg_query(22.2d1 epoch 194476) v3 -6601> 2014-08-13 17:23:20.954284 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.954284, event: waiting_for_osdmap, request: pg_query(22.2d1 epoch 194476) v3 -6600> 2014-08-13 17:23:20.954295 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.954295, event: started, request: pg_query(22.2d1 epoch 194476) v3 -6599> 2014-08-13 17:23:20.954333 7fcd0964c700 5 -- op tracker -- , seq: 69, time: 2014-08-13 17:23:20.954333, event: done, request: pg_query(22.2d1 epoch 194476) v3 -6598> 2014-08-13 17:23:20.954357 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 2 ==== pg_query(22.12a epoch 194476) v3 ==== 144+0+0 (3008252166 0 0) 0x2f96900 con 0x4f81fa0 -6597> 2014-08-13 17:23:20.954376 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.946215, event: header_read, request: pg_query(22.12a epoch 194476) v3 -6596> 2014-08-13 17:23:20.954386 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.946217, event: throttled, request: pg_query(22.12a epoch 194476) v3 -6595> 2014-08-13 17:23:20.954372 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.741775 3 0.000116 -6594> 2014-08-13 17:23:20.954395 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.946282, event: all_read, request: pg_query(22.12a epoch 194476) v3 -6593> 2014-08-13 17:23:20.954405 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.954374, event: dispatched, request: pg_query(22.12a epoch 194476) v3 -6592> 2014-08-13 17:23:20.954400 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.741854 0 0.000000 -6591> 2014-08-13 17:23:20.954387 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started/Stray 0.743777 3 0.000120 -6590> 2014-08-13 17:23:20.954420 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.954420, event: waiting_for_osdmap, request: pg_query(22.12a epoch 194476) v3 -6589> 2014-08-13 17:23:20.954429 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.954429, event: started, request: pg_query(22.12a epoch 194476) v3 -6588> 2014-08-13 17:23:20.954422 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194470/194470/192970) [371,4,21] r=1 lpr=194471 pi=192977-194469/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset -6587> 2014-08-13 17:23:20.954429 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started 0.743869 0 0.000000 -6586> 2014-08-13 17:23:20.954457 7fcd0964c700 5 -- op tracker -- , seq: 70, time: 2014-08-13 17:23:20.954456, event: done, request: pg_query(22.12a epoch 194476) v3 -6585> 2014-08-13 17:23:20.954455 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194470/194471/192978) [31,338,4] r=2 lpr=194471 pi=193125-194470/34 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset -6584> 2014-08-13 17:23:20.954483 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 2 ==== pg_query(22.3ad epoch 194476) v3 ==== 144+0+0 (472098115 0 0) 0x54fa3a0 con 0x4f870c0 -6583> 2014-08-13 17:23:20.954498 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.920381, event: header_read, request: pg_query(22.3ad epoch 194476) v3 -6582> 2014-08-13 17:23:20.954505 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.920383, event: throttled, request: pg_query(22.3ad epoch 194476) v3 -6581> 2014-08-13 17:23:20.954510 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.920437, event: all_read, request: pg_query(22.3ad epoch 194476) v3 -6580> 2014-08-13 17:23:20.954515 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.954496, event: dispatched, request: pg_query(22.3ad epoch 194476) v3 -6579> 2014-08-13 17:23:20.954524 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.954524, event: waiting_for_osdmap, request: pg_query(22.3ad epoch 194476) v3 -6578> 2014-08-13 17:23:20.954535 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.954535, event: started, request: pg_query(22.3ad epoch 194476) v3 -6577> 2014-08-13 17:23:20.954550 7fcd0964c700 5 -- op tracker -- , seq: 71, time: 2014-08-13 17:23:20.954550, event: done, request: pg_query(22.3ad epoch 194476) v3 -6576> 2014-08-13 17:23:20.954575 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 2 ==== pg_notify(22.4df(22) epoch 194476) v5 ==== 1944+0+0 (309562417 0 0) 0x54d3800 con 0x4fd86e0 -6575> 2014-08-13 17:23:20.954616 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.919757, event: header_read, request: pg_notify(22.4df(22) epoch 194476) v5 -6574> 2014-08-13 17:23:20.954629 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.919778, event: throttled, request: pg_notify(22.4df(22) epoch 194476) v5 -6573> 2014-08-13 17:23:20.954638 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.921144, event: all_read, request: pg_notify(22.4df(22) epoch 194476) v5 -6572> 2014-08-13 17:23:20.954627 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] exit Reset 0.000204 4 0.000164 -6571> 2014-08-13 17:23:20.954649 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.954613, event: dispatched, request: pg_notify(22.4df(22) epoch 194476) v5 -6570> 2014-08-13 17:23:20.954661 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.954661, event: waiting_for_osdmap, request: pg_notify(22.4df(22) epoch 194476) v5 -6569> 2014-08-13 17:23:20.954655 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] enter Started -6568> 2014-08-13 17:23:20.954673 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.954672, event: started, request: pg_notify(22.4df(22) epoch 194476) v5 -6567> 2014-08-13 17:23:20.954672 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] enter Start -6566> 2014-08-13 17:23:20.954689 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6565> 2014-08-13 17:23:20.954707 7fcd0964c700 5 -- op tracker -- , seq: 72, time: 2014-08-13 17:23:20.954706, event: done, request: pg_notify(22.4df(22) epoch 194476) v5 -6564> 2014-08-13 17:23:20.954688 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Reset 0.000232 4 0.000171 -6563> 2014-08-13 17:23:20.954711 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] exit Start 0.000039 0 0.000000 -6562> 2014-08-13 17:23:20.954738 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started -6561> 2014-08-13 17:23:20.954749 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.332 192.168.1.110:6806/42024 2 ==== pg_query(24.1bb epoch 194476) v3 ==== 144+0+0 (3559930960 0 0) 0x54fa580 con 0x4fdbde0 -6560> 2014-08-13 17:23:20.954749 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194471/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] enter Started/Stray -6559> 2014-08-13 17:23:20.954767 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.920459, event: header_read, request: pg_query(24.1bb epoch 194476) v3 -6558> 2014-08-13 17:23:20.954762 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Start -6557> 2014-08-13 17:23:20.954776 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.920461, event: throttled, request: pg_query(24.1bb epoch 194476) v3 -6556> 2014-08-13 17:23:20.954782 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.920545, event: all_read, request: pg_query(24.1bb epoch 194476) v3 -6555> 2014-08-13 17:23:20.954792 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.954764, event: dispatched, request: pg_query(24.1bb epoch 194476) v3 -6554> 2014-08-13 17:23:20.954783 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: transitioning to Stray -6553> 2014-08-13 17:23:20.954802 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.954802, event: waiting_for_osdmap, request: pg_query(24.1bb epoch 194476) v3 -6552> 2014-08-13 17:23:20.954814 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.954813, event: started, request: pg_query(24.1bb epoch 194476) v3 -6551> 2014-08-13 17:23:20.954805 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Start 0.000043 0 0.000000 -6550> 2014-08-13 17:23:20.954836 7fcd0964c700 5 -- op tracker -- , seq: 73, time: 2014-08-13 17:23:20.954836, event: done, request: pg_query(24.1bb epoch 194476) v3 -6549> 2014-08-13 17:23:20.954825 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194465/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started/Stray -6548> 2014-08-13 17:23:20.954861 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 2 ==== pg_notify(22.1d2(3) epoch 194476) v5 ==== 887+0+0 (579020996 0 0) 0x54d5940 con 0x4fde880 -6547> 2014-08-13 17:23:20.954881 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.946983, event: header_read, request: pg_notify(22.1d2(3) epoch 194476) v5 -6546> 2014-08-13 17:23:20.954892 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.946985, event: throttled, request: pg_notify(22.1d2(3) epoch 194476) v5 -6545> 2014-08-13 17:23:20.954903 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.947093, event: all_read, request: pg_notify(22.1d2(3) epoch 194476) v5 -6544> 2014-08-13 17:23:20.954911 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.954879, event: dispatched, request: pg_notify(22.1d2(3) epoch 194476) v5 -6543> 2014-08-13 17:23:20.954921 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.954921, event: waiting_for_osdmap, request: pg_notify(22.1d2(3) epoch 194476) v5 -6542> 2014-08-13 17:23:20.954910 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.742070 3 0.000136 -6541> 2014-08-13 17:23:20.954949 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started 0.742162 0 0.000000 -6540> 2014-08-13 17:23:20.954932 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.954932, event: started, request: pg_notify(22.1d2(3) epoch 194476) v5 -6539> 2014-08-13 17:23:20.954973 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194470/194471/192969) [356,4,334]/[356,4,32] r=1 lpr=194471 pi=193150-194470/29 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset -6538> 2014-08-13 17:23:20.955006 7fcd0964c700 5 -- op tracker -- , seq: 74, time: 2014-08-13 17:23:20.955006, event: done, request: pg_notify(22.1d2(3) epoch 194476) v5 -6537> 2014-08-13 17:23:20.955018 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(29) epoch 194476) v5 -- ?+0 0x54d08c0 con 0x4f80b00 -6536> 2014-08-13 17:23:20.955031 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 2 ==== pg_notify(22.22c(24) epoch 194476) v5 ==== 2062+0+0 (1668933989 0 0) 0x54d6900 con 0x4fde5c0 -6535> 2014-08-13 17:23:20.955052 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.947273, event: header_read, request: pg_notify(22.22c(24) epoch 194476) v5 -6534> 2014-08-13 17:23:20.955051 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(37) epoch 194476) v5 -- ?+0 0x4e50fc0 con 0x4ec56a0 -6533> 2014-08-13 17:23:20.955067 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.947275, event: throttled, request: pg_notify(22.22c(24) epoch 194476) v5 -6532> 2014-08-13 17:23:20.955077 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.947378, event: all_read, request: pg_notify(22.22c(24) epoch 194476) v5 -6531> 2014-08-13 17:23:20.955087 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.955049, event: dispatched, request: pg_notify(22.22c(24) epoch 194476) v5 -6530> 2014-08-13 17:23:20.955094 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.955094, event: waiting_for_osdmap, request: pg_notify(22.22c(24) epoch 194476) v5 -6529> 2014-08-13 17:23:20.955102 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.955102, event: started, request: pg_notify(22.22c(24) epoch 194476) v5 -6528> 2014-08-13 17:23:20.955123 7fcd0964c700 5 -- op tracker -- , seq: 75, time: 2014-08-13 17:23:20.955122, event: done, request: pg_notify(22.22c(24) epoch 194476) v5 -6527> 2014-08-13 17:23:20.955139 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.744351 3 0.000100 -6526> 2014-08-13 17:23:20.955151 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.24 192.168.1.107:6800/52669 2 ==== pg_notify(22.113(24) epoch 194476) v5 ==== 2062+0+0 (804026585 0 0) 0x54d3d40 con 0x4fd11e0 -6525> 2014-08-13 17:23:20.955175 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.921070, event: header_read, request: pg_notify(22.113(24) epoch 194476) v5 -6524> 2014-08-13 17:23:20.955167 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started 0.744417 0 0.000000 -6523> 2014-08-13 17:23:20.955183 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.921072, event: throttled, request: pg_notify(22.113(24) epoch 194476) v5 -6522> 2014-08-13 17:23:20.955189 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.921173, event: all_read, request: pg_notify(22.113(24) epoch 194476) v5 -6521> 2014-08-13 17:23:20.955196 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.955173, event: dispatched, request: pg_notify(22.113(24) epoch 194476) v5 -6520> 2014-08-13 17:23:20.955202 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.955202, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194476) v5 -6519> 2014-08-13 17:23:20.955188 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194470/194471/192975) [285,336,4] r=2 lpr=194471 pi=193125-194470/24 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset -6518> 2014-08-13 17:23:20.955210 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.955210, event: started, request: pg_notify(22.113(24) epoch 194476) v5 -6517> 2014-08-13 17:23:20.955231 7fcd0964c700 5 -- op tracker -- , seq: 76, time: 2014-08-13 17:23:20.955231, event: done, request: pg_notify(22.113(24) epoch 194476) v5 -6516> 2014-08-13 17:23:20.955259 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 4 ==== pg_notify(24.9b(26) epoch 194476) v5 ==== 2206+0+0 (2058123629 0 0) 0x5184b40 con 0x4fd3860 -6515> 2014-08-13 17:23:20.955279 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.886303, event: header_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6514> 2014-08-13 17:23:20.955289 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.886304, event: throttled, request: pg_notify(24.9b(26) epoch 194476) v5 -6513> 2014-08-13 17:23:20.955273 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000299 4 0.000183 -6512> 2014-08-13 17:23:20.955299 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.886440, event: all_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6511> 2014-08-13 17:23:20.955308 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.955277, event: dispatched, request: pg_notify(24.9b(26) epoch 194476) v5 -6510> 2014-08-13 17:23:20.955305 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started -6509> 2014-08-13 17:23:20.955320 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.955320, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194476) v5 -6508> 2014-08-13 17:23:20.955334 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Start -6507> 2014-08-13 17:23:20.955351 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.955351, event: started, request: pg_notify(24.9b(26) epoch 194476) v5 -6506> 2014-08-13 17:23:20.955355 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -6505> 2014-08-13 17:23:20.955378 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000044 0 0.000000 -6504> 2014-08-13 17:23:20.955378 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000190 4 0.000141 -6503> 2014-08-13 17:23:20.955403 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194466/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -6502> 2014-08-13 17:23:20.955426 7fcd0964c700 5 -- op tracker -- , seq: 77, time: 2014-08-13 17:23:20.955426, event: done, request: pg_notify(24.9b(26) epoch 194476) v5 -6501> 2014-08-13 17:23:20.955404 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started -6500> 2014-08-13 17:23:20.955455 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start -6499> 2014-08-13 17:23:20.955466 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 3 ==== pg_notify(22.481(34) epoch 194476) v5 ==== 2628+0+0 (3710564698 0 0) 0x54d0e00 con 0x4fd06e0 -6498> 2014-08-13 17:23:20.955478 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -6497> 2014-08-13 17:23:20.955494 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.915931, event: header_read, request: pg_notify(22.481(34) epoch 194476) v5 -6496> 2014-08-13 17:23:20.955506 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.915933, event: throttled, request: pg_notify(22.481(34) epoch 194476) v5 -6495> 2014-08-13 17:23:20.955498 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000042 0 0.000000 -6494> 2014-08-13 17:23:20.955520 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.916207, event: all_read, request: pg_notify(22.481(34) epoch 194476) v5 -6493> 2014-08-13 17:23:20.955527 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.955491, event: dispatched, request: pg_notify(22.481(34) epoch 194476) v5 -6492> 2014-08-13 17:23:20.955538 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.955538, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194476) v5 -6491> 2014-08-13 17:23:20.955530 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194466/194457 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -6490> 2014-08-13 17:23:20.955558 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.955557, event: started, request: pg_notify(22.481(34) epoch 194476) v5 -6489> 2014-08-13 17:23:20.955587 7fcd0964c700 5 -- op tracker -- , seq: 78, time: 2014-08-13 17:23:20.955586, event: done, request: pg_notify(22.481(34) epoch 194476) v5 -6488> 2014-08-13 17:23:20.955595 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_notify(24.b6(28) epoch 194476) v5 -- ?+0 0x5187c40 con 0x4ec5ac0 -6487> 2014-08-13 17:23:20.955616 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 4 ==== pg_query(22.1fb epoch 194476) v3 ==== 144+0+0 (2425598903 0 0) 0x2f92940 con 0x4fd32e0 -6486> 2014-08-13 17:23:20.955637 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.893900, event: header_read, request: pg_query(22.1fb epoch 194476) v3 -6485> 2014-08-13 17:23:20.955630 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6818/61186 -- pg_notify(24.3bf(5) epoch 194476) v5 -- ?+0 0x4e50c40 con 0x4f84ba0 -6484> 2014-08-13 17:23:20.955671 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_notify(24.136(26) epoch 194476) v5 -- ?+0 0x56a9f80 con 0x4eb7d20 -6483> 2014-08-13 17:23:20.955692 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(21) epoch 194476) v5 -- ?+0 0x56a9dc0 con 0x4fd8f20 -6482> 2014-08-13 17:23:20.955715 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(24.1f7(28) epoch 194476) v5 -- ?+0 0x56a9c00 con 0x4fdee00 -6481> 2014-08-13 17:23:20.955744 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(26),24.3b7(32) epoch 194476) v5 -- ?+0 0x54d6900 con 0x4ec4780 -6480> 2014-08-13 17:23:20.955736 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.744752 3 0.000128 -6479> 2014-08-13 17:23:20.955788 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.744875 0 0.000000 -6478> 2014-08-13 17:23:20.955807 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194470/194470/192978) [323,359,4] r=2 lpr=194471 pi=192978-194469/19 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset -6477> 2014-08-13 17:23:20.955962 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.000154 4 0.000143 -6476> 2014-08-13 17:23:20.956004 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(21) epoch 194476) v5 -- ?+0 0x54d5940 con 0x4eb6720 -6475> 2014-08-13 17:23:20.956011 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started -6474> 2014-08-13 17:23:20.956041 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(25) epoch 194476) v5 -- ?+0 0x54d3800 con 0x4fd86e0 -6473> 2014-08-13 17:23:20.956042 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start -6472> 2014-08-13 17:23:20.956057 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6471> 2014-08-13 17:23:20.956068 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(21) epoch 194476) v5 -- ?+0 0x54d1500 con 0x4fd32e0 -6470> 2014-08-13 17:23:20.956075 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 -6469> 2014-08-13 17:23:20.956093 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194471/194471 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray -6468> 2014-08-13 17:23:20.956175 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.741608 3 0.000109 -6467> 2014-08-13 17:23:20.956219 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.741693 0 0.000000 -6466> 2014-08-13 17:23:20.956224 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6465> 2014-08-13 17:23:20.955652 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.893901, event: throttled, request: pg_query(22.1fb epoch 194476) v3 -6464> 2014-08-13 17:23:20.956243 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194470/194470/192974) [376,4,303] r=1 lpr=194471 pi=192975-194469/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -6463> 2014-08-13 17:23:20.956250 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.745008 2 0.000129 -6462> 2014-08-13 17:23:20.956265 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.893975, event: all_read, request: pg_query(22.1fb epoch 194476) v3 -6461> 2014-08-13 17:23:20.956285 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.955635, event: dispatched, request: pg_query(22.1fb epoch 194476) v3 -6460> 2014-08-13 17:23:20.956278 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.745049 0 0.000000 -6459> 2014-08-13 17:23:20.956297 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.956296, event: waiting_for_osdmap, request: pg_query(22.1fb epoch 194476) v3 -6458> 2014-08-13 17:23:20.956304 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.745087 0 0.000000 -6457> 2014-08-13 17:23:20.956310 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.956309, event: started, request: pg_query(22.1fb epoch 194476) v3 -6456> 2014-08-13 17:23:20.956328 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started 0.745161 0 0.000000 -6455> 2014-08-13 17:23:20.956349 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194470/194470/194470) [4,44,362] r=0 lpr=194471 pi=193306-194469/23 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset -6454> 2014-08-13 17:23:20.956372 7fcd0964c700 5 -- op tracker -- , seq: 79, time: 2014-08-13 17:23:20.956372, event: done, request: pg_query(22.1fb epoch 194476) v3 -6453> 2014-08-13 17:23:20.956432 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 4 ==== pg_notify(22.1d2(24) epoch 194476) v5 ==== 2062+0+0 (2408723064 0 0) 0x54d3b80 con 0x5305ee0 -6452> 2014-08-13 17:23:20.956442 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] exit Reset 0.000199 4 0.000168 -6451> 2014-08-13 17:23:20.956460 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.919230, event: header_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -6450> 2014-08-13 17:23:20.956482 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.919233, event: throttled, request: pg_notify(22.1d2(24) epoch 194476) v5 -6449> 2014-08-13 17:23:20.956477 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] enter Started -6448> 2014-08-13 17:23:20.956493 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.921230, event: all_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -6447> 2014-08-13 17:23:20.956562 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.956456, event: dispatched, request: pg_notify(22.1d2(24) epoch 194476) v5 -6446> 2014-08-13 17:23:20.956554 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] exit Reset 0.000205 4 0.000227 -6445> 2014-08-13 17:23:20.956495 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] enter Start -6444> 2014-08-13 17:23:20.956581 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.956581, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194476) v5 -6443> 2014-08-13 17:23:20.956580 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] enter Started -6442> 2014-08-13 17:23:20.956594 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6441> 2014-08-13 17:23:20.956609 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] enter Start -6440> 2014-08-13 17:23:20.956592 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.956592, event: started, request: pg_notify(22.1d2(24) epoch 194476) v5 -6439> 2014-08-13 17:23:20.956639 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6438> 2014-08-13 17:23:20.956638 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] exit Start 0.000143 0 0.000000 -6437> 2014-08-13 17:23:20.956662 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194471/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] enter Started/Stray -6436> 2014-08-13 17:23:20.956665 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] exit Start 0.000055 0 0.000000 -6435> 2014-08-13 17:23:20.956683 7fcd0964c700 5 -- op tracker -- , seq: 80, time: 2014-08-13 17:23:20.956682, event: done, request: pg_notify(22.1d2(24) epoch 194476) v5 -6434> 2014-08-13 17:23:20.956686 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194466/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] enter Started/Stray -6433> 2014-08-13 17:23:20.956750 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 3 ==== pg_notify(24.9b(29) epoch 194476) v5 ==== 2383+0+0 (3603669103 0 0) 0x54d1180 con 0x53056a0 -6432> 2014-08-13 17:23:20.956784 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.916256, event: header_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6431> 2014-08-13 17:23:20.956796 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.916258, event: throttled, request: pg_notify(24.9b(29) epoch 194476) v5 -6430> 2014-08-13 17:23:20.956807 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.916729, event: all_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6429> 2014-08-13 17:23:20.956816 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.956780, event: dispatched, request: pg_notify(24.9b(29) epoch 194476) v5 -6428> 2014-08-13 17:23:20.956826 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.956826, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194476) v5 -6427> 2014-08-13 17:23:20.956839 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.956839, event: started, request: pg_notify(24.9b(29) epoch 194476) v5 -6426> 2014-08-13 17:23:20.956838 7fcd02841700 1 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6425> 2014-08-13 17:23:20.956849 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6424> 2014-08-13 17:23:20.956869 7fcd0964c700 5 -- op tracker -- , seq: 81, time: 2014-08-13 17:23:20.956869, event: done, request: pg_notify(24.9b(29) epoch 194476) v5 -6423> 2014-08-13 17:23:20.956872 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.742107 2 0.000104 -6422> 2014-08-13 17:23:20.956886 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.745443 2 0.000116 -6421> 2014-08-13 17:23:20.956895 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.742140 0 0.000000 -6420> 2014-08-13 17:23:20.956908 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 2 ==== pg_notify(24.9b(4) epoch 194476) v5 ==== 980+0+0 (2128401015 0 0) 0x54d5080 con 0x53053e0 -6419> 2014-08-13 17:23:20.956908 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.745476 0 0.000000 -6418> 2014-08-13 17:23:20.956928 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.946671, event: header_read, request: pg_notify(24.9b(4) epoch 194476) v5 -6417> 2014-08-13 17:23:20.956930 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.745508 0 0.000000 -6416> 2014-08-13 17:23:20.956957 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started 0.745583 0 0.000000 -6415> 2014-08-13 17:23:20.956921 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.742176 0 0.000000 -6414> 2014-08-13 17:23:20.956940 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.946673, event: throttled, request: pg_notify(24.9b(4) epoch 194476) v5 -6413> 2014-08-13 17:23:20.956985 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194470/194470/194470) [4,287,309] r=0 lpr=194471 pi=193306-194469/40 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset -6412> 2014-08-13 17:23:20.956987 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started 0.742284 0 0.000000 -6411> 2014-08-13 17:23:20.956993 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.946746, event: all_read, request: pg_notify(24.9b(4) epoch 194476) v5 -6410> 2014-08-13 17:23:20.957105 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.956925, event: dispatched, request: pg_notify(24.9b(4) epoch 194476) v5 -6409> 2014-08-13 17:23:20.957015 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194470/194470/194470) [4,338,376] r=0 lpr=194471 pi=193306-194469/41 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset -6408> 2014-08-13 17:23:20.957134 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.957134, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194476) v5 -6407> 2014-08-13 17:23:20.957131 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000145 4 0.000233 -6406> 2014-08-13 17:23:20.957162 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.957161, event: started, request: pg_notify(24.9b(4) epoch 194476) v5 -6405> 2014-08-13 17:23:20.957153 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started -6404> 2014-08-13 17:23:20.957199 7fcd0964c700 5 -- op tracker -- , seq: 82, time: 2014-08-13 17:23:20.957199, event: done, request: pg_notify(24.9b(4) epoch 194476) v5 -6403> 2014-08-13 17:23:20.957187 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start -6402> 2014-08-13 17:23:20.957215 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6401> 2014-08-13 17:23:20.957238 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000050 0 0.000000 -6400> 2014-08-13 17:23:20.957261 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6399> 2014-08-13 17:23:20.957265 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 3 ==== pg_notify(24.9b(26) epoch 194476) v5 ==== 2206+0+0 (2058123629 0 0) 0x4e50000 con 0x5305120 -6398> 2014-08-13 17:23:20.957281 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6397> 2014-08-13 17:23:20.957296 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.914361, event: header_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6396> 2014-08-13 17:23:20.957313 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.914363, event: throttled, request: pg_notify(24.9b(26) epoch 194476) v5 -6395> 2014-08-13 17:23:20.957301 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194466/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6394> 2014-08-13 17:23:20.957326 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.914692, event: all_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6393> 2014-08-13 17:23:20.957333 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.957292, event: dispatched, request: pg_notify(24.9b(26) epoch 194476) v5 -6392> 2014-08-13 17:23:20.957344 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.957343, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194476) v5 -6391> 2014-08-13 17:23:20.957355 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.957355, event: started, request: pg_notify(24.9b(26) epoch 194476) v5 -6390> 2014-08-13 17:23:20.957384 7fcd0964c700 5 -- op tracker -- , seq: 83, time: 2014-08-13 17:23:20.957384, event: done, request: pg_notify(24.9b(26) epoch 194476) v5 -6389> 2014-08-13 17:23:20.957394 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] exit Reset 0.000378 4 0.000419 -6388> 2014-08-13 17:23:20.957410 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 2 ==== pg_notify(22.f3(3) epoch 194476) v5 ==== 887+0+0 (345973583 0 0) 0x54d4b40 con 0x5307a60 -6387> 2014-08-13 17:23:20.957440 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.946416, event: header_read, request: pg_notify(22.f3(3) epoch 194476) v5 -6386> 2014-08-13 17:23:20.957435 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] enter Started -6385> 2014-08-13 17:23:20.957479 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.946418, event: throttled, request: pg_notify(22.f3(3) epoch 194476) v5 -6384> 2014-08-13 17:23:20.957487 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.946500, event: all_read, request: pg_notify(22.f3(3) epoch 194476) v5 -6383> 2014-08-13 17:23:20.957497 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.957437, event: dispatched, request: pg_notify(22.f3(3) epoch 194476) v5 -6382> 2014-08-13 17:23:20.957486 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] enter Start -6381> 2014-08-13 17:23:20.957512 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.957512, event: waiting_for_osdmap, request: pg_notify(22.f3(3) epoch 194476) v5 -6380> 2014-08-13 17:23:20.957493 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6379> 2014-08-13 17:23:20.957522 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.957522, event: started, request: pg_notify(22.f3(3) epoch 194476) v5 -6378> 2014-08-13 17:23:20.957519 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6377> 2014-08-13 17:23:20.957524 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.745878 2 0.000118 -6376> 2014-08-13 17:23:20.957543 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] exit Start 0.000056 0 0.000000 -6375> 2014-08-13 17:23:20.957545 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.745910 0 0.000000 -6374> 2014-08-13 17:23:20.957571 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194465/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] enter Started/Stray -6373> 2014-08-13 17:23:20.957586 7fcd0964c700 5 -- op tracker -- , seq: 84, time: 2014-08-13 17:23:20.957586, event: done, request: pg_notify(22.f3(3) epoch 194476) v5 -6372> 2014-08-13 17:23:20.957578 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.745954 0 0.000000 -6371> 2014-08-13 17:23:20.957602 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started 0.746030 0 0.000000 -6370> 2014-08-13 17:23:20.957608 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.311 192.168.1.101:6816/24270 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550a1c0 con 0x53077a0 -6369> 2014-08-13 17:23:20.957625 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194470/194470/194470) [4,289,341] r=0 lpr=194471 pi=192978-194469/21 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset -6368> 2014-08-13 17:23:20.957631 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6367> 2014-08-13 17:23:20.957674 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 3 ==== pg_notify(22.113(23) epoch 194476) v5 ==== 2003+0+0 (3841010039 0 0) 0x54d7e00 con 0x5305c20 -6366> 2014-08-13 17:23:20.957695 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.947552, event: header_read, request: pg_notify(22.113(23) epoch 194476) v5 -6365> 2014-08-13 17:23:20.957707 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.947555, event: throttled, request: pg_notify(22.113(23) epoch 194476) v5 -6364> 2014-08-13 17:23:20.957729 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.947660, event: all_read, request: pg_notify(22.113(23) epoch 194476) v5 -6363> 2014-08-13 17:23:20.957741 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.957692, event: dispatched, request: pg_notify(22.113(23) epoch 194476) v5 -6362> 2014-08-13 17:23:20.957751 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.957751, event: waiting_for_osdmap, request: pg_notify(22.113(23) epoch 194476) v5 -6361> 2014-08-13 17:23:20.957763 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.957763, event: started, request: pg_notify(22.113(23) epoch 194476) v5 -6360> 2014-08-13 17:23:20.957754 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6814/25893 -- pg_notify(22.3d9(44) epoch 194476) v5 -- ?+0 0x5184600 con 0x4f832e0 -6359> 2014-08-13 17:23:20.957794 7fcd0964c700 5 -- op tracker -- , seq: 85, time: 2014-08-13 17:23:20.957793, event: done, request: pg_notify(22.113(23) epoch 194476) v5 -6358> 2014-08-13 17:23:20.957790 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(25) epoch 194476) v5 -- ?+0 0x56ab100 con 0x4fd5960 -6357> 2014-08-13 17:23:20.957828 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] exit Reset 0.000204 4 0.000241 -6356> 2014-08-13 17:23:20.957859 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] enter Started -6355> 2014-08-13 17:23:20.957880 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] enter Start -6354> 2014-08-13 17:23:20.957899 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6353> 2014-08-13 17:23:20.957920 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] exit Start 0.000040 0 0.000000 -6352> 2014-08-13 17:23:20.957942 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194466/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] enter Started/Stray -6351> 2014-08-13 17:23:20.958064 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Reset 0.832776 6 0.000102 -6350> 2014-08-13 17:23:20.958080 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 2 ==== pg_notify(22.f3(20) epoch 194476) v5 ==== 1830+0+0 (514222415 0 0) 0x54d55c0 con 0x552a260 -6349> 2014-08-13 17:23:20.958084 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.746181 3 0.000136 -6348> 2014-08-13 17:23:20.958101 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started -6347> 2014-08-13 17:23:20.958108 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started 0.746254 0 0.000000 -6346> 2014-08-13 17:23:20.958125 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Start -6345> 2014-08-13 17:23:20.958136 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194470/194470/192972) [292,345,4] r=2 lpr=194471 pi=192975-194469/19 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset -6344> 2014-08-13 17:23:20.958146 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: transitioning to Stray -6343> 2014-08-13 17:23:20.958108 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.946889, event: header_read, request: pg_notify(22.f3(20) epoch 194476) v5 -6342> 2014-08-13 17:23:20.958171 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Start 0.000047 0 0.000000 -6341> 2014-08-13 17:23:20.958198 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.946892, event: throttled, request: pg_notify(22.f3(20) epoch 194476) v5 -6340> 2014-08-13 17:23:20.958201 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194472/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/Stray -6339> 2014-08-13 17:23:20.958213 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.947002, event: all_read, request: pg_notify(22.f3(20) epoch 194476) v5 -6338> 2014-08-13 17:23:20.958221 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.958104, event: dispatched, request: pg_notify(22.f3(20) epoch 194476) v5 -6337> 2014-08-13 17:23:20.958228 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.958228, event: waiting_for_osdmap, request: pg_notify(22.f3(20) epoch 194476) v5 -6336> 2014-08-13 17:23:20.958243 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.958242, event: started, request: pg_notify(22.f3(20) epoch 194476) v5 -6335> 2014-08-13 17:23:20.958282 7fcd0964c700 5 -- op tracker -- , seq: 86, time: 2014-08-13 17:23:20.958282, event: done, request: pg_notify(22.f3(20) epoch 194476) v5 -6334> 2014-08-13 17:23:20.958299 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.000163 4 0.000126 -6333> 2014-08-13 17:23:20.958326 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started -6332> 2014-08-13 17:23:20.958331 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 3 ==== pg_notify(22.113(3) epoch 194476) v5 ==== 887+0+0 (2650056438 0 0) 0x56a8540 con 0x5305960 -6331> 2014-08-13 17:23:20.958352 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.948115, event: header_read, request: pg_notify(22.113(3) epoch 194476) v5 -6330> 2014-08-13 17:23:20.958346 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start -6329> 2014-08-13 17:23:20.958359 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.948118, event: throttled, request: pg_notify(22.113(3) epoch 194476) v5 -6328> 2014-08-13 17:23:20.958366 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.948228, event: all_read, request: pg_notify(22.113(3) epoch 194476) v5 -6327> 2014-08-13 17:23:20.958361 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6326> 2014-08-13 17:23:20.958372 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.958349, event: dispatched, request: pg_notify(22.113(3) epoch 194476) v5 -6325> 2014-08-13 17:23:20.958371 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.744509 3 0.000129 -6324> 2014-08-13 17:23:20.958381 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 -6323> 2014-08-13 17:23:20.958408 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.744597 0 0.000000 -6322> 2014-08-13 17:23:20.958413 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194471/194471 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray -6321> 2014-08-13 17:23:20.958431 7fcd02841700 5 osd.4 pg_epoch: 194473 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194470/194470/192949) [328,4,20] r=1 lpr=194471 pi=192975-194469/20 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset -6320> 2014-08-13 17:23:20.958380 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.958380, event: waiting_for_osdmap, request: pg_notify(22.113(3) epoch 194476) v5 -6319> 2014-08-13 17:23:20.958479 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.958479, event: started, request: pg_notify(22.113(3) epoch 194476) v5 -6318> 2014-08-13 17:23:20.958507 7fcd0964c700 5 -- op tracker -- , seq: 87, time: 2014-08-13 17:23:20.958507, event: done, request: pg_notify(22.113(3) epoch 194476) v5 -6317> 2014-08-13 17:23:20.958531 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.302 192.168.1.113:6810/15731 2 ==== pg_notify(22.22c(3) epoch 194476) v5 ==== 887+0+0 (3579568949 0 0) 0x54d4280 con 0x552b9c0 -6316> 2014-08-13 17:23:20.958575 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.922202, event: header_read, request: pg_notify(22.22c(3) epoch 194476) v5 -6315> 2014-08-13 17:23:20.958587 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.922203, event: throttled, request: pg_notify(22.22c(3) epoch 194476) v5 -6314> 2014-08-13 17:23:20.958596 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.922281, event: all_read, request: pg_notify(22.22c(3) epoch 194476) v5 -6313> 2014-08-13 17:23:20.958596 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.000164 4 0.000140 -6312> 2014-08-13 17:23:20.958608 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.958572, event: dispatched, request: pg_notify(22.22c(3) epoch 194476) v5 -6311> 2014-08-13 17:23:20.958625 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started -6310> 2014-08-13 17:23:20.958640 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.958639, event: waiting_for_osdmap, request: pg_notify(22.22c(3) epoch 194476) v5 -6309> 2014-08-13 17:23:20.958641 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start -6308> 2014-08-13 17:23:20.958665 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.958665, event: started, request: pg_notify(22.22c(3) epoch 194476) v5 -6307> 2014-08-13 17:23:20.958671 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6306> 2014-08-13 17:23:20.958697 7fcd0964c700 5 -- op tracker -- , seq: 88, time: 2014-08-13 17:23:20.958696, event: done, request: pg_notify(22.22c(3) epoch 194476) v5 -6305> 2014-08-13 17:23:20.958695 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000054 0 0.000000 -6304> 2014-08-13 17:23:20.958720 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194471/194471 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray -6303> 2014-08-13 17:23:20.958733 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 2 ==== pg_notify(22.22c(22) epoch 194476) v5 ==== 1944+0+0 (917915763 0 0) 0x4e50a80 con 0x552b700 -6302> 2014-08-13 17:23:20.958756 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.945325, event: header_read, request: pg_notify(22.22c(22) epoch 194476) v5 -6301> 2014-08-13 17:23:20.958767 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.945328, event: throttled, request: pg_notify(22.22c(22) epoch 194476) v5 -6300> 2014-08-13 17:23:20.958755 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.871473 6 0.000097 -6299> 2014-08-13 17:23:20.958776 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.945460, event: all_read, request: pg_notify(22.22c(22) epoch 194476) v5 -6298> 2014-08-13 17:23:20.958799 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.958753, event: dispatched, request: pg_notify(22.22c(22) epoch 194476) v5 -6297> 2014-08-13 17:23:20.958793 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started -6296> 2014-08-13 17:23:20.958813 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.958813, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194476) v5 -6295> 2014-08-13 17:23:20.958814 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start -6294> 2014-08-13 17:23:20.958825 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.958825, event: started, request: pg_notify(22.22c(22) epoch 194476) v5 -6293> 2014-08-13 17:23:20.958838 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray -6292> 2014-08-13 17:23:20.958863 7fcd0964c700 5 -- op tracker -- , seq: 89, time: 2014-08-13 17:23:20.958862, event: done, request: pg_notify(22.22c(22) epoch 194476) v5 -6291> 2014-08-13 17:23:20.958861 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000046 0 0.000000 -6290> 2014-08-13 17:23:20.958883 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194472/194457 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray -6289> 2014-08-13 17:23:20.958886 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 2 ==== pg_notify(22.1d2(23) epoch 194476) v5 ==== 2003+0+0 (1106722567 0 0) 0x54d6ac0 con 0x552fa60 -6288> 2014-08-13 17:23:20.959020 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.947304, event: header_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -6287> 2014-08-13 17:23:20.959036 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.947306, event: throttled, request: pg_notify(22.1d2(23) epoch 194476) v5 -6286> 2014-08-13 17:23:20.959045 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.947403, event: all_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -6285> 2014-08-13 17:23:20.959029 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.796980 6 0.000094 -6284> 2014-08-13 17:23:20.959061 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started -6283> 2014-08-13 17:23:20.959056 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.959014, event: dispatched, request: pg_notify(22.1d2(23) epoch 194476) v5 -6282> 2014-08-13 17:23:20.959089 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.959089, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194476) v5 -6281> 2014-08-13 17:23:20.959083 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start -6280> 2014-08-13 17:23:20.959113 7fcd02841700 1 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray -6279> 2014-08-13 17:23:20.959108 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.959108, event: started, request: pg_notify(22.1d2(23) epoch 194476) v5 -6278> 2014-08-13 17:23:20.959140 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000058 0 0.000000 -6277> 2014-08-13 17:23:20.959163 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194466/194457 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray -6276> 2014-08-13 17:23:20.959176 7fcd0964c700 5 -- op tracker -- , seq: 90, time: 2014-08-13 17:23:20.959175, event: done, request: pg_notify(22.1d2(23) epoch 194476) v5 -6275> 2014-08-13 17:23:20.959203 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 2 ==== pg_notify(24.3d(27) epoch 194476) v5 ==== 2265+0+0 (1442779177 0 0) 0x54d6c80 con 0x552f7a0 -6274> 2014-08-13 17:23:20.959227 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.947359, event: header_read, request: pg_notify(24.3d(27) epoch 194476) v5 -6273> 2014-08-13 17:23:20.959237 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.947361, event: throttled, request: pg_notify(24.3d(27) epoch 194476) v5 -6272> 2014-08-13 17:23:20.959246 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.947480, event: all_read, request: pg_notify(24.3d(27) epoch 194476) v5 -6271> 2014-08-13 17:23:20.959230 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit Reset 0.914601 6 0.000122 -6270> 2014-08-13 17:23:20.959258 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.959224, event: dispatched, request: pg_notify(24.3d(27) epoch 194476) v5 -6269> 2014-08-13 17:23:20.959269 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.959269, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194476) v5 -6268> 2014-08-13 17:23:20.959264 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter Started -6267> 2014-08-13 17:23:20.959281 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.959281, event: started, request: pg_notify(24.3d(27) epoch 194476) v5 -6266> 2014-08-13 17:23:20.959282 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter Start -6265> 2014-08-13 17:23:20.959309 7fcd0964c700 5 -- op tracker -- , seq: 91, time: 2014-08-13 17:23:20.959309, event: done, request: pg_notify(24.3d(27) epoch 194476) v5 -6264> 2014-08-13 17:23:20.959308 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: transitioning to Stray -6263> 2014-08-13 17:23:20.959332 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit Start 0.000049 0 0.000000 -6262> 2014-08-13 17:23:20.959355 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194465/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter Started/Stray -6261> 2014-08-13 17:23:20.959356 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 2 ==== pg_query(24.e2 epoch 194476) v3 ==== 144+0+0 (2202314517 0 0) 0x54f8000 con 0x5520b00 -6260> 2014-08-13 17:23:20.959424 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.946469, event: header_read, request: pg_query(24.e2 epoch 194476) v3 -6259> 2014-08-13 17:23:20.959435 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.946471, event: throttled, request: pg_query(24.e2 epoch 194476) v3 -6258> 2014-08-13 17:23:20.959441 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.946531, event: all_read, request: pg_query(24.e2 epoch 194476) v3 -6257> 2014-08-13 17:23:20.959451 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.959403, event: dispatched, request: pg_query(24.e2 epoch 194476) v3 -6256> 2014-08-13 17:23:20.959461 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.959461, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194476) v3 -6255> 2014-08-13 17:23:20.959497 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.959497, event: started, request: pg_query(24.e2 epoch 194476) v3 -6254> 2014-08-13 17:23:20.959501 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.746514 3 0.000178 -6253> 2014-08-13 17:23:20.959526 7fcd0964c700 5 -- op tracker -- , seq: 92, time: 2014-08-13 17:23:20.959526, event: done, request: pg_query(24.e2 epoch 194476) v3 -6252> 2014-08-13 17:23:20.959528 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.746658 0 0.000000 -6251> 2014-08-13 17:23:20.959543 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 2 ==== pg_notify(22.481(39) epoch 194476) v5 ==== 2911+0+0 (3107951277 0 0) 0x56a9500 con 0x552d540 -6250> 2014-08-13 17:23:20.959547 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194470/194470/193205) [396,368,4]/[396,363] r=-1 lpr=194471 pi=194462-194469/3 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset -6249> 2014-08-13 17:23:20.959563 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.948325, event: header_read, request: pg_notify(22.481(39) epoch 194476) v5 -6248> 2014-08-13 17:23:20.959594 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.948328, event: throttled, request: pg_notify(22.481(39) epoch 194476) v5 -6247> 2014-08-13 17:23:20.959605 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.948456, event: all_read, request: pg_notify(22.481(39) epoch 194476) v5 -6246> 2014-08-13 17:23:20.959614 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.959560, event: dispatched, request: pg_notify(22.481(39) epoch 194476) v5 -6245> 2014-08-13 17:23:20.959624 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.959624, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194476) v5 -6244> 2014-08-13 17:23:20.959668 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.959667, event: started, request: pg_notify(22.481(39) epoch 194476) v5 -6243> 2014-08-13 17:23:20.959695 7fcd0964c700 5 -- op tracker -- , seq: 93, time: 2014-08-13 17:23:20.959694, event: done, request: pg_notify(22.481(39) epoch 194476) v5 -6242> 2014-08-13 17:23:20.959713 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.000166 4 0.000122 -6241> 2014-08-13 17:23:20.959727 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.352 192.168.1.108:6826/4273 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550e0c0 con 0x56b1fa0 -6240> 2014-08-13 17:23:20.959748 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6239> 2014-08-13 17:23:20.959745 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started -6238> 2014-08-13 17:23:20.959762 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(26) epoch 194476) v5 -- ?+0 0x56aabc0 con 0x4ec09a0 -6237> 2014-08-13 17:23:20.959766 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start -6236> 2014-08-13 17:23:20.959769 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 2 ==== pg_notify(22.f3(21) epoch 194476) v5 ==== 1889+0+0 (241229983 0 0) 0x54d32c0 con 0x4e314a0 -6235> 2014-08-13 17:23:20.959803 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.945621, event: header_read, request: pg_notify(22.f3(21) epoch 194476) v5 -6234> 2014-08-13 17:23:20.959799 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_notify(22.619(26) epoch 194476) v5 -- ?+0 0x56aaa00 con 0x4fd9760 -6233> 2014-08-13 17:23:20.959797 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray -6232> 2014-08-13 17:23:20.959820 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.945624, event: throttled, request: pg_notify(22.f3(21) epoch 194476) v5 -6231> 2014-08-13 17:23:20.959833 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(25) epoch 194476) v5 -- ?+0 0x56aa840 con 0x4fde300 -6230> 2014-08-13 17:23:20.959839 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.945801, event: all_read, request: pg_notify(22.f3(21) epoch 194476) v5 -6229> 2014-08-13 17:23:20.959847 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.959800, event: dispatched, request: pg_notify(22.f3(21) epoch 194476) v5 -6228> 2014-08-13 17:23:20.959838 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000073 0 0.000000 -6227> 2014-08-13 17:23:20.959852 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(22) epoch 194476) v5 -- ?+0 0x56aa680 con 0x4eb23c0 -6226> 2014-08-13 17:23:20.959862 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194471/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray -6225> 2014-08-13 17:23:20.959871 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(25) epoch 194476) v5 -- ?+0 0x56aa4c0 con 0x4fdee00 -6224> 2014-08-13 17:23:20.959854 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.959854, event: waiting_for_osdmap, request: pg_notify(22.f3(21) epoch 194476) v5 -6223> 2014-08-13 17:23:20.959888 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.959887, event: started, request: pg_notify(22.f3(21) epoch 194476) v5 -6222> 2014-08-13 17:23:20.959911 7fcd0964c700 5 -- op tracker -- , seq: 94, time: 2014-08-13 17:23:20.959911, event: done, request: pg_notify(22.f3(21) epoch 194476) v5 -6221> 2014-08-13 17:23:20.959936 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 4 ==== pg_query(22.362 epoch 194476) v3 ==== 144+0+0 (2430676563 0 0) 0x2f914a0 con 0x4e335a0 -6220> 2014-08-13 17:23:20.959959 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.875998, event: header_read, request: pg_query(22.362 epoch 194476) v3 -6219> 2014-08-13 17:23:20.959969 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.876000, event: throttled, request: pg_query(22.362 epoch 194476) v3 -6218> 2014-08-13 17:23:20.959977 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.876061, event: all_read, request: pg_query(22.362 epoch 194476) v3 -6217> 2014-08-13 17:23:20.959993 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.959956, event: dispatched, request: pg_query(22.362 epoch 194476) v3 -6216> 2014-08-13 17:23:20.960003 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.960003, event: waiting_for_osdmap, request: pg_query(22.362 epoch 194476) v3 -6215> 2014-08-13 17:23:20.960014 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.960014, event: started, request: pg_query(22.362 epoch 194476) v3 -6214> 2014-08-13 17:23:20.960053 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(29) epoch 194476) v5 -- ?+0 0x56aa300 con 0x4f83860 -6213> 2014-08-13 17:23:20.960086 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(42) epoch 194476) v5 -- ?+0 0x54d4b40 con 0x4eb18c0 -6212> 2014-08-13 17:23:20.960075 7fcd0964c700 5 -- op tracker -- , seq: 95, time: 2014-08-13 17:23:20.960075, event: done, request: pg_query(22.362 epoch 194476) v3 -6211> 2014-08-13 17:23:20.960093 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6210> 2014-08-13 17:23:20.960118 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.746802 2 0.000118 -6209> 2014-08-13 17:23:20.960142 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.746837 0 0.000000 -6208> 2014-08-13 17:23:20.960164 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.746870 0 0.000000 -6207> 2014-08-13 17:23:20.960128 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 2 ==== pg_query(22.28f epoch 194476) v3 ==== 144+0+0 (3400619831 0 0) 0x2f95fa0 con 0x4e34780 -6206> 2014-08-13 17:23:20.960182 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started 0.746938 0 0.000000 -6205> 2014-08-13 17:23:20.960192 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.947773, event: header_read, request: pg_query(22.28f epoch 194476) v3 -6204> 2014-08-13 17:23:20.960220 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.947774, event: throttled, request: pg_query(22.28f epoch 194476) v3 -6203> 2014-08-13 17:23:20.960242 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.947821, event: all_read, request: pg_query(22.28f epoch 194476) v3 -6202> 2014-08-13 17:23:20.960218 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194470/194470/194470) [4,343,286] r=0 lpr=194471 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset -6201> 2014-08-13 17:23:20.960255 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.960189, event: dispatched, request: pg_query(22.28f epoch 194476) v3 -6200> 2014-08-13 17:23:20.960271 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.960271, event: waiting_for_osdmap, request: pg_query(22.28f epoch 194476) v3 -6199> 2014-08-13 17:23:20.960283 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.960283, event: started, request: pg_query(22.28f epoch 194476) v3 -6198> 2014-08-13 17:23:20.960276 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(25) epoch 194476) v5 -- ?+0 0x54d5080 con 0x4fd5960 -6197> 2014-08-13 17:23:20.960344 7fcd0964c700 5 -- op tracker -- , seq: 96, time: 2014-08-13 17:23:20.960344, event: done, request: pg_query(22.28f epoch 194476) v3 -6196> 2014-08-13 17:23:20.960374 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x550cec0 con 0x4e35540 -6195> 2014-08-13 17:23:20.960399 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -6194> 2014-08-13 17:23:20.960399 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000181 4 0.000245 -6193> 2014-08-13 17:23:20.960415 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 4 ==== pg_notify(22.113(23) epoch 194476) v5 ==== 2003+0+0 (3933187701 0 0) 0x54d63c0 con 0x4ec09a0 -6192> 2014-08-13 17:23:20.960430 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started -6191> 2014-08-13 17:23:20.960456 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.947160, event: header_read, request: pg_notify(22.113(23) epoch 194476) v5 -6190> 2014-08-13 17:23:20.960468 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.947162, event: throttled, request: pg_notify(22.113(23) epoch 194476) v5 -6189> 2014-08-13 17:23:20.960459 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start -6188> 2014-08-13 17:23:20.960481 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.947291, event: all_read, request: pg_notify(22.113(23) epoch 194476) v5 -6187> 2014-08-13 17:23:20.960491 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6186> 2014-08-13 17:23:20.960491 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.960453, event: dispatched, request: pg_notify(22.113(23) epoch 194476) v5 -6185> 2014-08-13 17:23:20.960526 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.960526, event: waiting_for_osdmap, request: pg_notify(22.113(23) epoch 194476) v5 -6184> 2014-08-13 17:23:20.960520 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000061 0 0.000000 -6183> 2014-08-13 17:23:20.960551 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.960551, event: started, request: pg_notify(22.113(23) epoch 194476) v5 -6182> 2014-08-13 17:23:20.960551 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6181> 2014-08-13 17:23:20.960581 7fcd0964c700 5 -- op tracker -- , seq: 97, time: 2014-08-13 17:23:20.960580, event: done, request: pg_notify(22.113(23) epoch 194476) v5 -6180> 2014-08-13 17:23:20.960573 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6179> 2014-08-13 17:23:20.960604 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194471/194471 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6178> 2014-08-13 17:23:20.960630 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 2 ==== pg_query(24.33a epoch 194476) v3 ==== 144+0+0 (2893747789 0 0) 0x2f95dc0 con 0x4ec1ce0 -6177> 2014-08-13 17:23:20.960656 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.922541, event: header_read, request: pg_query(24.33a epoch 194476) v3 -6176> 2014-08-13 17:23:20.960666 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.922543, event: throttled, request: pg_query(24.33a epoch 194476) v3 -6175> 2014-08-13 17:23:20.960675 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.922610, event: all_read, request: pg_query(24.33a epoch 194476) v3 -6174> 2014-08-13 17:23:20.960685 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.960653, event: dispatched, request: pg_query(24.33a epoch 194476) v3 -6173> 2014-08-13 17:23:20.960694 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.960694, event: waiting_for_osdmap, request: pg_query(24.33a epoch 194476) v3 -6172> 2014-08-13 17:23:20.960734 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.960734, event: started, request: pg_query(24.33a epoch 194476) v3 -6171> 2014-08-13 17:23:20.960732 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.747057 3 0.000121 -6170> 2014-08-13 17:23:20.960760 7fcd0964c700 5 -- op tracker -- , seq: 98, time: 2014-08-13 17:23:20.960760, event: done, request: pg_query(24.33a epoch 194476) v3 -6169> 2014-08-13 17:23:20.960763 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.747137 0 0.000000 -6168> 2014-08-13 17:23:20.960786 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194470/194470/192971) [51,4,308] r=1 lpr=194471 pi=192975-194469/20 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset -6167> 2014-08-13 17:23:20.960784 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 2 ==== pg_notify(24.9b(29) epoch 194476) v5 ==== 2383+0+0 (2454009132 0 0) 0x54d5780 con 0x4ec27e0 -6166> 2014-08-13 17:23:20.960830 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.946941, event: header_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6165> 2014-08-13 17:23:20.960842 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.946943, event: throttled, request: pg_notify(24.9b(29) epoch 194476) v5 -6164> 2014-08-13 17:23:20.960852 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.947085, event: all_read, request: pg_notify(24.9b(29) epoch 194476) v5 -6163> 2014-08-13 17:23:20.960861 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.960825, event: dispatched, request: pg_notify(24.9b(29) epoch 194476) v5 -6162> 2014-08-13 17:23:20.960871 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.960870, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194476) v5 -6161> 2014-08-13 17:23:20.960927 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.960926, event: started, request: pg_notify(24.9b(29) epoch 194476) v5 -6160> 2014-08-13 17:23:20.960952 7fcd0964c700 5 -- op tracker -- , seq: 99, time: 2014-08-13 17:23:20.960952, event: done, request: pg_notify(24.9b(29) epoch 194476) v5 -6159> 2014-08-13 17:23:20.960947 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.000160 4 0.000132 -6158> 2014-08-13 17:23:20.960969 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -6157> 2014-08-13 17:23:20.960985 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -6156> 2014-08-13 17:23:20.960998 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6155> 2014-08-13 17:23:20.960999 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 3 ==== pg_query(24.5e epoch 194476) v3 ==== 144+0+0 (1930070268 0 0) 0x2f96ea0 con 0x4ec3440 -6154> 2014-08-13 17:23:20.961023 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.914906, event: header_read, request: pg_query(24.5e epoch 194476) v3 -6153> 2014-08-13 17:23:20.961015 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000 -6152> 2014-08-13 17:23:20.961047 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194471/194471 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -6151> 2014-08-13 17:23:20.961040 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.914908, event: throttled, request: pg_query(24.5e epoch 194476) v3 -6150> 2014-08-13 17:23:20.961077 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.914988, event: all_read, request: pg_query(24.5e epoch 194476) v3 -6149> 2014-08-13 17:23:20.961088 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.961020, event: dispatched, request: pg_query(24.5e epoch 194476) v3 -6148> 2014-08-13 17:23:20.961099 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.961098, event: waiting_for_osdmap, request: pg_query(24.5e epoch 194476) v3 -6147> 2014-08-13 17:23:20.961156 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_notify(22.14a(21) epoch 194476) v5 -- ?+0 0x54d3b80 con 0x4e335a0 -6146> 2014-08-13 17:23:20.961201 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6812/32413 -- pg_notify(22.707(26) epoch 194476) v5 -- ?+0 0x54d0e00 con 0x5303440 -6145> 2014-08-13 17:23:20.961195 7fcd01e40700 1 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset -6144> 2014-08-13 17:23:20.961240 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6836/6227 -- pg_notify(22.775(26) epoch 194476) v5 -- ?+0 0x54d3d40 con 0x53011e0 -6143> 2014-08-13 17:23:20.961278 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(26),24.3b7(32) epoch 194476) v5 -- ?+0 0x54d01c0 con 0x4ec4780 -6142> 2014-08-13 17:23:20.961299 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6830/52879 -- pg_notify(24.30b(29) epoch 194476) v5 -- ?+0 0x4e50000 con 0x4fd2520 -6141> 2014-08-13 17:23:20.961226 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.747262 2 0.000120 -6140> 2014-08-13 17:23:20.961404 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.747447 0 0.000000 -6139> 2014-08-13 17:23:20.961110 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.961110, event: started, request: pg_query(24.5e epoch 194476) v3 -6138> 2014-08-13 17:23:20.961425 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.747480 0 0.000000 -6137> 2014-08-13 17:23:20.961443 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started 0.747542 0 0.000000 -6136> 2014-08-13 17:23:20.961463 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194470/194470/194470) [4,340,347] r=0 lpr=194471 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset -6135> 2014-08-13 17:23:20.961547 7fcd0964c700 5 -- op tracker -- , seq: 100, time: 2014-08-13 17:23:20.961546, event: done, request: pg_query(24.5e epoch 194476) v3 -6134> 2014-08-13 17:23:20.961576 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 4 ==== pg_notify(22.4df(23) epoch 194476) v5 ==== 2003+0+0 (3530789080 0 0) 0x56a9880 con 0x4ec4780 -6133> 2014-08-13 17:23:20.961627 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.949263, event: header_read, request: pg_notify(22.4df(23) epoch 194476) v5 -6132> 2014-08-13 17:23:20.961619 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000156 4 0.000354 -6131> 2014-08-13 17:23:20.961643 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.949265, event: throttled, request: pg_notify(22.4df(23) epoch 194476) v5 -6130> 2014-08-13 17:23:20.961653 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.949389, event: all_read, request: pg_notify(22.4df(23) epoch 194476) v5 -6129> 2014-08-13 17:23:20.961665 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.961598, event: dispatched, request: pg_notify(22.4df(23) epoch 194476) v5 -6128> 2014-08-13 17:23:20.961647 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started -6127> 2014-08-13 17:23:20.961672 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.961672, event: waiting_for_osdmap, request: pg_notify(22.4df(23) epoch 194476) v5 -6126> 2014-08-13 17:23:20.961684 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start -6125> 2014-08-13 17:23:20.961698 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.961697, event: started, request: pg_notify(22.4df(23) epoch 194476) v5 -6124> 2014-08-13 17:23:20.961711 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -6123> 2014-08-13 17:23:20.961738 7fcd0964c700 5 -- op tracker -- , seq: 101, time: 2014-08-13 17:23:20.961738, event: done, request: pg_notify(22.4df(23) epoch 194476) v5 -6122> 2014-08-13 17:23:20.961733 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000049 0 0.000000 -6121> 2014-08-13 17:23:20.961755 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -6120> 2014-08-13 17:23:20.961769 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 4 ==== pg_query(24.2a1 epoch 194476) v3 ==== 144+0+0 (4212382656 0 0) 0x2f93840 con 0x4ec5ac0 -6119> 2014-08-13 17:23:20.961774 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -6118> 2014-08-13 17:23:20.961790 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.950860, event: header_read, request: pg_query(24.2a1 epoch 194476) v3 -6117> 2014-08-13 17:23:20.961790 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194471/194471 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -6116> 2014-08-13 17:23:20.961801 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.950862, event: throttled, request: pg_query(24.2a1 epoch 194476) v3 -6115> 2014-08-13 17:23:20.961822 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.950933, event: all_read, request: pg_query(24.2a1 epoch 194476) v3 -6114> 2014-08-13 17:23:20.961832 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.961788, event: dispatched, request: pg_query(24.2a1 epoch 194476) v3 -6113> 2014-08-13 17:23:20.961842 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.961842, event: waiting_for_osdmap, request: pg_query(24.2a1 epoch 194476) v3 -6112> 2014-08-13 17:23:20.961854 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.961854, event: started, request: pg_query(24.2a1 epoch 194476) v3 -6111> 2014-08-13 17:23:20.961876 7fcd0964c700 5 -- op tracker -- , seq: 102, time: 2014-08-13 17:23:20.961875, event: done, request: pg_query(24.2a1 epoch 194476) v3 -6110> 2014-08-13 17:23:20.961892 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 4 ==== pg_notify(22.6bd(1) epoch 194476) v5 ==== 769+0+0 (69560420 0 0) 0x54d0a80 con 0x4eb7d20 -6109> 2014-08-13 17:23:20.961912 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.954518, event: header_read, request: pg_notify(22.6bd(1) epoch 194476) v5 -6108> 2014-08-13 17:23:20.961904 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6826/52317 -- pg_log(24.1de epoch 194476 log log((194422'17074,194422'17074], crt=194422'17074) query_epoch 194476) v4 -- ?+0 0x56dbc00 con 0x4f83860 -6107> 2014-08-13 17:23:20.961922 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.954519, event: throttled, request: pg_notify(22.6bd(1) epoch 194476) v5 -6106> 2014-08-13 17:23:20.961924 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.747695 3 0.000113 -6105> 2014-08-13 17:23:20.961947 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.954642, event: all_read, request: pg_notify(22.6bd(1) epoch 194476) v5 -6104> 2014-08-13 17:23:20.961955 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.961909, event: dispatched, request: pg_notify(22.6bd(1) epoch 194476) v5 -6103> 2014-08-13 17:23:20.961948 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.747766 0 0.000000 -6102> 2014-08-13 17:23:20.961967 7fcd01e40700 5 osd.4 pg_epoch: 194473 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194470/194470/192981) [345,24,4] r=2 lpr=194471 pi=192984-194469/19 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset -6101> 2014-08-13 17:23:20.961962 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.961961, event: waiting_for_osdmap, request: pg_notify(22.6bd(1) epoch 194476) v5 -6100> 2014-08-13 17:23:20.961995 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.961995, event: started, request: pg_notify(22.6bd(1) epoch 194476) v5 -6099> 2014-08-13 17:23:20.962021 7fcd0964c700 5 -- op tracker -- , seq: 103, time: 2014-08-13 17:23:20.962021, event: done, request: pg_notify(22.6bd(1) epoch 194476) v5 -6098> 2014-08-13 17:23:20.962065 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 3 ==== pg_query(22.653 epoch 194476) v3 ==== 144+0+0 (2744949811 0 0) 0x54fa940 con 0x4fd86e0 -6097> 2014-08-13 17:23:20.962086 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.921173, event: header_read, request: pg_query(22.653 epoch 194476) v3 -6096> 2014-08-13 17:23:20.962094 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.921175, event: throttled, request: pg_query(22.653 epoch 194476) v3 -6095> 2014-08-13 17:23:20.962099 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.921228, event: all_read, request: pg_query(22.653 epoch 194476) v3 -6094> 2014-08-13 17:23:20.962104 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.962083, event: dispatched, request: pg_query(22.653 epoch 194476) v3 -6093> 2014-08-13 17:23:20.962110 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.962110, event: waiting_for_osdmap, request: pg_query(22.653 epoch 194476) v3 -6092> 2014-08-13 17:23:20.962135 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.000168 4 0.000115 -6091> 2014-08-13 17:23:20.962123 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.962123, event: started, request: pg_query(22.653 epoch 194476) v3 -6090> 2014-08-13 17:23:20.962167 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started -6089> 2014-08-13 17:23:20.962201 7fcd0964c700 5 -- op tracker -- , seq: 104, time: 2014-08-13 17:23:20.962201, event: done, request: pg_query(22.653 epoch 194476) v3 -6088> 2014-08-13 17:23:20.962186 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start -6087> 2014-08-13 17:23:20.962225 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -6086> 2014-08-13 17:23:20.962230 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 3 ==== pg_notify(22.1d2(3) epoch 194476) v5 ==== 887+0+0 (579020996 0 0) 0x54d6740 con 0x4fde880 -6085> 2014-08-13 17:23:20.962251 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000064 0 0.000000 -6084> 2014-08-13 17:23:20.962268 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.947213, event: header_read, request: pg_notify(22.1d2(3) epoch 194476) v5 -6083> 2014-08-13 17:23:20.962267 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(32) epoch 194476) v5 -- ?+0 0x4e501c0 con 0x4ec5ac0 -6082> 2014-08-13 17:23:20.962269 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194471/194471 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray -6081> 2014-08-13 17:23:20.962280 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.947215, event: throttled, request: pg_notify(22.1d2(3) epoch 194476) v5 -6080> 2014-08-13 17:23:20.962290 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.947341, event: all_read, request: pg_notify(22.1d2(3) epoch 194476) v5 -6079> 2014-08-13 17:23:20.962299 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.962265, event: dispatched, request: pg_notify(22.1d2(3) epoch 194476) v5 -6078> 2014-08-13 17:23:20.962296 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6818/61186 -- pg_notify(24.3bf(5) epoch 194476) v5 -- ?+0 0x5184980 con 0x4f84ba0 -6077> 2014-08-13 17:23:20.962313 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.962313, event: waiting_for_osdmap, request: pg_notify(22.1d2(3) epoch 194476) v5 -6076> 2014-08-13 17:23:20.962327 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- pg_notify(22.3e4(26) epoch 194476) v5 -- ?+0 0x56a8700 con 0x4ec5d80 -6075> 2014-08-13 17:23:20.962329 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.962329, event: started, request: pg_notify(22.1d2(3) epoch 194476) v5 -6074> 2014-08-13 17:23:20.962350 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.2b9(21) epoch 194476) v5 -- ?+0 0x56a88c0 con 0x4eb23c0 -6073> 2014-08-13 17:23:20.962370 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(21) epoch 194476) v5 -- ?+0 0x56a9180 con 0x4fde040 -6072> 2014-08-13 17:23:20.962373 7fcd0964c700 5 -- op tracker -- , seq: 105, time: 2014-08-13 17:23:20.962373, event: done, request: pg_notify(22.1d2(3) epoch 194476) v5 -6071> 2014-08-13 17:23:20.962392 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(21) epoch 194476) v5 -- ?+0 0x56a9500 con 0x4fd8f20 -6070> 2014-08-13 17:23:20.962410 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6814/25893 -- pg_notify(22.3d9(44) epoch 194476) v5 -- ?+0 0x56a8540 con 0x4f832e0 -6069> 2014-08-13 17:23:20.962426 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(24.1f7(28) epoch 194476) v5 -- ?+0 0x56a9340 con 0x4fdee00 -6068> 2014-08-13 17:23:20.962567 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.953335 6 0.000096 -6067> 2014-08-13 17:23:20.962589 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_notify(22.99(22) epoch 194476) v5 -- ?+0 0x54d7e00 con 0x4eb18c0 -6066> 2014-08-13 17:23:20.962609 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started -6065> 2014-08-13 17:23:20.962631 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start -6064> 2014-08-13 17:23:20.962649 7fcd01e40700 1 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray -6063> 2014-08-13 17:23:20.962405 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 3 ==== pg_notify(22.22c(24) epoch 194476) v5 ==== 2062+0+0 (1668933989 0 0) 0x54d7380 con 0x4fde5c0 -6062> 2014-08-13 17:23:20.962670 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000039 0 0.000000 -6061> 2014-08-13 17:23:20.962693 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194472/194457 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray -6060> 2014-08-13 17:23:20.962695 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.947396, event: header_read, request: pg_notify(22.22c(24) epoch 194476) v5 -6059> 2014-08-13 17:23:20.962733 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.947397, event: throttled, request: pg_notify(22.22c(24) epoch 194476) v5 -6058> 2014-08-13 17:23:20.962741 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.947499, event: all_read, request: pg_notify(22.22c(24) epoch 194476) v5 -6057> 2014-08-13 17:23:20.962747 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.962681, event: dispatched, request: pg_notify(22.22c(24) epoch 194476) v5 -6056> 2014-08-13 17:23:20.962757 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.962757, event: waiting_for_osdmap, request: pg_notify(22.22c(24) epoch 194476) v5 -6055> 2014-08-13 17:23:20.962773 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.962773, event: started, request: pg_notify(22.22c(24) epoch 194476) v5 -6054> 2014-08-13 17:23:20.962818 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(26) epoch 194476) v5 -- ?+0 0x56ac440 con 0x4fd0f20 -6053> 2014-08-13 17:23:20.962837 7fcd0964c700 5 -- op tracker -- , seq: 106, time: 2014-08-13 17:23:20.962836, event: done, request: pg_notify(22.22c(24) epoch 194476) v5 -6052> 2014-08-13 17:23:20.962849 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_notify(22.3af(26) epoch 194476) v5 -- ?+0 0x56ac280 con 0x4eb0c60 -6051> 2014-08-13 17:23:20.962863 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.24 192.168.1.107:6800/52669 3 ==== pg_notify(22.113(24) epoch 194476) v5 ==== 2062+0+0 (804026585 0 0) 0x56a81c0 con 0x4fd11e0 -6050> 2014-08-13 17:23:20.962893 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(22) epoch 194476) v5 -- ?+0 0x56ac0c0 con 0x4fda520 -6049> 2014-08-13 17:23:20.962934 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.947602, event: header_read, request: pg_notify(22.113(24) epoch 194476) v5 -6048> 2014-08-13 17:23:20.962944 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(26) epoch 194476) v5 -- ?+0 0x56abf00 con 0x4e314a0 -6047> 2014-08-13 17:23:20.962951 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.947603, event: throttled, request: pg_notify(22.113(24) epoch 194476) v5 -6046> 2014-08-13 17:23:20.962976 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_notify(22.48e(24) epoch 194476) v5 -- ?+0 0x56abd40 con 0x4ec3440 -6045> 2014-08-13 17:23:20.962998 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(21) epoch 194476) v5 -- ?+0 0x56abb80 con 0x4f870c0 -6044> 2014-08-13 17:23:20.963014 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(21) epoch 194476) v5 -- ?+0 0x56ab9c0 con 0x4f81fa0 -6043> 2014-08-13 17:23:20.963031 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6806/42024 -- pg_notify(24.1bb(26) epoch 194476) v5 -- ?+0 0x56ab800 con 0x4fdbde0 -6042> 2014-08-13 17:23:20.962957 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.947811, event: all_read, request: pg_notify(22.113(24) epoch 194476) v5 -6041> 2014-08-13 17:23:20.963197 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.962884, event: dispatched, request: pg_notify(22.113(24) epoch 194476) v5 -6040> 2014-08-13 17:23:20.963213 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.963213, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194476) v5 -6039> 2014-08-13 17:23:20.963215 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(26) epoch 194476) v5 -- ?+0 0x56ab640 con 0x4f86e00 -6038> 2014-08-13 17:23:20.963226 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.963226, event: started, request: pg_notify(22.113(24) epoch 194476) v5 -6037> 2014-08-13 17:23:20.963236 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(21) epoch 194476) v5 -- ?+0 0x56aaf40 con 0x4e32940 -6036> 2014-08-13 17:23:20.963259 7fcd0964c700 5 -- op tracker -- , seq: 107, time: 2014-08-13 17:23:20.963259, event: done, request: pg_notify(22.113(24) epoch 194476) v5 -6035> 2014-08-13 17:23:20.963266 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6814/41806 -- pg_notify(24.299(5) epoch 194476) v5 -- ?+0 0x56ad780 con 0x4fd3020 -6034> 2014-08-13 17:23:20.963291 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6820/19173 -- pg_query(22.3d5 epoch 194476) v3 -- ?+0 0x54fb2a0 con 0x4fdad60 -6033> 2014-08-13 17:23:20.963314 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6814/33435 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f97440 con 0x4f853e0 -6032> 2014-08-13 17:23:20.963335 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6802/30768 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f96180 con 0x4fd06e0 -6031> 2014-08-13 17:23:20.963339 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 5 ==== pg_notify(24.9b(26) epoch 194476) v5 ==== 2206+0+0 (2058123629 0 0) 0x5187380 con 0x4fd3860 -6030> 2014-08-13 17:23:20.963354 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6806/6812 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f95a00 con 0x552d540 -6029> 2014-08-13 17:23:20.963369 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.911626, event: header_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6028> 2014-08-13 17:23:20.963373 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6814/23703 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f96ae0 con 0x5303700 -6027> 2014-08-13 17:23:20.963389 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_query(22.1b8 epoch 194476) v3 -- ?+0 0x2f950a0 con 0x4e35540 -6026> 2014-08-13 17:23:20.963382 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.911628, event: throttled, request: pg_notify(24.9b(26) epoch 194476) v5 -6025> 2014-08-13 17:23:20.963406 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.911888, event: all_read, request: pg_notify(24.9b(26) epoch 194476) v5 -6024> 2014-08-13 17:23:20.963274 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.042349 5 0.000263 -6023> 2014-08-13 17:23:20.963415 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_query(22.3d5 epoch 194476) v3 -- ?+0 0x2f95fa0 con 0x4f86e00 -6022> 2014-08-13 17:23:20.963420 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.963365, event: dispatched, request: pg_notify(24.9b(26) epoch 194476) v5 -6021> 2014-08-13 17:23:20.963448 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -6020> 2014-08-13 17:23:20.963469 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.963469, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194476) v5 -6019> 2014-08-13 17:23:20.963487 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.963487, event: started, request: pg_notify(24.9b(26) epoch 194476) v5 -6018> 2014-08-13 17:23:20.963520 7fcd0964c700 5 -- op tracker -- , seq: 108, time: 2014-08-13 17:23:20.963520, event: done, request: pg_notify(24.9b(26) epoch 194476) v5 -6017> 2014-08-13 17:23:20.963449 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6800/4640 -- pg_query(22.1b8 epoch 194476) v3 -- ?+0 0x2f94b00 con 0x4fd1a20 -6016> 2014-08-13 17:23:20.963564 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 4 ==== pg_notify(24.3d(26) epoch 194476) v5 ==== 2206+0+0 (1292580142 0 0) 0x54d0c40 con 0x4fd06e0 -6015> 2014-08-13 17:23:20.963586 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.918889, event: header_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6014> 2014-08-13 17:23:20.963603 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.918891, event: throttled, request: pg_notify(24.3d(26) epoch 194476) v5 -6013> 2014-08-13 17:23:20.963588 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000140 0 0.000000 -6012> 2014-08-13 17:23:20.963609 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.921328, event: all_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6011> 2014-08-13 17:23:20.963615 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.963583, event: dispatched, request: pg_notify(24.3d(26) epoch 194476) v5 -6010> 2014-08-13 17:23:20.963622 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.963622, event: waiting_for_osdmap, request: pg_notify(24.3d(26) epoch 194476) v5 -6009> 2014-08-13 17:23:20.963630 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.963630, event: started, request: pg_notify(24.3d(26) epoch 194476) v5 -6008> 2014-08-13 17:23:20.963621 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -6007> 2014-08-13 17:23:20.963628 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6822/15390 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f910e0 con 0x578c620 -6006> 2014-08-13 17:23:20.963655 7fcd0964c700 5 -- op tracker -- , seq: 109, time: 2014-08-13 17:23:20.963655, event: done, request: pg_notify(24.3d(26) epoch 194476) v5 -6005> 2014-08-13 17:23:20.963717 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 5 ==== pg_notify(24.3d(26) epoch 194476) v5 ==== 2206+0+0 (3953936911 0 0) 0x54d3640 con 0x4fd32e0 -6004> 2014-08-13 17:23:20.963775 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.919651, event: header_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6003> 2014-08-13 17:23:20.963789 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.919652, event: throttled, request: pg_notify(24.3d(26) epoch 194476) v5 -6002> 2014-08-13 17:23:20.963798 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.920874, event: all_read, request: pg_notify(24.3d(26) epoch 194476) v5 -6001> 2014-08-13 17:23:20.963807 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.963771, event: dispatched, request: pg_notify(24.3d(26) epoch 194476) v5 -6000> 2014-08-13 17:23:20.963795 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.003191 3 0.000274 -5999> 2014-08-13 17:23:20.963817 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.963817, event: waiting_for_osdmap, request: pg_notify(24.3d(26) epoch 194476) v5 -5998> 2014-08-13 17:23:20.963811 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(21) epoch 194476) v5 -- ?+0 0x54d32c0 con 0x4e35120 -5997> 2014-08-13 17:23:20.963824 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5996> 2014-08-13 17:23:20.963838 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(32) epoch 194476) v5 -- ?+0 0x54d6c80 con 0x5305ee0 -5995> 2014-08-13 17:23:20.963856 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(22) epoch 194476) v5 -- ?+0 0x54d6ac0 con 0x4e32940 -5994> 2014-08-13 17:23:20.963877 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(21) epoch 194476) v5 -- ?+0 0x54d4b40 con 0x4fd3860 -5993> 2014-08-13 17:23:20.963829 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.963829, event: started, request: pg_notify(24.3d(26) epoch 194476) v5 -5992> 2014-08-13 17:23:20.963911 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_notify(22.39d(24) epoch 194476) v5 -- ?+0 0x56aec80 con 0x4fd32e0 -5991> 2014-08-13 17:23:20.963927 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000104 0 0.000000 -5990> 2014-08-13 17:23:20.963943 7fcd0964c700 5 -- op tracker -- , seq: 110, time: 2014-08-13 17:23:20.963943, event: done, request: pg_notify(24.3d(26) epoch 194476) v5 -5989> 2014-08-13 17:23:20.963950 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5988> 2014-08-13 17:23:20.963973 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 -5987> 2014-08-13 17:23:20.963993 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5986> 2014-08-13 17:23:20.964079 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x2f97260 con 0x4ec09a0 -5985> 2014-08-13 17:23:20.963974 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 3 ==== pg_notify(24.9b(4) epoch 194476) v5 ==== 980+0+0 (2128401015 0 0) 0x54d5400 con 0x53053e0 -5984> 2014-08-13 17:23:20.964111 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6818/45268 -- pg_query(22.113 epoch 194476) v3 -- ?+0 0x2f91680 con 0x5305c20 -5983> 2014-08-13 17:23:20.964113 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.946757, event: header_read, request: pg_notify(24.9b(4) epoch 194476) v5 -5982> 2014-08-13 17:23:20.964133 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.946758, event: throttled, request: pg_notify(24.9b(4) epoch 194476) v5 -5981> 2014-08-13 17:23:20.964139 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.946818, event: all_read, request: pg_notify(24.9b(4) epoch 194476) v5 -5980> 2014-08-13 17:23:20.964144 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.964109, event: dispatched, request: pg_notify(24.9b(4) epoch 194476) v5 -5979> 2014-08-13 17:23:20.964250 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.964250, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194476) v5 -5978> 2014-08-13 17:23:20.964304 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.964304, event: started, request: pg_notify(24.9b(4) epoch 194476) v5 -5977> 2014-08-13 17:23:20.964331 7fcd0964c700 5 -- op tracker -- , seq: 111, time: 2014-08-13 17:23:20.964331, event: done, request: pg_notify(24.9b(4) epoch 194476) v5 -5976> 2014-08-13 17:23:20.964391 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 3 ==== pg_notify(22.f3(3) epoch 194476) v5 ==== 887+0+0 (345973583 0 0) 0x54d4ec0 con 0x5307a60 -5975> 2014-08-13 17:23:20.964415 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.946524, event: header_read, request: pg_notify(22.f3(3) epoch 194476) v5 -5974> 2014-08-13 17:23:20.964429 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.946526, event: throttled, request: pg_notify(22.f3(3) epoch 194476) v5 -5973> 2014-08-13 17:23:20.964463 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.946597, event: all_read, request: pg_notify(22.f3(3) epoch 194476) v5 -5972> 2014-08-13 17:23:20.964472 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.964412, event: dispatched, request: pg_notify(22.f3(3) epoch 194476) v5 -5971> 2014-08-13 17:23:20.964483 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.964483, event: waiting_for_osdmap, request: pg_notify(22.f3(3) epoch 194476) v5 -5970> 2014-08-13 17:23:20.964542 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.964542, event: started, request: pg_notify(22.f3(3) epoch 194476) v5 -5969> 2014-08-13 17:23:20.964582 7fcd0964c700 5 -- op tracker -- , seq: 112, time: 2014-08-13 17:23:20.964582, event: done, request: pg_notify(22.f3(3) epoch 194476) v5 -5968> 2014-08-13 17:23:20.964633 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.311 192.168.1.101:6816/24270 2 ==== pg_notify(22.f3(20) epoch 194476) v5 ==== 1830+0+0 (514222415 0 0) 0x56a9a40 con 0x53077a0 -5967> 2014-08-13 17:23:20.964657 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.955097, event: header_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5966> 2014-08-13 17:23:20.964668 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.955098, event: throttled, request: pg_notify(22.f3(20) epoch 194476) v5 -5965> 2014-08-13 17:23:20.964677 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.955206, event: all_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5964> 2014-08-13 17:23:20.964686 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.964654, event: dispatched, request: pg_notify(22.f3(20) epoch 194476) v5 -5963> 2014-08-13 17:23:20.964696 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.964695, event: waiting_for_osdmap, request: pg_notify(22.f3(20) epoch 194476) v5 -5962> 2014-08-13 17:23:20.964707 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.964707, event: started, request: pg_notify(22.f3(20) epoch 194476) v5 -5961> 2014-08-13 17:23:20.964731 7fcd0964c700 5 -- op tracker -- , seq: 113, time: 2014-08-13 17:23:20.964730, event: done, request: pg_notify(22.f3(20) epoch 194476) v5 -5960> 2014-08-13 17:23:20.964936 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(26) epoch 194476) v5 -- ?+0 0x56ad400 con 0x4fd0f20 -5959> 2014-08-13 17:23:20.964965 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(29) epoch 194476) v5 -- ?+0 0x56ad240 con 0x4f80b00 -5958> 2014-08-13 17:23:20.965021 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(21) epoch 194476) v5 -- ?+0 0x56ad080 con 0x4ec3f40 -5957> 2014-08-13 17:23:20.965043 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(37) epoch 194476) v5 -- ?+0 0x56acec0 con 0x4ec56a0 -5956> 2014-08-13 17:23:20.965061 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_notify(22.3af(26) epoch 194476) v5 -- ?+0 0x56acd00 con 0x4eb0c60 -5955> 2014-08-13 17:23:20.964788 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 3 ==== pg_notify(22.f3(20) epoch 194476) v5 ==== 1830+0+0 (514222415 0 0) 0x54d5cc0 con 0x552a260 -5954> 2014-08-13 17:23:20.965096 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.947020, event: header_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5953> 2014-08-13 17:23:20.965107 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.947022, event: throttled, request: pg_notify(22.f3(20) epoch 194476) v5 -5952> 2014-08-13 17:23:20.965113 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.947159, event: all_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5951> 2014-08-13 17:23:20.965118 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.965092, event: dispatched, request: pg_notify(22.f3(20) epoch 194476) v5 -5950> 2014-08-13 17:23:20.965125 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.965124, event: waiting_for_osdmap, request: pg_notify(22.f3(20) epoch 194476) v5 -5949> 2014-08-13 17:23:20.965180 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.965180, event: started, request: pg_notify(22.f3(20) epoch 194476) v5 -5948> 2014-08-13 17:23:20.965223 7fcd0964c700 5 -- op tracker -- , seq: 114, time: 2014-08-13 17:23:20.965222, event: done, request: pg_notify(22.f3(20) epoch 194476) v5 -5947> 2014-08-13 17:23:20.965244 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(26) epoch 194476) v5 -- ?+0 0x56acb40 con 0x4ec09a0 -5946> 2014-08-13 17:23:20.965274 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6806/31286 -- pg_notify(24.181(31) epoch 194476) v5 -- ?+0 0x56ac980 con 0x4f81600 -5945> 2014-08-13 17:23:20.965289 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.302 192.168.1.113:6810/15731 3 ==== pg_notify(22.22c(3) epoch 194476) v5 ==== 887+0+0 (3579568949 0 0) 0x54d4600 con 0x552b9c0 -5944> 2014-08-13 17:23:20.965319 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(26) epoch 194476) v5 -- ?+0 0x56ac7c0 con 0x4f86e00 -5943> 2014-08-13 17:23:20.965334 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.922304, event: header_read, request: pg_notify(22.22c(3) epoch 194476) v5 -5942> 2014-08-13 17:23:20.965346 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.922306, event: throttled, request: pg_notify(22.22c(3) epoch 194476) v5 -5941> 2014-08-13 17:23:20.965355 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.922377, event: all_read, request: pg_notify(22.22c(3) epoch 194476) v5 -5940> 2014-08-13 17:23:20.965356 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(21) epoch 194476) v5 -- ?+0 0x56ac600 con 0x4e32940 -5939> 2014-08-13 17:23:20.965364 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.965330, event: dispatched, request: pg_notify(22.22c(3) epoch 194476) v5 -5938> 2014-08-13 17:23:20.965372 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.965372, event: waiting_for_osdmap, request: pg_notify(22.22c(3) epoch 194476) v5 -5937> 2014-08-13 17:23:20.965383 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.965383, event: started, request: pg_notify(22.22c(3) epoch 194476) v5 -5936> 2014-08-13 17:23:20.965409 7fcd0964c700 5 -- op tracker -- , seq: 115, time: 2014-08-13 17:23:20.965409, event: done, request: pg_notify(22.22c(3) epoch 194476) v5 -5935> 2014-08-13 17:23:20.965437 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 3 ==== pg_notify(22.22c(22) epoch 194476) v5 ==== 1944+0+0 (917915763 0 0) 0x5186e40 con 0x552b700 -5934> 2014-08-13 17:23:20.965482 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.945478, event: header_read, request: pg_notify(22.22c(22) epoch 194476) v5 -5933> 2014-08-13 17:23:20.965494 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.945479, event: throttled, request: pg_notify(22.22c(22) epoch 194476) v5 -5932> 2014-08-13 17:23:20.965504 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6814/41806 -- pg_notify(24.299(5) epoch 194476) v5 -- ?+0 0x56afe00 con 0x4fd3020 -5931> 2014-08-13 17:23:20.965503 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.945574, event: all_read, request: pg_notify(22.22c(22) epoch 194476) v5 -5930> 2014-08-13 17:23:20.965524 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.965479, event: dispatched, request: pg_notify(22.22c(22) epoch 194476) v5 -5929> 2014-08-13 17:23:20.965527 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(21) epoch 194476) v5 -- ?+0 0x56afc40 con 0x4fd53e0 -5928> 2014-08-13 17:23:20.965516 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- pg_notify(22.1f1(26) epoch 194476) v5 -- ?+0 0x56aeac0 con 0x4eb2680 -5927> 2014-08-13 17:23:20.965535 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.965535, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194476) v5 -5926> 2014-08-13 17:23:20.965557 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_notify(24.b6(28) epoch 194476) v5 -- ?+0 0x56ae900 con 0x4ec5ac0 -5925> 2014-08-13 17:23:20.965566 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.965565, event: started, request: pg_notify(22.22c(22) epoch 194476) v5 -5924> 2014-08-13 17:23:20.965578 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_notify(24.136(26) epoch 194476) v5 -- ?+0 0x56ae740 con 0x4eb7d20 -5923> 2014-08-13 17:23:20.965588 7fcd0964c700 5 -- op tracker -- , seq: 116, time: 2014-08-13 17:23:20.965588, event: done, request: pg_notify(22.22c(22) epoch 194476) v5 -5922> 2014-08-13 17:23:20.965619 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(21) epoch 194476) v5 -- ?+0 0x56ae580 con 0x4ec2aa0 -5921> 2014-08-13 17:23:20.965638 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.050834 4 0.000208 -5920> 2014-08-13 17:23:20.965837 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(21) epoch 194476) v5 -- ?+0 0x56ae3c0 con 0x4f870c0 -5919> 2014-08-13 17:23:20.965844 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5918> 2014-08-13 17:23:20.965839 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 3 ==== pg_notify(22.1d2(23) epoch 194476) v5 ==== 2003+0+0 (1106722567 0 0) 0x54d7700 con 0x552fa60 -5917> 2014-08-13 17:23:20.965858 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(21) epoch 194476) v5 -- ?+0 0x56ae200 con 0x4eb40a0 -5916> 2014-08-13 17:23:20.965870 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.947416, event: header_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -5915> 2014-08-13 17:23:20.965875 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(21) epoch 194476) v5 -- ?+0 0x56ae040 con 0x4f81fa0 -5914> 2014-08-13 17:23:20.965885 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.947417, event: throttled, request: pg_notify(22.1d2(23) epoch 194476) v5 -5913> 2014-08-13 17:23:20.965894 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.947507, event: all_read, request: pg_notify(22.1d2(23) epoch 194476) v5 -5912> 2014-08-13 17:23:20.965892 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(22) epoch 194476) v5 -- ?+0 0x56ade80 con 0x4eb23c0 -5911> 2014-08-13 17:23:20.965908 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.965867, event: dispatched, request: pg_notify(22.1d2(23) epoch 194476) v5 -5910> 2014-08-13 17:23:20.965920 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.965920, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194476) v5 -5909> 2014-08-13 17:23:20.965914 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000070 0 0.000000 -5908> 2014-08-13 17:23:20.965925 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6806/42024 -- pg_notify(24.1bb(26) epoch 194476) v5 -- ?+0 0x56adcc0 con 0x4fdbde0 -5907> 2014-08-13 17:23:20.965936 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.965936, event: started, request: pg_notify(22.1d2(23) epoch 194476) v5 -5906> 2014-08-13 17:23:20.965937 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5905> 2014-08-13 17:23:20.965953 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(21) epoch 194476) v5 -- ?+0 0x56adb00 con 0x4eb6720 -5904> 2014-08-13 17:23:20.965961 7fcd0964c700 5 -- op tracker -- , seq: 117, time: 2014-08-13 17:23:20.965961, event: done, request: pg_notify(22.1d2(23) epoch 194476) v5 -5903> 2014-08-13 17:23:20.965973 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(42) epoch 194476) v5 -- ?+0 0x56ad940 con 0x4eb18c0 -5902> 2014-08-13 17:23:20.966013 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 3 ==== pg_notify(24.3d(27) epoch 194476) v5 ==== 2265+0+0 (1442779177 0 0) 0x54d7a80 con 0x552f7a0 -5901> 2014-08-13 17:23:20.966118 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.947492, event: header_read, request: pg_notify(24.3d(27) epoch 194476) v5 -5900> 2014-08-13 17:23:20.966126 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.048772 8 0.000268 -5899> 2014-08-13 17:23:20.966157 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.947494, event: throttled, request: pg_notify(24.3d(27) epoch 194476) v5 -5898> 2014-08-13 17:23:20.966163 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5897> 2014-08-13 17:23:20.966169 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.947622, event: all_read, request: pg_notify(24.3d(27) epoch 194476) v5 -5896> 2014-08-13 17:23:20.966186 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.966030, event: dispatched, request: pg_notify(24.3d(27) epoch 194476) v5 -5895> 2014-08-13 17:23:20.966209 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.966209, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194476) v5 -5894> 2014-08-13 17:23:20.966223 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.966223, event: started, request: pg_notify(24.3d(27) epoch 194476) v5 -5893> 2014-08-13 17:23:20.966240 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000077 0 0.000000 -5892> 2014-08-13 17:23:20.966267 7fcd0964c700 5 -- op tracker -- , seq: 118, time: 2014-08-13 17:23:20.966266, event: done, request: pg_notify(24.3d(27) epoch 194476) v5 -5891> 2014-08-13 17:23:20.966266 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5890> 2014-08-13 17:23:20.966304 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 3 ==== pg_query(24.e2 epoch 194476) v3 ==== 144+0+0 (1889542248 0 0) 0x54fab20 con 0x5520b00 -5889> 2014-08-13 17:23:20.966332 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(21) epoch 194476) v5 -- ?+0 0x56af8c0 con 0x4fd32e0 -5888> 2014-08-13 17:23:20.966326 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.946548, event: header_read, request: pg_query(24.e2 epoch 194476) v3 -5887> 2014-08-13 17:23:20.966336 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.050855 8 0.000276 -5886> 2014-08-13 17:23:20.966363 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.946549, event: throttled, request: pg_query(24.e2 epoch 194476) v3 -5885> 2014-08-13 17:23:20.966363 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6814/16727 -- pg_query(22.22c epoch 194476) v3 -- ?+0 0x2f96720 con 0x4e35800 -5884> 2014-08-13 17:23:20.966375 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.946598, event: all_read, request: pg_query(24.e2 epoch 194476) v3 -5883> 2014-08-13 17:23:20.966366 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog -5882> 2014-08-13 17:23:20.966383 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.966323, event: dispatched, request: pg_query(24.e2 epoch 194476) v3 -5881> 2014-08-13 17:23:20.966393 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.966393, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194476) v3 -5880> 2014-08-13 17:23:20.966405 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.966405, event: started, request: pg_query(24.e2 epoch 194476) v3 -5879> 2014-08-13 17:23:20.966428 7fcd0964c700 5 -- op tracker -- , seq: 119, time: 2014-08-13 17:23:20.966428, event: done, request: pg_query(24.e2 epoch 194476) v3 -5878> 2014-08-13 17:23:20.966438 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.054408 4 0.000209 -5877> 2014-08-13 17:23:20.966447 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000081 0 0.000000 -5876> 2014-08-13 17:23:20.966451 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 3 ==== pg_notify(22.481(39) epoch 194476) v5 ==== 2911+0+0 (3107951277 0 0) 0x54d7380 con 0x552d540 -5875> 2014-08-13 17:23:20.966471 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing -5874> 2014-08-13 17:23:20.966486 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.964792, event: header_read, request: pg_notify(22.481(39) epoch 194476) v5 -5873> 2014-08-13 17:23:20.966497 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.964795, event: throttled, request: pg_notify(22.481(39) epoch 194476) v5 -5872> 2014-08-13 17:23:20.966477 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5871> 2014-08-13 17:23:20.966503 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.965167, event: all_read, request: pg_notify(22.481(39) epoch 194476) v5 -5870> 2014-08-13 17:23:20.966503 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.000032 0 0.000000 -5869> 2014-08-13 17:23:20.966531 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.966483, event: dispatched, request: pg_notify(22.481(39) epoch 194476) v5 -5868> 2014-08-13 17:23:20.966560 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000083 0 0.000000 -5867> 2014-08-13 17:23:20.966526 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru -5866> 2014-08-13 17:23:20.966591 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.966590, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194476) v5 -5865> 2014-08-13 17:23:20.966582 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5864> 2014-08-13 17:23:20.966603 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.966603, event: started, request: pg_notify(22.481(39) epoch 194476) v5 -5863> 2014-08-13 17:23:20.966631 7fcd0964c700 5 -- op tracker -- , seq: 120, time: 2014-08-13 17:23:20.966630, event: done, request: pg_notify(22.481(39) epoch 194476) v5 -5862> 2014-08-13 17:23:20.966672 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.058822 8 0.030599 -5861> 2014-08-13 17:23:20.966676 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x550cc80 con 0x56b2260 -5860> 2014-08-13 17:23:20.966710 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -5859> 2014-08-13 17:23:20.966730 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -5858> 2014-08-13 17:23:20.966752 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.352 192.168.1.108:6826/4273 2 ==== pg_notify(22.4df(3) epoch 194476) v5 ==== 887+0+0 (1125860178 0 0) 0x5187a80 con 0x56b1fa0 -5857> 2014-08-13 17:23:20.966795 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.959548, event: header_read, request: pg_notify(22.4df(3) epoch 194476) v5 -5856> 2014-08-13 17:23:20.966804 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.959549, event: throttled, request: pg_notify(22.4df(3) epoch 194476) v5 -5855> 2014-08-13 17:23:20.966796 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000086 0 0.000000 -5854> 2014-08-13 17:23:20.966814 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.959627, event: all_read, request: pg_notify(22.4df(3) epoch 194476) v5 -5853> 2014-08-13 17:23:20.966835 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.966792, event: dispatched, request: pg_notify(22.4df(3) epoch 194476) v5 -5852> 2014-08-13 17:23:20.966829 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -5851> 2014-08-13 17:23:20.966847 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.966846, event: waiting_for_osdmap, request: pg_notify(22.4df(3) epoch 194476) v5 -5850> 2014-08-13 17:23:20.966859 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.966858, event: started, request: pg_notify(22.4df(3) epoch 194476) v5 -5849> 2014-08-13 17:23:20.966858 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 -5848> 2014-08-13 17:23:20.966886 7fcd0964c700 5 -- op tracker -- , seq: 121, time: 2014-08-13 17:23:20.966886, event: done, request: pg_notify(22.4df(3) epoch 194476) v5 -5847> 2014-08-13 17:23:20.966885 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -5846> 2014-08-13 17:23:20.966911 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.380 192.168.1.114:6822/51844 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x5509200 con 0x56b1ce0 -5845> 2014-08-13 17:23:20.966932 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -5844> 2014-08-13 17:23:20.966948 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 3 ==== pg_notify(22.f3(21) epoch 194476) v5 ==== 1889+0+0 (241229983 0 0) 0x54d4980 con 0x4e314a0 -5843> 2014-08-13 17:23:20.966966 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.945838, event: header_read, request: pg_notify(22.f3(21) epoch 194476) v5 -5842> 2014-08-13 17:23:20.966977 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.945841, event: throttled, request: pg_notify(22.f3(21) epoch 194476) v5 -5841> 2014-08-13 17:23:20.966986 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.945957, event: all_read, request: pg_notify(22.f3(21) epoch 194476) v5 -5840> 2014-08-13 17:23:20.966996 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.966964, event: dispatched, request: pg_notify(22.f3(21) epoch 194476) v5 -5839> 2014-08-13 17:23:20.967031 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.967031, event: waiting_for_osdmap, request: pg_notify(22.f3(21) epoch 194476) v5 -5838> 2014-08-13 17:23:20.967029 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(21) epoch 194476) v5 -- ?+0 0x5889500 con 0x4e34780 -5837> 2014-08-13 17:23:20.967046 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.967046, event: started, request: pg_notify(22.f3(21) epoch 194476) v5 -5836> 2014-08-13 17:23:20.967010 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_log(24.2a1 epoch 194476 log log((194422'14353,194422'14353], crt=194422'14353) query_epoch 194476) v4 -- ?+0 0x56dce00 con 0x4ec5ac0 -5835> 2014-08-13 17:23:20.967058 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_notify(22.362(26) epoch 194476) v5 -- ?+0 0x5889340 con 0x4e335a0 -5834> 2014-08-13 17:23:20.967076 7fcd0964c700 5 -- op tracker -- , seq: 122, time: 2014-08-13 17:23:20.967076, event: done, request: pg_notify(22.f3(21) epoch 194476) v5 -5833> 2014-08-13 17:23:20.967108 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 5 ==== pg_notify(22.1d2(24) epoch 194476) v5 ==== 2062+0+0 (2408723064 0 0) 0x54d0700 con 0x4e335a0 -5832> 2014-08-13 17:23:20.967129 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.920754, event: header_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -5831> 2014-08-13 17:23:20.967140 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.920756, event: throttled, request: pg_notify(22.1d2(24) epoch 194476) v5 -5830> 2014-08-13 17:23:20.967199 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.920880, event: all_read, request: pg_notify(22.1d2(24) epoch 194476) v5 -5829> 2014-08-13 17:23:20.967208 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.967127, event: dispatched, request: pg_notify(22.1d2(24) epoch 194476) v5 -5828> 2014-08-13 17:23:20.967218 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.967218, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194476) v5 -5827> 2014-08-13 17:23:20.967260 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.967260, event: started, request: pg_notify(22.1d2(24) epoch 194476) v5 -5826> 2014-08-13 17:23:20.967272 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(39) epoch 194476) v5 -- ?+0 0x5889180 con 0x4ec3440 -5825> 2014-08-13 17:23:20.967295 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(26) epoch 194476) v5 -- ?+0 0x5888fc0 con 0x4ec1ce0 -5824> 2014-08-13 17:23:20.967314 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(28) epoch 194476) v5 -- ?+0 0x5888e00 con 0x5520b00 -5823> 2014-08-13 17:23:20.967319 7fcd0964c700 5 -- op tracker -- , seq: 123, time: 2014-08-13 17:23:20.967319, event: done, request: pg_notify(22.1d2(24) epoch 194476) v5 -5822> 2014-08-13 17:23:20.967352 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x2f97080 con 0x4e35ac0 -5821> 2014-08-13 17:23:20.967360 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 2 ==== pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 ==== 1494+0+0 (3973634273 0 0) 0x56aa140 con 0x4e35540 -5820> 2014-08-13 17:23:20.967398 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6800/61726 -- pg_query(22.1d2 epoch 194476) v3 -- ?+0 0x2f97620 con 0x552fa60 -5819> 2014-08-13 17:23:20.967405 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.957304, event: header_read, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5818> 2014-08-13 17:23:20.967426 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.957306, event: throttled, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5817> 2014-08-13 17:23:20.967450 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.957393, event: all_read, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5816> 2014-08-13 17:23:20.967461 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.967401, event: dispatched, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5815> 2014-08-13 17:23:20.967472 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.967472, event: waiting_for_osdmap, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5814> 2014-08-13 17:23:20.967490 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.967489, event: started, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5813> 2014-08-13 17:23:20.967538 7fcd0964c700 5 -- op tracker -- , seq: 124, time: 2014-08-13 17:23:20.967537, event: done, request: pg_notify(2.2a(1),1.2b(1) epoch 194476) v5 -5812> 2014-08-13 17:23:20.967568 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 5 ==== pg_query(24.369 epoch 194476) v3 ==== 144+0+0 (3026569778 0 0) 0x54f94a0 con 0x4ec09a0 -5811> 2014-08-13 17:23:20.967649 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(25) epoch 194476) v5 -- ?+0 0x56af540 con 0x4fd86e0 -5810> 2014-08-13 17:23:20.967672 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.960943, event: header_read, request: pg_query(24.369 epoch 194476) v3 -5809> 2014-08-13 17:23:20.967681 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.960944, event: throttled, request: pg_query(24.369 epoch 194476) v3 -5808> 2014-08-13 17:23:20.967686 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_query(22.f3 epoch 194476) v3 -- ?+0 0x2f90d20 con 0x4e314a0 -5807> 2014-08-13 17:23:20.967705 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.961003, event: all_read, request: pg_query(24.369 epoch 194476) v3 -5806> 2014-08-13 17:23:20.967712 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.967664, event: dispatched, request: pg_query(24.369 epoch 194476) v3 -5805> 2014-08-13 17:23:20.967707 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194476 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194476) v4 -- ?+0 0x56de000 con 0x5520b00 -5804> 2014-08-13 17:23:20.967723 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.967723, event: waiting_for_osdmap, request: pg_query(24.369 epoch 194476) v3 -5803> 2014-08-13 17:23:20.967735 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.967735, event: started, request: pg_query(24.369 epoch 194476) v3 -5802> 2014-08-13 17:23:20.967758 7fcd0964c700 5 -- op tracker -- , seq: 125, time: 2014-08-13 17:23:20.967758, event: done, request: pg_query(24.369 epoch 194476) v3 -5801> 2014-08-13 17:23:20.967777 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 3 ==== pg_notify(22.84(22) epoch 194476) v5 ==== 1948+0+0 (4138702873 0 0) 0x5187000 con 0x4ec1ce0 -5800> 2014-08-13 17:23:20.967798 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.924145, event: header_read, request: pg_notify(22.84(22) epoch 194476) v5 -5799> 2014-08-13 17:23:20.967805 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.924179, event: throttled, request: pg_notify(22.84(22) epoch 194476) v5 -5798> 2014-08-13 17:23:20.967811 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.924311, event: all_read, request: pg_notify(22.84(22) epoch 194476) v5 -5797> 2014-08-13 17:23:20.967817 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.967796, event: dispatched, request: pg_notify(22.84(22) epoch 194476) v5 -5796> 2014-08-13 17:23:20.967845 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.967845, event: waiting_for_osdmap, request: pg_notify(22.84(22) epoch 194476) v5 -5795> 2014-08-13 17:23:20.967855 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.967855, event: started, request: pg_notify(22.84(22) epoch 194476) v5 -5794> 2014-08-13 17:23:20.967906 7fcd0964c700 5 -- op tracker -- , seq: 126, time: 2014-08-13 17:23:20.967906, event: done, request: pg_notify(22.84(22) epoch 194476) v5 -5793> 2014-08-13 17:23:20.967937 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_log(24.369 epoch 194476 log log((194422'13576,194422'13576], crt=194422'13576) query_epoch 194476) v4 -- ?+0 0x56dec00 con 0x4ec09a0 -5792> 2014-08-13 17:23:20.967935 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.048239 2 0.000272 -5791> 2014-08-13 17:23:20.967960 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 3 ==== pg_notify(24.9b(29) epoch 194476) v5 ==== 2383+0+0 (2454009132 0 0) 0x54d6040 con 0x4ec27e0 -5790> 2014-08-13 17:23:20.967979 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.947110, event: header_read, request: pg_notify(24.9b(29) epoch 194476) v5 -5789> 2014-08-13 17:23:20.967980 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5788> 2014-08-13 17:23:20.967998 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.947112, event: throttled, request: pg_notify(24.9b(29) epoch 194476) v5 -5787> 2014-08-13 17:23:20.968036 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000056 0 0.000000 -5786> 2014-08-13 17:23:20.968043 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.947244, event: all_read, request: pg_notify(24.9b(29) epoch 194476) v5 -5785> 2014-08-13 17:23:20.968068 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.967977, event: dispatched, request: pg_notify(24.9b(29) epoch 194476) v5 -5784> 2014-08-13 17:23:20.968061 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5783> 2014-08-13 17:23:20.968077 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.968076, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194476) v5 -5782> 2014-08-13 17:23:20.968085 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.968085, event: started, request: pg_notify(24.9b(29) epoch 194476) v5 -5781> 2014-08-13 17:23:20.968088 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000027 0 0.000000 -5780> 2014-08-13 17:23:20.968128 7fcd0964c700 5 -- op tracker -- , seq: 127, time: 2014-08-13 17:23:20.968128, event: done, request: pg_notify(24.9b(29) epoch 194476) v5 -5779> 2014-08-13 17:23:20.968124 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5778> 2014-08-13 17:23:20.968149 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.020719 4 0.000182 -5777> 2014-08-13 17:23:20.968202 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5776> 2014-08-13 17:23:20.968219 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 4 ==== pg_notify(22.6bd(22) epoch 194476) v5 ==== 1948+0+0 (2308095656 0 0) 0x4e51180 con 0x4ec3440 -5775> 2014-08-13 17:23:20.968279 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.915007, event: header_read, request: pg_notify(22.6bd(22) epoch 194476) v5 -5774> 2014-08-13 17:23:20.968266 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.045761 2 0.000206 -5773> 2014-08-13 17:23:20.968270 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000068 0 0.000000 -5772> 2014-08-13 17:23:20.968293 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.915008, event: throttled, request: pg_notify(22.6bd(22) epoch 194476) v5 -5771> 2014-08-13 17:23:20.968304 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.915118, event: all_read, request: pg_notify(22.6bd(22) epoch 194476) v5 -5770> 2014-08-13 17:23:20.968360 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.968275, event: dispatched, request: pg_notify(22.6bd(22) epoch 194476) v5 -5769> 2014-08-13 17:23:20.968302 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5768> 2014-08-13 17:23:20.968298 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5767> 2014-08-13 17:23:20.968370 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.968370, event: waiting_for_osdmap, request: pg_notify(22.6bd(22) epoch 194476) v5 -5766> 2014-08-13 17:23:20.968385 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.968385, event: started, request: pg_notify(22.6bd(22) epoch 194476) v5 -5765> 2014-08-13 17:23:20.968382 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000081 0 0.000000 -5764> 2014-08-13 17:23:20.968413 7fcd0964c700 5 -- op tracker -- , seq: 128, time: 2014-08-13 17:23:20.968413, event: done, request: pg_notify(22.6bd(22) epoch 194476) v5 -5763> 2014-08-13 17:23:20.968410 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5762> 2014-08-13 17:23:20.968422 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000124 0 0.000000 -5761> 2014-08-13 17:23:20.968449 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5760> 2014-08-13 17:23:20.968454 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 5 ==== pg_query(24.350 epoch 194476) v3 ==== 144+0+0 (763810051 0 0) 0x2f95640 con 0x4ec4780 -5759> 2014-08-13 17:23:20.968473 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.957110, event: header_read, request: pg_query(24.350 epoch 194476) v3 -5758> 2014-08-13 17:23:20.968491 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.015877 4 0.000229 -5757> 2014-08-13 17:23:20.968474 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000025 0 0.000000 -5756> 2014-08-13 17:23:20.968517 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5755> 2014-08-13 17:23:20.968518 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.957111, event: throttled, request: pg_query(24.350 epoch 194476) v3 -5754> 2014-08-13 17:23:20.968555 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.957188, event: all_read, request: pg_query(24.350 epoch 194476) v3 -5753> 2014-08-13 17:23:20.968567 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.968470, event: dispatched, request: pg_query(24.350 epoch 194476) v3 -5752> 2014-08-13 17:23:20.968544 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5751> 2014-08-13 17:23:20.968585 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.968585, event: waiting_for_osdmap, request: pg_query(24.350 epoch 194476) v3 -5750> 2014-08-13 17:23:20.968595 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.968594, event: started, request: pg_query(24.350 epoch 194476) v3 -5749> 2014-08-13 17:23:20.968590 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000072 0 0.000000 -5748> 2014-08-13 17:23:20.968617 7fcd0964c700 5 -- op tracker -- , seq: 129, time: 2014-08-13 17:23:20.968617, event: done, request: pg_query(24.350 epoch 194476) v3 -5747> 2014-08-13 17:23:20.968615 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5746> 2014-08-13 17:23:20.968637 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 -5745> 2014-08-13 17:23:20.968646 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 4 ==== pg_query(24.2ec epoch 194476) v3 ==== 144+0+0 (3915811195 0 0) 0x2f94920 con 0x4eb18c0 -5744> 2014-08-13 17:23:20.968664 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.962107, event: header_read, request: pg_query(24.2ec epoch 194476) v3 -5743> 2014-08-13 17:23:20.968672 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.962108, event: throttled, request: pg_query(24.2ec epoch 194476) v3 -5742> 2014-08-13 17:23:20.968663 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5741> 2014-08-13 17:23:20.968686 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.962183, event: all_read, request: pg_query(24.2ec epoch 194476) v3 -5740> 2014-08-13 17:23:20.968692 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.968662, event: dispatched, request: pg_query(24.2ec epoch 194476) v3 -5739> 2014-08-13 17:23:20.968702 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.968701, event: waiting_for_osdmap, request: pg_query(24.2ec epoch 194476) v3 -5738> 2014-08-13 17:23:20.968712 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.968712, event: started, request: pg_query(24.2ec epoch 194476) v3 -5737> 2014-08-13 17:23:20.968724 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_log(24.350 epoch 194476 log log((194422'17433,194422'17433], crt=194422'17433) query_epoch 194476) v4 -- ?+0 0x56dda00 con 0x4ec4780 -5736> 2014-08-13 17:23:20.968762 7fcd0964c700 5 -- op tracker -- , seq: 130, time: 2014-08-13 17:23:20.968762, event: done, request: pg_query(24.2ec epoch 194476) v3 -5735> 2014-08-13 17:23:20.968786 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 4 ==== pg_notify(22.3d5(1) epoch 194476) v5 ==== 769+0+0 (767481504 0 0) 0x56ab640 con 0x4f86e00 -5734> 2014-08-13 17:23:20.968828 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.964231, event: header_read, request: pg_notify(22.3d5(1) epoch 194476) v5 -5733> 2014-08-13 17:23:20.968840 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.964233, event: throttled, request: pg_notify(22.3d5(1) epoch 194476) v5 -5732> 2014-08-13 17:23:20.968850 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.964315, event: all_read, request: pg_notify(22.3d5(1) epoch 194476) v5 -5731> 2014-08-13 17:23:20.968859 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.968826, event: dispatched, request: pg_notify(22.3d5(1) epoch 194476) v5 -5730> 2014-08-13 17:23:20.968868 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.968868, event: waiting_for_osdmap, request: pg_notify(22.3d5(1) epoch 194476) v5 -5729> 2014-08-13 17:23:20.968883 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.968883, event: started, request: pg_notify(22.3d5(1) epoch 194476) v5 -5728> 2014-08-13 17:23:20.968909 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_log(24.2ec epoch 194476 log log((194422'16449,194422'16449], crt=194422'16449) query_epoch 194476) v4 -- ?+0 0x56de600 con 0x4eb18c0 -5727> 2014-08-13 17:23:20.968931 7fcd0964c700 5 -- op tracker -- , seq: 131, time: 2014-08-13 17:23:20.968930, event: done, request: pg_notify(22.3d5(1) epoch 194476) v5 -5726> 2014-08-13 17:23:20.968975 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 4 ==== pg_notify(22.4df(22) epoch 194476) v5 ==== 1944+0+0 (309562417 0 0) 0x54d7000 con 0x4fd86e0 -5725> 2014-08-13 17:23:20.969004 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.949501, event: header_read, request: pg_notify(22.4df(22) epoch 194476) v5 -5724> 2014-08-13 17:23:20.969016 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.949503, event: throttled, request: pg_notify(22.4df(22) epoch 194476) v5 -5723> 2014-08-13 17:23:20.969025 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.949623, event: all_read, request: pg_notify(22.4df(22) epoch 194476) v5 -5722> 2014-08-13 17:23:20.969034 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.969001, event: dispatched, request: pg_notify(22.4df(22) epoch 194476) v5 -5721> 2014-08-13 17:23:20.969043 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.969043, event: waiting_for_osdmap, request: pg_notify(22.4df(22) epoch 194476) v5 -5720> 2014-08-13 17:23:20.969055 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.969054, event: started, request: pg_notify(22.4df(22) epoch 194476) v5 -5719> 2014-08-13 17:23:20.969079 7fcd0964c700 5 -- op tracker -- , seq: 132, time: 2014-08-13 17:23:20.969078, event: done, request: pg_notify(22.4df(22) epoch 194476) v5 -5718> 2014-08-13 17:23:20.969116 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 3 ==== pg_notify(22.3d5(22) epoch 194476) v5 ==== 1948+0+0 (1599452933 0 0) 0x56aaf40 con 0x4fdad60 -5717> 2014-08-13 17:23:20.969137 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.964092, event: header_read, request: pg_notify(22.3d5(22) epoch 194476) v5 -5716> 2014-08-13 17:23:20.969163 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.964093, event: throttled, request: pg_notify(22.3d5(22) epoch 194476) v5 -5715> 2014-08-13 17:23:20.969173 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.964211, event: all_read, request: pg_notify(22.3d5(22) epoch 194476) v5 -5714> 2014-08-13 17:23:20.969180 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.969135, event: dispatched, request: pg_notify(22.3d5(22) epoch 194476) v5 -5713> 2014-08-13 17:23:20.969196 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.969196, event: waiting_for_osdmap, request: pg_notify(22.3d5(22) epoch 194476) v5 -5712> 2014-08-13 17:23:20.969208 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.969208, event: started, request: pg_notify(22.3d5(22) epoch 194476) v5 -5711> 2014-08-13 17:23:20.969245 7fcd0964c700 5 -- op tracker -- , seq: 133, time: 2014-08-13 17:23:20.969244, event: done, request: pg_notify(22.3d5(22) epoch 194476) v5 -5710> 2014-08-13 17:23:20.969274 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 3 ==== pg_query(24.cd epoch 194476) v3 ==== 144+0+0 (1153571580 0 0) 0x2f94740 con 0x4fd0f20 -5709> 2014-08-13 17:23:20.969293 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.963900, event: header_read, request: pg_query(24.cd epoch 194476) v3 -5708> 2014-08-13 17:23:20.969303 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.963901, event: throttled, request: pg_query(24.cd epoch 194476) v3 -5707> 2014-08-13 17:23:20.969312 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.963976, event: all_read, request: pg_query(24.cd epoch 194476) v3 -5706> 2014-08-13 17:23:20.969321 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.969291, event: dispatched, request: pg_query(24.cd epoch 194476) v3 -5705> 2014-08-13 17:23:20.969330 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.969330, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194476) v3 -5704> 2014-08-13 17:23:20.969341 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.969341, event: started, request: pg_query(24.cd epoch 194476) v3 -5703> 2014-08-13 17:23:20.969365 7fcd0964c700 5 -- op tracker -- , seq: 134, time: 2014-08-13 17:23:20.969365, event: done, request: pg_query(24.cd epoch 194476) v3 -5702> 2014-08-13 17:23:20.969392 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 5 ==== pg_notify(22.481(34) epoch 194476) v5 ==== 2628+0+0 (3710564698 0 0) 0x54d0a80 con 0x4fd06e0 -5701> 2014-08-13 17:23:20.969411 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.964544, event: header_read, request: pg_notify(22.481(34) epoch 194476) v5 -5700> 2014-08-13 17:23:20.969407 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194476 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194476) v4 -- ?+0 0x58c9200 con 0x4fd0f20 -5699> 2014-08-13 17:23:20.969422 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.964547, event: throttled, request: pg_notify(22.481(34) epoch 194476) v5 -5698> 2014-08-13 17:23:20.969431 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.964750, event: all_read, request: pg_notify(22.481(34) epoch 194476) v5 -5697> 2014-08-13 17:23:20.969440 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.969409, event: dispatched, request: pg_notify(22.481(34) epoch 194476) v5 -5696> 2014-08-13 17:23:20.969450 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.969449, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194476) v5 -5695> 2014-08-13 17:23:20.969461 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.969460, event: started, request: pg_notify(22.481(34) epoch 194476) v5 -5694> 2014-08-13 17:23:20.969489 7fcd0964c700 5 -- op tracker -- , seq: 135, time: 2014-08-13 17:23:20.969488, event: done, request: pg_notify(22.481(34) epoch 194476) v5 -5693> 2014-08-13 17:23:20.969557 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 3 ==== pg_notify(22.481(34) epoch 194476) v5 ==== 2628+0+0 (996605771 0 0) 0x54d3640 con 0x5303700 -5692> 2014-08-13 17:23:20.969577 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.965178, event: header_read, request: pg_notify(22.481(34) epoch 194476) v5 -5691> 2014-08-13 17:23:20.969588 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.965181, event: throttled, request: pg_notify(22.481(34) epoch 194476) v5 -5690> 2014-08-13 17:23:20.969597 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.965307, event: all_read, request: pg_notify(22.481(34) epoch 194476) v5 -5689> 2014-08-13 17:23:20.969605 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.969575, event: dispatched, request: pg_notify(22.481(34) epoch 194476) v5 -5688> 2014-08-13 17:23:20.969633 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.969615, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194476) v5 -5687> 2014-08-13 17:23:20.969647 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.969647, event: started, request: pg_notify(22.481(34) epoch 194476) v5 -5686> 2014-08-13 17:23:20.969684 7fcd0964c700 5 -- op tracker -- , seq: 136, time: 2014-08-13 17:23:20.969684, event: done, request: pg_notify(22.481(34) epoch 194476) v5 -5685> 2014-08-13 17:23:20.969715 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 4 ==== pg_notify(22.481(34) epoch 194476) v5 ==== 2628+0+0 (944907138 0 0) 0x54d5240 con 0x53053e0 -5684> 2014-08-13 17:23:20.969736 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.946828, event: header_read, request: pg_notify(22.481(34) epoch 194476) v5 -5683> 2014-08-13 17:23:20.969746 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.946829, event: throttled, request: pg_notify(22.481(34) epoch 194476) v5 -5682> 2014-08-13 17:23:20.969755 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.946944, event: all_read, request: pg_notify(22.481(34) epoch 194476) v5 -5681> 2014-08-13 17:23:20.969764 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.969734, event: dispatched, request: pg_notify(22.481(34) epoch 194476) v5 -5680> 2014-08-13 17:23:20.969774 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.969774, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194476) v5 -5679> 2014-08-13 17:23:20.969811 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.969811, event: started, request: pg_notify(22.481(34) epoch 194476) v5 -5678> 2014-08-13 17:23:20.969865 7fcd0964c700 5 -- op tracker -- , seq: 137, time: 2014-08-13 17:23:20.969865, event: done, request: pg_notify(22.481(34) epoch 194476) v5 -5677> 2014-08-13 17:23:20.969919 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.311 192.168.1.101:6816/24270 3 ==== pg_notify(22.f3(20) epoch 194476) v5 ==== 1830+0+0 (514222415 0 0) 0x4e50700 con 0x53077a0 -5676> 2014-08-13 17:23:20.969944 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.955227, event: header_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5675> 2014-08-13 17:23:20.969955 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.955229, event: throttled, request: pg_notify(22.f3(20) epoch 194476) v5 -5674> 2014-08-13 17:23:20.969996 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.955353, event: all_read, request: pg_notify(22.f3(20) epoch 194476) v5 -5673> 2014-08-13 17:23:20.970046 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.969941, event: dispatched, request: pg_notify(22.f3(20) epoch 194476) v5 -5672> 2014-08-13 17:23:20.970016 7fccd7000700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530eb80 sd=398 :6820 s=0 pgs=0 cs=0 l=0 c=0x55f2940).accept sd=398 192.168.1.106:54821/0 -5671> 2014-08-13 17:23:20.970070 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.970069, event: waiting_for_osdmap, request: pg_notify(22.f3(20) epoch 194476) v5 -5670> 2014-08-13 17:23:20.970081 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.970081, event: started, request: pg_notify(22.f3(20) epoch 194476) v5 -5669> 2014-08-13 17:23:20.970139 7fccd6eff700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530f580 sd=399 :6808 s=0 pgs=0 cs=0 l=0 c=0x55f3020).accept sd=399 192.168.100.106:51074/0 -5668> 2014-08-13 17:23:20.970127 7fcd0964c700 5 -- op tracker -- , seq: 138, time: 2014-08-13 17:23:20.970126, event: done, request: pg_notify(22.f3(20) epoch 194476) v5 -5667> 2014-08-13 17:23:20.970256 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 4 ==== pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 ==== 3917+0+0 (4226024447 0 0) 0x56dc200 con 0x5305c20 -5666> 2014-08-13 17:23:20.970291 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.964844, event: header_read, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5665> 2014-08-13 17:23:20.970308 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.964846, event: throttled, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5664> 2014-08-13 17:23:20.970319 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.965039, event: all_read, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5663> 2014-08-13 17:23:20.970353 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.970288, event: dispatched, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5662> 2014-08-13 17:23:20.970366 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.970365, event: waiting_for_osdmap, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5661> 2014-08-13 17:23:20.970399 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.970398, event: started, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5660> 2014-08-13 17:23:20.970429 7fcd0964c700 5 -- op tracker -- , seq: 139, time: 2014-08-13 17:23:20.970429, event: done, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5659> 2014-08-13 17:23:20.970465 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 4 ==== pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 ==== 4055+0+0 (1502328404 0 0) 0x56df800 con 0x552fa60 -5658> 2014-08-13 17:23:20.970489 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.968521, event: header_read, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5657> 2014-08-13 17:23:20.970501 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.968522, event: throttled, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5656> 2014-08-13 17:23:20.970512 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.968639, event: all_read, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5655> 2014-08-13 17:23:20.970522 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.970486, event: dispatched, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5654> 2014-08-13 17:23:20.970535 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.970535, event: waiting_for_osdmap, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5653> 2014-08-13 17:23:20.970548 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.970548, event: started, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5652> 2014-08-13 17:23:20.970571 7fcd0964c700 5 -- op tracker -- , seq: 140, time: 2014-08-13 17:23:20.970571, event: done, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5651> 2014-08-13 17:23:20.970598 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 4 ==== pg_query(24.e2 epoch 194476) v3 ==== 144+0+0 (3467900948 0 0) 0x54f83c0 con 0x5520b00 -5650> 2014-08-13 17:23:20.970617 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.968975, event: header_read, request: pg_query(24.e2 epoch 194476) v3 -5649> 2014-08-13 17:23:20.970626 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.968977, event: throttled, request: pg_query(24.e2 epoch 194476) v3 -5648> 2014-08-13 17:23:20.970635 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.969032, event: all_read, request: pg_query(24.e2 epoch 194476) v3 -5647> 2014-08-13 17:23:20.970643 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.970614, event: dispatched, request: pg_query(24.e2 epoch 194476) v3 -5646> 2014-08-13 17:23:20.970670 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.970669, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194476) v3 -5645> 2014-08-13 17:23:20.970681 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.970681, event: started, request: pg_query(24.e2 epoch 194476) v3 -5644> 2014-08-13 17:23:20.970707 7fcd0964c700 5 -- op tracker -- , seq: 141, time: 2014-08-13 17:23:20.970707, event: done, request: pg_query(24.e2 epoch 194476) v3 -5643> 2014-08-13 17:23:20.970732 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 2 ==== pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 ==== 3994+0+0 (349501593 0 0) 0x54d4280 con 0x56b2260 -5642> 2014-08-13 17:23:20.970753 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.961435, event: header_read, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5641> 2014-08-13 17:23:20.970764 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.961438, event: throttled, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5640> 2014-08-13 17:23:20.970755 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194476 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194476) v4 -- ?+0 0x56dc200 con 0x5520b00 -5639> 2014-08-13 17:23:20.970774 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.961605, event: all_read, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5638> 2014-08-13 17:23:20.970783 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.970751, event: dispatched, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5637> 2014-08-13 17:23:20.970793 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.970793, event: waiting_for_osdmap, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5636> 2014-08-13 17:23:20.970804 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.970804, event: started, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5635> 2014-08-13 17:23:20.970865 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.022791 7 0.000375 -5634> 2014-08-13 17:23:20.970906 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5633> 2014-08-13 17:23:20.970977 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000071 0 0.000000 -5632> 2014-08-13 17:23:20.971003 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5631> 2014-08-13 17:23:20.971060 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x2f94560 con 0x4ec4780 -5630> 2014-08-13 17:23:20.971070 7fcd0964c700 5 -- op tracker -- , seq: 142, time: 2014-08-13 17:23:20.971069, event: done, request: pg_notify(22.4df(23),22.4df(23) epoch 194476) v5 -5629> 2014-08-13 17:23:20.971086 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6834/17224 -- pg_query(22.4df epoch 194476) v3 -- ?+0 0x2f94ce0 con 0x56b2260 -5628> 2014-08-13 17:23:20.971139 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.352 192.168.1.108:6826/4273 3 ==== pg_notify(22.4df(3) epoch 194476) v5 ==== 887+0+0 (1125860178 0 0) 0x5184ec0 con 0x56b1fa0 -5627> 2014-08-13 17:23:20.971174 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.960445, event: header_read, request: pg_notify(22.4df(3) epoch 194476) v5 -5626> 2014-08-13 17:23:20.971185 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.960449, event: throttled, request: pg_notify(22.4df(3) epoch 194476) v5 -5625> 2014-08-13 17:23:20.971194 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.960536, event: all_read, request: pg_notify(22.4df(3) epoch 194476) v5 -5624> 2014-08-13 17:23:20.971203 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.971170, event: dispatched, request: pg_notify(22.4df(3) epoch 194476) v5 -5623> 2014-08-13 17:23:20.971222 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.971222, event: waiting_for_osdmap, request: pg_notify(22.4df(3) epoch 194476) v5 -5622> 2014-08-13 17:23:20.971235 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.971235, event: started, request: pg_notify(22.4df(3) epoch 194476) v5 -5621> 2014-08-13 17:23:20.971268 7fcd0964c700 5 -- op tracker -- , seq: 143, time: 2014-08-13 17:23:20.971268, event: done, request: pg_notify(22.4df(3) epoch 194476) v5 -5620> 2014-08-13 17:23:20.971304 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.380 192.168.1.114:6822/51844 2 ==== pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 ==== 3876+0+0 (1638134845 0 0) 0x54d55c0 con 0x56b1ce0 -5619> 2014-08-13 17:23:20.971325 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.961381, event: header_read, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5618> 2014-08-13 17:23:20.971336 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.961383, event: throttled, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5617> 2014-08-13 17:23:20.971345 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.961516, event: all_read, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5616> 2014-08-13 17:23:20.971355 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.971322, event: dispatched, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5615> 2014-08-13 17:23:20.971365 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.971365, event: waiting_for_osdmap, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5614> 2014-08-13 17:23:20.971376 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.971376, event: started, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5613> 2014-08-13 17:23:20.971447 7fcd0964c700 5 -- op tracker -- , seq: 144, time: 2014-08-13 17:23:20.971446, event: done, request: pg_notify(22.4df(22),22.4df(22) epoch 194476) v5 -5612> 2014-08-13 17:23:20.971488 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 4 ==== pg_query(24.1e6 epoch 194476) v3 ==== 144+0+0 (703799464 0 0) 0x2f905a0 con 0x4e314a0 -5611> 2014-08-13 17:23:20.971509 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.946046, event: header_read, request: pg_query(24.1e6 epoch 194476) v3 -5610> 2014-08-13 17:23:20.971542 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.946048, event: throttled, request: pg_query(24.1e6 epoch 194476) v3 -5609> 2014-08-13 17:23:20.971552 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.946226, event: all_read, request: pg_query(24.1e6 epoch 194476) v3 -5608> 2014-08-13 17:23:20.971561 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.971507, event: dispatched, request: pg_query(24.1e6 epoch 194476) v3 -5607> 2014-08-13 17:23:20.971571 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.971571, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194476) v3 -5606> 2014-08-13 17:23:20.971582 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.971582, event: started, request: pg_query(24.1e6 epoch 194476) v3 -5605> 2014-08-13 17:23:20.971606 7fcd0964c700 5 -- op tracker -- , seq: 145, time: 2014-08-13 17:23:20.971606, event: done, request: pg_query(24.1e6 epoch 194476) v3 -5604> 2014-08-13 17:23:20.971630 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 4 ==== pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 ==== 4055+0+0 (3399392175 0 0) 0x56df200 con 0x4e35ac0 -5603> 2014-08-13 17:23:20.971652 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.968280, event: header_read, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5602> 2014-08-13 17:23:20.971664 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.968282, event: throttled, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5601> 2014-08-13 17:23:20.971675 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.968440, event: all_read, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5600> 2014-08-13 17:23:20.971668 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(26) epoch 194476) v5 -- ?+0 0x56af380 con 0x4e314a0 -5599> 2014-08-13 17:23:20.971686 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.971650, event: dispatched, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5598> 2014-08-13 17:23:20.971700 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.971700, event: waiting_for_osdmap, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5597> 2014-08-13 17:23:20.971733 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.971733, event: started, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5596> 2014-08-13 17:23:20.971756 7fcd0964c700 5 -- op tracker -- , seq: 146, time: 2014-08-13 17:23:20.971756, event: done, request: pg_log(22.1d2 epoch 194476 log log((194215'510,194215'510], crt=194215'510) query_epoch 194476) v4 -5595> 2014-08-13 17:23:20.971785 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 4 ==== pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 ==== 5227+0+0 (2330889007 0 0) 0x56dd400 con 0x4e35800 -5594> 2014-08-13 17:23:20.971782 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.005515 6 0.000214 -5593> 2014-08-13 17:23:20.971804 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.967329, event: header_read, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5592> 2014-08-13 17:23:20.971813 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.967331, event: throttled, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5591> 2014-08-13 17:23:20.971811 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5590> 2014-08-13 17:23:20.971825 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.967493, event: all_read, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5589> 2014-08-13 17:23:20.971837 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.971802, event: dispatched, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5588> 2014-08-13 17:23:20.971849 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.971849, event: waiting_for_osdmap, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5587> 2014-08-13 17:23:20.971861 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.971861, event: started, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5586> 2014-08-13 17:23:20.971896 7fcd0964c700 5 -- op tracker -- , seq: 147, time: 2014-08-13 17:23:20.971895, event: done, request: pg_log(22.22c epoch 194476 log log((194215'514,194215'514], crt=194215'514) query_epoch 194476) v4 -5585> 2014-08-13 17:23:20.971921 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 3 ==== pg_notify(22.1b8(1) epoch 194476) v5 ==== 769+0+0 (2424558164 0 0) 0x56ab480 con 0x4e35540 -5584> 2014-08-13 17:23:20.971940 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.957874, event: header_read, request: pg_notify(22.1b8(1) epoch 194476) v5 -5583> 2014-08-13 17:23:20.971923 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.005986 7 0.000358 -5582> 2014-08-13 17:23:20.971955 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.957875, event: throttled, request: pg_notify(22.1b8(1) epoch 194476) v5 -5581> 2014-08-13 17:23:20.971961 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.957980, event: all_read, request: pg_notify(22.1b8(1) epoch 194476) v5 -5580> 2014-08-13 17:23:20.971971 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.971938, event: dispatched, request: pg_notify(22.1b8(1) epoch 194476) v5 -5579> 2014-08-13 17:23:20.971966 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5578> 2014-08-13 17:23:20.971980 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.971980, event: waiting_for_osdmap, request: pg_notify(22.1b8(1) epoch 194476) v5 -5577> 2014-08-13 17:23:20.972002 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.972002, event: started, request: pg_notify(22.1b8(1) epoch 194476) v5 -5576> 2014-08-13 17:23:20.972032 7fcd0964c700 5 -- op tracker -- , seq: 148, time: 2014-08-13 17:23:20.972032, event: done, request: pg_notify(22.1b8(1) epoch 194476) v5 -5575> 2014-08-13 17:23:20.972086 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 6 ==== pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 ==== 3917+0+0 (1661620473 0 0) 0x56dc800 con 0x4ec09a0 -5574> 2014-08-13 17:23:20.972109 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.964901, event: header_read, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5573> 2014-08-13 17:23:20.972121 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.964902, event: throttled, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5572> 2014-08-13 17:23:20.972129 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.965035, event: all_read, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5571> 2014-08-13 17:23:20.972136 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.972106, event: dispatched, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5570> 2014-08-13 17:23:20.972144 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.972144, event: waiting_for_osdmap, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5569> 2014-08-13 17:23:20.972165 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.972165, event: started, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5568> 2014-08-13 17:23:20.972185 7fcd0964c700 5 -- op tracker -- , seq: 149, time: 2014-08-13 17:23:20.972185, event: done, request: pg_log(22.113 epoch 194476 log log((194215'528,194215'528], crt=194215'528) query_epoch 194476) v4 -5567> 2014-08-13 17:23:20.972231 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 4 ==== pg_query(24.33a epoch 194476) v3 ==== 144+0+0 (3784067404 0 0) 0x54faee0 con 0x4ec1ce0 -5566> 2014-08-13 17:23:20.972227 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.008606 9 0.000450 -5565> 2014-08-13 17:23:20.972258 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.947878, event: header_read, request: pg_query(24.33a epoch 194476) v3 -5564> 2014-08-13 17:23:20.972269 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.947879, event: throttled, request: pg_query(24.33a epoch 194476) v3 -5563> 2014-08-13 17:23:20.972260 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5562> 2014-08-13 17:23:20.972282 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.947968, event: all_read, request: pg_query(24.33a epoch 194476) v3 -5561> 2014-08-13 17:23:20.972302 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.972255, event: dispatched, request: pg_query(24.33a epoch 194476) v3 -5560> 2014-08-13 17:23:20.972313 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.972313, event: waiting_for_osdmap, request: pg_query(24.33a epoch 194476) v3 -5559> 2014-08-13 17:23:20.972324 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.972324, event: started, request: pg_query(24.33a epoch 194476) v3 -5558> 2014-08-13 17:23:20.972371 7fcd0964c700 5 -- op tracker -- , seq: 150, time: 2014-08-13 17:23:20.972370, event: done, request: pg_query(24.33a epoch 194476) v3 -5557> 2014-08-13 17:23:20.972393 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 5 ==== pg_query(22.48e epoch 194476) v3 ==== 144+0+0 (749060399 0 0) 0x2f97da0 con 0x4ec3440 -5556> 2014-08-13 17:23:20.972412 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.915666, event: header_read, request: pg_query(22.48e epoch 194476) v3 -5555> 2014-08-13 17:23:20.972422 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.915668, event: throttled, request: pg_query(22.48e epoch 194476) v3 -5554> 2014-08-13 17:23:20.972431 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.915732, event: all_read, request: pg_query(22.48e epoch 194476) v3 -5553> 2014-08-13 17:23:20.972422 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_log(24.33a epoch 194476 log log((194422'16034,194422'16034], crt=194422'16034) query_epoch 194476) v4 -- ?+0 0x56df200 con 0x4ec1ce0 -5552> 2014-08-13 17:23:20.972449 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.972410, event: dispatched, request: pg_query(22.48e epoch 194476) v3 -5551> 2014-08-13 17:23:20.972471 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.972471, event: waiting_for_osdmap, request: pg_query(22.48e epoch 194476) v3 -5550> 2014-08-13 17:23:20.972480 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.972480, event: started, request: pg_query(22.48e epoch 194476) v3 -5549> 2014-08-13 17:23:20.972499 7fcd0964c700 5 -- op tracker -- , seq: 151, time: 2014-08-13 17:23:20.972499, event: done, request: pg_query(22.48e epoch 194476) v3 -5548> 2014-08-13 17:23:20.972524 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 6 ==== pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 ==== 4878+0+0 (1584891814 0 0) 0x56dda00 con 0x4ec4780 -5547> 2014-08-13 17:23:20.972542 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.971798, event: header_read, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5546> 2014-08-13 17:23:20.972551 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.971800, event: throttled, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5545> 2014-08-13 17:23:20.972587 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.972026, event: all_read, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5544> 2014-08-13 17:23:20.972594 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.972540, event: dispatched, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5543> 2014-08-13 17:23:20.972602 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.972602, event: waiting_for_osdmap, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5542> 2014-08-13 17:23:20.972613 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.972613, event: started, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5541> 2014-08-13 17:23:20.972629 7fcd0964c700 5 -- op tracker -- , seq: 152, time: 2014-08-13 17:23:20.972629, event: done, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5540> 2014-08-13 17:23:20.972642 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_notify(22.48e(24) epoch 194476) v5 -- ?+0 0x56a81c0 con 0x4ec3440 -5539> 2014-08-13 17:23:20.972652 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 5 ==== pg_query(24.206 epoch 194476) v3 ==== 144+0+0 (532809369 0 0) 0x2f95fa0 con 0x4f86e00 -5538> 2014-08-13 17:23:20.972668 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.964382, event: header_read, request: pg_query(24.206 epoch 194476) v3 -5537> 2014-08-13 17:23:20.972680 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.964384, event: throttled, request: pg_query(24.206 epoch 194476) v3 -5536> 2014-08-13 17:23:20.972688 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.964438, event: all_read, request: pg_query(24.206 epoch 194476) v3 -5535> 2014-08-13 17:23:20.972696 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.972666, event: dispatched, request: pg_query(24.206 epoch 194476) v3 -5534> 2014-08-13 17:23:20.972706 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.972706, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194476) v3 -5533> 2014-08-13 17:23:20.972717 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.972716, event: started, request: pg_query(24.206 epoch 194476) v3 -5532> 2014-08-13 17:23:20.972767 7fcd0964c700 5 -- op tracker -- , seq: 153, time: 2014-08-13 17:23:20.972767, event: done, request: pg_query(24.206 epoch 194476) v3 -5531> 2014-08-13 17:23:20.972795 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x567af40 con 0x4fda520 -5530> 2014-08-13 17:23:20.972801 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_log(24.206 epoch 194476 log log((194422'14476,194422'14476], crt=194422'14476) query_epoch 194476) v4 -- ?+0 0x56dda00 con 0x4f86e00 -5529> 2014-08-13 17:23:20.972819 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -5528> 2014-08-13 17:23:20.972834 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 4 ==== pg_query(24.cd epoch 194476) v3 ==== 144+0+0 (1293378317 0 0) 0x2f90960 con 0x4fd0f20 -5527> 2014-08-13 17:23:20.972875 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.970707, event: header_read, request: pg_query(24.cd epoch 194476) v3 -5526> 2014-08-13 17:23:20.972887 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.970708, event: throttled, request: pg_query(24.cd epoch 194476) v3 -5525> 2014-08-13 17:23:20.972896 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.970762, event: all_read, request: pg_query(24.cd epoch 194476) v3 -5524> 2014-08-13 17:23:20.972905 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.972871, event: dispatched, request: pg_query(24.cd epoch 194476) v3 -5523> 2014-08-13 17:23:20.972914 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.972914, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194476) v3 -5522> 2014-08-13 17:23:20.972926 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.972925, event: started, request: pg_query(24.cd epoch 194476) v3 -5521> 2014-08-13 17:23:20.972951 7fcd0964c700 5 -- op tracker -- , seq: 154, time: 2014-08-13 17:23:20.972951, event: done, request: pg_query(24.cd epoch 194476) v3 -5520> 2014-08-13 17:23:20.972987 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194476 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194476) v4 -- ?+0 0x56dc800 con 0x4fd0f20 -5519> 2014-08-13 17:23:20.972979 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 3 ==== pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 ==== 4878+0+0 (3367018593 0 0) 0x58c9800 con 0x56b2260 -5518> 2014-08-13 17:23:20.973014 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.972383, event: header_read, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5517> 2014-08-13 17:23:20.973028 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.972386, event: throttled, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5516> 2014-08-13 17:23:20.973049 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.972530, event: all_read, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5515> 2014-08-13 17:23:20.973061 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.973011, event: dispatched, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5514> 2014-08-13 17:23:20.973072 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.973072, event: waiting_for_osdmap, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5513> 2014-08-13 17:23:20.973085 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.973085, event: started, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5512> 2014-08-13 17:23:20.973110 7fcd0964c700 5 -- op tracker -- , seq: 155, time: 2014-08-13 17:23:20.973110, event: done, request: pg_log(22.4df epoch 194476 log log((194215'497,194215'497], crt=194215'497) query_epoch 194476) v4 -5511> 2014-08-13 17:23:20.973133 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.002130 7 0.000229 -5510> 2014-08-13 17:23:20.973135 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 5 ==== pg_query(24.1e6 epoch 194476) v3 ==== 144+0+0 (1583904161 0 0) 0x2f979e0 con 0x4e314a0 -5509> 2014-08-13 17:23:20.973168 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.964074, event: header_read, request: pg_query(24.1e6 epoch 194476) v3 -5508> 2014-08-13 17:23:20.973167 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5507> 2014-08-13 17:23:20.973180 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.964076, event: throttled, request: pg_query(24.1e6 epoch 194476) v3 -5506> 2014-08-13 17:23:20.973189 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.964139, event: all_read, request: pg_query(24.1e6 epoch 194476) v3 -5505> 2014-08-13 17:23:20.973198 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.973165, event: dispatched, request: pg_query(24.1e6 epoch 194476) v3 -5504> 2014-08-13 17:23:20.973208 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.973207, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194476) v3 -5503> 2014-08-13 17:23:20.973219 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.973218, event: started, request: pg_query(24.1e6 epoch 194476) v3 -5502> 2014-08-13 17:23:20.973249 7fcd0964c700 5 -- op tracker -- , seq: 156, time: 2014-08-13 17:23:20.973249, event: done, request: pg_query(24.1e6 epoch 194476) v3 -5501> 2014-08-13 17:23:20.973280 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 4 ==== pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 ==== 1494+0+0 (2118873768 0 0) 0x56ab2c0 con 0x4e35540 -5500> 2014-08-13 17:23:20.973290 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_log(24.1e6 epoch 194476 log log((194422'17642,194422'17642], crt=194422'17642) query_epoch 194476) v4 -- ?+0 0x56dd400 con 0x4e314a0 -5499> 2014-08-13 17:23:20.973301 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.958105, event: header_read, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5498> 2014-08-13 17:23:20.973325 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.958107, event: throttled, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5497> 2014-08-13 17:23:20.973332 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.958271, event: all_read, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5496> 2014-08-13 17:23:20.973338 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.973298, event: dispatched, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5495> 2014-08-13 17:23:20.973345 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.973345, event: waiting_for_osdmap, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5494> 2014-08-13 17:23:20.973354 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.973354, event: started, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5493> 2014-08-13 17:23:20.973392 7fcd0964c700 5 -- op tracker -- , seq: 157, time: 2014-08-13 17:23:20.973392, event: done, request: pg_notify(1.2b(1),2.2a(1) epoch 194476) v5 -5492> 2014-08-13 17:23:20.973417 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 5 ==== pg_notify(22.84(22) epoch 194476) v5 ==== 1948+0+0 (4138702873 0 0) 0x54d1a40 con 0x4ec1ce0 -5491> 2014-08-13 17:23:20.973433 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.949733, event: header_read, request: pg_notify(22.84(22) epoch 194476) v5 -5490> 2014-08-13 17:23:20.973440 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.949735, event: throttled, request: pg_notify(22.84(22) epoch 194476) v5 -5489> 2014-08-13 17:23:20.973445 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.949835, event: all_read, request: pg_notify(22.84(22) epoch 194476) v5 -5488> 2014-08-13 17:23:20.973451 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.973431, event: dispatched, request: pg_notify(22.84(22) epoch 194476) v5 -5487> 2014-08-13 17:23:20.973457 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.973457, event: waiting_for_osdmap, request: pg_notify(22.84(22) epoch 194476) v5 -5486> 2014-08-13 17:23:20.973464 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.973464, event: started, request: pg_notify(22.84(22) epoch 194476) v5 -5485> 2014-08-13 17:23:20.973483 7fcd0964c700 5 -- op tracker -- , seq: 158, time: 2014-08-13 17:23:20.973483, event: done, request: pg_notify(22.84(22) epoch 194476) v5 -5484> 2014-08-13 17:23:20.973503 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 6 ==== pg_query(24.5e epoch 194476) v3 ==== 144+0+0 (76668405 0 0) 0x2f97bc0 con 0x4ec3440 -5483> 2014-08-13 17:23:20.973544 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.950412, event: header_read, request: pg_query(24.5e epoch 194476) v3 -5482> 2014-08-13 17:23:20.973556 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.950413, event: throttled, request: pg_query(24.5e epoch 194476) v3 -5481> 2014-08-13 17:23:20.973572 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.950479, event: all_read, request: pg_query(24.5e epoch 194476) v3 -5480> 2014-08-13 17:23:20.973581 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.973540, event: dispatched, request: pg_query(24.5e epoch 194476) v3 -5479> 2014-08-13 17:23:20.973590 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.973590, event: waiting_for_osdmap, request: pg_query(24.5e epoch 194476) v3 -5478> 2014-08-13 17:23:20.973601 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.973601, event: started, request: pg_query(24.5e epoch 194476) v3 -5477> 2014-08-13 17:23:20.973626 7fcd0964c700 5 -- op tracker -- , seq: 159, time: 2014-08-13 17:23:20.973625, event: done, request: pg_query(24.5e epoch 194476) v3 -5476> 2014-08-13 17:23:20.973653 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 2 ==== pg_query(22.537 epoch 194476) v3 ==== 144+0+0 (26773357 0 0) 0x2f95dc0 con 0x4fda520 -5475> 2014-08-13 17:23:20.973672 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.970498, event: header_read, request: pg_query(22.537 epoch 194476) v3 -5474> 2014-08-13 17:23:20.973682 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.970500, event: throttled, request: pg_query(22.537 epoch 194476) v3 -5473> 2014-08-13 17:23:20.973690 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.970545, event: all_read, request: pg_query(22.537 epoch 194476) v3 -5472> 2014-08-13 17:23:20.973699 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.973670, event: dispatched, request: pg_query(22.537 epoch 194476) v3 -5471> 2014-08-13 17:23:20.973708 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.973708, event: waiting_for_osdmap, request: pg_query(22.537 epoch 194476) v3 -5470> 2014-08-13 17:23:20.973697 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_log(24.5e epoch 194476 log log((194422'17034,194422'17034], crt=194422'17034) query_epoch 194476) v4 -- ?+0 0x58c9800 con 0x4ec3440 -5469> 2014-08-13 17:23:20.973723 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.973722, event: started, request: pg_query(22.537 epoch 194476) v3 -5468> 2014-08-13 17:23:20.973748 7fcd0964c700 5 -- op tracker -- , seq: 160, time: 2014-08-13 17:23:20.973748, event: done, request: pg_query(22.537 epoch 194476) v3 -5467> 2014-08-13 17:23:20.973776 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 6 ==== pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 ==== 4176+0+0 (1520273933 0 0) 0x58c8000 con 0x4e314a0 -5466> 2014-08-13 17:23:20.973797 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.968532, event: header_read, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5465> 2014-08-13 17:23:20.973809 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.968534, event: throttled, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5464> 2014-08-13 17:23:20.973819 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.968726, event: all_read, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5463> 2014-08-13 17:23:20.973831 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.973795, event: dispatched, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5462> 2014-08-13 17:23:20.973843 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.973842, event: waiting_for_osdmap, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5461> 2014-08-13 17:23:20.973857 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.973856, event: started, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5460> 2014-08-13 17:23:20.973876 7fcd0964c700 5 -- op tracker -- , seq: 161, time: 2014-08-13 17:23:20.973876, event: done, request: pg_log(22.f3 epoch 194476 log log((194215'506,194215'506], crt=194215'506) query_epoch 194476) v4 -5459> 2014-08-13 17:23:20.973901 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 5 ==== pg_notify(22.1b8(1) epoch 194476) v5 ==== 769+0+0 (2424558164 0 0) 0x56af1c0 con 0x4e35540 -5458> 2014-08-13 17:23:20.973906 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(22) epoch 194476) v5 -- ?+0 0x56a9880 con 0x4fda520 -5457> 2014-08-13 17:23:20.973920 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.964241, event: header_read, request: pg_notify(22.1b8(1) epoch 194476) v5 -5456> 2014-08-13 17:23:20.973920 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.007338 7 0.000218 -5455> 2014-08-13 17:23:20.973952 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.964243, event: throttled, request: pg_notify(22.1b8(1) epoch 194476) v5 -5454> 2014-08-13 17:23:20.973959 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.964323, event: all_read, request: pg_notify(22.1b8(1) epoch 194476) v5 -5453> 2014-08-13 17:23:20.973965 7fcd02841700 5 osd.4 pg_epoch: 194476 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5452> 2014-08-13 17:23:20.973978 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.973918, event: dispatched, request: pg_notify(22.1b8(1) epoch 194476) v5 -5451> 2014-08-13 17:23:20.974002 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.974002, event: waiting_for_osdmap, request: pg_notify(22.1b8(1) epoch 194476) v5 -5450> 2014-08-13 17:23:20.974015 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.974015, event: started, request: pg_notify(22.1b8(1) epoch 194476) v5 -5449> 2014-08-13 17:23:20.974043 7fcd0964c700 5 -- op tracker -- , seq: 162, time: 2014-08-13 17:23:20.974043, event: done, request: pg_notify(22.1b8(1) epoch 194476) v5 -5448> 2014-08-13 17:23:20.974072 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 7 ==== pg_notify(22.6bd(22) epoch 194476) v5 ==== 1948+0+0 (2308095656 0 0) 0x54d2d80 con 0x4ec3440 -5447> 2014-08-13 17:23:20.974093 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.954613, event: header_read, request: pg_notify(22.6bd(22) epoch 194476) v5 -5446> 2014-08-13 17:23:20.974103 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.954615, event: throttled, request: pg_notify(22.6bd(22) epoch 194476) v5 -5445> 2014-08-13 17:23:20.974120 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.954727, event: all_read, request: pg_notify(22.6bd(22) epoch 194476) v5 -5444> 2014-08-13 17:23:20.974129 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.974090, event: dispatched, request: pg_notify(22.6bd(22) epoch 194476) v5 -5443> 2014-08-13 17:23:20.974138 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.974138, event: waiting_for_osdmap, request: pg_notify(22.6bd(22) epoch 194476) v5 -5442> 2014-08-13 17:23:20.974158 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.974158, event: started, request: pg_notify(22.6bd(22) epoch 194476) v5 -5441> 2014-08-13 17:23:20.974186 7fcd0964c700 5 -- op tracker -- , seq: 163, time: 2014-08-13 17:23:20.974185, event: done, request: pg_notify(22.6bd(22) epoch 194476) v5 -5440> 2014-08-13 17:23:20.974498 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x567b600 con 0x4f853e0 -5439> 2014-08-13 17:23:20.974538 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -5438> 2014-08-13 17:23:20.974569 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.381 192.168.1.102:6822/15390 1 ==== osd_map(194473..194476 src has 185139..194476) v3 ==== 2841+0+0 (3915890033 0 0) 0x567bcc0 con 0x578c620 -5437> 2014-08-13 17:23:20.974589 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194473,194476], i have 194476, src has [185139,194476] -5436> 2014-08-13 17:23:20.974686 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 2 ==== pg_notify(22.481(41) epoch 194476) v5 ==== 3013+0+0 (1640144455 0 0) 0x4e55400 con 0x4f853e0 -5435> 2014-08-13 17:23:20.974708 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974501, event: header_read, request: pg_notify(22.481(41) epoch 194476) v5 -5434> 2014-08-13 17:23:20.974719 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974502, event: throttled, request: pg_notify(22.481(41) epoch 194476) v5 -5433> 2014-08-13 17:23:20.974730 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974664, event: all_read, request: pg_notify(22.481(41) epoch 194476) v5 -5432> 2014-08-13 17:23:20.974740 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974705, event: dispatched, request: pg_notify(22.481(41) epoch 194476) v5 -5431> 2014-08-13 17:23:20.974751 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974751, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194476) v5 -5430> 2014-08-13 17:23:20.974767 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974767, event: started, request: pg_notify(22.481(41) epoch 194476) v5 -5429> 2014-08-13 17:23:20.974799 7fcd0964c700 5 -- op tracker -- , seq: 164, time: 2014-08-13 17:23:20.974799, event: done, request: pg_notify(22.481(41) epoch 194476) v5 -5428> 2014-08-13 17:23:20.974825 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.017524 9 0.000288 -5427> 2014-08-13 17:23:20.974841 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.381 192.168.1.102:6822/15390 2 ==== pg_notify(22.481(41) epoch 194476) v5 ==== 3013+0+0 (3472876981 0 0) 0x4e50700 con 0x578c620 -5426> 2014-08-13 17:23:20.974859 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974564, event: header_read, request: pg_notify(22.481(41) epoch 194476) v5 -5425> 2014-08-13 17:23:20.974860 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5424> 2014-08-13 17:23:20.974870 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974565, event: throttled, request: pg_notify(22.481(41) epoch 194476) v5 -5423> 2014-08-13 17:23:20.974892 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974688, event: all_read, request: pg_notify(22.481(41) epoch 194476) v5 -5422> 2014-08-13 17:23:20.974903 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974856, event: dispatched, request: pg_notify(22.481(41) epoch 194476) v5 -5421> 2014-08-13 17:23:20.974913 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974913, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194476) v5 -5420> 2014-08-13 17:23:20.974928 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.974928, event: started, request: pg_notify(22.481(41) epoch 194476) v5 -5419> 2014-08-13 17:23:20.974933 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000073 0 0.000000 -5418> 2014-08-13 17:23:20.974959 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5417> 2014-08-13 17:23:20.975025 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6814/23703 -- pg_query(22.481 epoch 194476) v3 -- ?+0 0x2f90b40 con 0x5303700 -5416> 2014-08-13 17:23:20.975040 7fcd0964c700 5 -- op tracker -- , seq: 165, time: 2014-08-13 17:23:20.975040, event: done, request: pg_notify(22.481(41) epoch 194476) v5 -5415> 2014-08-13 17:23:20.975089 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 1 ==== osd_map(194472..194476 src has 185139..194476) v3 ==== 3135+0+0 (504121431 0 0) 0x567c380 con 0x4fd1a20 -5414> 2014-08-13 17:23:20.975121 7fcd0964c700 3 osd.4 194476 handle_osd_map epochs [194472,194476], i have 194476, src has [185139,194476] -5413> 2014-08-13 17:23:20.975249 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 2 ==== pg_notify(22.1b8(22) epoch 194476) v5 ==== 1948+0+0 (2078009479 0 0) 0x56aaf40 con 0x4fd1a20 -5412> 2014-08-13 17:23:20.975278 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975089, event: header_read, request: pg_notify(22.1b8(22) epoch 194476) v5 -5411> 2014-08-13 17:23:20.975292 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975091, event: throttled, request: pg_notify(22.1b8(22) epoch 194476) v5 -5410> 2014-08-13 17:23:20.975302 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975224, event: all_read, request: pg_notify(22.1b8(22) epoch 194476) v5 -5409> 2014-08-13 17:23:20.975311 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975273, event: dispatched, request: pg_notify(22.1b8(22) epoch 194476) v5 -5408> 2014-08-13 17:23:20.975327 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975327, event: waiting_for_osdmap, request: pg_notify(22.1b8(22) epoch 194476) v5 -5407> 2014-08-13 17:23:20.975340 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975340, event: started, request: pg_notify(22.1b8(22) epoch 194476) v5 -5406> 2014-08-13 17:23:20.975373 7fcd0964c700 5 -- op tracker -- , seq: 166, time: 2014-08-13 17:23:20.975373, event: done, request: pg_notify(22.1b8(22) epoch 194476) v5 -5405> 2014-08-13 17:23:20.975401 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 3 ==== pg_notify(22.481(41) epoch 194476) v5 ==== 3013+0+0 (1640144455 0 0) 0x56ab640 con 0x4f853e0 -5404> 2014-08-13 17:23:20.975422 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975015, event: header_read, request: pg_notify(22.481(41) epoch 194476) v5 -5403> 2014-08-13 17:23:20.975432 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975017, event: throttled, request: pg_notify(22.481(41) epoch 194476) v5 -5402> 2014-08-13 17:23:20.975413 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.013623 4 0.000237 -5401> 2014-08-13 17:23:20.975441 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975250, event: all_read, request: pg_notify(22.481(41) epoch 194476) v5 -5400> 2014-08-13 17:23:20.975464 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975420, event: dispatched, request: pg_notify(22.481(41) epoch 194476) v5 -5399> 2014-08-13 17:23:20.975460 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -5398> 2014-08-13 17:23:20.975475 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975475, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194476) v5 -5397> 2014-08-13 17:23:20.975487 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975487, event: started, request: pg_notify(22.481(41) epoch 194476) v5 -5396> 2014-08-13 17:23:20.975515 7fcd0964c700 5 -- op tracker -- , seq: 167, time: 2014-08-13 17:23:20.975515, event: done, request: pg_notify(22.481(41) epoch 194476) v5 -5395> 2014-08-13 17:23:20.975524 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000064 0 0.000000 -5394> 2014-08-13 17:23:20.975548 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.381 192.168.1.102:6822/15390 3 ==== pg_notify(22.481(41) epoch 194476) v5 ==== 3013+0+0 (3472876981 0 0) 0x56ad400 con 0x578c620 -5393> 2014-08-13 17:23:20.975550 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -5392> 2014-08-13 17:23:20.975573 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975200, event: header_read, request: pg_notify(22.481(41) epoch 194476) v5 -5391> 2014-08-13 17:23:20.975580 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975203, event: throttled, request: pg_notify(22.481(41) epoch 194476) v5 -5390> 2014-08-13 17:23:20.975590 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975336, event: all_read, request: pg_notify(22.481(41) epoch 194476) v5 -5389> 2014-08-13 17:23:20.975599 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975570, event: dispatched, request: pg_notify(22.481(41) epoch 194476) v5 -5388> 2014-08-13 17:23:20.975588 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000038 0 0.000000 -5387> 2014-08-13 17:23:20.975611 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975611, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194476) v5 -5386> 2014-08-13 17:23:20.975623 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975623, event: started, request: pg_notify(22.481(41) epoch 194476) v5 -5385> 2014-08-13 17:23:20.975614 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5384> 2014-08-13 17:23:20.975654 7fcd0964c700 5 -- op tracker -- , seq: 168, time: 2014-08-13 17:23:20.975653, event: done, request: pg_notify(22.481(41) epoch 194476) v5 -5383> 2014-08-13 17:23:20.976045 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 4 ==== pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 ==== 4530+0+0 (3538269406 0 0) 0x56daa00 con 0x5303700 -5382> 2014-08-13 17:23:20.976070 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.975877, event: header_read, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5381> 2014-08-13 17:23:20.976083 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.975880, event: throttled, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5380> 2014-08-13 17:23:20.976094 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.976024, event: all_read, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5379> 2014-08-13 17:23:20.976104 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.976067, event: dispatched, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5378> 2014-08-13 17:23:20.976116 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.976116, event: waiting_for_osdmap, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5377> 2014-08-13 17:23:20.976129 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.976129, event: started, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5376> 2014-08-13 17:23:20.976153 7fcd0964c700 5 -- op tracker -- , seq: 169, time: 2014-08-13 17:23:20.976153, event: done, request: pg_log(22.481 epoch 194476 log log((194215'519,194215'519], crt=194215'519) query_epoch 194476) v4 -5375> 2014-08-13 17:23:20.976188 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.001228 5 0.000200 -5374> 2014-08-13 17:23:20.976224 7fcd01e40700 5 osd.4 pg_epoch: 194476 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -5373> 2014-08-13 17:23:20.976515 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 3 ==== pg_notify(22.1b8(22) epoch 194476) v5 ==== 1948+0+0 (2078009479 0 0) 0x56a9a40 con 0x4fd1a20 -5372> 2014-08-13 17:23:20.976539 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976381, event: header_read, request: pg_notify(22.1b8(22) epoch 194476) v5 -5371> 2014-08-13 17:23:20.976550 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976383, event: throttled, request: pg_notify(22.1b8(22) epoch 194476) v5 -5370> 2014-08-13 17:23:20.976559 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976494, event: all_read, request: pg_notify(22.1b8(22) epoch 194476) v5 -5369> 2014-08-13 17:23:20.976568 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976536, event: dispatched, request: pg_notify(22.1b8(22) epoch 194476) v5 -5368> 2014-08-13 17:23:20.976577 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976577, event: waiting_for_osdmap, request: pg_notify(22.1b8(22) epoch 194476) v5 -5367> 2014-08-13 17:23:20.976589 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976589, event: started, request: pg_notify(22.1b8(22) epoch 194476) v5 -5366> 2014-08-13 17:23:20.976619 7fcd0964c700 5 -- op tracker -- , seq: 170, time: 2014-08-13 17:23:20.976619, event: done, request: pg_notify(22.1b8(22) epoch 194476) v5 -5365> 2014-08-13 17:23:20.977733 7fccd6bfc700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530e900 sd=400 :6820 s=0 pgs=0 cs=0 l=0 c=0x57f6880).accept sd=400 192.168.1.111:58731/0 -5364> 2014-08-13 17:23:20.977756 7fccd6afb700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530f300 sd=401 :6808 s=0 pgs=0 cs=0 l=0 c=0x57f70c0).accept sd=401 192.168.100.111:49048/0 -5363> 2014-08-13 17:23:21.021332 7fccd67f8700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x530e680 sd=402 :6820 s=0 pgs=0 cs=0 l=0 c=0x57f65c0).accept sd=402 192.168.1.104:46443/0 -5362> 2014-08-13 17:23:21.021399 7fccd66f7700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x530f080 sd=403 :6808 s=0 pgs=0 cs=0 l=0 c=0x57f6e00).accept sd=403 192.168.100.104:53866/0 -5361> 2014-08-13 17:23:21.083354 7fccd63f4700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5920a00 sd=404 :6820 s=0 pgs=0 cs=0 l=0 c=0x57f6300).accept sd=404 192.168.1.114:33922/0 -5360> 2014-08-13 17:23:21.083407 7fccd62f3700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5921680 sd=405 :6808 s=0 pgs=0 cs=0 l=0 c=0x57f6b40).accept sd=405 192.168.100.114:49548/0 -5359> 2014-08-13 17:23:21.084407 7fccd5ff0700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5920780 sd=406 :6820 s=0 pgs=0 cs=0 l=0 c=0x57f7d20).accept sd=406 192.168.1.106:54823/0 -5358> 2014-08-13 17:23:21.084464 7fccd5eef700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5921400 sd=407 :6808 s=0 pgs=0 cs=0 l=0 c=0x59286e0).accept sd=407 192.168.100.106:51076/0 -5357> 2014-08-13 17:23:21.091504 7fccd5bec700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5921180 sd=409 :6808 s=0 pgs=0 cs=0 l=0 c=0x5928420).accept sd=409 192.168.100.101:51713/0 -5356> 2014-08-13 17:23:21.091562 7fccd5aeb700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5920500 sd=408 :6820 s=0 pgs=0 cs=0 l=0 c=0x59298c0).accept sd=408 192.168.1.101:45760/0 -5355> 2014-08-13 17:23:21.092427 7fccd57e8700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5920280 sd=410 :6820 s=0 pgs=0 cs=0 l=0 c=0x5929600).accept sd=410 192.168.1.103:51533/0 -5354> 2014-08-13 17:23:21.092462 7fccd56e7700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5920f00 sd=411 :6808 s=0 pgs=0 cs=0 l=0 c=0x5928160).accept sd=411 192.168.100.103:48304/0 -5353> 2014-08-13 17:23:21.100776 7fccd53e4700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5920000 sd=412 :6820 s=0 pgs=0 cs=0 l=0 c=0x5929340).accept sd=412 192.168.1.103:51535/0 -5352> 2014-08-13 17:23:21.100800 7fccd52e3700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5920c80 sd=413 :6808 s=0 pgs=0 cs=0 l=0 c=0x592a940).accept sd=413 192.168.100.103:48306/0 -5351> 2014-08-13 17:23:21.179784 7fccd4fe0700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5922580 sd=414 :6820 s=0 pgs=0 cs=0 l=0 c=0x5929080).accept sd=414 192.168.1.104:46450/0 -5350> 2014-08-13 17:23:21.179815 7fccd4edf700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923480 sd=415 :6808 s=0 pgs=0 cs=0 l=0 c=0x592a680).accept sd=415 192.168.100.104:53873/0 -5349> 2014-08-13 17:23:21.187415 7fccd4bdc700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5922300 sd=416 :6820 s=0 pgs=0 cs=0 l=0 c=0x592bb20).accept sd=416 192.168.1.109:47471/0 -5348> 2014-08-13 17:23:21.187541 7fccd4adb700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923200 sd=417 :6808 s=0 pgs=0 cs=0 l=0 c=0x592a3c0).accept sd=417 192.168.100.109:55183/0 -5347> 2014-08-13 17:23:21.190858 7fccd47d8700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5922080 sd=418 :6820 s=0 pgs=0 cs=0 l=0 c=0x592b860).accept sd=418 192.168.1.114:33926/0 -5346> 2014-08-13 17:23:21.190959 7fccd46d7700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5922f80 sd=419 :6808 s=0 pgs=0 cs=0 l=0 c=0x592a100).accept sd=419 192.168.100.114:49552/0 -5345> 2014-08-13 17:23:21.191264 7fccd45d6700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5922d00 sd=420 :6808 s=0 pgs=0 cs=0 l=0 c=0x592ca40).accept sd=420 192.168.100.107:53832/0 -5344> 2014-08-13 17:23:21.191288 7fccd44d5700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5921e00 sd=421 :6820 s=0 pgs=0 cs=0 l=0 c=0x592b5a0).accept sd=421 192.168.1.107:41038/0 -5343> 2014-08-13 17:23:21.223649 7fccd3fd0700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5922a80 sd=422 :6808 s=0 pgs=0 cs=0 l=0 c=0x592c780).accept sd=422 192.168.100.107:53834/0 -5342> 2014-08-13 17:23:21.223678 7fccd3ecf700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5921b80 sd=423 :6820 s=0 pgs=0 cs=0 l=0 c=0x592b2e0).accept sd=423 192.168.1.107:41040/0 -5341> 2014-08-13 17:23:21.268820 7fccd3bcc700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5921900 sd=424 :6820 s=0 pgs=0 cs=0 l=0 c=0x592b020).accept sd=424 192.168.1.107:41042/0 -5340> 2014-08-13 17:23:21.268847 7fccd3acb700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5922800 sd=425 :6808 s=0 pgs=0 cs=0 l=0 c=0x592c4c0).accept sd=425 192.168.100.107:53836/0 -5339> 2014-08-13 17:23:21.288320 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.343 192.168.1.104:0/61960 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.288343) v2 ==== 47+0+0 (3269353023 0 0) 0x4e50e00 con 0x57f65c0 -5338> 2014-08-13 17:23:21.288413 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.288343) v2 -- ?+0 0x4e50c40 con 0x57f65c0 -5337> 2014-08-13 17:23:21.288375 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.343 192.168.1.104:0/61960 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.288343) v2 ==== 47+0+0 (3269353023 0 0) 0x4e50fc0 con 0x57f6e00 -5336> 2014-08-13 17:23:21.288465 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.288343) v2 -- ?+0 0x4e55400 con 0x57f6e00 -5335> 2014-08-13 17:23:21.291466 7fccd37c8700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5924600 sd=426 :6808 s=0 pgs=0 cs=0 l=0 c=0x592c200).accept sd=426 192.168.100.108:38583/0 -5334> 2014-08-13 17:23:21.291535 7fccd36c7700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925780 sd=427 :6820 s=0 pgs=0 cs=0 l=0 c=0x592dee0).accept sd=427 192.168.1.108:41031/0 -5333> 2014-08-13 17:23:21.304864 7fccd33c4700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925500 sd=428 :6820 s=0 pgs=0 cs=0 l=0 c=0x592dc20).accept sd=428 192.168.1.103:51540/0 -5332> 2014-08-13 17:23:21.304906 7fccd32c3700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5924380 sd=429 :6808 s=0 pgs=0 cs=0 l=0 c=0x592bf40).accept sd=429 192.168.100.103:48311/0 -5331> 2014-08-13 17:23:21.352664 7fccd2fc0700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925280 sd=430 :6820 s=0 pgs=0 cs=0 l=0 c=0x592d960).accept sd=430 192.168.1.108:41033/0 -5330> 2014-08-13 17:23:21.352713 7fccd2ebf700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5924100 sd=431 :6808 s=0 pgs=0 cs=0 l=0 c=0x592f220).accept sd=431 192.168.100.108:38585/0 -5329> 2014-08-13 17:23:21.379303 7fccd2bbc700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925000 sd=432 :6820 s=0 pgs=0 cs=0 l=0 c=0x592d6a0).accept sd=432 192.168.1.110:45991/0 -5328> 2014-08-13 17:23:21.379394 7fccd2abb700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923e80 sd=433 :6808 s=0 pgs=0 cs=0 l=0 c=0x592ef60).accept sd=433 192.168.100.110:39388/0 -5327> 2014-08-13 17:23:21.391872 7fccd27b8700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5924d80 sd=434 :6820 s=0 pgs=0 cs=0 l=0 c=0x592d3e0).accept sd=434 192.168.1.108:41035/0 -5326> 2014-08-13 17:23:21.392002 7fccd26b7700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923c00 sd=435 :6808 s=0 pgs=0 cs=0 l=0 c=0x592eca0).accept sd=435 192.168.100.108:38587/0 -5325> 2014-08-13 17:23:21.398902 7fccd23b4700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5924b00 sd=436 :6820 s=0 pgs=0 cs=0 l=0 c=0x592fd20).accept sd=436 192.168.1.111:58738/0 -5324> 2014-08-13 17:23:21.398940 7fccd22b3700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923980 sd=437 :6808 s=0 pgs=0 cs=0 l=0 c=0x592e9e0).accept sd=437 192.168.100.111:49055/0 -5323> 2014-08-13 17:23:21.440782 7fccd1fb0700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5924880 sd=438 :6820 s=0 pgs=0 cs=0 l=0 c=0x5469080).accept sd=438 192.168.1.115:44755/0 -5322> 2014-08-13 17:23:21.440853 7fccd1eaf700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5923700 sd=439 :6808 s=0 pgs=0 cs=0 l=0 c=0x592e720).accept sd=439 192.168.100.115:36477/0 -5321> 2014-08-13 17:23:21.472857 7fccd1bac700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5926b80 sd=440 :6820 s=0 pgs=0 cs=0 l=0 c=0x5468dc0).accept sd=440 192.168.1.102:54980/0 -5320> 2014-08-13 17:23:21.472952 7fccd1aab700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927d00 sd=441 :6808 s=0 pgs=0 cs=0 l=0 c=0x546a260).accept sd=441 192.168.100.102:33636/0 -5319> 2014-08-13 17:23:21.474246 7fccd17a8700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927a80 sd=442 :6808 s=0 pgs=0 cs=0 l=0 c=0x5469fa0).accept sd=442 192.168.100.107:53838/0 -5318> 2014-08-13 17:23:21.474286 7fccd16a7700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5926900 sd=443 :6820 s=0 pgs=0 cs=0 l=0 c=0x5468b00).accept sd=443 192.168.1.107:41044/0 -5317> 2014-08-13 17:23:21.503321 7fccd13a4700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927800 sd=444 :6808 s=0 pgs=0 cs=0 l=0 c=0x5469ce0).accept sd=444 192.168.100.104:53875/0 -5316> 2014-08-13 17:23:21.503411 7fccd12a3700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5926680 sd=445 :6820 s=0 pgs=0 cs=0 l=0 c=0x5468840).accept sd=445 192.168.1.104:46452/0 -5315> 2014-08-13 17:23:21.524192 7fccd0e9f700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927580 sd=447 :6808 s=0 pgs=0 cs=0 l=0 c=0x5469a20).accept sd=447 192.168.100.105:46749/0 -5314> 2014-08-13 17:23:21.524243 7fccd0fa0700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5926400 sd=446 :6820 s=0 pgs=0 cs=0 l=0 c=0x5468580).accept sd=446 192.168.1.105:57171/0 -5313> 2014-08-13 17:23:21.534628 7fccd0b9c700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927300 sd=448 :6808 s=0 pgs=0 cs=0 l=0 c=0x5469760).accept sd=448 192.168.100.111:49058/0 -5312> 2014-08-13 17:23:21.534749 7fccd0a9b700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5926180 sd=449 :6820 s=0 pgs=0 cs=0 l=0 c=0x54682c0).accept sd=449 192.168.1.111:58743/0 -5311> 2014-08-13 17:23:21.541880 7fccd0798700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925f00 sd=450 :6820 s=0 pgs=0 cs=0 l=0 c=0x546c200).accept sd=450 192.168.1.114:33928/0 -5310> 2014-08-13 17:23:21.541923 7fccd0697700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5927080 sd=451 :6808 s=0 pgs=0 cs=0 l=0 c=0x546d6a0).accept sd=451 192.168.100.114:49554/0 -5309> 2014-08-13 17:23:21.542171 7fccd0596700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925c80 sd=452 :6820 s=0 pgs=0 cs=0 l=0 c=0x546bf40).accept sd=452 192.168.1.105:57173/0 -5308> 2014-08-13 17:23:21.542334 7fccd0495700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5926e00 sd=453 :6808 s=0 pgs=0 cs=0 l=0 c=0x546d3e0).accept sd=453 192.168.100.105:46751/0 -5307> 2014-08-13 17:23:21.547651 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.376 192.168.1.108:0/6893 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.547062) v2 ==== 47+0+0 (4185598700 0 0) 0x4e50700 con 0x592f220 -5306> 2014-08-13 17:23:21.547703 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.547062) v2 -- ?+0 0x4e50fc0 con 0x592f220 -5305> 2014-08-13 17:23:21.547702 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.376 192.168.1.108:0/6893 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.547062) v2 ==== 47+0+0 (4185598700 0 0) 0x56a9c00 con 0x592d960 -5304> 2014-08-13 17:23:21.547746 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.547062) v2 -- ?+0 0x4e50e00 con 0x592d960 -5303> 2014-08-13 17:23:21.569876 7fcccff90700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5925a00 sd=454 :6820 s=0 pgs=0 cs=0 l=0 c=0x546bc80).accept sd=454 192.168.1.109:47473/0 -5302> 2014-08-13 17:23:21.569962 7fcccfe8f700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5931400 sd=455 :6808 s=0 pgs=0 cs=0 l=0 c=0x546d120).accept sd=455 192.168.100.109:55185/0 -5301> 2014-08-13 17:23:21.585435 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.321 192.168.1.110:0/43236 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.584750) v2 ==== 47+0+0 (3319785078 0 0) 0x56aa4c0 con 0x592ef60 -5300> 2014-08-13 17:23:21.585472 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.321 192.168.1.110:0/43236 1 ==== osd_ping(ping e194476 stamp 2014-08-13 17:23:21.584750) v2 ==== 47+0+0 (3319785078 0 0) 0x56abd40 con 0x592d6a0 -5299> 2014-08-13 17:23:21.585489 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.584750) v2 -- ?+0 0x4e50700 con 0x592ef60 -5298> 2014-08-13 17:23:21.585613 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.110:0/43236 -- osd_ping(ping_reply e194476 stamp 2014-08-13 17:23:21.584750) v2 -- ?+0 0x56a9c00 con 0x592d6a0 -5297> 2014-08-13 17:23:21.586033 7fcccfb8c700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5932a80 sd=456 :6820 s=0 pgs=0 cs=0 l=0 c=0x546b9c0).accept sd=456 192.168.1.106:54826/0 -5296> 2014-08-13 17:23:21.586060 7fcccfa8b700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5931180 sd=457 :6808 s=0 pgs=0 cs=0 l=0 c=0x546ce60).accept sd=457 192.168.100.106:51079/0 -5295> 2014-08-13 17:23:21.619989 7fcccf788700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5932800 sd=458 :6820 s=0 pgs=0 cs=0 l=0 c=0x546b700).accept sd=458 192.168.1.104:46454/0 -5294> 2014-08-13 17:23:21.620020 7fcccf687700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930f00 sd=459 :6808 s=0 pgs=0 cs=0 l=0 c=0x546cba0).accept sd=459 192.168.100.104:53877/0 -5293> 2014-08-13 17:23:21.632933 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 16 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x4f7f500 con 0x4e25d80 -5292> 2014-08-13 17:23:21.633056 7fcd0aa4e700 3 osd.4 194476 handle_osd_map epochs [194477,194477], i have 194476, src has [185139,194477] -5291> 2014-08-13 17:23:21.637280 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e60c0 con 0x4fd53e0 -5290> 2014-08-13 17:23:21.637586 7fcccf283700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930c80 sd=461 :6808 s=0 pgs=0 cs=0 l=0 c=0x546c8e0).accept sd=461 192.168.100.102:33639/0 -5289> 2014-08-13 17:23:21.637712 7fcccf384700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5932580 sd=460 :6820 s=0 pgs=0 cs=0 l=0 c=0x546b440).accept sd=460 192.168.1.102:54983/0 -5288> 2014-08-13 17:23:21.638233 7fcccef80700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930a00 sd=462 :6808 s=0 pgs=0 cs=0 l=0 c=0x546f900).accept sd=462 192.168.100.112:34686/0 -5287> 2014-08-13 17:23:21.638564 7fcd0aa4e700 0 osd.4 194477 crush map has features 2578087936000, adjusting msgr requires for mons -5286> 2014-08-13 17:23:21.638729 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 0.762344 4 0.000305 -5285> 2014-08-13 17:23:21.638779 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit Started 0.762499 0 0.000000 -5284> 2014-08-13 17:23:21.638807 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194476/194473) [4,286,375]/[375,10,301] r=-1 lpr=194476 pi=193640-194475/29 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter Reset -5283> 2014-08-13 17:23:21.638778 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.731087 4 0.024601 -5282> 2014-08-13 17:23:21.638839 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] exit Started 0.731242 0 0.000000 -5281> 2014-08-13 17:23:21.638865 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,58,343]/[58,343,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'415 lcod 0'0 remapped NOTIFY] enter Reset -5280> 2014-08-13 17:23:21.638905 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Reset 0.000098 1 0.000176 -5279> 2014-08-13 17:23:21.638936 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started -5278> 2014-08-13 17:23:21.638943 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000077 1 0.000164 -5277> 2014-08-13 17:23:21.638958 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Start -5276> 2014-08-13 17:23:21.638972 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started -5275> 2014-08-13 17:23:21.638982 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: transitioning to Primary -5274> 2014-08-13 17:23:21.638992 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start -5273> 2014-08-13 17:23:21.639009 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5272> 2014-08-13 17:23:21.639005 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Start 0.000047 0 0.000000 -5271> 2014-08-13 17:23:21.639030 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000038 0 0.000000 -5270> 2014-08-13 17:23:21.639038 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary -5269> 2014-08-13 17:23:21.639060 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5268> 2014-08-13 17:23:21.639064 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Peering -5267> 2014-08-13 17:23:21.639078 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5266> 2014-08-13 17:23:21.639085 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetInfo -5265> 2014-08-13 17:23:21.639101 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5264> 2014-08-13 17:23:21.639413 7fccced7e700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5932080 sd=464 :6820 s=0 pgs=0 cs=0 l=0 c=0x5948f20).accept sd=464 192.168.1.112:55022/0 -5263> 2014-08-13 17:23:21.639456 7fcd02841700 10 monclient: _send_mon_message to mon.storage0111-ib at 192.168.100.111:6789/0 -5262> 2014-08-13 17:23:21.639454 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.757741 4 0.000221 -5261> 2014-08-13 17:23:21.639475 7fcd02841700 1 -- 192.168.100.115:6805/31924 --> 192.168.100.111:6789/0 -- osd_alive(want up_thru 194477 have 194477) v1 -- ?+0 0x58888c0 con 0x4e25d80 -5260> 2014-08-13 17:23:21.639487 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] exit Started 0.757868 0 0.000000 -5259> 2014-08-13 17:23:21.639503 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_query(22.362 epoch 194477) v3 -- ?+0 0x2f96540 con 0x4e335a0 -5258> 2014-08-13 17:23:21.639507 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194476/192967) [351,4,18]/[351,18,52] r=-1 lpr=194476 pi=192970-194475/25 crt=192888'481 lcod 0'0 remapped NOTIFY] enter Reset -5257> 2014-08-13 17:23:21.639527 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_query(22.362 epoch 194477) v3 -- ?+0 0x2f97080 con 0x4e35ac0 -5256> 2014-08-13 17:23:21.639552 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- osd_map(194477..194477 src has 185139..194477) v3 -- ?+0 0x2e8b3c0 con 0x4f86e00 -5255> 2014-08-13 17:23:21.639572 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_query(22.362 epoch 194477) v3 -- ?+0 0x2f932a0 con 0x4f86e00 -5254> 2014-08-13 17:23:21.639627 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.000119 1 0.000113 -5253> 2014-08-13 17:23:21.639656 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started -5252> 2014-08-13 17:23:21.639672 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start -5251> 2014-08-13 17:23:21.639686 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -5250> 2014-08-13 17:23:21.639558 7fcccea7b700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930780 sd=466 :6808 s=0 pgs=0 cs=0 l=0 c=0x546f640).accept sd=466 192.168.100.112:34687/0 -5249> 2014-08-13 17:23:21.639704 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000032 0 0.000000 -5248> 2014-08-13 17:23:21.639723 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray -5247> 2014-08-13 17:23:21.639824 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.667858 2 0.000098 -5246> 2014-08-13 17:23:21.639843 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(26) epoch 194477) v5 -- ?+0 0x5184ec0 con 0x4fdee00 -5245> 2014-08-13 17:23:21.639509 7fcccee7f700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5932300 sd=463 :6820 s=0 pgs=0 cs=0 l=0 c=0x59491e0).accept sd=463 192.168.1.112:55021/0 -5244> 2014-08-13 17:23:21.639858 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.725076 0 0.000000 -5243> 2014-08-13 17:23:21.639876 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6815/60573 -- pg_query(24.30b epoch 194477) v3 -- ?+0 0x2f92ee0 con 0x4e30580 -5242> 2014-08-13 17:23:21.639887 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194471 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -5241> 2014-08-13 17:23:21.639895 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6820/19173 -- pg_query(24.30b epoch 194477) v3 -- ?+0 0x2f92580 con 0x4fdad60 -5240> 2014-08-13 17:23:21.639915 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6814/16727 -- pg_query(24.30b epoch 194477) v3 -- ?+0 0x2f921c0 con 0x4e35800 -5239> 2014-08-13 17:23:21.639932 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6830/52879 -- pg_query(24.30b epoch 194477) v3 -- ?+0 0x2f95820 con 0x4fd2520 -5238> 2014-08-13 17:23:21.640023 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.671612 4 0.000315 -5237> 2014-08-13 17:23:21.640051 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.692637 0 0.000000 -5236> 2014-08-13 17:23:21.640073 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -5235> 2014-08-13 17:23:21.640139 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194474/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -5234> 2014-08-13 17:23:21.640236 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -5233> 2014-08-13 17:23:21.640295 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.667128 2 0.000075 -5232> 2014-08-13 17:23:21.640322 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.692269 0 0.000000 -5231> 2014-08-13 17:23:21.640343 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194471 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -5230> 2014-08-13 17:23:21.640539 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.725140 4 0.000235 -5229> 2014-08-13 17:23:21.640572 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] exit Started 0.725253 0 0.000000 -5228> 2014-08-13 17:23:21.640592 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194476/194473) [4,65,62]/[65,62,44] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'439 lcod 0'0 remapped NOTIFY] enter Reset -5227> 2014-08-13 17:23:21.640703 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -5226> 2014-08-13 17:23:21.640798 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e14a0 -- 0x524c600 -5225> 2014-08-13 17:23:21.640845 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.691520 4 0.000250 -5224> 2014-08-13 17:23:21.640871 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] exit Started 0.691637 0 0.000000 -5223> 2014-08-13 17:23:21.640890 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/194473) [4,373,344]/[373,344,320] r=-1 lpr=194476 pi=193306-194475/24 crt=191727'416 lcod 0'0 remapped NOTIFY] enter Reset -5222> 2014-08-13 17:23:21.640925 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e11e0 -- 0x524c380 -5221> 2014-08-13 17:23:21.641101 7fcce3bcb700 2 -- 192.168.100.115:0/31924 >> 192.168.1.115:6811/59615 pipe(0x524c600 sd=293 :44962 s=4 pgs=135 cs=1 l=1 c=0x53e14a0).reader couldn't read tag, (0) Success -5220> 2014-08-13 17:23:21.641192 7fcce3bcb700 2 -- 192.168.100.115:0/31924 >> 192.168.1.115:6811/59615 pipe(0x524c600 sd=293 :44962 s=4 pgs=135 cs=1 l=1 c=0x53e14a0).fault (0) Success -5219> 2014-08-13 17:23:21.641707 7fccce071700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5931e00 sd=471 :6820 s=0 pgs=0 cs=0 l=0 c=0x5948c60).accept sd=471 192.168.1.110:45993/0 -5218> 2014-08-13 17:23:21.641889 7fcccdd6e700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5931b80 sd=475 :6820 s=0 pgs=0 cs=0 l=0 c=0x59489a0).accept sd=475 192.168.1.114:33931/0 -5217> 2014-08-13 17:23:21.641978 7fcccdf70700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930500 sd=472 :6808 s=0 pgs=0 cs=0 l=0 c=0x546f380).accept sd=472 192.168.100.110:39390/0 -5216> 2014-08-13 17:23:21.642388 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e65c0 -- 0x52c9b80 -5215> 2014-08-13 17:23:21.642476 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e6300 -- 0x52c9900 -5214> 2014-08-13 17:23:21.642531 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e35a0 -- 0x524e400 -5213> 2014-08-13 17:23:21.643009 7fccdd161700 2 -- 192.168.100.115:0/31924 >> 192.168.1.107:6814/53408 pipe(0x524e400 sd=343 :51727 s=4 pgs=128 cs=1 l=1 c=0x53e35a0).reader couldn't read tag, (0) Success -5212> 2014-08-13 17:23:21.644254 7fccdd161700 2 -- 192.168.100.115:0/31924 >> 192.168.1.107:6814/53408 pipe(0x524e400 sd=343 :51727 s=4 pgs=128 cs=1 l=1 c=0x53e35a0).fault (0) Success -5211> 2014-08-13 17:23:21.642348 7fcce34c4700 2 -- 192.168.100.115:0/31924 >> 192.168.100.115:6825/59615 pipe(0x524c380 sd=294 :51694 s=4 pgs=135 cs=1 l=1 c=0x53e11e0).reader couldn't read tag, (0) Success -5210> 2014-08-13 17:23:21.644511 7fcce34c4700 2 -- 192.168.100.115:0/31924 >> 192.168.100.115:6825/59615 pipe(0x524c380 sd=294 :51694 s=4 pgs=135 cs=1 l=1 c=0x53e11e0).fault (0) Success -5209> 2014-08-13 17:23:21.644533 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e32e0 -- 0x524e180 -5208> 2014-08-13 17:23:21.644589 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e40a0 -- 0x524ee00 -5207> 2014-08-13 17:23:21.642470 7fcce0191700 2 -- 192.168.100.115:0/31924 >> 192.168.1.103:6814/37929 pipe(0x52c9b80 sd=312 :42013 s=4 pgs=133 cs=1 l=1 c=0x53e65c0).reader couldn't read tag, (0) Success -5206> 2014-08-13 17:23:21.644613 7fcce0191700 2 -- 192.168.100.115:0/31924 >> 192.168.1.103:6814/37929 pipe(0x52c9b80 sd=312 :42013 s=4 pgs=133 cs=1 l=1 c=0x53e65c0).fault (0) Success -5205> 2014-08-13 17:23:21.644627 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e3de0 -- 0x524eb80 -5204> 2014-08-13 17:23:21.644683 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5377900 -- 0x533f080 -5203> 2014-08-13 17:23:21.644721 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5377640 -- 0x5249b80 -5202> 2014-08-13 17:23:21.644755 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5264fc0 -- 0x533a800 -5201> 2014-08-13 17:23:21.644784 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5264d00 -- 0x533a580 -5200> 2014-08-13 17:23:21.644819 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e6b40 -- 0x52ca080 -5199> 2014-08-13 17:23:21.644850 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e6880 -- 0x52c9e00 -5198> 2014-08-13 17:23:21.644875 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5264a40 -- 0x533a300 -5197> 2014-08-13 17:23:21.644899 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5264780 -- 0x533a080 -5196> 2014-08-13 17:23:21.644925 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e6040 -- 0x52c9680 -5195> 2014-08-13 17:23:21.644952 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e5d80 -- 0x52c9400 -5194> 2014-08-13 17:23:21.644982 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536dd80 -- 0x524d000 -5193> 2014-08-13 17:23:21.645009 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e1ce0 -- 0x524cd80 -5192> 2014-08-13 17:23:21.645032 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x52632e0 -- 0x5339180 -5191> 2014-08-13 17:23:21.645057 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5263020 -- 0x5338f00 -5190> 2014-08-13 17:23:21.645079 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e0f20 -- 0x524c100 -5189> 2014-08-13 17:23:21.645098 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e0c60 -- 0x524be80 -5188> 2014-08-13 17:23:21.645117 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5262d60 -- 0x5338c80 -5187> 2014-08-13 17:23:21.645139 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5265280 -- 0x533aa80 -5186> 2014-08-13 17:23:21.645158 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e4fc0 -- 0x52c8780 -5185> 2014-08-13 17:23:21.645188 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5269fa0 -- 0x52c8500 -5184> 2014-08-13 17:23:21.645209 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5269ce0 -- 0x52c8280 -5183> 2014-08-13 17:23:21.645232 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5269a20 -- 0x52c8000 -5182> 2014-08-13 17:23:21.645264 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e3b20 -- 0x524e900 -5181> 2014-08-13 17:23:21.645289 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e3860 -- 0x524e680 -5180> 2014-08-13 17:23:21.645320 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e09a0 -- 0x524fd00 -5179> 2014-08-13 17:23:21.645363 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e06e0 -- 0x524fa80 -5178> 2014-08-13 17:23:21.645303 7fcccd768700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5931900 sd=480 :6820 s=0 pgs=0 cs=0 l=0 c=0x59486e0).accept sd=480 192.168.1.102:54985/0 -5177> 2014-08-13 17:23:21.645391 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x52665c0 -- 0x5339e00 -5176> 2014-08-13 17:23:21.645425 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5266300 -- 0x5339b80 -5175> 2014-08-13 17:23:21.645452 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e0420 -- 0x524f800 -5174> 2014-08-13 17:23:21.645482 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e48e0 -- 0x524f580 -5173> 2014-08-13 17:23:21.645513 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536e880 -- 0x524da00 -5172> 2014-08-13 17:23:21.645541 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536e5c0 -- 0x524d780 -5171> 2014-08-13 17:23:21.645567 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526c780 -- 0x52cb700 -5170> 2014-08-13 17:23:21.645599 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526c4c0 -- 0x52cb480 -5169> 2014-08-13 17:23:21.645621 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536aec0 -- 0x5248f00 -5168> 2014-08-13 17:23:21.645645 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536ac00 -- 0x5248c80 -5167> 2014-08-13 17:23:21.645666 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526b180 -- 0x52cfa80 -5166> 2014-08-13 17:23:21.645697 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526aec0 -- 0x52cf800 -5165> 2014-08-13 17:23:21.645727 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526cd00 -- 0x52cbc00 -5164> 2014-08-13 17:23:21.645756 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526ca40 -- 0x52cb980 -5163> 2014-08-13 17:23:21.645776 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536f380 -- 0x524a080 -5162> 2014-08-13 17:23:21.645810 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536f0c0 -- 0x5249e00 -5161> 2014-08-13 17:23:21.645844 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e1a20 -- 0x524cb00 -5160> 2014-08-13 17:23:21.645876 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53e1760 -- 0x524c880 -5159> 2014-08-13 17:23:21.645907 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536a940 -- 0x5248a00 -5158> 2014-08-13 17:23:21.645857 7fcce4cdc700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6837/6757 pipe(0x5249e00 sd=282 :45237 s=4 pgs=158 cs=1 l=1 c=0x536f0c0).reader couldn't read tag, (0) Success -5157> 2014-08-13 17:23:21.645938 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536a680 -- 0x5248780 -5156> 2014-08-13 17:23:21.645938 7fcce4cdc700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6837/6757 pipe(0x5249e00 sd=282 :45237 s=4 pgs=158 cs=1 l=1 c=0x536f0c0).fault (0) Success -5155> 2014-08-13 17:23:21.645972 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536e300 -- 0x524d500 -5154> 2014-08-13 17:23:21.645907 7fcce3dcd700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6844/6227 pipe(0x524c880 sd=292 :49822 s=4 pgs=114 cs=1 l=1 c=0x53e1760).reader couldn't read tag, (0) Success -5153> 2014-08-13 17:23:21.645985 7fcce3dcd700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6844/6227 pipe(0x524c880 sd=292 :49822 s=4 pgs=114 cs=1 l=1 c=0x53e1760).fault (0) Success -5152> 2014-08-13 17:23:21.645998 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x536e040 -- 0x524d280 -5151> 2014-08-13 17:23:21.646039 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x5377380 -- 0x5249900 -5150> 2014-08-13 17:23:21.645986 7fcce6efe700 2 -- 192.168.100.115:0/31924 >> 192.168.100.102:6847/20017 pipe(0x5248780 sd=266 :49225 s=4 pgs=154 cs=1 l=1 c=0x536a680).reader couldn't read tag, (0) Success -5149> 2014-08-13 17:23:21.646067 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x53770c0 -- 0x5249680 -5148> 2014-08-13 17:23:21.646061 7fcce6efe700 2 -- 192.168.100.115:0/31924 >> 192.168.100.102:6847/20017 pipe(0x5248780 sd=266 :49225 s=4 pgs=154 cs=1 l=1 c=0x536a680).fault (0) Success -5147> 2014-08-13 17:23:21.646089 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526c200 -- 0x52cb200 -5146> 2014-08-13 17:23:21.646111 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526bf40 -- 0x52caf80 -5145> 2014-08-13 17:23:21.646131 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526f220 -- 0x52cf580 -5144> 2014-08-13 17:23:21.646151 7fcd0aa4e700 1 -- 192.168.100.115:0/31924 mark_down 0x526ef60 -- 0x52cf300 -5143> 2014-08-13 17:23:21.642526 7fcce0393700 2 -- 192.168.100.115:0/31924 >> 192.168.100.103:6821/37929 pipe(0x52c9900 sd=313 :56170 s=4 pgs=133 cs=1 l=1 c=0x53e6300).reader couldn't read tag, (0) Success -5142> 2014-08-13 17:23:21.646117 7fccdda6a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6839/6893 pipe(0x52cb200 sd=336 :52797 s=4 pgs=194 cs=1 l=1 c=0x526c200).reader couldn't read tag, (0) Success -5141> 2014-08-13 17:23:21.646169 7fcce0393700 2 -- 192.168.100.115:0/31924 >> 192.168.100.103:6821/37929 pipe(0x52c9900 sd=313 :56170 s=4 pgs=133 cs=1 l=1 c=0x53e6300).fault (0) Success -5140> 2014-08-13 17:23:21.646177 7fccdda6a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6839/6893 pipe(0x52cb200 sd=336 :52797 s=4 pgs=194 cs=1 l=1 c=0x526c200).fault (0) Success -5139> 2014-08-13 17:23:21.646174 7fcce7d0c700 2 -- 192.168.100.115:0/31924 >> 192.168.1.102:6829/16129 pipe(0x5249900 sd=259 :46567 s=4 pgs=183 cs=1 l=1 c=0x5377380).reader couldn't read tag, (0) Success -5138> 2014-08-13 17:23:21.646222 7fccdc858700 2 -- 192.168.100.115:0/31924 >> 192.168.100.105:6817/41806 pipe(0x52cf300 sd=349 :56804 s=4 pgs=71 cs=1 l=1 c=0x526ef60).reader couldn't read tag, (0) Success -5137> 2014-08-13 17:23:21.646267 7fcce7d0c700 2 -- 192.168.100.115:0/31924 >> 192.168.1.102:6829/16129 pipe(0x5249900 sd=259 :46567 s=4 pgs=183 cs=1 l=1 c=0x5377380).fault (0) Success -5136> 2014-08-13 17:23:21.646272 7fccdc858700 2 -- 192.168.100.115:0/31924 >> 192.168.100.105:6817/41806 pipe(0x52cf300 sd=349 :56804 s=4 pgs=71 cs=1 l=1 c=0x526ef60).fault (0) Success -5135> 2014-08-13 17:23:21.644678 7fcce24b4700 2 -- 192.168.100.115:0/31924 >> 192.168.100.107:6838/54786 pipe(0x524eb80 sd=304 :59296 s=4 pgs=133 cs=1 l=1 c=0x53e3de0).reader couldn't read tag, (0) Success -5134> 2014-08-13 17:23:21.646334 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5133> 2014-08-13 17:23:21.645457 7fcceb645700 2 -- 192.168.100.115:0/31924 >> 192.168.100.101:6803/22303 pipe(0x5339b80 sd=217 :57348 s=4 pgs=144 cs=1 l=1 c=0x5266300).reader couldn't read tag, (0) Success -5132> 2014-08-13 17:23:21.646339 7fcce24b4700 2 -- 192.168.100.115:0/31924 >> 192.168.100.107:6838/54786 pipe(0x524eb80 sd=304 :59296 s=4 pgs=133 cs=1 l=1 c=0x53e3de0).fault (0) Success -5131> 2014-08-13 17:23:21.646349 7fcceb645700 2 -- 192.168.100.115:0/31924 >> 192.168.100.101:6803/22303 pipe(0x5339b80 sd=217 :57348 s=4 pgs=144 cs=1 l=1 c=0x5266300).fault (0) Success -5130> 2014-08-13 17:23:21.646356 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e3840 con 0x4f86e00 -5129> 2014-08-13 17:23:21.646376 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5128> 2014-08-13 17:23:21.646389 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b98c0 con 0x4fd8f20 -5127> 2014-08-13 17:23:21.646404 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5126> 2014-08-13 17:23:21.646419 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e2f40 con 0x4fd86e0 -5125> 2014-08-13 17:23:21.646434 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5124> 2014-08-13 17:23:21.646443 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.332 192.168.1.110:6806/42024 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550a640 con 0x4fdbde0 -5123> 2014-08-13 17:23:21.646424 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.005832 1 0.000128 -5122> 2014-08-13 17:23:21.644748 7fcce8211700 2 -- 192.168.100.115:0/31924 >> 192.168.100.103:6850/41157 pipe(0x5249b80 sd=258 :41675 s=4 pgs=143 cs=1 l=1 c=0x5377640).reader couldn't read tag, (0) Success -5121> 2014-08-13 17:23:21.646457 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5120> 2014-08-13 17:23:21.646434 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.005544 1 0.000112 -5119> 2014-08-13 17:23:21.646461 7fcce8211700 2 -- 192.168.100.115:0/31924 >> 192.168.100.103:6850/41157 pipe(0x5249b80 sd=258 :41675 s=4 pgs=143 cs=1 l=1 c=0x5377640).fault (0) Success -5118> 2014-08-13 17:23:21.646467 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e5a00 con 0x4fdee00 -5117> 2014-08-13 17:23:21.646459 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started -5116> 2014-08-13 17:23:21.646482 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5115> 2014-08-13 17:23:21.646472 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started -5114> 2014-08-13 17:23:21.646485 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start -5113> 2014-08-13 17:23:21.646494 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start -5112> 2014-08-13 17:23:21.646503 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5111> 2014-08-13 17:23:21.646516 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -5110> 2014-08-13 17:23:21.646527 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000041 0 0.000000 -5109> 2014-08-13 17:23:21.646536 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 -5108> 2014-08-13 17:23:21.646549 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5107> 2014-08-13 17:23:21.646547 7fcce3bcb700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5934100 sd=293 :6808 s=0 pgs=0 cs=0 l=0 c=0x546eca0).accept sd=293 192.168.100.104:53879/0 -5106> 2014-08-13 17:23:21.646558 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -5105> 2014-08-13 17:23:21.646568 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5104> 2014-08-13 17:23:21.646576 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -5103> 2014-08-13 17:23:21.646586 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5102> 2014-08-13 17:23:21.646594 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -5101> 2014-08-13 17:23:21.645338 7fcced968700 2 -- 192.168.100.115:0/31924 >> 192.168.100.112:6810/5505 pipe(0x5338f00 sd=208 :45745 s=4 pgs=187 cs=1 l=1 c=0x5263020).reader couldn't read tag, (0) Success -5100> 2014-08-13 17:23:21.646638 7fcced968700 2 -- 192.168.100.115:0/31924 >> 192.168.100.112:6810/5505 pipe(0x5338f00 sd=208 :45745 s=4 pgs=187 cs=1 l=1 c=0x5263020).fault (0) Success -5099> 2014-08-13 17:23:21.644874 7fcce12a2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6815/12102 pipe(0x52c9e00 sd=311 :49844 s=4 pgs=201 cs=1 l=1 c=0x53e6880).reader couldn't read tag, (0) Success -5098> 2014-08-13 17:23:21.646735 7fcce12a2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6815/12102 pipe(0x52c9e00 sd=311 :49844 s=4 pgs=201 cs=1 l=1 c=0x53e6880).fault (0) Success -5097> 2014-08-13 17:23:21.644820 7fccecf5e700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6802/10782 pipe(0x533a580 sd=213 :49747 s=4 pgs=190 cs=1 l=1 c=0x5264d00).reader couldn't read tag, (0) Success -5096> 2014-08-13 17:23:21.644932 7fccecb5a700 2 -- 192.168.100.115:0/31924 >> 192.168.100.112:6842/8300 pipe(0x533a080 sd=215 :50448 s=4 pgs=131 cs=1 l=1 c=0x5264780).reader couldn't read tag, (0) Success -5095> 2014-08-13 17:23:21.646871 7fccecf5e700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6802/10782 pipe(0x533a580 sd=213 :49747 s=4 pgs=190 cs=1 l=1 c=0x5264d00).fault (0) Success -5094> 2014-08-13 17:23:21.646931 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b9680 con 0x4fde300 -5093> 2014-08-13 17:23:21.645211 7fcce42d2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6811/11446 pipe(0x524cd80 sd=290 :55494 s=4 pgs=170 cs=1 l=1 c=0x53e1ce0).reader couldn't read tag, (0) Success -5092> 2014-08-13 17:23:21.646953 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5091> 2014-08-13 17:23:21.644997 7fcce0999700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6833/14923 pipe(0x52c9400 sd=315 :39373 s=4 pgs=186 cs=1 l=1 c=0x53e5d80).reader couldn't read tag, (0) Success -5090> 2014-08-13 17:23:21.646955 7fcce42d2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6811/11446 pipe(0x524cd80 sd=290 :55494 s=4 pgs=170 cs=1 l=1 c=0x53e1ce0).fault (0) Success -5089> 2014-08-13 17:23:21.646963 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e3180 con 0x4fd3860 -5088> 2014-08-13 17:23:21.646962 7fcce0999700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6833/14923 pipe(0x52c9400 sd=315 :39373 s=4 pgs=186 cs=1 l=1 c=0x53e5d80).fault (0) Success -5087> 2014-08-13 17:23:21.646984 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5086> 2014-08-13 17:23:21.647004 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bb840 con 0x4fd0f20 -5085> 2014-08-13 17:23:21.645214 7fcced362700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6843/17345 pipe(0x5338c80 sd=209 :56241 s=4 pgs=199 cs=1 l=1 c=0x5262d60).reader couldn't read tag, (0) Success -5084> 2014-08-13 17:23:21.647046 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5083> 2014-08-13 17:23:21.647045 7fcced362700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6843/17345 pipe(0x5338c80 sd=209 :56241 s=4 pgs=199 cs=1 l=1 c=0x5262d60).fault (0) Success -5082> 2014-08-13 17:23:21.647059 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bc380 con 0x4fd5960 -5081> 2014-08-13 17:23:21.647082 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5080> 2014-08-13 17:23:21.647098 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e5580 con 0x4fd32e0 -5079> 2014-08-13 17:23:21.647124 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5078> 2014-08-13 17:23:21.647136 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e7500 con 0x4fd3020 -5077> 2014-08-13 17:23:21.647152 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5076> 2014-08-13 17:23:21.647173 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 4 ==== pg_notify(22.3d9(13) epoch 194477) v5 ==== 1453+0+0 (2371477594 0 0) 0x56a9dc0 con 0x4fd53e0 -5075> 2014-08-13 17:23:21.647192 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.637143, event: header_read, request: pg_notify(22.3d9(13) epoch 194477) v5 -5074> 2014-08-13 17:23:21.647204 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.637145, event: throttled, request: pg_notify(22.3d9(13) epoch 194477) v5 -5073> 2014-08-13 17:23:21.647211 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.637264, event: all_read, request: pg_notify(22.3d9(13) epoch 194477) v5 -5072> 2014-08-13 17:23:21.645235 7fccde777700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6807/31286 pipe(0x52c8780 sd=323 :36432 s=4 pgs=107 cs=1 l=1 c=0x53e4fc0).reader couldn't read tag, (0) Success -5071> 2014-08-13 17:23:21.647217 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.647187, event: dispatched, request: pg_notify(22.3d9(13) epoch 194477) v5 -5070> 2014-08-13 17:23:21.647225 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.647225, event: waiting_for_osdmap, request: pg_notify(22.3d9(13) epoch 194477) v5 -5069> 2014-08-13 17:23:21.647219 7fccde777700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6807/31286 pipe(0x52c8780 sd=323 :36432 s=4 pgs=107 cs=1 l=1 c=0x53e4fc0).fault (0) Success -5068> 2014-08-13 17:23:21.647235 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.647234, event: started, request: pg_notify(22.3d9(13) epoch 194477) v5 -5067> 2014-08-13 17:23:21.644578 7fcce1aaa700 2 -- 192.168.100.115:0/31924 >> 192.168.100.107:6819/53408 pipe(0x524e180 sd=306 :57164 s=4 pgs=128 cs=1 l=1 c=0x53e32e0).reader couldn't read tag, (0) Success -5066> 2014-08-13 17:23:21.647246 7fcce1aaa700 2 -- 192.168.100.115:0/31924 >> 192.168.100.107:6819/53408 pipe(0x524e180 sd=306 :57164 s=4 pgs=128 cs=1 l=1 c=0x53e32e0).fault (0) Success -5065> 2014-08-13 17:23:21.647267 7fcd0964c700 5 -- op tracker -- , seq: 171, time: 2014-08-13 17:23:21.647267, event: done, request: pg_notify(22.3d9(13) epoch 194477) v5 -5064> 2014-08-13 17:23:21.645279 7fcced564700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6842/17345 pipe(0x533aa80 sd=210 :42617 s=4 pgs=199 cs=1 l=1 c=0x5265280).reader couldn't read tag, (0) Success -5063> 2014-08-13 17:23:21.647293 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e6e40 con 0x4fd2520 -5062> 2014-08-13 17:23:21.647311 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5061> 2014-08-13 17:23:21.647320 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52be780 con 0x53011e0 -5060> 2014-08-13 17:23:21.647334 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5059> 2014-08-13 17:23:21.645065 7fccedb6a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.112:6825/5505 pipe(0x5339180 sd=207 :48635 s=4 pgs=187 cs=1 l=1 c=0x52632e0).reader couldn't read tag, (0) Success -5058> 2014-08-13 17:23:21.645292 7fcce2fbf700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6820/12680 pipe(0x524be80 sd=296 :34431 s=4 pgs=204 cs=1 l=1 c=0x53e0c60).reader couldn't read tag, (0) Success -5057> 2014-08-13 17:23:21.645283 7fccdfd8d700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6811/31016 pipe(0x52c8000 sd=322 :46135 s=4 pgs=127 cs=1 l=1 c=0x5269a20).reader couldn't read tag, (0) Success -5056> 2014-08-13 17:23:21.647347 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52be9c0 con 0x5303440 -5055> 2014-08-13 17:23:21.647345 7fccedb6a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.112:6825/5505 pipe(0x5339180 sd=207 :48635 s=4 pgs=187 cs=1 l=1 c=0x52632e0).fault (0) Success -5054> 2014-08-13 17:23:21.647367 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5053> 2014-08-13 17:23:21.647358 7fcce2fbf700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6820/12680 pipe(0x524be80 sd=296 :34431 s=4 pgs=204 cs=1 l=1 c=0x53e0c60).fault (0) Success -5052> 2014-08-13 17:23:21.647361 7fccdfd8d700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6811/31016 pipe(0x52c8000 sd=322 :46135 s=4 pgs=127 cs=1 l=1 c=0x5269a20).fault (0) Success -5051> 2014-08-13 17:23:21.647384 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 7 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550d580 con 0x4e314a0 -5050> 2014-08-13 17:23:21.647398 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5049> 2014-08-13 17:23:21.647406 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e0b40 con 0x4e335a0 -5048> 2014-08-13 17:23:21.647298 7fcced564700 2 -- 192.168.100.115:0/31924 >> 192.168.100.111:6842/17345 pipe(0x533aa80 sd=210 :42617 s=4 pgs=199 cs=1 l=1 c=0x5265280).fault (0) Success -5047> 2014-08-13 17:23:21.645312 7fcce22b2700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6813/32413 pipe(0x524e900 sd=305 :33694 s=4 pgs=152 cs=1 l=1 c=0x53e3b20).reader couldn't read tag, (0) Success -5046> 2014-08-13 17:23:21.647420 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5045> 2014-08-13 17:23:21.647401 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.680516 6 0.000297 -5044> 2014-08-13 17:23:21.647424 7fcce22b2700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6813/32413 pipe(0x524e900 sd=305 :33694 s=4 pgs=152 cs=1 l=1 c=0x53e3b20).fault (0) Success -5043> 2014-08-13 17:23:21.647429 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bf2c0 con 0x4e32940 -5042> 2014-08-13 17:23:21.647443 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5041> 2014-08-13 17:23:21.647439 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 0.739611 0 0.000000 -5040> 2014-08-13 17:23:21.647451 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5679b00 con 0x4e34780 -5039> 2014-08-13 17:23:21.647466 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5038> 2014-08-13 17:23:21.645530 7fcce2cbc700 2 -- 192.168.100.115:0/31924 >> 192.168.100.110:6840/43857 pipe(0x524f580 sd=300 :52226 s=4 pgs=147 cs=1 l=1 c=0x53e48e0).reader couldn't read tag, (0) Success -5037> 2014-08-13 17:23:21.647475 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5509680 con 0x4e35120 -5036> 2014-08-13 17:23:21.647469 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194472 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -5035> 2014-08-13 17:23:21.647478 7fcce2cbc700 2 -- 192.168.100.115:0/31924 >> 192.168.100.110:6840/43857 pipe(0x524f580 sd=300 :52226 s=4 pgs=147 cs=1 l=1 c=0x53e48e0).fault (0) Success -5034> 2014-08-13 17:23:21.647490 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5033> 2014-08-13 17:23:21.647501 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 7 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b8b40 con 0x4ec09a0 -5032> 2014-08-13 17:23:21.647517 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5031> 2014-08-13 17:23:21.647525 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550bf00 con 0x4ec1ce0 -5030> 2014-08-13 17:23:21.645353 7fcce1dad700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6833/32413 pipe(0x524e680 sd=309 :43937 s=4 pgs=152 cs=1 l=1 c=0x53e3860).reader couldn't read tag, (0) Success -5029> 2014-08-13 17:23:21.647539 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5028> 2014-08-13 17:23:21.647540 7fcce1dad700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6833/32413 pipe(0x524e680 sd=309 :43937 s=4 pgs=152 cs=1 l=1 c=0x53e3860).fault (0) Success -5027> 2014-08-13 17:23:21.647547 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bb180 con 0x4ec3f40 -5026> 2014-08-13 17:23:21.647560 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5025> 2014-08-13 17:23:21.647579 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e6780 con 0x4ec56a0 -5024> 2014-08-13 17:23:21.647594 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5023> 2014-08-13 17:23:21.645362 7fccdef7f700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6815/31286 pipe(0x52c8500 sd=321 :48512 s=4 pgs=107 cs=1 l=1 c=0x5269fa0).reader couldn't read tag, (0) Success -5022> 2014-08-13 17:23:21.647602 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 8 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e5c40 con 0x4ec3440 -5021> 2014-08-13 17:23:21.647607 7fccdef7f700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6815/31286 pipe(0x52c8500 sd=321 :48512 s=4 pgs=107 cs=1 l=1 c=0x5269fa0).fault (0) Success -5020> 2014-08-13 17:23:21.647619 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5019> 2014-08-13 17:23:21.647628 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 7 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b9d40 con 0x4ec4780 -5018> 2014-08-13 17:23:21.647647 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5017> 2014-08-13 17:23:21.646886 7fccecb5a700 2 -- 192.168.100.115:0/31924 >> 192.168.100.112:6842/8300 pipe(0x533a080 sd=215 :50448 s=4 pgs=131 cs=1 l=1 c=0x5264780).fault (0) Success -5016> 2014-08-13 17:23:21.647655 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bdc40 con 0x4ec5d80 -5015> 2014-08-13 17:23:21.647669 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5014> 2014-08-13 17:23:21.645845 7fcce48d8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6813/6757 pipe(0x524a080 sd=281 :42995 s=4 pgs=158 cs=1 l=1 c=0x536f380).reader couldn't read tag, (0) Success -5013> 2014-08-13 17:23:21.647677 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e7980 con 0x4ec5ac0 -5012> 2014-08-13 17:23:21.647691 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5011> 2014-08-13 17:23:21.647683 7fcce48d8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6813/6757 pipe(0x524a080 sd=281 :42995 s=4 pgs=158 cs=1 l=1 c=0x536f380).fault (0) Success -5010> 2014-08-13 17:23:21.647699 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e06c0 con 0x4eb0c60 -5009> 2014-08-13 17:23:21.644882 7fcce18a8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6829/12102 pipe(0x52ca080 sd=310 :58797 s=4 pgs=201 cs=1 l=1 c=0x53e6b40).reader couldn't read tag, (0) Success -5008> 2014-08-13 17:23:21.647713 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5007> 2014-08-13 17:23:21.647718 7fcce18a8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6829/12102 pipe(0x52ca080 sd=310 :58797 s=4 pgs=201 cs=1 l=1 c=0x53e6b40).fault (0) Success -5006> 2014-08-13 17:23:21.647721 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b8fc0 con 0x4eb23c0 -5005> 2014-08-13 17:23:21.645421 7fcce44d4700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6827/11446 pipe(0x524d000 sd=288 :35525 s=4 pgs=170 cs=1 l=1 c=0x536dd80).reader couldn't read tag, (0) Success -5004> 2014-08-13 17:23:21.647735 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -5003> 2014-08-13 17:23:21.647736 7fcce44d4700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6827/11446 pipe(0x524d000 sd=288 :35525 s=4 pgs=170 cs=1 l=1 c=0x536dd80).fault (0) Success -5002> 2014-08-13 17:23:21.645395 7fcce1eae700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6850/35445 pipe(0x524fa80 sd=298 :54142 s=4 pgs=138 cs=1 l=1 c=0x53e06e0).reader couldn't read tag, (0) Success -5001> 2014-08-13 17:23:21.647742 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52baac0 con 0x4eb18c0 -5000> 2014-08-13 17:23:21.647783 7fcce1eae700 2 -- 192.168.100.115:0/31924 >> 192.168.100.109:6850/35445 pipe(0x524fa80 sd=298 :54142 s=4 pgs=138 cs=1 l=1 c=0x53e06e0).fault (0) Success -4999> 2014-08-13 17:23:21.647803 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4998> 2014-08-13 17:23:21.647822 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b8480 con 0x4eb40a0 -4997> 2014-08-13 17:23:21.645232 7fcce31c1700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6831/12680 pipe(0x524c100 sd=295 :60517 s=4 pgs=204 cs=1 l=1 c=0x53e0f20).reader couldn't read tag, (0) Success -4996> 2014-08-13 17:23:21.647843 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4995> 2014-08-13 17:23:21.647843 7fcce31c1700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6831/12680 pipe(0x524c100 sd=295 :60517 s=4 pgs=204 cs=1 l=1 c=0x53e0f20).fault (0) Success -4994> 2014-08-13 17:23:21.647852 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bb600 con 0x4eb2680 -4993> 2014-08-13 17:23:21.647867 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4992> 2014-08-13 17:23:21.647875 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e7bc0 con 0x4eb7d20 -4991> 2014-08-13 17:23:21.647889 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4990> 2014-08-13 17:23:21.647897 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.296 192.168.1.109:6806/31286 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52be300 con 0x4f81600 -4989> 2014-08-13 17:23:21.645480 7fcce1faf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.110:6815/43857 pipe(0x524f800 sd=299 :42061 s=4 pgs=147 cs=1 l=1 c=0x53e0420).reader couldn't read tag, (0) Success -4988> 2014-08-13 17:23:21.647911 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4987> 2014-08-13 17:23:21.647912 7fcce1faf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.110:6815/43857 pipe(0x524f800 sd=299 :42061 s=4 pgs=147 cs=1 l=1 c=0x53e0420).fault (0) Success -4986> 2014-08-13 17:23:21.647900 7fcce41d1700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930000 sd=481 :6808 s=0 pgs=0 cs=0 l=0 c=0x546ee00).accept sd=481 192.168.100.102:33641/0 -4985> 2014-08-13 17:23:21.647920 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5508240 con 0x4eb6720 -4984> 2014-08-13 17:23:21.647935 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4983> 2014-08-13 17:23:21.647942 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52b8240 con 0x4f80b00 -4982> 2014-08-13 17:23:21.647956 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4981> 2014-08-13 17:23:21.647966 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.19 192.168.1.115:6818/61186 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e5100 con 0x4f84ba0 -4980> 2014-08-13 17:23:21.647979 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4979> 2014-08-13 17:23:21.647987 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e4c80 con 0x4f83860 -4978> 2014-08-13 17:23:21.648000 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4977> 2014-08-13 17:23:21.648010 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e18c0 con 0x4fd9760 -4976> 2014-08-13 17:23:21.648024 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4975> 2014-08-13 17:23:21.648036 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 7 ==== pg_notify(22.3d5(1) epoch 194477) v5 ==== 769+0+0 (4248740696 0 0) 0x56ac7c0 con 0x4f86e00 -4974> 2014-08-13 17:23:21.645600 7fccde373700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6805/5112 pipe(0x52cb700 sd=333 :60390 s=4 pgs=145 cs=1 l=1 c=0x526c780).reader couldn't read tag, (0) Success -4973> 2014-08-13 17:23:21.648055 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.637267, event: header_read, request: pg_notify(22.3d5(1) epoch 194477) v5 -4972> 2014-08-13 17:23:21.648057 7fccde373700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6805/5112 pipe(0x52cb700 sd=333 :60390 s=4 pgs=145 cs=1 l=1 c=0x526c780).fault (0) Success -4971> 2014-08-13 17:23:21.648067 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.637268, event: throttled, request: pg_notify(22.3d5(1) epoch 194477) v5 -4970> 2014-08-13 17:23:21.648074 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.637341, event: all_read, request: pg_notify(22.3d5(1) epoch 194477) v5 -4969> 2014-08-13 17:23:21.648080 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.648049, event: dispatched, request: pg_notify(22.3d5(1) epoch 194477) v5 -4968> 2014-08-13 17:23:21.648088 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.648088, event: waiting_for_osdmap, request: pg_notify(22.3d5(1) epoch 194477) v5 -4967> 2014-08-13 17:23:21.648098 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.648098, event: started, request: pg_notify(22.3d5(1) epoch 194477) v5 -4966> 2014-08-13 17:23:21.648125 7fcd0964c700 5 -- op tracker -- , seq: 172, time: 2014-08-13 17:23:21.648125, event: done, request: pg_notify(22.3d5(1) epoch 194477) v5 -4965> 2014-08-13 17:23:21.648146 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 4 ==== pg_notify(22.99(22) epoch 194477) v5 ==== 1944+0+0 (905055431 0 0) 0x56af700 con 0x4fd8f20 -4964> 2014-08-13 17:23:21.648163 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.641293, event: header_read, request: pg_notify(22.99(22) epoch 194477) v5 -4963> 2014-08-13 17:23:21.648170 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.641294, event: throttled, request: pg_notify(22.99(22) epoch 194477) v5 -4962> 2014-08-13 17:23:21.648176 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.641392, event: all_read, request: pg_notify(22.99(22) epoch 194477) v5 -4961> 2014-08-13 17:23:21.645628 7fccde171700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6813/5112 pipe(0x52cb480 sd=334 :33221 s=4 pgs=145 cs=1 l=1 c=0x526c4c0).reader couldn't read tag, (0) Success -4960> 2014-08-13 17:23:21.648181 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.648161, event: dispatched, request: pg_notify(22.99(22) epoch 194477) v5 -4959> 2014-08-13 17:23:21.648199 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.648199, event: waiting_for_osdmap, request: pg_notify(22.99(22) epoch 194477) v5 -4958> 2014-08-13 17:23:21.648207 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.648207, event: started, request: pg_notify(22.99(22) epoch 194477) v5 -4957> 2014-08-13 17:23:21.648193 7fccde171700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6813/5112 pipe(0x52cb480 sd=334 :33221 s=4 pgs=145 cs=1 l=1 c=0x526c4c0).fault (0) Success -4956> 2014-08-13 17:23:21.648229 7fcd0964c700 5 -- op tracker -- , seq: 173, time: 2014-08-13 17:23:21.648229, event: done, request: pg_notify(22.99(22) epoch 194477) v5 -4955> 2014-08-13 17:23:21.648248 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 6 ==== pg_notify(22.4df(22) epoch 194477) v5 ==== 1944+0+0 (140617638 0 0) 0x56af540 con 0x4fd86e0 -4954> 2014-08-13 17:23:21.648264 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.645426, event: header_read, request: pg_notify(22.4df(22) epoch 194477) v5 -4953> 2014-08-13 17:23:21.648270 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.645427, event: throttled, request: pg_notify(22.4df(22) epoch 194477) v5 -4952> 2014-08-13 17:23:21.648276 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.645529, event: all_read, request: pg_notify(22.4df(22) epoch 194477) v5 -4951> 2014-08-13 17:23:21.648281 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.648261, event: dispatched, request: pg_notify(22.4df(22) epoch 194477) v5 -4950> 2014-08-13 17:23:21.648288 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.648287, event: waiting_for_osdmap, request: pg_notify(22.4df(22) epoch 194477) v5 -4949> 2014-08-13 17:23:21.648295 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.648295, event: started, request: pg_notify(22.4df(22) epoch 194477) v5 -4948> 2014-08-13 17:23:21.645695 7fccdd060700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6815/25893 pipe(0x52cfa80 sd=342 :58000 s=4 pgs=44 cs=1 l=1 c=0x526b180).reader couldn't read tag, (0) Success -4947> 2014-08-13 17:23:21.648315 7fcd0964c700 5 -- op tracker -- , seq: 174, time: 2014-08-13 17:23:21.648314, event: done, request: pg_notify(22.4df(22) epoch 194477) v5 -4946> 2014-08-13 17:23:21.648316 7fccdd060700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6815/25893 pipe(0x52cfa80 sd=342 :58000 s=4 pgs=44 cs=1 l=1 c=0x526b180).fault (0) Success -4945> 2014-08-13 17:23:21.648342 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.332 192.168.1.110:6806/42024 4 ==== pg_notify(24.1bb(6) epoch 194477) v5 ==== 1094+0+0 (2664606886 0 0) 0x54d0540 con 0x4fdbde0 -4944> 2014-08-13 17:23:21.648360 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.644845, event: header_read, request: pg_notify(24.1bb(6) epoch 194477) v5 -4943> 2014-08-13 17:23:21.648367 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.644847, event: throttled, request: pg_notify(24.1bb(6) epoch 194477) v5 -4942> 2014-08-13 17:23:21.648373 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.644951, event: all_read, request: pg_notify(24.1bb(6) epoch 194477) v5 -4941> 2014-08-13 17:23:21.648379 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.648356, event: dispatched, request: pg_notify(24.1bb(6) epoch 194477) v5 -4940> 2014-08-13 17:23:21.648385 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.648384, event: waiting_for_osdmap, request: pg_notify(24.1bb(6) epoch 194477) v5 -4939> 2014-08-13 17:23:21.648392 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.648392, event: started, request: pg_notify(24.1bb(6) epoch 194477) v5 -4938> 2014-08-13 17:23:21.648417 7fcd0964c700 5 -- op tracker -- , seq: 175, time: 2014-08-13 17:23:21.648416, event: done, request: pg_notify(24.1bb(6) epoch 194477) v5 -4937> 2014-08-13 17:23:21.645789 7fcce68f8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.104:6805/62197 pipe(0x5248f00 sd=263 :55033 s=4 pgs=207 cs=1 l=1 c=0x536aec0).reader couldn't read tag, (0) Success -4936> 2014-08-13 17:23:21.648434 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 5 ==== pg_query(22.220 epoch 194477) v3 ==== 144+0+0 (4164157537 0 0) 0x2f92b20 con 0x4fdee00 -4935> 2014-08-13 17:23:21.648449 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.640081, event: header_read, request: pg_query(22.220 epoch 194477) v3 -4934> 2014-08-13 17:23:21.648444 7fcce68f8700 2 -- 192.168.100.115:0/31924 >> 192.168.1.104:6805/62197 pipe(0x5248f00 sd=263 :55033 s=4 pgs=207 cs=1 l=1 c=0x536aec0).fault (0) Success -4933> 2014-08-13 17:23:21.648456 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.640083, event: throttled, request: pg_query(22.220 epoch 194477) v3 -4932> 2014-08-13 17:23:21.648462 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.640139, event: all_read, request: pg_query(22.220 epoch 194477) v3 -4931> 2014-08-13 17:23:21.648467 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.648447, event: dispatched, request: pg_query(22.220 epoch 194477) v3 -4930> 2014-08-13 17:23:21.648473 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.648473, event: waiting_for_osdmap, request: pg_query(22.220 epoch 194477) v3 -4929> 2014-08-13 17:23:21.648481 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.648480, event: started, request: pg_query(22.220 epoch 194477) v3 -4928> 2014-08-13 17:23:21.648501 7fcd0964c700 5 -- op tracker -- , seq: 176, time: 2014-08-13 17:23:21.648501, event: done, request: pg_query(22.220 epoch 194477) v3 -4927> 2014-08-13 17:23:21.645815 7fccdd969700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6802/25893 pipe(0x52cf800 sd=341 :48799 s=4 pgs=44 cs=1 l=1 c=0x526aec0).reader couldn't read tag, (0) Success -4926> 2014-08-13 17:23:21.648517 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 4 ==== pg_query(22.581 epoch 194477) v3 ==== 144+0+0 (4240417718 0 0) 0x2f90960 con 0x4fde300 -4925> 2014-08-13 17:23:21.648523 7fccdd969700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6802/25893 pipe(0x52cf800 sd=341 :48799 s=4 pgs=44 cs=1 l=1 c=0x526aec0).fault (0) Success -4924> 2014-08-13 17:23:21.648532 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.640215, event: header_read, request: pg_query(22.581 epoch 194477) v3 -4923> 2014-08-13 17:23:21.648538 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.640218, event: throttled, request: pg_query(22.581 epoch 194477) v3 -4922> 2014-08-13 17:23:21.648544 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.640357, event: all_read, request: pg_query(22.581 epoch 194477) v3 -4921> 2014-08-13 17:23:21.648549 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.648530, event: dispatched, request: pg_query(22.581 epoch 194477) v3 -4920> 2014-08-13 17:23:21.648554 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.648554, event: waiting_for_osdmap, request: pg_query(22.581 epoch 194477) v3 -4919> 2014-08-13 17:23:21.648562 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.648562, event: started, request: pg_query(22.581 epoch 194477) v3 -4918> 2014-08-13 17:23:21.648577 7fcd0964c700 5 -- op tracker -- , seq: 177, time: 2014-08-13 17:23:21.648577, event: done, request: pg_query(22.581 epoch 194477) v3 -4917> 2014-08-13 17:23:21.645838 7fccdcf5f700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6819/9142 pipe(0x52cbc00 sd=345 :49010 s=4 pgs=137 cs=1 l=1 c=0x526cd00).reader couldn't read tag, (0) Success -4916> 2014-08-13 17:23:21.648587 7fccdcf5f700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6819/9142 pipe(0x52cbc00 sd=345 :49010 s=4 pgs=137 cs=1 l=1 c=0x526cd00).fault (0) Success -4915> 2014-08-13 17:23:21.648589 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e1440 con 0x4fde040 -4914> 2014-08-13 17:23:21.648608 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4913> 2014-08-13 17:23:21.648617 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 7 ==== pg_notify(24.9b(26) epoch 194477) v5 ==== 2206+0+0 (1971894860 0 0) 0x54d4b40 con 0x4fd3860 -4912> 2014-08-13 17:23:21.648630 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.639576, event: header_read, request: pg_notify(24.9b(26) epoch 194477) v5 -4911> 2014-08-13 17:23:21.648637 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.639577, event: throttled, request: pg_notify(24.9b(26) epoch 194477) v5 -4910> 2014-08-13 17:23:21.648643 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.639687, event: all_read, request: pg_notify(24.9b(26) epoch 194477) v5 -4909> 2014-08-13 17:23:21.648648 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.648628, event: dispatched, request: pg_notify(24.9b(26) epoch 194477) v5 -4908> 2014-08-13 17:23:21.648654 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.648654, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194477) v5 -4907> 2014-08-13 17:23:21.648662 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.648662, event: started, request: pg_notify(24.9b(26) epoch 194477) v5 -4906> 2014-08-13 17:23:21.648718 7fccccc5d700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5933c00 sd=486 :6808 s=0 pgs=0 cs=0 l=0 c=0x546e880).accept sd=486 192.168.100.104:53881/0 -4905> 2014-08-13 17:23:21.646004 7fcce40d0700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6829/4534 pipe(0x524d500 sd=286 :41288 s=4 pgs=128 cs=1 l=1 c=0x536e300).reader couldn't read tag, (0) Success -4904> 2014-08-13 17:23:21.648827 7fcce40d0700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6829/4534 pipe(0x524d500 sd=286 :41288 s=4 pgs=128 cs=1 l=1 c=0x536e300).fault (0) Success -4903> 2014-08-13 17:23:21.645877 7fcce3fcf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6837/6227 pipe(0x524cb00 sd=291 :35076 s=4 pgs=114 cs=1 l=1 c=0x53e1a20).reader couldn't read tag, (0) Success -4902> 2014-08-13 17:23:21.648945 7fcce3fcf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.108:6837/6227 pipe(0x524cb00 sd=291 :35076 s=4 pgs=114 cs=1 l=1 c=0x53e1a20).fault (0) Success -4901> 2014-08-13 17:23:21.648987 7fcccda6b700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5930280 sd=477 :6808 s=0 pgs=0 cs=0 l=0 c=0x546f0c0).accept sd=477 192.168.100.114:49555/0 -4900> 2014-08-13 17:23:21.644937 7fccecd5c700 2 -- 192.168.100.115:0/31924 >> 192.168.1.112:6839/8300 pipe(0x533a300 sd=214 :49894 s=4 pgs=131 cs=1 l=1 c=0x5264a40).reader couldn't read tag, (0) Success -4899> 2014-08-13 17:23:21.649038 7fccecd5c700 2 -- 192.168.100.115:0/31924 >> 192.168.1.112:6839/8300 pipe(0x533a300 sd=214 :49894 s=4 pgs=131 cs=1 l=1 c=0x5264a40).fault (0) Success -4898> 2014-08-13 17:23:21.646276 7fcce62f2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.102:6817/16129 pipe(0x5249680 sd=260 :36807 s=4 pgs=183 cs=1 l=1 c=0x53770c0).reader couldn't read tag, (0) Success -4897> 2014-08-13 17:23:21.649161 7fcce62f2700 2 -- 192.168.100.115:0/31924 >> 192.168.100.102:6817/16129 pipe(0x5249680 sd=260 :36807 s=4 pgs=183 cs=1 l=1 c=0x53770c0).fault (0) Success -4896> 2014-08-13 17:23:21.645670 7fcce6dfd700 2 -- 192.168.100.115:0/31924 >> 192.168.100.104:6816/62197 pipe(0x5248c80 sd=264 :45025 s=4 pgs=207 cs=1 l=1 c=0x536ac00).reader couldn't read tag, (0) Success -4895> 2014-08-13 17:23:21.649223 7fcce6dfd700 2 -- 192.168.100.115:0/31924 >> 192.168.100.104:6816/62197 pipe(0x5248c80 sd=264 :45025 s=4 pgs=207 cs=1 l=1 c=0x536ac00).fault (0) Success -4894> 2014-08-13 17:23:21.646285 7fccdc656700 2 -- 192.168.100.115:0/31924 >> 192.168.1.105:6815/41806 pipe(0x52cf580 sd=348 :48528 s=4 pgs=71 cs=1 l=1 c=0x526f220).reader couldn't read tag, (0) Success -4893> 2014-08-13 17:23:21.649252 7fccdc656700 2 -- 192.168.100.115:0/31924 >> 192.168.1.105:6815/41806 pipe(0x52cf580 sd=348 :48528 s=4 pgs=71 cs=1 l=1 c=0x526f220).fault (0) Success -4892> 2014-08-13 17:23:21.644662 7fcce27b7700 2 -- 192.168.100.115:0/31924 >> 192.168.1.107:6822/54786 pipe(0x524ee00 sd=303 :36167 s=4 pgs=133 cs=1 l=1 c=0x53e40a0).reader couldn't read tag, (0) Success -4891> 2014-08-13 17:23:21.649339 7fcce27b7700 2 -- 192.168.100.115:0/31924 >> 192.168.1.107:6822/54786 pipe(0x524ee00 sd=303 :36167 s=4 pgs=133 cs=1 l=1 c=0x53e40a0).fault (0) Success -4890> 2014-08-13 17:23:21.644924 7fcced160700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6823/10782 pipe(0x533a800 sd=212 :44385 s=4 pgs=190 cs=1 l=1 c=0x5264fc0).reader couldn't read tag, (0) Success -4889> 2014-08-13 17:23:21.649415 7fcced160700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6823/10782 pipe(0x533a800 sd=212 :44385 s=4 pgs=190 cs=1 l=1 c=0x5264fc0).fault (0) Success -4888> 2014-08-13 17:23:21.645263 7fccde676700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6805/31016 pipe(0x52c8280 sd=320 :56713 s=4 pgs=127 cs=1 l=1 c=0x5269ce0).reader couldn't read tag, (0) Success -4887> 2014-08-13 17:23:21.644950 7fcce0a9a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6837/14923 pipe(0x52c9680 sd=314 :48315 s=4 pgs=186 cs=1 l=1 c=0x53e6040).reader couldn't read tag, (0) Success -4886> 2014-08-13 17:23:21.649117 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -4885> 2014-08-13 17:23:21.649444 7fccde676700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6805/31016 pipe(0x52c8280 sd=320 :56713 s=4 pgs=127 cs=1 l=1 c=0x5269ce0).fault (0) Success -4884> 2014-08-13 17:23:21.649449 7fcce0a9a700 2 -- 192.168.100.115:0/31924 >> 192.168.1.111:6837/14923 pipe(0x52c9680 sd=314 :48315 s=4 pgs=186 cs=1 l=1 c=0x53e6040).fault (0) Success -4883> 2014-08-13 17:23:21.649306 7fcce17a7700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5935a00 sd=483 :6820 s=0 pgs=0 cs=0 l=0 c=0x5948160).accept sd=483 192.168.1.106:54828/0 -4882> 2014-08-13 17:23:21.649518 7fcce6bfb700 2 -- 192.168.100.115:0/31924 >> 192.168.1.102:6843/20017 pipe(0x5248a00 sd=265 :39708 s=4 pgs=154 cs=1 l=1 c=0x536a940).reader couldn't read tag, (0) Success -4881> 2014-08-13 17:23:21.649585 7fcce6bfb700 2 -- 192.168.100.115:0/31924 >> 192.168.1.102:6843/20017 pipe(0x5248a00 sd=265 :39708 s=4 pgs=154 cs=1 l=1 c=0x536a940).fault (0) Success -4880> 2014-08-13 17:23:21.649604 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.738132 4 0.000204 -4879> 2014-08-13 17:23:21.649639 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] exit Started 0.738236 0 0.000000 -4878> 2014-08-13 17:23:21.649660 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194476/192979) [320,4,30]/[320,30,12] r=-1 lpr=194476 pi=192979-194475/25 crt=192888'497 lcod 0'0 remapped NOTIFY] enter Reset -4877> 2014-08-13 17:23:21.649652 7fcccce5f700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5933e80 sd=484 :6808 s=0 pgs=0 cs=0 l=0 c=0x546eb40).accept sd=484 192.168.100.106:51081/0 -4876> 2014-08-13 17:23:21.649756 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.000096 1 0.000122 -4875> 2014-08-13 17:23:21.649779 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started -4874> 2014-08-13 17:23:21.649794 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start -4873> 2014-08-13 17:23:21.649814 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -4872> 2014-08-13 17:23:21.649833 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 -4871> 2014-08-13 17:23:21.649851 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray -4870> 2014-08-13 17:23:21.645371 7fcce25b5700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6821/35445 pipe(0x524fd00 sd=297 :53295 s=4 pgs=138 cs=1 l=1 c=0x53e09a0).reader couldn't read tag, (0) Success -4869> 2014-08-13 17:23:21.649926 7fcce25b5700 2 -- 192.168.100.115:0/31924 >> 192.168.1.109:6821/35445 pipe(0x524fd00 sd=297 :53295 s=4 pgs=138 cs=1 l=1 c=0x53e09a0).fault (0) Success -4868> 2014-08-13 17:23:21.649924 7fcd0964c700 5 -- op tracker -- , seq: 178, time: 2014-08-13 17:23:21.649924, event: done, request: pg_notify(24.9b(26) epoch 194477) v5 -4867> 2014-08-13 17:23:21.645553 7fcce4fdf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6821/10050 pipe(0x524da00 sd=285 :39117 s=4 pgs=154 cs=1 l=1 c=0x536e880).reader couldn't read tag, (0) Success -4866> 2014-08-13 17:23:21.649951 7fcce4fdf700 2 -- 192.168.100.115:0/31924 >> 192.168.1.106:6821/10050 pipe(0x524da00 sd=285 :39117 s=4 pgs=154 cs=1 l=1 c=0x536e880).fault (0) Success -4865> 2014-08-13 17:23:21.649968 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 6 ==== pg_info(1 pgs e194477:24.cd(1)) v4 ==== 799+0+0 (2773800722 0 0) 0x54d6040 con 0x4fd0f20 -4864> 2014-08-13 17:23:21.649989 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.645107, event: header_read, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4863> 2014-08-13 17:23:21.649997 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.645109, event: throttled, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4862> 2014-08-13 17:23:21.650004 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.645175, event: all_read, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4861> 2014-08-13 17:23:21.650011 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.649986, event: dispatched, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4860> 2014-08-13 17:23:21.650018 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.650018, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4859> 2014-08-13 17:23:21.650001 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.676036 2 0.000098 -4858> 2014-08-13 17:23:21.650034 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.650034, event: started, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4857> 2014-08-13 17:23:21.650028 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.738017 0 0.000000 -4856> 2014-08-13 17:23:21.650033 7fcccd465700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5931680 sd=482 :6820 s=0 pgs=0 cs=0 l=0 c=0x5948420).accept sd=482 192.168.1.104:46456/0 -4855> 2014-08-13 17:23:21.650051 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4854> 2014-08-13 17:23:21.650061 7fcd0964c700 5 -- op tracker -- , seq: 179, time: 2014-08-13 17:23:21.650060, event: done, request: pg_info(1 pgs e194477:24.cd(1)) v4 -4853> 2014-08-13 17:23:21.650085 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 4 ==== pg_notify(22.3d9(42) epoch 194477) v5 ==== 2996+0+0 (2760797936 0 0) 0x54d1dc0 con 0x4fd5960 -4852> 2014-08-13 17:23:21.650102 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.642933, event: header_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -4851> 2014-08-13 17:23:21.650110 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.642935, event: throttled, request: pg_notify(22.3d9(42) epoch 194477) v5 -4850> 2014-08-13 17:23:21.650116 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.643057, event: all_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -4849> 2014-08-13 17:23:21.645429 7fccec958700 2 -- 192.168.100.115:0/31924 >> 192.168.1.101:6803/22303 pipe(0x5339e00 sd=216 :37417 s=4 pgs=144 cs=1 l=1 c=0x52665c0).reader couldn't read tag, (0) Success -4848> 2014-08-13 17:23:21.650121 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.650100, event: dispatched, request: pg_notify(22.3d9(42) epoch 194477) v5 -4847> 2014-08-13 17:23:21.650128 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.650128, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194477) v5 -4846> 2014-08-13 17:23:21.650126 7fccec958700 2 -- 192.168.100.115:0/31924 >> 192.168.1.101:6803/22303 pipe(0x5339e00 sd=216 :37417 s=4 pgs=144 cs=1 l=1 c=0x52665c0).fault (0) Success -4845> 2014-08-13 17:23:21.650137 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.650136, event: started, request: pg_notify(22.3d9(42) epoch 194477) v5 -4844> 2014-08-13 17:23:21.650159 7fcd0964c700 5 -- op tracker -- , seq: 180, time: 2014-08-13 17:23:21.650159, event: done, request: pg_notify(22.3d9(42) epoch 194477) v5 -4843> 2014-08-13 17:23:21.646990 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 0.696359 4 0.000317 -4842> 2014-08-13 17:23:21.650201 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 7 ==== pg_notify(24.3d(26) epoch 194477) v5 ==== 2206+0+0 (3834153262 0 0) 0x56af8c0 con 0x4fd32e0 -4841> 2014-08-13 17:23:21.650220 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.638808, event: header_read, request: pg_notify(24.3d(26) epoch 194477) v5 -4840> 2014-08-13 17:23:21.650227 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.638809, event: throttled, request: pg_notify(24.3d(26) epoch 194477) v5 -4839> 2014-08-13 17:23:21.650211 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started 0.699683 0 0.000000 -4838> 2014-08-13 17:23:21.650233 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.638908, event: all_read, request: pg_notify(24.3d(26) epoch 194477) v5 -4837> 2014-08-13 17:23:21.650239 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.650217, event: dispatched, request: pg_notify(24.3d(26) epoch 194477) v5 -4836> 2014-08-13 17:23:21.650246 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.650245, event: waiting_for_osdmap, request: pg_notify(24.3d(26) epoch 194477) v5 -4835> 2014-08-13 17:23:21.650241 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194476/193227) [330,4,288]/[330,57,304] r=-1 lpr=194476 pi=193227-194475/32 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Reset -4834> 2014-08-13 17:23:21.650257 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.650257, event: started, request: pg_notify(24.3d(26) epoch 194477) v5 -4833> 2014-08-13 17:23:21.650280 7fcd0964c700 5 -- op tracker -- , seq: 181, time: 2014-08-13 17:23:21.650279, event: done, request: pg_notify(24.3d(26) epoch 194477) v5 -4832> 2014-08-13 17:23:21.650306 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 4 ==== pg_info(1 pgs e194477:24.299(5)) v4 ==== 1015+0+0 (3389183241 0 0) 0x56ad240 con 0x4fd3020 -4831> 2014-08-13 17:23:21.650322 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.638505, event: header_read, request: pg_info(1 pgs e194477:24.299(5)) v4 -4830> 2014-08-13 17:23:21.650329 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.638507, event: throttled, request: pg_info(1 pgs e194477:24.299(5)) v4 -4829> 2014-08-13 17:23:21.650335 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.638588, event: all_read, request: pg_info(1 pgs e194477:24.299(5)) v4 -4828> 2014-08-13 17:23:21.650340 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.650320, event: dispatched, request: pg_info(1 pgs e194477:24.299(5)) v4 -4827> 2014-08-13 17:23:21.650347 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.650347, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.299(5)) v4 -4826> 2014-08-13 17:23:21.650355 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.650354, event: started, request: pg_info(1 pgs e194477:24.299(5)) v4 -4825> 2014-08-13 17:23:21.650347 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Reset 0.000107 1 0.003310 -4824> 2014-08-13 17:23:21.650377 7fcd0964c700 5 -- op tracker -- , seq: 182, time: 2014-08-13 17:23:21.650377, event: done, request: pg_info(1 pgs e194477:24.299(5)) v4 -4823> 2014-08-13 17:23:21.650373 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started -4822> 2014-08-13 17:23:21.650391 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Start -4821> 2014-08-13 17:23:21.650394 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 5 ==== pg_info(1 pgs e194477:22.3b9(1)) v4 ==== 765+0+0 (1002479833 0 0) 0x56aa680 con 0x4fd53e0 -4820> 2014-08-13 17:23:21.650410 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.637277, event: header_read, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4819> 2014-08-13 17:23:21.650407 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: transitioning to Stray -4818> 2014-08-13 17:23:21.650424 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.637279, event: throttled, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4817> 2014-08-13 17:23:21.650431 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.637340, event: all_read, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4816> 2014-08-13 17:23:21.650437 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.650409, event: dispatched, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4815> 2014-08-13 17:23:21.650428 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Start 0.000037 0 0.000000 -4814> 2014-08-13 17:23:21.650444 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.650443, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4813> 2014-08-13 17:23:21.650452 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.650452, event: started, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4812> 2014-08-13 17:23:21.650448 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/Stray -4811> 2014-08-13 17:23:21.650473 7fcd0964c700 5 -- op tracker -- , seq: 183, time: 2014-08-13 17:23:21.650473, event: done, request: pg_info(1 pgs e194477:22.3b9(1)) v4 -4810> 2014-08-13 17:23:21.650509 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 4 ==== pg_notify(24.30b(33) epoch 194477) v5 ==== 2615+0+0 (4192084196 0 0) 0x4e50000 con 0x4fd2520 -4809> 2014-08-13 17:23:21.645805 7fccde878700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6846/9142 pipe(0x52cb980 sd=332 :36589 s=4 pgs=137 cs=1 l=1 c=0x526ca40).reader couldn't read tag, (0) Success -4808> 2014-08-13 17:23:21.650526 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.641652, event: header_read, request: pg_notify(24.30b(33) epoch 194477) v5 -4807> 2014-08-13 17:23:21.650533 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.641653, event: throttled, request: pg_notify(24.30b(33) epoch 194477) v5 -4806> 2014-08-13 17:23:21.650539 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.641776, event: all_read, request: pg_notify(24.30b(33) epoch 194477) v5 -4805> 2014-08-13 17:23:21.650545 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.650523, event: dispatched, request: pg_notify(24.30b(33) epoch 194477) v5 -4804> 2014-08-13 17:23:21.650531 7fccde878700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6846/9142 pipe(0x52cb980 sd=332 :36589 s=4 pgs=137 cs=1 l=1 c=0x526ca40).fault (0) Success -4803> 2014-08-13 17:23:21.650551 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.650551, event: waiting_for_osdmap, request: pg_notify(24.30b(33) epoch 194477) v5 -4802> 2014-08-13 17:23:21.650559 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.650559, event: started, request: pg_notify(24.30b(33) epoch 194477) v5 -4801> 2014-08-13 17:23:21.650581 7fcd0964c700 5 -- op tracker -- , seq: 184, time: 2014-08-13 17:23:21.650581, event: done, request: pg_notify(24.30b(33) epoch 194477) v5 -4800> 2014-08-13 17:23:21.650590 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.732847 4 0.000221 -4799> 2014-08-13 17:23:21.650609 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 4 ==== pg_notify(22.775(27) epoch 194477) v5 ==== 2231+0+0 (2222502980 0 0) 0x54d5080 con 0x53011e0 -4798> 2014-08-13 17:23:21.650626 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.642084, event: header_read, request: pg_notify(22.775(27) epoch 194477) v5 -4797> 2014-08-13 17:23:21.650615 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] exit Started 0.732946 0 0.000000 -4796> 2014-08-13 17:23:21.650633 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.642085, event: throttled, request: pg_notify(22.775(27) epoch 194477) v5 -4795> 2014-08-13 17:23:21.650639 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.642189, event: all_read, request: pg_notify(22.775(27) epoch 194477) v5 -4794> 2014-08-13 17:23:21.650645 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.650624, event: dispatched, request: pg_notify(22.775(27) epoch 194477) v5 -4793> 2014-08-13 17:23:21.650639 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,353,56]/[353,56,63] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'453 lcod 0'0 remapped NOTIFY] enter Reset -4792> 2014-08-13 17:23:21.650652 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.650652, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194477) v5 -4791> 2014-08-13 17:23:21.650660 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.650660, event: started, request: pg_notify(22.775(27) epoch 194477) v5 -4790> 2014-08-13 17:23:21.650705 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000067 1 0.000113 -4789> 2014-08-13 17:23:21.650728 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started -4788> 2014-08-13 17:23:21.650744 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start -4787> 2014-08-13 17:23:21.650759 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -4786> 2014-08-13 17:23:21.650777 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 -4785> 2014-08-13 17:23:21.650807 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -4784> 2014-08-13 17:23:21.650824 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4783> 2014-08-13 17:23:21.650842 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4782> 2014-08-13 17:23:21.650218 7fccccd5e700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5935780 sd=485 :6820 s=0 pgs=0 cs=0 l=0 c=0x5948000).accept sd=485 192.168.1.104:46458/0 -4781> 2014-08-13 17:23:21.650388 7fcce46d6700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6849/10050 pipe(0x524d780 sd=289 :44681 s=4 pgs=154 cs=1 l=1 c=0x536e5c0).reader couldn't read tag, (0) Success -4780> 2014-08-13 17:23:21.646214 7fccdcc5c700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6847/6893 pipe(0x52caf80 sd=340 :39250 s=4 pgs=194 cs=1 l=1 c=0x526bf40).reader couldn't read tag, (0) Success -4779> 2014-08-13 17:23:21.651226 7fcce46d6700 2 -- 192.168.100.115:0/31924 >> 192.168.100.106:6849/10050 pipe(0x524d780 sd=289 :44681 s=4 pgs=154 cs=1 l=1 c=0x536e5c0).fault (0) Success -4778> 2014-08-13 17:23:21.651236 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.729434 4 0.000580 -4777> 2014-08-13 17:23:21.651246 7fccdcc5c700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6847/6893 pipe(0x52caf80 sd=340 :39250 s=4 pgs=194 cs=1 l=1 c=0x526bf40).fault (0) Success -4776> 2014-08-13 17:23:21.646225 7fcce43d3700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6832/4534 pipe(0x524d280 sd=287 :46865 s=4 pgs=128 cs=1 l=1 c=0x536e040).reader couldn't read tag, (0) Success -4775> 2014-08-13 17:23:21.651265 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] exit Started 0.729541 0 0.000000 -4774> 2014-08-13 17:23:21.651280 7fcce43d3700 2 -- 192.168.100.115:0/31924 >> 192.168.100.108:6832/4534 pipe(0x524d280 sd=287 :46865 s=4 pgs=128 cs=1 l=1 c=0x536e040).fault (0) Success -4773> 2014-08-13 17:23:21.651315 7fcd0964c700 5 -- op tracker -- , seq: 185, time: 2014-08-13 17:23:21.651314, event: done, request: pg_notify(22.775(27) epoch 194477) v5 -4772> 2014-08-13 17:23:21.651355 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 4 ==== pg_notify(22.707(27) epoch 194477) v5 ==== 2231+0+0 (1488782986 0 0) 0x56ad5c0 con 0x5303440 -4771> 2014-08-13 17:23:21.651374 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.640916, event: header_read, request: pg_notify(22.707(27) epoch 194477) v5 -4770> 2014-08-13 17:23:21.651381 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.640917, event: throttled, request: pg_notify(22.707(27) epoch 194477) v5 -4769> 2014-08-13 17:23:21.651387 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.641021, event: all_read, request: pg_notify(22.707(27) epoch 194477) v5 -4768> 2014-08-13 17:23:21.651392 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.651371, event: dispatched, request: pg_notify(22.707(27) epoch 194477) v5 -4767> 2014-08-13 17:23:21.651399 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.651398, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194477) v5 -4766> 2014-08-13 17:23:21.651407 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.651407, event: started, request: pg_notify(22.707(27) epoch 194477) v5 -4765> 2014-08-13 17:23:21.651433 7fcd0964c700 5 -- op tracker -- , seq: 186, time: 2014-08-13 17:23:21.651433, event: done, request: pg_notify(22.707(27) epoch 194477) v5 -4764> 2014-08-13 17:23:21.651285 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194476/194473) [4,313,318]/[313,318,45] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'442 lcod 0'0 remapped NOTIFY] enter Reset -4763> 2014-08-13 17:23:21.651646 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bee40 con 0x5305ee0 -4762> 2014-08-13 17:23:21.651643 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000358 1 0.000404 -4761> 2014-08-13 17:23:21.651674 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4760> 2014-08-13 17:23:21.651669 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started -4759> 2014-08-13 17:23:21.651686 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start -4758> 2014-08-13 17:23:21.651701 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -4757> 2014-08-13 17:23:21.651719 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 -4756> 2014-08-13 17:23:21.651737 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -4755> 2014-08-13 17:23:21.651752 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4754> 2014-08-13 17:23:21.651767 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4753> 2014-08-13 17:23:21.652087 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.704569 4 0.000201 -4752> 2014-08-13 17:23:21.652117 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] exit Started 0.704672 0 0.000000 -4751> 2014-08-13 17:23:21.652136 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194476/194473) [4,310,21]/[310,21,15] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'417 lcod 0'0 remapped NOTIFY] enter Reset -4750> 2014-08-13 17:23:21.652209 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000073 1 0.000119 -4749> 2014-08-13 17:23:21.652231 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started -4748> 2014-08-13 17:23:21.652255 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start -4747> 2014-08-13 17:23:21.652276 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -4746> 2014-08-13 17:23:21.652294 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 -4745> 2014-08-13 17:23:21.652312 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -4744> 2014-08-13 17:23:21.652327 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4743> 2014-08-13 17:23:21.652347 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4742> 2014-08-13 17:23:21.644718 7fcce8615700 2 -- 192.168.100.115:0/31924 >> 192.168.1.103:6832/41157 pipe(0x533f080 sd=257 :49132 s=4 pgs=143 cs=1 l=1 c=0x5377900).reader couldn't read tag, (0) Success -4741> 2014-08-13 17:23:21.652593 7fcce8615700 2 -- 192.168.100.115:0/31924 >> 192.168.1.103:6832/41157 pipe(0x533f080 sd=257 :49132 s=4 pgs=143 cs=1 l=1 c=0x5377900).fault (0) Success -4740> 2014-08-13 17:23:21.652612 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.704013 4 0.000207 -4739> 2014-08-13 17:23:21.652641 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] exit Started 0.704112 0 0.000000 -4738> 2014-08-13 17:23:21.652660 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194476/194473) [4,5,47]/[5,47,11] r=-1 lpr=194476 pi=193306-194475/26 crt=191648'346 lcod 0'0 remapped NOTIFY] enter Reset -4737> 2014-08-13 17:23:21.652719 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000059 1 0.000105 -4736> 2014-08-13 17:23:21.652740 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started -4735> 2014-08-13 17:23:21.652755 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start -4734> 2014-08-13 17:23:21.652769 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -4733> 2014-08-13 17:23:21.652786 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000031 0 0.000000 -4732> 2014-08-13 17:23:21.651689 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e0240 con 0x5305120 -4731> 2014-08-13 17:23:21.652804 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -4730> 2014-08-13 17:23:21.652819 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4729> 2014-08-13 17:23:21.652825 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4728> 2014-08-13 17:23:21.652835 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4727> 2014-08-13 17:23:21.652848 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550c800 con 0x552b700 -4726> 2014-08-13 17:23:21.652869 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4725> 2014-08-13 17:23:21.652880 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550f080 con 0x5520b00 -4724> 2014-08-13 17:23:21.652897 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4723> 2014-08-13 17:23:21.652908 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.293 192.168.1.109:6816/33855 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52b8fc0 con 0x5724e60 -4722> 2014-08-13 17:23:21.652923 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4721> 2014-08-13 17:23:21.652935 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 8 ==== pg_notify(22.f3(21) epoch 194477) v5 ==== 1889+0+0 (1417314730 0 0) 0x56af380 con 0x4e314a0 -4720> 2014-08-13 17:23:21.652951 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.646503, event: header_read, request: pg_notify(22.f3(21) epoch 194477) v5 -4719> 2014-08-13 17:23:21.652960 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.646505, event: throttled, request: pg_notify(22.f3(21) epoch 194477) v5 -4718> 2014-08-13 17:23:21.652966 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.646717, event: all_read, request: pg_notify(22.f3(21) epoch 194477) v5 -4717> 2014-08-13 17:23:21.652971 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.652948, event: dispatched, request: pg_notify(22.f3(21) epoch 194477) v5 -4716> 2014-08-13 17:23:21.652978 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.652978, event: waiting_for_osdmap, request: pg_notify(22.f3(21) epoch 194477) v5 -4715> 2014-08-13 17:23:21.652987 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.652987, event: started, request: pg_notify(22.f3(21) epoch 194477) v5 -4714> 2014-08-13 17:23:21.653064 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit Started/Stray 0.702342 4 0.000468 -4713> 2014-08-13 17:23:21.653093 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit Started 0.702646 0 0.000000 -4712> 2014-08-13 17:23:21.653120 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194476/194473) [4,330,28]/[17,315,327] r=-1 lpr=194476 pi=193748-194475/28 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter Reset -4711> 2014-08-13 17:23:21.653167 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4710> 2014-08-13 17:23:21.653193 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.000073 1 0.000127 -4709> 2014-08-13 17:23:21.653219 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started -4708> 2014-08-13 17:23:21.653238 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Start -4707> 2014-08-13 17:23:21.653259 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary -4706> 2014-08-13 17:23:21.653279 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000042 0 0.000000 -4705> 2014-08-13 17:23:21.653298 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary -4704> 2014-08-13 17:23:21.653315 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering -4703> 2014-08-13 17:23:21.653339 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo -4702> 2014-08-13 17:23:21.653454 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 0.686928 6 0.000300 -4701> 2014-08-13 17:23:21.653495 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 0.738032 0 0.000000 -4700> 2014-08-13 17:23:21.653533 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194472 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active -4699> 2014-08-13 17:23:21.653627 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(32) epoch 194477) v5 -- ?+0 0x5184980 con 0x4ec5ac0 -4698> 2014-08-13 17:23:21.653666 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6818/61186 -- pg_notify(24.3bf(5) epoch 194477) v5 -- ?+0 0x5184600 con 0x4f84ba0 -4697> 2014-08-13 17:23:21.653690 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194477) v5 -- ?+0 0x5187c40 con 0x4e33de0 -4696> 2014-08-13 17:23:21.653709 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(39) epoch 194477) v5 -- ?+0 0x56a9a40 con 0x4ec3440 -4695> 2014-08-13 17:23:21.653729 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6812/23402 -- pg_notify(22.630(21) epoch 194477) v5 -- ?+0 0x56ad400 con 0x4eb40a0 -4694> 2014-08-13 17:23:21.653749 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.2b9(21) epoch 194477) v5 -- ?+0 0x56ab640 con 0x4eb23c0 -4693> 2014-08-13 17:23:21.653774 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(33) epoch 194477) v5 -- ?+0 0x56aaf40 con 0x5305ee0 -4692> 2014-08-13 17:23:21.653791 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6808/62751 -- pg_notify(22.69f(21) epoch 194477) v5 -- ?+0 0x56af1c0 con 0x4fde040 -4691> 2014-08-13 17:23:21.653809 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(28) epoch 194477) v5 -- ?+0 0x56ab2c0 con 0x5520b00 -4690> 2014-08-13 17:23:21.653825 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6824/52078 -- pg_notify(22.2d1(21) epoch 194477) v5 -- ?+0 0x56ab480 con 0x4eb6720 -4689> 2014-08-13 17:23:21.653846 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6840/55513 -- pg_notify(22.639(21) epoch 194477) v5 -- ?+0 0x588c440 con 0x4fd3860 -4688> 2014-08-13 17:23:21.653863 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_notify(22.1fb(21) epoch 194477) v5 -- ?+0 0x588c280 con 0x4fd32e0 -4687> 2014-08-13 17:23:21.653881 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- pg_query(22.1f1 epoch 194477) v3 -- ?+0 0x2f96180 con 0x4eb2680 -4686> 2014-08-13 17:23:21.654312 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6813/37929 -- pg_query(22.1f1 epoch 194477) v3 -- ?+0 0x2f91680 con 0x56ba940 -4685> 2014-08-13 17:23:21.654442 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_query(22.707 epoch 194477) v3 -- ?+0 0x2f96720 con 0x4f80b00 -4684> 2014-08-13 17:23:21.654555 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_query(24.b6 epoch 194477) v3 -- ?+0 0x2f903c0 con 0x4ec5ac0 -4683> 2014-08-13 17:23:21.654653 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6821/54786 -- pg_query(22.707 epoch 194477) v3 -- ?+0 0x54f94a0 con 0x4ec7e80 -4682> 2014-08-13 17:23:21.654777 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6805/37391 -- pg_query(24.b6 epoch 194477) v3 -- ?+0 0x54fab20 con 0x4e33f40 -4681> 2014-08-13 17:23:21.654844 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_query(22.3e4 epoch 194477) v3 -- ?+0 0x54fa940 con 0x4ec09a0 -4680> 2014-08-13 17:23:21.654964 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6828/12102 -- pg_query(22.1f1 epoch 194477) v3 -- ?+0 0x54faee0 con 0x57232e0 -4679> 2014-08-13 17:23:21.655082 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6826/11446 -- pg_query(22.775 epoch 194477) v3 -- ?+0 0x54f83c0 con 0x4eb48e0 -4678> 2014-08-13 17:23:21.655206 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6830/12680 -- pg_query(22.775 epoch 194477) v3 -- ?+0 0x2f930c0 con 0x5521e40 -4677> 2014-08-13 17:23:21.655319 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6812/32413 -- pg_query(22.707 epoch 194477) v3 -- ?+0 0x2f96900 con 0x5303440 -4676> 2014-08-13 17:23:21.655432 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- pg_query(22.3e4 epoch 194477) v3 -- ?+0 0x2f94920 con 0x4ec5d80 -4675> 2014-08-13 17:23:21.655499 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6800/14161 -- pg_query(24.b6 epoch 194477) v3 -- ?+0 0x2f95640 con 0x4eb6460 -4674> 2014-08-13 17:23:21.655564 7fcd0964c700 5 -- op tracker -- , seq: 187, time: 2014-08-13 17:23:21.655564, event: done, request: pg_notify(22.f3(21) epoch 194477) v5 -4673> 2014-08-13 17:23:21.655609 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 7 ==== pg_info(1 pgs e194477:22.14a(1)) v4 ==== 765+0+0 (836306210 0 0) 0x5889340 con 0x4e335a0 -4672> 2014-08-13 17:23:21.655636 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.641933, event: header_read, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4671> 2014-08-13 17:23:21.655649 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.641935, event: throttled, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4670> 2014-08-13 17:23:21.655659 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.642432, event: all_read, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4669> 2014-08-13 17:23:21.655670 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.655632, event: dispatched, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4668> 2014-08-13 17:23:21.655686 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.655686, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4667> 2014-08-13 17:23:21.655696 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.655696, event: started, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4666> 2014-08-13 17:23:21.655725 7fcd0964c700 5 -- op tracker -- , seq: 188, time: 2014-08-13 17:23:21.655725, event: done, request: pg_info(1 pgs e194477:22.14a(1)) v4 -4665> 2014-08-13 17:23:21.655743 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 5 ==== pg_info(1 pgs e194477:22.29a(1)) v4 ==== 765+0+0 (2021559218 0 0) 0x56ac600 con 0x4e32940 -4664> 2014-08-13 17:23:21.655760 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.640758, event: header_read, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4663> 2014-08-13 17:23:21.655767 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.640759, event: throttled, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4662> 2014-08-13 17:23:21.655772 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.641295, event: all_read, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4661> 2014-08-13 17:23:21.655778 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.655758, event: dispatched, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4660> 2014-08-13 17:23:21.655784 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.655784, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4659> 2014-08-13 17:23:21.655792 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.655792, event: started, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4658> 2014-08-13 17:23:21.655631 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6814/43857 -- pg_query(22.3e4 epoch 194477) v3 -- ?+0 0x2f93c00 con 0x55f2100 -4657> 2014-08-13 17:23:21.656016 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_query(22.39d epoch 194477) v3 -- ?+0 0x2f93840 con 0x4fde300 -4656> 2014-08-13 17:23:21.656143 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6804/5112 -- pg_query(24.b6 epoch 194477) v3 -- ?+0 0x2f90960 con 0x4fd35a0 -4655> 2014-08-13 17:23:21.656283 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_query(24.b6 epoch 194477) v3 -- ?+0 0x54f8780 con 0x5305ee0 -4654> 2014-08-13 17:23:21.656326 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6812/6757 -- pg_query(22.39d epoch 194477) v3 -- ?+0 0x54f8000 con 0x4ec70c0 -4653> 2014-08-13 17:23:21.656695 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6836/6227 -- pg_query(22.775 epoch 194477) v3 -- ?+0 0x54fa580 con 0x53011e0 -4652> 2014-08-13 17:23:21.657310 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_query(22.39d epoch 194477) v3 -- ?+0 0x54fa3a0 con 0x4fd32e0 -4651> 2014-08-13 17:23:21.657301 7fcccc455700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5935500 sd=313 :6820 s=0 pgs=0 cs=0 l=0 c=0x59f91e0).accept sd=313 192.168.1.112:55023/0 -4650> 2014-08-13 17:23:21.657458 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6805/37391 -- pg_info(1 pgs e194477:22.84(2)) v4 -- ?+0 0x588c0c0 con 0x4e33f40 -4649> 2014-08-13 17:23:21.657645 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_info(1 pgs e194477:22.84(2)) v4 -- ?+0 0x588bf00 con 0x4ec1ce0 -4648> 2014-08-13 17:23:21.657676 7fcd0964c700 5 -- op tracker -- , seq: 189, time: 2014-08-13 17:23:21.657676, event: done, request: pg_info(1 pgs e194477:22.29a(1)) v4 -4647> 2014-08-13 17:23:21.657709 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x2e8a400 con 0x4e33de0 -4646> 2014-08-13 17:23:21.657738 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4645> 2014-08-13 17:23:21.657750 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 4 ==== pg_info(1 pgs e194477:22.28f(1)) v4 ==== 765+0+0 (410328216 0 0) 0x5889500 con 0x4e34780 -4644> 2014-08-13 17:23:21.657766 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.645678, event: header_read, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4643> 2014-08-13 17:23:21.657775 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.645680, event: throttled, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4642> 2014-08-13 17:23:21.657781 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.645761, event: all_read, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4641> 2014-08-13 17:23:21.657787 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.657763, event: dispatched, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4640> 2014-08-13 17:23:21.657794 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.657794, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4639> 2014-08-13 17:23:21.657804 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.657804, event: started, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4638> 2014-08-13 17:23:21.657826 7fcd0964c700 5 -- op tracker -- , seq: 190, time: 2014-08-13 17:23:21.657825, event: done, request: pg_info(1 pgs e194477:22.28f(1)) v4 -4637> 2014-08-13 17:23:21.657841 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 4 ==== pg_info(1 pgs e194477:22.4aa(1)) v4 ==== 765+0+0 (1418452921 0 0) 0x54d40c0 con 0x4e35120 -4636> 2014-08-13 17:23:21.657857 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.642784, event: header_read, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4635> 2014-08-13 17:23:21.657864 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.642786, event: throttled, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4634> 2014-08-13 17:23:21.657870 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.642847, event: all_read, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4633> 2014-08-13 17:23:21.657876 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.657855, event: dispatched, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4632> 2014-08-13 17:23:21.657889 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.657889, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4631> 2014-08-13 17:23:21.657898 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.657898, event: started, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4630> 2014-08-13 17:23:21.657923 7fcd0964c700 5 -- op tracker -- , seq: 191, time: 2014-08-13 17:23:21.657923, event: done, request: pg_info(1 pgs e194477:22.4aa(1)) v4 -4629> 2014-08-13 17:23:21.657947 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e2ac0 con 0x4e33f40 -4628> 2014-08-13 17:23:21.657967 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4627> 2014-08-13 17:23:21.657937 7fcccc354700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5933980 sd=487 :6808 s=0 pgs=0 cs=0 l=0 c=0x59fa520).accept sd=487 192.168.100.112:34691/0 -4626> 2014-08-13 17:23:21.658039 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_info(1 pgs e194477:22.4df(23)) v4 -- ?+0 0x588bd40 con 0x4ec4780 -4625> 2014-08-13 17:23:21.658203 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e2400 con 0x4e35ac0 -4624> 2014-08-13 17:23:21.658235 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4623> 2014-08-13 17:23:21.658247 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52ba640 con 0x4e35800 -4622> 2014-08-13 17:23:21.658262 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4621> 2014-08-13 17:23:21.658275 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 8 ==== pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 ==== 4163+0+0 (3834676811 0 0) 0x56acb40 con 0x4ec09a0 -4620> 2014-08-13 17:23:21.658294 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.640175, event: header_read, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4619> 2014-08-13 17:23:21.658304 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.640176, event: throttled, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4618> 2014-08-13 17:23:21.658311 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.640335, event: all_read, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4617> 2014-08-13 17:23:21.658322 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.658289, event: dispatched, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4616> 2014-08-13 17:23:21.658338 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.658338, event: waiting_for_osdmap, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4615> 2014-08-13 17:23:21.658352 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.658352, event: started, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4614> 2014-08-13 17:23:21.658356 7fcce0999700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x524c380 sd=208 :6819 s=0 pgs=0 cs=0 l=0 c=0x552d280).accept sd=208 192.168.1.108:44647/0 -4613> 2014-08-13 17:23:21.658401 7fcd0964c700 5 -- op tracker -- , seq: 192, time: 2014-08-13 17:23:21.658401, event: done, request: pg_notify(22.113(23),22.3e4(26) epoch 194477) v5 -4612> 2014-08-13 17:23:21.658448 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5509b00 con 0x4ec2aa0 -4611> 2014-08-13 17:23:21.658467 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4610> 2014-08-13 17:23:21.658356 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6834/17224 -- pg_info(1 pgs e194477:22.4df(23)) v4 -- ?+0 0x588bb80 con 0x56b2260 -4609> 2014-08-13 17:23:21.658658 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 7 ==== pg_notify(22.84(22) epoch 194477) v5 ==== 1948+0+0 (3210431764 0 0) 0x5888fc0 con 0x4ec1ce0 -4608> 2014-08-13 17:23:21.658688 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.643991, event: header_read, request: pg_notify(22.84(22) epoch 194477) v5 -4607> 2014-08-13 17:23:21.658667 7fcce6efe700 1 -- 192.168.100.115:6808/31924 >> :/0 pipe(0x5933700 sd=266 :6808 s=0 pgs=0 cs=0 l=0 c=0x59fa260).accept sd=266 192.168.100.103:48312/0 -4606> 2014-08-13 17:23:21.658697 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.643993, event: throttled, request: pg_notify(22.84(22) epoch 194477) v5 -4605> 2014-08-13 17:23:21.658703 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.644083, event: all_read, request: pg_notify(22.84(22) epoch 194477) v5 -4604> 2014-08-13 17:23:21.658709 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.658684, event: dispatched, request: pg_notify(22.84(22) epoch 194477) v5 -4603> 2014-08-13 17:23:21.658716 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.658716, event: waiting_for_osdmap, request: pg_notify(22.84(22) epoch 194477) v5 -4602> 2014-08-13 17:23:21.658725 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.658725, event: started, request: pg_notify(22.84(22) epoch 194477) v5 -4601> 2014-08-13 17:23:21.658748 7fcd0964c700 5 -- op tracker -- , seq: 193, time: 2014-08-13 17:23:21.658748, event: done, request: pg_notify(22.84(22) epoch 194477) v5 -4600> 2014-08-13 17:23:21.658779 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 4 ==== pg_notify(24.34(50) epoch 194477) v5 ==== 3306+0+0 (4046455223 0 0) 0x54d4980 con 0x4ec3f40 -4599> 2014-08-13 17:23:21.658796 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.646146, event: header_read, request: pg_notify(24.34(50) epoch 194477) v5 -4598> 2014-08-13 17:23:21.658803 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.646148, event: throttled, request: pg_notify(24.34(50) epoch 194477) v5 -4597> 2014-08-13 17:23:21.658814 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.646292, event: all_read, request: pg_notify(24.34(50) epoch 194477) v5 -4596> 2014-08-13 17:23:21.658824 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.658793, event: dispatched, request: pg_notify(24.34(50) epoch 194477) v5 -4595> 2014-08-13 17:23:21.658834 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.658834, event: waiting_for_osdmap, request: pg_notify(24.34(50) epoch 194477) v5 -4594> 2014-08-13 17:23:21.658846 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.658846, event: started, request: pg_notify(24.34(50) epoch 194477) v5 -4593> 2014-08-13 17:23:21.658873 7fcd0964c700 5 -- op tracker -- , seq: 194, time: 2014-08-13 17:23:21.658872, event: done, request: pg_notify(24.34(50) epoch 194477) v5 -4592> 2014-08-13 17:23:21.658906 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 4 ==== pg_notify(22.99(23) epoch 194477) v5 ==== 2003+0+0 (1847996741 0 0) 0x56adb00 con 0x4ec56a0 -4591> 2014-08-13 17:23:21.658927 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.638203, event: header_read, request: pg_notify(22.99(23) epoch 194477) v5 -4590> 2014-08-13 17:23:21.658938 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.638205, event: throttled, request: pg_notify(22.99(23) epoch 194477) v5 -4589> 2014-08-13 17:23:21.658948 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.638318, event: all_read, request: pg_notify(22.99(23) epoch 194477) v5 -4588> 2014-08-13 17:23:21.658958 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.658924, event: dispatched, request: pg_notify(22.99(23) epoch 194477) v5 -4587> 2014-08-13 17:23:21.658968 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.658968, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194477) v5 -4586> 2014-08-13 17:23:21.658980 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.658980, event: started, request: pg_notify(22.99(23) epoch 194477) v5 -4585> 2014-08-13 17:23:21.659005 7fcd0964c700 5 -- op tracker -- , seq: 195, time: 2014-08-13 17:23:21.659004, event: done, request: pg_notify(22.99(23) epoch 194477) v5 -4584> 2014-08-13 17:23:21.659033 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 9 ==== pg_notify(22.6bd(22) epoch 194477) v5 ==== 1948+0+0 (3227937701 0 0) 0x5187a80 con 0x4ec3440 -4583> 2014-08-13 17:23:21.659056 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.641912, event: header_read, request: pg_notify(22.6bd(22) epoch 194477) v5 -4582> 2014-08-13 17:23:21.659067 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.641913, event: throttled, request: pg_notify(22.6bd(22) epoch 194477) v5 -4581> 2014-08-13 17:23:21.659077 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.642013, event: all_read, request: pg_notify(22.6bd(22) epoch 194477) v5 -4580> 2014-08-13 17:23:21.659087 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.659053, event: dispatched, request: pg_notify(22.6bd(22) epoch 194477) v5 -4579> 2014-08-13 17:23:21.659097 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.659097, event: waiting_for_osdmap, request: pg_notify(22.6bd(22) epoch 194477) v5 -4578> 2014-08-13 17:23:21.659105 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.659105, event: started, request: pg_notify(22.6bd(22) epoch 194477) v5 -4577> 2014-08-13 17:23:21.659125 7fcd0964c700 5 -- op tracker -- , seq: 196, time: 2014-08-13 17:23:21.659124, event: done, request: pg_notify(22.6bd(22) epoch 194477) v5 -4576> 2014-08-13 17:23:21.659164 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 8 ==== pg_notify(22.4df(23) epoch 194477) v5 ==== 2003+0+0 (3340126636 0 0) 0x54d01c0 con 0x4ec4780 -4575> 2014-08-13 17:23:21.658634 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating -4574> 2014-08-13 17:23:21.659339 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit Started/Stray 0.706339 4 0.000261 -4573> 2014-08-13 17:23:21.659419 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit Started 0.706523 0 0.000000 -4572> 2014-08-13 17:23:21.659186 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.640405, event: header_read, request: pg_notify(22.4df(23) epoch 194477) v5 -4571> 2014-08-13 17:23:21.659500 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.640406, event: throttled, request: pg_notify(22.4df(23) epoch 194477) v5 -4570> 2014-08-13 17:23:21.659524 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.640503, event: all_read, request: pg_notify(22.4df(23) epoch 194477) v5 -4569> 2014-08-13 17:23:21.659538 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.659184, event: dispatched, request: pg_notify(22.4df(23) epoch 194477) v5 -4568> 2014-08-13 17:23:21.659555 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.659555, event: waiting_for_osdmap, request: pg_notify(22.4df(23) epoch 194477) v5 -4567> 2014-08-13 17:23:21.659463 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194476/194473) [4,59,351]/[351,55,376] r=-1 lpr=194476 pi=193678-194475/28 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter Reset -4566> 2014-08-13 17:23:21.659566 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.659566, event: started, request: pg_notify(22.4df(23) epoch 194477) v5 -4565> 2014-08-13 17:23:21.659597 7fcd0964c700 5 -- op tracker -- , seq: 197, time: 2014-08-13 17:23:21.659597, event: done, request: pg_notify(22.4df(23) epoch 194477) v5 -4564> 2014-08-13 17:23:21.659606 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.687795 2 0.000074 -4563> 2014-08-13 17:23:21.659630 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 4 ==== pg_notify(22.3e4(27) epoch 194477) v5 ==== 2231+0+0 (3981404535 0 0) 0x5888c40 con 0x4ec5d80 -4562> 2014-08-13 17:23:21.659657 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.647094, event: header_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -4561> 2014-08-13 17:23:21.659664 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.647096, event: throttled, request: pg_notify(22.3e4(27) epoch 194477) v5 -4560> 2014-08-13 17:23:21.659674 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.647218, event: all_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -4559> 2014-08-13 17:23:21.659683 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.659653, event: dispatched, request: pg_notify(22.3e4(27) epoch 194477) v5 -4558> 2014-08-13 17:23:21.659651 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.742316 0 0.000000 -4557> 2014-08-13 17:23:21.659696 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.659695, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194477) v5 -4556> 2014-08-13 17:23:21.659710 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.659710, event: started, request: pg_notify(22.3e4(27) epoch 194477) v5 -4555> 2014-08-13 17:23:21.659715 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194471 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4554> 2014-08-13 17:23:21.659737 7fcd0964c700 5 -- op tracker -- , seq: 198, time: 2014-08-13 17:23:21.659737, event: done, request: pg_notify(22.3e4(27) epoch 194477) v5 -4553> 2014-08-13 17:23:21.659770 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 6 ==== pg_info(1 pgs e194477:24.2a1(1)) v4 ==== 799+0+0 (4039237035 0 0) 0x56ae900 con 0x4ec5ac0 -4552> 2014-08-13 17:23:21.659791 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.639260, event: header_read, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4551> 2014-08-13 17:23:21.659801 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.639261, event: throttled, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4550> 2014-08-13 17:23:21.659808 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.639329, event: all_read, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4549> 2014-08-13 17:23:21.659814 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.659786, event: dispatched, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4548> 2014-08-13 17:23:21.659823 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.659823, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4547> 2014-08-13 17:23:21.659835 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.659835, event: started, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4546> 2014-08-13 17:23:21.659654 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Reset 0.000190 1 0.000310 -4545> 2014-08-13 17:23:21.659863 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started -4544> 2014-08-13 17:23:21.659878 7fcd0964c700 5 -- op tracker -- , seq: 199, time: 2014-08-13 17:23:21.659878, event: done, request: pg_info(1 pgs e194477:24.2a1(1)) v4 -4543> 2014-08-13 17:23:21.659890 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Start -4542> 2014-08-13 17:23:21.659901 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 4 ==== pg_notify(22.619(5) epoch 194477) v5 ==== 1001+0+0 (1347682708 0 0) 0x56ad780 con 0x4eb0c60 -4541> 2014-08-13 17:23:21.659947 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.638531, event: header_read, request: pg_notify(22.619(5) epoch 194477) v5 -4540> 2014-08-13 17:23:21.659967 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.638533, event: throttled, request: pg_notify(22.619(5) epoch 194477) v5 -4539> 2014-08-13 17:23:21.659995 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.638611, event: all_read, request: pg_notify(22.619(5) epoch 194477) v5 -4538> 2014-08-13 17:23:21.660009 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.659943, event: dispatched, request: pg_notify(22.619(5) epoch 194477) v5 -4537> 2014-08-13 17:23:21.660018 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.660018, event: waiting_for_osdmap, request: pg_notify(22.619(5) epoch 194477) v5 -4536> 2014-08-13 17:23:21.660029 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.660028, event: started, request: pg_notify(22.619(5) epoch 194477) v5 -4535> 2014-08-13 17:23:21.660070 7fcd0964c700 5 -- op tracker -- , seq: 200, time: 2014-08-13 17:23:21.660070, event: done, request: pg_notify(22.619(5) epoch 194477) v5 -4534> 2014-08-13 17:23:21.660100 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 5 ==== pg_info(1 pgs e194477:22.2b9(1)) v4 ==== 765+0+0 (533631643 0 0) 0x56ade80 con 0x4eb23c0 -4533> 2014-08-13 17:23:21.660082 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4532> 2014-08-13 17:23:21.660130 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.643462, event: header_read, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4531> 2014-08-13 17:23:21.660138 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.643462, event: throttled, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4530> 2014-08-13 17:23:21.660148 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.643522, event: all_read, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4529> 2014-08-13 17:23:21.660167 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.660127, event: dispatched, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4528> 2014-08-13 17:23:21.660175 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.660174, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4527> 2014-08-13 17:23:21.660186 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.660186, event: started, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4526> 2014-08-13 17:23:21.660214 7fcd0964c700 5 -- op tracker -- , seq: 201, time: 2014-08-13 17:23:21.660213, event: done, request: pg_info(1 pgs e194477:22.2b9(1)) v4 -4525> 2014-08-13 17:23:21.659935 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: transitioning to Primary -4524> 2014-08-13 17:23:21.660242 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Start 0.000352 0 0.000000 -4523> 2014-08-13 17:23:21.660276 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary -4522> 2014-08-13 17:23:21.660295 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Peering -4521> 2014-08-13 17:23:21.660313 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetInfo -4520> 2014-08-13 17:23:21.660487 7fcce7b0a700 1 -- 192.168.1.115:6820/31924 >> :/0 pipe(0x5935280 sd=292 :6820 s=0 pgs=0 cs=0 l=0 c=0x59f8f20).accept sd=292 192.168.1.103:51543/0 -4519> 2014-08-13 17:23:21.660512 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit Started/Stray 0.706918 4 0.000348 -4518> 2014-08-13 17:23:21.660547 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit Started 0.707055 0 0.000000 -4517> 2014-08-13 17:23:21.660574 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194476/194473) [4,55,61]/[50,65,356] r=-1 lpr=194476 pi=193747-194475/26 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter Reset -4516> 2014-08-13 17:23:21.660647 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Reset 0.000074 1 0.000133 -4515> 2014-08-13 17:23:21.660676 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started -4514> 2014-08-13 17:23:21.660698 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Start -4513> 2014-08-13 17:23:21.660715 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: transitioning to Primary -4512> 2014-08-13 17:23:21.660734 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Start 0.000036 0 0.000000 -4511> 2014-08-13 17:23:21.660753 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary -4510> 2014-08-13 17:23:21.660771 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Peering -4509> 2014-08-13 17:23:21.660788 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetInfo -4508> 2014-08-13 17:23:21.661041 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.706292 4 0.000242 -4507> 2014-08-13 17:23:21.661077 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] exit Started 0.706423 0 0.000000 -4506> 2014-08-13 17:23:21.661102 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194476/192970) [371,4,21]/[371,21,24] r=-1 lpr=194476 pi=192977-194475/25 crt=192888'459 lcod 0'0 remapped NOTIFY] enter Reset -4505> 2014-08-13 17:23:21.660413 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.692288 4 0.000289 -4504> 2014-08-13 17:23:21.661157 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.741480 0 0.000000 -4503> 2014-08-13 17:23:21.661184 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x52b9680 con 0x4ec7e80 -4502> 2014-08-13 17:23:21.661213 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -4501> 2014-08-13 17:23:21.661230 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 6 ==== pg_notify(22.99(23) epoch 194477) v5 ==== 2003+0+0 (3218222013 0 0) 0x56ad940 con 0x4eb18c0 -4500> 2014-08-13 17:23:21.661219 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4499> 2014-08-13 17:23:21.661255 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.646609, event: header_read, request: pg_notify(22.99(23) epoch 194477) v5 -4498> 2014-08-13 17:23:21.661270 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.646610, event: throttled, request: pg_notify(22.99(23) epoch 194477) v5 -4497> 2014-08-13 17:23:21.661276 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.646717, event: all_read, request: pg_notify(22.99(23) epoch 194477) v5 -4496> 2014-08-13 17:23:21.661282 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.661252, event: dispatched, request: pg_notify(22.99(23) epoch 194477) v5 -4495> 2014-08-13 17:23:21.661289 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.661289, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194477) v5 -4494> 2014-08-13 17:23:21.661298 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.661297, event: started, request: pg_notify(22.99(23) epoch 194477) v5 -4493> 2014-08-13 17:23:21.661325 7fcd0964c700 5 -- op tracker -- , seq: 202, time: 2014-08-13 17:23:21.661325, event: done, request: pg_notify(22.99(23) epoch 194477) v5 -4492> 2014-08-13 17:23:21.661351 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 4 ==== pg_info(1 pgs e194477:22.630(1)) v4 ==== 765+0+0 (676913710 0 0) 0x54d2680 con 0x4eb40a0 -4491> 2014-08-13 17:23:21.661357 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4490> 2014-08-13 17:23:21.661375 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.642637, event: header_read, request: pg_info(1 pgs e194477:22.630(1)) v4 -4489> 2014-08-13 17:23:21.661385 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.642638, event: throttled, request: pg_info(1 pgs e194477:22.630(1)) v4 -4488> 2014-08-13 17:23:21.661393 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.642697, event: all_read, request: pg_info(1 pgs e194477:22.630(1)) v4 -4487> 2014-08-13 17:23:21.661400 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.661370, event: dispatched, request: pg_info(1 pgs e194477:22.630(1)) v4 -4486> 2014-08-13 17:23:21.661407 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.661407, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.630(1)) v4 -4485> 2014-08-13 17:23:21.661419 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.661418, event: started, request: pg_info(1 pgs e194477:22.630(1)) v4 -4484> 2014-08-13 17:23:21.661446 7fcd0964c700 5 -- op tracker -- , seq: 203, time: 2014-08-13 17:23:21.661446, event: done, request: pg_info(1 pgs e194477:22.630(1)) v4 -4483> 2014-08-13 17:23:21.661471 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 4 ==== pg_notify(22.1f1(27) epoch 194477) v5 ==== 2231+0+0 (2120929369 0 0) 0x54d2300 con 0x4eb2680 -4482> 2014-08-13 17:23:21.661493 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.643221, event: header_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -4481> 2014-08-13 17:23:21.661503 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.643223, event: throttled, request: pg_notify(22.1f1(27) epoch 194477) v5 -4480> 2014-08-13 17:23:21.661513 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.643332, event: all_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -4479> 2014-08-13 17:23:21.661522 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.661490, event: dispatched, request: pg_notify(22.1f1(27) epoch 194477) v5 -4478> 2014-08-13 17:23:21.661529 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.661529, event: waiting_for_osdmap, request: pg_notify(22.1f1(27) epoch 194477) v5 -4477> 2014-08-13 17:23:21.661544 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.661544, event: started, request: pg_notify(22.1f1(27) epoch 194477) v5 -4476> 2014-08-13 17:23:21.661575 7fcd0964c700 5 -- op tracker -- , seq: 204, time: 2014-08-13 17:23:21.661575, event: done, request: pg_notify(22.1f1(27) epoch 194477) v5 -4475> 2014-08-13 17:23:21.661557 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.689298 2 0.000096 -4474> 2014-08-13 17:23:21.661604 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 6 ==== pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 ==== 1851+0+0 (1069779892 0 0) 0x56ae740 con 0x4eb7d20 -4473> 2014-08-13 17:23:21.661629 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.643103, event: header_read, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4472> 2014-08-13 17:23:21.661623 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.740719 0 0.000000 -4471> 2014-08-13 17:23:21.661645 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.643105, event: throttled, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4470> 2014-08-13 17:23:21.661654 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.643290, event: all_read, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4469> 2014-08-13 17:23:21.661652 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194471 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4468> 2014-08-13 17:23:21.661668 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.661626, event: dispatched, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4467> 2014-08-13 17:23:21.661887 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.661887, event: waiting_for_osdmap, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4466> 2014-08-13 17:23:21.661916 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.661916, event: started, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4465> 2014-08-13 17:23:21.661962 7fcd0964c700 5 -- op tracker -- , seq: 205, time: 2014-08-13 17:23:21.661962, event: done, request: pg_notify(24.136(6),22.6bd(1) epoch 194477) v5 -4464> 2014-08-13 17:23:21.661968 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4463> 2014-08-13 17:23:21.661993 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.296 192.168.1.109:6806/31286 4 ==== pg_notify(24.181(28) epoch 194477) v5 ==== 2316+0+0 (2840360595 0 0) 0x54d0a80 con 0x4f81600 -4462> 2014-08-13 17:23:21.662020 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.645124, event: header_read, request: pg_notify(24.181(28) epoch 194477) v5 -4461> 2014-08-13 17:23:21.662030 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.645125, event: throttled, request: pg_notify(24.181(28) epoch 194477) v5 -4460> 2014-08-13 17:23:21.662040 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.645240, event: all_read, request: pg_notify(24.181(28) epoch 194477) v5 -4459> 2014-08-13 17:23:21.662050 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.662017, event: dispatched, request: pg_notify(24.181(28) epoch 194477) v5 -4458> 2014-08-13 17:23:21.662060 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.662060, event: waiting_for_osdmap, request: pg_notify(24.181(28) epoch 194477) v5 -4457> 2014-08-13 17:23:21.662072 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.662071, event: started, request: pg_notify(24.181(28) epoch 194477) v5 -4456> 2014-08-13 17:23:21.662096 7fcd0964c700 5 -- op tracker -- , seq: 206, time: 2014-08-13 17:23:21.662096, event: done, request: pg_notify(24.181(28) epoch 194477) v5 -4455> 2014-08-13 17:23:21.662133 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 4 ==== pg_info(1 pgs e194477:22.2d1(1)) v4 ==== 765+0+0 (3006076053 0 0) 0x54d5cc0 con 0x4eb6720 -4454> 2014-08-13 17:23:21.662157 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.646073, event: header_read, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4453> 2014-08-13 17:23:21.662165 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.646075, event: throttled, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4452> 2014-08-13 17:23:21.662171 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.646140, event: all_read, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4451> 2014-08-13 17:23:21.662177 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.662154, event: dispatched, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4450> 2014-08-13 17:23:21.662183 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.662183, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4449> 2014-08-13 17:23:21.662203 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.662191, event: started, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4448> 2014-08-13 17:23:21.662267 7fcd0964c700 5 -- op tracker -- , seq: 207, time: 2014-08-13 17:23:21.662267, event: done, request: pg_info(1 pgs e194477:22.2d1(1)) v4 -4447> 2014-08-13 17:23:21.662287 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 4 ==== pg_query(24.20e epoch 194477) v3 ==== 144+0+0 (3473736140 0 0) 0x2f97bc0 con 0x4f80b00 -4446> 2014-08-13 17:23:21.662304 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.640273, event: header_read, request: pg_query(24.20e epoch 194477) v3 -4445> 2014-08-13 17:23:21.662311 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.640275, event: throttled, request: pg_query(24.20e epoch 194477) v3 -4444> 2014-08-13 17:23:21.662316 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.640322, event: all_read, request: pg_query(24.20e epoch 194477) v3 -4443> 2014-08-13 17:23:21.662321 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.662302, event: dispatched, request: pg_query(24.20e epoch 194477) v3 -4442> 2014-08-13 17:23:21.662327 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.662327, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194477) v3 -4441> 2014-08-13 17:23:21.662335 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.662335, event: started, request: pg_query(24.20e epoch 194477) v3 -4440> 2014-08-13 17:23:21.662353 7fcd0964c700 5 -- op tracker -- , seq: 208, time: 2014-08-13 17:23:21.662353, event: done, request: pg_query(24.20e epoch 194477) v3 -4439> 2014-08-13 17:23:21.662347 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.693803 4 0.000357 -4438> 2014-08-13 17:23:21.662366 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.19 192.168.1.115:6818/61186 4 ==== pg_info(1 pgs e194477:24.3bf(5)) v4 ==== 1015+0+0 (239183506 0 0) 0x56afc40 con 0x4f84ba0 -4437> 2014-08-13 17:23:21.662382 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.639226, event: header_read, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4436> 2014-08-13 17:23:21.662376 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.739889 0 0.000000 -4435> 2014-08-13 17:23:21.662389 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.639230, event: throttled, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4434> 2014-08-13 17:23:21.662395 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.639310, event: all_read, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4433> 2014-08-13 17:23:21.662401 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.662380, event: dispatched, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4432> 2014-08-13 17:23:21.662408 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.662408, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4431> 2014-08-13 17:23:21.662399 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194471 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4430> 2014-08-13 17:23:21.662417 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.662416, event: started, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4429> 2014-08-13 17:23:21.662437 7fcd0964c700 5 -- op tracker -- , seq: 209, time: 2014-08-13 17:23:21.662437, event: done, request: pg_info(1 pgs e194477:24.3bf(5)) v4 -4428> 2014-08-13 17:23:21.662452 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550e540 con 0x4f81fa0 -4427> 2014-08-13 17:23:21.662471 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4426> 2014-08-13 17:23:21.662482 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 5 ==== pg_info(1 pgs e194477:24.1de(4)) v4 ==== 976+0+0 (2231284628 0 0) 0x54d1a40 con 0x4f83860 -4425> 2014-08-13 17:23:21.662504 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.645081, event: header_read, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4424> 2014-08-13 17:23:21.662512 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.645082, event: throttled, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4423> 2014-08-13 17:23:21.662518 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.645152, event: all_read, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4422> 2014-08-13 17:23:21.662524 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.662502, event: dispatched, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4421> 2014-08-13 17:23:21.662530 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.662530, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4420> 2014-08-13 17:23:21.662538 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.662538, event: started, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4419> 2014-08-13 17:23:21.662558 7fcd0964c700 5 -- op tracker -- , seq: 210, time: 2014-08-13 17:23:21.662558, event: done, request: pg_info(1 pgs e194477:24.1de(4)) v4 -4418> 2014-08-13 17:23:21.662555 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194474/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4417> 2014-08-13 17:23:21.662577 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bbcc0 con 0x4fd9a20 -4416> 2014-08-13 17:23:21.662598 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4415> 2014-08-13 17:23:21.662697 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.739687 3 0.000255 -4414> 2014-08-13 17:23:21.662729 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started 0.739802 0 0.000000 -4413> 2014-08-13 17:23:21.662748 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194473/194473) [26,30,340] r=-1 lpr=194473 pi=193125-194472/49 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Reset -4412> 2014-08-13 17:23:21.662830 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Reset 0.000081 1 0.000131 -4411> 2014-08-13 17:23:21.662860 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started -4410> 2014-08-13 17:23:21.662878 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Start -4409> 2014-08-13 17:23:21.662866 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.001764 1 0.000652 -4408> 2014-08-13 17:23:21.662894 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: transitioning to Primary -4407> 2014-08-13 17:23:21.662913 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Start 0.000035 0 0.000000 -4406> 2014-08-13 17:23:21.662932 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary -4405> 2014-08-13 17:23:21.662949 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Peering -4404> 2014-08-13 17:23:21.662939 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started -4403> 2014-08-13 17:23:21.662970 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start -4402> 2014-08-13 17:23:21.662966 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194461/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetInfo -4401> 2014-08-13 17:23:21.662977 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5509f80 con 0x4f870c0 -4400> 2014-08-13 17:23:21.663015 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -4399> 2014-08-13 17:23:21.663044 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4398> 2014-08-13 17:23:21.663047 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000077 0 0.000000 -4397> 2014-08-13 17:23:21.663065 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 4 ==== pg_notify(22.619(27) epoch 194477) v5 ==== 2231+0+0 (1628572869 0 0) 0x56aeac0 con 0x4fd9760 -4396> 2014-08-13 17:23:21.663089 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.638515, event: header_read, request: pg_notify(22.619(27) epoch 194477) v5 -4395> 2014-08-13 17:23:21.663103 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.638516, event: throttled, request: pg_notify(22.619(27) epoch 194477) v5 -4394> 2014-08-13 17:23:21.663113 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.638631, event: all_read, request: pg_notify(22.619(27) epoch 194477) v5 -4393> 2014-08-13 17:23:21.663123 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.663086, event: dispatched, request: pg_notify(22.619(27) epoch 194477) v5 -4392> 2014-08-13 17:23:21.663131 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.663131, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194477) v5 -4391> 2014-08-13 17:23:21.663140 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.663140, event: started, request: pg_notify(22.619(27) epoch 194477) v5 -4390> 2014-08-13 17:23:21.663165 7fcd0964c700 5 -- op tracker -- , seq: 211, time: 2014-08-13 17:23:21.663165, event: done, request: pg_notify(22.619(27) epoch 194477) v5 -4389> 2014-08-13 17:23:21.663202 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 8 ==== pg_info(1 pgs e194477:24.206(1)) v4 ==== 799+0+0 (4061741322 0 0) 0x5186e40 con 0x4f86e00 -4388> 2014-08-13 17:23:21.663220 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.640394, event: header_read, request: pg_info(1 pgs e194477:24.206(1)) v4 -4387> 2014-08-13 17:23:21.663227 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.640396, event: throttled, request: pg_info(1 pgs e194477:24.206(1)) v4 -4386> 2014-08-13 17:23:21.663233 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.640482, event: all_read, request: pg_info(1 pgs e194477:24.206(1)) v4 -4385> 2014-08-13 17:23:21.663239 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.663217, event: dispatched, request: pg_info(1 pgs e194477:24.206(1)) v4 -4384> 2014-08-13 17:23:21.663246 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.663246, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.206(1)) v4 -4383> 2014-08-13 17:23:21.663255 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.663254, event: started, request: pg_info(1 pgs e194477:24.206(1)) v4 -4382> 2014-08-13 17:23:21.663237 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.716367 4 0.000208 -4381> 2014-08-13 17:23:21.663277 7fcd0964c700 5 -- op tracker -- , seq: 212, time: 2014-08-13 17:23:21.663276, event: done, request: pg_info(1 pgs e194477:24.206(1)) v4 -4380> 2014-08-13 17:23:21.663267 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] exit Started 0.716473 0 0.000000 -4379> 2014-08-13 17:23:21.663288 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194476/194473) [4,374,31]/[374,31,20] r=-1 lpr=194476 pi=193306-194475/22 crt=191648'365 lcod 0'0 remapped NOTIFY] enter Reset -4378> 2014-08-13 17:23:21.663291 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 5 ==== pg_notify(24.1bb(30) epoch 194477) v5 ==== 2438+0+0 (2659733062 0 0) 0x56afa80 con 0x4fd8f20 -4377> 2014-08-13 17:23:21.663312 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.644003, event: header_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -4376> 2014-08-13 17:23:21.663319 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.644007, event: throttled, request: pg_notify(24.1bb(30) epoch 194477) v5 -4375> 2014-08-13 17:23:21.663325 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.646081, event: all_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -4374> 2014-08-13 17:23:21.663330 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.663310, event: dispatched, request: pg_notify(24.1bb(30) epoch 194477) v5 -4373> 2014-08-13 17:23:21.663337 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.663336, event: waiting_for_osdmap, request: pg_notify(24.1bb(30) epoch 194477) v5 -4372> 2014-08-13 17:23:21.663344 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.663344, event: started, request: pg_notify(24.1bb(30) epoch 194477) v5 -4371> 2014-08-13 17:23:21.663365 7fcd0964c700 5 -- op tracker -- , seq: 213, time: 2014-08-13 17:23:21.663364, event: done, request: pg_notify(24.1bb(30) epoch 194477) v5 -4370> 2014-08-13 17:23:21.663353 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000066 1 0.000114 -4369> 2014-08-13 17:23:21.663376 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started -4368> 2014-08-13 17:23:21.663391 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 7 ==== pg_query(22.653 epoch 194477) v3 ==== 144+0+0 (1342770675 0 0) 0x54fc1a0 con 0x4fd86e0 -4367> 2014-08-13 17:23:21.663391 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start -4366> 2014-08-13 17:23:21.663406 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.646585, event: header_read, request: pg_query(22.653 epoch 194477) v3 -4365> 2014-08-13 17:23:21.663414 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.646588, event: throttled, request: pg_query(22.653 epoch 194477) v3 -4364> 2014-08-13 17:23:21.663406 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -4363> 2014-08-13 17:23:21.663419 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.646649, event: all_read, request: pg_query(22.653 epoch 194477) v3 -4362> 2014-08-13 17:23:21.663425 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.663404, event: dispatched, request: pg_query(22.653 epoch 194477) v3 -4361> 2014-08-13 17:23:21.663431 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.663431, event: waiting_for_osdmap, request: pg_query(22.653 epoch 194477) v3 -4360> 2014-08-13 17:23:21.663426 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000035 0 0.000000 -4359> 2014-08-13 17:23:21.663439 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.663439, event: started, request: pg_query(22.653 epoch 194477) v3 -4358> 2014-08-13 17:23:21.663444 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -4357> 2014-08-13 17:23:21.663460 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4356> 2014-08-13 17:23:21.663093 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray -4355> 2014-08-13 17:23:21.663480 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4354> 2014-08-13 17:23:21.663617 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6817/53997 -- pg_notify(22.28f(21) epoch 194477) v5 -- ?+0 0x5888000 con 0x4e34780 -4353> 2014-08-13 17:23:21.663663 7fcd0964c700 5 -- op tracker -- , seq: 214, time: 2014-08-13 17:23:21.663663, event: done, request: pg_query(22.653 epoch 194477) v3 -4352> 2014-08-13 17:23:21.663691 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.363 192.168.1.102:6842/20017 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52be300 con 0x4fdb2e0 -4351> 2014-08-13 17:23:21.663717 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4350> 2014-08-13 17:23:21.663729 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x567b3c0 con 0x4fda520 -4349> 2014-08-13 17:23:21.663752 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4348> 2014-08-13 17:23:21.663762 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 6 ==== pg_notify(24.1f7(32) epoch 194477) v5 ==== 2556+0+0 (208552386 0 0) 0x5184ec0 con 0x4fdee00 -4347> 2014-08-13 17:23:21.663777 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.646781, event: header_read, request: pg_notify(24.1f7(32) epoch 194477) v5 -4346> 2014-08-13 17:23:21.663786 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.646812, event: throttled, request: pg_notify(24.1f7(32) epoch 194477) v5 -4345> 2014-08-13 17:23:21.663792 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.647023, event: all_read, request: pg_notify(24.1f7(32) epoch 194477) v5 -4344> 2014-08-13 17:23:21.663798 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.663774, event: dispatched, request: pg_notify(24.1f7(32) epoch 194477) v5 -4343> 2014-08-13 17:23:21.663804 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.663804, event: waiting_for_osdmap, request: pg_notify(24.1f7(32) epoch 194477) v5 -4342> 2014-08-13 17:23:21.663813 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.663813, event: started, request: pg_notify(24.1f7(32) epoch 194477) v5 -4341> 2014-08-13 17:23:21.663812 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6836/14923 -- pg_notify(22.30f(21) epoch 194477) v5 -- ?+0 0x588fa80 con 0x4ec2aa0 -4340> 2014-08-13 17:23:21.663838 7fcd0964c700 5 -- op tracker -- , seq: 215, time: 2014-08-13 17:23:21.663838, event: done, request: pg_notify(24.1f7(32) epoch 194477) v5 -4339> 2014-08-13 17:23:21.663843 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_notify(22.14a(21) epoch 194477) v5 -- ?+0 0x588f8c0 con 0x4e335a0 -4338> 2014-08-13 17:23:21.663871 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.379 192.168.1.108:6830/4861 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52bb840 con 0x4fdeb40 -4337> 2014-08-13 17:23:21.663893 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4336> 2014-08-13 17:23:21.663909 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5678240 con 0x4fde880 -4335> 2014-08-13 17:23:21.663924 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4334> 2014-08-13 17:23:21.663931 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5678900 con 0x4fde5c0 -4333> 2014-08-13 17:23:21.663946 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4332> 2014-08-13 17:23:21.663954 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 5 ==== pg_notify(22.39d(24) epoch 194477) v5 ==== 2058+0+0 (2245716050 0 0) 0x54d1c00 con 0x4fde300 -4331> 2014-08-13 17:23:21.663968 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.642982, event: header_read, request: pg_notify(22.39d(24) epoch 194477) v5 -4330> 2014-08-13 17:23:21.663976 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.642985, event: throttled, request: pg_notify(22.39d(24) epoch 194477) v5 -4329> 2014-08-13 17:23:21.663981 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.643100, event: all_read, request: pg_notify(22.39d(24) epoch 194477) v5 -4328> 2014-08-13 17:23:21.663987 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.663966, event: dispatched, request: pg_notify(22.39d(24) epoch 194477) v5 -4327> 2014-08-13 17:23:21.663982 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(26) epoch 194477) v5 -- ?+0 0x5888c40 con 0x4ec1ce0 -4326> 2014-08-13 17:23:21.663993 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.663993, event: waiting_for_osdmap, request: pg_notify(22.39d(24) epoch 194477) v5 -4325> 2014-08-13 17:23:21.664002 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.664001, event: started, request: pg_notify(22.39d(24) epoch 194477) v5 -4324> 2014-08-13 17:23:21.664022 7fcd0964c700 5 -- op tracker -- , seq: 216, time: 2014-08-13 17:23:21.664022, event: done, request: pg_notify(22.39d(24) epoch 194477) v5 -4323> 2014-08-13 17:23:21.664052 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 4 ==== pg_info(1 pgs e194477:22.69f(1)) v4 ==== 765+0+0 (640419502 0 0) 0x58896c0 con 0x4fde040 -4322> 2014-08-13 17:23:21.664069 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.647476, event: header_read, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4321> 2014-08-13 17:23:21.664076 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.647477, event: throttled, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4320> 2014-08-13 17:23:21.664082 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.647539, event: all_read, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4319> 2014-08-13 17:23:21.664088 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.664066, event: dispatched, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4318> 2014-08-13 17:23:21.664094 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.664094, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4317> 2014-08-13 17:23:21.664103 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.664102, event: started, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4316> 2014-08-13 17:23:21.664124 7fcd0964c700 5 -- op tracker -- , seq: 217, time: 2014-08-13 17:23:21.664124, event: done, request: pg_info(1 pgs e194477:22.69f(1)) v4 -4315> 2014-08-13 17:23:21.663672 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 0.708269 4 0.000348 -4314> 2014-08-13 17:23:21.664128 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(26) epoch 194477) v5 -- ?+0 0x5888fc0 con 0x4fde300 -4313> 2014-08-13 17:23:21.664139 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.24 192.168.1.107:6800/52669 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550ad00 con 0x4fd11e0 -4312> 2014-08-13 17:23:21.664157 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4311> 2014-08-13 17:23:21.664142 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Started 0.708838 0 0.000000 -4310> 2014-08-13 17:23:21.664166 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 8 ==== pg_info(1 pgs e194477:22.639(1)) v4 ==== 765+0+0 (282052485 0 0) 0x54d1f80 con 0x4fd3860 -4309> 2014-08-13 17:23:21.664181 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.639704, event: header_read, request: pg_info(1 pgs e194477:22.639(1)) v4 -4308> 2014-08-13 17:23:21.664188 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.639705, event: throttled, request: pg_info(1 pgs e194477:22.639(1)) v4 -4307> 2014-08-13 17:23:21.664176 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194476/192969) [356,4,334]/[356,32,322] r=-1 lpr=194476 pi=193150-194475/32 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Reset -4306> 2014-08-13 17:23:21.664198 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.639761, event: all_read, request: pg_info(1 pgs e194477:22.639(1)) v4 -4305> 2014-08-13 17:23:21.664214 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.664179, event: dispatched, request: pg_info(1 pgs e194477:22.639(1)) v4 -4304> 2014-08-13 17:23:21.664221 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.664221, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.639(1)) v4 -4303> 2014-08-13 17:23:21.664230 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.664229, event: started, request: pg_info(1 pgs e194477:22.639(1)) v4 -4302> 2014-08-13 17:23:21.664251 7fcd0964c700 5 -- op tracker -- , seq: 218, time: 2014-08-13 17:23:21.664250, event: done, request: pg_info(1 pgs e194477:22.639(1)) v4 -4301> 2014-08-13 17:23:21.664270 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x567ee40 con 0x4fd35a0 -4300> 2014-08-13 17:23:21.664287 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4299> 2014-08-13 17:23:21.664296 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bca40 con 0x4fd06e0 -4298> 2014-08-13 17:23:21.664292 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6812/43236 -- osd_map(194477..194477 src has 185139..194477) v3 -- ?+0 0x2e8b180 con 0x4e35120 -4297> 2014-08-13 17:23:21.664310 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4296> 2014-08-13 17:23:21.664318 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6812/43236 -- pg_notify(22.4aa(21) epoch 194477) v5 -- ?+0 0x54d0a80 con 0x4e35120 -4295> 2014-08-13 17:23:21.664319 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 5 ==== pg_notify(24.1f7(28) epoch 194477) v5 ==== 2320+0+0 (575630929 0 0) 0x54d2bc0 con 0x4fd5960 -4294> 2014-08-13 17:23:21.664333 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.645082, event: header_read, request: pg_notify(24.1f7(28) epoch 194477) v5 -4293> 2014-08-13 17:23:21.664342 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.645084, event: throttled, request: pg_notify(24.1f7(28) epoch 194477) v5 -4292> 2014-08-13 17:23:21.664348 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.645218, event: all_read, request: pg_notify(24.1f7(28) epoch 194477) v5 -4291> 2014-08-13 17:23:21.664353 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.664331, event: dispatched, request: pg_notify(24.1f7(28) epoch 194477) v5 -4290> 2014-08-13 17:23:21.664359 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.664359, event: waiting_for_osdmap, request: pg_notify(24.1f7(28) epoch 194477) v5 -4289> 2014-08-13 17:23:21.664367 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.664367, event: started, request: pg_notify(24.1f7(28) epoch 194477) v5 -4288> 2014-08-13 17:23:21.664387 7fcd0964c700 5 -- op tracker -- , seq: 219, time: 2014-08-13 17:23:21.664387, event: done, request: pg_notify(24.1f7(28) epoch 194477) v5 -4287> 2014-08-13 17:23:21.664414 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194477) v5 -- ?+0 0x54d2300 con 0x4fde5c0 -4286> 2014-08-13 17:23:21.664417 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 8 ==== pg_notify(22.39d(25) epoch 194477) v5 ==== 2117+0+0 (2057326087 0 0) 0x56aaa00 con 0x4fd32e0 -4285> 2014-08-13 17:23:21.664399 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000223 1 0.000576 -4284> 2014-08-13 17:23:21.664434 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.639695, event: header_read, request: pg_notify(22.39d(25) epoch 194477) v5 -4283> 2014-08-13 17:23:21.664435 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.116(22) epoch 194477) v5 -- ?+0 0x54d2680 con 0x4e32940 -4282> 2014-08-13 17:23:21.664441 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.639698, event: throttled, request: pg_notify(22.39d(25) epoch 194477) v5 -4281> 2014-08-13 17:23:21.664447 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.639812, event: all_read, request: pg_notify(22.39d(25) epoch 194477) v5 -4280> 2014-08-13 17:23:21.664437 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started -4279> 2014-08-13 17:23:21.664453 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.664432, event: dispatched, request: pg_notify(22.39d(25) epoch 194477) v5 -4278> 2014-08-13 17:23:21.664459 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(29) epoch 194477) v5 -- ?+0 0x54d01c0 con 0x4f83860 -4277> 2014-08-13 17:23:21.664457 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Start -4276> 2014-08-13 17:23:21.664460 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.664459, event: waiting_for_osdmap, request: pg_notify(22.39d(25) epoch 194477) v5 -4275> 2014-08-13 17:23:21.664474 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -4274> 2014-08-13 17:23:21.664497 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.664497, event: started, request: pg_notify(22.39d(25) epoch 194477) v5 -4273> 2014-08-13 17:23:21.664501 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000045 0 0.000000 -4272> 2014-08-13 17:23:21.664519 7fcd0964c700 5 -- op tracker -- , seq: 220, time: 2014-08-13 17:23:21.664519, event: done, request: pg_notify(22.39d(25) epoch 194477) v5 -4271> 2014-08-13 17:23:21.664522 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -4270> 2014-08-13 17:23:21.664544 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 5 ==== pg_notify(22.3d9(42) epoch 194477) v5 ==== 2996+0+0 (3769652863 0 0) 0x56ac980 con 0x4fd3020 -4269> 2014-08-13 17:23:21.664564 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.640522, event: header_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -4268> 2014-08-13 17:23:21.664571 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.640524, event: throttled, request: pg_notify(22.3d9(42) epoch 194477) v5 -4267> 2014-08-13 17:23:21.664577 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.640652, event: all_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -4266> 2014-08-13 17:23:21.664583 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.664561, event: dispatched, request: pg_notify(22.3d9(42) epoch 194477) v5 -4265> 2014-08-13 17:23:21.664589 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.664589, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194477) v5 -4264> 2014-08-13 17:23:21.664597 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.664597, event: started, request: pg_notify(22.3d9(42) epoch 194477) v5 -4263> 2014-08-13 17:23:21.664617 7fcd0964c700 5 -- op tracker -- , seq: 221, time: 2014-08-13 17:23:21.664617, event: done, request: pg_notify(22.3d9(42) epoch 194477) v5 -4262> 2014-08-13 17:23:21.664612 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_query(22.99 epoch 194477) v3 -- ?+0 0x2f94740 con 0x4e33de0 -4261> 2014-08-13 17:23:21.664637 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6802/58522 -- pg_query(24.34 epoch 194477) v3 -- ?+0 0x2f97440 con 0x55244c0 -4260> 2014-08-13 17:23:21.664642 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 5 ==== pg_notify(24.30b(33) epoch 194477) v5 ==== 2615+0+0 (4192084196 0 0) 0x54d1180 con 0x4fd2520 -4259> 2014-08-13 17:23:21.664653 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- pg_query(24.34 epoch 194477) v3 -- ?+0 0x2f910e0 con 0x4ec3f40 -4258> 2014-08-13 17:23:21.664658 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.642032, event: header_read, request: pg_notify(24.30b(33) epoch 194477) v5 -4257> 2014-08-13 17:23:21.664666 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.642034, event: throttled, request: pg_notify(24.30b(33) epoch 194477) v5 -4256> 2014-08-13 17:23:21.664671 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.642156, event: all_read, request: pg_notify(24.30b(33) epoch 194477) v5 -4255> 2014-08-13 17:23:21.664676 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.664656, event: dispatched, request: pg_notify(24.30b(33) epoch 194477) v5 -4254> 2014-08-13 17:23:21.664682 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.664682, event: waiting_for_osdmap, request: pg_notify(24.30b(33) epoch 194477) v5 -4253> 2014-08-13 17:23:21.664690 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.664690, event: started, request: pg_notify(24.30b(33) epoch 194477) v5 -4252> 2014-08-13 17:23:21.664709 7fcd0964c700 5 -- op tracker -- , seq: 222, time: 2014-08-13 17:23:21.664709, event: done, request: pg_notify(24.30b(33) epoch 194477) v5 -4251> 2014-08-13 17:23:21.664736 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 5 ==== pg_notify(22.775(27) epoch 194477) v5 ==== 2231+0+0 (2222502980 0 0) 0x54d5400 con 0x53011e0 -4250> 2014-08-13 17:23:21.664751 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.657457, event: header_read, request: pg_notify(22.775(27) epoch 194477) v5 -4249> 2014-08-13 17:23:21.664758 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.657460, event: throttled, request: pg_notify(22.775(27) epoch 194477) v5 -4248> 2014-08-13 17:23:21.664766 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.657584, event: all_read, request: pg_notify(22.775(27) epoch 194477) v5 -4247> 2014-08-13 17:23:21.664771 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.664749, event: dispatched, request: pg_notify(22.775(27) epoch 194477) v5 -4246> 2014-08-13 17:23:21.664777 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.664777, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194477) v5 -4245> 2014-08-13 17:23:21.664785 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.664785, event: started, request: pg_notify(22.775(27) epoch 194477) v5 -4244> 2014-08-13 17:23:21.664789 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_query(22.99 epoch 194477) v3 -- ?+0 0x2f94b00 con 0x4ec56a0 -4243> 2014-08-13 17:23:21.664804 7fcd0964c700 5 -- op tracker -- , seq: 223, time: 2014-08-13 17:23:21.664804, event: done, request: pg_notify(22.775(27) epoch 194477) v5 -4242> 2014-08-13 17:23:21.664826 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e1d40 con 0x5303700 -4241> 2014-08-13 17:23:21.664843 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4240> 2014-08-13 17:23:21.664858 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 5 ==== pg_notify(22.707(27) epoch 194477) v5 ==== 2231+0+0 (1488782986 0 0) 0x56aad80 con 0x5303440 -4239> 2014-08-13 17:23:21.664873 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.656110, event: header_read, request: pg_notify(22.707(27) epoch 194477) v5 -4238> 2014-08-13 17:23:21.664880 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.656113, event: throttled, request: pg_notify(22.707(27) epoch 194477) v5 -4237> 2014-08-13 17:23:21.664885 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.656237, event: all_read, request: pg_notify(22.707(27) epoch 194477) v5 -4236> 2014-08-13 17:23:21.664891 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.664870, event: dispatched, request: pg_notify(22.707(27) epoch 194477) v5 -4235> 2014-08-13 17:23:21.664897 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.664897, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194477) v5 -4234> 2014-08-13 17:23:21.664904 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.664904, event: started, request: pg_notify(22.707(27) epoch 194477) v5 -4233> 2014-08-13 17:23:21.664925 7fcd0964c700 5 -- op tracker -- , seq: 224, time: 2014-08-13 17:23:21.664925, event: done, request: pg_notify(22.707(27) epoch 194477) v5 -4232> 2014-08-13 17:23:21.664949 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 6 ==== pg_query(24.3f7 epoch 194477) v3 ==== 144+0+0 (3902582257 0 0) 0x54f8960 con 0x5305ee0 -4231> 2014-08-13 17:23:21.664964 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.647362, event: header_read, request: pg_query(24.3f7 epoch 194477) v3 -4230> 2014-08-13 17:23:21.664970 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.647364, event: throttled, request: pg_query(24.3f7 epoch 194477) v3 -4229> 2014-08-13 17:23:21.664975 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.647407, event: all_read, request: pg_query(24.3f7 epoch 194477) v3 -4228> 2014-08-13 17:23:21.664981 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.664962, event: dispatched, request: pg_query(24.3f7 epoch 194477) v3 -4227> 2014-08-13 17:23:21.664987 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.664987, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194477) v3 -4226> 2014-08-13 17:23:21.664994 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.664994, event: started, request: pg_query(24.3f7 epoch 194477) v3 -4225> 2014-08-13 17:23:21.665012 7fcd0964c700 5 -- op tracker -- , seq: 225, time: 2014-08-13 17:23:21.665011, event: done, request: pg_query(24.3f7 epoch 194477) v3 -4224> 2014-08-13 17:23:21.665026 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 5 ==== pg_notify(24.9b(26) epoch 194477) v5 ==== 2206+0+0 (1971894860 0 0) 0x4e50380 con 0x5305120 -4223> 2014-08-13 17:23:21.665010 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.696348 4 0.000231 -4222> 2014-08-13 17:23:21.665041 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.649636, event: header_read, request: pg_notify(24.9b(26) epoch 194477) v5 -4221> 2014-08-13 17:23:21.665048 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.649640, event: throttled, request: pg_notify(24.9b(26) epoch 194477) v5 -4220> 2014-08-13 17:23:21.665053 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.649764, event: all_read, request: pg_notify(24.9b(26) epoch 194477) v5 -4219> 2014-08-13 17:23:21.665059 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.665039, event: dispatched, request: pg_notify(24.9b(26) epoch 194477) v5 -4218> 2014-08-13 17:23:21.665046 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.712449 0 0.000000 -4217> 2014-08-13 17:23:21.665065 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.665065, event: waiting_for_osdmap, request: pg_notify(24.9b(26) epoch 194477) v5 -4216> 2014-08-13 17:23:21.665073 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.665073, event: started, request: pg_notify(24.9b(26) epoch 194477) v5 -4215> 2014-08-13 17:23:21.665071 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -4214> 2014-08-13 17:23:21.665095 7fcd0964c700 5 -- op tracker -- , seq: 226, time: 2014-08-13 17:23:21.665095, event: done, request: pg_notify(24.9b(26) epoch 194477) v5 -4213> 2014-08-13 17:23:21.665117 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bb3c0 con 0x5305c20 -4212> 2014-08-13 17:23:21.665134 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4211> 2014-08-13 17:23:21.665124 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194474/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -4210> 2014-08-13 17:23:21.665143 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5508b40 con 0x5305960 -4209> 2014-08-13 17:23:21.665157 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4208> 2014-08-13 17:23:21.665152 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_query(22.619 epoch 194477) v3 -- ?+0 0x54fa1c0 con 0x4eb0c60 -4207> 2014-08-13 17:23:21.665166 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.302 192.168.1.113:6810/15731 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550b840 con 0x552b9c0 -4206> 2014-08-13 17:23:21.665181 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4205> 2014-08-13 17:23:21.665190 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 5 ==== pg_notify(22.22c(22) epoch 194477) v5 ==== 1944+0+0 (748970980 0 0) 0x5187700 con 0x552b700 -4204> 2014-08-13 17:23:21.665216 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.649444, event: header_read, request: pg_notify(22.22c(22) epoch 194477) v5 -4203> 2014-08-13 17:23:21.665223 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.649445, event: throttled, request: pg_notify(22.22c(22) epoch 194477) v5 -4202> 2014-08-13 17:23:21.665229 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.649587, event: all_read, request: pg_notify(22.22c(22) epoch 194477) v5 -4201> 2014-08-13 17:23:21.665234 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.665213, event: dispatched, request: pg_notify(22.22c(22) epoch 194477) v5 -4200> 2014-08-13 17:23:21.665241 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.665240, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194477) v5 -4199> 2014-08-13 17:23:21.665248 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.665248, event: started, request: pg_notify(22.22c(22) epoch 194477) v5 -4198> 2014-08-13 17:23:21.665268 7fcd0964c700 5 -- op tracker -- , seq: 227, time: 2014-08-13 17:23:21.665268, event: done, request: pg_notify(22.22c(22) epoch 194477) v5 -4197> 2014-08-13 17:23:21.665286 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5678d80 con 0x552fa60 -4196> 2014-08-13 17:23:21.665306 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -4195> 2014-08-13 17:23:21.665314 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.63 192.168.1.111:6830/12680 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x567c380 con 0x5521e40 -4194> 2014-08-13 17:23:21.665331 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4193> 2014-08-13 17:23:21.665341 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 6 ==== pg_info(1 pgs e194477:24.e2(4)) v4 ==== 976+0+0 (3925902384 0 0) 0x5888e00 con 0x5520b00 -4192> 2014-08-13 17:23:21.665355 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.651615, event: header_read, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4191> 2014-08-13 17:23:21.665369 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.651616, event: throttled, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4190> 2014-08-13 17:23:21.665379 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.651729, event: all_read, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4189> 2014-08-13 17:23:21.665390 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.665353, event: dispatched, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4188> 2014-08-13 17:23:21.665398 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.665397, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4187> 2014-08-13 17:23:21.665406 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.665406, event: started, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4186> 2014-08-13 17:23:21.665427 7fcd0964c700 5 -- op tracker -- , seq: 228, time: 2014-08-13 17:23:21.665427, event: done, request: pg_info(1 pgs e194477:24.e2(4)) v4 -4185> 2014-08-13 17:23:21.665420 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6824/5505 -- pg_query(22.619 epoch 194477) v3 -- ?+0 0x54f9fe0 con 0x4e370c0 -4184> 2014-08-13 17:23:21.665454 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x52ba640 con 0x55f2100 -4183> 2014-08-13 17:23:21.665471 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -4182> 2014-08-13 17:23:21.665480 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x567e780 con 0x55244c0 -4181> 2014-08-13 17:23:21.665494 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4180> 2014-08-13 17:23:21.665502 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.314 192.168.1.101:6802/22303 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x5679f80 con 0x552d800 -4179> 2014-08-13 17:23:21.665516 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4178> 2014-08-13 17:23:21.665526 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.293 192.168.1.109:6816/33855 2 ==== pg_notify(22.3d9(46) epoch 194477) v5 ==== 3208+0+0 (821084124 0 0) 0x54d1dc0 con 0x5724e60 -4177> 2014-08-13 17:23:21.665540 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.651252, event: header_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -4176> 2014-08-13 17:23:21.665512 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started/Stray 0.712252 4 0.000282 -4175> 2014-08-13 17:23:21.665547 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.651253, event: throttled, request: pg_notify(22.3d9(46) epoch 194477) v5 -4174> 2014-08-13 17:23:21.665554 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.651382, event: all_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -4173> 2014-08-13 17:23:21.665559 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.665538, event: dispatched, request: pg_notify(22.3d9(46) epoch 194477) v5 -4172> 2014-08-13 17:23:21.665565 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.665565, event: waiting_for_osdmap, request: pg_notify(22.3d9(46) epoch 194477) v5 -4171> 2014-08-13 17:23:21.665554 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started 0.712403 0 0.000000 -4170> 2014-08-13 17:23:21.665573 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.665573, event: started, request: pg_notify(22.3d9(46) epoch 194477) v5 -4169> 2014-08-13 17:23:21.665566 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_query(22.619 epoch 194477) v3 -- ?+0 0x54faee0 con 0x4fd9760 -4168> 2014-08-13 17:23:21.665577 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194476/194473) [4,344,296]/[296,320,307] r=-1 lpr=194476 pi=193679-194475/31 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Reset -4167> 2014-08-13 17:23:21.665597 7fcd0964c700 5 -- op tracker -- , seq: 229, time: 2014-08-13 17:23:21.665597, event: done, request: pg_notify(22.3d9(46) epoch 194477) v5 -4166> 2014-08-13 17:23:21.665604 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_query(24.34 epoch 194477) v3 -- ?+0 0x54fab20 con 0x4e35540 -4165> 2014-08-13 17:23:21.665625 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_query(22.99 epoch 194477) v3 -- ?+0 0x54f9e00 con 0x4eb18c0 -4164> 2014-08-13 17:23:21.665624 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.47 192.168.1.111:6828/12102 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x51e69c0 con 0x57232e0 -4163> 2014-08-13 17:23:21.665641 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -4162> 2014-08-13 17:23:21.665651 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.382 192.168.1.108:6800/30527 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52bdc40 con 0x552d280 -4161> 2014-08-13 17:23:21.665665 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -4160> 2014-08-13 17:23:21.665676 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 9 ==== pg_info(1 pgs e194477:24.1e6(1)) v4 ==== 799+0+0 (4128952354 0 0) 0x56abf00 con 0x4e314a0 -4159> 2014-08-13 17:23:21.665690 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.647780, event: header_read, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4158> 2014-08-13 17:23:21.665701 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.647781, event: throttled, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4157> 2014-08-13 17:23:21.665715 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.647861, event: all_read, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4156> 2014-08-13 17:23:21.665721 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.665688, event: dispatched, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4155> 2014-08-13 17:23:21.665731 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.665731, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4154> 2014-08-13 17:23:21.665740 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.665740, event: started, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4153> 2014-08-13 17:23:21.665762 7fcd0964c700 5 -- op tracker -- , seq: 230, time: 2014-08-13 17:23:21.665761, event: done, request: pg_info(1 pgs e194477:24.1e6(1)) v4 -4152> 2014-08-13 17:23:21.665781 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 8 ==== pg_notify(22.1d2(24) epoch 194477) v5 ==== 2062+0+0 (1449907918 0 0) 0x54d3b80 con 0x4e335a0 -4151> 2014-08-13 17:23:21.665797 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.647126, event: header_read, request: pg_notify(22.1d2(24) epoch 194477) v5 -4150> 2014-08-13 17:23:21.665804 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.647128, event: throttled, request: pg_notify(22.1d2(24) epoch 194477) v5 -4149> 2014-08-13 17:23:21.665810 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.647256, event: all_read, request: pg_notify(22.1d2(24) epoch 194477) v5 -4148> 2014-08-13 17:23:21.665815 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.665795, event: dispatched, request: pg_notify(22.1d2(24) epoch 194477) v5 -4147> 2014-08-13 17:23:21.665792 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Reset 0.000214 1 0.000257 -4146> 2014-08-13 17:23:21.665821 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.665821, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194477) v5 -4145> 2014-08-13 17:23:21.665784 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_info(1 pgs e194477:24.3d(27)) v4 -- ?+0 0x54d4980 con 0x4e33de0 -4144> 2014-08-13 17:23:21.665829 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.665829, event: started, request: pg_notify(22.1d2(24) epoch 194477) v5 -4143> 2014-08-13 17:23:21.665824 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started -4142> 2014-08-13 17:23:21.665850 7fcd0964c700 5 -- op tracker -- , seq: 231, time: 2014-08-13 17:23:21.665850, event: done, request: pg_notify(22.1d2(24) epoch 194477) v5 -4141> 2014-08-13 17:23:21.665842 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Start -4140> 2014-08-13 17:23:21.665860 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: transitioning to Primary -4139> 2014-08-13 17:23:21.665875 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 6 ==== pg_info(1 pgs e194477:22.116(3)) v4 ==== 883+0+0 (1950771814 0 0) 0x54d6ac0 con 0x4e32940 -4138> 2014-08-13 17:23:21.665879 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Start 0.000037 0 0.000000 -4137> 2014-08-13 17:23:21.665891 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.641325, event: header_read, request: pg_info(1 pgs e194477:22.116(3)) v4 -4136> 2014-08-13 17:23:21.665899 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.641327, event: throttled, request: pg_info(1 pgs e194477:22.116(3)) v4 -4135> 2014-08-13 17:23:21.665905 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.641426, event: all_read, request: pg_info(1 pgs e194477:22.116(3)) v4 -4134> 2014-08-13 17:23:21.665911 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.665889, event: dispatched, request: pg_info(1 pgs e194477:22.116(3)) v4 -4133> 2014-08-13 17:23:21.665901 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary -4132> 2014-08-13 17:23:21.665918 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.665918, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.116(3)) v4 -4131> 2014-08-13 17:23:21.665927 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.665926, event: started, request: pg_info(1 pgs e194477:22.116(3)) v4 -4130> 2014-08-13 17:23:21.665956 7fcd0964c700 5 -- op tracker -- , seq: 232, time: 2014-08-13 17:23:21.665956, event: done, request: pg_info(1 pgs e194477:22.116(3)) v4 -4129> 2014-08-13 17:23:21.665919 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Peering -4128> 2014-08-13 17:23:21.665971 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetInfo -4127> 2014-08-13 17:23:21.665978 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6800/36920 -- pg_info(1 pgs e194477:24.3d(27)) v4 -- ?+0 0x56ae740 con 0x552f7a0 -4126> 2014-08-13 17:23:21.665977 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 5 ==== pg_notify(24.3d(4),22.99(22) epoch 194477) v5 ==== 2912+0+0 (2246666273 0 0) 0x56a8fc0 con 0x4e33de0 -4125> 2014-08-13 17:23:21.666006 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.653480, event: header_read, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4124> 2014-08-13 17:23:21.666019 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.653481, event: throttled, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4123> 2014-08-13 17:23:21.666026 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.653595, event: all_read, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4122> 2014-08-13 17:23:21.666032 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.666004, event: dispatched, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4121> 2014-08-13 17:23:21.666036 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_info(1 pgs e194477:22.113(23)) v4 -- ?+0 0x56ad940 con 0x4ec09a0 -4120> 2014-08-13 17:23:21.666043 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.666043, event: waiting_for_osdmap, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4119> 2014-08-13 17:23:21.666052 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.666051, event: started, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4118> 2014-08-13 17:23:21.666085 7fcd0964c700 5 -- op tracker -- , seq: 233, time: 2014-08-13 17:23:21.666084, event: done, request: pg_notify(24.3d(4),22.99(22) epoch 194477) v5 -4117> 2014-08-13 17:23:21.666093 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_info(1 pgs e194477:22.1d2(23)) v4 -- ?+0 0x56ade80 con 0x4e35ac0 -4116> 2014-08-13 17:23:21.666112 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 5 ==== pg_notify(22.84(1),24.b6(29) epoch 194477) v5 ==== 3136+0+0 (2063668183 0 0) 0x5187540 con 0x4e33f40 -4115> 2014-08-13 17:23:21.666130 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.651466, event: header_read, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4114> 2014-08-13 17:23:21.666137 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.651467, event: throttled, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4113> 2014-08-13 17:23:21.666144 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.652759, event: all_read, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4112> 2014-08-13 17:23:21.666150 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.666128, event: dispatched, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4111> 2014-08-13 17:23:21.666156 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.666156, event: waiting_for_osdmap, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4110> 2014-08-13 17:23:21.666164 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.666164, event: started, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4109> 2014-08-13 17:23:21.666197 7fcd0964c700 5 -- op tracker -- , seq: 234, time: 2014-08-13 17:23:21.666196, event: done, request: pg_notify(22.84(1),24.b6(29) epoch 194477) v5 -4108> 2014-08-13 17:23:21.666228 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 6 ==== pg_notify(22.3af(5) epoch 194477) v5 ==== 1001+0+0 (3495433800 0 0) 0x54d3f00 con 0x4e35ac0 -4107> 2014-08-13 17:23:21.666250 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.655626, event: header_read, request: pg_notify(22.3af(5) epoch 194477) v5 -4106> 2014-08-13 17:23:21.666267 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.655627, event: throttled, request: pg_notify(22.3af(5) epoch 194477) v5 -4105> 2014-08-13 17:23:21.666273 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.657718, event: all_read, request: pg_notify(22.3af(5) epoch 194477) v5 -4104> 2014-08-13 17:23:21.666279 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.666248, event: dispatched, request: pg_notify(22.3af(5) epoch 194477) v5 -4103> 2014-08-13 17:23:21.666286 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.666285, event: waiting_for_osdmap, request: pg_notify(22.3af(5) epoch 194477) v5 -4102> 2014-08-13 17:23:21.666298 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.666298, event: started, request: pg_notify(22.3af(5) epoch 194477) v5 -4101> 2014-08-13 17:23:21.666323 7fcd0964c700 5 -- op tracker -- , seq: 235, time: 2014-08-13 17:23:21.666323, event: done, request: pg_notify(22.3af(5) epoch 194477) v5 -4100> 2014-08-13 17:23:21.666343 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 6 ==== pg_notify(24.30b(6) epoch 194477) v5 ==== 1094+0+0 (3266582469 0 0) 0x54d1340 con 0x4e35800 -4099> 2014-08-13 17:23:21.666371 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.650669, event: header_read, request: pg_notify(24.30b(6) epoch 194477) v5 -4098> 2014-08-13 17:23:21.666333 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_info(1 pgs e194477:22.f3(21)) v4 -- ?+0 0x56ad780 con 0x4e314a0 -4097> 2014-08-13 17:23:21.666383 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.650670, event: throttled, request: pg_notify(24.30b(6) epoch 194477) v5 -4096> 2014-08-13 17:23:21.666389 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.650781, event: all_read, request: pg_notify(24.30b(6) epoch 194477) v5 -4095> 2014-08-13 17:23:21.666398 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.666369, event: dispatched, request: pg_notify(24.30b(6) epoch 194477) v5 -4094> 2014-08-13 17:23:21.666404 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.666404, event: waiting_for_osdmap, request: pg_notify(24.30b(6) epoch 194477) v5 -4093> 2014-08-13 17:23:21.666412 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.666412, event: started, request: pg_notify(24.30b(6) epoch 194477) v5 -4092> 2014-08-13 17:23:21.666410 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6820/25633 -- pg_info(1 pgs e194477:24.9b(26)) v4 -- ?+0 0x56ae900 con 0x4ec27e0 -4091> 2014-08-13 17:23:21.666432 7fcd0964c700 5 -- op tracker -- , seq: 236, time: 2014-08-13 17:23:21.666431, event: done, request: pg_notify(24.30b(6) epoch 194477) v5 -4090> 2014-08-13 17:23:21.666447 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 9 ==== pg_info(1 pgs e194477:24.369(1)) v4 ==== 799+0+0 (3275119278 0 0) 0x56aabc0 con 0x4ec09a0 -4089> 2014-08-13 17:23:21.666451 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6814/16727 -- pg_info(1 pgs e194477:22.22c(23)) v4 -- ?+0 0x56aad80 con 0x4e35800 -4088> 2014-08-13 17:23:21.666463 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.641257, event: header_read, request: pg_info(1 pgs e194477:24.369(1)) v4 -4087> 2014-08-13 17:23:21.666470 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.641259, event: throttled, request: pg_info(1 pgs e194477:24.369(1)) v4 -4086> 2014-08-13 17:23:21.666476 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.641333, event: all_read, request: pg_info(1 pgs e194477:24.369(1)) v4 -4085> 2014-08-13 17:23:21.666482 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.666461, event: dispatched, request: pg_info(1 pgs e194477:24.369(1)) v4 -4084> 2014-08-13 17:23:21.666488 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.666488, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.369(1)) v4 -4083> 2014-08-13 17:23:21.666496 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.666495, event: started, request: pg_info(1 pgs e194477:24.369(1)) v4 -4082> 2014-08-13 17:23:21.666514 7fcd0964c700 5 -- op tracker -- , seq: 237, time: 2014-08-13 17:23:21.666514, event: done, request: pg_info(1 pgs e194477:24.369(1)) v4 -4081> 2014-08-13 17:23:21.666527 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x52b8b40 con 0x4e370c0 -4080> 2014-08-13 17:23:21.666549 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -4079> 2014-08-13 17:23:21.666566 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 4 ==== pg_info(1 pgs e194477:22.30f(1)) v4 ==== 765+0+0 (1065930350 0 0) 0x56ae580 con 0x4ec2aa0 -4078> 2014-08-13 17:23:21.666584 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.647645, event: header_read, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4077> 2014-08-13 17:23:21.666594 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.647646, event: throttled, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4076> 2014-08-13 17:23:21.666604 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.647708, event: all_read, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4075> 2014-08-13 17:23:21.666612 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.666582, event: dispatched, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4074> 2014-08-13 17:23:21.666620 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.666619, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4073> 2014-08-13 17:23:21.666611 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6802/30768 -- pg_info(1 pgs e194477:24.3d(27)) v4 -- ?+0 0x56ac980 con 0x4fd06e0 -4072> 2014-08-13 17:23:21.666628 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.666627, event: started, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4071> 2014-08-13 17:23:21.666647 7fcd0964c700 5 -- op tracker -- , seq: 238, time: 2014-08-13 17:23:21.666647, event: done, request: pg_info(1 pgs e194477:22.30f(1)) v4 -4070> 2014-08-13 17:23:21.666650 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6816/17521 -- pg_info(1 pgs e194477:22.f3(21)) v4 -- ?+0 0x56aaa00 con 0x5307a60 -4069> 2014-08-13 17:23:21.666665 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 8 ==== pg_info(1 pgs e194477:24.33a(1)) v4 ==== 799+0+0 (1440889968 0 0) 0x54d32c0 con 0x4ec1ce0 -4068> 2014-08-13 17:23:21.666685 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.644141, event: header_read, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4067> 2014-08-13 17:23:21.666692 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.644143, event: throttled, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4066> 2014-08-13 17:23:21.666691 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6814/65209 -- pg_info(1 pgs e194477:22.22c(23)) v4 -- ?+0 0x56afa80 con 0x4fde5c0 -4065> 2014-08-13 17:23:21.666698 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.644222, event: all_read, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4064> 2014-08-13 17:23:21.666704 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.666683, event: dispatched, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4063> 2014-08-13 17:23:21.666711 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.666711, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4062> 2014-08-13 17:23:21.666719 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.666718, event: started, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4061> 2014-08-13 17:23:21.666737 7fcd0964c700 5 -- op tracker -- , seq: 239, time: 2014-08-13 17:23:21.666737, event: done, request: pg_info(1 pgs e194477:24.33a(1)) v4 -4060> 2014-08-13 17:23:21.666751 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 5 ==== pg_info(1 pgs e194477:22.791(1)) v4 ==== 765+0+0 (540146448 0 0) 0x54d7380 con 0x4ec3f40 -4059> 2014-08-13 17:23:21.666766 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.646304, event: header_read, request: pg_info(1 pgs e194477:22.791(1)) v4 -4058> 2014-08-13 17:23:21.666772 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.646305, event: throttled, request: pg_info(1 pgs e194477:22.791(1)) v4 -4057> 2014-08-13 17:23:21.666778 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.646362, event: all_read, request: pg_info(1 pgs e194477:22.791(1)) v4 -4056> 2014-08-13 17:23:21.666784 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.666764, event: dispatched, request: pg_info(1 pgs e194477:22.791(1)) v4 -4055> 2014-08-13 17:23:21.666790 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.666790, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.791(1)) v4 -4054> 2014-08-13 17:23:21.666797 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.666797, event: started, request: pg_info(1 pgs e194477:22.791(1)) v4 -4053> 2014-08-13 17:23:21.666816 7fcd0964c700 5 -- op tracker -- , seq: 240, time: 2014-08-13 17:23:21.666815, event: done, request: pg_info(1 pgs e194477:22.791(1)) v4 -4052> 2014-08-13 17:23:21.666834 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6818/45268 -- pg_info(1 pgs e194477:22.113(23)) v4 -- ?+0 0x56aeac0 con 0x5305c20 -4051> 2014-08-13 17:23:21.666872 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_info(2 pgs e194477:2.2a(2),1.2b(2)) v4 -- ?+0 0x56afc40 con 0x4e35540 -4050> 2014-08-13 17:23:21.666867 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 5 ==== pg_query(24.1e4 epoch 194477) v3 ==== 144+0+0 (193511596 0 0) 0x2f90b40 con 0x4ec56a0 -4049> 2014-08-13 17:23:21.666897 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6800/61726 -- pg_info(1 pgs e194477:22.1d2(23)) v4 -- ?+0 0x54d5400 con 0x552fa60 -4048> 2014-08-13 17:23:21.666898 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.639282, event: header_read, request: pg_query(24.1e4 epoch 194477) v3 -4047> 2014-08-13 17:23:21.666909 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.639284, event: throttled, request: pg_query(24.1e4 epoch 194477) v3 -4046> 2014-08-13 17:23:21.666915 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.639354, event: all_read, request: pg_query(24.1e4 epoch 194477) v3 -4045> 2014-08-13 17:23:21.666921 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.666893, event: dispatched, request: pg_query(24.1e4 epoch 194477) v3 -4044> 2014-08-13 17:23:21.666928 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.666928, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194477) v3 -4043> 2014-08-13 17:23:21.666938 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.666937, event: started, request: pg_query(24.1e4 epoch 194477) v3 -4042> 2014-08-13 17:23:21.666961 7fcd0964c700 5 -- op tracker -- , seq: 241, time: 2014-08-13 17:23:21.666961, event: done, request: pg_query(24.1e4 epoch 194477) v3 -4041> 2014-08-13 17:23:21.666979 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 10 ==== pg_info(1 pgs e194477:24.5e(1)) v4 ==== 799+0+0 (3045715463 0 0) 0x51847c0 con 0x4ec3440 -4040> 2014-08-13 17:23:21.666996 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.642024, event: header_read, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4039> 2014-08-13 17:23:21.667004 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.642025, event: throttled, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4038> 2014-08-13 17:23:21.667010 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.642079, event: all_read, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4037> 2014-08-13 17:23:21.667015 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.666994, event: dispatched, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4036> 2014-08-13 17:23:21.667022 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.667022, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4035> 2014-08-13 17:23:21.667035 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.667034, event: started, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4034> 2014-08-13 17:23:21.667072 7fcd0964c700 5 -- op tracker -- , seq: 242, time: 2014-08-13 17:23:21.667072, event: done, request: pg_info(1 pgs e194477:24.5e(1)) v4 -4033> 2014-08-13 17:23:21.667050 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6832/53185 -- pg_info(1 pgs e194477:24.9b(26)) v4 -- ?+0 0x54d1180 con 0x53056a0 -4032> 2014-08-13 17:23:21.667092 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 9 ==== pg_notify(24.136(26) epoch 194477) v5 ==== 2202+0+0 (2731362500 0 0) 0x54d6900 con 0x4ec4780 -4031> 2014-08-13 17:23:21.667104 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6842/8170 -- pg_info(1 pgs e194477:24.3d(27)) v4 -- ?+0 0x54d2bc0 con 0x4f86040 -4030> 2014-08-13 17:23:21.667109 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.642827, event: header_read, request: pg_notify(24.136(26) epoch 194477) v5 -4029> 2014-08-13 17:23:21.667117 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.642829, event: throttled, request: pg_notify(24.136(26) epoch 194477) v5 -4028> 2014-08-13 17:23:21.667122 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.642948, event: all_read, request: pg_notify(24.136(26) epoch 194477) v5 -4027> 2014-08-13 17:23:21.667128 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.667107, event: dispatched, request: pg_notify(24.136(26) epoch 194477) v5 -4026> 2014-08-13 17:23:21.667137 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.667137, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194477) v5 -4025> 2014-08-13 17:23:21.667147 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.667147, event: started, request: pg_notify(24.136(26) epoch 194477) v5 -4024> 2014-08-13 17:23:21.667144 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6840/55513 -- pg_info(1 pgs e194477:24.9b(26)) v4 -- ?+0 0x5b4a300 con 0x4fd3860 -4023> 2014-08-13 17:23:21.667179 7fcd0964c700 5 -- op tracker -- , seq: 243, time: 2014-08-13 17:23:21.667179, event: done, request: pg_notify(24.136(26) epoch 194477) v5 -4022> 2014-08-13 17:23:21.667177 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6840/19195 -- pg_info(1 pgs e194477:24.9b(26)) v4 -- ?+0 0x5b4a140 con 0x5305120 -4021> 2014-08-13 17:23:21.667225 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 5 ==== pg_notify(22.3e4(27) epoch 194477) v5 ==== 2231+0+0 (3981404535 0 0) 0x54d0700 con 0x4ec5d80 -4020> 2014-08-13 17:23:21.667242 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.658547, event: header_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -4019> 2014-08-13 17:23:21.667249 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.658549, event: throttled, request: pg_notify(22.3e4(27) epoch 194477) v5 -4018> 2014-08-13 17:23:21.667254 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.658754, event: all_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -4017> 2014-08-13 17:23:21.667260 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.667239, event: dispatched, request: pg_notify(22.3e4(27) epoch 194477) v5 -4016> 2014-08-13 17:23:21.667266 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.667266, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194477) v5 -4015> 2014-08-13 17:23:21.667274 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.667273, event: started, request: pg_notify(22.3e4(27) epoch 194477) v5 -4014> 2014-08-13 17:23:21.667294 7fcd0964c700 5 -- op tracker -- , seq: 244, time: 2014-08-13 17:23:21.667294, event: done, request: pg_notify(22.3e4(27) epoch 194477) v5 -4013> 2014-08-13 17:23:21.667317 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 7 ==== pg_notify(24.b6(32) epoch 194477) v5 ==== 2556+0+0 (371489873 0 0) 0x4e501c0 con 0x4ec5ac0 -4012> 2014-08-13 17:23:21.667332 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.639962, event: header_read, request: pg_notify(24.b6(32) epoch 194477) v5 -4011> 2014-08-13 17:23:21.667339 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.639963, event: throttled, request: pg_notify(24.b6(32) epoch 194477) v5 -4010> 2014-08-13 17:23:21.667344 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.640076, event: all_read, request: pg_notify(24.b6(32) epoch 194477) v5 -4009> 2014-08-13 17:23:21.667350 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.667330, event: dispatched, request: pg_notify(24.b6(32) epoch 194477) v5 -4008> 2014-08-13 17:23:21.667356 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.667356, event: waiting_for_osdmap, request: pg_notify(24.b6(32) epoch 194477) v5 -4007> 2014-08-13 17:23:21.667363 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.667363, event: started, request: pg_notify(24.b6(32) epoch 194477) v5 -4006> 2014-08-13 17:23:21.667384 7fcd0964c700 5 -- op tracker -- , seq: 245, time: 2014-08-13 17:23:21.667384, event: done, request: pg_notify(24.b6(32) epoch 194477) v5 -4005> 2014-08-13 17:23:21.667414 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 5 ==== pg_notify(22.3af(27) epoch 194477) v5 ==== 2231+0+0 (4199141443 0 0) 0x56a9180 con 0x4eb0c60 -4004> 2014-08-13 17:23:21.667429 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.641455, event: header_read, request: pg_notify(22.3af(27) epoch 194477) v5 -4003> 2014-08-13 17:23:21.667435 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.641457, event: throttled, request: pg_notify(22.3af(27) epoch 194477) v5 -4002> 2014-08-13 17:23:21.667441 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.641578, event: all_read, request: pg_notify(22.3af(27) epoch 194477) v5 -4001> 2014-08-13 17:23:21.667446 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.667427, event: dispatched, request: pg_notify(22.3af(27) epoch 194477) v5 -4000> 2014-08-13 17:23:21.667452 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.667452, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194477) v5 -3999> 2014-08-13 17:23:21.667459 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.667459, event: started, request: pg_notify(22.3af(27) epoch 194477) v5 -3998> 2014-08-13 17:23:21.667479 7fcd0964c700 5 -- op tracker -- , seq: 246, time: 2014-08-13 17:23:21.667479, event: done, request: pg_notify(22.3af(27) epoch 194477) v5 -3997> 2014-08-13 17:23:21.667498 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 6 ==== pg_info(1 pgs e194477:22.1a0(1)) v4 ==== 765+0+0 (1897639799 0 0) 0x56a88c0 con 0x4eb23c0 -3996> 2014-08-13 17:23:21.667514 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.647004, event: header_read, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3995> 2014-08-13 17:23:21.667521 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.647006, event: throttled, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3994> 2014-08-13 17:23:21.667527 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.647092, event: all_read, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3993> 2014-08-13 17:23:21.667532 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.667512, event: dispatched, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3992> 2014-08-13 17:23:21.667539 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.667538, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3991> 2014-08-13 17:23:21.667547 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.667546, event: started, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3990> 2014-08-13 17:23:21.667568 7fcd0964c700 5 -- op tracker -- , seq: 247, time: 2014-08-13 17:23:21.667568, event: done, request: pg_info(1 pgs e194477:22.1a0(1)) v4 -3989> 2014-08-13 17:23:21.667582 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 2 ==== pg_notify(22.707(27) epoch 194477) v5 ==== 2231+0+0 (1829887906 0 0) 0x588d080 con 0x4ec7e80 -3988> 2014-08-13 17:23:21.667597 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.656866, event: header_read, request: pg_notify(22.707(27) epoch 194477) v5 -3987> 2014-08-13 17:23:21.667603 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.656868, event: throttled, request: pg_notify(22.707(27) epoch 194477) v5 -3986> 2014-08-13 17:23:21.667643 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.656980, event: all_read, request: pg_notify(22.707(27) epoch 194477) v5 -3985> 2014-08-13 17:23:21.667649 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.667595, event: dispatched, request: pg_notify(22.707(27) epoch 194477) v5 -3984> 2014-08-13 17:23:21.667655 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.667655, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194477) v5 -3983> 2014-08-13 17:23:21.667663 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.667663, event: started, request: pg_notify(22.707(27) epoch 194477) v5 -3982> 2014-08-13 17:23:21.667683 7fcd0964c700 5 -- op tracker -- , seq: 248, time: 2014-08-13 17:23:21.667683, event: done, request: pg_notify(22.707(27) epoch 194477) v5 -3981> 2014-08-13 17:23:21.667706 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 7 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52ba880 con 0x4eb18c0 -3980> 2014-08-13 17:23:21.667728 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3979> 2014-08-13 17:23:21.667739 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x2e8a400 con 0x4eb48e0 -3978> 2014-08-13 17:23:21.667754 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -3977> 2014-08-13 17:23:21.667763 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 5 ==== pg_notify(22.1f1(27) epoch 194477) v5 ==== 2231+0+0 (2120929369 0 0) 0x54d2f40 con 0x4eb2680 -3976> 2014-08-13 17:23:21.667777 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.654778, event: header_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3975> 2014-08-13 17:23:21.667784 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.654779, event: throttled, request: pg_notify(22.1f1(27) epoch 194477) v5 -3974> 2014-08-13 17:23:21.667789 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.654903, event: all_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3973> 2014-08-13 17:23:21.667795 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.667774, event: dispatched, request: pg_notify(22.1f1(27) epoch 194477) v5 -3972> 2014-08-13 17:23:21.667801 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.667801, event: waiting_for_osdmap, request: pg_notify(22.1f1(27) epoch 194477) v5 -3971> 2014-08-13 17:23:21.667809 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.667808, event: started, request: pg_notify(22.1f1(27) epoch 194477) v5 -3970> 2014-08-13 17:23:21.667828 7fcd0964c700 5 -- op tracker -- , seq: 249, time: 2014-08-13 17:23:21.667828, event: done, request: pg_notify(22.1f1(27) epoch 194477) v5 -3969> 2014-08-13 17:23:21.667850 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 5 ==== pg_notify(22.707(28) epoch 194477) v5 ==== 2290+0+0 (3371788189 0 0) 0x54d0380 con 0x4f80b00 -3968> 2014-08-13 17:23:21.667864 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.644817, event: header_read, request: pg_notify(22.707(28) epoch 194477) v5 -3967> 2014-08-13 17:23:21.667871 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.644819, event: throttled, request: pg_notify(22.707(28) epoch 194477) v5 -3966> 2014-08-13 17:23:21.667876 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.644947, event: all_read, request: pg_notify(22.707(28) epoch 194477) v5 -3965> 2014-08-13 17:23:21.667882 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.667862, event: dispatched, request: pg_notify(22.707(28) epoch 194477) v5 -3964> 2014-08-13 17:23:21.667888 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.667888, event: waiting_for_osdmap, request: pg_notify(22.707(28) epoch 194477) v5 -3963> 2014-08-13 17:23:21.667895 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.667895, event: started, request: pg_notify(22.707(28) epoch 194477) v5 -3962> 2014-08-13 17:23:21.667914 7fcd0964c700 5 -- op tracker -- , seq: 250, time: 2014-08-13 17:23:21.667913, event: done, request: pg_notify(22.707(28) epoch 194477) v5 -3961> 2014-08-13 17:23:21.667936 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e3f00 con 0x4f832e0 -3960> 2014-08-13 17:23:21.667952 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3959> 2014-08-13 17:23:21.667961 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 4 ==== pg_info(1 pgs e194477:22.12a(1)) v4 ==== 765+0+0 (3754493879 0 0) 0x588abc0 con 0x4f81fa0 -3958> 2014-08-13 17:23:21.667975 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.653666, event: header_read, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3957> 2014-08-13 17:23:21.667982 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.653667, event: throttled, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3956> 2014-08-13 17:23:21.667988 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.653788, event: all_read, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3955> 2014-08-13 17:23:21.667994 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.667973, event: dispatched, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3954> 2014-08-13 17:23:21.668001 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.668000, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3953> 2014-08-13 17:23:21.668008 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.668008, event: started, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3952> 2014-08-13 17:23:21.668028 7fcd0964c700 5 -- op tracker -- , seq: 251, time: 2014-08-13 17:23:21.668027, event: done, request: pg_info(1 pgs e194477:22.12a(1)) v4 -3951> 2014-08-13 17:23:21.668042 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 5 ==== pg_notify(22.113(24) epoch 194477) v5 ==== 2062+0+0 (4128611439 0 0) 0x54d6e40 con 0x4fd9a20 -3950> 2014-08-13 17:23:21.668057 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.650622, event: header_read, request: pg_notify(22.113(24) epoch 194477) v5 -3949> 2014-08-13 17:23:21.668063 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.650623, event: throttled, request: pg_notify(22.113(24) epoch 194477) v5 -3948> 2014-08-13 17:23:21.668069 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.650723, event: all_read, request: pg_notify(22.113(24) epoch 194477) v5 -3947> 2014-08-13 17:23:21.668075 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.668055, event: dispatched, request: pg_notify(22.113(24) epoch 194477) v5 -3946> 2014-08-13 17:23:21.668081 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.668081, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194477) v5 -3945> 2014-08-13 17:23:21.668053 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 0.713816 4 0.000278 -3944> 2014-08-13 17:23:21.668089 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.668088, event: started, request: pg_notify(22.113(24) epoch 194477) v5 -3943> 2014-08-13 17:23:21.668093 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Started 0.713962 0 0.000000 -3942> 2014-08-13 17:23:21.668108 7fcd0964c700 5 -- op tracker -- , seq: 252, time: 2014-08-13 17:23:21.668108, event: done, request: pg_notify(22.113(24) epoch 194477) v5 -3941> 2014-08-13 17:23:21.668115 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194476/193264) [15,4,52]/[15,61,379] r=-1 lpr=194476 pi=193264-194475/29 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Reset -3940> 2014-08-13 17:23:21.668142 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 4 ==== pg_info(1 pgs e194477:22.3ad(1)) v4 ==== 765+0+0 (3121948747 0 0) 0x5889a40 con 0x4f870c0 -3939> 2014-08-13 17:23:21.668159 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.648866, event: header_read, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3938> 2014-08-13 17:23:21.668175 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.648867, event: throttled, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3937> 2014-08-13 17:23:21.668185 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.648941, event: all_read, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3936> 2014-08-13 17:23:21.668191 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.668156, event: dispatched, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3935> 2014-08-13 17:23:21.668197 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.668197, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3934> 2014-08-13 17:23:21.668205 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.668205, event: started, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3933> 2014-08-13 17:23:21.668224 7fcd0964c700 5 -- op tracker -- , seq: 253, time: 2014-08-13 17:23:21.668224, event: done, request: pg_info(1 pgs e194477:22.3ad(1)) v4 -3932> 2014-08-13 17:23:21.668246 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 5 ==== pg_notify(24.136(26) epoch 194477) v5 ==== 2202+0+0 (2731362500 0 0) 0x56ad080 con 0x4fd9760 -3931> 2014-08-13 17:23:21.668229 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.704236 5 0.000246 -3930> 2014-08-13 17:23:21.668266 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.642987, event: header_read, request: pg_notify(24.136(26) epoch 194477) v5 -3929> 2014-08-13 17:23:21.668264 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000149 1 0.000143 -3928> 2014-08-13 17:23:21.668291 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.642991, event: throttled, request: pg_notify(24.136(26) epoch 194477) v5 -3927> 2014-08-13 17:23:21.668285 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.707713 0 0.000000 -3926> 2014-08-13 17:23:21.668302 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.643119, event: all_read, request: pg_notify(24.136(26) epoch 194477) v5 -3925> 2014-08-13 17:23:21.668292 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started -3924> 2014-08-13 17:23:21.668309 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.668264, event: dispatched, request: pg_notify(24.136(26) epoch 194477) v5 -3923> 2014-08-13 17:23:21.668322 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Start -3922> 2014-08-13 17:23:21.668332 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.668331, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194477) v5 -3921> 2014-08-13 17:23:21.668324 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194471 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3920> 2014-08-13 17:23:21.668350 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.668350, event: started, request: pg_notify(24.136(26) epoch 194477) v5 -3919> 2014-08-13 17:23:21.668349 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray -3918> 2014-08-13 17:23:21.668373 7fcd0964c700 5 -- op tracker -- , seq: 254, time: 2014-08-13 17:23:21.668373, event: done, request: pg_notify(24.136(26) epoch 194477) v5 -3917> 2014-08-13 17:23:21.668370 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000048 0 0.000000 -3916> 2014-08-13 17:23:21.668385 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194474/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3915> 2014-08-13 17:23:21.668395 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray -3914> 2014-08-13 17:23:21.668407 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 9 ==== pg_notify(22.362(27) epoch 194477) v5 ==== 2231+0+0 (523403142 0 0) 0x5187000 con 0x4f86e00 -3913> 2014-08-13 17:23:21.668426 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.641448, event: header_read, request: pg_notify(22.362(27) epoch 194477) v5 -3912> 2014-08-13 17:23:21.668437 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.641450, event: throttled, request: pg_notify(22.362(27) epoch 194477) v5 -3911> 2014-08-13 17:23:21.668446 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.641578, event: all_read, request: pg_notify(22.362(27) epoch 194477) v5 -3910> 2014-08-13 17:23:21.668456 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.668423, event: dispatched, request: pg_notify(22.362(27) epoch 194477) v5 -3909> 2014-08-13 17:23:21.668466 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.668466, event: waiting_for_osdmap, request: pg_notify(22.362(27) epoch 194477) v5 -3908> 2014-08-13 17:23:21.668477 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.668477, event: started, request: pg_notify(22.362(27) epoch 194477) v5 -3907> 2014-08-13 17:23:21.668505 7fcd0964c700 5 -- op tracker -- , seq: 255, time: 2014-08-13 17:23:21.668504, event: done, request: pg_notify(22.362(27) epoch 194477) v5 -3906> 2014-08-13 17:23:21.668532 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 6 ==== pg_info(1 pgs e194477:22.4fe(1)) v4 ==== 765+0+0 (857157032 0 0) 0x54d7a80 con 0x4fd8f20 -3905> 2014-08-13 17:23:21.668552 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.646103, event: header_read, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3904> 2014-08-13 17:23:21.668559 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.646105, event: throttled, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3903> 2014-08-13 17:23:21.668566 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.646173, event: all_read, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3902> 2014-08-13 17:23:21.668556 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started/Stray 0.713731 4 0.000300 -3901> 2014-08-13 17:23:21.668577 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.668549, event: dispatched, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3900> 2014-08-13 17:23:21.668587 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.668587, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3899> 2014-08-13 17:23:21.668584 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started 0.713845 0 0.000000 -3898> 2014-08-13 17:23:21.668599 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.668599, event: started, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3897> 2014-08-13 17:23:21.668607 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194476/192978) [31,338,4]/[31,338,16] r=-1 lpr=194476 pi=193125-194475/37 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Reset -3896> 2014-08-13 17:23:21.668621 7fcd0964c700 5 -- op tracker -- , seq: 256, time: 2014-08-13 17:23:21.668621, event: done, request: pg_info(1 pgs e194477:22.4fe(1)) v4 -3895> 2014-08-13 17:23:21.668610 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.692996 4 0.000272 -3894> 2014-08-13 17:23:21.668671 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.706896 0 0.000000 -3893> 2014-08-13 17:23:21.668705 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3892> 2014-08-13 17:23:21.668750 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.000144 1 0.000118 -3891> 2014-08-13 17:23:21.668757 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194474/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3890> 2014-08-13 17:23:21.668784 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started -3889> 2014-08-13 17:23:21.668805 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start -3888> 2014-08-13 17:23:21.668828 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray -3887> 2014-08-13 17:23:21.668849 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000044 0 0.000000 -3886> 2014-08-13 17:23:21.668871 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray -3885> 2014-08-13 17:23:21.668925 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.363 192.168.1.102:6842/20017 2 ==== pg_notify(22.99(5),22.39d(24) epoch 194477) v5 ==== 3047+0+0 (2076861257 0 0) 0x588cec0 con 0x4fdb2e0 -3884> 2014-08-13 17:23:21.668955 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.656642, event: header_read, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3883> 2014-08-13 17:23:21.668968 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.656644, event: throttled, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3882> 2014-08-13 17:23:21.668978 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.656812, event: all_read, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3881> 2014-08-13 17:23:21.668988 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.668951, event: dispatched, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3880> 2014-08-13 17:23:21.668999 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.668999, event: waiting_for_osdmap, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3879> 2014-08-13 17:23:21.669009 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.669008, event: started, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3878> 2014-08-13 17:23:21.669049 7fcd0964c700 5 -- op tracker -- , seq: 257, time: 2014-08-13 17:23:21.669049, event: done, request: pg_notify(22.99(5),22.39d(24) epoch 194477) v5 -3877> 2014-08-13 17:23:21.669083 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 4 ==== pg_info(1 pgs e194477:22.537(1)) v4 ==== 765+0+0 (552553317 0 0) 0x56a9880 con 0x4fda520 -3876> 2014-08-13 17:23:21.669104 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.652624, event: header_read, request: pg_info(1 pgs e194477:22.537(1)) v4 -3875> 2014-08-13 17:23:21.669115 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.652626, event: throttled, request: pg_info(1 pgs e194477:22.537(1)) v4 -3874> 2014-08-13 17:23:21.669124 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.652721, event: all_read, request: pg_info(1 pgs e194477:22.537(1)) v4 -3873> 2014-08-13 17:23:21.669135 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.669101, event: dispatched, request: pg_info(1 pgs e194477:22.537(1)) v4 -3872> 2014-08-13 17:23:21.669144 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.669144, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.537(1)) v4 -3871> 2014-08-13 17:23:21.669156 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.669156, event: started, request: pg_info(1 pgs e194477:22.537(1)) v4 -3870> 2014-08-13 17:23:21.669151 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.712489 4 0.000373 -3869> 2014-08-13 17:23:21.669193 7fcd0964c700 5 -- op tracker -- , seq: 258, time: 2014-08-13 17:23:21.669192, event: done, request: pg_info(1 pgs e194477:22.537(1)) v4 -3868> 2014-08-13 17:23:21.669182 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] exit Started 0.712706 0 0.000000 -3867> 2014-08-13 17:23:21.669179 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.712494 4 0.000244 -3866> 2014-08-13 17:23:21.669213 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194476/192974) [376,4,303]/[376,303,313] r=-1 lpr=194476 pi=192975-194475/25 crt=192888'461 lcod 0'0 remapped NOTIFY] enter Reset -3865> 2014-08-13 17:23:21.669224 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] exit Started 0.712644 0 0.000000 -3864> 2014-08-13 17:23:21.669213 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.379 192.168.1.108:6830/4861 2 ==== pg_notify(24.1bb(30) epoch 194477) v5 ==== 2438+0+0 (2659733062 0 0) 0x588d240 con 0x4fdeb40 -3863> 2014-08-13 17:23:21.669250 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194476/194473) [4,44,362]/[44,362,61] r=-1 lpr=194476 pi=193306-194475/26 crt=191727'457 lcod 0'0 remapped NOTIFY] enter Reset -3862> 2014-08-13 17:23:21.669267 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.656902, event: header_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3861> 2014-08-13 17:23:21.669278 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.656903, event: throttled, request: pg_notify(24.1bb(30) epoch 194477) v5 -3860> 2014-08-13 17:23:21.669287 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.657014, event: all_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3859> 2014-08-13 17:23:21.669296 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.669264, event: dispatched, request: pg_notify(24.1bb(30) epoch 194477) v5 -3858> 2014-08-13 17:23:21.669306 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.669306, event: waiting_for_osdmap, request: pg_notify(24.1bb(30) epoch 194477) v5 -3857> 2014-08-13 17:23:21.669318 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.669317, event: started, request: pg_notify(24.1bb(30) epoch 194477) v5 -3856> 2014-08-13 17:23:21.669315 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000065 1 0.000132 -3855> 2014-08-13 17:23:21.669343 7fcd0964c700 5 -- op tracker -- , seq: 259, time: 2014-08-13 17:23:21.669343, event: done, request: pg_notify(24.1bb(30) epoch 194477) v5 -3854> 2014-08-13 17:23:21.669329 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.000116 1 0.000130 -3853> 2014-08-13 17:23:21.669347 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started -3852> 2014-08-13 17:23:21.669371 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 5 ==== pg_notify(22.1d2(3) epoch 194477) v5 ==== 887+0+0 (2045914784 0 0) 0x54d6580 con 0x4fde880 -3851> 2014-08-13 17:23:21.669375 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started -3850> 2014-08-13 17:23:21.669380 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start -3849> 2014-08-13 17:23:21.669391 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.656920, event: header_read, request: pg_notify(22.1d2(3) epoch 194477) v5 -3848> 2014-08-13 17:23:21.669406 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -3847> 2014-08-13 17:23:21.669394 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start -3846> 2014-08-13 17:23:21.669421 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.656921, event: throttled, request: pg_notify(22.1d2(3) epoch 194477) v5 -3845> 2014-08-13 17:23:21.669440 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.656993, event: all_read, request: pg_notify(22.1d2(3) epoch 194477) v5 -3844> 2014-08-13 17:23:21.669432 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000052 0 0.000000 -3843> 2014-08-13 17:23:21.669457 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.669389, event: dispatched, request: pg_notify(22.1d2(3) epoch 194477) v5 -3842> 2014-08-13 17:23:21.669443 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray -3841> 2014-08-13 17:23:21.669468 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.669468, event: waiting_for_osdmap, request: pg_notify(22.1d2(3) epoch 194477) v5 -3840> 2014-08-13 17:23:21.669467 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -3839> 2014-08-13 17:23:21.669483 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.669483, event: started, request: pg_notify(22.1d2(3) epoch 194477) v5 -3838> 2014-08-13 17:23:21.669473 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000080 0 0.000000 -3837> 2014-08-13 17:23:21.669491 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -3836> 2014-08-13 17:23:21.669495 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray -3835> 2014-08-13 17:23:21.669514 7fcd0964c700 5 -- op tracker -- , seq: 260, time: 2014-08-13 17:23:21.669513, event: done, request: pg_notify(22.1d2(3) epoch 194477) v5 -3834> 2014-08-13 17:23:21.669512 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -3833> 2014-08-13 17:23:21.669539 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 5 ==== pg_notify(22.22c(24) epoch 194477) v5 ==== 2062+0+0 (3128954323 0 0) 0x54d1880 con 0x4fde5c0 -3832> 2014-08-13 17:23:21.669566 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.656815, event: header_read, request: pg_notify(22.22c(24) epoch 194477) v5 -3831> 2014-08-13 17:23:21.669577 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.656817, event: throttled, request: pg_notify(22.22c(24) epoch 194477) v5 -3830> 2014-08-13 17:23:21.669587 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.656968, event: all_read, request: pg_notify(22.22c(24) epoch 194477) v5 -3829> 2014-08-13 17:23:21.669595 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.669563, event: dispatched, request: pg_notify(22.22c(24) epoch 194477) v5 -3828> 2014-08-13 17:23:21.669605 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.669605, event: waiting_for_osdmap, request: pg_notify(22.22c(24) epoch 194477) v5 -3827> 2014-08-13 17:23:21.669616 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.669616, event: started, request: pg_notify(22.22c(24) epoch 194477) v5 -3826> 2014-08-13 17:23:21.669641 7fcd0964c700 5 -- op tracker -- , seq: 261, time: 2014-08-13 17:23:21.669641, event: done, request: pg_notify(22.22c(24) epoch 194477) v5 -3825> 2014-08-13 17:23:21.669636 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.712065 4 0.000379 -3824> 2014-08-13 17:23:21.669666 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] exit Started 0.712232 0 0.000000 -3823> 2014-08-13 17:23:21.669676 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 6 ==== pg_notify(24.181(7) epoch 194477) v5 ==== 1149+0+0 (1634300321 0 0) 0x54d0e00 con 0x4fde300 -3822> 2014-08-13 17:23:21.669689 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194476/194473) [4,338,376]/[338,376,293] r=-1 lpr=194476 pi=193306-194475/44 crt=191727'441 lcod 0'0 remapped NOTIFY] enter Reset -3821> 2014-08-13 17:23:21.669698 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.643808, event: header_read, request: pg_notify(24.181(7) epoch 194477) v5 -3820> 2014-08-13 17:23:21.669701 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.693477 2 0.000087 -3819> 2014-08-13 17:23:21.669723 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.643811, event: throttled, request: pg_notify(24.181(7) epoch 194477) v5 -3818> 2014-08-13 17:23:21.669743 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.643902, event: all_read, request: pg_notify(24.181(7) epoch 194477) v5 -3817> 2014-08-13 17:23:21.669737 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.712456 0 0.000000 -3816> 2014-08-13 17:23:21.669754 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.669696, event: dispatched, request: pg_notify(24.181(7) epoch 194477) v5 -3815> 2014-08-13 17:23:21.669761 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194471 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3814> 2014-08-13 17:23:21.669763 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000074 1 0.000124 -3813> 2014-08-13 17:23:21.669787 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started -3812> 2014-08-13 17:23:21.669765 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.669764, event: waiting_for_osdmap, request: pg_notify(24.181(7) epoch 194477) v5 -3811> 2014-08-13 17:23:21.669803 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start -3810> 2014-08-13 17:23:21.669811 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.669811, event: started, request: pg_notify(24.181(7) epoch 194477) v5 -3809> 2014-08-13 17:23:21.669818 7fcd02841700 1 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -3808> 2014-08-13 17:23:21.669836 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 -3807> 2014-08-13 17:23:21.669845 7fcd0964c700 5 -- op tracker -- , seq: 262, time: 2014-08-13 17:23:21.669845, event: done, request: pg_notify(24.181(7) epoch 194477) v5 -3806> 2014-08-13 17:23:21.669854 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -3805> 2014-08-13 17:23:21.669870 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -3804> 2014-08-13 17:23:21.669872 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.24 192.168.1.107:6800/52669 5 ==== pg_notify(22.113(24) epoch 194477) v5 ==== 2062+0+0 (4128611439 0 0) 0x56a8000 con 0x4fd11e0 -3803> 2014-08-13 17:23:21.669891 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.653336, event: header_read, request: pg_notify(22.113(24) epoch 194477) v5 -3802> 2014-08-13 17:23:21.669886 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -3801> 2014-08-13 17:23:21.669898 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.653337, event: throttled, request: pg_notify(22.113(24) epoch 194477) v5 -3800> 2014-08-13 17:23:21.669904 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.654994, event: all_read, request: pg_notify(22.113(24) epoch 194477) v5 -3799> 2014-08-13 17:23:21.669910 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.669888, event: dispatched, request: pg_notify(22.113(24) epoch 194477) v5 -3798> 2014-08-13 17:23:21.669917 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.669917, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194477) v5 -3797> 2014-08-13 17:23:21.669925 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.669925, event: started, request: pg_notify(22.113(24) epoch 194477) v5 -3796> 2014-08-13 17:23:21.669963 7fcd0964c700 5 -- op tracker -- , seq: 263, time: 2014-08-13 17:23:21.669963, event: done, request: pg_notify(22.113(24) epoch 194477) v5 -3795> 2014-08-13 17:23:21.669987 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 2 ==== pg_notify(22.48e(24) epoch 194477) v5 ==== 2058+0+0 (1619785741 0 0) 0x588d940 con 0x4fd35a0 -3794> 2014-08-13 17:23:21.670007 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.657095, event: header_read, request: pg_notify(22.48e(24) epoch 194477) v5 -3793> 2014-08-13 17:23:21.670017 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.657097, event: throttled, request: pg_notify(22.48e(24) epoch 194477) v5 -3792> 2014-08-13 17:23:21.670026 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.657209, event: all_read, request: pg_notify(22.48e(24) epoch 194477) v5 -3791> 2014-08-13 17:23:21.670035 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.670004, event: dispatched, request: pg_notify(22.48e(24) epoch 194477) v5 -3790> 2014-08-13 17:23:21.670045 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.670044, event: waiting_for_osdmap, request: pg_notify(22.48e(24) epoch 194477) v5 -3789> 2014-08-13 17:23:21.670056 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.670055, event: started, request: pg_notify(22.48e(24) epoch 194477) v5 -3788> 2014-08-13 17:23:21.670080 7fcd0964c700 5 -- op tracker -- , seq: 264, time: 2014-08-13 17:23:21.670080, event: done, request: pg_notify(22.48e(24) epoch 194477) v5 -3787> 2014-08-13 17:23:21.670123 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 7 ==== pg_notify(22.3e4(5) epoch 194477) v5 ==== 1001+0+0 (3879056912 0 0) 0x54d5780 con 0x4fd06e0 -3786> 2014-08-13 17:23:21.670127 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -3785> 2014-08-13 17:23:21.670146 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.656291, event: header_read, request: pg_notify(22.3e4(5) epoch 194477) v5 -3784> 2014-08-13 17:23:21.670165 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.656292, event: throttled, request: pg_notify(22.3e4(5) epoch 194477) v5 -3783> 2014-08-13 17:23:21.670175 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.656396, event: all_read, request: pg_notify(22.3e4(5) epoch 194477) v5 -3782> 2014-08-13 17:23:21.670184 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.670143, event: dispatched, request: pg_notify(22.3e4(5) epoch 194477) v5 -3781> 2014-08-13 17:23:21.670197 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.670197, event: waiting_for_osdmap, request: pg_notify(22.3e4(5) epoch 194477) v5 -3780> 2014-08-13 17:23:21.670206 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.670205, event: started, request: pg_notify(22.3e4(5) epoch 194477) v5 -3779> 2014-08-13 17:23:21.670226 7fcd0964c700 5 -- op tracker -- , seq: 265, time: 2014-08-13 17:23:21.670226, event: done, request: pg_notify(22.3e4(5) epoch 194477) v5 -3778> 2014-08-13 17:23:21.670245 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 6 ==== pg_notify(22.48e(24) epoch 194477) v5 ==== 2058+0+0 (3288745529 0 0) 0x54d4280 con 0x4fd5960 -3777> 2014-08-13 17:23:21.670264 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.645571, event: header_read, request: pg_notify(22.48e(24) epoch 194477) v5 -3776> 2014-08-13 17:23:21.670274 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.645573, event: throttled, request: pg_notify(22.48e(24) epoch 194477) v5 -3775> 2014-08-13 17:23:21.670283 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.645689, event: all_read, request: pg_notify(22.48e(24) epoch 194477) v5 -3774> 2014-08-13 17:23:21.670292 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.670261, event: dispatched, request: pg_notify(22.48e(24) epoch 194477) v5 -3773> 2014-08-13 17:23:21.670301 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.670301, event: waiting_for_osdmap, request: pg_notify(22.48e(24) epoch 194477) v5 -3772> 2014-08-13 17:23:21.670312 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.670312, event: started, request: pg_notify(22.48e(24) epoch 194477) v5 -3771> 2014-08-13 17:23:21.670314 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.712372 4 0.000233 -3770> 2014-08-13 17:23:21.670346 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] exit Started 0.712488 0 0.000000 -3769> 2014-08-13 17:23:21.670370 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194476/194473) [4,289,341]/[289,341,376] r=-1 lpr=194476 pi=192978-194475/24 crt=191727'473 lcod 0'0 remapped NOTIFY] enter Reset -3768> 2014-08-13 17:23:21.670436 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000067 1 0.000121 -3767> 2014-08-13 17:23:21.670448 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(26) epoch 194477) v5 -- ?+0 0x5b49f80 con 0x4fd0f20 -3766> 2014-08-13 17:23:21.670463 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started -3765> 2014-08-13 17:23:21.670485 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(26) epoch 194477) v5 -- ?+0 0x5b49dc0 con 0x4ec09a0 -3764> 2014-08-13 17:23:21.670486 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start -3763> 2014-08-13 17:23:21.670527 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(22) epoch 194477) v5 -- ?+0 0x5b49c00 con 0x4fda520 -3762> 2014-08-13 17:23:21.670533 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -3761> 2014-08-13 17:23:21.670555 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000069 0 0.000000 -3760> 2014-08-13 17:23:21.670562 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6812/64303 -- pg_notify(22.1a0(22) epoch 194477) v5 -- ?+0 0x5b49a40 con 0x4eb23c0 -3759> 2014-08-13 17:23:21.670585 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -3758> 2014-08-13 17:23:21.670608 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -3757> 2014-08-13 17:23:21.670619 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(21) epoch 194477) v5 -- ?+0 0x5b49880 con 0x4e32940 -3756> 2014-08-13 17:23:21.670629 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -3755> 2014-08-13 17:23:21.670666 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(26) epoch 194477) v5 -- ?+0 0x5b496c0 con 0x4fdee00 -3754> 2014-08-13 17:23:21.670687 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(42) epoch 194477) v5 -- ?+0 0x5b49500 con 0x4eb18c0 -3753> 2014-08-13 17:23:21.670709 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- osd_map(194477..194477 src has 185139..194477) v3 -- ?+0 0x567d580 con 0x4fd5960 -3752> 2014-08-13 17:23:21.670732 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(26) epoch 194477) v5 -- ?+0 0x5b49340 con 0x4fd5960 -3751> 2014-08-13 17:23:21.670896 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6814/41806 -- pg_notify(24.299(5) epoch 194477) v5 -- ?+0 0x5b49180 con 0x4fd3020 -3750> 2014-08-13 17:23:21.670956 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6816/33855 -- pg_query(22.3d9 epoch 194477) v3 -- ?+0 0x2f92b20 con 0x5724e60 -3749> 2014-08-13 17:23:21.670955 7fcd0964c700 5 -- op tracker -- , seq: 266, time: 2014-08-13 17:23:21.670955, event: done, request: pg_notify(22.48e(24) epoch 194477) v5 -3748> 2014-08-13 17:23:21.670987 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6814/25893 -- pg_query(22.3d9 epoch 194477) v3 -- ?+0 0x2f93c00 con 0x4f832e0 -3747> 2014-08-13 17:23:21.671027 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 9 ==== pg_notify(22.3d9(39) epoch 194477) v5 ==== 2851+0+0 (4226451791 0 0) 0x56aa840 con 0x4fd32e0 -3746> 2014-08-13 17:23:21.671054 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.642194, event: header_read, request: pg_notify(22.3d9(39) epoch 194477) v5 -3745> 2014-08-13 17:23:21.671065 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.642195, event: throttled, request: pg_notify(22.3d9(39) epoch 194477) v5 -3744> 2014-08-13 17:23:21.671075 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.642309, event: all_read, request: pg_notify(22.3d9(39) epoch 194477) v5 -3743> 2014-08-13 17:23:21.671063 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_query(22.3d9 epoch 194477) v3 -- ?+0 0x54f9860 con 0x4fd5960 -3742> 2014-08-13 17:23:21.671084 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.671051, event: dispatched, request: pg_notify(22.3d9(39) epoch 194477) v5 -3741> 2014-08-13 17:23:21.671092 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.671091, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194477) v5 -3740> 2014-08-13 17:23:21.671100 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.671100, event: started, request: pg_notify(22.3d9(39) epoch 194477) v5 -3739> 2014-08-13 17:23:21.671098 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6820/19173 -- pg_info(1 pgs e194477:22.3d5(2)) v4 -- ?+0 0x5b48fc0 con 0x4fdad60 -3738> 2014-08-13 17:23:21.671118 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:22.1b8(2)) v4 -- ?+0 0x5b48e00 con 0x4e35540 -3737> 2014-08-13 17:23:21.671123 7fcd0964c700 5 -- op tracker -- , seq: 267, time: 2014-08-13 17:23:21.671123, event: done, request: pg_notify(22.3d9(39) epoch 194477) v5 -3736> 2014-08-13 17:23:21.671137 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_info(1 pgs e194477:22.3d5(2)) v4 -- ?+0 0x5b48c40 con 0x4f86e00 -3735> 2014-08-13 17:23:21.671155 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 6 ==== pg_notify(22.481(34) epoch 194477) v5 ==== 2628+0+0 (2057168624 0 0) 0x54d0c40 con 0x5303700 -3734> 2014-08-13 17:23:21.671171 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.650900, event: header_read, request: pg_notify(22.481(34) epoch 194477) v5 -3733> 2014-08-13 17:23:21.671178 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.650901, event: throttled, request: pg_notify(22.481(34) epoch 194477) v5 -3732> 2014-08-13 17:23:21.671184 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.651056, event: all_read, request: pg_notify(22.481(34) epoch 194477) v5 -3731> 2014-08-13 17:23:21.671176 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 0.711821 4 0.000477 -3730> 2014-08-13 17:23:21.671212 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.671169, event: dispatched, request: pg_notify(22.481(34) epoch 194477) v5 -3729> 2014-08-13 17:23:21.671225 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.671225, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194477) v5 -3728> 2014-08-13 17:23:21.671236 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.671236, event: started, request: pg_notify(22.481(34) epoch 194477) v5 -3727> 2014-08-13 17:23:21.671214 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit Started 0.711950 0 0.000000 -3726> 2014-08-13 17:23:21.671263 7fcd0964c700 5 -- op tracker -- , seq: 268, time: 2014-08-13 17:23:21.671263, event: done, request: pg_notify(22.481(34) epoch 194477) v5 -3725> 2014-08-13 17:23:21.671282 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194476/194473) [4,348,331]/[332,337,379] r=-1 lpr=194476 pi=193644-194475/26 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter Reset -3724> 2014-08-13 17:23:21.671301 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6800/4640 -- pg_info(1 pgs e194477:22.1b8(2)) v4 -- ?+0 0x5b48a80 con 0x4fd1a20 -3723> 2014-08-13 17:23:21.671309 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 7 ==== pg_notify(22.1d2(24) epoch 194477) v5 ==== 2062+0+0 (1449907918 0 0) 0x54d6c80 con 0x5305ee0 -3722> 2014-08-13 17:23:21.671333 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.647416, event: header_read, request: pg_notify(22.1d2(24) epoch 194477) v5 -3721> 2014-08-13 17:23:21.671344 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.647417, event: throttled, request: pg_notify(22.1d2(24) epoch 194477) v5 -3720> 2014-08-13 17:23:21.671353 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.647503, event: all_read, request: pg_notify(22.1d2(24) epoch 194477) v5 -3719> 2014-08-13 17:23:21.671362 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.671330, event: dispatched, request: pg_notify(22.1d2(24) epoch 194477) v5 -3718> 2014-08-13 17:23:21.671372 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.671372, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194477) v5 -3717> 2014-08-13 17:23:21.671359 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Reset 0.000076 1 0.000180 -3716> 2014-08-13 17:23:21.671383 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.671383, event: started, request: pg_notify(22.1d2(24) epoch 194477) v5 -3715> 2014-08-13 17:23:21.671385 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started -3714> 2014-08-13 17:23:21.671409 7fcd0964c700 5 -- op tracker -- , seq: 269, time: 2014-08-13 17:23:21.671409, event: done, request: pg_notify(22.1d2(24) epoch 194477) v5 -3713> 2014-08-13 17:23:21.671411 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Start -3712> 2014-08-13 17:23:21.671432 7fcd01e40700 1 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: transitioning to Primary -3711> 2014-08-13 17:23:21.671440 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.378 192.168.1.102:6840/19195 6 ==== pg_info(1 pgs e194477:24.9b) v4 ==== 748+0+0 (3105858428 0 0) 0x5b4a140 con 0x5305120 -3710> 2014-08-13 17:23:21.671466 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.668970, event: header_read, request: pg_info(1 pgs e194477:24.9b) v4 -3709> 2014-08-13 17:23:21.671456 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Start 0.000045 0 0.000000 -3708> 2014-08-13 17:23:21.671481 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.668973, event: throttled, request: pg_info(1 pgs e194477:24.9b) v4 -3707> 2014-08-13 17:23:21.671467 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194474/194474 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 0.708774 9 0.000437 -3706> 2014-08-13 17:23:21.671502 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.669050, event: all_read, request: pg_info(1 pgs e194477:24.9b) v4 -3705> 2014-08-13 17:23:21.671496 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary -3704> 2014-08-13 17:23:21.671513 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.671463, event: dispatched, request: pg_info(1 pgs e194477:24.9b) v4 -3703> 2014-08-13 17:23:21.671506 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194474/194474 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive -3702> 2014-08-13 17:23:21.671523 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.671523, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.9b) v4 -3701> 2014-08-13 17:23:21.671516 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Peering -3700> 2014-08-13 17:23:21.671532 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.671531, event: started, request: pg_info(1 pgs e194477:24.9b) v4 -3699> 2014-08-13 17:23:21.671538 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194472 n=53 ec=192688 les/c 194474/194474 194476/194476/192967) [6,374,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering -3698> 2014-08-13 17:23:21.671542 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetInfo -3697> 2014-08-13 17:23:21.671562 7fcd0964c700 5 -- op tracker -- , seq: 270, time: 2014-08-13 17:23:21.671561, event: done, request: pg_info(1 pgs e194477:24.9b) v4 -3696> 2014-08-13 17:23:21.671580 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 6 ==== pg_notify(22.3af(26) epoch 194477) v5 ==== 2172+0+0 (431353792 0 0) 0x54d7c40 con 0x5305c20 -3695> 2014-08-13 17:23:21.671600 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.660809, event: header_read, request: pg_notify(22.3af(26) epoch 194477) v5 -3694> 2014-08-13 17:23:21.671610 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.660810, event: throttled, request: pg_notify(22.3af(26) epoch 194477) v5 -3693> 2014-08-13 17:23:21.671619 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.660977, event: all_read, request: pg_notify(22.3af(26) epoch 194477) v5 -3692> 2014-08-13 17:23:21.671628 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.671598, event: dispatched, request: pg_notify(22.3af(26) epoch 194477) v5 -3691> 2014-08-13 17:23:21.671638 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.671637, event: waiting_for_osdmap, request: pg_notify(22.3af(26) epoch 194477) v5 -3690> 2014-08-13 17:23:21.671649 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.671648, event: started, request: pg_notify(22.3af(26) epoch 194477) v5 -3689> 2014-08-13 17:23:21.671671 7fcd0964c700 5 -- op tracker -- , seq: 271, time: 2014-08-13 17:23:21.671671, event: done, request: pg_notify(22.3af(26) epoch 194477) v5 -3688> 2014-08-13 17:23:21.671696 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(30) epoch 194477) v5 -- ?+0 0x588b480 con 0x4f80b00 -3687> 2014-08-13 17:23:21.671700 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550fbc0 con 0x552a260 -3686> 2014-08-13 17:23:21.671731 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- pg_notify(22.791(21) epoch 194477) v5 -- ?+0 0x588b2c0 con 0x4ec3f40 -3685> 2014-08-13 17:23:21.671736 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3684> 2014-08-13 17:23:21.671749 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(38) epoch 194477) v5 -- ?+0 0x588b100 con 0x4ec56a0 -3683> 2014-08-13 17:23:21.671748 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 5 ==== pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 ==== 3047+0+0 (1242334803 0 0) 0x56a8380 con 0x5305960 -3682> 2014-08-13 17:23:21.671765 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.659301, event: header_read, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3681> 2014-08-13 17:23:21.671765 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(26) epoch 194477) v5 -- ?+0 0x588af40 con 0x4e314a0 -3680> 2014-08-13 17:23:21.671774 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.659302, event: throttled, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3679> 2014-08-13 17:23:21.671780 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.659432, event: all_read, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3678> 2014-08-13 17:23:21.671781 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(21) epoch 194477) v5 -- ?+0 0x588ad80 con 0x4f870c0 -3677> 2014-08-13 17:23:21.671787 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.671762, event: dispatched, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3676> 2014-08-13 17:23:21.671799 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.671798, event: waiting_for_osdmap, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3675> 2014-08-13 17:23:21.671800 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(21) epoch 194477) v5 -- ?+0 0x54d5780 con 0x4f81fa0 -3674> 2014-08-13 17:23:21.671807 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.671807, event: started, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3673> 2014-08-13 17:23:21.671848 7fcd0964c700 5 -- op tracker -- , seq: 272, time: 2014-08-13 17:23:21.671848, event: done, request: pg_notify(22.113(3),22.3e4(26) epoch 194477) v5 -3672> 2014-08-13 17:23:21.671878 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.302 192.168.1.113:6810/15731 5 ==== pg_notify(22.22c(3) epoch 194477) v5 ==== 887+0+0 (2385175889 0 0) 0x54d4440 con 0x552b9c0 -3671> 2014-08-13 17:23:21.671899 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.660055, event: header_read, request: pg_notify(22.22c(3) epoch 194477) v5 -3670> 2014-08-13 17:23:21.671909 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.660056, event: throttled, request: pg_notify(22.22c(3) epoch 194477) v5 -3669> 2014-08-13 17:23:21.671918 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.660136, event: all_read, request: pg_notify(22.22c(3) epoch 194477) v5 -3668> 2014-08-13 17:23:21.671927 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.671896, event: dispatched, request: pg_notify(22.22c(3) epoch 194477) v5 -3667> 2014-08-13 17:23:21.671939 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.671939, event: waiting_for_osdmap, request: pg_notify(22.22c(3) epoch 194477) v5 -3666> 2014-08-13 17:23:21.671950 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.671950, event: started, request: pg_notify(22.22c(3) epoch 194477) v5 -3665> 2014-08-13 17:23:21.671956 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6804/62197 -- pg_notify(22.4fe(21) epoch 194477) v5 -- ?+0 0x54d0e00 con 0x4fd8f20 -3664> 2014-08-13 17:23:21.671974 7fcd0964c700 5 -- op tracker -- , seq: 273, time: 2014-08-13 17:23:21.671974, event: done, request: pg_notify(22.22c(3) epoch 194477) v5 -3663> 2014-08-13 17:23:21.671987 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(26) epoch 194477) v5 -- ?+0 0x54d1880 con 0x4f86e00 -3662> 2014-08-13 17:23:21.671989 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 6 ==== pg_notify(22.1f1(26) epoch 194477) v5 ==== 2172+0+0 (2620928995 0 0) 0x56aa680 con 0x552b700 -3661> 2014-08-13 17:23:21.672008 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.651395, event: header_read, request: pg_notify(22.1f1(26) epoch 194477) v5 -3660> 2014-08-13 17:23:21.672019 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.651397, event: throttled, request: pg_notify(22.1f1(26) epoch 194477) v5 -3659> 2014-08-13 17:23:21.672016 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(26),24.3b7(33) epoch 194477) v5 -- ?+0 0x54d6580 con 0x4ec4780 -3658> 2014-08-13 17:23:21.672030 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.651523, event: all_read, request: pg_notify(22.1f1(26) epoch 194477) v5 -3657> 2014-08-13 17:23:21.672042 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.672006, event: dispatched, request: pg_notify(22.1f1(26) epoch 194477) v5 -3656> 2014-08-13 17:23:21.672039 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(26) epoch 194477) v5 -- ?+0 0x54d7a80 con 0x4fd86e0 -3655> 2014-08-13 17:23:21.672049 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.672048, event: waiting_for_osdmap, request: pg_notify(22.1f1(26) epoch 194477) v5 -3654> 2014-08-13 17:23:21.672057 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.105:6808/41085 -- pg_notify(22.3b9(21) epoch 194477) v5 -- ?+0 0x54d6e40 con 0x4fd53e0 -3653> 2014-08-13 17:23:21.672074 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_query(22.3af epoch 194477) v3 -- ?+0 0x54fbfc0 con 0x4eb0c60 -3652> 2014-08-13 17:23:21.672093 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_query(24.136 epoch 194477) v3 -- ?+0 0x54f8b40 con 0x4eb7d20 -3651> 2014-08-13 17:23:21.672224 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.672224, event: started, request: pg_notify(22.1f1(26) epoch 194477) v5 -3650> 2014-08-13 17:23:21.672222 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6834/6966 -- pg_query(24.136,24.1f7 epoch 194477) v3 -- ?+0 0x54f81e0 con 0x4fd9a20 -3649> 2014-08-13 17:23:21.672255 7fcd0964c700 5 -- op tracker -- , seq: 274, time: 2014-08-13 17:23:21.672255, event: done, request: pg_notify(22.1f1(26) epoch 194477) v5 -3648> 2014-08-13 17:23:21.672289 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 6 ==== pg_notify(22.362(26) epoch 194477) v5 ==== 2172+0+0 (2886206003 0 0) 0x54d7540 con 0x552fa60 -3647> 2014-08-13 17:23:21.672295 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6822/5265 -- pg_query(24.1f7 epoch 194477) v3 -- ?+0 0x54f8d20 con 0x53e65c0 -3646> 2014-08-13 17:23:21.672314 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.658027, event: header_read, request: pg_notify(22.362(26) epoch 194477) v5 -3645> 2014-08-13 17:23:21.672318 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_query(22.3af,24.136 epoch 194477) v3 -- ?+0 0x54fb480 con 0x4e35ac0 -3644> 2014-08-13 17:23:21.672326 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.658028, event: throttled, request: pg_notify(22.362(26) epoch 194477) v5 -3643> 2014-08-13 17:23:21.672336 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.658215, event: all_read, request: pg_notify(22.362(26) epoch 194477) v5 -3642> 2014-08-13 17:23:21.672335 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_query(24.136 epoch 194477) v3 -- ?+0 0x54fb660 con 0x4fd9760 -3641> 2014-08-13 17:23:21.672342 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.672311, event: dispatched, request: pg_notify(22.362(26) epoch 194477) v5 -3640> 2014-08-13 17:23:21.672349 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.672349, event: waiting_for_osdmap, request: pg_notify(22.362(26) epoch 194477) v5 -3639> 2014-08-13 17:23:21.672352 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_query(22.48e epoch 194477) v3 -- ?+0 0x54fbc00 con 0x4ec3440 -3638> 2014-08-13 17:23:21.672357 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.672357, event: started, request: pg_notify(22.362(26) epoch 194477) v5 -3637> 2014-08-13 17:23:21.672369 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6806/31286 -- pg_query(24.181 epoch 194477) v3 -- ?+0 0x54fbde0 con 0x4f81600 -3636> 2014-08-13 17:23:21.672383 7fcd0964c700 5 -- op tracker -- , seq: 275, time: 2014-08-13 17:23:21.672383, event: done, request: pg_notify(22.362(26) epoch 194477) v5 -3635> 2014-08-13 17:23:21.672415 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.63 192.168.1.111:6830/12680 2 ==== pg_notify(22.775(26) epoch 194477) v5 ==== 2172+0+0 (2539764484 0 0) 0x588e3c0 con 0x5521e40 -3634> 2014-08-13 17:23:21.672436 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.657776, event: header_read, request: pg_notify(22.775(26) epoch 194477) v5 -3633> 2014-08-13 17:23:21.672446 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.657778, event: throttled, request: pg_notify(22.775(26) epoch 194477) v5 -3632> 2014-08-13 17:23:21.672455 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.657906, event: all_read, request: pg_notify(22.775(26) epoch 194477) v5 -3631> 2014-08-13 17:23:21.672464 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.672433, event: dispatched, request: pg_notify(22.775(26) epoch 194477) v5 -3630> 2014-08-13 17:23:21.672473 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.672473, event: waiting_for_osdmap, request: pg_notify(22.775(26) epoch 194477) v5 -3629> 2014-08-13 17:23:21.672484 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.672484, event: started, request: pg_notify(22.775(26) epoch 194477) v5 -3628> 2014-08-13 17:23:21.672480 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6804/31016 -- pg_query(24.181 epoch 194477) v3 -- ?+0 0x54fda00 con 0x56b4fc0 -3627> 2014-08-13 17:23:21.672510 7fcd0964c700 5 -- op tracker -- , seq: 276, time: 2014-08-13 17:23:21.672509, event: done, request: pg_notify(22.775(26) epoch 194477) v5 -3626> 2014-08-13 17:23:21.672507 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_query(24.181 epoch 194477) v3 -- ?+0 0x54fd820 con 0x4fde300 -3625> 2014-08-13 17:23:21.672536 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6816/837 -- pg_query(24.1bb epoch 194477) v3 -- ?+0 0x54fd640 con 0x4fde880 -3624> 2014-08-13 17:23:21.672539 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 2 ==== pg_notify(22.3e4(27) epoch 194477) v5 ==== 2231+0+0 (176340611 0 0) 0x56adb00 con 0x55f2100 -3623> 2014-08-13 17:23:21.672551 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6806/42024 -- pg_query(24.1bb epoch 194477) v3 -- ?+0 0x54fd460 con 0x4fdbde0 -3622> 2014-08-13 17:23:21.672555 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.663728, event: header_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -3621> 2014-08-13 17:23:21.672562 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.663730, event: throttled, request: pg_notify(22.3e4(27) epoch 194477) v5 -3620> 2014-08-13 17:23:21.672568 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.663850, event: all_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -3619> 2014-08-13 17:23:21.672570 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6804/62197 -- pg_query(24.1bb epoch 194477) v3 -- ?+0 0x54fd280 con 0x4fd8f20 -3618> 2014-08-13 17:23:21.672574 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.672553, event: dispatched, request: pg_notify(22.3e4(27) epoch 194477) v5 -3617> 2014-08-13 17:23:21.672584 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.672584, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194477) v5 -3616> 2014-08-13 17:23:21.672587 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6818/9142 -- pg_query(22.48e epoch 194477) v3 -- ?+0 0x54fd0a0 con 0x4f87900 -3615> 2014-08-13 17:23:21.672592 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.672592, event: started, request: pg_notify(22.3e4(27) epoch 194477) v5 -3614> 2014-08-13 17:23:21.672604 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6812/6757 -- pg_query(24.181 epoch 194477) v3 -- ?+0 0x54fcec0 con 0x4ec70c0 -3613> 2014-08-13 17:23:21.672612 7fcd0964c700 5 -- op tracker -- , seq: 277, time: 2014-08-13 17:23:21.672612, event: done, request: pg_notify(22.3e4(27) epoch 194477) v5 -3612> 2014-08-13 17:23:21.672633 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 2 ==== pg_notify(24.34(37) epoch 194477) v5 ==== 2623+0+0 (2256259889 0 0) 0x588d780 con 0x55244c0 -3611> 2014-08-13 17:23:21.672653 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.657044, event: header_read, request: pg_notify(24.34(37) epoch 194477) v5 -3610> 2014-08-13 17:23:21.672663 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.657046, event: throttled, request: pg_notify(24.34(37) epoch 194477) v5 -3609> 2014-08-13 17:23:21.672672 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.657180, event: all_read, request: pg_notify(24.34(37) epoch 194477) v5 -3608> 2014-08-13 17:23:21.672681 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.672651, event: dispatched, request: pg_notify(24.34(37) epoch 194477) v5 -3607> 2014-08-13 17:23:21.672691 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.672691, event: waiting_for_osdmap, request: pg_notify(24.34(37) epoch 194477) v5 -3606> 2014-08-13 17:23:21.672702 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.672701, event: started, request: pg_notify(24.34(37) epoch 194477) v5 -3605> 2014-08-13 17:23:21.672726 7fcd0964c700 5 -- op tracker -- , seq: 278, time: 2014-08-13 17:23:21.672726, event: done, request: pg_notify(24.34(37) epoch 194477) v5 -3604> 2014-08-13 17:23:21.672735 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6800/41204 -- pg_query(24.1bb epoch 194477) v3 -- ?+0 0x54fcce0 con 0x53e4fc0 -3603> 2014-08-13 17:23:21.672761 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_query(24.1f7 epoch 194477) v3 -- ?+0 0x54fcb00 con 0x4fdee00 -3602> 2014-08-13 17:23:21.672759 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.314 192.168.1.101:6802/22303 2 ==== pg_notify(22.619(26) epoch 194477) v5 ==== 2172+0+0 (3799552523 0 0) 0x588c600 con 0x552d800 -3601> 2014-08-13 17:23:21.672781 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.656094, event: header_read, request: pg_notify(22.619(26) epoch 194477) v5 -3600> 2014-08-13 17:23:21.672782 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_query(24.136 epoch 194477) v3 -- ?+0 0x54fc920 con 0x4ec4780 -3599> 2014-08-13 17:23:21.672794 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.656095, event: throttled, request: pg_notify(22.619(26) epoch 194477) v5 -3598> 2014-08-13 17:23:21.672803 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6842/8170 -- pg_query(22.3af epoch 194477) v3 -- ?+0 0x54fc740 con 0x4f86040 -3597> 2014-08-13 17:23:21.672806 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.656261, event: all_read, request: pg_notify(22.619(26) epoch 194477) v5 -3596> 2014-08-13 17:23:21.672816 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.672778, event: dispatched, request: pg_notify(22.619(26) epoch 194477) v5 -3595> 2014-08-13 17:23:21.672823 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.672822, event: waiting_for_osdmap, request: pg_notify(22.619(26) epoch 194477) v5 -3594> 2014-08-13 17:23:21.672830 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.672830, event: started, request: pg_notify(22.619(26) epoch 194477) v5 -3593> 2014-08-13 17:23:21.672849 7fcd0964c700 5 -- op tracker -- , seq: 279, time: 2014-08-13 17:23:21.672849, event: done, request: pg_notify(22.619(26) epoch 194477) v5 -3592> 2014-08-13 17:23:21.672869 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e7080 con 0x56b2260 -3591> 2014-08-13 17:23:21.672894 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3590> 2014-08-13 17:23:21.672907 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.11 192.168.1.103:6813/37929 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52bb840 con 0x56ba940 -3589> 2014-08-13 17:23:21.672925 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -3588> 2014-08-13 17:23:21.672940 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.293 192.168.1.109:6816/33855 3 ==== pg_notify(22.3d9(46) epoch 194477) v5 ==== 3208+0+0 (821084124 0 0) 0x54d0c40 con 0x5724e60 -3587> 2014-08-13 17:23:21.672957 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.672027, event: header_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -3586> 2014-08-13 17:23:21.672968 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.672029, event: throttled, request: pg_notify(22.3d9(46) epoch 194477) v5 -3585> 2014-08-13 17:23:21.672977 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.672204, event: all_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -3584> 2014-08-13 17:23:21.672986 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.672955, event: dispatched, request: pg_notify(22.3d9(46) epoch 194477) v5 -3583> 2014-08-13 17:23:21.672995 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.672995, event: waiting_for_osdmap, request: pg_notify(22.3d9(46) epoch 194477) v5 -3582> 2014-08-13 17:23:21.673006 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.673006, event: started, request: pg_notify(22.3d9(46) epoch 194477) v5 -3581> 2014-08-13 17:23:21.673043 7fcd0964c700 5 -- op tracker -- , seq: 280, time: 2014-08-13 17:23:21.673043, event: done, request: pg_notify(22.3d9(46) epoch 194477) v5 -3580> 2014-08-13 17:23:21.673078 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_query(22.48e,24.1f7 epoch 194477) v3 -- ?+0 0x54fc560 con 0x4fd5960 -3579> 2014-08-13 17:23:21.673095 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.47 192.168.1.111:6828/12102 2 ==== pg_notify(22.1f1(27) epoch 194477) v5 ==== 2231+0+0 (1112992215 0 0) 0x56af380 con 0x57232e0 -3578> 2014-08-13 17:23:21.673108 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6830/4861 -- pg_query(24.1bb epoch 194477) v3 -- ?+0 0x54fc380 con 0x4fdeb40 -3577> 2014-08-13 17:23:21.673117 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.658189, event: header_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3576> 2014-08-13 17:23:21.673127 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.658190, event: throttled, request: pg_notify(22.1f1(27) epoch 194477) v5 -3575> 2014-08-13 17:23:21.673130 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_info(1 pgs e194477:22.6bd(2)) v4 -- ?+0 0x56aa840 con 0x4eb7d20 -3574> 2014-08-13 17:23:21.673137 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.658624, event: all_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3573> 2014-08-13 17:23:21.673143 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.673114, event: dispatched, request: pg_notify(22.1f1(27) epoch 194477) v5 -3572> 2014-08-13 17:23:21.673155 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.673154, event: waiting_for_osdmap, request: pg_notify(22.1f1(27) epoch 194477) v5 -3571> 2014-08-13 17:23:21.673163 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.673163, event: started, request: pg_notify(22.1f1(27) epoch 194477) v5 -3570> 2014-08-13 17:23:21.673177 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6814/33435 -- pg_info(1 pgs e194477:22.481(42)) v4 -- ?+0 0x56a8000 con 0x4f853e0 -3569> 2014-08-13 17:23:21.673187 7fcd0964c700 5 -- op tracker -- , seq: 281, time: 2014-08-13 17:23:21.673187, event: done, request: pg_notify(22.1f1(27) epoch 194477) v5 -3568> 2014-08-13 17:23:21.673201 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_info(1 pgs e194477:22.6bd(2)) v4 -- ?+0 0x56a9880 con 0x4ec3440 -3567> 2014-08-13 17:23:21.673218 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.382 192.168.1.108:6800/30527 2 ==== pg_notify(22.3d9(46) epoch 194477) v5 ==== 3208+0+0 (440700594 0 0) 0x588f1c0 con 0x552d280 -3566> 2014-08-13 17:23:21.673235 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.661128, event: header_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -3565> 2014-08-13 17:23:21.673242 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.661129, event: throttled, request: pg_notify(22.3d9(46) epoch 194477) v5 -3564> 2014-08-13 17:23:21.673247 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.661379, event: all_read, request: pg_notify(22.3d9(46) epoch 194477) v5 -3563> 2014-08-13 17:23:21.673253 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.673232, event: dispatched, request: pg_notify(22.3d9(46) epoch 194477) v5 -3562> 2014-08-13 17:23:21.673259 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.673259, event: waiting_for_osdmap, request: pg_notify(22.3d9(46) epoch 194477) v5 -3561> 2014-08-13 17:23:21.673267 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.673266, event: started, request: pg_notify(22.3d9(46) epoch 194477) v5 -3560> 2014-08-13 17:23:21.673288 7fcd0964c700 5 -- op tracker -- , seq: 282, time: 2014-08-13 17:23:21.673288, event: done, request: pg_notify(22.3d9(46) epoch 194477) v5 -3559> 2014-08-13 17:23:21.673314 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 10 ==== pg_info(1 pgs e194477:22.f3) v4 ==== 714+0+0 (3452307152 0 0) 0x56ad780 con 0x4e314a0 -3558> 2014-08-13 17:23:21.673329 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.669445, event: header_read, request: pg_info(1 pgs e194477:22.f3) v4 -3557> 2014-08-13 17:23:21.673336 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.669447, event: throttled, request: pg_info(1 pgs e194477:22.f3) v4 -3556> 2014-08-13 17:23:21.673342 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.669521, event: all_read, request: pg_info(1 pgs e194477:22.f3) v4 -3555> 2014-08-13 17:23:21.673347 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.673327, event: dispatched, request: pg_info(1 pgs e194477:22.f3) v4 -3554> 2014-08-13 17:23:21.673353 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.673353, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.f3) v4 -3553> 2014-08-13 17:23:21.673361 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.673361, event: started, request: pg_info(1 pgs e194477:22.f3) v4 -3552> 2014-08-13 17:23:21.673381 7fcd0964c700 5 -- op tracker -- , seq: 283, time: 2014-08-13 17:23:21.673381, event: done, request: pg_info(1 pgs e194477:22.f3) v4 -3551> 2014-08-13 17:23:21.673400 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 9 ==== pg_notify(22.362(27) epoch 194477) v5 ==== 2231+0+0 (3551719807 0 0) 0x5889dc0 con 0x4e335a0 -3550> 2014-08-13 17:23:21.673420 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.650247, event: header_read, request: pg_notify(22.362(27) epoch 194477) v5 -3549> 2014-08-13 17:23:21.673427 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.650250, event: throttled, request: pg_notify(22.362(27) epoch 194477) v5 -3548> 2014-08-13 17:23:21.673433 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.650373, event: all_read, request: pg_notify(22.362(27) epoch 194477) v5 -3547> 2014-08-13 17:23:21.673438 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.673418, event: dispatched, request: pg_notify(22.362(27) epoch 194477) v5 -3546> 2014-08-13 17:23:21.673444 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.673444, event: waiting_for_osdmap, request: pg_notify(22.362(27) epoch 194477) v5 -3545> 2014-08-13 17:23:21.673452 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.673452, event: started, request: pg_notify(22.362(27) epoch 194477) v5 -3544> 2014-08-13 17:23:21.673472 7fcd0964c700 5 -- op tracker -- , seq: 284, time: 2014-08-13 17:23:21.673471, event: done, request: pg_notify(22.362(27) epoch 194477) v5 -3543> 2014-08-13 17:23:21.673468 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6814/23703 -- pg_info(1 pgs e194477:22.481(42)) v4 -- ?+0 0x56ad080 con 0x5303700 -3542> 2014-08-13 17:23:21.673516 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 6 ==== pg_notify(22.99(22) epoch 194477) v5 ==== 1944+0+0 (3288163467 0 0) 0x5187c40 con 0x4e33de0 -3541> 2014-08-13 17:23:21.673533 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.665520, event: header_read, request: pg_notify(22.99(22) epoch 194477) v5 -3540> 2014-08-13 17:23:21.673540 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.665522, event: throttled, request: pg_notify(22.99(22) epoch 194477) v5 -3539> 2014-08-13 17:23:21.673545 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.665660, event: all_read, request: pg_notify(22.99(22) epoch 194477) v5 -3538> 2014-08-13 17:23:21.673551 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.673531, event: dispatched, request: pg_notify(22.99(22) epoch 194477) v5 -3537> 2014-08-13 17:23:21.673557 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.673557, event: waiting_for_osdmap, request: pg_notify(22.99(22) epoch 194477) v5 -3536> 2014-08-13 17:23:21.673565 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.673564, event: started, request: pg_notify(22.99(22) epoch 194477) v5 -3535> 2014-08-13 17:23:21.673583 7fcd0964c700 5 -- op tracker -- , seq: 285, time: 2014-08-13 17:23:21.673583, event: done, request: pg_notify(22.99(22) epoch 194477) v5 -3534> 2014-08-13 17:23:21.673601 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 6 ==== pg_notify(24.b6(29) epoch 194477) v5 ==== 2379+0+0 (1511250803 0 0) 0x588c0c0 con 0x4e33f40 -3533> 2014-08-13 17:23:21.673616 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.658284, event: header_read, request: pg_notify(24.b6(29) epoch 194477) v5 -3532> 2014-08-13 17:23:21.673622 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.658286, event: throttled, request: pg_notify(24.b6(29) epoch 194477) v5 -3531> 2014-08-13 17:23:21.673628 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.658423, event: all_read, request: pg_notify(24.b6(29) epoch 194477) v5 -3530> 2014-08-13 17:23:21.673633 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.673614, event: dispatched, request: pg_notify(24.b6(29) epoch 194477) v5 -3529> 2014-08-13 17:23:21.673639 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.673639, event: waiting_for_osdmap, request: pg_notify(24.b6(29) epoch 194477) v5 -3528> 2014-08-13 17:23:21.673646 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.673646, event: started, request: pg_notify(24.b6(29) epoch 194477) v5 -3527> 2014-08-13 17:23:21.673666 7fcd0964c700 5 -- op tracker -- , seq: 286, time: 2014-08-13 17:23:21.673666, event: done, request: pg_notify(24.b6(29) epoch 194477) v5 -3526> 2014-08-13 17:23:21.673689 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 7 ==== pg_notify(22.1d2(23) epoch 194477) v5 ==== 2003+0+0 (1524527600 0 0) 0x56ac7c0 con 0x4e35ac0 -3525> 2014-08-13 17:23:21.673704 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.657950, event: header_read, request: pg_notify(22.1d2(23) epoch 194477) v5 -3524> 2014-08-13 17:23:21.673711 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.657951, event: throttled, request: pg_notify(22.1d2(23) epoch 194477) v5 -3523> 2014-08-13 17:23:21.673716 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.658041, event: all_read, request: pg_notify(22.1d2(23) epoch 194477) v5 -3522> 2014-08-13 17:23:21.673722 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.673702, event: dispatched, request: pg_notify(22.1d2(23) epoch 194477) v5 -3521> 2014-08-13 17:23:21.673728 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.673728, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194477) v5 -3520> 2014-08-13 17:23:21.673735 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.673735, event: started, request: pg_notify(22.1d2(23) epoch 194477) v5 -3519> 2014-08-13 17:23:21.673755 7fcd0964c700 5 -- op tracker -- , seq: 287, time: 2014-08-13 17:23:21.673755, event: done, request: pg_notify(22.1d2(23) epoch 194477) v5 -3518> 2014-08-13 17:23:21.673750 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.722224 7 0.000291 -3517> 2014-08-13 17:23:21.673781 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 7 ==== pg_notify(22.22c(23) epoch 194477) v5 ==== 2003+0+0 (2443476114 0 0) 0x588a300 con 0x4e35800 -3516> 2014-08-13 17:23:21.673784 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3515> 2014-08-13 17:23:21.673797 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.651575, event: header_read, request: pg_notify(22.22c(23) epoch 194477) v5 -3514> 2014-08-13 17:23:21.673804 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.651576, event: throttled, request: pg_notify(22.22c(23) epoch 194477) v5 -3513> 2014-08-13 17:23:21.673810 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.652613, event: all_read, request: pg_notify(22.22c(23) epoch 194477) v5 -3512> 2014-08-13 17:23:21.673815 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.673795, event: dispatched, request: pg_notify(22.22c(23) epoch 194477) v5 -3511> 2014-08-13 17:23:21.673821 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.673821, event: waiting_for_osdmap, request: pg_notify(22.22c(23) epoch 194477) v5 -3510> 2014-08-13 17:23:21.673829 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.673829, event: started, request: pg_notify(22.22c(23) epoch 194477) v5 -3509> 2014-08-13 17:23:21.673805 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192958) [397,343,4] r=2 lpr=194476 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3508> 2014-08-13 17:23:21.673850 7fcd0964c700 5 -- op tracker -- , seq: 288, time: 2014-08-13 17:23:21.673849, event: done, request: pg_notify(22.22c(23) epoch 194477) v5 -3507> 2014-08-13 17:23:21.673884 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 10 ==== pg_notify(22.3e4(26) epoch 194477) v5 ==== 2172+0+0 (1185747154 0 0) 0x54d6200 con 0x4ec09a0 -3506> 2014-08-13 17:23:21.673904 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.655832, event: header_read, request: pg_notify(22.3e4(26) epoch 194477) v5 -3505> 2014-08-13 17:23:21.673915 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.655833, event: throttled, request: pg_notify(22.3e4(26) epoch 194477) v5 -3504> 2014-08-13 17:23:21.673924 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.655950, event: all_read, request: pg_notify(22.3e4(26) epoch 194477) v5 -3503> 2014-08-13 17:23:21.673932 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.673902, event: dispatched, request: pg_notify(22.3e4(26) epoch 194477) v5 -3502> 2014-08-13 17:23:21.673942 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.673942, event: waiting_for_osdmap, request: pg_notify(22.3e4(26) epoch 194477) v5 -3501> 2014-08-13 17:23:21.673953 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.673953, event: started, request: pg_notify(22.3e4(26) epoch 194477) v5 -3500> 2014-08-13 17:23:21.673976 7fcd0964c700 5 -- op tracker -- , seq: 289, time: 2014-08-13 17:23:21.673976, event: done, request: pg_notify(22.3e4(26) epoch 194477) v5 -3499> 2014-08-13 17:23:21.674005 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 2 ==== pg_notify(22.619(27) epoch 194477) v5 ==== 2231+0+0 (857637660 0 0) 0x588fc40 con 0x4e370c0 -3498> 2014-08-13 17:23:21.674026 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.663955, event: header_read, request: pg_notify(22.619(27) epoch 194477) v5 -3497> 2014-08-13 17:23:21.674036 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.663957, event: throttled, request: pg_notify(22.619(27) epoch 194477) v5 -3496> 2014-08-13 17:23:21.674045 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.664344, event: all_read, request: pg_notify(22.619(27) epoch 194477) v5 -3495> 2014-08-13 17:23:21.674054 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.674024, event: dispatched, request: pg_notify(22.619(27) epoch 194477) v5 -3494> 2014-08-13 17:23:21.674064 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.674064, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194477) v5 -3493> 2014-08-13 17:23:21.674078 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.674077, event: started, request: pg_notify(22.619(27) epoch 194477) v5 -3492> 2014-08-13 17:23:21.674098 7fcd0964c700 5 -- op tracker -- , seq: 290, time: 2014-08-13 17:23:21.674098, event: done, request: pg_notify(22.619(27) epoch 194477) v5 -3491> 2014-08-13 17:23:21.674085 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194474/194474 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 0.750721 8 0.000450 -3490> 2014-08-13 17:23:21.674115 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194474/194474 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive -3489> 2014-08-13 17:23:21.674117 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 5 ==== pg_notify(22.1f1(26) epoch 194477) v5 ==== 2172+0+0 (2620928995 0 0) 0x54d6040 con 0x4ec2aa0 -3488> 2014-08-13 17:23:21.674133 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.650741, event: header_read, request: pg_notify(22.1f1(26) epoch 194477) v5 -3487> 2014-08-13 17:23:21.674140 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.650743, event: throttled, request: pg_notify(22.1f1(26) epoch 194477) v5 -3486> 2014-08-13 17:23:21.674135 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194472 n=52 ec=192688 les/c 194474/194474 194476/194476/193188) [17,4,395] r=1 lpr=194476 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering -3485> 2014-08-13 17:23:21.674146 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.650846, event: all_read, request: pg_notify(22.1f1(26) epoch 194477) v5 -3484> 2014-08-13 17:23:21.674157 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.674131, event: dispatched, request: pg_notify(22.1f1(26) epoch 194477) v5 -3483> 2014-08-13 17:23:21.674164 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.674164, event: waiting_for_osdmap, request: pg_notify(22.1f1(26) epoch 194477) v5 -3482> 2014-08-13 17:23:21.674171 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.674171, event: started, request: pg_notify(22.1f1(26) epoch 194477) v5 -3481> 2014-08-13 17:23:21.674190 7fcd0964c700 5 -- op tracker -- , seq: 291, time: 2014-08-13 17:23:21.674190, event: done, request: pg_notify(22.1f1(26) epoch 194477) v5 -3480> 2014-08-13 17:23:21.674222 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 9 ==== pg_info(1 pgs e194477:22.84) v4 ==== 714+0+0 (4141541345 0 0) 0x588bf00 con 0x4ec1ce0 -3479> 2014-08-13 17:23:21.674239 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.659216, event: header_read, request: pg_info(1 pgs e194477:22.84) v4 -3478> 2014-08-13 17:23:21.674246 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.659218, event: throttled, request: pg_info(1 pgs e194477:22.84) v4 -3477> 2014-08-13 17:23:21.674251 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.659288, event: all_read, request: pg_info(1 pgs e194477:22.84) v4 -3476> 2014-08-13 17:23:21.674257 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.674237, event: dispatched, request: pg_info(1 pgs e194477:22.84) v4 -3475> 2014-08-13 17:23:21.674263 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.674263, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.84) v4 -3474> 2014-08-13 17:23:21.674271 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.674271, event: started, request: pg_info(1 pgs e194477:22.84) v4 -3473> 2014-08-13 17:23:21.674290 7fcd0964c700 5 -- op tracker -- , seq: 292, time: 2014-08-13 17:23:21.674289, event: done, request: pg_info(1 pgs e194477:22.84) v4 -3472> 2014-08-13 17:23:21.674308 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 6 ==== pg_notify(24.34(50) epoch 194477) v5 ==== 3306+0+0 (4046455223 0 0) 0x54d1a40 con 0x4ec3f40 -3471> 2014-08-13 17:23:21.674323 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.665754, event: header_read, request: pg_notify(24.34(50) epoch 194477) v5 -3470> 2014-08-13 17:23:21.674330 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.665756, event: throttled, request: pg_notify(24.34(50) epoch 194477) v5 -3469> 2014-08-13 17:23:21.674335 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.665902, event: all_read, request: pg_notify(24.34(50) epoch 194477) v5 -3468> 2014-08-13 17:23:21.674341 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.674321, event: dispatched, request: pg_notify(24.34(50) epoch 194477) v5 -3467> 2014-08-13 17:23:21.674347 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.674347, event: waiting_for_osdmap, request: pg_notify(24.34(50) epoch 194477) v5 -3466> 2014-08-13 17:23:21.674355 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.674354, event: started, request: pg_notify(24.34(50) epoch 194477) v5 -3465> 2014-08-13 17:23:21.674380 7fcd0964c700 5 -- op tracker -- , seq: 293, time: 2014-08-13 17:23:21.674380, event: done, request: pg_notify(24.34(50) epoch 194477) v5 -3464> 2014-08-13 17:23:21.674385 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.397 192.168.1.105:6808/41085 -- pg_info(1 pgs e194477:22.3b9) v4 -- ?+0 0x5888a80 -3463> 2014-08-13 17:23:21.674408 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 6 ==== pg_notify(22.99(23) epoch 194477) v5 ==== 2003+0+0 (1847996741 0 0) 0x56a9500 con 0x4ec56a0 -3462> 2014-08-13 17:23:21.674423 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.665683, event: header_read, request: pg_notify(22.99(23) epoch 194477) v5 -3461> 2014-08-13 17:23:21.674435 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.665684, event: throttled, request: pg_notify(22.99(23) epoch 194477) v5 -3460> 2014-08-13 17:23:21.674441 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.665787, event: all_read, request: pg_notify(22.99(23) epoch 194477) v5 -3459> 2014-08-13 17:23:21.674447 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.674421, event: dispatched, request: pg_notify(22.99(23) epoch 194477) v5 -3458> 2014-08-13 17:23:21.674453 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.674453, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194477) v5 -3457> 2014-08-13 17:23:21.674461 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.674461, event: started, request: pg_notify(22.99(23) epoch 194477) v5 -3456> 2014-08-13 17:23:21.674480 7fcd0964c700 5 -- op tracker -- , seq: 294, time: 2014-08-13 17:23:21.674480, event: done, request: pg_notify(22.99(23) epoch 194477) v5 -3455> 2014-08-13 17:23:21.674496 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 11 ==== pg_notify(22.48e(5) epoch 194477) v5 ==== 1001+0+0 (1532929754 0 0) 0x5185080 con 0x4ec3440 -3454> 2014-08-13 17:23:21.674512 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.642794, event: header_read, request: pg_notify(22.48e(5) epoch 194477) v5 -3453> 2014-08-13 17:23:21.674518 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.642795, event: throttled, request: pg_notify(22.48e(5) epoch 194477) v5 -3452> 2014-08-13 17:23:21.674524 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.642873, event: all_read, request: pg_notify(22.48e(5) epoch 194477) v5 -3451> 2014-08-13 17:23:21.674529 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.674510, event: dispatched, request: pg_notify(22.48e(5) epoch 194477) v5 -3450> 2014-08-13 17:23:21.674536 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.674535, event: waiting_for_osdmap, request: pg_notify(22.48e(5) epoch 194477) v5 -3449> 2014-08-13 17:23:21.674543 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.674543, event: started, request: pg_notify(22.48e(5) epoch 194477) v5 -3448> 2014-08-13 17:23:21.674561 7fcd0964c700 5 -- op tracker -- , seq: 295, time: 2014-08-13 17:23:21.674561, event: done, request: pg_notify(22.48e(5) epoch 194477) v5 -3447> 2014-08-13 17:23:21.674575 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 10 ==== pg_query(24.3b7 epoch 194477) v3 ==== 144+0+0 (1548117580 0 0) 0x2f94560 con 0x4ec4780 -3446> 2014-08-13 17:23:21.674589 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.642963, event: header_read, request: pg_query(24.3b7 epoch 194477) v3 -3445> 2014-08-13 17:23:21.674596 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.642964, event: throttled, request: pg_query(24.3b7 epoch 194477) v3 -3444> 2014-08-13 17:23:21.674601 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.643007, event: all_read, request: pg_query(24.3b7 epoch 194477) v3 -3443> 2014-08-13 17:23:21.674607 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.674588, event: dispatched, request: pg_query(24.3b7 epoch 194477) v3 -3442> 2014-08-13 17:23:21.674618 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.674618, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194477) v3 -3441> 2014-08-13 17:23:21.674629 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.674629, event: started, request: pg_query(24.3b7 epoch 194477) v3 -3440> 2014-08-13 17:23:21.674651 7fcd0964c700 5 -- op tracker -- , seq: 296, time: 2014-08-13 17:23:21.674650, event: done, request: pg_query(24.3b7 epoch 194477) v3 -3439> 2014-08-13 17:23:21.674667 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 8 ==== pg_notify(24.b6(32) epoch 194477) v5 ==== 2556+0+0 (371489873 0 0) 0x5184980 con 0x4ec5ac0 -3438> 2014-08-13 17:23:21.674686 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.655532, event: header_read, request: pg_notify(24.b6(32) epoch 194477) v5 -3437> 2014-08-13 17:23:21.674696 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.655533, event: throttled, request: pg_notify(24.b6(32) epoch 194477) v5 -3436> 2014-08-13 17:23:21.674705 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.655659, event: all_read, request: pg_notify(24.b6(32) epoch 194477) v5 -3435> 2014-08-13 17:23:21.674714 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.674684, event: dispatched, request: pg_notify(24.b6(32) epoch 194477) v5 -3434> 2014-08-13 17:23:21.674723 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.674723, event: waiting_for_osdmap, request: pg_notify(24.b6(32) epoch 194477) v5 -3433> 2014-08-13 17:23:21.674734 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.674734, event: started, request: pg_notify(24.b6(32) epoch 194477) v5 -3432> 2014-08-13 17:23:21.674757 7fcd0964c700 5 -- op tracker -- , seq: 297, time: 2014-08-13 17:23:21.674757, event: done, request: pg_notify(24.b6(32) epoch 194477) v5 -3431> 2014-08-13 17:23:21.674747 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.714886 7 0.000237 -3430> 2014-08-13 17:23:21.674788 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive -3429> 2014-08-13 17:23:21.674795 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x52bee40 con 0x4ec70c0 -3428> 2014-08-13 17:23:21.674820 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -3427> 2014-08-13 17:23:21.674817 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3426> 2014-08-13 17:23:21.674835 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 6 ==== pg_notify(22.619(5) epoch 194477) v5 ==== 1001+0+0 (1347682708 0 0) 0x5b48380 con 0x4eb0c60 -3425> 2014-08-13 17:23:21.674853 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.666649, event: header_read, request: pg_notify(22.619(5) epoch 194477) v5 -3424> 2014-08-13 17:23:21.674864 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.666651, event: throttled, request: pg_notify(22.619(5) epoch 194477) v5 -3423> 2014-08-13 17:23:21.674874 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.666804, event: all_read, request: pg_notify(22.619(5) epoch 194477) v5 -3422> 2014-08-13 17:23:21.674886 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.674850, event: dispatched, request: pg_notify(22.619(5) epoch 194477) v5 -3421> 2014-08-13 17:23:21.674896 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.674896, event: waiting_for_osdmap, request: pg_notify(22.619(5) epoch 194477) v5 -3420> 2014-08-13 17:23:21.674907 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.674907, event: started, request: pg_notify(22.619(5) epoch 194477) v5 -3419> 2014-08-13 17:23:21.674931 7fcd0964c700 5 -- op tracker -- , seq: 298, time: 2014-08-13 17:23:21.674931, event: done, request: pg_notify(22.619(5) epoch 194477) v5 -3418> 2014-08-13 17:23:21.674956 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 3 ==== pg_notify(22.707(27) epoch 194477) v5 ==== 2231+0+0 (1829887906 0 0) 0x588d5c0 con 0x4ec7e80 -3417> 2014-08-13 17:23:21.674975 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.656999, event: header_read, request: pg_notify(22.707(27) epoch 194477) v5 -3416> 2014-08-13 17:23:21.674985 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.657000, event: throttled, request: pg_notify(22.707(27) epoch 194477) v5 -3415> 2014-08-13 17:23:21.674994 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.657099, event: all_read, request: pg_notify(22.707(27) epoch 194477) v5 -3414> 2014-08-13 17:23:21.675004 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.674973, event: dispatched, request: pg_notify(22.707(27) epoch 194477) v5 -3413> 2014-08-13 17:23:21.675013 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.675013, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194477) v5 -3412> 2014-08-13 17:23:21.675025 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.675025, event: started, request: pg_notify(22.707(27) epoch 194477) v5 -3411> 2014-08-13 17:23:21.675048 7fcd0964c700 5 -- op tracker -- , seq: 299, time: 2014-08-13 17:23:21.675048, event: done, request: pg_notify(22.707(27) epoch 194477) v5 -3410> 2014-08-13 17:23:21.675070 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 8 ==== pg_info(1 pgs e194477:24.2ec(4)) v4 ==== 976+0+0 (86883280 0 0) 0x54d7e00 con 0x4eb18c0 -3409> 2014-08-13 17:23:21.675062 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.762218 7 0.000263 -3408> 2014-08-13 17:23:21.675086 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.646788, event: header_read, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3407> 2014-08-13 17:23:21.675094 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.647998, event: throttled, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3406> 2014-08-13 17:23:21.675100 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.648093, event: all_read, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3405> 2014-08-13 17:23:21.675091 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3404> 2014-08-13 17:23:21.675109 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.675084, event: dispatched, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3403> 2014-08-13 17:23:21.675120 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.675120, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3402> 2014-08-13 17:23:21.675115 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192935) [58,28,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3401> 2014-08-13 17:23:21.675129 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.675129, event: started, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3400> 2014-08-13 17:23:21.675150 7fcd0964c700 5 -- op tracker -- , seq: 300, time: 2014-08-13 17:23:21.675150, event: done, request: pg_info(1 pgs e194477:24.2ec(4)) v4 -3399> 2014-08-13 17:23:21.675102 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.6 192.168.1.107:6809/53141 -- pg_info(1 pgs e194477:24.cd) v4 -- ?+0 0x5888540 -3398> 2014-08-13 17:23:21.675165 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 2 ==== pg_notify(22.775(27) epoch 194477) v5 ==== 2231+0+0 (1367275308 0 0) 0x54d1340 con 0x4eb48e0 -3397> 2014-08-13 17:23:21.675180 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.667602, event: header_read, request: pg_notify(22.775(27) epoch 194477) v5 -3396> 2014-08-13 17:23:21.675187 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.667603, event: throttled, request: pg_notify(22.775(27) epoch 194477) v5 -3395> 2014-08-13 17:23:21.675193 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.667710, event: all_read, request: pg_notify(22.775(27) epoch 194477) v5 -3394> 2014-08-13 17:23:21.675198 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.675178, event: dispatched, request: pg_notify(22.775(27) epoch 194477) v5 -3393> 2014-08-13 17:23:21.675212 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.675211, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194477) v5 -3392> 2014-08-13 17:23:21.675228 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.675228, event: started, request: pg_notify(22.775(27) epoch 194477) v5 -3391> 2014-08-13 17:23:21.675252 7fcd0964c700 5 -- op tracker -- , seq: 301, time: 2014-08-13 17:23:21.675252, event: done, request: pg_notify(22.775(27) epoch 194477) v5 -3390> 2014-08-13 17:23:21.675253 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.712984 7 0.000335 -3389> 2014-08-13 17:23:21.675281 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 7 ==== pg_notify(24.136(6) epoch 194477) v5 ==== 1094+0+0 (1572723595 0 0) 0x56a9f80 con 0x4eb7d20 -3388> 2014-08-13 17:23:21.675282 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3387> 2014-08-13 17:23:21.675297 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.673258, event: header_read, request: pg_notify(24.136(6) epoch 194477) v5 -3386> 2014-08-13 17:23:21.675305 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.673260, event: throttled, request: pg_notify(24.136(6) epoch 194477) v5 -3385> 2014-08-13 17:23:21.675310 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.673332, event: all_read, request: pg_notify(24.136(6) epoch 194477) v5 -3384> 2014-08-13 17:23:21.675302 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192981) [345,24,4] r=2 lpr=194476 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3383> 2014-08-13 17:23:21.675316 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.675295, event: dispatched, request: pg_notify(24.136(6) epoch 194477) v5 -3382> 2014-08-13 17:23:21.675323 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.675323, event: waiting_for_osdmap, request: pg_notify(24.136(6) epoch 194477) v5 -3381> 2014-08-13 17:23:21.675335 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.675335, event: started, request: pg_notify(24.136(6) epoch 194477) v5 -3380> 2014-08-13 17:23:21.675325 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.396 192.168.1.105:6814/41806 -- pg_info(1 pgs e194477:24.299) v4 -- ?+0 0x5888700 -3379> 2014-08-13 17:23:21.675363 7fcd0964c700 5 -- op tracker -- , seq: 302, time: 2014-08-13 17:23:21.675362, event: done, request: pg_notify(24.136(6) epoch 194477) v5 -3378> 2014-08-13 17:23:21.675381 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.296 192.168.1.109:6806/31286 5 ==== pg_notify(24.181(28) epoch 194477) v5 ==== 2316+0+0 (2840360595 0 0) 0x54d7000 con 0x4f81600 -3377> 2014-08-13 17:23:21.675399 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.673415, event: header_read, request: pg_notify(24.181(28) epoch 194477) v5 -3376> 2014-08-13 17:23:21.675409 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.673417, event: throttled, request: pg_notify(24.181(28) epoch 194477) v5 -3375> 2014-08-13 17:23:21.675419 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.673525, event: all_read, request: pg_notify(24.181(28) epoch 194477) v5 -3374> 2014-08-13 17:23:21.675411 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194474/194474 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.765612 7 0.001341 -3373> 2014-08-13 17:23:21.675428 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.675397, event: dispatched, request: pg_notify(24.181(28) epoch 194477) v5 -3372> 2014-08-13 17:23:21.675473 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.675473, event: waiting_for_osdmap, request: pg_notify(24.181(28) epoch 194477) v5 -3371> 2014-08-13 17:23:21.675444 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194474/194474 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3370> 2014-08-13 17:23:21.675489 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.675488, event: started, request: pg_notify(24.181(28) epoch 194477) v5 -3369> 2014-08-13 17:23:21.675524 7fcd0964c700 5 -- op tracker -- , seq: 303, time: 2014-08-13 17:23:21.675524, event: done, request: pg_notify(24.181(28) epoch 194477) v5 -3368> 2014-08-13 17:23:21.675513 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194471 n=42 ec=183096 les/c 194474/194474 194476/194476/192972) [9,292,4] r=2 lpr=194476 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3367> 2014-08-13 17:23:21.675554 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 6 ==== pg_notify(22.707(28) epoch 194477) v5 ==== 2290+0+0 (3371788189 0 0) 0x54d3100 con 0x4f80b00 -3366> 2014-08-13 17:23:21.675577 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.655411, event: header_read, request: pg_notify(22.707(28) epoch 194477) v5 -3365> 2014-08-13 17:23:21.675567 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194477:22.14a) v4 -- ?+0 0x588b9c0 -3364> 2014-08-13 17:23:21.675592 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.655412, event: throttled, request: pg_notify(22.707(28) epoch 194477) v5 -3363> 2014-08-13 17:23:21.675618 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.655521, event: all_read, request: pg_notify(22.707(28) epoch 194477) v5 -3362> 2014-08-13 17:23:21.675643 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.675574, event: dispatched, request: pg_notify(22.707(28) epoch 194477) v5 -3361> 2014-08-13 17:23:21.675654 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.675653, event: waiting_for_osdmap, request: pg_notify(22.707(28) epoch 194477) v5 -3360> 2014-08-13 17:23:21.675668 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.675667, event: started, request: pg_notify(22.707(28) epoch 194477) v5 -3359> 2014-08-13 17:23:21.675689 7fcd0964c700 5 -- op tracker -- , seq: 304, time: 2014-08-13 17:23:21.675688, event: done, request: pg_notify(22.707(28) epoch 194477) v5 -3358> 2014-08-13 17:23:21.675687 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.725787 7 0.000341 -3357> 2014-08-13 17:23:21.675715 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 4 ==== pg_notify(22.3d9(42) epoch 194477) v5 ==== 2996+0+0 (2682887844 0 0) 0x56acb40 con 0x4f832e0 -3356> 2014-08-13 17:23:21.675716 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3355> 2014-08-13 17:23:21.675744 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.663319, event: header_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -3354> 2014-08-13 17:23:21.675752 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.663320, event: throttled, request: pg_notify(22.3d9(42) epoch 194477) v5 -3353> 2014-08-13 17:23:21.675758 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.663453, event: all_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -3352> 2014-08-13 17:23:21.675764 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.675742, event: dispatched, request: pg_notify(22.3d9(42) epoch 194477) v5 -3351> 2014-08-13 17:23:21.675771 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.675771, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194477) v5 -3350> 2014-08-13 17:23:21.675761 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192949) [328,364,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3349> 2014-08-13 17:23:21.675779 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.675779, event: started, request: pg_notify(22.3d9(42) epoch 194477) v5 -3348> 2014-08-13 17:23:21.675755 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.759231 7 0.000283 -3347> 2014-08-13 17:23:21.675777 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194477:22.29a) v4 -- ?+0 0x58881c0 -3346> 2014-08-13 17:23:21.675799 7fcd0964c700 5 -- op tracker -- , seq: 305, time: 2014-08-13 17:23:21.675799, event: done, request: pg_notify(22.3d9(42) epoch 194477) v5 -3345> 2014-08-13 17:23:21.675793 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3344> 2014-08-13 17:23:21.675814 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194471 n=34 ec=183096 les/c 194474/194474 194476/194476/192971) [321,379,4] r=2 lpr=194476 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3343> 2014-08-13 17:23:21.675831 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 6 ==== pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 ==== 3343+0+0 (3306503829 0 0) 0x588a840 con 0x4fd9a20 -3342> 2014-08-13 17:23:21.675850 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.651824, event: header_read, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3341> 2014-08-13 17:23:21.675862 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.651827, event: throttled, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3340> 2014-08-13 17:23:21.675872 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.654205, event: all_read, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3339> 2014-08-13 17:23:21.675882 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.675847, event: dispatched, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3338> 2014-08-13 17:23:21.675893 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.675893, event: waiting_for_osdmap, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3337> 2014-08-13 17:23:21.675906 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.675905, event: started, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3336> 2014-08-13 17:23:21.675915 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.756930 7 0.000666 -3335> 2014-08-13 17:23:21.675941 7fcd0964c700 5 -- op tracker -- , seq: 306, time: 2014-08-13 17:23:21.675940, event: done, request: pg_notify(24.1f7(6),24.136(27) epoch 194477) v5 -3334> 2014-08-13 17:23:21.675950 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3333> 2014-08-13 17:23:21.675962 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 6 ==== pg_notify(22.619(27) epoch 194477) v5 ==== 2231+0+0 (1628572869 0 0) 0x54d3f00 con 0x4fd9760 -3332> 2014-08-13 17:23:21.675959 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(26) epoch 194477) v5 -- ?+0 0x51878c0 con 0x4fde300 -3331> 2014-08-13 17:23:21.675986 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.667193, event: header_read, request: pg_notify(22.619(27) epoch 194477) v5 -3330> 2014-08-13 17:23:21.675994 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.667196, event: throttled, request: pg_notify(22.619(27) epoch 194477) v5 -3329> 2014-08-13 17:23:21.675973 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194471 n=31 ec=183096 les/c 194474/194474 194476/194476/192968) [294,327,4] r=2 lpr=194476 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3328> 2014-08-13 17:23:21.676001 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.667337, event: all_read, request: pg_notify(22.619(27) epoch 194477) v5 -3327> 2014-08-13 17:23:21.676006 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.675984, event: dispatched, request: pg_notify(22.619(27) epoch 194477) v5 -3326> 2014-08-13 17:23:21.676012 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.676012, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194477) v5 -3325> 2014-08-13 17:23:21.676020 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.676020, event: started, request: pg_notify(22.619(27) epoch 194477) v5 -3324> 2014-08-13 17:23:21.676050 7fcd0964c700 5 -- op tracker -- , seq: 307, time: 2014-08-13 17:23:21.676049, event: done, request: pg_notify(22.619(27) epoch 194477) v5 -3323> 2014-08-13 17:23:21.676071 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.724759 7 0.000226 -3322> 2014-08-13 17:23:21.676097 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 10 ==== pg_notify(22.362(27) epoch 194477) v5 ==== 2231+0+0 (523403142 0 0) 0x54d1500 con 0x4f86e00 -3321> 2014-08-13 17:23:21.676101 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive -3320> 2014-08-13 17:23:21.676116 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.642221, event: header_read, request: pg_notify(22.362(27) epoch 194477) v5 -3319> 2014-08-13 17:23:21.676124 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.642223, event: throttled, request: pg_notify(22.362(27) epoch 194477) v5 -3318> 2014-08-13 17:23:21.676130 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.642342, event: all_read, request: pg_notify(22.362(27) epoch 194477) v5 -3317> 2014-08-13 17:23:21.676123 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194474/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194476 pi=194462-194475/5 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3316> 2014-08-13 17:23:21.676136 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.676114, event: dispatched, request: pg_notify(22.362(27) epoch 194477) v5 -3315> 2014-08-13 17:23:21.676143 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.676143, event: waiting_for_osdmap, request: pg_notify(22.362(27) epoch 194477) v5 -3314> 2014-08-13 17:23:21.676151 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.676151, event: started, request: pg_notify(22.362(27) epoch 194477) v5 -3313> 2014-08-13 17:23:21.676180 7fcd0964c700 5 -- op tracker -- , seq: 308, time: 2014-08-13 17:23:21.676180, event: done, request: pg_notify(22.362(27) epoch 194477) v5 -3312> 2014-08-13 17:23:21.676200 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.726299 7 0.000375 -3311> 2014-08-13 17:23:21.676227 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 7 ==== pg_notify(24.1bb(30) epoch 194477) v5 ==== 2438+0+0 (2659733062 0 0) 0x54d0e00 con 0x4fd8f20 -3310> 2014-08-13 17:23:21.676227 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3309> 2014-08-13 17:23:21.676252 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.673315, event: header_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3308> 2014-08-13 17:23:21.676260 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.673318, event: throttled, request: pg_notify(24.1bb(30) epoch 194477) v5 -3307> 2014-08-13 17:23:21.676253 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192966) [358,374,4] r=2 lpr=194476 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3306> 2014-08-13 17:23:21.676266 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.673429, event: all_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3305> 2014-08-13 17:23:21.676272 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.676246, event: dispatched, request: pg_notify(24.1bb(30) epoch 194477) v5 -3304> 2014-08-13 17:23:21.676278 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.676278, event: waiting_for_osdmap, request: pg_notify(24.1bb(30) epoch 194477) v5 -3303> 2014-08-13 17:23:21.676286 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.676286, event: started, request: pg_notify(24.1bb(30) epoch 194477) v5 -3302> 2014-08-13 17:23:21.676306 7fcd0964c700 5 -- op tracker -- , seq: 309, time: 2014-08-13 17:23:21.676306, event: done, request: pg_notify(24.1bb(30) epoch 194477) v5 -3301> 2014-08-13 17:23:21.676330 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.332 192.168.1.110:6806/42024 5 ==== pg_notify(24.1bb(6) epoch 194477) v5 ==== 1094+0+0 (2664606886 0 0) 0x54d3d40 con 0x4fdbde0 -3300> 2014-08-13 17:23:21.676346 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.673583, event: header_read, request: pg_notify(24.1bb(6) epoch 194477) v5 -3299> 2014-08-13 17:23:21.676353 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.673584, event: throttled, request: pg_notify(24.1bb(6) epoch 194477) v5 -3298> 2014-08-13 17:23:21.676358 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.673661, event: all_read, request: pg_notify(24.1bb(6) epoch 194477) v5 -3297> 2014-08-13 17:23:21.676363 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.676343, event: dispatched, request: pg_notify(24.1bb(6) epoch 194477) v5 -3296> 2014-08-13 17:23:21.676369 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.676369, event: waiting_for_osdmap, request: pg_notify(24.1bb(6) epoch 194477) v5 -3295> 2014-08-13 17:23:21.676377 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.676377, event: started, request: pg_notify(24.1bb(6) epoch 194477) v5 -3294> 2014-08-13 17:23:21.676400 7fcd0964c700 5 -- op tracker -- , seq: 310, time: 2014-08-13 17:23:21.676400, event: done, request: pg_notify(24.1bb(6) epoch 194477) v5 -3293> 2014-08-13 17:23:21.676416 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 7 ==== pg_notify(24.1f7(32) epoch 194477) v5 ==== 2556+0+0 (208552386 0 0) 0x5b496c0 con 0x4fdee00 -3292> 2014-08-13 17:23:21.676437 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.674025, event: header_read, request: pg_notify(24.1f7(32) epoch 194477) v5 -3291> 2014-08-13 17:23:21.676444 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.674028, event: throttled, request: pg_notify(24.1f7(32) epoch 194477) v5 -3290> 2014-08-13 17:23:21.676452 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.674245, event: all_read, request: pg_notify(24.1f7(32) epoch 194477) v5 -3289> 2014-08-13 17:23:21.676462 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.676434, event: dispatched, request: pg_notify(24.1f7(32) epoch 194477) v5 -3288> 2014-08-13 17:23:21.676468 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.676468, event: waiting_for_osdmap, request: pg_notify(24.1f7(32) epoch 194477) v5 -3287> 2014-08-13 17:23:21.676476 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.676475, event: started, request: pg_notify(24.1f7(32) epoch 194477) v5 -3286> 2014-08-13 17:23:21.676494 7fcd0964c700 5 -- op tracker -- , seq: 311, time: 2014-08-13 17:23:21.676494, event: done, request: pg_notify(24.1f7(32) epoch 194477) v5 -3285> 2014-08-13 17:23:21.676539 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.379 192.168.1.108:6830/4861 3 ==== pg_notify(24.1bb(30) epoch 194477) v5 ==== 2438+0+0 (2659733062 0 0) 0x54d0c40 con 0x4fdeb40 -3284> 2014-08-13 17:23:21.676564 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.674213, event: header_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3283> 2014-08-13 17:23:21.676558 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(30) epoch 194477) v5 -- ?+0 0x588d240 con 0x4f80b00 -3282> 2014-08-13 17:23:21.676575 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.674216, event: throttled, request: pg_notify(24.1bb(30) epoch 194477) v5 -3281> 2014-08-13 17:23:21.676585 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.674335, event: all_read, request: pg_notify(24.1bb(30) epoch 194477) v5 -3280> 2014-08-13 17:23:21.676595 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.676562, event: dispatched, request: pg_notify(24.1bb(30) epoch 194477) v5 -3279> 2014-08-13 17:23:21.676604 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.676604, event: waiting_for_osdmap, request: pg_notify(24.1bb(30) epoch 194477) v5 -3278> 2014-08-13 17:23:21.676616 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.676615, event: started, request: pg_notify(24.1bb(30) epoch 194477) v5 -3277> 2014-08-13 17:23:21.676639 7fcd0964c700 5 -- op tracker -- , seq: 312, time: 2014-08-13 17:23:21.676639, event: done, request: pg_notify(24.1bb(30) epoch 194477) v5 -3276> 2014-08-13 17:23:21.676666 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 6 ==== pg_notify(24.1bb(27) epoch 194477) v5 ==== 2261+0+0 (3277685512 0 0) 0x56ad5c0 con 0x4fde880 -3275> 2014-08-13 17:23:21.676687 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.658096, event: header_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -3274> 2014-08-13 17:23:21.676695 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.658099, event: throttled, request: pg_notify(24.1bb(27) epoch 194477) v5 -3273> 2014-08-13 17:23:21.676700 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.658232, event: all_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -3272> 2014-08-13 17:23:21.676706 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.676684, event: dispatched, request: pg_notify(24.1bb(27) epoch 194477) v5 -3271> 2014-08-13 17:23:21.676685 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194474/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 0.757215 9 0.000567 -3270> 2014-08-13 17:23:21.676713 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.676712, event: waiting_for_osdmap, request: pg_notify(24.1bb(27) epoch 194477) v5 -3269> 2014-08-13 17:23:21.676721 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.676720, event: started, request: pg_notify(24.1bb(27) epoch 194477) v5 -3268> 2014-08-13 17:23:21.676717 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194474/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive -3267> 2014-08-13 17:23:21.676741 7fcd0964c700 5 -- op tracker -- , seq: 313, time: 2014-08-13 17:23:21.676741, event: done, request: pg_notify(24.1bb(27) epoch 194477) v5 -3266> 2014-08-13 17:23:21.676741 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194472 n=69 ec=192688 les/c 194474/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194476 pi=193125-194475/28 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering -3265> 2014-08-13 17:23:21.676759 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.324 192.168.1.104:6814/65209 6 ==== pg_info(1 pgs e194477:22.22c) v4 ==== 714+0+0 (3764557236 0 0) 0x56afa80 con 0x4fde5c0 -3264> 2014-08-13 17:23:21.676774 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.668361, event: header_read, request: pg_info(1 pgs e194477:22.22c) v4 -3263> 2014-08-13 17:23:21.676781 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.668365, event: throttled, request: pg_info(1 pgs e194477:22.22c) v4 -3262> 2014-08-13 17:23:21.676787 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.668462, event: all_read, request: pg_info(1 pgs e194477:22.22c) v4 -3261> 2014-08-13 17:23:21.676797 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.676773, event: dispatched, request: pg_info(1 pgs e194477:22.22c) v4 -3260> 2014-08-13 17:23:21.676806 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.676806, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.22c) v4 -3259> 2014-08-13 17:23:21.676817 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.676817, event: started, request: pg_info(1 pgs e194477:22.22c) v4 -3258> 2014-08-13 17:23:21.676842 7fcd0964c700 5 -- op tracker -- , seq: 314, time: 2014-08-13 17:23:21.676841, event: done, request: pg_info(1 pgs e194477:22.22c) v4 -3257> 2014-08-13 17:23:21.676843 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194477:24.2a1) v4 -- ?+0 0x5888380 -3256> 2014-08-13 17:23:21.676860 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 7 ==== pg_notify(22.39d(24) epoch 194477) v5 ==== 2058+0+0 (2245716050 0 0) 0x588de80 con 0x4fde300 -3255> 2014-08-13 17:23:21.676876 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.657159, event: header_read, request: pg_notify(22.39d(24) epoch 194477) v5 -3254> 2014-08-13 17:23:21.676888 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.657162, event: throttled, request: pg_notify(22.39d(24) epoch 194477) v5 -3253> 2014-08-13 17:23:21.676894 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.657279, event: all_read, request: pg_notify(22.39d(24) epoch 194477) v5 -3252> 2014-08-13 17:23:21.676903 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.676874, event: dispatched, request: pg_notify(22.39d(24) epoch 194477) v5 -3251> 2014-08-13 17:23:21.676909 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.676909, event: waiting_for_osdmap, request: pg_notify(22.39d(24) epoch 194477) v5 -3250> 2014-08-13 17:23:21.676920 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.676920, event: started, request: pg_notify(22.39d(24) epoch 194477) v5 -3249> 2014-08-13 17:23:21.676902 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.9 192.168.1.107:6817/53997 -- pg_info(1 pgs e194477:22.28f) v4 -- ?+0 0x588a300 -3248> 2014-08-13 17:23:21.676943 7fcd0964c700 5 -- op tracker -- , seq: 315, time: 2014-08-13 17:23:21.676943, event: done, request: pg_notify(22.39d(24) epoch 194477) v5 -3247> 2014-08-13 17:23:21.676975 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 9 ==== pg_info(1 pgs e194477:24.9b) v4 ==== 748+0+0 (3105858428 0 0) 0x5b4a300 con 0x4fd3860 -3246> 2014-08-13 17:23:21.676996 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.669806, event: header_read, request: pg_info(1 pgs e194477:24.9b) v4 -3245> 2014-08-13 17:23:21.676995 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194477:22.2b9) v4 -- ?+0 0x56acb40 -3244> 2014-08-13 17:23:21.677006 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.669807, event: throttled, request: pg_info(1 pgs e194477:24.9b) v4 -3243> 2014-08-13 17:23:21.677016 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.669891, event: all_read, request: pg_info(1 pgs e194477:24.9b) v4 -3242> 2014-08-13 17:23:21.677022 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.676994, event: dispatched, request: pg_info(1 pgs e194477:24.9b) v4 -3241> 2014-08-13 17:23:21.677029 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.677029, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.9b) v4 -3240> 2014-08-13 17:23:21.677037 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.677036, event: started, request: pg_info(1 pgs e194477:24.9b) v4 -3239> 2014-08-13 17:23:21.677032 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.321 192.168.1.110:6812/43236 -- pg_info(1 pgs e194477:22.4aa) v4 -- ?+0 0x56a9f80 -3238> 2014-08-13 17:23:21.677057 7fcd0964c700 5 -- op tracker -- , seq: 316, time: 2014-08-13 17:23:21.677057, event: done, request: pg_info(1 pgs e194477:24.9b) v4 -3237> 2014-08-13 17:23:21.677071 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.6 192.168.1.107:6809/53141 7 ==== pg_info(1 pgs e194477:24.cd) v4 ==== 748+0+0 (3135928259 0 0) 0x5888540 con 0x4fd0f20 -3236> 2014-08-13 17:23:21.677086 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.676185, event: header_read, request: pg_info(1 pgs e194477:24.cd) v4 -3235> 2014-08-13 17:23:21.677092 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.676187, event: throttled, request: pg_info(1 pgs e194477:24.cd) v4 -3234> 2014-08-13 17:23:21.677097 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.676255, event: all_read, request: pg_info(1 pgs e194477:24.cd) v4 -3233> 2014-08-13 17:23:21.677103 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.677084, event: dispatched, request: pg_info(1 pgs e194477:24.cd) v4 -3232> 2014-08-13 17:23:21.677109 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.677109, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.cd) v4 -3231> 2014-08-13 17:23:21.677121 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.677120, event: started, request: pg_info(1 pgs e194477:24.cd) v4 -3230> 2014-08-13 17:23:21.677144 7fcd0964c700 5 -- op tracker -- , seq: 317, time: 2014-08-13 17:23:21.677144, event: done, request: pg_info(1 pgs e194477:24.cd) v4 -3229> 2014-08-13 17:23:21.677161 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 3 ==== pg_notify(24.b6(6) epoch 194477) v5 ==== 1094+0+0 (694073278 0 0) 0x588e200 con 0x4fd35a0 -3228> 2014-08-13 17:23:21.677185 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.657398, event: header_read, request: pg_notify(24.b6(6) epoch 194477) v5 -3227> 2014-08-13 17:23:21.677213 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.657399, event: throttled, request: pg_notify(24.b6(6) epoch 194477) v5 -3226> 2014-08-13 17:23:21.677224 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.657481, event: all_read, request: pg_notify(24.b6(6) epoch 194477) v5 -3225> 2014-08-13 17:23:21.677233 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.677183, event: dispatched, request: pg_notify(24.b6(6) epoch 194477) v5 -3224> 2014-08-13 17:23:21.677243 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.677243, event: waiting_for_osdmap, request: pg_notify(24.b6(6) epoch 194477) v5 -3223> 2014-08-13 17:23:21.677254 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.677254, event: started, request: pg_notify(24.b6(6) epoch 194477) v5 -3222> 2014-08-13 17:23:21.677278 7fcd0964c700 5 -- op tracker -- , seq: 318, time: 2014-08-13 17:23:21.677277, event: done, request: pg_notify(24.b6(6) epoch 194477) v5 -3221> 2014-08-13 17:23:21.677297 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 8 ==== pg_notify(22.707(5),24.3d(26) epoch 194477) v5 ==== 3195+0+0 (470973262 0 0) 0x54d63c0 con 0x4fd06e0 -3220> 2014-08-13 17:23:21.677314 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.656415, event: header_read, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3219> 2014-08-13 17:23:21.677322 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.656416, event: throttled, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3218> 2014-08-13 17:23:21.677306 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.294 192.168.1.101:6812/23402 -- pg_info(1 pgs e194477:22.630) v4 -- ?+0 0x56a9500 -3217> 2014-08-13 17:23:21.677328 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.656535, event: all_read, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3216> 2014-08-13 17:23:21.677334 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.677312, event: dispatched, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3215> 2014-08-13 17:23:21.677341 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.677341, event: waiting_for_osdmap, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3214> 2014-08-13 17:23:21.677349 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.677348, event: started, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3213> 2014-08-13 17:23:21.677380 7fcd0964c700 5 -- op tracker -- , seq: 319, time: 2014-08-13 17:23:21.677379, event: done, request: pg_notify(22.707(5),24.3d(26) epoch 194477) v5 -3212> 2014-08-13 17:23:21.677412 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 7 ==== pg_query(22.226 epoch 194477) v3 ==== 144+0+0 (2333532171 0 0) 0x2f96ea0 con 0x4fd5960 -3211> 2014-08-13 17:23:21.677432 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.645709, event: header_read, request: pg_query(22.226 epoch 194477) v3 -3210> 2014-08-13 17:23:21.677442 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.645710, event: throttled, request: pg_query(22.226 epoch 194477) v3 -3209> 2014-08-13 17:23:21.677452 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.645770, event: all_read, request: pg_query(22.226 epoch 194477) v3 -3208> 2014-08-13 17:23:21.677460 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.677430, event: dispatched, request: pg_query(22.226 epoch 194477) v3 -3207> 2014-08-13 17:23:21.677470 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.677470, event: waiting_for_osdmap, request: pg_query(22.226 epoch 194477) v3 -3206> 2014-08-13 17:23:21.677481 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.677480, event: started, request: pg_query(22.226 epoch 194477) v3 -3205> 2014-08-13 17:23:21.677502 7fcd0964c700 5 -- op tracker -- , seq: 320, time: 2014-08-13 17:23:21.677502, event: done, request: pg_query(22.226 epoch 194477) v3 -3204> 2014-08-13 17:23:21.677519 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 10 ==== pg_info(1 pgs e194477:22.1fb(1)) v4 ==== 765+0+0 (3537679781 0 0) 0x56a8700 con 0x4fd32e0 -3203> 2014-08-13 17:23:21.677538 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.642328, event: header_read, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3202> 2014-08-13 17:23:21.677548 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.642330, event: throttled, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3201> 2014-08-13 17:23:21.677558 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.642387, event: all_read, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3200> 2014-08-13 17:23:21.677567 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.677536, event: dispatched, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3199> 2014-08-13 17:23:21.677576 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.677576, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3198> 2014-08-13 17:23:21.677587 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.677587, event: started, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3197> 2014-08-13 17:23:21.677610 7fcd0964c700 5 -- op tracker -- , seq: 321, time: 2014-08-13 17:23:21.677610, event: done, request: pg_info(1 pgs e194477:22.1fb(1)) v4 -3196> 2014-08-13 17:23:21.677629 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 6 ==== pg_info(1 pgs e194477:22.3b9) v4 ==== 714+0+0 (572121974 0 0) 0x5888a80 con 0x4fd53e0 -3195> 2014-08-13 17:23:21.677647 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.675092, event: header_read, request: pg_info(1 pgs e194477:22.3b9) v4 -3194> 2014-08-13 17:23:21.677658 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.675093, event: throttled, request: pg_info(1 pgs e194477:22.3b9) v4 -3193> 2014-08-13 17:23:21.677667 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.675195, event: all_read, request: pg_info(1 pgs e194477:22.3b9) v4 -3192> 2014-08-13 17:23:21.677675 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.677645, event: dispatched, request: pg_info(1 pgs e194477:22.3b9) v4 -3191> 2014-08-13 17:23:21.677682 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.677682, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3b9) v4 -3190> 2014-08-13 17:23:21.677689 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.677689, event: started, request: pg_info(1 pgs e194477:22.3b9) v4 -3189> 2014-08-13 17:23:21.677708 7fcd0964c700 5 -- op tracker -- , seq: 322, time: 2014-08-13 17:23:21.677707, event: done, request: pg_info(1 pgs e194477:22.3b9) v4 -3188> 2014-08-13 17:23:21.677726 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 8 ==== pg_notify(24.b6(29) epoch 194477) v5 ==== 2379+0+0 (1511250803 0 0) 0x56a8c40 con 0x5305ee0 -3187> 2014-08-13 17:23:21.677745 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.648676, event: header_read, request: pg_notify(24.b6(29) epoch 194477) v5 -3186> 2014-08-13 17:23:21.677755 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.648678, event: throttled, request: pg_notify(24.b6(29) epoch 194477) v5 -3185> 2014-08-13 17:23:21.677764 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.648930, event: all_read, request: pg_notify(24.b6(29) epoch 194477) v5 -3184> 2014-08-13 17:23:21.677773 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.677743, event: dispatched, request: pg_notify(24.b6(29) epoch 194477) v5 -3183> 2014-08-13 17:23:21.677782 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.677782, event: waiting_for_osdmap, request: pg_notify(24.b6(29) epoch 194477) v5 -3182> 2014-08-13 17:23:21.677795 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.677795, event: started, request: pg_notify(24.b6(29) epoch 194477) v5 -3181> 2014-08-13 17:23:21.677819 7fcd0964c700 5 -- op tracker -- , seq: 323, time: 2014-08-13 17:23:21.677818, event: done, request: pg_notify(24.b6(29) epoch 194477) v5 -3180> 2014-08-13 17:23:21.677842 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bf980 con 0x53056a0 -3179> 2014-08-13 17:23:21.677901 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3178> 2014-08-13 17:23:21.677912 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 5 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550f740 con 0x53053e0 -3177> 2014-08-13 17:23:21.677931 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3176> 2014-08-13 17:23:21.677942 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550ec00 con 0x5307a60 -3175> 2014-08-13 17:23:21.677959 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3174> 2014-08-13 17:23:21.677970 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 7 ==== pg_notify(22.113(23) epoch 194477) v5 ==== 2003+0+0 (4053069827 0 0) 0x588f700 con 0x5305c20 -3173> 2014-08-13 17:23:21.677990 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.661387, event: header_read, request: pg_notify(22.113(23) epoch 194477) v5 -3172> 2014-08-13 17:23:21.677998 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.661391, event: throttled, request: pg_notify(22.113(23) epoch 194477) v5 -3171> 2014-08-13 17:23:21.678007 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.661550, event: all_read, request: pg_notify(22.113(23) epoch 194477) v5 -3170> 2014-08-13 17:23:21.678016 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.677988, event: dispatched, request: pg_notify(22.113(23) epoch 194477) v5 -3169> 2014-08-13 17:23:21.678025 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.678025, event: waiting_for_osdmap, request: pg_notify(22.113(23) epoch 194477) v5 -3168> 2014-08-13 17:23:21.678040 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.678040, event: started, request: pg_notify(22.113(23) epoch 194477) v5 -3167> 2014-08-13 17:23:21.678061 7fcd0964c700 5 -- op tracker -- , seq: 324, time: 2014-08-13 17:23:21.678061, event: done, request: pg_notify(22.113(23) epoch 194477) v5 -3166> 2014-08-13 17:23:21.678079 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 5 ==== pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 ==== 3990+0+0 (1445093824 0 0) 0x54d5b00 con 0x552a260 -3165> 2014-08-13 17:23:21.678095 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.670681, event: header_read, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3164> 2014-08-13 17:23:21.678103 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.670684, event: throttled, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3163> 2014-08-13 17:23:21.678109 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.670825, event: all_read, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3162> 2014-08-13 17:23:21.678115 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.678093, event: dispatched, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3161> 2014-08-13 17:23:21.678122 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.678121, event: waiting_for_osdmap, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3160> 2014-08-13 17:23:21.678129 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.678129, event: started, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3159> 2014-08-13 17:23:21.678161 7fcd0964c700 5 -- op tracker -- , seq: 325, time: 2014-08-13 17:23:21.678161, event: done, request: pg_notify(22.3af(26),22.f3(20) epoch 194477) v5 -3158> 2014-08-13 17:23:21.678208 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 6 ==== pg_notify(22.707(28) epoch 194477) v5 ==== 2290+0+0 (3967541694 0 0) 0x588f000 con 0x5305960 -3157> 2014-08-13 17:23:21.678230 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.660825, event: header_read, request: pg_notify(22.707(28) epoch 194477) v5 -3156> 2014-08-13 17:23:21.678241 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.660826, event: throttled, request: pg_notify(22.707(28) epoch 194477) v5 -3155> 2014-08-13 17:23:21.678250 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.661358, event: all_read, request: pg_notify(22.707(28) epoch 194477) v5 -3154> 2014-08-13 17:23:21.678260 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.678227, event: dispatched, request: pg_notify(22.707(28) epoch 194477) v5 -3153> 2014-08-13 17:23:21.678269 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.678269, event: waiting_for_osdmap, request: pg_notify(22.707(28) epoch 194477) v5 -3152> 2014-08-13 17:23:21.678280 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.678280, event: started, request: pg_notify(22.707(28) epoch 194477) v5 -3151> 2014-08-13 17:23:21.678304 7fcd0964c700 5 -- op tracker -- , seq: 326, time: 2014-08-13 17:23:21.678303, event: done, request: pg_notify(22.707(28) epoch 194477) v5 -3150> 2014-08-13 17:23:21.678331 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 7 ==== pg_notify(22.1d2(23) epoch 194477) v5 ==== 2003+0+0 (1418913395 0 0) 0x54d4b40 con 0x552fa60 -3149> 2014-08-13 17:23:21.678352 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.658785, event: header_read, request: pg_notify(22.1d2(23) epoch 194477) v5 -3148> 2014-08-13 17:23:21.678362 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.658786, event: throttled, request: pg_notify(22.1d2(23) epoch 194477) v5 -3147> 2014-08-13 17:23:21.678371 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.658971, event: all_read, request: pg_notify(22.1d2(23) epoch 194477) v5 -3146> 2014-08-13 17:23:21.678384 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.678349, event: dispatched, request: pg_notify(22.1d2(23) epoch 194477) v5 -3145> 2014-08-13 17:23:21.678394 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.678394, event: waiting_for_osdmap, request: pg_notify(22.1d2(23) epoch 194477) v5 -3144> 2014-08-13 17:23:21.678405 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.678404, event: started, request: pg_notify(22.1d2(23) epoch 194477) v5 -3143> 2014-08-13 17:23:21.678430 7fcd0964c700 5 -- op tracker -- , seq: 327, time: 2014-08-13 17:23:21.678429, event: done, request: pg_notify(22.1d2(23) epoch 194477) v5 -3142> 2014-08-13 17:23:21.678314 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194474/194474 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.722784 8 0.000410 -3141> 2014-08-13 17:23:21.678518 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5679200 con 0x552f7a0 -3140> 2014-08-13 17:23:21.678540 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3139> 2014-08-13 17:23:21.678540 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194474/194474 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -3138> 2014-08-13 17:23:21.678558 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.63 192.168.1.111:6830/12680 3 ==== pg_notify(22.775(26) epoch 194477) v5 ==== 2172+0+0 (2539764484 0 0) 0x588eac0 con 0x5521e40 -3137> 2014-08-13 17:23:21.678577 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.660770, event: header_read, request: pg_notify(22.775(26) epoch 194477) v5 -3136> 2014-08-13 17:23:21.678570 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194448 n=48 ec=192688 les/c 194474/194474 194476/194476/192975) [285,336,4] r=2 lpr=194476 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -3135> 2014-08-13 17:23:21.678594 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.660772, event: throttled, request: pg_notify(22.775(26) epoch 194477) v5 -3134> 2014-08-13 17:23:21.678604 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194474/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 0.759556 8 0.000609 -3133> 2014-08-13 17:23:21.678624 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.660893, event: all_read, request: pg_notify(22.775(26) epoch 194477) v5 -3132> 2014-08-13 17:23:21.678636 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.678575, event: dispatched, request: pg_notify(22.775(26) epoch 194477) v5 -3131> 2014-08-13 17:23:21.678643 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.678643, event: waiting_for_osdmap, request: pg_notify(22.775(26) epoch 194477) v5 -3130> 2014-08-13 17:23:21.678633 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194474/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive -3129> 2014-08-13 17:23:21.678659 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.678659, event: started, request: pg_notify(22.775(26) epoch 194477) v5 -3128> 2014-08-13 17:23:21.678624 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.19 192.168.1.115:6818/61186 -- pg_info(1 pgs e194477:24.3bf) v4 -- ?+0 0x54d6040 -3127> 2014-08-13 17:23:21.678681 7fcd0964c700 5 -- op tracker -- , seq: 328, time: 2014-08-13 17:23:21.678681, event: done, request: pg_notify(22.775(26) epoch 194477) v5 -3126> 2014-08-13 17:23:21.678671 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194472 n=49 ec=192688 les/c 194474/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194476 pi=193125-194475/29 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering -3125> 2014-08-13 17:23:21.678705 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 3 ==== pg_notify(22.3e4(27) epoch 194477) v5 ==== 2231+0+0 (176340611 0 0) 0x588fe00 con 0x55f2100 -3124> 2014-08-13 17:23:21.678713 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.358 192.168.1.114:6824/52078 -- pg_info(1 pgs e194477:22.2d1) v4 -- ?+0 0x54d5b00 -3123> 2014-08-13 17:23:21.678722 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.664130, event: header_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -3122> 2014-08-13 17:23:21.678729 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.664131, event: throttled, request: pg_notify(22.3e4(27) epoch 194477) v5 -3121> 2014-08-13 17:23:21.678735 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.664260, event: all_read, request: pg_notify(22.3e4(27) epoch 194477) v5 -3120> 2014-08-13 17:23:21.678741 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.678720, event: dispatched, request: pg_notify(22.3e4(27) epoch 194477) v5 -3119> 2014-08-13 17:23:21.678747 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.678746, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194477) v5 -3118> 2014-08-13 17:23:21.678754 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.678754, event: started, request: pg_notify(22.3e4(27) epoch 194477) v5 -3117> 2014-08-13 17:23:21.678773 7fcd0964c700 5 -- op tracker -- , seq: 329, time: 2014-08-13 17:23:21.678773, event: done, request: pg_notify(22.3e4(27) epoch 194477) v5 -3116> 2014-08-13 17:23:21.678800 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 3 ==== pg_notify(24.34(37) epoch 194477) v5 ==== 2623+0+0 (2256259889 0 0) 0x5184ec0 con 0x55244c0 -3115> 2014-08-13 17:23:21.678816 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.665574, event: header_read, request: pg_notify(24.34(37) epoch 194477) v5 -3114> 2014-08-13 17:23:21.678823 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.665576, event: throttled, request: pg_notify(24.34(37) epoch 194477) v5 -3113> 2014-08-13 17:23:21.678828 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.665753, event: all_read, request: pg_notify(24.34(37) epoch 194477) v5 -3112> 2014-08-13 17:23:21.678833 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.678814, event: dispatched, request: pg_notify(24.34(37) epoch 194477) v5 -3111> 2014-08-13 17:23:21.678840 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.678839, event: waiting_for_osdmap, request: pg_notify(24.34(37) epoch 194477) v5 -3110> 2014-08-13 17:23:21.678847 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.678847, event: started, request: pg_notify(24.34(37) epoch 194477) v5 -3109> 2014-08-13 17:23:21.678867 7fcd0964c700 5 -- op tracker -- , seq: 330, time: 2014-08-13 17:23:21.678866, event: done, request: pg_notify(24.34(37) epoch 194477) v5 -3108> 2014-08-13 17:23:21.678895 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 5 ==== pg_notify(22.4df(23) epoch 194477) v5 ==== 2003+0+0 (3293394395 0 0) 0x588bb80 con 0x56b2260 -3107> 2014-08-13 17:23:21.678911 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.669710, event: header_read, request: pg_notify(22.4df(23) epoch 194477) v5 -3106> 2014-08-13 17:23:21.678918 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.669712, event: throttled, request: pg_notify(22.4df(23) epoch 194477) v5 -3105> 2014-08-13 17:23:21.678932 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.669827, event: all_read, request: pg_notify(22.4df(23) epoch 194477) v5 -3104> 2014-08-13 17:23:21.678937 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.678909, event: dispatched, request: pg_notify(22.4df(23) epoch 194477) v5 -3103> 2014-08-13 17:23:21.678943 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.678943, event: waiting_for_osdmap, request: pg_notify(22.4df(23) epoch 194477) v5 -3102> 2014-08-13 17:23:21.678951 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.678950, event: started, request: pg_notify(22.4df(23) epoch 194477) v5 -3101> 2014-08-13 17:23:21.678970 7fcd0964c700 5 -- op tracker -- , seq: 331, time: 2014-08-13 17:23:21.678970, event: done, request: pg_notify(22.4df(23) epoch 194477) v5 -3100> 2014-08-13 17:23:21.678996 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.11 192.168.1.103:6813/37929 2 ==== pg_notify(22.1f1(5) epoch 194477) v5 ==== 1001+0+0 (2117454945 0 0) 0x56a88c0 con 0x56ba940 -3099> 2014-08-13 17:23:21.679013 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.670551, event: header_read, request: pg_notify(22.1f1(5) epoch 194477) v5 -3098> 2014-08-13 17:23:21.679020 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.670552, event: throttled, request: pg_notify(22.1f1(5) epoch 194477) v5 -3097> 2014-08-13 17:23:21.679026 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.670636, event: all_read, request: pg_notify(22.1f1(5) epoch 194477) v5 -3096> 2014-08-13 17:23:21.679031 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.679011, event: dispatched, request: pg_notify(22.1f1(5) epoch 194477) v5 -3095> 2014-08-13 17:23:21.679037 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.679037, event: waiting_for_osdmap, request: pg_notify(22.1f1(5) epoch 194477) v5 -3094> 2014-08-13 17:23:21.679044 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.679044, event: started, request: pg_notify(22.1f1(5) epoch 194477) v5 -3093> 2014-08-13 17:23:21.679063 7fcd0964c700 5 -- op tracker -- , seq: 332, time: 2014-08-13 17:23:21.679063, event: done, request: pg_notify(22.1f1(5) epoch 194477) v5 -3092> 2014-08-13 17:23:21.679078 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.47 192.168.1.111:6828/12102 3 ==== pg_notify(22.1f1(27) epoch 194477) v5 ==== 2231+0+0 (1112992215 0 0) 0x54d40c0 con 0x57232e0 -3091> 2014-08-13 17:23:21.679097 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.659219, event: header_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3090> 2014-08-13 17:23:21.679104 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.659221, event: throttled, request: pg_notify(22.1f1(27) epoch 194477) v5 -3089> 2014-08-13 17:23:21.679113 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.659333, event: all_read, request: pg_notify(22.1f1(27) epoch 194477) v5 -3088> 2014-08-13 17:23:21.679122 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.679095, event: dispatched, request: pg_notify(22.1f1(27) epoch 194477) v5 -3087> 2014-08-13 17:23:21.679110 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.346 192.168.1.104:6820/2308 -- pg_info(1 pgs e194477:24.e2) v4 -- ?+0 0x588f000 -3086> 2014-08-13 17:23:21.679131 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.679131, event: waiting_for_osdmap, request: pg_notify(22.1f1(27) epoch 194477) v5 -3085> 2014-08-13 17:23:21.679142 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.679142, event: started, request: pg_notify(22.1f1(27) epoch 194477) v5 -3084> 2014-08-13 17:23:21.679162 7fcd0964c700 5 -- op tracker -- , seq: 333, time: 2014-08-13 17:23:21.679161, event: done, request: pg_notify(22.1f1(27) epoch 194477) v5 -3083> 2014-08-13 17:23:21.679185 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 10 ==== pg_notify(22.362(27) epoch 194477) v5 ==== 2231+0+0 (3551719807 0 0) 0x5889c00 con 0x4e335a0 -3082> 2014-08-13 17:23:21.679208 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.652061, event: header_read, request: pg_notify(22.362(27) epoch 194477) v5 -3081> 2014-08-13 17:23:21.679216 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.652063, event: throttled, request: pg_notify(22.362(27) epoch 194477) v5 -3080> 2014-08-13 17:23:21.679222 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.653206, event: all_read, request: pg_notify(22.362(27) epoch 194477) v5 -3079> 2014-08-13 17:23:21.679227 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.679198, event: dispatched, request: pg_notify(22.362(27) epoch 194477) v5 -3078> 2014-08-13 17:23:21.679233 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.679233, event: waiting_for_osdmap, request: pg_notify(22.362(27) epoch 194477) v5 -3077> 2014-08-13 17:23:21.679241 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.679240, event: started, request: pg_notify(22.362(27) epoch 194477) v5 -3076> 2014-08-13 17:23:21.679265 7fcd0964c700 5 -- op tracker -- , seq: 334, time: 2014-08-13 17:23:21.679264, event: done, request: pg_notify(22.362(27) epoch 194477) v5 -3075> 2014-08-13 17:23:21.679292 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 5 ==== pg_info(1 pgs e194477:22.28f) v4 ==== 714+0+0 (1995194604 0 0) 0x588a300 con 0x4e34780 -3074> 2014-08-13 17:23:21.679312 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.677939, event: header_read, request: pg_info(1 pgs e194477:22.28f) v4 -3073> 2014-08-13 17:23:21.679322 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.677941, event: throttled, request: pg_info(1 pgs e194477:22.28f) v4 -3072> 2014-08-13 17:23:21.679331 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.678011, event: all_read, request: pg_info(1 pgs e194477:22.28f) v4 -3071> 2014-08-13 17:23:21.679339 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.679310, event: dispatched, request: pg_info(1 pgs e194477:22.28f) v4 -3070> 2014-08-13 17:23:21.679351 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.679350, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.28f) v4 -3069> 2014-08-13 17:23:21.679359 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.679359, event: started, request: pg_info(1 pgs e194477:22.28f) v4 -3068> 2014-08-13 17:23:21.679397 7fcd0964c700 5 -- op tracker -- , seq: 335, time: 2014-08-13 17:23:21.679397, event: done, request: pg_info(1 pgs e194477:22.28f) v4 -3067> 2014-08-13 17:23:21.679421 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 8 ==== pg_notify(22.362(5) epoch 194477) v5 ==== 1001+0+0 (822319021 0 0) 0x56a9dc0 con 0x4e35ac0 -3066> 2014-08-13 17:23:21.679437 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.660133, event: header_read, request: pg_notify(22.362(5) epoch 194477) v5 -3065> 2014-08-13 17:23:21.679443 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.660135, event: throttled, request: pg_notify(22.362(5) epoch 194477) v5 -3064> 2014-08-13 17:23:21.679449 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.660232, event: all_read, request: pg_notify(22.362(5) epoch 194477) v5 -3063> 2014-08-13 17:23:21.679455 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.679434, event: dispatched, request: pg_notify(22.362(5) epoch 194477) v5 -3062> 2014-08-13 17:23:21.679461 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.679460, event: waiting_for_osdmap, request: pg_notify(22.362(5) epoch 194477) v5 -3061> 2014-08-13 17:23:21.679471 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.679470, event: started, request: pg_notify(22.362(5) epoch 194477) v5 -3060> 2014-08-13 17:23:21.679489 7fcd0964c700 5 -- op tracker -- , seq: 336, time: 2014-08-13 17:23:21.679489, event: done, request: pg_notify(22.362(5) epoch 194477) v5 -3059> 2014-08-13 17:23:21.679504 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 8 ==== pg_notify(24.30b(6) epoch 194477) v5 ==== 1094+0+0 (3266582469 0 0) 0x588a140 con 0x4e35800 -3058> 2014-08-13 17:23:21.679519 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.652638, event: header_read, request: pg_notify(24.30b(6) epoch 194477) v5 -3057> 2014-08-13 17:23:21.679525 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.652639, event: throttled, request: pg_notify(24.30b(6) epoch 194477) v5 -3056> 2014-08-13 17:23:21.679531 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.652711, event: all_read, request: pg_notify(24.30b(6) epoch 194477) v5 -3055> 2014-08-13 17:23:21.679536 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.679517, event: dispatched, request: pg_notify(24.30b(6) epoch 194477) v5 -3054> 2014-08-13 17:23:21.679542 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.679542, event: waiting_for_osdmap, request: pg_notify(24.30b(6) epoch 194477) v5 -3053> 2014-08-13 17:23:21.679549 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.679549, event: started, request: pg_notify(24.30b(6) epoch 194477) v5 -3052> 2014-08-13 17:23:21.679568 7fcd0964c700 5 -- op tracker -- , seq: 337, time: 2014-08-13 17:23:21.679568, event: done, request: pg_notify(24.30b(6) epoch 194477) v5 -3051> 2014-08-13 17:23:21.679587 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 6 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550b180 con 0x4e35540 -3050> 2014-08-13 17:23:21.679605 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -3049> 2014-08-13 17:23:21.679615 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 11 ==== pg_info(1 pgs e194477:22.113) v4 ==== 714+0+0 (2374253081 0 0) 0x56ad940 con 0x4ec09a0 -3048> 2014-08-13 17:23:21.679633 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.667847, event: header_read, request: pg_info(1 pgs e194477:22.113) v4 -3047> 2014-08-13 17:23:21.679644 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.667849, event: throttled, request: pg_info(1 pgs e194477:22.113) v4 -3046> 2014-08-13 17:23:21.679652 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.667932, event: all_read, request: pg_info(1 pgs e194477:22.113) v4 -3045> 2014-08-13 17:23:21.679662 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.679631, event: dispatched, request: pg_info(1 pgs e194477:22.113) v4 -3044> 2014-08-13 17:23:21.679671 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.679671, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.113) v4 -3043> 2014-08-13 17:23:21.679683 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.679682, event: started, request: pg_info(1 pgs e194477:22.113) v4 -3042> 2014-08-13 17:23:21.679706 7fcd0964c700 5 -- op tracker -- , seq: 338, time: 2014-08-13 17:23:21.679706, event: done, request: pg_info(1 pgs e194477:22.113) v4 -3041> 2014-08-13 17:23:21.679723 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 3 ==== pg_notify(22.619(27) epoch 194477) v5 ==== 2231+0+0 (857637660 0 0) 0x54d1f80 con 0x4e370c0 -3040> 2014-08-13 17:23:21.679744 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.666243, event: header_read, request: pg_notify(22.619(27) epoch 194477) v5 -3039> 2014-08-13 17:23:21.679751 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.666245, event: throttled, request: pg_notify(22.619(27) epoch 194477) v5 -3038> 2014-08-13 17:23:21.679757 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.666356, event: all_read, request: pg_notify(22.619(27) epoch 194477) v5 -3037> 2014-08-13 17:23:21.679766 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.679742, event: dispatched, request: pg_notify(22.619(27) epoch 194477) v5 -3036> 2014-08-13 17:23:21.679778 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.679777, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194477) v5 -3035> 2014-08-13 17:23:21.679789 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.679789, event: started, request: pg_notify(22.619(27) epoch 194477) v5 -3034> 2014-08-13 17:23:21.679816 7fcd0964c700 5 -- op tracker -- , seq: 339, time: 2014-08-13 17:23:21.679815, event: done, request: pg_notify(22.619(27) epoch 194477) v5 -3033> 2014-08-13 17:23:21.679949 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 7 ==== pg_query(24.1e4 epoch 194477) v3 ==== 144+0+0 (3386637667 0 0) 0x2f94b00 con 0x4ec56a0 -3032> 2014-08-13 17:23:21.680010 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.673187, event: header_read, request: pg_query(24.1e4 epoch 194477) v3 -3031> 2014-08-13 17:23:21.680023 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.673188, event: throttled, request: pg_query(24.1e4 epoch 194477) v3 -3030> 2014-08-13 17:23:21.680032 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.673314, event: all_read, request: pg_query(24.1e4 epoch 194477) v3 -3029> 2014-08-13 17:23:21.680041 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.680006, event: dispatched, request: pg_query(24.1e4 epoch 194477) v3 -3028> 2014-08-13 17:23:21.680089 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.680088, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194477) v3 -3027> 2014-08-13 17:23:21.680099 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.680099, event: started, request: pg_query(24.1e4 epoch 194477) v3 -3026> 2014-08-13 17:23:21.680123 7fcd0964c700 5 -- op tracker -- , seq: 340, time: 2014-08-13 17:23:21.680123, event: done, request: pg_query(24.1e4 epoch 194477) v3 -3025> 2014-08-13 17:23:21.680142 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 12 ==== pg_notify(22.48e(5) epoch 194477) v5 ==== 1001+0+0 (1532929754 0 0) 0x56a9a40 con 0x4ec3440 -3024> 2014-08-13 17:23:21.680164 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.673034, event: header_read, request: pg_notify(22.48e(5) epoch 194477) v5 -3023> 2014-08-13 17:23:21.680173 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194474/194474 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 0.721291 8 0.000376 -3022> 2014-08-13 17:23:21.680171 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.673037, event: throttled, request: pg_notify(22.48e(5) epoch 194477) v5 -3021> 2014-08-13 17:23:21.680226 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.673128, event: all_read, request: pg_notify(22.48e(5) epoch 194477) v5 -3020> 2014-08-13 17:23:21.680241 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.680161, event: dispatched, request: pg_notify(22.48e(5) epoch 194477) v5 -3019> 2014-08-13 17:23:21.680252 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.680252, event: waiting_for_osdmap, request: pg_notify(22.48e(5) epoch 194477) v5 -3018> 2014-08-13 17:23:21.680265 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.680265, event: started, request: pg_notify(22.48e(5) epoch 194477) v5 -3017> 2014-08-13 17:23:21.680255 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194474/194474 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive -3016> 2014-08-13 17:23:21.680288 7fcd0964c700 5 -- op tracker -- , seq: 341, time: 2014-08-13 17:23:21.680288, event: done, request: pg_notify(22.48e(5) epoch 194477) v5 -3015> 2014-08-13 17:23:21.680283 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194472 n=43 ec=192688 les/c 194474/194474 194476/194476/193262) [343,45,4] r=2 lpr=194476 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering -3014> 2014-08-13 17:23:21.680403 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 11 ==== pg_info(1 pgs e194477:24.350(1)) v4 ==== 799+0+0 (1638735827 0 0) 0x56a96c0 con 0x4ec4780 -3013> 2014-08-13 17:23:21.680404 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.765379 7 0.000261 -3012> 2014-08-13 17:23:21.680479 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.643746, event: header_read, request: pg_info(1 pgs e194477:24.350(1)) v4 -3011> 2014-08-13 17:23:21.680496 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.643747, event: throttled, request: pg_info(1 pgs e194477:24.350(1)) v4 -3010> 2014-08-13 17:23:21.680479 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -3009> 2014-08-13 17:23:21.680506 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.643814, event: all_read, request: pg_info(1 pgs e194477:24.350(1)) v4 -3008> 2014-08-13 17:23:21.680513 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194471 n=29 ec=183096 les/c 194474/194465 194476/194476/192981) [345,4,374] r=1 lpr=194476 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -3007> 2014-08-13 17:23:21.680676 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.017196 5 0.000200 -3006> 2014-08-13 17:23:21.680744 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -3005> 2014-08-13 17:23:21.680549 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.680474, event: dispatched, request: pg_info(1 pgs e194477:24.350(1)) v4 -3004> 2014-08-13 17:23:21.680802 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.680802, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.350(1)) v4 -3003> 2014-08-13 17:23:21.680832 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.680832, event: started, request: pg_info(1 pgs e194477:24.350(1)) v4 -3002> 2014-08-13 17:23:21.680871 7fcd0964c700 5 -- op tracker -- , seq: 342, time: 2014-08-13 17:23:21.680870, event: done, request: pg_info(1 pgs e194477:24.350(1)) v4 -3001> 2014-08-13 17:23:21.680852 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000109 0 0.000000 -3000> 2014-08-13 17:23:21.680983 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2999> 2014-08-13 17:23:21.681279 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194474/194474 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.722116 8 0.000374 -2998> 2014-08-13 17:23:21.681317 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194474/194474 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -2997> 2014-08-13 17:23:21.681631 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194472 n=51 ec=192688 les/c 194474/194474 194476/194476/193186) [45,319,4] r=2 lpr=194476 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -2996> 2014-08-13 17:23:21.680903 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 2 ==== pg_notify(22.39d(25) epoch 194477) v5 ==== 2117+0+0 (1249656499 0 0) 0x56abf00 con 0x4ec70c0 -2995> 2014-08-13 17:23:21.681809 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.668351, event: header_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2994> 2014-08-13 17:23:21.682087 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.668353, event: throttled, request: pg_notify(22.39d(25) epoch 194477) v5 -2993> 2014-08-13 17:23:21.682686 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.668475, event: all_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2992> 2014-08-13 17:23:21.682873 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.761634 7 0.000601 -2991> 2014-08-13 17:23:21.683053 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2990> 2014-08-13 17:23:21.683098 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192946) [334,289,4] r=2 lpr=194476 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2989> 2014-08-13 17:23:21.683105 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.681803, event: dispatched, request: pg_notify(22.39d(25) epoch 194477) v5 -2988> 2014-08-13 17:23:21.683189 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.683188, event: waiting_for_osdmap, request: pg_notify(22.39d(25) epoch 194477) v5 -2987> 2014-08-13 17:23:21.683219 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.683218, event: started, request: pg_notify(22.39d(25) epoch 194477) v5 -2986> 2014-08-13 17:23:21.683247 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.736303 7 0.000241 -2985> 2014-08-13 17:23:21.683280 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2984> 2014-08-13 17:23:21.683289 7fcd0964c700 5 -- op tracker -- , seq: 343, time: 2014-08-13 17:23:21.683289, event: done, request: pg_notify(22.39d(25) epoch 194477) v5 -2983> 2014-08-13 17:23:21.683309 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194471 n=33 ec=183096 les/c 194474/194474 194476/194476/192976) [367,44,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2982> 2014-08-13 17:23:21.683355 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.364 192.168.1.114:6826/52317 -- pg_info(1 pgs e194477:24.1de) v4 -- ?+0 0x588f700 -2981> 2014-08-13 17:23:21.683436 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 7 ==== pg_notify(22.3af(27) epoch 194477) v5 ==== 2231+0+0 (4199141443 0 0) 0x5b481c0 con 0x4eb0c60 -2980> 2014-08-13 17:23:21.683474 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.672900, event: header_read, request: pg_notify(22.3af(27) epoch 194477) v5 -2979> 2014-08-13 17:23:21.683497 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.672902, event: throttled, request: pg_notify(22.3af(27) epoch 194477) v5 -2978> 2014-08-13 17:23:21.683517 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.673018, event: all_read, request: pg_notify(22.3af(27) epoch 194477) v5 -2977> 2014-08-13 17:23:21.683537 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.683470, event: dispatched, request: pg_notify(22.3af(27) epoch 194477) v5 -2976> 2014-08-13 17:23:21.683557 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.683557, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194477) v5 -2975> 2014-08-13 17:23:21.683671 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194474/194474 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.807263 7 0.000347 -2974> 2014-08-13 17:23:21.683705 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194474/194474 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2973> 2014-08-13 17:23:21.683729 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194471 n=27 ec=183096 les/c 194474/194474 194476/194476/192936) [54,331,4] r=2 lpr=194476 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2972> 2014-08-13 17:23:21.683735 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(33) epoch 194477) v5 -- ?+0 0x5187e00 con 0x5305ee0 -2971> 2014-08-13 17:23:21.683772 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(26) epoch 194477) v5 -- ?+0 0x56ac7c0 con 0x4fd86e0 -2970> 2014-08-13 17:23:21.683777 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.683777, event: started, request: pg_notify(22.3af(27) epoch 194477) v5 -2969> 2014-08-13 17:23:21.683841 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194474/194474 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 0.762219 8 0.000332 -2968> 2014-08-13 17:23:21.683875 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194474/194474 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive -2967> 2014-08-13 17:23:21.683883 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194477:24.1e6) v4 -- ?+0 0x56a8c40 -2966> 2014-08-13 17:23:21.683895 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194472 n=60 ec=192688 les/c 194474/194474 194476/194476/193250) [304,323,4] r=2 lpr=194476 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering -2965> 2014-08-13 17:23:21.683852 7fcd0964c700 5 -- op tracker -- , seq: 344, time: 2014-08-13 17:23:21.683852, event: done, request: pg_notify(22.3af(27) epoch 194477) v5 -2964> 2014-08-13 17:23:21.683952 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 7 ==== pg_info(1 pgs e194477:22.2b9) v4 ==== 714+0+0 (181265300 0 0) 0x588fc40 con 0x4eb23c0 -2963> 2014-08-13 17:23:21.683971 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.677731, event: header_read, request: pg_info(1 pgs e194477:22.2b9) v4 -2962> 2014-08-13 17:23:21.683979 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.677733, event: throttled, request: pg_info(1 pgs e194477:22.2b9) v4 -2961> 2014-08-13 17:23:21.683986 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.677865, event: all_read, request: pg_info(1 pgs e194477:22.2b9) v4 -2960> 2014-08-13 17:23:21.683992 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.683968, event: dispatched, request: pg_info(1 pgs e194477:22.2b9) v4 -2959> 2014-08-13 17:23:21.683998 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.683998, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2b9) v4 -2958> 2014-08-13 17:23:21.684007 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.684007, event: started, request: pg_info(1 pgs e194477:22.2b9) v4 -2957> 2014-08-13 17:23:21.684031 7fcd0964c700 5 -- op tracker -- , seq: 345, time: 2014-08-13 17:23:21.684031, event: done, request: pg_info(1 pgs e194477:22.2b9) v4 -2956> 2014-08-13 17:23:21.684047 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 9 ==== pg_notify(22.99(23) epoch 194477) v5 ==== 2003+0+0 (3218222013 0 0) 0x5b48700 con 0x4eb18c0 -2955> 2014-08-13 17:23:21.684070 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.666930, event: header_read, request: pg_notify(22.99(23) epoch 194477) v5 -2954> 2014-08-13 17:23:21.684077 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.666931, event: throttled, request: pg_notify(22.99(23) epoch 194477) v5 -2953> 2014-08-13 17:23:21.684083 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.667037, event: all_read, request: pg_notify(22.99(23) epoch 194477) v5 -2952> 2014-08-13 17:23:21.684088 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.684068, event: dispatched, request: pg_notify(22.99(23) epoch 194477) v5 -2951> 2014-08-13 17:23:21.684094 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.684094, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194477) v5 -2950> 2014-08-13 17:23:21.684102 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.684102, event: started, request: pg_notify(22.99(23) epoch 194477) v5 -2949> 2014-08-13 17:23:21.684107 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194477:22.116) v4 -- ?+0 0x588fe00 -2948> 2014-08-13 17:23:21.684123 7fcd0964c700 5 -- op tracker -- , seq: 346, time: 2014-08-13 17:23:21.684123, event: done, request: pg_notify(22.99(23) epoch 194477) v5 -2947> 2014-08-13 17:23:21.684181 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 3 ==== pg_notify(22.775(27) epoch 194477) v5 ==== 2231+0+0 (1367275308 0 0) 0x4e501c0 con 0x4eb48e0 -2946> 2014-08-13 17:23:21.684202 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.668050, event: header_read, request: pg_notify(22.775(27) epoch 194477) v5 -2945> 2014-08-13 17:23:21.684210 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.668051, event: throttled, request: pg_notify(22.775(27) epoch 194477) v5 -2944> 2014-08-13 17:23:21.684215 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.668147, event: all_read, request: pg_notify(22.775(27) epoch 194477) v5 -2943> 2014-08-13 17:23:21.684221 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.684199, event: dispatched, request: pg_notify(22.775(27) epoch 194477) v5 -2942> 2014-08-13 17:23:21.684227 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.684227, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194477) v5 -2941> 2014-08-13 17:23:21.684235 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.684235, event: started, request: pg_notify(22.775(27) epoch 194477) v5 -2940> 2014-08-13 17:23:21.684257 7fcd0964c700 5 -- op tracker -- , seq: 347, time: 2014-08-13 17:23:21.684256, event: done, request: pg_notify(22.775(27) epoch 194477) v5 -2939> 2014-08-13 17:23:21.684265 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194474/194474 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.725545 7 0.000262 -2938> 2014-08-13 17:23:21.684282 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 5 ==== pg_info(1 pgs e194477:22.630) v4 ==== 714+0+0 (2530651777 0 0) 0x56a9500 con 0x4eb40a0 -2937> 2014-08-13 17:23:21.684298 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.678071, event: header_read, request: pg_info(1 pgs e194477:22.630) v4 -2936> 2014-08-13 17:23:21.684305 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.678073, event: throttled, request: pg_info(1 pgs e194477:22.630) v4 -2935> 2014-08-13 17:23:21.684311 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.678151, event: all_read, request: pg_info(1 pgs e194477:22.630) v4 -2934> 2014-08-13 17:23:21.684302 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194474/194474 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2933> 2014-08-13 17:23:21.684317 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.684295, event: dispatched, request: pg_info(1 pgs e194477:22.630) v4 -2932> 2014-08-13 17:23:21.684324 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.684323, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.630) v4 -2931> 2014-08-13 17:23:21.684332 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.684332, event: started, request: pg_info(1 pgs e194477:22.630) v4 -2930> 2014-08-13 17:23:21.684323 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194471 n=30 ec=183096 les/c 194474/194474 194476/194476/192949) [328,4,20] r=1 lpr=194476 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2929> 2014-08-13 17:23:21.684352 7fcd0964c700 5 -- op tracker -- , seq: 348, time: 2014-08-13 17:23:21.684351, event: done, request: pg_info(1 pgs e194477:22.630) v4 -2928> 2014-08-13 17:23:21.684366 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 8 ==== pg_info(1 pgs e194477:22.6bd) v4 ==== 714+0+0 (1048725863 0 0) 0x56aa840 con 0x4eb7d20 -2927> 2014-08-13 17:23:21.684380 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.674747, event: header_read, request: pg_info(1 pgs e194477:22.6bd) v4 -2926> 2014-08-13 17:23:21.684387 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.674749, event: throttled, request: pg_info(1 pgs e194477:22.6bd) v4 -2925> 2014-08-13 17:23:21.684392 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.674818, event: all_read, request: pg_info(1 pgs e194477:22.6bd) v4 -2924> 2014-08-13 17:23:21.684397 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.684378, event: dispatched, request: pg_info(1 pgs e194477:22.6bd) v4 -2923> 2014-08-13 17:23:21.684404 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.684404, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.6bd) v4 -2922> 2014-08-13 17:23:21.684411 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.684411, event: started, request: pg_info(1 pgs e194477:22.6bd) v4 -2921> 2014-08-13 17:23:21.684400 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194477:24.206) v4 -- ?+0 0x588eac0 -2920> 2014-08-13 17:23:21.684429 7fcd0964c700 5 -- op tracker -- , seq: 349, time: 2014-08-13 17:23:21.684429, event: done, request: pg_info(1 pgs e194477:22.6bd) v4 -2919> 2014-08-13 17:23:21.684450 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 5 ==== pg_info(1 pgs e194477:22.2d1) v4 ==== 714+0+0 (3432422329 0 0) 0x54d5b00 con 0x4eb6720 -2918> 2014-08-13 17:23:21.684464 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.679283, event: header_read, request: pg_info(1 pgs e194477:22.2d1) v4 -2917> 2014-08-13 17:23:21.684471 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.679286, event: throttled, request: pg_info(1 pgs e194477:22.2d1) v4 -2916> 2014-08-13 17:23:21.684476 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.679365, event: all_read, request: pg_info(1 pgs e194477:22.2d1) v4 -2915> 2014-08-13 17:23:21.684482 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.684462, event: dispatched, request: pg_info(1 pgs e194477:22.2d1) v4 -2914> 2014-08-13 17:23:21.684488 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.684487, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2d1) v4 -2913> 2014-08-13 17:23:21.684495 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.684495, event: started, request: pg_info(1 pgs e194477:22.2d1) v4 -2912> 2014-08-13 17:23:21.684513 7fcd0964c700 5 -- op tracker -- , seq: 350, time: 2014-08-13 17:23:21.684513, event: done, request: pg_info(1 pgs e194477:22.2d1) v4 -2911> 2014-08-13 17:23:21.684527 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 7 ==== pg_query(24.20e epoch 194477) v3 ==== 144+0+0 (224279555 0 0) 0x2f96720 con 0x4f80b00 -2910> 2014-08-13 17:23:21.684540 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.672968, event: header_read, request: pg_query(24.20e epoch 194477) v3 -2909> 2014-08-13 17:23:21.684547 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.672970, event: throttled, request: pg_query(24.20e epoch 194477) v3 -2908> 2014-08-13 17:23:21.684552 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.673085, event: all_read, request: pg_query(24.20e epoch 194477) v3 -2907> 2014-08-13 17:23:21.684558 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.684539, event: dispatched, request: pg_query(24.20e epoch 194477) v3 -2906> 2014-08-13 17:23:21.684564 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.684563, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194477) v3 -2905> 2014-08-13 17:23:21.684572 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.684571, event: started, request: pg_query(24.20e epoch 194477) v3 -2904> 2014-08-13 17:23:21.684590 7fcd0964c700 5 -- op tracker -- , seq: 351, time: 2014-08-13 17:23:21.684590, event: done, request: pg_query(24.20e epoch 194477) v3 -2903> 2014-08-13 17:23:21.684573 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.032226 5 0.000202 -2902> 2014-08-13 17:23:21.684604 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.315 192.168.1.113:6800/14161 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52ba640 con 0x4eb6460 -2901> 2014-08-13 17:23:21.684606 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2900> 2014-08-13 17:23:21.684624 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -2899> 2014-08-13 17:23:21.684634 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 5 ==== pg_notify(22.3d9(42) epoch 194477) v5 ==== 2996+0+0 (2682887844 0 0) 0x5187a80 con 0x4f832e0 -2898> 2014-08-13 17:23:21.684649 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.672142, event: header_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -2897> 2014-08-13 17:23:21.684656 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.672144, event: throttled, request: pg_notify(22.3d9(42) epoch 194477) v5 -2896> 2014-08-13 17:23:21.684662 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.672366, event: all_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -2895> 2014-08-13 17:23:21.684667 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.684646, event: dispatched, request: pg_notify(22.3d9(42) epoch 194477) v5 -2894> 2014-08-13 17:23:21.684674 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.684674, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194477) v5 -2893> 2014-08-13 17:23:21.684681 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.684681, event: started, request: pg_notify(22.3d9(42) epoch 194477) v5 -2892> 2014-08-13 17:23:21.684671 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000065 0 0.000000 -2891> 2014-08-13 17:23:21.684702 7fcd0964c700 5 -- op tracker -- , seq: 352, time: 2014-08-13 17:23:21.684701, event: done, request: pg_notify(22.3d9(42) epoch 194477) v5 -2890> 2014-08-13 17:23:21.684693 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2889> 2014-08-13 17:23:21.684728 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x4f7fbc0 con 0x4f86040 -2888> 2014-08-13 17:23:21.684747 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2887> 2014-08-13 17:23:21.684756 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.364 192.168.1.114:6826/52317 6 ==== pg_info(1 pgs e194477:24.1de) v4 ==== 748+0+0 (4166564109 0 0) 0x588f700 con 0x4f83860 -2886> 2014-08-13 17:23:21.684769 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.683977, event: header_read, request: pg_info(1 pgs e194477:24.1de) v4 -2885> 2014-08-13 17:23:21.684776 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.683979, event: throttled, request: pg_info(1 pgs e194477:24.1de) v4 -2884> 2014-08-13 17:23:21.684782 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.684085, event: all_read, request: pg_info(1 pgs e194477:24.1de) v4 -2883> 2014-08-13 17:23:21.684788 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.684767, event: dispatched, request: pg_info(1 pgs e194477:24.1de) v4 -2882> 2014-08-13 17:23:21.684740 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194474/194474 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.728646 7 0.000285 -2881> 2014-08-13 17:23:21.684794 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.684794, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.1de) v4 -2880> 2014-08-13 17:23:21.684802 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.684802, event: started, request: pg_info(1 pgs e194477:24.1de) v4 -2879> 2014-08-13 17:23:21.684797 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194474/194474 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2878> 2014-08-13 17:23:21.684800 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.334 192.168.1.104:6808/62751 -- pg_info(1 pgs e194477:22.69f) v4 -- ?+0 0x5888a80 -2877> 2014-08-13 17:23:21.684824 7fcd0964c700 5 -- op tracker -- , seq: 353, time: 2014-08-13 17:23:21.684824, event: done, request: pg_info(1 pgs e194477:24.1de) v4 -2876> 2014-08-13 17:23:21.684816 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194471 n=41 ec=183096 les/c 194474/194474 194476/194476/192978) [323,359,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2875> 2014-08-13 17:23:21.684841 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 7 ==== pg_notify(24.181(7) epoch 194477) v5 ==== 1149+0+0 (159208590 0 0) 0x588a680 con 0x4fd9a20 -2874> 2014-08-13 17:23:21.684856 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.654229, event: header_read, request: pg_notify(24.181(7) epoch 194477) v5 -2873> 2014-08-13 17:23:21.684864 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.654231, event: throttled, request: pg_notify(24.181(7) epoch 194477) v5 -2872> 2014-08-13 17:23:21.684870 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.654337, event: all_read, request: pg_notify(24.181(7) epoch 194477) v5 -2871> 2014-08-13 17:23:21.684876 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.684854, event: dispatched, request: pg_notify(24.181(7) epoch 194477) v5 -2870> 2014-08-13 17:23:21.684882 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.684881, event: waiting_for_osdmap, request: pg_notify(24.181(7) epoch 194477) v5 -2869> 2014-08-13 17:23:21.684889 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.684889, event: started, request: pg_notify(24.181(7) epoch 194477) v5 -2868> 2014-08-13 17:23:21.684911 7fcd0964c700 5 -- op tracker -- , seq: 354, time: 2014-08-13 17:23:21.684911, event: done, request: pg_notify(24.181(7) epoch 194477) v5 -2867> 2014-08-13 17:23:21.684928 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 7 ==== pg_notify(24.136(26) epoch 194477) v5 ==== 2202+0+0 (2731362500 0 0) 0x56ae580 con 0x4fd9760 -2866> 2014-08-13 17:23:21.684942 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.673860, event: header_read, request: pg_notify(24.136(26) epoch 194477) v5 -2865> 2014-08-13 17:23:21.684949 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.673863, event: throttled, request: pg_notify(24.136(26) epoch 194477) v5 -2864> 2014-08-13 17:23:21.684955 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.673972, event: all_read, request: pg_notify(24.136(26) epoch 194477) v5 -2863> 2014-08-13 17:23:21.684960 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.684941, event: dispatched, request: pg_notify(24.136(26) epoch 194477) v5 -2862> 2014-08-13 17:23:21.684946 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.726533 7 0.000273 -2861> 2014-08-13 17:23:21.684967 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.684967, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194477) v5 -2860> 2014-08-13 17:23:21.684975 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.684975, event: started, request: pg_notify(24.136(26) epoch 194477) v5 -2859> 2014-08-13 17:23:21.684974 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2858> 2014-08-13 17:23:21.684995 7fcd0964c700 5 -- op tracker -- , seq: 355, time: 2014-08-13 17:23:21.684995, event: done, request: pg_notify(24.136(26) epoch 194477) v5 -2857> 2014-08-13 17:23:21.684994 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194471 n=37 ec=183096 les/c 194474/194474 194476/194476/192972) [292,345,4] r=2 lpr=194476 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2856> 2014-08-13 17:23:21.685029 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 11 ==== pg_info(1 pgs e194477:22.3d5) v4 ==== 714+0+0 (3944782751 0 0) 0x54d1880 con 0x4f86e00 -2855> 2014-08-13 17:23:21.685045 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.672766, event: header_read, request: pg_info(1 pgs e194477:22.3d5) v4 -2854> 2014-08-13 17:23:21.685053 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.672768, event: throttled, request: pg_info(1 pgs e194477:22.3d5) v4 -2853> 2014-08-13 17:23:21.685059 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.672842, event: all_read, request: pg_info(1 pgs e194477:22.3d5) v4 -2852> 2014-08-13 17:23:21.685068 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.685043, event: dispatched, request: pg_info(1 pgs e194477:22.3d5) v4 -2851> 2014-08-13 17:23:21.685080 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.685080, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3d5) v4 -2850> 2014-08-13 17:23:21.685091 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.685091, event: started, request: pg_info(1 pgs e194477:22.3d5) v4 -2849> 2014-08-13 17:23:21.685115 7fcd0964c700 5 -- op tracker -- , seq: 356, time: 2014-08-13 17:23:21.685115, event: done, request: pg_info(1 pgs e194477:22.3d5) v4 -2848> 2014-08-13 17:23:21.685135 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x52bd340 con 0x4fdad60 -2847> 2014-08-13 17:23:21.685131 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.730909 7 0.000353 -2846> 2014-08-13 17:23:21.685153 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2845> 2014-08-13 17:23:21.685157 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2844> 2014-08-13 17:23:21.685162 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x567c800 con 0x4fd1a20 -2843> 2014-08-13 17:23:21.685187 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2842> 2014-08-13 17:23:21.685176 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194471 n=35 ec=183096 les/c 194474/194474 194476/194476/192943) [337,20,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2841> 2014-08-13 17:23:21.685196 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 7 ==== pg_notify(24.1bb(27) epoch 194477) v5 ==== 2261+0+0 (3277685512 0 0) 0x56af540 con 0x4fde880 -2840> 2014-08-13 17:23:21.685214 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.673438, event: header_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2839> 2014-08-13 17:23:21.685222 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.673442, event: throttled, request: pg_notify(24.1bb(27) epoch 194477) v5 -2838> 2014-08-13 17:23:21.685228 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.673562, event: all_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2837> 2014-08-13 17:23:21.685236 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.685211, event: dispatched, request: pg_notify(24.1bb(27) epoch 194477) v5 -2836> 2014-08-13 17:23:21.685247 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.685247, event: waiting_for_osdmap, request: pg_notify(24.1bb(27) epoch 194477) v5 -2835> 2014-08-13 17:23:21.685259 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.685259, event: started, request: pg_notify(24.1bb(27) epoch 194477) v5 -2834> 2014-08-13 17:23:21.685284 7fcd0964c700 5 -- op tracker -- , seq: 357, time: 2014-08-13 17:23:21.685284, event: done, request: pg_notify(24.1bb(27) epoch 194477) v5 -2833> 2014-08-13 17:23:21.685319 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 8 ==== pg_notify(24.181(7) epoch 194477) v5 ==== 1149+0+0 (1634300321 0 0) 0x5888fc0 con 0x4fde300 -2832> 2014-08-13 17:23:21.685322 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194474/194474 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.724275 7 0.000275 -2831> 2014-08-13 17:23:21.685340 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.673669, event: header_read, request: pg_notify(24.181(7) epoch 194477) v5 -2830> 2014-08-13 17:23:21.685362 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194474/194474 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2829> 2014-08-13 17:23:21.685371 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.673672, event: throttled, request: pg_notify(24.181(7) epoch 194477) v5 -2828> 2014-08-13 17:23:21.685393 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.673745, event: all_read, request: pg_notify(24.181(7) epoch 194477) v5 -2827> 2014-08-13 17:23:21.685410 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.685337, event: dispatched, request: pg_notify(24.181(7) epoch 194477) v5 -2826> 2014-08-13 17:23:21.685421 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.685421, event: waiting_for_osdmap, request: pg_notify(24.181(7) epoch 194477) v5 -2825> 2014-08-13 17:23:21.685435 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.685435, event: started, request: pg_notify(24.181(7) epoch 194477) v5 -2824> 2014-08-13 17:23:21.685426 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194477:22.639) v4 -- ?+0 0x588e200 -2823> 2014-08-13 17:23:21.685386 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194471 n=28 ec=183096 les/c 194474/194474 194476/194476/192971) [51,4,308] r=1 lpr=194476 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2822> 2014-08-13 17:23:21.685474 7fcd0964c700 5 -- op tracker -- , seq: 358, time: 2014-08-13 17:23:21.685474, event: done, request: pg_notify(24.181(7) epoch 194477) v5 -2821> 2014-08-13 17:23:21.685509 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 4 ==== pg_notify(24.b6(6) epoch 194477) v5 ==== 1094+0+0 (694073278 0 0) 0x588e040 con 0x4fd35a0 -2820> 2014-08-13 17:23:21.685532 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.659901, event: header_read, request: pg_notify(24.b6(6) epoch 194477) v5 -2819> 2014-08-13 17:23:21.685518 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194474/194474 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.733481 7 0.000274 -2818> 2014-08-13 17:23:21.685540 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.659902, event: throttled, request: pg_notify(24.b6(6) epoch 194477) v5 -2817> 2014-08-13 17:23:21.685546 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.660216, event: all_read, request: pg_notify(24.b6(6) epoch 194477) v5 -2816> 2014-08-13 17:23:21.685553 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.685529, event: dispatched, request: pg_notify(24.b6(6) epoch 194477) v5 -2815> 2014-08-13 17:23:21.685544 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194474/194474 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2814> 2014-08-13 17:23:21.685560 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.685560, event: waiting_for_osdmap, request: pg_notify(24.b6(6) epoch 194477) v5 -2813> 2014-08-13 17:23:21.685569 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.685569, event: started, request: pg_notify(24.b6(6) epoch 194477) v5 -2812> 2014-08-13 17:23:21.685565 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194471 n=26 ec=183096 les/c 194474/194474 194476/194476/192976) [30,335,4] r=2 lpr=194476 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2811> 2014-08-13 17:23:21.685592 7fcd0964c700 5 -- op tracker -- , seq: 359, time: 2014-08-13 17:23:21.685592, event: done, request: pg_notify(24.b6(6) epoch 194477) v5 -2810> 2014-08-13 17:23:21.685626 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 9 ==== pg_notify(22.481(34) epoch 194477) v5 ==== 2628+0+0 (2630983905 0 0) 0x588cd00 con 0x4fd06e0 -2809> 2014-08-13 17:23:21.685648 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.656551, event: header_read, request: pg_notify(22.481(34) epoch 194477) v5 -2808> 2014-08-13 17:23:21.685655 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.656553, event: throttled, request: pg_notify(22.481(34) epoch 194477) v5 -2807> 2014-08-13 17:23:21.685661 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.656667, event: all_read, request: pg_notify(22.481(34) epoch 194477) v5 -2806> 2014-08-13 17:23:21.685667 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.685645, event: dispatched, request: pg_notify(22.481(34) epoch 194477) v5 -2805> 2014-08-13 17:23:21.685676 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.685675, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194477) v5 -2804> 2014-08-13 17:23:21.685683 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.685683, event: started, request: pg_notify(22.481(34) epoch 194477) v5 -2803> 2014-08-13 17:23:21.685704 7fcd0964c700 5 -- op tracker -- , seq: 360, time: 2014-08-13 17:23:21.685703, event: done, request: pg_notify(22.481(34) epoch 194477) v5 -2802> 2014-08-13 17:23:21.685728 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 8 ==== pg_notify(22.3d9(42) epoch 194477) v5 ==== 2996+0+0 (2760797936 0 0) 0x5b49340 con 0x4fd5960 -2801> 2014-08-13 17:23:21.685750 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.671916, event: header_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -2800> 2014-08-13 17:23:21.685744 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6821/54786 -- pg_query(22.707 epoch 194477) v3 -- ?+0 0x54f83c0 con 0x4ec7e80 -2799> 2014-08-13 17:23:21.685761 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.671918, event: throttled, request: pg_notify(22.3d9(42) epoch 194477) v5 -2798> 2014-08-13 17:23:21.685769 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.672106, event: all_read, request: pg_notify(22.3d9(42) epoch 194477) v5 -2797> 2014-08-13 17:23:21.685778 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.685748, event: dispatched, request: pg_notify(22.3d9(42) epoch 194477) v5 -2796> 2014-08-13 17:23:21.685795 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.685795, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194477) v5 -2795> 2014-08-13 17:23:21.685808 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.685808, event: started, request: pg_notify(22.3d9(42) epoch 194477) v5 -2794> 2014-08-13 17:23:21.685835 7fcd0964c700 5 -- op tracker -- , seq: 361, time: 2014-08-13 17:23:21.685835, event: done, request: pg_notify(22.3d9(42) epoch 194477) v5 -2793> 2014-08-13 17:23:21.685874 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 11 ==== pg_notify(22.39d(25) epoch 194477) v5 ==== 2117+0+0 (2057326087 0 0) 0x588c280 con 0x4fd32e0 -2792> 2014-08-13 17:23:21.685892 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.658441, event: header_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2791> 2014-08-13 17:23:21.685899 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.658443, event: throttled, request: pg_notify(22.39d(25) epoch 194477) v5 -2790> 2014-08-13 17:23:21.685904 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.658548, event: all_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2789> 2014-08-13 17:23:21.685910 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.685890, event: dispatched, request: pg_notify(22.39d(25) epoch 194477) v5 -2788> 2014-08-13 17:23:21.685916 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.685916, event: waiting_for_osdmap, request: pg_notify(22.39d(25) epoch 194477) v5 -2787> 2014-08-13 17:23:21.685924 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.685924, event: started, request: pg_notify(22.39d(25) epoch 194477) v5 -2786> 2014-08-13 17:23:21.685943 7fcd0964c700 5 -- op tracker -- , seq: 362, time: 2014-08-13 17:23:21.685943, event: done, request: pg_notify(22.39d(25) epoch 194477) v5 -2785> 2014-08-13 17:23:21.685968 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 7 ==== pg_info(1 pgs e194477:22.3b9) v4 ==== 714+0+0 (983423061 0 0) 0x54d6e40 con 0x4fd53e0 -2784> 2014-08-13 17:23:21.685984 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.675310, event: header_read, request: pg_info(1 pgs e194477:22.3b9) v4 -2783> 2014-08-13 17:23:21.685991 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.675313, event: throttled, request: pg_info(1 pgs e194477:22.3b9) v4 -2782> 2014-08-13 17:23:21.685997 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.675437, event: all_read, request: pg_info(1 pgs e194477:22.3b9) v4 -2781> 2014-08-13 17:23:21.685987 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194477:24.369) v4 -- ?+0 0x5888540 -2780> 2014-08-13 17:23:21.686003 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.685982, event: dispatched, request: pg_info(1 pgs e194477:22.3b9) v4 -2779> 2014-08-13 17:23:21.686010 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.686009, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3b9) v4 -2778> 2014-08-13 17:23:21.686017 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.686017, event: started, request: pg_info(1 pgs e194477:22.3b9) v4 -2777> 2014-08-13 17:23:21.686038 7fcd0964c700 5 -- op tracker -- , seq: 363, time: 2014-08-13 17:23:21.686038, event: done, request: pg_info(1 pgs e194477:22.3b9) v4 -2776> 2014-08-13 17:23:21.686052 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.309 192.168.1.101:6814/23703 7 ==== pg_info(1 pgs e194477:22.481) v4 ==== 714+0+0 (3482583824 0 0) 0x56ad080 con 0x5303700 -2775> 2014-08-13 17:23:21.686066 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.677703, event: header_read, request: pg_info(1 pgs e194477:22.481) v4 -2774> 2014-08-13 17:23:21.686072 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.677705, event: throttled, request: pg_info(1 pgs e194477:22.481) v4 -2773> 2014-08-13 17:23:21.686078 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.677768, event: all_read, request: pg_info(1 pgs e194477:22.481) v4 -2772> 2014-08-13 17:23:21.686083 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.686064, event: dispatched, request: pg_info(1 pgs e194477:22.481) v4 -2771> 2014-08-13 17:23:21.686089 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.686089, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.481) v4 -2770> 2014-08-13 17:23:21.686097 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.686097, event: started, request: pg_info(1 pgs e194477:22.481) v4 -2769> 2014-08-13 17:23:21.686116 7fcd0964c700 5 -- op tracker -- , seq: 364, time: 2014-08-13 17:23:21.686116, event: done, request: pg_info(1 pgs e194477:22.481) v4 -2768> 2014-08-13 17:23:21.686129 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 9 ==== pg_notify(24.b6(29) epoch 194477) v5 ==== 2379+0+0 (1511250803 0 0) 0x56aaf40 con 0x5305ee0 -2767> 2014-08-13 17:23:21.686150 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.657578, event: header_read, request: pg_notify(24.b6(29) epoch 194477) v5 -2766> 2014-08-13 17:23:21.686166 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.657580, event: throttled, request: pg_notify(24.b6(29) epoch 194477) v5 -2765> 2014-08-13 17:23:21.686175 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.657849, event: all_read, request: pg_notify(24.b6(29) epoch 194477) v5 -2764> 2014-08-13 17:23:21.686185 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.686148, event: dispatched, request: pg_notify(24.b6(29) epoch 194477) v5 -2763> 2014-08-13 17:23:21.686192 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.686192, event: waiting_for_osdmap, request: pg_notify(24.b6(29) epoch 194477) v5 -2762> 2014-08-13 17:23:21.686203 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.686203, event: started, request: pg_notify(24.b6(29) epoch 194477) v5 -2761> 2014-08-13 17:23:21.686190 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.54 192.168.1.111:6836/14923 -- pg_info(1 pgs e194477:22.30f) v4 -- ?+0 0x588de80 -2760> 2014-08-13 17:23:21.686225 7fcd0964c700 5 -- op tracker -- , seq: 365, time: 2014-08-13 17:23:21.686225, event: done, request: pg_notify(24.b6(29) epoch 194477) v5 -2759> 2014-08-13 17:23:21.686253 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 5 ==== pg_notify(24.9b(29) epoch 194477) v5 ==== 2383+0+0 (2768490881 0 0) 0x54d1180 con 0x53056a0 -2758> 2014-08-13 17:23:21.686040 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6812/32413 -- pg_query(22.707 epoch 194477) v3 -- ?+0 0x54f8000 con 0x5303440 -2757> 2014-08-13 17:23:21.686280 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.672945, event: header_read, request: pg_notify(24.9b(29) epoch 194477) v5 -2756> 2014-08-13 17:23:21.686288 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.672946, event: throttled, request: pg_notify(24.9b(29) epoch 194477) v5 -2755> 2014-08-13 17:23:21.686294 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.673055, event: all_read, request: pg_notify(24.9b(29) epoch 194477) v5 -2754> 2014-08-13 17:23:21.686299 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.686278, event: dispatched, request: pg_notify(24.9b(29) epoch 194477) v5 -2753> 2014-08-13 17:23:21.686305 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.686305, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194477) v5 -2752> 2014-08-13 17:23:21.686313 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.686312, event: started, request: pg_notify(24.9b(29) epoch 194477) v5 -2751> 2014-08-13 17:23:21.686332 7fcd0964c700 5 -- op tracker -- , seq: 366, time: 2014-08-13 17:23:21.686332, event: done, request: pg_notify(24.9b(29) epoch 194477) v5 -2750> 2014-08-13 17:23:21.686349 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194477:22.1a0) v4 -- ?+0 0x56aa840 -2749> 2014-08-13 17:23:21.686354 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 6 ==== pg_notify(24.9b(4) epoch 194477) v5 ==== 980+0+0 (711207696 0 0) 0x54d4440 con 0x53053e0 -2748> 2014-08-13 17:23:21.686374 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.673785, event: header_read, request: pg_notify(24.9b(4) epoch 194477) v5 -2747> 2014-08-13 17:23:21.686381 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.673788, event: throttled, request: pg_notify(24.9b(4) epoch 194477) v5 -2746> 2014-08-13 17:23:21.686387 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.673865, event: all_read, request: pg_notify(24.9b(4) epoch 194477) v5 -2745> 2014-08-13 17:23:21.686393 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.686372, event: dispatched, request: pg_notify(24.9b(4) epoch 194477) v5 -2744> 2014-08-13 17:23:21.686399 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.686399, event: waiting_for_osdmap, request: pg_notify(24.9b(4) epoch 194477) v5 -2743> 2014-08-13 17:23:21.686406 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.686406, event: started, request: pg_notify(24.9b(4) epoch 194477) v5 -2742> 2014-08-13 17:23:21.686425 7fcd0964c700 5 -- op tracker -- , seq: 367, time: 2014-08-13 17:23:21.686425, event: done, request: pg_notify(24.9b(4) epoch 194477) v5 -2741> 2014-08-13 17:23:21.686441 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 5 ==== pg_notify(22.f3(3) epoch 194477) v5 ==== 887+0+0 (1341007147 0 0) 0x56aaa00 con 0x5307a60 -2740> 2014-08-13 17:23:21.686456 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.675936, event: header_read, request: pg_notify(22.f3(3) epoch 194477) v5 -2739> 2014-08-13 17:23:21.686462 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.675938, event: throttled, request: pg_notify(22.f3(3) epoch 194477) v5 -2738> 2014-08-13 17:23:21.686468 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.676065, event: all_read, request: pg_notify(22.f3(3) epoch 194477) v5 -2737> 2014-08-13 17:23:21.686473 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.686454, event: dispatched, request: pg_notify(22.f3(3) epoch 194477) v5 -2736> 2014-08-13 17:23:21.686479 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.686479, event: waiting_for_osdmap, request: pg_notify(22.f3(3) epoch 194477) v5 -2735> 2014-08-13 17:23:21.686520 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.686520, event: started, request: pg_notify(22.f3(3) epoch 194477) v5 -2734> 2014-08-13 17:23:21.686540 7fcd0964c700 5 -- op tracker -- , seq: 368, time: 2014-08-13 17:23:21.686540, event: done, request: pg_notify(22.f3(3) epoch 194477) v5 -2733> 2014-08-13 17:23:21.686556 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 8 ==== pg_info(1 pgs e194477:22.113) v4 ==== 714+0+0 (1469345488 0 0) 0x56aeac0 con 0x5305c20 -2732> 2014-08-13 17:23:21.686576 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.684038, event: header_read, request: pg_info(1 pgs e194477:22.113) v4 -2731> 2014-08-13 17:23:21.686584 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.684040, event: throttled, request: pg_info(1 pgs e194477:22.113) v4 -2730> 2014-08-13 17:23:21.686589 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.684203, event: all_read, request: pg_info(1 pgs e194477:22.113) v4 -2729> 2014-08-13 17:23:21.686595 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.686574, event: dispatched, request: pg_info(1 pgs e194477:22.113) v4 -2728> 2014-08-13 17:23:21.686601 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.686601, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.113) v4 -2727> 2014-08-13 17:23:21.686608 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.686608, event: started, request: pg_info(1 pgs e194477:22.113) v4 -2726> 2014-08-13 17:23:21.686628 7fcd0964c700 5 -- op tracker -- , seq: 369, time: 2014-08-13 17:23:21.686628, event: done, request: pg_info(1 pgs e194477:22.113) v4 -2725> 2014-08-13 17:23:21.686650 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.369 192.168.1.108:6824/4037 6 ==== pg_notify(22.775(5) epoch 194477) v5 ==== 1001+0+0 (2869863447 0 0) 0x54d0700 con 0x552a260 -2724> 2014-08-13 17:23:21.686666 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.672530, event: header_read, request: pg_notify(22.775(5) epoch 194477) v5 -2723> 2014-08-13 17:23:21.686678 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.672533, event: throttled, request: pg_notify(22.775(5) epoch 194477) v5 -2722> 2014-08-13 17:23:21.686684 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.672607, event: all_read, request: pg_notify(22.775(5) epoch 194477) v5 -2721> 2014-08-13 17:23:21.686689 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.686664, event: dispatched, request: pg_notify(22.775(5) epoch 194477) v5 -2720> 2014-08-13 17:23:21.686696 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.686695, event: waiting_for_osdmap, request: pg_notify(22.775(5) epoch 194477) v5 -2719> 2014-08-13 17:23:21.686707 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.686707, event: started, request: pg_notify(22.775(5) epoch 194477) v5 -2718> 2014-08-13 17:23:21.686731 7fcd0964c700 5 -- op tracker -- , seq: 370, time: 2014-08-13 17:23:21.686731, event: done, request: pg_notify(22.775(5) epoch 194477) v5 -2717> 2014-08-13 17:23:21.686749 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 8 ==== pg_info(1 pgs e194477:22.1d2) v4 ==== 714+0+0 (2213653931 0 0) 0x54d5400 con 0x552fa60 -2716> 2014-08-13 17:23:21.686769 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.668564, event: header_read, request: pg_info(1 pgs e194477:22.1d2) v4 -2715> 2014-08-13 17:23:21.686776 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.668566, event: throttled, request: pg_info(1 pgs e194477:22.1d2) v4 -2714> 2014-08-13 17:23:21.686782 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.668642, event: all_read, request: pg_info(1 pgs e194477:22.1d2) v4 -2713> 2014-08-13 17:23:21.687197 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.686767, event: dispatched, request: pg_info(1 pgs e194477:22.1d2) v4 -2712> 2014-08-13 17:23:21.687212 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.687211, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1d2) v4 -2711> 2014-08-13 17:23:21.687226 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.687225, event: started, request: pg_info(1 pgs e194477:22.1d2) v4 -2710> 2014-08-13 17:23:21.687254 7fcd0964c700 5 -- op tracker -- , seq: 371, time: 2014-08-13 17:23:21.687254, event: done, request: pg_info(1 pgs e194477:22.1d2) v4 -2709> 2014-08-13 17:23:21.687284 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 5 ==== pg_notify(24.3d(27) epoch 194477) v5 ==== 2265+0+0 (3804825511 0 0) 0x56ae740 con 0x552f7a0 -2708> 2014-08-13 17:23:21.687308 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.674435, event: header_read, request: pg_notify(24.3d(27) epoch 194477) v5 -2707> 2014-08-13 17:23:21.687316 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.674436, event: throttled, request: pg_notify(24.3d(27) epoch 194477) v5 -2706> 2014-08-13 17:23:21.687321 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.674577, event: all_read, request: pg_notify(24.3d(27) epoch 194477) v5 -2705> 2014-08-13 17:23:21.687327 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.687305, event: dispatched, request: pg_notify(24.3d(27) epoch 194477) v5 -2704> 2014-08-13 17:23:21.687333 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.687333, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194477) v5 -2703> 2014-08-13 17:23:21.687342 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.687341, event: started, request: pg_notify(24.3d(27) epoch 194477) v5 -2702> 2014-08-13 17:23:21.687367 7fcd0964c700 5 -- op tracker -- , seq: 372, time: 2014-08-13 17:23:21.687367, event: done, request: pg_notify(24.3d(27) epoch 194477) v5 -2701> 2014-08-13 17:23:21.687396 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.346 192.168.1.104:6820/2308 7 ==== pg_info(1 pgs e194477:24.e2) v4 ==== 748+0+0 (1066906522 0 0) 0x56ad5c0 con 0x5520b00 -2700> 2014-08-13 17:23:21.687412 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.679784, event: header_read, request: pg_info(1 pgs e194477:24.e2) v4 -2699> 2014-08-13 17:23:21.687424 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.679786, event: throttled, request: pg_info(1 pgs e194477:24.e2) v4 -2698> 2014-08-13 17:23:21.687436 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.679866, event: all_read, request: pg_info(1 pgs e194477:24.e2) v4 -2697> 2014-08-13 17:23:21.687442 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.687410, event: dispatched, request: pg_info(1 pgs e194477:24.e2) v4 -2696> 2014-08-13 17:23:21.687448 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.687448, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.e2) v4 -2695> 2014-08-13 17:23:21.687456 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.687456, event: started, request: pg_info(1 pgs e194477:24.e2) v4 -2694> 2014-08-13 17:23:21.687479 7fcd0964c700 5 -- op tracker -- , seq: 373, time: 2014-08-13 17:23:21.687478, event: done, request: pg_info(1 pgs e194477:24.e2) v4 -2693> 2014-08-13 17:23:21.687504 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.357 192.168.1.102:6834/17224 6 ==== pg_info(1 pgs e194477:22.4df) v4 ==== 714+0+0 (195025360 0 0) 0x54d3b80 con 0x56b2260 -2692> 2014-08-13 17:23:21.687524 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.671095, event: header_read, request: pg_info(1 pgs e194477:22.4df) v4 -2691> 2014-08-13 17:23:21.687535 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.671098, event: throttled, request: pg_info(1 pgs e194477:22.4df) v4 -2690> 2014-08-13 17:23:21.687545 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.671179, event: all_read, request: pg_info(1 pgs e194477:22.4df) v4 -2689> 2014-08-13 17:23:21.687555 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.687521, event: dispatched, request: pg_info(1 pgs e194477:22.4df) v4 -2688> 2014-08-13 17:23:21.687573 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.687572, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4df) v4 -2687> 2014-08-13 17:23:21.687586 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.687585, event: started, request: pg_info(1 pgs e194477:22.4df) v4 -2686> 2014-08-13 17:23:21.687610 7fcd0964c700 5 -- op tracker -- , seq: 374, time: 2014-08-13 17:23:21.687610, event: done, request: pg_info(1 pgs e194477:22.4df) v4 -2685> 2014-08-13 17:23:21.687629 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.11 192.168.1.103:6813/37929 3 ==== pg_notify(22.1f1(5) epoch 194477) v5 ==== 1001+0+0 (2117454945 0 0) 0x588d080 con 0x56ba940 -2684> 2014-08-13 17:23:21.687668 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.670792, event: header_read, request: pg_notify(22.1f1(5) epoch 194477) v5 -2683> 2014-08-13 17:23:21.687689 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.670793, event: throttled, request: pg_notify(22.1f1(5) epoch 194477) v5 -2682> 2014-08-13 17:23:21.687709 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.670912, event: all_read, request: pg_notify(22.1f1(5) epoch 194477) v5 -2681> 2014-08-13 17:23:21.687728 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.687665, event: dispatched, request: pg_notify(22.1f1(5) epoch 194477) v5 -2680> 2014-08-13 17:23:21.687739 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.687739, event: waiting_for_osdmap, request: pg_notify(22.1f1(5) epoch 194477) v5 -2679> 2014-08-13 17:23:21.687748 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.687748, event: started, request: pg_notify(22.1f1(5) epoch 194477) v5 -2678> 2014-08-13 17:23:21.685760 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194474/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 0.737747 8 0.000467 -2677> 2014-08-13 17:23:21.687808 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194474/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive -2676> 2014-08-13 17:23:21.687856 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194448 n=63 ec=192688 les/c 194474/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive/RepNotRecovering -2675> 2014-08-13 17:23:21.687820 7fcd0964c700 5 -- op tracker -- , seq: 375, time: 2014-08-13 17:23:21.687819, event: done, request: pg_notify(22.1f1(5) epoch 194477) v5 -2674> 2014-08-13 17:23:21.688451 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.307 192.168.1.109:6804/31016 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x51e7080 con 0x56b4fc0 -2673> 2014-08-13 17:23:21.688485 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -2672> 2014-08-13 17:23:21.688548 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.348 192.168.1.110:6800/41204 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52b8900 con 0x53e4fc0 -2671> 2014-08-13 17:23:21.688574 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -2670> 2014-08-13 17:23:21.688587 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.285 192.168.1.101:6822/26295 11 ==== pg_info(1 pgs e194477:24.1e6) v4 ==== 748+0+0 (2381046100 0 0) 0x56a8c40 con 0x4e314a0 -2669> 2014-08-13 17:23:21.688606 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.684815, event: header_read, request: pg_info(1 pgs e194477:24.1e6) v4 -2668> 2014-08-13 17:23:21.688619 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.684816, event: throttled, request: pg_info(1 pgs e194477:24.1e6) v4 -2667> 2014-08-13 17:23:21.688631 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.684892, event: all_read, request: pg_info(1 pgs e194477:24.1e6) v4 -2666> 2014-08-13 17:23:21.688659 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.688603, event: dispatched, request: pg_info(1 pgs e194477:24.1e6) v4 -2665> 2014-08-13 17:23:21.688671 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.688671, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.1e6) v4 -2664> 2014-08-13 17:23:21.688684 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.688684, event: started, request: pg_info(1 pgs e194477:24.1e6) v4 -2663> 2014-08-13 17:23:21.688716 7fcd0964c700 5 -- op tracker -- , seq: 376, time: 2014-08-13 17:23:21.688715, event: done, request: pg_info(1 pgs e194477:24.1e6) v4 -2662> 2014-08-13 17:23:21.688768 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.9 192.168.1.107:6817/53997 6 ==== pg_info(1 pgs e194477:22.28f) v4 ==== 714+0+0 (3395307666 0 0) 0x5888000 con 0x4e34780 -2661> 2014-08-13 17:23:21.688794 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.678824, event: header_read, request: pg_info(1 pgs e194477:22.28f) v4 -2660> 2014-08-13 17:23:21.688807 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.678826, event: throttled, request: pg_info(1 pgs e194477:22.28f) v4 -2659> 2014-08-13 17:23:21.688816 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.678940, event: all_read, request: pg_info(1 pgs e194477:22.28f) v4 -2658> 2014-08-13 17:23:21.688827 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.688791, event: dispatched, request: pg_info(1 pgs e194477:22.28f) v4 -2657> 2014-08-13 17:23:21.688834 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.688834, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.28f) v4 -2656> 2014-08-13 17:23:21.688843 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.688842, event: started, request: pg_info(1 pgs e194477:22.28f) v4 -2655> 2014-08-13 17:23:21.688865 7fcd0964c700 5 -- op tracker -- , seq: 377, time: 2014-08-13 17:23:21.688865, event: done, request: pg_info(1 pgs e194477:22.28f) v4 -2654> 2014-08-13 17:23:21.688965 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 5 ==== pg_info(1 pgs e194477:22.4aa) v4 ==== 714+0+0 (1904617758 0 0) 0x56a9f80 con 0x4e35120 -2653> 2014-08-13 17:23:21.689010 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.679333, event: header_read, request: pg_info(1 pgs e194477:22.4aa) v4 -2652> 2014-08-13 17:23:21.689151 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.679335, event: throttled, request: pg_info(1 pgs e194477:22.4aa) v4 -2651> 2014-08-13 17:23:21.689176 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.682074, event: all_read, request: pg_info(1 pgs e194477:22.4aa) v4 -2650> 2014-08-13 17:23:21.689186 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.689006, event: dispatched, request: pg_info(1 pgs e194477:22.4aa) v4 -2649> 2014-08-13 17:23:21.689195 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.689195, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4aa) v4 -2648> 2014-08-13 17:23:21.689216 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.689216, event: started, request: pg_info(1 pgs e194477:22.4aa) v4 -2647> 2014-08-13 17:23:21.689253 7fcd0964c700 5 -- op tracker -- , seq: 378, time: 2014-08-13 17:23:21.689253, event: done, request: pg_info(1 pgs e194477:22.4aa) v4 -2646> 2014-08-13 17:23:21.689280 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 9 ==== pg_notify(22.362(5) epoch 194477) v5 ==== 1001+0+0 (822319021 0 0) 0x4e50000 con 0x4e35ac0 -2645> 2014-08-13 17:23:21.689312 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.660252, event: header_read, request: pg_notify(22.362(5) epoch 194477) v5 -2644> 2014-08-13 17:23:21.689325 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.660253, event: throttled, request: pg_notify(22.362(5) epoch 194477) v5 -2643> 2014-08-13 17:23:21.689335 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.660321, event: all_read, request: pg_notify(22.362(5) epoch 194477) v5 -2642> 2014-08-13 17:23:21.689342 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.689309, event: dispatched, request: pg_notify(22.362(5) epoch 194477) v5 -2641> 2014-08-13 17:23:21.689352 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.689352, event: waiting_for_osdmap, request: pg_notify(22.362(5) epoch 194477) v5 -2640> 2014-08-13 17:23:21.689365 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.689365, event: started, request: pg_notify(22.362(5) epoch 194477) v5 -2639> 2014-08-13 17:23:21.689395 7fcd0964c700 5 -- op tracker -- , seq: 379, time: 2014-08-13 17:23:21.689395, event: done, request: pg_notify(22.362(5) epoch 194477) v5 -2638> 2014-08-13 17:23:21.689417 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 9 ==== pg_info(1 pgs e194477:22.22c) v4 ==== 714+0+0 (3144481058 0 0) 0x56aad80 con 0x4e35800 -2637> 2014-08-13 17:23:21.689438 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.667951, event: header_read, request: pg_info(1 pgs e194477:22.22c) v4 -2636> 2014-08-13 17:23:21.689449 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.667952, event: throttled, request: pg_info(1 pgs e194477:22.22c) v4 -2635> 2014-08-13 17:23:21.689458 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.668024, event: all_read, request: pg_info(1 pgs e194477:22.22c) v4 -2634> 2014-08-13 17:23:21.689468 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.689435, event: dispatched, request: pg_info(1 pgs e194477:22.22c) v4 -2633> 2014-08-13 17:23:21.689479 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.689478, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.22c) v4 -2632> 2014-08-13 17:23:21.689491 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.689491, event: started, request: pg_info(1 pgs e194477:22.22c) v4 -2631> 2014-08-13 17:23:21.689515 7fcd0964c700 5 -- op tracker -- , seq: 380, time: 2014-08-13 17:23:21.689515, event: done, request: pg_info(1 pgs e194477:22.22c) v4 -2630> 2014-08-13 17:23:21.689533 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 7 ==== pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 ==== 4788+0+0 (3444273008 0 0) 0x5b48e00 con 0x4e35540 -2629> 2014-08-13 17:23:21.689554 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.676849, event: header_read, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2628> 2014-08-13 17:23:21.689565 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.676851, event: throttled, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2627> 2014-08-13 17:23:21.689575 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.677110, event: all_read, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2626> 2014-08-13 17:23:21.689585 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.689552, event: dispatched, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2625> 2014-08-13 17:23:21.689596 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.689596, event: waiting_for_osdmap, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2624> 2014-08-13 17:23:21.689608 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.689608, event: started, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2623> 2014-08-13 17:23:21.689661 7fcd0964c700 5 -- op tracker -- , seq: 381, time: 2014-08-13 17:23:21.689661, event: done, request: pg_notify(2.2a(1),1.2b(1),24.34(50) epoch 194477) v5 -2622> 2014-08-13 17:23:21.689697 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.45 192.168.1.112:6828/6040 12 ==== pg_info(1 pgs e194477:24.369) v4 ==== 748+0+0 (3499698355 0 0) 0x5888540 con 0x4ec09a0 -2621> 2014-08-13 17:23:21.689717 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.686835, event: header_read, request: pg_info(1 pgs e194477:24.369) v4 -2620> 2014-08-13 17:23:21.689727 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.686836, event: throttled, request: pg_info(1 pgs e194477:24.369) v4 -2619> 2014-08-13 17:23:21.689736 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.686913, event: all_read, request: pg_info(1 pgs e194477:24.369) v4 -2618> 2014-08-13 17:23:21.689744 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.689714, event: dispatched, request: pg_info(1 pgs e194477:24.369) v4 -2617> 2014-08-13 17:23:21.689754 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.689754, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.369) v4 -2616> 2014-08-13 17:23:21.689765 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.689765, event: started, request: pg_info(1 pgs e194477:24.369) v4 -2615> 2014-08-13 17:23:21.689745 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.037978 7 0.000201 -2614> 2014-08-13 17:23:21.689790 7fcd0964c700 5 -- op tracker -- , seq: 382, time: 2014-08-13 17:23:21.689789, event: done, request: pg_info(1 pgs e194477:24.369) v4 -2613> 2014-08-13 17:23:21.689794 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2612> 2014-08-13 17:23:21.689808 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 6 ==== pg_info(1 pgs e194477:22.30f) v4 ==== 714+0+0 (272523132 0 0) 0x588de80 con 0x4ec2aa0 -2611> 2014-08-13 17:23:21.689826 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.687536, event: header_read, request: pg_info(1 pgs e194477:22.30f) v4 -2610> 2014-08-13 17:23:21.689836 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.687538, event: throttled, request: pg_info(1 pgs e194477:22.30f) v4 -2609> 2014-08-13 17:23:21.689845 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.687623, event: all_read, request: pg_info(1 pgs e194477:22.30f) v4 -2608> 2014-08-13 17:23:21.689854 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.689824, event: dispatched, request: pg_info(1 pgs e194477:22.30f) v4 -2607> 2014-08-13 17:23:21.689863 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.689863, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.30f) v4 -2606> 2014-08-13 17:23:21.689874 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.689874, event: started, request: pg_info(1 pgs e194477:22.30f) v4 -2605> 2014-08-13 17:23:21.689896 7fcd0964c700 5 -- op tracker -- , seq: 383, time: 2014-08-13 17:23:21.689896, event: done, request: pg_info(1 pgs e194477:22.30f) v4 -2604> 2014-08-13 17:23:21.689903 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000110 0 0.000000 -2603> 2014-08-13 17:23:21.689913 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x5670240 con 0x4ec27e0 -2602> 2014-08-13 17:23:21.689933 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2601> 2014-08-13 17:23:21.689932 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2600> 2014-08-13 17:23:21.689947 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 13 ==== pg_info(1 pgs e194477:22.6bd) v4 ==== 714+0+0 (2448491805 0 0) 0x56a9880 con 0x4ec3440 -2599> 2014-08-13 17:23:21.689964 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.676293, event: header_read, request: pg_info(1 pgs e194477:22.6bd) v4 -2598> 2014-08-13 17:23:21.689974 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.676295, event: throttled, request: pg_info(1 pgs e194477:22.6bd) v4 -2597> 2014-08-13 17:23:21.689985 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.676365, event: all_read, request: pg_info(1 pgs e194477:22.6bd) v4 -2596> 2014-08-13 17:23:21.689990 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.689962, event: dispatched, request: pg_info(1 pgs e194477:22.6bd) v4 -2595> 2014-08-13 17:23:21.689997 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.689997, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.6bd) v4 -2594> 2014-08-13 17:23:21.690005 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.690005, event: started, request: pg_info(1 pgs e194477:22.6bd) v4 -2593> 2014-08-13 17:23:21.690024 7fcd0964c700 5 -- op tracker -- , seq: 384, time: 2014-08-13 17:23:21.690024, event: done, request: pg_info(1 pgs e194477:22.6bd) v4 -2592> 2014-08-13 17:23:21.690038 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 12 ==== pg_info(1 pgs e194477:22.4df) v4 ==== 714+0+0 (2297579165 0 0) 0x588bd40 con 0x4ec4780 -2591> 2014-08-13 17:23:21.690053 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.659702, event: header_read, request: pg_info(1 pgs e194477:22.4df) v4 -2590> 2014-08-13 17:23:21.690046 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- pg_query(22.3e4 epoch 194477) v3 -- ?+0 0x2f94740 con 0x4ec5d80 -2589> 2014-08-13 17:23:21.690064 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.659704, event: throttled, request: pg_info(1 pgs e194477:22.4df) v4 -2588> 2014-08-13 17:23:21.690073 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.659791, event: all_read, request: pg_info(1 pgs e194477:22.4df) v4 -2587> 2014-08-13 17:23:21.690073 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6814/43857 -- pg_query(22.3e4 epoch 194477) v3 -- ?+0 0x2f96360 con 0x55f2100 -2586> 2014-08-13 17:23:21.690079 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.690051, event: dispatched, request: pg_info(1 pgs e194477:22.4df) v4 -2585> 2014-08-13 17:23:21.690085 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.690085, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4df) v4 -2584> 2014-08-13 17:23:21.690093 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.690093, event: started, request: pg_info(1 pgs e194477:22.4df) v4 -2583> 2014-08-13 17:23:21.690124 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(38) epoch 194477) v5 -- ?+0 0x58881c0 con 0x4ec56a0 -2582> 2014-08-13 17:23:21.690191 7fcd0964c700 5 -- op tracker -- , seq: 385, time: 2014-08-13 17:23:21.690190, event: done, request: pg_info(1 pgs e194477:22.4df) v4 -2581> 2014-08-13 17:23:21.690221 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 9 ==== pg_info(1 pgs e194477:24.2a1) v4 ==== 748+0+0 (3109247085 0 0) 0x5888380 con 0x4ec5ac0 -2580> 2014-08-13 17:23:21.690219 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.323 192.168.1.106:6816/8118 -- pg_info(1 pgs e194477:22.12a) v4 -- ?+0 0x56a9500 -2579> 2014-08-13 17:23:21.690244 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.682906, event: header_read, request: pg_info(1 pgs e194477:24.2a1) v4 -2578> 2014-08-13 17:23:21.690254 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.682907, event: throttled, request: pg_info(1 pgs e194477:24.2a1) v4 -2577> 2014-08-13 17:23:21.690263 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.682987, event: all_read, request: pg_info(1 pgs e194477:24.2a1) v4 -2576> 2014-08-13 17:23:21.690272 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.690241, event: dispatched, request: pg_info(1 pgs e194477:24.2a1) v4 -2575> 2014-08-13 17:23:21.690282 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.690281, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.2a1) v4 -2574> 2014-08-13 17:23:21.690282 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194477:22.4fe) v4 -- ?+0 0x56abf00 -2573> 2014-08-13 17:23:21.690294 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.690293, event: started, request: pg_info(1 pgs e194477:24.2a1) v4 -2572> 2014-08-13 17:23:21.690321 7fcd0964c700 5 -- op tracker -- , seq: 386, time: 2014-08-13 17:23:21.690320, event: done, request: pg_info(1 pgs e194477:24.2a1) v4 -2571> 2014-08-13 17:23:21.690327 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194477:24.33a) v4 -- ?+0 0x56a96c0 -2570> 2014-08-13 17:23:21.690334 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 3 ==== pg_notify(24.181(28) epoch 194477) v5 ==== 2316+0+0 (2891192339 0 0) 0x56a9180 con 0x4ec70c0 -2569> 2014-08-13 17:23:21.690385 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.669029, event: header_read, request: pg_notify(24.181(28) epoch 194477) v5 -2568> 2014-08-13 17:23:21.690393 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.669032, event: throttled, request: pg_notify(24.181(28) epoch 194477) v5 -2567> 2014-08-13 17:23:21.690398 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.669193, event: all_read, request: pg_notify(24.181(28) epoch 194477) v5 -2566> 2014-08-13 17:23:21.690404 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.690383, event: dispatched, request: pg_notify(24.181(28) epoch 194477) v5 -2565> 2014-08-13 17:23:21.690403 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.51 192.168.1.112:6832/6655 -- pg_info(1 pgs e194477:22.537) v4 -- ?+0 0x56a9a40 -2564> 2014-08-13 17:23:21.690410 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.690410, event: waiting_for_osdmap, request: pg_notify(24.181(28) epoch 194477) v5 -2563> 2014-08-13 17:23:21.690419 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.690419, event: started, request: pg_notify(24.181(28) epoch 194477) v5 -2562> 2014-08-13 17:23:21.690442 7fcd0964c700 5 -- op tracker -- , seq: 387, time: 2014-08-13 17:23:21.690442, event: done, request: pg_notify(24.181(28) epoch 194477) v5 -2561> 2014-08-13 17:23:21.690442 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194477:22.791) v4 -- ?+0 0x56ad940 -2560> 2014-08-13 17:23:21.690467 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 8 ==== pg_info(1 pgs e194477:22.2b9) v4 ==== 714+0+0 (1517463766 0 0) 0x56acb40 con 0x4eb23c0 -2559> 2014-08-13 17:23:21.690487 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.678543, event: header_read, request: pg_info(1 pgs e194477:22.2b9) v4 -2558> 2014-08-13 17:23:21.690494 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.678545, event: throttled, request: pg_info(1 pgs e194477:22.2b9) v4 -2557> 2014-08-13 17:23:21.690500 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.678613, event: all_read, request: pg_info(1 pgs e194477:22.2b9) v4 -2556> 2014-08-13 17:23:21.690506 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.690485, event: dispatched, request: pg_info(1 pgs e194477:22.2b9) v4 -2555> 2014-08-13 17:23:21.690512 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.690512, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2b9) v4 -2554> 2014-08-13 17:23:21.690520 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.690519, event: started, request: pg_info(1 pgs e194477:22.2b9) v4 -2553> 2014-08-13 17:23:21.690497 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194474/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 0.732296 8 0.000601 -2552> 2014-08-13 17:23:21.690539 7fcd0964c700 5 -- op tracker -- , seq: 388, time: 2014-08-13 17:23:21.690538, event: done, request: pg_info(1 pgs e194477:22.2b9) v4 -2551> 2014-08-13 17:23:21.690537 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194474/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive -2550> 2014-08-13 17:23:21.690561 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194472 n=52 ec=192688 les/c 194474/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194476 pi=192688-194475/42 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering -2549> 2014-08-13 17:23:21.690563 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.292 192.168.1.113:6812/16056 -- pg_info(1 pgs e194477:22.3ad) v4 -- ?+0 0x56a9dc0 -2548> 2014-08-13 17:23:21.690577 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 4 ==== pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 ==== 2996+0+0 (125624844 0 0) 0x56dec00 con 0x4ec7e80 -2547> 2014-08-13 17:23:21.690596 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.687003, event: header_read, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2546> 2014-08-13 17:23:21.690605 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.687005, event: throttled, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2545> 2014-08-13 17:23:21.690612 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.687134, event: all_read, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2544> 2014-08-13 17:23:21.690620 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.690594, event: dispatched, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2543> 2014-08-13 17:23:21.690628 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.690627, event: waiting_for_osdmap, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2542> 2014-08-13 17:23:21.690637 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.690637, event: started, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2541> 2014-08-13 17:23:21.690655 7fcd0964c700 5 -- op tracker -- , seq: 389, time: 2014-08-13 17:23:21.690654, event: done, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2540> 2014-08-13 17:23:21.690677 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.294 192.168.1.101:6812/23402 6 ==== pg_info(1 pgs e194477:22.630) v4 ==== 714+0+0 (3864745313 0 0) 0x56ad400 con 0x4eb40a0 -2539> 2014-08-13 17:23:21.690692 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.678689, event: header_read, request: pg_info(1 pgs e194477:22.630) v4 -2538> 2014-08-13 17:23:21.690699 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.678691, event: throttled, request: pg_info(1 pgs e194477:22.630) v4 -2537> 2014-08-13 17:23:21.690704 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.678773, event: all_read, request: pg_info(1 pgs e194477:22.630) v4 -2536> 2014-08-13 17:23:21.690710 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.690690, event: dispatched, request: pg_info(1 pgs e194477:22.630) v4 -2535> 2014-08-13 17:23:21.690716 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.690716, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.630) v4 -2534> 2014-08-13 17:23:21.690723 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.690723, event: started, request: pg_info(1 pgs e194477:22.630) v4 -2533> 2014-08-13 17:23:21.690707 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_query(22.99 epoch 194477) v3 -- ?+0 0x54fe540 con 0x4ec56a0 -2532> 2014-08-13 17:23:21.690742 7fcd0964c700 5 -- op tracker -- , seq: 390, time: 2014-08-13 17:23:21.690742, event: done, request: pg_info(1 pgs e194477:22.630) v4 -2531> 2014-08-13 17:23:21.690742 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_query(22.99 epoch 194477) v3 -- ?+0 0x54fe360 con 0x4eb18c0 -2530> 2014-08-13 17:23:21.690764 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.358 192.168.1.114:6824/52078 6 ==== pg_info(1 pgs e194477:22.2d1) v4 ==== 714+0+0 (373703566 0 0) 0x56ab480 con 0x4eb6720 -2529> 2014-08-13 17:23:21.690779 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.679501, event: header_read, request: pg_info(1 pgs e194477:22.2d1) v4 -2528> 2014-08-13 17:23:21.690785 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.679503, event: throttled, request: pg_info(1 pgs e194477:22.2d1) v4 -2527> 2014-08-13 17:23:21.690791 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.679572, event: all_read, request: pg_info(1 pgs e194477:22.2d1) v4 -2526> 2014-08-13 17:23:21.690796 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.690777, event: dispatched, request: pg_info(1 pgs e194477:22.2d1) v4 -2525> 2014-08-13 17:23:21.690802 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.690802, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.2d1) v4 -2524> 2014-08-13 17:23:21.690810 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.690809, event: started, request: pg_info(1 pgs e194477:22.2d1) v4 -2523> 2014-08-13 17:23:21.690827 7fcd0964c700 5 -- op tracker -- , seq: 391, time: 2014-08-13 17:23:21.690827, event: done, request: pg_info(1 pgs e194477:22.2d1) v4 -2522> 2014-08-13 17:23:21.690849 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.315 192.168.1.113:6800/14161 2 ==== pg_notify(24.b6(32) epoch 194477) v5 ==== 2556+0+0 (25117777 0 0) 0x56afa80 con 0x4eb6460 -2521> 2014-08-13 17:23:21.690864 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.682526, event: header_read, request: pg_notify(24.b6(32) epoch 194477) v5 -2520> 2014-08-13 17:23:21.690871 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.682527, event: throttled, request: pg_notify(24.b6(32) epoch 194477) v5 -2519> 2014-08-13 17:23:21.690876 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.682634, event: all_read, request: pg_notify(24.b6(32) epoch 194477) v5 -2518> 2014-08-13 17:23:21.690882 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.690862, event: dispatched, request: pg_notify(24.b6(32) epoch 194477) v5 -2517> 2014-08-13 17:23:21.690888 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.690888, event: waiting_for_osdmap, request: pg_notify(24.b6(32) epoch 194477) v5 -2516> 2014-08-13 17:23:21.690895 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.690895, event: started, request: pg_notify(24.b6(32) epoch 194477) v5 -2515> 2014-08-13 17:23:21.690914 7fcd0964c700 5 -- op tracker -- , seq: 392, time: 2014-08-13 17:23:21.690914, event: done, request: pg_notify(24.b6(32) epoch 194477) v5 -2514> 2014-08-13 17:23:21.690919 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194477:24.5e) v4 -- ?+0 0x54d3b80 -2513> 2014-08-13 17:23:21.690948 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 5 ==== pg_notify(24.3d(27) epoch 194477) v5 ==== 2265+0+0 (3804825511 0 0) 0x54d2bc0 con 0x4f86040 -2512> 2014-08-13 17:23:21.690964 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.676356, event: header_read, request: pg_notify(24.3d(27) epoch 194477) v5 -2511> 2014-08-13 17:23:21.690970 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.676357, event: throttled, request: pg_notify(24.3d(27) epoch 194477) v5 -2510> 2014-08-13 17:23:21.690976 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.676495, event: all_read, request: pg_notify(24.3d(27) epoch 194477) v5 -2509> 2014-08-13 17:23:21.690981 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.690961, event: dispatched, request: pg_notify(24.3d(27) epoch 194477) v5 -2508> 2014-08-13 17:23:21.690987 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.690987, event: waiting_for_osdmap, request: pg_notify(24.3d(27) epoch 194477) v5 -2507> 2014-08-13 17:23:21.690995 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.690995, event: started, request: pg_notify(24.3d(27) epoch 194477) v5 -2506> 2014-08-13 17:23:21.691015 7fcd0964c700 5 -- op tracker -- , seq: 393, time: 2014-08-13 17:23:21.691015, event: done, request: pg_notify(24.3d(27) epoch 194477) v5 -2505> 2014-08-13 17:23:21.691043 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x567ba80 con 0x4f853e0 -2504> 2014-08-13 17:23:21.691062 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2503> 2014-08-13 17:23:21.691072 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 1 ==== osd_map(194472..194477 src has 185139..194477) v3 ==== 4367+0+0 (2378190549 0 0) 0x51e0240 con 0x4f87900 -2502> 2014-08-13 17:23:21.691057 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.044470 5 0.006002 -2501> 2014-08-13 17:23:21.691086 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194472,194477], i have 194477, src has [185139,194477] -2500> 2014-08-13 17:23:21.691092 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2499> 2014-08-13 17:23:21.691101 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 8 ==== pg_notify(24.1f7(6) epoch 194477) v5 ==== 1094+0+0 (3116339385 0 0) 0x588a4c0 con 0x4fd9a20 -2498> 2014-08-13 17:23:21.691116 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.673376, event: header_read, request: pg_notify(24.1f7(6) epoch 194477) v5 -2497> 2014-08-13 17:23:21.691124 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.673399, event: throttled, request: pg_notify(24.1f7(6) epoch 194477) v5 -2496> 2014-08-13 17:23:21.691129 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.673627, event: all_read, request: pg_notify(24.1f7(6) epoch 194477) v5 -2495> 2014-08-13 17:23:21.691135 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.691114, event: dispatched, request: pg_notify(24.1f7(6) epoch 194477) v5 -2494> 2014-08-13 17:23:21.691142 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.691141, event: waiting_for_osdmap, request: pg_notify(24.1f7(6) epoch 194477) v5 -2493> 2014-08-13 17:23:21.691150 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.691149, event: started, request: pg_notify(24.1f7(6) epoch 194477) v5 -2492> 2014-08-13 17:23:21.691169 7fcd0964c700 5 -- op tracker -- , seq: 394, time: 2014-08-13 17:23:21.691169, event: done, request: pg_notify(24.1f7(6) epoch 194477) v5 -2491> 2014-08-13 17:23:21.691162 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000070 0 0.000000 -2490> 2014-08-13 17:23:21.691195 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2489> 2014-08-13 17:23:21.691194 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 12 ==== pg_info(1 pgs e194477:24.206) v4 ==== 748+0+0 (766931044 0 0) 0x588eac0 con 0x4f86e00 -2488> 2014-08-13 17:23:21.691220 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.685190, event: header_read, request: pg_info(1 pgs e194477:24.206) v4 -2487> 2014-08-13 17:23:21.691227 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.685192, event: throttled, request: pg_info(1 pgs e194477:24.206) v4 -2486> 2014-08-13 17:23:21.691233 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.685279, event: all_read, request: pg_info(1 pgs e194477:24.206) v4 -2485> 2014-08-13 17:23:21.691239 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.691217, event: dispatched, request: pg_info(1 pgs e194477:24.206) v4 -2484> 2014-08-13 17:23:21.691245 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.691245, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.206) v4 -2483> 2014-08-13 17:23:21.691253 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.691252, event: started, request: pg_info(1 pgs e194477:24.206) v4 -2482> 2014-08-13 17:23:21.691274 7fcd0964c700 5 -- op tracker -- , seq: 395, time: 2014-08-13 17:23:21.691274, event: done, request: pg_info(1 pgs e194477:24.206) v4 -2481> 2014-08-13 17:23:21.691293 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 5 ==== pg_notify(24.30b(30) epoch 194477) v5 ==== 2438+0+0 (1242858415 0 0) 0x5b48fc0 con 0x4fdad60 -2480> 2014-08-13 17:23:21.691308 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.682960, event: header_read, request: pg_notify(24.30b(30) epoch 194477) v5 -2479> 2014-08-13 17:23:21.691315 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.682962, event: throttled, request: pg_notify(24.30b(30) epoch 194477) v5 -2478> 2014-08-13 17:23:21.691320 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.683062, event: all_read, request: pg_notify(24.30b(30) epoch 194477) v5 -2477> 2014-08-13 17:23:21.691326 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.691306, event: dispatched, request: pg_notify(24.30b(30) epoch 194477) v5 -2476> 2014-08-13 17:23:21.691332 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.691332, event: waiting_for_osdmap, request: pg_notify(24.30b(30) epoch 194477) v5 -2475> 2014-08-13 17:23:21.691340 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.691339, event: started, request: pg_notify(24.30b(30) epoch 194477) v5 -2474> 2014-08-13 17:23:21.691344 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_notify(24.3b7(33) epoch 194477) v5 -- ?+0 0x588d080 con 0x4ec4780 -2473> 2014-08-13 17:23:21.691360 7fcd0964c700 5 -- op tracker -- , seq: 396, time: 2014-08-13 17:23:21.691360, event: done, request: pg_notify(24.30b(30) epoch 194477) v5 -2472> 2014-08-13 17:23:21.691395 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 5 ==== pg_notify(22.1b8(22) epoch 194477) v5 ==== 1948+0+0 (841956746 0 0) 0x5b48a80 con 0x4fd1a20 -2471> 2014-08-13 17:23:21.691411 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.683598, event: header_read, request: pg_notify(22.1b8(22) epoch 194477) v5 -2470> 2014-08-13 17:23:21.691418 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.683600, event: throttled, request: pg_notify(22.1b8(22) epoch 194477) v5 -2469> 2014-08-13 17:23:21.691423 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.683721, event: all_read, request: pg_notify(22.1b8(22) epoch 194477) v5 -2468> 2014-08-13 17:23:21.691429 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.691409, event: dispatched, request: pg_notify(22.1b8(22) epoch 194477) v5 -2467> 2014-08-13 17:23:21.691446 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.691446, event: waiting_for_osdmap, request: pg_notify(22.1b8(22) epoch 194477) v5 -2466> 2014-08-13 17:23:21.691454 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.691454, event: started, request: pg_notify(22.1b8(22) epoch 194477) v5 -2465> 2014-08-13 17:23:21.691473 7fcd0964c700 5 -- op tracker -- , seq: 397, time: 2014-08-13 17:23:21.691473, event: done, request: pg_notify(22.1b8(22) epoch 194477) v5 -2464> 2014-08-13 17:23:21.691565 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 5 ==== pg_info(1 pgs e194477:22.69f) v4 ==== 714+0+0 (2463671517 0 0) 0x5888a80 con 0x4fde040 -2463> 2014-08-13 17:23:21.691582 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6824/5505 -- pg_query(22.619 epoch 194477) v3 -- ?+0 0x54fe180 con 0x4e370c0 -2462> 2014-08-13 17:23:21.691615 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.689051, event: header_read, request: pg_info(1 pgs e194477:22.69f) v4 -2461> 2014-08-13 17:23:21.691626 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.689053, event: throttled, request: pg_info(1 pgs e194477:22.69f) v4 -2460> 2014-08-13 17:23:21.691636 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.689233, event: all_read, request: pg_info(1 pgs e194477:22.69f) v4 -2459> 2014-08-13 17:23:21.691645 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.691610, event: dispatched, request: pg_info(1 pgs e194477:22.69f) v4 -2458> 2014-08-13 17:23:21.691656 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.691655, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.69f) v4 -2457> 2014-08-13 17:23:21.691665 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.691665, event: started, request: pg_info(1 pgs e194477:22.69f) v4 -2456> 2014-08-13 17:23:21.691690 7fcd0964c700 5 -- op tracker -- , seq: 398, time: 2014-08-13 17:23:21.691690, event: done, request: pg_info(1 pgs e194477:22.69f) v4 -2455> 2014-08-13 17:23:21.691743 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 10 ==== pg_info(1 pgs e194477:24.3d) v4 ==== 748+0+0 (1541086399 0 0) 0x56ac980 con 0x4fd06e0 -2454> 2014-08-13 17:23:21.691764 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.668897, event: header_read, request: pg_info(1 pgs e194477:24.3d) v4 -2453> 2014-08-13 17:23:21.691751 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_query(22.619 epoch 194477) v3 -- ?+0 0x54fdfa0 con 0x4fd9760 -2452> 2014-08-13 17:23:21.691772 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.668899, event: throttled, request: pg_info(1 pgs e194477:24.3d) v4 -2451> 2014-08-13 17:23:21.691778 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.668990, event: all_read, request: pg_info(1 pgs e194477:24.3d) v4 -2450> 2014-08-13 17:23:21.691784 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.691761, event: dispatched, request: pg_info(1 pgs e194477:24.3d) v4 -2449> 2014-08-13 17:23:21.691798 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.691798, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3d) v4 -2448> 2014-08-13 17:23:21.691811 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.691811, event: started, request: pg_info(1 pgs e194477:24.3d) v4 -2447> 2014-08-13 17:23:21.691834 7fcd0964c700 5 -- op tracker -- , seq: 399, time: 2014-08-13 17:23:21.691833, event: done, request: pg_info(1 pgs e194477:24.3d) v4 -2446> 2014-08-13 17:23:21.691865 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 9 ==== pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 ==== 4366+0+0 (3614162458 0 0) 0x54d5240 con 0x4fd5960 -2445> 2014-08-13 17:23:21.691887 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.674193, event: header_read, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2444> 2014-08-13 17:23:21.691898 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.674195, event: throttled, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2443> 2014-08-13 17:23:21.691908 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.674347, event: all_read, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2442> 2014-08-13 17:23:21.691918 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.691884, event: dispatched, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2441> 2014-08-13 17:23:21.691928 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.691928, event: waiting_for_osdmap, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2440> 2014-08-13 17:23:21.691940 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.691940, event: started, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2439> 2014-08-13 17:23:21.691980 7fcd0964c700 5 -- op tracker -- , seq: 400, time: 2014-08-13 17:23:21.691980, event: done, request: pg_notify(22.48e(24),24.1f7(28) epoch 194477) v5 -2438> 2014-08-13 17:23:21.692035 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 6 ==== pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 ==== 2996+0+0 (462451155 0 0) 0x58c9200 con 0x5303440 -2437> 2014-08-13 17:23:21.692077 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.687384, event: header_read, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2436> 2014-08-13 17:23:21.692105 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.687387, event: throttled, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2435> 2014-08-13 17:23:21.692128 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.687513, event: all_read, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2434> 2014-08-13 17:23:21.692149 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.692074, event: dispatched, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2433> 2014-08-13 17:23:21.692140 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.740231 7 0.000286 -2432> 2014-08-13 17:23:21.692172 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.692172, event: waiting_for_osdmap, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2431> 2014-08-13 17:23:21.692183 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -2430> 2014-08-13 17:23:21.692232 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194471 n=29 ec=183096 les/c 194474/194474 194476/194476/192978) [373,360,4] r=2 lpr=194476 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -2429> 2014-08-13 17:23:21.692286 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.041444 5 0.000194 -2428> 2014-08-13 17:23:21.692325 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.692324, event: started, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2427> 2014-08-13 17:23:21.692331 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2426> 2014-08-13 17:23:21.692356 7fcd0964c700 5 -- op tracker -- , seq: 401, time: 2014-08-13 17:23:21.692356, event: done, request: pg_log(22.707 epoch 194477 log log((194215'483,194215'483], crt=194215'483) query_epoch 194477) v4 -2425> 2014-08-13 17:23:21.692393 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 10 ==== pg_query(24.3f7 epoch 194477) v3 ==== 144+0+0 (2775315184 0 0) 0x54f8780 con 0x5305ee0 -2424> 2014-08-13 17:23:21.692404 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000073 0 0.000000 -2423> 2014-08-13 17:23:21.692416 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.662570, event: header_read, request: pg_query(24.3f7 epoch 194477) v3 -2422> 2014-08-13 17:23:21.692424 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.662574, event: throttled, request: pg_query(24.3f7 epoch 194477) v3 -2421> 2014-08-13 17:23:21.692430 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.662652, event: all_read, request: pg_query(24.3f7 epoch 194477) v3 -2420> 2014-08-13 17:23:21.692435 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.692414, event: dispatched, request: pg_query(24.3f7 epoch 194477) v3 -2419> 2014-08-13 17:23:21.692428 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2418> 2014-08-13 17:23:21.692442 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.692442, event: waiting_for_osdmap, request: pg_query(24.3f7 epoch 194477) v3 -2417> 2014-08-13 17:23:21.692450 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.692450, event: started, request: pg_query(24.3f7 epoch 194477) v3 -2416> 2014-08-13 17:23:21.692469 7fcd0964c700 5 -- op tracker -- , seq: 402, time: 2014-08-13 17:23:21.692469, event: done, request: pg_query(24.3f7 epoch 194477) v3 -2415> 2014-08-13 17:23:21.692493 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.350 192.168.1.114:6832/53185 6 ==== pg_info(1 pgs e194477:24.9b) v4 ==== 748+0+0 (956953779 0 0) 0x54d0fc0 con 0x53056a0 -2414> 2014-08-13 17:23:21.692509 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.673601, event: header_read, request: pg_info(1 pgs e194477:24.9b) v4 -2413> 2014-08-13 17:23:21.692516 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.673603, event: throttled, request: pg_info(1 pgs e194477:24.9b) v4 -2412> 2014-08-13 17:23:21.692522 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.673666, event: all_read, request: pg_info(1 pgs e194477:24.9b) v4 -2411> 2014-08-13 17:23:21.692527 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.692507, event: dispatched, request: pg_info(1 pgs e194477:24.9b) v4 -2410> 2014-08-13 17:23:21.692534 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.692534, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.9b) v4 -2409> 2014-08-13 17:23:21.692542 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.692542, event: started, request: pg_info(1 pgs e194477:24.9b) v4 -2408> 2014-08-13 17:23:21.692525 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.022639 9 0.000570 -2407> 2014-08-13 17:23:21.692563 7fcd0964c700 5 -- op tracker -- , seq: 403, time: 2014-08-13 17:23:21.692562, event: done, request: pg_info(1 pgs e194477:24.9b) v4 -2406> 2014-08-13 17:23:21.692553 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2405> 2014-08-13 17:23:21.692554 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(26) epoch 194477) v5 -- ?+0 0x588c280 con 0x4fd5960 -2404> 2014-08-13 17:23:21.692587 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.365 192.168.1.102:6824/15623 7 ==== pg_notify(22.481(34) epoch 194477) v5 ==== 2628+0+0 (2041133625 0 0) 0x54d7c40 con 0x53053e0 -2403> 2014-08-13 17:23:21.692603 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.674192, event: header_read, request: pg_notify(22.481(34) epoch 194477) v5 -2402> 2014-08-13 17:23:21.692610 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.674194, event: throttled, request: pg_notify(22.481(34) epoch 194477) v5 -2401> 2014-08-13 17:23:21.692616 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.674321, event: all_read, request: pg_notify(22.481(34) epoch 194477) v5 -2400> 2014-08-13 17:23:21.692622 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.692601, event: dispatched, request: pg_notify(22.481(34) epoch 194477) v5 -2399> 2014-08-13 17:23:21.692628 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.692628, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194477) v5 -2398> 2014-08-13 17:23:21.692616 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000063 0 0.000000 -2397> 2014-08-13 17:23:21.692637 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.692636, event: started, request: pg_notify(22.481(34) epoch 194477) v5 -2396> 2014-08-13 17:23:21.692639 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2395> 2014-08-13 17:23:21.692658 7fcd0964c700 5 -- op tracker -- , seq: 404, time: 2014-08-13 17:23:21.692658, event: done, request: pg_notify(22.481(34) epoch 194477) v5 -2394> 2014-08-13 17:23:21.692688 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.312 192.168.1.113:6816/17521 6 ==== pg_info(1 pgs e194477:22.f3) v4 ==== 714+0+0 (3629016234 0 0) 0x54d4d00 con 0x5307a60 -2393> 2014-08-13 17:23:21.692704 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.678316, event: header_read, request: pg_info(1 pgs e194477:22.f3) v4 -2392> 2014-08-13 17:23:21.692711 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.678322, event: throttled, request: pg_info(1 pgs e194477:22.f3) v4 -2391> 2014-08-13 17:23:21.692716 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.678402, event: all_read, request: pg_info(1 pgs e194477:22.f3) v4 -2390> 2014-08-13 17:23:21.692722 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.692702, event: dispatched, request: pg_info(1 pgs e194477:22.f3) v4 -2389> 2014-08-13 17:23:21.692728 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.692728, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.f3) v4 -2388> 2014-08-13 17:23:21.692736 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.692735, event: started, request: pg_info(1 pgs e194477:22.f3) v4 -2387> 2014-08-13 17:23:21.692755 7fcd0964c700 5 -- op tracker -- , seq: 405, time: 2014-08-13 17:23:21.692754, event: done, request: pg_info(1 pgs e194477:22.f3) v4 -2386> 2014-08-13 17:23:21.692778 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.25 192.168.1.103:6800/36920 6 ==== pg_info(1 pgs e194477:24.3d) v4 ==== 748+0+0 (4111432871 0 0) 0x54d78c0 con 0x552f7a0 -2385> 2014-08-13 17:23:21.692792 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.675596, event: header_read, request: pg_info(1 pgs e194477:24.3d) v4 -2384> 2014-08-13 17:23:21.692799 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.675598, event: throttled, request: pg_info(1 pgs e194477:24.3d) v4 -2383> 2014-08-13 17:23:21.692804 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.675697, event: all_read, request: pg_info(1 pgs e194477:24.3d) v4 -2382> 2014-08-13 17:23:21.692809 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.692790, event: dispatched, request: pg_info(1 pgs e194477:24.3d) v4 -2381> 2014-08-13 17:23:21.692815 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.692815, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3d) v4 -2380> 2014-08-13 17:23:21.692823 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.692822, event: started, request: pg_info(1 pgs e194477:24.3d) v4 -2379> 2014-08-13 17:23:21.692843 7fcd0964c700 5 -- op tracker -- , seq: 406, time: 2014-08-13 17:23:21.692843, event: done, request: pg_info(1 pgs e194477:24.3d) v4 -2378> 2014-08-13 17:23:21.692867 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 4 ==== pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 ==== 4499+0+0 (883769128 0 0) 0x58c9e00 con 0x55f2100 -2377> 2014-08-13 17:23:21.692884 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.691009, event: header_read, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2376> 2014-08-13 17:23:21.692892 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.691011, event: throttled, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2375> 2014-08-13 17:23:21.692899 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.691153, event: all_read, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2374> 2014-08-13 17:23:21.692906 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.692881, event: dispatched, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2373> 2014-08-13 17:23:21.692914 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.692914, event: waiting_for_osdmap, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2372> 2014-08-13 17:23:21.692924 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.692923, event: started, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2371> 2014-08-13 17:23:21.692939 7fcd0964c700 5 -- op tracker -- , seq: 407, time: 2014-08-13 17:23:21.692939, event: done, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2370> 2014-08-13 17:23:21.692963 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x567a400 con 0x552d540 -2369> 2014-08-13 17:23:21.692982 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2368> 2014-08-13 17:23:21.692992 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.307 192.168.1.109:6804/31016 2 ==== pg_notify(24.181(35) epoch 194477) v5 ==== 2717+0+0 (1371071092 0 0) 0x588f000 con 0x56b4fc0 -2367> 2014-08-13 17:23:21.693006 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.684037, event: header_read, request: pg_notify(24.181(35) epoch 194477) v5 -2366> 2014-08-13 17:23:21.693012 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.684039, event: throttled, request: pg_notify(24.181(35) epoch 194477) v5 -2365> 2014-08-13 17:23:21.693018 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.684157, event: all_read, request: pg_notify(24.181(35) epoch 194477) v5 -2364> 2014-08-13 17:23:21.693023 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.693003, event: dispatched, request: pg_notify(24.181(35) epoch 194477) v5 -2363> 2014-08-13 17:23:21.693046 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.693046, event: waiting_for_osdmap, request: pg_notify(24.181(35) epoch 194477) v5 -2362> 2014-08-13 17:23:21.693054 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.693054, event: started, request: pg_notify(24.181(35) epoch 194477) v5 -2361> 2014-08-13 17:23:21.693075 7fcd0964c700 5 -- op tracker -- , seq: 408, time: 2014-08-13 17:23:21.693075, event: done, request: pg_notify(24.181(35) epoch 194477) v5 -2360> 2014-08-13 17:23:21.693111 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.59 192.168.1.112:6822/5265 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x51e7980 con 0x53e65c0 -2359> 2014-08-13 17:23:21.693129 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -2358> 2014-08-13 17:23:21.693138 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.348 192.168.1.110:6800/41204 2 ==== pg_notify(24.1bb(27) epoch 194477) v5 ==== 2261+0+0 (3277685512 0 0) 0x588a300 con 0x53e4fc0 -2357> 2014-08-13 17:23:21.693153 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.686955, event: header_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2356> 2014-08-13 17:23:21.693160 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.686956, event: throttled, request: pg_notify(24.1bb(27) epoch 194477) v5 -2355> 2014-08-13 17:23:21.693166 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.687056, event: all_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2354> 2014-08-13 17:23:21.693171 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.693150, event: dispatched, request: pg_notify(24.1bb(27) epoch 194477) v5 -2353> 2014-08-13 17:23:21.693177 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.693177, event: waiting_for_osdmap, request: pg_notify(24.1bb(27) epoch 194477) v5 -2352> 2014-08-13 17:23:21.693185 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.693185, event: started, request: pg_notify(24.1bb(27) epoch 194477) v5 -2351> 2014-08-13 17:23:21.693226 7fcd0964c700 5 -- op tracker -- , seq: 409, time: 2014-08-13 17:23:21.693226, event: done, request: pg_notify(24.1bb(27) epoch 194477) v5 -2350> 2014-08-13 17:23:21.693257 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 7 ==== pg_info(1 pgs e194477:22.116) v4 ==== 714+0+0 (1591787633 0 0) 0x588fe00 con 0x4e32940 -2349> 2014-08-13 17:23:21.693274 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.689489, event: header_read, request: pg_info(1 pgs e194477:22.116) v4 -2348> 2014-08-13 17:23:21.693281 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.689490, event: throttled, request: pg_info(1 pgs e194477:22.116) v4 -2347> 2014-08-13 17:23:21.693287 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.689571, event: all_read, request: pg_info(1 pgs e194477:22.116) v4 -2346> 2014-08-13 17:23:21.693292 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.693272, event: dispatched, request: pg_info(1 pgs e194477:22.116) v4 -2345> 2014-08-13 17:23:21.693298 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.693298, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.116) v4 -2344> 2014-08-13 17:23:21.693306 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.693306, event: started, request: pg_info(1 pgs e194477:22.116) v4 -2343> 2014-08-13 17:23:21.693325 7fcd0964c700 5 -- op tracker -- , seq: 410, time: 2014-08-13 17:23:21.693325, event: done, request: pg_info(1 pgs e194477:22.116) v4 -2342> 2014-08-13 17:23:21.693348 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.321 192.168.1.110:6812/43236 6 ==== pg_info(1 pgs e194477:22.4aa) v4 ==== 714+0+0 (3649246169 0 0) 0x54d0a80 con 0x4e35120 -2341> 2014-08-13 17:23:21.693363 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.682102, event: header_read, request: pg_info(1 pgs e194477:22.4aa) v4 -2340> 2014-08-13 17:23:21.693369 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.682103, event: throttled, request: pg_info(1 pgs e194477:22.4aa) v4 -2339> 2014-08-13 17:23:21.693375 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.682180, event: all_read, request: pg_info(1 pgs e194477:22.4aa) v4 -2338> 2014-08-13 17:23:21.693380 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.693361, event: dispatched, request: pg_info(1 pgs e194477:22.4aa) v4 -2337> 2014-08-13 17:23:21.693386 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.693386, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4aa) v4 -2336> 2014-08-13 17:23:21.693394 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.693394, event: started, request: pg_info(1 pgs e194477:22.4aa) v4 -2335> 2014-08-13 17:23:21.693412 7fcd0964c700 5 -- op tracker -- , seq: 411, time: 2014-08-13 17:23:21.693412, event: done, request: pg_info(1 pgs e194477:22.4aa) v4 -2334> 2014-08-13 17:23:21.693435 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 10 ==== pg_notify(24.136(27) epoch 194477) v5 ==== 2261+0+0 (59036701 0 0) 0x588e740 con 0x4e35ac0 -2333> 2014-08-13 17:23:21.693451 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.660648, event: header_read, request: pg_notify(24.136(27) epoch 194477) v5 -2332> 2014-08-13 17:23:21.693457 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.660650, event: throttled, request: pg_notify(24.136(27) epoch 194477) v5 -2331> 2014-08-13 17:23:21.693463 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.660775, event: all_read, request: pg_notify(24.136(27) epoch 194477) v5 -2330> 2014-08-13 17:23:21.693468 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.693449, event: dispatched, request: pg_notify(24.136(27) epoch 194477) v5 -2329> 2014-08-13 17:23:21.693474 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.693474, event: waiting_for_osdmap, request: pg_notify(24.136(27) epoch 194477) v5 -2328> 2014-08-13 17:23:21.693482 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.693482, event: started, request: pg_notify(24.136(27) epoch 194477) v5 -2327> 2014-08-13 17:23:21.693503 7fcd0964c700 5 -- op tracker -- , seq: 412, time: 2014-08-13 17:23:21.693502, event: done, request: pg_notify(24.136(27) epoch 194477) v5 -2326> 2014-08-13 17:23:21.693525 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 8 ==== pg_notify(22.1b8(1) epoch 194477) v5 ==== 769+0+0 (1073981356 0 0) 0x56afc40 con 0x4e35540 -2325> 2014-08-13 17:23:21.693540 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.677130, event: header_read, request: pg_notify(22.1b8(1) epoch 194477) v5 -2324> 2014-08-13 17:23:21.693547 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.677131, event: throttled, request: pg_notify(22.1b8(1) epoch 194477) v5 -2323> 2014-08-13 17:23:21.693552 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.677254, event: all_read, request: pg_notify(22.1b8(1) epoch 194477) v5 -2322> 2014-08-13 17:23:21.693558 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.693538, event: dispatched, request: pg_notify(22.1b8(1) epoch 194477) v5 -2321> 2014-08-13 17:23:21.693564 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.693563, event: waiting_for_osdmap, request: pg_notify(22.1b8(1) epoch 194477) v5 -2320> 2014-08-13 17:23:21.693571 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.693571, event: started, request: pg_notify(22.1b8(1) epoch 194477) v5 -2319> 2014-08-13 17:23:21.693589 7fcd0964c700 5 -- op tracker -- , seq: 413, time: 2014-08-13 17:23:21.693589, event: done, request: pg_notify(22.1b8(1) epoch 194477) v5 -2318> 2014-08-13 17:23:21.693614 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 4 ==== pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 ==== 4171+0+0 (1909078766 0 0) 0x58cb000 con 0x4e370c0 -2317> 2014-08-13 17:23:21.693631 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.692548, event: header_read, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2316> 2014-08-13 17:23:21.693639 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.692550, event: throttled, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2315> 2014-08-13 17:23:21.693647 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.692750, event: all_read, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2314> 2014-08-13 17:23:21.693654 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.693629, event: dispatched, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2313> 2014-08-13 17:23:21.693662 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.693662, event: waiting_for_osdmap, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2312> 2014-08-13 17:23:21.693671 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.693671, event: started, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2311> 2014-08-13 17:23:21.693692 7fcd0964c700 5 -- op tracker -- , seq: 414, time: 2014-08-13 17:23:21.693692, event: done, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2310> 2014-08-13 17:23:21.693720 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.54 192.168.1.111:6836/14923 7 ==== pg_info(1 pgs e194477:22.30f) v4 ==== 714+0+0 (2607999189 0 0) 0x588fa80 con 0x4ec2aa0 -2309> 2014-08-13 17:23:21.693735 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.688527, event: header_read, request: pg_info(1 pgs e194477:22.30f) v4 -2308> 2014-08-13 17:23:21.693741 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.688529, event: throttled, request: pg_info(1 pgs e194477:22.30f) v4 -2307> 2014-08-13 17:23:21.693747 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.688618, event: all_read, request: pg_info(1 pgs e194477:22.30f) v4 -2306> 2014-08-13 17:23:21.693752 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.693733, event: dispatched, request: pg_info(1 pgs e194477:22.30f) v4 -2305> 2014-08-13 17:23:21.693758 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.693758, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.30f) v4 -2304> 2014-08-13 17:23:21.693766 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.693766, event: started, request: pg_info(1 pgs e194477:22.30f) v4 -2303> 2014-08-13 17:23:21.693794 7fcd0964c700 5 -- op tracker -- , seq: 415, time: 2014-08-13 17:23:21.693794, event: done, request: pg_info(1 pgs e194477:22.30f) v4 -2302> 2014-08-13 17:23:21.693823 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.304 192.168.1.101:6808/22782 10 ==== pg_info(1 pgs e194477:24.33a) v4 ==== 748+0+0 (2199503681 0 0) 0x56a96c0 con 0x4ec1ce0 -2301> 2014-08-13 17:23:21.693843 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.691377, event: header_read, request: pg_info(1 pgs e194477:24.33a) v4 -2300> 2014-08-13 17:23:21.693850 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.691378, event: throttled, request: pg_info(1 pgs e194477:24.33a) v4 -2299> 2014-08-13 17:23:21.693856 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.691483, event: all_read, request: pg_info(1 pgs e194477:24.33a) v4 -2298> 2014-08-13 17:23:21.693862 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.693841, event: dispatched, request: pg_info(1 pgs e194477:24.33a) v4 -2297> 2014-08-13 17:23:21.693891 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.693890, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.33a) v4 -2296> 2014-08-13 17:23:21.693900 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.693900, event: started, request: pg_info(1 pgs e194477:24.33a) v4 -2295> 2014-08-13 17:23:21.693922 7fcd0964c700 5 -- op tracker -- , seq: 416, time: 2014-08-13 17:23:21.693921, event: done, request: pg_info(1 pgs e194477:24.33a) v4 -2294> 2014-08-13 17:23:21.693949 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 5 ==== pg_notify(24.9b(29) epoch 194477) v5 ==== 2383+0+0 (3784131778 0 0) 0x56ae900 con 0x4ec27e0 -2293> 2014-08-13 17:23:21.693971 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.683192, event: header_read, request: pg_notify(24.9b(29) epoch 194477) v5 -2292> 2014-08-13 17:23:21.693982 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.683193, event: throttled, request: pg_notify(24.9b(29) epoch 194477) v5 -2291> 2014-08-13 17:23:21.693991 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.683292, event: all_read, request: pg_notify(24.9b(29) epoch 194477) v5 -2290> 2014-08-13 17:23:21.694000 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.693969, event: dispatched, request: pg_notify(24.9b(29) epoch 194477) v5 -2289> 2014-08-13 17:23:21.694007 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.694007, event: waiting_for_osdmap, request: pg_notify(24.9b(29) epoch 194477) v5 -2288> 2014-08-13 17:23:21.694018 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.694018, event: started, request: pg_notify(24.9b(29) epoch 194477) v5 -2287> 2014-08-13 17:23:21.694041 7fcd0964c700 5 -- op tracker -- , seq: 417, time: 2014-08-13 17:23:21.694041, event: done, request: pg_notify(24.9b(29) epoch 194477) v5 -2286> 2014-08-13 17:23:21.694081 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 7 ==== pg_info(1 pgs e194477:22.791) v4 ==== 714+0+0 (1121292605 0 0) 0x56ad940 con 0x4ec3f40 -2285> 2014-08-13 17:23:21.694103 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.691216, event: header_read, request: pg_info(1 pgs e194477:22.791) v4 -2284> 2014-08-13 17:23:21.694110 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.691218, event: throttled, request: pg_info(1 pgs e194477:22.791) v4 -2283> 2014-08-13 17:23:21.694116 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.691298, event: all_read, request: pg_info(1 pgs e194477:22.791) v4 -2282> 2014-08-13 17:23:21.694121 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.694100, event: dispatched, request: pg_info(1 pgs e194477:22.791) v4 -2281> 2014-08-13 17:23:21.694127 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.694127, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.791) v4 -2280> 2014-08-13 17:23:21.694135 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.694135, event: started, request: pg_info(1 pgs e194477:22.791) v4 -2279> 2014-08-13 17:23:21.694121 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194477:24.2ec) v4 -- ?+0 0x54d5400 -2278> 2014-08-13 17:23:21.694204 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194477:22.1fb) v4 -- ?+0 0x54d0700 -2277> 2014-08-13 17:23:21.694180 7fcd0964c700 5 -- op tracker -- , seq: 418, time: 2014-08-13 17:23:21.694180, event: done, request: pg_info(1 pgs e194477:22.791) v4 -2276> 2014-08-13 17:23:21.694371 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6826/11446 -- pg_query(22.775 epoch 194477) v3 -- ?+0 0x2f91680 con 0x4eb48e0 -2275> 2014-08-13 17:23:21.694405 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6836/6227 -- pg_query(22.775 epoch 194477) v3 -- ?+0 0x54fdfa0 con 0x53011e0 -2274> 2014-08-13 17:23:21.694423 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_query(22.3d9 epoch 194477) v3 -- ?+0 0x54fb660 con 0x4fd5960 -2273> 2014-08-13 17:23:21.694453 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 8 ==== pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 ==== 4150+0+0 (588011005 0 0) 0x58caa00 con 0x4ec56a0 -2272> 2014-08-13 17:23:21.694498 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.691594, event: header_read, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2271> 2014-08-13 17:23:21.694509 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.691596, event: throttled, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2270> 2014-08-13 17:23:21.694517 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.691779, event: all_read, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2269> 2014-08-13 17:23:21.694529 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.694494, event: dispatched, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2268> 2014-08-13 17:23:21.694537 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.694537, event: waiting_for_osdmap, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2267> 2014-08-13 17:23:21.694548 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.694548, event: started, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2266> 2014-08-13 17:23:21.694567 7fcd0964c700 5 -- op tracker -- , seq: 419, time: 2014-08-13 17:23:21.694567, event: done, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2265> 2014-08-13 17:23:21.694591 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 14 ==== pg_info(1 pgs e194477:24.5e) v4 ==== 748+0+0 (3374637135 0 0) 0x54d3b80 con 0x4ec3440 -2264> 2014-08-13 17:23:21.694615 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.691682, event: header_read, request: pg_info(1 pgs e194477:24.5e) v4 -2263> 2014-08-13 17:23:21.694628 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.691684, event: throttled, request: pg_info(1 pgs e194477:24.5e) v4 -2262> 2014-08-13 17:23:21.694673 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.691760, event: all_read, request: pg_info(1 pgs e194477:24.5e) v4 -2261> 2014-08-13 17:23:21.694679 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.694612, event: dispatched, request: pg_info(1 pgs e194477:24.5e) v4 -2260> 2014-08-13 17:23:21.694686 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.694686, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.5e) v4 -2259> 2014-08-13 17:23:21.694694 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.694693, event: started, request: pg_info(1 pgs e194477:24.5e) v4 -2258> 2014-08-13 17:23:21.694715 7fcd0964c700 5 -- op tracker -- , seq: 420, time: 2014-08-13 17:23:21.694715, event: done, request: pg_info(1 pgs e194477:24.5e) v4 -2257> 2014-08-13 17:23:21.694742 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 13 ==== pg_query(24.3b7 epoch 194477) v3 ==== 144+0+0 (2556083031 0 0) 0x54fc920 con 0x4ec4780 -2256> 2014-08-13 17:23:21.694758 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.673357, event: header_read, request: pg_query(24.3b7 epoch 194477) v3 -2255> 2014-08-13 17:23:21.694765 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.673359, event: throttled, request: pg_query(24.3b7 epoch 194477) v3 -2254> 2014-08-13 17:23:21.694738 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.041903 7 0.000195 -2253> 2014-08-13 17:23:21.694770 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.673409, event: all_read, request: pg_query(24.3b7 epoch 194477) v3 -2252> 2014-08-13 17:23:21.694776 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.694756, event: dispatched, request: pg_query(24.3b7 epoch 194477) v3 -2251> 2014-08-13 17:23:21.694783 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.694782, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194477) v3 -2250> 2014-08-13 17:23:21.694778 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2249> 2014-08-13 17:23:21.694790 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.694790, event: started, request: pg_query(24.3b7 epoch 194477) v3 -2248> 2014-08-13 17:23:21.694816 7fcd0964c700 5 -- op tracker -- , seq: 421, time: 2014-08-13 17:23:21.694816, event: done, request: pg_query(24.3b7 epoch 194477) v3 -2247> 2014-08-13 17:23:21.694838 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 6 ==== pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 ==== 4499+0+0 (3151600845 0 0) 0x58ca400 con 0x4ec5d80 -2246> 2014-08-13 17:23:21.694855 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.691134, event: header_read, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2245> 2014-08-13 17:23:21.694850 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000072 0 0.000000 -2244> 2014-08-13 17:23:21.694864 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.691136, event: throttled, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2243> 2014-08-13 17:23:21.694872 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.691296, event: all_read, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2242> 2014-08-13 17:23:21.694880 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.694853, event: dispatched, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2241> 2014-08-13 17:23:21.694872 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2240> 2014-08-13 17:23:21.694888 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.694888, event: waiting_for_osdmap, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2239> 2014-08-13 17:23:21.694898 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.694898, event: started, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2238> 2014-08-13 17:23:21.694913 7fcd0964c700 5 -- op tracker -- , seq: 422, time: 2014-08-13 17:23:21.694913, event: done, request: pg_log(22.3e4 epoch 194477 log log((194215'518,194215'518], crt=194215'518) query_epoch 194477) v4 -2237> 2014-08-13 17:23:21.694932 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 4 ==== pg_notify(22.39d(25) epoch 194477) v5 ==== 2117+0+0 (1249656499 0 0) 0x56a8fc0 con 0x4ec70c0 -2236> 2014-08-13 17:23:21.694947 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.669764, event: header_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2235> 2014-08-13 17:23:21.694954 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.669766, event: throttled, request: pg_notify(22.39d(25) epoch 194477) v5 -2234> 2014-08-13 17:23:21.694940 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.055840 7 0.000231 -2233> 2014-08-13 17:23:21.694960 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.669882, event: all_read, request: pg_notify(22.39d(25) epoch 194477) v5 -2232> 2014-08-13 17:23:21.694966 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.694945, event: dispatched, request: pg_notify(22.39d(25) epoch 194477) v5 -2231> 2014-08-13 17:23:21.694973 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.694973, event: waiting_for_osdmap, request: pg_notify(22.39d(25) epoch 194477) v5 -2230> 2014-08-13 17:23:21.694966 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -2229> 2014-08-13 17:23:21.694980 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.694980, event: started, request: pg_notify(22.39d(25) epoch 194477) v5 -2228> 2014-08-13 17:23:21.695001 7fcd0964c700 5 -- op tracker -- , seq: 423, time: 2014-08-13 17:23:21.695001, event: done, request: pg_notify(22.39d(25) epoch 194477) v5 -2227> 2014-08-13 17:23:21.695024 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000058 0 0.000000 -2226> 2014-08-13 17:23:21.695030 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 10 ==== pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 ==== 4150+0+0 (3559791880 0 0) 0x56de600 con 0x4eb18c0 -2225> 2014-08-13 17:23:21.695047 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.691850, event: header_read, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2224> 2014-08-13 17:23:21.695044 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -2223> 2014-08-13 17:23:21.695056 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.691852, event: throttled, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2222> 2014-08-13 17:23:21.695064 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.691994, event: all_read, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2221> 2014-08-13 17:23:21.695071 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.695045, event: dispatched, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2220> 2014-08-13 17:23:21.695079 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.695079, event: waiting_for_osdmap, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2219> 2014-08-13 17:23:21.695088 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.695088, event: started, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2218> 2014-08-13 17:23:21.695102 7fcd0964c700 5 -- op tracker -- , seq: 424, time: 2014-08-13 17:23:21.695102, event: done, request: pg_log(22.99 epoch 194477 log log((194215'504,194215'504], crt=194215'504) query_epoch 194477) v4 -2217> 2014-08-13 17:23:21.695136 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.315 192.168.1.113:6800/14161 3 ==== pg_notify(24.b6(32) epoch 194477) v5 ==== 2556+0+0 (25117777 0 0) 0x56a88c0 con 0x4eb6460 -2216> 2014-08-13 17:23:21.695151 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.682787, event: header_read, request: pg_notify(24.b6(32) epoch 194477) v5 -2215> 2014-08-13 17:23:21.695158 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.682789, event: throttled, request: pg_notify(24.b6(32) epoch 194477) v5 -2214> 2014-08-13 17:23:21.695164 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.682921, event: all_read, request: pg_notify(24.b6(32) epoch 194477) v5 -2213> 2014-08-13 17:23:21.695169 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.695149, event: dispatched, request: pg_notify(24.b6(32) epoch 194477) v5 -2212> 2014-08-13 17:23:21.695220 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_log(24.20e epoch 194477 log log((194422'11806,194422'11806], crt=194422'11806) query_epoch 194477) v4 -- ?+0 0x58c8000 con 0x4f80b00 -2211> 2014-08-13 17:23:21.695527 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- pg_query(22.1f1 epoch 194477) v3 -- ?+0 0x54faee0 con 0x4eb2680 -2210> 2014-08-13 17:23:21.695559 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6828/12102 -- pg_query(22.1f1 epoch 194477) v3 -- ?+0 0x2f95640 con 0x57232e0 -2209> 2014-08-13 17:23:21.695578 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_query(22.362 epoch 194477) v3 -- ?+0 0x2f92580 con 0x4e335a0 -2208> 2014-08-13 17:23:21.695597 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_query(22.362 epoch 194477) v3 -- ?+0 0x2f905a0 con 0x4f86e00 -2207> 2014-08-13 17:23:21.695801 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.695800, event: waiting_for_osdmap, request: pg_notify(24.b6(32) epoch 194477) v5 -2206> 2014-08-13 17:23:21.695824 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.695824, event: started, request: pg_notify(24.b6(32) epoch 194477) v5 -2205> 2014-08-13 17:23:21.695874 7fcd0964c700 5 -- op tracker -- , seq: 425, time: 2014-08-13 17:23:21.695873, event: done, request: pg_notify(24.b6(32) epoch 194477) v5 -2204> 2014-08-13 17:23:21.695921 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 6 ==== pg_notify(22.362(26) epoch 194477) v5 ==== 2172+0+0 (2886206003 0 0) 0x54d2840 con 0x4f86040 -2203> 2014-08-13 17:23:21.695941 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.676524, event: header_read, request: pg_notify(22.362(26) epoch 194477) v5 -2202> 2014-08-13 17:23:21.695949 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.676526, event: throttled, request: pg_notify(22.362(26) epoch 194477) v5 -2201> 2014-08-13 17:23:21.695954 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.676627, event: all_read, request: pg_notify(22.362(26) epoch 194477) v5 -2200> 2014-08-13 17:23:21.695960 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.695938, event: dispatched, request: pg_notify(22.362(26) epoch 194477) v5 -2199> 2014-08-13 17:23:21.695966 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.695966, event: waiting_for_osdmap, request: pg_notify(22.362(26) epoch 194477) v5 -2198> 2014-08-13 17:23:21.695974 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.695974, event: started, request: pg_notify(22.362(26) epoch 194477) v5 -2197> 2014-08-13 17:23:21.695996 7fcd0964c700 5 -- op tracker -- , seq: 426, time: 2014-08-13 17:23:21.695995, event: done, request: pg_notify(22.362(26) epoch 194477) v5 -2196> 2014-08-13 17:23:21.696016 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 5 ==== pg_notify(22.481(41) epoch 194477) v5 ==== 3013+0+0 (985792803 0 0) 0x56a8000 con 0x4f853e0 -2195> 2014-08-13 17:23:21.696030 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.689617, event: header_read, request: pg_notify(22.481(41) epoch 194477) v5 -2194> 2014-08-13 17:23:21.696037 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.689618, event: throttled, request: pg_notify(22.481(41) epoch 194477) v5 -2193> 2014-08-13 17:23:21.696043 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.689771, event: all_read, request: pg_notify(22.481(41) epoch 194477) v5 -2192> 2014-08-13 17:23:21.696048 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.696028, event: dispatched, request: pg_notify(22.481(41) epoch 194477) v5 -2191> 2014-08-13 17:23:21.696054 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.696054, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194477) v5 -2190> 2014-08-13 17:23:21.696062 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.696062, event: started, request: pg_notify(22.481(41) epoch 194477) v5 -2189> 2014-08-13 17:23:21.696048 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.301 192.168.1.113:6814/16727 -- pg_info(1 pgs e194477:22.22c) v4 -- ?+0 0x56a8380 -2188> 2014-08-13 17:23:21.696081 7fcd0964c700 5 -- op tracker -- , seq: 427, time: 2014-08-13 17:23:21.696080, event: done, request: pg_notify(22.481(41) epoch 194477) v5 -2187> 2014-08-13 17:23:21.696091 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.324 192.168.1.104:6814/65209 -- pg_info(1 pgs e194477:22.22c) v4 -- ?+0 0x588c0c0 -2186> 2014-08-13 17:23:21.696129 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 2 ==== pg_notify(22.48e(25) epoch 194477) v5 ==== 2117+0+0 (952916012 0 0) 0x4e501c0 con 0x4f87900 -2185> 2014-08-13 17:23:21.696146 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.685900, event: header_read, request: pg_notify(22.48e(25) epoch 194477) v5 -2184> 2014-08-13 17:23:21.696153 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.685902, event: throttled, request: pg_notify(22.48e(25) epoch 194477) v5 -2183> 2014-08-13 17:23:21.696158 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.686022, event: all_read, request: pg_notify(22.48e(25) epoch 194477) v5 -2182> 2014-08-13 17:23:21.696163 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.696144, event: dispatched, request: pg_notify(22.48e(25) epoch 194477) v5 -2181> 2014-08-13 17:23:21.696169 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.696169, event: waiting_for_osdmap, request: pg_notify(22.48e(25) epoch 194477) v5 -2180> 2014-08-13 17:23:21.696223 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.696223, event: started, request: pg_notify(22.48e(25) epoch 194477) v5 -2179> 2014-08-13 17:23:21.696249 7fcd0964c700 5 -- op tracker -- , seq: 428, time: 2014-08-13 17:23:21.696249, event: done, request: pg_notify(22.48e(25) epoch 194477) v5 -2178> 2014-08-13 17:23:21.696254 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.033288 6 0.000259 -2177> 2014-08-13 17:23:21.696281 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 9 ==== pg_notify(24.136(27) epoch 194477) v5 ==== 2261+0+0 (59036701 0 0) 0x5187540 con 0x4fd9a20 -2176> 2014-08-13 17:23:21.696287 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog -2175> 2014-08-13 17:23:21.696299 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.673717, event: header_read, request: pg_notify(24.136(27) epoch 194477) v5 -2174> 2014-08-13 17:23:21.696309 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.673718, event: throttled, request: pg_notify(24.136(27) epoch 194477) v5 -2173> 2014-08-13 17:23:21.696315 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.673829, event: all_read, request: pg_notify(24.136(27) epoch 194477) v5 -2172> 2014-08-13 17:23:21.696321 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.696296, event: dispatched, request: pg_notify(24.136(27) epoch 194477) v5 -2171> 2014-08-13 17:23:21.696328 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.696328, event: waiting_for_osdmap, request: pg_notify(24.136(27) epoch 194477) v5 -2170> 2014-08-13 17:23:21.696336 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.696336, event: started, request: pg_notify(24.136(27) epoch 194477) v5 -2169> 2014-08-13 17:23:21.696362 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000075 0 0.000000 -2168> 2014-08-13 17:23:21.696385 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing -2167> 2014-08-13 17:23:21.696420 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000036 0 0.000000 -2166> 2014-08-13 17:23:21.696445 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru -2165> 2014-08-13 17:23:21.696468 7fcd0964c700 5 -- op tracker -- , seq: 429, time: 2014-08-13 17:23:21.696357, event: done, request: pg_notify(24.136(27) epoch 194477) v5 -2164> 2014-08-13 17:23:21.696509 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 5 ==== pg_info(1 pgs e194477:22.3ad) v4 ==== 714+0+0 (756878566 0 0) 0x56a9dc0 con 0x4f870c0 -2163> 2014-08-13 17:23:21.696528 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.691361, event: header_read, request: pg_info(1 pgs e194477:22.3ad) v4 -2162> 2014-08-13 17:23:21.696535 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.691362, event: throttled, request: pg_info(1 pgs e194477:22.3ad) v4 -2161> 2014-08-13 17:23:21.696540 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.691437, event: all_read, request: pg_info(1 pgs e194477:22.3ad) v4 -2160> 2014-08-13 17:23:21.696546 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.696525, event: dispatched, request: pg_info(1 pgs e194477:22.3ad) v4 -2159> 2014-08-13 17:23:21.696552 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.696552, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3ad) v4 -2158> 2014-08-13 17:23:21.696561 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.696561, event: started, request: pg_info(1 pgs e194477:22.3ad) v4 -2157> 2014-08-13 17:23:21.696583 7fcd0964c700 5 -- op tracker -- , seq: 430, time: 2014-08-13 17:23:21.696582, event: done, request: pg_info(1 pgs e194477:22.3ad) v4 -2156> 2014-08-13 17:23:21.696589 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194477:22.6bd) v4 -- ?+0 0x5889dc0 -2155> 2014-08-13 17:23:21.696617 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194477:22.6bd) v4 -- ?+0 0x588f1c0 -2154> 2014-08-13 17:23:21.696616 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 8 ==== pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 ==== 4171+0+0 (735328068 0 0) 0x58cb600 con 0x4fd9760 -2153> 2014-08-13 17:23:21.696635 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.693265, event: header_read, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2152> 2014-08-13 17:23:21.696649 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.693268, event: throttled, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2151> 2014-08-13 17:23:21.696657 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.693410, event: all_read, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2150> 2014-08-13 17:23:21.696664 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.696633, event: dispatched, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2149> 2014-08-13 17:23:21.696673 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.696673, event: waiting_for_osdmap, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2148> 2014-08-13 17:23:21.696683 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.696682, event: started, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2147> 2014-08-13 17:23:21.696766 7fcd0964c700 5 -- op tracker -- , seq: 431, time: 2014-08-13 17:23:21.696766, event: done, request: pg_log(22.619 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2146> 2014-08-13 17:23:21.696800 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 8 ==== pg_info(1 pgs e194477:22.4fe) v4 ==== 714+0+0 (899035732 0 0) 0x56abf00 con 0x4fd8f20 -2145> 2014-08-13 17:23:21.696820 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.694372, event: header_read, request: pg_info(1 pgs e194477:22.4fe) v4 -2144> 2014-08-13 17:23:21.696937 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.694378, event: throttled, request: pg_info(1 pgs e194477:22.4fe) v4 -2143> 2014-08-13 17:23:21.696946 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.694450, event: all_read, request: pg_info(1 pgs e194477:22.4fe) v4 -2142> 2014-08-13 17:23:21.696952 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.696817, event: dispatched, request: pg_info(1 pgs e194477:22.4fe) v4 -2141> 2014-08-13 17:23:21.696959 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.696959, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4fe) v4 -2140> 2014-08-13 17:23:21.696968 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.696968, event: started, request: pg_info(1 pgs e194477:22.4fe) v4 -2139> 2014-08-13 17:23:21.697054 7fcd0964c700 5 -- op tracker -- , seq: 432, time: 2014-08-13 17:23:21.697054, event: done, request: pg_info(1 pgs e194477:22.4fe) v4 -2138> 2014-08-13 17:23:21.697076 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 6 ==== pg_notify(24.30b(30) epoch 194477) v5 ==== 2438+0+0 (1242858415 0 0) 0x54d2140 con 0x4fdad60 -2137> 2014-08-13 17:23:21.697095 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.683495, event: header_read, request: pg_notify(24.30b(30) epoch 194477) v5 -2136> 2014-08-13 17:23:21.697102 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.683496, event: throttled, request: pg_notify(24.30b(30) epoch 194477) v5 -2135> 2014-08-13 17:23:21.697108 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.683607, event: all_read, request: pg_notify(24.30b(30) epoch 194477) v5 -2134> 2014-08-13 17:23:21.697136 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.697093, event: dispatched, request: pg_notify(24.30b(30) epoch 194477) v5 -2133> 2014-08-13 17:23:21.697147 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.697147, event: waiting_for_osdmap, request: pg_notify(24.30b(30) epoch 194477) v5 -2132> 2014-08-13 17:23:21.697160 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.697160, event: started, request: pg_notify(24.30b(30) epoch 194477) v5 -2131> 2014-08-13 17:23:21.697138 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.043800 10 0.000417 -2130> 2014-08-13 17:23:21.697190 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -2129> 2014-08-13 17:23:21.697277 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000087 0 0.000000 -2128> 2014-08-13 17:23:21.697308 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -2127> 2014-08-13 17:23:21.697334 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000027 0 0.000000 -2126> 2014-08-13 17:23:21.697356 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -2125> 2014-08-13 17:23:21.697387 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_log(24.1e4 epoch 194477 log log((194422'18567,194422'18567], crt=194422'18567) query_epoch 194477) v4 -- ?+0 0x56daa00 con 0x4ec56a0 -2124> 2014-08-13 17:23:21.697438 7fcd0964c700 5 -- op tracker -- , seq: 433, time: 2014-08-13 17:23:21.697438, event: done, request: pg_notify(24.30b(30) epoch 194477) v5 -2123> 2014-08-13 17:23:21.697494 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.347 192.168.1.106:6800/4640 6 ==== pg_info(1 pgs e194477:22.1b8) v4 ==== 714+0+0 (1067714062 0 0) 0x56ac0c0 con 0x4fd1a20 -2122> 2014-08-13 17:23:21.697514 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.683742, event: header_read, request: pg_info(1 pgs e194477:22.1b8) v4 -2121> 2014-08-13 17:23:21.697522 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.683743, event: throttled, request: pg_info(1 pgs e194477:22.1b8) v4 -2120> 2014-08-13 17:23:21.697528 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.683802, event: all_read, request: pg_info(1 pgs e194477:22.1b8) v4 -2119> 2014-08-13 17:23:21.697534 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.697510, event: dispatched, request: pg_info(1 pgs e194477:22.1b8) v4 -2118> 2014-08-13 17:23:21.697541 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.697541, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1b8) v4 -2117> 2014-08-13 17:23:21.697550 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.697549, event: started, request: pg_info(1 pgs e194477:22.1b8) v4 -2116> 2014-08-13 17:23:21.697573 7fcd0964c700 5 -- op tracker -- , seq: 434, time: 2014-08-13 17:23:21.697573, event: done, request: pg_info(1 pgs e194477:22.1b8) v4 -2115> 2014-08-13 17:23:21.697602 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.334 192.168.1.104:6808/62751 6 ==== pg_info(1 pgs e194477:22.69f) v4 ==== 714+0+0 (2993817345 0 0) 0x56af1c0 con 0x4fde040 -2114> 2014-08-13 17:23:21.697625 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.689257, event: header_read, request: pg_info(1 pgs e194477:22.69f) v4 -2113> 2014-08-13 17:23:21.697633 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.689259, event: throttled, request: pg_info(1 pgs e194477:22.69f) v4 -2112> 2014-08-13 17:23:21.697638 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.689383, event: all_read, request: pg_info(1 pgs e194477:22.69f) v4 -2111> 2014-08-13 17:23:21.697644 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.697623, event: dispatched, request: pg_info(1 pgs e194477:22.69f) v4 -2110> 2014-08-13 17:23:21.697651 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.697650, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.69f) v4 -2109> 2014-08-13 17:23:21.697659 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.697658, event: started, request: pg_info(1 pgs e194477:22.69f) v4 -2108> 2014-08-13 17:23:21.697678 7fcd0964c700 5 -- op tracker -- , seq: 435, time: 2014-08-13 17:23:21.697678, event: done, request: pg_info(1 pgs e194477:22.69f) v4 -2107> 2014-08-13 17:23:21.697705 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 10 ==== pg_info(1 pgs e194477:22.639) v4 ==== 714+0+0 (677029327 0 0) 0x588e200 con 0x4fd3860 -2106> 2014-08-13 17:23:21.697725 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.694860, event: header_read, request: pg_info(1 pgs e194477:22.639) v4 -2105> 2014-08-13 17:23:21.697736 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.694862, event: throttled, request: pg_info(1 pgs e194477:22.639) v4 -2104> 2014-08-13 17:23:21.697745 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.694938, event: all_read, request: pg_info(1 pgs e194477:22.639) v4 -2103> 2014-08-13 17:23:21.697756 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.697723, event: dispatched, request: pg_info(1 pgs e194477:22.639) v4 -2102> 2014-08-13 17:23:21.697763 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.697763, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.639) v4 -2101> 2014-08-13 17:23:21.697774 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.697774, event: started, request: pg_info(1 pgs e194477:22.639) v4 -2100> 2014-08-13 17:23:21.697829 7fcd0964c700 5 -- op tracker -- , seq: 436, time: 2014-08-13 17:23:21.697829, event: done, request: pg_info(1 pgs e194477:22.639) v4 -2099> 2014-08-13 17:23:21.697859 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 10 ==== pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 ==== 4056+0+0 (2554542883 0 0) 0x58cc200 con 0x4fd5960 -2098> 2014-08-13 17:23:21.697881 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.695371, event: header_read, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2097> 2014-08-13 17:23:21.697890 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.695373, event: throttled, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2096> 2014-08-13 17:23:21.697897 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.695520, event: all_read, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2095> 2014-08-13 17:23:21.697904 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.697878, event: dispatched, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2094> 2014-08-13 17:23:21.697912 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.697912, event: waiting_for_osdmap, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2093> 2014-08-13 17:23:21.697926 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.697926, event: started, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2092> 2014-08-13 17:23:21.697948 7fcd0964c700 5 -- op tracker -- , seq: 437, time: 2014-08-13 17:23:21.697948, event: done, request: pg_log(22.3d9 epoch 194477 log log((194215'511,194215'511], crt=194215'511) query_epoch 194477) v4 -2091> 2014-08-13 17:23:21.697976 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 12 ==== pg_info(1 pgs e194477:22.1fb) v4 ==== 714+0+0 (4186034584 0 0) 0x54d0700 con 0x4fd32e0 -2090> 2014-08-13 17:23:21.697991 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.694885, event: header_read, request: pg_info(1 pgs e194477:22.1fb) v4 -2089> 2014-08-13 17:23:21.697998 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.694887, event: throttled, request: pg_info(1 pgs e194477:22.1fb) v4 -2088> 2014-08-13 17:23:21.698004 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.694967, event: all_read, request: pg_info(1 pgs e194477:22.1fb) v4 -2087> 2014-08-13 17:23:21.698009 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.697989, event: dispatched, request: pg_info(1 pgs e194477:22.1fb) v4 -2086> 2014-08-13 17:23:21.698015 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.698015, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1fb) v4 -2085> 2014-08-13 17:23:21.698023 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.698022, event: started, request: pg_info(1 pgs e194477:22.1fb) v4 -2084> 2014-08-13 17:23:21.698042 7fcd0964c700 5 -- op tracker -- , seq: 438, time: 2014-08-13 17:23:21.698042, event: done, request: pg_info(1 pgs e194477:22.1fb) v4 -2083> 2014-08-13 17:23:21.698066 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 6 ==== pg_info(1 pgs e194477:24.299) v4 ==== 744+0+0 (2008976712 0 0) 0x5888700 con 0x4fd3020 -2082> 2014-08-13 17:23:21.698081 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.693669, event: header_read, request: pg_info(1 pgs e194477:24.299) v4 -2081> 2014-08-13 17:23:21.698087 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.693671, event: throttled, request: pg_info(1 pgs e194477:24.299) v4 -2080> 2014-08-13 17:23:21.698093 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.693853, event: all_read, request: pg_info(1 pgs e194477:24.299) v4 -2079> 2014-08-13 17:23:21.698098 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.698079, event: dispatched, request: pg_info(1 pgs e194477:24.299) v4 -2078> 2014-08-13 17:23:21.698104 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.698104, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.299) v4 -2077> 2014-08-13 17:23:21.698112 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.698112, event: started, request: pg_info(1 pgs e194477:24.299) v4 -2076> 2014-08-13 17:23:21.698131 7fcd0964c700 5 -- op tracker -- , seq: 439, time: 2014-08-13 17:23:21.698131, event: done, request: pg_info(1 pgs e194477:24.299) v4 -2075> 2014-08-13 17:23:21.698152 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 6 ==== pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 ==== 4504+0+0 (1572101677 0 0) 0x58cc800 con 0x53011e0 -2074> 2014-08-13 17:23:21.698169 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.695869, event: header_read, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2073> 2014-08-13 17:23:21.698177 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.695872, event: throttled, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2072> 2014-08-13 17:23:21.698185 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.696046, event: all_read, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2071> 2014-08-13 17:23:21.698192 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.698167, event: dispatched, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2070> 2014-08-13 17:23:21.698209 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.698209, event: waiting_for_osdmap, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2069> 2014-08-13 17:23:21.698220 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.698220, event: started, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2068> 2014-08-13 17:23:21.698235 7fcd0964c700 5 -- op tracker -- , seq: 440, time: 2014-08-13 17:23:21.698235, event: done, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -2067> 2014-08-13 17:23:21.698261 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 5 ==== pg_notify(22.481(39) epoch 194477) v5 ==== 2911+0+0 (3525017014 0 0) 0x54d6740 con 0x552d540 -2066> 2014-08-13 17:23:21.698278 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.688507, event: header_read, request: pg_notify(22.481(39) epoch 194477) v5 -2065> 2014-08-13 17:23:21.698285 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.688508, event: throttled, request: pg_notify(22.481(39) epoch 194477) v5 -2064> 2014-08-13 17:23:21.698291 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.688631, event: all_read, request: pg_notify(22.481(39) epoch 194477) v5 -2063> 2014-08-13 17:23:21.698296 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.698276, event: dispatched, request: pg_notify(22.481(39) epoch 194477) v5 -2062> 2014-08-13 17:23:21.698302 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.698302, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194477) v5 -2061> 2014-08-13 17:23:21.698310 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.698310, event: started, request: pg_notify(22.481(39) epoch 194477) v5 -2060> 2014-08-13 17:23:21.698339 7fcd0964c700 5 -- op tracker -- , seq: 441, time: 2014-08-13 17:23:21.698339, event: done, request: pg_notify(22.481(39) epoch 194477) v5 -2059> 2014-08-13 17:23:21.698374 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.352 192.168.1.108:6826/4273 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x55086c0 con 0x56b1fa0 -2058> 2014-08-13 17:23:21.698399 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -2057> 2014-08-13 17:23:21.698422 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.307 192.168.1.109:6804/31016 3 ==== pg_notify(24.181(35) epoch 194477) v5 ==== 2717+0+0 (1371071092 0 0) 0x54d0c40 con 0x56b4fc0 -2056> 2014-08-13 17:23:21.698442 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.685470, event: header_read, request: pg_notify(24.181(35) epoch 194477) v5 -2055> 2014-08-13 17:23:21.698454 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.685472, event: throttled, request: pg_notify(24.181(35) epoch 194477) v5 -2054> 2014-08-13 17:23:21.698464 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.685593, event: all_read, request: pg_notify(24.181(35) epoch 194477) v5 -2053> 2014-08-13 17:23:21.698474 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.698440, event: dispatched, request: pg_notify(24.181(35) epoch 194477) v5 -2052> 2014-08-13 17:23:21.698484 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.698484, event: waiting_for_osdmap, request: pg_notify(24.181(35) epoch 194477) v5 -2051> 2014-08-13 17:23:21.698496 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.698496, event: started, request: pg_notify(24.181(35) epoch 194477) v5 -2050> 2014-08-13 17:23:21.698522 7fcd0964c700 5 -- op tracker -- , seq: 442, time: 2014-08-13 17:23:21.698522, event: done, request: pg_notify(24.181(35) epoch 194477) v5 -2049> 2014-08-13 17:23:21.698547 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.47 192.168.1.111:6828/12102 4 ==== pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 ==== 4878+0+0 (2329973560 0 0) 0x58cda00 con 0x57232e0 -2048> 2014-08-13 17:23:21.698570 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.696791, event: header_read, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2047> 2014-08-13 17:23:21.698582 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.696793, event: throttled, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2046> 2014-08-13 17:23:21.698593 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.696962, event: all_read, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2045> 2014-08-13 17:23:21.698604 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.698567, event: dispatched, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2044> 2014-08-13 17:23:21.698615 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.698615, event: waiting_for_osdmap, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2043> 2014-08-13 17:23:21.698628 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.698628, event: started, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2042> 2014-08-13 17:23:21.698648 7fcd0964c700 5 -- op tracker -- , seq: 443, time: 2014-08-13 17:23:21.698647, event: done, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -2041> 2014-08-13 17:23:21.698666 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.59 192.168.1.112:6822/5265 2 ==== pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 ==== 4746+0+0 (2881551850 0 0) 0x588f700 con 0x53e65c0 -2040> 2014-08-13 17:23:21.698683 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.688901, event: header_read, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2039> 2014-08-13 17:23:21.698690 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.688903, event: throttled, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2038> 2014-08-13 17:23:21.698696 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.689147, event: all_read, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2037> 2014-08-13 17:23:21.698702 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.698681, event: dispatched, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2036> 2014-08-13 17:23:21.698709 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.698709, event: waiting_for_osdmap, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2035> 2014-08-13 17:23:21.698718 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.698717, event: started, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2034> 2014-08-13 17:23:21.698755 7fcd0964c700 5 -- op tracker -- , seq: 444, time: 2014-08-13 17:23:21.698755, event: done, request: pg_notify(24.1f7(29),24.1f7(29) epoch 194477) v5 -2033> 2014-08-13 17:23:21.698794 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.348 192.168.1.110:6800/41204 3 ==== pg_notify(24.1bb(27) epoch 194477) v5 ==== 2261+0+0 (3277685512 0 0) 0x588fc40 con 0x53e4fc0 -2032> 2014-08-13 17:23:21.698815 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.687286, event: header_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2031> 2014-08-13 17:23:21.698825 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.687290, event: throttled, request: pg_notify(24.1bb(27) epoch 194477) v5 -2030> 2014-08-13 17:23:21.698834 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.687433, event: all_read, request: pg_notify(24.1bb(27) epoch 194477) v5 -2029> 2014-08-13 17:23:21.698844 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.698812, event: dispatched, request: pg_notify(24.1bb(27) epoch 194477) v5 -2028> 2014-08-13 17:23:21.698855 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.698855, event: waiting_for_osdmap, request: pg_notify(24.1bb(27) epoch 194477) v5 -2027> 2014-08-13 17:23:21.698867 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.698867, event: started, request: pg_notify(24.1bb(27) epoch 194477) v5 -2026> 2014-08-13 17:23:21.698891 7fcd0964c700 5 -- op tracker -- , seq: 445, time: 2014-08-13 17:23:21.698891, event: done, request: pg_notify(24.1bb(27) epoch 194477) v5 -2025> 2014-08-13 17:23:21.698918 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 11 ==== pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 ==== 3470+0+0 (2480947458 0 0) 0x58cd400 con 0x4e335a0 -2024> 2014-08-13 17:23:21.698976 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.696750, event: header_read, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2023> 2014-08-13 17:23:21.698989 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.696752, event: throttled, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2022> 2014-08-13 17:23:21.698999 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.696885, event: all_read, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2021> 2014-08-13 17:23:21.699010 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.698973, event: dispatched, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2020> 2014-08-13 17:23:21.699021 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.699021, event: waiting_for_osdmap, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2019> 2014-08-13 17:23:21.699034 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.699034, event: started, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2018> 2014-08-13 17:23:21.699053 7fcd0964c700 5 -- op tracker -- , seq: 446, time: 2014-08-13 17:23:21.699053, event: done, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -2017> 2014-08-13 17:23:21.699072 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 8 ==== pg_info(1 pgs e194477:22.29a) v4 ==== 714+0+0 (3418476361 0 0) 0x588bd40 con 0x4e32940 -2016> 2014-08-13 17:23:21.699093 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.696731, event: header_read, request: pg_info(1 pgs e194477:22.29a) v4 -2015> 2014-08-13 17:23:21.699100 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.696733, event: throttled, request: pg_info(1 pgs e194477:22.29a) v4 -2014> 2014-08-13 17:23:21.699106 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.696899, event: all_read, request: pg_info(1 pgs e194477:22.29a) v4 -2013> 2014-08-13 17:23:21.699111 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.699091, event: dispatched, request: pg_info(1 pgs e194477:22.29a) v4 -2012> 2014-08-13 17:23:21.699118 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.699118, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.29a) v4 -2011> 2014-08-13 17:23:21.699128 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.699128, event: started, request: pg_info(1 pgs e194477:22.29a) v4 -2010> 2014-08-13 17:23:21.699152 7fcd0964c700 5 -- op tracker -- , seq: 447, time: 2014-08-13 17:23:21.699152, event: done, request: pg_info(1 pgs e194477:22.29a) v4 -2009> 2014-08-13 17:23:21.699171 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 7 ==== pg_info(1 pgs e194477:24.3d) v4 ==== 748+0+0 (790035466 0 0) 0x54d4980 con 0x4e33de0 -2008> 2014-08-13 17:23:21.699190 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.695058, event: header_read, request: pg_info(1 pgs e194477:24.3d) v4 -2007> 2014-08-13 17:23:21.699213 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.695060, event: throttled, request: pg_info(1 pgs e194477:24.3d) v4 -2006> 2014-08-13 17:23:21.699234 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.695232, event: all_read, request: pg_info(1 pgs e194477:24.3d) v4 -2005> 2014-08-13 17:23:21.699244 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.699187, event: dispatched, request: pg_info(1 pgs e194477:24.3d) v4 -2004> 2014-08-13 17:23:21.699253 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.699253, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3d) v4 -2003> 2014-08-13 17:23:21.699265 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.699264, event: started, request: pg_info(1 pgs e194477:24.3d) v4 -2002> 2014-08-13 17:23:21.699492 7fcd0964c700 5 -- op tracker -- , seq: 448, time: 2014-08-13 17:23:21.699491, event: done, request: pg_info(1 pgs e194477:24.3d) v4 -2001> 2014-08-13 17:23:21.699526 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 11 ==== pg_notify(22.3af(5),24.136(27) epoch 194477) v5 ==== 3250+0+0 (1672358396 0 0) 0x56ade80 con 0x4e35ac0 -2000> 2014-08-13 17:23:21.699547 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.673495, event: header_read, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1999> 2014-08-13 17:23:21.699555 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.673502, event: throttled, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1998> 2014-08-13 17:23:21.699562 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.673626, event: all_read, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1997> 2014-08-13 17:23:21.699569 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.699544, event: dispatched, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1996> 2014-08-13 17:23:21.699577 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.699577, event: waiting_for_osdmap, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1995> 2014-08-13 17:23:21.699586 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.699586, event: started, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1994> 2014-08-13 17:23:21.699623 7fcd0964c700 5 -- op tracker -- , seq: 449, time: 2014-08-13 17:23:21.699623, event: done, request: pg_notify(22.3af(5),24.136(27) epoch 194477) v5 -1993> 2014-08-13 17:23:21.699644 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 9 ==== pg_info(1 pgs e194477:22.1b8) v4 ==== 714+0+0 (1710515684 0 0) 0x56af000 con 0x4e35540 -1992> 2014-08-13 17:23:21.699660 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.677554, event: header_read, request: pg_info(1 pgs e194477:22.1b8) v4 -1991> 2014-08-13 17:23:21.699645 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.014952 7 0.000209 -1990> 2014-08-13 17:23:21.699667 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.677556, event: throttled, request: pg_info(1 pgs e194477:22.1b8) v4 -1989> 2014-08-13 17:23:21.699673 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.677625, event: all_read, request: pg_info(1 pgs e194477:22.1b8) v4 -1988> 2014-08-13 17:23:21.699678 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.699658, event: dispatched, request: pg_info(1 pgs e194477:22.1b8) v4 -1987> 2014-08-13 17:23:21.699685 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.699684, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1b8) v4 -1986> 2014-08-13 17:23:21.699679 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1985> 2014-08-13 17:23:21.699693 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.699692, event: started, request: pg_info(1 pgs e194477:22.1b8) v4 -1984> 2014-08-13 17:23:21.699732 7fcd0964c700 5 -- op tracker -- , seq: 450, time: 2014-08-13 17:23:21.699731, event: done, request: pg_info(1 pgs e194477:22.1b8) v4 -1983> 2014-08-13 17:23:21.699743 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_log(24.3f7 epoch 194477 log log((194422'15537,194422'15537], crt=194422'15537) query_epoch 194477) v4 -- ?+0 0x58c9200 con 0x5305ee0 -1982> 2014-08-13 17:23:21.699751 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 8 ==== pg_info(1 pgs e194477:22.791) v4 ==== 714+0+0 (2862539511 0 0) 0x588b2c0 con 0x4ec3f40 -1981> 2014-08-13 17:23:21.699766 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.691915, event: header_read, request: pg_info(1 pgs e194477:22.791) v4 -1980> 2014-08-13 17:23:21.699773 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.691917, event: throttled, request: pg_info(1 pgs e194477:22.791) v4 -1979> 2014-08-13 17:23:21.699778 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.691995, event: all_read, request: pg_info(1 pgs e194477:22.791) v4 -1978> 2014-08-13 17:23:21.699784 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.699764, event: dispatched, request: pg_info(1 pgs e194477:22.791) v4 -1977> 2014-08-13 17:23:21.699789 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.699789, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.791) v4 -1976> 2014-08-13 17:23:21.699797 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.699797, event: started, request: pg_info(1 pgs e194477:22.791) v4 -1975> 2014-08-13 17:23:21.699818 7fcd0964c700 5 -- op tracker -- , seq: 451, time: 2014-08-13 17:23:21.699818, event: done, request: pg_info(1 pgs e194477:22.791) v4 -1974> 2014-08-13 17:23:21.699834 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.289 192.168.1.113:6808/15284 15 ==== MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 ==== 26+0+0 (1867627219 0 0) 0x54d0fc0 con 0x4ec3440 -1973> 2014-08-13 17:23:21.699850 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.693070, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1972> 2014-08-13 17:23:21.699856 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.693073, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1971> 2014-08-13 17:23:21.699862 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.693140, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1970> 2014-08-13 17:23:21.699845 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.033873 8 0.000374 -1969> 2014-08-13 17:23:21.699868 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.699847, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1968> 2014-08-13 17:23:21.699875 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.699874, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1967> 2014-08-13 17:23:21.699876 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog -1966> 2014-08-13 17:23:21.699899 7fcd0964c700 5 -- op tracker -- , seq: 452, time: 2014-08-13 17:23:21.699898, event: done, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194477 v2 -1965> 2014-08-13 17:23:21.699913 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 14 ==== pg_notify(24.136(26) epoch 194477) v5 ==== 2202+0+0 (2731362500 0 0) 0x54d6580 con 0x4ec4780 -1964> 2014-08-13 17:23:21.699928 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.673489, event: header_read, request: pg_notify(24.136(26) epoch 194477) v5 -1963> 2014-08-13 17:23:21.699934 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.673498, event: throttled, request: pg_notify(24.136(26) epoch 194477) v5 -1962> 2014-08-13 17:23:21.699940 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.673622, event: all_read, request: pg_notify(24.136(26) epoch 194477) v5 -1961> 2014-08-13 17:23:21.699946 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.699926, event: dispatched, request: pg_notify(24.136(26) epoch 194477) v5 -1960> 2014-08-13 17:23:21.699952 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.699952, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194477) v5 -1959> 2014-08-13 17:23:21.699959 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.699959, event: started, request: pg_notify(24.136(26) epoch 194477) v5 -1958> 2014-08-13 17:23:21.699961 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000086 0 0.000000 -1957> 2014-08-13 17:23:21.699979 7fcd0964c700 5 -- op tracker -- , seq: 453, time: 2014-08-13 17:23:21.699979, event: done, request: pg_notify(24.136(26) epoch 194477) v5 -1956> 2014-08-13 17:23:21.699986 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing -1955> 2014-08-13 17:23:21.700003 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 5 ==== pg_notify(24.181(28) epoch 194477) v5 ==== 2316+0+0 (2891192339 0 0) 0x54d32c0 con 0x4ec70c0 -1954> 2014-08-13 17:23:21.700018 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.673696, event: header_read, request: pg_notify(24.181(28) epoch 194477) v5 -1953> 2014-08-13 17:23:21.700025 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.673698, event: throttled, request: pg_notify(24.181(28) epoch 194477) v5 -1952> 2014-08-13 17:23:21.700031 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.673820, event: all_read, request: pg_notify(24.181(28) epoch 194477) v5 -1951> 2014-08-13 17:23:21.700036 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.700016, event: dispatched, request: pg_notify(24.181(28) epoch 194477) v5 -1950> 2014-08-13 17:23:21.700042 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.700042, event: waiting_for_osdmap, request: pg_notify(24.181(28) epoch 194477) v5 -1949> 2014-08-13 17:23:21.700049 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.700049, event: started, request: pg_notify(24.181(28) epoch 194477) v5 -1948> 2014-08-13 17:23:21.700009 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000024 0 0.000000 -1947> 2014-08-13 17:23:21.700060 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru -1946> 2014-08-13 17:23:21.700125 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.028583 10 0.000290 -1945> 2014-08-13 17:23:21.700152 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog -1944> 2014-08-13 17:23:21.700247 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000095 0 0.000000 -1943> 2014-08-13 17:23:21.700266 7fcd0964c700 5 -- op tracker -- , seq: 454, time: 2014-08-13 17:23:21.700265, event: done, request: pg_notify(24.181(28) epoch 194477) v5 -1942> 2014-08-13 17:23:21.700277 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing -1941> 2014-08-13 17:23:21.700302 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 9 ==== pg_info(1 pgs e194477:22.1a0) v4 ==== 714+0+0 (3622964740 0 0) 0x56aa840 con 0x4eb23c0 -1940> 2014-08-13 17:23:21.700303 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000027 0 0.000000 -1939> 2014-08-13 17:23:21.700322 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.695079, event: header_read, request: pg_info(1 pgs e194477:22.1a0) v4 -1938> 2014-08-13 17:23:21.700330 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.695082, event: throttled, request: pg_info(1 pgs e194477:22.1a0) v4 -1937> 2014-08-13 17:23:21.700336 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.695149, event: all_read, request: pg_info(1 pgs e194477:22.1a0) v4 -1936> 2014-08-13 17:23:21.700325 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru -1935> 2014-08-13 17:23:21.700342 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.700319, event: dispatched, request: pg_info(1 pgs e194477:22.1a0) v4 -1934> 2014-08-13 17:23:21.700349 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.700349, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1a0) v4 -1933> 2014-08-13 17:23:21.700391 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.700391, event: started, request: pg_info(1 pgs e194477:22.1a0) v4 -1932> 2014-08-13 17:23:21.700532 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.039744 8 0.000237 -1931> 2014-08-13 17:23:21.700565 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog -1930> 2014-08-13 17:23:21.700640 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000076 0 0.000000 -1929> 2014-08-13 17:23:21.700663 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing -1928> 2014-08-13 17:23:21.700687 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000024 0 0.000000 -1927> 2014-08-13 17:23:21.700713 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru -1926> 2014-08-13 17:23:21.700717 7fcd0964c700 5 -- op tracker -- , seq: 455, time: 2014-08-13 17:23:21.700716, event: done, request: pg_info(1 pgs e194477:22.1a0) v4 -1925> 2014-08-13 17:23:21.700752 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 4 ==== pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 ==== 4504+0+0 (3618143431 0 0) 0x58cbc00 con 0x4eb48e0 -1924> 2014-08-13 17:23:21.700774 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.695321, event: header_read, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1923> 2014-08-13 17:23:21.700783 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.695323, event: throttled, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1922> 2014-08-13 17:23:21.700791 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.695476, event: all_read, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1921> 2014-08-13 17:23:21.700799 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.700771, event: dispatched, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1920> 2014-08-13 17:23:21.700807 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.700807, event: waiting_for_osdmap, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1919> 2014-08-13 17:23:21.700818 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.700817, event: started, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1918> 2014-08-13 17:23:21.700838 7fcd0964c700 5 -- op tracker -- , seq: 456, time: 2014-08-13 17:23:21.700838, event: done, request: pg_log(22.775 epoch 194477 log log((194215'513,194215'513], crt=194215'513) query_epoch 194477) v4 -1917> 2014-08-13 17:23:21.700858 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 6 ==== pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 ==== 4878+0+0 (718292743 0 0) 0x58cce00 con 0x4eb2680 -1916> 2014-08-13 17:23:21.700875 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.696677, event: header_read, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1915> 2014-08-13 17:23:21.700884 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.696679, event: throttled, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1914> 2014-08-13 17:23:21.700891 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.696832, event: all_read, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1913> 2014-08-13 17:23:21.700899 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.700873, event: dispatched, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1912> 2014-08-13 17:23:21.700907 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.700907, event: waiting_for_osdmap, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1911> 2014-08-13 17:23:21.700916 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.700916, event: started, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1910> 2014-08-13 17:23:21.700933 7fcd0964c700 5 -- op tracker -- , seq: 457, time: 2014-08-13 17:23:21.700933, event: done, request: pg_log(22.1f1 epoch 194477 log log((194215'493,194215'493], crt=194215'493) query_epoch 194477) v4 -1909> 2014-08-13 17:23:21.700953 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 5 ==== pg_info(1 pgs e194477:22.12a) v4 ==== 714+0+0 (4259955323 0 0) 0x56a9500 con 0x4f81fa0 -1908> 2014-08-13 17:23:21.700967 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700086, event: header_read, request: pg_info(1 pgs e194477:22.12a) v4 -1907> 2014-08-13 17:23:21.700974 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700088, event: throttled, request: pg_info(1 pgs e194477:22.12a) v4 -1906> 2014-08-13 17:23:21.700980 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700271, event: all_read, request: pg_info(1 pgs e194477:22.12a) v4 -1905> 2014-08-13 17:23:21.700985 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700965, event: dispatched, request: pg_info(1 pgs e194477:22.12a) v4 -1904> 2014-08-13 17:23:21.700991 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700991, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.12a) v4 -1903> 2014-08-13 17:23:21.700999 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.700998, event: started, request: pg_info(1 pgs e194477:22.12a) v4 -1902> 2014-08-13 17:23:21.701018 7fcd0964c700 5 -- op tracker -- , seq: 458, time: 2014-08-13 17:23:21.701018, event: done, request: pg_info(1 pgs e194477:22.12a) v4 -1901> 2014-08-13 17:23:21.701031 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 7 ==== pg_notify(22.3af(27) epoch 194477) v5 ==== 2231+0+0 (1918832129 0 0) 0x54d1340 con 0x4f86040 -1900> 2014-08-13 17:23:21.701046 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.678677, event: header_read, request: pg_notify(22.3af(27) epoch 194477) v5 -1899> 2014-08-13 17:23:21.701053 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.678679, event: throttled, request: pg_notify(22.3af(27) epoch 194477) v5 -1898> 2014-08-13 17:23:21.701040 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194477 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194477) v4 -- ?+0 0x58c9e00 con 0x4ec4780 -1897> 2014-08-13 17:23:21.701058 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.678781, event: all_read, request: pg_notify(22.3af(27) epoch 194477) v5 -1896> 2014-08-13 17:23:21.701064 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.701044, event: dispatched, request: pg_notify(22.3af(27) epoch 194477) v5 -1895> 2014-08-13 17:23:21.701070 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.701070, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194477) v5 -1894> 2014-08-13 17:23:21.701078 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.701078, event: started, request: pg_notify(22.3af(27) epoch 194477) v5 -1893> 2014-08-13 17:23:21.701098 7fcd0964c700 5 -- op tracker -- , seq: 459, time: 2014-08-13 17:23:21.701098, event: done, request: pg_notify(22.3af(27) epoch 194477) v5 -1892> 2014-08-13 17:23:21.701094 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.011163 5 0.000258 -1891> 2014-08-13 17:23:21.701121 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.287 192.168.1.109:6814/33435 6 ==== pg_info(1 pgs e194477:22.481) v4 ==== 714+0+0 (3593085414 0 0) 0x56aa140 con 0x4f853e0 -1890> 2014-08-13 17:23:21.701121 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1889> 2014-08-13 17:23:21.701137 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.689791, event: header_read, request: pg_info(1 pgs e194477:22.481) v4 -1888> 2014-08-13 17:23:21.701144 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.689792, event: throttled, request: pg_info(1 pgs e194477:22.481) v4 -1887> 2014-08-13 17:23:21.701149 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.689848, event: all_read, request: pg_info(1 pgs e194477:22.481) v4 -1886> 2014-08-13 17:23:21.701155 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.701135, event: dispatched, request: pg_info(1 pgs e194477:22.481) v4 -1885> 2014-08-13 17:23:21.701161 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.701161, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.481) v4 -1884> 2014-08-13 17:23:21.701168 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.701168, event: started, request: pg_info(1 pgs e194477:22.481) v4 -1883> 2014-08-13 17:23:21.701186 7fcd0964c700 5 -- op tracker -- , seq: 460, time: 2014-08-13 17:23:21.701186, event: done, request: pg_info(1 pgs e194477:22.481) v4 -1882> 2014-08-13 17:23:21.701214 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 3 ==== pg_notify(22.48e(25) epoch 194477) v5 ==== 2117+0+0 (952916012 0 0) 0x56af540 con 0x4f87900 -1881> 2014-08-13 17:23:21.701231 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.686384, event: header_read, request: pg_notify(22.48e(25) epoch 194477) v5 -1880> 2014-08-13 17:23:21.701238 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.686386, event: throttled, request: pg_notify(22.48e(25) epoch 194477) v5 -1879> 2014-08-13 17:23:21.701244 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.686506, event: all_read, request: pg_notify(22.48e(25) epoch 194477) v5 -1878> 2014-08-13 17:23:21.701249 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.701229, event: dispatched, request: pg_notify(22.48e(25) epoch 194477) v5 -1877> 2014-08-13 17:23:21.701255 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.701255, event: waiting_for_osdmap, request: pg_notify(22.48e(25) epoch 194477) v5 -1876> 2014-08-13 17:23:21.701263 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.701263, event: started, request: pg_notify(22.48e(25) epoch 194477) v5 -1875> 2014-08-13 17:23:21.701283 7fcd0964c700 5 -- op tracker -- , seq: 461, time: 2014-08-13 17:23:21.701283, event: done, request: pg_notify(22.48e(25) epoch 194477) v5 -1874> 2014-08-13 17:23:21.701308 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.292 192.168.1.113:6812/16056 6 ==== pg_info(1 pgs e194477:22.3ad) v4 ==== 714+0+0 (1539556767 0 0) 0x588ad80 con 0x4f870c0 -1873> 2014-08-13 17:23:21.701322 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.691717, event: header_read, request: pg_info(1 pgs e194477:22.3ad) v4 -1872> 2014-08-13 17:23:21.701329 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.691755, event: throttled, request: pg_info(1 pgs e194477:22.3ad) v4 -1871> 2014-08-13 17:23:21.701334 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.691847, event: all_read, request: pg_info(1 pgs e194477:22.3ad) v4 -1870> 2014-08-13 17:23:21.701339 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.701320, event: dispatched, request: pg_info(1 pgs e194477:22.3ad) v4 -1869> 2014-08-13 17:23:21.701346 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.701345, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3ad) v4 -1868> 2014-08-13 17:23:21.701353 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.701353, event: started, request: pg_info(1 pgs e194477:22.3ad) v4 -1867> 2014-08-13 17:23:21.701371 7fcd0964c700 5 -- op tracker -- , seq: 462, time: 2014-08-13 17:23:21.701371, event: done, request: pg_info(1 pgs e194477:22.3ad) v4 -1866> 2014-08-13 17:23:21.701384 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 13 ==== pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 ==== 3470+0+0 (1888486293 0 0) 0x56dda00 con 0x4f86e00 -1865> 2014-08-13 17:23:21.701400 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.696567, event: header_read, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1864> 2014-08-13 17:23:21.701409 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.696569, event: throttled, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1863> 2014-08-13 17:23:21.701416 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.696693, event: all_read, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1862> 2014-08-13 17:23:21.701424 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.701398, event: dispatched, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1861> 2014-08-13 17:23:21.701432 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.701432, event: waiting_for_osdmap, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1860> 2014-08-13 17:23:21.701441 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.701441, event: started, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1859> 2014-08-13 17:23:21.701456 7fcd0964c700 5 -- op tracker -- , seq: 463, time: 2014-08-13 17:23:21.701456, event: done, request: pg_log(22.362 epoch 194477 log log((194215'479,194215'479], crt=194215'479) query_epoch 194477) v4 -1858> 2014-08-13 17:23:21.701449 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.030820 7 0.000261 -1857> 2014-08-13 17:23:21.701471 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 9 ==== pg_info(1 pgs e194477:22.4fe) v4 ==== 714+0+0 (3827136916 0 0) 0x54d7700 con 0x4fd8f20 -1856> 2014-08-13 17:23:21.701476 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -1855> 2014-08-13 17:23:21.701485 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.695338, event: header_read, request: pg_info(1 pgs e194477:22.4fe) v4 -1854> 2014-08-13 17:23:21.701498 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.695341, event: throttled, request: pg_info(1 pgs e194477:22.4fe) v4 -1853> 2014-08-13 17:23:21.701503 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.695471, event: all_read, request: pg_info(1 pgs e194477:22.4fe) v4 -1852> 2014-08-13 17:23:21.701509 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.701483, event: dispatched, request: pg_info(1 pgs e194477:22.4fe) v4 -1851> 2014-08-13 17:23:21.701515 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.701515, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.4fe) v4 -1850> 2014-08-13 17:23:21.701523 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.701522, event: started, request: pg_info(1 pgs e194477:22.4fe) v4 -1849> 2014-08-13 17:23:21.701541 7fcd0964c700 5 -- op tracker -- , seq: 464, time: 2014-08-13 17:23:21.701541, event: done, request: pg_info(1 pgs e194477:22.4fe) v4 -1848> 2014-08-13 17:23:21.701540 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000065 0 0.000000 -1847> 2014-08-13 17:23:21.701554 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 7 ==== pg_notify(22.3d5(22) epoch 194477) v5 ==== 1948+0+0 (379675144 0 0) 0x54d0e00 con 0x4fdad60 -1846> 2014-08-13 17:23:21.701563 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -1845> 2014-08-13 17:23:21.701572 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.684920, event: header_read, request: pg_notify(22.3d5(22) epoch 194477) v5 -1844> 2014-08-13 17:23:21.701580 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.684921, event: throttled, request: pg_notify(22.3d5(22) epoch 194477) v5 -1843> 2014-08-13 17:23:21.701585 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.685012, event: all_read, request: pg_notify(22.3d5(22) epoch 194477) v5 -1842> 2014-08-13 17:23:21.701590 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.701570, event: dispatched, request: pg_notify(22.3d5(22) epoch 194477) v5 -1841> 2014-08-13 17:23:21.701597 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.701596, event: waiting_for_osdmap, request: pg_notify(22.3d5(22) epoch 194477) v5 -1840> 2014-08-13 17:23:21.701604 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.701604, event: started, request: pg_notify(22.3d5(22) epoch 194477) v5 -1839> 2014-08-13 17:23:21.701608 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6818/9142 -- pg_query(22.48e epoch 194477) v3 -- ?+0 0x2f96720 con 0x4f87900 -1838> 2014-08-13 17:23:21.701623 7fcd0964c700 5 -- op tracker -- , seq: 465, time: 2014-08-13 17:23:21.701623, event: done, request: pg_notify(22.3d5(22) epoch 194477) v5 -1837> 2014-08-13 17:23:21.701646 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 5 ==== pg_info(1 pgs e194477:22.537) v4 ==== 714+0+0 (1565943020 0 0) 0x588a4c0 con 0x4fda520 -1836> 2014-08-13 17:23:21.701661 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.697678, event: header_read, request: pg_info(1 pgs e194477:22.537) v4 -1835> 2014-08-13 17:23:21.701668 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.697680, event: throttled, request: pg_info(1 pgs e194477:22.537) v4 -1834> 2014-08-13 17:23:21.701673 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.697760, event: all_read, request: pg_info(1 pgs e194477:22.537) v4 -1833> 2014-08-13 17:23:21.701679 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.701659, event: dispatched, request: pg_info(1 pgs e194477:22.537) v4 -1832> 2014-08-13 17:23:21.701685 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.701684, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.537) v4 -1831> 2014-08-13 17:23:21.701693 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.701693, event: started, request: pg_info(1 pgs e194477:22.537) v4 -1830> 2014-08-13 17:23:21.701712 7fcd0964c700 5 -- op tracker -- , seq: 466, time: 2014-08-13 17:23:21.701712, event: done, request: pg_info(1 pgs e194477:22.537) v4 -1829> 2014-08-13 17:23:21.701726 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.367 192.168.1.114:6840/55513 11 ==== pg_info(1 pgs e194477:22.639) v4 ==== 714+0+0 (1514036736 0 0) 0x588c440 con 0x4fd3860 -1828> 2014-08-13 17:23:21.701740 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.695078, event: header_read, request: pg_info(1 pgs e194477:22.639) v4 -1827> 2014-08-13 17:23:21.701746 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.695081, event: throttled, request: pg_info(1 pgs e194477:22.639) v4 -1826> 2014-08-13 17:23:21.701754 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.695175, event: all_read, request: pg_info(1 pgs e194477:22.639) v4 -1825> 2014-08-13 17:23:21.701759 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.701738, event: dispatched, request: pg_info(1 pgs e194477:22.639) v4 -1824> 2014-08-13 17:23:21.701765 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.701765, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.639) v4 -1823> 2014-08-13 17:23:21.701773 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.701773, event: started, request: pg_info(1 pgs e194477:22.639) v4 -1822> 2014-08-13 17:23:21.701791 7fcd0964c700 5 -- op tracker -- , seq: 467, time: 2014-08-13 17:23:21.701791, event: done, request: pg_info(1 pgs e194477:22.639) v4 -1821> 2014-08-13 17:23:21.701804 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 13 ==== pg_info(1 pgs e194477:22.1fb) v4 ==== 714+0+0 (3948647923 0 0) 0x56ae200 con 0x4fd32e0 -1820> 2014-08-13 17:23:21.701818 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.695455, event: header_read, request: pg_info(1 pgs e194477:22.1fb) v4 -1819> 2014-08-13 17:23:21.701825 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.695457, event: throttled, request: pg_info(1 pgs e194477:22.1fb) v4 -1818> 2014-08-13 17:23:21.701830 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.695524, event: all_read, request: pg_info(1 pgs e194477:22.1fb) v4 -1817> 2014-08-13 17:23:21.701890 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194474/194474 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 0.749420 8 0.000472 -1816> 2014-08-13 17:23:21.701930 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194474/194474 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive -1815> 2014-08-13 17:23:21.701953 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194448 n=53 ec=192688 les/c 194474/194474 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive/RepNotRecovering -1814> 2014-08-13 17:23:21.701979 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.009340 4 0.000160 -1813> 2014-08-13 17:23:21.702006 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1812> 2014-08-13 17:23:21.701836 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.701816, event: dispatched, request: pg_info(1 pgs e194477:22.1fb) v4 -1811> 2014-08-13 17:23:21.702184 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.702184, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1fb) v4 -1810> 2014-08-13 17:23:21.702222 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.702221, event: started, request: pg_info(1 pgs e194477:22.1fb) v4 -1809> 2014-08-13 17:23:21.702229 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.041915 8 0.000759 -1808> 2014-08-13 17:23:21.702261 7fcd0964c700 5 -- op tracker -- , seq: 468, time: 2014-08-13 17:23:21.702261, event: done, request: pg_info(1 pgs e194477:22.1fb) v4 -1807> 2014-08-13 17:23:21.702267 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog -1806> 2014-08-13 17:23:21.702303 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.396 192.168.1.105:6814/41806 7 ==== MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 ==== 30+0+0 (1582489391 0 0) 0x5b49180 con 0x4fd3020 -1805> 2014-08-13 17:23:21.702494 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.694373, event: header_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1804> 2014-08-13 17:23:21.702508 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.694375, event: throttled, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1803> 2014-08-13 17:23:21.702515 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.694464, event: all_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1802> 2014-08-13 17:23:21.702341 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000075 0 0.000000 -1801> 2014-08-13 17:23:21.702594 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing -1800> 2014-08-13 17:23:21.702637 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000044 0 0.000000 -1799> 2014-08-13 17:23:21.702666 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru -1798> 2014-08-13 17:23:21.703226 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.056631 6 0.005707 -1797> 2014-08-13 17:23:21.703267 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -1796> 2014-08-13 17:23:21.703276 7fcced766700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x524cd80 sd=212 :6819 s=0 pgs=0 cs=0 l=0 c=0x5bef900).accept sd=212 192.168.1.115:53811/0 -1795> 2014-08-13 17:23:21.703339 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000073 0 0.000000 -1794> 2014-08-13 17:23:21.703367 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -1793> 2014-08-13 17:23:21.703508 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6812/6757 -- pg_query(22.39d epoch 194477) v3 -- ?+0 0x54fdbe0 con 0x4ec70c0 -1792> 2014-08-13 17:23:21.703541 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_query(22.39d epoch 194477) v3 -- ?+0 0x54f81e0 con 0x4fd32e0 -1791> 2014-08-13 17:23:21.702521 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.702489, event: dispatched, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1790> 2014-08-13 17:23:21.703735 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.703734, event: waiting_for_osdmap, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1789> 2014-08-13 17:23:21.703796 7fcd0964c700 5 -- op tracker -- , seq: 469, time: 2014-08-13 17:23:21.703796, event: done, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194477, prio: 1 v3 -1788> 2014-08-13 17:23:21.703823 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.352 192.168.1.108:6826/4273 5 ==== pg_notify(22.4df(3) epoch 194477) v5 ==== 887+0+0 (409617718 0 0) 0x4e50a80 con 0x56b1fa0 -1787> 2014-08-13 17:23:21.703843 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.693396, event: header_read, request: pg_notify(22.4df(3) epoch 194477) v5 -1786> 2014-08-13 17:23:21.703851 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.693398, event: throttled, request: pg_notify(22.4df(3) epoch 194477) v5 -1785> 2014-08-13 17:23:21.703856 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.693465, event: all_read, request: pg_notify(22.4df(3) epoch 194477) v5 -1784> 2014-08-13 17:23:21.703862 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.703840, event: dispatched, request: pg_notify(22.4df(3) epoch 194477) v5 -1783> 2014-08-13 17:23:21.703868 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.703868, event: waiting_for_osdmap, request: pg_notify(22.4df(3) epoch 194477) v5 -1782> 2014-08-13 17:23:21.703876 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.703875, event: started, request: pg_notify(22.4df(3) epoch 194477) v5 -1781> 2014-08-13 17:23:21.703898 7fcd0964c700 5 -- op tracker -- , seq: 470, time: 2014-08-13 17:23:21.703898, event: done, request: pg_notify(22.4df(3) epoch 194477) v5 -1780> 2014-08-13 17:23:21.703883 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194477:22.113) v4 -- ?+0 0x56aad80 -1779> 2014-08-13 17:23:21.703913 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.380 192.168.1.114:6822/51844 3 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x51e0fc0 con 0x56b1ce0 -1778> 2014-08-13 17:23:21.703932 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -1777> 2014-08-13 17:23:21.703927 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.339 192.168.1.110:6818/45268 -- pg_info(1 pgs e194477:22.113) v4 -- ?+0 0x56a9f80 -1776> 2014-08-13 17:23:21.703943 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.10 192.168.1.115:6815/60573 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52bd340 con 0x4e30580 -1775> 2014-08-13 17:23:21.704201 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:22.1b8) v4 -- ?+0 0x56ad940 -1774> 2014-08-13 17:23:21.703958 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -1773> 2014-08-13 17:23:21.704261 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.345 192.168.1.110:6820/46337 9 ==== pg_info(1 pgs e194477:22.29a) v4 ==== 714+0+0 (4288711704 0 0) 0x588de80 con 0x4e32940 -1772> 2014-08-13 17:23:21.704258 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194477:22.1b8) v4 -- ?+0 0x588fa80 -1771> 2014-08-13 17:23:21.704278 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.697126, event: header_read, request: pg_info(1 pgs e194477:22.29a) v4 -1770> 2014-08-13 17:23:21.704287 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.697128, event: throttled, request: pg_info(1 pgs e194477:22.29a) v4 -1769> 2014-08-13 17:23:21.704293 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.697220, event: all_read, request: pg_info(1 pgs e194477:22.29a) v4 -1768> 2014-08-13 17:23:21.704299 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.704275, event: dispatched, request: pg_info(1 pgs e194477:22.29a) v4 -1767> 2014-08-13 17:23:21.704306 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.704305, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.29a) v4 -1766> 2014-08-13 17:23:21.704314 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.704314, event: started, request: pg_info(1 pgs e194477:22.29a) v4 -1765> 2014-08-13 17:23:21.704307 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194477:22.4df) v4 -- ?+0 0x56a8c40 -1764> 2014-08-13 17:23:21.704337 7fcd0964c700 5 -- op tracker -- , seq: 471, time: 2014-08-13 17:23:21.704337, event: done, request: pg_info(1 pgs e194477:22.29a) v4 -1763> 2014-08-13 17:23:21.704360 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.357 192.168.1.102:6834/17224 -- pg_info(1 pgs e194477:22.4df) v4 -- ?+0 0x56ad5c0 -1762> 2014-08-13 17:23:21.704589 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] exit Started/ReplicaActive/RepNotRecovering 0.016734 3 0.004234 -1761> 2014-08-13 17:23:21.704629 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] enter Started/ReplicaActive/RepWaitRecoveryReserved -1760> 2014-08-13 17:23:21.704591 7fccecf5e700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x524eb80 sd=210 :6819 s=0 pgs=0 cs=0 l=0 c=0x5befa60).accept sd=210 192.168.1.108:44653/0 -1759> 2014-08-13 17:23:21.704690 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194477:22.f3) v4 -- ?+0 0x56aeac0 -1758> 2014-08-13 17:23:21.704752 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.312 192.168.1.113:6816/17521 -- pg_info(1 pgs e194477:22.f3) v4 -- ?+0 0x56aaa00 -1757> 2014-08-13 17:23:21.704836 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.012409 6 0.000213 -1756> 2014-08-13 17:23:21.704872 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1755> 2014-08-13 17:23:21.704354 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 10 ==== pg_info(1 pgs e194477:2.2a) v4 ==== 706+0+0 (2535784854 0 0) 0x56aee40 con 0x4e35540 -1754> 2014-08-13 17:23:21.704915 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.677807, event: header_read, request: pg_info(1 pgs e194477:2.2a) v4 -1753> 2014-08-13 17:23:21.704932 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.677808, event: throttled, request: pg_info(1 pgs e194477:2.2a) v4 -1752> 2014-08-13 17:23:21.704940 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.677868, event: all_read, request: pg_info(1 pgs e194477:2.2a) v4 -1751> 2014-08-13 17:23:21.704949 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.704909, event: dispatched, request: pg_info(1 pgs e194477:2.2a) v4 -1750> 2014-08-13 17:23:21.704960 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.704959, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:2.2a) v4 -1749> 2014-08-13 17:23:21.704972 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.704972, event: started, request: pg_info(1 pgs e194477:2.2a) v4 -1748> 2014-08-13 17:23:21.704999 7fcd0964c700 5 -- op tracker -- , seq: 472, time: 2014-08-13 17:23:21.704999, event: done, request: pg_info(1 pgs e194477:2.2a) v4 -1747> 2014-08-13 17:23:21.705021 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.291 192.168.1.101:6820/25633 6 ==== pg_info(1 pgs e194477:24.9b) v4 ==== 748+0+0 (3363454884 0 0) 0x54d5e80 con 0x4ec27e0 -1746> 2014-08-13 17:23:21.705043 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.700663, event: header_read, request: pg_info(1 pgs e194477:24.9b) v4 -1745> 2014-08-13 17:23:21.705053 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.700666, event: throttled, request: pg_info(1 pgs e194477:24.9b) v4 -1744> 2014-08-13 17:23:21.705062 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.700754, event: all_read, request: pg_info(1 pgs e194477:24.9b) v4 -1743> 2014-08-13 17:23:21.705071 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.705040, event: dispatched, request: pg_info(1 pgs e194477:24.9b) v4 -1742> 2014-08-13 17:23:21.705078 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.705077, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.9b) v4 -1741> 2014-08-13 17:23:21.705085 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.705085, event: started, request: pg_info(1 pgs e194477:24.9b) v4 -1740> 2014-08-13 17:23:21.705108 7fcd0964c700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:23:21.705108, event: done, request: pg_info(1 pgs e194477:24.9b) v4 -1739> 2014-08-13 17:23:21.705126 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 15 ==== pg_query(24.3b7 epoch 194477) v3 ==== 144+0+0 (2652199811 0 0) 0x2f912c0 con 0x4ec4780 -1738> 2014-08-13 17:23:21.705145 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.702387, event: header_read, request: pg_query(24.3b7 epoch 194477) v3 -1737> 2014-08-13 17:23:21.705155 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.702389, event: throttled, request: pg_query(24.3b7 epoch 194477) v3 -1736> 2014-08-13 17:23:21.705165 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.702459, event: all_read, request: pg_query(24.3b7 epoch 194477) v3 -1735> 2014-08-13 17:23:21.705173 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.705143, event: dispatched, request: pg_query(24.3b7 epoch 194477) v3 -1734> 2014-08-13 17:23:21.705183 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.705182, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194477) v3 -1733> 2014-08-13 17:23:21.705237 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.705237, event: started, request: pg_query(24.3b7 epoch 194477) v3 -1732> 2014-08-13 17:23:21.705266 7fcd0964c700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:23:21.705265, event: done, request: pg_query(24.3b7 epoch 194477) v3 -1731> 2014-08-13 17:23:21.705285 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 6 ==== pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 ==== 3468+0+0 (3587757758 0 0) 0x56dec00 con 0x4ec70c0 -1730> 2014-08-13 17:23:21.705307 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.704783, event: header_read, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1729> 2014-08-13 17:23:21.705319 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.704785, event: throttled, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1728> 2014-08-13 17:23:21.705331 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.705033, event: all_read, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1727> 2014-08-13 17:23:21.705341 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.705305, event: dispatched, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1726> 2014-08-13 17:23:21.705353 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.705353, event: waiting_for_osdmap, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1725> 2014-08-13 17:23:21.705366 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.705365, event: started, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1724> 2014-08-13 17:23:21.705387 7fcd0964c700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:23:21.705386, event: done, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1723> 2014-08-13 17:23:21.705378 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194477:24.350) v4 -- ?+0 0x54d4440 -1722> 2014-08-13 17:23:21.705407 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.328 192.168.1.104:6812/64303 10 ==== pg_info(1 pgs e194477:22.1a0) v4 ==== 714+0+0 (1974427080 0 0) 0x56ab640 con 0x4eb23c0 -1721> 2014-08-13 17:23:21.705422 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.695350, event: header_read, request: pg_info(1 pgs e194477:22.1a0) v4 -1720> 2014-08-13 17:23:21.705429 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.695352, event: throttled, request: pg_info(1 pgs e194477:22.1a0) v4 -1719> 2014-08-13 17:23:21.705449 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.695422, event: all_read, request: pg_info(1 pgs e194477:22.1a0) v4 -1718> 2014-08-13 17:23:21.705455 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.705420, event: dispatched, request: pg_info(1 pgs e194477:22.1a0) v4 -1717> 2014-08-13 17:23:21.705461 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.705461, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1a0) v4 -1716> 2014-08-13 17:23:21.705469 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.705469, event: started, request: pg_info(1 pgs e194477:22.1a0) v4 -1715> 2014-08-13 17:23:21.705454 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.010581 5 0.000188 -1714> 2014-08-13 17:23:21.705489 7fcd0964c700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:23:21.705489, event: done, request: pg_info(1 pgs e194477:22.1a0) v4 -1713> 2014-08-13 17:23:21.705490 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1712> 2014-08-13 17:23:21.705503 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 11 ==== pg_info(1 pgs e194477:24.2ec) v4 ==== 748+0+0 (997361271 0 0) 0x54d5400 con 0x4eb18c0 -1711> 2014-08-13 17:23:21.705518 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705307, event: header_read, request: pg_info(1 pgs e194477:24.2ec) v4 -1710> 2014-08-13 17:23:21.705524 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705309, event: throttled, request: pg_info(1 pgs e194477:24.2ec) v4 -1709> 2014-08-13 17:23:21.705530 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705378, event: all_read, request: pg_info(1 pgs e194477:24.2ec) v4 -1708> 2014-08-13 17:23:21.705536 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705516, event: dispatched, request: pg_info(1 pgs e194477:24.2ec) v4 -1707> 2014-08-13 17:23:21.705544 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705543, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.2ec) v4 -1706> 2014-08-13 17:23:21.705551 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705551, event: started, request: pg_info(1 pgs e194477:24.2ec) v4 -1705> 2014-08-13 17:23:21.705570 7fcd0964c700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:23:21.705570, event: done, request: pg_info(1 pgs e194477:24.2ec) v4 -1704> 2014-08-13 17:23:21.705582 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.19 192.168.1.115:6818/61186 5 ==== pg_info(1 pgs e194477:24.3bf) v4 ==== 744+0+0 (1450727866 0 0) 0x54d6040 con 0x4f84ba0 -1703> 2014-08-13 17:23:21.705534 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.024552 7 0.000466 -1702> 2014-08-13 17:23:21.705597 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.703301, event: header_read, request: pg_info(1 pgs e194477:24.3bf) v4 -1701> 2014-08-13 17:23:21.705604 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.703303, event: throttled, request: pg_info(1 pgs e194477:24.3bf) v4 -1700> 2014-08-13 17:23:21.705610 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.703384, event: all_read, request: pg_info(1 pgs e194477:24.3bf) v4 -1699> 2014-08-13 17:23:21.705601 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1698> 2014-08-13 17:23:21.705615 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.705595, event: dispatched, request: pg_info(1 pgs e194477:24.3bf) v4 -1697> 2014-08-13 17:23:21.705603 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.287 192.168.1.109:6814/33435 -- pg_info(1 pgs e194477:22.481) v4 -- ?+0 0x588e740 -1696> 2014-08-13 17:23:21.705622 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.705622, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3bf) v4 -1695> 2014-08-13 17:23:21.705630 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.705629, event: started, request: pg_info(1 pgs e194477:24.3bf) v4 -1694> 2014-08-13 17:23:21.705642 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.309 192.168.1.101:6814/23703 -- pg_info(1 pgs e194477:22.481) v4 -- ?+0 0x588fe00 -1693> 2014-08-13 17:23:21.705648 7fcd0964c700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:23:21.705648, event: done, request: pg_info(1 pgs e194477:24.3bf) v4 -1692> 2014-08-13 17:23:21.705663 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.323 192.168.1.106:6816/8118 6 ==== pg_info(1 pgs e194477:22.12a) v4 ==== 714+0+0 (3594254271 0 0) 0x54d5780 con 0x4f81fa0 -1691> 2014-08-13 17:23:21.705677 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.700439, event: header_read, request: pg_info(1 pgs e194477:22.12a) v4 -1690> 2014-08-13 17:23:21.705683 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.700440, event: throttled, request: pg_info(1 pgs e194477:22.12a) v4 -1689> 2014-08-13 17:23:21.705688 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.700537, event: all_read, request: pg_info(1 pgs e194477:22.12a) v4 -1688> 2014-08-13 17:23:21.705694 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.705675, event: dispatched, request: pg_info(1 pgs e194477:22.12a) v4 -1687> 2014-08-13 17:23:21.705700 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.705700, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.12a) v4 -1686> 2014-08-13 17:23:21.705707 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.705707, event: started, request: pg_info(1 pgs e194477:22.12a) v4 -1685> 2014-08-13 17:23:21.705725 7fcd0964c700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:23:21.705725, event: done, request: pg_info(1 pgs e194477:22.12a) v4 -1684> 2014-08-13 17:23:21.705741 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 8 ==== pg_info(1 pgs e194477:24.3d) v4 ==== 748+0+0 (4111432871 0 0) 0x54d7e00 con 0x4f86040 -1683> 2014-08-13 17:23:21.705756 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.678817, event: header_read, request: pg_info(1 pgs e194477:24.3d) v4 -1682> 2014-08-13 17:23:21.705763 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.678818, event: throttled, request: pg_info(1 pgs e194477:24.3d) v4 -1681> 2014-08-13 17:23:21.705770 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.678874, event: all_read, request: pg_info(1 pgs e194477:24.3d) v4 -1680> 2014-08-13 17:23:21.705775 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.705754, event: dispatched, request: pg_info(1 pgs e194477:24.3d) v4 -1679> 2014-08-13 17:23:21.705782 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.705781, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.3d) v4 -1678> 2014-08-13 17:23:21.705761 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194477/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.065526 10 0.000796 -1677> 2014-08-13 17:23:21.705789 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.705789, event: started, request: pg_info(1 pgs e194477:24.3d) v4 -1676> 2014-08-13 17:23:21.705798 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194477/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1675> 2014-08-13 17:23:21.705810 7fcd0964c700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:23:21.705810, event: done, request: pg_info(1 pgs e194477:24.3d) v4 -1674> 2014-08-13 17:23:21.705790 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.014595 6 0.000207 -1673> 2014-08-13 17:23:21.705830 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 4 ==== pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 ==== 5462+0+0 (1371705662 0 0) 0x56df800 con 0x4f87900 -1672> 2014-08-13 17:23:21.705829 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1671> 2014-08-13 17:23:21.705848 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.702698, event: header_read, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1670> 2014-08-13 17:23:21.705862 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.702700, event: throttled, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1669> 2014-08-13 17:23:21.705874 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.702868, event: all_read, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1668> 2014-08-13 17:23:21.705885 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.705846, event: dispatched, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1667> 2014-08-13 17:23:21.705896 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.705896, event: waiting_for_osdmap, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1666> 2014-08-13 17:23:21.705892 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.036379 8 0.000300 -1665> 2014-08-13 17:23:21.705909 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.705909, event: started, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1664> 2014-08-13 17:23:21.705929 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -1663> 2014-08-13 17:23:21.705942 7fcd0964c700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:23:21.705942, event: done, request: pg_log(22.48e epoch 194477 log log((194215'543,194215'543], crt=194215'543) query_epoch 194477) v4 -1662> 2014-08-13 17:23:21.705932 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.010888 5 0.000170 -1661> 2014-08-13 17:23:21.705959 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.286 192.168.1.113:6820/19173 8 ==== pg_info(1 pgs e194477:22.3d5) v4 ==== 714+0+0 (2175247549 0 0) 0x5889c00 con 0x4fdad60 -1660> 2014-08-13 17:23:21.705965 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1659> 2014-08-13 17:23:21.705978 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.686158, event: header_read, request: pg_info(1 pgs e194477:22.3d5) v4 -1658> 2014-08-13 17:23:21.706000 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.686160, event: throttled, request: pg_info(1 pgs e194477:22.3d5) v4 -1657> 2014-08-13 17:23:21.706011 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.686238, event: all_read, request: pg_info(1 pgs e194477:22.3d5) v4 -1656> 2014-08-13 17:23:21.706020 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.705976, event: dispatched, request: pg_info(1 pgs e194477:22.3d5) v4 -1655> 2014-08-13 17:23:21.706003 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000074 0 0.000000 -1654> 2014-08-13 17:23:21.706035 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.706035, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.3d5) v4 -1653> 2014-08-13 17:23:21.706044 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.706043, event: started, request: pg_info(1 pgs e194477:22.3d5) v4 -1652> 2014-08-13 17:23:21.706048 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -1651> 2014-08-13 17:23:21.706069 7fcd0964c700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:23:21.706069, event: done, request: pg_info(1 pgs e194477:22.3d5) v4 -1650> 2014-08-13 17:23:21.706092 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.51 192.168.1.112:6832/6655 6 ==== pg_info(1 pgs e194477:22.537) v4 ==== 714+0+0 (1356600570 0 0) 0x5888380 con 0x4fda520 -1649> 2014-08-13 17:23:21.706108 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.698283, event: header_read, request: pg_info(1 pgs e194477:22.537) v4 -1648> 2014-08-13 17:23:21.706115 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.698285, event: throttled, request: pg_info(1 pgs e194477:22.537) v4 -1647> 2014-08-13 17:23:21.706121 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.698374, event: all_read, request: pg_info(1 pgs e194477:22.537) v4 -1646> 2014-08-13 17:23:21.706126 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.706106, event: dispatched, request: pg_info(1 pgs e194477:22.537) v4 -1645> 2014-08-13 17:23:21.706133 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.706132, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.537) v4 -1644> 2014-08-13 17:23:21.706140 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.706140, event: started, request: pg_info(1 pgs e194477:22.537) v4 -1643> 2014-08-13 17:23:21.706172 7fcd0964c700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:23:21.706172, event: done, request: pg_info(1 pgs e194477:22.537) v4 -1642> 2014-08-13 17:23:21.706169 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.041046 8 0.000249 -1641> 2014-08-13 17:23:21.706200 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 14 ==== pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 ==== 3468+0+0 (3743530752 0 0) 0x58c8600 con 0x4fd32e0 -1640> 2014-08-13 17:23:21.706207 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1639> 2014-08-13 17:23:21.706219 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.705089, event: header_read, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1638> 2014-08-13 17:23:21.706229 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.705091, event: throttled, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1637> 2014-08-13 17:23:21.706236 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.705218, event: all_read, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1636> 2014-08-13 17:23:21.706244 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.706216, event: dispatched, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1635> 2014-08-13 17:23:21.706240 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000033 0 0.000000 -1634> 2014-08-13 17:23:21.706253 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.706253, event: waiting_for_osdmap, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1633> 2014-08-13 17:23:21.706246 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.289 192.168.1.113:6808/15284 -- MRecoveryReserve GRANT pgid: 24.5e, query_epoch: 194477 v2 -- ?+0 0x588eac0 -1632> 2014-08-13 17:23:21.706264 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.706263, event: started, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1631> 2014-08-13 17:23:21.706264 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194474 194476/194476/194476) [4,50,289] r=0 lpr=194476 pi=194470-194475/2 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1630> 2014-08-13 17:23:21.706280 7fcd0964c700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:23:21.706280, event: done, request: pg_log(22.39d epoch 194477 log log((194215'480,194215'480], crt=194215'480) query_epoch 194477) v4 -1629> 2014-08-13 17:23:21.706275 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.001646 1 0.000087 -1628> 2014-08-13 17:23:21.706296 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.380 192.168.1.114:6822/51844 4 ==== pg_notify(22.4df(22) epoch 194477) v5 ==== 1944+0+0 (140617638 0 0) 0x588e200 con 0x56b1ce0 -1627> 2014-08-13 17:23:21.706298 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194477:22.6bd) v4 -- ?+0 0x56aaf40 -1626> 2014-08-13 17:23:21.706311 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.701262, event: header_read, request: pg_notify(22.4df(22) epoch 194477) v5 -1625> 2014-08-13 17:23:21.706305 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194476 pi=193125-194475/39 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] enter Started/ReplicaActive/RepRecovering -1624> 2014-08-13 17:23:21.706318 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.701264, event: throttled, request: pg_notify(22.4df(22) epoch 194477) v5 -1623> 2014-08-13 17:23:21.706324 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.701368, event: all_read, request: pg_notify(22.4df(22) epoch 194477) v5 -1622> 2014-08-13 17:23:21.706330 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.706309, event: dispatched, request: pg_notify(22.4df(22) epoch 194477) v5 -1621> 2014-08-13 17:23:21.706341 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.706341, event: waiting_for_osdmap, request: pg_notify(22.4df(22) epoch 194477) v5 -1620> 2014-08-13 17:23:21.706354 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.706354, event: started, request: pg_notify(22.4df(22) epoch 194477) v5 -1619> 2014-08-13 17:23:21.706349 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194477:22.6bd) v4 -- ?+0 0x56ad080 -1618> 2014-08-13 17:23:21.706379 7fcd0964c700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:23:21.706378, event: done, request: pg_notify(22.4df(22) epoch 194477) v5 -1617> 2014-08-13 17:23:21.706400 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.366 192.168.1.108:6828/4534 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x51e3180 con 0x5befa60 -1616> 2014-08-13 17:23:21.706420 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -1615> 2014-08-13 17:23:21.706439 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.0 192.168.1.115:6810/59615 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x52ba640 con 0x5bef900 -1614> 2014-08-13 17:23:21.706454 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -1613> 2014-08-13 17:23:21.706499 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.10 192.168.1.115:6815/60573 2 ==== pg_notify(24.30b(29) epoch 194477) v5 ==== 2379+0+0 (2275116288 0 0) 0x588d940 con 0x4e30580 -1612> 2014-08-13 17:23:21.706514 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.698879, event: header_read, request: pg_notify(24.30b(29) epoch 194477) v5 -1611> 2014-08-13 17:23:21.706522 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.698880, event: throttled, request: pg_notify(24.30b(29) epoch 194477) v5 -1610> 2014-08-13 17:23:21.706528 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.698999, event: all_read, request: pg_notify(24.30b(29) epoch 194477) v5 -1609> 2014-08-13 17:23:21.706534 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.706512, event: dispatched, request: pg_notify(24.30b(29) epoch 194477) v5 -1608> 2014-08-13 17:23:21.706540 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.706540, event: waiting_for_osdmap, request: pg_notify(24.30b(29) epoch 194477) v5 -1607> 2014-08-13 17:23:21.706548 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.706548, event: started, request: pg_notify(24.30b(29) epoch 194477) v5 -1606> 2014-08-13 17:23:21.706575 7fcd0964c700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:23:21.706575, event: done, request: pg_notify(24.30b(29) epoch 194477) v5 -1605> 2014-08-13 17:23:21.706599 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 11 ==== pg_notify(24.34(50) epoch 194477) v5 ==== 3306+0+0 (1066846088 0 0) 0x54d6200 con 0x4e35540 -1604> 2014-08-13 17:23:21.706615 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.677904, event: header_read, request: pg_notify(24.34(50) epoch 194477) v5 -1603> 2014-08-13 17:23:21.706622 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.677905, event: throttled, request: pg_notify(24.34(50) epoch 194477) v5 -1602> 2014-08-13 17:23:21.706628 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.678030, event: all_read, request: pg_notify(24.34(50) epoch 194477) v5 -1601> 2014-08-13 17:23:21.706634 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.706613, event: dispatched, request: pg_notify(24.34(50) epoch 194477) v5 -1600> 2014-08-13 17:23:21.706640 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.706640, event: waiting_for_osdmap, request: pg_notify(24.34(50) epoch 194477) v5 -1599> 2014-08-13 17:23:21.706648 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.706647, event: started, request: pg_notify(24.34(50) epoch 194477) v5 -1598> 2014-08-13 17:23:21.706627 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 luod=0'0 crt=0'0 lcod 0'0 active+remapped] exit Started/ReplicaActive/RepNotRecovering 0.031811 3 0.000162 -1597> 2014-08-13 17:23:21.706669 7fcd0964c700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:23:21.706669, event: done, request: pg_notify(24.34(50) epoch 194477) v5 -1596> 2014-08-13 17:23:21.706661 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194476 pi=194462-194475/5 luod=0'0 crt=0'0 lcod 0'0 active+remapped] enter Started/ReplicaActive/RepWaitBackfillReserved -1595> 2014-08-13 17:23:21.706704 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 16 ==== pg_info(1 pgs e194477:24.350) v4 ==== 748+0+0 (1649174343 0 0) 0x54d4440 con 0x4ec4780 -1594> 2014-08-13 17:23:21.706719 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706363, event: header_read, request: pg_info(1 pgs e194477:24.350) v4 -1593> 2014-08-13 17:23:21.706726 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706365, event: throttled, request: pg_info(1 pgs e194477:24.350) v4 -1592> 2014-08-13 17:23:21.706731 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706466, event: all_read, request: pg_info(1 pgs e194477:24.350) v4 -1591> 2014-08-13 17:23:21.706736 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706717, event: dispatched, request: pg_info(1 pgs e194477:24.350) v4 -1590> 2014-08-13 17:23:21.706742 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706742, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:24.350) v4 -1589> 2014-08-13 17:23:21.706750 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706750, event: started, request: pg_info(1 pgs e194477:24.350) v4 -1588> 2014-08-13 17:23:21.706775 7fcd0964c700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:23:21.706775, event: done, request: pg_info(1 pgs e194477:24.350) v4 -1587> 2014-08-13 17:23:21.706792 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 9 ==== pg_notify(22.3af(27) epoch 194477) v5 ==== 2231+0+0 (1918832129 0 0) 0x54d1a40 con 0x4f86040 -1586> 2014-08-13 17:23:21.706811 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.678885, event: header_read, request: pg_notify(22.3af(27) epoch 194477) v5 -1585> 2014-08-13 17:23:21.706821 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.678886, event: throttled, request: pg_notify(22.3af(27) epoch 194477) v5 -1584> 2014-08-13 17:23:21.706830 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.678982, event: all_read, request: pg_notify(22.3af(27) epoch 194477) v5 -1583> 2014-08-13 17:23:21.706839 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.706809, event: dispatched, request: pg_notify(22.3af(27) epoch 194477) v5 -1582> 2014-08-13 17:23:21.706849 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.706849, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194477) v5 -1581> 2014-08-13 17:23:21.706860 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.706860, event: started, request: pg_notify(22.3af(27) epoch 194477) v5 -1580> 2014-08-13 17:23:21.706885 7fcd0964c700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:23:21.706884, event: done, request: pg_notify(22.3af(27) epoch 194477) v5 -1579> 2014-08-13 17:23:21.706913 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.366 192.168.1.108:6828/4534 2 ==== pg_notify(22.39d(5) epoch 194477) v5 ==== 1001+0+0 (4286074226 0 0) 0x56ac0c0 con 0x5befa60 -1578> 2014-08-13 17:23:21.706936 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706062, event: header_read, request: pg_notify(22.39d(5) epoch 194477) v5 -1577> 2014-08-13 17:23:21.706943 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706065, event: throttled, request: pg_notify(22.39d(5) epoch 194477) v5 -1576> 2014-08-13 17:23:21.706952 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706203, event: all_read, request: pg_notify(22.39d(5) epoch 194477) v5 -1575> 2014-08-13 17:23:21.706961 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706933, event: dispatched, request: pg_notify(22.39d(5) epoch 194477) v5 -1574> 2014-08-13 17:23:21.706970 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706970, event: waiting_for_osdmap, request: pg_notify(22.39d(5) epoch 194477) v5 -1573> 2014-08-13 17:23:21.706981 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.706981, event: started, request: pg_notify(22.39d(5) epoch 194477) v5 -1572> 2014-08-13 17:23:21.707004 7fcd0964c700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:23:21.707004, event: done, request: pg_notify(22.39d(5) epoch 194477) v5 -1571> 2014-08-13 17:23:21.707022 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.0 192.168.1.115:6810/59615 2 ==== pg_notify(22.775(26) epoch 194477) v5 ==== 2172+0+0 (3521734717 0 0) 0x56abf00 con 0x5bef900 -1570> 2014-08-13 17:23:21.707042 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.704074, event: header_read, request: pg_notify(22.775(26) epoch 194477) v5 -1569> 2014-08-13 17:23:21.707052 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.704076, event: throttled, request: pg_notify(22.775(26) epoch 194477) v5 -1568> 2014-08-13 17:23:21.707061 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.704184, event: all_read, request: pg_notify(22.775(26) epoch 194477) v5 -1567> 2014-08-13 17:23:21.707070 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.707039, event: dispatched, request: pg_notify(22.775(26) epoch 194477) v5 -1566> 2014-08-13 17:23:21.707079 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.707079, event: waiting_for_osdmap, request: pg_notify(22.775(26) epoch 194477) v5 -1565> 2014-08-13 17:23:21.707090 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.707090, event: started, request: pg_notify(22.775(26) epoch 194477) v5 -1564> 2014-08-13 17:23:21.707113 7fcd0964c700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:23:21.707112, event: done, request: pg_notify(22.775(26) epoch 194477) v5 -1563> 2014-08-13 17:23:21.707138 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.10 192.168.1.115:6815/60573 3 ==== pg_notify(24.30b(29) epoch 194477) v5 ==== 2379+0+0 (2275116288 0 0) 0x56a9dc0 con 0x4e30580 -1562> 2014-08-13 17:23:21.707157 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.703980, event: header_read, request: pg_notify(24.30b(29) epoch 194477) v5 -1561> 2014-08-13 17:23:21.707164 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.703982, event: throttled, request: pg_notify(24.30b(29) epoch 194477) v5 -1560> 2014-08-13 17:23:21.707170 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.704107, event: all_read, request: pg_notify(24.30b(29) epoch 194477) v5 -1559> 2014-08-13 17:23:21.707175 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.707155, event: dispatched, request: pg_notify(24.30b(29) epoch 194477) v5 -1558> 2014-08-13 17:23:21.707181 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.707181, event: waiting_for_osdmap, request: pg_notify(24.30b(29) epoch 194477) v5 -1557> 2014-08-13 17:23:21.707189 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.707188, event: started, request: pg_notify(24.30b(29) epoch 194477) v5 -1556> 2014-08-13 17:23:21.707217 7fcd0964c700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:23:21.707217, event: done, request: pg_notify(24.30b(29) epoch 194477) v5 -1555> 2014-08-13 17:23:21.707235 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 12 ==== pg_info(1 pgs e194477:1.2b) v4 ==== 706+0+0 (700365585 0 0) 0x588a840 con 0x4e35540 -1554> 2014-08-13 17:23:21.707257 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.678229, event: header_read, request: pg_info(1 pgs e194477:1.2b) v4 -1553> 2014-08-13 17:23:21.707268 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.678231, event: throttled, request: pg_info(1 pgs e194477:1.2b) v4 -1552> 2014-08-13 17:23:21.707274 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.678296, event: all_read, request: pg_info(1 pgs e194477:1.2b) v4 -1551> 2014-08-13 17:23:21.707267 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:2.2a) v4 -- ?+0 0x4e50000 -1550> 2014-08-13 17:23:21.707280 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.707254, event: dispatched, request: pg_info(1 pgs e194477:1.2b) v4 -1549> 2014-08-13 17:23:21.707322 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.707322, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:1.2b) v4 -1548> 2014-08-13 17:23:21.707311 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194477 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194477) v4 -- ?+0 0x56dda00 con 0x4ec4780 -1547> 2014-08-13 17:23:21.707349 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.707349, event: started, request: pg_info(1 pgs e194477:1.2b) v4 -1546> 2014-08-13 17:23:21.707381 7fcd0964c700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:23:21.707381, event: done, request: pg_info(1 pgs e194477:1.2b) v4 -1545> 2014-08-13 17:23:21.707406 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 17 ==== MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 ==== 26+0+0 (1306618377 0 0) 0x588d080 con 0x4ec4780 -1544> 2014-08-13 17:23:21.707417 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.291 192.168.1.101:6820/25633 -- pg_info(1 pgs e194477:24.9b) v4 -- ?+0 0x56ade80 -1543> 2014-08-13 17:23:21.707431 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707221, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1542> 2014-08-13 17:23:21.707440 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707223, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1541> 2014-08-13 17:23:21.707449 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707289, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1540> 2014-08-13 17:23:21.707457 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.350 192.168.1.114:6832/53185 -- pg_info(1 pgs e194477:24.9b) v4 -- ?+0 0x56af1c0 -1539> 2014-08-13 17:23:21.707458 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707428, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1538> 2014-08-13 17:23:21.707483 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707483, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1537> 2014-08-13 17:23:21.707488 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194477:24.9b) v4 -- ?+0 0x56ab640 -1536> 2014-08-13 17:23:21.707512 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.378 192.168.1.102:6840/19195 -- pg_info(1 pgs e194477:24.9b) v4 -- ?+0 0x56aee40 -1535> 2014-08-13 17:23:21.707502 7fcd0964c700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:23:21.707501, event: done, request: MRecoveryReserve REQUEST pgid: 24.350, query_epoch: 194477 v2 -1534> 2014-08-13 17:23:21.707627 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_query(22.3af epoch 194477) v3 -- ?+0 0x54f92c0 con 0x4eb0c60 -1533> 2014-08-13 17:23:21.707659 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6842/8170 -- pg_query(22.3af epoch 194477) v3 -- ?+0 0x54fc740 con 0x4f86040 -1532> 2014-08-13 17:23:21.707806 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.004438 5 0.000217 -1531> 2014-08-13 17:23:21.707838 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1530> 2014-08-13 17:23:21.707893 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetInfo 0.068807 8 0.000285 -1529> 2014-08-13 17:23:21.707923 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetLog -1528> 2014-08-13 17:23:21.708003 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetLog 0.000080 0 0.000000 -1527> 2014-08-13 17:23:21.708031 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetMissing -1526> 2014-08-13 17:23:21.708060 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 -1525> 2014-08-13 17:23:21.708085 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/WaitUpThru -1524> 2014-08-13 17:23:21.708140 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194477/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.046173 11 0.000753 -1523> 2014-08-13 17:23:21.708180 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194477/194466 194476/194476/194476) [4,339,45] r=0 lpr=194476 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1522> 2014-08-13 17:23:21.708211 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194477/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.067508 11 0.000872 -1521> 2014-08-13 17:23:21.708235 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194477/194466 194476/194476/194476) [4,356,357] r=0 lpr=194476 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1520> 2014-08-13 17:23:21.708291 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.055124 9 0.003528 -1519> 2014-08-13 17:23:21.708320 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/194476) [4,312,285] r=0 lpr=194476 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1518> 2014-08-13 17:23:21.708408 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:1.2b) v4 -- ?+0 0x56aa140 -1517> 2014-08-13 17:23:21.708578 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194477:24.3d) v4 -- ?+0 0x5888000 -1516> 2014-08-13 17:23:21.708661 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.25 192.168.1.103:6800/36920 -- pg_info(1 pgs e194477:24.3d) v4 -- ?+0 0x588abc0 -1515> 2014-08-13 17:23:21.708692 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.303 192.168.1.109:6802/30768 -- pg_info(1 pgs e194477:24.3d) v4 -- ?+0 0x5889a40 -1514> 2014-08-13 17:23:21.708691 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 10 ==== pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 ==== 3560+0+0 (3974518537 0 0) 0x56de600 con 0x4f86040 -1513> 2014-08-13 17:23:21.708720 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708519, event: header_read, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1512> 2014-08-13 17:23:21.708722 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194477:24.3d) v4 -- ?+0 0x588cec0 -1511> 2014-08-13 17:23:21.708738 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708520, event: throttled, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1510> 2014-08-13 17:23:21.708780 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708661, event: all_read, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1509> 2014-08-13 17:23:21.708790 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708716, event: dispatched, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1508> 2014-08-13 17:23:21.708798 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708798, event: waiting_for_osdmap, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1507> 2014-08-13 17:23:21.708809 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708809, event: started, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1506> 2014-08-13 17:23:21.708828 7fcd0964c700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:23:21.708828, event: done, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1505> 2014-08-13 17:23:21.708846 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 8 ==== pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 ==== 3560+0+0 (1483714220 0 0) 0x58c9e00 con 0x4eb0c60 -1504> 2014-08-13 17:23:21.708868 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708671, event: header_read, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1503> 2014-08-13 17:23:21.708881 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708672, event: throttled, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1502> 2014-08-13 17:23:21.708857 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194477/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.003059 1 0.000074 -1501> 2014-08-13 17:23:21.708896 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708810, event: all_read, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1500> 2014-08-13 17:23:21.708904 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708865, event: dispatched, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1499> 2014-08-13 17:23:21.708916 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708915, event: waiting_for_osdmap, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1498> 2014-08-13 17:23:21.708908 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194477/194465 194476/194476/194476) [4,301,324] r=0 lpr=194476 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved -1497> 2014-08-13 17:23:21.708929 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708928, event: started, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1496> 2014-08-13 17:23:21.708966 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6814/16727 -- MRecoveryReserve REQUEST pgid: 22.22c, query_epoch: 194477 v2 -- ?+0 0x588b2c0 con 0x4e35800 -1495> 2014-08-13 17:23:21.708973 7fcd0964c700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:23:21.708973, event: done, request: pg_log(22.3af epoch 194477 log log((194215'539,194215'539], crt=194215'539) query_epoch 194477) v4 -1494> 2014-08-13 17:23:21.708779 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194477/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 -1493> 2014-08-13 17:23:21.709098 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.007535 3 0.000157 -1492> 2014-08-13 17:23:21.709130 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1491> 2014-08-13 17:23:21.709254 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194477:22.3d5) v4 -- ?+0 0x588bd40 -1490> 2014-08-13 17:23:21.709287 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194477:22.3d5) v4 -- ?+0 0x588fc40 -1489> 2014-08-13 17:23:21.709102 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194477/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 -1488> 2014-08-13 17:23:21.709465 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.040707 8 0.000395 -1487> 2014-08-13 17:23:21.709497 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1486> 2014-08-13 17:23:21.709526 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000029 0 0.000000 -1485> 2014-08-13 17:23:21.709551 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194474 194476/194476/194476) [4,340,347] r=0 lpr=194476 pi=194470-194475/2 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1484> 2014-08-13 17:23:21.709627 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:22.1b8) v4 -- ?+0 0x588f700 -1483> 2014-08-13 17:23:21.709657 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194477:22.1b8) v4 -- ?+0 0x5888700 -1482> 2014-08-13 17:23:21.709423 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194477/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 -1481> 2014-08-13 17:23:21.709928 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] exit Started/ReplicaActive/RepNotRecovering 0.007936 3 0.001202 -1480> 2014-08-13 17:23:21.709968 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194476 pi=193151-194475/26 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] enter Started/ReplicaActive/RepWaitRecoveryReserved -1479> 2014-08-13 17:23:21.710026 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194477/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.039899 11 0.000844 -1478> 2014-08-13 17:23:21.710052 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194477/194304 194476/194476/194476) [4,287,309] r=0 lpr=194476 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1477> 2014-08-13 17:23:21.710155 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.048799 6 0.001070 -1476> 2014-08-13 17:23:21.710197 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1475> 2014-08-13 17:23:21.710222 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000026 0 0.000000 -1474> 2014-08-13 17:23:21.710245 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1473> 2014-08-13 17:23:21.710278 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:2.2a) v4 -- ?+0 0x56a9500 -1472> 2014-08-13 17:23:21.710385 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.311 192.168.1.101:6816/24270 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x550de80 con 0x53077a0 -1471> 2014-08-13 17:23:21.710427 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -1470> 2014-08-13 17:23:21.710586 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.311 192.168.1.101:6816/24270 5 ==== pg_notify(22.f3(20) epoch 194477) v5 ==== 1830+0+0 (3919976725 0 0) 0x32a8000 con 0x53077a0 -1469> 2014-08-13 17:23:21.710610 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710385, event: header_read, request: pg_notify(22.f3(20) epoch 194477) v5 -1468> 2014-08-13 17:23:21.710624 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710387, event: throttled, request: pg_notify(22.f3(20) epoch 194477) v5 -1467> 2014-08-13 17:23:21.710635 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710566, event: all_read, request: pg_notify(22.f3(20) epoch 194477) v5 -1466> 2014-08-13 17:23:21.710642 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710606, event: dispatched, request: pg_notify(22.f3(20) epoch 194477) v5 -1465> 2014-08-13 17:23:21.710652 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710652, event: waiting_for_osdmap, request: pg_notify(22.f3(20) epoch 194477) v5 -1464> 2014-08-13 17:23:21.710665 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710664, event: started, request: pg_notify(22.f3(20) epoch 194477) v5 -1463> 2014-08-13 17:23:21.710389 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194477/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] exit Started/Primary/Active/Activating 0.061273 13 0.003898 -1462> 2014-08-13 17:23:21.710708 7fcd0964c700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:23:21.710707, event: done, request: pg_notify(22.f3(20) epoch 194477) v5 -1461> 2014-08-13 17:23:21.710480 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194477/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 -1460> 2014-08-13 17:23:21.710714 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194477/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194476 pi=193637-194475/26 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] enter Started/Primary/Active/WaitLocalRecoveryReserved -1459> 2014-08-13 17:23:21.711069 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.005021 4 0.000222 -1458> 2014-08-13 17:23:21.710803 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194477/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 -1457> 2014-08-13 17:23:21.711102 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1456> 2014-08-13 17:23:21.711154 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.048599 6 0.000374 -1455> 2014-08-13 17:23:21.711190 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1454> 2014-08-13 17:23:21.711215 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000024 0 0.000000 -1453> 2014-08-13 17:23:21.711237 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194474 194476/194476/194476) [4,340] r=0 lpr=194476 pi=194470-194475/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1452> 2014-08-13 17:23:21.711267 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194477:1.2b) v4 -- ?+0 0x56a8c40 -1451> 2014-08-13 17:23:21.711107 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194477/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 -1450> 2014-08-13 17:23:21.711920 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.381 192.168.1.102:6822/15390 4 ==== osd_map(194477..194477 src has 185139..194477) v3 ==== 1264+0+0 (3461956090 0 0) 0x567c140 con 0x578c620 -1449> 2014-08-13 17:23:21.711959 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194477,194477], i have 194477, src has [185139,194477] -1448> 2014-08-13 17:23:21.712178 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.381 192.168.1.102:6822/15390 5 ==== pg_notify(22.481(41) epoch 194477) v5 ==== 3013+0+0 (2516548305 0 0) 0x56ac440 con 0x578c620 -1447> 2014-08-13 17:23:21.712214 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712019, event: header_read, request: pg_notify(22.481(41) epoch 194477) v5 -1446> 2014-08-13 17:23:21.712226 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712020, event: throttled, request: pg_notify(22.481(41) epoch 194477) v5 -1445> 2014-08-13 17:23:21.712275 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712145, event: all_read, request: pg_notify(22.481(41) epoch 194477) v5 -1444> 2014-08-13 17:23:21.712285 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712207, event: dispatched, request: pg_notify(22.481(41) epoch 194477) v5 -1443> 2014-08-13 17:23:21.712295 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712295, event: waiting_for_osdmap, request: pg_notify(22.481(41) epoch 194477) v5 -1442> 2014-08-13 17:23:21.711953 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194477/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] exit Started/Primary/Active/Activating 0.053320 13 0.008001 -1441> 2014-08-13 17:23:21.712307 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712307, event: started, request: pg_notify(22.481(41) epoch 194477) v5 -1440> 2014-08-13 17:23:21.712352 7fcd0964c700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:23:21.712352, event: done, request: pg_notify(22.481(41) epoch 194477) v5 -1439> 2014-08-13 17:23:21.712325 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194477/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194476 pi=193635-194475/27 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] enter Started/Primary/Active/WaitLocalRecoveryReserved -1438> 2014-08-13 17:23:21.712644 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.044259 8 0.000330 -1437> 2014-08-13 17:23:21.712670 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1436> 2014-08-13 17:23:21.712694 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000024 0 0.000000 -1435> 2014-08-13 17:23:21.712717 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194474 194476/194476/194476) [4,343,286] r=0 lpr=194476 pi=194470-194475/2 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1434> 2014-08-13 17:23:21.712753 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194477:22.3d5) v4 -- ?+0 0x56a9880 -1433> 2014-08-13 17:23:21.712787 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194477:22.3d5) v4 -- ?+0 0x56a9a40 -1432> 2014-08-13 17:23:21.734938 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.28 192.168.1.103:6805/37391 7 ==== pg_info(1 pgs e194477:22.84) v4 ==== 714+0+0 (485479817 0 0) 0x56ad400 con 0x4e33f40 -1431> 2014-08-13 17:23:21.734975 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.734808, event: header_read, request: pg_info(1 pgs e194477:22.84) v4 -1430> 2014-08-13 17:23:21.734991 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.734810, event: throttled, request: pg_info(1 pgs e194477:22.84) v4 -1429> 2014-08-13 17:23:21.735001 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.734899, event: all_read, request: pg_info(1 pgs e194477:22.84) v4 -1428> 2014-08-13 17:23:21.735010 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.734970, event: dispatched, request: pg_info(1 pgs e194477:22.84) v4 -1427> 2014-08-13 17:23:21.735022 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.735022, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.84) v4 -1426> 2014-08-13 17:23:21.735033 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.735033, event: started, request: pg_info(1 pgs e194477:22.84) v4 -1425> 2014-08-13 17:23:21.735065 7fcd0964c700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:23:21.735065, event: done, request: pg_info(1 pgs e194477:22.84) v4 -1424> 2014-08-13 17:23:21.735214 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194477:22.84) v4 -- ?+0 0x5888380 -1423> 2014-08-13 17:23:21.735255 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194477:22.84) v4 -- ?+0 0x5889c00 -1422> 2014-08-13 17:23:21.735331 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.095192 8 0.000282 -1421> 2014-08-13 17:23:21.735362 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -1420> 2014-08-13 17:23:21.735385 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000023 0 0.000000 -1419> 2014-08-13 17:23:21.735404 7fcd01e40700 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194474 194476/194476/194476) [4,28,304] r=0 lpr=194476 pi=194470-194475/2 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -1418> 2014-08-13 17:23:21.735432 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194477:22.84) v4 -- ?+0 0x588de80 -1417> 2014-08-13 17:23:21.735604 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 12 ==== pg_info(1 pgs e194477:22.14a) v4 ==== 714+0+0 (2348437832 0 0) 0x588b9c0 con 0x4e335a0 -1416> 2014-08-13 17:23:21.735640 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735347, event: header_read, request: pg_info(1 pgs e194477:22.14a) v4 -1415> 2014-08-13 17:23:21.735651 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735350, event: throttled, request: pg_info(1 pgs e194477:22.14a) v4 -1414> 2014-08-13 17:23:21.735657 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735434, event: all_read, request: pg_info(1 pgs e194477:22.14a) v4 -1413> 2014-08-13 17:23:21.735663 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735636, event: dispatched, request: pg_info(1 pgs e194477:22.14a) v4 -1412> 2014-08-13 17:23:21.735670 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735669, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.14a) v4 -1411> 2014-08-13 17:23:21.735679 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735679, event: started, request: pg_info(1 pgs e194477:22.14a) v4 -1410> 2014-08-13 17:23:21.735709 7fcd0964c700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:23:21.735709, event: done, request: pg_info(1 pgs e194477:22.14a) v4 -1409> 2014-08-13 17:23:21.735755 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194477:22.84) v4 -- ?+0 0x588c440 -1408> 2014-08-13 17:23:21.736344 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.58 192.168.1.111:6834/14316 13 ==== pg_info(1 pgs e194477:22.14a) v4 ==== 714+0+0 (3980288637 0 0) 0x588f8c0 con 0x4e335a0 -1407> 2014-08-13 17:23:21.736374 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.735976, event: header_read, request: pg_info(1 pgs e194477:22.14a) v4 -1406> 2014-08-13 17:23:21.736387 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.735978, event: throttled, request: pg_info(1 pgs e194477:22.14a) v4 -1405> 2014-08-13 17:23:21.736399 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.736052, event: all_read, request: pg_info(1 pgs e194477:22.14a) v4 -1404> 2014-08-13 17:23:21.736409 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.736370, event: dispatched, request: pg_info(1 pgs e194477:22.14a) v4 -1403> 2014-08-13 17:23:21.736419 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.736419, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.14a) v4 -1402> 2014-08-13 17:23:21.736433 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.736432, event: started, request: pg_info(1 pgs e194477:22.14a) v4 -1401> 2014-08-13 17:23:21.736471 7fcd0964c700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:23:21.736471, event: done, request: pg_info(1 pgs e194477:22.14a) v4 -1400> 2014-08-13 17:23:21.739319 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 12 ==== pg_info(1 pgs e194477:22.1d2) v4 ==== 714+0+0 (2346202861 0 0) 0x588e580 con 0x4e35ac0 -1399> 2014-08-13 17:23:21.739358 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739175, event: header_read, request: pg_info(1 pgs e194477:22.1d2) v4 -1398> 2014-08-13 17:23:21.739373 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739177, event: throttled, request: pg_info(1 pgs e194477:22.1d2) v4 -1397> 2014-08-13 17:23:21.739383 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739274, event: all_read, request: pg_info(1 pgs e194477:22.1d2) v4 -1396> 2014-08-13 17:23:21.739392 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739353, event: dispatched, request: pg_info(1 pgs e194477:22.1d2) v4 -1395> 2014-08-13 17:23:21.739403 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739403, event: waiting_for_osdmap, request: pg_info(1 pgs e194477:22.1d2) v4 -1394> 2014-08-13 17:23:21.739416 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739416, event: started, request: pg_info(1 pgs e194477:22.1d2) v4 -1393> 2014-08-13 17:23:21.739452 7fcd0964c700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:23:21.739452, event: done, request: pg_info(1 pgs e194477:22.1d2) v4 -1392> 2014-08-13 17:23:21.739483 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194477:22.1d2) v4 -- ?+0 0x56adb00 -1391> 2014-08-13 17:23:21.739522 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194477:22.1d2) v4 -- ?+0 0x56af380 -1390> 2014-08-13 17:23:21.739824 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194477/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.079742 11 0.000828 -1389> 2014-08-13 17:23:21.739857 7fcd02841700 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194477/194466 194476/194476/194476) [4,349,61] r=0 lpr=194476 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -1388> 2014-08-13 17:23:21.777132 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.313 192.168.1.109:0/35445 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.776992) v2 ==== 47+0+0 (4066663083 0 0) 0x56ab480 con 0x546d120 -1387> 2014-08-13 17:23:21.777222 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.109:0/35445 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.776992) v2 -- ?+0 0x56aa4c0 con 0x546d120 -1386> 2014-08-13 17:23:21.777538 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.313 192.168.1.109:0/35445 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.776992) v2 ==== 47+0+0 (4066663083 0 0) 0x56afa80 con 0x546bc80 -1385> 2014-08-13 17:23:21.777587 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.109:0/35445 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.776992) v2 -- ?+0 0x56abd40 con 0x546bc80 -1384> 2014-08-13 17:23:21.785695 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.373 192.168.1.108:0/7435 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.785218) v2 ==== 47+0+0 (321411781 0 0) 0x56afc40 con 0x592dee0 -1383> 2014-08-13 17:23:21.785737 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.785218) v2 -- ?+0 0x56afa80 con 0x592dee0 -1382> 2014-08-13 17:23:21.785740 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.373 192.168.1.108:0/7435 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.785218) v2 ==== 47+0+0 (321411781 0 0) 0x56a96c0 con 0x592c200 -1381> 2014-08-13 17:23:21.785838 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.108:0/7435 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.785218) v2 -- ?+0 0x56ab480 con 0x592c200 -1380> 2014-08-13 17:23:21.787289 7fcce8615700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5338f00 sd=213 :6819 s=0 pgs=0 cs=0 l=0 c=0x5bef7a0).accept sd=213 192.168.1.115:53812/0 -1379> 2014-08-13 17:23:21.792114 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.792307) v2 ==== 47+0+0 (2841339678 0 0) 0x54d32c0 con 0x57f65c0 -1378> 2014-08-13 17:23:21.792124 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.792307) v2 ==== 47+0+0 (2841339678 0 0) 0x54d0fc0 con 0x57f6e00 -1377> 2014-08-13 17:23:21.792174 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.792307) v2 -- ?+0 0x56afc40 con 0x57f65c0 -1376> 2014-08-13 17:23:21.792236 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.792307) v2 -- ?+0 0x56a96c0 con 0x57f6e00 -1375> 2014-08-13 17:23:21.815786 7fccdd161700 1 -- 192.168.1.115:6819/31924 >> :/0 pipe(0x5339b80 sd=214 :6819 s=0 pgs=0 cs=0 l=0 c=0x5bef640).accept sd=214 192.168.1.112:41551/0 -1374> 2014-08-13 17:23:21.816846 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.48 192.168.1.112:6838/8300 1 ==== osd_map(194473..194477 src has 185139..194477) v3 ==== 4073+0+0 (468953081 0 0) 0x550fbc0 con 0x5bef640 -1373> 2014-08-13 17:23:21.816896 7fcd0964c700 3 osd.4 194477 handle_osd_map epochs [194473,194477], i have 194477, src has [185139,194477] -1372> 2014-08-13 17:23:21.817006 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.48 192.168.1.112:6838/8300 2 ==== pg_notify(22.619(26) epoch 194477) v5 ==== 2172+0+0 (3799552523 0 0) 0x4e55400 con 0x5bef640 -1371> 2014-08-13 17:23:21.817051 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.816811, event: header_read, request: pg_notify(22.619(26) epoch 194477) v5 -1370> 2014-08-13 17:23:21.817064 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.816813, event: throttled, request: pg_notify(22.619(26) epoch 194477) v5 -1369> 2014-08-13 17:23:21.817070 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.816938, event: all_read, request: pg_notify(22.619(26) epoch 194477) v5 -1368> 2014-08-13 17:23:21.817076 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.817044, event: dispatched, request: pg_notify(22.619(26) epoch 194477) v5 -1367> 2014-08-13 17:23:21.817084 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.817084, event: waiting_for_osdmap, request: pg_notify(22.619(26) epoch 194477) v5 -1366> 2014-08-13 17:23:21.817097 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.817097, event: started, request: pg_notify(22.619(26) epoch 194477) v5 -1365> 2014-08-13 17:23:21.817137 7fcd0964c700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:23:21.817137, event: done, request: pg_notify(22.619(26) epoch 194477) v5 -1364> 2014-08-13 17:23:21.863694 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.863178) v2 ==== 47+0+0 (3260260108 0 0) 0x4e50c40 con 0x55f2940 -1363> 2014-08-13 17:23:21.863751 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.863178) v2 -- ?+0 0x54d32c0 con 0x55f2940 -1362> 2014-08-13 17:23:21.863769 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.863178) v2 ==== 47+0+0 (3260260108 0 0) 0x56a96c0 con 0x55f3020 -1361> 2014-08-13 17:23:21.863819 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.863178) v2 -- ?+0 0x54d0fc0 con 0x55f3020 -1360> 2014-08-13 17:23:21.866743 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.866485) v2 ==== 47+0+0 (3983175176 0 0) 0x56afc40 con 0x5468b00 -1359> 2014-08-13 17:23:21.866800 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.866485) v2 -- ?+0 0x4e50c40 con 0x5468b00 -1358> 2014-08-13 17:23:21.866747 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.866485) v2 ==== 47+0+0 (3983175176 0 0) 0x54d1340 con 0x5469fa0 -1357> 2014-08-13 17:23:21.866907 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.866485) v2 -- ?+0 0x56a96c0 con 0x5469fa0 -1356> 2014-08-13 17:23:21.902686 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.902487) v2 ==== 47+0+0 (869941864 0 0) 0x56ac7c0 con 0x59298c0 -1355> 2014-08-13 17:23:21.902687 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.285 192.168.100.101:0/26295 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.902487) v2 ==== 47+0+0 (869941864 0 0) 0x588f1c0 con 0x5928420 -1354> 2014-08-13 17:23:21.902740 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.902487) v2 -- ?+0 0x56afc40 con 0x59298c0 -1353> 2014-08-13 17:23:21.902913 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.101:0/26295 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.902487) v2 -- ?+0 0x54d1340 con 0x5928420 -1352> 2014-08-13 17:23:21.936816 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.936663) v2 ==== 47+0+0 (878165000 0 0) 0x56a9a40 con 0x592b5a0 -1351> 2014-08-13 17:23:21.936858 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.936663) v2 ==== 47+0+0 (878165000 0 0) 0x4e50000 con 0x592ca40 -1350> 2014-08-13 17:23:21.936889 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.936663) v2 -- ?+0 0x56ac7c0 con 0x592b5a0 -1349> 2014-08-13 17:23:21.936965 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.936663) v2 -- ?+0 0x588f1c0 con 0x592ca40 -1348> 2014-08-13 17:23:21.980318 7fcd18aa9700 5 asok(0x2e78230) AdminSocket: request 'get_command_descriptions' '' to 0x2e30150 returned 2965 bytes -1347> 2014-08-13 17:23:21.981909 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.9 192.168.100.107:0/53997 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.981628) v2 ==== 47+0+0 (3014500964 0 0) 0x56ad940 con 0x592b020 -1346> 2014-08-13 17:23:21.981964 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.981628) v2 -- ?+0 0x56a9a40 con 0x592b020 -1345> 2014-08-13 17:23:21.981946 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.9 192.168.100.107:0/53997 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:21.981628) v2 ==== 47+0+0 (3014500964 0 0) 0x56aa140 con 0x592c4c0 -1344> 2014-08-13 17:23:21.982098 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:21.981628) v2 -- ?+0 0x4e50000 con 0x592c4c0 -1343> 2014-08-13 17:23:22.008481 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.007606) v2 ==== 47+0+0 (2227448133 0 0) 0x5888c40 con 0x552a940 -1342> 2014-08-13 17:23:22.008501 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.345 192.168.1.110:0/46337 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.007606) v2 ==== 47+0+0 (2227448133 0 0) 0x5889c00 con 0x5306e00 -1341> 2014-08-13 17:23:22.008532 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.007606) v2 -- ?+0 0x56ad940 con 0x552a940 -1340> 2014-08-13 17:23:22.008589 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.110:0/46337 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.007606) v2 -- ?+0 0x56aa140 con 0x5306e00 -1339> 2014-08-13 17:23:22.010639 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.50 192.168.100.112:0/9607 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.010439) v2 ==== 47+0+0 (2520565625 0 0) 0x588c440 con 0x546f640 -1338> 2014-08-13 17:23:22.010684 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.112:0/9607 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.010439) v2 -- ?+0 0x5889c00 con 0x546f640 -1337> 2014-08-13 17:23:22.010843 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.50 192.168.100.112:0/9607 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.010439) v2 ==== 47+0+0 (2520565625 0 0) 0x588fc40 con 0x59491e0 -1336> 2014-08-13 17:23:22.010882 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.112:0/9607 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.010439) v2 -- ?+0 0x5888c40 con 0x59491e0 -1335> 2014-08-13 17:23:22.012174 7fcd18aa9700 5 asok(0x2e78230) AdminSocket: request 'version' '' to 0x2e280b0 returned 20 bytes -1334> 2014-08-13 17:23:22.090534 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.089997) v2 ==== 47+0+0 (434881882 0 0) 0x588bf00 con 0x59286e0 -1333> 2014-08-13 17:23:22.090582 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.089997) v2 -- ?+0 0x588c440 con 0x59286e0 -1332> 2014-08-13 17:23:22.090637 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.089997) v2 ==== 47+0+0 (434881882 0 0) 0x588d5c0 con 0x57f7d20 -1331> 2014-08-13 17:23:22.090675 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.089997) v2 -- ?+0 0x588fc40 con 0x57f7d20 -1330> 2014-08-13 17:23:22.114106 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.20 192.168.1.103:0/38269 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.113704) v2 ==== 47+0+0 (3230095801 0 0) 0x54d0000 con 0x59fa260 -1329> 2014-08-13 17:23:22.114151 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.113704) v2 -- ?+0 0x588bf00 con 0x59fa260 -1328> 2014-08-13 17:23:22.114149 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.20 192.168.1.103:0/38269 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.113704) v2 ==== 47+0+0 (3230095801 0 0) 0x588f700 con 0x59f8f20 -1327> 2014-08-13 17:23:22.114204 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.103:0/38269 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.113704) v2 -- ?+0 0x588d5c0 con 0x59f8f20 -1326> 2014-08-13 17:23:22.151921 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.338 192.168.100.106:0/25893 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.151246) v2 ==== 47+0+0 (3644479092 0 0) 0x54d16c0 con 0x546b9c0 -1325> 2014-08-13 17:23:22.151971 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.106:0/25893 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.151246) v2 -- ?+0 0x588f700 con 0x546b9c0 -1324> 2014-08-13 17:23:22.151939 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.338 192.168.100.106:0/25893 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.151246) v2 ==== 47+0+0 (3644479092 0 0) 0x54d3480 con 0x546ce60 -1323> 2014-08-13 17:23:22.152017 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.106:0/25893 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.151246) v2 -- ?+0 0x54d0000 con 0x546ce60 -1322> 2014-08-13 17:23:22.174215 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.356 192.168.1.108:0/5235 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.173639) v2 ==== 47+0+0 (2047406027 0 0) 0x54d5240 con 0x55f32e0 -1321> 2014-08-13 17:23:22.174261 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.173639) v2 -- ?+0 0x54d3480 con 0x55f32e0 -1320> 2014-08-13 17:23:22.174351 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.356 192.168.1.108:0/5235 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.173639) v2 ==== 47+0+0 (2047406027 0 0) 0x54d3800 con 0x55f2c00 -1319> 2014-08-13 17:23:22.174400 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.108:0/5235 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.173639) v2 -- ?+0 0x54d16c0 con 0x55f2c00 -1318> 2014-08-13 17:23:22.187658 7fccd4bdc700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.109:0/31286 pipe(0x5922300 sd=416 :6820 s=2 pgs=487 cs=1 l=1 c=0x592bb20).reader couldn't read tag, (0) Success -1317> 2014-08-13 17:23:22.187711 7fccd4bdc700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.109:0/31286 pipe(0x5922300 sd=416 :6820 s=2 pgs=487 cs=1 l=1 c=0x592bb20).fault (0) Success -1316> 2014-08-13 17:23:22.187686 7fccd4adb700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.109:0/31286 pipe(0x5923200 sd=417 :6808 s=2 pgs=488 cs=1 l=1 c=0x592a3c0).reader couldn't read tag, (0) Success -1315> 2014-08-13 17:23:22.187741 7fccd4adb700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.109:0/31286 pipe(0x5923200 sd=417 :6808 s=2 pgs=488 cs=1 l=1 c=0x592a3c0).fault (0) Success -1314> 2014-08-13 17:23:22.191156 7fccd46d7700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.114:0/52879 pipe(0x5922f80 sd=419 :6808 s=2 pgs=342 cs=1 l=1 c=0x592a100).reader couldn't read tag, (0) Success -1313> 2014-08-13 17:23:22.191202 7fccd47d8700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.114:0/52879 pipe(0x5922080 sd=418 :6820 s=2 pgs=341 cs=1 l=1 c=0x592b860).reader couldn't read tag, (0) Success -1312> 2014-08-13 17:23:22.191227 7fccd46d7700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.114:0/52879 pipe(0x5922f80 sd=419 :6808 s=2 pgs=342 cs=1 l=1 c=0x592a100).fault (0) Success -1311> 2014-08-13 17:23:22.191259 7fccd47d8700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.114:0/52879 pipe(0x5922080 sd=418 :6820 s=2 pgs=341 cs=1 l=1 c=0x592b860).fault (0) Success -1310> 2014-08-13 17:23:22.309644 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.346 192.168.100.104:0/2308 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.309606) v2 ==== 47+0+0 (2394353593 0 0) 0x54d7a80 con 0x5948000 -1309> 2014-08-13 17:23:22.309694 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.309606) v2 -- ?+0 0x54d3800 con 0x5948000 -1308> 2014-08-13 17:23:22.309683 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.346 192.168.100.104:0/2308 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.309606) v2 ==== 47+0+0 (2394353593 0 0) 0x54d7700 con 0x546e880 -1307> 2014-08-13 17:23:22.309742 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.104:0/2308 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.309606) v2 -- ?+0 0x54d5240 con 0x546e880 -1306> 2014-08-13 17:23:22.379018 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.378779) v2 ==== 47+0+0 (3746806474 0 0) 0x54d5e80 con 0x546c200 -1305> 2014-08-13 17:23:22.379031 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.378779) v2 ==== 47+0+0 (3746806474 0 0) 0x54d0e00 con 0x546d6a0 -1304> 2014-08-13 17:23:22.379060 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.378779) v2 -- ?+0 0x54d7a80 con 0x546c200 -1303> 2014-08-13 17:23:22.379569 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.378779) v2 -- ?+0 0x54d7700 con 0x546d6a0 -1302> 2014-08-13 17:23:22.391960 7fccd27b8700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.108:0/6227 pipe(0x5924d80 sd=434 :6820 s=2 pgs=361 cs=1 l=1 c=0x592d3e0).reader couldn't read tag, (0) Success -1301> 2014-08-13 17:23:22.392009 7fccd27b8700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.108:0/6227 pipe(0x5924d80 sd=434 :6820 s=2 pgs=361 cs=1 l=1 c=0x592d3e0).fault (0) Success -1300> 2014-08-13 17:23:22.392003 7fccd26b7700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.108:0/6227 pipe(0x5923c00 sd=435 :6808 s=2 pgs=362 cs=1 l=1 c=0x592eca0).reader couldn't read tag, (0) Success -1299> 2014-08-13 17:23:22.392059 7fccd26b7700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.108:0/6227 pipe(0x5923c00 sd=435 :6808 s=2 pgs=362 cs=1 l=1 c=0x592eca0).fault (0) Success -1298> 2014-08-13 17:23:22.399375 7fccd23b4700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.111:0/17345 pipe(0x5924b00 sd=436 :6820 s=2 pgs=33034 cs=1 l=1 c=0x592fd20).reader couldn't read tag, (0) Success -1297> 2014-08-13 17:23:22.399425 7fccd23b4700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.111:0/17345 pipe(0x5924b00 sd=436 :6820 s=2 pgs=33034 cs=1 l=1 c=0x592fd20).fault (0) Success -1296> 2014-08-13 17:23:22.399502 7fccd22b3700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.111:0/17345 pipe(0x5923980 sd=437 :6808 s=2 pgs=33035 cs=1 l=1 c=0x592e9e0).reader couldn't read tag, (0) Success -1295> 2014-08-13 17:23:22.399563 7fccd22b3700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.111:0/17345 pipe(0x5923980 sd=437 :6808 s=2 pgs=33035 cs=1 l=1 c=0x592e9e0).fault (0) Success -1294> 2014-08-13 17:23:22.430613 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.374 192.168.100.102:0/16129 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.430310) v2 ==== 47+0+0 (72588180 0 0) 0x54d5400 con 0x5468dc0 -1293> 2014-08-13 17:23:22.430628 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.374 192.168.100.102:0/16129 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.430310) v2 ==== 47+0+0 (72588180 0 0) 0x54d6040 con 0x546a260 -1292> 2014-08-13 17:23:22.430664 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.430310) v2 -- ?+0 0x54d5e80 con 0x5468dc0 -1291> 2014-08-13 17:23:22.430718 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.102:0/16129 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.430310) v2 -- ?+0 0x54d0e00 con 0x546a260 -1290> 2014-08-13 17:23:22.455682 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.5 192.168.1.103:0/40192 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.455345) v2 ==== 47+0+0 (2207211985 0 0) 0x54d5780 con 0x592dc20 -1289> 2014-08-13 17:23:22.455733 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.455345) v2 -- ?+0 0x54d5400 con 0x592dc20 -1288> 2014-08-13 17:23:22.456081 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.5 192.168.1.103:0/40192 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.455345) v2 ==== 47+0+0 (2207211985 0 0) 0x54d7e00 con 0x592bf40 -1287> 2014-08-13 17:23:22.456137 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.103:0/40192 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.455345) v2 -- ?+0 0x54d6040 con 0x592bf40 -1286> 2014-08-13 17:23:22.534752 7fccd0a9b700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.111:0/10782 pipe(0x5926180 sd=449 :6820 s=2 pgs=1801 cs=1 l=1 c=0x54682c0).reader couldn't read tag, (0) Success -1285> 2014-08-13 17:23:22.534804 7fccd0a9b700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.111:0/10782 pipe(0x5926180 sd=449 :6820 s=2 pgs=1801 cs=1 l=1 c=0x54682c0).fault (0) Success -1284> 2014-08-13 17:23:22.534782 7fccd0b9c700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.111:0/10782 pipe(0x5927300 sd=448 :6808 s=2 pgs=1800 cs=1 l=1 c=0x5469760).reader couldn't read tag, (0) Success -1283> 2014-08-13 17:23:22.534836 7fccd0b9c700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.111:0/10782 pipe(0x5927300 sd=448 :6808 s=2 pgs=1800 cs=1 l=1 c=0x5469760).fault (0) Success -1282> 2014-08-13 17:23:22.570919 7fcccff90700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.109:0/35445 pipe(0x5925a00 sd=454 :6820 s=2 pgs=32461 cs=1 l=1 c=0x546bc80).reader couldn't read tag, (0) Success -1281> 2014-08-13 17:23:22.570970 7fcccff90700 2 -- 192.168.1.115:6820/31924 >> 192.168.1.109:0/35445 pipe(0x5925a00 sd=454 :6820 s=2 pgs=32461 cs=1 l=1 c=0x546bc80).fault (0) Success -1280> 2014-08-13 17:23:22.570963 7fcccfe8f700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.109:0/35445 pipe(0x5931400 sd=455 :6808 s=2 pgs=32462 cs=1 l=1 c=0x546d120).reader couldn't read tag, (0) Success -1279> 2014-08-13 17:23:22.571019 7fcccfe8f700 2 -- 192.168.100.115:6808/31924 >> 192.168.1.109:0/35445 pipe(0x5931400 sd=455 :6808 s=2 pgs=32462 cs=1 l=1 c=0x546d120).fault (0) Success -1278> 2014-08-13 17:23:22.586377 7fcccfa8b700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.106:0/25893 pipe(0x5931180 sd=457 :6808 s=2 pgs=170 cs=1 l=1 c=0x546ce60).reader couldn't read tag, (0) Success -1277> 2014-08-13 17:23:22.586430 7fcccfa8b700 2 -- 192.168.100.115:6808/31924 >> 192.168.100.106:0/25893 pipe(0x5931180 sd=457 :6808 s=2 pgs=170 cs=1 l=1 c=0x546ce60).fault (0) Success -1276> 2014-08-13 17:23:22.586490 7fcccfb8c700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.106:0/25893 pipe(0x5932a80 sd=456 :6820 s=2 pgs=169 cs=1 l=1 c=0x546b9c0).reader couldn't read tag, (0) Success -1275> 2014-08-13 17:23:22.586547 7fcccfb8c700 2 -- 192.168.1.115:6820/31924 >> 192.168.100.106:0/25893 pipe(0x5932a80 sd=456 :6820 s=2 pgs=169 cs=1 l=1 c=0x546b9c0).fault (0) Success -1274> 2014-08-13 17:23:22.592477 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.323 192.168.100.106:0/8118 2 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.591690) v2 ==== 47+0+0 (4214728337 0 0) 0x56aa4c0 con 0x59286e0 -1273> 2014-08-13 17:23:22.592528 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.591690) v2 -- ?+0 0x54d7e00 con 0x59286e0 -1272> 2014-08-13 17:23:22.592614 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.323 192.168.100.106:0/8118 2 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.591690) v2 ==== 47+0+0 (4214728337 0 0) 0x54d0000 con 0x57f7d20 -1271> 2014-08-13 17:23:22.592660 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194477 stamp 2014-08-13 17:23:22.591690) v2 -- ?+0 0x54d5780 con 0x57f7d20 -1270> 2014-08-13 17:23:22.630814 7fcd0aa4e700 1 -- 192.168.100.115:6805/31924 <== mon.0 192.168.100.111:6789/0 17 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x550bf00 con 0x4e25d80 -1269> 2014-08-13 17:23:22.630907 7fcd0aa4e700 3 osd.4 194477 handle_osd_map epochs [194478,194478], i have 194477, src has [185139,194478] -1268> 2014-08-13 17:23:22.636410 7fcd0aa4e700 0 osd.4 194478 crush map has features 2578087936000, adjusting msgr requires for mons -1267> 2014-08-13 17:23:22.636662 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 0.928577 4 0.000256 -1266> 2014-08-13 17:23:22.636719 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 0.997655 0 0.000000 -1265> 2014-08-13 17:23:22.636753 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194472 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active -1264> 2014-08-13 17:23:22.639445 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e6540 con 0x4ec56a0 -1263> 2014-08-13 17:23:22.642917 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194477 pi=193640-194476/30 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating -1262> 2014-08-13 17:23:22.647401 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.941436 2 0.000087 -1261> 2014-08-13 17:23:22.647458 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.008380 0 0.000000 -1260> 2014-08-13 17:23:22.647491 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1259> 2014-08-13 17:23:22.647784 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194477 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1258> 2014-08-13 17:23:22.652294 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.946465 3 0.000121 -1257> 2014-08-13 17:23:22.652339 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.005771 0 0.000000 -1256> 2014-08-13 17:23:22.652373 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194471 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1255> 2014-08-13 17:23:22.652569 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 0.956124 4 0.000248 -1254> 2014-08-13 17:23:22.652624 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 0.989674 0 0.000000 -1253> 2014-08-13 17:23:22.652660 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194472 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active -1252> 2014-08-13 17:23:22.652757 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194477 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1251> 2014-08-13 17:23:22.652991 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating -1250> 2014-08-13 17:23:22.653193 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.947593 2 0.000112 -1249> 2014-08-13 17:23:22.653237 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.989777 0 0.000000 -1248> 2014-08-13 17:23:22.653265 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194471 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1247> 2014-08-13 17:23:22.653582 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1246> 2014-08-13 17:23:22.654364 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.946525 3 0.000087 -1245> 2014-08-13 17:23:22.654397 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.007821 0 0.000000 -1244> 2014-08-13 17:23:22.654423 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1243> 2014-08-13 17:23:22.654493 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x2e8af40 con 0x4fde300 -1242> 2014-08-13 17:23:22.654526 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_notify(22.581(26) epoch 194478) v5 -- ?+0 0x588ad80 con 0x4fde300 -1241> 2014-08-13 17:23:22.654613 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_notify(22.220(26) epoch 194478) v5 -- ?+0 0x4e50a80 con 0x4fdee00 -1240> 2014-08-13 17:23:22.654655 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6815/60573 -- pg_info(1 pgs e194478:24.30b(30)) v4 -- ?+0 0x5b4cb40 con 0x4e30580 -1239> 2014-08-13 17:23:22.654686 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6834/14316 -- pg_info(1 pgs e194478:22.362(27)) v4 -- ?+0 0x5b4c980 con 0x4e335a0 -1238> 2014-08-13 17:23:22.654692 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1237> 2014-08-13 17:23:22.654717 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6824/5505 -- pg_info(1 pgs e194478:22.619(27)) v4 -- ?+0 0x5b4c7c0 con 0x4e370c0 -1236> 2014-08-13 17:23:22.654755 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_info(1 pgs e194478:22.619(27)) v4 -- ?+0 0x5b4c600 con 0x4fd9760 -1235> 2014-08-13 17:23:22.654785 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6820/19173 -- pg_info(1 pgs e194478:24.30b(30)) v4 -- ?+0 0x5b4c440 con 0x4fdad60 -1234> 2014-08-13 17:23:22.654810 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x2e8ad00 con 0x4f86e00 -1233> 2014-08-13 17:23:22.654840 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6802/61960 -- pg_info(1 pgs e194478:22.362(27)) v4 -- ?+0 0x5b4c280 con 0x4f86e00 -1232> 2014-08-13 17:23:22.654886 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.114:6830/52879 -- pg_info(1 pgs e194478:24.30b(30)) v4 -- ?+0 0x5b4c0c0 con 0x4fd2520 -1231> 2014-08-13 17:23:22.654932 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1230> 2014-08-13 17:23:22.654957 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 10 ==== pg_notify(22.99(23) epoch 194478) v5 ==== 2003+0+0 (1367890649 0 0) 0x58881c0 con 0x4ec56a0 -1229> 2014-08-13 17:23:22.654986 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.639467, event: header_read, request: pg_notify(22.99(23) epoch 194478) v5 -1228> 2014-08-13 17:23:22.654998 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.639470, event: throttled, request: pg_notify(22.99(23) epoch 194478) v5 -1227> 2014-08-13 17:23:22.655004 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.639605, event: all_read, request: pg_notify(22.99(23) epoch 194478) v5 -1226> 2014-08-13 17:23:22.655014 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.654982, event: dispatched, request: pg_notify(22.99(23) epoch 194478) v5 -1225> 2014-08-13 17:23:22.655022 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.655022, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194478) v5 -1224> 2014-08-13 17:23:22.655036 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.655036, event: started, request: pg_notify(22.99(23) epoch 194478) v5 -1223> 2014-08-13 17:23:22.655065 7fcd0964c700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:23:22.655065, event: done, request: pg_notify(22.99(23) epoch 194478) v5 -1222> 2014-08-13 17:23:22.655091 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 18 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x55086c0 con 0x4ec4780 -1221> 2014-08-13 17:23:22.655114 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1220> 2014-08-13 17:23:22.655097 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.955418 2 0.000076 -1219> 2014-08-13 17:23:22.655127 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bda00 con 0x4ec5d80 -1218> 2014-08-13 17:23:22.655145 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1217> 2014-08-13 17:23:22.655139 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.002812 0 0.000000 -1216> 2014-08-13 17:23:22.655164 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52be780 con 0x4ec7e80 -1215> 2014-08-13 17:23:22.655182 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1214> 2014-08-13 17:23:22.655166 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194471 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1213> 2014-08-13 17:23:22.655194 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b8000 con 0x4f80b00 -1212> 2014-08-13 17:23:22.655222 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1211> 2014-08-13 17:23:22.655234 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 10 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bad00 con 0x4fd8f20 -1210> 2014-08-13 17:23:22.655252 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1209> 2014-08-13 17:23:22.655263 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e33c0 con 0x4fd86e0 -1208> 2014-08-13 17:23:22.655281 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1207> 2014-08-13 17:23:22.655295 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e57c0 con 0x4fdee00 -1206> 2014-08-13 17:23:22.655313 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1205> 2014-08-13 17:23:22.655328 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x5678000 con 0x4fde880 -1204> 2014-08-13 17:23:22.655345 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1203> 2014-08-13 17:23:22.655373 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b9440 con 0x4fde300 -1202> 2014-08-13 17:23:22.655390 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1201> 2014-08-13 17:23:22.655410 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 11 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567d580 con 0x4fd5960 -1200> 2014-08-13 17:23:22.655410 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194477 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1199> 2014-08-13 17:23:22.655428 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1198> 2014-08-13 17:23:22.655437 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 11 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bec00 con 0x5305ee0 -1197> 2014-08-13 17:23:22.655455 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1196> 2014-08-13 17:23:22.655465 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x550c5c0 con 0x552b700 -1195> 2014-08-13 17:23:22.655486 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1194> 2014-08-13 17:23:22.655496 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52baf40 con 0x4ec3f40 -1193> 2014-08-13 17:23:22.655514 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1192> 2014-08-13 17:23:22.655525 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 11 ==== pg_info(1 pgs e194478:24.1e4(15)) v4 ==== 1585+0+0 (4246411451 0 0) 0x588b100 con 0x4ec56a0 -1191> 2014-08-13 17:23:22.655543 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.642186, event: header_read, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1190> 2014-08-13 17:23:22.655555 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.642189, event: throttled, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1189> 2014-08-13 17:23:22.655565 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.642310, event: all_read, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1188> 2014-08-13 17:23:22.655576 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.655540, event: dispatched, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1187> 2014-08-13 17:23:22.655587 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.655586, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1186> 2014-08-13 17:23:22.655599 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.655598, event: started, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1185> 2014-08-13 17:23:22.655626 7fcd0964c700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:23:22.655625, event: done, request: pg_info(1 pgs e194478:24.1e4(15)) v4 -1184> 2014-08-13 17:23:22.655611 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.950121 2 0.000086 -1183> 2014-08-13 17:23:22.655639 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.002820 0 0.000000 -1182> 2014-08-13 17:23:22.655648 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 19 ==== pg_notify(24.136(26) epoch 194478) v5 ==== 2202+0+0 (942611436 0 0) 0x54d1500 con 0x4ec4780 -1181> 2014-08-13 17:23:22.655664 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.641885, event: header_read, request: pg_notify(24.136(26) epoch 194478) v5 -1180> 2014-08-13 17:23:22.655672 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.641888, event: throttled, request: pg_notify(24.136(26) epoch 194478) v5 -1179> 2014-08-13 17:23:22.655667 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194471 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1178> 2014-08-13 17:23:22.655678 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.642028, event: all_read, request: pg_notify(24.136(26) epoch 194478) v5 -1177> 2014-08-13 17:23:22.655687 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.655662, event: dispatched, request: pg_notify(24.136(26) epoch 194478) v5 -1176> 2014-08-13 17:23:22.655698 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.655698, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194478) v5 -1175> 2014-08-13 17:23:22.655709 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.655709, event: started, request: pg_notify(24.136(26) epoch 194478) v5 -1174> 2014-08-13 17:23:22.655746 7fcd0964c700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:23:22.655745, event: done, request: pg_notify(24.136(26) epoch 194478) v5 -1173> 2014-08-13 17:23:22.655813 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 8 ==== pg_notify(22.3e4(27) epoch 194478) v5 ==== 2231+0+0 (1924376369 0 0) 0x5b488c0 con 0x4ec5d80 -1172> 2014-08-13 17:23:22.655835 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.648035, event: header_read, request: pg_notify(22.3e4(27) epoch 194478) v5 -1171> 2014-08-13 17:23:22.655886 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.648037, event: throttled, request: pg_notify(22.3e4(27) epoch 194478) v5 -1170> 2014-08-13 17:23:22.655895 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.648167, event: all_read, request: pg_notify(22.3e4(27) epoch 194478) v5 -1169> 2014-08-13 17:23:22.655904 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.655832, event: dispatched, request: pg_notify(22.3e4(27) epoch 194478) v5 -1168> 2014-08-13 17:23:22.655914 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.655914, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194478) v5 -1167> 2014-08-13 17:23:22.655929 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.655929, event: started, request: pg_notify(22.3e4(27) epoch 194478) v5 -1166> 2014-08-13 17:23:22.655959 7fcd0964c700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:23:22.655959, event: done, request: pg_notify(22.3e4(27) epoch 194478) v5 -1165> 2014-08-13 17:23:22.655986 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.21 192.168.1.107:6821/54786 6 ==== pg_notify(22.707(27) epoch 194478) v5 ==== 2231+0+0 (4075622884 0 0) 0x588d400 con 0x4ec7e80 -1164> 2014-08-13 17:23:22.656006 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.649566, event: header_read, request: pg_notify(22.707(27) epoch 194478) v5 -1163> 2014-08-13 17:23:22.656017 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.649567, event: throttled, request: pg_notify(22.707(27) epoch 194478) v5 -1162> 2014-08-13 17:23:22.656026 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.649682, event: all_read, request: pg_notify(22.707(27) epoch 194478) v5 -1161> 2014-08-13 17:23:22.656035 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.656004, event: dispatched, request: pg_notify(22.707(27) epoch 194478) v5 -1160> 2014-08-13 17:23:22.656044 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.656044, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194478) v5 -1159> 2014-08-13 17:23:22.656055 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.656055, event: started, request: pg_notify(22.707(27) epoch 194478) v5 -1158> 2014-08-13 17:23:22.656059 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194477 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -1157> 2014-08-13 17:23:22.656079 7fcd0964c700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:23:22.656079, event: done, request: pg_notify(22.707(27) epoch 194478) v5 -1156> 2014-08-13 17:23:22.656097 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 9 ==== pg_notify(22.707(28) epoch 194478) v5 ==== 2290+0+0 (3334078298 0 0) 0x588d240 con 0x4f80b00 -1155> 2014-08-13 17:23:22.656116 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.649462, event: header_read, request: pg_notify(22.707(28) epoch 194478) v5 -1154> 2014-08-13 17:23:22.656126 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.649463, event: throttled, request: pg_notify(22.707(28) epoch 194478) v5 -1153> 2014-08-13 17:23:22.656135 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.649756, event: all_read, request: pg_notify(22.707(28) epoch 194478) v5 -1152> 2014-08-13 17:23:22.656144 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.656114, event: dispatched, request: pg_notify(22.707(28) epoch 194478) v5 -1151> 2014-08-13 17:23:22.656153 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.656153, event: waiting_for_osdmap, request: pg_notify(22.707(28) epoch 194478) v5 -1150> 2014-08-13 17:23:22.656164 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.656164, event: started, request: pg_notify(22.707(28) epoch 194478) v5 -1149> 2014-08-13 17:23:22.656187 7fcd0964c700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:23:22.656186, event: done, request: pg_notify(22.707(28) epoch 194478) v5 -1148> 2014-08-13 17:23:22.656215 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 11 ==== pg_notify(22.99(22) epoch 194478) v5 ==== 1944+0+0 (465858686 0 0) 0x54d4600 con 0x4fd8f20 -1147> 2014-08-13 17:23:22.656235 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.647455, event: header_read, request: pg_notify(22.99(22) epoch 194478) v5 -1146> 2014-08-13 17:23:22.656246 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.647456, event: throttled, request: pg_notify(22.99(22) epoch 194478) v5 -1145> 2014-08-13 17:23:22.656255 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.647584, event: all_read, request: pg_notify(22.99(22) epoch 194478) v5 -1144> 2014-08-13 17:23:22.656265 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.656233, event: dispatched, request: pg_notify(22.99(22) epoch 194478) v5 -1143> 2014-08-13 17:23:22.656272 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.656271, event: waiting_for_osdmap, request: pg_notify(22.99(22) epoch 194478) v5 -1142> 2014-08-13 17:23:22.656282 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.656282, event: started, request: pg_notify(22.99(22) epoch 194478) v5 -1141> 2014-08-13 17:23:22.656265 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.958910 4 0.000270 -1140> 2014-08-13 17:23:22.656302 7fcd0964c700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:23:22.656302, event: done, request: pg_notify(22.99(22) epoch 194478) v5 -1139> 2014-08-13 17:23:22.656298 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.002983 0 0.000000 -1138> 2014-08-13 17:23:22.656328 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 9 ==== pg_info(1 pgs e194478:22.653(5)) v4 ==== 997+0+0 (2396643312 0 0) 0x5b4a300 con 0x4fd86e0 -1137> 2014-08-13 17:23:22.656334 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194472 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -1136> 2014-08-13 17:23:22.656345 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.650506, event: header_read, request: pg_info(1 pgs e194478:22.653(5)) v4 -1135> 2014-08-13 17:23:22.656356 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.650507, event: throttled, request: pg_info(1 pgs e194478:22.653(5)) v4 -1134> 2014-08-13 17:23:22.656362 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.650581, event: all_read, request: pg_info(1 pgs e194478:22.653(5)) v4 -1133> 2014-08-13 17:23:22.656368 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.656343, event: dispatched, request: pg_info(1 pgs e194478:22.653(5)) v4 -1132> 2014-08-13 17:23:22.656374 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.656374, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.653(5)) v4 -1131> 2014-08-13 17:23:22.656382 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.656382, event: started, request: pg_info(1 pgs e194478:22.653(5)) v4 -1130> 2014-08-13 17:23:22.656402 7fcd0964c700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:23:22.656402, event: done, request: pg_info(1 pgs e194478:22.653(5)) v4 -1129> 2014-08-13 17:23:22.656417 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 9 ==== pg_info(1 pgs e194478:22.220(5)) v4 ==== 997+0+0 (714769224 0 0) 0x56a9340 con 0x4fdee00 -1128> 2014-08-13 17:23:22.656431 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.649053, event: header_read, request: pg_info(1 pgs e194478:22.220(5)) v4 -1127> 2014-08-13 17:23:22.656438 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.649055, event: throttled, request: pg_info(1 pgs e194478:22.220(5)) v4 -1126> 2014-08-13 17:23:22.656443 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.649132, event: all_read, request: pg_info(1 pgs e194478:22.220(5)) v4 -1125> 2014-08-13 17:23:22.656449 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.656429, event: dispatched, request: pg_info(1 pgs e194478:22.220(5)) v4 -1124> 2014-08-13 17:23:22.656456 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.656455, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.220(5)) v4 -1123> 2014-08-13 17:23:22.656463 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.656463, event: started, request: pg_info(1 pgs e194478:22.220(5)) v4 -1122> 2014-08-13 17:23:22.656482 7fcd0964c700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:23:22.656482, event: done, request: pg_info(1 pgs e194478:22.220(5)) v4 -1121> 2014-08-13 17:23:22.656507 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 9 ==== pg_notify(22.1d2(3) epoch 194478) v5 ==== 887+0+0 (2489394188 0 0) 0x56af700 con 0x4fde880 -1120> 2014-08-13 17:23:22.656522 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.653427, event: header_read, request: pg_notify(22.1d2(3) epoch 194478) v5 -1119> 2014-08-13 17:23:22.656529 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.653428, event: throttled, request: pg_notify(22.1d2(3) epoch 194478) v5 -1118> 2014-08-13 17:23:22.656534 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.653508, event: all_read, request: pg_notify(22.1d2(3) epoch 194478) v5 -1117> 2014-08-13 17:23:22.656540 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.656520, event: dispatched, request: pg_notify(22.1d2(3) epoch 194478) v5 -1116> 2014-08-13 17:23:22.656546 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.656546, event: waiting_for_osdmap, request: pg_notify(22.1d2(3) epoch 194478) v5 -1115> 2014-08-13 17:23:22.656553 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.656553, event: started, request: pg_notify(22.1d2(3) epoch 194478) v5 -1114> 2014-08-13 17:23:22.656573 7fcd0964c700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:23:22.656572, event: done, request: pg_notify(22.1d2(3) epoch 194478) v5 -1113> 2014-08-13 17:23:22.656590 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 10 ==== pg_info(1 pgs e194478:22.581(5)) v4 ==== 997+0+0 (1666961312 0 0) 0x51878c0 con 0x4fde300 -1112> 2014-08-13 17:23:22.656606 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.640126, event: header_read, request: pg_info(1 pgs e194478:22.581(5)) v4 -1111> 2014-08-13 17:23:22.656613 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.640129, event: throttled, request: pg_info(1 pgs e194478:22.581(5)) v4 -1110> 2014-08-13 17:23:22.656619 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.640228, event: all_read, request: pg_info(1 pgs e194478:22.581(5)) v4 -1109> 2014-08-13 17:23:22.656629 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.656604, event: dispatched, request: pg_info(1 pgs e194478:22.581(5)) v4 -1108> 2014-08-13 17:23:22.656639 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.656639, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.581(5)) v4 -1107> 2014-08-13 17:23:22.656650 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.656650, event: started, request: pg_info(1 pgs e194478:22.581(5)) v4 -1106> 2014-08-13 17:23:22.656673 7fcd0964c700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:23:22.656673, event: done, request: pg_info(1 pgs e194478:22.581(5)) v4 -1105> 2014-08-13 17:23:22.656691 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 12 ==== pg_notify(22.3d9(42) epoch 194478) v5 ==== 2996+0+0 (2914928597 0 0) 0x588c280 con 0x4fd5960 -1104> 2014-08-13 17:23:22.656709 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.649120, event: header_read, request: pg_notify(22.3d9(42) epoch 194478) v5 -1103> 2014-08-13 17:23:22.656719 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.649122, event: throttled, request: pg_notify(22.3d9(42) epoch 194478) v5 -1102> 2014-08-13 17:23:22.656728 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.649271, event: all_read, request: pg_notify(22.3d9(42) epoch 194478) v5 -1101> 2014-08-13 17:23:22.656737 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.656707, event: dispatched, request: pg_notify(22.3d9(42) epoch 194478) v5 -1100> 2014-08-13 17:23:22.656747 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.656746, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194478) v5 -1099> 2014-08-13 17:23:22.656757 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.656757, event: started, request: pg_notify(22.3d9(42) epoch 194478) v5 -1098> 2014-08-13 17:23:22.656780 7fcd0964c700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:23:22.656780, event: done, request: pg_notify(22.3d9(42) epoch 194478) v5 -1097> 2014-08-13 17:23:22.656808 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x4f7f980 con 0x53011e0 -1096> 2014-08-13 17:23:22.656829 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1095> 2014-08-13 17:23:22.656841 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 12 ==== pg_notify(22.1d2(24) epoch 194478) v5 ==== 2062+0+0 (965578213 0 0) 0x5187e00 con 0x5305ee0 -1094> 2014-08-13 17:23:22.656859 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.641200, event: header_read, request: pg_notify(22.1d2(24) epoch 194478) v5 -1093> 2014-08-13 17:23:22.656870 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.641202, event: throttled, request: pg_notify(22.1d2(24) epoch 194478) v5 -1092> 2014-08-13 17:23:22.656879 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.641317, event: all_read, request: pg_notify(22.1d2(24) epoch 194478) v5 -1091> 2014-08-13 17:23:22.656888 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.656857, event: dispatched, request: pg_notify(22.1d2(24) epoch 194478) v5 -1090> 2014-08-13 17:23:22.656897 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.656897, event: waiting_for_osdmap, request: pg_notify(22.1d2(24) epoch 194478) v5 -1089> 2014-08-13 17:23:22.656908 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.656908, event: started, request: pg_notify(22.1d2(24) epoch 194478) v5 -1088> 2014-08-13 17:23:22.656931 7fcd0964c700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:23:22.656931, event: done, request: pg_notify(22.1d2(24) epoch 194478) v5 -1087> 2014-08-13 17:23:22.656952 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 8 ==== pg_notify(22.22c(22) epoch 194478) v5 ==== 1944+0+0 (43130717 0 0) 0x56ad240 con 0x552b700 -1086> 2014-08-13 17:23:22.656970 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.647546, event: header_read, request: pg_notify(22.22c(22) epoch 194478) v5 -1085> 2014-08-13 17:23:22.656980 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.647568, event: throttled, request: pg_notify(22.22c(22) epoch 194478) v5 -1084> 2014-08-13 17:23:22.656989 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.647883, event: all_read, request: pg_notify(22.22c(22) epoch 194478) v5 -1083> 2014-08-13 17:23:22.656998 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.656968, event: dispatched, request: pg_notify(22.22c(22) epoch 194478) v5 -1082> 2014-08-13 17:23:22.657007 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.657007, event: waiting_for_osdmap, request: pg_notify(22.22c(22) epoch 194478) v5 -1081> 2014-08-13 17:23:22.657018 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.657018, event: started, request: pg_notify(22.22c(22) epoch 194478) v5 -1080> 2014-08-13 17:23:22.657037 7fcd0964c700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:23:22.657037, event: done, request: pg_notify(22.22c(22) epoch 194478) v5 -1079> 2014-08-13 17:23:22.657064 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x5678b40 con 0x552fa60 -1078> 2014-08-13 17:23:22.657085 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -1077> 2014-08-13 17:23:22.657097 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 10 ==== pg_notify(24.34(50) epoch 194478) v5 ==== 3306+0+0 (3083843268 0 0) 0x54d5cc0 con 0x4ec3f40 -1076> 2014-08-13 17:23:22.657114 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.650685, event: header_read, request: pg_notify(24.34(50) epoch 194478) v5 -1075> 2014-08-13 17:23:22.657125 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.650686, event: throttled, request: pg_notify(24.34(50) epoch 194478) v5 -1074> 2014-08-13 17:23:22.657134 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.650883, event: all_read, request: pg_notify(24.34(50) epoch 194478) v5 -1073> 2014-08-13 17:23:22.657140 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.657112, event: dispatched, request: pg_notify(24.34(50) epoch 194478) v5 -1072> 2014-08-13 17:23:22.657149 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.657149, event: waiting_for_osdmap, request: pg_notify(24.34(50) epoch 194478) v5 -1071> 2014-08-13 17:23:22.657160 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.657160, event: started, request: pg_notify(24.34(50) epoch 194478) v5 -1070> 2014-08-13 17:23:22.657186 7fcd0964c700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:23:22.657185, event: done, request: pg_notify(24.34(50) epoch 194478) v5 -1069> 2014-08-13 17:23:22.657226 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.356 192.168.1.108:6832/5235 20 ==== pg_info(1 pgs e194478:24.3b7(6)) v4 ==== 1090+0+0 (4233285941 0 0) 0x54d3f00 con 0x4ec4780 -1068> 2014-08-13 17:23:22.657248 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.642247, event: header_read, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1067> 2014-08-13 17:23:22.657258 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.642249, event: throttled, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1066> 2014-08-13 17:23:22.657268 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.642359, event: all_read, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1065> 2014-08-13 17:23:22.657279 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.657245, event: dispatched, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1064> 2014-08-13 17:23:22.657289 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.657289, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1063> 2014-08-13 17:23:22.657300 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.657300, event: started, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1062> 2014-08-13 17:23:22.657324 7fcd0964c700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:23:22.657324, event: done, request: pg_info(1 pgs e194478:24.3b7(6)) v4 -1061> 2014-08-13 17:23:22.657347 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 10 ==== pg_info(1 pgs e194478:24.20e(6)) v4 ==== 1090+0+0 (3329752791 0 0) 0x588b480 con 0x4f80b00 -1060> 2014-08-13 17:23:22.657366 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.650174, event: header_read, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1059> 2014-08-13 17:23:22.657377 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.650175, event: throttled, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1058> 2014-08-13 17:23:22.657385 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.650247, event: all_read, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1057> 2014-08-13 17:23:22.657394 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.657364, event: dispatched, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1056> 2014-08-13 17:23:22.657404 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.657404, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1055> 2014-08-13 17:23:22.657415 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.657415, event: started, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1054> 2014-08-13 17:23:22.657438 7fcd0964c700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:23:22.657438, event: done, request: pg_info(1 pgs e194478:24.20e(6)) v4 -1053> 2014-08-13 17:23:22.657457 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 12 ==== pg_notify(24.1bb(30) epoch 194478) v5 ==== 2438+0+0 (3887558717 0 0) 0x5b49340 con 0x4fd8f20 -1052> 2014-08-13 17:23:22.657475 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.648929, event: header_read, request: pg_notify(24.1bb(30) epoch 194478) v5 -1051> 2014-08-13 17:23:22.657485 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.648931, event: throttled, request: pg_notify(24.1bb(30) epoch 194478) v5 -1050> 2014-08-13 17:23:22.657494 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.649119, event: all_read, request: pg_notify(24.1bb(30) epoch 194478) v5 -1049> 2014-08-13 17:23:22.657503 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.657473, event: dispatched, request: pg_notify(24.1bb(30) epoch 194478) v5 -1048> 2014-08-13 17:23:22.657512 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.657512, event: waiting_for_osdmap, request: pg_notify(24.1bb(30) epoch 194478) v5 -1047> 2014-08-13 17:23:22.657523 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.657523, event: started, request: pg_notify(24.1bb(30) epoch 194478) v5 -1046> 2014-08-13 17:23:22.657546 7fcd0964c700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:23:22.657546, event: done, request: pg_notify(24.1bb(30) epoch 194478) v5 -1045> 2014-08-13 17:23:22.657568 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 10 ==== pg_notify(22.4df(22) epoch 194478) v5 ==== 1944+0+0 (643291935 0 0) 0x5b496c0 con 0x4fd86e0 -1044> 2014-08-13 17:23:22.657586 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.650601, event: header_read, request: pg_notify(22.4df(22) epoch 194478) v5 -1043> 2014-08-13 17:23:22.657596 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.650602, event: throttled, request: pg_notify(22.4df(22) epoch 194478) v5 -1042> 2014-08-13 17:23:22.657605 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.650696, event: all_read, request: pg_notify(22.4df(22) epoch 194478) v5 -1041> 2014-08-13 17:23:22.657614 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.657584, event: dispatched, request: pg_notify(22.4df(22) epoch 194478) v5 -1040> 2014-08-13 17:23:22.657624 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.657623, event: waiting_for_osdmap, request: pg_notify(22.4df(22) epoch 194478) v5 -1039> 2014-08-13 17:23:22.657635 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.657635, event: started, request: pg_notify(22.4df(22) epoch 194478) v5 -1038> 2014-08-13 17:23:22.657658 7fcd0964c700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:23:22.657657, event: done, request: pg_notify(22.4df(22) epoch 194478) v5 -1037> 2014-08-13 17:23:22.657678 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 10 ==== pg_notify(24.1f7(32) epoch 194478) v5 ==== 2556+0+0 (4237589848 0 0) 0x5b49880 con 0x4fdee00 -1036> 2014-08-13 17:23:22.657695 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.649678, event: header_read, request: pg_notify(24.1f7(32) epoch 194478) v5 -1035> 2014-08-13 17:23:22.657705 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.649681, event: throttled, request: pg_notify(24.1f7(32) epoch 194478) v5 -1034> 2014-08-13 17:23:22.657714 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.649816, event: all_read, request: pg_notify(24.1f7(32) epoch 194478) v5 -1033> 2014-08-13 17:23:22.657723 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.657693, event: dispatched, request: pg_notify(24.1f7(32) epoch 194478) v5 -1032> 2014-08-13 17:23:22.657732 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.657732, event: waiting_for_osdmap, request: pg_notify(24.1f7(32) epoch 194478) v5 -1031> 2014-08-13 17:23:22.657743 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.657743, event: started, request: pg_notify(24.1f7(32) epoch 194478) v5 -1030> 2014-08-13 17:23:22.657766 7fcd0964c700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:23:22.657765, event: done, request: pg_notify(24.1f7(32) epoch 194478) v5 -1029> 2014-08-13 17:23:22.657795 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 10 ==== pg_notify(24.1bb(27) epoch 194478) v5 ==== 2261+0+0 (3277018795 0 0) 0x5b4abc0 con 0x4fde880 -1028> 2014-08-13 17:23:22.657814 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.653661, event: header_read, request: pg_notify(24.1bb(27) epoch 194478) v5 -1027> 2014-08-13 17:23:22.657824 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.653663, event: throttled, request: pg_notify(24.1bb(27) epoch 194478) v5 -1026> 2014-08-13 17:23:22.657833 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.653782, event: all_read, request: pg_notify(24.1bb(27) epoch 194478) v5 -1025> 2014-08-13 17:23:22.657842 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.657812, event: dispatched, request: pg_notify(24.1bb(27) epoch 194478) v5 -1024> 2014-08-13 17:23:22.657851 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.657851, event: waiting_for_osdmap, request: pg_notify(24.1bb(27) epoch 194478) v5 -1023> 2014-08-13 17:23:22.657862 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.657862, event: started, request: pg_notify(24.1bb(27) epoch 194478) v5 -1022> 2014-08-13 17:23:22.657889 7fcd0964c700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:23:22.657889, event: done, request: pg_notify(24.1bb(27) epoch 194478) v5 -1021> 2014-08-13 17:23:22.657920 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 11 ==== pg_notify(22.39d(24),24.181(7) epoch 194478) v5 ==== 3195+0+0 (2893397850 0 0) 0x588dcc0 con 0x4fde300 -1020> 2014-08-13 17:23:22.657941 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.641389, event: header_read, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1019> 2014-08-13 17:23:22.657953 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.641392, event: throttled, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1018> 2014-08-13 17:23:22.657966 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.641533, event: all_read, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1017> 2014-08-13 17:23:22.657975 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.657939, event: dispatched, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1016> 2014-08-13 17:23:22.657986 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.657985, event: waiting_for_osdmap, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1015> 2014-08-13 17:23:22.657997 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.657997, event: started, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1014> 2014-08-13 17:23:22.658031 7fcd0964c700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:23:22.658031, event: done, request: pg_notify(22.39d(24),24.181(7) epoch 194478) v5 -1013> 2014-08-13 17:23:22.657938 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194477 pi=193748-194476/29 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -1012> 2014-08-13 17:23:22.658254 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 13 ==== pg_notify(22.48e(24) epoch 194478) v5 ==== 2058+0+0 (3287541215 0 0) 0x54d3640 con 0x4fd5960 -1011> 2014-08-13 17:23:22.658275 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.650392, event: header_read, request: pg_notify(22.48e(24) epoch 194478) v5 -1010> 2014-08-13 17:23:22.658288 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.650395, event: throttled, request: pg_notify(22.48e(24) epoch 194478) v5 -1009> 2014-08-13 17:23:22.658298 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.650520, event: all_read, request: pg_notify(22.48e(24) epoch 194478) v5 -1008> 2014-08-13 17:23:22.658277 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.953404 3 0.000098 -1007> 2014-08-13 17:23:22.658307 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.658272, event: dispatched, request: pg_notify(22.48e(24) epoch 194478) v5 -1006> 2014-08-13 17:23:22.658334 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.658334, event: waiting_for_osdmap, request: pg_notify(22.48e(24) epoch 194478) v5 -1005> 2014-08-13 17:23:22.658326 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.007503 0 0.000000 -1004> 2014-08-13 17:23:22.658348 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.658348, event: started, request: pg_notify(22.48e(24) epoch 194478) v5 -1003> 2014-08-13 17:23:22.658357 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -1002> 2014-08-13 17:23:22.658374 7fcd0964c700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:23:22.658374, event: done, request: pg_notify(22.48e(24) epoch 194478) v5 -1001> 2014-08-13 17:23:22.658396 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 8 ==== pg_notify(22.775(27) epoch 194478) v5 ==== 2231+0+0 (461666818 0 0) 0x5b4d080 con 0x53011e0 -1000> 2014-08-13 17:23:22.658417 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.656255, event: header_read, request: pg_notify(22.775(27) epoch 194478) v5 -999> 2014-08-13 17:23:22.658427 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.656257, event: throttled, request: pg_notify(22.775(27) epoch 194478) v5 -998> 2014-08-13 17:23:22.658436 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.656391, event: all_read, request: pg_notify(22.775(27) epoch 194478) v5 -997> 2014-08-13 17:23:22.658445 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.658415, event: dispatched, request: pg_notify(22.775(27) epoch 194478) v5 -996> 2014-08-13 17:23:22.658454 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.658454, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194478) v5 -995> 2014-08-13 17:23:22.658465 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.658465, event: started, request: pg_notify(22.775(27) epoch 194478) v5 -994> 2014-08-13 17:23:22.658525 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 0.955859 4 0.000490 -993> 2014-08-13 17:23:22.658559 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 0.998264 0 0.000000 -992> 2014-08-13 17:23:22.658587 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194472 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active -991> 2014-08-13 17:23:22.658715 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -990> 2014-08-13 17:23:22.658874 7fcd0964c700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:23:22.658874, event: done, request: pg_notify(22.775(27) epoch 194478) v5 -989> 2014-08-13 17:23:22.658901 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 13 ==== pg_notify(24.b6(29) epoch 194478) v5 ==== 2379+0+0 (2390649565 0 0) 0x54d39c0 con 0x5305ee0 -988> 2014-08-13 17:23:22.658921 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.642001, event: header_read, request: pg_notify(24.b6(29) epoch 194478) v5 -987> 2014-08-13 17:23:22.658931 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.642004, event: throttled, request: pg_notify(24.b6(29) epoch 194478) v5 -986> 2014-08-13 17:23:22.658940 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.642127, event: all_read, request: pg_notify(24.b6(29) epoch 194478) v5 -985> 2014-08-13 17:23:22.658949 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.658919, event: dispatched, request: pg_notify(24.b6(29) epoch 194478) v5 -984> 2014-08-13 17:23:22.658958 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.658958, event: waiting_for_osdmap, request: pg_notify(24.b6(29) epoch 194478) v5 -983> 2014-08-13 17:23:22.658971 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.658970, event: started, request: pg_notify(24.b6(29) epoch 194478) v5 -982> 2014-08-13 17:23:22.658992 7fcd0964c700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:23:22.658992, event: done, request: pg_notify(24.b6(29) epoch 194478) v5 -981> 2014-08-13 17:23:22.658988 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.957868 2 0.000067 -980> 2014-08-13 17:23:22.659023 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x5508900 con 0x5305960 -979> 2014-08-13 17:23:22.659022 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.007270 0 0.000000 -978> 2014-08-13 17:23:22.659046 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -977> 2014-08-13 17:23:22.659047 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194471 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -976> 2014-08-13 17:23:22.659059 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.368 192.168.1.102:6826/15861 9 ==== pg_notify(22.1f1(26) epoch 194478) v5 ==== 2172+0+0 (532328834 0 0) 0x56af8c0 con 0x552b700 -975> 2014-08-13 17:23:22.659077 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.649938, event: header_read, request: pg_notify(22.1f1(26) epoch 194478) v5 -974> 2014-08-13 17:23:22.659088 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.649940, event: throttled, request: pg_notify(22.1f1(26) epoch 194478) v5 -973> 2014-08-13 17:23:22.659097 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.650053, event: all_read, request: pg_notify(22.1f1(26) epoch 194478) v5 -972> 2014-08-13 17:23:22.659105 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.659074, event: dispatched, request: pg_notify(22.1f1(26) epoch 194478) v5 -971> 2014-08-13 17:23:22.659115 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.659115, event: waiting_for_osdmap, request: pg_notify(22.1f1(26) epoch 194478) v5 -970> 2014-08-13 17:23:22.659126 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.659126, event: started, request: pg_notify(22.1f1(26) epoch 194478) v5 -969> 2014-08-13 17:23:22.659149 7fcd0964c700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:23:22.659149, event: done, request: pg_notify(22.1f1(26) epoch 194478) v5 -968> 2014-08-13 17:23:22.659170 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.349 192.168.1.104:6800/61726 10 ==== pg_notify(22.362(26) epoch 194478) v5 ==== 2172+0+0 (797605970 0 0) 0x56af380 con 0x552fa60 -967> 2014-08-13 17:23:22.659189 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.655757, event: header_read, request: pg_notify(22.362(26) epoch 194478) v5 -966> 2014-08-13 17:23:22.659199 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.655758, event: throttled, request: pg_notify(22.362(26) epoch 194478) v5 -965> 2014-08-13 17:23:22.659215 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.655860, event: all_read, request: pg_notify(22.362(26) epoch 194478) v5 -964> 2014-08-13 17:23:22.659224 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.659187, event: dispatched, request: pg_notify(22.362(26) epoch 194478) v5 -963> 2014-08-13 17:23:22.659234 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.659234, event: waiting_for_osdmap, request: pg_notify(22.362(26) epoch 194478) v5 -962> 2014-08-13 17:23:22.659245 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.659244, event: started, request: pg_notify(22.362(26) epoch 194478) v5 -961> 2014-08-13 17:23:22.659268 7fcd0964c700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:23:22.659267, event: done, request: pg_notify(22.362(26) epoch 194478) v5 -960> 2014-08-13 17:23:22.659297 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 13 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e21c0 con 0x4e35ac0 -959> 2014-08-13 17:23:22.659318 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -958> 2014-08-13 17:23:22.659329 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 10 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bba80 con 0x4fd9a20 -957> 2014-08-13 17:23:22.659347 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -956> 2014-08-13 17:23:22.659358 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 14 ==== pg_notify(24.1f7(28) epoch 194478) v5 ==== 2320+0+0 (2923320972 0 0) 0x5b4a4c0 con 0x4fd5960 -955> 2014-08-13 17:23:22.659374 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.650601, event: header_read, request: pg_notify(24.1f7(28) epoch 194478) v5 -954> 2014-08-13 17:23:22.659385 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.650604, event: throttled, request: pg_notify(24.1f7(28) epoch 194478) v5 -953> 2014-08-13 17:23:22.659378 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -952> 2014-08-13 17:23:22.659394 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.650738, event: all_read, request: pg_notify(24.1f7(28) epoch 194478) v5 -951> 2014-08-13 17:23:22.659417 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.659372, event: dispatched, request: pg_notify(24.1f7(28) epoch 194478) v5 -950> 2014-08-13 17:23:22.659429 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.659429, event: waiting_for_osdmap, request: pg_notify(24.1f7(28) epoch 194478) v5 -949> 2014-08-13 17:23:22.659440 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.659440, event: started, request: pg_notify(24.1f7(28) epoch 194478) v5 -948> 2014-08-13 17:23:22.659516 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x567d340 con 0x4e33de0 -947> 2014-08-13 17:23:22.659548 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194478) v5 -- ?+0 0x56ae740 con 0x4e33de0 -946> 2014-08-13 17:23:22.659628 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194478) v5 -- ?+0 0x56a8fc0 con 0x4fde5c0 -945> 2014-08-13 17:23:22.659652 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(33) epoch 194478) v5 -- ?+0 0x56a88c0 con 0x5305ee0 -944> 2014-08-13 17:23:22.659688 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.115:6802/58522 -- pg_info(1 pgs e194478:24.34(50)) v4 -- ?+0 0x588f000 con 0x55244c0 -943> 2014-08-13 17:23:22.659710 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x567d100 con 0x4ec3f40 -942> 2014-08-13 17:23:22.659756 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6823/55491 -- pg_info(1 pgs e194478:24.34(50)) v4 -- ?+0 0x588a300 con 0x4ec3f40 -941> 2014-08-13 17:23:22.659777 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_info(1 pgs e194478:22.99(23)) v4 -- ?+0 0x588e040 con 0x4ec56a0 -940> 2014-08-13 17:23:22.659810 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6826/11446 -- pg_info(1 pgs e194478:22.775(27)) v4 -- ?+0 0x588b800 con 0x4eb48e0 -939> 2014-08-13 17:23:22.659830 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x52bad00 con 0x4ec5d80 -938> 2014-08-13 17:23:22.659855 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6820/35445 -- pg_info(1 pgs e194478:22.3e4(27)) v4 -- ?+0 0x588b640 con 0x4ec5d80 -937> 2014-08-13 17:23:22.659960 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6814/43857 -- pg_info(1 pgs e194478:22.3e4(27)) v4 -- ?+0 0x588e3c0 con 0x55f2100 -936> 2014-08-13 17:23:22.660019 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194478:24.34(50)) v4 -- ?+0 0x588d780 con 0x4e35540 -935> 2014-08-13 17:23:22.660048 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6812/6757 -- pg_info(1 pgs e194478:22.39d(25)) v4 -- ?+0 0x588c600 con 0x4ec70c0 -934> 2014-08-13 17:23:22.660237 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6836/6227 -- pg_info(1 pgs e194478:22.775(27)) v4 -- ?+0 0x588cd00 con 0x53011e0 -933> 2014-08-13 17:23:22.660273 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x52b8000 con 0x4fd32e0 -932> 2014-08-13 17:23:22.660315 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6840/7435 -- pg_info(1 pgs e194478:22.39d(25)) v4 -- ?+0 0x56a9340 con 0x4fd32e0 -931> 2014-08-13 17:23:22.660333 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x5678b40 con 0x4eb18c0 -930> 2014-08-13 17:23:22.660358 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6828/16129 -- pg_info(1 pgs e194478:22.99(23)) v4 -- ?+0 0x56ad400 con 0x4eb18c0 -929> 2014-08-13 17:23:22.660818 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.949717 2 0.000071 -928> 2014-08-13 17:23:22.660863 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.991372 0 0.000000 -927> 2014-08-13 17:23:22.660893 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194471 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -926> 2014-08-13 17:23:22.660750 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194477 pi=193678-194476/29 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating -925> 2014-08-13 17:23:22.661187 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -924> 2014-08-13 17:23:22.661230 7fcd0964c700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:23:22.661229, event: done, request: pg_notify(24.1f7(28) epoch 194478) v5 -923> 2014-08-13 17:23:22.661261 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 0.960549 6 0.000236 -922> 2014-08-13 17:23:22.661283 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 14 ==== pg_info(1 pgs e194478:24.3f7(6)) v4 ==== 1090+0+0 (2763189364 0 0) 0x5187c40 con 0x5305ee0 -921> 2014-08-13 17:23:22.661297 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 1.000525 0 0.000000 -920> 2014-08-13 17:23:22.661313 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.642662, event: header_read, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -919> 2014-08-13 17:23:22.661323 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.642665, event: throttled, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -918> 2014-08-13 17:23:22.661334 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.642749, event: all_read, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -917> 2014-08-13 17:23:22.661326 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194472 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active -916> 2014-08-13 17:23:22.661344 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.661310, event: dispatched, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -915> 2014-08-13 17:23:22.661352 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.661352, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -914> 2014-08-13 17:23:22.661365 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.661365, event: started, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -913> 2014-08-13 17:23:22.661394 7fcd0964c700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:23:22.661393, event: done, request: pg_info(1 pgs e194478:24.3f7(6)) v4 -912> 2014-08-13 17:23:22.661427 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 8 ==== pg_notify(22.113(3) epoch 194478) v5 ==== 887+0+0 (722905662 0 0) 0x588ee40 con 0x5305960 -911> 2014-08-13 17:23:22.661451 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.657351, event: header_read, request: pg_notify(22.113(3) epoch 194478) v5 -910> 2014-08-13 17:23:22.661460 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.657352, event: throttled, request: pg_notify(22.113(3) epoch 194478) v5 -909> 2014-08-13 17:23:22.661471 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.657426, event: all_read, request: pg_notify(22.113(3) epoch 194478) v5 -908> 2014-08-13 17:23:22.661477 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.661449, event: dispatched, request: pg_notify(22.113(3) epoch 194478) v5 -907> 2014-08-13 17:23:22.661919 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.952789 2 0.000147 -906> 2014-08-13 17:23:22.661949 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.991342 0 0.000000 -905> 2014-08-13 17:23:22.661972 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194471 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -904> 2014-08-13 17:23:22.661483 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.661483, event: waiting_for_osdmap, request: pg_notify(22.113(3) epoch 194478) v5 -903> 2014-08-13 17:23:22.662100 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.662100, event: started, request: pg_notify(22.113(3) epoch 194478) v5 -902> 2014-08-13 17:23:22.662139 7fcd0964c700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:23:22.662139, event: done, request: pg_notify(22.113(3) epoch 194478) v5 -901> 2014-08-13 17:23:22.662182 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 14 ==== pg_notify(22.3af(5) epoch 194478) v5 ==== 1001+0+0 (365653847 0 0) 0x56adb00 con 0x4e35ac0 -900> 2014-08-13 17:23:22.662209 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.658159, event: header_read, request: pg_notify(22.3af(5) epoch 194478) v5 -899> 2014-08-13 17:23:22.662219 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.658160, event: throttled, request: pg_notify(22.3af(5) epoch 194478) v5 -898> 2014-08-13 17:23:22.662229 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.658249, event: all_read, request: pg_notify(22.3af(5) epoch 194478) v5 -897> 2014-08-13 17:23:22.662241 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.662205, event: dispatched, request: pg_notify(22.3af(5) epoch 194478) v5 -896> 2014-08-13 17:23:22.662252 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.662252, event: waiting_for_osdmap, request: pg_notify(22.3af(5) epoch 194478) v5 -895> 2014-08-13 17:23:22.662263 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.662263, event: started, request: pg_notify(22.3af(5) epoch 194478) v5 -894> 2014-08-13 17:23:22.662289 7fcd0964c700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:23:22.662288, event: done, request: pg_notify(22.3af(5) epoch 194478) v5 -893> 2014-08-13 17:23:22.662319 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 12 ==== pg_info(1 pgs e194478:22.99) v4 ==== 714+0+0 (1978090172 0 0) 0x588e040 con 0x4ec56a0 -892> 2014-08-13 17:23:22.662339 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.661467, event: header_read, request: pg_info(1 pgs e194478:22.99) v4 -891> 2014-08-13 17:23:22.662328 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194477 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -890> 2014-08-13 17:23:22.662350 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.661468, event: throttled, request: pg_info(1 pgs e194478:22.99) v4 -889> 2014-08-13 17:23:22.662359 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.661535, event: all_read, request: pg_info(1 pgs e194478:22.99) v4 -888> 2014-08-13 17:23:22.662368 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.662337, event: dispatched, request: pg_info(1 pgs e194478:22.99) v4 -887> 2014-08-13 17:23:22.662378 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.662378, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.99) v4 -886> 2014-08-13 17:23:22.662390 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.662389, event: started, request: pg_info(1 pgs e194478:22.99) v4 -885> 2014-08-13 17:23:22.662417 7fcd0964c700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:23:22.662417, event: done, request: pg_info(1 pgs e194478:22.99) v4 -884> 2014-08-13 17:23:22.662436 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.313 192.168.1.109:6820/35445 9 ==== pg_info(1 pgs e194478:22.3e4) v4 ==== 714+0+0 (3657874177 0 0) 0x588b640 con 0x4ec5d80 -883> 2014-08-13 17:23:22.662456 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.661696, event: header_read, request: pg_info(1 pgs e194478:22.3e4) v4 -882> 2014-08-13 17:23:22.662465 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.661697, event: throttled, request: pg_info(1 pgs e194478:22.3e4) v4 -881> 2014-08-13 17:23:22.662474 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.661799, event: all_read, request: pg_info(1 pgs e194478:22.3e4) v4 -880> 2014-08-13 17:23:22.662483 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.662453, event: dispatched, request: pg_info(1 pgs e194478:22.3e4) v4 -879> 2014-08-13 17:23:22.662492 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.662492, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3e4) v4 -878> 2014-08-13 17:23:22.662503 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.662503, event: started, request: pg_info(1 pgs e194478:22.3e4) v4 -877> 2014-08-13 17:23:22.662526 7fcd0964c700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:23:22.662525, event: done, request: pg_info(1 pgs e194478:22.3e4) v4 -876> 2014-08-13 17:23:22.662542 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 6 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e3cc0 con 0x4f832e0 -875> 2014-08-13 17:23:22.662561 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -874> 2014-08-13 17:23:22.662555 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 0.962230 4 0.000245 -873> 2014-08-13 17:23:22.662572 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 11 ==== pg_notify(22.113(24) epoch 194478) v5 ==== 2062+0+0 (2582890308 0 0) 0x5187700 con 0x4fd9a20 -872> 2014-08-13 17:23:22.662605 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.657995, event: header_read, request: pg_notify(22.113(24) epoch 194478) v5 -871> 2014-08-13 17:23:22.662596 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 0.991080 0 0.000000 -870> 2014-08-13 17:23:22.662619 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.657996, event: throttled, request: pg_notify(22.113(24) epoch 194478) v5 -869> 2014-08-13 17:23:22.662629 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.658262, event: all_read, request: pg_notify(22.113(24) epoch 194478) v5 -868> 2014-08-13 17:23:22.662638 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.662602, event: dispatched, request: pg_notify(22.113(24) epoch 194478) v5 -867> 2014-08-13 17:23:22.662627 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194472 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active -866> 2014-08-13 17:23:22.662655 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.662655, event: waiting_for_osdmap, request: pg_notify(22.113(24) epoch 194478) v5 -865> 2014-08-13 17:23:22.662672 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.662671, event: started, request: pg_notify(22.113(24) epoch 194478) v5 -864> 2014-08-13 17:23:22.662699 7fcd0964c700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:23:22.662699, event: done, request: pg_notify(22.113(24) epoch 194478) v5 -863> 2014-08-13 17:23:22.662727 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 11 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bcc80 con 0x4fd06e0 -862> 2014-08-13 17:23:22.662749 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -861> 2014-08-13 17:23:22.662767 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 15 ==== pg_info(1 pgs e194478:22.226(5)) v4 ==== 997+0+0 (2937778253 0 0) 0x5b4a140 con 0x4fd5960 -860> 2014-08-13 17:23:22.662785 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.650767, event: header_read, request: pg_info(1 pgs e194478:22.226(5)) v4 -859> 2014-08-13 17:23:22.662796 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.650769, event: throttled, request: pg_info(1 pgs e194478:22.226(5)) v4 -858> 2014-08-13 17:23:22.662805 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.650854, event: all_read, request: pg_info(1 pgs e194478:22.226(5)) v4 -857> 2014-08-13 17:23:22.662814 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.662783, event: dispatched, request: pg_info(1 pgs e194478:22.226(5)) v4 -856> 2014-08-13 17:23:22.662824 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.662824, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.226(5)) v4 -855> 2014-08-13 17:23:22.662836 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.662836, event: started, request: pg_info(1 pgs e194478:22.226(5)) v4 -854> 2014-08-13 17:23:22.662861 7fcd0964c700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:23:22.662860, event: done, request: pg_info(1 pgs e194478:22.226(5)) v4 -853> 2014-08-13 17:23:22.662879 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.353 192.168.1.108:6836/6227 9 ==== pg_info(1 pgs e194478:22.775) v4 ==== 714+0+0 (4042284257 0 0) 0x588cd00 con 0x53011e0 -852> 2014-08-13 17:23:22.662897 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662251, event: header_read, request: pg_info(1 pgs e194478:22.775) v4 -851> 2014-08-13 17:23:22.662907 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662254, event: throttled, request: pg_info(1 pgs e194478:22.775) v4 -850> 2014-08-13 17:23:22.662915 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662327, event: all_read, request: pg_info(1 pgs e194478:22.775) v4 -849> 2014-08-13 17:23:22.662924 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662895, event: dispatched, request: pg_info(1 pgs e194478:22.775) v4 -848> 2014-08-13 17:23:22.662934 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662934, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.775) v4 -847> 2014-08-13 17:23:22.662945 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662944, event: started, request: pg_info(1 pgs e194478:22.775) v4 -846> 2014-08-13 17:23:22.662967 7fcd0964c700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:23:22.662967, event: done, request: pg_info(1 pgs e194478:22.775) v4 -845> 2014-08-13 17:23:22.662984 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 9 ==== pg_notify(22.3e4(26) epoch 194478) v5 ==== 2172+0+0 (2208776586 0 0) 0x588ec80 con 0x5305960 -844> 2014-08-13 17:23:22.663002 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.657913, event: header_read, request: pg_notify(22.3e4(26) epoch 194478) v5 -843> 2014-08-13 17:23:22.663012 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.657915, event: throttled, request: pg_notify(22.3e4(26) epoch 194478) v5 -842> 2014-08-13 17:23:22.663021 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.658036, event: all_read, request: pg_notify(22.3e4(26) epoch 194478) v5 -841> 2014-08-13 17:23:22.663030 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.663000, event: dispatched, request: pg_notify(22.3e4(26) epoch 194478) v5 -840> 2014-08-13 17:23:22.663039 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.663039, event: waiting_for_osdmap, request: pg_notify(22.3e4(26) epoch 194478) v5 -839> 2014-08-13 17:23:22.663050 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.663050, event: started, request: pg_notify(22.3e4(26) epoch 194478) v5 -838> 2014-08-13 17:23:22.663082 7fcd0964c700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:23:22.663082, event: done, request: pg_notify(22.3e4(26) epoch 194478) v5 -837> 2014-08-13 17:23:22.663118 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 15 ==== pg_notify(22.362(5) epoch 194478) v5 ==== 1001+0+0 (4103138994 0 0) 0x5889500 con 0x4e35ac0 -836> 2014-08-13 17:23:22.663138 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.661927, event: header_read, request: pg_notify(22.362(5) epoch 194478) v5 -835> 2014-08-13 17:23:22.663149 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.661929, event: throttled, request: pg_notify(22.362(5) epoch 194478) v5 -834> 2014-08-13 17:23:22.663170 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.662036, event: all_read, request: pg_notify(22.362(5) epoch 194478) v5 -833> 2014-08-13 17:23:22.663212 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.663136, event: dispatched, request: pg_notify(22.362(5) epoch 194478) v5 -832> 2014-08-13 17:23:22.663223 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.663223, event: waiting_for_osdmap, request: pg_notify(22.362(5) epoch 194478) v5 -831> 2014-08-13 17:23:22.663235 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.663235, event: started, request: pg_notify(22.362(5) epoch 194478) v5 -830> 2014-08-13 17:23:22.663263 7fcd0964c700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:23:22.663263, event: done, request: pg_notify(22.362(5) epoch 194478) v5 -829> 2014-08-13 17:23:22.663292 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 7 ==== pg_notify(22.3d9(42) epoch 194478) v5 ==== 2996+0+0 (2530850689 0 0) 0x5b4a300 con 0x4f832e0 -828> 2014-08-13 17:23:22.663316 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.661753, event: header_read, request: pg_notify(22.3d9(42) epoch 194478) v5 -827> 2014-08-13 17:23:22.663324 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.661754, event: throttled, request: pg_notify(22.3d9(42) epoch 194478) v5 -826> 2014-08-13 17:23:22.663335 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.661925, event: all_read, request: pg_notify(22.3d9(42) epoch 194478) v5 -825> 2014-08-13 17:23:22.663345 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.663311, event: dispatched, request: pg_notify(22.3d9(42) epoch 194478) v5 -824> 2014-08-13 17:23:22.663354 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.663354, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194478) v5 -823> 2014-08-13 17:23:22.663366 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.663365, event: started, request: pg_notify(22.3d9(42) epoch 194478) v5 -822> 2014-08-13 17:23:22.663391 7fcd0964c700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:23:22.663391, event: done, request: pg_notify(22.3d9(42) epoch 194478) v5 -821> 2014-08-13 17:23:22.663429 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 12 ==== pg_notify(24.1f7(6) epoch 194478) v5 ==== 1094+0+0 (66306695 0 0) 0x5187000 con 0x4fd9a20 -820> 2014-08-13 17:23:22.663445 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.658639, event: header_read, request: pg_notify(24.1f7(6) epoch 194478) v5 -819> 2014-08-13 17:23:22.663456 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.658640, event: throttled, request: pg_notify(24.1f7(6) epoch 194478) v5 -818> 2014-08-13 17:23:22.663465 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.658716, event: all_read, request: pg_notify(24.1f7(6) epoch 194478) v5 -817> 2014-08-13 17:23:22.663474 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.663442, event: dispatched, request: pg_notify(24.1f7(6) epoch 194478) v5 -816> 2014-08-13 17:23:22.663484 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.663483, event: waiting_for_osdmap, request: pg_notify(24.1f7(6) epoch 194478) v5 -815> 2014-08-13 17:23:22.663494 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.663494, event: started, request: pg_notify(24.1f7(6) epoch 194478) v5 -814> 2014-08-13 17:23:22.663225 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194477 pi=193747-194476/27 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating -813> 2014-08-13 17:23:22.663520 7fcd0964c700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:23:22.663520, event: done, request: pg_notify(24.1f7(6) epoch 194478) v5 -812> 2014-08-13 17:23:22.663541 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 12 ==== pg_notify(22.707(5) epoch 194478) v5 ==== 1001+0+0 (1360284176 0 0) 0x5889a40 con 0x4fd06e0 -811> 2014-08-13 17:23:22.663560 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.659990, event: header_read, request: pg_notify(22.707(5) epoch 194478) v5 -810> 2014-08-13 17:23:22.663570 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.659991, event: throttled, request: pg_notify(22.707(5) epoch 194478) v5 -809> 2014-08-13 17:23:22.663579 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.660096, event: all_read, request: pg_notify(22.707(5) epoch 194478) v5 -808> 2014-08-13 17:23:22.663588 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.663558, event: dispatched, request: pg_notify(22.707(5) epoch 194478) v5 -807> 2014-08-13 17:23:22.663597 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.663597, event: waiting_for_osdmap, request: pg_notify(22.707(5) epoch 194478) v5 -806> 2014-08-13 17:23:22.663610 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.663609, event: started, request: pg_notify(22.707(5) epoch 194478) v5 -805> 2014-08-13 17:23:22.663629 7fcd0964c700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:23:22.663628, event: done, request: pg_notify(22.707(5) epoch 194478) v5 -804> 2014-08-13 17:23:22.663649 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 6 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e6c00 con 0x4fd2520 -803> 2014-08-13 17:23:22.663667 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -802> 2014-08-13 17:23:22.663677 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.342 192.168.1.110:6802/41438 10 ==== pg_notify(22.707(28) epoch 194478) v5 ==== 2290+0+0 (3795548537 0 0) 0x588d940 con 0x5305960 -801> 2014-08-13 17:23:22.663695 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.658748, event: header_read, request: pg_notify(22.707(28) epoch 194478) v5 -800> 2014-08-13 17:23:22.663707 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.658750, event: throttled, request: pg_notify(22.707(28) epoch 194478) v5 -799> 2014-08-13 17:23:22.663715 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.658845, event: all_read, request: pg_notify(22.707(28) epoch 194478) v5 -798> 2014-08-13 17:23:22.663725 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.663693, event: dispatched, request: pg_notify(22.707(28) epoch 194478) v5 -797> 2014-08-13 17:23:22.663732 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.663732, event: waiting_for_osdmap, request: pg_notify(22.707(28) epoch 194478) v5 -796> 2014-08-13 17:23:22.663740 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.663740, event: started, request: pg_notify(22.707(28) epoch 194478) v5 -795> 2014-08-13 17:23:22.663759 7fcd0964c700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:23:22.663759, event: done, request: pg_notify(22.707(28) epoch 194478) v5 -794> 2014-08-13 17:23:22.663787 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 16 ==== pg_notify(24.136(27) epoch 194478) v5 ==== 2261+0+0 (59376062 0 0) 0x5889340 con 0x4e35ac0 -793> 2014-08-13 17:23:22.663803 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.662072, event: header_read, request: pg_notify(24.136(27) epoch 194478) v5 -792> 2014-08-13 17:23:22.663810 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.662073, event: throttled, request: pg_notify(24.136(27) epoch 194478) v5 -791> 2014-08-13 17:23:22.663815 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.662229, event: all_read, request: pg_notify(24.136(27) epoch 194478) v5 -790> 2014-08-13 17:23:22.663820 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.663801, event: dispatched, request: pg_notify(24.136(27) epoch 194478) v5 -789> 2014-08-13 17:23:22.663827 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.663826, event: waiting_for_osdmap, request: pg_notify(24.136(27) epoch 194478) v5 -788> 2014-08-13 17:23:22.663834 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.663834, event: started, request: pg_notify(24.136(27) epoch 194478) v5 -787> 2014-08-13 17:23:22.663855 7fcd0964c700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:23:22.663854, event: done, request: pg_notify(24.136(27) epoch 194478) v5 -786> 2014-08-13 17:23:22.663872 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 13 ==== pg_notify(24.136(27),24.181(7) epoch 194478) v5 ==== 3398+0+0 (3017190879 0 0) 0x54d7380 con 0x4fd9a20 -785> 2014-08-13 17:23:22.663887 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.659064, event: header_read, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -784> 2014-08-13 17:23:22.663894 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.659066, event: throttled, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -783> 2014-08-13 17:23:22.663900 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.659214, event: all_read, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -782> 2014-08-13 17:23:22.663906 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.663885, event: dispatched, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -781> 2014-08-13 17:23:22.663913 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.663912, event: waiting_for_osdmap, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -780> 2014-08-13 17:23:22.663920 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.663920, event: started, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -779> 2014-08-13 17:23:22.663918 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/WaitUpThru 0.963858 5 0.000266 -778> 2014-08-13 17:23:22.663950 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering 0.998031 0 0.000000 -777> 2014-08-13 17:23:22.663975 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194449 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active -776> 2014-08-13 17:23:22.664755 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194477 pi=193644-194476/27 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating -775> 2014-08-13 17:23:22.665539 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.963533 2 0.000066 -774> 2014-08-13 17:23:22.665575 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.995704 0 0.000000 -773> 2014-08-13 17:23:22.665601 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194471 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -772> 2014-08-13 17:23:22.665655 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194477 pi=193679-194476/32 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active/Activating -771> 2014-08-13 17:23:22.666047 7fcd0964c700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:23:22.666046, event: done, request: pg_notify(24.136(27),24.181(7) epoch 194478) v5 -770> 2014-08-13 17:23:22.666132 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x5671680 con 0x4f80b00 -769> 2014-08-13 17:23:22.666146 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 13 ==== pg_notify(22.3e4(5) epoch 194478) v5 ==== 1001+0+0 (581357327 0 0) 0x588cb40 con 0x4fd06e0 -768> 2014-08-13 17:23:22.666181 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.660161, event: header_read, request: pg_notify(22.3e4(5) epoch 194478) v5 -767> 2014-08-13 17:23:22.666189 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.660163, event: throttled, request: pg_notify(22.3e4(5) epoch 194478) v5 -766> 2014-08-13 17:23:22.666195 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.660287, event: all_read, request: pg_notify(22.3e4(5) epoch 194478) v5 -765> 2014-08-13 17:23:22.666201 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.666177, event: dispatched, request: pg_notify(22.3e4(5) epoch 194478) v5 -764> 2014-08-13 17:23:22.666208 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.666208, event: waiting_for_osdmap, request: pg_notify(22.3e4(5) epoch 194478) v5 -763> 2014-08-13 17:23:22.666182 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -762> 2014-08-13 17:23:22.666236 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.666236, event: started, request: pg_notify(22.3e4(5) epoch 194478) v5 -761> 2014-08-13 17:23:22.666264 7fcd0964c700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:23:22.666264, event: done, request: pg_notify(22.3e4(5) epoch 194478) v5 -760> 2014-08-13 17:23:22.666277 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(30) epoch 194478) v5 -- ?+0 0x5b4bf00 con 0x4f80b00 -759> 2014-08-13 17:23:22.666282 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 7 ==== pg_notify(24.30b(33) epoch 194478) v5 ==== 2615+0+0 (2787529404 0 0) 0x5b4c0c0 con 0x4fd2520 -758> 2014-08-13 17:23:22.666299 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.663316, event: header_read, request: pg_notify(24.30b(33) epoch 194478) v5 -757> 2014-08-13 17:23:22.666306 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.663317, event: throttled, request: pg_notify(24.30b(33) epoch 194478) v5 -756> 2014-08-13 17:23:22.666312 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.663432, event: all_read, request: pg_notify(24.30b(33) epoch 194478) v5 -755> 2014-08-13 17:23:22.666317 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.666297, event: dispatched, request: pg_notify(24.30b(33) epoch 194478) v5 -754> 2014-08-13 17:23:22.666324 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.666323, event: waiting_for_osdmap, request: pg_notify(24.30b(33) epoch 194478) v5 -753> 2014-08-13 17:23:22.666331 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.666331, event: started, request: pg_notify(24.30b(33) epoch 194478) v5 -752> 2014-08-13 17:23:22.666350 7fcd0964c700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:23:22.666350, event: done, request: pg_notify(24.30b(33) epoch 194478) v5 -751> 2014-08-13 17:23:22.666423 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 4 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567af40 con 0x55244c0 -750> 2014-08-13 17:23:22.666449 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -749> 2014-08-13 17:23:22.666460 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e2400 con 0x4e370c0 -748> 2014-08-13 17:23:22.666480 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -747> 2014-08-13 17:23:22.666475 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x5671440 con 0x4ec4780 -746> 2014-08-13 17:23:22.666492 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 14 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x2e8ad00 con 0x4f86e00 -745> 2014-08-13 17:23:22.666510 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -744> 2014-08-13 17:23:22.666521 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.303 192.168.1.109:6802/30768 14 ==== pg_notify(22.481(34) epoch 194478) v5 ==== 2628+0+0 (1591511596 0 0) 0x588c980 con 0x4fd06e0 -743> 2014-08-13 17:23:22.666646 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.661443, event: header_read, request: pg_notify(22.481(34) epoch 194478) v5 -742> 2014-08-13 17:23:22.666665 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.661445, event: throttled, request: pg_notify(22.481(34) epoch 194478) v5 -741> 2014-08-13 17:23:22.666674 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.661579, event: all_read, request: pg_notify(22.481(34) epoch 194478) v5 -740> 2014-08-13 17:23:22.666683 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.666641, event: dispatched, request: pg_notify(22.481(34) epoch 194478) v5 -739> 2014-08-13 17:23:22.666697 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.666697, event: waiting_for_osdmap, request: pg_notify(22.481(34) epoch 194478) v5 -738> 2014-08-13 17:23:22.666694 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(38) epoch 194478) v5 -- ?+0 0x588d400 con 0x4ec56a0 -737> 2014-08-13 17:23:22.666587 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6832/5235 -- pg_notify(24.3b7(33) epoch 194478) v5 -- ?+0 0x5b4bd40 con 0x4ec4780 -736> 2014-08-13 17:23:22.666735 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.666735, event: started, request: pg_notify(22.481(34) epoch 194478) v5 -735> 2014-08-13 17:23:22.666732 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x56718c0 con 0x4fd5960 -734> 2014-08-13 17:23:22.666770 7fcd0964c700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:23:22.666770, event: done, request: pg_notify(22.481(34) epoch 194478) v5 -733> 2014-08-13 17:23:22.666765 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6832/16753 -- pg_notify(22.653(26) epoch 194478) v5 -- ?+0 0x5b4bb80 con 0x4fd86e0 -732> 2014-08-13 17:23:22.666805 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.375 192.168.1.114:6830/52879 8 ==== pg_info(1 pgs e194478:24.30b) v4 ==== 748+0+0 (2228521417 0 0) 0x54d4440 con 0x4fd2520 -731> 2014-08-13 17:23:22.666824 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.665971, event: header_read, request: pg_info(1 pgs e194478:24.30b) v4 -730> 2014-08-13 17:23:22.666831 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.665974, event: throttled, request: pg_info(1 pgs e194478:24.30b) v4 -729> 2014-08-13 17:23:22.666841 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.666074, event: all_read, request: pg_info(1 pgs e194478:24.30b) v4 -728> 2014-08-13 17:23:22.666847 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.666821, event: dispatched, request: pg_info(1 pgs e194478:24.30b) v4 -727> 2014-08-13 17:23:22.666853 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.666853, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.30b) v4 -726> 2014-08-13 17:23:22.666861 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.666861, event: started, request: pg_info(1 pgs e194478:24.30b) v4 -725> 2014-08-13 17:23:22.666883 7fcd0964c700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:23:22.666883, event: done, request: pg_info(1 pgs e194478:24.30b) v4 -724> 2014-08-13 17:23:22.666878 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(26) epoch 194478) v5 -- ?+0 0x588b100 con 0x4fd5960 -723> 2014-08-13 17:23:22.666894 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x5671200 con 0x4eb2680 -722> 2014-08-13 17:23:22.666897 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 5 ==== pg_notify(24.34(37) epoch 194478) v5 ==== 2623+0+0 (2862086254 0 0) 0x588f000 con 0x55244c0 -721> 2014-08-13 17:23:22.666912 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666108, event: header_read, request: pg_notify(24.34(37) epoch 194478) v5 -720> 2014-08-13 17:23:22.666919 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666109, event: throttled, request: pg_notify(24.34(37) epoch 194478) v5 -719> 2014-08-13 17:23:22.666924 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666237, event: all_read, request: pg_notify(24.34(37) epoch 194478) v5 -718> 2014-08-13 17:23:22.666929 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666910, event: dispatched, request: pg_notify(24.34(37) epoch 194478) v5 -717> 2014-08-13 17:23:22.666936 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666935, event: waiting_for_osdmap, request: pg_notify(24.34(37) epoch 194478) v5 -716> 2014-08-13 17:23:22.666943 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666943, event: started, request: pg_notify(24.34(37) epoch 194478) v5 -715> 2014-08-13 17:23:22.666964 7fcd0964c700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:23:22.666964, event: done, request: pg_notify(24.34(37) epoch 194478) v5 -714> 2014-08-13 17:23:22.666988 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.62 192.168.1.112:6824/5505 6 ==== pg_notify(22.619(27) epoch 194478) v5 ==== 2231+0+0 (2900512090 0 0) 0x5b4c7c0 con 0x4e370c0 -713> 2014-08-13 17:23:22.667009 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.664731, event: header_read, request: pg_notify(22.619(27) epoch 194478) v5 -712> 2014-08-13 17:23:22.667016 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.664733, event: throttled, request: pg_notify(22.619(27) epoch 194478) v5 -711> 2014-08-13 17:23:22.667022 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.664851, event: all_read, request: pg_notify(22.619(27) epoch 194478) v5 -710> 2014-08-13 17:23:22.667031 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.667007, event: dispatched, request: pg_notify(22.619(27) epoch 194478) v5 -709> 2014-08-13 17:23:22.667041 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.667040, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194478) v5 -708> 2014-08-13 17:23:22.667052 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.667052, event: started, request: pg_notify(22.619(27) epoch 194478) v5 -707> 2014-08-13 17:23:22.667045 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6822/10782 -- pg_info(1 pgs e194478:22.3af(27)) v4 -- ?+0 0x58881c0 con 0x4eb0c60 -706> 2014-08-13 17:23:22.667076 7fcd0964c700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:23:22.667075, event: done, request: pg_notify(22.619(27) epoch 194478) v5 -705> 2014-08-13 17:23:22.667083 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6808/15284 -- pg_info(1 pgs e194478:22.48e(25)) v4 -- ?+0 0x588e580 con 0x4ec3440 -704> 2014-08-13 17:23:22.667078 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6827/40192 -- pg_info(1 pgs e194478:22.1f1(27)) v4 -- ?+0 0x5b4b9c0 con 0x4eb2680 -703> 2014-08-13 17:23:22.667097 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 15 ==== pg_notify(22.362(27) epoch 194478) v5 ==== 2231+0+0 (2161029568 0 0) 0x5b4c280 con 0x4f86e00 -702> 2014-08-13 17:23:22.667118 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.666327, event: header_read, request: pg_notify(22.362(27) epoch 194478) v5 -701> 2014-08-13 17:23:22.667118 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6816/837 -- pg_info(1 pgs e194478:24.1bb(27)) v4 -- ?+0 0x588f8c0 con 0x4fde880 -700> 2014-08-13 17:23:22.667129 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.666328, event: throttled, request: pg_notify(22.362(27) epoch 194478) v5 -699> 2014-08-13 17:23:22.667138 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.666435, event: all_read, request: pg_notify(22.362(27) epoch 194478) v5 -698> 2014-08-13 17:23:22.667147 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.667116, event: dispatched, request: pg_notify(22.362(27) epoch 194478) v5 -697> 2014-08-13 17:23:22.667154 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.667153, event: waiting_for_osdmap, request: pg_notify(22.362(27) epoch 194478) v5 -696> 2014-08-13 17:23:22.667178 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.667178, event: started, request: pg_notify(22.362(27) epoch 194478) v5 -695> 2014-08-13 17:23:22.667204 7fcd0964c700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:23:22.667203, event: done, request: pg_notify(22.362(27) epoch 194478) v5 -694> 2014-08-13 17:23:22.667398 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.343 192.168.1.104:6802/61960 16 ==== pg_info(1 pgs e194478:22.362) v4 ==== 714+0+0 (328940323 0 0) 0x54d3f00 con 0x4f86e00 -693> 2014-08-13 17:23:22.667124 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6811/37658 -- pg_info(1 pgs e194478:24.b6(29)) v4 -- ?+0 0x5b4b800 con 0x4ec5ac0 -692> 2014-08-13 17:23:22.667333 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6806/42024 -- pg_info(1 pgs e194478:24.1bb(27)) v4 -- ?+0 0x588b9c0 con 0x4fdbde0 -691> 2014-08-13 17:23:22.667421 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667226, event: header_read, request: pg_info(1 pgs e194478:22.362) v4 -690> 2014-08-13 17:23:22.667429 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667229, event: throttled, request: pg_info(1 pgs e194478:22.362) v4 -689> 2014-08-13 17:23:22.667435 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667357, event: all_read, request: pg_info(1 pgs e194478:22.362) v4 -688> 2014-08-13 17:23:22.667440 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667417, event: dispatched, request: pg_info(1 pgs e194478:22.362) v4 -687> 2014-08-13 17:23:22.667446 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667446, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.362) v4 -686> 2014-08-13 17:23:22.667454 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667454, event: started, request: pg_info(1 pgs e194478:22.362) v4 -685> 2014-08-13 17:23:22.667449 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.107:6821/54786 -- pg_info(1 pgs e194478:22.707(27)) v4 -- ?+0 0x5b4b640 con 0x4ec7e80 -684> 2014-08-13 17:23:22.667459 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.104:6804/62197 -- pg_info(1 pgs e194478:24.1bb(27)) v4 -- ?+0 0x588d080 con 0x4fd8f20 -683> 2014-08-13 17:23:22.667479 7fcd0964c700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:23:22.667479, event: done, request: pg_info(1 pgs e194478:22.362) v4 -682> 2014-08-13 17:23:22.667500 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.382 192.168.1.108:6800/30527 3 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bb180 con 0x552d280 -681> 2014-08-13 17:23:22.667524 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -680> 2014-08-13 17:23:22.667635 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 12 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x5678b40 con 0x4eb18c0 -679> 2014-08-13 17:23:22.667676 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -678> 2014-08-13 17:23:22.667746 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.382 192.168.1.108:6800/30527 4 ==== pg_notify(22.3d9(46) epoch 194478) v5 ==== 3208+0+0 (2037149407 0 0) 0x54d39c0 con 0x552d280 -677> 2014-08-13 17:23:22.667759 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.103:6805/37391 -- pg_info(1 pgs e194478:24.b6(29)) v4 -- ?+0 0x5b4b480 con 0x4e33f40 -676> 2014-08-13 17:23:22.667770 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667527, event: header_read, request: pg_notify(22.3d9(46) epoch 194478) v5 -675> 2014-08-13 17:23:22.667796 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667533, event: throttled, request: pg_notify(22.3d9(46) epoch 194478) v5 -674> 2014-08-13 17:23:22.667804 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667720, event: all_read, request: pg_notify(22.3d9(46) epoch 194478) v5 -673> 2014-08-13 17:23:22.667810 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667767, event: dispatched, request: pg_notify(22.3d9(46) epoch 194478) v5 -672> 2014-08-13 17:23:22.667818 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667817, event: waiting_for_osdmap, request: pg_notify(22.3d9(46) epoch 194478) v5 -671> 2014-08-13 17:23:22.667810 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6814/25893 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x51e57c0 con 0x4f832e0 -670> 2014-08-13 17:23:22.667827 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667827, event: started, request: pg_notify(22.3d9(46) epoch 194478) v5 -669> 2014-08-13 17:23:22.667828 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6828/12102 -- pg_info(1 pgs e194478:22.1f1(27)) v4 -- ?+0 0x5b4b2c0 con 0x57232e0 -668> 2014-08-13 17:23:22.667855 7fcd0964c700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:23:22.667855, event: done, request: pg_notify(22.3d9(46) epoch 194478) v5 -667> 2014-08-13 17:23:22.667857 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x56733c0 con 0x4eb7d20 -666> 2014-08-13 17:23:22.667861 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6814/25893 -- pg_info(1 pgs e194478:22.3d9(45)) v4 -- ?+0 0x588a840 con 0x4f832e0 -665> 2014-08-13 17:23:22.667903 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6842/9607 -- pg_info(1 pgs e194478:24.136(27)) v4 -- ?+0 0x5b4b100 con 0x4eb7d20 -664> 2014-08-13 17:23:22.667908 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e1680 con 0x4fd9760 -663> 2014-08-13 17:23:22.667929 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -662> 2014-08-13 17:23:22.667939 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 13 ==== pg_notify(22.99(23) epoch 194478) v5 ==== 2003+0+0 (2155626529 0 0) 0x56ad400 con 0x4eb18c0 -661> 2014-08-13 17:23:22.667959 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.667637, event: header_read, request: pg_notify(22.99(23) epoch 194478) v5 -660> 2014-08-13 17:23:22.667970 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.667638, event: throttled, request: pg_notify(22.99(23) epoch 194478) v5 -659> 2014-08-13 17:23:22.667979 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.667759, event: all_read, request: pg_notify(22.99(23) epoch 194478) v5 -658> 2014-08-13 17:23:22.667988 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.667956, event: dispatched, request: pg_notify(22.99(23) epoch 194478) v5 -657> 2014-08-13 17:23:22.668000 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.668000, event: waiting_for_osdmap, request: pg_notify(22.99(23) epoch 194478) v5 -656> 2014-08-13 17:23:22.668013 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.668013, event: started, request: pg_notify(22.99(23) epoch 194478) v5 -655> 2014-08-13 17:23:22.668029 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6818/9142 -- pg_info(1 pgs e194478:22.48e(25)) v4 -- ?+0 0x5b49340 con 0x4f87900 -654> 2014-08-13 17:23:22.668035 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6834/6966 -- pg_info(2 pgs e194478:24.1f7(29),24.136(27)) v4 -- ?+0 0x5b4af40 con 0x4fd9a20 -653> 2014-08-13 17:23:22.668076 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.110:6800/41204 -- pg_info(1 pgs e194478:24.1bb(27)) v4 -- ?+0 0x5889500 con 0x53e4fc0 -652> 2014-08-13 17:23:22.668039 7fcd0964c700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:23:22.668039, event: done, request: pg_notify(22.99(23) epoch 194478) v5 -651> 2014-08-13 17:23:22.668099 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.112:6822/5265 -- pg_info(1 pgs e194478:24.1f7(29)) v4 -- ?+0 0x5b4ad80 con 0x53e65c0 -650> 2014-08-13 17:23:22.668104 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6842/8170 -- pg_info(1 pgs e194478:22.3af(27)) v4 -- ?+0 0x588ec80 con 0x4f86040 -649> 2014-08-13 17:23:22.668124 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 10 ==== pg_notify(22.619(27) epoch 194478) v5 ==== 2231+0+0 (4277060227 0 0) 0x5b4c600 con 0x4fd9760 -648> 2014-08-13 17:23:22.668143 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.667770, event: header_read, request: pg_notify(22.619(27) epoch 194478) v5 -647> 2014-08-13 17:23:22.668150 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.667772, event: throttled, request: pg_notify(22.619(27) epoch 194478) v5 -646> 2014-08-13 17:23:22.668155 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.667890, event: all_read, request: pg_notify(22.619(27) epoch 194478) v5 -645> 2014-08-13 17:23:22.668161 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.668140, event: dispatched, request: pg_notify(22.619(27) epoch 194478) v5 -644> 2014-08-13 17:23:22.668168 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.668167, event: waiting_for_osdmap, request: pg_notify(22.619(27) epoch 194478) v5 -643> 2014-08-13 17:23:22.668159 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.108:6838/6893 -- pg_info(1 pgs e194478:22.3d9(45)) v4 -- ?+0 0x588cd00 con 0x4fd5960 -642> 2014-08-13 17:23:22.668175 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.668175, event: started, request: pg_notify(22.619(27) epoch 194478) v5 -641> 2014-08-13 17:23:22.668211 7fcd0964c700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:23:22.668210, event: done, request: pg_notify(22.619(27) epoch 194478) v5 -640> 2014-08-13 17:23:22.668201 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6840/16566 -- pg_info(1 pgs e194478:24.136(27)) v4 -- ?+0 0x5b4e740 con 0x4e35ac0 -639> 2014-08-13 17:23:22.668296 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.111:6842/17345 -- pg_info(1 pgs e194478:24.136(27)) v4 -- ?+0 0x5b4e580 con 0x4fd9760 -638> 2014-08-13 17:23:22.668339 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6806/31286 -- pg_info(1 pgs e194478:24.181(32)) v4 -- ?+0 0x5b4e3c0 con 0x4f81600 -637> 2014-08-13 17:23:22.668464 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.109:6812/32413 -- pg_info(1 pgs e194478:22.707(27)) v4 -- ?+0 0x5b4e200 con 0x5303440 -636> 2014-08-13 17:23:22.668513 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.113:6800/14161 -- pg_info(1 pgs e194478:24.b6(29)) v4 -- ?+0 0x5b4e040 con 0x4eb6460 -635> 2014-08-13 17:23:22.668548 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6820/10050 -- pg_info(1 pgs e194478:24.181(32)) v4 -- ?+0 0x5b4de80 con 0x4fde300 -634> 2014-08-13 17:23:22.668528 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started/Stray 0.999656 8 0.000447 -633> 2014-08-13 17:23:22.668586 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6802/4876 -- pg_info(1 pgs e194478:24.b6(29)) v4 -- ?+0 0x5b4dcc0 con 0x5305ee0 -632> 2014-08-13 17:23:22.668607 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive -631> 2014-08-13 17:23:22.668641 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.106:6812/6757 -- pg_info(1 pgs e194478:24.181(32)) v4 -- ?+0 0x5b4db00 con 0x4ec70c0 -630> 2014-08-13 17:23:22.668641 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194449 n=50 ec=192688 les/c 194475/194372 194476/194477/192978) [31,338,4] r=2 lpr=194477 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive/RepNotRecovering -629> 2014-08-13 17:23:22.668672 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.102:6830/16444 -- pg_info(1 pgs e194478:24.1f7(29)) v4 -- ?+0 0x5b4d940 con 0x4fdee00 -628> 2014-08-13 17:23:22.668913 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.005820 7 0.001929 -627> 2014-08-13 17:23:22.668953 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -626> 2014-08-13 17:23:22.668978 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194471 n=23 ec=183096 les/c 194475/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -625> 2014-08-13 17:23:22.669113 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.004591 9 0.000670 -624> 2014-08-13 17:23:22.669144 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive -623> 2014-08-13 17:23:22.669168 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194449 n=56 ec=192688 les/c 194475/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194477 pi=193150-194476/33 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering -622> 2014-08-13 17:23:22.669716 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567d340 con 0x4e33de0 -621> 2014-08-13 17:23:22.669752 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -620> 2014-08-13 17:23:22.669915 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.331 192.168.1.104:6816/837 11 ==== pg_info(1 pgs e194478:24.1bb) v4 ==== 748+0+0 (3143823142 0 0) 0x588f8c0 con 0x4fde880 -619> 2014-08-13 17:23:22.669956 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.669793, event: header_read, request: pg_info(1 pgs e194478:24.1bb) v4 -618> 2014-08-13 17:23:22.669970 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.669794, event: throttled, request: pg_info(1 pgs e194478:24.1bb) v4 -617> 2014-08-13 17:23:22.669979 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.669861, event: all_read, request: pg_info(1 pgs e194478:24.1bb) v4 -616> 2014-08-13 17:23:22.669990 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.669951, event: dispatched, request: pg_info(1 pgs e194478:24.1bb) v4 -615> 2014-08-13 17:23:22.669998 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.669997, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1bb) v4 -614> 2014-08-13 17:23:22.670011 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.670011, event: started, request: pg_info(1 pgs e194478:24.1bb) v4 -613> 2014-08-13 17:23:22.670039 7fcd0964c700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:23:22.670039, event: done, request: pg_info(1 pgs e194478:24.1bb) v4 -612> 2014-08-13 17:23:22.670061 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.20 192.168.1.103:6815/38269 9 ==== pg_notify(22.99(22) epoch 194478) v5 ==== 1944+0+0 (3989510194 0 0) 0x56ae740 con 0x4e33de0 -611> 2014-08-13 17:23:22.670081 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.669849, event: header_read, request: pg_notify(22.99(22) epoch 194478) v5 -610> 2014-08-13 17:23:22.670092 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.669851, event: throttled, request: pg_notify(22.99(22) epoch 194478) v5 -609> 2014-08-13 17:23:22.670101 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.669967, event: all_read, request: pg_notify(22.99(22) epoch 194478) v5 -608> 2014-08-13 17:23:22.670110 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.670079, event: dispatched, request: pg_notify(22.99(22) epoch 194478) v5 -607> 2014-08-13 17:23:22.670120 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.670119, event: waiting_for_osdmap, request: pg_notify(22.99(22) epoch 194478) v5 -606> 2014-08-13 17:23:22.670128 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.670128, event: started, request: pg_notify(22.99(22) epoch 194478) v5 -605> 2014-08-13 17:23:22.670148 7fcd0964c700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:23:22.670148, event: done, request: pg_notify(22.99(22) epoch 194478) v5 -604> 2014-08-13 17:23:22.670169 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 16 ==== pg_info(1 pgs e194478:22.3d9) v4 ==== 714+0+0 (2855332168 0 0) 0x588cd00 con 0x4fd5960 -603> 2014-08-13 17:23:22.670198 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.669874, event: header_read, request: pg_info(1 pgs e194478:22.3d9) v4 -602> 2014-08-13 17:23:22.670206 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.669876, event: throttled, request: pg_info(1 pgs e194478:22.3d9) v4 -601> 2014-08-13 17:23:22.670215 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.669966, event: all_read, request: pg_info(1 pgs e194478:22.3d9) v4 -600> 2014-08-13 17:23:22.670224 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.670196, event: dispatched, request: pg_info(1 pgs e194478:22.3d9) v4 -599> 2014-08-13 17:23:22.670234 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.670234, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3d9) v4 -598> 2014-08-13 17:23:22.670246 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.670246, event: started, request: pg_info(1 pgs e194478:22.3d9) v4 -597> 2014-08-13 17:23:22.670235 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.371 192.168.1.102:6832/16753 -- pg_info(1 pgs e194478:22.653) v4 -- ?+0 0x54d1180 -596> 2014-08-13 17:23:22.670270 7fcd0964c700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:23:22.670270, event: done, request: pg_info(1 pgs e194478:22.3d9) v4 -595> 2014-08-13 17:23:22.670288 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b9680 con 0x55f2100 -594> 2014-08-13 17:23:22.670311 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -593> 2014-08-13 17:23:22.670324 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.337 192.168.1.104:6804/62197 13 ==== pg_info(1 pgs e194478:24.1bb) v4 ==== 748+0+0 (3492542700 0 0) 0x588d080 con 0x4fd8f20 -592> 2014-08-13 17:23:22.670341 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670103, event: header_read, request: pg_info(1 pgs e194478:24.1bb) v4 -591> 2014-08-13 17:23:22.670351 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670104, event: throttled, request: pg_info(1 pgs e194478:24.1bb) v4 -590> 2014-08-13 17:23:22.670360 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670180, event: all_read, request: pg_info(1 pgs e194478:24.1bb) v4 -589> 2014-08-13 17:23:22.670369 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670339, event: dispatched, request: pg_info(1 pgs e194478:24.1bb) v4 -588> 2014-08-13 17:23:22.670378 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670378, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1bb) v4 -587> 2014-08-13 17:23:22.670390 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670389, event: started, request: pg_info(1 pgs e194478:24.1bb) v4 -586> 2014-08-13 17:23:22.670413 7fcd0964c700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:23:22.670413, event: done, request: pg_info(1 pgs e194478:24.1bb) v4 -585> 2014-08-13 17:23:22.670430 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 15 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b8000 con 0x4fd32e0 -584> 2014-08-13 17:23:22.670451 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -583> 2014-08-13 17:23:22.670462 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 6 ==== pg_notify(22.3e4(27) epoch 194478) v5 ==== 2231+0+0 (2508091589 0 0) 0x588e3c0 con 0x55f2100 -582> 2014-08-13 17:23:22.670480 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670016, event: header_read, request: pg_notify(22.3e4(27) epoch 194478) v5 -581> 2014-08-13 17:23:22.670490 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670017, event: throttled, request: pg_notify(22.3e4(27) epoch 194478) v5 -580> 2014-08-13 17:23:22.670501 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670115, event: all_read, request: pg_notify(22.3e4(27) epoch 194478) v5 -579> 2014-08-13 17:23:22.670507 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670478, event: dispatched, request: pg_notify(22.3e4(27) epoch 194478) v5 -578> 2014-08-13 17:23:22.670516 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670516, event: waiting_for_osdmap, request: pg_notify(22.3e4(27) epoch 194478) v5 -577> 2014-08-13 17:23:22.670527 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670527, event: started, request: pg_notify(22.3e4(27) epoch 194478) v5 -576> 2014-08-13 17:23:22.670552 7fcd0964c700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:23:22.670551, event: done, request: pg_notify(22.3e4(27) epoch 194478) v5 -575> 2014-08-13 17:23:22.670575 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.002180 8 0.000508 -574> 2014-08-13 17:23:22.670601 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 12 ==== pg_info(1 pgs e194478:24.181) v4 ==== 748+0+0 (1928527014 0 0) 0x5b4de80 con 0x4fde300 -573> 2014-08-13 17:23:22.670608 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive -572> 2014-08-13 17:23:22.670624 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670411, event: header_read, request: pg_info(1 pgs e194478:24.181) v4 -571> 2014-08-13 17:23:22.670632 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670414, event: throttled, request: pg_info(1 pgs e194478:24.181) v4 -570> 2014-08-13 17:23:22.670638 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670485, event: all_read, request: pg_info(1 pgs e194478:24.181) v4 -569> 2014-08-13 17:23:22.670629 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194449 n=59 ec=192688 les/c 194475/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering -568> 2014-08-13 17:23:22.670643 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670621, event: dispatched, request: pg_info(1 pgs e194478:24.181) v4 -567> 2014-08-13 17:23:22.670650 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670650, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.181) v4 -566> 2014-08-13 17:23:22.670658 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670657, event: started, request: pg_info(1 pgs e194478:24.181) v4 -565> 2014-08-13 17:23:22.670677 7fcd0964c700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:23:22.670677, event: done, request: pg_info(1 pgs e194478:24.181) v4 -564> 2014-08-13 17:23:22.670697 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 13 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x550d7c0 con 0x4e35540 -563> 2014-08-13 17:23:22.670714 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -562> 2014-08-13 17:23:22.670723 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.318 192.168.1.110:6814/43857 7 ==== pg_info(1 pgs e194478:22.3e4) v4 ==== 714+0+0 (1353716569 0 0) 0x5b4ec80 con 0x55f2100 -561> 2014-08-13 17:23:22.670736 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670584, event: header_read, request: pg_info(1 pgs e194478:22.3e4) v4 -560> 2014-08-13 17:23:22.670743 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670586, event: throttled, request: pg_info(1 pgs e194478:22.3e4) v4 -559> 2014-08-13 17:23:22.670748 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670652, event: all_read, request: pg_info(1 pgs e194478:22.3e4) v4 -558> 2014-08-13 17:23:22.670754 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670734, event: dispatched, request: pg_info(1 pgs e194478:22.3e4) v4 -557> 2014-08-13 17:23:22.670760 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670759, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3e4) v4 -556> 2014-08-13 17:23:22.670767 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670767, event: started, request: pg_info(1 pgs e194478:22.3e4) v4 -555> 2014-08-13 17:23:22.670786 7fcd0964c700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:23:22.670786, event: done, request: pg_info(1 pgs e194478:22.3e4) v4 -554> 2014-08-13 17:23:22.670800 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 16 ==== pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 ==== 4311+0+0 (3248766514 0 0) 0x56a9340 con 0x4fd32e0 -553> 2014-08-13 17:23:22.670815 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670301, event: header_read, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -552> 2014-08-13 17:23:22.670822 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670302, event: throttled, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -551> 2014-08-13 17:23:22.670828 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670728, event: all_read, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -550> 2014-08-13 17:23:22.670834 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670813, event: dispatched, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -549> 2014-08-13 17:23:22.670841 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670841, event: waiting_for_osdmap, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -548> 2014-08-13 17:23:22.670849 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670848, event: started, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -547> 2014-08-13 17:23:22.670881 7fcd0964c700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:23:22.670881, event: done, request: pg_notify(24.3d(26),22.39d(25) epoch 194478) v5 -546> 2014-08-13 17:23:22.670910 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 14 ==== pg_notify(24.34(50) epoch 194478) v5 ==== 3306+0+0 (2036953339 0 0) 0x588d780 con 0x4e35540 -545> 2014-08-13 17:23:22.670926 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670720, event: header_read, request: pg_notify(24.34(50) epoch 194478) v5 -544> 2014-08-13 17:23:22.670933 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670722, event: throttled, request: pg_notify(24.34(50) epoch 194478) v5 -543> 2014-08-13 17:23:22.670939 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670870, event: all_read, request: pg_notify(24.34(50) epoch 194478) v5 -542> 2014-08-13 17:23:22.670944 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670924, event: dispatched, request: pg_notify(24.34(50) epoch 194478) v5 -541> 2014-08-13 17:23:22.670950 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670950, event: waiting_for_osdmap, request: pg_notify(24.34(50) epoch 194478) v5 -540> 2014-08-13 17:23:22.670958 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670957, event: started, request: pg_notify(24.34(50) epoch 194478) v5 -539> 2014-08-13 17:23:22.670961 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194478:24.3b7) v4 -- ?+0 0x54d6e40 -538> 2014-08-13 17:23:22.670978 7fcd0964c700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:23:22.670978, event: done, request: pg_notify(24.34(50) epoch 194478) v5 -537> 2014-08-13 17:23:22.671023 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52be300 con 0x4ec70c0 -536> 2014-08-13 17:23:22.671044 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -535> 2014-08-13 17:23:22.671053 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 8 ==== pg_notify(22.39d(25) epoch 194478) v5 ==== 2117+0+0 (2440020879 0 0) 0x5b4db00 con 0x4ec70c0 -534> 2014-08-13 17:23:22.671067 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.670911, event: header_read, request: pg_notify(22.39d(25) epoch 194478) v5 -533> 2014-08-13 17:23:22.671075 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.670912, event: throttled, request: pg_notify(22.39d(25) epoch 194478) v5 -532> 2014-08-13 17:23:22.671081 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.671030, event: all_read, request: pg_notify(22.39d(25) epoch 194478) v5 -531> 2014-08-13 17:23:22.671086 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.671065, event: dispatched, request: pg_notify(22.39d(25) epoch 194478) v5 -530> 2014-08-13 17:23:22.671092 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.671092, event: waiting_for_osdmap, request: pg_notify(22.39d(25) epoch 194478) v5 -529> 2014-08-13 17:23:22.671100 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.671100, event: started, request: pg_notify(22.39d(25) epoch 194478) v5 -528> 2014-08-13 17:23:22.671126 7fcd0964c700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:23:22.671126, event: done, request: pg_notify(22.39d(25) epoch 194478) v5 -527> 2014-08-13 17:23:22.671338 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b8b40 con 0x4eb48e0 -526> 2014-08-13 17:23:22.671375 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -525> 2014-08-13 17:23:22.671391 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.374 192.168.1.102:6828/16129 14 ==== pg_info(1 pgs e194478:22.99) v4 ==== 714+0+0 (2897583940 0 0) 0x5b49500 con 0x4eb18c0 -524> 2014-08-13 17:23:22.671410 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671240, event: header_read, request: pg_info(1 pgs e194478:22.99) v4 -523> 2014-08-13 17:23:22.671423 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671243, event: throttled, request: pg_info(1 pgs e194478:22.99) v4 -522> 2014-08-13 17:23:22.671432 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671313, event: all_read, request: pg_info(1 pgs e194478:22.99) v4 -521> 2014-08-13 17:23:22.671441 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671406, event: dispatched, request: pg_info(1 pgs e194478:22.99) v4 -520> 2014-08-13 17:23:22.671448 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671448, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.99) v4 -519> 2014-08-13 17:23:22.671457 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671456, event: started, request: pg_info(1 pgs e194478:22.99) v4 -518> 2014-08-13 17:23:22.671481 7fcd0964c700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:23:22.671481, event: done, request: pg_info(1 pgs e194478:22.99) v4 -517> 2014-08-13 17:23:22.671498 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 6 ==== pg_notify(22.775(27) epoch 194478) v5 ==== 2231+0+0 (3464640874 0 0) 0x588b800 con 0x4eb48e0 -516> 2014-08-13 17:23:22.671513 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671245, event: header_read, request: pg_notify(22.775(27) epoch 194478) v5 -515> 2014-08-13 17:23:22.671520 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671246, event: throttled, request: pg_notify(22.775(27) epoch 194478) v5 -514> 2014-08-13 17:23:22.671551 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671350, event: all_read, request: pg_notify(22.775(27) epoch 194478) v5 -513> 2014-08-13 17:23:22.671558 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671511, event: dispatched, request: pg_notify(22.775(27) epoch 194478) v5 -512> 2014-08-13 17:23:22.671564 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671564, event: waiting_for_osdmap, request: pg_notify(22.775(27) epoch 194478) v5 -511> 2014-08-13 17:23:22.671572 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671572, event: started, request: pg_notify(22.775(27) epoch 194478) v5 -510> 2014-08-13 17:23:22.671594 7fcd0964c700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:23:22.671594, event: done, request: pg_notify(22.775(27) epoch 194478) v5 -509> 2014-08-13 17:23:22.671629 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 17 ==== pg_notify(22.3d9(39) epoch 194478) v5 ==== 2851+0+0 (1647953117 0 0) 0x5b4f540 con 0x4fd32e0 -508> 2014-08-13 17:23:22.671654 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671209, event: header_read, request: pg_notify(22.3d9(39) epoch 194478) v5 -507> 2014-08-13 17:23:22.671664 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671211, event: throttled, request: pg_notify(22.3d9(39) epoch 194478) v5 -506> 2014-08-13 17:23:22.671674 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671350, event: all_read, request: pg_notify(22.3d9(39) epoch 194478) v5 -505> 2014-08-13 17:23:22.671683 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671651, event: dispatched, request: pg_notify(22.3d9(39) epoch 194478) v5 -504> 2014-08-13 17:23:22.671692 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671692, event: waiting_for_osdmap, request: pg_notify(22.3d9(39) epoch 194478) v5 -503> 2014-08-13 17:23:22.671703 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671703, event: started, request: pg_notify(22.3d9(39) epoch 194478) v5 -502> 2014-08-13 17:23:22.671727 7fcd0964c700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:23:22.671727, event: done, request: pg_notify(22.3d9(39) epoch 194478) v5 -501> 2014-08-13 17:23:22.671750 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 15 ==== pg_info(1 pgs e194478:24.b6) v4 ==== 748+0+0 (3629619006 0 0) 0x5b4dcc0 con 0x5305ee0 -500> 2014-08-13 17:23:22.671769 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671489, event: header_read, request: pg_info(1 pgs e194478:24.b6) v4 -499> 2014-08-13 17:23:22.671779 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671492, event: throttled, request: pg_info(1 pgs e194478:24.b6) v4 -498> 2014-08-13 17:23:22.671788 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671556, event: all_read, request: pg_info(1 pgs e194478:24.b6) v4 -497> 2014-08-13 17:23:22.671797 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671767, event: dispatched, request: pg_info(1 pgs e194478:24.b6) v4 -496> 2014-08-13 17:23:22.671806 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671806, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.b6) v4 -495> 2014-08-13 17:23:22.671817 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671817, event: started, request: pg_info(1 pgs e194478:24.b6) v4 -494> 2014-08-13 17:23:22.671842 7fcd0964c700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:23:22.671842, event: done, request: pg_info(1 pgs e194478:24.b6) v4 -493> 2014-08-13 17:23:22.671860 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 9 ==== pg_notify(24.181(28) epoch 194478) v5 ==== 2316+0+0 (3298561275 0 0) 0x588c600 con 0x4ec70c0 -492> 2014-08-13 17:23:22.671879 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671404, event: header_read, request: pg_notify(24.181(28) epoch 194478) v5 -491> 2014-08-13 17:23:22.671889 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671406, event: throttled, request: pg_notify(24.181(28) epoch 194478) v5 -490> 2014-08-13 17:23:22.671901 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671513, event: all_read, request: pg_notify(24.181(28) epoch 194478) v5 -489> 2014-08-13 17:23:22.671907 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671877, event: dispatched, request: pg_notify(24.181(28) epoch 194478) v5 -488> 2014-08-13 17:23:22.671913 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671913, event: waiting_for_osdmap, request: pg_notify(24.181(28) epoch 194478) v5 -487> 2014-08-13 17:23:22.671921 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671920, event: started, request: pg_notify(24.181(28) epoch 194478) v5 -486> 2014-08-13 17:23:22.671860 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.032136 7 0.000280 -485> 2014-08-13 17:23:22.671941 7fcd0964c700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:23:22.671941, event: done, request: pg_notify(24.181(28) epoch 194478) v5 -484> 2014-08-13 17:23:22.671940 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -483> 2014-08-13 17:23:22.671959 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 11 ==== pg_notify(24.136(26) epoch 194478) v5 ==== 2202+0+0 (942611436 0 0) 0x5b4e580 con 0x4fd9760 -482> 2014-08-13 17:23:22.671964 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194471 n=36 ec=183096 les/c 194475/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -481> 2014-08-13 17:23:22.671974 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.671488, event: header_read, request: pg_notify(24.136(26) epoch 194478) v5 -480> 2014-08-13 17:23:22.671981 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.671490, event: throttled, request: pg_notify(24.136(26) epoch 194478) v5 -479> 2014-08-13 17:23:22.671987 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.671594, event: all_read, request: pg_notify(24.136(26) epoch 194478) v5 -478> 2014-08-13 17:23:22.671992 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.671972, event: dispatched, request: pg_notify(24.136(26) epoch 194478) v5 -477> 2014-08-13 17:23:22.671998 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.671998, event: waiting_for_osdmap, request: pg_notify(24.136(26) epoch 194478) v5 -476> 2014-08-13 17:23:22.672006 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.672006, event: started, request: pg_notify(24.136(26) epoch 194478) v5 -475> 2014-08-13 17:23:22.672025 7fcd0964c700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:23:22.672025, event: done, request: pg_notify(24.136(26) epoch 194478) v5 -474> 2014-08-13 17:23:22.672056 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.373 192.168.1.108:6840/7435 18 ==== pg_info(1 pgs e194478:22.39d) v4 ==== 714+0+0 (1742187011 0 0) 0x5b4f380 con 0x4fd32e0 -473> 2014-08-13 17:23:22.672070 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.671706, event: header_read, request: pg_info(1 pgs e194478:22.39d) v4 -472> 2014-08-13 17:23:22.672077 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.671708, event: throttled, request: pg_info(1 pgs e194478:22.39d) v4 -471> 2014-08-13 17:23:22.672082 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.671774, event: all_read, request: pg_info(1 pgs e194478:22.39d) v4 -470> 2014-08-13 17:23:22.672088 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.672068, event: dispatched, request: pg_info(1 pgs e194478:22.39d) v4 -469> 2014-08-13 17:23:22.672094 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.672094, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.39d) v4 -468> 2014-08-13 17:23:22.672101 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.672101, event: started, request: pg_info(1 pgs e194478:22.39d) v4 -467> 2014-08-13 17:23:22.672121 7fcd0964c700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:23:22.672121, event: done, request: pg_info(1 pgs e194478:22.39d) v4 -466> 2014-08-13 17:23:22.672110 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.022259 7 0.000270 -465> 2014-08-13 17:23:22.672136 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -464> 2014-08-13 17:23:22.672164 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194471 n=40 ec=183096 les/c 194475/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -463> 2014-08-13 17:23:22.672453 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.340 192.168.1.104:6806/62463 15 ==== pg_info(1 pgs e194478:24.34) v4 ==== 740+0+0 (555506992 0 0) 0x56a8c40 con 0x4e35540 -462> 2014-08-13 17:23:22.672486 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672354, event: header_read, request: pg_info(1 pgs e194478:24.34) v4 -461> 2014-08-13 17:23:22.672500 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672356, event: throttled, request: pg_info(1 pgs e194478:24.34) v4 -460> 2014-08-13 17:23:22.672509 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672430, event: all_read, request: pg_info(1 pgs e194478:24.34) v4 -459> 2014-08-13 17:23:22.672518 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672482, event: dispatched, request: pg_info(1 pgs e194478:24.34) v4 -458> 2014-08-13 17:23:22.672529 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672529, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.34) v4 -457> 2014-08-13 17:23:22.672543 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672543, event: started, request: pg_info(1 pgs e194478:24.34) v4 -456> 2014-08-13 17:23:22.672526 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.003032 7 0.000408 -455> 2014-08-13 17:23:22.672557 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -454> 2014-08-13 17:23:22.672571 7fcd0964c700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:23:22.672571, event: done, request: pg_info(1 pgs e194478:24.34) v4 -453> 2014-08-13 17:23:22.672556 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194478:24.1e4) v4 -- ?+0 0x5888fc0 -452> 2014-08-13 17:23:22.672578 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194471 n=33 ec=183096 les/c 194475/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -451> 2014-08-13 17:23:22.672798 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.351 192.168.1.102:6830/16444 -- pg_info(1 pgs e194478:22.220) v4 -- ?+0 0x588a680 -450> 2014-08-13 17:23:22.672966 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.320 192.168.1.106:6820/10050 -- pg_info(1 pgs e194478:22.581) v4 -- ?+0 0x5b4f8c0 -449> 2014-08-13 17:23:22.673061 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.313 192.168.1.109:6820/35445 -- pg_info(1 pgs e194478:22.3e4) v4 -- ?+0 0x588e040 -448> 2014-08-13 17:23:22.673108 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.318 192.168.1.110:6814/43857 -- pg_info(1 pgs e194478:22.3e4) v4 -- ?+0 0x588ee40 -447> 2014-08-13 17:23:22.673256 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194478:22.226) v4 -- ?+0 0x5b4f700 -446> 2014-08-13 17:23:22.673327 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194478:22.99) v4 -- ?+0 0x588dcc0 -445> 2014-08-13 17:23:22.673375 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194478:22.99) v4 -- ?+0 0x588b480 -444> 2014-08-13 17:23:22.673476 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.15 192.168.1.107:6812/53408 -- pg_info(1 pgs e194478:24.20e) v4 -- ?+0 0x5b49500 -443> 2014-08-13 17:23:22.673588 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 1.023140 8 0.000561 -442> 2014-08-13 17:23:22.673633 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive -441> 2014-08-13 17:23:22.673661 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194472 n=66 ec=192688 les/c 194475/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering -440> 2014-08-13 17:23:22.673927 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.55 192.168.1.112:6834/6966 14 ==== pg_info(1 pgs e194478:24.136) v4 ==== 748+0+0 (3747609268 0 0) 0x5b4af40 con 0x4fd9a20 -439> 2014-08-13 17:23:22.673951 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.673808, event: header_read, request: pg_info(1 pgs e194478:24.136) v4 -438> 2014-08-13 17:23:22.673963 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.673810, event: throttled, request: pg_info(1 pgs e194478:24.136) v4 -437> 2014-08-13 17:23:22.673972 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.673904, event: all_read, request: pg_info(1 pgs e194478:24.136) v4 -436> 2014-08-13 17:23:22.673981 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.673946, event: dispatched, request: pg_info(1 pgs e194478:24.136) v4 -435> 2014-08-13 17:23:22.673975 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194478/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.014596 10 0.000727 -434> 2014-08-13 17:23:22.673999 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.673999, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.136) v4 -433> 2014-08-13 17:23:22.674024 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.674024, event: started, request: pg_info(1 pgs e194478:24.136) v4 -432> 2014-08-13 17:23:22.674018 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194478/194466 194476/194477/194477) [4,313,318] r=0 lpr=194477 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -431> 2014-08-13 17:23:22.674056 7fcd0964c700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:23:22.674055, event: done, request: pg_info(1 pgs e194478:24.136) v4 -430> 2014-08-13 17:23:22.674051 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194478/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.020469 10 0.000705 -429> 2014-08-13 17:23:22.674072 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194478/194465 194476/194477/194477) [4,374,31] r=0 lpr=194477 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -428> 2014-08-13 17:23:22.674081 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 17 ==== pg_info(1 pgs e194478:22.226) v4 ==== 714+0+0 (3214876449 0 0) 0x5b4f700 con 0x4fd5960 -427> 2014-08-13 17:23:22.674097 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.673896, event: header_read, request: pg_info(1 pgs e194478:22.226) v4 -426> 2014-08-13 17:23:22.674108 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.673898, event: throttled, request: pg_info(1 pgs e194478:22.226) v4 -425> 2014-08-13 17:23:22.674117 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.673962, event: all_read, request: pg_info(1 pgs e194478:22.226) v4 -424> 2014-08-13 17:23:22.674126 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.674095, event: dispatched, request: pg_info(1 pgs e194478:22.226) v4 -423> 2014-08-13 17:23:22.674135 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.674135, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.226) v4 -422> 2014-08-13 17:23:22.674146 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.674146, event: started, request: pg_info(1 pgs e194478:22.226) v4 -421> 2014-08-13 17:23:22.674172 7fcd0964c700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:23:22.674172, event: done, request: pg_info(1 pgs e194478:22.226) v4 -420> 2014-08-13 17:23:22.674534 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.376 192.168.1.108:6838/6893 18 ==== MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 ==== 26+0+0 (2091757571 0 0) 0x588b100 con 0x4fd5960 -419> 2014-08-13 17:23:22.674563 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674450, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -418> 2014-08-13 17:23:22.674575 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674452, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -417> 2014-08-13 17:23:22.674584 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674511, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -416> 2014-08-13 17:23:22.674593 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674559, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -415> 2014-08-13 17:23:22.674603 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674602, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -414> 2014-08-13 17:23:22.674630 7fcd0964c700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:23:22.674630, event: done, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194478 v2 -413> 2014-08-13 17:23:22.674644 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.002065 3 0.000089 -412> 2014-08-13 17:23:22.674680 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194477 pi=192975-194476/26 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -411> 2014-08-13 17:23:22.675599 7fcd0e685700 1 -- 192.168.1.115:6819/31924 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194478:24.3f7) v4 -- ?+0 0x5b4db00 -410> 2014-08-13 17:23:22.676132 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e0480 con 0x4eb0c60 -409> 2014-08-13 17:23:22.676159 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -408> 2014-08-13 17:23:22.676259 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 10 ==== pg_notify(22.619(5) epoch 194478) v5 ==== 1001+0+0 (2512877707 0 0) 0x58881c0 con 0x4eb0c60 -407> 2014-08-13 17:23:22.676289 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676137, event: header_read, request: pg_notify(22.619(5) epoch 194478) v5 -406> 2014-08-13 17:23:22.676301 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676144, event: throttled, request: pg_notify(22.619(5) epoch 194478) v5 -405> 2014-08-13 17:23:22.676309 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676221, event: all_read, request: pg_notify(22.619(5) epoch 194478) v5 -404> 2014-08-13 17:23:22.676318 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676285, event: dispatched, request: pg_notify(22.619(5) epoch 194478) v5 -403> 2014-08-13 17:23:22.676328 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676328, event: waiting_for_osdmap, request: pg_notify(22.619(5) epoch 194478) v5 -402> 2014-08-13 17:23:22.676340 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676340, event: started, request: pg_notify(22.619(5) epoch 194478) v5 -401> 2014-08-13 17:23:22.676371 7fcd0964c700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:23:22.676371, event: done, request: pg_notify(22.619(5) epoch 194478) v5 -400> 2014-08-13 17:23:22.677090 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e0d80 con 0x5303440 -399> 2014-08-13 17:23:22.677121 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -398> 2014-08-13 17:23:22.677329 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 8 ==== pg_notify(22.707(27) epoch 194478) v5 ==== 2231+0+0 (3342977228 0 0) 0x5b4e200 con 0x5303440 -397> 2014-08-13 17:23:22.677374 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677119, event: header_read, request: pg_notify(22.707(27) epoch 194478) v5 -396> 2014-08-13 17:23:22.677387 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677122, event: throttled, request: pg_notify(22.707(27) epoch 194478) v5 -395> 2014-08-13 17:23:22.677393 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677265, event: all_read, request: pg_notify(22.707(27) epoch 194478) v5 -394> 2014-08-13 17:23:22.677399 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677368, event: dispatched, request: pg_notify(22.707(27) epoch 194478) v5 -393> 2014-08-13 17:23:22.677406 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677406, event: waiting_for_osdmap, request: pg_notify(22.707(27) epoch 194478) v5 -392> 2014-08-13 17:23:22.677416 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677416, event: started, request: pg_notify(22.707(27) epoch 194478) v5 -391> 2014-08-13 17:23:22.677459 7fcd0964c700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:23:22.677459, event: done, request: pg_notify(22.707(27) epoch 194478) v5 -390> 2014-08-13 17:23:22.677759 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 11 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x2e8aac0 con 0x4f86040 -389> 2014-08-13 17:23:22.677790 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -388> 2014-08-13 17:23:22.677980 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e0b40 con 0x4f87900 -387> 2014-08-13 17:23:22.678002 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -386> 2014-08-13 17:23:22.678050 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 12 ==== pg_notify(22.362(26) epoch 194478) v5 ==== 2172+0+0 (797605970 0 0) 0x588ec80 con 0x4f86040 -385> 2014-08-13 17:23:22.678070 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.677875, event: header_read, request: pg_notify(22.362(26) epoch 194478) v5 -384> 2014-08-13 17:23:22.678081 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.677879, event: throttled, request: pg_notify(22.362(26) epoch 194478) v5 -383> 2014-08-13 17:23:22.678090 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.678016, event: all_read, request: pg_notify(22.362(26) epoch 194478) v5 -382> 2014-08-13 17:23:22.678099 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.678067, event: dispatched, request: pg_notify(22.362(26) epoch 194478) v5 -381> 2014-08-13 17:23:22.678109 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.678109, event: waiting_for_osdmap, request: pg_notify(22.362(26) epoch 194478) v5 -380> 2014-08-13 17:23:22.678127 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.678127, event: started, request: pg_notify(22.362(26) epoch 194478) v5 -379> 2014-08-13 17:23:22.678162 7fcd0964c700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:23:22.678162, event: done, request: pg_notify(22.362(26) epoch 194478) v5 -378> 2014-08-13 17:23:22.678217 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.341 192.168.1.106:6818/9142 6 ==== pg_notify(22.48e(25) epoch 194478) v5 ==== 2117+0+0 (3823084816 0 0) 0x5b49340 con 0x4f87900 -377> 2014-08-13 17:23:22.678239 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.677996, event: header_read, request: pg_notify(22.48e(25) epoch 194478) v5 -376> 2014-08-13 17:23:22.678250 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.677997, event: throttled, request: pg_notify(22.48e(25) epoch 194478) v5 -375> 2014-08-13 17:23:22.678259 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.678107, event: all_read, request: pg_notify(22.48e(25) epoch 194478) v5 -374> 2014-08-13 17:23:22.678268 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.678237, event: dispatched, request: pg_notify(22.48e(25) epoch 194478) v5 -373> 2014-08-13 17:23:22.678278 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.678277, event: waiting_for_osdmap, request: pg_notify(22.48e(25) epoch 194478) v5 -372> 2014-08-13 17:23:22.678289 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.678288, event: started, request: pg_notify(22.48e(25) epoch 194478) v5 -371> 2014-08-13 17:23:22.678323 7fcd0964c700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:23:22.678323, event: done, request: pg_notify(22.48e(25) epoch 194478) v5 -370> 2014-08-13 17:23:22.678383 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 11 ==== pg_notify(22.3af(27) epoch 194478) v5 ==== 2231+0+0 (1706368517 0 0) 0x5b48000 con 0x4eb0c60 -369> 2014-08-13 17:23:22.678407 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678213, event: header_read, request: pg_notify(22.3af(27) epoch 194478) v5 -368> 2014-08-13 17:23:22.678418 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678215, event: throttled, request: pg_notify(22.3af(27) epoch 194478) v5 -367> 2014-08-13 17:23:22.678424 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678357, event: all_read, request: pg_notify(22.3af(27) epoch 194478) v5 -366> 2014-08-13 17:23:22.678433 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678404, event: dispatched, request: pg_notify(22.3af(27) epoch 194478) v5 -365> 2014-08-13 17:23:22.678442 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678442, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194478) v5 -364> 2014-08-13 17:23:22.678453 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678453, event: started, request: pg_notify(22.3af(27) epoch 194478) v5 -363> 2014-08-13 17:23:22.678478 7fcd0964c700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:23:22.678478, event: done, request: pg_notify(22.3af(27) epoch 194478) v5 -362> 2014-08-13 17:23:22.678502 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 7 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x5671200 con 0x4eb2680 -361> 2014-08-13 17:23:22.678524 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -360> 2014-08-13 17:23:22.678701 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.5 192.168.1.103:6827/40192 8 ==== pg_notify(22.1f1(27) epoch 194478) v5 ==== 2231+0+0 (3784711711 0 0) 0x5b4b9c0 con 0x4eb2680 -359> 2014-08-13 17:23:22.678721 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678567, event: header_read, request: pg_notify(22.1f1(27) epoch 194478) v5 -358> 2014-08-13 17:23:22.678729 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678568, event: throttled, request: pg_notify(22.1f1(27) epoch 194478) v5 -357> 2014-08-13 17:23:22.678735 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678681, event: all_read, request: pg_notify(22.1f1(27) epoch 194478) v5 -356> 2014-08-13 17:23:22.678744 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678718, event: dispatched, request: pg_notify(22.1f1(27) epoch 194478) v5 -355> 2014-08-13 17:23:22.678750 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678750, event: waiting_for_osdmap, request: pg_notify(22.1f1(27) epoch 194478) v5 -354> 2014-08-13 17:23:22.678758 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678758, event: started, request: pg_notify(22.1f1(27) epoch 194478) v5 -353> 2014-08-13 17:23:22.678782 7fcd0964c700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:23:22.678782, event: done, request: pg_notify(22.1f1(27) epoch 194478) v5 -352> 2014-08-13 17:23:22.678948 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 8 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e5e80 con 0x4fd53e0 -351> 2014-08-13 17:23:22.678971 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -350> 2014-08-13 17:23:22.678996 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b9200 con 0x5305c20 -349> 2014-08-13 17:23:22.679018 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -348> 2014-08-13 17:23:22.679094 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.397 192.168.1.105:6808/41085 9 ==== pg_notify(22.3d9(13) epoch 194478) v5 ==== 1453+0+0 (2675562068 0 0) 0x54d4440 con 0x4fd53e0 -347> 2014-08-13 17:23:22.679112 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.678980, event: header_read, request: pg_notify(22.3d9(13) epoch 194478) v5 -346> 2014-08-13 17:23:22.679119 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.678982, event: throttled, request: pg_notify(22.3d9(13) epoch 194478) v5 -345> 2014-08-13 17:23:22.679125 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.679075, event: all_read, request: pg_notify(22.3d9(13) epoch 194478) v5 -344> 2014-08-13 17:23:22.679130 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.679109, event: dispatched, request: pg_notify(22.3d9(13) epoch 194478) v5 -343> 2014-08-13 17:23:22.679136 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.679136, event: waiting_for_osdmap, request: pg_notify(22.3d9(13) epoch 194478) v5 -342> 2014-08-13 17:23:22.679144 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.679144, event: started, request: pg_notify(22.3d9(13) epoch 194478) v5 -341> 2014-08-13 17:23:22.679169 7fcd0964c700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:23:22.679169, event: done, request: pg_notify(22.3d9(13) epoch 194478) v5 -340> 2014-08-13 17:23:22.679216 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.339 192.168.1.110:6818/45268 10 ==== pg_notify(22.3af(26) epoch 194478) v5 ==== 2172+0+0 (2587323809 0 0) 0x56a9f80 con 0x5305c20 -339> 2014-08-13 17:23:22.679238 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679001, event: header_read, request: pg_notify(22.3af(26) epoch 194478) v5 -338> 2014-08-13 17:23:22.679249 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679003, event: throttled, request: pg_notify(22.3af(26) epoch 194478) v5 -337> 2014-08-13 17:23:22.679258 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679144, event: all_read, request: pg_notify(22.3af(26) epoch 194478) v5 -336> 2014-08-13 17:23:22.679267 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679235, event: dispatched, request: pg_notify(22.3af(26) epoch 194478) v5 -335> 2014-08-13 17:23:22.679276 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679276, event: waiting_for_osdmap, request: pg_notify(22.3af(26) epoch 194478) v5 -334> 2014-08-13 17:23:22.679287 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679287, event: started, request: pg_notify(22.3af(26) epoch 194478) v5 -333> 2014-08-13 17:23:22.679314 7fcd0964c700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:23:22.679314, event: done, request: pg_notify(22.3af(26) epoch 194478) v5 -332> 2014-08-13 17:23:22.679342 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 10 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e7740 con 0x4ec5ac0 -331> 2014-08-13 17:23:22.679366 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -330> 2014-08-13 17:23:22.679382 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 11 ==== pg_info(1 pgs e194478:22.653) v4 ==== 714+0+0 (907433385 0 0) 0x54d1180 con 0x4fd86e0 -329> 2014-08-13 17:23:22.679400 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679239, event: header_read, request: pg_info(1 pgs e194478:22.653) v4 -328> 2014-08-13 17:23:22.679411 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679243, event: throttled, request: pg_info(1 pgs e194478:22.653) v4 -327> 2014-08-13 17:23:22.679420 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679334, event: all_read, request: pg_info(1 pgs e194478:22.653) v4 -326> 2014-08-13 17:23:22.679428 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679398, event: dispatched, request: pg_info(1 pgs e194478:22.653) v4 -325> 2014-08-13 17:23:22.679438 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679438, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.653) v4 -324> 2014-08-13 17:23:22.679450 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679449, event: started, request: pg_info(1 pgs e194478:22.653) v4 -323> 2014-08-13 17:23:22.679479 7fcd0964c700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:23:22.679479, event: done, request: pg_info(1 pgs e194478:22.653) v4 -322> 2014-08-13 17:23:22.679498 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.17 192.168.1.103:6811/37658 11 ==== pg_notify(24.b6(32) epoch 194478) v5 ==== 2556+0+0 (3873325259 0 0) 0x5b4b800 con 0x4ec5ac0 -321> 2014-08-13 17:23:22.679516 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679244, event: header_read, request: pg_notify(24.b6(32) epoch 194478) v5 -320> 2014-08-13 17:23:22.679526 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679245, event: throttled, request: pg_notify(24.b6(32) epoch 194478) v5 -319> 2014-08-13 17:23:22.679536 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679374, event: all_read, request: pg_notify(24.b6(32) epoch 194478) v5 -318> 2014-08-13 17:23:22.679544 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679514, event: dispatched, request: pg_notify(24.b6(32) epoch 194478) v5 -317> 2014-08-13 17:23:22.679554 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679554, event: waiting_for_osdmap, request: pg_notify(24.b6(32) epoch 194478) v5 -316> 2014-08-13 17:23:22.679565 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679565, event: started, request: pg_notify(24.b6(32) epoch 194478) v5 -315> 2014-08-13 17:23:22.679593 7fcd0964c700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:23:22.679593, event: done, request: pg_notify(24.b6(32) epoch 194478) v5 -314> 2014-08-13 17:23:22.679612 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.59 192.168.1.112:6822/5265 3 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x51e5c40 con 0x53e65c0 -313> 2014-08-13 17:23:22.679629 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -312> 2014-08-13 17:23:22.679657 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.59 192.168.1.112:6822/5265 4 ==== pg_notify(24.1f7(29) epoch 194478) v5 ==== 2379+0+0 (2652506049 0 0) 0x5b4ad80 con 0x53e65c0 -311> 2014-08-13 17:23:22.679680 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679523, event: header_read, request: pg_notify(24.1f7(29) epoch 194478) v5 -310> 2014-08-13 17:23:22.679691 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679524, event: throttled, request: pg_notify(24.1f7(29) epoch 194478) v5 -309> 2014-08-13 17:23:22.679700 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679636, event: all_read, request: pg_notify(24.1f7(29) epoch 194478) v5 -308> 2014-08-13 17:23:22.679708 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679677, event: dispatched, request: pg_notify(24.1f7(29) epoch 194478) v5 -307> 2014-08-13 17:23:22.679715 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679715, event: waiting_for_osdmap, request: pg_notify(24.1f7(29) epoch 194478) v5 -306> 2014-08-13 17:23:22.679723 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679722, event: started, request: pg_notify(24.1f7(29) epoch 194478) v5 -305> 2014-08-13 17:23:22.679749 7fcd0964c700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:23:22.679749, event: done, request: pg_notify(24.1f7(29) epoch 194478) v5 -304> 2014-08-13 17:23:22.679882 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.371 192.168.1.102:6832/16753 12 ==== MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 ==== 26+0+0 (278605955 0 0) 0x5b4bb80 con 0x4fd86e0 -303> 2014-08-13 17:23:22.679918 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679785, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -302> 2014-08-13 17:23:22.679928 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679788, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -301> 2014-08-13 17:23:22.679935 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679842, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -300> 2014-08-13 17:23:22.679940 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679913, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -299> 2014-08-13 17:23:22.679947 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679947, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -298> 2014-08-13 17:23:22.679973 7fcd0964c700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:23:22.679973, event: done, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194478 v2 -297> 2014-08-13 17:23:22.679992 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 9 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x56733c0 con 0x4eb7d20 -296> 2014-08-13 17:23:22.679986 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.011009 3 0.000106 -295> 2014-08-13 17:23:22.680017 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -294> 2014-08-13 17:23:22.680047 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194477 pi=192977-194476/26 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -293> 2014-08-13 17:23:22.680053 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 10 ==== pg_notify(24.136(6) epoch 194478) v5 ==== 1094+0+0 (3891366325 0 0) 0x5889dc0 con 0x4eb7d20 -292> 2014-08-13 17:23:22.680074 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.679928, event: header_read, request: pg_notify(24.136(6) epoch 194478) v5 -291> 2014-08-13 17:23:22.680086 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.679930, event: throttled, request: pg_notify(24.136(6) epoch 194478) v5 -290> 2014-08-13 17:23:22.680095 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.680032, event: all_read, request: pg_notify(24.136(6) epoch 194478) v5 -289> 2014-08-13 17:23:22.680104 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.680071, event: dispatched, request: pg_notify(24.136(6) epoch 194478) v5 -288> 2014-08-13 17:23:22.680114 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.680114, event: waiting_for_osdmap, request: pg_notify(24.136(6) epoch 194478) v5 -287> 2014-08-13 17:23:22.680126 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.680126, event: started, request: pg_notify(24.136(6) epoch 194478) v5 -286> 2014-08-13 17:23:22.680159 7fcd0964c700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:23:22.680159, event: done, request: pg_notify(24.136(6) epoch 194478) v5 -285> 2014-08-13 17:23:22.680214 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 13 ==== pg_notify(22.3af(27) epoch 194478) v5 ==== 2231+0+0 (3986931783 0 0) 0x588cec0 con 0x4f86040 -284> 2014-08-13 17:23:22.680238 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680055, event: header_read, request: pg_notify(22.3af(27) epoch 194478) v5 -283> 2014-08-13 17:23:22.680248 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680057, event: throttled, request: pg_notify(22.3af(27) epoch 194478) v5 -282> 2014-08-13 17:23:22.680257 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680190, event: all_read, request: pg_notify(22.3af(27) epoch 194478) v5 -281> 2014-08-13 17:23:22.680266 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680235, event: dispatched, request: pg_notify(22.3af(27) epoch 194478) v5 -280> 2014-08-13 17:23:22.680276 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680276, event: waiting_for_osdmap, request: pg_notify(22.3af(27) epoch 194478) v5 -279> 2014-08-13 17:23:22.680287 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680287, event: started, request: pg_notify(22.3af(27) epoch 194478) v5 -278> 2014-08-13 17:23:22.680316 7fcd0964c700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:23:22.680316, event: done, request: pg_notify(22.3af(27) epoch 194478) v5 -277> 2014-08-13 17:23:22.680477 7fcd07849700 1 -- 192.168.100.115:6808/31924 <== osd.294 192.168.1.101:0/23402 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.680323) v2 ==== 47+0+0 (2598014630 0 0) 0x5889a40 con 0x5523440 -276> 2014-08-13 17:23:22.680519 7fcd07849700 1 -- 192.168.100.115:6808/31924 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194478 stamp 2014-08-13 17:23:22.680323) v2 -- ?+0 0x56aa4c0 con 0x5523440 -275> 2014-08-13 17:23:22.680551 7fcd07849700 1 -- 192.168.1.115:6819/31924 --> 192.168.1.101:6812/23402 -- osd_map(194478..194478 src has 185139..194478) v3 -- ?+0 0x5675e80 con 0x4eb40a0 -274> 2014-08-13 17:23:22.680580 7fcd06447700 1 -- 192.168.1.115:6820/31924 <== osd.294 192.168.1.101:0/23402 1 ==== osd_ping(ping e194477 stamp 2014-08-13 17:23:22.680323) v2 ==== 47+0+0 (2598014630 0 0) 0x588d940 con 0x5522c00 -273> 2014-08-13 17:23:22.680620 7fcd06447700 1 -- 192.168.1.115:6820/31924 --> 192.168.1.101:0/23402 -- osd_ping(ping_reply e194478 stamp 2014-08-13 17:23:22.680323) v2 -- ?+0 0x54d0000 con 0x5522c00 -272> 2014-08-13 17:23:22.680756 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.362 192.168.1.108:6842/8170 14 ==== pg_info(1 pgs e194478:22.3af) v4 ==== 714+0+0 (2046608248 0 0) 0x588f000 con 0x4f86040 -271> 2014-08-13 17:23:22.680788 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680569, event: header_read, request: pg_info(1 pgs e194478:22.3af) v4 -270> 2014-08-13 17:23:22.680800 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680571, event: throttled, request: pg_info(1 pgs e194478:22.3af) v4 -269> 2014-08-13 17:23:22.680809 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680660, event: all_read, request: pg_info(1 pgs e194478:22.3af) v4 -268> 2014-08-13 17:23:22.680820 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680784, event: dispatched, request: pg_info(1 pgs e194478:22.3af) v4 -267> 2014-08-13 17:23:22.680830 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680830, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3af) v4 -266> 2014-08-13 17:23:22.680842 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680842, event: started, request: pg_info(1 pgs e194478:22.3af) v4 -265> 2014-08-13 17:23:22.680875 7fcd0964c700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:23:22.680875, event: done, request: pg_info(1 pgs e194478:22.3af) v4 -264> 2014-08-13 17:23:22.680896 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.44 192.168.1.111:6822/10782 12 ==== pg_info(1 pgs e194478:22.3af) v4 ==== 714+0+0 (2109539738 0 0) 0x588b800 con 0x4eb0c60 -263> 2014-08-13 17:23:22.680915 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680668, event: header_read, request: pg_info(1 pgs e194478:22.3af) v4 -262> 2014-08-13 17:23:22.680925 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680670, event: throttled, request: pg_info(1 pgs e194478:22.3af) v4 -261> 2014-08-13 17:23:22.680934 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680745, event: all_read, request: pg_info(1 pgs e194478:22.3af) v4 -260> 2014-08-13 17:23:22.680942 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680913, event: dispatched, request: pg_info(1 pgs e194478:22.3af) v4 -259> 2014-08-13 17:23:22.680952 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680952, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3af) v4 -258> 2014-08-13 17:23:22.680963 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680963, event: started, request: pg_info(1 pgs e194478:22.3af) v4 -257> 2014-08-13 17:23:22.680989 7fcd0964c700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:23:22.680989, event: done, request: pg_info(1 pgs e194478:22.3af) v4 -256> 2014-08-13 17:23:22.681016 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.44 192.168.1.111:6822/10782 -- pg_info(1 pgs e194478:22.3af) v4 -- ?+0 0x5187700 -255> 2014-08-13 17:23:22.681057 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194478:22.3af) v4 -- ?+0 0x5187c40 -254> 2014-08-13 17:23:22.681384 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194478/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.020197 10 0.000703 -253> 2014-08-13 17:23:22.681405 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.59 192.168.1.112:6822/5265 5 ==== pg_info(1 pgs e194478:24.1f7) v4 ==== 748+0+0 (4184763088 0 0) 0x56aaf40 con 0x53e65c0 -252> 2014-08-13 17:23:22.681418 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194478/194466 194476/194477/194477) [4,44,362] r=0 lpr=194477 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -251> 2014-08-13 17:23:22.681428 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681314, event: header_read, request: pg_info(1 pgs e194478:24.1f7) v4 -250> 2014-08-13 17:23:22.681445 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681316, event: throttled, request: pg_info(1 pgs e194478:24.1f7) v4 -249> 2014-08-13 17:23:22.681455 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681385, event: all_read, request: pg_info(1 pgs e194478:24.1f7) v4 -248> 2014-08-13 17:23:22.681464 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681425, event: dispatched, request: pg_info(1 pgs e194478:24.1f7) v4 -247> 2014-08-13 17:23:22.681473 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681473, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1f7) v4 -246> 2014-08-13 17:23:22.681485 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681485, event: started, request: pg_info(1 pgs e194478:24.1f7) v4 -245> 2014-08-13 17:23:22.681514 7fcd0964c700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:23:22.681514, event: done, request: pg_info(1 pgs e194478:24.1f7) v4 -244> 2014-08-13 17:23:22.681791 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.50 192.168.1.112:6842/9607 11 ==== pg_info(1 pgs e194478:24.136) v4 ==== 748+0+0 (1723983963 0 0) 0x54d39c0 con 0x4eb7d20 -243> 2014-08-13 17:23:22.681813 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681686, event: header_read, request: pg_info(1 pgs e194478:24.136) v4 -242> 2014-08-13 17:23:22.681823 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681688, event: throttled, request: pg_info(1 pgs e194478:24.136) v4 -241> 2014-08-13 17:23:22.681832 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681771, event: all_read, request: pg_info(1 pgs e194478:24.136) v4 -240> 2014-08-13 17:23:22.681840 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681810, event: dispatched, request: pg_info(1 pgs e194478:24.136) v4 -239> 2014-08-13 17:23:22.681881 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681881, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.136) v4 -238> 2014-08-13 17:23:22.681893 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681893, event: started, request: pg_info(1 pgs e194478:24.136) v4 -237> 2014-08-13 17:23:22.681920 7fcd0964c700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:23:22.681920, event: done, request: pg_info(1 pgs e194478:24.136) v4 -236> 2014-08-13 17:23:22.682579 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.310 192.168.1.109:6812/32413 9 ==== pg_info(1 pgs e194478:22.707) v4 ==== 714+0+0 (2139653632 0 0) 0x56a8c40 con 0x5303440 -235> 2014-08-13 17:23:22.682608 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682464, event: header_read, request: pg_info(1 pgs e194478:22.707) v4 -234> 2014-08-13 17:23:22.682619 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682467, event: throttled, request: pg_info(1 pgs e194478:22.707) v4 -233> 2014-08-13 17:23:22.682628 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682542, event: all_read, request: pg_info(1 pgs e194478:22.707) v4 -232> 2014-08-13 17:23:22.682637 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682604, event: dispatched, request: pg_info(1 pgs e194478:22.707) v4 -231> 2014-08-13 17:23:22.682646 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682646, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.707) v4 -230> 2014-08-13 17:23:22.682657 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682657, event: started, request: pg_info(1 pgs e194478:22.707) v4 -229> 2014-08-13 17:23:22.682685 7fcd0964c700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:23:22.682685, event: done, request: pg_info(1 pgs e194478:22.707) v4 -228> 2014-08-13 17:23:22.682951 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.63 192.168.1.111:6830/12680 4 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567f2c0 con 0x5521e40 -227> 2014-08-13 17:23:22.682976 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -226> 2014-08-13 17:23:22.683089 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.63 192.168.1.111:6830/12680 5 ==== pg_notify(22.775(26) epoch 194478) v5 ==== 2172+0+0 (350435685 0 0) 0x588e900 con 0x5521e40 -225> 2014-08-13 17:23:22.683109 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.682965, event: header_read, request: pg_notify(22.775(26) epoch 194478) v5 -224> 2014-08-13 17:23:22.683120 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.682967, event: throttled, request: pg_notify(22.775(26) epoch 194478) v5 -223> 2014-08-13 17:23:22.683129 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.683069, event: all_read, request: pg_notify(22.775(26) epoch 194478) v5 -222> 2014-08-13 17:23:22.683138 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.683107, event: dispatched, request: pg_notify(22.775(26) epoch 194478) v5 -221> 2014-08-13 17:23:22.683148 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.683147, event: waiting_for_osdmap, request: pg_notify(22.775(26) epoch 194478) v5 -220> 2014-08-13 17:23:22.683160 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.683160, event: started, request: pg_notify(22.775(26) epoch 194478) v5 -219> 2014-08-13 17:23:22.683207 7fcd0964c700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:23:22.683207, event: done, request: pg_notify(22.775(26) epoch 194478) v5 -218> 2014-08-13 17:23:22.685610 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.293 192.168.1.109:6816/33855 4 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52b8480 con 0x5724e60 -217> 2014-08-13 17:23:22.685641 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -216> 2014-08-13 17:23:22.685693 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 13 ==== pg_info(1 pgs e194478:22.581) v4 ==== 714+0+0 (1394839257 0 0) 0x5b4f8c0 con 0x4fde300 -215> 2014-08-13 17:23:22.685715 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685569, event: header_read, request: pg_info(1 pgs e194478:22.581) v4 -214> 2014-08-13 17:23:22.685723 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685573, event: throttled, request: pg_info(1 pgs e194478:22.581) v4 -213> 2014-08-13 17:23:22.685733 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685667, event: all_read, request: pg_info(1 pgs e194478:22.581) v4 -212> 2014-08-13 17:23:22.685742 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685711, event: dispatched, request: pg_info(1 pgs e194478:22.581) v4 -211> 2014-08-13 17:23:22.685752 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685752, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.581) v4 -210> 2014-08-13 17:23:22.685763 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685763, event: started, request: pg_info(1 pgs e194478:22.581) v4 -209> 2014-08-13 17:23:22.685794 7fcd0964c700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:23:22.685794, event: done, request: pg_info(1 pgs e194478:22.581) v4 -208> 2014-08-13 17:23:22.685813 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.293 192.168.1.109:6816/33855 5 ==== pg_notify(22.3d9(46) epoch 194478) v5 ==== 3208+0+0 (1406689201 0 0) 0x54d4280 con 0x5724e60 -207> 2014-08-13 17:23:22.685832 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685579, event: header_read, request: pg_notify(22.3d9(46) epoch 194478) v5 -206> 2014-08-13 17:23:22.685842 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685580, event: throttled, request: pg_notify(22.3d9(46) epoch 194478) v5 -205> 2014-08-13 17:23:22.685851 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685712, event: all_read, request: pg_notify(22.3d9(46) epoch 194478) v5 -204> 2014-08-13 17:23:22.685860 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685830, event: dispatched, request: pg_notify(22.3d9(46) epoch 194478) v5 -203> 2014-08-13 17:23:22.685870 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685870, event: waiting_for_osdmap, request: pg_notify(22.3d9(46) epoch 194478) v5 -202> 2014-08-13 17:23:22.685881 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685881, event: started, request: pg_notify(22.3d9(46) epoch 194478) v5 -201> 2014-08-13 17:23:22.685908 7fcd0964c700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:23:22.685908, event: done, request: pg_notify(22.3d9(46) epoch 194478) v5 -200> 2014-08-13 17:23:22.686110 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.56 192.168.1.111:6826/11446 7 ==== pg_info(1 pgs e194478:22.775) v4 ==== 714+0+0 (874888681 0 0) 0x4e50380 con 0x4eb48e0 -199> 2014-08-13 17:23:22.686138 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686007, event: header_read, request: pg_info(1 pgs e194478:22.775) v4 -198> 2014-08-13 17:23:22.686150 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686009, event: throttled, request: pg_info(1 pgs e194478:22.775) v4 -197> 2014-08-13 17:23:22.686159 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686087, event: all_read, request: pg_info(1 pgs e194478:22.775) v4 -196> 2014-08-13 17:23:22.686168 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686134, event: dispatched, request: pg_info(1 pgs e194478:22.775) v4 -195> 2014-08-13 17:23:22.686178 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686178, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.775) v4 -194> 2014-08-13 17:23:22.686197 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686197, event: started, request: pg_info(1 pgs e194478:22.775) v4 -193> 2014-08-13 17:23:22.686223 7fcd0964c700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:23:22.686223, event: done, request: pg_info(1 pgs e194478:22.775) v4 -192> 2014-08-13 17:23:22.686254 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.56 192.168.1.111:6826/11446 -- pg_info(1 pgs e194478:22.775) v4 -- ?+0 0x5b4d780 -191> 2014-08-13 17:23:22.686295 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.353 192.168.1.108:6836/6227 -- pg_info(1 pgs e194478:22.775) v4 -- ?+0 0x5b4d5c0 -190> 2014-08-13 17:23:22.686489 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.320 192.168.1.106:6820/10050 14 ==== MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 ==== 26+0+0 (2546581235 0 0) 0x588ad80 con 0x4fde300 -189> 2014-08-13 17:23:22.686511 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686398, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -188> 2014-08-13 17:23:22.686522 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686401, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -187> 2014-08-13 17:23:22.686530 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686468, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -186> 2014-08-13 17:23:22.686539 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686508, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -185> 2014-08-13 17:23:22.686548 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686548, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -184> 2014-08-13 17:23:22.686572 7fcd0964c700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:23:22.686571, event: done, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194478 v2 -183> 2014-08-13 17:23:22.686586 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.014422 3 0.000093 -182> 2014-08-13 17:23:22.686627 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194477 pi=192979-194476/26 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -181> 2014-08-13 17:23:22.686664 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194478/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.027949 9 0.000791 -180> 2014-08-13 17:23:22.686704 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194478/194466 194476/194477/194477) [4,353,56] r=0 lpr=194477 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -179> 2014-08-13 17:23:22.687040 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 11 ==== pg_info(1 pgs e194478:24.1f7) v4 ==== 748+0+0 (2447981883 0 0) 0x5b4d940 con 0x4fdee00 -178> 2014-08-13 17:23:22.687061 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.686944, event: header_read, request: pg_info(1 pgs e194478:24.1f7) v4 -177> 2014-08-13 17:23:22.687072 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.686946, event: throttled, request: pg_info(1 pgs e194478:24.1f7) v4 -176> 2014-08-13 17:23:22.687081 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.687019, event: all_read, request: pg_info(1 pgs e194478:24.1f7) v4 -175> 2014-08-13 17:23:22.687089 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.687059, event: dispatched, request: pg_info(1 pgs e194478:24.1f7) v4 -174> 2014-08-13 17:23:22.687099 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.687098, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1f7) v4 -173> 2014-08-13 17:23:22.687110 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.687110, event: started, request: pg_info(1 pgs e194478:24.1f7) v4 -172> 2014-08-13 17:23:22.687138 7fcd0964c700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:23:22.687137, event: done, request: pg_info(1 pgs e194478:24.1f7) v4 -171> 2014-08-13 17:23:22.687532 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 11 ==== pg_info(1 pgs e194478:24.20e) v4 ==== 748+0+0 (3214501591 0 0) 0x5b49500 con 0x4f80b00 -170> 2014-08-13 17:23:22.687558 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687431, event: header_read, request: pg_info(1 pgs e194478:24.20e) v4 -169> 2014-08-13 17:23:22.687569 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687433, event: throttled, request: pg_info(1 pgs e194478:24.20e) v4 -168> 2014-08-13 17:23:22.687578 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687508, event: all_read, request: pg_info(1 pgs e194478:24.20e) v4 -167> 2014-08-13 17:23:22.687587 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687554, event: dispatched, request: pg_info(1 pgs e194478:24.20e) v4 -166> 2014-08-13 17:23:22.687596 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687596, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.20e) v4 -165> 2014-08-13 17:23:22.687608 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687608, event: started, request: pg_info(1 pgs e194478:24.20e) v4 -164> 2014-08-13 17:23:22.687636 7fcd0964c700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:23:22.687636, event: done, request: pg_info(1 pgs e194478:24.20e) v4 -163> 2014-08-13 17:23:22.688631 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.11 192.168.1.103:6813/37929 4 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52bb3c0 con 0x56ba940 -162> 2014-08-13 17:23:22.688657 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -161> 2014-08-13 17:23:22.688725 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.11 192.168.1.103:6813/37929 5 ==== pg_notify(22.1f1(5) epoch 194478) v5 ==== 1001+0+0 (3148264830 0 0) 0x5888e00 con 0x56ba940 -160> 2014-08-13 17:23:22.688746 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688631, event: header_read, request: pg_notify(22.1f1(5) epoch 194478) v5 -159> 2014-08-13 17:23:22.688757 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688632, event: throttled, request: pg_notify(22.1f1(5) epoch 194478) v5 -158> 2014-08-13 17:23:22.688766 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688706, event: all_read, request: pg_notify(22.1f1(5) epoch 194478) v5 -157> 2014-08-13 17:23:22.688775 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688743, event: dispatched, request: pg_notify(22.1f1(5) epoch 194478) v5 -156> 2014-08-13 17:23:22.688784 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688784, event: waiting_for_osdmap, request: pg_notify(22.1f1(5) epoch 194478) v5 -155> 2014-08-13 17:23:22.688793 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688793, event: started, request: pg_notify(22.1f1(5) epoch 194478) v5 -154> 2014-08-13 17:23:22.688820 7fcd0964c700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:23:22.688820, event: done, request: pg_notify(22.1f1(5) epoch 194478) v5 -153> 2014-08-13 17:23:22.688935 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.15 192.168.1.107:6812/53408 12 ==== MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 ==== 26+0+0 (3312421685 0 0) 0x5b4bf00 con 0x4f80b00 -152> 2014-08-13 17:23:22.688959 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.688837, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -151> 2014-08-13 17:23:22.688970 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.688839, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -150> 2014-08-13 17:23:22.688978 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.688906, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -149> 2014-08-13 17:23:22.688987 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.688956, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -148> 2014-08-13 17:23:22.689001 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.689001, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -147> 2014-08-13 17:23:22.689023 7fcd0964c700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:23:22.689023, event: done, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194478 v2 -146> 2014-08-13 17:23:22.689042 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 0.018413 3 0.002390 -145> 2014-08-13 17:23:22.689085 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194477 pi=193264-194476/30 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved -144> 2014-08-13 17:23:22.690991 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.338 192.168.1.106:6814/25893 8 ==== pg_info(1 pgs e194478:22.3d9) v4 ==== 714+0+0 (2468825207 0 0) 0x588a840 con 0x4f832e0 -143> 2014-08-13 17:23:22.691012 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.690846, event: header_read, request: pg_info(1 pgs e194478:22.3d9) v4 -142> 2014-08-13 17:23:22.691019 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.690847, event: throttled, request: pg_info(1 pgs e194478:22.3d9) v4 -141> 2014-08-13 17:23:22.691025 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.690969, event: all_read, request: pg_info(1 pgs e194478:22.3d9) v4 -140> 2014-08-13 17:23:22.691031 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.691009, event: dispatched, request: pg_info(1 pgs e194478:22.3d9) v4 -139> 2014-08-13 17:23:22.691042 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.691042, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.3d9) v4 -138> 2014-08-13 17:23:22.691054 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.691054, event: started, request: pg_info(1 pgs e194478:22.3d9) v4 -137> 2014-08-13 17:23:22.691081 7fcd0964c700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:23:22.691081, event: done, request: pg_info(1 pgs e194478:22.3d9) v4 -136> 2014-08-13 17:23:22.691112 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.338 192.168.1.106:6814/25893 -- pg_info(1 pgs e194478:22.3d9) v4 -- ?+0 0x5187e00 -135> 2014-08-13 17:23:22.691158 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194478:22.3d9) v4 -- ?+0 0x51878c0 -134> 2014-08-13 17:23:22.691549 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194478/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.025368 12 0.001000 -133> 2014-08-13 17:23:22.691587 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194478/194389 194476/194477/194477) [4,338,376] r=0 lpr=194477 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -132> 2014-08-13 17:23:22.691655 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.31 192.168.1.103:6831/41157 13 ==== pg_info(1 pgs e194478:24.1e4) v4 ==== 748+0+0 (664562453 0 0) 0x588dcc0 con 0x4ec56a0 -131> 2014-08-13 17:23:22.691699 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691510, event: header_read, request: pg_info(1 pgs e194478:24.1e4) v4 -130> 2014-08-13 17:23:22.691715 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691512, event: throttled, request: pg_info(1 pgs e194478:24.1e4) v4 -129> 2014-08-13 17:23:22.691725 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691585, event: all_read, request: pg_info(1 pgs e194478:24.1e4) v4 -128> 2014-08-13 17:23:22.691734 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691693, event: dispatched, request: pg_info(1 pgs e194478:24.1e4) v4 -127> 2014-08-13 17:23:22.691745 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691745, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.1e4) v4 -126> 2014-08-13 17:23:22.691758 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691758, event: started, request: pg_info(1 pgs e194478:24.1e4) v4 -125> 2014-08-13 17:23:22.691792 7fcd0964c700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:23:22.691791, event: done, request: pg_info(1 pgs e194478:24.1e4) v4 -124> 2014-08-13 17:23:22.691827 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 16 ==== pg_info(1 pgs e194478:24.3f7) v4 ==== 748+0+0 (3233911737 0 0) 0x5b4db00 con 0x5305ee0 -123> 2014-08-13 17:23:22.691848 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691672, event: header_read, request: pg_info(1 pgs e194478:24.3f7) v4 -122> 2014-08-13 17:23:22.691859 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691675, event: throttled, request: pg_info(1 pgs e194478:24.3f7) v4 -121> 2014-08-13 17:23:22.691869 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691739, event: all_read, request: pg_info(1 pgs e194478:24.3f7) v4 -120> 2014-08-13 17:23:22.691878 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691846, event: dispatched, request: pg_info(1 pgs e194478:24.3f7) v4 -119> 2014-08-13 17:23:22.691888 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691887, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.3f7) v4 -118> 2014-08-13 17:23:22.691899 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691899, event: started, request: pg_info(1 pgs e194478:24.3f7) v4 -117> 2014-08-13 17:23:22.691932 7fcd0964c700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:23:22.691932, event: done, request: pg_info(1 pgs e194478:24.3f7) v4 -116> 2014-08-13 17:23:22.692540 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.65 192.168.1.111:6842/17345 12 ==== pg_info(1 pgs e194478:22.619) v4 ==== 714+0+0 (351628231 0 0) 0x56aabc0 con 0x4fd9760 -115> 2014-08-13 17:23:22.692570 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692119, event: header_read, request: pg_info(1 pgs e194478:22.619) v4 -114> 2014-08-13 17:23:22.692587 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692122, event: throttled, request: pg_info(1 pgs e194478:22.619) v4 -113> 2014-08-13 17:23:22.692597 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692235, event: all_read, request: pg_info(1 pgs e194478:22.619) v4 -112> 2014-08-13 17:23:22.692606 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692566, event: dispatched, request: pg_info(1 pgs e194478:22.619) v4 -111> 2014-08-13 17:23:22.692617 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692616, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.619) v4 -110> 2014-08-13 17:23:22.692629 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692629, event: started, request: pg_info(1 pgs e194478:22.619) v4 -109> 2014-08-13 17:23:22.692660 7fcd0964c700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:23:22.692660, event: done, request: pg_info(1 pgs e194478:22.619) v4 -108> 2014-08-13 17:23:22.693670 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.330 192.168.1.106:6802/4876 17 ==== MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 ==== 26+0+0 (1792451784 0 0) 0x56a88c0 con 0x5305ee0 -107> 2014-08-13 17:23:22.693695 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693583, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -106> 2014-08-13 17:23:22.693706 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693587, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -105> 2014-08-13 17:23:22.693715 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693649, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -104> 2014-08-13 17:23:22.693725 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693692, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -103> 2014-08-13 17:23:22.693735 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693735, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -102> 2014-08-13 17:23:22.693758 7fcd0964c700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:23:22.693758, event: done, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194478 v2 -101> 2014-08-13 17:23:22.693821 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] exit Started/ReplicaActive/RepNotRecovering 0.020160 3 0.001387 -100> 2014-08-13 17:23:22.693874 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194477 pi=193227-194476/33 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] enter Started/ReplicaActive/RepWaitRecoveryReserved -99> 2014-08-13 17:23:22.694643 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 6 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567a1c0 con 0x552d540 -98> 2014-08-13 17:23:22.694676 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -97> 2014-08-13 17:23:22.694850 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.306 192.168.1.101:6806/6812 7 ==== pg_notify(22.481(39) epoch 194478) v5 ==== 2911+0+0 (1870023835 0 0) 0x56a8c40 con 0x552d540 -96> 2014-08-13 17:23:22.694879 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694669, event: header_read, request: pg_notify(22.481(39) epoch 194478) v5 -95> 2014-08-13 17:23:22.694888 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694671, event: throttled, request: pg_notify(22.481(39) epoch 194478) v5 -94> 2014-08-13 17:23:22.694894 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694798, event: all_read, request: pg_notify(22.481(39) epoch 194478) v5 -93> 2014-08-13 17:23:22.694900 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694875, event: dispatched, request: pg_notify(22.481(39) epoch 194478) v5 -92> 2014-08-13 17:23:22.694908 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694908, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194478) v5 -91> 2014-08-13 17:23:22.694917 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694917, event: started, request: pg_notify(22.481(39) epoch 194478) v5 -90> 2014-08-13 17:23:22.694948 7fcd0964c700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:23:22.694948, event: done, request: pg_notify(22.481(39) epoch 194478) v5 -89> 2014-08-13 17:23:22.695278 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 5 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x567bcc0 con 0x4fd35a0 -88> 2014-08-13 17:23:22.695306 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -87> 2014-08-13 17:23:22.695462 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.327 192.168.1.106:6804/5112 6 ==== pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 ==== 3140+0+0 (2169497203 0 0) 0x54d39c0 con 0x4fd35a0 -86> 2014-08-13 17:23:22.695484 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695309, event: header_read, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -85> 2014-08-13 17:23:22.695493 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695311, event: throttled, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -84> 2014-08-13 17:23:22.695499 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695441, event: all_read, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -83> 2014-08-13 17:23:22.695505 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695481, event: dispatched, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -82> 2014-08-13 17:23:22.695512 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695512, event: waiting_for_osdmap, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -81> 2014-08-13 17:23:22.695520 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695520, event: started, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -80> 2014-08-13 17:23:22.695562 7fcd0964c700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:23:22.695561, event: done, request: pg_notify(24.b6(6),22.48e(24) epoch 194478) v5 -79> 2014-08-13 17:23:22.697217 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.26 192.168.1.115:6802/58522 6 ==== pg_info(1 pgs e194478:24.34) v4 ==== 740+0+0 (3733486732 0 0) 0x5186e40 con 0x55244c0 -78> 2014-08-13 17:23:22.697247 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697094, event: header_read, request: pg_info(1 pgs e194478:24.34) v4 -77> 2014-08-13 17:23:22.697257 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697096, event: throttled, request: pg_info(1 pgs e194478:24.34) v4 -76> 2014-08-13 17:23:22.697263 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697190, event: all_read, request: pg_info(1 pgs e194478:24.34) v4 -75> 2014-08-13 17:23:22.697268 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697242, event: dispatched, request: pg_info(1 pgs e194478:24.34) v4 -74> 2014-08-13 17:23:22.697275 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697275, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.34) v4 -73> 2014-08-13 17:23:22.697287 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697287, event: started, request: pg_info(1 pgs e194478:24.34) v4 -72> 2014-08-13 17:23:22.697321 7fcd0964c700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:23:22.697321, event: done, request: pg_info(1 pgs e194478:24.34) v4 -71> 2014-08-13 17:23:22.699592 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 10 ==== pg_info(1 pgs e194478:24.181) v4 ==== 748+0+0 (3244150365 0 0) 0x54d0380 con 0x4ec70c0 -70> 2014-08-13 17:23:22.699636 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699392, event: header_read, request: pg_info(1 pgs e194478:24.181) v4 -69> 2014-08-13 17:23:22.699649 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699396, event: throttled, request: pg_info(1 pgs e194478:24.181) v4 -68> 2014-08-13 17:23:22.699655 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699513, event: all_read, request: pg_info(1 pgs e194478:24.181) v4 -67> 2014-08-13 17:23:22.699661 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699630, event: dispatched, request: pg_info(1 pgs e194478:24.181) v4 -66> 2014-08-13 17:23:22.699671 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699671, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.181) v4 -65> 2014-08-13 17:23:22.699681 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699681, event: started, request: pg_info(1 pgs e194478:24.181) v4 -64> 2014-08-13 17:23:22.699719 7fcd0964c700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:23:22.699719, event: done, request: pg_info(1 pgs e194478:24.181) v4 -63> 2014-08-13 17:23:22.699743 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.344 192.168.1.106:6812/6757 11 ==== pg_info(1 pgs e194478:22.39d) v4 ==== 714+0+0 (1011494820 0 0) 0x54d2f40 con 0x4ec70c0 -62> 2014-08-13 17:23:22.699759 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699536, event: header_read, request: pg_info(1 pgs e194478:22.39d) v4 -61> 2014-08-13 17:23:22.699765 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699537, event: throttled, request: pg_info(1 pgs e194478:22.39d) v4 -60> 2014-08-13 17:23:22.699771 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699602, event: all_read, request: pg_info(1 pgs e194478:22.39d) v4 -59> 2014-08-13 17:23:22.699781 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699756, event: dispatched, request: pg_info(1 pgs e194478:22.39d) v4 -58> 2014-08-13 17:23:22.699793 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699792, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.39d) v4 -57> 2014-08-13 17:23:22.699804 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699804, event: started, request: pg_info(1 pgs e194478:22.39d) v4 -56> 2014-08-13 17:23:22.699832 7fcd0964c700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:23:22.699832, event: done, request: pg_info(1 pgs e194478:22.39d) v4 -55> 2014-08-13 17:23:22.699908 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194478:22.39d) v4 -- ?+0 0x56adb00 -54> 2014-08-13 17:23:22.699955 7fcd02841700 1 -- 192.168.1.115:6819/31924 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194478:22.39d) v4 -- ?+0 0x56af380 -53> 2014-08-13 17:23:22.700347 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.045654 9 0.000648 -52> 2014-08-13 17:23:22.700386 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/194477) [4,373,344] r=0 lpr=194477 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -51> 2014-08-13 17:23:22.700680 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 12 ==== pg_info(1 pgs e194478:22.220) v4 ==== 714+0+0 (3645992202 0 0) 0x5b48e00 con 0x4fdee00 -50> 2014-08-13 17:23:22.700718 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700529, event: header_read, request: pg_info(1 pgs e194478:22.220) v4 -49> 2014-08-13 17:23:22.700739 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700532, event: throttled, request: pg_info(1 pgs e194478:22.220) v4 -48> 2014-08-13 17:23:22.700746 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700639, event: all_read, request: pg_info(1 pgs e194478:22.220) v4 -47> 2014-08-13 17:23:22.700751 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700713, event: dispatched, request: pg_info(1 pgs e194478:22.220) v4 -46> 2014-08-13 17:23:22.700759 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700759, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:22.220) v4 -45> 2014-08-13 17:23:22.700774 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700773, event: started, request: pg_info(1 pgs e194478:22.220) v4 -44> 2014-08-13 17:23:22.700808 7fcd0964c700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:23:22.700808, event: done, request: pg_info(1 pgs e194478:22.220) v4 -43> 2014-08-13 17:23:22.701507 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.351 192.168.1.102:6830/16444 13 ==== MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 ==== 26+0+0 (3105558320 0 0) 0x588a680 con 0x4fdee00 -42> 2014-08-13 17:23:22.701548 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701421, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -41> 2014-08-13 17:23:22.701560 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701423, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -40> 2014-08-13 17:23:22.701567 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701482, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -39> 2014-08-13 17:23:22.701573 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701543, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -38> 2014-08-13 17:23:22.701579 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701579, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -37> 2014-08-13 17:23:22.701611 7fcd0964c700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:23:22.701610, event: done, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194478 v2 -36> 2014-08-13 17:23:22.701628 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.029663 3 0.000146 -35> 2014-08-13 17:23:22.701658 7fcd02841700 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194477 pi=192970-194476/26 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -34> 2014-08-13 17:23:22.703035 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.61 192.168.1.111:6840/16566 17 ==== pg_info(1 pgs e194478:24.136) v4 ==== 748+0+0 (3747609268 0 0) 0x5b4e740 con 0x4e35ac0 -33> 2014-08-13 17:23:22.703065 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.702921, event: header_read, request: pg_info(1 pgs e194478:24.136) v4 -32> 2014-08-13 17:23:22.703074 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.702923, event: throttled, request: pg_info(1 pgs e194478:24.136) v4 -31> 2014-08-13 17:23:22.703080 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.703003, event: all_read, request: pg_info(1 pgs e194478:24.136) v4 -30> 2014-08-13 17:23:22.703090 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.703061, event: dispatched, request: pg_info(1 pgs e194478:24.136) v4 -29> 2014-08-13 17:23:22.703097 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.703096, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.136) v4 -28> 2014-08-13 17:23:22.703113 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.703113, event: started, request: pg_info(1 pgs e194478:24.136) v4 -27> 2014-08-13 17:23:22.703149 7fcd0964c700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:23:22.703149, event: done, request: pg_info(1 pgs e194478:24.136) v4 -26> 2014-08-13 17:23:22.703179 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 10 ==== osd_map(194478..194478 src has 185139..194478) v3 ==== 278+0+0 (1737859085 0 0) 0x52ba400 con 0x4e35800 -25> 2014-08-13 17:23:22.703201 7fcd0964c700 3 osd.4 194478 handle_osd_map epochs [194478,194478], i have 194478, src has [185139,194478] -24> 2014-08-13 17:23:22.703226 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.301 192.168.1.113:6814/16727 11 ==== pg_notify(24.30b(6) epoch 194478) v5 ==== 1094+0+0 (2029536763 0 0) 0x588b2c0 con 0x4e35800 -23> 2014-08-13 17:23:22.703247 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703110, event: header_read, request: pg_notify(24.30b(6) epoch 194478) v5 -22> 2014-08-13 17:23:22.703255 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703111, event: throttled, request: pg_notify(24.30b(6) epoch 194478) v5 -21> 2014-08-13 17:23:22.703261 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703208, event: all_read, request: pg_notify(24.30b(6) epoch 194478) v5 -20> 2014-08-13 17:23:22.703267 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703244, event: dispatched, request: pg_notify(24.30b(6) epoch 194478) v5 -19> 2014-08-13 17:23:22.703273 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703273, event: waiting_for_osdmap, request: pg_notify(24.30b(6) epoch 194478) v5 -18> 2014-08-13 17:23:22.703282 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703282, event: started, request: pg_notify(24.30b(6) epoch 194478) v5 -17> 2014-08-13 17:23:22.703310 7fcd0964c700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:23:22.703310, event: done, request: pg_notify(24.30b(6) epoch 194478) v5 -16> 2014-08-13 17:23:22.704969 7fcd0964c700 1 -- 192.168.1.115:6819/31924 <== osd.30 192.168.1.107:6823/55491 11 ==== pg_info(1 pgs e194478:24.34) v4 ==== 740+0+0 (156754386 0 0) 0x588a300 con 0x4ec3f40 -15> 2014-08-13 17:23:22.705012 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.704791, event: header_read, request: pg_info(1 pgs e194478:24.34) v4 -14> 2014-08-13 17:23:22.705024 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.704794, event: throttled, request: pg_info(1 pgs e194478:24.34) v4 -13> 2014-08-13 17:23:22.705031 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.704884, event: all_read, request: pg_info(1 pgs e194478:24.34) v4 -12> 2014-08-13 17:23:22.705036 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.705006, event: dispatched, request: pg_info(1 pgs e194478:24.34) v4 -11> 2014-08-13 17:23:22.705043 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.705043, event: waiting_for_osdmap, request: pg_info(1 pgs e194478:24.34) v4 -10> 2014-08-13 17:23:22.705063 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.705063, event: started, request: pg_info(1 pgs e194478:24.34) v4 -9> 2014-08-13 17:23:22.705098 7fcd0964c700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:23:22.705098, event: done, request: pg_info(1 pgs e194478:24.34) v4 -8> 2014-08-13 17:23:22.705127 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.26 192.168.1.115:6802/58522 -- pg_info(1 pgs e194478:24.34) v4 -- ?+0 0x5b4d240 -7> 2014-08-13 17:23:22.705184 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194478:24.34) v4 -- ?+0 0x5b4a140 -6> 2014-08-13 17:23:22.705213 7fcd01e40700 1 -- 192.168.1.115:6819/31924 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194478:24.34) v4 -- ?+0 0x5b4a4c0 -5> 2014-08-13 17:23:22.705292 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 -4> 2014-08-13 17:23:22.705325 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 -3> 2014-08-13 17:23:22.705343 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 -2> 2014-08-13 17:23:22.705359 7fcd01e40700 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194477 pi=193125-194476/50 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 -1> 2014-08-13 17:23:22.705745 7fcd01e40700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 0> 2014-08-13 17:23:22.710795 7fcd01e40700 -1 *** Caught signal (Floating point exception) ** in thread 7fcd01e40700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fcd1c3af710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fcd1c3a79d1] 16: (clone()+0x6d) [0x7fcd1b337b6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.4.log --- end dump of recent events --- 2014-08-13 17:23:22.735075 7fcd02841700 0 log [WRN] : pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 17:23:22.758655 7fcd02841700 0 log [WRN] : pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 17:38:01.948739 7fe9bae887a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 34805 2014-08-13 17:38:01.956921 7fe9bae887a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:38:01.956951 7fe9bae887a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-08-13 17:38:01.959278 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:38:01.959296 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:38:01.978705 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:38:01.978825 7fe9bae887a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:38:01.993053 7fe9bae887a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-13 17:38:01.998358 7fe9bae887a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:38:02.327776 7fe9bae887a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:38:02.339893 7fe9bae887a0 1 journal close /var/lib/ceph/osd/ceph-4/journal 2014-08-13 17:38:02.341683 7fe9bae887a0 2 osd.4 0 mounting /var/lib/ceph/osd/ceph-4 /var/lib/ceph/osd/ceph-4/journal 2014-08-13 17:38:02.341775 7fe9bae887a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 17:38:02.344134 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 17:38:02.344184 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 17:38:02.358559 7fe9bae887a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 17:38:02.358653 7fe9bae887a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 17:38:02.372139 7fe9bae887a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-13 17:38:02.377774 7fe9bae887a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 22: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:38:02.501565 7fe9bae887a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 22: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 17:38:02.501978 7fe9bae887a0 2 osd.4 0 boot 2014-08-13 17:38:02.502207 7fe9bae887a0 10 osd.4 0 read_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194478 [185139,194478] lci=[194476,194478]) 2014-08-13 17:38:02.502394 7fe9bae887a0 10 open_all_classes 2014-08-13 17:38:02.502490 7fe9bae887a0 10 open_all_classes found rbd 2014-08-13 17:38:02.502554 7fe9bae887a0 10 _get_class adding new class name rbd 0x2a881e8 2014-08-13 17:38:02.502567 7fe9bae887a0 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so 2014-08-13 17:38:02.503517 7fe9bae887a0 10 register_class rbd status 3 2014-08-13 17:38:02.503534 7fe9bae887a0 10 register_cxx_method rbd.create flags 3 0x7fe9b16fd8b0 2014-08-13 17:38:02.503541 7fe9bae887a0 10 register_cxx_method rbd.get_features flags 1 0x7fe9b1702830 2014-08-13 17:38:02.503544 7fe9bae887a0 10 register_cxx_method rbd.get_size flags 1 0x7fe9b1703d50 2014-08-13 17:38:02.503548 7fe9bae887a0 10 register_cxx_method rbd.set_size flags 3 0x7fe9b1702bd0 2014-08-13 17:38:02.503551 7fe9bae887a0 10 register_cxx_method rbd.get_snapcontext flags 1 0x7fe9b16fe8b0 2014-08-13 17:38:02.503565 7fe9bae887a0 10 register_cxx_method rbd.get_object_prefix flags 1 0x7fe9b1702330 2014-08-13 17:38:02.503590 7fe9bae887a0 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7fe9b1701070 2014-08-13 17:38:02.503596 7fe9bae887a0 10 register_cxx_method rbd.snapshot_add flags 3 0x7fe9b17031a0 2014-08-13 17:38:02.503599 7fe9bae887a0 10 register_cxx_method rbd.snapshot_remove flags 3 0x7fe9b1700e20 2014-08-13 17:38:02.503609 7fe9bae887a0 10 register_cxx_method rbd.get_all_features flags 1 0x7fe9b16fa3f0 2014-08-13 17:38:02.503612 7fe9bae887a0 10 register_cxx_method rbd.copyup flags 3 0x7fe9b16fa370 2014-08-13 17:38:02.503615 7fe9bae887a0 10 register_cxx_method rbd.get_parent flags 1 0x7fe9b17041c0 2014-08-13 17:38:02.503619 7fe9bae887a0 10 register_cxx_method rbd.set_parent flags 3 0x7fe9b1701d20 2014-08-13 17:38:02.503622 7fe9bae887a0 10 register_cxx_method rbd.remove_parent flags 3 0x7fe9b1701b10 2014-08-13 17:38:02.503625 7fe9bae887a0 10 register_cxx_method rbd.set_protection_status flags 3 0x7fe9b1701530 2014-08-13 17:38:02.503628 7fe9bae887a0 10 register_cxx_method rbd.get_protection_status flags 1 0x7fe9b17012c0 2014-08-13 17:38:02.503631 7fe9bae887a0 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7fe9b1702550 2014-08-13 17:38:02.503634 7fe9bae887a0 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7fe9b16fc3d0 2014-08-13 17:38:02.503637 7fe9bae887a0 10 register_cxx_method rbd.add_child flags 3 0x7fe9b16fef60 2014-08-13 17:38:02.503640 7fe9bae887a0 10 register_cxx_method rbd.remove_child flags 3 0x7fe9b17002e0 2014-08-13 17:38:02.503643 7fe9bae887a0 10 register_cxx_method rbd.get_children flags 1 0x7fe9b16ff4c0 2014-08-13 17:38:02.503646 7fe9bae887a0 10 register_cxx_method rbd.get_id flags 1 0x7fe9b16fa960 2014-08-13 17:38:02.503650 7fe9bae887a0 10 register_cxx_method rbd.set_id flags 3 0x7fe9b16fecd0 2014-08-13 17:38:02.503652 7fe9bae887a0 10 register_cxx_method rbd.dir_get_id flags 1 0x7fe9b16fbfd0 2014-08-13 17:38:02.503655 7fe9bae887a0 10 register_cxx_method rbd.dir_get_name flags 1 0x7fe9b16fc7d0 2014-08-13 17:38:02.503658 7fe9bae887a0 10 register_cxx_method rbd.dir_list flags 1 0x7fe9b16fcd70 2014-08-13 17:38:02.503661 7fe9bae887a0 10 register_cxx_method rbd.dir_add_image flags 3 0x7fe9b16fe670 2014-08-13 17:38:02.503665 7fe9bae887a0 10 register_cxx_method rbd.dir_remove_image flags 3 0x7fe9b16fbdb0 2014-08-13 17:38:02.503668 7fe9bae887a0 10 register_cxx_method rbd.dir_rename_image flags 3 0x7fe9b16fe390 2014-08-13 17:38:02.503671 7fe9bae887a0 10 register_cxx_method rbd.snap_list flags 1 0x7fe9b1701860 2014-08-13 17:38:02.503674 7fe9bae887a0 10 register_cxx_method rbd.snap_add flags 3 0x7fe9b16ff830 2014-08-13 17:38:02.503677 7fe9bae887a0 10 register_cxx_method rbd.snap_remove flags 3 0x7fe9b16ffd50 2014-08-13 17:38:02.503680 7fe9bae887a0 10 _load_class rbd success 2014-08-13 17:38:02.503684 7fe9bae887a0 10 open_all_classes found refcount 2014-08-13 17:38:02.503688 7fe9bae887a0 10 _get_class adding new class name refcount 0x2a88108 2014-08-13 17:38:02.503692 7fe9bae887a0 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so 2014-08-13 17:38:02.508544 7fe9bae887a0 10 register_class refcount status 3 2014-08-13 17:38:02.508547 7fe9bae887a0 10 register_cxx_method refcount.get flags 3 0x7fe9b13cd050 2014-08-13 17:38:02.508552 7fe9bae887a0 10 register_cxx_method refcount.put flags 3 0x7fe9b13cd610 2014-08-13 17:38:02.508555 7fe9bae887a0 10 register_cxx_method refcount.set flags 3 0x7fe9b13cbc60 2014-08-13 17:38:02.508558 7fe9bae887a0 10 register_cxx_method refcount.read flags 1 0x7fe9b13ccaa0 2014-08-13 17:38:02.508561 7fe9bae887a0 10 _load_class refcount success 2014-08-13 17:38:02.508564 7fe9bae887a0 10 open_all_classes found replica_log 2014-08-13 17:38:02.508568 7fe9bae887a0 10 _get_class adding new class name replica_log 0x2a88028 2014-08-13 17:38:02.508572 7fe9bae887a0 10 _load_class replica_log from /usr/lib64/rados-classes/libcls_replica_log.so 2014-08-13 17:38:02.508780 7fe9bae887a0 10 register_class replica_log status 3 2014-08-13 17:38:02.508782 7fe9bae887a0 10 register_cxx_method replica_log.set flags 3 0x7fe9b1122b30 2014-08-13 17:38:02.508786 7fe9bae887a0 10 register_cxx_method replica_log.get flags 1 0x7fe9b1121ac0 2014-08-13 17:38:02.508789 7fe9bae887a0 10 register_cxx_method replica_log.delete flags 3 0x7fe9b11225b0 2014-08-13 17:38:02.508792 7fe9bae887a0 10 _load_class replica_log success 2014-08-13 17:38:02.508795 7fe9bae887a0 10 open_all_classes found statelog 2014-08-13 17:38:02.508804 7fe9bae887a0 10 _get_class adding new class name statelog 0x2a88568 2014-08-13 17:38:02.508807 7fe9bae887a0 10 _load_class statelog from /usr/lib64/rados-classes/libcls_statelog.so 2014-08-13 17:38:02.509011 7fe9bae887a0 10 register_class statelog status 3 2014-08-13 17:38:02.509014 7fe9bae887a0 10 register_cxx_method statelog.add flags 3 0x7fe9a8d03360 2014-08-13 17:38:02.509018 7fe9bae887a0 10 register_cxx_method statelog.list flags 1 0x7fe9a8d020e0 2014-08-13 17:38:02.509021 7fe9bae887a0 10 register_cxx_method statelog.remove flags 3 0x7fe9a8d04330 2014-08-13 17:38:02.509032 7fe9bae887a0 10 register_cxx_method statelog.check_state flags 1 0x7fe9a8d03aa0 2014-08-13 17:38:02.509036 7fe9bae887a0 10 _load_class statelog success 2014-08-13 17:38:02.509038 7fe9bae887a0 10 open_all_classes found rgw 2014-08-13 17:38:02.509042 7fe9bae887a0 10 _get_class adding new class name rgw 0x2a88488 2014-08-13 17:38:02.509045 7fe9bae887a0 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so 2014-08-13 17:38:02.513142 7fe9bae887a0 10 register_class rgw status 3 2014-08-13 17:38:02.513153 7fe9bae887a0 10 register_cxx_method rgw.bucket_init_index flags 3 0x7fe9a89bccc0 2014-08-13 17:38:02.513159 7fe9bae887a0 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7fe9a89bc740 2014-08-13 17:38:02.513162 7fe9bae887a0 10 register_cxx_method rgw.bucket_list flags 1 0x7fe9a89c3480 2014-08-13 17:38:02.513166 7fe9bae887a0 10 register_cxx_method rgw.bucket_check_index flags 1 0x7fe9a89c3030 2014-08-13 17:38:02.513169 7fe9bae887a0 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7fe9a89c32d0 2014-08-13 17:38:02.513172 7fe9bae887a0 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7fe9a89c50f0 2014-08-13 17:38:02.513175 7fe9bae887a0 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7fe9a89c5af0 2014-08-13 17:38:02.513178 7fe9bae887a0 10 register_cxx_method rgw.bi_log_list flags 1 0x7fe9a89c1280 2014-08-13 17:38:02.513185 7fe9bae887a0 10 register_cxx_method rgw.bi_log_trim flags 3 0x7fe9a89c0df0 2014-08-13 17:38:02.513188 7fe9bae887a0 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7fe9a89c45e0 2014-08-13 17:38:02.513192 7fe9bae887a0 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7fe9a89c0000 2014-08-13 17:38:02.513194 7fe9bae887a0 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7fe9a89bdf30 2014-08-13 17:38:02.513197 7fe9bae887a0 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7fe9a89bdbd0 2014-08-13 17:38:02.513201 7fe9bae887a0 10 register_cxx_method rgw.gc_set_entry flags 3 0x7fe9a89bf960 2014-08-13 17:38:02.513204 7fe9bae887a0 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7fe9a89bf660 2014-08-13 17:38:02.513208 7fe9bae887a0 10 register_cxx_method rgw.gc_list flags 1 0x7fe9a89be7d0 2014-08-13 17:38:02.513211 7fe9bae887a0 10 register_cxx_method rgw.gc_remove flags 3 0x7fe9a89bfc00 2014-08-13 17:38:02.513214 7fe9bae887a0 10 _load_class rgw success 2014-08-13 17:38:02.513217 7fe9bae887a0 10 open_all_classes found version 2014-08-13 17:38:02.513221 7fe9bae887a0 10 _get_class adding new class name version 0x2a883a8 2014-08-13 17:38:02.513224 7fe9bae887a0 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so 2014-08-13 17:38:02.513438 7fe9bae887a0 10 register_class version status 3 2014-08-13 17:38:02.513475 7fe9bae887a0 10 register_cxx_method version.set flags 3 0x7fe9a870d3c0 2014-08-13 17:38:02.513480 7fe9bae887a0 10 register_cxx_method version.inc flags 3 0x7fe9a870e6b0 2014-08-13 17:38:02.513483 7fe9bae887a0 10 register_cxx_method version.inc_conds flags 3 0x7fe9a870e6b0 2014-08-13 17:38:02.513486 7fe9bae887a0 10 register_cxx_method version.read flags 1 0x7fe9a870dd20 2014-08-13 17:38:02.513489 7fe9bae887a0 10 register_cxx_method version.check_conds flags 1 0x7fe9a870e0a0 2014-08-13 17:38:02.513492 7fe9bae887a0 10 _load_class version success 2014-08-13 17:38:02.513494 7fe9bae887a0 10 open_all_classes found lock 2014-08-13 17:38:02.513498 7fe9bae887a0 10 _get_class adding new class name lock 0x2a882c8 2014-08-13 17:38:02.513501 7fe9bae887a0 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so 2014-08-13 17:38:02.513738 7fe9bae887a0 10 register_class lock status 3 2014-08-13 17:38:02.513745 7fe9bae887a0 10 register_cxx_method lock.lock flags 3 0x7fe9a85055f0 2014-08-13 17:38:02.513750 7fe9bae887a0 10 register_cxx_method lock.unlock flags 3 0x7fe9a8506510 2014-08-13 17:38:02.513753 7fe9bae887a0 10 register_cxx_method lock.break_lock flags 3 0x7fe9a8506950 2014-08-13 17:38:02.513756 7fe9bae887a0 10 register_cxx_method lock.get_info flags 1 0x7fe9a8504ad0 2014-08-13 17:38:02.513759 7fe9bae887a0 10 register_cxx_method lock.list_locks flags 1 0x7fe9a8503630 2014-08-13 17:38:02.513762 7fe9bae887a0 10 _load_class lock success 2014-08-13 17:38:02.513765 7fe9bae887a0 10 open_all_classes found kvs 2014-08-13 17:38:02.513769 7fe9bae887a0 10 _get_class adding new class name kvs 0x2a889c8 2014-08-13 17:38:02.513772 7fe9bae887a0 10 _load_class kvs from /usr/lib64/rados-classes/libcls_kvs.so 2014-08-13 17:38:02.514041 7fe9bae887a0 10 register_class kvs status 3 2014-08-13 17:38:02.514048 7fe9bae887a0 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7fe9a82f6b10 2014-08-13 17:38:02.514052 7fe9bae887a0 10 register_cxx_method kvs.get_next_idata flags 1 0x7fe9a82f5b70 2014-08-13 17:38:02.514055 7fe9bae887a0 10 register_cxx_method kvs.get_prev_idata flags 1 0x7fe9a82f7560 2014-08-13 17:38:02.514058 7fe9bae887a0 10 register_cxx_method kvs.read_many flags 1 0x7fe9a82f6100 2014-08-13 17:38:02.514061 7fe9bae887a0 10 register_cxx_method kvs.check_writable flags 3 0x7fe9a82f3ab0 2014-08-13 17:38:02.514064 7fe9bae887a0 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7fe9a82f3ec0 2014-08-13 17:38:02.514069 7fe9bae887a0 10 register_cxx_method kvs.omap_insert flags 2 0x7fe9a82f5390 2014-08-13 17:38:02.514072 7fe9bae887a0 10 register_cxx_method kvs.create_with_omap flags 2 0x7fe9a82f40e0 2014-08-13 17:38:02.514075 7fe9bae887a0 10 register_cxx_method kvs.omap_remove flags 2 0x7fe9a82f4c80 2014-08-13 17:38:02.514078 7fe9bae887a0 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7fe9a82f44a0 2014-08-13 17:38:02.514081 7fe9bae887a0 10 _load_class kvs success 2014-08-13 17:38:02.514084 7fe9bae887a0 10 open_all_classes found hello 2014-08-13 17:38:02.514087 7fe9bae887a0 10 _get_class adding new class name hello 0x2a888e8 2014-08-13 17:38:02.514091 7fe9bae887a0 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so 2014-08-13 17:38:02.514296 7fe9bae887a0 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-13 17:38:02.514304 7fe9bae887a0 10 register_class hello status 3 2014-08-13 17:38:02.514306 7fe9bae887a0 10 register_cxx_method hello.say_hello flags 1 0x7fe9a80ec1f0 2014-08-13 17:38:02.514311 7fe9bae887a0 10 register_cxx_method hello.record_hello flags 2 0x7fe9a80ec3e0 2014-08-13 17:38:02.514314 7fe9bae887a0 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7fe9a80ec960 2014-08-13 17:38:02.514317 7fe9bae887a0 10 register_cxx_method hello.replay flags 1 0x7fe9a80ec1c0 2014-08-13 17:38:02.514320 7fe9bae887a0 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7fe9a80ecc40 2014-08-13 17:38:02.514323 7fe9bae887a0 10 register_cxx_method hello.bad_reader flags 2 0x7fe9a80ec1b0 2014-08-13 17:38:02.514326 7fe9bae887a0 10 register_cxx_method hello.bad_writer flags 1 0x7fe9a80ec1a0 2014-08-13 17:38:02.514329 7fe9bae887a0 10 _load_class hello success 2014-08-13 17:38:02.514331 7fe9bae887a0 10 open_all_classes found log 2014-08-13 17:38:02.514335 7fe9bae887a0 10 _get_class adding new class name log 0x2a88808 2014-08-13 17:38:02.514338 7fe9bae887a0 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so 2014-08-13 17:38:02.514565 7fe9bae887a0 10 register_class log status 3 2014-08-13 17:38:02.514568 7fe9bae887a0 10 register_cxx_method log.add flags 3 0x7fe9a7ee5370 2014-08-13 17:38:02.514572 7fe9bae887a0 10 register_cxx_method log.list flags 1 0x7fe9a7ee61f0 2014-08-13 17:38:02.514575 7fe9bae887a0 10 register_cxx_method log.trim flags 3 0x7fe9a7ee4b40 2014-08-13 17:38:02.514578 7fe9bae887a0 10 register_cxx_method log.info flags 1 0x7fe9a7ee75d0 2014-08-13 17:38:02.514581 7fe9bae887a0 10 _load_class log success 2014-08-13 17:38:02.514584 7fe9bae887a0 10 open_all_classes found user 2014-08-13 17:38:02.514588 7fe9bae887a0 10 _get_class adding new class name user 0x2a88728 2014-08-13 17:38:02.514591 7fe9bae887a0 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so 2014-08-13 17:38:02.515055 7fe9bae887a0 10 register_class user status 3 2014-08-13 17:38:02.515058 7fe9bae887a0 10 register_cxx_method user.set_buckets_info flags 3 0x7fe9a7cdd220 2014-08-13 17:38:02.515117 7fe9bae887a0 10 register_cxx_method user.complete_stats_sync flags 3 0x7fe9a7cdbff0 2014-08-13 17:38:02.515128 7fe9bae887a0 10 register_cxx_method user.remove_bucket flags 3 0x7fe9a7cde210 2014-08-13 17:38:02.515132 7fe9bae887a0 10 register_cxx_method user.list_buckets flags 1 0x7fe9a7cdc3c0 2014-08-13 17:38:02.515135 7fe9bae887a0 10 register_cxx_method user.get_header flags 1 0x7fe9a7cdbc50 2014-08-13 17:38:02.515138 7fe9bae887a0 10 _load_class user success 2014-08-13 17:38:02.515188 7fe9bae887a0 20 osd.4 0 get_map 194478 - loading and decoding 0x2745400 2014-08-13 17:38:02.515733 7fe9bae887a0 10 osd.4 0 add_map_bl 194478 268447 bytes 2014-08-13 17:38:02.517453 7fe9bae887a0 0 osd.4 194478 crush map has features 2303210029056, adjusting msgr requires for clients 2014-08-13 17:38:02.517476 7fe9bae887a0 0 osd.4 194478 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:38:02.517485 7fe9bae887a0 0 osd.4 194478 crush map has features 2578087936000, adjusting msgr requires for osds 2014-08-13 17:38:02.517493 7fe9bae887a0 10 osd.4 194478 create_recoverystate_perf 2014-08-13 17:38:02.517517 7fe9bae887a0 0 osd.4 194478 load_pgs 2014-08-13 17:38:02.518095 7fe9bae887a0 10 osd.4 194478 load_pgs ignoring unrecognized meta 2014-08-13 17:38:02.518346 7fe9bae887a0 10 osd.4 194478 pgid 1.2b coll 1.2b_head 2014-08-13 17:38:02.518834 7fe9bae887a0 20 osd.4 0 get_map 194477 - loading and decoding 0x2745200 2014-08-13 17:38:02.519257 7fe9bae887a0 10 osd.4 0 add_map_bl 194477 268447 bytes 2014-08-13 17:38:02.522199 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 1.2b 2014-08-13 17:38:02.522224 7fe9bae887a0 10 osd.4 194478 _get_pool 1 2014-08-13 17:38:02.522350 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[1.2b(unlocked)] enter Initial 2014-08-13 17:38:02.522406 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[1.2b(unlocked)] enter NotTrimming 2014-08-13 17:38:02.522719 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.523084 7fe9bae887a0 20 read_log 192888'1 (0'0) modify 968c7c2b/benchmark_data_storage0111-ib.pocstor_11336_object26/head//1 by client.1312877.0:27 2014-08-11 16:25:04.098381 2014-08-13 17:38:02.523157 7fe9bae887a0 20 read_log 192888'2 (0'0) modify 458d82b/benchmark_data_storage0111-ib.pocstor_11336_object85/head//1 by client.1312877.0:86 2014-08-11 16:25:04.370979 2014-08-13 17:38:02.523186 7fe9bae887a0 20 read_log 192888'3 (0'0) modify 8d3a1cab/benchmark_data_storage0111-ib.pocstor_11336_object115/head//1 by client.1312877.0:116 2014-08-11 16:25:04.477993 2014-08-13 17:38:02.523210 7fe9bae887a0 20 read_log 192888'4 (0'0) modify 174a7a6b/benchmark_data_storage0111-ib.pocstor_11336_object120/head//1 by client.1312877.0:121 2014-08-11 16:25:04.495532 2014-08-13 17:38:02.523232 7fe9bae887a0 20 read_log 192888'5 (0'0) modify 709b852b/benchmark_data_storage0111-ib.pocstor_11336_object514/head//1 by client.1312877.0:515 2014-08-11 16:25:05.966566 2014-08-13 17:38:02.523254 7fe9bae887a0 20 read_log 192888'6 (0'0) modify f9c4ff6b/benchmark_data_storage0111-ib.pocstor_11336_object529/head//1 by client.1312877.0:530 2014-08-11 16:25:06.020839 2014-08-13 17:38:02.523278 7fe9bae887a0 20 read_log 192888'7 (0'0) modify 5c599e6b/benchmark_data_storage0111-ib.pocstor_11336_object584/head//1 by client.1312877.0:585 2014-08-11 16:25:06.289680 2014-08-13 17:38:02.523300 7fe9bae887a0 20 read_log 192888'8 (0'0) modify cc5ad2ab/benchmark_data_storage0111-ib.pocstor_11336_object589/head//1 by client.1312877.0:590 2014-08-11 16:25:06.316558 2014-08-13 17:38:02.523321 7fe9bae887a0 20 read_log 192888'9 (0'0) modify e2cda9ab/benchmark_data_storage0111-ib.pocstor_11336_object680/head//1 by client.1312877.0:681 2014-08-11 16:25:06.690737 2014-08-13 17:38:02.523344 7fe9bae887a0 20 read_log 192888'10 (0'0) modify 6a2d76b/benchmark_data_storage0111-ib.pocstor_11336_object727/head//1 by client.1312877.0:728 2014-08-11 16:25:06.857642 2014-08-13 17:38:02.523366 7fe9bae887a0 20 read_log 192888'11 (0'0) modify d4542b6b/benchmark_data_storage0111-ib.pocstor_11336_object785/head//1 by client.1312877.0:786 2014-08-11 16:25:07.094331 2014-08-13 17:38:02.523388 7fe9bae887a0 20 read_log 192888'12 (0'0) modify 13924b6b/benchmark_data_storage0111-ib.pocstor_11336_object893/head//1 by client.1312877.0:894 2014-08-11 16:25:07.534706 2014-08-13 17:38:02.523417 7fe9bae887a0 20 read_log 192888'13 (0'0) modify 2df8bfab/benchmark_data_storage0111-ib.pocstor_11336_object920/head//1 by client.1312877.0:921 2014-08-11 16:25:07.631660 2014-08-13 17:38:02.523451 7fe9bae887a0 20 read_log 192888'14 (0'0) modify 1142b5eb/benchmark_data_storage0111-ib.pocstor_11336_object942/head//1 by client.1312877.0:943 2014-08-11 16:25:07.711037 2014-08-13 17:38:02.523478 7fe9bae887a0 20 read_log 192888'15 (0'0) modify afd4156b/benchmark_data_storage0111-ib.pocstor_11336_object1007/head//1 by client.1312877.0:1008 2014-08-11 16:25:07.946317 2014-08-13 17:38:02.523503 7fe9bae887a0 20 read_log 192888'16 (0'0) modify 1bd77ceb/benchmark_data_storage0111-ib.pocstor_11336_object1053/head//1 by client.1312877.0:1054 2014-08-11 16:25:08.111621 2014-08-13 17:38:02.523543 7fe9bae887a0 20 read_log 192888'17 (0'0) modify 4c3c3eab/benchmark_data_storage0111-ib.pocstor_11336_object1107/head//1 by client.1312877.0:1108 2014-08-11 16:25:08.304277 2014-08-13 17:38:02.523742 7fe9bae887a0 20 read_log 192888'18 (0'0) modify 84e0c2b/benchmark_data_storage0111-ib.pocstor_11336_object1197/head//1 by client.1312877.0:1198 2014-08-11 16:25:08.623939 2014-08-13 17:38:02.523807 7fe9bae887a0 20 read_log 192888'19 (0'0) modify 5acc686b/benchmark_data_storage0111-ib.pocstor_11336_object1402/head//1 by client.1312877.0:1403 2014-08-11 16:25:09.360339 2014-08-13 17:38:02.523838 7fe9bae887a0 20 read_log 192888'20 (0'0) modify cd6caab/benchmark_data_storage0111-ib.pocstor_11336_object1404/head//1 by client.1312877.0:1405 2014-08-11 16:25:09.367539 2014-08-13 17:38:02.523861 7fe9bae887a0 20 read_log 192888'21 (0'0) modify f337eceb/benchmark_data_storage0111-ib.pocstor_11336_object1448/head//1 by client.1312877.0:1449 2014-08-11 16:25:09.524131 2014-08-13 17:38:02.523883 7fe9bae887a0 20 read_log 192888'22 (0'0) modify fdf8f3eb/benchmark_data_storage0111-ib.pocstor_11336_object1668/head//1 by client.1312877.0:1669 2014-08-11 16:25:10.333172 2014-08-13 17:38:02.523905 7fe9bae887a0 20 read_log 192888'23 (0'0) modify 784e352b/benchmark_data_storage0111-ib.pocstor_11336_object1682/head//1 by client.1312877.0:1683 2014-08-11 16:25:10.382879 2014-08-13 17:38:02.523927 7fe9bae887a0 20 read_log 192888'24 (0'0) modify dda332b/benchmark_data_storage0111-ib.pocstor_11336_object1930/head//1 by client.1312877.0:1931 2014-08-11 16:25:11.400791 2014-08-13 17:38:02.523953 7fe9bae887a0 20 read_log 192888'25 (0'0) modify 93f31b2b/benchmark_data_storage0111-ib.pocstor_11336_object2038/head//1 by client.1312877.0:2039 2014-08-11 16:25:11.783817 2014-08-13 17:38:02.523975 7fe9bae887a0 20 read_log 192888'26 (0'0) modify 357eddeb/benchmark_data_storage0111-ib.pocstor_11336_object2073/head//1 by client.1312877.0:2074 2014-08-11 16:25:11.908139 2014-08-13 17:38:02.523998 7fe9bae887a0 20 read_log 192888'27 (0'0) modify 27ad09eb/benchmark_data_storage0111-ib.pocstor_11336_object2157/head//1 by client.1312877.0:2158 2014-08-11 16:25:12.208789 2014-08-13 17:38:02.524023 7fe9bae887a0 20 read_log 192888'28 (0'0) modify 78fc9beb/benchmark_data_storage0111-ib.pocstor_11336_object2224/head//1 by client.1312877.0:2225 2014-08-11 16:25:12.447857 2014-08-13 17:38:02.524045 7fe9bae887a0 20 read_log 192888'29 (0'0) modify 4721d42b/benchmark_data_storage0111-ib.pocstor_11336_object2247/head//1 by client.1312877.0:2248 2014-08-11 16:25:12.530070 2014-08-13 17:38:02.524075 7fe9bae887a0 20 read_log 192888'30 (0'0) modify d935726b/benchmark_data_storage0111-ib.pocstor_11336_object2299/head//1 by client.1312877.0:2300 2014-08-11 16:25:12.715972 2014-08-13 17:38:02.524097 7fe9bae887a0 20 read_log 192888'31 (0'0) modify 28d2c6ab/benchmark_data_storage0111-ib.pocstor_11336_object2510/head//1 by client.1312877.0:2511 2014-08-11 16:25:13.464648 2014-08-13 17:38:02.524119 7fe9bae887a0 20 read_log 192888'32 (0'0) modify a9ff716b/benchmark_data_storage0111-ib.pocstor_11336_object2683/head//1 by client.1312877.0:2684 2014-08-11 16:25:14.225172 2014-08-13 17:38:02.524140 7fe9bae887a0 20 read_log 192888'33 (0'0) modify 6eac06ab/benchmark_data_storage0111-ib.pocstor_11336_object2861/head//1 by client.1312877.0:2862 2014-08-11 16:25:14.854269 2014-08-13 17:38:02.524178 7fe9bae887a0 20 read_log 192888'34 (0'0) modify e39450ab/benchmark_data_storage0111-ib.pocstor_11336_object2944/head//1 by client.1312877.0:2945 2014-08-11 16:25:15.149385 2014-08-13 17:38:02.524201 7fe9bae887a0 20 read_log 192888'35 (0'0) modify 372d6ab/benchmark_data_storage0111-ib.pocstor_11336_object2981/head//1 by client.1312877.0:2982 2014-08-11 16:25:15.278781 2014-08-13 17:38:02.524223 7fe9bae887a0 20 read_log 192888'36 (0'0) modify aaad6a2b/benchmark_data_storage0111-ib.pocstor_11336_object2989/head//1 by client.1312877.0:2990 2014-08-11 16:25:15.306677 2014-08-13 17:38:02.524249 7fe9bae887a0 20 read_log 192888'37 (0'0) modify dbd623ab/benchmark_data_storage0111-ib.pocstor_11336_object3227/head//1 by client.1312877.0:3228 2014-08-11 16:25:16.152623 2014-08-13 17:38:02.524271 7fe9bae887a0 20 read_log 192888'38 (0'0) modify f13350eb/benchmark_data_storage0111-ib.pocstor_11336_object3246/head//1 by client.1312877.0:3247 2014-08-11 16:25:16.219261 2014-08-13 17:38:02.524316 7fe9bae887a0 20 read_log 192888'39 (0'0) modify 752770eb/benchmark_data_storage0111-ib.pocstor_11336_object3306/head//1 by client.1312877.0:3307 2014-08-11 16:25:16.433463 2014-08-13 17:38:02.524344 7fe9bae887a0 20 read_log 192888'40 (0'0) modify dd84e9ab/benchmark_data_storage0111-ib.pocstor_11336_object3341/head//1 by client.1312877.0:3342 2014-08-11 16:25:16.556429 2014-08-13 17:38:02.524366 7fe9bae887a0 20 read_log 192888'41 (0'0) modify de0a9deb/benchmark_data_storage0111-ib.pocstor_11336_object3370/head//1 by client.1312877.0:3371 2014-08-11 16:25:16.659507 2014-08-13 17:38:02.524388 7fe9bae887a0 20 read_log 192888'42 (0'0) modify 7d7620ab/benchmark_data_storage0111-ib.pocstor_11336_object3373/head//1 by client.1312877.0:3374 2014-08-11 16:25:16.670278 2014-08-13 17:38:02.524411 7fe9bae887a0 20 read_log 192888'43 (0'0) modify 83a675ab/benchmark_data_storage0111-ib.pocstor_11336_object3566/head//1 by client.1312877.0:3567 2014-08-11 16:25:17.379436 2014-08-13 17:38:02.524433 7fe9bae887a0 20 read_log 192888'44 (0'0) modify 8ef2e3eb/benchmark_data_storage0111-ib.pocstor_11336_object3609/head//1 by client.1312877.0:3610 2014-08-11 16:25:17.542587 2014-08-13 17:38:02.524459 7fe9bae887a0 20 read_log 192888'45 (0'0) modify c40043eb/benchmark_data_storage0111-ib.pocstor_11336_object3630/head//1 by client.1312877.0:3631 2014-08-11 16:25:17.621035 2014-08-13 17:38:02.524481 7fe9bae887a0 20 read_log 192888'46 (0'0) modify 83bd3aeb/benchmark_data_storage0111-ib.pocstor_11336_object3634/head//1 by client.1312877.0:3635 2014-08-11 16:25:17.635998 2014-08-13 17:38:02.524515 7fe9bae887a0 20 read_log 192888'47 (0'0) modify a7339fab/benchmark_data_storage0111-ib.pocstor_11336_object3636/head//1 by client.1312877.0:3637 2014-08-11 16:25:17.643430 2014-08-13 17:38:02.524537 7fe9bae887a0 20 read_log 192888'48 (0'0) modify b51ef32b/benchmark_data_storage0111-ib.pocstor_11336_object3841/head//1 by client.1312877.0:3842 2014-08-11 16:25:18.437037 2014-08-13 17:38:02.524559 7fe9bae887a0 20 read_log 192888'49 (0'0) modify e3142eab/benchmark_data_storage0111-ib.pocstor_11336_object4049/head//1 by client.1312877.0:4050 2014-08-11 16:25:19.254584 2014-08-13 17:38:02.524581 7fe9bae887a0 20 read_log 192888'50 (0'0) modify 2e39aeb/benchmark_data_storage0111-ib.pocstor_11336_object4076/head//1 by client.1312877.0:4077 2014-08-11 16:25:19.360926 2014-08-13 17:38:02.524603 7fe9bae887a0 20 read_log 192888'51 (0'0) modify 52e94fab/benchmark_data_storage0111-ib.pocstor_11336_object4179/head//1 by client.1312877.0:4180 2014-08-11 16:25:19.979133 2014-08-13 17:38:02.524624 7fe9bae887a0 20 read_log 192888'52 (0'0) modify a4d05a6b/benchmark_data_storage0111-ib.pocstor_11336_object4217/head//1 by client.1312877.0:4218 2014-08-11 16:25:20.149909 2014-08-13 17:38:02.524664 7fe9bae887a0 20 read_log 192888'53 (0'0) modify 8e026b6b/benchmark_data_storage0111-ib.pocstor_11336_object4269/head//1 by client.1312877.0:4270 2014-08-11 16:25:20.388934 2014-08-13 17:38:02.524691 7fe9bae887a0 20 read_log 192888'54 (0'0) modify 45e77deb/benchmark_data_storage0111-ib.pocstor_11336_object4277/head//1 by client.1312877.0:4278 2014-08-11 16:25:20.425898 2014-08-13 17:38:02.524713 7fe9bae887a0 20 read_log 192888'55 (0'0) modify 2779eb/benchmark_data_storage0111-ib.pocstor_11336_object4350/head//1 by client.1312877.0:4351 2014-08-11 16:25:20.748320 2014-08-13 17:38:02.524737 7fe9bae887a0 20 read_log 192888'56 (0'0) modify 200acdeb/benchmark_data_storage0111-ib.pocstor_11336_object4425/head//1 by client.1312877.0:4426 2014-08-11 16:25:21.033727 2014-08-13 17:38:02.524760 7fe9bae887a0 20 read_log 192888'57 (0'0) modify 4967bfeb/benchmark_data_storage0111-ib.pocstor_11336_object4506/head//1 by client.1312877.0:4507 2014-08-11 16:25:21.338278 2014-08-13 17:38:02.524781 7fe9bae887a0 20 read_log 192888'58 (0'0) modify 13ab5f6b/benchmark_data_storage0111-ib.pocstor_11336_object4629/head//1 by client.1312877.0:4630 2014-08-11 16:25:21.813661 2014-08-13 17:38:02.524815 7fe9bae887a0 20 read_log 192888'59 (0'0) modify e9c088eb/benchmark_data_storage0111-ib.pocstor_11336_object4640/head//1 by client.1312877.0:4641 2014-08-11 16:25:21.854777 2014-08-13 17:38:02.524843 7fe9bae887a0 20 read_log 192888'60 (0'0) modify 84ba19eb/benchmark_data_storage0111-ib.pocstor_11336_object4722/head//1 by client.1312877.0:4723 2014-08-11 16:25:22.710537 2014-08-13 17:38:02.524865 7fe9bae887a0 20 read_log 192888'61 (0'0) modify 64f0a7ab/benchmark_data_storage0111-ib.pocstor_11336_object4767/head//1 by client.1312877.0:4768 2014-08-11 16:25:23.037677 2014-08-13 17:38:02.524887 7fe9bae887a0 20 read_log 192888'62 (0'0) modify 18830eeb/benchmark_data_storage0111-ib.pocstor_11336_object4790/head//1 by client.1312877.0:4791 2014-08-11 16:25:23.150607 2014-08-13 17:38:02.524909 7fe9bae887a0 20 read_log 192888'63 (0'0) modify 80cba86b/benchmark_data_storage0111-ib.pocstor_11336_object4818/head//1 by client.1312877.0:4819 2014-08-11 16:25:23.257310 2014-08-13 17:38:02.524931 7fe9bae887a0 20 read_log 192888'64 (0'0) modify 7b42aa6b/benchmark_data_storage0111-ib.pocstor_11336_object4861/head//1 by client.1312877.0:4862 2014-08-11 16:25:23.422893 2014-08-13 17:38:02.524952 7fe9bae887a0 20 read_log 192888'65 (0'0) modify cb176c6b/benchmark_data_storage0111-ib.pocstor_11336_object4943/head//1 by client.1312877.0:4944 2014-08-11 16:25:23.742523 2014-08-13 17:38:02.524974 7fe9bae887a0 20 read_log 192888'66 (0'0) modify 34a403eb/benchmark_data_storage0111-ib.pocstor_11336_object5103/head//1 by client.1312877.0:5104 2014-08-11 16:25:24.664707 2014-08-13 17:38:02.525003 7fe9bae887a0 20 read_log 192888'67 (0'0) modify 43cf3beb/benchmark_data_storage0111-ib.pocstor_11336_object5115/head//1 by client.1312877.0:5116 2014-08-11 16:25:24.711221 2014-08-13 17:38:02.525028 7fe9bae887a0 20 read_log 192888'68 (0'0) modify ef97dfab/benchmark_data_storage0111-ib.pocstor_11336_object5267/head//1 by client.1312877.0:5268 2014-08-11 16:25:25.292076 2014-08-13 17:38:02.525050 7fe9bae887a0 20 read_log 192888'69 (0'0) modify 2a7643eb/benchmark_data_storage0111-ib.pocstor_11336_object5291/head//1 by client.1312877.0:5292 2014-08-11 16:25:25.383847 2014-08-13 17:38:02.525072 7fe9bae887a0 20 read_log 192888'70 (0'0) modify c91d172b/benchmark_data_storage0111-ib.pocstor_11336_object5349/head//1 by client.1312877.0:5350 2014-08-11 16:25:25.605487 2014-08-13 17:38:02.525094 7fe9bae887a0 20 read_log 192888'71 (0'0) modify 86f5e06b/benchmark_data_storage0111-ib.pocstor_11336_object5415/head//1 by client.1312877.0:5416 2014-08-11 16:25:25.855272 2014-08-13 17:38:02.525117 7fe9bae887a0 20 read_log 192888'72 (0'0) modify 199e7beb/benchmark_data_storage0111-ib.pocstor_11336_object5466/head//1 by client.1312877.0:5467 2014-08-11 16:25:26.048325 2014-08-13 17:38:02.525138 7fe9bae887a0 20 read_log 192888'73 (0'0) modify 7c15dbab/benchmark_data_storage0111-ib.pocstor_11336_object5510/head//1 by client.1312877.0:5511 2014-08-11 16:25:26.215683 2014-08-13 17:38:02.525167 7fe9bae887a0 20 read_log 192888'74 (0'0) modify 5bad7f6b/benchmark_data_storage0111-ib.pocstor_11336_object5611/head//1 by client.1312877.0:5612 2014-08-11 16:25:26.598058 2014-08-13 17:38:02.525190 7fe9bae887a0 20 read_log 192888'75 (0'0) modify b65b286b/benchmark_data_storage0111-ib.pocstor_11336_object5618/head//1 by client.1312877.0:5619 2014-08-11 16:25:26.624552 2014-08-13 17:38:02.525216 7fe9bae887a0 20 read_log 192888'76 (0'0) modify 51e9e32b/benchmark_data_storage0111-ib.pocstor_11336_object5649/head//1 by client.1312877.0:5650 2014-08-11 16:25:26.741722 2014-08-13 17:38:02.525239 7fe9bae887a0 20 read_log 192888'77 (0'0) modify be6f1b2b/benchmark_data_storage0111-ib.pocstor_11336_object5663/head//1 by client.1312877.0:5664 2014-08-11 16:25:26.796655 2014-08-13 17:38:02.525260 7fe9bae887a0 20 read_log 192888'78 (0'0) modify ef81c46b/benchmark_data_storage0111-ib.pocstor_11336_object5705/head//1 by client.1312877.0:5706 2014-08-11 16:25:26.955338 2014-08-13 17:38:02.525294 7fe9bae887a0 20 read_log 192888'79 (0'0) modify 7460ea2b/benchmark_data_storage0111-ib.pocstor_11336_object5758/head//1 by client.1312877.0:5759 2014-08-11 16:25:27.156703 2014-08-13 17:38:02.525317 7fe9bae887a0 20 read_log 192888'80 (0'0) modify d8711a2b/benchmark_data_storage0111-ib.pocstor_11336_object5802/head//1 by client.1312877.0:5803 2014-08-11 16:25:27.323639 2014-08-13 17:38:02.525349 7fe9bae887a0 20 read_log 192888'81 (0'0) modify 7cea612b/benchmark_data_storage0111-ib.pocstor_11336_object5943/head//1 by client.1312877.0:5944 2014-08-11 16:25:27.878556 2014-08-13 17:38:02.525371 7fe9bae887a0 20 read_log 192888'82 (0'0) modify b862b52b/benchmark_data_storage0111-ib.pocstor_11336_object5996/head//1 by client.1312877.0:5997 2014-08-11 16:25:28.083412 2014-08-13 17:38:02.525392 7fe9bae887a0 20 read_log 192888'83 (0'0) modify aa0fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object6043/head//1 by client.1312877.0:6044 2014-08-11 16:25:28.262041 2014-08-13 17:38:02.525415 7fe9bae887a0 20 read_log 192888'84 (0'0) modify 1f2e13eb/benchmark_data_storage0111-ib.pocstor_11336_object6114/head//1 by client.1312877.0:6115 2014-08-11 16:25:28.535889 2014-08-13 17:38:02.525436 7fe9bae887a0 20 read_log 192888'85 (0'0) modify 26185aeb/benchmark_data_storage0111-ib.pocstor_11336_object6115/head//1 by client.1312877.0:6116 2014-08-11 16:25:28.539681 2014-08-13 17:38:02.525458 7fe9bae887a0 20 read_log 192888'86 (0'0) modify bc9f81ab/benchmark_data_storage0111-ib.pocstor_11336_object6256/head//1 by client.1312877.0:6257 2014-08-11 16:25:29.081741 2014-08-13 17:38:02.525480 7fe9bae887a0 20 read_log 192888'87 (0'0) modify 665d8cab/benchmark_data_storage0111-ib.pocstor_11336_object6345/head//1 by client.1312877.0:6346 2014-08-11 16:25:29.421793 2014-08-13 17:38:02.525502 7fe9bae887a0 20 read_log 192888'88 (0'0) modify 2797a2b/benchmark_data_storage0111-ib.pocstor_11336_object6356/head//1 by client.1312877.0:6357 2014-08-11 16:25:29.463723 2014-08-13 17:38:02.525523 7fe9bae887a0 20 read_log 192888'89 (0'0) modify dd86336b/benchmark_data_storage0111-ib.pocstor_11336_object6360/head//1 by client.1312877.0:6361 2014-08-11 16:25:29.479031 2014-08-13 17:38:02.525545 7fe9bae887a0 20 read_log 192888'90 (0'0) modify 81e68deb/benchmark_data_storage0111-ib.pocstor_11336_object6546/head//1 by client.1312877.0:6547 2014-08-11 16:25:30.256045 2014-08-13 17:38:02.525567 7fe9bae887a0 20 read_log 192888'91 (0'0) modify cacbb16b/benchmark_data_storage0111-ib.pocstor_11336_object6567/head//1 by client.1312877.0:6568 2014-08-11 16:25:30.336123 2014-08-13 17:38:02.525588 7fe9bae887a0 20 read_log 192888'92 (0'0) modify 982d3c2b/benchmark_data_storage0111-ib.pocstor_11336_object6610/head//1 by client.1312877.0:6611 2014-08-11 16:25:30.502431 2014-08-13 17:38:02.525610 7fe9bae887a0 20 read_log 192888'93 (0'0) modify 7c69c2b/benchmark_data_storage0111-ib.pocstor_11336_object6688/head//1 by client.1312877.0:6689 2014-08-11 16:25:30.801607 2014-08-13 17:38:02.525631 7fe9bae887a0 20 read_log 192888'94 (0'0) modify 4db680ab/benchmark_data_storage0111-ib.pocstor_11336_object6735/head//1 by client.1312877.0:6736 2014-08-11 16:25:30.982438 2014-08-13 17:38:02.525653 7fe9bae887a0 20 read_log 192888'95 (0'0) modify ee8926eb/benchmark_data_storage0111-ib.pocstor_11336_object6785/head//1 by client.1312877.0:6786 2014-08-11 16:25:31.174880 2014-08-13 17:38:02.525674 7fe9bae887a0 20 read_log 192888'96 (0'0) modify 183f4fab/benchmark_data_storage0111-ib.pocstor_11336_object6867/head//1 by client.1312877.0:6868 2014-08-11 16:25:31.489574 2014-08-13 17:38:02.525696 7fe9bae887a0 20 read_log 192888'97 (0'0) modify 77e164ab/benchmark_data_storage0111-ib.pocstor_11336_object6990/head//1 by client.1312877.0:6991 2014-08-11 16:25:31.957041 2014-08-13 17:38:02.525717 7fe9bae887a0 20 read_log 192888'98 (0'0) modify e34b5a2b/benchmark_data_storage0111-ib.pocstor_11336_object6999/head//1 by client.1312877.0:7000 2014-08-11 16:25:31.990958 2014-08-13 17:38:02.525749 7fe9bae887a0 20 read_log 192888'99 (0'0) modify aa9237eb/benchmark_data_storage0111-ib.pocstor_11336_object7008/head//1 by client.1312877.0:7009 2014-08-11 16:25:32.026460 2014-08-13 17:38:02.525775 7fe9bae887a0 20 read_log 192888'100 (0'0) modify ac4a732b/benchmark_data_storage0111-ib.pocstor_11336_object7094/head//1 by client.1312877.0:7095 2014-08-11 16:25:32.355711 2014-08-13 17:38:02.525798 7fe9bae887a0 20 read_log 192888'101 (0'0) modify c413e7ab/benchmark_data_storage0111-ib.pocstor_11336_object7152/head//1 by client.1312877.0:7153 2014-08-11 16:25:32.576689 2014-08-13 17:38:02.525820 7fe9bae887a0 20 read_log 192888'102 (0'0) modify ef342f6b/benchmark_data_storage0111-ib.pocstor_11336_object7178/head//1 by client.1312877.0:7179 2014-08-11 16:25:32.674886 2014-08-13 17:38:02.525843 7fe9bae887a0 20 read_log 192888'103 (0'0) modify 581f046b/benchmark_data_storage0111-ib.pocstor_11336_object7210/head//1 by client.1312877.0:7211 2014-08-11 16:25:32.798047 2014-08-13 17:38:02.525865 7fe9bae887a0 20 read_log 192888'104 (0'0) modify 6e01b76b/benchmark_data_storage0111-ib.pocstor_11336_object7236/head//1 by client.1312877.0:7237 2014-08-11 16:25:32.895808 2014-08-13 17:38:02.525886 7fe9bae887a0 20 read_log 192888'105 (0'0) modify d200a6eb/benchmark_data_storage0111-ib.pocstor_11336_object7281/head//1 by client.1312877.0:7282 2014-08-11 16:25:33.067087 2014-08-13 17:38:02.525908 7fe9bae887a0 20 read_log 192888'106 (0'0) modify 84fb60eb/benchmark_data_storage0111-ib.pocstor_11336_object7370/head//1 by client.1312877.0:7371 2014-08-11 16:25:33.403090 2014-08-13 17:38:02.525930 7fe9bae887a0 20 read_log 192888'107 (0'0) modify d6b5f52b/benchmark_data_storage0111-ib.pocstor_11336_object7447/head//1 by client.1312877.0:7448 2014-08-11 16:25:33.697163 2014-08-13 17:38:02.525952 7fe9bae887a0 20 read_log 192888'108 (0'0) modify 8552af2b/benchmark_data_storage0111-ib.pocstor_11336_object7477/head//1 by client.1312877.0:7478 2014-08-11 16:25:33.810783 2014-08-13 17:38:02.525975 7fe9bae887a0 20 read_log 192888'109 (0'0) modify d583a86b/benchmark_data_storage0111-ib.pocstor_11336_object7517/head//1 by client.1312877.0:7518 2014-08-11 16:25:33.963205 2014-08-13 17:38:02.525996 7fe9bae887a0 20 read_log 192888'110 (0'0) modify 5fb5a3eb/benchmark_data_storage0111-ib.pocstor_11336_object7554/head//1 by client.1312877.0:7555 2014-08-11 16:25:34.105265 2014-08-13 17:38:02.526018 7fe9bae887a0 20 read_log 192888'111 (0'0) modify fb03b0eb/benchmark_data_storage0111-ib.pocstor_11336_object7647/head//1 by client.1312877.0:7648 2014-08-11 16:25:34.459397 2014-08-13 17:38:02.526040 7fe9bae887a0 20 read_log 192888'112 (0'0) modify 6b05496b/benchmark_data_storage0111-ib.pocstor_11336_object7737/head//1 by client.1312877.0:7738 2014-08-11 16:25:34.798043 2014-08-13 17:38:02.526061 7fe9bae887a0 20 read_log 192888'113 (0'0) modify 8326d86b/benchmark_data_storage0111-ib.pocstor_11336_object7812/head//1 by client.1312877.0:7813 2014-08-11 16:25:35.091267 2014-08-13 17:38:02.526088 7fe9bae887a0 20 read_log 192888'114 (0'0) modify e8e52e2b/benchmark_data_storage0111-ib.pocstor_11336_object7900/head//1 by client.1312877.0:7901 2014-08-11 16:25:35.566020 2014-08-13 17:38:02.526110 7fe9bae887a0 20 read_log 192888'115 (0'0) modify e48691eb/benchmark_data_storage0111-ib.pocstor_11336_object7922/head//1 by client.1312877.0:7923 2014-08-11 16:25:35.648126 2014-08-13 17:38:02.526134 7fe9bae887a0 20 read_log 192888'116 (0'0) modify 10dd5aeb/benchmark_data_storage0111-ib.pocstor_11336_object7942/head//1 by client.1312877.0:7943 2014-08-11 16:25:35.722210 2014-08-13 17:38:02.526166 7fe9bae887a0 20 read_log 192888'117 (0'0) modify 1dd889ab/benchmark_data_storage0111-ib.pocstor_11336_object8027/head//1 by client.1312877.0:8028 2014-08-11 16:25:36.105574 2014-08-13 17:38:02.526192 7fe9bae887a0 20 read_log 192888'118 (0'0) modify 8d332b2b/benchmark_data_storage0111-ib.pocstor_11336_object8120/head//1 by client.1312877.0:8121 2014-08-11 16:25:36.593921 2014-08-13 17:38:02.526234 7fe9bae887a0 20 read_log 192888'119 (0'0) modify 1793a06b/benchmark_data_storage0111-ib.pocstor_11336_object8121/head//1 by client.1312877.0:8122 2014-08-11 16:25:36.597541 2014-08-13 17:38:02.526263 7fe9bae887a0 20 read_log 192888'120 (0'0) modify b4faa32b/benchmark_data_storage0111-ib.pocstor_11336_object8144/head//1 by client.1312877.0:8145 2014-08-11 16:25:36.689453 2014-08-13 17:38:02.526285 7fe9bae887a0 20 read_log 192888'121 (0'0) modify 4035182b/benchmark_data_storage0111-ib.pocstor_11336_object8213/head//1 by client.1312877.0:8214 2014-08-11 16:25:37.082509 2014-08-13 17:38:02.526341 7fe9bae887a0 20 read_log 192888'122 (0'0) modify 461c62b/benchmark_data_storage0111-ib.pocstor_11336_object8226/head//1 by client.1312877.0:8227 2014-08-11 16:25:37.144136 2014-08-13 17:38:02.526367 7fe9bae887a0 20 read_log 192888'123 (0'0) modify fef695ab/benchmark_data_storage0111-ib.pocstor_11336_object8282/head//1 by client.1312877.0:8283 2014-08-11 16:25:37.439393 2014-08-13 17:38:02.526389 7fe9bae887a0 20 read_log 192888'124 (0'0) modify 133eef2b/benchmark_data_storage0111-ib.pocstor_11336_object8316/head//1 by client.1312877.0:8317 2014-08-11 16:25:37.582436 2014-08-13 17:38:02.526411 7fe9bae887a0 20 read_log 192888'125 (0'0) modify 5905afab/benchmark_data_storage0111-ib.pocstor_11336_object8508/head//1 by client.1312877.0:8509 2014-08-11 16:25:38.443785 2014-08-13 17:38:02.526432 7fe9bae887a0 20 read_log 192888'126 (0'0) modify 1c3d296b/benchmark_data_storage0111-ib.pocstor_11336_object8550/head//1 by client.1312877.0:8551 2014-08-11 16:25:38.887026 2014-08-13 17:38:02.526454 7fe9bae887a0 20 read_log 192888'127 (0'0) modify 582141eb/benchmark_data_storage0111-ib.pocstor_11336_object8559/head//1 by client.1312877.0:8560 2014-08-11 16:25:38.927100 2014-08-13 17:38:02.526475 7fe9bae887a0 20 read_log 192888'128 (0'0) modify 39758f6b/benchmark_data_storage0111-ib.pocstor_11336_object8581/head//1 by client.1312877.0:8582 2014-08-11 16:25:39.007912 2014-08-13 17:38:02.526497 7fe9bae887a0 20 read_log 192888'129 (0'0) modify 5016fc6b/benchmark_data_storage0111-ib.pocstor_11336_object8602/head//1 by client.1312877.0:8603 2014-08-11 16:25:39.100287 2014-08-13 17:38:02.526519 7fe9bae887a0 20 read_log 192888'130 (0'0) modify 11560a2b/benchmark_data_storage0111-ib.pocstor_11336_object8708/head//1 by client.1312877.0:8709 2014-08-11 16:25:39.514490 2014-08-13 17:38:02.526541 7fe9bae887a0 20 read_log 192888'131 (0'0) modify 22b07aab/benchmark_data_storage0111-ib.pocstor_11336_object8772/head//1 by client.1312877.0:8773 2014-08-11 16:25:39.820079 2014-08-13 17:38:02.526563 7fe9bae887a0 20 read_log 192888'132 (0'0) modify 7e5d59ab/benchmark_data_storage0111-ib.pocstor_11336_object8779/head//1 by client.1312877.0:8780 2014-08-11 16:25:39.863261 2014-08-13 17:38:02.526584 7fe9bae887a0 20 read_log 192888'133 (0'0) modify ec37116b/benchmark_data_storage0111-ib.pocstor_11336_object8784/head//1 by client.1312877.0:8785 2014-08-11 16:25:39.888051 2014-08-13 17:38:02.526608 7fe9bae887a0 20 read_log 192888'134 (0'0) modify b0de23eb/benchmark_data_storage0111-ib.pocstor_11336_object8926/head//1 by client.1312877.0:8927 2014-08-11 16:25:40.838236 2014-08-13 17:38:02.526630 7fe9bae887a0 20 read_log 192888'135 (0'0) modify bb89eab/benchmark_data_storage0111-ib.pocstor_11336_object8969/head//1 by client.1312877.0:8970 2014-08-11 16:25:41.099505 2014-08-13 17:38:02.526652 7fe9bae887a0 20 read_log 192888'136 (0'0) modify 1573a02b/benchmark_data_storage0111-ib.pocstor_11336_object9147/head//1 by client.1312877.0:9148 2014-08-11 16:25:41.911075 2014-08-13 17:38:02.526719 7fe9bae887a0 20 read_log 192888'137 (0'0) modify 9b3d4deb/benchmark_data_storage0111-ib.pocstor_11336_object9217/head//1 by client.1312877.0:9218 2014-08-11 16:25:42.164072 2014-08-13 17:38:02.526744 7fe9bae887a0 20 read_log 192888'138 (0'0) modify 7c824dab/benchmark_data_storage0111-ib.pocstor_11336_object9339/head//1 by client.1312877.0:9340 2014-08-11 16:25:42.595095 2014-08-13 17:38:02.526778 7fe9bae887a0 20 read_log 192888'139 (0'0) modify 7ce138eb/benchmark_data_storage0111-ib.pocstor_11336_object9430/head//1 by client.1312877.0:9431 2014-08-11 16:25:43.020432 2014-08-13 17:38:02.526802 7fe9bae887a0 20 read_log 192888'140 (0'0) modify 5dd698ab/benchmark_data_storage0111-ib.pocstor_11336_object9467/head//1 by client.1312877.0:9468 2014-08-11 16:25:43.149613 2014-08-13 17:38:02.526824 7fe9bae887a0 20 read_log 192888'141 (0'0) modify 6fa2c96b/benchmark_data_storage0111-ib.pocstor_11336_object9547/head//1 by client.1312877.0:9548 2014-08-11 16:25:43.467545 2014-08-13 17:38:02.526846 7fe9bae887a0 20 read_log 192888'142 (0'0) modify af93b72b/benchmark_data_storage0111-ib.pocstor_11336_object9599/head//1 by client.1312877.0:9600 2014-08-11 16:25:43.665776 2014-08-13 17:38:02.526867 7fe9bae887a0 20 read_log 192888'143 (0'0) modify 9e00ec6b/benchmark_data_storage0111-ib.pocstor_11336_object9624/head//1 by client.1312877.0:9625 2014-08-11 16:25:43.884548 2014-08-13 17:38:02.526889 7fe9bae887a0 20 read_log 192888'144 (0'0) modify 30d7d7ab/benchmark_data_storage0111-ib.pocstor_11336_object9654/head//1 by client.1312877.0:9655 2014-08-11 16:25:44.037661 2014-08-13 17:38:02.526911 7fe9bae887a0 20 read_log 192888'145 (0'0) modify c30ef8eb/benchmark_data_storage0111-ib.pocstor_11336_object9675/head//1 by client.1312877.0:9676 2014-08-11 16:25:44.146318 2014-08-13 17:38:02.526965 7fe9bae887a0 20 read_log 192888'146 (0'0) modify e5c5686b/benchmark_data_storage0111-ib.pocstor_11336_object9798/head//1 by client.1312877.0:9799 2014-08-11 16:25:44.702352 2014-08-13 17:38:02.526991 7fe9bae887a0 20 read_log 192888'147 (0'0) modify f0c1866b/benchmark_data_storage0111-ib.pocstor_11336_object9809/head//1 by client.1312877.0:9810 2014-08-11 16:25:44.740451 2014-08-13 17:38:02.527014 7fe9bae887a0 20 read_log 192888'148 (0'0) modify 7dfac8eb/benchmark_data_storage0111-ib.pocstor_11336_object9862/head//1 by client.1312877.0:9863 2014-08-11 16:25:44.925703 2014-08-13 17:38:02.527036 7fe9bae887a0 20 read_log 192888'149 (0'0) modify 7385cbab/benchmark_data_storage0111-ib.pocstor_11336_object10044/head//1 by client.1312877.0:10045 2014-08-11 16:25:45.589416 2014-08-13 17:38:02.527058 7fe9bae887a0 20 read_log 192888'150 (0'0) modify 8cef43ab/benchmark_data_storage0111-ib.pocstor_11336_object10070/head//1 by client.1312877.0:10071 2014-08-11 16:25:45.682268 2014-08-13 17:38:02.527082 7fe9bae887a0 20 read_log 192888'151 (0'0) modify a97eac2b/benchmark_data_storage0111-ib.pocstor_11336_object10215/head//1 by client.1312877.0:10216 2014-08-11 16:25:46.234787 2014-08-13 17:38:02.527105 7fe9bae887a0 20 read_log 192888'152 (0'0) modify 9dccadab/benchmark_data_storage0111-ib.pocstor_11336_object10293/head//1 by client.1312877.0:10294 2014-08-11 16:25:46.668768 2014-08-13 17:38:02.527128 7fe9bae887a0 20 read_log 192888'153 (0'0) modify 403f876b/benchmark_data_storage0111-ib.pocstor_11336_object10422/head//1 by client.1312877.0:10423 2014-08-11 16:25:47.419241 2014-08-13 17:38:02.527150 7fe9bae887a0 20 read_log 192888'154 (0'0) modify d913feab/benchmark_data_storage0111-ib.pocstor_11336_object10499/head//1 by client.1312877.0:10500 2014-08-11 16:25:47.706624 2014-08-13 17:38:02.527185 7fe9bae887a0 20 read_log 192888'155 (0'0) modify 1067f22b/benchmark_data_storage0111-ib.pocstor_11336_object10519/head//1 by client.1312877.0:10520 2014-08-11 16:25:47.779852 2014-08-13 17:38:02.527214 7fe9bae887a0 20 read_log 192888'156 (0'0) modify 4ca0adab/benchmark_data_storage0111-ib.pocstor_11336_object10563/head//1 by client.1312877.0:10564 2014-08-11 16:25:47.936464 2014-08-13 17:38:02.527245 7fe9bae887a0 20 read_log 192888'157 (0'0) modify cd10626b/benchmark_data_storage0111-ib.pocstor_11336_object10629/head//1 by client.1312877.0:10630 2014-08-11 16:25:48.170968 2014-08-13 17:38:02.527277 7fe9bae887a0 20 read_log 192888'158 (0'0) modify 2d074d6b/benchmark_data_storage0111-ib.pocstor_11336_object10684/head//1 by client.1312877.0:10685 2014-08-11 16:25:48.362526 2014-08-13 17:38:02.527326 7fe9bae887a0 20 read_log 192888'159 (0'0) modify 7094446b/benchmark_data_storage0111-ib.pocstor_11336_object10863/head//1 by client.1312877.0:10864 2014-08-11 16:25:49.013359 2014-08-13 17:38:02.527356 7fe9bae887a0 20 read_log 192888'160 (0'0) modify 24fc1eab/benchmark_data_storage0111-ib.pocstor_11336_object10891/head//1 by client.1312877.0:10892 2014-08-11 16:25:49.135547 2014-08-13 17:38:02.527379 7fe9bae887a0 20 read_log 192888'161 (0'0) modify ba27f86b/benchmark_data_storage0111-ib.pocstor_11336_object10963/head//1 by client.1312877.0:10964 2014-08-11 16:25:49.451940 2014-08-13 17:38:02.527403 7fe9bae887a0 20 read_log 192888'162 (0'0) modify 979816b/benchmark_data_storage0111-ib.pocstor_11336_object10980/head//1 by client.1312877.0:10981 2014-08-11 16:25:49.537095 2014-08-13 17:38:02.527424 7fe9bae887a0 20 read_log 192888'163 (0'0) modify 7f2fe8eb/benchmark_data_storage0111-ib.pocstor_11336_object10990/head//1 by client.1312877.0:10991 2014-08-11 16:25:49.595577 2014-08-13 17:38:02.527446 7fe9bae887a0 20 read_log 192888'164 (0'0) modify 53e63eb/benchmark_data_storage0111-ib.pocstor_11336_object11017/head//1 by client.1312877.0:11018 2014-08-11 16:25:49.733507 2014-08-13 17:38:02.527470 7fe9bae887a0 20 read_log 192888'165 (0'0) modify b5c20a6b/benchmark_data_storage0111-ib.pocstor_11336_object11120/head//1 by client.1312877.0:11121 2014-08-11 16:25:50.148548 2014-08-13 17:38:02.527491 7fe9bae887a0 20 read_log 192888'166 (0'0) modify 2c9accab/benchmark_data_storage0111-ib.pocstor_11336_object11135/head//1 by client.1312877.0:11136 2014-08-11 16:25:50.227359 2014-08-13 17:38:02.527514 7fe9bae887a0 20 read_log 192888'167 (0'0) modify 8cd4cbab/benchmark_data_storage0111-ib.pocstor_11336_object11148/head//1 by client.1312877.0:11149 2014-08-11 16:25:50.297770 2014-08-13 17:38:02.527536 7fe9bae887a0 20 read_log 192888'168 (0'0) modify 5a894beb/benchmark_data_storage0111-ib.pocstor_11336_object11176/head//1 by client.1312877.0:11177 2014-08-11 16:25:50.461762 2014-08-13 17:38:02.527557 7fe9bae887a0 20 read_log 192888'169 (0'0) modify a326b22b/benchmark_data_storage0111-ib.pocstor_11336_object11234/head//1 by client.1312877.0:11235 2014-08-11 16:25:50.769210 2014-08-13 17:38:02.527578 7fe9bae887a0 20 read_log 192888'170 (0'0) modify f27ed76b/benchmark_data_storage0111-ib.pocstor_11336_object11321/head//1 by client.1312877.0:11322 2014-08-11 16:25:51.348582 2014-08-13 17:38:02.527600 7fe9bae887a0 20 read_log 192888'171 (0'0) modify 36effaeb/benchmark_data_storage0111-ib.pocstor_11336_object11355/head//1 by client.1312877.0:11356 2014-08-11 16:25:51.761371 2014-08-13 17:38:02.527621 7fe9bae887a0 20 read_log 192888'172 (0'0) modify c959e92b/benchmark_data_storage0111-ib.pocstor_11336_object11363/head//1 by client.1312877.0:11364 2014-08-11 16:25:51.874245 2014-08-13 17:38:02.527643 7fe9bae887a0 20 read_log 192888'173 (0'0) modify 8a569feb/benchmark_data_storage0111-ib.pocstor_11336_object11379/head//1 by client.1312877.0:11380 2014-08-11 16:25:52.056754 2014-08-13 17:38:02.527666 7fe9bae887a0 20 read_log 192888'174 (0'0) modify 265609ab/benchmark_data_storage0111-ib.pocstor_11336_object11434/head//1 by client.1312877.0:11435 2014-08-11 16:25:52.547532 2014-08-13 17:38:02.527688 7fe9bae887a0 20 read_log 192888'175 (0'0) modify 2f3985ab/benchmark_data_storage0111-ib.pocstor_11336_object11591/head//1 by client.1312877.0:11592 2014-08-11 16:25:53.158672 2014-08-13 17:38:02.527710 7fe9bae887a0 20 read_log 192888'176 (0'0) modify 2218ab6b/benchmark_data_storage0111-ib.pocstor_11336_object11594/head//1 by client.1312877.0:11595 2014-08-11 16:25:53.170196 2014-08-13 17:38:02.527735 7fe9bae887a0 20 read_log 192888'177 (0'0) modify 780536ab/benchmark_data_storage0111-ib.pocstor_11336_object11678/head//1 by client.1312877.0:11679 2014-08-11 16:25:53.520527 2014-08-13 17:38:02.527766 7fe9bae887a0 20 read_log 192888'178 (0'0) modify a4d98e2b/benchmark_data_storage0111-ib.pocstor_11336_object11711/head//1 by client.1312877.0:11712 2014-08-11 16:25:53.675553 2014-08-13 17:38:02.527836 7fe9bae887a0 20 read_log 192888'179 (0'0) modify 3ae3c7eb/benchmark_data_storage0111-ib.pocstor_11336_object11840/head//1 by client.1312877.0:11841 2014-08-11 16:25:54.146662 2014-08-13 17:38:02.527864 7fe9bae887a0 20 read_log 192888'180 (0'0) modify 28cb2aeb/benchmark_data_storage0111-ib.pocstor_11336_object11864/head//1 by client.1312877.0:11865 2014-08-11 16:25:54.242230 2014-08-13 17:38:02.527886 7fe9bae887a0 20 read_log 192888'181 (0'0) modify a7751f6b/benchmark_data_storage0111-ib.pocstor_11336_object11902/head//1 by client.1312877.0:11903 2014-08-11 16:25:54.454656 2014-08-13 17:38:02.527908 7fe9bae887a0 20 read_log 192888'182 (0'0) modify 4b1d62ab/benchmark_data_storage0111-ib.pocstor_11336_object11920/head//1 by client.1312877.0:11921 2014-08-11 16:25:54.552529 2014-08-13 17:38:02.527929 7fe9bae887a0 20 read_log 192888'183 (0'0) modify 9ef299ab/benchmark_data_storage0111-ib.pocstor_11336_object11921/head//1 by client.1312877.0:11922 2014-08-11 16:25:54.558418 2014-08-13 17:38:02.527951 7fe9bae887a0 20 read_log 192888'184 (0'0) modify 47510d6b/benchmark_data_storage0111-ib.pocstor_11336_object12100/head//1 by client.1312877.0:12101 2014-08-11 16:25:55.404498 2014-08-13 17:38:02.527974 7fe9bae887a0 20 read_log 192888'185 (0'0) modify 69a89f2b/benchmark_data_storage0111-ib.pocstor_11336_object12168/head//1 by client.1312877.0:12169 2014-08-11 16:25:55.783872 2014-08-13 17:38:02.528003 7fe9bae887a0 20 read_log 192888'186 (0'0) modify 6b100eeb/benchmark_data_storage0111-ib.pocstor_11336_object12257/head//1 by client.1312877.0:12258 2014-08-11 16:25:56.099515 2014-08-13 17:38:02.528025 7fe9bae887a0 20 read_log 192888'187 (0'0) modify cdb5142b/benchmark_data_storage0111-ib.pocstor_11336_object12280/head//1 by client.1312877.0:12281 2014-08-11 16:25:56.181331 2014-08-13 17:38:02.528046 7fe9bae887a0 20 read_log 192888'188 (0'0) modify 50af2eeb/benchmark_data_storage0111-ib.pocstor_11336_object12322/head//1 by client.1312877.0:12323 2014-08-11 16:25:56.330605 2014-08-13 17:38:02.528069 7fe9bae887a0 20 read_log 192888'189 (0'0) modify d5d1452b/benchmark_data_storage0111-ib.pocstor_11336_object12326/head//1 by client.1312877.0:12327 2014-08-11 16:25:56.345229 2014-08-13 17:38:02.528091 7fe9bae887a0 20 read_log 192888'190 (0'0) modify a8750ceb/benchmark_data_storage0111-ib.pocstor_11336_object12327/head//1 by client.1312877.0:12328 2014-08-11 16:25:56.348840 2014-08-13 17:38:02.528112 7fe9bae887a0 20 read_log 192888'191 (0'0) modify 6cd4e4eb/benchmark_data_storage0111-ib.pocstor_11336_object12360/head//1 by client.1312877.0:12361 2014-08-11 16:25:56.464192 2014-08-13 17:38:02.528134 7fe9bae887a0 20 read_log 192888'192 (0'0) modify e3f5d66b/benchmark_data_storage0111-ib.pocstor_11336_object12391/head//1 by client.1312877.0:12392 2014-08-11 16:25:56.576819 2014-08-13 17:38:02.528164 7fe9bae887a0 20 read_log 192888'193 (0'0) modify 75ceb6eb/benchmark_data_storage0111-ib.pocstor_11336_object12417/head//1 by client.1312877.0:12418 2014-08-11 16:25:56.669960 2014-08-13 17:38:02.528187 7fe9bae887a0 20 read_log 192888'194 (0'0) modify 8e72486b/benchmark_data_storage0111-ib.pocstor_11336_object12632/head//1 by client.1312877.0:12633 2014-08-11 16:25:57.439549 2014-08-13 17:38:02.528212 7fe9bae887a0 20 read_log 192888'195 (0'0) modify cce08cab/benchmark_data_storage0111-ib.pocstor_11336_object12674/head//1 by client.1312877.0:12675 2014-08-11 16:25:57.604735 2014-08-13 17:38:02.528234 7fe9bae887a0 20 read_log 192888'196 (0'0) modify 3b28fc6b/benchmark_data_storage0111-ib.pocstor_11336_object12837/head//1 by client.1312877.0:12838 2014-08-11 16:25:58.218206 2014-08-13 17:38:02.528255 7fe9bae887a0 20 read_log 192888'197 (0'0) modify d4d8526b/benchmark_data_storage0111-ib.pocstor_11336_object12951/head//1 by client.1312877.0:12952 2014-08-11 16:25:58.735928 2014-08-13 17:38:02.528277 7fe9bae887a0 20 read_log 192888'198 (0'0) modify 1454bcab/benchmark_data_storage0111-ib.pocstor_11336_object12984/head//1 by client.1312877.0:12985 2014-08-11 16:25:58.874452 2014-08-13 17:38:02.528310 7fe9bae887a0 20 read_log 192888'199 (0'0) modify 7ad66aab/benchmark_data_storage0111-ib.pocstor_11336_object13208/head//1 by client.1312877.0:13209 2014-08-11 16:25:59.940374 2014-08-13 17:38:02.528332 7fe9bae887a0 20 read_log 192888'200 (0'0) modify ef9a3c6b/benchmark_data_storage0111-ib.pocstor_11336_object13239/head//1 by client.1312877.0:13240 2014-08-11 16:26:00.105622 2014-08-13 17:38:02.528354 7fe9bae887a0 20 read_log 192888'201 (0'0) modify a87af4eb/benchmark_data_storage0111-ib.pocstor_11336_object13360/head//1 by client.1312877.0:13361 2014-08-11 16:26:00.687417 2014-08-13 17:38:02.528376 7fe9bae887a0 20 read_log 192888'202 (0'0) modify 2d5e4eab/benchmark_data_storage0111-ib.pocstor_11336_object13477/head//1 by client.1312877.0:13478 2014-08-11 16:26:01.317237 2014-08-13 17:38:02.528398 7fe9bae887a0 20 read_log 192888'203 (0'0) modify 6a50cd6b/benchmark_data_storage0111-ib.pocstor_11336_object13497/head//1 by client.1312877.0:13498 2014-08-11 16:26:01.393200 2014-08-13 17:38:02.528419 7fe9bae887a0 20 read_log 192888'204 (0'0) modify 34362b6b/benchmark_data_storage0111-ib.pocstor_11336_object13615/head//1 by client.1312877.0:13616 2014-08-11 16:26:02.038657 2014-08-13 17:38:02.528441 7fe9bae887a0 20 read_log 192888'205 (0'0) modify dfdebe2b/benchmark_data_storage0111-ib.pocstor_11336_object13622/head//1 by client.1312877.0:13623 2014-08-11 16:26:02.075108 2014-08-13 17:38:02.528467 7fe9bae887a0 20 read_log 192888'206 (0'0) modify 4c4339eb/benchmark_data_storage0111-ib.pocstor_11336_object13623/head//1 by client.1312877.0:13624 2014-08-11 16:26:02.080817 2014-08-13 17:38:02.528489 7fe9bae887a0 20 read_log 192888'207 (0'0) modify 898a53ab/benchmark_data_storage0111-ib.pocstor_11336_object14154/head//1 by client.1312877.0:14155 2014-08-11 16:26:04.271930 2014-08-13 17:38:02.528511 7fe9bae887a0 20 read_log 192888'208 (0'0) modify 48a3ff2b/benchmark_data_storage0111-ib.pocstor_11336_object14170/head//1 by client.1312877.0:14171 2014-08-11 16:26:04.328388 2014-08-13 17:38:02.528535 7fe9bae887a0 20 read_log 192888'209 (0'0) modify 6545c6ab/benchmark_data_storage0111-ib.pocstor_11336_object14196/head//1 by client.1312877.0:14197 2014-08-11 16:26:04.416323 2014-08-13 17:38:02.528562 7fe9bae887a0 20 read_log 192888'210 (0'0) modify da0040ab/benchmark_data_storage0111-ib.pocstor_11336_object14203/head//1 by client.1312877.0:14204 2014-08-11 16:26:04.439554 2014-08-13 17:38:02.528596 7fe9bae887a0 20 read_log 192888'211 (0'0) modify 3b3442ab/benchmark_data_storage0111-ib.pocstor_11336_object14209/head//1 by client.1312877.0:14210 2014-08-11 16:26:04.464633 2014-08-13 17:38:02.528635 7fe9bae887a0 20 read_log 192888'212 (0'0) modify 138d602b/benchmark_data_storage0111-ib.pocstor_11336_object14220/head//1 by client.1312877.0:14221 2014-08-11 16:26:04.501975 2014-08-13 17:38:02.528657 7fe9bae887a0 20 read_log 192888'213 (0'0) modify d6709e6b/benchmark_data_storage0111-ib.pocstor_11336_object14223/head//1 by client.1312877.0:14224 2014-08-11 16:26:04.512638 2014-08-13 17:38:02.528678 7fe9bae887a0 20 read_log 192888'214 (0'0) modify fe3dc06b/benchmark_data_storage0111-ib.pocstor_11336_object14364/head//1 by client.1312877.0:14365 2014-08-11 16:26:05.097526 2014-08-13 17:38:02.528700 7fe9bae887a0 20 read_log 192888'215 (0'0) modify 4a1483ab/benchmark_data_storage0111-ib.pocstor_11336_object14418/head//1 by client.1312877.0:14419 2014-08-11 16:26:05.296619 2014-08-13 17:38:02.528722 7fe9bae887a0 20 read_log 192888'216 (0'0) modify 89e9b56b/benchmark_data_storage0111-ib.pocstor_11336_object14424/head//1 by client.1312877.0:14425 2014-08-11 16:26:05.317343 2014-08-13 17:38:02.528744 7fe9bae887a0 20 read_log 192888'217 (0'0) modify d45ab2ab/benchmark_data_storage0111-ib.pocstor_11336_object14441/head//1 by client.1312877.0:14442 2014-08-11 16:26:05.377716 2014-08-13 17:38:02.528765 7fe9bae887a0 20 read_log 192888'218 (0'0) modify 35e2e1eb/benchmark_data_storage0111-ib.pocstor_11336_object14593/head//1 by client.1312877.0:14594 2014-08-11 16:26:05.968657 2014-08-13 17:38:02.528798 7fe9bae887a0 20 read_log 192888'219 (0'0) modify f9cba92b/benchmark_data_storage0111-ib.pocstor_11336_object14598/head//1 by client.1312877.0:14599 2014-08-11 16:26:05.990404 2014-08-13 17:38:02.528825 7fe9bae887a0 20 read_log 192888'220 (0'0) modify 5ddbe7eb/benchmark_data_storage0111-ib.pocstor_11336_object14613/head//1 by client.1312877.0:14614 2014-08-11 16:26:06.049295 2014-08-13 17:38:02.528848 7fe9bae887a0 20 read_log 192888'221 (0'0) modify 44537cab/benchmark_data_storage0111-ib.pocstor_11336_object14743/head//1 by client.1312877.0:14744 2014-08-11 16:26:06.545889 2014-08-13 17:38:02.528870 7fe9bae887a0 20 read_log 192888'222 (0'0) modify 3cd17fab/benchmark_data_storage0111-ib.pocstor_11336_object14794/head//1 by client.1312877.0:14795 2014-08-11 16:26:06.811632 2014-08-13 17:38:02.528891 7fe9bae887a0 20 read_log 192888'223 (0'0) modify fa8a42eb/benchmark_data_storage0111-ib.pocstor_11336_object14952/head//1 by client.1312877.0:14953 2014-08-11 16:26:07.385380 2014-08-13 17:38:02.528913 7fe9bae887a0 20 read_log 192888'224 (0'0) modify 96108eb/benchmark_data_storage0111-ib.pocstor_11336_object15084/head//1 by client.1312877.0:15085 2014-08-11 16:26:07.925003 2014-08-13 17:38:02.528938 7fe9bae887a0 20 read_log 192888'225 (0'0) modify 20b9f32b/benchmark_data_storage0111-ib.pocstor_11336_object15097/head//1 by client.1312877.0:15098 2014-08-11 16:26:07.975770 2014-08-13 17:38:02.528964 7fe9bae887a0 20 read_log 192888'226 (0'0) modify f6085eb/benchmark_data_storage0111-ib.pocstor_11336_object15386/head//1 by client.1312877.0:15387 2014-08-11 16:26:09.437707 2014-08-13 17:38:02.528986 7fe9bae887a0 20 read_log 192888'227 (0'0) modify f1deeceb/benchmark_data_storage0111-ib.pocstor_11336_object15406/head//1 by client.1312877.0:15407 2014-08-11 16:26:09.561403 2014-08-13 17:38:02.529008 7fe9bae887a0 20 read_log 192888'228 (0'0) modify 211e31ab/benchmark_data_storage0111-ib.pocstor_11336_object15650/head//1 by client.1312877.0:15651 2014-08-11 16:26:10.467129 2014-08-13 17:38:02.529029 7fe9bae887a0 20 read_log 192888'229 (0'0) modify 5ea9c7eb/benchmark_data_storage0111-ib.pocstor_11336_object15752/head//1 by client.1312877.0:15753 2014-08-11 16:26:10.830397 2014-08-13 17:38:02.529051 7fe9bae887a0 20 read_log 192888'230 (0'0) modify b8c42aab/benchmark_data_storage0111-ib.pocstor_11336_object15766/head//1 by client.1312877.0:15767 2014-08-11 16:26:10.877610 2014-08-13 17:38:02.529074 7fe9bae887a0 20 read_log 192888'231 (0'0) modify e8b2fbab/benchmark_data_storage0111-ib.pocstor_11336_object15797/head//1 by client.1312877.0:15798 2014-08-11 16:26:11.015944 2014-08-13 17:38:02.529096 7fe9bae887a0 20 read_log 192888'232 (0'0) modify 81b9caab/benchmark_data_storage0111-ib.pocstor_11336_object15801/head//1 by client.1312877.0:15802 2014-08-11 16:26:11.036260 2014-08-13 17:38:02.529119 7fe9bae887a0 20 read_log 192888'233 (0'0) modify 459c8bab/benchmark_data_storage0111-ib.pocstor_11336_object15913/head//1 by client.1312877.0:15914 2014-08-11 16:26:11.533291 2014-08-13 17:38:02.529202 7fe9bae887a0 20 read_log 192888'234 (0'0) modify 8fbd32b/benchmark_data_storage0111-ib.pocstor_11336_object15924/head//1 by client.1312877.0:15925 2014-08-11 16:26:11.585885 2014-08-13 17:38:02.529224 7fe9bae887a0 20 read_log 192888'235 (0'0) modify 709995eb/benchmark_data_storage0111-ib.pocstor_11336_object15934/head//1 by client.1312877.0:15935 2014-08-11 16:26:11.633813 2014-08-13 17:38:02.529247 7fe9bae887a0 20 read_log 192888'236 (0'0) modify 4f2410eb/benchmark_data_storage0111-ib.pocstor_11336_object15947/head//1 by client.1312877.0:15948 2014-08-11 16:26:11.682349 2014-08-13 17:38:02.529273 7fe9bae887a0 20 read_log 192888'237 (0'0) modify 4fadd6ab/benchmark_data_storage0111-ib.pocstor_11336_object16038/head//1 by client.1312877.0:16039 2014-08-11 16:26:12.059985 2014-08-13 17:38:02.529295 7fe9bae887a0 20 read_log 192888'238 (0'0) modify 5b14ab6b/benchmark_data_storage0111-ib.pocstor_11336_object16162/head//1 by client.1312877.0:16163 2014-08-11 16:26:12.982216 2014-08-13 17:38:02.529354 7fe9bae887a0 20 read_log 192888'239 (0'0) modify 6ab0302b/benchmark_data_storage0111-ib.pocstor_11336_object16219/head//1 by client.1312877.0:16220 2014-08-11 16:26:13.216689 2014-08-13 17:38:02.529382 7fe9bae887a0 20 read_log 192888'240 (0'0) modify b76318eb/benchmark_data_storage0111-ib.pocstor_11336_object16290/head//1 by client.1312877.0:16291 2014-08-11 16:26:13.583907 2014-08-13 17:38:02.529405 7fe9bae887a0 20 read_log 192888'241 (0'0) modify 89d8c5eb/benchmark_data_storage0111-ib.pocstor_11336_object16371/head//1 by client.1312877.0:16372 2014-08-11 16:26:13.878059 2014-08-13 17:38:02.529430 7fe9bae887a0 20 read_log 192888'242 (0'0) modify 88c4ec2b/benchmark_data_storage0111-ib.pocstor_11336_object16404/head//1 by client.1312877.0:16405 2014-08-11 16:26:13.992092 2014-08-13 17:38:02.529457 7fe9bae887a0 20 read_log 192888'243 (0'0) modify d2930fab/benchmark_data_storage0111-ib.pocstor_11336_object16420/head//1 by client.1312877.0:16421 2014-08-11 16:26:14.081966 2014-08-13 17:38:02.529483 7fe9bae887a0 20 read_log 192888'244 (0'0) modify 4bcd04eb/benchmark_data_storage0111-ib.pocstor_11336_object16464/head//1 by client.1312877.0:16465 2014-08-11 16:26:14.296561 2014-08-13 17:38:02.529505 7fe9bae887a0 20 read_log 192888'245 (0'0) modify f5411b6b/benchmark_data_storage0111-ib.pocstor_11336_object16539/head//1 by client.1312877.0:16540 2014-08-11 16:26:14.572867 2014-08-13 17:38:02.529530 7fe9bae887a0 20 read_log 192888'246 (0'0) modify 5b8992ab/benchmark_data_storage0111-ib.pocstor_11336_object16582/head//1 by client.1312877.0:16583 2014-08-11 16:26:14.800789 2014-08-13 17:38:02.529552 7fe9bae887a0 20 read_log 192888'247 (0'0) modify 968ceeeb/benchmark_data_storage0111-ib.pocstor_11336_object16597/head//1 by client.1312877.0:16598 2014-08-11 16:26:14.894820 2014-08-13 17:38:02.529577 7fe9bae887a0 20 read_log 192888'248 (0'0) modify 478cf8eb/benchmark_data_storage0111-ib.pocstor_11336_object16708/head//1 by client.1312877.0:16709 2014-08-11 16:26:15.468936 2014-08-13 17:38:02.529599 7fe9bae887a0 20 read_log 192888'249 (0'0) modify e3afee6b/benchmark_data_storage0111-ib.pocstor_11336_object16711/head//1 by client.1312877.0:16712 2014-08-11 16:26:15.484023 2014-08-13 17:38:02.529625 7fe9bae887a0 20 read_log 192888'250 (0'0) modify b42d546b/benchmark_data_storage0111-ib.pocstor_11336_object16737/head//1 by client.1312877.0:16738 2014-08-11 16:26:15.601881 2014-08-13 17:38:02.529653 7fe9bae887a0 20 read_log 192888'251 (0'0) modify 6b39d7ab/benchmark_data_storage0111-ib.pocstor_11336_object16757/head//1 by client.1312877.0:16758 2014-08-11 16:26:15.689243 2014-08-13 17:38:02.529675 7fe9bae887a0 20 read_log 192888'252 (0'0) modify 6c846b6b/benchmark_data_storage0111-ib.pocstor_11336_object16841/head//1 by client.1312877.0:16842 2014-08-11 16:26:16.021770 2014-08-13 17:38:02.529697 7fe9bae887a0 20 read_log 192888'253 (0'0) modify 15deb2eb/benchmark_data_storage0111-ib.pocstor_11336_object16904/head//1 by client.1312877.0:16905 2014-08-11 16:26:16.355920 2014-08-13 17:38:02.529719 7fe9bae887a0 20 read_log 192888'254 (0'0) modify 16e9aaeb/benchmark_data_storage0111-ib.pocstor_11336_object17042/head//1 by client.1312877.0:17043 2014-08-11 16:26:16.843505 2014-08-13 17:38:02.529745 7fe9bae887a0 20 read_log 192888'255 (0'0) modify d0004aeb/benchmark_data_storage0111-ib.pocstor_11336_object17045/head//1 by client.1312877.0:17046 2014-08-11 16:26:16.854420 2014-08-13 17:38:02.529767 7fe9bae887a0 20 read_log 192888'256 (0'0) modify 96728e6b/benchmark_data_storage0111-ib.pocstor_11336_object17143/head//1 by client.1312877.0:17144 2014-08-11 16:26:17.192096 2014-08-13 17:38:02.529793 7fe9bae887a0 20 read_log 192888'257 (0'0) modify f7e3ef2b/benchmark_data_storage0111-ib.pocstor_11336_object17271/head//1 by client.1312877.0:17272 2014-08-11 16:26:17.640454 2014-08-13 17:38:02.529815 7fe9bae887a0 20 read_log 192888'258 (0'0) modify cb867cab/benchmark_data_storage0111-ib.pocstor_11336_object17286/head//1 by client.1312877.0:17287 2014-08-11 16:26:17.692229 2014-08-13 17:38:02.529851 7fe9bae887a0 20 read_log 192888'259 (0'0) modify 286a36b/benchmark_data_storage0111-ib.pocstor_11336_object17353/head//1 by client.1312877.0:17354 2014-08-11 16:26:17.948725 2014-08-13 17:38:02.529877 7fe9bae887a0 20 read_log 192888'260 (0'0) modify 57509cab/benchmark_data_storage0111-ib.pocstor_11336_object17406/head//1 by client.1312877.0:17407 2014-08-11 16:26:18.185142 2014-08-13 17:38:02.529900 7fe9bae887a0 20 read_log 192888'261 (0'0) modify 1978a66b/benchmark_data_storage0111-ib.pocstor_11336_object17504/head//1 by client.1312877.0:17505 2014-08-11 16:26:18.535967 2014-08-13 17:38:02.529921 7fe9bae887a0 20 read_log 192888'262 (0'0) modify 1ff709eb/benchmark_data_storage0111-ib.pocstor_11336_object17536/head//1 by client.1312877.0:17537 2014-08-11 16:26:18.652135 2014-08-13 17:38:02.529943 7fe9bae887a0 20 read_log 192888'263 (0'0) modify 504892ab/benchmark_data_storage0111-ib.pocstor_11336_object17552/head//1 by client.1312877.0:17553 2014-08-11 16:26:18.706436 2014-08-13 17:38:02.529968 7fe9bae887a0 20 read_log 192888'264 (0'0) modify aa8340ab/benchmark_data_storage0111-ib.pocstor_11336_object17582/head//1 by client.1312877.0:17583 2014-08-11 16:26:18.812276 2014-08-13 17:38:02.529991 7fe9bae887a0 20 read_log 192888'265 (0'0) modify b666282b/benchmark_data_storage0111-ib.pocstor_11336_object17624/head//1 by client.1312877.0:17625 2014-08-11 16:26:18.956782 2014-08-13 17:38:02.530017 7fe9bae887a0 20 read_log 192888'266 (0'0) modify 792ade6b/benchmark_data_storage0111-ib.pocstor_11336_object17776/head//1 by client.1312877.0:17777 2014-08-11 16:26:19.572287 2014-08-13 17:38:02.530043 7fe9bae887a0 20 read_log 192888'267 (0'0) modify e4e4e3ab/benchmark_data_storage0111-ib.pocstor_11336_object17791/head//1 by client.1312877.0:17792 2014-08-11 16:26:19.628627 2014-08-13 17:38:02.530072 7fe9bae887a0 20 read_log 192888'268 (0'0) modify e22515eb/benchmark_data_storage0111-ib.pocstor_11336_object17812/head//1 by client.1312877.0:17813 2014-08-11 16:26:19.699954 2014-08-13 17:38:02.530094 7fe9bae887a0 20 read_log 192888'269 (0'0) modify 378d10eb/benchmark_data_storage0111-ib.pocstor_11336_object17841/head//1 by client.1312877.0:17842 2014-08-11 16:26:19.801003 2014-08-13 17:38:02.530119 7fe9bae887a0 20 read_log 192888'270 (0'0) modify a1c1b56b/benchmark_data_storage0111-ib.pocstor_11336_object17879/head//1 by client.1312877.0:17880 2014-08-11 16:26:19.943818 2014-08-13 17:38:02.530141 7fe9bae887a0 20 read_log 192888'271 (0'0) modify 2d29506b/benchmark_data_storage0111-ib.pocstor_11336_object18047/head//1 by client.1312877.0:18048 2014-08-11 16:26:20.739758 2014-08-13 17:38:02.530182 7fe9bae887a0 20 read_log 192888'272 (0'0) modify a954006b/benchmark_data_storage0111-ib.pocstor_11336_object18067/head//1 by client.1312877.0:18068 2014-08-11 16:26:20.923256 2014-08-13 17:38:02.530213 7fe9bae887a0 20 read_log 192888'273 (0'0) modify 3ff41aeb/benchmark_data_storage0111-ib.pocstor_11336_object18232/head//1 by client.1312877.0:18233 2014-08-11 16:26:21.998756 2014-08-13 17:38:02.530237 7fe9bae887a0 20 read_log 192888'274 (0'0) modify 1953402b/benchmark_data_storage0111-ib.pocstor_11336_object18323/head//1 by client.1312877.0:18324 2014-08-11 16:26:22.460038 2014-08-13 17:38:02.530262 7fe9bae887a0 20 read_log 192888'275 (0'0) modify c2fa4d2b/benchmark_data_storage0111-ib.pocstor_11336_object18387/head//1 by client.1312877.0:18388 2014-08-11 16:26:22.691623 2014-08-13 17:38:02.530291 7fe9bae887a0 20 read_log 192888'276 (0'0) modify 9efed92b/benchmark_data_storage0111-ib.pocstor_11336_object18517/head//1 by client.1312877.0:18518 2014-08-11 16:26:23.145076 2014-08-13 17:38:02.530316 7fe9bae887a0 20 read_log 192888'277 (0'0) modify 1846c0eb/benchmark_data_storage0111-ib.pocstor_11336_object18527/head//1 by client.1312877.0:18528 2014-08-11 16:26:23.181015 2014-08-13 17:38:02.530346 7fe9bae887a0 20 read_log 192888'278 (0'0) modify 275089ab/benchmark_data_storage0111-ib.pocstor_11336_object18528/head//1 by client.1312877.0:18529 2014-08-11 16:26:23.183648 2014-08-13 17:38:02.530383 7fe9bae887a0 20 read_log 192888'279 (0'0) modify bcd079eb/benchmark_data_storage0111-ib.pocstor_11336_object18542/head//1 by client.1312877.0:18543 2014-08-11 16:26:23.233281 2014-08-13 17:38:02.530410 7fe9bae887a0 20 read_log 192888'280 (0'0) modify 52868e6b/benchmark_data_storage0111-ib.pocstor_11336_object18592/head//1 by client.1312877.0:18593 2014-08-11 16:26:23.408111 2014-08-13 17:38:02.530432 7fe9bae887a0 20 read_log 192888'281 (0'0) modify 426531eb/benchmark_data_storage0111-ib.pocstor_11336_object18604/head//1 by client.1312877.0:18605 2014-08-11 16:26:23.449747 2014-08-13 17:38:02.530457 7fe9bae887a0 20 read_log 192888'282 (0'0) modify 2aad1c6b/benchmark_data_storage0111-ib.pocstor_11336_object18663/head//1 by client.1312877.0:18664 2014-08-11 16:26:23.661060 2014-08-13 17:38:02.530479 7fe9bae887a0 20 read_log 192888'283 (0'0) modify 5869a02b/benchmark_data_storage0111-ib.pocstor_11336_object18781/head//1 by client.1312877.0:18782 2014-08-11 16:26:24.108098 2014-08-13 17:38:02.530500 7fe9bae887a0 20 read_log 192888'284 (0'0) modify 4187e2eb/benchmark_data_storage0111-ib.pocstor_11336_object18834/head//1 by client.1312877.0:18835 2014-08-11 16:26:24.289663 2014-08-13 17:38:02.530522 7fe9bae887a0 20 read_log 192888'285 (0'0) modify 4f44796b/benchmark_data_storage0111-ib.pocstor_11336_object18875/head//1 by client.1312877.0:18876 2014-08-11 16:26:24.427360 2014-08-13 17:38:02.530543 7fe9bae887a0 20 read_log 192888'286 (0'0) modify fd4b836b/benchmark_data_storage0111-ib.pocstor_11336_object18888/head//1 by client.1312877.0:18889 2014-08-11 16:26:24.473249 2014-08-13 17:38:02.530569 7fe9bae887a0 20 read_log 192888'287 (0'0) modify 2b80b52b/benchmark_data_storage0111-ib.pocstor_11336_object18990/head//1 by client.1312877.0:18991 2014-08-11 16:26:24.828966 2014-08-13 17:38:02.530594 7fe9bae887a0 20 read_log 192888'288 (0'0) modify 1c3083ab/benchmark_data_storage0111-ib.pocstor_11336_object19003/head//1 by client.1312877.0:19004 2014-08-11 16:26:24.875878 2014-08-13 17:38:02.530617 7fe9bae887a0 20 read_log 192888'289 (0'0) modify c2464ceb/benchmark_data_storage0111-ib.pocstor_11336_object19103/head//1 by client.1312877.0:19104 2014-08-11 16:26:25.228126 2014-08-13 17:38:02.530639 7fe9bae887a0 20 read_log 192888'290 (0'0) modify 122a33ab/benchmark_data_storage0111-ib.pocstor_11336_object19227/head//1 by client.1312877.0:19228 2014-08-11 16:26:25.665095 2014-08-13 17:38:02.530668 7fe9bae887a0 20 read_log 192888'291 (0'0) modify e7ba396b/benchmark_data_storage0111-ib.pocstor_11336_object19274/head//1 by client.1312877.0:19275 2014-08-11 16:26:25.858676 2014-08-13 17:38:02.530692 7fe9bae887a0 20 read_log 192888'292 (0'0) modify 9bd5cdeb/benchmark_data_storage0111-ib.pocstor_11336_object19328/head//1 by client.1312877.0:19329 2014-08-11 16:26:26.119132 2014-08-13 17:38:02.530713 7fe9bae887a0 20 read_log 192888'293 (0'0) modify d0eba76b/benchmark_data_storage0111-ib.pocstor_11336_object19418/head//1 by client.1312877.0:19419 2014-08-11 16:26:26.505366 2014-08-13 17:38:02.530735 7fe9bae887a0 20 read_log 192888'294 (0'0) modify 1dc295eb/benchmark_data_storage0111-ib.pocstor_11336_object19457/head//1 by client.1312877.0:19458 2014-08-11 16:26:26.700667 2014-08-13 17:38:02.530757 7fe9bae887a0 20 read_log 192888'295 (0'0) modify 1dd11fab/benchmark_data_storage0111-ib.pocstor_11336_object19469/head//1 by client.1312877.0:19470 2014-08-11 16:26:26.743086 2014-08-13 17:38:02.530778 7fe9bae887a0 20 read_log 192888'296 (0'0) modify c64d172b/benchmark_data_storage0111-ib.pocstor_11336_object19516/head//1 by client.1312877.0:19517 2014-08-11 16:26:26.918807 2014-08-13 17:38:02.530800 7fe9bae887a0 20 read_log 192888'297 (0'0) modify 475791eb/benchmark_data_storage0111-ib.pocstor_11336_object19587/head//1 by client.1312877.0:19588 2014-08-11 16:26:27.171672 2014-08-13 17:38:02.530821 7fe9bae887a0 20 read_log 192888'298 (0'0) modify e2681e6b/benchmark_data_storage0111-ib.pocstor_11336_object19629/head//1 by client.1312877.0:19630 2014-08-11 16:26:27.322787 2014-08-13 17:38:02.530854 7fe9bae887a0 20 read_log 192888'299 (0'0) modify e272fb6b/benchmark_data_storage0111-ib.pocstor_11336_object19824/head//1 by client.1312877.0:19825 2014-08-11 16:26:28.356552 2014-08-13 17:38:02.530878 7fe9bae887a0 20 read_log 192888'300 (0'0) modify 24e902b/benchmark_data_storage0111-ib.pocstor_11336_object19842/head//1 by client.1312877.0:19843 2014-08-11 16:26:28.474235 2014-08-13 17:38:02.530905 7fe9bae887a0 20 read_log 192888'301 (0'0) modify d95a362b/benchmark_data_storage0111-ib.pocstor_11336_object19869/head//1 by client.1312877.0:19870 2014-08-11 16:26:28.733567 2014-08-13 17:38:02.530927 7fe9bae887a0 20 read_log 192888'302 (0'0) modify 46edc8eb/benchmark_data_storage0111-ib.pocstor_11336_object19881/head//1 by client.1312877.0:19882 2014-08-11 16:26:28.894063 2014-08-13 17:38:02.530949 7fe9bae887a0 20 read_log 192888'303 (0'0) modify 17820beb/benchmark_data_storage0111-ib.pocstor_11336_object19886/head//1 by client.1312877.0:19887 2014-08-11 16:26:29.384621 2014-08-13 17:38:02.530973 7fe9bae887a0 20 read_log 192888'304 (0'0) modify 86a8e56b/benchmark_data_storage0111-ib.pocstor_11336_object19954/head//1 by client.1312877.0:19955 2014-08-11 16:26:29.691965 2014-08-13 17:38:02.531001 7fe9bae887a0 20 read_log 192888'305 (0'0) modify 68ddf0ab/benchmark_data_storage0111-ib.pocstor_11336_object19987/head//1 by client.1312877.0:19988 2014-08-11 16:26:29.808390 2014-08-13 17:38:02.531023 7fe9bae887a0 20 read_log 192888'306 (0'0) modify 1fe4d72b/benchmark_data_storage0111-ib.pocstor_11336_object20033/head//1 by client.1312877.0:20034 2014-08-11 16:26:29.971244 2014-08-13 17:38:02.531044 7fe9bae887a0 20 read_log 192888'307 (0'0) modify a91445eb/benchmark_data_storage0111-ib.pocstor_11336_object20056/head//1 by client.1312877.0:20057 2014-08-11 16:26:30.052028 2014-08-13 17:38:02.531066 7fe9bae887a0 20 read_log 192888'308 (0'0) modify 7faf2f6b/benchmark_data_storage0111-ib.pocstor_11336_object20068/head//1 by client.1312877.0:20069 2014-08-11 16:26:30.096129 2014-08-13 17:38:02.531087 7fe9bae887a0 20 read_log 192888'309 (0'0) modify 8b39baab/benchmark_data_storage0111-ib.pocstor_11336_object20099/head//1 by client.1312877.0:20100 2014-08-11 16:26:30.204525 2014-08-13 17:38:02.531108 7fe9bae887a0 20 read_log 192888'310 (0'0) modify f67c52b/benchmark_data_storage0111-ib.pocstor_11336_object20190/head//1 by client.1312877.0:20191 2014-08-11 16:26:30.526991 2014-08-13 17:38:02.531130 7fe9bae887a0 20 read_log 192888'311 (0'0) modify 9b83842b/benchmark_data_storage0111-ib.pocstor_11336_object20253/head//1 by client.1312877.0:20254 2014-08-11 16:26:30.760511 2014-08-13 17:38:02.531151 7fe9bae887a0 20 read_log 192888'312 (0'0) modify 5f94746b/benchmark_data_storage0111-ib.pocstor_11336_object20281/head//1 by client.1312877.0:20282 2014-08-11 16:26:30.863300 2014-08-13 17:38:02.531182 7fe9bae887a0 20 read_log 192888'313 (0'0) modify f708caab/benchmark_data_storage0111-ib.pocstor_11336_object20467/head//1 by client.1312877.0:20468 2014-08-11 16:26:31.851549 2014-08-13 17:38:02.531204 7fe9bae887a0 20 read_log 192888'314 (0'0) modify 5b14d96b/benchmark_data_storage0111-ib.pocstor_11336_object20478/head//1 by client.1312877.0:20479 2014-08-11 16:26:31.980115 2014-08-13 17:38:02.531226 7fe9bae887a0 20 read_log 192888'315 (0'0) modify b820eb2b/benchmark_data_storage0111-ib.pocstor_11336_object20492/head//1 by client.1312877.0:20493 2014-08-11 16:26:32.232457 2014-08-13 17:38:02.531252 7fe9bae887a0 20 read_log 192888'316 (0'0) modify d119eab/benchmark_data_storage0111-ib.pocstor_11336_object20632/head//1 by client.1312877.0:20633 2014-08-11 16:26:32.902642 2014-08-13 17:38:02.531278 7fe9bae887a0 20 read_log 192888'317 (0'0) modify 11cba56b/benchmark_data_storage0111-ib.pocstor_11336_object20947/head//1 by client.1312877.0:20948 2014-08-11 16:26:34.361669 2014-08-13 17:38:02.531304 7fe9bae887a0 20 read_log 192888'318 (0'0) modify 60a1adeb/benchmark_data_storage0111-ib.pocstor_11336_object21180/head//1 by client.1312877.0:21181 2014-08-11 16:26:35.381536 2014-08-13 17:38:02.531370 7fe9bae887a0 20 read_log 192888'319 (0'0) modify 7125f2ab/benchmark_data_storage0111-ib.pocstor_11336_object21256/head//1 by client.1312877.0:21257 2014-08-11 16:26:35.733707 2014-08-13 17:38:02.531403 7fe9bae887a0 20 read_log 192888'320 (0'0) modify 7726eb2b/benchmark_data_storage0111-ib.pocstor_11336_object21372/head//1 by client.1312877.0:21373 2014-08-11 16:26:36.885139 2014-08-13 17:38:02.531426 7fe9bae887a0 20 read_log 192888'321 (0'0) modify 242250ab/benchmark_data_storage0111-ib.pocstor_11336_object21400/head//1 by client.1312877.0:21401 2014-08-11 16:26:36.989791 2014-08-13 17:38:02.531448 7fe9bae887a0 20 read_log 192888'322 (0'0) modify 54fc23ab/benchmark_data_storage0111-ib.pocstor_11336_object21424/head//1 by client.1312877.0:21425 2014-08-11 16:26:37.081022 2014-08-13 17:38:02.531470 7fe9bae887a0 20 read_log 192888'323 (0'0) modify e546a22b/benchmark_data_storage0111-ib.pocstor_11336_object21443/head//1 by client.1312877.0:21444 2014-08-11 16:26:37.175245 2014-08-13 17:38:02.531491 7fe9bae887a0 20 read_log 192888'324 (0'0) modify b69c4a2b/benchmark_data_storage0111-ib.pocstor_11336_object21466/head//1 by client.1312877.0:21467 2014-08-11 16:26:37.297286 2014-08-13 17:38:02.531513 7fe9bae887a0 20 read_log 192888'325 (0'0) modify 5923652b/benchmark_data_storage0111-ib.pocstor_11336_object21528/head//1 by client.1312877.0:21529 2014-08-11 16:26:37.596583 2014-08-13 17:38:02.531535 7fe9bae887a0 20 read_log 192888'326 (0'0) modify da17c26b/benchmark_data_storage0111-ib.pocstor_11336_object21763/head//1 by client.1312877.0:21764 2014-08-11 16:26:38.579171 2014-08-13 17:38:02.531556 7fe9bae887a0 20 read_log 192888'327 (0'0) modify 87148eb/benchmark_data_storage0111-ib.pocstor_11336_object21812/head//1 by client.1312877.0:21813 2014-08-11 16:26:38.780873 2014-08-13 17:38:02.531578 7fe9bae887a0 20 read_log 192888'328 (0'0) modify 9101e96b/benchmark_data_storage0111-ib.pocstor_11336_object21868/head//1 by client.1312877.0:21869 2014-08-11 16:26:38.994835 2014-08-13 17:38:02.531599 7fe9bae887a0 20 read_log 192888'329 (0'0) modify 7c4fe4ab/benchmark_data_storage0111-ib.pocstor_11336_object21885/head//1 by client.1312877.0:21886 2014-08-11 16:26:39.079840 2014-08-13 17:38:02.531621 7fe9bae887a0 20 read_log 192888'330 (0'0) modify ad24d8ab/benchmark_data_storage0111-ib.pocstor_11336_object21935/head//1 by client.1312877.0:21936 2014-08-11 16:26:39.313238 2014-08-13 17:38:02.531642 7fe9bae887a0 20 read_log 192888'331 (0'0) modify 787deceb/benchmark_data_storage0111-ib.pocstor_11336_object21956/head//1 by client.1312877.0:21957 2014-08-11 16:26:39.391347 2014-08-13 17:38:02.531698 7fe9bae887a0 20 read_log 192888'332 (0'0) modify f978002b/benchmark_data_storage0111-ib.pocstor_11336_object21997/head//1 by client.1312877.0:21998 2014-08-11 16:26:39.547116 2014-08-13 17:38:02.531726 7fe9bae887a0 20 read_log 192888'333 (0'0) modify 959dd26b/benchmark_data_storage0111-ib.pocstor_11336_object22000/head//1 by client.1312877.0:22001 2014-08-11 16:26:39.558471 2014-08-13 17:38:02.531748 7fe9bae887a0 20 read_log 192888'334 (0'0) modify 4e472ceb/benchmark_data_storage0111-ib.pocstor_11336_object22034/head//1 by client.1312877.0:22035 2014-08-11 16:26:39.687552 2014-08-13 17:38:02.531770 7fe9bae887a0 20 read_log 192888'335 (0'0) modify 6c1ac06b/benchmark_data_storage0111-ib.pocstor_11336_object22051/head//1 by client.1312877.0:22052 2014-08-11 16:26:39.751101 2014-08-13 17:38:02.531791 7fe9bae887a0 20 read_log 192888'336 (0'0) modify 373a052b/benchmark_data_storage0111-ib.pocstor_11336_object22091/head//1 by client.1312877.0:22092 2014-08-11 16:26:39.900417 2014-08-13 17:38:02.531813 7fe9bae887a0 20 read_log 192888'337 (0'0) modify 31cfc02b/benchmark_data_storage0111-ib.pocstor_11336_object22258/head//1 by client.1312877.0:22259 2014-08-11 16:26:40.598516 2014-08-13 17:38:02.531835 7fe9bae887a0 20 read_log 192888'338 (0'0) modify 6de7eaab/benchmark_data_storage0111-ib.pocstor_11336_object22272/head//1 by client.1312877.0:22273 2014-08-11 16:26:40.663523 2014-08-13 17:38:02.531875 7fe9bae887a0 20 read_log 192888'339 (0'0) modify 1781c26b/benchmark_data_storage0111-ib.pocstor_11336_object22296/head//1 by client.1312877.0:22297 2014-08-11 16:26:40.758480 2014-08-13 17:38:02.531903 7fe9bae887a0 20 read_log 192888'340 (0'0) modify f157deab/benchmark_data_storage0111-ib.pocstor_11336_object22346/head//1 by client.1312877.0:22347 2014-08-11 16:26:40.955046 2014-08-13 17:38:02.531925 7fe9bae887a0 20 read_log 192888'341 (0'0) modify f68908eb/benchmark_data_storage0111-ib.pocstor_11336_object22441/head//1 by client.1312877.0:22442 2014-08-11 16:26:41.304878 2014-08-13 17:38:02.531947 7fe9bae887a0 20 read_log 192888'342 (0'0) modify a20011ab/benchmark_data_storage0111-ib.pocstor_11336_object22467/head//1 by client.1312877.0:22468 2014-08-11 16:26:41.397941 2014-08-13 17:38:02.531968 7fe9bae887a0 20 read_log 192888'343 (0'0) modify 82490b2b/benchmark_data_storage0111-ib.pocstor_11336_object22469/head//1 by client.1312877.0:22470 2014-08-11 16:26:41.405354 2014-08-13 17:38:02.531989 7fe9bae887a0 20 read_log 192888'344 (0'0) modify 520cf2eb/benchmark_data_storage0111-ib.pocstor_11336_object22473/head//1 by client.1312877.0:22474 2014-08-11 16:26:41.418601 2014-08-13 17:38:02.532013 7fe9bae887a0 20 read_log 192888'345 (0'0) modify 42e912ab/benchmark_data_storage0111-ib.pocstor_11336_object22545/head//1 by client.1312877.0:22546 2014-08-11 16:26:41.803872 2014-08-13 17:38:02.532039 7fe9bae887a0 20 read_log 192888'346 (0'0) modify 96bbf22b/benchmark_data_storage0111-ib.pocstor_11336_object22579/head//1 by client.1312877.0:22580 2014-08-11 16:26:41.923292 2014-08-13 17:38:02.532062 7fe9bae887a0 20 read_log 192888'347 (0'0) modify a0f15c6b/benchmark_data_storage0111-ib.pocstor_11336_object22587/head//1 by client.1312877.0:22588 2014-08-11 16:26:41.956565 2014-08-13 17:38:02.532088 7fe9bae887a0 20 read_log 192888'348 (0'0) modify 698c546b/benchmark_data_storage0111-ib.pocstor_11336_object22636/head//1 by client.1312877.0:22637 2014-08-11 16:26:42.159265 2014-08-13 17:38:02.532109 7fe9bae887a0 20 read_log 192888'349 (0'0) modify a9a305eb/benchmark_data_storage0111-ib.pocstor_11336_object22716/head//1 by client.1312877.0:22717 2014-08-11 16:26:42.550061 2014-08-13 17:38:02.532135 7fe9bae887a0 20 read_log 192888'350 (0'0) modify 20cdd2b/benchmark_data_storage0111-ib.pocstor_11336_object22738/head//1 by client.1312877.0:22739 2014-08-11 16:26:42.642179 2014-08-13 17:38:02.532170 7fe9bae887a0 20 read_log 192888'351 (0'0) modify 13e43f2b/benchmark_data_storage0111-ib.pocstor_11336_object22869/head//1 by client.1312877.0:22870 2014-08-11 16:26:43.345894 2014-08-13 17:38:02.532196 7fe9bae887a0 20 read_log 192888'352 (0'0) modify efa1d42b/benchmark_data_storage0111-ib.pocstor_11336_object22914/head//1 by client.1312877.0:22915 2014-08-11 16:26:43.690619 2014-08-13 17:38:02.532218 7fe9bae887a0 20 read_log 192888'353 (0'0) modify 96aeefeb/benchmark_data_storage0111-ib.pocstor_11336_object22932/head//1 by client.1312877.0:22933 2014-08-11 16:26:43.849886 2014-08-13 17:38:02.532240 7fe9bae887a0 20 read_log 192888'354 (0'0) modify 1a986b6b/benchmark_data_storage0111-ib.pocstor_11336_object22937/head//1 by client.1312877.0:22938 2014-08-11 16:26:43.880915 2014-08-13 17:38:02.532262 7fe9bae887a0 20 read_log 192888'355 (0'0) modify a0706d2b/benchmark_data_storage0111-ib.pocstor_11336_object22960/head//1 by client.1312877.0:22961 2014-08-11 16:26:44.122557 2014-08-13 17:38:02.532286 7fe9bae887a0 20 read_log 192888'356 (0'0) modify 8e2579ab/benchmark_data_storage0111-ib.pocstor_11336_object22962/head//1 by client.1312877.0:22963 2014-08-11 16:26:44.141679 2014-08-13 17:38:02.532307 7fe9bae887a0 20 read_log 192888'357 (0'0) modify 7ebd15eb/benchmark_data_storage0111-ib.pocstor_11336_object23023/head//1 by client.1312877.0:23024 2014-08-11 16:26:44.487575 2014-08-13 17:38:02.532333 7fe9bae887a0 20 read_log 192888'358 (0'0) modify 53266aeb/benchmark_data_storage0111-ib.pocstor_11336_object23033/head//1 by client.1312877.0:23034 2014-08-11 16:26:44.525347 2014-08-13 17:38:02.532366 7fe9bae887a0 20 read_log 192888'359 (0'0) modify 45b50ab/benchmark_data_storage0111-ib.pocstor_11336_object23056/head//1 by client.1312877.0:23057 2014-08-11 16:26:44.603328 2014-08-13 17:38:02.532393 7fe9bae887a0 20 read_log 192888'360 (0'0) modify 686380ab/benchmark_data_storage0111-ib.pocstor_11336_object23170/head//1 by client.1312877.0:23171 2014-08-11 16:26:45.014761 2014-08-13 17:38:02.532415 7fe9bae887a0 20 read_log 192888'361 (0'0) modify a62bfc6b/benchmark_data_storage0111-ib.pocstor_11336_object23205/head//1 by client.1312877.0:23206 2014-08-11 16:26:45.167732 2014-08-13 17:38:02.532437 7fe9bae887a0 20 read_log 192888'362 (0'0) modify b6784d6b/benchmark_data_storage0111-ib.pocstor_11336_object23214/head//1 by client.1312877.0:23215 2014-08-11 16:26:45.202423 2014-08-13 17:38:02.532459 7fe9bae887a0 20 read_log 192888'363 (0'0) modify e4717fab/benchmark_data_storage0111-ib.pocstor_11336_object23363/head//1 by client.1312877.0:23364 2014-08-11 16:26:45.763882 2014-08-13 17:38:02.532481 7fe9bae887a0 20 read_log 192888'364 (0'0) modify 793e702b/benchmark_data_storage0111-ib.pocstor_11336_object23422/head//1 by client.1312877.0:23423 2014-08-11 16:26:45.976633 2014-08-13 17:38:02.532502 7fe9bae887a0 20 read_log 192888'365 (0'0) modify 90f044eb/benchmark_data_storage0111-ib.pocstor_11336_object23495/head//1 by client.1312877.0:23496 2014-08-11 16:26:46.326997 2014-08-13 17:38:02.532523 7fe9bae887a0 20 read_log 192888'366 (0'0) modify 30bf61ab/benchmark_data_storage0111-ib.pocstor_11336_object23554/head//1 by client.1312877.0:23555 2014-08-11 16:26:46.693405 2014-08-13 17:38:02.532545 7fe9bae887a0 20 read_log 192888'367 (0'0) modify c494546b/benchmark_data_storage0111-ib.pocstor_11336_object23580/head//1 by client.1312877.0:23581 2014-08-11 16:26:46.880615 2014-08-13 17:38:02.532566 7fe9bae887a0 20 read_log 192888'368 (0'0) modify 60e4f62b/benchmark_data_storage0111-ib.pocstor_11336_object23637/head//1 by client.1312877.0:23638 2014-08-11 16:26:47.341780 2014-08-13 17:38:02.532588 7fe9bae887a0 20 read_log 192888'369 (0'0) modify 4c4bf4ab/benchmark_data_storage0111-ib.pocstor_11336_object23654/head//1 by client.1312877.0:23655 2014-08-11 16:26:47.403860 2014-08-13 17:38:02.532609 7fe9bae887a0 20 read_log 192888'370 (0'0) modify a946212b/benchmark_data_storage0111-ib.pocstor_11336_object23753/head//1 by client.1312877.0:23754 2014-08-11 16:26:47.833072 2014-08-13 17:38:02.532630 7fe9bae887a0 20 read_log 192888'371 (0'0) modify a1fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object23786/head//1 by client.1312877.0:23787 2014-08-11 16:26:47.966950 2014-08-13 17:38:02.532652 7fe9bae887a0 20 read_log 192888'372 (0'0) modify b45acb2b/benchmark_data_storage0111-ib.pocstor_11336_object23815/head//1 by client.1312877.0:23816 2014-08-11 16:26:48.088858 2014-08-13 17:38:02.532673 7fe9bae887a0 20 read_log 192888'373 (0'0) modify f2e080eb/benchmark_data_storage0111-ib.pocstor_11336_object23867/head//1 by client.1312877.0:23868 2014-08-11 16:26:48.418532 2014-08-13 17:38:02.532699 7fe9bae887a0 20 read_log 192888'374 (0'0) modify a083866b/benchmark_data_storage0111-ib.pocstor_11336_object23921/head//1 by client.1312877.0:23922 2014-08-11 16:26:48.844292 2014-08-13 17:38:02.532721 7fe9bae887a0 20 read_log 192888'375 (0'0) modify 8bbc9eeb/benchmark_data_storage0111-ib.pocstor_11336_object23984/head//1 by client.1312877.0:23985 2014-08-11 16:26:49.142340 2014-08-13 17:38:02.532742 7fe9bae887a0 20 read_log 192888'376 (0'0) modify e7d972b/benchmark_data_storage0111-ib.pocstor_11336_object24002/head//1 by client.1312877.0:24003 2014-08-11 16:26:49.206748 2014-08-13 17:38:02.532763 7fe9bae887a0 20 read_log 192888'377 (0'0) modify a7b66c6b/benchmark_data_storage0111-ib.pocstor_11336_object24011/head//1 by client.1312877.0:24012 2014-08-11 16:26:49.238714 2014-08-13 17:38:02.532785 7fe9bae887a0 20 read_log 192888'378 (0'0) modify 8d09746b/benchmark_data_storage0111-ib.pocstor_11336_object24032/head//1 by client.1312877.0:24033 2014-08-11 16:26:49.328222 2014-08-13 17:38:02.532817 7fe9bae887a0 20 read_log 192888'379 (0'0) modify f96e4feb/benchmark_data_storage0111-ib.pocstor_11336_object24132/head//1 by client.1312877.0:24133 2014-08-11 16:26:49.772456 2014-08-13 17:38:02.532840 7fe9bae887a0 20 read_log 192888'380 (0'0) modify 12d73bab/benchmark_data_storage0111-ib.pocstor_11336_object24218/head//1 by client.1312877.0:24219 2014-08-11 16:26:50.086561 2014-08-13 17:38:02.532861 7fe9bae887a0 20 read_log 192888'381 (0'0) modify 9866d8eb/benchmark_data_storage0111-ib.pocstor_11336_object24269/head//1 by client.1312877.0:24270 2014-08-11 16:26:50.351521 2014-08-13 17:38:02.532883 7fe9bae887a0 20 read_log 192888'382 (0'0) modify de90c7eb/benchmark_data_storage0111-ib.pocstor_11336_object24341/head//1 by client.1312877.0:24342 2014-08-11 16:26:50.788055 2014-08-13 17:38:02.532904 7fe9bae887a0 20 read_log 192888'383 (0'0) modify 537521eb/benchmark_data_storage0111-ib.pocstor_11336_object24410/head//1 by client.1312877.0:24411 2014-08-11 16:26:51.184347 2014-08-13 17:38:02.532925 7fe9bae887a0 20 read_log 192888'384 (0'0) modify e6d1e6ab/benchmark_data_storage0111-ib.pocstor_11336_object24502/head//1 by client.1312877.0:24503 2014-08-11 16:26:51.807228 2014-08-13 17:38:02.532949 7fe9bae887a0 20 read_log 192888'385 (0'0) modify 570d676b/benchmark_data_storage0111-ib.pocstor_11336_object24513/head//1 by client.1312877.0:24514 2014-08-11 16:26:51.846662 2014-08-13 17:38:02.532971 7fe9bae887a0 20 read_log 192888'386 (0'0) modify 4d3698ab/benchmark_data_storage0111-ib.pocstor_11336_object24640/head//1 by client.1312877.0:24641 2014-08-11 16:26:52.400721 2014-08-13 17:38:02.532992 7fe9bae887a0 20 read_log 192888'387 (0'0) modify 6fd4ceb/benchmark_data_storage0111-ib.pocstor_11336_object24678/head//1 by client.1312877.0:24679 2014-08-11 16:26:52.584560 2014-08-13 17:38:02.533015 7fe9bae887a0 20 read_log 192888'388 (0'0) modify dda641eb/benchmark_data_storage0111-ib.pocstor_11336_object24696/head//1 by client.1312877.0:24697 2014-08-11 16:26:52.678130 2014-08-13 17:38:02.533036 7fe9bae887a0 20 read_log 192888'389 (0'0) modify 3edb27eb/benchmark_data_storage0111-ib.pocstor_11336_object24704/head//1 by client.1312877.0:24705 2014-08-11 16:26:52.720373 2014-08-13 17:38:02.533058 7fe9bae887a0 20 read_log 192888'390 (0'0) modify c80d832b/benchmark_data_storage0111-ib.pocstor_11336_object24729/head//1 by client.1312877.0:24730 2014-08-11 16:26:52.821854 2014-08-13 17:38:02.533117 7fe9bae887a0 20 read_log 192888'391 (0'0) modify 7fdffdeb/benchmark_data_storage0111-ib.pocstor_11336_object24731/head//1 by client.1312877.0:24732 2014-08-11 16:26:52.829238 2014-08-13 17:38:02.533139 7fe9bae887a0 20 read_log 192888'392 (0'0) modify fbb74cab/benchmark_data_storage0111-ib.pocstor_11336_object24889/head//1 by client.1312877.0:24890 2014-08-11 16:26:53.602759 2014-08-13 17:38:02.533174 7fe9bae887a0 20 read_log 192888'393 (0'0) modify abc784eb/benchmark_data_storage0111-ib.pocstor_11336_object24918/head//1 by client.1312877.0:24919 2014-08-11 16:26:53.755724 2014-08-13 17:38:02.533197 7fe9bae887a0 20 read_log 192888'394 (0'0) modify aa96b5eb/benchmark_data_storage0111-ib.pocstor_11336_object25001/head//1 by client.1312877.0:25002 2014-08-11 16:26:54.064487 2014-08-13 17:38:02.533221 7fe9bae887a0 20 read_log 192888'395 (0'0) modify f622b72b/benchmark_data_storage0111-ib.pocstor_11336_object25080/head//1 by client.1312877.0:25081 2014-08-11 16:26:54.352433 2014-08-13 17:38:02.533243 7fe9bae887a0 20 read_log 192888'396 (0'0) modify 42235aeb/benchmark_data_storage0111-ib.pocstor_11336_object25120/head//1 by client.1312877.0:25121 2014-08-11 16:26:54.493495 2014-08-13 17:38:02.533264 7fe9bae887a0 20 read_log 192888'397 (0'0) modify 965b2cab/benchmark_data_storage0111-ib.pocstor_11336_object25176/head//1 by client.1312877.0:25177 2014-08-11 16:26:54.691821 2014-08-13 17:38:02.533286 7fe9bae887a0 20 read_log 192888'398 (0'0) modify 802de92b/benchmark_data_storage0111-ib.pocstor_11336_object25189/head//1 by client.1312877.0:25190 2014-08-11 16:26:54.737328 2014-08-13 17:38:02.533319 7fe9bae887a0 20 read_log 192888'399 (0'0) modify 9409426b/benchmark_data_storage0111-ib.pocstor_11336_object25191/head//1 by client.1312877.0:25192 2014-08-11 16:26:54.744548 2014-08-13 17:38:02.533346 7fe9bae887a0 20 read_log 192888'400 (0'0) modify c26e8ceb/benchmark_data_storage0111-ib.pocstor_11336_object25320/head//1 by client.1312877.0:25321 2014-08-11 16:26:55.410502 2014-08-13 17:38:02.533368 7fe9bae887a0 20 read_log 192888'401 (0'0) modify 5bd198eb/benchmark_data_storage0111-ib.pocstor_11336_object25342/head//1 by client.1312877.0:25343 2014-08-11 16:26:55.488592 2014-08-13 17:38:02.533390 7fe9bae887a0 20 read_log 192888'402 (0'0) modify f2bae72b/benchmark_data_storage0111-ib.pocstor_11336_object25411/head//1 by client.1312877.0:25412 2014-08-11 16:26:55.732698 2014-08-13 17:38:02.533411 7fe9bae887a0 20 read_log 192888'403 (0'0) modify 22b339ab/benchmark_data_storage0111-ib.pocstor_11336_object25485/head//1 by client.1312877.0:25486 2014-08-11 16:26:55.993717 2014-08-13 17:38:02.533432 7fe9bae887a0 20 read_log 192888'404 (0'0) modify 4e638a6b/benchmark_data_storage0111-ib.pocstor_11336_object25526/head//1 by client.1312877.0:25527 2014-08-11 16:26:56.202836 2014-08-13 17:38:02.533453 7fe9bae887a0 20 read_log 192888'405 (0'0) modify 1129b96b/benchmark_data_storage0111-ib.pocstor_11336_object25572/head//1 by client.1312877.0:25573 2014-08-11 16:26:56.395695 2014-08-13 17:38:02.533475 7fe9bae887a0 20 read_log 192888'406 (0'0) modify 934612eb/benchmark_data_storage0111-ib.pocstor_11336_object25811/head//1 by client.1312877.0:25812 2014-08-11 16:26:57.761164 2014-08-13 17:38:02.533496 7fe9bae887a0 20 read_log 192888'407 (0'0) modify 8a0e956b/benchmark_data_storage0111-ib.pocstor_11336_object25812/head//1 by client.1312877.0:25813 2014-08-11 16:26:57.764724 2014-08-13 17:38:02.533517 7fe9bae887a0 20 read_log 192888'408 (0'0) modify 5275656b/benchmark_data_storage0111-ib.pocstor_11336_object25868/head//1 by client.1312877.0:25869 2014-08-11 16:26:57.962180 2014-08-13 17:38:02.533538 7fe9bae887a0 20 read_log 192888'409 (0'0) modify 75b001eb/benchmark_data_storage0111-ib.pocstor_11336_object26161/head//1 by client.1312877.0:26162 2014-08-11 16:26:59.376527 2014-08-13 17:38:02.533560 7fe9bae887a0 20 read_log 192888'410 (0'0) modify b69ac26b/benchmark_data_storage0111-ib.pocstor_11336_object26351/head//1 by client.1312877.0:26352 2014-08-11 16:27:00.037628 2014-08-13 17:38:02.533582 7fe9bae887a0 20 read_log 192888'411 (0'0) modify 784d40ab/benchmark_data_storage0111-ib.pocstor_11336_object26478/head//1 by client.1312877.0:26479 2014-08-11 16:27:00.596340 2014-08-13 17:38:02.533603 7fe9bae887a0 20 read_log 192888'412 (0'0) modify b4b81d2b/benchmark_data_storage0111-ib.pocstor_11336_object26538/head//1 by client.1312877.0:26539 2014-08-11 16:27:00.805272 2014-08-13 17:38:02.533624 7fe9bae887a0 20 read_log 192888'413 (0'0) modify 940ab0ab/benchmark_data_storage0111-ib.pocstor_11336_object26596/head//1 by client.1312877.0:26597 2014-08-11 16:27:01.010879 2014-08-13 17:38:02.533648 7fe9bae887a0 20 read_log 192888'414 (0'0) modify f7f71f6b/benchmark_data_storage0111-ib.pocstor_11336_object26678/head//1 by client.1312877.0:26679 2014-08-11 16:27:01.300260 2014-08-13 17:38:02.533670 7fe9bae887a0 20 read_log 192888'415 (0'0) modify 2b79936b/benchmark_data_storage0111-ib.pocstor_11336_object26822/head//1 by client.1312877.0:26823 2014-08-11 16:27:01.800044 2014-08-13 17:38:02.533692 7fe9bae887a0 20 read_log 192888'416 (0'0) modify 9d858b2b/benchmark_data_storage0111-ib.pocstor_11336_object26873/head//1 by client.1312877.0:26874 2014-08-11 16:27:01.977353 2014-08-13 17:38:02.533715 7fe9bae887a0 20 read_log 192888'417 (0'0) modify d46b0eab/benchmark_data_storage0111-ib.pocstor_11336_object26900/head//1 by client.1312877.0:26901 2014-08-11 16:27:02.070385 2014-08-13 17:38:02.533736 7fe9bae887a0 20 read_log 192888'418 (0'0) modify 3596656b/benchmark_data_storage0111-ib.pocstor_11336_object27149/head//1 by client.1312877.0:27150 2014-08-11 16:27:03.049271 2014-08-13 17:38:02.533772 7fe9bae887a0 20 read_log 192888'419 (0'0) modify 800c4c2b/benchmark_data_storage0111-ib.pocstor_11336_object27214/head//1 by client.1312877.0:27215 2014-08-11 16:27:03.570119 2014-08-13 17:38:02.533794 7fe9bae887a0 20 read_log 192888'420 (192888'1) delete 968c7c2b/benchmark_data_storage0111-ib.pocstor_11336_object26/head//1 by client.1312877.0:27310 2014-08-11 16:27:05.039769 2014-08-13 17:38:02.533821 7fe9bae887a0 20 read_log 192888'421 (192888'2) delete 458d82b/benchmark_data_storage0111-ib.pocstor_11336_object85/head//1 by client.1312877.0:27369 2014-08-11 16:27:05.065866 2014-08-13 17:38:02.533844 7fe9bae887a0 20 read_log 192888'422 (192888'3) delete 8d3a1cab/benchmark_data_storage0111-ib.pocstor_11336_object115/head//1 by client.1312877.0:27399 2014-08-11 16:27:05.115545 2014-08-13 17:38:02.533867 7fe9bae887a0 20 read_log 192888'423 (192888'4) delete 174a7a6b/benchmark_data_storage0111-ib.pocstor_11336_object120/head//1 by client.1312877.0:27404 2014-08-11 16:27:05.130440 2014-08-13 17:38:02.533889 7fe9bae887a0 20 read_log 192888'424 (192888'5) delete 709b852b/benchmark_data_storage0111-ib.pocstor_11336_object514/head//1 by client.1312877.0:27798 2014-08-11 16:27:05.352177 2014-08-13 17:38:02.533943 7fe9bae887a0 20 read_log 192888'425 (192888'6) delete f9c4ff6b/benchmark_data_storage0111-ib.pocstor_11336_object529/head//1 by client.1312877.0:27813 2014-08-11 16:27:05.368026 2014-08-13 17:38:02.533972 7fe9bae887a0 20 read_log 192888'426 (192888'7) delete 5c599e6b/benchmark_data_storage0111-ib.pocstor_11336_object584/head//1 by client.1312877.0:27868 2014-08-11 16:27:05.457356 2014-08-13 17:38:02.533995 7fe9bae887a0 20 read_log 192888'427 (192888'8) delete cc5ad2ab/benchmark_data_storage0111-ib.pocstor_11336_object589/head//1 by client.1312877.0:27873 2014-08-11 16:27:05.460574 2014-08-13 17:38:02.534017 7fe9bae887a0 20 read_log 192888'428 (192888'9) delete e2cda9ab/benchmark_data_storage0111-ib.pocstor_11336_object680/head//1 by client.1312877.0:27964 2014-08-11 16:27:05.529342 2014-08-13 17:38:02.534039 7fe9bae887a0 20 read_log 192888'429 (192888'10) delete 6a2d76b/benchmark_data_storage0111-ib.pocstor_11336_object727/head//1 by client.1312877.0:28011 2014-08-11 16:27:05.584039 2014-08-13 17:38:02.534061 7fe9bae887a0 20 read_log 192888'430 (192888'11) delete d4542b6b/benchmark_data_storage0111-ib.pocstor_11336_object785/head//1 by client.1312877.0:28069 2014-08-11 16:27:05.677339 2014-08-13 17:38:02.534083 7fe9bae887a0 20 read_log 192888'431 (192888'12) delete 13924b6b/benchmark_data_storage0111-ib.pocstor_11336_object893/head//1 by client.1312877.0:28177 2014-08-11 16:27:05.734545 2014-08-13 17:38:02.534105 7fe9bae887a0 20 read_log 192888'432 (192888'13) delete 2df8bfab/benchmark_data_storage0111-ib.pocstor_11336_object920/head//1 by client.1312877.0:28204 2014-08-11 16:27:05.750387 2014-08-13 17:38:02.534126 7fe9bae887a0 20 read_log 192888'433 (192888'14) delete 1142b5eb/benchmark_data_storage0111-ib.pocstor_11336_object942/head//1 by client.1312877.0:28226 2014-08-11 16:27:05.771329 2014-08-13 17:38:02.534167 7fe9bae887a0 20 read_log 192888'434 (192888'15) delete afd4156b/benchmark_data_storage0111-ib.pocstor_11336_object1007/head//1 by client.1312877.0:28291 2014-08-11 16:27:06.082181 2014-08-13 17:38:02.534206 7fe9bae887a0 20 read_log 192888'435 (192888'16) delete 1bd77ceb/benchmark_data_storage0111-ib.pocstor_11336_object1053/head//1 by client.1312877.0:28337 2014-08-11 16:27:06.103443 2014-08-13 17:38:02.534230 7fe9bae887a0 20 read_log 192888'436 (192888'17) delete 4c3c3eab/benchmark_data_storage0111-ib.pocstor_11336_object1107/head//1 by client.1312877.0:28391 2014-08-11 16:27:06.129095 2014-08-13 17:38:02.534252 7fe9bae887a0 20 read_log 192888'437 (192888'18) delete 84e0c2b/benchmark_data_storage0111-ib.pocstor_11336_object1197/head//1 by client.1312877.0:28481 2014-08-11 16:27:06.205483 2014-08-13 17:38:02.534274 7fe9bae887a0 20 read_log 192888'438 (192888'19) delete 5acc686b/benchmark_data_storage0111-ib.pocstor_11336_object1402/head//1 by client.1312877.0:28686 2014-08-11 16:27:06.372340 2014-08-13 17:38:02.534309 7fe9bae887a0 20 read_log 192888'439 (192888'20) delete cd6caab/benchmark_data_storage0111-ib.pocstor_11336_object1404/head//1 by client.1312877.0:28688 2014-08-11 16:27:06.374471 2014-08-13 17:38:02.534336 7fe9bae887a0 20 read_log 192888'440 (192888'21) delete f337eceb/benchmark_data_storage0111-ib.pocstor_11336_object1448/head//1 by client.1312877.0:28732 2014-08-11 16:27:06.390419 2014-08-13 17:38:02.534359 7fe9bae887a0 20 read_log 192888'441 (192888'22) delete fdf8f3eb/benchmark_data_storage0111-ib.pocstor_11336_object1668/head//1 by client.1312877.0:28952 2014-08-11 16:27:06.466848 2014-08-13 17:38:02.534381 7fe9bae887a0 20 read_log 192888'442 (192888'23) delete 784e352b/benchmark_data_storage0111-ib.pocstor_11336_object1682/head//1 by client.1312877.0:28966 2014-08-11 16:27:06.472175 2014-08-13 17:38:02.534403 7fe9bae887a0 20 read_log 192888'443 (192888'24) delete dda332b/benchmark_data_storage0111-ib.pocstor_11336_object1930/head//1 by client.1312877.0:29214 2014-08-11 16:27:06.539051 2014-08-13 17:38:02.534427 7fe9bae887a0 20 read_log 192888'444 (192888'25) delete 93f31b2b/benchmark_data_storage0111-ib.pocstor_11336_object2038/head//1 by client.1312877.0:29322 2014-08-11 16:27:06.564439 2014-08-13 17:38:02.534450 7fe9bae887a0 20 read_log 192888'445 (192888'26) delete 357eddeb/benchmark_data_storage0111-ib.pocstor_11336_object2073/head//1 by client.1312877.0:29357 2014-08-11 16:27:06.574050 2014-08-13 17:38:02.534472 7fe9bae887a0 20 read_log 192888'446 (192888'27) delete 27ad09eb/benchmark_data_storage0111-ib.pocstor_11336_object2157/head//1 by client.1312877.0:29441 2014-08-11 16:27:06.608517 2014-08-13 17:38:02.534495 7fe9bae887a0 20 read_log 192888'447 (192888'28) delete 78fc9beb/benchmark_data_storage0111-ib.pocstor_11336_object2224/head//1 by client.1312877.0:29508 2014-08-11 16:27:06.671425 2014-08-13 17:38:02.534521 7fe9bae887a0 20 read_log 192888'448 (192888'29) delete 4721d42b/benchmark_data_storage0111-ib.pocstor_11336_object2247/head//1 by client.1312877.0:29531 2014-08-11 16:27:06.679764 2014-08-13 17:38:02.534543 7fe9bae887a0 20 read_log 192888'449 (192888'30) delete d935726b/benchmark_data_storage0111-ib.pocstor_11336_object2299/head//1 by client.1312877.0:29583 2014-08-11 16:27:06.701980 2014-08-13 17:38:02.534565 7fe9bae887a0 20 read_log 192888'450 (192888'31) delete 28d2c6ab/benchmark_data_storage0111-ib.pocstor_11336_object2510/head//1 by client.1312877.0:29794 2014-08-11 16:27:06.966418 2014-08-13 17:38:02.534587 7fe9bae887a0 20 read_log 192888'451 (192888'32) delete a9ff716b/benchmark_data_storage0111-ib.pocstor_11336_object2683/head//1 by client.1312877.0:29967 2014-08-11 16:27:07.010376 2014-08-13 17:38:02.534611 7fe9bae887a0 20 read_log 192888'452 (192888'33) delete 6eac06ab/benchmark_data_storage0111-ib.pocstor_11336_object2861/head//1 by client.1312877.0:30145 2014-08-11 16:27:07.127585 2014-08-13 17:38:02.534632 7fe9bae887a0 20 read_log 192888'453 (192888'34) delete e39450ab/benchmark_data_storage0111-ib.pocstor_11336_object2944/head//1 by client.1312877.0:30228 2014-08-11 16:27:07.148317 2014-08-13 17:38:02.534656 7fe9bae887a0 20 read_log 192888'454 (192888'35) delete 372d6ab/benchmark_data_storage0111-ib.pocstor_11336_object2981/head//1 by client.1312877.0:30265 2014-08-11 16:27:07.157884 2014-08-13 17:38:02.534678 7fe9bae887a0 20 read_log 192888'455 (192888'36) delete aaad6a2b/benchmark_data_storage0111-ib.pocstor_11336_object2989/head//1 by client.1312877.0:30273 2014-08-11 16:27:07.159726 2014-08-13 17:38:02.534709 7fe9bae887a0 20 read_log 192888'456 (192888'37) delete dbd623ab/benchmark_data_storage0111-ib.pocstor_11336_object3227/head//1 by client.1312877.0:30511 2014-08-11 16:27:07.220869 2014-08-13 17:38:02.534731 7fe9bae887a0 20 read_log 192888'457 (192888'38) delete f13350eb/benchmark_data_storage0111-ib.pocstor_11336_object3246/head//1 by client.1312877.0:30530 2014-08-11 16:27:07.229216 2014-08-13 17:38:02.534754 7fe9bae887a0 20 read_log 192888'458 (192888'39) delete 752770eb/benchmark_data_storage0111-ib.pocstor_11336_object3306/head//1 by client.1312877.0:30590 2014-08-11 16:27:07.257440 2014-08-13 17:38:02.534826 7fe9bae887a0 20 read_log 192888'459 (192888'40) delete dd84e9ab/benchmark_data_storage0111-ib.pocstor_11336_object3341/head//1 by client.1312877.0:30625 2014-08-11 16:27:07.279826 2014-08-13 17:38:02.534854 7fe9bae887a0 20 read_log 192888'460 (192888'41) delete de0a9deb/benchmark_data_storage0111-ib.pocstor_11336_object3370/head//1 by client.1312877.0:30654 2014-08-11 16:27:07.330510 2014-08-13 17:38:02.534877 7fe9bae887a0 20 read_log 192888'461 (192888'42) delete 7d7620ab/benchmark_data_storage0111-ib.pocstor_11336_object3373/head//1 by client.1312877.0:30657 2014-08-11 16:27:07.337355 2014-08-13 17:38:02.534898 7fe9bae887a0 20 read_log 192888'462 (192888'43) delete 83a675ab/benchmark_data_storage0111-ib.pocstor_11336_object3566/head//1 by client.1312877.0:30850 2014-08-11 16:27:07.796474 2014-08-13 17:38:02.534920 7fe9bae887a0 20 read_log 192888'463 (192888'44) delete 8ef2e3eb/benchmark_data_storage0111-ib.pocstor_11336_object3609/head//1 by client.1312877.0:30893 2014-08-11 16:27:07.818724 2014-08-13 17:38:02.534942 7fe9bae887a0 20 read_log 192888'464 (192888'45) delete c40043eb/benchmark_data_storage0111-ib.pocstor_11336_object3630/head//1 by client.1312877.0:30914 2014-08-11 16:27:07.826470 2014-08-13 17:38:02.534964 7fe9bae887a0 20 read_log 192888'465 (192888'46) delete 83bd3aeb/benchmark_data_storage0111-ib.pocstor_11336_object3634/head//1 by client.1312877.0:30918 2014-08-11 16:27:07.827428 2014-08-13 17:38:02.534986 7fe9bae887a0 20 read_log 192888'466 (192888'47) delete a7339fab/benchmark_data_storage0111-ib.pocstor_11336_object3636/head//1 by client.1312877.0:30920 2014-08-11 16:27:07.828803 2014-08-13 17:38:02.535008 7fe9bae887a0 20 read_log 192888'467 (192888'48) delete b51ef32b/benchmark_data_storage0111-ib.pocstor_11336_object3841/head//1 by client.1312877.0:31125 2014-08-11 16:27:07.888104 2014-08-13 17:38:02.535041 7fe9bae887a0 20 read_log 192888'468 (192888'49) delete e3142eab/benchmark_data_storage0111-ib.pocstor_11336_object4049/head//1 by client.1312877.0:31333 2014-08-11 16:27:07.963841 2014-08-13 17:38:02.535064 7fe9bae887a0 20 read_log 192888'469 (192888'50) delete 2e39aeb/benchmark_data_storage0111-ib.pocstor_11336_object4076/head//1 by client.1312877.0:31360 2014-08-11 16:27:07.990807 2014-08-13 17:38:02.535086 7fe9bae887a0 20 read_log 192888'470 (192888'51) delete 52e94fab/benchmark_data_storage0111-ib.pocstor_11336_object4179/head//1 by client.1312877.0:31463 2014-08-11 16:27:08.082818 2014-08-13 17:38:02.535107 7fe9bae887a0 20 read_log 192888'471 (192888'52) delete a4d05a6b/benchmark_data_storage0111-ib.pocstor_11336_object4217/head//1 by client.1312877.0:31501 2014-08-11 16:27:08.213040 2014-08-13 17:38:02.535129 7fe9bae887a0 20 read_log 192888'472 (192888'53) delete 8e026b6b/benchmark_data_storage0111-ib.pocstor_11336_object4269/head//1 by client.1312877.0:31553 2014-08-11 16:27:08.230094 2014-08-13 17:38:02.535151 7fe9bae887a0 20 read_log 192888'473 (192888'54) delete 45e77deb/benchmark_data_storage0111-ib.pocstor_11336_object4277/head//1 by client.1312877.0:31561 2014-08-11 16:27:08.232812 2014-08-13 17:38:02.535181 7fe9bae887a0 20 read_log 192888'474 (192888'55) delete 2779eb/benchmark_data_storage0111-ib.pocstor_11336_object4350/head//1 by client.1312877.0:31634 2014-08-11 16:27:08.253031 2014-08-13 17:38:02.535206 7fe9bae887a0 20 read_log 192888'475 (192888'56) delete 200acdeb/benchmark_data_storage0111-ib.pocstor_11336_object4425/head//1 by client.1312877.0:31709 2014-08-11 16:27:08.276387 2014-08-13 17:38:02.535228 7fe9bae887a0 20 read_log 192888'476 (192888'57) delete 4967bfeb/benchmark_data_storage0111-ib.pocstor_11336_object4506/head//1 by client.1312877.0:31790 2014-08-11 16:27:08.296640 2014-08-13 17:38:02.535250 7fe9bae887a0 20 read_log 192888'477 (192888'58) delete 13ab5f6b/benchmark_data_storage0111-ib.pocstor_11336_object4629/head//1 by client.1312877.0:31913 2014-08-11 16:27:08.347008 2014-08-13 17:38:02.535276 7fe9bae887a0 20 read_log 192888'478 (192888'59) delete e9c088eb/benchmark_data_storage0111-ib.pocstor_11336_object4640/head//1 by client.1312877.0:31924 2014-08-11 16:27:08.352284 2014-08-13 17:38:02.535311 7fe9bae887a0 20 read_log 192888'479 (192888'60) delete 84ba19eb/benchmark_data_storage0111-ib.pocstor_11336_object4722/head//1 by client.1312877.0:32006 2014-08-11 16:27:08.386537 2014-08-13 17:38:02.535337 7fe9bae887a0 20 read_log 192888'480 (192888'61) delete 64f0a7ab/benchmark_data_storage0111-ib.pocstor_11336_object4767/head//1 by client.1312877.0:32051 2014-08-11 16:27:08.397934 2014-08-13 17:38:02.535361 7fe9bae887a0 20 read_log 192888'481 (192888'62) delete 18830eeb/benchmark_data_storage0111-ib.pocstor_11336_object4790/head//1 by client.1312877.0:32074 2014-08-11 16:27:08.403833 2014-08-13 17:38:02.535383 7fe9bae887a0 20 read_log 192888'482 (192888'63) delete 80cba86b/benchmark_data_storage0111-ib.pocstor_11336_object4818/head//1 by client.1312877.0:32102 2014-08-11 16:27:08.409580 2014-08-13 17:38:02.535405 7fe9bae887a0 20 read_log 192888'483 (192888'64) delete 7b42aa6b/benchmark_data_storage0111-ib.pocstor_11336_object4861/head//1 by client.1312877.0:32145 2014-08-11 16:27:08.419858 2014-08-13 17:38:02.535426 7fe9bae887a0 20 read_log 192888'484 (192888'65) delete cb176c6b/benchmark_data_storage0111-ib.pocstor_11336_object4943/head//1 by client.1312877.0:32227 2014-08-11 16:27:08.444235 2014-08-13 17:38:02.535448 7fe9bae887a0 20 read_log 192888'485 (192888'66) delete 34a403eb/benchmark_data_storage0111-ib.pocstor_11336_object5103/head//1 by client.1312877.0:32387 2014-08-11 16:27:08.525121 2014-08-13 17:38:02.535470 7fe9bae887a0 20 read_log 192888'486 (192888'67) delete 43cf3beb/benchmark_data_storage0111-ib.pocstor_11336_object5115/head//1 by client.1312877.0:32399 2014-08-11 16:27:08.529820 2014-08-13 17:38:02.535492 7fe9bae887a0 20 read_log 192888'487 (192888'68) delete ef97dfab/benchmark_data_storage0111-ib.pocstor_11336_object5267/head//1 by client.1312877.0:32551 2014-08-11 16:27:08.608980 2014-08-13 17:38:02.535517 7fe9bae887a0 20 read_log 192888'488 (192888'69) delete 2a7643eb/benchmark_data_storage0111-ib.pocstor_11336_object5291/head//1 by client.1312877.0:32575 2014-08-11 16:27:08.615061 2014-08-13 17:38:02.535539 7fe9bae887a0 20 read_log 192888'489 (192888'70) delete c91d172b/benchmark_data_storage0111-ib.pocstor_11336_object5349/head//1 by client.1312877.0:32633 2014-08-11 16:27:08.629343 2014-08-13 17:38:02.535560 7fe9bae887a0 20 read_log 192888'490 (192888'71) delete 86f5e06b/benchmark_data_storage0111-ib.pocstor_11336_object5415/head//1 by client.1312877.0:32699 2014-08-11 16:27:08.644360 2014-08-13 17:38:02.535582 7fe9bae887a0 20 read_log 192888'491 (192888'72) delete 199e7beb/benchmark_data_storage0111-ib.pocstor_11336_object5466/head//1 by client.1312877.0:32750 2014-08-11 16:27:08.656118 2014-08-13 17:38:02.535605 7fe9bae887a0 20 read_log 192888'492 (192888'73) delete 7c15dbab/benchmark_data_storage0111-ib.pocstor_11336_object5510/head//1 by client.1312877.0:32794 2014-08-11 16:27:08.666479 2014-08-13 17:38:02.535627 7fe9bae887a0 20 read_log 192888'493 (192888'74) delete 5bad7f6b/benchmark_data_storage0111-ib.pocstor_11336_object5611/head//1 by client.1312877.0:32895 2014-08-11 16:27:08.688927 2014-08-13 17:38:02.535649 7fe9bae887a0 20 read_log 192888'494 (192888'75) delete b65b286b/benchmark_data_storage0111-ib.pocstor_11336_object5618/head//1 by client.1312877.0:32902 2014-08-11 16:27:08.690725 2014-08-13 17:38:02.535679 7fe9bae887a0 20 read_log 192888'495 (192888'76) delete 51e9e32b/benchmark_data_storage0111-ib.pocstor_11336_object5649/head//1 by client.1312877.0:32933 2014-08-11 16:27:08.697349 2014-08-13 17:38:02.535701 7fe9bae887a0 20 read_log 192888'496 (192888'77) delete be6f1b2b/benchmark_data_storage0111-ib.pocstor_11336_object5663/head//1 by client.1312877.0:32947 2014-08-11 16:27:08.701244 2014-08-13 17:38:02.535723 7fe9bae887a0 20 read_log 192888'497 (192888'78) delete ef81c46b/benchmark_data_storage0111-ib.pocstor_11336_object5705/head//1 by client.1312877.0:32989 2014-08-11 16:27:08.711661 2014-08-13 17:38:02.535745 7fe9bae887a0 20 read_log 192888'498 (192888'79) delete 7460ea2b/benchmark_data_storage0111-ib.pocstor_11336_object5758/head//1 by client.1312877.0:33042 2014-08-11 16:27:08.724624 2014-08-13 17:38:02.535778 7fe9bae887a0 20 read_log 192888'499 (192888'80) delete d8711a2b/benchmark_data_storage0111-ib.pocstor_11336_object5802/head//1 by client.1312877.0:33086 2014-08-11 16:27:08.738423 2014-08-13 17:38:02.535806 7fe9bae887a0 20 read_log 192888'500 (192888'81) delete 7cea612b/benchmark_data_storage0111-ib.pocstor_11336_object5943/head//1 by client.1312877.0:33227 2014-08-11 16:27:08.791516 2014-08-13 17:38:02.535828 7fe9bae887a0 20 read_log 192888'501 (192888'82) delete b862b52b/benchmark_data_storage0111-ib.pocstor_11336_object5996/head//1 by client.1312877.0:33280 2014-08-11 16:27:08.821243 2014-08-13 17:38:02.535850 7fe9bae887a0 20 read_log 192888'502 (192888'83) delete aa0fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object6043/head//1 by client.1312877.0:33327 2014-08-11 16:27:08.832412 2014-08-13 17:38:02.535872 7fe9bae887a0 20 read_log 192888'503 (192888'84) delete 1f2e13eb/benchmark_data_storage0111-ib.pocstor_11336_object6114/head//1 by client.1312877.0:33398 2014-08-11 16:27:08.850424 2014-08-13 17:38:02.535893 7fe9bae887a0 20 read_log 192888'504 (192888'85) delete 26185aeb/benchmark_data_storage0111-ib.pocstor_11336_object6115/head//1 by client.1312877.0:33399 2014-08-11 16:27:08.850627 2014-08-13 17:38:02.535915 7fe9bae887a0 20 read_log 192888'505 (192888'86) delete bc9f81ab/benchmark_data_storage0111-ib.pocstor_11336_object6256/head//1 by client.1312877.0:33540 2014-08-11 16:27:08.888229 2014-08-13 17:38:02.535937 7fe9bae887a0 20 read_log 192888'506 (192888'87) delete 665d8cab/benchmark_data_storage0111-ib.pocstor_11336_object6345/head//1 by client.1312877.0:33629 2014-08-11 16:27:08.909014 2014-08-13 17:38:02.535959 7fe9bae887a0 20 read_log 192888'507 (192888'88) delete 2797a2b/benchmark_data_storage0111-ib.pocstor_11336_object6356/head//1 by client.1312877.0:33640 2014-08-11 16:27:08.912070 2014-08-13 17:38:02.535980 7fe9bae887a0 20 read_log 192888'508 (192888'89) delete dd86336b/benchmark_data_storage0111-ib.pocstor_11336_object6360/head//1 by client.1312877.0:33644 2014-08-11 16:27:08.912470 2014-08-13 17:38:02.536002 7fe9bae887a0 20 read_log 192888'509 (192888'90) delete 81e68deb/benchmark_data_storage0111-ib.pocstor_11336_object6546/head//1 by client.1312877.0:33830 2014-08-11 16:27:08.965431 2014-08-13 17:38:02.536036 7fe9bae887a0 20 read_log 192888'510 (192888'91) delete cacbb16b/benchmark_data_storage0111-ib.pocstor_11336_object6567/head//1 by client.1312877.0:33851 2014-08-11 16:27:08.970488 2014-08-13 17:38:02.536059 7fe9bae887a0 20 read_log 192888'511 (192888'92) delete 982d3c2b/benchmark_data_storage0111-ib.pocstor_11336_object6610/head//1 by client.1312877.0:33894 2014-08-11 16:27:08.980162 2014-08-13 17:38:02.536081 7fe9bae887a0 20 read_log 192888'512 (192888'93) delete 7c69c2b/benchmark_data_storage0111-ib.pocstor_11336_object6688/head//1 by client.1312877.0:33972 2014-08-11 16:27:08.998024 2014-08-13 17:38:02.536102 7fe9bae887a0 20 read_log 192888'513 (192888'94) delete 4db680ab/benchmark_data_storage0111-ib.pocstor_11336_object6735/head//1 by client.1312877.0:34019 2014-08-11 16:27:09.010135 2014-08-13 17:38:02.536124 7fe9bae887a0 20 read_log 192888'514 (192888'95) delete ee8926eb/benchmark_data_storage0111-ib.pocstor_11336_object6785/head//1 by client.1312877.0:34069 2014-08-11 16:27:09.022110 2014-08-13 17:38:02.536147 7fe9bae887a0 20 read_log 192888'515 (192888'96) delete 183f4fab/benchmark_data_storage0111-ib.pocstor_11336_object6867/head//1 by client.1312877.0:34151 2014-08-11 16:27:09.044992 2014-08-13 17:38:02.536178 7fe9bae887a0 20 read_log 192888'516 (192888'97) delete 77e164ab/benchmark_data_storage0111-ib.pocstor_11336_object6990/head//1 by client.1312877.0:34274 2014-08-11 16:27:09.077190 2014-08-13 17:38:02.536201 7fe9bae887a0 20 read_log 192888'517 (192888'98) delete e34b5a2b/benchmark_data_storage0111-ib.pocstor_11336_object6999/head//1 by client.1312877.0:34283 2014-08-11 16:27:09.078934 2014-08-13 17:38:02.536254 7fe9bae887a0 20 read_log 192888'518 (192888'99) delete aa9237eb/benchmark_data_storage0111-ib.pocstor_11336_object7008/head//1 by client.1312877.0:34292 2014-08-11 16:27:09.081592 2014-08-13 17:38:02.536293 7fe9bae887a0 20 read_log 192888'519 (192888'100) delete ac4a732b/benchmark_data_storage0111-ib.pocstor_11336_object7094/head//1 by client.1312877.0:34378 2014-08-11 16:27:09.105363 2014-08-13 17:38:02.536320 7fe9bae887a0 20 read_log 192888'520 (192888'101) delete c413e7ab/benchmark_data_storage0111-ib.pocstor_11336_object7152/head//1 by client.1312877.0:34436 2014-08-11 16:27:09.119085 2014-08-13 17:38:02.536343 7fe9bae887a0 20 read_log 192888'521 (192888'102) delete ef342f6b/benchmark_data_storage0111-ib.pocstor_11336_object7178/head//1 by client.1312877.0:34462 2014-08-11 16:27:09.124925 2014-08-13 17:38:02.536365 7fe9bae887a0 20 read_log 192888'522 (192888'103) delete 581f046b/benchmark_data_storage0111-ib.pocstor_11336_object7210/head//1 by client.1312877.0:34494 2014-08-11 16:27:09.135867 2014-08-13 17:38:02.536387 7fe9bae887a0 20 read_log 192888'523 (192888'104) delete 6e01b76b/benchmark_data_storage0111-ib.pocstor_11336_object7236/head//1 by client.1312877.0:34520 2014-08-11 16:27:09.141889 2014-08-13 17:38:02.536409 7fe9bae887a0 20 read_log 192888'524 (192888'105) delete d200a6eb/benchmark_data_storage0111-ib.pocstor_11336_object7281/head//1 by client.1312877.0:34565 2014-08-11 16:27:09.153200 2014-08-13 17:38:02.536430 7fe9bae887a0 20 read_log 192888'525 (192888'106) delete 84fb60eb/benchmark_data_storage0111-ib.pocstor_11336_object7370/head//1 by client.1312877.0:34654 2014-08-11 16:27:09.173412 2014-08-13 17:38:02.536454 7fe9bae887a0 20 read_log 192888'526 (192888'107) delete d6b5f52b/benchmark_data_storage0111-ib.pocstor_11336_object7447/head//1 by client.1312877.0:34731 2014-08-11 16:27:09.196895 2014-08-13 17:38:02.536476 7fe9bae887a0 20 read_log 192888'527 (192888'108) delete 8552af2b/benchmark_data_storage0111-ib.pocstor_11336_object7477/head//1 by client.1312877.0:34761 2014-08-11 16:27:09.203097 2014-08-13 17:38:02.536498 7fe9bae887a0 20 read_log 192888'528 (192888'109) delete d583a86b/benchmark_data_storage0111-ib.pocstor_11336_object7517/head//1 by client.1312877.0:34801 2014-08-11 16:27:09.214764 2014-08-13 17:38:02.536519 7fe9bae887a0 20 read_log 192888'529 (192888'110) delete 5fb5a3eb/benchmark_data_storage0111-ib.pocstor_11336_object7554/head//1 by client.1312877.0:34838 2014-08-11 16:27:09.225151 2014-08-13 17:38:02.536541 7fe9bae887a0 20 read_log 192888'530 (192888'111) delete fb03b0eb/benchmark_data_storage0111-ib.pocstor_11336_object7647/head//1 by client.1312877.0:34931 2014-08-11 16:27:09.247817 2014-08-13 17:38:02.536563 7fe9bae887a0 20 read_log 192888'531 (192888'112) delete 6b05496b/benchmark_data_storage0111-ib.pocstor_11336_object7737/head//1 by client.1312877.0:35021 2014-08-11 16:27:09.269725 2014-08-13 17:38:02.536585 7fe9bae887a0 20 read_log 192888'532 (192888'113) delete 8326d86b/benchmark_data_storage0111-ib.pocstor_11336_object7812/head//1 by client.1312877.0:35096 2014-08-11 16:27:09.290064 2014-08-13 17:38:02.536606 7fe9bae887a0 20 read_log 192888'533 (192888'114) delete e8e52e2b/benchmark_data_storage0111-ib.pocstor_11336_object7900/head//1 by client.1312877.0:35184 2014-08-11 16:27:09.316802 2014-08-13 17:38:02.536661 7fe9bae887a0 20 read_log 192888'534 (192888'115) delete e48691eb/benchmark_data_storage0111-ib.pocstor_11336_object7922/head//1 by client.1312877.0:35206 2014-08-11 16:27:09.322732 2014-08-13 17:38:02.536691 7fe9bae887a0 20 read_log 192888'535 (192888'116) delete 10dd5aeb/benchmark_data_storage0111-ib.pocstor_11336_object7942/head//1 by client.1312877.0:35226 2014-08-11 16:27:09.327566 2014-08-13 17:38:02.536714 7fe9bae887a0 20 read_log 192888'536 (192888'117) delete 1dd889ab/benchmark_data_storage0111-ib.pocstor_11336_object8027/head//1 by client.1312877.0:35311 2014-08-11 16:27:09.352224 2014-08-13 17:38:02.536736 7fe9bae887a0 20 read_log 192888'537 (192888'118) delete 8d332b2b/benchmark_data_storage0111-ib.pocstor_11336_object8120/head//1 by client.1312877.0:35404 2014-08-11 16:27:09.375595 2014-08-13 17:38:02.536757 7fe9bae887a0 20 read_log 192888'538 (192888'119) delete 1793a06b/benchmark_data_storage0111-ib.pocstor_11336_object8121/head//1 by client.1312877.0:35405 2014-08-11 16:27:09.376030 2014-08-13 17:38:02.536794 7fe9bae887a0 20 read_log 192888'539 (192888'120) delete b4faa32b/benchmark_data_storage0111-ib.pocstor_11336_object8144/head//1 by client.1312877.0:35428 2014-08-11 16:27:09.381408 2014-08-13 17:38:02.536821 7fe9bae887a0 20 read_log 192888'540 (192888'121) delete 4035182b/benchmark_data_storage0111-ib.pocstor_11336_object8213/head//1 by client.1312877.0:35497 2014-08-11 16:27:09.400123 2014-08-13 17:38:02.536843 7fe9bae887a0 20 read_log 192888'541 (192888'122) delete 461c62b/benchmark_data_storage0111-ib.pocstor_11336_object8226/head//1 by client.1312877.0:35510 2014-08-11 16:27:09.403041 2014-08-13 17:38:02.536865 7fe9bae887a0 20 read_log 192888'542 (192888'123) delete fef695ab/benchmark_data_storage0111-ib.pocstor_11336_object8282/head//1 by client.1312877.0:35566 2014-08-11 16:27:09.415444 2014-08-13 17:38:02.536899 7fe9bae887a0 20 read_log 192888'543 (192888'124) delete 133eef2b/benchmark_data_storage0111-ib.pocstor_11336_object8316/head//1 by client.1312877.0:35600 2014-08-11 16:27:09.423728 2014-08-13 17:38:02.536922 7fe9bae887a0 20 read_log 192888'544 (192888'125) delete 5905afab/benchmark_data_storage0111-ib.pocstor_11336_object8508/head//1 by client.1312877.0:35792 2014-08-11 16:27:09.472886 2014-08-13 17:38:02.536944 7fe9bae887a0 20 read_log 192888'545 (192888'126) delete 1c3d296b/benchmark_data_storage0111-ib.pocstor_11336_object8550/head//1 by client.1312877.0:35834 2014-08-11 16:27:09.483813 2014-08-13 17:38:02.536966 7fe9bae887a0 20 read_log 192888'546 (192888'127) delete 582141eb/benchmark_data_storage0111-ib.pocstor_11336_object8559/head//1 by client.1312877.0:35843 2014-08-11 16:27:09.485813 2014-08-13 17:38:02.536988 7fe9bae887a0 20 read_log 192888'547 (192888'128) delete 39758f6b/benchmark_data_storage0111-ib.pocstor_11336_object8581/head//1 by client.1312877.0:35865 2014-08-11 16:27:09.491999 2014-08-13 17:38:02.537049 7fe9bae887a0 20 read_log 192888'548 (192888'129) delete 5016fc6b/benchmark_data_storage0111-ib.pocstor_11336_object8602/head//1 by client.1312877.0:35886 2014-08-11 16:27:09.496789 2014-08-13 17:38:02.537071 7fe9bae887a0 20 read_log 192888'549 (192888'130) delete 11560a2b/benchmark_data_storage0111-ib.pocstor_11336_object8708/head//1 by client.1312877.0:35992 2014-08-11 16:27:09.522619 2014-08-13 17:38:02.537094 7fe9bae887a0 20 read_log 192888'550 (192888'131) delete 22b07aab/benchmark_data_storage0111-ib.pocstor_11336_object8772/head//1 by client.1312877.0:36056 2014-08-11 16:27:09.541103 2014-08-13 17:38:02.537116 7fe9bae887a0 20 read_log 192888'551 (192888'132) delete 7e5d59ab/benchmark_data_storage0111-ib.pocstor_11336_object8779/head//1 by client.1312877.0:36063 2014-08-11 16:27:09.542679 2014-08-13 17:38:02.537138 7fe9bae887a0 20 read_log 192888'552 (192888'133) delete ec37116b/benchmark_data_storage0111-ib.pocstor_11336_object8784/head//1 by client.1312877.0:36068 2014-08-11 16:27:09.543661 2014-08-13 17:38:02.537168 7fe9bae887a0 20 read_log 192888'553 (192888'134) delete b0de23eb/benchmark_data_storage0111-ib.pocstor_11336_object8926/head//1 by client.1312877.0:36210 2014-08-11 16:27:09.579567 2014-08-13 17:38:02.537193 7fe9bae887a0 20 read_log 192888'554 (192888'135) delete bb89eab/benchmark_data_storage0111-ib.pocstor_11336_object8969/head//1 by client.1312877.0:36253 2014-08-11 16:27:09.590497 2014-08-13 17:38:02.537215 7fe9bae887a0 20 read_log 192888'555 (192888'136) delete 1573a02b/benchmark_data_storage0111-ib.pocstor_11336_object9147/head//1 by client.1312877.0:36431 2014-08-11 16:27:09.636140 2014-08-13 17:38:02.537238 7fe9bae887a0 20 read_log 192888'556 (192888'137) delete 9b3d4deb/benchmark_data_storage0111-ib.pocstor_11336_object9217/head//1 by client.1312877.0:36501 2014-08-11 16:27:09.651926 2014-08-13 17:38:02.537259 7fe9bae887a0 20 read_log 192888'557 (192888'138) delete 7c824dab/benchmark_data_storage0111-ib.pocstor_11336_object9339/head//1 by client.1312877.0:36623 2014-08-11 16:27:09.681301 2014-08-13 17:38:02.537281 7fe9bae887a0 20 read_log 192888'558 (192888'139) delete 7ce138eb/benchmark_data_storage0111-ib.pocstor_11336_object9430/head//1 by client.1312877.0:36714 2014-08-11 16:27:09.704268 2014-08-13 17:38:02.537315 7fe9bae887a0 20 read_log 192888'559 (192888'140) delete 5dd698ab/benchmark_data_storage0111-ib.pocstor_11336_object9467/head//1 by client.1312877.0:36751 2014-08-11 16:27:09.713700 2014-08-13 17:38:02.537342 7fe9bae887a0 20 read_log 192888'560 (192888'141) delete 6fa2c96b/benchmark_data_storage0111-ib.pocstor_11336_object9547/head//1 by client.1312877.0:36831 2014-08-11 16:27:09.735681 2014-08-13 17:38:02.537366 7fe9bae887a0 20 read_log 192888'561 (192888'142) delete af93b72b/benchmark_data_storage0111-ib.pocstor_11336_object9599/head//1 by client.1312877.0:36883 2014-08-11 16:27:09.750025 2014-08-13 17:38:02.537388 7fe9bae887a0 20 read_log 192888'562 (192888'143) delete 9e00ec6b/benchmark_data_storage0111-ib.pocstor_11336_object9624/head//1 by client.1312877.0:36908 2014-08-11 16:27:09.756796 2014-08-13 17:38:02.537409 7fe9bae887a0 20 read_log 192888'563 (192888'144) delete 30d7d7ab/benchmark_data_storage0111-ib.pocstor_11336_object9654/head//1 by client.1312877.0:36938 2014-08-11 16:27:09.763927 2014-08-13 17:38:02.537467 7fe9bae887a0 20 read_log 192888'564 (192888'145) delete c30ef8eb/benchmark_data_storage0111-ib.pocstor_11336_object9675/head//1 by client.1312877.0:36959 2014-08-11 16:27:09.768622 2014-08-13 17:38:02.537490 7fe9bae887a0 20 read_log 192888'565 (192888'146) delete e5c5686b/benchmark_data_storage0111-ib.pocstor_11336_object9798/head//1 by client.1312877.0:37082 2014-08-11 16:27:09.802523 2014-08-13 17:38:02.537511 7fe9bae887a0 20 read_log 192888'566 (192888'147) delete f0c1866b/benchmark_data_storage0111-ib.pocstor_11336_object9809/head//1 by client.1312877.0:37093 2014-08-11 16:27:09.803864 2014-08-13 17:38:02.537533 7fe9bae887a0 20 read_log 192888'567 (192888'148) delete 7dfac8eb/benchmark_data_storage0111-ib.pocstor_11336_object9862/head//1 by client.1312877.0:37146 2014-08-11 16:27:09.817216 2014-08-13 17:38:02.537555 7fe9bae887a0 20 read_log 192888'568 (192888'149) delete 7385cbab/benchmark_data_storage0111-ib.pocstor_11336_object10044/head//1 by client.1312877.0:37328 2014-08-11 16:27:09.863174 2014-08-13 17:38:02.537577 7fe9bae887a0 20 read_log 192888'569 (192888'150) delete 8cef43ab/benchmark_data_storage0111-ib.pocstor_11336_object10070/head//1 by client.1312877.0:37354 2014-08-11 16:27:09.869667 2014-08-13 17:38:02.537599 7fe9bae887a0 20 read_log 192888'570 (192888'151) delete a97eac2b/benchmark_data_storage0111-ib.pocstor_11336_object10215/head//1 by client.1312877.0:37499 2014-08-11 16:27:09.907559 2014-08-13 17:38:02.537621 7fe9bae887a0 20 read_log 192888'571 (192888'152) delete 9dccadab/benchmark_data_storage0111-ib.pocstor_11336_object10293/head//1 by client.1312877.0:37577 2014-08-11 16:27:09.924944 2014-08-13 17:38:02.537643 7fe9bae887a0 20 read_log 192888'572 (192888'153) delete 403f876b/benchmark_data_storage0111-ib.pocstor_11336_object10422/head//1 by client.1312877.0:37706 2014-08-11 16:27:09.956368 2014-08-13 17:38:02.537664 7fe9bae887a0 20 read_log 192888'573 (192888'154) delete d913feab/benchmark_data_storage0111-ib.pocstor_11336_object10499/head//1 by client.1312877.0:37783 2014-08-11 16:27:09.973350 2014-08-13 17:38:02.537686 7fe9bae887a0 20 read_log 192888'574 (192888'155) delete 1067f22b/benchmark_data_storage0111-ib.pocstor_11336_object10519/head//1 by client.1312877.0:37803 2014-08-11 16:27:09.977347 2014-08-13 17:38:02.537708 7fe9bae887a0 20 read_log 192888'575 (192888'156) delete 4ca0adab/benchmark_data_storage0111-ib.pocstor_11336_object10563/head//1 by client.1312877.0:37847 2014-08-11 16:27:09.986805 2014-08-13 17:38:02.537730 7fe9bae887a0 20 read_log 192888'576 (192888'157) delete cd10626b/benchmark_data_storage0111-ib.pocstor_11336_object10629/head//1 by client.1312877.0:37913 2014-08-11 16:27:10.002592 2014-08-13 17:38:02.537753 7fe9bae887a0 20 read_log 192888'577 (192888'158) delete 2d074d6b/benchmark_data_storage0111-ib.pocstor_11336_object10684/head//1 by client.1312877.0:37968 2014-08-11 16:27:10.016038 2014-08-13 17:38:02.537775 7fe9bae887a0 20 read_log 192888'578 (192888'159) delete 7094446b/benchmark_data_storage0111-ib.pocstor_11336_object10863/head//1 by client.1312877.0:38147 2014-08-11 16:27:10.060241 2014-08-13 17:38:02.537812 7fe9bae887a0 20 read_log 192888'579 (192888'160) delete 24fc1eab/benchmark_data_storage0111-ib.pocstor_11336_object10891/head//1 by client.1312877.0:38175 2014-08-11 16:27:10.066778 2014-08-13 17:38:02.537840 7fe9bae887a0 20 read_log 192888'580 (192888'161) delete ba27f86b/benchmark_data_storage0111-ib.pocstor_11336_object10963/head//1 by client.1312877.0:38247 2014-08-11 16:27:10.083951 2014-08-13 17:38:02.537863 7fe9bae887a0 20 read_log 192888'581 (192888'162) delete 979816b/benchmark_data_storage0111-ib.pocstor_11336_object10980/head//1 by client.1312877.0:38264 2014-08-11 16:27:10.087771 2014-08-13 17:38:02.537885 7fe9bae887a0 20 read_log 192888'582 (192888'163) delete 7f2fe8eb/benchmark_data_storage0111-ib.pocstor_11336_object10990/head//1 by client.1312877.0:38274 2014-08-11 16:27:10.090581 2014-08-13 17:38:02.537907 7fe9bae887a0 20 read_log 192888'583 (192888'164) delete 53e63eb/benchmark_data_storage0111-ib.pocstor_11336_object11017/head//1 by client.1312877.0:38301 2014-08-11 16:27:10.098585 2014-08-13 17:38:02.537931 7fe9bae887a0 20 read_log 192888'584 (192888'165) delete b5c20a6b/benchmark_data_storage0111-ib.pocstor_11336_object11120/head//1 by client.1312877.0:38404 2014-08-11 16:27:10.123768 2014-08-13 17:38:02.537953 7fe9bae887a0 20 read_log 192888'585 (192888'166) delete 2c9accab/benchmark_data_storage0111-ib.pocstor_11336_object11135/head//1 by client.1312877.0:38419 2014-08-11 16:27:10.127628 2014-08-13 17:38:02.537976 7fe9bae887a0 20 read_log 192888'586 (192888'167) delete 8cd4cbab/benchmark_data_storage0111-ib.pocstor_11336_object11148/head//1 by client.1312877.0:38432 2014-08-11 16:27:10.130794 2014-08-13 17:38:02.537999 7fe9bae887a0 20 read_log 192888'587 (192888'168) delete 5a894beb/benchmark_data_storage0111-ib.pocstor_11336_object11176/head//1 by client.1312877.0:38460 2014-08-11 16:27:10.137972 2014-08-13 17:38:02.538020 7fe9bae887a0 20 read_log 192888'588 (192888'169) delete a326b22b/benchmark_data_storage0111-ib.pocstor_11336_object11234/head//1 by client.1312877.0:38518 2014-08-11 16:27:10.150589 2014-08-13 17:38:02.538042 7fe9bae887a0 20 read_log 192888'589 (192888'170) delete f27ed76b/benchmark_data_storage0111-ib.pocstor_11336_object11321/head//1 by client.1312877.0:38605 2014-08-11 16:27:10.171756 2014-08-13 17:38:02.538064 7fe9bae887a0 20 read_log 192888'590 (192888'171) delete 36effaeb/benchmark_data_storage0111-ib.pocstor_11336_object11355/head//1 by client.1312877.0:38639 2014-08-11 16:27:10.182191 2014-08-13 17:38:02.538088 7fe9bae887a0 20 read_log 192888'591 (192888'172) delete c959e92b/benchmark_data_storage0111-ib.pocstor_11336_object11363/head//1 by client.1312877.0:38647 2014-08-11 16:27:10.183439 2014-08-13 17:38:02.538109 7fe9bae887a0 20 read_log 192888'592 (192888'173) delete 8a569feb/benchmark_data_storage0111-ib.pocstor_11336_object11379/head//1 by client.1312877.0:38663 2014-08-11 16:27:10.189154 2014-08-13 17:38:02.538131 7fe9bae887a0 20 read_log 192888'593 (192888'174) delete 265609ab/benchmark_data_storage0111-ib.pocstor_11336_object11434/head//1 by client.1312877.0:38718 2014-08-11 16:27:10.201777 2014-08-13 17:38:02.538166 7fe9bae887a0 20 read_log 192888'594 (192888'175) delete 2f3985ab/benchmark_data_storage0111-ib.pocstor_11336_object11591/head//1 by client.1312877.0:38875 2014-08-11 16:27:10.243339 2014-08-13 17:38:02.538190 7fe9bae887a0 20 read_log 192888'595 (192888'176) delete 2218ab6b/benchmark_data_storage0111-ib.pocstor_11336_object11594/head//1 by client.1312877.0:38878 2014-08-11 16:27:10.244598 2014-08-13 17:38:02.538220 7fe9bae887a0 20 read_log 192888'596 (192888'177) delete 780536ab/benchmark_data_storage0111-ib.pocstor_11336_object11678/head//1 by client.1312877.0:38962 2014-08-11 16:27:10.268573 2014-08-13 17:38:02.538242 7fe9bae887a0 20 read_log 192888'597 (192888'178) delete a4d98e2b/benchmark_data_storage0111-ib.pocstor_11336_object11711/head//1 by client.1312877.0:38995 2014-08-11 16:27:10.275492 2014-08-13 17:38:02.538264 7fe9bae887a0 20 read_log 192888'598 (192888'179) delete 3ae3c7eb/benchmark_data_storage0111-ib.pocstor_11336_object11840/head//1 by client.1312877.0:39124 2014-08-11 16:27:10.308037 2014-08-13 17:38:02.538332 7fe9bae887a0 20 read_log 192888'599 (192888'180) delete 28cb2aeb/benchmark_data_storage0111-ib.pocstor_11336_object11864/head//1 by client.1312877.0:39148 2014-08-11 16:27:10.316166 2014-08-13 17:38:02.538362 7fe9bae887a0 20 read_log 192888'600 (192888'181) delete a7751f6b/benchmark_data_storage0111-ib.pocstor_11336_object11902/head//1 by client.1312877.0:39186 2014-08-11 16:27:10.324062 2014-08-13 17:38:02.538384 7fe9bae887a0 20 read_log 192888'601 (192888'182) delete 4b1d62ab/benchmark_data_storage0111-ib.pocstor_11336_object11920/head//1 by client.1312877.0:39204 2014-08-11 16:27:10.328075 2014-08-13 17:38:02.538406 7fe9bae887a0 20 read_log 192888'602 (192888'183) delete 9ef299ab/benchmark_data_storage0111-ib.pocstor_11336_object11921/head//1 by client.1312877.0:39205 2014-08-11 16:27:10.328180 2014-08-13 17:38:02.538429 7fe9bae887a0 20 read_log 192888'603 (192888'184) delete 47510d6b/benchmark_data_storage0111-ib.pocstor_11336_object12100/head//1 by client.1312877.0:39384 2014-08-11 16:27:10.373175 2014-08-13 17:38:02.538454 7fe9bae887a0 20 read_log 192888'604 (192888'185) delete 69a89f2b/benchmark_data_storage0111-ib.pocstor_11336_object12168/head//1 by client.1312877.0:39452 2014-08-11 16:27:10.390862 2014-08-13 17:38:02.538479 7fe9bae887a0 20 read_log 192888'605 (192888'186) delete 6b100eeb/benchmark_data_storage0111-ib.pocstor_11336_object12257/head//1 by client.1312877.0:39541 2014-08-11 16:27:10.412329 2014-08-13 17:38:02.538502 7fe9bae887a0 20 read_log 192888'606 (192888'187) delete cdb5142b/benchmark_data_storage0111-ib.pocstor_11336_object12280/head//1 by client.1312877.0:39564 2014-08-11 16:27:10.418654 2014-08-13 17:38:02.538523 7fe9bae887a0 20 read_log 192888'607 (192888'188) delete 50af2eeb/benchmark_data_storage0111-ib.pocstor_11336_object12322/head//1 by client.1312877.0:39606 2014-08-11 16:27:10.429707 2014-08-13 17:38:02.538545 7fe9bae887a0 20 read_log 192888'608 (192888'189) delete d5d1452b/benchmark_data_storage0111-ib.pocstor_11336_object12326/head//1 by client.1312877.0:39610 2014-08-11 16:27:10.430450 2014-08-13 17:38:02.538567 7fe9bae887a0 20 read_log 192888'609 (192888'190) delete a8750ceb/benchmark_data_storage0111-ib.pocstor_11336_object12327/head//1 by client.1312877.0:39611 2014-08-11 16:27:10.430839 2014-08-13 17:38:02.538588 7fe9bae887a0 20 read_log 192888'610 (192888'191) delete 6cd4e4eb/benchmark_data_storage0111-ib.pocstor_11336_object12360/head//1 by client.1312877.0:39644 2014-08-11 16:27:10.440934 2014-08-13 17:38:02.538640 7fe9bae887a0 20 read_log 192888'611 (192888'192) delete e3f5d66b/benchmark_data_storage0111-ib.pocstor_11336_object12391/head//1 by client.1312877.0:39675 2014-08-11 16:27:10.447294 2014-08-13 17:38:02.538667 7fe9bae887a0 20 read_log 192888'612 (192888'193) delete 75ceb6eb/benchmark_data_storage0111-ib.pocstor_11336_object12417/head//1 by client.1312877.0:39701 2014-08-11 16:27:10.453521 2014-08-13 17:38:02.538690 7fe9bae887a0 20 read_log 192888'613 (192888'194) delete 8e72486b/benchmark_data_storage0111-ib.pocstor_11336_object12632/head//1 by client.1312877.0:39916 2014-08-11 16:27:10.505622 2014-08-13 17:38:02.538712 7fe9bae887a0 20 read_log 192888'614 (192888'195) delete cce08cab/benchmark_data_storage0111-ib.pocstor_11336_object12674/head//1 by client.1312877.0:39958 2014-08-11 16:27:10.515088 2014-08-13 17:38:02.538734 7fe9bae887a0 20 read_log 192888'615 (192888'196) delete 3b28fc6b/benchmark_data_storage0111-ib.pocstor_11336_object12837/head//1 by client.1312877.0:40121 2014-08-11 16:27:10.558608 2014-08-13 17:38:02.538756 7fe9bae887a0 20 read_log 192888'616 (192888'197) delete d4d8526b/benchmark_data_storage0111-ib.pocstor_11336_object12951/head//1 by client.1312877.0:40235 2014-08-11 16:27:10.585349 2014-08-13 17:38:02.538778 7fe9bae887a0 20 read_log 192888'617 (192888'198) delete 1454bcab/benchmark_data_storage0111-ib.pocstor_11336_object12984/head//1 by client.1312877.0:40268 2014-08-11 16:27:10.593713 2014-08-13 17:38:02.538799 7fe9bae887a0 20 read_log 192888'618 (192888'199) delete 7ad66aab/benchmark_data_storage0111-ib.pocstor_11336_object13208/head//1 by client.1312877.0:40492 2014-08-11 16:27:10.652949 2014-08-13 17:38:02.538833 7fe9bae887a0 20 read_log 192888'619 (192888'200) delete ef9a3c6b/benchmark_data_storage0111-ib.pocstor_11336_object13239/head//1 by client.1312877.0:40523 2014-08-11 16:27:10.660620 2014-08-13 17:38:02.538860 7fe9bae887a0 20 read_log 192888'620 (192888'201) delete a87af4eb/benchmark_data_storage0111-ib.pocstor_11336_object13360/head//1 by client.1312877.0:40644 2014-08-11 16:27:10.695009 2014-08-13 17:38:02.538882 7fe9bae887a0 20 read_log 192888'621 (192888'202) delete 2d5e4eab/benchmark_data_storage0111-ib.pocstor_11336_object13477/head//1 by client.1312877.0:40761 2014-08-11 16:27:10.724492 2014-08-13 17:38:02.538904 7fe9bae887a0 20 read_log 192888'622 (192888'203) delete 6a50cd6b/benchmark_data_storage0111-ib.pocstor_11336_object13497/head//1 by client.1312877.0:40781 2014-08-11 16:27:10.729012 2014-08-13 17:38:02.538925 7fe9bae887a0 20 read_log 192888'623 (192888'204) delete 34362b6b/benchmark_data_storage0111-ib.pocstor_11336_object13615/head//1 by client.1312877.0:40899 2014-08-11 16:27:10.758717 2014-08-13 17:38:02.538947 7fe9bae887a0 20 read_log 192888'624 (192888'205) delete dfdebe2b/benchmark_data_storage0111-ib.pocstor_11336_object13622/head//1 by client.1312877.0:40906 2014-08-11 16:27:10.760123 2014-08-13 17:38:02.538969 7fe9bae887a0 20 read_log 192888'625 (192888'206) delete 4c4339eb/benchmark_data_storage0111-ib.pocstor_11336_object13623/head//1 by client.1312877.0:40907 2014-08-11 16:27:10.760230 2014-08-13 17:38:02.538990 7fe9bae887a0 20 read_log 192888'626 (192888'207) delete 898a53ab/benchmark_data_storage0111-ib.pocstor_11336_object14154/head//1 by client.1312877.0:41438 2014-08-11 16:27:10.906781 2014-08-13 17:38:02.539013 7fe9bae887a0 20 read_log 192888'627 (192888'208) delete 48a3ff2b/benchmark_data_storage0111-ib.pocstor_11336_object14170/head//1 by client.1312877.0:41454 2014-08-11 16:27:10.910138 2014-08-13 17:38:02.539035 7fe9bae887a0 20 read_log 192888'628 (192888'209) delete 6545c6ab/benchmark_data_storage0111-ib.pocstor_11336_object14196/head//1 by client.1312877.0:41480 2014-08-11 16:27:10.917163 2014-08-13 17:38:02.539057 7fe9bae887a0 20 read_log 192888'629 (192888'210) delete da0040ab/benchmark_data_storage0111-ib.pocstor_11336_object14203/head//1 by client.1312877.0:41487 2014-08-11 16:27:10.918588 2014-08-13 17:38:02.539079 7fe9bae887a0 20 read_log 192888'630 (192888'211) delete 3b3442ab/benchmark_data_storage0111-ib.pocstor_11336_object14209/head//1 by client.1312877.0:41493 2014-08-11 16:27:10.920307 2014-08-13 17:38:02.539101 7fe9bae887a0 20 read_log 192888'631 (192888'212) delete 138d602b/benchmark_data_storage0111-ib.pocstor_11336_object14220/head//1 by client.1312877.0:41504 2014-08-11 16:27:10.922793 2014-08-13 17:38:02.539122 7fe9bae887a0 20 read_log 192888'632 (192888'213) delete d6709e6b/benchmark_data_storage0111-ib.pocstor_11336_object14223/head//1 by client.1312877.0:41507 2014-08-11 16:27:10.923563 2014-08-13 17:38:02.539144 7fe9bae887a0 20 read_log 192888'633 (192888'214) delete fe3dc06b/benchmark_data_storage0111-ib.pocstor_11336_object14364/head//1 by client.1312877.0:41648 2014-08-11 16:27:10.960824 2014-08-13 17:38:02.539183 7fe9bae887a0 20 read_log 192888'634 (192888'215) delete 4a1483ab/benchmark_data_storage0111-ib.pocstor_11336_object14418/head//1 by client.1312877.0:41702 2014-08-11 16:27:10.974038 2014-08-13 17:38:02.539206 7fe9bae887a0 20 read_log 192888'635 (192888'216) delete 89e9b56b/benchmark_data_storage0111-ib.pocstor_11336_object14424/head//1 by client.1312877.0:41708 2014-08-11 16:27:10.976360 2014-08-13 17:38:02.539228 7fe9bae887a0 20 read_log 192888'636 (192888'217) delete d45ab2ab/benchmark_data_storage0111-ib.pocstor_11336_object14441/head//1 by client.1312877.0:41725 2014-08-11 16:27:10.981286 2014-08-13 17:38:02.539254 7fe9bae887a0 20 read_log 192888'637 (192888'218) delete 35e2e1eb/benchmark_data_storage0111-ib.pocstor_11336_object14593/head//1 by client.1312877.0:41877 2014-08-11 16:27:11.020569 2014-08-13 17:38:02.539276 7fe9bae887a0 20 read_log 192888'638 (192888'219) delete f9cba92b/benchmark_data_storage0111-ib.pocstor_11336_object14598/head//1 by client.1312877.0:41882 2014-08-11 16:27:11.021511 2014-08-13 17:38:02.539315 7fe9bae887a0 20 read_log 192888'639 (192888'220) delete 5ddbe7eb/benchmark_data_storage0111-ib.pocstor_11336_object14613/head//1 by client.1312877.0:41897 2014-08-11 16:27:11.025250 2014-08-13 17:38:02.539342 7fe9bae887a0 20 read_log 192888'640 (192888'221) delete 44537cab/benchmark_data_storage0111-ib.pocstor_11336_object14743/head//1 by client.1312877.0:42027 2014-08-11 16:27:11.059361 2014-08-13 17:38:02.539365 7fe9bae887a0 20 read_log 192888'641 (192888'222) delete 3cd17fab/benchmark_data_storage0111-ib.pocstor_11336_object14794/head//1 by client.1312877.0:42078 2014-08-11 16:27:11.073162 2014-08-13 17:38:02.539387 7fe9bae887a0 20 read_log 192888'642 (192888'223) delete fa8a42eb/benchmark_data_storage0111-ib.pocstor_11336_object14952/head//1 by client.1312877.0:42236 2014-08-11 16:27:11.114397 2014-08-13 17:38:02.539413 7fe9bae887a0 20 read_log 192888'643 (192888'224) delete 96108eb/benchmark_data_storage0111-ib.pocstor_11336_object15084/head//1 by client.1312877.0:42368 2014-08-11 16:27:11.147236 2014-08-13 17:38:02.539435 7fe9bae887a0 20 read_log 192888'644 (192888'225) delete 20b9f32b/benchmark_data_storage0111-ib.pocstor_11336_object15097/head//1 by client.1312877.0:42381 2014-08-11 16:27:11.150480 2014-08-13 17:38:02.539457 7fe9bae887a0 20 read_log 192888'645 (192888'226) delete f6085eb/benchmark_data_storage0111-ib.pocstor_11336_object15386/head//1 by client.1312877.0:42670 2014-08-11 16:27:11.219754 2014-08-13 17:38:02.539482 7fe9bae887a0 20 read_log 192888'646 (192888'227) delete f1deeceb/benchmark_data_storage0111-ib.pocstor_11336_object15406/head//1 by client.1312877.0:42690 2014-08-11 16:27:11.226261 2014-08-13 17:38:02.539504 7fe9bae887a0 20 read_log 192888'647 (192888'228) delete 211e31ab/benchmark_data_storage0111-ib.pocstor_11336_object15650/head//1 by client.1312877.0:42934 2014-08-11 16:27:11.287529 2014-08-13 17:38:02.539526 7fe9bae887a0 20 read_log 192888'648 (192888'229) delete 5ea9c7eb/benchmark_data_storage0111-ib.pocstor_11336_object15752/head//1 by client.1312877.0:43036 2014-08-11 16:27:11.311606 2014-08-13 17:38:02.539549 7fe9bae887a0 20 read_log 192888'649 (192888'230) delete b8c42aab/benchmark_data_storage0111-ib.pocstor_11336_object15766/head//1 by client.1312877.0:43050 2014-08-11 16:27:11.315986 2014-08-13 17:38:02.539573 7fe9bae887a0 20 read_log 192888'650 (192888'231) delete e8b2fbab/benchmark_data_storage0111-ib.pocstor_11336_object15797/head//1 by client.1312877.0:43081 2014-08-11 16:27:11.323149 2014-08-13 17:38:02.539595 7fe9bae887a0 20 read_log 192888'651 (192888'232) delete 81b9caab/benchmark_data_storage0111-ib.pocstor_11336_object15801/head//1 by client.1312877.0:43085 2014-08-11 16:27:11.323687 2014-08-13 17:38:02.539616 7fe9bae887a0 20 read_log 192888'652 (192888'233) delete 459c8bab/benchmark_data_storage0111-ib.pocstor_11336_object15913/head//1 by client.1312877.0:43197 2014-08-11 16:27:11.351091 2014-08-13 17:38:02.539638 7fe9bae887a0 20 read_log 192888'653 (192888'234) delete 8fbd32b/benchmark_data_storage0111-ib.pocstor_11336_object15924/head//1 by client.1312877.0:43208 2014-08-11 16:27:11.353056 2014-08-13 17:38:02.539660 7fe9bae887a0 20 read_log 192888'654 (192888'235) delete 709995eb/benchmark_data_storage0111-ib.pocstor_11336_object15934/head//1 by client.1312877.0:43218 2014-08-11 16:27:11.355462 2014-08-13 17:38:02.539682 7fe9bae887a0 20 read_log 192888'655 (192888'236) delete 4f2410eb/benchmark_data_storage0111-ib.pocstor_11336_object15947/head//1 by client.1312877.0:43231 2014-08-11 16:27:11.359137 2014-08-13 17:38:02.539704 7fe9bae887a0 20 read_log 192888'656 (192888'237) delete 4fadd6ab/benchmark_data_storage0111-ib.pocstor_11336_object16038/head//1 by client.1312877.0:43322 2014-08-11 16:27:11.380007 2014-08-13 17:38:02.539726 7fe9bae887a0 20 read_log 192888'657 (192888'238) delete 5b14ab6b/benchmark_data_storage0111-ib.pocstor_11336_object16162/head//1 by client.1312877.0:43446 2014-08-11 16:27:11.414025 2014-08-13 17:38:02.539747 7fe9bae887a0 20 read_log 192888'658 (192888'239) delete 6ab0302b/benchmark_data_storage0111-ib.pocstor_11336_object16219/head//1 by client.1312877.0:43503 2014-08-11 16:27:11.428505 2014-08-13 17:38:02.539780 7fe9bae887a0 20 read_log 192888'659 (192888'240) delete b76318eb/benchmark_data_storage0111-ib.pocstor_11336_object16290/head//1 by client.1312877.0:43574 2014-08-11 16:27:11.446474 2014-08-13 17:38:02.539808 7fe9bae887a0 20 read_log 192888'660 (192888'241) delete 89d8c5eb/benchmark_data_storage0111-ib.pocstor_11336_object16371/head//1 by client.1312877.0:43655 2014-08-11 16:27:11.468964 2014-08-13 17:38:02.539830 7fe9bae887a0 20 read_log 192888'661 (192888'242) delete 88c4ec2b/benchmark_data_storage0111-ib.pocstor_11336_object16404/head//1 by client.1312877.0:43688 2014-08-11 16:27:11.478367 2014-08-13 17:38:02.539852 7fe9bae887a0 20 read_log 192888'662 (192888'243) delete d2930fab/benchmark_data_storage0111-ib.pocstor_11336_object16420/head//1 by client.1312877.0:43704 2014-08-11 16:27:11.482998 2014-08-13 17:38:02.539873 7fe9bae887a0 20 read_log 192888'663 (192888'244) delete 4bcd04eb/benchmark_data_storage0111-ib.pocstor_11336_object16464/head//1 by client.1312877.0:43748 2014-08-11 16:27:11.493244 2014-08-13 17:38:02.539895 7fe9bae887a0 20 read_log 192888'664 (192888'245) delete f5411b6b/benchmark_data_storage0111-ib.pocstor_11336_object16539/head//1 by client.1312877.0:43823 2014-08-11 16:27:11.513397 2014-08-13 17:38:02.539917 7fe9bae887a0 20 read_log 192888'665 (192888'246) delete 5b8992ab/benchmark_data_storage0111-ib.pocstor_11336_object16582/head//1 by client.1312877.0:43866 2014-08-11 16:27:11.526696 2014-08-13 17:38:02.539939 7fe9bae887a0 20 read_log 192888'666 (192888'247) delete 968ceeeb/benchmark_data_storage0111-ib.pocstor_11336_object16597/head//1 by client.1312877.0:43881 2014-08-11 16:27:11.530468 2014-08-13 17:38:02.539961 7fe9bae887a0 20 read_log 192888'667 (192888'248) delete 478cf8eb/benchmark_data_storage0111-ib.pocstor_11336_object16708/head//1 by client.1312877.0:43992 2014-08-11 16:27:11.557651 2014-08-13 17:38:02.539983 7fe9bae887a0 20 read_log 192888'668 (192888'249) delete e3afee6b/benchmark_data_storage0111-ib.pocstor_11336_object16711/head//1 by client.1312877.0:43995 2014-08-11 16:27:11.559620 2014-08-13 17:38:02.540004 7fe9bae887a0 20 read_log 192888'669 (192888'250) delete b42d546b/benchmark_data_storage0111-ib.pocstor_11336_object16737/head//1 by client.1312877.0:44021 2014-08-11 16:27:11.565532 2014-08-13 17:38:02.540027 7fe9bae887a0 20 read_log 192888'670 (192888'251) delete 6b39d7ab/benchmark_data_storage0111-ib.pocstor_11336_object16757/head//1 by client.1312877.0:44041 2014-08-11 16:27:11.571703 2014-08-13 17:38:02.540051 7fe9bae887a0 20 read_log 192888'671 (192888'252) delete 6c846b6b/benchmark_data_storage0111-ib.pocstor_11336_object16841/head//1 by client.1312877.0:44125 2014-08-11 16:27:11.594960 2014-08-13 17:38:02.540073 7fe9bae887a0 20 read_log 192888'672 (192888'253) delete 15deb2eb/benchmark_data_storage0111-ib.pocstor_11336_object16904/head//1 by client.1312877.0:44188 2014-08-11 16:27:11.610823 2014-08-13 17:38:02.540095 7fe9bae887a0 20 read_log 192888'673 (192888'254) delete 16e9aaeb/benchmark_data_storage0111-ib.pocstor_11336_object17042/head//1 by client.1312877.0:44326 2014-08-11 16:27:11.645859 2014-08-13 17:38:02.540118 7fe9bae887a0 20 read_log 192888'674 (192888'255) delete d0004aeb/benchmark_data_storage0111-ib.pocstor_11336_object17045/head//1 by client.1312877.0:44329 2014-08-11 16:27:11.646718 2014-08-13 17:38:02.540141 7fe9bae887a0 20 read_log 192888'675 (192888'256) delete 96728e6b/benchmark_data_storage0111-ib.pocstor_11336_object17143/head//1 by client.1312877.0:44427 2014-08-11 16:27:11.671980 2014-08-13 17:38:02.540173 7fe9bae887a0 20 read_log 192888'676 (192888'257) delete f7e3ef2b/benchmark_data_storage0111-ib.pocstor_11336_object17271/head//1 by client.1312877.0:44555 2014-08-11 16:27:11.708031 2014-08-13 17:38:02.540196 7fe9bae887a0 20 read_log 192888'677 (192888'258) delete cb867cab/benchmark_data_storage0111-ib.pocstor_11336_object17286/head//1 by client.1312877.0:44570 2014-08-11 16:27:11.712021 2014-08-13 17:38:02.540218 7fe9bae887a0 20 read_log 192888'678 (192888'259) delete 286a36b/benchmark_data_storage0111-ib.pocstor_11336_object17353/head//1 by client.1312877.0:44637 2014-08-11 16:27:11.728949 2014-08-13 17:38:02.540251 7fe9bae887a0 20 read_log 192888'679 (192888'260) delete 57509cab/benchmark_data_storage0111-ib.pocstor_11336_object17406/head//1 by client.1312877.0:44690 2014-08-11 16:27:11.743606 2014-08-13 17:38:02.540274 7fe9bae887a0 20 read_log 192888'680 (192888'261) delete 1978a66b/benchmark_data_storage0111-ib.pocstor_11336_object17504/head//1 by client.1312877.0:44788 2014-08-11 16:27:11.770086 2014-08-13 17:38:02.540296 7fe9bae887a0 20 read_log 192888'681 (192888'262) delete 1ff709eb/benchmark_data_storage0111-ib.pocstor_11336_object17536/head//1 by client.1312877.0:44820 2014-08-11 16:27:11.777013 2014-08-13 17:38:02.540322 7fe9bae887a0 20 read_log 192888'682 (192888'263) delete 504892ab/benchmark_data_storage0111-ib.pocstor_11336_object17552/head//1 by client.1312877.0:44836 2014-08-11 16:27:11.781393 2014-08-13 17:38:02.540345 7fe9bae887a0 20 read_log 192888'683 (192888'264) delete aa8340ab/benchmark_data_storage0111-ib.pocstor_11336_object17582/head//1 by client.1312877.0:44866 2014-08-11 16:27:11.788763 2014-08-13 17:38:02.540367 7fe9bae887a0 20 read_log 192888'684 (192888'265) delete b666282b/benchmark_data_storage0111-ib.pocstor_11336_object17624/head//1 by client.1312877.0:44908 2014-08-11 16:27:11.798476 2014-08-13 17:38:02.540389 7fe9bae887a0 20 read_log 192888'685 (192888'266) delete 792ade6b/benchmark_data_storage0111-ib.pocstor_11336_object17776/head//1 by client.1312877.0:45060 2014-08-11 16:27:11.836781 2014-08-13 17:38:02.540410 7fe9bae887a0 20 read_log 192888'686 (192888'267) delete e4e4e3ab/benchmark_data_storage0111-ib.pocstor_11336_object17791/head//1 by client.1312877.0:45075 2014-08-11 16:27:11.840190 2014-08-13 17:38:02.540432 7fe9bae887a0 20 read_log 192888'687 (192888'268) delete e22515eb/benchmark_data_storage0111-ib.pocstor_11336_object17812/head//1 by client.1312877.0:45096 2014-08-11 16:27:11.845145 2014-08-13 17:38:02.540454 7fe9bae887a0 20 read_log 192888'688 (192888'269) delete 378d10eb/benchmark_data_storage0111-ib.pocstor_11336_object17841/head//1 by client.1312877.0:45125 2014-08-11 16:27:11.851959 2014-08-13 17:38:02.540476 7fe9bae887a0 20 read_log 192888'689 (192888'270) delete a1c1b56b/benchmark_data_storage0111-ib.pocstor_11336_object17879/head//1 by client.1312877.0:45163 2014-08-11 16:27:11.860414 2014-08-13 17:38:02.540499 7fe9bae887a0 20 read_log 192888'690 (192888'271) delete 2d29506b/benchmark_data_storage0111-ib.pocstor_11336_object18047/head//1 by client.1312877.0:45331 2014-08-11 16:27:11.901604 2014-08-13 17:38:02.540521 7fe9bae887a0 20 read_log 192888'691 (192888'272) delete a954006b/benchmark_data_storage0111-ib.pocstor_11336_object18067/head//1 by client.1312877.0:45351 2014-08-11 16:27:11.906077 2014-08-13 17:38:02.540543 7fe9bae887a0 20 read_log 192888'692 (192888'273) delete 3ff41aeb/benchmark_data_storage0111-ib.pocstor_11336_object18232/head//1 by client.1312877.0:45516 2014-08-11 16:27:11.947543 2014-08-13 17:38:02.540566 7fe9bae887a0 20 read_log 192888'693 (192888'274) delete 1953402b/benchmark_data_storage0111-ib.pocstor_11336_object18323/head//1 by client.1312877.0:45607 2014-08-11 16:27:11.971845 2014-08-13 17:38:02.540588 7fe9bae887a0 20 read_log 192888'694 (192888'275) delete c2fa4d2b/benchmark_data_storage0111-ib.pocstor_11336_object18387/head//1 by client.1312877.0:45671 2014-08-11 16:27:11.986546 2014-08-13 17:38:02.540611 7fe9bae887a0 20 read_log 192888'695 (192888'276) delete 9efed92b/benchmark_data_storage0111-ib.pocstor_11336_object18517/head//1 by client.1312877.0:45801 2014-08-11 16:27:12.016387 2014-08-13 17:38:02.540638 7fe9bae887a0 20 read_log 192888'696 (192888'277) delete 1846c0eb/benchmark_data_storage0111-ib.pocstor_11336_object18527/head//1 by client.1312877.0:45811 2014-08-11 16:27:12.019480 2014-08-13 17:38:02.540663 7fe9bae887a0 20 read_log 192888'697 (192888'278) delete 275089ab/benchmark_data_storage0111-ib.pocstor_11336_object18528/head//1 by client.1312877.0:45812 2014-08-11 16:27:12.019664 2014-08-13 17:38:02.540685 7fe9bae887a0 20 read_log 192888'698 (192888'279) delete bcd079eb/benchmark_data_storage0111-ib.pocstor_11336_object18542/head//1 by client.1312877.0:45826 2014-08-11 16:27:12.023626 2014-08-13 17:38:02.540720 7fe9bae887a0 20 read_log 192888'699 (192888'280) delete 52868e6b/benchmark_data_storage0111-ib.pocstor_11336_object18592/head//1 by client.1312877.0:45876 2014-08-11 16:27:12.036502 2014-08-13 17:38:02.540748 7fe9bae887a0 20 read_log 192888'700 (192888'281) delete 426531eb/benchmark_data_storage0111-ib.pocstor_11336_object18604/head//1 by client.1312877.0:45888 2014-08-11 16:27:12.039485 2014-08-13 17:38:02.540771 7fe9bae887a0 20 read_log 192888'701 (192888'282) delete 2aad1c6b/benchmark_data_storage0111-ib.pocstor_11336_object18663/head//1 by client.1312877.0:45947 2014-08-11 16:27:12.052571 2014-08-13 17:38:02.540793 7fe9bae887a0 20 read_log 192888'702 (192888'283) delete 5869a02b/benchmark_data_storage0111-ib.pocstor_11336_object18781/head//1 by client.1312877.0:46065 2014-08-11 16:27:12.081703 2014-08-13 17:38:02.540815 7fe9bae887a0 20 read_log 192888'703 (192888'284) delete 4187e2eb/benchmark_data_storage0111-ib.pocstor_11336_object18834/head//1 by client.1312877.0:46118 2014-08-11 16:27:12.093643 2014-08-13 17:38:02.540870 7fe9bae887a0 20 read_log 192888'704 (192888'285) delete 4f44796b/benchmark_data_storage0111-ib.pocstor_11336_object18875/head//1 by client.1312877.0:46159 2014-08-11 16:27:12.105002 2014-08-13 17:38:02.540928 7fe9bae887a0 20 read_log 192888'705 (192888'286) delete fd4b836b/benchmark_data_storage0111-ib.pocstor_11336_object18888/head//1 by client.1312877.0:46172 2014-08-11 16:27:12.108257 2014-08-13 17:38:02.540951 7fe9bae887a0 20 read_log 192888'706 (192888'287) delete 2b80b52b/benchmark_data_storage0111-ib.pocstor_11336_object18990/head//1 by client.1312877.0:46274 2014-08-11 16:27:12.133362 2014-08-13 17:38:02.540973 7fe9bae887a0 20 read_log 192888'707 (192888'288) delete 1c3083ab/benchmark_data_storage0111-ib.pocstor_11336_object19003/head//1 by client.1312877.0:46287 2014-08-11 16:27:12.136537 2014-08-13 17:38:02.540995 7fe9bae887a0 20 read_log 192888'708 (192888'289) delete c2464ceb/benchmark_data_storage0111-ib.pocstor_11336_object19103/head//1 by client.1312877.0:46387 2014-08-11 16:27:12.160144 2014-08-13 17:38:02.541017 7fe9bae887a0 20 read_log 192888'709 (192888'290) delete 122a33ab/benchmark_data_storage0111-ib.pocstor_11336_object19227/head//1 by client.1312877.0:46511 2014-08-11 16:27:12.190816 2014-08-13 17:38:02.541039 7fe9bae887a0 20 read_log 192888'710 (192888'291) delete e7ba396b/benchmark_data_storage0111-ib.pocstor_11336_object19274/head//1 by client.1312877.0:46558 2014-08-11 16:27:12.202176 2014-08-13 17:38:02.541060 7fe9bae887a0 20 read_log 192888'711 (192888'292) delete 9bd5cdeb/benchmark_data_storage0111-ib.pocstor_11336_object19328/head//1 by client.1312877.0:46612 2014-08-11 16:27:12.216483 2014-08-13 17:38:02.541083 7fe9bae887a0 20 read_log 192888'712 (192888'293) delete d0eba76b/benchmark_data_storage0111-ib.pocstor_11336_object19418/head//1 by client.1312877.0:46702 2014-08-11 16:27:12.239150 2014-08-13 17:38:02.541105 7fe9bae887a0 20 read_log 192888'713 (192888'294) delete 1dc295eb/benchmark_data_storage0111-ib.pocstor_11336_object19457/head//1 by client.1312877.0:46741 2014-08-11 16:27:12.248567 2014-08-13 17:38:02.541127 7fe9bae887a0 20 read_log 192888'714 (192888'295) delete 1dd11fab/benchmark_data_storage0111-ib.pocstor_11336_object19469/head//1 by client.1312877.0:46753 2014-08-11 16:27:12.252748 2014-08-13 17:38:02.541149 7fe9bae887a0 20 read_log 192888'715 (192888'296) delete c64d172b/benchmark_data_storage0111-ib.pocstor_11336_object19516/head//1 by client.1312877.0:46800 2014-08-11 16:27:12.267414 2014-08-13 17:38:02.541180 7fe9bae887a0 20 read_log 192888'716 (192888'297) delete 475791eb/benchmark_data_storage0111-ib.pocstor_11336_object19587/head//1 by client.1312877.0:46871 2014-08-11 16:27:12.283858 2014-08-13 17:38:02.541203 7fe9bae887a0 20 read_log 192888'717 (192888'298) delete e2681e6b/benchmark_data_storage0111-ib.pocstor_11336_object19629/head//1 by client.1312877.0:46913 2014-08-11 16:27:12.295019 2014-08-13 17:38:02.541225 7fe9bae887a0 20 read_log 192888'718 (192888'299) delete e272fb6b/benchmark_data_storage0111-ib.pocstor_11336_object19824/head//1 by client.1312877.0:47108 2014-08-11 16:27:12.348318 2014-08-13 17:38:02.541258 7fe9bae887a0 20 read_log 192888'719 (192888'300) delete 24e902b/benchmark_data_storage0111-ib.pocstor_11336_object19842/head//1 by client.1312877.0:47126 2014-08-11 16:27:12.352277 2014-08-13 17:38:02.541288 7fe9bae887a0 20 read_log 192888'720 (192888'301) delete d95a362b/benchmark_data_storage0111-ib.pocstor_11336_object19869/head//1 by client.1312877.0:47153 2014-08-11 16:27:12.359775 2014-08-13 17:38:02.541311 7fe9bae887a0 20 read_log 192888'721 (192888'302) delete 46edc8eb/benchmark_data_storage0111-ib.pocstor_11336_object19881/head//1 by client.1312877.0:47165 2014-08-11 16:27:12.362302 2014-08-13 17:38:02.541333 7fe9bae887a0 20 read_log 192888'722 (192888'303) delete 17820beb/benchmark_data_storage0111-ib.pocstor_11336_object19886/head//1 by client.1312877.0:47170 2014-08-11 16:27:12.363323 2014-08-13 17:38:02.541354 7fe9bae887a0 20 read_log 192888'723 (192888'304) delete 86a8e56b/benchmark_data_storage0111-ib.pocstor_11336_object19954/head//1 by client.1312877.0:47238 2014-08-11 16:27:12.381422 2014-08-13 17:38:02.541376 7fe9bae887a0 20 read_log 192888'724 (192888'305) delete 68ddf0ab/benchmark_data_storage0111-ib.pocstor_11336_object19987/head//1 by client.1312877.0:47271 2014-08-11 16:27:12.388873 2014-08-13 17:38:02.541399 7fe9bae887a0 20 read_log 192888'725 (192888'306) delete 1fe4d72b/benchmark_data_storage0111-ib.pocstor_11336_object20033/head//1 by client.1312877.0:47317 2014-08-11 16:27:12.399721 2014-08-13 17:38:02.541421 7fe9bae887a0 20 read_log 192888'726 (192888'307) delete a91445eb/benchmark_data_storage0111-ib.pocstor_11336_object20056/head//1 by client.1312877.0:47340 2014-08-11 16:27:12.405372 2014-08-13 17:38:02.541443 7fe9bae887a0 20 read_log 192888'727 (192888'308) delete 7faf2f6b/benchmark_data_storage0111-ib.pocstor_11336_object20068/head//1 by client.1312877.0:47352 2014-08-11 16:27:12.408134 2014-08-13 17:38:02.541466 7fe9bae887a0 20 read_log 192888'728 (192888'309) delete 8b39baab/benchmark_data_storage0111-ib.pocstor_11336_object20099/head//1 by client.1312877.0:47383 2014-08-11 16:27:12.417092 2014-08-13 17:38:02.541487 7fe9bae887a0 20 read_log 192888'729 (192888'310) delete f67c52b/benchmark_data_storage0111-ib.pocstor_11336_object20190/head//1 by client.1312877.0:47474 2014-08-11 16:27:12.439320 2014-08-13 17:38:02.541509 7fe9bae887a0 20 read_log 192888'730 (192888'311) delete 9b83842b/benchmark_data_storage0111-ib.pocstor_11336_object20253/head//1 by client.1312877.0:47537 2014-08-11 16:27:12.454398 2014-08-13 17:38:02.541531 7fe9bae887a0 20 read_log 192888'731 (192888'312) delete 5f94746b/benchmark_data_storage0111-ib.pocstor_11336_object20281/head//1 by client.1312877.0:47565 2014-08-11 16:27:12.460701 2014-08-13 17:38:02.541553 7fe9bae887a0 20 read_log 192888'732 (192888'313) delete f708caab/benchmark_data_storage0111-ib.pocstor_11336_object20467/head//1 by client.1312877.0:47751 2014-08-11 16:27:12.508290 2014-08-13 17:38:02.541574 7fe9bae887a0 20 read_log 192888'733 (192888'314) delete 5b14d96b/benchmark_data_storage0111-ib.pocstor_11336_object20478/head//1 by client.1312877.0:47762 2014-08-11 16:27:12.511251 2014-08-13 17:38:02.541636 7fe9bae887a0 20 read_log 192888'734 (192888'315) delete b820eb2b/benchmark_data_storage0111-ib.pocstor_11336_object20492/head//1 by client.1312877.0:47776 2014-08-11 16:27:12.514511 2014-08-13 17:38:02.541659 7fe9bae887a0 20 read_log 192888'735 (192888'316) delete d119eab/benchmark_data_storage0111-ib.pocstor_11336_object20632/head//1 by client.1312877.0:47916 2014-08-11 16:27:12.549861 2014-08-13 17:38:02.541681 7fe9bae887a0 20 read_log 192888'736 (192888'317) delete 11cba56b/benchmark_data_storage0111-ib.pocstor_11336_object20947/head//1 by client.1312877.0:48231 2014-08-11 16:27:12.635101 2014-08-13 17:38:02.541703 7fe9bae887a0 20 read_log 192888'737 (192888'318) delete 60a1adeb/benchmark_data_storage0111-ib.pocstor_11336_object21180/head//1 by client.1312877.0:48464 2014-08-11 16:27:12.698629 2014-08-13 17:38:02.541725 7fe9bae887a0 20 read_log 192888'738 (192888'319) delete 7125f2ab/benchmark_data_storage0111-ib.pocstor_11336_object21256/head//1 by client.1312877.0:48540 2014-08-11 16:27:12.719218 2014-08-13 17:38:02.541789 7fe9bae887a0 20 read_log 192888'739 (192888'320) delete 7726eb2b/benchmark_data_storage0111-ib.pocstor_11336_object21372/head//1 by client.1312877.0:48656 2014-08-11 16:27:12.749523 2014-08-13 17:38:02.541818 7fe9bae887a0 20 read_log 192888'740 (192888'321) delete 242250ab/benchmark_data_storage0111-ib.pocstor_11336_object21400/head//1 by client.1312877.0:48684 2014-08-11 16:27:12.755612 2014-08-13 17:38:02.541840 7fe9bae887a0 20 read_log 192888'741 (192888'322) delete 54fc23ab/benchmark_data_storage0111-ib.pocstor_11336_object21424/head//1 by client.1312877.0:48708 2014-08-11 16:27:12.763093 2014-08-13 17:38:02.541863 7fe9bae887a0 20 read_log 192888'742 (192888'323) delete e546a22b/benchmark_data_storage0111-ib.pocstor_11336_object21443/head//1 by client.1312877.0:48727 2014-08-11 16:27:12.767557 2014-08-13 17:38:02.541884 7fe9bae887a0 20 read_log 192888'743 (192888'324) delete b69c4a2b/benchmark_data_storage0111-ib.pocstor_11336_object21466/head//1 by client.1312877.0:48750 2014-08-11 16:27:12.773584 2014-08-13 17:38:02.541907 7fe9bae887a0 20 read_log 192888'744 (192888'325) delete 5923652b/benchmark_data_storage0111-ib.pocstor_11336_object21528/head//1 by client.1312877.0:48812 2014-08-11 16:27:12.787682 2014-08-13 17:38:02.541929 7fe9bae887a0 20 read_log 192888'745 (192888'326) delete da17c26b/benchmark_data_storage0111-ib.pocstor_11336_object21763/head//1 by client.1312877.0:49047 2014-08-11 16:27:12.844874 2014-08-13 17:38:02.541953 7fe9bae887a0 20 read_log 192888'746 (192888'327) delete 87148eb/benchmark_data_storage0111-ib.pocstor_11336_object21812/head//1 by client.1312877.0:49096 2014-08-11 16:27:12.856122 2014-08-13 17:38:02.541976 7fe9bae887a0 20 read_log 192888'747 (192888'328) delete 9101e96b/benchmark_data_storage0111-ib.pocstor_11336_object21868/head//1 by client.1312877.0:49152 2014-08-11 16:27:12.872203 2014-08-13 17:38:02.542000 7fe9bae887a0 20 read_log 192888'748 (192888'329) delete 7c4fe4ab/benchmark_data_storage0111-ib.pocstor_11336_object21885/head//1 by client.1312877.0:49169 2014-08-11 16:27:12.876884 2014-08-13 17:38:02.542022 7fe9bae887a0 20 read_log 192888'749 (192888'330) delete ad24d8ab/benchmark_data_storage0111-ib.pocstor_11336_object21935/head//1 by client.1312877.0:49219 2014-08-11 16:27:12.889064 2014-08-13 17:38:02.542044 7fe9bae887a0 20 read_log 192888'750 (192888'331) delete 787deceb/benchmark_data_storage0111-ib.pocstor_11336_object21956/head//1 by client.1312877.0:49240 2014-08-11 16:27:12.895379 2014-08-13 17:38:02.542065 7fe9bae887a0 20 read_log 192888'751 (192888'332) delete f978002b/benchmark_data_storage0111-ib.pocstor_11336_object21997/head//1 by client.1312877.0:49281 2014-08-11 16:27:12.905336 2014-08-13 17:38:02.542087 7fe9bae887a0 20 read_log 192888'752 (192888'333) delete 959dd26b/benchmark_data_storage0111-ib.pocstor_11336_object22000/head//1 by client.1312877.0:49284 2014-08-11 16:27:12.907143 2014-08-13 17:38:02.542109 7fe9bae887a0 20 read_log 192888'753 (192888'334) delete 4e472ceb/benchmark_data_storage0111-ib.pocstor_11336_object22034/head//1 by client.1312877.0:49318 2014-08-11 16:27:12.914790 2014-08-13 17:38:02.542130 7fe9bae887a0 20 read_log 192888'754 (192888'335) delete 6c1ac06b/benchmark_data_storage0111-ib.pocstor_11336_object22051/head//1 by client.1312877.0:49335 2014-08-11 16:27:12.918804 2014-08-13 17:38:02.542152 7fe9bae887a0 20 read_log 192888'755 (192888'336) delete 373a052b/benchmark_data_storage0111-ib.pocstor_11336_object22091/head//1 by client.1312877.0:49375 2014-08-11 16:27:12.928357 2014-08-13 17:38:02.542196 7fe9bae887a0 20 read_log 192888'756 (192888'337) delete 31cfc02b/benchmark_data_storage0111-ib.pocstor_11336_object22258/head//1 by client.1312877.0:49542 2014-08-11 16:27:12.967129 2014-08-13 17:38:02.542219 7fe9bae887a0 20 read_log 192888'757 (192888'338) delete 6de7eaab/benchmark_data_storage0111-ib.pocstor_11336_object22272/head//1 by client.1312877.0:49556 2014-08-11 16:27:12.971208 2014-08-13 17:38:02.542241 7fe9bae887a0 20 read_log 192888'758 (192888'339) delete 1781c26b/benchmark_data_storage0111-ib.pocstor_11336_object22296/head//1 by client.1312877.0:49580 2014-08-11 16:27:12.977309 2014-08-13 17:38:02.542275 7fe9bae887a0 20 read_log 192888'759 (192888'340) delete f157deab/benchmark_data_storage0111-ib.pocstor_11336_object22346/head//1 by client.1312877.0:49630 2014-08-11 16:27:12.988370 2014-08-13 17:38:02.542302 7fe9bae887a0 20 read_log 192888'760 (192888'341) delete f68908eb/benchmark_data_storage0111-ib.pocstor_11336_object22441/head//1 by client.1312877.0:49725 2014-08-11 16:27:13.011609 2014-08-13 17:38:02.542324 7fe9bae887a0 20 read_log 192888'761 (192888'342) delete a20011ab/benchmark_data_storage0111-ib.pocstor_11336_object22467/head//1 by client.1312877.0:49751 2014-08-11 16:27:13.017487 2014-08-13 17:38:02.542347 7fe9bae887a0 20 read_log 192888'762 (192888'343) delete 82490b2b/benchmark_data_storage0111-ib.pocstor_11336_object22469/head//1 by client.1312877.0:49753 2014-08-11 16:27:13.018165 2014-08-13 17:38:02.542369 7fe9bae887a0 20 read_log 192888'763 (192888'344) delete 520cf2eb/benchmark_data_storage0111-ib.pocstor_11336_object22473/head//1 by client.1312877.0:49757 2014-08-11 16:27:13.019544 2014-08-13 17:38:02.542391 7fe9bae887a0 20 read_log 192888'764 (192888'345) delete 42e912ab/benchmark_data_storage0111-ib.pocstor_11336_object22545/head//1 by client.1312877.0:49829 2014-08-11 16:27:13.037029 2014-08-13 17:38:02.542413 7fe9bae887a0 20 read_log 192888'765 (192888'346) delete 96bbf22b/benchmark_data_storage0111-ib.pocstor_11336_object22579/head//1 by client.1312877.0:49863 2014-08-11 16:27:13.046206 2014-08-13 17:38:02.542434 7fe9bae887a0 20 read_log 192888'766 (192888'347) delete a0f15c6b/benchmark_data_storage0111-ib.pocstor_11336_object22587/head//1 by client.1312877.0:49871 2014-08-11 16:27:13.047879 2014-08-13 17:38:02.542456 7fe9bae887a0 20 read_log 192888'767 (192888'348) delete 698c546b/benchmark_data_storage0111-ib.pocstor_11336_object22636/head//1 by client.1312877.0:49920 2014-08-11 16:27:13.060644 2014-08-13 17:38:02.542478 7fe9bae887a0 20 read_log 192888'768 (192888'349) delete a9a305eb/benchmark_data_storage0111-ib.pocstor_11336_object22716/head//1 by client.1312877.0:50000 2014-08-11 16:27:13.082160 2014-08-13 17:38:02.542503 7fe9bae887a0 20 read_log 192888'769 (192888'350) delete 20cdd2b/benchmark_data_storage0111-ib.pocstor_11336_object22738/head//1 by client.1312877.0:50022 2014-08-11 16:27:13.087488 2014-08-13 17:38:02.542526 7fe9bae887a0 20 read_log 192888'770 (192888'351) delete 13e43f2b/benchmark_data_storage0111-ib.pocstor_11336_object22869/head//1 by client.1312877.0:50153 2014-08-11 16:27:13.123032 2014-08-13 17:38:02.542548 7fe9bae887a0 20 read_log 192888'771 (192888'352) delete efa1d42b/benchmark_data_storage0111-ib.pocstor_11336_object22914/head//1 by client.1312877.0:50198 2014-08-11 16:27:13.134496 2014-08-13 17:38:02.542570 7fe9bae887a0 20 read_log 192888'772 (192888'353) delete 96aeefeb/benchmark_data_storage0111-ib.pocstor_11336_object22932/head//1 by client.1312877.0:50216 2014-08-11 16:27:13.139077 2014-08-13 17:38:02.542592 7fe9bae887a0 20 read_log 192888'773 (192888'354) delete 1a986b6b/benchmark_data_storage0111-ib.pocstor_11336_object22937/head//1 by client.1312877.0:50221 2014-08-11 16:27:13.140583 2014-08-13 17:38:02.542613 7fe9bae887a0 20 read_log 192888'774 (192888'355) delete a0706d2b/benchmark_data_storage0111-ib.pocstor_11336_object22960/head//1 by client.1312877.0:50244 2014-08-11 16:27:13.147693 2014-08-13 17:38:02.542635 7fe9bae887a0 20 read_log 192888'775 (192888'356) delete 8e2579ab/benchmark_data_storage0111-ib.pocstor_11336_object22962/head//1 by client.1312877.0:50246 2014-08-11 16:27:13.148019 2014-08-13 17:38:02.542660 7fe9bae887a0 20 read_log 192888'776 (192888'357) delete 7ebd15eb/benchmark_data_storage0111-ib.pocstor_11336_object23023/head//1 by client.1312877.0:50307 2014-08-11 16:27:13.161802 2014-08-13 17:38:02.542682 7fe9bae887a0 20 read_log 192888'777 (192888'358) delete 53266aeb/benchmark_data_storage0111-ib.pocstor_11336_object23033/head//1 by client.1312877.0:50317 2014-08-11 16:27:13.164869 2014-08-13 17:38:02.542708 7fe9bae887a0 20 read_log 192888'778 (192888'359) delete 45b50ab/benchmark_data_storage0111-ib.pocstor_11336_object23056/head//1 by client.1312877.0:50340 2014-08-11 16:27:13.169617 2014-08-13 17:38:02.542741 7fe9bae887a0 20 read_log 192888'779 (192888'360) delete 686380ab/benchmark_data_storage0111-ib.pocstor_11336_object23170/head//1 by client.1312877.0:50454 2014-08-11 16:27:13.198670 2014-08-13 17:38:02.542768 7fe9bae887a0 20 read_log 192888'780 (192888'361) delete a62bfc6b/benchmark_data_storage0111-ib.pocstor_11336_object23205/head//1 by client.1312877.0:50489 2014-08-11 16:27:13.206738 2014-08-13 17:38:02.542790 7fe9bae887a0 20 read_log 192888'781 (192888'362) delete b6784d6b/benchmark_data_storage0111-ib.pocstor_11336_object23214/head//1 by client.1312877.0:50498 2014-08-11 16:27:13.208506 2014-08-13 17:38:02.542812 7fe9bae887a0 20 read_log 192888'782 (192888'363) delete e4717fab/benchmark_data_storage0111-ib.pocstor_11336_object23363/head//1 by client.1312877.0:50647 2014-08-11 16:27:13.247860 2014-08-13 17:38:02.542834 7fe9bae887a0 20 read_log 192888'783 (192888'364) delete 793e702b/benchmark_data_storage0111-ib.pocstor_11336_object23422/head//1 by client.1312877.0:50706 2014-08-11 16:27:13.262483 2014-08-13 17:38:02.542856 7fe9bae887a0 20 read_log 192888'784 (192888'365) delete 90f044eb/benchmark_data_storage0111-ib.pocstor_11336_object23495/head//1 by client.1312877.0:50779 2014-08-11 16:27:13.278767 2014-08-13 17:38:02.542877 7fe9bae887a0 20 read_log 192888'785 (192888'366) delete 30bf61ab/benchmark_data_storage0111-ib.pocstor_11336_object23554/head//1 by client.1312877.0:50838 2014-08-11 16:27:13.293079 2014-08-13 17:38:02.542899 7fe9bae887a0 20 read_log 192888'786 (192888'367) delete c494546b/benchmark_data_storage0111-ib.pocstor_11336_object23580/head//1 by client.1312877.0:50864 2014-08-11 16:27:13.299444 2014-08-13 17:38:02.542920 7fe9bae887a0 20 read_log 192888'787 (192888'368) delete 60e4f62b/benchmark_data_storage0111-ib.pocstor_11336_object23637/head//1 by client.1312877.0:50921 2014-08-11 16:27:13.311435 2014-08-13 17:38:02.542942 7fe9bae887a0 20 read_log 192888'788 (192888'369) delete 4c4bf4ab/benchmark_data_storage0111-ib.pocstor_11336_object23654/head//1 by client.1312877.0:50938 2014-08-11 16:27:13.315264 2014-08-13 17:38:02.542965 7fe9bae887a0 20 read_log 192888'789 (192888'370) delete a946212b/benchmark_data_storage0111-ib.pocstor_11336_object23753/head//1 by client.1312877.0:51037 2014-08-11 16:27:13.337745 2014-08-13 17:38:02.542989 7fe9bae887a0 20 read_log 192888'790 (192888'371) delete a1fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object23786/head//1 by client.1312877.0:51070 2014-08-11 16:27:13.346893 2014-08-13 17:38:02.543016 7fe9bae887a0 20 read_log 192888'791 (192888'372) delete b45acb2b/benchmark_data_storage0111-ib.pocstor_11336_object23815/head//1 by client.1312877.0:51099 2014-08-11 16:27:13.352368 2014-08-13 17:38:02.543038 7fe9bae887a0 20 read_log 192888'792 (192888'373) delete f2e080eb/benchmark_data_storage0111-ib.pocstor_11336_object23867/head//1 by client.1312877.0:51151 2014-08-11 16:27:13.368798 2014-08-13 17:38:02.543064 7fe9bae887a0 20 read_log 192888'793 (192888'374) delete a083866b/benchmark_data_storage0111-ib.pocstor_11336_object23921/head//1 by client.1312877.0:51205 2014-08-11 16:27:13.382086 2014-08-13 17:38:02.543088 7fe9bae887a0 20 read_log 192888'794 (192888'375) delete 8bbc9eeb/benchmark_data_storage0111-ib.pocstor_11336_object23984/head//1 by client.1312877.0:51268 2014-08-11 16:27:13.397936 2014-08-13 17:38:02.543110 7fe9bae887a0 20 read_log 192888'795 (192888'376) delete e7d972b/benchmark_data_storage0111-ib.pocstor_11336_object24002/head//1 by client.1312877.0:51286 2014-08-11 16:27:13.402011 2014-08-13 17:38:02.543133 7fe9bae887a0 20 read_log 192888'796 (192888'377) delete a7b66c6b/benchmark_data_storage0111-ib.pocstor_11336_object24011/head//1 by client.1312877.0:51295 2014-08-11 16:27:13.404012 2014-08-13 17:38:02.543200 7fe9bae887a0 20 read_log 192888'797 (192888'378) delete 8d09746b/benchmark_data_storage0111-ib.pocstor_11336_object24032/head//1 by client.1312877.0:51316 2014-08-11 16:27:13.411714 2014-08-13 17:38:02.543228 7fe9bae887a0 20 read_log 192888'798 (192888'379) delete f96e4feb/benchmark_data_storage0111-ib.pocstor_11336_object24132/head//1 by client.1312877.0:51416 2014-08-11 16:27:13.436529 2014-08-13 17:38:02.543263 7fe9bae887a0 20 read_log 192888'799 (192888'380) delete 12d73bab/benchmark_data_storage0111-ib.pocstor_11336_object24218/head//1 by client.1312877.0:51502 2014-08-11 16:27:13.461534 2014-08-13 17:38:02.543290 7fe9bae887a0 20 read_log 192888'800 (192888'381) delete 9866d8eb/benchmark_data_storage0111-ib.pocstor_11336_object24269/head//1 by client.1312877.0:51553 2014-08-11 16:27:13.475672 2014-08-13 17:38:02.543312 7fe9bae887a0 20 read_log 192888'801 (192888'382) delete de90c7eb/benchmark_data_storage0111-ib.pocstor_11336_object24341/head//1 by client.1312877.0:51625 2014-08-11 16:27:13.492981 2014-08-13 17:38:02.543334 7fe9bae887a0 20 read_log 192888'802 (192888'383) delete 537521eb/benchmark_data_storage0111-ib.pocstor_11336_object24410/head//1 by client.1312877.0:51694 2014-08-11 16:27:13.513164 2014-08-13 17:38:02.543356 7fe9bae887a0 20 read_log 192888'803 (192888'384) delete e6d1e6ab/benchmark_data_storage0111-ib.pocstor_11336_object24502/head//1 by client.1312877.0:51786 2014-08-11 16:27:13.536056 2014-08-13 17:38:02.543379 7fe9bae887a0 20 read_log 192888'804 (192888'385) delete 570d676b/benchmark_data_storage0111-ib.pocstor_11336_object24513/head//1 by client.1312877.0:51797 2014-08-11 16:27:13.538322 2014-08-13 17:38:02.543402 7fe9bae887a0 20 read_log 192888'805 (192888'386) delete 4d3698ab/benchmark_data_storage0111-ib.pocstor_11336_object24640/head//1 by client.1312877.0:51924 2014-08-11 16:27:13.572565 2014-08-13 17:38:02.543424 7fe9bae887a0 20 read_log 192888'806 (192888'387) delete 6fd4ceb/benchmark_data_storage0111-ib.pocstor_11336_object24678/head//1 by client.1312877.0:51962 2014-08-11 16:27:13.580577 2014-08-13 17:38:02.543445 7fe9bae887a0 20 read_log 192888'807 (192888'388) delete dda641eb/benchmark_data_storage0111-ib.pocstor_11336_object24696/head//1 by client.1312877.0:51980 2014-08-11 16:27:13.585828 2014-08-13 17:38:02.543467 7fe9bae887a0 20 read_log 192888'808 (192888'389) delete 3edb27eb/benchmark_data_storage0111-ib.pocstor_11336_object24704/head//1 by client.1312877.0:51988 2014-08-11 16:27:13.587950 2014-08-13 17:38:02.543489 7fe9bae887a0 20 read_log 192888'809 (192888'390) delete c80d832b/benchmark_data_storage0111-ib.pocstor_11336_object24729/head//1 by client.1312877.0:52013 2014-08-11 16:27:13.593963 2014-08-13 17:38:02.543511 7fe9bae887a0 20 read_log 192888'810 (192888'391) delete 7fdffdeb/benchmark_data_storage0111-ib.pocstor_11336_object24731/head//1 by client.1312877.0:52015 2014-08-11 16:27:13.594288 2014-08-13 17:38:02.543533 7fe9bae887a0 20 read_log 192888'811 (192888'392) delete fbb74cab/benchmark_data_storage0111-ib.pocstor_11336_object24889/head//1 by client.1312877.0:52173 2014-08-11 16:27:13.635479 2014-08-13 17:38:02.543555 7fe9bae887a0 20 read_log 192888'812 (192888'393) delete abc784eb/benchmark_data_storage0111-ib.pocstor_11336_object24918/head//1 by client.1312877.0:52202 2014-08-11 16:27:13.643201 2014-08-13 17:38:02.543577 7fe9bae887a0 20 read_log 192888'813 (192888'394) delete aa96b5eb/benchmark_data_storage0111-ib.pocstor_11336_object25001/head//1 by client.1312877.0:52285 2014-08-11 16:27:13.664317 2014-08-13 17:38:02.543599 7fe9bae887a0 20 read_log 192888'814 (192888'395) delete f622b72b/benchmark_data_storage0111-ib.pocstor_11336_object25080/head//1 by client.1312877.0:52364 2014-08-11 16:27:13.686984 2014-08-13 17:38:02.543620 7fe9bae887a0 20 read_log 192888'815 (192888'396) delete 42235aeb/benchmark_data_storage0111-ib.pocstor_11336_object25120/head//1 by client.1312877.0:52404 2014-08-11 16:27:13.698099 2014-08-13 17:38:02.543642 7fe9bae887a0 20 read_log 192888'816 (192888'397) delete 965b2cab/benchmark_data_storage0111-ib.pocstor_11336_object25176/head//1 by client.1312877.0:52460 2014-08-11 16:27:13.710829 2014-08-13 17:38:02.543664 7fe9bae887a0 20 read_log 192888'817 (192888'398) delete 802de92b/benchmark_data_storage0111-ib.pocstor_11336_object25189/head//1 by client.1312877.0:52473 2014-08-11 16:27:13.713793 2014-08-13 17:38:02.543686 7fe9bae887a0 20 read_log 192888'818 (192888'399) delete 9409426b/benchmark_data_storage0111-ib.pocstor_11336_object25191/head//1 by client.1312877.0:52475 2014-08-11 16:27:13.715060 2014-08-13 17:38:02.543719 7fe9bae887a0 20 read_log 192888'819 (192888'400) delete c26e8ceb/benchmark_data_storage0111-ib.pocstor_11336_object25320/head//1 by client.1312877.0:52604 2014-08-11 16:27:13.751180 2014-08-13 17:38:02.543746 7fe9bae887a0 20 read_log 192888'820 (192888'401) delete 5bd198eb/benchmark_data_storage0111-ib.pocstor_11336_object25342/head//1 by client.1312877.0:52626 2014-08-11 16:27:13.760748 2014-08-13 17:38:02.543770 7fe9bae887a0 20 read_log 192888'821 (192888'402) delete f2bae72b/benchmark_data_storage0111-ib.pocstor_11336_object25411/head//1 by client.1312877.0:52695 2014-08-11 16:27:13.777063 2014-08-13 17:38:02.543792 7fe9bae887a0 20 read_log 192888'822 (192888'403) delete 22b339ab/benchmark_data_storage0111-ib.pocstor_11336_object25485/head//1 by client.1312877.0:52769 2014-08-11 16:27:13.796341 2014-08-13 17:38:02.543814 7fe9bae887a0 20 read_log 192888'823 (192888'404) delete 4e638a6b/benchmark_data_storage0111-ib.pocstor_11336_object25526/head//1 by client.1312877.0:52810 2014-08-11 16:27:13.804427 2014-08-13 17:38:02.543836 7fe9bae887a0 20 read_log 192888'824 (192888'405) delete 1129b96b/benchmark_data_storage0111-ib.pocstor_11336_object25572/head//1 by client.1312877.0:52856 2014-08-11 16:27:13.815918 2014-08-13 17:38:02.543857 7fe9bae887a0 20 read_log 192888'825 (192888'406) delete 934612eb/benchmark_data_storage0111-ib.pocstor_11336_object25811/head//1 by client.1312877.0:53095 2014-08-11 16:27:13.880971 2014-08-13 17:38:02.543879 7fe9bae887a0 20 read_log 192888'826 (192888'407) delete 8a0e956b/benchmark_data_storage0111-ib.pocstor_11336_object25812/head//1 by client.1312877.0:53096 2014-08-11 16:27:13.881648 2014-08-13 17:38:02.543901 7fe9bae887a0 20 read_log 192888'827 (192888'408) delete 5275656b/benchmark_data_storage0111-ib.pocstor_11336_object25868/head//1 by client.1312877.0:53152 2014-08-11 16:27:13.895301 2014-08-13 17:38:02.543923 7fe9bae887a0 20 read_log 192888'828 (192888'409) delete 75b001eb/benchmark_data_storage0111-ib.pocstor_11336_object26161/head//1 by client.1312877.0:53445 2014-08-11 16:27:13.969101 2014-08-13 17:38:02.543944 7fe9bae887a0 20 read_log 192888'829 (192888'410) delete b69ac26b/benchmark_data_storage0111-ib.pocstor_11336_object26351/head//1 by client.1312877.0:53635 2014-08-11 16:27:14.016727 2014-08-13 17:38:02.543966 7fe9bae887a0 20 read_log 192888'830 (192888'411) delete 784d40ab/benchmark_data_storage0111-ib.pocstor_11336_object26478/head//1 by client.1312877.0:53762 2014-08-11 16:27:14.049132 2014-08-13 17:38:02.543988 7fe9bae887a0 20 read_log 192888'831 (192888'412) delete b4b81d2b/benchmark_data_storage0111-ib.pocstor_11336_object26538/head//1 by client.1312877.0:53822 2014-08-11 16:27:14.062870 2014-08-13 17:38:02.544011 7fe9bae887a0 20 read_log 192888'832 (192888'413) delete 940ab0ab/benchmark_data_storage0111-ib.pocstor_11336_object26596/head//1 by client.1312877.0:53880 2014-08-11 16:27:14.077266 2014-08-13 17:38:02.544037 7fe9bae887a0 20 read_log 192888'833 (192888'414) delete f7f71f6b/benchmark_data_storage0111-ib.pocstor_11336_object26678/head//1 by client.1312877.0:53962 2014-08-11 16:27:14.097705 2014-08-13 17:38:02.544060 7fe9bae887a0 20 read_log 192888'834 (192888'415) delete 2b79936b/benchmark_data_storage0111-ib.pocstor_11336_object26822/head//1 by client.1312877.0:54106 2014-08-11 16:27:14.132971 2014-08-13 17:38:02.544086 7fe9bae887a0 20 read_log 192888'835 (192888'416) delete 9d858b2b/benchmark_data_storage0111-ib.pocstor_11336_object26873/head//1 by client.1312877.0:54157 2014-08-11 16:27:14.144678 2014-08-13 17:38:02.544108 7fe9bae887a0 20 read_log 192888'836 (192888'417) delete d46b0eab/benchmark_data_storage0111-ib.pocstor_11336_object26900/head//1 by client.1312877.0:54184 2014-08-11 16:27:14.151013 2014-08-13 17:38:02.544134 7fe9bae887a0 20 read_log 192888'837 (192888'418) delete 3596656b/benchmark_data_storage0111-ib.pocstor_11336_object27149/head//1 by client.1312877.0:54433 2014-08-11 16:27:14.211365 2014-08-13 17:38:02.544164 7fe9bae887a0 20 read_log 192888'838 (192888'419) delete 800c4c2b/benchmark_data_storage0111-ib.pocstor_11336_object27214/head//1 by client.1312877.0:54498 2014-08-11 16:27:14.225984 2014-08-13 17:38:02.544204 7fe9bae887a0 20 read_log 194215'839 (0'0) modify 6fb923eb/benchmark_data_storage0111-ib.pocstor_19374_object0/head//1 by client.1048726.0:1 2014-08-12 17:10:00.590828 2014-08-13 17:38:02.544232 7fe9bae887a0 20 read_log 194215'840 (0'0) modify f1c09dab/benchmark_data_storage0111-ib.pocstor_19374_object53/head//1 by client.1048726.0:54 2014-08-12 17:10:00.857408 2014-08-13 17:38:02.544294 7fe9bae887a0 20 read_log 194215'841 (0'0) modify 322ce82b/benchmark_data_storage0111-ib.pocstor_19374_object55/head//1 by client.1048726.0:56 2014-08-12 17:10:00.865181 2014-08-13 17:38:02.544319 7fe9bae887a0 20 read_log 194215'842 (0'0) modify 42b2c8ab/benchmark_data_storage0111-ib.pocstor_19374_object77/head//1 by client.1048726.0:78 2014-08-12 17:10:00.949824 2014-08-13 17:38:02.544340 7fe9bae887a0 20 read_log 194215'843 (0'0) modify 3792f3eb/benchmark_data_storage0111-ib.pocstor_19374_object104/head//1 by client.1048726.0:105 2014-08-12 17:10:01.054574 2014-08-13 17:38:02.544363 7fe9bae887a0 20 read_log 194215'844 (0'0) modify c21e90ab/benchmark_data_storage0111-ib.pocstor_19374_object106/head//1 by client.1048726.0:107 2014-08-12 17:10:01.061936 2014-08-13 17:38:02.544388 7fe9bae887a0 20 read_log 194215'845 (0'0) modify f13767eb/benchmark_data_storage0111-ib.pocstor_19374_object112/head//1 by client.1048726.0:113 2014-08-12 17:10:01.084846 2014-08-13 17:38:02.544411 7fe9bae887a0 20 read_log 194215'846 (0'0) modify 90f4b5eb/benchmark_data_storage0111-ib.pocstor_19374_object117/head//1 by client.1048726.0:118 2014-08-12 17:10:01.103934 2014-08-13 17:38:02.544436 7fe9bae887a0 20 read_log 194215'847 (0'0) modify ec96382b/benchmark_data_storage0111-ib.pocstor_19374_object134/head//1 by client.1048726.0:135 2014-08-12 17:10:01.168850 2014-08-13 17:38:02.544458 7fe9bae887a0 20 read_log 194215'848 (0'0) modify 58c1d5eb/benchmark_data_storage0111-ib.pocstor_19374_object146/head//1 by client.1048726.0:147 2014-08-12 17:10:01.214706 2014-08-13 17:38:02.544484 7fe9bae887a0 20 read_log 194215'849 (0'0) modify fb95d6ab/benchmark_data_storage0111-ib.pocstor_19374_object204/head//1 by client.1048726.0:205 2014-08-12 17:10:01.451445 2014-08-13 17:38:02.544506 7fe9bae887a0 20 read_log 194215'850 (0'0) modify b112016b/benchmark_data_storage0111-ib.pocstor_19374_object273/head//1 by client.1048726.0:274 2014-08-12 17:10:01.789491 2014-08-13 17:38:02.544531 7fe9bae887a0 20 read_log 194215'851 (0'0) modify a02a832b/benchmark_data_storage0111-ib.pocstor_19374_object284/head//1 by client.1048726.0:285 2014-08-12 17:10:01.830122 2014-08-13 17:38:02.544553 7fe9bae887a0 20 read_log 194215'852 (0'0) modify a5225cab/benchmark_data_storage0111-ib.pocstor_19374_object539/head//1 by client.1048726.0:540 2014-08-12 17:10:02.825535 2014-08-13 17:38:02.544579 7fe9bae887a0 20 read_log 194215'853 (0'0) modify a60191ab/benchmark_data_storage0111-ib.pocstor_19374_object649/head//1 by client.1048726.0:650 2014-08-12 17:10:03.256870 2014-08-13 17:38:02.544601 7fe9bae887a0 20 read_log 194215'854 (0'0) modify 6fc2f02b/benchmark_data_storage0111-ib.pocstor_19374_object659/head//1 by client.1048726.0:660 2014-08-12 17:10:03.294233 2014-08-13 17:38:02.544626 7fe9bae887a0 20 read_log 194215'855 (0'0) modify 7f48662b/benchmark_data_storage0111-ib.pocstor_19374_object1055/head//1 by client.1048726.0:1056 2014-08-12 17:10:04.742733 2014-08-13 17:38:02.544648 7fe9bae887a0 20 read_log 194215'856 (0'0) modify 17650d2b/benchmark_data_storage0111-ib.pocstor_19374_object1157/head//1 by client.1048726.0:1158 2014-08-12 17:10:05.115254 2014-08-13 17:38:02.544676 7fe9bae887a0 20 read_log 194215'857 (0'0) modify 39bc27ab/benchmark_data_storage0111-ib.pocstor_19374_object1180/head//1 by client.1048726.0:1181 2014-08-12 17:10:05.209249 2014-08-13 17:38:02.544698 7fe9bae887a0 20 read_log 194215'858 (0'0) modify e51879ab/benchmark_data_storage0111-ib.pocstor_19374_object1244/head//1 by client.1048726.0:1245 2014-08-12 17:10:05.444267 2014-08-13 17:38:02.544735 7fe9bae887a0 20 read_log 194215'859 (0'0) modify fefa32eb/benchmark_data_storage0111-ib.pocstor_19374_object1265/head//1 by client.1048726.0:1266 2014-08-12 17:10:05.520797 2014-08-13 17:38:02.544762 7fe9bae887a0 20 read_log 194215'860 (0'0) modify ee0c2aab/benchmark_data_storage0111-ib.pocstor_19374_object1278/head//1 by client.1048726.0:1279 2014-08-12 17:10:05.568368 2014-08-13 17:38:02.544786 7fe9bae887a0 20 read_log 194215'861 (0'0) modify 3560e9ab/benchmark_data_storage0111-ib.pocstor_19374_object1318/head//1 by client.1048726.0:1319 2014-08-12 17:10:05.715945 2014-08-13 17:38:02.544845 7fe9bae887a0 20 read_log 194215'862 (0'0) modify 510c1a6b/benchmark_data_storage0111-ib.pocstor_19374_object1337/head//1 by client.1048726.0:1338 2014-08-12 17:10:05.784135 2014-08-13 17:38:02.544867 7fe9bae887a0 20 read_log 194215'863 (0'0) modify 2298276b/benchmark_data_storage0111-ib.pocstor_19374_object1339/head//1 by client.1048726.0:1340 2014-08-12 17:10:05.791557 2014-08-13 17:38:02.544893 7fe9bae887a0 20 read_log 194215'864 (0'0) modify 6340222b/benchmark_data_storage0111-ib.pocstor_19374_object1349/head//1 by client.1048726.0:1350 2014-08-12 17:10:05.827220 2014-08-13 17:38:02.544919 7fe9bae887a0 20 read_log 194215'865 (0'0) modify 96f019ab/benchmark_data_storage0111-ib.pocstor_19374_object1487/head//1 by client.1048726.0:1488 2014-08-12 17:10:06.346758 2014-08-13 17:38:02.544941 7fe9bae887a0 20 read_log 194215'866 (0'0) modify 309188eb/benchmark_data_storage0111-ib.pocstor_19374_object1517/head//1 by client.1048726.0:1518 2014-08-12 17:10:06.456804 2014-08-13 17:38:02.544967 7fe9bae887a0 20 read_log 194215'867 (0'0) modify 5e9b32eb/benchmark_data_storage0111-ib.pocstor_19374_object1568/head//1 by client.1048726.0:1569 2014-08-12 17:10:06.660441 2014-08-13 17:38:02.544989 7fe9bae887a0 20 read_log 194215'868 (0'0) modify bb2497ab/benchmark_data_storage0111-ib.pocstor_19374_object1632/head//1 by client.1048726.0:1633 2014-08-12 17:10:06.899760 2014-08-13 17:38:02.545014 7fe9bae887a0 20 read_log 194215'869 (0'0) modify cf5e382b/benchmark_data_storage0111-ib.pocstor_19374_object1744/head//1 by client.1048726.0:1745 2014-08-12 17:10:07.316858 2014-08-13 17:38:02.545036 7fe9bae887a0 20 read_log 194215'870 (0'0) modify 3c5c7dab/benchmark_data_storage0111-ib.pocstor_19374_object1756/head//1 by client.1048726.0:1757 2014-08-12 17:10:07.362150 2014-08-13 17:38:02.545061 7fe9bae887a0 20 read_log 194215'871 (0'0) modify 724fd5eb/benchmark_data_storage0111-ib.pocstor_19374_object1757/head//1 by client.1048726.0:1758 2014-08-12 17:10:07.365884 2014-08-13 17:38:02.545083 7fe9bae887a0 20 read_log 194215'872 (0'0) modify 912098ab/benchmark_data_storage0111-ib.pocstor_19374_object1776/head//1 by client.1048726.0:1777 2014-08-12 17:10:07.436954 2014-08-13 17:38:02.545108 7fe9bae887a0 20 read_log 194215'873 (0'0) modify 3939e16b/benchmark_data_storage0111-ib.pocstor_19374_object1894/head//1 by client.1048726.0:1895 2014-08-12 17:10:07.879247 2014-08-13 17:38:02.545130 7fe9bae887a0 20 read_log 194215'874 (0'0) modify 6a0792eb/benchmark_data_storage0111-ib.pocstor_19374_object1896/head//1 by client.1048726.0:1897 2014-08-12 17:10:07.886451 2014-08-13 17:38:02.545161 7fe9bae887a0 20 read_log 194215'875 (0'0) modify 6f099eeb/benchmark_data_storage0111-ib.pocstor_19374_object2059/head//1 by client.1048726.0:2060 2014-08-12 17:10:08.500386 2014-08-13 17:38:02.545188 7fe9bae887a0 20 read_log 194215'876 (0'0) modify eafb902b/benchmark_data_storage0111-ib.pocstor_19374_object2077/head//1 by client.1048726.0:2078 2014-08-12 17:10:08.565170 2014-08-13 17:38:02.545210 7fe9bae887a0 20 read_log 194215'877 (0'0) modify 946c36eb/benchmark_data_storage0111-ib.pocstor_19374_object2102/head//1 by client.1048726.0:2103 2014-08-12 17:10:08.657330 2014-08-13 17:38:02.545235 7fe9bae887a0 20 read_log 194215'878 (0'0) modify f47e62b/benchmark_data_storage0111-ib.pocstor_19374_object2180/head//1 by client.1048726.0:2181 2014-08-12 17:10:08.950520 2014-08-13 17:38:02.545301 7fe9bae887a0 20 read_log 194215'879 (0'0) modify 934f442b/benchmark_data_storage0111-ib.pocstor_19374_object2262/head//1 by client.1048726.0:2263 2014-08-12 17:10:09.262995 2014-08-13 17:38:02.545330 7fe9bae887a0 20 read_log 194215'880 (0'0) modify 7f57086b/benchmark_data_storage0111-ib.pocstor_19374_object2343/head//1 by client.1048726.0:2344 2014-08-12 17:10:09.563691 2014-08-13 17:38:02.545352 7fe9bae887a0 20 read_log 194215'881 (0'0) modify 13d2392b/benchmark_data_storage0111-ib.pocstor_19374_object2352/head//1 by client.1048726.0:2353 2014-08-12 17:10:09.596336 2014-08-13 17:38:02.545373 7fe9bae887a0 20 read_log 194215'882 (0'0) modify 3d536ceb/benchmark_data_storage0111-ib.pocstor_19374_object2361/head//1 by client.1048726.0:2362 2014-08-12 17:10:09.631118 2014-08-13 17:38:02.545395 7fe9bae887a0 20 read_log 194215'883 (0'0) modify c6e9f26b/benchmark_data_storage0111-ib.pocstor_19374_object2394/head//1 by client.1048726.0:2395 2014-08-12 17:10:09.751730 2014-08-13 17:38:02.545418 7fe9bae887a0 20 read_log 194215'884 (0'0) modify 85e6a52b/benchmark_data_storage0111-ib.pocstor_19374_object2418/head//1 by client.1048726.0:2419 2014-08-12 17:10:09.839253 2014-08-13 17:38:02.545444 7fe9bae887a0 20 read_log 194215'885 (0'0) modify c7cc53ab/benchmark_data_storage0111-ib.pocstor_19374_object2432/head//1 by client.1048726.0:2433 2014-08-12 17:10:09.891934 2014-08-13 17:38:02.545472 7fe9bae887a0 20 read_log 194215'886 (0'0) modify 3a7fe4ab/benchmark_data_storage0111-ib.pocstor_19374_object2594/head//1 by client.1048726.0:2595 2014-08-12 17:10:10.492847 2014-08-13 17:38:02.545494 7fe9bae887a0 20 read_log 194215'887 (0'0) modify 4305176b/benchmark_data_storage0111-ib.pocstor_19374_object2684/head//1 by client.1048726.0:2685 2014-08-12 17:10:10.840476 2014-08-13 17:38:02.545516 7fe9bae887a0 20 read_log 194215'888 (0'0) modify e7b7062b/benchmark_data_storage0111-ib.pocstor_19374_object2725/head//1 by client.1048726.0:2726 2014-08-12 17:10:10.991411 2014-08-13 17:38:02.545542 7fe9bae887a0 20 read_log 194215'889 (0'0) modify a1e105ab/benchmark_data_storage0111-ib.pocstor_19374_object2744/head//1 by client.1048726.0:2745 2014-08-12 17:10:11.061577 2014-08-13 17:38:02.545601 7fe9bae887a0 20 read_log 194215'890 (0'0) modify 9bee462b/benchmark_data_storage0111-ib.pocstor_19374_object2763/head//1 by client.1048726.0:2764 2014-08-12 17:10:11.131220 2014-08-13 17:38:02.545630 7fe9bae887a0 20 read_log 194215'891 (0'0) modify 2d10016b/benchmark_data_storage0111-ib.pocstor_19374_object2791/head//1 by client.1048726.0:2792 2014-08-12 17:10:11.234844 2014-08-13 17:38:02.545652 7fe9bae887a0 20 read_log 194215'892 (0'0) modify 7464b42b/benchmark_data_storage0111-ib.pocstor_19374_object2865/head//1 by client.1048726.0:2866 2014-08-12 17:10:11.527280 2014-08-13 17:38:02.545673 7fe9bae887a0 20 read_log 194215'893 (0'0) modify 6a28beb/benchmark_data_storage0111-ib.pocstor_19374_object2901/head//1 by client.1048726.0:2902 2014-08-12 17:10:11.661259 2014-08-13 17:38:02.545768 7fe9bae887a0 20 read_log 194215'894 (0'0) modify df15306b/benchmark_data_storage0111-ib.pocstor_19374_object2944/head//1 by client.1048726.0:2945 2014-08-12 17:10:11.835788 2014-08-13 17:38:02.545793 7fe9bae887a0 20 read_log 194215'895 (0'0) modify 505175ab/benchmark_data_storage0111-ib.pocstor_19374_object2950/head//1 by client.1048726.0:2951 2014-08-12 17:10:11.857698 2014-08-13 17:38:02.545814 7fe9bae887a0 20 read_log 194215'896 (0'0) modify 3fa3deeb/benchmark_data_storage0111-ib.pocstor_19374_object3045/head//1 by client.1048726.0:3046 2014-08-12 17:10:12.212259 2014-08-13 17:38:02.545836 7fe9bae887a0 20 read_log 194215'897 (0'0) modify f1bcb3ab/benchmark_data_storage0111-ib.pocstor_19374_object3073/head//1 by client.1048726.0:3074 2014-08-12 17:10:12.318267 2014-08-13 17:38:02.545857 7fe9bae887a0 20 read_log 194215'898 (0'0) modify 6eb5e6ab/benchmark_data_storage0111-ib.pocstor_19374_object3178/head//1 by client.1048726.0:3179 2014-08-12 17:10:12.723788 2014-08-13 17:38:02.545895 7fe9bae887a0 20 read_log 194215'899 (0'0) modify 71213f6b/benchmark_data_storage0111-ib.pocstor_19374_object3222/head//1 by client.1048726.0:3223 2014-08-12 17:10:12.883970 2014-08-13 17:38:02.545919 7fe9bae887a0 20 read_log 194215'900 (0'0) modify 2847d6ab/benchmark_data_storage0111-ib.pocstor_19374_object3286/head//1 by client.1048726.0:3287 2014-08-12 17:10:13.119342 2014-08-13 17:38:02.545940 7fe9bae887a0 20 read_log 194215'901 (0'0) modify b38c2f6b/benchmark_data_storage0111-ib.pocstor_19374_object3415/head//1 by client.1048726.0:3416 2014-08-12 17:10:13.609279 2014-08-13 17:38:02.545962 7fe9bae887a0 20 read_log 194215'902 (0'0) modify 518a656b/benchmark_data_storage0111-ib.pocstor_19374_object3454/head//1 by client.1048726.0:3455 2014-08-12 17:10:13.755288 2014-08-13 17:38:02.545983 7fe9bae887a0 20 read_log 194215'903 (0'0) modify cad818ab/benchmark_data_storage0111-ib.pocstor_19374_object3466/head//1 by client.1048726.0:3467 2014-08-12 17:10:13.799432 2014-08-13 17:38:02.546005 7fe9bae887a0 20 read_log 194215'904 (0'0) modify 52ed8aab/benchmark_data_storage0111-ib.pocstor_19374_object3511/head//1 by client.1048726.0:3512 2014-08-12 17:10:13.975505 2014-08-13 17:38:02.546026 7fe9bae887a0 20 read_log 194215'905 (0'0) modify 376b872b/benchmark_data_storage0111-ib.pocstor_19374_object3523/head//1 by client.1048726.0:3524 2014-08-12 17:10:14.018984 2014-08-13 17:38:02.546048 7fe9bae887a0 20 read_log 194215'906 (0'0) modify 69cdf92b/benchmark_data_storage0111-ib.pocstor_19374_object3584/head//1 by client.1048726.0:3585 2014-08-12 17:10:14.248686 2014-08-13 17:38:02.546069 7fe9bae887a0 20 read_log 194215'907 (0'0) modify 5dad4cab/benchmark_data_storage0111-ib.pocstor_19374_object3612/head//1 by client.1048726.0:3613 2014-08-12 17:10:14.351100 2014-08-13 17:38:02.546090 7fe9bae887a0 20 read_log 194215'908 (0'0) modify 8a29ddeb/benchmark_data_storage0111-ib.pocstor_19374_object3677/head//1 by client.1048726.0:3678 2014-08-12 17:10:14.592968 2014-08-13 17:38:02.546112 7fe9bae887a0 20 read_log 194215'909 (0'0) modify cb77c3ab/benchmark_data_storage0111-ib.pocstor_19374_object3717/head//1 by client.1048726.0:3718 2014-08-12 17:10:14.742560 2014-08-13 17:38:02.546133 7fe9bae887a0 20 read_log 194215'910 (0'0) modify 7096f52b/benchmark_data_storage0111-ib.pocstor_19374_object3810/head//1 by client.1048726.0:3811 2014-08-12 17:10:15.096844 2014-08-13 17:38:02.546166 7fe9bae887a0 20 read_log 194215'911 (0'0) modify ee1fffeb/benchmark_data_storage0111-ib.pocstor_19374_object3837/head//1 by client.1048726.0:3838 2014-08-12 17:10:15.195941 2014-08-13 17:38:02.546189 7fe9bae887a0 20 read_log 194215'912 (0'0) modify a32a26eb/benchmark_data_storage0111-ib.pocstor_19374_object3838/head//1 by client.1048726.0:3839 2014-08-12 17:10:15.199764 2014-08-13 17:38:02.546211 7fe9bae887a0 20 read_log 194215'913 (0'0) modify 6fb1deeb/benchmark_data_storage0111-ib.pocstor_19374_object4048/head//1 by client.1048726.0:4049 2014-08-12 17:10:15.978347 2014-08-13 17:38:02.546232 7fe9bae887a0 20 read_log 194215'914 (0'0) modify 6c58c7ab/benchmark_data_storage0111-ib.pocstor_19374_object4059/head//1 by client.1048726.0:4060 2014-08-12 17:10:16.027647 2014-08-13 17:38:02.546254 7fe9bae887a0 20 read_log 194215'915 (0'0) modify 660f996b/benchmark_data_storage0111-ib.pocstor_19374_object4105/head//1 by client.1048726.0:4106 2014-08-12 17:10:16.233117 2014-08-13 17:38:02.546276 7fe9bae887a0 20 read_log 194215'916 (0'0) modify 612f78ab/benchmark_data_storage0111-ib.pocstor_19374_object4125/head//1 by client.1048726.0:4126 2014-08-12 17:10:16.346790 2014-08-13 17:38:02.546297 7fe9bae887a0 20 read_log 194215'917 (0'0) modify bd30982b/benchmark_data_storage0111-ib.pocstor_19374_object4136/head//1 by client.1048726.0:4137 2014-08-12 17:10:16.387277 2014-08-13 17:38:02.546318 7fe9bae887a0 20 read_log 194215'918 (0'0) modify d1ec98ab/benchmark_data_storage0111-ib.pocstor_19374_object4183/head//1 by client.1048726.0:4184 2014-08-12 17:10:16.564420 2014-08-13 17:38:02.546352 7fe9bae887a0 20 read_log 194215'919 (0'0) modify cd9ad76b/benchmark_data_storage0111-ib.pocstor_19374_object4187/head//1 by client.1048726.0:4188 2014-08-12 17:10:16.579161 2014-08-13 17:38:02.546375 7fe9bae887a0 20 read_log 194215'920 (0'0) modify 6236346b/benchmark_data_storage0111-ib.pocstor_19374_object4191/head//1 by client.1048726.0:4192 2014-08-12 17:10:16.594033 2014-08-13 17:38:02.546397 7fe9bae887a0 20 read_log 194215'921 (0'0) modify c47eb16b/benchmark_data_storage0111-ib.pocstor_19374_object4242/head//1 by client.1048726.0:4243 2014-08-12 17:10:16.780918 2014-08-13 17:38:02.546418 7fe9bae887a0 20 read_log 194215'922 (0'0) modify 7f4a59eb/benchmark_data_storage0111-ib.pocstor_19374_object4360/head//1 by client.1048726.0:4361 2014-08-12 17:10:17.287713 2014-08-13 17:38:02.546440 7fe9bae887a0 20 read_log 194215'923 (0'0) modify 3ac7842b/benchmark_data_storage0111-ib.pocstor_19374_object4440/head//1 by client.1048726.0:4441 2014-08-12 17:10:17.867496 2014-08-13 17:38:02.546461 7fe9bae887a0 20 read_log 194215'924 (0'0) modify 3be00eeb/benchmark_data_storage0111-ib.pocstor_19374_object4498/head//1 by client.1048726.0:4499 2014-08-12 17:10:18.103629 2014-08-13 17:38:02.546483 7fe9bae887a0 20 read_log 194215'925 (0'0) modify 20baeeab/benchmark_data_storage0111-ib.pocstor_19374_object4674/head//1 by client.1048726.0:4675 2014-08-12 17:10:18.772248 2014-08-13 17:38:02.546505 7fe9bae887a0 20 read_log 194215'926 (0'0) modify ae2e9d2b/benchmark_data_storage0111-ib.pocstor_19374_object4826/head//1 by client.1048726.0:4827 2014-08-12 17:10:19.348107 2014-08-13 17:38:02.546526 7fe9bae887a0 20 read_log 194215'927 (0'0) modify e1d72b2b/benchmark_data_storage0111-ib.pocstor_19374_object4840/head//1 by client.1048726.0:4841 2014-08-12 17:10:19.416924 2014-08-13 17:38:02.546547 7fe9bae887a0 20 read_log 194215'928 (0'0) modify 14b2daeb/benchmark_data_storage0111-ib.pocstor_19374_object4851/head//1 by client.1048726.0:4852 2014-08-12 17:10:19.467367 2014-08-13 17:38:02.546569 7fe9bae887a0 20 read_log 194215'929 (0'0) modify f058a86b/benchmark_data_storage0111-ib.pocstor_19374_object4891/head//1 by client.1048726.0:4892 2014-08-12 17:10:19.634432 2014-08-13 17:38:02.546590 7fe9bae887a0 20 read_log 194215'930 (0'0) modify 1cc0022b/benchmark_data_storage0111-ib.pocstor_19374_object4938/head//1 by client.1048726.0:4939 2014-08-12 17:10:19.950373 2014-08-13 17:38:02.546612 7fe9bae887a0 20 read_log 194215'931 (0'0) modify add0cc2b/benchmark_data_storage0111-ib.pocstor_19374_object5085/head//1 by client.1048726.0:5086 2014-08-12 17:10:20.624221 2014-08-13 17:38:02.546635 7fe9bae887a0 20 read_log 194215'932 (0'0) modify 942a0e2b/benchmark_data_storage0111-ib.pocstor_19374_object5225/head//1 by client.1048726.0:5226 2014-08-12 17:10:21.444376 2014-08-13 17:38:02.546657 7fe9bae887a0 20 read_log 194215'933 (0'0) modify b41983eb/benchmark_data_storage0111-ib.pocstor_19374_object5345/head//1 by client.1048726.0:5346 2014-08-12 17:10:22.029188 2014-08-13 17:38:02.546681 7fe9bae887a0 20 read_log 194215'934 (0'0) modify 6f06a4eb/benchmark_data_storage0111-ib.pocstor_19374_object5356/head//1 by client.1048726.0:5357 2014-08-12 17:10:22.069586 2014-08-13 17:38:02.546703 7fe9bae887a0 20 read_log 194215'935 (0'0) modify a176b/benchmark_data_storage0111-ib.pocstor_19374_object5547/head//1 by client.1048726.0:5548 2014-08-12 17:10:22.811798 2014-08-13 17:38:02.546725 7fe9bae887a0 20 read_log 194215'936 (0'0) modify 2bbd78ab/benchmark_data_storage0111-ib.pocstor_19374_object5562/head//1 by client.1048726.0:5563 2014-08-12 17:10:22.870801 2014-08-13 17:38:02.546747 7fe9bae887a0 20 read_log 194215'937 (0'0) modify 54ab51eb/benchmark_data_storage0111-ib.pocstor_19374_object5571/head//1 by client.1048726.0:5572 2014-08-12 17:10:22.917778 2014-08-13 17:38:02.546771 7fe9bae887a0 20 read_log 194215'938 (0'0) modify 53bf81eb/benchmark_data_storage0111-ib.pocstor_19374_object5590/head//1 by client.1048726.0:5591 2014-08-12 17:10:22.989437 2014-08-13 17:38:02.546803 7fe9bae887a0 20 read_log 194215'939 (0'0) modify 93b6132b/benchmark_data_storage0111-ib.pocstor_19374_object5651/head//1 by client.1048726.0:5652 2014-08-12 17:10:23.226343 2014-08-13 17:38:02.546832 7fe9bae887a0 20 read_log 194215'940 (0'0) modify 2f4423eb/benchmark_data_storage0111-ib.pocstor_19374_object5849/head//1 by client.1048726.0:5850 2014-08-12 17:10:24.092365 2014-08-13 17:38:02.546854 7fe9bae887a0 20 read_log 194215'941 (0'0) modify 37100d2b/benchmark_data_storage0111-ib.pocstor_19374_object5900/head//1 by client.1048726.0:5901 2014-08-12 17:10:24.281509 2014-08-13 17:38:02.546875 7fe9bae887a0 20 read_log 194215'942 (0'0) modify 2dd007ab/benchmark_data_storage0111-ib.pocstor_19374_object5917/head//1 by client.1048726.0:5918 2014-08-12 17:10:24.344833 2014-08-13 17:38:02.546934 7fe9bae887a0 20 read_log 194215'943 (0'0) modify 86a4fcab/benchmark_data_storage0111-ib.pocstor_19374_object5945/head//1 by client.1048726.0:5946 2014-08-12 17:10:24.458547 2014-08-13 17:38:02.546964 7fe9bae887a0 20 read_log 194215'944 (0'0) modify 920492b/benchmark_data_storage0111-ib.pocstor_19374_object5965/head//1 by client.1048726.0:5966 2014-08-12 17:10:24.532532 2014-08-13 17:38:02.546986 7fe9bae887a0 20 read_log 194215'945 (0'0) modify 1055bb2b/benchmark_data_storage0111-ib.pocstor_19374_object5969/head//1 by client.1048726.0:5970 2014-08-12 17:10:24.547212 2014-08-13 17:38:02.547009 7fe9bae887a0 20 read_log 194215'946 (0'0) modify 58e49beb/benchmark_data_storage0111-ib.pocstor_19374_object6067/head//1 by client.1048726.0:6068 2014-08-12 17:10:25.363336 2014-08-13 17:38:02.547030 7fe9bae887a0 20 read_log 194215'947 (0'0) modify f187cf6b/benchmark_data_storage0111-ib.pocstor_19374_object6085/head//1 by client.1048726.0:6086 2014-08-12 17:10:25.533802 2014-08-13 17:38:02.547052 7fe9bae887a0 20 read_log 194215'948 (0'0) modify 700ddfab/benchmark_data_storage0111-ib.pocstor_19374_object6199/head//1 by client.1048726.0:6200 2014-08-12 17:10:26.633810 2014-08-13 17:38:02.547078 7fe9bae887a0 20 read_log 194215'949 (0'0) modify fd0e98ab/benchmark_data_storage0111-ib.pocstor_19374_object6210/head//1 by client.1048726.0:6211 2014-08-12 17:10:26.685672 2014-08-13 17:38:02.547100 7fe9bae887a0 20 read_log 194215'950 (0'0) modify b470446b/benchmark_data_storage0111-ib.pocstor_19374_object6353/head//1 by client.1048726.0:6354 2014-08-12 17:10:27.231921 2014-08-13 17:38:02.547121 7fe9bae887a0 20 read_log 194215'951 (0'0) modify e85327ab/benchmark_data_storage0111-ib.pocstor_19374_object6362/head//1 by client.1048726.0:6363 2014-08-12 17:10:27.264520 2014-08-13 17:38:02.547150 7fe9bae887a0 20 read_log 194215'952 (0'0) modify e40363ab/benchmark_data_storage0111-ib.pocstor_19374_object6440/head//1 by client.1048726.0:6441 2014-08-12 17:10:27.563351 2014-08-13 17:38:02.547181 7fe9bae887a0 20 read_log 194215'953 (0'0) modify 2dd5496b/benchmark_data_storage0111-ib.pocstor_19374_object6446/head//1 by client.1048726.0:6447 2014-08-12 17:10:27.596203 2014-08-13 17:38:02.547203 7fe9bae887a0 20 read_log 194215'954 (0'0) modify 75bccceb/benchmark_data_storage0111-ib.pocstor_19374_object6497/head//1 by client.1048726.0:6498 2014-08-12 17:10:27.784276 2014-08-13 17:38:02.547225 7fe9bae887a0 20 read_log 194215'955 (0'0) modify df89832b/benchmark_data_storage0111-ib.pocstor_19374_object6513/head//1 by client.1048726.0:6514 2014-08-12 17:10:27.854578 2014-08-13 17:38:02.547246 7fe9bae887a0 20 read_log 194215'956 (0'0) modify f02cbe6b/benchmark_data_storage0111-ib.pocstor_19374_object6524/head//1 by client.1048726.0:6525 2014-08-12 17:10:27.895191 2014-08-13 17:38:02.547268 7fe9bae887a0 20 read_log 194215'957 (0'0) modify a3faf6b/benchmark_data_storage0111-ib.pocstor_19374_object6613/head//1 by client.1048726.0:6614 2014-08-12 17:10:28.323216 2014-08-13 17:38:02.547289 7fe9bae887a0 20 read_log 194215'958 (0'0) modify ff3983eb/benchmark_data_storage0111-ib.pocstor_19374_object6657/head//1 by client.1048726.0:6658 2014-08-12 17:10:28.558897 2014-08-13 17:38:02.547323 7fe9bae887a0 20 read_log 194215'959 (0'0) modify 1f301fab/benchmark_data_storage0111-ib.pocstor_19374_object6816/head//1 by client.1048726.0:6817 2014-08-12 17:10:29.158546 2014-08-13 17:38:02.547350 7fe9bae887a0 20 read_log 194215'960 (0'0) modify f6de92eb/benchmark_data_storage0111-ib.pocstor_19374_object6841/head//1 by client.1048726.0:6842 2014-08-12 17:10:29.251845 2014-08-13 17:38:02.547372 7fe9bae887a0 20 read_log 194215'961 (0'0) modify b6c1042b/benchmark_data_storage0111-ib.pocstor_19374_object6885/head//1 by client.1048726.0:6886 2014-08-12 17:10:29.420715 2014-08-13 17:38:02.547394 7fe9bae887a0 20 read_log 194215'962 (0'0) modify af36f9eb/benchmark_data_storage0111-ib.pocstor_19374_object6904/head//1 by client.1048726.0:6905 2014-08-12 17:10:29.490280 2014-08-13 17:38:02.547416 7fe9bae887a0 20 read_log 194215'963 (0'0) modify a5777ab/benchmark_data_storage0111-ib.pocstor_19374_object7029/head//1 by client.1048726.0:7030 2014-08-12 17:10:29.950996 2014-08-13 17:38:02.547437 7fe9bae887a0 20 read_log 194215'964 (0'0) modify 6919b6b/benchmark_data_storage0111-ib.pocstor_19374_object7059/head//1 by client.1048726.0:7060 2014-08-12 17:10:30.060800 2014-08-13 17:38:02.547459 7fe9bae887a0 20 read_log 194215'965 (0'0) modify efbee52b/benchmark_data_storage0111-ib.pocstor_19374_object7069/head//1 by client.1048726.0:7070 2014-08-12 17:10:30.097601 2014-08-13 17:38:02.547480 7fe9bae887a0 20 read_log 194215'966 (0'0) modify 5f8c13eb/benchmark_data_storage0111-ib.pocstor_19374_object7102/head//1 by client.1048726.0:7103 2014-08-12 17:10:30.219407 2014-08-13 17:38:02.547501 7fe9bae887a0 20 read_log 194215'967 (0'0) modify 651b9fab/benchmark_data_storage0111-ib.pocstor_19374_object7248/head//1 by client.1048726.0:7249 2014-08-12 17:10:30.761181 2014-08-13 17:38:02.547523 7fe9bae887a0 20 read_log 194215'968 (0'0) modify 4698106b/benchmark_data_storage0111-ib.pocstor_19374_object7302/head//1 by client.1048726.0:7303 2014-08-12 17:10:30.968787 2014-08-13 17:38:02.547544 7fe9bae887a0 20 read_log 194215'969 (0'0) modify 95b2bd2b/benchmark_data_storage0111-ib.pocstor_19374_object7355/head//1 by client.1048726.0:7356 2014-08-12 17:10:31.166111 2014-08-13 17:38:02.547565 7fe9bae887a0 20 read_log 194215'970 (0'0) modify 3565086b/benchmark_data_storage0111-ib.pocstor_19374_object7394/head//1 by client.1048726.0:7395 2014-08-12 17:10:31.311296 2014-08-13 17:38:02.547587 7fe9bae887a0 20 read_log 194215'971 (0'0) modify 7723e2eb/benchmark_data_storage0111-ib.pocstor_19374_object7432/head//1 by client.1048726.0:7433 2014-08-12 17:10:31.451457 2014-08-13 17:38:02.547608 7fe9bae887a0 20 read_log 194215'972 (0'0) modify 3876f66b/benchmark_data_storage0111-ib.pocstor_19374_object7456/head//1 by client.1048726.0:7457 2014-08-12 17:10:31.553376 2014-08-13 17:38:02.547629 7fe9bae887a0 20 read_log 194215'973 (0'0) modify 7c404ceb/benchmark_data_storage0111-ib.pocstor_19374_object7471/head//1 by client.1048726.0:7472 2014-08-12 17:10:31.608965 2014-08-13 17:38:02.547650 7fe9bae887a0 20 read_log 194215'974 (0'0) modify dee414ab/benchmark_data_storage0111-ib.pocstor_19374_object7498/head//1 by client.1048726.0:7499 2014-08-12 17:10:31.708966 2014-08-13 17:38:02.547672 7fe9bae887a0 20 read_log 194215'975 (0'0) modify b0b328ab/benchmark_data_storage0111-ib.pocstor_19374_object7503/head//1 by client.1048726.0:7504 2014-08-12 17:10:31.727589 2014-08-13 17:38:02.547693 7fe9bae887a0 20 read_log 194215'976 (0'0) modify fbaf77ab/benchmark_data_storage0111-ib.pocstor_19374_object7653/head//1 by client.1048726.0:7654 2014-08-12 17:10:32.300647 2014-08-13 17:38:02.547714 7fe9bae887a0 20 read_log 194215'977 (0'0) modify f080c82b/benchmark_data_storage0111-ib.pocstor_19374_object7664/head//1 by client.1048726.0:7665 2014-08-12 17:10:32.353688 2014-08-13 17:38:02.547736 7fe9bae887a0 20 read_log 194215'978 (0'0) modify cf53d6eb/benchmark_data_storage0111-ib.pocstor_19374_object7819/head//1 by client.1048726.0:7820 2014-08-12 17:10:33.004816 2014-08-13 17:38:02.547770 7fe9bae887a0 20 read_log 194215'979 (0'0) modify 7e7fc76b/benchmark_data_storage0111-ib.pocstor_19374_object7822/head//1 by client.1048726.0:7823 2014-08-12 17:10:33.016213 2014-08-13 17:38:02.547793 7fe9bae887a0 20 read_log 194215'980 (0'0) modify fa90ac2b/benchmark_data_storage0111-ib.pocstor_19374_object7839/head//1 by client.1048726.0:7840 2014-08-12 17:10:33.090661 2014-08-13 17:38:02.547814 7fe9bae887a0 20 read_log 194215'981 (0'0) modify 5918ea2b/benchmark_data_storage0111-ib.pocstor_19374_object7899/head//1 by client.1048726.0:7900 2014-08-12 17:10:33.314113 2014-08-13 17:38:02.547836 7fe9bae887a0 20 read_log 194215'982 (0'0) modify 4b04ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8097/head//1 by client.1048726.0:8098 2014-08-12 17:10:34.059672 2014-08-13 17:38:02.547890 7fe9bae887a0 20 read_log 194215'983 (0'0) modify 3b37c4ab/benchmark_data_storage0111-ib.pocstor_19374_object8171/head//1 by client.1048726.0:8172 2014-08-12 17:10:34.335508 2014-08-13 17:38:02.547917 7fe9bae887a0 20 read_log 194215'984 (0'0) modify fabb5dab/benchmark_data_storage0111-ib.pocstor_19374_object8252/head//1 by client.1048726.0:8253 2014-08-12 17:10:34.635614 2014-08-13 17:38:02.547939 7fe9bae887a0 20 read_log 194215'985 (0'0) modify be4caceb/benchmark_data_storage0111-ib.pocstor_19374_object8295/head//1 by client.1048726.0:8296 2014-08-12 17:10:34.807030 2014-08-13 17:38:02.547961 7fe9bae887a0 20 read_log 194215'986 (0'0) modify 95c4ed2b/benchmark_data_storage0111-ib.pocstor_19374_object8373/head//1 by client.1048726.0:8374 2014-08-12 17:10:35.105503 2014-08-13 17:38:02.547982 7fe9bae887a0 20 read_log 194215'987 (0'0) modify bfddf56b/benchmark_data_storage0111-ib.pocstor_19374_object8440/head//1 by client.1048726.0:8441 2014-08-12 17:10:35.355647 2014-08-13 17:38:02.548003 7fe9bae887a0 20 read_log 194215'988 (0'0) modify 63bb63eb/benchmark_data_storage0111-ib.pocstor_19374_object8570/head//1 by client.1048726.0:8571 2014-08-12 17:10:35.944718 2014-08-13 17:38:02.548025 7fe9bae887a0 20 read_log 194215'989 (0'0) modify 819622eb/benchmark_data_storage0111-ib.pocstor_19374_object8723/head//1 by client.1048726.0:8724 2014-08-12 17:10:36.530742 2014-08-13 17:38:02.548046 7fe9bae887a0 20 read_log 194215'990 (0'0) modify 872863eb/benchmark_data_storage0111-ib.pocstor_19374_object8755/head//1 by client.1048726.0:8756 2014-08-12 17:10:36.649383 2014-08-13 17:38:02.548068 7fe9bae887a0 20 read_log 194215'991 (0'0) modify ba99d16b/benchmark_data_storage0111-ib.pocstor_19374_object8788/head//1 by client.1048726.0:8789 2014-08-12 17:10:36.773391 2014-08-13 17:38:02.548089 7fe9bae887a0 20 read_log 194215'992 (0'0) modify 89e660ab/benchmark_data_storage0111-ib.pocstor_19374_object8796/head//1 by client.1048726.0:8797 2014-08-12 17:10:36.802335 2014-08-13 17:38:02.548113 7fe9bae887a0 20 read_log 194215'993 (0'0) modify 3d2f22b/benchmark_data_storage0111-ib.pocstor_19374_object8882/head//1 by client.1048726.0:8883 2014-08-12 17:10:37.125816 2014-08-13 17:38:02.548139 7fe9bae887a0 20 read_log 194215'994 (0'0) modify 5366ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8986/head//1 by client.1048726.0:8987 2014-08-12 17:10:37.553142 2014-08-13 17:38:02.548167 7fe9bae887a0 20 read_log 194215'995 (0'0) modify e3a50aab/benchmark_data_storage0111-ib.pocstor_19374_object9008/head//1 by client.1048726.0:9009 2014-08-12 17:10:37.667724 2014-08-13 17:38:02.548193 7fe9bae887a0 20 read_log 194215'996 (0'0) modify 96aab4eb/benchmark_data_storage0111-ib.pocstor_19374_object9145/head//1 by client.1048726.0:9146 2014-08-12 17:10:38.188224 2014-08-13 17:38:02.548215 7fe9bae887a0 20 read_log 194215'997 (0'0) modify 14f1502b/benchmark_data_storage0111-ib.pocstor_19374_object9193/head//1 by client.1048726.0:9194 2014-08-12 17:10:38.366272 2014-08-13 17:38:02.548238 7fe9bae887a0 20 read_log 194215'998 (0'0) modify 6a91646b/benchmark_data_storage0111-ib.pocstor_19374_object9229/head//1 by client.1048726.0:9230 2014-08-12 17:10:38.505359 2014-08-13 17:38:02.548271 7fe9bae887a0 20 read_log 194215'999 (0'0) modify c93002eb/benchmark_data_storage0111-ib.pocstor_19374_object9287/head//1 by client.1048726.0:9288 2014-08-12 17:10:38.720418 2014-08-13 17:38:02.548294 7fe9bae887a0 20 read_log 194215'1000 (0'0) modify 2257b96b/benchmark_data_storage0111-ib.pocstor_19374_object9441/head//1 by client.1048726.0:9442 2014-08-12 17:10:39.330165 2014-08-13 17:38:02.548316 7fe9bae887a0 20 read_log 194215'1001 (0'0) modify 33077deb/benchmark_data_storage0111-ib.pocstor_19374_object9442/head//1 by client.1048726.0:9443 2014-08-12 17:10:39.333815 2014-08-13 17:38:02.548338 7fe9bae887a0 20 read_log 194215'1002 (0'0) modify e2ce132b/benchmark_data_storage0111-ib.pocstor_19374_object9679/head//1 by client.1048726.0:9680 2014-08-12 17:10:40.321794 2014-08-13 17:38:02.548361 7fe9bae887a0 20 read_log 194215'1003 (0'0) modify 9ce657ab/benchmark_data_storage0111-ib.pocstor_19374_object9686/head//1 by client.1048726.0:9687 2014-08-12 17:10:40.353095 2014-08-13 17:38:02.548383 7fe9bae887a0 20 read_log 194215'1004 (0'0) modify a535d1ab/benchmark_data_storage0111-ib.pocstor_19374_object9692/head//1 by client.1048726.0:9693 2014-08-12 17:10:40.382097 2014-08-13 17:38:02.548405 7fe9bae887a0 20 read_log 194215'1005 (0'0) modify 3ad652eb/benchmark_data_storage0111-ib.pocstor_19374_object9703/head//1 by client.1048726.0:9704 2014-08-12 17:10:40.443125 2014-08-13 17:38:02.548426 7fe9bae887a0 20 read_log 194215'1006 (0'0) modify 2daa772b/benchmark_data_storage0111-ib.pocstor_19374_object9929/head//1 by client.1048726.0:9930 2014-08-12 17:10:41.728624 2014-08-13 17:38:02.548447 7fe9bae887a0 20 read_log 194215'1007 (0'0) modify 1659f86b/benchmark_data_storage0111-ib.pocstor_19374_object9992/head//1 by client.1048726.0:9993 2014-08-12 17:10:42.087794 2014-08-13 17:38:02.548469 7fe9bae887a0 20 read_log 194215'1008 (0'0) modify 50cd18ab/benchmark_data_storage0111-ib.pocstor_19374_object10038/head//1 by client.1048726.0:10039 2014-08-12 17:10:42.327808 2014-08-13 17:38:02.548491 7fe9bae887a0 20 read_log 194215'1009 (0'0) modify 57ad5fab/benchmark_data_storage0111-ib.pocstor_19374_object10124/head//1 by client.1048726.0:10125 2014-08-12 17:10:42.748778 2014-08-13 17:38:02.548514 7fe9bae887a0 20 read_log 194215'1010 (0'0) modify 20113dab/benchmark_data_storage0111-ib.pocstor_19374_object10313/head//1 by client.1048726.0:10314 2014-08-12 17:10:43.554899 2014-08-13 17:38:02.548536 7fe9bae887a0 20 read_log 194215'1011 (0'0) modify fc13012b/benchmark_data_storage0111-ib.pocstor_19374_object10319/head//1 by client.1048726.0:10320 2014-08-12 17:10:43.577183 2014-08-13 17:38:02.548558 7fe9bae887a0 20 read_log 194215'1012 (0'0) modify 55f420ab/benchmark_data_storage0111-ib.pocstor_19374_object10414/head//1 by client.1048726.0:10415 2014-08-12 17:10:43.973605 2014-08-13 17:38:02.548580 7fe9bae887a0 20 read_log 194215'1013 (0'0) modify 44b26d2b/benchmark_data_storage0111-ib.pocstor_19374_object10437/head//1 by client.1048726.0:10438 2014-08-12 17:10:44.060397 2014-08-13 17:38:02.548602 7fe9bae887a0 20 read_log 194215'1014 (0'0) modify 6c33baab/benchmark_data_storage0111-ib.pocstor_19374_object10493/head//1 by client.1048726.0:10494 2014-08-12 17:10:44.265021 2014-08-13 17:38:02.548624 7fe9bae887a0 20 read_log 194215'1015 (0'0) modify 56652dab/benchmark_data_storage0111-ib.pocstor_19374_object10603/head//1 by client.1048726.0:10604 2014-08-12 17:10:44.841636 2014-08-13 17:38:02.548646 7fe9bae887a0 20 read_log 194215'1016 (0'0) modify 7aabe3eb/benchmark_data_storage0111-ib.pocstor_19374_object10674/head//1 by client.1048726.0:10675 2014-08-12 17:10:45.204485 2014-08-13 17:38:02.548668 7fe9bae887a0 20 read_log 194215'1017 (0'0) modify bdbfb66b/benchmark_data_storage0111-ib.pocstor_19374_object10688/head//1 by client.1048726.0:10689 2014-08-12 17:10:45.264289 2014-08-13 17:38:02.548690 7fe9bae887a0 20 read_log 194215'1018 (0'0) modify 9f6f4eeb/benchmark_data_storage0111-ib.pocstor_19374_object10736/head//1 by client.1048726.0:10737 2014-08-12 17:10:45.459936 2014-08-13 17:38:02.548791 7fe9bae887a0 20 read_log 194215'1019 (0'0) modify 55cd0b2b/benchmark_data_storage0111-ib.pocstor_19374_object10796/head//1 by client.1048726.0:10797 2014-08-12 17:10:45.734965 2014-08-13 17:38:02.548819 7fe9bae887a0 20 read_log 194215'1020 (0'0) modify 190a722b/benchmark_data_storage0111-ib.pocstor_19374_object10846/head//1 by client.1048726.0:10847 2014-08-12 17:10:45.932635 2014-08-13 17:38:02.548845 7fe9bae887a0 20 read_log 194215'1021 (0'0) modify 960912b/benchmark_data_storage0111-ib.pocstor_19374_object10955/head//1 by client.1048726.0:10956 2014-08-12 17:10:46.748083 2014-08-13 17:38:02.548867 7fe9bae887a0 20 read_log 194215'1022 (0'0) modify fcdecfeb/benchmark_data_storage0111-ib.pocstor_19374_object11112/head//1 by client.1048726.0:11113 2014-08-12 17:10:47.539122 2014-08-13 17:38:02.548889 7fe9bae887a0 20 read_log 194215'1023 (0'0) modify d5fcf32b/benchmark_data_storage0111-ib.pocstor_19374_object11303/head//1 by client.1048726.0:11304 2014-08-12 17:10:48.536652 2014-08-13 17:38:02.548910 7fe9bae887a0 20 read_log 194215'1024 (0'0) modify 75ac9eb/benchmark_data_storage0111-ib.pocstor_19374_object11344/head//1 by client.1048726.0:11345 2014-08-12 17:10:48.702631 2014-08-13 17:38:02.548932 7fe9bae887a0 20 read_log 194215'1025 (0'0) modify 3bda6ab/benchmark_data_storage0111-ib.pocstor_19374_object11393/head//1 by client.1048726.0:11394 2014-08-12 17:10:48.899216 2014-08-13 17:38:02.548953 7fe9bae887a0 20 read_log 194215'1026 (0'0) modify 940c086b/benchmark_data_storage0111-ib.pocstor_19374_object11474/head//1 by client.1048726.0:11475 2014-08-12 17:10:49.253261 2014-08-13 17:38:02.548975 7fe9bae887a0 20 read_log 194215'1027 (0'0) modify 316ace6b/benchmark_data_storage0111-ib.pocstor_19374_object11516/head//1 by client.1048726.0:11517 2014-08-12 17:10:49.420810 2014-08-13 17:38:02.548997 7fe9bae887a0 20 read_log 194215'1028 (0'0) modify dd29fdeb/benchmark_data_storage0111-ib.pocstor_19374_object11523/head//1 by client.1048726.0:11524 2014-08-12 17:10:49.447159 2014-08-13 17:38:02.549018 7fe9bae887a0 20 read_log 194215'1029 (0'0) modify f5bded2b/benchmark_data_storage0111-ib.pocstor_19374_object11543/head//1 by client.1048726.0:11544 2014-08-12 17:10:49.523414 2014-08-13 17:38:02.549040 7fe9bae887a0 20 read_log 194215'1030 (0'0) modify c4b3dc6b/benchmark_data_storage0111-ib.pocstor_19374_object11655/head//1 by client.1048726.0:11656 2014-08-12 17:10:50.068298 2014-08-13 17:38:02.549062 7fe9bae887a0 20 read_log 194215'1031 (0'0) modify 2095576b/benchmark_data_storage0111-ib.pocstor_19374_object11672/head//1 by client.1048726.0:11673 2014-08-12 17:10:50.139254 2014-08-13 17:38:02.549083 7fe9bae887a0 20 read_log 194215'1032 (0'0) modify a6115a2b/benchmark_data_storage0111-ib.pocstor_19374_object11714/head//1 by client.1048726.0:11715 2014-08-12 17:10:50.332239 2014-08-13 17:38:02.549107 7fe9bae887a0 20 read_log 194215'1033 (0'0) modify 8aafa9ab/benchmark_data_storage0111-ib.pocstor_19374_object11744/head//1 by client.1048726.0:11745 2014-08-12 17:10:50.525189 2014-08-13 17:38:02.549129 7fe9bae887a0 20 read_log 194215'1034 (0'0) modify d9131ceb/benchmark_data_storage0111-ib.pocstor_19374_object11815/head//1 by client.1048726.0:11816 2014-08-12 17:10:51.027982 2014-08-13 17:38:02.549150 7fe9bae887a0 20 read_log 194215'1035 (0'0) modify 7091a6b/benchmark_data_storage0111-ib.pocstor_19374_object12125/head//1 by client.1048726.0:12126 2014-08-12 17:10:52.717446 2014-08-13 17:38:02.549182 7fe9bae887a0 20 read_log 194215'1036 (0'0) modify 96552bab/benchmark_data_storage0111-ib.pocstor_19374_object12215/head//1 by client.1048726.0:12216 2014-08-12 17:10:53.169165 2014-08-13 17:38:02.549204 7fe9bae887a0 20 read_log 194215'1037 (0'0) modify 84c0f3eb/benchmark_data_storage0111-ib.pocstor_19374_object12617/head//1 by client.1048726.0:12618 2014-08-12 17:10:55.405083 2014-08-13 17:38:02.549226 7fe9bae887a0 20 read_log 194215'1038 (0'0) modify 6b222c6b/benchmark_data_storage0111-ib.pocstor_19374_object12626/head//1 by client.1048726.0:12627 2014-08-12 17:10:55.470454 2014-08-13 17:38:02.549260 7fe9bae887a0 20 read_log 194215'1039 (0'0) modify 304a35eb/benchmark_data_storage0111-ib.pocstor_19374_object12721/head//1 by client.1048726.0:12722 2014-08-12 17:10:56.018848 2014-08-13 17:38:02.549283 7fe9bae887a0 20 read_log 194215'1040 (0'0) modify ce855f2b/benchmark_data_storage0111-ib.pocstor_19374_object12758/head//1 by client.1048726.0:12759 2014-08-12 17:10:56.160994 2014-08-13 17:38:02.549305 7fe9bae887a0 20 read_log 194215'1041 (0'0) modify 91d427ab/benchmark_data_storage0111-ib.pocstor_19374_object12778/head//1 by client.1048726.0:12779 2014-08-12 17:10:56.233117 2014-08-13 17:38:02.549326 7fe9bae887a0 20 read_log 194215'1042 (0'0) modify fb4560ab/benchmark_data_storage0111-ib.pocstor_19374_object12863/head//1 by client.1048726.0:12864 2014-08-12 17:10:56.573943 2014-08-13 17:38:02.549348 7fe9bae887a0 20 read_log 194215'1043 (0'0) modify 883592b/benchmark_data_storage0111-ib.pocstor_19374_object12865/head//1 by client.1048726.0:12866 2014-08-12 17:10:56.584963 2014-08-13 17:38:02.549369 7fe9bae887a0 20 read_log 194215'1044 (0'0) modify 64527e2b/benchmark_data_storage0111-ib.pocstor_19374_object12908/head//1 by client.1048726.0:12909 2014-08-12 17:10:56.782793 2014-08-13 17:38:02.549391 7fe9bae887a0 20 read_log 194215'1045 (0'0) modify bb45c1eb/benchmark_data_storage0111-ib.pocstor_19374_object12937/head//1 by client.1048726.0:12938 2014-08-12 17:10:56.911124 2014-08-13 17:38:02.549413 7fe9bae887a0 20 read_log 194215'1046 (0'0) modify 4f81c16b/benchmark_data_storage0111-ib.pocstor_19374_object12968/head//1 by client.1048726.0:12969 2014-08-12 17:10:57.059413 2014-08-13 17:38:02.549434 7fe9bae887a0 20 read_log 194215'1047 (0'0) modify ca6ba7eb/benchmark_data_storage0111-ib.pocstor_19374_object12985/head//1 by client.1048726.0:12986 2014-08-12 17:10:57.153096 2014-08-13 17:38:02.549456 7fe9bae887a0 20 read_log 194215'1048 (0'0) modify d6caf0eb/benchmark_data_storage0111-ib.pocstor_19374_object13037/head//1 by client.1048726.0:13038 2014-08-12 17:10:57.434942 2014-08-13 17:38:02.549478 7fe9bae887a0 20 read_log 194215'1049 (0'0) modify 995517eb/benchmark_data_storage0111-ib.pocstor_19374_object13098/head//1 by client.1048726.0:13099 2014-08-12 17:10:57.754754 2014-08-13 17:38:02.549500 7fe9bae887a0 20 read_log 194215'1050 (0'0) modify 69b7392b/benchmark_data_storage0111-ib.pocstor_19374_object13278/head//1 by client.1048726.0:13279 2014-08-12 17:10:58.809494 2014-08-13 17:38:02.549521 7fe9bae887a0 20 read_log 194215'1051 (0'0) modify 5d3e5d6b/benchmark_data_storage0111-ib.pocstor_19374_object13368/head//1 by client.1048726.0:13369 2014-08-12 17:10:59.479526 2014-08-13 17:38:02.549543 7fe9bae887a0 20 read_log 194215'1052 (0'0) modify b7bdedeb/benchmark_data_storage0111-ib.pocstor_19374_object13400/head//1 by client.1048726.0:13401 2014-08-12 17:10:59.596906 2014-08-13 17:38:02.549565 7fe9bae887a0 20 read_log 194215'1053 (0'0) modify 6dc64eeb/benchmark_data_storage0111-ib.pocstor_19374_object13453/head//1 by client.1048726.0:13454 2014-08-12 17:10:59.795662 2014-08-13 17:38:02.549586 7fe9bae887a0 20 read_log 194215'1054 (0'0) modify 8c0f1d2b/benchmark_data_storage0111-ib.pocstor_19374_object13462/head//1 by client.1048726.0:13463 2014-08-12 17:10:59.829868 2014-08-13 17:38:02.549608 7fe9bae887a0 20 read_log 194215'1055 (0'0) modify 37a0adab/benchmark_data_storage0111-ib.pocstor_19374_object13587/head//1 by client.1048726.0:13588 2014-08-12 17:11:00.345687 2014-08-13 17:38:02.549630 7fe9bae887a0 20 read_log 194215'1056 (0'0) modify 5a6f74eb/benchmark_data_storage0111-ib.pocstor_19374_object13600/head//1 by client.1048726.0:13601 2014-08-12 17:11:00.392623 2014-08-13 17:38:02.549657 7fe9bae887a0 20 read_log 194215'1057 (0'0) modify d3458feb/benchmark_data_storage0111-ib.pocstor_19374_object13641/head//1 by client.1048726.0:13642 2014-08-12 17:11:00.542529 2014-08-13 17:38:02.549680 7fe9bae887a0 20 read_log 194215'1058 (0'0) modify b1e814eb/benchmark_data_storage0111-ib.pocstor_19374_object13681/head//1 by client.1048726.0:13682 2014-08-12 17:11:00.696193 2014-08-13 17:38:02.549714 7fe9bae887a0 20 read_log 194215'1059 (0'0) modify 97b4952b/benchmark_data_storage0111-ib.pocstor_19374_object13697/head//1 by client.1048726.0:13698 2014-08-12 17:11:00.753113 2014-08-13 17:38:02.549741 7fe9bae887a0 20 read_log 194215'1060 (0'0) modify b156fbab/benchmark_data_storage0111-ib.pocstor_19374_object13795/head//1 by client.1048726.0:13796 2014-08-12 17:11:01.117694 2014-08-13 17:38:02.549763 7fe9bae887a0 20 read_log 194215'1061 (0'0) modify b9abdaeb/benchmark_data_storage0111-ib.pocstor_19374_object13915/head//1 by client.1048726.0:13916 2014-08-12 17:11:01.568882 2014-08-13 17:38:02.549785 7fe9bae887a0 20 read_log 194215'1062 (0'0) modify 15bb4aeb/benchmark_data_storage0111-ib.pocstor_19374_object14018/head//1 by client.1048726.0:14019 2014-08-12 17:11:01.956741 2014-08-13 17:38:02.549806 7fe9bae887a0 20 read_log 194215'1063 (0'0) modify 86d77e6b/benchmark_data_storage0111-ib.pocstor_19374_object14045/head//1 by client.1048726.0:14046 2014-08-12 17:11:02.054905 2014-08-13 17:38:02.549830 7fe9bae887a0 20 read_log 194215'1064 (0'0) modify f762602b/benchmark_data_storage0111-ib.pocstor_19374_object14072/head//1 by client.1048726.0:14073 2014-08-12 17:11:02.155399 2014-08-13 17:38:02.549852 7fe9bae887a0 20 read_log 194215'1065 (0'0) modify 656a9d6b/benchmark_data_storage0111-ib.pocstor_19374_object14108/head//1 by client.1048726.0:14109 2014-08-12 17:11:02.288002 2014-08-13 17:38:02.549874 7fe9bae887a0 20 read_log 194215'1066 (0'0) modify 12017d6b/benchmark_data_storage0111-ib.pocstor_19374_object14130/head//1 by client.1048726.0:14131 2014-08-12 17:11:02.380293 2014-08-13 17:38:02.549895 7fe9bae887a0 20 read_log 194215'1067 (0'0) modify 6837d1eb/benchmark_data_storage0111-ib.pocstor_19374_object14218/head//1 by client.1048726.0:14219 2014-08-12 17:11:02.782849 2014-08-13 17:38:02.549917 7fe9bae887a0 20 read_log 194215'1068 (0'0) modify e0890e6b/benchmark_data_storage0111-ib.pocstor_19374_object14272/head//1 by client.1048726.0:14273 2014-08-12 17:11:02.980867 2014-08-13 17:38:02.549938 7fe9bae887a0 20 read_log 194215'1069 (0'0) modify 51a9f76b/benchmark_data_storage0111-ib.pocstor_19374_object14273/head//1 by client.1048726.0:14274 2014-08-12 17:11:02.984570 2014-08-13 17:38:02.549960 7fe9bae887a0 20 read_log 194215'1070 (0'0) modify ff105aeb/benchmark_data_storage0111-ib.pocstor_19374_object14288/head//1 by client.1048726.0:14289 2014-08-12 17:11:03.038863 2014-08-13 17:38:02.549981 7fe9bae887a0 20 read_log 194215'1071 (0'0) modify 7c054ceb/benchmark_data_storage0111-ib.pocstor_19374_object14442/head//1 by client.1048726.0:14443 2014-08-12 17:11:03.778734 2014-08-13 17:38:02.550003 7fe9bae887a0 20 read_log 194215'1072 (0'0) modify fc1d02eb/benchmark_data_storage0111-ib.pocstor_19374_object14443/head//1 by client.1048726.0:14444 2014-08-12 17:11:03.784749 2014-08-13 17:38:02.550024 7fe9bae887a0 20 read_log 194215'1073 (0'0) modify b7463aeb/benchmark_data_storage0111-ib.pocstor_19374_object14446/head//1 by client.1048726.0:14447 2014-08-12 17:11:03.802855 2014-08-13 17:38:02.550046 7fe9bae887a0 20 read_log 194215'1074 (0'0) modify 56be36eb/benchmark_data_storage0111-ib.pocstor_19374_object14581/head//1 by client.1048726.0:14582 2014-08-12 17:11:04.570010 2014-08-13 17:38:02.550067 7fe9bae887a0 20 read_log 194215'1075 (0'0) modify bc8678eb/benchmark_data_storage0111-ib.pocstor_19374_object14608/head//1 by client.1048726.0:14609 2014-08-12 17:11:04.710252 2014-08-13 17:38:02.550130 7fe9bae887a0 20 read_log 194215'1076 (0'0) modify 49787bab/benchmark_data_storage0111-ib.pocstor_19374_object14741/head//1 by client.1048726.0:14742 2014-08-12 17:11:05.631531 2014-08-13 17:38:02.550165 7fe9bae887a0 20 read_log 194215'1077 (0'0) modify af3d7a2b/benchmark_data_storage0111-ib.pocstor_19374_object14751/head//1 by client.1048726.0:14752 2014-08-12 17:11:05.713953 2014-08-13 17:38:02.550225 7fe9bae887a0 20 read_log 194215'1078 (0'0) modify 3cec11eb/benchmark_data_storage0111-ib.pocstor_19374_object14764/head//1 by client.1048726.0:14765 2014-08-12 17:11:05.774351 2014-08-13 17:38:02.550260 7fe9bae887a0 20 read_log 194215'1079 (0'0) modify 97eeeab/benchmark_data_storage0111-ib.pocstor_19374_object14796/head//1 by client.1048726.0:14797 2014-08-12 17:11:05.926313 2014-08-13 17:38:02.550287 7fe9bae887a0 20 read_log 194215'1080 (0'0) modify 708ad02b/benchmark_data_storage0111-ib.pocstor_19374_object14818/head//1 by client.1048726.0:14819 2014-08-12 17:11:06.055132 2014-08-13 17:38:02.550309 7fe9bae887a0 20 read_log 194215'1081 (0'0) modify ed21ddeb/benchmark_data_storage0111-ib.pocstor_19374_object14866/head//1 by client.1048726.0:14867 2014-08-12 17:11:06.282019 2014-08-13 17:38:02.550331 7fe9bae887a0 20 read_log 194215'1082 (0'0) modify 7847fa6b/benchmark_data_storage0111-ib.pocstor_19374_object15056/head//1 by client.1048726.0:15057 2014-08-12 17:11:07.237033 2014-08-13 17:38:02.550353 7fe9bae887a0 20 read_log 194215'1083 (0'0) modify 91326eeb/benchmark_data_storage0111-ib.pocstor_19374_object15087/head//1 by client.1048726.0:15088 2014-08-12 17:11:07.431239 2014-08-13 17:38:02.550374 7fe9bae887a0 20 read_log 194215'1084 (0'0) modify f966fa2b/benchmark_data_storage0111-ib.pocstor_19374_object15209/head//1 by client.1048726.0:15210 2014-08-12 17:11:07.900154 2014-08-13 17:38:02.550395 7fe9bae887a0 20 read_log 194215'1085 (0'0) modify 228e7c6b/benchmark_data_storage0111-ib.pocstor_19374_object15227/head//1 by client.1048726.0:15228 2014-08-12 17:11:07.967280 2014-08-13 17:38:02.550417 7fe9bae887a0 20 read_log 194215'1086 (0'0) modify 7acf24ab/benchmark_data_storage0111-ib.pocstor_19374_object15235/head//1 by client.1048726.0:15236 2014-08-12 17:11:07.996324 2014-08-13 17:38:02.550438 7fe9bae887a0 20 read_log 194215'1087 (0'0) modify c50ebeeb/benchmark_data_storage0111-ib.pocstor_19374_object15256/head//1 by client.1048726.0:15257 2014-08-12 17:11:08.074474 2014-08-13 17:38:02.550460 7fe9bae887a0 20 read_log 194215'1088 (0'0) modify ce6e186b/benchmark_data_storage0111-ib.pocstor_19374_object15291/head//1 by client.1048726.0:15292 2014-08-12 17:11:08.202782 2014-08-13 17:38:02.550482 7fe9bae887a0 20 read_log 194215'1089 (0'0) modify 4248daab/benchmark_data_storage0111-ib.pocstor_19374_object15301/head//1 by client.1048726.0:15302 2014-08-12 17:11:08.239529 2014-08-13 17:38:02.550504 7fe9bae887a0 20 read_log 194215'1090 (0'0) modify 7db6d4eb/benchmark_data_storage0111-ib.pocstor_19374_object15375/head//1 by client.1048726.0:15376 2014-08-12 17:11:08.579730 2014-08-13 17:38:02.550525 7fe9bae887a0 20 read_log 194215'1091 (0'0) modify d27cd12b/benchmark_data_storage0111-ib.pocstor_19374_object15494/head//1 by client.1048726.0:15495 2014-08-12 17:11:09.064518 2014-08-13 17:38:02.550547 7fe9bae887a0 20 read_log 194215'1092 (0'0) modify 29efcab/benchmark_data_storage0111-ib.pocstor_19374_object15598/head//1 by client.1048726.0:15599 2014-08-12 17:11:09.753328 2014-08-13 17:38:02.550568 7fe9bae887a0 20 read_log 194215'1093 (0'0) modify 725d332b/benchmark_data_storage0111-ib.pocstor_19374_object15691/head//1 by client.1048726.0:15692 2014-08-12 17:11:10.201985 2014-08-13 17:38:02.550590 7fe9bae887a0 20 read_log 194215'1094 (0'0) modify 427577eb/benchmark_data_storage0111-ib.pocstor_19374_object15706/head//1 by client.1048726.0:15707 2014-08-12 17:11:10.258348 2014-08-13 17:38:02.550611 7fe9bae887a0 20 read_log 194215'1095 (0'0) modify 3774496b/benchmark_data_storage0111-ib.pocstor_19374_object15775/head//1 by client.1048726.0:15776 2014-08-12 17:11:10.511897 2014-08-13 17:38:02.550633 7fe9bae887a0 20 read_log 194215'1096 (0'0) modify fb1ddcab/benchmark_data_storage0111-ib.pocstor_19374_object15797/head//1 by client.1048726.0:15798 2014-08-12 17:11:10.592274 2014-08-13 17:38:02.550655 7fe9bae887a0 20 read_log 194215'1097 (0'0) modify 72b1916b/benchmark_data_storage0111-ib.pocstor_19374_object15974/head//1 by client.1048726.0:15975 2014-08-12 17:11:11.421520 2014-08-13 17:38:02.550676 7fe9bae887a0 20 read_log 194215'1098 (0'0) modify 1afad6eb/benchmark_data_storage0111-ib.pocstor_19374_object16016/head//1 by client.1048726.0:16017 2014-08-12 17:11:11.659355 2014-08-13 17:38:02.550709 7fe9bae887a0 20 read_log 194215'1099 (0'0) modify 36c3d4ab/benchmark_data_storage0111-ib.pocstor_19374_object16089/head//1 by client.1048726.0:16090 2014-08-12 17:11:11.942172 2014-08-13 17:38:02.550736 7fe9bae887a0 20 read_log 194215'1100 (0'0) modify 41ee41eb/benchmark_data_storage0111-ib.pocstor_19374_object16099/head//1 by client.1048726.0:16100 2014-08-12 17:11:11.989962 2014-08-13 17:38:02.550759 7fe9bae887a0 20 read_log 194215'1101 (0'0) modify f52c0f6b/benchmark_data_storage0111-ib.pocstor_19374_object16114/head//1 by client.1048726.0:16115 2014-08-12 17:11:12.070909 2014-08-13 17:38:02.550781 7fe9bae887a0 20 read_log 194215'1102 (0'0) modify 343512ab/benchmark_data_storage0111-ib.pocstor_19374_object16151/head//1 by client.1048726.0:16152 2014-08-12 17:11:12.261179 2014-08-13 17:38:02.550802 7fe9bae887a0 20 read_log 194215'1103 (0'0) modify 5811406b/benchmark_data_storage0111-ib.pocstor_19374_object16504/head//1 by client.1048726.0:16505 2014-08-12 17:11:13.777636 2014-08-13 17:38:02.550825 7fe9bae887a0 20 read_log 194215'1104 (0'0) modify 9896e7ab/benchmark_data_storage0111-ib.pocstor_19374_object16534/head//1 by client.1048726.0:16535 2014-08-12 17:11:13.887725 2014-08-13 17:38:02.550847 7fe9bae887a0 20 read_log 194215'1105 (0'0) modify d7af0eab/benchmark_data_storage0111-ib.pocstor_19374_object16646/head//1 by client.1048726.0:16647 2014-08-12 17:11:14.311649 2014-08-13 17:38:02.550868 7fe9bae887a0 20 read_log 194215'1106 (0'0) modify 915c5a2b/benchmark_data_storage0111-ib.pocstor_19374_object16648/head//1 by client.1048726.0:16649 2014-08-12 17:11:14.318961 2014-08-13 17:38:02.550890 7fe9bae887a0 20 read_log 194215'1107 (0'0) modify a73d206b/benchmark_data_storage0111-ib.pocstor_19374_object16746/head//1 by client.1048726.0:16747 2014-08-12 17:11:14.681228 2014-08-13 17:38:02.550911 7fe9bae887a0 20 read_log 194215'1108 (0'0) modify c47c9beb/benchmark_data_storage0111-ib.pocstor_19374_object16760/head//1 by client.1048726.0:16761 2014-08-12 17:11:14.734844 2014-08-13 17:38:02.550935 7fe9bae887a0 20 read_log 194215'1109 (0'0) modify dea573ab/benchmark_data_storage0111-ib.pocstor_19374_object16841/head//1 by client.1048726.0:16842 2014-08-12 17:11:15.046110 2014-08-13 17:38:02.550959 7fe9bae887a0 20 read_log 194215'1110 (0'0) modify b9e0d1ab/benchmark_data_storage0111-ib.pocstor_19374_object16881/head//1 by client.1048726.0:16882 2014-08-12 17:11:15.193801 2014-08-13 17:38:02.550984 7fe9bae887a0 20 read_log 194215'1111 (0'0) modify 3755296b/benchmark_data_storage0111-ib.pocstor_19374_object16916/head//1 by client.1048726.0:16917 2014-08-12 17:11:15.352323 2014-08-13 17:38:02.551006 7fe9bae887a0 20 read_log 194215'1112 (0'0) modify c45cbd6b/benchmark_data_storage0111-ib.pocstor_19374_object16925/head//1 by client.1048726.0:16926 2014-08-12 17:11:15.409489 2014-08-13 17:38:02.551027 7fe9bae887a0 20 read_log 194215'1113 (0'0) modify d5f0a92b/benchmark_data_storage0111-ib.pocstor_19374_object16947/head//1 by client.1048726.0:16948 2014-08-12 17:11:15.552798 2014-08-13 17:38:02.551050 7fe9bae887a0 20 read_log 194215'1114 (0'0) modify 5fceb76b/benchmark_data_storage0111-ib.pocstor_19374_object17225/head//1 by client.1048726.0:17226 2014-08-12 17:11:16.642545 2014-08-13 17:38:02.551071 7fe9bae887a0 20 read_log 194215'1115 (0'0) modify bf0f31eb/benchmark_data_storage0111-ib.pocstor_19374_object17240/head//1 by client.1048726.0:17241 2014-08-12 17:11:16.727026 2014-08-13 17:38:02.551093 7fe9bae887a0 20 read_log 194215'1116 (0'0) modify ab105e2b/benchmark_data_storage0111-ib.pocstor_19374_object17300/head//1 by client.1048726.0:17301 2014-08-12 17:11:16.984455 2014-08-13 17:38:02.551114 7fe9bae887a0 20 read_log 194215'1117 (0'0) modify 8b06b7ab/benchmark_data_storage0111-ib.pocstor_19374_object17532/head//1 by client.1048726.0:17533 2014-08-12 17:11:18.127760 2014-08-13 17:38:02.551135 7fe9bae887a0 20 read_log 194215'1118 (0'0) modify ff580c6b/benchmark_data_storage0111-ib.pocstor_19374_object17566/head//1 by client.1048726.0:17567 2014-08-12 17:11:18.391187 2014-08-13 17:38:02.551177 7fe9bae887a0 20 read_log 194215'1119 (0'0) modify feff186b/benchmark_data_storage0111-ib.pocstor_19374_object17666/head//1 by client.1048726.0:17667 2014-08-12 17:11:18.814107 2014-08-13 17:38:02.551205 7fe9bae887a0 20 read_log 194215'1120 (0'0) modify 92eb096b/benchmark_data_storage0111-ib.pocstor_19374_object17678/head//1 by client.1048726.0:17679 2014-08-12 17:11:18.868790 2014-08-13 17:38:02.551227 7fe9bae887a0 20 read_log 194215'1121 (0'0) modify 4795ccab/benchmark_data_storage0111-ib.pocstor_19374_object17689/head//1 by client.1048726.0:17690 2014-08-12 17:11:18.910377 2014-08-13 17:38:02.551249 7fe9bae887a0 20 read_log 194215'1122 (0'0) modify 7610516b/benchmark_data_storage0111-ib.pocstor_19374_object17776/head//1 by client.1048726.0:17777 2014-08-12 17:11:19.346114 2014-08-13 17:38:02.551270 7fe9bae887a0 20 read_log 194215'1123 (0'0) modify 1941352b/benchmark_data_storage0111-ib.pocstor_19374_object17799/head//1 by client.1048726.0:17800 2014-08-12 17:11:19.430094 2014-08-13 17:38:02.551292 7fe9bae887a0 20 read_log 194215'1124 (0'0) modify a663f52b/benchmark_data_storage0111-ib.pocstor_19374_object17824/head//1 by client.1048726.0:17825 2014-08-12 17:11:19.520873 2014-08-13 17:38:02.551313 7fe9bae887a0 20 read_log 194215'1125 (0'0) modify 85edc6ab/benchmark_data_storage0111-ib.pocstor_19374_object17839/head//1 by client.1048726.0:17840 2014-08-12 17:11:19.582707 2014-08-13 17:38:02.551337 7fe9bae887a0 20 read_log 194215'1126 (0'0) modify 3a62892b/benchmark_data_storage0111-ib.pocstor_19374_object17894/head//1 by client.1048726.0:17895 2014-08-12 17:11:19.786243 2014-08-13 17:38:02.551359 7fe9bae887a0 20 read_log 194215'1127 (0'0) modify a9f9016b/benchmark_data_storage0111-ib.pocstor_19374_object17942/head//1 by client.1048726.0:17943 2014-08-12 17:11:19.960010 2014-08-13 17:38:02.551383 7fe9bae887a0 20 read_log 194215'1128 (0'0) modify 49c8032b/benchmark_data_storage0111-ib.pocstor_19374_object18015/head//1 by client.1048726.0:18016 2014-08-12 17:11:20.228997 2014-08-13 17:38:02.551404 7fe9bae887a0 20 read_log 194215'1129 (0'0) modify 63b44fab/benchmark_data_storage0111-ib.pocstor_19374_object18232/head//1 by client.1048726.0:18233 2014-08-12 17:11:21.110630 2014-08-13 17:38:02.551426 7fe9bae887a0 20 read_log 194215'1130 (0'0) modify c05a8beb/benchmark_data_storage0111-ib.pocstor_19374_object18250/head//1 by client.1048726.0:18251 2014-08-12 17:11:21.176143 2014-08-13 17:38:02.551448 7fe9bae887a0 20 read_log 194215'1131 (0'0) modify 622415eb/benchmark_data_storage0111-ib.pocstor_19374_object18344/head//1 by client.1048726.0:18345 2014-08-12 17:11:21.589700 2014-08-13 17:38:02.551470 7fe9bae887a0 20 read_log 194215'1132 (0'0) modify d404aeb/benchmark_data_storage0111-ib.pocstor_19374_object18373/head//1 by client.1048726.0:18374 2014-08-12 17:11:21.697244 2014-08-13 17:38:02.551491 7fe9bae887a0 20 read_log 194215'1133 (0'0) modify 19dc8e2b/benchmark_data_storage0111-ib.pocstor_19374_object18496/head//1 by client.1048726.0:18497 2014-08-12 17:11:22.168059 2014-08-13 17:38:02.551512 7fe9bae887a0 20 read_log 194215'1134 (0'0) modify 379abaab/benchmark_data_storage0111-ib.pocstor_19374_object18497/head//1 by client.1048726.0:18498 2014-08-12 17:11:22.174034 2014-08-13 17:38:02.551534 7fe9bae887a0 20 read_log 194215'1135 (0'0) modify 64e0802b/benchmark_data_storage0111-ib.pocstor_19374_object18498/head//1 by client.1048726.0:18499 2014-08-12 17:11:22.178663 2014-08-13 17:38:02.551555 7fe9bae887a0 20 read_log 194215'1136 (0'0) modify c6e8922b/benchmark_data_storage0111-ib.pocstor_19374_object18505/head//1 by client.1048726.0:18506 2014-08-12 17:11:22.210691 2014-08-13 17:38:02.551577 7fe9bae887a0 20 read_log 194215'1137 (0'0) modify 2855efeb/benchmark_data_storage0111-ib.pocstor_19374_object18551/head//1 by client.1048726.0:18552 2014-08-12 17:11:22.558483 2014-08-13 17:38:02.551599 7fe9bae887a0 20 read_log 194215'1138 (0'0) modify 646eadab/benchmark_data_storage0111-ib.pocstor_19374_object18577/head//1 by client.1048726.0:18578 2014-08-12 17:11:22.727357 2014-08-13 17:38:02.551631 7fe9bae887a0 20 read_log 194215'1139 (0'0) modify 6c5018eb/benchmark_data_storage0111-ib.pocstor_19374_object18780/head//1 by client.1048726.0:18781 2014-08-12 17:11:23.601122 2014-08-13 17:38:02.551658 7fe9bae887a0 20 read_log 194215'1140 (0'0) modify d3893eeb/benchmark_data_storage0111-ib.pocstor_19374_object18819/head//1 by client.1048726.0:18820 2014-08-12 17:11:23.828227 2014-08-13 17:38:02.551680 7fe9bae887a0 20 read_log 194215'1141 (0'0) modify f022432b/benchmark_data_storage0111-ib.pocstor_19374_object18842/head//1 by client.1048726.0:18843 2014-08-12 17:11:24.025785 2014-08-13 17:38:02.551702 7fe9bae887a0 20 read_log 194215'1142 (0'0) modify 2476c96b/benchmark_data_storage0111-ib.pocstor_19374_object18852/head//1 by client.1048726.0:18853 2014-08-12 17:11:24.087535 2014-08-13 17:38:02.551724 7fe9bae887a0 20 read_log 194215'1143 (0'0) modify e1cd2ceb/benchmark_data_storage0111-ib.pocstor_19374_object19065/head//1 by client.1048726.0:19066 2014-08-12 17:11:24.935380 2014-08-13 17:38:02.551745 7fe9bae887a0 20 read_log 194215'1144 (0'0) modify ae4f962b/benchmark_data_storage0111-ib.pocstor_19374_object19226/head//1 by client.1048726.0:19227 2014-08-12 17:11:25.634967 2014-08-13 17:38:02.551767 7fe9bae887a0 20 read_log 194215'1145 (0'0) modify b8cf256b/benchmark_data_storage0111-ib.pocstor_19374_object19231/head//1 by client.1048726.0:19232 2014-08-12 17:11:25.670638 2014-08-13 17:38:02.551788 7fe9bae887a0 20 read_log 194215'1146 (0'0) modify f0edd2b/benchmark_data_storage0111-ib.pocstor_19374_object19233/head//1 by client.1048726.0:19234 2014-08-12 17:11:25.683402 2014-08-13 17:38:02.551810 7fe9bae887a0 20 read_log 194215'1147 (0'0) modify 2f22b1ab/benchmark_data_storage0111-ib.pocstor_19374_object19274/head//1 by client.1048726.0:19275 2014-08-12 17:11:26.361721 2014-08-13 17:38:02.551831 7fe9bae887a0 20 read_log 194215'1148 (0'0) modify 115748eb/benchmark_data_storage0111-ib.pocstor_19374_object19293/head//1 by client.1048726.0:19294 2014-08-12 17:11:26.466906 2014-08-13 17:38:02.551853 7fe9bae887a0 20 read_log 194215'1149 (0'0) modify ce72ce6b/benchmark_data_storage0111-ib.pocstor_19374_object19399/head//1 by client.1048726.0:19400 2014-08-12 17:11:26.872335 2014-08-13 17:38:02.551875 7fe9bae887a0 20 read_log 194215'1150 (0'0) modify d56f90ab/benchmark_data_storage0111-ib.pocstor_19374_object19623/head//1 by client.1048726.0:19624 2014-08-12 17:11:27.697981 2014-08-13 17:38:02.551898 7fe9bae887a0 20 read_log 194215'1151 (0'0) modify ee91eceb/benchmark_data_storage0111-ib.pocstor_19374_object19625/head//1 by client.1048726.0:19626 2014-08-12 17:11:27.705429 2014-08-13 17:38:02.551919 7fe9bae887a0 20 read_log 194215'1152 (0'0) modify 563df2ab/benchmark_data_storage0111-ib.pocstor_19374_object19626/head//1 by client.1048726.0:19627 2014-08-12 17:11:27.709066 2014-08-13 17:38:02.551941 7fe9bae887a0 20 read_log 194215'1153 (0'0) modify f16196eb/benchmark_data_storage0111-ib.pocstor_19374_object19875/head//1 by client.1048726.0:19876 2014-08-12 17:11:28.659242 2014-08-13 17:38:02.551962 7fe9bae887a0 20 read_log 194215'1154 (0'0) modify 43e8486b/benchmark_data_storage0111-ib.pocstor_19374_object19917/head//1 by client.1048726.0:19918 2014-08-12 17:11:28.895145 2014-08-13 17:38:02.551984 7fe9bae887a0 20 read_log 194215'1155 (0'0) modify 3847442b/benchmark_data_storage0111-ib.pocstor_19374_object19991/head//1 by client.1048726.0:19992 2014-08-12 17:11:29.242744 2014-08-13 17:38:02.552006 7fe9bae887a0 20 read_log 194215'1156 (0'0) modify 2e8e12ab/benchmark_data_storage0111-ib.pocstor_19374_object20016/head//1 by client.1048726.0:20017 2014-08-12 17:11:29.336224 2014-08-13 17:38:02.552027 7fe9bae887a0 20 read_log 194215'1157 (0'0) modify 9ea9826b/benchmark_data_storage0111-ib.pocstor_19374_object20100/head//1 by client.1048726.0:20101 2014-08-12 17:11:29.651300 2014-08-13 17:38:02.552049 7fe9bae887a0 20 read_log 194215'1158 (0'0) modify de30edeb/benchmark_data_storage0111-ib.pocstor_19374_object20132/head//1 by client.1048726.0:20133 2014-08-12 17:11:29.768937 2014-08-13 17:38:02.552114 7fe9bae887a0 20 read_log 194215'1159 (0'0) modify 47ed002b/benchmark_data_storage0111-ib.pocstor_19374_object20189/head//1 by client.1048726.0:20190 2014-08-12 17:11:29.981605 2014-08-13 17:38:02.552143 7fe9bae887a0 20 read_log 194215'1160 (0'0) modify 5be3cf2b/benchmark_data_storage0111-ib.pocstor_19374_object20205/head//1 by client.1048726.0:20206 2014-08-12 17:11:30.040316 2014-08-13 17:38:02.552173 7fe9bae887a0 20 read_log 194215'1161 (0'0) modify fbdfe52b/benchmark_data_storage0111-ib.pocstor_19374_object20231/head//1 by client.1048726.0:20232 2014-08-12 17:11:30.138232 2014-08-13 17:38:02.552202 7fe9bae887a0 20 read_log 194215'1162 (0'0) modify 45de5dab/benchmark_data_storage0111-ib.pocstor_19374_object20287/head//1 by client.1048726.0:20288 2014-08-12 17:11:30.357060 2014-08-13 17:38:02.552224 7fe9bae887a0 20 read_log 194215'1163 (0'0) modify 94efc5eb/benchmark_data_storage0111-ib.pocstor_19374_object20346/head//1 by client.1048726.0:20347 2014-08-12 17:11:30.703703 2014-08-13 17:38:02.552246 7fe9bae887a0 20 read_log 194215'1164 (0'0) modify 1347d22b/benchmark_data_storage0111-ib.pocstor_19374_object20407/head//1 by client.1048726.0:20408 2014-08-12 17:11:30.987836 2014-08-13 17:38:02.552267 7fe9bae887a0 20 read_log 194215'1165 (0'0) modify e33e54ab/benchmark_data_storage0111-ib.pocstor_19374_object20410/head//1 by client.1048726.0:20411 2014-08-12 17:11:30.998699 2014-08-13 17:38:02.552289 7fe9bae887a0 20 read_log 194215'1166 (0'0) modify e858402b/benchmark_data_storage0111-ib.pocstor_19374_object20437/head//1 by client.1048726.0:20438 2014-08-12 17:11:31.099054 2014-08-13 17:38:02.552311 7fe9bae887a0 20 read_log 194215'1167 (0'0) modify 19a5632b/benchmark_data_storage0111-ib.pocstor_19374_object20464/head//1 by client.1048726.0:20465 2014-08-12 17:11:31.197916 2014-08-13 17:38:02.552332 7fe9bae887a0 20 read_log 194215'1168 (0'0) modify 19fb1ceb/benchmark_data_storage0111-ib.pocstor_19374_object20580/head//1 by client.1048726.0:20581 2014-08-12 17:11:31.636049 2014-08-13 17:38:02.552386 7fe9bae887a0 20 read_log 194215'1169 (0'0) modify 63afe4eb/benchmark_data_storage0111-ib.pocstor_19374_object20590/head//1 by client.1048726.0:20591 2014-08-12 17:11:31.674216 2014-08-13 17:38:02.552409 7fe9bae887a0 20 read_log 194215'1170 (0'0) modify 5e2d476b/benchmark_data_storage0111-ib.pocstor_19374_object20594/head//1 by client.1048726.0:20595 2014-08-12 17:11:31.690890 2014-08-13 17:38:02.552430 7fe9bae887a0 20 read_log 194215'1171 (0'0) modify 7ffdd76b/benchmark_data_storage0111-ib.pocstor_19374_object20637/head//1 by client.1048726.0:20638 2014-08-12 17:11:31.856883 2014-08-13 17:38:02.552452 7fe9bae887a0 20 read_log 194215'1172 (0'0) modify 595617eb/benchmark_data_storage0111-ib.pocstor_19374_object20646/head//1 by client.1048726.0:20647 2014-08-12 17:11:31.894409 2014-08-13 17:38:02.552475 7fe9bae887a0 20 read_log 194215'1173 (0'0) modify 2860b42b/benchmark_data_storage0111-ib.pocstor_19374_object20753/head//1 by client.1048726.0:20754 2014-08-12 17:11:32.554768 2014-08-13 17:38:02.552496 7fe9bae887a0 20 read_log 194215'1174 (0'0) modify 496c846b/benchmark_data_storage0111-ib.pocstor_19374_object20823/head//1 by client.1048726.0:20824 2014-08-12 17:11:32.826697 2014-08-13 17:38:02.552518 7fe9bae887a0 20 read_log 194215'1175 (0'0) modify 22036eeb/benchmark_data_storage0111-ib.pocstor_19374_object20913/head//1 by client.1048726.0:20914 2014-08-12 17:11:33.261477 2014-08-13 17:38:02.552572 7fe9bae887a0 20 read_log 194215'1176 (0'0) modify ab250fab/benchmark_data_storage0111-ib.pocstor_19374_object20914/head//1 by client.1048726.0:20915 2014-08-12 17:11:33.265285 2014-08-13 17:38:02.552597 7fe9bae887a0 20 read_log 194215'1177 (0'0) modify ae70372b/benchmark_data_storage0111-ib.pocstor_19374_object21028/head//1 by client.1048726.0:21029 2014-08-12 17:11:33.809639 2014-08-13 17:38:02.552619 7fe9bae887a0 20 read_log 194215'1178 (0'0) modify 1036baeb/benchmark_data_storage0111-ib.pocstor_19374_object21032/head//1 by client.1048726.0:21033 2014-08-12 17:11:33.842624 2014-08-13 17:38:02.552652 7fe9bae887a0 20 read_log 194215'1179 (0'0) modify 4a8e3aeb/benchmark_data_storage0111-ib.pocstor_19374_object21109/head//1 by client.1048726.0:21110 2014-08-12 17:11:34.314900 2014-08-13 17:38:02.552680 7fe9bae887a0 20 read_log 194215'1180 (0'0) modify 753fa2ab/benchmark_data_storage0111-ib.pocstor_19374_object21247/head//1 by client.1048726.0:21248 2014-08-12 17:11:35.099555 2014-08-13 17:38:02.552702 7fe9bae887a0 20 read_log 194215'1181 (0'0) modify f67584eb/benchmark_data_storage0111-ib.pocstor_19374_object21267/head//1 by client.1048726.0:21268 2014-08-12 17:11:35.186587 2014-08-13 17:38:02.552724 7fe9bae887a0 20 read_log 194215'1182 (0'0) modify fe828eab/benchmark_data_storage0111-ib.pocstor_19374_object21345/head//1 by client.1048726.0:21346 2014-08-12 17:11:36.038996 2014-08-13 17:38:02.552746 7fe9bae887a0 20 read_log 194215'1183 (0'0) modify 2ef9f66b/benchmark_data_storage0111-ib.pocstor_19374_object21352/head//1 by client.1048726.0:21353 2014-08-12 17:11:36.144223 2014-08-13 17:38:02.552767 7fe9bae887a0 20 read_log 194215'1184 (0'0) modify 3560716b/benchmark_data_storage0111-ib.pocstor_19374_object21409/head//1 by client.1048726.0:21410 2014-08-12 17:11:36.488438 2014-08-13 17:38:02.552789 7fe9bae887a0 20 read_log 194215'1185 (0'0) modify 7876b26b/benchmark_data_storage0111-ib.pocstor_19374_object21474/head//1 by client.1048726.0:21475 2014-08-12 17:11:36.881618 2014-08-13 17:38:02.552811 7fe9bae887a0 20 read_log 194215'1186 (0'0) modify a090132b/benchmark_data_storage0111-ib.pocstor_19374_object21525/head//1 by client.1048726.0:21526 2014-08-12 17:11:37.351706 2014-08-13 17:38:02.552834 7fe9bae887a0 20 read_log 194215'1187 (0'0) modify 2b02b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21551/head//1 by client.1048726.0:21552 2014-08-12 17:11:37.581206 2014-08-13 17:38:02.552856 7fe9bae887a0 20 read_log 194215'1188 (0'0) modify 2af0d52b/benchmark_data_storage0111-ib.pocstor_19374_object21651/head//1 by client.1048726.0:21652 2014-08-12 17:11:37.956609 2014-08-13 17:38:02.552879 7fe9bae887a0 20 read_log 194215'1189 (0'0) modify df94b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21660/head//1 by client.1048726.0:21661 2014-08-12 17:11:37.992064 2014-08-13 17:38:02.552901 7fe9bae887a0 20 read_log 194215'1190 (0'0) modify 157f89ab/benchmark_data_storage0111-ib.pocstor_19374_object21754/head//1 by client.1048726.0:21755 2014-08-12 17:11:38.345697 2014-08-13 17:38:02.552923 7fe9bae887a0 20 read_log 194215'1191 (0'0) modify 267742eb/benchmark_data_storage0111-ib.pocstor_19374_object21769/head//1 by client.1048726.0:21770 2014-08-12 17:11:38.400622 2014-08-13 17:38:02.552949 7fe9bae887a0 20 read_log 194215'1192 (0'0) modify 7086f06b/benchmark_data_storage0111-ib.pocstor_19374_object21770/head//1 by client.1048726.0:21771 2014-08-12 17:11:38.404254 2014-08-13 17:38:02.552971 7fe9bae887a0 20 read_log 194215'1193 (0'0) modify 5d6f6f6b/benchmark_data_storage0111-ib.pocstor_19374_object21877/head//1 by client.1048726.0:21878 2014-08-12 17:11:39.417066 2014-08-13 17:38:02.552996 7fe9bae887a0 20 read_log 194215'1194 (0'0) modify 6a5b3deb/benchmark_data_storage0111-ib.pocstor_19374_object21945/head//1 by client.1048726.0:21946 2014-08-12 17:11:39.685592 2014-08-13 17:38:02.553018 7fe9bae887a0 20 read_log 194215'1195 (0'0) modify 66a13f2b/benchmark_data_storage0111-ib.pocstor_19374_object22113/head//1 by client.1048726.0:22114 2014-08-12 17:11:40.338964 2014-08-13 17:38:02.553043 7fe9bae887a0 20 read_log 194215'1196 (0'0) modify 207ab2eb/benchmark_data_storage0111-ib.pocstor_19374_object22174/head//1 by client.1048726.0:22175 2014-08-12 17:11:40.569044 2014-08-13 17:38:02.553069 7fe9bae887a0 20 read_log 194215'1197 (0'0) modify 752cbeab/benchmark_data_storage0111-ib.pocstor_19374_object22208/head//1 by client.1048726.0:22209 2014-08-12 17:11:40.693348 2014-08-13 17:38:02.553094 7fe9bae887a0 20 read_log 194215'1198 (0'0) modify 5f7e08eb/benchmark_data_storage0111-ib.pocstor_19374_object22348/head//1 by client.1048726.0:22349 2014-08-12 17:11:41.214701 2014-08-13 17:38:02.553131 7fe9bae887a0 20 read_log 194215'1199 (0'0) modify 1c91296b/benchmark_data_storage0111-ib.pocstor_19374_object22399/head//1 by client.1048726.0:22400 2014-08-12 17:11:41.403636 2014-08-13 17:38:02.553165 7fe9bae887a0 20 read_log 194215'1200 (0'0) modify ed68a1eb/benchmark_data_storage0111-ib.pocstor_19374_object22466/head//1 by client.1048726.0:22467 2014-08-12 17:11:41.711256 2014-08-13 17:38:02.553188 7fe9bae887a0 20 read_log 194215'1201 (0'0) modify db847beb/benchmark_data_storage0111-ib.pocstor_19374_object22490/head//1 by client.1048726.0:22491 2014-08-12 17:11:41.840569 2014-08-13 17:38:02.553210 7fe9bae887a0 20 read_log 194215'1202 (0'0) modify 9b61f7eb/benchmark_data_storage0111-ib.pocstor_19374_object22596/head//1 by client.1048726.0:22597 2014-08-12 17:11:42.312656 2014-08-13 17:38:02.553232 7fe9bae887a0 20 read_log 194215'1203 (0'0) modify 8df4ebab/benchmark_data_storage0111-ib.pocstor_19374_object22647/head//1 by client.1048726.0:22648 2014-08-12 17:11:42.552403 2014-08-13 17:38:02.553253 7fe9bae887a0 20 read_log 194215'1204 (0'0) modify ade90cab/benchmark_data_storage0111-ib.pocstor_19374_object22661/head//1 by client.1048726.0:22662 2014-08-12 17:11:42.626530 2014-08-13 17:38:02.553274 7fe9bae887a0 20 read_log 194215'1205 (0'0) modify d2df432b/benchmark_data_storage0111-ib.pocstor_19374_object22688/head//1 by client.1048726.0:22689 2014-08-12 17:11:42.727273 2014-08-13 17:38:02.553296 7fe9bae887a0 20 read_log 194215'1206 (0'0) modify f0a1f42b/benchmark_data_storage0111-ib.pocstor_19374_object22740/head//1 by client.1048726.0:22741 2014-08-12 17:11:42.942474 2014-08-13 17:38:02.553317 7fe9bae887a0 20 read_log 194215'1207 (0'0) modify b467b16b/benchmark_data_storage0111-ib.pocstor_19374_object22893/head//1 by client.1048726.0:22894 2014-08-12 17:11:43.526333 2014-08-13 17:38:02.553338 7fe9bae887a0 20 read_log 194215'1208 (0'0) modify 2febd7ab/benchmark_data_storage0111-ib.pocstor_19374_object22894/head//1 by client.1048726.0:22895 2014-08-12 17:11:43.530043 2014-08-13 17:38:02.553360 7fe9bae887a0 20 read_log 194215'1209 (0'0) modify e0db75ab/benchmark_data_storage0111-ib.pocstor_19374_object22965/head//1 by client.1048726.0:22966 2014-08-12 17:11:43.911346 2014-08-13 17:38:02.553383 7fe9bae887a0 20 read_log 194215'1210 (0'0) modify 6344e4eb/benchmark_data_storage0111-ib.pocstor_19374_object22992/head//1 by client.1048726.0:22993 2014-08-12 17:11:44.143999 2014-08-13 17:38:02.553406 7fe9bae887a0 20 read_log 194215'1211 (0'0) modify 7ba9fd2b/benchmark_data_storage0111-ib.pocstor_19374_object23012/head//1 by client.1048726.0:23013 2014-08-12 17:11:44.237251 2014-08-13 17:38:02.553428 7fe9bae887a0 20 read_log 194215'1212 (0'0) modify 8666e06b/benchmark_data_storage0111-ib.pocstor_19374_object23095/head//1 by client.1048726.0:23096 2014-08-12 17:11:44.628537 2014-08-13 17:38:02.553449 7fe9bae887a0 20 read_log 194215'1213 (0'0) modify ed901d2b/benchmark_data_storage0111-ib.pocstor_19374_object23113/head//1 by client.1048726.0:23114 2014-08-12 17:11:44.719472 2014-08-13 17:38:02.553471 7fe9bae887a0 20 read_log 194215'1214 (0'0) modify 84443b6b/benchmark_data_storage0111-ib.pocstor_19374_object23132/head//1 by client.1048726.0:23133 2014-08-12 17:11:44.788734 2014-08-13 17:38:02.553492 7fe9bae887a0 20 read_log 194215'1215 (0'0) modify 8cff746b/benchmark_data_storage0111-ib.pocstor_19374_object23361/head//1 by client.1048726.0:23362 2014-08-12 17:11:46.106681 2014-08-13 17:38:02.553514 7fe9bae887a0 20 read_log 194215'1216 (0'0) modify 4edb3b2b/benchmark_data_storage0111-ib.pocstor_19374_object23379/head//1 by client.1048726.0:23380 2014-08-12 17:11:46.174136 2014-08-13 17:38:02.553535 7fe9bae887a0 20 read_log 194215'1217 (0'0) modify 1c53ca2b/benchmark_data_storage0111-ib.pocstor_19374_object23426/head//1 by client.1048726.0:23427 2014-08-12 17:11:46.350067 2014-08-13 17:38:02.553556 7fe9bae887a0 20 read_log 194215'1218 (0'0) modify fa5b282b/benchmark_data_storage0111-ib.pocstor_19374_object23429/head//1 by client.1048726.0:23430 2014-08-12 17:11:46.361085 2014-08-13 17:38:02.553589 7fe9bae887a0 20 read_log 194215'1219 (0'0) modify 5334536b/benchmark_data_storage0111-ib.pocstor_19374_object23433/head//1 by client.1048726.0:23434 2014-08-12 17:11:46.376233 2014-08-13 17:38:02.553611 7fe9bae887a0 20 read_log 194215'1220 (0'0) modify 6211b72b/benchmark_data_storage0111-ib.pocstor_19374_object23453/head//1 by client.1048726.0:23454 2014-08-12 17:11:46.452007 2014-08-13 17:38:02.553633 7fe9bae887a0 20 read_log 194215'1221 (0'0) modify 2514846b/benchmark_data_storage0111-ib.pocstor_19374_object23592/head//1 by client.1048726.0:23593 2014-08-12 17:11:47.010317 2014-08-13 17:38:02.553655 7fe9bae887a0 20 read_log 194215'1222 (0'0) modify aae84e2b/benchmark_data_storage0111-ib.pocstor_19374_object23706/head//1 by client.1048726.0:23707 2014-08-12 17:11:47.503011 2014-08-13 17:38:02.553679 7fe9bae887a0 20 read_log 194215'1223 (0'0) modify b9e840ab/benchmark_data_storage0111-ib.pocstor_19374_object23839/head//1 by client.1048726.0:23840 2014-08-12 17:11:48.002145 2014-08-13 17:38:02.553706 7fe9bae887a0 20 read_log 194215'1224 (0'0) modify 4cb71eb/benchmark_data_storage0111-ib.pocstor_19374_object23922/head//1 by client.1048726.0:23923 2014-08-12 17:11:48.315188 2014-08-13 17:38:02.553732 7fe9bae887a0 20 read_log 194215'1225 (0'0) modify e4b2e06b/benchmark_data_storage0111-ib.pocstor_19374_object23948/head//1 by client.1048726.0:23949 2014-08-12 17:11:48.412255 2014-08-13 17:38:02.553754 7fe9bae887a0 20 read_log 194215'1226 (0'0) modify d835436b/benchmark_data_storage0111-ib.pocstor_19374_object24094/head//1 by client.1048726.0:24095 2014-08-12 17:11:48.970521 2014-08-13 17:38:02.553775 7fe9bae887a0 20 read_log 194215'1227 (0'0) modify a7483a2b/benchmark_data_storage0111-ib.pocstor_19374_object24133/head//1 by client.1048726.0:24134 2014-08-12 17:11:49.165289 2014-08-13 17:38:02.553797 7fe9bae887a0 20 read_log 194215'1228 (0'0) modify 4afb40eb/benchmark_data_storage0111-ib.pocstor_19374_object24160/head//1 by client.1048726.0:24161 2014-08-12 17:11:49.424597 2014-08-13 17:38:02.553818 7fe9bae887a0 20 read_log 194215'1229 (0'0) modify 273f5cab/benchmark_data_storage0111-ib.pocstor_19374_object24162/head//1 by client.1048726.0:24163 2014-08-12 17:11:49.437223 2014-08-13 17:38:02.553840 7fe9bae887a0 20 read_log 194215'1230 (0'0) modify 438a14eb/benchmark_data_storage0111-ib.pocstor_19374_object24263/head//1 by client.1048726.0:24264 2014-08-12 17:11:49.898535 2014-08-13 17:38:02.553861 7fe9bae887a0 20 read_log 194215'1231 (0'0) modify 33c0b76b/benchmark_data_storage0111-ib.pocstor_19374_object24565/head//1 by client.1048726.0:24566 2014-08-12 17:11:51.610931 2014-08-13 17:38:02.553882 7fe9bae887a0 20 read_log 194215'1232 (0'0) modify c08eceab/benchmark_data_storage0111-ib.pocstor_19374_object24597/head//1 by client.1048726.0:24598 2014-08-12 17:11:51.732984 2014-08-13 17:38:02.553904 7fe9bae887a0 20 read_log 194215'1233 (0'0) modify befd7d6b/benchmark_data_storage0111-ib.pocstor_19374_object24728/head//1 by client.1048726.0:24729 2014-08-12 17:11:52.255888 2014-08-13 17:38:02.553925 7fe9bae887a0 20 read_log 194215'1234 (0'0) modify e425ca6b/benchmark_data_storage0111-ib.pocstor_19374_object24734/head//1 by client.1048726.0:24735 2014-08-12 17:11:52.278843 2014-08-13 17:38:02.553947 7fe9bae887a0 20 read_log 194215'1235 (0'0) modify 961e2f2b/benchmark_data_storage0111-ib.pocstor_19374_object24748/head//1 by client.1048726.0:24749 2014-08-12 17:11:52.345353 2014-08-13 17:38:02.553968 7fe9bae887a0 20 read_log 194215'1236 (0'0) modify 4a2485eb/benchmark_data_storage0111-ib.pocstor_19374_object24798/head//1 by client.1048726.0:24799 2014-08-12 17:11:52.679872 2014-08-13 17:38:02.553990 7fe9bae887a0 20 read_log 194215'1237 (0'0) modify 66eb74eb/benchmark_data_storage0111-ib.pocstor_19374_object24869/head//1 by client.1048726.0:24870 2014-08-12 17:11:53.059794 2014-08-13 17:38:02.554011 7fe9bae887a0 20 read_log 194215'1238 (0'0) modify cdd7d8eb/benchmark_data_storage0111-ib.pocstor_19374_object24973/head//1 by client.1048726.0:24974 2014-08-12 17:11:53.452075 2014-08-13 17:38:02.554045 7fe9bae887a0 20 read_log 194215'1239 (0'0) modify a8ff1b2b/benchmark_data_storage0111-ib.pocstor_19374_object25163/head//1 by client.1048726.0:25164 2014-08-12 17:11:54.516344 2014-08-13 17:38:02.554067 7fe9bae887a0 20 read_log 194215'1240 (0'0) modify 80a012eb/benchmark_data_storage0111-ib.pocstor_19374_object25209/head//1 by client.1048726.0:25210 2014-08-12 17:11:54.712278 2014-08-13 17:38:02.554089 7fe9bae887a0 20 read_log 194215'1241 (0'0) modify 6a06c22b/benchmark_data_storage0111-ib.pocstor_19374_object25224/head//1 by client.1048726.0:25225 2014-08-12 17:11:54.768778 2014-08-13 17:38:02.554111 7fe9bae887a0 20 read_log 194215'1242 (0'0) modify 1c47e46b/benchmark_data_storage0111-ib.pocstor_19374_object25488/head//1 by client.1048726.0:25489 2014-08-12 17:11:56.124305 2014-08-13 17:38:02.554132 7fe9bae887a0 20 read_log 194215'1243 (0'0) modify d92e436b/benchmark_data_storage0111-ib.pocstor_19374_object25630/head//1 by client.1048726.0:25631 2014-08-12 17:11:56.760259 2014-08-13 17:38:02.554160 7fe9bae887a0 20 read_log 194215'1244 (0'0) modify abae06ab/benchmark_data_storage0111-ib.pocstor_19374_object25701/head//1 by client.1048726.0:25702 2014-08-12 17:11:57.038710 2014-08-13 17:38:02.554184 7fe9bae887a0 20 read_log 194215'1245 (0'0) modify 63f6f26b/benchmark_data_storage0111-ib.pocstor_19374_object25733/head//1 by client.1048726.0:25734 2014-08-12 17:11:57.157771 2014-08-13 17:38:02.554205 7fe9bae887a0 20 read_log 194215'1246 (0'0) modify 65c6a56b/benchmark_data_storage0111-ib.pocstor_19374_object25806/head//1 by client.1048726.0:25807 2014-08-12 17:11:57.429477 2014-08-13 17:38:02.554227 7fe9bae887a0 20 read_log 194215'1247 (0'0) modify f2f70f6b/benchmark_data_storage0111-ib.pocstor_19374_object25849/head//1 by client.1048726.0:25850 2014-08-12 17:11:57.588962 2014-08-13 17:38:02.554249 7fe9bae887a0 20 read_log 194215'1248 (0'0) modify c4335c2b/benchmark_data_storage0111-ib.pocstor_19374_object25922/head//1 by client.1048726.0:25923 2014-08-12 17:11:57.923719 2014-08-13 17:38:02.554270 7fe9bae887a0 20 read_log 194215'1249 (0'0) modify 559f96eb/benchmark_data_storage0111-ib.pocstor_19374_object26064/head//1 by client.1048726.0:26065 2014-08-12 17:11:58.551746 2014-08-13 17:38:02.554292 7fe9bae887a0 20 read_log 194215'1250 (0'0) modify b6adc82b/benchmark_data_storage0111-ib.pocstor_19374_object26221/head//1 by client.1048726.0:26222 2014-08-12 17:11:59.139076 2014-08-13 17:38:02.554316 7fe9bae887a0 20 read_log 194215'1251 (0'0) modify 76fd52ab/benchmark_data_storage0111-ib.pocstor_19374_object26229/head//1 by client.1048726.0:26230 2014-08-12 17:11:59.168189 2014-08-13 17:38:02.554338 7fe9bae887a0 20 read_log 194215'1252 (0'0) modify 94dc9c2b/benchmark_data_storage0111-ib.pocstor_19374_object26322/head//1 by client.1048726.0:26323 2014-08-12 17:11:59.523186 2014-08-13 17:38:02.554395 7fe9bae887a0 20 read_log 194215'1253 (0'0) modify 7560a26b/benchmark_data_storage0111-ib.pocstor_19374_object26345/head//1 by client.1048726.0:26346 2014-08-12 17:11:59.608713 2014-08-13 17:38:02.554418 7fe9bae887a0 20 read_log 194215'1254 (0'0) modify 1f77502b/benchmark_data_storage0111-ib.pocstor_19374_object26385/head//1 by client.1048726.0:26386 2014-08-12 17:11:59.764776 2014-08-13 17:38:02.554440 7fe9bae887a0 20 read_log 194215'1255 (0'0) modify 6b90772b/benchmark_data_storage0111-ib.pocstor_19374_object26540/head//1 by client.1048726.0:26541 2014-08-12 17:12:00.347803 2014-08-13 17:38:02.554462 7fe9bae887a0 20 read_log 194215'1256 (194215'839) delete 6fb923eb/benchmark_data_storage0111-ib.pocstor_19374_object0/head//1 by client.1048726.0:26608 2014-08-12 17:12:01.646592 2014-08-13 17:38:02.554489 7fe9bae887a0 20 read_log 194215'1257 (194215'840) delete f1c09dab/benchmark_data_storage0111-ib.pocstor_19374_object53/head//1 by client.1048726.0:26661 2014-08-12 17:12:01.665773 2014-08-13 17:38:02.554511 7fe9bae887a0 20 read_log 194215'1258 (194215'841) delete 322ce82b/benchmark_data_storage0111-ib.pocstor_19374_object55/head//1 by client.1048726.0:26663 2014-08-12 17:12:01.665897 2014-08-13 17:38:02.554551 7fe9bae887a0 20 read_log 194215'1259 (194215'842) delete 42b2c8ab/benchmark_data_storage0111-ib.pocstor_19374_object77/head//1 by client.1048726.0:26685 2014-08-12 17:12:01.681085 2014-08-13 17:38:02.554578 7fe9bae887a0 20 read_log 194215'1260 (194215'843) delete 3792f3eb/benchmark_data_storage0111-ib.pocstor_19374_object104/head//1 by client.1048726.0:26712 2014-08-12 17:12:01.698569 2014-08-13 17:38:02.554601 7fe9bae887a0 20 read_log 194215'1261 (194215'844) delete c21e90ab/benchmark_data_storage0111-ib.pocstor_19374_object106/head//1 by client.1048726.0:26714 2014-08-12 17:12:01.699070 2014-08-13 17:38:02.554655 7fe9bae887a0 20 read_log 194215'1262 (194215'845) delete f13767eb/benchmark_data_storage0111-ib.pocstor_19374_object112/head//1 by client.1048726.0:26720 2014-08-12 17:12:01.703304 2014-08-13 17:38:02.554683 7fe9bae887a0 20 read_log 194215'1263 (194215'846) delete 90f4b5eb/benchmark_data_storage0111-ib.pocstor_19374_object117/head//1 by client.1048726.0:26725 2014-08-12 17:12:01.708569 2014-08-13 17:38:02.554705 7fe9bae887a0 20 read_log 194215'1264 (194215'847) delete ec96382b/benchmark_data_storage0111-ib.pocstor_19374_object134/head//1 by client.1048726.0:26742 2014-08-12 17:12:01.726473 2014-08-13 17:38:02.554727 7fe9bae887a0 20 read_log 194215'1265 (194215'848) delete 58c1d5eb/benchmark_data_storage0111-ib.pocstor_19374_object146/head//1 by client.1048726.0:26754 2014-08-12 17:12:01.733928 2014-08-13 17:38:02.554749 7fe9bae887a0 20 read_log 194215'1266 (194215'849) delete fb95d6ab/benchmark_data_storage0111-ib.pocstor_19374_object204/head//1 by client.1048726.0:26812 2014-08-12 17:12:01.805343 2014-08-13 17:38:02.554770 7fe9bae887a0 20 read_log 194215'1267 (194215'850) delete b112016b/benchmark_data_storage0111-ib.pocstor_19374_object273/head//1 by client.1048726.0:26881 2014-08-12 17:12:01.970029 2014-08-13 17:38:02.554792 7fe9bae887a0 20 read_log 194215'1268 (194215'851) delete a02a832b/benchmark_data_storage0111-ib.pocstor_19374_object284/head//1 by client.1048726.0:26892 2014-08-12 17:12:01.978514 2014-08-13 17:38:02.554814 7fe9bae887a0 20 read_log 194215'1269 (194215'852) delete a5225cab/benchmark_data_storage0111-ib.pocstor_19374_object539/head//1 by client.1048726.0:27147 2014-08-12 17:12:03.613247 2014-08-13 17:38:02.554840 7fe9bae887a0 20 read_log 194215'1270 (194215'853) delete a60191ab/benchmark_data_storage0111-ib.pocstor_19374_object649/head//1 by client.1048726.0:27257 2014-08-12 17:12:03.717744 2014-08-13 17:38:02.554862 7fe9bae887a0 20 read_log 194215'1271 (194215'854) delete 6fc2f02b/benchmark_data_storage0111-ib.pocstor_19374_object659/head//1 by client.1048726.0:27267 2014-08-12 17:12:03.725953 2014-08-13 17:38:02.554884 7fe9bae887a0 20 read_log 194215'1272 (194215'855) delete 7f48662b/benchmark_data_storage0111-ib.pocstor_19374_object1055/head//1 by client.1048726.0:27663 2014-08-12 17:12:04.650034 2014-08-13 17:38:02.554906 7fe9bae887a0 20 read_log 194215'1273 (194215'856) delete 17650d2b/benchmark_data_storage0111-ib.pocstor_19374_object1157/head//1 by client.1048726.0:27765 2014-08-12 17:12:04.690623 2014-08-13 17:38:02.554930 7fe9bae887a0 20 read_log 194215'1274 (194215'857) delete 39bc27ab/benchmark_data_storage0111-ib.pocstor_19374_object1180/head//1 by client.1048726.0:27788 2014-08-12 17:12:04.699424 2014-08-13 17:38:02.554953 7fe9bae887a0 20 read_log 194215'1275 (194215'858) delete e51879ab/benchmark_data_storage0111-ib.pocstor_19374_object1244/head//1 by client.1048726.0:27852 2014-08-12 17:12:04.726231 2014-08-13 17:38:02.554975 7fe9bae887a0 20 read_log 194215'1276 (194215'859) delete fefa32eb/benchmark_data_storage0111-ib.pocstor_19374_object1265/head//1 by client.1048726.0:27873 2014-08-12 17:12:04.734258 2014-08-13 17:38:02.554997 7fe9bae887a0 20 read_log 194215'1277 (194215'860) delete ee0c2aab/benchmark_data_storage0111-ib.pocstor_19374_object1278/head//1 by client.1048726.0:27886 2014-08-12 17:12:04.743340 2014-08-13 17:38:02.555019 7fe9bae887a0 20 read_log 194215'1278 (194215'861) delete 3560e9ab/benchmark_data_storage0111-ib.pocstor_19374_object1318/head//1 by client.1048726.0:27926 2014-08-12 17:12:04.760426 2014-08-13 17:38:02.555052 7fe9bae887a0 20 read_log 194215'1279 (194215'862) delete 510c1a6b/benchmark_data_storage0111-ib.pocstor_19374_object1337/head//1 by client.1048726.0:27945 2014-08-12 17:12:04.771262 2014-08-13 17:38:02.555079 7fe9bae887a0 20 read_log 194215'1280 (194215'863) delete 2298276b/benchmark_data_storage0111-ib.pocstor_19374_object1339/head//1 by client.1048726.0:27947 2014-08-12 17:12:04.772474 2014-08-13 17:38:02.555101 7fe9bae887a0 20 read_log 194215'1281 (194215'864) delete 6340222b/benchmark_data_storage0111-ib.pocstor_19374_object1349/head//1 by client.1048726.0:27957 2014-08-12 17:12:04.777947 2014-08-13 17:38:02.555123 7fe9bae887a0 20 read_log 194215'1282 (194215'865) delete 96f019ab/benchmark_data_storage0111-ib.pocstor_19374_object1487/head//1 by client.1048726.0:28095 2014-08-12 17:12:05.020409 2014-08-13 17:38:02.555145 7fe9bae887a0 20 read_log 194215'1283 (194215'866) delete 309188eb/benchmark_data_storage0111-ib.pocstor_19374_object1517/head//1 by client.1048726.0:28125 2014-08-12 17:12:05.050683 2014-08-13 17:38:02.555177 7fe9bae887a0 20 read_log 194215'1284 (194215'867) delete 5e9b32eb/benchmark_data_storage0111-ib.pocstor_19374_object1568/head//1 by client.1048726.0:28176 2014-08-12 17:12:05.080999 2014-08-13 17:38:02.555199 7fe9bae887a0 20 read_log 194215'1285 (194215'868) delete bb2497ab/benchmark_data_storage0111-ib.pocstor_19374_object1632/head//1 by client.1048726.0:28240 2014-08-12 17:12:05.101271 2014-08-13 17:38:02.555221 7fe9bae887a0 20 read_log 194215'1286 (194215'869) delete cf5e382b/benchmark_data_storage0111-ib.pocstor_19374_object1744/head//1 by client.1048726.0:28352 2014-08-12 17:12:05.136706 2014-08-13 17:38:02.555243 7fe9bae887a0 20 read_log 194215'1287 (194215'870) delete 3c5c7dab/benchmark_data_storage0111-ib.pocstor_19374_object1756/head//1 by client.1048726.0:28364 2014-08-12 17:12:05.139966 2014-08-13 17:38:02.555265 7fe9bae887a0 20 read_log 194215'1288 (194215'871) delete 724fd5eb/benchmark_data_storage0111-ib.pocstor_19374_object1757/head//1 by client.1048726.0:28365 2014-08-12 17:12:05.140017 2014-08-13 17:38:02.555287 7fe9bae887a0 20 read_log 194215'1289 (194215'872) delete 912098ab/benchmark_data_storage0111-ib.pocstor_19374_object1776/head//1 by client.1048726.0:28384 2014-08-12 17:12:05.147015 2014-08-13 17:38:02.555309 7fe9bae887a0 20 read_log 194215'1290 (194215'873) delete 3939e16b/benchmark_data_storage0111-ib.pocstor_19374_object1894/head//1 by client.1048726.0:28502 2014-08-12 17:12:05.188935 2014-08-13 17:38:02.555331 7fe9bae887a0 20 read_log 194215'1291 (194215'874) delete 6a0792eb/benchmark_data_storage0111-ib.pocstor_19374_object1896/head//1 by client.1048726.0:28504 2014-08-12 17:12:05.189089 2014-08-13 17:38:02.555359 7fe9bae887a0 20 read_log 194215'1292 (194215'875) delete 6f099eeb/benchmark_data_storage0111-ib.pocstor_19374_object2059/head//1 by client.1048726.0:28667 2014-08-12 17:12:05.242177 2014-08-13 17:38:02.555382 7fe9bae887a0 20 read_log 194215'1293 (194215'876) delete eafb902b/benchmark_data_storage0111-ib.pocstor_19374_object2077/head//1 by client.1048726.0:28685 2014-08-12 17:12:05.248508 2014-08-13 17:38:02.555404 7fe9bae887a0 20 read_log 194215'1294 (194215'877) delete 946c36eb/benchmark_data_storage0111-ib.pocstor_19374_object2102/head//1 by client.1048726.0:28710 2014-08-12 17:12:05.258439 2014-08-13 17:38:02.555425 7fe9bae887a0 20 read_log 194215'1295 (194215'878) delete f47e62b/benchmark_data_storage0111-ib.pocstor_19374_object2180/head//1 by client.1048726.0:28788 2014-08-12 17:12:05.287419 2014-08-13 17:38:02.555447 7fe9bae887a0 20 read_log 194215'1296 (194215'879) delete 934f442b/benchmark_data_storage0111-ib.pocstor_19374_object2262/head//1 by client.1048726.0:28870 2014-08-12 17:12:05.327409 2014-08-13 17:38:02.555469 7fe9bae887a0 20 read_log 194215'1297 (194215'880) delete 7f57086b/benchmark_data_storage0111-ib.pocstor_19374_object2343/head//1 by client.1048726.0:28951 2014-08-12 17:12:05.359161 2014-08-13 17:38:02.555491 7fe9bae887a0 20 read_log 194215'1298 (194215'881) delete 13d2392b/benchmark_data_storage0111-ib.pocstor_19374_object2352/head//1 by client.1048726.0:28960 2014-08-12 17:12:05.362707 2014-08-13 17:38:02.555558 7fe9bae887a0 20 read_log 194215'1299 (194215'882) delete 3d536ceb/benchmark_data_storage0111-ib.pocstor_19374_object2361/head//1 by client.1048726.0:28969 2014-08-12 17:12:05.366551 2014-08-13 17:38:02.555587 7fe9bae887a0 20 read_log 194215'1300 (194215'883) delete c6e9f26b/benchmark_data_storage0111-ib.pocstor_19374_object2394/head//1 by client.1048726.0:29002 2014-08-12 17:12:05.401353 2014-08-13 17:38:02.555609 7fe9bae887a0 20 read_log 194215'1301 (194215'884) delete 85e6a52b/benchmark_data_storage0111-ib.pocstor_19374_object2418/head//1 by client.1048726.0:29026 2014-08-12 17:12:05.476470 2014-08-13 17:38:02.555631 7fe9bae887a0 20 read_log 194215'1302 (194215'885) delete c7cc53ab/benchmark_data_storage0111-ib.pocstor_19374_object2432/head//1 by client.1048726.0:29040 2014-08-12 17:12:05.483029 2014-08-13 17:38:02.555653 7fe9bae887a0 20 read_log 194215'1303 (194215'886) delete 3a7fe4ab/benchmark_data_storage0111-ib.pocstor_19374_object2594/head//1 by client.1048726.0:29202 2014-08-12 17:12:05.607483 2014-08-13 17:38:02.555678 7fe9bae887a0 20 read_log 194215'1304 (194215'887) delete 4305176b/benchmark_data_storage0111-ib.pocstor_19374_object2684/head//1 by client.1048726.0:29292 2014-08-12 17:12:06.259135 2014-08-13 17:38:02.555701 7fe9bae887a0 20 read_log 194215'1305 (194215'888) delete e7b7062b/benchmark_data_storage0111-ib.pocstor_19374_object2725/head//1 by client.1048726.0:29333 2014-08-12 17:12:06.306363 2014-08-13 17:38:02.555723 7fe9bae887a0 20 read_log 194215'1306 (194215'889) delete a1e105ab/benchmark_data_storage0111-ib.pocstor_19374_object2744/head//1 by client.1048726.0:29352 2014-08-12 17:12:06.316619 2014-08-13 17:38:02.555745 7fe9bae887a0 20 read_log 194215'1307 (194215'890) delete 9bee462b/benchmark_data_storage0111-ib.pocstor_19374_object2763/head//1 by client.1048726.0:29371 2014-08-12 17:12:06.322373 2014-08-13 17:38:02.555767 7fe9bae887a0 20 read_log 194215'1308 (194215'891) delete 2d10016b/benchmark_data_storage0111-ib.pocstor_19374_object2791/head//1 by client.1048726.0:29399 2014-08-12 17:12:06.334504 2014-08-13 17:38:02.555790 7fe9bae887a0 20 read_log 194215'1309 (194215'892) delete 7464b42b/benchmark_data_storage0111-ib.pocstor_19374_object2865/head//1 by client.1048726.0:29473 2014-08-12 17:12:06.375063 2014-08-13 17:38:02.555812 7fe9bae887a0 20 read_log 194215'1310 (194215'893) delete 6a28beb/benchmark_data_storage0111-ib.pocstor_19374_object2901/head//1 by client.1048726.0:29509 2014-08-12 17:12:06.403708 2014-08-13 17:38:02.555834 7fe9bae887a0 20 read_log 194215'1311 (194215'894) delete df15306b/benchmark_data_storage0111-ib.pocstor_19374_object2944/head//1 by client.1048726.0:29552 2014-08-12 17:12:06.459753 2014-08-13 17:38:02.555856 7fe9bae887a0 20 read_log 194215'1312 (194215'895) delete 505175ab/benchmark_data_storage0111-ib.pocstor_19374_object2950/head//1 by client.1048726.0:29558 2014-08-12 17:12:06.463067 2014-08-13 17:38:02.555878 7fe9bae887a0 20 read_log 194215'1313 (194215'896) delete 3fa3deeb/benchmark_data_storage0111-ib.pocstor_19374_object3045/head//1 by client.1048726.0:29653 2014-08-12 17:12:06.496969 2014-08-13 17:38:02.555900 7fe9bae887a0 20 read_log 194215'1314 (194215'897) delete f1bcb3ab/benchmark_data_storage0111-ib.pocstor_19374_object3073/head//1 by client.1048726.0:29681 2014-08-12 17:12:06.508057 2014-08-13 17:38:02.555921 7fe9bae887a0 20 read_log 194215'1315 (194215'898) delete 6eb5e6ab/benchmark_data_storage0111-ib.pocstor_19374_object3178/head//1 by client.1048726.0:29786 2014-08-12 17:12:06.540511 2014-08-13 17:38:02.555943 7fe9bae887a0 20 read_log 194215'1316 (194215'899) delete 71213f6b/benchmark_data_storage0111-ib.pocstor_19374_object3222/head//1 by client.1048726.0:29830 2014-08-12 17:12:06.553801 2014-08-13 17:38:02.555965 7fe9bae887a0 20 read_log 194215'1317 (194215'900) delete 2847d6ab/benchmark_data_storage0111-ib.pocstor_19374_object3286/head//1 by client.1048726.0:29894 2014-08-12 17:12:06.576296 2014-08-13 17:38:02.555990 7fe9bae887a0 20 read_log 194215'1318 (194215'901) delete b38c2f6b/benchmark_data_storage0111-ib.pocstor_19374_object3415/head//1 by client.1048726.0:30023 2014-08-12 17:12:06.615270 2014-08-13 17:38:02.556024 7fe9bae887a0 20 read_log 194215'1319 (194215'902) delete 518a656b/benchmark_data_storage0111-ib.pocstor_19374_object3454/head//1 by client.1048726.0:30062 2014-08-12 17:12:06.627505 2014-08-13 17:38:02.556051 7fe9bae887a0 20 read_log 194215'1320 (194215'903) delete cad818ab/benchmark_data_storage0111-ib.pocstor_19374_object3466/head//1 by client.1048726.0:30074 2014-08-12 17:12:06.631630 2014-08-13 17:38:02.556074 7fe9bae887a0 20 read_log 194215'1321 (194215'904) delete 52ed8aab/benchmark_data_storage0111-ib.pocstor_19374_object3511/head//1 by client.1048726.0:30119 2014-08-12 17:12:06.644221 2014-08-13 17:38:02.556096 7fe9bae887a0 20 read_log 194215'1322 (194215'905) delete 376b872b/benchmark_data_storage0111-ib.pocstor_19374_object3523/head//1 by client.1048726.0:30131 2014-08-12 17:12:06.647796 2014-08-13 17:38:02.556117 7fe9bae887a0 20 read_log 194215'1323 (194215'906) delete 69cdf92b/benchmark_data_storage0111-ib.pocstor_19374_object3584/head//1 by client.1048726.0:30192 2014-08-12 17:12:06.672741 2014-08-13 17:38:02.556139 7fe9bae887a0 20 read_log 194215'1324 (194215'907) delete 5dad4cab/benchmark_data_storage0111-ib.pocstor_19374_object3612/head//1 by client.1048726.0:30220 2014-08-12 17:12:06.684853 2014-08-13 17:38:02.556170 7fe9bae887a0 20 read_log 194215'1325 (194215'908) delete 8a29ddeb/benchmark_data_storage0111-ib.pocstor_19374_object3677/head//1 by client.1048726.0:30285 2014-08-12 17:12:06.772314 2014-08-13 17:38:02.556193 7fe9bae887a0 20 read_log 194215'1326 (194215'909) delete cb77c3ab/benchmark_data_storage0111-ib.pocstor_19374_object3717/head//1 by client.1048726.0:30325 2014-08-12 17:12:06.796707 2014-08-13 17:38:02.556215 7fe9bae887a0 20 read_log 194215'1327 (194215'910) delete 7096f52b/benchmark_data_storage0111-ib.pocstor_19374_object3810/head//1 by client.1048726.0:30418 2014-08-12 17:12:06.841884 2014-08-13 17:38:02.556237 7fe9bae887a0 20 read_log 194215'1328 (194215'911) delete ee1fffeb/benchmark_data_storage0111-ib.pocstor_19374_object3837/head//1 by client.1048726.0:30445 2014-08-12 17:12:06.858370 2014-08-13 17:38:02.556259 7fe9bae887a0 20 read_log 194215'1329 (194215'912) delete a32a26eb/benchmark_data_storage0111-ib.pocstor_19374_object3838/head//1 by client.1048726.0:30446 2014-08-12 17:12:06.858972 2014-08-13 17:38:02.556281 7fe9bae887a0 20 read_log 194215'1330 (194215'913) delete 6fb1deeb/benchmark_data_storage0111-ib.pocstor_19374_object4048/head//1 by client.1048726.0:30656 2014-08-12 17:12:06.964556 2014-08-13 17:38:02.556303 7fe9bae887a0 20 read_log 194215'1331 (194215'914) delete 6c58c7ab/benchmark_data_storage0111-ib.pocstor_19374_object4059/head//1 by client.1048726.0:30667 2014-08-12 17:12:06.969486 2014-08-13 17:38:02.556326 7fe9bae887a0 20 read_log 194215'1332 (194215'915) delete 660f996b/benchmark_data_storage0111-ib.pocstor_19374_object4105/head//1 by client.1048726.0:30713 2014-08-12 17:12:06.999342 2014-08-13 17:38:02.556388 7fe9bae887a0 20 read_log 194215'1333 (194215'916) delete 612f78ab/benchmark_data_storage0111-ib.pocstor_19374_object4125/head//1 by client.1048726.0:30733 2014-08-12 17:12:07.010599 2014-08-13 17:38:02.556411 7fe9bae887a0 20 read_log 194215'1334 (194215'917) delete bd30982b/benchmark_data_storage0111-ib.pocstor_19374_object4136/head//1 by client.1048726.0:30744 2014-08-12 17:12:07.016489 2014-08-13 17:38:02.556434 7fe9bae887a0 20 read_log 194215'1335 (194215'918) delete d1ec98ab/benchmark_data_storage0111-ib.pocstor_19374_object4183/head//1 by client.1048726.0:30791 2014-08-12 17:12:07.107874 2014-08-13 17:38:02.556456 7fe9bae887a0 20 read_log 194215'1336 (194215'919) delete cd9ad76b/benchmark_data_storage0111-ib.pocstor_19374_object4187/head//1 by client.1048726.0:30795 2014-08-12 17:12:07.127328 2014-08-13 17:38:02.556478 7fe9bae887a0 20 read_log 194215'1337 (194215'920) delete 6236346b/benchmark_data_storage0111-ib.pocstor_19374_object4191/head//1 by client.1048726.0:30799 2014-08-12 17:12:07.136540 2014-08-13 17:38:02.556499 7fe9bae887a0 20 read_log 194215'1338 (194215'921) delete c47eb16b/benchmark_data_storage0111-ib.pocstor_19374_object4242/head//1 by client.1048726.0:30850 2014-08-12 17:12:07.156218 2014-08-13 17:38:02.556533 7fe9bae887a0 20 read_log 194215'1339 (194215'922) delete 7f4a59eb/benchmark_data_storage0111-ib.pocstor_19374_object4360/head//1 by client.1048726.0:30968 2014-08-12 17:12:07.196367 2014-08-13 17:38:02.556560 7fe9bae887a0 20 read_log 194215'1340 (194215'923) delete 3ac7842b/benchmark_data_storage0111-ib.pocstor_19374_object4440/head//1 by client.1048726.0:31048 2014-08-12 17:12:07.239846 2014-08-13 17:38:02.556582 7fe9bae887a0 20 read_log 194215'1341 (194215'924) delete 3be00eeb/benchmark_data_storage0111-ib.pocstor_19374_object4498/head//1 by client.1048726.0:31106 2014-08-12 17:12:07.285475 2014-08-13 17:38:02.556604 7fe9bae887a0 20 read_log 194215'1342 (194215'925) delete 20baeeab/benchmark_data_storage0111-ib.pocstor_19374_object4674/head//1 by client.1048726.0:31282 2014-08-12 17:12:07.362385 2014-08-13 17:38:02.556626 7fe9bae887a0 20 read_log 194215'1343 (194215'926) delete ae2e9d2b/benchmark_data_storage0111-ib.pocstor_19374_object4826/head//1 by client.1048726.0:31434 2014-08-12 17:12:07.425514 2014-08-13 17:38:02.556648 7fe9bae887a0 20 read_log 194215'1344 (194215'927) delete e1d72b2b/benchmark_data_storage0111-ib.pocstor_19374_object4840/head//1 by client.1048726.0:31448 2014-08-12 17:12:07.431609 2014-08-13 17:38:02.556669 7fe9bae887a0 20 read_log 194215'1345 (194215'928) delete 14b2daeb/benchmark_data_storage0111-ib.pocstor_19374_object4851/head//1 by client.1048726.0:31459 2014-08-12 17:12:07.433564 2014-08-13 17:38:02.556691 7fe9bae887a0 20 read_log 194215'1346 (194215'929) delete f058a86b/benchmark_data_storage0111-ib.pocstor_19374_object4891/head//1 by client.1048726.0:31499 2014-08-12 17:12:07.450015 2014-08-13 17:38:02.556713 7fe9bae887a0 20 read_log 194215'1347 (194215'930) delete 1cc0022b/benchmark_data_storage0111-ib.pocstor_19374_object4938/head//1 by client.1048726.0:31546 2014-08-12 17:12:07.470916 2014-08-13 17:38:02.556735 7fe9bae887a0 20 read_log 194215'1348 (194215'931) delete add0cc2b/benchmark_data_storage0111-ib.pocstor_19374_object5085/head//1 by client.1048726.0:31693 2014-08-12 17:12:07.520933 2014-08-13 17:38:02.556757 7fe9bae887a0 20 read_log 194215'1349 (194215'932) delete 942a0e2b/benchmark_data_storage0111-ib.pocstor_19374_object5225/head//1 by client.1048726.0:31833 2014-08-12 17:12:07.565376 2014-08-13 17:38:02.556780 7fe9bae887a0 20 read_log 194215'1350 (194215'933) delete b41983eb/benchmark_data_storage0111-ib.pocstor_19374_object5345/head//1 by client.1048726.0:31953 2014-08-12 17:12:07.603937 2014-08-13 17:38:02.556802 7fe9bae887a0 20 read_log 194215'1351 (194215'934) delete 6f06a4eb/benchmark_data_storage0111-ib.pocstor_19374_object5356/head//1 by client.1048726.0:31964 2014-08-12 17:12:07.608232 2014-08-13 17:38:02.556859 7fe9bae887a0 20 read_log 194215'1352 (194215'935) delete a176b/benchmark_data_storage0111-ib.pocstor_19374_object5547/head//1 by client.1048726.0:32155 2014-08-12 17:12:07.667029 2014-08-13 17:38:02.556890 7fe9bae887a0 20 read_log 194215'1353 (194215'936) delete 2bbd78ab/benchmark_data_storage0111-ib.pocstor_19374_object5562/head//1 by client.1048726.0:32170 2014-08-12 17:12:07.671544 2014-08-13 17:38:02.556912 7fe9bae887a0 20 read_log 194215'1354 (194215'937) delete 54ab51eb/benchmark_data_storage0111-ib.pocstor_19374_object5571/head//1 by client.1048726.0:32179 2014-08-12 17:12:07.676824 2014-08-13 17:38:02.556966 7fe9bae887a0 20 read_log 194215'1355 (194215'938) delete 53bf81eb/benchmark_data_storage0111-ib.pocstor_19374_object5590/head//1 by client.1048726.0:32198 2014-08-12 17:12:07.682597 2014-08-13 17:38:02.556993 7fe9bae887a0 20 read_log 194215'1356 (194215'939) delete 93b6132b/benchmark_data_storage0111-ib.pocstor_19374_object5651/head//1 by client.1048726.0:32259 2014-08-12 17:12:07.701780 2014-08-13 17:38:02.557015 7fe9bae887a0 20 read_log 194215'1357 (194215'940) delete 2f4423eb/benchmark_data_storage0111-ib.pocstor_19374_object5849/head//1 by client.1048726.0:32457 2014-08-12 17:12:07.764354 2014-08-13 17:38:02.557037 7fe9bae887a0 20 read_log 194215'1358 (194215'941) delete 37100d2b/benchmark_data_storage0111-ib.pocstor_19374_object5900/head//1 by client.1048726.0:32508 2014-08-12 17:12:07.779429 2014-08-13 17:38:02.557071 7fe9bae887a0 20 read_log 194215'1359 (194215'942) delete 2dd007ab/benchmark_data_storage0111-ib.pocstor_19374_object5917/head//1 by client.1048726.0:32525 2014-08-12 17:12:07.785188 2014-08-13 17:38:02.557098 7fe9bae887a0 20 read_log 194215'1360 (194215'943) delete 86a4fcab/benchmark_data_storage0111-ib.pocstor_19374_object5945/head//1 by client.1048726.0:32553 2014-08-12 17:12:07.792698 2014-08-13 17:38:02.557121 7fe9bae887a0 20 read_log 194215'1361 (194215'944) delete 920492b/benchmark_data_storage0111-ib.pocstor_19374_object5965/head//1 by client.1048726.0:32573 2014-08-12 17:12:07.797787 2014-08-13 17:38:02.557143 7fe9bae887a0 20 read_log 194215'1362 (194215'945) delete 1055bb2b/benchmark_data_storage0111-ib.pocstor_19374_object5969/head//1 by client.1048726.0:32577 2014-08-12 17:12:07.799086 2014-08-13 17:38:02.557173 7fe9bae887a0 20 read_log 194215'1363 (194215'946) delete 58e49beb/benchmark_data_storage0111-ib.pocstor_19374_object6067/head//1 by client.1048726.0:32675 2014-08-12 17:12:07.828662 2014-08-13 17:38:02.557196 7fe9bae887a0 20 read_log 194215'1364 (194215'947) delete f187cf6b/benchmark_data_storage0111-ib.pocstor_19374_object6085/head//1 by client.1048726.0:32693 2014-08-12 17:12:07.833464 2014-08-13 17:38:02.557218 7fe9bae887a0 20 read_log 194215'1365 (194215'948) delete 700ddfab/benchmark_data_storage0111-ib.pocstor_19374_object6199/head//1 by client.1048726.0:32807 2014-08-12 17:12:07.868800 2014-08-13 17:38:02.557240 7fe9bae887a0 20 read_log 194215'1366 (194215'949) delete fd0e98ab/benchmark_data_storage0111-ib.pocstor_19374_object6210/head//1 by client.1048726.0:32818 2014-08-12 17:12:07.871794 2014-08-13 17:38:02.557262 7fe9bae887a0 20 read_log 194215'1367 (194215'950) delete b470446b/benchmark_data_storage0111-ib.pocstor_19374_object6353/head//1 by client.1048726.0:32961 2014-08-12 17:12:07.917575 2014-08-13 17:38:02.557284 7fe9bae887a0 20 read_log 194215'1368 (194215'951) delete e85327ab/benchmark_data_storage0111-ib.pocstor_19374_object6362/head//1 by client.1048726.0:32970 2014-08-12 17:12:07.920429 2014-08-13 17:38:02.557306 7fe9bae887a0 20 read_log 194215'1369 (194215'952) delete e40363ab/benchmark_data_storage0111-ib.pocstor_19374_object6440/head//1 by client.1048726.0:33048 2014-08-12 17:12:07.946771 2014-08-13 17:38:02.557328 7fe9bae887a0 20 read_log 194215'1370 (194215'953) delete 2dd5496b/benchmark_data_storage0111-ib.pocstor_19374_object6446/head//1 by client.1048726.0:33054 2014-08-12 17:12:07.949162 2014-08-13 17:38:02.557350 7fe9bae887a0 20 read_log 194215'1371 (194215'954) delete 75bccceb/benchmark_data_storage0111-ib.pocstor_19374_object6497/head//1 by client.1048726.0:33105 2014-08-12 17:12:07.967320 2014-08-13 17:38:02.557372 7fe9bae887a0 20 read_log 194215'1372 (194215'955) delete df89832b/benchmark_data_storage0111-ib.pocstor_19374_object6513/head//1 by client.1048726.0:33121 2014-08-12 17:12:07.971893 2014-08-13 17:38:02.557396 7fe9bae887a0 20 read_log 194215'1373 (194215'956) delete f02cbe6b/benchmark_data_storage0111-ib.pocstor_19374_object6524/head//1 by client.1048726.0:33132 2014-08-12 17:12:07.975849 2014-08-13 17:38:02.557418 7fe9bae887a0 20 read_log 194215'1374 (194215'957) delete a3faf6b/benchmark_data_storage0111-ib.pocstor_19374_object6613/head//1 by client.1048726.0:33221 2014-08-12 17:12:08.003763 2014-08-13 17:38:02.557440 7fe9bae887a0 20 read_log 194215'1375 (194215'958) delete ff3983eb/benchmark_data_storage0111-ib.pocstor_19374_object6657/head//1 by client.1048726.0:33265 2014-08-12 17:12:08.016655 2014-08-13 17:38:02.557462 7fe9bae887a0 20 read_log 194215'1376 (194215'959) delete 1f301fab/benchmark_data_storage0111-ib.pocstor_19374_object6816/head//1 by client.1048726.0:33424 2014-08-12 17:12:08.064019 2014-08-13 17:38:02.557484 7fe9bae887a0 20 read_log 194215'1377 (194215'960) delete f6de92eb/benchmark_data_storage0111-ib.pocstor_19374_object6841/head//1 by client.1048726.0:33449 2014-08-12 17:12:08.073605 2014-08-13 17:38:02.557506 7fe9bae887a0 20 read_log 194215'1378 (194215'961) delete b6c1042b/benchmark_data_storage0111-ib.pocstor_19374_object6885/head//1 by client.1048726.0:33493 2014-08-12 17:12:08.084838 2014-08-13 17:38:02.557543 7fe9bae887a0 20 read_log 194215'1379 (194215'962) delete af36f9eb/benchmark_data_storage0111-ib.pocstor_19374_object6904/head//1 by client.1048726.0:33512 2014-08-12 17:12:08.090667 2014-08-13 17:38:02.557566 7fe9bae887a0 20 read_log 194215'1380 (194215'963) delete a5777ab/benchmark_data_storage0111-ib.pocstor_19374_object7029/head//1 by client.1048726.0:33637 2014-08-12 17:12:08.127986 2014-08-13 17:38:02.557588 7fe9bae887a0 20 read_log 194215'1381 (194215'964) delete 6919b6b/benchmark_data_storage0111-ib.pocstor_19374_object7059/head//1 by client.1048726.0:33667 2014-08-12 17:12:08.137666 2014-08-13 17:38:02.557610 7fe9bae887a0 20 read_log 194215'1382 (194215'965) delete efbee52b/benchmark_data_storage0111-ib.pocstor_19374_object7069/head//1 by client.1048726.0:33677 2014-08-12 17:12:08.140866 2014-08-13 17:38:02.557632 7fe9bae887a0 20 read_log 194215'1383 (194215'966) delete 5f8c13eb/benchmark_data_storage0111-ib.pocstor_19374_object7102/head//1 by client.1048726.0:33710 2014-08-12 17:12:08.151016 2014-08-13 17:38:02.557654 7fe9bae887a0 20 read_log 194215'1384 (194215'967) delete 651b9fab/benchmark_data_storage0111-ib.pocstor_19374_object7248/head//1 by client.1048726.0:33856 2014-08-12 17:12:08.212719 2014-08-13 17:38:02.557675 7fe9bae887a0 20 read_log 194215'1385 (194215'968) delete 4698106b/benchmark_data_storage0111-ib.pocstor_19374_object7302/head//1 by client.1048726.0:33910 2014-08-12 17:12:08.228384 2014-08-13 17:38:02.557700 7fe9bae887a0 20 read_log 194215'1386 (194215'969) delete 95b2bd2b/benchmark_data_storage0111-ib.pocstor_19374_object7355/head//1 by client.1048726.0:33963 2014-08-12 17:12:08.245593 2014-08-13 17:38:02.557722 7fe9bae887a0 20 read_log 194215'1387 (194215'970) delete 3565086b/benchmark_data_storage0111-ib.pocstor_19374_object7394/head//1 by client.1048726.0:34002 2014-08-12 17:12:08.256276 2014-08-13 17:38:02.557744 7fe9bae887a0 20 read_log 194215'1388 (194215'971) delete 7723e2eb/benchmark_data_storage0111-ib.pocstor_19374_object7432/head//1 by client.1048726.0:34040 2014-08-12 17:12:08.266736 2014-08-13 17:38:02.557766 7fe9bae887a0 20 read_log 194215'1389 (194215'972) delete 3876f66b/benchmark_data_storage0111-ib.pocstor_19374_object7456/head//1 by client.1048726.0:34064 2014-08-12 17:12:08.276472 2014-08-13 17:38:02.557788 7fe9bae887a0 20 read_log 194215'1390 (194215'973) delete 7c404ceb/benchmark_data_storage0111-ib.pocstor_19374_object7471/head//1 by client.1048726.0:34079 2014-08-12 17:12:08.281546 2014-08-13 17:38:02.557812 7fe9bae887a0 20 read_log 194215'1391 (194215'974) delete dee414ab/benchmark_data_storage0111-ib.pocstor_19374_object7498/head//1 by client.1048726.0:34106 2014-08-12 17:12:08.288998 2014-08-13 17:38:02.557834 7fe9bae887a0 20 read_log 194215'1392 (194215'975) delete b0b328ab/benchmark_data_storage0111-ib.pocstor_19374_object7503/head//1 by client.1048726.0:34111 2014-08-12 17:12:08.290744 2014-08-13 17:38:02.557856 7fe9bae887a0 20 read_log 194215'1393 (194215'976) delete fbaf77ab/benchmark_data_storage0111-ib.pocstor_19374_object7653/head//1 by client.1048726.0:34261 2014-08-12 17:12:08.337138 2014-08-13 17:38:02.557878 7fe9bae887a0 20 read_log 194215'1394 (194215'977) delete f080c82b/benchmark_data_storage0111-ib.pocstor_19374_object7664/head//1 by client.1048726.0:34272 2014-08-12 17:12:08.341162 2014-08-13 17:38:02.557900 7fe9bae887a0 20 read_log 194215'1395 (194215'978) delete cf53d6eb/benchmark_data_storage0111-ib.pocstor_19374_object7819/head//1 by client.1048726.0:34427 2014-08-12 17:12:08.389764 2014-08-13 17:38:02.557921 7fe9bae887a0 20 read_log 194215'1396 (194215'979) delete 7e7fc76b/benchmark_data_storage0111-ib.pocstor_19374_object7822/head//1 by client.1048726.0:34430 2014-08-12 17:12:08.390837 2014-08-13 17:38:02.557943 7fe9bae887a0 20 read_log 194215'1397 (194215'980) delete fa90ac2b/benchmark_data_storage0111-ib.pocstor_19374_object7839/head//1 by client.1048726.0:34447 2014-08-12 17:12:08.395610 2014-08-13 17:38:02.557965 7fe9bae887a0 20 read_log 194215'1398 (194215'981) delete 5918ea2b/benchmark_data_storage0111-ib.pocstor_19374_object7899/head//1 by client.1048726.0:34507 2014-08-12 17:12:08.415096 2014-08-13 17:38:02.558001 7fe9bae887a0 20 read_log 194215'1399 (194215'982) delete 4b04ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8097/head//1 by client.1048726.0:34705 2014-08-12 17:12:08.477220 2014-08-13 17:38:02.558024 7fe9bae887a0 20 read_log 194215'1400 (194215'983) delete 3b37c4ab/benchmark_data_storage0111-ib.pocstor_19374_object8171/head//1 by client.1048726.0:34779 2014-08-12 17:12:08.501921 2014-08-13 17:38:02.558050 7fe9bae887a0 20 read_log 194215'1401 (194215'984) delete fabb5dab/benchmark_data_storage0111-ib.pocstor_19374_object8252/head//1 by client.1048726.0:34860 2014-08-12 17:12:08.526699 2014-08-13 17:38:02.558073 7fe9bae887a0 20 read_log 194215'1402 (194215'985) delete be4caceb/benchmark_data_storage0111-ib.pocstor_19374_object8295/head//1 by client.1048726.0:34903 2014-08-12 17:12:08.539493 2014-08-13 17:38:02.558094 7fe9bae887a0 20 read_log 194215'1403 (194215'986) delete 95c4ed2b/benchmark_data_storage0111-ib.pocstor_19374_object8373/head//1 by client.1048726.0:34981 2014-08-12 17:12:08.566000 2014-08-13 17:38:02.558116 7fe9bae887a0 20 read_log 194215'1404 (194215'987) delete bfddf56b/benchmark_data_storage0111-ib.pocstor_19374_object8440/head//1 by client.1048726.0:35048 2014-08-12 17:12:08.585129 2014-08-13 17:38:02.558138 7fe9bae887a0 20 read_log 194215'1405 (194215'988) delete 63bb63eb/benchmark_data_storage0111-ib.pocstor_19374_object8570/head//1 by client.1048726.0:35178 2014-08-12 17:12:08.623071 2014-08-13 17:38:02.558166 7fe9bae887a0 20 read_log 194215'1406 (194215'989) delete 819622eb/benchmark_data_storage0111-ib.pocstor_19374_object8723/head//1 by client.1048726.0:35331 2014-08-12 17:12:08.677012 2014-08-13 17:38:02.558189 7fe9bae887a0 20 read_log 194215'1407 (194215'990) delete 872863eb/benchmark_data_storage0111-ib.pocstor_19374_object8755/head//1 by client.1048726.0:35363 2014-08-12 17:12:08.686847 2014-08-13 17:38:02.558211 7fe9bae887a0 20 read_log 194215'1408 (194215'991) delete ba99d16b/benchmark_data_storage0111-ib.pocstor_19374_object8788/head//1 by client.1048726.0:35396 2014-08-12 17:12:08.696027 2014-08-13 17:38:02.558232 7fe9bae887a0 20 read_log 194215'1409 (194215'992) delete 89e660ab/benchmark_data_storage0111-ib.pocstor_19374_object8796/head//1 by client.1048726.0:35404 2014-08-12 17:12:08.698316 2014-08-13 17:38:02.558254 7fe9bae887a0 20 read_log 194215'1410 (194215'993) delete 3d2f22b/benchmark_data_storage0111-ib.pocstor_19374_object8882/head//1 by client.1048726.0:35490 2014-08-12 17:12:08.723702 2014-08-13 17:38:02.558276 7fe9bae887a0 20 read_log 194215'1411 (194215'994) delete 5366ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8986/head//1 by client.1048726.0:35594 2014-08-12 17:12:08.757162 2014-08-13 17:38:02.558298 7fe9bae887a0 20 read_log 194215'1412 (194215'995) delete e3a50aab/benchmark_data_storage0111-ib.pocstor_19374_object9008/head//1 by client.1048726.0:35616 2014-08-12 17:12:08.764539 2014-08-13 17:38:02.558322 7fe9bae887a0 20 read_log 194215'1413 (194215'996) delete 96aab4eb/benchmark_data_storage0111-ib.pocstor_19374_object9145/head//1 by client.1048726.0:35753 2014-08-12 17:12:08.805106 2014-08-13 17:38:02.558349 7fe9bae887a0 20 read_log 194215'1414 (194215'997) delete 14f1502b/benchmark_data_storage0111-ib.pocstor_19374_object9193/head//1 by client.1048726.0:35801 2014-08-12 17:12:08.820123 2014-08-13 17:38:02.558373 7fe9bae887a0 20 read_log 194215'1415 (194215'998) delete 6a91646b/benchmark_data_storage0111-ib.pocstor_19374_object9229/head//1 by client.1048726.0:35837 2014-08-12 17:12:08.832538 2014-08-13 17:38:02.558395 7fe9bae887a0 20 read_log 194215'1416 (194215'999) delete c93002eb/benchmark_data_storage0111-ib.pocstor_19374_object9287/head//1 by client.1048726.0:35895 2014-08-12 17:12:08.849201 2014-08-13 17:38:02.558417 7fe9bae887a0 20 read_log 194215'1417 (194215'1000) delete 2257b96b/benchmark_data_storage0111-ib.pocstor_19374_object9441/head//1 by client.1048726.0:36049 2014-08-12 17:12:08.899088 2014-08-13 17:38:02.558439 7fe9bae887a0 20 read_log 194215'1418 (194215'1001) delete 33077deb/benchmark_data_storage0111-ib.pocstor_19374_object9442/head//1 by client.1048726.0:36050 2014-08-12 17:12:08.899792 2014-08-13 17:38:02.558474 7fe9bae887a0 20 read_log 194215'1419 (194215'1002) delete e2ce132b/benchmark_data_storage0111-ib.pocstor_19374_object9679/head//1 by client.1048726.0:36287 2014-08-12 17:12:08.975858 2014-08-13 17:38:02.558500 7fe9bae887a0 20 read_log 194215'1420 (194215'1003) delete 9ce657ab/benchmark_data_storage0111-ib.pocstor_19374_object9686/head//1 by client.1048726.0:36294 2014-08-12 17:12:08.979043 2014-08-13 17:38:02.558523 7fe9bae887a0 20 read_log 194215'1421 (194215'1004) delete a535d1ab/benchmark_data_storage0111-ib.pocstor_19374_object9692/head//1 by client.1048726.0:36300 2014-08-12 17:12:08.979946 2014-08-13 17:38:02.558544 7fe9bae887a0 20 read_log 194215'1422 (194215'1005) delete 3ad652eb/benchmark_data_storage0111-ib.pocstor_19374_object9703/head//1 by client.1048726.0:36311 2014-08-12 17:12:08.982873 2014-08-13 17:38:02.558603 7fe9bae887a0 20 read_log 194215'1423 (194215'1006) delete 2daa772b/benchmark_data_storage0111-ib.pocstor_19374_object9929/head//1 by client.1048726.0:36537 2014-08-12 17:12:09.052370 2014-08-13 17:38:02.558626 7fe9bae887a0 20 read_log 194215'1424 (194215'1007) delete 1659f86b/benchmark_data_storage0111-ib.pocstor_19374_object9992/head//1 by client.1048726.0:36600 2014-08-12 17:12:09.073368 2014-08-13 17:38:02.558649 7fe9bae887a0 20 read_log 194215'1425 (194215'1008) delete 50cd18ab/benchmark_data_storage0111-ib.pocstor_19374_object10038/head//1 by client.1048726.0:36646 2014-08-12 17:12:09.094311 2014-08-13 17:38:02.558671 7fe9bae887a0 20 read_log 194215'1426 (194215'1009) delete 57ad5fab/benchmark_data_storage0111-ib.pocstor_19374_object10124/head//1 by client.1048726.0:36732 2014-08-12 17:12:09.119654 2014-08-13 17:38:02.558693 7fe9bae887a0 20 read_log 194215'1427 (194215'1010) delete 20113dab/benchmark_data_storage0111-ib.pocstor_19374_object10313/head//1 by client.1048726.0:36921 2014-08-12 17:12:09.183933 2014-08-13 17:38:02.558715 7fe9bae887a0 20 read_log 194215'1428 (194215'1011) delete fc13012b/benchmark_data_storage0111-ib.pocstor_19374_object10319/head//1 by client.1048726.0:36927 2014-08-12 17:12:09.184336 2014-08-13 17:38:02.558740 7fe9bae887a0 20 read_log 194215'1429 (194215'1012) delete 55f420ab/benchmark_data_storage0111-ib.pocstor_19374_object10414/head//1 by client.1048726.0:37022 2014-08-12 17:12:09.220648 2014-08-13 17:38:02.558762 7fe9bae887a0 20 read_log 194215'1430 (194215'1013) delete 44b26d2b/benchmark_data_storage0111-ib.pocstor_19374_object10437/head//1 by client.1048726.0:37045 2014-08-12 17:12:09.229211 2014-08-13 17:38:02.558784 7fe9bae887a0 20 read_log 194215'1431 (194215'1014) delete 6c33baab/benchmark_data_storage0111-ib.pocstor_19374_object10493/head//1 by client.1048726.0:37101 2014-08-12 17:12:09.245366 2014-08-13 17:38:02.558806 7fe9bae887a0 20 read_log 194215'1432 (194215'1015) delete 56652dab/benchmark_data_storage0111-ib.pocstor_19374_object10603/head//1 by client.1048726.0:37211 2014-08-12 17:12:09.284058 2014-08-13 17:38:02.558827 7fe9bae887a0 20 read_log 194215'1433 (194215'1016) delete 7aabe3eb/benchmark_data_storage0111-ib.pocstor_19374_object10674/head//1 by client.1048726.0:37282 2014-08-12 17:12:09.308402 2014-08-13 17:38:02.558849 7fe9bae887a0 20 read_log 194215'1434 (194215'1017) delete bdbfb66b/benchmark_data_storage0111-ib.pocstor_19374_object10688/head//1 by client.1048726.0:37296 2014-08-12 17:12:09.314929 2014-08-13 17:38:02.558871 7fe9bae887a0 20 read_log 194215'1435 (194215'1018) delete 9f6f4eeb/benchmark_data_storage0111-ib.pocstor_19374_object10736/head//1 by client.1048726.0:37344 2014-08-12 17:12:09.332252 2014-08-13 17:38:02.558893 7fe9bae887a0 20 read_log 194215'1436 (194215'1019) delete 55cd0b2b/benchmark_data_storage0111-ib.pocstor_19374_object10796/head//1 by client.1048726.0:37404 2014-08-12 17:12:09.353540 2014-08-13 17:38:02.558915 7fe9bae887a0 20 read_log 194215'1437 (194215'1020) delete 190a722b/benchmark_data_storage0111-ib.pocstor_19374_object10846/head//1 by client.1048726.0:37454 2014-08-12 17:12:09.368913 2014-08-13 17:38:02.558937 7fe9bae887a0 20 read_log 194215'1438 (194215'1021) delete 960912b/benchmark_data_storage0111-ib.pocstor_19374_object10955/head//1 by client.1048726.0:37563 2014-08-12 17:12:09.410246 2014-08-13 17:38:02.559001 7fe9bae887a0 20 read_log 194215'1439 (194215'1022) delete fcdecfeb/benchmark_data_storage0111-ib.pocstor_19374_object11112/head//1 by client.1048726.0:37720 2014-08-12 17:12:09.465773 2014-08-13 17:38:02.559030 7fe9bae887a0 20 read_log 194215'1440 (194215'1023) delete d5fcf32b/benchmark_data_storage0111-ib.pocstor_19374_object11303/head//1 by client.1048726.0:37911 2014-08-12 17:12:09.523005 2014-08-13 17:38:02.559052 7fe9bae887a0 20 read_log 194215'1441 (194215'1024) delete 75ac9eb/benchmark_data_storage0111-ib.pocstor_19374_object11344/head//1 by client.1048726.0:37952 2014-08-12 17:12:09.535472 2014-08-13 17:38:02.559074 7fe9bae887a0 20 read_log 194215'1442 (194215'1025) delete 3bda6ab/benchmark_data_storage0111-ib.pocstor_19374_object11393/head//1 by client.1048726.0:38001 2014-08-12 17:12:09.550396 2014-08-13 17:38:02.559096 7fe9bae887a0 20 read_log 194215'1443 (194215'1026) delete 940c086b/benchmark_data_storage0111-ib.pocstor_19374_object11474/head//1 by client.1048726.0:38082 2014-08-12 17:12:09.575617 2014-08-13 17:38:02.559118 7fe9bae887a0 20 read_log 194215'1444 (194215'1027) delete 316ace6b/benchmark_data_storage0111-ib.pocstor_19374_object11516/head//1 by client.1048726.0:38124 2014-08-12 17:12:09.587722 2014-08-13 17:38:02.559139 7fe9bae887a0 20 read_log 194215'1445 (194215'1028) delete dd29fdeb/benchmark_data_storage0111-ib.pocstor_19374_object11523/head//1 by client.1048726.0:38131 2014-08-12 17:12:09.590937 2014-08-13 17:38:02.559169 7fe9bae887a0 20 read_log 194215'1446 (194215'1029) delete f5bded2b/benchmark_data_storage0111-ib.pocstor_19374_object11543/head//1 by client.1048726.0:38151 2014-08-12 17:12:09.597176 2014-08-13 17:38:02.559192 7fe9bae887a0 20 read_log 194215'1447 (194215'1030) delete c4b3dc6b/benchmark_data_storage0111-ib.pocstor_19374_object11655/head//1 by client.1048726.0:38263 2014-08-12 17:12:09.637386 2014-08-13 17:38:02.559247 7fe9bae887a0 20 read_log 194215'1448 (194215'1031) delete 2095576b/benchmark_data_storage0111-ib.pocstor_19374_object11672/head//1 by client.1048726.0:38280 2014-08-12 17:12:09.645359 2014-08-13 17:38:02.559274 7fe9bae887a0 20 read_log 194215'1449 (194215'1032) delete a6115a2b/benchmark_data_storage0111-ib.pocstor_19374_object11714/head//1 by client.1048726.0:38322 2014-08-12 17:12:09.660996 2014-08-13 17:38:02.559296 7fe9bae887a0 20 read_log 194215'1450 (194215'1033) delete 8aafa9ab/benchmark_data_storage0111-ib.pocstor_19374_object11744/head//1 by client.1048726.0:38352 2014-08-12 17:12:09.670450 2014-08-13 17:38:02.559318 7fe9bae887a0 20 read_log 194215'1451 (194215'1034) delete d9131ceb/benchmark_data_storage0111-ib.pocstor_19374_object11815/head//1 by client.1048726.0:38423 2014-08-12 17:12:09.694564 2014-08-13 17:38:02.559340 7fe9bae887a0 20 read_log 194215'1452 (194215'1035) delete 7091a6b/benchmark_data_storage0111-ib.pocstor_19374_object12125/head//1 by client.1048726.0:38733 2014-08-12 17:12:09.804142 2014-08-13 17:38:02.559366 7fe9bae887a0 20 read_log 194215'1453 (194215'1036) delete 96552bab/benchmark_data_storage0111-ib.pocstor_19374_object12215/head//1 by client.1048726.0:38823 2014-08-12 17:12:09.839432 2014-08-13 17:38:02.559388 7fe9bae887a0 20 read_log 194215'1454 (194215'1037) delete 84c0f3eb/benchmark_data_storage0111-ib.pocstor_19374_object12617/head//1 by client.1048726.0:39225 2014-08-12 17:12:09.982090 2014-08-13 17:38:02.559410 7fe9bae887a0 20 read_log 194215'1455 (194215'1038) delete 6b222c6b/benchmark_data_storage0111-ib.pocstor_19374_object12626/head//1 by client.1048726.0:39234 2014-08-12 17:12:09.986105 2014-08-13 17:38:02.559432 7fe9bae887a0 20 read_log 194215'1456 (194215'1039) delete 304a35eb/benchmark_data_storage0111-ib.pocstor_19374_object12721/head//1 by client.1048726.0:39329 2014-08-12 17:12:10.013472 2014-08-13 17:38:02.559454 7fe9bae887a0 20 read_log 194215'1457 (194215'1040) delete ce855f2b/benchmark_data_storage0111-ib.pocstor_19374_object12758/head//1 by client.1048726.0:39366 2014-08-12 17:12:10.024379 2014-08-13 17:38:02.559476 7fe9bae887a0 20 read_log 194215'1458 (194215'1041) delete 91d427ab/benchmark_data_storage0111-ib.pocstor_19374_object12778/head//1 by client.1048726.0:39386 2014-08-12 17:12:10.029642 2014-08-13 17:38:02.559509 7fe9bae887a0 20 read_log 194215'1459 (194215'1042) delete fb4560ab/benchmark_data_storage0111-ib.pocstor_19374_object12863/head//1 by client.1048726.0:39471 2014-08-12 17:12:10.058395 2014-08-13 17:38:02.559537 7fe9bae887a0 20 read_log 194215'1460 (194215'1043) delete 883592b/benchmark_data_storage0111-ib.pocstor_19374_object12865/head//1 by client.1048726.0:39473 2014-08-12 17:12:10.060577 2014-08-13 17:38:02.559559 7fe9bae887a0 20 read_log 194215'1461 (194215'1044) delete 64527e2b/benchmark_data_storage0111-ib.pocstor_19374_object12908/head//1 by client.1048726.0:39516 2014-08-12 17:12:10.074158 2014-08-13 17:38:02.559582 7fe9bae887a0 20 read_log 194215'1462 (194215'1045) delete bb45c1eb/benchmark_data_storage0111-ib.pocstor_19374_object12937/head//1 by client.1048726.0:39545 2014-08-12 17:12:10.084668 2014-08-13 17:38:02.559603 7fe9bae887a0 20 read_log 194215'1463 (194215'1046) delete 4f81c16b/benchmark_data_storage0111-ib.pocstor_19374_object12968/head//1 by client.1048726.0:39576 2014-08-12 17:12:10.096728 2014-08-13 17:38:02.559629 7fe9bae887a0 20 read_log 194215'1464 (194215'1047) delete ca6ba7eb/benchmark_data_storage0111-ib.pocstor_19374_object12985/head//1 by client.1048726.0:39593 2014-08-12 17:12:10.103326 2014-08-13 17:38:02.559652 7fe9bae887a0 20 read_log 194215'1465 (194215'1048) delete d6caf0eb/benchmark_data_storage0111-ib.pocstor_19374_object13037/head//1 by client.1048726.0:39645 2014-08-12 17:12:10.118536 2014-08-13 17:38:02.559673 7fe9bae887a0 20 read_log 194215'1466 (194215'1049) delete 995517eb/benchmark_data_storage0111-ib.pocstor_19374_object13098/head//1 by client.1048726.0:39706 2014-08-12 17:12:10.137401 2014-08-13 17:38:02.559701 7fe9bae887a0 20 read_log 194215'1467 (194215'1050) delete 69b7392b/benchmark_data_storage0111-ib.pocstor_19374_object13278/head//1 by client.1048726.0:39886 2014-08-12 17:12:10.198794 2014-08-13 17:38:02.559724 7fe9bae887a0 20 read_log 194215'1468 (194215'1051) delete 5d3e5d6b/benchmark_data_storage0111-ib.pocstor_19374_object13368/head//1 by client.1048726.0:39976 2014-08-12 17:12:10.226623 2014-08-13 17:38:02.559745 7fe9bae887a0 20 read_log 194215'1469 (194215'1052) delete b7bdedeb/benchmark_data_storage0111-ib.pocstor_19374_object13400/head//1 by client.1048726.0:40008 2014-08-12 17:12:10.239865 2014-08-13 17:38:02.559767 7fe9bae887a0 20 read_log 194215'1470 (194215'1053) delete 6dc64eeb/benchmark_data_storage0111-ib.pocstor_19374_object13453/head//1 by client.1048726.0:40061 2014-08-12 17:12:10.260396 2014-08-13 17:38:02.559789 7fe9bae887a0 20 read_log 194215'1471 (194215'1054) delete 8c0f1d2b/benchmark_data_storage0111-ib.pocstor_19374_object13462/head//1 by client.1048726.0:40070 2014-08-12 17:12:10.263464 2014-08-13 17:38:02.559810 7fe9bae887a0 20 read_log 194215'1472 (194215'1055) delete 37a0adab/benchmark_data_storage0111-ib.pocstor_19374_object13587/head//1 by client.1048726.0:40195 2014-08-12 17:12:10.321025 2014-08-13 17:38:02.559836 7fe9bae887a0 20 read_log 194215'1473 (194215'1056) delete 5a6f74eb/benchmark_data_storage0111-ib.pocstor_19374_object13600/head//1 by client.1048726.0:40208 2014-08-12 17:12:10.328063 2014-08-13 17:38:02.559859 7fe9bae887a0 20 read_log 194215'1474 (194215'1057) delete d3458feb/benchmark_data_storage0111-ib.pocstor_19374_object13641/head//1 by client.1048726.0:40249 2014-08-12 17:12:10.347720 2014-08-13 17:38:02.559880 7fe9bae887a0 20 read_log 194215'1475 (194215'1058) delete b1e814eb/benchmark_data_storage0111-ib.pocstor_19374_object13681/head//1 by client.1048726.0:40289 2014-08-12 17:12:10.366181 2014-08-13 17:38:02.559902 7fe9bae887a0 20 read_log 194215'1476 (194215'1059) delete 97b4952b/benchmark_data_storage0111-ib.pocstor_19374_object13697/head//1 by client.1048726.0:40305 2014-08-12 17:12:10.373564 2014-08-13 17:38:02.559928 7fe9bae887a0 20 read_log 194215'1477 (194215'1060) delete b156fbab/benchmark_data_storage0111-ib.pocstor_19374_object13795/head//1 by client.1048726.0:40403 2014-08-12 17:12:10.413287 2014-08-13 17:38:02.559955 7fe9bae887a0 20 read_log 194215'1478 (194215'1061) delete b9abdaeb/benchmark_data_storage0111-ib.pocstor_19374_object13915/head//1 by client.1048726.0:40523 2014-08-12 17:12:10.462657 2014-08-13 17:38:02.559990 7fe9bae887a0 20 read_log 194215'1479 (194215'1062) delete 15bb4aeb/benchmark_data_storage0111-ib.pocstor_19374_object14018/head//1 by client.1048726.0:40626 2014-08-12 17:12:10.496295 2014-08-13 17:38:02.560016 7fe9bae887a0 20 read_log 194215'1480 (194215'1063) delete 86d77e6b/benchmark_data_storage0111-ib.pocstor_19374_object14045/head//1 by client.1048726.0:40653 2014-08-12 17:12:10.503974 2014-08-13 17:38:02.560038 7fe9bae887a0 20 read_log 194215'1481 (194215'1064) delete f762602b/benchmark_data_storage0111-ib.pocstor_19374_object14072/head//1 by client.1048726.0:40680 2014-08-12 17:12:10.512713 2014-08-13 17:38:02.560059 7fe9bae887a0 20 read_log 194215'1482 (194215'1065) delete 656a9d6b/benchmark_data_storage0111-ib.pocstor_19374_object14108/head//1 by client.1048726.0:40716 2014-08-12 17:12:10.527145 2014-08-13 17:38:02.560081 7fe9bae887a0 20 read_log 194215'1483 (194215'1066) delete 12017d6b/benchmark_data_storage0111-ib.pocstor_19374_object14130/head//1 by client.1048726.0:40738 2014-08-12 17:12:10.534227 2014-08-13 17:38:02.560102 7fe9bae887a0 20 read_log 194215'1484 (194215'1067) delete 6837d1eb/benchmark_data_storage0111-ib.pocstor_19374_object14218/head//1 by client.1048726.0:40826 2014-08-12 17:12:10.566668 2014-08-13 17:38:02.560124 7fe9bae887a0 20 read_log 194215'1485 (194215'1068) delete e0890e6b/benchmark_data_storage0111-ib.pocstor_19374_object14272/head//1 by client.1048726.0:40880 2014-08-12 17:12:10.584392 2014-08-13 17:38:02.560145 7fe9bae887a0 20 read_log 194215'1486 (194215'1069) delete 51a9f76b/benchmark_data_storage0111-ib.pocstor_19374_object14273/head//1 by client.1048726.0:40881 2014-08-12 17:12:10.584524 2014-08-13 17:38:02.560181 7fe9bae887a0 20 read_log 194215'1487 (194215'1070) delete ff105aeb/benchmark_data_storage0111-ib.pocstor_19374_object14288/head//1 by client.1048726.0:40896 2014-08-12 17:12:10.590143 2014-08-13 17:38:02.560205 7fe9bae887a0 20 read_log 194215'1488 (194215'1071) delete 7c054ceb/benchmark_data_storage0111-ib.pocstor_19374_object14442/head//1 by client.1048726.0:41050 2014-08-12 17:12:10.657515 2014-08-13 17:38:02.560271 7fe9bae887a0 20 read_log 194215'1489 (194215'1072) delete fc1d02eb/benchmark_data_storage0111-ib.pocstor_19374_object14443/head//1 by client.1048726.0:41051 2014-08-12 17:12:10.657688 2014-08-13 17:38:02.560293 7fe9bae887a0 20 read_log 194215'1490 (194215'1073) delete b7463aeb/benchmark_data_storage0111-ib.pocstor_19374_object14446/head//1 by client.1048726.0:41054 2014-08-12 17:12:10.660490 2014-08-13 17:38:02.560315 7fe9bae887a0 20 read_log 194215'1491 (194215'1074) delete 56be36eb/benchmark_data_storage0111-ib.pocstor_19374_object14581/head//1 by client.1048726.0:41189 2014-08-12 17:12:10.731834 2014-08-13 17:38:02.560337 7fe9bae887a0 20 read_log 194215'1492 (194215'1075) delete bc8678eb/benchmark_data_storage0111-ib.pocstor_19374_object14608/head//1 by client.1048726.0:41216 2014-08-12 17:12:10.744467 2014-08-13 17:38:02.560359 7fe9bae887a0 20 read_log 194215'1493 (194215'1076) delete 49787bab/benchmark_data_storage0111-ib.pocstor_19374_object14741/head//1 by client.1048726.0:41349 2014-08-12 17:12:10.808010 2014-08-13 17:38:02.560380 7fe9bae887a0 20 read_log 194215'1494 (194215'1077) delete af3d7a2b/benchmark_data_storage0111-ib.pocstor_19374_object14751/head//1 by client.1048726.0:41359 2014-08-12 17:12:10.811951 2014-08-13 17:38:02.560402 7fe9bae887a0 20 read_log 194215'1495 (194215'1078) delete 3cec11eb/benchmark_data_storage0111-ib.pocstor_19374_object14764/head//1 by client.1048726.0:41372 2014-08-12 17:12:10.816174 2014-08-13 17:38:02.560427 7fe9bae887a0 20 read_log 194215'1496 (194215'1079) delete 97eeeab/benchmark_data_storage0111-ib.pocstor_19374_object14796/head//1 by client.1048726.0:41404 2014-08-12 17:12:10.833680 2014-08-13 17:38:02.560448 7fe9bae887a0 20 read_log 194215'1497 (194215'1080) delete 708ad02b/benchmark_data_storage0111-ib.pocstor_19374_object14818/head//1 by client.1048726.0:41426 2014-08-12 17:12:10.848022 2014-08-13 17:38:02.560470 7fe9bae887a0 20 read_log 194215'1498 (194215'1081) delete ed21ddeb/benchmark_data_storage0111-ib.pocstor_19374_object14866/head//1 by client.1048726.0:41474 2014-08-12 17:12:10.877986 2014-08-13 17:38:02.560504 7fe9bae887a0 20 read_log 194215'1499 (194215'1082) delete 7847fa6b/benchmark_data_storage0111-ib.pocstor_19374_object15056/head//1 by client.1048726.0:41664 2014-08-12 17:12:10.968868 2014-08-13 17:38:02.560528 7fe9bae887a0 20 read_log 194215'1500 (194215'1083) delete 91326eeb/benchmark_data_storage0111-ib.pocstor_19374_object15087/head//1 by client.1048726.0:41695 2014-08-12 17:12:10.979866 2014-08-13 17:38:02.560549 7fe9bae887a0 20 read_log 194215'1501 (194215'1084) delete f966fa2b/benchmark_data_storage0111-ib.pocstor_19374_object15209/head//1 by client.1048726.0:41817 2014-08-12 17:12:11.025827 2014-08-13 17:38:02.560571 7fe9bae887a0 20 read_log 194215'1502 (194215'1085) delete 228e7c6b/benchmark_data_storage0111-ib.pocstor_19374_object15227/head//1 by client.1048726.0:41835 2014-08-12 17:12:11.032740 2014-08-13 17:38:02.560593 7fe9bae887a0 20 read_log 194215'1503 (194215'1086) delete 7acf24ab/benchmark_data_storage0111-ib.pocstor_19374_object15235/head//1 by client.1048726.0:41843 2014-08-12 17:12:11.036396 2014-08-13 17:38:02.560615 7fe9bae887a0 20 read_log 194215'1504 (194215'1087) delete c50ebeeb/benchmark_data_storage0111-ib.pocstor_19374_object15256/head//1 by client.1048726.0:41864 2014-08-12 17:12:11.047265 2014-08-13 17:38:02.560636 7fe9bae887a0 20 read_log 194215'1505 (194215'1088) delete ce6e186b/benchmark_data_storage0111-ib.pocstor_19374_object15291/head//1 by client.1048726.0:41899 2014-08-12 17:12:11.062901 2014-08-13 17:38:02.560658 7fe9bae887a0 20 read_log 194215'1506 (194215'1089) delete 4248daab/benchmark_data_storage0111-ib.pocstor_19374_object15301/head//1 by client.1048726.0:41909 2014-08-12 17:12:11.067919 2014-08-13 17:38:02.560679 7fe9bae887a0 20 read_log 194215'1507 (194215'1090) delete 7db6d4eb/benchmark_data_storage0111-ib.pocstor_19374_object15375/head//1 by client.1048726.0:41983 2014-08-12 17:12:11.105704 2014-08-13 17:38:02.560703 7fe9bae887a0 20 read_log 194215'1508 (194215'1091) delete d27cd12b/benchmark_data_storage0111-ib.pocstor_19374_object15494/head//1 by client.1048726.0:42102 2014-08-12 17:12:11.163798 2014-08-13 17:38:02.560725 7fe9bae887a0 20 read_log 194215'1509 (194215'1092) delete 29efcab/benchmark_data_storage0111-ib.pocstor_19374_object15598/head//1 by client.1048726.0:42206 2014-08-12 17:12:11.216115 2014-08-13 17:38:02.560747 7fe9bae887a0 20 read_log 194215'1510 (194215'1093) delete 725d332b/benchmark_data_storage0111-ib.pocstor_19374_object15691/head//1 by client.1048726.0:42299 2014-08-12 17:12:11.254170 2014-08-13 17:38:02.560769 7fe9bae887a0 20 read_log 194215'1511 (194215'1094) delete 427577eb/benchmark_data_storage0111-ib.pocstor_19374_object15706/head//1 by client.1048726.0:42314 2014-08-12 17:12:11.259763 2014-08-13 17:38:02.560792 7fe9bae887a0 20 read_log 194215'1512 (194215'1095) delete 3774496b/benchmark_data_storage0111-ib.pocstor_19374_object15775/head//1 by client.1048726.0:42383 2014-08-12 17:12:11.282713 2014-08-13 17:38:02.560813 7fe9bae887a0 20 read_log 194215'1513 (194215'1096) delete fb1ddcab/benchmark_data_storage0111-ib.pocstor_19374_object15797/head//1 by client.1048726.0:42405 2014-08-12 17:12:11.290841 2014-08-13 17:38:02.560835 7fe9bae887a0 20 read_log 194215'1514 (194215'1097) delete 72b1916b/benchmark_data_storage0111-ib.pocstor_19374_object15974/head//1 by client.1048726.0:42582 2014-08-12 17:12:11.365685 2014-08-13 17:38:02.560856 7fe9bae887a0 20 read_log 194215'1515 (194215'1098) delete 1afad6eb/benchmark_data_storage0111-ib.pocstor_19374_object16016/head//1 by client.1048726.0:42624 2014-08-12 17:12:11.380790 2014-08-13 17:38:02.560878 7fe9bae887a0 20 read_log 194215'1516 (194215'1099) delete 36c3d4ab/benchmark_data_storage0111-ib.pocstor_19374_object16089/head//1 by client.1048726.0:42697 2014-08-12 17:12:11.410101 2014-08-13 17:38:02.560899 7fe9bae887a0 20 read_log 194215'1517 (194215'1100) delete 41ee41eb/benchmark_data_storage0111-ib.pocstor_19374_object16099/head//1 by client.1048726.0:42707 2014-08-12 17:12:11.415323 2014-08-13 17:38:02.560920 7fe9bae887a0 20 read_log 194215'1518 (194215'1101) delete f52c0f6b/benchmark_data_storage0111-ib.pocstor_19374_object16114/head//1 by client.1048726.0:42722 2014-08-12 17:12:11.423609 2014-08-13 17:38:02.560953 7fe9bae887a0 20 read_log 194215'1519 (194215'1102) delete 343512ab/benchmark_data_storage0111-ib.pocstor_19374_object16151/head//1 by client.1048726.0:42759 2014-08-12 17:12:11.440682 2014-08-13 17:38:02.560977 7fe9bae887a0 20 read_log 194215'1520 (194215'1103) delete 5811406b/benchmark_data_storage0111-ib.pocstor_19374_object16504/head//1 by client.1048726.0:43112 2014-08-12 17:12:11.585163 2014-08-13 17:38:02.560998 7fe9bae887a0 20 read_log 194215'1521 (194215'1104) delete 9896e7ab/benchmark_data_storage0111-ib.pocstor_19374_object16534/head//1 by client.1048726.0:43142 2014-08-12 17:12:11.599824 2014-08-13 17:38:02.561020 7fe9bae887a0 20 read_log 194215'1522 (194215'1105) delete d7af0eab/benchmark_data_storage0111-ib.pocstor_19374_object16646/head//1 by client.1048726.0:43254 2014-08-12 17:12:11.656798 2014-08-13 17:38:02.561041 7fe9bae887a0 20 read_log 194215'1523 (194215'1106) delete 915c5a2b/benchmark_data_storage0111-ib.pocstor_19374_object16648/head//1 by client.1048726.0:43256 2014-08-12 17:12:11.658431 2014-08-13 17:38:02.561062 7fe9bae887a0 20 read_log 194215'1524 (194215'1107) delete a73d206b/benchmark_data_storage0111-ib.pocstor_19374_object16746/head//1 by client.1048726.0:43354 2014-08-12 17:12:11.708415 2014-08-13 17:38:02.561083 7fe9bae887a0 20 read_log 194215'1525 (194215'1108) delete c47c9beb/benchmark_data_storage0111-ib.pocstor_19374_object16760/head//1 by client.1048726.0:43368 2014-08-12 17:12:11.712816 2014-08-13 17:38:02.561105 7fe9bae887a0 20 read_log 194215'1526 (194215'1109) delete dea573ab/benchmark_data_storage0111-ib.pocstor_19374_object16841/head//1 by client.1048726.0:43449 2014-08-12 17:12:11.746116 2014-08-13 17:38:02.561126 7fe9bae887a0 20 read_log 194215'1527 (194215'1110) delete b9e0d1ab/benchmark_data_storage0111-ib.pocstor_19374_object16881/head//1 by client.1048726.0:43489 2014-08-12 17:12:11.762373 2014-08-13 17:38:02.561148 7fe9bae887a0 20 read_log 194215'1528 (194215'1111) delete 3755296b/benchmark_data_storage0111-ib.pocstor_19374_object16916/head//1 by client.1048726.0:43524 2014-08-12 17:12:11.777259 2014-08-13 17:38:02.561178 7fe9bae887a0 20 read_log 194215'1529 (194215'1112) delete c45cbd6b/benchmark_data_storage0111-ib.pocstor_19374_object16925/head//1 by client.1048726.0:43533 2014-08-12 17:12:11.781025 2014-08-13 17:38:02.561201 7fe9bae887a0 20 read_log 194215'1530 (194215'1113) delete d5f0a92b/benchmark_data_storage0111-ib.pocstor_19374_object16947/head//1 by client.1048726.0:43555 2014-08-12 17:12:11.791448 2014-08-13 17:38:02.561222 7fe9bae887a0 20 read_log 194215'1531 (194215'1114) delete 5fceb76b/benchmark_data_storage0111-ib.pocstor_19374_object17225/head//1 by client.1048726.0:43833 2014-08-12 17:12:11.939847 2014-08-13 17:38:02.561244 7fe9bae887a0 20 read_log 194215'1532 (194215'1115) delete bf0f31eb/benchmark_data_storage0111-ib.pocstor_19374_object17240/head//1 by client.1048726.0:43848 2014-08-12 17:12:11.948426 2014-08-13 17:38:02.561265 7fe9bae887a0 20 read_log 194215'1533 (194215'1116) delete ab105e2b/benchmark_data_storage0111-ib.pocstor_19374_object17300/head//1 by client.1048726.0:43908 2014-08-12 17:12:11.983474 2014-08-13 17:38:02.561286 7fe9bae887a0 20 read_log 194215'1534 (194215'1117) delete 8b06b7ab/benchmark_data_storage0111-ib.pocstor_19374_object17532/head//1 by client.1048726.0:44140 2014-08-12 17:12:12.123563 2014-08-13 17:38:02.561307 7fe9bae887a0 20 read_log 194215'1535 (194215'1118) delete ff580c6b/benchmark_data_storage0111-ib.pocstor_19374_object17566/head//1 by client.1048726.0:44174 2014-08-12 17:12:12.145512 2014-08-13 17:38:02.561329 7fe9bae887a0 20 read_log 194215'1536 (194215'1119) delete feff186b/benchmark_data_storage0111-ib.pocstor_19374_object17666/head//1 by client.1048726.0:44274 2014-08-12 17:12:12.221742 2014-08-13 17:38:02.561350 7fe9bae887a0 20 read_log 194215'1537 (194215'1120) delete 92eb096b/benchmark_data_storage0111-ib.pocstor_19374_object17678/head//1 by client.1048726.0:44286 2014-08-12 17:12:12.229387 2014-08-13 17:38:02.561372 7fe9bae887a0 20 read_log 194215'1538 (194215'1121) delete 4795ccab/benchmark_data_storage0111-ib.pocstor_19374_object17689/head//1 by client.1048726.0:44297 2014-08-12 17:12:12.236925 2014-08-13 17:38:02.561405 7fe9bae887a0 20 read_log 194215'1539 (194215'1122) delete 7610516b/benchmark_data_storage0111-ib.pocstor_19374_object17776/head//1 by client.1048726.0:44384 2014-08-12 17:12:12.299358 2014-08-13 17:38:02.561432 7fe9bae887a0 20 read_log 194215'1540 (194215'1123) delete 1941352b/benchmark_data_storage0111-ib.pocstor_19374_object17799/head//1 by client.1048726.0:44407 2014-08-12 17:12:12.308405 2014-08-13 17:38:02.561493 7fe9bae887a0 20 read_log 194215'1541 (194215'1124) delete a663f52b/benchmark_data_storage0111-ib.pocstor_19374_object17824/head//1 by client.1048726.0:44432 2014-08-12 17:12:12.319432 2014-08-13 17:38:02.561516 7fe9bae887a0 20 read_log 194215'1542 (194215'1125) delete 85edc6ab/benchmark_data_storage0111-ib.pocstor_19374_object17839/head//1 by client.1048726.0:44447 2014-08-12 17:12:12.325370 2014-08-13 17:38:02.561537 7fe9bae887a0 20 read_log 194215'1543 (194215'1126) delete 3a62892b/benchmark_data_storage0111-ib.pocstor_19374_object17894/head//1 by client.1048726.0:44502 2014-08-12 17:12:12.349169 2014-08-13 17:38:02.561559 7fe9bae887a0 20 read_log 194215'1544 (194215'1127) delete a9f9016b/benchmark_data_storage0111-ib.pocstor_19374_object17942/head//1 by client.1048726.0:44550 2014-08-12 17:12:12.366679 2014-08-13 17:38:02.561580 7fe9bae887a0 20 read_log 194215'1545 (194215'1128) delete 49c8032b/benchmark_data_storage0111-ib.pocstor_19374_object18015/head//1 by client.1048726.0:44623 2014-08-12 17:12:12.398194 2014-08-13 17:38:02.561602 7fe9bae887a0 20 read_log 194215'1546 (194215'1129) delete 63b44fab/benchmark_data_storage0111-ib.pocstor_19374_object18232/head//1 by client.1048726.0:44840 2014-08-12 17:12:12.494064 2014-08-13 17:38:02.561623 7fe9bae887a0 20 read_log 194215'1547 (194215'1130) delete c05a8beb/benchmark_data_storage0111-ib.pocstor_19374_object18250/head//1 by client.1048726.0:44858 2014-08-12 17:12:12.501151 2014-08-13 17:38:02.561644 7fe9bae887a0 20 read_log 194215'1548 (194215'1131) delete 622415eb/benchmark_data_storage0111-ib.pocstor_19374_object18344/head//1 by client.1048726.0:44952 2014-08-12 17:12:12.533542 2014-08-13 17:38:02.561665 7fe9bae887a0 20 read_log 194215'1549 (194215'1132) delete d404aeb/benchmark_data_storage0111-ib.pocstor_19374_object18373/head//1 by client.1048726.0:44981 2014-08-12 17:12:12.544879 2014-08-13 17:38:02.561687 7fe9bae887a0 20 read_log 194215'1550 (194215'1133) delete 19dc8e2b/benchmark_data_storage0111-ib.pocstor_19374_object18496/head//1 by client.1048726.0:45104 2014-08-12 17:12:12.592582 2014-08-13 17:38:02.561709 7fe9bae887a0 20 read_log 194215'1551 (194215'1134) delete 379abaab/benchmark_data_storage0111-ib.pocstor_19374_object18497/head//1 by client.1048726.0:45105 2014-08-12 17:12:12.592775 2014-08-13 17:38:02.561730 7fe9bae887a0 20 read_log 194215'1552 (194215'1135) delete 64e0802b/benchmark_data_storage0111-ib.pocstor_19374_object18498/head//1 by client.1048726.0:45106 2014-08-12 17:12:12.592891 2014-08-13 17:38:02.561751 7fe9bae887a0 20 read_log 194215'1553 (194215'1136) delete c6e8922b/benchmark_data_storage0111-ib.pocstor_19374_object18505/head//1 by client.1048726.0:45113 2014-08-12 17:12:12.597526 2014-08-13 17:38:02.561776 7fe9bae887a0 20 read_log 194215'1554 (194215'1137) delete 2855efeb/benchmark_data_storage0111-ib.pocstor_19374_object18551/head//1 by client.1048726.0:45159 2014-08-12 17:12:12.616209 2014-08-13 17:38:02.561798 7fe9bae887a0 20 read_log 194215'1555 (194215'1138) delete 646eadab/benchmark_data_storage0111-ib.pocstor_19374_object18577/head//1 by client.1048726.0:45185 2014-08-12 17:12:12.626993 2014-08-13 17:38:02.561821 7fe9bae887a0 20 read_log 194215'1556 (194215'1139) delete 6c5018eb/benchmark_data_storage0111-ib.pocstor_19374_object18780/head//1 by client.1048726.0:45388 2014-08-12 17:12:12.710014 2014-08-13 17:38:02.561843 7fe9bae887a0 20 read_log 194215'1557 (194215'1140) delete d3893eeb/benchmark_data_storage0111-ib.pocstor_19374_object18819/head//1 by client.1048726.0:45427 2014-08-12 17:12:12.724198 2014-08-13 17:38:02.561867 7fe9bae887a0 20 read_log 194215'1558 (194215'1141) delete f022432b/benchmark_data_storage0111-ib.pocstor_19374_object18842/head//1 by client.1048726.0:45450 2014-08-12 17:12:12.732856 2014-08-13 17:38:02.561901 7fe9bae887a0 20 read_log 194215'1559 (194215'1142) delete 2476c96b/benchmark_data_storage0111-ib.pocstor_19374_object18852/head//1 by client.1048726.0:45460 2014-08-12 17:12:12.735763 2014-08-13 17:38:02.561924 7fe9bae887a0 20 read_log 194215'1560 (194215'1143) delete e1cd2ceb/benchmark_data_storage0111-ib.pocstor_19374_object19065/head//1 by client.1048726.0:45673 2014-08-12 17:12:12.818077 2014-08-13 17:38:02.561946 7fe9bae887a0 20 read_log 194215'1561 (194215'1144) delete ae4f962b/benchmark_data_storage0111-ib.pocstor_19374_object19226/head//1 by client.1048726.0:45834 2014-08-12 17:12:12.879829 2014-08-13 17:38:02.561967 7fe9bae887a0 20 read_log 194215'1562 (194215'1145) delete b8cf256b/benchmark_data_storage0111-ib.pocstor_19374_object19231/head//1 by client.1048726.0:45839 2014-08-12 17:12:12.881421 2014-08-13 17:38:02.561989 7fe9bae887a0 20 read_log 194215'1563 (194215'1146) delete f0edd2b/benchmark_data_storage0111-ib.pocstor_19374_object19233/head//1 by client.1048726.0:45841 2014-08-12 17:12:12.882068 2014-08-13 17:38:02.562011 7fe9bae887a0 20 read_log 194215'1564 (194215'1147) delete 2f22b1ab/benchmark_data_storage0111-ib.pocstor_19374_object19274/head//1 by client.1048726.0:45882 2014-08-12 17:12:12.895240 2014-08-13 17:38:02.562032 7fe9bae887a0 20 read_log 194215'1565 (194215'1148) delete 115748eb/benchmark_data_storage0111-ib.pocstor_19374_object19293/head//1 by client.1048726.0:45901 2014-08-12 17:12:12.901849 2014-08-13 17:38:02.562053 7fe9bae887a0 20 read_log 194215'1566 (194215'1149) delete ce72ce6b/benchmark_data_storage0111-ib.pocstor_19374_object19399/head//1 by client.1048726.0:46007 2014-08-12 17:12:12.935554 2014-08-13 17:38:02.562075 7fe9bae887a0 20 read_log 194215'1567 (194215'1150) delete d56f90ab/benchmark_data_storage0111-ib.pocstor_19374_object19623/head//1 by client.1048726.0:46231 2014-08-12 17:12:13.008926 2014-08-13 17:38:02.562096 7fe9bae887a0 20 read_log 194215'1568 (194215'1151) delete ee91eceb/benchmark_data_storage0111-ib.pocstor_19374_object19625/head//1 by client.1048726.0:46233 2014-08-12 17:12:13.009384 2014-08-13 17:38:02.562118 7fe9bae887a0 20 read_log 194215'1569 (194215'1152) delete 563df2ab/benchmark_data_storage0111-ib.pocstor_19374_object19626/head//1 by client.1048726.0:46234 2014-08-12 17:12:13.009468 2014-08-13 17:38:02.562139 7fe9bae887a0 20 read_log 194215'1570 (194215'1153) delete f16196eb/benchmark_data_storage0111-ib.pocstor_19374_object19875/head//1 by client.1048726.0:46483 2014-08-12 17:12:13.091595 2014-08-13 17:38:02.562186 7fe9bae887a0 20 read_log 194215'1571 (194215'1154) delete 43e8486b/benchmark_data_storage0111-ib.pocstor_19374_object19917/head//1 by client.1048726.0:46525 2014-08-12 17:12:13.107435 2014-08-13 17:38:02.562211 7fe9bae887a0 20 read_log 194215'1572 (194215'1155) delete 3847442b/benchmark_data_storage0111-ib.pocstor_19374_object19991/head//1 by client.1048726.0:46599 2014-08-12 17:12:13.132025 2014-08-13 17:38:02.562235 7fe9bae887a0 20 read_log 194215'1573 (194215'1156) delete 2e8e12ab/benchmark_data_storage0111-ib.pocstor_19374_object20016/head//1 by client.1048726.0:46624 2014-08-12 17:12:13.143630 2014-08-13 17:38:02.562257 7fe9bae887a0 20 read_log 194215'1574 (194215'1157) delete 9ea9826b/benchmark_data_storage0111-ib.pocstor_19374_object20100/head//1 by client.1048726.0:46708 2014-08-12 17:12:13.169849 2014-08-13 17:38:02.562278 7fe9bae887a0 20 read_log 194215'1575 (194215'1158) delete de30edeb/benchmark_data_storage0111-ib.pocstor_19374_object20132/head//1 by client.1048726.0:46740 2014-08-12 17:12:13.179443 2014-08-13 17:38:02.562300 7fe9bae887a0 20 read_log 194215'1576 (194215'1159) delete 47ed002b/benchmark_data_storage0111-ib.pocstor_19374_object20189/head//1 by client.1048726.0:46797 2014-08-12 17:12:13.199224 2014-08-13 17:38:02.562322 7fe9bae887a0 20 read_log 194215'1577 (194215'1160) delete 5be3cf2b/benchmark_data_storage0111-ib.pocstor_19374_object20205/head//1 by client.1048726.0:46813 2014-08-12 17:12:13.204286 2014-08-13 17:38:02.562344 7fe9bae887a0 20 read_log 194215'1578 (194215'1161) delete fbdfe52b/benchmark_data_storage0111-ib.pocstor_19374_object20231/head//1 by client.1048726.0:46839 2014-08-12 17:12:13.213770 2014-08-13 17:38:02.562411 7fe9bae887a0 20 read_log 194215'1579 (194215'1162) delete 45de5dab/benchmark_data_storage0111-ib.pocstor_19374_object20287/head//1 by client.1048726.0:46895 2014-08-12 17:12:13.230430 2014-08-13 17:38:02.562435 7fe9bae887a0 20 read_log 194215'1580 (194215'1163) delete 94efc5eb/benchmark_data_storage0111-ib.pocstor_19374_object20346/head//1 by client.1048726.0:46954 2014-08-12 17:12:13.249723 2014-08-13 17:38:02.562457 7fe9bae887a0 20 read_log 194215'1581 (194215'1164) delete 1347d22b/benchmark_data_storage0111-ib.pocstor_19374_object20407/head//1 by client.1048726.0:47015 2014-08-12 17:12:13.271648 2014-08-13 17:38:02.562479 7fe9bae887a0 20 read_log 194215'1582 (194215'1165) delete e33e54ab/benchmark_data_storage0111-ib.pocstor_19374_object20410/head//1 by client.1048726.0:47018 2014-08-12 17:12:13.273639 2014-08-13 17:38:02.562501 7fe9bae887a0 20 read_log 194215'1583 (194215'1166) delete e858402b/benchmark_data_storage0111-ib.pocstor_19374_object20437/head//1 by client.1048726.0:47045 2014-08-12 17:12:13.282106 2014-08-13 17:38:02.562525 7fe9bae887a0 20 read_log 194215'1584 (194215'1167) delete 19a5632b/benchmark_data_storage0111-ib.pocstor_19374_object20464/head//1 by client.1048726.0:47072 2014-08-12 17:12:13.290914 2014-08-13 17:38:02.562549 7fe9bae887a0 20 read_log 194215'1585 (194215'1168) delete 19fb1ceb/benchmark_data_storage0111-ib.pocstor_19374_object20580/head//1 by client.1048726.0:47188 2014-08-12 17:12:13.330303 2014-08-13 17:38:02.562571 7fe9bae887a0 20 read_log 194215'1586 (194215'1169) delete 63afe4eb/benchmark_data_storage0111-ib.pocstor_19374_object20590/head//1 by client.1048726.0:47198 2014-08-12 17:12:13.333747 2014-08-13 17:38:02.562596 7fe9bae887a0 20 read_log 194215'1587 (194215'1170) delete 5e2d476b/benchmark_data_storage0111-ib.pocstor_19374_object20594/head//1 by client.1048726.0:47202 2014-08-12 17:12:13.337531 2014-08-13 17:38:02.562618 7fe9bae887a0 20 read_log 194215'1588 (194215'1171) delete 7ffdd76b/benchmark_data_storage0111-ib.pocstor_19374_object20637/head//1 by client.1048726.0:47245 2014-08-12 17:12:13.349506 2014-08-13 17:38:02.562640 7fe9bae887a0 20 read_log 194215'1589 (194215'1172) delete 595617eb/benchmark_data_storage0111-ib.pocstor_19374_object20646/head//1 by client.1048726.0:47254 2014-08-12 17:12:13.351589 2014-08-13 17:38:02.562661 7fe9bae887a0 20 read_log 194215'1590 (194215'1173) delete 2860b42b/benchmark_data_storage0111-ib.pocstor_19374_object20753/head//1 by client.1048726.0:47361 2014-08-12 17:12:13.386695 2014-08-13 17:38:02.562683 7fe9bae887a0 20 read_log 194215'1591 (194215'1174) delete 496c846b/benchmark_data_storage0111-ib.pocstor_19374_object20823/head//1 by client.1048726.0:47431 2014-08-12 17:12:13.409981 2014-08-13 17:38:02.562704 7fe9bae887a0 20 read_log 194215'1592 (194215'1175) delete 22036eeb/benchmark_data_storage0111-ib.pocstor_19374_object20913/head//1 by client.1048726.0:47521 2014-08-12 17:12:13.436982 2014-08-13 17:38:02.562759 7fe9bae887a0 20 read_log 194215'1593 (194215'1176) delete ab250fab/benchmark_data_storage0111-ib.pocstor_19374_object20914/head//1 by client.1048726.0:47522 2014-08-12 17:12:13.439482 2014-08-13 17:38:02.562782 7fe9bae887a0 20 read_log 194215'1594 (194215'1177) delete ae70372b/benchmark_data_storage0111-ib.pocstor_19374_object21028/head//1 by client.1048726.0:47636 2014-08-12 17:12:13.478139 2014-08-13 17:38:02.562804 7fe9bae887a0 20 read_log 194215'1595 (194215'1178) delete 1036baeb/benchmark_data_storage0111-ib.pocstor_19374_object21032/head//1 by client.1048726.0:47640 2014-08-12 17:12:13.478452 2014-08-13 17:38:02.562825 7fe9bae887a0 20 read_log 194215'1596 (194215'1179) delete 4a8e3aeb/benchmark_data_storage0111-ib.pocstor_19374_object21109/head//1 by client.1048726.0:47717 2014-08-12 17:12:13.505238 2014-08-13 17:38:02.562847 7fe9bae887a0 20 read_log 194215'1597 (194215'1180) delete 753fa2ab/benchmark_data_storage0111-ib.pocstor_19374_object21247/head//1 by client.1048726.0:47855 2014-08-12 17:12:13.551714 2014-08-13 17:38:02.562868 7fe9bae887a0 20 read_log 194215'1598 (194215'1181) delete f67584eb/benchmark_data_storage0111-ib.pocstor_19374_object21267/head//1 by client.1048726.0:47875 2014-08-12 17:12:13.558048 2014-08-13 17:38:02.562902 7fe9bae887a0 20 read_log 194215'1599 (194215'1182) delete fe828eab/benchmark_data_storage0111-ib.pocstor_19374_object21345/head//1 by client.1048726.0:47953 2014-08-12 17:12:13.579470 2014-08-13 17:38:02.562926 7fe9bae887a0 20 read_log 194215'1600 (194215'1183) delete 2ef9f66b/benchmark_data_storage0111-ib.pocstor_19374_object21352/head//1 by client.1048726.0:47960 2014-08-12 17:12:13.580847 2014-08-13 17:38:02.562948 7fe9bae887a0 20 read_log 194215'1601 (194215'1184) delete 3560716b/benchmark_data_storage0111-ib.pocstor_19374_object21409/head//1 by client.1048726.0:48017 2014-08-12 17:12:13.602273 2014-08-13 17:38:02.562970 7fe9bae887a0 20 read_log 194215'1602 (194215'1185) delete 7876b26b/benchmark_data_storage0111-ib.pocstor_19374_object21474/head//1 by client.1048726.0:48082 2014-08-12 17:12:13.625859 2014-08-13 17:38:02.562996 7fe9bae887a0 20 read_log 194215'1603 (194215'1186) delete a090132b/benchmark_data_storage0111-ib.pocstor_19374_object21525/head//1 by client.1048726.0:48133 2014-08-12 17:12:13.641818 2014-08-13 17:38:02.563019 7fe9bae887a0 20 read_log 194215'1604 (194215'1187) delete 2b02b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21551/head//1 by client.1048726.0:48159 2014-08-12 17:12:13.650570 2014-08-13 17:38:02.563042 7fe9bae887a0 20 read_log 194215'1605 (194215'1188) delete 2af0d52b/benchmark_data_storage0111-ib.pocstor_19374_object21651/head//1 by client.1048726.0:48259 2014-08-12 17:12:13.683146 2014-08-13 17:38:02.563064 7fe9bae887a0 20 read_log 194215'1606 (194215'1189) delete df94b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21660/head//1 by client.1048726.0:48268 2014-08-12 17:12:13.685663 2014-08-13 17:38:02.563087 7fe9bae887a0 20 read_log 194215'1607 (194215'1190) delete 157f89ab/benchmark_data_storage0111-ib.pocstor_19374_object21754/head//1 by client.1048726.0:48362 2014-08-12 17:12:13.717148 2014-08-13 17:38:02.563110 7fe9bae887a0 20 read_log 194215'1608 (194215'1191) delete 267742eb/benchmark_data_storage0111-ib.pocstor_19374_object21769/head//1 by client.1048726.0:48377 2014-08-12 17:12:13.723023 2014-08-13 17:38:02.563134 7fe9bae887a0 20 read_log 194215'1609 (194215'1192) delete 7086f06b/benchmark_data_storage0111-ib.pocstor_19374_object21770/head//1 by client.1048726.0:48378 2014-08-12 17:12:13.723428 2014-08-13 17:38:02.563166 7fe9bae887a0 20 read_log 194215'1610 (194215'1193) delete 5d6f6f6b/benchmark_data_storage0111-ib.pocstor_19374_object21877/head//1 by client.1048726.0:48485 2014-08-12 17:12:13.756359 2014-08-13 17:38:02.563189 7fe9bae887a0 20 read_log 194215'1611 (194215'1194) delete 6a5b3deb/benchmark_data_storage0111-ib.pocstor_19374_object21945/head//1 by client.1048726.0:48553 2014-08-12 17:12:13.777382 2014-08-13 17:38:02.563212 7fe9bae887a0 20 read_log 194215'1612 (194215'1195) delete 66a13f2b/benchmark_data_storage0111-ib.pocstor_19374_object22113/head//1 by client.1048726.0:48721 2014-08-12 17:12:13.828658 2014-08-13 17:38:02.563234 7fe9bae887a0 20 read_log 194215'1613 (194215'1196) delete 207ab2eb/benchmark_data_storage0111-ib.pocstor_19374_object22174/head//1 by client.1048726.0:48782 2014-08-12 17:12:13.847221 2014-08-13 17:38:02.563256 7fe9bae887a0 20 read_log 194215'1614 (194215'1197) delete 752cbeab/benchmark_data_storage0111-ib.pocstor_19374_object22208/head//1 by client.1048726.0:48816 2014-08-12 17:12:13.859637 2014-08-13 17:38:02.563278 7fe9bae887a0 20 read_log 194215'1615 (194215'1198) delete 5f7e08eb/benchmark_data_storage0111-ib.pocstor_19374_object22348/head//1 by client.1048726.0:48956 2014-08-12 17:12:13.907818 2014-08-13 17:38:02.563300 7fe9bae887a0 20 read_log 194215'1616 (194215'1199) delete 1c91296b/benchmark_data_storage0111-ib.pocstor_19374_object22399/head//1 by client.1048726.0:49007 2014-08-12 17:12:13.925737 2014-08-13 17:38:02.563323 7fe9bae887a0 20 read_log 194215'1617 (194215'1200) delete ed68a1eb/benchmark_data_storage0111-ib.pocstor_19374_object22466/head//1 by client.1048726.0:49074 2014-08-12 17:12:13.947357 2014-08-13 17:38:02.563345 7fe9bae887a0 20 read_log 194215'1618 (194215'1201) delete db847beb/benchmark_data_storage0111-ib.pocstor_19374_object22490/head//1 by client.1048726.0:49098 2014-08-12 17:12:13.954034 2014-08-13 17:38:02.563414 7fe9bae887a0 20 read_log 194215'1619 (194215'1202) delete 9b61f7eb/benchmark_data_storage0111-ib.pocstor_19374_object22596/head//1 by client.1048726.0:49204 2014-08-12 17:12:13.992700 2014-08-13 17:38:02.563440 7fe9bae887a0 20 read_log 194215'1620 (194215'1203) delete 8df4ebab/benchmark_data_storage0111-ib.pocstor_19374_object22647/head//1 by client.1048726.0:49255 2014-08-12 17:12:14.009508 2014-08-13 17:38:02.563461 7fe9bae887a0 20 read_log 194215'1621 (194215'1204) delete ade90cab/benchmark_data_storage0111-ib.pocstor_19374_object22661/head//1 by client.1048726.0:49269 2014-08-12 17:12:14.013083 2014-08-13 17:38:02.563485 7fe9bae887a0 20 read_log 194215'1622 (194215'1205) delete d2df432b/benchmark_data_storage0111-ib.pocstor_19374_object22688/head//1 by client.1048726.0:49296 2014-08-12 17:12:14.022525 2014-08-13 17:38:02.563507 7fe9bae887a0 20 read_log 194215'1623 (194215'1206) delete f0a1f42b/benchmark_data_storage0111-ib.pocstor_19374_object22740/head//1 by client.1048726.0:49348 2014-08-12 17:12:14.041292 2014-08-13 17:38:02.563529 7fe9bae887a0 20 read_log 194215'1624 (194215'1207) delete b467b16b/benchmark_data_storage0111-ib.pocstor_19374_object22893/head//1 by client.1048726.0:49501 2014-08-12 17:12:14.097520 2014-08-13 17:38:02.563551 7fe9bae887a0 20 read_log 194215'1625 (194215'1208) delete 2febd7ab/benchmark_data_storage0111-ib.pocstor_19374_object22894/head//1 by client.1048726.0:49502 2014-08-12 17:12:14.097577 2014-08-13 17:38:02.563575 7fe9bae887a0 20 read_log 194215'1626 (194215'1209) delete e0db75ab/benchmark_data_storage0111-ib.pocstor_19374_object22965/head//1 by client.1048726.0:49573 2014-08-12 17:12:14.123655 2014-08-13 17:38:02.563597 7fe9bae887a0 20 read_log 194215'1627 (194215'1210) delete 6344e4eb/benchmark_data_storage0111-ib.pocstor_19374_object22992/head//1 by client.1048726.0:49600 2014-08-12 17:12:14.132342 2014-08-13 17:38:02.563619 7fe9bae887a0 20 read_log 194215'1628 (194215'1211) delete 7ba9fd2b/benchmark_data_storage0111-ib.pocstor_19374_object23012/head//1 by client.1048726.0:49620 2014-08-12 17:12:14.138767 2014-08-13 17:38:02.563641 7fe9bae887a0 20 read_log 194215'1629 (194215'1212) delete 8666e06b/benchmark_data_storage0111-ib.pocstor_19374_object23095/head//1 by client.1048726.0:49703 2014-08-12 17:12:14.171793 2014-08-13 17:38:02.563662 7fe9bae887a0 20 read_log 194215'1630 (194215'1213) delete ed901d2b/benchmark_data_storage0111-ib.pocstor_19374_object23113/head//1 by client.1048726.0:49721 2014-08-12 17:12:14.178096 2014-08-13 17:38:02.563684 7fe9bae887a0 20 read_log 194215'1631 (194215'1214) delete 84443b6b/benchmark_data_storage0111-ib.pocstor_19374_object23132/head//1 by client.1048726.0:49740 2014-08-12 17:12:14.183503 2014-08-13 17:38:02.563705 7fe9bae887a0 20 read_log 194215'1632 (194215'1215) delete 8cff746b/benchmark_data_storage0111-ib.pocstor_19374_object23361/head//1 by client.1048726.0:49969 2014-08-12 17:12:14.259734 2014-08-13 17:38:02.563726 7fe9bae887a0 20 read_log 194215'1633 (194215'1216) delete 4edb3b2b/benchmark_data_storage0111-ib.pocstor_19374_object23379/head//1 by client.1048726.0:49987 2014-08-12 17:12:14.264414 2014-08-13 17:38:02.563781 7fe9bae887a0 20 read_log 194215'1634 (194215'1217) delete 1c53ca2b/benchmark_data_storage0111-ib.pocstor_19374_object23426/head//1 by client.1048726.0:50034 2014-08-12 17:12:14.278685 2014-08-13 17:38:02.563803 7fe9bae887a0 20 read_log 194215'1635 (194215'1218) delete fa5b282b/benchmark_data_storage0111-ib.pocstor_19374_object23429/head//1 by client.1048726.0:50037 2014-08-12 17:12:14.279660 2014-08-13 17:38:02.563825 7fe9bae887a0 20 read_log 194215'1636 (194215'1219) delete 5334536b/benchmark_data_storage0111-ib.pocstor_19374_object23433/head//1 by client.1048726.0:50041 2014-08-12 17:12:14.281084 2014-08-13 17:38:02.563846 7fe9bae887a0 20 read_log 194215'1637 (194215'1220) delete 6211b72b/benchmark_data_storage0111-ib.pocstor_19374_object23453/head//1 by client.1048726.0:50061 2014-08-12 17:12:14.289776 2014-08-13 17:38:02.563868 7fe9bae887a0 20 read_log 194215'1638 (194215'1221) delete 2514846b/benchmark_data_storage0111-ib.pocstor_19374_object23592/head//1 by client.1048726.0:50200 2014-08-12 17:12:14.335200 2014-08-13 17:38:02.563901 7fe9bae887a0 20 read_log 194215'1639 (194215'1222) delete aae84e2b/benchmark_data_storage0111-ib.pocstor_19374_object23706/head//1 by client.1048726.0:50314 2014-08-12 17:12:14.376996 2014-08-13 17:38:02.563924 7fe9bae887a0 20 read_log 194215'1640 (194215'1223) delete b9e840ab/benchmark_data_storage0111-ib.pocstor_19374_object23839/head//1 by client.1048726.0:50447 2014-08-12 17:12:14.422302 2014-08-13 17:38:02.563948 7fe9bae887a0 20 read_log 194215'1641 (194215'1224) delete 4cb71eb/benchmark_data_storage0111-ib.pocstor_19374_object23922/head//1 by client.1048726.0:50530 2014-08-12 17:12:14.451835 2014-08-13 17:38:02.563970 7fe9bae887a0 20 read_log 194215'1642 (194215'1225) delete e4b2e06b/benchmark_data_storage0111-ib.pocstor_19374_object23948/head//1 by client.1048726.0:50556 2014-08-12 17:12:14.461716 2014-08-13 17:38:02.563991 7fe9bae887a0 20 read_log 194215'1643 (194215'1226) delete d835436b/benchmark_data_storage0111-ib.pocstor_19374_object24094/head//1 by client.1048726.0:50702 2014-08-12 17:12:14.522150 2014-08-13 17:38:02.564012 7fe9bae887a0 20 read_log 194215'1644 (194215'1227) delete a7483a2b/benchmark_data_storage0111-ib.pocstor_19374_object24133/head//1 by client.1048726.0:50741 2014-08-12 17:12:14.538926 2014-08-13 17:38:02.564034 7fe9bae887a0 20 read_log 194215'1645 (194215'1228) delete 4afb40eb/benchmark_data_storage0111-ib.pocstor_19374_object24160/head//1 by client.1048726.0:50768 2014-08-12 17:12:14.551542 2014-08-13 17:38:02.564090 7fe9bae887a0 20 read_log 194215'1646 (194215'1229) delete 273f5cab/benchmark_data_storage0111-ib.pocstor_19374_object24162/head//1 by client.1048726.0:50770 2014-08-12 17:12:14.554395 2014-08-13 17:38:02.564112 7fe9bae887a0 20 read_log 194215'1647 (194215'1230) delete 438a14eb/benchmark_data_storage0111-ib.pocstor_19374_object24263/head//1 by client.1048726.0:50871 2014-08-12 17:12:14.612903 2014-08-13 17:38:02.564133 7fe9bae887a0 20 read_log 194215'1648 (194215'1231) delete 33c0b76b/benchmark_data_storage0111-ib.pocstor_19374_object24565/head//1 by client.1048726.0:51173 2014-08-12 17:12:14.777834 2014-08-13 17:38:02.564162 7fe9bae887a0 20 read_log 194215'1649 (194215'1232) delete c08eceab/benchmark_data_storage0111-ib.pocstor_19374_object24597/head//1 by client.1048726.0:51205 2014-08-12 17:12:14.789974 2014-08-13 17:38:02.564185 7fe9bae887a0 20 read_log 194215'1650 (194215'1233) delete befd7d6b/benchmark_data_storage0111-ib.pocstor_19374_object24728/head//1 by client.1048726.0:51336 2014-08-12 17:12:14.838717 2014-08-13 17:38:02.564207 7fe9bae887a0 20 read_log 194215'1651 (194215'1234) delete e425ca6b/benchmark_data_storage0111-ib.pocstor_19374_object24734/head//1 by client.1048726.0:51342 2014-08-12 17:12:14.841588 2014-08-13 17:38:02.564229 7fe9bae887a0 20 read_log 194215'1652 (194215'1235) delete 961e2f2b/benchmark_data_storage0111-ib.pocstor_19374_object24748/head//1 by client.1048726.0:51356 2014-08-12 17:12:14.845298 2014-08-13 17:38:02.564250 7fe9bae887a0 20 read_log 194215'1653 (194215'1236) delete 4a2485eb/benchmark_data_storage0111-ib.pocstor_19374_object24798/head//1 by client.1048726.0:51406 2014-08-12 17:12:14.863505 2014-08-13 17:38:02.564271 7fe9bae887a0 20 read_log 194215'1654 (194215'1237) delete 66eb74eb/benchmark_data_storage0111-ib.pocstor_19374_object24869/head//1 by client.1048726.0:51477 2014-08-12 17:12:14.889857 2014-08-13 17:38:02.564293 7fe9bae887a0 20 read_log 194215'1655 (194215'1238) delete cdd7d8eb/benchmark_data_storage0111-ib.pocstor_19374_object24973/head//1 by client.1048726.0:51581 2014-08-12 17:12:14.930499 2014-08-13 17:38:02.564314 7fe9bae887a0 20 read_log 194215'1656 (194215'1239) delete a8ff1b2b/benchmark_data_storage0111-ib.pocstor_19374_object25163/head//1 by client.1048726.0:51771 2014-08-12 17:12:15.003035 2014-08-13 17:38:02.564336 7fe9bae887a0 20 read_log 194215'1657 (194215'1240) delete 80a012eb/benchmark_data_storage0111-ib.pocstor_19374_object25209/head//1 by client.1048726.0:51817 2014-08-12 17:12:15.018532 2014-08-13 17:38:02.564357 7fe9bae887a0 20 read_log 194215'1658 (194215'1241) delete 6a06c22b/benchmark_data_storage0111-ib.pocstor_19374_object25224/head//1 by client.1048726.0:51832 2014-08-12 17:12:15.024416 2014-08-13 17:38:02.564391 7fe9bae887a0 20 read_log 194215'1659 (194215'1242) delete 1c47e46b/benchmark_data_storage0111-ib.pocstor_19374_object25488/head//1 by client.1048726.0:52096 2014-08-12 17:12:15.114148 2014-08-13 17:38:02.564414 7fe9bae887a0 20 read_log 194215'1660 (194215'1243) delete d92e436b/benchmark_data_storage0111-ib.pocstor_19374_object25630/head//1 by client.1048726.0:52238 2014-08-12 17:12:15.160166 2014-08-13 17:38:02.564435 7fe9bae887a0 20 read_log 194215'1661 (194215'1244) delete abae06ab/benchmark_data_storage0111-ib.pocstor_19374_object25701/head//1 by client.1048726.0:52309 2014-08-12 17:12:15.185855 2014-08-13 17:38:02.564457 7fe9bae887a0 20 read_log 194215'1662 (194215'1245) delete 63f6f26b/benchmark_data_storage0111-ib.pocstor_19374_object25733/head//1 by client.1048726.0:52341 2014-08-12 17:12:15.195757 2014-08-13 17:38:02.564478 7fe9bae887a0 20 read_log 194215'1663 (194215'1246) delete 65c6a56b/benchmark_data_storage0111-ib.pocstor_19374_object25806/head//1 by client.1048726.0:52414 2014-08-12 17:12:15.220855 2014-08-13 17:38:02.564499 7fe9bae887a0 20 read_log 194215'1664 (194215'1247) delete f2f70f6b/benchmark_data_storage0111-ib.pocstor_19374_object25849/head//1 by client.1048726.0:52457 2014-08-12 17:12:15.237658 2014-08-13 17:38:02.564521 7fe9bae887a0 20 read_log 194215'1665 (194215'1248) delete c4335c2b/benchmark_data_storage0111-ib.pocstor_19374_object25922/head//1 by client.1048726.0:52530 2014-08-12 17:12:15.272334 2014-08-13 17:38:02.564542 7fe9bae887a0 20 read_log 194215'1666 (194215'1249) delete 559f96eb/benchmark_data_storage0111-ib.pocstor_19374_object26064/head//1 by client.1048726.0:52672 2014-08-12 17:12:15.328418 2014-08-13 17:38:02.564563 7fe9bae887a0 20 read_log 194215'1667 (194215'1250) delete b6adc82b/benchmark_data_storage0111-ib.pocstor_19374_object26221/head//1 by client.1048726.0:52829 2014-08-12 17:12:15.392797 2014-08-13 17:38:02.564585 7fe9bae887a0 20 read_log 194215'1668 (194215'1251) delete 76fd52ab/benchmark_data_storage0111-ib.pocstor_19374_object26229/head//1 by client.1048726.0:52837 2014-08-12 17:12:15.397904 2014-08-13 17:38:02.564606 7fe9bae887a0 20 read_log 194215'1669 (194215'1252) delete 94dc9c2b/benchmark_data_storage0111-ib.pocstor_19374_object26322/head//1 by client.1048726.0:52930 2014-08-12 17:12:15.436794 2014-08-13 17:38:02.564627 7fe9bae887a0 20 read_log 194215'1670 (194215'1253) delete 7560a26b/benchmark_data_storage0111-ib.pocstor_19374_object26345/head//1 by client.1048726.0:52953 2014-08-12 17:12:15.444980 2014-08-13 17:38:02.564649 7fe9bae887a0 20 read_log 194215'1671 (194215'1254) delete 1f77502b/benchmark_data_storage0111-ib.pocstor_19374_object26385/head//1 by client.1048726.0:52993 2014-08-12 17:12:15.462192 2014-08-13 17:38:02.564670 7fe9bae887a0 20 read_log 194215'1672 (194215'1255) delete 6b90772b/benchmark_data_storage0111-ib.pocstor_19374_object26540/head//1 by client.1048726.0:53148 2014-08-12 17:12:15.531821 2014-08-13 17:38:02.578112 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.578230 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.578281 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.055933 0 0.000000 2014-08-13 17:38:02.578308 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.578323 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.578339 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'1672], crt=0'0) 2014-08-13 17:38:02.578355 7fe9bae887a0 10 osd.4 194478 pgid 2.2a coll 2.2a_head 2014-08-13 17:38:02.578701 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 2.2a 2014-08-13 17:38:02.578719 7fe9bae887a0 10 osd.4 194478 _get_pool 2 2014-08-13 17:38:02.578777 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[2.2a(unlocked)] enter Initial 2014-08-13 17:38:02.578795 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[2.2a(unlocked)] enter NotTrimming 2014-08-13 17:38:02.578982 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.579206 7fe9bae887a0 20 read_log 194215'1 (0'0) modify 68edfb2a/benchmark_data_storage0111-ib.pocstor_48865_object150/head//2 by client.2191635.0:151 2014-08-12 17:14:57.943540 2014-08-13 17:38:02.579246 7fe9bae887a0 20 read_log 194215'2 (0'0) modify 11847ea/benchmark_data_storage0111-ib.pocstor_48865_object183/head//2 by client.2191635.0:184 2014-08-12 17:14:58.069710 2014-08-13 17:38:02.579271 7fe9bae887a0 20 read_log 194215'3 (0'0) modify 282e8a2a/benchmark_data_storage0111-ib.pocstor_48865_object269/head//2 by client.2191635.0:270 2014-08-12 17:14:58.390170 2014-08-13 17:38:02.579297 7fe9bae887a0 20 read_log 194215'4 (0'0) modify 2975e0ea/benchmark_data_storage0111-ib.pocstor_48865_object333/head//2 by client.2191635.0:334 2014-08-12 17:14:58.630161 2014-08-13 17:38:02.579319 7fe9bae887a0 20 read_log 194215'5 (0'0) modify 65323c2a/benchmark_data_storage0111-ib.pocstor_48865_object358/head//2 by client.2191635.0:359 2014-08-12 17:14:58.725101 2014-08-13 17:38:02.579340 7fe9bae887a0 20 read_log 194215'6 (0'0) modify 80c081ea/benchmark_data_storage0111-ib.pocstor_48865_object538/head//2 by client.2191635.0:539 2014-08-12 17:14:59.397550 2014-08-13 17:38:02.579362 7fe9bae887a0 20 read_log 194215'7 (0'0) modify 277a9c6a/benchmark_data_storage0111-ib.pocstor_48865_object629/head//2 by client.2191635.0:630 2014-08-12 17:14:59.742225 2014-08-13 17:38:02.579382 7fe9bae887a0 20 read_log 194215'8 (0'0) modify ce42036a/benchmark_data_storage0111-ib.pocstor_48865_object668/head//2 by client.2191635.0:669 2014-08-12 17:14:59.892946 2014-08-13 17:38:02.579403 7fe9bae887a0 20 read_log 194215'9 (0'0) modify 5955272a/benchmark_data_storage0111-ib.pocstor_48865_object715/head//2 by client.2191635.0:716 2014-08-12 17:15:00.103594 2014-08-13 17:38:02.579424 7fe9bae887a0 20 read_log 194215'10 (0'0) modify 52f5892a/benchmark_data_storage0111-ib.pocstor_48865_object726/head//2 by client.2191635.0:727 2014-08-12 17:15:00.171781 2014-08-13 17:38:02.579446 7fe9bae887a0 20 read_log 194215'11 (0'0) modify d1e810aa/benchmark_data_storage0111-ib.pocstor_48865_object1003/head//2 by client.2191635.0:1004 2014-08-12 17:15:01.226924 2014-08-13 17:38:02.579467 7fe9bae887a0 20 read_log 194215'12 (0'0) modify 70ee6b6a/benchmark_data_storage0111-ib.pocstor_48865_object1014/head//2 by client.2191635.0:1015 2014-08-12 17:15:01.268015 2014-08-13 17:38:02.579488 7fe9bae887a0 20 read_log 194215'13 (0'0) modify 81dd632a/benchmark_data_storage0111-ib.pocstor_48865_object1055/head//2 by client.2191635.0:1056 2014-08-12 17:15:01.421283 2014-08-13 17:38:02.579509 7fe9bae887a0 20 read_log 194215'14 (0'0) modify a31ef9aa/benchmark_data_storage0111-ib.pocstor_48865_object1066/head//2 by client.2191635.0:1067 2014-08-12 17:15:01.462438 2014-08-13 17:38:02.579543 7fe9bae887a0 20 read_log 194215'15 (0'0) modify bb8082a/benchmark_data_storage0111-ib.pocstor_48865_object1073/head//2 by client.2191635.0:1074 2014-08-12 17:15:01.488055 2014-08-13 17:38:02.579566 7fe9bae887a0 20 read_log 194215'16 (0'0) modify 8c5d0caa/benchmark_data_storage0111-ib.pocstor_48865_object1105/head//2 by client.2191635.0:1106 2014-08-12 17:15:01.607193 2014-08-13 17:38:02.579586 7fe9bae887a0 20 read_log 194215'17 (0'0) modify e781e86a/benchmark_data_storage0111-ib.pocstor_48865_object1126/head//2 by client.2191635.0:1127 2014-08-12 17:15:01.687538 2014-08-13 17:38:02.579608 7fe9bae887a0 20 read_log 194215'18 (0'0) modify bb59aaea/benchmark_data_storage0111-ib.pocstor_48865_object1239/head//2 by client.2191635.0:1240 2014-08-12 17:15:02.164730 2014-08-13 17:38:02.579629 7fe9bae887a0 20 read_log 194215'19 (0'0) modify f812992a/benchmark_data_storage0111-ib.pocstor_48865_object1316/head//2 by client.2191635.0:1317 2014-08-12 17:15:02.454095 2014-08-13 17:38:02.579649 7fe9bae887a0 20 read_log 194215'20 (0'0) modify 8845246a/benchmark_data_storage0111-ib.pocstor_48865_object1340/head//2 by client.2191635.0:1341 2014-08-12 17:15:02.544008 2014-08-13 17:38:02.579674 7fe9bae887a0 20 read_log 194215'21 (0'0) modify ac40caa/benchmark_data_storage0111-ib.pocstor_48865_object1424/head//2 by client.2191635.0:1425 2014-08-12 17:15:02.868950 2014-08-13 17:38:02.579695 7fe9bae887a0 20 read_log 194215'22 (0'0) modify ff0610aa/benchmark_data_storage0111-ib.pocstor_48865_object1571/head//2 by client.2191635.0:1572 2014-08-12 17:15:03.430301 2014-08-13 17:38:02.579717 7fe9bae887a0 20 read_log 194215'23 (0'0) modify 95a794ea/benchmark_data_storage0111-ib.pocstor_48865_object1601/head//2 by client.2191635.0:1602 2014-08-12 17:15:03.546025 2014-08-13 17:38:02.579738 7fe9bae887a0 20 read_log 194215'24 (0'0) modify cae8062a/benchmark_data_storage0111-ib.pocstor_48865_object1620/head//2 by client.2191635.0:1621 2014-08-12 17:15:03.618251 2014-08-13 17:38:02.579758 7fe9bae887a0 20 read_log 194215'25 (0'0) modify 920ef56a/benchmark_data_storage0111-ib.pocstor_48865_object1759/head//2 by client.2191635.0:1760 2014-08-12 17:15:04.156730 2014-08-13 17:38:02.579779 7fe9bae887a0 20 read_log 194215'26 (0'0) modify a732e2a/benchmark_data_storage0111-ib.pocstor_48865_object1807/head//2 by client.2191635.0:1808 2014-08-12 17:15:04.343397 2014-08-13 17:38:02.579800 7fe9bae887a0 20 read_log 194215'27 (0'0) modify c4c702aa/benchmark_data_storage0111-ib.pocstor_48865_object1871/head//2 by client.2191635.0:1872 2014-08-12 17:15:04.591112 2014-08-13 17:38:02.579821 7fe9bae887a0 20 read_log 194215'28 (0'0) modify 279c346a/benchmark_data_storage0111-ib.pocstor_48865_object1875/head//2 by client.2191635.0:1876 2014-08-12 17:15:04.606512 2014-08-13 17:38:02.579842 7fe9bae887a0 20 read_log 194215'29 (0'0) modify 553263ea/benchmark_data_storage0111-ib.pocstor_48865_object2043/head//2 by client.2191635.0:2044 2014-08-12 17:15:05.590851 2014-08-13 17:38:02.579863 7fe9bae887a0 20 read_log 194215'30 (0'0) modify f16ac3ea/benchmark_data_storage0111-ib.pocstor_48865_object2102/head//2 by client.2191635.0:2103 2014-08-12 17:15:05.812638 2014-08-13 17:38:02.579883 7fe9bae887a0 20 read_log 194215'31 (0'0) modify 5fc8512a/benchmark_data_storage0111-ib.pocstor_48865_object2104/head//2 by client.2191635.0:2105 2014-08-12 17:15:05.820138 2014-08-13 17:38:02.579904 7fe9bae887a0 20 read_log 194215'32 (0'0) modify a896f2ea/benchmark_data_storage0111-ib.pocstor_48865_object2127/head//2 by client.2191635.0:2128 2014-08-12 17:15:05.905528 2014-08-13 17:38:02.579925 7fe9bae887a0 20 read_log 194215'33 (0'0) modify b516fdaa/benchmark_data_storage0111-ib.pocstor_48865_object2151/head//2 by client.2191635.0:2152 2014-08-12 17:15:05.994619 2014-08-13 17:38:02.579945 7fe9bae887a0 20 read_log 194215'34 (0'0) modify 8cda0e6a/benchmark_data_storage0111-ib.pocstor_48865_object2185/head//2 by client.2191635.0:2186 2014-08-12 17:15:06.120792 2014-08-13 17:38:02.579982 7fe9bae887a0 20 read_log 194215'35 (0'0) modify 130ec9aa/benchmark_data_storage0111-ib.pocstor_48865_object2195/head//2 by client.2191635.0:2196 2014-08-12 17:15:06.157124 2014-08-13 17:38:02.580005 7fe9bae887a0 20 read_log 194215'36 (0'0) modify d4544c2a/benchmark_data_storage0111-ib.pocstor_48865_object2225/head//2 by client.2191635.0:2226 2014-08-12 17:15:06.267584 2014-08-13 17:38:02.580026 7fe9bae887a0 20 read_log 194215'37 (0'0) modify 8855e5aa/benchmark_data_storage0111-ib.pocstor_48865_object2244/head//2 by client.2191635.0:2245 2014-08-12 17:15:06.338488 2014-08-13 17:38:02.580046 7fe9bae887a0 20 read_log 194215'38 (0'0) modify 875f70ea/benchmark_data_storage0111-ib.pocstor_48865_object2251/head//2 by client.2191635.0:2252 2014-08-12 17:15:06.364235 2014-08-13 17:38:02.580070 7fe9bae887a0 20 read_log 194215'39 (0'0) modify 365a75ea/benchmark_data_storage0111-ib.pocstor_48865_object2593/head//2 by client.2191635.0:2594 2014-08-12 17:15:07.629790 2014-08-13 17:38:02.580091 7fe9bae887a0 20 read_log 194215'40 (0'0) modify 1cc465aa/benchmark_data_storage0111-ib.pocstor_48865_object2601/head//2 by client.2191635.0:2602 2014-08-12 17:15:07.662124 2014-08-13 17:38:02.580112 7fe9bae887a0 20 read_log 194215'41 (0'0) modify b7b2a22a/benchmark_data_storage0111-ib.pocstor_48865_object2629/head//2 by client.2191635.0:2630 2014-08-12 17:15:07.765164 2014-08-13 17:38:02.580133 7fe9bae887a0 20 read_log 194215'42 (0'0) modify 109903aa/benchmark_data_storage0111-ib.pocstor_48865_object2691/head//2 by client.2191635.0:2692 2014-08-12 17:15:07.996861 2014-08-13 17:38:02.580172 7fe9bae887a0 20 read_log 194215'43 (0'0) modify db63af6a/benchmark_data_storage0111-ib.pocstor_48865_object2699/head//2 by client.2191635.0:2700 2014-08-12 17:15:08.027377 2014-08-13 17:38:02.580199 7fe9bae887a0 20 read_log 194215'44 (0'0) modify 3dfdc96a/benchmark_data_storage0111-ib.pocstor_48865_object2712/head//2 by client.2191635.0:2713 2014-08-12 17:15:08.074848 2014-08-13 17:38:02.580225 7fe9bae887a0 20 read_log 194215'45 (0'0) modify 839a1faa/benchmark_data_storage0111-ib.pocstor_48865_object2786/head//2 by client.2191635.0:2787 2014-08-12 17:15:08.347678 2014-08-13 17:38:02.580247 7fe9bae887a0 20 read_log 194215'46 (0'0) modify 7d0316aa/benchmark_data_storage0111-ib.pocstor_48865_object2888/head//2 by client.2191635.0:2889 2014-08-12 17:15:08.721395 2014-08-13 17:38:02.580267 7fe9bae887a0 20 read_log 194215'47 (0'0) modify 8fd59f2a/benchmark_data_storage0111-ib.pocstor_48865_object3016/head//2 by client.2191635.0:3017 2014-08-12 17:15:09.188152 2014-08-13 17:38:02.580288 7fe9bae887a0 20 read_log 194215'48 (0'0) modify 2f6cc42a/benchmark_data_storage0111-ib.pocstor_48865_object3021/head//2 by client.2191635.0:3022 2014-08-12 17:15:09.206474 2014-08-13 17:38:02.580309 7fe9bae887a0 20 read_log 194215'49 (0'0) modify 32c490ea/benchmark_data_storage0111-ib.pocstor_48865_object3081/head//2 by client.2191635.0:3082 2014-08-12 17:15:09.426514 2014-08-13 17:38:02.580330 7fe9bae887a0 20 read_log 194215'50 (0'0) modify ead47aea/benchmark_data_storage0111-ib.pocstor_48865_object3105/head//2 by client.2191635.0:3106 2014-08-12 17:15:09.514927 2014-08-13 17:38:02.580382 7fe9bae887a0 20 read_log 194215'51 (0'0) modify bddd492a/benchmark_data_storage0111-ib.pocstor_48865_object3141/head//2 by client.2191635.0:3142 2014-08-12 17:15:09.648681 2014-08-13 17:38:02.580437 7fe9bae887a0 20 read_log 194215'52 (0'0) modify f9981caa/benchmark_data_storage0111-ib.pocstor_48865_object3321/head//2 by client.2191635.0:3322 2014-08-12 17:15:10.397453 2014-08-13 17:38:02.580464 7fe9bae887a0 20 read_log 194215'53 (0'0) modify d01bb4ea/benchmark_data_storage0111-ib.pocstor_48865_object3371/head//2 by client.2191635.0:3372 2014-08-12 17:15:10.624977 2014-08-13 17:38:02.580485 7fe9bae887a0 20 read_log 194215'54 (0'0) modify f0690b6a/benchmark_data_storage0111-ib.pocstor_48865_object3420/head//2 by client.2191635.0:3421 2014-08-12 17:15:10.802546 2014-08-13 17:38:02.580518 7fe9bae887a0 20 read_log 194215'55 (0'0) modify ef7fe82a/benchmark_data_storage0111-ib.pocstor_48865_object3426/head//2 by client.2191635.0:3427 2014-08-12 17:15:10.825579 2014-08-13 17:38:02.580540 7fe9bae887a0 20 read_log 194215'56 (0'0) modify 4f5aa4ea/benchmark_data_storage0111-ib.pocstor_48865_object3456/head//2 by client.2191635.0:3457 2014-08-12 17:15:10.934509 2014-08-13 17:38:02.580561 7fe9bae887a0 20 read_log 194215'57 (0'0) modify e192b76a/benchmark_data_storage0111-ib.pocstor_48865_object3465/head//2 by client.2191635.0:3466 2014-08-12 17:15:10.967414 2014-08-13 17:38:02.580582 7fe9bae887a0 20 read_log 194215'58 (0'0) modify ed61ad6a/benchmark_data_storage0111-ib.pocstor_48865_object3599/head//2 by client.2191635.0:3600 2014-08-12 17:15:11.462967 2014-08-13 17:38:02.580602 7fe9bae887a0 20 read_log 194215'59 (0'0) modify 284476ea/benchmark_data_storage0111-ib.pocstor_48865_object3637/head//2 by client.2191635.0:3638 2014-08-12 17:15:11.604438 2014-08-13 17:38:02.580623 7fe9bae887a0 20 read_log 194215'60 (0'0) modify 69ffe8ea/benchmark_data_storage0111-ib.pocstor_48865_object3643/head//2 by client.2191635.0:3644 2014-08-12 17:15:11.626588 2014-08-13 17:38:02.580643 7fe9bae887a0 20 read_log 194215'61 (0'0) modify 4443936a/benchmark_data_storage0111-ib.pocstor_48865_object3673/head//2 by client.2191635.0:3674 2014-08-12 17:15:11.738317 2014-08-13 17:38:02.580667 7fe9bae887a0 20 read_log 194215'62 (0'0) modify b1fb3aaa/benchmark_data_storage0111-ib.pocstor_48865_object3684/head//2 by client.2191635.0:3685 2014-08-12 17:15:11.778152 2014-08-13 17:38:02.580688 7fe9bae887a0 20 read_log 194215'63 (0'0) modify abbf486a/benchmark_data_storage0111-ib.pocstor_48865_object3708/head//2 by client.2191635.0:3709 2014-08-12 17:15:11.868006 2014-08-13 17:38:02.580708 7fe9bae887a0 20 read_log 194215'64 (0'0) modify d2774aaa/benchmark_data_storage0111-ib.pocstor_48865_object3723/head//2 by client.2191635.0:3724 2014-08-12 17:15:11.922298 2014-08-13 17:38:02.580729 7fe9bae887a0 20 read_log 194215'65 (0'0) modify d4716aaa/benchmark_data_storage0111-ib.pocstor_48865_object3873/head//2 by client.2191635.0:3874 2014-08-12 17:15:12.588987 2014-08-13 17:38:02.580749 7fe9bae887a0 20 read_log 194215'66 (0'0) modify 2546412a/benchmark_data_storage0111-ib.pocstor_48865_object3982/head//2 by client.2191635.0:3983 2014-08-12 17:15:13.175586 2014-08-13 17:38:02.580772 7fe9bae887a0 20 read_log 194215'67 (0'0) modify fce3fb2a/benchmark_data_storage0111-ib.pocstor_48865_object4047/head//2 by client.2191635.0:4048 2014-08-12 17:15:13.522121 2014-08-13 17:38:02.580795 7fe9bae887a0 20 read_log 194215'68 (0'0) modify 880c6cea/benchmark_data_storage0111-ib.pocstor_48865_object4061/head//2 by client.2191635.0:4062 2014-08-12 17:15:13.593244 2014-08-13 17:38:02.580816 7fe9bae887a0 20 read_log 194215'69 (0'0) modify da367caa/benchmark_data_storage0111-ib.pocstor_48865_object4094/head//2 by client.2191635.0:4095 2014-08-12 17:15:13.797370 2014-08-13 17:38:02.580836 7fe9bae887a0 20 read_log 194215'70 (0'0) modify fbd37baa/benchmark_data_storage0111-ib.pocstor_48865_object4145/head//2 by client.2191635.0:4146 2014-08-12 17:15:14.564150 2014-08-13 17:38:02.580859 7fe9bae887a0 20 read_log 194215'71 (0'0) modify df68caea/benchmark_data_storage0111-ib.pocstor_48865_object4151/head//2 by client.2191635.0:4152 2014-08-12 17:15:14.594531 2014-08-13 17:38:02.580880 7fe9bae887a0 20 read_log 194215'72 (0'0) modify 7757da6a/benchmark_data_storage0111-ib.pocstor_48865_object4234/head//2 by client.2191635.0:4235 2014-08-12 17:15:14.898093 2014-08-13 17:38:02.580900 7fe9bae887a0 20 read_log 194215'73 (0'0) modify aef9282a/benchmark_data_storage0111-ib.pocstor_48865_object4266/head//2 by client.2191635.0:4267 2014-08-12 17:15:15.017677 2014-08-13 17:38:02.580921 7fe9bae887a0 20 read_log 194215'74 (0'0) modify ca7790ea/benchmark_data_storage0111-ib.pocstor_48865_object4302/head//2 by client.2191635.0:4303 2014-08-12 17:15:15.183823 2014-08-13 17:38:02.580979 7fe9bae887a0 20 read_log 194215'75 (0'0) modify 5c96bb6a/benchmark_data_storage0111-ib.pocstor_48865_object4312/head//2 by client.2191635.0:4313 2014-08-12 17:15:15.229432 2014-08-13 17:38:02.581002 7fe9bae887a0 20 read_log 194215'76 (0'0) modify 463bdeaa/benchmark_data_storage0111-ib.pocstor_48865_object4392/head//2 by client.2191635.0:4393 2014-08-12 17:15:15.546223 2014-08-13 17:38:02.581023 7fe9bae887a0 20 read_log 194215'77 (0'0) modify d37ec2ea/benchmark_data_storage0111-ib.pocstor_48865_object4404/head//2 by client.2191635.0:4405 2014-08-12 17:15:15.589482 2014-08-13 17:38:02.581044 7fe9bae887a0 20 read_log 194215'78 (0'0) modify 50c68dea/benchmark_data_storage0111-ib.pocstor_48865_object4406/head//2 by client.2191635.0:4407 2014-08-12 17:15:15.596585 2014-08-13 17:38:02.581065 7fe9bae887a0 20 read_log 194215'79 (0'0) modify 4e5fdaaa/benchmark_data_storage0111-ib.pocstor_48865_object4437/head//2 by client.2191635.0:4438 2014-08-12 17:15:15.711200 2014-08-13 17:38:02.581085 7fe9bae887a0 20 read_log 194215'80 (0'0) modify c56423aa/benchmark_data_storage0111-ib.pocstor_48865_object4456/head//2 by client.2191635.0:4457 2014-08-12 17:15:15.780426 2014-08-13 17:38:02.581106 7fe9bae887a0 20 read_log 194215'81 (0'0) modify 48228f2a/benchmark_data_storage0111-ib.pocstor_48865_object4543/head//2 by client.2191635.0:4544 2014-08-12 17:15:16.099710 2014-08-13 17:38:02.581129 7fe9bae887a0 20 read_log 194215'82 (0'0) modify f096ccea/benchmark_data_storage0111-ib.pocstor_48865_object4594/head//2 by client.2191635.0:4595 2014-08-12 17:15:16.287510 2014-08-13 17:38:02.581151 7fe9bae887a0 20 read_log 194215'83 (0'0) modify 775974ea/benchmark_data_storage0111-ib.pocstor_48865_object4605/head//2 by client.2191635.0:4606 2014-08-12 17:15:16.328242 2014-08-13 17:38:02.581182 7fe9bae887a0 20 read_log 194215'84 (0'0) modify dee4596a/benchmark_data_storage0111-ib.pocstor_48865_object4660/head//2 by client.2191635.0:4661 2014-08-12 17:15:16.531495 2014-08-13 17:38:02.581204 7fe9bae887a0 20 read_log 194215'85 (0'0) modify b3d8632a/benchmark_data_storage0111-ib.pocstor_48865_object4719/head//2 by client.2191635.0:4720 2014-08-12 17:15:16.750627 2014-08-13 17:38:02.581225 7fe9bae887a0 20 read_log 194215'86 (0'0) modify dfa2a76a/benchmark_data_storage0111-ib.pocstor_48865_object4725/head//2 by client.2191635.0:4726 2014-08-12 17:15:16.773159 2014-08-13 17:38:02.581246 7fe9bae887a0 20 read_log 194215'87 (0'0) modify ea96c2aa/benchmark_data_storage0111-ib.pocstor_48865_object4778/head//2 by client.2191635.0:4779 2014-08-12 17:15:16.969685 2014-08-13 17:38:02.581266 7fe9bae887a0 20 read_log 194215'88 (0'0) modify 37ca6bea/benchmark_data_storage0111-ib.pocstor_48865_object4949/head//2 by client.2191635.0:4950 2014-08-12 17:15:17.604129 2014-08-13 17:38:02.581287 7fe9bae887a0 20 read_log 194215'89 (0'0) modify 62eca2ea/benchmark_data_storage0111-ib.pocstor_48865_object4961/head//2 by client.2191635.0:4962 2014-08-12 17:15:17.647996 2014-08-13 17:38:02.581307 7fe9bae887a0 20 read_log 194215'90 (0'0) modify 489460ea/benchmark_data_storage0111-ib.pocstor_48865_object4986/head//2 by client.2191635.0:4987 2014-08-12 17:15:17.742016 2014-08-13 17:38:02.581364 7fe9bae887a0 20 read_log 194215'91 (0'0) modify fc746b2a/benchmark_data_storage0111-ib.pocstor_48865_object5031/head//2 by client.2191635.0:5032 2014-08-12 17:15:17.906698 2014-08-13 17:38:02.581386 7fe9bae887a0 20 read_log 194215'92 (0'0) modify dc8ffd6a/benchmark_data_storage0111-ib.pocstor_48865_object5063/head//2 by client.2191635.0:5064 2014-08-12 17:15:18.026716 2014-08-13 17:38:02.581407 7fe9bae887a0 20 read_log 194215'93 (0'0) modify 98f0566a/benchmark_data_storage0111-ib.pocstor_48865_object5274/head//2 by client.2191635.0:5275 2014-08-12 17:15:18.976790 2014-08-13 17:38:02.581428 7fe9bae887a0 20 read_log 194215'94 (0'0) modify a673bdea/benchmark_data_storage0111-ib.pocstor_48865_object5308/head//2 by client.2191635.0:5309 2014-08-12 17:15:19.156388 2014-08-13 17:38:02.581460 7fe9bae887a0 20 read_log 194215'95 (0'0) modify dd5e12aa/benchmark_data_storage0111-ib.pocstor_48865_object5321/head//2 by client.2191635.0:5322 2014-08-12 17:15:19.218769 2014-08-13 17:38:02.581483 7fe9bae887a0 20 read_log 194215'96 (0'0) modify 743f146a/benchmark_data_storage0111-ib.pocstor_48865_object5342/head//2 by client.2191635.0:5343 2014-08-12 17:15:19.297748 2014-08-13 17:38:02.581503 7fe9bae887a0 20 read_log 194215'97 (0'0) modify 4b19ae2a/benchmark_data_storage0111-ib.pocstor_48865_object5365/head//2 by client.2191635.0:5366 2014-08-12 17:15:19.382771 2014-08-13 17:38:02.581524 7fe9bae887a0 20 read_log 194215'98 (0'0) modify 4187d4aa/benchmark_data_storage0111-ib.pocstor_48865_object5418/head//2 by client.2191635.0:5419 2014-08-12 17:15:19.579006 2014-08-13 17:38:02.581545 7fe9bae887a0 20 read_log 194215'99 (0'0) modify 76e737aa/benchmark_data_storage0111-ib.pocstor_48865_object5442/head//2 by client.2191635.0:5443 2014-08-12 17:15:19.667852 2014-08-13 17:38:02.581565 7fe9bae887a0 20 read_log 194215'100 (0'0) modify 90d3712a/benchmark_data_storage0111-ib.pocstor_48865_object5446/head//2 by client.2191635.0:5447 2014-08-12 17:15:19.684038 2014-08-13 17:38:02.581586 7fe9bae887a0 20 read_log 194215'101 (0'0) modify 59cb2aea/benchmark_data_storage0111-ib.pocstor_48865_object5453/head//2 by client.2191635.0:5454 2014-08-12 17:15:19.710685 2014-08-13 17:38:02.581607 7fe9bae887a0 20 read_log 194215'102 (0'0) modify d0ed1c6a/benchmark_data_storage0111-ib.pocstor_48865_object5498/head//2 by client.2191635.0:5499 2014-08-12 17:15:19.929410 2014-08-13 17:38:02.581628 7fe9bae887a0 20 read_log 194215'103 (0'0) modify 48e796a/benchmark_data_storage0111-ib.pocstor_48865_object5511/head//2 by client.2191635.0:5512 2014-08-12 17:15:20.000046 2014-08-13 17:38:02.581649 7fe9bae887a0 20 read_log 194215'104 (0'0) modify b179e16a/benchmark_data_storage0111-ib.pocstor_48865_object5567/head//2 by client.2191635.0:5568 2014-08-12 17:15:20.225598 2014-08-13 17:38:02.581669 7fe9bae887a0 20 read_log 194215'105 (0'0) modify d38639aa/benchmark_data_storage0111-ib.pocstor_48865_object5574/head//2 by client.2191635.0:5575 2014-08-12 17:15:20.251008 2014-08-13 17:38:02.581690 7fe9bae887a0 20 read_log 194215'106 (0'0) modify 6cb251aa/benchmark_data_storage0111-ib.pocstor_48865_object5598/head//2 by client.2191635.0:5599 2014-08-12 17:15:20.337605 2014-08-13 17:38:02.581710 7fe9bae887a0 20 read_log 194215'107 (0'0) modify eaccd2a/benchmark_data_storage0111-ib.pocstor_48865_object5617/head//2 by client.2191635.0:5618 2014-08-12 17:15:20.407130 2014-08-13 17:38:02.581731 7fe9bae887a0 20 read_log 194215'108 (0'0) modify a3f082aa/benchmark_data_storage0111-ib.pocstor_48865_object5622/head//2 by client.2191635.0:5623 2014-08-12 17:15:20.425080 2014-08-13 17:38:02.581751 7fe9bae887a0 20 read_log 194215'109 (0'0) modify a3af342a/benchmark_data_storage0111-ib.pocstor_48865_object5745/head//2 by client.2191635.0:5746 2014-08-12 17:15:20.875893 2014-08-13 17:38:02.581772 7fe9bae887a0 20 read_log 194215'110 (0'0) modify e572e62a/benchmark_data_storage0111-ib.pocstor_48865_object5964/head//2 by client.2191635.0:5965 2014-08-12 17:15:21.668664 2014-08-13 17:38:02.581793 7fe9bae887a0 20 read_log 194215'111 (0'0) modify b874d76a/benchmark_data_storage0111-ib.pocstor_48865_object5981/head//2 by client.2191635.0:5982 2014-08-12 17:15:21.731061 2014-08-13 17:38:02.581813 7fe9bae887a0 20 read_log 194215'112 (0'0) modify b9b163ea/benchmark_data_storage0111-ib.pocstor_48865_object5987/head//2 by client.2191635.0:5988 2014-08-12 17:15:21.752577 2014-08-13 17:38:02.581834 7fe9bae887a0 20 read_log 194215'113 (0'0) modify 48d2f52a/benchmark_data_storage0111-ib.pocstor_48865_object6057/head//2 by client.2191635.0:6058 2014-08-12 17:15:22.003642 2014-08-13 17:38:02.581855 7fe9bae887a0 20 read_log 194215'114 (0'0) modify 857610ea/benchmark_data_storage0111-ib.pocstor_48865_object6104/head//2 by client.2191635.0:6105 2014-08-12 17:15:22.217161 2014-08-13 17:38:02.581890 7fe9bae887a0 20 read_log 194215'115 (0'0) modify 2628a2ea/benchmark_data_storage0111-ib.pocstor_48865_object6300/head//2 by client.2191635.0:6301 2014-08-12 17:15:22.958415 2014-08-13 17:38:02.581913 7fe9bae887a0 20 read_log 194215'116 (0'0) modify 6374406a/benchmark_data_storage0111-ib.pocstor_48865_object6344/head//2 by client.2191635.0:6345 2014-08-12 17:15:23.119532 2014-08-13 17:38:02.581933 7fe9bae887a0 20 read_log 194215'117 (0'0) modify 1a2b33ea/benchmark_data_storage0111-ib.pocstor_48865_object6404/head//2 by client.2191635.0:6405 2014-08-12 17:15:23.334451 2014-08-13 17:38:02.581956 7fe9bae887a0 20 read_log 194215'118 (0'0) modify 56ba632a/benchmark_data_storage0111-ib.pocstor_48865_object6415/head//2 by client.2191635.0:6416 2014-08-12 17:15:23.374708 2014-08-13 17:38:02.582006 7fe9bae887a0 20 read_log 194215'119 (0'0) modify 7073ea/benchmark_data_storage0111-ib.pocstor_48865_object6523/head//2 by client.2191635.0:6524 2014-08-12 17:15:23.765336 2014-08-13 17:38:02.582027 7fe9bae887a0 20 read_log 194215'120 (0'0) modify 4be729ea/benchmark_data_storage0111-ib.pocstor_48865_object6640/head//2 by client.2191635.0:6641 2014-08-12 17:15:24.190079 2014-08-13 17:38:02.582048 7fe9bae887a0 20 read_log 194215'121 (0'0) modify f386bea/benchmark_data_storage0111-ib.pocstor_48865_object6650/head//2 by client.2191635.0:6651 2014-08-12 17:15:24.228296 2014-08-13 17:38:02.582069 7fe9bae887a0 20 read_log 194215'122 (0'0) modify 64fac96a/benchmark_data_storage0111-ib.pocstor_48865_object6732/head//2 by client.2191635.0:6733 2014-08-12 17:15:24.521396 2014-08-13 17:38:02.582089 7fe9bae887a0 20 read_log 194215'123 (0'0) modify 68644faa/benchmark_data_storage0111-ib.pocstor_48865_object6763/head//2 by client.2191635.0:6764 2014-08-12 17:15:24.633924 2014-08-13 17:38:02.582112 7fe9bae887a0 20 read_log 194215'124 (0'0) modify e27b086a/benchmark_data_storage0111-ib.pocstor_48865_object6779/head//2 by client.2191635.0:6780 2014-08-12 17:15:24.690074 2014-08-13 17:38:02.582133 7fe9bae887a0 20 read_log 194215'125 (0'0) modify db1f17ea/benchmark_data_storage0111-ib.pocstor_48865_object6852/head//2 by client.2191635.0:6853 2014-08-12 17:15:25.125365 2014-08-13 17:38:02.582161 7fe9bae887a0 20 read_log 194215'126 (0'0) modify 45f4e1ea/benchmark_data_storage0111-ib.pocstor_48865_object6955/head//2 by client.2191635.0:6956 2014-08-12 17:15:25.530331 2014-08-13 17:38:02.582183 7fe9bae887a0 20 read_log 194215'127 (0'0) modify 223884ea/benchmark_data_storage0111-ib.pocstor_48865_object7007/head//2 by client.2191635.0:7008 2014-08-12 17:15:25.717952 2014-08-13 17:38:02.582204 7fe9bae887a0 20 read_log 194215'128 (0'0) modify 84d19fea/benchmark_data_storage0111-ib.pocstor_48865_object7058/head//2 by client.2191635.0:7059 2014-08-12 17:15:25.901598 2014-08-13 17:38:02.582225 7fe9bae887a0 20 read_log 194215'129 (0'0) modify a64a352a/benchmark_data_storage0111-ib.pocstor_48865_object7111/head//2 by client.2191635.0:7112 2014-08-12 17:15:26.091912 2014-08-13 17:38:02.582246 7fe9bae887a0 20 read_log 194215'130 (0'0) modify 9abeec2a/benchmark_data_storage0111-ib.pocstor_48865_object7201/head//2 by client.2191635.0:7202 2014-08-12 17:15:26.413796 2014-08-13 17:38:02.582266 7fe9bae887a0 20 read_log 194215'131 (0'0) modify 2a4d06aa/benchmark_data_storage0111-ib.pocstor_48865_object7255/head//2 by client.2191635.0:7256 2014-08-12 17:15:26.606790 2014-08-13 17:38:02.582287 7fe9bae887a0 20 read_log 194215'132 (0'0) modify 64ec0faa/benchmark_data_storage0111-ib.pocstor_48865_object7261/head//2 by client.2191635.0:7262 2014-08-12 17:15:26.627828 2014-08-13 17:38:02.582307 7fe9bae887a0 20 read_log 194215'133 (0'0) modify d703ceaa/benchmark_data_storage0111-ib.pocstor_48865_object7323/head//2 by client.2191635.0:7324 2014-08-12 17:15:26.851411 2014-08-13 17:38:02.582328 7fe9bae887a0 20 read_log 194215'134 (0'0) modify afc50fea/benchmark_data_storage0111-ib.pocstor_48865_object7361/head//2 by client.2191635.0:7362 2014-08-12 17:15:26.988591 2014-08-13 17:38:02.582360 7fe9bae887a0 20 read_log 194215'135 (0'0) modify 70d399aa/benchmark_data_storage0111-ib.pocstor_48865_object7567/head//2 by client.2191635.0:7568 2014-08-12 17:15:27.729882 2014-08-13 17:38:02.582384 7fe9bae887a0 20 read_log 194215'136 (0'0) modify 7e608b6a/benchmark_data_storage0111-ib.pocstor_48865_object7574/head//2 by client.2191635.0:7575 2014-08-12 17:15:27.755390 2014-08-13 17:38:02.582405 7fe9bae887a0 20 read_log 194215'137 (0'0) modify 55a67f2a/benchmark_data_storage0111-ib.pocstor_48865_object7751/head//2 by client.2191635.0:7752 2014-08-12 17:15:28.392786 2014-08-13 17:38:02.582426 7fe9bae887a0 20 read_log 194215'138 (0'0) modify c7ab856a/benchmark_data_storage0111-ib.pocstor_48865_object7992/head//2 by client.2191635.0:7993 2014-08-12 17:15:30.007218 2014-08-13 17:38:02.582448 7fe9bae887a0 20 read_log 194215'139 (0'0) modify 89882f2a/benchmark_data_storage0111-ib.pocstor_48865_object8149/head//2 by client.2191635.0:8150 2014-08-12 17:15:30.596244 2014-08-13 17:38:02.582468 7fe9bae887a0 20 read_log 194215'140 (0'0) modify 560df9ea/benchmark_data_storage0111-ib.pocstor_48865_object8228/head//2 by client.2191635.0:8229 2014-08-12 17:15:30.880045 2014-08-13 17:38:02.582489 7fe9bae887a0 20 read_log 194215'141 (0'0) modify 88c4606a/benchmark_data_storage0111-ib.pocstor_48865_object8266/head//2 by client.2191635.0:8267 2014-08-12 17:15:31.016827 2014-08-13 17:38:02.582511 7fe9bae887a0 20 read_log 194215'142 (0'0) modify addd1f6a/benchmark_data_storage0111-ib.pocstor_48865_object8285/head//2 by client.2191635.0:8286 2014-08-12 17:15:31.118403 2014-08-13 17:38:02.582535 7fe9bae887a0 20 read_log 194215'143 (0'0) modify 53f2bbaa/benchmark_data_storage0111-ib.pocstor_48865_object8316/head//2 by client.2191635.0:8317 2014-08-12 17:15:31.312440 2014-08-13 17:38:02.582556 7fe9bae887a0 20 read_log 194215'144 (0'0) modify e60a5caa/benchmark_data_storage0111-ib.pocstor_48865_object8605/head//2 by client.2191635.0:8606 2014-08-12 17:15:32.690445 2014-08-13 17:38:02.582607 7fe9bae887a0 20 read_log 194215'145 (0'0) modify e9f3b4aa/benchmark_data_storage0111-ib.pocstor_48865_object8698/head//2 by client.2191635.0:8699 2014-08-12 17:15:33.027523 2014-08-13 17:38:02.582630 7fe9bae887a0 20 read_log 194215'146 (0'0) modify 66d1cb6a/benchmark_data_storage0111-ib.pocstor_48865_object8716/head//2 by client.2191635.0:8717 2014-08-12 17:15:33.091962 2014-08-13 17:38:02.582651 7fe9bae887a0 20 read_log 194215'147 (0'0) modify fbb9d36a/benchmark_data_storage0111-ib.pocstor_48865_object8783/head//2 by client.2191635.0:8784 2014-08-12 17:15:33.336870 2014-08-13 17:38:02.582672 7fe9bae887a0 20 read_log 194215'148 (0'0) modify c746816a/benchmark_data_storage0111-ib.pocstor_48865_object8787/head//2 by client.2191635.0:8788 2014-08-12 17:15:33.351242 2014-08-13 17:38:02.582696 7fe9bae887a0 20 read_log 194215'149 (0'0) modify 3a4a1e6a/benchmark_data_storage0111-ib.pocstor_48865_object8947/head//2 by client.2191635.0:8948 2014-08-12 17:15:33.930406 2014-08-13 17:38:02.582719 7fe9bae887a0 20 read_log 194215'150 (0'0) modify bc379f6a/benchmark_data_storage0111-ib.pocstor_48865_object9016/head//2 by client.2191635.0:9017 2014-08-12 17:15:34.181016 2014-08-13 17:38:02.582740 7fe9bae887a0 20 read_log 194215'151 (0'0) modify cb6d2d6a/benchmark_data_storage0111-ib.pocstor_48865_object9125/head//2 by client.2191635.0:9126 2014-08-12 17:15:34.576941 2014-08-13 17:38:02.582761 7fe9bae887a0 20 read_log 194215'152 (0'0) modify 1ef336ea/benchmark_data_storage0111-ib.pocstor_48865_object9155/head//2 by client.2191635.0:9156 2014-08-12 17:15:34.687613 2014-08-13 17:38:02.582782 7fe9bae887a0 20 read_log 194215'153 (0'0) modify 7bc5afea/benchmark_data_storage0111-ib.pocstor_48865_object9205/head//2 by client.2191635.0:9206 2014-08-12 17:15:34.868034 2014-08-13 17:38:02.582802 7fe9bae887a0 20 read_log 194215'154 (0'0) modify 4d0f22ea/benchmark_data_storage0111-ib.pocstor_48865_object9210/head//2 by client.2191635.0:9211 2014-08-12 17:15:34.885872 2014-08-13 17:38:02.582835 7fe9bae887a0 20 read_log 194215'155 (0'0) modify 33b536ea/benchmark_data_storage0111-ib.pocstor_48865_object9217/head//2 by client.2191635.0:9218 2014-08-12 17:15:34.912468 2014-08-13 17:38:02.582857 7fe9bae887a0 20 read_log 194215'156 (0'0) modify c09e996a/benchmark_data_storage0111-ib.pocstor_48865_object9321/head//2 by client.2191635.0:9322 2014-08-12 17:15:35.289806 2014-08-13 17:38:02.582878 7fe9bae887a0 20 read_log 194215'157 (0'0) modify 884b2aa/benchmark_data_storage0111-ib.pocstor_48865_object9349/head//2 by client.2191635.0:9350 2014-08-12 17:15:35.428529 2014-08-13 17:38:02.582899 7fe9bae887a0 20 read_log 194215'158 (0'0) modify 7bea2ea/benchmark_data_storage0111-ib.pocstor_48865_object9369/head//2 by client.2191635.0:9370 2014-08-12 17:15:35.528811 2014-08-13 17:38:02.582920 7fe9bae887a0 20 read_log 194215'159 (0'0) modify d092c3ea/benchmark_data_storage0111-ib.pocstor_48865_object9464/head//2 by client.2191635.0:9465 2014-08-12 17:15:35.887006 2014-08-13 17:38:02.582940 7fe9bae887a0 20 read_log 194215'160 (0'0) modify f2d72e2a/benchmark_data_storage0111-ib.pocstor_48865_object9497/head//2 by client.2191635.0:9498 2014-08-12 17:15:36.006585 2014-08-13 17:38:02.582961 7fe9bae887a0 20 read_log 194215'161 (0'0) modify 7ed5766a/benchmark_data_storage0111-ib.pocstor_48865_object9512/head//2 by client.2191635.0:9513 2014-08-12 17:15:36.059959 2014-08-13 17:38:02.582982 7fe9bae887a0 20 read_log 194215'162 (0'0) modify 3108d22a/benchmark_data_storage0111-ib.pocstor_48865_object9561/head//2 by client.2191635.0:9562 2014-08-12 17:15:36.237341 2014-08-13 17:38:02.583003 7fe9bae887a0 20 read_log 194215'163 (0'0) modify 55a7182a/benchmark_data_storage0111-ib.pocstor_48865_object9618/head//2 by client.2191635.0:9619 2014-08-12 17:15:36.443842 2014-08-13 17:38:02.583023 7fe9bae887a0 20 read_log 194215'164 (0'0) modify dfc811ea/benchmark_data_storage0111-ib.pocstor_48865_object9687/head//2 by client.2191635.0:9688 2014-08-12 17:15:36.785253 2014-08-13 17:38:02.583044 7fe9bae887a0 20 read_log 194215'165 (0'0) modify 9b7a8baa/benchmark_data_storage0111-ib.pocstor_48865_object9869/head//2 by client.2191635.0:9870 2014-08-12 17:15:37.706117 2014-08-13 17:38:02.583064 7fe9bae887a0 20 read_log 194215'166 (0'0) modify 2b1b79ea/benchmark_data_storage0111-ib.pocstor_48865_object9880/head//2 by client.2191635.0:9881 2014-08-12 17:15:37.759971 2014-08-13 17:38:02.583085 7fe9bae887a0 20 read_log 194215'167 (0'0) modify 21a23aea/benchmark_data_storage0111-ib.pocstor_48865_object9933/head//2 by client.2191635.0:9934 2014-08-12 17:15:38.050817 2014-08-13 17:38:02.583106 7fe9bae887a0 20 read_log 194215'168 (0'0) modify b5c1d3aa/benchmark_data_storage0111-ib.pocstor_48865_object10008/head//2 by client.2191635.0:10009 2014-08-12 17:15:38.403669 2014-08-13 17:38:02.583127 7fe9bae887a0 20 read_log 194215'169 (0'0) modify 87325caa/benchmark_data_storage0111-ib.pocstor_48865_object10072/head//2 by client.2191635.0:10073 2014-08-12 17:15:38.767291 2014-08-13 17:38:02.583147 7fe9bae887a0 20 read_log 194215'170 (0'0) modify a5bef9aa/benchmark_data_storage0111-ib.pocstor_48865_object10123/head//2 by client.2191635.0:10124 2014-08-12 17:15:38.950737 2014-08-13 17:38:02.583178 7fe9bae887a0 20 read_log 194215'171 (0'0) modify 839ef8aa/benchmark_data_storage0111-ib.pocstor_48865_object10189/head//2 by client.2191635.0:10190 2014-08-12 17:15:39.206925 2014-08-13 17:38:02.583199 7fe9bae887a0 20 read_log 194215'172 (0'0) modify a72c32aa/benchmark_data_storage0111-ib.pocstor_48865_object10248/head//2 by client.2191635.0:10249 2014-08-12 17:15:39.434879 2014-08-13 17:38:02.583220 7fe9bae887a0 20 read_log 194215'173 (0'0) modify 3759752a/benchmark_data_storage0111-ib.pocstor_48865_object10253/head//2 by client.2191635.0:10254 2014-08-12 17:15:39.453378 2014-08-13 17:38:02.583241 7fe9bae887a0 20 read_log 194215'174 (0'0) modify 83b5eaa/benchmark_data_storage0111-ib.pocstor_48865_object10313/head//2 by client.2191635.0:10314 2014-08-12 17:15:39.671946 2014-08-13 17:38:02.583277 7fe9bae887a0 20 read_log 194215'175 (0'0) modify 6a9613aa/benchmark_data_storage0111-ib.pocstor_48865_object10396/head//2 by client.2191635.0:10397 2014-08-12 17:15:40.014965 2014-08-13 17:38:02.583299 7fe9bae887a0 20 read_log 194215'176 (0'0) modify b37aadaa/benchmark_data_storage0111-ib.pocstor_48865_object10453/head//2 by client.2191635.0:10454 2014-08-12 17:15:40.227327 2014-08-13 17:38:02.583320 7fe9bae887a0 20 read_log 194215'177 (0'0) modify 292338aa/benchmark_data_storage0111-ib.pocstor_48865_object10482/head//2 by client.2191635.0:10483 2014-08-12 17:15:40.333410 2014-08-13 17:38:02.583341 7fe9bae887a0 20 read_log 194215'178 (0'0) modify cb33af2a/benchmark_data_storage0111-ib.pocstor_48865_object10644/head//2 by client.2191635.0:10645 2014-08-12 17:15:40.970800 2014-08-13 17:38:02.583362 7fe9bae887a0 20 read_log 194215'179 (0'0) modify 1d50792a/benchmark_data_storage0111-ib.pocstor_48865_object10690/head//2 by client.2191635.0:10691 2014-08-12 17:15:41.224375 2014-08-13 17:38:02.583383 7fe9bae887a0 20 read_log 194215'180 (0'0) modify f9ca0fea/benchmark_data_storage0111-ib.pocstor_48865_object10730/head//2 by client.2191635.0:10731 2014-08-12 17:15:41.397617 2014-08-13 17:38:02.583406 7fe9bae887a0 20 read_log 194215'181 (0'0) modify a596172a/benchmark_data_storage0111-ib.pocstor_48865_object10814/head//2 by client.2191635.0:10815 2014-08-12 17:15:41.855059 2014-08-13 17:38:02.583427 7fe9bae887a0 20 read_log 194215'182 (0'0) modify 2e3f30aa/benchmark_data_storage0111-ib.pocstor_48865_object10886/head//2 by client.2191635.0:10887 2014-08-12 17:15:42.256517 2014-08-13 17:38:02.583447 7fe9bae887a0 20 read_log 194215'183 (0'0) modify 3359f32a/benchmark_data_storage0111-ib.pocstor_48865_object10910/head//2 by client.2191635.0:10911 2014-08-12 17:15:42.388701 2014-08-13 17:38:02.583468 7fe9bae887a0 20 read_log 194215'184 (0'0) modify d5ffa2a/benchmark_data_storage0111-ib.pocstor_48865_object10933/head//2 by client.2191635.0:10934 2014-08-12 17:15:42.494485 2014-08-13 17:38:02.583489 7fe9bae887a0 20 read_log 194215'185 (0'0) modify 5db5596a/benchmark_data_storage0111-ib.pocstor_48865_object10936/head//2 by client.2191635.0:10937 2014-08-12 17:15:42.505098 2014-08-13 17:38:02.583509 7fe9bae887a0 20 read_log 194215'186 (0'0) modify aa36ecaa/benchmark_data_storage0111-ib.pocstor_48865_object10979/head//2 by client.2191635.0:10980 2014-08-12 17:15:42.658928 2014-08-13 17:38:02.583530 7fe9bae887a0 20 read_log 194215'187 (0'0) modify 7039232a/benchmark_data_storage0111-ib.pocstor_48865_object11094/head//2 by client.2191635.0:11095 2014-08-12 17:15:43.109776 2014-08-13 17:38:02.583550 7fe9bae887a0 20 read_log 194215'188 (0'0) modify 5b3c0d2a/benchmark_data_storage0111-ib.pocstor_48865_object11111/head//2 by client.2191635.0:11112 2014-08-12 17:15:43.190326 2014-08-13 17:38:02.583571 7fe9bae887a0 20 read_log 194215'189 (0'0) modify 3f026e2a/benchmark_data_storage0111-ib.pocstor_48865_object11117/head//2 by client.2191635.0:11118 2014-08-12 17:15:43.227487 2014-08-13 17:38:02.583595 7fe9bae887a0 20 read_log 194215'190 (0'0) modify 6072102a/benchmark_data_storage0111-ib.pocstor_48865_object11176/head//2 by client.2191635.0:11177 2014-08-12 17:15:43.593006 2014-08-13 17:38:02.583615 7fe9bae887a0 20 read_log 194215'191 (0'0) modify b6ed77aa/benchmark_data_storage0111-ib.pocstor_48865_object11187/head//2 by client.2191635.0:11188 2014-08-12 17:15:43.632697 2014-08-13 17:38:02.583638 7fe9bae887a0 20 read_log 194215'192 (0'0) modify 205c17aa/benchmark_data_storage0111-ib.pocstor_48865_object11236/head//2 by client.2191635.0:11237 2014-08-12 17:15:43.817950 2014-08-13 17:38:02.583659 7fe9bae887a0 20 read_log 194215'193 (0'0) modify 8d426cea/benchmark_data_storage0111-ib.pocstor_48865_object11349/head//2 by client.2191635.0:11350 2014-08-12 17:15:44.242918 2014-08-13 17:38:02.583682 7fe9bae887a0 20 read_log 194215'194 (0'0) modify d297416a/benchmark_data_storage0111-ib.pocstor_48865_object11397/head//2 by client.2191635.0:11398 2014-08-12 17:15:44.413801 2014-08-13 17:38:02.583716 7fe9bae887a0 20 read_log 194215'195 (0'0) modify d6cefa2a/benchmark_data_storage0111-ib.pocstor_48865_object11498/head//2 by client.2191635.0:11499 2014-08-12 17:15:44.781522 2014-08-13 17:38:02.583738 7fe9bae887a0 20 read_log 194215'196 (0'0) modify 53e842ea/benchmark_data_storage0111-ib.pocstor_48865_object11552/head//2 by client.2191635.0:11553 2014-08-12 17:15:44.985019 2014-08-13 17:38:02.583759 7fe9bae887a0 20 read_log 194215'197 (0'0) modify 6030336a/benchmark_data_storage0111-ib.pocstor_48865_object11556/head//2 by client.2191635.0:11557 2014-08-12 17:15:45.024211 2014-08-13 17:38:02.583782 7fe9bae887a0 20 read_log 194215'198 (0'0) modify 89d46bea/benchmark_data_storage0111-ib.pocstor_48865_object11776/head//2 by client.2191635.0:11777 2014-08-12 17:15:46.004313 2014-08-13 17:38:02.583803 7fe9bae887a0 20 read_log 194215'199 (0'0) modify 35ffcaa/benchmark_data_storage0111-ib.pocstor_48865_object11861/head//2 by client.2191635.0:11862 2014-08-12 17:15:46.336605 2014-08-13 17:38:02.583824 7fe9bae887a0 20 read_log 194215'200 (0'0) modify 5b650a2a/benchmark_data_storage0111-ib.pocstor_48865_object11920/head//2 by client.2191635.0:11921 2014-08-12 17:15:46.549607 2014-08-13 17:38:02.583845 7fe9bae887a0 20 read_log 194215'201 (0'0) modify 4a1b016a/benchmark_data_storage0111-ib.pocstor_48865_object12014/head//2 by client.2191635.0:12015 2014-08-12 17:15:46.886851 2014-08-13 17:38:02.583865 7fe9bae887a0 20 read_log 194215'202 (0'0) modify 6c2c76aa/benchmark_data_storage0111-ib.pocstor_48865_object12017/head//2 by client.2191635.0:12018 2014-08-12 17:15:46.897357 2014-08-13 17:38:02.583886 7fe9bae887a0 20 read_log 194215'203 (0'0) modify b1b052a/benchmark_data_storage0111-ib.pocstor_48865_object12039/head//2 by client.2191635.0:12040 2014-08-12 17:15:46.975486 2014-08-13 17:38:02.583907 7fe9bae887a0 20 read_log 194215'204 (0'0) modify 18d92c2a/benchmark_data_storage0111-ib.pocstor_48865_object12137/head//2 by client.2191635.0:12138 2014-08-12 17:15:47.326641 2014-08-13 17:38:02.583927 7fe9bae887a0 20 read_log 194215'205 (0'0) modify e293eb6a/benchmark_data_storage0111-ib.pocstor_48865_object12200/head//2 by client.2191635.0:12201 2014-08-12 17:15:47.552061 2014-08-13 17:38:02.583947 7fe9bae887a0 20 read_log 194215'206 (0'0) modify aec82caa/benchmark_data_storage0111-ib.pocstor_48865_object12205/head//2 by client.2191635.0:12206 2014-08-12 17:15:47.569872 2014-08-13 17:38:02.583968 7fe9bae887a0 20 read_log 194215'207 (0'0) modify e4c381ea/benchmark_data_storage0111-ib.pocstor_48865_object12239/head//2 by client.2191635.0:12240 2014-08-12 17:15:47.691485 2014-08-13 17:38:02.583988 7fe9bae887a0 20 read_log 194215'208 (0'0) modify 5597feaa/benchmark_data_storage0111-ib.pocstor_48865_object12386/head//2 by client.2191635.0:12387 2014-08-12 17:15:48.391108 2014-08-13 17:38:02.584009 7fe9bae887a0 20 read_log 194215'209 (0'0) modify 43f2212a/benchmark_data_storage0111-ib.pocstor_48865_object12393/head//2 by client.2191635.0:12394 2014-08-12 17:15:48.416114 2014-08-13 17:38:02.584030 7fe9bae887a0 20 read_log 194215'210 (0'0) modify 2b5bb7ea/benchmark_data_storage0111-ib.pocstor_48865_object12418/head//2 by client.2191635.0:12419 2014-08-12 17:15:48.546188 2014-08-13 17:38:02.584050 7fe9bae887a0 20 read_log 194215'211 (0'0) modify b49f39aa/benchmark_data_storage0111-ib.pocstor_48865_object12547/head//2 by client.2191635.0:12548 2014-08-12 17:15:49.977299 2014-08-13 17:38:02.584071 7fe9bae887a0 20 read_log 194215'212 (0'0) modify affb68aa/benchmark_data_storage0111-ib.pocstor_48865_object12599/head//2 by client.2191635.0:12600 2014-08-12 17:15:50.219212 2014-08-13 17:38:02.584093 7fe9bae887a0 20 read_log 194215'213 (0'0) modify 2e68e72a/benchmark_data_storage0111-ib.pocstor_48865_object12623/head//2 by client.2191635.0:12624 2014-08-12 17:15:50.321994 2014-08-13 17:38:02.584114 7fe9bae887a0 20 read_log 194215'214 (0'0) modify b033de6a/benchmark_data_storage0111-ib.pocstor_48865_object12628/head//2 by client.2191635.0:12629 2014-08-12 17:15:50.342151 2014-08-13 17:38:02.584191 7fe9bae887a0 20 read_log 194215'215 (0'0) modify ae98b7ea/benchmark_data_storage0111-ib.pocstor_48865_object12659/head//2 by client.2191635.0:12660 2014-08-12 17:15:50.476108 2014-08-13 17:38:02.584216 7fe9bae887a0 20 read_log 194215'216 (0'0) modify 2aaf702a/benchmark_data_storage0111-ib.pocstor_48865_object12668/head//2 by client.2191635.0:12669 2014-08-12 17:15:50.515635 2014-08-13 17:38:02.584237 7fe9bae887a0 20 read_log 194215'217 (0'0) modify f57663aa/benchmark_data_storage0111-ib.pocstor_48865_object12750/head//2 by client.2191635.0:12751 2014-08-12 17:15:50.854116 2014-08-13 17:38:02.584258 7fe9bae887a0 20 read_log 194215'218 (0'0) modify f4212caa/benchmark_data_storage0111-ib.pocstor_48865_object12838/head//2 by client.2191635.0:12839 2014-08-12 17:15:51.171945 2014-08-13 17:38:02.584279 7fe9bae887a0 20 read_log 194215'219 (0'0) modify 7240e7aa/benchmark_data_storage0111-ib.pocstor_48865_object12999/head//2 by client.2191635.0:13000 2014-08-12 17:15:51.757431 2014-08-13 17:38:02.584300 7fe9bae887a0 20 read_log 194215'220 (0'0) modify 667d846a/benchmark_data_storage0111-ib.pocstor_48865_object13090/head//2 by client.2191635.0:13091 2014-08-12 17:15:52.102955 2014-08-13 17:38:02.584324 7fe9bae887a0 20 read_log 194215'221 (0'0) modify 4728a26a/benchmark_data_storage0111-ib.pocstor_48865_object13104/head//2 by client.2191635.0:13105 2014-08-12 17:15:52.177461 2014-08-13 17:38:02.584344 7fe9bae887a0 20 read_log 194215'222 (0'0) modify 6830202a/benchmark_data_storage0111-ib.pocstor_48865_object13121/head//2 by client.2191635.0:13122 2014-08-12 17:15:52.255868 2014-08-13 17:38:02.584365 7fe9bae887a0 20 read_log 194215'223 (0'0) modify e698f6ea/benchmark_data_storage0111-ib.pocstor_48865_object13189/head//2 by client.2191635.0:13190 2014-08-12 17:15:52.612221 2014-08-13 17:38:02.584386 7fe9bae887a0 20 read_log 194215'224 (0'0) modify 1d7d93ea/benchmark_data_storage0111-ib.pocstor_48865_object13262/head//2 by client.2191635.0:13263 2014-08-12 17:15:52.877192 2014-08-13 17:38:02.584407 7fe9bae887a0 20 read_log 194215'225 (0'0) modify 4a9e062a/benchmark_data_storage0111-ib.pocstor_48865_object13318/head//2 by client.2191635.0:13319 2014-08-12 17:15:53.079486 2014-08-13 17:38:02.584427 7fe9bae887a0 20 read_log 194215'226 (0'0) modify 71b2d72a/benchmark_data_storage0111-ib.pocstor_48865_object13371/head//2 by client.2191635.0:13372 2014-08-12 17:15:53.272076 2014-08-13 17:38:02.584448 7fe9bae887a0 20 read_log 194215'227 (0'0) modify 4abb7a2a/benchmark_data_storage0111-ib.pocstor_48865_object13374/head//2 by client.2191635.0:13375 2014-08-12 17:15:53.282562 2014-08-13 17:38:02.584468 7fe9bae887a0 20 read_log 194215'228 (0'0) modify 8e43f9aa/benchmark_data_storage0111-ib.pocstor_48865_object13392/head//2 by client.2191635.0:13393 2014-08-12 17:15:53.349656 2014-08-13 17:38:02.584489 7fe9bae887a0 20 read_log 194215'229 (0'0) modify 4f37bbea/benchmark_data_storage0111-ib.pocstor_48865_object13561/head//2 by client.2191635.0:13562 2014-08-12 17:15:53.962136 2014-08-13 17:38:02.584509 7fe9bae887a0 20 read_log 194215'230 (0'0) modify 19487bea/benchmark_data_storage0111-ib.pocstor_48865_object13635/head//2 by client.2191635.0:13636 2014-08-12 17:15:54.231884 2014-08-13 17:38:02.584530 7fe9bae887a0 20 read_log 194215'231 (0'0) modify dd220cea/benchmark_data_storage0111-ib.pocstor_48865_object13725/head//2 by client.2191635.0:13726 2014-08-12 17:15:54.554549 2014-08-13 17:38:02.584551 7fe9bae887a0 20 read_log 194215'232 (0'0) modify 9440cd2a/benchmark_data_storage0111-ib.pocstor_48865_object13735/head//2 by client.2191635.0:13736 2014-08-12 17:15:54.590517 2014-08-13 17:38:02.584572 7fe9bae887a0 20 read_log 194215'233 (0'0) modify d720b2a/benchmark_data_storage0111-ib.pocstor_48865_object13783/head//2 by client.2191635.0:13784 2014-08-12 17:15:54.763533 2014-08-13 17:38:02.584592 7fe9bae887a0 20 read_log 194215'234 (0'0) modify c9345c2a/benchmark_data_storage0111-ib.pocstor_48865_object13852/head//2 by client.2191635.0:13853 2014-08-12 17:15:55.053480 2014-08-13 17:38:02.584627 7fe9bae887a0 20 read_log 194215'235 (0'0) modify ea0b76a/benchmark_data_storage0111-ib.pocstor_48865_object13973/head//2 by client.2191635.0:13974 2014-08-12 17:15:55.567926 2014-08-13 17:38:02.584650 7fe9bae887a0 20 read_log 194215'236 (0'0) modify 2a0c802a/benchmark_data_storage0111-ib.pocstor_48865_object14145/head//2 by client.2191635.0:14146 2014-08-12 17:15:56.291566 2014-08-13 17:38:02.584671 7fe9bae887a0 20 read_log 194215'237 (0'0) modify 63208a6a/benchmark_data_storage0111-ib.pocstor_48865_object14286/head//2 by client.2191635.0:14287 2014-08-12 17:15:56.800077 2014-08-13 17:38:02.584723 7fe9bae887a0 20 read_log 194215'238 (0'0) modify 40f6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object14317/head//2 by client.2191635.0:14318 2014-08-12 17:15:56.912607 2014-08-13 17:38:02.584745 7fe9bae887a0 20 read_log 194215'239 (0'0) modify 41de21ea/benchmark_data_storage0111-ib.pocstor_48865_object14598/head//2 by client.2191635.0:14599 2014-08-12 17:15:58.065932 2014-08-13 17:38:02.584766 7fe9bae887a0 20 read_log 194215'240 (0'0) modify b8a8542a/benchmark_data_storage0111-ib.pocstor_48865_object14672/head//2 by client.2191635.0:14673 2014-08-12 17:15:58.333255 2014-08-13 17:38:02.584787 7fe9bae887a0 20 read_log 194215'241 (0'0) modify 4a6306aa/benchmark_data_storage0111-ib.pocstor_48865_object14713/head//2 by client.2191635.0:14714 2014-08-12 17:15:58.483677 2014-08-13 17:38:02.584807 7fe9bae887a0 20 read_log 194215'242 (0'0) modify 2991aaaa/benchmark_data_storage0111-ib.pocstor_48865_object14741/head//2 by client.2191635.0:14742 2014-08-12 17:15:58.584487 2014-08-13 17:38:02.584828 7fe9bae887a0 20 read_log 194215'243 (0'0) modify 565f662a/benchmark_data_storage0111-ib.pocstor_48865_object14767/head//2 by client.2191635.0:14768 2014-08-12 17:15:58.679345 2014-08-13 17:38:02.584849 7fe9bae887a0 20 read_log 194215'244 (0'0) modify 5192f8aa/benchmark_data_storage0111-ib.pocstor_48865_object14776/head//2 by client.2191635.0:14777 2014-08-12 17:15:58.712169 2014-08-13 17:38:02.584869 7fe9bae887a0 20 read_log 194215'245 (0'0) modify eeea566a/benchmark_data_storage0111-ib.pocstor_48865_object14778/head//2 by client.2191635.0:14779 2014-08-12 17:15:58.719188 2014-08-13 17:38:02.584890 7fe9bae887a0 20 read_log 194215'246 (0'0) modify 8558d5aa/benchmark_data_storage0111-ib.pocstor_48865_object14801/head//2 by client.2191635.0:14802 2014-08-12 17:15:58.818408 2014-08-13 17:38:02.584911 7fe9bae887a0 20 read_log 194215'247 (0'0) modify 9d0eaaea/benchmark_data_storage0111-ib.pocstor_48865_object14866/head//2 by client.2191635.0:14867 2014-08-12 17:15:59.065600 2014-08-13 17:38:02.584932 7fe9bae887a0 20 read_log 194215'248 (0'0) modify 305b2a6a/benchmark_data_storage0111-ib.pocstor_48865_object14950/head//2 by client.2191635.0:14951 2014-08-12 17:15:59.370240 2014-08-13 17:38:02.584952 7fe9bae887a0 20 read_log 194215'249 (0'0) modify 731b0e6a/benchmark_data_storage0111-ib.pocstor_48865_object15136/head//2 by client.2191635.0:15137 2014-08-12 17:16:00.043103 2014-08-13 17:38:02.584975 7fe9bae887a0 20 read_log 194215'250 (0'0) modify 65f3ae6a/benchmark_data_storage0111-ib.pocstor_48865_object15161/head//2 by client.2191635.0:15162 2014-08-12 17:16:00.132222 2014-08-13 17:38:02.584996 7fe9bae887a0 20 read_log 194215'251 (0'0) modify 77250b2a/benchmark_data_storage0111-ib.pocstor_48865_object15203/head//2 by client.2191635.0:15204 2014-08-12 17:16:00.285023 2014-08-13 17:38:02.585017 7fe9bae887a0 20 read_log 194215'252 (0'0) modify daba3e6a/benchmark_data_storage0111-ib.pocstor_48865_object15306/head//2 by client.2191635.0:15307 2014-08-12 17:16:00.665089 2014-08-13 17:38:02.585038 7fe9bae887a0 20 read_log 194215'253 (0'0) modify 8338bcea/benchmark_data_storage0111-ib.pocstor_48865_object15415/head//2 by client.2191635.0:15416 2014-08-12 17:16:01.090982 2014-08-13 17:38:02.585059 7fe9bae887a0 20 read_log 194215'254 (0'0) modify 79bf0b6a/benchmark_data_storage0111-ib.pocstor_48865_object15441/head//2 by client.2191635.0:15442 2014-08-12 17:16:01.217189 2014-08-13 17:38:02.585091 7fe9bae887a0 20 read_log 194215'255 (0'0) modify 22e21cea/benchmark_data_storage0111-ib.pocstor_48865_object15451/head//2 by client.2191635.0:15452 2014-08-12 17:16:01.262365 2014-08-13 17:38:02.585113 7fe9bae887a0 20 read_log 194215'256 (0'0) modify 722c47ea/benchmark_data_storage0111-ib.pocstor_48865_object15566/head//2 by client.2191635.0:15567 2014-08-12 17:16:01.962561 2014-08-13 17:38:02.585134 7fe9bae887a0 20 read_log 194215'257 (0'0) modify a865f92a/benchmark_data_storage0111-ib.pocstor_48865_object15671/head//2 by client.2191635.0:15672 2014-08-12 17:16:02.535524 2014-08-13 17:38:02.585161 7fe9bae887a0 20 read_log 194215'258 (0'0) modify eb9d4f6a/benchmark_data_storage0111-ib.pocstor_48865_object15686/head//2 by client.2191635.0:15687 2014-08-12 17:16:02.614819 2014-08-13 17:38:02.585183 7fe9bae887a0 20 read_log 194215'259 (0'0) modify bfafc2a/benchmark_data_storage0111-ib.pocstor_48865_object15765/head//2 by client.2191635.0:15766 2014-08-12 17:16:02.989209 2014-08-13 17:38:02.585205 7fe9bae887a0 20 read_log 194215'260 (0'0) modify c4918aea/benchmark_data_storage0111-ib.pocstor_48865_object15774/head//2 by client.2191635.0:15775 2014-08-12 17:16:03.026942 2014-08-13 17:38:02.585226 7fe9bae887a0 20 read_log 194215'261 (0'0) modify 577cf22a/benchmark_data_storage0111-ib.pocstor_48865_object15886/head//2 by client.2191635.0:15887 2014-08-12 17:16:03.434523 2014-08-13 17:38:02.585247 7fe9bae887a0 20 read_log 194215'262 (0'0) modify 47b95a6a/benchmark_data_storage0111-ib.pocstor_48865_object15917/head//2 by client.2191635.0:15918 2014-08-12 17:16:03.549837 2014-08-13 17:38:02.585267 7fe9bae887a0 20 read_log 194215'263 (0'0) modify d8a6b6ea/benchmark_data_storage0111-ib.pocstor_48865_object15948/head//2 by client.2191635.0:15949 2014-08-12 17:16:03.661576 2014-08-13 17:38:02.585288 7fe9bae887a0 20 read_log 194215'264 (0'0) modify b36d1d6a/benchmark_data_storage0111-ib.pocstor_48865_object15982/head//2 by client.2191635.0:15983 2014-08-12 17:16:03.787983 2014-08-13 17:38:02.585308 7fe9bae887a0 20 read_log 194215'265 (0'0) modify 45f476ea/benchmark_data_storage0111-ib.pocstor_48865_object16063/head//2 by client.2191635.0:16064 2014-08-12 17:16:04.080351 2014-08-13 17:38:02.585329 7fe9bae887a0 20 read_log 194215'266 (0'0) modify c3d0156a/benchmark_data_storage0111-ib.pocstor_48865_object16141/head//2 by client.2191635.0:16142 2014-08-12 17:16:04.366973 2014-08-13 17:38:02.585350 7fe9bae887a0 20 read_log 194215'267 (0'0) modify 81a3d32a/benchmark_data_storage0111-ib.pocstor_48865_object16150/head//2 by client.2191635.0:16151 2014-08-12 17:16:04.398979 2014-08-13 17:38:02.585370 7fe9bae887a0 20 read_log 194215'268 (0'0) modify 8c0d182a/benchmark_data_storage0111-ib.pocstor_48865_object16296/head//2 by client.2191635.0:16297 2014-08-12 17:16:04.926972 2014-08-13 17:38:02.585391 7fe9bae887a0 20 read_log 194215'269 (0'0) modify a521d6aa/benchmark_data_storage0111-ib.pocstor_48865_object16315/head//2 by client.2191635.0:16316 2014-08-12 17:16:04.997455 2014-08-13 17:38:02.585412 7fe9bae887a0 20 read_log 194215'270 (0'0) modify df2ea2ea/benchmark_data_storage0111-ib.pocstor_48865_object16352/head//2 by client.2191635.0:16353 2014-08-12 17:16:05.130873 2014-08-13 17:38:02.585433 7fe9bae887a0 20 read_log 194215'271 (0'0) modify 826376aa/benchmark_data_storage0111-ib.pocstor_48865_object16361/head//2 by client.2191635.0:16362 2014-08-12 17:16:05.162821 2014-08-13 17:38:02.585453 7fe9bae887a0 20 read_log 194215'272 (0'0) modify 6afdb52a/benchmark_data_storage0111-ib.pocstor_48865_object16427/head//2 by client.2191635.0:16428 2014-08-12 17:16:05.404967 2014-08-13 17:38:02.585476 7fe9bae887a0 20 read_log 194215'273 (0'0) modify bc7adeaa/benchmark_data_storage0111-ib.pocstor_48865_object16515/head//2 by client.2191635.0:16516 2014-08-12 17:16:05.724474 2014-08-13 17:38:02.585497 7fe9bae887a0 20 read_log 194215'274 (0'0) modify ccb6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object16564/head//2 by client.2191635.0:16565 2014-08-12 17:16:05.905227 2014-08-13 17:38:02.585529 7fe9bae887a0 20 read_log 194215'275 (0'0) modify 7f37d7aa/benchmark_data_storage0111-ib.pocstor_48865_object16565/head//2 by client.2191635.0:16566 2014-08-12 17:16:05.908721 2014-08-13 17:38:02.585582 7fe9bae887a0 20 read_log 194215'276 (0'0) modify 8cf4382a/benchmark_data_storage0111-ib.pocstor_48865_object16604/head//2 by client.2191635.0:16605 2014-08-12 17:16:06.046947 2014-08-13 17:38:02.585604 7fe9bae887a0 20 read_log 194215'277 (0'0) modify fbdbd5aa/benchmark_data_storage0111-ib.pocstor_48865_object16791/head//2 by client.2191635.0:16792 2014-08-12 17:16:06.727006 2014-08-13 17:38:02.585625 7fe9bae887a0 20 read_log 194215'278 (0'0) modify 46242aea/benchmark_data_storage0111-ib.pocstor_48865_object16862/head//2 by client.2191635.0:16863 2014-08-12 17:16:06.980908 2014-08-13 17:38:02.585645 7fe9bae887a0 20 read_log 194215'279 (0'0) modify 8e0b2d2a/benchmark_data_storage0111-ib.pocstor_48865_object16863/head//2 by client.2191635.0:16864 2014-08-12 17:16:06.984476 2014-08-13 17:38:02.585666 7fe9bae887a0 20 read_log 194215'280 (0'0) modify d8478eea/benchmark_data_storage0111-ib.pocstor_48865_object16870/head//2 by client.2191635.0:16871 2014-08-12 17:16:07.009231 2014-08-13 17:38:02.585687 7fe9bae887a0 20 read_log 194215'281 (0'0) modify 23a472a/benchmark_data_storage0111-ib.pocstor_48865_object16907/head//2 by client.2191635.0:16908 2014-08-12 17:16:07.145148 2014-08-13 17:38:02.585708 7fe9bae887a0 20 read_log 194215'282 (0'0) modify 861c596a/benchmark_data_storage0111-ib.pocstor_48865_object16946/head//2 by client.2191635.0:16947 2014-08-12 17:16:07.342214 2014-08-13 17:38:02.585729 7fe9bae887a0 20 read_log 194215'283 (0'0) modify 70e3cf2a/benchmark_data_storage0111-ib.pocstor_48865_object16999/head//2 by client.2191635.0:17000 2014-08-12 17:16:07.624824 2014-08-13 17:38:02.585749 7fe9bae887a0 20 read_log 194215'284 (0'0) modify 66eb7d2a/benchmark_data_storage0111-ib.pocstor_48865_object17022/head//2 by client.2191635.0:17023 2014-08-12 17:16:07.711065 2014-08-13 17:38:02.585770 7fe9bae887a0 20 read_log 194215'285 (0'0) modify a97b1aa/benchmark_data_storage0111-ib.pocstor_48865_object17041/head//2 by client.2191635.0:17042 2014-08-12 17:16:07.777491 2014-08-13 17:38:02.585791 7fe9bae887a0 20 read_log 194215'286 (0'0) modify 36b9c7ea/benchmark_data_storage0111-ib.pocstor_48865_object17067/head//2 by client.2191635.0:17068 2014-08-12 17:16:07.871923 2014-08-13 17:38:02.585811 7fe9bae887a0 20 read_log 194215'287 (0'0) modify d2246d2a/benchmark_data_storage0111-ib.pocstor_48865_object17159/head//2 by client.2191635.0:17160 2014-08-12 17:16:08.218770 2014-08-13 17:38:02.585832 7fe9bae887a0 20 read_log 194215'288 (0'0) modify 9e6f64ea/benchmark_data_storage0111-ib.pocstor_48865_object17166/head//2 by client.2191635.0:17167 2014-08-12 17:16:08.267875 2014-08-13 17:38:02.585853 7fe9bae887a0 20 read_log 194215'289 (0'0) modify 1ce7ab2a/benchmark_data_storage0111-ib.pocstor_48865_object17300/head//2 by client.2191635.0:17301 2014-08-12 17:16:09.093686 2014-08-13 17:38:02.585873 7fe9bae887a0 20 read_log 194215'290 (0'0) modify 93cde52a/benchmark_data_storage0111-ib.pocstor_48865_object17356/head//2 by client.2191635.0:17357 2014-08-12 17:16:09.435192 2014-08-13 17:38:02.585894 7fe9bae887a0 20 read_log 194215'291 (0'0) modify 6543b36a/benchmark_data_storage0111-ib.pocstor_48865_object17395/head//2 by client.2191635.0:17396 2014-08-12 17:16:09.657999 2014-08-13 17:38:02.585917 7fe9bae887a0 20 read_log 194215'292 (0'0) modify 907fe66a/benchmark_data_storage0111-ib.pocstor_48865_object17420/head//2 by client.2191635.0:17421 2014-08-12 17:16:09.764588 2014-08-13 17:38:02.585938 7fe9bae887a0 20 read_log 194215'293 (0'0) modify f04168aa/benchmark_data_storage0111-ib.pocstor_48865_object17493/head//2 by client.2191635.0:17494 2014-08-12 17:16:10.029089 2014-08-13 17:38:02.585959 7fe9bae887a0 20 read_log 194215'294 (0'0) modify 3dd28a2a/benchmark_data_storage0111-ib.pocstor_48865_object17533/head//2 by client.2191635.0:17534 2014-08-12 17:16:10.175906 2014-08-13 17:38:02.585994 7fe9bae887a0 20 read_log 194215'295 (0'0) modify 1e4657ea/benchmark_data_storage0111-ib.pocstor_48865_object17612/head//2 by client.2191635.0:17613 2014-08-12 17:16:10.608645 2014-08-13 17:38:02.586016 7fe9bae887a0 20 read_log 194215'296 (0'0) modify 6041b6ea/benchmark_data_storage0111-ib.pocstor_48865_object17635/head//2 by client.2191635.0:17636 2014-08-12 17:16:10.770191 2014-08-13 17:38:02.586038 7fe9bae887a0 20 read_log 194215'297 (0'0) modify 224e726a/benchmark_data_storage0111-ib.pocstor_48865_object17676/head//2 by client.2191635.0:17677 2014-08-12 17:16:11.018959 2014-08-13 17:38:02.586058 7fe9bae887a0 20 read_log 194215'298 (0'0) modify b888422a/benchmark_data_storage0111-ib.pocstor_48865_object17741/head//2 by client.2191635.0:17742 2014-08-12 17:16:11.332637 2014-08-13 17:38:02.586079 7fe9bae887a0 20 read_log 194215'299 (0'0) modify 6161082a/benchmark_data_storage0111-ib.pocstor_48865_object17851/head//2 by client.2191635.0:17852 2014-08-12 17:16:11.727213 2014-08-13 17:38:02.586100 7fe9bae887a0 20 read_log 194215'300 (0'0) modify f91dcc2a/benchmark_data_storage0111-ib.pocstor_48865_object18013/head//2 by client.2191635.0:18014 2014-08-12 17:16:12.405398 2014-08-13 17:38:02.586121 7fe9bae887a0 20 read_log 194215'301 (0'0) modify d714946a/benchmark_data_storage0111-ib.pocstor_48865_object18071/head//2 by client.2191635.0:18072 2014-08-12 17:16:12.711413 2014-08-13 17:38:02.586142 7fe9bae887a0 20 read_log 194215'302 (0'0) modify 76da69aa/benchmark_data_storage0111-ib.pocstor_48865_object18175/head//2 by client.2191635.0:18176 2014-08-12 17:16:13.088383 2014-08-13 17:38:02.586172 7fe9bae887a0 20 read_log 194215'303 (0'0) modify ca0bf7ea/benchmark_data_storage0111-ib.pocstor_48865_object18353/head//2 by client.2191635.0:18354 2014-08-12 17:16:13.731627 2014-08-13 17:38:02.586193 7fe9bae887a0 20 read_log 194215'304 (0'0) modify cb5fc6ea/benchmark_data_storage0111-ib.pocstor_48865_object18362/head//2 by client.2191635.0:18363 2014-08-12 17:16:13.763791 2014-08-13 17:38:02.586217 7fe9bae887a0 20 read_log 194215'305 (0'0) modify ba58eaaa/benchmark_data_storage0111-ib.pocstor_48865_object18438/head//2 by client.2191635.0:18439 2014-08-12 17:16:14.061156 2014-08-13 17:38:02.586238 7fe9bae887a0 20 read_log 194215'306 (0'0) modify 774fb76a/benchmark_data_storage0111-ib.pocstor_48865_object18602/head//2 by client.2191635.0:18603 2014-08-12 17:16:14.916162 2014-08-13 17:38:02.586258 7fe9bae887a0 20 read_log 194215'307 (0'0) modify 3f75e6a/benchmark_data_storage0111-ib.pocstor_48865_object18627/head//2 by client.2191635.0:18628 2014-08-12 17:16:15.017009 2014-08-13 17:38:02.586281 7fe9bae887a0 20 read_log 194215'308 (0'0) modify 76eaf2a/benchmark_data_storage0111-ib.pocstor_48865_object18644/head//2 by client.2191635.0:18645 2014-08-12 17:16:15.076769 2014-08-13 17:38:02.586302 7fe9bae887a0 20 read_log 194215'309 (0'0) modify 5bc40b6a/benchmark_data_storage0111-ib.pocstor_48865_object18652/head//2 by client.2191635.0:18653 2014-08-12 17:16:15.109627 2014-08-13 17:38:02.586325 7fe9bae887a0 20 read_log 194215'310 (0'0) modify 9c195bea/benchmark_data_storage0111-ib.pocstor_48865_object18750/head//2 by client.2191635.0:18751 2014-08-12 17:16:15.542111 2014-08-13 17:38:02.586346 7fe9bae887a0 20 read_log 194215'311 (0'0) modify bcd1ddea/benchmark_data_storage0111-ib.pocstor_48865_object18803/head//2 by client.2191635.0:18804 2014-08-12 17:16:15.953270 2014-08-13 17:38:02.586367 7fe9bae887a0 20 read_log 194215'312 (0'0) modify 57e7232a/benchmark_data_storage0111-ib.pocstor_48865_object18851/head//2 by client.2191635.0:18852 2014-08-12 17:16:17.205211 2014-08-13 17:38:02.586388 7fe9bae887a0 20 read_log 194215'313 (0'0) modify 3f8d12a/benchmark_data_storage0111-ib.pocstor_48865_object18898/head//2 by client.2191635.0:18899 2014-08-12 17:16:17.423178 2014-08-13 17:38:02.586408 7fe9bae887a0 20 read_log 194215'314 (0'0) modify eda4702a/benchmark_data_storage0111-ib.pocstor_48865_object18934/head//2 by client.2191635.0:18935 2014-08-12 17:16:17.557912 2014-08-13 17:38:02.586441 7fe9bae887a0 20 read_log 194215'315 (0'0) modify 8ad8c9ea/benchmark_data_storage0111-ib.pocstor_48865_object19074/head//2 by client.2191635.0:19075 2014-08-12 17:16:18.063779 2014-08-13 17:38:02.586464 7fe9bae887a0 20 read_log 194215'316 (0'0) modify 45feb6aa/benchmark_data_storage0111-ib.pocstor_48865_object19194/head//2 by client.2191635.0:19195 2014-08-12 17:16:18.851198 2014-08-13 17:38:02.586485 7fe9bae887a0 20 read_log 194215'317 (0'0) modify 67fe712a/benchmark_data_storage0111-ib.pocstor_48865_object19218/head//2 by client.2191635.0:19219 2014-08-12 17:16:18.937502 2014-08-13 17:38:02.586505 7fe9bae887a0 20 read_log 194215'318 (0'0) modify 87a9a2ea/benchmark_data_storage0111-ib.pocstor_48865_object19230/head//2 by client.2191635.0:19231 2014-08-12 17:16:18.979622 2014-08-13 17:38:02.586527 7fe9bae887a0 20 read_log 194215'319 (0'0) modify d4cfa56a/benchmark_data_storage0111-ib.pocstor_48865_object19259/head//2 by client.2191635.0:19260 2014-08-12 17:16:19.084466 2014-08-13 17:38:02.586548 7fe9bae887a0 20 read_log 194215'320 (0'0) modify 3d60586a/benchmark_data_storage0111-ib.pocstor_48865_object19315/head//2 by client.2191635.0:19316 2014-08-12 17:16:19.286260 2014-08-13 17:38:02.586569 7fe9bae887a0 20 read_log 194215'321 (0'0) modify a5cf01aa/benchmark_data_storage0111-ib.pocstor_48865_object19413/head//2 by client.2191635.0:19414 2014-08-12 17:16:19.736977 2014-08-13 17:38:02.586590 7fe9bae887a0 20 read_log 194215'322 (0'0) modify eac666ea/benchmark_data_storage0111-ib.pocstor_48865_object19418/head//2 by client.2191635.0:19419 2014-08-12 17:16:19.769140 2014-08-13 17:38:02.586611 7fe9bae887a0 20 read_log 194215'323 (0'0) modify 701512a/benchmark_data_storage0111-ib.pocstor_48865_object19425/head//2 by client.2191635.0:19426 2014-08-12 17:16:19.814572 2014-08-13 17:38:02.586631 7fe9bae887a0 20 read_log 194215'324 (0'0) modify 109f076a/benchmark_data_storage0111-ib.pocstor_48865_object19461/head//2 by client.2191635.0:19462 2014-08-12 17:16:20.028673 2014-08-13 17:38:02.586652 7fe9bae887a0 20 read_log 194215'325 (0'0) modify b65a122a/benchmark_data_storage0111-ib.pocstor_48865_object19517/head//2 by client.2191635.0:19518 2014-08-12 17:16:20.285607 2014-08-13 17:38:02.586673 7fe9bae887a0 20 read_log 194215'326 (0'0) modify bca3f02a/benchmark_data_storage0111-ib.pocstor_48865_object19526/head//2 by client.2191635.0:19527 2014-08-12 17:16:20.317334 2014-08-13 17:38:02.586694 7fe9bae887a0 20 read_log 194215'327 (0'0) modify cf2e0e2a/benchmark_data_storage0111-ib.pocstor_48865_object19566/head//2 by client.2191635.0:19567 2014-08-12 17:16:20.463822 2014-08-13 17:38:02.586714 7fe9bae887a0 20 read_log 194215'328 (0'0) modify 56fd8d6a/benchmark_data_storage0111-ib.pocstor_48865_object19582/head//2 by client.2191635.0:19583 2014-08-12 17:16:20.521305 2014-08-13 17:38:02.586735 7fe9bae887a0 20 read_log 194215'329 (0'0) modify bea29c2a/benchmark_data_storage0111-ib.pocstor_48865_object19690/head//2 by client.2191635.0:19691 2014-08-12 17:16:20.911949 2014-08-13 17:38:02.586757 7fe9bae887a0 20 read_log 194215'330 (0'0) modify 971f7a6a/benchmark_data_storage0111-ib.pocstor_48865_object19692/head//2 by client.2191635.0:19693 2014-08-12 17:16:20.918927 2014-08-13 17:38:02.586807 7fe9bae887a0 20 read_log 194215'331 (0'0) modify 9a63936a/benchmark_data_storage0111-ib.pocstor_48865_object19899/head//2 by client.2191635.0:19900 2014-08-12 17:16:21.762855 2014-08-13 17:38:02.586829 7fe9bae887a0 20 read_log 194215'332 (0'0) modify f7da2aea/benchmark_data_storage0111-ib.pocstor_48865_object19982/head//2 by client.2191635.0:19983 2014-08-12 17:16:22.225361 2014-08-13 17:38:02.586850 7fe9bae887a0 20 read_log 194215'333 (0'0) modify 5bb981aa/benchmark_data_storage0111-ib.pocstor_48865_object20036/head//2 by client.2191635.0:20037 2014-08-12 17:16:22.426859 2014-08-13 17:38:02.586871 7fe9bae887a0 20 read_log 194215'334 (0'0) modify cf11302a/benchmark_data_storage0111-ib.pocstor_48865_object20045/head//2 by client.2191635.0:20046 2014-08-12 17:16:22.458846 2014-08-13 17:38:02.586902 7fe9bae887a0 20 read_log 194215'335 (0'0) modify fe4a30aa/benchmark_data_storage0111-ib.pocstor_48865_object20057/head//2 by client.2191635.0:20058 2014-08-12 17:16:22.502342 2014-08-13 17:38:02.586924 7fe9bae887a0 20 read_log 194215'336 (0'0) modify 44997bea/benchmark_data_storage0111-ib.pocstor_48865_object20125/head//2 by client.2191635.0:20126 2014-08-12 17:16:22.789954 2014-08-13 17:38:02.586945 7fe9bae887a0 20 read_log 194215'337 (0'0) modify c8c401aa/benchmark_data_storage0111-ib.pocstor_48865_object20134/head//2 by client.2191635.0:20135 2014-08-12 17:16:22.845537 2014-08-13 17:38:02.586965 7fe9bae887a0 20 read_log 194215'338 (0'0) modify ae295caa/benchmark_data_storage0111-ib.pocstor_48865_object20201/head//2 by client.2191635.0:20202 2014-08-12 17:16:23.652760 2014-08-13 17:38:02.586986 7fe9bae887a0 20 read_log 194215'339 (0'0) modify f203046a/benchmark_data_storage0111-ib.pocstor_48865_object20204/head//2 by client.2191635.0:20205 2014-08-12 17:16:23.687471 2014-08-13 17:38:02.587007 7fe9bae887a0 20 read_log 194215'340 (0'0) modify de6e16aa/benchmark_data_storage0111-ib.pocstor_48865_object20326/head//2 by client.2191635.0:20327 2014-08-12 17:16:24.292247 2014-08-13 17:38:02.587027 7fe9bae887a0 20 read_log 194215'341 (0'0) modify e793e8aa/benchmark_data_storage0111-ib.pocstor_48865_object20420/head//2 by client.2191635.0:20421 2014-08-12 17:16:24.642523 2014-08-13 17:38:02.587048 7fe9bae887a0 20 read_log 194215'342 (0'0) modify 97c150ea/benchmark_data_storage0111-ib.pocstor_48865_object20422/head//2 by client.2191635.0:20423 2014-08-12 17:16:24.661076 2014-08-13 17:38:02.587068 7fe9bae887a0 20 read_log 194215'343 (0'0) modify c581916a/benchmark_data_storage0111-ib.pocstor_48865_object20680/head//2 by client.2191635.0:20681 2014-08-12 17:16:26.226605 2014-08-13 17:38:02.587089 7fe9bae887a0 20 read_log 194215'344 (0'0) modify 226308aa/benchmark_data_storage0111-ib.pocstor_48865_object20698/head//2 by client.2191635.0:20699 2014-08-12 17:16:26.290535 2014-08-13 17:38:02.587110 7fe9bae887a0 20 read_log 194215'345 (0'0) modify 217b9c2a/benchmark_data_storage0111-ib.pocstor_48865_object20701/head//2 by client.2191635.0:20702 2014-08-12 17:16:26.301012 2014-08-13 17:38:02.587131 7fe9bae887a0 20 read_log 194215'346 (0'0) modify d22aaeea/benchmark_data_storage0111-ib.pocstor_48865_object20724/head//2 by client.2191635.0:20725 2014-08-12 17:16:26.385467 2014-08-13 17:38:02.587152 7fe9bae887a0 20 read_log 194215'347 (0'0) modify 3056b9aa/benchmark_data_storage0111-ib.pocstor_48865_object20781/head//2 by client.2191635.0:20782 2014-08-12 17:16:26.588930 2014-08-13 17:38:02.587182 7fe9bae887a0 20 read_log 194215'348 (0'0) modify e128ee2a/benchmark_data_storage0111-ib.pocstor_48865_object20832/head//2 by client.2191635.0:20833 2014-08-12 17:16:26.780106 2014-08-13 17:38:02.587205 7fe9bae887a0 20 read_log 194215'349 (0'0) modify a037016a/benchmark_data_storage0111-ib.pocstor_48865_object21051/head//2 by client.2191635.0:21052 2014-08-12 17:16:27.760213 2014-08-13 17:38:02.587227 7fe9bae887a0 20 read_log 194215'350 (0'0) modify b10367ea/benchmark_data_storage0111-ib.pocstor_48865_object21068/head//2 by client.2191635.0:21069 2014-08-12 17:16:27.826846 2014-08-13 17:38:02.587248 7fe9bae887a0 20 read_log 194215'351 (0'0) modify d46985aa/benchmark_data_storage0111-ib.pocstor_48865_object21162/head//2 by client.2191635.0:21163 2014-08-12 17:16:28.737133 2014-08-13 17:38:02.587268 7fe9bae887a0 20 read_log 194215'352 (0'0) modify 971d17ea/benchmark_data_storage0111-ib.pocstor_48865_object21246/head//2 by client.2191635.0:21247 2014-08-12 17:16:29.113516 2014-08-13 17:38:02.587289 7fe9bae887a0 20 read_log 194215'353 (0'0) modify 41d7a72a/benchmark_data_storage0111-ib.pocstor_48865_object21255/head//2 by client.2191635.0:21256 2014-08-12 17:16:29.144956 2014-08-13 17:38:02.587310 7fe9bae887a0 20 read_log 194215'354 (0'0) modify 6adc306a/benchmark_data_storage0111-ib.pocstor_48865_object21329/head//2 by client.2191635.0:21330 2014-08-12 17:16:29.411652 2014-08-13 17:38:02.587372 7fe9bae887a0 20 read_log 194215'355 (0'0) modify 19802aaa/benchmark_data_storage0111-ib.pocstor_48865_object21368/head//2 by client.2191635.0:21369 2014-08-12 17:16:29.576418 2014-08-13 17:38:02.587396 7fe9bae887a0 20 read_log 194215'356 (0'0) modify 87842baa/benchmark_data_storage0111-ib.pocstor_48865_object21403/head//2 by client.2191635.0:21404 2014-08-12 17:16:29.801240 2014-08-13 17:38:02.587417 7fe9bae887a0 20 read_log 194215'357 (0'0) modify b59226a/benchmark_data_storage0111-ib.pocstor_48865_object21404/head//2 by client.2191635.0:21405 2014-08-12 17:16:29.812016 2014-08-13 17:38:02.587438 7fe9bae887a0 20 read_log 194215'358 (0'0) modify d68ecdaa/benchmark_data_storage0111-ib.pocstor_48865_object21520/head//2 by client.2191635.0:21521 2014-08-12 17:16:30.562160 2014-08-13 17:38:02.587461 7fe9bae887a0 20 read_log 194215'359 (0'0) modify 37caa06a/benchmark_data_storage0111-ib.pocstor_48865_object21568/head//2 by client.2191635.0:21569 2014-08-12 17:16:30.735149 2014-08-13 17:38:02.587482 7fe9bae887a0 20 read_log 194215'360 (0'0) modify 615151ea/benchmark_data_storage0111-ib.pocstor_48865_object21657/head//2 by client.2191635.0:21658 2014-08-12 17:16:31.062712 2014-08-13 17:38:02.587504 7fe9bae887a0 20 read_log 194215'361 (0'0) modify 7abdc1ea/benchmark_data_storage0111-ib.pocstor_48865_object21668/head//2 by client.2191635.0:21669 2014-08-12 17:16:31.102711 2014-08-13 17:38:02.587525 7fe9bae887a0 20 read_log 194215'362 (0'0) modify 97743f2a/benchmark_data_storage0111-ib.pocstor_48865_object21779/head//2 by client.2191635.0:21780 2014-08-12 17:16:31.541261 2014-08-13 17:38:02.587546 7fe9bae887a0 20 read_log 194215'363 (0'0) modify 605fde2a/benchmark_data_storage0111-ib.pocstor_48865_object21799/head//2 by client.2191635.0:21800 2014-08-12 17:16:31.622469 2014-08-13 17:38:02.587567 7fe9bae887a0 20 read_log 194215'364 (0'0) modify 7e36cb6a/benchmark_data_storage0111-ib.pocstor_48865_object21980/head//2 by client.2191635.0:21981 2014-08-12 17:16:32.463014 2014-08-13 17:38:02.587588 7fe9bae887a0 20 read_log 194215'365 (0'0) modify 31ca3faa/benchmark_data_storage0111-ib.pocstor_48865_object22026/head//2 by client.2191635.0:22027 2014-08-12 17:16:32.627850 2014-08-13 17:38:02.587610 7fe9bae887a0 20 read_log 194215'366 (0'0) modify b22ed1aa/benchmark_data_storage0111-ib.pocstor_48865_object22036/head//2 by client.2191635.0:22037 2014-08-12 17:16:32.663051 2014-08-13 17:38:02.587631 7fe9bae887a0 20 read_log 194215'367 (0'0) modify 9d60f46a/benchmark_data_storage0111-ib.pocstor_48865_object22243/head//2 by client.2191635.0:22244 2014-08-12 17:16:33.639658 2014-08-13 17:38:02.587651 7fe9bae887a0 20 read_log 194215'368 (0'0) modify d38544aa/benchmark_data_storage0111-ib.pocstor_48865_object22284/head//2 by client.2191635.0:22285 2014-08-12 17:16:33.815647 2014-08-13 17:38:02.587672 7fe9bae887a0 20 read_log 194215'369 (0'0) modify 4248b2ea/benchmark_data_storage0111-ib.pocstor_48865_object22314/head//2 by client.2191635.0:22315 2014-08-12 17:16:33.929669 2014-08-13 17:38:02.587693 7fe9bae887a0 20 read_log 194215'370 (0'0) modify 9d1470aa/benchmark_data_storage0111-ib.pocstor_48865_object22348/head//2 by client.2191635.0:22349 2014-08-12 17:16:34.050102 2014-08-13 17:38:02.587713 7fe9bae887a0 20 read_log 194215'371 (0'0) modify 8501b66a/benchmark_data_storage0111-ib.pocstor_48865_object22363/head//2 by client.2191635.0:22364 2014-08-12 17:16:34.107552 2014-08-13 17:38:02.587734 7fe9bae887a0 20 read_log 194215'372 (0'0) modify 4baa0dea/benchmark_data_storage0111-ib.pocstor_48865_object22414/head//2 by client.2191635.0:22415 2014-08-12 17:16:34.373672 2014-08-13 17:38:02.587755 7fe9bae887a0 20 read_log 194215'373 (0'0) modify c76fe86a/benchmark_data_storage0111-ib.pocstor_48865_object22497/head//2 by client.2191635.0:22498 2014-08-12 17:16:34.702994 2014-08-13 17:38:02.587775 7fe9bae887a0 20 read_log 194215'374 (0'0) modify 1378006a/benchmark_data_storage0111-ib.pocstor_48865_object22538/head//2 by client.2191635.0:22539 2014-08-12 17:16:34.853009 2014-08-13 17:38:02.587806 7fe9bae887a0 20 read_log 194215'375 (0'0) modify 51bf9b2a/benchmark_data_storage0111-ib.pocstor_48865_object22569/head//2 by client.2191635.0:22570 2014-08-12 17:16:34.962874 2014-08-13 17:38:02.587828 7fe9bae887a0 20 read_log 194215'376 (0'0) modify d91a75ea/benchmark_data_storage0111-ib.pocstor_48865_object22581/head//2 by client.2191635.0:22582 2014-08-12 17:16:35.005339 2014-08-13 17:38:02.587849 7fe9bae887a0 20 read_log 194215'377 (0'0) modify 877f52a/benchmark_data_storage0111-ib.pocstor_48865_object22590/head//2 by client.2191635.0:22591 2014-08-12 17:16:35.039981 2014-08-13 17:38:02.587870 7fe9bae887a0 20 read_log 194215'378 (0'0) modify 7772482a/benchmark_data_storage0111-ib.pocstor_48865_object22594/head//2 by client.2191635.0:22595 2014-08-12 17:16:35.055278 2014-08-13 17:38:02.587890 7fe9bae887a0 20 read_log 194215'379 (0'0) modify 4f9362a/benchmark_data_storage0111-ib.pocstor_48865_object22640/head//2 by client.2191635.0:22641 2014-08-12 17:16:35.250146 2014-08-13 17:38:02.587914 7fe9bae887a0 20 read_log 194215'380 (0'0) modify 5af1bdaa/benchmark_data_storage0111-ib.pocstor_48865_object22673/head//2 by client.2191635.0:22674 2014-08-12 17:16:35.399686 2014-08-13 17:38:02.587935 7fe9bae887a0 20 read_log 194215'381 (0'0) modify 8f9836aa/benchmark_data_storage0111-ib.pocstor_48865_object22690/head//2 by client.2191635.0:22691 2014-08-12 17:16:35.514386 2014-08-13 17:38:02.587955 7fe9bae887a0 20 read_log 194215'382 (0'0) modify 1e47b9ea/benchmark_data_storage0111-ib.pocstor_48865_object22731/head//2 by client.2191635.0:22732 2014-08-12 17:16:35.736001 2014-08-13 17:38:02.588008 7fe9bae887a0 20 read_log 194215'383 (0'0) modify dfa6716a/benchmark_data_storage0111-ib.pocstor_48865_object22765/head//2 by client.2191635.0:22766 2014-08-12 17:16:35.858332 2014-08-13 17:38:02.588030 7fe9bae887a0 20 read_log 194215'384 (0'0) modify 10a9a06a/benchmark_data_storage0111-ib.pocstor_48865_object22886/head//2 by client.2191635.0:22887 2014-08-12 17:16:36.298808 2014-08-13 17:38:02.588051 7fe9bae887a0 20 read_log 194215'385 (0'0) modify 8c0ab1aa/benchmark_data_storage0111-ib.pocstor_48865_object22899/head//2 by client.2191635.0:22900 2014-08-12 17:16:36.345587 2014-08-13 17:38:02.588071 7fe9bae887a0 20 read_log 194215'386 (0'0) modify 8bceefea/benchmark_data_storage0111-ib.pocstor_48865_object22913/head//2 by client.2191635.0:22914 2014-08-12 17:16:36.395120 2014-08-13 17:38:02.588092 7fe9bae887a0 20 read_log 194215'387 (0'0) modify 2fbf146a/benchmark_data_storage0111-ib.pocstor_48865_object22932/head//2 by client.2191635.0:22933 2014-08-12 17:16:36.462020 2014-08-13 17:38:02.588113 7fe9bae887a0 20 read_log 194215'388 (0'0) modify 7df62c2a/benchmark_data_storage0111-ib.pocstor_48865_object23119/head//2 by client.2191635.0:23120 2014-08-12 17:16:37.141437 2014-08-13 17:38:02.588133 7fe9bae887a0 20 read_log 194215'389 (0'0) modify b614c76a/benchmark_data_storage0111-ib.pocstor_48865_object23197/head//2 by client.2191635.0:23198 2014-08-12 17:16:37.475105 2014-08-13 17:38:02.588161 7fe9bae887a0 20 read_log 194215'390 (0'0) modify 62b055ea/benchmark_data_storage0111-ib.pocstor_48865_object23212/head//2 by client.2191635.0:23213 2014-08-12 17:16:37.542049 2014-08-13 17:38:02.588183 7fe9bae887a0 20 read_log 194215'391 (0'0) modify 972960ea/benchmark_data_storage0111-ib.pocstor_48865_object23269/head//2 by client.2191635.0:23270 2014-08-12 17:16:37.814742 2014-08-13 17:38:02.588205 7fe9bae887a0 20 read_log 194215'392 (0'0) modify 64c4eb6a/benchmark_data_storage0111-ib.pocstor_48865_object23325/head//2 by client.2191635.0:23326 2014-08-12 17:16:38.093389 2014-08-13 17:38:02.588226 7fe9bae887a0 20 read_log 194215'393 (0'0) modify c10f2b2a/benchmark_data_storage0111-ib.pocstor_48865_object23339/head//2 by client.2191635.0:23340 2014-08-12 17:16:38.194023 2014-08-13 17:38:02.588246 7fe9bae887a0 20 read_log 194215'394 (0'0) modify f00c10aa/benchmark_data_storage0111-ib.pocstor_48865_object23473/head//2 by client.2191635.0:23474 2014-08-12 17:16:39.514178 2014-08-13 17:38:02.588278 7fe9bae887a0 20 read_log 194215'395 (0'0) modify e2f975aa/benchmark_data_storage0111-ib.pocstor_48865_object23699/head//2 by client.2191635.0:23700 2014-08-12 17:16:40.566220 2014-08-13 17:38:02.588300 7fe9bae887a0 20 read_log 194215'396 (0'0) modify 68a661ea/benchmark_data_storage0111-ib.pocstor_48865_object23758/head//2 by client.2191635.0:23759 2014-08-12 17:16:40.871178 2014-08-13 17:38:02.588321 7fe9bae887a0 20 read_log 194215'397 (0'0) modify baa697aa/benchmark_data_storage0111-ib.pocstor_48865_object23818/head//2 by client.2191635.0:23819 2014-08-12 17:16:41.102698 2014-08-13 17:38:02.588342 7fe9bae887a0 20 read_log 194215'398 (0'0) modify 6b48b46a/benchmark_data_storage0111-ib.pocstor_48865_object23835/head//2 by client.2191635.0:23836 2014-08-12 17:16:41.162200 2014-08-13 17:38:02.588363 7fe9bae887a0 20 read_log 194215'399 (0'0) modify d8ebb3aa/benchmark_data_storage0111-ib.pocstor_48865_object23888/head//2 by client.2191635.0:23889 2014-08-12 17:16:41.380060 2014-08-13 17:38:02.588383 7fe9bae887a0 20 read_log 194215'400 (0'0) modify 34bd3aaa/benchmark_data_storage0111-ib.pocstor_48865_object23938/head//2 by client.2191635.0:23939 2014-08-12 17:16:41.659877 2014-08-13 17:38:02.588404 7fe9bae887a0 20 read_log 194215'401 (0'0) modify 7504a3aa/benchmark_data_storage0111-ib.pocstor_48865_object23964/head//2 by client.2191635.0:23965 2014-08-12 17:16:41.843009 2014-08-13 17:38:02.588425 7fe9bae887a0 20 read_log 194215'402 (0'0) modify e16bca6a/benchmark_data_storage0111-ib.pocstor_48865_object24022/head//2 by client.2191635.0:24023 2014-08-12 17:16:42.218380 2014-08-13 17:38:02.588448 7fe9bae887a0 20 read_log 194215'403 (0'0) modify 4b4b8cea/benchmark_data_storage0111-ib.pocstor_48865_object24030/head//2 by client.2191635.0:24031 2014-08-12 17:16:42.257717 2014-08-13 17:38:02.588469 7fe9bae887a0 20 read_log 194215'404 (0'0) modify e441ef6a/benchmark_data_storage0111-ib.pocstor_48865_object24080/head//2 by client.2191635.0:24081 2014-08-12 17:16:42.460966 2014-08-13 17:38:02.588492 7fe9bae887a0 20 read_log 194215'405 (0'0) modify 5fcc2aaa/benchmark_data_storage0111-ib.pocstor_48865_object24241/head//2 by client.2191635.0:24242 2014-08-12 17:16:43.044977 2014-08-13 17:38:02.588513 7fe9bae887a0 20 read_log 194215'406 (0'0) modify 3c8eaeaa/benchmark_data_storage0111-ib.pocstor_48865_object24315/head//2 by client.2191635.0:24316 2014-08-12 17:16:43.315794 2014-08-13 17:38:02.588534 7fe9bae887a0 20 read_log 194215'407 (0'0) modify fb00662a/benchmark_data_storage0111-ib.pocstor_48865_object24327/head//2 by client.2191635.0:24328 2014-08-12 17:16:43.358796 2014-08-13 17:38:02.588555 7fe9bae887a0 20 read_log 194215'408 (0'0) modify b50f7ea/benchmark_data_storage0111-ib.pocstor_48865_object24343/head//2 by client.2191635.0:24344 2014-08-12 17:16:43.415391 2014-08-13 17:38:02.588576 7fe9bae887a0 20 read_log 194215'409 (0'0) modify f5caafaa/benchmark_data_storage0111-ib.pocstor_48865_object24349/head//2 by client.2191635.0:24350 2014-08-12 17:16:43.436839 2014-08-13 17:38:02.588597 7fe9bae887a0 20 read_log 194215'410 (0'0) modify d65f39aa/benchmark_data_storage0111-ib.pocstor_48865_object24366/head//2 by client.2191635.0:24367 2014-08-12 17:16:43.497237 2014-08-13 17:38:02.588618 7fe9bae887a0 20 read_log 194215'411 (0'0) modify d785ae2a/benchmark_data_storage0111-ib.pocstor_48865_object24461/head//2 by client.2191635.0:24462 2014-08-12 17:16:43.926709 2014-08-13 17:38:02.588639 7fe9bae887a0 20 read_log 194215'412 (0'0) modify 94e6396a/benchmark_data_storage0111-ib.pocstor_48865_object24499/head//2 by client.2191635.0:24500 2014-08-12 17:16:44.064613 2014-08-13 17:38:02.588659 7fe9bae887a0 20 read_log 194215'413 (0'0) modify ec9780ea/benchmark_data_storage0111-ib.pocstor_48865_object24551/head//2 by client.2191635.0:24552 2014-08-12 17:16:44.252457 2014-08-13 17:38:02.588680 7fe9bae887a0 20 read_log 194215'414 (0'0) modify 76527aaa/benchmark_data_storage0111-ib.pocstor_48865_object24564/head//2 by client.2191635.0:24565 2014-08-12 17:16:44.298816 2014-08-13 17:38:02.588711 7fe9bae887a0 20 read_log 194215'415 (0'0) modify 1fe906ea/benchmark_data_storage0111-ib.pocstor_48865_object24630/head//2 by client.2191635.0:24631 2014-08-12 17:16:44.537451 2014-08-13 17:38:02.588733 7fe9bae887a0 20 read_log 194215'416 (0'0) modify d088b7aa/benchmark_data_storage0111-ib.pocstor_48865_object24638/head//2 by client.2191635.0:24639 2014-08-12 17:16:44.566378 2014-08-13 17:38:02.588753 7fe9bae887a0 20 read_log 194215'417 (0'0) modify c957872a/benchmark_data_storage0111-ib.pocstor_48865_object24706/head//2 by client.2191635.0:24707 2014-08-12 17:16:44.809434 2014-08-13 17:38:02.588774 7fe9bae887a0 20 read_log 194215'418 (0'0) modify 7e977daa/benchmark_data_storage0111-ib.pocstor_48865_object24941/head//2 by client.2191635.0:24942 2014-08-12 17:16:45.770427 2014-08-13 17:38:02.588795 7fe9bae887a0 20 read_log 194215'419 (0'0) modify c441e36a/benchmark_data_storage0111-ib.pocstor_48865_object25044/head//2 by client.2191635.0:25045 2014-08-12 17:16:46.214769 2014-08-13 17:38:02.588816 7fe9bae887a0 20 read_log 194215'420 (0'0) modify 3b05822a/benchmark_data_storage0111-ib.pocstor_48865_object25074/head//2 by client.2191635.0:25075 2014-08-12 17:16:46.364290 2014-08-13 17:38:02.588836 7fe9bae887a0 20 read_log 194215'421 (0'0) modify dac3a2ea/benchmark_data_storage0111-ib.pocstor_48865_object25144/head//2 by client.2191635.0:25145 2014-08-12 17:16:46.618312 2014-08-13 17:38:02.588857 7fe9bae887a0 20 read_log 194215'422 (0'0) modify ed943f2a/benchmark_data_storage0111-ib.pocstor_48865_object25159/head//2 by client.2191635.0:25160 2014-08-12 17:16:46.671518 2014-08-13 17:38:02.588877 7fe9bae887a0 20 read_log 194215'423 (0'0) modify 68276bea/benchmark_data_storage0111-ib.pocstor_48865_object25166/head//2 by client.2191635.0:25167 2014-08-12 17:16:46.696500 2014-08-13 17:38:02.588929 7fe9bae887a0 20 read_log 194215'424 (0'0) modify bd3e026a/benchmark_data_storage0111-ib.pocstor_48865_object25310/head//2 by client.2191635.0:25311 2014-08-12 17:16:47.211674 2014-08-13 17:38:02.588951 7fe9bae887a0 20 read_log 194215'425 (0'0) modify 48d2462a/benchmark_data_storage0111-ib.pocstor_48865_object25331/head//2 by client.2191635.0:25332 2014-08-12 17:16:47.288629 2014-08-13 17:38:02.588974 7fe9bae887a0 20 read_log 194215'426 (0'0) modify 9c95526a/benchmark_data_storage0111-ib.pocstor_48865_object25369/head//2 by client.2191635.0:25370 2014-08-12 17:16:47.423563 2014-08-13 17:38:02.588995 7fe9bae887a0 20 read_log 194215'427 (0'0) modify 9e2944aa/benchmark_data_storage0111-ib.pocstor_48865_object25391/head//2 by client.2191635.0:25392 2014-08-12 17:16:47.504594 2014-08-13 17:38:02.589016 7fe9bae887a0 20 read_log 194215'428 (0'0) modify aff1fc6a/benchmark_data_storage0111-ib.pocstor_48865_object25463/head//2 by client.2191635.0:25464 2014-08-12 17:16:47.766553 2014-08-13 17:38:02.589036 7fe9bae887a0 20 read_log 194215'429 (0'0) modify 9f6824aa/benchmark_data_storage0111-ib.pocstor_48865_object25483/head//2 by client.2191635.0:25484 2014-08-12 17:16:47.839544 2014-08-13 17:38:02.589057 7fe9bae887a0 20 read_log 194215'430 (0'0) modify 9cc1c82a/benchmark_data_storage0111-ib.pocstor_48865_object25512/head//2 by client.2191635.0:25513 2014-08-12 17:16:47.944868 2014-08-13 17:38:02.589077 7fe9bae887a0 20 read_log 194215'431 (0'0) modify f2fcef2a/benchmark_data_storage0111-ib.pocstor_48865_object25660/head//2 by client.2191635.0:25661 2014-08-12 17:16:49.637166 2014-08-13 17:38:02.589098 7fe9bae887a0 20 read_log 194215'432 (0'0) modify deeda82a/benchmark_data_storage0111-ib.pocstor_48865_object25706/head//2 by client.2191635.0:25707 2014-08-12 17:16:49.805121 2014-08-13 17:38:02.589150 7fe9bae887a0 20 read_log 194215'433 (0'0) modify b504f56a/benchmark_data_storage0111-ib.pocstor_48865_object25726/head//2 by client.2191635.0:25727 2014-08-12 17:16:49.875936 2014-08-13 17:38:02.589199 7fe9bae887a0 20 read_log 194215'434 (0'0) modify 652e19ea/benchmark_data_storage0111-ib.pocstor_48865_object25735/head//2 by client.2191635.0:25736 2014-08-12 17:16:49.907590 2014-08-13 17:38:02.589236 7fe9bae887a0 20 read_log 194215'435 (0'0) modify ff82672a/benchmark_data_storage0111-ib.pocstor_48865_object25879/head//2 by client.2191635.0:25880 2014-08-12 17:16:50.483030 2014-08-13 17:38:02.589258 7fe9bae887a0 20 read_log 194215'436 (0'0) modify 1deec7ea/benchmark_data_storage0111-ib.pocstor_48865_object25948/head//2 by client.2191635.0:25949 2014-08-12 17:16:50.750259 2014-08-13 17:38:02.589279 7fe9bae887a0 20 read_log 194215'437 (0'0) modify 6898cd6a/benchmark_data_storage0111-ib.pocstor_48865_object25978/head//2 by client.2191635.0:25979 2014-08-12 17:16:50.855998 2014-08-13 17:38:02.589300 7fe9bae887a0 20 read_log 194215'438 (0'0) modify 9c55f66a/benchmark_data_storage0111-ib.pocstor_48865_object26060/head//2 by client.2191635.0:26061 2014-08-12 17:16:51.261313 2014-08-13 17:38:02.589320 7fe9bae887a0 20 read_log 194215'439 (0'0) modify 99abde2a/benchmark_data_storage0111-ib.pocstor_48865_object26093/head//2 by client.2191635.0:26094 2014-08-12 17:16:51.409383 2014-08-13 17:38:02.589341 7fe9bae887a0 20 read_log 194215'440 (0'0) modify 1696b9ea/benchmark_data_storage0111-ib.pocstor_48865_object26147/head//2 by client.2191635.0:26148 2014-08-12 17:16:51.608739 2014-08-13 17:38:02.589364 7fe9bae887a0 20 read_log 194215'441 (0'0) modify 29311a2a/benchmark_data_storage0111-ib.pocstor_48865_object26180/head//2 by client.2191635.0:26181 2014-08-12 17:16:51.729545 2014-08-13 17:38:02.589385 7fe9bae887a0 20 read_log 194215'442 (0'0) modify f9505d2a/benchmark_data_storage0111-ib.pocstor_48865_object26181/head//2 by client.2191635.0:26182 2014-08-12 17:16:51.733103 2014-08-13 17:38:02.589410 7fe9bae887a0 20 read_log 194215'443 (0'0) modify 5991912a/benchmark_data_storage0111-ib.pocstor_48865_object26212/head//2 by client.2191635.0:26213 2014-08-12 17:16:51.844592 2014-08-13 17:38:02.589431 7fe9bae887a0 20 read_log 194215'444 (0'0) modify b036cdea/benchmark_data_storage0111-ib.pocstor_48865_object26253/head//2 by client.2191635.0:26254 2014-08-12 17:16:51.994567 2014-08-13 17:38:02.589452 7fe9bae887a0 20 read_log 194215'445 (0'0) modify 3571b8ea/benchmark_data_storage0111-ib.pocstor_48865_object26474/head//2 by client.2191635.0:26475 2014-08-12 17:16:52.798075 2014-08-13 17:38:02.589473 7fe9bae887a0 20 read_log 194215'446 (0'0) modify cdf9102a/benchmark_data_storage0111-ib.pocstor_48865_object26574/head//2 by client.2191635.0:26575 2014-08-12 17:16:53.164843 2014-08-13 17:38:02.589494 7fe9bae887a0 20 read_log 194215'447 (0'0) modify acb3246a/benchmark_data_storage0111-ib.pocstor_48865_object26583/head//2 by client.2191635.0:26584 2014-08-12 17:16:53.196926 2014-08-13 17:38:02.589514 7fe9bae887a0 20 read_log 194215'448 (0'0) modify c64e22ea/benchmark_data_storage0111-ib.pocstor_48865_object26626/head//2 by client.2191635.0:26627 2014-08-12 17:16:53.354760 2014-08-13 17:38:02.589535 7fe9bae887a0 20 read_log 194215'449 (0'0) modify d778c46a/benchmark_data_storage0111-ib.pocstor_48865_object26634/head//2 by client.2191635.0:26635 2014-08-12 17:16:53.383282 2014-08-13 17:38:02.589556 7fe9bae887a0 20 read_log 194215'450 (0'0) modify 6da2d2ea/benchmark_data_storage0111-ib.pocstor_48865_object26711/head//2 by client.2191635.0:26712 2014-08-12 17:16:53.661811 2014-08-13 17:38:02.589577 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 6655c36a/benchmark_data_storage0111-ib.pocstor_48865_object26778/head//2 by client.2191635.0:26779 2014-08-12 17:16:53.907259 2014-08-13 17:38:02.589597 7fe9bae887a0 20 read_log 194215'452 (0'0) modify d935beaa/benchmark_data_storage0111-ib.pocstor_48865_object26946/head//2 by client.2191635.0:26947 2014-08-12 17:16:54.585600 2014-08-13 17:38:02.589618 7fe9bae887a0 20 read_log 194215'453 (0'0) modify 143d096a/benchmark_data_storage0111-ib.pocstor_48865_object26976/head//2 by client.2191635.0:26977 2014-08-12 17:16:54.786591 2014-08-13 17:38:02.589639 7fe9bae887a0 20 read_log 194215'454 (0'0) modify a5899caa/benchmark_data_storage0111-ib.pocstor_48865_object27029/head//2 by client.2191635.0:27030 2014-08-12 17:16:55.110411 2014-08-13 17:38:02.589673 7fe9bae887a0 20 read_log 194215'455 (0'0) modify b23c082a/benchmark_data_storage0111-ib.pocstor_48865_object27049/head//2 by client.2191635.0:27050 2014-08-12 17:16:55.184521 2014-08-13 17:38:02.589696 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 128e18aa/benchmark_data_storage0111-ib.pocstor_48865_object27097/head//2 by client.2191635.0:27098 2014-08-12 17:16:55.352536 2014-08-13 17:38:02.589716 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 1fef4b2a/benchmark_data_storage0111-ib.pocstor_48865_object27156/head//2 by client.2191635.0:27157 2014-08-12 17:16:55.597599 2014-08-13 17:38:02.589737 7fe9bae887a0 20 read_log 194215'458 (0'0) modify 53f22aaa/benchmark_data_storage0111-ib.pocstor_48865_object27164/head//2 by client.2191635.0:27165 2014-08-12 17:16:55.625229 2014-08-13 17:38:02.589758 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 25a647ea/benchmark_data_storage0111-ib.pocstor_48865_object27294/head//2 by client.2191635.0:27295 2014-08-12 17:16:56.172477 2014-08-13 17:38:02.589812 7fe9bae887a0 20 read_log 194215'460 (0'0) modify edbf666a/benchmark_data_storage0111-ib.pocstor_48865_object27302/head//2 by client.2191635.0:27303 2014-08-12 17:16:56.201858 2014-08-13 17:38:02.589835 7fe9bae887a0 20 read_log 194215'461 (0'0) modify e0414e6a/benchmark_data_storage0111-ib.pocstor_48865_object27325/head//2 by client.2191635.0:27326 2014-08-12 17:16:56.279786 2014-08-13 17:38:02.589857 7fe9bae887a0 20 read_log 194215'462 (0'0) modify fa7d5f6a/benchmark_data_storage0111-ib.pocstor_48865_object27402/head//2 by client.2191635.0:27403 2014-08-12 17:16:56.549606 2014-08-13 17:38:02.589878 7fe9bae887a0 20 read_log 194215'463 (0'0) modify fee55a2a/benchmark_data_storage0111-ib.pocstor_48865_object27451/head//2 by client.2191635.0:27452 2014-08-12 17:16:56.752869 2014-08-13 17:38:02.589898 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 9b85faaa/benchmark_data_storage0111-ib.pocstor_48865_object27523/head//2 by client.2191635.0:27524 2014-08-12 17:16:57.044696 2014-08-13 17:38:02.589919 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 233bee2a/benchmark_data_storage0111-ib.pocstor_48865_object27577/head//2 by client.2191635.0:27578 2014-08-12 17:16:57.235154 2014-08-13 17:38:02.589940 7fe9bae887a0 20 read_log 194215'466 (194215'1) delete 68edfb2a/benchmark_data_storage0111-ib.pocstor_48865_object150/head//2 by client.2191635.0:27747 2014-08-12 17:16:58.740066 2014-08-13 17:38:02.589961 7fe9bae887a0 20 read_log 194215'467 (194215'2) delete 11847ea/benchmark_data_storage0111-ib.pocstor_48865_object183/head//2 by client.2191635.0:27780 2014-08-12 17:16:58.764497 2014-08-13 17:38:02.589983 7fe9bae887a0 20 read_log 194215'468 (194215'3) delete 282e8a2a/benchmark_data_storage0111-ib.pocstor_48865_object269/head//2 by client.2191635.0:27866 2014-08-12 17:16:58.888264 2014-08-13 17:38:02.590004 7fe9bae887a0 20 read_log 194215'469 (194215'4) delete 2975e0ea/benchmark_data_storage0111-ib.pocstor_48865_object333/head//2 by client.2191635.0:27930 2014-08-12 17:16:58.907677 2014-08-13 17:38:02.590025 7fe9bae887a0 20 read_log 194215'470 (194215'5) delete 65323c2a/benchmark_data_storage0111-ib.pocstor_48865_object358/head//2 by client.2191635.0:27955 2014-08-12 17:16:58.915552 2014-08-13 17:38:02.590048 7fe9bae887a0 20 read_log 194215'471 (194215'6) delete 80c081ea/benchmark_data_storage0111-ib.pocstor_48865_object538/head//2 by client.2191635.0:28135 2014-08-12 17:16:58.969893 2014-08-13 17:38:02.590069 7fe9bae887a0 20 read_log 194215'472 (194215'7) delete 277a9c6a/benchmark_data_storage0111-ib.pocstor_48865_object629/head//2 by client.2191635.0:28226 2014-08-12 17:16:58.996966 2014-08-13 17:38:02.590090 7fe9bae887a0 20 read_log 194215'473 (194215'8) delete ce42036a/benchmark_data_storage0111-ib.pocstor_48865_object668/head//2 by client.2191635.0:28265 2014-08-12 17:16:59.009266 2014-08-13 17:38:02.590111 7fe9bae887a0 20 read_log 194215'474 (194215'9) delete 5955272a/benchmark_data_storage0111-ib.pocstor_48865_object715/head//2 by client.2191635.0:28312 2014-08-12 17:16:59.023657 2014-08-13 17:38:02.590145 7fe9bae887a0 20 read_log 194215'475 (194215'10) delete 52f5892a/benchmark_data_storage0111-ib.pocstor_48865_object726/head//2 by client.2191635.0:28323 2014-08-12 17:16:59.028038 2014-08-13 17:38:02.590179 7fe9bae887a0 20 read_log 194215'476 (194215'11) delete d1e810aa/benchmark_data_storage0111-ib.pocstor_48865_object1003/head//2 by client.2191635.0:28600 2014-08-12 17:16:59.115602 2014-08-13 17:38:02.590201 7fe9bae887a0 20 read_log 194215'477 (194215'12) delete 70ee6b6a/benchmark_data_storage0111-ib.pocstor_48865_object1014/head//2 by client.2191635.0:28611 2014-08-12 17:16:59.116995 2014-08-13 17:38:02.590222 7fe9bae887a0 20 read_log 194215'478 (194215'13) delete 81dd632a/benchmark_data_storage0111-ib.pocstor_48865_object1055/head//2 by client.2191635.0:28652 2014-08-12 17:16:59.128804 2014-08-13 17:38:02.590243 7fe9bae887a0 20 read_log 194215'479 (194215'14) delete a31ef9aa/benchmark_data_storage0111-ib.pocstor_48865_object1066/head//2 by client.2191635.0:28663 2014-08-12 17:16:59.131648 2014-08-13 17:38:02.590264 7fe9bae887a0 20 read_log 194215'480 (194215'15) delete bb8082a/benchmark_data_storage0111-ib.pocstor_48865_object1073/head//2 by client.2191635.0:28670 2014-08-12 17:16:59.133969 2014-08-13 17:38:02.590285 7fe9bae887a0 20 read_log 194215'481 (194215'16) delete 8c5d0caa/benchmark_data_storage0111-ib.pocstor_48865_object1105/head//2 by client.2191635.0:28702 2014-08-12 17:16:59.142775 2014-08-13 17:38:02.590306 7fe9bae887a0 20 read_log 194215'482 (194215'17) delete e781e86a/benchmark_data_storage0111-ib.pocstor_48865_object1126/head//2 by client.2191635.0:28723 2014-08-12 17:16:59.149257 2014-08-13 17:38:02.590327 7fe9bae887a0 20 read_log 194215'483 (194215'18) delete bb59aaea/benchmark_data_storage0111-ib.pocstor_48865_object1239/head//2 by client.2191635.0:28836 2014-08-12 17:16:59.184349 2014-08-13 17:38:02.590352 7fe9bae887a0 20 read_log 194215'484 (194215'19) delete f812992a/benchmark_data_storage0111-ib.pocstor_48865_object1316/head//2 by client.2191635.0:28913 2014-08-12 17:16:59.211942 2014-08-13 17:38:02.590373 7fe9bae887a0 20 read_log 194215'485 (194215'20) delete 8845246a/benchmark_data_storage0111-ib.pocstor_48865_object1340/head//2 by client.2191635.0:28937 2014-08-12 17:16:59.215979 2014-08-13 17:38:02.590394 7fe9bae887a0 20 read_log 194215'486 (194215'21) delete ac40caa/benchmark_data_storage0111-ib.pocstor_48865_object1424/head//2 by client.2191635.0:29021 2014-08-12 17:16:59.238876 2014-08-13 17:38:02.590415 7fe9bae887a0 20 read_log 194215'487 (194215'22) delete ff0610aa/benchmark_data_storage0111-ib.pocstor_48865_object1571/head//2 by client.2191635.0:29168 2014-08-12 17:16:59.276674 2014-08-13 17:38:02.590436 7fe9bae887a0 20 read_log 194215'488 (194215'23) delete 95a794ea/benchmark_data_storage0111-ib.pocstor_48865_object1601/head//2 by client.2191635.0:29198 2014-08-12 17:16:59.287444 2014-08-13 17:38:02.590457 7fe9bae887a0 20 read_log 194215'489 (194215'24) delete cae8062a/benchmark_data_storage0111-ib.pocstor_48865_object1620/head//2 by client.2191635.0:29217 2014-08-12 17:16:59.292860 2014-08-13 17:38:02.590478 7fe9bae887a0 20 read_log 194215'490 (194215'25) delete 920ef56a/benchmark_data_storage0111-ib.pocstor_48865_object1759/head//2 by client.2191635.0:29356 2014-08-12 17:16:59.329537 2014-08-13 17:38:02.590499 7fe9bae887a0 20 read_log 194215'491 (194215'26) delete a732e2a/benchmark_data_storage0111-ib.pocstor_48865_object1807/head//2 by client.2191635.0:29404 2014-08-12 17:16:59.341555 2014-08-13 17:38:02.590520 7fe9bae887a0 20 read_log 194215'492 (194215'27) delete c4c702aa/benchmark_data_storage0111-ib.pocstor_48865_object1871/head//2 by client.2191635.0:29468 2014-08-12 17:16:59.358921 2014-08-13 17:38:02.590541 7fe9bae887a0 20 read_log 194215'493 (194215'28) delete 279c346a/benchmark_data_storage0111-ib.pocstor_48865_object1875/head//2 by client.2191635.0:29472 2014-08-12 17:16:59.359980 2014-08-13 17:38:02.590562 7fe9bae887a0 20 read_log 194215'494 (194215'29) delete 553263ea/benchmark_data_storage0111-ib.pocstor_48865_object2043/head//2 by client.2191635.0:29640 2014-08-12 17:16:59.407869 2014-08-13 17:38:02.590628 7fe9bae887a0 20 read_log 194215'495 (194215'30) delete f16ac3ea/benchmark_data_storage0111-ib.pocstor_48865_object2102/head//2 by client.2191635.0:29699 2014-08-12 17:16:59.424162 2014-08-13 17:38:02.590652 7fe9bae887a0 20 read_log 194215'496 (194215'31) delete 5fc8512a/benchmark_data_storage0111-ib.pocstor_48865_object2104/head//2 by client.2191635.0:29701 2014-08-12 17:16:59.424878 2014-08-13 17:38:02.590673 7fe9bae887a0 20 read_log 194215'497 (194215'32) delete a896f2ea/benchmark_data_storage0111-ib.pocstor_48865_object2127/head//2 by client.2191635.0:29724 2014-08-12 17:16:59.430160 2014-08-13 17:38:02.590696 7fe9bae887a0 20 read_log 194215'498 (194215'33) delete b516fdaa/benchmark_data_storage0111-ib.pocstor_48865_object2151/head//2 by client.2191635.0:29748 2014-08-12 17:16:59.436895 2014-08-13 17:38:02.590717 7fe9bae887a0 20 read_log 194215'499 (194215'34) delete 8cda0e6a/benchmark_data_storage0111-ib.pocstor_48865_object2185/head//2 by client.2191635.0:29782 2014-08-12 17:16:59.445768 2014-08-13 17:38:02.590738 7fe9bae887a0 20 read_log 194215'500 (194215'35) delete 130ec9aa/benchmark_data_storage0111-ib.pocstor_48865_object2195/head//2 by client.2191635.0:29792 2014-08-12 17:16:59.448530 2014-08-13 17:38:02.590759 7fe9bae887a0 20 read_log 194215'501 (194215'36) delete d4544c2a/benchmark_data_storage0111-ib.pocstor_48865_object2225/head//2 by client.2191635.0:29822 2014-08-12 17:16:59.456450 2014-08-13 17:38:02.590779 7fe9bae887a0 20 read_log 194215'502 (194215'37) delete 8855e5aa/benchmark_data_storage0111-ib.pocstor_48865_object2244/head//2 by client.2191635.0:29841 2014-08-12 17:16:59.461293 2014-08-13 17:38:02.590800 7fe9bae887a0 20 read_log 194215'503 (194215'38) delete 875f70ea/benchmark_data_storage0111-ib.pocstor_48865_object2251/head//2 by client.2191635.0:29848 2014-08-12 17:16:59.463802 2014-08-13 17:38:02.590821 7fe9bae887a0 20 read_log 194215'504 (194215'39) delete 365a75ea/benchmark_data_storage0111-ib.pocstor_48865_object2593/head//2 by client.2191635.0:30190 2014-08-12 17:16:59.567147 2014-08-13 17:38:02.590842 7fe9bae887a0 20 read_log 194215'505 (194215'40) delete 1cc465aa/benchmark_data_storage0111-ib.pocstor_48865_object2601/head//2 by client.2191635.0:30198 2014-08-12 17:16:59.569483 2014-08-13 17:38:02.590863 7fe9bae887a0 20 read_log 194215'506 (194215'41) delete b7b2a22a/benchmark_data_storage0111-ib.pocstor_48865_object2629/head//2 by client.2191635.0:30226 2014-08-12 17:16:59.580240 2014-08-13 17:38:02.590884 7fe9bae887a0 20 read_log 194215'507 (194215'42) delete 109903aa/benchmark_data_storage0111-ib.pocstor_48865_object2691/head//2 by client.2191635.0:30288 2014-08-12 17:16:59.601237 2014-08-13 17:38:02.590905 7fe9bae887a0 20 read_log 194215'508 (194215'43) delete db63af6a/benchmark_data_storage0111-ib.pocstor_48865_object2699/head//2 by client.2191635.0:30296 2014-08-12 17:16:59.602450 2014-08-13 17:38:02.590926 7fe9bae887a0 20 read_log 194215'509 (194215'44) delete 3dfdc96a/benchmark_data_storage0111-ib.pocstor_48865_object2712/head//2 by client.2191635.0:30309 2014-08-12 17:16:59.609500 2014-08-13 17:38:02.590947 7fe9bae887a0 20 read_log 194215'510 (194215'45) delete 839a1faa/benchmark_data_storage0111-ib.pocstor_48865_object2786/head//2 by client.2191635.0:30383 2014-08-12 17:16:59.631825 2014-08-13 17:38:02.590968 7fe9bae887a0 20 read_log 194215'511 (194215'46) delete 7d0316aa/benchmark_data_storage0111-ib.pocstor_48865_object2888/head//2 by client.2191635.0:30485 2014-08-12 17:16:59.661130 2014-08-13 17:38:02.590989 7fe9bae887a0 20 read_log 194215'512 (194215'47) delete 8fd59f2a/benchmark_data_storage0111-ib.pocstor_48865_object3016/head//2 by client.2191635.0:30613 2014-08-12 17:16:59.696494 2014-08-13 17:38:02.591010 7fe9bae887a0 20 read_log 194215'513 (194215'48) delete 2f6cc42a/benchmark_data_storage0111-ib.pocstor_48865_object3021/head//2 by client.2191635.0:30618 2014-08-12 17:16:59.698194 2014-08-13 17:38:02.591031 7fe9bae887a0 20 read_log 194215'514 (194215'49) delete 32c490ea/benchmark_data_storage0111-ib.pocstor_48865_object3081/head//2 by client.2191635.0:30678 2014-08-12 17:16:59.714601 2014-08-13 17:38:02.591063 7fe9bae887a0 20 read_log 194215'515 (194215'50) delete ead47aea/benchmark_data_storage0111-ib.pocstor_48865_object3105/head//2 by client.2191635.0:30702 2014-08-12 17:16:59.721304 2014-08-13 17:38:02.591086 7fe9bae887a0 20 read_log 194215'516 (194215'51) delete bddd492a/benchmark_data_storage0111-ib.pocstor_48865_object3141/head//2 by client.2191635.0:30738 2014-08-12 17:16:59.731462 2014-08-13 17:38:02.591134 7fe9bae887a0 20 read_log 194215'517 (194215'52) delete f9981caa/benchmark_data_storage0111-ib.pocstor_48865_object3321/head//2 by client.2191635.0:30918 2014-08-12 17:16:59.784791 2014-08-13 17:38:02.591164 7fe9bae887a0 20 read_log 194215'518 (194215'53) delete d01bb4ea/benchmark_data_storage0111-ib.pocstor_48865_object3371/head//2 by client.2191635.0:30968 2014-08-12 17:16:59.799870 2014-08-13 17:38:02.591187 7fe9bae887a0 20 read_log 194215'519 (194215'54) delete f0690b6a/benchmark_data_storage0111-ib.pocstor_48865_object3420/head//2 by client.2191635.0:31017 2014-08-12 17:16:59.813627 2014-08-13 17:38:02.591208 7fe9bae887a0 20 read_log 194215'520 (194215'55) delete ef7fe82a/benchmark_data_storage0111-ib.pocstor_48865_object3426/head//2 by client.2191635.0:31023 2014-08-12 17:16:59.815586 2014-08-13 17:38:02.591230 7fe9bae887a0 20 read_log 194215'521 (194215'56) delete 4f5aa4ea/benchmark_data_storage0111-ib.pocstor_48865_object3456/head//2 by client.2191635.0:31053 2014-08-12 17:16:59.824320 2014-08-13 17:38:02.591251 7fe9bae887a0 20 read_log 194215'522 (194215'57) delete e192b76a/benchmark_data_storage0111-ib.pocstor_48865_object3465/head//2 by client.2191635.0:31062 2014-08-12 17:16:59.828559 2014-08-13 17:38:02.591272 7fe9bae887a0 20 read_log 194215'523 (194215'58) delete ed61ad6a/benchmark_data_storage0111-ib.pocstor_48865_object3599/head//2 by client.2191635.0:31196 2014-08-12 17:16:59.873026 2014-08-13 17:38:02.591293 7fe9bae887a0 20 read_log 194215'524 (194215'59) delete 284476ea/benchmark_data_storage0111-ib.pocstor_48865_object3637/head//2 by client.2191635.0:31234 2014-08-12 17:16:59.886034 2014-08-13 17:38:02.591313 7fe9bae887a0 20 read_log 194215'525 (194215'60) delete 69ffe8ea/benchmark_data_storage0111-ib.pocstor_48865_object3643/head//2 by client.2191635.0:31240 2014-08-12 17:16:59.886460 2014-08-13 17:38:02.591334 7fe9bae887a0 20 read_log 194215'526 (194215'61) delete 4443936a/benchmark_data_storage0111-ib.pocstor_48865_object3673/head//2 by client.2191635.0:31270 2014-08-12 17:16:59.896764 2014-08-13 17:38:02.591355 7fe9bae887a0 20 read_log 194215'527 (194215'62) delete b1fb3aaa/benchmark_data_storage0111-ib.pocstor_48865_object3684/head//2 by client.2191635.0:31281 2014-08-12 17:16:59.899705 2014-08-13 17:38:02.591375 7fe9bae887a0 20 read_log 194215'528 (194215'63) delete abbf486a/benchmark_data_storage0111-ib.pocstor_48865_object3708/head//2 by client.2191635.0:31305 2014-08-12 17:16:59.907925 2014-08-13 17:38:02.591396 7fe9bae887a0 20 read_log 194215'529 (194215'64) delete d2774aaa/benchmark_data_storage0111-ib.pocstor_48865_object3723/head//2 by client.2191635.0:31320 2014-08-12 17:16:59.911646 2014-08-13 17:38:02.591416 7fe9bae887a0 20 read_log 194215'530 (194215'65) delete d4716aaa/benchmark_data_storage0111-ib.pocstor_48865_object3873/head//2 by client.2191635.0:31470 2014-08-12 17:16:59.957188 2014-08-13 17:38:02.591437 7fe9bae887a0 20 read_log 194215'531 (194215'66) delete 2546412a/benchmark_data_storage0111-ib.pocstor_48865_object3982/head//2 by client.2191635.0:31579 2014-08-12 17:16:59.989662 2014-08-13 17:38:02.591457 7fe9bae887a0 20 read_log 194215'532 (194215'67) delete fce3fb2a/benchmark_data_storage0111-ib.pocstor_48865_object4047/head//2 by client.2191635.0:31644 2014-08-12 17:17:00.010584 2014-08-13 17:38:02.591478 7fe9bae887a0 20 read_log 194215'533 (194215'68) delete 880c6cea/benchmark_data_storage0111-ib.pocstor_48865_object4061/head//2 by client.2191635.0:31658 2014-08-12 17:17:00.015500 2014-08-13 17:38:02.591499 7fe9bae887a0 20 read_log 194215'534 (194215'69) delete da367caa/benchmark_data_storage0111-ib.pocstor_48865_object4094/head//2 by client.2191635.0:31691 2014-08-12 17:17:00.026961 2014-08-13 17:38:02.591532 7fe9bae887a0 20 read_log 194215'535 (194215'70) delete fbd37baa/benchmark_data_storage0111-ib.pocstor_48865_object4145/head//2 by client.2191635.0:31742 2014-08-12 17:17:00.041234 2014-08-13 17:38:02.591554 7fe9bae887a0 20 read_log 194215'536 (194215'71) delete df68caea/benchmark_data_storage0111-ib.pocstor_48865_object4151/head//2 by client.2191635.0:31748 2014-08-12 17:17:00.042368 2014-08-13 17:38:02.591575 7fe9bae887a0 20 read_log 194215'537 (194215'72) delete 7757da6a/benchmark_data_storage0111-ib.pocstor_48865_object4234/head//2 by client.2191635.0:31831 2014-08-12 17:17:00.069978 2014-08-13 17:38:02.591596 7fe9bae887a0 20 read_log 194215'538 (194215'73) delete aef9282a/benchmark_data_storage0111-ib.pocstor_48865_object4266/head//2 by client.2191635.0:31863 2014-08-12 17:17:00.078478 2014-08-13 17:38:02.591621 7fe9bae887a0 20 read_log 194215'539 (194215'74) delete ca7790ea/benchmark_data_storage0111-ib.pocstor_48865_object4302/head//2 by client.2191635.0:31899 2014-08-12 17:17:00.089311 2014-08-13 17:38:02.591643 7fe9bae887a0 20 read_log 194215'540 (194215'75) delete 5c96bb6a/benchmark_data_storage0111-ib.pocstor_48865_object4312/head//2 by client.2191635.0:31909 2014-08-12 17:17:00.091635 2014-08-13 17:38:02.591664 7fe9bae887a0 20 read_log 194215'541 (194215'76) delete 463bdeaa/benchmark_data_storage0111-ib.pocstor_48865_object4392/head//2 by client.2191635.0:31989 2014-08-12 17:17:00.115263 2014-08-13 17:38:02.591685 7fe9bae887a0 20 read_log 194215'542 (194215'77) delete d37ec2ea/benchmark_data_storage0111-ib.pocstor_48865_object4404/head//2 by client.2191635.0:32001 2014-08-12 17:17:00.118611 2014-08-13 17:38:02.591706 7fe9bae887a0 20 read_log 194215'543 (194215'78) delete 50c68dea/benchmark_data_storage0111-ib.pocstor_48865_object4406/head//2 by client.2191635.0:32003 2014-08-12 17:17:00.119336 2014-08-13 17:38:02.591758 7fe9bae887a0 20 read_log 194215'544 (194215'79) delete 4e5fdaaa/benchmark_data_storage0111-ib.pocstor_48865_object4437/head//2 by client.2191635.0:32034 2014-08-12 17:17:00.127312 2014-08-13 17:38:02.591780 7fe9bae887a0 20 read_log 194215'545 (194215'80) delete c56423aa/benchmark_data_storage0111-ib.pocstor_48865_object4456/head//2 by client.2191635.0:32053 2014-08-12 17:17:00.135904 2014-08-13 17:38:02.591801 7fe9bae887a0 20 read_log 194215'546 (194215'81) delete 48228f2a/benchmark_data_storage0111-ib.pocstor_48865_object4543/head//2 by client.2191635.0:32140 2014-08-12 17:17:00.164226 2014-08-13 17:38:02.591822 7fe9bae887a0 20 read_log 194215'547 (194215'82) delete f096ccea/benchmark_data_storage0111-ib.pocstor_48865_object4594/head//2 by client.2191635.0:32191 2014-08-12 17:17:00.178487 2014-08-13 17:38:02.591842 7fe9bae887a0 20 read_log 194215'548 (194215'83) delete 775974ea/benchmark_data_storage0111-ib.pocstor_48865_object4605/head//2 by client.2191635.0:32202 2014-08-12 17:17:00.180353 2014-08-13 17:38:02.591864 7fe9bae887a0 20 read_log 194215'549 (194215'84) delete dee4596a/benchmark_data_storage0111-ib.pocstor_48865_object4660/head//2 by client.2191635.0:32257 2014-08-12 17:17:00.201680 2014-08-13 17:38:02.591885 7fe9bae887a0 20 read_log 194215'550 (194215'85) delete b3d8632a/benchmark_data_storage0111-ib.pocstor_48865_object4719/head//2 by client.2191635.0:32316 2014-08-12 17:17:00.216758 2014-08-13 17:38:02.591906 7fe9bae887a0 20 read_log 194215'551 (194215'86) delete dfa2a76a/benchmark_data_storage0111-ib.pocstor_48865_object4725/head//2 by client.2191635.0:32322 2014-08-12 17:17:00.217721 2014-08-13 17:38:02.591927 7fe9bae887a0 20 read_log 194215'552 (194215'87) delete ea96c2aa/benchmark_data_storage0111-ib.pocstor_48865_object4778/head//2 by client.2191635.0:32375 2014-08-12 17:17:00.235434 2014-08-13 17:38:02.591948 7fe9bae887a0 20 read_log 194215'553 (194215'88) delete 37ca6bea/benchmark_data_storage0111-ib.pocstor_48865_object4949/head//2 by client.2191635.0:32546 2014-08-12 17:17:00.281456 2014-08-13 17:38:02.591968 7fe9bae887a0 20 read_log 194215'554 (194215'89) delete 62eca2ea/benchmark_data_storage0111-ib.pocstor_48865_object4961/head//2 by client.2191635.0:32558 2014-08-12 17:17:00.284197 2014-08-13 17:38:02.592005 7fe9bae887a0 20 read_log 194215'555 (194215'90) delete 489460ea/benchmark_data_storage0111-ib.pocstor_48865_object4986/head//2 by client.2191635.0:32583 2014-08-12 17:17:00.290632 2014-08-13 17:38:02.592027 7fe9bae887a0 20 read_log 194215'556 (194215'91) delete fc746b2a/benchmark_data_storage0111-ib.pocstor_48865_object5031/head//2 by client.2191635.0:32628 2014-08-12 17:17:00.303341 2014-08-13 17:38:02.592049 7fe9bae887a0 20 read_log 194215'557 (194215'92) delete dc8ffd6a/benchmark_data_storage0111-ib.pocstor_48865_object5063/head//2 by client.2191635.0:32660 2014-08-12 17:17:00.312776 2014-08-13 17:38:02.592070 7fe9bae887a0 20 read_log 194215'558 (194215'93) delete 98f0566a/benchmark_data_storage0111-ib.pocstor_48865_object5274/head//2 by client.2191635.0:32871 2014-08-12 17:17:00.368232 2014-08-13 17:38:02.592091 7fe9bae887a0 20 read_log 194215'559 (194215'94) delete a673bdea/benchmark_data_storage0111-ib.pocstor_48865_object5308/head//2 by client.2191635.0:32905 2014-08-12 17:17:00.377189 2014-08-13 17:38:02.592112 7fe9bae887a0 20 read_log 194215'560 (194215'95) delete dd5e12aa/benchmark_data_storage0111-ib.pocstor_48865_object5321/head//2 by client.2191635.0:32918 2014-08-12 17:17:00.381361 2014-08-13 17:38:02.592133 7fe9bae887a0 20 read_log 194215'561 (194215'96) delete 743f146a/benchmark_data_storage0111-ib.pocstor_48865_object5342/head//2 by client.2191635.0:32939 2014-08-12 17:17:00.386094 2014-08-13 17:38:02.592161 7fe9bae887a0 20 read_log 194215'562 (194215'97) delete 4b19ae2a/benchmark_data_storage0111-ib.pocstor_48865_object5365/head//2 by client.2191635.0:32962 2014-08-12 17:17:00.393665 2014-08-13 17:38:02.592183 7fe9bae887a0 20 read_log 194215'563 (194215'98) delete 4187d4aa/benchmark_data_storage0111-ib.pocstor_48865_object5418/head//2 by client.2191635.0:33015 2014-08-12 17:17:00.407853 2014-08-13 17:38:02.592204 7fe9bae887a0 20 read_log 194215'564 (194215'99) delete 76e737aa/benchmark_data_storage0111-ib.pocstor_48865_object5442/head//2 by client.2191635.0:33039 2014-08-12 17:17:00.413907 2014-08-13 17:38:02.592225 7fe9bae887a0 20 read_log 194215'565 (194215'100) delete 90d3712a/benchmark_data_storage0111-ib.pocstor_48865_object5446/head//2 by client.2191635.0:33043 2014-08-12 17:17:00.415286 2014-08-13 17:38:02.592247 7fe9bae887a0 20 read_log 194215'566 (194215'101) delete 59cb2aea/benchmark_data_storage0111-ib.pocstor_48865_object5453/head//2 by client.2191635.0:33050 2014-08-12 17:17:00.418769 2014-08-13 17:38:02.592268 7fe9bae887a0 20 read_log 194215'567 (194215'102) delete d0ed1c6a/benchmark_data_storage0111-ib.pocstor_48865_object5498/head//2 by client.2191635.0:33095 2014-08-12 17:17:00.432742 2014-08-13 17:38:02.592289 7fe9bae887a0 20 read_log 194215'568 (194215'103) delete 48e796a/benchmark_data_storage0111-ib.pocstor_48865_object5511/head//2 by client.2191635.0:33108 2014-08-12 17:17:00.437389 2014-08-13 17:38:02.592310 7fe9bae887a0 20 read_log 194215'569 (194215'104) delete b179e16a/benchmark_data_storage0111-ib.pocstor_48865_object5567/head//2 by client.2191635.0:33164 2014-08-12 17:17:00.453265 2014-08-13 17:38:02.592331 7fe9bae887a0 20 read_log 194215'570 (194215'105) delete d38639aa/benchmark_data_storage0111-ib.pocstor_48865_object5574/head//2 by client.2191635.0:33171 2014-08-12 17:17:00.454545 2014-08-13 17:38:02.592353 7fe9bae887a0 20 read_log 194215'571 (194215'106) delete 6cb251aa/benchmark_data_storage0111-ib.pocstor_48865_object5598/head//2 by client.2191635.0:33195 2014-08-12 17:17:00.461774 2014-08-13 17:38:02.592374 7fe9bae887a0 20 read_log 194215'572 (194215'107) delete eaccd2a/benchmark_data_storage0111-ib.pocstor_48865_object5617/head//2 by client.2191635.0:33214 2014-08-12 17:17:00.469469 2014-08-13 17:38:02.592395 7fe9bae887a0 20 read_log 194215'573 (194215'108) delete a3f082aa/benchmark_data_storage0111-ib.pocstor_48865_object5622/head//2 by client.2191635.0:33219 2014-08-12 17:17:00.470477 2014-08-13 17:38:02.592416 7fe9bae887a0 20 read_log 194215'574 (194215'109) delete a3af342a/benchmark_data_storage0111-ib.pocstor_48865_object5745/head//2 by client.2191635.0:33342 2014-08-12 17:17:00.503283 2014-08-13 17:38:02.592448 7fe9bae887a0 20 read_log 194215'575 (194215'110) delete e572e62a/benchmark_data_storage0111-ib.pocstor_48865_object5964/head//2 by client.2191635.0:33561 2014-08-12 17:17:00.571300 2014-08-13 17:38:02.592471 7fe9bae887a0 20 read_log 194215'576 (194215'111) delete b874d76a/benchmark_data_storage0111-ib.pocstor_48865_object5981/head//2 by client.2191635.0:33578 2014-08-12 17:17:00.576497 2014-08-13 17:38:02.592494 7fe9bae887a0 20 read_log 194215'577 (194215'112) delete b9b163ea/benchmark_data_storage0111-ib.pocstor_48865_object5987/head//2 by client.2191635.0:33584 2014-08-12 17:17:00.578577 2014-08-13 17:38:02.592516 7fe9bae887a0 20 read_log 194215'578 (194215'113) delete 48d2f52a/benchmark_data_storage0111-ib.pocstor_48865_object6057/head//2 by client.2191635.0:33654 2014-08-12 17:17:00.598759 2014-08-13 17:38:02.592537 7fe9bae887a0 20 read_log 194215'579 (194215'114) delete 857610ea/benchmark_data_storage0111-ib.pocstor_48865_object6104/head//2 by client.2191635.0:33701 2014-08-12 17:17:00.611316 2014-08-13 17:38:02.592559 7fe9bae887a0 20 read_log 194215'580 (194215'115) delete 2628a2ea/benchmark_data_storage0111-ib.pocstor_48865_object6300/head//2 by client.2191635.0:33897 2014-08-12 17:17:00.670677 2014-08-13 17:38:02.592580 7fe9bae887a0 20 read_log 194215'581 (194215'116) delete 6374406a/benchmark_data_storage0111-ib.pocstor_48865_object6344/head//2 by client.2191635.0:33941 2014-08-12 17:17:00.684812 2014-08-13 17:38:02.592601 7fe9bae887a0 20 read_log 194215'582 (194215'117) delete 1a2b33ea/benchmark_data_storage0111-ib.pocstor_48865_object6404/head//2 by client.2191635.0:34001 2014-08-12 17:17:00.701931 2014-08-13 17:38:02.592622 7fe9bae887a0 20 read_log 194215'583 (194215'118) delete 56ba632a/benchmark_data_storage0111-ib.pocstor_48865_object6415/head//2 by client.2191635.0:34012 2014-08-12 17:17:00.706675 2014-08-13 17:38:02.592643 7fe9bae887a0 20 read_log 194215'584 (194215'119) delete 7073ea/benchmark_data_storage0111-ib.pocstor_48865_object6523/head//2 by client.2191635.0:34120 2014-08-12 17:17:00.741268 2014-08-13 17:38:02.592664 7fe9bae887a0 20 read_log 194215'585 (194215'120) delete 4be729ea/benchmark_data_storage0111-ib.pocstor_48865_object6640/head//2 by client.2191635.0:34237 2014-08-12 17:17:00.775265 2014-08-13 17:38:02.592685 7fe9bae887a0 20 read_log 194215'586 (194215'121) delete f386bea/benchmark_data_storage0111-ib.pocstor_48865_object6650/head//2 by client.2191635.0:34247 2014-08-12 17:17:00.778133 2014-08-13 17:38:02.592706 7fe9bae887a0 20 read_log 194215'587 (194215'122) delete 64fac96a/benchmark_data_storage0111-ib.pocstor_48865_object6732/head//2 by client.2191635.0:34329 2014-08-12 17:17:00.803044 2014-08-13 17:38:02.592727 7fe9bae887a0 20 read_log 194215'588 (194215'123) delete 68644faa/benchmark_data_storage0111-ib.pocstor_48865_object6763/head//2 by client.2191635.0:34360 2014-08-12 17:17:00.811578 2014-08-13 17:38:02.592748 7fe9bae887a0 20 read_log 194215'589 (194215'124) delete e27b086a/benchmark_data_storage0111-ib.pocstor_48865_object6779/head//2 by client.2191635.0:34376 2014-08-12 17:17:00.815761 2014-08-13 17:38:02.592805 7fe9bae887a0 20 read_log 194215'590 (194215'125) delete db1f17ea/benchmark_data_storage0111-ib.pocstor_48865_object6852/head//2 by client.2191635.0:34449 2014-08-12 17:17:00.837682 2014-08-13 17:38:02.592827 7fe9bae887a0 20 read_log 194215'591 (194215'126) delete 45f4e1ea/benchmark_data_storage0111-ib.pocstor_48865_object6955/head//2 by client.2191635.0:34552 2014-08-12 17:17:00.868563 2014-08-13 17:38:02.592848 7fe9bae887a0 20 read_log 194215'592 (194215'127) delete 223884ea/benchmark_data_storage0111-ib.pocstor_48865_object7007/head//2 by client.2191635.0:34604 2014-08-12 17:17:00.885514 2014-08-13 17:38:02.592870 7fe9bae887a0 20 read_log 194215'593 (194215'128) delete 84d19fea/benchmark_data_storage0111-ib.pocstor_48865_object7058/head//2 by client.2191635.0:34655 2014-08-12 17:17:00.899884 2014-08-13 17:38:02.592891 7fe9bae887a0 20 read_log 194215'594 (194215'129) delete a64a352a/benchmark_data_storage0111-ib.pocstor_48865_object7111/head//2 by client.2191635.0:34708 2014-08-12 17:17:00.915067 2014-08-13 17:38:02.592925 7fe9bae887a0 20 read_log 194215'595 (194215'130) delete 9abeec2a/benchmark_data_storage0111-ib.pocstor_48865_object7201/head//2 by client.2191635.0:34798 2014-08-12 17:17:00.944448 2014-08-13 17:38:02.592952 7fe9bae887a0 20 read_log 194215'596 (194215'131) delete 2a4d06aa/benchmark_data_storage0111-ib.pocstor_48865_object7255/head//2 by client.2191635.0:34852 2014-08-12 17:17:00.960979 2014-08-13 17:38:02.592974 7fe9bae887a0 20 read_log 194215'597 (194215'132) delete 64ec0faa/benchmark_data_storage0111-ib.pocstor_48865_object7261/head//2 by client.2191635.0:34858 2014-08-12 17:17:00.962247 2014-08-13 17:38:02.592995 7fe9bae887a0 20 read_log 194215'598 (194215'133) delete d703ceaa/benchmark_data_storage0111-ib.pocstor_48865_object7323/head//2 by client.2191635.0:34920 2014-08-12 17:17:00.979774 2014-08-13 17:38:02.593016 7fe9bae887a0 20 read_log 194215'599 (194215'134) delete afc50fea/benchmark_data_storage0111-ib.pocstor_48865_object7361/head//2 by client.2191635.0:34958 2014-08-12 17:17:00.991800 2014-08-13 17:38:02.593036 7fe9bae887a0 20 read_log 194215'600 (194215'135) delete 70d399aa/benchmark_data_storage0111-ib.pocstor_48865_object7567/head//2 by client.2191635.0:35164 2014-08-12 17:17:01.053507 2014-08-13 17:38:02.593058 7fe9bae887a0 20 read_log 194215'601 (194215'136) delete 7e608b6a/benchmark_data_storage0111-ib.pocstor_48865_object7574/head//2 by client.2191635.0:35171 2014-08-12 17:17:01.056899 2014-08-13 17:38:02.593079 7fe9bae887a0 20 read_log 194215'602 (194215'137) delete 55a67f2a/benchmark_data_storage0111-ib.pocstor_48865_object7751/head//2 by client.2191635.0:35348 2014-08-12 17:17:01.115762 2014-08-13 17:38:02.593100 7fe9bae887a0 20 read_log 194215'603 (194215'138) delete c7ab856a/benchmark_data_storage0111-ib.pocstor_48865_object7992/head//2 by client.2191635.0:35589 2014-08-12 17:17:01.198568 2014-08-13 17:38:02.593121 7fe9bae887a0 20 read_log 194215'604 (194215'139) delete 89882f2a/benchmark_data_storage0111-ib.pocstor_48865_object8149/head//2 by client.2191635.0:35746 2014-08-12 17:17:01.245238 2014-08-13 17:38:02.593142 7fe9bae887a0 20 read_log 194215'605 (194215'140) delete 560df9ea/benchmark_data_storage0111-ib.pocstor_48865_object8228/head//2 by client.2191635.0:35825 2014-08-12 17:17:01.265276 2014-08-13 17:38:02.593178 7fe9bae887a0 20 read_log 194215'606 (194215'141) delete 88c4606a/benchmark_data_storage0111-ib.pocstor_48865_object8266/head//2 by client.2191635.0:35863 2014-08-12 17:17:01.277415 2014-08-13 17:38:02.593200 7fe9bae887a0 20 read_log 194215'607 (194215'142) delete addd1f6a/benchmark_data_storage0111-ib.pocstor_48865_object8285/head//2 by client.2191635.0:35882 2014-08-12 17:17:01.282786 2014-08-13 17:38:02.593221 7fe9bae887a0 20 read_log 194215'608 (194215'143) delete 53f2bbaa/benchmark_data_storage0111-ib.pocstor_48865_object8316/head//2 by client.2191635.0:35913 2014-08-12 17:17:01.290901 2014-08-13 17:38:02.593242 7fe9bae887a0 20 read_log 194215'609 (194215'144) delete e60a5caa/benchmark_data_storage0111-ib.pocstor_48865_object8605/head//2 by client.2191635.0:36202 2014-08-12 17:17:01.380864 2014-08-13 17:38:02.593293 7fe9bae887a0 20 read_log 194215'610 (194215'145) delete e9f3b4aa/benchmark_data_storage0111-ib.pocstor_48865_object8698/head//2 by client.2191635.0:36295 2014-08-12 17:17:01.410528 2014-08-13 17:38:02.593316 7fe9bae887a0 20 read_log 194215'611 (194215'146) delete 66d1cb6a/benchmark_data_storage0111-ib.pocstor_48865_object8716/head//2 by client.2191635.0:36313 2014-08-12 17:17:01.415226 2014-08-13 17:38:02.593337 7fe9bae887a0 20 read_log 194215'612 (194215'147) delete fbb9d36a/benchmark_data_storage0111-ib.pocstor_48865_object8783/head//2 by client.2191635.0:36380 2014-08-12 17:17:01.438106 2014-08-13 17:38:02.593358 7fe9bae887a0 20 read_log 194215'613 (194215'148) delete c746816a/benchmark_data_storage0111-ib.pocstor_48865_object8787/head//2 by client.2191635.0:36384 2014-08-12 17:17:01.438574 2014-08-13 17:38:02.593379 7fe9bae887a0 20 read_log 194215'614 (194215'149) delete 3a4a1e6a/benchmark_data_storage0111-ib.pocstor_48865_object8947/head//2 by client.2191635.0:36544 2014-08-12 17:17:01.489975 2014-08-13 17:38:02.593415 7fe9bae887a0 20 read_log 194215'615 (194215'150) delete bc379f6a/benchmark_data_storage0111-ib.pocstor_48865_object9016/head//2 by client.2191635.0:36613 2014-08-12 17:17:01.513775 2014-08-13 17:38:02.593438 7fe9bae887a0 20 read_log 194215'616 (194215'151) delete cb6d2d6a/benchmark_data_storage0111-ib.pocstor_48865_object9125/head//2 by client.2191635.0:36722 2014-08-12 17:17:01.545025 2014-08-13 17:38:02.593459 7fe9bae887a0 20 read_log 194215'617 (194215'152) delete 1ef336ea/benchmark_data_storage0111-ib.pocstor_48865_object9155/head//2 by client.2191635.0:36752 2014-08-12 17:17:01.555386 2014-08-13 17:38:02.593480 7fe9bae887a0 20 read_log 194215'618 (194215'153) delete 7bc5afea/benchmark_data_storage0111-ib.pocstor_48865_object9205/head//2 by client.2191635.0:36802 2014-08-12 17:17:01.571963 2014-08-13 17:38:02.593503 7fe9bae887a0 20 read_log 194215'619 (194215'154) delete 4d0f22ea/benchmark_data_storage0111-ib.pocstor_48865_object9210/head//2 by client.2191635.0:36807 2014-08-12 17:17:01.573478 2014-08-13 17:38:02.593525 7fe9bae887a0 20 read_log 194215'620 (194215'155) delete 33b536ea/benchmark_data_storage0111-ib.pocstor_48865_object9217/head//2 by client.2191635.0:36814 2014-08-12 17:17:01.576447 2014-08-13 17:38:02.593546 7fe9bae887a0 20 read_log 194215'621 (194215'156) delete c09e996a/benchmark_data_storage0111-ib.pocstor_48865_object9321/head//2 by client.2191635.0:36918 2014-08-12 17:17:01.607609 2014-08-13 17:38:02.593567 7fe9bae887a0 20 read_log 194215'622 (194215'157) delete 884b2aa/benchmark_data_storage0111-ib.pocstor_48865_object9349/head//2 by client.2191635.0:36946 2014-08-12 17:17:01.614956 2014-08-13 17:38:02.593588 7fe9bae887a0 20 read_log 194215'623 (194215'158) delete 7bea2ea/benchmark_data_storage0111-ib.pocstor_48865_object9369/head//2 by client.2191635.0:36966 2014-08-12 17:17:01.623012 2014-08-13 17:38:02.593609 7fe9bae887a0 20 read_log 194215'624 (194215'159) delete d092c3ea/benchmark_data_storage0111-ib.pocstor_48865_object9464/head//2 by client.2191635.0:37061 2014-08-12 17:17:01.652005 2014-08-13 17:38:02.593630 7fe9bae887a0 20 read_log 194215'625 (194215'160) delete f2d72e2a/benchmark_data_storage0111-ib.pocstor_48865_object9497/head//2 by client.2191635.0:37094 2014-08-12 17:17:01.661834 2014-08-13 17:38:02.593651 7fe9bae887a0 20 read_log 194215'626 (194215'161) delete 7ed5766a/benchmark_data_storage0111-ib.pocstor_48865_object9512/head//2 by client.2191635.0:37109 2014-08-12 17:17:01.664897 2014-08-13 17:38:02.593672 7fe9bae887a0 20 read_log 194215'627 (194215'162) delete 3108d22a/benchmark_data_storage0111-ib.pocstor_48865_object9561/head//2 by client.2191635.0:37158 2014-08-12 17:17:01.679067 2014-08-13 17:38:02.593693 7fe9bae887a0 20 read_log 194215'628 (194215'163) delete 55a7182a/benchmark_data_storage0111-ib.pocstor_48865_object9618/head//2 by client.2191635.0:37215 2014-08-12 17:17:01.694910 2014-08-13 17:38:02.593714 7fe9bae887a0 20 read_log 194215'629 (194215'164) delete dfc811ea/benchmark_data_storage0111-ib.pocstor_48865_object9687/head//2 by client.2191635.0:37284 2014-08-12 17:17:01.716646 2014-08-13 17:38:02.593738 7fe9bae887a0 20 read_log 194215'630 (194215'165) delete 9b7a8baa/benchmark_data_storage0111-ib.pocstor_48865_object9869/head//2 by client.2191635.0:37466 2014-08-12 17:17:01.771122 2014-08-13 17:38:02.593759 7fe9bae887a0 20 read_log 194215'631 (194215'166) delete 2b1b79ea/benchmark_data_storage0111-ib.pocstor_48865_object9880/head//2 by client.2191635.0:37477 2014-08-12 17:17:01.774093 2014-08-13 17:38:02.593780 7fe9bae887a0 20 read_log 194215'632 (194215'167) delete 21a23aea/benchmark_data_storage0111-ib.pocstor_48865_object9933/head//2 by client.2191635.0:37530 2014-08-12 17:17:01.789386 2014-08-13 17:38:02.593801 7fe9bae887a0 20 read_log 194215'633 (194215'168) delete b5c1d3aa/benchmark_data_storage0111-ib.pocstor_48865_object10008/head//2 by client.2191635.0:37605 2014-08-12 17:17:01.809652 2014-08-13 17:38:02.593822 7fe9bae887a0 20 read_log 194215'634 (194215'169) delete 87325caa/benchmark_data_storage0111-ib.pocstor_48865_object10072/head//2 by client.2191635.0:37669 2014-08-12 17:17:01.826989 2014-08-13 17:38:02.593884 7fe9bae887a0 20 read_log 194215'635 (194215'170) delete a5bef9aa/benchmark_data_storage0111-ib.pocstor_48865_object10123/head//2 by client.2191635.0:37720 2014-08-12 17:17:01.840452 2014-08-13 17:38:02.593908 7fe9bae887a0 20 read_log 194215'636 (194215'171) delete 839ef8aa/benchmark_data_storage0111-ib.pocstor_48865_object10189/head//2 by client.2191635.0:37786 2014-08-12 17:17:01.858320 2014-08-13 17:38:02.593930 7fe9bae887a0 20 read_log 194215'637 (194215'172) delete a72c32aa/benchmark_data_storage0111-ib.pocstor_48865_object10248/head//2 by client.2191635.0:37845 2014-08-12 17:17:01.875657 2014-08-13 17:38:02.593971 7fe9bae887a0 20 read_log 194215'638 (194215'173) delete 3759752a/benchmark_data_storage0111-ib.pocstor_48865_object10253/head//2 by client.2191635.0:37850 2014-08-12 17:17:01.877538 2014-08-13 17:38:02.593993 7fe9bae887a0 20 read_log 194215'639 (194215'174) delete 83b5eaa/benchmark_data_storage0111-ib.pocstor_48865_object10313/head//2 by client.2191635.0:37910 2014-08-12 17:17:01.895381 2014-08-13 17:38:02.594015 7fe9bae887a0 20 read_log 194215'640 (194215'175) delete 6a9613aa/benchmark_data_storage0111-ib.pocstor_48865_object10396/head//2 by client.2191635.0:37993 2014-08-12 17:17:01.921356 2014-08-13 17:38:02.594036 7fe9bae887a0 20 read_log 194215'641 (194215'176) delete b37aadaa/benchmark_data_storage0111-ib.pocstor_48865_object10453/head//2 by client.2191635.0:38050 2014-08-12 17:17:01.937071 2014-08-13 17:38:02.594057 7fe9bae887a0 20 read_log 194215'642 (194215'177) delete 292338aa/benchmark_data_storage0111-ib.pocstor_48865_object10482/head//2 by client.2191635.0:38079 2014-08-12 17:17:01.944349 2014-08-13 17:38:02.594079 7fe9bae887a0 20 read_log 194215'643 (194215'178) delete cb33af2a/benchmark_data_storage0111-ib.pocstor_48865_object10644/head//2 by client.2191635.0:38241 2014-08-12 17:17:01.994035 2014-08-13 17:38:02.594100 7fe9bae887a0 20 read_log 194215'644 (194215'179) delete 1d50792a/benchmark_data_storage0111-ib.pocstor_48865_object10690/head//2 by client.2191635.0:38287 2014-08-12 17:17:02.008780 2014-08-13 17:38:02.594121 7fe9bae887a0 20 read_log 194215'645 (194215'180) delete f9ca0fea/benchmark_data_storage0111-ib.pocstor_48865_object10730/head//2 by client.2191635.0:38327 2014-08-12 17:17:02.019746 2014-08-13 17:38:02.594142 7fe9bae887a0 20 read_log 194215'646 (194215'181) delete a596172a/benchmark_data_storage0111-ib.pocstor_48865_object10814/head//2 by client.2191635.0:38411 2014-08-12 17:17:02.046679 2014-08-13 17:38:02.594172 7fe9bae887a0 20 read_log 194215'647 (194215'182) delete 2e3f30aa/benchmark_data_storage0111-ib.pocstor_48865_object10886/head//2 by client.2191635.0:38483 2014-08-12 17:17:02.071115 2014-08-13 17:38:02.594194 7fe9bae887a0 20 read_log 194215'648 (194215'183) delete 3359f32a/benchmark_data_storage0111-ib.pocstor_48865_object10910/head//2 by client.2191635.0:38507 2014-08-12 17:17:02.077350 2014-08-13 17:38:02.594220 7fe9bae887a0 20 read_log 194215'649 (194215'184) delete d5ffa2a/benchmark_data_storage0111-ib.pocstor_48865_object10933/head//2 by client.2191635.0:38530 2014-08-12 17:17:02.085659 2014-08-13 17:38:02.594242 7fe9bae887a0 20 read_log 194215'650 (194215'185) delete 5db5596a/benchmark_data_storage0111-ib.pocstor_48865_object10936/head//2 by client.2191635.0:38533 2014-08-12 17:17:02.086218 2014-08-13 17:38:02.594264 7fe9bae887a0 20 read_log 194215'651 (194215'186) delete aa36ecaa/benchmark_data_storage0111-ib.pocstor_48865_object10979/head//2 by client.2191635.0:38576 2014-08-12 17:17:02.098229 2014-08-13 17:38:02.594285 7fe9bae887a0 20 read_log 194215'652 (194215'187) delete 7039232a/benchmark_data_storage0111-ib.pocstor_48865_object11094/head//2 by client.2191635.0:38691 2014-08-12 17:17:02.131366 2014-08-13 17:38:02.594307 7fe9bae887a0 20 read_log 194215'653 (194215'188) delete 5b3c0d2a/benchmark_data_storage0111-ib.pocstor_48865_object11111/head//2 by client.2191635.0:38708 2014-08-12 17:17:02.136343 2014-08-13 17:38:02.594328 7fe9bae887a0 20 read_log 194215'654 (194215'189) delete 3f026e2a/benchmark_data_storage0111-ib.pocstor_48865_object11117/head//2 by client.2191635.0:38714 2014-08-12 17:17:02.137690 2014-08-13 17:38:02.594362 7fe9bae887a0 20 read_log 194215'655 (194215'190) delete 6072102a/benchmark_data_storage0111-ib.pocstor_48865_object11176/head//2 by client.2191635.0:38773 2014-08-12 17:17:02.156822 2014-08-13 17:38:02.594384 7fe9bae887a0 20 read_log 194215'656 (194215'191) delete b6ed77aa/benchmark_data_storage0111-ib.pocstor_48865_object11187/head//2 by client.2191635.0:38784 2014-08-12 17:17:02.160391 2014-08-13 17:38:02.594405 7fe9bae887a0 20 read_log 194215'657 (194215'192) delete 205c17aa/benchmark_data_storage0111-ib.pocstor_48865_object11236/head//2 by client.2191635.0:38833 2014-08-12 17:17:02.178340 2014-08-13 17:38:02.594427 7fe9bae887a0 20 read_log 194215'658 (194215'193) delete 8d426cea/benchmark_data_storage0111-ib.pocstor_48865_object11349/head//2 by client.2191635.0:38946 2014-08-12 17:17:02.211427 2014-08-13 17:38:02.594448 7fe9bae887a0 20 read_log 194215'659 (194215'194) delete d297416a/benchmark_data_storage0111-ib.pocstor_48865_object11397/head//2 by client.2191635.0:38994 2014-08-12 17:17:02.230013 2014-08-13 17:38:02.594469 7fe9bae887a0 20 read_log 194215'660 (194215'195) delete d6cefa2a/benchmark_data_storage0111-ib.pocstor_48865_object11498/head//2 by client.2191635.0:39095 2014-08-12 17:17:02.263973 2014-08-13 17:38:02.594491 7fe9bae887a0 20 read_log 194215'661 (194215'196) delete 53e842ea/benchmark_data_storage0111-ib.pocstor_48865_object11552/head//2 by client.2191635.0:39149 2014-08-12 17:17:02.280025 2014-08-13 17:38:02.594512 7fe9bae887a0 20 read_log 194215'662 (194215'197) delete 6030336a/benchmark_data_storage0111-ib.pocstor_48865_object11556/head//2 by client.2191635.0:39153 2014-08-12 17:17:02.282002 2014-08-13 17:38:02.594533 7fe9bae887a0 20 read_log 194215'663 (194215'198) delete 89d46bea/benchmark_data_storage0111-ib.pocstor_48865_object11776/head//2 by client.2191635.0:39373 2014-08-12 17:17:02.344881 2014-08-13 17:38:02.594554 7fe9bae887a0 20 read_log 194215'664 (194215'199) delete 35ffcaa/benchmark_data_storage0111-ib.pocstor_48865_object11861/head//2 by client.2191635.0:39458 2014-08-12 17:17:02.367192 2014-08-13 17:38:02.594575 7fe9bae887a0 20 read_log 194215'665 (194215'200) delete 5b650a2a/benchmark_data_storage0111-ib.pocstor_48865_object11920/head//2 by client.2191635.0:39517 2014-08-12 17:17:02.384450 2014-08-13 17:38:02.594596 7fe9bae887a0 20 read_log 194215'666 (194215'201) delete 4a1b016a/benchmark_data_storage0111-ib.pocstor_48865_object12014/head//2 by client.2191635.0:39611 2014-08-12 17:17:02.409950 2014-08-13 17:38:02.594617 7fe9bae887a0 20 read_log 194215'667 (194215'202) delete 6c2c76aa/benchmark_data_storage0111-ib.pocstor_48865_object12017/head//2 by client.2191635.0:39614 2014-08-12 17:17:02.411267 2014-08-13 17:38:02.594638 7fe9bae887a0 20 read_log 194215'668 (194215'203) delete b1b052a/benchmark_data_storage0111-ib.pocstor_48865_object12039/head//2 by client.2191635.0:39636 2014-08-12 17:17:02.417871 2014-08-13 17:38:02.594659 7fe9bae887a0 20 read_log 194215'669 (194215'204) delete 18d92c2a/benchmark_data_storage0111-ib.pocstor_48865_object12137/head//2 by client.2191635.0:39734 2014-08-12 17:17:02.447524 2014-08-13 17:38:02.594680 7fe9bae887a0 20 read_log 194215'670 (194215'205) delete e293eb6a/benchmark_data_storage0111-ib.pocstor_48865_object12200/head//2 by client.2191635.0:39797 2014-08-12 17:17:02.466436 2014-08-13 17:38:02.594701 7fe9bae887a0 20 read_log 194215'671 (194215'206) delete aec82caa/benchmark_data_storage0111-ib.pocstor_48865_object12205/head//2 by client.2191635.0:39802 2014-08-12 17:17:02.467324 2014-08-13 17:38:02.594725 7fe9bae887a0 20 read_log 194215'672 (194215'207) delete e4c381ea/benchmark_data_storage0111-ib.pocstor_48865_object12239/head//2 by client.2191635.0:39836 2014-08-12 17:17:02.480375 2014-08-13 17:38:02.594747 7fe9bae887a0 20 read_log 194215'673 (194215'208) delete 5597feaa/benchmark_data_storage0111-ib.pocstor_48865_object12386/head//2 by client.2191635.0:39983 2014-08-12 17:17:02.527260 2014-08-13 17:38:02.594768 7fe9bae887a0 20 read_log 194215'674 (194215'209) delete 43f2212a/benchmark_data_storage0111-ib.pocstor_48865_object12393/head//2 by client.2191635.0:39990 2014-08-12 17:17:02.528065 2014-08-13 17:38:02.594804 7fe9bae887a0 20 read_log 194215'675 (194215'210) delete 2b5bb7ea/benchmark_data_storage0111-ib.pocstor_48865_object12418/head//2 by client.2191635.0:40015 2014-08-12 17:17:02.538756 2014-08-13 17:38:02.594827 7fe9bae887a0 20 read_log 194215'676 (194215'211) delete b49f39aa/benchmark_data_storage0111-ib.pocstor_48865_object12547/head//2 by client.2191635.0:40144 2014-08-12 17:17:02.575749 2014-08-13 17:38:02.594849 7fe9bae887a0 20 read_log 194215'677 (194215'212) delete affb68aa/benchmark_data_storage0111-ib.pocstor_48865_object12599/head//2 by client.2191635.0:40196 2014-08-12 17:17:02.594413 2014-08-13 17:38:02.594870 7fe9bae887a0 20 read_log 194215'678 (194215'213) delete 2e68e72a/benchmark_data_storage0111-ib.pocstor_48865_object12623/head//2 by client.2191635.0:40220 2014-08-12 17:17:02.602057 2014-08-13 17:38:02.594891 7fe9bae887a0 20 read_log 194215'679 (194215'214) delete b033de6a/benchmark_data_storage0111-ib.pocstor_48865_object12628/head//2 by client.2191635.0:40225 2014-08-12 17:17:02.602276 2014-08-13 17:38:02.594912 7fe9bae887a0 20 read_log 194215'680 (194215'215) delete ae98b7ea/benchmark_data_storage0111-ib.pocstor_48865_object12659/head//2 by client.2191635.0:40256 2014-08-12 17:17:02.612988 2014-08-13 17:38:02.594933 7fe9bae887a0 20 read_log 194215'681 (194215'216) delete 2aaf702a/benchmark_data_storage0111-ib.pocstor_48865_object12668/head//2 by client.2191635.0:40265 2014-08-12 17:17:02.615687 2014-08-13 17:38:02.594954 7fe9bae887a0 20 read_log 194215'682 (194215'217) delete f57663aa/benchmark_data_storage0111-ib.pocstor_48865_object12750/head//2 by client.2191635.0:40347 2014-08-12 17:17:02.642122 2014-08-13 17:38:02.594975 7fe9bae887a0 20 read_log 194215'683 (194215'218) delete f4212caa/benchmark_data_storage0111-ib.pocstor_48865_object12838/head//2 by client.2191635.0:40435 2014-08-12 17:17:02.666168 2014-08-13 17:38:02.594996 7fe9bae887a0 20 read_log 194215'684 (194215'219) delete 7240e7aa/benchmark_data_storage0111-ib.pocstor_48865_object12999/head//2 by client.2191635.0:40596 2014-08-12 17:17:02.718920 2014-08-13 17:38:02.595017 7fe9bae887a0 20 read_log 194215'685 (194215'220) delete 667d846a/benchmark_data_storage0111-ib.pocstor_48865_object13090/head//2 by client.2191635.0:40687 2014-08-12 17:17:02.746435 2014-08-13 17:38:02.595038 7fe9bae887a0 20 read_log 194215'686 (194215'221) delete 4728a26a/benchmark_data_storage0111-ib.pocstor_48865_object13104/head//2 by client.2191635.0:40701 2014-08-12 17:17:02.750471 2014-08-13 17:38:02.595059 7fe9bae887a0 20 read_log 194215'687 (194215'222) delete 6830202a/benchmark_data_storage0111-ib.pocstor_48865_object13121/head//2 by client.2191635.0:40718 2014-08-12 17:17:02.755839 2014-08-13 17:38:02.595081 7fe9bae887a0 20 read_log 194215'688 (194215'223) delete e698f6ea/benchmark_data_storage0111-ib.pocstor_48865_object13189/head//2 by client.2191635.0:40786 2014-08-12 17:17:02.774956 2014-08-13 17:38:02.595102 7fe9bae887a0 20 read_log 194215'689 (194215'224) delete 1d7d93ea/benchmark_data_storage0111-ib.pocstor_48865_object13262/head//2 by client.2191635.0:40859 2014-08-12 17:17:02.798534 2014-08-13 17:38:02.595123 7fe9bae887a0 20 read_log 194215'690 (194215'225) delete 4a9e062a/benchmark_data_storage0111-ib.pocstor_48865_object13318/head//2 by client.2191635.0:40915 2014-08-12 17:17:02.815048 2014-08-13 17:38:02.595144 7fe9bae887a0 20 read_log 194215'691 (194215'226) delete 71b2d72a/benchmark_data_storage0111-ib.pocstor_48865_object13371/head//2 by client.2191635.0:40968 2014-08-12 17:17:02.830144 2014-08-13 17:38:02.595172 7fe9bae887a0 20 read_log 194215'692 (194215'227) delete 4abb7a2a/benchmark_data_storage0111-ib.pocstor_48865_object13374/head//2 by client.2191635.0:40971 2014-08-12 17:17:02.830829 2014-08-13 17:38:02.595194 7fe9bae887a0 20 read_log 194215'693 (194215'228) delete 8e43f9aa/benchmark_data_storage0111-ib.pocstor_48865_object13392/head//2 by client.2191635.0:40989 2014-08-12 17:17:02.835994 2014-08-13 17:38:02.595216 7fe9bae887a0 20 read_log 194215'694 (194215'229) delete 4f37bbea/benchmark_data_storage0111-ib.pocstor_48865_object13561/head//2 by client.2191635.0:41158 2014-08-12 17:17:02.886568 2014-08-13 17:38:02.595249 7fe9bae887a0 20 read_log 194215'695 (194215'230) delete 19487bea/benchmark_data_storage0111-ib.pocstor_48865_object13635/head//2 by client.2191635.0:41232 2014-08-12 17:17:02.907478 2014-08-13 17:38:02.595271 7fe9bae887a0 20 read_log 194215'696 (194215'231) delete dd220cea/benchmark_data_storage0111-ib.pocstor_48865_object13725/head//2 by client.2191635.0:41322 2014-08-12 17:17:02.933577 2014-08-13 17:38:02.595292 7fe9bae887a0 20 read_log 194215'697 (194215'232) delete 9440cd2a/benchmark_data_storage0111-ib.pocstor_48865_object13735/head//2 by client.2191635.0:41332 2014-08-12 17:17:02.936490 2014-08-13 17:38:02.595316 7fe9bae887a0 20 read_log 194215'698 (194215'233) delete d720b2a/benchmark_data_storage0111-ib.pocstor_48865_object13783/head//2 by client.2191635.0:41380 2014-08-12 17:17:02.951374 2014-08-13 17:38:02.595339 7fe9bae887a0 20 read_log 194215'699 (194215'234) delete c9345c2a/benchmark_data_storage0111-ib.pocstor_48865_object13852/head//2 by client.2191635.0:41449 2014-08-12 17:17:02.972106 2014-08-13 17:38:02.595364 7fe9bae887a0 20 read_log 194215'700 (194215'235) delete ea0b76a/benchmark_data_storage0111-ib.pocstor_48865_object13973/head//2 by client.2191635.0:41570 2014-08-12 17:17:03.012603 2014-08-13 17:38:02.595386 7fe9bae887a0 20 read_log 194215'701 (194215'236) delete 2a0c802a/benchmark_data_storage0111-ib.pocstor_48865_object14145/head//2 by client.2191635.0:41742 2014-08-12 17:17:03.062495 2014-08-13 17:38:02.595407 7fe9bae887a0 20 read_log 194215'702 (194215'237) delete 63208a6a/benchmark_data_storage0111-ib.pocstor_48865_object14286/head//2 by client.2191635.0:41883 2014-08-12 17:17:03.102690 2014-08-13 17:38:02.595457 7fe9bae887a0 20 read_log 194215'703 (194215'238) delete 40f6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object14317/head//2 by client.2191635.0:41914 2014-08-12 17:17:03.112314 2014-08-13 17:38:02.595480 7fe9bae887a0 20 read_log 194215'704 (194215'239) delete 41de21ea/benchmark_data_storage0111-ib.pocstor_48865_object14598/head//2 by client.2191635.0:42195 2014-08-12 17:17:03.199725 2014-08-13 17:38:02.595501 7fe9bae887a0 20 read_log 194215'705 (194215'240) delete b8a8542a/benchmark_data_storage0111-ib.pocstor_48865_object14672/head//2 by client.2191635.0:42269 2014-08-12 17:17:03.220543 2014-08-13 17:38:02.595522 7fe9bae887a0 20 read_log 194215'706 (194215'241) delete 4a6306aa/benchmark_data_storage0111-ib.pocstor_48865_object14713/head//2 by client.2191635.0:42310 2014-08-12 17:17:03.231459 2014-08-13 17:38:02.595543 7fe9bae887a0 20 read_log 194215'707 (194215'242) delete 2991aaaa/benchmark_data_storage0111-ib.pocstor_48865_object14741/head//2 by client.2191635.0:42338 2014-08-12 17:17:03.239683 2014-08-13 17:38:02.595565 7fe9bae887a0 20 read_log 194215'708 (194215'243) delete 565f662a/benchmark_data_storage0111-ib.pocstor_48865_object14767/head//2 by client.2191635.0:42364 2014-08-12 17:17:03.248980 2014-08-13 17:38:02.595586 7fe9bae887a0 20 read_log 194215'709 (194215'244) delete 5192f8aa/benchmark_data_storage0111-ib.pocstor_48865_object14776/head//2 by client.2191635.0:42373 2014-08-12 17:17:03.250696 2014-08-13 17:38:02.595607 7fe9bae887a0 20 read_log 194215'710 (194215'245) delete eeea566a/benchmark_data_storage0111-ib.pocstor_48865_object14778/head//2 by client.2191635.0:42375 2014-08-12 17:17:03.251470 2014-08-13 17:38:02.595628 7fe9bae887a0 20 read_log 194215'711 (194215'246) delete 8558d5aa/benchmark_data_storage0111-ib.pocstor_48865_object14801/head//2 by client.2191635.0:42398 2014-08-12 17:17:03.258463 2014-08-13 17:38:02.595650 7fe9bae887a0 20 read_log 194215'712 (194215'247) delete 9d0eaaea/benchmark_data_storage0111-ib.pocstor_48865_object14866/head//2 by client.2191635.0:42463 2014-08-12 17:17:03.278564 2014-08-13 17:38:02.595670 7fe9bae887a0 20 read_log 194215'713 (194215'248) delete 305b2a6a/benchmark_data_storage0111-ib.pocstor_48865_object14950/head//2 by client.2191635.0:42547 2014-08-12 17:17:03.304029 2014-08-13 17:38:02.595726 7fe9bae887a0 20 read_log 194215'714 (194215'249) delete 731b0e6a/benchmark_data_storage0111-ib.pocstor_48865_object15136/head//2 by client.2191635.0:42733 2014-08-12 17:17:03.358968 2014-08-13 17:38:02.595761 7fe9bae887a0 20 read_log 194215'715 (194215'250) delete 65f3ae6a/benchmark_data_storage0111-ib.pocstor_48865_object15161/head//2 by client.2191635.0:42758 2014-08-12 17:17:03.364942 2014-08-13 17:38:02.595783 7fe9bae887a0 20 read_log 194215'716 (194215'251) delete 77250b2a/benchmark_data_storage0111-ib.pocstor_48865_object15203/head//2 by client.2191635.0:42800 2014-08-12 17:17:03.376741 2014-08-13 17:38:02.595804 7fe9bae887a0 20 read_log 194215'717 (194215'252) delete daba3e6a/benchmark_data_storage0111-ib.pocstor_48865_object15306/head//2 by client.2191635.0:42903 2014-08-12 17:17:03.409742 2014-08-13 17:38:02.595826 7fe9bae887a0 20 read_log 194215'718 (194215'253) delete 8338bcea/benchmark_data_storage0111-ib.pocstor_48865_object15415/head//2 by client.2191635.0:43012 2014-08-12 17:17:03.442888 2014-08-13 17:38:02.595847 7fe9bae887a0 20 read_log 194215'719 (194215'254) delete 79bf0b6a/benchmark_data_storage0111-ib.pocstor_48865_object15441/head//2 by client.2191635.0:43038 2014-08-12 17:17:03.451008 2014-08-13 17:38:02.595868 7fe9bae887a0 20 read_log 194215'720 (194215'255) delete 22e21cea/benchmark_data_storage0111-ib.pocstor_48865_object15451/head//2 by client.2191635.0:43048 2014-08-12 17:17:03.454459 2014-08-13 17:38:02.595889 7fe9bae887a0 20 read_log 194215'721 (194215'256) delete 722c47ea/benchmark_data_storage0111-ib.pocstor_48865_object15566/head//2 by client.2191635.0:43163 2014-08-12 17:17:03.487870 2014-08-13 17:38:02.595910 7fe9bae887a0 20 read_log 194215'722 (194215'257) delete a865f92a/benchmark_data_storage0111-ib.pocstor_48865_object15671/head//2 by client.2191635.0:43268 2014-08-12 17:17:03.522171 2014-08-13 17:38:02.595931 7fe9bae887a0 20 read_log 194215'723 (194215'258) delete eb9d4f6a/benchmark_data_storage0111-ib.pocstor_48865_object15686/head//2 by client.2191635.0:43283 2014-08-12 17:17:03.526607 2014-08-13 17:38:02.595952 7fe9bae887a0 20 read_log 194215'724 (194215'259) delete bfafc2a/benchmark_data_storage0111-ib.pocstor_48865_object15765/head//2 by client.2191635.0:43362 2014-08-12 17:17:03.549829 2014-08-13 17:38:02.595973 7fe9bae887a0 20 read_log 194215'725 (194215'260) delete c4918aea/benchmark_data_storage0111-ib.pocstor_48865_object15774/head//2 by client.2191635.0:43371 2014-08-12 17:17:03.553490 2014-08-13 17:38:02.595994 7fe9bae887a0 20 read_log 194215'726 (194215'261) delete 577cf22a/benchmark_data_storage0111-ib.pocstor_48865_object15886/head//2 by client.2191635.0:43483 2014-08-12 17:17:03.586559 2014-08-13 17:38:02.596016 7fe9bae887a0 20 read_log 194215'727 (194215'262) delete 47b95a6a/benchmark_data_storage0111-ib.pocstor_48865_object15917/head//2 by client.2191635.0:43514 2014-08-12 17:17:03.595653 2014-08-13 17:38:02.596036 7fe9bae887a0 20 read_log 194215'728 (194215'263) delete d8a6b6ea/benchmark_data_storage0111-ib.pocstor_48865_object15948/head//2 by client.2191635.0:43545 2014-08-12 17:17:03.605682 2014-08-13 17:38:02.596057 7fe9bae887a0 20 read_log 194215'729 (194215'264) delete b36d1d6a/benchmark_data_storage0111-ib.pocstor_48865_object15982/head//2 by client.2191635.0:43579 2014-08-12 17:17:03.615985 2014-08-13 17:38:02.596078 7fe9bae887a0 20 read_log 194215'730 (194215'265) delete 45f476ea/benchmark_data_storage0111-ib.pocstor_48865_object16063/head//2 by client.2191635.0:43660 2014-08-12 17:17:03.638650 2014-08-13 17:38:02.596099 7fe9bae887a0 20 read_log 194215'731 (194215'266) delete c3d0156a/benchmark_data_storage0111-ib.pocstor_48865_object16141/head//2 by client.2191635.0:43738 2014-08-12 17:17:03.660512 2014-08-13 17:38:02.596120 7fe9bae887a0 20 read_log 194215'732 (194215'267) delete 81a3d32a/benchmark_data_storage0111-ib.pocstor_48865_object16150/head//2 by client.2191635.0:43747 2014-08-12 17:17:03.663635 2014-08-13 17:38:02.596141 7fe9bae887a0 20 read_log 194215'733 (194215'268) delete 8c0d182a/benchmark_data_storage0111-ib.pocstor_48865_object16296/head//2 by client.2191635.0:43893 2014-08-12 17:17:03.707017 2014-08-13 17:38:02.596169 7fe9bae887a0 20 read_log 194215'734 (194215'269) delete a521d6aa/benchmark_data_storage0111-ib.pocstor_48865_object16315/head//2 by client.2191635.0:43912 2014-08-12 17:17:03.712091 2014-08-13 17:38:02.596208 7fe9bae887a0 20 read_log 194215'735 (194215'270) delete df2ea2ea/benchmark_data_storage0111-ib.pocstor_48865_object16352/head//2 by client.2191635.0:43949 2014-08-12 17:17:03.724378 2014-08-13 17:38:02.596231 7fe9bae887a0 20 read_log 194215'736 (194215'271) delete 826376aa/benchmark_data_storage0111-ib.pocstor_48865_object16361/head//2 by client.2191635.0:43958 2014-08-12 17:17:03.725546 2014-08-13 17:38:02.596252 7fe9bae887a0 20 read_log 194215'737 (194215'272) delete 6afdb52a/benchmark_data_storage0111-ib.pocstor_48865_object16427/head//2 by client.2191635.0:44024 2014-08-12 17:17:03.744165 2014-08-13 17:38:02.596273 7fe9bae887a0 20 read_log 194215'738 (194215'273) delete bc7adeaa/benchmark_data_storage0111-ib.pocstor_48865_object16515/head//2 by client.2191635.0:44112 2014-08-12 17:17:03.770449 2014-08-13 17:38:02.596294 7fe9bae887a0 20 read_log 194215'739 (194215'274) delete ccb6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object16564/head//2 by client.2191635.0:44161 2014-08-12 17:17:03.783113 2014-08-13 17:38:02.596315 7fe9bae887a0 20 read_log 194215'740 (194215'275) delete 7f37d7aa/benchmark_data_storage0111-ib.pocstor_48865_object16565/head//2 by client.2191635.0:44162 2014-08-12 17:17:03.783180 2014-08-13 17:38:02.596336 7fe9bae887a0 20 read_log 194215'741 (194215'276) delete 8cf4382a/benchmark_data_storage0111-ib.pocstor_48865_object16604/head//2 by client.2191635.0:44201 2014-08-12 17:17:03.794113 2014-08-13 17:38:02.596357 7fe9bae887a0 20 read_log 194215'742 (194215'277) delete fbdbd5aa/benchmark_data_storage0111-ib.pocstor_48865_object16791/head//2 by client.2191635.0:44388 2014-08-12 17:17:03.848527 2014-08-13 17:38:02.596378 7fe9bae887a0 20 read_log 194215'743 (194215'278) delete 46242aea/benchmark_data_storage0111-ib.pocstor_48865_object16862/head//2 by client.2191635.0:44459 2014-08-12 17:17:03.870692 2014-08-13 17:38:02.596399 7fe9bae887a0 20 read_log 194215'744 (194215'279) delete 8e0b2d2a/benchmark_data_storage0111-ib.pocstor_48865_object16863/head//2 by client.2191635.0:44460 2014-08-12 17:17:03.871189 2014-08-13 17:38:02.596420 7fe9bae887a0 20 read_log 194215'745 (194215'280) delete d8478eea/benchmark_data_storage0111-ib.pocstor_48865_object16870/head//2 by client.2191635.0:44467 2014-08-12 17:17:03.872842 2014-08-13 17:38:02.596442 7fe9bae887a0 20 read_log 194215'746 (194215'281) delete 23a472a/benchmark_data_storage0111-ib.pocstor_48865_object16907/head//2 by client.2191635.0:44504 2014-08-12 17:17:03.882741 2014-08-13 17:38:02.596493 7fe9bae887a0 20 read_log 194215'747 (194215'282) delete 861c596a/benchmark_data_storage0111-ib.pocstor_48865_object16946/head//2 by client.2191635.0:44543 2014-08-12 17:17:03.895417 2014-08-13 17:38:02.596514 7fe9bae887a0 20 read_log 194215'748 (194215'283) delete 70e3cf2a/benchmark_data_storage0111-ib.pocstor_48865_object16999/head//2 by client.2191635.0:44596 2014-08-12 17:17:03.910478 2014-08-13 17:38:02.596536 7fe9bae887a0 20 read_log 194215'749 (194215'284) delete 66eb7d2a/benchmark_data_storage0111-ib.pocstor_48865_object17022/head//2 by client.2191635.0:44619 2014-08-12 17:17:03.918986 2014-08-13 17:38:02.596557 7fe9bae887a0 20 read_log 194215'750 (194215'285) delete a97b1aa/benchmark_data_storage0111-ib.pocstor_48865_object17041/head//2 by client.2191635.0:44638 2014-08-12 17:17:03.923117 2014-08-13 17:38:02.596578 7fe9bae887a0 20 read_log 194215'751 (194215'286) delete 36b9c7ea/benchmark_data_storage0111-ib.pocstor_48865_object17067/head//2 by client.2191635.0:44664 2014-08-12 17:17:03.931740 2014-08-13 17:38:02.596599 7fe9bae887a0 20 read_log 194215'752 (194215'287) delete d2246d2a/benchmark_data_storage0111-ib.pocstor_48865_object17159/head//2 by client.2191635.0:44756 2014-08-12 17:17:03.959638 2014-08-13 17:38:02.596620 7fe9bae887a0 20 read_log 194215'753 (194215'288) delete 9e6f64ea/benchmark_data_storage0111-ib.pocstor_48865_object17166/head//2 by client.2191635.0:44763 2014-08-12 17:17:03.960583 2014-08-13 17:38:02.596641 7fe9bae887a0 20 read_log 194215'754 (194215'289) delete 1ce7ab2a/benchmark_data_storage0111-ib.pocstor_48865_object17300/head//2 by client.2191635.0:44897 2014-08-12 17:17:04.000255 2014-08-13 17:38:02.596674 7fe9bae887a0 20 read_log 194215'755 (194215'290) delete 93cde52a/benchmark_data_storage0111-ib.pocstor_48865_object17356/head//2 by client.2191635.0:44953 2014-08-12 17:17:04.017759 2014-08-13 17:38:02.596697 7fe9bae887a0 20 read_log 194215'756 (194215'291) delete 6543b36a/benchmark_data_storage0111-ib.pocstor_48865_object17395/head//2 by client.2191635.0:44992 2014-08-12 17:17:04.027964 2014-08-13 17:38:02.596718 7fe9bae887a0 20 read_log 194215'757 (194215'292) delete 907fe66a/benchmark_data_storage0111-ib.pocstor_48865_object17420/head//2 by client.2191635.0:45017 2014-08-12 17:17:04.034809 2014-08-13 17:38:02.596739 7fe9bae887a0 20 read_log 194215'758 (194215'293) delete f04168aa/benchmark_data_storage0111-ib.pocstor_48865_object17493/head//2 by client.2191635.0:45090 2014-08-12 17:17:04.058589 2014-08-13 17:38:02.596761 7fe9bae887a0 20 read_log 194215'759 (194215'294) delete 3dd28a2a/benchmark_data_storage0111-ib.pocstor_48865_object17533/head//2 by client.2191635.0:45130 2014-08-12 17:17:04.070768 2014-08-13 17:38:02.596782 7fe9bae887a0 20 read_log 194215'760 (194215'295) delete 1e4657ea/benchmark_data_storage0111-ib.pocstor_48865_object17612/head//2 by client.2191635.0:45209 2014-08-12 17:17:04.098224 2014-08-13 17:38:02.596803 7fe9bae887a0 20 read_log 194215'761 (194215'296) delete 6041b6ea/benchmark_data_storage0111-ib.pocstor_48865_object17635/head//2 by client.2191635.0:45232 2014-08-12 17:17:04.107833 2014-08-13 17:38:02.596824 7fe9bae887a0 20 read_log 194215'762 (194215'297) delete 224e726a/benchmark_data_storage0111-ib.pocstor_48865_object17676/head//2 by client.2191635.0:45273 2014-08-12 17:17:04.120504 2014-08-13 17:38:02.596845 7fe9bae887a0 20 read_log 194215'763 (194215'298) delete b888422a/benchmark_data_storage0111-ib.pocstor_48865_object17741/head//2 by client.2191635.0:45338 2014-08-12 17:17:04.141765 2014-08-13 17:38:02.596866 7fe9bae887a0 20 read_log 194215'764 (194215'299) delete 6161082a/benchmark_data_storage0111-ib.pocstor_48865_object17851/head//2 by client.2191635.0:45448 2014-08-12 17:17:04.171589 2014-08-13 17:38:02.596887 7fe9bae887a0 20 read_log 194215'765 (194215'300) delete f91dcc2a/benchmark_data_storage0111-ib.pocstor_48865_object18013/head//2 by client.2191635.0:45610 2014-08-12 17:17:04.224937 2014-08-13 17:38:02.596908 7fe9bae887a0 20 read_log 194215'766 (194215'301) delete d714946a/benchmark_data_storage0111-ib.pocstor_48865_object18071/head//2 by client.2191635.0:45668 2014-08-12 17:17:04.243752 2014-08-13 17:38:02.596929 7fe9bae887a0 20 read_log 194215'767 (194215'302) delete 76da69aa/benchmark_data_storage0111-ib.pocstor_48865_object18175/head//2 by client.2191635.0:45772 2014-08-12 17:17:04.276736 2014-08-13 17:38:02.596952 7fe9bae887a0 20 read_log 194215'768 (194215'303) delete ca0bf7ea/benchmark_data_storage0111-ib.pocstor_48865_object18353/head//2 by client.2191635.0:45950 2014-08-12 17:17:04.334746 2014-08-13 17:38:02.596973 7fe9bae887a0 20 read_log 194215'769 (194215'304) delete cb5fc6ea/benchmark_data_storage0111-ib.pocstor_48865_object18362/head//2 by client.2191635.0:45959 2014-08-12 17:17:04.338043 2014-08-13 17:38:02.596997 7fe9bae887a0 20 read_log 194215'770 (194215'305) delete ba58eaaa/benchmark_data_storage0111-ib.pocstor_48865_object18438/head//2 by client.2191635.0:46035 2014-08-12 17:17:04.363405 2014-08-13 17:38:02.597018 7fe9bae887a0 20 read_log 194215'771 (194215'306) delete 774fb76a/benchmark_data_storage0111-ib.pocstor_48865_object18602/head//2 by client.2191635.0:46199 2014-08-12 17:17:04.414922 2014-08-13 17:38:02.597040 7fe9bae887a0 20 read_log 194215'772 (194215'307) delete 3f75e6a/benchmark_data_storage0111-ib.pocstor_48865_object18627/head//2 by client.2191635.0:46224 2014-08-12 17:17:04.423816 2014-08-13 17:38:02.597060 7fe9bae887a0 20 read_log 194215'773 (194215'308) delete 76eaf2a/benchmark_data_storage0111-ib.pocstor_48865_object18644/head//2 by client.2191635.0:46241 2014-08-12 17:17:04.428065 2014-08-13 17:38:02.597082 7fe9bae887a0 20 read_log 194215'774 (194215'309) delete 5bc40b6a/benchmark_data_storage0111-ib.pocstor_48865_object18652/head//2 by client.2191635.0:46249 2014-08-12 17:17:04.430713 2014-08-13 17:38:02.597143 7fe9bae887a0 20 read_log 194215'775 (194215'310) delete 9c195bea/benchmark_data_storage0111-ib.pocstor_48865_object18750/head//2 by client.2191635.0:46347 2014-08-12 17:17:04.463048 2014-08-13 17:38:02.597175 7fe9bae887a0 20 read_log 194215'776 (194215'311) delete bcd1ddea/benchmark_data_storage0111-ib.pocstor_48865_object18803/head//2 by client.2191635.0:46400 2014-08-12 17:17:04.477059 2014-08-13 17:38:02.597197 7fe9bae887a0 20 read_log 194215'777 (194215'312) delete 57e7232a/benchmark_data_storage0111-ib.pocstor_48865_object18851/head//2 by client.2191635.0:46448 2014-08-12 17:17:04.491213 2014-08-13 17:38:02.597219 7fe9bae887a0 20 read_log 194215'778 (194215'313) delete 3f8d12a/benchmark_data_storage0111-ib.pocstor_48865_object18898/head//2 by client.2191635.0:46495 2014-08-12 17:17:04.504495 2014-08-13 17:38:02.597240 7fe9bae887a0 20 read_log 194215'779 (194215'314) delete eda4702a/benchmark_data_storage0111-ib.pocstor_48865_object18934/head//2 by client.2191635.0:46531 2014-08-12 17:17:04.514927 2014-08-13 17:38:02.597261 7fe9bae887a0 20 read_log 194215'780 (194215'315) delete 8ad8c9ea/benchmark_data_storage0111-ib.pocstor_48865_object19074/head//2 by client.2191635.0:46671 2014-08-12 17:17:04.551775 2014-08-13 17:38:02.597282 7fe9bae887a0 20 read_log 194215'781 (194215'316) delete 45feb6aa/benchmark_data_storage0111-ib.pocstor_48865_object19194/head//2 by client.2191635.0:46791 2014-08-12 17:17:04.585919 2014-08-13 17:38:02.597305 7fe9bae887a0 20 read_log 194215'782 (194215'317) delete 67fe712a/benchmark_data_storage0111-ib.pocstor_48865_object19218/head//2 by client.2191635.0:46815 2014-08-12 17:17:04.592239 2014-08-13 17:38:02.597328 7fe9bae887a0 20 read_log 194215'783 (194215'318) delete 87a9a2ea/benchmark_data_storage0111-ib.pocstor_48865_object19230/head//2 by client.2191635.0:46827 2014-08-12 17:17:04.596308 2014-08-13 17:38:02.597349 7fe9bae887a0 20 read_log 194215'784 (194215'319) delete d4cfa56a/benchmark_data_storage0111-ib.pocstor_48865_object19259/head//2 by client.2191635.0:46856 2014-08-12 17:17:04.604276 2014-08-13 17:38:02.597370 7fe9bae887a0 20 read_log 194215'785 (194215'320) delete 3d60586a/benchmark_data_storage0111-ib.pocstor_48865_object19315/head//2 by client.2191635.0:46912 2014-08-12 17:17:04.618845 2014-08-13 17:38:02.597392 7fe9bae887a0 20 read_log 194215'786 (194215'321) delete a5cf01aa/benchmark_data_storage0111-ib.pocstor_48865_object19413/head//2 by client.2191635.0:47010 2014-08-12 17:17:04.646090 2014-08-13 17:38:02.597412 7fe9bae887a0 20 read_log 194215'787 (194215'322) delete eac666ea/benchmark_data_storage0111-ib.pocstor_48865_object19418/head//2 by client.2191635.0:47015 2014-08-12 17:17:04.647628 2014-08-13 17:38:02.597478 7fe9bae887a0 20 read_log 194215'788 (194215'323) delete 701512a/benchmark_data_storage0111-ib.pocstor_48865_object19425/head//2 by client.2191635.0:47022 2014-08-12 17:17:04.649464 2014-08-13 17:38:02.597503 7fe9bae887a0 20 read_log 194215'789 (194215'324) delete 109f076a/benchmark_data_storage0111-ib.pocstor_48865_object19461/head//2 by client.2191635.0:47058 2014-08-12 17:17:04.660461 2014-08-13 17:38:02.597525 7fe9bae887a0 20 read_log 194215'790 (194215'325) delete b65a122a/benchmark_data_storage0111-ib.pocstor_48865_object19517/head//2 by client.2191635.0:47114 2014-08-12 17:17:04.678486 2014-08-13 17:38:02.597546 7fe9bae887a0 20 read_log 194215'791 (194215'326) delete bca3f02a/benchmark_data_storage0111-ib.pocstor_48865_object19526/head//2 by client.2191635.0:47123 2014-08-12 17:17:04.681640 2014-08-13 17:38:02.597568 7fe9bae887a0 20 read_log 194215'792 (194215'327) delete cf2e0e2a/benchmark_data_storage0111-ib.pocstor_48865_object19566/head//2 by client.2191635.0:47163 2014-08-12 17:17:04.693856 2014-08-13 17:38:02.597589 7fe9bae887a0 20 read_log 194215'793 (194215'328) delete 56fd8d6a/benchmark_data_storage0111-ib.pocstor_48865_object19582/head//2 by client.2191635.0:47179 2014-08-12 17:17:04.697927 2014-08-13 17:38:02.597610 7fe9bae887a0 20 read_log 194215'794 (194215'329) delete bea29c2a/benchmark_data_storage0111-ib.pocstor_48865_object19690/head//2 by client.2191635.0:47287 2014-08-12 17:17:04.730034 2014-08-13 17:38:02.597649 7fe9bae887a0 20 read_log 194215'795 (194215'330) delete 971f7a6a/benchmark_data_storage0111-ib.pocstor_48865_object19692/head//2 by client.2191635.0:47289 2014-08-12 17:17:04.730147 2014-08-13 17:38:02.597699 7fe9bae887a0 20 read_log 194215'796 (194215'331) delete 9a63936a/benchmark_data_storage0111-ib.pocstor_48865_object19899/head//2 by client.2191635.0:47496 2014-08-12 17:17:04.795100 2014-08-13 17:38:02.597721 7fe9bae887a0 20 read_log 194215'797 (194215'332) delete f7da2aea/benchmark_data_storage0111-ib.pocstor_48865_object19982/head//2 by client.2191635.0:47579 2014-08-12 17:17:04.823185 2014-08-13 17:38:02.597743 7fe9bae887a0 20 read_log 194215'798 (194215'333) delete 5bb981aa/benchmark_data_storage0111-ib.pocstor_48865_object20036/head//2 by client.2191635.0:47633 2014-08-12 17:17:04.838572 2014-08-13 17:38:02.597764 7fe9bae887a0 20 read_log 194215'799 (194215'334) delete cf11302a/benchmark_data_storage0111-ib.pocstor_48865_object20045/head//2 by client.2191635.0:47642 2014-08-12 17:17:04.841469 2014-08-13 17:38:02.597785 7fe9bae887a0 20 read_log 194215'800 (194215'335) delete fe4a30aa/benchmark_data_storage0111-ib.pocstor_48865_object20057/head//2 by client.2191635.0:47654 2014-08-12 17:17:04.844276 2014-08-13 17:38:02.597806 7fe9bae887a0 20 read_log 194215'801 (194215'336) delete 44997bea/benchmark_data_storage0111-ib.pocstor_48865_object20125/head//2 by client.2191635.0:47722 2014-08-12 17:17:04.860248 2014-08-13 17:38:02.597828 7fe9bae887a0 20 read_log 194215'802 (194215'337) delete c8c401aa/benchmark_data_storage0111-ib.pocstor_48865_object20134/head//2 by client.2191635.0:47731 2014-08-12 17:17:04.862174 2014-08-13 17:38:02.597849 7fe9bae887a0 20 read_log 194215'803 (194215'338) delete ae295caa/benchmark_data_storage0111-ib.pocstor_48865_object20201/head//2 by client.2191635.0:47798 2014-08-12 17:17:04.881751 2014-08-13 17:38:02.597870 7fe9bae887a0 20 read_log 194215'804 (194215'339) delete f203046a/benchmark_data_storage0111-ib.pocstor_48865_object20204/head//2 by client.2191635.0:47801 2014-08-12 17:17:04.882237 2014-08-13 17:38:02.597892 7fe9bae887a0 20 read_log 194215'805 (194215'340) delete de6e16aa/benchmark_data_storage0111-ib.pocstor_48865_object20326/head//2 by client.2191635.0:47923 2014-08-12 17:17:04.921246 2014-08-13 17:38:02.597913 7fe9bae887a0 20 read_log 194215'806 (194215'341) delete e793e8aa/benchmark_data_storage0111-ib.pocstor_48865_object20420/head//2 by client.2191635.0:48017 2014-08-12 17:17:04.948763 2014-08-13 17:38:02.597935 7fe9bae887a0 20 read_log 194215'807 (194215'342) delete 97c150ea/benchmark_data_storage0111-ib.pocstor_48865_object20422/head//2 by client.2191635.0:48019 2014-08-12 17:17:04.949670 2014-08-13 17:38:02.597956 7fe9bae887a0 20 read_log 194215'808 (194215'343) delete c581916a/benchmark_data_storage0111-ib.pocstor_48865_object20680/head//2 by client.2191635.0:48277 2014-08-12 17:17:05.023896 2014-08-13 17:38:02.597977 7fe9bae887a0 20 read_log 194215'809 (194215'344) delete 226308aa/benchmark_data_storage0111-ib.pocstor_48865_object20698/head//2 by client.2191635.0:48295 2014-08-12 17:17:05.028773 2014-08-13 17:38:02.597998 7fe9bae887a0 20 read_log 194215'810 (194215'345) delete 217b9c2a/benchmark_data_storage0111-ib.pocstor_48865_object20701/head//2 by client.2191635.0:48298 2014-08-12 17:17:05.029894 2014-08-13 17:38:02.598020 7fe9bae887a0 20 read_log 194215'811 (194215'346) delete d22aaeea/benchmark_data_storage0111-ib.pocstor_48865_object20724/head//2 by client.2191635.0:48321 2014-08-12 17:17:05.035948 2014-08-13 17:38:02.598041 7fe9bae887a0 20 read_log 194215'812 (194215'347) delete 3056b9aa/benchmark_data_storage0111-ib.pocstor_48865_object20781/head//2 by client.2191635.0:48378 2014-08-12 17:17:05.053283 2014-08-13 17:38:02.598062 7fe9bae887a0 20 read_log 194215'813 (194215'348) delete e128ee2a/benchmark_data_storage0111-ib.pocstor_48865_object20832/head//2 by client.2191635.0:48429 2014-08-12 17:17:05.068034 2014-08-13 17:38:02.598083 7fe9bae887a0 20 read_log 194215'814 (194215'349) delete a037016a/benchmark_data_storage0111-ib.pocstor_48865_object21051/head//2 by client.2191635.0:48648 2014-08-12 17:17:05.134132 2014-08-13 17:38:02.598116 7fe9bae887a0 20 read_log 194215'815 (194215'350) delete b10367ea/benchmark_data_storage0111-ib.pocstor_48865_object21068/head//2 by client.2191635.0:48665 2014-08-12 17:17:05.139039 2014-08-13 17:38:02.598138 7fe9bae887a0 20 read_log 194215'816 (194215'351) delete d46985aa/benchmark_data_storage0111-ib.pocstor_48865_object21162/head//2 by client.2191635.0:48759 2014-08-12 17:17:05.165129 2014-08-13 17:38:02.598170 7fe9bae887a0 20 read_log 194215'817 (194215'352) delete 971d17ea/benchmark_data_storage0111-ib.pocstor_48865_object21246/head//2 by client.2191635.0:48843 2014-08-12 17:17:05.190199 2014-08-13 17:38:02.598192 7fe9bae887a0 20 read_log 194215'818 (194215'353) delete 41d7a72a/benchmark_data_storage0111-ib.pocstor_48865_object21255/head//2 by client.2191635.0:48852 2014-08-12 17:17:05.192642 2014-08-13 17:38:02.598214 7fe9bae887a0 20 read_log 194215'819 (194215'354) delete 6adc306a/benchmark_data_storage0111-ib.pocstor_48865_object21329/head//2 by client.2191635.0:48926 2014-08-12 17:17:05.215068 2014-08-13 17:38:02.598235 7fe9bae887a0 20 read_log 194215'820 (194215'355) delete 19802aaa/benchmark_data_storage0111-ib.pocstor_48865_object21368/head//2 by client.2191635.0:48965 2014-08-12 17:17:05.226607 2014-08-13 17:38:02.598256 7fe9bae887a0 20 read_log 194215'821 (194215'356) delete 87842baa/benchmark_data_storage0111-ib.pocstor_48865_object21403/head//2 by client.2191635.0:49000 2014-08-12 17:17:05.237146 2014-08-13 17:38:02.598278 7fe9bae887a0 20 read_log 194215'822 (194215'357) delete b59226a/benchmark_data_storage0111-ib.pocstor_48865_object21404/head//2 by client.2191635.0:49001 2014-08-12 17:17:05.237302 2014-08-13 17:38:02.598299 7fe9bae887a0 20 read_log 194215'823 (194215'358) delete d68ecdaa/benchmark_data_storage0111-ib.pocstor_48865_object21520/head//2 by client.2191635.0:49117 2014-08-12 17:17:05.272097 2014-08-13 17:38:02.598320 7fe9bae887a0 20 read_log 194215'824 (194215'359) delete 37caa06a/benchmark_data_storage0111-ib.pocstor_48865_object21568/head//2 by client.2191635.0:49165 2014-08-12 17:17:05.286430 2014-08-13 17:38:02.598341 7fe9bae887a0 20 read_log 194215'825 (194215'360) delete 615151ea/benchmark_data_storage0111-ib.pocstor_48865_object21657/head//2 by client.2191635.0:49254 2014-08-12 17:17:05.310822 2014-08-13 17:38:02.598362 7fe9bae887a0 20 read_log 194215'826 (194215'361) delete 7abdc1ea/benchmark_data_storage0111-ib.pocstor_48865_object21668/head//2 by client.2191635.0:49265 2014-08-12 17:17:05.313722 2014-08-13 17:38:02.598386 7fe9bae887a0 20 read_log 194215'827 (194215'362) delete 97743f2a/benchmark_data_storage0111-ib.pocstor_48865_object21779/head//2 by client.2191635.0:49376 2014-08-12 17:17:05.347539 2014-08-13 17:38:02.598407 7fe9bae887a0 20 read_log 194215'828 (194215'363) delete 605fde2a/benchmark_data_storage0111-ib.pocstor_48865_object21799/head//2 by client.2191635.0:49396 2014-08-12 17:17:05.354213 2014-08-13 17:38:02.598428 7fe9bae887a0 20 read_log 194215'829 (194215'364) delete 7e36cb6a/benchmark_data_storage0111-ib.pocstor_48865_object21980/head//2 by client.2191635.0:49577 2014-08-12 17:17:05.406600 2014-08-13 17:38:02.598449 7fe9bae887a0 20 read_log 194215'830 (194215'365) delete 31ca3faa/benchmark_data_storage0111-ib.pocstor_48865_object22026/head//2 by client.2191635.0:49623 2014-08-12 17:17:05.421925 2014-08-13 17:38:02.598470 7fe9bae887a0 20 read_log 194215'831 (194215'366) delete b22ed1aa/benchmark_data_storage0111-ib.pocstor_48865_object22036/head//2 by client.2191635.0:49633 2014-08-12 17:17:05.424507 2014-08-13 17:38:02.598491 7fe9bae887a0 20 read_log 194215'832 (194215'367) delete 9d60f46a/benchmark_data_storage0111-ib.pocstor_48865_object22243/head//2 by client.2191635.0:49840 2014-08-12 17:17:05.481920 2014-08-13 17:38:02.598512 7fe9bae887a0 20 read_log 194215'833 (194215'368) delete d38544aa/benchmark_data_storage0111-ib.pocstor_48865_object22284/head//2 by client.2191635.0:49881 2014-08-12 17:17:05.495587 2014-08-13 17:38:02.598533 7fe9bae887a0 20 read_log 194215'834 (194215'369) delete 4248b2ea/benchmark_data_storage0111-ib.pocstor_48865_object22314/head//2 by client.2191635.0:49911 2014-08-12 17:17:05.504481 2014-08-13 17:38:02.598565 7fe9bae887a0 20 read_log 194215'835 (194215'370) delete 9d1470aa/benchmark_data_storage0111-ib.pocstor_48865_object22348/head//2 by client.2191635.0:49945 2014-08-12 17:17:05.514218 2014-08-13 17:38:02.598587 7fe9bae887a0 20 read_log 194215'836 (194215'371) delete 8501b66a/benchmark_data_storage0111-ib.pocstor_48865_object22363/head//2 by client.2191635.0:49960 2014-08-12 17:17:05.519001 2014-08-13 17:38:02.598608 7fe9bae887a0 20 read_log 194215'837 (194215'372) delete 4baa0dea/benchmark_data_storage0111-ib.pocstor_48865_object22414/head//2 by client.2191635.0:50011 2014-08-12 17:17:05.532797 2014-08-13 17:38:02.598630 7fe9bae887a0 20 read_log 194215'838 (194215'373) delete c76fe86a/benchmark_data_storage0111-ib.pocstor_48865_object22497/head//2 by client.2191635.0:50094 2014-08-12 17:17:05.556744 2014-08-13 17:38:02.598650 7fe9bae887a0 20 read_log 194215'839 (194215'374) delete 1378006a/benchmark_data_storage0111-ib.pocstor_48865_object22538/head//2 by client.2191635.0:50135 2014-08-12 17:17:05.567810 2014-08-13 17:38:02.598671 7fe9bae887a0 20 read_log 194215'840 (194215'375) delete 51bf9b2a/benchmark_data_storage0111-ib.pocstor_48865_object22569/head//2 by client.2191635.0:50166 2014-08-12 17:17:05.579086 2014-08-13 17:38:02.598692 7fe9bae887a0 20 read_log 194215'841 (194215'376) delete d91a75ea/benchmark_data_storage0111-ib.pocstor_48865_object22581/head//2 by client.2191635.0:50178 2014-08-12 17:17:05.583122 2014-08-13 17:38:02.598713 7fe9bae887a0 20 read_log 194215'842 (194215'377) delete 877f52a/benchmark_data_storage0111-ib.pocstor_48865_object22590/head//2 by client.2191635.0:50187 2014-08-12 17:17:05.584999 2014-08-13 17:38:02.598734 7fe9bae887a0 20 read_log 194215'843 (194215'378) delete 7772482a/benchmark_data_storage0111-ib.pocstor_48865_object22594/head//2 by client.2191635.0:50191 2014-08-12 17:17:05.586579 2014-08-13 17:38:02.598755 7fe9bae887a0 20 read_log 194215'844 (194215'379) delete 4f9362a/benchmark_data_storage0111-ib.pocstor_48865_object22640/head//2 by client.2191635.0:50237 2014-08-12 17:17:05.600634 2014-08-13 17:38:02.598776 7fe9bae887a0 20 read_log 194215'845 (194215'380) delete 5af1bdaa/benchmark_data_storage0111-ib.pocstor_48865_object22673/head//2 by client.2191635.0:50270 2014-08-12 17:17:05.610069 2014-08-13 17:38:02.598797 7fe9bae887a0 20 read_log 194215'846 (194215'381) delete 8f9836aa/benchmark_data_storage0111-ib.pocstor_48865_object22690/head//2 by client.2191635.0:50287 2014-08-12 17:17:05.615026 2014-08-13 17:38:02.598818 7fe9bae887a0 20 read_log 194215'847 (194215'382) delete 1e47b9ea/benchmark_data_storage0111-ib.pocstor_48865_object22731/head//2 by client.2191635.0:50328 2014-08-12 17:17:05.627444 2014-08-13 17:38:02.598842 7fe9bae887a0 20 read_log 194215'848 (194215'383) delete dfa6716a/benchmark_data_storage0111-ib.pocstor_48865_object22765/head//2 by client.2191635.0:50362 2014-08-12 17:17:05.638534 2014-08-13 17:38:02.598863 7fe9bae887a0 20 read_log 194215'849 (194215'384) delete 10a9a06a/benchmark_data_storage0111-ib.pocstor_48865_object22886/head//2 by client.2191635.0:50483 2014-08-12 17:17:05.677317 2014-08-13 17:38:02.598884 7fe9bae887a0 20 read_log 194215'850 (194215'385) delete 8c0ab1aa/benchmark_data_storage0111-ib.pocstor_48865_object22899/head//2 by client.2191635.0:50496 2014-08-12 17:17:05.683824 2014-08-13 17:38:02.598906 7fe9bae887a0 20 read_log 194215'851 (194215'386) delete 8bceefea/benchmark_data_storage0111-ib.pocstor_48865_object22913/head//2 by client.2191635.0:50510 2014-08-12 17:17:05.687141 2014-08-13 17:38:02.598927 7fe9bae887a0 20 read_log 194215'852 (194215'387) delete 2fbf146a/benchmark_data_storage0111-ib.pocstor_48865_object22932/head//2 by client.2191635.0:50529 2014-08-12 17:17:05.692264 2014-08-13 17:38:02.598948 7fe9bae887a0 20 read_log 194215'853 (194215'388) delete 7df62c2a/benchmark_data_storage0111-ib.pocstor_48865_object23119/head//2 by client.2191635.0:50716 2014-08-12 17:17:05.745373 2014-08-13 17:38:02.598969 7fe9bae887a0 20 read_log 194215'854 (194215'389) delete b614c76a/benchmark_data_storage0111-ib.pocstor_48865_object23197/head//2 by client.2191635.0:50794 2014-08-12 17:17:05.769603 2014-08-13 17:38:02.599004 7fe9bae887a0 20 read_log 194215'855 (194215'390) delete 62b055ea/benchmark_data_storage0111-ib.pocstor_48865_object23212/head//2 by client.2191635.0:50809 2014-08-12 17:17:05.772777 2014-08-13 17:38:02.599027 7fe9bae887a0 20 read_log 194215'856 (194215'391) delete 972960ea/benchmark_data_storage0111-ib.pocstor_48865_object23269/head//2 by client.2191635.0:50866 2014-08-12 17:17:05.788953 2014-08-13 17:38:02.599051 7fe9bae887a0 20 read_log 194215'857 (194215'392) delete 64c4eb6a/benchmark_data_storage0111-ib.pocstor_48865_object23325/head//2 by client.2191635.0:50922 2014-08-12 17:17:05.804096 2014-08-13 17:38:02.599072 7fe9bae887a0 20 read_log 194215'858 (194215'393) delete c10f2b2a/benchmark_data_storage0111-ib.pocstor_48865_object23339/head//2 by client.2191635.0:50936 2014-08-12 17:17:05.810037 2014-08-13 17:38:02.599093 7fe9bae887a0 20 read_log 194215'859 (194215'394) delete f00c10aa/benchmark_data_storage0111-ib.pocstor_48865_object23473/head//2 by client.2191635.0:51070 2014-08-12 17:17:05.849116 2014-08-13 17:38:02.599114 7fe9bae887a0 20 read_log 194215'860 (194215'395) delete e2f975aa/benchmark_data_storage0111-ib.pocstor_48865_object23699/head//2 by client.2191635.0:51296 2014-08-12 17:17:05.914597 2014-08-13 17:38:02.599135 7fe9bae887a0 20 read_log 194215'861 (194215'396) delete 68a661ea/benchmark_data_storage0111-ib.pocstor_48865_object23758/head//2 by client.2191635.0:51355 2014-08-12 17:17:05.931338 2014-08-13 17:38:02.599163 7fe9bae887a0 20 read_log 194215'862 (194215'397) delete baa697aa/benchmark_data_storage0111-ib.pocstor_48865_object23818/head//2 by client.2191635.0:51415 2014-08-12 17:17:05.950516 2014-08-13 17:38:02.599186 7fe9bae887a0 20 read_log 194215'863 (194215'398) delete 6b48b46a/benchmark_data_storage0111-ib.pocstor_48865_object23835/head//2 by client.2191635.0:51432 2014-08-12 17:17:05.954788 2014-08-13 17:38:02.599207 7fe9bae887a0 20 read_log 194215'864 (194215'399) delete d8ebb3aa/benchmark_data_storage0111-ib.pocstor_48865_object23888/head//2 by client.2191635.0:51485 2014-08-12 17:17:05.970616 2014-08-13 17:38:02.599229 7fe9bae887a0 20 read_log 194215'865 (194215'400) delete 34bd3aaa/benchmark_data_storage0111-ib.pocstor_48865_object23938/head//2 by client.2191635.0:51535 2014-08-12 17:17:05.984325 2014-08-13 17:38:02.599251 7fe9bae887a0 20 read_log 194215'866 (194215'401) delete 7504a3aa/benchmark_data_storage0111-ib.pocstor_48865_object23964/head//2 by client.2191635.0:51561 2014-08-12 17:17:05.992081 2014-08-13 17:38:02.599273 7fe9bae887a0 20 read_log 194215'867 (194215'402) delete e16bca6a/benchmark_data_storage0111-ib.pocstor_48865_object24022/head//2 by client.2191635.0:51619 2014-08-12 17:17:06.009384 2014-08-13 17:38:02.599294 7fe9bae887a0 20 read_log 194215'868 (194215'403) delete 4b4b8cea/benchmark_data_storage0111-ib.pocstor_48865_object24030/head//2 by client.2191635.0:51627 2014-08-12 17:17:06.012324 2014-08-13 17:38:02.599349 7fe9bae887a0 20 read_log 194215'869 (194215'404) delete e441ef6a/benchmark_data_storage0111-ib.pocstor_48865_object24080/head//2 by client.2191635.0:51677 2014-08-12 17:17:06.028919 2014-08-13 17:38:02.599374 7fe9bae887a0 20 read_log 194215'870 (194215'405) delete 5fcc2aaa/benchmark_data_storage0111-ib.pocstor_48865_object24241/head//2 by client.2191635.0:51838 2014-08-12 17:17:06.077254 2014-08-13 17:38:02.599396 7fe9bae887a0 20 read_log 194215'871 (194215'406) delete 3c8eaeaa/benchmark_data_storage0111-ib.pocstor_48865_object24315/head//2 by client.2191635.0:51912 2014-08-12 17:17:06.100277 2014-08-13 17:38:02.599424 7fe9bae887a0 20 read_log 194215'872 (194215'407) delete fb00662a/benchmark_data_storage0111-ib.pocstor_48865_object24327/head//2 by client.2191635.0:51924 2014-08-12 17:17:06.105571 2014-08-13 17:38:02.599446 7fe9bae887a0 20 read_log 194215'873 (194215'408) delete b50f7ea/benchmark_data_storage0111-ib.pocstor_48865_object24343/head//2 by client.2191635.0:51940 2014-08-12 17:17:06.109352 2014-08-13 17:38:02.599467 7fe9bae887a0 20 read_log 194215'874 (194215'409) delete f5caafaa/benchmark_data_storage0111-ib.pocstor_48865_object24349/head//2 by client.2191635.0:51946 2014-08-12 17:17:06.111288 2014-08-13 17:38:02.599500 7fe9bae887a0 20 read_log 194215'875 (194215'410) delete d65f39aa/benchmark_data_storage0111-ib.pocstor_48865_object24366/head//2 by client.2191635.0:51963 2014-08-12 17:17:06.116141 2014-08-13 17:38:02.599522 7fe9bae887a0 20 read_log 194215'876 (194215'411) delete d785ae2a/benchmark_data_storage0111-ib.pocstor_48865_object24461/head//2 by client.2191635.0:52058 2014-08-12 17:17:06.153743 2014-08-13 17:38:02.599544 7fe9bae887a0 20 read_log 194215'877 (194215'412) delete 94e6396a/benchmark_data_storage0111-ib.pocstor_48865_object24499/head//2 by client.2191635.0:52096 2014-08-12 17:17:06.164450 2014-08-13 17:38:02.599566 7fe9bae887a0 20 read_log 194215'878 (194215'413) delete ec9780ea/benchmark_data_storage0111-ib.pocstor_48865_object24551/head//2 by client.2191635.0:52148 2014-08-12 17:17:06.180755 2014-08-13 17:38:02.599588 7fe9bae887a0 20 read_log 194215'879 (194215'414) delete 76527aaa/benchmark_data_storage0111-ib.pocstor_48865_object24564/head//2 by client.2191635.0:52161 2014-08-12 17:17:06.184804 2014-08-13 17:38:02.599609 7fe9bae887a0 20 read_log 194215'880 (194215'415) delete 1fe906ea/benchmark_data_storage0111-ib.pocstor_48865_object24630/head//2 by client.2191635.0:52227 2014-08-12 17:17:06.207428 2014-08-13 17:38:02.599630 7fe9bae887a0 20 read_log 194215'881 (194215'416) delete d088b7aa/benchmark_data_storage0111-ib.pocstor_48865_object24638/head//2 by client.2191635.0:52235 2014-08-12 17:17:06.209632 2014-08-13 17:38:02.599651 7fe9bae887a0 20 read_log 194215'882 (194215'417) delete c957872a/benchmark_data_storage0111-ib.pocstor_48865_object24706/head//2 by client.2191635.0:52303 2014-08-12 17:17:06.228196 2014-08-13 17:38:02.599671 7fe9bae887a0 20 read_log 194215'883 (194215'418) delete 7e977daa/benchmark_data_storage0111-ib.pocstor_48865_object24941/head//2 by client.2191635.0:52538 2014-08-12 17:17:06.300089 2014-08-13 17:38:02.599722 7fe9bae887a0 20 read_log 194215'884 (194215'419) delete c441e36a/benchmark_data_storage0111-ib.pocstor_48865_object25044/head//2 by client.2191635.0:52641 2014-08-12 17:17:06.329291 2014-08-13 17:38:02.599745 7fe9bae887a0 20 read_log 194215'885 (194215'420) delete 3b05822a/benchmark_data_storage0111-ib.pocstor_48865_object25074/head//2 by client.2191635.0:52671 2014-08-12 17:17:06.336455 2014-08-13 17:38:02.599766 7fe9bae887a0 20 read_log 194215'886 (194215'421) delete dac3a2ea/benchmark_data_storage0111-ib.pocstor_48865_object25144/head//2 by client.2191635.0:52741 2014-08-12 17:17:06.357227 2014-08-13 17:38:02.599787 7fe9bae887a0 20 read_log 194215'887 (194215'422) delete ed943f2a/benchmark_data_storage0111-ib.pocstor_48865_object25159/head//2 by client.2191635.0:52756 2014-08-12 17:17:06.362533 2014-08-13 17:38:02.599809 7fe9bae887a0 20 read_log 194215'888 (194215'423) delete 68276bea/benchmark_data_storage0111-ib.pocstor_48865_object25166/head//2 by client.2191635.0:52763 2014-08-12 17:17:06.364580 2014-08-13 17:38:02.599855 7fe9bae887a0 20 read_log 194215'889 (194215'424) delete bd3e026a/benchmark_data_storage0111-ib.pocstor_48865_object25310/head//2 by client.2191635.0:52907 2014-08-12 17:17:06.409555 2014-08-13 17:38:02.599878 7fe9bae887a0 20 read_log 194215'890 (194215'425) delete 48d2462a/benchmark_data_storage0111-ib.pocstor_48865_object25331/head//2 by client.2191635.0:52928 2014-08-12 17:17:06.416164 2014-08-13 17:38:02.599899 7fe9bae887a0 20 read_log 194215'891 (194215'426) delete 9c95526a/benchmark_data_storage0111-ib.pocstor_48865_object25369/head//2 by client.2191635.0:52966 2014-08-12 17:17:06.428690 2014-08-13 17:38:02.599921 7fe9bae887a0 20 read_log 194215'892 (194215'427) delete 9e2944aa/benchmark_data_storage0111-ib.pocstor_48865_object25391/head//2 by client.2191635.0:52988 2014-08-12 17:17:06.436814 2014-08-13 17:38:02.599942 7fe9bae887a0 20 read_log 194215'893 (194215'428) delete aff1fc6a/benchmark_data_storage0111-ib.pocstor_48865_object25463/head//2 by client.2191635.0:53060 2014-08-12 17:17:06.457812 2014-08-13 17:38:02.599963 7fe9bae887a0 20 read_log 194215'894 (194215'429) delete 9f6824aa/benchmark_data_storage0111-ib.pocstor_48865_object25483/head//2 by client.2191635.0:53080 2014-08-12 17:17:06.464333 2014-08-13 17:38:02.599996 7fe9bae887a0 20 read_log 194215'895 (194215'430) delete 9cc1c82a/benchmark_data_storage0111-ib.pocstor_48865_object25512/head//2 by client.2191635.0:53109 2014-08-12 17:17:06.473807 2014-08-13 17:38:02.600018 7fe9bae887a0 20 read_log 194215'896 (194215'431) delete f2fcef2a/benchmark_data_storage0111-ib.pocstor_48865_object25660/head//2 by client.2191635.0:53257 2014-08-12 17:17:06.521469 2014-08-13 17:38:02.600040 7fe9bae887a0 20 read_log 194215'897 (194215'432) delete deeda82a/benchmark_data_storage0111-ib.pocstor_48865_object25706/head//2 by client.2191635.0:53303 2014-08-12 17:17:06.534550 2014-08-13 17:38:02.600061 7fe9bae887a0 20 read_log 194215'898 (194215'433) delete b504f56a/benchmark_data_storage0111-ib.pocstor_48865_object25726/head//2 by client.2191635.0:53323 2014-08-12 17:17:06.540764 2014-08-13 17:38:02.600082 7fe9bae887a0 20 read_log 194215'899 (194215'434) delete 652e19ea/benchmark_data_storage0111-ib.pocstor_48865_object25735/head//2 by client.2191635.0:53332 2014-08-12 17:17:06.543907 2014-08-13 17:38:02.600106 7fe9bae887a0 20 read_log 194215'900 (194215'435) delete ff82672a/benchmark_data_storage0111-ib.pocstor_48865_object25879/head//2 by client.2191635.0:53476 2014-08-12 17:17:06.584962 2014-08-13 17:38:02.600128 7fe9bae887a0 20 read_log 194215'901 (194215'436) delete 1deec7ea/benchmark_data_storage0111-ib.pocstor_48865_object25948/head//2 by client.2191635.0:53545 2014-08-12 17:17:06.611299 2014-08-13 17:38:02.600149 7fe9bae887a0 20 read_log 194215'902 (194215'437) delete 6898cd6a/benchmark_data_storage0111-ib.pocstor_48865_object25978/head//2 by client.2191635.0:53575 2014-08-12 17:17:06.620961 2014-08-13 17:38:02.600179 7fe9bae887a0 20 read_log 194215'903 (194215'438) delete 9c55f66a/benchmark_data_storage0111-ib.pocstor_48865_object26060/head//2 by client.2191635.0:53657 2014-08-12 17:17:06.647657 2014-08-13 17:38:02.600232 7fe9bae887a0 20 read_log 194215'904 (194215'439) delete 99abde2a/benchmark_data_storage0111-ib.pocstor_48865_object26093/head//2 by client.2191635.0:53690 2014-08-12 17:17:06.656776 2014-08-13 17:38:02.600254 7fe9bae887a0 20 read_log 194215'905 (194215'440) delete 1696b9ea/benchmark_data_storage0111-ib.pocstor_48865_object26147/head//2 by client.2191635.0:53744 2014-08-12 17:17:06.672322 2014-08-13 17:38:02.600275 7fe9bae887a0 20 read_log 194215'906 (194215'441) delete 29311a2a/benchmark_data_storage0111-ib.pocstor_48865_object26180/head//2 by client.2191635.0:53777 2014-08-12 17:17:06.682090 2014-08-13 17:38:02.600296 7fe9bae887a0 20 read_log 194215'907 (194215'442) delete f9505d2a/benchmark_data_storage0111-ib.pocstor_48865_object26181/head//2 by client.2191635.0:53778 2014-08-12 17:17:06.682204 2014-08-13 17:38:02.600318 7fe9bae887a0 20 read_log 194215'908 (194215'443) delete 5991912a/benchmark_data_storage0111-ib.pocstor_48865_object26212/head//2 by client.2191635.0:53809 2014-08-12 17:17:06.691957 2014-08-13 17:38:02.600339 7fe9bae887a0 20 read_log 194215'909 (194215'444) delete b036cdea/benchmark_data_storage0111-ib.pocstor_48865_object26253/head//2 by client.2191635.0:53850 2014-08-12 17:17:06.703696 2014-08-13 17:38:02.600360 7fe9bae887a0 20 read_log 194215'910 (194215'445) delete 3571b8ea/benchmark_data_storage0111-ib.pocstor_48865_object26474/head//2 by client.2191635.0:54071 2014-08-12 17:17:06.764192 2014-08-13 17:38:02.600381 7fe9bae887a0 20 read_log 194215'911 (194215'446) delete cdf9102a/benchmark_data_storage0111-ib.pocstor_48865_object26574/head//2 by client.2191635.0:54171 2014-08-12 17:17:06.792174 2014-08-13 17:38:02.600402 7fe9bae887a0 20 read_log 194215'912 (194215'447) delete acb3246a/benchmark_data_storage0111-ib.pocstor_48865_object26583/head//2 by client.2191635.0:54180 2014-08-12 17:17:06.794112 2014-08-13 17:38:02.600424 7fe9bae887a0 20 read_log 194215'913 (194215'448) delete c64e22ea/benchmark_data_storage0111-ib.pocstor_48865_object26626/head//2 by client.2191635.0:54223 2014-08-12 17:17:06.807865 2014-08-13 17:38:02.600445 7fe9bae887a0 20 read_log 194215'914 (194215'449) delete d778c46a/benchmark_data_storage0111-ib.pocstor_48865_object26634/head//2 by client.2191635.0:54231 2014-08-12 17:17:06.809462 2014-08-13 17:38:02.600511 7fe9bae887a0 20 read_log 194215'915 (194215'450) delete 6da2d2ea/benchmark_data_storage0111-ib.pocstor_48865_object26711/head//2 by client.2191635.0:54308 2014-08-12 17:17:06.831468 2014-08-13 17:38:02.600535 7fe9bae887a0 20 read_log 194215'916 (194215'451) delete 6655c36a/benchmark_data_storage0111-ib.pocstor_48865_object26778/head//2 by client.2191635.0:54375 2014-08-12 17:17:06.848835 2014-08-13 17:38:02.600557 7fe9bae887a0 20 read_log 194215'917 (194215'452) delete d935beaa/benchmark_data_storage0111-ib.pocstor_48865_object26946/head//2 by client.2191635.0:54543 2014-08-12 17:17:06.892700 2014-08-13 17:38:02.600578 7fe9bae887a0 20 read_log 194215'918 (194215'453) delete 143d096a/benchmark_data_storage0111-ib.pocstor_48865_object26976/head//2 by client.2191635.0:54573 2014-08-12 17:17:06.901272 2014-08-13 17:38:02.600599 7fe9bae887a0 20 read_log 194215'919 (194215'454) delete a5899caa/benchmark_data_storage0111-ib.pocstor_48865_object27029/head//2 by client.2191635.0:54626 2014-08-12 17:17:06.915040 2014-08-13 17:38:02.600620 7fe9bae887a0 20 read_log 194215'920 (194215'455) delete b23c082a/benchmark_data_storage0111-ib.pocstor_48865_object27049/head//2 by client.2191635.0:54646 2014-08-12 17:17:06.921441 2014-08-13 17:38:02.600641 7fe9bae887a0 20 read_log 194215'921 (194215'456) delete 128e18aa/benchmark_data_storage0111-ib.pocstor_48865_object27097/head//2 by client.2191635.0:54694 2014-08-12 17:17:06.935428 2014-08-13 17:38:02.600662 7fe9bae887a0 20 read_log 194215'922 (194215'457) delete 1fef4b2a/benchmark_data_storage0111-ib.pocstor_48865_object27156/head//2 by client.2191635.0:54753 2014-08-12 17:17:06.952099 2014-08-13 17:38:02.600683 7fe9bae887a0 20 read_log 194215'923 (194215'458) delete 53f22aaa/benchmark_data_storage0111-ib.pocstor_48865_object27164/head//2 by client.2191635.0:54761 2014-08-12 17:17:06.953868 2014-08-13 17:38:02.600704 7fe9bae887a0 20 read_log 194215'924 (194215'459) delete 25a647ea/benchmark_data_storage0111-ib.pocstor_48865_object27294/head//2 by client.2191635.0:54891 2014-08-12 17:17:06.989198 2014-08-13 17:38:02.600725 7fe9bae887a0 20 read_log 194215'925 (194215'460) delete edbf666a/benchmark_data_storage0111-ib.pocstor_48865_object27302/head//2 by client.2191635.0:54899 2014-08-12 17:17:06.992355 2014-08-13 17:38:02.600746 7fe9bae887a0 20 read_log 194215'926 (194215'461) delete e0414e6a/benchmark_data_storage0111-ib.pocstor_48865_object27325/head//2 by client.2191635.0:54922 2014-08-12 17:17:06.997593 2014-08-13 17:38:02.600767 7fe9bae887a0 20 read_log 194215'927 (194215'462) delete fa7d5f6a/benchmark_data_storage0111-ib.pocstor_48865_object27402/head//2 by client.2191635.0:54999 2014-08-12 17:17:07.019557 2014-08-13 17:38:02.600788 7fe9bae887a0 20 read_log 194215'928 (194215'463) delete fee55a2a/benchmark_data_storage0111-ib.pocstor_48865_object27451/head//2 by client.2191635.0:55048 2014-08-12 17:17:07.035724 2014-08-13 17:38:02.600809 7fe9bae887a0 20 read_log 194215'929 (194215'464) delete 9b85faaa/benchmark_data_storage0111-ib.pocstor_48865_object27523/head//2 by client.2191635.0:55120 2014-08-12 17:17:07.057376 2014-08-13 17:38:02.600830 7fe9bae887a0 20 read_log 194215'930 (194215'465) delete 233bee2a/benchmark_data_storage0111-ib.pocstor_48865_object27577/head//2 by client.2191635.0:55174 2014-08-12 17:17:07.076475 2014-08-13 17:38:02.602030 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.602083 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.602109 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.023331 0 0.000000 2014-08-13 17:38:02.602127 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.602141 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.602162 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'930], crt=0'0) 2014-08-13 17:38:02.602179 7fe9bae887a0 10 osd.4 194478 pgid 22.84 coll 22.84_head 2014-08-13 17:38:02.602423 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.84 2014-08-13 17:38:02.602435 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.602484 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.84(unlocked)] enter Initial 2014-08-13 17:38:02.602500 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.84(unlocked)] enter NotTrimming 2014-08-13 17:38:02.602683 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.602884 7fe9bae887a0 20 read_log 191648'20 (0'0) modify 205a9884/benchmark_data_storage0111-ib.pocstor_40745_object1580/head//22 by client.273158.0:1581 2014-07-30 14:08:54.591086 2014-08-13 17:38:02.602916 7fe9bae887a0 20 read_log 191648'56 (0'0) modify b5421084/benchmark_data_storage0111-ib.pocstor_40745_object3978/head//22 by client.273158.0:3979 2014-07-30 14:09:06.140534 2014-08-13 17:38:02.602939 7fe9bae887a0 20 read_log 191648'254 (0'0) modify 85632084/benchmark_data_storage0111-ib.pocstor_40745_object18447/head//22 by client.273158.0:18448 2014-07-30 14:10:20.511565 2014-08-13 17:38:02.602960 7fe9bae887a0 20 read_log 191648'268 (0'0) modify 2a0ab084/benchmark_data_storage0111-ib.pocstor_40745_object18918/head//22 by client.273158.0:18919 2014-07-30 14:10:23.231329 2014-08-13 17:38:02.602997 7fe9bae887a0 20 read_log 191648'285 (0'0) modify bf866884/benchmark_data_storage0111-ib.pocstor_40745_object19765/head//22 by client.273158.0:19766 2014-07-30 14:10:30.417399 2014-08-13 17:38:02.603020 7fe9bae887a0 20 read_log 191648'303 (0'0) modify c7aba884/benchmark_data_storage0111-ib.pocstor_40745_object20683/head//22 by client.273158.0:20684 2014-07-30 14:10:34.707216 2014-08-13 17:38:02.603041 7fe9bae887a0 20 read_log 191727'341 (0'0) modify aec4084/benchmark_data_storage0111-ib.pocstor_51285_object1142/head//22 by client.275175.0:1143 2014-07-30 14:18:37.978666 2014-08-13 17:38:02.603062 7fe9bae887a0 20 read_log 191727'343 (0'0) modify 57eff084/benchmark_data_storage0111-ib.pocstor_51285_object2198/head//22 by client.275175.0:2199 2014-07-30 14:18:41.542418 2014-08-13 17:38:02.603083 7fe9bae887a0 20 read_log 191727'348 (0'0) modify ad464084/benchmark_data_storage0111-ib.pocstor_51285_object6729/head//22 by client.275175.0:6730 2014-07-30 14:18:55.826527 2014-08-13 17:38:02.603104 7fe9bae887a0 20 read_log 191727'349 (0'0) modify 81d55084/benchmark_data_storage0111-ib.pocstor_51285_object6936/head//22 by client.275175.0:6937 2014-07-30 14:18:56.541008 2014-08-13 17:38:02.603125 7fe9bae887a0 20 read_log 191727'350 (0'0) modify 14822084/benchmark_data_storage0111-ib.pocstor_51285_object7084/head//22 by client.275175.0:7085 2014-07-30 14:18:56.956303 2014-08-13 17:38:02.603145 7fe9bae887a0 20 read_log 191727'351 (0'0) modify bfbb8084/benchmark_data_storage0111-ib.pocstor_51285_object7632/head//22 by client.275175.0:7633 2014-07-30 14:18:58.500817 2014-08-13 17:38:02.603178 7fe9bae887a0 20 read_log 191727'356 (0'0) modify 974d1084/benchmark_data_storage0111-ib.pocstor_51285_object10697/head//22 by client.275175.0:10698 2014-07-30 14:19:07.195713 2014-08-13 17:38:02.603200 7fe9bae887a0 20 read_log 191727'360 (0'0) modify f8db0084/benchmark_data_storage0111-ib.pocstor_51285_object12830/head//22 by client.275175.0:12831 2014-07-30 14:19:13.964975 2014-08-13 17:38:02.603221 7fe9bae887a0 20 read_log 191727'364 (0'0) modify 21650884/benchmark_data_storage0111-ib.pocstor_51285_object15290/head//22 by client.275175.0:15291 2014-07-30 14:19:21.640370 2014-08-13 17:38:02.603241 7fe9bae887a0 20 read_log 191727'365 (0'0) modify 2b9d4084/benchmark_data_storage0111-ib.pocstor_51285_object15665/head//22 by client.275175.0:15666 2014-07-30 14:19:22.876135 2014-08-13 17:38:02.603262 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 8211d084/benchmark_data_storage0111-ib.pocstor_51285_object18767/head//22 by client.275175.0:18768 2014-07-30 14:19:32.369003 2014-08-13 17:38:02.603283 7fe9bae887a0 20 read_log 191727'400 (0'0) modify 1729d084/benchmark_data_storage0111-ib.pocstor_51285_object27218/head//22 by client.275175.0:27219 2014-07-30 14:19:58.436718 2014-08-13 17:38:02.603304 7fe9bae887a0 20 read_log 191727'401 (0'0) modify fdfec084/benchmark_data_storage0111-ib.pocstor_51285_object28283/head//22 by client.275175.0:28284 2014-07-30 14:20:01.818767 2014-08-13 17:38:02.603324 7fe9bae887a0 20 read_log 191727'402 (0'0) modify db3e7084/benchmark_data_storage0111-ib.pocstor_51285_object29027/head//22 by client.275175.0:29028 2014-07-30 14:20:03.966527 2014-08-13 17:38:02.603347 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 46c9a084/benchmark_data_storage0111-ib.pocstor_51285_object29801/head//22 by client.275175.0:29802 2014-07-30 14:20:06.390757 2014-08-13 17:38:02.603369 7fe9bae887a0 20 read_log 191727'409 (0'0) modify 7778f884/benchmark_data_storage0111-ib.pocstor_51285_object32228/head//22 by client.275175.0:32229 2014-07-30 14:20:13.557324 2014-08-13 17:38:02.603390 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 5df73884/benchmark_data_storage0111-ib.pocstor_51285_object33582/head//22 by client.275175.0:33583 2014-07-30 14:20:17.799915 2014-08-13 17:38:02.603427 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 7c177884/benchmark_data_storage0111-ib.pocstor_51285_object33968/head//22 by client.275175.0:33969 2014-07-30 14:20:18.902465 2014-08-13 17:38:02.603449 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 1a64b084/benchmark_data_storage0111-ib.pocstor_51285_object34143/head//22 by client.275175.0:34144 2014-07-30 14:20:19.458283 2014-08-13 17:38:02.603470 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 8b01b884/benchmark_data_storage0111-ib.pocstor_51285_object34402/head//22 by client.275175.0:34403 2014-07-30 14:20:20.267697 2014-08-13 17:38:02.603490 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 1622d084/benchmark_data_storage0111-ib.pocstor_51285_object36130/head//22 by client.275175.0:36131 2014-07-30 14:20:25.665252 2014-08-13 17:38:02.603511 7fe9bae887a0 20 read_log 191727'425 (0'0) modify c7bb9884/benchmark_data_storage0111-ib.pocstor_51285_object37519/head//22 by client.275175.0:37520 2014-07-30 14:20:29.903390 2014-08-13 17:38:02.603531 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 2c757084/benchmark_data_storage0111-ib.pocstor_51285_object37881/head//22 by client.275175.0:37882 2014-07-30 14:20:30.906287 2014-08-13 17:38:02.603552 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 36c52884/benchmark_data_storage0111-ib.pocstor_35625_object1157/head//22 by client.940330.0:1158 2014-08-11 16:29:45.375958 2014-08-13 17:38:02.603573 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 81389084/benchmark_data_storage0111-ib.pocstor_35625_object3335/head//22 by client.940330.0:3336 2014-08-11 16:29:53.868950 2014-08-13 17:38:02.603594 7fe9bae887a0 20 read_log 192888'430 (0'0) modify b96f2884/benchmark_data_storage0111-ib.pocstor_35625_object6702/head//22 by client.940330.0:6703 2014-08-11 16:30:06.266578 2014-08-13 17:38:02.603614 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 40cef084/benchmark_data_storage0111-ib.pocstor_35625_object7298/head//22 by client.940330.0:7299 2014-08-11 16:30:08.371085 2014-08-13 17:38:02.603635 7fe9bae887a0 20 read_log 192888'432 (0'0) modify cf459884/benchmark_data_storage0111-ib.pocstor_35625_object8956/head//22 by client.940330.0:8957 2014-08-11 16:30:14.180582 2014-08-13 17:38:02.603655 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 1a881884/benchmark_data_storage0111-ib.pocstor_35625_object10500/head//22 by client.940330.0:10501 2014-08-11 16:30:19.802513 2014-08-13 17:38:02.603676 7fe9bae887a0 20 read_log 192888'434 (0'0) modify ad5e9084/benchmark_data_storage0111-ib.pocstor_35625_object10730/head//22 by client.940330.0:10731 2014-08-11 16:30:20.751785 2014-08-13 17:38:02.603699 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 33576084/benchmark_data_storage0111-ib.pocstor_35625_object11454/head//22 by client.940330.0:11455 2014-08-11 16:30:23.616894 2014-08-13 17:38:02.603719 7fe9bae887a0 20 read_log 192888'436 (0'0) modify b8ec0884/benchmark_data_storage0111-ib.pocstor_35625_object13040/head//22 by client.940330.0:13041 2014-08-11 16:30:29.995136 2014-08-13 17:38:02.603740 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 51b7b884/benchmark_data_storage0111-ib.pocstor_35625_object14783/head//22 by client.940330.0:14784 2014-08-11 16:30:36.684727 2014-08-13 17:38:02.603760 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 7dfd8884/benchmark_data_storage0111-ib.pocstor_35625_object15735/head//22 by client.940330.0:15736 2014-08-11 16:30:40.233008 2014-08-13 17:38:02.603781 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 408a9884/benchmark_data_storage0111-ib.pocstor_35625_object16251/head//22 by client.940330.0:16252 2014-08-11 16:30:42.169432 2014-08-13 17:38:02.603801 7fe9bae887a0 20 read_log 192888'440 (0'0) modify b6e47884/benchmark_data_storage0111-ib.pocstor_35625_object23093/head//22 by client.940330.0:23094 2014-08-11 16:31:08.756569 2014-08-13 17:38:02.603836 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 9f5b6084/benchmark_data_storage0111-ib.pocstor_35625_object24964/head//22 by client.940330.0:24965 2014-08-11 16:31:15.730467 2014-08-13 17:38:02.603858 7fe9bae887a0 20 read_log 192888'442 (0'0) modify d8b82084/benchmark_data_storage0111-ib.pocstor_35625_object26836/head//22 by client.940330.0:26837 2014-08-11 16:31:23.170115 2014-08-13 17:38:02.603879 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 49743084/benchmark_data_storage0111-ib.pocstor_35625_object26861/head//22 by client.940330.0:26862 2014-08-11 16:31:23.265734 2014-08-13 17:38:02.603900 7fe9bae887a0 20 read_log 192888'444 (192888'428) delete 36c52884/benchmark_data_storage0111-ib.pocstor_35625_object1157/head//22 by client.940330.0:32465 2014-08-11 16:31:41.787500 2014-08-13 17:38:02.603921 7fe9bae887a0 20 read_log 192888'445 (192888'429) delete 81389084/benchmark_data_storage0111-ib.pocstor_35625_object3335/head//22 by client.940330.0:34643 2014-08-11 16:31:42.344781 2014-08-13 17:38:02.603942 7fe9bae887a0 20 read_log 192888'446 (192888'430) delete b96f2884/benchmark_data_storage0111-ib.pocstor_35625_object6702/head//22 by client.940330.0:38010 2014-08-11 16:31:43.263000 2014-08-13 17:38:02.603963 7fe9bae887a0 20 read_log 192888'447 (192888'431) delete 40cef084/benchmark_data_storage0111-ib.pocstor_35625_object7298/head//22 by client.940330.0:38606 2014-08-11 16:31:43.414228 2014-08-13 17:38:02.603984 7fe9bae887a0 20 read_log 192888'448 (192888'432) delete cf459884/benchmark_data_storage0111-ib.pocstor_35625_object8956/head//22 by client.940330.0:40264 2014-08-11 16:31:43.830718 2014-08-13 17:38:02.604005 7fe9bae887a0 20 read_log 192888'449 (192888'433) delete 1a881884/benchmark_data_storage0111-ib.pocstor_35625_object10500/head//22 by client.940330.0:41808 2014-08-11 16:31:44.253052 2014-08-13 17:38:02.604061 7fe9bae887a0 20 read_log 192888'450 (192888'434) delete ad5e9084/benchmark_data_storage0111-ib.pocstor_35625_object10730/head//22 by client.940330.0:42038 2014-08-11 16:31:44.320351 2014-08-13 17:38:02.604086 7fe9bae887a0 20 read_log 192888'451 (192888'435) delete 33576084/benchmark_data_storage0111-ib.pocstor_35625_object11454/head//22 by client.940330.0:42762 2014-08-11 16:31:44.529030 2014-08-13 17:38:02.604108 7fe9bae887a0 20 read_log 192888'452 (192888'436) delete b8ec0884/benchmark_data_storage0111-ib.pocstor_35625_object13040/head//22 by client.940330.0:44348 2014-08-11 16:31:44.929482 2014-08-13 17:38:02.604129 7fe9bae887a0 20 read_log 192888'453 (192888'437) delete 51b7b884/benchmark_data_storage0111-ib.pocstor_35625_object14783/head//22 by client.940330.0:46091 2014-08-11 16:31:45.394940 2014-08-13 17:38:02.604151 7fe9bae887a0 20 read_log 192888'454 (192888'438) delete 7dfd8884/benchmark_data_storage0111-ib.pocstor_35625_object15735/head//22 by client.940330.0:47043 2014-08-11 16:31:45.654188 2014-08-13 17:38:02.604181 7fe9bae887a0 20 read_log 192888'455 (192888'439) delete 408a9884/benchmark_data_storage0111-ib.pocstor_35625_object16251/head//22 by client.940330.0:47559 2014-08-11 16:31:45.791300 2014-08-13 17:38:02.604202 7fe9bae887a0 20 read_log 192888'456 (192888'440) delete b6e47884/benchmark_data_storage0111-ib.pocstor_35625_object23093/head//22 by client.940330.0:54401 2014-08-11 16:31:47.789243 2014-08-13 17:38:02.604225 7fe9bae887a0 20 read_log 192888'457 (192888'441) delete 9f5b6084/benchmark_data_storage0111-ib.pocstor_35625_object24964/head//22 by client.940330.0:56272 2014-08-11 16:31:48.329240 2014-08-13 17:38:02.604246 7fe9bae887a0 20 read_log 192888'458 (192888'442) delete d8b82084/benchmark_data_storage0111-ib.pocstor_35625_object26836/head//22 by client.940330.0:58144 2014-08-11 16:31:48.882049 2014-08-13 17:38:02.604268 7fe9bae887a0 20 read_log 192888'459 (192888'443) delete 49743084/benchmark_data_storage0111-ib.pocstor_35625_object26861/head//22 by client.940330.0:58169 2014-08-11 16:31:48.890513 2014-08-13 17:38:02.604289 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 4fc3b884/benchmark_data_storage0111-ib.pocstor_64498_object792/head//22 by client.1049635.0:793 2014-08-12 17:17:19.683040 2014-08-13 17:38:02.604324 7fe9bae887a0 20 read_log 194215'461 (0'0) modify a4984884/benchmark_data_storage0111-ib.pocstor_64498_object5513/head//22 by client.1049635.0:5514 2014-08-12 17:17:34.654642 2014-08-13 17:38:02.604346 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 1943a084/benchmark_data_storage0111-ib.pocstor_64498_object9092/head//22 by client.1049635.0:9093 2014-08-12 17:17:45.285566 2014-08-13 17:38:02.604367 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 66030084/benchmark_data_storage0111-ib.pocstor_64498_object10169/head//22 by client.1049635.0:10170 2014-08-12 17:17:48.411223 2014-08-13 17:38:02.604388 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 10f56084/benchmark_data_storage0111-ib.pocstor_64498_object12292/head//22 by client.1049635.0:12293 2014-08-12 17:17:55.327966 2014-08-13 17:38:02.604409 7fe9bae887a0 20 read_log 194215'465 (0'0) modify cc164884/benchmark_data_storage0111-ib.pocstor_64498_object13405/head//22 by client.1049635.0:13406 2014-08-12 17:17:58.663314 2014-08-13 17:38:02.604430 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 756e084/benchmark_data_storage0111-ib.pocstor_64498_object14092/head//22 by client.1049635.0:14093 2014-08-12 17:18:00.797029 2014-08-13 17:38:02.604451 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 426ac884/benchmark_data_storage0111-ib.pocstor_64498_object15892/head//22 by client.1049635.0:15893 2014-08-12 17:18:07.151254 2014-08-13 17:38:02.604472 7fe9bae887a0 20 read_log 194215'468 (0'0) modify f5950884/benchmark_data_storage0111-ib.pocstor_64498_object17120/head//22 by client.1049635.0:17121 2014-08-12 17:18:11.822259 2014-08-13 17:38:02.604493 7fe9bae887a0 20 read_log 194215'469 (0'0) modify ed32f884/benchmark_data_storage0111-ib.pocstor_64498_object18769/head//22 by client.1049635.0:18770 2014-08-12 17:18:18.067064 2014-08-13 17:38:02.604514 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 42e7c084/benchmark_data_storage0111-ib.pocstor_64498_object18881/head//22 by client.1049635.0:18882 2014-08-12 17:18:18.481676 2014-08-13 17:38:02.604535 7fe9bae887a0 20 read_log 194215'471 (0'0) modify f2ae884/benchmark_data_storage0111-ib.pocstor_64498_object20404/head//22 by client.1049635.0:20405 2014-08-12 17:18:24.313106 2014-08-13 17:38:02.604559 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 4495d084/benchmark_data_storage0111-ib.pocstor_64498_object22141/head//22 by client.1049635.0:22142 2014-08-12 17:18:30.861306 2014-08-13 17:38:02.604580 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 32449884/benchmark_data_storage0111-ib.pocstor_64498_object24160/head//22 by client.1049635.0:24161 2014-08-12 17:18:38.448951 2014-08-13 17:38:02.604601 7fe9bae887a0 20 read_log 194215'474 (0'0) modify cce09084/benchmark_data_storage0111-ib.pocstor_64498_object26483/head//22 by client.1049635.0:26484 2014-08-12 17:18:47.227456 2014-08-13 17:38:02.604621 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 19ee6084/benchmark_data_storage0111-ib.pocstor_64498_object27649/head//22 by client.1049635.0:27650 2014-08-12 17:18:51.571620 2014-08-13 17:38:02.604642 7fe9bae887a0 20 read_log 194215'476 (0'0) modify c5a2e884/benchmark_data_storage0111-ib.pocstor_64498_object28642/head//22 by client.1049635.0:28643 2014-08-12 17:18:55.268252 2014-08-13 17:38:02.604663 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 9d68c884/benchmark_data_storage0111-ib.pocstor_64498_object29164/head//22 by client.1049635.0:29165 2014-08-12 17:18:57.208449 2014-08-13 17:38:02.604684 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 525c3884/benchmark_data_storage0111-ib.pocstor_64498_object33928/head//22 by client.1049635.0:33929 2014-08-12 17:19:15.908121 2014-08-13 17:38:02.604705 7fe9bae887a0 20 read_log 194215'479 (194215'460) delete 4fc3b884/benchmark_data_storage0111-ib.pocstor_64498_object792/head//22 by client.1049635.0:34995 2014-08-12 17:19:18.249519 2014-08-13 17:38:02.604726 7fe9bae887a0 20 read_log 194215'480 (194215'461) delete a4984884/benchmark_data_storage0111-ib.pocstor_64498_object5513/head//22 by client.1049635.0:39716 2014-08-12 17:19:19.775451 2014-08-13 17:38:02.604793 7fe9bae887a0 20 read_log 194215'481 (194215'462) delete 1943a084/benchmark_data_storage0111-ib.pocstor_64498_object9092/head//22 by client.1049635.0:43295 2014-08-12 17:19:20.951142 2014-08-13 17:38:02.604816 7fe9bae887a0 20 read_log 194215'482 (194215'463) delete 66030084/benchmark_data_storage0111-ib.pocstor_64498_object10169/head//22 by client.1049635.0:44372 2014-08-12 17:19:21.317144 2014-08-13 17:38:02.604838 7fe9bae887a0 20 read_log 194215'483 (194215'464) delete 10f56084/benchmark_data_storage0111-ib.pocstor_64498_object12292/head//22 by client.1049635.0:46495 2014-08-12 17:19:21.989034 2014-08-13 17:38:02.604859 7fe9bae887a0 20 read_log 194215'484 (194215'465) delete cc164884/benchmark_data_storage0111-ib.pocstor_64498_object13405/head//22 by client.1049635.0:47608 2014-08-12 17:19:22.362622 2014-08-13 17:38:02.604880 7fe9bae887a0 20 read_log 194215'485 (194215'466) delete 756e084/benchmark_data_storage0111-ib.pocstor_64498_object14092/head//22 by client.1049635.0:48295 2014-08-12 17:19:22.591906 2014-08-13 17:38:02.604901 7fe9bae887a0 20 read_log 194215'486 (194215'467) delete 426ac884/benchmark_data_storage0111-ib.pocstor_64498_object15892/head//22 by client.1049635.0:50095 2014-08-12 17:19:23.156735 2014-08-13 17:38:02.604922 7fe9bae887a0 20 read_log 194215'487 (194215'468) delete f5950884/benchmark_data_storage0111-ib.pocstor_64498_object17120/head//22 by client.1049635.0:51323 2014-08-12 17:19:23.544119 2014-08-13 17:38:02.604942 7fe9bae887a0 20 read_log 194215'488 (194215'469) delete ed32f884/benchmark_data_storage0111-ib.pocstor_64498_object18769/head//22 by client.1049635.0:52972 2014-08-12 17:19:24.086206 2014-08-13 17:38:02.604963 7fe9bae887a0 20 read_log 194215'489 (194215'470) delete 42e7c084/benchmark_data_storage0111-ib.pocstor_64498_object18881/head//22 by client.1049635.0:53084 2014-08-12 17:19:24.125119 2014-08-13 17:38:02.604983 7fe9bae887a0 20 read_log 194215'490 (194215'471) delete f2ae884/benchmark_data_storage0111-ib.pocstor_64498_object20404/head//22 by client.1049635.0:54607 2014-08-12 17:19:24.670421 2014-08-13 17:38:02.605007 7fe9bae887a0 20 read_log 194215'491 (194215'472) delete 4495d084/benchmark_data_storage0111-ib.pocstor_64498_object22141/head//22 by client.1049635.0:56344 2014-08-12 17:19:25.286401 2014-08-13 17:38:02.605028 7fe9bae887a0 20 read_log 194215'492 (194215'473) delete 32449884/benchmark_data_storage0111-ib.pocstor_64498_object24160/head//22 by client.1049635.0:58363 2014-08-12 17:19:26.017810 2014-08-13 17:38:02.605049 7fe9bae887a0 20 read_log 194215'493 (194215'474) delete cce09084/benchmark_data_storage0111-ib.pocstor_64498_object26483/head//22 by client.1049635.0:60686 2014-08-12 17:19:26.816526 2014-08-13 17:38:02.605069 7fe9bae887a0 20 read_log 194215'494 (194215'475) delete 19ee6084/benchmark_data_storage0111-ib.pocstor_64498_object27649/head//22 by client.1049635.0:61852 2014-08-12 17:19:27.229367 2014-08-13 17:38:02.605090 7fe9bae887a0 20 read_log 194215'495 (194215'476) delete c5a2e884/benchmark_data_storage0111-ib.pocstor_64498_object28642/head//22 by client.1049635.0:62845 2014-08-12 17:19:27.575359 2014-08-13 17:38:02.605112 7fe9bae887a0 20 read_log 194215'496 (194215'477) delete 9d68c884/benchmark_data_storage0111-ib.pocstor_64498_object29164/head//22 by client.1049635.0:63367 2014-08-12 17:19:27.766786 2014-08-13 17:38:02.605133 7fe9bae887a0 20 read_log 194215'497 (194215'478) delete 525c3884/benchmark_data_storage0111-ib.pocstor_64498_object33928/head//22 by client.1049635.0:68131 2014-08-12 17:19:29.452313 2014-08-13 17:38:02.605292 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.605328 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.605351 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002867 0 0.000000 2014-08-13 17:38:02.605370 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.605383 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.605397 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'497], crt=191648'337) 2014-08-13 17:38:02.605414 7fe9bae887a0 10 osd.4 194478 pgid 22.99 coll 22.99_head 2014-08-13 17:38:02.605606 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.99 2014-08-13 17:38:02.605617 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.605680 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.99(unlocked)] enter Initial 2014-08-13 17:38:02.605696 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.99(unlocked)] enter NotTrimming 2014-08-13 17:38:02.605913 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.606102 7fe9bae887a0 20 read_log 191648'2 (0'0) modify e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 by client.273158.0:185 2014-07-30 14:08:49.401106 2014-08-13 17:38:02.606133 7fe9bae887a0 20 read_log 191648'21 (0'0) modify ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 by client.273158.0:1462 2014-07-30 14:08:54.158576 2014-08-13 17:38:02.606165 7fe9bae887a0 20 read_log 191648'54 (0'0) modify 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 by client.273158.0:3013 2014-07-30 14:09:01.437743 2014-08-13 17:38:02.606188 7fe9bae887a0 20 read_log 191648'67 (0'0) modify a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 by client.273158.0:3715 2014-07-30 14:09:04.874600 2014-08-13 17:38:02.606212 7fe9bae887a0 20 read_log 191648'116 (0'0) modify c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 by client.273158.0:6616 2014-07-30 14:09:19.415313 2014-08-13 17:38:02.606236 7fe9bae887a0 20 read_log 191648'166 (0'0) modify b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 by client.273158.0:9878 2014-07-30 14:09:35.189406 2014-08-13 17:38:02.606257 7fe9bae887a0 20 read_log 191648'172 (0'0) modify 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 by client.273158.0:10200 2014-07-30 14:09:36.828464 2014-08-13 17:38:02.606310 7fe9bae887a0 20 read_log 191648'196 (0'0) modify 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 by client.273158.0:11571 2014-07-30 14:09:45.102648 2014-08-13 17:38:02.606331 7fe9bae887a0 20 read_log 191648'202 (0'0) modify d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 by client.273158.0:11893 2014-07-30 14:09:46.431387 2014-08-13 17:38:02.606353 7fe9bae887a0 20 read_log 191648'310 (0'0) modify 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 by client.273158.0:19422 2014-07-30 14:10:25.983506 2014-08-13 17:38:02.606374 7fe9bae887a0 20 read_log 191648'343 (0'0) modify b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 by client.273158.0:21536 2014-07-30 14:10:39.632431 2014-08-13 17:38:02.606418 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 by client.275175.0:754 2014-07-30 14:18:36.412444 2014-08-13 17:38:02.606442 7fe9bae887a0 20 read_log 191727'369 (0'0) modify 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 by client.275175.0:1570 2014-07-30 14:18:39.452929 2014-08-13 17:38:02.606463 7fe9bae887a0 20 read_log 191727'375 (0'0) modify ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 by client.275175.0:4555 2014-07-30 14:18:49.024949 2014-08-13 17:38:02.606484 7fe9bae887a0 20 read_log 191727'387 (0'0) modify c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 by client.275175.0:10914 2014-07-30 14:19:07.800919 2014-08-13 17:38:02.606505 7fe9bae887a0 20 read_log 191727'395 (0'0) modify a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 by client.275175.0:14436 2014-07-30 14:19:19.105976 2014-08-13 17:38:02.606525 7fe9bae887a0 20 read_log 191727'397 (0'0) modify e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 by client.275175.0:14939 2014-07-30 14:19:20.529496 2014-08-13 17:38:02.606546 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 by client.275175.0:15114 2014-07-30 14:19:21.090244 2014-08-13 17:38:02.606566 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 by client.275175.0:15494 2014-07-30 14:19:22.397025 2014-08-13 17:38:02.606587 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 by client.275175.0:19337 2014-07-30 14:19:34.319796 2014-08-13 17:38:02.606608 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 by client.275175.0:21149 2014-07-30 14:19:39.551859 2014-08-13 17:38:02.606628 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 by client.275175.0:21336 2014-07-30 14:19:40.119379 2014-08-13 17:38:02.606649 7fe9bae887a0 20 read_log 191727'420 (0'0) modify e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 by client.275175.0:23727 2014-07-30 14:19:48.063239 2014-08-13 17:38:02.606672 7fe9bae887a0 20 read_log 191727'423 (0'0) modify b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 by client.275175.0:25206 2014-07-30 14:19:52.433927 2014-08-13 17:38:02.606692 7fe9bae887a0 20 read_log 191727'428 (0'0) modify 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 by client.275175.0:29448 2014-07-30 14:20:05.231231 2014-08-13 17:38:02.606713 7fe9bae887a0 20 read_log 191727'432 (0'0) modify 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 by client.275175.0:30598 2014-07-30 14:20:08.846399 2014-08-13 17:38:02.606734 7fe9bae887a0 20 read_log 191727'434 (0'0) modify f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 by client.275175.0:32207 2014-07-30 14:20:13.496910 2014-08-13 17:38:02.606754 7fe9bae887a0 20 read_log 191727'440 (0'0) modify b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 by client.275175.0:38337 2014-07-30 14:20:32.203438 2014-08-13 17:38:02.606775 7fe9bae887a0 20 read_log 192674'441 (0'0) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8671 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606796 7fe9bae887a0 20 read_log 192674'442 (192674'441) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8672 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606817 7fe9bae887a0 20 read_log 192674'443 (192674'442) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8673 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606850 7fe9bae887a0 20 read_log 192674'444 (192674'443) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8674 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606872 7fe9bae887a0 20 read_log 192674'445 (192674'444) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8675 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606893 7fe9bae887a0 20 read_log 192674'446 (192674'445) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8676 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606913 7fe9bae887a0 20 read_log 192674'447 (192674'446) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8677 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606933 7fe9bae887a0 20 read_log 192674'448 (192674'447) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8678 2014-07-31 18:03:50.857000 2014-08-13 17:38:02.606975 7fe9bae887a0 20 read_log 192888'449 (0'0) modify fc63b099/benchmark_data_storage0111-ib.pocstor_35625_object1416/head//22 by client.940330.0:1417 2014-08-11 16:29:46.368792 2014-08-13 17:38:02.606998 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 4ee8c899/benchmark_data_storage0111-ib.pocstor_35625_object3692/head//22 by client.940330.0:3693 2014-08-11 16:29:55.207679 2014-08-13 17:38:02.607019 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 10403899/benchmark_data_storage0111-ib.pocstor_35625_object4202/head//22 by client.940330.0:4203 2014-08-11 16:29:57.065365 2014-08-13 17:38:02.607040 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 730f6099/benchmark_data_storage0111-ib.pocstor_35625_object6761/head//22 by client.940330.0:6762 2014-08-11 16:30:06.478093 2014-08-13 17:38:02.607061 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 27f82899/benchmark_data_storage0111-ib.pocstor_35625_object14667/head//22 by client.940330.0:14668 2014-08-11 16:30:36.255223 2014-08-13 17:38:02.607082 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 9568e099/benchmark_data_storage0111-ib.pocstor_35625_object20645/head//22 by client.940330.0:20646 2014-08-11 16:30:59.228846 2014-08-13 17:38:02.607102 7fe9bae887a0 20 read_log 192888'455 (0'0) modify abd87899/benchmark_data_storage0111-ib.pocstor_35625_object23696/head//22 by client.940330.0:23697 2014-08-11 16:31:11.037530 2014-08-13 17:38:02.607123 7fe9bae887a0 20 read_log 192888'456 (0'0) modify cbc83099/benchmark_data_storage0111-ib.pocstor_35625_object24625/head//22 by client.940330.0:24626 2014-08-11 16:31:14.437461 2014-08-13 17:38:02.607195 7fe9bae887a0 20 read_log 192888'457 (0'0) modify f3eac099/benchmark_data_storage0111-ib.pocstor_35625_object24808/head//22 by client.940330.0:24809 2014-08-11 16:31:15.125160 2014-08-13 17:38:02.607218 7fe9bae887a0 20 read_log 192888'458 (0'0) modify fb915099/benchmark_data_storage0111-ib.pocstor_35625_object26265/head//22 by client.940330.0:26266 2014-08-11 16:31:20.833187 2014-08-13 17:38:02.607239 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 97271899/benchmark_data_storage0111-ib.pocstor_35625_object30552/head//22 by client.940330.0:30553 2014-08-11 16:31:37.547016 2014-08-13 17:38:02.607262 7fe9bae887a0 20 read_log 192888'460 (192888'449) delete fc63b099/benchmark_data_storage0111-ib.pocstor_35625_object1416/head//22 by client.940330.0:32724 2014-08-11 16:31:41.847343 2014-08-13 17:38:02.607284 7fe9bae887a0 20 read_log 192888'461 (192888'450) delete 4ee8c899/benchmark_data_storage0111-ib.pocstor_35625_object3692/head//22 by client.940330.0:35000 2014-08-11 16:31:42.439818 2014-08-13 17:38:02.607305 7fe9bae887a0 20 read_log 192888'462 (192888'451) delete 10403899/benchmark_data_storage0111-ib.pocstor_35625_object4202/head//22 by client.940330.0:35510 2014-08-11 16:31:42.593220 2014-08-13 17:38:02.607329 7fe9bae887a0 20 read_log 192888'463 (192888'452) delete 730f6099/benchmark_data_storage0111-ib.pocstor_35625_object6761/head//22 by client.940330.0:38069 2014-08-11 16:31:43.278783 2014-08-13 17:38:02.607368 7fe9bae887a0 20 read_log 192888'464 (192888'453) delete 27f82899/benchmark_data_storage0111-ib.pocstor_35625_object14667/head//22 by client.940330.0:45975 2014-08-11 16:31:45.363455 2014-08-13 17:38:02.607390 7fe9bae887a0 20 read_log 192888'465 (192888'454) delete 9568e099/benchmark_data_storage0111-ib.pocstor_35625_object20645/head//22 by client.940330.0:51953 2014-08-11 16:31:47.064844 2014-08-13 17:38:02.607412 7fe9bae887a0 20 read_log 192888'466 (192888'455) delete abd87899/benchmark_data_storage0111-ib.pocstor_35625_object23696/head//22 by client.940330.0:55004 2014-08-11 16:31:47.950649 2014-08-13 17:38:02.607433 7fe9bae887a0 20 read_log 192888'467 (192888'456) delete cbc83099/benchmark_data_storage0111-ib.pocstor_35625_object24625/head//22 by client.940330.0:55933 2014-08-11 16:31:48.224016 2014-08-13 17:38:02.607454 7fe9bae887a0 20 read_log 192888'468 (192888'457) delete f3eac099/benchmark_data_storage0111-ib.pocstor_35625_object24808/head//22 by client.940330.0:56116 2014-08-11 16:31:48.275672 2014-08-13 17:38:02.607475 7fe9bae887a0 20 read_log 192888'469 (192888'458) delete fb915099/benchmark_data_storage0111-ib.pocstor_35625_object26265/head//22 by client.940330.0:57573 2014-08-11 16:31:48.723950 2014-08-13 17:38:02.607495 7fe9bae887a0 20 read_log 192888'470 (192888'459) delete 97271899/benchmark_data_storage0111-ib.pocstor_35625_object30552/head//22 by client.940330.0:61860 2014-08-11 16:31:49.915565 2014-08-13 17:38:02.607517 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 5a58c899/benchmark_data_storage0111-ib.pocstor_64498_object867/head//22 by client.1049635.0:868 2014-08-12 17:17:19.916802 2014-08-13 17:38:02.607537 7fe9bae887a0 20 read_log 194215'472 (0'0) modify f18bb899/benchmark_data_storage0111-ib.pocstor_64498_object4334/head//22 by client.1049635.0:4335 2014-08-12 17:17:31.042066 2014-08-13 17:38:02.607558 7fe9bae887a0 20 read_log 194215'473 (0'0) modify ee8c1899/benchmark_data_storage0111-ib.pocstor_64498_object4463/head//22 by client.1049635.0:4464 2014-08-12 17:17:31.399839 2014-08-13 17:38:02.607579 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 7bd8a099/benchmark_data_storage0111-ib.pocstor_64498_object12364/head//22 by client.1049635.0:12365 2014-08-12 17:17:55.523666 2014-08-13 17:38:02.607600 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 8c4a5099/benchmark_data_storage0111-ib.pocstor_64498_object12958/head//22 by client.1049635.0:12959 2014-08-12 17:17:57.236712 2014-08-13 17:38:02.607621 7fe9bae887a0 20 read_log 194215'476 (0'0) modify e4bb2899/benchmark_data_storage0111-ib.pocstor_64498_object13516/head//22 by client.1049635.0:13517 2014-08-12 17:17:59.033256 2014-08-13 17:38:02.607645 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 9c431899/benchmark_data_storage0111-ib.pocstor_64498_object13576/head//22 by client.1049635.0:13577 2014-08-12 17:17:59.278724 2014-08-13 17:38:02.607666 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 2319b899/benchmark_data_storage0111-ib.pocstor_64498_object20266/head//22 by client.1049635.0:20267 2014-08-12 17:18:23.787060 2014-08-13 17:38:02.607687 7fe9bae887a0 20 read_log 194215'479 (0'0) modify ed8e8099/benchmark_data_storage0111-ib.pocstor_64498_object23250/head//22 by client.1049635.0:23251 2014-08-12 17:18:35.061591 2014-08-13 17:38:02.607708 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 2e3d9899/benchmark_data_storage0111-ib.pocstor_64498_object24116/head//22 by client.1049635.0:24117 2014-08-12 17:18:38.284606 2014-08-13 17:38:02.607729 7fe9bae887a0 20 read_log 194215'481 (0'0) modify bcb54899/benchmark_data_storage0111-ib.pocstor_64498_object24337/head//22 by client.1049635.0:24338 2014-08-12 17:18:39.112025 2014-08-13 17:38:02.607750 7fe9bae887a0 20 read_log 194215'482 (0'0) modify f1a02099/benchmark_data_storage0111-ib.pocstor_64498_object24577/head//22 by client.1049635.0:24578 2014-08-12 17:18:40.007564 2014-08-13 17:38:02.607783 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 467bb899/benchmark_data_storage0111-ib.pocstor_64498_object25821/head//22 by client.1049635.0:25822 2014-08-12 17:18:44.759396 2014-08-13 17:38:02.607805 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 14320099/benchmark_data_storage0111-ib.pocstor_64498_object26372/head//22 by client.1049635.0:26373 2014-08-12 17:18:46.803700 2014-08-13 17:38:02.607826 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 56034899/benchmark_data_storage0111-ib.pocstor_64498_object27271/head//22 by client.1049635.0:27272 2014-08-12 17:18:50.156872 2014-08-13 17:38:02.607847 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 7d537099/benchmark_data_storage0111-ib.pocstor_64498_object29215/head//22 by client.1049635.0:29216 2014-08-12 17:18:57.392101 2014-08-13 17:38:02.607868 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 89178099/benchmark_data_storage0111-ib.pocstor_64498_object31705/head//22 by client.1049635.0:31706 2014-08-12 17:19:07.432383 2014-08-13 17:38:02.607889 7fe9bae887a0 20 read_log 194215'488 (194215'471) delete 5a58c899/benchmark_data_storage0111-ib.pocstor_64498_object867/head//22 by client.1049635.0:35070 2014-08-12 17:19:18.271290 2014-08-13 17:38:02.607911 7fe9bae887a0 20 read_log 194215'489 (194215'472) delete f18bb899/benchmark_data_storage0111-ib.pocstor_64498_object4334/head//22 by client.1049635.0:38537 2014-08-12 17:19:19.382495 2014-08-13 17:38:02.607932 7fe9bae887a0 20 read_log 194215'490 (194215'473) delete ee8c1899/benchmark_data_storage0111-ib.pocstor_64498_object4463/head//22 by client.1049635.0:38666 2014-08-12 17:19:19.426154 2014-08-13 17:38:02.607953 7fe9bae887a0 20 read_log 194215'491 (194215'474) delete 7bd8a099/benchmark_data_storage0111-ib.pocstor_64498_object12364/head//22 by client.1049635.0:46567 2014-08-12 17:19:22.011447 2014-08-13 17:38:02.607974 7fe9bae887a0 20 read_log 194215'492 (194215'475) delete 8c4a5099/benchmark_data_storage0111-ib.pocstor_64498_object12958/head//22 by client.1049635.0:47161 2014-08-12 17:19:22.199112 2014-08-13 17:38:02.607995 7fe9bae887a0 20 read_log 194215'493 (194215'476) delete e4bb2899/benchmark_data_storage0111-ib.pocstor_64498_object13516/head//22 by client.1049635.0:47719 2014-08-12 17:19:22.405676 2014-08-13 17:38:02.608015 7fe9bae887a0 20 read_log 194215'494 (194215'477) delete 9c431899/benchmark_data_storage0111-ib.pocstor_64498_object13576/head//22 by client.1049635.0:47779 2014-08-12 17:19:22.431245 2014-08-13 17:38:02.608036 7fe9bae887a0 20 read_log 194215'495 (194215'478) delete 2319b899/benchmark_data_storage0111-ib.pocstor_64498_object20266/head//22 by client.1049635.0:54469 2014-08-12 17:19:24.620091 2014-08-13 17:38:02.608056 7fe9bae887a0 20 read_log 194215'496 (194215'479) delete ed8e8099/benchmark_data_storage0111-ib.pocstor_64498_object23250/head//22 by client.1049635.0:57453 2014-08-12 17:19:25.708866 2014-08-13 17:38:02.608077 7fe9bae887a0 20 read_log 194215'497 (194215'480) delete 2e3d9899/benchmark_data_storage0111-ib.pocstor_64498_object24116/head//22 by client.1049635.0:58319 2014-08-12 17:19:26.002724 2014-08-13 17:38:02.608098 7fe9bae887a0 20 read_log 194215'498 (194215'481) delete bcb54899/benchmark_data_storage0111-ib.pocstor_64498_object24337/head//22 by client.1049635.0:58540 2014-08-12 17:19:26.078611 2014-08-13 17:38:02.608119 7fe9bae887a0 20 read_log 194215'499 (194215'482) delete f1a02099/benchmark_data_storage0111-ib.pocstor_64498_object24577/head//22 by client.1049635.0:58780 2014-08-12 17:19:26.165694 2014-08-13 17:38:02.608140 7fe9bae887a0 20 read_log 194215'500 (194215'483) delete 467bb899/benchmark_data_storage0111-ib.pocstor_64498_object25821/head//22 by client.1049635.0:60024 2014-08-12 17:19:26.597681 2014-08-13 17:38:02.608169 7fe9bae887a0 20 read_log 194215'501 (194215'484) delete 14320099/benchmark_data_storage0111-ib.pocstor_64498_object26372/head//22 by client.1049635.0:60575 2014-08-12 17:19:26.782295 2014-08-13 17:38:02.608191 7fe9bae887a0 20 read_log 194215'502 (194215'485) delete 56034899/benchmark_data_storage0111-ib.pocstor_64498_object27271/head//22 by client.1049635.0:61474 2014-08-12 17:19:27.097012 2014-08-13 17:38:02.608224 7fe9bae887a0 20 read_log 194215'503 (194215'486) delete 7d537099/benchmark_data_storage0111-ib.pocstor_64498_object29215/head//22 by client.1049635.0:63418 2014-08-12 17:19:27.781807 2014-08-13 17:38:02.608247 7fe9bae887a0 20 read_log 194215'504 (194215'487) delete 89178099/benchmark_data_storage0111-ib.pocstor_64498_object31705/head//22 by client.1049635.0:65908 2014-08-12 17:19:28.642175 2014-08-13 17:38:02.608383 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.608412 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=0 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.608435 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=0 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002756 0 0.000000 2014-08-13 17:38:02.608455 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=0 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.608470 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=0 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.608485 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'504], crt=191648'365) 2014-08-13 17:38:02.608501 7fe9bae887a0 10 osd.4 194478 pgid 22.f3 coll 22.f3_head 2014-08-13 17:38:02.608716 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.f3 2014-08-13 17:38:02.608727 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.608772 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.f3(unlocked)] enter Initial 2014-08-13 17:38:02.608787 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.f3(unlocked)] enter NotTrimming 2014-08-13 17:38:02.609001 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.609213 7fe9bae887a0 20 read_log 191648'159 (0'0) modify 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 by client.273158.0:10812 2014-07-30 14:09:40.255053 2014-08-13 17:38:02.609244 7fe9bae887a0 20 read_log 191648'163 (0'0) modify 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 by client.273158.0:11293 2014-07-30 14:09:43.480720 2014-08-13 17:38:02.609266 7fe9bae887a0 20 read_log 191648'219 (0'0) modify e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 by client.273158.0:14986 2014-07-30 14:10:02.628657 2014-08-13 17:38:02.609287 7fe9bae887a0 20 read_log 191648'326 (0'0) modify 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 by client.273158.0:21658 2014-07-30 14:10:41.075122 2014-08-13 17:38:02.609311 7fe9bae887a0 20 read_log 191648'332 (0'0) modify f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 by client.273158.0:22197 2014-07-30 14:10:44.963874 2014-08-13 17:38:02.609332 7fe9bae887a0 20 read_log 191727'344 (0'0) modify b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 by client.275175.0:772 2014-07-30 14:18:36.498159 2014-08-13 17:38:02.609353 7fe9bae887a0 20 read_log 191727'347 (0'0) modify 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 by client.275175.0:1539 2014-07-30 14:18:39.343225 2014-08-13 17:38:02.609386 7fe9bae887a0 20 read_log 191727'351 (0'0) modify dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 by client.275175.0:2722 2014-07-30 14:18:43.326956 2014-08-13 17:38:02.609408 7fe9bae887a0 20 read_log 191727'353 (0'0) modify 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 by client.275175.0:3410 2014-07-30 14:18:45.512736 2014-08-13 17:38:02.609429 7fe9bae887a0 20 read_log 191727'355 (0'0) modify bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 by client.275175.0:5007 2014-07-30 14:18:50.370849 2014-08-13 17:38:02.609450 7fe9bae887a0 20 read_log 191727'357 (0'0) modify 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 by client.275175.0:5814 2014-07-30 14:18:52.707995 2014-08-13 17:38:02.609471 7fe9bae887a0 20 read_log 191727'360 (0'0) modify d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 by client.275175.0:6318 2014-07-30 14:18:54.202712 2014-08-13 17:38:02.609491 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 by client.275175.0:14781 2014-07-30 14:19:20.075602 2014-08-13 17:38:02.609515 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 by client.275175.0:15186 2014-07-30 14:19:21.330810 2014-08-13 17:38:02.609535 7fe9bae887a0 20 read_log 191727'377 (0'0) modify c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 by client.275175.0:18666 2014-07-30 14:19:31.947870 2014-08-13 17:38:02.609556 7fe9bae887a0 20 read_log 191727'378 (0'0) modify aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 by client.275175.0:20665 2014-07-30 14:19:38.168090 2014-08-13 17:38:02.609577 7fe9bae887a0 20 read_log 191727'380 (0'0) modify a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 by client.275175.0:20875 2014-07-30 14:19:38.772419 2014-08-13 17:38:02.609599 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 by client.275175.0:22622 2014-07-30 14:19:44.766224 2014-08-13 17:38:02.609620 7fe9bae887a0 20 read_log 191727'388 (0'0) modify a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 by client.275175.0:23630 2014-07-30 14:19:47.790262 2014-08-13 17:38:02.609640 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 by client.275175.0:23748 2014-07-30 14:19:48.121416 2014-08-13 17:38:02.609661 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 by client.275175.0:24358 2014-07-30 14:19:49.829737 2014-08-13 17:38:02.609681 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 by client.275175.0:26651 2014-07-30 14:19:56.778705 2014-08-13 17:38:02.609702 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 by client.275175.0:27046 2014-07-30 14:19:57.938973 2014-08-13 17:38:02.609722 7fe9bae887a0 20 read_log 191727'397 (0'0) modify acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 by client.275175.0:27158 2014-07-30 14:19:58.259424 2014-08-13 17:38:02.609743 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 by client.275175.0:27195 2014-07-30 14:19:58.367206 2014-08-13 17:38:02.609764 7fe9bae887a0 20 read_log 191727'405 (0'0) modify f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 by client.275175.0:31675 2014-07-30 14:20:11.938008 2014-08-13 17:38:02.609784 7fe9bae887a0 20 read_log 191727'409 (0'0) modify 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 by client.275175.0:35270 2014-07-30 14:20:22.784675 2014-08-13 17:38:02.609824 7fe9bae887a0 20 read_log 191727'414 (0'0) modify aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 by client.275175.0:37144 2014-07-30 14:20:28.774611 2014-08-13 17:38:02.609847 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 by client.275175.0:37255 2014-07-30 14:20:29.080004 2014-08-13 17:38:02.609868 7fe9bae887a0 20 read_log 192888'417 (0'0) modify ede050f3/benchmark_data_storage0111-ib.pocstor_35625_object102/head//22 by client.940330.0:103 2014-08-11 16:29:40.940627 2014-08-13 17:38:02.609888 7fe9bae887a0 20 read_log 192888'418 (0'0) modify 93d410f3/benchmark_data_storage0111-ib.pocstor_35625_object1085/head//22 by client.940330.0:1086 2014-08-11 16:29:45.110786 2014-08-13 17:38:02.609909 7fe9bae887a0 20 read_log 192888'419 (0'0) modify b955e0f3/benchmark_data_storage0111-ib.pocstor_35625_object1305/head//22 by client.940330.0:1306 2014-08-11 16:29:45.926415 2014-08-13 17:38:02.609930 7fe9bae887a0 20 read_log 192888'420 (0'0) modify ebaaa0f3/benchmark_data_storage0111-ib.pocstor_35625_object1953/head//22 by client.940330.0:1954 2014-08-11 16:29:48.364829 2014-08-13 17:38:02.609950 7fe9bae887a0 20 read_log 192888'421 (0'0) modify 8387f0f3/benchmark_data_storage0111-ib.pocstor_35625_object2593/head//22 by client.940330.0:2594 2014-08-11 16:29:50.836801 2014-08-13 17:38:02.609971 7fe9bae887a0 20 read_log 192888'422 (0'0) modify 5df5f8f3/benchmark_data_storage0111-ib.pocstor_35625_object2757/head//22 by client.940330.0:2758 2014-08-11 16:29:51.464343 2014-08-13 17:38:02.609994 7fe9bae887a0 20 read_log 192888'423 (0'0) modify 69a528f3/benchmark_data_storage0111-ib.pocstor_35625_object4192/head//22 by client.940330.0:4193 2014-08-11 16:29:57.030578 2014-08-13 17:38:02.610015 7fe9bae887a0 20 read_log 192888'424 (0'0) modify 76a6c0f3/benchmark_data_storage0111-ib.pocstor_35625_object6937/head//22 by client.940330.0:6938 2014-08-11 16:30:07.096967 2014-08-13 17:38:02.610037 7fe9bae887a0 20 read_log 192888'425 (0'0) modify 438d80f3/benchmark_data_storage0111-ib.pocstor_35625_object8266/head//22 by client.940330.0:8267 2014-08-11 16:30:11.770462 2014-08-13 17:38:02.610058 7fe9bae887a0 20 read_log 192888'426 (0'0) modify 8f1f40f3/benchmark_data_storage0111-ib.pocstor_35625_object10408/head//22 by client.940330.0:10409 2014-08-11 16:30:19.334590 2014-08-13 17:38:02.610078 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 72d960f3/benchmark_data_storage0111-ib.pocstor_35625_object12528/head//22 by client.940330.0:12529 2014-08-11 16:30:27.998239 2014-08-13 17:38:02.610099 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 5b4b58f3/benchmark_data_storage0111-ib.pocstor_35625_object13002/head//22 by client.940330.0:13003 2014-08-11 16:30:29.799170 2014-08-13 17:38:02.610119 7fe9bae887a0 20 read_log 192888'429 (0'0) modify f07590f3/benchmark_data_storage0111-ib.pocstor_35625_object16269/head//22 by client.940330.0:16270 2014-08-11 16:30:42.235521 2014-08-13 17:38:02.610176 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 349030f3/benchmark_data_storage0111-ib.pocstor_35625_object16327/head//22 by client.940330.0:16328 2014-08-11 16:30:42.450013 2014-08-13 17:38:02.610199 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 9da710f3/benchmark_data_storage0111-ib.pocstor_35625_object16516/head//22 by client.940330.0:16517 2014-08-11 16:30:43.155094 2014-08-13 17:38:02.610220 7fe9bae887a0 20 read_log 192888'432 (0'0) modify df1ea8f3/benchmark_data_storage0111-ib.pocstor_35625_object16937/head//22 by client.940330.0:16938 2014-08-11 16:30:44.730897 2014-08-13 17:38:02.610240 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 9a9b90f3/benchmark_data_storage0111-ib.pocstor_35625_object17532/head//22 by client.940330.0:17533 2014-08-11 16:30:46.957078 2014-08-13 17:38:02.610274 7fe9bae887a0 20 read_log 192888'434 (0'0) modify e34040f3/benchmark_data_storage0111-ib.pocstor_35625_object19743/head//22 by client.940330.0:19744 2014-08-11 16:30:55.787083 2014-08-13 17:38:02.610295 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 800ba8f3/benchmark_data_storage0111-ib.pocstor_35625_object24244/head//22 by client.940330.0:24245 2014-08-11 16:31:13.039710 2014-08-13 17:38:02.610316 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 23a640f3/benchmark_data_storage0111-ib.pocstor_35625_object25406/head//22 by client.940330.0:25407 2014-08-11 16:31:17.475217 2014-08-13 17:38:02.610337 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 7c0760f3/benchmark_data_storage0111-ib.pocstor_35625_object25751/head//22 by client.940330.0:25752 2014-08-11 16:31:18.814698 2014-08-13 17:38:02.610358 7fe9bae887a0 20 read_log 192888'438 (0'0) modify df9d80f3/benchmark_data_storage0111-ib.pocstor_35625_object25874/head//22 by client.940330.0:25875 2014-08-11 16:31:19.297911 2014-08-13 17:38:02.610378 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 3d4dc0f3/benchmark_data_storage0111-ib.pocstor_35625_object29385/head//22 by client.940330.0:29386 2014-08-11 16:31:33.134814 2014-08-13 17:38:02.610401 7fe9bae887a0 20 read_log 192888'440 (0'0) modify a96ce0f3/benchmark_data_storage0111-ib.pocstor_35625_object30850/head//22 by client.940330.0:30851 2014-08-11 16:31:38.656249 2014-08-13 17:38:02.610422 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 4f5540f3/benchmark_data_storage0111-ib.pocstor_35625_object31154/head//22 by client.940330.0:31155 2014-08-11 16:31:39.800042 2014-08-13 17:38:02.610442 7fe9bae887a0 20 read_log 192888'442 (192888'417) delete ede050f3/benchmark_data_storage0111-ib.pocstor_35625_object102/head//22 by client.940330.0:31410 2014-08-11 16:31:41.480819 2014-08-13 17:38:02.610463 7fe9bae887a0 20 read_log 192888'443 (192888'418) delete 93d410f3/benchmark_data_storage0111-ib.pocstor_35625_object1085/head//22 by client.940330.0:32393 2014-08-11 16:31:41.771030 2014-08-13 17:38:02.610485 7fe9bae887a0 20 read_log 192888'444 (192888'419) delete b955e0f3/benchmark_data_storage0111-ib.pocstor_35625_object1305/head//22 by client.940330.0:32613 2014-08-11 16:31:41.821440 2014-08-13 17:38:02.610506 7fe9bae887a0 20 read_log 192888'445 (192888'420) delete ebaaa0f3/benchmark_data_storage0111-ib.pocstor_35625_object1953/head//22 by client.940330.0:33261 2014-08-11 16:31:41.980003 2014-08-13 17:38:02.610527 7fe9bae887a0 20 read_log 192888'446 (192888'421) delete 8387f0f3/benchmark_data_storage0111-ib.pocstor_35625_object2593/head//22 by client.940330.0:33901 2014-08-11 16:31:42.143719 2014-08-13 17:38:02.610548 7fe9bae887a0 20 read_log 192888'447 (192888'422) delete 5df5f8f3/benchmark_data_storage0111-ib.pocstor_35625_object2757/head//22 by client.940330.0:34065 2014-08-11 16:31:42.185179 2014-08-13 17:38:02.610608 7fe9bae887a0 20 read_log 192888'448 (192888'423) delete 69a528f3/benchmark_data_storage0111-ib.pocstor_35625_object4192/head//22 by client.940330.0:35500 2014-08-11 16:31:42.590687 2014-08-13 17:38:02.610630 7fe9bae887a0 20 read_log 192888'449 (192888'424) delete 76a6c0f3/benchmark_data_storage0111-ib.pocstor_35625_object6937/head//22 by client.940330.0:38245 2014-08-11 16:31:43.319425 2014-08-13 17:38:02.610651 7fe9bae887a0 20 read_log 192888'450 (192888'425) delete 438d80f3/benchmark_data_storage0111-ib.pocstor_35625_object8266/head//22 by client.940330.0:39574 2014-08-11 16:31:43.658641 2014-08-13 17:38:02.610672 7fe9bae887a0 20 read_log 192888'451 (192888'426) delete 8f1f40f3/benchmark_data_storage0111-ib.pocstor_35625_object10408/head//22 by client.940330.0:41716 2014-08-11 16:31:44.227474 2014-08-13 17:38:02.610693 7fe9bae887a0 20 read_log 192888'452 (192888'427) delete 72d960f3/benchmark_data_storage0111-ib.pocstor_35625_object12528/head//22 by client.940330.0:43836 2014-08-11 16:31:44.790119 2014-08-13 17:38:02.610714 7fe9bae887a0 20 read_log 192888'453 (192888'428) delete 5b4b58f3/benchmark_data_storage0111-ib.pocstor_35625_object13002/head//22 by client.940330.0:44310 2014-08-11 16:31:44.918773 2014-08-13 17:38:02.610748 7fe9bae887a0 20 read_log 192888'454 (192888'429) delete f07590f3/benchmark_data_storage0111-ib.pocstor_35625_object16269/head//22 by client.940330.0:47577 2014-08-11 16:31:45.796495 2014-08-13 17:38:02.610771 7fe9bae887a0 20 read_log 192888'455 (192888'430) delete 349030f3/benchmark_data_storage0111-ib.pocstor_35625_object16327/head//22 by client.940330.0:47635 2014-08-11 16:31:45.810418 2014-08-13 17:38:02.610792 7fe9bae887a0 20 read_log 192888'456 (192888'431) delete 9da710f3/benchmark_data_storage0111-ib.pocstor_35625_object16516/head//22 by client.940330.0:47824 2014-08-11 16:31:45.858683 2014-08-13 17:38:02.610813 7fe9bae887a0 20 read_log 192888'457 (192888'432) delete df1ea8f3/benchmark_data_storage0111-ib.pocstor_35625_object16937/head//22 by client.940330.0:48245 2014-08-11 16:31:45.970799 2014-08-13 17:38:02.610834 7fe9bae887a0 20 read_log 192888'458 (192888'433) delete 9a9b90f3/benchmark_data_storage0111-ib.pocstor_35625_object17532/head//22 by client.940330.0:48840 2014-08-11 16:31:46.137092 2014-08-13 17:38:02.610855 7fe9bae887a0 20 read_log 192888'459 (192888'434) delete e34040f3/benchmark_data_storage0111-ib.pocstor_35625_object19743/head//22 by client.940330.0:51051 2014-08-11 16:31:46.793475 2014-08-13 17:38:02.610876 7fe9bae887a0 20 read_log 192888'460 (192888'435) delete 800ba8f3/benchmark_data_storage0111-ib.pocstor_35625_object24244/head//22 by client.940330.0:55552 2014-08-11 16:31:48.112868 2014-08-13 17:38:02.610896 7fe9bae887a0 20 read_log 192888'461 (192888'436) delete 23a640f3/benchmark_data_storage0111-ib.pocstor_35625_object25406/head//22 by client.940330.0:56714 2014-08-11 16:31:48.469191 2014-08-13 17:38:02.610917 7fe9bae887a0 20 read_log 192888'462 (192888'437) delete 7c0760f3/benchmark_data_storage0111-ib.pocstor_35625_object25751/head//22 by client.940330.0:57059 2014-08-11 16:31:48.573741 2014-08-13 17:38:02.610964 7fe9bae887a0 20 read_log 192888'463 (192888'438) delete df9d80f3/benchmark_data_storage0111-ib.pocstor_35625_object25874/head//22 by client.940330.0:57182 2014-08-11 16:31:48.609202 2014-08-13 17:38:02.610986 7fe9bae887a0 20 read_log 192888'464 (192888'439) delete 3d4dc0f3/benchmark_data_storage0111-ib.pocstor_35625_object29385/head//22 by client.940330.0:60693 2014-08-11 16:31:49.579925 2014-08-13 17:38:02.611007 7fe9bae887a0 20 read_log 192888'465 (192888'440) delete a96ce0f3/benchmark_data_storage0111-ib.pocstor_35625_object30850/head//22 by client.940330.0:62158 2014-08-11 16:31:50.008413 2014-08-13 17:38:02.611028 7fe9bae887a0 20 read_log 192888'466 (192888'441) delete 4f5540f3/benchmark_data_storage0111-ib.pocstor_35625_object31154/head//22 by client.940330.0:62462 2014-08-11 16:31:50.106344 2014-08-13 17:38:02.611050 7fe9bae887a0 20 read_log 194215'467 (0'0) modify c933f8f3/benchmark_data_storage0111-ib.pocstor_64498_object4282/head//22 by client.1049635.0:4283 2014-08-12 17:17:30.900696 2014-08-13 17:38:02.611071 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 5bbf98f3/benchmark_data_storage0111-ib.pocstor_64498_object5387/head//22 by client.1049635.0:5388 2014-08-12 17:17:34.120280 2014-08-13 17:38:02.611091 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 2056f0f3/benchmark_data_storage0111-ib.pocstor_64498_object8530/head//22 by client.1049635.0:8531 2014-08-12 17:17:43.660009 2014-08-13 17:38:02.611113 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 7400f0f3/benchmark_data_storage0111-ib.pocstor_64498_object10337/head//22 by client.1049635.0:10338 2014-08-12 17:17:48.885257 2014-08-13 17:38:02.611134 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 2fbb90f3/benchmark_data_storage0111-ib.pocstor_64498_object12591/head//22 by client.1049635.0:12592 2014-08-12 17:17:56.146414 2014-08-13 17:38:02.611165 7fe9bae887a0 20 read_log 194215'472 (0'0) modify dcde18f3/benchmark_data_storage0111-ib.pocstor_64498_object12798/head//22 by client.1049635.0:12799 2014-08-12 17:17:56.738055 2014-08-13 17:38:02.611204 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 8f2420f3/benchmark_data_storage0111-ib.pocstor_64498_object15056/head//22 by client.1049635.0:15057 2014-08-12 17:18:04.152103 2014-08-13 17:38:02.611228 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 6f2f88f3/benchmark_data_storage0111-ib.pocstor_64498_object15434/head//22 by client.1049635.0:15435 2014-08-12 17:18:05.415548 2014-08-13 17:38:02.611249 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 4f6b70f3/benchmark_data_storage0111-ib.pocstor_64498_object16535/head//22 by client.1049635.0:16536 2014-08-12 17:18:09.595642 2014-08-13 17:38:02.611270 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 1b7b68f3/benchmark_data_storage0111-ib.pocstor_64498_object19505/head//22 by client.1049635.0:19506 2014-08-12 17:18:20.926361 2014-08-13 17:38:02.611291 7fe9bae887a0 20 read_log 194215'477 (0'0) modify d8a7b0f3/benchmark_data_storage0111-ib.pocstor_64498_object22023/head//22 by client.1049635.0:22024 2014-08-12 17:18:30.415875 2014-08-13 17:38:02.611313 7fe9bae887a0 20 read_log 194215'478 (0'0) modify ded0d0f3/benchmark_data_storage0111-ib.pocstor_64498_object23683/head//22 by client.1049635.0:23684 2014-08-12 17:18:36.678849 2014-08-13 17:38:02.611334 7fe9bae887a0 20 read_log 194215'479 (0'0) modify ecd880f3/benchmark_data_storage0111-ib.pocstor_64498_object24505/head//22 by client.1049635.0:24506 2014-08-12 17:18:39.734691 2014-08-13 17:38:02.611357 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 8594a8f3/benchmark_data_storage0111-ib.pocstor_64498_object24824/head//22 by client.1049635.0:24825 2014-08-12 17:18:40.951288 2014-08-13 17:38:02.611378 7fe9bae887a0 20 read_log 194215'481 (0'0) modify c3b8d0f3/benchmark_data_storage0111-ib.pocstor_64498_object24852/head//22 by client.1049635.0:24853 2014-08-12 17:18:41.103829 2014-08-13 17:38:02.611399 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 72dd18f3/benchmark_data_storage0111-ib.pocstor_64498_object31703/head//22 by client.1049635.0:31704 2014-08-12 17:19:07.424649 2014-08-13 17:38:02.611421 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 288d68f3/benchmark_data_storage0111-ib.pocstor_64498_object32568/head//22 by client.1049635.0:32569 2014-08-12 17:19:10.753373 2014-08-13 17:38:02.611442 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 5d2d68f3/benchmark_data_storage0111-ib.pocstor_64498_object33193/head//22 by client.1049635.0:33194 2014-08-12 17:19:13.120862 2014-08-13 17:38:02.611463 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 4cab58f3/benchmark_data_storage0111-ib.pocstor_64498_object33522/head//22 by client.1049635.0:33523 2014-08-12 17:19:14.368449 2014-08-13 17:38:02.611484 7fe9bae887a0 20 read_log 194215'486 (0'0) modify c3f9d0f3/benchmark_data_storage0111-ib.pocstor_64498_object33780/head//22 by client.1049635.0:33781 2014-08-12 17:19:15.348101 2014-08-13 17:38:02.611505 7fe9bae887a0 20 read_log 194215'487 (194215'467) delete c933f8f3/benchmark_data_storage0111-ib.pocstor_64498_object4282/head//22 by client.1049635.0:38485 2014-08-12 17:19:19.366885 2014-08-13 17:38:02.611526 7fe9bae887a0 20 read_log 194215'488 (194215'468) delete 5bbf98f3/benchmark_data_storage0111-ib.pocstor_64498_object5387/head//22 by client.1049635.0:39590 2014-08-12 17:19:19.734019 2014-08-13 17:38:02.611547 7fe9bae887a0 20 read_log 194215'489 (194215'469) delete 2056f0f3/benchmark_data_storage0111-ib.pocstor_64498_object8530/head//22 by client.1049635.0:42733 2014-08-12 17:19:20.770215 2014-08-13 17:38:02.611569 7fe9bae887a0 20 read_log 194215'490 (194215'470) delete 7400f0f3/benchmark_data_storage0111-ib.pocstor_64498_object10337/head//22 by client.1049635.0:44540 2014-08-12 17:19:21.371302 2014-08-13 17:38:02.611590 7fe9bae887a0 20 read_log 194215'491 (194215'471) delete 2fbb90f3/benchmark_data_storage0111-ib.pocstor_64498_object12591/head//22 by client.1049635.0:46794 2014-08-12 17:19:22.082017 2014-08-13 17:38:02.611611 7fe9bae887a0 20 read_log 194215'492 (194215'472) delete dcde18f3/benchmark_data_storage0111-ib.pocstor_64498_object12798/head//22 by client.1049635.0:47001 2014-08-12 17:19:22.147529 2014-08-13 17:38:02.611649 7fe9bae887a0 20 read_log 194215'493 (194215'473) delete 8f2420f3/benchmark_data_storage0111-ib.pocstor_64498_object15056/head//22 by client.1049635.0:49259 2014-08-12 17:19:22.891587 2014-08-13 17:38:02.611672 7fe9bae887a0 20 read_log 194215'494 (194215'474) delete 6f2f88f3/benchmark_data_storage0111-ib.pocstor_64498_object15434/head//22 by client.1049635.0:49637 2014-08-12 17:19:23.012225 2014-08-13 17:38:02.611693 7fe9bae887a0 20 read_log 194215'495 (194215'475) delete 4f6b70f3/benchmark_data_storage0111-ib.pocstor_64498_object16535/head//22 by client.1049635.0:50738 2014-08-12 17:19:23.362654 2014-08-13 17:38:02.611714 7fe9bae887a0 20 read_log 194215'496 (194215'476) delete 1b7b68f3/benchmark_data_storage0111-ib.pocstor_64498_object19505/head//22 by client.1049635.0:53708 2014-08-12 17:19:24.351261 2014-08-13 17:38:02.611735 7fe9bae887a0 20 read_log 194215'497 (194215'477) delete d8a7b0f3/benchmark_data_storage0111-ib.pocstor_64498_object22023/head//22 by client.1049635.0:56226 2014-08-12 17:19:25.245997 2014-08-13 17:38:02.611756 7fe9bae887a0 20 read_log 194215'498 (194215'478) delete ded0d0f3/benchmark_data_storage0111-ib.pocstor_64498_object23683/head//22 by client.1049635.0:57886 2014-08-12 17:19:25.860231 2014-08-13 17:38:02.611777 7fe9bae887a0 20 read_log 194215'499 (194215'479) delete ecd880f3/benchmark_data_storage0111-ib.pocstor_64498_object24505/head//22 by client.1049635.0:58708 2014-08-12 17:19:26.137638 2014-08-13 17:38:02.611798 7fe9bae887a0 20 read_log 194215'500 (194215'480) delete 8594a8f3/benchmark_data_storage0111-ib.pocstor_64498_object24824/head//22 by client.1049635.0:59027 2014-08-12 17:19:26.263642 2014-08-13 17:38:02.611818 7fe9bae887a0 20 read_log 194215'501 (194215'481) delete c3b8d0f3/benchmark_data_storage0111-ib.pocstor_64498_object24852/head//22 by client.1049635.0:59055 2014-08-12 17:19:26.274854 2014-08-13 17:38:02.611839 7fe9bae887a0 20 read_log 194215'502 (194215'482) delete 72dd18f3/benchmark_data_storage0111-ib.pocstor_64498_object31703/head//22 by client.1049635.0:65906 2014-08-12 17:19:28.641464 2014-08-13 17:38:02.611860 7fe9bae887a0 20 read_log 194215'503 (194215'483) delete 288d68f3/benchmark_data_storage0111-ib.pocstor_64498_object32568/head//22 by client.1049635.0:66771 2014-08-12 17:19:28.954974 2014-08-13 17:38:02.611881 7fe9bae887a0 20 read_log 194215'504 (194215'484) delete 5d2d68f3/benchmark_data_storage0111-ib.pocstor_64498_object33193/head//22 by client.1049635.0:67396 2014-08-12 17:19:29.182515 2014-08-13 17:38:02.611902 7fe9bae887a0 20 read_log 194215'505 (194215'485) delete 4cab58f3/benchmark_data_storage0111-ib.pocstor_64498_object33522/head//22 by client.1049635.0:67725 2014-08-12 17:19:29.299821 2014-08-13 17:38:02.611922 7fe9bae887a0 20 read_log 194215'506 (194215'486) delete c3f9d0f3/benchmark_data_storage0111-ib.pocstor_64498_object33780/head//22 by client.1049635.0:67983 2014-08-12 17:19:29.398243 2014-08-13 17:38:02.612092 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.612121 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=0 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.612144 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=0 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003373 0 0.000000 2014-08-13 17:38:02.612173 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=0 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.612188 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=0 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.612203 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'506], crt=191648'343) 2014-08-13 17:38:02.612220 7fe9bae887a0 10 osd.4 194478 pgid 22.113 coll 22.113_head 2014-08-13 17:38:02.612479 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.113 2014-08-13 17:38:02.612491 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.612536 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.113(unlocked)] enter Initial 2014-08-13 17:38:02.612551 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.113(unlocked)] enter NotTrimming 2014-08-13 17:38:02.612783 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.612988 7fe9bae887a0 20 read_log 191648'77 (0'0) modify 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 by client.273158.0:4384 2014-07-30 14:09:08.285865 2014-08-13 17:38:02.613018 7fe9bae887a0 20 read_log 191648'103 (0'0) modify 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 by client.273158.0:5574 2014-07-30 14:09:13.732468 2014-08-13 17:38:02.613040 7fe9bae887a0 20 read_log 191648'105 (0'0) modify 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 by client.273158.0:5622 2014-07-30 14:09:13.925369 2014-08-13 17:38:02.613061 7fe9bae887a0 20 read_log 191648'140 (0'0) modify 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 by client.273158.0:7672 2014-07-30 14:09:24.203072 2014-08-13 17:38:02.613082 7fe9bae887a0 20 read_log 191648'168 (0'0) modify 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 by client.273158.0:9439 2014-07-30 14:09:33.051653 2014-08-13 17:38:02.613102 7fe9bae887a0 20 read_log 191648'177 (0'0) modify 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 by client.273158.0:10164 2014-07-30 14:09:36.668970 2014-08-13 17:38:02.613123 7fe9bae887a0 20 read_log 191648'198 (0'0) modify 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 by client.273158.0:11899 2014-07-30 14:09:46.454073 2014-08-13 17:38:02.613144 7fe9bae887a0 20 read_log 191648'330 (0'0) modify ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 by client.273158.0:19187 2014-07-30 14:10:24.334369 2014-08-13 17:38:02.613175 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 by client.275175.0:527 2014-07-30 14:18:35.577293 2014-08-13 17:38:02.613197 7fe9bae887a0 20 read_log 191727'400 (0'0) modify 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 by client.275175.0:9936 2014-07-30 14:19:04.999388 2014-08-13 17:38:02.613219 7fe9bae887a0 20 read_log 191727'404 (0'0) modify bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 by client.275175.0:12690 2014-07-30 14:19:13.575432 2014-08-13 17:38:02.613240 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 by client.275175.0:13683 2014-07-30 14:19:16.758273 2014-08-13 17:38:02.613261 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 by client.275175.0:15903 2014-07-30 14:19:23.542605 2014-08-13 17:38:02.613318 7fe9bae887a0 20 read_log 191727'417 (0'0) modify de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 by client.275175.0:19817 2014-07-30 14:19:35.653150 2014-08-13 17:38:02.613363 7fe9bae887a0 20 read_log 191727'418 (0'0) modify ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 by client.275175.0:21132 2014-07-30 14:19:39.502002 2014-08-13 17:38:02.613386 7fe9bae887a0 20 read_log 191727'422 (0'0) modify b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 by client.275175.0:22569 2014-07-30 14:19:44.534397 2014-08-13 17:38:02.613407 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 by client.275175.0:22734 2014-07-30 14:19:45.236432 2014-08-13 17:38:02.613427 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 by client.275175.0:23832 2014-07-30 14:19:48.357355 2014-08-13 17:38:02.613448 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 by client.275175.0:26391 2014-07-30 14:19:55.973261 2014-08-13 17:38:02.613469 7fe9bae887a0 20 read_log 191727'436 (0'0) modify d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 by client.275175.0:27708 2014-07-30 14:19:59.812197 2014-08-13 17:38:02.613489 7fe9bae887a0 20 read_log 191727'439 (0'0) modify 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 by client.275175.0:28887 2014-07-30 14:20:03.571262 2014-08-13 17:38:02.613510 7fe9bae887a0 20 read_log 191727'443 (0'0) modify c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 by client.275175.0:29730 2014-07-30 14:20:06.131708 2014-08-13 17:38:02.613530 7fe9bae887a0 20 read_log 191727'445 (0'0) modify 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 by client.275175.0:30639 2014-07-30 14:20:08.959585 2014-08-13 17:38:02.613550 7fe9bae887a0 20 read_log 191727'451 (0'0) modify 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 by client.275175.0:34068 2014-07-30 14:20:19.184755 2014-08-13 17:38:02.613571 7fe9bae887a0 20 read_log 191727'453 (0'0) modify 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 by client.275175.0:34534 2014-07-30 14:20:20.642953 2014-08-13 17:38:02.613608 7fe9bae887a0 20 read_log 191727'462 (0'0) modify d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 by client.275175.0:38324 2014-07-30 14:20:32.165867 2014-08-13 17:38:02.613630 7fe9bae887a0 20 read_log 192674'463 (0'0) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6975 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613651 7fe9bae887a0 20 read_log 192674'464 (192674'463) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6976 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613672 7fe9bae887a0 20 read_log 192674'465 (192674'464) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6977 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613693 7fe9bae887a0 20 read_log 192674'466 (192674'465) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6978 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613713 7fe9bae887a0 20 read_log 192674'467 (192674'466) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6979 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613735 7fe9bae887a0 20 read_log 192674'468 (192674'467) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6980 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613755 7fe9bae887a0 20 read_log 192674'469 (192674'468) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6981 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613775 7fe9bae887a0 20 read_log 192674'470 (192674'469) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6982 2014-07-31 18:03:49.048000 2014-08-13 17:38:02.613814 7fe9bae887a0 20 read_log 192888'471 (0'0) modify a29a7113/benchmark_data_storage0111-ib.pocstor_35625_object487/head//22 by client.940330.0:488 2014-08-11 16:29:42.654200 2014-08-13 17:38:02.613837 7fe9bae887a0 20 read_log 192888'472 (0'0) modify 3ee94913/benchmark_data_storage0111-ib.pocstor_35625_object1034/head//22 by client.940330.0:1035 2014-08-11 16:29:44.920692 2014-08-13 17:38:02.613858 7fe9bae887a0 20 read_log 192888'473 (0'0) modify 5ff60913/benchmark_data_storage0111-ib.pocstor_35625_object1433/head//22 by client.940330.0:1434 2014-08-11 16:29:46.441610 2014-08-13 17:38:02.613879 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 991e8113/benchmark_data_storage0111-ib.pocstor_35625_object4649/head//22 by client.940330.0:4650 2014-08-11 16:29:58.796035 2014-08-13 17:38:02.613900 7fe9bae887a0 20 read_log 192888'475 (0'0) modify 59df0113/benchmark_data_storage0111-ib.pocstor_35625_object6133/head//22 by client.940330.0:6134 2014-08-11 16:30:04.056385 2014-08-13 17:38:02.613921 7fe9bae887a0 20 read_log 192888'476 (0'0) modify 1cbc7113/benchmark_data_storage0111-ib.pocstor_35625_object6435/head//22 by client.940330.0:6436 2014-08-11 16:30:05.117149 2014-08-13 17:38:02.613941 7fe9bae887a0 20 read_log 192888'477 (0'0) modify fff7e913/benchmark_data_storage0111-ib.pocstor_35625_object9788/head//22 by client.940330.0:9789 2014-08-11 16:30:17.088891 2014-08-13 17:38:02.613993 7fe9bae887a0 20 read_log 192888'478 (0'0) modify adc70113/benchmark_data_storage0111-ib.pocstor_35625_object10411/head//22 by client.940330.0:10412 2014-08-11 16:30:19.344429 2014-08-13 17:38:02.614016 7fe9bae887a0 20 read_log 192888'479 (0'0) modify e34aa913/benchmark_data_storage0111-ib.pocstor_35625_object14746/head//22 by client.940330.0:14747 2014-08-11 16:30:36.547234 2014-08-13 17:38:02.614037 7fe9bae887a0 20 read_log 192888'480 (0'0) modify c6825113/benchmark_data_storage0111-ib.pocstor_35625_object15937/head//22 by client.940330.0:15938 2014-08-11 16:30:41.001055 2014-08-13 17:38:02.614059 7fe9bae887a0 20 read_log 192888'481 (0'0) modify ffa21113/benchmark_data_storage0111-ib.pocstor_35625_object21650/head//22 by client.940330.0:21651 2014-08-11 16:31:03.227759 2014-08-13 17:38:02.614082 7fe9bae887a0 20 read_log 192888'482 (0'0) modify ed496113/benchmark_data_storage0111-ib.pocstor_35625_object24733/head//22 by client.940330.0:24734 2014-08-11 16:31:14.838225 2014-08-13 17:38:02.614103 7fe9bae887a0 20 read_log 192888'483 (0'0) modify 1015d113/benchmark_data_storage0111-ib.pocstor_35625_object28967/head//22 by client.940330.0:28968 2014-08-11 16:31:31.461202 2014-08-13 17:38:02.614127 7fe9bae887a0 20 read_log 192888'484 (0'0) modify 4e926913/benchmark_data_storage0111-ib.pocstor_35625_object29849/head//22 by client.940330.0:29850 2014-08-11 16:31:34.888119 2014-08-13 17:38:02.614148 7fe9bae887a0 20 read_log 192888'485 (192888'471) delete a29a7113/benchmark_data_storage0111-ib.pocstor_35625_object487/head//22 by client.940330.0:31795 2014-08-11 16:31:41.619035 2014-08-13 17:38:02.614179 7fe9bae887a0 20 read_log 192888'486 (192888'472) delete 3ee94913/benchmark_data_storage0111-ib.pocstor_35625_object1034/head//22 by client.940330.0:32342 2014-08-11 16:31:41.757850 2014-08-13 17:38:02.614201 7fe9bae887a0 20 read_log 192888'487 (192888'473) delete 5ff60913/benchmark_data_storage0111-ib.pocstor_35625_object1433/head//22 by client.940330.0:32741 2014-08-11 16:31:41.851358 2014-08-13 17:38:02.614222 7fe9bae887a0 20 read_log 192888'488 (192888'474) delete 991e8113/benchmark_data_storage0111-ib.pocstor_35625_object4649/head//22 by client.940330.0:35957 2014-08-11 16:31:42.724099 2014-08-13 17:38:02.614243 7fe9bae887a0 20 read_log 192888'489 (192888'475) delete 59df0113/benchmark_data_storage0111-ib.pocstor_35625_object6133/head//22 by client.940330.0:37441 2014-08-11 16:31:43.112713 2014-08-13 17:38:02.614264 7fe9bae887a0 20 read_log 192888'490 (192888'476) delete 1cbc7113/benchmark_data_storage0111-ib.pocstor_35625_object6435/head//22 by client.940330.0:37743 2014-08-11 16:31:43.191394 2014-08-13 17:38:02.614300 7fe9bae887a0 20 read_log 192888'491 (192888'477) delete fff7e913/benchmark_data_storage0111-ib.pocstor_35625_object9788/head//22 by client.940330.0:41096 2014-08-11 16:31:44.055721 2014-08-13 17:38:02.614323 7fe9bae887a0 20 read_log 192888'492 (192888'478) delete adc70113/benchmark_data_storage0111-ib.pocstor_35625_object10411/head//22 by client.940330.0:41719 2014-08-11 16:31:44.227819 2014-08-13 17:38:02.614345 7fe9bae887a0 20 read_log 192888'493 (192888'479) delete e34aa913/benchmark_data_storage0111-ib.pocstor_35625_object14746/head//22 by client.940330.0:46054 2014-08-11 16:31:45.384329 2014-08-13 17:38:02.614366 7fe9bae887a0 20 read_log 192888'494 (192888'480) delete c6825113/benchmark_data_storage0111-ib.pocstor_35625_object15937/head//22 by client.940330.0:47245 2014-08-11 16:31:45.707721 2014-08-13 17:38:02.614387 7fe9bae887a0 20 read_log 192888'495 (192888'481) delete ffa21113/benchmark_data_storage0111-ib.pocstor_35625_object21650/head//22 by client.940330.0:52958 2014-08-11 16:31:47.336090 2014-08-13 17:38:02.614408 7fe9bae887a0 20 read_log 192888'496 (192888'482) delete ed496113/benchmark_data_storage0111-ib.pocstor_35625_object24733/head//22 by client.940330.0:56041 2014-08-11 16:31:48.254288 2014-08-13 17:38:02.614429 7fe9bae887a0 20 read_log 192888'497 (192888'483) delete 1015d113/benchmark_data_storage0111-ib.pocstor_35625_object28967/head//22 by client.940330.0:60275 2014-08-11 16:31:49.462419 2014-08-13 17:38:02.614450 7fe9bae887a0 20 read_log 192888'498 (192888'484) delete 4e926913/benchmark_data_storage0111-ib.pocstor_35625_object29849/head//22 by client.940330.0:61157 2014-08-11 16:31:49.713518 2014-08-13 17:38:02.614471 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 2dace913/benchmark_data_storage0111-ib.pocstor_64498_object547/head//22 by client.1049635.0:548 2014-08-12 17:17:18.885541 2014-08-13 17:38:02.614492 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 7ad1c113/benchmark_data_storage0111-ib.pocstor_64498_object5043/head//22 by client.1049635.0:5044 2014-08-12 17:17:33.048664 2014-08-13 17:38:02.614515 7fe9bae887a0 20 read_log 194215'501 (0'0) modify d650e113/benchmark_data_storage0111-ib.pocstor_64498_object6498/head//22 by client.1049635.0:6499 2014-08-12 17:17:37.568170 2014-08-13 17:38:02.614536 7fe9bae887a0 20 read_log 194215'502 (0'0) modify a88fe913/benchmark_data_storage0111-ib.pocstor_64498_object9300/head//22 by client.1049635.0:9301 2014-08-12 17:17:45.890116 2014-08-13 17:38:02.614557 7fe9bae887a0 20 read_log 194215'503 (0'0) modify 69ba6113/benchmark_data_storage0111-ib.pocstor_64498_object11501/head//22 by client.1049635.0:11502 2014-08-12 17:17:52.889304 2014-08-13 17:38:02.614578 7fe9bae887a0 20 read_log 194215'504 (0'0) modify c46bc113/benchmark_data_storage0111-ib.pocstor_64498_object12235/head//22 by client.1049635.0:12236 2014-08-12 17:17:55.173580 2014-08-13 17:38:02.614599 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 1cf87913/benchmark_data_storage0111-ib.pocstor_64498_object17919/head//22 by client.1049635.0:17920 2014-08-12 17:18:14.857735 2014-08-13 17:38:02.614620 7fe9bae887a0 20 read_log 194215'506 (0'0) modify 94522913/benchmark_data_storage0111-ib.pocstor_64498_object21505/head//22 by client.1049635.0:21506 2014-08-12 17:18:28.472680 2014-08-13 17:38:02.614641 7fe9bae887a0 20 read_log 194215'507 (0'0) modify fb4bc113/benchmark_data_storage0111-ib.pocstor_64498_object21970/head//22 by client.1049635.0:21971 2014-08-12 17:18:30.219047 2014-08-13 17:38:02.614664 7fe9bae887a0 20 read_log 194215'508 (0'0) modify 5e3e8913/benchmark_data_storage0111-ib.pocstor_64498_object23403/head//22 by client.1049635.0:23404 2014-08-12 17:18:35.627203 2014-08-13 17:38:02.614685 7fe9bae887a0 20 read_log 194215'509 (0'0) modify f49eb913/benchmark_data_storage0111-ib.pocstor_64498_object24983/head//22 by client.1049635.0:24984 2014-08-12 17:18:41.652724 2014-08-13 17:38:02.614718 7fe9bae887a0 20 read_log 194215'510 (0'0) modify ecba113/benchmark_data_storage0111-ib.pocstor_64498_object29063/head//22 by client.1049635.0:29064 2014-08-12 17:18:56.832696 2014-08-13 17:38:02.614740 7fe9bae887a0 20 read_log 194215'511 (0'0) modify 5d317113/benchmark_data_storage0111-ib.pocstor_64498_object29569/head//22 by client.1049635.0:29570 2014-08-12 17:18:58.831244 2014-08-13 17:38:02.614761 7fe9bae887a0 20 read_log 194215'512 (0'0) modify c20d6113/benchmark_data_storage0111-ib.pocstor_64498_object31859/head//22 by client.1049635.0:31860 2014-08-12 17:19:08.020374 2014-08-13 17:38:02.614782 7fe9bae887a0 20 read_log 194215'513 (0'0) modify a561d913/benchmark_data_storage0111-ib.pocstor_64498_object32728/head//22 by client.1049635.0:32729 2014-08-12 17:19:11.360882 2014-08-13 17:38:02.614803 7fe9bae887a0 20 read_log 194215'514 (194215'499) delete 2dace913/benchmark_data_storage0111-ib.pocstor_64498_object547/head//22 by client.1049635.0:34750 2014-08-12 17:19:18.171575 2014-08-13 17:38:02.614824 7fe9bae887a0 20 read_log 194215'515 (194215'500) delete 7ad1c113/benchmark_data_storage0111-ib.pocstor_64498_object5043/head//22 by client.1049635.0:39246 2014-08-12 17:19:19.618790 2014-08-13 17:38:02.614846 7fe9bae887a0 20 read_log 194215'516 (194215'501) delete d650e113/benchmark_data_storage0111-ib.pocstor_64498_object6498/head//22 by client.1049635.0:40701 2014-08-12 17:19:20.116449 2014-08-13 17:38:02.614866 7fe9bae887a0 20 read_log 194215'517 (194215'502) delete a88fe913/benchmark_data_storage0111-ib.pocstor_64498_object9300/head//22 by client.1049635.0:43503 2014-08-12 17:19:21.024506 2014-08-13 17:38:02.614888 7fe9bae887a0 20 read_log 194215'518 (194215'503) delete 69ba6113/benchmark_data_storage0111-ib.pocstor_64498_object11501/head//22 by client.1049635.0:45704 2014-08-12 17:19:21.737115 2014-08-13 17:38:02.614909 7fe9bae887a0 20 read_log 194215'519 (194215'504) delete c46bc113/benchmark_data_storage0111-ib.pocstor_64498_object12235/head//22 by client.1049635.0:46438 2014-08-12 17:19:21.969929 2014-08-13 17:38:02.614929 7fe9bae887a0 20 read_log 194215'520 (194215'505) delete 1cf87913/benchmark_data_storage0111-ib.pocstor_64498_object17919/head//22 by client.1049635.0:52122 2014-08-12 17:19:23.800541 2014-08-13 17:38:02.614950 7fe9bae887a0 20 read_log 194215'521 (194215'506) delete 94522913/benchmark_data_storage0111-ib.pocstor_64498_object21505/head//22 by client.1049635.0:55708 2014-08-12 17:19:25.057474 2014-08-13 17:38:02.614971 7fe9bae887a0 20 read_log 194215'522 (194215'507) delete fb4bc113/benchmark_data_storage0111-ib.pocstor_64498_object21970/head//22 by client.1049635.0:56173 2014-08-12 17:19:25.222178 2014-08-13 17:38:02.615022 7fe9bae887a0 20 read_log 194215'523 (194215'508) delete 5e3e8913/benchmark_data_storage0111-ib.pocstor_64498_object23403/head//22 by client.1049635.0:57606 2014-08-12 17:19:25.763617 2014-08-13 17:38:02.615044 7fe9bae887a0 20 read_log 194215'524 (194215'509) delete f49eb913/benchmark_data_storage0111-ib.pocstor_64498_object24983/head//22 by client.1049635.0:59186 2014-08-12 17:19:26.317695 2014-08-13 17:38:02.615065 7fe9bae887a0 20 read_log 194215'525 (194215'510) delete ecba113/benchmark_data_storage0111-ib.pocstor_64498_object29063/head//22 by client.1049635.0:63266 2014-08-12 17:19:27.727565 2014-08-13 17:38:02.615086 7fe9bae887a0 20 read_log 194215'526 (194215'511) delete 5d317113/benchmark_data_storage0111-ib.pocstor_64498_object29569/head//22 by client.1049635.0:63772 2014-08-12 17:19:27.907729 2014-08-13 17:38:02.615107 7fe9bae887a0 20 read_log 194215'527 (194215'512) delete c20d6113/benchmark_data_storage0111-ib.pocstor_64498_object31859/head//22 by client.1049635.0:66062 2014-08-12 17:19:28.692136 2014-08-13 17:38:02.615128 7fe9bae887a0 20 read_log 194215'528 (194215'513) delete a561d913/benchmark_data_storage0111-ib.pocstor_64498_object32728/head//22 by client.1049635.0:66931 2014-08-12 17:19:29.014971 2014-08-13 17:38:02.615298 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.615328 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=0 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.615351 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=0 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002816 0 0.000000 2014-08-13 17:38:02.615371 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=0 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.615386 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=0 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.615401 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'528], crt=191648'383) 2014-08-13 17:38:02.615418 7fe9bae887a0 10 osd.4 194478 pgid 22.116 coll 22.116_head 2014-08-13 17:38:02.615611 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.116 2014-08-13 17:38:02.615622 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.615666 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.116(unlocked)] enter Initial 2014-08-13 17:38:02.615682 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.116(unlocked)] enter NotTrimming 2014-08-13 17:38:02.615893 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.616133 7fe9bae887a0 20 read_log 191648'85 (0'0) modify 8fdc916/benchmark_data_storage0111-ib.pocstor_40745_object5633/head//22 by client.273158.0:5634 2014-07-30 14:09:13.974314 2014-08-13 17:38:02.616175 7fe9bae887a0 20 read_log 191648'99 (0'0) modify f1132916/benchmark_data_storage0111-ib.pocstor_40745_object6261/head//22 by client.273158.0:6262 2014-07-30 14:09:17.289800 2014-08-13 17:38:02.616198 7fe9bae887a0 20 read_log 191648'118 (0'0) modify 7efac116/benchmark_data_storage0111-ib.pocstor_40745_object6993/head//22 by client.273158.0:6994 2014-07-30 14:09:21.065974 2014-08-13 17:38:02.616220 7fe9bae887a0 20 read_log 191648'122 (0'0) modify 5e6c0916/benchmark_data_storage0111-ib.pocstor_40745_object7304/head//22 by client.273158.0:7305 2014-07-30 14:09:22.292581 2014-08-13 17:38:02.616242 7fe9bae887a0 20 read_log 191648'132 (0'0) modify f6c20116/benchmark_data_storage0111-ib.pocstor_40745_object8285/head//22 by client.273158.0:8286 2014-07-30 14:09:27.764161 2014-08-13 17:38:02.616263 7fe9bae887a0 20 read_log 191648'182 (0'0) modify c621f116/benchmark_data_storage0111-ib.pocstor_40745_object11977/head//22 by client.273158.0:11978 2014-07-30 14:09:46.927074 2014-08-13 17:38:02.616284 7fe9bae887a0 20 read_log 191648'189 (0'0) modify d25fb116/benchmark_data_storage0111-ib.pocstor_40745_object12496/head//22 by client.273158.0:12497 2014-07-30 14:09:49.991885 2014-08-13 17:38:02.616304 7fe9bae887a0 20 read_log 191648'206 (0'0) modify 6fd2e916/benchmark_data_storage0111-ib.pocstor_40745_object13441/head//22 by client.273158.0:13442 2014-07-30 14:09:54.468489 2014-08-13 17:38:02.616325 7fe9bae887a0 20 read_log 191648'214 (0'0) modify d7cd7916/benchmark_data_storage0111-ib.pocstor_40745_object13744/head//22 by client.273158.0:13745 2014-07-30 14:09:55.808656 2014-08-13 17:38:02.616362 7fe9bae887a0 20 read_log 191648'247 (0'0) modify 9a579916/benchmark_data_storage0111-ib.pocstor_40745_object15685/head//22 by client.273158.0:15686 2014-07-30 14:10:05.622621 2014-08-13 17:38:02.616385 7fe9bae887a0 20 read_log 191648'256 (0'0) modify 1d60a116/benchmark_data_storage0111-ib.pocstor_40745_object16275/head//22 by client.273158.0:16276 2014-07-30 14:10:08.961420 2014-08-13 17:38:02.616438 7fe9bae887a0 20 read_log 191648'337 (0'0) modify be72a116/benchmark_data_storage0111-ib.pocstor_40745_object21012/head//22 by client.273158.0:21013 2014-07-30 14:10:37.305279 2014-08-13 17:38:02.616461 7fe9bae887a0 20 read_log 191727'366 (0'0) modify e5e51116/benchmark_data_storage0111-ib.pocstor_51285_object2257/head//22 by client.275175.0:2258 2014-07-30 14:18:41.735075 2014-08-13 17:38:02.616482 7fe9bae887a0 20 read_log 191727'369 (0'0) modify ce6e4916/benchmark_data_storage0111-ib.pocstor_51285_object3050/head//22 by client.275175.0:3051 2014-07-30 14:18:44.384996 2014-08-13 17:38:02.616503 7fe9bae887a0 20 read_log 191727'374 (0'0) modify 1cb1b116/benchmark_data_storage0111-ib.pocstor_51285_object4253/head//22 by client.275175.0:4254 2014-07-30 14:18:48.126636 2014-08-13 17:38:02.616524 7fe9bae887a0 20 read_log 191727'383 (0'0) modify 795f7916/benchmark_data_storage0111-ib.pocstor_51285_object9359/head//22 by client.275175.0:9360 2014-07-30 14:19:03.389468 2014-08-13 17:38:02.616544 7fe9bae887a0 20 read_log 191727'385 (0'0) modify ec53c916/benchmark_data_storage0111-ib.pocstor_51285_object10011/head//22 by client.275175.0:10012 2014-07-30 14:19:05.214295 2014-08-13 17:38:02.616565 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 5b347116/benchmark_data_storage0111-ib.pocstor_51285_object15330/head//22 by client.275175.0:15331 2014-07-30 14:19:21.797340 2014-08-13 17:38:02.616586 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 800d4116/benchmark_data_storage0111-ib.pocstor_51285_object17063/head//22 by client.275175.0:17064 2014-07-30 14:19:27.338250 2014-08-13 17:38:02.616606 7fe9bae887a0 20 read_log 191727'396 (0'0) modify ec971116/benchmark_data_storage0111-ib.pocstor_51285_object17381/head//22 by client.275175.0:17382 2014-07-30 14:19:28.217820 2014-08-13 17:38:02.616631 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 587dc916/benchmark_data_storage0111-ib.pocstor_51285_object18232/head//22 by client.275175.0:18233 2014-07-30 14:19:30.669624 2014-08-13 17:38:02.616652 7fe9bae887a0 20 read_log 191727'408 (0'0) modify de7c3116/benchmark_data_storage0111-ib.pocstor_51285_object23390/head//22 by client.275175.0:23391 2014-07-30 14:19:47.128678 2014-08-13 17:38:02.616673 7fe9bae887a0 20 read_log 191727'417 (0'0) modify c0e70916/benchmark_data_storage0111-ib.pocstor_51285_object26712/head//22 by client.275175.0:26713 2014-07-30 14:19:56.952443 2014-08-13 17:38:02.616693 7fe9bae887a0 20 read_log 191727'419 (0'0) modify 1195b916/benchmark_data_storage0111-ib.pocstor_51285_object28254/head//22 by client.275175.0:28255 2014-07-30 14:20:01.739328 2014-08-13 17:38:02.616714 7fe9bae887a0 20 read_log 191727'428 (0'0) modify 49106916/benchmark_data_storage0111-ib.pocstor_51285_object33237/head//22 by client.275175.0:33238 2014-07-30 14:20:16.558157 2014-08-13 17:38:02.616735 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 3ba6d116/benchmark_data_storage0111-ib.pocstor_51285_object33603/head//22 by client.275175.0:33604 2014-07-30 14:20:17.861400 2014-08-13 17:38:02.616755 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 77a43116/benchmark_data_storage0111-ib.pocstor_51285_object34152/head//22 by client.275175.0:34153 2014-07-30 14:20:19.495304 2014-08-13 17:38:02.616776 7fe9bae887a0 20 read_log 191727'433 (0'0) modify 6ca4a116/benchmark_data_storage0111-ib.pocstor_51285_object34574/head//22 by client.275175.0:34575 2014-07-30 14:20:20.759030 2014-08-13 17:38:02.616808 7fe9bae887a0 20 read_log 191727'438 (0'0) modify 6db26116/benchmark_data_storage0111-ib.pocstor_51285_object38680/head//22 by client.275175.0:38681 2014-07-30 14:20:33.184814 2014-08-13 17:38:02.616830 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 624a4116/benchmark_data_storage0111-ib.pocstor_35625_object1207/head//22 by client.940330.0:1208 2014-08-11 16:29:45.563713 2014-08-13 17:38:02.616851 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 3dc95916/benchmark_data_storage0111-ib.pocstor_35625_object2937/head//22 by client.940330.0:2938 2014-08-11 16:29:52.350322 2014-08-13 17:38:02.616871 7fe9bae887a0 20 read_log 192888'441 (0'0) modify b75d8916/benchmark_data_storage0111-ib.pocstor_35625_object3841/head//22 by client.940330.0:3842 2014-08-11 16:29:55.750183 2014-08-13 17:38:02.616892 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 977ff916/benchmark_data_storage0111-ib.pocstor_35625_object5894/head//22 by client.940330.0:5895 2014-08-11 16:30:03.188692 2014-08-13 17:38:02.616913 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 70db5916/benchmark_data_storage0111-ib.pocstor_35625_object7685/head//22 by client.940330.0:7686 2014-08-11 16:30:09.726737 2014-08-13 17:38:02.616933 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 14e69116/benchmark_data_storage0111-ib.pocstor_35625_object16925/head//22 by client.940330.0:16926 2014-08-11 16:30:44.686558 2014-08-13 17:38:02.616953 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 17fe5916/benchmark_data_storage0111-ib.pocstor_35625_object20454/head//22 by client.940330.0:20455 2014-08-11 16:30:58.526998 2014-08-13 17:38:02.616976 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 40fbb116/benchmark_data_storage0111-ib.pocstor_35625_object23666/head//22 by client.940330.0:23667 2014-08-11 16:31:10.909061 2014-08-13 17:38:02.616997 7fe9bae887a0 20 read_log 192888'447 (0'0) modify a6c2c116/benchmark_data_storage0111-ib.pocstor_35625_object23743/head//22 by client.940330.0:23744 2014-08-11 16:31:11.207063 2014-08-13 17:38:02.617017 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 52f62916/benchmark_data_storage0111-ib.pocstor_35625_object24451/head//22 by client.940330.0:24452 2014-08-11 16:31:13.788043 2014-08-13 17:38:02.617040 7fe9bae887a0 20 read_log 192888'449 (0'0) modify a3cc5916/benchmark_data_storage0111-ib.pocstor_35625_object24649/head//22 by client.940330.0:24650 2014-08-11 16:31:14.524787 2014-08-13 17:38:02.617061 7fe9bae887a0 20 read_log 192888'450 (0'0) modify ad34116/benchmark_data_storage0111-ib.pocstor_35625_object26568/head//22 by client.940330.0:26569 2014-08-11 16:31:22.061749 2014-08-13 17:38:02.617082 7fe9bae887a0 20 read_log 192888'451 (0'0) modify a15ab916/benchmark_data_storage0111-ib.pocstor_35625_object31265/head//22 by client.940330.0:31266 2014-08-11 16:31:40.273133 2014-08-13 17:38:02.617103 7fe9bae887a0 20 read_log 192888'452 (192888'439) delete 624a4116/benchmark_data_storage0111-ib.pocstor_35625_object1207/head//22 by client.940330.0:32515 2014-08-11 16:31:41.798468 2014-08-13 17:38:02.617124 7fe9bae887a0 20 read_log 192888'453 (192888'440) delete 3dc95916/benchmark_data_storage0111-ib.pocstor_35625_object2937/head//22 by client.940330.0:34245 2014-08-11 16:31:42.235722 2014-08-13 17:38:02.617145 7fe9bae887a0 20 read_log 192888'454 (192888'441) delete b75d8916/benchmark_data_storage0111-ib.pocstor_35625_object3841/head//22 by client.940330.0:35149 2014-08-11 16:31:42.486955 2014-08-13 17:38:02.617174 7fe9bae887a0 20 read_log 192888'455 (192888'442) delete 977ff916/benchmark_data_storage0111-ib.pocstor_35625_object5894/head//22 by client.940330.0:37202 2014-08-11 16:31:43.047469 2014-08-13 17:38:02.617196 7fe9bae887a0 20 read_log 192888'456 (192888'443) delete 70db5916/benchmark_data_storage0111-ib.pocstor_35625_object7685/head//22 by client.940330.0:38993 2014-08-11 16:31:43.510337 2014-08-13 17:38:02.617230 7fe9bae887a0 20 read_log 192888'457 (192888'444) delete 14e69116/benchmark_data_storage0111-ib.pocstor_35625_object16925/head//22 by client.940330.0:48233 2014-08-11 16:31:45.969859 2014-08-13 17:38:02.617252 7fe9bae887a0 20 read_log 192888'458 (192888'445) delete 17fe5916/benchmark_data_storage0111-ib.pocstor_35625_object20454/head//22 by client.940330.0:51762 2014-08-11 16:31:47.009868 2014-08-13 17:38:02.617274 7fe9bae887a0 20 read_log 192888'459 (192888'446) delete 40fbb116/benchmark_data_storage0111-ib.pocstor_35625_object23666/head//22 by client.940330.0:54974 2014-08-11 16:31:47.941262 2014-08-13 17:38:02.617295 7fe9bae887a0 20 read_log 192888'460 (192888'447) delete a6c2c116/benchmark_data_storage0111-ib.pocstor_35625_object23743/head//22 by client.940330.0:55051 2014-08-11 16:31:47.962117 2014-08-13 17:38:02.617315 7fe9bae887a0 20 read_log 192888'461 (192888'448) delete 52f62916/benchmark_data_storage0111-ib.pocstor_35625_object24451/head//22 by client.940330.0:55759 2014-08-11 16:31:48.171608 2014-08-13 17:38:02.617339 7fe9bae887a0 20 read_log 192888'462 (192888'449) delete a3cc5916/benchmark_data_storage0111-ib.pocstor_35625_object24649/head//22 by client.940330.0:55957 2014-08-11 16:31:48.230535 2014-08-13 17:38:02.617360 7fe9bae887a0 20 read_log 192888'463 (192888'450) delete ad34116/benchmark_data_storage0111-ib.pocstor_35625_object26568/head//22 by client.940330.0:57876 2014-08-11 16:31:48.810270 2014-08-13 17:38:02.617381 7fe9bae887a0 20 read_log 192888'464 (192888'451) delete a15ab916/benchmark_data_storage0111-ib.pocstor_35625_object31265/head//22 by client.940330.0:62573 2014-08-11 16:31:50.142084 2014-08-13 17:38:02.617405 7fe9bae887a0 20 read_log 194215'465 (0'0) modify a4087116/benchmark_data_storage0111-ib.pocstor_64498_object1090/head//22 by client.1049635.0:1091 2014-08-12 17:17:20.734071 2014-08-13 17:38:02.617426 7fe9bae887a0 20 read_log 194215'466 (0'0) modify fac8a116/benchmark_data_storage0111-ib.pocstor_64498_object5156/head//22 by client.1049635.0:5157 2014-08-12 17:17:33.394579 2014-08-13 17:38:02.617447 7fe9bae887a0 20 read_log 194215'467 (0'0) modify a140b916/benchmark_data_storage0111-ib.pocstor_64498_object5365/head//22 by client.1049635.0:5366 2014-08-12 17:17:34.040797 2014-08-13 17:38:02.617468 7fe9bae887a0 20 read_log 194215'468 (0'0) modify f673e916/benchmark_data_storage0111-ib.pocstor_64498_object8586/head//22 by client.1049635.0:8587 2014-08-12 17:17:43.841742 2014-08-13 17:38:02.617488 7fe9bae887a0 20 read_log 194215'469 (0'0) modify ac992916/benchmark_data_storage0111-ib.pocstor_64498_object8839/head//22 by client.1049635.0:8840 2014-08-12 17:17:44.565023 2014-08-13 17:38:02.617509 7fe9bae887a0 20 read_log 194215'470 (0'0) modify dc13a116/benchmark_data_storage0111-ib.pocstor_64498_object12504/head//22 by client.1049635.0:12505 2014-08-12 17:17:55.906356 2014-08-13 17:38:02.617530 7fe9bae887a0 20 read_log 194215'471 (0'0) modify e3fe2916/benchmark_data_storage0111-ib.pocstor_64498_object17820/head//22 by client.1049635.0:17821 2014-08-12 17:18:14.480255 2014-08-13 17:38:02.617551 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 4e9f4916/benchmark_data_storage0111-ib.pocstor_64498_object18225/head//22 by client.1049635.0:18226 2014-08-12 17:18:16.011056 2014-08-13 17:38:02.617572 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 71470116/benchmark_data_storage0111-ib.pocstor_64498_object24781/head//22 by client.1049635.0:24782 2014-08-12 17:18:40.761724 2014-08-13 17:38:02.617593 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 19214916/benchmark_data_storage0111-ib.pocstor_64498_object26570/head//22 by client.1049635.0:26571 2014-08-12 17:18:47.545961 2014-08-13 17:38:02.617613 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 36749916/benchmark_data_storage0111-ib.pocstor_64498_object26633/head//22 by client.1049635.0:26634 2014-08-12 17:18:47.784194 2014-08-13 17:38:02.617650 7fe9bae887a0 20 read_log 194215'476 (0'0) modify b477a116/benchmark_data_storage0111-ib.pocstor_64498_object28846/head//22 by client.1049635.0:28847 2014-08-12 17:18:56.030367 2014-08-13 17:38:02.617673 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 9fb36916/benchmark_data_storage0111-ib.pocstor_64498_object29857/head//22 by client.1049635.0:29858 2014-08-12 17:18:59.912795 2014-08-13 17:38:02.617695 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 6c957116/benchmark_data_storage0111-ib.pocstor_64498_object30231/head//22 by client.1049635.0:30232 2014-08-12 17:19:01.472489 2014-08-13 17:38:02.617716 7fe9bae887a0 20 read_log 194215'479 (0'0) modify f48be116/benchmark_data_storage0111-ib.pocstor_64498_object32849/head//22 by client.1049635.0:32850 2014-08-12 17:19:11.819207 2014-08-13 17:38:02.617737 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 9153f116/benchmark_data_storage0111-ib.pocstor_64498_object34000/head//22 by client.1049635.0:34001 2014-08-12 17:19:16.183194 2014-08-13 17:38:02.617758 7fe9bae887a0 20 read_log 194215'481 (194215'465) delete a4087116/benchmark_data_storage0111-ib.pocstor_64498_object1090/head//22 by client.1049635.0:35293 2014-08-12 17:19:18.348349 2014-08-13 17:38:02.617779 7fe9bae887a0 20 read_log 194215'482 (194215'466) delete fac8a116/benchmark_data_storage0111-ib.pocstor_64498_object5156/head//22 by client.1049635.0:39359 2014-08-12 17:19:19.658271 2014-08-13 17:38:02.617800 7fe9bae887a0 20 read_log 194215'483 (194215'467) delete a140b916/benchmark_data_storage0111-ib.pocstor_64498_object5365/head//22 by client.1049635.0:39568 2014-08-12 17:19:19.727550 2014-08-13 17:38:02.617821 7fe9bae887a0 20 read_log 194215'484 (194215'468) delete f673e916/benchmark_data_storage0111-ib.pocstor_64498_object8586/head//22 by client.1049635.0:42789 2014-08-12 17:19:20.791626 2014-08-13 17:38:02.617842 7fe9bae887a0 20 read_log 194215'485 (194215'469) delete ac992916/benchmark_data_storage0111-ib.pocstor_64498_object8839/head//22 by client.1049635.0:43042 2014-08-12 17:19:20.873686 2014-08-13 17:38:02.617863 7fe9bae887a0 20 read_log 194215'486 (194215'470) delete dc13a116/benchmark_data_storage0111-ib.pocstor_64498_object12504/head//22 by client.1049635.0:46707 2014-08-12 17:19:22.054246 2014-08-13 17:38:02.617884 7fe9bae887a0 20 read_log 194215'487 (194215'471) delete e3fe2916/benchmark_data_storage0111-ib.pocstor_64498_object17820/head//22 by client.1049635.0:52023 2014-08-12 17:19:23.766792 2014-08-13 17:38:02.617904 7fe9bae887a0 20 read_log 194215'488 (194215'472) delete 4e9f4916/benchmark_data_storage0111-ib.pocstor_64498_object18225/head//22 by client.1049635.0:52428 2014-08-12 17:19:23.904939 2014-08-13 17:38:02.617925 7fe9bae887a0 20 read_log 194215'489 (194215'473) delete 71470116/benchmark_data_storage0111-ib.pocstor_64498_object24781/head//22 by client.1049635.0:58984 2014-08-12 17:19:26.246505 2014-08-13 17:38:02.617946 7fe9bae887a0 20 read_log 194215'490 (194215'474) delete 19214916/benchmark_data_storage0111-ib.pocstor_64498_object26570/head//22 by client.1049635.0:60773 2014-08-12 17:19:26.846081 2014-08-13 17:38:02.617967 7fe9bae887a0 20 read_log 194215'491 (194215'475) delete 36749916/benchmark_data_storage0111-ib.pocstor_64498_object26633/head//22 by client.1049635.0:60836 2014-08-12 17:19:26.868477 2014-08-13 17:38:02.617991 7fe9bae887a0 20 read_log 194215'492 (194215'476) delete b477a116/benchmark_data_storage0111-ib.pocstor_64498_object28846/head//22 by client.1049635.0:63049 2014-08-12 17:19:27.643104 2014-08-13 17:38:02.618012 7fe9bae887a0 20 read_log 194215'493 (194215'477) delete 9fb36916/benchmark_data_storage0111-ib.pocstor_64498_object29857/head//22 by client.1049635.0:64060 2014-08-12 17:19:28.002724 2014-08-13 17:38:02.618033 7fe9bae887a0 20 read_log 194215'494 (194215'478) delete 6c957116/benchmark_data_storage0111-ib.pocstor_64498_object30231/head//22 by client.1049635.0:64434 2014-08-12 17:19:28.118949 2014-08-13 17:38:02.618054 7fe9bae887a0 20 read_log 194215'495 (194215'479) delete f48be116/benchmark_data_storage0111-ib.pocstor_64498_object32849/head//22 by client.1049635.0:67052 2014-08-12 17:19:29.058729 2014-08-13 17:38:02.618089 7fe9bae887a0 20 read_log 194215'496 (194215'480) delete 9153f116/benchmark_data_storage0111-ib.pocstor_64498_object34000/head//22 by client.1049635.0:68203 2014-08-12 17:19:29.482676 2014-08-13 17:38:02.618232 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.618259 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=0 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.618282 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=0 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Initial 0.002616 0 0.000000 2014-08-13 17:38:02.618302 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=0 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.618316 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=0 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.618330 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] log((0'0,194215'496], crt=192888'464) 2014-08-13 17:38:02.618347 7fe9bae887a0 10 osd.4 194478 pgid 22.12a coll 22.12a_head 2014-08-13 17:38:02.618549 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.12a 2014-08-13 17:38:02.618559 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.618622 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.12a(unlocked)] enter Initial 2014-08-13 17:38:02.618638 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.12a(unlocked)] enter NotTrimming 2014-08-13 17:38:02.618838 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.619033 7fe9bae887a0 20 read_log 191648'70 (0'0) modify 87c1e92a/benchmark_data_storage0111-ib.pocstor_40745_object4743/head//22 by client.273158.0:4744 2014-07-30 14:09:09.824350 2014-08-13 17:38:02.619069 7fe9bae887a0 20 read_log 191648'106 (0'0) modify 647592a/benchmark_data_storage0111-ib.pocstor_40745_object7036/head//22 by client.273158.0:7037 2014-07-30 14:09:21.235503 2014-08-13 17:38:02.619091 7fe9bae887a0 20 read_log 191648'142 (0'0) modify 2a19992a/benchmark_data_storage0111-ib.pocstor_40745_object9701/head//22 by client.273158.0:9702 2014-07-30 14:09:34.401634 2014-08-13 17:38:02.619115 7fe9bae887a0 20 read_log 191648'148 (0'0) modify 6969c92a/benchmark_data_storage0111-ib.pocstor_40745_object9826/head//22 by client.273158.0:9827 2014-07-30 14:09:34.901010 2014-08-13 17:38:02.619137 7fe9bae887a0 20 read_log 191648'220 (0'0) modify 581c92a/benchmark_data_storage0111-ib.pocstor_40745_object15560/head//22 by client.273158.0:15561 2014-07-30 14:10:05.073565 2014-08-13 17:38:02.619172 7fe9bae887a0 20 read_log 191648'251 (0'0) modify af6ef92a/benchmark_data_storage0111-ib.pocstor_40745_object17068/head//22 by client.273158.0:17069 2014-07-30 14:10:13.401297 2014-08-13 17:38:02.619194 7fe9bae887a0 20 read_log 191648'256 (0'0) modify b6e8092a/benchmark_data_storage0111-ib.pocstor_40745_object17182/head//22 by client.273158.0:17183 2014-07-30 14:10:13.997832 2014-08-13 17:38:02.619216 7fe9bae887a0 20 read_log 191648'266 (0'0) modify 8a7d912a/benchmark_data_storage0111-ib.pocstor_40745_object17584/head//22 by client.273158.0:17585 2014-07-30 14:10:16.023032 2014-08-13 17:38:02.619237 7fe9bae887a0 20 read_log 191648'324 (0'0) modify 7be2592a/benchmark_data_storage0111-ib.pocstor_40745_object22107/head//22 by client.273158.0:22108 2014-07-30 14:10:44.617145 2014-08-13 17:38:02.619258 7fe9bae887a0 20 read_log 191727'336 (0'0) modify d77d992a/benchmark_data_storage0111-ib.pocstor_51285_object251/head//22 by client.275175.0:252 2014-07-30 14:18:34.407369 2014-08-13 17:38:02.619278 7fe9bae887a0 20 read_log 191727'337 (0'0) modify f612012a/benchmark_data_storage0111-ib.pocstor_51285_object374/head//22 by client.275175.0:375 2014-07-30 14:18:34.938935 2014-08-13 17:38:02.619299 7fe9bae887a0 20 read_log 191727'339 (0'0) modify 7257392a/benchmark_data_storage0111-ib.pocstor_51285_object1244/head//22 by client.275175.0:1245 2014-07-30 14:18:38.332118 2014-08-13 17:38:02.619320 7fe9bae887a0 20 read_log 191727'340 (0'0) modify 2cd3812a/benchmark_data_storage0111-ib.pocstor_51285_object1631/head//22 by client.275175.0:1632 2014-07-30 14:18:39.666150 2014-08-13 17:38:02.619341 7fe9bae887a0 20 read_log 191727'341 (0'0) modify fc9ba92a/benchmark_data_storage0111-ib.pocstor_51285_object1852/head//22 by client.275175.0:1853 2014-07-30 14:18:40.417035 2014-08-13 17:38:02.619392 7fe9bae887a0 20 read_log 191727'344 (0'0) modify b088992a/benchmark_data_storage0111-ib.pocstor_51285_object3975/head//22 by client.275175.0:3976 2014-07-30 14:18:47.280165 2014-08-13 17:38:02.619414 7fe9bae887a0 20 read_log 191727'349 (0'0) modify 8733812a/benchmark_data_storage0111-ib.pocstor_51285_object7812/head//22 by client.275175.0:7813 2014-07-30 14:18:59.001810 2014-08-13 17:38:02.619436 7fe9bae887a0 20 read_log 191727'350 (0'0) modify aaf8612a/benchmark_data_storage0111-ib.pocstor_51285_object8521/head//22 by client.275175.0:8522 2014-07-30 14:19:00.987795 2014-08-13 17:38:02.619457 7fe9bae887a0 20 read_log 191727'357 (0'0) modify 7568a92a/benchmark_data_storage0111-ib.pocstor_51285_object10652/head//22 by client.275175.0:10653 2014-07-30 14:19:07.072632 2014-08-13 17:38:02.619479 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 825e692a/benchmark_data_storage0111-ib.pocstor_51285_object12969/head//22 by client.275175.0:12970 2014-07-30 14:19:14.531347 2014-08-13 17:38:02.619513 7fe9bae887a0 20 read_log 191727'362 (0'0) modify 297912a/benchmark_data_storage0111-ib.pocstor_51285_object15500/head//22 by client.275175.0:15501 2014-07-30 14:19:22.416357 2014-08-13 17:38:02.619535 7fe9bae887a0 20 read_log 191727'363 (0'0) modify b98d192a/benchmark_data_storage0111-ib.pocstor_51285_object16368/head//22 by client.275175.0:16369 2014-07-30 14:19:25.261820 2014-08-13 17:38:02.619555 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 4eaad12a/benchmark_data_storage0111-ib.pocstor_51285_object18134/head//22 by client.275175.0:18135 2014-07-30 14:19:30.333846 2014-08-13 17:38:02.619576 7fe9bae887a0 20 read_log 191727'369 (0'0) modify 885ff12a/benchmark_data_storage0111-ib.pocstor_51285_object18417/head//22 by client.275175.0:18418 2014-07-30 14:19:31.248060 2014-08-13 17:38:02.619597 7fe9bae887a0 20 read_log 191727'370 (0'0) modify bbe0292a/benchmark_data_storage0111-ib.pocstor_51285_object18753/head//22 by client.275175.0:18754 2014-07-30 14:19:32.290424 2014-08-13 17:38:02.619618 7fe9bae887a0 20 read_log 191727'374 (0'0) modify e3e3792a/benchmark_data_storage0111-ib.pocstor_51285_object19780/head//22 by client.275175.0:19781 2014-07-30 14:19:35.553511 2014-08-13 17:38:02.619638 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 1f27792a/benchmark_data_storage0111-ib.pocstor_51285_object19874/head//22 by client.275175.0:19875 2014-07-30 14:19:35.817071 2014-08-13 17:38:02.619659 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 3d0f312a/benchmark_data_storage0111-ib.pocstor_51285_object21500/head//22 by client.275175.0:21501 2014-07-30 14:19:40.666866 2014-08-13 17:38:02.619679 7fe9bae887a0 20 read_log 191727'385 (0'0) modify fe3c392a/benchmark_data_storage0111-ib.pocstor_51285_object25479/head//22 by client.275175.0:25480 2014-07-30 14:19:53.203645 2014-08-13 17:38:02.619700 7fe9bae887a0 20 read_log 191727'387 (0'0) modify c78b192a/benchmark_data_storage0111-ib.pocstor_51285_object25908/head//22 by client.275175.0:25909 2014-07-30 14:19:54.468897 2014-08-13 17:38:02.619721 7fe9bae887a0 20 read_log 191727'388 (0'0) modify d6fff92a/benchmark_data_storage0111-ib.pocstor_51285_object26164/head//22 by client.275175.0:26165 2014-07-30 14:19:55.335846 2014-08-13 17:38:02.619741 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 4e28012a/benchmark_data_storage0111-ib.pocstor_51285_object27916/head//22 by client.275175.0:27917 2014-07-30 14:20:00.768446 2014-08-13 17:38:02.619762 7fe9bae887a0 20 read_log 191727'393 (0'0) modify 881da92a/benchmark_data_storage0111-ib.pocstor_51285_object28150/head//22 by client.275175.0:28151 2014-07-30 14:20:01.450873 2014-08-13 17:38:02.619782 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 3899192a/benchmark_data_storage0111-ib.pocstor_51285_object30322/head//22 by client.275175.0:30323 2014-07-30 14:20:08.078527 2014-08-13 17:38:02.619803 7fe9bae887a0 20 read_log 191727'398 (0'0) modify c732312a/benchmark_data_storage0111-ib.pocstor_51285_object30774/head//22 by client.275175.0:30775 2014-07-30 14:20:09.341375 2014-08-13 17:38:02.619824 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 996a012a/benchmark_data_storage0111-ib.pocstor_51285_object31879/head//22 by client.275175.0:31880 2014-07-30 14:20:12.510839 2014-08-13 17:38:02.619844 7fe9bae887a0 20 read_log 191727'404 (0'0) modify 431b992a/benchmark_data_storage0111-ib.pocstor_51285_object32036/head//22 by client.275175.0:32037 2014-07-30 14:20:12.953760 2014-08-13 17:38:02.619865 7fe9bae887a0 20 read_log 191727'407 (0'0) modify f51f812a/benchmark_data_storage0111-ib.pocstor_51285_object32537/head//22 by client.275175.0:32538 2014-07-30 14:20:14.406656 2014-08-13 17:38:02.619888 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 898bc12a/benchmark_data_storage0111-ib.pocstor_51285_object35016/head//22 by client.275175.0:35017 2014-07-30 14:20:22.031759 2014-08-13 17:38:02.619909 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 3952d92a/benchmark_data_storage0111-ib.pocstor_51285_object37282/head//22 by client.275175.0:37283 2014-07-30 14:20:29.156295 2014-08-13 17:38:02.619944 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 4949f12a/benchmark_data_storage0111-ib.pocstor_51285_object38749/head//22 by client.275175.0:38750 2014-07-30 14:20:33.377345 2014-08-13 17:38:02.619999 7fe9bae887a0 20 read_log 192674'422 (0'0) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2463 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620021 7fe9bae887a0 20 read_log 192674'423 (192674'422) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2464 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620042 7fe9bae887a0 20 read_log 192674'424 (192674'423) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2465 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620062 7fe9bae887a0 20 read_log 192674'425 (192674'424) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2466 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620083 7fe9bae887a0 20 read_log 192674'426 (192674'425) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2467 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620103 7fe9bae887a0 20 read_log 192674'427 (192674'426) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2468 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620124 7fe9bae887a0 20 read_log 192674'428 (192674'427) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2469 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620144 7fe9bae887a0 20 read_log 192674'429 (192674'428) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2470 2014-07-31 18:03:44.302000 2014-08-13 17:38:02.620174 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 2ae2f12a/benchmark_data_storage0111-ib.pocstor_35625_object255/head//22 by client.940330.0:256 2014-08-11 16:29:41.740666 2014-08-13 17:38:02.620197 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 680d612a/benchmark_data_storage0111-ib.pocstor_35625_object2015/head//22 by client.940330.0:2016 2014-08-11 16:29:48.613779 2014-08-13 17:38:02.620218 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 82b8892a/benchmark_data_storage0111-ib.pocstor_35625_object7489/head//22 by client.940330.0:7490 2014-08-11 16:30:09.039951 2014-08-13 17:38:02.620239 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 9ff3d92a/benchmark_data_storage0111-ib.pocstor_35625_object7813/head//22 by client.940330.0:7814 2014-08-11 16:30:10.179044 2014-08-13 17:38:02.620260 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 13fa912a/benchmark_data_storage0111-ib.pocstor_35625_object8599/head//22 by client.940330.0:8600 2014-08-11 16:30:12.931333 2014-08-13 17:38:02.620281 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 4120392a/benchmark_data_storage0111-ib.pocstor_35625_object13555/head//22 by client.940330.0:13556 2014-08-11 16:30:31.988268 2014-08-13 17:38:02.620302 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 3a90012a/benchmark_data_storage0111-ib.pocstor_35625_object15524/head//22 by client.940330.0:15525 2014-08-11 16:30:39.449368 2014-08-13 17:38:02.620323 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 575c912a/benchmark_data_storage0111-ib.pocstor_35625_object21817/head//22 by client.940330.0:21818 2014-08-11 16:31:03.889463 2014-08-13 17:38:02.620344 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 3bd3a92a/benchmark_data_storage0111-ib.pocstor_35625_object23944/head//22 by client.940330.0:23945 2014-08-11 16:31:11.947531 2014-08-13 17:38:02.620364 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 5b4a092a/benchmark_data_storage0111-ib.pocstor_35625_object24360/head//22 by client.940330.0:24361 2014-08-11 16:31:13.460155 2014-08-13 17:38:02.620385 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 8aa3912a/benchmark_data_storage0111-ib.pocstor_35625_object24503/head//22 by client.940330.0:24504 2014-08-11 16:31:13.985335 2014-08-13 17:38:02.620423 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 7044892a/benchmark_data_storage0111-ib.pocstor_35625_object24607/head//22 by client.940330.0:24608 2014-08-11 16:31:14.372515 2014-08-13 17:38:02.620446 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 11d0392a/benchmark_data_storage0111-ib.pocstor_35625_object25772/head//22 by client.940330.0:25773 2014-08-11 16:31:18.895464 2014-08-13 17:38:02.620467 7fe9bae887a0 20 read_log 192888'443 (0'0) modify b513c92a/benchmark_data_storage0111-ib.pocstor_35625_object25997/head//22 by client.940330.0:25998 2014-08-11 16:31:19.781326 2014-08-13 17:38:02.620488 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 1fd5612a/benchmark_data_storage0111-ib.pocstor_35625_object26880/head//22 by client.940330.0:26881 2014-08-11 16:31:23.340354 2014-08-13 17:38:02.620509 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 6407f92a/benchmark_data_storage0111-ib.pocstor_35625_object28894/head//22 by client.940330.0:28895 2014-08-11 16:31:31.184224 2014-08-13 17:38:02.620530 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 29b9392a/benchmark_data_storage0111-ib.pocstor_35625_object31078/head//22 by client.940330.0:31079 2014-08-11 16:31:39.518108 2014-08-13 17:38:02.620550 7fe9bae887a0 20 read_log 192888'447 (192888'430) delete 2ae2f12a/benchmark_data_storage0111-ib.pocstor_35625_object255/head//22 by client.940330.0:31563 2014-08-11 16:31:41.546524 2014-08-13 17:38:02.620572 7fe9bae887a0 20 read_log 192888'448 (192888'431) delete 680d612a/benchmark_data_storage0111-ib.pocstor_35625_object2015/head//22 by client.940330.0:33323 2014-08-11 16:31:41.995430 2014-08-13 17:38:02.620594 7fe9bae887a0 20 read_log 192888'449 (192888'432) delete 82b8892a/benchmark_data_storage0111-ib.pocstor_35625_object7489/head//22 by client.940330.0:38797 2014-08-11 16:31:43.461043 2014-08-13 17:38:02.620615 7fe9bae887a0 20 read_log 192888'450 (192888'433) delete 9ff3d92a/benchmark_data_storage0111-ib.pocstor_35625_object7813/head//22 by client.940330.0:39121 2014-08-11 16:31:43.543246 2014-08-13 17:38:02.620639 7fe9bae887a0 20 read_log 192888'451 (192888'434) delete 13fa912a/benchmark_data_storage0111-ib.pocstor_35625_object8599/head//22 by client.940330.0:39907 2014-08-11 16:31:43.739005 2014-08-13 17:38:02.620661 7fe9bae887a0 20 read_log 192888'452 (192888'435) delete 4120392a/benchmark_data_storage0111-ib.pocstor_35625_object13555/head//22 by client.940330.0:44863 2014-08-11 16:31:45.068963 2014-08-13 17:38:02.620683 7fe9bae887a0 20 read_log 192888'453 (192888'436) delete 3a90012a/benchmark_data_storage0111-ib.pocstor_35625_object15524/head//22 by client.940330.0:46832 2014-08-11 16:31:45.596697 2014-08-13 17:38:02.620704 7fe9bae887a0 20 read_log 192888'454 (192888'437) delete 575c912a/benchmark_data_storage0111-ib.pocstor_35625_object21817/head//22 by client.940330.0:53125 2014-08-11 16:31:47.389161 2014-08-13 17:38:02.620725 7fe9bae887a0 20 read_log 192888'455 (192888'438) delete 3bd3a92a/benchmark_data_storage0111-ib.pocstor_35625_object23944/head//22 by client.940330.0:55252 2014-08-11 16:31:48.021494 2014-08-13 17:38:02.620747 7fe9bae887a0 20 read_log 192888'456 (192888'439) delete 5b4a092a/benchmark_data_storage0111-ib.pocstor_35625_object24360/head//22 by client.940330.0:55668 2014-08-11 16:31:48.145378 2014-08-13 17:38:02.620768 7fe9bae887a0 20 read_log 192888'457 (192888'440) delete 8aa3912a/benchmark_data_storage0111-ib.pocstor_35625_object24503/head//22 by client.940330.0:55811 2014-08-11 16:31:48.190113 2014-08-13 17:38:02.620789 7fe9bae887a0 20 read_log 192888'458 (192888'441) delete 7044892a/benchmark_data_storage0111-ib.pocstor_35625_object24607/head//22 by client.940330.0:55915 2014-08-11 16:31:48.218663 2014-08-13 17:38:02.620811 7fe9bae887a0 20 read_log 192888'459 (192888'442) delete 11d0392a/benchmark_data_storage0111-ib.pocstor_35625_object25772/head//22 by client.940330.0:57080 2014-08-11 16:31:48.579845 2014-08-13 17:38:02.620832 7fe9bae887a0 20 read_log 192888'460 (192888'443) delete b513c92a/benchmark_data_storage0111-ib.pocstor_35625_object25997/head//22 by client.940330.0:57305 2014-08-11 16:31:48.647121 2014-08-13 17:38:02.620866 7fe9bae887a0 20 read_log 192888'461 (192888'444) delete 1fd5612a/benchmark_data_storage0111-ib.pocstor_35625_object26880/head//22 by client.940330.0:58188 2014-08-11 16:31:48.895189 2014-08-13 17:38:02.620888 7fe9bae887a0 20 read_log 192888'462 (192888'445) delete 6407f92a/benchmark_data_storage0111-ib.pocstor_35625_object28894/head//22 by client.940330.0:60202 2014-08-11 16:31:49.444197 2014-08-13 17:38:02.620909 7fe9bae887a0 20 read_log 192888'463 (192888'446) delete 29b9392a/benchmark_data_storage0111-ib.pocstor_35625_object31078/head//22 by client.940330.0:62386 2014-08-11 16:31:50.080511 2014-08-13 17:38:02.620930 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 2ee612a/benchmark_data_storage0111-ib.pocstor_64498_object379/head//22 by client.1049635.0:380 2014-08-12 17:17:18.353063 2014-08-13 17:38:02.620952 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 2c48c12a/benchmark_data_storage0111-ib.pocstor_64498_object1059/head//22 by client.1049635.0:1060 2014-08-12 17:17:20.610912 2014-08-13 17:38:02.620973 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 8af2112a/benchmark_data_storage0111-ib.pocstor_64498_object4083/head//22 by client.1049635.0:4084 2014-08-12 17:17:30.333738 2014-08-13 17:38:02.620997 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 147dc92a/benchmark_data_storage0111-ib.pocstor_64498_object10160/head//22 by client.1049635.0:10161 2014-08-12 17:17:48.385336 2014-08-13 17:38:02.621018 7fe9bae887a0 20 read_log 194215'468 (0'0) modify d568392a/benchmark_data_storage0111-ib.pocstor_64498_object10885/head//22 by client.1049635.0:10886 2014-08-12 17:17:50.767452 2014-08-13 17:38:02.621039 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 9e11f12a/benchmark_data_storage0111-ib.pocstor_64498_object14759/head//22 by client.1049635.0:14760 2014-08-12 17:18:03.193799 2014-08-13 17:38:02.621061 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 9e17192a/benchmark_data_storage0111-ib.pocstor_64498_object16689/head//22 by client.1049635.0:16690 2014-08-12 17:18:10.190169 2014-08-13 17:38:02.621082 7fe9bae887a0 20 read_log 194215'471 (0'0) modify ffa1c12a/benchmark_data_storage0111-ib.pocstor_64498_object22262/head//22 by client.1049635.0:22263 2014-08-12 17:18:31.317648 2014-08-13 17:38:02.621103 7fe9bae887a0 20 read_log 194215'472 (0'0) modify f6b0e92a/benchmark_data_storage0111-ib.pocstor_64498_object23093/head//22 by client.1049635.0:23094 2014-08-12 17:18:34.475902 2014-08-13 17:38:02.621124 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 7f64192a/benchmark_data_storage0111-ib.pocstor_64498_object24449/head//22 by client.1049635.0:24450 2014-08-12 17:18:39.524038 2014-08-13 17:38:02.621144 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 70cf912a/benchmark_data_storage0111-ib.pocstor_64498_object24585/head//22 by client.1049635.0:24586 2014-08-12 17:18:40.036578 2014-08-13 17:38:02.621173 7fe9bae887a0 20 read_log 194215'475 (0'0) modify e836812a/benchmark_data_storage0111-ib.pocstor_64498_object24977/head//22 by client.1049635.0:24978 2014-08-12 17:18:41.631543 2014-08-13 17:38:02.621195 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 172e592a/benchmark_data_storage0111-ib.pocstor_64498_object29066/head//22 by client.1049635.0:29067 2014-08-12 17:18:56.843588 2014-08-13 17:38:02.621217 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 6ec1012a/benchmark_data_storage0111-ib.pocstor_64498_object29425/head//22 by client.1049635.0:29426 2014-08-12 17:18:58.191693 2014-08-13 17:38:02.621238 7fe9bae887a0 20 read_log 194215'478 (194215'464) delete 2ee612a/benchmark_data_storage0111-ib.pocstor_64498_object379/head//22 by client.1049635.0:34582 2014-08-12 17:19:18.121056 2014-08-13 17:38:02.621259 7fe9bae887a0 20 read_log 194215'479 (194215'465) delete 2c48c12a/benchmark_data_storage0111-ib.pocstor_64498_object1059/head//22 by client.1049635.0:35262 2014-08-12 17:19:18.339151 2014-08-13 17:38:02.621294 7fe9bae887a0 20 read_log 194215'480 (194215'466) delete 8af2112a/benchmark_data_storage0111-ib.pocstor_64498_object4083/head//22 by client.1049635.0:38286 2014-08-12 17:19:19.301383 2014-08-13 17:38:02.621316 7fe9bae887a0 20 read_log 194215'481 (194215'467) delete 147dc92a/benchmark_data_storage0111-ib.pocstor_64498_object10160/head//22 by client.1049635.0:44363 2014-08-12 17:19:21.314842 2014-08-13 17:38:02.621337 7fe9bae887a0 20 read_log 194215'482 (194215'468) delete d568392a/benchmark_data_storage0111-ib.pocstor_64498_object10885/head//22 by client.1049635.0:45088 2014-08-12 17:19:21.543309 2014-08-13 17:38:02.621358 7fe9bae887a0 20 read_log 194215'483 (194215'469) delete 9e11f12a/benchmark_data_storage0111-ib.pocstor_64498_object14759/head//22 by client.1049635.0:48962 2014-08-12 17:19:22.802175 2014-08-13 17:38:02.621379 7fe9bae887a0 20 read_log 194215'484 (194215'470) delete 9e17192a/benchmark_data_storage0111-ib.pocstor_64498_object16689/head//22 by client.1049635.0:50892 2014-08-12 17:19:23.414098 2014-08-13 17:38:02.621399 7fe9bae887a0 20 read_log 194215'485 (194215'471) delete ffa1c12a/benchmark_data_storage0111-ib.pocstor_64498_object22262/head//22 by client.1049635.0:56465 2014-08-12 17:19:25.328865 2014-08-13 17:38:02.621420 7fe9bae887a0 20 read_log 194215'486 (194215'472) delete f6b0e92a/benchmark_data_storage0111-ib.pocstor_64498_object23093/head//22 by client.1049635.0:57296 2014-08-12 17:19:25.654589 2014-08-13 17:38:02.621441 7fe9bae887a0 20 read_log 194215'487 (194215'473) delete 7f64192a/benchmark_data_storage0111-ib.pocstor_64498_object24449/head//22 by client.1049635.0:58652 2014-08-12 17:19:26.115871 2014-08-13 17:38:02.621461 7fe9bae887a0 20 read_log 194215'488 (194215'474) delete 70cf912a/benchmark_data_storage0111-ib.pocstor_64498_object24585/head//22 by client.1049635.0:58788 2014-08-12 17:19:26.168053 2014-08-13 17:38:02.621513 7fe9bae887a0 20 read_log 194215'489 (194215'475) delete e836812a/benchmark_data_storage0111-ib.pocstor_64498_object24977/head//22 by client.1049635.0:59180 2014-08-12 17:19:26.314211 2014-08-13 17:38:02.621536 7fe9bae887a0 20 read_log 194215'490 (194215'476) delete 172e592a/benchmark_data_storage0111-ib.pocstor_64498_object29066/head//22 by client.1049635.0:63269 2014-08-12 17:19:27.727725 2014-08-13 17:38:02.621557 7fe9bae887a0 20 read_log 194215'491 (194215'477) delete 6ec1012a/benchmark_data_storage0111-ib.pocstor_64498_object29425/head//22 by client.1049635.0:63628 2014-08-12 17:19:27.859889 2014-08-13 17:38:02.621761 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.621788 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.621811 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.003189 0 0.000000 2014-08-13 17:38:02.621831 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.621845 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.621860 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] log((0'0,194215'491], crt=192888'463) 2014-08-13 17:38:02.621876 7fe9bae887a0 10 osd.4 194478 pgid 22.14a coll 22.14a_head 2014-08-13 17:38:02.622064 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.14a 2014-08-13 17:38:02.622075 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.622118 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.14a(unlocked)] enter Initial 2014-08-13 17:38:02.622134 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.14a(unlocked)] enter NotTrimming 2014-08-13 17:38:02.622348 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.622527 7fe9bae887a0 20 read_log 191648'11 (0'0) modify 58d3594a/benchmark_data_storage0111-ib.pocstor_40745_object984/head//22 by client.273158.0:985 2014-07-30 14:08:52.424666 2014-08-13 17:38:02.622563 7fe9bae887a0 20 read_log 191648'36 (0'0) modify 441e514a/benchmark_data_storage0111-ib.pocstor_40745_object2331/head//22 by client.273158.0:2332 2014-07-30 14:08:57.801631 2014-08-13 17:38:02.622586 7fe9bae887a0 20 read_log 191648'52 (0'0) modify e483894a/benchmark_data_storage0111-ib.pocstor_40745_object3123/head//22 by client.273158.0:3124 2014-07-30 14:09:01.863211 2014-08-13 17:38:02.622607 7fe9bae887a0 20 read_log 191648'58 (0'0) modify 5752f94a/benchmark_data_storage0111-ib.pocstor_40745_object3516/head//22 by client.273158.0:3517 2014-07-30 14:09:03.797950 2014-08-13 17:38:02.622628 7fe9bae887a0 20 read_log 191648'126 (0'0) modify 8bfb394a/benchmark_data_storage0111-ib.pocstor_40745_object7135/head//22 by client.273158.0:7136 2014-07-30 14:09:21.622499 2014-08-13 17:38:02.622650 7fe9bae887a0 20 read_log 191648'127 (0'0) modify dba1094a/benchmark_data_storage0111-ib.pocstor_40745_object7202/head//22 by client.273158.0:7203 2014-07-30 14:09:21.886306 2014-08-13 17:38:02.622721 7fe9bae887a0 20 read_log 191648'142 (0'0) modify 4475314a/benchmark_data_storage0111-ib.pocstor_40745_object8182/head//22 by client.273158.0:8183 2014-07-30 14:09:27.071935 2014-08-13 17:38:02.622750 7fe9bae887a0 20 read_log 191648'227 (0'0) modify 1ed6894a/benchmark_data_storage0111-ib.pocstor_40745_object14094/head//22 by client.273158.0:14095 2014-07-30 14:09:57.444232 2014-08-13 17:38:02.622772 7fe9bae887a0 20 read_log 191648'230 (0'0) modify 1b50c14a/benchmark_data_storage0111-ib.pocstor_40745_object14494/head//22 by client.273158.0:14495 2014-07-30 14:10:00.064442 2014-08-13 17:38:02.622794 7fe9bae887a0 20 read_log 191648'232 (0'0) modify cfc9e14a/benchmark_data_storage0111-ib.pocstor_40745_object14681/head//22 by client.273158.0:14682 2014-07-30 14:10:00.915715 2014-08-13 17:38:02.622814 7fe9bae887a0 20 read_log 191648'268 (0'0) modify 7221e14a/benchmark_data_storage0111-ib.pocstor_40745_object16901/head//22 by client.273158.0:16902 2014-07-30 14:10:12.219844 2014-08-13 17:38:02.622838 7fe9bae887a0 20 read_log 191648'281 (0'0) modify 5ef0814a/benchmark_data_storage0111-ib.pocstor_40745_object17758/head//22 by client.273158.0:17759 2014-07-30 14:10:16.898139 2014-08-13 17:38:02.622860 7fe9bae887a0 20 read_log 191648'292 (0'0) modify 64c5414a/benchmark_data_storage0111-ib.pocstor_40745_object18496/head//22 by client.273158.0:18497 2014-07-30 14:10:20.854550 2014-08-13 17:38:02.622880 7fe9bae887a0 20 read_log 191648'301 (0'0) modify d5b3a14a/benchmark_data_storage0111-ib.pocstor_40745_object19469/head//22 by client.273158.0:19470 2014-07-30 14:10:26.248542 2014-08-13 17:38:02.622901 7fe9bae887a0 20 read_log 191727'361 (0'0) modify ed57514a/benchmark_data_storage0111-ib.pocstor_51285_object993/head//22 by client.275175.0:994 2014-07-30 14:18:37.425929 2014-08-13 17:38:02.622921 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 4488d94a/benchmark_data_storage0111-ib.pocstor_51285_object4714/head//22 by client.275175.0:4715 2014-07-30 14:18:49.499713 2014-08-13 17:38:02.622942 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 6011494a/benchmark_data_storage0111-ib.pocstor_51285_object5151/head//22 by client.275175.0:5152 2014-07-30 14:18:50.796850 2014-08-13 17:38:02.622962 7fe9bae887a0 20 read_log 191727'377 (0'0) modify 7a4fa94a/benchmark_data_storage0111-ib.pocstor_51285_object8499/head//22 by client.275175.0:8500 2014-07-30 14:19:00.924572 2014-08-13 17:38:02.622983 7fe9bae887a0 20 read_log 191727'380 (0'0) modify 4ed2b94a/benchmark_data_storage0111-ib.pocstor_51285_object9611/head//22 by client.275175.0:9612 2014-07-30 14:19:04.101174 2014-08-13 17:38:02.623004 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 9b4c114a/benchmark_data_storage0111-ib.pocstor_51285_object10789/head//22 by client.275175.0:10790 2014-07-30 14:19:07.453352 2014-08-13 17:38:02.623024 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 96ffb14a/benchmark_data_storage0111-ib.pocstor_51285_object13539/head//22 by client.275175.0:13540 2014-07-30 14:19:16.354459 2014-08-13 17:38:02.623047 7fe9bae887a0 20 read_log 191727'387 (0'0) modify a5c2114a/benchmark_data_storage0111-ib.pocstor_51285_object14513/head//22 by client.275175.0:14514 2014-07-30 14:19:19.328359 2014-08-13 17:38:02.623067 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 314bd94a/benchmark_data_storage0111-ib.pocstor_51285_object20377/head//22 by client.275175.0:20378 2014-07-30 14:19:37.347686 2014-08-13 17:38:02.623088 7fe9bae887a0 20 read_log 191727'397 (0'0) modify ecde14a/benchmark_data_storage0111-ib.pocstor_51285_object20680/head//22 by client.275175.0:20681 2014-07-30 14:19:38.214708 2014-08-13 17:38:02.623111 7fe9bae887a0 20 read_log 191727'399 (0'0) modify c1b5e94a/benchmark_data_storage0111-ib.pocstor_51285_object22884/head//22 by client.275175.0:22885 2014-07-30 14:19:45.695087 2014-08-13 17:38:02.623143 7fe9bae887a0 20 read_log 191727'410 (0'0) modify f668f14a/benchmark_data_storage0111-ib.pocstor_51285_object27436/head//22 by client.275175.0:27437 2014-07-30 14:19:59.050386 2014-08-13 17:38:02.623179 7fe9bae887a0 20 read_log 191727'412 (0'0) modify a60f594a/benchmark_data_storage0111-ib.pocstor_51285_object27958/head//22 by client.275175.0:27959 2014-07-30 14:20:00.913642 2014-08-13 17:38:02.623200 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 3bf314a/benchmark_data_storage0111-ib.pocstor_51285_object28306/head//22 by client.275175.0:28307 2014-07-30 14:20:01.883309 2014-08-13 17:38:02.623221 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 5bef594a/benchmark_data_storage0111-ib.pocstor_51285_object30136/head//22 by client.275175.0:30137 2014-07-30 14:20:07.403920 2014-08-13 17:38:02.623242 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 433d714a/benchmark_data_storage0111-ib.pocstor_51285_object32336/head//22 by client.275175.0:32337 2014-07-30 14:20:13.856982 2014-08-13 17:38:02.623262 7fe9bae887a0 20 read_log 191727'431 (0'0) modify a310894a/benchmark_data_storage0111-ib.pocstor_51285_object38685/head//22 by client.275175.0:38686 2014-07-30 14:20:33.197517 2014-08-13 17:38:02.623283 7fe9bae887a0 20 read_log 192674'432 (0'0) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2143 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623304 7fe9bae887a0 20 read_log 192674'433 (192674'432) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2144 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623324 7fe9bae887a0 20 read_log 192674'434 (192674'433) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2145 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623345 7fe9bae887a0 20 read_log 192674'435 (192674'434) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2146 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623366 7fe9bae887a0 20 read_log 192674'436 (192674'435) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2147 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623386 7fe9bae887a0 20 read_log 192674'437 (192674'436) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2148 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623406 7fe9bae887a0 20 read_log 192674'438 (192674'437) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2149 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623427 7fe9bae887a0 20 read_log 192674'439 (192674'438) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2150 2014-07-31 18:03:43.955000 2014-08-13 17:38:02.623450 7fe9bae887a0 20 read_log 192674'440 (0'0) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4151 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623471 7fe9bae887a0 20 read_log 192674'441 (192674'440) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4152 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623491 7fe9bae887a0 20 read_log 192674'442 (192674'441) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4153 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623512 7fe9bae887a0 20 read_log 192674'443 (192674'442) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4154 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623532 7fe9bae887a0 20 read_log 192674'444 (192674'443) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4155 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623552 7fe9bae887a0 20 read_log 192674'445 (192674'444) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4156 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623572 7fe9bae887a0 20 read_log 192674'446 (192674'445) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4157 2014-07-31 18:03:46.080000 2014-08-13 17:38:02.623633 7fe9bae887a0 20 read_log 192674'447 (192674'446) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4158 2014-07-31 18:03:46.113000 2014-08-13 17:38:02.623655 7fe9bae887a0 20 read_log 192674'448 (0'0) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6463 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623676 7fe9bae887a0 20 read_log 192674'449 (192674'448) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6464 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623697 7fe9bae887a0 20 read_log 192674'450 (192674'449) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6465 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623717 7fe9bae887a0 20 read_log 192674'451 (192674'450) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6466 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623738 7fe9bae887a0 20 read_log 192674'452 (192674'451) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6467 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623758 7fe9bae887a0 20 read_log 192674'453 (192674'452) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6468 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623779 7fe9bae887a0 20 read_log 192674'454 (192674'453) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6469 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623799 7fe9bae887a0 20 read_log 192674'455 (192674'454) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6470 2014-07-31 18:03:48.593000 2014-08-13 17:38:02.623820 7fe9bae887a0 20 read_log 192888'456 (0'0) modify f1ce114a/benchmark_data_storage0111-ib.pocstor_35625_object1859/head//22 by client.940330.0:1860 2014-08-11 16:29:48.025890 2014-08-13 17:38:02.623841 7fe9bae887a0 20 read_log 192888'457 (0'0) modify c410b94a/benchmark_data_storage0111-ib.pocstor_35625_object2761/head//22 by client.940330.0:2762 2014-08-11 16:29:51.480235 2014-08-13 17:38:02.623862 7fe9bae887a0 20 read_log 192888'458 (0'0) modify bd08994a/benchmark_data_storage0111-ib.pocstor_35625_object5903/head//22 by client.940330.0:5904 2014-08-11 16:30:03.219611 2014-08-13 17:38:02.623882 7fe9bae887a0 20 read_log 192888'459 (0'0) modify f186c94a/benchmark_data_storage0111-ib.pocstor_35625_object9344/head//22 by client.940330.0:9345 2014-08-11 16:30:15.535506 2014-08-13 17:38:02.623904 7fe9bae887a0 20 read_log 192888'460 (0'0) modify b1b4914a/benchmark_data_storage0111-ib.pocstor_35625_object9958/head//22 by client.940330.0:9959 2014-08-11 16:30:17.709794 2014-08-13 17:38:02.623924 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 98e014a/benchmark_data_storage0111-ib.pocstor_35625_object13525/head//22 by client.940330.0:13526 2014-08-11 16:30:31.880706 2014-08-13 17:38:02.623945 7fe9bae887a0 20 read_log 192888'462 (0'0) modify ff0bf14a/benchmark_data_storage0111-ib.pocstor_35625_object13835/head//22 by client.940330.0:13836 2014-08-11 16:30:33.028338 2014-08-13 17:38:02.623966 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 658ec14a/benchmark_data_storage0111-ib.pocstor_35625_object17710/head//22 by client.940330.0:17711 2014-08-11 16:30:47.626530 2014-08-13 17:38:02.623986 7fe9bae887a0 20 read_log 192888'464 (0'0) modify d832994a/benchmark_data_storage0111-ib.pocstor_35625_object20941/head//22 by client.940330.0:20942 2014-08-11 16:31:00.478687 2014-08-13 17:38:02.624006 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 383bb14a/benchmark_data_storage0111-ib.pocstor_35625_object22915/head//22 by client.940330.0:22916 2014-08-11 16:31:08.109239 2014-08-13 17:38:02.624027 7fe9bae887a0 20 read_log 192888'466 (0'0) modify c32af94a/benchmark_data_storage0111-ib.pocstor_35625_object27150/head//22 by client.940330.0:27151 2014-08-11 16:31:24.381537 2014-08-13 17:38:02.624064 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 9984b14a/benchmark_data_storage0111-ib.pocstor_35625_object29527/head//22 by client.940330.0:29528 2014-08-11 16:31:33.673173 2014-08-13 17:38:02.624086 7fe9bae887a0 20 read_log 192888'468 (0'0) modify 6888194a/benchmark_data_storage0111-ib.pocstor_35625_object29996/head//22 by client.940330.0:29997 2014-08-11 16:31:35.446557 2014-08-13 17:38:02.624106 7fe9bae887a0 20 read_log 192888'469 (192888'456) delete f1ce114a/benchmark_data_storage0111-ib.pocstor_35625_object1859/head//22 by client.940330.0:33167 2014-08-11 16:31:41.955710 2014-08-13 17:38:02.624128 7fe9bae887a0 20 read_log 192888'470 (192888'457) delete c410b94a/benchmark_data_storage0111-ib.pocstor_35625_object2761/head//22 by client.940330.0:34069 2014-08-11 16:31:42.186630 2014-08-13 17:38:02.624149 7fe9bae887a0 20 read_log 192888'471 (192888'458) delete bd08994a/benchmark_data_storage0111-ib.pocstor_35625_object5903/head//22 by client.940330.0:37211 2014-08-11 16:31:43.049509 2014-08-13 17:38:02.624179 7fe9bae887a0 20 read_log 192888'472 (192888'459) delete f186c94a/benchmark_data_storage0111-ib.pocstor_35625_object9344/head//22 by client.940330.0:40652 2014-08-11 16:31:43.932129 2014-08-13 17:38:02.624201 7fe9bae887a0 20 read_log 192888'473 (192888'460) delete b1b4914a/benchmark_data_storage0111-ib.pocstor_35625_object9958/head//22 by client.940330.0:41266 2014-08-11 16:31:44.102231 2014-08-13 17:38:02.624222 7fe9bae887a0 20 read_log 192888'474 (192888'461) delete 98e014a/benchmark_data_storage0111-ib.pocstor_35625_object13525/head//22 by client.940330.0:44833 2014-08-11 16:31:45.060642 2014-08-13 17:38:02.624243 7fe9bae887a0 20 read_log 192888'475 (192888'462) delete ff0bf14a/benchmark_data_storage0111-ib.pocstor_35625_object13835/head//22 by client.940330.0:45143 2014-08-11 16:31:45.140180 2014-08-13 17:38:02.624296 7fe9bae887a0 20 read_log 192888'476 (192888'463) delete 658ec14a/benchmark_data_storage0111-ib.pocstor_35625_object17710/head//22 by client.940330.0:49018 2014-08-11 16:31:46.194261 2014-08-13 17:38:02.624317 7fe9bae887a0 20 read_log 192888'477 (192888'464) delete d832994a/benchmark_data_storage0111-ib.pocstor_35625_object20941/head//22 by client.940330.0:52249 2014-08-11 16:31:47.146610 2014-08-13 17:38:02.624338 7fe9bae887a0 20 read_log 192888'478 (192888'465) delete 383bb14a/benchmark_data_storage0111-ib.pocstor_35625_object22915/head//22 by client.940330.0:54223 2014-08-11 16:31:47.734145 2014-08-13 17:38:02.624359 7fe9bae887a0 20 read_log 192888'479 (192888'466) delete c32af94a/benchmark_data_storage0111-ib.pocstor_35625_object27150/head//22 by client.940330.0:58458 2014-08-11 16:31:48.966898 2014-08-13 17:38:02.624380 7fe9bae887a0 20 read_log 192888'480 (192888'467) delete 9984b14a/benchmark_data_storage0111-ib.pocstor_35625_object29527/head//22 by client.940330.0:60835 2014-08-11 16:31:49.621632 2014-08-13 17:38:02.624400 7fe9bae887a0 20 read_log 192888'481 (192888'468) delete 6888194a/benchmark_data_storage0111-ib.pocstor_35625_object29996/head//22 by client.940330.0:61304 2014-08-11 16:31:49.754541 2014-08-13 17:38:02.624422 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 717dd14a/benchmark_data_storage0111-ib.pocstor_64498_object1244/head//22 by client.1049635.0:1245 2014-08-12 17:17:21.182110 2014-08-13 17:38:02.624443 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 874d314a/benchmark_data_storage0111-ib.pocstor_64498_object2274/head//22 by client.1049635.0:2275 2014-08-12 17:17:24.419676 2014-08-13 17:38:02.624464 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 2d5914a/benchmark_data_storage0111-ib.pocstor_64498_object6707/head//22 by client.1049635.0:6708 2014-08-12 17:17:38.192236 2014-08-13 17:38:02.624484 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 39c6194a/benchmark_data_storage0111-ib.pocstor_64498_object7150/head//22 by client.1049635.0:7151 2014-08-12 17:17:39.556441 2014-08-13 17:38:02.624517 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 8062814a/benchmark_data_storage0111-ib.pocstor_64498_object10380/head//22 by client.1049635.0:10381 2014-08-12 17:17:49.005388 2014-08-13 17:38:02.624539 7fe9bae887a0 20 read_log 194215'487 (0'0) modify c3f4a94a/benchmark_data_storage0111-ib.pocstor_64498_object10397/head//22 by client.1049635.0:10398 2014-08-12 17:17:49.059861 2014-08-13 17:38:02.624590 7fe9bae887a0 20 read_log 194215'488 (0'0) modify a279614a/benchmark_data_storage0111-ib.pocstor_64498_object10557/head//22 by client.1049635.0:10558 2014-08-12 17:17:49.738377 2014-08-13 17:38:02.624612 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 1477f14a/benchmark_data_storage0111-ib.pocstor_64498_object16559/head//22 by client.1049635.0:16560 2014-08-12 17:18:09.684658 2014-08-13 17:38:02.624633 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 7685e14a/benchmark_data_storage0111-ib.pocstor_64498_object20125/head//22 by client.1049635.0:20126 2014-08-12 17:18:23.251118 2014-08-13 17:38:02.624654 7fe9bae887a0 20 read_log 194215'491 (0'0) modify a54ef94a/benchmark_data_storage0111-ib.pocstor_64498_object20451/head//22 by client.1049635.0:20452 2014-08-12 17:18:24.487405 2014-08-13 17:38:02.624675 7fe9bae887a0 20 read_log 194215'492 (0'0) modify cd23894a/benchmark_data_storage0111-ib.pocstor_64498_object22255/head//22 by client.1049635.0:22256 2014-08-12 17:18:31.291835 2014-08-13 17:38:02.624696 7fe9bae887a0 20 read_log 194215'493 (0'0) modify bd71494a/benchmark_data_storage0111-ib.pocstor_64498_object22587/head//22 by client.1049635.0:22588 2014-08-12 17:18:32.598005 2014-08-13 17:38:02.624716 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 56e0814a/benchmark_data_storage0111-ib.pocstor_64498_object23154/head//22 by client.1049635.0:23155 2014-08-12 17:18:34.705690 2014-08-13 17:38:02.624737 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 6ba494a/benchmark_data_storage0111-ib.pocstor_64498_object26578/head//22 by client.1049635.0:26579 2014-08-12 17:18:47.575725 2014-08-13 17:38:02.624758 7fe9bae887a0 20 read_log 194215'496 (0'0) modify fe95014a/benchmark_data_storage0111-ib.pocstor_64498_object26843/head//22 by client.1049635.0:26844 2014-08-12 17:18:48.565141 2014-08-13 17:38:02.624779 7fe9bae887a0 20 read_log 194215'497 (0'0) modify b1a6514a/benchmark_data_storage0111-ib.pocstor_64498_object33383/head//22 by client.1049635.0:33384 2014-08-12 17:19:13.830703 2014-08-13 17:38:02.624799 7fe9bae887a0 20 read_log 194215'498 (194215'482) delete 717dd14a/benchmark_data_storage0111-ib.pocstor_64498_object1244/head//22 by client.1049635.0:35447 2014-08-12 17:19:18.399394 2014-08-13 17:38:02.624820 7fe9bae887a0 20 read_log 194215'499 (194215'483) delete 874d314a/benchmark_data_storage0111-ib.pocstor_64498_object2274/head//22 by client.1049635.0:36477 2014-08-12 17:19:18.725286 2014-08-13 17:38:02.624841 7fe9bae887a0 20 read_log 194215'500 (194215'484) delete 2d5914a/benchmark_data_storage0111-ib.pocstor_64498_object6707/head//22 by client.1049635.0:40910 2014-08-12 17:19:20.188979 2014-08-13 17:38:02.624862 7fe9bae887a0 20 read_log 194215'501 (194215'485) delete 39c6194a/benchmark_data_storage0111-ib.pocstor_64498_object7150/head//22 by client.1049635.0:41353 2014-08-12 17:19:20.334217 2014-08-13 17:38:02.624883 7fe9bae887a0 20 read_log 194215'502 (194215'486) delete 8062814a/benchmark_data_storage0111-ib.pocstor_64498_object10380/head//22 by client.1049635.0:44583 2014-08-12 17:19:21.383194 2014-08-13 17:38:02.624904 7fe9bae887a0 20 read_log 194215'503 (194215'487) delete c3f4a94a/benchmark_data_storage0111-ib.pocstor_64498_object10397/head//22 by client.1049635.0:44600 2014-08-12 17:19:21.388686 2014-08-13 17:38:02.624924 7fe9bae887a0 20 read_log 194215'504 (194215'488) delete a279614a/benchmark_data_storage0111-ib.pocstor_64498_object10557/head//22 by client.1049635.0:44760 2014-08-12 17:19:21.438346 2014-08-13 17:38:02.624945 7fe9bae887a0 20 read_log 194215'505 (194215'489) delete 1477f14a/benchmark_data_storage0111-ib.pocstor_64498_object16559/head//22 by client.1049635.0:50762 2014-08-12 17:19:23.369862 2014-08-13 17:38:02.624978 7fe9bae887a0 20 read_log 194215'506 (194215'490) delete 7685e14a/benchmark_data_storage0111-ib.pocstor_64498_object20125/head//22 by client.1049635.0:54328 2014-08-12 17:19:24.569386 2014-08-13 17:38:02.625000 7fe9bae887a0 20 read_log 194215'507 (194215'491) delete a54ef94a/benchmark_data_storage0111-ib.pocstor_64498_object20451/head//22 by client.1049635.0:54654 2014-08-12 17:19:24.684566 2014-08-13 17:38:02.625021 7fe9bae887a0 20 read_log 194215'508 (194215'492) delete cd23894a/benchmark_data_storage0111-ib.pocstor_64498_object22255/head//22 by client.1049635.0:56458 2014-08-12 17:19:25.327311 2014-08-13 17:38:02.625041 7fe9bae887a0 20 read_log 194215'509 (194215'493) delete bd71494a/benchmark_data_storage0111-ib.pocstor_64498_object22587/head//22 by client.1049635.0:56790 2014-08-12 17:19:25.453033 2014-08-13 17:38:02.625062 7fe9bae887a0 20 read_log 194215'510 (194215'494) delete 56e0814a/benchmark_data_storage0111-ib.pocstor_64498_object23154/head//22 by client.1049635.0:57357 2014-08-12 17:19:25.674075 2014-08-13 17:38:02.625083 7fe9bae887a0 20 read_log 194215'511 (194215'495) delete 6ba494a/benchmark_data_storage0111-ib.pocstor_64498_object26578/head//22 by client.1049635.0:60781 2014-08-12 17:19:26.849926 2014-08-13 17:38:02.625104 7fe9bae887a0 20 read_log 194215'512 (194215'496) delete fe95014a/benchmark_data_storage0111-ib.pocstor_64498_object26843/head//22 by client.1049635.0:61046 2014-08-12 17:19:26.940480 2014-08-13 17:38:02.625125 7fe9bae887a0 20 read_log 194215'513 (194215'497) delete b1a6514a/benchmark_data_storage0111-ib.pocstor_64498_object33383/head//22 by client.1049635.0:67586 2014-08-12 17:19:29.249756 2014-08-13 17:38:02.625322 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.625356 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.625379 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.003261 0 0.000000 2014-08-13 17:38:02.625399 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.625413 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.625428 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] log((0'0,194215'513], crt=192888'481) 2014-08-13 17:38:02.625443 7fe9bae887a0 10 osd.4 194478 pgid 22.1a0 coll 22.1a0_head 2014-08-13 17:38:02.625655 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.1a0 2014-08-13 17:38:02.625666 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.625710 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1a0(unlocked)] enter Initial 2014-08-13 17:38:02.625732 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1a0(unlocked)] enter NotTrimming 2014-08-13 17:38:02.625934 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.626199 7fe9bae887a0 20 read_log 191648'1 (0'0) modify 600b39a0/benchmark_data_storage0111-ib.pocstor_40745_object110/head//22 by client.273158.0:111 2014-07-30 14:08:49.119811 2014-08-13 17:38:02.626238 7fe9bae887a0 20 read_log 191648'26 (0'0) modify 83a281a0/benchmark_data_storage0111-ib.pocstor_40745_object1682/head//22 by client.273158.0:1683 2014-07-30 14:08:54.961875 2014-08-13 17:38:02.626260 7fe9bae887a0 20 read_log 191648'28 (0'0) modify b9ba9a0/benchmark_data_storage0111-ib.pocstor_40745_object1712/head//22 by client.273158.0:1713 2014-07-30 14:08:55.072096 2014-08-13 17:38:02.626282 7fe9bae887a0 20 read_log 191648'86 (0'0) modify fa9371a0/benchmark_data_storage0111-ib.pocstor_40745_object5208/head//22 by client.273158.0:5209 2014-07-30 14:09:12.265080 2014-08-13 17:38:02.626303 7fe9bae887a0 20 read_log 191648'94 (0'0) modify 7e2321a0/benchmark_data_storage0111-ib.pocstor_40745_object5523/head//22 by client.273158.0:5524 2014-07-30 14:09:13.529328 2014-08-13 17:38:02.626324 7fe9bae887a0 20 read_log 191648'150 (0'0) modify d713f9a0/benchmark_data_storage0111-ib.pocstor_40745_object8715/head//22 by client.273158.0:8716 2014-07-30 14:09:30.052874 2014-08-13 17:38:02.626345 7fe9bae887a0 20 read_log 191648'275 (0'0) modify cfba71a0/benchmark_data_storage0111-ib.pocstor_40745_object16955/head//22 by client.273158.0:16956 2014-07-30 14:10:12.812878 2014-08-13 17:38:02.626365 7fe9bae887a0 20 read_log 191648'325 (0'0) modify 82ee1a0/benchmark_data_storage0111-ib.pocstor_40745_object19964/head//22 by client.273158.0:19965 2014-07-30 14:10:31.165516 2014-08-13 17:38:02.626386 7fe9bae887a0 20 read_log 191648'337 (0'0) modify 98ad99a0/benchmark_data_storage0111-ib.pocstor_40745_object20610/head//22 by client.273158.0:20611 2014-07-30 14:10:34.369909 2014-08-13 17:38:02.626407 7fe9bae887a0 20 read_log 191727'373 (0'0) modify b2cb59a0/benchmark_data_storage0111-ib.pocstor_51285_object401/head//22 by client.275175.0:402 2014-07-30 14:18:35.044411 2014-08-13 17:38:02.626428 7fe9bae887a0 20 read_log 191727'374 (0'0) modify f07e29a0/benchmark_data_storage0111-ib.pocstor_51285_object569/head//22 by client.275175.0:570 2014-07-30 14:18:35.720956 2014-08-13 17:38:02.626449 7fe9bae887a0 20 read_log 191727'379 (0'0) modify ebe239a0/benchmark_data_storage0111-ib.pocstor_51285_object2631/head//22 by client.275175.0:2632 2014-07-30 14:18:43.039442 2014-08-13 17:38:02.626470 7fe9bae887a0 20 read_log 191727'381 (0'0) modify 590059a0/benchmark_data_storage0111-ib.pocstor_51285_object3955/head//22 by client.275175.0:3956 2014-07-30 14:18:47.219045 2014-08-13 17:38:02.626491 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 5a8a61a0/benchmark_data_storage0111-ib.pocstor_51285_object4222/head//22 by client.275175.0:4223 2014-07-30 14:18:48.030043 2014-08-13 17:38:02.626512 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 133461a0/benchmark_data_storage0111-ib.pocstor_51285_object7769/head//22 by client.275175.0:7770 2014-07-30 14:18:58.883388 2014-08-13 17:38:02.626535 7fe9bae887a0 20 read_log 191727'389 (0'0) modify b4f0c1a0/benchmark_data_storage0111-ib.pocstor_51285_object8978/head//22 by client.275175.0:8979 2014-07-30 14:19:02.286745 2014-08-13 17:38:02.626569 7fe9bae887a0 20 read_log 191727'395 (0'0) modify f61981a0/benchmark_data_storage0111-ib.pocstor_51285_object11051/head//22 by client.275175.0:11052 2014-07-30 14:19:08.188204 2014-08-13 17:38:02.626590 7fe9bae887a0 20 read_log 191727'396 (0'0) modify be1ee9a0/benchmark_data_storage0111-ib.pocstor_51285_object12293/head//22 by client.275175.0:12294 2014-07-30 14:19:12.460330 2014-08-13 17:38:02.626611 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 9ff0e1a0/benchmark_data_storage0111-ib.pocstor_51285_object12356/head//22 by client.275175.0:12357 2014-07-30 14:19:12.633360 2014-08-13 17:38:02.626632 7fe9bae887a0 20 read_log 191727'405 (0'0) modify d15511a0/benchmark_data_storage0111-ib.pocstor_51285_object16469/head//22 by client.275175.0:16470 2014-07-30 14:19:25.579341 2014-08-13 17:38:02.626654 7fe9bae887a0 20 read_log 191727'410 (0'0) modify eb4a59a0/benchmark_data_storage0111-ib.pocstor_51285_object20107/head//22 by client.275175.0:20108 2014-07-30 14:19:36.565767 2014-08-13 17:38:02.626675 7fe9bae887a0 20 read_log 191727'416 (0'0) modify a29c11a0/benchmark_data_storage0111-ib.pocstor_51285_object22774/head//22 by client.275175.0:22775 2014-07-30 14:19:45.351281 2014-08-13 17:38:02.626695 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 440b39a0/benchmark_data_storage0111-ib.pocstor_51285_object23168/head//22 by client.275175.0:23169 2014-07-30 14:19:46.498028 2014-08-13 17:38:02.626716 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 47c059a0/benchmark_data_storage0111-ib.pocstor_51285_object24890/head//22 by client.275175.0:24891 2014-07-30 14:19:51.330283 2014-08-13 17:38:02.626739 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 426b9a0/benchmark_data_storage0111-ib.pocstor_51285_object30670/head//22 by client.275175.0:30671 2014-07-30 14:20:09.052033 2014-08-13 17:38:02.626762 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 87c9b1a0/benchmark_data_storage0111-ib.pocstor_51285_object30816/head//22 by client.275175.0:30817 2014-07-30 14:20:09.460213 2014-08-13 17:38:02.626783 7fe9bae887a0 20 read_log 191727'430 (0'0) modify b0e561a0/benchmark_data_storage0111-ib.pocstor_51285_object31260/head//22 by client.275175.0:31261 2014-07-30 14:20:10.724977 2014-08-13 17:38:02.626804 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 546a11a0/benchmark_data_storage0111-ib.pocstor_51285_object31654/head//22 by client.275175.0:31655 2014-07-30 14:20:11.883103 2014-08-13 17:38:02.626825 7fe9bae887a0 20 read_log 191727'441 (0'0) modify 338bd1a0/benchmark_data_storage0111-ib.pocstor_51285_object35804/head//22 by client.275175.0:35805 2014-07-30 14:20:24.662355 2014-08-13 17:38:02.626847 7fe9bae887a0 20 read_log 192674'447 (0'0) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6383 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626867 7fe9bae887a0 20 read_log 192674'448 (192674'447) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6384 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626888 7fe9bae887a0 20 read_log 192674'449 (192674'448) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6385 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626909 7fe9bae887a0 20 read_log 192674'450 (192674'449) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6386 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626929 7fe9bae887a0 20 read_log 192674'451 (192674'450) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6387 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626950 7fe9bae887a0 20 read_log 192674'452 (192674'451) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6388 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.626970 7fe9bae887a0 20 read_log 192674'453 (192674'452) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6389 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.627007 7fe9bae887a0 20 read_log 192674'454 (192674'453) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6390 2014-07-31 18:03:48.454000 2014-08-13 17:38:02.627035 7fe9bae887a0 20 read_log 192888'455 (0'0) modify 5b49f1a0/benchmark_data_storage0111-ib.pocstor_35625_object376/head//22 by client.940330.0:377 2014-08-11 16:29:42.200388 2014-08-13 17:38:02.627057 7fe9bae887a0 20 read_log 192888'456 (0'0) modify ded991a0/benchmark_data_storage0111-ib.pocstor_35625_object1054/head//22 by client.940330.0:1055 2014-08-11 16:29:44.993610 2014-08-13 17:38:02.627078 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 6c4139a0/benchmark_data_storage0111-ib.pocstor_35625_object3140/head//22 by client.940330.0:3141 2014-08-11 16:29:53.077569 2014-08-13 17:38:02.627103 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 6bee09a0/benchmark_data_storage0111-ib.pocstor_35625_object3321/head//22 by client.940330.0:3322 2014-08-11 16:29:53.817583 2014-08-13 17:38:02.627124 7fe9bae887a0 20 read_log 192888'459 (0'0) modify a731a1a0/benchmark_data_storage0111-ib.pocstor_35625_object4118/head//22 by client.940330.0:4119 2014-08-11 16:29:56.760193 2014-08-13 17:38:02.627145 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 702041a0/benchmark_data_storage0111-ib.pocstor_35625_object6608/head//22 by client.940330.0:6609 2014-08-11 16:30:05.859199 2014-08-13 17:38:02.627180 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 440fb9a0/benchmark_data_storage0111-ib.pocstor_35625_object8841/head//22 by client.940330.0:8842 2014-08-11 16:30:13.777862 2014-08-13 17:38:02.627207 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 90a701a0/benchmark_data_storage0111-ib.pocstor_35625_object9970/head//22 by client.940330.0:9971 2014-08-11 16:30:17.769307 2014-08-13 17:38:02.627228 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 6653b9a0/benchmark_data_storage0111-ib.pocstor_35625_object21052/head//22 by client.940330.0:21053 2014-08-11 16:31:01.029940 2014-08-13 17:38:02.627249 7fe9bae887a0 20 read_log 192888'464 (0'0) modify ad1361a0/benchmark_data_storage0111-ib.pocstor_35625_object21274/head//22 by client.940330.0:21275 2014-08-11 16:31:01.852186 2014-08-13 17:38:02.627274 7fe9bae887a0 20 read_log 192888'465 (0'0) modify ae5089a0/benchmark_data_storage0111-ib.pocstor_35625_object22073/head//22 by client.940330.0:22074 2014-08-11 16:31:04.844227 2014-08-13 17:38:02.627295 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 1aad71a0/benchmark_data_storage0111-ib.pocstor_35625_object29233/head//22 by client.940330.0:29234 2014-08-11 16:31:32.559058 2014-08-13 17:38:02.627316 7fe9bae887a0 20 read_log 192888'467 (192888'455) delete 5b49f1a0/benchmark_data_storage0111-ib.pocstor_35625_object376/head//22 by client.940330.0:31684 2014-08-11 16:31:41.586859 2014-08-13 17:38:02.627337 7fe9bae887a0 20 read_log 192888'468 (192888'456) delete ded991a0/benchmark_data_storage0111-ib.pocstor_35625_object1054/head//22 by client.940330.0:32362 2014-08-11 16:31:41.762189 2014-08-13 17:38:02.627358 7fe9bae887a0 20 read_log 192888'469 (192888'457) delete 6c4139a0/benchmark_data_storage0111-ib.pocstor_35625_object3140/head//22 by client.940330.0:34448 2014-08-11 16:31:42.291153 2014-08-13 17:38:02.627380 7fe9bae887a0 20 read_log 192888'470 (192888'458) delete 6bee09a0/benchmark_data_storage0111-ib.pocstor_35625_object3321/head//22 by client.940330.0:34629 2014-08-11 16:31:42.341146 2014-08-13 17:38:02.627401 7fe9bae887a0 20 read_log 192888'471 (192888'459) delete a731a1a0/benchmark_data_storage0111-ib.pocstor_35625_object4118/head//22 by client.940330.0:35426 2014-08-11 16:31:42.568026 2014-08-13 17:38:02.627421 7fe9bae887a0 20 read_log 192888'472 (192888'460) delete 702041a0/benchmark_data_storage0111-ib.pocstor_35625_object6608/head//22 by client.940330.0:37916 2014-08-11 16:31:43.236526 2014-08-13 17:38:02.627442 7fe9bae887a0 20 read_log 192888'473 (192888'461) delete 440fb9a0/benchmark_data_storage0111-ib.pocstor_35625_object8841/head//22 by client.940330.0:40149 2014-08-11 16:31:43.804048 2014-08-13 17:38:02.627480 7fe9bae887a0 20 read_log 192888'474 (192888'462) delete 90a701a0/benchmark_data_storage0111-ib.pocstor_35625_object9970/head//22 by client.940330.0:41278 2014-08-11 16:31:44.104809 2014-08-13 17:38:02.627503 7fe9bae887a0 20 read_log 192888'475 (192888'463) delete 6653b9a0/benchmark_data_storage0111-ib.pocstor_35625_object21052/head//22 by client.940330.0:52360 2014-08-11 16:31:47.176053 2014-08-13 17:38:02.627524 7fe9bae887a0 20 read_log 192888'476 (192888'464) delete ad1361a0/benchmark_data_storage0111-ib.pocstor_35625_object21274/head//22 by client.940330.0:52582 2014-08-11 16:31:47.233382 2014-08-13 17:38:02.627545 7fe9bae887a0 20 read_log 192888'477 (192888'465) delete ae5089a0/benchmark_data_storage0111-ib.pocstor_35625_object22073/head//22 by client.940330.0:53381 2014-08-11 16:31:47.464951 2014-08-13 17:38:02.627567 7fe9bae887a0 20 read_log 192888'478 (192888'466) delete 1aad71a0/benchmark_data_storage0111-ib.pocstor_35625_object29233/head//22 by client.940330.0:60541 2014-08-11 16:31:49.533375 2014-08-13 17:38:02.627588 7fe9bae887a0 20 read_log 194215'479 (0'0) modify c87389a0/benchmark_data_storage0111-ib.pocstor_64498_object1497/head//22 by client.1049635.0:1498 2014-08-12 17:17:21.985253 2014-08-13 17:38:02.627609 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 448151a0/benchmark_data_storage0111-ib.pocstor_64498_object2771/head//22 by client.1049635.0:2772 2014-08-12 17:17:26.046075 2014-08-13 17:38:02.627630 7fe9bae887a0 20 read_log 194215'481 (0'0) modify c79731a0/benchmark_data_storage0111-ib.pocstor_64498_object3592/head//22 by client.1049635.0:3593 2014-08-12 17:17:28.779857 2014-08-13 17:38:02.627679 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 150129a0/benchmark_data_storage0111-ib.pocstor_64498_object8387/head//22 by client.1049635.0:8388 2014-08-12 17:17:43.232331 2014-08-13 17:38:02.627701 7fe9bae887a0 20 read_log 194215'483 (0'0) modify b16921a0/benchmark_data_storage0111-ib.pocstor_64498_object10172/head//22 by client.1049635.0:10173 2014-08-12 17:17:48.419376 2014-08-13 17:38:02.627722 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 9c1c49a0/benchmark_data_storage0111-ib.pocstor_64498_object11620/head//22 by client.1049635.0:11621 2014-08-12 17:17:53.272455 2014-08-13 17:38:02.627743 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 528881a0/benchmark_data_storage0111-ib.pocstor_64498_object15022/head//22 by client.1049635.0:15023 2014-08-12 17:18:04.042953 2014-08-13 17:38:02.627764 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 786519a0/benchmark_data_storage0111-ib.pocstor_64498_object15081/head//22 by client.1049635.0:15082 2014-08-12 17:18:04.233294 2014-08-13 17:38:02.627785 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 8d821a0/benchmark_data_storage0111-ib.pocstor_64498_object16553/head//22 by client.1049635.0:16554 2014-08-12 17:18:09.662764 2014-08-13 17:38:02.627806 7fe9bae887a0 20 read_log 194215'488 (0'0) modify b4c219a0/benchmark_data_storage0111-ib.pocstor_64498_object16949/head//22 by client.1049635.0:16950 2014-08-12 17:18:11.172551 2014-08-13 17:38:02.627827 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 98d7d1a0/benchmark_data_storage0111-ib.pocstor_64498_object17756/head//22 by client.1049635.0:17757 2014-08-12 17:18:14.232965 2014-08-13 17:38:02.627850 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 427319a0/benchmark_data_storage0111-ib.pocstor_64498_object23448/head//22 by client.1049635.0:23449 2014-08-12 17:18:35.794615 2014-08-13 17:38:02.627871 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 5028e9a0/benchmark_data_storage0111-ib.pocstor_64498_object24117/head//22 by client.1049635.0:24118 2014-08-12 17:18:38.288347 2014-08-13 17:38:02.627892 7fe9bae887a0 20 read_log 194215'492 (0'0) modify e52d1a0/benchmark_data_storage0111-ib.pocstor_64498_object25642/head//22 by client.1049635.0:25643 2014-08-12 17:18:44.097285 2014-08-13 17:38:02.627925 7fe9bae887a0 20 read_log 194215'493 (0'0) modify b1a129a0/benchmark_data_storage0111-ib.pocstor_64498_object28097/head//22 by client.1049635.0:28098 2014-08-12 17:18:53.235708 2014-08-13 17:38:02.627980 7fe9bae887a0 20 read_log 194215'494 (0'0) modify cfd061a0/benchmark_data_storage0111-ib.pocstor_64498_object29846/head//22 by client.1049635.0:29847 2014-08-12 17:18:59.872725 2014-08-13 17:38:02.628004 7fe9bae887a0 20 read_log 194215'495 (0'0) modify a3dfc1a0/benchmark_data_storage0111-ib.pocstor_64498_object32736/head//22 by client.1049635.0:32737 2014-08-12 17:19:11.390281 2014-08-13 17:38:02.628028 7fe9bae887a0 20 read_log 194215'496 (194215'479) delete c87389a0/benchmark_data_storage0111-ib.pocstor_64498_object1497/head//22 by client.1049635.0:35700 2014-08-12 17:19:18.484174 2014-08-13 17:38:02.628050 7fe9bae887a0 20 read_log 194215'497 (194215'480) delete 448151a0/benchmark_data_storage0111-ib.pocstor_64498_object2771/head//22 by client.1049635.0:36974 2014-08-12 17:19:18.890511 2014-08-13 17:38:02.628071 7fe9bae887a0 20 read_log 194215'498 (194215'481) delete c79731a0/benchmark_data_storage0111-ib.pocstor_64498_object3592/head//22 by client.1049635.0:37795 2014-08-12 17:19:19.147688 2014-08-13 17:38:02.628091 7fe9bae887a0 20 read_log 194215'499 (194215'482) delete 150129a0/benchmark_data_storage0111-ib.pocstor_64498_object8387/head//22 by client.1049635.0:42590 2014-08-12 17:19:20.725944 2014-08-13 17:38:02.628113 7fe9bae887a0 20 read_log 194215'500 (194215'483) delete b16921a0/benchmark_data_storage0111-ib.pocstor_64498_object10172/head//22 by client.1049635.0:44375 2014-08-12 17:19:21.318279 2014-08-13 17:38:02.628134 7fe9bae887a0 20 read_log 194215'501 (194215'484) delete 9c1c49a0/benchmark_data_storage0111-ib.pocstor_64498_object11620/head//22 by client.1049635.0:45823 2014-08-12 17:19:21.772568 2014-08-13 17:38:02.628162 7fe9bae887a0 20 read_log 194215'502 (194215'485) delete 528881a0/benchmark_data_storage0111-ib.pocstor_64498_object15022/head//22 by client.1049635.0:49225 2014-08-12 17:19:22.882110 2014-08-13 17:38:02.628185 7fe9bae887a0 20 read_log 194215'503 (194215'486) delete 786519a0/benchmark_data_storage0111-ib.pocstor_64498_object15081/head//22 by client.1049635.0:49284 2014-08-12 17:19:22.899606 2014-08-13 17:38:02.628206 7fe9bae887a0 20 read_log 194215'504 (194215'487) delete 8d821a0/benchmark_data_storage0111-ib.pocstor_64498_object16553/head//22 by client.1049635.0:50756 2014-08-12 17:19:23.368538 2014-08-13 17:38:02.628227 7fe9bae887a0 20 read_log 194215'505 (194215'488) delete b4c219a0/benchmark_data_storage0111-ib.pocstor_64498_object16949/head//22 by client.1049635.0:51152 2014-08-12 17:19:23.492091 2014-08-13 17:38:02.628248 7fe9bae887a0 20 read_log 194215'506 (194215'489) delete 98d7d1a0/benchmark_data_storage0111-ib.pocstor_64498_object17756/head//22 by client.1049635.0:51959 2014-08-12 17:19:23.746670 2014-08-13 17:38:02.628269 7fe9bae887a0 20 read_log 194215'507 (194215'490) delete 427319a0/benchmark_data_storage0111-ib.pocstor_64498_object23448/head//22 by client.1049635.0:57651 2014-08-12 17:19:25.778543 2014-08-13 17:38:02.628289 7fe9bae887a0 20 read_log 194215'508 (194215'491) delete 5028e9a0/benchmark_data_storage0111-ib.pocstor_64498_object24117/head//22 by client.1049635.0:58320 2014-08-12 17:19:26.003026 2014-08-13 17:38:02.628311 7fe9bae887a0 20 read_log 194215'509 (194215'492) delete e52d1a0/benchmark_data_storage0111-ib.pocstor_64498_object25642/head//22 by client.1049635.0:59845 2014-08-12 17:19:26.534634 2014-08-13 17:38:02.628332 7fe9bae887a0 20 read_log 194215'510 (194215'493) delete b1a129a0/benchmark_data_storage0111-ib.pocstor_64498_object28097/head//22 by client.1049635.0:62300 2014-08-12 17:19:27.382909 2014-08-13 17:38:02.628353 7fe9bae887a0 20 read_log 194215'511 (194215'494) delete cfd061a0/benchmark_data_storage0111-ib.pocstor_64498_object29846/head//22 by client.1049635.0:64049 2014-08-12 17:19:28.000324 2014-08-13 17:38:02.628374 7fe9bae887a0 20 read_log 194215'512 (194215'495) delete a3dfc1a0/benchmark_data_storage0111-ib.pocstor_64498_object32736/head//22 by client.1049635.0:66939 2014-08-12 17:19:29.016427 2014-08-13 17:38:02.628528 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.628558 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=0 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.628581 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=0 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.002872 0 0.000000 2014-08-13 17:38:02.628602 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=0 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.628616 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=0 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.628631 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] log((0'0,194215'512], crt=192888'478) 2014-08-13 17:38:02.628651 7fe9bae887a0 10 osd.4 194478 pgid 22.1b8 coll 22.1b8_head 2014-08-13 17:38:02.628838 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.1b8 2014-08-13 17:38:02.628855 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.628900 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1b8(unlocked)] enter Initial 2014-08-13 17:38:02.628916 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1b8(unlocked)] enter NotTrimming 2014-08-13 17:38:02.629087 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.629340 7fe9bae887a0 20 read_log 191648'2 (0'0) modify 4f4b9b8/benchmark_data_storage0111-ib.pocstor_40745_object152/head//22 by client.273158.0:153 2014-07-30 14:08:49.277162 2014-08-13 17:38:02.629378 7fe9bae887a0 20 read_log 191648'42 (0'0) modify 1d5b09b8/benchmark_data_storage0111-ib.pocstor_40745_object2665/head//22 by client.273158.0:2666 2014-07-30 14:08:59.710222 2014-08-13 17:38:02.629415 7fe9bae887a0 20 read_log 191648'55 (0'0) modify 23bf39b8/benchmark_data_storage0111-ib.pocstor_40745_object3547/head//22 by client.273158.0:3548 2014-07-30 14:09:03.912460 2014-08-13 17:38:02.629441 7fe9bae887a0 20 read_log 191648'80 (0'0) modify d1d411b8/benchmark_data_storage0111-ib.pocstor_40745_object5405/head//22 by client.273158.0:5406 2014-07-30 14:09:13.052785 2014-08-13 17:38:02.629463 7fe9bae887a0 20 read_log 191648'97 (0'0) modify a3e0b9b8/benchmark_data_storage0111-ib.pocstor_40745_object6665/head//22 by client.273158.0:6666 2014-07-30 14:09:19.617243 2014-08-13 17:38:02.629484 7fe9bae887a0 20 read_log 191648'109 (0'0) modify fbb4b1b8/benchmark_data_storage0111-ib.pocstor_40745_object7277/head//22 by client.273158.0:7278 2014-07-30 14:09:22.185254 2014-08-13 17:38:02.629505 7fe9bae887a0 20 read_log 191648'135 (0'0) modify b519e1b8/benchmark_data_storage0111-ib.pocstor_40745_object9043/head//22 by client.273158.0:9044 2014-07-30 14:09:31.349216 2014-08-13 17:38:02.629526 7fe9bae887a0 20 read_log 191648'138 (0'0) modify 926801b8/benchmark_data_storage0111-ib.pocstor_40745_object9158/head//22 by client.273158.0:9159 2014-07-30 14:09:31.795808 2014-08-13 17:38:02.629547 7fe9bae887a0 20 read_log 191648'152 (0'0) modify 467fc9b8/benchmark_data_storage0111-ib.pocstor_40745_object9922/head//22 by client.273158.0:9923 2014-07-30 14:09:35.412045 2014-08-13 17:38:02.629569 7fe9bae887a0 20 read_log 191648'232 (0'0) modify 5e601b8/benchmark_data_storage0111-ib.pocstor_40745_object14236/head//22 by client.273158.0:14237 2014-07-30 14:09:58.964895 2014-08-13 17:38:02.629590 7fe9bae887a0 20 read_log 191648'257 (0'0) modify 21c779b8/benchmark_data_storage0111-ib.pocstor_40745_object16169/head//22 by client.273158.0:16170 2014-07-30 14:10:08.403247 2014-08-13 17:38:02.629610 7fe9bae887a0 20 read_log 191648'259 (0'0) modify 4ef7b9b8/benchmark_data_storage0111-ib.pocstor_40745_object16189/head//22 by client.273158.0:16190 2014-07-30 14:10:08.476618 2014-08-13 17:38:02.629631 7fe9bae887a0 20 read_log 191648'281 (0'0) modify 1b9919b8/benchmark_data_storage0111-ib.pocstor_40745_object17481/head//22 by client.273158.0:17482 2014-07-30 14:10:15.640829 2014-08-13 17:38:02.629652 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 77b2f9b8/benchmark_data_storage0111-ib.pocstor_51285_object1125/head//22 by client.275175.0:1126 2014-07-30 14:18:37.920252 2014-08-13 17:38:02.629674 7fe9bae887a0 20 read_log 191727'372 (0'0) modify b63a79b8/benchmark_data_storage0111-ib.pocstor_51285_object4473/head//22 by client.275175.0:4474 2014-07-30 14:18:48.776633 2014-08-13 17:38:02.629694 7fe9bae887a0 20 read_log 191727'374 (0'0) modify caadc1b8/benchmark_data_storage0111-ib.pocstor_51285_object7163/head//22 by client.275175.0:7164 2014-07-30 14:18:57.183927 2014-08-13 17:38:02.629715 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 2b4699b8/benchmark_data_storage0111-ib.pocstor_51285_object7806/head//22 by client.275175.0:7807 2014-07-30 14:18:58.984867 2014-08-13 17:38:02.629736 7fe9bae887a0 20 read_log 191727'377 (0'0) modify 76ab11b8/benchmark_data_storage0111-ib.pocstor_51285_object9639/head//22 by client.275175.0:9640 2014-07-30 14:19:04.178076 2014-08-13 17:38:02.629757 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 557d01b8/benchmark_data_storage0111-ib.pocstor_51285_object10400/head//22 by client.275175.0:10401 2014-07-30 14:19:06.354520 2014-08-13 17:38:02.629778 7fe9bae887a0 20 read_log 191727'380 (0'0) modify 4075b1b8/benchmark_data_storage0111-ib.pocstor_51285_object10661/head//22 by client.275175.0:10662 2014-07-30 14:19:07.097451 2014-08-13 17:38:02.629799 7fe9bae887a0 20 read_log 191727'381 (0'0) modify c80c1b8/benchmark_data_storage0111-ib.pocstor_51285_object11066/head//22 by client.275175.0:11067 2014-07-30 14:19:08.229162 2014-08-13 17:38:02.629856 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 5b0781b8/benchmark_data_storage0111-ib.pocstor_51285_object11101/head//22 by client.275175.0:11102 2014-07-30 14:19:08.328105 2014-08-13 17:38:02.629885 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 70bf71b8/benchmark_data_storage0111-ib.pocstor_51285_object12994/head//22 by client.275175.0:12995 2014-07-30 14:19:14.644235 2014-08-13 17:38:02.629906 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 12c599b8/benchmark_data_storage0111-ib.pocstor_51285_object16328/head//22 by client.275175.0:16329 2014-07-30 14:19:25.074944 2014-08-13 17:38:02.629927 7fe9bae887a0 20 read_log 191727'390 (0'0) modify e330d1b8/benchmark_data_storage0111-ib.pocstor_51285_object17896/head//22 by client.275175.0:17897 2014-07-30 14:19:29.665509 2014-08-13 17:38:02.629948 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 676db1b8/benchmark_data_storage0111-ib.pocstor_51285_object19875/head//22 by client.275175.0:19876 2014-07-30 14:19:35.820034 2014-08-13 17:38:02.629969 7fe9bae887a0 20 read_log 191727'398 (0'0) modify da7959b8/benchmark_data_storage0111-ib.pocstor_51285_object24072/head//22 by client.275175.0:24073 2014-07-30 14:19:49.036353 2014-08-13 17:38:02.629990 7fe9bae887a0 20 read_log 191727'402 (0'0) modify f25ec9b8/benchmark_data_storage0111-ib.pocstor_51285_object25508/head//22 by client.275175.0:25509 2014-07-30 14:19:53.284666 2014-08-13 17:38:02.630011 7fe9bae887a0 20 read_log 191727'409 (0'0) modify 8bbe29b8/benchmark_data_storage0111-ib.pocstor_51285_object29216/head//22 by client.275175.0:29217 2014-07-30 14:20:04.584603 2014-08-13 17:38:02.630032 7fe9bae887a0 20 read_log 191727'411 (0'0) modify ff9289b8/benchmark_data_storage0111-ib.pocstor_51285_object29649/head//22 by client.275175.0:29650 2014-07-30 14:20:05.908631 2014-08-13 17:38:02.630053 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 6298a9b8/benchmark_data_storage0111-ib.pocstor_51285_object34696/head//22 by client.275175.0:34697 2014-07-30 14:20:21.108304 2014-08-13 17:38:02.630074 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 358889b8/benchmark_data_storage0111-ib.pocstor_51285_object36189/head//22 by client.275175.0:36190 2014-07-30 14:20:25.879477 2014-08-13 17:38:02.630094 7fe9bae887a0 20 read_log 191727'423 (0'0) modify 3b29e9b8/benchmark_data_storage0111-ib.pocstor_51285_object37895/head//22 by client.275175.0:37896 2014-07-30 14:20:30.945923 2014-08-13 17:38:02.630115 7fe9bae887a0 20 read_log 191727'425 (0'0) modify c28c69b8/benchmark_data_storage0111-ib.pocstor_51285_object38627/head//22 by client.275175.0:38628 2014-07-30 14:20:33.032499 2014-08-13 17:38:02.630136 7fe9bae887a0 20 read_log 192888'426 (0'0) modify b76ac1b8/benchmark_data_storage0111-ib.pocstor_35625_object1026/head//22 by client.940330.0:1027 2014-08-11 16:29:44.888953 2014-08-13 17:38:02.630167 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 809961b8/benchmark_data_storage0111-ib.pocstor_35625_object1642/head//22 by client.940330.0:1643 2014-08-11 16:29:47.239078 2014-08-13 17:38:02.630190 7fe9bae887a0 20 read_log 192888'428 (0'0) modify aba971b8/benchmark_data_storage0111-ib.pocstor_35625_object12080/head//22 by client.940330.0:12081 2014-08-11 16:30:26.268715 2014-08-13 17:38:02.630211 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 6f0bb1b8/benchmark_data_storage0111-ib.pocstor_35625_object13391/head//22 by client.940330.0:13392 2014-08-11 16:30:31.384561 2014-08-13 17:38:02.630232 7fe9bae887a0 20 read_log 192888'430 (0'0) modify ac4411b8/benchmark_data_storage0111-ib.pocstor_35625_object19519/head//22 by client.940330.0:19520 2014-08-11 16:30:54.825002 2014-08-13 17:38:02.630252 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 2b7e99b8/benchmark_data_storage0111-ib.pocstor_35625_object20876/head//22 by client.940330.0:20877 2014-08-11 16:31:00.231789 2014-08-13 17:38:02.630286 7fe9bae887a0 20 read_log 192888'432 (0'0) modify ad29b8/benchmark_data_storage0111-ib.pocstor_35625_object21963/head//22 by client.940330.0:21964 2014-08-11 16:31:04.430730 2014-08-13 17:38:02.630313 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 354e31b8/benchmark_data_storage0111-ib.pocstor_35625_object22084/head//22 by client.940330.0:22085 2014-08-11 16:31:04.888418 2014-08-13 17:38:02.630337 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 3f6c51b8/benchmark_data_storage0111-ib.pocstor_35625_object23682/head//22 by client.940330.0:23683 2014-08-11 16:31:10.979217 2014-08-13 17:38:02.630358 7fe9bae887a0 20 read_log 192888'435 (0'0) modify b3c819b8/benchmark_data_storage0111-ib.pocstor_35625_object25353/head//22 by client.940330.0:25354 2014-08-11 16:31:17.267199 2014-08-13 17:38:02.630378 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 737ab9b8/benchmark_data_storage0111-ib.pocstor_35625_object27066/head//22 by client.940330.0:27067 2014-08-11 16:31:24.059526 2014-08-13 17:38:02.630399 7fe9bae887a0 20 read_log 192888'437 (192888'426) delete b76ac1b8/benchmark_data_storage0111-ib.pocstor_35625_object1026/head//22 by client.940330.0:32334 2014-08-11 16:31:41.755100 2014-08-13 17:38:02.630421 7fe9bae887a0 20 read_log 192888'438 (192888'427) delete 809961b8/benchmark_data_storage0111-ib.pocstor_35625_object1642/head//22 by client.940330.0:32950 2014-08-11 16:31:41.900893 2014-08-13 17:38:02.630442 7fe9bae887a0 20 read_log 192888'439 (192888'428) delete aba971b8/benchmark_data_storage0111-ib.pocstor_35625_object12080/head//22 by client.940330.0:43388 2014-08-11 16:31:44.678392 2014-08-13 17:38:02.630464 7fe9bae887a0 20 read_log 192888'440 (192888'429) delete 6f0bb1b8/benchmark_data_storage0111-ib.pocstor_35625_object13391/head//22 by client.940330.0:44699 2014-08-11 16:31:45.028674 2014-08-13 17:38:02.630486 7fe9bae887a0 20 read_log 192888'441 (192888'430) delete ac4411b8/benchmark_data_storage0111-ib.pocstor_35625_object19519/head//22 by client.940330.0:50827 2014-08-11 16:31:46.725415 2014-08-13 17:38:02.630507 7fe9bae887a0 20 read_log 192888'442 (192888'431) delete 2b7e99b8/benchmark_data_storage0111-ib.pocstor_35625_object20876/head//22 by client.940330.0:52184 2014-08-11 16:31:47.127125 2014-08-13 17:38:02.630528 7fe9bae887a0 20 read_log 192888'443 (192888'432) delete ad29b8/benchmark_data_storage0111-ib.pocstor_35625_object21963/head//22 by client.940330.0:53271 2014-08-11 16:31:47.434894 2014-08-13 17:38:02.630550 7fe9bae887a0 20 read_log 192888'444 (192888'433) delete 354e31b8/benchmark_data_storage0111-ib.pocstor_35625_object22084/head//22 by client.940330.0:53392 2014-08-11 16:31:47.467808 2014-08-13 17:38:02.630571 7fe9bae887a0 20 read_log 192888'445 (192888'434) delete 3f6c51b8/benchmark_data_storage0111-ib.pocstor_35625_object23682/head//22 by client.940330.0:54990 2014-08-11 16:31:47.945650 2014-08-13 17:38:02.630592 7fe9bae887a0 20 read_log 192888'446 (192888'435) delete b3c819b8/benchmark_data_storage0111-ib.pocstor_35625_object25353/head//22 by client.940330.0:56661 2014-08-11 16:31:48.455759 2014-08-13 17:38:02.630613 7fe9bae887a0 20 read_log 192888'447 (192888'436) delete 737ab9b8/benchmark_data_storage0111-ib.pocstor_35625_object27066/head//22 by client.940330.0:58374 2014-08-11 16:31:48.944948 2014-08-13 17:38:02.630634 7fe9bae887a0 20 read_log 194215'448 (0'0) modify f30ba1b8/benchmark_data_storage0111-ib.pocstor_64498_object5394/head//22 by client.1049635.0:5395 2014-08-12 17:17:34.146143 2014-08-13 17:38:02.630655 7fe9bae887a0 20 read_log 194215'449 (0'0) modify 271989b8/benchmark_data_storage0111-ib.pocstor_64498_object6054/head//22 by client.1049635.0:6055 2014-08-12 17:17:36.235998 2014-08-13 17:38:02.630676 7fe9bae887a0 20 read_log 194215'450 (0'0) modify 4d3b79b8/benchmark_data_storage0111-ib.pocstor_64498_object9577/head//22 by client.1049635.0:9578 2014-08-12 17:17:46.723113 2014-08-13 17:38:02.630732 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 3a2449b8/benchmark_data_storage0111-ib.pocstor_64498_object9678/head//22 by client.1049635.0:9679 2014-08-12 17:17:47.008518 2014-08-13 17:38:02.630772 7fe9bae887a0 20 read_log 194215'452 (0'0) modify 1a5079b8/benchmark_data_storage0111-ib.pocstor_64498_object9933/head//22 by client.1049635.0:9934 2014-08-12 17:17:47.733902 2014-08-13 17:38:02.630799 7fe9bae887a0 20 read_log 194215'453 (0'0) modify 33d451b8/benchmark_data_storage0111-ib.pocstor_64498_object13096/head//22 by client.1049635.0:13097 2014-08-12 17:17:57.666365 2014-08-13 17:38:02.630820 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 6882e1b8/benchmark_data_storage0111-ib.pocstor_64498_object14280/head//22 by client.1049635.0:14281 2014-08-12 17:18:01.445569 2014-08-13 17:38:02.630842 7fe9bae887a0 20 read_log 194215'455 (0'0) modify 1b25c1b8/benchmark_data_storage0111-ib.pocstor_64498_object14315/head//22 by client.1049635.0:14316 2014-08-12 17:18:01.599549 2014-08-13 17:38:02.630863 7fe9bae887a0 20 read_log 194215'456 (0'0) modify c20819b8/benchmark_data_storage0111-ib.pocstor_64498_object16050/head//22 by client.1049635.0:16051 2014-08-12 17:18:07.738220 2014-08-13 17:38:02.630884 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 1957a9b8/benchmark_data_storage0111-ib.pocstor_64498_object17244/head//22 by client.1049635.0:17245 2014-08-12 17:18:12.297996 2014-08-13 17:38:02.630905 7fe9bae887a0 20 read_log 194215'458 (0'0) modify f0e649b8/benchmark_data_storage0111-ib.pocstor_64498_object19542/head//22 by client.1049635.0:19543 2014-08-12 17:18:21.065537 2014-08-13 17:38:02.630927 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 5ce6e1b8/benchmark_data_storage0111-ib.pocstor_64498_object20366/head//22 by client.1049635.0:20367 2014-08-12 17:18:24.169038 2014-08-13 17:38:02.630948 7fe9bae887a0 20 read_log 194215'460 (0'0) modify aad9b9b8/benchmark_data_storage0111-ib.pocstor_64498_object21736/head//22 by client.1049635.0:21737 2014-08-12 17:18:29.339374 2014-08-13 17:38:02.630969 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 4f2061b8/benchmark_data_storage0111-ib.pocstor_64498_object25898/head//22 by client.1049635.0:25899 2014-08-12 17:18:45.042049 2014-08-13 17:38:02.630990 7fe9bae887a0 20 read_log 194215'462 (0'0) modify e4f079b8/benchmark_data_storage0111-ib.pocstor_64498_object29203/head//22 by client.1049635.0:29204 2014-08-12 17:18:57.348701 2014-08-13 17:38:02.631011 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 9b2ac9b8/benchmark_data_storage0111-ib.pocstor_64498_object29565/head//22 by client.1049635.0:29566 2014-08-12 17:18:58.816754 2014-08-13 17:38:02.631032 7fe9bae887a0 20 read_log 194215'464 (194215'448) delete f30ba1b8/benchmark_data_storage0111-ib.pocstor_64498_object5394/head//22 by client.1049635.0:39597 2014-08-12 17:19:19.735444 2014-08-13 17:38:02.631056 7fe9bae887a0 20 read_log 194215'465 (194215'449) delete 271989b8/benchmark_data_storage0111-ib.pocstor_64498_object6054/head//22 by client.1049635.0:40257 2014-08-12 17:19:19.973959 2014-08-13 17:38:02.631078 7fe9bae887a0 20 read_log 194215'466 (194215'450) delete 4d3b79b8/benchmark_data_storage0111-ib.pocstor_64498_object9577/head//22 by client.1049635.0:43780 2014-08-12 17:19:21.113480 2014-08-13 17:38:02.631099 7fe9bae887a0 20 read_log 194215'467 (194215'451) delete 3a2449b8/benchmark_data_storage0111-ib.pocstor_64498_object9678/head//22 by client.1049635.0:43881 2014-08-12 17:19:21.148094 2014-08-13 17:38:02.631121 7fe9bae887a0 20 read_log 194215'468 (194215'452) delete 1a5079b8/benchmark_data_storage0111-ib.pocstor_64498_object9933/head//22 by client.1049635.0:44136 2014-08-12 17:19:21.238998 2014-08-13 17:38:02.631142 7fe9bae887a0 20 read_log 194215'469 (194215'453) delete 33d451b8/benchmark_data_storage0111-ib.pocstor_64498_object13096/head//22 by client.1049635.0:47299 2014-08-12 17:19:22.253610 2014-08-13 17:38:02.631177 7fe9bae887a0 20 read_log 194215'470 (194215'454) delete 6882e1b8/benchmark_data_storage0111-ib.pocstor_64498_object14280/head//22 by client.1049635.0:48483 2014-08-12 17:19:22.652556 2014-08-13 17:38:02.631200 7fe9bae887a0 20 read_log 194215'471 (194215'455) delete 1b25c1b8/benchmark_data_storage0111-ib.pocstor_64498_object14315/head//22 by client.1049635.0:48518 2014-08-12 17:19:22.664758 2014-08-13 17:38:02.631223 7fe9bae887a0 20 read_log 194215'472 (194215'456) delete c20819b8/benchmark_data_storage0111-ib.pocstor_64498_object16050/head//22 by client.1049635.0:50253 2014-08-12 17:19:23.205316 2014-08-13 17:38:02.631245 7fe9bae887a0 20 read_log 194215'473 (194215'457) delete 1957a9b8/benchmark_data_storage0111-ib.pocstor_64498_object17244/head//22 by client.1049635.0:51447 2014-08-12 17:19:23.582695 2014-08-13 17:38:02.631266 7fe9bae887a0 20 read_log 194215'474 (194215'458) delete f0e649b8/benchmark_data_storage0111-ib.pocstor_64498_object19542/head//22 by client.1049635.0:53745 2014-08-12 17:19:24.364664 2014-08-13 17:38:02.631287 7fe9bae887a0 20 read_log 194215'475 (194215'459) delete 5ce6e1b8/benchmark_data_storage0111-ib.pocstor_64498_object20366/head//22 by client.1049635.0:54569 2014-08-12 17:19:24.656346 2014-08-13 17:38:02.631308 7fe9bae887a0 20 read_log 194215'476 (194215'460) delete aad9b9b8/benchmark_data_storage0111-ib.pocstor_64498_object21736/head//22 by client.1049635.0:55939 2014-08-12 17:19:25.138994 2014-08-13 17:38:02.631329 7fe9bae887a0 20 read_log 194215'477 (194215'461) delete 4f2061b8/benchmark_data_storage0111-ib.pocstor_64498_object25898/head//22 by client.1049635.0:60101 2014-08-12 17:19:26.624223 2014-08-13 17:38:02.631349 7fe9bae887a0 20 read_log 194215'478 (194215'462) delete e4f079b8/benchmark_data_storage0111-ib.pocstor_64498_object29203/head//22 by client.1049635.0:63406 2014-08-12 17:19:27.778813 2014-08-13 17:38:02.631370 7fe9bae887a0 20 read_log 194215'479 (194215'463) delete 9b2ac9b8/benchmark_data_storage0111-ib.pocstor_64498_object29565/head//22 by client.1049635.0:63768 2014-08-12 17:19:27.907566 2014-08-13 17:38:02.631501 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.631541 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.631567 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002667 0 0.000000 2014-08-13 17:38:02.631586 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.631600 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.631614 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'479], crt=191648'362) 2014-08-13 17:38:02.631630 7fe9bae887a0 10 osd.4 194478 pgid 22.1d2 coll 22.1d2_head 2014-08-13 17:38:02.631831 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.1d2 2014-08-13 17:38:02.631848 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.631908 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1d2(unlocked)] enter Initial 2014-08-13 17:38:02.631928 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1d2(unlocked)] enter NotTrimming 2014-08-13 17:38:02.632134 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.632353 7fe9bae887a0 20 read_log 191648'55 (0'0) modify 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 by client.273158.0:3470 2014-07-30 14:09:03.578797 2014-08-13 17:38:02.632389 7fe9bae887a0 20 read_log 191648'82 (0'0) modify cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 by client.273158.0:4875 2014-07-30 14:09:10.421448 2014-08-13 17:38:02.632411 7fe9bae887a0 20 read_log 191648'119 (0'0) modify 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 by client.273158.0:7271 2014-07-30 14:09:22.157475 2014-08-13 17:38:02.632432 7fe9bae887a0 20 read_log 191648'140 (0'0) modify f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 by client.273158.0:8766 2014-07-30 14:09:30.247316 2014-08-13 17:38:02.632453 7fe9bae887a0 20 read_log 191648'176 (0'0) modify d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 by client.273158.0:11133 2014-07-30 14:09:42.819988 2014-08-13 17:38:02.632474 7fe9bae887a0 20 read_log 191648'183 (0'0) modify c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 by client.273158.0:11613 2014-07-30 14:09:45.258056 2014-08-13 17:38:02.632495 7fe9bae887a0 20 read_log 191648'208 (0'0) modify db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 by client.273158.0:12899 2014-07-30 14:09:51.928276 2014-08-13 17:38:02.632517 7fe9bae887a0 20 read_log 191648'231 (0'0) modify 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 by client.273158.0:14393 2014-07-30 14:09:59.636690 2014-08-13 17:38:02.632538 7fe9bae887a0 20 read_log 191648'246 (0'0) modify f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 by client.273158.0:15322 2014-07-30 14:10:03.958818 2014-08-13 17:38:02.632558 7fe9bae887a0 20 read_log 191648'265 (0'0) modify a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 by client.273158.0:16470 2014-07-30 14:10:10.039712 2014-08-13 17:38:02.632579 7fe9bae887a0 20 read_log 191648'294 (0'0) modify 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 by client.273158.0:18082 2014-07-30 14:10:18.452616 2014-08-13 17:38:02.632600 7fe9bae887a0 20 read_log 191648'297 (0'0) modify 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 by client.273158.0:18255 2014-07-30 14:10:19.335978 2014-08-13 17:38:02.632620 7fe9bae887a0 20 read_log 191648'332 (0'0) modify 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 by client.273158.0:20895 2014-07-30 14:10:36.449402 2014-08-13 17:38:02.632641 7fe9bae887a0 20 read_log 191648'345 (0'0) modify d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 by client.273158.0:21464 2014-07-30 14:10:39.332678 2014-08-13 17:38:02.632661 7fe9bae887a0 20 read_log 191727'377 (0'0) modify a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 by client.275175.0:1832 2014-07-30 14:18:40.346523 2014-08-13 17:38:02.632682 7fe9bae887a0 20 read_log 191727'378 (0'0) modify e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 by client.275175.0:1837 2014-07-30 14:18:40.364150 2014-08-13 17:38:02.632703 7fe9bae887a0 20 read_log 191727'382 (0'0) modify f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 by client.275175.0:5230 2014-07-30 14:18:51.028490 2014-08-13 17:38:02.632724 7fe9bae887a0 20 read_log 191727'383 (0'0) modify b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 by client.275175.0:5426 2014-07-30 14:18:51.599515 2014-08-13 17:38:02.632745 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 by client.275175.0:7723 2014-07-30 14:18:58.754197 2014-08-13 17:38:02.632779 7fe9bae887a0 20 read_log 191727'390 (0'0) modify 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 by client.275175.0:8994 2014-07-30 14:19:02.329439 2014-08-13 17:38:02.632801 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 by client.275175.0:9727 2014-07-30 14:19:04.417396 2014-08-13 17:38:02.632822 7fe9bae887a0 20 read_log 191727'401 (0'0) modify 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 by client.275175.0:15557 2014-07-30 14:19:22.575913 2014-08-13 17:38:02.632842 7fe9bae887a0 20 read_log 191727'415 (0'0) modify ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 by client.275175.0:24055 2014-07-30 14:19:48.986063 2014-08-13 17:38:02.632863 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 by client.275175.0:25855 2014-07-30 14:19:54.305346 2014-08-13 17:38:02.632887 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 by client.275175.0:26718 2014-07-30 14:19:56.966150 2014-08-13 17:38:02.632908 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 by client.275175.0:31402 2014-07-30 14:20:11.118127 2014-08-13 17:38:02.632929 7fe9bae887a0 20 read_log 191727'435 (0'0) modify 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 by client.275175.0:33452 2014-07-30 14:20:17.262020 2014-08-13 17:38:02.632949 7fe9bae887a0 20 read_log 191727'445 (0'0) modify 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 by client.275175.0:37148 2014-07-30 14:20:28.785511 2014-08-13 17:38:02.632975 7fe9bae887a0 20 read_log 192888'447 (0'0) modify fcbd01d2/benchmark_data_storage0111-ib.pocstor_35625_object497/head//22 by client.940330.0:498 2014-08-11 16:29:42.712906 2014-08-13 17:38:02.632996 7fe9bae887a0 20 read_log 192888'448 (0'0) modify e381d1d2/benchmark_data_storage0111-ib.pocstor_35625_object1349/head//22 by client.940330.0:1350 2014-08-11 16:29:46.094608 2014-08-13 17:38:02.633017 7fe9bae887a0 20 read_log 192888'449 (0'0) modify bf70f9d2/benchmark_data_storage0111-ib.pocstor_35625_object1562/head//22 by client.940330.0:1563 2014-08-11 16:29:46.935283 2014-08-13 17:38:02.633038 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 24da71d2/benchmark_data_storage0111-ib.pocstor_35625_object2329/head//22 by client.940330.0:2330 2014-08-11 16:29:49.744395 2014-08-13 17:38:02.633058 7fe9bae887a0 20 read_log 192888'451 (0'0) modify eeb259d2/benchmark_data_storage0111-ib.pocstor_35625_object4607/head//22 by client.940330.0:4608 2014-08-11 16:29:58.646409 2014-08-13 17:38:02.633081 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 91c09d2/benchmark_data_storage0111-ib.pocstor_35625_object7055/head//22 by client.940330.0:7056 2014-08-11 16:30:07.512600 2014-08-13 17:38:02.633102 7fe9bae887a0 20 read_log 192888'453 (0'0) modify d07a49d2/benchmark_data_storage0111-ib.pocstor_35625_object11324/head//22 by client.940330.0:11325 2014-08-11 16:30:23.111637 2014-08-13 17:38:02.633122 7fe9bae887a0 20 read_log 192888'454 (0'0) modify f8b119d2/benchmark_data_storage0111-ib.pocstor_35625_object12245/head//22 by client.940330.0:12246 2014-08-11 16:30:26.929677 2014-08-13 17:38:02.633143 7fe9bae887a0 20 read_log 192888'455 (0'0) modify f02ea9d2/benchmark_data_storage0111-ib.pocstor_35625_object12699/head//22 by client.940330.0:12700 2014-08-11 16:30:28.635629 2014-08-13 17:38:02.633174 7fe9bae887a0 20 read_log 192888'456 (0'0) modify b8ea29d2/benchmark_data_storage0111-ib.pocstor_35625_object13182/head//22 by client.940330.0:13183 2014-08-11 16:30:30.580365 2014-08-13 17:38:02.633209 7fe9bae887a0 20 read_log 192888'457 (0'0) modify cef5b1d2/benchmark_data_storage0111-ib.pocstor_35625_object14545/head//22 by client.940330.0:14546 2014-08-11 16:30:35.798230 2014-08-13 17:38:02.633236 7fe9bae887a0 20 read_log 192888'458 (0'0) modify aa71c1d2/benchmark_data_storage0111-ib.pocstor_35625_object14881/head//22 by client.940330.0:14882 2014-08-11 16:30:37.049651 2014-08-13 17:38:02.633257 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 9caf81d2/benchmark_data_storage0111-ib.pocstor_35625_object21727/head//22 by client.940330.0:21728 2014-08-11 16:31:03.511077 2014-08-13 17:38:02.633278 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 803b51d2/benchmark_data_storage0111-ib.pocstor_35625_object25016/head//22 by client.940330.0:25017 2014-08-11 16:31:15.945599 2014-08-13 17:38:02.633303 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 97c009d2/benchmark_data_storage0111-ib.pocstor_35625_object25699/head//22 by client.940330.0:25700 2014-08-11 16:31:18.609937 2014-08-13 17:38:02.633327 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 59c19d2/benchmark_data_storage0111-ib.pocstor_35625_object25956/head//22 by client.940330.0:25957 2014-08-11 16:31:19.621437 2014-08-13 17:38:02.633348 7fe9bae887a0 20 read_log 192888'463 (0'0) modify dc3401d2/benchmark_data_storage0111-ib.pocstor_35625_object26482/head//22 by client.940330.0:26483 2014-08-11 16:31:21.715035 2014-08-13 17:38:02.633369 7fe9bae887a0 20 read_log 192888'464 (0'0) modify c2faa1d2/benchmark_data_storage0111-ib.pocstor_35625_object27601/head//22 by client.940330.0:27602 2014-08-11 16:31:26.136550 2014-08-13 17:38:02.633389 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 3f0d59d2/benchmark_data_storage0111-ib.pocstor_35625_object30310/head//22 by client.940330.0:30311 2014-08-11 16:31:36.633627 2014-08-13 17:38:02.633410 7fe9bae887a0 20 read_log 192888'466 (192888'447) delete fcbd01d2/benchmark_data_storage0111-ib.pocstor_35625_object497/head//22 by client.940330.0:31805 2014-08-11 16:31:41.620514 2014-08-13 17:38:02.633432 7fe9bae887a0 20 read_log 192888'467 (192888'448) delete e381d1d2/benchmark_data_storage0111-ib.pocstor_35625_object1349/head//22 by client.940330.0:32657 2014-08-11 16:31:41.832125 2014-08-13 17:38:02.633453 7fe9bae887a0 20 read_log 192888'468 (192888'449) delete bf70f9d2/benchmark_data_storage0111-ib.pocstor_35625_object1562/head//22 by client.940330.0:32870 2014-08-11 16:31:41.882208 2014-08-13 17:38:02.633475 7fe9bae887a0 20 read_log 192888'469 (192888'450) delete 24da71d2/benchmark_data_storage0111-ib.pocstor_35625_object2329/head//22 by client.940330.0:33637 2014-08-11 16:31:42.076153 2014-08-13 17:38:02.633502 7fe9bae887a0 20 read_log 192888'470 (192888'451) delete eeb259d2/benchmark_data_storage0111-ib.pocstor_35625_object4607/head//22 by client.940330.0:35915 2014-08-11 16:31:42.710660 2014-08-13 17:38:02.633524 7fe9bae887a0 20 read_log 192888'471 (192888'452) delete 91c09d2/benchmark_data_storage0111-ib.pocstor_35625_object7055/head//22 by client.940330.0:38363 2014-08-11 16:31:43.349072 2014-08-13 17:38:02.633545 7fe9bae887a0 20 read_log 192888'472 (192888'453) delete d07a49d2/benchmark_data_storage0111-ib.pocstor_35625_object11324/head//22 by client.940330.0:42632 2014-08-11 16:31:44.493637 2014-08-13 17:38:02.633567 7fe9bae887a0 20 read_log 192888'473 (192888'454) delete f8b119d2/benchmark_data_storage0111-ib.pocstor_35625_object12245/head//22 by client.940330.0:43553 2014-08-11 16:31:44.718601 2014-08-13 17:38:02.633588 7fe9bae887a0 20 read_log 192888'474 (192888'455) delete f02ea9d2/benchmark_data_storage0111-ib.pocstor_35625_object12699/head//22 by client.940330.0:44007 2014-08-11 16:31:44.835154 2014-08-13 17:38:02.633609 7fe9bae887a0 20 read_log 192888'475 (192888'456) delete b8ea29d2/benchmark_data_storage0111-ib.pocstor_35625_object13182/head//22 by client.940330.0:44490 2014-08-11 16:31:44.969645 2014-08-13 17:38:02.633675 7fe9bae887a0 20 read_log 192888'476 (192888'457) delete cef5b1d2/benchmark_data_storage0111-ib.pocstor_35625_object14545/head//22 by client.940330.0:45853 2014-08-11 16:31:45.329587 2014-08-13 17:38:02.633711 7fe9bae887a0 20 read_log 192888'477 (192888'458) delete aa71c1d2/benchmark_data_storage0111-ib.pocstor_35625_object14881/head//22 by client.940330.0:46189 2014-08-11 16:31:45.423893 2014-08-13 17:38:02.633733 7fe9bae887a0 20 read_log 192888'478 (192888'459) delete 9caf81d2/benchmark_data_storage0111-ib.pocstor_35625_object21727/head//22 by client.940330.0:53035 2014-08-11 16:31:47.360711 2014-08-13 17:38:02.633757 7fe9bae887a0 20 read_log 192888'479 (192888'460) delete 803b51d2/benchmark_data_storage0111-ib.pocstor_35625_object25016/head//22 by client.940330.0:56324 2014-08-11 16:31:48.345550 2014-08-13 17:38:02.633778 7fe9bae887a0 20 read_log 192888'480 (192888'461) delete 97c009d2/benchmark_data_storage0111-ib.pocstor_35625_object25699/head//22 by client.940330.0:57007 2014-08-11 16:31:48.559287 2014-08-13 17:38:02.633800 7fe9bae887a0 20 read_log 192888'481 (192888'462) delete 59c19d2/benchmark_data_storage0111-ib.pocstor_35625_object25956/head//22 by client.940330.0:57264 2014-08-11 16:31:48.635211 2014-08-13 17:38:02.633846 7fe9bae887a0 20 read_log 192888'482 (192888'463) delete dc3401d2/benchmark_data_storage0111-ib.pocstor_35625_object26482/head//22 by client.940330.0:57790 2014-08-11 16:31:48.786918 2014-08-13 17:38:02.633872 7fe9bae887a0 20 read_log 192888'483 (192888'464) delete c2faa1d2/benchmark_data_storage0111-ib.pocstor_35625_object27601/head//22 by client.940330.0:58909 2014-08-11 16:31:49.094411 2014-08-13 17:38:02.633894 7fe9bae887a0 20 read_log 192888'484 (192888'465) delete 3f0d59d2/benchmark_data_storage0111-ib.pocstor_35625_object30310/head//22 by client.940330.0:61618 2014-08-11 16:31:49.847240 2014-08-13 17:38:02.633915 7fe9bae887a0 20 read_log 194215'485 (0'0) modify f95681d2/benchmark_data_storage0111-ib.pocstor_64498_object2601/head//22 by client.1049635.0:2602 2014-08-12 17:17:25.540184 2014-08-13 17:38:02.633936 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 9f8571d2/benchmark_data_storage0111-ib.pocstor_64498_object6629/head//22 by client.1049635.0:6630 2014-08-12 17:17:37.943883 2014-08-13 17:38:02.633957 7fe9bae887a0 20 read_log 194215'487 (0'0) modify cd39a1d2/benchmark_data_storage0111-ib.pocstor_64498_object9295/head//22 by client.1049635.0:9296 2014-08-12 17:17:45.876471 2014-08-13 17:38:02.633978 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 90e6f1d2/benchmark_data_storage0111-ib.pocstor_64498_object17081/head//22 by client.1049635.0:17082 2014-08-12 17:18:11.671638 2014-08-13 17:38:02.633999 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 49eb19d2/benchmark_data_storage0111-ib.pocstor_64498_object19560/head//22 by client.1049635.0:19561 2014-08-12 17:18:21.131607 2014-08-13 17:38:02.634021 7fe9bae887a0 20 read_log 194215'490 (0'0) modify da00a1d2/benchmark_data_storage0111-ib.pocstor_64498_object20336/head//22 by client.1049635.0:20337 2014-08-12 17:18:24.057806 2014-08-13 17:38:02.634042 7fe9bae887a0 20 read_log 194215'491 (0'0) modify abb989d2/benchmark_data_storage0111-ib.pocstor_64498_object20369/head//22 by client.1049635.0:20370 2014-08-12 17:18:24.179605 2014-08-13 17:38:02.634063 7fe9bae887a0 20 read_log 194215'492 (0'0) modify 4edaf1d2/benchmark_data_storage0111-ib.pocstor_64498_object23643/head//22 by client.1049635.0:23644 2014-08-12 17:18:36.527064 2014-08-13 17:38:02.634085 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 5be11d2/benchmark_data_storage0111-ib.pocstor_64498_object25489/head//22 by client.1049635.0:25490 2014-08-12 17:18:43.523704 2014-08-13 17:38:02.634106 7fe9bae887a0 20 read_log 194215'494 (0'0) modify f7139d2/benchmark_data_storage0111-ib.pocstor_64498_object29946/head//22 by client.1049635.0:29947 2014-08-12 17:19:00.245924 2014-08-13 17:38:02.634127 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 5d4e11d2/benchmark_data_storage0111-ib.pocstor_64498_object31153/head//22 by client.1049635.0:31154 2014-08-12 17:19:05.155529 2014-08-13 17:38:02.634151 7fe9bae887a0 20 read_log 194215'496 (0'0) modify df0179d2/benchmark_data_storage0111-ib.pocstor_64498_object33542/head//22 by client.1049635.0:33543 2014-08-12 17:19:14.442093 2014-08-13 17:38:02.634186 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 13441d2/benchmark_data_storage0111-ib.pocstor_64498_object33985/head//22 by client.1049635.0:33986 2014-08-12 17:19:16.123634 2014-08-13 17:38:02.634208 7fe9bae887a0 20 read_log 194215'498 (194215'485) delete f95681d2/benchmark_data_storage0111-ib.pocstor_64498_object2601/head//22 by client.1049635.0:36804 2014-08-12 17:19:18.831807 2014-08-13 17:38:02.634229 7fe9bae887a0 20 read_log 194215'499 (194215'486) delete 9f8571d2/benchmark_data_storage0111-ib.pocstor_64498_object6629/head//22 by client.1049635.0:40832 2014-08-12 17:19:20.158876 2014-08-13 17:38:02.634251 7fe9bae887a0 20 read_log 194215'500 (194215'487) delete cd39a1d2/benchmark_data_storage0111-ib.pocstor_64498_object9295/head//22 by client.1049635.0:43498 2014-08-12 17:19:21.023468 2014-08-13 17:38:02.634272 7fe9bae887a0 20 read_log 194215'501 (194215'488) delete 90e6f1d2/benchmark_data_storage0111-ib.pocstor_64498_object17081/head//22 by client.1049635.0:51284 2014-08-12 17:19:23.533305 2014-08-13 17:38:02.634297 7fe9bae887a0 20 read_log 194215'502 (194215'489) delete 49eb19d2/benchmark_data_storage0111-ib.pocstor_64498_object19560/head//22 by client.1049635.0:53763 2014-08-12 17:19:24.371102 2014-08-13 17:38:02.634319 7fe9bae887a0 20 read_log 194215'503 (194215'490) delete da00a1d2/benchmark_data_storage0111-ib.pocstor_64498_object20336/head//22 by client.1049635.0:54539 2014-08-12 17:19:24.643201 2014-08-13 17:38:02.634340 7fe9bae887a0 20 read_log 194215'504 (194215'491) delete abb989d2/benchmark_data_storage0111-ib.pocstor_64498_object20369/head//22 by client.1049635.0:54572 2014-08-12 17:19:24.657249 2014-08-13 17:38:02.634361 7fe9bae887a0 20 read_log 194215'505 (194215'492) delete 4edaf1d2/benchmark_data_storage0111-ib.pocstor_64498_object23643/head//22 by client.1049635.0:57846 2014-08-12 17:19:25.846737 2014-08-13 17:38:02.634382 7fe9bae887a0 20 read_log 194215'506 (194215'493) delete 5be11d2/benchmark_data_storage0111-ib.pocstor_64498_object25489/head//22 by client.1049635.0:59692 2014-08-12 17:19:26.480900 2014-08-13 17:38:02.634403 7fe9bae887a0 20 read_log 194215'507 (194215'494) delete f7139d2/benchmark_data_storage0111-ib.pocstor_64498_object29946/head//22 by client.1049635.0:64149 2014-08-12 17:19:28.029913 2014-08-13 17:38:02.634424 7fe9bae887a0 20 read_log 194215'508 (194215'495) delete 5d4e11d2/benchmark_data_storage0111-ib.pocstor_64498_object31153/head//22 by client.1049635.0:65356 2014-08-12 17:19:28.432108 2014-08-13 17:38:02.634445 7fe9bae887a0 20 read_log 194215'509 (194215'496) delete df0179d2/benchmark_data_storage0111-ib.pocstor_64498_object33542/head//22 by client.1049635.0:67745 2014-08-12 17:19:29.308447 2014-08-13 17:38:02.634468 7fe9bae887a0 20 read_log 194215'510 (194215'497) delete 13441d2/benchmark_data_storage0111-ib.pocstor_64498_object33985/head//22 by client.1049635.0:68188 2014-08-12 17:19:29.475150 2014-08-13 17:38:02.634601 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.634633 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=0 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.634660 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=0 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002751 0 0.000000 2014-08-13 17:38:02.634680 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=0 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.634694 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=0 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.634712 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'510], crt=191648'372) 2014-08-13 17:38:02.634729 7fe9bae887a0 10 osd.4 194478 pgid 22.1f1 coll 22.1f1_head 2014-08-13 17:38:02.634915 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.1f1 2014-08-13 17:38:02.634931 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.634976 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.1f1(unlocked)] enter Initial 2014-08-13 17:38:02.634995 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.1f1(unlocked)] enter NotTrimming 2014-08-13 17:38:02.635219 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.635433 7fe9bae887a0 20 read_log 191648'8 (0'0) modify 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 by client.273158.0:441 2014-07-30 14:08:50.431491 2014-08-13 17:38:02.635468 7fe9bae887a0 20 read_log 191648'10 (0'0) modify f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 by client.273158.0:593 2014-07-30 14:08:50.988411 2014-08-13 17:38:02.635491 7fe9bae887a0 20 read_log 191648'16 (0'0) modify c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 by client.273158.0:1144 2014-07-30 14:08:53.004008 2014-08-13 17:38:02.635513 7fe9bae887a0 20 read_log 191648'25 (0'0) modify 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 by client.273158.0:1583 2014-07-30 14:08:54.598458 2014-08-13 17:38:02.635534 7fe9bae887a0 20 read_log 191648'64 (0'0) modify 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 by client.273158.0:4386 2014-07-30 14:09:08.293062 2014-08-13 17:38:02.635555 7fe9bae887a0 20 read_log 191648'68 (0'0) modify a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 by client.273158.0:4725 2014-07-30 14:09:09.714701 2014-08-13 17:38:02.635576 7fe9bae887a0 20 read_log 191648'80 (0'0) modify 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 by client.273158.0:5479 2014-07-30 14:09:13.350131 2014-08-13 17:38:02.635598 7fe9bae887a0 20 read_log 191648'109 (0'0) modify a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 by client.273158.0:7623 2014-07-30 14:09:23.969602 2014-08-13 17:38:02.635618 7fe9bae887a0 20 read_log 191648'127 (0'0) modify d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 by client.273158.0:8471 2014-07-30 14:09:28.734832 2014-08-13 17:38:02.635639 7fe9bae887a0 20 read_log 191648'167 (0'0) modify bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 by client.273158.0:10846 2014-07-30 14:09:40.610335 2014-08-13 17:38:02.635660 7fe9bae887a0 20 read_log 191648'199 (0'0) modify d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 by client.273158.0:13343 2014-07-30 14:09:53.998432 2014-08-13 17:38:02.635682 7fe9bae887a0 20 read_log 191648'200 (0'0) modify 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 by client.273158.0:13425 2014-07-30 14:09:54.405244 2014-08-13 17:38:02.635702 7fe9bae887a0 20 read_log 191648'206 (0'0) modify c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 by client.273158.0:13680 2014-07-30 14:09:55.478228 2014-08-13 17:38:02.635723 7fe9bae887a0 20 read_log 191648'220 (0'0) modify 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 by client.273158.0:14805 2014-07-30 14:10:01.920444 2014-08-13 17:38:02.635745 7fe9bae887a0 20 read_log 191648'345 (0'0) modify 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 by client.273158.0:22731 2014-07-30 14:10:47.990653 2014-08-13 17:38:02.635766 7fe9bae887a0 20 read_log 191727'353 (0'0) modify 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 by client.275175.0:5300 2014-07-30 14:18:51.233571 2014-08-13 17:38:02.635787 7fe9bae887a0 20 read_log 191727'354 (0'0) modify 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 by client.275175.0:5871 2014-07-30 14:18:52.876107 2014-08-13 17:38:02.635857 7fe9bae887a0 20 read_log 191727'355 (0'0) modify ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 by client.275175.0:6315 2014-07-30 14:18:54.193940 2014-08-13 17:38:02.635886 7fe9bae887a0 20 read_log 191727'365 (0'0) modify 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 by client.275175.0:10384 2014-07-30 14:19:06.306709 2014-08-13 17:38:02.635908 7fe9bae887a0 20 read_log 191727'366 (0'0) modify d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 by client.275175.0:11576 2014-07-30 14:19:09.978218 2014-08-13 17:38:02.635929 7fe9bae887a0 20 read_log 191727'371 (0'0) modify a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 by client.275175.0:13816 2014-07-30 14:19:17.131022 2014-08-13 17:38:02.635950 7fe9bae887a0 20 read_log 191727'372 (0'0) modify e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 by client.275175.0:14725 2014-07-30 14:19:19.920346 2014-08-13 17:38:02.635970 7fe9bae887a0 20 read_log 191727'378 (0'0) modify e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 by client.275175.0:16928 2014-07-30 14:19:26.954126 2014-08-13 17:38:02.635991 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 by client.275175.0:18003 2014-07-30 14:19:29.964849 2014-08-13 17:38:02.636012 7fe9bae887a0 20 read_log 191727'388 (0'0) modify b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 by client.275175.0:19821 2014-07-30 14:19:35.664807 2014-08-13 17:38:02.636033 7fe9bae887a0 20 read_log 191727'389 (0'0) modify cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 by client.275175.0:20074 2014-07-30 14:19:36.461904 2014-08-13 17:38:02.636054 7fe9bae887a0 20 read_log 191727'395 (0'0) modify c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 by client.275175.0:24931 2014-07-30 14:19:51.440949 2014-08-13 17:38:02.636076 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 by client.275175.0:26005 2014-07-30 14:19:54.735969 2014-08-13 17:38:02.636097 7fe9bae887a0 20 read_log 191727'404 (0'0) modify f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 by client.275175.0:30474 2014-07-30 14:20:08.503529 2014-08-13 17:38:02.636117 7fe9bae887a0 20 read_log 191727'406 (0'0) modify e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 by client.275175.0:31180 2014-07-30 14:20:10.498636 2014-08-13 17:38:02.636141 7fe9bae887a0 20 read_log 191727'411 (0'0) modify ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 by client.275175.0:32722 2014-07-30 14:20:14.949205 2014-08-13 17:38:02.636172 7fe9bae887a0 20 read_log 191727'414 (0'0) modify d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 by client.275175.0:33477 2014-07-30 14:20:17.384036 2014-08-13 17:38:02.636194 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 by client.275175.0:33865 2014-07-30 14:20:18.602629 2014-08-13 17:38:02.636216 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 by client.275175.0:34627 2014-07-30 14:20:20.909357 2014-08-13 17:38:02.636237 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 by client.275175.0:35150 2014-07-30 14:20:22.408807 2014-08-13 17:38:02.636258 7fe9bae887a0 20 read_log 192888'422 (0'0) modify 62b871f1/benchmark_data_storage0111-ib.pocstor_35625_object2004/head//22 by client.940330.0:2005 2014-08-11 16:29:48.571298 2014-08-13 17:38:02.636331 7fe9bae887a0 20 read_log 192888'423 (0'0) modify 951251f1/benchmark_data_storage0111-ib.pocstor_35625_object3369/head//22 by client.940330.0:3370 2014-08-11 16:29:53.991930 2014-08-13 17:38:02.636358 7fe9bae887a0 20 read_log 192888'424 (0'0) modify b35761f1/benchmark_data_storage0111-ib.pocstor_35625_object5059/head//22 by client.940330.0:5060 2014-08-11 16:30:00.241162 2014-08-13 17:38:02.636379 7fe9bae887a0 20 read_log 192888'425 (0'0) modify 27eca1f1/benchmark_data_storage0111-ib.pocstor_35625_object5670/head//22 by client.940330.0:5671 2014-08-11 16:30:02.399752 2014-08-13 17:38:02.636401 7fe9bae887a0 20 read_log 192888'426 (0'0) modify 8b8029f1/benchmark_data_storage0111-ib.pocstor_35625_object6113/head//22 by client.940330.0:6114 2014-08-11 16:30:03.985223 2014-08-13 17:38:02.636422 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 5cdf71f1/benchmark_data_storage0111-ib.pocstor_35625_object6531/head//22 by client.940330.0:6532 2014-08-11 16:30:05.557542 2014-08-13 17:38:02.636443 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 689941f1/benchmark_data_storage0111-ib.pocstor_35625_object7206/head//22 by client.940330.0:7207 2014-08-11 16:30:08.047077 2014-08-13 17:38:02.636466 7fe9bae887a0 20 read_log 192888'429 (0'0) modify b5c5d1f1/benchmark_data_storage0111-ib.pocstor_35625_object7842/head//22 by client.940330.0:7843 2014-08-11 16:30:10.280738 2014-08-13 17:38:02.636488 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 57a7b9f1/benchmark_data_storage0111-ib.pocstor_35625_object12142/head//22 by client.940330.0:12143 2014-08-11 16:30:26.542445 2014-08-13 17:38:02.636509 7fe9bae887a0 20 read_log 192888'431 (0'0) modify aa43c1f1/benchmark_data_storage0111-ib.pocstor_35625_object16350/head//22 by client.940330.0:16351 2014-08-11 16:30:42.538811 2014-08-13 17:38:02.636529 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 50f111f1/benchmark_data_storage0111-ib.pocstor_35625_object16386/head//22 by client.940330.0:16387 2014-08-11 16:30:42.671121 2014-08-13 17:38:02.636550 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 7a2541f1/benchmark_data_storage0111-ib.pocstor_35625_object21915/head//22 by client.940330.0:21916 2014-08-11 16:31:04.254575 2014-08-13 17:38:02.636571 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 601a51f1/benchmark_data_storage0111-ib.pocstor_35625_object23749/head//22 by client.940330.0:23750 2014-08-11 16:31:11.228612 2014-08-13 17:38:02.636591 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 45bc29f1/benchmark_data_storage0111-ib.pocstor_35625_object25493/head//22 by client.940330.0:25494 2014-08-11 16:31:17.812384 2014-08-13 17:38:02.636612 7fe9bae887a0 20 read_log 192888'436 (0'0) modify dcbec1f1/benchmark_data_storage0111-ib.pocstor_35625_object27340/head//22 by client.940330.0:27341 2014-08-11 16:31:25.118134 2014-08-13 17:38:02.636633 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 292b59f1/benchmark_data_storage0111-ib.pocstor_35625_object29457/head//22 by client.940330.0:29458 2014-08-11 16:31:33.404355 2014-08-13 17:38:02.636653 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 1bb829f1/benchmark_data_storage0111-ib.pocstor_35625_object30141/head//22 by client.940330.0:30142 2014-08-11 16:31:35.988255 2014-08-13 17:38:02.636674 7fe9bae887a0 20 read_log 192888'439 (192888'422) delete 62b871f1/benchmark_data_storage0111-ib.pocstor_35625_object2004/head//22 by client.940330.0:33312 2014-08-11 16:31:41.992955 2014-08-13 17:38:02.636697 7fe9bae887a0 20 read_log 192888'440 (192888'423) delete 951251f1/benchmark_data_storage0111-ib.pocstor_35625_object3369/head//22 by client.940330.0:34677 2014-08-11 16:31:42.353131 2014-08-13 17:38:02.636749 7fe9bae887a0 20 read_log 192888'441 (192888'424) delete b35761f1/benchmark_data_storage0111-ib.pocstor_35625_object5059/head//22 by client.940330.0:36367 2014-08-11 16:31:42.832710 2014-08-13 17:38:02.636772 7fe9bae887a0 20 read_log 192888'442 (192888'425) delete 27eca1f1/benchmark_data_storage0111-ib.pocstor_35625_object5670/head//22 by client.940330.0:36978 2014-08-11 16:31:42.988614 2014-08-13 17:38:02.636806 7fe9bae887a0 20 read_log 192888'443 (192888'426) delete 8b8029f1/benchmark_data_storage0111-ib.pocstor_35625_object6113/head//22 by client.940330.0:37421 2014-08-11 16:31:43.107436 2014-08-13 17:38:02.636827 7fe9bae887a0 20 read_log 192888'444 (192888'427) delete 5cdf71f1/benchmark_data_storage0111-ib.pocstor_35625_object6531/head//22 by client.940330.0:37839 2014-08-11 16:31:43.215123 2014-08-13 17:38:02.636849 7fe9bae887a0 20 read_log 192888'445 (192888'428) delete 689941f1/benchmark_data_storage0111-ib.pocstor_35625_object7206/head//22 by client.940330.0:38514 2014-08-11 16:31:43.386297 2014-08-13 17:38:02.636870 7fe9bae887a0 20 read_log 192888'446 (192888'429) delete b5c5d1f1/benchmark_data_storage0111-ib.pocstor_35625_object7842/head//22 by client.940330.0:39150 2014-08-11 16:31:43.550652 2014-08-13 17:38:02.636891 7fe9bae887a0 20 read_log 192888'447 (192888'430) delete 57a7b9f1/benchmark_data_storage0111-ib.pocstor_35625_object12142/head//22 by client.940330.0:43450 2014-08-11 16:31:44.694172 2014-08-13 17:38:02.636941 7fe9bae887a0 20 read_log 192888'448 (192888'431) delete aa43c1f1/benchmark_data_storage0111-ib.pocstor_35625_object16350/head//22 by client.940330.0:47658 2014-08-11 16:31:45.816008 2014-08-13 17:38:02.636964 7fe9bae887a0 20 read_log 192888'449 (192888'432) delete 50f111f1/benchmark_data_storage0111-ib.pocstor_35625_object16386/head//22 by client.940330.0:47694 2014-08-11 16:31:45.825806 2014-08-13 17:38:02.636985 7fe9bae887a0 20 read_log 192888'450 (192888'433) delete 7a2541f1/benchmark_data_storage0111-ib.pocstor_35625_object21915/head//22 by client.940330.0:53223 2014-08-11 16:31:47.418220 2014-08-13 17:38:02.637006 7fe9bae887a0 20 read_log 192888'451 (192888'434) delete 601a51f1/benchmark_data_storage0111-ib.pocstor_35625_object23749/head//22 by client.940330.0:55057 2014-08-11 16:31:47.964273 2014-08-13 17:38:02.637028 7fe9bae887a0 20 read_log 192888'452 (192888'435) delete 45bc29f1/benchmark_data_storage0111-ib.pocstor_35625_object25493/head//22 by client.940330.0:56801 2014-08-11 16:31:48.497357 2014-08-13 17:38:02.637049 7fe9bae887a0 20 read_log 192888'453 (192888'436) delete dcbec1f1/benchmark_data_storage0111-ib.pocstor_35625_object27340/head//22 by client.940330.0:58648 2014-08-11 16:31:49.020609 2014-08-13 17:38:02.637070 7fe9bae887a0 20 read_log 192888'454 (192888'437) delete 292b59f1/benchmark_data_storage0111-ib.pocstor_35625_object29457/head//22 by client.940330.0:60765 2014-08-11 16:31:49.599119 2014-08-13 17:38:02.637091 7fe9bae887a0 20 read_log 192888'455 (192888'438) delete 1bb829f1/benchmark_data_storage0111-ib.pocstor_35625_object30141/head//22 by client.940330.0:61449 2014-08-11 16:31:49.799778 2014-08-13 17:38:02.637112 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 2e1be1f1/benchmark_data_storage0111-ib.pocstor_64498_object1155/head//22 by client.1049635.0:1156 2014-08-12 17:17:20.919622 2014-08-13 17:38:02.637133 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 6ec2f1f1/benchmark_data_storage0111-ib.pocstor_64498_object2536/head//22 by client.1049635.0:2537 2014-08-12 17:17:25.355718 2014-08-13 17:38:02.637165 7fe9bae887a0 20 read_log 194215'458 (0'0) modify fed131f1/benchmark_data_storage0111-ib.pocstor_64498_object3266/head//22 by client.1049635.0:3267 2014-08-12 17:17:27.722683 2014-08-13 17:38:02.637190 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 89b1d9f1/benchmark_data_storage0111-ib.pocstor_64498_object3515/head//22 by client.1049635.0:3516 2014-08-12 17:17:28.547918 2014-08-13 17:38:02.637212 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 606191f1/benchmark_data_storage0111-ib.pocstor_64498_object3528/head//22 by client.1049635.0:3529 2014-08-12 17:17:28.584991 2014-08-13 17:38:02.637233 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 971491f1/benchmark_data_storage0111-ib.pocstor_64498_object4989/head//22 by client.1049635.0:4990 2014-08-12 17:17:32.897095 2014-08-13 17:38:02.637267 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 8717b9f1/benchmark_data_storage0111-ib.pocstor_64498_object9514/head//22 by client.1049635.0:9515 2014-08-12 17:17:46.534206 2014-08-13 17:38:02.637289 7fe9bae887a0 20 read_log 194215'463 (0'0) modify e95e99f1/benchmark_data_storage0111-ib.pocstor_64498_object9713/head//22 by client.1049635.0:9714 2014-08-12 17:17:47.111547 2014-08-13 17:38:02.637309 7fe9bae887a0 20 read_log 194215'464 (0'0) modify a5b441f1/benchmark_data_storage0111-ib.pocstor_64498_object9854/head//22 by client.1049635.0:9855 2014-08-12 17:17:47.510961 2014-08-13 17:38:02.637330 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 60d2b1f1/benchmark_data_storage0111-ib.pocstor_64498_object9915/head//22 by client.1049635.0:9916 2014-08-12 17:17:47.681505 2014-08-13 17:38:02.637353 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 12f0b1f1/benchmark_data_storage0111-ib.pocstor_64498_object10755/head//22 by client.1049635.0:10756 2014-08-12 17:17:50.398895 2014-08-13 17:38:02.637377 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 6c7771f1/benchmark_data_storage0111-ib.pocstor_64498_object12514/head//22 by client.1049635.0:12515 2014-08-12 17:17:55.935626 2014-08-13 17:38:02.637399 7fe9bae887a0 20 read_log 194215'468 (0'0) modify fed49f1/benchmark_data_storage0111-ib.pocstor_64498_object12637/head//22 by client.1049635.0:12638 2014-08-12 17:17:56.270754 2014-08-13 17:38:02.637420 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 9f789f1/benchmark_data_storage0111-ib.pocstor_64498_object17683/head//22 by client.1049635.0:17684 2014-08-12 17:18:13.956777 2014-08-13 17:38:02.637440 7fe9bae887a0 20 read_log 194215'470 (0'0) modify e1aea1f1/benchmark_data_storage0111-ib.pocstor_64498_object21412/head//22 by client.1049635.0:21413 2014-08-12 17:18:28.125117 2014-08-13 17:38:02.637461 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 680e49f1/benchmark_data_storage0111-ib.pocstor_64498_object22252/head//22 by client.1049635.0:22253 2014-08-12 17:18:31.280620 2014-08-13 17:38:02.637483 7fe9bae887a0 20 read_log 194215'472 (0'0) modify d74ca1f1/benchmark_data_storage0111-ib.pocstor_64498_object25767/head//22 by client.1049635.0:25768 2014-08-12 17:18:44.559114 2014-08-13 17:38:02.637504 7fe9bae887a0 20 read_log 194215'473 (0'0) modify b1ba19f1/benchmark_data_storage0111-ib.pocstor_64498_object27424/head//22 by client.1049635.0:27425 2014-08-12 17:18:50.739556 2014-08-13 17:38:02.637524 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 146f59f1/benchmark_data_storage0111-ib.pocstor_64498_object27765/head//22 by client.1049635.0:27766 2014-08-12 17:18:52.004242 2014-08-13 17:38:02.637545 7fe9bae887a0 20 read_log 194215'475 (194215'456) delete 2e1be1f1/benchmark_data_storage0111-ib.pocstor_64498_object1155/head//22 by client.1049635.0:35358 2014-08-12 17:19:18.369557 2014-08-13 17:38:02.637566 7fe9bae887a0 20 read_log 194215'476 (194215'457) delete 6ec2f1f1/benchmark_data_storage0111-ib.pocstor_64498_object2536/head//22 by client.1049635.0:36739 2014-08-12 17:19:18.812742 2014-08-13 17:38:02.637588 7fe9bae887a0 20 read_log 194215'477 (194215'458) delete fed131f1/benchmark_data_storage0111-ib.pocstor_64498_object3266/head//22 by client.1049635.0:37469 2014-08-12 17:19:19.049289 2014-08-13 17:38:02.637609 7fe9bae887a0 20 read_log 194215'478 (194215'459) delete 89b1d9f1/benchmark_data_storage0111-ib.pocstor_64498_object3515/head//22 by client.1049635.0:37718 2014-08-12 17:19:19.126273 2014-08-13 17:38:02.637630 7fe9bae887a0 20 read_log 194215'479 (194215'460) delete 606191f1/benchmark_data_storage0111-ib.pocstor_64498_object3528/head//22 by client.1049635.0:37731 2014-08-12 17:19:19.130057 2014-08-13 17:38:02.637651 7fe9bae887a0 20 read_log 194215'480 (194215'461) delete 971491f1/benchmark_data_storage0111-ib.pocstor_64498_object4989/head//22 by client.1049635.0:39192 2014-08-12 17:19:19.597482 2014-08-13 17:38:02.637672 7fe9bae887a0 20 read_log 194215'481 (194215'462) delete 8717b9f1/benchmark_data_storage0111-ib.pocstor_64498_object9514/head//22 by client.1049635.0:43717 2014-08-12 17:19:21.094660 2014-08-13 17:38:02.637696 7fe9bae887a0 20 read_log 194215'482 (194215'463) delete e95e99f1/benchmark_data_storage0111-ib.pocstor_64498_object9713/head//22 by client.1049635.0:43916 2014-08-12 17:19:21.158584 2014-08-13 17:38:02.637717 7fe9bae887a0 20 read_log 194215'483 (194215'464) delete a5b441f1/benchmark_data_storage0111-ib.pocstor_64498_object9854/head//22 by client.1049635.0:44057 2014-08-12 17:19:21.211585 2014-08-13 17:38:02.637738 7fe9bae887a0 20 read_log 194215'484 (194215'465) delete 60d2b1f1/benchmark_data_storage0111-ib.pocstor_64498_object9915/head//22 by client.1049635.0:44118 2014-08-12 17:19:21.229546 2014-08-13 17:38:02.637760 7fe9bae887a0 20 read_log 194215'485 (194215'466) delete 12f0b1f1/benchmark_data_storage0111-ib.pocstor_64498_object10755/head//22 by client.1049635.0:44958 2014-08-12 17:19:21.502449 2014-08-13 17:38:02.637781 7fe9bae887a0 20 read_log 194215'486 (194215'467) delete 6c7771f1/benchmark_data_storage0111-ib.pocstor_64498_object12514/head//22 by client.1049635.0:46717 2014-08-12 17:19:22.058385 2014-08-13 17:38:02.637802 7fe9bae887a0 20 read_log 194215'487 (194215'468) delete fed49f1/benchmark_data_storage0111-ib.pocstor_64498_object12637/head//22 by client.1049635.0:46840 2014-08-12 17:19:22.097029 2014-08-13 17:38:02.637823 7fe9bae887a0 20 read_log 194215'488 (194215'469) delete 9f789f1/benchmark_data_storage0111-ib.pocstor_64498_object17683/head//22 by client.1049635.0:51886 2014-08-12 17:19:23.721819 2014-08-13 17:38:02.637844 7fe9bae887a0 20 read_log 194215'489 (194215'470) delete e1aea1f1/benchmark_data_storage0111-ib.pocstor_64498_object21412/head//22 by client.1049635.0:55615 2014-08-12 17:19:25.023712 2014-08-13 17:38:02.637865 7fe9bae887a0 20 read_log 194215'490 (194215'471) delete 680e49f1/benchmark_data_storage0111-ib.pocstor_64498_object22252/head//22 by client.1049635.0:56455 2014-08-12 17:19:25.326738 2014-08-13 17:38:02.637886 7fe9bae887a0 20 read_log 194215'491 (194215'472) delete d74ca1f1/benchmark_data_storage0111-ib.pocstor_64498_object25767/head//22 by client.1049635.0:59970 2014-08-12 17:19:26.580689 2014-08-13 17:38:02.637907 7fe9bae887a0 20 read_log 194215'492 (194215'473) delete b1ba19f1/benchmark_data_storage0111-ib.pocstor_64498_object27424/head//22 by client.1049635.0:61627 2014-08-12 17:19:27.152497 2014-08-13 17:38:02.637928 7fe9bae887a0 20 read_log 194215'493 (194215'474) delete 146f59f1/benchmark_data_storage0111-ib.pocstor_64498_object27765/head//22 by client.1049635.0:61968 2014-08-12 17:19:27.265758 2014-08-13 17:38:02.638088 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.638115 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=0 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.638138 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=0 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003162 0 0.000000 2014-08-13 17:38:02.638167 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=0 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.638210 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=0 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.638225 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'493], crt=191648'346) 2014-08-13 17:38:02.638242 7fe9bae887a0 10 osd.4 194478 pgid 22.1fb coll 22.1fb_head 2014-08-13 17:38:02.638453 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.1fb 2014-08-13 17:38:02.638464 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.638507 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1fb(unlocked)] enter Initial 2014-08-13 17:38:02.638522 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1fb(unlocked)] enter NotTrimming 2014-08-13 17:38:02.638725 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.638903 7fe9bae887a0 20 read_log 191648'30 (0'0) modify 136be9fb/benchmark_data_storage0111-ib.pocstor_40745_object1798/head//22 by client.273158.0:1799 2014-07-30 14:08:55.383993 2014-08-13 17:38:02.638938 7fe9bae887a0 20 read_log 191648'57 (0'0) modify 938109fb/benchmark_data_storage0111-ib.pocstor_40745_object3302/head//22 by client.273158.0:3303 2014-07-30 14:09:02.709759 2014-08-13 17:38:02.638961 7fe9bae887a0 20 read_log 191648'62 (0'0) modify c60759fb/benchmark_data_storage0111-ib.pocstor_40745_object3555/head//22 by client.273158.0:3556 2014-07-30 14:09:03.941073 2014-08-13 17:38:02.638982 7fe9bae887a0 20 read_log 191648'94 (0'0) modify 50ba91fb/benchmark_data_storage0111-ib.pocstor_40745_object5695/head//22 by client.273158.0:5696 2014-07-30 14:09:14.225153 2014-08-13 17:38:02.639003 7fe9bae887a0 20 read_log 191648'109 (0'0) modify c49c01fb/benchmark_data_storage0111-ib.pocstor_40745_object6507/head//22 by client.273158.0:6508 2014-07-30 14:09:18.748658 2014-08-13 17:38:02.639024 7fe9bae887a0 20 read_log 191648'112 (0'0) modify 92a179fb/benchmark_data_storage0111-ib.pocstor_40745_object6668/head//22 by client.273158.0:6669 2014-07-30 14:09:19.628958 2014-08-13 17:38:02.639045 7fe9bae887a0 20 read_log 191648'250 (0'0) modify 45f161fb/benchmark_data_storage0111-ib.pocstor_40745_object16692/head//22 by client.273158.0:16693 2014-07-30 14:10:11.289369 2014-08-13 17:38:02.639066 7fe9bae887a0 20 read_log 191648'304 (0'0) modify 94a299fb/benchmark_data_storage0111-ib.pocstor_40745_object19853/head//22 by client.273158.0:19854 2014-07-30 14:10:30.750352 2014-08-13 17:38:02.639087 7fe9bae887a0 20 read_log 191648'313 (0'0) modify 7aacc9fb/benchmark_data_storage0111-ib.pocstor_40745_object20227/head//22 by client.273158.0:20228 2014-07-30 14:10:32.369551 2014-08-13 17:38:02.639108 7fe9bae887a0 20 read_log 191648'322 (0'0) modify fd17e1fb/benchmark_data_storage0111-ib.pocstor_40745_object20836/head//22 by client.273158.0:20837 2014-07-30 14:10:36.215961 2014-08-13 17:38:02.639130 7fe9bae887a0 20 read_log 191727'351 (0'0) modify f41419fb/benchmark_data_storage0111-ib.pocstor_51285_object81/head//22 by client.275175.0:82 2014-07-30 14:18:33.786266 2014-08-13 17:38:02.639151 7fe9bae887a0 20 read_log 191727'353 (0'0) modify 1fea39fb/benchmark_data_storage0111-ib.pocstor_51285_object1851/head//22 by client.275175.0:1852 2014-07-30 14:18:40.413134 2014-08-13 17:38:02.639188 7fe9bae887a0 20 read_log 191727'354 (0'0) modify 11ca11fb/benchmark_data_storage0111-ib.pocstor_51285_object1954/head//22 by client.275175.0:1955 2014-07-30 14:18:40.749952 2014-08-13 17:38:02.639210 7fe9bae887a0 20 read_log 191727'358 (0'0) modify 9e9599fb/benchmark_data_storage0111-ib.pocstor_51285_object5856/head//22 by client.275175.0:5857 2014-07-30 14:18:52.835925 2014-08-13 17:38:02.639231 7fe9bae887a0 20 read_log 191727'368 (0'0) modify a94181fb/benchmark_data_storage0111-ib.pocstor_51285_object9622/head//22 by client.275175.0:9623 2014-07-30 14:19:04.131710 2014-08-13 17:38:02.639252 7fe9bae887a0 20 read_log 191727'371 (0'0) modify f7a0f1fb/benchmark_data_storage0111-ib.pocstor_51285_object10117/head//22 by client.275175.0:10118 2014-07-30 14:19:05.508789 2014-08-13 17:38:02.639273 7fe9bae887a0 20 read_log 191727'372 (0'0) modify 617e1fb/benchmark_data_storage0111-ib.pocstor_51285_object10998/head//22 by client.275175.0:10999 2014-07-30 14:19:08.039689 2014-08-13 17:38:02.639294 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 94ada1fb/benchmark_data_storage0111-ib.pocstor_51285_object12487/head//22 by client.275175.0:12488 2014-07-30 14:19:13.003252 2014-08-13 17:38:02.639315 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 6e061fb/benchmark_data_storage0111-ib.pocstor_51285_object20938/head//22 by client.275175.0:20939 2014-07-30 14:19:38.955297 2014-08-13 17:38:02.639387 7fe9bae887a0 20 read_log 191727'392 (0'0) modify c82459fb/benchmark_data_storage0111-ib.pocstor_51285_object21217/head//22 by client.275175.0:21218 2014-07-30 14:19:39.755515 2014-08-13 17:38:02.639416 7fe9bae887a0 20 read_log 191727'394 (0'0) modify f676f9fb/benchmark_data_storage0111-ib.pocstor_51285_object21392/head//22 by client.275175.0:21393 2014-07-30 14:19:40.280549 2014-08-13 17:38:02.639439 7fe9bae887a0 20 read_log 191727'396 (0'0) modify d13631fb/benchmark_data_storage0111-ib.pocstor_51285_object24265/head//22 by client.275175.0:24266 2014-07-30 14:19:49.573530 2014-08-13 17:38:02.639460 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 64d59fb/benchmark_data_storage0111-ib.pocstor_51285_object26346/head//22 by client.275175.0:26347 2014-07-30 14:19:55.846545 2014-08-13 17:38:02.639483 7fe9bae887a0 20 read_log 191727'408 (0'0) modify a3ba89fb/benchmark_data_storage0111-ib.pocstor_51285_object28449/head//22 by client.275175.0:28450 2014-07-30 14:20:02.279194 2014-08-13 17:38:02.639504 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 66e8b9fb/benchmark_data_storage0111-ib.pocstor_51285_object31368/head//22 by client.275175.0:31369 2014-07-30 14:20:11.022851 2014-08-13 17:38:02.639524 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 81bb51fb/benchmark_data_storage0111-ib.pocstor_51285_object32061/head//22 by client.275175.0:32062 2014-07-30 14:20:13.024473 2014-08-13 17:38:02.639548 7fe9bae887a0 20 read_log 191727'422 (0'0) modify cec859fb/benchmark_data_storage0111-ib.pocstor_51285_object36315/head//22 by client.275175.0:36316 2014-07-30 14:20:26.236339 2014-08-13 17:38:02.639572 7fe9bae887a0 20 read_log 191727'423 (0'0) modify 5f3b99fb/benchmark_data_storage0111-ib.pocstor_51285_object37312/head//22 by client.275175.0:37313 2014-07-30 14:20:29.242057 2014-08-13 17:38:02.639594 7fe9bae887a0 20 read_log 192666'425 (0'0) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1175 2014-07-30 16:28:47.187955 2014-08-13 17:38:02.639615 7fe9bae887a0 20 read_log 192666'426 (192666'425) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1177 2014-07-30 16:28:47.194955 2014-08-13 17:38:02.639636 7fe9bae887a0 20 read_log 192666'427 (192666'426) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1178 2014-07-30 16:28:47.198955 2014-08-13 17:38:02.639657 7fe9bae887a0 20 read_log 192666'428 (192666'427) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1179 2014-07-30 16:28:47.202954 2014-08-13 17:38:02.639678 7fe9bae887a0 20 read_log 192666'429 (192666'428) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1180 2014-07-30 16:28:47.206954 2014-08-13 17:38:02.639700 7fe9bae887a0 20 read_log 192666'430 (192666'429) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1181 2014-07-30 16:28:47.211954 2014-08-13 17:38:02.639720 7fe9bae887a0 20 read_log 192666'431 (192666'430) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1182 2014-07-30 16:28:47.215954 2014-08-13 17:38:02.639741 7fe9bae887a0 20 read_log 192666'432 (192666'431) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1183 2014-07-30 16:28:47.218954 2014-08-13 17:38:02.639761 7fe9bae887a0 20 read_log 192666'433 (192666'432) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1184 2014-07-30 16:28:47.222954 2014-08-13 17:38:02.639782 7fe9bae887a0 20 read_log 192666'434 (192666'433) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1185 2014-07-30 16:28:47.226954 2014-08-13 17:38:02.639802 7fe9bae887a0 20 read_log 192666'435 (192666'434) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1186 2014-07-30 16:28:47.230953 2014-08-13 17:38:02.639822 7fe9bae887a0 20 read_log 192666'436 (192666'435) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1187 2014-07-30 16:28:47.234953 2014-08-13 17:38:02.639854 7fe9bae887a0 20 read_log 192666'437 (192666'436) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1188 2014-07-30 16:28:47.238953 2014-08-13 17:38:02.639876 7fe9bae887a0 20 read_log 192666'438 (192666'437) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1189 2014-07-30 16:28:47.241953 2014-08-13 17:38:02.639896 7fe9bae887a0 20 read_log 192666'439 (192666'438) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1190 2014-07-30 16:28:47.244953 2014-08-13 17:38:02.639917 7fe9bae887a0 20 read_log 192666'440 (192666'439) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1191 2014-07-30 16:28:47.248953 2014-08-13 17:38:02.639964 7fe9bae887a0 20 read_log 192666'441 (192666'440) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1193 2014-07-30 16:28:47.291951 2014-08-13 17:38:02.639986 7fe9bae887a0 20 read_log 192666'442 (192666'441) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:799 2014-07-30 17:22:40.202000 2014-08-13 17:38:02.640007 7fe9bae887a0 20 read_log 192666'443 (192666'442) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:801 2014-07-30 17:22:40.211000 2014-08-13 17:38:02.640029 7fe9bae887a0 20 read_log 192666'444 (192666'443) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:802 2014-07-30 17:22:40.217000 2014-08-13 17:38:02.640050 7fe9bae887a0 20 read_log 192666'445 (192666'444) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:803 2014-07-30 17:22:40.223000 2014-08-13 17:38:02.640070 7fe9bae887a0 20 read_log 192666'446 (192666'445) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:804 2014-07-30 17:22:40.229000 2014-08-13 17:38:02.640091 7fe9bae887a0 20 read_log 192666'447 (192666'446) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:805 2014-07-30 17:22:40.235000 2014-08-13 17:38:02.640111 7fe9bae887a0 20 read_log 192666'448 (192666'447) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:806 2014-07-30 17:22:40.240000 2014-08-13 17:38:02.640131 7fe9bae887a0 20 read_log 192666'449 (192666'448) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:807 2014-07-30 17:22:40.245000 2014-08-13 17:38:02.640151 7fe9bae887a0 20 read_log 192666'450 (192666'449) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:808 2014-07-30 17:22:40.251000 2014-08-13 17:38:02.640181 7fe9bae887a0 20 read_log 192666'451 (192666'450) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:809 2014-07-30 17:22:40.256000 2014-08-13 17:38:02.640204 7fe9bae887a0 20 read_log 192666'452 (192666'451) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:810 2014-07-30 17:22:40.261000 2014-08-13 17:38:02.640225 7fe9bae887a0 20 read_log 192666'453 (192666'452) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:811 2014-07-30 17:22:40.266000 2014-08-13 17:38:02.640245 7fe9bae887a0 20 read_log 192666'454 (192666'453) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:812 2014-07-30 17:22:40.272000 2014-08-13 17:38:02.640266 7fe9bae887a0 20 read_log 192666'455 (192666'454) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:813 2014-07-30 17:22:40.277000 2014-08-13 17:38:02.640286 7fe9bae887a0 20 read_log 192666'456 (192666'455) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:814 2014-07-30 17:22:40.283000 2014-08-13 17:38:02.640306 7fe9bae887a0 20 read_log 192666'457 (192666'456) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:815 2014-07-30 17:22:40.288000 2014-08-13 17:38:02.640327 7fe9bae887a0 20 read_log 192666'458 (192666'457) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:817 2014-07-30 17:22:40.294000 2014-08-13 17:38:02.640388 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 921031fb/benchmark_data_storage0111-ib.pocstor_35625_object579/head//22 by client.940330.0:580 2014-08-11 16:29:43.040595 2014-08-13 17:38:02.640412 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 18f419fb/benchmark_data_storage0111-ib.pocstor_35625_object4140/head//22 by client.940330.0:4141 2014-08-11 16:29:56.840380 2014-08-13 17:38:02.640434 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 2c26a9fb/benchmark_data_storage0111-ib.pocstor_35625_object6864/head//22 by client.940330.0:6865 2014-08-11 16:30:06.841979 2014-08-13 17:38:02.640455 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 4de21fb/benchmark_data_storage0111-ib.pocstor_35625_object9106/head//22 by client.940330.0:9107 2014-08-11 16:30:14.702129 2014-08-13 17:38:02.640476 7fe9bae887a0 20 read_log 192888'463 (0'0) modify df5061fb/benchmark_data_storage0111-ib.pocstor_35625_object9110/head//22 by client.940330.0:9111 2014-08-11 16:30:14.716489 2014-08-13 17:38:02.640497 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 126a79fb/benchmark_data_storage0111-ib.pocstor_35625_object12346/head//22 by client.940330.0:12347 2014-08-11 16:30:27.305793 2014-08-13 17:38:02.640517 7fe9bae887a0 20 read_log 192888'465 (0'0) modify e9d2d1fb/benchmark_data_storage0111-ib.pocstor_35625_object12374/head//22 by client.940330.0:12375 2014-08-11 16:30:27.412367 2014-08-13 17:38:02.640538 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 15ab9fb/benchmark_data_storage0111-ib.pocstor_35625_object14057/head//22 by client.940330.0:14058 2014-08-11 16:30:33.849231 2014-08-13 17:38:02.640558 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 201e01fb/benchmark_data_storage0111-ib.pocstor_35625_object17661/head//22 by client.940330.0:17662 2014-08-11 16:30:47.449097 2014-08-13 17:38:02.640579 7fe9bae887a0 20 read_log 192888'468 (0'0) modify ec9299fb/benchmark_data_storage0111-ib.pocstor_35625_object18051/head//22 by client.940330.0:18052 2014-08-11 16:30:48.927590 2014-08-13 17:38:02.640599 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 430bd1fb/benchmark_data_storage0111-ib.pocstor_35625_object21376/head//22 by client.940330.0:21377 2014-08-11 16:31:02.225114 2014-08-13 17:38:02.640620 7fe9bae887a0 20 read_log 192888'470 (0'0) modify fa2891fb/benchmark_data_storage0111-ib.pocstor_35625_object22572/head//22 by client.940330.0:22573 2014-08-11 16:31:06.798464 2014-08-13 17:38:02.640640 7fe9bae887a0 20 read_log 192888'471 (0'0) modify 4012c9fb/benchmark_data_storage0111-ib.pocstor_35625_object24261/head//22 by client.940330.0:24262 2014-08-11 16:31:13.100712 2014-08-13 17:38:02.640661 7fe9bae887a0 20 read_log 192888'472 (0'0) modify e49cc9fb/benchmark_data_storage0111-ib.pocstor_35625_object28716/head//22 by client.940330.0:28717 2014-08-11 16:31:30.504701 2014-08-13 17:38:02.640681 7fe9bae887a0 20 read_log 192888'473 (0'0) modify 2143b9fb/benchmark_data_storage0111-ib.pocstor_35625_object28772/head//22 by client.940330.0:28773 2014-08-11 16:31:30.715309 2014-08-13 17:38:02.640702 7fe9bae887a0 20 read_log 192888'474 (0'0) modify f65409fb/benchmark_data_storage0111-ib.pocstor_35625_object28848/head//22 by client.940330.0:28849 2014-08-11 16:31:31.003813 2014-08-13 17:38:02.640723 7fe9bae887a0 20 read_log 192888'475 (0'0) modify 11c731fb/benchmark_data_storage0111-ib.pocstor_35625_object31226/head//22 by client.940330.0:31227 2014-08-11 16:31:40.122664 2014-08-13 17:38:02.640743 7fe9bae887a0 20 read_log 192888'476 (192888'459) delete 921031fb/benchmark_data_storage0111-ib.pocstor_35625_object579/head//22 by client.940330.0:31887 2014-08-11 16:31:41.648094 2014-08-13 17:38:02.640764 7fe9bae887a0 20 read_log 192888'477 (192888'460) delete 18f419fb/benchmark_data_storage0111-ib.pocstor_35625_object4140/head//22 by client.940330.0:35448 2014-08-11 16:31:42.575077 2014-08-13 17:38:02.640786 7fe9bae887a0 20 read_log 192888'478 (192888'461) delete 2c26a9fb/benchmark_data_storage0111-ib.pocstor_35625_object6864/head//22 by client.940330.0:38172 2014-08-11 16:31:43.302762 2014-08-13 17:38:02.640825 7fe9bae887a0 20 read_log 192888'479 (192888'462) delete 4de21fb/benchmark_data_storage0111-ib.pocstor_35625_object9106/head//22 by client.940330.0:40414 2014-08-11 16:31:43.867858 2014-08-13 17:38:02.640848 7fe9bae887a0 20 read_log 192888'480 (192888'463) delete df5061fb/benchmark_data_storage0111-ib.pocstor_35625_object9110/head//22 by client.940330.0:40418 2014-08-11 16:31:43.868880 2014-08-13 17:38:02.640869 7fe9bae887a0 20 read_log 192888'481 (192888'464) delete 126a79fb/benchmark_data_storage0111-ib.pocstor_35625_object12346/head//22 by client.940330.0:43654 2014-08-11 16:31:44.744195 2014-08-13 17:38:02.640891 7fe9bae887a0 20 read_log 192888'482 (192888'465) delete e9d2d1fb/benchmark_data_storage0111-ib.pocstor_35625_object12374/head//22 by client.940330.0:43682 2014-08-11 16:31:44.750673 2014-08-13 17:38:02.640912 7fe9bae887a0 20 read_log 192888'483 (192888'466) delete 15ab9fb/benchmark_data_storage0111-ib.pocstor_35625_object14057/head//22 by client.940330.0:45365 2014-08-11 16:31:45.197280 2014-08-13 17:38:02.640933 7fe9bae887a0 20 read_log 192888'484 (192888'467) delete 201e01fb/benchmark_data_storage0111-ib.pocstor_35625_object17661/head//22 by client.940330.0:48969 2014-08-11 16:31:46.180881 2014-08-13 17:38:02.640954 7fe9bae887a0 20 read_log 192888'485 (192888'468) delete ec9299fb/benchmark_data_storage0111-ib.pocstor_35625_object18051/head//22 by client.940330.0:49359 2014-08-11 16:31:46.300935 2014-08-13 17:38:02.640975 7fe9bae887a0 20 read_log 192888'486 (192888'469) delete 430bd1fb/benchmark_data_storage0111-ib.pocstor_35625_object21376/head//22 by client.940330.0:52684 2014-08-11 16:31:47.261403 2014-08-13 17:38:02.640996 7fe9bae887a0 20 read_log 192888'487 (192888'470) delete fa2891fb/benchmark_data_storage0111-ib.pocstor_35625_object22572/head//22 by client.940330.0:53880 2014-08-11 16:31:47.625966 2014-08-13 17:38:02.641017 7fe9bae887a0 20 read_log 192888'488 (192888'471) delete 4012c9fb/benchmark_data_storage0111-ib.pocstor_35625_object24261/head//22 by client.940330.0:55569 2014-08-11 16:31:48.116812 2014-08-13 17:38:02.641038 7fe9bae887a0 20 read_log 192888'489 (192888'472) delete e49cc9fb/benchmark_data_storage0111-ib.pocstor_35625_object28716/head//22 by client.940330.0:60024 2014-08-11 16:31:49.395968 2014-08-13 17:38:02.641059 7fe9bae887a0 20 read_log 192888'490 (192888'473) delete 2143b9fb/benchmark_data_storage0111-ib.pocstor_35625_object28772/head//22 by client.940330.0:60080 2014-08-11 16:31:49.411129 2014-08-13 17:38:02.641080 7fe9bae887a0 20 read_log 192888'491 (192888'474) delete f65409fb/benchmark_data_storage0111-ib.pocstor_35625_object28848/head//22 by client.940330.0:60156 2014-08-11 16:31:49.432132 2014-08-13 17:38:02.641101 7fe9bae887a0 20 read_log 192888'492 (192888'475) delete 11c731fb/benchmark_data_storage0111-ib.pocstor_35625_object31226/head//22 by client.940330.0:62534 2014-08-11 16:31:50.128190 2014-08-13 17:38:02.641122 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 9903e1fb/benchmark_data_storage0111-ib.pocstor_64498_object324/head//22 by client.1049635.0:325 2014-08-12 17:17:18.136507 2014-08-13 17:38:02.641143 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 897231fb/benchmark_data_storage0111-ib.pocstor_64498_object1672/head//22 by client.1049635.0:1673 2014-08-12 17:17:22.498188 2014-08-13 17:38:02.641173 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 508bb1fb/benchmark_data_storage0111-ib.pocstor_64498_object11163/head//22 by client.1049635.0:11164 2014-08-12 17:17:51.726622 2014-08-13 17:38:02.641195 7fe9bae887a0 20 read_log 194215'496 (0'0) modify f42729fb/benchmark_data_storage0111-ib.pocstor_64498_object12326/head//22 by client.1049635.0:12327 2014-08-12 17:17:55.420344 2014-08-13 17:38:02.641240 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 2ce931fb/benchmark_data_storage0111-ib.pocstor_64498_object14190/head//22 by client.1049635.0:14191 2014-08-12 17:18:01.082516 2014-08-13 17:38:02.641277 7fe9bae887a0 20 read_log 194215'498 (0'0) modify b22b59fb/benchmark_data_storage0111-ib.pocstor_64498_object16283/head//22 by client.1049635.0:16284 2014-08-12 17:18:08.632257 2014-08-13 17:38:02.641301 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 713421fb/benchmark_data_storage0111-ib.pocstor_64498_object17669/head//22 by client.1049635.0:17670 2014-08-12 17:18:13.905150 2014-08-13 17:38:02.641323 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 3f5a51fb/benchmark_data_storage0111-ib.pocstor_64498_object20027/head//22 by client.1049635.0:20028 2014-08-12 17:18:22.883952 2014-08-13 17:38:02.641344 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 78cd61fb/benchmark_data_storage0111-ib.pocstor_64498_object20894/head//22 by client.1049635.0:20895 2014-08-12 17:18:26.131271 2014-08-13 17:38:02.641365 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 6eb279fb/benchmark_data_storage0111-ib.pocstor_64498_object25470/head//22 by client.1049635.0:25471 2014-08-12 17:18:43.454644 2014-08-13 17:38:02.641389 7fe9bae887a0 20 read_log 194215'503 (0'0) modify 33fe11fb/benchmark_data_storage0111-ib.pocstor_64498_object27034/head//22 by client.1049635.0:27035 2014-08-12 17:18:49.275326 2014-08-13 17:38:02.641410 7fe9bae887a0 20 read_log 194215'504 (0'0) modify 4e3d69fb/benchmark_data_storage0111-ib.pocstor_64498_object27074/head//22 by client.1049635.0:27075 2014-08-12 17:18:49.425237 2014-08-13 17:38:02.641431 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 7aa881fb/benchmark_data_storage0111-ib.pocstor_64498_object28819/head//22 by client.1049635.0:28820 2014-08-12 17:18:55.925099 2014-08-13 17:38:02.641452 7fe9bae887a0 20 read_log 194215'506 (0'0) modify 630141fb/benchmark_data_storage0111-ib.pocstor_64498_object30109/head//22 by client.1049635.0:30110 2014-08-12 17:19:00.901195 2014-08-13 17:38:02.641473 7fe9bae887a0 20 read_log 194215'507 (0'0) modify 92ba79fb/benchmark_data_storage0111-ib.pocstor_64498_object30676/head//22 by client.1049635.0:30677 2014-08-12 17:19:03.174201 2014-08-13 17:38:02.641494 7fe9bae887a0 20 read_log 194215'508 (194215'493) delete 9903e1fb/benchmark_data_storage0111-ib.pocstor_64498_object324/head//22 by client.1049635.0:34527 2014-08-12 17:19:18.098805 2014-08-13 17:38:02.641516 7fe9bae887a0 20 read_log 194215'509 (194215'494) delete 897231fb/benchmark_data_storage0111-ib.pocstor_64498_object1672/head//22 by client.1049635.0:35875 2014-08-12 17:19:18.541691 2014-08-13 17:38:02.641537 7fe9bae887a0 20 read_log 194215'510 (194215'495) delete 508bb1fb/benchmark_data_storage0111-ib.pocstor_64498_object11163/head//22 by client.1049635.0:45366 2014-08-12 17:19:21.630442 2014-08-13 17:38:02.641558 7fe9bae887a0 20 read_log 194215'511 (194215'496) delete f42729fb/benchmark_data_storage0111-ib.pocstor_64498_object12326/head//22 by client.1049635.0:46529 2014-08-12 17:19:22.000955 2014-08-13 17:38:02.641579 7fe9bae887a0 20 read_log 194215'512 (194215'497) delete 2ce931fb/benchmark_data_storage0111-ib.pocstor_64498_object14190/head//22 by client.1049635.0:48393 2014-08-12 17:19:22.619583 2014-08-13 17:38:02.641600 7fe9bae887a0 20 read_log 194215'513 (194215'498) delete b22b59fb/benchmark_data_storage0111-ib.pocstor_64498_object16283/head//22 by client.1049635.0:50486 2014-08-12 17:19:23.278073 2014-08-13 17:38:02.641621 7fe9bae887a0 20 read_log 194215'514 (194215'499) delete 713421fb/benchmark_data_storage0111-ib.pocstor_64498_object17669/head//22 by client.1049635.0:51872 2014-08-12 17:19:23.717968 2014-08-13 17:38:02.641642 7fe9bae887a0 20 read_log 194215'515 (194215'500) delete 3f5a51fb/benchmark_data_storage0111-ib.pocstor_64498_object20027/head//22 by client.1049635.0:54230 2014-08-12 17:19:24.523712 2014-08-13 17:38:02.641665 7fe9bae887a0 20 read_log 194215'516 (194215'501) delete 78cd61fb/benchmark_data_storage0111-ib.pocstor_64498_object20894/head//22 by client.1049635.0:55097 2014-08-12 17:19:24.847906 2014-08-13 17:38:02.641686 7fe9bae887a0 20 read_log 194215'517 (194215'502) delete 6eb279fb/benchmark_data_storage0111-ib.pocstor_64498_object25470/head//22 by client.1049635.0:59673 2014-08-12 17:19:26.474435 2014-08-13 17:38:02.641724 7fe9bae887a0 20 read_log 194215'518 (194215'503) delete 33fe11fb/benchmark_data_storage0111-ib.pocstor_64498_object27034/head//22 by client.1049635.0:61237 2014-08-12 17:19:27.012541 2014-08-13 17:38:02.641747 7fe9bae887a0 20 read_log 194215'519 (194215'504) delete 4e3d69fb/benchmark_data_storage0111-ib.pocstor_64498_object27074/head//22 by client.1049635.0:61277 2014-08-12 17:19:27.023532 2014-08-13 17:38:02.641769 7fe9bae887a0 20 read_log 194215'520 (194215'505) delete 7aa881fb/benchmark_data_storage0111-ib.pocstor_64498_object28819/head//22 by client.1049635.0:63022 2014-08-12 17:19:27.634200 2014-08-13 17:38:02.641790 7fe9bae887a0 20 read_log 194215'521 (194215'506) delete 630141fb/benchmark_data_storage0111-ib.pocstor_64498_object30109/head//22 by client.1049635.0:64312 2014-08-12 17:19:28.080511 2014-08-13 17:38:02.641838 7fe9bae887a0 20 read_log 194215'522 (194215'507) delete 92ba79fb/benchmark_data_storage0111-ib.pocstor_64498_object30676/head//22 by client.1049635.0:64879 2014-08-12 17:19:28.266623 2014-08-13 17:38:02.642041 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.642070 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.642093 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.003585 0 0.000000 2014-08-13 17:38:02.642113 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.642127 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=0 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.642142 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] log((0'0,194215'522], crt=192888'492) 2014-08-13 17:38:02.642168 7fe9bae887a0 10 osd.4 194478 pgid 22.220 coll 22.220_head 2014-08-13 17:38:02.642367 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.220 2014-08-13 17:38:02.642384 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.642429 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.220(unlocked)] enter Initial 2014-08-13 17:38:02.642445 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.220(unlocked)] enter NotTrimming 2014-08-13 17:38:02.642661 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.642881 7fe9bae887a0 20 read_log 191648'7 (0'0) modify d8526a20/benchmark_data_storage0111-ib.pocstor_40745_object555/head//22 by client.273158.0:556 2014-07-30 14:08:50.853061 2014-08-13 17:38:02.642918 7fe9bae887a0 20 read_log 191648'27 (0'0) modify a2194a20/benchmark_data_storage0111-ib.pocstor_40745_object1699/head//22 by client.273158.0:1700 2014-07-30 14:08:55.025610 2014-08-13 17:38:02.642941 7fe9bae887a0 20 read_log 191648'50 (0'0) modify 2d49ba20/benchmark_data_storage0111-ib.pocstor_40745_object2873/head//22 by client.273158.0:2874 2014-07-30 14:09:00.853219 2014-08-13 17:38:02.642962 7fe9bae887a0 20 read_log 191648'59 (0'0) modify c125d220/benchmark_data_storage0111-ib.pocstor_40745_object3402/head//22 by client.273158.0:3403 2014-07-30 14:09:03.246494 2014-08-13 17:38:02.642983 7fe9bae887a0 20 read_log 191648'80 (0'0) modify 83db220/benchmark_data_storage0111-ib.pocstor_40745_object4815/head//22 by client.273158.0:4816 2014-07-30 14:09:10.206601 2014-08-13 17:38:02.643004 7fe9bae887a0 20 read_log 191648'88 (0'0) modify 123d0220/benchmark_data_storage0111-ib.pocstor_40745_object5226/head//22 by client.273158.0:5227 2014-07-30 14:09:12.335079 2014-08-13 17:38:02.643025 7fe9bae887a0 20 read_log 191648'110 (0'0) modify fa035a20/benchmark_data_storage0111-ib.pocstor_40745_object6605/head//22 by client.273158.0:6606 2014-07-30 14:09:19.375385 2014-08-13 17:38:02.643046 7fe9bae887a0 20 read_log 191648'144 (0'0) modify 51a55220/benchmark_data_storage0111-ib.pocstor_40745_object8304/head//22 by client.273158.0:8305 2014-07-30 14:09:27.845135 2014-08-13 17:38:02.643098 7fe9bae887a0 20 read_log 191648'149 (0'0) modify 10b7b220/benchmark_data_storage0111-ib.pocstor_40745_object8710/head//22 by client.273158.0:8711 2014-07-30 14:09:30.032909 2014-08-13 17:38:02.643125 7fe9bae887a0 20 read_log 191648'213 (0'0) modify f2ad0220/benchmark_data_storage0111-ib.pocstor_40745_object12399/head//22 by client.273158.0:12400 2014-07-30 14:09:49.578673 2014-08-13 17:38:02.643196 7fe9bae887a0 20 read_log 191648'217 (0'0) modify aa7f9220/benchmark_data_storage0111-ib.pocstor_40745_object13009/head//22 by client.273158.0:13010 2014-07-30 14:09:52.399293 2014-08-13 17:38:02.643218 7fe9bae887a0 20 read_log 191648'241 (0'0) modify df2a9a20/benchmark_data_storage0111-ib.pocstor_40745_object14295/head//22 by client.273158.0:14296 2014-07-30 14:09:59.191602 2014-08-13 17:38:02.643239 7fe9bae887a0 20 read_log 191648'258 (0'0) modify a1011a20/benchmark_data_storage0111-ib.pocstor_40745_object16025/head//22 by client.273158.0:16026 2014-07-30 14:10:07.702993 2014-08-13 17:38:02.643262 7fe9bae887a0 20 read_log 191648'329 (0'0) modify 4235d220/benchmark_data_storage0111-ib.pocstor_40745_object20169/head//22 by client.273158.0:20170 2014-07-30 14:10:32.155820 2014-08-13 17:38:02.643284 7fe9bae887a0 20 read_log 191727'374 (0'0) modify fa3ada20/benchmark_data_storage0111-ib.pocstor_51285_object1425/head//22 by client.275175.0:1426 2014-07-30 14:18:38.954091 2014-08-13 17:38:02.643305 7fe9bae887a0 20 read_log 191727'375 (0'0) modify b1065a20/benchmark_data_storage0111-ib.pocstor_51285_object2148/head//22 by client.275175.0:2149 2014-07-30 14:18:41.381351 2014-08-13 17:38:02.643344 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 9f56aa20/benchmark_data_storage0111-ib.pocstor_51285_object4051/head//22 by client.275175.0:4052 2014-07-30 14:18:47.514397 2014-08-13 17:38:02.643371 7fe9bae887a0 20 read_log 191727'380 (0'0) modify 66b72220/benchmark_data_storage0111-ib.pocstor_51285_object4608/head//22 by client.275175.0:4609 2014-07-30 14:18:49.185942 2014-08-13 17:38:02.643392 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 53151220/benchmark_data_storage0111-ib.pocstor_51285_object5602/head//22 by client.275175.0:5603 2014-07-30 14:18:52.108010 2014-08-13 17:38:02.643413 7fe9bae887a0 20 read_log 191727'389 (0'0) modify c009ea20/benchmark_data_storage0111-ib.pocstor_51285_object9325/head//22 by client.275175.0:9326 2014-07-30 14:19:03.292991 2014-08-13 17:38:02.643434 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 6864a220/benchmark_data_storage0111-ib.pocstor_51285_object12381/head//22 by client.275175.0:12382 2014-07-30 14:19:12.703588 2014-08-13 17:38:02.643455 7fe9bae887a0 20 read_log 191727'407 (0'0) modify d357220/benchmark_data_storage0111-ib.pocstor_51285_object19103/head//22 by client.275175.0:19104 2014-07-30 14:19:33.624767 2014-08-13 17:38:02.643476 7fe9bae887a0 20 read_log 191727'408 (0'0) modify 9ce4da20/benchmark_data_storage0111-ib.pocstor_51285_object19841/head//22 by client.275175.0:19842 2014-07-30 14:19:35.726695 2014-08-13 17:38:02.643497 7fe9bae887a0 20 read_log 191727'409 (0'0) modify b8288a20/benchmark_data_storage0111-ib.pocstor_51285_object20289/head//22 by client.275175.0:20290 2014-07-30 14:19:37.093241 2014-08-13 17:38:02.643517 7fe9bae887a0 20 read_log 191727'410 (0'0) modify c5849220/benchmark_data_storage0111-ib.pocstor_51285_object20433/head//22 by client.275175.0:20434 2014-07-30 14:19:37.505847 2014-08-13 17:38:02.643538 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 9732e220/benchmark_data_storage0111-ib.pocstor_51285_object22699/head//22 by client.275175.0:22700 2014-07-30 14:19:45.090339 2014-08-13 17:38:02.643558 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 73fc1220/benchmark_data_storage0111-ib.pocstor_51285_object23070/head//22 by client.275175.0:23071 2014-07-30 14:19:46.223055 2014-08-13 17:38:02.643579 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 73e59a20/benchmark_data_storage0111-ib.pocstor_51285_object24191/head//22 by client.275175.0:24192 2014-07-30 14:19:49.365760 2014-08-13 17:38:02.643599 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 2c01ba20/benchmark_data_storage0111-ib.pocstor_51285_object24662/head//22 by client.275175.0:24663 2014-07-30 14:19:50.692751 2014-08-13 17:38:02.643619 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 2b3daa20/benchmark_data_storage0111-ib.pocstor_51285_object28342/head//22 by client.275175.0:28343 2014-07-30 14:20:01.981635 2014-08-13 17:38:02.643640 7fe9bae887a0 20 read_log 191727'432 (0'0) modify 4bec1a20/benchmark_data_storage0111-ib.pocstor_51285_object31025/head//22 by client.275175.0:31026 2014-07-30 14:20:10.054598 2014-08-13 17:38:02.643661 7fe9bae887a0 20 read_log 191727'434 (0'0) modify 1af60a20/benchmark_data_storage0111-ib.pocstor_51285_object31610/head//22 by client.275175.0:31611 2014-07-30 14:20:11.759890 2014-08-13 17:38:02.643681 7fe9bae887a0 20 read_log 191727'436 (0'0) modify 78132220/benchmark_data_storage0111-ib.pocstor_51285_object32172/head//22 by client.275175.0:32173 2014-07-30 14:20:13.386253 2014-08-13 17:38:02.643702 7fe9bae887a0 20 read_log 191727'437 (0'0) modify 1595f220/benchmark_data_storage0111-ib.pocstor_51285_object32760/head//22 by client.275175.0:32761 2014-07-30 14:20:15.096708 2014-08-13 17:38:02.643723 7fe9bae887a0 20 read_log 191727'442 (0'0) modify bb376a20/benchmark_data_storage0111-ib.pocstor_51285_object34868/head//22 by client.275175.0:34869 2014-07-30 14:20:21.610610 2014-08-13 17:38:02.643743 7fe9bae887a0 20 read_log 191727'449 (0'0) modify ea85ba20/benchmark_data_storage0111-ib.pocstor_51285_object38693/head//22 by client.275175.0:38694 2014-07-30 14:20:33.220940 2014-08-13 17:38:02.643775 7fe9bae887a0 20 read_log 192888'450 (0'0) modify e903ea20/benchmark_data_storage0111-ib.pocstor_35625_object941/head//22 by client.940330.0:942 2014-08-11 16:29:44.463380 2014-08-13 17:38:02.643798 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 65670220/benchmark_data_storage0111-ib.pocstor_35625_object4252/head//22 by client.940330.0:4253 2014-08-11 16:29:57.245207 2014-08-13 17:38:02.643819 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 3ce8ca20/benchmark_data_storage0111-ib.pocstor_35625_object5743/head//22 by client.940330.0:5744 2014-08-11 16:30:02.656987 2014-08-13 17:38:02.643839 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 2edd3220/benchmark_data_storage0111-ib.pocstor_35625_object8808/head//22 by client.940330.0:8809 2014-08-11 16:30:13.662150 2014-08-13 17:38:02.643860 7fe9bae887a0 20 read_log 192888'454 (0'0) modify d8068a20/benchmark_data_storage0111-ib.pocstor_35625_object9121/head//22 by client.940330.0:9122 2014-08-11 16:30:14.755318 2014-08-13 17:38:02.643881 7fe9bae887a0 20 read_log 192888'455 (0'0) modify ffe40220/benchmark_data_storage0111-ib.pocstor_35625_object9551/head//22 by client.940330.0:9552 2014-08-11 16:30:16.258341 2014-08-13 17:38:02.643902 7fe9bae887a0 20 read_log 192888'456 (0'0) modify 2f067220/benchmark_data_storage0111-ib.pocstor_35625_object10780/head//22 by client.940330.0:10781 2014-08-11 16:30:20.946971 2014-08-13 17:38:02.643923 7fe9bae887a0 20 read_log 192888'457 (0'0) modify edd4aa20/benchmark_data_storage0111-ib.pocstor_35625_object12386/head//22 by client.940330.0:12387 2014-08-11 16:30:27.456582 2014-08-13 17:38:02.643943 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 4dd7da20/benchmark_data_storage0111-ib.pocstor_35625_object12499/head//22 by client.940330.0:12500 2014-08-11 16:30:27.890650 2014-08-13 17:38:02.643964 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 274eba20/benchmark_data_storage0111-ib.pocstor_35625_object14106/head//22 by client.940330.0:14107 2014-08-11 16:30:34.138783 2014-08-13 17:38:02.643985 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 82d6da20/benchmark_data_storage0111-ib.pocstor_35625_object17347/head//22 by client.940330.0:17348 2014-08-11 16:30:46.264014 2014-08-13 17:38:02.644005 7fe9bae887a0 20 read_log 192888'461 (0'0) modify e3284a20/benchmark_data_storage0111-ib.pocstor_35625_object18589/head//22 by client.940330.0:18590 2014-08-11 16:30:51.093356 2014-08-13 17:38:02.644026 7fe9bae887a0 20 read_log 192888'462 (0'0) modify bd89e220/benchmark_data_storage0111-ib.pocstor_35625_object18980/head//22 by client.940330.0:18981 2014-08-11 16:30:52.595003 2014-08-13 17:38:02.644046 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 2f326a20/benchmark_data_storage0111-ib.pocstor_35625_object20637/head//22 by client.940330.0:20638 2014-08-11 16:30:59.200285 2014-08-13 17:38:02.644067 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 92dc1a20/benchmark_data_storage0111-ib.pocstor_35625_object24416/head//22 by client.940330.0:24417 2014-08-11 16:31:13.662052 2014-08-13 17:38:02.644087 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 85b54220/benchmark_data_storage0111-ib.pocstor_35625_object24660/head//22 by client.940330.0:24661 2014-08-11 16:31:14.564175 2014-08-13 17:38:02.644108 7fe9bae887a0 20 read_log 192888'466 (192888'450) delete e903ea20/benchmark_data_storage0111-ib.pocstor_35625_object941/head//22 by client.940330.0:32249 2014-08-11 16:31:41.735824 2014-08-13 17:38:02.644129 7fe9bae887a0 20 read_log 192888'467 (192888'451) delete 65670220/benchmark_data_storage0111-ib.pocstor_35625_object4252/head//22 by client.940330.0:35560 2014-08-11 16:31:42.607456 2014-08-13 17:38:02.644150 7fe9bae887a0 20 read_log 192888'468 (192888'452) delete 3ce8ca20/benchmark_data_storage0111-ib.pocstor_35625_object5743/head//22 by client.940330.0:37051 2014-08-11 16:31:43.008377 2014-08-13 17:38:02.644182 7fe9bae887a0 20 read_log 192888'469 (192888'453) delete 2edd3220/benchmark_data_storage0111-ib.pocstor_35625_object8808/head//22 by client.940330.0:40116 2014-08-11 16:31:43.793671 2014-08-13 17:38:02.644246 7fe9bae887a0 20 read_log 192888'470 (192888'454) delete d8068a20/benchmark_data_storage0111-ib.pocstor_35625_object9121/head//22 by client.940330.0:40429 2014-08-11 16:31:43.871541 2014-08-13 17:38:02.644270 7fe9bae887a0 20 read_log 192888'471 (192888'455) delete ffe40220/benchmark_data_storage0111-ib.pocstor_35625_object9551/head//22 by client.940330.0:40859 2014-08-11 16:31:43.990218 2014-08-13 17:38:02.644291 7fe9bae887a0 20 read_log 192888'472 (192888'456) delete 2f067220/benchmark_data_storage0111-ib.pocstor_35625_object10780/head//22 by client.940330.0:42088 2014-08-11 16:31:44.336405 2014-08-13 17:38:02.644313 7fe9bae887a0 20 read_log 192888'473 (192888'457) delete edd4aa20/benchmark_data_storage0111-ib.pocstor_35625_object12386/head//22 by client.940330.0:43694 2014-08-11 16:31:44.753447 2014-08-13 17:38:02.644334 7fe9bae887a0 20 read_log 192888'474 (192888'458) delete 4dd7da20/benchmark_data_storage0111-ib.pocstor_35625_object12499/head//22 by client.940330.0:43807 2014-08-11 16:31:44.782374 2014-08-13 17:38:02.644355 7fe9bae887a0 20 read_log 192888'475 (192888'459) delete 274eba20/benchmark_data_storage0111-ib.pocstor_35625_object14106/head//22 by client.940330.0:45414 2014-08-11 16:31:45.211039 2014-08-13 17:38:02.644376 7fe9bae887a0 20 read_log 192888'476 (192888'460) delete 82d6da20/benchmark_data_storage0111-ib.pocstor_35625_object17347/head//22 by client.940330.0:48655 2014-08-11 16:31:46.088339 2014-08-13 17:38:02.644397 7fe9bae887a0 20 read_log 192888'477 (192888'461) delete e3284a20/benchmark_data_storage0111-ib.pocstor_35625_object18589/head//22 by client.940330.0:49897 2014-08-11 16:31:46.470984 2014-08-13 17:38:02.644418 7fe9bae887a0 20 read_log 192888'478 (192888'462) delete bd89e220/benchmark_data_storage0111-ib.pocstor_35625_object18980/head//22 by client.940330.0:50288 2014-08-11 16:31:46.576234 2014-08-13 17:38:02.644439 7fe9bae887a0 20 read_log 192888'479 (192888'463) delete 2f326a20/benchmark_data_storage0111-ib.pocstor_35625_object20637/head//22 by client.940330.0:51945 2014-08-11 16:31:47.062947 2014-08-13 17:38:02.644460 7fe9bae887a0 20 read_log 192888'480 (192888'464) delete 92dc1a20/benchmark_data_storage0111-ib.pocstor_35625_object24416/head//22 by client.940330.0:55724 2014-08-11 16:31:48.162523 2014-08-13 17:38:02.644481 7fe9bae887a0 20 read_log 192888'481 (192888'465) delete 85b54220/benchmark_data_storage0111-ib.pocstor_35625_object24660/head//22 by client.940330.0:55968 2014-08-11 16:31:48.233028 2014-08-13 17:38:02.644502 7fe9bae887a0 20 read_log 194215'482 (0'0) modify daa2d220/benchmark_data_storage0111-ib.pocstor_64498_object2015/head//22 by client.1049635.0:2016 2014-08-12 17:17:23.601792 2014-08-13 17:38:02.644524 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 81737220/benchmark_data_storage0111-ib.pocstor_64498_object3735/head//22 by client.1049635.0:3736 2014-08-12 17:17:29.197328 2014-08-13 17:38:02.644544 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 70ce3220/benchmark_data_storage0111-ib.pocstor_64498_object4977/head//22 by client.1049635.0:4978 2014-08-12 17:17:32.864213 2014-08-13 17:38:02.644565 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 120e3a20/benchmark_data_storage0111-ib.pocstor_64498_object7336/head//22 by client.1049635.0:7337 2014-08-12 17:17:40.076761 2014-08-13 17:38:02.644586 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 3cff3a20/benchmark_data_storage0111-ib.pocstor_64498_object8659/head//22 by client.1049635.0:8660 2014-08-12 17:17:44.050581 2014-08-13 17:38:02.644606 7fe9bae887a0 20 read_log 194215'487 (0'0) modify de9e5a20/benchmark_data_storage0111-ib.pocstor_64498_object8728/head//22 by client.1049635.0:8729 2014-08-12 17:17:44.250155 2014-08-13 17:38:02.644627 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 9237a220/benchmark_data_storage0111-ib.pocstor_64498_object11397/head//22 by client.1049635.0:11398 2014-08-12 17:17:52.504411 2014-08-13 17:38:02.644663 7fe9bae887a0 20 read_log 194215'489 (0'0) modify d8282220/benchmark_data_storage0111-ib.pocstor_64498_object11630/head//22 by client.1049635.0:11631 2014-08-12 17:17:53.300539 2014-08-13 17:38:02.644686 7fe9bae887a0 20 read_log 194215'490 (0'0) modify f78eb220/benchmark_data_storage0111-ib.pocstor_64498_object11845/head//22 by client.1049635.0:11846 2014-08-12 17:17:53.949671 2014-08-13 17:38:02.644707 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 87457220/benchmark_data_storage0111-ib.pocstor_64498_object16664/head//22 by client.1049635.0:16665 2014-08-12 17:18:10.085813 2014-08-13 17:38:02.644728 7fe9bae887a0 20 read_log 194215'492 (0'0) modify 6d694220/benchmark_data_storage0111-ib.pocstor_64498_object18060/head//22 by client.1049635.0:18061 2014-08-12 17:18:15.392886 2014-08-13 17:38:02.644749 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 91fd6220/benchmark_data_storage0111-ib.pocstor_64498_object18135/head//22 by client.1049635.0:18136 2014-08-12 17:18:15.672932 2014-08-13 17:38:02.644770 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 882f2a20/benchmark_data_storage0111-ib.pocstor_64498_object19891/head//22 by client.1049635.0:19892 2014-08-12 17:18:22.368645 2014-08-13 17:38:02.644791 7fe9bae887a0 20 read_log 194215'495 (0'0) modify fa663a20/benchmark_data_storage0111-ib.pocstor_64498_object19967/head//22 by client.1049635.0:19968 2014-08-12 17:18:22.659504 2014-08-13 17:38:02.644812 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 615e7220/benchmark_data_storage0111-ib.pocstor_64498_object21627/head//22 by client.1049635.0:21628 2014-08-12 17:18:28.929614 2014-08-13 17:38:02.644833 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 484f4220/benchmark_data_storage0111-ib.pocstor_64498_object24344/head//22 by client.1049635.0:24345 2014-08-12 17:18:39.137481 2014-08-13 17:38:02.644853 7fe9bae887a0 20 read_log 194215'498 (0'0) modify ace15220/benchmark_data_storage0111-ib.pocstor_64498_object26081/head//22 by client.1049635.0:26082 2014-08-12 17:18:45.725292 2014-08-13 17:38:02.644874 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 15490220/benchmark_data_storage0111-ib.pocstor_64498_object28324/head//22 by client.1049635.0:28325 2014-08-12 17:18:54.084046 2014-08-13 17:38:02.644895 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 7c1a0220/benchmark_data_storage0111-ib.pocstor_64498_object30361/head//22 by client.1049635.0:30362 2014-08-12 17:19:01.981843 2014-08-13 17:38:02.644916 7fe9bae887a0 20 read_log 194215'501 (0'0) modify a7166220/benchmark_data_storage0111-ib.pocstor_64498_object33579/head//22 by client.1049635.0:33580 2014-08-12 17:19:14.586261 2014-08-13 17:38:02.644937 7fe9bae887a0 20 read_log 194215'502 (194215'482) delete daa2d220/benchmark_data_storage0111-ib.pocstor_64498_object2015/head//22 by client.1049635.0:36218 2014-08-12 17:19:18.652846 2014-08-13 17:38:02.644958 7fe9bae887a0 20 read_log 194215'503 (194215'483) delete 81737220/benchmark_data_storage0111-ib.pocstor_64498_object3735/head//22 by client.1049635.0:37938 2014-08-12 17:19:19.194478 2014-08-13 17:38:02.644979 7fe9bae887a0 20 read_log 194215'504 (194215'484) delete 70ce3220/benchmark_data_storage0111-ib.pocstor_64498_object4977/head//22 by client.1049635.0:39180 2014-08-12 17:19:19.595325 2014-08-13 17:38:02.645000 7fe9bae887a0 20 read_log 194215'505 (194215'485) delete 120e3a20/benchmark_data_storage0111-ib.pocstor_64498_object7336/head//22 by client.1049635.0:41539 2014-08-12 17:19:20.395976 2014-08-13 17:38:02.645021 7fe9bae887a0 20 read_log 194215'506 (194215'486) delete 3cff3a20/benchmark_data_storage0111-ib.pocstor_64498_object8659/head//22 by client.1049635.0:42862 2014-08-12 17:19:20.817571 2014-08-13 17:38:02.645042 7fe9bae887a0 20 read_log 194215'507 (194215'487) delete de9e5a20/benchmark_data_storage0111-ib.pocstor_64498_object8728/head//22 by client.1049635.0:42931 2014-08-12 17:19:20.840932 2014-08-13 17:38:02.645063 7fe9bae887a0 20 read_log 194215'508 (194215'488) delete 9237a220/benchmark_data_storage0111-ib.pocstor_64498_object11397/head//22 by client.1049635.0:45600 2014-08-12 17:19:21.703329 2014-08-13 17:38:02.645097 7fe9bae887a0 20 read_log 194215'509 (194215'489) delete d8282220/benchmark_data_storage0111-ib.pocstor_64498_object11630/head//22 by client.1049635.0:45833 2014-08-12 17:19:21.776101 2014-08-13 17:38:02.645122 7fe9bae887a0 20 read_log 194215'510 (194215'490) delete f78eb220/benchmark_data_storage0111-ib.pocstor_64498_object11845/head//22 by client.1049635.0:46048 2014-08-12 17:19:21.854274 2014-08-13 17:38:02.645143 7fe9bae887a0 20 read_log 194215'511 (194215'491) delete 87457220/benchmark_data_storage0111-ib.pocstor_64498_object16664/head//22 by client.1049635.0:50867 2014-08-12 17:19:23.401820 2014-08-13 17:38:02.645173 7fe9bae887a0 20 read_log 194215'512 (194215'492) delete 6d694220/benchmark_data_storage0111-ib.pocstor_64498_object18060/head//22 by client.1049635.0:52263 2014-08-12 17:19:23.851232 2014-08-13 17:38:02.645195 7fe9bae887a0 20 read_log 194215'513 (194215'493) delete 91fd6220/benchmark_data_storage0111-ib.pocstor_64498_object18135/head//22 by client.1049635.0:52338 2014-08-12 17:19:23.874911 2014-08-13 17:38:02.645216 7fe9bae887a0 20 read_log 194215'514 (194215'494) delete 882f2a20/benchmark_data_storage0111-ib.pocstor_64498_object19891/head//22 by client.1049635.0:54094 2014-08-12 17:19:24.475404 2014-08-13 17:38:02.645389 7fe9bae887a0 20 read_log 194215'515 (194215'495) delete fa663a20/benchmark_data_storage0111-ib.pocstor_64498_object19967/head//22 by client.1049635.0:54170 2014-08-12 17:19:24.501717 2014-08-13 17:38:02.645415 7fe9bae887a0 20 read_log 194215'516 (194215'496) delete 615e7220/benchmark_data_storage0111-ib.pocstor_64498_object21627/head//22 by client.1049635.0:55830 2014-08-12 17:19:25.098019 2014-08-13 17:38:02.645436 7fe9bae887a0 20 read_log 194215'517 (194215'497) delete 484f4220/benchmark_data_storage0111-ib.pocstor_64498_object24344/head//22 by client.1049635.0:58547 2014-08-12 17:19:26.081970 2014-08-13 17:38:02.645458 7fe9bae887a0 20 read_log 194215'518 (194215'498) delete ace15220/benchmark_data_storage0111-ib.pocstor_64498_object26081/head//22 by client.1049635.0:60284 2014-08-12 17:19:26.684738 2014-08-13 17:38:02.645481 7fe9bae887a0 20 read_log 194215'519 (194215'499) delete 15490220/benchmark_data_storage0111-ib.pocstor_64498_object28324/head//22 by client.1049635.0:62527 2014-08-12 17:19:27.462991 2014-08-13 17:38:02.645502 7fe9bae887a0 20 read_log 194215'520 (194215'500) delete 7c1a0220/benchmark_data_storage0111-ib.pocstor_64498_object30361/head//22 by client.1049635.0:64564 2014-08-12 17:19:28.162993 2014-08-13 17:38:02.645524 7fe9bae887a0 20 read_log 194215'521 (194215'501) delete a7166220/benchmark_data_storage0111-ib.pocstor_64498_object33579/head//22 by client.1049635.0:67782 2014-08-12 17:19:29.322810 2014-08-13 17:38:02.645681 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.645708 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=0 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.645731 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=0 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.003302 0 0.000000 2014-08-13 17:38:02.645751 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=0 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.645765 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=0 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194478 2014-08-13 17:38:02.645779 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] log((0'0,194215'521], crt=192888'481) 2014-08-13 17:38:02.645795 7fe9bae887a0 10 osd.4 194478 pgid 22.226 coll 22.226_head 2014-08-13 17:38:02.646013 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.226 2014-08-13 17:38:02.646024 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.646087 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.226(unlocked)] enter Initial 2014-08-13 17:38:02.646109 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.226(unlocked)] enter NotTrimming 2014-08-13 17:38:02.646335 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.646557 7fe9bae887a0 20 read_log 191648'79 (0'0) modify 5beeaa26/benchmark_data_storage0111-ib.pocstor_40745_object5094/head//22 by client.273158.0:5095 2014-07-30 14:09:11.434191 2014-08-13 17:38:02.646593 7fe9bae887a0 20 read_log 191648'87 (0'0) modify b564b226/benchmark_data_storage0111-ib.pocstor_40745_object5447/head//22 by client.273158.0:5448 2014-07-30 14:09:13.219988 2014-08-13 17:38:02.646616 7fe9bae887a0 20 read_log 191648'135 (0'0) modify 26c2aa26/benchmark_data_storage0111-ib.pocstor_40745_object8810/head//22 by client.273158.0:8811 2014-07-30 14:09:30.424843 2014-08-13 17:38:02.646638 7fe9bae887a0 20 read_log 191648'183 (0'0) modify e0ea9226/benchmark_data_storage0111-ib.pocstor_40745_object12276/head//22 by client.273158.0:12277 2014-07-30 14:09:48.976178 2014-08-13 17:38:02.646659 7fe9bae887a0 20 read_log 191648'186 (0'0) modify 533a226/benchmark_data_storage0111-ib.pocstor_40745_object12558/head//22 by client.273158.0:12559 2014-07-30 14:09:50.348134 2014-08-13 17:38:02.646680 7fe9bae887a0 20 read_log 191648'253 (0'0) modify f140a26/benchmark_data_storage0111-ib.pocstor_40745_object16781/head//22 by client.273158.0:16782 2014-07-30 14:10:11.620938 2014-08-13 17:38:02.646701 7fe9bae887a0 20 read_log 191648'254 (0'0) modify 70cc2226/benchmark_data_storage0111-ib.pocstor_40745_object16947/head//22 by client.273158.0:16948 2014-07-30 14:10:12.765066 2014-08-13 17:38:02.646722 7fe9bae887a0 20 read_log 191648'258 (0'0) modify f2a2fa26/benchmark_data_storage0111-ib.pocstor_40745_object17298/head//22 by client.273158.0:17299 2014-07-30 14:10:14.743060 2014-08-13 17:38:02.646742 7fe9bae887a0 20 read_log 191727'348 (0'0) modify 43d2ca26/benchmark_data_storage0111-ib.pocstor_51285_object980/head//22 by client.275175.0:981 2014-07-30 14:18:37.381352 2014-08-13 17:38:02.646763 7fe9bae887a0 20 read_log 191727'349 (0'0) modify e76be226/benchmark_data_storage0111-ib.pocstor_51285_object1838/head//22 by client.275175.0:1839 2014-07-30 14:18:40.370995 2014-08-13 17:38:02.646784 7fe9bae887a0 20 read_log 191727'353 (0'0) modify 9feffa26/benchmark_data_storage0111-ib.pocstor_51285_object3273/head//22 by client.275175.0:3274 2014-07-30 14:18:45.081706 2014-08-13 17:38:02.646818 7fe9bae887a0 20 read_log 191727'354 (0'0) modify 66e3ba26/benchmark_data_storage0111-ib.pocstor_51285_object3294/head//22 by client.275175.0:3295 2014-07-30 14:18:45.146923 2014-08-13 17:38:02.646840 7fe9bae887a0 20 read_log 191727'355 (0'0) modify 9f24aa26/benchmark_data_storage0111-ib.pocstor_51285_object4200/head//22 by client.275175.0:4201 2014-07-30 14:18:47.964507 2014-08-13 17:38:02.646860 7fe9bae887a0 20 read_log 191727'357 (0'0) modify f4072226/benchmark_data_storage0111-ib.pocstor_51285_object4791/head//22 by client.275175.0:4792 2014-07-30 14:18:49.732130 2014-08-13 17:38:02.646881 7fe9bae887a0 20 read_log 191727'359 (0'0) modify 7ef38226/benchmark_data_storage0111-ib.pocstor_51285_object5242/head//22 by client.275175.0:5243 2014-07-30 14:18:51.066689 2014-08-13 17:38:02.646902 7fe9bae887a0 20 read_log 191727'363 (0'0) modify 351f2226/benchmark_data_storage0111-ib.pocstor_51285_object6926/head//22 by client.275175.0:6927 2014-07-30 14:18:56.511740 2014-08-13 17:38:02.646924 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 1c452a26/benchmark_data_storage0111-ib.pocstor_51285_object11110/head//22 by client.275175.0:11111 2014-07-30 14:19:08.352260 2014-08-13 17:38:02.646944 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 44974a26/benchmark_data_storage0111-ib.pocstor_51285_object13020/head//22 by client.275175.0:13021 2014-07-30 14:19:14.753154 2014-08-13 17:38:02.646965 7fe9bae887a0 20 read_log 191727'381 (0'0) modify 6eddea26/benchmark_data_storage0111-ib.pocstor_51285_object18207/head//22 by client.275175.0:18208 2014-07-30 14:19:30.579104 2014-08-13 17:38:02.646987 7fe9bae887a0 20 read_log 191727'387 (0'0) modify 417f7a26/benchmark_data_storage0111-ib.pocstor_51285_object25116/head//22 by client.275175.0:25117 2014-07-30 14:19:52.081451 2014-08-13 17:38:02.647010 7fe9bae887a0 20 read_log 191727'390 (0'0) modify a156f226/benchmark_data_storage0111-ib.pocstor_51285_object25875/head//22 by client.275175.0:25876 2014-07-30 14:19:54.379013 2014-08-13 17:38:02.647033 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 316b4a26/benchmark_data_storage0111-ib.pocstor_51285_object26998/head//22 by client.275175.0:26999 2014-07-30 14:19:57.807183 2014-08-13 17:38:02.647054 7fe9bae887a0 20 read_log 191727'393 (0'0) modify c5382a26/benchmark_data_storage0111-ib.pocstor_51285_object27844/head//22 by client.275175.0:27845 2014-07-30 14:20:00.384169 2014-08-13 17:38:02.647075 7fe9bae887a0 20 read_log 191727'394 (0'0) modify d759ea26/benchmark_data_storage0111-ib.pocstor_51285_object27845/head//22 by client.275175.0:27846 2014-07-30 14:20:00.388669 2014-08-13 17:38:02.647096 7fe9bae887a0 20 read_log 191727'404 (0'0) modify 31e4a226/benchmark_data_storage0111-ib.pocstor_51285_object31642/head//22 by client.275175.0:31643 2014-07-30 14:20:11.850327 2014-08-13 17:38:02.647116 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 59b5226/benchmark_data_storage0111-ib.pocstor_51285_object32712/head//22 by client.275175.0:32713 2014-07-30 14:20:14.920491 2014-08-13 17:38:02.647137 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 296226/benchmark_data_storage0111-ib.pocstor_51285_object32919/head//22 by client.275175.0:32920 2014-07-30 14:20:15.571968 2014-08-13 17:38:02.647173 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 2d612a26/benchmark_data_storage0111-ib.pocstor_51285_object34599/head//22 by client.275175.0:34600 2014-07-30 14:20:20.830881 2014-08-13 17:38:02.647196 7fe9bae887a0 20 read_log 191727'414 (0'0) modify befcca26/benchmark_data_storage0111-ib.pocstor_51285_object35253/head//22 by client.275175.0:35254 2014-07-30 14:20:22.730597 2014-08-13 17:38:02.647217 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 874ffa26/benchmark_data_storage0111-ib.pocstor_51285_object35730/head//22 by client.275175.0:35731 2014-07-30 14:20:24.352034 2014-08-13 17:38:02.647238 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 39971a26/benchmark_data_storage0111-ib.pocstor_51285_object35961/head//22 by client.275175.0:35962 2014-07-30 14:20:25.143962 2014-08-13 17:38:02.647274 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 75ee6226/benchmark_data_storage0111-ib.pocstor_51285_object38501/head//22 by client.275175.0:38502 2014-07-30 14:20:32.671869 2014-08-13 17:38:02.647296 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 8edbd226/benchmark_data_storage0111-ib.pocstor_51285_object38593/head//22 by client.275175.0:38594 2014-07-30 14:20:32.936333 2014-08-13 17:38:02.647317 7fe9bae887a0 20 read_log 192888'426 (0'0) modify 28169226/benchmark_data_storage0111-ib.pocstor_35625_object930/head//22 by client.940330.0:931 2014-08-11 16:29:44.421330 2014-08-13 17:38:02.647338 7fe9bae887a0 20 read_log 192888'427 (0'0) modify c0a2e226/benchmark_data_storage0111-ib.pocstor_35625_object2409/head//22 by client.940330.0:2410 2014-08-11 16:29:50.032639 2014-08-13 17:38:02.647359 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 7a624226/benchmark_data_storage0111-ib.pocstor_35625_object3210/head//22 by client.940330.0:3211 2014-08-11 16:29:53.377910 2014-08-13 17:38:02.647380 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 5bbd5226/benchmark_data_storage0111-ib.pocstor_35625_object7386/head//22 by client.940330.0:7387 2014-08-11 16:30:08.679068 2014-08-13 17:38:02.647401 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 69cf6226/benchmark_data_storage0111-ib.pocstor_35625_object7675/head//22 by client.940330.0:7676 2014-08-11 16:30:09.690306 2014-08-13 17:38:02.647422 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 87567a26/benchmark_data_storage0111-ib.pocstor_35625_object8402/head//22 by client.940330.0:8403 2014-08-11 16:30:12.249023 2014-08-13 17:38:02.647443 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 6a30c226/benchmark_data_storage0111-ib.pocstor_35625_object8972/head//22 by client.940330.0:8973 2014-08-11 16:30:14.237050 2014-08-13 17:38:02.647464 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 723e9a26/benchmark_data_storage0111-ib.pocstor_35625_object9822/head//22 by client.940330.0:9823 2014-08-11 16:30:17.208055 2014-08-13 17:38:02.647484 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 523a1226/benchmark_data_storage0111-ib.pocstor_35625_object12469/head//22 by client.940330.0:12470 2014-08-11 16:30:27.778100 2014-08-13 17:38:02.647505 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 6a83a26/benchmark_data_storage0111-ib.pocstor_35625_object15689/head//22 by client.940330.0:15690 2014-08-11 16:30:40.052246 2014-08-13 17:38:02.647526 7fe9bae887a0 20 read_log 192888'436 (0'0) modify abe2d226/benchmark_data_storage0111-ib.pocstor_35625_object17639/head//22 by client.940330.0:17640 2014-08-11 16:30:47.363707 2014-08-13 17:38:02.647546 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 46e56226/benchmark_data_storage0111-ib.pocstor_35625_object21450/head//22 by client.940330.0:21451 2014-08-11 16:31:02.498031 2014-08-13 17:38:02.647567 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 71520226/benchmark_data_storage0111-ib.pocstor_35625_object22343/head//22 by client.940330.0:22344 2014-08-11 16:31:05.842558 2014-08-13 17:38:02.647588 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 67012a26/benchmark_data_storage0111-ib.pocstor_35625_object23662/head//22 by client.940330.0:23663 2014-08-11 16:31:10.889579 2014-08-13 17:38:02.647640 7fe9bae887a0 20 read_log 192888'440 (0'0) modify da86a226/benchmark_data_storage0111-ib.pocstor_35625_object23987/head//22 by client.940330.0:23988 2014-08-11 16:31:12.104568 2014-08-13 17:38:02.647661 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 9f30c226/benchmark_data_storage0111-ib.pocstor_35625_object26445/head//22 by client.940330.0:26446 2014-08-11 16:31:21.555578 2014-08-13 17:38:02.647682 7fe9bae887a0 20 read_log 192888'442 (0'0) modify b5366a26/benchmark_data_storage0111-ib.pocstor_35625_object26754/head//22 by client.940330.0:26755 2014-08-11 16:31:22.852783 2014-08-13 17:38:02.647744 7fe9bae887a0 20 read_log 192888'443 (0'0) modify a849e226/benchmark_data_storage0111-ib.pocstor_35625_object30156/head//22 by client.940330.0:30157 2014-08-11 16:31:36.047032 2014-08-13 17:38:02.647767 7fe9bae887a0 20 read_log 192888'444 (192888'426) delete 28169226/benchmark_data_storage0111-ib.pocstor_35625_object930/head//22 by client.940330.0:32238 2014-08-11 16:31:41.732762 2014-08-13 17:38:02.647821 7fe9bae887a0 20 read_log 192888'445 (192888'427) delete c0a2e226/benchmark_data_storage0111-ib.pocstor_35625_object2409/head//22 by client.940330.0:33717 2014-08-11 16:31:42.097585 2014-08-13 17:38:02.647843 7fe9bae887a0 20 read_log 192888'446 (192888'428) delete 7a624226/benchmark_data_storage0111-ib.pocstor_35625_object3210/head//22 by client.940330.0:34518 2014-08-11 16:31:42.310660 2014-08-13 17:38:02.647865 7fe9bae887a0 20 read_log 192888'447 (192888'429) delete 5bbd5226/benchmark_data_storage0111-ib.pocstor_35625_object7386/head//22 by client.940330.0:38694 2014-08-11 16:31:43.435033 2014-08-13 17:38:02.647886 7fe9bae887a0 20 read_log 192888'448 (192888'430) delete 69cf6226/benchmark_data_storage0111-ib.pocstor_35625_object7675/head//22 by client.940330.0:38983 2014-08-11 16:31:43.509036 2014-08-13 17:38:02.647907 7fe9bae887a0 20 read_log 192888'449 (192888'431) delete 87567a26/benchmark_data_storage0111-ib.pocstor_35625_object8402/head//22 by client.940330.0:39710 2014-08-11 16:31:43.690306 2014-08-13 17:38:02.647928 7fe9bae887a0 20 read_log 192888'450 (192888'432) delete 6a30c226/benchmark_data_storage0111-ib.pocstor_35625_object8972/head//22 by client.940330.0:40280 2014-08-11 16:31:43.834632 2014-08-13 17:38:02.647949 7fe9bae887a0 20 read_log 192888'451 (192888'433) delete 723e9a26/benchmark_data_storage0111-ib.pocstor_35625_object9822/head//22 by client.940330.0:41130 2014-08-11 16:31:44.064228 2014-08-13 17:38:02.647971 7fe9bae887a0 20 read_log 192888'452 (192888'434) delete 523a1226/benchmark_data_storage0111-ib.pocstor_35625_object12469/head//22 by client.940330.0:43777 2014-08-11 16:31:44.774837 2014-08-13 17:38:02.647992 7fe9bae887a0 20 read_log 192888'453 (192888'435) delete 6a83a26/benchmark_data_storage0111-ib.pocstor_35625_object15689/head//22 by client.940330.0:46997 2014-08-11 16:31:45.640074 2014-08-13 17:38:02.648013 7fe9bae887a0 20 read_log 192888'454 (192888'436) delete abe2d226/benchmark_data_storage0111-ib.pocstor_35625_object17639/head//22 by client.940330.0:48947 2014-08-11 16:31:46.173835 2014-08-13 17:38:02.648034 7fe9bae887a0 20 read_log 192888'455 (192888'437) delete 46e56226/benchmark_data_storage0111-ib.pocstor_35625_object21450/head//22 by client.940330.0:52758 2014-08-11 16:31:47.281494 2014-08-13 17:38:02.648055 7fe9bae887a0 20 read_log 192888'456 (192888'438) delete 71520226/benchmark_data_storage0111-ib.pocstor_35625_object22343/head//22 by client.940330.0:53651 2014-08-11 16:31:47.557965 2014-08-13 17:38:02.648076 7fe9bae887a0 20 read_log 192888'457 (192888'439) delete 67012a26/benchmark_data_storage0111-ib.pocstor_35625_object23662/head//22 by client.940330.0:54970 2014-08-11 16:31:47.940802 2014-08-13 17:38:02.648097 7fe9bae887a0 20 read_log 192888'458 (192888'440) delete da86a226/benchmark_data_storage0111-ib.pocstor_35625_object23987/head//22 by client.940330.0:55295 2014-08-11 16:31:48.034722 2014-08-13 17:38:02.648118 7fe9bae887a0 20 read_log 192888'459 (192888'441) delete 9f30c226/benchmark_data_storage0111-ib.pocstor_35625_object26445/head//22 by client.940330.0:57753 2014-08-11 16:31:48.776965 2014-08-13 17:38:02.648139 7fe9bae887a0 20 read_log 192888'460 (192888'442) delete b5366a26/benchmark_data_storage0111-ib.pocstor_35625_object26754/head//22 by client.940330.0:58062 2014-08-11 16:31:48.858663 2014-08-13 17:38:02.648168 7fe9bae887a0 20 read_log 192888'461 (192888'443) delete a849e226/benchmark_data_storage0111-ib.pocstor_35625_object30156/head//22 by client.940330.0:61464 2014-08-11 16:31:49.803069 2014-08-13 17:38:02.648191 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 9a5d0a26/benchmark_data_storage0111-ib.pocstor_64498_object12036/head//22 by client.1049635.0:12037 2014-08-12 17:17:54.507665 2014-08-13 17:38:02.648225 7fe9bae887a0 20 read_log 194215'463 (0'0) modify e05ca226/benchmark_data_storage0111-ib.pocstor_64498_object12225/head//22 by client.1049635.0:12226 2014-08-12 17:17:55.146239 2014-08-13 17:38:02.648247 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 9b348a26/benchmark_data_storage0111-ib.pocstor_64498_object16311/head//22 by client.1049635.0:16312 2014-08-12 17:18:08.737621 2014-08-13 17:38:02.648268 7fe9bae887a0 20 read_log 194215'465 (0'0) modify c255226/benchmark_data_storage0111-ib.pocstor_64498_object16715/head//22 by client.1049635.0:16716 2014-08-12 17:18:10.286659 2014-08-13 17:38:02.648289 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 5fa5ca26/benchmark_data_storage0111-ib.pocstor_64498_object18878/head//22 by client.1049635.0:18879 2014-08-12 17:18:18.470497 2014-08-13 17:38:02.648310 7fe9bae887a0 20 read_log 194215'467 (0'0) modify fa698226/benchmark_data_storage0111-ib.pocstor_64498_object21474/head//22 by client.1049635.0:21475 2014-08-12 17:18:28.360914 2014-08-13 17:38:02.648332 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 9000ea26/benchmark_data_storage0111-ib.pocstor_64498_object21572/head//22 by client.1049635.0:21573 2014-08-12 17:18:28.725271 2014-08-13 17:38:02.648353 7fe9bae887a0 20 read_log 194215'469 (0'0) modify d4445a26/benchmark_data_storage0111-ib.pocstor_64498_object26701/head//22 by client.1049635.0:26702 2014-08-12 17:18:48.035766 2014-08-13 17:38:02.648378 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 31ba5226/benchmark_data_storage0111-ib.pocstor_64498_object27404/head//22 by client.1049635.0:27405 2014-08-12 17:18:50.662091 2014-08-13 17:38:02.648400 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 79839226/benchmark_data_storage0111-ib.pocstor_64498_object31667/head//22 by client.1049635.0:31668 2014-08-12 17:19:07.282091 2014-08-13 17:38:02.648423 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 50bcd226/benchmark_data_storage0111-ib.pocstor_64498_object32530/head//22 by client.1049635.0:32531 2014-08-12 17:19:10.604750 2014-08-13 17:38:02.648476 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 2bcaea26/benchmark_data_storage0111-ib.pocstor_64498_object33220/head//22 by client.1049635.0:33221 2014-08-12 17:19:13.226539 2014-08-13 17:38:02.648513 7fe9bae887a0 20 read_log 194215'474 (194215'462) delete 9a5d0a26/benchmark_data_storage0111-ib.pocstor_64498_object12036/head//22 by client.1049635.0:46239 2014-08-12 17:19:21.910020 2014-08-13 17:38:02.648536 7fe9bae887a0 20 read_log 194215'475 (194215'463) delete e05ca226/benchmark_data_storage0111-ib.pocstor_64498_object12225/head//22 by client.1049635.0:46428 2014-08-12 17:19:21.967181 2014-08-13 17:38:02.648586 7fe9bae887a0 20 read_log 194215'476 (194215'464) delete 9b348a26/benchmark_data_storage0111-ib.pocstor_64498_object16311/head//22 by client.1049635.0:50514 2014-08-12 17:19:23.288652 2014-08-13 17:38:02.648612 7fe9bae887a0 20 read_log 194215'477 (194215'465) delete c255226/benchmark_data_storage0111-ib.pocstor_64498_object16715/head//22 by client.1049635.0:50918 2014-08-12 17:19:23.421406 2014-08-13 17:38:02.648634 7fe9bae887a0 20 read_log 194215'478 (194215'466) delete 5fa5ca26/benchmark_data_storage0111-ib.pocstor_64498_object18878/head//22 by client.1049635.0:53081 2014-08-12 17:19:24.123799 2014-08-13 17:38:02.648656 7fe9bae887a0 20 read_log 194215'479 (194215'467) delete fa698226/benchmark_data_storage0111-ib.pocstor_64498_object21474/head//22 by client.1049635.0:55677 2014-08-12 17:19:25.048645 2014-08-13 17:38:02.648680 7fe9bae887a0 20 read_log 194215'480 (194215'468) delete 9000ea26/benchmark_data_storage0111-ib.pocstor_64498_object21572/head//22 by client.1049635.0:55775 2014-08-12 17:19:25.079939 2014-08-13 17:38:02.648704 7fe9bae887a0 20 read_log 194215'481 (194215'469) delete d4445a26/benchmark_data_storage0111-ib.pocstor_64498_object26701/head//22 by client.1049635.0:60904 2014-08-12 17:19:26.890537 2014-08-13 17:38:02.648725 7fe9bae887a0 20 read_log 194215'482 (194215'470) delete 31ba5226/benchmark_data_storage0111-ib.pocstor_64498_object27404/head//22 by client.1049635.0:61607 2014-08-12 17:19:27.146544 2014-08-13 17:38:02.648764 7fe9bae887a0 20 read_log 194215'483 (194215'471) delete 79839226/benchmark_data_storage0111-ib.pocstor_64498_object31667/head//22 by client.1049635.0:65870 2014-08-12 17:19:28.627095 2014-08-13 17:38:02.648786 7fe9bae887a0 20 read_log 194215'484 (194215'472) delete 50bcd226/benchmark_data_storage0111-ib.pocstor_64498_object32530/head//22 by client.1049635.0:66733 2014-08-12 17:19:28.941097 2014-08-13 17:38:02.648808 7fe9bae887a0 20 read_log 194215'485 (194215'473) delete 2bcaea26/benchmark_data_storage0111-ib.pocstor_64498_object33220/head//22 by client.1049635.0:67423 2014-08-12 17:19:29.194183 2014-08-13 17:38:02.648955 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.648987 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=0 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.649013 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=0 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.002927 0 0.000000 2014-08-13 17:38:02.649033 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=0 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.649047 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=0 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194478 2014-08-13 17:38:02.649062 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] log((0'0,194215'485], crt=192888'461) 2014-08-13 17:38:02.649079 7fe9bae887a0 10 osd.4 194478 pgid 22.22c coll 22.22c_head 2014-08-13 17:38:02.649319 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.22c 2014-08-13 17:38:02.649336 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.649381 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.22c(unlocked)] enter Initial 2014-08-13 17:38:02.649398 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.22c(unlocked)] enter NotTrimming 2014-08-13 17:38:02.649617 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.649795 7fe9bae887a0 20 read_log 191648'4 (0'0) modify a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 by client.273158.0:85 2014-07-30 14:08:49.018175 2014-08-13 17:38:02.649832 7fe9bae887a0 20 read_log 191648'33 (0'0) modify 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 by client.273158.0:1707 2014-07-30 14:08:55.050437 2014-08-13 17:38:02.649854 7fe9bae887a0 20 read_log 191648'48 (0'0) modify cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 by client.273158.0:2886 2014-07-30 14:09:00.935167 2014-08-13 17:38:02.649876 7fe9bae887a0 20 read_log 191648'79 (0'0) modify 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 by client.273158.0:4481 2014-07-30 14:09:08.657039 2014-08-13 17:38:02.649897 7fe9bae887a0 20 read_log 191648'108 (0'0) modify a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 by client.273158.0:6329 2014-07-30 14:09:17.707193 2014-08-13 17:38:02.649918 7fe9bae887a0 20 read_log 191648'137 (0'0) modify 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 by client.273158.0:8601 2014-07-30 14:09:29.583263 2014-08-13 17:38:02.649939 7fe9bae887a0 20 read_log 191648'165 (0'0) modify 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 by client.273158.0:10170 2014-07-30 14:09:36.691027 2014-08-13 17:38:02.649963 7fe9bae887a0 20 read_log 191648'272 (0'0) modify 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 by client.273158.0:17240 2014-07-30 14:10:14.414754 2014-08-13 17:38:02.649984 7fe9bae887a0 20 read_log 191648'309 (0'0) modify 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 by client.273158.0:19165 2014-07-30 14:10:24.252127 2014-08-13 17:38:02.650005 7fe9bae887a0 20 read_log 191648'319 (0'0) modify f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 by client.273158.0:19544 2014-07-30 14:10:26.613669 2014-08-13 17:38:02.650026 7fe9bae887a0 20 read_log 191648'351 (0'0) modify e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 by client.273158.0:22218 2014-07-30 14:10:45.059339 2014-08-13 17:38:02.650048 7fe9bae887a0 20 read_log 191727'364 (0'0) modify 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 by client.275175.0:306 2014-07-30 14:18:34.607368 2014-08-13 17:38:02.650069 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 by client.275175.0:1488 2014-07-30 14:18:39.166845 2014-08-13 17:38:02.650093 7fe9bae887a0 20 read_log 191727'367 (0'0) modify 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 by client.275175.0:1567 2014-07-30 14:18:39.442377 2014-08-13 17:38:02.650128 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 by client.275175.0:2315 2014-07-30 14:18:41.922939 2014-08-13 17:38:02.650151 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 by client.275175.0:4220 2014-07-30 14:18:48.020490 2014-08-13 17:38:02.650184 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 by client.275175.0:4949 2014-07-30 14:18:50.201939 2014-08-13 17:38:02.650210 7fe9bae887a0 20 read_log 191727'381 (0'0) modify f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 by client.275175.0:6030 2014-07-30 14:18:53.332588 2014-08-13 17:38:02.650231 7fe9bae887a0 20 read_log 191727'382 (0'0) modify ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 by client.275175.0:6141 2014-07-30 14:18:53.697273 2014-08-13 17:38:02.650252 7fe9bae887a0 20 read_log 191727'386 (0'0) modify ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 by client.275175.0:8352 2014-07-30 14:19:00.504331 2014-08-13 17:38:02.650273 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 by client.275175.0:9585 2014-07-30 14:19:04.025852 2014-08-13 17:38:02.650294 7fe9bae887a0 20 read_log 191727'392 (0'0) modify f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 by client.275175.0:10497 2014-07-30 14:19:06.629527 2014-08-13 17:38:02.650315 7fe9bae887a0 20 read_log 191727'399 (0'0) modify fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 by client.275175.0:14386 2014-07-30 14:19:18.967524 2014-08-13 17:38:02.650336 7fe9bae887a0 20 read_log 191727'401 (0'0) modify e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 by client.275175.0:16209 2014-07-30 14:19:24.659592 2014-08-13 17:38:02.650356 7fe9bae887a0 20 read_log 191727'406 (0'0) modify fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 by client.275175.0:18817 2014-07-30 14:19:32.614990 2014-08-13 17:38:02.650377 7fe9bae887a0 20 read_log 191727'408 (0'0) modify df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 by client.275175.0:19175 2014-07-30 14:19:33.867449 2014-08-13 17:38:02.650397 7fe9bae887a0 20 read_log 191727'409 (0'0) modify ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 by client.275175.0:19483 2014-07-30 14:19:34.729754 2014-08-13 17:38:02.650418 7fe9bae887a0 20 read_log 191727'412 (0'0) modify a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 by client.275175.0:21428 2014-07-30 14:19:40.412254 2014-08-13 17:38:02.650438 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 by client.275175.0:23618 2014-07-30 14:19:47.757239 2014-08-13 17:38:02.650459 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 by client.275175.0:25895 2014-07-30 14:19:54.430752 2014-08-13 17:38:02.650479 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 by client.275175.0:26500 2014-07-30 14:19:56.358670 2014-08-13 17:38:02.650500 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 by client.275175.0:28003 2014-07-30 14:20:01.036910 2014-08-13 17:38:02.650521 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 by client.275175.0:31104 2014-07-30 14:20:10.277085 2014-08-13 17:38:02.650544 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 by client.275175.0:31578 2014-07-30 14:20:11.666639 2014-08-13 17:38:02.650577 7fe9bae887a0 20 read_log 191727'432 (0'0) modify a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 by client.275175.0:31976 2014-07-30 14:20:12.783822 2014-08-13 17:38:02.650599 7fe9bae887a0 20 read_log 191727'434 (0'0) modify d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 by client.275175.0:32956 2014-07-30 14:20:15.674789 2014-08-13 17:38:02.650619 7fe9bae887a0 20 read_log 191727'437 (0'0) modify 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 by client.275175.0:34706 2014-07-30 14:20:21.133299 2014-08-13 17:38:02.650640 7fe9bae887a0 20 read_log 191727'440 (0'0) modify 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 by client.275175.0:36163 2014-07-30 14:20:25.787147 2014-08-13 17:38:02.650661 7fe9bae887a0 20 read_log 192888'447 (0'0) modify f520ea2c/benchmark_data_storage0111-ib.pocstor_35625_object1093/head//22 by client.940330.0:1094 2014-08-11 16:29:45.140333 2014-08-13 17:38:02.650682 7fe9bae887a0 20 read_log 192888'448 (0'0) modify fd913a2c/benchmark_data_storage0111-ib.pocstor_35625_object6790/head//22 by client.940330.0:6791 2014-08-11 16:30:06.582473 2014-08-13 17:38:02.650702 7fe9bae887a0 20 read_log 192888'449 (0'0) modify c9c5b22c/benchmark_data_storage0111-ib.pocstor_35625_object7325/head//22 by client.940330.0:7326 2014-08-11 16:30:08.466079 2014-08-13 17:38:02.650723 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 92138a2c/benchmark_data_storage0111-ib.pocstor_35625_object15082/head//22 by client.940330.0:15083 2014-08-11 16:30:37.799336 2014-08-13 17:38:02.650743 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 2fb0fa2c/benchmark_data_storage0111-ib.pocstor_35625_object15095/head//22 by client.940330.0:15096 2014-08-11 16:30:37.846177 2014-08-13 17:38:02.650764 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 431f7a2c/benchmark_data_storage0111-ib.pocstor_35625_object18501/head//22 by client.940330.0:18502 2014-08-11 16:30:50.772548 2014-08-13 17:38:02.650784 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 1da4b22c/benchmark_data_storage0111-ib.pocstor_35625_object19175/head//22 by client.940330.0:19176 2014-08-11 16:30:53.317936 2014-08-13 17:38:02.650804 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 7ff722c/benchmark_data_storage0111-ib.pocstor_35625_object24393/head//22 by client.940330.0:24394 2014-08-11 16:31:13.578587 2014-08-13 17:38:02.650824 7fe9bae887a0 20 read_log 192888'455 (0'0) modify 8c32922c/benchmark_data_storage0111-ib.pocstor_35625_object27444/head//22 by client.940330.0:27445 2014-08-11 16:31:25.521409 2014-08-13 17:38:02.650845 7fe9bae887a0 20 read_log 192888'456 (0'0) modify f3f4da2c/benchmark_data_storage0111-ib.pocstor_35625_object27828/head//22 by client.940330.0:27829 2014-08-11 16:31:27.014716 2014-08-13 17:38:02.650865 7fe9bae887a0 20 read_log 192888'457 (192888'447) delete f520ea2c/benchmark_data_storage0111-ib.pocstor_35625_object1093/head//22 by client.940330.0:32401 2014-08-11 16:31:41.772162 2014-08-13 17:38:02.650887 7fe9bae887a0 20 read_log 192888'458 (192888'448) delete fd913a2c/benchmark_data_storage0111-ib.pocstor_35625_object6790/head//22 by client.940330.0:38098 2014-08-11 16:31:43.285386 2014-08-13 17:38:02.650908 7fe9bae887a0 20 read_log 192888'459 (192888'449) delete c9c5b22c/benchmark_data_storage0111-ib.pocstor_35625_object7325/head//22 by client.940330.0:38633 2014-08-11 16:31:43.419293 2014-08-13 17:38:02.650929 7fe9bae887a0 20 read_log 192888'460 (192888'450) delete 92138a2c/benchmark_data_storage0111-ib.pocstor_35625_object15082/head//22 by client.940330.0:46390 2014-08-11 16:31:45.478417 2014-08-13 17:38:02.650950 7fe9bae887a0 20 read_log 192888'461 (192888'451) delete 2fb0fa2c/benchmark_data_storage0111-ib.pocstor_35625_object15095/head//22 by client.940330.0:46403 2014-08-11 16:31:45.480921 2014-08-13 17:38:02.650990 7fe9bae887a0 20 read_log 192888'462 (192888'452) delete 431f7a2c/benchmark_data_storage0111-ib.pocstor_35625_object18501/head//22 by client.940330.0:49809 2014-08-11 16:31:46.447227 2014-08-13 17:38:02.651013 7fe9bae887a0 20 read_log 192888'463 (192888'453) delete 1da4b22c/benchmark_data_storage0111-ib.pocstor_35625_object19175/head//22 by client.940330.0:50483 2014-08-11 16:31:46.627118 2014-08-13 17:38:02.651034 7fe9bae887a0 20 read_log 192888'464 (192888'454) delete 7ff722c/benchmark_data_storage0111-ib.pocstor_35625_object24393/head//22 by client.940330.0:55701 2014-08-11 16:31:48.153730 2014-08-13 17:38:02.651056 7fe9bae887a0 20 read_log 192888'465 (192888'455) delete 8c32922c/benchmark_data_storage0111-ib.pocstor_35625_object27444/head//22 by client.940330.0:58752 2014-08-11 16:31:49.052237 2014-08-13 17:38:02.651077 7fe9bae887a0 20 read_log 192888'466 (192888'456) delete f3f4da2c/benchmark_data_storage0111-ib.pocstor_35625_object27828/head//22 by client.940330.0:59136 2014-08-11 16:31:49.159514 2014-08-13 17:38:02.651098 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 4999a22c/benchmark_data_storage0111-ib.pocstor_64498_object95/head//22 by client.1049635.0:96 2014-08-12 17:17:17.321755 2014-08-13 17:38:02.651119 7fe9bae887a0 20 read_log 194215'468 (0'0) modify ef8ca2c/benchmark_data_storage0111-ib.pocstor_64498_object522/head//22 by client.1049635.0:523 2014-08-12 17:17:18.801354 2014-08-13 17:38:02.651140 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 43617a2c/benchmark_data_storage0111-ib.pocstor_64498_object1409/head//22 by client.1049635.0:1410 2014-08-12 17:17:21.723544 2014-08-13 17:38:02.651168 7fe9bae887a0 20 read_log 194215'470 (0'0) modify cc3a322c/benchmark_data_storage0111-ib.pocstor_64498_object2728/head//22 by client.1049635.0:2729 2014-08-12 17:17:25.919402 2014-08-13 17:38:02.651190 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 9e98922c/benchmark_data_storage0111-ib.pocstor_64498_object3040/head//22 by client.1049635.0:3041 2014-08-12 17:17:26.956656 2014-08-13 17:38:02.651211 7fe9bae887a0 20 read_log 194215'472 (0'0) modify b7c4d22c/benchmark_data_storage0111-ib.pocstor_64498_object4936/head//22 by client.1049635.0:4937 2014-08-12 17:17:32.753191 2014-08-13 17:38:02.651231 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 10f1fa2c/benchmark_data_storage0111-ib.pocstor_64498_object6276/head//22 by client.1049635.0:6277 2014-08-12 17:17:36.922032 2014-08-13 17:38:02.651252 7fe9bae887a0 20 read_log 194215'474 (0'0) modify c5e61a2c/benchmark_data_storage0111-ib.pocstor_64498_object9119/head//22 by client.1049635.0:9120 2014-08-12 17:17:45.362286 2014-08-13 17:38:02.651272 7fe9bae887a0 20 read_log 194215'475 (0'0) modify cb8faa2c/benchmark_data_storage0111-ib.pocstor_64498_object13032/head//22 by client.1049635.0:13033 2014-08-12 17:17:57.488064 2014-08-13 17:38:02.651293 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 4e848a2c/benchmark_data_storage0111-ib.pocstor_64498_object16398/head//22 by client.1049635.0:16399 2014-08-12 17:18:09.077039 2014-08-13 17:38:02.651315 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 29e82a2c/benchmark_data_storage0111-ib.pocstor_64498_object17030/head//22 by client.1049635.0:17031 2014-08-12 17:18:11.475226 2014-08-13 17:38:02.651336 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 1571d22c/benchmark_data_storage0111-ib.pocstor_64498_object18729/head//22 by client.1049635.0:18730 2014-08-12 17:18:17.919097 2014-08-13 17:38:02.651357 7fe9bae887a0 20 read_log 194215'479 (0'0) modify b31e822c/benchmark_data_storage0111-ib.pocstor_64498_object19736/head//22 by client.1049635.0:19737 2014-08-12 17:18:21.791546 2014-08-13 17:38:02.651378 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 95c0922c/benchmark_data_storage0111-ib.pocstor_64498_object24070/head//22 by client.1049635.0:24071 2014-08-12 17:18:38.111832 2014-08-13 17:38:02.651453 7fe9bae887a0 20 read_log 194215'481 (0'0) modify aa10a2c/benchmark_data_storage0111-ib.pocstor_64498_object26853/head//22 by client.1049635.0:26854 2014-08-12 17:18:48.601610 2014-08-13 17:38:02.651477 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 1f99d22c/benchmark_data_storage0111-ib.pocstor_64498_object27334/head//22 by client.1049635.0:27335 2014-08-12 17:18:50.394231 2014-08-13 17:38:02.651500 7fe9bae887a0 20 read_log 194215'483 (0'0) modify e8a5d22c/benchmark_data_storage0111-ib.pocstor_64498_object27510/head//22 by client.1049635.0:27511 2014-08-12 17:18:51.058586 2014-08-13 17:38:02.651521 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 58d3c22c/benchmark_data_storage0111-ib.pocstor_64498_object28531/head//22 by client.1049635.0:28532 2014-08-12 17:18:54.859383 2014-08-13 17:38:02.651542 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 58916a2c/benchmark_data_storage0111-ib.pocstor_64498_object29137/head//22 by client.1049635.0:29138 2014-08-12 17:18:57.106440 2014-08-13 17:38:02.651563 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 395d922c/benchmark_data_storage0111-ib.pocstor_64498_object29811/head//22 by client.1049635.0:29812 2014-08-12 17:18:59.741211 2014-08-13 17:38:02.651583 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 5fbada2c/benchmark_data_storage0111-ib.pocstor_64498_object29889/head//22 by client.1049635.0:29890 2014-08-12 17:19:00.034149 2014-08-13 17:38:02.651604 7fe9bae887a0 20 read_log 194215'488 (0'0) modify e1b76a2c/benchmark_data_storage0111-ib.pocstor_64498_object30695/head//22 by client.1049635.0:30696 2014-08-12 17:19:03.269800 2014-08-13 17:38:02.651651 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 30bd7a2c/benchmark_data_storage0111-ib.pocstor_64498_object31147/head//22 by client.1049635.0:31148 2014-08-12 17:19:05.133313 2014-08-13 17:38:02.651673 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 8f37322c/benchmark_data_storage0111-ib.pocstor_64498_object31995/head//22 by client.1049635.0:31996 2014-08-12 17:19:08.554610 2014-08-13 17:38:02.651694 7fe9bae887a0 20 read_log 194215'491 (194215'467) delete 4999a22c/benchmark_data_storage0111-ib.pocstor_64498_object95/head//22 by client.1049635.0:34298 2014-08-12 17:19:18.012152 2014-08-13 17:38:02.651716 7fe9bae887a0 20 read_log 194215'492 (194215'468) delete ef8ca2c/benchmark_data_storage0111-ib.pocstor_64498_object522/head//22 by client.1049635.0:34725 2014-08-12 17:19:18.166130 2014-08-13 17:38:02.651737 7fe9bae887a0 20 read_log 194215'493 (194215'469) delete 43617a2c/benchmark_data_storage0111-ib.pocstor_64498_object1409/head//22 by client.1049635.0:35612 2014-08-12 17:19:18.454796 2014-08-13 17:38:02.651758 7fe9bae887a0 20 read_log 194215'494 (194215'470) delete cc3a322c/benchmark_data_storage0111-ib.pocstor_64498_object2728/head//22 by client.1049635.0:36931 2014-08-12 17:19:18.876406 2014-08-13 17:38:02.651779 7fe9bae887a0 20 read_log 194215'495 (194215'471) delete 9e98922c/benchmark_data_storage0111-ib.pocstor_64498_object3040/head//22 by client.1049635.0:37243 2014-08-12 17:19:18.980881 2014-08-13 17:38:02.651800 7fe9bae887a0 20 read_log 194215'496 (194215'472) delete b7c4d22c/benchmark_data_storage0111-ib.pocstor_64498_object4936/head//22 by client.1049635.0:39139 2014-08-12 17:19:19.582320 2014-08-13 17:38:02.651821 7fe9bae887a0 20 read_log 194215'497 (194215'473) delete 10f1fa2c/benchmark_data_storage0111-ib.pocstor_64498_object6276/head//22 by client.1049635.0:40479 2014-08-12 17:19:20.046112 2014-08-13 17:38:02.651842 7fe9bae887a0 20 read_log 194215'498 (194215'474) delete c5e61a2c/benchmark_data_storage0111-ib.pocstor_64498_object9119/head//22 by client.1049635.0:43322 2014-08-12 17:19:20.960325 2014-08-13 17:38:02.651863 7fe9bae887a0 20 read_log 194215'499 (194215'475) delete cb8faa2c/benchmark_data_storage0111-ib.pocstor_64498_object13032/head//22 by client.1049635.0:47235 2014-08-12 17:19:22.228760 2014-08-13 17:38:02.651884 7fe9bae887a0 20 read_log 194215'500 (194215'476) delete 4e848a2c/benchmark_data_storage0111-ib.pocstor_64498_object16398/head//22 by client.1049635.0:50601 2014-08-12 17:19:23.318639 2014-08-13 17:38:02.651918 7fe9bae887a0 20 read_log 194215'501 (194215'477) delete 29e82a2c/benchmark_data_storage0111-ib.pocstor_64498_object17030/head//22 by client.1049635.0:51233 2014-08-12 17:19:23.517296 2014-08-13 17:38:02.651941 7fe9bae887a0 20 read_log 194215'502 (194215'478) delete 1571d22c/benchmark_data_storage0111-ib.pocstor_64498_object18729/head//22 by client.1049635.0:52932 2014-08-12 17:19:24.073802 2014-08-13 17:38:02.651961 7fe9bae887a0 20 read_log 194215'503 (194215'479) delete b31e822c/benchmark_data_storage0111-ib.pocstor_64498_object19736/head//22 by client.1049635.0:53939 2014-08-12 17:19:24.426879 2014-08-13 17:38:02.651982 7fe9bae887a0 20 read_log 194215'504 (194215'480) delete 95c0922c/benchmark_data_storage0111-ib.pocstor_64498_object24070/head//22 by client.1049635.0:58273 2014-08-12 17:19:25.985682 2014-08-13 17:38:02.652006 7fe9bae887a0 20 read_log 194215'505 (194215'481) delete aa10a2c/benchmark_data_storage0111-ib.pocstor_64498_object26853/head//22 by client.1049635.0:61056 2014-08-12 17:19:26.946228 2014-08-13 17:38:02.652028 7fe9bae887a0 20 read_log 194215'506 (194215'482) delete 1f99d22c/benchmark_data_storage0111-ib.pocstor_64498_object27334/head//22 by client.1049635.0:61537 2014-08-12 17:19:27.119514 2014-08-13 17:38:02.652048 7fe9bae887a0 20 read_log 194215'507 (194215'483) delete e8a5d22c/benchmark_data_storage0111-ib.pocstor_64498_object27510/head//22 by client.1049635.0:61713 2014-08-12 17:19:27.183830 2014-08-13 17:38:02.652101 7fe9bae887a0 20 read_log 194215'508 (194215'484) delete 58d3c22c/benchmark_data_storage0111-ib.pocstor_64498_object28531/head//22 by client.1049635.0:62734 2014-08-12 17:19:27.534482 2014-08-13 17:38:02.652122 7fe9bae887a0 20 read_log 194215'509 (194215'485) delete 58916a2c/benchmark_data_storage0111-ib.pocstor_64498_object29137/head//22 by client.1049635.0:63340 2014-08-12 17:19:27.752110 2014-08-13 17:38:02.652143 7fe9bae887a0 20 read_log 194215'510 (194215'486) delete 395d922c/benchmark_data_storage0111-ib.pocstor_64498_object29811/head//22 by client.1049635.0:64014 2014-08-12 17:19:27.988983 2014-08-13 17:38:02.652177 7fe9bae887a0 20 read_log 194215'511 (194215'487) delete 5fbada2c/benchmark_data_storage0111-ib.pocstor_64498_object29889/head//22 by client.1049635.0:64092 2014-08-12 17:19:28.012836 2014-08-13 17:38:02.652199 7fe9bae887a0 20 read_log 194215'512 (194215'488) delete e1b76a2c/benchmark_data_storage0111-ib.pocstor_64498_object30695/head//22 by client.1049635.0:64898 2014-08-12 17:19:28.272136 2014-08-13 17:38:02.652220 7fe9bae887a0 20 read_log 194215'513 (194215'489) delete 30bd7a2c/benchmark_data_storage0111-ib.pocstor_64498_object31147/head//22 by client.1049635.0:65350 2014-08-12 17:19:28.431292 2014-08-13 17:38:02.652240 7fe9bae887a0 20 read_log 194215'514 (194215'490) delete 8f37322c/benchmark_data_storage0111-ib.pocstor_64498_object31995/head//22 by client.1049635.0:66198 2014-08-12 17:19:28.741494 2014-08-13 17:38:02.652405 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.652435 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=0 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.652459 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=0 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003079 0 0.000000 2014-08-13 17:38:02.652479 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=0 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.652493 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=0 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.652509 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'514], crt=191727'446) 2014-08-13 17:38:02.652525 7fe9bae887a0 10 osd.4 194478 pgid 22.28f coll 22.28f_head 2014-08-13 17:38:02.652736 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.28f 2014-08-13 17:38:02.652747 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.652792 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.28f(unlocked)] enter Initial 2014-08-13 17:38:02.652810 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.28f(unlocked)] enter NotTrimming 2014-08-13 17:38:02.653187 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.653374 7fe9bae887a0 20 read_log 191648'21 (0'0) modify fba01a8f/benchmark_data_storage0111-ib.pocstor_40745_object1856/head//22 by client.273158.0:1857 2014-07-30 14:08:55.607854 2014-08-13 17:38:02.653406 7fe9bae887a0 20 read_log 191648'69 (0'0) modify d62b3a8f/benchmark_data_storage0111-ib.pocstor_40745_object5137/head//22 by client.273158.0:5138 2014-07-30 14:09:11.692933 2014-08-13 17:38:02.653428 7fe9bae887a0 20 read_log 191648'82 (0'0) modify 4945728f/benchmark_data_storage0111-ib.pocstor_40745_object5835/head//22 by client.273158.0:5836 2014-07-30 14:09:15.068476 2014-08-13 17:38:02.653449 7fe9bae887a0 20 read_log 191648'102 (0'0) modify ffe228f/benchmark_data_storage0111-ib.pocstor_40745_object6850/head//22 by client.273158.0:6851 2014-07-30 14:09:20.504439 2014-08-13 17:38:02.653470 7fe9bae887a0 20 read_log 191648'131 (0'0) modify ae4e3a8f/benchmark_data_storage0111-ib.pocstor_40745_object8910/head//22 by client.273158.0:8911 2014-07-30 14:09:30.816666 2014-08-13 17:38:02.653491 7fe9bae887a0 20 read_log 191648'161 (0'0) modify 1719428f/benchmark_data_storage0111-ib.pocstor_40745_object10040/head//22 by client.273158.0:10041 2014-07-30 14:09:36.004066 2014-08-13 17:38:02.653513 7fe9bae887a0 20 read_log 191648'221 (0'0) modify f0c1528f/benchmark_data_storage0111-ib.pocstor_40745_object14371/head//22 by client.273158.0:14372 2014-07-30 14:09:59.558968 2014-08-13 17:38:02.653533 7fe9bae887a0 20 read_log 191648'251 (0'0) modify b2a9928f/benchmark_data_storage0111-ib.pocstor_40745_object16400/head//22 by client.273158.0:16401 2014-07-30 14:10:09.607719 2014-08-13 17:38:02.653554 7fe9bae887a0 20 read_log 191648'276 (0'0) modify 80fa3a8f/benchmark_data_storage0111-ib.pocstor_40745_object18322/head//22 by client.273158.0:18323 2014-07-30 14:10:19.764902 2014-08-13 17:38:02.653574 7fe9bae887a0 20 read_log 191648'294 (0'0) modify 1610628f/benchmark_data_storage0111-ib.pocstor_40745_object19372/head//22 by client.273158.0:19373 2014-07-30 14:10:25.733467 2014-08-13 17:38:02.653595 7fe9bae887a0 20 read_log 191648'303 (0'0) modify 9e0c828f/benchmark_data_storage0111-ib.pocstor_40745_object19886/head//22 by client.273158.0:19887 2014-07-30 14:10:30.869865 2014-08-13 17:38:02.653615 7fe9bae887a0 20 read_log 191648'309 (0'0) modify eb4b4a8f/benchmark_data_storage0111-ib.pocstor_40745_object20390/head//22 by client.273158.0:20391 2014-07-30 14:10:32.975475 2014-08-13 17:38:02.653636 7fe9bae887a0 20 read_log 191648'315 (0'0) modify c0ffc28f/benchmark_data_storage0111-ib.pocstor_40745_object20597/head//22 by client.273158.0:20598 2014-07-30 14:10:34.320948 2014-08-13 17:38:02.653656 7fe9bae887a0 20 read_log 191648'344 (0'0) modify 3d17228f/benchmark_data_storage0111-ib.pocstor_40745_object22319/head//22 by client.273158.0:22320 2014-07-30 14:10:46.009537 2014-08-13 17:38:02.653677 7fe9bae887a0 20 read_log 191727'353 (0'0) modify 711a828f/benchmark_data_storage0111-ib.pocstor_51285_object625/head//22 by client.275175.0:626 2014-07-30 14:18:35.904129 2014-08-13 17:38:02.653699 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 99411a8f/benchmark_data_storage0111-ib.pocstor_51285_object3094/head//22 by client.275175.0:3095 2014-07-30 14:18:44.522806 2014-08-13 17:38:02.653719 7fe9bae887a0 20 read_log 191727'362 (0'0) modify 814928f/benchmark_data_storage0111-ib.pocstor_51285_object3641/head//22 by client.275175.0:3642 2014-07-30 14:18:46.250602 2014-08-13 17:38:02.653756 7fe9bae887a0 20 read_log 191727'365 (0'0) modify 3c21328f/benchmark_data_storage0111-ib.pocstor_51285_object4804/head//22 by client.275175.0:4805 2014-07-30 14:18:49.770478 2014-08-13 17:38:02.653778 7fe9bae887a0 20 read_log 191727'370 (0'0) modify 7fb7828f/benchmark_data_storage0111-ib.pocstor_51285_object8195/head//22 by client.275175.0:8196 2014-07-30 14:19:00.066191 2014-08-13 17:38:02.653799 7fe9bae887a0 20 read_log 191727'376 (0'0) modify 4eea7a8f/benchmark_data_storage0111-ib.pocstor_51285_object12838/head//22 by client.275175.0:12839 2014-07-30 14:19:13.994371 2014-08-13 17:38:02.653820 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 2efd828f/benchmark_data_storage0111-ib.pocstor_51285_object14257/head//22 by client.275175.0:14258 2014-07-30 14:19:18.602821 2014-08-13 17:38:02.653841 7fe9bae887a0 20 read_log 191727'380 (0'0) modify b94d4a8f/benchmark_data_storage0111-ib.pocstor_51285_object14787/head//22 by client.275175.0:14788 2014-07-30 14:19:20.095525 2014-08-13 17:38:02.653861 7fe9bae887a0 20 read_log 191727'383 (0'0) modify 32dea8f/benchmark_data_storage0111-ib.pocstor_51285_object17769/head//22 by client.275175.0:17770 2014-07-30 14:19:29.308785 2014-08-13 17:38:02.653882 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 475db28f/benchmark_data_storage0111-ib.pocstor_51285_object18973/head//22 by client.275175.0:18974 2014-07-30 14:19:33.093278 2014-08-13 17:38:02.653903 7fe9bae887a0 20 read_log 191727'393 (0'0) modify 837f0a8f/benchmark_data_storage0111-ib.pocstor_51285_object25134/head//22 by client.275175.0:25135 2014-07-30 14:19:52.149190 2014-08-13 17:38:02.653923 7fe9bae887a0 20 read_log 191727'394 (0'0) modify bfcafa8f/benchmark_data_storage0111-ib.pocstor_51285_object25377/head//22 by client.275175.0:25378 2014-07-30 14:19:52.917477 2014-08-13 17:38:02.653944 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 1a3c5a8f/benchmark_data_storage0111-ib.pocstor_51285_object25503/head//22 by client.275175.0:25504 2014-07-30 14:19:53.270547 2014-08-13 17:38:02.653964 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 2e8ff28f/benchmark_data_storage0111-ib.pocstor_51285_object26529/head//22 by client.275175.0:26530 2014-07-30 14:19:56.444655 2014-08-13 17:38:02.653985 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 2224ca8f/benchmark_data_storage0111-ib.pocstor_51285_object27625/head//22 by client.275175.0:27626 2014-07-30 14:19:59.585256 2014-08-13 17:38:02.654005 7fe9bae887a0 20 read_log 191727'400 (0'0) modify 82876a8f/benchmark_data_storage0111-ib.pocstor_51285_object27747/head//22 by client.275175.0:27748 2014-07-30 14:19:59.977856 2014-08-13 17:38:02.654026 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 280128f/benchmark_data_storage0111-ib.pocstor_51285_object28968/head//22 by client.275175.0:28969 2014-07-30 14:20:03.799756 2014-08-13 17:38:02.654046 7fe9bae887a0 20 read_log 191727'406 (0'0) modify feb6ca8f/benchmark_data_storage0111-ib.pocstor_51285_object29765/head//22 by client.275175.0:29766 2014-07-30 14:20:06.258438 2014-08-13 17:38:02.654067 7fe9bae887a0 20 read_log 191727'409 (0'0) modify 78db928f/benchmark_data_storage0111-ib.pocstor_51285_object30073/head//22 by client.275175.0:30074 2014-07-30 14:20:07.154491 2014-08-13 17:38:02.654088 7fe9bae887a0 20 read_log 191727'410 (0'0) modify c1e0528f/benchmark_data_storage0111-ib.pocstor_51285_object30495/head//22 by client.275175.0:30496 2014-07-30 14:20:08.565892 2014-08-13 17:38:02.654109 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 4e63428f/benchmark_data_storage0111-ib.pocstor_51285_object32294/head//22 by client.275175.0:32295 2014-07-30 14:20:13.740559 2014-08-13 17:38:02.654130 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 8ec19a8f/benchmark_data_storage0111-ib.pocstor_51285_object33758/head//22 by client.275175.0:33759 2014-07-30 14:20:18.303809 2014-08-13 17:38:02.654163 7fe9bae887a0 20 read_log 191727'417 (0'0) modify d62dda8f/benchmark_data_storage0111-ib.pocstor_51285_object34064/head//22 by client.275175.0:34065 2014-07-30 14:20:19.175784 2014-08-13 17:38:02.654200 7fe9bae887a0 20 read_log 191727'419 (0'0) modify 5297ba8f/benchmark_data_storage0111-ib.pocstor_51285_object35422/head//22 by client.275175.0:35423 2014-07-30 14:20:23.224880 2014-08-13 17:38:02.654222 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 19f3928f/benchmark_data_storage0111-ib.pocstor_51285_object35711/head//22 by client.275175.0:35712 2014-07-30 14:20:24.184303 2014-08-13 17:38:02.654243 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 9922128f/benchmark_data_storage0111-ib.pocstor_51285_object37419/head//22 by client.275175.0:37420 2014-07-30 14:20:29.592777 2014-08-13 17:38:02.654264 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 3ef1628f/benchmark_data_storage0111-ib.pocstor_51285_object38330/head//22 by client.275175.0:38331 2014-07-30 14:20:32.185409 2014-08-13 17:38:02.654284 7fe9bae887a0 20 read_log 192674'430 (0'0) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4839 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654305 7fe9bae887a0 20 read_log 192674'431 (192674'430) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4840 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654326 7fe9bae887a0 20 read_log 192674'432 (192674'431) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4841 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654346 7fe9bae887a0 20 read_log 192674'433 (192674'432) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4842 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654367 7fe9bae887a0 20 read_log 192674'434 (192674'433) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4843 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654387 7fe9bae887a0 20 read_log 192674'435 (192674'434) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4844 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654407 7fe9bae887a0 20 read_log 192674'436 (192674'435) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4845 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654428 7fe9bae887a0 20 read_log 192674'437 (192674'436) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4846 2014-07-31 18:03:46.802000 2014-08-13 17:38:02.654449 7fe9bae887a0 20 read_log 192888'438 (0'0) modify cfa9ea8f/benchmark_data_storage0111-ib.pocstor_35625_object3856/head//22 by client.940330.0:3857 2014-08-11 16:29:55.804830 2014-08-13 17:38:02.654470 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 175dd28f/benchmark_data_storage0111-ib.pocstor_35625_object3875/head//22 by client.940330.0:3876 2014-08-11 16:29:55.874346 2014-08-13 17:38:02.654491 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 76af428f/benchmark_data_storage0111-ib.pocstor_35625_object10037/head//22 by client.940330.0:10038 2014-08-11 16:30:18.036344 2014-08-13 17:38:02.654512 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 1e90c28f/benchmark_data_storage0111-ib.pocstor_35625_object11027/head//22 by client.940330.0:11028 2014-08-11 16:30:21.904001 2014-08-13 17:38:02.654533 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 3795328f/benchmark_data_storage0111-ib.pocstor_35625_object11154/head//22 by client.940330.0:11155 2014-08-11 16:30:22.405929 2014-08-13 17:38:02.654553 7fe9bae887a0 20 read_log 192888'443 (0'0) modify acd28f/benchmark_data_storage0111-ib.pocstor_35625_object14560/head//22 by client.940330.0:14561 2014-08-11 16:30:35.855325 2014-08-13 17:38:02.654574 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 1c4f8a8f/benchmark_data_storage0111-ib.pocstor_35625_object19867/head//22 by client.940330.0:19868 2014-08-11 16:30:56.328983 2014-08-13 17:38:02.654595 7fe9bae887a0 20 read_log 192888'445 (0'0) modify e927428f/benchmark_data_storage0111-ib.pocstor_35625_object21493/head//22 by client.940330.0:21494 2014-08-11 16:31:02.652402 2014-08-13 17:38:02.654628 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 9ea78a8f/benchmark_data_storage0111-ib.pocstor_35625_object25950/head//22 by client.940330.0:25951 2014-08-11 16:31:19.598189 2014-08-13 17:38:02.654649 7fe9bae887a0 20 read_log 192888'447 (0'0) modify 5e8fca8f/benchmark_data_storage0111-ib.pocstor_35625_object26627/head//22 by client.940330.0:26628 2014-08-11 16:31:22.352056 2014-08-13 17:38:02.654670 7fe9bae887a0 20 read_log 192888'448 (0'0) modify ef611a8f/benchmark_data_storage0111-ib.pocstor_35625_object28786/head//22 by client.940330.0:28787 2014-08-11 16:31:30.767306 2014-08-13 17:38:02.654690 7fe9bae887a0 20 read_log 192888'449 (0'0) modify affc4a8f/benchmark_data_storage0111-ib.pocstor_35625_object30183/head//22 by client.940330.0:30184 2014-08-11 16:31:36.145393 2014-08-13 17:38:02.654711 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 8bf19a8f/benchmark_data_storage0111-ib.pocstor_35625_object30875/head//22 by client.940330.0:30876 2014-08-11 16:31:38.747546 2014-08-13 17:38:02.654732 7fe9bae887a0 20 read_log 192888'451 (192888'438) delete cfa9ea8f/benchmark_data_storage0111-ib.pocstor_35625_object3856/head//22 by client.940330.0:35164 2014-08-11 16:31:42.489926 2014-08-13 17:38:02.654754 7fe9bae887a0 20 read_log 192888'452 (192888'439) delete 175dd28f/benchmark_data_storage0111-ib.pocstor_35625_object3875/head//22 by client.940330.0:35183 2014-08-11 16:31:42.495559 2014-08-13 17:38:02.654814 7fe9bae887a0 20 read_log 192888'453 (192888'440) delete 76af428f/benchmark_data_storage0111-ib.pocstor_35625_object10037/head//22 by client.940330.0:41345 2014-08-11 16:31:44.123624 2014-08-13 17:38:02.654838 7fe9bae887a0 20 read_log 192888'454 (192888'441) delete 1e90c28f/benchmark_data_storage0111-ib.pocstor_35625_object11027/head//22 by client.940330.0:42335 2014-08-11 16:31:44.406913 2014-08-13 17:38:02.654859 7fe9bae887a0 20 read_log 192888'455 (192888'442) delete 3795328f/benchmark_data_storage0111-ib.pocstor_35625_object11154/head//22 by client.940330.0:42462 2014-08-11 16:31:44.443937 2014-08-13 17:38:02.654883 7fe9bae887a0 20 read_log 192888'456 (192888'443) delete acd28f/benchmark_data_storage0111-ib.pocstor_35625_object14560/head//22 by client.940330.0:45868 2014-08-11 16:31:45.335000 2014-08-13 17:38:02.654905 7fe9bae887a0 20 read_log 192888'457 (192888'444) delete 1c4f8a8f/benchmark_data_storage0111-ib.pocstor_35625_object19867/head//22 by client.940330.0:51175 2014-08-11 16:31:46.830389 2014-08-13 17:38:02.654926 7fe9bae887a0 20 read_log 192888'458 (192888'445) delete e927428f/benchmark_data_storage0111-ib.pocstor_35625_object21493/head//22 by client.940330.0:52801 2014-08-11 16:31:47.293627 2014-08-13 17:38:02.654947 7fe9bae887a0 20 read_log 192888'459 (192888'446) delete 9ea78a8f/benchmark_data_storage0111-ib.pocstor_35625_object25950/head//22 by client.940330.0:57258 2014-08-11 16:31:48.633681 2014-08-13 17:38:02.654968 7fe9bae887a0 20 read_log 192888'460 (192888'447) delete 5e8fca8f/benchmark_data_storage0111-ib.pocstor_35625_object26627/head//22 by client.940330.0:57935 2014-08-11 16:31:48.824593 2014-08-13 17:38:02.654989 7fe9bae887a0 20 read_log 192888'461 (192888'448) delete ef611a8f/benchmark_data_storage0111-ib.pocstor_35625_object28786/head//22 by client.940330.0:60094 2014-08-11 16:31:49.414662 2014-08-13 17:38:02.655010 7fe9bae887a0 20 read_log 192888'462 (192888'449) delete affc4a8f/benchmark_data_storage0111-ib.pocstor_35625_object30183/head//22 by client.940330.0:61491 2014-08-11 16:31:49.810039 2014-08-13 17:38:02.655051 7fe9bae887a0 20 read_log 192888'463 (192888'450) delete 8bf19a8f/benchmark_data_storage0111-ib.pocstor_35625_object30875/head//22 by client.940330.0:62183 2014-08-11 16:31:50.015629 2014-08-13 17:38:02.655074 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 9da8428f/benchmark_data_storage0111-ib.pocstor_64498_object51/head//22 by client.1049635.0:52 2014-08-12 17:17:17.146640 2014-08-13 17:38:02.655114 7fe9bae887a0 20 read_log 194215'465 (0'0) modify ea24828f/benchmark_data_storage0111-ib.pocstor_64498_object190/head//22 by client.1049635.0:191 2014-08-12 17:17:17.625114 2014-08-13 17:38:02.655137 7fe9bae887a0 20 read_log 194215'466 (0'0) modify c8af3a8f/benchmark_data_storage0111-ib.pocstor_64498_object1053/head//22 by client.1049635.0:1054 2014-08-12 17:17:20.586727 2014-08-13 17:38:02.655166 7fe9bae887a0 20 read_log 194215'467 (0'0) modify ba25f28f/benchmark_data_storage0111-ib.pocstor_64498_object4858/head//22 by client.1049635.0:4859 2014-08-12 17:17:32.498633 2014-08-13 17:38:02.655188 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 2e05f28f/benchmark_data_storage0111-ib.pocstor_64498_object5728/head//22 by client.1049635.0:5729 2014-08-12 17:17:35.278269 2014-08-13 17:38:02.655209 7fe9bae887a0 20 read_log 194215'469 (0'0) modify a29c1a8f/benchmark_data_storage0111-ib.pocstor_64498_object8190/head//22 by client.1049635.0:8191 2014-08-12 17:17:42.665876 2014-08-13 17:38:02.655232 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 8186028f/benchmark_data_storage0111-ib.pocstor_64498_object8787/head//22 by client.1049635.0:8788 2014-08-12 17:17:44.419176 2014-08-13 17:38:02.655252 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 18b5a8f/benchmark_data_storage0111-ib.pocstor_64498_object10205/head//22 by client.1049635.0:10206 2014-08-12 17:17:48.511178 2014-08-13 17:38:02.655273 7fe9bae887a0 20 read_log 194215'472 (0'0) modify ea7fca8f/benchmark_data_storage0111-ib.pocstor_64498_object12352/head//22 by client.1049635.0:12353 2014-08-12 17:17:55.491521 2014-08-13 17:38:02.655294 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 377d8a8f/benchmark_data_storage0111-ib.pocstor_64498_object13263/head//22 by client.1049635.0:13264 2014-08-12 17:17:58.202154 2014-08-13 17:38:02.655315 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 88536a8f/benchmark_data_storage0111-ib.pocstor_64498_object14782/head//22 by client.1049635.0:14783 2014-08-12 17:18:03.266048 2014-08-13 17:38:02.655336 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 418a6a8f/benchmark_data_storage0111-ib.pocstor_64498_object22899/head//22 by client.1049635.0:22900 2014-08-12 17:18:33.759711 2014-08-13 17:38:02.655357 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 49bb6a8f/benchmark_data_storage0111-ib.pocstor_64498_object24885/head//22 by client.1049635.0:24886 2014-08-12 17:18:41.233275 2014-08-13 17:38:02.655378 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 3c022a8f/benchmark_data_storage0111-ib.pocstor_64498_object24891/head//22 by client.1049635.0:24892 2014-08-12 17:18:41.264968 2014-08-13 17:38:02.655399 7fe9bae887a0 20 read_log 194215'478 (0'0) modify fdc9428f/benchmark_data_storage0111-ib.pocstor_64498_object25881/head//22 by client.1049635.0:25882 2014-08-12 17:18:44.981241 2014-08-13 17:38:02.655419 7fe9bae887a0 20 read_log 194215'479 (0'0) modify e988e28f/benchmark_data_storage0111-ib.pocstor_64498_object33720/head//22 by client.1049635.0:33721 2014-08-12 17:19:15.118184 2014-08-13 17:38:02.655440 7fe9bae887a0 20 read_log 194215'480 (194215'464) delete 9da8428f/benchmark_data_storage0111-ib.pocstor_64498_object51/head//22 by client.1049635.0:34254 2014-08-12 17:19:17.996438 2014-08-13 17:38:02.655462 7fe9bae887a0 20 read_log 194215'481 (194215'465) delete ea24828f/benchmark_data_storage0111-ib.pocstor_64498_object190/head//22 by client.1049635.0:34393 2014-08-12 17:19:18.048768 2014-08-13 17:38:02.655483 7fe9bae887a0 20 read_log 194215'482 (194215'466) delete c8af3a8f/benchmark_data_storage0111-ib.pocstor_64498_object1053/head//22 by client.1049635.0:35256 2014-08-12 17:19:18.338035 2014-08-13 17:38:02.655504 7fe9bae887a0 20 read_log 194215'483 (194215'467) delete ba25f28f/benchmark_data_storage0111-ib.pocstor_64498_object4858/head//22 by client.1049635.0:39061 2014-08-12 17:19:19.558205 2014-08-13 17:38:02.655525 7fe9bae887a0 20 read_log 194215'484 (194215'468) delete 2e05f28f/benchmark_data_storage0111-ib.pocstor_64498_object5728/head//22 by client.1049635.0:39931 2014-08-12 17:19:19.864606 2014-08-13 17:38:02.655587 7fe9bae887a0 20 read_log 194215'485 (194215'469) delete a29c1a8f/benchmark_data_storage0111-ib.pocstor_64498_object8190/head//22 by client.1049635.0:42393 2014-08-12 17:19:20.664573 2014-08-13 17:38:02.655639 7fe9bae887a0 20 read_log 194215'486 (194215'470) delete 8186028f/benchmark_data_storage0111-ib.pocstor_64498_object8787/head//22 by client.1049635.0:42990 2014-08-12 17:19:20.857336 2014-08-13 17:38:02.655662 7fe9bae887a0 20 read_log 194215'487 (194215'471) delete 18b5a8f/benchmark_data_storage0111-ib.pocstor_64498_object10205/head//22 by client.1049635.0:44408 2014-08-12 17:19:21.329823 2014-08-13 17:38:02.655684 7fe9bae887a0 20 read_log 194215'488 (194215'472) delete ea7fca8f/benchmark_data_storage0111-ib.pocstor_64498_object12352/head//22 by client.1049635.0:46555 2014-08-12 17:19:22.008293 2014-08-13 17:38:02.655705 7fe9bae887a0 20 read_log 194215'489 (194215'473) delete 377d8a8f/benchmark_data_storage0111-ib.pocstor_64498_object13263/head//22 by client.1049635.0:47466 2014-08-12 17:19:22.310773 2014-08-13 17:38:02.655729 7fe9bae887a0 20 read_log 194215'490 (194215'474) delete 88536a8f/benchmark_data_storage0111-ib.pocstor_64498_object14782/head//22 by client.1049635.0:48985 2014-08-12 17:19:22.808377 2014-08-13 17:38:02.655751 7fe9bae887a0 20 read_log 194215'491 (194215'475) delete 418a6a8f/benchmark_data_storage0111-ib.pocstor_64498_object22899/head//22 by client.1049635.0:57102 2014-08-12 17:19:25.579191 2014-08-13 17:38:02.655772 7fe9bae887a0 20 read_log 194215'492 (194215'476) delete 49bb6a8f/benchmark_data_storage0111-ib.pocstor_64498_object24885/head//22 by client.1049635.0:59088 2014-08-12 17:19:26.284165 2014-08-13 17:38:02.655792 7fe9bae887a0 20 read_log 194215'493 (194215'477) delete 3c022a8f/benchmark_data_storage0111-ib.pocstor_64498_object24891/head//22 by client.1049635.0:59094 2014-08-12 17:19:26.285426 2014-08-13 17:38:02.655813 7fe9bae887a0 20 read_log 194215'494 (194215'478) delete fdc9428f/benchmark_data_storage0111-ib.pocstor_64498_object25881/head//22 by client.1049635.0:60084 2014-08-12 17:19:26.620124 2014-08-13 17:38:02.655833 7fe9bae887a0 20 read_log 194215'495 (194215'479) delete e988e28f/benchmark_data_storage0111-ib.pocstor_64498_object33720/head//22 by client.1049635.0:67923 2014-08-12 17:19:29.371767 2014-08-13 17:38:02.655993 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.656021 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=0 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.656044 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=0 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.003253 0 0.000000 2014-08-13 17:38:02.656064 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=0 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.656078 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=0 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.656093 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] log((0'0,194215'495], crt=192888'463) 2014-08-13 17:38:02.656110 7fe9bae887a0 10 osd.4 194478 pgid 22.29a coll 22.29a_head 2014-08-13 17:38:02.656346 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.29a 2014-08-13 17:38:02.656358 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.656405 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.29a(unlocked)] enter Initial 2014-08-13 17:38:02.656420 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.29a(unlocked)] enter NotTrimming 2014-08-13 17:38:02.656625 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.656807 7fe9bae887a0 20 read_log 191648'13 (0'0) modify 689fa29a/benchmark_data_storage0111-ib.pocstor_40745_object537/head//22 by client.273158.0:538 2014-07-30 14:08:50.784943 2014-08-13 17:38:02.656844 7fe9bae887a0 20 read_log 191648'41 (0'0) modify 90b5ca9a/benchmark_data_storage0111-ib.pocstor_40745_object2071/head//22 by client.273158.0:2072 2014-07-30 14:08:56.563366 2014-08-13 17:38:02.656867 7fe9bae887a0 20 read_log 191648'151 (0'0) modify 70ae429a/benchmark_data_storage0111-ib.pocstor_40745_object9938/head//22 by client.273158.0:9939 2014-07-30 14:09:35.518262 2014-08-13 17:38:02.656889 7fe9bae887a0 20 read_log 191648'152 (0'0) modify 40dd329a/benchmark_data_storage0111-ib.pocstor_40745_object9953/head//22 by client.273158.0:9954 2014-07-30 14:09:35.613585 2014-08-13 17:38:02.656910 7fe9bae887a0 20 read_log 191648'233 (0'0) modify 7023129a/benchmark_data_storage0111-ib.pocstor_40745_object14605/head//22 by client.273158.0:14606 2014-07-30 14:10:00.612468 2014-08-13 17:38:02.656932 7fe9bae887a0 20 read_log 191648'237 (0'0) modify 334a429a/benchmark_data_storage0111-ib.pocstor_40745_object14901/head//22 by client.273158.0:14902 2014-07-30 14:10:02.313537 2014-08-13 17:38:02.656953 7fe9bae887a0 20 read_log 191648'297 (0'0) modify 68297a9a/benchmark_data_storage0111-ib.pocstor_40745_object19664/head//22 by client.273158.0:19665 2014-07-30 14:10:27.682482 2014-08-13 17:38:02.656986 7fe9bae887a0 20 read_log 191727'362 (0'0) modify 88fe1a9a/benchmark_data_storage0111-ib.pocstor_51285_object3877/head//22 by client.275175.0:3878 2014-07-30 14:18:46.978801 2014-08-13 17:38:02.657012 7fe9bae887a0 20 read_log 191727'366 (0'0) modify ec98fa9a/benchmark_data_storage0111-ib.pocstor_51285_object6988/head//22 by client.275175.0:6989 2014-07-30 14:18:56.685492 2014-08-13 17:38:02.657034 7fe9bae887a0 20 read_log 191727'369 (0'0) modify ae04329a/benchmark_data_storage0111-ib.pocstor_51285_object8973/head//22 by client.275175.0:8974 2014-07-30 14:19:02.272865 2014-08-13 17:38:02.657055 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 8b24829a/benchmark_data_storage0111-ib.pocstor_51285_object10296/head//22 by client.275175.0:10297 2014-07-30 14:19:06.053286 2014-08-13 17:38:02.657076 7fe9bae887a0 20 read_log 191727'374 (0'0) modify 89c9a9a/benchmark_data_storage0111-ib.pocstor_51285_object10997/head//22 by client.275175.0:10998 2014-07-30 14:19:08.036743 2014-08-13 17:38:02.657097 7fe9bae887a0 20 read_log 191727'377 (0'0) modify cc2a0a9a/benchmark_data_storage0111-ib.pocstor_51285_object12431/head//22 by client.275175.0:12432 2014-07-30 14:19:12.847221 2014-08-13 17:38:02.657120 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 292e629a/benchmark_data_storage0111-ib.pocstor_51285_object13162/head//22 by client.275175.0:13163 2014-07-30 14:19:15.151537 2014-08-13 17:38:02.657141 7fe9bae887a0 20 read_log 191727'380 (0'0) modify c022e29a/benchmark_data_storage0111-ib.pocstor_51285_object15276/head//22 by client.275175.0:15277 2014-07-30 14:19:21.586933 2014-08-13 17:38:02.657174 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 9d99aa9a/benchmark_data_storage0111-ib.pocstor_51285_object16536/head//22 by client.275175.0:16537 2014-07-30 14:19:25.800780 2014-08-13 17:38:02.657196 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 5415fa9a/benchmark_data_storage0111-ib.pocstor_51285_object16853/head//22 by client.275175.0:16854 2014-07-30 14:19:26.743260 2014-08-13 17:38:02.657217 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 881129a/benchmark_data_storage0111-ib.pocstor_51285_object16957/head//22 by client.275175.0:16958 2014-07-30 14:19:27.036928 2014-08-13 17:38:02.657238 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 880a3a9a/benchmark_data_storage0111-ib.pocstor_51285_object17909/head//22 by client.275175.0:17910 2014-07-30 14:19:29.702951 2014-08-13 17:38:02.657296 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 9210f29a/benchmark_data_storage0111-ib.pocstor_51285_object19452/head//22 by client.275175.0:19453 2014-07-30 14:19:34.648687 2014-08-13 17:38:02.657318 7fe9bae887a0 20 read_log 191727'393 (0'0) modify e6ddaa9a/benchmark_data_storage0111-ib.pocstor_51285_object22035/head//22 by client.275175.0:22036 2014-07-30 14:19:42.563224 2014-08-13 17:38:02.657339 7fe9bae887a0 20 read_log 191727'396 (0'0) modify ec1ce29a/benchmark_data_storage0111-ib.pocstor_51285_object22784/head//22 by client.275175.0:22785 2014-07-30 14:19:45.380009 2014-08-13 17:38:02.657360 7fe9bae887a0 20 read_log 191727'399 (0'0) modify dc88029a/benchmark_data_storage0111-ib.pocstor_51285_object23752/head//22 by client.275175.0:23753 2014-07-30 14:19:48.135751 2014-08-13 17:38:02.657381 7fe9bae887a0 20 read_log 191727'402 (0'0) modify f36fba9a/benchmark_data_storage0111-ib.pocstor_51285_object25553/head//22 by client.275175.0:25554 2014-07-30 14:19:53.409010 2014-08-13 17:38:02.657401 7fe9bae887a0 20 read_log 191727'404 (0'0) modify c82e4a9a/benchmark_data_storage0111-ib.pocstor_51285_object26110/head//22 by client.275175.0:26111 2014-07-30 14:19:55.176914 2014-08-13 17:38:02.657422 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 28c9ca9a/benchmark_data_storage0111-ib.pocstor_51285_object27944/head//22 by client.275175.0:27945 2014-07-30 14:20:00.873965 2014-08-13 17:38:02.657443 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 747a029a/benchmark_data_storage0111-ib.pocstor_51285_object30016/head//22 by client.275175.0:30017 2014-07-30 14:20:06.996280 2014-08-13 17:38:02.657477 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 5aa5829a/benchmark_data_storage0111-ib.pocstor_51285_object31438/head//22 by client.275175.0:31439 2014-07-30 14:20:11.220391 2014-08-13 17:38:02.657499 7fe9bae887a0 20 read_log 191727'424 (0'0) modify d540329a/benchmark_data_storage0111-ib.pocstor_51285_object34231/head//22 by client.275175.0:34232 2014-07-30 14:20:19.714751 2014-08-13 17:38:02.657519 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 82820a9a/benchmark_data_storage0111-ib.pocstor_51285_object34369/head//22 by client.275175.0:34370 2014-07-30 14:20:20.154262 2014-08-13 17:38:02.657539 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 5812ea9a/benchmark_data_storage0111-ib.pocstor_51285_object36541/head//22 by client.275175.0:36542 2014-07-30 14:20:26.976725 2014-08-13 17:38:02.657560 7fe9bae887a0 20 read_log 192888'435 (0'0) modify a6d1629a/benchmark_data_storage0111-ib.pocstor_35625_object3338/head//22 by client.940330.0:3339 2014-08-11 16:29:53.880535 2014-08-13 17:38:02.657581 7fe9bae887a0 20 read_log 192888'436 (0'0) modify bf9f929a/benchmark_data_storage0111-ib.pocstor_35625_object5242/head//22 by client.940330.0:5243 2014-08-11 16:30:00.885204 2014-08-13 17:38:02.657602 7fe9bae887a0 20 read_log 192888'437 (0'0) modify ee73329a/benchmark_data_storage0111-ib.pocstor_35625_object7331/head//22 by client.940330.0:7332 2014-08-11 16:30:08.485592 2014-08-13 17:38:02.657623 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 5d63da9a/benchmark_data_storage0111-ib.pocstor_35625_object9065/head//22 by client.940330.0:9066 2014-08-11 16:30:14.560038 2014-08-13 17:38:02.657643 7fe9bae887a0 20 read_log 192888'439 (0'0) modify ff775a9a/benchmark_data_storage0111-ib.pocstor_35625_object12302/head//22 by client.940330.0:12303 2014-08-11 16:30:27.141225 2014-08-13 17:38:02.657664 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 18ffaa9a/benchmark_data_storage0111-ib.pocstor_35625_object13517/head//22 by client.940330.0:13518 2014-08-11 16:30:31.852520 2014-08-13 17:38:02.657685 7fe9bae887a0 20 read_log 192888'441 (0'0) modify b04f629a/benchmark_data_storage0111-ib.pocstor_35625_object14185/head//22 by client.940330.0:14186 2014-08-11 16:30:34.442736 2014-08-13 17:38:02.657706 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 15ac1a9a/benchmark_data_storage0111-ib.pocstor_35625_object18959/head//22 by client.940330.0:18960 2014-08-11 16:30:52.518495 2014-08-13 17:38:02.657726 7fe9bae887a0 20 read_log 192888'443 (0'0) modify d285829a/benchmark_data_storage0111-ib.pocstor_35625_object20871/head//22 by client.940330.0:20872 2014-08-11 16:31:00.213662 2014-08-13 17:38:02.657747 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 8f1e0a9a/benchmark_data_storage0111-ib.pocstor_35625_object20965/head//22 by client.940330.0:20966 2014-08-11 16:31:00.574395 2014-08-13 17:38:02.657768 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 4d7c8a9a/benchmark_data_storage0111-ib.pocstor_35625_object28021/head//22 by client.940330.0:28022 2014-08-11 16:31:27.810272 2014-08-13 17:38:02.657788 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 94293a9a/benchmark_data_storage0111-ib.pocstor_35625_object31158/head//22 by client.940330.0:31159 2014-08-11 16:31:39.815238 2014-08-13 17:38:02.657809 7fe9bae887a0 20 read_log 192888'447 (192888'435) delete a6d1629a/benchmark_data_storage0111-ib.pocstor_35625_object3338/head//22 by client.940330.0:34646 2014-08-11 16:31:42.345397 2014-08-13 17:38:02.657830 7fe9bae887a0 20 read_log 192888'448 (192888'436) delete bf9f929a/benchmark_data_storage0111-ib.pocstor_35625_object5242/head//22 by client.940330.0:36550 2014-08-11 16:31:42.878912 2014-08-13 17:38:02.657851 7fe9bae887a0 20 read_log 192888'449 (192888'437) delete ee73329a/benchmark_data_storage0111-ib.pocstor_35625_object7331/head//22 by client.940330.0:38639 2014-08-11 16:31:43.421986 2014-08-13 17:38:02.657889 7fe9bae887a0 20 read_log 192888'450 (192888'438) delete 5d63da9a/benchmark_data_storage0111-ib.pocstor_35625_object9065/head//22 by client.940330.0:40373 2014-08-11 16:31:43.858090 2014-08-13 17:38:02.657948 7fe9bae887a0 20 read_log 192888'451 (192888'439) delete ff775a9a/benchmark_data_storage0111-ib.pocstor_35625_object12302/head//22 by client.940330.0:43610 2014-08-11 16:31:44.733705 2014-08-13 17:38:02.657972 7fe9bae887a0 20 read_log 192888'452 (192888'440) delete 18ffaa9a/benchmark_data_storage0111-ib.pocstor_35625_object13517/head//22 by client.940330.0:44825 2014-08-11 16:31:45.057974 2014-08-13 17:38:02.657993 7fe9bae887a0 20 read_log 192888'453 (192888'441) delete b04f629a/benchmark_data_storage0111-ib.pocstor_35625_object14185/head//22 by client.940330.0:45493 2014-08-11 16:31:45.230848 2014-08-13 17:38:02.658014 7fe9bae887a0 20 read_log 192888'454 (192888'442) delete 15ac1a9a/benchmark_data_storage0111-ib.pocstor_35625_object18959/head//22 by client.940330.0:50267 2014-08-11 16:31:46.570405 2014-08-13 17:38:02.658035 7fe9bae887a0 20 read_log 192888'455 (192888'443) delete d285829a/benchmark_data_storage0111-ib.pocstor_35625_object20871/head//22 by client.940330.0:52179 2014-08-11 16:31:47.126257 2014-08-13 17:38:02.658056 7fe9bae887a0 20 read_log 192888'456 (192888'444) delete 8f1e0a9a/benchmark_data_storage0111-ib.pocstor_35625_object20965/head//22 by client.940330.0:52273 2014-08-11 16:31:47.152356 2014-08-13 17:38:02.658077 7fe9bae887a0 20 read_log 192888'457 (192888'445) delete 4d7c8a9a/benchmark_data_storage0111-ib.pocstor_35625_object28021/head//22 by client.940330.0:59329 2014-08-11 16:31:49.211905 2014-08-13 17:38:02.658098 7fe9bae887a0 20 read_log 192888'458 (192888'446) delete 94293a9a/benchmark_data_storage0111-ib.pocstor_35625_object31158/head//22 by client.940330.0:62466 2014-08-11 16:31:50.108233 2014-08-13 17:38:02.658119 7fe9bae887a0 20 read_log 194215'459 (0'0) modify a3e6429a/benchmark_data_storage0111-ib.pocstor_64498_object1440/head//22 by client.1049635.0:1441 2014-08-12 17:17:21.820545 2014-08-13 17:38:02.658140 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 2cd79a9a/benchmark_data_storage0111-ib.pocstor_64498_object5515/head//22 by client.1049635.0:5516 2014-08-12 17:17:34.660015 2014-08-13 17:38:02.658186 7fe9bae887a0 20 read_log 194215'461 (0'0) modify d9bff29a/benchmark_data_storage0111-ib.pocstor_64498_object7457/head//22 by client.1049635.0:7458 2014-08-12 17:17:40.416193 2014-08-13 17:38:02.658209 7fe9bae887a0 20 read_log 194215'462 (0'0) modify c8a7ea9a/benchmark_data_storage0111-ib.pocstor_64498_object8377/head//22 by client.1049635.0:8378 2014-08-12 17:17:43.202221 2014-08-13 17:38:02.658230 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 8b7c4a9a/benchmark_data_storage0111-ib.pocstor_64498_object8879/head//22 by client.1049635.0:8880 2014-08-12 17:17:44.677970 2014-08-13 17:38:02.658251 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 4c7eba9a/benchmark_data_storage0111-ib.pocstor_64498_object9444/head//22 by client.1049635.0:9445 2014-08-12 17:17:46.313717 2014-08-13 17:38:02.658271 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 31221a9a/benchmark_data_storage0111-ib.pocstor_64498_object9464/head//22 by client.1049635.0:9465 2014-08-12 17:17:46.371214 2014-08-13 17:38:02.658292 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 9f517a9a/benchmark_data_storage0111-ib.pocstor_64498_object10178/head//22 by client.1049635.0:10179 2014-08-12 17:17:48.435379 2014-08-13 17:38:02.658314 7fe9bae887a0 20 read_log 194215'467 (0'0) modify bd4b1a9a/benchmark_data_storage0111-ib.pocstor_64498_object14186/head//22 by client.1049635.0:14187 2014-08-12 17:18:01.069249 2014-08-13 17:38:02.658335 7fe9bae887a0 20 read_log 194215'468 (0'0) modify dda5129a/benchmark_data_storage0111-ib.pocstor_64498_object16118/head//22 by client.1049635.0:16119 2014-08-12 17:18:07.997347 2014-08-13 17:38:02.658370 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 258f529a/benchmark_data_storage0111-ib.pocstor_64498_object18818/head//22 by client.1049635.0:18819 2014-08-12 17:18:18.245369 2014-08-13 17:38:02.658393 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 9b491a9a/benchmark_data_storage0111-ib.pocstor_64498_object18967/head//22 by client.1049635.0:18968 2014-08-12 17:18:18.813307 2014-08-13 17:38:02.658414 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 740a3a9a/benchmark_data_storage0111-ib.pocstor_64498_object19482/head//22 by client.1049635.0:19483 2014-08-12 17:18:20.842384 2014-08-13 17:38:02.658435 7fe9bae887a0 20 read_log 194215'472 (0'0) modify d0db4a9a/benchmark_data_storage0111-ib.pocstor_64498_object25472/head//22 by client.1049635.0:25473 2014-08-12 17:18:43.462266 2014-08-13 17:38:02.658456 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 7cbd0a9a/benchmark_data_storage0111-ib.pocstor_64498_object32075/head//22 by client.1049635.0:32076 2014-08-12 17:19:08.855735 2014-08-13 17:38:02.658477 7fe9bae887a0 20 read_log 194215'474 (194215'459) delete a3e6429a/benchmark_data_storage0111-ib.pocstor_64498_object1440/head//22 by client.1049635.0:35643 2014-08-12 17:19:18.465088 2014-08-13 17:38:02.658498 7fe9bae887a0 20 read_log 194215'475 (194215'460) delete 2cd79a9a/benchmark_data_storage0111-ib.pocstor_64498_object5515/head//22 by client.1049635.0:39718 2014-08-12 17:19:19.776840 2014-08-13 17:38:02.658520 7fe9bae887a0 20 read_log 194215'476 (194215'461) delete d9bff29a/benchmark_data_storage0111-ib.pocstor_64498_object7457/head//22 by client.1049635.0:41660 2014-08-12 17:19:20.436838 2014-08-13 17:38:02.658541 7fe9bae887a0 20 read_log 194215'477 (194215'462) delete c8a7ea9a/benchmark_data_storage0111-ib.pocstor_64498_object8377/head//22 by client.1049635.0:42580 2014-08-12 17:19:20.720441 2014-08-13 17:38:02.658561 7fe9bae887a0 20 read_log 194215'478 (194215'463) delete 8b7c4a9a/benchmark_data_storage0111-ib.pocstor_64498_object8879/head//22 by client.1049635.0:43082 2014-08-12 17:19:20.885714 2014-08-13 17:38:02.658583 7fe9bae887a0 20 read_log 194215'479 (194215'464) delete 4c7eba9a/benchmark_data_storage0111-ib.pocstor_64498_object9444/head//22 by client.1049635.0:43647 2014-08-12 17:19:21.074107 2014-08-13 17:38:02.658604 7fe9bae887a0 20 read_log 194215'480 (194215'465) delete 31221a9a/benchmark_data_storage0111-ib.pocstor_64498_object9464/head//22 by client.1049635.0:43667 2014-08-12 17:19:21.079405 2014-08-13 17:38:02.658625 7fe9bae887a0 20 read_log 194215'481 (194215'466) delete 9f517a9a/benchmark_data_storage0111-ib.pocstor_64498_object10178/head//22 by client.1049635.0:44381 2014-08-12 17:19:21.321322 2014-08-13 17:38:02.658649 7fe9bae887a0 20 read_log 194215'482 (194215'467) delete bd4b1a9a/benchmark_data_storage0111-ib.pocstor_64498_object14186/head//22 by client.1049635.0:48389 2014-08-12 17:19:22.618870 2014-08-13 17:38:02.658671 7fe9bae887a0 20 read_log 194215'483 (194215'468) delete dda5129a/benchmark_data_storage0111-ib.pocstor_64498_object16118/head//22 by client.1049635.0:50321 2014-08-12 17:19:23.226018 2014-08-13 17:38:02.658692 7fe9bae887a0 20 read_log 194215'484 (194215'469) delete 258f529a/benchmark_data_storage0111-ib.pocstor_64498_object18818/head//22 by client.1049635.0:53021 2014-08-12 17:19:24.100637 2014-08-13 17:38:02.658713 7fe9bae887a0 20 read_log 194215'485 (194215'470) delete 9b491a9a/benchmark_data_storage0111-ib.pocstor_64498_object18967/head//22 by client.1049635.0:53170 2014-08-12 17:19:24.157210 2014-08-13 17:38:02.658734 7fe9bae887a0 20 read_log 194215'486 (194215'471) delete 740a3a9a/benchmark_data_storage0111-ib.pocstor_64498_object19482/head//22 by client.1049635.0:53685 2014-08-12 17:19:24.344152 2014-08-13 17:38:02.658755 7fe9bae887a0 20 read_log 194215'487 (194215'472) delete d0db4a9a/benchmark_data_storage0111-ib.pocstor_64498_object25472/head//22 by client.1049635.0:59675 2014-08-12 17:19:26.475418 2014-08-13 17:38:02.658776 7fe9bae887a0 20 read_log 194215'488 (194215'473) delete 7cbd0a9a/benchmark_data_storage0111-ib.pocstor_64498_object32075/head//22 by client.1049635.0:66278 2014-08-12 17:19:28.771812 2014-08-13 17:38:02.658917 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.658944 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=0 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.658967 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=0 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.002562 0 0.000000 2014-08-13 17:38:02.658986 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=0 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.658999 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=0 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.659016 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] log((0'0,194215'488], crt=192888'458) 2014-08-13 17:38:02.659031 7fe9bae887a0 10 osd.4 194478 pgid 22.2b9 coll 22.2b9_head 2014-08-13 17:38:02.659238 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.2b9 2014-08-13 17:38:02.659249 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.659308 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2b9(unlocked)] enter Initial 2014-08-13 17:38:02.659323 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.2b9(unlocked)] enter NotTrimming 2014-08-13 17:38:02.659524 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.659737 7fe9bae887a0 20 read_log 191648'35 (0'0) modify d3f8f2b9/benchmark_data_storage0111-ib.pocstor_40745_object2578/head//22 by client.273158.0:2579 2014-07-30 14:08:59.235134 2014-08-13 17:38:02.659773 7fe9bae887a0 20 read_log 191648'50 (0'0) modify 7f207ab9/benchmark_data_storage0111-ib.pocstor_40745_object3459/head//22 by client.273158.0:3460 2014-07-30 14:09:03.526944 2014-08-13 17:38:02.659796 7fe9bae887a0 20 read_log 191648'57 (0'0) modify c0f0dab9/benchmark_data_storage0111-ib.pocstor_40745_object4202/head//22 by client.273158.0:4203 2014-07-30 14:09:07.625024 2014-08-13 17:38:02.659818 7fe9bae887a0 20 read_log 191648'122 (0'0) modify d72ab9/benchmark_data_storage0111-ib.pocstor_40745_object8496/head//22 by client.273158.0:8497 2014-07-30 14:09:28.890608 2014-08-13 17:38:02.659839 7fe9bae887a0 20 read_log 191648'131 (0'0) modify 8cb8ab9/benchmark_data_storage0111-ib.pocstor_40745_object9314/head//22 by client.273158.0:9315 2014-07-30 14:09:32.469869 2014-08-13 17:38:02.659863 7fe9bae887a0 20 read_log 191648'146 (0'0) modify ef02b2b9/benchmark_data_storage0111-ib.pocstor_40745_object10074/head//22 by client.273158.0:10075 2014-07-30 14:09:36.129504 2014-08-13 17:38:02.659885 7fe9bae887a0 20 read_log 191648'176 (0'0) modify 907fd2b9/benchmark_data_storage0111-ib.pocstor_40745_object11811/head//22 by client.273158.0:11812 2014-07-30 14:09:46.128749 2014-08-13 17:38:02.659956 7fe9bae887a0 20 read_log 191648'261 (0'0) modify 89e652b9/benchmark_data_storage0111-ib.pocstor_40745_object16679/head//22 by client.273158.0:16680 2014-07-30 14:10:11.242493 2014-08-13 17:38:02.659984 7fe9bae887a0 20 read_log 191648'267 (0'0) modify 146cb2b9/benchmark_data_storage0111-ib.pocstor_40745_object17206/head//22 by client.273158.0:17207 2014-07-30 14:10:14.122501 2014-08-13 17:38:02.660006 7fe9bae887a0 20 read_log 191648'272 (0'0) modify d0ea0ab9/benchmark_data_storage0111-ib.pocstor_40745_object17659/head//22 by client.273158.0:17660 2014-07-30 14:10:16.328821 2014-08-13 17:38:02.660027 7fe9bae887a0 20 read_log 191648'286 (0'0) modify 28e2fab9/benchmark_data_storage0111-ib.pocstor_40745_object18574/head//22 by client.273158.0:18575 2014-07-30 14:10:21.484947 2014-08-13 17:38:02.660048 7fe9bae887a0 20 read_log 191727'356 (0'0) modify 15b2eab9/benchmark_data_storage0111-ib.pocstor_51285_object1015/head//22 by client.275175.0:1016 2014-07-30 14:18:37.510632 2014-08-13 17:38:02.660069 7fe9bae887a0 20 read_log 191727'358 (0'0) modify ee6d2ab9/benchmark_data_storage0111-ib.pocstor_51285_object2208/head//22 by client.275175.0:2209 2014-07-30 14:18:41.573650 2014-08-13 17:38:02.660089 7fe9bae887a0 20 read_log 191727'360 (0'0) modify 113e8ab9/benchmark_data_storage0111-ib.pocstor_51285_object3238/head//22 by client.275175.0:3239 2014-07-30 14:18:44.972331 2014-08-13 17:38:02.660110 7fe9bae887a0 20 read_log 191727'365 (0'0) modify e8205ab9/benchmark_data_storage0111-ib.pocstor_51285_object4988/head//22 by client.275175.0:4989 2014-07-30 14:18:50.317372 2014-08-13 17:38:02.660131 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 6b5daab9/benchmark_data_storage0111-ib.pocstor_51285_object9391/head//22 by client.275175.0:9392 2014-07-30 14:19:03.480266 2014-08-13 17:38:02.660152 7fe9bae887a0 20 read_log 191727'378 (0'0) modify f85552b9/benchmark_data_storage0111-ib.pocstor_51285_object12132/head//22 by client.275175.0:12133 2014-07-30 14:19:12.009009 2014-08-13 17:38:02.660189 7fe9bae887a0 20 read_log 191727'380 (0'0) modify 80068ab9/benchmark_data_storage0111-ib.pocstor_51285_object12795/head//22 by client.275175.0:12796 2014-07-30 14:19:13.870536 2014-08-13 17:38:02.660210 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 26189ab9/benchmark_data_storage0111-ib.pocstor_51285_object14011/head//22 by client.275175.0:14012 2014-07-30 14:19:17.913542 2014-08-13 17:38:02.660231 7fe9bae887a0 20 read_log 191727'387 (0'0) modify 9fdee2b9/benchmark_data_storage0111-ib.pocstor_51285_object15409/head//22 by client.275175.0:15410 2014-07-30 14:19:22.120013 2014-08-13 17:38:02.660252 7fe9bae887a0 20 read_log 191727'388 (0'0) modify af35eab9/benchmark_data_storage0111-ib.pocstor_51285_object15464/head//22 by client.275175.0:15465 2014-07-30 14:19:22.313137 2014-08-13 17:38:02.660273 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 11a47ab9/benchmark_data_storage0111-ib.pocstor_51285_object18360/head//22 by client.275175.0:18361 2014-07-30 14:19:31.056235 2014-08-13 17:38:02.660294 7fe9bae887a0 20 read_log 191727'395 (0'0) modify b6478ab9/benchmark_data_storage0111-ib.pocstor_51285_object19555/head//22 by client.275175.0:19556 2014-07-30 14:19:34.931007 2014-08-13 17:38:02.660318 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 6c9ff2b9/benchmark_data_storage0111-ib.pocstor_51285_object20232/head//22 by client.275175.0:20233 2014-07-30 14:19:36.927640 2014-08-13 17:38:02.660339 7fe9bae887a0 20 read_log 191727'398 (0'0) modify ec344ab9/benchmark_data_storage0111-ib.pocstor_51285_object20497/head//22 by client.275175.0:20498 2014-07-30 14:19:37.686646 2014-08-13 17:38:02.660360 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 5de7eab9/benchmark_data_storage0111-ib.pocstor_51285_object25019/head//22 by client.275175.0:25020 2014-07-30 14:19:51.736299 2014-08-13 17:38:02.660393 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 183982b9/benchmark_data_storage0111-ib.pocstor_51285_object28811/head//22 by client.275175.0:28812 2014-07-30 14:20:03.335783 2014-08-13 17:38:02.660415 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 52bdc2b9/benchmark_data_storage0111-ib.pocstor_51285_object32990/head//22 by client.275175.0:32991 2014-07-30 14:20:15.770962 2014-08-13 17:38:02.660436 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 3fac2ab9/benchmark_data_storage0111-ib.pocstor_51285_object34206/head//22 by client.275175.0:34207 2014-07-30 14:20:19.645396 2014-08-13 17:38:02.660485 7fe9bae887a0 20 read_log 191727'426 (0'0) modify ea98fab9/benchmark_data_storage0111-ib.pocstor_51285_object35788/head//22 by client.275175.0:35789 2014-07-30 14:20:24.618718 2014-08-13 17:38:02.660506 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 7fc9dab9/benchmark_data_storage0111-ib.pocstor_51285_object35837/head//22 by client.275175.0:35838 2014-07-30 14:20:24.753367 2014-08-13 17:38:02.660526 7fe9bae887a0 20 read_log 191727'428 (0'0) modify 891bbab9/benchmark_data_storage0111-ib.pocstor_51285_object37239/head//22 by client.275175.0:37240 2014-07-30 14:20:29.038620 2014-08-13 17:38:02.660547 7fe9bae887a0 20 read_log 191727'430 (0'0) modify af75cab9/benchmark_data_storage0111-ib.pocstor_51285_object37400/head//22 by client.275175.0:37401 2014-07-30 14:20:29.520223 2014-08-13 17:38:02.660568 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 262f02b9/benchmark_data_storage0111-ib.pocstor_35625_object92/head//22 by client.940330.0:93 2014-08-11 16:29:40.880779 2014-08-13 17:38:02.660588 7fe9bae887a0 20 read_log 192888'434 (0'0) modify d0eb1ab9/benchmark_data_storage0111-ib.pocstor_35625_object439/head//22 by client.940330.0:440 2014-08-11 16:29:42.433265 2014-08-13 17:38:02.660609 7fe9bae887a0 20 read_log 192888'435 (0'0) modify f282cab9/benchmark_data_storage0111-ib.pocstor_35625_object1820/head//22 by client.940330.0:1821 2014-08-11 16:29:47.885147 2014-08-13 17:38:02.660629 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 7ecbcab9/benchmark_data_storage0111-ib.pocstor_35625_object7012/head//22 by client.940330.0:7013 2014-08-11 16:30:07.360659 2014-08-13 17:38:02.660649 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 8c6df2b9/benchmark_data_storage0111-ib.pocstor_35625_object9464/head//22 by client.940330.0:9465 2014-08-11 16:30:15.954132 2014-08-13 17:38:02.660670 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 6864dab9/benchmark_data_storage0111-ib.pocstor_35625_object11075/head//22 by client.940330.0:11076 2014-08-11 16:30:22.091043 2014-08-13 17:38:02.660691 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 6e8472b9/benchmark_data_storage0111-ib.pocstor_35625_object12694/head//22 by client.940330.0:12695 2014-08-11 16:30:28.618262 2014-08-13 17:38:02.660711 7fe9bae887a0 20 read_log 192888'440 (0'0) modify ef062b9/benchmark_data_storage0111-ib.pocstor_35625_object13219/head//22 by client.940330.0:13220 2014-08-11 16:30:30.739260 2014-08-13 17:38:02.660732 7fe9bae887a0 20 read_log 192888'441 (0'0) modify a366b2b9/benchmark_data_storage0111-ib.pocstor_35625_object16802/head//22 by client.940330.0:16803 2014-08-11 16:30:44.227856 2014-08-13 17:38:02.660752 7fe9bae887a0 20 read_log 192888'442 (0'0) modify da46d2b9/benchmark_data_storage0111-ib.pocstor_35625_object17401/head//22 by client.940330.0:17402 2014-08-11 16:30:46.462131 2014-08-13 17:38:02.660772 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 578d2ab9/benchmark_data_storage0111-ib.pocstor_35625_object21585/head//22 by client.940330.0:21586 2014-08-11 16:31:02.988341 2014-08-13 17:38:02.660793 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 610412b9/benchmark_data_storage0111-ib.pocstor_35625_object22270/head//22 by client.940330.0:22271 2014-08-11 16:31:05.576452 2014-08-13 17:38:02.660825 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 9303d2b9/benchmark_data_storage0111-ib.pocstor_35625_object24253/head//22 by client.940330.0:24254 2014-08-11 16:31:13.072121 2014-08-13 17:38:02.660847 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 5047b2b9/benchmark_data_storage0111-ib.pocstor_35625_object24409/head//22 by client.940330.0:24410 2014-08-11 16:31:13.635303 2014-08-13 17:38:02.660868 7fe9bae887a0 20 read_log 192888'447 (0'0) modify 34d39ab9/benchmark_data_storage0111-ib.pocstor_35625_object25823/head//22 by client.940330.0:25824 2014-08-11 16:31:19.098064 2014-08-13 17:38:02.660888 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 9fc76ab9/benchmark_data_storage0111-ib.pocstor_35625_object27778/head//22 by client.940330.0:27779 2014-08-11 16:31:26.821318 2014-08-13 17:38:02.660909 7fe9bae887a0 20 read_log 192888'449 (0'0) modify e9267ab9/benchmark_data_storage0111-ib.pocstor_35625_object27786/head//22 by client.940330.0:27787 2014-08-11 16:31:26.852202 2014-08-13 17:38:02.660929 7fe9bae887a0 20 read_log 192888'450 (0'0) modify e5fc3ab9/benchmark_data_storage0111-ib.pocstor_35625_object28447/head//22 by client.940330.0:28448 2014-08-11 16:31:29.452060 2014-08-13 17:38:02.660950 7fe9bae887a0 20 read_log 192888'451 (0'0) modify bfa62b9/benchmark_data_storage0111-ib.pocstor_35625_object29176/head//22 by client.940330.0:29177 2014-08-11 16:31:32.344930 2014-08-13 17:38:02.661005 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 4f51dab9/benchmark_data_storage0111-ib.pocstor_35625_object30528/head//22 by client.940330.0:30529 2014-08-11 16:31:37.456588 2014-08-13 17:38:02.661027 7fe9bae887a0 20 read_log 192888'453 (192888'433) delete 262f02b9/benchmark_data_storage0111-ib.pocstor_35625_object92/head//22 by client.940330.0:31400 2014-08-11 16:31:41.474492 2014-08-13 17:38:02.661049 7fe9bae887a0 20 read_log 192888'454 (192888'434) delete d0eb1ab9/benchmark_data_storage0111-ib.pocstor_35625_object439/head//22 by client.940330.0:31747 2014-08-11 16:31:41.605606 2014-08-13 17:38:02.661070 7fe9bae887a0 20 read_log 192888'455 (192888'435) delete f282cab9/benchmark_data_storage0111-ib.pocstor_35625_object1820/head//22 by client.940330.0:33128 2014-08-11 16:31:41.945367 2014-08-13 17:38:02.661091 7fe9bae887a0 20 read_log 192888'456 (192888'436) delete 7ecbcab9/benchmark_data_storage0111-ib.pocstor_35625_object7012/head//22 by client.940330.0:38320 2014-08-11 16:31:43.338398 2014-08-13 17:38:02.661112 7fe9bae887a0 20 read_log 192888'457 (192888'437) delete 8c6df2b9/benchmark_data_storage0111-ib.pocstor_35625_object9464/head//22 by client.940330.0:40772 2014-08-11 16:31:43.965609 2014-08-13 17:38:02.661135 7fe9bae887a0 20 read_log 192888'458 (192888'438) delete 6864dab9/benchmark_data_storage0111-ib.pocstor_35625_object11075/head//22 by client.940330.0:42383 2014-08-11 16:31:44.419791 2014-08-13 17:38:02.661165 7fe9bae887a0 20 read_log 192888'459 (192888'439) delete 6e8472b9/benchmark_data_storage0111-ib.pocstor_35625_object12694/head//22 by client.940330.0:44002 2014-08-11 16:31:44.832866 2014-08-13 17:38:02.661188 7fe9bae887a0 20 read_log 192888'460 (192888'440) delete ef062b9/benchmark_data_storage0111-ib.pocstor_35625_object13219/head//22 by client.940330.0:44527 2014-08-11 16:31:44.980969 2014-08-13 17:38:02.661209 7fe9bae887a0 20 read_log 192888'461 (192888'441) delete a366b2b9/benchmark_data_storage0111-ib.pocstor_35625_object16802/head//22 by client.940330.0:48110 2014-08-11 16:31:45.934941 2014-08-13 17:38:02.661231 7fe9bae887a0 20 read_log 192888'462 (192888'442) delete da46d2b9/benchmark_data_storage0111-ib.pocstor_35625_object17401/head//22 by client.940330.0:48709 2014-08-11 16:31:46.101895 2014-08-13 17:38:02.661251 7fe9bae887a0 20 read_log 192888'463 (192888'443) delete 578d2ab9/benchmark_data_storage0111-ib.pocstor_35625_object21585/head//22 by client.940330.0:52893 2014-08-11 16:31:47.317679 2014-08-13 17:38:02.661272 7fe9bae887a0 20 read_log 192888'464 (192888'444) delete 610412b9/benchmark_data_storage0111-ib.pocstor_35625_object22270/head//22 by client.940330.0:53578 2014-08-11 16:31:47.534444 2014-08-13 17:38:02.661310 7fe9bae887a0 20 read_log 192888'465 (192888'445) delete 9303d2b9/benchmark_data_storage0111-ib.pocstor_35625_object24253/head//22 by client.940330.0:55561 2014-08-11 16:31:48.115829 2014-08-13 17:38:02.661332 7fe9bae887a0 20 read_log 192888'466 (192888'446) delete 5047b2b9/benchmark_data_storage0111-ib.pocstor_35625_object24409/head//22 by client.940330.0:55717 2014-08-11 16:31:48.159218 2014-08-13 17:38:02.661353 7fe9bae887a0 20 read_log 192888'467 (192888'447) delete 34d39ab9/benchmark_data_storage0111-ib.pocstor_35625_object25823/head//22 by client.940330.0:57131 2014-08-11 16:31:48.594401 2014-08-13 17:38:02.661374 7fe9bae887a0 20 read_log 192888'468 (192888'448) delete 9fc76ab9/benchmark_data_storage0111-ib.pocstor_35625_object27778/head//22 by client.940330.0:59086 2014-08-11 16:31:49.144956 2014-08-13 17:38:02.661395 7fe9bae887a0 20 read_log 192888'469 (192888'449) delete e9267ab9/benchmark_data_storage0111-ib.pocstor_35625_object27786/head//22 by client.940330.0:59094 2014-08-11 16:31:49.147349 2014-08-13 17:38:02.661416 7fe9bae887a0 20 read_log 192888'470 (192888'450) delete e5fc3ab9/benchmark_data_storage0111-ib.pocstor_35625_object28447/head//22 by client.940330.0:59755 2014-08-11 16:31:49.323373 2014-08-13 17:38:02.661437 7fe9bae887a0 20 read_log 192888'471 (192888'451) delete bfa62b9/benchmark_data_storage0111-ib.pocstor_35625_object29176/head//22 by client.940330.0:60484 2014-08-11 16:31:49.517446 2014-08-13 17:38:02.661458 7fe9bae887a0 20 read_log 192888'472 (192888'452) delete 4f51dab9/benchmark_data_storage0111-ib.pocstor_35625_object30528/head//22 by client.940330.0:61836 2014-08-11 16:31:49.909042 2014-08-13 17:38:02.661479 7fe9bae887a0 20 read_log 194215'473 (0'0) modify a9d172b9/benchmark_data_storage0111-ib.pocstor_64498_object3760/head//22 by client.1049635.0:3761 2014-08-12 17:17:29.272366 2014-08-13 17:38:02.661500 7fe9bae887a0 20 read_log 194215'474 (0'0) modify e90592b9/benchmark_data_storage0111-ib.pocstor_64498_object3986/head//22 by client.1049635.0:3987 2014-08-12 17:17:30.044775 2014-08-13 17:38:02.661521 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 8ac7e2b9/benchmark_data_storage0111-ib.pocstor_64498_object4273/head//22 by client.1049635.0:4274 2014-08-12 17:17:30.876383 2014-08-13 17:38:02.661541 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 91e7ab9/benchmark_data_storage0111-ib.pocstor_64498_object4663/head//22 by client.1049635.0:4664 2014-08-12 17:17:31.947733 2014-08-13 17:38:02.661563 7fe9bae887a0 20 read_log 194215'477 (0'0) modify f97aeab9/benchmark_data_storage0111-ib.pocstor_64498_object12653/head//22 by client.1049635.0:12654 2014-08-12 17:17:56.313997 2014-08-13 17:38:02.661584 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 8f9d4ab9/benchmark_data_storage0111-ib.pocstor_64498_object13940/head//22 by client.1049635.0:13941 2014-08-12 17:18:00.341466 2014-08-13 17:38:02.661638 7fe9bae887a0 20 read_log 194215'479 (0'0) modify 5ba662b9/benchmark_data_storage0111-ib.pocstor_64498_object15216/head//22 by client.1049635.0:15217 2014-08-12 17:18:04.690502 2014-08-13 17:38:02.661659 7fe9bae887a0 20 read_log 194215'480 (0'0) modify cdfe3ab9/benchmark_data_storage0111-ib.pocstor_64498_object16216/head//22 by client.1049635.0:16217 2014-08-12 17:18:08.376666 2014-08-13 17:38:02.661681 7fe9bae887a0 20 read_log 194215'481 (0'0) modify 3e4a9ab9/benchmark_data_storage0111-ib.pocstor_64498_object18182/head//22 by client.1049635.0:18183 2014-08-12 17:18:15.851499 2014-08-13 17:38:02.661702 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 4bc3ab9/benchmark_data_storage0111-ib.pocstor_64498_object18556/head//22 by client.1049635.0:18557 2014-08-12 17:18:17.261424 2014-08-13 17:38:02.661723 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 47b7dab9/benchmark_data_storage0111-ib.pocstor_64498_object21260/head//22 by client.1049635.0:21261 2014-08-12 17:18:27.561650 2014-08-13 17:38:02.661755 7fe9bae887a0 20 read_log 194215'484 (0'0) modify a2ea42b9/benchmark_data_storage0111-ib.pocstor_64498_object22370/head//22 by client.1049635.0:22371 2014-08-12 17:18:31.721020 2014-08-13 17:38:02.661777 7fe9bae887a0 20 read_log 194215'485 (0'0) modify c476aab9/benchmark_data_storage0111-ib.pocstor_64498_object24913/head//22 by client.1049635.0:24914 2014-08-12 17:18:41.373895 2014-08-13 17:38:02.661825 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 315662b9/benchmark_data_storage0111-ib.pocstor_64498_object26498/head//22 by client.1049635.0:26499 2014-08-12 17:18:47.281435 2014-08-13 17:38:02.661846 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 29a7dab9/benchmark_data_storage0111-ib.pocstor_64498_object27780/head//22 by client.1049635.0:27781 2014-08-12 17:18:52.065774 2014-08-13 17:38:02.661868 7fe9bae887a0 20 read_log 194215'488 (0'0) modify cd2c82b9/benchmark_data_storage0111-ib.pocstor_64498_object28703/head//22 by client.1049635.0:28704 2014-08-12 17:18:55.495257 2014-08-13 17:38:02.661889 7fe9bae887a0 20 read_log 194215'489 (0'0) modify eb9c7ab9/benchmark_data_storage0111-ib.pocstor_64498_object29655/head//22 by client.1049635.0:29656 2014-08-12 17:18:59.157579 2014-08-13 17:38:02.661909 7fe9bae887a0 20 read_log 194215'490 (194215'473) delete a9d172b9/benchmark_data_storage0111-ib.pocstor_64498_object3760/head//22 by client.1049635.0:37963 2014-08-12 17:19:19.201888 2014-08-13 17:38:02.661930 7fe9bae887a0 20 read_log 194215'491 (194215'474) delete e90592b9/benchmark_data_storage0111-ib.pocstor_64498_object3986/head//22 by client.1049635.0:38189 2014-08-12 17:19:19.270674 2014-08-13 17:38:02.661951 7fe9bae887a0 20 read_log 194215'492 (194215'475) delete 8ac7e2b9/benchmark_data_storage0111-ib.pocstor_64498_object4273/head//22 by client.1049635.0:38476 2014-08-12 17:19:19.364508 2014-08-13 17:38:02.661972 7fe9bae887a0 20 read_log 194215'493 (194215'476) delete 91e7ab9/benchmark_data_storage0111-ib.pocstor_64498_object4663/head//22 by client.1049635.0:38866 2014-08-12 17:19:19.490071 2014-08-13 17:38:02.661993 7fe9bae887a0 20 read_log 194215'494 (194215'477) delete f97aeab9/benchmark_data_storage0111-ib.pocstor_64498_object12653/head//22 by client.1049635.0:46856 2014-08-12 17:19:22.101362 2014-08-13 17:38:02.662014 7fe9bae887a0 20 read_log 194215'495 (194215'478) delete 8f9d4ab9/benchmark_data_storage0111-ib.pocstor_64498_object13940/head//22 by client.1049635.0:48143 2014-08-12 17:19:22.545011 2014-08-13 17:38:02.662035 7fe9bae887a0 20 read_log 194215'496 (194215'479) delete 5ba662b9/benchmark_data_storage0111-ib.pocstor_64498_object15216/head//22 by client.1049635.0:49419 2014-08-12 17:19:22.942355 2014-08-13 17:38:02.662055 7fe9bae887a0 20 read_log 194215'497 (194215'480) delete cdfe3ab9/benchmark_data_storage0111-ib.pocstor_64498_object16216/head//22 by client.1049635.0:50419 2014-08-12 17:19:23.256361 2014-08-13 17:38:02.662076 7fe9bae887a0 20 read_log 194215'498 (194215'481) delete 3e4a9ab9/benchmark_data_storage0111-ib.pocstor_64498_object18182/head//22 by client.1049635.0:52385 2014-08-12 17:19:23.891171 2014-08-13 17:38:02.662097 7fe9bae887a0 20 read_log 194215'499 (194215'482) delete 4bc3ab9/benchmark_data_storage0111-ib.pocstor_64498_object18556/head//22 by client.1049635.0:52759 2014-08-12 17:19:24.016127 2014-08-13 17:38:02.662118 7fe9bae887a0 20 read_log 194215'500 (194215'483) delete 47b7dab9/benchmark_data_storage0111-ib.pocstor_64498_object21260/head//22 by client.1049635.0:55463 2014-08-12 17:19:24.975482 2014-08-13 17:38:02.662139 7fe9bae887a0 20 read_log 194215'501 (194215'484) delete a2ea42b9/benchmark_data_storage0111-ib.pocstor_64498_object22370/head//22 by client.1049635.0:56573 2014-08-12 17:19:25.371796 2014-08-13 17:38:02.662168 7fe9bae887a0 20 read_log 194215'502 (194215'485) delete c476aab9/benchmark_data_storage0111-ib.pocstor_64498_object24913/head//22 by client.1049635.0:59116 2014-08-12 17:19:26.292249 2014-08-13 17:38:02.662190 7fe9bae887a0 20 read_log 194215'503 (194215'486) delete 315662b9/benchmark_data_storage0111-ib.pocstor_64498_object26498/head//22 by client.1049635.0:60701 2014-08-12 17:19:26.821811 2014-08-13 17:38:02.662223 7fe9bae887a0 20 read_log 194215'504 (194215'487) delete 29a7dab9/benchmark_data_storage0111-ib.pocstor_64498_object27780/head//22 by client.1049635.0:61983 2014-08-12 17:19:27.270597 2014-08-13 17:38:02.662245 7fe9bae887a0 20 read_log 194215'505 (194215'488) delete cd2c82b9/benchmark_data_storage0111-ib.pocstor_64498_object28703/head//22 by client.1049635.0:62906 2014-08-12 17:19:27.598334 2014-08-13 17:38:02.662266 7fe9bae887a0 20 read_log 194215'506 (194215'489) delete eb9c7ab9/benchmark_data_storage0111-ib.pocstor_64498_object29655/head//22 by client.1049635.0:63858 2014-08-12 17:19:27.936472 2014-08-13 17:38:02.662448 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.662476 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.662499 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Initial 0.003192 0 0.000000 2014-08-13 17:38:02.662519 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.662533 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.662548 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] log((0'0,194215'506], crt=192888'472) 2014-08-13 17:38:02.662564 7fe9bae887a0 10 osd.4 194478 pgid 22.2d1 coll 22.2d1_head 2014-08-13 17:38:02.662781 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.2d1 2014-08-13 17:38:02.662792 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.662837 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2d1(unlocked)] enter Initial 2014-08-13 17:38:02.662852 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.2d1(unlocked)] enter NotTrimming 2014-08-13 17:38:02.663059 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.663292 7fe9bae887a0 20 read_log 191648'32 (0'0) modify ad8a5ad1/benchmark_data_storage0111-ib.pocstor_40745_object2205/head//22 by client.273158.0:2206 2014-07-30 14:08:57.183342 2014-08-13 17:38:02.663328 7fe9bae887a0 20 read_log 191648'41 (0'0) modify 6051ead1/benchmark_data_storage0111-ib.pocstor_40745_object2679/head//22 by client.273158.0:2680 2014-07-30 14:08:59.759974 2014-08-13 17:38:02.663351 7fe9bae887a0 20 read_log 191648'49 (0'0) modify 33ceb2d1/benchmark_data_storage0111-ib.pocstor_40745_object3158/head//22 by client.273158.0:3159 2014-07-30 14:09:02.039944 2014-08-13 17:38:02.663372 7fe9bae887a0 20 read_log 191648'66 (0'0) modify 2aa79ad1/benchmark_data_storage0111-ib.pocstor_40745_object4564/head//22 by client.273158.0:4565 2014-07-30 14:09:09.120361 2014-08-13 17:38:02.663393 7fe9bae887a0 20 read_log 191648'122 (0'0) modify 9bf872d1/benchmark_data_storage0111-ib.pocstor_40745_object7240/head//22 by client.273158.0:7241 2014-07-30 14:09:22.037233 2014-08-13 17:38:02.663414 7fe9bae887a0 20 read_log 191648'147 (0'0) modify c6ef32d1/benchmark_data_storage0111-ib.pocstor_40745_object8901/head//22 by client.273158.0:8902 2014-07-30 14:09:30.781815 2014-08-13 17:38:02.663435 7fe9bae887a0 20 read_log 191648'167 (0'0) modify b1ca12d1/benchmark_data_storage0111-ib.pocstor_40745_object10248/head//22 by client.273158.0:10249 2014-07-30 14:09:37.043171 2014-08-13 17:38:02.663457 7fe9bae887a0 20 read_log 191648'171 (0'0) modify 19d27ad1/benchmark_data_storage0111-ib.pocstor_40745_object10417/head//22 by client.273158.0:10418 2014-07-30 14:09:37.905583 2014-08-13 17:38:02.663477 7fe9bae887a0 20 read_log 191648'185 (0'0) modify ed430ad1/benchmark_data_storage0111-ib.pocstor_40745_object10953/head//22 by client.273158.0:10954 2014-07-30 14:09:41.416379 2014-08-13 17:38:02.663499 7fe9bae887a0 20 read_log 191648'198 (0'0) modify b751aad1/benchmark_data_storage0111-ib.pocstor_40745_object11545/head//22 by client.273158.0:11546 2014-07-30 14:09:45.003651 2014-08-13 17:38:02.663520 7fe9bae887a0 20 read_log 191648'208 (0'0) modify 3ff242d1/benchmark_data_storage0111-ib.pocstor_40745_object12109/head//22 by client.273158.0:12110 2014-07-30 14:09:48.208782 2014-08-13 17:38:02.663541 7fe9bae887a0 20 read_log 191648'225 (0'0) modify 25bfa2d1/benchmark_data_storage0111-ib.pocstor_40745_object13090/head//22 by client.273158.0:13091 2014-07-30 14:09:52.861537 2014-08-13 17:38:02.663562 7fe9bae887a0 20 read_log 191648'233 (0'0) modify 5c2c4ad1/benchmark_data_storage0111-ib.pocstor_40745_object13619/head//22 by client.273158.0:13620 2014-07-30 14:09:55.245600 2014-08-13 17:38:02.663583 7fe9bae887a0 20 read_log 191648'274 (0'0) modify 8a96b2d1/benchmark_data_storage0111-ib.pocstor_40745_object16184/head//22 by client.273158.0:16185 2014-07-30 14:10:08.458019 2014-08-13 17:38:02.663604 7fe9bae887a0 20 read_log 191648'299 (0'0) modify 767962d1/benchmark_data_storage0111-ib.pocstor_40745_object17600/head//22 by client.273158.0:17601 2014-07-30 14:10:16.082096 2014-08-13 17:38:02.663625 7fe9bae887a0 20 read_log 191648'339 (0'0) modify cb9e4ad1/benchmark_data_storage0111-ib.pocstor_40745_object20445/head//22 by client.273158.0:20446 2014-07-30 14:10:33.191753 2014-08-13 17:38:02.663646 7fe9bae887a0 20 read_log 191648'357 (0'0) modify a3b7c2d1/benchmark_data_storage0111-ib.pocstor_40745_object21782/head//22 by client.273158.0:21783 2014-07-30 14:10:41.582122 2014-08-13 17:38:02.663667 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 4d3de2d1/benchmark_data_storage0111-ib.pocstor_51285_object8734/head//22 by client.275175.0:8735 2014-07-30 14:19:01.591699 2014-08-13 17:38:02.663688 7fe9bae887a0 20 read_log 191727'388 (0'0) modify eeee2d1/benchmark_data_storage0111-ib.pocstor_51285_object11614/head//22 by client.275175.0:11615 2014-07-30 14:19:10.087029 2014-08-13 17:38:02.663729 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 8c379ad1/benchmark_data_storage0111-ib.pocstor_51285_object12033/head//22 by client.275175.0:12034 2014-07-30 14:19:11.730939 2014-08-13 17:38:02.663752 7fe9bae887a0 20 read_log 191727'394 (0'0) modify f5c1aad1/benchmark_data_storage0111-ib.pocstor_51285_object14555/head//22 by client.275175.0:14556 2014-07-30 14:19:19.446768 2014-08-13 17:38:02.663773 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 9b83ead1/benchmark_data_storage0111-ib.pocstor_51285_object14639/head//22 by client.275175.0:14640 2014-07-30 14:19:19.685388 2014-08-13 17:38:02.663794 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 965722d1/benchmark_data_storage0111-ib.pocstor_51285_object16439/head//22 by client.275175.0:16440 2014-07-30 14:19:25.497346 2014-08-13 17:38:02.663815 7fe9bae887a0 20 read_log 191727'410 (0'0) modify 7ea23ad1/benchmark_data_storage0111-ib.pocstor_51285_object23247/head//22 by client.275175.0:23248 2014-07-30 14:19:46.728335 2014-08-13 17:38:02.663836 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 8fb5e2d1/benchmark_data_storage0111-ib.pocstor_51285_object23868/head//22 by client.275175.0:23869 2014-07-30 14:19:48.464742 2014-08-13 17:38:02.663857 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 4644dad1/benchmark_data_storage0111-ib.pocstor_51285_object32821/head//22 by client.275175.0:32822 2014-07-30 14:20:15.294467 2014-08-13 17:38:02.663878 7fe9bae887a0 20 read_log 191727'427 (0'0) modify f1dc3ad1/benchmark_data_storage0111-ib.pocstor_51285_object33057/head//22 by client.275175.0:33058 2014-07-30 14:20:15.957920 2014-08-13 17:38:02.663902 7fe9bae887a0 20 read_log 191727'428 (0'0) modify fae99ad1/benchmark_data_storage0111-ib.pocstor_51285_object33486/head//22 by client.275175.0:33487 2014-07-30 14:20:17.431133 2014-08-13 17:38:02.663923 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 11d45ad1/benchmark_data_storage0111-ib.pocstor_51285_object33754/head//22 by client.275175.0:33755 2014-07-30 14:20:18.293502 2014-08-13 17:38:02.663944 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 41c42d1/benchmark_data_storage0111-ib.pocstor_51285_object33986/head//22 by client.275175.0:33987 2014-07-30 14:20:18.953694 2014-08-13 17:38:02.663965 7fe9bae887a0 20 read_log 191727'433 (0'0) modify aa6cd2d1/benchmark_data_storage0111-ib.pocstor_51285_object35827/head//22 by client.275175.0:35828 2014-07-30 14:20:24.726318 2014-08-13 17:38:02.663986 7fe9bae887a0 20 read_log 191727'435 (0'0) modify f9d62d1/benchmark_data_storage0111-ib.pocstor_51285_object36377/head//22 by client.275175.0:36378 2014-07-30 14:20:26.407419 2014-08-13 17:38:02.664007 7fe9bae887a0 20 read_log 191727'436 (0'0) modify a8372d1/benchmark_data_storage0111-ib.pocstor_51285_object37621/head//22 by client.275175.0:37622 2014-07-30 14:20:30.186969 2014-08-13 17:38:02.664028 7fe9bae887a0 20 read_log 191727'437 (0'0) modify d2b93ad1/benchmark_data_storage0111-ib.pocstor_51285_object38275/head//22 by client.275175.0:38276 2014-07-30 14:20:32.030022 2014-08-13 17:38:02.664049 7fe9bae887a0 20 read_log 192666'438 (0'0) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1273 2014-07-30 16:28:47.899928 2014-08-13 17:38:02.664104 7fe9bae887a0 20 read_log 192666'439 (192666'438) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1283 2014-07-30 16:28:48.102920 2014-08-13 17:38:02.664131 7fe9bae887a0 20 read_log 192666'440 (192666'439) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1284 2014-07-30 16:28:48.106920 2014-08-13 17:38:02.664152 7fe9bae887a0 20 read_log 192666'441 (192666'440) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1285 2014-07-30 16:28:48.149918 2014-08-13 17:38:02.664215 7fe9bae887a0 20 read_log 192666'442 (192666'441) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1286 2014-07-30 16:28:48.153918 2014-08-13 17:38:02.664281 7fe9bae887a0 20 read_log 192666'443 (192666'442) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1348 2014-07-30 16:28:48.910889 2014-08-13 17:38:02.664310 7fe9bae887a0 20 read_log 192666'444 (192666'443) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1349 2014-07-30 16:28:48.914889 2014-08-13 17:38:02.664331 7fe9bae887a0 20 read_log 192666'445 (192666'444) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:662 2014-07-30 17:22:39.389000 2014-08-13 17:38:02.664352 7fe9bae887a0 20 read_log 192666'446 (192666'445) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:897 2014-07-30 17:22:40.662000 2014-08-13 17:38:02.664372 7fe9bae887a0 20 read_log 192666'447 (192666'446) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:907 2014-07-30 17:22:40.700000 2014-08-13 17:38:02.664396 7fe9bae887a0 20 read_log 192666'448 (192666'447) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:908 2014-07-30 17:22:40.705000 2014-08-13 17:38:02.664416 7fe9bae887a0 20 read_log 192666'449 (192666'448) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:909 2014-07-30 17:22:40.708000 2014-08-13 17:38:02.664437 7fe9bae887a0 20 read_log 192666'450 (192666'449) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:910 2014-07-30 17:22:40.711000 2014-08-13 17:38:02.664457 7fe9bae887a0 20 read_log 192666'451 (192666'450) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:972 2014-07-30 17:22:41.059000 2014-08-13 17:38:02.664478 7fe9bae887a0 20 read_log 192666'452 (192666'451) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:973 2014-07-30 17:22:41.063000 2014-08-13 17:38:02.664499 7fe9bae887a0 20 read_log 192674'453 (0'0) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1710 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664519 7fe9bae887a0 20 read_log 192674'454 (192674'453) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1711 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664540 7fe9bae887a0 20 read_log 192674'455 (192674'454) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1712 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664560 7fe9bae887a0 20 read_log 192674'456 (192674'455) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1713 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664581 7fe9bae887a0 20 read_log 192674'457 (192674'456) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1714 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664601 7fe9bae887a0 20 read_log 192674'458 (192674'457) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1715 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664622 7fe9bae887a0 20 read_log 192674'459 (192674'458) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1716 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664642 7fe9bae887a0 20 read_log 192674'460 (192674'459) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1717 2014-07-31 18:03:43.479000 2014-08-13 17:38:02.664662 7fe9bae887a0 20 read_log 192674'461 (0'0) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4759 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664683 7fe9bae887a0 20 read_log 192674'462 (192674'461) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4760 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664703 7fe9bae887a0 20 read_log 192674'463 (192674'462) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4761 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664723 7fe9bae887a0 20 read_log 192674'464 (192674'463) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4762 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664756 7fe9bae887a0 20 read_log 192674'465 (192674'464) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4763 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664778 7fe9bae887a0 20 read_log 192674'466 (192674'465) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4764 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664798 7fe9bae887a0 20 read_log 192674'467 (192674'466) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4765 2014-07-31 18:03:46.699000 2014-08-13 17:38:02.664819 7fe9bae887a0 20 read_log 192674'468 (192674'467) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4766 2014-07-31 18:03:46.726000 2014-08-13 17:38:02.664840 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 5ee3e2d1/benchmark_data_storage0111-ib.pocstor_35625_object2742/head//22 by client.940330.0:2743 2014-08-11 16:29:51.404484 2014-08-13 17:38:02.664861 7fe9bae887a0 20 read_log 192888'470 (0'0) modify 7d6252d1/benchmark_data_storage0111-ib.pocstor_35625_object2854/head//22 by client.940330.0:2855 2014-08-11 16:29:51.988624 2014-08-13 17:38:02.664885 7fe9bae887a0 20 read_log 192888'471 (0'0) modify 15bb5ad1/benchmark_data_storage0111-ib.pocstor_35625_object3322/head//22 by client.940330.0:3323 2014-08-11 16:29:53.821111 2014-08-13 17:38:02.664906 7fe9bae887a0 20 read_log 192888'472 (0'0) modify 232732d1/benchmark_data_storage0111-ib.pocstor_35625_object10456/head//22 by client.940330.0:10457 2014-08-11 16:30:19.567983 2014-08-13 17:38:02.664926 7fe9bae887a0 20 read_log 192888'473 (0'0) modify 33b6b2d1/benchmark_data_storage0111-ib.pocstor_35625_object10838/head//22 by client.940330.0:10839 2014-08-11 16:30:21.172030 2014-08-13 17:38:02.664947 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 891c8ad1/benchmark_data_storage0111-ib.pocstor_35625_object10903/head//22 by client.940330.0:10904 2014-08-11 16:30:21.422951 2014-08-13 17:38:02.664967 7fe9bae887a0 20 read_log 192888'475 (0'0) modify 841902d1/benchmark_data_storage0111-ib.pocstor_35625_object11927/head//22 by client.940330.0:11928 2014-08-11 16:30:25.562521 2014-08-13 17:38:02.664987 7fe9bae887a0 20 read_log 192888'476 (0'0) modify c26f02d1/benchmark_data_storage0111-ib.pocstor_35625_object22095/head//22 by client.940330.0:22096 2014-08-11 16:31:04.928366 2014-08-13 17:38:02.665008 7fe9bae887a0 20 read_log 192888'477 (0'0) modify cbe69ad1/benchmark_data_storage0111-ib.pocstor_35625_object23792/head//22 by client.940330.0:23793 2014-08-11 16:31:11.390383 2014-08-13 17:38:02.665029 7fe9bae887a0 20 read_log 192888'478 (0'0) modify 45ea3ad1/benchmark_data_storage0111-ib.pocstor_35625_object25656/head//22 by client.940330.0:25657 2014-08-11 16:31:18.443058 2014-08-13 17:38:02.665049 7fe9bae887a0 20 read_log 192888'479 (0'0) modify d94e7ad1/benchmark_data_storage0111-ib.pocstor_35625_object27182/head//22 by client.940330.0:27183 2014-08-11 16:31:24.506100 2014-08-13 17:38:02.665069 7fe9bae887a0 20 read_log 192888'480 (0'0) modify 160f5ad1/benchmark_data_storage0111-ib.pocstor_35625_object27494/head//22 by client.940330.0:27495 2014-08-11 16:31:25.714233 2014-08-13 17:38:02.665090 7fe9bae887a0 20 read_log 192888'481 (0'0) modify 153ef2d1/benchmark_data_storage0111-ib.pocstor_35625_object27643/head//22 by client.940330.0:27644 2014-08-11 16:31:26.296654 2014-08-13 17:38:02.665110 7fe9bae887a0 20 read_log 192888'482 (0'0) modify 34074ad1/benchmark_data_storage0111-ib.pocstor_35625_object28159/head//22 by client.940330.0:28160 2014-08-11 16:31:28.352484 2014-08-13 17:38:02.665130 7fe9bae887a0 20 read_log 192888'483 (0'0) modify b0cf2ad1/benchmark_data_storage0111-ib.pocstor_35625_object30237/head//22 by client.940330.0:30238 2014-08-11 16:31:36.355808 2014-08-13 17:38:02.665151 7fe9bae887a0 20 read_log 192888'484 (0'0) modify e61dad1/benchmark_data_storage0111-ib.pocstor_35625_object30899/head//22 by client.940330.0:30900 2014-08-11 16:31:38.836073 2014-08-13 17:38:02.665223 7fe9bae887a0 20 read_log 192888'485 (192888'469) delete 5ee3e2d1/benchmark_data_storage0111-ib.pocstor_35625_object2742/head//22 by client.940330.0:34050 2014-08-11 16:31:42.181877 2014-08-13 17:38:02.665253 7fe9bae887a0 20 read_log 192888'486 (192888'470) delete 7d6252d1/benchmark_data_storage0111-ib.pocstor_35625_object2854/head//22 by client.940330.0:34162 2014-08-11 16:31:42.212112 2014-08-13 17:38:02.665275 7fe9bae887a0 20 read_log 192888'487 (192888'471) delete 15bb5ad1/benchmark_data_storage0111-ib.pocstor_35625_object3322/head//22 by client.940330.0:34630 2014-08-11 16:31:42.341265 2014-08-13 17:38:02.665296 7fe9bae887a0 20 read_log 192888'488 (192888'472) delete 232732d1/benchmark_data_storage0111-ib.pocstor_35625_object10456/head//22 by client.940330.0:41764 2014-08-11 16:31:44.240347 2014-08-13 17:38:02.665318 7fe9bae887a0 20 read_log 192888'489 (192888'473) delete 33b6b2d1/benchmark_data_storage0111-ib.pocstor_35625_object10838/head//22 by client.940330.0:42146 2014-08-11 16:31:44.352435 2014-08-13 17:38:02.665339 7fe9bae887a0 20 read_log 192888'490 (192888'474) delete 891c8ad1/benchmark_data_storage0111-ib.pocstor_35625_object10903/head//22 by client.940330.0:42211 2014-08-11 16:31:44.370802 2014-08-13 17:38:02.665360 7fe9bae887a0 20 read_log 192888'491 (192888'475) delete 841902d1/benchmark_data_storage0111-ib.pocstor_35625_object11927/head//22 by client.940330.0:43235 2014-08-11 16:31:44.638929 2014-08-13 17:38:02.665380 7fe9bae887a0 20 read_log 192888'492 (192888'476) delete c26f02d1/benchmark_data_storage0111-ib.pocstor_35625_object22095/head//22 by client.940330.0:53403 2014-08-11 16:31:47.472130 2014-08-13 17:38:02.665401 7fe9bae887a0 20 read_log 192888'493 (192888'477) delete cbe69ad1/benchmark_data_storage0111-ib.pocstor_35625_object23792/head//22 by client.940330.0:55100 2014-08-11 16:31:47.974299 2014-08-13 17:38:02.665422 7fe9bae887a0 20 read_log 192888'494 (192888'478) delete 45ea3ad1/benchmark_data_storage0111-ib.pocstor_35625_object25656/head//22 by client.940330.0:56964 2014-08-11 16:31:48.545771 2014-08-13 17:38:02.665443 7fe9bae887a0 20 read_log 192888'495 (192888'479) delete d94e7ad1/benchmark_data_storage0111-ib.pocstor_35625_object27182/head//22 by client.940330.0:58490 2014-08-11 16:31:48.978093 2014-08-13 17:38:02.665464 7fe9bae887a0 20 read_log 192888'496 (192888'480) delete 160f5ad1/benchmark_data_storage0111-ib.pocstor_35625_object27494/head//22 by client.940330.0:58802 2014-08-11 16:31:49.063934 2014-08-13 17:38:02.665485 7fe9bae887a0 20 read_log 192888'497 (192888'481) delete 153ef2d1/benchmark_data_storage0111-ib.pocstor_35625_object27643/head//22 by client.940330.0:58951 2014-08-11 16:31:49.105221 2014-08-13 17:38:02.665506 7fe9bae887a0 20 read_log 192888'498 (192888'482) delete 34074ad1/benchmark_data_storage0111-ib.pocstor_35625_object28159/head//22 by client.940330.0:59467 2014-08-11 16:31:49.248838 2014-08-13 17:38:02.665527 7fe9bae887a0 20 read_log 192888'499 (192888'483) delete b0cf2ad1/benchmark_data_storage0111-ib.pocstor_35625_object30237/head//22 by client.940330.0:61545 2014-08-11 16:31:49.826953 2014-08-13 17:38:02.665548 7fe9bae887a0 20 read_log 192888'500 (192888'484) delete e61dad1/benchmark_data_storage0111-ib.pocstor_35625_object30899/head//22 by client.940330.0:62207 2014-08-11 16:31:50.023502 2014-08-13 17:38:02.665570 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 99a1d2d1/benchmark_data_storage0111-ib.pocstor_64498_object3998/head//22 by client.1049635.0:3999 2014-08-12 17:17:30.080865 2014-08-13 17:38:02.665592 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 3493cad1/benchmark_data_storage0111-ib.pocstor_64498_object7740/head//22 by client.1049635.0:7741 2014-08-12 17:17:41.278668 2014-08-13 17:38:02.665616 7fe9bae887a0 20 read_log 194215'503 (0'0) modify 4268cad1/benchmark_data_storage0111-ib.pocstor_64498_object8489/head//22 by client.1049635.0:8490 2014-08-12 17:17:43.528281 2014-08-13 17:38:02.665637 7fe9bae887a0 20 read_log 194215'504 (0'0) modify 30f31ad1/benchmark_data_storage0111-ib.pocstor_64498_object10219/head//22 by client.1049635.0:10220 2014-08-12 17:17:48.549443 2014-08-13 17:38:02.665671 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 7ceea2d1/benchmark_data_storage0111-ib.pocstor_64498_object13546/head//22 by client.1049635.0:13547 2014-08-12 17:17:59.146307 2014-08-13 17:38:02.665693 7fe9bae887a0 20 read_log 194215'506 (0'0) modify 70405ad1/benchmark_data_storage0111-ib.pocstor_64498_object18763/head//22 by client.1049635.0:18764 2014-08-12 17:18:18.042437 2014-08-13 17:38:02.665714 7fe9bae887a0 20 read_log 194215'507 (0'0) modify e09ae2d1/benchmark_data_storage0111-ib.pocstor_64498_object20569/head//22 by client.1049635.0:20570 2014-08-12 17:18:24.926539 2014-08-13 17:38:02.665735 7fe9bae887a0 20 read_log 194215'508 (0'0) modify 411892d1/benchmark_data_storage0111-ib.pocstor_64498_object25202/head//22 by client.1049635.0:25203 2014-08-12 17:18:42.459424 2014-08-13 17:38:02.665756 7fe9bae887a0 20 read_log 194215'509 (0'0) modify 6e233ad1/benchmark_data_storage0111-ib.pocstor_64498_object25327/head//22 by client.1049635.0:25328 2014-08-12 17:18:42.926392 2014-08-13 17:38:02.665777 7fe9bae887a0 20 read_log 194215'510 (0'0) modify 874ddad1/benchmark_data_storage0111-ib.pocstor_64498_object25950/head//22 by client.1049635.0:25951 2014-08-12 17:18:45.240085 2014-08-13 17:38:02.665797 7fe9bae887a0 20 read_log 194215'511 (0'0) modify 4a9dcad1/benchmark_data_storage0111-ib.pocstor_64498_object26261/head//22 by client.1049635.0:26262 2014-08-12 17:18:46.393837 2014-08-13 17:38:02.665818 7fe9bae887a0 20 read_log 194215'512 (0'0) modify 504c7ad1/benchmark_data_storage0111-ib.pocstor_64498_object27514/head//22 by client.1049635.0:27515 2014-08-12 17:18:51.073620 2014-08-13 17:38:02.665839 7fe9bae887a0 20 read_log 194215'513 (0'0) modify 8934e2d1/benchmark_data_storage0111-ib.pocstor_64498_object31138/head//22 by client.1049635.0:31139 2014-08-12 17:19:05.099541 2014-08-13 17:38:02.665859 7fe9bae887a0 20 read_log 194215'514 (194215'501) delete 99a1d2d1/benchmark_data_storage0111-ib.pocstor_64498_object3998/head//22 by client.1049635.0:38201 2014-08-12 17:19:19.274120 2014-08-13 17:38:02.665880 7fe9bae887a0 20 read_log 194215'515 (194215'502) delete 3493cad1/benchmark_data_storage0111-ib.pocstor_64498_object7740/head//22 by client.1049635.0:41943 2014-08-12 17:19:20.523490 2014-08-13 17:38:02.665901 7fe9bae887a0 20 read_log 194215'516 (194215'503) delete 4268cad1/benchmark_data_storage0111-ib.pocstor_64498_object8489/head//22 by client.1049635.0:42692 2014-08-12 17:19:20.756972 2014-08-13 17:38:02.665922 7fe9bae887a0 20 read_log 194215'517 (194215'504) delete 30f31ad1/benchmark_data_storage0111-ib.pocstor_64498_object10219/head//22 by client.1049635.0:44422 2014-08-12 17:19:21.335209 2014-08-13 17:38:02.665943 7fe9bae887a0 20 read_log 194215'518 (194215'505) delete 7ceea2d1/benchmark_data_storage0111-ib.pocstor_64498_object13546/head//22 by client.1049635.0:47749 2014-08-12 17:19:22.416834 2014-08-13 17:38:02.665964 7fe9bae887a0 20 read_log 194215'519 (194215'506) delete 70405ad1/benchmark_data_storage0111-ib.pocstor_64498_object18763/head//22 by client.1049635.0:52966 2014-08-12 17:19:24.085344 2014-08-13 17:38:02.665985 7fe9bae887a0 20 read_log 194215'520 (194215'507) delete e09ae2d1/benchmark_data_storage0111-ib.pocstor_64498_object20569/head//22 by client.1049635.0:54772 2014-08-12 17:19:24.729533 2014-08-13 17:38:02.666044 7fe9bae887a0 20 read_log 194215'521 (194215'508) delete 411892d1/benchmark_data_storage0111-ib.pocstor_64498_object25202/head//22 by client.1049635.0:59405 2014-08-12 17:19:26.390213 2014-08-13 17:38:02.666066 7fe9bae887a0 20 read_log 194215'522 (194215'509) delete 6e233ad1/benchmark_data_storage0111-ib.pocstor_64498_object25327/head//22 by client.1049635.0:59530 2014-08-12 17:19:26.428187 2014-08-13 17:38:02.666087 7fe9bae887a0 20 read_log 194215'523 (194215'510) delete 874ddad1/benchmark_data_storage0111-ib.pocstor_64498_object25950/head//22 by client.1049635.0:60153 2014-08-12 17:19:26.639323 2014-08-13 17:38:02.666107 7fe9bae887a0 20 read_log 194215'524 (194215'511) delete 4a9dcad1/benchmark_data_storage0111-ib.pocstor_64498_object26261/head//22 by client.1049635.0:60464 2014-08-12 17:19:26.745244 2014-08-13 17:38:02.666130 7fe9bae887a0 20 read_log 194215'525 (194215'512) delete 504c7ad1/benchmark_data_storage0111-ib.pocstor_64498_object27514/head//22 by client.1049635.0:61717 2014-08-12 17:19:27.186207 2014-08-13 17:38:02.666151 7fe9bae887a0 20 read_log 194215'526 (194215'513) delete 8934e2d1/benchmark_data_storage0111-ib.pocstor_64498_object31138/head//22 by client.1049635.0:65341 2014-08-12 17:19:28.428102 2014-08-13 17:38:02.666360 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.666392 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=0 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.666418 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=0 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Initial 0.003582 0 0.000000 2014-08-13 17:38:02.666440 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=0 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.666454 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=0 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.666469 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] log((0'0,194215'526], crt=192888'500) 2014-08-13 17:38:02.666485 7fe9bae887a0 10 osd.4 194478 pgid 22.30f coll 22.30f_head 2014-08-13 17:38:02.666682 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.30f 2014-08-13 17:38:02.666698 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.666744 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.30f(unlocked)] enter Initial 2014-08-13 17:38:02.666763 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.30f(unlocked)] enter NotTrimming 2014-08-13 17:38:02.666969 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.667149 7fe9bae887a0 20 read_log 191648'14 (0'0) modify 5257fb0f/benchmark_data_storage0111-ib.pocstor_40745_object1025/head//22 by client.273158.0:1026 2014-07-30 14:08:52.574943 2014-08-13 17:38:02.667220 7fe9bae887a0 20 read_log 191648'207 (0'0) modify 4417b30f/benchmark_data_storage0111-ib.pocstor_40745_object13399/head//22 by client.273158.0:13400 2014-07-30 14:09:54.272143 2014-08-13 17:38:02.667278 7fe9bae887a0 20 read_log 191648'262 (0'0) modify d4209b0f/benchmark_data_storage0111-ib.pocstor_40745_object16906/head//22 by client.273158.0:16907 2014-07-30 14:10:12.278627 2014-08-13 17:38:02.667304 7fe9bae887a0 20 read_log 191648'285 (0'0) modify ebabb0f/benchmark_data_storage0111-ib.pocstor_40745_object18866/head//22 by client.273158.0:18867 2014-07-30 14:10:23.030276 2014-08-13 17:38:02.667326 7fe9bae887a0 20 read_log 191648'307 (0'0) modify 39bab0f/benchmark_data_storage0111-ib.pocstor_40745_object20214/head//22 by client.273158.0:20215 2014-07-30 14:10:32.320903 2014-08-13 17:38:02.667348 7fe9bae887a0 20 read_log 191648'335 (0'0) modify e2cd530f/benchmark_data_storage0111-ib.pocstor_40745_object21690/head//22 by client.273158.0:21691 2014-07-30 14:10:41.219192 2014-08-13 17:38:02.667369 7fe9bae887a0 20 read_log 191648'338 (0'0) modify 17b1db0f/benchmark_data_storage0111-ib.pocstor_40745_object22062/head//22 by client.273158.0:22063 2014-07-30 14:10:44.432712 2014-08-13 17:38:02.667390 7fe9bae887a0 20 read_log 191727'357 (0'0) modify 22167b0f/benchmark_data_storage0111-ib.pocstor_51285_object4699/head//22 by client.275175.0:4700 2014-07-30 14:18:49.453647 2014-08-13 17:38:02.667411 7fe9bae887a0 20 read_log 191727'359 (0'0) modify bdb78b0f/benchmark_data_storage0111-ib.pocstor_51285_object5308/head//22 by client.275175.0:5309 2014-07-30 14:18:51.261359 2014-08-13 17:38:02.667431 7fe9bae887a0 20 read_log 191727'362 (0'0) modify 1761cb0f/benchmark_data_storage0111-ib.pocstor_51285_object8972/head//22 by client.275175.0:8973 2014-07-30 14:19:02.269923 2014-08-13 17:38:02.667452 7fe9bae887a0 20 read_log 191727'367 (0'0) modify a8aea30f/benchmark_data_storage0111-ib.pocstor_51285_object11092/head//22 by client.275175.0:11093 2014-07-30 14:19:08.302941 2014-08-13 17:38:02.667473 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 3eedfb0f/benchmark_data_storage0111-ib.pocstor_51285_object11862/head//22 by client.275175.0:11863 2014-07-30 14:19:10.909377 2014-08-13 17:38:02.667494 7fe9bae887a0 20 read_log 191727'370 (0'0) modify 8ac5db0f/benchmark_data_storage0111-ib.pocstor_51285_object12222/head//22 by client.275175.0:12223 2014-07-30 14:19:12.261947 2014-08-13 17:38:02.667515 7fe9bae887a0 20 read_log 191727'374 (0'0) modify f534cb0f/benchmark_data_storage0111-ib.pocstor_51285_object13640/head//22 by client.275175.0:13641 2014-07-30 14:19:16.640561 2014-08-13 17:38:02.667536 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 2983130f/benchmark_data_storage0111-ib.pocstor_51285_object18003/head//22 by client.275175.0:18004 2014-07-30 14:19:29.967637 2014-08-13 17:38:02.667559 7fe9bae887a0 20 read_log 191727'380 (0'0) modify c59ab0f/benchmark_data_storage0111-ib.pocstor_51285_object18111/head//22 by client.275175.0:18112 2014-07-30 14:19:30.266706 2014-08-13 17:38:02.667581 7fe9bae887a0 20 read_log 191727'381 (0'0) modify b2726b0f/benchmark_data_storage0111-ib.pocstor_51285_object18395/head//22 by client.275175.0:18396 2014-07-30 14:19:31.178824 2014-08-13 17:38:02.667603 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 4c94630f/benchmark_data_storage0111-ib.pocstor_51285_object18541/head//22 by client.275175.0:18542 2014-07-30 14:19:31.602079 2014-08-13 17:38:02.667623 7fe9bae887a0 20 read_log 191727'389 (0'0) modify f7d22b0f/benchmark_data_storage0111-ib.pocstor_51285_object23433/head//22 by client.275175.0:23434 2014-07-30 14:19:47.249000 2014-08-13 17:38:02.667644 7fe9bae887a0 20 read_log 191727'394 (0'0) modify e8f9c30f/benchmark_data_storage0111-ib.pocstor_51285_object28246/head//22 by client.275175.0:28247 2014-07-30 14:20:01.717667 2014-08-13 17:38:02.667678 7fe9bae887a0 20 read_log 191727'395 (0'0) modify a59f830f/benchmark_data_storage0111-ib.pocstor_51285_object28870/head//22 by client.275175.0:28871 2014-07-30 14:20:03.527055 2014-08-13 17:38:02.667700 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 534c0b0f/benchmark_data_storage0111-ib.pocstor_51285_object28997/head//22 by client.275175.0:28998 2014-07-30 14:20:03.880283 2014-08-13 17:38:02.667721 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 1a415b0f/benchmark_data_storage0111-ib.pocstor_51285_object31144/head//22 by client.275175.0:31145 2014-07-30 14:20:10.396827 2014-08-13 17:38:02.667741 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 471e6b0f/benchmark_data_storage0111-ib.pocstor_51285_object33212/head//22 by client.275175.0:33213 2014-07-30 14:20:16.456685 2014-08-13 17:38:02.667762 7fe9bae887a0 20 read_log 191727'405 (0'0) modify 20ae0b0f/benchmark_data_storage0111-ib.pocstor_51285_object36282/head//22 by client.275175.0:36283 2014-07-30 14:20:26.143656 2014-08-13 17:38:02.667783 7fe9bae887a0 20 read_log 191727'406 (0'0) modify fe11330f/benchmark_data_storage0111-ib.pocstor_51285_object36427/head//22 by client.275175.0:36428 2014-07-30 14:20:26.555124 2014-08-13 17:38:02.667804 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 8426d30f/benchmark_data_storage0111-ib.pocstor_51285_object36896/head//22 by client.275175.0:36897 2014-07-30 14:20:28.086699 2014-08-13 17:38:02.667825 7fe9bae887a0 20 read_log 192888'412 (0'0) modify 6a64b30f/benchmark_data_storage0111-ib.pocstor_35625_object426/head//22 by client.940330.0:427 2014-08-11 16:29:42.386662 2014-08-13 17:38:02.667846 7fe9bae887a0 20 read_log 192888'413 (0'0) modify bd36830f/benchmark_data_storage0111-ib.pocstor_35625_object10675/head//22 by client.940330.0:10676 2014-08-11 16:30:20.536512 2014-08-13 17:38:02.667866 7fe9bae887a0 20 read_log 192888'414 (0'0) modify 43beb0f/benchmark_data_storage0111-ib.pocstor_35625_object11193/head//22 by client.940330.0:11194 2014-08-11 16:30:22.556156 2014-08-13 17:38:02.667891 7fe9bae887a0 20 read_log 192888'415 (0'0) modify d6919b0f/benchmark_data_storage0111-ib.pocstor_35625_object11219/head//22 by client.940330.0:11220 2014-08-11 16:30:22.659707 2014-08-13 17:38:02.667912 7fe9bae887a0 20 read_log 192888'416 (0'0) modify c9ad7b0f/benchmark_data_storage0111-ib.pocstor_35625_object18202/head//22 by client.940330.0:18203 2014-08-11 16:30:49.550388 2014-08-13 17:38:02.667933 7fe9bae887a0 20 read_log 192888'417 (0'0) modify 3598bb0f/benchmark_data_storage0111-ib.pocstor_35625_object19630/head//22 by client.940330.0:19631 2014-08-11 16:30:55.257991 2014-08-13 17:38:02.667954 7fe9bae887a0 20 read_log 192888'418 (0'0) modify 8f59430f/benchmark_data_storage0111-ib.pocstor_35625_object22716/head//22 by client.940330.0:22717 2014-08-11 16:31:07.352494 2014-08-13 17:38:02.667974 7fe9bae887a0 20 read_log 192888'419 (0'0) modify 203f9b0f/benchmark_data_storage0111-ib.pocstor_35625_object23891/head//22 by client.940330.0:23892 2014-08-11 16:31:11.750035 2014-08-13 17:38:02.667995 7fe9bae887a0 20 read_log 192888'420 (0'0) modify 11a26b0f/benchmark_data_storage0111-ib.pocstor_35625_object25834/head//22 by client.940330.0:25835 2014-08-11 16:31:19.140129 2014-08-13 17:38:02.668019 7fe9bae887a0 20 read_log 192888'421 (0'0) modify 3c5b5b0f/benchmark_data_storage0111-ib.pocstor_35625_object26759/head//22 by client.940330.0:26760 2014-08-11 16:31:22.871968 2014-08-13 17:38:02.668040 7fe9bae887a0 20 read_log 192888'422 (192888'412) delete 6a64b30f/benchmark_data_storage0111-ib.pocstor_35625_object426/head//22 by client.940330.0:31734 2014-08-11 16:31:41.601325 2014-08-13 17:38:02.668062 7fe9bae887a0 20 read_log 192888'423 (192888'413) delete bd36830f/benchmark_data_storage0111-ib.pocstor_35625_object10675/head//22 by client.940330.0:41983 2014-08-11 16:31:44.302893 2014-08-13 17:38:02.668133 7fe9bae887a0 20 read_log 192888'424 (192888'414) delete 43beb0f/benchmark_data_storage0111-ib.pocstor_35625_object11193/head//22 by client.940330.0:42501 2014-08-11 16:31:44.455158 2014-08-13 17:38:02.668185 7fe9bae887a0 20 read_log 192888'425 (192888'415) delete d6919b0f/benchmark_data_storage0111-ib.pocstor_35625_object11219/head//22 by client.940330.0:42527 2014-08-11 16:31:44.461937 2014-08-13 17:38:02.668213 7fe9bae887a0 20 read_log 192888'426 (192888'416) delete c9ad7b0f/benchmark_data_storage0111-ib.pocstor_35625_object18202/head//22 by client.940330.0:49510 2014-08-11 16:31:46.352948 2014-08-13 17:38:02.668235 7fe9bae887a0 20 read_log 192888'427 (192888'417) delete 3598bb0f/benchmark_data_storage0111-ib.pocstor_35625_object19630/head//22 by client.940330.0:50938 2014-08-11 16:31:46.759002 2014-08-13 17:38:02.668256 7fe9bae887a0 20 read_log 192888'428 (192888'418) delete 8f59430f/benchmark_data_storage0111-ib.pocstor_35625_object22716/head//22 by client.940330.0:54024 2014-08-11 16:31:47.669167 2014-08-13 17:38:02.668277 7fe9bae887a0 20 read_log 192888'429 (192888'419) delete 203f9b0f/benchmark_data_storage0111-ib.pocstor_35625_object23891/head//22 by client.940330.0:55199 2014-08-11 16:31:48.003009 2014-08-13 17:38:02.668298 7fe9bae887a0 20 read_log 192888'430 (192888'420) delete 11a26b0f/benchmark_data_storage0111-ib.pocstor_35625_object25834/head//22 by client.940330.0:57142 2014-08-11 16:31:48.597797 2014-08-13 17:38:02.668319 7fe9bae887a0 20 read_log 192888'431 (192888'421) delete 3c5b5b0f/benchmark_data_storage0111-ib.pocstor_35625_object26759/head//22 by client.940330.0:58067 2014-08-11 16:31:48.859550 2014-08-13 17:38:02.668344 7fe9bae887a0 20 read_log 194215'432 (0'0) modify 8dbbab0f/benchmark_data_storage0111-ib.pocstor_64498_object422/head//22 by client.1049635.0:423 2014-08-12 17:17:18.504715 2014-08-13 17:38:02.668365 7fe9bae887a0 20 read_log 194215'433 (0'0) modify 9bd41b0f/benchmark_data_storage0111-ib.pocstor_64498_object2589/head//22 by client.1049635.0:2590 2014-08-12 17:17:25.506150 2014-08-13 17:38:02.668386 7fe9bae887a0 20 read_log 194215'434 (0'0) modify 827e730f/benchmark_data_storage0111-ib.pocstor_64498_object2931/head//22 by client.1049635.0:2932 2014-08-12 17:17:26.603816 2014-08-13 17:38:02.668407 7fe9bae887a0 20 read_log 194215'435 (0'0) modify 8e27830f/benchmark_data_storage0111-ib.pocstor_64498_object2978/head//22 by client.1049635.0:2979 2014-08-12 17:17:26.777824 2014-08-13 17:38:02.668428 7fe9bae887a0 20 read_log 194215'436 (0'0) modify 546f6b0f/benchmark_data_storage0111-ib.pocstor_64498_object3949/head//22 by client.1049635.0:3950 2014-08-12 17:17:29.933062 2014-08-13 17:38:02.668449 7fe9bae887a0 20 read_log 194215'437 (0'0) modify 3f5d730f/benchmark_data_storage0111-ib.pocstor_64498_object6863/head//22 by client.1049635.0:6864 2014-08-12 17:17:38.693985 2014-08-13 17:38:02.668474 7fe9bae887a0 20 read_log 194215'438 (0'0) modify 6d99130f/benchmark_data_storage0111-ib.pocstor_64498_object6897/head//22 by client.1049635.0:6898 2014-08-12 17:17:38.820152 2014-08-13 17:38:02.668495 7fe9bae887a0 20 read_log 194215'439 (0'0) modify e5f43b0f/benchmark_data_storage0111-ib.pocstor_64498_object9706/head//22 by client.1049635.0:9707 2014-08-12 17:17:47.090526 2014-08-13 17:38:02.668516 7fe9bae887a0 20 read_log 194215'440 (0'0) modify 92df230f/benchmark_data_storage0111-ib.pocstor_64498_object10963/head//22 by client.1049635.0:10964 2014-08-12 17:17:50.987623 2014-08-13 17:38:02.668537 7fe9bae887a0 20 read_log 194215'441 (0'0) modify e0a1630f/benchmark_data_storage0111-ib.pocstor_64498_object12246/head//22 by client.1049635.0:12247 2014-08-12 17:17:55.203215 2014-08-13 17:38:02.668558 7fe9bae887a0 20 read_log 194215'442 (0'0) modify 561af30f/benchmark_data_storage0111-ib.pocstor_64498_object20013/head//22 by client.1049635.0:20014 2014-08-12 17:18:22.827657 2014-08-13 17:38:02.668594 7fe9bae887a0 20 read_log 194215'443 (0'0) modify 6012730f/benchmark_data_storage0111-ib.pocstor_64498_object21158/head//22 by client.1049635.0:21159 2014-08-12 17:18:27.184614 2014-08-13 17:38:02.668617 7fe9bae887a0 20 read_log 194215'444 (0'0) modify c09930f/benchmark_data_storage0111-ib.pocstor_64498_object22881/head//22 by client.1049635.0:22882 2014-08-12 17:18:33.694128 2014-08-13 17:38:02.668638 7fe9bae887a0 20 read_log 194215'445 (0'0) modify e433430f/benchmark_data_storage0111-ib.pocstor_64498_object26306/head//22 by client.1049635.0:26307 2014-08-12 17:18:46.561042 2014-08-13 17:38:02.668658 7fe9bae887a0 20 read_log 194215'446 (0'0) modify fbe3fb0f/benchmark_data_storage0111-ib.pocstor_64498_object26769/head//22 by client.1049635.0:26770 2014-08-12 17:18:48.291820 2014-08-13 17:38:02.668680 7fe9bae887a0 20 read_log 194215'447 (0'0) modify 79d5030f/benchmark_data_storage0111-ib.pocstor_64498_object27485/head//22 by client.1049635.0:27486 2014-08-12 17:18:50.967995 2014-08-13 17:38:02.668701 7fe9bae887a0 20 read_log 194215'448 (0'0) modify c54c4b0f/benchmark_data_storage0111-ib.pocstor_64498_object29235/head//22 by client.1049635.0:29236 2014-08-12 17:18:57.468862 2014-08-13 17:38:02.668722 7fe9bae887a0 20 read_log 194215'449 (0'0) modify b20d530f/benchmark_data_storage0111-ib.pocstor_64498_object30511/head//22 by client.1049635.0:30512 2014-08-12 17:19:02.541578 2014-08-13 17:38:02.668743 7fe9bae887a0 20 read_log 194215'450 (0'0) modify 38837b0f/benchmark_data_storage0111-ib.pocstor_64498_object30555/head//22 by client.1049635.0:30556 2014-08-12 17:19:02.708705 2014-08-13 17:38:02.668764 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 64ba1b0f/benchmark_data_storage0111-ib.pocstor_64498_object31114/head//22 by client.1049635.0:31115 2014-08-12 17:19:05.003841 2014-08-13 17:38:02.668785 7fe9bae887a0 20 read_log 194215'452 (0'0) modify 81b69b0f/benchmark_data_storage0111-ib.pocstor_64498_object32132/head//22 by client.1049635.0:32133 2014-08-12 17:19:09.076002 2014-08-13 17:38:02.668807 7fe9bae887a0 20 read_log 194215'453 (194215'432) delete 8dbbab0f/benchmark_data_storage0111-ib.pocstor_64498_object422/head//22 by client.1049635.0:34625 2014-08-12 17:19:18.135736 2014-08-13 17:38:02.668828 7fe9bae887a0 20 read_log 194215'454 (194215'433) delete 9bd41b0f/benchmark_data_storage0111-ib.pocstor_64498_object2589/head//22 by client.1049635.0:36792 2014-08-12 17:19:18.827465 2014-08-13 17:38:02.668851 7fe9bae887a0 20 read_log 194215'455 (194215'434) delete 827e730f/benchmark_data_storage0111-ib.pocstor_64498_object2931/head//22 by client.1049635.0:37134 2014-08-12 17:19:18.947392 2014-08-13 17:38:02.668872 7fe9bae887a0 20 read_log 194215'456 (194215'435) delete 8e27830f/benchmark_data_storage0111-ib.pocstor_64498_object2978/head//22 by client.1049635.0:37181 2014-08-12 17:19:18.962142 2014-08-13 17:38:02.668893 7fe9bae887a0 20 read_log 194215'457 (194215'436) delete 546f6b0f/benchmark_data_storage0111-ib.pocstor_64498_object3949/head//22 by client.1049635.0:38152 2014-08-12 17:19:19.259394 2014-08-13 17:38:02.668915 7fe9bae887a0 20 read_log 194215'458 (194215'437) delete 3f5d730f/benchmark_data_storage0111-ib.pocstor_64498_object6863/head//22 by client.1049635.0:41066 2014-08-12 17:19:20.240054 2014-08-13 17:38:02.668936 7fe9bae887a0 20 read_log 194215'459 (194215'438) delete 6d99130f/benchmark_data_storage0111-ib.pocstor_64498_object6897/head//22 by client.1049635.0:41100 2014-08-12 17:19:20.250815 2014-08-13 17:38:02.668958 7fe9bae887a0 20 read_log 194215'460 (194215'439) delete e5f43b0f/benchmark_data_storage0111-ib.pocstor_64498_object9706/head//22 by client.1049635.0:43909 2014-08-12 17:19:21.156697 2014-08-13 17:38:02.668979 7fe9bae887a0 20 read_log 194215'461 (194215'440) delete 92df230f/benchmark_data_storage0111-ib.pocstor_64498_object10963/head//22 by client.1049635.0:45166 2014-08-12 17:19:21.567271 2014-08-13 17:38:02.669000 7fe9bae887a0 20 read_log 194215'462 (194215'441) delete e0a1630f/benchmark_data_storage0111-ib.pocstor_64498_object12246/head//22 by client.1049635.0:46449 2014-08-12 17:19:21.973284 2014-08-13 17:38:02.669035 7fe9bae887a0 20 read_log 194215'463 (194215'442) delete 561af30f/benchmark_data_storage0111-ib.pocstor_64498_object20013/head//22 by client.1049635.0:54216 2014-08-12 17:19:24.518900 2014-08-13 17:38:02.669057 7fe9bae887a0 20 read_log 194215'464 (194215'443) delete 6012730f/benchmark_data_storage0111-ib.pocstor_64498_object21158/head//22 by client.1049635.0:55361 2014-08-12 17:19:24.941039 2014-08-13 17:38:02.669078 7fe9bae887a0 20 read_log 194215'465 (194215'444) delete c09930f/benchmark_data_storage0111-ib.pocstor_64498_object22881/head//22 by client.1049635.0:57084 2014-08-12 17:19:25.569026 2014-08-13 17:38:02.669100 7fe9bae887a0 20 read_log 194215'466 (194215'445) delete e433430f/benchmark_data_storage0111-ib.pocstor_64498_object26306/head//22 by client.1049635.0:60509 2014-08-12 17:19:26.761825 2014-08-13 17:38:02.669121 7fe9bae887a0 20 read_log 194215'467 (194215'446) delete fbe3fb0f/benchmark_data_storage0111-ib.pocstor_64498_object26769/head//22 by client.1049635.0:60972 2014-08-12 17:19:26.914422 2014-08-13 17:38:02.669142 7fe9bae887a0 20 read_log 194215'468 (194215'447) delete 79d5030f/benchmark_data_storage0111-ib.pocstor_64498_object27485/head//22 by client.1049635.0:61688 2014-08-12 17:19:27.175483 2014-08-13 17:38:02.669172 7fe9bae887a0 20 read_log 194215'469 (194215'448) delete c54c4b0f/benchmark_data_storage0111-ib.pocstor_64498_object29235/head//22 by client.1049635.0:63438 2014-08-12 17:19:27.787522 2014-08-13 17:38:02.669194 7fe9bae887a0 20 read_log 194215'470 (194215'449) delete b20d530f/benchmark_data_storage0111-ib.pocstor_64498_object30511/head//22 by client.1049635.0:64714 2014-08-12 17:19:28.212271 2014-08-13 17:38:02.669215 7fe9bae887a0 20 read_log 194215'471 (194215'450) delete 38837b0f/benchmark_data_storage0111-ib.pocstor_64498_object30555/head//22 by client.1049635.0:64758 2014-08-12 17:19:28.227520 2014-08-13 17:38:02.669237 7fe9bae887a0 20 read_log 194215'472 (194215'451) delete 64ba1b0f/benchmark_data_storage0111-ib.pocstor_64498_object31114/head//22 by client.1049635.0:65317 2014-08-12 17:19:28.418275 2014-08-13 17:38:02.669257 7fe9bae887a0 20 read_log 194215'473 (194215'452) delete 81b69b0f/benchmark_data_storage0111-ib.pocstor_64498_object32132/head//22 by client.1049635.0:66335 2014-08-12 17:19:28.794563 2014-08-13 17:38:02.669391 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.669422 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=0 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.669448 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=0 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Initial 0.002705 0 0.000000 2014-08-13 17:38:02.669468 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=0 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.669482 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=0 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.669497 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] log((0'0,194215'473], crt=192888'431) 2014-08-13 17:38:02.669513 7fe9bae887a0 10 osd.4 194478 pgid 22.362 coll 22.362_head 2014-08-13 17:38:02.669723 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.362 2014-08-13 17:38:02.669740 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.669786 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.362(unlocked)] enter Initial 2014-08-13 17:38:02.669805 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.362(unlocked)] enter NotTrimming 2014-08-13 17:38:02.670025 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.670257 7fe9bae887a0 20 read_log 191648'4 (0'0) modify 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 by client.273158.0:281 2014-07-30 14:08:49.759341 2014-08-13 17:38:02.670294 7fe9bae887a0 20 read_log 191648'80 (0'0) modify 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 by client.273158.0:5691 2014-07-30 14:09:14.204942 2014-08-13 17:38:02.670317 7fe9bae887a0 20 read_log 191648'82 (0'0) modify 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 by client.273158.0:5737 2014-07-30 14:09:14.390026 2014-08-13 17:38:02.670367 7fe9bae887a0 20 read_log 191648'135 (0'0) modify 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 by client.273158.0:9883 2014-07-30 14:09:35.219906 2014-08-13 17:38:02.670394 7fe9bae887a0 20 read_log 191648'162 (0'0) modify 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 by client.273158.0:11469 2014-07-30 14:09:44.540936 2014-08-13 17:38:02.670416 7fe9bae887a0 20 read_log 191648'198 (0'0) modify 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 by client.273158.0:13764 2014-07-30 14:09:55.896649 2014-08-13 17:38:02.670437 7fe9bae887a0 20 read_log 191648'224 (0'0) modify 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 by client.273158.0:16213 2014-07-30 14:10:08.572385 2014-08-13 17:38:02.670458 7fe9bae887a0 20 read_log 191648'242 (0'0) modify cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 by client.273158.0:17384 2014-07-30 14:10:15.207746 2014-08-13 17:38:02.670479 7fe9bae887a0 20 read_log 191648'273 (0'0) modify 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 by client.273158.0:19090 2014-07-30 14:10:23.975481 2014-08-13 17:38:02.670500 7fe9bae887a0 20 read_log 191648'290 (0'0) modify 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 by client.273158.0:20003 2014-07-30 14:10:31.307318 2014-08-13 17:38:02.670534 7fe9bae887a0 20 read_log 191727'347 (0'0) modify 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 by client.275175.0:6281 2014-07-30 14:18:54.097670 2014-08-13 17:38:02.670585 7fe9bae887a0 20 read_log 191727'358 (0'0) modify 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 by client.275175.0:10814 2014-07-30 14:19:07.520159 2014-08-13 17:38:02.670613 7fe9bae887a0 20 read_log 191727'360 (0'0) modify 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 by client.275175.0:11507 2014-07-30 14:19:09.737911 2014-08-13 17:38:02.670634 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 by client.275175.0:12519 2014-07-30 14:19:13.090565 2014-08-13 17:38:02.670656 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 by client.275175.0:15582 2014-07-30 14:19:22.645697 2014-08-13 17:38:02.670676 7fe9bae887a0 20 read_log 191727'370 (0'0) modify 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 by client.275175.0:17541 2014-07-30 14:19:28.659125 2014-08-13 17:38:02.670698 7fe9bae887a0 20 read_log 191727'390 (0'0) modify bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 by client.275175.0:27026 2014-07-30 14:19:57.884253 2014-08-13 17:38:02.670718 7fe9bae887a0 20 read_log 191727'394 (0'0) modify c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 by client.275175.0:29144 2014-07-30 14:20:04.349500 2014-08-13 17:38:02.670739 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 by client.275175.0:29212 2014-07-30 14:20:04.571099 2014-08-13 17:38:02.670760 7fe9bae887a0 20 read_log 191727'397 (0'0) modify b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 by client.275175.0:29912 2014-07-30 14:20:06.697728 2014-08-13 17:38:02.670781 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 by client.275175.0:29970 2014-07-30 14:20:06.859872 2014-08-13 17:38:02.670803 7fe9bae887a0 20 read_log 191727'411 (0'0) modify a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 by client.275175.0:37515 2014-07-30 14:20:29.889612 2014-08-13 17:38:02.670824 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 by client.275175.0:38715 2014-07-30 14:20:33.280192 2014-08-13 17:38:02.670846 7fe9bae887a0 20 read_log 192888'416 (0'0) modify bfeb4b62/benchmark_data_storage0111-ib.pocstor_35625_object214/head//22 by client.940330.0:215 2014-08-11 16:29:41.514064 2014-08-13 17:38:02.670867 7fe9bae887a0 20 read_log 192888'417 (0'0) modify 3a11cb62/benchmark_data_storage0111-ib.pocstor_35625_object256/head//22 by client.940330.0:257 2014-08-11 16:29:41.744578 2014-08-13 17:38:02.670888 7fe9bae887a0 20 read_log 192888'418 (0'0) modify 9a2d3b62/benchmark_data_storage0111-ib.pocstor_35625_object7122/head//22 by client.940330.0:7123 2014-08-11 16:30:07.750277 2014-08-13 17:38:02.670911 7fe9bae887a0 20 read_log 192888'419 (0'0) modify 96e2cb62/benchmark_data_storage0111-ib.pocstor_35625_object7908/head//22 by client.940330.0:7909 2014-08-11 16:30:10.514287 2014-08-13 17:38:02.670932 7fe9bae887a0 20 read_log 192888'420 (0'0) modify 12f7b362/benchmark_data_storage0111-ib.pocstor_35625_object8754/head//22 by client.940330.0:8755 2014-08-11 16:30:13.473690 2014-08-13 17:38:02.670953 7fe9bae887a0 20 read_log 192888'421 (0'0) modify e572b62/benchmark_data_storage0111-ib.pocstor_35625_object10737/head//22 by client.940330.0:10738 2014-08-11 16:30:20.779742 2014-08-13 17:38:02.670991 7fe9bae887a0 20 read_log 192888'422 (0'0) modify d7500b62/benchmark_data_storage0111-ib.pocstor_35625_object11247/head//22 by client.940330.0:11248 2014-08-11 16:30:22.767286 2014-08-13 17:38:02.671013 7fe9bae887a0 20 read_log 192888'423 (0'0) modify 9600b362/benchmark_data_storage0111-ib.pocstor_35625_object11628/head//22 by client.940330.0:11629 2014-08-11 16:30:24.275568 2014-08-13 17:38:02.671034 7fe9bae887a0 20 read_log 192888'424 (0'0) modify ffb48362/benchmark_data_storage0111-ib.pocstor_35625_object13210/head//22 by client.940330.0:13211 2014-08-11 16:30:30.701867 2014-08-13 17:38:02.671055 7fe9bae887a0 20 read_log 192888'425 (0'0) modify f5d1e362/benchmark_data_storage0111-ib.pocstor_35625_object15540/head//22 by client.940330.0:15541 2014-08-11 16:30:39.506202 2014-08-13 17:38:02.671076 7fe9bae887a0 20 read_log 192888'426 (0'0) modify dc951362/benchmark_data_storage0111-ib.pocstor_35625_object17520/head//22 by client.940330.0:17521 2014-08-11 16:30:46.902659 2014-08-13 17:38:02.671097 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 5779f362/benchmark_data_storage0111-ib.pocstor_35625_object22069/head//22 by client.940330.0:22070 2014-08-11 16:31:04.828566 2014-08-13 17:38:02.671118 7fe9bae887a0 20 read_log 192888'428 (0'0) modify e1296b62/benchmark_data_storage0111-ib.pocstor_35625_object24856/head//22 by client.940330.0:24857 2014-08-11 16:31:15.313885 2014-08-13 17:38:02.671144 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 7a051b62/benchmark_data_storage0111-ib.pocstor_35625_object29317/head//22 by client.940330.0:29318 2014-08-11 16:31:32.875685 2014-08-13 17:38:02.671175 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 6294362/benchmark_data_storage0111-ib.pocstor_35625_object29614/head//22 by client.940330.0:29615 2014-08-11 16:31:34.002687 2014-08-13 17:38:02.671234 7fe9bae887a0 20 read_log 192888'431 (0'0) modify c6045b62/benchmark_data_storage0111-ib.pocstor_35625_object30231/head//22 by client.940330.0:30232 2014-08-11 16:31:36.332403 2014-08-13 17:38:02.671256 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 91906362/benchmark_data_storage0111-ib.pocstor_35625_object30275/head//22 by client.940330.0:30276 2014-08-11 16:31:36.500706 2014-08-13 17:38:02.671277 7fe9bae887a0 20 read_log 192888'433 (192888'416) delete bfeb4b62/benchmark_data_storage0111-ib.pocstor_35625_object214/head//22 by client.940330.0:31522 2014-08-11 16:31:41.528597 2014-08-13 17:38:02.671298 7fe9bae887a0 20 read_log 192888'434 (192888'417) delete 3a11cb62/benchmark_data_storage0111-ib.pocstor_35625_object256/head//22 by client.940330.0:31564 2014-08-11 16:31:41.547229 2014-08-13 17:38:02.671320 7fe9bae887a0 20 read_log 192888'435 (192888'418) delete 9a2d3b62/benchmark_data_storage0111-ib.pocstor_35625_object7122/head//22 by client.940330.0:38430 2014-08-11 16:31:43.365382 2014-08-13 17:38:02.671341 7fe9bae887a0 20 read_log 192888'436 (192888'419) delete 96e2cb62/benchmark_data_storage0111-ib.pocstor_35625_object7908/head//22 by client.940330.0:39216 2014-08-11 16:31:43.566807 2014-08-13 17:38:02.671362 7fe9bae887a0 20 read_log 192888'437 (192888'420) delete 12f7b362/benchmark_data_storage0111-ib.pocstor_35625_object8754/head//22 by client.940330.0:40062 2014-08-11 16:31:43.778644 2014-08-13 17:38:02.671384 7fe9bae887a0 20 read_log 192888'438 (192888'421) delete e572b62/benchmark_data_storage0111-ib.pocstor_35625_object10737/head//22 by client.940330.0:42045 2014-08-11 16:31:44.323252 2014-08-13 17:38:02.671410 7fe9bae887a0 20 read_log 192888'439 (192888'422) delete d7500b62/benchmark_data_storage0111-ib.pocstor_35625_object11247/head//22 by client.940330.0:42555 2014-08-11 16:31:44.470958 2014-08-13 17:38:02.671431 7fe9bae887a0 20 read_log 192888'440 (192888'423) delete 9600b362/benchmark_data_storage0111-ib.pocstor_35625_object11628/head//22 by client.940330.0:42936 2014-08-11 16:31:44.568683 2014-08-13 17:38:02.671453 7fe9bae887a0 20 read_log 192888'441 (192888'424) delete ffb48362/benchmark_data_storage0111-ib.pocstor_35625_object13210/head//22 by client.940330.0:44518 2014-08-11 16:31:44.977500 2014-08-13 17:38:02.671517 7fe9bae887a0 20 read_log 192888'442 (192888'425) delete f5d1e362/benchmark_data_storage0111-ib.pocstor_35625_object15540/head//22 by client.940330.0:46848 2014-08-11 16:31:45.601562 2014-08-13 17:38:02.671544 7fe9bae887a0 20 read_log 192888'443 (192888'426) delete dc951362/benchmark_data_storage0111-ib.pocstor_35625_object17520/head//22 by client.940330.0:48828 2014-08-11 16:31:46.133742 2014-08-13 17:38:02.671566 7fe9bae887a0 20 read_log 192888'444 (192888'427) delete 5779f362/benchmark_data_storage0111-ib.pocstor_35625_object22069/head//22 by client.940330.0:53377 2014-08-11 16:31:47.463568 2014-08-13 17:38:02.671587 7fe9bae887a0 20 read_log 192888'445 (192888'428) delete e1296b62/benchmark_data_storage0111-ib.pocstor_35625_object24856/head//22 by client.940330.0:56164 2014-08-11 16:31:48.292418 2014-08-13 17:38:02.671609 7fe9bae887a0 20 read_log 192888'446 (192888'429) delete 7a051b62/benchmark_data_storage0111-ib.pocstor_35625_object29317/head//22 by client.940330.0:60625 2014-08-11 16:31:49.558048 2014-08-13 17:38:02.671630 7fe9bae887a0 20 read_log 192888'447 (192888'430) delete 6294362/benchmark_data_storage0111-ib.pocstor_35625_object29614/head//22 by client.940330.0:60922 2014-08-11 16:31:49.647528 2014-08-13 17:38:02.671651 7fe9bae887a0 20 read_log 192888'448 (192888'431) delete c6045b62/benchmark_data_storage0111-ib.pocstor_35625_object30231/head//22 by client.940330.0:61539 2014-08-11 16:31:49.823001 2014-08-13 17:38:02.671672 7fe9bae887a0 20 read_log 192888'449 (192888'432) delete 91906362/benchmark_data_storage0111-ib.pocstor_35625_object30275/head//22 by client.940330.0:61583 2014-08-11 16:31:49.837947 2014-08-13 17:38:02.671694 7fe9bae887a0 20 read_log 194215'450 (0'0) modify a6c07362/benchmark_data_storage0111-ib.pocstor_64498_object7055/head//22 by client.1049635.0:7056 2014-08-12 17:17:39.287541 2014-08-13 17:38:02.671715 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 2a611362/benchmark_data_storage0111-ib.pocstor_64498_object9383/head//22 by client.1049635.0:9384 2014-08-12 17:17:46.137210 2014-08-13 17:38:02.671736 7fe9bae887a0 20 read_log 194215'452 (0'0) modify 7f7d2362/benchmark_data_storage0111-ib.pocstor_64498_object10645/head//22 by client.1049635.0:10646 2014-08-12 17:17:50.063541 2014-08-13 17:38:02.671758 7fe9bae887a0 20 read_log 194215'453 (0'0) modify 9d377362/benchmark_data_storage0111-ib.pocstor_64498_object10960/head//22 by client.1049635.0:10961 2014-08-12 17:17:50.979302 2014-08-13 17:38:02.671779 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 446fb62/benchmark_data_storage0111-ib.pocstor_64498_object12968/head//22 by client.1049635.0:12969 2014-08-12 17:17:57.264085 2014-08-13 17:38:02.671800 7fe9bae887a0 20 read_log 194215'455 (0'0) modify 69232b62/benchmark_data_storage0111-ib.pocstor_64498_object13029/head//22 by client.1049635.0:13030 2014-08-12 17:17:57.479853 2014-08-13 17:38:02.671821 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 8dbd8362/benchmark_data_storage0111-ib.pocstor_64498_object13953/head//22 by client.1049635.0:13954 2014-08-12 17:18:00.383857 2014-08-13 17:38:02.671842 7fe9bae887a0 20 read_log 194215'457 (0'0) modify b6528b62/benchmark_data_storage0111-ib.pocstor_64498_object14620/head//22 by client.1049635.0:14621 2014-08-12 17:18:02.676161 2014-08-13 17:38:02.671863 7fe9bae887a0 20 read_log 194215'458 (0'0) modify 6d78c362/benchmark_data_storage0111-ib.pocstor_64498_object17345/head//22 by client.1049635.0:17346 2014-08-12 17:18:12.683975 2014-08-13 17:38:02.671884 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 8c33c362/benchmark_data_storage0111-ib.pocstor_64498_object17682/head//22 by client.1049635.0:17683 2014-08-12 17:18:13.953180 2014-08-13 17:38:02.671906 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 6b28e362/benchmark_data_storage0111-ib.pocstor_64498_object19659/head//22 by client.1049635.0:19660 2014-08-12 17:18:21.504723 2014-08-13 17:38:02.671939 7fe9bae887a0 20 read_log 194215'461 (0'0) modify c0644b62/benchmark_data_storage0111-ib.pocstor_64498_object25743/head//22 by client.1049635.0:25744 2014-08-12 17:18:44.469757 2014-08-13 17:38:02.671962 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 8023e362/benchmark_data_storage0111-ib.pocstor_64498_object26771/head//22 by client.1049635.0:26772 2014-08-12 17:18:48.299105 2014-08-13 17:38:02.671983 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 343362/benchmark_data_storage0111-ib.pocstor_64498_object29465/head//22 by client.1049635.0:29466 2014-08-12 17:18:58.436180 2014-08-13 17:38:02.672004 7fe9bae887a0 20 read_log 194215'464 (0'0) modify e0debb62/benchmark_data_storage0111-ib.pocstor_64498_object32182/head//22 by client.1049635.0:32183 2014-08-12 17:19:09.269924 2014-08-13 17:38:02.672026 7fe9bae887a0 20 read_log 194215'465 (194215'450) delete a6c07362/benchmark_data_storage0111-ib.pocstor_64498_object7055/head//22 by client.1049635.0:41258 2014-08-12 17:19:20.298677 2014-08-13 17:38:02.672047 7fe9bae887a0 20 read_log 194215'466 (194215'451) delete 2a611362/benchmark_data_storage0111-ib.pocstor_64498_object9383/head//22 by client.1049635.0:43586 2014-08-12 17:19:21.054960 2014-08-13 17:38:02.672069 7fe9bae887a0 20 read_log 194215'467 (194215'452) delete 7f7d2362/benchmark_data_storage0111-ib.pocstor_64498_object10645/head//22 by client.1049635.0:44848 2014-08-12 17:19:21.464491 2014-08-13 17:38:02.672090 7fe9bae887a0 20 read_log 194215'468 (194215'453) delete 9d377362/benchmark_data_storage0111-ib.pocstor_64498_object10960/head//22 by client.1049635.0:45163 2014-08-12 17:19:21.565895 2014-08-13 17:38:02.672111 7fe9bae887a0 20 read_log 194215'469 (194215'454) delete 446fb62/benchmark_data_storage0111-ib.pocstor_64498_object12968/head//22 by client.1049635.0:47171 2014-08-12 17:19:22.202596 2014-08-13 17:38:02.672133 7fe9bae887a0 20 read_log 194215'470 (194215'455) delete 69232b62/benchmark_data_storage0111-ib.pocstor_64498_object13029/head//22 by client.1049635.0:47232 2014-08-12 17:19:22.227935 2014-08-13 17:38:02.672162 7fe9bae887a0 20 read_log 194215'471 (194215'456) delete 8dbd8362/benchmark_data_storage0111-ib.pocstor_64498_object13953/head//22 by client.1049635.0:48156 2014-08-12 17:19:22.548643 2014-08-13 17:38:02.672185 7fe9bae887a0 20 read_log 194215'472 (194215'457) delete b6528b62/benchmark_data_storage0111-ib.pocstor_64498_object14620/head//22 by client.1049635.0:48823 2014-08-12 17:19:22.760422 2014-08-13 17:38:02.672206 7fe9bae887a0 20 read_log 194215'473 (194215'458) delete 6d78c362/benchmark_data_storage0111-ib.pocstor_64498_object17345/head//22 by client.1049635.0:51548 2014-08-12 17:19:23.616644 2014-08-13 17:38:02.672228 7fe9bae887a0 20 read_log 194215'474 (194215'459) delete 8c33c362/benchmark_data_storage0111-ib.pocstor_64498_object17682/head//22 by client.1049635.0:51885 2014-08-12 17:19:23.721755 2014-08-13 17:38:02.672249 7fe9bae887a0 20 read_log 194215'475 (194215'460) delete 6b28e362/benchmark_data_storage0111-ib.pocstor_64498_object19659/head//22 by client.1049635.0:53862 2014-08-12 17:19:24.403353 2014-08-13 17:38:02.672270 7fe9bae887a0 20 read_log 194215'476 (194215'461) delete c0644b62/benchmark_data_storage0111-ib.pocstor_64498_object25743/head//22 by client.1049635.0:59946 2014-08-12 17:19:26.572396 2014-08-13 17:38:02.672293 7fe9bae887a0 20 read_log 194215'477 (194215'462) delete 8023e362/benchmark_data_storage0111-ib.pocstor_64498_object26771/head//22 by client.1049635.0:60974 2014-08-12 17:19:26.914512 2014-08-13 17:38:02.672314 7fe9bae887a0 20 read_log 194215'478 (194215'463) delete 343362/benchmark_data_storage0111-ib.pocstor_64498_object29465/head//22 by client.1049635.0:63668 2014-08-12 17:19:27.874509 2014-08-13 17:38:02.672336 7fe9bae887a0 20 read_log 194215'479 (194215'464) delete e0debb62/benchmark_data_storage0111-ib.pocstor_64498_object32182/head//22 by client.1049635.0:66385 2014-08-12 17:19:28.811195 2014-08-13 17:38:02.672570 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.672608 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=0 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.672636 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=0 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002851 0 0.000000 2014-08-13 17:38:02.672657 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=0 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.672671 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=0 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.672687 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'479], crt=191727'415) 2014-08-13 17:38:02.672704 7fe9bae887a0 10 osd.4 194478 pgid 22.39d coll 22.39d_head 2014-08-13 17:38:02.672893 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.39d 2014-08-13 17:38:02.672909 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.672969 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.39d(unlocked)] enter Initial 2014-08-13 17:38:02.672989 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.39d(unlocked)] enter NotTrimming 2014-08-13 17:38:02.673217 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.673442 7fe9bae887a0 20 read_log 191648'26 (0'0) modify e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 by client.273158.0:1299 2014-07-30 14:08:53.563044 2014-08-13 17:38:02.673481 7fe9bae887a0 20 read_log 191648'122 (0'0) modify 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 by client.273158.0:7733 2014-07-30 14:09:24.642346 2014-08-13 17:38:02.673504 7fe9bae887a0 20 read_log 191648'139 (0'0) modify d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 by client.273158.0:8772 2014-07-30 14:09:30.270496 2014-08-13 17:38:02.673526 7fe9bae887a0 20 read_log 191648'196 (0'0) modify 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 by client.273158.0:12774 2014-07-30 14:09:51.305241 2014-08-13 17:38:02.673547 7fe9bae887a0 20 read_log 191648'231 (0'0) modify bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 by client.273158.0:15130 2014-07-30 14:10:03.184496 2014-08-13 17:38:02.673568 7fe9bae887a0 20 read_log 191648'238 (0'0) modify 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 by client.273158.0:15587 2014-07-30 14:10:05.248338 2014-08-13 17:38:02.673622 7fe9bae887a0 20 read_log 191727'348 (0'0) modify d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 by client.275175.0:460 2014-07-30 14:18:35.295944 2014-08-13 17:38:02.673666 7fe9bae887a0 20 read_log 191727'354 (0'0) modify 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 by client.275175.0:4105 2014-07-30 14:18:47.676496 2014-08-13 17:38:02.673693 7fe9bae887a0 20 read_log 191727'359 (0'0) modify 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 by client.275175.0:7849 2014-07-30 14:18:59.101798 2014-08-13 17:38:02.673715 7fe9bae887a0 20 read_log 191727'362 (0'0) modify e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 by client.275175.0:9837 2014-07-30 14:19:04.722733 2014-08-13 17:38:02.673736 7fe9bae887a0 20 read_log 191727'363 (0'0) modify 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 by client.275175.0:10098 2014-07-30 14:19:05.451862 2014-08-13 17:38:02.673757 7fe9bae887a0 20 read_log 191727'365 (0'0) modify e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 by client.275175.0:12885 2014-07-30 14:19:14.144831 2014-08-13 17:38:02.673778 7fe9bae887a0 20 read_log 191727'368 (0'0) modify e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 by client.275175.0:13708 2014-07-30 14:19:16.827656 2014-08-13 17:38:02.673799 7fe9bae887a0 20 read_log 191727'374 (0'0) modify ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 by client.275175.0:16549 2014-07-30 14:19:25.843951 2014-08-13 17:38:02.673820 7fe9bae887a0 20 read_log 191727'383 (0'0) modify 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 by client.275175.0:22486 2014-07-30 14:19:44.167235 2014-08-13 17:38:02.673841 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 by client.275175.0:22601 2014-07-30 14:19:44.678490 2014-08-13 17:38:02.673862 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 by client.275175.0:24145 2014-07-30 14:19:49.236554 2014-08-13 17:38:02.673885 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 by client.275175.0:24634 2014-07-30 14:19:50.611965 2014-08-13 17:38:02.673906 7fe9bae887a0 20 read_log 191727'390 (0'0) modify 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 by client.275175.0:25668 2014-07-30 14:19:53.722499 2014-08-13 17:38:02.673927 7fe9bae887a0 20 read_log 191727'393 (0'0) modify e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 by client.275175.0:26106 2014-07-30 14:19:55.164355 2014-08-13 17:38:02.673949 7fe9bae887a0 20 read_log 191727'398 (0'0) modify d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 by client.275175.0:28266 2014-07-30 14:20:01.770201 2014-08-13 17:38:02.673971 7fe9bae887a0 20 read_log 191727'403 (0'0) modify e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 by client.275175.0:31330 2014-07-30 14:20:10.912444 2014-08-13 17:38:02.673991 7fe9bae887a0 20 read_log 191727'411 (0'0) modify aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 by client.275175.0:35464 2014-07-30 14:20:23.339331 2014-08-13 17:38:02.674012 7fe9bae887a0 20 read_log 192888'417 (0'0) modify aff7939d/benchmark_data_storage0111-ib.pocstor_35625_object1287/head//22 by client.940330.0:1288 2014-08-11 16:29:45.861815 2014-08-13 17:38:02.674036 7fe9bae887a0 20 read_log 192888'418 (0'0) modify 3b158b9d/benchmark_data_storage0111-ib.pocstor_35625_object1448/head//22 by client.940330.0:1449 2014-08-11 16:29:46.505073 2014-08-13 17:38:02.674057 7fe9bae887a0 20 read_log 192888'419 (0'0) modify 96f7e39d/benchmark_data_storage0111-ib.pocstor_35625_object2238/head//22 by client.940330.0:2239 2014-08-11 16:29:49.420246 2014-08-13 17:38:02.674078 7fe9bae887a0 20 read_log 192888'420 (0'0) modify f730c39d/benchmark_data_storage0111-ib.pocstor_35625_object5880/head//22 by client.940330.0:5881 2014-08-11 16:30:03.137399 2014-08-13 17:38:02.674112 7fe9bae887a0 20 read_log 192888'421 (0'0) modify a5e58b9d/benchmark_data_storage0111-ib.pocstor_35625_object9479/head//22 by client.940330.0:9480 2014-08-11 16:30:16.008439 2014-08-13 17:38:02.674134 7fe9bae887a0 20 read_log 192888'422 (0'0) modify 47e94b9d/benchmark_data_storage0111-ib.pocstor_35625_object9885/head//22 by client.940330.0:9886 2014-08-11 16:30:17.428162 2014-08-13 17:38:02.674166 7fe9bae887a0 20 read_log 192888'423 (0'0) modify dba47b9d/benchmark_data_storage0111-ib.pocstor_35625_object10685/head//22 by client.940330.0:10686 2014-08-11 16:30:20.576240 2014-08-13 17:38:02.674189 7fe9bae887a0 20 read_log 192888'424 (0'0) modify 49060b9d/benchmark_data_storage0111-ib.pocstor_35625_object11965/head//22 by client.940330.0:11966 2014-08-11 16:30:25.779919 2014-08-13 17:38:02.674210 7fe9bae887a0 20 read_log 192888'425 (0'0) modify 357a439d/benchmark_data_storage0111-ib.pocstor_35625_object13078/head//22 by client.940330.0:13079 2014-08-11 16:30:30.197668 2014-08-13 17:38:02.674231 7fe9bae887a0 20 read_log 192888'426 (0'0) modify c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object15709/head//22 by client.940330.0:15710 2014-08-11 16:30:40.137252 2014-08-13 17:38:02.674252 7fe9bae887a0 20 read_log 192888'427 (0'0) modify a99c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object16423/head//22 by client.940330.0:16424 2014-08-11 16:30:42.811556 2014-08-13 17:38:02.674273 7fe9bae887a0 20 read_log 192888'428 (0'0) modify a7798b9d/benchmark_data_storage0111-ib.pocstor_35625_object17705/head//22 by client.940330.0:17706 2014-08-11 16:30:47.608735 2014-08-13 17:38:02.674296 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 96cf639d/benchmark_data_storage0111-ib.pocstor_35625_object21027/head//22 by client.940330.0:21028 2014-08-11 16:31:00.916326 2014-08-13 17:38:02.674318 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 9093db9d/benchmark_data_storage0111-ib.pocstor_35625_object22274/head//22 by client.940330.0:22275 2014-08-11 16:31:05.590831 2014-08-13 17:38:02.674340 7fe9bae887a0 20 read_log 192888'431 (0'0) modify a4c0bb9d/benchmark_data_storage0111-ib.pocstor_35625_object26157/head//22 by client.940330.0:26158 2014-08-11 16:31:20.407958 2014-08-13 17:38:02.674360 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 8d647b9d/benchmark_data_storage0111-ib.pocstor_35625_object29398/head//22 by client.940330.0:29399 2014-08-11 16:31:33.185068 2014-08-13 17:38:02.674381 7fe9bae887a0 20 read_log 192888'433 (0'0) modify b5d4fb9d/benchmark_data_storage0111-ib.pocstor_35625_object30589/head//22 by client.940330.0:30590 2014-08-11 16:31:37.682976 2014-08-13 17:38:02.674402 7fe9bae887a0 20 read_log 192888'434 (192888'417) delete aff7939d/benchmark_data_storage0111-ib.pocstor_35625_object1287/head//22 by client.940330.0:32595 2014-08-11 16:31:41.817054 2014-08-13 17:38:02.674423 7fe9bae887a0 20 read_log 192888'435 (192888'418) delete 3b158b9d/benchmark_data_storage0111-ib.pocstor_35625_object1448/head//22 by client.940330.0:32756 2014-08-11 16:31:41.855371 2014-08-13 17:38:02.674445 7fe9bae887a0 20 read_log 192888'436 (192888'419) delete 96f7e39d/benchmark_data_storage0111-ib.pocstor_35625_object2238/head//22 by client.940330.0:33546 2014-08-11 16:31:42.052572 2014-08-13 17:38:02.674466 7fe9bae887a0 20 read_log 192888'437 (192888'420) delete f730c39d/benchmark_data_storage0111-ib.pocstor_35625_object5880/head//22 by client.940330.0:37188 2014-08-11 16:31:43.043874 2014-08-13 17:38:02.674488 7fe9bae887a0 20 read_log 192888'438 (192888'421) delete a5e58b9d/benchmark_data_storage0111-ib.pocstor_35625_object9479/head//22 by client.940330.0:40787 2014-08-11 16:31:43.969923 2014-08-13 17:38:02.674509 7fe9bae887a0 20 read_log 192888'439 (192888'422) delete 47e94b9d/benchmark_data_storage0111-ib.pocstor_35625_object9885/head//22 by client.940330.0:41193 2014-08-11 16:31:44.082557 2014-08-13 17:38:02.674530 7fe9bae887a0 20 read_log 192888'440 (192888'423) delete dba47b9d/benchmark_data_storage0111-ib.pocstor_35625_object10685/head//22 by client.940330.0:41993 2014-08-11 16:31:44.305708 2014-08-13 17:38:02.674564 7fe9bae887a0 20 read_log 192888'441 (192888'424) delete 49060b9d/benchmark_data_storage0111-ib.pocstor_35625_object11965/head//22 by client.940330.0:43273 2014-08-11 16:31:44.648388 2014-08-13 17:38:02.674594 7fe9bae887a0 20 read_log 192888'442 (192888'425) delete 357a439d/benchmark_data_storage0111-ib.pocstor_35625_object13078/head//22 by client.940330.0:44386 2014-08-11 16:31:44.940655 2014-08-13 17:38:02.674616 7fe9bae887a0 20 read_log 192888'443 (192888'426) delete c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object15709/head//22 by client.940330.0:47017 2014-08-11 16:31:45.646283 2014-08-13 17:38:02.674638 7fe9bae887a0 20 read_log 192888'444 (192888'427) delete a99c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object16423/head//22 by client.940330.0:47731 2014-08-11 16:31:45.834947 2014-08-13 17:38:02.674659 7fe9bae887a0 20 read_log 192888'445 (192888'428) delete a7798b9d/benchmark_data_storage0111-ib.pocstor_35625_object17705/head//22 by client.940330.0:49013 2014-08-11 16:31:46.193497 2014-08-13 17:38:02.674681 7fe9bae887a0 20 read_log 192888'446 (192888'429) delete 96cf639d/benchmark_data_storage0111-ib.pocstor_35625_object21027/head//22 by client.940330.0:52335 2014-08-11 16:31:47.168840 2014-08-13 17:38:02.674702 7fe9bae887a0 20 read_log 192888'447 (192888'430) delete 9093db9d/benchmark_data_storage0111-ib.pocstor_35625_object22274/head//22 by client.940330.0:53582 2014-08-11 16:31:47.534706 2014-08-13 17:38:02.674723 7fe9bae887a0 20 read_log 192888'448 (192888'431) delete a4c0bb9d/benchmark_data_storage0111-ib.pocstor_35625_object26157/head//22 by client.940330.0:57465 2014-08-11 16:31:48.690196 2014-08-13 17:38:02.674744 7fe9bae887a0 20 read_log 192888'449 (192888'432) delete 8d647b9d/benchmark_data_storage0111-ib.pocstor_35625_object29398/head//22 by client.940330.0:60706 2014-08-11 16:31:49.584116 2014-08-13 17:38:02.674765 7fe9bae887a0 20 read_log 192888'450 (192888'433) delete b5d4fb9d/benchmark_data_storage0111-ib.pocstor_35625_object30589/head//22 by client.940330.0:61897 2014-08-11 16:31:49.926372 2014-08-13 17:38:02.674823 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 9aa9a39d/benchmark_data_storage0111-ib.pocstor_64498_object1198/head//22 by client.1049635.0:1199 2014-08-12 17:17:21.039948 2014-08-13 17:38:02.674846 7fe9bae887a0 20 read_log 194215'452 (0'0) modify ee4f839d/benchmark_data_storage0111-ib.pocstor_64498_object7499/head//22 by client.1049635.0:7500 2014-08-12 17:17:40.535063 2014-08-13 17:38:02.674867 7fe9bae887a0 20 read_log 194215'453 (0'0) modify f1ff839d/benchmark_data_storage0111-ib.pocstor_64498_object8496/head//22 by client.1049635.0:8497 2014-08-12 17:17:43.549867 2014-08-13 17:38:02.674888 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 55c4439d/benchmark_data_storage0111-ib.pocstor_64498_object9283/head//22 by client.1049635.0:9284 2014-08-12 17:17:45.842712 2014-08-13 17:38:02.674909 7fe9bae887a0 20 read_log 194215'455 (0'0) modify 6e112b9d/benchmark_data_storage0111-ib.pocstor_64498_object11955/head//22 by client.1049635.0:11956 2014-08-12 17:17:54.258163 2014-08-13 17:38:02.674931 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 4c1b039d/benchmark_data_storage0111-ib.pocstor_64498_object13203/head//22 by client.1049635.0:13204 2014-08-12 17:17:57.996560 2014-08-13 17:38:02.674952 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 14bedb9d/benchmark_data_storage0111-ib.pocstor_64498_object13760/head//22 by client.1049635.0:13761 2014-08-12 17:17:59.822949 2014-08-13 17:38:02.674974 7fe9bae887a0 20 read_log 194215'458 (0'0) modify 929fd39d/benchmark_data_storage0111-ib.pocstor_64498_object14114/head//22 by client.1049635.0:14115 2014-08-12 17:18:00.860191 2014-08-13 17:38:02.674995 7fe9bae887a0 20 read_log 194215'459 (0'0) modify e1f7139d/benchmark_data_storage0111-ib.pocstor_64498_object14470/head//22 by client.1049635.0:14471 2014-08-12 17:18:02.151050 2014-08-13 17:38:02.675056 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 9530db9d/benchmark_data_storage0111-ib.pocstor_64498_object14696/head//22 by client.1049635.0:14697 2014-08-12 17:18:02.996079 2014-08-13 17:38:02.675085 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 77269b9d/benchmark_data_storage0111-ib.pocstor_64498_object15171/head//22 by client.1049635.0:15172 2014-08-12 17:18:04.534230 2014-08-13 17:38:02.675106 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 729a2b9d/benchmark_data_storage0111-ib.pocstor_64498_object24867/head//22 by client.1049635.0:24868 2014-08-12 17:18:41.158950 2014-08-13 17:38:02.675127 7fe9bae887a0 20 read_log 194215'463 (0'0) modify ed5ae39d/benchmark_data_storage0111-ib.pocstor_64498_object24923/head//22 by client.1049635.0:24924 2014-08-12 17:18:41.419768 2014-08-13 17:38:02.675149 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 185c5b9d/benchmark_data_storage0111-ib.pocstor_64498_object29834/head//22 by client.1049635.0:29835 2014-08-12 17:18:59.828932 2014-08-13 17:38:02.675179 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 713a339d/benchmark_data_storage0111-ib.pocstor_64498_object31111/head//22 by client.1049635.0:31112 2014-08-12 17:19:04.992614 2014-08-13 17:38:02.675201 7fe9bae887a0 20 read_log 194215'466 (194215'451) delete 9aa9a39d/benchmark_data_storage0111-ib.pocstor_64498_object1198/head//22 by client.1049635.0:35401 2014-08-12 17:19:18.383901 2014-08-13 17:38:02.675223 7fe9bae887a0 20 read_log 194215'467 (194215'452) delete ee4f839d/benchmark_data_storage0111-ib.pocstor_64498_object7499/head//22 by client.1049635.0:41702 2014-08-12 17:19:20.448516 2014-08-13 17:38:02.675244 7fe9bae887a0 20 read_log 194215'468 (194215'453) delete f1ff839d/benchmark_data_storage0111-ib.pocstor_64498_object8496/head//22 by client.1049635.0:42699 2014-08-12 17:19:20.758592 2014-08-13 17:38:02.675265 7fe9bae887a0 20 read_log 194215'469 (194215'454) delete 55c4439d/benchmark_data_storage0111-ib.pocstor_64498_object9283/head//22 by client.1049635.0:43486 2014-08-12 17:19:21.018972 2014-08-13 17:38:02.675287 7fe9bae887a0 20 read_log 194215'470 (194215'455) delete 6e112b9d/benchmark_data_storage0111-ib.pocstor_64498_object11955/head//22 by client.1049635.0:46158 2014-08-12 17:19:21.885671 2014-08-13 17:38:02.675308 7fe9bae887a0 20 read_log 194215'471 (194215'456) delete 4c1b039d/benchmark_data_storage0111-ib.pocstor_64498_object13203/head//22 by client.1049635.0:47406 2014-08-12 17:19:22.291853 2014-08-13 17:38:02.675330 7fe9bae887a0 20 read_log 194215'472 (194215'457) delete 14bedb9d/benchmark_data_storage0111-ib.pocstor_64498_object13760/head//22 by client.1049635.0:47963 2014-08-12 17:19:22.492087 2014-08-13 17:38:02.675351 7fe9bae887a0 20 read_log 194215'473 (194215'458) delete 929fd39d/benchmark_data_storage0111-ib.pocstor_64498_object14114/head//22 by client.1049635.0:48317 2014-08-12 17:19:22.597693 2014-08-13 17:38:02.675372 7fe9bae887a0 20 read_log 194215'474 (194215'459) delete e1f7139d/benchmark_data_storage0111-ib.pocstor_64498_object14470/head//22 by client.1049635.0:48673 2014-08-12 17:19:22.711244 2014-08-13 17:38:02.675394 7fe9bae887a0 20 read_log 194215'475 (194215'460) delete 9530db9d/benchmark_data_storage0111-ib.pocstor_64498_object14696/head//22 by client.1049635.0:48899 2014-08-12 17:19:22.783840 2014-08-13 17:38:02.675416 7fe9bae887a0 20 read_log 194215'476 (194215'461) delete 77269b9d/benchmark_data_storage0111-ib.pocstor_64498_object15171/head//22 by client.1049635.0:49374 2014-08-12 17:19:22.926265 2014-08-13 17:38:02.675437 7fe9bae887a0 20 read_log 194215'477 (194215'462) delete 729a2b9d/benchmark_data_storage0111-ib.pocstor_64498_object24867/head//22 by client.1049635.0:59070 2014-08-12 17:19:26.278811 2014-08-13 17:38:02.675458 7fe9bae887a0 20 read_log 194215'478 (194215'463) delete ed5ae39d/benchmark_data_storage0111-ib.pocstor_64498_object24923/head//22 by client.1049635.0:59126 2014-08-12 17:19:26.295113 2014-08-13 17:38:02.675479 7fe9bae887a0 20 read_log 194215'479 (194215'464) delete 185c5b9d/benchmark_data_storage0111-ib.pocstor_64498_object29834/head//22 by client.1049635.0:64037 2014-08-12 17:19:27.996815 2014-08-13 17:38:02.675513 7fe9bae887a0 20 read_log 194215'480 (194215'465) delete 713a339d/benchmark_data_storage0111-ib.pocstor_64498_object31111/head//22 by client.1049635.0:65314 2014-08-12 17:19:28.417085 2014-08-13 17:38:02.675654 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.675687 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=0 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.675714 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=0 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002745 0 0.000000 2014-08-13 17:38:02.675735 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=0 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.675749 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=0 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.675764 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'480], crt=191727'416) 2014-08-13 17:38:02.675781 7fe9bae887a0 10 osd.4 194478 pgid 22.3ad coll 22.3ad_head 2014-08-13 17:38:02.675994 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3ad 2014-08-13 17:38:02.676010 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.676058 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3ad(unlocked)] enter Initial 2014-08-13 17:38:02.676115 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3ad(unlocked)] enter NotTrimming 2014-08-13 17:38:02.676341 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.676538 7fe9bae887a0 20 read_log 191648'9 (0'0) modify f5b70bad/benchmark_data_storage0111-ib.pocstor_40745_object361/head//22 by client.273158.0:362 2014-07-30 14:08:50.141745 2014-08-13 17:38:02.676574 7fe9bae887a0 20 read_log 191648'20 (0'0) modify de67fbad/benchmark_data_storage0111-ib.pocstor_40745_object878/head//22 by client.273158.0:879 2014-07-30 14:08:52.037400 2014-08-13 17:38:02.676597 7fe9bae887a0 20 read_log 191648'55 (0'0) modify 631053ad/benchmark_data_storage0111-ib.pocstor_40745_object2799/head//22 by client.273158.0:2800 2014-07-30 14:09:00.349553 2014-08-13 17:38:02.676619 7fe9bae887a0 20 read_log 191648'72 (0'0) modify 5b437bad/benchmark_data_storage0111-ib.pocstor_40745_object3931/head//22 by client.273158.0:3932 2014-07-30 14:09:05.958864 2014-08-13 17:38:02.676640 7fe9bae887a0 20 read_log 191648'80 (0'0) modify 594613ad/benchmark_data_storage0111-ib.pocstor_40745_object4707/head//22 by client.273158.0:4708 2014-07-30 14:09:09.642401 2014-08-13 17:38:02.676661 7fe9bae887a0 20 read_log 191648'171 (0'0) modify f9557bad/benchmark_data_storage0111-ib.pocstor_40745_object11374/head//22 by client.273158.0:11375 2014-07-30 14:09:43.783183 2014-08-13 17:38:02.676698 7fe9bae887a0 20 read_log 191648'187 (0'0) modify a1240bad/benchmark_data_storage0111-ib.pocstor_40745_object12129/head//22 by client.273158.0:12130 2014-07-30 14:09:48.343980 2014-08-13 17:38:02.676725 7fe9bae887a0 20 read_log 191648'252 (0'0) modify 2c903bad/benchmark_data_storage0111-ib.pocstor_40745_object16338/head//22 by client.273158.0:16339 2014-07-30 14:10:09.295782 2014-08-13 17:38:02.676746 7fe9bae887a0 20 read_log 191648'294 (0'0) modify 3a24cbad/benchmark_data_storage0111-ib.pocstor_40745_object18210/head//22 by client.273158.0:18211 2014-07-30 14:10:19.109377 2014-08-13 17:38:02.676804 7fe9bae887a0 20 read_log 191648'301 (0'0) modify 2e3273ad/benchmark_data_storage0111-ib.pocstor_40745_object18704/head//22 by client.273158.0:18705 2014-07-30 14:10:22.342769 2014-08-13 17:38:02.676830 7fe9bae887a0 20 read_log 191648'313 (0'0) modify 5da043ad/benchmark_data_storage0111-ib.pocstor_40745_object19378/head//22 by client.273158.0:19379 2014-07-30 14:10:25.771154 2014-08-13 17:38:02.676852 7fe9bae887a0 20 read_log 191648'317 (0'0) modify 7658d3ad/benchmark_data_storage0111-ib.pocstor_40745_object19695/head//22 by client.273158.0:19696 2014-07-30 14:10:27.834449 2014-08-13 17:38:02.676872 7fe9bae887a0 20 read_log 191648'343 (0'0) modify 108f6bad/benchmark_data_storage0111-ib.pocstor_40745_object21182/head//22 by client.273158.0:21183 2014-07-30 14:10:38.020512 2014-08-13 17:38:02.676893 7fe9bae887a0 20 read_log 191648'347 (0'0) modify df803ad/benchmark_data_storage0111-ib.pocstor_40745_object21453/head//22 by client.273158.0:21454 2014-07-30 14:10:39.267993 2014-08-13 17:38:02.676915 7fe9bae887a0 20 read_log 191727'374 (0'0) modify 186f6bad/benchmark_data_storage0111-ib.pocstor_51285_object527/head//22 by client.275175.0:528 2014-07-30 14:18:35.580093 2014-08-13 17:38:02.676935 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 367a93ad/benchmark_data_storage0111-ib.pocstor_51285_object5248/head//22 by client.275175.0:5249 2014-07-30 14:18:51.084411 2014-08-13 17:38:02.676956 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 9d6ed3ad/benchmark_data_storage0111-ib.pocstor_51285_object6613/head//22 by client.275175.0:6614 2014-07-30 14:18:55.218980 2014-08-13 17:38:02.676977 7fe9bae887a0 20 read_log 191727'385 (0'0) modify bfe27bad/benchmark_data_storage0111-ib.pocstor_51285_object7152/head//22 by client.275175.0:7153 2014-07-30 14:18:57.152350 2014-08-13 17:38:02.676998 7fe9bae887a0 20 read_log 191727'387 (0'0) modify bd663ad/benchmark_data_storage0111-ib.pocstor_51285_object8531/head//22 by client.275175.0:8532 2014-07-30 14:19:01.016015 2014-08-13 17:38:02.677019 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 480e73ad/benchmark_data_storage0111-ib.pocstor_51285_object8759/head//22 by client.275175.0:8760 2014-07-30 14:19:01.662527 2014-08-13 17:38:02.677040 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 825783ad/benchmark_data_storage0111-ib.pocstor_51285_object12762/head//22 by client.275175.0:12763 2014-07-30 14:19:13.779839 2014-08-13 17:38:02.677060 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 5d358bad/benchmark_data_storage0111-ib.pocstor_51285_object13078/head//22 by client.275175.0:13079 2014-07-30 14:19:14.916472 2014-08-13 17:38:02.677081 7fe9bae887a0 20 read_log 191727'401 (0'0) modify 206cb3ad/benchmark_data_storage0111-ib.pocstor_51285_object14457/head//22 by client.275175.0:14458 2014-07-30 14:19:19.169473 2014-08-13 17:38:02.677102 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 6b955bad/benchmark_data_storage0111-ib.pocstor_51285_object15248/head//22 by client.275175.0:15249 2014-07-30 14:19:21.506584 2014-08-13 17:38:02.677123 7fe9bae887a0 20 read_log 191727'409 (0'0) modify a63efbad/benchmark_data_storage0111-ib.pocstor_51285_object20929/head//22 by client.275175.0:20930 2014-07-30 14:19:38.930325 2014-08-13 17:38:02.677170 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 89403ad/benchmark_data_storage0111-ib.pocstor_51285_object24803/head//22 by client.275175.0:24804 2014-07-30 14:19:51.086663 2014-08-13 17:38:02.677196 7fe9bae887a0 20 read_log 191727'419 (0'0) modify a9532bad/benchmark_data_storage0111-ib.pocstor_51285_object25781/head//22 by client.275175.0:25782 2014-07-30 14:19:54.035199 2014-08-13 17:38:02.677217 7fe9bae887a0 20 read_log 191727'421 (0'0) modify d2d3bad/benchmark_data_storage0111-ib.pocstor_51285_object25880/head//22 by client.275175.0:25881 2014-07-30 14:19:54.392441 2014-08-13 17:38:02.677238 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 70940bad/benchmark_data_storage0111-ib.pocstor_51285_object27497/head//22 by client.275175.0:27498 2014-07-30 14:19:59.226875 2014-08-13 17:38:02.677259 7fe9bae887a0 20 read_log 191727'424 (0'0) modify eeebf3ad/benchmark_data_storage0111-ib.pocstor_51285_object29626/head//22 by client.275175.0:29627 2014-07-30 14:20:05.840665 2014-08-13 17:38:02.677280 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 356933ad/benchmark_data_storage0111-ib.pocstor_51285_object31396/head//22 by client.275175.0:31397 2014-07-30 14:20:11.103763 2014-08-13 17:38:02.677301 7fe9bae887a0 20 read_log 191727'432 (0'0) modify 90f48bad/benchmark_data_storage0111-ib.pocstor_51285_object33661/head//22 by client.275175.0:33662 2014-07-30 14:20:18.026837 2014-08-13 17:38:02.677325 7fe9bae887a0 20 read_log 191727'435 (0'0) modify a3894bad/benchmark_data_storage0111-ib.pocstor_51285_object34965/head//22 by client.275175.0:34966 2014-07-30 14:20:21.890626 2014-08-13 17:38:02.677346 7fe9bae887a0 20 read_log 191727'436 (0'0) modify 621a33ad/benchmark_data_storage0111-ib.pocstor_51285_object35329/head//22 by client.275175.0:35330 2014-07-30 14:20:22.965074 2014-08-13 17:38:02.677367 7fe9bae887a0 20 read_log 191727'438 (0'0) modify 7bb8c3ad/benchmark_data_storage0111-ib.pocstor_51285_object35925/head//22 by client.275175.0:35926 2014-07-30 14:20:25.019068 2014-08-13 17:38:02.677388 7fe9bae887a0 20 read_log 191727'444 (0'0) modify 3e2d83ad/benchmark_data_storage0111-ib.pocstor_51285_object37914/head//22 by client.275175.0:37915 2014-07-30 14:20:31.000553 2014-08-13 17:38:02.677412 7fe9bae887a0 20 read_log 192674'448 (0'0) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6879 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677433 7fe9bae887a0 20 read_log 192674'449 (192674'448) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6880 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677454 7fe9bae887a0 20 read_log 192674'450 (192674'449) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6881 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677474 7fe9bae887a0 20 read_log 192674'451 (192674'450) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6882 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677495 7fe9bae887a0 20 read_log 192674'452 (192674'451) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6883 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677516 7fe9bae887a0 20 read_log 192674'453 (192674'452) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6884 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677537 7fe9bae887a0 20 read_log 192674'454 (192674'453) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6885 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677558 7fe9bae887a0 20 read_log 192674'455 (192674'454) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6886 2014-07-31 18:03:48.965000 2014-08-13 17:38:02.677578 7fe9bae887a0 20 read_log 192888'456 (0'0) modify 7070bbad/benchmark_data_storage0111-ib.pocstor_35625_object475/head//22 by client.940330.0:476 2014-08-11 16:29:42.587733 2014-08-13 17:38:02.677612 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 57448bad/benchmark_data_storage0111-ib.pocstor_35625_object4103/head//22 by client.940330.0:4104 2014-08-11 16:29:56.705615 2014-08-13 17:38:02.677636 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 4b262bad/benchmark_data_storage0111-ib.pocstor_35625_object4361/head//22 by client.940330.0:4362 2014-08-11 16:29:57.713970 2014-08-13 17:38:02.677657 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 78bc73ad/benchmark_data_storage0111-ib.pocstor_35625_object5581/head//22 by client.940330.0:5582 2014-08-11 16:30:02.085856 2014-08-13 17:38:02.677681 7fe9bae887a0 20 read_log 192888'460 (0'0) modify d5a703ad/benchmark_data_storage0111-ib.pocstor_35625_object17802/head//22 by client.940330.0:17803 2014-08-11 16:30:47.977241 2014-08-13 17:38:02.677708 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 9fe73bad/benchmark_data_storage0111-ib.pocstor_35625_object18683/head//22 by client.940330.0:18684 2014-08-11 16:30:51.501260 2014-08-13 17:38:02.677729 7fe9bae887a0 20 read_log 192888'462 (0'0) modify d6194bad/benchmark_data_storage0111-ib.pocstor_35625_object19308/head//22 by client.940330.0:19309 2014-08-11 16:30:53.810500 2014-08-13 17:38:02.677750 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 6fa56bad/benchmark_data_storage0111-ib.pocstor_35625_object21887/head//22 by client.940330.0:21888 2014-08-11 16:31:04.151087 2014-08-13 17:38:02.677771 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 4f4eebad/benchmark_data_storage0111-ib.pocstor_35625_object24509/head//22 by client.940330.0:24510 2014-08-11 16:31:14.007156 2014-08-13 17:38:02.677791 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 2bb3abad/benchmark_data_storage0111-ib.pocstor_35625_object25296/head//22 by client.940330.0:25297 2014-08-11 16:31:17.047030 2014-08-13 17:38:02.677812 7fe9bae887a0 20 read_log 192888'466 (0'0) modify ec9edbad/benchmark_data_storage0111-ib.pocstor_35625_object30472/head//22 by client.940330.0:30473 2014-08-11 16:31:37.250025 2014-08-13 17:38:02.677833 7fe9bae887a0 20 read_log 192888'467 (192888'456) delete 7070bbad/benchmark_data_storage0111-ib.pocstor_35625_object475/head//22 by client.940330.0:31783 2014-08-11 16:31:41.614663 2014-08-13 17:38:02.677855 7fe9bae887a0 20 read_log 192888'468 (192888'457) delete 57448bad/benchmark_data_storage0111-ib.pocstor_35625_object4103/head//22 by client.940330.0:35411 2014-08-11 16:31:42.564040 2014-08-13 17:38:02.677876 7fe9bae887a0 20 read_log 192888'469 (192888'458) delete 4b262bad/benchmark_data_storage0111-ib.pocstor_35625_object4361/head//22 by client.940330.0:35669 2014-08-11 16:31:42.638555 2014-08-13 17:38:02.677897 7fe9bae887a0 20 read_log 192888'470 (192888'459) delete 78bc73ad/benchmark_data_storage0111-ib.pocstor_35625_object5581/head//22 by client.940330.0:36889 2014-08-11 16:31:42.964034 2014-08-13 17:38:02.677918 7fe9bae887a0 20 read_log 192888'471 (192888'460) delete d5a703ad/benchmark_data_storage0111-ib.pocstor_35625_object17802/head//22 by client.940330.0:49110 2014-08-11 16:31:46.226350 2014-08-13 17:38:02.677939 7fe9bae887a0 20 read_log 192888'472 (192888'461) delete 9fe73bad/benchmark_data_storage0111-ib.pocstor_35625_object18683/head//22 by client.940330.0:49991 2014-08-11 16:31:46.496118 2014-08-13 17:38:02.677960 7fe9bae887a0 20 read_log 192888'473 (192888'462) delete d6194bad/benchmark_data_storage0111-ib.pocstor_35625_object19308/head//22 by client.940330.0:50616 2014-08-11 16:31:46.666018 2014-08-13 17:38:02.677982 7fe9bae887a0 20 read_log 192888'474 (192888'463) delete 6fa56bad/benchmark_data_storage0111-ib.pocstor_35625_object21887/head//22 by client.940330.0:53195 2014-08-11 16:31:47.410477 2014-08-13 17:38:02.678003 7fe9bae887a0 20 read_log 192888'475 (192888'464) delete 4f4eebad/benchmark_data_storage0111-ib.pocstor_35625_object24509/head//22 by client.940330.0:55817 2014-08-11 16:31:48.191393 2014-08-13 17:38:02.678024 7fe9bae887a0 20 read_log 192888'476 (192888'465) delete 2bb3abad/benchmark_data_storage0111-ib.pocstor_35625_object25296/head//22 by client.940330.0:56604 2014-08-11 16:31:48.436407 2014-08-13 17:38:02.678086 7fe9bae887a0 20 read_log 192888'477 (192888'466) delete ec9edbad/benchmark_data_storage0111-ib.pocstor_35625_object30472/head//22 by client.940330.0:61780 2014-08-11 16:31:49.892392 2014-08-13 17:38:02.678115 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 363cd3ad/benchmark_data_storage0111-ib.pocstor_64498_object1247/head//22 by client.1049635.0:1248 2014-08-12 17:17:21.190729 2014-08-13 17:38:02.678136 7fe9bae887a0 20 read_log 194215'479 (0'0) modify 2d1823ad/benchmark_data_storage0111-ib.pocstor_64498_object4302/head//22 by client.1049635.0:4303 2014-08-12 17:17:30.955348 2014-08-13 17:38:02.678167 7fe9bae887a0 20 read_log 194215'480 (0'0) modify a79e73ad/benchmark_data_storage0111-ib.pocstor_64498_object6240/head//22 by client.1049635.0:6241 2014-08-12 17:17:36.820480 2014-08-13 17:38:02.678194 7fe9bae887a0 20 read_log 194215'481 (0'0) modify e07a9bad/benchmark_data_storage0111-ib.pocstor_64498_object6324/head//22 by client.1049635.0:6325 2014-08-12 17:17:37.057291 2014-08-13 17:38:02.678216 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 2afacbad/benchmark_data_storage0111-ib.pocstor_64498_object6446/head//22 by client.1049635.0:6447 2014-08-12 17:17:37.400204 2014-08-13 17:38:02.678236 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 95b453ad/benchmark_data_storage0111-ib.pocstor_64498_object6913/head//22 by client.1049635.0:6914 2014-08-12 17:17:38.876489 2014-08-13 17:38:02.678257 7fe9bae887a0 20 read_log 194215'484 (0'0) modify c1c25bad/benchmark_data_storage0111-ib.pocstor_64498_object7198/head//22 by client.1049635.0:7199 2014-08-12 17:17:39.691508 2014-08-13 17:38:02.678278 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 507e53ad/benchmark_data_storage0111-ib.pocstor_64498_object9948/head//22 by client.1049635.0:9949 2014-08-12 17:17:47.775579 2014-08-13 17:38:02.678298 7fe9bae887a0 20 read_log 194215'486 (0'0) modify bccc03ad/benchmark_data_storage0111-ib.pocstor_64498_object11182/head//22 by client.1049635.0:11183 2014-08-12 17:17:51.778783 2014-08-13 17:38:02.678320 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 8ee75bad/benchmark_data_storage0111-ib.pocstor_64498_object16389/head//22 by client.1049635.0:16390 2014-08-12 17:18:09.043693 2014-08-13 17:38:02.678343 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 909993ad/benchmark_data_storage0111-ib.pocstor_64498_object17543/head//22 by client.1049635.0:17544 2014-08-12 17:18:13.434157 2014-08-13 17:38:02.678365 7fe9bae887a0 20 read_log 194215'489 (0'0) modify eacbfbad/benchmark_data_storage0111-ib.pocstor_64498_object22206/head//22 by client.1049635.0:22207 2014-08-12 17:18:31.102950 2014-08-13 17:38:02.678386 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 1adf3ad/benchmark_data_storage0111-ib.pocstor_64498_object26457/head//22 by client.1049635.0:26458 2014-08-12 17:18:47.120737 2014-08-13 17:38:02.678407 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 943dcbad/benchmark_data_storage0111-ib.pocstor_64498_object30277/head//22 by client.1049635.0:30278 2014-08-12 17:19:01.664544 2014-08-13 17:38:02.678430 7fe9bae887a0 20 read_log 194215'492 (0'0) modify dd6bb3ad/benchmark_data_storage0111-ib.pocstor_64498_object30900/head//22 by client.1049635.0:30901 2014-08-12 17:19:04.185559 2014-08-13 17:38:02.678452 7fe9bae887a0 20 read_log 194215'493 (0'0) modify c2b6ebad/benchmark_data_storage0111-ib.pocstor_64498_object31613/head//22 by client.1049635.0:31614 2014-08-12 17:19:07.068733 2014-08-13 17:38:02.678473 7fe9bae887a0 20 read_log 194215'494 (0'0) modify bbfc6bad/benchmark_data_storage0111-ib.pocstor_64498_object32995/head//22 by client.1049635.0:32996 2014-08-12 17:19:12.373899 2014-08-13 17:38:02.678495 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 1c93a3ad/benchmark_data_storage0111-ib.pocstor_64498_object33298/head//22 by client.1049635.0:33299 2014-08-12 17:19:13.515117 2014-08-13 17:38:02.678519 7fe9bae887a0 20 read_log 194215'496 (194215'478) delete 363cd3ad/benchmark_data_storage0111-ib.pocstor_64498_object1247/head//22 by client.1049635.0:35450 2014-08-12 17:19:18.399580 2014-08-13 17:38:02.678541 7fe9bae887a0 20 read_log 194215'497 (194215'479) delete 2d1823ad/benchmark_data_storage0111-ib.pocstor_64498_object4302/head//22 by client.1049635.0:38505 2014-08-12 17:19:19.373433 2014-08-13 17:38:02.678562 7fe9bae887a0 20 read_log 194215'498 (194215'480) delete a79e73ad/benchmark_data_storage0111-ib.pocstor_64498_object6240/head//22 by client.1049635.0:40443 2014-08-12 17:19:20.036063 2014-08-13 17:38:02.678584 7fe9bae887a0 20 read_log 194215'499 (194215'481) delete e07a9bad/benchmark_data_storage0111-ib.pocstor_64498_object6324/head//22 by client.1049635.0:40527 2014-08-12 17:19:20.061978 2014-08-13 17:38:02.678605 7fe9bae887a0 20 read_log 194215'500 (194215'482) delete 2afacbad/benchmark_data_storage0111-ib.pocstor_64498_object6446/head//22 by client.1049635.0:40649 2014-08-12 17:19:20.097510 2014-08-13 17:38:02.678626 7fe9bae887a0 20 read_log 194215'501 (194215'483) delete 95b453ad/benchmark_data_storage0111-ib.pocstor_64498_object6913/head//22 by client.1049635.0:41116 2014-08-12 17:19:20.255432 2014-08-13 17:38:02.678647 7fe9bae887a0 20 read_log 194215'502 (194215'484) delete c1c25bad/benchmark_data_storage0111-ib.pocstor_64498_object7198/head//22 by client.1049635.0:41401 2014-08-12 17:19:20.351115 2014-08-13 17:38:02.678667 7fe9bae887a0 20 read_log 194215'503 (194215'485) delete 507e53ad/benchmark_data_storage0111-ib.pocstor_64498_object9948/head//22 by client.1049635.0:44151 2014-08-12 17:19:21.244621 2014-08-13 17:38:02.678688 7fe9bae887a0 20 read_log 194215'504 (194215'486) delete bccc03ad/benchmark_data_storage0111-ib.pocstor_64498_object11182/head//22 by client.1049635.0:45385 2014-08-12 17:19:21.636639 2014-08-13 17:38:02.678710 7fe9bae887a0 20 read_log 194215'505 (194215'487) delete 8ee75bad/benchmark_data_storage0111-ib.pocstor_64498_object16389/head//22 by client.1049635.0:50592 2014-08-12 17:19:23.318138 2014-08-13 17:38:02.678730 7fe9bae887a0 20 read_log 194215'506 (194215'488) delete 909993ad/benchmark_data_storage0111-ib.pocstor_64498_object17543/head//22 by client.1049635.0:51746 2014-08-12 17:19:23.681103 2014-08-13 17:38:02.678753 7fe9bae887a0 20 read_log 194215'507 (194215'489) delete eacbfbad/benchmark_data_storage0111-ib.pocstor_64498_object22206/head//22 by client.1049635.0:56409 2014-08-12 17:19:25.310630 2014-08-13 17:38:02.678774 7fe9bae887a0 20 read_log 194215'508 (194215'490) delete 1adf3ad/benchmark_data_storage0111-ib.pocstor_64498_object26457/head//22 by client.1049635.0:60660 2014-08-12 17:19:26.808026 2014-08-13 17:38:02.678795 7fe9bae887a0 20 read_log 194215'509 (194215'491) delete 943dcbad/benchmark_data_storage0111-ib.pocstor_64498_object30277/head//22 by client.1049635.0:64480 2014-08-12 17:19:28.137180 2014-08-13 17:38:02.678816 7fe9bae887a0 20 read_log 194215'510 (194215'492) delete dd6bb3ad/benchmark_data_storage0111-ib.pocstor_64498_object30900/head//22 by client.1049635.0:65103 2014-08-12 17:19:28.349001 2014-08-13 17:38:02.678837 7fe9bae887a0 20 read_log 194215'511 (194215'493) delete c2b6ebad/benchmark_data_storage0111-ib.pocstor_64498_object31613/head//22 by client.1049635.0:65816 2014-08-12 17:19:28.604932 2014-08-13 17:38:02.678859 7fe9bae887a0 20 read_log 194215'512 (194215'494) delete bbfc6bad/benchmark_data_storage0111-ib.pocstor_64498_object32995/head//22 by client.1049635.0:67198 2014-08-12 17:19:29.112824 2014-08-13 17:38:02.678880 7fe9bae887a0 20 read_log 194215'513 (194215'495) delete 1c93a3ad/benchmark_data_storage0111-ib.pocstor_64498_object33298/head//22 by client.1049635.0:67501 2014-08-12 17:19:29.217217 2014-08-13 17:38:02.679056 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.679087 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=0 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.679113 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=0 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Initial 0.003056 0 0.000000 2014-08-13 17:38:02.679133 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=0 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.679147 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=0 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.679172 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] log((0'0,194215'513], crt=192888'477) 2014-08-13 17:38:02.679189 7fe9bae887a0 10 osd.4 194478 pgid 22.3af coll 22.3af_head 2014-08-13 17:38:02.679375 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3af 2014-08-13 17:38:02.679391 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.679437 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3af(unlocked)] enter Initial 2014-08-13 17:38:02.679456 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3af(unlocked)] enter NotTrimming 2014-08-13 17:38:02.679664 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.679855 7fe9bae887a0 20 read_log 191648'76 (0'0) modify 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 by client.273158.0:4306 2014-07-30 14:09:07.998582 2014-08-13 17:38:02.679891 7fe9bae887a0 20 read_log 191648'109 (0'0) modify 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 by client.273158.0:6338 2014-07-30 14:09:17.770657 2014-08-13 17:38:02.679913 7fe9bae887a0 20 read_log 191648'140 (0'0) modify 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 by client.273158.0:8285 2014-07-30 14:09:27.760006 2014-08-13 17:38:02.679934 7fe9bae887a0 20 read_log 191648'149 (0'0) modify 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 by client.273158.0:8900 2014-07-30 14:09:30.773627 2014-08-13 17:38:02.679956 7fe9bae887a0 20 read_log 191648'165 (0'0) modify f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 by client.273158.0:10091 2014-07-30 14:09:36.195486 2014-08-13 17:38:02.679977 7fe9bae887a0 20 read_log 191648'208 (0'0) modify 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 by client.273158.0:12745 2014-07-30 14:09:51.188747 2014-08-13 17:38:02.679998 7fe9bae887a0 20 read_log 191648'320 (0'0) modify b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 by client.273158.0:18769 2014-07-30 14:10:22.639242 2014-08-13 17:38:02.680019 7fe9bae887a0 20 read_log 191648'355 (0'0) modify 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 by client.273158.0:20631 2014-07-30 14:10:34.448792 2014-08-13 17:38:02.680040 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 by client.275175.0:1876 2014-07-30 14:18:40.498228 2014-08-13 17:38:02.680061 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 by client.275175.0:3580 2014-07-30 14:18:46.047221 2014-08-13 17:38:02.680082 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 by client.275175.0:3942 2014-07-30 14:18:47.177767 2014-08-13 17:38:02.680102 7fe9bae887a0 20 read_log 191727'397 (0'0) modify aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 by client.275175.0:5493 2014-07-30 14:18:51.793595 2014-08-13 17:38:02.680123 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 by client.275175.0:6917 2014-07-30 14:18:56.483335 2014-08-13 17:38:02.680146 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 by client.275175.0:8854 2014-07-30 14:19:01.927942 2014-08-13 17:38:02.680217 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 by client.275175.0:8969 2014-07-30 14:19:02.258534 2014-08-13 17:38:02.680247 7fe9bae887a0 20 read_log 191727'406 (0'0) modify c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 by client.275175.0:10506 2014-07-30 14:19:06.655258 2014-08-13 17:38:02.680268 7fe9bae887a0 20 read_log 191727'409 (0'0) modify ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 by client.275175.0:11865 2014-07-30 14:19:10.918722 2014-08-13 17:38:02.680289 7fe9bae887a0 20 read_log 191727'413 (0'0) modify f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 by client.275175.0:14382 2014-07-30 14:19:18.956215 2014-08-13 17:38:02.680310 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 by client.275175.0:16495 2014-07-30 14:19:25.649635 2014-08-13 17:38:02.680331 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 by client.275175.0:18154 2014-07-30 14:19:30.385252 2014-08-13 17:38:02.680352 7fe9bae887a0 20 read_log 191727'428 (0'0) modify 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 by client.275175.0:20701 2014-07-30 14:19:38.272502 2014-08-13 17:38:02.680372 7fe9bae887a0 20 read_log 191727'437 (0'0) modify b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 by client.275175.0:24201 2014-07-30 14:19:49.391611 2014-08-13 17:38:02.680393 7fe9bae887a0 20 read_log 191727'438 (0'0) modify 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 by client.275175.0:24689 2014-07-30 14:19:50.765134 2014-08-13 17:38:02.680414 7fe9bae887a0 20 read_log 192674'458 (0'0) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7007 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680434 7fe9bae887a0 20 read_log 192674'459 (192674'458) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7008 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680455 7fe9bae887a0 20 read_log 192674'460 (192674'459) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7009 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680476 7fe9bae887a0 20 read_log 192674'461 (192674'460) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7010 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680496 7fe9bae887a0 20 read_log 192674'462 (192674'461) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7011 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680519 7fe9bae887a0 20 read_log 192674'463 (192674'462) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7012 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680539 7fe9bae887a0 20 read_log 192674'464 (192674'463) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7013 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680560 7fe9bae887a0 20 read_log 192674'465 (192674'464) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7014 2014-07-31 18:03:49.071000 2014-08-13 17:38:02.680584 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 5bcdd3af/benchmark_data_storage0111-ib.pocstor_35625_object0/head//22 by client.940330.0:1 2014-08-11 16:29:40.419425 2014-08-13 17:38:02.680605 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 6512cbaf/benchmark_data_storage0111-ib.pocstor_35625_object485/head//22 by client.940330.0:486 2014-08-11 16:29:42.642917 2014-08-13 17:38:02.680626 7fe9bae887a0 20 read_log 192888'468 (0'0) modify 8b144baf/benchmark_data_storage0111-ib.pocstor_35625_object1526/head//22 by client.940330.0:1527 2014-08-11 16:29:46.799832 2014-08-13 17:38:02.680661 7fe9bae887a0 20 read_log 192888'469 (0'0) modify c27f6baf/benchmark_data_storage0111-ib.pocstor_35625_object2677/head//22 by client.940330.0:2678 2014-08-11 16:29:51.141644 2014-08-13 17:38:02.680688 7fe9bae887a0 20 read_log 192888'470 (0'0) modify c329a3af/benchmark_data_storage0111-ib.pocstor_35625_object4914/head//22 by client.940330.0:4915 2014-08-11 16:29:59.729074 2014-08-13 17:38:02.680709 7fe9bae887a0 20 read_log 192888'471 (0'0) modify 746763af/benchmark_data_storage0111-ib.pocstor_35625_object5143/head//22 by client.940330.0:5144 2014-08-11 16:30:00.537737 2014-08-13 17:38:02.680730 7fe9bae887a0 20 read_log 192888'472 (0'0) modify b3abc3af/benchmark_data_storage0111-ib.pocstor_35625_object5366/head//22 by client.940330.0:5367 2014-08-11 16:30:01.330123 2014-08-13 17:38:02.680751 7fe9bae887a0 20 read_log 192888'473 (0'0) modify ead2dbaf/benchmark_data_storage0111-ib.pocstor_35625_object13251/head//22 by client.940330.0:13252 2014-08-11 16:30:30.859372 2014-08-13 17:38:02.680772 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 84c513af/benchmark_data_storage0111-ib.pocstor_35625_object13740/head//22 by client.940330.0:13741 2014-08-11 16:30:32.659321 2014-08-13 17:38:02.680833 7fe9bae887a0 20 read_log 192888'475 (0'0) modify f01003af/benchmark_data_storage0111-ib.pocstor_35625_object13922/head//22 by client.940330.0:13923 2014-08-11 16:30:33.352296 2014-08-13 17:38:02.680854 7fe9bae887a0 20 read_log 192888'476 (0'0) modify 343b43af/benchmark_data_storage0111-ib.pocstor_35625_object14810/head//22 by client.940330.0:14811 2014-08-11 16:30:36.783848 2014-08-13 17:38:02.680875 7fe9bae887a0 20 read_log 192888'477 (0'0) modify 67d23baf/benchmark_data_storage0111-ib.pocstor_35625_object19907/head//22 by client.940330.0:19908 2014-08-11 16:30:56.471441 2014-08-13 17:38:02.680896 7fe9bae887a0 20 read_log 192888'478 (0'0) modify 5474b3af/benchmark_data_storage0111-ib.pocstor_35625_object20104/head//22 by client.940330.0:20105 2014-08-11 16:30:57.198105 2014-08-13 17:38:02.680919 7fe9bae887a0 20 read_log 192888'479 (0'0) modify c60f93af/benchmark_data_storage0111-ib.pocstor_35625_object22423/head//22 by client.940330.0:22424 2014-08-11 16:31:06.133146 2014-08-13 17:38:02.680940 7fe9bae887a0 20 read_log 192888'480 (0'0) modify b1891baf/benchmark_data_storage0111-ib.pocstor_35625_object24479/head//22 by client.940330.0:24480 2014-08-11 16:31:13.896141 2014-08-13 17:38:02.680961 7fe9bae887a0 20 read_log 192888'481 (0'0) modify bcfc53af/benchmark_data_storage0111-ib.pocstor_35625_object25442/head//22 by client.940330.0:25443 2014-08-11 16:31:17.614531 2014-08-13 17:38:02.680981 7fe9bae887a0 20 read_log 192888'482 (0'0) modify a46103af/benchmark_data_storage0111-ib.pocstor_35625_object25479/head//22 by client.940330.0:25480 2014-08-11 16:31:17.756817 2014-08-13 17:38:02.681002 7fe9bae887a0 20 read_log 192888'483 (0'0) modify 103f5baf/benchmark_data_storage0111-ib.pocstor_35625_object26257/head//22 by client.940330.0:26258 2014-08-11 16:31:20.802762 2014-08-13 17:38:02.681022 7fe9bae887a0 20 read_log 192888'484 (0'0) modify 86588baf/benchmark_data_storage0111-ib.pocstor_35625_object29331/head//22 by client.940330.0:29332 2014-08-11 16:31:32.927228 2014-08-13 17:38:02.681042 7fe9bae887a0 20 read_log 192888'485 (0'0) modify 5d0573af/benchmark_data_storage0111-ib.pocstor_35625_object31175/head//22 by client.940330.0:31176 2014-08-11 16:31:39.880561 2014-08-13 17:38:02.681063 7fe9bae887a0 20 read_log 192888'486 (192888'466) delete 5bcdd3af/benchmark_data_storage0111-ib.pocstor_35625_object0/head//22 by client.940330.0:31308 2014-08-11 16:31:41.446696 2014-08-13 17:38:02.681084 7fe9bae887a0 20 read_log 192888'487 (192888'467) delete 6512cbaf/benchmark_data_storage0111-ib.pocstor_35625_object485/head//22 by client.940330.0:31793 2014-08-11 16:31:41.618691 2014-08-13 17:38:02.681106 7fe9bae887a0 20 read_log 192888'488 (192888'468) delete 8b144baf/benchmark_data_storage0111-ib.pocstor_35625_object1526/head//22 by client.940330.0:32834 2014-08-11 16:31:41.873868 2014-08-13 17:38:02.681140 7fe9bae887a0 20 read_log 192888'489 (192888'469) delete c27f6baf/benchmark_data_storage0111-ib.pocstor_35625_object2677/head//22 by client.940330.0:33985 2014-08-11 16:31:42.165785 2014-08-13 17:38:02.681174 7fe9bae887a0 20 read_log 192888'490 (192888'470) delete c329a3af/benchmark_data_storage0111-ib.pocstor_35625_object4914/head//22 by client.940330.0:36222 2014-08-11 16:31:42.795676 2014-08-13 17:38:02.681196 7fe9bae887a0 20 read_log 192888'491 (192888'471) delete 746763af/benchmark_data_storage0111-ib.pocstor_35625_object5143/head//22 by client.940330.0:36451 2014-08-11 16:31:42.852981 2014-08-13 17:38:02.681217 7fe9bae887a0 20 read_log 192888'492 (192888'472) delete b3abc3af/benchmark_data_storage0111-ib.pocstor_35625_object5366/head//22 by client.940330.0:36674 2014-08-11 16:31:42.909023 2014-08-13 17:38:02.681238 7fe9bae887a0 20 read_log 192888'493 (192888'473) delete ead2dbaf/benchmark_data_storage0111-ib.pocstor_35625_object13251/head//22 by client.940330.0:44559 2014-08-11 16:31:44.990613 2014-08-13 17:38:02.681259 7fe9bae887a0 20 read_log 192888'494 (192888'474) delete 84c513af/benchmark_data_storage0111-ib.pocstor_35625_object13740/head//22 by client.940330.0:45048 2014-08-11 16:31:45.114960 2014-08-13 17:38:02.681283 7fe9bae887a0 20 read_log 192888'495 (192888'475) delete f01003af/benchmark_data_storage0111-ib.pocstor_35625_object13922/head//22 by client.940330.0:45230 2014-08-11 16:31:45.162688 2014-08-13 17:38:02.681305 7fe9bae887a0 20 read_log 192888'496 (192888'476) delete 343b43af/benchmark_data_storage0111-ib.pocstor_35625_object14810/head//22 by client.940330.0:46118 2014-08-11 16:31:45.402568 2014-08-13 17:38:02.681326 7fe9bae887a0 20 read_log 192888'497 (192888'477) delete 67d23baf/benchmark_data_storage0111-ib.pocstor_35625_object19907/head//22 by client.940330.0:51215 2014-08-11 16:31:46.840277 2014-08-13 17:38:02.681347 7fe9bae887a0 20 read_log 192888'498 (192888'478) delete 5474b3af/benchmark_data_storage0111-ib.pocstor_35625_object20104/head//22 by client.940330.0:51412 2014-08-11 16:31:46.911880 2014-08-13 17:38:02.681368 7fe9bae887a0 20 read_log 192888'499 (192888'479) delete c60f93af/benchmark_data_storage0111-ib.pocstor_35625_object22423/head//22 by client.940330.0:53731 2014-08-11 16:31:47.581184 2014-08-13 17:38:02.681389 7fe9bae887a0 20 read_log 192888'500 (192888'480) delete b1891baf/benchmark_data_storage0111-ib.pocstor_35625_object24479/head//22 by client.940330.0:55787 2014-08-11 16:31:48.183283 2014-08-13 17:38:02.681410 7fe9bae887a0 20 read_log 192888'501 (192888'481) delete bcfc53af/benchmark_data_storage0111-ib.pocstor_35625_object25442/head//22 by client.940330.0:56750 2014-08-11 16:31:48.481125 2014-08-13 17:38:02.681431 7fe9bae887a0 20 read_log 192888'502 (192888'482) delete a46103af/benchmark_data_storage0111-ib.pocstor_35625_object25479/head//22 by client.940330.0:56787 2014-08-11 16:31:48.493557 2014-08-13 17:38:02.681452 7fe9bae887a0 20 read_log 192888'503 (192888'483) delete 103f5baf/benchmark_data_storage0111-ib.pocstor_35625_object26257/head//22 by client.940330.0:57565 2014-08-11 16:31:48.720632 2014-08-13 17:38:02.681473 7fe9bae887a0 20 read_log 192888'504 (192888'484) delete 86588baf/benchmark_data_storage0111-ib.pocstor_35625_object29331/head//22 by client.940330.0:60639 2014-08-11 16:31:49.561774 2014-08-13 17:38:02.681493 7fe9bae887a0 20 read_log 192888'505 (192888'485) delete 5d0573af/benchmark_data_storage0111-ib.pocstor_35625_object31175/head//22 by client.940330.0:62483 2014-08-11 16:31:50.112879 2014-08-13 17:38:02.681518 7fe9bae887a0 20 read_log 194215'506 (0'0) modify 66b83af/benchmark_data_storage0111-ib.pocstor_64498_object131/head//22 by client.1049635.0:132 2014-08-12 17:17:17.447484 2014-08-13 17:38:02.681540 7fe9bae887a0 20 read_log 194215'507 (0'0) modify 903563af/benchmark_data_storage0111-ib.pocstor_64498_object262/head//22 by client.1049635.0:263 2014-08-12 17:17:17.922454 2014-08-13 17:38:02.681573 7fe9bae887a0 20 read_log 194215'508 (0'0) modify 2cdbaf/benchmark_data_storage0111-ib.pocstor_64498_object359/head//22 by client.1049635.0:360 2014-08-12 17:17:18.277079 2014-08-13 17:38:02.681600 7fe9bae887a0 20 read_log 194215'509 (0'0) modify fb8ab3af/benchmark_data_storage0111-ib.pocstor_64498_object425/head//22 by client.1049635.0:426 2014-08-12 17:17:18.513313 2014-08-13 17:38:02.681621 7fe9bae887a0 20 read_log 194215'510 (0'0) modify f542cbaf/benchmark_data_storage0111-ib.pocstor_64498_object763/head//22 by client.1049635.0:764 2014-08-12 17:17:19.583517 2014-08-13 17:38:02.681642 7fe9bae887a0 20 read_log 194215'511 (0'0) modify 2a7e03af/benchmark_data_storage0111-ib.pocstor_64498_object943/head//22 by client.1049635.0:944 2014-08-12 17:17:20.169978 2014-08-13 17:38:02.681663 7fe9bae887a0 20 read_log 194215'512 (0'0) modify dd537baf/benchmark_data_storage0111-ib.pocstor_64498_object9149/head//22 by client.1049635.0:9150 2014-08-12 17:17:45.449474 2014-08-13 17:38:02.681683 7fe9bae887a0 20 read_log 194215'513 (0'0) modify 1be2b3af/benchmark_data_storage0111-ib.pocstor_64498_object11766/head//22 by client.1049635.0:11767 2014-08-12 17:17:53.700790 2014-08-13 17:38:02.681704 7fe9bae887a0 20 read_log 194215'514 (0'0) modify 9e45bbaf/benchmark_data_storage0111-ib.pocstor_64498_object16982/head//22 by client.1049635.0:16983 2014-08-12 17:18:11.297403 2014-08-13 17:38:02.681725 7fe9bae887a0 20 read_log 194215'515 (0'0) modify 3f4eb3af/benchmark_data_storage0111-ib.pocstor_64498_object17847/head//22 by client.1049635.0:17848 2014-08-12 17:18:14.580045 2014-08-13 17:38:02.681746 7fe9bae887a0 20 read_log 194215'516 (0'0) modify 45e603af/benchmark_data_storage0111-ib.pocstor_64498_object19221/head//22 by client.1049635.0:19222 2014-08-12 17:18:19.868625 2014-08-13 17:38:02.681767 7fe9bae887a0 20 read_log 194215'517 (0'0) modify 842aabaf/benchmark_data_storage0111-ib.pocstor_64498_object20868/head//22 by client.1049635.0:20869 2014-08-12 17:18:26.035064 2014-08-13 17:38:02.681788 7fe9bae887a0 20 read_log 194215'518 (0'0) modify 5aa89baf/benchmark_data_storage0111-ib.pocstor_64498_object21961/head//22 by client.1049635.0:21962 2014-08-12 17:18:30.185481 2014-08-13 17:38:02.681809 7fe9bae887a0 20 read_log 194215'519 (0'0) modify cdd48baf/benchmark_data_storage0111-ib.pocstor_64498_object22353/head//22 by client.1049635.0:22354 2014-08-12 17:18:31.658260 2014-08-13 17:38:02.681830 7fe9bae887a0 20 read_log 194215'520 (0'0) modify 3a232baf/benchmark_data_storage0111-ib.pocstor_64498_object24288/head//22 by client.1049635.0:24289 2014-08-12 17:18:38.929540 2014-08-13 17:38:02.681851 7fe9bae887a0 20 read_log 194215'521 (0'0) modify 5b6a63af/benchmark_data_storage0111-ib.pocstor_64498_object30696/head//22 by client.1049635.0:30697 2014-08-12 17:19:03.274748 2014-08-13 17:38:02.681874 7fe9bae887a0 20 read_log 194215'522 (0'0) modify 82777baf/benchmark_data_storage0111-ib.pocstor_64498_object32715/head//22 by client.1049635.0:32716 2014-08-12 17:19:11.312579 2014-08-13 17:38:02.681895 7fe9bae887a0 20 read_log 194215'523 (194215'506) delete 66b83af/benchmark_data_storage0111-ib.pocstor_64498_object131/head//22 by client.1049635.0:34334 2014-08-12 17:19:18.029163 2014-08-13 17:38:02.681916 7fe9bae887a0 20 read_log 194215'524 (194215'507) delete 903563af/benchmark_data_storage0111-ib.pocstor_64498_object262/head//22 by client.1049635.0:34465 2014-08-12 17:19:18.079370 2014-08-13 17:38:02.681969 7fe9bae887a0 20 read_log 194215'525 (194215'508) delete 2cdbaf/benchmark_data_storage0111-ib.pocstor_64498_object359/head//22 by client.1049635.0:34562 2014-08-12 17:19:18.110409 2014-08-13 17:38:02.681997 7fe9bae887a0 20 read_log 194215'526 (194215'509) delete fb8ab3af/benchmark_data_storage0111-ib.pocstor_64498_object425/head//22 by client.1049635.0:34628 2014-08-12 17:19:18.136084 2014-08-13 17:38:02.682019 7fe9bae887a0 20 read_log 194215'527 (194215'510) delete f542cbaf/benchmark_data_storage0111-ib.pocstor_64498_object763/head//22 by client.1049635.0:34966 2014-08-12 17:19:18.240414 2014-08-13 17:38:02.682042 7fe9bae887a0 20 read_log 194215'528 (194215'511) delete 2a7e03af/benchmark_data_storage0111-ib.pocstor_64498_object943/head//22 by client.1049635.0:35146 2014-08-12 17:19:18.301070 2014-08-13 17:38:02.682064 7fe9bae887a0 20 read_log 194215'529 (194215'512) delete dd537baf/benchmark_data_storage0111-ib.pocstor_64498_object9149/head//22 by client.1049635.0:43352 2014-08-12 17:19:20.969513 2014-08-13 17:38:02.682085 7fe9bae887a0 20 read_log 194215'530 (194215'513) delete 1be2b3af/benchmark_data_storage0111-ib.pocstor_64498_object11766/head//22 by client.1049635.0:45969 2014-08-12 17:19:21.830180 2014-08-13 17:38:02.682107 7fe9bae887a0 20 read_log 194215'531 (194215'514) delete 9e45bbaf/benchmark_data_storage0111-ib.pocstor_64498_object16982/head//22 by client.1049635.0:51185 2014-08-12 17:19:23.503350 2014-08-13 17:38:02.682128 7fe9bae887a0 20 read_log 194215'532 (194215'515) delete 3f4eb3af/benchmark_data_storage0111-ib.pocstor_64498_object17847/head//22 by client.1049635.0:52050 2014-08-12 17:19:23.776275 2014-08-13 17:38:02.682149 7fe9bae887a0 20 read_log 194215'533 (194215'516) delete 45e603af/benchmark_data_storage0111-ib.pocstor_64498_object19221/head//22 by client.1049635.0:53424 2014-08-12 17:19:24.241367 2014-08-13 17:38:02.682179 7fe9bae887a0 20 read_log 194215'534 (194215'517) delete 842aabaf/benchmark_data_storage0111-ib.pocstor_64498_object20868/head//22 by client.1049635.0:55071 2014-08-12 17:19:24.839028 2014-08-13 17:38:02.682200 7fe9bae887a0 20 read_log 194215'535 (194215'518) delete 5aa89baf/benchmark_data_storage0111-ib.pocstor_64498_object21961/head//22 by client.1049635.0:56164 2014-08-12 17:19:25.218864 2014-08-13 17:38:02.682221 7fe9bae887a0 20 read_log 194215'536 (194215'519) delete cdd48baf/benchmark_data_storage0111-ib.pocstor_64498_object22353/head//22 by client.1049635.0:56556 2014-08-12 17:19:25.363899 2014-08-13 17:38:02.682242 7fe9bae887a0 20 read_log 194215'537 (194215'520) delete 3a232baf/benchmark_data_storage0111-ib.pocstor_64498_object24288/head//22 by client.1049635.0:58491 2014-08-12 17:19:26.061637 2014-08-13 17:38:02.682263 7fe9bae887a0 20 read_log 194215'538 (194215'521) delete 5b6a63af/benchmark_data_storage0111-ib.pocstor_64498_object30696/head//22 by client.1049635.0:64899 2014-08-12 17:19:28.272326 2014-08-13 17:38:02.682284 7fe9bae887a0 20 read_log 194215'539 (194215'522) delete 82777baf/benchmark_data_storage0111-ib.pocstor_64498_object32715/head//22 by client.1049635.0:66918 2014-08-12 17:19:29.010187 2014-08-13 17:38:02.682468 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.682501 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=0 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.682530 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=0 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003093 0 0.000000 2014-08-13 17:38:02.682551 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=0 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.682565 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=0 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.682580 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'539], crt=191727'457) 2014-08-13 17:38:02.682597 7fe9bae887a0 10 osd.4 194478 pgid 22.3b9 coll 22.3b9_head 2014-08-13 17:38:02.682817 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3b9 2014-08-13 17:38:02.682833 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.682878 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3b9(unlocked)] enter Initial 2014-08-13 17:38:02.682897 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3b9(unlocked)] enter NotTrimming 2014-08-13 17:38:02.683100 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.683323 7fe9bae887a0 20 read_log 191648'28 (0'0) modify 2a5b3bb9/benchmark_data_storage0111-ib.pocstor_40745_object1810/head//22 by client.273158.0:1811 2014-07-30 14:08:55.427627 2014-08-13 17:38:02.683359 7fe9bae887a0 20 read_log 191648'115 (0'0) modify 21a30bb9/benchmark_data_storage0111-ib.pocstor_40745_object8125/head//22 by client.273158.0:8126 2014-07-30 14:09:26.690956 2014-08-13 17:38:02.683382 7fe9bae887a0 20 read_log 191648'123 (0'0) modify 693b9/benchmark_data_storage0111-ib.pocstor_40745_object8507/head//22 by client.273158.0:8508 2014-07-30 14:09:28.932306 2014-08-13 17:38:02.683406 7fe9bae887a0 20 read_log 191648'188 (0'0) modify 203ef3b9/benchmark_data_storage0111-ib.pocstor_40745_object12560/head//22 by client.273158.0:12561 2014-07-30 14:09:50.361312 2014-08-13 17:38:02.683431 7fe9bae887a0 20 read_log 191648'214 (0'0) modify 567f43b9/benchmark_data_storage0111-ib.pocstor_40745_object14002/head//22 by client.273158.0:14003 2014-07-30 14:09:57.014126 2014-08-13 17:38:02.683453 7fe9bae887a0 20 read_log 191648'225 (0'0) modify e8d303b9/benchmark_data_storage0111-ib.pocstor_40745_object14493/head//22 by client.273158.0:14494 2014-07-30 14:10:00.057762 2014-08-13 17:38:02.683474 7fe9bae887a0 20 read_log 191648'235 (0'0) modify 2ca733b9/benchmark_data_storage0111-ib.pocstor_40745_object15068/head//22 by client.273158.0:15069 2014-07-30 14:10:02.937032 2014-08-13 17:38:02.683495 7fe9bae887a0 20 read_log 191648'236 (0'0) modify 1ce98bb9/benchmark_data_storage0111-ib.pocstor_40745_object15366/head//22 by client.273158.0:15367 2014-07-30 14:10:04.129794 2014-08-13 17:38:02.683516 7fe9bae887a0 20 read_log 191648'318 (0'0) modify e0efbb9/benchmark_data_storage0111-ib.pocstor_40745_object20766/head//22 by client.273158.0:20767 2014-07-30 14:10:35.381754 2014-08-13 17:38:02.683537 7fe9bae887a0 20 read_log 191648'352 (0'0) modify a9dcd3b9/benchmark_data_storage0111-ib.pocstor_40745_object22760/head//22 by client.273158.0:22761 2014-07-30 14:10:48.105926 2014-08-13 17:38:02.683558 7fe9bae887a0 20 read_log 191727'356 (0'0) modify e2d5f3b9/benchmark_data_storage0111-ib.pocstor_51285_object2765/head//22 by client.275175.0:2766 2014-07-30 14:18:43.475350 2014-08-13 17:38:02.683580 7fe9bae887a0 20 read_log 191727'362 (0'0) modify 320a23b9/benchmark_data_storage0111-ib.pocstor_51285_object5104/head//22 by client.275175.0:5105 2014-07-30 14:18:50.657353 2014-08-13 17:38:02.683601 7fe9bae887a0 20 read_log 191727'367 (0'0) modify f4667bb9/benchmark_data_storage0111-ib.pocstor_51285_object6287/head//22 by client.275175.0:6288 2014-07-30 14:18:54.117804 2014-08-13 17:38:02.683622 7fe9bae887a0 20 read_log 191727'369 (0'0) modify cd3d63b9/benchmark_data_storage0111-ib.pocstor_51285_object7539/head//22 by client.275175.0:7540 2014-07-30 14:18:58.242087 2014-08-13 17:38:02.683643 7fe9bae887a0 20 read_log 191727'370 (0'0) modify c4ca0bb9/benchmark_data_storage0111-ib.pocstor_51285_object7759/head//22 by client.275175.0:7760 2014-07-30 14:18:58.856387 2014-08-13 17:38:02.683664 7fe9bae887a0 20 read_log 191727'372 (0'0) modify f3844bb9/benchmark_data_storage0111-ib.pocstor_51285_object8114/head//22 by client.275175.0:8115 2014-07-30 14:18:59.841862 2014-08-13 17:38:02.683685 7fe9bae887a0 20 read_log 191727'376 (0'0) modify ba6093b9/benchmark_data_storage0111-ib.pocstor_51285_object10104/head//22 by client.275175.0:10105 2014-07-30 14:19:05.471588 2014-08-13 17:38:02.683706 7fe9bae887a0 20 read_log 191727'382 (0'0) modify e50d7bb9/benchmark_data_storage0111-ib.pocstor_51285_object14050/head//22 by client.275175.0:14051 2014-07-30 14:19:18.021904 2014-08-13 17:38:02.683742 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 751c93b9/benchmark_data_storage0111-ib.pocstor_51285_object18539/head//22 by client.275175.0:18540 2014-07-30 14:19:31.596072 2014-08-13 17:38:02.683764 7fe9bae887a0 20 read_log 191727'393 (0'0) modify 9d994bb9/benchmark_data_storage0111-ib.pocstor_51285_object19826/head//22 by client.275175.0:19827 2014-07-30 14:19:35.682687 2014-08-13 17:38:02.683785 7fe9bae887a0 20 read_log 191727'402 (0'0) modify e5b053b9/benchmark_data_storage0111-ib.pocstor_51285_object25986/head//22 by client.275175.0:25987 2014-07-30 14:19:54.685530 2014-08-13 17:38:02.683805 7fe9bae887a0 20 read_log 191727'403 (0'0) modify cfadbb9/benchmark_data_storage0111-ib.pocstor_51285_object26228/head//22 by client.275175.0:26229 2014-07-30 14:19:55.517587 2014-08-13 17:38:02.683826 7fe9bae887a0 20 read_log 191727'404 (0'0) modify ee764bb9/benchmark_data_storage0111-ib.pocstor_51285_object26342/head//22 by client.275175.0:26343 2014-07-30 14:19:55.835792 2014-08-13 17:38:02.683846 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 91a0ebb9/benchmark_data_storage0111-ib.pocstor_51285_object28143/head//22 by client.275175.0:28144 2014-07-30 14:20:01.430194 2014-08-13 17:38:02.683867 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 36d8ebb9/benchmark_data_storage0111-ib.pocstor_51285_object30225/head//22 by client.275175.0:30226 2014-07-30 14:20:07.791509 2014-08-13 17:38:02.683887 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 54abbbb9/benchmark_data_storage0111-ib.pocstor_51285_object30594/head//22 by client.275175.0:30595 2014-07-30 14:20:08.839064 2014-08-13 17:38:02.683907 7fe9bae887a0 20 read_log 191727'416 (0'0) modify c9b053b9/benchmark_data_storage0111-ib.pocstor_51285_object32643/head//22 by client.275175.0:32644 2014-07-30 14:20:14.704920 2014-08-13 17:38:02.683928 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 6eeb43b9/benchmark_data_storage0111-ib.pocstor_51285_object33445/head//22 by client.275175.0:33446 2014-07-30 14:20:17.234508 2014-08-13 17:38:02.683948 7fe9bae887a0 20 read_log 191727'419 (0'0) modify 67d0e3b9/benchmark_data_storage0111-ib.pocstor_51285_object33946/head//22 by client.275175.0:33947 2014-07-30 14:20:18.839143 2014-08-13 17:38:02.683969 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 803ccbb9/benchmark_data_storage0111-ib.pocstor_51285_object34472/head//22 by client.275175.0:34473 2014-07-30 14:20:20.470415 2014-08-13 17:38:02.683989 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 46957bb9/benchmark_data_storage0111-ib.pocstor_51285_object34917/head//22 by client.275175.0:34918 2014-07-30 14:20:21.751892 2014-08-13 17:38:02.684010 7fe9bae887a0 20 read_log 191727'432 (0'0) modify 680b5bb9/benchmark_data_storage0111-ib.pocstor_51285_object37458/head//22 by client.275175.0:37459 2014-07-30 14:20:29.732351 2014-08-13 17:38:02.684030 7fe9bae887a0 20 read_log 191727'433 (0'0) modify 969c53b9/benchmark_data_storage0111-ib.pocstor_51285_object37635/head//22 by client.275175.0:37636 2014-07-30 14:20:30.225573 2014-08-13 17:38:02.684051 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 738eabb9/benchmark_data_storage0111-ib.pocstor_35625_object2657/head//22 by client.940330.0:2658 2014-08-11 16:29:51.070731 2014-08-13 17:38:02.684071 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 9aea63b9/benchmark_data_storage0111-ib.pocstor_35625_object3811/head//22 by client.940330.0:3812 2014-08-11 16:29:55.641650 2014-08-13 17:38:02.684092 7fe9bae887a0 20 read_log 192888'439 (0'0) modify b1ea83b9/benchmark_data_storage0111-ib.pocstor_35625_object4015/head//22 by client.940330.0:4016 2014-08-11 16:29:56.386850 2014-08-13 17:38:02.684112 7fe9bae887a0 20 read_log 192888'440 (0'0) modify a8846bb9/benchmark_data_storage0111-ib.pocstor_35625_object4917/head//22 by client.940330.0:4918 2014-08-11 16:29:59.739930 2014-08-13 17:38:02.684188 7fe9bae887a0 20 read_log 192888'441 (0'0) modify be9243b9/benchmark_data_storage0111-ib.pocstor_35625_object5330/head//22 by client.940330.0:5331 2014-08-11 16:30:01.202480 2014-08-13 17:38:02.684217 7fe9bae887a0 20 read_log 192888'442 (0'0) modify ab2ee3b9/benchmark_data_storage0111-ib.pocstor_35625_object8334/head//22 by client.940330.0:8335 2014-08-11 16:30:12.010065 2014-08-13 17:38:02.684239 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 5c7df3b9/benchmark_data_storage0111-ib.pocstor_35625_object8868/head//22 by client.940330.0:8869 2014-08-11 16:30:13.872213 2014-08-13 17:38:02.684261 7fe9bae887a0 20 read_log 192888'444 (0'0) modify cbdec3b9/benchmark_data_storage0111-ib.pocstor_35625_object9820/head//22 by client.940330.0:9821 2014-08-11 16:30:17.201017 2014-08-13 17:38:02.684286 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 9851f3b9/benchmark_data_storage0111-ib.pocstor_35625_object10015/head//22 by client.940330.0:10016 2014-08-11 16:30:17.960196 2014-08-13 17:38:02.684306 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 76ca73b9/benchmark_data_storage0111-ib.pocstor_35625_object10720/head//22 by client.940330.0:10721 2014-08-11 16:30:20.713060 2014-08-13 17:38:02.684366 7fe9bae887a0 20 read_log 192888'447 (0'0) modify db73c3b9/benchmark_data_storage0111-ib.pocstor_35625_object11988/head//22 by client.940330.0:11989 2014-08-11 16:30:25.895466 2014-08-13 17:38:02.684394 7fe9bae887a0 20 read_log 192888'448 (0'0) modify cf8063b9/benchmark_data_storage0111-ib.pocstor_35625_object12943/head//22 by client.940330.0:12944 2014-08-11 16:30:29.551052 2014-08-13 17:38:02.684415 7fe9bae887a0 20 read_log 192888'449 (0'0) modify 254a23b9/benchmark_data_storage0111-ib.pocstor_35625_object18472/head//22 by client.940330.0:18473 2014-08-11 16:30:50.664448 2014-08-13 17:38:02.684436 7fe9bae887a0 20 read_log 192888'450 (0'0) modify b22e63b9/benchmark_data_storage0111-ib.pocstor_35625_object18627/head//22 by client.940330.0:18628 2014-08-11 16:30:51.233268 2014-08-13 17:38:02.684461 7fe9bae887a0 20 read_log 192888'451 (0'0) modify e132a3b9/benchmark_data_storage0111-ib.pocstor_35625_object19274/head//22 by client.940330.0:19275 2014-08-11 16:30:53.679987 2014-08-13 17:38:02.684481 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 620e8bb9/benchmark_data_storage0111-ib.pocstor_35625_object19312/head//22 by client.940330.0:19313 2014-08-11 16:30:53.824769 2014-08-13 17:38:02.684502 7fe9bae887a0 20 read_log 192888'453 (0'0) modify de36b3b9/benchmark_data_storage0111-ib.pocstor_35625_object20095/head//22 by client.940330.0:20096 2014-08-11 16:30:57.165253 2014-08-13 17:38:02.684523 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 8752b3b9/benchmark_data_storage0111-ib.pocstor_35625_object23292/head//22 by client.940330.0:23293 2014-08-11 16:31:09.484221 2014-08-13 17:38:02.684543 7fe9bae887a0 20 read_log 192888'455 (0'0) modify ec5a63b9/benchmark_data_storage0111-ib.pocstor_35625_object23813/head//22 by client.940330.0:23814 2014-08-11 16:31:11.466674 2014-08-13 17:38:02.684595 7fe9bae887a0 20 read_log 192888'456 (0'0) modify 39ac9bb9/benchmark_data_storage0111-ib.pocstor_35625_object24168/head//22 by client.940330.0:24169 2014-08-11 16:31:12.763904 2014-08-13 17:38:02.684617 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 452b53b9/benchmark_data_storage0111-ib.pocstor_35625_object31276/head//22 by client.940330.0:31277 2014-08-11 16:31:40.313662 2014-08-13 17:38:02.684638 7fe9bae887a0 20 read_log 192888'458 (192888'437) delete 738eabb9/benchmark_data_storage0111-ib.pocstor_35625_object2657/head//22 by client.940330.0:33965 2014-08-11 16:31:42.159731 2014-08-13 17:38:02.684659 7fe9bae887a0 20 read_log 192888'459 (192888'438) delete 9aea63b9/benchmark_data_storage0111-ib.pocstor_35625_object3811/head//22 by client.940330.0:35119 2014-08-11 16:31:42.476657 2014-08-13 17:38:02.684681 7fe9bae887a0 20 read_log 192888'460 (192888'439) delete b1ea83b9/benchmark_data_storage0111-ib.pocstor_35625_object4015/head//22 by client.940330.0:35323 2014-08-11 16:31:42.535797 2014-08-13 17:38:02.684715 7fe9bae887a0 20 read_log 192888'461 (192888'440) delete a8846bb9/benchmark_data_storage0111-ib.pocstor_35625_object4917/head//22 by client.940330.0:36225 2014-08-11 16:31:42.796438 2014-08-13 17:38:02.684742 7fe9bae887a0 20 read_log 192888'462 (192888'441) delete be9243b9/benchmark_data_storage0111-ib.pocstor_35625_object5330/head//22 by client.940330.0:36638 2014-08-11 16:31:42.899713 2014-08-13 17:38:02.684764 7fe9bae887a0 20 read_log 192888'463 (192888'442) delete ab2ee3b9/benchmark_data_storage0111-ib.pocstor_35625_object8334/head//22 by client.940330.0:39642 2014-08-11 16:31:43.674961 2014-08-13 17:38:02.684785 7fe9bae887a0 20 read_log 192888'464 (192888'443) delete 5c7df3b9/benchmark_data_storage0111-ib.pocstor_35625_object8868/head//22 by client.940330.0:40176 2014-08-11 16:31:43.809752 2014-08-13 17:38:02.684810 7fe9bae887a0 20 read_log 192888'465 (192888'444) delete cbdec3b9/benchmark_data_storage0111-ib.pocstor_35625_object9820/head//22 by client.940330.0:41128 2014-08-11 16:31:44.063966 2014-08-13 17:38:02.684832 7fe9bae887a0 20 read_log 192888'466 (192888'445) delete 9851f3b9/benchmark_data_storage0111-ib.pocstor_35625_object10015/head//22 by client.940330.0:41323 2014-08-11 16:31:44.118237 2014-08-13 17:38:02.684853 7fe9bae887a0 20 read_log 192888'467 (192888'446) delete 76ca73b9/benchmark_data_storage0111-ib.pocstor_35625_object10720/head//22 by client.940330.0:42028 2014-08-11 16:31:44.317599 2014-08-13 17:38:02.684874 7fe9bae887a0 20 read_log 192888'468 (192888'447) delete db73c3b9/benchmark_data_storage0111-ib.pocstor_35625_object11988/head//22 by client.940330.0:43296 2014-08-11 16:31:44.654943 2014-08-13 17:38:02.684895 7fe9bae887a0 20 read_log 192888'469 (192888'448) delete cf8063b9/benchmark_data_storage0111-ib.pocstor_35625_object12943/head//22 by client.940330.0:44251 2014-08-11 16:31:44.903657 2014-08-13 17:38:02.684916 7fe9bae887a0 20 read_log 192888'470 (192888'449) delete 254a23b9/benchmark_data_storage0111-ib.pocstor_35625_object18472/head//22 by client.940330.0:49780 2014-08-11 16:31:46.439613 2014-08-13 17:38:02.684937 7fe9bae887a0 20 read_log 192888'471 (192888'450) delete b22e63b9/benchmark_data_storage0111-ib.pocstor_35625_object18627/head//22 by client.940330.0:49935 2014-08-11 16:31:46.480119 2014-08-13 17:38:02.684958 7fe9bae887a0 20 read_log 192888'472 (192888'451) delete e132a3b9/benchmark_data_storage0111-ib.pocstor_35625_object19274/head//22 by client.940330.0:50582 2014-08-11 16:31:46.654929 2014-08-13 17:38:02.684981 7fe9bae887a0 20 read_log 192888'473 (192888'452) delete 620e8bb9/benchmark_data_storage0111-ib.pocstor_35625_object19312/head//22 by client.940330.0:50620 2014-08-11 16:31:46.666345 2014-08-13 17:38:02.685002 7fe9bae887a0 20 read_log 192888'474 (192888'453) delete de36b3b9/benchmark_data_storage0111-ib.pocstor_35625_object20095/head//22 by client.940330.0:51403 2014-08-11 16:31:46.908417 2014-08-13 17:38:02.685023 7fe9bae887a0 20 read_log 192888'475 (192888'454) delete 8752b3b9/benchmark_data_storage0111-ib.pocstor_35625_object23292/head//22 by client.940330.0:54600 2014-08-11 16:31:47.842640 2014-08-13 17:38:02.685044 7fe9bae887a0 20 read_log 192888'476 (192888'455) delete ec5a63b9/benchmark_data_storage0111-ib.pocstor_35625_object23813/head//22 by client.940330.0:55121 2014-08-11 16:31:47.980801 2014-08-13 17:38:02.685067 7fe9bae887a0 20 read_log 192888'477 (192888'456) delete 39ac9bb9/benchmark_data_storage0111-ib.pocstor_35625_object24168/head//22 by client.940330.0:55476 2014-08-11 16:31:48.091476 2014-08-13 17:38:02.685088 7fe9bae887a0 20 read_log 192888'478 (192888'457) delete 452b53b9/benchmark_data_storage0111-ib.pocstor_35625_object31276/head//22 by client.940330.0:62584 2014-08-11 16:31:50.144047 2014-08-13 17:38:02.685139 7fe9bae887a0 20 read_log 194215'479 (0'0) modify 6b5bebb9/benchmark_data_storage0111-ib.pocstor_64498_object2843/head//22 by client.1049635.0:2844 2014-08-12 17:17:26.266332 2014-08-13 17:38:02.685185 7fe9bae887a0 20 read_log 194215'480 (0'0) modify ddfd53b9/benchmark_data_storage0111-ib.pocstor_64498_object4265/head//22 by client.1049635.0:4266 2014-08-12 17:17:30.854922 2014-08-13 17:38:02.685213 7fe9bae887a0 20 read_log 194215'481 (0'0) modify b842abb9/benchmark_data_storage0111-ib.pocstor_64498_object5602/head//22 by client.1049635.0:5603 2014-08-12 17:17:34.907348 2014-08-13 17:38:02.685234 7fe9bae887a0 20 read_log 194215'482 (0'0) modify aa9f53b9/benchmark_data_storage0111-ib.pocstor_64498_object8634/head//22 by client.1049635.0:8635 2014-08-12 17:17:43.977204 2014-08-13 17:38:02.685255 7fe9bae887a0 20 read_log 194215'483 (0'0) modify fa5c43b9/benchmark_data_storage0111-ib.pocstor_64498_object9329/head//22 by client.1049635.0:9330 2014-08-12 17:17:45.975330 2014-08-13 17:38:02.685308 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 8eb7fbb9/benchmark_data_storage0111-ib.pocstor_64498_object10300/head//22 by client.1049635.0:10301 2014-08-12 17:17:48.781179 2014-08-13 17:38:02.685330 7fe9bae887a0 20 read_log 194215'485 (0'0) modify b0b14bb9/benchmark_data_storage0111-ib.pocstor_64498_object10614/head//22 by client.1049635.0:10615 2014-08-12 17:17:49.975104 2014-08-13 17:38:02.685351 7fe9bae887a0 20 read_log 194215'486 (0'0) modify d36a03b9/benchmark_data_storage0111-ib.pocstor_64498_object13616/head//22 by client.1049635.0:13617 2014-08-12 17:17:59.412824 2014-08-13 17:38:02.685372 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 58709bb9/benchmark_data_storage0111-ib.pocstor_64498_object16289/head//22 by client.1049635.0:16290 2014-08-12 17:18:08.654953 2014-08-13 17:38:02.685394 7fe9bae887a0 20 read_log 194215'488 (0'0) modify b64933b9/benchmark_data_storage0111-ib.pocstor_64498_object16505/head//22 by client.1049635.0:16506 2014-08-12 17:18:09.484027 2014-08-13 17:38:02.685415 7fe9bae887a0 20 read_log 194215'489 (0'0) modify ef739bb9/benchmark_data_storage0111-ib.pocstor_64498_object17678/head//22 by client.1049635.0:17679 2014-08-12 17:18:13.938898 2014-08-13 17:38:02.685435 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 80fdfbb9/benchmark_data_storage0111-ib.pocstor_64498_object22606/head//22 by client.1049635.0:22607 2014-08-12 17:18:32.668514 2014-08-13 17:38:02.685457 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 9dc7f3b9/benchmark_data_storage0111-ib.pocstor_64498_object25208/head//22 by client.1049635.0:25209 2014-08-12 17:18:42.485172 2014-08-13 17:38:02.685477 7fe9bae887a0 20 read_log 194215'492 (0'0) modify e53a93b9/benchmark_data_storage0111-ib.pocstor_64498_object27601/head//22 by client.1049635.0:27602 2014-08-12 17:18:51.393771 2014-08-13 17:38:02.685498 7fe9bae887a0 20 read_log 194215'493 (0'0) modify cd3df3b9/benchmark_data_storage0111-ib.pocstor_64498_object27690/head//22 by client.1049635.0:27691 2014-08-12 17:18:51.728081 2014-08-13 17:38:02.685520 7fe9bae887a0 20 read_log 194215'494 (0'0) modify e4e593b9/benchmark_data_storage0111-ib.pocstor_64498_object29725/head//22 by client.1049635.0:29726 2014-08-12 17:18:59.421720 2014-08-13 17:38:02.685541 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 3070a3b9/benchmark_data_storage0111-ib.pocstor_64498_object30382/head//22 by client.1049635.0:30383 2014-08-12 17:19:02.058064 2014-08-13 17:38:02.685562 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 65e683b9/benchmark_data_storage0111-ib.pocstor_64498_object31058/head//22 by client.1049635.0:31059 2014-08-12 17:19:04.786856 2014-08-13 17:38:02.685583 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 149c8bb9/benchmark_data_storage0111-ib.pocstor_64498_object33877/head//22 by client.1049635.0:33878 2014-08-12 17:19:15.720801 2014-08-13 17:38:02.685604 7fe9bae887a0 20 read_log 194215'498 (0'0) modify f3b8d3b9/benchmark_data_storage0111-ib.pocstor_64498_object33910/head//22 by client.1049635.0:33911 2014-08-12 17:19:15.841767 2014-08-13 17:38:02.685625 7fe9bae887a0 20 read_log 194215'499 (194215'479) delete 6b5bebb9/benchmark_data_storage0111-ib.pocstor_64498_object2843/head//22 by client.1049635.0:37046 2014-08-12 17:19:18.916625 2014-08-13 17:38:02.685662 7fe9bae887a0 20 read_log 194215'500 (194215'480) delete ddfd53b9/benchmark_data_storage0111-ib.pocstor_64498_object4265/head//22 by client.1049635.0:38468 2014-08-12 17:19:19.362711 2014-08-13 17:38:02.685687 7fe9bae887a0 20 read_log 194215'501 (194215'481) delete b842abb9/benchmark_data_storage0111-ib.pocstor_64498_object5602/head//22 by client.1049635.0:39805 2014-08-12 17:19:19.814380 2014-08-13 17:38:02.685708 7fe9bae887a0 20 read_log 194215'502 (194215'482) delete aa9f53b9/benchmark_data_storage0111-ib.pocstor_64498_object8634/head//22 by client.1049635.0:42837 2014-08-12 17:19:20.806279 2014-08-13 17:38:02.685729 7fe9bae887a0 20 read_log 194215'503 (194215'483) delete fa5c43b9/benchmark_data_storage0111-ib.pocstor_64498_object9329/head//22 by client.1049635.0:43532 2014-08-12 17:19:21.034170 2014-08-13 17:38:02.685750 7fe9bae887a0 20 read_log 194215'504 (194215'484) delete 8eb7fbb9/benchmark_data_storage0111-ib.pocstor_64498_object10300/head//22 by client.1049635.0:44503 2014-08-12 17:19:21.358604 2014-08-13 17:38:02.685771 7fe9bae887a0 20 read_log 194215'505 (194215'485) delete b0b14bb9/benchmark_data_storage0111-ib.pocstor_64498_object10614/head//22 by client.1049635.0:44817 2014-08-12 17:19:21.454425 2014-08-13 17:38:02.685792 7fe9bae887a0 20 read_log 194215'506 (194215'486) delete d36a03b9/benchmark_data_storage0111-ib.pocstor_64498_object13616/head//22 by client.1049635.0:47819 2014-08-12 17:19:22.444043 2014-08-13 17:38:02.685813 7fe9bae887a0 20 read_log 194215'507 (194215'487) delete 58709bb9/benchmark_data_storage0111-ib.pocstor_64498_object16289/head//22 by client.1049635.0:50492 2014-08-12 17:19:23.280667 2014-08-13 17:38:02.685840 7fe9bae887a0 20 read_log 194215'508 (194215'488) delete b64933b9/benchmark_data_storage0111-ib.pocstor_64498_object16505/head//22 by client.1049635.0:50708 2014-08-12 17:19:23.353655 2014-08-13 17:38:02.685862 7fe9bae887a0 20 read_log 194215'509 (194215'489) delete ef739bb9/benchmark_data_storage0111-ib.pocstor_64498_object17678/head//22 by client.1049635.0:51881 2014-08-12 17:19:23.721221 2014-08-13 17:38:02.685883 7fe9bae887a0 20 read_log 194215'510 (194215'490) delete 80fdfbb9/benchmark_data_storage0111-ib.pocstor_64498_object22606/head//22 by client.1049635.0:56809 2014-08-12 17:19:25.458512 2014-08-13 17:38:02.685904 7fe9bae887a0 20 read_log 194215'511 (194215'491) delete 9dc7f3b9/benchmark_data_storage0111-ib.pocstor_64498_object25208/head//22 by client.1049635.0:59411 2014-08-12 17:19:26.392107 2014-08-13 17:38:02.685925 7fe9bae887a0 20 read_log 194215'512 (194215'492) delete e53a93b9/benchmark_data_storage0111-ib.pocstor_64498_object27601/head//22 by client.1049635.0:61804 2014-08-12 17:19:27.213760 2014-08-13 17:38:02.685945 7fe9bae887a0 20 read_log 194215'513 (194215'493) delete cd3df3b9/benchmark_data_storage0111-ib.pocstor_64498_object27690/head//22 by client.1049635.0:61893 2014-08-12 17:19:27.241593 2014-08-13 17:38:02.685966 7fe9bae887a0 20 read_log 194215'514 (194215'494) delete e4e593b9/benchmark_data_storage0111-ib.pocstor_64498_object29725/head//22 by client.1049635.0:63928 2014-08-12 17:19:27.959511 2014-08-13 17:38:02.685987 7fe9bae887a0 20 read_log 194215'515 (194215'495) delete 3070a3b9/benchmark_data_storage0111-ib.pocstor_64498_object30382/head//22 by client.1049635.0:64585 2014-08-12 17:19:28.169382 2014-08-13 17:38:02.686048 7fe9bae887a0 20 read_log 194215'516 (194215'496) delete 65e683b9/benchmark_data_storage0111-ib.pocstor_64498_object31058/head//22 by client.1049635.0:65261 2014-08-12 17:19:28.400366 2014-08-13 17:38:02.686070 7fe9bae887a0 20 read_log 194215'517 (194215'497) delete 149c8bb9/benchmark_data_storage0111-ib.pocstor_64498_object33877/head//22 by client.1049635.0:68080 2014-08-12 17:19:29.433613 2014-08-13 17:38:02.686092 7fe9bae887a0 20 read_log 194215'518 (194215'498) delete f3b8d3b9/benchmark_data_storage0111-ib.pocstor_64498_object33910/head//22 by client.1049635.0:68113 2014-08-12 17:19:29.447013 2014-08-13 17:38:02.686278 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.686310 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=0 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.686337 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=0 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.003459 0 0.000000 2014-08-13 17:38:02.686358 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=0 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.686372 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=0 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.686386 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] log((0'0,194215'518], crt=192888'478) 2014-08-13 17:38:02.686403 7fe9bae887a0 10 osd.4 194478 pgid 22.3d5 coll 22.3d5_head 2014-08-13 17:38:02.686592 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3d5 2014-08-13 17:38:02.686607 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.686668 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3d5(unlocked)] enter Initial 2014-08-13 17:38:02.686687 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3d5(unlocked)] enter NotTrimming 2014-08-13 17:38:02.686862 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.687090 7fe9bae887a0 20 read_log 191648'3 (0'0) modify d6591bd5/benchmark_data_storage0111-ib.pocstor_40745_object136/head//22 by client.273158.0:137 2014-07-30 14:08:49.215673 2014-08-13 17:38:02.687125 7fe9bae887a0 20 read_log 191648'144 (0'0) modify deb973d5/benchmark_data_storage0111-ib.pocstor_40745_object8838/head//22 by client.273158.0:8839 2014-07-30 14:09:30.534650 2014-08-13 17:38:02.687148 7fe9bae887a0 20 read_log 191648'200 (0'0) modify 336bbbd5/benchmark_data_storage0111-ib.pocstor_40745_object12424/head//22 by client.273158.0:12425 2014-07-30 14:09:49.721435 2014-08-13 17:38:02.687181 7fe9bae887a0 20 read_log 191648'233 (0'0) modify a25f53d5/benchmark_data_storage0111-ib.pocstor_40745_object15186/head//22 by client.273158.0:15187 2014-07-30 14:10:03.428862 2014-08-13 17:38:02.687203 7fe9bae887a0 20 read_log 191648'269 (0'0) modify 6d90bbd5/benchmark_data_storage0111-ib.pocstor_40745_object17932/head//22 by client.273158.0:17933 2014-07-30 14:10:17.565080 2014-08-13 17:38:02.687224 7fe9bae887a0 20 read_log 191648'288 (0'0) modify 5b1cb3d5/benchmark_data_storage0111-ib.pocstor_40745_object19102/head//22 by client.273158.0:19103 2014-07-30 14:10:24.022907 2014-08-13 17:38:02.687245 7fe9bae887a0 20 read_log 191648'300 (0'0) modify 3cce4bd5/benchmark_data_storage0111-ib.pocstor_40745_object19916/head//22 by client.273158.0:19917 2014-07-30 14:10:30.988403 2014-08-13 17:38:02.687266 7fe9bae887a0 20 read_log 191648'347 (0'0) modify 7edc6bd5/benchmark_data_storage0111-ib.pocstor_40745_object22239/head//22 by client.273158.0:22240 2014-07-30 14:10:45.339247 2014-08-13 17:38:02.687287 7fe9bae887a0 20 read_log 191727'360 (0'0) modify adc4dbd5/benchmark_data_storage0111-ib.pocstor_51285_object1367/head//22 by client.275175.0:1368 2014-07-30 14:18:38.753795 2014-08-13 17:38:02.687322 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 16d2fbd5/benchmark_data_storage0111-ib.pocstor_51285_object2277/head//22 by client.275175.0:2278 2014-07-30 14:18:41.799399 2014-08-13 17:38:02.687348 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 73e533d5/benchmark_data_storage0111-ib.pocstor_51285_object3033/head//22 by client.275175.0:3034 2014-07-30 14:18:44.332016 2014-08-13 17:38:02.687370 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 95f50bd5/benchmark_data_storage0111-ib.pocstor_51285_object5286/head//22 by client.275175.0:5287 2014-07-30 14:18:51.196822 2014-08-13 17:38:02.687390 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 11de9bd5/benchmark_data_storage0111-ib.pocstor_51285_object7534/head//22 by client.275175.0:7535 2014-07-30 14:18:58.227507 2014-08-13 17:38:02.687411 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 43483bd5/benchmark_data_storage0111-ib.pocstor_51285_object8358/head//22 by client.275175.0:8359 2014-07-30 14:19:00.524356 2014-08-13 17:38:02.687432 7fe9bae887a0 20 read_log 191727'381 (0'0) modify 7abc73d5/benchmark_data_storage0111-ib.pocstor_51285_object10650/head//22 by client.275175.0:10651 2014-07-30 14:19:07.067114 2014-08-13 17:38:02.687453 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 43e9cbd5/benchmark_data_storage0111-ib.pocstor_51285_object11563/head//22 by client.275175.0:11564 2014-07-30 14:19:09.943863 2014-08-13 17:38:02.687474 7fe9bae887a0 20 read_log 191727'396 (0'0) modify ac0d8bd5/benchmark_data_storage0111-ib.pocstor_51285_object19412/head//22 by client.275175.0:19413 2014-07-30 14:19:34.536872 2014-08-13 17:38:02.687495 7fe9bae887a0 20 read_log 191727'404 (0'0) modify 7e0c53d5/benchmark_data_storage0111-ib.pocstor_51285_object25639/head//22 by client.275175.0:25640 2014-07-30 14:19:53.645674 2014-08-13 17:38:02.687515 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 6a8a7bd5/benchmark_data_storage0111-ib.pocstor_51285_object28386/head//22 by client.275175.0:28387 2014-07-30 14:20:02.102272 2014-08-13 17:38:02.687536 7fe9bae887a0 20 read_log 191727'410 (0'0) modify d24dfbd5/benchmark_data_storage0111-ib.pocstor_51285_object29132/head//22 by client.275175.0:29133 2014-07-30 14:20:04.308036 2014-08-13 17:38:02.687557 7fe9bae887a0 20 read_log 191727'412 (0'0) modify a92cbd5/benchmark_data_storage0111-ib.pocstor_51285_object29675/head//22 by client.275175.0:29676 2014-07-30 14:20:05.981278 2014-08-13 17:38:02.687578 7fe9bae887a0 20 read_log 191727'419 (0'0) modify bf621bd5/benchmark_data_storage0111-ib.pocstor_51285_object32066/head//22 by client.275175.0:32067 2014-07-30 14:20:13.038102 2014-08-13 17:38:02.687599 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 723913d5/benchmark_data_storage0111-ib.pocstor_51285_object35967/head//22 by client.275175.0:35968 2014-07-30 14:20:25.165793 2014-08-13 17:38:02.687619 7fe9bae887a0 20 read_log 191727'423 (0'0) modify 8cbf9bd5/benchmark_data_storage0111-ib.pocstor_51285_object36398/head//22 by client.275175.0:36399 2014-07-30 14:20:26.468067 2014-08-13 17:38:02.687640 7fe9bae887a0 20 read_log 191727'424 (0'0) modify b1d983d5/benchmark_data_storage0111-ib.pocstor_51285_object36492/head//22 by client.275175.0:36493 2014-07-30 14:20:26.797460 2014-08-13 17:38:02.687660 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 40865bd5/benchmark_data_storage0111-ib.pocstor_35625_object2794/head//22 by client.940330.0:2795 2014-08-11 16:29:51.639378 2014-08-13 17:38:02.687681 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 265433d5/benchmark_data_storage0111-ib.pocstor_35625_object9571/head//22 by client.940330.0:9572 2014-08-11 16:30:16.328912 2014-08-13 17:38:02.687701 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 293be3d5/benchmark_data_storage0111-ib.pocstor_35625_object11267/head//22 by client.940330.0:11268 2014-08-11 16:30:22.862199 2014-08-13 17:38:02.687722 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 580673d5/benchmark_data_storage0111-ib.pocstor_35625_object11489/head//22 by client.940330.0:11490 2014-08-11 16:30:23.750993 2014-08-13 17:38:02.687786 7fe9bae887a0 20 read_log 192888'431 (0'0) modify ab9a83d5/benchmark_data_storage0111-ib.pocstor_35625_object15695/head//22 by client.940330.0:15696 2014-08-11 16:30:40.073999 2014-08-13 17:38:02.687815 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 1f7d33d5/benchmark_data_storage0111-ib.pocstor_35625_object16402/head//22 by client.940330.0:16403 2014-08-11 16:30:42.733090 2014-08-13 17:38:02.687837 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 385ec3d5/benchmark_data_storage0111-ib.pocstor_35625_object16568/head//22 by client.940330.0:16569 2014-08-11 16:30:43.349925 2014-08-13 17:38:02.687857 7fe9bae887a0 20 read_log 192888'434 (0'0) modify d0f1fbd5/benchmark_data_storage0111-ib.pocstor_35625_object18645/head//22 by client.940330.0:18646 2014-08-11 16:30:51.305392 2014-08-13 17:38:02.687878 7fe9bae887a0 20 read_log 192888'435 (0'0) modify b0311bd5/benchmark_data_storage0111-ib.pocstor_35625_object25800/head//22 by client.940330.0:25801 2014-08-11 16:31:19.008561 2014-08-13 17:38:02.687899 7fe9bae887a0 20 read_log 192888'436 (0'0) modify c13143d5/benchmark_data_storage0111-ib.pocstor_35625_object27975/head//22 by client.940330.0:27976 2014-08-11 16:31:27.638150 2014-08-13 17:38:02.687920 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 8e8163d5/benchmark_data_storage0111-ib.pocstor_35625_object30712/head//22 by client.940330.0:30713 2014-08-11 16:31:38.141350 2014-08-13 17:38:02.687941 7fe9bae887a0 20 read_log 192888'438 (192888'427) delete 40865bd5/benchmark_data_storage0111-ib.pocstor_35625_object2794/head//22 by client.940330.0:34102 2014-08-11 16:31:42.194518 2014-08-13 17:38:02.687962 7fe9bae887a0 20 read_log 192888'439 (192888'428) delete 265433d5/benchmark_data_storage0111-ib.pocstor_35625_object9571/head//22 by client.940330.0:40879 2014-08-11 16:31:43.996919 2014-08-13 17:38:02.687983 7fe9bae887a0 20 read_log 192888'440 (192888'429) delete 293be3d5/benchmark_data_storage0111-ib.pocstor_35625_object11267/head//22 by client.940330.0:42575 2014-08-11 16:31:44.476987 2014-08-13 17:38:02.688004 7fe9bae887a0 20 read_log 192888'441 (192888'430) delete 580673d5/benchmark_data_storage0111-ib.pocstor_35625_object11489/head//22 by client.940330.0:42797 2014-08-11 16:31:44.537743 2014-08-13 17:38:02.688026 7fe9bae887a0 20 read_log 192888'442 (192888'431) delete ab9a83d5/benchmark_data_storage0111-ib.pocstor_35625_object15695/head//22 by client.940330.0:47003 2014-08-11 16:31:45.642692 2014-08-13 17:38:02.688046 7fe9bae887a0 20 read_log 192888'443 (192888'432) delete 1f7d33d5/benchmark_data_storage0111-ib.pocstor_35625_object16402/head//22 by client.940330.0:47710 2014-08-11 16:31:45.829864 2014-08-13 17:38:02.688067 7fe9bae887a0 20 read_log 192888'444 (192888'433) delete 385ec3d5/benchmark_data_storage0111-ib.pocstor_35625_object16568/head//22 by client.940330.0:47876 2014-08-11 16:31:45.873092 2014-08-13 17:38:02.688091 7fe9bae887a0 20 read_log 192888'445 (192888'434) delete d0f1fbd5/benchmark_data_storage0111-ib.pocstor_35625_object18645/head//22 by client.940330.0:49953 2014-08-11 16:31:46.485201 2014-08-13 17:38:02.688112 7fe9bae887a0 20 read_log 192888'446 (192888'435) delete b0311bd5/benchmark_data_storage0111-ib.pocstor_35625_object25800/head//22 by client.940330.0:57108 2014-08-11 16:31:48.588643 2014-08-13 17:38:02.688133 7fe9bae887a0 20 read_log 192888'447 (192888'436) delete c13143d5/benchmark_data_storage0111-ib.pocstor_35625_object27975/head//22 by client.940330.0:59283 2014-08-11 16:31:49.198758 2014-08-13 17:38:02.688162 7fe9bae887a0 20 read_log 192888'448 (192888'437) delete 8e8163d5/benchmark_data_storage0111-ib.pocstor_35625_object30712/head//22 by client.940330.0:62020 2014-08-11 16:31:49.964239 2014-08-13 17:38:02.688185 7fe9bae887a0 20 read_log 194215'449 (0'0) modify 8ae673d5/benchmark_data_storage0111-ib.pocstor_64498_object3640/head//22 by client.1049635.0:3641 2014-08-12 17:17:28.918286 2014-08-13 17:38:02.688223 7fe9bae887a0 20 read_log 194215'450 (0'0) modify df23abd5/benchmark_data_storage0111-ib.pocstor_64498_object12875/head//22 by client.1049635.0:12876 2014-08-12 17:17:57.007666 2014-08-13 17:38:02.688251 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 56eb3d5/benchmark_data_storage0111-ib.pocstor_64498_object15254/head//22 by client.1049635.0:15255 2014-08-12 17:18:04.817459 2014-08-13 17:38:02.688302 7fe9bae887a0 20 read_log 194215'452 (0'0) modify 6ba9dbd5/benchmark_data_storage0111-ib.pocstor_64498_object19389/head//22 by client.1049635.0:19390 2014-08-12 17:18:20.495989 2014-08-13 17:38:02.688329 7fe9bae887a0 20 read_log 194215'453 (0'0) modify e7db63d5/benchmark_data_storage0111-ib.pocstor_64498_object19764/head//22 by client.1049635.0:19765 2014-08-12 17:18:21.894548 2014-08-13 17:38:02.688351 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 572cfbd5/benchmark_data_storage0111-ib.pocstor_64498_object20280/head//22 by client.1049635.0:20281 2014-08-12 17:18:23.838974 2014-08-13 17:38:02.688372 7fe9bae887a0 20 read_log 194215'455 (0'0) modify e72ddbd5/benchmark_data_storage0111-ib.pocstor_64498_object22442/head//22 by client.1049635.0:22443 2014-08-12 17:18:32.005459 2014-08-13 17:38:02.688393 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 591063d5/benchmark_data_storage0111-ib.pocstor_64498_object30588/head//22 by client.1049635.0:30589 2014-08-12 17:19:02.831451 2014-08-13 17:38:02.688414 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 7d6343d5/benchmark_data_storage0111-ib.pocstor_64498_object32355/head//22 by client.1049635.0:32356 2014-08-12 17:19:09.938017 2014-08-13 17:38:02.688436 7fe9bae887a0 20 read_log 194215'458 (194215'449) delete 8ae673d5/benchmark_data_storage0111-ib.pocstor_64498_object3640/head//22 by client.1049635.0:37843 2014-08-12 17:19:19.162746 2014-08-13 17:38:02.688457 7fe9bae887a0 20 read_log 194215'459 (194215'450) delete df23abd5/benchmark_data_storage0111-ib.pocstor_64498_object12875/head//22 by client.1049635.0:47078 2014-08-12 17:19:22.174177 2014-08-13 17:38:02.688478 7fe9bae887a0 20 read_log 194215'460 (194215'451) delete 56eb3d5/benchmark_data_storage0111-ib.pocstor_64498_object15254/head//22 by client.1049635.0:49457 2014-08-12 17:19:22.954064 2014-08-13 17:38:02.688500 7fe9bae887a0 20 read_log 194215'461 (194215'452) delete 6ba9dbd5/benchmark_data_storage0111-ib.pocstor_64498_object19389/head//22 by client.1049635.0:53592 2014-08-12 17:19:24.313301 2014-08-13 17:38:02.688521 7fe9bae887a0 20 read_log 194215'462 (194215'453) delete e7db63d5/benchmark_data_storage0111-ib.pocstor_64498_object19764/head//22 by client.1049635.0:53967 2014-08-12 17:19:24.437265 2014-08-13 17:38:02.688541 7fe9bae887a0 20 read_log 194215'463 (194215'454) delete 572cfbd5/benchmark_data_storage0111-ib.pocstor_64498_object20280/head//22 by client.1049635.0:54483 2014-08-12 17:19:24.624134 2014-08-13 17:38:02.688562 7fe9bae887a0 20 read_log 194215'464 (194215'455) delete e72ddbd5/benchmark_data_storage0111-ib.pocstor_64498_object22442/head//22 by client.1049635.0:56645 2014-08-12 17:19:25.401091 2014-08-13 17:38:02.688583 7fe9bae887a0 20 read_log 194215'465 (194215'456) delete 591063d5/benchmark_data_storage0111-ib.pocstor_64498_object30588/head//22 by client.1049635.0:64791 2014-08-12 17:19:28.239253 2014-08-13 17:38:02.688604 7fe9bae887a0 20 read_log 194215'466 (194215'457) delete 7d6343d5/benchmark_data_storage0111-ib.pocstor_64498_object32355/head//22 by client.1049635.0:66558 2014-08-12 17:19:28.875006 2014-08-13 17:38:02.688704 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.688740 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.688768 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002101 0 0.000000 2014-08-13 17:38:02.688787 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.688800 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.688814 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'466], crt=191727'426) 2014-08-13 17:38:02.688831 7fe9bae887a0 10 osd.4 194478 pgid 22.3d9 coll 22.3d9_head 2014-08-13 17:38:02.689018 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3d9 2014-08-13 17:38:02.689034 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.689079 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3d9(unlocked)] enter Initial 2014-08-13 17:38:02.689098 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3d9(unlocked)] enter NotTrimming 2014-08-13 17:38:02.689346 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.689576 7fe9bae887a0 20 read_log 191648'8 (0'0) modify 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 by client.273158.0:822 2014-07-30 14:08:51.828841 2014-08-13 17:38:02.689612 7fe9bae887a0 20 read_log 191648'48 (0'0) modify 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 by client.273158.0:2771 2014-07-30 14:09:00.243066 2014-08-13 17:38:02.689634 7fe9bae887a0 20 read_log 191648'58 (0'0) modify bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 by client.273158.0:3174 2014-07-30 14:09:02.147864 2014-08-13 17:38:02.689656 7fe9bae887a0 20 read_log 191648'126 (0'0) modify a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 by client.273158.0:7307 2014-07-30 14:09:22.300377 2014-08-13 17:38:02.689677 7fe9bae887a0 20 read_log 191648'153 (0'0) modify 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 by client.273158.0:8984 2014-07-30 14:09:31.110735 2014-08-13 17:38:02.689698 7fe9bae887a0 20 read_log 191648'189 (0'0) modify 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 by client.273158.0:11165 2014-07-30 14:09:43.006434 2014-08-13 17:38:02.689719 7fe9bae887a0 20 read_log 191648'198 (0'0) modify a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 by client.273158.0:11654 2014-07-30 14:09:45.413047 2014-08-13 17:38:02.689740 7fe9bae887a0 20 read_log 191648'258 (0'0) modify 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 by client.273158.0:15056 2014-07-30 14:10:02.887398 2014-08-13 17:38:02.689761 7fe9bae887a0 20 read_log 191648'304 (0'0) modify 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 by client.273158.0:19107 2014-07-30 14:10:24.037122 2014-08-13 17:38:02.689782 7fe9bae887a0 20 read_log 191648'320 (0'0) modify e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 by client.273158.0:19927 2014-07-30 14:10:31.025716 2014-08-13 17:38:02.689803 7fe9bae887a0 20 read_log 191648'337 (0'0) modify b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 by client.273158.0:21174 2014-07-30 14:10:37.984880 2014-08-13 17:38:02.689826 7fe9bae887a0 20 read_log 191727'370 (0'0) modify 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 by client.275175.0:2604 2014-07-30 14:18:42.948238 2014-08-13 17:38:02.689848 7fe9bae887a0 20 read_log 191727'371 (0'0) modify e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 by client.275175.0:4312 2014-07-30 14:18:48.297703 2014-08-13 17:38:02.689869 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 by client.275175.0:5364 2014-07-30 14:18:51.420098 2014-08-13 17:38:02.689909 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 by client.275175.0:7043 2014-07-30 14:18:56.837569 2014-08-13 17:38:02.689932 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 by client.275175.0:12054 2014-07-30 14:19:11.790011 2014-08-13 17:38:02.689953 7fe9bae887a0 20 read_log 191727'390 (0'0) modify 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 by client.275175.0:13697 2014-07-30 14:19:16.795566 2014-08-13 17:38:02.689974 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 by client.275175.0:14177 2014-07-30 14:19:18.375565 2014-08-13 17:38:02.689994 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 by client.275175.0:17324 2014-07-30 14:19:28.057673 2014-08-13 17:38:02.690015 7fe9bae887a0 20 read_log 191727'404 (0'0) modify a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 by client.275175.0:21166 2014-07-30 14:19:39.599975 2014-08-13 17:38:02.690036 7fe9bae887a0 20 read_log 191727'405 (0'0) modify 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 by client.275175.0:21243 2014-07-30 14:19:39.841571 2014-08-13 17:38:02.690056 7fe9bae887a0 20 read_log 191727'411 (0'0) modify a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 by client.275175.0:24210 2014-07-30 14:19:49.416172 2014-08-13 17:38:02.690077 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 by client.275175.0:24640 2014-07-30 14:19:50.627549 2014-08-13 17:38:02.690098 7fe9bae887a0 20 read_log 191727'415 (0'0) modify b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 by client.275175.0:25870 2014-07-30 14:19:54.362210 2014-08-13 17:38:02.690120 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 by client.275175.0:29894 2014-07-30 14:20:06.646699 2014-08-13 17:38:02.690140 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 by client.275175.0:30423 2014-07-30 14:20:08.361387 2014-08-13 17:38:02.690176 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 by client.275175.0:30533 2014-07-30 14:20:08.668276 2014-08-13 17:38:02.690199 7fe9bae887a0 20 read_log 191727'441 (0'0) modify d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 by client.275175.0:38262 2014-07-30 14:20:31.988938 2014-08-13 17:38:02.690221 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 836a9bd9/benchmark_data_storage0111-ib.pocstor_35625_object1500/head//22 by client.940330.0:1501 2014-08-11 16:29:46.702668 2014-08-13 17:38:02.690241 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 877e03d9/benchmark_data_storage0111-ib.pocstor_35625_object4171/head//22 by client.940330.0:4172 2014-08-11 16:29:56.953485 2014-08-13 17:38:02.690262 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 2adb73d9/benchmark_data_storage0111-ib.pocstor_35625_object4984/head//22 by client.940330.0:4985 2014-08-11 16:29:59.977474 2014-08-13 17:38:02.690283 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 490d0bd9/benchmark_data_storage0111-ib.pocstor_35625_object7078/head//22 by client.940330.0:7079 2014-08-11 16:30:07.594504 2014-08-13 17:38:02.690304 7fe9bae887a0 20 read_log 192888'446 (0'0) modify d99333d9/benchmark_data_storage0111-ib.pocstor_35625_object7200/head//22 by client.940330.0:7201 2014-08-11 16:30:08.025142 2014-08-13 17:38:02.690380 7fe9bae887a0 20 read_log 192888'447 (0'0) modify aa7a2bd9/benchmark_data_storage0111-ib.pocstor_35625_object7219/head//22 by client.940330.0:7220 2014-08-11 16:30:08.092820 2014-08-13 17:38:02.690406 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 1a7dcbd9/benchmark_data_storage0111-ib.pocstor_35625_object9000/head//22 by client.940330.0:9001 2014-08-11 16:30:14.334130 2014-08-13 17:38:02.690427 7fe9bae887a0 20 read_log 192888'449 (0'0) modify 5f4fe3d9/benchmark_data_storage0111-ib.pocstor_35625_object11838/head//22 by client.940330.0:11839 2014-08-11 16:30:25.136512 2014-08-13 17:38:02.690449 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 179893d9/benchmark_data_storage0111-ib.pocstor_35625_object14597/head//22 by client.940330.0:14598 2014-08-11 16:30:35.993758 2014-08-13 17:38:02.690469 7fe9bae887a0 20 read_log 192888'451 (0'0) modify b00db3d9/benchmark_data_storage0111-ib.pocstor_35625_object14757/head//22 by client.940330.0:14758 2014-08-11 16:30:36.586978 2014-08-13 17:38:02.690490 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 81912bd9/benchmark_data_storage0111-ib.pocstor_35625_object16479/head//22 by client.940330.0:16480 2014-08-11 16:30:43.019375 2014-08-13 17:38:02.690511 7fe9bae887a0 20 read_log 192888'453 (0'0) modify aa4553d9/benchmark_data_storage0111-ib.pocstor_35625_object16599/head//22 by client.940330.0:16600 2014-08-11 16:30:43.467916 2014-08-13 17:38:02.690532 7fe9bae887a0 20 read_log 192888'454 (0'0) modify d753e3d9/benchmark_data_storage0111-ib.pocstor_35625_object18936/head//22 by client.940330.0:18937 2014-08-11 16:30:52.434070 2014-08-13 17:38:02.690552 7fe9bae887a0 20 read_log 192888'455 (0'0) modify 8e1a3bd9/benchmark_data_storage0111-ib.pocstor_35625_object19639/head//22 by client.940330.0:19640 2014-08-11 16:30:55.290566 2014-08-13 17:38:02.690572 7fe9bae887a0 20 read_log 192888'456 (0'0) modify f93ebbd9/benchmark_data_storage0111-ib.pocstor_35625_object20302/head//22 by client.940330.0:20303 2014-08-11 16:30:57.963829 2014-08-13 17:38:02.690594 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 301b4bd9/benchmark_data_storage0111-ib.pocstor_35625_object20450/head//22 by client.940330.0:20451 2014-08-11 16:30:58.511994 2014-08-13 17:38:02.690615 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 532d13d9/benchmark_data_storage0111-ib.pocstor_35625_object20472/head//22 by client.940330.0:20473 2014-08-11 16:30:58.593007 2014-08-13 17:38:02.690635 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 10a87bd9/benchmark_data_storage0111-ib.pocstor_35625_object24843/head//22 by client.940330.0:24844 2014-08-11 16:31:15.260307 2014-08-13 17:38:02.690658 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 336c5bd9/benchmark_data_storage0111-ib.pocstor_35625_object27982/head//22 by client.940330.0:27983 2014-08-11 16:31:27.664109 2014-08-13 17:38:02.690679 7fe9bae887a0 20 read_log 192888'461 (0'0) modify e94bcbd9/benchmark_data_storage0111-ib.pocstor_35625_object28206/head//22 by client.940330.0:28207 2014-08-11 16:31:28.530840 2014-08-13 17:38:02.690699 7fe9bae887a0 20 read_log 192888'462 (192888'442) delete 836a9bd9/benchmark_data_storage0111-ib.pocstor_35625_object1500/head//22 by client.940330.0:32808 2014-08-11 16:31:41.867513 2014-08-13 17:38:02.690721 7fe9bae887a0 20 read_log 192888'463 (192888'443) delete 877e03d9/benchmark_data_storage0111-ib.pocstor_35625_object4171/head//22 by client.940330.0:35479 2014-08-11 16:31:42.584379 2014-08-13 17:38:02.690742 7fe9bae887a0 20 read_log 192888'464 (192888'444) delete 2adb73d9/benchmark_data_storage0111-ib.pocstor_35625_object4984/head//22 by client.940330.0:36292 2014-08-11 16:31:42.813027 2014-08-13 17:38:02.690763 7fe9bae887a0 20 read_log 192888'465 (192888'445) delete 490d0bd9/benchmark_data_storage0111-ib.pocstor_35625_object7078/head//22 by client.940330.0:38386 2014-08-11 16:31:43.353768 2014-08-13 17:38:02.690784 7fe9bae887a0 20 read_log 192888'466 (192888'446) delete d99333d9/benchmark_data_storage0111-ib.pocstor_35625_object7200/head//22 by client.940330.0:38508 2014-08-11 16:31:43.385351 2014-08-13 17:38:02.690817 7fe9bae887a0 20 read_log 192888'467 (192888'447) delete aa7a2bd9/benchmark_data_storage0111-ib.pocstor_35625_object7219/head//22 by client.940330.0:38527 2014-08-11 16:31:43.390890 2014-08-13 17:38:02.690839 7fe9bae887a0 20 read_log 192888'468 (192888'448) delete 1a7dcbd9/benchmark_data_storage0111-ib.pocstor_35625_object9000/head//22 by client.940330.0:40308 2014-08-11 16:31:43.841388 2014-08-13 17:38:02.690860 7fe9bae887a0 20 read_log 192888'469 (192888'449) delete 5f4fe3d9/benchmark_data_storage0111-ib.pocstor_35625_object11838/head//22 by client.940330.0:43146 2014-08-11 16:31:44.617396 2014-08-13 17:38:02.690881 7fe9bae887a0 20 read_log 192888'470 (192888'450) delete 179893d9/benchmark_data_storage0111-ib.pocstor_35625_object14597/head//22 by client.940330.0:45905 2014-08-11 16:31:45.345671 2014-08-13 17:38:02.690905 7fe9bae887a0 20 read_log 192888'471 (192888'451) delete b00db3d9/benchmark_data_storage0111-ib.pocstor_35625_object14757/head//22 by client.940330.0:46065 2014-08-11 16:31:45.387938 2014-08-13 17:38:02.690927 7fe9bae887a0 20 read_log 192888'472 (192888'452) delete 81912bd9/benchmark_data_storage0111-ib.pocstor_35625_object16479/head//22 by client.940330.0:47787 2014-08-11 16:31:45.848768 2014-08-13 17:38:02.690947 7fe9bae887a0 20 read_log 192888'473 (192888'453) delete aa4553d9/benchmark_data_storage0111-ib.pocstor_35625_object16599/head//22 by client.940330.0:47907 2014-08-11 16:31:45.880783 2014-08-13 17:38:02.690968 7fe9bae887a0 20 read_log 192888'474 (192888'454) delete d753e3d9/benchmark_data_storage0111-ib.pocstor_35625_object18936/head//22 by client.940330.0:50244 2014-08-11 16:31:46.564270 2014-08-13 17:38:02.690989 7fe9bae887a0 20 read_log 192888'475 (192888'455) delete 8e1a3bd9/benchmark_data_storage0111-ib.pocstor_35625_object19639/head//22 by client.940330.0:50947 2014-08-11 16:31:46.759804 2014-08-13 17:38:02.691010 7fe9bae887a0 20 read_log 192888'476 (192888'456) delete f93ebbd9/benchmark_data_storage0111-ib.pocstor_35625_object20302/head//22 by client.940330.0:51610 2014-08-11 16:31:46.968713 2014-08-13 17:38:02.691031 7fe9bae887a0 20 read_log 192888'477 (192888'457) delete 301b4bd9/benchmark_data_storage0111-ib.pocstor_35625_object20450/head//22 by client.940330.0:51758 2014-08-11 16:31:47.009068 2014-08-13 17:38:02.691052 7fe9bae887a0 20 read_log 192888'478 (192888'458) delete 532d13d9/benchmark_data_storage0111-ib.pocstor_35625_object20472/head//22 by client.940330.0:51780 2014-08-11 16:31:47.014473 2014-08-13 17:38:02.691073 7fe9bae887a0 20 read_log 192888'479 (192888'459) delete 10a87bd9/benchmark_data_storage0111-ib.pocstor_35625_object24843/head//22 by client.940330.0:56151 2014-08-11 16:31:48.288935 2014-08-13 17:38:02.691094 7fe9bae887a0 20 read_log 192888'480 (192888'460) delete 336c5bd9/benchmark_data_storage0111-ib.pocstor_35625_object27982/head//22 by client.940330.0:59290 2014-08-11 16:31:49.199621 2014-08-13 17:38:02.691115 7fe9bae887a0 20 read_log 192888'481 (192888'461) delete e94bcbd9/benchmark_data_storage0111-ib.pocstor_35625_object28206/head//22 by client.940330.0:59514 2014-08-11 16:31:49.261201 2014-08-13 17:38:02.691136 7fe9bae887a0 20 read_log 194215'482 (0'0) modify b091e3d9/benchmark_data_storage0111-ib.pocstor_64498_object1407/head//22 by client.1049635.0:1408 2014-08-12 17:17:21.717610 2014-08-13 17:38:02.691165 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 660853d9/benchmark_data_storage0111-ib.pocstor_64498_object3877/head//22 by client.1049635.0:3878 2014-08-12 17:17:29.676304 2014-08-13 17:38:02.691188 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 5136d3d9/benchmark_data_storage0111-ib.pocstor_64498_object4412/head//22 by client.1049635.0:4413 2014-08-12 17:17:31.257814 2014-08-13 17:38:02.691209 7fe9bae887a0 20 read_log 194215'485 (0'0) modify 100ee3d9/benchmark_data_storage0111-ib.pocstor_64498_object6374/head//22 by client.1049635.0:6375 2014-08-12 17:17:37.201333 2014-08-13 17:38:02.691252 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 74b4cbd9/benchmark_data_storage0111-ib.pocstor_64498_object9876/head//22 by client.1049635.0:9877 2014-08-12 17:17:47.572381 2014-08-13 17:38:02.691279 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 34bc63d9/benchmark_data_storage0111-ib.pocstor_64498_object11189/head//22 by client.1049635.0:11190 2014-08-12 17:17:51.798215 2014-08-13 17:38:02.691301 7fe9bae887a0 20 read_log 194215'488 (0'0) modify a7758bd9/benchmark_data_storage0111-ib.pocstor_64498_object15202/head//22 by client.1049635.0:15203 2014-08-12 17:18:04.645181 2014-08-13 17:38:02.691322 7fe9bae887a0 20 read_log 194215'489 (0'0) modify ebc083d9/benchmark_data_storage0111-ib.pocstor_64498_object15324/head//22 by client.1049635.0:15325 2014-08-12 17:18:05.053228 2014-08-13 17:38:02.691344 7fe9bae887a0 20 read_log 194215'490 (0'0) modify ecd3abd9/benchmark_data_storage0111-ib.pocstor_64498_object15674/head//22 by client.1049635.0:15675 2014-08-12 17:18:06.353886 2014-08-13 17:38:02.691396 7fe9bae887a0 20 read_log 194215'491 (0'0) modify fdafebd9/benchmark_data_storage0111-ib.pocstor_64498_object16076/head//22 by client.1049635.0:16077 2014-08-12 17:18:07.838477 2014-08-13 17:38:02.691423 7fe9bae887a0 20 read_log 194215'492 (0'0) modify cfb0cbd9/benchmark_data_storage0111-ib.pocstor_64498_object20689/head//22 by client.1049635.0:20690 2014-08-12 17:18:25.370515 2014-08-13 17:38:02.691445 7fe9bae887a0 20 read_log 194215'493 (0'0) modify f9f693d9/benchmark_data_storage0111-ib.pocstor_64498_object20727/head//22 by client.1049635.0:20728 2014-08-12 17:18:25.508224 2014-08-13 17:38:02.691466 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 44d9b3d9/benchmark_data_storage0111-ib.pocstor_64498_object23494/head//22 by client.1049635.0:23495 2014-08-12 17:18:35.969397 2014-08-13 17:38:02.691487 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 97346bd9/benchmark_data_storage0111-ib.pocstor_64498_object29753/head//22 by client.1049635.0:29754 2014-08-12 17:18:59.523995 2014-08-13 17:38:02.691508 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:30300 2014-08-12 17:19:01.749852 2014-08-13 17:38:02.691529 7fe9bae887a0 20 read_log 194215'497 (194215'482) delete b091e3d9/benchmark_data_storage0111-ib.pocstor_64498_object1407/head//22 by client.1049635.0:35610 2014-08-12 17:19:18.454089 2014-08-13 17:38:02.691551 7fe9bae887a0 20 read_log 194215'498 (194215'483) delete 660853d9/benchmark_data_storage0111-ib.pocstor_64498_object3877/head//22 by client.1049635.0:38080 2014-08-12 17:19:19.237653 2014-08-13 17:38:02.691572 7fe9bae887a0 20 read_log 194215'499 (194215'484) delete 5136d3d9/benchmark_data_storage0111-ib.pocstor_64498_object4412/head//22 by client.1049635.0:38615 2014-08-12 17:19:19.408602 2014-08-13 17:38:02.691593 7fe9bae887a0 20 read_log 194215'500 (194215'485) delete 100ee3d9/benchmark_data_storage0111-ib.pocstor_64498_object6374/head//22 by client.1049635.0:40577 2014-08-12 17:19:20.077232 2014-08-13 17:38:02.691614 7fe9bae887a0 20 read_log 194215'501 (194215'486) delete 74b4cbd9/benchmark_data_storage0111-ib.pocstor_64498_object9876/head//22 by client.1049635.0:44079 2014-08-12 17:19:21.216186 2014-08-13 17:38:02.691639 7fe9bae887a0 20 read_log 194215'502 (194215'487) delete 34bc63d9/benchmark_data_storage0111-ib.pocstor_64498_object11189/head//22 by client.1049635.0:45392 2014-08-12 17:19:21.637715 2014-08-13 17:38:02.691662 7fe9bae887a0 20 read_log 194215'503 (194215'488) delete a7758bd9/benchmark_data_storage0111-ib.pocstor_64498_object15202/head//22 by client.1049635.0:49405 2014-08-12 17:19:22.937262 2014-08-13 17:38:02.691683 7fe9bae887a0 20 read_log 194215'504 (194215'489) delete ebc083d9/benchmark_data_storage0111-ib.pocstor_64498_object15324/head//22 by client.1049635.0:49527 2014-08-12 17:19:22.974200 2014-08-13 17:38:02.691704 7fe9bae887a0 20 read_log 194215'505 (194215'490) delete ecd3abd9/benchmark_data_storage0111-ib.pocstor_64498_object15674/head//22 by client.1049635.0:49877 2014-08-12 17:19:23.086653 2014-08-13 17:38:02.692041 7fe9bae887a0 20 read_log 194215'506 (194215'491) delete fdafebd9/benchmark_data_storage0111-ib.pocstor_64498_object16076/head//22 by client.1049635.0:50279 2014-08-12 17:19:23.212421 2014-08-13 17:38:02.692057 7fe9bae887a0 20 read_log 194215'507 (194215'492) delete cfb0cbd9/benchmark_data_storage0111-ib.pocstor_64498_object20689/head//22 by client.1049635.0:54892 2014-08-12 17:19:24.770872 2014-08-13 17:38:02.692071 7fe9bae887a0 20 read_log 194215'508 (194215'493) delete f9f693d9/benchmark_data_storage0111-ib.pocstor_64498_object20727/head//22 by client.1049635.0:54930 2014-08-12 17:19:24.784831 2014-08-13 17:38:02.692084 7fe9bae887a0 20 read_log 194215'509 (194215'494) delete 44d9b3d9/benchmark_data_storage0111-ib.pocstor_64498_object23494/head//22 by client.1049635.0:57697 2014-08-12 17:19:25.796840 2014-08-13 17:38:02.692096 7fe9bae887a0 20 read_log 194215'510 (194215'495) delete 97346bd9/benchmark_data_storage0111-ib.pocstor_64498_object29753/head//22 by client.1049635.0:63956 2014-08-12 17:19:27.970049 2014-08-13 17:38:02.692110 7fe9bae887a0 20 read_log 194215'511 (194215'496) delete 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:64502 2014-08-12 17:19:28.143258 2014-08-13 17:38:02.692213 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.692232 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=0 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.692249 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=0 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003171 0 0.000000 2014-08-13 17:38:02.692261 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=0 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.692269 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=0 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.692278 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'511], crt=191727'441) 2014-08-13 17:38:02.692287 7fe9bae887a0 10 osd.4 194478 pgid 22.3e4 coll 22.3e4_head 2014-08-13 17:38:02.692414 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.3e4 2014-08-13 17:38:02.692424 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.692450 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3e4(unlocked)] enter Initial 2014-08-13 17:38:02.692461 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3e4(unlocked)] enter NotTrimming 2014-08-13 17:38:02.692579 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.692701 7fe9bae887a0 20 read_log 191648'13 (0'0) modify ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 by client.273158.0:660 2014-07-30 14:08:51.236138 2014-08-13 17:38:02.692721 7fe9bae887a0 20 read_log 191648'20 (0'0) modify c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 by client.273158.0:1374 2014-07-30 14:08:53.839019 2014-08-13 17:38:02.692734 7fe9bae887a0 20 read_log 191648'118 (0'0) modify 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 by client.273158.0:7408 2014-07-30 14:09:22.807776 2014-08-13 17:38:02.692747 7fe9bae887a0 20 read_log 191648'158 (0'0) modify e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 by client.273158.0:9202 2014-07-30 14:09:31.967465 2014-08-13 17:38:02.692759 7fe9bae887a0 20 read_log 191648'196 (0'0) modify 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 by client.273158.0:11828 2014-07-30 14:09:46.186522 2014-08-13 17:38:02.692771 7fe9bae887a0 20 read_log 191648'215 (0'0) modify 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 by client.273158.0:13103 2014-07-30 14:09:52.930745 2014-08-13 17:38:02.692783 7fe9bae887a0 20 read_log 191648'261 (0'0) modify f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 by client.273158.0:16500 2014-07-30 14:10:10.181943 2014-08-13 17:38:02.692794 7fe9bae887a0 20 read_log 191648'268 (0'0) modify 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 by client.273158.0:16897 2014-07-30 14:10:12.167252 2014-08-13 17:38:02.692806 7fe9bae887a0 20 read_log 191648'269 (0'0) modify e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 by client.273158.0:16910 2014-07-30 14:10:12.323702 2014-08-13 17:38:02.692818 7fe9bae887a0 20 read_log 191648'295 (0'0) modify 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 by client.273158.0:18817 2014-07-30 14:10:22.815945 2014-08-13 17:38:02.692830 7fe9bae887a0 20 read_log 191727'370 (0'0) modify e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 by client.275175.0:171 2014-07-30 14:18:34.112158 2014-08-13 17:38:02.692842 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 by client.275175.0:850 2014-07-30 14:18:36.863512 2014-08-13 17:38:02.692854 7fe9bae887a0 20 read_log 191727'375 (0'0) modify a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 by client.275175.0:2335 2014-07-30 14:18:41.990442 2014-08-13 17:38:02.692865 7fe9bae887a0 20 read_log 191727'379 (0'0) modify b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 by client.275175.0:3618 2014-07-30 14:18:46.163538 2014-08-13 17:38:02.692877 7fe9bae887a0 20 read_log 191727'380 (0'0) modify e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 by client.275175.0:3871 2014-07-30 14:18:46.957245 2014-08-13 17:38:02.692889 7fe9bae887a0 20 read_log 191727'387 (0'0) modify 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 by client.275175.0:7584 2014-07-30 14:18:58.364406 2014-08-13 17:38:02.692901 7fe9bae887a0 20 read_log 191727'390 (0'0) modify 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 by client.275175.0:8185 2014-07-30 14:19:00.035853 2014-08-13 17:38:02.692913 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 by client.275175.0:8512 2014-07-30 14:19:00.959329 2014-08-13 17:38:02.692933 7fe9bae887a0 20 read_log 191727'393 (0'0) modify 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 by client.275175.0:9117 2014-07-30 14:19:02.682215 2014-08-13 17:38:02.692945 7fe9bae887a0 20 read_log 191727'399 (0'0) modify c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 by client.275175.0:12240 2014-07-30 14:19:12.310416 2014-08-13 17:38:02.692957 7fe9bae887a0 20 read_log 191727'401 (0'0) modify fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 by client.275175.0:13471 2014-07-30 14:19:16.161477 2014-08-13 17:38:02.692969 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 by client.275175.0:13496 2014-07-30 14:19:16.233859 2014-08-13 17:38:02.692981 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 by client.275175.0:14901 2014-07-30 14:19:20.424930 2014-08-13 17:38:02.692993 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 by client.275175.0:18558 2014-07-30 14:19:31.647507 2014-08-13 17:38:02.693005 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 by client.275175.0:21790 2014-07-30 14:19:41.808153 2014-08-13 17:38:02.693017 7fe9bae887a0 20 read_log 191727'419 (0'0) modify e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 by client.275175.0:23147 2014-07-30 14:19:46.435420 2014-08-13 17:38:02.693032 7fe9bae887a0 20 read_log 191727'420 (0'0) modify fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 by client.275175.0:24195 2014-07-30 14:19:49.374369 2014-08-13 17:38:02.693044 7fe9bae887a0 20 read_log 191727'427 (0'0) modify b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 by client.275175.0:31110 2014-07-30 14:20:10.294807 2014-08-13 17:38:02.693059 7fe9bae887a0 20 read_log 191727'437 (0'0) modify 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 by client.275175.0:37692 2014-07-30 14:20:30.380147 2014-08-13 17:38:02.693074 7fe9bae887a0 20 read_log 191727'439 (0'0) modify 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 by client.275175.0:38662 2014-07-30 14:20:33.130508 2014-08-13 17:38:02.693086 7fe9bae887a0 20 read_log 191727'442 (0'0) modify a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 by client.275175.0:38712 2014-07-30 14:20:33.271825 2014-08-13 17:38:02.693100 7fe9bae887a0 20 read_log 192674'443 (0'0) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2663 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693112 7fe9bae887a0 20 read_log 192674'444 (192674'443) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2664 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693126 7fe9bae887a0 20 read_log 192674'445 (192674'444) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2665 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693138 7fe9bae887a0 20 read_log 192674'446 (192674'445) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2666 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693150 7fe9bae887a0 20 read_log 192674'447 (192674'446) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2667 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693171 7fe9bae887a0 20 read_log 192674'448 (192674'447) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2668 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693183 7fe9bae887a0 20 read_log 192674'449 (192674'448) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2669 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693203 7fe9bae887a0 20 read_log 192674'450 (192674'449) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2670 2014-07-31 18:03:44.524000 2014-08-13 17:38:02.693216 7fe9bae887a0 20 read_log 192888'451 (0'0) modify ab1eabe4/benchmark_data_storage0111-ib.pocstor_35625_object252/head//22 by client.940330.0:253 2014-08-11 16:29:41.724807 2014-08-13 17:38:02.693230 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 251db3e4/benchmark_data_storage0111-ib.pocstor_35625_object1039/head//22 by client.940330.0:1040 2014-08-11 16:29:44.939051 2014-08-13 17:38:02.693247 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 18092be4/benchmark_data_storage0111-ib.pocstor_35625_object2883/head//22 by client.940330.0:2884 2014-08-11 16:29:52.154787 2014-08-13 17:38:02.693259 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 294b1be4/benchmark_data_storage0111-ib.pocstor_35625_object3206/head//22 by client.940330.0:3207 2014-08-11 16:29:53.356262 2014-08-13 17:38:02.693271 7fe9bae887a0 20 read_log 192888'455 (0'0) modify a86353e4/benchmark_data_storage0111-ib.pocstor_35625_object4168/head//22 by client.940330.0:4169 2014-08-11 16:29:56.942276 2014-08-13 17:38:02.693285 7fe9bae887a0 20 read_log 192888'456 (0'0) modify df38d3e4/benchmark_data_storage0111-ib.pocstor_35625_object6828/head//22 by client.940330.0:6829 2014-08-11 16:30:06.715874 2014-08-13 17:38:02.693297 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 3a3d9be4/benchmark_data_storage0111-ib.pocstor_35625_object8496/head//22 by client.940330.0:8497 2014-08-11 16:30:12.573964 2014-08-13 17:38:02.693309 7fe9bae887a0 20 read_log 192888'458 (0'0) modify b860b3e4/benchmark_data_storage0111-ib.pocstor_35625_object10286/head//22 by client.940330.0:10287 2014-08-11 16:30:18.908661 2014-08-13 17:38:02.693321 7fe9bae887a0 20 read_log 192888'459 (0'0) modify c7e76be4/benchmark_data_storage0111-ib.pocstor_35625_object13242/head//22 by client.940330.0:13243 2014-08-11 16:30:30.826113 2014-08-13 17:38:02.693333 7fe9bae887a0 20 read_log 192888'460 (0'0) modify b1987be4/benchmark_data_storage0111-ib.pocstor_35625_object14257/head//22 by client.940330.0:14258 2014-08-11 16:30:34.721306 2014-08-13 17:38:02.693345 7fe9bae887a0 20 read_log 192888'461 (0'0) modify cf3f43e4/benchmark_data_storage0111-ib.pocstor_35625_object14945/head//22 by client.940330.0:14946 2014-08-11 16:30:37.287712 2014-08-13 17:38:02.693357 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 6e1813e4/benchmark_data_storage0111-ib.pocstor_35625_object15096/head//22 by client.940330.0:15097 2014-08-11 16:30:37.849964 2014-08-13 17:38:02.693369 7fe9bae887a0 20 read_log 192888'463 (0'0) modify f136abe4/benchmark_data_storage0111-ib.pocstor_35625_object17411/head//22 by client.940330.0:17412 2014-08-11 16:30:46.501450 2014-08-13 17:38:02.693381 7fe9bae887a0 20 read_log 192888'464 (0'0) modify a12053e4/benchmark_data_storage0111-ib.pocstor_35625_object17529/head//22 by client.940330.0:17530 2014-08-11 16:30:46.935071 2014-08-13 17:38:02.693393 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 56f9fbe4/benchmark_data_storage0111-ib.pocstor_35625_object19448/head//22 by client.940330.0:19449 2014-08-11 16:30:54.437749 2014-08-13 17:38:02.693405 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 548183e4/benchmark_data_storage0111-ib.pocstor_35625_object21478/head//22 by client.940330.0:21479 2014-08-11 16:31:02.598556 2014-08-13 17:38:02.693417 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 5bd8be4/benchmark_data_storage0111-ib.pocstor_35625_object22736/head//22 by client.940330.0:22737 2014-08-11 16:31:07.432192 2014-08-13 17:38:02.693429 7fe9bae887a0 20 read_log 192888'468 (0'0) modify ae923e4/benchmark_data_storage0111-ib.pocstor_35625_object23845/head//22 by client.940330.0:23846 2014-08-11 16:31:11.584553 2014-08-13 17:38:02.693441 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 72aa3e4/benchmark_data_storage0111-ib.pocstor_35625_object23952/head//22 by client.940330.0:23953 2014-08-11 16:31:11.975897 2014-08-13 17:38:02.693460 7fe9bae887a0 20 read_log 192888'470 (0'0) modify 9a123be4/benchmark_data_storage0111-ib.pocstor_35625_object29395/head//22 by client.940330.0:29396 2014-08-11 16:31:33.173709 2014-08-13 17:38:02.693475 7fe9bae887a0 20 read_log 192888'471 (0'0) modify 9b4fbe4/benchmark_data_storage0111-ib.pocstor_35625_object30551/head//22 by client.940330.0:30552 2014-08-11 16:31:37.543505 2014-08-13 17:38:02.693489 7fe9bae887a0 20 read_log 192888'472 (192888'451) delete ab1eabe4/benchmark_data_storage0111-ib.pocstor_35625_object252/head//22 by client.940330.0:31560 2014-08-11 16:31:41.545995 2014-08-13 17:38:02.693502 7fe9bae887a0 20 read_log 192888'473 (192888'452) delete 251db3e4/benchmark_data_storage0111-ib.pocstor_35625_object1039/head//22 by client.940330.0:32347 2014-08-11 16:31:41.758561 2014-08-13 17:38:02.693514 7fe9bae887a0 20 read_log 192888'474 (192888'453) delete 18092be4/benchmark_data_storage0111-ib.pocstor_35625_object2883/head//22 by client.940330.0:34191 2014-08-11 16:31:42.221088 2014-08-13 17:38:02.693526 7fe9bae887a0 20 read_log 192888'475 (192888'454) delete 294b1be4/benchmark_data_storage0111-ib.pocstor_35625_object3206/head//22 by client.940330.0:34514 2014-08-11 16:31:42.310168 2014-08-13 17:38:02.693538 7fe9bae887a0 20 read_log 192888'476 (192888'455) delete a86353e4/benchmark_data_storage0111-ib.pocstor_35625_object4168/head//22 by client.940330.0:35476 2014-08-11 16:31:42.584017 2014-08-13 17:38:02.693550 7fe9bae887a0 20 read_log 192888'477 (192888'456) delete df38d3e4/benchmark_data_storage0111-ib.pocstor_35625_object6828/head//22 by client.940330.0:38136 2014-08-11 16:31:43.294875 2014-08-13 17:38:02.693562 7fe9bae887a0 20 read_log 192888'478 (192888'457) delete 3a3d9be4/benchmark_data_storage0111-ib.pocstor_35625_object8496/head//22 by client.940330.0:39804 2014-08-11 16:31:43.715291 2014-08-13 17:38:02.693574 7fe9bae887a0 20 read_log 192888'479 (192888'458) delete b860b3e4/benchmark_data_storage0111-ib.pocstor_35625_object10286/head//22 by client.940330.0:41594 2014-08-11 16:31:44.195254 2014-08-13 17:38:02.693586 7fe9bae887a0 20 read_log 192888'480 (192888'459) delete c7e76be4/benchmark_data_storage0111-ib.pocstor_35625_object13242/head//22 by client.940330.0:44550 2014-08-11 16:31:44.987190 2014-08-13 17:38:02.693621 7fe9bae887a0 20 read_log 192888'481 (192888'460) delete b1987be4/benchmark_data_storage0111-ib.pocstor_35625_object14257/head//22 by client.940330.0:45565 2014-08-11 16:31:45.249495 2014-08-13 17:38:02.693634 7fe9bae887a0 20 read_log 192888'482 (192888'461) delete cf3f43e4/benchmark_data_storage0111-ib.pocstor_35625_object14945/head//22 by client.940330.0:46253 2014-08-11 16:31:45.441438 2014-08-13 17:38:02.693646 7fe9bae887a0 20 read_log 192888'483 (192888'462) delete 6e1813e4/benchmark_data_storage0111-ib.pocstor_35625_object15096/head//22 by client.940330.0:46404 2014-08-11 16:31:45.481628 2014-08-13 17:38:02.693659 7fe9bae887a0 20 read_log 192888'484 (192888'463) delete f136abe4/benchmark_data_storage0111-ib.pocstor_35625_object17411/head//22 by client.940330.0:48719 2014-08-11 16:31:46.105085 2014-08-13 17:38:02.693671 7fe9bae887a0 20 read_log 192888'485 (192888'464) delete a12053e4/benchmark_data_storage0111-ib.pocstor_35625_object17529/head//22 by client.940330.0:48837 2014-08-11 16:31:46.135320 2014-08-13 17:38:02.693683 7fe9bae887a0 20 read_log 192888'486 (192888'465) delete 56f9fbe4/benchmark_data_storage0111-ib.pocstor_35625_object19448/head//22 by client.940330.0:50756 2014-08-11 16:31:46.704365 2014-08-13 17:38:02.693696 7fe9bae887a0 20 read_log 192888'487 (192888'466) delete 548183e4/benchmark_data_storage0111-ib.pocstor_35625_object21478/head//22 by client.940330.0:52786 2014-08-11 16:31:47.289428 2014-08-13 17:38:02.693708 7fe9bae887a0 20 read_log 192888'488 (192888'467) delete 5bd8be4/benchmark_data_storage0111-ib.pocstor_35625_object22736/head//22 by client.940330.0:54044 2014-08-11 16:31:47.679853 2014-08-13 17:38:02.693720 7fe9bae887a0 20 read_log 192888'489 (192888'468) delete ae923e4/benchmark_data_storage0111-ib.pocstor_35625_object23845/head//22 by client.940330.0:55153 2014-08-11 16:31:47.988873 2014-08-13 17:38:02.693739 7fe9bae887a0 20 read_log 192888'490 (192888'469) delete 72aa3e4/benchmark_data_storage0111-ib.pocstor_35625_object23952/head//22 by client.940330.0:55260 2014-08-11 16:31:48.022299 2014-08-13 17:38:02.693752 7fe9bae887a0 20 read_log 192888'491 (192888'470) delete 9a123be4/benchmark_data_storage0111-ib.pocstor_35625_object29395/head//22 by client.940330.0:60703 2014-08-11 16:31:49.583299 2014-08-13 17:38:02.693784 7fe9bae887a0 20 read_log 192888'492 (192888'471) delete 9b4fbe4/benchmark_data_storage0111-ib.pocstor_35625_object30551/head//22 by client.940330.0:61859 2014-08-11 16:31:49.915067 2014-08-13 17:38:02.693796 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 6ea8be4/benchmark_data_storage0111-ib.pocstor_64498_object3003/head//22 by client.1049635.0:3004 2014-08-12 17:17:26.852274 2014-08-13 17:38:02.693808 7fe9bae887a0 20 read_log 194215'494 (0'0) modify d902e3e4/benchmark_data_storage0111-ib.pocstor_64498_object3395/head//22 by client.1049635.0:3396 2014-08-12 17:17:28.190801 2014-08-13 17:38:02.693820 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 62240be4/benchmark_data_storage0111-ib.pocstor_64498_object3504/head//22 by client.1049635.0:3505 2014-08-12 17:17:28.515281 2014-08-13 17:38:02.693832 7fe9bae887a0 20 read_log 194215'496 (0'0) modify b085f3e4/benchmark_data_storage0111-ib.pocstor_64498_object5535/head//22 by client.1049635.0:5536 2014-08-12 17:17:34.716840 2014-08-13 17:38:02.693844 7fe9bae887a0 20 read_log 194215'497 (0'0) modify b89993e4/benchmark_data_storage0111-ib.pocstor_64498_object11414/head//22 by client.1049635.0:11415 2014-08-12 17:17:52.564972 2014-08-13 17:38:02.693856 7fe9bae887a0 20 read_log 194215'498 (0'0) modify d53a4be4/benchmark_data_storage0111-ib.pocstor_64498_object14828/head//22 by client.1049635.0:14829 2014-08-12 17:18:03.413077 2014-08-13 17:38:02.693868 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 55eb0be4/benchmark_data_storage0111-ib.pocstor_64498_object20658/head//22 by client.1049635.0:20659 2014-08-12 17:18:25.253171 2014-08-13 17:38:02.693880 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 8cf67be4/benchmark_data_storage0111-ib.pocstor_64498_object21331/head//22 by client.1049635.0:21332 2014-08-12 17:18:27.827817 2014-08-13 17:38:02.693892 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 37bc5be4/benchmark_data_storage0111-ib.pocstor_64498_object22829/head//22 by client.1049635.0:22830 2014-08-12 17:18:33.499888 2014-08-13 17:38:02.693904 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 1d0f93e4/benchmark_data_storage0111-ib.pocstor_64498_object26258/head//22 by client.1049635.0:26259 2014-08-12 17:18:46.382980 2014-08-13 17:38:02.693916 7fe9bae887a0 20 read_log 194215'503 (0'0) modify ade7abe4/benchmark_data_storage0111-ib.pocstor_64498_object27199/head//22 by client.1049635.0:27200 2014-08-12 17:18:49.891666 2014-08-13 17:38:02.693928 7fe9bae887a0 20 read_log 194215'504 (0'0) modify 982fdbe4/benchmark_data_storage0111-ib.pocstor_64498_object33594/head//22 by client.1049635.0:33595 2014-08-12 17:19:14.641389 2014-08-13 17:38:02.693939 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 59e2a3e4/benchmark_data_storage0111-ib.pocstor_64498_object33849/head//22 by client.1049635.0:33850 2014-08-12 17:19:15.608005 2014-08-13 17:38:02.693952 7fe9bae887a0 20 read_log 194215'506 (194215'493) delete 6ea8be4/benchmark_data_storage0111-ib.pocstor_64498_object3003/head//22 by client.1049635.0:37206 2014-08-12 17:19:18.970763 2014-08-13 17:38:02.693964 7fe9bae887a0 20 read_log 194215'507 (194215'494) delete d902e3e4/benchmark_data_storage0111-ib.pocstor_64498_object3395/head//22 by client.1049635.0:37598 2014-08-12 17:19:19.087637 2014-08-13 17:38:02.693976 7fe9bae887a0 20 read_log 194215'508 (194215'495) delete 62240be4/benchmark_data_storage0111-ib.pocstor_64498_object3504/head//22 by client.1049635.0:37707 2014-08-12 17:19:19.120739 2014-08-13 17:38:02.693990 7fe9bae887a0 20 read_log 194215'509 (194215'496) delete b085f3e4/benchmark_data_storage0111-ib.pocstor_64498_object5535/head//22 by client.1049635.0:39738 2014-08-12 17:19:19.784670 2014-08-13 17:38:02.694002 7fe9bae887a0 20 read_log 194215'510 (194215'497) delete b89993e4/benchmark_data_storage0111-ib.pocstor_64498_object11414/head//22 by client.1049635.0:45617 2014-08-12 17:19:21.708168 2014-08-13 17:38:02.694014 7fe9bae887a0 20 read_log 194215'511 (194215'498) delete d53a4be4/benchmark_data_storage0111-ib.pocstor_64498_object14828/head//22 by client.1049635.0:49031 2014-08-12 17:19:22.823027 2014-08-13 17:38:02.694026 7fe9bae887a0 20 read_log 194215'512 (194215'499) delete 55eb0be4/benchmark_data_storage0111-ib.pocstor_64498_object20658/head//22 by client.1049635.0:54861 2014-08-12 17:19:24.760654 2014-08-13 17:38:02.694038 7fe9bae887a0 20 read_log 194215'513 (194215'500) delete 8cf67be4/benchmark_data_storage0111-ib.pocstor_64498_object21331/head//22 by client.1049635.0:55534 2014-08-12 17:19:24.998445 2014-08-13 17:38:02.694053 7fe9bae887a0 20 read_log 194215'514 (194215'501) delete 37bc5be4/benchmark_data_storage0111-ib.pocstor_64498_object22829/head//22 by client.1049635.0:57032 2014-08-12 17:19:25.547867 2014-08-13 17:38:02.694065 7fe9bae887a0 20 read_log 194215'515 (194215'502) delete 1d0f93e4/benchmark_data_storage0111-ib.pocstor_64498_object26258/head//22 by client.1049635.0:60461 2014-08-12 17:19:26.743747 2014-08-13 17:38:02.694079 7fe9bae887a0 20 read_log 194215'516 (194215'503) delete ade7abe4/benchmark_data_storage0111-ib.pocstor_64498_object27199/head//22 by client.1049635.0:61402 2014-08-12 17:19:27.073145 2014-08-13 17:38:02.694091 7fe9bae887a0 20 read_log 194215'517 (194215'504) delete 982fdbe4/benchmark_data_storage0111-ib.pocstor_64498_object33594/head//22 by client.1049635.0:67797 2014-08-12 17:19:29.328082 2014-08-13 17:38:02.694105 7fe9bae887a0 20 read_log 194215'518 (194215'505) delete 59e2a3e4/benchmark_data_storage0111-ib.pocstor_64498_object33849/head//22 by client.1049635.0:68052 2014-08-12 17:19:29.423476 2014-08-13 17:38:02.694217 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.694236 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=0 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.694250 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=0 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001801 0 0.000000 2014-08-13 17:38:02.694261 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=0 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.694269 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=0 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.694278 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'518], crt=191727'442) 2014-08-13 17:38:02.694287 7fe9bae887a0 10 osd.4 194478 pgid 22.481 coll 22.481_head 2014-08-13 17:38:02.694415 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.481 2014-08-13 17:38:02.694424 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.694451 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.481(unlocked)] enter Initial 2014-08-13 17:38:02.694460 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.481(unlocked)] enter NotTrimming 2014-08-13 17:38:02.694614 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.694740 7fe9bae887a0 20 read_log 191648'32 (0'0) modify f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 by client.273158.0:2001 2014-07-30 14:08:56.306848 2014-08-13 17:38:02.694761 7fe9bae887a0 20 read_log 191648'39 (0'0) modify 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 by client.273158.0:2550 2014-07-30 14:08:59.035976 2014-08-13 17:38:02.694773 7fe9bae887a0 20 read_log 191648'128 (0'0) modify c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 by client.273158.0:7329 2014-07-30 14:09:22.405062 2014-08-13 17:38:02.694786 7fe9bae887a0 20 read_log 191648'160 (0'0) modify e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 by client.273158.0:9612 2014-07-30 14:09:34.063892 2014-08-13 17:38:02.694798 7fe9bae887a0 20 read_log 191648'168 (0'0) modify dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 by client.273158.0:9898 2014-07-30 14:09:35.299912 2014-08-13 17:38:02.694810 7fe9bae887a0 20 read_log 191648'179 (0'0) modify a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 by client.273158.0:10617 2014-07-30 14:09:39.132313 2014-08-13 17:38:02.694845 7fe9bae887a0 20 read_log 191648'214 (0'0) modify 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 by client.273158.0:12472 2014-07-30 14:09:49.900374 2014-08-13 17:38:02.694859 7fe9bae887a0 20 read_log 191648'245 (0'0) modify 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 by client.273158.0:14322 2014-07-30 14:09:59.304984 2014-08-13 17:38:02.694871 7fe9bae887a0 20 read_log 191648'355 (0'0) modify a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 by client.273158.0:20931 2014-07-30 14:10:36.683387 2014-08-13 17:38:02.694883 7fe9bae887a0 20 read_log 191648'366 (0'0) modify fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 by client.273158.0:21582 2014-07-30 14:10:39.987529 2014-08-13 17:38:02.694895 7fe9bae887a0 20 read_log 191648'385 (0'0) modify 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 by client.273158.0:22847 2014-07-30 14:10:48.513560 2014-08-13 17:38:02.694908 7fe9bae887a0 20 read_log 191727'394 (0'0) modify bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 by client.275175.0:5071 2014-07-30 14:18:50.558639 2014-08-13 17:38:02.694921 7fe9bae887a0 20 read_log 191727'398 (0'0) modify f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 by client.275175.0:9016 2014-07-30 14:19:02.392588 2014-08-13 17:38:02.694933 7fe9bae887a0 20 read_log 191727'401 (0'0) modify 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 by client.275175.0:9586 2014-07-30 14:19:04.028818 2014-08-13 17:38:02.694945 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 by client.275175.0:10570 2014-07-30 14:19:06.837076 2014-08-13 17:38:02.694959 7fe9bae887a0 20 read_log 191727'405 (0'0) modify 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 by client.275175.0:11276 2014-07-30 14:19:08.876081 2014-08-13 17:38:02.694993 7fe9bae887a0 20 read_log 191727'410 (0'0) modify 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 by client.275175.0:13113 2014-07-30 14:19:15.010034 2014-08-13 17:38:02.695006 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 by client.275175.0:13180 2014-07-30 14:19:15.201136 2014-08-13 17:38:02.695019 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 by client.275175.0:14907 2014-07-30 14:19:20.441176 2014-08-13 17:38:02.695031 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 by client.275175.0:21338 2014-07-30 14:19:40.124246 2014-08-13 17:38:02.695043 7fe9bae887a0 20 read_log 191727'428 (0'0) modify b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 by client.275175.0:23755 2014-07-30 14:19:48.141566 2014-08-13 17:38:02.695055 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 by client.275175.0:24227 2014-07-30 14:19:49.462453 2014-08-13 17:38:02.695067 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 by client.275175.0:24242 2014-07-30 14:19:49.505140 2014-08-13 17:38:02.695079 7fe9bae887a0 20 read_log 191727'433 (0'0) modify f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 by client.275175.0:25072 2014-07-30 14:19:51.907957 2014-08-13 17:38:02.695091 7fe9bae887a0 20 read_log 191727'437 (0'0) modify e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 by client.275175.0:27294 2014-07-30 14:19:58.644997 2014-08-13 17:38:02.695110 7fe9bae887a0 20 read_log 191727'439 (0'0) modify f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 by client.275175.0:29700 2014-07-30 14:20:06.045307 2014-08-13 17:38:02.695122 7fe9bae887a0 20 read_log 191727'445 (0'0) modify 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 by client.275175.0:31689 2014-07-30 14:20:11.975694 2014-08-13 17:38:02.695134 7fe9bae887a0 20 read_log 191727'447 (0'0) modify 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 by client.275175.0:31920 2014-07-30 14:20:12.623608 2014-08-13 17:38:02.695146 7fe9bae887a0 20 read_log 191727'448 (0'0) modify f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 by client.275175.0:32224 2014-07-30 14:20:13.543611 2014-08-13 17:38:02.695164 7fe9bae887a0 20 read_log 191727'455 (0'0) modify 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 by client.275175.0:33789 2014-07-30 14:20:18.389634 2014-08-13 17:38:02.695176 7fe9bae887a0 20 read_log 191727'457 (0'0) modify d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 by client.275175.0:34793 2014-07-30 14:20:21.386137 2014-08-13 17:38:02.695188 7fe9bae887a0 20 read_log 191727'462 (0'0) modify 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 by client.275175.0:35835 2014-07-30 14:20:24.745200 2014-08-13 17:38:02.695201 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 616f5481/benchmark_data_storage0111-ib.pocstor_35625_object651/head//22 by client.940330.0:652 2014-08-11 16:29:43.355266 2014-08-13 17:38:02.695213 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 3154a481/benchmark_data_storage0111-ib.pocstor_35625_object760/head//22 by client.940330.0:761 2014-08-11 16:29:43.795479 2014-08-13 17:38:02.695225 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 9622ec81/benchmark_data_storage0111-ib.pocstor_35625_object1690/head//22 by client.940330.0:1691 2014-08-11 16:29:47.414585 2014-08-13 17:38:02.695236 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 48ce2481/benchmark_data_storage0111-ib.pocstor_35625_object2240/head//22 by client.940330.0:2241 2014-08-11 16:29:49.427494 2014-08-13 17:38:02.695248 7fe9bae887a0 20 read_log 192888'468 (0'0) modify 7b2bd481/benchmark_data_storage0111-ib.pocstor_35625_object9590/head//22 by client.940330.0:9591 2014-08-11 16:30:16.394039 2014-08-13 17:38:02.695260 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 1dba8c81/benchmark_data_storage0111-ib.pocstor_35625_object9759/head//22 by client.940330.0:9760 2014-08-11 16:30:16.983964 2014-08-13 17:38:02.695272 7fe9bae887a0 20 read_log 192888'470 (0'0) modify c2b20481/benchmark_data_storage0111-ib.pocstor_35625_object11967/head//22 by client.940330.0:11968 2014-08-11 16:30:25.790406 2014-08-13 17:38:02.695284 7fe9bae887a0 20 read_log 192888'471 (0'0) modify c56f481/benchmark_data_storage0111-ib.pocstor_35625_object17540/head//22 by client.940330.0:17541 2014-08-11 16:30:46.986613 2014-08-13 17:38:02.695296 7fe9bae887a0 20 read_log 192888'472 (0'0) modify a4f3dc81/benchmark_data_storage0111-ib.pocstor_35625_object17688/head//22 by client.940330.0:17689 2014-08-11 16:30:47.548075 2014-08-13 17:38:02.695307 7fe9bae887a0 20 read_log 192888'473 (0'0) modify 9b88bc81/benchmark_data_storage0111-ib.pocstor_35625_object17932/head//22 by client.940330.0:17933 2014-08-11 16:30:48.460851 2014-08-13 17:38:02.695319 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 415da481/benchmark_data_storage0111-ib.pocstor_35625_object21802/head//22 by client.940330.0:21803 2014-08-11 16:31:03.811189 2014-08-13 17:38:02.695331 7fe9bae887a0 20 read_log 192888'475 (0'0) modify 5a879481/benchmark_data_storage0111-ib.pocstor_35625_object24308/head//22 by client.940330.0:24309 2014-08-11 16:31:13.270508 2014-08-13 17:38:02.695354 7fe9bae887a0 20 read_log 192888'476 (192888'464) delete 616f5481/benchmark_data_storage0111-ib.pocstor_35625_object651/head//22 by client.940330.0:31959 2014-08-11 16:31:41.663758 2014-08-13 17:38:02.695368 7fe9bae887a0 20 read_log 192888'477 (192888'465) delete 3154a481/benchmark_data_storage0111-ib.pocstor_35625_object760/head//22 by client.940330.0:32068 2014-08-11 16:31:41.690227 2014-08-13 17:38:02.695380 7fe9bae887a0 20 read_log 192888'478 (192888'466) delete 9622ec81/benchmark_data_storage0111-ib.pocstor_35625_object1690/head//22 by client.940330.0:32998 2014-08-11 16:31:41.912839 2014-08-13 17:38:02.695392 7fe9bae887a0 20 read_log 192888'479 (192888'467) delete 48ce2481/benchmark_data_storage0111-ib.pocstor_35625_object2240/head//22 by client.940330.0:33548 2014-08-11 16:31:42.052680 2014-08-13 17:38:02.695405 7fe9bae887a0 20 read_log 192888'480 (192888'468) delete 7b2bd481/benchmark_data_storage0111-ib.pocstor_35625_object9590/head//22 by client.940330.0:40898 2014-08-11 16:31:44.002025 2014-08-13 17:38:02.695417 7fe9bae887a0 20 read_log 192888'481 (192888'469) delete 1dba8c81/benchmark_data_storage0111-ib.pocstor_35625_object9759/head//22 by client.940330.0:41067 2014-08-11 16:31:44.048525 2014-08-13 17:38:02.695430 7fe9bae887a0 20 read_log 192888'482 (192888'470) delete c2b20481/benchmark_data_storage0111-ib.pocstor_35625_object11967/head//22 by client.940330.0:43275 2014-08-11 16:31:44.650779 2014-08-13 17:38:02.695442 7fe9bae887a0 20 read_log 192888'483 (192888'471) delete c56f481/benchmark_data_storage0111-ib.pocstor_35625_object17540/head//22 by client.940330.0:48848 2014-08-11 16:31:46.139149 2014-08-13 17:38:02.695470 7fe9bae887a0 20 read_log 192888'484 (192888'472) delete a4f3dc81/benchmark_data_storage0111-ib.pocstor_35625_object17688/head//22 by client.940330.0:48996 2014-08-11 16:31:46.188777 2014-08-13 17:38:02.695486 7fe9bae887a0 20 read_log 192888'485 (192888'473) delete 9b88bc81/benchmark_data_storage0111-ib.pocstor_35625_object17932/head//22 by client.940330.0:49240 2014-08-11 16:31:46.263695 2014-08-13 17:38:02.695498 7fe9bae887a0 20 read_log 192888'486 (192888'474) delete 415da481/benchmark_data_storage0111-ib.pocstor_35625_object21802/head//22 by client.940330.0:53110 2014-08-11 16:31:47.384654 2014-08-13 17:38:02.695511 7fe9bae887a0 20 read_log 192888'487 (192888'475) delete 5a879481/benchmark_data_storage0111-ib.pocstor_35625_object24308/head//22 by client.940330.0:55616 2014-08-11 16:31:48.130594 2014-08-13 17:38:02.695523 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 73420c81/benchmark_data_storage0111-ib.pocstor_64498_object2591/head//22 by client.1049635.0:2592 2014-08-12 17:17:25.511792 2014-08-13 17:38:02.695535 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 7cfb3481/benchmark_data_storage0111-ib.pocstor_64498_object2640/head//22 by client.1049635.0:2641 2014-08-12 17:17:25.657717 2014-08-13 17:38:02.695547 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 227d6481/benchmark_data_storage0111-ib.pocstor_64498_object5057/head//22 by client.1049635.0:5058 2014-08-12 17:17:33.087833 2014-08-13 17:38:02.695559 7fe9bae887a0 20 read_log 194215'491 (0'0) modify f98e6481/benchmark_data_storage0111-ib.pocstor_64498_object8557/head//22 by client.1049635.0:8558 2014-08-12 17:17:43.748065 2014-08-13 17:38:02.695571 7fe9bae887a0 20 read_log 194215'492 (0'0) modify 6b624c81/benchmark_data_storage0111-ib.pocstor_64498_object10778/head//22 by client.1049635.0:10779 2014-08-12 17:17:50.466707 2014-08-13 17:38:02.695584 7fe9bae887a0 20 read_log 194215'493 (0'0) modify b9e75c81/benchmark_data_storage0111-ib.pocstor_64498_object12122/head//22 by client.1049635.0:12123 2014-08-12 17:17:54.858295 2014-08-13 17:38:02.695596 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 7ecb2c81/benchmark_data_storage0111-ib.pocstor_64498_object15034/head//22 by client.1049635.0:15035 2014-08-12 17:18:04.082533 2014-08-13 17:38:02.695672 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 79f93c81/benchmark_data_storage0111-ib.pocstor_64498_object19207/head//22 by client.1049635.0:19208 2014-08-12 17:18:19.817849 2014-08-13 17:38:02.695692 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 150b5c81/benchmark_data_storage0111-ib.pocstor_64498_object19519/head//22 by client.1049635.0:19520 2014-08-12 17:18:20.977017 2014-08-13 17:38:02.695704 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 7b752c81/benchmark_data_storage0111-ib.pocstor_64498_object19713/head//22 by client.1049635.0:19714 2014-08-12 17:18:21.703346 2014-08-13 17:38:02.695717 7fe9bae887a0 20 read_log 194215'498 (0'0) modify 110ea481/benchmark_data_storage0111-ib.pocstor_64498_object25890/head//22 by client.1049635.0:25891 2014-08-12 17:18:45.013390 2014-08-13 17:38:02.695729 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 91a68481/benchmark_data_storage0111-ib.pocstor_64498_object26478/head//22 by client.1049635.0:26479 2014-08-12 17:18:47.209202 2014-08-13 17:38:02.695741 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 7530cc81/benchmark_data_storage0111-ib.pocstor_64498_object28052/head//22 by client.1049635.0:28053 2014-08-12 17:18:53.066060 2014-08-13 17:38:02.695753 7fe9bae887a0 20 read_log 194215'501 (0'0) modify b6f13c81/benchmark_data_storage0111-ib.pocstor_64498_object28758/head//22 by client.1049635.0:28759 2014-08-12 17:18:55.695417 2014-08-13 17:38:02.695765 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 60f45481/benchmark_data_storage0111-ib.pocstor_64498_object28921/head//22 by client.1049635.0:28922 2014-08-12 17:18:56.308411 2014-08-13 17:38:02.695777 7fe9bae887a0 20 read_log 194215'503 (0'0) modify 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:30136 2014-08-12 17:19:01.018764 2014-08-13 17:38:02.695789 7fe9bae887a0 20 read_log 194215'504 (194215'488) delete 73420c81/benchmark_data_storage0111-ib.pocstor_64498_object2591/head//22 by client.1049635.0:36794 2014-08-12 17:19:18.827659 2014-08-13 17:38:02.695802 7fe9bae887a0 20 read_log 194215'505 (194215'489) delete 7cfb3481/benchmark_data_storage0111-ib.pocstor_64498_object2640/head//22 by client.1049635.0:36843 2014-08-12 17:19:18.843070 2014-08-13 17:38:02.695814 7fe9bae887a0 20 read_log 194215'506 (194215'490) delete 227d6481/benchmark_data_storage0111-ib.pocstor_64498_object5057/head//22 by client.1049635.0:39260 2014-08-12 17:19:19.624355 2014-08-13 17:38:02.695826 7fe9bae887a0 20 read_log 194215'507 (194215'491) delete f98e6481/benchmark_data_storage0111-ib.pocstor_64498_object8557/head//22 by client.1049635.0:42760 2014-08-12 17:19:20.780629 2014-08-13 17:38:02.695838 7fe9bae887a0 20 read_log 194215'508 (194215'492) delete 6b624c81/benchmark_data_storage0111-ib.pocstor_64498_object10778/head//22 by client.1049635.0:44981 2014-08-12 17:19:21.510483 2014-08-13 17:38:02.695850 7fe9bae887a0 20 read_log 194215'509 (194215'493) delete b9e75c81/benchmark_data_storage0111-ib.pocstor_64498_object12122/head//22 by client.1049635.0:46325 2014-08-12 17:19:21.935751 2014-08-13 17:38:02.695862 7fe9bae887a0 20 read_log 194215'510 (194215'494) delete 7ecb2c81/benchmark_data_storage0111-ib.pocstor_64498_object15034/head//22 by client.1049635.0:49237 2014-08-12 17:19:22.885215 2014-08-13 17:38:02.695874 7fe9bae887a0 20 read_log 194215'511 (194215'495) delete 79f93c81/benchmark_data_storage0111-ib.pocstor_64498_object19207/head//22 by client.1049635.0:53410 2014-08-12 17:19:24.235640 2014-08-13 17:38:02.695886 7fe9bae887a0 20 read_log 194215'512 (194215'496) delete 150b5c81/benchmark_data_storage0111-ib.pocstor_64498_object19519/head//22 by client.1049635.0:53722 2014-08-12 17:19:24.355477 2014-08-13 17:38:02.695899 7fe9bae887a0 20 read_log 194215'513 (194215'497) delete 7b752c81/benchmark_data_storage0111-ib.pocstor_64498_object19713/head//22 by client.1049635.0:53916 2014-08-12 17:19:24.419889 2014-08-13 17:38:02.695911 7fe9bae887a0 20 read_log 194215'514 (194215'498) delete 110ea481/benchmark_data_storage0111-ib.pocstor_64498_object25890/head//22 by client.1049635.0:60093 2014-08-12 17:19:26.621415 2014-08-13 17:38:02.695925 7fe9bae887a0 20 read_log 194215'515 (194215'499) delete 91a68481/benchmark_data_storage0111-ib.pocstor_64498_object26478/head//22 by client.1049635.0:60681 2014-08-12 17:19:26.814269 2014-08-13 17:38:02.695937 7fe9bae887a0 20 read_log 194215'516 (194215'500) delete 7530cc81/benchmark_data_storage0111-ib.pocstor_64498_object28052/head//22 by client.1049635.0:62255 2014-08-12 17:19:27.364377 2014-08-13 17:38:02.695949 7fe9bae887a0 20 read_log 194215'517 (194215'501) delete b6f13c81/benchmark_data_storage0111-ib.pocstor_64498_object28758/head//22 by client.1049635.0:62961 2014-08-12 17:19:27.615796 2014-08-13 17:38:02.695961 7fe9bae887a0 20 read_log 194215'518 (194215'502) delete 60f45481/benchmark_data_storage0111-ib.pocstor_64498_object28921/head//22 by client.1049635.0:63124 2014-08-12 17:19:27.675242 2014-08-13 17:38:02.695973 7fe9bae887a0 20 read_log 194215'519 (194215'503) delete 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:64338 2014-08-12 17:19:28.088153 2014-08-13 17:38:02.696082 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.696101 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=0 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.696115 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=0 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001664 0 0.000000 2014-08-13 17:38:02.696126 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=0 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.696134 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=0 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.696143 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'519], crt=191727'463) 2014-08-13 17:38:02.696152 7fe9bae887a0 10 osd.4 194478 pgid 22.48e coll 22.48e_head 2014-08-13 17:38:02.696270 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.48e 2014-08-13 17:38:02.696279 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.696315 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.48e(unlocked)] enter Initial 2014-08-13 17:38:02.696324 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.48e(unlocked)] enter NotTrimming 2014-08-13 17:38:02.696442 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.696533 7fe9bae887a0 20 read_log 191648'1 (0'0) modify c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 by client.273158.0:26 2014-07-30 14:08:48.646718 2014-08-13 17:38:02.696553 7fe9bae887a0 20 read_log 191648'8 (0'0) modify 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 by client.273158.0:594 2014-07-30 14:08:50.992311 2014-08-13 17:38:02.696566 7fe9bae887a0 20 read_log 191648'11 (0'0) modify ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 by client.273158.0:725 2014-07-30 14:08:51.474548 2014-08-13 17:38:02.696578 7fe9bae887a0 20 read_log 191648'22 (0'0) modify 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 by client.273158.0:1181 2014-07-30 14:08:53.136462 2014-08-13 17:38:02.696590 7fe9bae887a0 20 read_log 191648'51 (0'0) modify 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 by client.273158.0:2702 2014-07-30 14:08:59.851949 2014-08-13 17:38:02.696602 7fe9bae887a0 20 read_log 191648'67 (0'0) modify 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 by client.273158.0:3870 2014-07-30 14:09:05.668495 2014-08-13 17:38:02.696613 7fe9bae887a0 20 read_log 191648'109 (0'0) modify 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 by client.273158.0:6497 2014-07-30 14:09:18.669611 2014-08-13 17:38:02.696625 7fe9bae887a0 20 read_log 191648'110 (0'0) modify 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 by client.273158.0:6617 2014-07-30 14:09:19.419227 2014-08-13 17:38:02.696637 7fe9bae887a0 20 read_log 191648'125 (0'0) modify f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 by client.273158.0:7117 2014-07-30 14:09:21.548833 2014-08-13 17:38:02.696649 7fe9bae887a0 20 read_log 191648'159 (0'0) modify fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 by client.273158.0:8616 2014-07-30 14:09:29.656074 2014-08-13 17:38:02.696661 7fe9bae887a0 20 read_log 191648'246 (0'0) modify cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 by client.273158.0:14282 2014-07-30 14:09:59.139806 2014-08-13 17:38:02.696673 7fe9bae887a0 20 read_log 191648'265 (0'0) modify dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 by client.273158.0:15742 2014-07-30 14:10:05.831344 2014-08-13 17:38:02.696684 7fe9bae887a0 20 read_log 191648'273 (0'0) modify 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 by client.273158.0:16606 2014-07-30 14:10:10.763484 2014-08-13 17:38:02.696697 7fe9bae887a0 20 read_log 191648'309 (0'0) modify 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 by client.273158.0:18717 2014-07-30 14:10:22.416198 2014-08-13 17:38:02.696709 7fe9bae887a0 20 read_log 191648'371 (0'0) modify 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 by client.273158.0:22737 2014-07-30 14:10:48.013719 2014-08-13 17:38:02.696721 7fe9bae887a0 20 read_log 191727'380 (0'0) modify d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 by client.275175.0:2402 2014-07-30 14:18:42.259382 2014-08-13 17:38:02.696733 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 by client.275175.0:3270 2014-07-30 14:18:45.069555 2014-08-13 17:38:02.696745 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 by client.275175.0:3368 2014-07-30 14:18:45.378896 2014-08-13 17:38:02.696775 7fe9bae887a0 20 read_log 191727'398 (0'0) modify cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 by client.275175.0:7523 2014-07-30 14:18:58.193895 2014-08-13 17:38:02.696796 7fe9bae887a0 20 read_log 191727'399 (0'0) modify f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 by client.275175.0:7571 2014-07-30 14:18:58.328199 2014-08-13 17:38:02.696808 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 by client.275175.0:8445 2014-07-30 14:19:00.766469 2014-08-13 17:38:02.696820 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 by client.275175.0:10455 2014-07-30 14:19:06.510270 2014-08-13 17:38:02.696832 7fe9bae887a0 20 read_log 191727'408 (0'0) modify fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 by client.275175.0:10940 2014-07-30 14:19:07.874091 2014-08-13 17:38:02.696844 7fe9bae887a0 20 read_log 191727'410 (0'0) modify df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 by client.275175.0:11110 2014-07-30 14:19:08.349474 2014-08-13 17:38:02.696856 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 by client.275175.0:11412 2014-07-30 14:19:09.273830 2014-08-13 17:38:02.696868 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 by client.275175.0:14260 2014-07-30 14:19:18.608762 2014-08-13 17:38:02.696880 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 by client.275175.0:18340 2014-07-30 14:19:30.982002 2014-08-13 17:38:02.696891 7fe9bae887a0 20 read_log 191727'428 (0'0) modify 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 by client.275175.0:19101 2014-07-30 14:19:33.610221 2014-08-13 17:38:02.696903 7fe9bae887a0 20 read_log 191727'435 (0'0) modify bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 by client.275175.0:22750 2014-07-30 14:19:45.279965 2014-08-13 17:38:02.696915 7fe9bae887a0 20 read_log 191727'437 (0'0) modify efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 by client.275175.0:23762 2014-07-30 14:19:48.161179 2014-08-13 17:38:02.696927 7fe9bae887a0 20 read_log 191727'450 (0'0) modify 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 by client.275175.0:29672 2014-07-30 14:20:05.969957 2014-08-13 17:38:02.696939 7fe9bae887a0 20 read_log 191727'452 (0'0) modify c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 by client.275175.0:30865 2014-07-30 14:20:09.595983 2014-08-13 17:38:02.696951 7fe9bae887a0 20 read_log 191727'453 (0'0) modify e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 by client.275175.0:31893 2014-07-30 14:20:12.546838 2014-08-13 17:38:02.696964 7fe9bae887a0 20 read_log 191727'455 (0'0) modify 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 by client.275175.0:32199 2014-07-30 14:20:13.471727 2014-08-13 17:38:02.696976 7fe9bae887a0 20 read_log 191727'456 (0'0) modify 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 by client.275175.0:32241 2014-07-30 14:20:13.592020 2014-08-13 17:38:02.696988 7fe9bae887a0 20 read_log 191727'458 (0'0) modify 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 by client.275175.0:33955 2014-07-30 14:20:18.862563 2014-08-13 17:38:02.697000 7fe9bae887a0 20 read_log 191727'461 (0'0) modify 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 by client.275175.0:35063 2014-07-30 14:20:22.161828 2014-08-13 17:38:02.697011 7fe9bae887a0 20 read_log 191727'469 (0'0) modify d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 by client.275175.0:36638 2014-07-30 14:20:27.297370 2014-08-13 17:38:02.697023 7fe9bae887a0 20 read_log 191727'470 (0'0) modify 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 by client.275175.0:36838 2014-07-30 14:20:27.921555 2014-08-13 17:38:02.697046 7fe9bae887a0 20 read_log 191727'473 (0'0) modify e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 by client.275175.0:38457 2014-07-30 14:20:32.542729 2014-08-13 17:38:02.697059 7fe9bae887a0 20 read_log 192888'474 (0'0) modify cd2c6c8e/benchmark_data_storage0111-ib.pocstor_35625_object322/head//22 by client.940330.0:323 2014-08-11 16:29:41.998487 2014-08-13 17:38:02.697071 7fe9bae887a0 20 read_log 192888'475 (0'0) modify 669cac8e/benchmark_data_storage0111-ib.pocstor_35625_object538/head//22 by client.940330.0:539 2014-08-11 16:29:42.883101 2014-08-13 17:38:02.697083 7fe9bae887a0 20 read_log 192888'476 (0'0) modify 5bd1b48e/benchmark_data_storage0111-ib.pocstor_35625_object4062/head//22 by client.940330.0:4063 2014-08-11 16:29:56.558578 2014-08-13 17:38:02.697095 7fe9bae887a0 20 read_log 192888'477 (0'0) modify 3318b48e/benchmark_data_storage0111-ib.pocstor_35625_object5841/head//22 by client.940330.0:5842 2014-08-11 16:30:03.002894 2014-08-13 17:38:02.697107 7fe9bae887a0 20 read_log 192888'478 (0'0) modify 917aa48e/benchmark_data_storage0111-ib.pocstor_35625_object7990/head//22 by client.940330.0:7991 2014-08-11 16:30:10.803982 2014-08-13 17:38:02.697119 7fe9bae887a0 20 read_log 192888'479 (0'0) modify c160b48e/benchmark_data_storage0111-ib.pocstor_35625_object10453/head//22 by client.940330.0:10454 2014-08-11 16:30:19.552772 2014-08-13 17:38:02.697131 7fe9bae887a0 20 read_log 192888'480 (0'0) modify ea0e2c8e/benchmark_data_storage0111-ib.pocstor_35625_object10824/head//22 by client.940330.0:10825 2014-08-11 16:30:21.115997 2014-08-13 17:38:02.697146 7fe9bae887a0 20 read_log 192888'481 (0'0) modify 5a41b48e/benchmark_data_storage0111-ib.pocstor_35625_object15525/head//22 by client.940330.0:15526 2014-08-11 16:30:39.452952 2014-08-13 17:38:02.697170 7fe9bae887a0 20 read_log 192888'482 (0'0) modify a4ac8e/benchmark_data_storage0111-ib.pocstor_35625_object16743/head//22 by client.940330.0:16744 2014-08-11 16:30:44.009417 2014-08-13 17:38:02.697183 7fe9bae887a0 20 read_log 192888'483 (0'0) modify a707ec8e/benchmark_data_storage0111-ib.pocstor_35625_object16957/head//22 by client.940330.0:16958 2014-08-11 16:30:44.804103 2014-08-13 17:38:02.697195 7fe9bae887a0 20 read_log 192888'484 (0'0) modify fa9f248e/benchmark_data_storage0111-ib.pocstor_35625_object17327/head//22 by client.940330.0:17328 2014-08-11 16:30:46.189079 2014-08-13 17:38:02.697212 7fe9bae887a0 20 read_log 192888'485 (0'0) modify 7ed82c8e/benchmark_data_storage0111-ib.pocstor_35625_object21083/head//22 by client.940330.0:21084 2014-08-11 16:31:01.151956 2014-08-13 17:38:02.697225 7fe9bae887a0 20 read_log 192888'486 (0'0) modify 5d78fc8e/benchmark_data_storage0111-ib.pocstor_35625_object23545/head//22 by client.940330.0:23546 2014-08-11 16:31:10.422534 2014-08-13 17:38:02.697240 7fe9bae887a0 20 read_log 192888'487 (0'0) modify 42a7848e/benchmark_data_storage0111-ib.pocstor_35625_object26231/head//22 by client.940330.0:26232 2014-08-11 16:31:20.701197 2014-08-13 17:38:02.697269 7fe9bae887a0 20 read_log 192888'488 (0'0) modify 3bde748e/benchmark_data_storage0111-ib.pocstor_35625_object30416/head//22 by client.940330.0:30417 2014-08-11 16:31:37.036759 2014-08-13 17:38:02.697284 7fe9bae887a0 20 read_log 192888'489 (192888'474) delete cd2c6c8e/benchmark_data_storage0111-ib.pocstor_35625_object322/head//22 by client.940330.0:31630 2014-08-11 16:31:41.571063 2014-08-13 17:38:02.697297 7fe9bae887a0 20 read_log 192888'490 (192888'475) delete 669cac8e/benchmark_data_storage0111-ib.pocstor_35625_object538/head//22 by client.940330.0:31846 2014-08-11 16:31:41.632023 2014-08-13 17:38:02.697309 7fe9bae887a0 20 read_log 192888'491 (192888'476) delete 5bd1b48e/benchmark_data_storage0111-ib.pocstor_35625_object4062/head//22 by client.940330.0:35370 2014-08-11 16:31:42.552595 2014-08-13 17:38:02.697332 7fe9bae887a0 20 read_log 192888'492 (192888'477) delete 3318b48e/benchmark_data_storage0111-ib.pocstor_35625_object5841/head//22 by client.940330.0:37149 2014-08-11 16:31:43.034086 2014-08-13 17:38:02.697345 7fe9bae887a0 20 read_log 192888'493 (192888'478) delete 917aa48e/benchmark_data_storage0111-ib.pocstor_35625_object7990/head//22 by client.940330.0:39298 2014-08-11 16:31:43.587500 2014-08-13 17:38:02.697357 7fe9bae887a0 20 read_log 192888'494 (192888'479) delete c160b48e/benchmark_data_storage0111-ib.pocstor_35625_object10453/head//22 by client.940330.0:41761 2014-08-11 16:31:44.240058 2014-08-13 17:38:02.697369 7fe9bae887a0 20 read_log 192888'495 (192888'480) delete ea0e2c8e/benchmark_data_storage0111-ib.pocstor_35625_object10824/head//22 by client.940330.0:42132 2014-08-11 16:31:44.349897 2014-08-13 17:38:02.697386 7fe9bae887a0 20 read_log 192888'496 (192888'481) delete 5a41b48e/benchmark_data_storage0111-ib.pocstor_35625_object15525/head//22 by client.940330.0:46833 2014-08-11 16:31:45.597270 2014-08-13 17:38:02.697401 7fe9bae887a0 20 read_log 192888'497 (192888'482) delete a4ac8e/benchmark_data_storage0111-ib.pocstor_35625_object16743/head//22 by client.940330.0:48051 2014-08-11 16:31:45.918183 2014-08-13 17:38:02.697413 7fe9bae887a0 20 read_log 192888'498 (192888'483) delete a707ec8e/benchmark_data_storage0111-ib.pocstor_35625_object16957/head//22 by client.940330.0:48265 2014-08-11 16:31:45.977173 2014-08-13 17:38:02.697425 7fe9bae887a0 20 read_log 192888'499 (192888'484) delete fa9f248e/benchmark_data_storage0111-ib.pocstor_35625_object17327/head//22 by client.940330.0:48635 2014-08-11 16:31:46.081888 2014-08-13 17:38:02.697437 7fe9bae887a0 20 read_log 192888'500 (192888'485) delete 7ed82c8e/benchmark_data_storage0111-ib.pocstor_35625_object21083/head//22 by client.940330.0:52391 2014-08-11 16:31:47.183825 2014-08-13 17:38:02.697452 7fe9bae887a0 20 read_log 192888'501 (192888'486) delete 5d78fc8e/benchmark_data_storage0111-ib.pocstor_35625_object23545/head//22 by client.940330.0:54853 2014-08-11 16:31:47.908453 2014-08-13 17:38:02.697464 7fe9bae887a0 20 read_log 192888'502 (192888'487) delete 42a7848e/benchmark_data_storage0111-ib.pocstor_35625_object26231/head//22 by client.940330.0:57539 2014-08-11 16:31:48.713078 2014-08-13 17:38:02.697478 7fe9bae887a0 20 read_log 192888'503 (192888'488) delete 3bde748e/benchmark_data_storage0111-ib.pocstor_35625_object30416/head//22 by client.940330.0:61724 2014-08-11 16:31:49.876608 2014-08-13 17:38:02.697490 7fe9bae887a0 20 read_log 194215'504 (0'0) modify f98fdc8e/benchmark_data_storage0111-ib.pocstor_64498_object28/head//22 by client.1049635.0:29 2014-08-12 17:17:17.032421 2014-08-13 17:38:02.697505 7fe9bae887a0 20 read_log 194215'505 (0'0) modify ed082c8e/benchmark_data_storage0111-ib.pocstor_64498_object2105/head//22 by client.1049635.0:2106 2014-08-12 17:17:23.864246 2014-08-13 17:38:02.697517 7fe9bae887a0 20 read_log 194215'506 (0'0) modify fa38148e/benchmark_data_storage0111-ib.pocstor_64498_object2397/head//22 by client.1049635.0:2398 2014-08-12 17:17:24.923333 2014-08-13 17:38:02.697532 7fe9bae887a0 20 read_log 194215'507 (0'0) modify 46696c8e/benchmark_data_storage0111-ib.pocstor_64498_object5518/head//22 by client.1049635.0:5519 2014-08-12 17:17:34.668262 2014-08-13 17:38:02.697544 7fe9bae887a0 20 read_log 194215'508 (0'0) modify df5eb48e/benchmark_data_storage0111-ib.pocstor_64498_object5652/head//22 by client.1049635.0:5653 2014-08-12 17:17:35.045714 2014-08-13 17:38:02.697558 7fe9bae887a0 20 read_log 194215'509 (0'0) modify 8184a48e/benchmark_data_storage0111-ib.pocstor_64498_object7318/head//22 by client.1049635.0:7319 2014-08-12 17:17:40.025030 2014-08-13 17:38:02.697571 7fe9bae887a0 20 read_log 194215'510 (0'0) modify 80d8b48e/benchmark_data_storage0111-ib.pocstor_64498_object10767/head//22 by client.1049635.0:10768 2014-08-12 17:17:50.434516 2014-08-13 17:38:02.697592 7fe9bae887a0 20 read_log 194215'511 (0'0) modify b4523c8e/benchmark_data_storage0111-ib.pocstor_64498_object15072/head//22 by client.1049635.0:15073 2014-08-12 17:18:04.204502 2014-08-13 17:38:02.697607 7fe9bae887a0 20 read_log 194215'512 (0'0) modify 9291c48e/benchmark_data_storage0111-ib.pocstor_64498_object16532/head//22 by client.1049635.0:16533 2014-08-12 17:18:09.583986 2014-08-13 17:38:02.697619 7fe9bae887a0 20 read_log 194215'513 (0'0) modify 6fc1248e/benchmark_data_storage0111-ib.pocstor_64498_object17593/head//22 by client.1049635.0:17594 2014-08-12 17:18:13.622871 2014-08-13 17:38:02.697631 7fe9bae887a0 20 read_log 194215'514 (0'0) modify b180648e/benchmark_data_storage0111-ib.pocstor_64498_object21571/head//22 by client.1049635.0:21572 2014-08-12 17:18:28.721561 2014-08-13 17:38:02.697645 7fe9bae887a0 20 read_log 194215'515 (0'0) modify 717a5c8e/benchmark_data_storage0111-ib.pocstor_64498_object22512/head//22 by client.1049635.0:22513 2014-08-12 17:18:32.316627 2014-08-13 17:38:02.697657 7fe9bae887a0 20 read_log 194215'516 (0'0) modify d57b948e/benchmark_data_storage0111-ib.pocstor_64498_object22835/head//22 by client.1049635.0:22836 2014-08-12 17:18:33.521701 2014-08-13 17:38:02.697671 7fe9bae887a0 20 read_log 194215'517 (0'0) modify e04f548e/benchmark_data_storage0111-ib.pocstor_64498_object24458/head//22 by client.1049635.0:24459 2014-08-12 17:18:39.556547 2014-08-13 17:38:02.697683 7fe9bae887a0 20 read_log 194215'518 (0'0) modify d6678c8e/benchmark_data_storage0111-ib.pocstor_64498_object24663/head//22 by client.1049635.0:24664 2014-08-12 17:18:40.321956 2014-08-13 17:38:02.697695 7fe9bae887a0 20 read_log 194215'519 (0'0) modify 149bb48e/benchmark_data_storage0111-ib.pocstor_64498_object26333/head//22 by client.1049635.0:26334 2014-08-12 17:18:46.658119 2014-08-13 17:38:02.697707 7fe9bae887a0 20 read_log 194215'520 (0'0) modify cff48c8e/benchmark_data_storage0111-ib.pocstor_64498_object29414/head//22 by client.1049635.0:29415 2014-08-12 17:18:58.134134 2014-08-13 17:38:02.697720 7fe9bae887a0 20 read_log 194215'521 (0'0) modify 4c84148e/benchmark_data_storage0111-ib.pocstor_64498_object31608/head//22 by client.1049635.0:31609 2014-08-12 17:19:07.049100 2014-08-13 17:38:02.697734 7fe9bae887a0 20 read_log 194215'522 (0'0) modify 798a448e/benchmark_data_storage0111-ib.pocstor_64498_object33432/head//22 by client.1049635.0:33433 2014-08-12 17:19:14.019189 2014-08-13 17:38:02.697746 7fe9bae887a0 20 read_log 194215'523 (0'0) modify cd52148e/benchmark_data_storage0111-ib.pocstor_64498_object33494/head//22 by client.1049635.0:33495 2014-08-12 17:19:14.253182 2014-08-13 17:38:02.697760 7fe9bae887a0 20 read_log 194215'524 (194215'504) delete f98fdc8e/benchmark_data_storage0111-ib.pocstor_64498_object28/head//22 by client.1049635.0:34231 2014-08-12 17:19:17.986064 2014-08-13 17:38:02.697772 7fe9bae887a0 20 read_log 194215'525 (194215'505) delete ed082c8e/benchmark_data_storage0111-ib.pocstor_64498_object2105/head//22 by client.1049635.0:36308 2014-08-12 17:19:18.676401 2014-08-13 17:38:02.697786 7fe9bae887a0 20 read_log 194215'526 (194215'506) delete fa38148e/benchmark_data_storage0111-ib.pocstor_64498_object2397/head//22 by client.1049635.0:36600 2014-08-12 17:19:18.767261 2014-08-13 17:38:02.697798 7fe9bae887a0 20 read_log 194215'527 (194215'507) delete 46696c8e/benchmark_data_storage0111-ib.pocstor_64498_object5518/head//22 by client.1049635.0:39721 2014-08-12 17:19:19.778798 2014-08-13 17:38:02.697815 7fe9bae887a0 20 read_log 194215'528 (194215'508) delete df5eb48e/benchmark_data_storage0111-ib.pocstor_64498_object5652/head//22 by client.1049635.0:39855 2014-08-12 17:19:19.833243 2014-08-13 17:38:02.697829 7fe9bae887a0 20 read_log 194215'529 (194215'509) delete 8184a48e/benchmark_data_storage0111-ib.pocstor_64498_object7318/head//22 by client.1049635.0:41521 2014-08-12 17:19:20.389816 2014-08-13 17:38:02.697842 7fe9bae887a0 20 read_log 194215'530 (194215'510) delete 80d8b48e/benchmark_data_storage0111-ib.pocstor_64498_object10767/head//22 by client.1049635.0:44970 2014-08-12 17:19:21.507297 2014-08-13 17:38:02.697863 7fe9bae887a0 20 read_log 194215'531 (194215'511) delete b4523c8e/benchmark_data_storage0111-ib.pocstor_64498_object15072/head//22 by client.1049635.0:49275 2014-08-12 17:19:22.896798 2014-08-13 17:38:02.697878 7fe9bae887a0 20 read_log 194215'532 (194215'512) delete 9291c48e/benchmark_data_storage0111-ib.pocstor_64498_object16532/head//22 by client.1049635.0:50735 2014-08-12 17:19:23.361790 2014-08-13 17:38:02.697890 7fe9bae887a0 20 read_log 194215'533 (194215'513) delete 6fc1248e/benchmark_data_storage0111-ib.pocstor_64498_object17593/head//22 by client.1049635.0:51796 2014-08-12 17:19:23.695362 2014-08-13 17:38:02.697904 7fe9bae887a0 20 read_log 194215'534 (194215'514) delete b180648e/benchmark_data_storage0111-ib.pocstor_64498_object21571/head//22 by client.1049635.0:55774 2014-08-12 17:19:25.078168 2014-08-13 17:38:02.697916 7fe9bae887a0 20 read_log 194215'535 (194215'515) delete 717a5c8e/benchmark_data_storage0111-ib.pocstor_64498_object22512/head//22 by client.1049635.0:56715 2014-08-12 17:19:25.426543 2014-08-13 17:38:02.697930 7fe9bae887a0 20 read_log 194215'536 (194215'516) delete d57b948e/benchmark_data_storage0111-ib.pocstor_64498_object22835/head//22 by client.1049635.0:57038 2014-08-12 17:19:25.556395 2014-08-13 17:38:02.697943 7fe9bae887a0 20 read_log 194215'537 (194215'517) delete e04f548e/benchmark_data_storage0111-ib.pocstor_64498_object24458/head//22 by client.1049635.0:58661 2014-08-12 17:19:26.119173 2014-08-13 17:38:02.697955 7fe9bae887a0 20 read_log 194215'538 (194215'518) delete d6678c8e/benchmark_data_storage0111-ib.pocstor_64498_object24663/head//22 by client.1049635.0:58866 2014-08-12 17:19:26.197839 2014-08-13 17:38:02.697967 7fe9bae887a0 20 read_log 194215'539 (194215'519) delete 149bb48e/benchmark_data_storage0111-ib.pocstor_64498_object26333/head//22 by client.1049635.0:60536 2014-08-12 17:19:26.769915 2014-08-13 17:38:02.697979 7fe9bae887a0 20 read_log 194215'540 (194215'520) delete cff48c8e/benchmark_data_storage0111-ib.pocstor_64498_object29414/head//22 by client.1049635.0:63617 2014-08-12 17:19:27.856722 2014-08-13 17:38:02.697992 7fe9bae887a0 20 read_log 194215'541 (194215'521) delete 4c84148e/benchmark_data_storage0111-ib.pocstor_64498_object31608/head//22 by client.1049635.0:65811 2014-08-12 17:19:28.602741 2014-08-13 17:38:02.698005 7fe9bae887a0 20 read_log 194215'542 (194215'522) delete 798a448e/benchmark_data_storage0111-ib.pocstor_64498_object33432/head//22 by client.1049635.0:67635 2014-08-12 17:19:29.268938 2014-08-13 17:38:02.698019 7fe9bae887a0 20 read_log 194215'543 (194215'523) delete cd52148e/benchmark_data_storage0111-ib.pocstor_64498_object33494/head//22 by client.1049635.0:67697 2014-08-12 17:19:29.289388 2014-08-13 17:38:02.698126 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.698145 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=0 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.698165 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=0 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001850 0 0.000000 2014-08-13 17:38:02.698176 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=0 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.698185 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=0 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.698194 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'543], crt=191727'473) 2014-08-13 17:38:02.698203 7fe9bae887a0 10 osd.4 194478 pgid 22.4aa coll 22.4aa_head 2014-08-13 17:38:02.698342 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.4aa 2014-08-13 17:38:02.698352 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.698377 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4aa(unlocked)] enter Initial 2014-08-13 17:38:02.698388 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4aa(unlocked)] enter NotTrimming 2014-08-13 17:38:02.698506 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.698606 7fe9bae887a0 20 read_log 191648'3 (0'0) modify b51acaa/benchmark_data_storage0111-ib.pocstor_40745_object45/head//22 by client.273158.0:46 2014-07-30 14:08:48.787550 2014-08-13 17:38:02.698626 7fe9bae887a0 20 read_log 191648'71 (0'0) modify 99ae54aa/benchmark_data_storage0111-ib.pocstor_40745_object4754/head//22 by client.273158.0:4755 2014-07-30 14:09:09.888332 2014-08-13 17:38:02.698639 7fe9bae887a0 20 read_log 191648'112 (0'0) modify af2f1caa/benchmark_data_storage0111-ib.pocstor_40745_object7536/head//22 by client.273158.0:7537 2014-07-30 14:09:23.628445 2014-08-13 17:38:02.698651 7fe9bae887a0 20 read_log 191648'159 (0'0) modify f33d4caa/benchmark_data_storage0111-ib.pocstor_40745_object10718/head//22 by client.273158.0:10719 2014-07-30 14:09:39.611444 2014-08-13 17:38:02.698664 7fe9bae887a0 20 read_log 191648'180 (0'0) modify 99ebd4aa/benchmark_data_storage0111-ib.pocstor_40745_object12728/head//22 by client.273158.0:12729 2014-07-30 14:09:51.130131 2014-08-13 17:38:02.698676 7fe9bae887a0 20 read_log 191648'218 (0'0) modify 457144aa/benchmark_data_storage0111-ib.pocstor_40745_object15319/head//22 by client.273158.0:15320 2014-07-30 14:10:03.951159 2014-08-13 17:38:02.698688 7fe9bae887a0 20 read_log 191648'328 (0'0) modify 7ce87caa/benchmark_data_storage0111-ib.pocstor_40745_object22191/head//22 by client.273158.0:22192 2014-07-30 14:10:44.944594 2014-08-13 17:38:02.698700 7fe9bae887a0 20 read_log 191727'336 (0'0) modify 891174aa/benchmark_data_storage0111-ib.pocstor_51285_object128/head//22 by client.275175.0:129 2014-07-30 14:18:33.960086 2014-08-13 17:38:02.698712 7fe9bae887a0 20 read_log 191727'338 (0'0) modify 93360caa/benchmark_data_storage0111-ib.pocstor_51285_object429/head//22 by client.275175.0:430 2014-07-30 14:18:35.151171 2014-08-13 17:38:02.698724 7fe9bae887a0 20 read_log 191727'339 (0'0) modify 1a7634aa/benchmark_data_storage0111-ib.pocstor_51285_object592/head//22 by client.275175.0:593 2014-07-30 14:18:35.798009 2014-08-13 17:38:02.698736 7fe9bae887a0 20 read_log 191727'342 (0'0) modify 99c09caa/benchmark_data_storage0111-ib.pocstor_51285_object1952/head//22 by client.275175.0:1953 2014-07-30 14:18:40.743701 2014-08-13 17:38:02.698748 7fe9bae887a0 20 read_log 191727'347 (0'0) modify 67c484aa/benchmark_data_storage0111-ib.pocstor_51285_object5443/head//22 by client.275175.0:5444 2014-07-30 14:18:51.655065 2014-08-13 17:38:02.698760 7fe9bae887a0 20 read_log 191727'349 (0'0) modify ed9f84aa/benchmark_data_storage0111-ib.pocstor_51285_object5952/head//22 by client.275175.0:5953 2014-07-30 14:18:53.108922 2014-08-13 17:38:02.698772 7fe9bae887a0 20 read_log 191727'352 (0'0) modify b7d22caa/benchmark_data_storage0111-ib.pocstor_51285_object6321/head//22 by client.275175.0:6322 2014-07-30 14:18:54.214929 2014-08-13 17:38:02.698784 7fe9bae887a0 20 read_log 191727'355 (0'0) modify 174a74aa/benchmark_data_storage0111-ib.pocstor_51285_object8009/head//22 by client.275175.0:8010 2014-07-30 14:18:59.549430 2014-08-13 17:38:02.698795 7fe9bae887a0 20 read_log 191727'357 (0'0) modify e823bcaa/benchmark_data_storage0111-ib.pocstor_51285_object8940/head//22 by client.275175.0:8941 2014-07-30 14:19:02.179241 2014-08-13 17:38:02.698807 7fe9bae887a0 20 read_log 191727'359 (0'0) modify 1d1b44aa/benchmark_data_storage0111-ib.pocstor_51285_object9631/head//22 by client.275175.0:9632 2014-07-30 14:19:04.156961 2014-08-13 17:38:02.698819 7fe9bae887a0 20 read_log 191727'363 (0'0) modify 9de244aa/benchmark_data_storage0111-ib.pocstor_51285_object13048/head//22 by client.275175.0:13049 2014-07-30 14:19:14.832051 2014-08-13 17:38:02.698833 7fe9bae887a0 20 read_log 191727'367 (0'0) modify e5da4caa/benchmark_data_storage0111-ib.pocstor_51285_object15028/head//22 by client.275175.0:15029 2014-07-30 14:19:20.782242 2014-08-13 17:38:02.698853 7fe9bae887a0 20 read_log 191727'371 (0'0) modify f5dddcaa/benchmark_data_storage0111-ib.pocstor_51285_object15992/head//22 by client.275175.0:15993 2014-07-30 14:19:23.813770 2014-08-13 17:38:02.698865 7fe9bae887a0 20 read_log 191727'380 (0'0) modify f632fcaa/benchmark_data_storage0111-ib.pocstor_51285_object20990/head//22 by client.275175.0:20991 2014-07-30 14:19:39.104325 2014-08-13 17:38:02.698877 7fe9bae887a0 20 read_log 191727'381 (0'0) modify 6d6534aa/benchmark_data_storage0111-ib.pocstor_51285_object20991/head//22 by client.275175.0:20992 2014-07-30 14:19:39.106129 2014-08-13 17:38:02.698889 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 43aa2caa/benchmark_data_storage0111-ib.pocstor_51285_object21283/head//22 by client.275175.0:21284 2014-07-30 14:19:39.970931 2014-08-13 17:38:02.698901 7fe9bae887a0 20 read_log 191727'383 (0'0) modify 6f764aa/benchmark_data_storage0111-ib.pocstor_51285_object22801/head//22 by client.275175.0:22802 2014-07-30 14:19:45.426390 2014-08-13 17:38:02.698913 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 214004aa/benchmark_data_storage0111-ib.pocstor_51285_object25497/head//22 by client.275175.0:25498 2014-07-30 14:19:53.253545 2014-08-13 17:38:02.698925 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 5a2104aa/benchmark_data_storage0111-ib.pocstor_51285_object29411/head//22 by client.275175.0:29412 2014-07-30 14:20:05.132669 2014-08-13 17:38:02.698937 7fe9bae887a0 20 read_log 191727'397 (0'0) modify ae4dcaa/benchmark_data_storage0111-ib.pocstor_51285_object31225/head//22 by client.275175.0:31226 2014-07-30 14:20:10.627748 2014-08-13 17:38:02.698948 7fe9bae887a0 20 read_log 191727'401 (0'0) modify ec101caa/benchmark_data_storage0111-ib.pocstor_51285_object33130/head//22 by client.275175.0:33131 2014-07-30 14:20:16.161536 2014-08-13 17:38:02.698961 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 7e4da4aa/benchmark_data_storage0111-ib.pocstor_51285_object33892/head//22 by client.275175.0:33893 2014-07-30 14:20:18.683513 2014-08-13 17:38:02.698973 7fe9bae887a0 20 read_log 191727'404 (0'0) modify f0dd54aa/benchmark_data_storage0111-ib.pocstor_51285_object34289/head//22 by client.275175.0:34290 2014-07-30 14:20:19.881177 2014-08-13 17:38:02.698985 7fe9bae887a0 20 read_log 191727'408 (0'0) modify 1ce2dcaa/benchmark_data_storage0111-ib.pocstor_51285_object36532/head//22 by client.275175.0:36533 2014-07-30 14:20:26.943308 2014-08-13 17:38:02.698997 7fe9bae887a0 20 read_log 191727'409 (0'0) modify c1364caa/benchmark_data_storage0111-ib.pocstor_51285_object37195/head//22 by client.275175.0:37196 2014-07-30 14:20:28.917566 2014-08-13 17:38:02.699008 7fe9bae887a0 20 read_log 191727'412 (0'0) modify a72e64aa/benchmark_data_storage0111-ib.pocstor_51285_object37846/head//22 by client.275175.0:37847 2014-07-30 14:20:30.811454 2014-08-13 17:38:02.699020 7fe9bae887a0 20 read_log 192674'416 (0'0) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5879 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699050 7fe9bae887a0 20 read_log 192674'417 (192674'416) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5880 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699065 7fe9bae887a0 20 read_log 192674'418 (192674'417) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5881 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699077 7fe9bae887a0 20 read_log 192674'419 (192674'418) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5882 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699090 7fe9bae887a0 20 read_log 192674'420 (192674'419) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5883 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699104 7fe9bae887a0 20 read_log 192674'421 (192674'420) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5884 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699124 7fe9bae887a0 20 read_log 192674'422 (192674'421) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5885 2014-07-31 18:03:47.835000 2014-08-13 17:38:02.699138 7fe9bae887a0 20 read_log 192674'423 (192674'422) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5886 2014-07-31 18:03:47.871000 2014-08-13 17:38:02.699151 7fe9bae887a0 20 read_log 192888'424 (0'0) modify 94d964aa/benchmark_data_storage0111-ib.pocstor_35625_object1907/head//22 by client.940330.0:1908 2014-08-11 16:29:48.198825 2014-08-13 17:38:02.699172 7fe9bae887a0 20 read_log 192888'425 (0'0) modify d2e02caa/benchmark_data_storage0111-ib.pocstor_35625_object2345/head//22 by client.940330.0:2346 2014-08-11 16:29:49.802854 2014-08-13 17:38:02.699185 7fe9bae887a0 20 read_log 192888'426 (0'0) modify 3f205caa/benchmark_data_storage0111-ib.pocstor_35625_object3104/head//22 by client.940330.0:3105 2014-08-11 16:29:52.947718 2014-08-13 17:38:02.699197 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 321694aa/benchmark_data_storage0111-ib.pocstor_35625_object3185/head//22 by client.940330.0:3186 2014-08-11 16:29:53.245566 2014-08-13 17:38:02.699209 7fe9bae887a0 20 read_log 192888'428 (0'0) modify f31f9caa/benchmark_data_storage0111-ib.pocstor_35625_object5271/head//22 by client.940330.0:5272 2014-08-11 16:30:00.987260 2014-08-13 17:38:02.699223 7fe9bae887a0 20 read_log 192888'429 (0'0) modify a28564aa/benchmark_data_storage0111-ib.pocstor_35625_object6086/head//22 by client.940330.0:6087 2014-08-11 16:30:03.889478 2014-08-13 17:38:02.699235 7fe9bae887a0 20 read_log 192888'430 (0'0) modify a4abdcaa/benchmark_data_storage0111-ib.pocstor_35625_object15804/head//22 by client.940330.0:15805 2014-08-11 16:30:40.498672 2014-08-13 17:38:02.699248 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 8d7974aa/benchmark_data_storage0111-ib.pocstor_35625_object17338/head//22 by client.940330.0:17339 2014-08-11 16:30:46.229075 2014-08-13 17:38:02.699260 7fe9bae887a0 20 read_log 192888'432 (0'0) modify dd609caa/benchmark_data_storage0111-ib.pocstor_35625_object18795/head//22 by client.940330.0:18796 2014-08-11 16:30:51.920245 2014-08-13 17:38:02.699273 7fe9bae887a0 20 read_log 192888'433 (0'0) modify f8b78caa/benchmark_data_storage0111-ib.pocstor_35625_object24492/head//22 by client.940330.0:24493 2014-08-11 16:31:13.945173 2014-08-13 17:38:02.699284 7fe9bae887a0 20 read_log 192888'434 (0'0) modify b5bf4caa/benchmark_data_storage0111-ib.pocstor_35625_object25590/head//22 by client.940330.0:25591 2014-08-11 16:31:18.187183 2014-08-13 17:38:02.699296 7fe9bae887a0 20 read_log 192888'435 (0'0) modify d91b5caa/benchmark_data_storage0111-ib.pocstor_35625_object27134/head//22 by client.940330.0:27135 2014-08-11 16:31:24.319755 2014-08-13 17:38:02.699332 7fe9bae887a0 20 read_log 192888'436 (0'0) modify ec22b4aa/benchmark_data_storage0111-ib.pocstor_35625_object28562/head//22 by client.940330.0:28563 2014-08-11 16:31:29.905926 2014-08-13 17:38:02.699345 7fe9bae887a0 20 read_log 192888'437 (0'0) modify bff364aa/benchmark_data_storage0111-ib.pocstor_35625_object30784/head//22 by client.940330.0:30785 2014-08-11 16:31:38.408271 2014-08-13 17:38:02.699357 7fe9bae887a0 20 read_log 192888'438 (192888'424) delete 94d964aa/benchmark_data_storage0111-ib.pocstor_35625_object1907/head//22 by client.940330.0:33215 2014-08-11 16:31:41.967945 2014-08-13 17:38:02.699369 7fe9bae887a0 20 read_log 192888'439 (192888'425) delete d2e02caa/benchmark_data_storage0111-ib.pocstor_35625_object2345/head//22 by client.940330.0:33653 2014-08-11 16:31:42.081424 2014-08-13 17:38:02.699382 7fe9bae887a0 20 read_log 192888'440 (192888'426) delete 3f205caa/benchmark_data_storage0111-ib.pocstor_35625_object3104/head//22 by client.940330.0:34412 2014-08-11 16:31:42.279530 2014-08-13 17:38:02.699394 7fe9bae887a0 20 read_log 192888'441 (192888'427) delete 321694aa/benchmark_data_storage0111-ib.pocstor_35625_object3185/head//22 by client.940330.0:34493 2014-08-11 16:31:42.304669 2014-08-13 17:38:02.699414 7fe9bae887a0 20 read_log 192888'442 (192888'428) delete f31f9caa/benchmark_data_storage0111-ib.pocstor_35625_object5271/head//22 by client.940330.0:36579 2014-08-11 16:31:42.885612 2014-08-13 17:38:02.699429 7fe9bae887a0 20 read_log 192888'443 (192888'429) delete a28564aa/benchmark_data_storage0111-ib.pocstor_35625_object6086/head//22 by client.940330.0:37394 2014-08-11 16:31:43.099669 2014-08-13 17:38:02.699441 7fe9bae887a0 20 read_log 192888'444 (192888'430) delete a4abdcaa/benchmark_data_storage0111-ib.pocstor_35625_object15804/head//22 by client.940330.0:47112 2014-08-11 16:31:45.673141 2014-08-13 17:38:02.699454 7fe9bae887a0 20 read_log 192888'445 (192888'431) delete 8d7974aa/benchmark_data_storage0111-ib.pocstor_35625_object17338/head//22 by client.940330.0:48646 2014-08-11 16:31:46.085531 2014-08-13 17:38:02.699466 7fe9bae887a0 20 read_log 192888'446 (192888'432) delete dd609caa/benchmark_data_storage0111-ib.pocstor_35625_object18795/head//22 by client.940330.0:50103 2014-08-11 16:31:46.525223 2014-08-13 17:38:02.699478 7fe9bae887a0 20 read_log 192888'447 (192888'433) delete f8b78caa/benchmark_data_storage0111-ib.pocstor_35625_object24492/head//22 by client.940330.0:55800 2014-08-11 16:31:48.184714 2014-08-13 17:38:02.699490 7fe9bae887a0 20 read_log 192888'448 (192888'434) delete b5bf4caa/benchmark_data_storage0111-ib.pocstor_35625_object25590/head//22 by client.940330.0:56898 2014-08-11 16:31:48.529000 2014-08-13 17:38:02.699502 7fe9bae887a0 20 read_log 192888'449 (192888'435) delete d91b5caa/benchmark_data_storage0111-ib.pocstor_35625_object27134/head//22 by client.940330.0:58442 2014-08-11 16:31:48.962032 2014-08-13 17:38:02.699514 7fe9bae887a0 20 read_log 192888'450 (192888'436) delete ec22b4aa/benchmark_data_storage0111-ib.pocstor_35625_object28562/head//22 by client.940330.0:59870 2014-08-11 16:31:49.353460 2014-08-13 17:38:02.699526 7fe9bae887a0 20 read_log 192888'451 (192888'437) delete bff364aa/benchmark_data_storage0111-ib.pocstor_35625_object30784/head//22 by client.940330.0:62092 2014-08-11 16:31:49.985182 2014-08-13 17:38:02.699538 7fe9bae887a0 20 read_log 194215'452 (0'0) modify d3493caa/benchmark_data_storage0111-ib.pocstor_64498_object1388/head//22 by client.1049635.0:1389 2014-08-12 17:17:21.657762 2014-08-13 17:38:02.699550 7fe9bae887a0 20 read_log 194215'453 (0'0) modify 2f813caa/benchmark_data_storage0111-ib.pocstor_64498_object1557/head//22 by client.1049635.0:1558 2014-08-12 17:17:22.170318 2014-08-13 17:38:02.699562 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 142d74aa/benchmark_data_storage0111-ib.pocstor_64498_object2336/head//22 by client.1049635.0:2337 2014-08-12 17:17:24.658250 2014-08-13 17:38:02.699574 7fe9bae887a0 20 read_log 194215'455 (0'0) modify 9e73a4aa/benchmark_data_storage0111-ib.pocstor_64498_object4572/head//22 by client.1049635.0:4573 2014-08-12 17:17:31.696988 2014-08-13 17:38:02.699586 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 3ffaecaa/benchmark_data_storage0111-ib.pocstor_64498_object5431/head//22 by client.1049635.0:5432 2014-08-12 17:17:34.312310 2014-08-13 17:38:02.699598 7fe9bae887a0 20 read_log 194215'457 (0'0) modify fab7fcaa/benchmark_data_storage0111-ib.pocstor_64498_object11361/head//22 by client.1049635.0:11362 2014-08-12 17:17:52.361725 2014-08-13 17:38:02.699610 7fe9bae887a0 20 read_log 194215'458 (0'0) modify ad684caa/benchmark_data_storage0111-ib.pocstor_64498_object13689/head//22 by client.1049635.0:13690 2014-08-12 17:17:59.617001 2014-08-13 17:38:02.699622 7fe9bae887a0 20 read_log 194215'459 (0'0) modify d7bd84aa/benchmark_data_storage0111-ib.pocstor_64498_object17483/head//22 by client.1049635.0:17484 2014-08-12 17:18:13.201423 2014-08-13 17:38:02.699635 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 96d0a4aa/benchmark_data_storage0111-ib.pocstor_64498_object19126/head//22 by client.1049635.0:19127 2014-08-12 17:18:19.494667 2014-08-13 17:38:02.699648 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 4dd09caa/benchmark_data_storage0111-ib.pocstor_64498_object23563/head//22 by client.1049635.0:23564 2014-08-12 17:18:36.228407 2014-08-13 17:38:02.699660 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 69484caa/benchmark_data_storage0111-ib.pocstor_64498_object23826/head//22 by client.1049635.0:23827 2014-08-12 17:18:37.207392 2014-08-13 17:38:02.699672 7fe9bae887a0 20 read_log 194215'463 (194215'452) delete d3493caa/benchmark_data_storage0111-ib.pocstor_64498_object1388/head//22 by client.1049635.0:35591 2014-08-12 17:19:18.448214 2014-08-13 17:38:02.699684 7fe9bae887a0 20 read_log 194215'464 (194215'453) delete 2f813caa/benchmark_data_storage0111-ib.pocstor_64498_object1557/head//22 by client.1049635.0:35760 2014-08-12 17:19:18.506422 2014-08-13 17:38:02.699696 7fe9bae887a0 20 read_log 194215'465 (194215'454) delete 142d74aa/benchmark_data_storage0111-ib.pocstor_64498_object2336/head//22 by client.1049635.0:36539 2014-08-12 17:19:18.749574 2014-08-13 17:38:02.699708 7fe9bae887a0 20 read_log 194215'466 (194215'455) delete 9e73a4aa/benchmark_data_storage0111-ib.pocstor_64498_object4572/head//22 by client.1049635.0:38775 2014-08-12 17:19:19.460370 2014-08-13 17:38:02.699720 7fe9bae887a0 20 read_log 194215'467 (194215'456) delete 3ffaecaa/benchmark_data_storage0111-ib.pocstor_64498_object5431/head//22 by client.1049635.0:39634 2014-08-12 17:19:19.748345 2014-08-13 17:38:02.699732 7fe9bae887a0 20 read_log 194215'468 (194215'457) delete fab7fcaa/benchmark_data_storage0111-ib.pocstor_64498_object11361/head//22 by client.1049635.0:45564 2014-08-12 17:19:21.691808 2014-08-13 17:38:02.699745 7fe9bae887a0 20 read_log 194215'469 (194215'458) delete ad684caa/benchmark_data_storage0111-ib.pocstor_64498_object13689/head//22 by client.1049635.0:47892 2014-08-12 17:19:22.468399 2014-08-13 17:38:02.699758 7fe9bae887a0 20 read_log 194215'470 (194215'459) delete d7bd84aa/benchmark_data_storage0111-ib.pocstor_64498_object17483/head//22 by client.1049635.0:51686 2014-08-12 17:19:23.661793 2014-08-13 17:38:02.699769 7fe9bae887a0 20 read_log 194215'471 (194215'460) delete 96d0a4aa/benchmark_data_storage0111-ib.pocstor_64498_object19126/head//22 by client.1049635.0:53329 2014-08-12 17:19:24.209204 2014-08-13 17:38:02.699781 7fe9bae887a0 20 read_log 194215'472 (194215'461) delete 4dd09caa/benchmark_data_storage0111-ib.pocstor_64498_object23563/head//22 by client.1049635.0:57766 2014-08-12 17:19:25.819288 2014-08-13 17:38:02.699793 7fe9bae887a0 20 read_log 194215'473 (194215'462) delete 69484caa/benchmark_data_storage0111-ib.pocstor_64498_object23826/head//22 by client.1049635.0:58029 2014-08-12 17:19:25.904558 2014-08-13 17:38:02.699870 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.699887 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=0 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.699902 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=0 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Initial 0.001525 0 0.000000 2014-08-13 17:38:02.699914 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=0 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.699922 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=0 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.699931 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] log((0'0,194215'473], crt=192888'451) 2014-08-13 17:38:02.699940 7fe9bae887a0 10 osd.4 194478 pgid 22.4df coll 22.4df_head 2014-08-13 17:38:02.700052 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.4df 2014-08-13 17:38:02.700061 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.700088 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4df(unlocked)] enter Initial 2014-08-13 17:38:02.700104 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4df(unlocked)] enter NotTrimming 2014-08-13 17:38:02.700230 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.700334 7fe9bae887a0 20 read_log 191648'7 (0'0) modify c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 by client.273158.0:417 2014-07-30 14:08:50.342842 2014-08-13 17:38:02.700354 7fe9bae887a0 20 read_log 191648'34 (0'0) modify cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 by client.273158.0:1923 2014-07-30 14:08:55.972827 2014-08-13 17:38:02.700367 7fe9bae887a0 20 read_log 191648'37 (0'0) modify 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 by client.273158.0:2195 2014-07-30 14:08:57.141519 2014-08-13 17:38:02.700379 7fe9bae887a0 20 read_log 191648'39 (0'0) modify 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 by client.273158.0:2228 2014-07-30 14:08:57.262622 2014-08-13 17:38:02.700391 7fe9bae887a0 20 read_log 191648'101 (0'0) modify 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 by client.273158.0:6390 2014-07-30 14:09:18.140107 2014-08-13 17:38:02.700403 7fe9bae887a0 20 read_log 191648'125 (0'0) modify d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 by client.273158.0:8184 2014-07-30 14:09:27.078589 2014-08-13 17:38:02.700415 7fe9bae887a0 20 read_log 191648'140 (0'0) modify 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 by client.273158.0:9292 2014-07-30 14:09:32.357356 2014-08-13 17:38:02.700427 7fe9bae887a0 20 read_log 191648'149 (0'0) modify 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 by client.273158.0:10005 2014-07-30 14:09:35.864018 2014-08-13 17:38:02.700439 7fe9bae887a0 20 read_log 191648'191 (0'0) modify 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 by client.273158.0:12482 2014-07-30 14:09:49.937056 2014-08-13 17:38:02.700451 7fe9bae887a0 20 read_log 191648'209 (0'0) modify 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 by client.273158.0:13383 2014-07-30 14:09:54.156483 2014-08-13 17:38:02.700462 7fe9bae887a0 20 read_log 191648'212 (0'0) modify dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 by client.273158.0:13431 2014-07-30 14:09:54.428087 2014-08-13 17:38:02.700474 7fe9bae887a0 20 read_log 191648'318 (0'0) modify a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 by client.273158.0:21241 2014-07-30 14:10:38.249873 2014-08-13 17:38:02.700486 7fe9bae887a0 20 read_log 191648'319 (0'0) modify 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 by client.273158.0:21297 2014-07-30 14:10:38.473365 2014-08-13 17:38:02.700507 7fe9bae887a0 20 read_log 191648'326 (0'0) modify 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 by client.273158.0:21669 2014-07-30 14:10:41.133320 2014-08-13 17:38:02.700520 7fe9bae887a0 20 read_log 191648'329 (0'0) modify 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 by client.273158.0:21822 2014-07-30 14:10:41.737108 2014-08-13 17:38:02.700533 7fe9bae887a0 20 read_log 191727'355 (0'0) modify 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 by client.275175.0:2976 2014-07-30 14:18:44.142832 2014-08-13 17:38:02.700545 7fe9bae887a0 20 read_log 191727'358 (0'0) modify e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 by client.275175.0:3589 2014-07-30 14:18:46.073779 2014-08-13 17:38:02.700557 7fe9bae887a0 20 read_log 191727'360 (0'0) modify 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 by client.275175.0:3807 2014-07-30 14:18:46.759267 2014-08-13 17:38:02.700570 7fe9bae887a0 20 read_log 191727'368 (0'0) modify 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 by client.275175.0:7295 2014-07-30 14:18:57.554106 2014-08-13 17:38:02.700583 7fe9bae887a0 20 read_log 191727'377 (0'0) modify fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 by client.275175.0:9108 2014-07-30 14:19:02.656325 2014-08-13 17:38:02.700595 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 by client.275175.0:9203 2014-07-30 14:19:02.929198 2014-08-13 17:38:02.700606 7fe9bae887a0 20 read_log 191727'379 (0'0) modify b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 by client.275175.0:10101 2014-07-30 14:19:05.461112 2014-08-13 17:38:02.700618 7fe9bae887a0 20 read_log 191727'381 (0'0) modify 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 by client.275175.0:11109 2014-07-30 14:19:08.346682 2014-08-13 17:38:02.700630 7fe9bae887a0 20 read_log 191727'382 (0'0) modify a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 by client.275175.0:13112 2014-07-30 14:19:15.008064 2014-08-13 17:38:02.700642 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 by client.275175.0:14090 2014-07-30 14:19:18.131063 2014-08-13 17:38:02.700654 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 by client.275175.0:17352 2014-07-30 14:19:28.134585 2014-08-13 17:38:02.700666 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 by client.275175.0:20351 2014-07-30 14:19:37.271027 2014-08-13 17:38:02.700677 7fe9bae887a0 20 read_log 191727'406 (0'0) modify a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 by client.275175.0:23326 2014-07-30 14:19:46.949147 2014-08-13 17:38:02.700689 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 by client.275175.0:26110 2014-07-30 14:19:55.174260 2014-08-13 17:38:02.700701 7fe9bae887a0 20 read_log 191727'413 (0'0) modify c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 by client.275175.0:26590 2014-07-30 14:19:56.609822 2014-08-13 17:38:02.700713 7fe9bae887a0 20 read_log 191727'414 (0'0) modify 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 by client.275175.0:27000 2014-07-30 14:19:57.810192 2014-08-13 17:38:02.700724 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 by client.275175.0:27589 2014-07-30 14:19:59.484122 2014-08-13 17:38:02.700743 7fe9bae887a0 20 read_log 191727'418 (0'0) modify d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 by client.275175.0:28171 2014-07-30 14:20:01.508142 2014-08-13 17:38:02.700775 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 by client.275175.0:35530 2014-07-30 14:20:23.524439 2014-08-13 17:38:02.700791 7fe9bae887a0 20 read_log 191727'431 (0'0) modify ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 by client.275175.0:37978 2014-07-30 14:20:31.179032 2014-08-13 17:38:02.700803 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 9efca4df/benchmark_data_storage0111-ib.pocstor_35625_object49/head//22 by client.940330.0:50 2014-08-11 16:29:40.666238 2014-08-13 17:38:02.700815 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 6fbf6cdf/benchmark_data_storage0111-ib.pocstor_35625_object572/head//22 by client.940330.0:573 2014-08-11 16:29:43.011003 2014-08-13 17:38:02.700829 7fe9bae887a0 20 read_log 192888'436 (0'0) modify a4dc74df/benchmark_data_storage0111-ib.pocstor_35625_object5561/head//22 by client.940330.0:5562 2014-08-11 16:30:02.016590 2014-08-13 17:38:02.700842 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 492c9cdf/benchmark_data_storage0111-ib.pocstor_35625_object6312/head//22 by client.940330.0:6313 2014-08-11 16:30:04.683541 2014-08-13 17:38:02.700856 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 9ceb7cdf/benchmark_data_storage0111-ib.pocstor_35625_object7989/head//22 by client.940330.0:7990 2014-08-11 16:30:10.800237 2014-08-13 17:38:02.700868 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 5ca1cdf/benchmark_data_storage0111-ib.pocstor_35625_object10740/head//22 by client.940330.0:10741 2014-08-11 16:30:20.791227 2014-08-13 17:38:02.700882 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 511044df/benchmark_data_storage0111-ib.pocstor_35625_object11089/head//22 by client.940330.0:11090 2014-08-11 16:30:22.145716 2014-08-13 17:38:02.700894 7fe9bae887a0 20 read_log 192888'441 (0'0) modify e58fb4df/benchmark_data_storage0111-ib.pocstor_35625_object15023/head//22 by client.940330.0:15024 2014-08-11 16:30:37.580948 2014-08-13 17:38:02.700905 7fe9bae887a0 20 read_log 192888'442 (0'0) modify e5936cdf/benchmark_data_storage0111-ib.pocstor_35625_object16272/head//22 by client.940330.0:16273 2014-08-11 16:30:42.246174 2014-08-13 17:38:02.700917 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 39a0b4df/benchmark_data_storage0111-ib.pocstor_35625_object17400/head//22 by client.940330.0:17401 2014-08-11 16:30:46.458370 2014-08-13 17:38:02.700929 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 90f5c4df/benchmark_data_storage0111-ib.pocstor_35625_object18527/head//22 by client.940330.0:18528 2014-08-11 16:30:50.868250 2014-08-13 17:38:02.700941 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 61343cdf/benchmark_data_storage0111-ib.pocstor_35625_object18817/head//22 by client.940330.0:18818 2014-08-11 16:30:51.999198 2014-08-13 17:38:02.700955 7fe9bae887a0 20 read_log 192888'446 (0'0) modify d30cd4df/benchmark_data_storage0111-ib.pocstor_35625_object21466/head//22 by client.940330.0:21467 2014-08-11 16:31:02.555551 2014-08-13 17:38:02.700967 7fe9bae887a0 20 read_log 192888'447 (0'0) modify ef929cdf/benchmark_data_storage0111-ib.pocstor_35625_object28670/head//22 by client.940330.0:28671 2014-08-11 16:31:30.329470 2014-08-13 17:38:02.700981 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 6f6e34df/benchmark_data_storage0111-ib.pocstor_35625_object29034/head//22 by client.940330.0:29035 2014-08-11 16:31:31.811822 2014-08-13 17:38:02.700993 7fe9bae887a0 20 read_log 192888'449 (0'0) modify 637db4df/benchmark_data_storage0111-ib.pocstor_35625_object30208/head//22 by client.940330.0:30209 2014-08-11 16:31:36.236705 2014-08-13 17:38:02.701015 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 791f7cdf/benchmark_data_storage0111-ib.pocstor_35625_object31220/head//22 by client.940330.0:31221 2014-08-11 16:31:40.095813 2014-08-13 17:38:02.701030 7fe9bae887a0 20 read_log 192888'451 (192888'434) delete 9efca4df/benchmark_data_storage0111-ib.pocstor_35625_object49/head//22 by client.940330.0:31357 2014-08-11 16:31:41.459217 2014-08-13 17:38:02.701042 7fe9bae887a0 20 read_log 192888'452 (192888'435) delete 6fbf6cdf/benchmark_data_storage0111-ib.pocstor_35625_object572/head//22 by client.940330.0:31880 2014-08-11 16:31:41.644861 2014-08-13 17:38:02.701056 7fe9bae887a0 20 read_log 192888'453 (192888'436) delete a4dc74df/benchmark_data_storage0111-ib.pocstor_35625_object5561/head//22 by client.940330.0:36869 2014-08-11 16:31:42.959155 2014-08-13 17:38:02.701071 7fe9bae887a0 20 read_log 192888'454 (192888'437) delete 492c9cdf/benchmark_data_storage0111-ib.pocstor_35625_object6312/head//22 by client.940330.0:37620 2014-08-11 16:31:43.160463 2014-08-13 17:38:02.701083 7fe9bae887a0 20 read_log 192888'455 (192888'438) delete 9ceb7cdf/benchmark_data_storage0111-ib.pocstor_35625_object7989/head//22 by client.940330.0:39297 2014-08-11 16:31:43.587256 2014-08-13 17:38:02.701097 7fe9bae887a0 20 read_log 192888'456 (192888'439) delete 5ca1cdf/benchmark_data_storage0111-ib.pocstor_35625_object10740/head//22 by client.940330.0:42048 2014-08-11 16:31:44.324738 2014-08-13 17:38:02.701109 7fe9bae887a0 20 read_log 192888'457 (192888'440) delete 511044df/benchmark_data_storage0111-ib.pocstor_35625_object11089/head//22 by client.940330.0:42397 2014-08-11 16:31:44.423895 2014-08-13 17:38:02.701123 7fe9bae887a0 20 read_log 192888'458 (192888'441) delete e58fb4df/benchmark_data_storage0111-ib.pocstor_35625_object15023/head//22 by client.940330.0:46331 2014-08-11 16:31:45.463375 2014-08-13 17:38:02.701135 7fe9bae887a0 20 read_log 192888'459 (192888'442) delete e5936cdf/benchmark_data_storage0111-ib.pocstor_35625_object16272/head//22 by client.940330.0:47580 2014-08-11 16:31:45.796683 2014-08-13 17:38:02.701150 7fe9bae887a0 20 read_log 192888'460 (192888'443) delete 39a0b4df/benchmark_data_storage0111-ib.pocstor_35625_object17400/head//22 by client.940330.0:48708 2014-08-11 16:31:46.101825 2014-08-13 17:38:02.701170 7fe9bae887a0 20 read_log 192888'461 (192888'444) delete 90f5c4df/benchmark_data_storage0111-ib.pocstor_35625_object18527/head//22 by client.940330.0:49835 2014-08-11 16:31:46.453477 2014-08-13 17:38:02.701183 7fe9bae887a0 20 read_log 192888'462 (192888'445) delete 61343cdf/benchmark_data_storage0111-ib.pocstor_35625_object18817/head//22 by client.940330.0:50125 2014-08-11 16:31:46.532344 2014-08-13 17:38:02.701195 7fe9bae887a0 20 read_log 192888'463 (192888'446) delete d30cd4df/benchmark_data_storage0111-ib.pocstor_35625_object21466/head//22 by client.940330.0:52774 2014-08-11 16:31:47.285847 2014-08-13 17:38:02.701207 7fe9bae887a0 20 read_log 192888'464 (192888'447) delete ef929cdf/benchmark_data_storage0111-ib.pocstor_35625_object28670/head//22 by client.940330.0:59978 2014-08-11 16:31:49.382148 2014-08-13 17:38:02.701219 7fe9bae887a0 20 read_log 192888'465 (192888'448) delete 6f6e34df/benchmark_data_storage0111-ib.pocstor_35625_object29034/head//22 by client.940330.0:60342 2014-08-11 16:31:49.481296 2014-08-13 17:38:02.701233 7fe9bae887a0 20 read_log 192888'466 (192888'449) delete 637db4df/benchmark_data_storage0111-ib.pocstor_35625_object30208/head//22 by client.940330.0:61516 2014-08-11 16:31:49.816975 2014-08-13 17:38:02.701245 7fe9bae887a0 20 read_log 192888'467 (192888'450) delete 791f7cdf/benchmark_data_storage0111-ib.pocstor_35625_object31220/head//22 by client.940330.0:62528 2014-08-11 16:31:50.127007 2014-08-13 17:38:02.701259 7fe9bae887a0 20 read_log 194215'468 (0'0) modify fd3654df/benchmark_data_storage0111-ib.pocstor_64498_object982/head//22 by client.1049635.0:983 2014-08-12 17:17:20.297293 2014-08-13 17:38:02.701296 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 55040cdf/benchmark_data_storage0111-ib.pocstor_64498_object1167/head//22 by client.1049635.0:1168 2014-08-12 17:17:20.953796 2014-08-13 17:38:02.701312 7fe9bae887a0 20 read_log 194215'470 (0'0) modify d01aa4df/benchmark_data_storage0111-ib.pocstor_64498_object3257/head//22 by client.1049635.0:3258 2014-08-12 17:17:27.694100 2014-08-13 17:38:02.701324 7fe9bae887a0 20 read_log 194215'471 (0'0) modify c581ecdf/benchmark_data_storage0111-ib.pocstor_64498_object5466/head//22 by client.1049635.0:5467 2014-08-12 17:17:34.475010 2014-08-13 17:38:02.701337 7fe9bae887a0 20 read_log 194215'472 (0'0) modify e54744df/benchmark_data_storage0111-ib.pocstor_64498_object6134/head//22 by client.1049635.0:6135 2014-08-12 17:17:36.491274 2014-08-13 17:38:02.701351 7fe9bae887a0 20 read_log 194215'473 (0'0) modify f7e44df/benchmark_data_storage0111-ib.pocstor_64498_object7395/head//22 by client.1049635.0:7396 2014-08-12 17:17:40.239530 2014-08-13 17:38:02.701364 7fe9bae887a0 20 read_log 194215'474 (0'0) modify f4af5cdf/benchmark_data_storage0111-ib.pocstor_64498_object13769/head//22 by client.1049635.0:13770 2014-08-12 17:17:59.848404 2014-08-13 17:38:02.701376 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 30962cdf/benchmark_data_storage0111-ib.pocstor_64498_object15259/head//22 by client.1049635.0:15260 2014-08-12 17:18:04.831917 2014-08-13 17:38:02.701388 7fe9bae887a0 20 read_log 194215'476 (0'0) modify fcb474df/benchmark_data_storage0111-ib.pocstor_64498_object15865/head//22 by client.1049635.0:15866 2014-08-12 17:18:07.047647 2014-08-13 17:38:02.701400 7fe9bae887a0 20 read_log 194215'477 (0'0) modify f66fdcdf/benchmark_data_storage0111-ib.pocstor_64498_object19484/head//22 by client.1049635.0:19485 2014-08-12 17:18:20.849600 2014-08-13 17:38:02.701412 7fe9bae887a0 20 read_log 194215'478 (0'0) modify eed494df/benchmark_data_storage0111-ib.pocstor_64498_object22229/head//22 by client.1049635.0:22230 2014-08-12 17:18:31.197050 2014-08-13 17:38:02.701424 7fe9bae887a0 20 read_log 194215'479 (0'0) modify c6719cdf/benchmark_data_storage0111-ib.pocstor_64498_object25872/head//22 by client.1049635.0:25873 2014-08-12 17:18:44.948569 2014-08-13 17:38:02.701436 7fe9bae887a0 20 read_log 194215'480 (0'0) modify a9cf34df/benchmark_data_storage0111-ib.pocstor_64498_object30446/head//22 by client.1049635.0:30447 2014-08-12 17:19:02.297618 2014-08-13 17:38:02.701448 7fe9bae887a0 20 read_log 194215'481 (0'0) modify ff0264df/benchmark_data_storage0111-ib.pocstor_64498_object31710/head//22 by client.1049635.0:31711 2014-08-12 17:19:07.451430 2014-08-13 17:38:02.701460 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 4eab34df/benchmark_data_storage0111-ib.pocstor_64498_object33007/head//22 by client.1049635.0:33008 2014-08-12 17:19:12.417594 2014-08-13 17:38:02.701472 7fe9bae887a0 20 read_log 194215'483 (194215'468) delete fd3654df/benchmark_data_storage0111-ib.pocstor_64498_object982/head//22 by client.1049635.0:35185 2014-08-12 17:19:18.314503 2014-08-13 17:38:02.701484 7fe9bae887a0 20 read_log 194215'484 (194215'469) delete 55040cdf/benchmark_data_storage0111-ib.pocstor_64498_object1167/head//22 by client.1049635.0:35370 2014-08-12 17:19:18.374305 2014-08-13 17:38:02.701496 7fe9bae887a0 20 read_log 194215'485 (194215'470) delete d01aa4df/benchmark_data_storage0111-ib.pocstor_64498_object3257/head//22 by client.1049635.0:37460 2014-08-12 17:19:19.046850 2014-08-13 17:38:02.701509 7fe9bae887a0 20 read_log 194215'486 (194215'471) delete c581ecdf/benchmark_data_storage0111-ib.pocstor_64498_object5466/head//22 by client.1049635.0:39669 2014-08-12 17:19:19.759466 2014-08-13 17:38:02.701522 7fe9bae887a0 20 read_log 194215'487 (194215'472) delete e54744df/benchmark_data_storage0111-ib.pocstor_64498_object6134/head//22 by client.1049635.0:40337 2014-08-12 17:19:20.001142 2014-08-13 17:38:02.701534 7fe9bae887a0 20 read_log 194215'488 (194215'473) delete f7e44df/benchmark_data_storage0111-ib.pocstor_64498_object7395/head//22 by client.1049635.0:41598 2014-08-12 17:19:20.415453 2014-08-13 17:38:02.701548 7fe9bae887a0 20 read_log 194215'489 (194215'474) delete f4af5cdf/benchmark_data_storage0111-ib.pocstor_64498_object13769/head//22 by client.1049635.0:47972 2014-08-12 17:19:22.494377 2014-08-13 17:38:02.701560 7fe9bae887a0 20 read_log 194215'490 (194215'475) delete 30962cdf/benchmark_data_storage0111-ib.pocstor_64498_object15259/head//22 by client.1049635.0:49462 2014-08-12 17:19:22.956087 2014-08-13 17:38:02.701572 7fe9bae887a0 20 read_log 194215'491 (194215'476) delete fcb474df/benchmark_data_storage0111-ib.pocstor_64498_object15865/head//22 by client.1049635.0:50068 2014-08-12 17:19:23.145147 2014-08-13 17:38:02.701584 7fe9bae887a0 20 read_log 194215'492 (194215'477) delete f66fdcdf/benchmark_data_storage0111-ib.pocstor_64498_object19484/head//22 by client.1049635.0:53687 2014-08-12 17:19:24.344397 2014-08-13 17:38:02.701596 7fe9bae887a0 20 read_log 194215'493 (194215'478) delete eed494df/benchmark_data_storage0111-ib.pocstor_64498_object22229/head//22 by client.1049635.0:56432 2014-08-12 17:19:25.317710 2014-08-13 17:38:02.701608 7fe9bae887a0 20 read_log 194215'494 (194215'479) delete c6719cdf/benchmark_data_storage0111-ib.pocstor_64498_object25872/head//22 by client.1049635.0:60075 2014-08-12 17:19:26.616397 2014-08-13 17:38:02.701620 7fe9bae887a0 20 read_log 194215'495 (194215'480) delete a9cf34df/benchmark_data_storage0111-ib.pocstor_64498_object30446/head//22 by client.1049635.0:64649 2014-08-12 17:19:28.189539 2014-08-13 17:38:02.701632 7fe9bae887a0 20 read_log 194215'496 (194215'481) delete ff0264df/benchmark_data_storage0111-ib.pocstor_64498_object31710/head//22 by client.1049635.0:65913 2014-08-12 17:19:28.644050 2014-08-13 17:38:02.701646 7fe9bae887a0 20 read_log 194215'497 (194215'482) delete 4eab34df/benchmark_data_storage0111-ib.pocstor_64498_object33007/head//22 by client.1049635.0:67210 2014-08-12 17:19:29.118303 2014-08-13 17:38:02.701727 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.701746 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=0 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.701762 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=0 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001675 0 0.000000 2014-08-13 17:38:02.701774 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=0 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.701782 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=0 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.701791 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'497], crt=191727'433) 2014-08-13 17:38:02.701800 7fe9bae887a0 10 osd.4 194478 pgid 22.4fe coll 22.4fe_head 2014-08-13 17:38:02.701922 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.4fe 2014-08-13 17:38:02.701933 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.701979 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4fe(unlocked)] enter Initial 2014-08-13 17:38:02.701989 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4fe(unlocked)] enter NotTrimming 2014-08-13 17:38:02.702131 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.702271 7fe9bae887a0 20 read_log 191648'4 (0'0) modify 58b384fe/benchmark_data_storage0111-ib.pocstor_40745_object471/head//22 by client.273158.0:472 2014-07-30 14:08:50.543500 2014-08-13 17:38:02.702292 7fe9bae887a0 20 read_log 191648'5 (0'0) modify f07e4fe/benchmark_data_storage0111-ib.pocstor_40745_object524/head//22 by client.273158.0:525 2014-07-30 14:08:50.737009 2014-08-13 17:38:02.702305 7fe9bae887a0 20 read_log 191648'6 (0'0) modify eb685cfe/benchmark_data_storage0111-ib.pocstor_40745_object531/head//22 by client.273158.0:532 2014-07-30 14:08:50.763301 2014-08-13 17:38:02.702318 7fe9bae887a0 20 read_log 191648'8 (0'0) modify 1b7684fe/benchmark_data_storage0111-ib.pocstor_40745_object823/head//22 by client.273158.0:824 2014-07-30 14:08:51.836025 2014-08-13 17:38:02.702330 7fe9bae887a0 20 read_log 191648'16 (0'0) modify 9872acfe/benchmark_data_storage0111-ib.pocstor_40745_object1221/head//22 by client.273158.0:1222 2014-07-30 14:08:53.283920 2014-08-13 17:38:02.702342 7fe9bae887a0 20 read_log 191648'51 (0'0) modify 2e57a4fe/benchmark_data_storage0111-ib.pocstor_40745_object3406/head//22 by client.273158.0:3407 2014-07-30 14:09:03.260646 2014-08-13 17:38:02.702354 7fe9bae887a0 20 read_log 191648'125 (0'0) modify 6afcdcfe/benchmark_data_storage0111-ib.pocstor_40745_object8785/head//22 by client.273158.0:8786 2014-07-30 14:09:30.325059 2014-08-13 17:38:02.702366 7fe9bae887a0 20 read_log 191648'150 (0'0) modify dad8f4fe/benchmark_data_storage0111-ib.pocstor_40745_object10683/head//22 by client.273158.0:10684 2014-07-30 14:09:39.428634 2014-08-13 17:38:02.702378 7fe9bae887a0 20 read_log 191648'214 (0'0) modify 78d17cfe/benchmark_data_storage0111-ib.pocstor_40745_object14902/head//22 by client.273158.0:14903 2014-07-30 14:10:02.317253 2014-08-13 17:38:02.702391 7fe9bae887a0 20 read_log 191648'224 (0'0) modify be8f44fe/benchmark_data_storage0111-ib.pocstor_40745_object15617/head//22 by client.273158.0:15618 2014-07-30 14:10:05.362651 2014-08-13 17:38:02.702403 7fe9bae887a0 20 read_log 191648'304 (0'0) modify 5a7bbcfe/benchmark_data_storage0111-ib.pocstor_40745_object19788/head//22 by client.273158.0:19789 2014-07-30 14:10:30.509366 2014-08-13 17:38:02.702414 7fe9bae887a0 20 read_log 191648'323 (0'0) modify cc265cfe/benchmark_data_storage0111-ib.pocstor_40745_object20820/head//22 by client.273158.0:20821 2014-07-30 14:10:36.155940 2014-08-13 17:38:02.702426 7fe9bae887a0 20 read_log 191648'353 (0'0) modify 294a84fe/benchmark_data_storage0111-ib.pocstor_40745_object22653/head//22 by client.273158.0:22654 2014-07-30 14:10:47.690078 2014-08-13 17:38:02.702438 7fe9bae887a0 20 read_log 191727'357 (0'0) modify c2c2f4fe/benchmark_data_storage0111-ib.pocstor_51285_object1659/head//22 by client.275175.0:1660 2014-07-30 14:18:39.768504 2014-08-13 17:38:02.702451 7fe9bae887a0 20 read_log 191727'359 (0'0) modify 3363f4fe/benchmark_data_storage0111-ib.pocstor_51285_object2105/head//22 by client.275175.0:2106 2014-07-30 14:18:41.239850 2014-08-13 17:38:02.702462 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 4b0054fe/benchmark_data_storage0111-ib.pocstor_51285_object2330/head//22 by client.275175.0:2331 2014-07-30 14:18:41.977090 2014-08-13 17:38:02.702476 7fe9bae887a0 20 read_log 191727'369 (0'0) modify 7f6154fe/benchmark_data_storage0111-ib.pocstor_51285_object3571/head//22 by client.275175.0:3572 2014-07-30 14:18:46.022522 2014-08-13 17:38:02.702488 7fe9bae887a0 20 read_log 191727'371 (0'0) modify b62524fe/benchmark_data_storage0111-ib.pocstor_51285_object4359/head//22 by client.275175.0:4360 2014-07-30 14:18:48.436460 2014-08-13 17:38:02.702500 7fe9bae887a0 20 read_log 191727'379 (0'0) modify 77e26cfe/benchmark_data_storage0111-ib.pocstor_51285_object7964/head//22 by client.275175.0:7965 2014-07-30 14:18:59.424286 2014-08-13 17:38:02.702520 7fe9bae887a0 20 read_log 191727'380 (0'0) modify 57a63cfe/benchmark_data_storage0111-ib.pocstor_51285_object8099/head//22 by client.275175.0:8100 2014-07-30 14:18:59.800414 2014-08-13 17:38:02.702532 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 84e90cfe/benchmark_data_storage0111-ib.pocstor_51285_object10730/head//22 by client.275175.0:10731 2014-07-30 14:19:07.287570 2014-08-13 17:38:02.702544 7fe9bae887a0 20 read_log 191727'390 (0'0) modify cd05e4fe/benchmark_data_storage0111-ib.pocstor_51285_object12138/head//22 by client.275175.0:12139 2014-07-30 14:19:12.026464 2014-08-13 17:38:02.702556 7fe9bae887a0 20 read_log 191727'396 (0'0) modify a1374fe/benchmark_data_storage0111-ib.pocstor_51285_object18591/head//22 by client.275175.0:18592 2014-07-30 14:19:31.743005 2014-08-13 17:38:02.702568 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 354eacfe/benchmark_data_storage0111-ib.pocstor_51285_object19372/head//22 by client.275175.0:19373 2014-07-30 14:19:34.421185 2014-08-13 17:38:02.702599 7fe9bae887a0 20 read_log 191727'399 (0'0) modify 206d04fe/benchmark_data_storage0111-ib.pocstor_51285_object19678/head//22 by client.275175.0:19679 2014-07-30 14:19:35.269935 2014-08-13 17:38:02.702615 7fe9bae887a0 20 read_log 191727'404 (0'0) modify ac2bfcfe/benchmark_data_storage0111-ib.pocstor_51285_object21140/head//22 by client.275175.0:21141 2014-07-30 14:19:39.528982 2014-08-13 17:38:02.702627 7fe9bae887a0 20 read_log 191727'405 (0'0) modify 63cd54fe/benchmark_data_storage0111-ib.pocstor_51285_object21538/head//22 by client.275175.0:21539 2014-07-30 14:19:40.804188 2014-08-13 17:38:02.702639 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 2e27ecfe/benchmark_data_storage0111-ib.pocstor_51285_object23873/head//22 by client.275175.0:23874 2014-07-30 14:19:48.479183 2014-08-13 17:38:02.702650 7fe9bae887a0 20 read_log 191727'408 (0'0) modify e9c5d4fe/benchmark_data_storage0111-ib.pocstor_51285_object25026/head//22 by client.275175.0:25027 2014-07-30 14:19:51.760474 2014-08-13 17:38:02.702662 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 287344fe/benchmark_data_storage0111-ib.pocstor_51285_object25412/head//22 by client.275175.0:25413 2014-07-30 14:19:53.016988 2014-08-13 17:38:02.702674 7fe9bae887a0 20 read_log 191727'414 (0'0) modify a21dc4fe/benchmark_data_storage0111-ib.pocstor_51285_object26863/head//22 by client.275175.0:26864 2014-07-30 14:19:57.426501 2014-08-13 17:38:02.702686 7fe9bae887a0 20 read_log 191727'415 (0'0) modify f42634fe/benchmark_data_storage0111-ib.pocstor_51285_object27351/head//22 by client.275175.0:27352 2014-07-30 14:19:58.810621 2014-08-13 17:38:02.702698 7fe9bae887a0 20 read_log 191727'416 (0'0) modify f39decfe/benchmark_data_storage0111-ib.pocstor_51285_object28444/head//22 by client.275175.0:28445 2014-07-30 14:20:02.264942 2014-08-13 17:38:02.702710 7fe9bae887a0 20 read_log 191727'425 (0'0) modify cc6a34fe/benchmark_data_storage0111-ib.pocstor_51285_object34347/head//22 by client.275175.0:34348 2014-07-30 14:20:20.075351 2014-08-13 17:38:02.702723 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 19dd7cfe/benchmark_data_storage0111-ib.pocstor_51285_object38492/head//22 by client.275175.0:38493 2014-07-30 14:20:32.645282 2014-08-13 17:38:02.702737 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 2788bcfe/benchmark_data_storage0111-ib.pocstor_35625_object897/head//22 by client.940330.0:898 2014-08-11 16:29:44.300577 2014-08-13 17:38:02.702749 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 5d77cfe/benchmark_data_storage0111-ib.pocstor_35625_object2367/head//22 by client.940330.0:2368 2014-08-11 16:29:49.879704 2014-08-13 17:38:02.702761 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 9f4e7cfe/benchmark_data_storage0111-ib.pocstor_35625_object2460/head//22 by client.940330.0:2461 2014-08-11 16:29:50.284171 2014-08-13 17:38:02.702773 7fe9bae887a0 20 read_log 192888'433 (0'0) modify a463d4fe/benchmark_data_storage0111-ib.pocstor_35625_object12266/head//22 by client.940330.0:12267 2014-08-11 16:30:27.008081 2014-08-13 17:38:02.702793 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 856b94fe/benchmark_data_storage0111-ib.pocstor_35625_object13173/head//22 by client.940330.0:13174 2014-08-11 16:30:30.547326 2014-08-13 17:38:02.702809 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 639c1cfe/benchmark_data_storage0111-ib.pocstor_35625_object15473/head//22 by client.940330.0:15474 2014-08-11 16:30:39.265453 2014-08-13 17:38:02.702821 7fe9bae887a0 20 read_log 192888'436 (0'0) modify ea2dccfe/benchmark_data_storage0111-ib.pocstor_35625_object15539/head//22 by client.940330.0:15540 2014-08-11 16:30:39.502579 2014-08-13 17:38:02.702836 7fe9bae887a0 20 read_log 192888'437 (0'0) modify d3f8ccfe/benchmark_data_storage0111-ib.pocstor_35625_object18332/head//22 by client.940330.0:18333 2014-08-11 16:30:50.045756 2014-08-13 17:38:02.702849 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 7f25b4fe/benchmark_data_storage0111-ib.pocstor_35625_object18376/head//22 by client.940330.0:18377 2014-08-11 16:30:50.212848 2014-08-13 17:38:02.702863 7fe9bae887a0 20 read_log 192888'439 (0'0) modify a96f0cfe/benchmark_data_storage0111-ib.pocstor_35625_object24128/head//22 by client.940330.0:24129 2014-08-11 16:31:12.614270 2014-08-13 17:38:02.702875 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 875b14fe/benchmark_data_storage0111-ib.pocstor_35625_object24796/head//22 by client.940330.0:24797 2014-08-11 16:31:15.079797 2014-08-13 17:38:02.702887 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 1c68c4fe/benchmark_data_storage0111-ib.pocstor_35625_object28381/head//22 by client.940330.0:28382 2014-08-11 16:31:29.190103 2014-08-13 17:38:02.702900 7fe9bae887a0 20 read_log 192888'442 (0'0) modify fac364fe/benchmark_data_storage0111-ib.pocstor_35625_object28439/head//22 by client.940330.0:28440 2014-08-11 16:31:29.421054 2014-08-13 17:38:02.702912 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 74f5e4fe/benchmark_data_storage0111-ib.pocstor_35625_object29630/head//22 by client.940330.0:29631 2014-08-11 16:31:34.062030 2014-08-13 17:38:02.702925 7fe9bae887a0 20 read_log 192888'444 (0'0) modify c1f7dcfe/benchmark_data_storage0111-ib.pocstor_35625_object30504/head//22 by client.940330.0:30505 2014-08-11 16:31:37.368485 2014-08-13 17:38:02.702939 7fe9bae887a0 20 read_log 192888'445 (192888'430) delete 2788bcfe/benchmark_data_storage0111-ib.pocstor_35625_object897/head//22 by client.940330.0:32205 2014-08-11 16:31:41.724590 2014-08-13 17:38:02.702952 7fe9bae887a0 20 read_log 192888'446 (192888'431) delete 5d77cfe/benchmark_data_storage0111-ib.pocstor_35625_object2367/head//22 by client.940330.0:33675 2014-08-11 16:31:42.085861 2014-08-13 17:38:02.702966 7fe9bae887a0 20 read_log 192888'447 (192888'432) delete 9f4e7cfe/benchmark_data_storage0111-ib.pocstor_35625_object2460/head//22 by client.940330.0:33768 2014-08-11 16:31:42.110540 2014-08-13 17:38:02.702979 7fe9bae887a0 20 read_log 192888'448 (192888'433) delete a463d4fe/benchmark_data_storage0111-ib.pocstor_35625_object12266/head//22 by client.940330.0:43574 2014-08-11 16:31:44.724608 2014-08-13 17:38:02.702991 7fe9bae887a0 20 read_log 192888'449 (192888'434) delete 856b94fe/benchmark_data_storage0111-ib.pocstor_35625_object13173/head//22 by client.940330.0:44481 2014-08-11 16:31:44.967098 2014-08-13 17:38:02.703003 7fe9bae887a0 20 read_log 192888'450 (192888'435) delete 639c1cfe/benchmark_data_storage0111-ib.pocstor_35625_object15473/head//22 by client.940330.0:46781 2014-08-11 16:31:45.584355 2014-08-13 17:38:02.703017 7fe9bae887a0 20 read_log 192888'451 (192888'436) delete ea2dccfe/benchmark_data_storage0111-ib.pocstor_35625_object15539/head//22 by client.940330.0:46847 2014-08-11 16:31:45.600863 2014-08-13 17:38:02.703030 7fe9bae887a0 20 read_log 192888'452 (192888'437) delete d3f8ccfe/benchmark_data_storage0111-ib.pocstor_35625_object18332/head//22 by client.940330.0:49640 2014-08-11 16:31:46.402516 2014-08-13 17:38:02.703044 7fe9bae887a0 20 read_log 192888'453 (192888'438) delete 7f25b4fe/benchmark_data_storage0111-ib.pocstor_35625_object18376/head//22 by client.940330.0:49684 2014-08-11 16:31:46.413892 2014-08-13 17:38:02.703064 7fe9bae887a0 20 read_log 192888'454 (192888'439) delete a96f0cfe/benchmark_data_storage0111-ib.pocstor_35625_object24128/head//22 by client.940330.0:55436 2014-08-11 16:31:48.080317 2014-08-13 17:38:02.703080 7fe9bae887a0 20 read_log 192888'455 (192888'440) delete 875b14fe/benchmark_data_storage0111-ib.pocstor_35625_object24796/head//22 by client.940330.0:56104 2014-08-11 16:31:48.273488 2014-08-13 17:38:02.703092 7fe9bae887a0 20 read_log 192888'456 (192888'441) delete 1c68c4fe/benchmark_data_storage0111-ib.pocstor_35625_object28381/head//22 by client.940330.0:59689 2014-08-11 16:31:49.305566 2014-08-13 17:38:02.703106 7fe9bae887a0 20 read_log 192888'457 (192888'442) delete fac364fe/benchmark_data_storage0111-ib.pocstor_35625_object28439/head//22 by client.940330.0:59747 2014-08-11 16:31:49.322330 2014-08-13 17:38:02.703118 7fe9bae887a0 20 read_log 192888'458 (192888'443) delete 74f5e4fe/benchmark_data_storage0111-ib.pocstor_35625_object29630/head//22 by client.940330.0:60938 2014-08-11 16:31:49.652241 2014-08-13 17:38:02.703132 7fe9bae887a0 20 read_log 192888'459 (192888'444) delete c1f7dcfe/benchmark_data_storage0111-ib.pocstor_35625_object30504/head//22 by client.940330.0:61812 2014-08-11 16:31:49.903415 2014-08-13 17:38:02.703145 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 41e73cfe/benchmark_data_storage0111-ib.pocstor_64498_object1675/head//22 by client.1049635.0:1676 2014-08-12 17:17:22.506625 2014-08-13 17:38:02.703180 7fe9bae887a0 20 read_log 194215'461 (0'0) modify d87c44fe/benchmark_data_storage0111-ib.pocstor_64498_object2418/head//22 by client.1049635.0:2419 2014-08-12 17:17:24.991169 2014-08-13 17:38:02.703197 7fe9bae887a0 20 read_log 194215'462 (0'0) modify f3738cfe/benchmark_data_storage0111-ib.pocstor_64498_object3262/head//22 by client.1049635.0:3263 2014-08-12 17:17:27.710487 2014-08-13 17:38:02.703209 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 87f1dcfe/benchmark_data_storage0111-ib.pocstor_64498_object4820/head//22 by client.1049635.0:4821 2014-08-12 17:17:32.378595 2014-08-13 17:38:02.703223 7fe9bae887a0 20 read_log 194215'464 (0'0) modify f77c3cfe/benchmark_data_storage0111-ib.pocstor_64498_object5164/head//22 by client.1049635.0:5165 2014-08-12 17:17:33.421968 2014-08-13 17:38:02.703235 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 20153cfe/benchmark_data_storage0111-ib.pocstor_64498_object6002/head//22 by client.1049635.0:6003 2014-08-12 17:17:36.083392 2014-08-13 17:38:02.703249 7fe9bae887a0 20 read_log 194215'466 (0'0) modify ec2644fe/benchmark_data_storage0111-ib.pocstor_64498_object11299/head//22 by client.1049635.0:11300 2014-08-12 17:17:52.111746 2014-08-13 17:38:02.703262 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 1acf5cfe/benchmark_data_storage0111-ib.pocstor_64498_object13628/head//22 by client.1049635.0:13629 2014-08-12 17:17:59.445232 2014-08-13 17:38:02.703276 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 903b14fe/benchmark_data_storage0111-ib.pocstor_64498_object15571/head//22 by client.1049635.0:15572 2014-08-12 17:18:05.996779 2014-08-13 17:38:02.703289 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 63ba7cfe/benchmark_data_storage0111-ib.pocstor_64498_object15746/head//22 by client.1049635.0:15747 2014-08-12 17:18:06.614892 2014-08-13 17:38:02.703301 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 5606ecfe/benchmark_data_storage0111-ib.pocstor_64498_object16345/head//22 by client.1049635.0:16346 2014-08-12 17:18:08.871425 2014-08-13 17:38:02.703313 7fe9bae887a0 20 read_log 194215'471 (0'0) modify dbfe6cfe/benchmark_data_storage0111-ib.pocstor_64498_object19601/head//22 by client.1049635.0:19602 2014-08-12 17:18:21.288159 2014-08-13 17:38:02.703327 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 7bdb1cfe/benchmark_data_storage0111-ib.pocstor_64498_object22759/head//22 by client.1049635.0:22760 2014-08-12 17:18:33.240760 2014-08-13 17:38:02.703347 7fe9bae887a0 20 read_log 194215'473 (0'0) modify d3aec4fe/benchmark_data_storage0111-ib.pocstor_64498_object25801/head//22 by client.1049635.0:25802 2014-08-12 17:18:44.682417 2014-08-13 17:38:02.703362 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 89bc84fe/benchmark_data_storage0111-ib.pocstor_64498_object26010/head//22 by client.1049635.0:26011 2014-08-12 17:18:45.462098 2014-08-13 17:38:02.703394 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 33f0ccfe/benchmark_data_storage0111-ib.pocstor_64498_object27370/head//22 by client.1049635.0:27371 2014-08-12 17:18:50.535886 2014-08-13 17:38:02.703410 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 8e498cfe/benchmark_data_storage0111-ib.pocstor_64498_object27382/head//22 by client.1049635.0:27383 2014-08-12 17:18:50.581043 2014-08-13 17:38:02.703422 7fe9bae887a0 20 read_log 194215'477 (0'0) modify d172ecfe/benchmark_data_storage0111-ib.pocstor_64498_object31545/head//22 by client.1049635.0:31546 2014-08-12 17:19:06.803234 2014-08-13 17:38:02.703434 7fe9bae887a0 20 read_log 194215'478 (194215'460) delete 41e73cfe/benchmark_data_storage0111-ib.pocstor_64498_object1675/head//22 by client.1049635.0:35878 2014-08-12 17:19:18.544663 2014-08-13 17:38:02.703447 7fe9bae887a0 20 read_log 194215'479 (194215'461) delete d87c44fe/benchmark_data_storage0111-ib.pocstor_64498_object2418/head//22 by client.1049635.0:36621 2014-08-12 17:19:18.773378 2014-08-13 17:38:02.703459 7fe9bae887a0 20 read_log 194215'480 (194215'462) delete f3738cfe/benchmark_data_storage0111-ib.pocstor_64498_object3262/head//22 by client.1049635.0:37465 2014-08-12 17:19:19.047973 2014-08-13 17:38:02.703471 7fe9bae887a0 20 read_log 194215'481 (194215'463) delete 87f1dcfe/benchmark_data_storage0111-ib.pocstor_64498_object4820/head//22 by client.1049635.0:39023 2014-08-12 17:19:19.542409 2014-08-13 17:38:02.703483 7fe9bae887a0 20 read_log 194215'482 (194215'464) delete f77c3cfe/benchmark_data_storage0111-ib.pocstor_64498_object5164/head//22 by client.1049635.0:39367 2014-08-12 17:19:19.660564 2014-08-13 17:38:02.703495 7fe9bae887a0 20 read_log 194215'483 (194215'465) delete 20153cfe/benchmark_data_storage0111-ib.pocstor_64498_object6002/head//22 by client.1049635.0:40205 2014-08-12 17:19:19.957174 2014-08-13 17:38:02.703508 7fe9bae887a0 20 read_log 194215'484 (194215'466) delete ec2644fe/benchmark_data_storage0111-ib.pocstor_64498_object11299/head//22 by client.1049635.0:45502 2014-08-12 17:19:21.672810 2014-08-13 17:38:02.703520 7fe9bae887a0 20 read_log 194215'485 (194215'467) delete 1acf5cfe/benchmark_data_storage0111-ib.pocstor_64498_object13628/head//22 by client.1049635.0:47831 2014-08-12 17:19:22.446745 2014-08-13 17:38:02.703532 7fe9bae887a0 20 read_log 194215'486 (194215'468) delete 903b14fe/benchmark_data_storage0111-ib.pocstor_64498_object15571/head//22 by client.1049635.0:49774 2014-08-12 17:19:23.055648 2014-08-13 17:38:02.703544 7fe9bae887a0 20 read_log 194215'487 (194215'469) delete 63ba7cfe/benchmark_data_storage0111-ib.pocstor_64498_object15746/head//22 by client.1049635.0:49949 2014-08-12 17:19:23.106847 2014-08-13 17:38:02.703556 7fe9bae887a0 20 read_log 194215'488 (194215'470) delete 5606ecfe/benchmark_data_storage0111-ib.pocstor_64498_object16345/head//22 by client.1049635.0:50548 2014-08-12 17:19:23.299188 2014-08-13 17:38:02.703570 7fe9bae887a0 20 read_log 194215'489 (194215'471) delete dbfe6cfe/benchmark_data_storage0111-ib.pocstor_64498_object19601/head//22 by client.1049635.0:53804 2014-08-12 17:19:24.386105 2014-08-13 17:38:02.703582 7fe9bae887a0 20 read_log 194215'490 (194215'472) delete 7bdb1cfe/benchmark_data_storage0111-ib.pocstor_64498_object22759/head//22 by client.1049635.0:56962 2014-08-12 17:19:25.523527 2014-08-13 17:38:02.703595 7fe9bae887a0 20 read_log 194215'491 (194215'473) delete d3aec4fe/benchmark_data_storage0111-ib.pocstor_64498_object25801/head//22 by client.1049635.0:60004 2014-08-12 17:19:26.591970 2014-08-13 17:38:02.703607 7fe9bae887a0 20 read_log 194215'492 (194215'474) delete 89bc84fe/benchmark_data_storage0111-ib.pocstor_64498_object26010/head//22 by client.1049635.0:60213 2014-08-12 17:19:26.658456 2014-08-13 17:38:02.703620 7fe9bae887a0 20 read_log 194215'493 (194215'475) delete 33f0ccfe/benchmark_data_storage0111-ib.pocstor_64498_object27370/head//22 by client.1049635.0:61573 2014-08-12 17:19:27.136365 2014-08-13 17:38:02.703651 7fe9bae887a0 20 read_log 194215'494 (194215'476) delete 8e498cfe/benchmark_data_storage0111-ib.pocstor_64498_object27382/head//22 by client.1049635.0:61585 2014-08-12 17:19:27.139273 2014-08-13 17:38:02.703664 7fe9bae887a0 20 read_log 194215'495 (194215'477) delete d172ecfe/benchmark_data_storage0111-ib.pocstor_64498_object31545/head//22 by client.1049635.0:65748 2014-08-12 17:19:28.573673 2014-08-13 17:38:02.703747 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.703764 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.703779 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001801 0 0.000000 2014-08-13 17:38:02.703791 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.703799 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.703808 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] log((0'0,194215'495], crt=192888'459) 2014-08-13 17:38:02.703817 7fe9bae887a0 10 osd.4 194478 pgid 22.537 coll 22.537_head 2014-08-13 17:38:02.703927 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.537 2014-08-13 17:38:02.703936 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.703976 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.537(unlocked)] enter Initial 2014-08-13 17:38:02.703987 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.537(unlocked)] enter NotTrimming 2014-08-13 17:38:02.704105 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.704227 7fe9bae887a0 20 read_log 191648'64 (0'0) modify 9f2d3537/benchmark_data_storage0111-ib.pocstor_40745_object4149/head//22 by client.273158.0:4150 2014-07-30 14:09:07.396439 2014-08-13 17:38:02.704248 7fe9bae887a0 20 read_log 191648'102 (0'0) modify e92e2537/benchmark_data_storage0111-ib.pocstor_40745_object6017/head//22 by client.273158.0:6018 2014-07-30 14:09:15.942553 2014-08-13 17:38:02.704261 7fe9bae887a0 20 read_log 191648'149 (0'0) modify 5cce3d37/benchmark_data_storage0111-ib.pocstor_40745_object9140/head//22 by client.273158.0:9141 2014-07-30 14:09:31.727335 2014-08-13 17:38:02.704273 7fe9bae887a0 20 read_log 191648'162 (0'0) modify 5abeb537/benchmark_data_storage0111-ib.pocstor_40745_object10175/head//22 by client.273158.0:10176 2014-07-30 14:09:36.713070 2014-08-13 17:38:02.704285 7fe9bae887a0 20 read_log 191648'217 (0'0) modify 163ced37/benchmark_data_storage0111-ib.pocstor_40745_object14668/head//22 by client.273158.0:14669 2014-07-30 14:10:00.860410 2014-08-13 17:38:02.704297 7fe9bae887a0 20 read_log 191648'268 (0'0) modify bcd06d37/benchmark_data_storage0111-ib.pocstor_40745_object17041/head//22 by client.273158.0:17042 2014-07-30 14:10:13.209255 2014-08-13 17:38:02.704333 7fe9bae887a0 20 read_log 191648'281 (0'0) modify e0988d37/benchmark_data_storage0111-ib.pocstor_40745_object17851/head//22 by client.273158.0:17852 2014-07-30 14:10:17.245423 2014-08-13 17:38:02.704349 7fe9bae887a0 20 read_log 191648'302 (0'0) modify a6dd7537/benchmark_data_storage0111-ib.pocstor_40745_object19182/head//22 by client.273158.0:19183 2014-07-30 14:10:24.319763 2014-08-13 17:38:02.704362 7fe9bae887a0 20 read_log 191648'329 (0'0) modify 321ecd37/benchmark_data_storage0111-ib.pocstor_40745_object21131/head//22 by client.273158.0:21132 2014-07-30 14:10:37.821865 2014-08-13 17:38:02.704374 7fe9bae887a0 20 read_log 191648'345 (0'0) modify 561e0d37/benchmark_data_storage0111-ib.pocstor_40745_object21999/head//22 by client.273158.0:22000 2014-07-30 14:10:44.192191 2014-08-13 17:38:02.704386 7fe9bae887a0 20 read_log 191648'353 (0'0) modify bbb3d37/benchmark_data_storage0111-ib.pocstor_40745_object22437/head//22 by client.273158.0:22438 2014-07-30 14:10:46.478246 2014-08-13 17:38:02.704398 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 9f800d37/benchmark_data_storage0111-ib.pocstor_51285_object2624/head//22 by client.275175.0:2625 2014-07-30 14:18:43.014894 2014-08-13 17:38:02.704410 7fe9bae887a0 20 read_log 191727'370 (0'0) modify e4a0c537/benchmark_data_storage0111-ib.pocstor_51285_object5657/head//22 by client.275175.0:5658 2014-07-30 14:18:52.265763 2014-08-13 17:38:02.704439 7fe9bae887a0 20 read_log 191727'371 (0'0) modify ab82537/benchmark_data_storage0111-ib.pocstor_51285_object6609/head//22 by client.275175.0:6610 2014-07-30 14:18:55.202703 2014-08-13 17:38:02.704454 7fe9bae887a0 20 read_log 191727'372 (0'0) modify 3c3d3537/benchmark_data_storage0111-ib.pocstor_51285_object6751/head//22 by client.275175.0:6752 2014-07-30 14:18:55.961904 2014-08-13 17:38:02.704466 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 533c7d37/benchmark_data_storage0111-ib.pocstor_51285_object7393/head//22 by client.275175.0:7394 2014-07-30 14:18:57.835060 2014-08-13 17:38:02.704478 7fe9bae887a0 20 read_log 191727'374 (0'0) modify 626e7d37/benchmark_data_storage0111-ib.pocstor_51285_object7462/head//22 by client.275175.0:7463 2014-07-30 14:18:58.026429 2014-08-13 17:38:02.704490 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 6d7d0537/benchmark_data_storage0111-ib.pocstor_51285_object7884/head//22 by client.275175.0:7885 2014-07-30 14:18:59.200890 2014-08-13 17:38:02.704502 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 8e980d37/benchmark_data_storage0111-ib.pocstor_51285_object11945/head//22 by client.275175.0:11946 2014-07-30 14:19:11.296820 2014-08-13 17:38:02.704514 7fe9bae887a0 20 read_log 191727'389 (0'0) modify b9cd1537/benchmark_data_storage0111-ib.pocstor_51285_object13273/head//22 by client.275175.0:13274 2014-07-30 14:19:15.465684 2014-08-13 17:38:02.704526 7fe9bae887a0 20 read_log 191727'393 (0'0) modify b73cc537/benchmark_data_storage0111-ib.pocstor_51285_object15116/head//22 by client.275175.0:15117 2014-07-30 14:19:21.100978 2014-08-13 17:38:02.704538 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 8b9bcd37/benchmark_data_storage0111-ib.pocstor_51285_object15958/head//22 by client.275175.0:15959 2014-07-30 14:19:23.703644 2014-08-13 17:38:02.704550 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 76fcfd37/benchmark_data_storage0111-ib.pocstor_51285_object16661/head//22 by client.275175.0:16662 2014-07-30 14:19:26.204367 2014-08-13 17:38:02.704562 7fe9bae887a0 20 read_log 191727'399 (0'0) modify bf420537/benchmark_data_storage0111-ib.pocstor_51285_object17034/head//22 by client.275175.0:17035 2014-07-30 14:19:27.255682 2014-08-13 17:38:02.704576 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 853f5d37/benchmark_data_storage0111-ib.pocstor_51285_object18814/head//22 by client.275175.0:18815 2014-07-30 14:19:32.605231 2014-08-13 17:38:02.704595 7fe9bae887a0 20 read_log 191727'414 (0'0) modify cc4fb537/benchmark_data_storage0111-ib.pocstor_51285_object28366/head//22 by client.275175.0:28367 2014-07-30 14:20:02.046246 2014-08-13 17:38:02.704608 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 2eae2537/benchmark_data_storage0111-ib.pocstor_51285_object35246/head//22 by client.275175.0:35247 2014-07-30 14:20:22.704942 2014-08-13 17:38:02.704620 7fe9bae887a0 20 read_log 191727'430 (0'0) modify e81b5537/benchmark_data_storage0111-ib.pocstor_51285_object36909/head//22 by client.275175.0:36910 2014-07-30 14:20:28.124031 2014-08-13 17:38:02.704632 7fe9bae887a0 20 read_log 192888'434 (0'0) modify fe549537/benchmark_data_storage0111-ib.pocstor_35625_object1111/head//22 by client.940330.0:1112 2014-08-11 16:29:45.205610 2014-08-13 17:38:02.704644 7fe9bae887a0 20 read_log 192888'435 (0'0) modify f0a82537/benchmark_data_storage0111-ib.pocstor_35625_object1868/head//22 by client.940330.0:1869 2014-08-11 16:29:48.057971 2014-08-13 17:38:02.704658 7fe9bae887a0 20 read_log 192888'436 (0'0) modify d6609d37/benchmark_data_storage0111-ib.pocstor_35625_object3626/head//22 by client.940330.0:3627 2014-08-11 16:29:54.913776 2014-08-13 17:38:02.704670 7fe9bae887a0 20 read_log 192888'437 (0'0) modify 7f693d37/benchmark_data_storage0111-ib.pocstor_35625_object4793/head//22 by client.940330.0:4794 2014-08-11 16:29:59.302241 2014-08-13 17:38:02.704682 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 48a16537/benchmark_data_storage0111-ib.pocstor_35625_object8829/head//22 by client.940330.0:8830 2014-08-11 16:30:13.735937 2014-08-13 17:38:02.704694 7fe9bae887a0 20 read_log 192888'439 (0'0) modify c38fc537/benchmark_data_storage0111-ib.pocstor_35625_object9379/head//22 by client.940330.0:9380 2014-08-11 16:30:15.657145 2014-08-13 17:38:02.704706 7fe9bae887a0 20 read_log 192888'440 (0'0) modify c01a3537/benchmark_data_storage0111-ib.pocstor_35625_object11996/head//22 by client.940330.0:11997 2014-08-11 16:30:25.927545 2014-08-13 17:38:02.704718 7fe9bae887a0 20 read_log 192888'441 (0'0) modify db259d37/benchmark_data_storage0111-ib.pocstor_35625_object14749/head//22 by client.940330.0:14750 2014-08-11 16:30:36.558485 2014-08-13 17:38:02.704730 7fe9bae887a0 20 read_log 192888'442 (0'0) modify d353b537/benchmark_data_storage0111-ib.pocstor_35625_object16821/head//22 by client.940330.0:16822 2014-08-11 16:30:44.298436 2014-08-13 17:38:02.704742 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 139a5537/benchmark_data_storage0111-ib.pocstor_35625_object20363/head//22 by client.940330.0:20364 2014-08-11 16:30:58.187839 2014-08-13 17:38:02.704756 7fe9bae887a0 20 read_log 192888'444 (0'0) modify defd8d37/benchmark_data_storage0111-ib.pocstor_35625_object24671/head//22 by client.940330.0:24672 2014-08-11 16:31:14.611331 2014-08-13 17:38:02.704768 7fe9bae887a0 20 read_log 192888'445 (0'0) modify e245d37/benchmark_data_storage0111-ib.pocstor_35625_object26935/head//22 by client.940330.0:26936 2014-08-11 16:31:23.553779 2014-08-13 17:38:02.704799 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 9b2c4537/benchmark_data_storage0111-ib.pocstor_35625_object29549/head//22 by client.940330.0:29550 2014-08-11 16:31:33.756040 2014-08-13 17:38:02.704812 7fe9bae887a0 20 read_log 192888'447 (0'0) modify b11edd37/benchmark_data_storage0111-ib.pocstor_35625_object30249/head//22 by client.940330.0:30250 2014-08-11 16:31:36.402100 2014-08-13 17:38:02.704825 7fe9bae887a0 20 read_log 192888'448 (192888'434) delete fe549537/benchmark_data_storage0111-ib.pocstor_35625_object1111/head//22 by client.940330.0:32419 2014-08-11 16:31:41.776212 2014-08-13 17:38:02.704838 7fe9bae887a0 20 read_log 192888'449 (192888'435) delete f0a82537/benchmark_data_storage0111-ib.pocstor_35625_object1868/head//22 by client.940330.0:33176 2014-08-11 16:31:41.957605 2014-08-13 17:38:02.704858 7fe9bae887a0 20 read_log 192888'450 (192888'436) delete d6609d37/benchmark_data_storage0111-ib.pocstor_35625_object3626/head//22 by client.940330.0:34934 2014-08-11 16:31:42.420971 2014-08-13 17:38:02.704873 7fe9bae887a0 20 read_log 192888'451 (192888'437) delete 7f693d37/benchmark_data_storage0111-ib.pocstor_35625_object4793/head//22 by client.940330.0:36101 2014-08-11 16:31:42.764209 2014-08-13 17:38:02.704885 7fe9bae887a0 20 read_log 192888'452 (192888'438) delete 48a16537/benchmark_data_storage0111-ib.pocstor_35625_object8829/head//22 by client.940330.0:40137 2014-08-11 16:31:43.799881 2014-08-13 17:38:02.704900 7fe9bae887a0 20 read_log 192888'453 (192888'439) delete c38fc537/benchmark_data_storage0111-ib.pocstor_35625_object9379/head//22 by client.940330.0:40687 2014-08-11 16:31:43.942556 2014-08-13 17:38:02.704915 7fe9bae887a0 20 read_log 192888'454 (192888'440) delete c01a3537/benchmark_data_storage0111-ib.pocstor_35625_object11996/head//22 by client.940330.0:43304 2014-08-11 16:31:44.656311 2014-08-13 17:38:02.704930 7fe9bae887a0 20 read_log 192888'455 (192888'441) delete db259d37/benchmark_data_storage0111-ib.pocstor_35625_object14749/head//22 by client.940330.0:46057 2014-08-11 16:31:45.386195 2014-08-13 17:38:02.704942 7fe9bae887a0 20 read_log 192888'456 (192888'442) delete d353b537/benchmark_data_storage0111-ib.pocstor_35625_object16821/head//22 by client.940330.0:48129 2014-08-11 16:31:45.939955 2014-08-13 17:38:02.704956 7fe9bae887a0 20 read_log 192888'457 (192888'443) delete 139a5537/benchmark_data_storage0111-ib.pocstor_35625_object20363/head//22 by client.940330.0:51671 2014-08-11 16:31:46.983849 2014-08-13 17:38:02.704969 7fe9bae887a0 20 read_log 192888'458 (192888'444) delete defd8d37/benchmark_data_storage0111-ib.pocstor_35625_object24671/head//22 by client.940330.0:55979 2014-08-11 16:31:48.235593 2014-08-13 17:38:02.704983 7fe9bae887a0 20 read_log 192888'459 (192888'445) delete e245d37/benchmark_data_storage0111-ib.pocstor_35625_object26935/head//22 by client.940330.0:58243 2014-08-11 16:31:48.911101 2014-08-13 17:38:02.704995 7fe9bae887a0 20 read_log 192888'460 (192888'446) delete 9b2c4537/benchmark_data_storage0111-ib.pocstor_35625_object29549/head//22 by client.940330.0:60857 2014-08-11 16:31:49.628931 2014-08-13 17:38:02.705009 7fe9bae887a0 20 read_log 192888'461 (192888'447) delete b11edd37/benchmark_data_storage0111-ib.pocstor_35625_object30249/head//22 by client.940330.0:61557 2014-08-11 16:31:49.828387 2014-08-13 17:38:02.705022 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 1caf6537/benchmark_data_storage0111-ib.pocstor_64498_object889/head//22 by client.1049635.0:890 2014-08-12 17:17:19.992609 2014-08-13 17:38:02.705036 7fe9bae887a0 20 read_log 194215'463 (0'0) modify ee9dbd37/benchmark_data_storage0111-ib.pocstor_64498_object1881/head//22 by client.1049635.0:1882 2014-08-12 17:17:23.176455 2014-08-13 17:38:02.705048 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 68637d37/benchmark_data_storage0111-ib.pocstor_64498_object4355/head//22 by client.1049635.0:4356 2014-08-12 17:17:31.099047 2014-08-13 17:38:02.705063 7fe9bae887a0 20 read_log 194215'465 (0'0) modify f4ec8537/benchmark_data_storage0111-ib.pocstor_64498_object9681/head//22 by client.1049635.0:9682 2014-08-12 17:17:47.016142 2014-08-13 17:38:02.705075 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 32adc537/benchmark_data_storage0111-ib.pocstor_64498_object10221/head//22 by client.1049635.0:10222 2014-08-12 17:17:48.555078 2014-08-13 17:38:02.705089 7fe9bae887a0 20 read_log 194215'467 (0'0) modify a86d5537/benchmark_data_storage0111-ib.pocstor_64498_object11839/head//22 by client.1049635.0:11840 2014-08-12 17:17:53.933524 2014-08-13 17:38:02.705102 7fe9bae887a0 20 read_log 194215'468 (0'0) modify b2a34537/benchmark_data_storage0111-ib.pocstor_64498_object14642/head//22 by client.1049635.0:14643 2014-08-12 17:18:02.766985 2014-08-13 17:38:02.705121 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 3afb8537/benchmark_data_storage0111-ib.pocstor_64498_object14690/head//22 by client.1049635.0:14691 2014-08-12 17:18:02.977905 2014-08-13 17:38:02.705135 7fe9bae887a0 20 read_log 194215'470 (0'0) modify f79d1d37/benchmark_data_storage0111-ib.pocstor_64498_object15153/head//22 by client.1049635.0:15154 2014-08-12 17:18:04.475618 2014-08-13 17:38:02.705148 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 4cb72537/benchmark_data_storage0111-ib.pocstor_64498_object15792/head//22 by client.1049635.0:15793 2014-08-12 17:18:06.783276 2014-08-13 17:38:02.705165 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 574a4537/benchmark_data_storage0111-ib.pocstor_64498_object18217/head//22 by client.1049635.0:18218 2014-08-12 17:18:15.981564 2014-08-13 17:38:02.705180 7fe9bae887a0 20 read_log 194215'473 (0'0) modify f84c6d37/benchmark_data_storage0111-ib.pocstor_64498_object18315/head//22 by client.1049635.0:18316 2014-08-12 17:18:16.355372 2014-08-13 17:38:02.705192 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 77140537/benchmark_data_storage0111-ib.pocstor_64498_object18943/head//22 by client.1049635.0:18944 2014-08-12 17:18:18.721127 2014-08-13 17:38:02.705204 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 30607d37/benchmark_data_storage0111-ib.pocstor_64498_object19997/head//22 by client.1049635.0:19998 2014-08-12 17:18:22.769519 2014-08-13 17:38:02.705218 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 43d19537/benchmark_data_storage0111-ib.pocstor_64498_object20224/head//22 by client.1049635.0:20225 2014-08-12 17:18:23.624964 2014-08-13 17:38:02.705230 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 941bd537/benchmark_data_storage0111-ib.pocstor_64498_object23926/head//22 by client.1049635.0:23927 2014-08-12 17:18:37.579130 2014-08-13 17:38:02.705243 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 74970537/benchmark_data_storage0111-ib.pocstor_64498_object24055/head//22 by client.1049635.0:24056 2014-08-12 17:18:38.057385 2014-08-13 17:38:02.705257 7fe9bae887a0 20 read_log 194215'479 (0'0) modify f848ad37/benchmark_data_storage0111-ib.pocstor_64498_object26220/head//22 by client.1049635.0:26221 2014-08-12 17:18:46.242193 2014-08-13 17:38:02.705269 7fe9bae887a0 20 read_log 194215'480 (0'0) modify ec0d3d37/benchmark_data_storage0111-ib.pocstor_64498_object26486/head//22 by client.1049635.0:26487 2014-08-12 17:18:47.238199 2014-08-13 17:38:02.705283 7fe9bae887a0 20 read_log 194215'481 (0'0) modify a97aad37/benchmark_data_storage0111-ib.pocstor_64498_object32816/head//22 by client.1049635.0:32817 2014-08-12 17:19:11.689309 2014-08-13 17:38:02.705295 7fe9bae887a0 20 read_log 194215'482 (194215'462) delete 1caf6537/benchmark_data_storage0111-ib.pocstor_64498_object889/head//22 by client.1049635.0:35092 2014-08-12 17:19:18.277797 2014-08-13 17:38:02.705310 7fe9bae887a0 20 read_log 194215'483 (194215'463) delete ee9dbd37/benchmark_data_storage0111-ib.pocstor_64498_object1881/head//22 by client.1049635.0:36084 2014-08-12 17:19:18.610953 2014-08-13 17:38:02.705322 7fe9bae887a0 20 read_log 194215'484 (194215'464) delete 68637d37/benchmark_data_storage0111-ib.pocstor_64498_object4355/head//22 by client.1049635.0:38558 2014-08-12 17:19:19.388681 2014-08-13 17:38:02.705336 7fe9bae887a0 20 read_log 194215'485 (194215'465) delete f4ec8537/benchmark_data_storage0111-ib.pocstor_64498_object9681/head//22 by client.1049635.0:43884 2014-08-12 17:19:21.148558 2014-08-13 17:38:02.705351 7fe9bae887a0 20 read_log 194215'486 (194215'466) delete 32adc537/benchmark_data_storage0111-ib.pocstor_64498_object10221/head//22 by client.1049635.0:44424 2014-08-12 17:19:21.335372 2014-08-13 17:38:02.705363 7fe9bae887a0 20 read_log 194215'487 (194215'467) delete a86d5537/benchmark_data_storage0111-ib.pocstor_64498_object11839/head//22 by client.1049635.0:46042 2014-08-12 17:19:21.850386 2014-08-13 17:38:02.705375 7fe9bae887a0 20 read_log 194215'488 (194215'468) delete b2a34537/benchmark_data_storage0111-ib.pocstor_64498_object14642/head//22 by client.1049635.0:48845 2014-08-12 17:19:22.767963 2014-08-13 17:38:02.705389 7fe9bae887a0 20 read_log 194215'489 (194215'469) delete 3afb8537/benchmark_data_storage0111-ib.pocstor_64498_object14690/head//22 by client.1049635.0:48893 2014-08-12 17:19:22.782057 2014-08-13 17:38:02.705401 7fe9bae887a0 20 read_log 194215'490 (194215'470) delete f79d1d37/benchmark_data_storage0111-ib.pocstor_64498_object15153/head//22 by client.1049635.0:49356 2014-08-12 17:19:22.921072 2014-08-13 17:38:02.705413 7fe9bae887a0 20 read_log 194215'491 (194215'471) delete 4cb72537/benchmark_data_storage0111-ib.pocstor_64498_object15792/head//22 by client.1049635.0:49995 2014-08-12 17:19:23.119120 2014-08-13 17:38:02.705425 7fe9bae887a0 20 read_log 194215'492 (194215'472) delete 574a4537/benchmark_data_storage0111-ib.pocstor_64498_object18217/head//22 by client.1049635.0:52420 2014-08-12 17:19:23.901882 2014-08-13 17:38:02.705437 7fe9bae887a0 20 read_log 194215'493 (194215'473) delete f84c6d37/benchmark_data_storage0111-ib.pocstor_64498_object18315/head//22 by client.1049635.0:52518 2014-08-12 17:19:23.936367 2014-08-13 17:38:02.705449 7fe9bae887a0 20 read_log 194215'494 (194215'474) delete 77140537/benchmark_data_storage0111-ib.pocstor_64498_object18943/head//22 by client.1049635.0:53146 2014-08-12 17:19:24.145069 2014-08-13 17:38:02.705461 7fe9bae887a0 20 read_log 194215'495 (194215'475) delete 30607d37/benchmark_data_storage0111-ib.pocstor_64498_object19997/head//22 by client.1049635.0:54200 2014-08-12 17:19:24.513925 2014-08-13 17:38:02.705473 7fe9bae887a0 20 read_log 194215'496 (194215'476) delete 43d19537/benchmark_data_storage0111-ib.pocstor_64498_object20224/head//22 by client.1049635.0:54427 2014-08-12 17:19:24.606307 2014-08-13 17:38:02.705485 7fe9bae887a0 20 read_log 194215'497 (194215'477) delete 941bd537/benchmark_data_storage0111-ib.pocstor_64498_object23926/head//22 by client.1049635.0:58129 2014-08-12 17:19:25.937475 2014-08-13 17:38:02.705497 7fe9bae887a0 20 read_log 194215'498 (194215'478) delete 74970537/benchmark_data_storage0111-ib.pocstor_64498_object24055/head//22 by client.1049635.0:58258 2014-08-12 17:19:25.980347 2014-08-13 17:38:02.705509 7fe9bae887a0 20 read_log 194215'499 (194215'479) delete f848ad37/benchmark_data_storage0111-ib.pocstor_64498_object26220/head//22 by client.1049635.0:60423 2014-08-12 17:19:26.732956 2014-08-13 17:38:02.705521 7fe9bae887a0 20 read_log 194215'500 (194215'480) delete ec0d3d37/benchmark_data_storage0111-ib.pocstor_64498_object26486/head//22 by client.1049635.0:60689 2014-08-12 17:19:26.817198 2014-08-13 17:38:02.705534 7fe9bae887a0 20 read_log 194215'501 (194215'481) delete a97aad37/benchmark_data_storage0111-ib.pocstor_64498_object32816/head//22 by client.1049635.0:67019 2014-08-12 17:19:29.045914 2014-08-13 17:38:02.705613 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.705631 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=0 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.705646 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=0 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001671 0 0.000000 2014-08-13 17:38:02.705658 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=0 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.705666 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=0 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.705674 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] log((0'0,194215'501], crt=192888'461) 2014-08-13 17:38:02.705684 7fe9bae887a0 10 osd.4 194478 pgid 22.581 coll 22.581_head 2014-08-13 17:38:02.705803 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.581 2014-08-13 17:38:02.705812 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.705839 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.581(unlocked)] enter Initial 2014-08-13 17:38:02.705850 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.581(unlocked)] enter NotTrimming 2014-08-13 17:38:02.705973 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.706096 7fe9bae887a0 20 read_log 191648'1 (0'0) modify d146b581/benchmark_data_storage0111-ib.pocstor_40745_object11/head//22 by client.273158.0:12 2014-07-30 14:08:48.538992 2014-08-13 17:38:02.706116 7fe9bae887a0 20 read_log 191648'16 (0'0) modify 272b0d81/benchmark_data_storage0111-ib.pocstor_40745_object1340/head//22 by client.273158.0:1341 2014-07-30 14:08:53.717594 2014-08-13 17:38:02.706129 7fe9bae887a0 20 read_log 191648'24 (0'0) modify 2c065581/benchmark_data_storage0111-ib.pocstor_40745_object1791/head//22 by client.273158.0:1792 2014-07-30 14:08:55.358268 2014-08-13 17:38:02.706141 7fe9bae887a0 20 read_log 191648'59 (0'0) modify 5068dd81/benchmark_data_storage0111-ib.pocstor_40745_object3427/head//22 by client.273158.0:3428 2014-07-30 14:09:03.353286 2014-08-13 17:38:02.706162 7fe9bae887a0 20 read_log 191648'65 (0'0) modify f5682581/benchmark_data_storage0111-ib.pocstor_40745_object3784/head//22 by client.273158.0:3785 2014-07-30 14:09:05.182665 2014-08-13 17:38:02.706178 7fe9bae887a0 20 read_log 191648'78 (0'0) modify 397aad81/benchmark_data_storage0111-ib.pocstor_40745_object4453/head//22 by client.273158.0:4454 2014-07-30 14:09:08.548982 2014-08-13 17:38:02.706191 7fe9bae887a0 20 read_log 191648'98 (0'0) modify e2aa9d81/benchmark_data_storage0111-ib.pocstor_40745_object5708/head//22 by client.273158.0:5709 2014-07-30 14:09:14.276817 2014-08-13 17:38:02.706225 7fe9bae887a0 20 read_log 191648'105 (0'0) modify 14eae581/benchmark_data_storage0111-ib.pocstor_40745_object6217/head//22 by client.273158.0:6218 2014-07-30 14:09:17.111124 2014-08-13 17:38:02.706241 7fe9bae887a0 20 read_log 191648'113 (0'0) modify 11464d81/benchmark_data_storage0111-ib.pocstor_40745_object6648/head//22 by client.273158.0:6649 2014-07-30 14:09:19.549437 2014-08-13 17:38:02.706253 7fe9bae887a0 20 read_log 191648'138 (0'0) modify 63fb9d81/benchmark_data_storage0111-ib.pocstor_40745_object8043/head//22 by client.273158.0:8044 2014-07-30 14:09:26.300856 2014-08-13 17:38:02.706265 7fe9bae887a0 20 read_log 191648'140 (0'0) modify 41112d81/benchmark_data_storage0111-ib.pocstor_40745_object8385/head//22 by client.273158.0:8386 2014-07-30 14:09:28.163359 2014-08-13 17:38:02.706277 7fe9bae887a0 20 read_log 191648'177 (0'0) modify 6741bd81/benchmark_data_storage0111-ib.pocstor_40745_object10398/head//22 by client.273158.0:10399 2014-07-30 14:09:37.827925 2014-08-13 17:38:02.706289 7fe9bae887a0 20 read_log 191648'191 (0'0) modify 9fe2581/benchmark_data_storage0111-ib.pocstor_40745_object11558/head//22 by client.273158.0:11559 2014-07-30 14:09:45.058157 2014-08-13 17:38:02.706302 7fe9bae887a0 20 read_log 191648'238 (0'0) modify f8824d81/benchmark_data_storage0111-ib.pocstor_40745_object14131/head//22 by client.273158.0:14132 2014-07-30 14:09:57.630527 2014-08-13 17:38:02.706314 7fe9bae887a0 20 read_log 191648'255 (0'0) modify a52c1d81/benchmark_data_storage0111-ib.pocstor_40745_object14762/head//22 by client.273158.0:14763 2014-07-30 14:10:01.462413 2014-08-13 17:38:02.706326 7fe9bae887a0 20 read_log 191648'272 (0'0) modify dfe2581/benchmark_data_storage0111-ib.pocstor_40745_object15659/head//22 by client.273158.0:15660 2014-07-30 14:10:05.527327 2014-08-13 17:38:02.706348 7fe9bae887a0 20 read_log 191648'360 (0'0) modify fe3c5d81/benchmark_data_storage0111-ib.pocstor_40745_object21320/head//22 by client.273158.0:21321 2014-07-30 14:10:38.582548 2014-08-13 17:38:02.706361 7fe9bae887a0 20 read_log 191648'377 (0'0) modify 9a720d81/benchmark_data_storage0111-ib.pocstor_40745_object22468/head//22 by client.273158.0:22469 2014-07-30 14:10:46.949357 2014-08-13 17:38:02.706374 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 5d0f2581/benchmark_data_storage0111-ib.pocstor_51285_object1297/head//22 by client.275175.0:1298 2014-07-30 14:18:38.510951 2014-08-13 17:38:02.706386 7fe9bae887a0 20 read_log 191727'390 (0'0) modify ed21ed81/benchmark_data_storage0111-ib.pocstor_51285_object1342/head//22 by client.275175.0:1343 2014-07-30 14:18:38.669717 2014-08-13 17:38:02.706398 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 1759ad81/benchmark_data_storage0111-ib.pocstor_51285_object2146/head//22 by client.275175.0:2147 2014-07-30 14:18:41.373638 2014-08-13 17:38:02.706410 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 75719d81/benchmark_data_storage0111-ib.pocstor_51285_object3172/head//22 by client.275175.0:3173 2014-07-30 14:18:44.769729 2014-08-13 17:38:02.706422 7fe9bae887a0 20 read_log 191727'398 (0'0) modify 5f4a7d81/benchmark_data_storage0111-ib.pocstor_51285_object5271/head//22 by client.275175.0:5272 2014-07-30 14:18:51.151681 2014-08-13 17:38:02.706433 7fe9bae887a0 20 read_log 191727'406 (0'0) modify 60441581/benchmark_data_storage0111-ib.pocstor_51285_object10141/head//22 by client.275175.0:10142 2014-07-30 14:19:05.575559 2014-08-13 17:38:02.706445 7fe9bae887a0 20 read_log 191727'411 (0'0) modify cbbdbd81/benchmark_data_storage0111-ib.pocstor_51285_object10895/head//22 by client.275175.0:10896 2014-07-30 14:19:07.750675 2014-08-13 17:38:02.706457 7fe9bae887a0 20 read_log 191727'413 (0'0) modify 30080d81/benchmark_data_storage0111-ib.pocstor_51285_object12202/head//22 by client.275175.0:12203 2014-07-30 14:19:12.207065 2014-08-13 17:38:02.706469 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 36ebcd81/benchmark_data_storage0111-ib.pocstor_51285_object13646/head//22 by client.275175.0:13647 2014-07-30 14:19:16.657749 2014-08-13 17:38:02.706481 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 55007581/benchmark_data_storage0111-ib.pocstor_51285_object19691/head//22 by client.275175.0:19692 2014-07-30 14:19:35.305345 2014-08-13 17:38:02.706493 7fe9bae887a0 20 read_log 191727'429 (0'0) modify a9112d81/benchmark_data_storage0111-ib.pocstor_51285_object20829/head//22 by client.275175.0:20830 2014-07-30 14:19:38.645198 2014-08-13 17:38:02.706505 7fe9bae887a0 20 read_log 191727'430 (0'0) modify dd9a2d81/benchmark_data_storage0111-ib.pocstor_51285_object21118/head//22 by client.275175.0:21119 2014-07-30 14:19:39.465456 2014-08-13 17:38:02.706517 7fe9bae887a0 20 read_log 191727'434 (0'0) modify beb91581/benchmark_data_storage0111-ib.pocstor_51285_object21905/head//22 by client.275175.0:21906 2014-07-30 14:19:42.155331 2014-08-13 17:38:02.706529 7fe9bae887a0 20 read_log 191727'445 (0'0) modify 6f609d81/benchmark_data_storage0111-ib.pocstor_51285_object25276/head//22 by client.275175.0:25277 2014-07-30 14:19:52.636574 2014-08-13 17:38:02.706541 7fe9bae887a0 20 read_log 191727'450 (0'0) modify d77ce581/benchmark_data_storage0111-ib.pocstor_51285_object26437/head//22 by client.275175.0:26438 2014-07-30 14:19:56.136719 2014-08-13 17:38:02.706552 7fe9bae887a0 20 read_log 191727'454 (0'0) modify b3379d81/benchmark_data_storage0111-ib.pocstor_51285_object28839/head//22 by client.275175.0:28840 2014-07-30 14:20:03.433788 2014-08-13 17:38:02.706564 7fe9bae887a0 20 read_log 191727'456 (0'0) modify 1c28c581/benchmark_data_storage0111-ib.pocstor_51285_object29540/head//22 by client.275175.0:29541 2014-07-30 14:20:05.554898 2014-08-13 17:38:02.706583 7fe9bae887a0 20 read_log 191727'458 (0'0) modify a129581/benchmark_data_storage0111-ib.pocstor_51285_object30379/head//22 by client.275175.0:30380 2014-07-30 14:20:08.240810 2014-08-13 17:38:02.706595 7fe9bae887a0 20 read_log 191727'460 (0'0) modify c552fd81/benchmark_data_storage0111-ib.pocstor_51285_object31719/head//22 by client.275175.0:31720 2014-07-30 14:20:12.062727 2014-08-13 17:38:02.706607 7fe9bae887a0 20 read_log 191727'461 (0'0) modify 6a495d81/benchmark_data_storage0111-ib.pocstor_51285_object32013/head//22 by client.275175.0:32014 2014-07-30 14:20:12.887831 2014-08-13 17:38:02.706619 7fe9bae887a0 20 read_log 191727'465 (0'0) modify 2d9d5581/benchmark_data_storage0111-ib.pocstor_51285_object34460/head//22 by client.275175.0:34461 2014-07-30 14:20:20.435826 2014-08-13 17:38:02.706631 7fe9bae887a0 20 read_log 191727'475 (0'0) modify 70e8f581/benchmark_data_storage0111-ib.pocstor_51285_object38678/head//22 by client.275175.0:38679 2014-07-30 14:20:33.179159 2014-08-13 17:38:02.706643 7fe9bae887a0 20 read_log 192888'476 (0'0) modify 28013d81/benchmark_data_storage0111-ib.pocstor_35625_object1558/head//22 by client.940330.0:1559 2014-08-11 16:29:46.921107 2014-08-13 17:38:02.706655 7fe9bae887a0 20 read_log 192888'477 (0'0) modify 8576c581/benchmark_data_storage0111-ib.pocstor_35625_object4090/head//22 by client.940330.0:4091 2014-08-11 16:29:56.660294 2014-08-13 17:38:02.706667 7fe9bae887a0 20 read_log 192888'478 (0'0) modify 821fed81/benchmark_data_storage0111-ib.pocstor_35625_object8710/head//22 by client.940330.0:8711 2014-08-11 16:30:13.321367 2014-08-13 17:38:02.706679 7fe9bae887a0 20 read_log 192888'479 (0'0) modify 458f1d81/benchmark_data_storage0111-ib.pocstor_35625_object12452/head//22 by client.940330.0:12453 2014-08-11 16:30:27.715445 2014-08-13 17:38:02.706690 7fe9bae887a0 20 read_log 192888'480 (0'0) modify bd1a0581/benchmark_data_storage0111-ib.pocstor_35625_object12722/head//22 by client.940330.0:12723 2014-08-11 16:30:28.720865 2014-08-13 17:38:02.706702 7fe9bae887a0 20 read_log 192888'481 (0'0) modify cfa2bd81/benchmark_data_storage0111-ib.pocstor_35625_object15617/head//22 by client.940330.0:15618 2014-08-11 16:30:39.789438 2014-08-13 17:38:02.706714 7fe9bae887a0 20 read_log 192888'482 (0'0) modify 781e1d81/benchmark_data_storage0111-ib.pocstor_35625_object19145/head//22 by client.940330.0:19146 2014-08-11 16:30:53.201350 2014-08-13 17:38:02.706726 7fe9bae887a0 20 read_log 192888'483 (0'0) modify 7f69cd81/benchmark_data_storage0111-ib.pocstor_35625_object25229/head//22 by client.940330.0:25230 2014-08-11 16:31:16.787923 2014-08-13 17:38:02.706738 7fe9bae887a0 20 read_log 192888'484 (0'0) modify 5dd64581/benchmark_data_storage0111-ib.pocstor_35625_object25720/head//22 by client.940330.0:25721 2014-08-11 16:31:18.691446 2014-08-13 17:38:02.706749 7fe9bae887a0 20 read_log 192888'485 (0'0) modify 75688d81/benchmark_data_storage0111-ib.pocstor_35625_object26050/head//22 by client.940330.0:26051 2014-08-11 16:31:19.994972 2014-08-13 17:38:02.706761 7fe9bae887a0 20 read_log 192888'486 (0'0) modify 4c108d81/benchmark_data_storage0111-ib.pocstor_35625_object26980/head//22 by client.940330.0:26981 2014-08-11 16:31:23.724757 2014-08-13 17:38:02.706773 7fe9bae887a0 20 read_log 192888'487 (192888'476) delete 28013d81/benchmark_data_storage0111-ib.pocstor_35625_object1558/head//22 by client.940330.0:32866 2014-08-11 16:31:41.881756 2014-08-13 17:38:02.706785 7fe9bae887a0 20 read_log 192888'488 (192888'477) delete 8576c581/benchmark_data_storage0111-ib.pocstor_35625_object4090/head//22 by client.940330.0:35398 2014-08-11 16:31:42.561525 2014-08-13 17:38:02.706798 7fe9bae887a0 20 read_log 192888'489 (192888'478) delete 821fed81/benchmark_data_storage0111-ib.pocstor_35625_object8710/head//22 by client.940330.0:40018 2014-08-11 16:31:43.766973 2014-08-13 17:38:02.706832 7fe9bae887a0 20 read_log 192888'490 (192888'479) delete 458f1d81/benchmark_data_storage0111-ib.pocstor_35625_object12452/head//22 by client.940330.0:43760 2014-08-11 16:31:44.770827 2014-08-13 17:38:02.706849 7fe9bae887a0 20 read_log 192888'491 (192888'480) delete bd1a0581/benchmark_data_storage0111-ib.pocstor_35625_object12722/head//22 by client.940330.0:44030 2014-08-11 16:31:44.841348 2014-08-13 17:38:02.706861 7fe9bae887a0 20 read_log 192888'492 (192888'481) delete cfa2bd81/benchmark_data_storage0111-ib.pocstor_35625_object15617/head//22 by client.940330.0:46925 2014-08-11 16:31:45.622949 2014-08-13 17:38:02.706873 7fe9bae887a0 20 read_log 192888'493 (192888'482) delete 781e1d81/benchmark_data_storage0111-ib.pocstor_35625_object19145/head//22 by client.940330.0:50453 2014-08-11 16:31:46.620029 2014-08-13 17:38:02.706885 7fe9bae887a0 20 read_log 192888'494 (192888'483) delete 7f69cd81/benchmark_data_storage0111-ib.pocstor_35625_object25229/head//22 by client.940330.0:56537 2014-08-11 16:31:48.414216 2014-08-13 17:38:02.706897 7fe9bae887a0 20 read_log 192888'495 (192888'484) delete 5dd64581/benchmark_data_storage0111-ib.pocstor_35625_object25720/head//22 by client.940330.0:57028 2014-08-11 16:31:48.565309 2014-08-13 17:38:02.706909 7fe9bae887a0 20 read_log 192888'496 (192888'485) delete 75688d81/benchmark_data_storage0111-ib.pocstor_35625_object26050/head//22 by client.940330.0:57358 2014-08-11 16:31:48.662122 2014-08-13 17:38:02.706922 7fe9bae887a0 20 read_log 192888'497 (192888'486) delete 4c108d81/benchmark_data_storage0111-ib.pocstor_35625_object26980/head//22 by client.940330.0:58288 2014-08-11 16:31:48.921349 2014-08-13 17:38:02.706934 7fe9bae887a0 20 read_log 194215'498 (0'0) modify 9d807d81/benchmark_data_storage0111-ib.pocstor_64498_object5781/head//22 by client.1049635.0:5782 2014-08-12 17:17:35.440085 2014-08-13 17:38:02.706947 7fe9bae887a0 20 read_log 194215'499 (0'0) modify c3b61581/benchmark_data_storage0111-ib.pocstor_64498_object7847/head//22 by client.1049635.0:7848 2014-08-12 17:17:41.678216 2014-08-13 17:38:02.706960 7fe9bae887a0 20 read_log 194215'500 (0'0) modify ad4a0581/benchmark_data_storage0111-ib.pocstor_64498_object7922/head//22 by client.1049635.0:7923 2014-08-12 17:17:41.898104 2014-08-13 17:38:02.706972 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 4074d81/benchmark_data_storage0111-ib.pocstor_64498_object8983/head//22 by client.1049635.0:8984 2014-08-12 17:17:44.974246 2014-08-13 17:38:02.706984 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 74106d81/benchmark_data_storage0111-ib.pocstor_64498_object11247/head//22 by client.1049635.0:11248 2014-08-12 17:17:51.962968 2014-08-13 17:38:02.706996 7fe9bae887a0 20 read_log 194215'503 (0'0) modify f725c581/benchmark_data_storage0111-ib.pocstor_64498_object12443/head//22 by client.1049635.0:12444 2014-08-12 17:17:55.739942 2014-08-13 17:38:02.707008 7fe9bae887a0 20 read_log 194215'504 (0'0) modify 2b3e2d81/benchmark_data_storage0111-ib.pocstor_64498_object14452/head//22 by client.1049635.0:14453 2014-08-12 17:18:02.095885 2014-08-13 17:38:02.707020 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 6396d581/benchmark_data_storage0111-ib.pocstor_64498_object16192/head//22 by client.1049635.0:16193 2014-08-12 17:18:08.282803 2014-08-13 17:38:02.707032 7fe9bae887a0 20 read_log 194215'506 (0'0) modify a2166d81/benchmark_data_storage0111-ib.pocstor_64498_object20000/head//22 by client.1049635.0:20001 2014-08-12 17:18:22.780614 2014-08-13 17:38:02.707044 7fe9bae887a0 20 read_log 194215'507 (0'0) modify f73f2d81/benchmark_data_storage0111-ib.pocstor_64498_object21974/head//22 by client.1049635.0:21975 2014-08-12 17:18:30.233423 2014-08-13 17:38:02.707056 7fe9bae887a0 20 read_log 194215'508 (0'0) modify 3ae62d81/benchmark_data_storage0111-ib.pocstor_64498_object23221/head//22 by client.1049635.0:23222 2014-08-12 17:18:34.953135 2014-08-13 17:38:02.707076 7fe9bae887a0 20 read_log 194215'509 (0'0) modify bb607d81/benchmark_data_storage0111-ib.pocstor_64498_object24641/head//22 by client.1049635.0:24642 2014-08-12 17:18:40.242869 2014-08-13 17:38:02.707088 7fe9bae887a0 20 read_log 194215'510 (0'0) modify c4f65d81/benchmark_data_storage0111-ib.pocstor_64498_object27772/head//22 by client.1049635.0:27773 2014-08-12 17:18:52.030026 2014-08-13 17:38:02.707100 7fe9bae887a0 20 read_log 194215'511 (0'0) modify 94586581/benchmark_data_storage0111-ib.pocstor_64498_object28398/head//22 by client.1049635.0:28399 2014-08-12 17:18:54.364140 2014-08-13 17:38:02.707112 7fe9bae887a0 20 read_log 194215'512 (0'0) modify c9722581/benchmark_data_storage0111-ib.pocstor_64498_object32664/head//22 by client.1049635.0:32665 2014-08-12 17:19:11.116031 2014-08-13 17:38:02.707124 7fe9bae887a0 20 read_log 194215'513 (194215'498) delete 9d807d81/benchmark_data_storage0111-ib.pocstor_64498_object5781/head//22 by client.1049635.0:39984 2014-08-12 17:19:19.881311 2014-08-13 17:38:02.707136 7fe9bae887a0 20 read_log 194215'514 (194215'499) delete c3b61581/benchmark_data_storage0111-ib.pocstor_64498_object7847/head//22 by client.1049635.0:42050 2014-08-12 17:19:20.557152 2014-08-13 17:38:02.707149 7fe9bae887a0 20 read_log 194215'515 (194215'500) delete ad4a0581/benchmark_data_storage0111-ib.pocstor_64498_object7922/head//22 by client.1049635.0:42125 2014-08-12 17:19:20.579298 2014-08-13 17:38:02.707166 7fe9bae887a0 20 read_log 194215'516 (194215'501) delete 4074d81/benchmark_data_storage0111-ib.pocstor_64498_object8983/head//22 by client.1049635.0:43186 2014-08-12 17:19:20.918859 2014-08-13 17:38:02.707179 7fe9bae887a0 20 read_log 194215'517 (194215'502) delete 74106d81/benchmark_data_storage0111-ib.pocstor_64498_object11247/head//22 by client.1049635.0:45450 2014-08-12 17:19:21.656410 2014-08-13 17:38:02.707191 7fe9bae887a0 20 read_log 194215'518 (194215'503) delete f725c581/benchmark_data_storage0111-ib.pocstor_64498_object12443/head//22 by client.1049635.0:46646 2014-08-12 17:19:22.035881 2014-08-13 17:38:02.707203 7fe9bae887a0 20 read_log 194215'519 (194215'504) delete 2b3e2d81/benchmark_data_storage0111-ib.pocstor_64498_object14452/head//22 by client.1049635.0:48655 2014-08-12 17:19:22.705833 2014-08-13 17:38:02.707215 7fe9bae887a0 20 read_log 194215'520 (194215'505) delete 6396d581/benchmark_data_storage0111-ib.pocstor_64498_object16192/head//22 by client.1049635.0:50395 2014-08-12 17:19:23.248446 2014-08-13 17:38:02.707230 7fe9bae887a0 20 read_log 194215'521 (194215'506) delete a2166d81/benchmark_data_storage0111-ib.pocstor_64498_object20000/head//22 by client.1049635.0:54203 2014-08-12 17:19:24.514357 2014-08-13 17:38:02.707242 7fe9bae887a0 20 read_log 194215'522 (194215'507) delete f73f2d81/benchmark_data_storage0111-ib.pocstor_64498_object21974/head//22 by client.1049635.0:56177 2014-08-12 17:19:25.223531 2014-08-13 17:38:02.707254 7fe9bae887a0 20 read_log 194215'523 (194215'508) delete 3ae62d81/benchmark_data_storage0111-ib.pocstor_64498_object23221/head//22 by client.1049635.0:57424 2014-08-12 17:19:25.698341 2014-08-13 17:38:02.707266 7fe9bae887a0 20 read_log 194215'524 (194215'509) delete bb607d81/benchmark_data_storage0111-ib.pocstor_64498_object24641/head//22 by client.1049635.0:58844 2014-08-12 17:19:26.189294 2014-08-13 17:38:02.707300 7fe9bae887a0 20 read_log 194215'525 (194215'510) delete c4f65d81/benchmark_data_storage0111-ib.pocstor_64498_object27772/head//22 by client.1049635.0:61975 2014-08-12 17:19:27.270115 2014-08-13 17:38:02.707313 7fe9bae887a0 20 read_log 194215'526 (194215'511) delete 94586581/benchmark_data_storage0111-ib.pocstor_64498_object28398/head//22 by client.1049635.0:62601 2014-08-12 17:19:27.489178 2014-08-13 17:38:02.707325 7fe9bae887a0 20 read_log 194215'527 (194215'512) delete c9722581/benchmark_data_storage0111-ib.pocstor_64498_object32664/head//22 by client.1049635.0:66867 2014-08-12 17:19:28.987628 2014-08-13 17:38:02.707403 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.707421 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=0 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.707436 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=0 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Initial 0.001598 0 0.000000 2014-08-13 17:38:02.707448 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=0 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.707456 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=0 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194478 2014-08-13 17:38:02.707464 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] log((0'0,194215'527], crt=192888'497) 2014-08-13 17:38:02.707474 7fe9bae887a0 10 osd.4 194478 pgid 22.619 coll 22.619_head 2014-08-13 17:38:02.707581 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.619 2014-08-13 17:38:02.707590 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.707616 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.619(unlocked)] enter Initial 2014-08-13 17:38:02.707627 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.619(unlocked)] enter NotTrimming 2014-08-13 17:38:02.707746 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.707857 7fe9bae887a0 20 read_log 191648'29 (0'0) modify b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 by client.273158.0:1797 2014-07-30 14:08:55.376645 2014-08-13 17:38:02.707877 7fe9bae887a0 20 read_log 191648'35 (0'0) modify 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 by client.273158.0:2102 2014-07-30 14:08:56.673851 2014-08-13 17:38:02.707890 7fe9bae887a0 20 read_log 191648'155 (0'0) modify e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 by client.273158.0:9105 2014-07-30 14:09:31.580296 2014-08-13 17:38:02.707902 7fe9bae887a0 20 read_log 191648'157 (0'0) modify e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 by client.273158.0:9252 2014-07-30 14:09:32.163490 2014-08-13 17:38:02.707914 7fe9bae887a0 20 read_log 191648'200 (0'0) modify ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 by client.273158.0:11709 2014-07-30 14:09:45.618534 2014-08-13 17:38:02.707955 7fe9bae887a0 20 read_log 191648'207 (0'0) modify 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 by client.273158.0:12031 2014-07-30 14:09:47.326717 2014-08-13 17:38:02.707971 7fe9bae887a0 20 read_log 191648'215 (0'0) modify 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 by client.273158.0:12401 2014-07-30 14:09:49.584897 2014-08-13 17:38:02.707983 7fe9bae887a0 20 read_log 191648'216 (0'0) modify f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 by client.273158.0:12403 2014-07-30 14:09:49.597034 2014-08-13 17:38:02.707995 7fe9bae887a0 20 read_log 191648'222 (0'0) modify e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 by client.273158.0:12736 2014-07-30 14:09:51.156005 2014-08-13 17:38:02.708007 7fe9bae887a0 20 read_log 191648'295 (0'0) modify d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 by client.273158.0:18288 2014-07-30 14:10:19.550998 2014-08-13 17:38:02.708019 7fe9bae887a0 20 read_log 191648'298 (0'0) modify a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 by client.273158.0:18354 2014-07-30 14:10:19.908154 2014-08-13 17:38:02.708031 7fe9bae887a0 20 read_log 191648'361 (0'0) modify da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 by client.273158.0:22717 2014-07-30 14:10:47.933850 2014-08-13 17:38:02.708043 7fe9bae887a0 20 read_log 191727'366 (0'0) modify 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 by client.275175.0:618 2014-07-30 14:18:35.877657 2014-08-13 17:38:02.708055 7fe9bae887a0 20 read_log 191727'369 (0'0) modify 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 by client.275175.0:2413 2014-07-30 14:18:42.305242 2014-08-13 17:38:02.708067 7fe9bae887a0 20 read_log 191727'371 (0'0) modify fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 by client.275175.0:4043 2014-07-30 14:18:47.486778 2014-08-13 17:38:02.708079 7fe9bae887a0 20 read_log 191727'372 (0'0) modify 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 by client.275175.0:4443 2014-07-30 14:18:48.684121 2014-08-13 17:38:02.708090 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 by client.275175.0:5945 2014-07-30 14:18:53.087326 2014-08-13 17:38:02.708102 7fe9bae887a0 20 read_log 191727'374 (0'0) modify 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 by client.275175.0:6169 2014-07-30 14:18:53.780895 2014-08-13 17:38:02.708114 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 by client.275175.0:6426 2014-07-30 14:18:54.515886 2014-08-13 17:38:02.708125 7fe9bae887a0 20 read_log 191727'377 (0'0) modify 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 by client.275175.0:7435 2014-07-30 14:18:57.947408 2014-08-13 17:38:02.708137 7fe9bae887a0 20 read_log 191727'397 (0'0) modify 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 by client.275175.0:17498 2014-07-30 14:19:28.540203 2014-08-13 17:38:02.708149 7fe9bae887a0 20 read_log 191727'408 (0'0) modify 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 by client.275175.0:21276 2014-07-30 14:19:39.945185 2014-08-13 17:38:02.708167 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 by client.275175.0:26508 2014-07-30 14:19:56.383979 2014-08-13 17:38:02.708179 7fe9bae887a0 20 read_log 191727'420 (0'0) modify 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 by client.275175.0:28600 2014-07-30 14:20:02.711208 2014-08-13 17:38:02.708199 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 by client.275175.0:33054 2014-07-30 14:20:15.946725 2014-08-13 17:38:02.708211 7fe9bae887a0 20 read_log 191727'429 (0'0) modify dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 by client.275175.0:33392 2014-07-30 14:20:17.004800 2014-08-13 17:38:02.708223 7fe9bae887a0 20 read_log 191727'435 (0'0) modify 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 by client.275175.0:36750 2014-07-30 14:20:27.675461 2014-08-13 17:38:02.708235 7fe9bae887a0 20 read_log 191727'437 (0'0) modify d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 by client.275175.0:36944 2014-07-30 14:20:28.218693 2014-08-13 17:38:02.708247 7fe9bae887a0 20 read_log 191727'438 (0'0) modify 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 by client.275175.0:37179 2014-07-30 14:20:28.869719 2014-08-13 17:38:02.708260 7fe9bae887a0 20 read_log 192888'440 (0'0) modify d8799e19/benchmark_data_storage0111-ib.pocstor_35625_object2250/head//22 by client.940330.0:2251 2014-08-11 16:29:49.464596 2014-08-13 17:38:02.708272 7fe9bae887a0 20 read_log 192888'441 (0'0) modify a139de19/benchmark_data_storage0111-ib.pocstor_35625_object2379/head//22 by client.940330.0:2380 2014-08-11 16:29:49.922455 2014-08-13 17:38:02.708283 7fe9bae887a0 20 read_log 192888'442 (0'0) modify e01de19/benchmark_data_storage0111-ib.pocstor_35625_object2430/head//22 by client.940330.0:2431 2014-08-11 16:29:50.130489 2014-08-13 17:38:02.708295 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 3ded4619/benchmark_data_storage0111-ib.pocstor_35625_object3012/head//22 by client.940330.0:3013 2014-08-11 16:29:52.617984 2014-08-13 17:38:02.708308 7fe9bae887a0 20 read_log 192888'444 (0'0) modify c6684619/benchmark_data_storage0111-ib.pocstor_35625_object4761/head//22 by client.940330.0:4762 2014-08-11 16:29:59.190071 2014-08-13 17:38:02.708320 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 4a0eb619/benchmark_data_storage0111-ib.pocstor_35625_object6023/head//22 by client.940330.0:6024 2014-08-11 16:30:03.656338 2014-08-13 17:38:02.708333 7fe9bae887a0 20 read_log 192888'446 (0'0) modify f6306619/benchmark_data_storage0111-ib.pocstor_35625_object8063/head//22 by client.940330.0:8064 2014-08-11 16:30:11.061091 2014-08-13 17:38:02.708345 7fe9bae887a0 20 read_log 192888'447 (0'0) modify bee55619/benchmark_data_storage0111-ib.pocstor_35625_object10707/head//22 by client.940330.0:10708 2014-08-11 16:30:20.662088 2014-08-13 17:38:02.708357 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 29104619/benchmark_data_storage0111-ib.pocstor_35625_object18576/head//22 by client.940330.0:18577 2014-08-11 16:30:51.046149 2014-08-13 17:38:02.708369 7fe9bae887a0 20 read_log 192888'449 (0'0) modify ceda9e19/benchmark_data_storage0111-ib.pocstor_35625_object19159/head//22 by client.940330.0:19160 2014-08-11 16:30:53.251335 2014-08-13 17:38:02.708381 7fe9bae887a0 20 read_log 192888'450 (0'0) modify dcddfe19/benchmark_data_storage0111-ib.pocstor_35625_object24902/head//22 by client.940330.0:24903 2014-08-11 16:31:15.493569 2014-08-13 17:38:02.708393 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 3a4aa619/benchmark_data_storage0111-ib.pocstor_35625_object25750/head//22 by client.940330.0:25751 2014-08-11 16:31:18.810831 2014-08-13 17:38:02.708405 7fe9bae887a0 20 read_log 192888'452 (0'0) modify c6c53e19/benchmark_data_storage0111-ib.pocstor_35625_object25895/head//22 by client.940330.0:25896 2014-08-11 16:31:19.379818 2014-08-13 17:38:02.708417 7fe9bae887a0 20 read_log 192888'453 (0'0) modify fec94e19/benchmark_data_storage0111-ib.pocstor_35625_object29351/head//22 by client.940330.0:29352 2014-08-11 16:31:33.004646 2014-08-13 17:38:02.708440 7fe9bae887a0 20 read_log 192888'454 (192888'440) delete d8799e19/benchmark_data_storage0111-ib.pocstor_35625_object2250/head//22 by client.940330.0:33558 2014-08-11 16:31:42.056390 2014-08-13 17:38:02.708454 7fe9bae887a0 20 read_log 192888'455 (192888'441) delete a139de19/benchmark_data_storage0111-ib.pocstor_35625_object2379/head//22 by client.940330.0:33687 2014-08-11 16:31:42.090135 2014-08-13 17:38:02.708466 7fe9bae887a0 20 read_log 192888'456 (192888'442) delete e01de19/benchmark_data_storage0111-ib.pocstor_35625_object2430/head//22 by client.940330.0:33738 2014-08-11 16:31:42.103048 2014-08-13 17:38:02.708479 7fe9bae887a0 20 read_log 192888'457 (192888'443) delete 3ded4619/benchmark_data_storage0111-ib.pocstor_35625_object3012/head//22 by client.940330.0:34320 2014-08-11 16:31:42.253654 2014-08-13 17:38:02.708491 7fe9bae887a0 20 read_log 192888'458 (192888'444) delete c6684619/benchmark_data_storage0111-ib.pocstor_35625_object4761/head//22 by client.940330.0:36069 2014-08-11 16:31:42.756067 2014-08-13 17:38:02.708503 7fe9bae887a0 20 read_log 192888'459 (192888'445) delete 4a0eb619/benchmark_data_storage0111-ib.pocstor_35625_object6023/head//22 by client.940330.0:37331 2014-08-11 16:31:43.082103 2014-08-13 17:38:02.708515 7fe9bae887a0 20 read_log 192888'460 (192888'446) delete f6306619/benchmark_data_storage0111-ib.pocstor_35625_object8063/head//22 by client.940330.0:39371 2014-08-11 16:31:43.608541 2014-08-13 17:38:02.708527 7fe9bae887a0 20 read_log 192888'461 (192888'447) delete bee55619/benchmark_data_storage0111-ib.pocstor_35625_object10707/head//22 by client.940330.0:42015 2014-08-11 16:31:44.314017 2014-08-13 17:38:02.708540 7fe9bae887a0 20 read_log 192888'462 (192888'448) delete 29104619/benchmark_data_storage0111-ib.pocstor_35625_object18576/head//22 by client.940330.0:49884 2014-08-11 16:31:46.467810 2014-08-13 17:38:02.708552 7fe9bae887a0 20 read_log 192888'463 (192888'449) delete ceda9e19/benchmark_data_storage0111-ib.pocstor_35625_object19159/head//22 by client.940330.0:50467 2014-08-11 16:31:46.623093 2014-08-13 17:38:02.708564 7fe9bae887a0 20 read_log 192888'464 (192888'450) delete dcddfe19/benchmark_data_storage0111-ib.pocstor_35625_object24902/head//22 by client.940330.0:56210 2014-08-11 16:31:48.310835 2014-08-13 17:38:02.708576 7fe9bae887a0 20 read_log 192888'465 (192888'451) delete 3a4aa619/benchmark_data_storage0111-ib.pocstor_35625_object25750/head//22 by client.940330.0:57058 2014-08-11 16:31:48.573647 2014-08-13 17:38:02.708588 7fe9bae887a0 20 read_log 192888'466 (192888'452) delete c6c53e19/benchmark_data_storage0111-ib.pocstor_35625_object25895/head//22 by client.940330.0:57203 2014-08-11 16:31:48.615773 2014-08-13 17:38:02.708600 7fe9bae887a0 20 read_log 192888'467 (192888'453) delete fec94e19/benchmark_data_storage0111-ib.pocstor_35625_object29351/head//22 by client.940330.0:60659 2014-08-11 16:31:49.569061 2014-08-13 17:38:02.708612 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 90dbce19/benchmark_data_storage0111-ib.pocstor_64498_object2782/head//22 by client.1049635.0:2783 2014-08-12 17:17:26.077991 2014-08-13 17:38:02.708624 7fe9bae887a0 20 read_log 194215'469 (0'0) modify d0476e19/benchmark_data_storage0111-ib.pocstor_64498_object4601/head//22 by client.1049635.0:4602 2014-08-12 17:17:31.776557 2014-08-13 17:38:02.708637 7fe9bae887a0 20 read_log 194215'470 (0'0) modify 568b619/benchmark_data_storage0111-ib.pocstor_64498_object4720/head//22 by client.1049635.0:4721 2014-08-12 17:17:32.102126 2014-08-13 17:38:02.708648 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 41a81619/benchmark_data_storage0111-ib.pocstor_64498_object6022/head//22 by client.1049635.0:6023 2014-08-12 17:17:36.139569 2014-08-13 17:38:02.708660 7fe9bae887a0 20 read_log 194215'472 (0'0) modify f9b9ee19/benchmark_data_storage0111-ib.pocstor_64498_object10725/head//22 by client.1049635.0:10726 2014-08-12 17:17:50.306474 2014-08-13 17:38:02.708685 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 93b14e19/benchmark_data_storage0111-ib.pocstor_64498_object11619/head//22 by client.1049635.0:11620 2014-08-12 17:17:53.269704 2014-08-13 17:38:02.708698 7fe9bae887a0 20 read_log 194215'474 (0'0) modify 5e8eee19/benchmark_data_storage0111-ib.pocstor_64498_object12617/head//22 by client.1049635.0:12618 2014-08-12 17:17:56.215775 2014-08-13 17:38:02.708712 7fe9bae887a0 20 read_log 194215'475 (0'0) modify eb67c619/benchmark_data_storage0111-ib.pocstor_64498_object12901/head//22 by client.1049635.0:12902 2014-08-12 17:17:57.081811 2014-08-13 17:38:02.708724 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 58943e19/benchmark_data_storage0111-ib.pocstor_64498_object14044/head//22 by client.1049635.0:14045 2014-08-12 17:18:00.657127 2014-08-13 17:38:02.708736 7fe9bae887a0 20 read_log 194215'477 (0'0) modify 5c9ebe19/benchmark_data_storage0111-ib.pocstor_64498_object15633/head//22 by client.1049635.0:15634 2014-08-12 17:18:06.212147 2014-08-13 17:38:02.708748 7fe9bae887a0 20 read_log 194215'478 (0'0) modify 6b2cde19/benchmark_data_storage0111-ib.pocstor_64498_object15829/head//22 by client.1049635.0:15830 2014-08-12 17:18:06.915291 2014-08-13 17:38:02.708760 7fe9bae887a0 20 read_log 194215'479 (0'0) modify eb12e19/benchmark_data_storage0111-ib.pocstor_64498_object16157/head//22 by client.1049635.0:16158 2014-08-12 17:18:08.151499 2014-08-13 17:38:02.708772 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 7e639619/benchmark_data_storage0111-ib.pocstor_64498_object17752/head//22 by client.1049635.0:17753 2014-08-12 17:18:14.217650 2014-08-13 17:38:02.708783 7fe9bae887a0 20 read_log 194215'481 (0'0) modify e9bed619/benchmark_data_storage0111-ib.pocstor_64498_object21543/head//22 by client.1049635.0:21544 2014-08-12 17:18:28.615139 2014-08-13 17:38:02.708796 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 5f294e19/benchmark_data_storage0111-ib.pocstor_64498_object25229/head//22 by client.1049635.0:25230 2014-08-12 17:18:42.562190 2014-08-13 17:38:02.708808 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 25894619/benchmark_data_storage0111-ib.pocstor_64498_object26112/head//22 by client.1049635.0:26113 2014-08-12 17:18:45.842059 2014-08-13 17:38:02.708819 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 2c8d6619/benchmark_data_storage0111-ib.pocstor_64498_object29086/head//22 by client.1049635.0:29087 2014-08-12 17:18:56.914299 2014-08-13 17:38:02.708831 7fe9bae887a0 20 read_log 194215'485 (0'0) modify a48d1619/benchmark_data_storage0111-ib.pocstor_64498_object29131/head//22 by client.1049635.0:29132 2014-08-12 17:18:57.084320 2014-08-13 17:38:02.708843 7fe9bae887a0 20 read_log 194215'486 (0'0) modify b4bc4e19/benchmark_data_storage0111-ib.pocstor_64498_object29243/head//22 by client.1049635.0:29244 2014-08-12 17:18:57.498700 2014-08-13 17:38:02.708855 7fe9bae887a0 20 read_log 194215'487 (0'0) modify a5b04e19/benchmark_data_storage0111-ib.pocstor_64498_object29441/head//22 by client.1049635.0:29442 2014-08-12 17:18:58.287329 2014-08-13 17:38:02.708867 7fe9bae887a0 20 read_log 194215'488 (0'0) modify cef83e19/benchmark_data_storage0111-ib.pocstor_64498_object30080/head//22 by client.1049635.0:30081 2014-08-12 17:19:00.755689 2014-08-13 17:38:02.708879 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 4b5fee19/benchmark_data_storage0111-ib.pocstor_64498_object32902/head//22 by client.1049635.0:32903 2014-08-12 17:19:12.024870 2014-08-13 17:38:02.708892 7fe9bae887a0 20 read_log 194215'490 (194215'468) delete 90dbce19/benchmark_data_storage0111-ib.pocstor_64498_object2782/head//22 by client.1049635.0:36985 2014-08-12 17:19:18.893947 2014-08-13 17:38:02.708904 7fe9bae887a0 20 read_log 194215'491 (194215'469) delete d0476e19/benchmark_data_storage0111-ib.pocstor_64498_object4601/head//22 by client.1049635.0:38804 2014-08-12 17:19:19.469360 2014-08-13 17:38:02.708916 7fe9bae887a0 20 read_log 194215'492 (194215'470) delete 568b619/benchmark_data_storage0111-ib.pocstor_64498_object4720/head//22 by client.1049635.0:38923 2014-08-12 17:19:19.508658 2014-08-13 17:38:02.708936 7fe9bae887a0 20 read_log 194215'493 (194215'471) delete 41a81619/benchmark_data_storage0111-ib.pocstor_64498_object6022/head//22 by client.1049635.0:40225 2014-08-12 17:19:19.965279 2014-08-13 17:38:02.708949 7fe9bae887a0 20 read_log 194215'494 (194215'472) delete f9b9ee19/benchmark_data_storage0111-ib.pocstor_64498_object10725/head//22 by client.1049635.0:44928 2014-08-12 17:19:21.492121 2014-08-13 17:38:02.708964 7fe9bae887a0 20 read_log 194215'495 (194215'473) delete 93b14e19/benchmark_data_storage0111-ib.pocstor_64498_object11619/head//22 by client.1049635.0:45822 2014-08-12 17:19:21.772387 2014-08-13 17:38:02.708977 7fe9bae887a0 20 read_log 194215'496 (194215'474) delete 5e8eee19/benchmark_data_storage0111-ib.pocstor_64498_object12617/head//22 by client.1049635.0:46820 2014-08-12 17:19:22.089537 2014-08-13 17:38:02.708989 7fe9bae887a0 20 read_log 194215'497 (194215'475) delete eb67c619/benchmark_data_storage0111-ib.pocstor_64498_object12901/head//22 by client.1049635.0:47104 2014-08-12 17:19:22.182088 2014-08-13 17:38:02.709001 7fe9bae887a0 20 read_log 194215'498 (194215'476) delete 58943e19/benchmark_data_storage0111-ib.pocstor_64498_object14044/head//22 by client.1049635.0:48247 2014-08-12 17:19:22.577446 2014-08-13 17:38:02.709012 7fe9bae887a0 20 read_log 194215'499 (194215'477) delete 5c9ebe19/benchmark_data_storage0111-ib.pocstor_64498_object15633/head//22 by client.1049635.0:49836 2014-08-12 17:19:23.074197 2014-08-13 17:38:02.709025 7fe9bae887a0 20 read_log 194215'500 (194215'478) delete 6b2cde19/benchmark_data_storage0111-ib.pocstor_64498_object15829/head//22 by client.1049635.0:50032 2014-08-12 17:19:23.129989 2014-08-13 17:38:02.709037 7fe9bae887a0 20 read_log 194215'501 (194215'479) delete eb12e19/benchmark_data_storage0111-ib.pocstor_64498_object16157/head//22 by client.1049635.0:50360 2014-08-12 17:19:23.236921 2014-08-13 17:38:02.709049 7fe9bae887a0 20 read_log 194215'502 (194215'480) delete 7e639619/benchmark_data_storage0111-ib.pocstor_64498_object17752/head//22 by client.1049635.0:51955 2014-08-12 17:19:23.743676 2014-08-13 17:38:02.709061 7fe9bae887a0 20 read_log 194215'503 (194215'481) delete e9bed619/benchmark_data_storage0111-ib.pocstor_64498_object21543/head//22 by client.1049635.0:55746 2014-08-12 17:19:25.068569 2014-08-13 17:38:02.709073 7fe9bae887a0 20 read_log 194215'504 (194215'482) delete 5f294e19/benchmark_data_storage0111-ib.pocstor_64498_object25229/head//22 by client.1049635.0:59432 2014-08-12 17:19:26.398253 2014-08-13 17:38:02.709085 7fe9bae887a0 20 read_log 194215'505 (194215'483) delete 25894619/benchmark_data_storage0111-ib.pocstor_64498_object26112/head//22 by client.1049635.0:60315 2014-08-12 17:19:26.692702 2014-08-13 17:38:02.709096 7fe9bae887a0 20 read_log 194215'506 (194215'484) delete 2c8d6619/benchmark_data_storage0111-ib.pocstor_64498_object29086/head//22 by client.1049635.0:63289 2014-08-12 17:19:27.733861 2014-08-13 17:38:02.709108 7fe9bae887a0 20 read_log 194215'507 (194215'485) delete a48d1619/benchmark_data_storage0111-ib.pocstor_64498_object29131/head//22 by client.1049635.0:63334 2014-08-12 17:19:27.749972 2014-08-13 17:38:02.709120 7fe9bae887a0 20 read_log 194215'508 (194215'486) delete b4bc4e19/benchmark_data_storage0111-ib.pocstor_64498_object29243/head//22 by client.1049635.0:63446 2014-08-12 17:19:27.792695 2014-08-13 17:38:02.709151 7fe9bae887a0 20 read_log 194215'509 (194215'487) delete a5b04e19/benchmark_data_storage0111-ib.pocstor_64498_object29441/head//22 by client.1049635.0:63644 2014-08-12 17:19:27.864105 2014-08-13 17:38:02.709173 7fe9bae887a0 20 read_log 194215'510 (194215'488) delete cef83e19/benchmark_data_storage0111-ib.pocstor_64498_object30080/head//22 by client.1049635.0:64283 2014-08-12 17:19:28.072083 2014-08-13 17:38:02.709185 7fe9bae887a0 20 read_log 194215'511 (194215'489) delete 4b5fee19/benchmark_data_storage0111-ib.pocstor_64498_object32902/head//22 by client.1049635.0:67105 2014-08-12 17:19:29.083326 2014-08-13 17:38:02.709307 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.709326 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=0 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.709342 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=0 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001727 0 0.000000 2014-08-13 17:38:02.709354 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=0 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.709362 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=0 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.709371 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'511], crt=191727'439) 2014-08-13 17:38:02.709380 7fe9bae887a0 10 osd.4 194478 pgid 22.630 coll 22.630_head 2014-08-13 17:38:02.709540 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.630 2014-08-13 17:38:02.709549 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.709576 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.630(unlocked)] enter Initial 2014-08-13 17:38:02.709587 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.630(unlocked)] enter NotTrimming 2014-08-13 17:38:02.709712 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.709823 7fe9bae887a0 20 read_log 191648'66 (0'0) modify ccb6de30/benchmark_data_storage0111-ib.pocstor_40745_object4310/head//22 by client.273158.0:4311 2014-07-30 14:09:08.019610 2014-08-13 17:38:02.709843 7fe9bae887a0 20 read_log 191648'186 (0'0) modify edbf4e30/benchmark_data_storage0111-ib.pocstor_40745_object11670/head//22 by client.273158.0:11671 2014-07-30 14:09:45.475408 2014-08-13 17:38:02.709856 7fe9bae887a0 20 read_log 191648'188 (0'0) modify 4c565630/benchmark_data_storage0111-ib.pocstor_40745_object11781/head//22 by client.273158.0:11782 2014-07-30 14:09:45.941658 2014-08-13 17:38:02.709868 7fe9bae887a0 20 read_log 191648'196 (0'0) modify c197f630/benchmark_data_storage0111-ib.pocstor_40745_object12230/head//22 by client.273158.0:12231 2014-07-30 14:09:48.741592 2014-08-13 17:38:02.709880 7fe9bae887a0 20 read_log 191648'198 (0'0) modify 2fac6e30/benchmark_data_storage0111-ib.pocstor_40745_object12340/head//22 by client.273158.0:12341 2014-07-30 14:09:49.247038 2014-08-13 17:38:02.709892 7fe9bae887a0 20 read_log 191648'212 (0'0) modify 49753e30/benchmark_data_storage0111-ib.pocstor_40745_object13073/head//22 by client.273158.0:13074 2014-07-30 14:09:52.795488 2014-08-13 17:38:02.709904 7fe9bae887a0 20 read_log 191648'242 (0'0) modify 5f5dae30/benchmark_data_storage0111-ib.pocstor_40745_object14641/head//22 by client.273158.0:14642 2014-07-30 14:10:00.755469 2014-08-13 17:38:02.709917 7fe9bae887a0 20 read_log 191648'251 (0'0) modify 3560de30/benchmark_data_storage0111-ib.pocstor_40745_object15315/head//22 by client.273158.0:15316 2014-07-30 14:10:03.924044 2014-08-13 17:38:02.709929 7fe9bae887a0 20 read_log 191648'256 (0'0) modify 3e3d2630/benchmark_data_storage0111-ib.pocstor_40745_object15942/head//22 by client.273158.0:15943 2014-07-30 14:10:07.226671 2014-08-13 17:38:02.709941 7fe9bae887a0 20 read_log 191648'287 (0'0) modify 9065630/benchmark_data_storage0111-ib.pocstor_40745_object17781/head//22 by client.273158.0:17782 2014-07-30 14:10:16.980555 2014-08-13 17:38:02.709952 7fe9bae887a0 20 read_log 191648'316 (0'0) modify 2efb8630/benchmark_data_storage0111-ib.pocstor_40745_object19161/head//22 by client.273158.0:19162 2014-07-30 14:10:24.241436 2014-08-13 17:38:02.709964 7fe9bae887a0 20 read_log 191648'323 (0'0) modify 997c3630/benchmark_data_storage0111-ib.pocstor_40745_object19577/head//22 by client.273158.0:19578 2014-07-30 14:10:27.016986 2014-08-13 17:38:02.709976 7fe9bae887a0 20 read_log 191648'330 (0'0) modify bfb22630/benchmark_data_storage0111-ib.pocstor_40745_object20063/head//22 by client.273158.0:20064 2014-07-30 14:10:31.541128 2014-08-13 17:38:02.709988 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 90abe30/benchmark_data_storage0111-ib.pocstor_51285_object7394/head//22 by client.275175.0:7395 2014-07-30 14:18:57.836886 2014-08-13 17:38:02.710000 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 9e946e30/benchmark_data_storage0111-ib.pocstor_51285_object10525/head//22 by client.275175.0:10526 2014-07-30 14:19:06.711768 2014-08-13 17:38:02.710012 7fe9bae887a0 20 read_log 191727'395 (0'0) modify a4132e30/benchmark_data_storage0111-ib.pocstor_51285_object10791/head//22 by client.275175.0:10792 2014-07-30 14:19:07.459118 2014-08-13 17:38:02.710031 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 890d3630/benchmark_data_storage0111-ib.pocstor_51285_object10893/head//22 by client.275175.0:10894 2014-07-30 14:19:07.745140 2014-08-13 17:38:02.710044 7fe9bae887a0 20 read_log 191727'400 (0'0) modify ff3ece30/benchmark_data_storage0111-ib.pocstor_51285_object13102/head//22 by client.275175.0:13103 2014-07-30 14:19:14.982817 2014-08-13 17:38:02.710056 7fe9bae887a0 20 read_log 191727'401 (0'0) modify 80578e30/benchmark_data_storage0111-ib.pocstor_51285_object13945/head//22 by client.275175.0:13946 2014-07-30 14:19:17.604438 2014-08-13 17:38:02.710067 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 2699b630/benchmark_data_storage0111-ib.pocstor_51285_object15300/head//22 by client.275175.0:15301 2014-07-30 14:19:21.673280 2014-08-13 17:38:02.710079 7fe9bae887a0 20 read_log 191727'408 (0'0) modify 22037e30/benchmark_data_storage0111-ib.pocstor_51285_object18542/head//22 by client.275175.0:18543 2014-07-30 14:19:31.604818 2014-08-13 17:38:02.710091 7fe9bae887a0 20 read_log 191727'411 (0'0) modify 55029630/benchmark_data_storage0111-ib.pocstor_51285_object19583/head//22 by client.275175.0:19584 2014-07-30 14:19:35.009655 2014-08-13 17:38:02.710103 7fe9bae887a0 20 read_log 191727'417 (0'0) modify d804ce30/benchmark_data_storage0111-ib.pocstor_51285_object23812/head//22 by client.275175.0:23813 2014-07-30 14:19:48.303786 2014-08-13 17:38:02.710115 7fe9bae887a0 20 read_log 191727'423 (0'0) modify 1aeece30/benchmark_data_storage0111-ib.pocstor_51285_object26166/head//22 by client.275175.0:26167 2014-07-30 14:19:55.341759 2014-08-13 17:38:02.710127 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 5c244e30/benchmark_data_storage0111-ib.pocstor_51285_object27581/head//22 by client.275175.0:27582 2014-07-30 14:19:59.465720 2014-08-13 17:38:02.710138 7fe9bae887a0 20 read_log 191727'428 (0'0) modify f0465630/benchmark_data_storage0111-ib.pocstor_51285_object27756/head//22 by client.275175.0:27757 2014-07-30 14:20:00.030483 2014-08-13 17:38:02.710150 7fe9bae887a0 20 read_log 191727'431 (0'0) modify 7062f630/benchmark_data_storage0111-ib.pocstor_51285_object29294/head//22 by client.275175.0:29295 2014-07-30 14:20:04.802449 2014-08-13 17:38:02.710175 7fe9bae887a0 20 read_log 191727'432 (0'0) modify 5c3ed630/benchmark_data_storage0111-ib.pocstor_51285_object29583/head//22 by client.275175.0:29584 2014-07-30 14:20:05.711911 2014-08-13 17:38:02.710188 7fe9bae887a0 20 read_log 191727'442 (0'0) modify 2f19fe30/benchmark_data_storage0111-ib.pocstor_51285_object33514/head//22 by client.275175.0:33515 2014-07-30 14:20:17.549561 2014-08-13 17:38:02.710220 7fe9bae887a0 20 read_log 191727'446 (0'0) modify f02a7e30/benchmark_data_storage0111-ib.pocstor_51285_object35075/head//22 by client.275175.0:35076 2014-07-30 14:20:22.197940 2014-08-13 17:38:02.710232 7fe9bae887a0 20 read_log 191727'448 (0'0) modify defca630/benchmark_data_storage0111-ib.pocstor_51285_object35754/head//22 by client.275175.0:35755 2014-07-30 14:20:24.466230 2014-08-13 17:38:02.710245 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 1e072e30/benchmark_data_storage0111-ib.pocstor_35625_object253/head//22 by client.940330.0:254 2014-08-11 16:29:41.729995 2014-08-13 17:38:02.710257 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 86e1d630/benchmark_data_storage0111-ib.pocstor_35625_object1389/head//22 by client.940330.0:1390 2014-08-11 16:29:46.243928 2014-08-13 17:38:02.710268 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 14227630/benchmark_data_storage0111-ib.pocstor_35625_object6455/head//22 by client.940330.0:6456 2014-08-11 16:30:05.186232 2014-08-13 17:38:02.710280 7fe9bae887a0 20 read_log 192888'455 (0'0) modify 6dee7e30/benchmark_data_storage0111-ib.pocstor_35625_object7227/head//22 by client.940330.0:7228 2014-08-11 16:30:08.121350 2014-08-13 17:38:02.710300 7fe9bae887a0 20 read_log 192888'456 (0'0) modify 886a7e30/benchmark_data_storage0111-ib.pocstor_35625_object7279/head//22 by client.940330.0:7280 2014-08-11 16:30:08.305273 2014-08-13 17:38:02.710315 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 46c8e630/benchmark_data_storage0111-ib.pocstor_35625_object10117/head//22 by client.940330.0:10118 2014-08-11 16:30:18.318983 2014-08-13 17:38:02.710327 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 65cf7e30/benchmark_data_storage0111-ib.pocstor_35625_object12056/head//22 by client.940330.0:12057 2014-08-11 16:30:26.150625 2014-08-13 17:38:02.710339 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 506cce30/benchmark_data_storage0111-ib.pocstor_35625_object13218/head//22 by client.940330.0:13219 2014-08-11 16:30:30.735058 2014-08-13 17:38:02.710351 7fe9bae887a0 20 read_log 192888'460 (0'0) modify c2866630/benchmark_data_storage0111-ib.pocstor_35625_object14065/head//22 by client.940330.0:14066 2014-08-11 16:30:33.891881 2014-08-13 17:38:02.710362 7fe9bae887a0 20 read_log 192888'461 (0'0) modify ec85a630/benchmark_data_storage0111-ib.pocstor_35625_object16118/head//22 by client.940330.0:16119 2014-08-11 16:30:41.674790 2014-08-13 17:38:02.710374 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 9efb6630/benchmark_data_storage0111-ib.pocstor_35625_object17793/head//22 by client.940330.0:17794 2014-08-11 16:30:47.943539 2014-08-13 17:38:02.710386 7fe9bae887a0 20 read_log 192888'463 (0'0) modify ca56fe30/benchmark_data_storage0111-ib.pocstor_35625_object21019/head//22 by client.940330.0:21020 2014-08-11 16:31:00.879709 2014-08-13 17:38:02.710398 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 2ec3de30/benchmark_data_storage0111-ib.pocstor_35625_object21235/head//22 by client.940330.0:21236 2014-08-11 16:31:01.708108 2014-08-13 17:38:02.710409 7fe9bae887a0 20 read_log 192888'465 (0'0) modify 5652b630/benchmark_data_storage0111-ib.pocstor_35625_object23775/head//22 by client.940330.0:23776 2014-08-11 16:31:11.321911 2014-08-13 17:38:02.710421 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 17f87e30/benchmark_data_storage0111-ib.pocstor_35625_object24460/head//22 by client.940330.0:24461 2014-08-11 16:31:13.820467 2014-08-13 17:38:02.710433 7fe9bae887a0 20 read_log 192888'467 (0'0) modify e3936e30/benchmark_data_storage0111-ib.pocstor_35625_object26293/head//22 by client.940330.0:26294 2014-08-11 16:31:20.939065 2014-08-13 17:38:02.710444 7fe9bae887a0 20 read_log 192888'468 (192888'452) delete 1e072e30/benchmark_data_storage0111-ib.pocstor_35625_object253/head//22 by client.940330.0:31561 2014-08-11 16:31:41.546269 2014-08-13 17:38:02.710457 7fe9bae887a0 20 read_log 192888'469 (192888'453) delete 86e1d630/benchmark_data_storage0111-ib.pocstor_35625_object1389/head//22 by client.940330.0:32697 2014-08-11 16:31:41.841847 2014-08-13 17:38:02.710469 7fe9bae887a0 20 read_log 192888'470 (192888'454) delete 14227630/benchmark_data_storage0111-ib.pocstor_35625_object6455/head//22 by client.940330.0:37763 2014-08-11 16:31:43.196760 2014-08-13 17:38:02.710481 7fe9bae887a0 20 read_log 192888'471 (192888'455) delete 6dee7e30/benchmark_data_storage0111-ib.pocstor_35625_object7227/head//22 by client.940330.0:38535 2014-08-11 16:31:43.392899 2014-08-13 17:38:02.710493 7fe9bae887a0 20 read_log 192888'472 (192888'456) delete 886a7e30/benchmark_data_storage0111-ib.pocstor_35625_object7279/head//22 by client.940330.0:38587 2014-08-11 16:31:43.407930 2014-08-13 17:38:02.710505 7fe9bae887a0 20 read_log 192888'473 (192888'457) delete 46c8e630/benchmark_data_storage0111-ib.pocstor_35625_object10117/head//22 by client.940330.0:41425 2014-08-11 16:31:44.147606 2014-08-13 17:38:02.710517 7fe9bae887a0 20 read_log 192888'474 (192888'458) delete 65cf7e30/benchmark_data_storage0111-ib.pocstor_35625_object12056/head//22 by client.940330.0:43364 2014-08-11 16:31:44.671607 2014-08-13 17:38:02.710529 7fe9bae887a0 20 read_log 192888'475 (192888'459) delete 506cce30/benchmark_data_storage0111-ib.pocstor_35625_object13218/head//22 by client.940330.0:44526 2014-08-11 16:31:44.980844 2014-08-13 17:38:02.710548 7fe9bae887a0 20 read_log 192888'476 (192888'460) delete c2866630/benchmark_data_storage0111-ib.pocstor_35625_object14065/head//22 by client.940330.0:45373 2014-08-11 16:31:45.200174 2014-08-13 17:38:02.710561 7fe9bae887a0 20 read_log 192888'477 (192888'461) delete ec85a630/benchmark_data_storage0111-ib.pocstor_35625_object16118/head//22 by client.940330.0:47426 2014-08-11 16:31:45.756335 2014-08-13 17:38:02.710573 7fe9bae887a0 20 read_log 192888'478 (192888'462) delete 9efb6630/benchmark_data_storage0111-ib.pocstor_35625_object17793/head//22 by client.940330.0:49101 2014-08-11 16:31:46.220009 2014-08-13 17:38:02.710585 7fe9bae887a0 20 read_log 192888'479 (192888'463) delete ca56fe30/benchmark_data_storage0111-ib.pocstor_35625_object21019/head//22 by client.940330.0:52327 2014-08-11 16:31:47.167955 2014-08-13 17:38:02.710597 7fe9bae887a0 20 read_log 192888'480 (192888'464) delete 2ec3de30/benchmark_data_storage0111-ib.pocstor_35625_object21235/head//22 by client.940330.0:52543 2014-08-11 16:31:47.223453 2014-08-13 17:38:02.710609 7fe9bae887a0 20 read_log 192888'481 (192888'465) delete 5652b630/benchmark_data_storage0111-ib.pocstor_35625_object23775/head//22 by client.940330.0:55083 2014-08-11 16:31:47.970380 2014-08-13 17:38:02.710621 7fe9bae887a0 20 read_log 192888'482 (192888'466) delete 17f87e30/benchmark_data_storage0111-ib.pocstor_35625_object24460/head//22 by client.940330.0:55768 2014-08-11 16:31:48.175319 2014-08-13 17:38:02.710633 7fe9bae887a0 20 read_log 192888'483 (192888'467) delete e3936e30/benchmark_data_storage0111-ib.pocstor_35625_object26293/head//22 by client.940330.0:57601 2014-08-11 16:31:48.731381 2014-08-13 17:38:02.710645 7fe9bae887a0 20 read_log 194215'484 (0'0) modify e6629e30/benchmark_data_storage0111-ib.pocstor_64498_object778/head//22 by client.1049635.0:779 2014-08-12 17:17:19.634853 2014-08-13 17:38:02.710658 7fe9bae887a0 20 read_log 194215'485 (0'0) modify c9e6630/benchmark_data_storage0111-ib.pocstor_64498_object4406/head//22 by client.1049635.0:4407 2014-08-12 17:17:31.240668 2014-08-13 17:38:02.710670 7fe9bae887a0 20 read_log 194215'486 (0'0) modify 3f817630/benchmark_data_storage0111-ib.pocstor_64498_object4640/head//22 by client.1049635.0:4641 2014-08-12 17:17:31.884887 2014-08-13 17:38:02.710682 7fe9bae887a0 20 read_log 194215'487 (0'0) modify a8bdf630/benchmark_data_storage0111-ib.pocstor_64498_object4686/head//22 by client.1049635.0:4687 2014-08-12 17:17:32.009525 2014-08-13 17:38:02.710693 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 12e28630/benchmark_data_storage0111-ib.pocstor_64498_object4929/head//22 by client.1049635.0:4930 2014-08-12 17:17:32.733804 2014-08-13 17:38:02.710705 7fe9bae887a0 20 read_log 194215'489 (0'0) modify e80d5e30/benchmark_data_storage0111-ib.pocstor_64498_object6302/head//22 by client.1049635.0:6303 2014-08-12 17:17:36.996353 2014-08-13 17:38:02.710717 7fe9bae887a0 20 read_log 194215'490 (0'0) modify fed24630/benchmark_data_storage0111-ib.pocstor_64498_object7465/head//22 by client.1049635.0:7466 2014-08-12 17:17:40.438736 2014-08-13 17:38:02.710729 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 6aba4630/benchmark_data_storage0111-ib.pocstor_64498_object14935/head//22 by client.1049635.0:14936 2014-08-12 17:18:03.754370 2014-08-13 17:38:02.710741 7fe9bae887a0 20 read_log 194215'492 (0'0) modify 25bb1e30/benchmark_data_storage0111-ib.pocstor_64498_object19012/head//22 by client.1049635.0:19013 2014-08-12 17:18:18.984034 2014-08-13 17:38:02.710753 7fe9bae887a0 20 read_log 194215'493 (0'0) modify ef399630/benchmark_data_storage0111-ib.pocstor_64498_object19842/head//22 by client.1049635.0:19843 2014-08-12 17:18:22.189380 2014-08-13 17:38:02.710765 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 7e542630/benchmark_data_storage0111-ib.pocstor_64498_object23129/head//22 by client.1049635.0:23130 2014-08-12 17:18:34.613226 2014-08-13 17:38:02.710805 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 5c645630/benchmark_data_storage0111-ib.pocstor_64498_object24835/head//22 by client.1049635.0:24836 2014-08-12 17:18:41.012317 2014-08-13 17:38:02.710821 7fe9bae887a0 20 read_log 194215'496 (0'0) modify dac93e30/benchmark_data_storage0111-ib.pocstor_64498_object26247/head//22 by client.1049635.0:26248 2014-08-12 17:18:46.343797 2014-08-13 17:38:02.710834 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 18270630/benchmark_data_storage0111-ib.pocstor_64498_object26408/head//22 by client.1049635.0:26409 2014-08-12 17:18:46.942457 2014-08-13 17:38:02.710845 7fe9bae887a0 20 read_log 194215'498 (0'0) modify 58c28630/benchmark_data_storage0111-ib.pocstor_64498_object28675/head//22 by client.1049635.0:28676 2014-08-12 17:18:55.393165 2014-08-13 17:38:02.710857 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 6dc85630/benchmark_data_storage0111-ib.pocstor_64498_object31292/head//22 by client.1049635.0:31293 2014-08-12 17:19:05.697133 2014-08-13 17:38:02.710869 7fe9bae887a0 20 read_log 194215'500 (0'0) modify 9a8bc630/benchmark_data_storage0111-ib.pocstor_64498_object31461/head//22 by client.1049635.0:31462 2014-08-12 17:19:06.448808 2014-08-13 17:38:02.710882 7fe9bae887a0 20 read_log 194215'501 (194215'484) delete e6629e30/benchmark_data_storage0111-ib.pocstor_64498_object778/head//22 by client.1049635.0:34981 2014-08-12 17:19:18.245287 2014-08-13 17:38:02.710894 7fe9bae887a0 20 read_log 194215'502 (194215'485) delete c9e6630/benchmark_data_storage0111-ib.pocstor_64498_object4406/head//22 by client.1049635.0:38609 2014-08-12 17:19:19.408174 2014-08-13 17:38:02.710906 7fe9bae887a0 20 read_log 194215'503 (194215'486) delete 3f817630/benchmark_data_storage0111-ib.pocstor_64498_object4640/head//22 by client.1049635.0:38843 2014-08-12 17:19:19.482886 2014-08-13 17:38:02.710919 7fe9bae887a0 20 read_log 194215'504 (194215'487) delete a8bdf630/benchmark_data_storage0111-ib.pocstor_64498_object4686/head//22 by client.1049635.0:38889 2014-08-12 17:19:19.496888 2014-08-13 17:38:02.710931 7fe9bae887a0 20 read_log 194215'505 (194215'488) delete 12e28630/benchmark_data_storage0111-ib.pocstor_64498_object4929/head//22 by client.1049635.0:39132 2014-08-12 17:19:19.579881 2014-08-13 17:38:02.710944 7fe9bae887a0 20 read_log 194215'506 (194215'489) delete e80d5e30/benchmark_data_storage0111-ib.pocstor_64498_object6302/head//22 by client.1049635.0:40505 2014-08-12 17:19:20.055383 2014-08-13 17:38:02.710957 7fe9bae887a0 20 read_log 194215'507 (194215'490) delete fed24630/benchmark_data_storage0111-ib.pocstor_64498_object7465/head//22 by client.1049635.0:41668 2014-08-12 17:19:20.438798 2014-08-13 17:38:02.711025 7fe9bae887a0 20 read_log 194215'508 (194215'491) delete 6aba4630/benchmark_data_storage0111-ib.pocstor_64498_object14935/head//22 by client.1049635.0:49138 2014-08-12 17:19:22.855414 2014-08-13 17:38:02.711043 7fe9bae887a0 20 read_log 194215'509 (194215'492) delete 25bb1e30/benchmark_data_storage0111-ib.pocstor_64498_object19012/head//22 by client.1049635.0:53215 2014-08-12 17:19:24.170332 2014-08-13 17:38:02.711055 7fe9bae887a0 20 read_log 194215'510 (194215'493) delete ef399630/benchmark_data_storage0111-ib.pocstor_64498_object19842/head//22 by client.1049635.0:54045 2014-08-12 17:19:24.460480 2014-08-13 17:38:02.711068 7fe9bae887a0 20 read_log 194215'511 (194215'494) delete 7e542630/benchmark_data_storage0111-ib.pocstor_64498_object23129/head//22 by client.1049635.0:57332 2014-08-12 17:19:25.665517 2014-08-13 17:38:02.711080 7fe9bae887a0 20 read_log 194215'512 (194215'495) delete 5c645630/benchmark_data_storage0111-ib.pocstor_64498_object24835/head//22 by client.1049635.0:59038 2014-08-12 17:19:26.266897 2014-08-13 17:38:02.711092 7fe9bae887a0 20 read_log 194215'513 (194215'496) delete dac93e30/benchmark_data_storage0111-ib.pocstor_64498_object26247/head//22 by client.1049635.0:60450 2014-08-12 17:19:26.741059 2014-08-13 17:38:02.711104 7fe9bae887a0 20 read_log 194215'514 (194215'497) delete 18270630/benchmark_data_storage0111-ib.pocstor_64498_object26408/head//22 by client.1049635.0:60611 2014-08-12 17:19:26.792218 2014-08-13 17:38:02.711118 7fe9bae887a0 20 read_log 194215'515 (194215'498) delete 58c28630/benchmark_data_storage0111-ib.pocstor_64498_object28675/head//22 by client.1049635.0:62878 2014-08-12 17:19:27.588020 2014-08-13 17:38:02.711130 7fe9bae887a0 20 read_log 194215'516 (194215'499) delete 6dc85630/benchmark_data_storage0111-ib.pocstor_64498_object31292/head//22 by client.1049635.0:65495 2014-08-12 17:19:28.482205 2014-08-13 17:38:02.711142 7fe9bae887a0 20 read_log 194215'517 (194215'500) delete 9a8bc630/benchmark_data_storage0111-ib.pocstor_64498_object31461/head//22 by client.1049635.0:65664 2014-08-12 17:19:28.543841 2014-08-13 17:38:02.711227 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.711245 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=0 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.711260 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=0 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Initial 0.001685 0 0.000000 2014-08-13 17:38:02.711272 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=0 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.711280 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=0 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.711288 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] log((0'0,194215'517], crt=192888'483) 2014-08-13 17:38:02.711297 7fe9bae887a0 10 osd.4 194478 pgid 22.639 coll 22.639_head 2014-08-13 17:38:02.711405 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.639 2014-08-13 17:38:02.711414 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.711449 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.639(unlocked)] enter Initial 2014-08-13 17:38:02.711460 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.639(unlocked)] enter NotTrimming 2014-08-13 17:38:02.711573 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.711684 7fe9bae887a0 20 read_log 191648'1 (0'0) modify 481ec639/benchmark_data_storage0111-ib.pocstor_40745_object275/head//22 by client.273158.0:276 2014-07-30 14:08:49.740856 2014-08-13 17:38:02.711704 7fe9bae887a0 20 read_log 191648'37 (0'0) modify 2730639/benchmark_data_storage0111-ib.pocstor_40745_object2663/head//22 by client.273158.0:2664 2014-07-30 14:08:59.702920 2014-08-13 17:38:02.711717 7fe9bae887a0 20 read_log 191648'121 (0'0) modify 2dab0e39/benchmark_data_storage0111-ib.pocstor_40745_object8454/head//22 by client.273158.0:8455 2014-07-30 14:09:28.563838 2014-08-13 17:38:02.711729 7fe9bae887a0 20 read_log 191648'128 (0'0) modify 4cf57e39/benchmark_data_storage0111-ib.pocstor_40745_object9021/head//22 by client.273158.0:9022 2014-07-30 14:09:31.263988 2014-08-13 17:38:02.711741 7fe9bae887a0 20 read_log 191648'173 (0'0) modify 2f595639/benchmark_data_storage0111-ib.pocstor_40745_object11682/head//22 by client.273158.0:11683 2014-07-30 14:09:45.522309 2014-08-13 17:38:02.711753 7fe9bae887a0 20 read_log 191648'183 (0'0) modify 1ac56e39/benchmark_data_storage0111-ib.pocstor_40745_object12321/head//22 by client.273158.0:12322 2014-07-30 14:09:49.146674 2014-08-13 17:38:02.711765 7fe9bae887a0 20 read_log 191648'305 (0'0) modify 6e85f639/benchmark_data_storage0111-ib.pocstor_40745_object19685/head//22 by client.273158.0:19686 2014-07-30 14:10:27.772334 2014-08-13 17:38:02.711777 7fe9bae887a0 20 read_log 191648'347 (0'0) modify c0f8e39/benchmark_data_storage0111-ib.pocstor_40745_object22642/head//22 by client.273158.0:22643 2014-07-30 14:10:47.646378 2014-08-13 17:38:02.711789 7fe9bae887a0 20 read_log 191727'355 (0'0) modify c1b06639/benchmark_data_storage0111-ib.pocstor_51285_object1020/head//22 by client.275175.0:1021 2014-07-30 14:18:37.530957 2014-08-13 17:38:02.711801 7fe9bae887a0 20 read_log 191727'356 (0'0) modify ede4ce39/benchmark_data_storage0111-ib.pocstor_51285_object1230/head//22 by client.275175.0:1231 2014-07-30 14:18:38.285023 2014-08-13 17:38:02.711813 7fe9bae887a0 20 read_log 191727'359 (0'0) modify b4408639/benchmark_data_storage0111-ib.pocstor_51285_object4502/head//22 by client.275175.0:4503 2014-07-30 14:18:48.864614 2014-08-13 17:38:02.711825 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 4f8a6e39/benchmark_data_storage0111-ib.pocstor_51285_object6571/head//22 by client.275175.0:6572 2014-07-30 14:18:55.037271 2014-08-13 17:38:02.711837 7fe9bae887a0 20 read_log 191727'363 (0'0) modify 2a18de39/benchmark_data_storage0111-ib.pocstor_51285_object7315/head//22 by client.275175.0:7316 2014-07-30 14:18:57.617343 2014-08-13 17:38:02.711849 7fe9bae887a0 20 read_log 191727'369 (0'0) modify ada5c639/benchmark_data_storage0111-ib.pocstor_51285_object12282/head//22 by client.275175.0:12283 2014-07-30 14:19:12.430522 2014-08-13 17:38:02.711869 7fe9bae887a0 20 read_log 191727'375 (0'0) modify d58cfe39/benchmark_data_storage0111-ib.pocstor_51285_object13669/head//22 by client.275175.0:13670 2014-07-30 14:19:16.721800 2014-08-13 17:38:02.711881 7fe9bae887a0 20 read_log 191727'377 (0'0) modify f8884e39/benchmark_data_storage0111-ib.pocstor_51285_object15378/head//22 by client.275175.0:15379 2014-07-30 14:19:21.995669 2014-08-13 17:38:02.711893 7fe9bae887a0 20 read_log 191727'385 (0'0) modify d9b2e39/benchmark_data_storage0111-ib.pocstor_51285_object18936/head//22 by client.275175.0:18937 2014-07-30 14:19:32.991312 2014-08-13 17:38:02.711905 7fe9bae887a0 20 read_log 191727'387 (0'0) modify 8af2e639/benchmark_data_storage0111-ib.pocstor_51285_object19448/head//22 by client.275175.0:19449 2014-07-30 14:19:34.637565 2014-08-13 17:38:02.711917 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 7dc09e39/benchmark_data_storage0111-ib.pocstor_51285_object21016/head//22 by client.275175.0:21017 2014-07-30 14:19:39.177880 2014-08-13 17:38:02.711929 7fe9bae887a0 20 read_log 191727'392 (0'0) modify aafbbe39/benchmark_data_storage0111-ib.pocstor_51285_object22198/head//22 by client.275175.0:22199 2014-07-30 14:19:43.227643 2014-08-13 17:38:02.711941 7fe9bae887a0 20 read_log 191727'394 (0'0) modify e42f2e39/benchmark_data_storage0111-ib.pocstor_51285_object23735/head//22 by client.275175.0:23736 2014-07-30 14:19:48.087461 2014-08-13 17:38:02.711953 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 1c862639/benchmark_data_storage0111-ib.pocstor_51285_object24523/head//22 by client.275175.0:24524 2014-07-30 14:19:50.301945 2014-08-13 17:38:02.711965 7fe9bae887a0 20 read_log 191727'397 (0'0) modify b23b0e39/benchmark_data_storage0111-ib.pocstor_51285_object24643/head//22 by client.275175.0:24644 2014-07-30 14:19:50.638951 2014-08-13 17:38:02.711977 7fe9bae887a0 20 read_log 191727'400 (0'0) modify e6a2e639/benchmark_data_storage0111-ib.pocstor_51285_object26373/head//22 by client.275175.0:26374 2014-07-30 14:19:55.924454 2014-08-13 17:38:02.711991 7fe9bae887a0 20 read_log 191727'406 (0'0) modify c784d639/benchmark_data_storage0111-ib.pocstor_51285_object28589/head//22 by client.275175.0:28590 2014-07-30 14:20:02.683096 2014-08-13 17:38:02.712003 7fe9bae887a0 20 read_log 191727'407 (0'0) modify f6798639/benchmark_data_storage0111-ib.pocstor_51285_object28817/head//22 by client.275175.0:28818 2014-07-30 14:20:03.356140 2014-08-13 17:38:02.712015 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 72f1ae39/benchmark_data_storage0111-ib.pocstor_51285_object31279/head//22 by client.275175.0:31280 2014-07-30 14:20:10.775092 2014-08-13 17:38:02.712027 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 5d1bbe39/benchmark_data_storage0111-ib.pocstor_51285_object32701/head//22 by client.275175.0:32702 2014-07-30 14:20:14.875553 2014-08-13 17:38:02.712039 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 89ba5e39/benchmark_data_storage0111-ib.pocstor_51285_object33105/head//22 by client.275175.0:33106 2014-07-30 14:20:16.092023 2014-08-13 17:38:02.712051 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 37d2ae39/benchmark_data_storage0111-ib.pocstor_51285_object33517/head//22 by client.275175.0:33518 2014-07-30 14:20:17.562583 2014-08-13 17:38:02.712063 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 46606e39/benchmark_data_storage0111-ib.pocstor_51285_object35605/head//22 by client.275175.0:35606 2014-07-30 14:20:23.737240 2014-08-13 17:38:02.712076 7fe9bae887a0 20 read_log 191727'422 (0'0) modify e0ad9e39/benchmark_data_storage0111-ib.pocstor_51285_object35854/head//22 by client.275175.0:35855 2014-07-30 14:20:24.801181 2014-08-13 17:38:02.712088 7fe9bae887a0 20 read_log 191727'426 (0'0) modify 5da08639/benchmark_data_storage0111-ib.pocstor_51285_object38666/head//22 by client.275175.0:38667 2014-07-30 14:20:33.144862 2014-08-13 17:38:02.712100 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 55a70639/benchmark_data_storage0111-ib.pocstor_35625_object199/head//22 by client.940330.0:200 2014-08-11 16:29:41.430738 2014-08-13 17:38:02.712119 7fe9bae887a0 20 read_log 192888'428 (0'0) modify 1ffeb639/benchmark_data_storage0111-ib.pocstor_35625_object2658/head//22 by client.940330.0:2659 2014-08-11 16:29:51.074318 2014-08-13 17:38:02.712135 7fe9bae887a0 20 read_log 192888'429 (0'0) modify 7c3cc639/benchmark_data_storage0111-ib.pocstor_35625_object3860/head//22 by client.940330.0:3861 2014-08-11 16:29:55.819658 2014-08-13 17:38:02.712147 7fe9bae887a0 20 read_log 192888'430 (0'0) modify 5b7c2e39/benchmark_data_storage0111-ib.pocstor_35625_object4180/head//22 by client.940330.0:4181 2014-08-11 16:29:56.984870 2014-08-13 17:38:02.712165 7fe9bae887a0 20 read_log 192888'431 (0'0) modify 7c2fe39/benchmark_data_storage0111-ib.pocstor_35625_object4739/head//22 by client.940330.0:4740 2014-08-11 16:29:59.111739 2014-08-13 17:38:02.712180 7fe9bae887a0 20 read_log 192888'432 (0'0) modify 3b3a2e39/benchmark_data_storage0111-ib.pocstor_35625_object10805/head//22 by client.940330.0:10806 2014-08-11 16:30:21.042933 2014-08-13 17:38:02.712192 7fe9bae887a0 20 read_log 192888'433 (0'0) modify 42e43e39/benchmark_data_storage0111-ib.pocstor_35625_object14551/head//22 by client.940330.0:14552 2014-08-11 16:30:35.821354 2014-08-13 17:38:02.712204 7fe9bae887a0 20 read_log 192888'434 (0'0) modify 5a550639/benchmark_data_storage0111-ib.pocstor_35625_object16663/head//22 by client.940330.0:16664 2014-08-11 16:30:43.712293 2014-08-13 17:38:02.712218 7fe9bae887a0 20 read_log 192888'435 (0'0) modify 95ea8639/benchmark_data_storage0111-ib.pocstor_35625_object17898/head//22 by client.940330.0:17899 2014-08-11 16:30:48.332880 2014-08-13 17:38:02.712230 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 1e344e39/benchmark_data_storage0111-ib.pocstor_35625_object19146/head//22 by client.940330.0:19147 2014-08-11 16:30:53.204896 2014-08-13 17:38:02.712242 7fe9bae887a0 20 read_log 192888'437 (0'0) modify e2ccee39/benchmark_data_storage0111-ib.pocstor_35625_object19457/head//22 by client.940330.0:19458 2014-08-11 16:30:54.485156 2014-08-13 17:38:02.712256 7fe9bae887a0 20 read_log 192888'438 (0'0) modify fcc97639/benchmark_data_storage0111-ib.pocstor_35625_object21724/head//22 by client.940330.0:21725 2014-08-11 16:31:03.500175 2014-08-13 17:38:02.712268 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 78b00e39/benchmark_data_storage0111-ib.pocstor_35625_object26694/head//22 by client.940330.0:26695 2014-08-11 16:31:22.621379 2014-08-13 17:38:02.712280 7fe9bae887a0 20 read_log 192888'440 (0'0) modify 782e6639/benchmark_data_storage0111-ib.pocstor_35625_object26891/head//22 by client.940330.0:26892 2014-08-11 16:31:23.382190 2014-08-13 17:38:02.712292 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 82c62e39/benchmark_data_storage0111-ib.pocstor_35625_object28193/head//22 by client.940330.0:28194 2014-08-11 16:31:28.477998 2014-08-13 17:38:02.712304 7fe9bae887a0 20 read_log 192888'442 (0'0) modify e79c5639/benchmark_data_storage0111-ib.pocstor_35625_object29711/head//22 by client.940330.0:29712 2014-08-11 16:31:34.367474 2014-08-13 17:38:02.712316 7fe9bae887a0 20 read_log 192888'443 (192888'427) delete 55a70639/benchmark_data_storage0111-ib.pocstor_35625_object199/head//22 by client.940330.0:31507 2014-08-11 16:31:41.519756 2014-08-13 17:38:02.712328 7fe9bae887a0 20 read_log 192888'444 (192888'428) delete 1ffeb639/benchmark_data_storage0111-ib.pocstor_35625_object2658/head//22 by client.940330.0:33966 2014-08-11 16:31:42.160017 2014-08-13 17:38:02.712340 7fe9bae887a0 20 read_log 192888'445 (192888'429) delete 7c3cc639/benchmark_data_storage0111-ib.pocstor_35625_object3860/head//22 by client.940330.0:35168 2014-08-11 16:31:42.491133 2014-08-13 17:38:02.712352 7fe9bae887a0 20 read_log 192888'446 (192888'430) delete 5b7c2e39/benchmark_data_storage0111-ib.pocstor_35625_object4180/head//22 by client.940330.0:35488 2014-08-11 16:31:42.587388 2014-08-13 17:38:02.712364 7fe9bae887a0 20 read_log 192888'447 (192888'431) delete 7c2fe39/benchmark_data_storage0111-ib.pocstor_35625_object4739/head//22 by client.940330.0:36047 2014-08-11 16:31:42.750204 2014-08-13 17:38:02.712405 7fe9bae887a0 20 read_log 192888'448 (192888'432) delete 3b3a2e39/benchmark_data_storage0111-ib.pocstor_35625_object10805/head//22 by client.940330.0:42113 2014-08-11 16:31:44.343890 2014-08-13 17:38:02.712421 7fe9bae887a0 20 read_log 192888'449 (192888'433) delete 42e43e39/benchmark_data_storage0111-ib.pocstor_35625_object14551/head//22 by client.940330.0:45859 2014-08-11 16:31:45.332360 2014-08-13 17:38:02.712434 7fe9bae887a0 20 read_log 192888'450 (192888'434) delete 5a550639/benchmark_data_storage0111-ib.pocstor_35625_object16663/head//22 by client.940330.0:47971 2014-08-11 16:31:45.898185 2014-08-13 17:38:02.712446 7fe9bae887a0 20 read_log 192888'451 (192888'435) delete 95ea8639/benchmark_data_storage0111-ib.pocstor_35625_object17898/head//22 by client.940330.0:49206 2014-08-11 16:31:46.253072 2014-08-13 17:38:02.712458 7fe9bae887a0 20 read_log 192888'452 (192888'436) delete 1e344e39/benchmark_data_storage0111-ib.pocstor_35625_object19146/head//22 by client.940330.0:50454 2014-08-11 16:31:46.620527 2014-08-13 17:38:02.712470 7fe9bae887a0 20 read_log 192888'453 (192888'437) delete e2ccee39/benchmark_data_storage0111-ib.pocstor_35625_object19457/head//22 by client.940330.0:50765 2014-08-11 16:31:46.707819 2014-08-13 17:38:02.712482 7fe9bae887a0 20 read_log 192888'454 (192888'438) delete fcc97639/benchmark_data_storage0111-ib.pocstor_35625_object21724/head//22 by client.940330.0:53032 2014-08-11 16:31:47.358412 2014-08-13 17:38:02.712494 7fe9bae887a0 20 read_log 192888'455 (192888'439) delete 78b00e39/benchmark_data_storage0111-ib.pocstor_35625_object26694/head//22 by client.940330.0:58002 2014-08-11 16:31:48.843060 2014-08-13 17:38:02.712506 7fe9bae887a0 20 read_log 192888'456 (192888'440) delete 782e6639/benchmark_data_storage0111-ib.pocstor_35625_object26891/head//22 by client.940330.0:58199 2014-08-11 16:31:48.899753 2014-08-13 17:38:02.712518 7fe9bae887a0 20 read_log 192888'457 (192888'441) delete 82c62e39/benchmark_data_storage0111-ib.pocstor_35625_object28193/head//22 by client.940330.0:59501 2014-08-11 16:31:49.256998 2014-08-13 17:38:02.712530 7fe9bae887a0 20 read_log 192888'458 (192888'442) delete e79c5639/benchmark_data_storage0111-ib.pocstor_35625_object29711/head//22 by client.940330.0:61019 2014-08-11 16:31:49.676944 2014-08-13 17:38:02.712542 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 9e3f7639/benchmark_data_storage0111-ib.pocstor_64498_object1607/head//22 by client.1049635.0:1608 2014-08-12 17:17:22.313360 2014-08-13 17:38:02.712554 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 9188fe39/benchmark_data_storage0111-ib.pocstor_64498_object2060/head//22 by client.1049635.0:2061 2014-08-12 17:17:23.731623 2014-08-13 17:38:02.712566 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 9e90ce39/benchmark_data_storage0111-ib.pocstor_64498_object3326/head//22 by client.1049635.0:3327 2014-08-12 17:17:27.941857 2014-08-13 17:38:02.712578 7fe9bae887a0 20 read_log 194215'462 (0'0) modify f0463639/benchmark_data_storage0111-ib.pocstor_64498_object7865/head//22 by client.1049635.0:7866 2014-08-12 17:17:41.735718 2014-08-13 17:38:02.712589 7fe9bae887a0 20 read_log 194215'463 (0'0) modify 5a7f2e39/benchmark_data_storage0111-ib.pocstor_64498_object9034/head//22 by client.1049635.0:9035 2014-08-12 17:17:45.119487 2014-08-13 17:38:02.712602 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 9db63639/benchmark_data_storage0111-ib.pocstor_64498_object12088/head//22 by client.1049635.0:12089 2014-08-12 17:17:54.722646 2014-08-13 17:38:02.712614 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 5e5ede39/benchmark_data_storage0111-ib.pocstor_64498_object12243/head//22 by client.1049635.0:12244 2014-08-12 17:17:55.194568 2014-08-13 17:38:02.712626 7fe9bae887a0 20 read_log 194215'466 (0'0) modify 232ec639/benchmark_data_storage0111-ib.pocstor_64498_object12575/head//22 by client.1049635.0:12576 2014-08-12 17:17:56.103236 2014-08-13 17:38:02.712645 7fe9bae887a0 20 read_log 194215'467 (0'0) modify 91c27e39/benchmark_data_storage0111-ib.pocstor_64498_object13743/head//22 by client.1049635.0:13744 2014-08-12 17:17:59.772821 2014-08-13 17:38:02.712660 7fe9bae887a0 20 read_log 194215'468 (0'0) modify 69ce1e39/benchmark_data_storage0111-ib.pocstor_64498_object13794/head//22 by client.1049635.0:13795 2014-08-12 17:17:59.916989 2014-08-13 17:38:02.712672 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 13e1a639/benchmark_data_storage0111-ib.pocstor_64498_object14679/head//22 by client.1049635.0:14680 2014-08-12 17:18:02.928486 2014-08-13 17:38:02.712685 7fe9bae887a0 20 read_log 194215'470 (0'0) modify a4ea639/benchmark_data_storage0111-ib.pocstor_64498_object15315/head//22 by client.1049635.0:15316 2014-08-12 17:18:05.018218 2014-08-13 17:38:02.712716 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 75b9fe39/benchmark_data_storage0111-ib.pocstor_64498_object18236/head//22 by client.1049635.0:18237 2014-08-12 17:18:16.058331 2014-08-13 17:38:02.712729 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 8adce39/benchmark_data_storage0111-ib.pocstor_64498_object22869/head//22 by client.1049635.0:22870 2014-08-12 17:18:33.645837 2014-08-13 17:38:02.712741 7fe9bae887a0 20 read_log 194215'473 (0'0) modify a5a8c639/benchmark_data_storage0111-ib.pocstor_64498_object32892/head//22 by client.1049635.0:32893 2014-08-12 17:19:11.983903 2014-08-13 17:38:02.712772 7fe9bae887a0 20 read_log 194215'474 (0'0) modify e4bf7e39/benchmark_data_storage0111-ib.pocstor_64498_object33512/head//22 by client.1049635.0:33513 2014-08-12 17:19:14.321114 2014-08-13 17:38:02.712788 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 89c00639/benchmark_data_storage0111-ib.pocstor_64498_object34002/head//22 by client.1049635.0:34003 2014-08-12 17:19:16.190410 2014-08-13 17:38:02.712800 7fe9bae887a0 20 read_log 194215'476 (194215'459) delete 9e3f7639/benchmark_data_storage0111-ib.pocstor_64498_object1607/head//22 by client.1049635.0:35810 2014-08-12 17:19:18.520175 2014-08-13 17:38:02.712812 7fe9bae887a0 20 read_log 194215'477 (194215'460) delete 9188fe39/benchmark_data_storage0111-ib.pocstor_64498_object2060/head//22 by client.1049635.0:36263 2014-08-12 17:19:18.664074 2014-08-13 17:38:02.712824 7fe9bae887a0 20 read_log 194215'478 (194215'461) delete 9e90ce39/benchmark_data_storage0111-ib.pocstor_64498_object3326/head//22 by client.1049635.0:37529 2014-08-12 17:19:19.065813 2014-08-13 17:38:02.712837 7fe9bae887a0 20 read_log 194215'479 (194215'462) delete f0463639/benchmark_data_storage0111-ib.pocstor_64498_object7865/head//22 by client.1049635.0:42068 2014-08-12 17:19:20.561698 2014-08-13 17:38:02.712849 7fe9bae887a0 20 read_log 194215'480 (194215'463) delete 5a7f2e39/benchmark_data_storage0111-ib.pocstor_64498_object9034/head//22 by client.1049635.0:43237 2014-08-12 17:19:20.935454 2014-08-13 17:38:02.712861 7fe9bae887a0 20 read_log 194215'481 (194215'464) delete 9db63639/benchmark_data_storage0111-ib.pocstor_64498_object12088/head//22 by client.1049635.0:46291 2014-08-12 17:19:21.924279 2014-08-13 17:38:02.712873 7fe9bae887a0 20 read_log 194215'482 (194215'465) delete 5e5ede39/benchmark_data_storage0111-ib.pocstor_64498_object12243/head//22 by client.1049635.0:46446 2014-08-12 17:19:21.972416 2014-08-13 17:38:02.712885 7fe9bae887a0 20 read_log 194215'483 (194215'466) delete 232ec639/benchmark_data_storage0111-ib.pocstor_64498_object12575/head//22 by client.1049635.0:46778 2014-08-12 17:19:22.076791 2014-08-13 17:38:02.712897 7fe9bae887a0 20 read_log 194215'484 (194215'467) delete 91c27e39/benchmark_data_storage0111-ib.pocstor_64498_object13743/head//22 by client.1049635.0:47946 2014-08-12 17:19:22.488199 2014-08-13 17:38:02.712910 7fe9bae887a0 20 read_log 194215'485 (194215'468) delete 69ce1e39/benchmark_data_storage0111-ib.pocstor_64498_object13794/head//22 by client.1049635.0:47997 2014-08-12 17:19:22.501441 2014-08-13 17:38:02.712922 7fe9bae887a0 20 read_log 194215'486 (194215'469) delete 13e1a639/benchmark_data_storage0111-ib.pocstor_64498_object14679/head//22 by client.1049635.0:48882 2014-08-12 17:19:22.778021 2014-08-13 17:38:02.712942 7fe9bae887a0 20 read_log 194215'487 (194215'470) delete a4ea639/benchmark_data_storage0111-ib.pocstor_64498_object15315/head//22 by client.1049635.0:49518 2014-08-12 17:19:22.972058 2014-08-13 17:38:02.712955 7fe9bae887a0 20 read_log 194215'488 (194215'471) delete 75b9fe39/benchmark_data_storage0111-ib.pocstor_64498_object18236/head//22 by client.1049635.0:52439 2014-08-12 17:19:23.907650 2014-08-13 17:38:02.712967 7fe9bae887a0 20 read_log 194215'489 (194215'472) delete 8adce39/benchmark_data_storage0111-ib.pocstor_64498_object22869/head//22 by client.1049635.0:57072 2014-08-12 17:19:25.564703 2014-08-13 17:38:02.712979 7fe9bae887a0 20 read_log 194215'490 (194215'473) delete a5a8c639/benchmark_data_storage0111-ib.pocstor_64498_object32892/head//22 by client.1049635.0:67095 2014-08-12 17:19:29.079034 2014-08-13 17:38:02.712991 7fe9bae887a0 20 read_log 194215'491 (194215'474) delete e4bf7e39/benchmark_data_storage0111-ib.pocstor_64498_object33512/head//22 by client.1049635.0:67715 2014-08-12 17:19:29.295637 2014-08-13 17:38:02.713003 7fe9bae887a0 20 read_log 194215'492 (194215'475) delete 89c00639/benchmark_data_storage0111-ib.pocstor_64498_object34002/head//22 by client.1049635.0:68205 2014-08-12 17:19:29.482783 2014-08-13 17:38:02.713088 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.713105 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.713120 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001671 0 0.000000 2014-08-13 17:38:02.713131 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.713139 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.713148 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] log((0'0,194215'492], crt=192888'458) 2014-08-13 17:38:02.713163 7fe9bae887a0 10 osd.4 194478 pgid 22.653 coll 22.653_head 2014-08-13 17:38:02.713293 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.653 2014-08-13 17:38:02.713302 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.713328 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.653(unlocked)] enter Initial 2014-08-13 17:38:02.713339 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.653(unlocked)] enter NotTrimming 2014-08-13 17:38:02.713458 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.713569 7fe9bae887a0 20 read_log 191648'50 (0'0) modify df526e53/benchmark_data_storage0111-ib.pocstor_40745_object2992/head//22 by client.273158.0:2993 2014-07-30 14:09:01.366616 2014-08-13 17:38:02.713590 7fe9bae887a0 20 read_log 191648'99 (0'0) modify 5462a653/benchmark_data_storage0111-ib.pocstor_40745_object5509/head//22 by client.273158.0:5510 2014-07-30 14:09:13.472861 2014-08-13 17:38:02.713603 7fe9bae887a0 20 read_log 191648'141 (0'0) modify 5af0be53/benchmark_data_storage0111-ib.pocstor_40745_object7720/head//22 by client.273158.0:7721 2014-07-30 14:09:24.548679 2014-08-13 17:38:02.713615 7fe9bae887a0 20 read_log 191648'164 (0'0) modify 26873e53/benchmark_data_storage0111-ib.pocstor_40745_object9236/head//22 by client.273158.0:9237 2014-07-30 14:09:32.104715 2014-08-13 17:38:02.713627 7fe9bae887a0 20 read_log 191648'196 (0'0) modify 4c04ce53/benchmark_data_storage0111-ib.pocstor_40745_object11704/head//22 by client.273158.0:11705 2014-07-30 14:09:45.603765 2014-08-13 17:38:02.713639 7fe9bae887a0 20 read_log 191648'236 (0'0) modify 11479653/benchmark_data_storage0111-ib.pocstor_40745_object13796/head//22 by client.273158.0:13797 2014-07-30 14:09:56.026201 2014-08-13 17:38:02.713651 7fe9bae887a0 20 read_log 191648'243 (0'0) modify d3a72e53/benchmark_data_storage0111-ib.pocstor_40745_object14204/head//22 by client.273158.0:14205 2014-07-30 14:09:58.369131 2014-08-13 17:38:02.713663 7fe9bae887a0 20 read_log 191648'276 (0'0) modify 323c653/benchmark_data_storage0111-ib.pocstor_40745_object15872/head//22 by client.273158.0:15873 2014-07-30 14:10:06.843641 2014-08-13 17:38:02.713675 7fe9bae887a0 20 read_log 191648'279 (0'0) modify 5beba653/benchmark_data_storage0111-ib.pocstor_40745_object15948/head//22 by client.273158.0:15949 2014-07-30 14:10:07.260442 2014-08-13 17:38:02.713687 7fe9bae887a0 20 read_log 191648'350 (0'0) modify 3f228653/benchmark_data_storage0111-ib.pocstor_40745_object20444/head//22 by client.273158.0:20445 2014-07-30 14:10:33.185905 2014-08-13 17:38:02.713700 7fe9bae887a0 20 read_log 191648'367 (0'0) modify 13f4b653/benchmark_data_storage0111-ib.pocstor_40745_object21587/head//22 by client.273158.0:21588 2014-07-30 14:10:40.045428 2014-08-13 17:38:02.713720 7fe9bae887a0 20 read_log 191648'376 (0'0) modify 3788e653/benchmark_data_storage0111-ib.pocstor_40745_object22096/head//22 by client.273158.0:22097 2014-07-30 14:10:44.575946 2014-08-13 17:38:02.713735 7fe9bae887a0 20 read_log 191727'389 (0'0) modify 338d4e53/benchmark_data_storage0111-ib.pocstor_51285_object5636/head//22 by client.275175.0:5637 2014-07-30 14:18:52.207507 2014-08-13 17:38:02.713747 7fe9bae887a0 20 read_log 191727'390 (0'0) modify a1fe7e53/benchmark_data_storage0111-ib.pocstor_51285_object5883/head//22 by client.275175.0:5884 2014-07-30 14:18:52.913352 2014-08-13 17:38:02.713759 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 40cd6e53/benchmark_data_storage0111-ib.pocstor_51285_object10185/head//22 by client.275175.0:10186 2014-07-30 14:19:05.694861 2014-08-13 17:38:02.713771 7fe9bae887a0 20 read_log 191727'399 (0'0) modify d8d8a653/benchmark_data_storage0111-ib.pocstor_51285_object14490/head//22 by client.275175.0:14491 2014-07-30 14:19:19.263602 2014-08-13 17:38:02.713783 7fe9bae887a0 20 read_log 191727'400 (0'0) modify 77371e53/benchmark_data_storage0111-ib.pocstor_51285_object15005/head//22 by client.275175.0:15006 2014-07-30 14:19:20.716254 2014-08-13 17:38:02.713795 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 4f588653/benchmark_data_storage0111-ib.pocstor_51285_object17088/head//22 by client.275175.0:17089 2014-07-30 14:19:27.407011 2014-08-13 17:38:02.713807 7fe9bae887a0 20 read_log 191727'408 (0'0) modify 90392653/benchmark_data_storage0111-ib.pocstor_51285_object18667/head//22 by client.275175.0:18668 2014-07-30 14:19:31.953450 2014-08-13 17:38:02.713821 7fe9bae887a0 20 read_log 191727'424 (0'0) modify 21400e53/benchmark_data_storage0111-ib.pocstor_51285_object29524/head//22 by client.275175.0:29525 2014-07-30 14:20:05.494414 2014-08-13 17:38:02.713833 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 33bbde53/benchmark_data_storage0111-ib.pocstor_51285_object32140/head//22 by client.275175.0:32141 2014-07-30 14:20:13.270369 2014-08-13 17:38:02.713845 7fe9bae887a0 20 read_log 191727'431 (0'0) modify c1284653/benchmark_data_storage0111-ib.pocstor_51285_object34371/head//22 by client.275175.0:34372 2014-07-30 14:20:20.162069 2014-08-13 17:38:02.713857 7fe9bae887a0 20 read_log 191727'435 (0'0) modify 9181d653/benchmark_data_storage0111-ib.pocstor_51285_object38743/head//22 by client.275175.0:38744 2014-07-30 14:20:33.360591 2014-08-13 17:38:02.713869 7fe9bae887a0 20 read_log 192888'436 (0'0) modify 99f0ee53/benchmark_data_storage0111-ib.pocstor_35625_object3177/head//22 by client.940330.0:3178 2014-08-11 16:29:53.210028 2014-08-13 17:38:02.713882 7fe9bae887a0 20 read_log 192888'437 (0'0) modify a5049653/benchmark_data_storage0111-ib.pocstor_35625_object3827/head//22 by client.940330.0:3828 2014-08-11 16:29:55.700114 2014-08-13 17:38:02.713894 7fe9bae887a0 20 read_log 192888'438 (0'0) modify 2799d653/benchmark_data_storage0111-ib.pocstor_35625_object7714/head//22 by client.940330.0:7715 2014-08-11 16:30:09.827621 2014-08-13 17:38:02.713906 7fe9bae887a0 20 read_log 192888'439 (0'0) modify 63322e53/benchmark_data_storage0111-ib.pocstor_35625_object8106/head//22 by client.940330.0:8107 2014-08-11 16:30:11.212499 2014-08-13 17:38:02.713918 7fe9bae887a0 20 read_log 192888'440 (0'0) modify f3be0e53/benchmark_data_storage0111-ib.pocstor_35625_object11131/head//22 by client.940330.0:11132 2014-08-11 16:30:22.314295 2014-08-13 17:38:02.713930 7fe9bae887a0 20 read_log 192888'441 (0'0) modify b2889653/benchmark_data_storage0111-ib.pocstor_35625_object14766/head//22 by client.940330.0:14767 2014-08-11 16:30:36.621453 2014-08-13 17:38:02.713942 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 914f653/benchmark_data_storage0111-ib.pocstor_35625_object15593/head//22 by client.940330.0:15594 2014-08-11 16:30:39.702497 2014-08-13 17:38:02.713961 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 8bf33e53/benchmark_data_storage0111-ib.pocstor_35625_object16627/head//22 by client.940330.0:16628 2014-08-11 16:30:43.574868 2014-08-13 17:38:02.713974 7fe9bae887a0 20 read_log 192888'444 (0'0) modify 7fcff653/benchmark_data_storage0111-ib.pocstor_35625_object18979/head//22 by client.940330.0:18980 2014-08-11 16:30:52.591538 2014-08-13 17:38:02.713986 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 5db8fe53/benchmark_data_storage0111-ib.pocstor_35625_object19163/head//22 by client.940330.0:19164 2014-08-11 16:30:53.265434 2014-08-13 17:38:02.713998 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 2db68e53/benchmark_data_storage0111-ib.pocstor_35625_object30402/head//22 by client.940330.0:30403 2014-08-11 16:31:36.980670 2014-08-13 17:38:02.714013 7fe9bae887a0 20 read_log 192888'447 (0'0) modify 827f3e53/benchmark_data_storage0111-ib.pocstor_35625_object31033/head//22 by client.940330.0:31034 2014-08-11 16:31:39.350480 2014-08-13 17:38:02.714026 7fe9bae887a0 20 read_log 192888'448 (192888'436) delete 99f0ee53/benchmark_data_storage0111-ib.pocstor_35625_object3177/head//22 by client.940330.0:34485 2014-08-11 16:31:42.301018 2014-08-13 17:38:02.714040 7fe9bae887a0 20 read_log 192888'449 (192888'437) delete a5049653/benchmark_data_storage0111-ib.pocstor_35625_object3827/head//22 by client.940330.0:35135 2014-08-11 16:31:42.482825 2014-08-13 17:38:02.714053 7fe9bae887a0 20 read_log 192888'450 (192888'438) delete 2799d653/benchmark_data_storage0111-ib.pocstor_35625_object7714/head//22 by client.940330.0:39022 2014-08-11 16:31:43.518473 2014-08-13 17:38:02.714067 7fe9bae887a0 20 read_log 192888'451 (192888'439) delete 63322e53/benchmark_data_storage0111-ib.pocstor_35625_object8106/head//22 by client.940330.0:39414 2014-08-11 16:31:43.619474 2014-08-13 17:38:02.714079 7fe9bae887a0 20 read_log 192888'452 (192888'440) delete f3be0e53/benchmark_data_storage0111-ib.pocstor_35625_object11131/head//22 by client.940330.0:42439 2014-08-11 16:31:44.437385 2014-08-13 17:38:02.714093 7fe9bae887a0 20 read_log 192888'453 (192888'441) delete b2889653/benchmark_data_storage0111-ib.pocstor_35625_object14766/head//22 by client.940330.0:46074 2014-08-11 16:31:45.390471 2014-08-13 17:38:02.714106 7fe9bae887a0 20 read_log 192888'454 (192888'442) delete 914f653/benchmark_data_storage0111-ib.pocstor_35625_object15593/head//22 by client.940330.0:46901 2014-08-11 16:31:45.616019 2014-08-13 17:38:02.714120 7fe9bae887a0 20 read_log 192888'455 (192888'443) delete 8bf33e53/benchmark_data_storage0111-ib.pocstor_35625_object16627/head//22 by client.940330.0:47935 2014-08-11 16:31:45.888754 2014-08-13 17:38:02.714134 7fe9bae887a0 20 read_log 192888'456 (192888'444) delete 7fcff653/benchmark_data_storage0111-ib.pocstor_35625_object18979/head//22 by client.940330.0:50287 2014-08-11 16:31:46.576056 2014-08-13 17:38:02.714148 7fe9bae887a0 20 read_log 192888'457 (192888'445) delete 5db8fe53/benchmark_data_storage0111-ib.pocstor_35625_object19163/head//22 by client.940330.0:50471 2014-08-11 16:31:46.624824 2014-08-13 17:38:02.714172 7fe9bae887a0 20 read_log 192888'458 (192888'446) delete 2db68e53/benchmark_data_storage0111-ib.pocstor_35625_object30402/head//22 by client.940330.0:61710 2014-08-11 16:31:49.872201 2014-08-13 17:38:02.714186 7fe9bae887a0 20 read_log 192888'459 (192888'447) delete 827f3e53/benchmark_data_storage0111-ib.pocstor_35625_object31033/head//22 by client.940330.0:62341 2014-08-11 16:31:50.067541 2014-08-13 17:38:02.714198 7fe9bae887a0 20 read_log 194215'460 (0'0) modify 8c2bae53/benchmark_data_storage0111-ib.pocstor_64498_object726/head//22 by client.1049635.0:727 2014-08-12 17:17:19.463649 2014-08-13 17:38:02.714210 7fe9bae887a0 20 read_log 194215'461 (0'0) modify 825c2653/benchmark_data_storage0111-ib.pocstor_64498_object2005/head//22 by client.1049635.0:2006 2014-08-12 17:17:23.569963 2014-08-13 17:38:02.714232 7fe9bae887a0 20 read_log 194215'462 (0'0) modify 4d7d8653/benchmark_data_storage0111-ib.pocstor_64498_object2439/head//22 by client.1049635.0:2440 2014-08-12 17:17:25.064373 2014-08-13 17:38:02.714247 7fe9bae887a0 20 read_log 194215'463 (0'0) modify a254d653/benchmark_data_storage0111-ib.pocstor_64498_object2520/head//22 by client.1049635.0:2521 2014-08-12 17:17:25.308170 2014-08-13 17:38:02.714259 7fe9bae887a0 20 read_log 194215'464 (0'0) modify 380b8e53/benchmark_data_storage0111-ib.pocstor_64498_object2697/head//22 by client.1049635.0:2698 2014-08-12 17:17:25.827860 2014-08-13 17:38:02.714273 7fe9bae887a0 20 read_log 194215'465 (0'0) modify 16d6fe53/benchmark_data_storage0111-ib.pocstor_64498_object2776/head//22 by client.1049635.0:2777 2014-08-12 17:17:26.060757 2014-08-13 17:38:02.714285 7fe9bae887a0 20 read_log 194215'466 (0'0) modify fd5dbe53/benchmark_data_storage0111-ib.pocstor_64498_object3902/head//22 by client.1049635.0:3903 2014-08-12 17:17:29.781465 2014-08-13 17:38:02.714299 7fe9bae887a0 20 read_log 194215'467 (0'0) modify ca4a8653/benchmark_data_storage0111-ib.pocstor_64498_object9634/head//22 by client.1049635.0:9635 2014-08-12 17:17:46.883178 2014-08-13 17:38:02.714311 7fe9bae887a0 20 read_log 194215'468 (0'0) modify cb0fe53/benchmark_data_storage0111-ib.pocstor_64498_object10866/head//22 by client.1049635.0:10867 2014-08-12 17:17:50.713783 2014-08-13 17:38:02.714325 7fe9bae887a0 20 read_log 194215'469 (0'0) modify 10761653/benchmark_data_storage0111-ib.pocstor_64498_object18349/head//22 by client.1049635.0:18350 2014-08-12 17:18:16.479157 2014-08-13 17:38:02.714337 7fe9bae887a0 20 read_log 194215'470 (0'0) modify f928e53/benchmark_data_storage0111-ib.pocstor_64498_object22570/head//22 by client.1049635.0:22571 2014-08-12 17:18:32.532578 2014-08-13 17:38:02.714351 7fe9bae887a0 20 read_log 194215'471 (0'0) modify 6ccc6653/benchmark_data_storage0111-ib.pocstor_64498_object22728/head//22 by client.1049635.0:22729 2014-08-12 17:18:33.120476 2014-08-13 17:38:02.714363 7fe9bae887a0 20 read_log 194215'472 (0'0) modify 5153be53/benchmark_data_storage0111-ib.pocstor_64498_object24699/head//22 by client.1049635.0:24700 2014-08-12 17:18:40.458748 2014-08-13 17:38:02.714377 7fe9bae887a0 20 read_log 194215'473 (0'0) modify 1c3a1653/benchmark_data_storage0111-ib.pocstor_64498_object25680/head//22 by client.1049635.0:25681 2014-08-12 17:18:44.235916 2014-08-13 17:38:02.714389 7fe9bae887a0 20 read_log 194215'474 (0'0) modify b531e53/benchmark_data_storage0111-ib.pocstor_64498_object27038/head//22 by client.1049635.0:27039 2014-08-12 17:18:49.289440 2014-08-13 17:38:02.714403 7fe9bae887a0 20 read_log 194215'475 (0'0) modify 74122653/benchmark_data_storage0111-ib.pocstor_64498_object27078/head//22 by client.1049635.0:27079 2014-08-12 17:18:49.439743 2014-08-13 17:38:02.714415 7fe9bae887a0 20 read_log 194215'476 (0'0) modify 1a1dc653/benchmark_data_storage0111-ib.pocstor_64498_object27081/head//22 by client.1049635.0:27082 2014-08-12 17:18:49.450738 2014-08-13 17:38:02.714432 7fe9bae887a0 20 read_log 194215'477 (194215'460) delete 8c2bae53/benchmark_data_storage0111-ib.pocstor_64498_object726/head//22 by client.1049635.0:34929 2014-08-12 17:19:18.227827 2014-08-13 17:38:02.714444 7fe9bae887a0 20 read_log 194215'478 (194215'461) delete 825c2653/benchmark_data_storage0111-ib.pocstor_64498_object2005/head//22 by client.1049635.0:36208 2014-08-12 17:19:18.648082 2014-08-13 17:38:02.714457 7fe9bae887a0 20 read_log 194215'479 (194215'462) delete 4d7d8653/benchmark_data_storage0111-ib.pocstor_64498_object2439/head//22 by client.1049635.0:36642 2014-08-12 17:19:18.780168 2014-08-13 17:38:02.714469 7fe9bae887a0 20 read_log 194215'480 (194215'463) delete a254d653/benchmark_data_storage0111-ib.pocstor_64498_object2520/head//22 by client.1049635.0:36723 2014-08-12 17:19:18.807512 2014-08-13 17:38:02.714483 7fe9bae887a0 20 read_log 194215'481 (194215'464) delete 380b8e53/benchmark_data_storage0111-ib.pocstor_64498_object2697/head//22 by client.1049635.0:36900 2014-08-12 17:19:18.863824 2014-08-13 17:38:02.714497 7fe9bae887a0 20 read_log 194215'482 (194215'465) delete 16d6fe53/benchmark_data_storage0111-ib.pocstor_64498_object2776/head//22 by client.1049635.0:36979 2014-08-12 17:19:18.892572 2014-08-13 17:38:02.714511 7fe9bae887a0 20 read_log 194215'483 (194215'466) delete fd5dbe53/benchmark_data_storage0111-ib.pocstor_64498_object3902/head//22 by client.1049635.0:38105 2014-08-12 17:19:19.246990 2014-08-13 17:38:02.714541 7fe9bae887a0 20 read_log 194215'484 (194215'467) delete ca4a8653/benchmark_data_storage0111-ib.pocstor_64498_object9634/head//22 by client.1049635.0:43837 2014-08-12 17:19:21.132284 2014-08-13 17:38:02.714557 7fe9bae887a0 20 read_log 194215'485 (194215'468) delete cb0fe53/benchmark_data_storage0111-ib.pocstor_64498_object10866/head//22 by client.1049635.0:45069 2014-08-12 17:19:21.537160 2014-08-13 17:38:02.714569 7fe9bae887a0 20 read_log 194215'486 (194215'469) delete 10761653/benchmark_data_storage0111-ib.pocstor_64498_object18349/head//22 by client.1049635.0:52552 2014-08-12 17:19:23.947075 2014-08-13 17:38:02.714583 7fe9bae887a0 20 read_log 194215'487 (194215'470) delete f928e53/benchmark_data_storage0111-ib.pocstor_64498_object22570/head//22 by client.1049635.0:56773 2014-08-12 17:19:25.447544 2014-08-13 17:38:02.714596 7fe9bae887a0 20 read_log 194215'488 (194215'471) delete 6ccc6653/benchmark_data_storage0111-ib.pocstor_64498_object22728/head//22 by client.1049635.0:56931 2014-08-12 17:19:25.508161 2014-08-13 17:38:02.714610 7fe9bae887a0 20 read_log 194215'489 (194215'472) delete 5153be53/benchmark_data_storage0111-ib.pocstor_64498_object24699/head//22 by client.1049635.0:58902 2014-08-12 17:19:26.213327 2014-08-13 17:38:02.714622 7fe9bae887a0 20 read_log 194215'490 (194215'473) delete 1c3a1653/benchmark_data_storage0111-ib.pocstor_64498_object25680/head//22 by client.1049635.0:59883 2014-08-12 17:19:26.548351 2014-08-13 17:38:02.714634 7fe9bae887a0 20 read_log 194215'491 (194215'474) delete b531e53/benchmark_data_storage0111-ib.pocstor_64498_object27038/head//22 by client.1049635.0:61241 2014-08-12 17:19:27.013718 2014-08-13 17:38:02.714648 7fe9bae887a0 20 read_log 194215'492 (194215'475) delete 74122653/benchmark_data_storage0111-ib.pocstor_64498_object27078/head//22 by client.1049635.0:61281 2014-08-12 17:19:27.025291 2014-08-13 17:38:02.714661 7fe9bae887a0 20 read_log 194215'493 (194215'476) delete 1a1dc653/benchmark_data_storage0111-ib.pocstor_64498_object27081/head//22 by client.1049635.0:61284 2014-08-12 17:19:27.027157 2014-08-13 17:38:02.714735 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.714753 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=0 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.714767 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=0 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001439 0 0.000000 2014-08-13 17:38:02.714778 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=0 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.714786 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=0 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194478 2014-08-13 17:38:02.714795 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] log((0'0,194215'493], crt=192888'459) 2014-08-13 17:38:02.714804 7fe9bae887a0 10 osd.4 194478 pgid 22.69f coll 22.69f_head 2014-08-13 17:38:02.714910 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.69f 2014-08-13 17:38:02.714919 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.714945 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.69f(unlocked)] enter Initial 2014-08-13 17:38:02.714956 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.69f(unlocked)] enter NotTrimming 2014-08-13 17:38:02.715072 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.715172 7fe9bae887a0 20 read_log 191648'26 (0'0) modify 59d5ce9f/benchmark_data_storage0111-ib.pocstor_40745_object1607/head//22 by client.273158.0:1608 2014-07-30 14:08:54.688978 2014-08-13 17:38:02.715192 7fe9bae887a0 20 read_log 191648'30 (0'0) modify 6a669f/benchmark_data_storage0111-ib.pocstor_40745_object1755/head//22 by client.273158.0:1756 2014-07-30 14:08:55.225036 2014-08-13 17:38:02.715205 7fe9bae887a0 20 read_log 191648'69 (0'0) modify fcd08e9f/benchmark_data_storage0111-ib.pocstor_40745_object4264/head//22 by client.273158.0:4265 2014-07-30 14:09:07.850442 2014-08-13 17:38:02.715220 7fe9bae887a0 20 read_log 191648'103 (0'0) modify ca2b0e9f/benchmark_data_storage0111-ib.pocstor_40745_object6471/head//22 by client.273158.0:6472 2014-07-30 14:09:18.567582 2014-08-13 17:38:02.715232 7fe9bae887a0 20 read_log 191648'184 (0'0) modify 2478969f/benchmark_data_storage0111-ib.pocstor_40745_object11905/head//22 by client.273158.0:11906 2014-07-30 14:09:46.482004 2014-08-13 17:38:02.715247 7fe9bae887a0 20 read_log 191648'192 (0'0) modify ff54369f/benchmark_data_storage0111-ib.pocstor_40745_object12517/head//22 by client.273158.0:12518 2014-07-30 14:09:50.070797 2014-08-13 17:38:02.715260 7fe9bae887a0 20 read_log 191648'223 (0'0) modify f0b1ee9f/benchmark_data_storage0111-ib.pocstor_40745_object14293/head//22 by client.273158.0:14294 2014-07-30 14:09:59.184372 2014-08-13 17:38:02.715274 7fe9bae887a0 20 read_log 191648'229 (0'0) modify 5b09de9f/benchmark_data_storage0111-ib.pocstor_40745_object14690/head//22 by client.273158.0:14691 2014-07-30 14:10:00.950754 2014-08-13 17:38:02.715298 7fe9bae887a0 20 read_log 191727'356 (0'0) modify 3c64a69f/benchmark_data_storage0111-ib.pocstor_51285_object2992/head//22 by client.275175.0:2993 2014-07-30 14:18:44.196632 2014-08-13 17:38:02.715313 7fe9bae887a0 20 read_log 191727'358 (0'0) modify 444a2e9f/benchmark_data_storage0111-ib.pocstor_51285_object3482/head//22 by client.275175.0:3483 2014-07-30 14:18:45.748545 2014-08-13 17:38:02.715325 7fe9bae887a0 20 read_log 191727'361 (0'0) modify 8282169f/benchmark_data_storage0111-ib.pocstor_51285_object5121/head//22 by client.275175.0:5122 2014-07-30 14:18:50.707225 2014-08-13 17:38:02.715340 7fe9bae887a0 20 read_log 191727'365 (0'0) modify a398f69f/benchmark_data_storage0111-ib.pocstor_51285_object6081/head//22 by client.275175.0:6082 2014-07-30 14:18:53.501523 2014-08-13 17:38:02.715352 7fe9bae887a0 20 read_log 191727'371 (0'0) modify a5b4e69f/benchmark_data_storage0111-ib.pocstor_51285_object7846/head//22 by client.275175.0:7847 2014-07-30 14:18:59.096908 2014-08-13 17:38:02.715366 7fe9bae887a0 20 read_log 191727'372 (0'0) modify 57937e9f/benchmark_data_storage0111-ib.pocstor_51285_object8504/head//22 by client.275175.0:8505 2014-07-30 14:19:00.938907 2014-08-13 17:38:02.715378 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 1db82e9f/benchmark_data_storage0111-ib.pocstor_51285_object9022/head//22 by client.275175.0:9023 2014-07-30 14:19:02.413649 2014-08-13 17:38:02.715390 7fe9bae887a0 20 read_log 191727'380 (0'0) modify ee27569f/benchmark_data_storage0111-ib.pocstor_51285_object11244/head//22 by client.275175.0:11245 2014-07-30 14:19:08.782711 2014-08-13 17:38:02.715402 7fe9bae887a0 20 read_log 191727'382 (0'0) modify 33bf469f/benchmark_data_storage0111-ib.pocstor_51285_object12465/head//22 by client.275175.0:12466 2014-07-30 14:19:12.942283 2014-08-13 17:38:02.715414 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 582869f/benchmark_data_storage0111-ib.pocstor_51285_object13347/head//22 by client.275175.0:13348 2014-07-30 14:19:15.727298 2014-08-13 17:38:02.715426 7fe9bae887a0 20 read_log 191727'385 (0'0) modify b964f69f/benchmark_data_storage0111-ib.pocstor_51285_object13791/head//22 by client.275175.0:13792 2014-07-30 14:19:17.064618 2014-08-13 17:38:02.715438 7fe9bae887a0 20 read_log 191727'392 (0'0) modify 6f08569f/benchmark_data_storage0111-ib.pocstor_51285_object16375/head//22 by client.275175.0:16376 2014-07-30 14:19:25.291650 2014-08-13 17:38:02.715450 7fe9bae887a0 20 read_log 191727'393 (0'0) modify 5a62669f/benchmark_data_storage0111-ib.pocstor_51285_object16670/head//22 by client.275175.0:16671 2014-07-30 14:19:26.229709 2014-08-13 17:38:02.715462 7fe9bae887a0 20 read_log 191727'396 (0'0) modify a8a5469f/benchmark_data_storage0111-ib.pocstor_51285_object17641/head//22 by client.275175.0:17642 2014-07-30 14:19:28.942289 2014-08-13 17:38:02.715474 7fe9bae887a0 20 read_log 191727'404 (0'0) modify 3300ee9f/benchmark_data_storage0111-ib.pocstor_51285_object20797/head//22 by client.275175.0:20798 2014-07-30 14:19:38.558042 2014-08-13 17:38:02.715486 7fe9bae887a0 20 read_log 191727'405 (0'0) modify 2ba9069f/benchmark_data_storage0111-ib.pocstor_51285_object20912/head//22 by client.275175.0:20913 2014-07-30 14:19:38.881399 2014-08-13 17:38:02.715498 7fe9bae887a0 20 read_log 191727'407 (0'0) modify 4146d69f/benchmark_data_storage0111-ib.pocstor_51285_object22447/head//22 by client.275175.0:22448 2014-07-30 14:19:44.047292 2014-08-13 17:38:02.715510 7fe9bae887a0 20 read_log 191727'409 (0'0) modify 3e13069f/benchmark_data_storage0111-ib.pocstor_51285_object25432/head//22 by client.275175.0:25433 2014-07-30 14:19:53.073705 2014-08-13 17:38:02.715522 7fe9bae887a0 20 read_log 191727'410 (0'0) modify 6481469f/benchmark_data_storage0111-ib.pocstor_51285_object26802/head//22 by client.275175.0:26803 2014-07-30 14:19:57.228522 2014-08-13 17:38:02.715534 7fe9bae887a0 20 read_log 191727'411 (0'0) modify df8ca69f/benchmark_data_storage0111-ib.pocstor_51285_object26958/head//22 by client.275175.0:26959 2014-07-30 14:19:57.694261 2014-08-13 17:38:02.715557 7fe9bae887a0 20 read_log 191727'412 (0'0) modify 13261e9f/benchmark_data_storage0111-ib.pocstor_51285_object27006/head//22 by client.275175.0:27007 2014-07-30 14:19:57.829539 2014-08-13 17:38:02.715572 7fe9bae887a0 20 read_log 191727'414 (0'0) modify ae27869f/benchmark_data_storage0111-ib.pocstor_51285_object27935/head//22 by client.275175.0:27936 2014-07-30 14:20:00.843512 2014-08-13 17:38:02.715604 7fe9bae887a0 20 read_log 191727'415 (0'0) modify 1c06669f/benchmark_data_storage0111-ib.pocstor_51285_object27987/head//22 by client.275175.0:27988 2014-07-30 14:20:00.995534 2014-08-13 17:38:02.715617 7fe9bae887a0 20 read_log 191727'418 (0'0) modify 935cf69f/benchmark_data_storage0111-ib.pocstor_51285_object30756/head//22 by client.275175.0:30757 2014-07-30 14:20:09.292499 2014-08-13 17:38:02.715628 7fe9bae887a0 20 read_log 191727'419 (0'0) modify 6d5efe9f/benchmark_data_storage0111-ib.pocstor_51285_object30784/head//22 by client.275175.0:30785 2014-07-30 14:20:09.370229 2014-08-13 17:38:02.715641 7fe9bae887a0 20 read_log 191727'433 (0'0) modify 25faae9f/benchmark_data_storage0111-ib.pocstor_51285_object36563/head//22 by client.275175.0:36564 2014-07-30 14:20:27.053999 2014-08-13 17:38:02.715653 7fe9bae887a0 20 read_log 191727'434 (0'0) modify b6c0ae9f/benchmark_data_storage0111-ib.pocstor_51285_object36680/head//22 by client.275175.0:36681 2014-07-30 14:20:27.483371 2014-08-13 17:38:02.715664 7fe9bae887a0 20 read_log 191727'437 (0'0) modify 2b32d69f/benchmark_data_storage0111-ib.pocstor_51285_object37213/head//22 by client.275175.0:37214 2014-07-30 14:20:28.966627 2014-08-13 17:38:02.715676 7fe9bae887a0 20 read_log 191727'439 (0'0) modify 26a4e69f/benchmark_data_storage0111-ib.pocstor_51285_object38269/head//22 by client.275175.0:38270 2014-07-30 14:20:32.014094 2014-08-13 17:38:02.715688 7fe9bae887a0 20 read_log 192888'441 (0'0) modify 54659e9f/benchmark_data_storage0111-ib.pocstor_35625_object11/head//22 by client.940330.0:12 2014-08-11 16:29:40.425568 2014-08-13 17:38:02.715700 7fe9bae887a0 20 read_log 192888'442 (0'0) modify 7837a69f/benchmark_data_storage0111-ib.pocstor_35625_object213/head//22 by client.940330.0:214 2014-08-11 16:29:41.508684 2014-08-13 17:38:02.715712 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 30a4569f/benchmark_data_storage0111-ib.pocstor_35625_object1188/head//22 by client.940330.0:1189 2014-08-11 16:29:45.491826 2014-08-13 17:38:02.715724 7fe9bae887a0 20 read_log 192888'444 (0'0) modify a97ece9f/benchmark_data_storage0111-ib.pocstor_35625_object1686/head//22 by client.940330.0:1687 2014-08-11 16:29:47.399999 2014-08-13 17:38:02.715736 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 62db4e9f/benchmark_data_storage0111-ib.pocstor_35625_object3016/head//22 by client.940330.0:3017 2014-08-11 16:29:52.631795 2014-08-13 17:38:02.715748 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 1b4e369f/benchmark_data_storage0111-ib.pocstor_35625_object3864/head//22 by client.940330.0:3865 2014-08-11 16:29:55.833961 2014-08-13 17:38:02.715760 7fe9bae887a0 20 read_log 192888'447 (0'0) modify 2abf869f/benchmark_data_storage0111-ib.pocstor_35625_object6049/head//22 by client.940330.0:6050 2014-08-11 16:30:03.757963 2014-08-13 17:38:02.715772 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 4871369f/benchmark_data_storage0111-ib.pocstor_35625_object7757/head//22 by client.940330.0:7758 2014-08-11 16:30:09.981196 2014-08-13 17:38:02.715784 7fe9bae887a0 20 read_log 192888'449 (0'0) modify fbc2769f/benchmark_data_storage0111-ib.pocstor_35625_object9075/head//22 by client.940330.0:9076 2014-08-11 16:30:14.595048 2014-08-13 17:38:02.715796 7fe9bae887a0 20 read_log 192888'450 (0'0) modify 8aae2e9f/benchmark_data_storage0111-ib.pocstor_35625_object11428/head//22 by client.940330.0:11429 2014-08-11 16:30:23.516874 2014-08-13 17:38:02.715834 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 9e78e9f/benchmark_data_storage0111-ib.pocstor_35625_object11505/head//22 by client.940330.0:11506 2014-08-11 16:30:23.812116 2014-08-13 17:38:02.715851 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 1d90b69f/benchmark_data_storage0111-ib.pocstor_35625_object13707/head//22 by client.940330.0:13708 2014-08-11 16:30:32.541308 2014-08-13 17:38:02.715863 7fe9bae887a0 20 read_log 192888'453 (0'0) modify ddca869f/benchmark_data_storage0111-ib.pocstor_35625_object14104/head//22 by client.940330.0:14105 2014-08-11 16:30:34.128146 2014-08-13 17:38:02.715893 7fe9bae887a0 20 read_log 192888'454 (0'0) modify b8d7ae9f/benchmark_data_storage0111-ib.pocstor_35625_object16641/head//22 by client.940330.0:16642 2014-08-11 16:30:43.628439 2014-08-13 17:38:02.715905 7fe9bae887a0 20 read_log 192888'455 (0'0) modify 68de7e9f/benchmark_data_storage0111-ib.pocstor_35625_object18613/head//22 by client.940330.0:18614 2014-08-11 16:30:51.182550 2014-08-13 17:38:02.715917 7fe9bae887a0 20 read_log 192888'456 (0'0) modify acbe2e9f/benchmark_data_storage0111-ib.pocstor_35625_object18837/head//22 by client.940330.0:18838 2014-08-11 16:30:52.071132 2014-08-13 17:38:02.715929 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 850c469f/benchmark_data_storage0111-ib.pocstor_35625_object19850/head//22 by client.940330.0:19851 2014-08-11 16:30:56.257301 2014-08-13 17:38:02.715941 7fe9bae887a0 20 read_log 192888'458 (0'0) modify c68e569f/benchmark_data_storage0111-ib.pocstor_35625_object23380/head//22 by client.940330.0:23381 2014-08-11 16:31:09.803097 2014-08-13 17:38:02.715953 7fe9bae887a0 20 read_log 192888'459 (0'0) modify 88bc469f/benchmark_data_storage0111-ib.pocstor_35625_object24089/head//22 by client.940330.0:24090 2014-08-11 16:31:12.472541 2014-08-13 17:38:02.715965 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 61d0c69f/benchmark_data_storage0111-ib.pocstor_35625_object26963/head//22 by client.940330.0:26964 2014-08-11 16:31:23.660290 2014-08-13 17:38:02.715977 7fe9bae887a0 20 read_log 192888'461 (0'0) modify 5eb66e9f/benchmark_data_storage0111-ib.pocstor_35625_object27334/head//22 by client.940330.0:27335 2014-08-11 16:31:25.095400 2014-08-13 17:38:02.715988 7fe9bae887a0 20 read_log 192888'462 (0'0) modify d405e69f/benchmark_data_storage0111-ib.pocstor_35625_object27468/head//22 by client.940330.0:27469 2014-08-11 16:31:25.613767 2014-08-13 17:38:02.716000 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 1ee2a69f/benchmark_data_storage0111-ib.pocstor_35625_object29830/head//22 by client.940330.0:29831 2014-08-11 16:31:34.816061 2014-08-13 17:38:02.716012 7fe9bae887a0 20 read_log 192888'464 (0'0) modify b59869f/benchmark_data_storage0111-ib.pocstor_35625_object31218/head//22 by client.940330.0:31219 2014-08-11 16:31:40.086476 2014-08-13 17:38:02.716024 7fe9bae887a0 20 read_log 192888'465 (192888'441) delete 54659e9f/benchmark_data_storage0111-ib.pocstor_35625_object11/head//22 by client.940330.0:31319 2014-08-11 16:31:41.447201 2014-08-13 17:38:02.716036 7fe9bae887a0 20 read_log 192888'466 (192888'442) delete 7837a69f/benchmark_data_storage0111-ib.pocstor_35625_object213/head//22 by client.940330.0:31521 2014-08-11 16:31:41.527264 2014-08-13 17:38:02.716048 7fe9bae887a0 20 read_log 192888'467 (192888'443) delete 30a4569f/benchmark_data_storage0111-ib.pocstor_35625_object1188/head//22 by client.940330.0:32496 2014-08-11 16:31:41.794200 2014-08-13 17:38:02.716060 7fe9bae887a0 20 read_log 192888'468 (192888'444) delete a97ece9f/benchmark_data_storage0111-ib.pocstor_35625_object1686/head//22 by client.940330.0:32994 2014-08-11 16:31:41.912087 2014-08-13 17:38:02.716075 7fe9bae887a0 20 read_log 192888'469 (192888'445) delete 62db4e9f/benchmark_data_storage0111-ib.pocstor_35625_object3016/head//22 by client.940330.0:34324 2014-08-11 16:31:42.254306 2014-08-13 17:38:02.716088 7fe9bae887a0 20 read_log 192888'470 (192888'446) delete 1b4e369f/benchmark_data_storage0111-ib.pocstor_35625_object3864/head//22 by client.940330.0:35172 2014-08-11 16:31:42.492670 2014-08-13 17:38:02.716111 7fe9bae887a0 20 read_log 192888'471 (192888'447) delete 2abf869f/benchmark_data_storage0111-ib.pocstor_35625_object6049/head//22 by client.940330.0:37357 2014-08-11 16:31:43.089554 2014-08-13 17:38:02.716126 7fe9bae887a0 20 read_log 192888'472 (192888'448) delete 4871369f/benchmark_data_storage0111-ib.pocstor_35625_object7757/head//22 by client.940330.0:39065 2014-08-11 16:31:43.529035 2014-08-13 17:38:02.716138 7fe9bae887a0 20 read_log 192888'473 (192888'449) delete fbc2769f/benchmark_data_storage0111-ib.pocstor_35625_object9075/head//22 by client.940330.0:40383 2014-08-11 16:31:43.859908 2014-08-13 17:38:02.716152 7fe9bae887a0 20 read_log 192888'474 (192888'450) delete 8aae2e9f/benchmark_data_storage0111-ib.pocstor_35625_object11428/head//22 by client.940330.0:42736 2014-08-11 16:31:44.521816 2014-08-13 17:38:02.716175 7fe9bae887a0 20 read_log 192888'475 (192888'451) delete 9e78e9f/benchmark_data_storage0111-ib.pocstor_35625_object11505/head//22 by client.940330.0:42813 2014-08-11 16:31:44.541226 2014-08-13 17:38:02.716188 7fe9bae887a0 20 read_log 192888'476 (192888'452) delete 1d90b69f/benchmark_data_storage0111-ib.pocstor_35625_object13707/head//22 by client.940330.0:45015 2014-08-11 16:31:45.106344 2014-08-13 17:38:02.716202 7fe9bae887a0 20 read_log 192888'477 (192888'453) delete ddca869f/benchmark_data_storage0111-ib.pocstor_35625_object14104/head//22 by client.940330.0:45412 2014-08-11 16:31:45.209866 2014-08-13 17:38:02.716215 7fe9bae887a0 20 read_log 192888'478 (192888'454) delete b8d7ae9f/benchmark_data_storage0111-ib.pocstor_35625_object16641/head//22 by client.940330.0:47949 2014-08-11 16:31:45.892229 2014-08-13 17:38:02.716227 7fe9bae887a0 20 read_log 192888'479 (192888'455) delete 68de7e9f/benchmark_data_storage0111-ib.pocstor_35625_object18613/head//22 by client.940330.0:49921 2014-08-11 16:31:46.476621 2014-08-13 17:38:02.716239 7fe9bae887a0 20 read_log 192888'480 (192888'456) delete acbe2e9f/benchmark_data_storage0111-ib.pocstor_35625_object18837/head//22 by client.940330.0:50145 2014-08-11 16:31:46.537627 2014-08-13 17:38:02.716251 7fe9bae887a0 20 read_log 192888'481 (192888'457) delete 850c469f/benchmark_data_storage0111-ib.pocstor_35625_object19850/head//22 by client.940330.0:51158 2014-08-11 16:31:46.825575 2014-08-13 17:38:02.716265 7fe9bae887a0 20 read_log 192888'482 (192888'458) delete c68e569f/benchmark_data_storage0111-ib.pocstor_35625_object23380/head//22 by client.940330.0:54688 2014-08-11 16:31:47.866428 2014-08-13 17:38:02.716277 7fe9bae887a0 20 read_log 192888'483 (192888'459) delete 88bc469f/benchmark_data_storage0111-ib.pocstor_35625_object24089/head//22 by client.940330.0:55397 2014-08-11 16:31:48.066324 2014-08-13 17:38:02.716309 7fe9bae887a0 20 read_log 192888'484 (192888'460) delete 61d0c69f/benchmark_data_storage0111-ib.pocstor_35625_object26963/head//22 by client.940330.0:58271 2014-08-11 16:31:48.916389 2014-08-13 17:38:02.716324 7fe9bae887a0 20 read_log 192888'485 (192888'461) delete 5eb66e9f/benchmark_data_storage0111-ib.pocstor_35625_object27334/head//22 by client.940330.0:58642 2014-08-11 16:31:49.018357 2014-08-13 17:38:02.716337 7fe9bae887a0 20 read_log 192888'486 (192888'462) delete d405e69f/benchmark_data_storage0111-ib.pocstor_35625_object27468/head//22 by client.940330.0:58776 2014-08-11 16:31:49.057124 2014-08-13 17:38:02.716351 7fe9bae887a0 20 read_log 192888'487 (192888'463) delete 1ee2a69f/benchmark_data_storage0111-ib.pocstor_35625_object29830/head//22 by client.940330.0:61138 2014-08-11 16:31:49.709055 2014-08-13 17:38:02.716364 7fe9bae887a0 20 read_log 192888'488 (192888'464) delete b59869f/benchmark_data_storage0111-ib.pocstor_35625_object31218/head//22 by client.940330.0:62526 2014-08-11 16:31:50.126683 2014-08-13 17:38:02.716378 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 3ecec69f/benchmark_data_storage0111-ib.pocstor_64498_object1404/head//22 by client.1049635.0:1405 2014-08-12 17:17:21.708199 2014-08-13 17:38:02.716397 7fe9bae887a0 20 read_log 194215'490 (0'0) modify 9ad4069f/benchmark_data_storage0111-ib.pocstor_64498_object3587/head//22 by client.1049635.0:3588 2014-08-12 17:17:28.766281 2014-08-13 17:38:02.716412 7fe9bae887a0 20 read_log 194215'491 (0'0) modify 24506e9f/benchmark_data_storage0111-ib.pocstor_64498_object4377/head//22 by client.1049635.0:4378 2014-08-12 17:17:31.161057 2014-08-13 17:38:02.716425 7fe9bae887a0 20 read_log 194215'492 (0'0) modify d1a7769f/benchmark_data_storage0111-ib.pocstor_64498_object5307/head//22 by client.1049635.0:5308 2014-08-12 17:17:33.857025 2014-08-13 17:38:02.716439 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 6fb34e9f/benchmark_data_storage0111-ib.pocstor_64498_object6549/head//22 by client.1049635.0:6550 2014-08-12 17:17:37.719641 2014-08-13 17:38:02.716451 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 22619e9f/benchmark_data_storage0111-ib.pocstor_64498_object9935/head//22 by client.1049635.0:9936 2014-08-12 17:17:47.739205 2014-08-13 17:38:02.716465 7fe9bae887a0 20 read_log 194215'495 (0'0) modify faf5a69f/benchmark_data_storage0111-ib.pocstor_64498_object9949/head//22 by client.1049635.0:9950 2014-08-12 17:17:47.778378 2014-08-13 17:38:02.716477 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 54de3e9f/benchmark_data_storage0111-ib.pocstor_64498_object10247/head//22 by client.1049635.0:10248 2014-08-12 17:17:48.628216 2014-08-13 17:38:02.716492 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 3eee969f/benchmark_data_storage0111-ib.pocstor_64498_object12153/head//22 by client.1049635.0:12154 2014-08-12 17:17:54.945107 2014-08-13 17:38:02.716504 7fe9bae887a0 20 read_log 194215'498 (0'0) modify 54d3069f/benchmark_data_storage0111-ib.pocstor_64498_object13146/head//22 by client.1049635.0:13147 2014-08-12 17:17:57.809358 2014-08-13 17:38:02.716516 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 67ec69f/benchmark_data_storage0111-ib.pocstor_64498_object13220/head//22 by client.1049635.0:13221 2014-08-12 17:17:58.059747 2014-08-13 17:38:02.716528 7fe9bae887a0 20 read_log 194215'500 (0'0) modify f8fbbe9f/benchmark_data_storage0111-ib.pocstor_64498_object14873/head//22 by client.1049635.0:14874 2014-08-12 17:18:03.556650 2014-08-13 17:38:02.716543 7fe9bae887a0 20 read_log 194215'501 (0'0) modify bd477e9f/benchmark_data_storage0111-ib.pocstor_64498_object15889/head//22 by client.1049635.0:15890 2014-08-12 17:18:07.139499 2014-08-13 17:38:02.716555 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 669ce69f/benchmark_data_storage0111-ib.pocstor_64498_object16308/head//22 by client.1049635.0:16309 2014-08-12 17:18:08.727523 2014-08-13 17:38:02.716569 7fe9bae887a0 20 read_log 194215'503 (0'0) modify 1de5e9f/benchmark_data_storage0111-ib.pocstor_64498_object20675/head//22 by client.1049635.0:20676 2014-08-12 17:18:25.320137 2014-08-13 17:38:02.716581 7fe9bae887a0 20 read_log 194215'504 (0'0) modify 23b8069f/benchmark_data_storage0111-ib.pocstor_64498_object23355/head//22 by client.1049635.0:23356 2014-08-12 17:18:35.451912 2014-08-13 17:38:02.716596 7fe9bae887a0 20 read_log 194215'505 (0'0) modify 4df4be9f/benchmark_data_storage0111-ib.pocstor_64498_object24264/head//22 by client.1049635.0:24265 2014-08-12 17:18:38.843329 2014-08-13 17:38:02.716608 7fe9bae887a0 20 read_log 194215'506 (0'0) modify c07e169f/benchmark_data_storage0111-ib.pocstor_64498_object24680/head//22 by client.1049635.0:24681 2014-08-12 17:18:40.386998 2014-08-13 17:38:02.716622 7fe9bae887a0 20 read_log 194215'507 (0'0) modify a896469f/benchmark_data_storage0111-ib.pocstor_64498_object24850/head//22 by client.1049635.0:24851 2014-08-12 17:18:41.096674 2014-08-13 17:38:02.716636 7fe9bae887a0 20 read_log 194215'508 (0'0) modify 1febae9f/benchmark_data_storage0111-ib.pocstor_64498_object27173/head//22 by client.1049635.0:27174 2014-08-12 17:18:49.792518 2014-08-13 17:38:02.716650 7fe9bae887a0 20 read_log 194215'509 (194215'489) delete 3ecec69f/benchmark_data_storage0111-ib.pocstor_64498_object1404/head//22 by client.1049635.0:35607 2014-08-12 17:19:18.453300 2014-08-13 17:38:02.716669 7fe9bae887a0 20 read_log 194215'510 (194215'490) delete 9ad4069f/benchmark_data_storage0111-ib.pocstor_64498_object3587/head//22 by client.1049635.0:37790 2014-08-12 17:19:19.144988 2014-08-13 17:38:02.716684 7fe9bae887a0 20 read_log 194215'511 (194215'491) delete 24506e9f/benchmark_data_storage0111-ib.pocstor_64498_object4377/head//22 by client.1049635.0:38580 2014-08-12 17:19:19.395858 2014-08-13 17:38:02.716696 7fe9bae887a0 20 read_log 194215'512 (194215'492) delete d1a7769f/benchmark_data_storage0111-ib.pocstor_64498_object5307/head//22 by client.1049635.0:39510 2014-08-12 17:19:19.711809 2014-08-13 17:38:02.716710 7fe9bae887a0 20 read_log 194215'513 (194215'493) delete 6fb34e9f/benchmark_data_storage0111-ib.pocstor_64498_object6549/head//22 by client.1049635.0:40752 2014-08-12 17:19:20.132582 2014-08-13 17:38:02.716723 7fe9bae887a0 20 read_log 194215'514 (194215'494) delete 22619e9f/benchmark_data_storage0111-ib.pocstor_64498_object9935/head//22 by client.1049635.0:44138 2014-08-12 17:19:21.239146 2014-08-13 17:38:02.716737 7fe9bae887a0 20 read_log 194215'515 (194215'495) delete faf5a69f/benchmark_data_storage0111-ib.pocstor_64498_object9949/head//22 by client.1049635.0:44152 2014-08-12 17:19:21.244711 2014-08-13 17:38:02.716749 7fe9bae887a0 20 read_log 194215'516 (194215'496) delete 54de3e9f/benchmark_data_storage0111-ib.pocstor_64498_object10247/head//22 by client.1049635.0:44450 2014-08-12 17:19:21.341615 2014-08-13 17:38:02.716763 7fe9bae887a0 20 read_log 194215'517 (194215'497) delete 3eee969f/benchmark_data_storage0111-ib.pocstor_64498_object12153/head//22 by client.1049635.0:46356 2014-08-12 17:19:21.945293 2014-08-13 17:38:02.716775 7fe9bae887a0 20 read_log 194215'518 (194215'498) delete 54d3069f/benchmark_data_storage0111-ib.pocstor_64498_object13146/head//22 by client.1049635.0:47349 2014-08-12 17:19:22.275502 2014-08-13 17:38:02.716787 7fe9bae887a0 20 read_log 194215'519 (194215'499) delete 67ec69f/benchmark_data_storage0111-ib.pocstor_64498_object13220/head//22 by client.1049635.0:47423 2014-08-12 17:19:22.297624 2014-08-13 17:38:02.716799 7fe9bae887a0 20 read_log 194215'520 (194215'500) delete f8fbbe9f/benchmark_data_storage0111-ib.pocstor_64498_object14873/head//22 by client.1049635.0:49076 2014-08-12 17:19:22.836593 2014-08-13 17:38:02.716811 7fe9bae887a0 20 read_log 194215'521 (194215'501) delete bd477e9f/benchmark_data_storage0111-ib.pocstor_64498_object15889/head//22 by client.1049635.0:50092 2014-08-12 17:19:23.156553 2014-08-13 17:38:02.716823 7fe9bae887a0 20 read_log 194215'522 (194215'502) delete 669ce69f/benchmark_data_storage0111-ib.pocstor_64498_object16308/head//22 by client.1049635.0:50511 2014-08-12 17:19:23.288147 2014-08-13 17:38:02.716838 7fe9bae887a0 20 read_log 194215'523 (194215'503) delete 1de5e9f/benchmark_data_storage0111-ib.pocstor_64498_object20675/head//22 by client.1049635.0:54878 2014-08-12 17:19:24.767212 2014-08-13 17:38:02.716850 7fe9bae887a0 20 read_log 194215'524 (194215'504) delete 23b8069f/benchmark_data_storage0111-ib.pocstor_64498_object23355/head//22 by client.1049635.0:57558 2014-08-12 17:19:25.747190 2014-08-13 17:38:02.716864 7fe9bae887a0 20 read_log 194215'525 (194215'505) delete 4df4be9f/benchmark_data_storage0111-ib.pocstor_64498_object24264/head//22 by client.1049635.0:58467 2014-08-12 17:19:26.051697 2014-08-13 17:38:02.716877 7fe9bae887a0 20 read_log 194215'526 (194215'506) delete c07e169f/benchmark_data_storage0111-ib.pocstor_64498_object24680/head//22 by client.1049635.0:58883 2014-08-12 17:19:26.202197 2014-08-13 17:38:02.716891 7fe9bae887a0 20 read_log 194215'527 (194215'507) delete a896469f/benchmark_data_storage0111-ib.pocstor_64498_object24850/head//22 by client.1049635.0:59053 2014-08-12 17:19:26.273128 2014-08-13 17:38:02.716903 7fe9bae887a0 20 read_log 194215'528 (194215'508) delete 1febae9f/benchmark_data_storage0111-ib.pocstor_64498_object27173/head//22 by client.1049635.0:61376 2014-08-12 17:19:27.064621 2014-08-13 17:38:02.717004 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.717022 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=0 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.717037 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=0 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Initial 0.002092 0 0.000000 2014-08-13 17:38:02.717049 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=0 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.717057 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=0 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.717065 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] log((0'0,194215'528], crt=192888'488) 2014-08-13 17:38:02.717074 7fe9bae887a0 10 osd.4 194478 pgid 22.6bd coll 22.6bd_head 2014-08-13 17:38:02.717192 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.6bd 2014-08-13 17:38:02.717201 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.717228 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.6bd(unlocked)] enter Initial 2014-08-13 17:38:02.717239 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.6bd(unlocked)] enter NotTrimming 2014-08-13 17:38:02.717340 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.717440 7fe9bae887a0 20 read_log 191648'69 (0'0) modify 6c0a7ebd/benchmark_data_storage0111-ib.pocstor_40745_object3741/head//22 by client.273158.0:3742 2014-07-30 14:09:04.972872 2014-08-13 17:38:02.717460 7fe9bae887a0 20 read_log 191648'89 (0'0) modify 587b1ebd/benchmark_data_storage0111-ib.pocstor_40745_object5009/head//22 by client.273158.0:5010 2014-07-30 14:09:11.065311 2014-08-13 17:38:02.717473 7fe9bae887a0 20 read_log 191648'93 (0'0) modify dc4a7ebd/benchmark_data_storage0111-ib.pocstor_40745_object5164/head//22 by client.273158.0:5165 2014-07-30 14:09:11.962809 2014-08-13 17:38:02.717485 7fe9bae887a0 20 read_log 191648'105 (0'0) modify 28253ebd/benchmark_data_storage0111-ib.pocstor_40745_object5962/head//22 by client.273158.0:5963 2014-07-30 14:09:15.655307 2014-08-13 17:38:02.717498 7fe9bae887a0 20 read_log 191648'114 (0'0) modify 937596bd/benchmark_data_storage0111-ib.pocstor_40745_object6376/head//22 by client.273158.0:6377 2014-07-30 14:09:18.064751 2014-08-13 17:38:02.717510 7fe9bae887a0 20 read_log 191648'119 (0'0) modify f1d2febd/benchmark_data_storage0111-ib.pocstor_40745_object6723/head//22 by client.273158.0:6724 2014-07-30 14:09:19.844380 2014-08-13 17:38:02.717533 7fe9bae887a0 20 read_log 191648'174 (0'0) modify 76f636bd/benchmark_data_storage0111-ib.pocstor_40745_object9923/head//22 by client.273158.0:9924 2014-07-30 14:09:35.421885 2014-08-13 17:38:02.717549 7fe9bae887a0 20 read_log 191648'226 (0'0) modify ae50eebd/benchmark_data_storage0111-ib.pocstor_40745_object13564/head//22 by client.273158.0:13565 2014-07-30 14:09:54.956422 2014-08-13 17:38:02.717561 7fe9bae887a0 20 read_log 191648'249 (0'0) modify 77816bd/benchmark_data_storage0111-ib.pocstor_40745_object15019/head//22 by client.273158.0:15020 2014-07-30 14:10:02.754545 2014-08-13 17:38:02.717573 7fe9bae887a0 20 read_log 191648'250 (0'0) modify ab7eebd/benchmark_data_storage0111-ib.pocstor_40745_object15035/head//22 by client.273158.0:15036 2014-07-30 14:10:02.813743 2014-08-13 17:38:02.717585 7fe9bae887a0 20 read_log 191648'255 (0'0) modify dbab9ebd/benchmark_data_storage0111-ib.pocstor_40745_object15269/head//22 by client.273158.0:15270 2014-07-30 14:10:03.747777 2014-08-13 17:38:02.717597 7fe9bae887a0 20 read_log 191648'261 (0'0) modify f292a6bd/benchmark_data_storage0111-ib.pocstor_40745_object15669/head//22 by client.273158.0:15670 2014-07-30 14:10:05.563476 2014-08-13 17:38:02.717609 7fe9bae887a0 20 read_log 191648'287 (0'0) modify b1c366bd/benchmark_data_storage0111-ib.pocstor_40745_object17364/head//22 by client.273158.0:17365 2014-07-30 14:10:15.035417 2014-08-13 17:38:02.717621 7fe9bae887a0 20 read_log 191648'295 (0'0) modify bda64ebd/benchmark_data_storage0111-ib.pocstor_40745_object17841/head//22 by client.273158.0:17842 2014-07-30 14:10:17.208194 2014-08-13 17:38:02.717633 7fe9bae887a0 20 read_log 191727'371 (0'0) modify 6f376ebd/benchmark_data_storage0111-ib.pocstor_51285_object1941/head//22 by client.275175.0:1942 2014-07-30 14:18:40.709689 2014-08-13 17:38:02.717645 7fe9bae887a0 20 read_log 191727'375 (0'0) modify c8078ebd/benchmark_data_storage0111-ib.pocstor_51285_object2881/head//22 by client.275175.0:2882 2014-07-30 14:18:43.846370 2014-08-13 17:38:02.717657 7fe9bae887a0 20 read_log 191727'376 (0'0) modify c21b76bd/benchmark_data_storage0111-ib.pocstor_51285_object3086/head//22 by client.275175.0:3087 2014-07-30 14:18:44.498279 2014-08-13 17:38:02.717669 7fe9bae887a0 20 read_log 191727'379 (0'0) modify ca9096bd/benchmark_data_storage0111-ib.pocstor_51285_object4142/head//22 by client.275175.0:4143 2014-07-30 14:18:47.793031 2014-08-13 17:38:02.717681 7fe9bae887a0 20 read_log 191727'385 (0'0) modify c4e5c6bd/benchmark_data_storage0111-ib.pocstor_51285_object6793/head//22 by client.275175.0:6794 2014-07-30 14:18:56.139313 2014-08-13 17:38:02.717693 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 238d56bd/benchmark_data_storage0111-ib.pocstor_51285_object9014/head//22 by client.275175.0:9015 2014-07-30 14:19:02.389751 2014-08-13 17:38:02.717705 7fe9bae887a0 20 read_log 191727'391 (0'0) modify 683b8ebd/benchmark_data_storage0111-ib.pocstor_51285_object9699/head//22 by client.275175.0:9700 2014-07-30 14:19:04.343122 2014-08-13 17:38:02.717718 7fe9bae887a0 20 read_log 191727'394 (0'0) modify 8f4eeebd/benchmark_data_storage0111-ib.pocstor_51285_object11551/head//22 by client.275175.0:11552 2014-07-30 14:19:09.910373 2014-08-13 17:38:02.717730 7fe9bae887a0 20 read_log 191727'395 (0'0) modify 95c68ebd/benchmark_data_storage0111-ib.pocstor_51285_object11606/head//22 by client.275175.0:11607 2014-07-30 14:19:10.064654 2014-08-13 17:38:02.717743 7fe9bae887a0 20 read_log 191727'399 (0'0) modify e079debd/benchmark_data_storage0111-ib.pocstor_51285_object15815/head//22 by client.275175.0:15816 2014-07-30 14:19:23.297614 2014-08-13 17:38:02.717755 7fe9bae887a0 20 read_log 191727'401 (0'0) modify 42e596bd/benchmark_data_storage0111-ib.pocstor_51285_object17487/head//22 by client.275175.0:17488 2014-07-30 14:19:28.512848 2014-08-13 17:38:02.717773 7fe9bae887a0 20 read_log 191727'403 (0'0) modify d484e6bd/benchmark_data_storage0111-ib.pocstor_51285_object18608/head//22 by client.275175.0:18609 2014-07-30 14:19:31.790986 2014-08-13 17:38:02.717786 7fe9bae887a0 20 read_log 191727'406 (0'0) modify d65536bd/benchmark_data_storage0111-ib.pocstor_51285_object21059/head//22 by client.275175.0:21060 2014-07-30 14:19:39.300085 2014-08-13 17:38:02.717798 7fe9bae887a0 20 read_log 191727'411 (0'0) modify b96bebd/benchmark_data_storage0111-ib.pocstor_51285_object22891/head//22 by client.275175.0:22892 2014-07-30 14:19:45.717804 2014-08-13 17:38:02.717810 7fe9bae887a0 20 read_log 191727'417 (0'0) modify 6eebd/benchmark_data_storage0111-ib.pocstor_51285_object27237/head//22 by client.275175.0:27238 2014-07-30 14:19:58.489338 2014-08-13 17:38:02.717822 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 2f09b6bd/benchmark_data_storage0111-ib.pocstor_51285_object28161/head//22 by client.275175.0:28162 2014-07-30 14:20:01.481766 2014-08-13 17:38:02.717834 7fe9bae887a0 20 read_log 191727'422 (0'0) modify 267cfebd/benchmark_data_storage0111-ib.pocstor_51285_object28855/head//22 by client.275175.0:28856 2014-07-30 14:20:03.484327 2014-08-13 17:38:02.717846 7fe9bae887a0 20 read_log 191727'425 (0'0) modify 31c3b6bd/benchmark_data_storage0111-ib.pocstor_51285_object30210/head//22 by client.275175.0:30211 2014-07-30 14:20:07.731180 2014-08-13 17:38:02.717858 7fe9bae887a0 20 read_log 191727'427 (0'0) modify 338abebd/benchmark_data_storage0111-ib.pocstor_51285_object31017/head//22 by client.275175.0:31018 2014-07-30 14:20:10.031751 2014-08-13 17:38:02.717870 7fe9bae887a0 20 read_log 191727'429 (0'0) modify 6c8fdebd/benchmark_data_storage0111-ib.pocstor_51285_object31418/head//22 by client.275175.0:31419 2014-07-30 14:20:11.165503 2014-08-13 17:38:02.717882 7fe9bae887a0 20 read_log 191727'430 (0'0) modify d87febd/benchmark_data_storage0111-ib.pocstor_51285_object32524/head//22 by client.275175.0:32525 2014-07-30 14:20:14.371434 2014-08-13 17:38:02.717896 7fe9bae887a0 20 read_log 192888'443 (0'0) modify 499fe6bd/benchmark_data_storage0111-ib.pocstor_35625_object1877/head//22 by client.940330.0:1878 2014-08-11 16:29:48.090530 2014-08-13 17:38:02.717909 7fe9bae887a0 20 read_log 192888'444 (0'0) modify ec534ebd/benchmark_data_storage0111-ib.pocstor_35625_object3073/head//22 by client.940330.0:3074 2014-08-11 16:29:52.837572 2014-08-13 17:38:02.717923 7fe9bae887a0 20 read_log 192888'445 (0'0) modify 59202ebd/benchmark_data_storage0111-ib.pocstor_35625_object4800/head//22 by client.940330.0:4801 2014-08-11 16:29:59.327603 2014-08-13 17:38:02.717935 7fe9bae887a0 20 read_log 192888'446 (0'0) modify 84445ebd/benchmark_data_storage0111-ib.pocstor_35625_object6105/head//22 by client.940330.0:6106 2014-08-11 16:30:03.955124 2014-08-13 17:38:02.717947 7fe9bae887a0 20 read_log 192888'447 (0'0) modify 2b64debd/benchmark_data_storage0111-ib.pocstor_35625_object7407/head//22 by client.940330.0:7408 2014-08-11 16:30:08.753977 2014-08-13 17:38:02.717959 7fe9bae887a0 20 read_log 192888'448 (0'0) modify 89a2a6bd/benchmark_data_storage0111-ib.pocstor_35625_object9352/head//22 by client.940330.0:9353 2014-08-11 16:30:15.562790 2014-08-13 17:38:02.717972 7fe9bae887a0 20 read_log 192888'449 (0'0) modify ac29a6bd/benchmark_data_storage0111-ib.pocstor_35625_object11055/head//22 by client.940330.0:11056 2014-08-11 16:30:22.013217 2014-08-13 17:38:02.717986 7fe9bae887a0 20 read_log 192888'450 (0'0) modify fdbb26bd/benchmark_data_storage0111-ib.pocstor_35625_object12712/head//22 by client.940330.0:12713 2014-08-11 16:30:28.684301 2014-08-13 17:38:02.717999 7fe9bae887a0 20 read_log 192888'451 (0'0) modify 8ecafebd/benchmark_data_storage0111-ib.pocstor_35625_object13683/head//22 by client.940330.0:13684 2014-08-11 16:30:32.452537 2014-08-13 17:38:02.718018 7fe9bae887a0 20 read_log 192888'452 (0'0) modify 5f603ebd/benchmark_data_storage0111-ib.pocstor_35625_object18152/head//22 by client.940330.0:18153 2014-08-11 16:30:49.357352 2014-08-13 17:38:02.718052 7fe9bae887a0 20 read_log 192888'453 (0'0) modify 2eaaebd/benchmark_data_storage0111-ib.pocstor_35625_object20659/head//22 by client.940330.0:20660 2014-08-11 16:30:59.279279 2014-08-13 17:38:02.718067 7fe9bae887a0 20 read_log 192888'454 (0'0) modify 7ad70ebd/benchmark_data_storage0111-ib.pocstor_35625_object23277/head//22 by client.940330.0:23278 2014-08-11 16:31:09.430039 2014-08-13 17:38:02.718079 7fe9bae887a0 20 read_log 192888'455 (0'0) modify e84c7ebd/benchmark_data_storage0111-ib.pocstor_35625_object23404/head//22 by client.940330.0:23405 2014-08-11 16:31:09.900039 2014-08-13 17:38:02.718092 7fe9bae887a0 20 read_log 192888'456 (0'0) modify 901d86bd/benchmark_data_storage0111-ib.pocstor_35625_object23801/head//22 by client.940330.0:23802 2014-08-11 16:31:11.422792 2014-08-13 17:38:02.718103 7fe9bae887a0 20 read_log 192888'457 (0'0) modify d762ebd/benchmark_data_storage0111-ib.pocstor_35625_object24691/head//22 by client.940330.0:24692 2014-08-11 16:31:14.684344 2014-08-13 17:38:02.718137 7fe9bae887a0 20 read_log 192888'458 (0'0) modify df3c4ebd/benchmark_data_storage0111-ib.pocstor_35625_object28332/head//22 by client.940330.0:28333 2014-08-11 16:31:29.007161 2014-08-13 17:38:02.718149 7fe9bae887a0 20 read_log 192888'459 (0'0) modify dbe02ebd/benchmark_data_storage0111-ib.pocstor_35625_object28470/head//22 by client.940330.0:28471 2014-08-11 16:31:29.549910 2014-08-13 17:38:02.718169 7fe9bae887a0 20 read_log 192888'460 (0'0) modify ee9126bd/benchmark_data_storage0111-ib.pocstor_35625_object30366/head//22 by client.940330.0:30367 2014-08-11 16:31:36.850174 2014-08-13 17:38:02.718182 7fe9bae887a0 20 read_log 192888'461 (192888'443) delete 499fe6bd/benchmark_data_storage0111-ib.pocstor_35625_object1877/head//22 by client.940330.0:33185 2014-08-11 16:31:41.960603 2014-08-13 17:38:02.718195 7fe9bae887a0 20 read_log 192888'462 (192888'444) delete ec534ebd/benchmark_data_storage0111-ib.pocstor_35625_object3073/head//22 by client.940330.0:34381 2014-08-11 16:31:42.270012 2014-08-13 17:38:02.718207 7fe9bae887a0 20 read_log 192888'463 (192888'445) delete 59202ebd/benchmark_data_storage0111-ib.pocstor_35625_object4800/head//22 by client.940330.0:36108 2014-08-11 16:31:42.766412 2014-08-13 17:38:02.718219 7fe9bae887a0 20 read_log 192888'464 (192888'446) delete 84445ebd/benchmark_data_storage0111-ib.pocstor_35625_object6105/head//22 by client.940330.0:37413 2014-08-11 16:31:43.105164 2014-08-13 17:38:02.718231 7fe9bae887a0 20 read_log 192888'465 (192888'447) delete 2b64debd/benchmark_data_storage0111-ib.pocstor_35625_object7407/head//22 by client.940330.0:38715 2014-08-11 16:31:43.440721 2014-08-13 17:38:02.718243 7fe9bae887a0 20 read_log 192888'466 (192888'448) delete 89a2a6bd/benchmark_data_storage0111-ib.pocstor_35625_object9352/head//22 by client.940330.0:40660 2014-08-11 16:31:43.933025 2014-08-13 17:38:02.718276 7fe9bae887a0 20 read_log 192888'467 (192888'449) delete ac29a6bd/benchmark_data_storage0111-ib.pocstor_35625_object11055/head//22 by client.940330.0:42363 2014-08-11 16:31:44.414944 2014-08-13 17:38:02.718289 7fe9bae887a0 20 read_log 192888'468 (192888'450) delete fdbb26bd/benchmark_data_storage0111-ib.pocstor_35625_object12712/head//22 by client.940330.0:44020 2014-08-11 16:31:44.838387 2014-08-13 17:38:02.718301 7fe9bae887a0 20 read_log 192888'469 (192888'451) delete 8ecafebd/benchmark_data_storage0111-ib.pocstor_35625_object13683/head//22 by client.940330.0:44991 2014-08-11 16:31:45.100298 2014-08-13 17:38:02.718314 7fe9bae887a0 20 read_log 192888'470 (192888'452) delete 5f603ebd/benchmark_data_storage0111-ib.pocstor_35625_object18152/head//22 by client.940330.0:49460 2014-08-11 16:31:46.338441 2014-08-13 17:38:02.718326 7fe9bae887a0 20 read_log 192888'471 (192888'453) delete 2eaaebd/benchmark_data_storage0111-ib.pocstor_35625_object20659/head//22 by client.940330.0:51967 2014-08-11 16:31:47.068080 2014-08-13 17:38:02.718362 7fe9bae887a0 20 read_log 192888'472 (192888'454) delete 7ad70ebd/benchmark_data_storage0111-ib.pocstor_35625_object23277/head//22 by client.940330.0:54585 2014-08-11 16:31:47.837831 2014-08-13 17:38:02.718378 7fe9bae887a0 20 read_log 192888'473 (192888'455) delete e84c7ebd/benchmark_data_storage0111-ib.pocstor_35625_object23404/head//22 by client.940330.0:54712 2014-08-11 16:31:47.871150 2014-08-13 17:38:02.718391 7fe9bae887a0 20 read_log 192888'474 (192888'456) delete 901d86bd/benchmark_data_storage0111-ib.pocstor_35625_object23801/head//22 by client.940330.0:55109 2014-08-11 16:31:47.976701 2014-08-13 17:38:02.718403 7fe9bae887a0 20 read_log 192888'475 (192888'457) delete d762ebd/benchmark_data_storage0111-ib.pocstor_35625_object24691/head//22 by client.940330.0:55999 2014-08-11 16:31:48.241379 2014-08-13 17:38:02.718415 7fe9bae887a0 20 read_log 192888'476 (192888'458) delete df3c4ebd/benchmark_data_storage0111-ib.pocstor_35625_object28332/head//22 by client.940330.0:59640 2014-08-11 16:31:49.293771 2014-08-13 17:38:02.718427 7fe9bae887a0 20 read_log 192888'477 (192888'459) delete dbe02ebd/benchmark_data_storage0111-ib.pocstor_35625_object28470/head//22 by client.940330.0:59778 2014-08-11 16:31:49.329750 2014-08-13 17:38:02.718439 7fe9bae887a0 20 read_log 192888'478 (192888'460) delete ee9126bd/benchmark_data_storage0111-ib.pocstor_35625_object30366/head//22 by client.940330.0:61674 2014-08-11 16:31:49.863707 2014-08-13 17:38:02.718451 7fe9bae887a0 20 read_log 194215'479 (0'0) modify d80e4ebd/benchmark_data_storage0111-ib.pocstor_64498_object7326/head//22 by client.1049635.0:7327 2014-08-12 17:17:40.049105 2014-08-13 17:38:02.718464 7fe9bae887a0 20 read_log 194215'480 (0'0) modify 3bceb6bd/benchmark_data_storage0111-ib.pocstor_64498_object12432/head//22 by client.1049635.0:12433 2014-08-12 17:17:55.711235 2014-08-13 17:38:02.718476 7fe9bae887a0 20 read_log 194215'481 (0'0) modify 176e3ebd/benchmark_data_storage0111-ib.pocstor_64498_object13452/head//22 by client.1049635.0:13453 2014-08-12 17:17:58.810035 2014-08-13 17:38:02.718488 7fe9bae887a0 20 read_log 194215'482 (0'0) modify 4e7d2ebd/benchmark_data_storage0111-ib.pocstor_64498_object13989/head//22 by client.1049635.0:13990 2014-08-12 17:18:00.496105 2014-08-13 17:38:02.718500 7fe9bae887a0 20 read_log 194215'483 (0'0) modify 7fc8b6bd/benchmark_data_storage0111-ib.pocstor_64498_object15927/head//22 by client.1049635.0:15928 2014-08-12 17:18:07.278169 2014-08-13 17:38:02.718513 7fe9bae887a0 20 read_log 194215'484 (0'0) modify 1ff32ebd/benchmark_data_storage0111-ib.pocstor_64498_object16682/head//22 by client.1049635.0:16683 2014-08-12 17:18:10.164760 2014-08-13 17:38:02.718525 7fe9bae887a0 20 read_log 194215'485 (0'0) modify b2a0eebd/benchmark_data_storage0111-ib.pocstor_64498_object20409/head//22 by client.1049635.0:20410 2014-08-12 17:18:24.330984 2014-08-13 17:38:02.718537 7fe9bae887a0 20 read_log 194215'486 (0'0) modify a611b6bd/benchmark_data_storage0111-ib.pocstor_64498_object21741/head//22 by client.1049635.0:21742 2014-08-12 17:18:29.357621 2014-08-13 17:38:02.718550 7fe9bae887a0 20 read_log 194215'487 (0'0) modify 64ff96bd/benchmark_data_storage0111-ib.pocstor_64498_object25750/head//22 by client.1049635.0:25751 2014-08-12 17:18:44.499040 2014-08-13 17:38:02.718562 7fe9bae887a0 20 read_log 194215'488 (0'0) modify 98271ebd/benchmark_data_storage0111-ib.pocstor_64498_object29249/head//22 by client.1049635.0:29250 2014-08-12 17:18:57.521174 2014-08-13 17:38:02.718574 7fe9bae887a0 20 read_log 194215'489 (0'0) modify 65a15ebd/benchmark_data_storage0111-ib.pocstor_64498_object31539/head//22 by client.1049635.0:31540 2014-08-12 17:19:06.778643 2014-08-13 17:38:02.718603 7fe9bae887a0 20 read_log 194215'490 (0'0) modify cd7296bd/benchmark_data_storage0111-ib.pocstor_64498_object32223/head//22 by client.1049635.0:32224 2014-08-12 17:19:09.425686 2014-08-13 17:38:02.718628 7fe9bae887a0 20 read_log 194215'491 (0'0) modify caa3cebd/benchmark_data_storage0111-ib.pocstor_64498_object32334/head//22 by client.1049635.0:32335 2014-08-12 17:19:09.856014 2014-08-13 17:38:02.718643 7fe9bae887a0 20 read_log 194215'492 (194215'479) delete d80e4ebd/benchmark_data_storage0111-ib.pocstor_64498_object7326/head//22 by client.1049635.0:41529 2014-08-12 17:19:20.394683 2014-08-13 17:38:02.718655 7fe9bae887a0 20 read_log 194215'493 (194215'480) delete 3bceb6bd/benchmark_data_storage0111-ib.pocstor_64498_object12432/head//22 by client.1049635.0:46635 2014-08-12 17:19:22.033587 2014-08-13 17:38:02.718668 7fe9bae887a0 20 read_log 194215'494 (194215'481) delete 176e3ebd/benchmark_data_storage0111-ib.pocstor_64498_object13452/head//22 by client.1049635.0:47655 2014-08-12 17:19:22.378598 2014-08-13 17:38:02.718680 7fe9bae887a0 20 read_log 194215'495 (194215'482) delete 4e7d2ebd/benchmark_data_storage0111-ib.pocstor_64498_object13989/head//22 by client.1049635.0:48192 2014-08-12 17:19:22.560768 2014-08-13 17:38:02.718692 7fe9bae887a0 20 read_log 194215'496 (194215'483) delete 7fc8b6bd/benchmark_data_storage0111-ib.pocstor_64498_object15927/head//22 by client.1049635.0:50130 2014-08-12 17:19:23.168613 2014-08-13 17:38:02.718704 7fe9bae887a0 20 read_log 194215'497 (194215'484) delete 1ff32ebd/benchmark_data_storage0111-ib.pocstor_64498_object16682/head//22 by client.1049635.0:50885 2014-08-12 17:19:23.410536 2014-08-13 17:38:02.718716 7fe9bae887a0 20 read_log 194215'498 (194215'485) delete b2a0eebd/benchmark_data_storage0111-ib.pocstor_64498_object20409/head//22 by client.1049635.0:54612 2014-08-12 17:19:24.670792 2014-08-13 17:38:02.718729 7fe9bae887a0 20 read_log 194215'499 (194215'486) delete a611b6bd/benchmark_data_storage0111-ib.pocstor_64498_object21741/head//22 by client.1049635.0:55944 2014-08-12 17:19:25.139696 2014-08-13 17:38:02.718741 7fe9bae887a0 20 read_log 194215'500 (194215'487) delete 64ff96bd/benchmark_data_storage0111-ib.pocstor_64498_object25750/head//22 by client.1049635.0:59953 2014-08-12 17:19:26.575293 2014-08-13 17:38:02.718753 7fe9bae887a0 20 read_log 194215'501 (194215'488) delete 98271ebd/benchmark_data_storage0111-ib.pocstor_64498_object29249/head//22 by client.1049635.0:63452 2014-08-12 17:19:27.801207 2014-08-13 17:38:02.718765 7fe9bae887a0 20 read_log 194215'502 (194215'489) delete 65a15ebd/benchmark_data_storage0111-ib.pocstor_64498_object31539/head//22 by client.1049635.0:65742 2014-08-12 17:19:28.572689 2014-08-13 17:38:02.718777 7fe9bae887a0 20 read_log 194215'503 (194215'490) delete cd7296bd/benchmark_data_storage0111-ib.pocstor_64498_object32223/head//22 by client.1049635.0:66426 2014-08-12 17:19:28.827080 2014-08-13 17:38:02.718789 7fe9bae887a0 20 read_log 194215'504 (194215'491) delete caa3cebd/benchmark_data_storage0111-ib.pocstor_64498_object32334/head//22 by client.1049635.0:66537 2014-08-12 17:19:28.867432 2014-08-13 17:38:02.718868 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.718888 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.718903 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001675 0 0.000000 2014-08-13 17:38:02.718913 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.718921 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194477 2014-08-13 17:38:02.718929 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'504], crt=191727'442) 2014-08-13 17:38:02.718938 7fe9bae887a0 10 osd.4 194478 pgid 22.707 coll 22.707_head 2014-08-13 17:38:02.719052 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.707 2014-08-13 17:38:02.719061 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.719099 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.707(unlocked)] enter Initial 2014-08-13 17:38:02.719115 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.707(unlocked)] enter NotTrimming 2014-08-13 17:38:02.719243 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.719354 7fe9bae887a0 20 read_log 191648'85 (0'0) modify 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 by client.273158.0:5717 2014-07-30 14:09:14.310106 2014-08-13 17:38:02.719375 7fe9bae887a0 20 read_log 191648'92 (0'0) modify e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 by client.273158.0:6406 2014-07-30 14:09:18.250213 2014-08-13 17:38:02.719396 7fe9bae887a0 20 read_log 191648'116 (0'0) modify 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 by client.273158.0:8225 2014-07-30 14:09:27.485296 2014-08-13 17:38:02.719411 7fe9bae887a0 20 read_log 191648'130 (0'0) modify 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 by client.273158.0:8996 2014-07-30 14:09:31.157678 2014-08-13 17:38:02.719423 7fe9bae887a0 20 read_log 191648'282 (0'0) modify 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 by client.273158.0:18038 2014-07-30 14:10:18.225194 2014-08-13 17:38:02.719435 7fe9bae887a0 20 read_log 191648'285 (0'0) modify e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 by client.273158.0:18159 2014-07-30 14:10:18.774337 2014-08-13 17:38:02.719447 7fe9bae887a0 20 read_log 191648'312 (0'0) modify dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 by client.273158.0:20042 2014-07-30 14:10:31.453445 2014-08-13 17:38:02.719459 7fe9bae887a0 20 read_log 191648'320 (0'0) modify 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 by client.273158.0:20422 2014-07-30 14:10:33.090957 2014-08-13 17:38:02.719471 7fe9bae887a0 20 read_log 191727'351 (0'0) modify b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 by client.275175.0:262 2014-07-30 14:18:34.439021 2014-08-13 17:38:02.719483 7fe9bae887a0 20 read_log 191727'352 (0'0) modify 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 by client.275175.0:597 2014-07-30 14:18:35.811348 2014-08-13 17:38:02.719495 7fe9bae887a0 20 read_log 191727'373 (0'0) modify 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 by client.275175.0:10862 2014-07-30 14:19:07.658492 2014-08-13 17:38:02.719507 7fe9bae887a0 20 read_log 191727'374 (0'0) modify a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 by client.275175.0:11395 2014-07-30 14:19:09.208503 2014-08-13 17:38:02.719519 7fe9bae887a0 20 read_log 191727'379 (0'0) modify d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 by client.275175.0:14707 2014-07-30 14:19:19.872066 2014-08-13 17:38:02.719531 7fe9bae887a0 20 read_log 191727'385 (0'0) modify 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 by client.275175.0:18723 2014-07-30 14:19:32.163026 2014-08-13 17:38:02.719543 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 by client.275175.0:18847 2014-07-30 14:19:32.745583 2014-08-13 17:38:02.719555 7fe9bae887a0 20 read_log 191727'388 (0'0) modify 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 by client.275175.0:19494 2014-07-30 14:19:34.761816 2014-08-13 17:38:02.719567 7fe9bae887a0 20 read_log 191727'391 (0'0) modify e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 by client.275175.0:21352 2014-07-30 14:19:40.165345 2014-08-13 17:38:02.719579 7fe9bae887a0 20 read_log 191727'414 (0'0) modify f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 by client.275175.0:36300 2014-07-30 14:20:26.191623 2014-08-13 17:38:02.719590 7fe9bae887a0 20 read_log 191727'416 (0'0) modify effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 by client.275175.0:37312 2014-07-30 14:20:29.239298 2014-08-13 17:38:02.719603 7fe9bae887a0 20 read_log 192888'418 (0'0) modify ffdaa707/benchmark_data_storage0111-ib.pocstor_35625_object1811/head//22 by client.940330.0:1812 2014-08-11 16:29:47.854039 2014-08-13 17:38:02.719615 7fe9bae887a0 20 read_log 192888'419 (0'0) modify 192d8f07/benchmark_data_storage0111-ib.pocstor_35625_object9977/head//22 by client.940330.0:9978 2014-08-11 16:30:17.804275 2014-08-13 17:38:02.719638 7fe9bae887a0 20 read_log 192888'420 (0'0) modify a7600f07/benchmark_data_storage0111-ib.pocstor_35625_object11904/head//22 by client.940330.0:11905 2014-08-11 16:30:25.450136 2014-08-13 17:38:02.719650 7fe9bae887a0 20 read_log 192888'421 (0'0) modify 7132d707/benchmark_data_storage0111-ib.pocstor_35625_object14936/head//22 by client.940330.0:14937 2014-08-11 16:30:37.254766 2014-08-13 17:38:02.719662 7fe9bae887a0 20 read_log 192888'422 (0'0) modify 17fd8707/benchmark_data_storage0111-ib.pocstor_35625_object16064/head//22 by client.940330.0:16065 2014-08-11 16:30:41.474318 2014-08-13 17:38:02.719674 7fe9bae887a0 20 read_log 192888'423 (0'0) modify 7aadaf07/benchmark_data_storage0111-ib.pocstor_35625_object18029/head//22 by client.940330.0:18030 2014-08-11 16:30:48.833523 2014-08-13 17:38:02.719686 7fe9bae887a0 20 read_log 192888'424 (0'0) modify f68bdf07/benchmark_data_storage0111-ib.pocstor_35625_object19965/head//22 by client.940330.0:19966 2014-08-11 16:30:56.686064 2014-08-13 17:38:02.719698 7fe9bae887a0 20 read_log 192888'425 (0'0) modify f804a707/benchmark_data_storage0111-ib.pocstor_35625_object20351/head//22 by client.940330.0:20352 2014-08-11 16:30:58.144701 2014-08-13 17:38:02.719710 7fe9bae887a0 20 read_log 192888'426 (0'0) modify 2a46b707/benchmark_data_storage0111-ib.pocstor_35625_object23245/head//22 by client.940330.0:23246 2014-08-11 16:31:09.311945 2014-08-13 17:38:02.719722 7fe9bae887a0 20 read_log 192888'427 (0'0) modify 66df6707/benchmark_data_storage0111-ib.pocstor_35625_object23606/head//22 by client.940330.0:23607 2014-08-11 16:31:10.644784 2014-08-13 17:38:02.719733 7fe9bae887a0 20 read_log 192888'428 (192888'418) delete ffdaa707/benchmark_data_storage0111-ib.pocstor_35625_object1811/head//22 by client.940330.0:33119 2014-08-11 16:31:41.942777 2014-08-13 17:38:02.719746 7fe9bae887a0 20 read_log 192888'429 (192888'419) delete 192d8f07/benchmark_data_storage0111-ib.pocstor_35625_object9977/head//22 by client.940330.0:41285 2014-08-11 16:31:44.107939 2014-08-13 17:38:02.719758 7fe9bae887a0 20 read_log 192888'430 (192888'420) delete a7600f07/benchmark_data_storage0111-ib.pocstor_35625_object11904/head//22 by client.940330.0:43212 2014-08-11 16:31:44.634586 2014-08-13 17:38:02.719770 7fe9bae887a0 20 read_log 192888'431 (192888'421) delete 7132d707/benchmark_data_storage0111-ib.pocstor_35625_object14936/head//22 by client.940330.0:46244 2014-08-11 16:31:45.440139 2014-08-13 17:38:02.719782 7fe9bae887a0 20 read_log 192888'432 (192888'422) delete 17fd8707/benchmark_data_storage0111-ib.pocstor_35625_object16064/head//22 by client.940330.0:47372 2014-08-11 16:31:45.743082 2014-08-13 17:38:02.719794 7fe9bae887a0 20 read_log 192888'433 (192888'423) delete 7aadaf07/benchmark_data_storage0111-ib.pocstor_35625_object18029/head//22 by client.940330.0:49337 2014-08-11 16:31:46.295067 2014-08-13 17:38:02.719806 7fe9bae887a0 20 read_log 192888'434 (192888'424) delete f68bdf07/benchmark_data_storage0111-ib.pocstor_35625_object19965/head//22 by client.940330.0:51273 2014-08-11 16:31:46.859799 2014-08-13 17:38:02.719819 7fe9bae887a0 20 read_log 192888'435 (192888'425) delete f804a707/benchmark_data_storage0111-ib.pocstor_35625_object20351/head//22 by client.940330.0:51659 2014-08-11 16:31:46.980897 2014-08-13 17:38:02.719831 7fe9bae887a0 20 read_log 192888'436 (192888'426) delete 2a46b707/benchmark_data_storage0111-ib.pocstor_35625_object23245/head//22 by client.940330.0:54553 2014-08-11 16:31:47.829576 2014-08-13 17:38:02.719843 7fe9bae887a0 20 read_log 192888'437 (192888'427) delete 66df6707/benchmark_data_storage0111-ib.pocstor_35625_object23606/head//22 by client.940330.0:54914 2014-08-11 16:31:47.925474 2014-08-13 17:38:02.719855 7fe9bae887a0 20 read_log 194215'438 (0'0) modify b74b7f07/benchmark_data_storage0111-ib.pocstor_64498_object628/head//22 by client.1049635.0:629 2014-08-12 17:17:19.139874 2014-08-13 17:38:02.719896 7fe9bae887a0 20 read_log 194215'439 (0'0) modify a6b69707/benchmark_data_storage0111-ib.pocstor_64498_object5166/head//22 by client.1049635.0:5167 2014-08-12 17:17:33.429115 2014-08-13 17:38:02.719912 7fe9bae887a0 20 read_log 194215'440 (0'0) modify cd906f07/benchmark_data_storage0111-ib.pocstor_64498_object5298/head//22 by client.1049635.0:5299 2014-08-12 17:17:33.829471 2014-08-13 17:38:02.719925 7fe9bae887a0 20 read_log 194215'441 (0'0) modify 3c992707/benchmark_data_storage0111-ib.pocstor_64498_object6667/head//22 by client.1049635.0:6668 2014-08-12 17:17:38.051169 2014-08-13 17:38:02.719939 7fe9bae887a0 20 read_log 194215'442 (0'0) modify 78daf707/benchmark_data_storage0111-ib.pocstor_64498_object7060/head//22 by client.1049635.0:7061 2014-08-12 17:17:39.302128 2014-08-13 17:38:02.719951 7fe9bae887a0 20 read_log 194215'443 (0'0) modify f54fbf07/benchmark_data_storage0111-ib.pocstor_64498_object9880/head//22 by client.1049635.0:9881 2014-08-12 17:17:47.583868 2014-08-13 17:38:02.719965 7fe9bae887a0 20 read_log 194215'444 (0'0) modify ffcca707/benchmark_data_storage0111-ib.pocstor_64498_object11309/head//22 by client.1049635.0:11310 2014-08-12 17:17:52.146349 2014-08-13 17:38:02.719978 7fe9bae887a0 20 read_log 194215'445 (0'0) modify edf48f07/benchmark_data_storage0111-ib.pocstor_64498_object14376/head//22 by client.1049635.0:14377 2014-08-12 17:18:01.858342 2014-08-13 17:38:02.719992 7fe9bae887a0 20 read_log 194215'446 (0'0) modify a8ffa707/benchmark_data_storage0111-ib.pocstor_64498_object16042/head//22 by client.1049635.0:16043 2014-08-12 17:18:07.709330 2014-08-13 17:38:02.720005 7fe9bae887a0 20 read_log 194215'447 (0'0) modify cf65af07/benchmark_data_storage0111-ib.pocstor_64498_object16219/head//22 by client.1049635.0:16220 2014-08-12 17:18:08.388202 2014-08-13 17:38:02.720019 7fe9bae887a0 20 read_log 194215'448 (0'0) modify 9a61b707/benchmark_data_storage0111-ib.pocstor_64498_object16970/head//22 by client.1049635.0:16971 2014-08-12 17:18:11.249491 2014-08-13 17:38:02.720031 7fe9bae887a0 20 read_log 194215'449 (0'0) modify fa523707/benchmark_data_storage0111-ib.pocstor_64498_object18011/head//22 by client.1049635.0:18012 2014-08-12 17:18:15.205555 2014-08-13 17:38:02.720046 7fe9bae887a0 20 read_log 194215'450 (0'0) modify 13a8c707/benchmark_data_storage0111-ib.pocstor_64498_object18082/head//22 by client.1049635.0:18083 2014-08-12 17:18:15.475173 2014-08-13 17:38:02.720058 7fe9bae887a0 20 read_log 194215'451 (0'0) modify 54a42707/benchmark_data_storage0111-ib.pocstor_64498_object18214/head//22 by client.1049635.0:18215 2014-08-12 17:18:15.970648 2014-08-13 17:38:02.720070 7fe9bae887a0 20 read_log 194215'452 (0'0) modify e6c1e707/benchmark_data_storage0111-ib.pocstor_64498_object19980/head//22 by client.1049635.0:19981 2014-08-12 17:18:22.707134 2014-08-13 17:38:02.720082 7fe9bae887a0 20 read_log 194215'453 (0'0) modify 6cf86f07/benchmark_data_storage0111-ib.pocstor_64498_object20600/head//22 by client.1049635.0:20601 2014-08-12 17:18:25.039329 2014-08-13 17:38:02.720094 7fe9bae887a0 20 read_log 194215'454 (0'0) modify 2b385f07/benchmark_data_storage0111-ib.pocstor_64498_object24619/head//22 by client.1049635.0:24620 2014-08-12 17:18:40.163976 2014-08-13 17:38:02.720106 7fe9bae887a0 20 read_log 194215'455 (0'0) modify 63021f07/benchmark_data_storage0111-ib.pocstor_64498_object25371/head//22 by client.1049635.0:25372 2014-08-12 17:18:43.086215 2014-08-13 17:38:02.720120 7fe9bae887a0 20 read_log 194215'456 (0'0) modify 74249707/benchmark_data_storage0111-ib.pocstor_64498_object26493/head//22 by client.1049635.0:26494 2014-08-12 17:18:47.262989 2014-08-13 17:38:02.720133 7fe9bae887a0 20 read_log 194215'457 (0'0) modify 244f6707/benchmark_data_storage0111-ib.pocstor_64498_object27386/head//22 by client.1049635.0:27387 2014-08-12 17:18:50.595924 2014-08-13 17:38:02.720147 7fe9bae887a0 20 read_log 194215'458 (0'0) modify 384ad707/benchmark_data_storage0111-ib.pocstor_64498_object28792/head//22 by client.1049635.0:28793 2014-08-12 17:18:55.824861 2014-08-13 17:38:02.720176 7fe9bae887a0 20 read_log 194215'459 (0'0) modify 91cec707/benchmark_data_storage0111-ib.pocstor_64498_object28812/head//22 by client.1049635.0:28813 2014-08-12 17:18:55.898241 2014-08-13 17:38:02.720192 7fe9bae887a0 20 read_log 194215'460 (0'0) modify d05a7707/benchmark_data_storage0111-ib.pocstor_64498_object34198/head//22 by client.1049635.0:34199 2014-08-12 17:19:16.932840 2014-08-13 17:38:02.720205 7fe9bae887a0 20 read_log 194215'461 (194215'438) delete b74b7f07/benchmark_data_storage0111-ib.pocstor_64498_object628/head//22 by client.1049635.0:34831 2014-08-12 17:19:18.199222 2014-08-13 17:38:02.720219 7fe9bae887a0 20 read_log 194215'462 (194215'439) delete a6b69707/benchmark_data_storage0111-ib.pocstor_64498_object5166/head//22 by client.1049635.0:39369 2014-08-12 17:19:19.660729 2014-08-13 17:38:02.720231 7fe9bae887a0 20 read_log 194215'463 (194215'440) delete cd906f07/benchmark_data_storage0111-ib.pocstor_64498_object5298/head//22 by client.1049635.0:39501 2014-08-12 17:19:19.707448 2014-08-13 17:38:02.720246 7fe9bae887a0 20 read_log 194215'464 (194215'441) delete 3c992707/benchmark_data_storage0111-ib.pocstor_64498_object6667/head//22 by client.1049635.0:40870 2014-08-12 17:19:20.176318 2014-08-13 17:38:02.720260 7fe9bae887a0 20 read_log 194215'465 (194215'442) delete 78daf707/benchmark_data_storage0111-ib.pocstor_64498_object7060/head//22 by client.1049635.0:41263 2014-08-12 17:19:20.300553 2014-08-13 17:38:02.720273 7fe9bae887a0 20 read_log 194215'466 (194215'443) delete f54fbf07/benchmark_data_storage0111-ib.pocstor_64498_object9880/head//22 by client.1049635.0:44083 2014-08-12 17:19:21.219078 2014-08-13 17:38:02.720285 7fe9bae887a0 20 read_log 194215'467 (194215'444) delete ffcca707/benchmark_data_storage0111-ib.pocstor_64498_object11309/head//22 by client.1049635.0:45512 2014-08-12 17:19:21.675554 2014-08-13 17:38:02.720297 7fe9bae887a0 20 read_log 194215'468 (194215'445) delete edf48f07/benchmark_data_storage0111-ib.pocstor_64498_object14376/head//22 by client.1049635.0:48579 2014-08-12 17:19:22.682512 2014-08-13 17:38:02.720309 7fe9bae887a0 20 read_log 194215'469 (194215'446) delete a8ffa707/benchmark_data_storage0111-ib.pocstor_64498_object16042/head//22 by client.1049635.0:50245 2014-08-12 17:19:23.203423 2014-08-13 17:38:02.720321 7fe9bae887a0 20 read_log 194215'470 (194215'447) delete cf65af07/benchmark_data_storage0111-ib.pocstor_64498_object16219/head//22 by client.1049635.0:50422 2014-08-12 17:19:23.260054 2014-08-13 17:38:02.720333 7fe9bae887a0 20 read_log 194215'471 (194215'448) delete 9a61b707/benchmark_data_storage0111-ib.pocstor_64498_object16970/head//22 by client.1049635.0:51173 2014-08-12 17:19:23.499810 2014-08-13 17:38:02.720345 7fe9bae887a0 20 read_log 194215'472 (194215'449) delete fa523707/benchmark_data_storage0111-ib.pocstor_64498_object18011/head//22 by client.1049635.0:52214 2014-08-12 17:19:23.833534 2014-08-13 17:38:02.720357 7fe9bae887a0 20 read_log 194215'473 (194215'450) delete 13a8c707/benchmark_data_storage0111-ib.pocstor_64498_object18082/head//22 by client.1049635.0:52285 2014-08-12 17:19:23.859171 2014-08-13 17:38:02.720369 7fe9bae887a0 20 read_log 194215'474 (194215'451) delete 54a42707/benchmark_data_storage0111-ib.pocstor_64498_object18214/head//22 by client.1049635.0:52417 2014-08-12 17:19:23.901419 2014-08-13 17:38:02.720381 7fe9bae887a0 20 read_log 194215'475 (194215'452) delete e6c1e707/benchmark_data_storage0111-ib.pocstor_64498_object19980/head//22 by client.1049635.0:54183 2014-08-12 17:19:24.507027 2014-08-13 17:38:02.720393 7fe9bae887a0 20 read_log 194215'476 (194215'453) delete 6cf86f07/benchmark_data_storage0111-ib.pocstor_64498_object20600/head//22 by client.1049635.0:54803 2014-08-12 17:19:24.741678 2014-08-13 17:38:02.720405 7fe9bae887a0 20 read_log 194215'477 (194215'454) delete 2b385f07/benchmark_data_storage0111-ib.pocstor_64498_object24619/head//22 by client.1049635.0:58822 2014-08-12 17:19:26.180530 2014-08-13 17:38:02.720419 7fe9bae887a0 20 read_log 194215'478 (194215'455) delete 63021f07/benchmark_data_storage0111-ib.pocstor_64498_object25371/head//22 by client.1049635.0:59574 2014-08-12 17:19:26.440954 2014-08-13 17:38:02.720432 7fe9bae887a0 20 read_log 194215'479 (194215'456) delete 74249707/benchmark_data_storage0111-ib.pocstor_64498_object26493/head//22 by client.1049635.0:60696 2014-08-12 17:19:26.819296 2014-08-13 17:38:02.720444 7fe9bae887a0 20 read_log 194215'480 (194215'457) delete 244f6707/benchmark_data_storage0111-ib.pocstor_64498_object27386/head//22 by client.1049635.0:61589 2014-08-12 17:19:27.140798 2014-08-13 17:38:02.720456 7fe9bae887a0 20 read_log 194215'481 (194215'458) delete 384ad707/benchmark_data_storage0111-ib.pocstor_64498_object28792/head//22 by client.1049635.0:62995 2014-08-12 17:19:27.627696 2014-08-13 17:38:02.720468 7fe9bae887a0 20 read_log 194215'482 (194215'459) delete 91cec707/benchmark_data_storage0111-ib.pocstor_64498_object28812/head//22 by client.1049635.0:63015 2014-08-12 17:19:27.632442 2014-08-13 17:38:02.720483 7fe9bae887a0 20 read_log 194215'483 (194215'460) delete d05a7707/benchmark_data_storage0111-ib.pocstor_64498_object34198/head//22 by client.1049635.0:68401 2014-08-12 17:19:29.552584 2014-08-13 17:38:02.720573 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.720591 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=0 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.720606 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=0 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001508 0 0.000000 2014-08-13 17:38:02.720618 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=0 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.720626 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=0 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.720635 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'483], crt=191727'417) 2014-08-13 17:38:02.720645 7fe9bae887a0 10 osd.4 194478 pgid 22.775 coll 22.775_head 2014-08-13 17:38:02.720753 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.775 2014-08-13 17:38:02.720762 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.720787 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.775(unlocked)] enter Initial 2014-08-13 17:38:02.720798 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.775(unlocked)] enter NotTrimming 2014-08-13 17:38:02.720922 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.721033 7fe9bae887a0 20 read_log 191648'55 (0'0) modify f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 by client.273158.0:3019 2014-07-30 14:09:01.459405 2014-08-13 17:38:02.721074 7fe9bae887a0 20 read_log 191648'59 (0'0) modify d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 by client.273158.0:3104 2014-07-30 14:09:01.784898 2014-08-13 17:38:02.721087 7fe9bae887a0 20 read_log 191648'93 (0'0) modify c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 by client.273158.0:4778 2014-07-30 14:09:10.036012 2014-08-13 17:38:02.721099 7fe9bae887a0 20 read_log 191648'107 (0'0) modify e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 by client.273158.0:5727 2014-07-30 14:09:14.350650 2014-08-13 17:38:02.721111 7fe9bae887a0 20 read_log 191648'130 (0'0) modify 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 by client.273158.0:6835 2014-07-30 14:09:20.420522 2014-08-13 17:38:02.721124 7fe9bae887a0 20 read_log 191648'138 (0'0) modify f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 by client.273158.0:7388 2014-07-30 14:09:22.717046 2014-08-13 17:38:02.721136 7fe9bae887a0 20 read_log 191648'199 (0'0) modify f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 by client.273158.0:11404 2014-07-30 14:09:43.889130 2014-08-13 17:38:02.721148 7fe9bae887a0 20 read_log 191648'275 (0'0) modify c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 by client.273158.0:15851 2014-07-30 14:10:06.539259 2014-08-13 17:38:02.721166 7fe9bae887a0 20 read_log 191648'281 (0'0) modify 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 by client.273158.0:16442 2014-07-30 14:10:09.851192 2014-08-13 17:38:02.721179 7fe9bae887a0 20 read_log 191648'308 (0'0) modify 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 by client.273158.0:18988 2014-07-30 14:10:23.577343 2014-08-13 17:38:02.721193 7fe9bae887a0 20 read_log 191648'323 (0'0) modify 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 by client.273158.0:20241 2014-07-30 14:10:32.420087 2014-08-13 17:38:02.721208 7fe9bae887a0 20 read_log 191727'370 (0'0) modify 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 by client.275175.0:1991 2014-07-30 14:18:40.868696 2014-08-13 17:38:02.721221 7fe9bae887a0 20 read_log 191727'374 (0'0) modify ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 by client.275175.0:3077 2014-07-30 14:18:44.466494 2014-08-13 17:38:02.721233 7fe9bae887a0 20 read_log 191727'383 (0'0) modify cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 by client.275175.0:5894 2014-07-30 14:18:52.942030 2014-08-13 17:38:02.721245 7fe9bae887a0 20 read_log 191727'384 (0'0) modify 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 by client.275175.0:7758 2014-07-30 14:18:58.850466 2014-08-13 17:38:02.721257 7fe9bae887a0 20 read_log 191727'388 (0'0) modify fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 by client.275175.0:9745 2014-07-30 14:19:04.467815 2014-08-13 17:38:02.721269 7fe9bae887a0 20 read_log 191727'389 (0'0) modify ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 by client.275175.0:11770 2014-07-30 14:19:10.522279 2014-08-13 17:38:02.721281 7fe9bae887a0 20 read_log 191727'392 (0'0) modify a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 by client.275175.0:13336 2014-07-30 14:19:15.680276 2014-08-13 17:38:02.721293 7fe9bae887a0 20 read_log 191727'399 (0'0) modify ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 by client.275175.0:16060 2014-07-30 14:19:24.102573 2014-08-13 17:38:02.721332 7fe9bae887a0 20 read_log 191727'400 (0'0) modify 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 by client.275175.0:16311 2014-07-30 14:19:24.994509 2014-08-13 17:38:02.721349 7fe9bae887a0 20 read_log 191727'401 (0'0) modify d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 by client.275175.0:16477 2014-07-30 14:19:25.598642 2014-08-13 17:38:02.721361 7fe9bae887a0 20 read_log 191727'402 (0'0) modify 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 by client.275175.0:16647 2014-07-30 14:19:26.162746 2014-08-13 17:38:02.721373 7fe9bae887a0 20 read_log 191727'410 (0'0) modify 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 by client.275175.0:19897 2014-07-30 14:19:35.877318 2014-08-13 17:38:02.721387 7fe9bae887a0 20 read_log 191727'421 (0'0) modify 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 by client.275175.0:23920 2014-07-30 14:19:48.609469 2014-08-13 17:38:02.721400 7fe9bae887a0 20 read_log 191727'429 (0'0) modify fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 by client.275175.0:28023 2014-07-30 14:20:01.091986 2014-08-13 17:38:02.721412 7fe9bae887a0 20 read_log 191727'433 (0'0) modify 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 by client.275175.0:29710 2014-07-30 14:20:06.074603 2014-08-13 17:38:02.721424 7fe9bae887a0 20 read_log 191727'434 (0'0) modify 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 by client.275175.0:30317 2014-07-30 14:20:08.061941 2014-08-13 17:38:02.721436 7fe9bae887a0 20 read_log 191727'441 (0'0) modify 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 by client.275175.0:34100 2014-07-30 14:20:19.275316 2014-08-13 17:38:02.721448 7fe9bae887a0 20 read_log 191727'442 (0'0) modify 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 by client.275175.0:34242 2014-07-30 14:20:19.744007 2014-08-13 17:38:02.721460 7fe9bae887a0 20 read_log 191727'445 (0'0) modify 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 by client.275175.0:34778 2014-07-30 14:20:21.343072 2014-08-13 17:38:02.721471 7fe9bae887a0 20 read_log 191727'450 (0'0) modify 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 by client.275175.0:36744 2014-07-30 14:20:27.658549 2014-08-13 17:38:02.721483 7fe9bae887a0 20 read_log 192674'454 (0'0) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1150 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721495 7fe9bae887a0 20 read_log 192674'455 (192674'454) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1151 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721507 7fe9bae887a0 20 read_log 192674'456 (192674'455) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1152 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721519 7fe9bae887a0 20 read_log 192674'457 (192674'456) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1153 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721531 7fe9bae887a0 20 read_log 192674'458 (192674'457) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1154 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721543 7fe9bae887a0 20 read_log 192674'459 (192674'458) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1155 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721555 7fe9bae887a0 20 read_log 192674'460 (192674'459) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1156 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721567 7fe9bae887a0 20 read_log 192674'461 (192674'460) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1157 2014-07-31 18:03:42.896000 2014-08-13 17:38:02.721586 7fe9bae887a0 20 read_log 192888'462 (0'0) modify 701fa775/benchmark_data_storage0111-ib.pocstor_35625_object2166/head//22 by client.940330.0:2167 2014-08-11 16:29:49.159399 2014-08-13 17:38:02.721603 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 773a2f75/benchmark_data_storage0111-ib.pocstor_35625_object4806/head//22 by client.940330.0:4807 2014-08-11 16:29:59.347606 2014-08-13 17:38:02.721618 7fe9bae887a0 20 read_log 192888'464 (0'0) modify 99d48775/benchmark_data_storage0111-ib.pocstor_35625_object13110/head//22 by client.940330.0:13111 2014-08-11 16:30:30.315121 2014-08-13 17:38:02.721630 7fe9bae887a0 20 read_log 192888'465 (0'0) modify c8262775/benchmark_data_storage0111-ib.pocstor_35625_object14655/head//22 by client.940330.0:14656 2014-08-11 16:30:36.210678 2014-08-13 17:38:02.721644 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 89a37f75/benchmark_data_storage0111-ib.pocstor_35625_object15019/head//22 by client.940330.0:15020 2014-08-11 16:30:37.565249 2014-08-13 17:38:02.721656 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 5e9ca775/benchmark_data_storage0111-ib.pocstor_35625_object16815/head//22 by client.940330.0:16816 2014-08-11 16:30:44.275872 2014-08-13 17:38:02.721686 7fe9bae887a0 20 read_log 192888'468 (0'0) modify 787ff775/benchmark_data_storage0111-ib.pocstor_35625_object18765/head//22 by client.940330.0:18766 2014-08-11 16:30:51.806638 2014-08-13 17:38:02.721700 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 570bf775/benchmark_data_storage0111-ib.pocstor_35625_object18912/head//22 by client.940330.0:18913 2014-08-11 16:30:52.344670 2014-08-13 17:38:02.721712 7fe9bae887a0 20 read_log 192888'470 (0'0) modify 7f5e6f75/benchmark_data_storage0111-ib.pocstor_35625_object20067/head//22 by client.940330.0:20068 2014-08-11 16:30:57.063322 2014-08-13 17:38:02.721727 7fe9bae887a0 20 read_log 192888'471 (0'0) modify 2ffa9f75/benchmark_data_storage0111-ib.pocstor_35625_object20283/head//22 by client.940330.0:20284 2014-08-11 16:30:57.895618 2014-08-13 17:38:02.721739 7fe9bae887a0 20 read_log 192888'472 (0'0) modify 51c84775/benchmark_data_storage0111-ib.pocstor_35625_object20461/head//22 by client.940330.0:20462 2014-08-11 16:30:58.552190 2014-08-13 17:38:02.721752 7fe9bae887a0 20 read_log 192888'473 (0'0) modify c08a1775/benchmark_data_storage0111-ib.pocstor_35625_object22215/head//22 by client.940330.0:22216 2014-08-11 16:31:05.372008 2014-08-13 17:38:02.721765 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 1032bf75/benchmark_data_storage0111-ib.pocstor_35625_object24071/head//22 by client.940330.0:24072 2014-08-11 16:31:12.408178 2014-08-13 17:38:02.721778 7fe9bae887a0 20 read_log 192888'475 (0'0) modify d1c43f75/benchmark_data_storage0111-ib.pocstor_35625_object25401/head//22 by client.940330.0:25402 2014-08-11 16:31:17.456028 2014-08-13 17:38:02.721790 7fe9bae887a0 20 read_log 192888'476 (0'0) modify adcbe775/benchmark_data_storage0111-ib.pocstor_35625_object26205/head//22 by client.940330.0:26206 2014-08-11 16:31:20.600409 2014-08-13 17:38:02.721804 7fe9bae887a0 20 read_log 192888'477 (192888'462) delete 701fa775/benchmark_data_storage0111-ib.pocstor_35625_object2166/head//22 by client.940330.0:33474 2014-08-11 16:31:42.034247 2014-08-13 17:38:02.721818 7fe9bae887a0 20 read_log 192888'478 (192888'463) delete 773a2f75/benchmark_data_storage0111-ib.pocstor_35625_object4806/head//22 by client.940330.0:36114 2014-08-11 16:31:42.769490 2014-08-13 17:38:02.721830 7fe9bae887a0 20 read_log 192888'479 (192888'464) delete 99d48775/benchmark_data_storage0111-ib.pocstor_35625_object13110/head//22 by client.940330.0:44418 2014-08-11 16:31:44.949156 2014-08-13 17:38:02.721842 7fe9bae887a0 20 read_log 192888'480 (192888'465) delete c8262775/benchmark_data_storage0111-ib.pocstor_35625_object14655/head//22 by client.940330.0:45963 2014-08-11 16:31:45.361611 2014-08-13 17:38:02.721855 7fe9bae887a0 20 read_log 192888'481 (192888'466) delete 89a37f75/benchmark_data_storage0111-ib.pocstor_35625_object15019/head//22 by client.940330.0:46327 2014-08-11 16:31:45.462967 2014-08-13 17:38:02.721876 7fe9bae887a0 20 read_log 192888'482 (192888'467) delete 5e9ca775/benchmark_data_storage0111-ib.pocstor_35625_object16815/head//22 by client.940330.0:48123 2014-08-11 16:31:45.938397 2014-08-13 17:38:02.721891 7fe9bae887a0 20 read_log 192888'483 (192888'468) delete 787ff775/benchmark_data_storage0111-ib.pocstor_35625_object18765/head//22 by client.940330.0:50073 2014-08-11 16:31:46.516766 2014-08-13 17:38:02.721903 7fe9bae887a0 20 read_log 192888'484 (192888'469) delete 570bf775/benchmark_data_storage0111-ib.pocstor_35625_object18912/head//22 by client.940330.0:50220 2014-08-11 16:31:46.557309 2014-08-13 17:38:02.721917 7fe9bae887a0 20 read_log 192888'485 (192888'470) delete 7f5e6f75/benchmark_data_storage0111-ib.pocstor_35625_object20067/head//22 by client.940330.0:51375 2014-08-11 16:31:46.899816 2014-08-13 17:38:02.721934 7fe9bae887a0 20 read_log 192888'486 (192888'471) delete 2ffa9f75/benchmark_data_storage0111-ib.pocstor_35625_object20283/head//22 by client.940330.0:51591 2014-08-11 16:31:46.963907 2014-08-13 17:38:02.721948 7fe9bae887a0 20 read_log 192888'487 (192888'472) delete 51c84775/benchmark_data_storage0111-ib.pocstor_35625_object20461/head//22 by client.940330.0:51769 2014-08-11 16:31:47.011071 2014-08-13 17:38:02.721961 7fe9bae887a0 20 read_log 192888'488 (192888'473) delete c08a1775/benchmark_data_storage0111-ib.pocstor_35625_object22215/head//22 by client.940330.0:53523 2014-08-11 16:31:47.515081 2014-08-13 17:38:02.721975 7fe9bae887a0 20 read_log 192888'489 (192888'474) delete 1032bf75/benchmark_data_storage0111-ib.pocstor_35625_object24071/head//22 by client.940330.0:55379 2014-08-11 16:31:48.061982 2014-08-13 17:38:02.721987 7fe9bae887a0 20 read_log 192888'490 (192888'475) delete d1c43f75/benchmark_data_storage0111-ib.pocstor_35625_object25401/head//22 by client.940330.0:56709 2014-08-11 16:31:48.468905 2014-08-13 17:38:02.722002 7fe9bae887a0 20 read_log 192888'491 (192888'476) delete adcbe775/benchmark_data_storage0111-ib.pocstor_35625_object26205/head//22 by client.940330.0:57513 2014-08-11 16:31:48.706086 2014-08-13 17:38:02.722014 7fe9bae887a0 20 read_log 194215'492 (0'0) modify d8fe4f75/benchmark_data_storage0111-ib.pocstor_64498_object6040/head//22 by client.1049635.0:6041 2014-08-12 17:17:36.191556 2014-08-13 17:38:02.722028 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 50715775/benchmark_data_storage0111-ib.pocstor_64498_object9862/head//22 by client.1049635.0:9863 2014-08-12 17:17:47.533366 2014-08-13 17:38:02.722040 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 1edfcf75/benchmark_data_storage0111-ib.pocstor_64498_object13368/head//22 by client.1049635.0:13369 2014-08-12 17:17:58.551763 2014-08-13 17:38:02.722054 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 6f2d9775/benchmark_data_storage0111-ib.pocstor_64498_object13671/head//22 by client.1049635.0:13672 2014-08-12 17:17:59.565349 2014-08-13 17:38:02.722069 7fe9bae887a0 20 read_log 194215'496 (0'0) modify c57baf75/benchmark_data_storage0111-ib.pocstor_64498_object15930/head//22 by client.1049635.0:15931 2014-08-12 17:18:07.290047 2014-08-13 17:38:02.722083 7fe9bae887a0 20 read_log 194215'497 (0'0) modify e23e3775/benchmark_data_storage0111-ib.pocstor_64498_object17738/head//22 by client.1049635.0:17739 2014-08-12 17:18:14.163227 2014-08-13 17:38:02.722097 7fe9bae887a0 20 read_log 194215'498 (0'0) modify c52be775/benchmark_data_storage0111-ib.pocstor_64498_object20965/head//22 by client.1049635.0:20966 2014-08-12 17:18:26.393736 2014-08-13 17:38:02.722110 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 7aab3f75/benchmark_data_storage0111-ib.pocstor_64498_object22635/head//22 by client.1049635.0:22636 2014-08-12 17:18:32.773532 2014-08-13 17:38:02.722122 7fe9bae887a0 20 read_log 194215'500 (0'0) modify ca9adf75/benchmark_data_storage0111-ib.pocstor_64498_object24677/head//22 by client.1049635.0:24678 2014-08-12 17:18:40.376462 2014-08-13 17:38:02.722135 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 6c2ecf75/benchmark_data_storage0111-ib.pocstor_64498_object26415/head//22 by client.1049635.0:26416 2014-08-12 17:18:46.967325 2014-08-13 17:38:02.722149 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 9942ff75/benchmark_data_storage0111-ib.pocstor_64498_object29962/head//22 by client.1049635.0:29963 2014-08-12 17:19:00.304655 2014-08-13 17:38:02.722172 7fe9bae887a0 20 read_log 194215'503 (194215'492) delete d8fe4f75/benchmark_data_storage0111-ib.pocstor_64498_object6040/head//22 by client.1049635.0:40243 2014-08-12 17:19:19.969487 2014-08-13 17:38:02.722184 7fe9bae887a0 20 read_log 194215'504 (194215'493) delete 50715775/benchmark_data_storage0111-ib.pocstor_64498_object9862/head//22 by client.1049635.0:44065 2014-08-12 17:19:21.212105 2014-08-13 17:38:02.722199 7fe9bae887a0 20 read_log 194215'505 (194215'494) delete 1edfcf75/benchmark_data_storage0111-ib.pocstor_64498_object13368/head//22 by client.1049635.0:47571 2014-08-12 17:19:22.348911 2014-08-13 17:38:02.722211 7fe9bae887a0 20 read_log 194215'506 (194215'495) delete 6f2d9775/benchmark_data_storage0111-ib.pocstor_64498_object13671/head//22 by client.1049635.0:47874 2014-08-12 17:19:22.463990 2014-08-13 17:38:02.722226 7fe9bae887a0 20 read_log 194215'507 (194215'496) delete c57baf75/benchmark_data_storage0111-ib.pocstor_64498_object15930/head//22 by client.1049635.0:50133 2014-08-12 17:19:23.168753 2014-08-13 17:38:02.722238 7fe9bae887a0 20 read_log 194215'508 (194215'497) delete e23e3775/benchmark_data_storage0111-ib.pocstor_64498_object17738/head//22 by client.1049635.0:51941 2014-08-12 17:19:23.739302 2014-08-13 17:38:02.722252 7fe9bae887a0 20 read_log 194215'509 (194215'498) delete c52be775/benchmark_data_storage0111-ib.pocstor_64498_object20965/head//22 by client.1049635.0:55168 2014-08-12 17:19:24.869220 2014-08-13 17:38:02.722264 7fe9bae887a0 20 read_log 194215'510 (194215'499) delete 7aab3f75/benchmark_data_storage0111-ib.pocstor_64498_object22635/head//22 by client.1049635.0:56838 2014-08-12 17:19:25.472617 2014-08-13 17:38:02.722278 7fe9bae887a0 20 read_log 194215'511 (194215'500) delete ca9adf75/benchmark_data_storage0111-ib.pocstor_64498_object24677/head//22 by client.1049635.0:58880 2014-08-12 17:19:26.201755 2014-08-13 17:38:02.722291 7fe9bae887a0 20 read_log 194215'512 (194215'501) delete 6c2ecf75/benchmark_data_storage0111-ib.pocstor_64498_object26415/head//22 by client.1049635.0:60618 2014-08-12 17:19:26.795142 2014-08-13 17:38:02.722305 7fe9bae887a0 20 read_log 194215'513 (194215'502) delete 9942ff75/benchmark_data_storage0111-ib.pocstor_64498_object29962/head//22 by client.1049635.0:64165 2014-08-12 17:19:28.033744 2014-08-13 17:38:02.722381 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.722399 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=0 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.722414 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=0 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001627 0 0.000000 2014-08-13 17:38:02.722426 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=0 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.722434 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=0 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.722443 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'513], crt=191727'453) 2014-08-13 17:38:02.722453 7fe9bae887a0 10 osd.4 194478 pgid 22.791 coll 22.791_head 2014-08-13 17:38:02.722559 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 22.791 2014-08-13 17:38:02.722568 7fe9bae887a0 10 osd.4 194478 _get_pool 22 2014-08-13 17:38:02.722594 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.791(unlocked)] enter Initial 2014-08-13 17:38:02.722605 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.791(unlocked)] enter NotTrimming 2014-08-13 17:38:02.722726 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.722837 7fe9bae887a0 20 read_log 191648'84 (0'0) modify db435f91/benchmark_data_storage0111-ib.pocstor_40745_object5693/head//22 by client.273158.0:5694 2014-07-30 14:09:14.216971 2014-08-13 17:38:02.722857 7fe9bae887a0 20 read_log 191648'115 (0'0) modify c5b0af91/benchmark_data_storage0111-ib.pocstor_40745_object6884/head//22 by client.273158.0:6885 2014-07-30 14:09:20.637862 2014-08-13 17:38:02.722870 7fe9bae887a0 20 read_log 191648'130 (0'0) modify c1783f91/benchmark_data_storage0111-ib.pocstor_40745_object7518/head//22 by client.273158.0:7519 2014-07-30 14:09:23.517134 2014-08-13 17:38:02.722882 7fe9bae887a0 20 read_log 191648'362 (0'0) modify d32a2f91/benchmark_data_storage0111-ib.pocstor_40745_object22333/head//22 by client.273158.0:22334 2014-07-30 14:10:46.067130 2014-08-13 17:38:02.722894 7fe9bae887a0 20 read_log 191727'373 (0'0) modify badb7f91/benchmark_data_storage0111-ib.pocstor_51285_object1837/head//22 by client.275175.0:1838 2014-07-30 14:18:40.367113 2014-08-13 17:38:02.722908 7fe9bae887a0 20 read_log 191727'375 (0'0) modify 110af791/benchmark_data_storage0111-ib.pocstor_51285_object3444/head//22 by client.275175.0:3445 2014-07-30 14:18:45.623214 2014-08-13 17:38:02.722920 7fe9bae887a0 20 read_log 191727'376 (0'0) modify 976a7791/benchmark_data_storage0111-ib.pocstor_51285_object4069/head//22 by client.275175.0:4070 2014-07-30 14:18:47.568962 2014-08-13 17:38:02.722932 7fe9bae887a0 20 read_log 191727'377 (0'0) modify 9a6bcf91/benchmark_data_storage0111-ib.pocstor_51285_object4125/head//22 by client.275175.0:4126 2014-07-30 14:18:47.741611 2014-08-13 17:38:02.722955 7fe9bae887a0 20 read_log 191727'378 (0'0) modify 45125791/benchmark_data_storage0111-ib.pocstor_51285_object4653/head//22 by client.275175.0:4654 2014-07-30 14:18:49.316676 2014-08-13 17:38:02.722971 7fe9bae887a0 20 read_log 191727'381 (0'0) modify f5483791/benchmark_data_storage0111-ib.pocstor_51285_object5709/head//22 by client.275175.0:5710 2014-07-30 14:18:52.414873 2014-08-13 17:38:02.722983 7fe9bae887a0 20 read_log 191727'384 (0'0) modify ab7df91/benchmark_data_storage0111-ib.pocstor_51285_object7282/head//22 by client.275175.0:7283 2014-07-30 14:18:57.518568 2014-08-13 17:38:02.722995 7fe9bae887a0 20 read_log 191727'386 (0'0) modify 4fc9f91/benchmark_data_storage0111-ib.pocstor_51285_object8150/head//22 by client.275175.0:8151 2014-07-30 14:18:59.942267 2014-08-13 17:38:02.723007 7fe9bae887a0 20 read_log 191727'396 (0'0) modify 1500e791/benchmark_data_storage0111-ib.pocstor_51285_object13316/head//22 by client.275175.0:13317 2014-07-30 14:19:15.586725 2014-08-13 17:38:02.723019 7fe9bae887a0 20 read_log 191727'398 (0'0) modify db9d3f91/benchmark_data_storage0111-ib.pocstor_51285_object14622/head//22 by client.275175.0:14623 2014-07-30 14:19:19.637423 2014-08-13 17:38:02.723030 7fe9bae887a0 20 read_log 191727'400 (0'0) modify b5261f91/benchmark_data_storage0111-ib.pocstor_51285_object16103/head//22 by client.275175.0:16104 2014-07-30 14:19:24.259928 2014-08-13 17:38:02.723042 7fe9bae887a0 20 read_log 191727'403 (0'0) modify 9d524791/benchmark_data_storage0111-ib.pocstor_51285_object16930/head//22 by client.275175.0:16931 2014-07-30 14:19:26.962855 2014-08-13 17:38:02.723054 7fe9bae887a0 20 read_log 191727'410 (0'0) modify bcf81791/benchmark_data_storage0111-ib.pocstor_51285_object19006/head//22 by client.275175.0:19007 2014-07-30 14:19:33.226863 2014-08-13 17:38:02.723066 7fe9bae887a0 20 read_log 191727'416 (0'0) modify 740d3f91/benchmark_data_storage0111-ib.pocstor_51285_object22673/head//22 by client.275175.0:22674 2014-07-30 14:19:44.993875 2014-08-13 17:38:02.723078 7fe9bae887a0 20 read_log 191727'421 (0'0) modify daaae791/benchmark_data_storage0111-ib.pocstor_51285_object25399/head//22 by client.275175.0:25400 2014-07-30 14:19:52.980564 2014-08-13 17:38:02.723089 7fe9bae887a0 20 read_log 191727'425 (0'0) modify ed5c0791/benchmark_data_storage0111-ib.pocstor_51285_object26739/head//22 by client.275175.0:26740 2014-07-30 14:19:57.027759 2014-08-13 17:38:02.723101 7fe9bae887a0 20 read_log 191727'430 (0'0) modify 8ae1cf91/benchmark_data_storage0111-ib.pocstor_51285_object29190/head//22 by client.275175.0:29191 2014-07-30 14:20:04.513232 2014-08-13 17:38:02.723113 7fe9bae887a0 20 read_log 191727'439 (0'0) modify dce47f91/benchmark_data_storage0111-ib.pocstor_51285_object34594/head//22 by client.275175.0:34595 2014-07-30 14:20:20.816333 2014-08-13 17:38:02.723125 7fe9bae887a0 20 read_log 191727'440 (0'0) modify 285adf91/benchmark_data_storage0111-ib.pocstor_51285_object35212/head//22 by client.275175.0:35213 2014-07-30 14:20:22.587266 2014-08-13 17:38:02.723137 7fe9bae887a0 20 read_log 191727'443 (0'0) modify 1b9ad791/benchmark_data_storage0111-ib.pocstor_51285_object35779/head//22 by client.275175.0:35780 2014-07-30 14:20:24.581943 2014-08-13 17:38:02.723149 7fe9bae887a0 20 read_log 191727'448 (0'0) modify 829a3f91/benchmark_data_storage0111-ib.pocstor_51285_object37867/head//22 by client.275175.0:37868 2014-07-30 14:20:30.868306 2014-08-13 17:38:02.723167 7fe9bae887a0 20 read_log 192674'449 (0'0) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1414 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723180 7fe9bae887a0 20 read_log 192674'450 (192674'449) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1415 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723192 7fe9bae887a0 20 read_log 192674'451 (192674'450) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1416 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723211 7fe9bae887a0 20 read_log 192674'452 (192674'451) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1417 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723223 7fe9bae887a0 20 read_log 192674'453 (192674'452) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1418 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723238 7fe9bae887a0 20 read_log 192674'454 (192674'453) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1419 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723250 7fe9bae887a0 20 read_log 192674'455 (192674'454) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1420 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723262 7fe9bae887a0 20 read_log 192674'456 (192674'455) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1421 2014-07-31 18:03:43.156000 2014-08-13 17:38:02.723274 7fe9bae887a0 20 read_log 192888'457 (0'0) modify 139ccf91/benchmark_data_storage0111-ib.pocstor_35625_object1200/head//22 by client.940330.0:1201 2014-08-11 16:29:45.538384 2014-08-13 17:38:02.723286 7fe9bae887a0 20 read_log 192888'458 (0'0) modify 70b79f91/benchmark_data_storage0111-ib.pocstor_35625_object7819/head//22 by client.940330.0:7820 2014-08-11 16:30:10.200604 2014-08-13 17:38:02.723298 7fe9bae887a0 20 read_log 192888'459 (0'0) modify a1c82f91/benchmark_data_storage0111-ib.pocstor_35625_object10305/head//22 by client.940330.0:10306 2014-08-11 16:30:18.974795 2014-08-13 17:38:02.723310 7fe9bae887a0 20 read_log 192888'460 (0'0) modify 82e97791/benchmark_data_storage0111-ib.pocstor_35625_object11477/head//22 by client.940330.0:11478 2014-08-11 16:30:23.704692 2014-08-13 17:38:02.723322 7fe9bae887a0 20 read_log 192888'461 (0'0) modify a06c6791/benchmark_data_storage0111-ib.pocstor_35625_object12714/head//22 by client.940330.0:12715 2014-08-11 16:30:28.691008 2014-08-13 17:38:02.723333 7fe9bae887a0 20 read_log 192888'462 (0'0) modify a2be6791/benchmark_data_storage0111-ib.pocstor_35625_object13953/head//22 by client.940330.0:13954 2014-08-11 16:30:33.466283 2014-08-13 17:38:02.723345 7fe9bae887a0 20 read_log 192888'463 (0'0) modify 99adf791/benchmark_data_storage0111-ib.pocstor_35625_object16306/head//22 by client.940330.0:16307 2014-08-11 16:30:42.373749 2014-08-13 17:38:02.723357 7fe9bae887a0 20 read_log 192888'464 (0'0) modify b0241791/benchmark_data_storage0111-ib.pocstor_35625_object19501/head//22 by client.940330.0:19502 2014-08-11 16:30:54.724858 2014-08-13 17:38:02.723369 7fe9bae887a0 20 read_log 192888'465 (0'0) modify efffbf91/benchmark_data_storage0111-ib.pocstor_35625_object20833/head//22 by client.940330.0:20834 2014-08-11 16:31:00.051004 2014-08-13 17:38:02.723381 7fe9bae887a0 20 read_log 192888'466 (0'0) modify 3f46791/benchmark_data_storage0111-ib.pocstor_35625_object21203/head//22 by client.940330.0:21204 2014-08-11 16:31:01.591759 2014-08-13 17:38:02.723393 7fe9bae887a0 20 read_log 192888'467 (0'0) modify 5b76f791/benchmark_data_storage0111-ib.pocstor_35625_object21469/head//22 by client.940330.0:21470 2014-08-11 16:31:02.566494 2014-08-13 17:38:02.723424 7fe9bae887a0 20 read_log 192888'468 (0'0) modify 4850c791/benchmark_data_storage0111-ib.pocstor_35625_object25594/head//22 by client.940330.0:25595 2014-08-11 16:31:18.202556 2014-08-13 17:38:02.723440 7fe9bae887a0 20 read_log 192888'469 (0'0) modify 22f57791/benchmark_data_storage0111-ib.pocstor_35625_object25976/head//22 by client.940330.0:25977 2014-08-11 16:31:19.698737 2014-08-13 17:38:02.723452 7fe9bae887a0 20 read_log 192888'470 (0'0) modify 57df1791/benchmark_data_storage0111-ib.pocstor_35625_object26789/head//22 by client.940330.0:26790 2014-08-11 16:31:22.989421 2014-08-13 17:38:02.723464 7fe9bae887a0 20 read_log 192888'471 (0'0) modify b6f4e791/benchmark_data_storage0111-ib.pocstor_35625_object26941/head//22 by client.940330.0:26942 2014-08-11 16:31:23.576652 2014-08-13 17:38:02.723483 7fe9bae887a0 20 read_log 192888'472 (0'0) modify 431a1f91/benchmark_data_storage0111-ib.pocstor_35625_object28219/head//22 by client.940330.0:28220 2014-08-11 16:31:28.580294 2014-08-13 17:38:02.723498 7fe9bae887a0 20 read_log 192888'473 (0'0) modify 10d69f91/benchmark_data_storage0111-ib.pocstor_35625_object28901/head//22 by client.940330.0:28902 2014-08-11 16:31:31.212324 2014-08-13 17:38:02.723510 7fe9bae887a0 20 read_log 192888'474 (0'0) modify 7f511791/benchmark_data_storage0111-ib.pocstor_35625_object30817/head//22 by client.940330.0:30818 2014-08-11 16:31:38.531910 2014-08-13 17:38:02.723522 7fe9bae887a0 20 read_log 192888'475 (192888'457) delete 139ccf91/benchmark_data_storage0111-ib.pocstor_35625_object1200/head//22 by client.940330.0:32508 2014-08-11 16:31:41.797572 2014-08-13 17:38:02.723553 7fe9bae887a0 20 read_log 192888'476 (192888'458) delete 70b79f91/benchmark_data_storage0111-ib.pocstor_35625_object7819/head//22 by client.940330.0:39127 2014-08-11 16:31:43.544852 2014-08-13 17:38:02.723566 7fe9bae887a0 20 read_log 192888'477 (192888'459) delete a1c82f91/benchmark_data_storage0111-ib.pocstor_35625_object10305/head//22 by client.940330.0:41613 2014-08-11 16:31:44.201391 2014-08-13 17:38:02.723578 7fe9bae887a0 20 read_log 192888'478 (192888'460) delete 82e97791/benchmark_data_storage0111-ib.pocstor_35625_object11477/head//22 by client.940330.0:42785 2014-08-11 16:31:44.535017 2014-08-13 17:38:02.723613 7fe9bae887a0 20 read_log 192888'479 (192888'461) delete a06c6791/benchmark_data_storage0111-ib.pocstor_35625_object12714/head//22 by client.940330.0:44022 2014-08-11 16:31:44.838569 2014-08-13 17:38:02.723626 7fe9bae887a0 20 read_log 192888'480 (192888'462) delete a2be6791/benchmark_data_storage0111-ib.pocstor_35625_object13953/head//22 by client.940330.0:45261 2014-08-11 16:31:45.170661 2014-08-13 17:38:02.723638 7fe9bae887a0 20 read_log 192888'481 (192888'463) delete 99adf791/benchmark_data_storage0111-ib.pocstor_35625_object16306/head//22 by client.940330.0:47614 2014-08-11 16:31:45.805416 2014-08-13 17:38:02.723650 7fe9bae887a0 20 read_log 192888'482 (192888'464) delete b0241791/benchmark_data_storage0111-ib.pocstor_35625_object19501/head//22 by client.940330.0:50809 2014-08-11 16:31:46.721006 2014-08-13 17:38:02.723662 7fe9bae887a0 20 read_log 192888'483 (192888'465) delete efffbf91/benchmark_data_storage0111-ib.pocstor_35625_object20833/head//22 by client.940330.0:52141 2014-08-11 16:31:47.115820 2014-08-13 17:38:02.723674 7fe9bae887a0 20 read_log 192888'484 (192888'466) delete 3f46791/benchmark_data_storage0111-ib.pocstor_35625_object21203/head//22 by client.940330.0:52511 2014-08-11 16:31:47.215150 2014-08-13 17:38:02.723686 7fe9bae887a0 20 read_log 192888'485 (192888'467) delete 5b76f791/benchmark_data_storage0111-ib.pocstor_35625_object21469/head//22 by client.940330.0:52777 2014-08-11 16:31:47.288636 2014-08-13 17:38:02.723698 7fe9bae887a0 20 read_log 192888'486 (192888'468) delete 4850c791/benchmark_data_storage0111-ib.pocstor_35625_object25594/head//22 by client.940330.0:56902 2014-08-11 16:31:48.529552 2014-08-13 17:38:02.723710 7fe9bae887a0 20 read_log 192888'487 (192888'469) delete 22f57791/benchmark_data_storage0111-ib.pocstor_35625_object25976/head//22 by client.940330.0:57284 2014-08-11 16:31:48.640579 2014-08-13 17:38:02.723722 7fe9bae887a0 20 read_log 192888'488 (192888'470) delete 57df1791/benchmark_data_storage0111-ib.pocstor_35625_object26789/head//22 by client.940330.0:58097 2014-08-11 16:31:48.870198 2014-08-13 17:38:02.723734 7fe9bae887a0 20 read_log 192888'489 (192888'471) delete b6f4e791/benchmark_data_storage0111-ib.pocstor_35625_object26941/head//22 by client.940330.0:58249 2014-08-11 16:31:48.912097 2014-08-13 17:38:02.723746 7fe9bae887a0 20 read_log 192888'490 (192888'472) delete 431a1f91/benchmark_data_storage0111-ib.pocstor_35625_object28219/head//22 by client.940330.0:59527 2014-08-11 16:31:49.264801 2014-08-13 17:38:02.723758 7fe9bae887a0 20 read_log 192888'491 (192888'473) delete 10d69f91/benchmark_data_storage0111-ib.pocstor_35625_object28901/head//22 by client.940330.0:60209 2014-08-11 16:31:49.446411 2014-08-13 17:38:02.723777 7fe9bae887a0 20 read_log 192888'492 (192888'474) delete 7f511791/benchmark_data_storage0111-ib.pocstor_35625_object30817/head//22 by client.940330.0:62125 2014-08-11 16:31:49.999423 2014-08-13 17:38:02.723789 7fe9bae887a0 20 read_log 194215'493 (0'0) modify 8a262791/benchmark_data_storage0111-ib.pocstor_64498_object3589/head//22 by client.1049635.0:3590 2014-08-12 17:17:28.771731 2014-08-13 17:38:02.723801 7fe9bae887a0 20 read_log 194215'494 (0'0) modify 9118a791/benchmark_data_storage0111-ib.pocstor_64498_object4103/head//22 by client.1049635.0:4104 2014-08-12 17:17:30.393766 2014-08-13 17:38:02.723813 7fe9bae887a0 20 read_log 194215'495 (0'0) modify 3ea80f91/benchmark_data_storage0111-ib.pocstor_64498_object8825/head//22 by client.1049635.0:8826 2014-08-12 17:17:44.525646 2014-08-13 17:38:02.723825 7fe9bae887a0 20 read_log 194215'496 (0'0) modify 6c896f91/benchmark_data_storage0111-ib.pocstor_64498_object9105/head//22 by client.1049635.0:9106 2014-08-12 17:17:45.323147 2014-08-13 17:38:02.723837 7fe9bae887a0 20 read_log 194215'497 (0'0) modify 2f23bf91/benchmark_data_storage0111-ib.pocstor_64498_object15443/head//22 by client.1049635.0:15444 2014-08-12 17:18:05.456863 2014-08-13 17:38:02.723850 7fe9bae887a0 20 read_log 194215'498 (0'0) modify 85e41f91/benchmark_data_storage0111-ib.pocstor_64498_object17017/head//22 by client.1049635.0:17018 2014-08-12 17:18:11.426990 2014-08-13 17:38:02.723862 7fe9bae887a0 20 read_log 194215'499 (0'0) modify 3c01df91/benchmark_data_storage0111-ib.pocstor_64498_object17633/head//22 by client.1049635.0:17634 2014-08-12 17:18:13.768857 2014-08-13 17:38:02.723874 7fe9bae887a0 20 read_log 194215'500 (0'0) modify fb1c6791/benchmark_data_storage0111-ib.pocstor_64498_object20467/head//22 by client.1049635.0:20468 2014-08-12 17:18:24.545801 2014-08-13 17:38:02.723886 7fe9bae887a0 20 read_log 194215'501 (0'0) modify 8a565791/benchmark_data_storage0111-ib.pocstor_64498_object20850/head//22 by client.1049635.0:20851 2014-08-12 17:18:25.964691 2014-08-13 17:38:02.723898 7fe9bae887a0 20 read_log 194215'502 (0'0) modify 7b045791/benchmark_data_storage0111-ib.pocstor_64498_object26311/head//22 by client.1049635.0:26312 2014-08-12 17:18:46.579387 2014-08-13 17:38:02.723910 7fe9bae887a0 20 read_log 194215'503 (0'0) modify e9d4c791/benchmark_data_storage0111-ib.pocstor_64498_object30275/head//22 by client.1049635.0:30276 2014-08-12 17:19:01.657233 2014-08-13 17:38:02.723922 7fe9bae887a0 20 read_log 194215'504 (0'0) modify a55c0f91/benchmark_data_storage0111-ib.pocstor_64498_object30969/head//22 by client.1049635.0:30970 2014-08-12 17:19:04.446525 2014-08-13 17:38:02.723935 7fe9bae887a0 20 read_log 194215'505 (0'0) modify e1d0ff91/benchmark_data_storage0111-ib.pocstor_64498_object31121/head//22 by client.1049635.0:31122 2014-08-12 17:19:05.030906 2014-08-13 17:38:02.723947 7fe9bae887a0 20 read_log 194215'506 (194215'493) delete 8a262791/benchmark_data_storage0111-ib.pocstor_64498_object3589/head//22 by client.1049635.0:37792 2014-08-12 17:19:19.145625 2014-08-13 17:38:02.723962 7fe9bae887a0 20 read_log 194215'507 (194215'494) delete 9118a791/benchmark_data_storage0111-ib.pocstor_64498_object4103/head//22 by client.1049635.0:38306 2014-08-12 17:19:19.307379 2014-08-13 17:38:02.723974 7fe9bae887a0 20 read_log 194215'508 (194215'495) delete 3ea80f91/benchmark_data_storage0111-ib.pocstor_64498_object8825/head//22 by client.1049635.0:43028 2014-08-12 17:19:20.869633 2014-08-13 17:38:02.723988 7fe9bae887a0 20 read_log 194215'509 (194215'496) delete 6c896f91/benchmark_data_storage0111-ib.pocstor_64498_object9105/head//22 by client.1049635.0:43308 2014-08-12 17:19:20.955008 2014-08-13 17:38:02.724001 7fe9bae887a0 20 read_log 194215'510 (194215'497) delete 2f23bf91/benchmark_data_storage0111-ib.pocstor_64498_object15443/head//22 by client.1049635.0:49646 2014-08-12 17:19:23.015177 2014-08-13 17:38:02.724017 7fe9bae887a0 20 read_log 194215'511 (194215'498) delete 85e41f91/benchmark_data_storage0111-ib.pocstor_64498_object17017/head//22 by client.1049635.0:51220 2014-08-12 17:19:23.513435 2014-08-13 17:38:02.724031 7fe9bae887a0 20 read_log 194215'512 (194215'499) delete 3c01df91/benchmark_data_storage0111-ib.pocstor_64498_object17633/head//22 by client.1049635.0:51836 2014-08-12 17:19:23.706481 2014-08-13 17:38:02.724043 7fe9bae887a0 20 read_log 194215'513 (194215'500) delete fb1c6791/benchmark_data_storage0111-ib.pocstor_64498_object20467/head//22 by client.1049635.0:54670 2014-08-12 17:19:24.692984 2014-08-13 17:38:02.724057 7fe9bae887a0 20 read_log 194215'514 (194215'501) delete 8a565791/benchmark_data_storage0111-ib.pocstor_64498_object20850/head//22 by client.1049635.0:55053 2014-08-12 17:19:24.834124 2014-08-13 17:38:02.724070 7fe9bae887a0 20 read_log 194215'515 (194215'502) delete 7b045791/benchmark_data_storage0111-ib.pocstor_64498_object26311/head//22 by client.1049635.0:60514 2014-08-12 17:19:26.762089 2014-08-13 17:38:02.724084 7fe9bae887a0 20 read_log 194215'516 (194215'503) delete e9d4c791/benchmark_data_storage0111-ib.pocstor_64498_object30275/head//22 by client.1049635.0:64478 2014-08-12 17:19:28.134152 2014-08-13 17:38:02.724098 7fe9bae887a0 20 read_log 194215'517 (194215'504) delete a55c0f91/benchmark_data_storage0111-ib.pocstor_64498_object30969/head//22 by client.1049635.0:65172 2014-08-12 17:19:28.371418 2014-08-13 17:38:02.724113 7fe9bae887a0 20 read_log 194215'518 (194215'505) delete e1d0ff91/benchmark_data_storage0111-ib.pocstor_64498_object31121/head//22 by client.1049635.0:65324 2014-08-12 17:19:28.423632 2014-08-13 17:38:02.724247 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.724269 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.724284 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.001690 0 0.000000 2014-08-13 17:38:02.724296 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.724304 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=0 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.724313 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] log((0'0,194215'518], crt=192888'492) 2014-08-13 17:38:02.724322 7fe9bae887a0 10 osd.4 194478 pgid 24.2c coll 24.2c_head 2014-08-13 17:38:02.724462 7fe9bae887a0 20 osd.4 0 get_map 194470 - loading and decoding 0x2746400 2014-08-13 17:38:02.724726 7fe9bae887a0 10 osd.4 0 add_map_bl 194470 268258 bytes 2014-08-13 17:38:02.726369 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.2c 2014-08-13 17:38:02.726384 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.726417 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] enter Initial 2014-08-13 17:38:02.726429 7fe9bae887a0 20 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] enter NotTrimming 2014-08-13 17:38:02.726582 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.726712 7fe9bae887a0 20 read_log 192834'13423 (192834'13423) modify 2c/hit_set_24.2c_archive_2014-08-11 10:24:43.594071_2014-08-11 15:23:38.213353/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:38.213751 2014-08-13 17:38:02.726735 7fe9bae887a0 20 read_log 192834'13424 (192718'12963) delete 2c/hit_set_24.2c_archive_2014-08-06 14:29:46.086849_2014-08-06 15:00:27.044493/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:38.213751 2014-08-13 17:38:02.726749 7fe9bae887a0 20 read_log 192834'13437 (0'0) promote bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:14886 2014-08-11 15:23:40.131592 2014-08-13 17:38:02.726762 7fe9bae887a0 20 read_log 192834'13438 (192834'13437) modify bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by client.933448.0:1152 2014-08-11 15:23:40.124130 2014-08-13 17:38:02.726775 7fe9bae887a0 20 read_log 192834'13449 (0'0) promote b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by osd.20.0:14898 2014-08-11 15:23:44.965483 2014-08-13 17:38:02.726787 7fe9bae887a0 20 read_log 192834'13450 (192834'13449) modify b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.933448.0:2102 2014-08-11 15:23:44.958283 2014-08-13 17:38:02.726799 7fe9bae887a0 20 read_log 192834'13459 (0'0) promote b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by osd.20.0:14908 2014-08-11 15:23:49.940768 2014-08-13 17:38:02.726811 7fe9bae887a0 20 read_log 192834'13460 (192834'13459) modify b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.933448.0:3049 2014-08-11 15:23:49.933885 2014-08-13 17:38:02.726823 7fe9bae887a0 20 read_log 192834'13461 (0'0) promote 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by osd.20.0:14910 2014-08-11 15:23:52.013558 2014-08-13 17:38:02.726835 7fe9bae887a0 20 read_log 192834'13462 (192834'13461) modify 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.933448.0:3487 2014-08-11 15:23:52.004988 2014-08-13 17:38:02.726848 7fe9bae887a0 20 read_log 192834'13471 (0'0) promote 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:14920 2014-08-11 15:23:54.439774 2014-08-13 17:38:02.726860 7fe9bae887a0 20 read_log 192834'13472 (192834'13471) modify 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by client.933448.0:3907 2014-08-11 15:23:54.431556 2014-08-13 17:38:02.726872 7fe9bae887a0 20 read_log 192834'13475 (0'0) promote 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by osd.20.0:14924 2014-08-11 15:23:55.915896 2014-08-13 17:38:02.726884 7fe9bae887a0 20 read_log 192834'13476 (192834'13475) modify 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.933448.0:4213 2014-08-11 15:23:55.907164 2014-08-13 17:38:02.726897 7fe9bae887a0 20 read_log 192834'13481 (0'0) promote 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by osd.20.0:14930 2014-08-11 15:23:57.596052 2014-08-13 17:38:02.726909 7fe9bae887a0 20 read_log 192834'13482 (192834'13481) modify 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.933448.0:4559 2014-08-11 15:23:57.587823 2014-08-13 17:38:02.726929 7fe9bae887a0 20 read_log 192834'13495 (0'0) promote c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by osd.20.0:14944 2014-08-11 15:24:04.133269 2014-08-13 17:38:02.726944 7fe9bae887a0 20 read_log 192834'13496 (192834'13495) modify c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.933448.0:6105 2014-08-11 15:24:04.125430 2014-08-13 17:38:02.726957 7fe9bae887a0 20 read_log 192834'13529 (0'0) promote 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:14978 2014-08-11 15:24:12.004496 2014-08-13 17:38:02.726969 7fe9bae887a0 20 read_log 192834'13530 (192834'13529) modify 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by client.933448.0:7946 2014-08-11 15:24:11.996307 2014-08-13 17:38:02.726982 7fe9bae887a0 20 read_log 192834'13531 (0'0) promote c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by osd.20.0:14980 2014-08-11 15:24:12.819542 2014-08-13 17:38:02.726993 7fe9bae887a0 20 read_log 192834'13532 (192834'13531) modify c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.933448.0:8123 2014-08-11 15:24:12.810969 2014-08-13 17:38:02.727006 7fe9bae887a0 20 read_log 192834'13541 (0'0) promote 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by osd.20.0:14990 2014-08-11 15:24:17.843326 2014-08-13 17:38:02.727017 7fe9bae887a0 20 read_log 192834'13542 (192834'13541) modify 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.933448.0:9178 2014-08-11 15:24:17.835345 2014-08-13 17:38:02.727030 7fe9bae887a0 20 read_log 192834'13555 (0'0) promote 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by osd.20.0:15004 2014-08-11 15:24:22.438019 2014-08-13 17:38:02.727042 7fe9bae887a0 20 read_log 192834'13556 (192834'13555) modify 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.933448.0:10293 2014-08-11 15:24:22.428624 2014-08-13 17:38:02.727054 7fe9bae887a0 20 read_log 192834'13557 (0'0) promote 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15006 2014-08-11 15:24:22.536695 2014-08-13 17:38:02.727066 7fe9bae887a0 20 read_log 192834'13558 (192834'13557) modify 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by client.933448.0:10314 2014-08-11 15:24:22.527540 2014-08-13 17:38:02.727078 7fe9bae887a0 20 read_log 192834'13571 (0'0) promote 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by osd.20.0:15020 2014-08-11 15:24:27.431954 2014-08-13 17:38:02.727090 7fe9bae887a0 20 read_log 192834'13572 (192834'13571) modify 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.933448.0:11361 2014-08-11 15:24:27.422655 2014-08-13 17:38:02.727102 7fe9bae887a0 20 read_log 192834'13585 (0'0) promote cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by osd.20.0:15034 2014-08-11 15:24:29.656490 2014-08-13 17:38:02.727114 7fe9bae887a0 20 read_log 192834'13586 (192834'13585) modify cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.933448.0:11858 2014-08-11 15:24:29.649098 2014-08-13 17:38:02.727126 7fe9bae887a0 20 read_log 192834'13593 (0'0) promote 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:15042 2014-08-11 15:24:31.521354 2014-08-13 17:38:02.727138 7fe9bae887a0 20 read_log 192834'13594 (192834'13593) modify 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by client.933448.0:12214 2014-08-11 15:24:31.510472 2014-08-13 17:38:02.727150 7fe9bae887a0 20 read_log 192834'13607 (0'0) promote a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by osd.20.0:15056 2014-08-11 15:24:33.666961 2014-08-13 17:38:02.727171 7fe9bae887a0 20 read_log 192834'13608 (192834'13607) modify a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.933448.0:12737 2014-08-11 15:24:33.657343 2014-08-13 17:38:02.727194 7fe9bae887a0 20 read_log 192834'13625 (0'0) promote d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by osd.20.0:15074 2014-08-11 15:24:40.270540 2014-08-13 17:38:02.727209 7fe9bae887a0 20 read_log 192834'13626 (192834'13625) modify d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.933448.0:13872 2014-08-11 15:24:40.261490 2014-08-13 17:38:02.727222 7fe9bae887a0 20 read_log 192834'13629 (0'0) promote fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15078 2014-08-11 15:24:41.745306 2014-08-13 17:38:02.727255 7fe9bae887a0 20 read_log 192834'13630 (192834'13629) modify fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by client.933448.0:14154 2014-08-11 15:24:41.734143 2014-08-13 17:38:02.727270 7fe9bae887a0 20 read_log 192834'13639 (0'0) promote 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by osd.20.0:15088 2014-08-11 15:24:46.657282 2014-08-13 17:38:02.727282 7fe9bae887a0 20 read_log 192834'13640 (192834'13639) modify 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.933448.0:14669 2014-08-11 15:24:46.648684 2014-08-13 17:38:02.727295 7fe9bae887a0 20 read_log 192834'13651 (0'0) promote 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by osd.20.0:15100 2014-08-11 15:24:50.030814 2014-08-13 17:38:02.727309 7fe9bae887a0 20 read_log 192834'13652 (192834'13651) modify 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.933448.0:15539 2014-08-11 15:24:50.022585 2014-08-13 17:38:02.727322 7fe9bae887a0 20 read_log 192834'13665 (0'0) promote bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by osd.20.0:15114 2014-08-11 15:24:52.730496 2014-08-13 17:38:02.727336 7fe9bae887a0 20 read_log 192834'13666 (192834'13665) modify bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.933448.0:16065 2014-08-11 15:24:52.718086 2014-08-13 17:38:02.727349 7fe9bae887a0 20 read_log 192834'13667 (0'0) promote 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by osd.20.0:15116 2014-08-11 15:24:53.406304 2014-08-13 17:38:02.727364 7fe9bae887a0 20 read_log 192834'13668 (192834'13667) modify 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.933448.0:16174 2014-08-11 15:24:53.396511 2014-08-13 17:38:02.727378 7fe9bae887a0 20 read_log 192834'13669 (0'0) promote 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15118 2014-08-11 15:24:54.173432 2014-08-13 17:38:02.727390 7fe9bae887a0 20 read_log 192834'13670 (192834'13669) modify 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by client.933448.0:16339 2014-08-11 15:24:54.163611 2014-08-13 17:38:02.727402 7fe9bae887a0 20 read_log 192834'13673 (0'0) promote 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:15122 2014-08-11 15:24:54.336992 2014-08-13 17:38:02.727414 7fe9bae887a0 20 read_log 192834'13674 (192834'13673) modify 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by client.933448.0:16363 2014-08-11 15:24:54.327759 2014-08-13 17:38:02.727426 7fe9bae887a0 20 read_log 192834'13689 (0'0) promote 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by osd.20.0:15138 2014-08-11 15:24:58.665082 2014-08-13 17:38:02.727439 7fe9bae887a0 20 read_log 192834'13690 (192834'13689) modify 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.933448.0:17159 2014-08-11 15:24:58.657118 2014-08-13 17:38:02.727453 7fe9bae887a0 20 read_log 192834'13697 (0'0) promote e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15146 2014-08-11 15:24:59.517027 2014-08-13 17:38:02.727465 7fe9bae887a0 20 read_log 192834'13698 (192834'13697) modify e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by client.933448.0:17388 2014-08-11 15:24:59.508471 2014-08-13 17:38:02.727484 7fe9bae887a0 20 read_log 192834'13701 (0'0) promote 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by osd.20.0:15150 2014-08-11 15:25:00.989399 2014-08-13 17:38:02.727499 7fe9bae887a0 20 read_log 192834'13702 (192834'13701) modify 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.933448.0:17673 2014-08-11 15:25:00.978745 2014-08-13 17:38:02.727511 7fe9bae887a0 20 read_log 192834'13719 (0'0) promote 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by osd.20.0:15168 2014-08-11 15:25:11.461623 2014-08-13 17:38:02.727525 7fe9bae887a0 20 read_log 192834'13720 (192834'13719) modify 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.933448.0:19320 2014-08-11 15:25:11.452356 2014-08-13 17:38:02.727539 7fe9bae887a0 20 read_log 192834'13743 (0'0) promote 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by osd.20.0:15192 2014-08-11 15:25:35.166266 2014-08-13 17:38:02.727551 7fe9bae887a0 20 read_log 192834'13744 (192834'13743) modify 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.933448.0:22235 2014-08-11 15:25:35.156244 2014-08-13 17:38:02.727563 7fe9bae887a0 20 read_log 192834'13753 (192834'13438) modify bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by client.933448.0:23443 2014-08-11 15:25:39.356068 2014-08-13 17:38:02.727577 7fe9bae887a0 20 read_log 192834'13759 (192834'13450) modify b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.933448.0:24393 2014-08-11 15:25:40.750615 2014-08-13 17:38:02.727590 7fe9bae887a0 20 read_log 192834'13764 (192834'13460) modify b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.933448.0:25340 2014-08-11 15:25:42.015264 2014-08-13 17:38:02.727604 7fe9bae887a0 20 read_log 192834'13765 (192834'13462) modify 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.933448.0:25778 2014-08-11 15:25:42.434633 2014-08-13 17:38:02.727616 7fe9bae887a0 20 read_log 192834'13770 (192834'13472) modify 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by client.933448.0:26198 2014-08-11 15:25:43.103178 2014-08-13 17:38:02.727631 7fe9bae887a0 20 read_log 192834'13772 (192834'13476) modify 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.933448.0:26504 2014-08-11 15:25:43.468688 2014-08-13 17:38:02.727643 7fe9bae887a0 20 read_log 192834'13775 (192834'13482) modify 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.933448.0:26850 2014-08-11 15:25:43.815289 2014-08-13 17:38:02.727655 7fe9bae887a0 20 read_log 192834'13782 (192834'13496) modify c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.933448.0:28396 2014-08-11 15:25:46.131223 2014-08-13 17:38:02.727667 7fe9bae887a0 20 read_log 192834'13799 (192834'13530) modify 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by client.933448.0:30237 2014-08-11 15:25:48.623926 2014-08-13 17:38:02.727681 7fe9bae887a0 20 read_log 192834'13800 (192834'13532) modify c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.933448.0:30414 2014-08-11 15:25:48.821373 2014-08-13 17:38:02.727694 7fe9bae887a0 20 read_log 192834'13805 (192834'13542) modify 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.933448.0:31469 2014-08-11 15:25:50.098570 2014-08-13 17:38:02.727706 7fe9bae887a0 20 read_log 192834'13812 (192834'13556) modify 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.933448.0:32584 2014-08-11 15:25:52.046910 2014-08-13 17:38:02.727718 7fe9bae887a0 20 read_log 192834'13813 (192834'13558) modify 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by client.933448.0:32605 2014-08-11 15:25:52.074978 2014-08-13 17:38:02.727730 7fe9bae887a0 20 read_log 192834'13820 (192834'13572) modify 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.933448.0:33652 2014-08-11 15:25:53.915063 2014-08-13 17:38:02.727750 7fe9bae887a0 20 read_log 192834'13827 (192834'13586) modify cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.933448.0:34149 2014-08-11 15:25:54.641298 2014-08-13 17:38:02.727764 7fe9bae887a0 20 read_log 192834'13831 (192834'13594) modify 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by client.933448.0:34505 2014-08-11 15:25:55.091364 2014-08-13 17:38:02.727777 7fe9bae887a0 20 read_log 192834'13838 (192834'13608) modify a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.933448.0:35028 2014-08-11 15:25:55.720319 2014-08-13 17:38:02.727789 7fe9bae887a0 20 read_log 192834'13847 (192834'13626) modify d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.933448.0:36163 2014-08-11 15:25:57.313549 2014-08-13 17:38:02.727803 7fe9bae887a0 20 read_log 192834'13849 (192834'13630) modify fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by client.933448.0:36445 2014-08-11 15:25:57.735095 2014-08-13 17:38:02.727815 7fe9bae887a0 20 read_log 192834'13854 (192834'13640) modify 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.933448.0:36960 2014-08-11 15:25:58.575684 2014-08-13 17:38:02.727827 7fe9bae887a0 20 read_log 192834'13860 (192834'13652) modify 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.933448.0:37830 2014-08-11 15:26:00.125055 2014-08-13 17:38:02.727841 7fe9bae887a0 20 read_log 192834'13867 (192834'13666) modify bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.933448.0:38356 2014-08-11 15:26:00.808258 2014-08-13 17:38:02.727853 7fe9bae887a0 20 read_log 192834'13868 (192834'13668) modify 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.933448.0:38465 2014-08-11 15:26:00.895984 2014-08-13 17:38:02.727881 7fe9bae887a0 20 read_log 192834'13869 (192834'13670) modify 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by client.933448.0:38630 2014-08-11 15:26:01.075093 2014-08-13 17:38:02.727897 7fe9bae887a0 20 read_log 192834'13871 (192834'13674) modify 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by client.933448.0:38654 2014-08-11 15:26:01.107710 2014-08-13 17:38:02.727911 7fe9bae887a0 20 read_log 192834'13879 (192834'13690) modify 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.933448.0:39450 2014-08-11 15:26:01.886662 2014-08-13 17:38:02.727923 7fe9bae887a0 20 read_log 192834'13883 (192834'13698) modify e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by client.933448.0:39679 2014-08-11 15:26:02.026322 2014-08-13 17:38:02.727935 7fe9bae887a0 20 read_log 192834'13885 (192834'13702) modify 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.933448.0:39964 2014-08-11 15:26:02.189860 2014-08-13 17:38:02.727947 7fe9bae887a0 20 read_log 192834'13894 (192834'13720) modify 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.933448.0:41611 2014-08-11 15:26:02.997683 2014-08-13 17:38:02.727959 7fe9bae887a0 20 read_log 192834'13906 (192834'13744) modify 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.933448.0:44526 2014-08-11 15:26:03.975678 2014-08-13 17:38:02.727971 7fe9bae887a0 20 read_log 192834'13912 (0'0) promote f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15361 2014-08-11 15:26:15.957866 2014-08-13 17:38:02.727985 7fe9bae887a0 20 read_log 192834'13913 (192834'13912) modify f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by client.1883634.0:399 2014-08-11 15:26:15.950165 2014-08-13 17:38:02.727998 7fe9bae887a0 20 read_log 192834'13924 (0'0) promote 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by osd.20.0:15373 2014-08-11 15:26:17.336723 2014-08-13 17:38:02.728012 7fe9bae887a0 20 read_log 192834'13925 (192834'13924) modify 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.1883634.0:780 2014-08-11 15:26:17.325386 2014-08-13 17:38:02.728035 7fe9bae887a0 20 read_log 192834'13926 (0'0) promote 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15375 2014-08-11 15:26:17.587906 2014-08-13 17:38:02.728050 7fe9bae887a0 20 read_log 192834'13927 (192834'13926) modify 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by client.1883634.0:850 2014-08-11 15:26:17.577891 2014-08-13 17:38:02.728063 7fe9bae887a0 20 read_log 192834'13976 (0'0) promote 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by osd.20.0:15425 2014-08-11 15:26:30.182370 2014-08-13 17:38:02.728075 7fe9bae887a0 20 read_log 192834'13977 (192834'13976) modify 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.1883634.0:4335 2014-08-11 15:26:30.174160 2014-08-13 17:38:02.728090 7fe9bae887a0 20 read_log 192834'13978 (0'0) promote c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15427 2014-08-11 15:26:30.197686 2014-08-13 17:38:02.728102 7fe9bae887a0 20 read_log 192834'13979 (192834'13978) modify c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by client.1883634.0:4340 2014-08-11 15:26:30.190365 2014-08-13 17:38:02.728116 7fe9bae887a0 20 read_log 192834'14078 (0'0) promote 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15527 2014-08-11 15:26:53.540340 2014-08-13 17:38:02.728132 7fe9bae887a0 20 read_log 192834'14079 (192834'14078) modify 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by client.1883634.0:10057 2014-08-11 15:26:53.531842 2014-08-13 17:38:02.728144 7fe9bae887a0 20 read_log 192834'14080 (0'0) promote d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by osd.20.0:15529 2014-08-11 15:26:53.875358 2014-08-13 17:38:02.728163 7fe9bae887a0 20 read_log 192834'14081 (192834'14080) modify d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.1883634.0:10152 2014-08-11 15:26:53.868108 2014-08-13 17:38:02.728176 7fe9bae887a0 20 read_log 192834'14114 (0'0) promote 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by osd.20.0:15563 2014-08-11 15:27:01.006122 2014-08-13 17:38:02.728190 7fe9bae887a0 20 read_log 192834'14115 (192834'14114) modify 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.1883634.0:11696 2014-08-11 15:27:00.996432 2014-08-13 17:38:02.728203 7fe9bae887a0 20 read_log 192834'14148 (0'0) promote 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by osd.20.0:15597 2014-08-11 15:27:11.751591 2014-08-13 17:38:02.728215 7fe9bae887a0 20 read_log 192834'14149 (192834'14148) modify 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.1883634.0:14052 2014-08-11 15:27:11.742603 2014-08-13 17:38:02.728227 7fe9bae887a0 20 read_log 192834'14164 (0'0) promote 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by osd.20.0:15613 2014-08-11 15:27:15.740326 2014-08-13 17:38:02.728240 7fe9bae887a0 20 read_log 192834'14165 (192834'14164) modify 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.1883634.0:14862 2014-08-11 15:27:15.731500 2014-08-13 17:38:02.728252 7fe9bae887a0 20 read_log 192834'14172 (0'0) promote 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by osd.20.0:15621 2014-08-11 15:27:17.651940 2014-08-13 17:38:02.728264 7fe9bae887a0 20 read_log 192834'14173 (192834'14172) modify 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.1883634.0:15203 2014-08-11 15:27:17.642707 2014-08-13 17:38:02.728278 7fe9bae887a0 20 read_log 192834'14194 (0'0) promote 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by osd.20.0:15643 2014-08-11 15:27:27.821265 2014-08-13 17:38:02.728290 7fe9bae887a0 20 read_log 192834'14195 (192834'14194) modify 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.1883634.0:16494 2014-08-11 15:27:27.811592 2014-08-13 17:38:02.728329 7fe9bae887a0 20 read_log 192834'14196 (0'0) promote 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:15645 2014-08-11 15:27:28.115025 2014-08-13 17:38:02.728345 7fe9bae887a0 20 read_log 192834'14197 (192834'14196) modify 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by client.1883634.0:16530 2014-08-11 15:27:28.104925 2014-08-13 17:38:02.728358 7fe9bae887a0 20 read_log 192834'14222 (0'0) promote 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:15671 2014-08-11 15:27:49.212801 2014-08-13 17:38:02.728371 7fe9bae887a0 20 read_log 192834'14223 (192834'14222) modify 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by client.1883634.0:19067 2014-08-11 15:27:49.203240 2014-08-13 17:38:02.728385 7fe9bae887a0 20 read_log 192834'14226 (0'0) promote ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:15675 2014-08-11 15:27:51.244259 2014-08-13 17:38:02.728397 7fe9bae887a0 20 read_log 192834'14227 (192834'14226) modify ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by client.1883634.0:19320 2014-08-11 15:27:51.235439 2014-08-13 17:38:02.728409 7fe9bae887a0 20 read_log 192834'14238 (0'0) promote a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by osd.20.0:15687 2014-08-11 15:27:53.627656 2014-08-13 17:38:02.728423 7fe9bae887a0 20 read_log 192834'14239 (192834'14238) modify a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.1883634.0:19638 2014-08-11 15:27:53.616912 2014-08-13 17:38:02.728436 7fe9bae887a0 20 read_log 192834'14250 (0'0) promote a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:15699 2014-08-11 15:27:56.885319 2014-08-13 17:38:02.728447 7fe9bae887a0 20 read_log 192834'14251 (192834'14250) modify a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by client.1883634.0:20431 2014-08-11 15:27:56.875240 2014-08-13 17:38:02.728461 7fe9bae887a0 20 read_log 192834'14274 (0'0) promote 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by osd.20.0:15723 2014-08-11 15:28:04.979843 2014-08-13 17:38:02.728473 7fe9bae887a0 20 read_log 192834'14275 (192834'14274) modify 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.1883634.0:21691 2014-08-11 15:28:04.968934 2014-08-13 17:38:02.728485 7fe9bae887a0 20 read_log 192834'14292 (192834'13813) clean 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15741 2014-08-11 15:28:10.703623 2014-08-13 17:38:02.728498 7fe9bae887a0 20 read_log 192834'14293 (192834'13753) clean bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:15742 2014-08-11 15:28:10.707760 2014-08-13 17:38:02.728510 7fe9bae887a0 20 read_log 192834'14294 (192834'13979) clean c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15743 2014-08-11 15:28:10.770799 2014-08-13 17:38:02.728522 7fe9bae887a0 20 read_log 192834'14295 (192834'13849) clean fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15744 2014-08-11 15:28:10.780297 2014-08-13 17:38:02.728550 7fe9bae887a0 20 read_log 192834'14296 (192834'13883) clean e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15745 2014-08-11 15:28:10.801151 2014-08-13 17:38:02.728565 7fe9bae887a0 20 read_log 192834'14297 (192834'13799) clean 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:15746 2014-08-11 15:28:10.805351 2014-08-13 17:38:02.728578 7fe9bae887a0 20 read_log 192834'14298 (192834'13869) clean 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15747 2014-08-11 15:28:10.834660 2014-08-13 17:38:02.728589 7fe9bae887a0 20 read_log 192834'14299 (192834'13913) clean f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15748 2014-08-11 15:28:10.854708 2014-08-13 17:38:02.728609 7fe9bae887a0 20 read_log 192834'14300 (192834'13770) clean 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:15749 2014-08-11 15:28:11.064491 2014-08-13 17:38:02.728624 7fe9bae887a0 20 read_log 192834'14301 (192834'14079) clean 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15750 2014-08-11 15:28:11.064817 2014-08-13 17:38:02.728639 7fe9bae887a0 20 read_log 192834'14303 (192834'13927) clean 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15752 2014-08-11 15:28:11.179010 2014-08-13 17:38:02.728651 7fe9bae887a0 20 read_log 192834'14311 (192834'14292) delete 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15760 2014-08-11 15:28:11.529122 2014-08-13 17:38:02.728663 7fe9bae887a0 20 read_log 192834'14312 (192834'14294) delete c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15761 2014-08-11 15:28:11.529584 2014-08-13 17:38:02.728676 7fe9bae887a0 20 read_log 192834'14313 (192834'14293) delete bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:15762 2014-08-11 15:28:11.533848 2014-08-13 17:38:02.728688 7fe9bae887a0 20 read_log 192834'14314 (192834'14299) delete f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15763 2014-08-11 15:28:11.534422 2014-08-13 17:38:02.728700 7fe9bae887a0 20 read_log 192834'14316 (192834'14295) delete fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15765 2014-08-11 15:28:11.540842 2014-08-13 17:38:02.728711 7fe9bae887a0 20 read_log 192834'14317 (192834'14296) delete e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15766 2014-08-11 15:28:11.541287 2014-08-13 17:38:02.728723 7fe9bae887a0 20 read_log 192834'14318 (192834'14301) delete 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15767 2014-08-11 15:28:11.546450 2014-08-13 17:38:02.728736 7fe9bae887a0 20 read_log 192834'14319 (192834'14297) delete 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:15768 2014-08-11 15:28:11.546881 2014-08-13 17:38:02.728748 7fe9bae887a0 20 read_log 192834'14321 (192834'14298) delete 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15770 2014-08-11 15:28:11.551686 2014-08-13 17:38:02.728760 7fe9bae887a0 20 read_log 192834'14322 (192834'14303) delete 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15771 2014-08-11 15:28:11.552063 2014-08-13 17:38:02.728772 7fe9bae887a0 20 read_log 192834'14323 (192834'14300) delete 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:15772 2014-08-11 15:28:11.556394 2014-08-13 17:38:02.728785 7fe9bae887a0 20 read_log 192834'14335 (0'0) promote a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by osd.20.0:15784 2014-08-11 15:28:12.106564 2014-08-13 17:38:02.728798 7fe9bae887a0 20 read_log 192834'14336 (192834'14335) modify a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.1883634.0:23021 2014-08-11 15:28:12.097108 2014-08-13 17:38:02.728810 7fe9bae887a0 20 read_log 192834'14345 (0'0) promote f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15794 2014-08-11 15:28:17.626691 2014-08-13 17:38:02.728822 7fe9bae887a0 20 read_log 192834'14346 (192834'14345) modify f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by client.1883634.0:23723 2014-08-11 15:28:17.601677 2014-08-13 17:38:02.728834 7fe9bae887a0 20 read_log 192834'14352 (192834'13925) modify 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.1883634.0:24104 2014-08-11 15:28:18.361842 2014-08-13 17:38:02.728846 7fe9bae887a0 20 read_log 192834'14353 (0'0) promote 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15802 2014-08-11 15:28:18.462067 2014-08-13 17:38:02.728868 7fe9bae887a0 20 read_log 192834'14354 (192834'14353) modify 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by client.1883634.0:24174 2014-08-11 15:28:18.440741 2014-08-13 17:38:02.728884 7fe9bae887a0 20 read_log 192834'14379 (192834'13977) modify 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.1883634.0:27659 2014-08-11 15:28:24.752531 2014-08-13 17:38:02.728897 7fe9bae887a0 20 read_log 192834'14380 (0'0) promote c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15829 2014-08-11 15:28:24.792537 2014-08-13 17:38:02.728908 7fe9bae887a0 20 read_log 192834'14381 (192834'14380) modify c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by client.1883634.0:27664 2014-08-11 15:28:24.770652 2014-08-13 17:38:02.728920 7fe9bae887a0 20 read_log 192834'14436 (0'0) promote 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15885 2014-08-11 15:28:33.111085 2014-08-13 17:38:02.728932 7fe9bae887a0 20 read_log 192834'14437 (192834'14436) modify 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by client.1883634.0:33381 2014-08-11 15:28:33.085247 2014-08-13 17:38:02.728944 7fe9bae887a0 20 read_log 192834'14438 (192834'14081) modify d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.1883634.0:33476 2014-08-11 15:28:33.251623 2014-08-13 17:38:02.728957 7fe9bae887a0 20 read_log 192834'14453 (192834'14453) modify 2c/hit_set_24.2c_archive_2014-08-11 15:23:38.213813_2014-08-11 15:28:35.729117/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:35.729424 2014-08-13 17:38:02.728969 7fe9bae887a0 20 read_log 192834'14454 (192834'13423) delete 2c/hit_set_24.2c_archive_2014-08-11 10:24:43.594071_2014-08-11 15:23:38.213353/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:35.729424 2014-08-13 17:38:02.728982 7fe9bae887a0 20 read_log 192834'14457 (192834'14115) modify 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.1883634.0:35020 2014-08-11 15:28:35.856612 2014-08-13 17:38:02.728994 7fe9bae887a0 20 read_log 192834'14474 (192834'14149) modify 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.1883634.0:37376 2014-08-11 15:28:39.248162 2014-08-13 17:38:02.729006 7fe9bae887a0 20 read_log 192834'14482 (192834'14165) modify 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.1883634.0:38186 2014-08-11 15:28:40.335473 2014-08-13 17:38:02.729018 7fe9bae887a0 20 read_log 192834'14486 (192834'14173) modify 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.1883634.0:38527 2014-08-11 15:28:40.770378 2014-08-13 17:38:02.729031 7fe9bae887a0 20 read_log 192834'14497 (192834'14195) modify 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.1883634.0:39818 2014-08-11 15:28:42.609669 2014-08-13 17:38:02.729044 7fe9bae887a0 20 read_log 192834'14498 (192834'14197) modify 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by client.1883634.0:39854 2014-08-11 15:28:42.666827 2014-08-13 17:38:02.729056 7fe9bae887a0 20 read_log 192834'14511 (192834'14223) modify 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by client.1883634.0:42391 2014-08-11 15:28:46.228290 2014-08-13 17:38:02.729067 7fe9bae887a0 20 read_log 192834'14513 (192834'14227) modify ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by client.1883634.0:42644 2014-08-11 15:28:46.309661 2014-08-13 17:38:02.729079 7fe9bae887a0 20 read_log 192834'14519 (192834'14239) modify a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.1883634.0:42962 2014-08-11 15:28:46.415267 2014-08-13 17:38:02.729091 7fe9bae887a0 20 read_log 192834'14525 (192834'14251) modify a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by client.1883634.0:43755 2014-08-11 15:28:46.653316 2014-08-13 17:38:02.729135 7fe9bae887a0 20 read_log 192834'14537 (192834'14275) modify 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.1883634.0:45015 2014-08-11 15:28:47.122151 2014-08-13 17:38:02.729150 7fe9bae887a0 20 read_log 192834'14548 (192834'14336) modify a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.1883634.0:46345 2014-08-11 15:28:47.563588 2014-08-13 17:38:02.729169 7fe9bae887a0 20 read_log 192888'14552 (192888'14552) modify 2c/hit_set_24.2c_archive_2014-08-11 16:02:02.168428_2014-08-11 16:44:57.627389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.627917 2014-08-13 17:38:02.729182 7fe9bae887a0 20 read_log 192888'14553 (192834'14453) delete 2c/hit_set_24.2c_archive_2014-08-11 15:23:38.213813_2014-08-11 15:28:35.729117/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.627917 2014-08-13 17:38:02.729194 7fe9bae887a0 20 read_log 192888'14554 (0'0) promote 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:16314 2014-08-11 16:44:57.632293 2014-08-13 17:38:02.729206 7fe9bae887a0 20 read_log 192888'14555 (192888'14554) modify 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by client.1315259.0:613 2014-08-11 16:44:57.599462 2014-08-13 17:38:02.729218 7fe9bae887a0 20 read_log 192888'14558 (0'0) promote 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:16325 2014-08-11 16:45:02.951673 2014-08-13 17:38:02.729230 7fe9bae887a0 20 read_log 192888'14559 (192888'14558) modify 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by client.1315259.0:2021 2014-08-11 16:45:02.943705 2014-08-13 17:38:02.729243 7fe9bae887a0 20 read_log 192888'14560 (0'0) promote 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:16329 2014-08-11 16:45:04.059429 2014-08-13 17:38:02.729255 7fe9bae887a0 20 read_log 192888'14561 (192888'14560) modify 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by client.1315259.0:2321 2014-08-11 16:45:04.051921 2014-08-13 17:38:02.729267 7fe9bae887a0 20 read_log 192888'14566 (0'0) promote f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:16369 2014-08-11 16:45:17.809009 2014-08-13 17:38:02.729279 7fe9bae887a0 20 read_log 192888'14567 (192888'14566) modify f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by client.1315259.0:6007 2014-08-11 16:45:17.800013 2014-08-13 17:38:02.729291 7fe9bae887a0 20 read_log 192888'14568 (0'0) promote d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by osd.20.0:16373 2014-08-11 16:45:19.521818 2014-08-13 17:38:02.729305 7fe9bae887a0 20 read_log 192888'14569 (192888'14568) modify d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.1315259.0:6481 2014-08-11 16:45:19.513337 2014-08-13 17:38:02.729317 7fe9bae887a0 20 read_log 192888'14574 (0'0) promote 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by osd.20.0:16389 2014-08-11 16:45:23.849899 2014-08-13 17:38:02.729329 7fe9bae887a0 20 read_log 192888'14575 (192888'14574) modify 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.1315259.0:7680 2014-08-11 16:45:23.841994 2014-08-13 17:38:02.729342 7fe9bae887a0 20 read_log 192888'14578 (0'0) promote 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by osd.20.0:16397 2014-08-11 16:45:25.589587 2014-08-13 17:38:02.729354 7fe9bae887a0 20 read_log 192888'14579 (192888'14578) modify 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.1315259.0:8152 2014-08-11 16:45:25.581531 2014-08-13 17:38:02.729367 7fe9bae887a0 20 read_log 192888'14586 (0'0) promote 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by osd.20.0:16435 2014-08-11 16:45:38.627442 2014-08-13 17:38:02.729388 7fe9bae887a0 20 read_log 192888'14587 (192888'14586) modify 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.1315259.0:11508 2014-08-11 16:45:38.619645 2014-08-13 17:38:02.729401 7fe9bae887a0 20 read_log 192888'14588 (0'0) promote 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:16443 2014-08-11 16:45:44.323018 2014-08-13 17:38:02.729413 7fe9bae887a0 20 read_log 192888'14589 (192888'14588) modify 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by client.1315259.0:12875 2014-08-11 16:45:44.314861 2014-08-13 17:38:02.729425 7fe9bae887a0 20 read_log 192888'14594 (0'0) promote 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by osd.20.0:16451 2014-08-11 16:45:47.256906 2014-08-13 17:38:02.729437 7fe9bae887a0 20 read_log 192888'14595 (192888'14594) modify 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.1315259.0:13595 2014-08-11 16:45:47.247068 2014-08-13 17:38:02.729449 7fe9bae887a0 20 read_log 192888'14598 (0'0) promote ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by osd.20.0:16471 2014-08-11 16:45:51.435871 2014-08-13 17:38:02.729461 7fe9bae887a0 20 read_log 192888'14599 (192888'14598) modify ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.1315259.0:14704 2014-08-11 16:45:51.427885 2014-08-13 17:38:02.729473 7fe9bae887a0 20 read_log 192888'14600 (0'0) promote 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by osd.20.0:16473 2014-08-11 16:45:51.877645 2014-08-13 17:38:02.729485 7fe9bae887a0 20 read_log 192888'14601 (192888'14600) modify 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.1315259.0:14823 2014-08-11 16:45:51.870221 2014-08-13 17:38:02.729497 7fe9bae887a0 20 read_log 192888'14604 (0'0) promote 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by osd.20.0:16479 2014-08-11 16:45:54.192535 2014-08-13 17:38:02.729509 7fe9bae887a0 20 read_log 192888'14605 (192888'14604) modify 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.1315259.0:15412 2014-08-11 16:45:54.184254 2014-08-13 17:38:02.729521 7fe9bae887a0 20 read_log 192888'14606 (0'0) promote fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:16487 2014-08-11 16:45:56.770989 2014-08-13 17:38:02.729533 7fe9bae887a0 20 read_log 192888'14607 (192888'14606) modify fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by client.1315259.0:16090 2014-08-11 16:45:56.762083 2014-08-13 17:38:02.729545 7fe9bae887a0 20 read_log 192888'14610 (0'0) promote 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by osd.20.0:16507 2014-08-11 16:46:08.348284 2014-08-13 17:38:02.729557 7fe9bae887a0 20 read_log 192888'14611 (192888'14610) modify 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.1315259.0:19023 2014-08-11 16:46:08.339815 2014-08-13 17:38:02.729569 7fe9bae887a0 20 read_log 192888'14614 (0'0) promote d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by osd.20.0:16521 2014-08-11 16:46:12.588079 2014-08-13 17:38:02.729581 7fe9bae887a0 20 read_log 192888'14615 (192888'14614) modify d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.1315259.0:20127 2014-08-11 16:46:12.578599 2014-08-13 17:38:02.729593 7fe9bae887a0 20 read_log 192888'14620 (0'0) promote ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by osd.20.0:16547 2014-08-11 16:46:21.150570 2014-08-13 17:38:02.729605 7fe9bae887a0 20 read_log 192888'14621 (192888'14620) modify ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.1315259.0:22377 2014-08-11 16:46:21.142139 2014-08-13 17:38:02.729617 7fe9bae887a0 20 read_log 192888'14632 (0'0) promote ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by osd.20.0:16597 2014-08-11 16:46:38.635444 2014-08-13 17:38:02.729638 7fe9bae887a0 20 read_log 192888'14633 (192888'14632) modify ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.1315259.0:26690 2014-08-11 16:46:38.627527 2014-08-13 17:38:02.729651 7fe9bae887a0 20 read_log 192888'14634 (0'0) promote a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:16607 2014-08-11 16:46:39.834220 2014-08-13 17:38:02.729663 7fe9bae887a0 20 read_log 192888'14635 (192888'14634) modify a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by client.1315259.0:26990 2014-08-11 16:46:39.826626 2014-08-13 17:38:02.729675 7fe9bae887a0 20 read_log 192888'14636 (0'0) promote 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by osd.20.0:16609 2014-08-11 16:46:39.853956 2014-08-13 17:38:02.729690 7fe9bae887a0 20 read_log 192888'14637 (192888'14636) modify 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.1315259.0:26994 2014-08-11 16:46:39.847017 2014-08-13 17:38:02.729703 7fe9bae887a0 20 read_log 192888'14640 (0'0) promote a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:16621 2014-08-11 16:46:45.291346 2014-08-13 17:38:02.729715 7fe9bae887a0 20 read_log 192888'14641 (192888'14640) modify a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by client.1315259.0:28317 2014-08-11 16:46:45.283367 2014-08-13 17:38:02.729727 7fe9bae887a0 20 read_log 192888'14642 (0'0) promote 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by osd.20.0:16623 2014-08-11 16:46:45.318065 2014-08-13 17:38:02.729739 7fe9bae887a0 20 read_log 192888'14643 (192888'14642) modify 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.1315259.0:28324 2014-08-11 16:46:45.310496 2014-08-13 17:38:02.729751 7fe9bae887a0 20 read_log 192888'14646 (0'0) promote 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:16629 2014-08-11 16:46:48.709227 2014-08-13 17:38:02.729763 7fe9bae887a0 20 read_log 192888'14647 (192888'14646) modify 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by client.1315259.0:29175 2014-08-11 16:46:48.699853 2014-08-13 17:38:02.729793 7fe9bae887a0 20 read_log 192888'14648 (192888'14555) modify 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by client.1315259.0:31340 2014-08-11 16:46:56.066437 2014-08-13 17:38:02.729806 7fe9bae887a0 20 read_log 192888'14650 (192888'14559) modify 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by client.1315259.0:32748 2014-08-11 16:46:56.459492 2014-08-13 17:38:02.729818 7fe9bae887a0 20 read_log 192888'14651 (192888'14561) modify 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by client.1315259.0:33048 2014-08-11 16:46:56.541018 2014-08-13 17:38:02.729830 7fe9bae887a0 20 read_log 192888'14654 (192888'14567) modify f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by client.1315259.0:36734 2014-08-11 16:46:57.499069 2014-08-13 17:38:02.729842 7fe9bae887a0 20 read_log 192888'14655 (192888'14569) modify d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.1315259.0:37208 2014-08-11 16:46:57.622055 2014-08-13 17:38:02.729856 7fe9bae887a0 20 read_log 192888'14658 (192888'14575) modify 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.1315259.0:38407 2014-08-11 16:46:57.938860 2014-08-13 17:38:02.729872 7fe9bae887a0 20 read_log 192888'14660 (192888'14579) modify 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.1315259.0:38879 2014-08-11 16:46:58.068998 2014-08-13 17:38:02.729884 7fe9bae887a0 20 read_log 192888'14664 (192888'14587) modify 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.1315259.0:42235 2014-08-11 16:46:58.954832 2014-08-13 17:38:02.729896 7fe9bae887a0 20 read_log 192888'14665 (192888'14589) modify 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by client.1315259.0:43602 2014-08-11 16:46:59.336384 2014-08-13 17:38:02.729916 7fe9bae887a0 20 read_log 192888'14668 (192888'14595) modify 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.1315259.0:44322 2014-08-11 16:46:59.545884 2014-08-13 17:38:02.729931 7fe9bae887a0 20 read_log 192888'14670 (192888'14599) modify ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.1315259.0:45431 2014-08-11 16:46:59.868316 2014-08-13 17:38:02.729944 7fe9bae887a0 20 read_log 192888'14671 (192888'14601) modify 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.1315259.0:45550 2014-08-11 16:46:59.902936 2014-08-13 17:38:02.729956 7fe9bae887a0 20 read_log 192888'14673 (192888'14605) modify 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.1315259.0:46139 2014-08-11 16:47:00.070596 2014-08-13 17:38:02.729969 7fe9bae887a0 20 read_log 192888'14674 (192888'14607) modify fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by client.1315259.0:46817 2014-08-11 16:47:00.266746 2014-08-13 17:38:02.729981 7fe9bae887a0 20 read_log 192888'14676 (192888'14611) modify 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.1315259.0:49750 2014-08-11 16:47:01.162706 2014-08-13 17:38:02.729993 7fe9bae887a0 20 read_log 192888'14678 (192888'14615) modify d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.1315259.0:50854 2014-08-11 16:47:01.497844 2014-08-13 17:38:02.730005 7fe9bae887a0 20 read_log 192888'14681 (192888'14621) modify ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.1315259.0:53104 2014-08-11 16:47:02.156349 2014-08-13 17:38:02.730017 7fe9bae887a0 20 read_log 192888'14687 (192888'14633) modify ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.1315259.0:57417 2014-08-11 16:47:03.505942 2014-08-13 17:38:02.730029 7fe9bae887a0 20 read_log 192888'14688 (192888'14635) modify a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by client.1315259.0:57717 2014-08-11 16:47:03.592040 2014-08-13 17:38:02.730064 7fe9bae887a0 20 read_log 192888'14689 (192888'14637) modify 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.1315259.0:57721 2014-08-11 16:47:03.592927 2014-08-13 17:38:02.730077 7fe9bae887a0 20 read_log 192888'14691 (192888'14641) modify a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by client.1315259.0:59044 2014-08-11 16:47:03.981590 2014-08-13 17:38:02.730089 7fe9bae887a0 20 read_log 192888'14692 (192888'14643) modify 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.1315259.0:59051 2014-08-11 16:47:03.983701 2014-08-13 17:38:02.730101 7fe9bae887a0 20 read_log 192888'14694 (192888'14647) modify 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by client.1315259.0:59902 2014-08-11 16:47:04.278553 2014-08-13 17:38:02.730114 7fe9bae887a0 20 read_log 192888'14695 (192888'14695) modify 2c/hit_set_24.2c_archive_2014-08-11 16:44:57.627923_2014-08-11 17:05:15.447969/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.448742 2014-08-13 17:38:02.730126 7fe9bae887a0 20 read_log 192888'14696 (192888'14552) delete 2c/hit_set_24.2c_archive_2014-08-11 16:02:02.168428_2014-08-11 16:44:57.627389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.448742 2014-08-13 17:38:02.730138 7fe9bae887a0 20 read_log 192888'14701 (0'0) promote b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:16828 2014-08-11 17:05:17.495891 2014-08-13 17:38:02.730151 7fe9bae887a0 20 read_log 192888'14702 (192888'14701) modify b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by client.942868.0:607 2014-08-11 17:05:17.489278 2014-08-13 17:38:02.730172 7fe9bae887a0 20 read_log 192888'14703 (0'0) promote 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by osd.20.0:16833 2014-08-11 17:05:19.458162 2014-08-13 17:38:02.730210 7fe9bae887a0 20 read_log 192888'14704 (192888'14703) modify 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.942868.0:1087 2014-08-11 17:05:19.448933 2014-08-13 17:38:02.730227 7fe9bae887a0 20 read_log 192888'14707 (0'0) promote 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:16843 2014-08-11 17:05:21.300430 2014-08-13 17:38:02.730239 7fe9bae887a0 20 read_log 192888'14708 (192888'14707) modify 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by client.942868.0:1535 2014-08-11 17:05:21.289657 2014-08-13 17:38:02.730253 7fe9bae887a0 20 read_log 192888'14711 (0'0) promote 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by osd.20.0:16861 2014-08-11 17:05:29.541776 2014-08-13 17:38:02.730265 7fe9bae887a0 20 read_log 192888'14712 (192888'14711) modify 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.942868.0:3676 2014-08-11 17:05:29.533913 2014-08-13 17:38:02.730277 7fe9bae887a0 20 read_log 192888'14721 (0'0) promote badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by osd.20.0:16885 2014-08-11 17:05:36.723562 2014-08-13 17:38:02.730289 7fe9bae887a0 20 read_log 192888'14722 (192888'14721) modify badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.942868.0:5562 2014-08-11 17:05:36.715254 2014-08-13 17:38:02.730301 7fe9bae887a0 20 read_log 192888'14725 (0'0) promote c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:16897 2014-08-11 17:05:40.565429 2014-08-13 17:38:02.730313 7fe9bae887a0 20 read_log 192888'14726 (192888'14725) modify c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by client.942868.0:6623 2014-08-11 17:05:40.557477 2014-08-13 17:38:02.730326 7fe9bae887a0 20 read_log 192888'14729 (0'0) promote c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by osd.20.0:16905 2014-08-11 17:05:45.335005 2014-08-13 17:38:02.730337 7fe9bae887a0 20 read_log 192888'14730 (192888'14729) modify c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.942868.0:7862 2014-08-11 17:05:45.323751 2014-08-13 17:38:02.730350 7fe9bae887a0 20 read_log 192888'14733 (0'0) promote 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by osd.20.0:16909 2014-08-11 17:05:45.516811 2014-08-13 17:38:02.730361 7fe9bae887a0 20 read_log 192888'14734 (192888'14733) modify 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.942868.0:7913 2014-08-11 17:05:45.507752 2014-08-13 17:38:02.730373 7fe9bae887a0 20 read_log 192888'14739 (0'0) promote 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by osd.20.0:16925 2014-08-11 17:05:49.189002 2014-08-13 17:38:02.730385 7fe9bae887a0 20 read_log 192888'14740 (192888'14739) modify 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.942868.0:8908 2014-08-11 17:05:49.181303 2014-08-13 17:38:02.730397 7fe9bae887a0 20 read_log 192888'14747 (0'0) promote f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:16941 2014-08-11 17:05:57.488387 2014-08-13 17:38:02.730409 7fe9bae887a0 20 read_log 192888'14748 (192888'14747) modify f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by client.942868.0:11173 2014-08-11 17:05:57.480126 2014-08-13 17:38:02.730421 7fe9bae887a0 20 read_log 192888'14749 (0'0) promote 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:16951 2014-08-11 17:05:59.044874 2014-08-13 17:38:02.730433 7fe9bae887a0 20 read_log 192888'14750 (192888'14749) modify 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by client.942868.0:11604 2014-08-11 17:05:59.035058 2014-08-13 17:38:02.730445 7fe9bae887a0 20 read_log 192888'14755 (0'0) promote cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by osd.20.0:16967 2014-08-11 17:06:05.305636 2014-08-13 17:38:02.730466 7fe9bae887a0 20 read_log 192888'14756 (192888'14755) modify cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.942868.0:13308 2014-08-11 17:06:05.297336 2014-08-13 17:38:02.730479 7fe9bae887a0 20 read_log 192888'14757 (0'0) promote 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by osd.20.0:16969 2014-08-11 17:06:06.037330 2014-08-13 17:38:02.730491 7fe9bae887a0 20 read_log 192888'14758 (192888'14757) modify 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.942868.0:13467 2014-08-11 17:06:06.029406 2014-08-13 17:38:02.730504 7fe9bae887a0 20 read_log 192888'14761 (0'0) promote 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by osd.20.0:16985 2014-08-11 17:06:08.855077 2014-08-13 17:38:02.730517 7fe9bae887a0 20 read_log 192888'14762 (192888'14761) modify 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.942868.0:14153 2014-08-11 17:06:08.847597 2014-08-13 17:38:02.730529 7fe9bae887a0 20 read_log 192888'14763 (0'0) promote 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by osd.20.0:16989 2014-08-11 17:06:10.156218 2014-08-13 17:38:02.730541 7fe9bae887a0 20 read_log 192888'14764 (192888'14763) modify 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.942868.0:14477 2014-08-11 17:06:10.147480 2014-08-13 17:38:02.730553 7fe9bae887a0 20 read_log 192888'14765 (0'0) promote 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:16993 2014-08-11 17:06:11.791551 2014-08-13 17:38:02.730565 7fe9bae887a0 20 read_log 192888'14766 (192888'14765) modify 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by client.942868.0:14889 2014-08-11 17:06:11.781368 2014-08-13 17:38:02.730577 7fe9bae887a0 20 read_log 192888'14769 (0'0) promote 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by osd.20.0:17003 2014-08-11 17:06:13.618827 2014-08-13 17:38:02.730589 7fe9bae887a0 20 read_log 192888'14770 (192888'14769) modify 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.942868.0:15358 2014-08-11 17:06:13.610355 2014-08-13 17:38:02.730601 7fe9bae887a0 20 read_log 192888'14771 (0'0) promote 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by osd.20.0:17009 2014-08-11 17:06:15.239023 2014-08-13 17:38:02.730613 7fe9bae887a0 20 read_log 192888'14772 (192888'14771) modify 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.942868.0:15775 2014-08-11 17:06:15.231576 2014-08-13 17:38:02.730625 7fe9bae887a0 20 read_log 192888'14773 (0'0) promote 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:17013 2014-08-11 17:06:16.836399 2014-08-13 17:38:02.730637 7fe9bae887a0 20 read_log 192888'14774 (192888'14773) modify 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by client.942868.0:16189 2014-08-11 17:06:16.828252 2014-08-13 17:38:02.730649 7fe9bae887a0 20 read_log 192888'14775 (0'0) promote 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by osd.20.0:17035 2014-08-11 17:06:23.378175 2014-08-13 17:38:02.730661 7fe9bae887a0 20 read_log 192888'14776 (192888'14775) modify 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.942868.0:17879 2014-08-11 17:06:23.370234 2014-08-13 17:38:02.730673 7fe9bae887a0 20 read_log 192888'14787 (0'0) promote 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by osd.20.0:17061 2014-08-11 17:06:27.443121 2014-08-13 17:38:02.730685 7fe9bae887a0 20 read_log 192888'14788 (192888'14787) modify 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.942868.0:18960 2014-08-11 17:06:27.434935 2014-08-13 17:38:02.730697 7fe9bae887a0 20 read_log 192888'14789 (0'0) promote e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:17063 2014-08-11 17:06:28.229584 2014-08-13 17:38:02.730715 7fe9bae887a0 20 read_log 192888'14790 (192888'14789) modify e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by client.942868.0:19167 2014-08-11 17:06:28.220544 2014-08-13 17:38:02.730728 7fe9bae887a0 20 read_log 192888'14791 (0'0) promote ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:17067 2014-08-11 17:06:29.182067 2014-08-13 17:38:02.730740 7fe9bae887a0 20 read_log 192888'14792 (192888'14791) modify ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by client.942868.0:19421 2014-08-11 17:06:29.173724 2014-08-13 17:38:02.730752 7fe9bae887a0 20 read_log 192888'14793 (0'0) promote 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by osd.20.0:17073 2014-08-11 17:06:29.808701 2014-08-13 17:38:02.730764 7fe9bae887a0 20 read_log 192888'14794 (192888'14793) modify 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.942868.0:19589 2014-08-11 17:06:29.800747 2014-08-13 17:38:02.730777 7fe9bae887a0 20 read_log 192888'14795 (0'0) promote 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by osd.20.0:17089 2014-08-11 17:06:35.107506 2014-08-13 17:38:02.730789 7fe9bae887a0 20 read_log 192888'14796 (192888'14795) modify 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.942868.0:20975 2014-08-11 17:06:35.100073 2014-08-13 17:38:02.730801 7fe9bae887a0 20 read_log 192888'14799 (0'0) promote fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by osd.20.0:17095 2014-08-11 17:06:36.740182 2014-08-13 17:38:02.730813 7fe9bae887a0 20 read_log 192888'14800 (192888'14799) modify fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.942868.0:21399 2014-08-11 17:06:36.731758 2014-08-13 17:38:02.730825 7fe9bae887a0 20 read_log 192888'14801 (0'0) promote 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by osd.20.0:17099 2014-08-11 17:06:37.312404 2014-08-13 17:38:02.730837 7fe9bae887a0 20 read_log 192888'14802 (192888'14801) modify 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.942868.0:21554 2014-08-11 17:06:37.304605 2014-08-13 17:38:02.730849 7fe9bae887a0 20 read_log 192888'14805 (0'0) promote 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:17105 2014-08-11 17:06:39.721971 2014-08-13 17:38:02.730861 7fe9bae887a0 20 read_log 192888'14806 (192888'14805) modify 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by client.942868.0:22200 2014-08-11 17:06:39.713726 2014-08-13 17:38:02.730873 7fe9bae887a0 20 read_log 192888'14807 (0'0) promote 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by osd.20.0:17107 2014-08-11 17:06:40.081221 2014-08-13 17:38:02.730885 7fe9bae887a0 20 read_log 192888'14808 (192888'14807) modify 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.942868.0:22284 2014-08-11 17:06:40.072658 2014-08-13 17:38:02.730897 7fe9bae887a0 20 read_log 192888'14813 (0'0) promote f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by osd.20.0:17123 2014-08-11 17:06:46.132967 2014-08-13 17:38:02.730909 7fe9bae887a0 20 read_log 192888'14814 (192888'14813) modify f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.942868.0:23841 2014-08-11 17:06:46.122947 2014-08-13 17:38:02.730921 7fe9bae887a0 20 read_log 192888'14815 (0'0) promote 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by osd.20.0:17129 2014-08-11 17:06:47.845770 2014-08-13 17:38:02.730933 7fe9bae887a0 20 read_log 192888'14816 (192888'14815) modify 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.942868.0:24308 2014-08-11 17:06:47.837685 2014-08-13 17:38:02.730945 7fe9bae887a0 20 read_log 192888'14817 (0'0) promote a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by osd.20.0:17133 2014-08-11 17:06:48.223069 2014-08-13 17:38:02.730964 7fe9bae887a0 20 read_log 192888'14818 (192888'14817) modify a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.942868.0:24408 2014-08-11 17:06:48.215048 2014-08-13 17:38:02.730976 7fe9bae887a0 20 read_log 192888'14819 (0'0) promote 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:17137 2014-08-11 17:06:48.880798 2014-08-13 17:38:02.730988 7fe9bae887a0 20 read_log 192888'14820 (192888'14819) modify 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by client.942868.0:24585 2014-08-11 17:06:48.873097 2014-08-13 17:38:02.731000 7fe9bae887a0 20 read_log 192888'14827 (0'0) promote 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:17153 2014-08-11 17:06:54.669624 2014-08-13 17:38:02.731012 7fe9bae887a0 20 read_log 192888'14828 (192888'14827) modify 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by client.942868.0:26062 2014-08-11 17:06:54.660605 2014-08-13 17:38:02.731041 7fe9bae887a0 20 read_log 192888'14833 (0'0) promote 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by osd.20.0:17171 2014-08-11 17:06:57.906683 2014-08-13 17:38:02.731057 7fe9bae887a0 20 read_log 192888'14834 (192888'14833) modify 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.942868.0:26912 2014-08-11 17:06:57.896047 2014-08-13 17:38:02.731069 7fe9bae887a0 20 read_log 192888'14839 (0'0) promote 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:17189 2014-08-11 17:07:01.843597 2014-08-13 17:38:02.731081 7fe9bae887a0 20 read_log 192888'14840 (192888'14839) modify 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by client.942868.0:27958 2014-08-11 17:07:01.834611 2014-08-13 17:38:02.731094 7fe9bae887a0 20 read_log 192888'14847 (0'0) promote 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:17225 2014-08-11 17:07:12.171013 2014-08-13 17:38:02.731106 7fe9bae887a0 20 read_log 192888'14848 (192888'14847) modify 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by client.942868.0:30586 2014-08-11 17:07:12.162935 2014-08-13 17:38:02.731119 7fe9bae887a0 20 read_log 192888'14853 (192888'14702) modify b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by client.942868.0:31873 2014-08-11 17:07:16.082792 2014-08-13 17:38:02.731131 7fe9bae887a0 20 read_log 192888'14854 (192888'14704) modify 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.942868.0:32353 2014-08-11 17:07:16.230538 2014-08-13 17:38:02.731143 7fe9bae887a0 20 read_log 192888'14856 (192888'14708) modify 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by client.942868.0:32801 2014-08-11 17:07:16.348817 2014-08-13 17:38:02.731159 7fe9bae887a0 20 read_log 192888'14858 (192888'14712) modify 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.942868.0:34942 2014-08-11 17:07:16.939124 2014-08-13 17:38:02.731172 7fe9bae887a0 20 read_log 192888'14863 (192888'14722) modify badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.942868.0:36828 2014-08-11 17:07:17.444676 2014-08-13 17:38:02.731185 7fe9bae887a0 20 read_log 192888'14865 (192888'14726) modify c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by client.942868.0:37889 2014-08-11 17:07:17.735915 2014-08-13 17:38:02.731197 7fe9bae887a0 20 read_log 192888'14867 (192888'14730) modify c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.942868.0:39128 2014-08-11 17:07:18.090783 2014-08-13 17:38:02.731209 7fe9bae887a0 20 read_log 192888'14869 (192888'14734) modify 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.942868.0:39179 2014-08-11 17:07:18.108917 2014-08-13 17:38:02.731226 7fe9bae887a0 20 read_log 192888'14872 (192888'14740) modify 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.942868.0:40174 2014-08-11 17:07:18.383430 2014-08-13 17:38:02.731248 7fe9bae887a0 20 read_log 192888'14876 (192888'14748) modify f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by client.942868.0:42439 2014-08-11 17:07:18.997524 2014-08-13 17:38:02.731263 7fe9bae887a0 20 read_log 192888'14877 (192888'14750) modify 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by client.942868.0:42870 2014-08-11 17:07:19.114749 2014-08-13 17:38:02.731276 7fe9bae887a0 20 read_log 192888'14880 (192888'14756) modify cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.942868.0:44574 2014-08-11 17:07:19.603762 2014-08-13 17:38:02.731288 7fe9bae887a0 20 read_log 192888'14881 (192888'14758) modify 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.942868.0:44733 2014-08-11 17:07:19.649621 2014-08-13 17:38:02.731300 7fe9bae887a0 20 read_log 192888'14883 (192888'14762) modify 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.942868.0:45419 2014-08-11 17:07:19.833115 2014-08-13 17:38:02.731312 7fe9bae887a0 20 read_log 192888'14884 (192888'14764) modify 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.942868.0:45743 2014-08-11 17:07:19.921961 2014-08-13 17:38:02.731324 7fe9bae887a0 20 read_log 192888'14885 (192888'14766) modify 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by client.942868.0:46155 2014-08-11 17:07:20.035826 2014-08-13 17:38:02.731337 7fe9bae887a0 20 read_log 192888'14887 (192888'14770) modify 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.942868.0:46624 2014-08-11 17:07:20.174695 2014-08-13 17:38:02.731349 7fe9bae887a0 20 read_log 192888'14888 (192888'14772) modify 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.942868.0:47041 2014-08-11 17:07:20.291054 2014-08-13 17:38:02.731361 7fe9bae887a0 20 read_log 192888'14889 (192888'14774) modify 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by client.942868.0:47455 2014-08-11 17:07:20.412858 2014-08-13 17:38:02.731373 7fe9bae887a0 20 read_log 192888'14890 (192888'14776) modify 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.942868.0:49145 2014-08-11 17:07:20.922348 2014-08-13 17:38:02.731385 7fe9bae887a0 20 read_log 192888'14896 (192888'14788) modify 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.942868.0:50226 2014-08-11 17:07:21.252820 2014-08-13 17:38:02.731397 7fe9bae887a0 20 read_log 192888'14897 (192888'14790) modify e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by client.942868.0:50433 2014-08-11 17:07:21.317042 2014-08-13 17:38:02.731409 7fe9bae887a0 20 read_log 192888'14898 (192888'14792) modify ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by client.942868.0:50687 2014-08-11 17:07:21.399975 2014-08-13 17:38:02.731422 7fe9bae887a0 20 read_log 192888'14899 (192888'14794) modify 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.942868.0:50855 2014-08-11 17:07:21.453814 2014-08-13 17:38:02.731434 7fe9bae887a0 20 read_log 192888'14900 (192888'14796) modify 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.942868.0:52241 2014-08-11 17:07:21.880614 2014-08-13 17:38:02.731446 7fe9bae887a0 20 read_log 192888'14902 (192888'14800) modify fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.942868.0:52665 2014-08-11 17:07:22.013879 2014-08-13 17:38:02.731460 7fe9bae887a0 20 read_log 192888'14903 (192888'14802) modify 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.942868.0:52820 2014-08-11 17:07:22.061200 2014-08-13 17:38:02.731473 7fe9bae887a0 20 read_log 192888'14905 (192888'14806) modify 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by client.942868.0:53466 2014-08-11 17:07:22.271683 2014-08-13 17:38:02.731485 7fe9bae887a0 20 read_log 192888'14906 (192888'14808) modify 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.942868.0:53550 2014-08-11 17:07:22.301891 2014-08-13 17:38:02.731505 7fe9bae887a0 20 read_log 192888'14909 (192888'14814) modify f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.942868.0:55107 2014-08-11 17:07:22.745832 2014-08-13 17:38:02.731518 7fe9bae887a0 20 read_log 192888'14910 (192888'14816) modify 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.942868.0:55574 2014-08-11 17:07:22.887859 2014-08-13 17:38:02.731530 7fe9bae887a0 20 read_log 192888'14911 (192888'14818) modify a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.942868.0:55674 2014-08-11 17:07:22.918229 2014-08-13 17:38:02.731542 7fe9bae887a0 20 read_log 192888'14912 (192888'14820) modify 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by client.942868.0:55851 2014-08-11 17:07:22.978100 2014-08-13 17:38:02.731554 7fe9bae887a0 20 read_log 192888'14916 (192888'14828) modify 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by client.942868.0:57328 2014-08-11 17:07:23.424175 2014-08-13 17:38:02.731566 7fe9bae887a0 20 read_log 192888'14919 (192888'14834) modify 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.942868.0:58178 2014-08-11 17:07:23.664832 2014-08-13 17:38:02.731578 7fe9bae887a0 20 read_log 192888'14922 (192888'14840) modify 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by client.942868.0:59224 2014-08-11 17:07:23.983203 2014-08-13 17:38:02.731590 7fe9bae887a0 20 read_log 192888'14926 (192888'14848) modify 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by client.942868.0:61852 2014-08-11 17:07:24.772496 2014-08-13 17:38:02.731603 7fe9bae887a0 20 read_log 193305'14928 (193305'14928) modify 2c/hit_set_24.2c_archive_2014-08-11 18:37:04.911332_2014-08-11 18:42:33.028272/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.028549 2014-08-13 17:38:02.731615 7fe9bae887a0 20 read_log 193305'14929 (192888'14695) delete 2c/hit_set_24.2c_archive_2014-08-11 16:44:57.627923_2014-08-11 17:05:15.447969/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.028549 2014-08-13 17:38:02.731627 7fe9bae887a0 20 read_log 193305'14930 (0'0) promote 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by osd.20.0:9 2014-08-11 18:42:33.058887 2014-08-13 17:38:02.731640 7fe9bae887a0 20 read_log 193305'14931 (193305'14930) modify 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.951337.0:55 2014-08-11 18:42:33.023511 2014-08-13 17:38:02.731652 7fe9bae887a0 20 read_log 193305'14932 (0'0) promote fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by osd.20.0:19 2014-08-11 18:42:34.941479 2014-08-13 17:38:02.731664 7fe9bae887a0 20 read_log 193305'14933 (193305'14932) modify fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.951337.0:476 2014-08-11 18:42:34.935078 2014-08-13 17:38:02.731676 7fe9bae887a0 20 read_log 193305'14934 (0'0) promote 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by osd.20.0:21 2014-08-11 18:42:36.166828 2014-08-13 17:38:02.731688 7fe9bae887a0 20 read_log 193305'14935 (193305'14934) modify 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.951337.0:776 2014-08-11 18:42:36.159603 2014-08-13 17:38:02.731700 7fe9bae887a0 20 read_log 193305'14936 (0'0) promote d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by osd.20.0:50 2014-08-11 18:42:41.544733 2014-08-13 17:38:02.731712 7fe9bae887a0 20 read_log 193305'14937 (193305'14936) modify d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.951337.0:2216 2014-08-11 18:42:41.537162 2014-08-13 17:38:02.731724 7fe9bae887a0 20 read_log 193305'14938 (0'0) promote 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by osd.20.0:60 2014-08-11 18:42:44.272800 2014-08-13 17:38:02.731743 7fe9bae887a0 20 read_log 193305'14939 (193305'14938) modify 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.951337.0:2968 2014-08-11 18:42:44.265838 2014-08-13 17:38:02.731755 7fe9bae887a0 20 read_log 193305'14940 (0'0) promote 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:78 2014-08-11 18:42:46.283035 2014-08-13 17:38:02.731767 7fe9bae887a0 20 read_log 193305'14941 (193305'14940) modify 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by client.951337.0:3521 2014-08-11 18:42:46.275848 2014-08-13 17:38:02.731780 7fe9bae887a0 20 read_log 193305'14942 (0'0) promote dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:100 2014-08-11 18:42:51.146267 2014-08-13 17:38:02.731795 7fe9bae887a0 20 read_log 193305'14943 (193305'14942) modify dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by client.951337.0:4842 2014-08-11 18:42:51.139496 2014-08-13 17:38:02.731807 7fe9bae887a0 20 read_log 193305'14944 (0'0) promote 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by osd.20.0:158 2014-08-11 18:43:03.200041 2014-08-13 17:38:02.731819 7fe9bae887a0 20 read_log 193305'14945 (193305'14944) modify 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.951337.0:8144 2014-08-11 18:43:03.192415 2014-08-13 17:38:02.731831 7fe9bae887a0 20 read_log 193305'14946 (0'0) promote 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by osd.20.0:204 2014-08-11 18:43:11.152094 2014-08-13 17:38:02.731843 7fe9bae887a0 20 read_log 193305'14947 (193305'14946) modify 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.951337.0:10238 2014-08-11 18:43:11.144066 2014-08-13 17:38:02.731855 7fe9bae887a0 20 read_log 193305'14948 (0'0) promote 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:220 2014-08-11 18:43:13.834120 2014-08-13 17:38:02.731867 7fe9bae887a0 20 read_log 193305'14949 (193305'14948) modify 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by client.951337.0:10953 2014-08-11 18:43:13.826040 2014-08-13 17:38:02.731879 7fe9bae887a0 20 read_log 193305'14950 (0'0) promote 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:224 2014-08-11 18:43:15.328106 2014-08-13 17:38:02.731893 7fe9bae887a0 20 read_log 193305'14951 (193305'14950) modify 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by client.951337.0:11307 2014-08-11 18:43:15.319833 2014-08-13 17:38:02.731905 7fe9bae887a0 20 read_log 193305'14952 (0'0) promote 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:242 2014-08-11 18:43:19.060587 2014-08-13 17:38:02.731917 7fe9bae887a0 20 read_log 193305'14953 (193305'14952) modify 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by client.951337.0:12320 2014-08-11 18:43:19.053956 2014-08-13 17:38:02.731931 7fe9bae887a0 20 read_log 193305'14954 (0'0) promote 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:266 2014-08-11 18:43:22.725989 2014-08-13 17:38:02.731944 7fe9bae887a0 20 read_log 193305'14955 (193305'14954) modify 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by client.951337.0:13295 2014-08-11 18:43:22.718816 2014-08-13 17:38:02.731956 7fe9bae887a0 20 read_log 193305'14956 (0'0) promote b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:312 2014-08-11 18:43:30.401994 2014-08-13 17:38:02.731968 7fe9bae887a0 20 read_log 193305'14957 (193305'14956) modify b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by client.951337.0:15423 2014-08-11 18:43:30.394410 2014-08-13 17:38:02.731979 7fe9bae887a0 20 read_log 193305'14958 (0'0) promote 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by osd.20.0:318 2014-08-11 18:43:32.381306 2014-08-13 17:38:02.732018 7fe9bae887a0 20 read_log 193305'14959 (193305'14958) modify 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.951337.0:15995 2014-08-11 18:43:32.373416 2014-08-13 17:38:02.732034 7fe9bae887a0 20 read_log 193305'14960 (0'0) promote 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by osd.20.0:352 2014-08-11 18:43:40.697337 2014-08-13 17:38:02.732047 7fe9bae887a0 20 read_log 193305'14961 (193305'14960) modify 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.951337.0:18373 2014-08-11 18:43:40.689363 2014-08-13 17:38:02.732061 7fe9bae887a0 20 read_log 193305'14962 (0'0) promote b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:424 2014-08-11 18:43:51.504104 2014-08-13 17:38:02.732076 7fe9bae887a0 20 read_log 193305'14963 (193305'14962) modify b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by client.951337.0:21363 2014-08-11 18:43:51.496690 2014-08-13 17:38:02.732088 7fe9bae887a0 20 read_log 193305'14964 (0'0) promote bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by osd.20.0:428 2014-08-11 18:43:51.602208 2014-08-13 17:38:02.732102 7fe9bae887a0 20 read_log 193305'14965 (193305'14964) modify bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.951337.0:21392 2014-08-11 18:43:51.594314 2014-08-13 17:38:02.732116 7fe9bae887a0 20 read_log 193305'14966 (0'0) promote 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by osd.20.0:436 2014-08-11 18:43:54.051374 2014-08-13 17:38:02.732128 7fe9bae887a0 20 read_log 193305'14967 (193305'14966) modify 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.951337.0:22069 2014-08-11 18:43:54.042889 2014-08-13 17:38:02.732146 7fe9bae887a0 20 read_log 193305'14968 (0'0) promote e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by osd.20.0:466 2014-08-11 18:43:57.775627 2014-08-13 17:38:02.732163 7fe9bae887a0 20 read_log 193305'14969 (193305'14968) modify e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.951337.0:23096 2014-08-11 18:43:57.767140 2014-08-13 17:38:02.732176 7fe9bae887a0 20 read_log 193305'14970 (0'0) promote 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:506 2014-08-11 18:44:05.582627 2014-08-13 17:38:02.732188 7fe9bae887a0 20 read_log 193305'14971 (193305'14970) modify 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by client.951337.0:25259 2014-08-11 18:44:05.574492 2014-08-13 17:38:02.732202 7fe9bae887a0 20 read_log 193305'14972 (0'0) promote 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by osd.20.0:508 2014-08-11 18:44:05.631590 2014-08-13 17:38:02.732215 7fe9bae887a0 20 read_log 193305'14973 (193305'14972) modify 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.951337.0:25273 2014-08-11 18:44:05.624267 2014-08-13 17:38:02.732227 7fe9bae887a0 20 read_log 193305'14974 (0'0) promote 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:518 2014-08-11 18:44:08.160754 2014-08-13 17:38:02.732241 7fe9bae887a0 20 read_log 193305'14975 (193305'14974) modify 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by client.951337.0:25949 2014-08-11 18:44:08.153172 2014-08-13 17:38:02.732253 7fe9bae887a0 20 read_log 193305'14976 (0'0) promote aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:533 2014-08-11 18:44:09.145927 2014-08-13 17:38:02.732265 7fe9bae887a0 20 read_log 193305'14977 (193305'14976) modify aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by client.951337.0:26204 2014-08-11 18:44:09.137887 2014-08-13 17:38:02.732296 7fe9bae887a0 20 read_log 193305'14978 (0'0) promote a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by osd.20.0:535 2014-08-11 18:44:09.278703 2014-08-13 17:38:02.732318 7fe9bae887a0 20 read_log 193305'14979 (193305'14978) modify a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.951337.0:26241 2014-08-11 18:44:09.271564 2014-08-13 17:38:02.732333 7fe9bae887a0 20 read_log 193305'14980 (0'0) promote b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by osd.20.0:541 2014-08-11 18:44:10.630058 2014-08-13 17:38:02.732346 7fe9bae887a0 20 read_log 193305'14981 (193305'14980) modify b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.951337.0:26620 2014-08-11 18:44:10.621159 2014-08-13 17:38:02.732358 7fe9bae887a0 20 read_log 193305'14982 (0'0) promote 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by osd.20.0:545 2014-08-11 18:44:11.424946 2014-08-13 17:38:02.732370 7fe9bae887a0 20 read_log 193305'14983 (193305'14982) modify 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.951337.0:26844 2014-08-11 18:44:11.416571 2014-08-13 17:38:02.732382 7fe9bae887a0 20 read_log 193305'14984 (0'0) promote 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by osd.20.0:556 2014-08-11 18:44:12.959526 2014-08-13 17:38:02.732394 7fe9bae887a0 20 read_log 193305'14985 (193305'14984) modify 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.951337.0:27242 2014-08-11 18:44:12.950878 2014-08-13 17:38:02.732406 7fe9bae887a0 20 read_log 193305'14986 (0'0) promote 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by osd.20.0:566 2014-08-11 18:44:14.213485 2014-08-13 17:38:02.732418 7fe9bae887a0 20 read_log 193305'14987 (193305'14986) modify 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.951337.0:27594 2014-08-11 18:44:14.206000 2014-08-13 17:38:02.732430 7fe9bae887a0 20 read_log 193305'14988 (0'0) promote 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by osd.20.0:581 2014-08-11 18:44:14.966156 2014-08-13 17:38:02.732442 7fe9bae887a0 20 read_log 193305'14989 (193305'14988) modify 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.951337.0:27805 2014-08-11 18:44:14.954950 2014-08-13 17:38:02.732454 7fe9bae887a0 20 read_log 193305'14990 (0'0) promote 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:604 2014-08-11 18:44:19.044537 2014-08-13 17:38:02.732465 7fe9bae887a0 20 read_log 193305'14991 (193305'14990) modify 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by client.951337.0:28916 2014-08-11 18:44:19.035988 2014-08-13 17:38:02.732478 7fe9bae887a0 20 read_log 193305'14992 (0'0) promote ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:612 2014-08-11 18:44:20.688863 2014-08-13 17:38:02.732490 7fe9bae887a0 20 read_log 193305'14993 (193305'14992) modify ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by client.951337.0:29370 2014-08-11 18:44:20.681012 2014-08-13 17:38:02.732501 7fe9bae887a0 20 read_log 193305'14994 (0'0) promote 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by osd.20.0:616 2014-08-11 18:44:21.177696 2014-08-13 17:38:02.732513 7fe9bae887a0 20 read_log 193305'14995 (193305'14994) modify 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.951337.0:29503 2014-08-11 18:44:21.170083 2014-08-13 17:38:02.732525 7fe9bae887a0 20 read_log 193305'14996 (0'0) promote 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:626 2014-08-11 18:44:22.758610 2014-08-13 17:38:02.732538 7fe9bae887a0 20 read_log 193305'14997 (193305'14996) modify 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by client.951337.0:29935 2014-08-11 18:44:22.751340 2014-08-13 17:38:02.732550 7fe9bae887a0 20 read_log 193305'14998 (0'0) promote b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by osd.20.0:630 2014-08-11 18:44:23.176905 2014-08-13 17:38:02.732569 7fe9bae887a0 20 read_log 193305'14999 (193305'14998) modify b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.951337.0:30051 2014-08-11 18:44:23.170567 2014-08-13 17:38:02.732582 7fe9bae887a0 20 read_log 193305'15000 (0'0) promote 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by osd.20.0:638 2014-08-11 18:44:24.080425 2014-08-13 17:38:02.732594 7fe9bae887a0 20 read_log 193305'15001 (193305'15000) modify 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.951337.0:30267 2014-08-11 18:44:24.073436 2014-08-13 17:38:02.732607 7fe9bae887a0 20 read_log 193305'15002 (0'0) promote 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by osd.20.0:668 2014-08-11 18:44:28.266617 2014-08-13 17:38:02.732619 7fe9bae887a0 20 read_log 193305'15003 (193305'15002) modify 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.951337.0:31410 2014-08-11 18:44:28.257958 2014-08-13 17:38:02.732631 7fe9bae887a0 20 read_log 193305'15004 (0'0) promote a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by osd.20.0:674 2014-08-11 18:44:28.902414 2014-08-13 17:38:02.732642 7fe9bae887a0 20 read_log 193305'15005 (193305'15004) modify a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.951337.0:31601 2014-08-11 18:44:28.896396 2014-08-13 17:38:02.732655 7fe9bae887a0 20 read_log 193305'15006 (0'0) promote e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by osd.20.0:686 2014-08-11 18:44:31.176743 2014-08-13 17:38:02.732666 7fe9bae887a0 20 read_log 193305'15007 (193305'15006) modify e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.951337.0:32251 2014-08-11 18:44:31.168779 2014-08-13 17:38:02.732680 7fe9bae887a0 20 read_log 193305'15008 (0'0) promote c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by osd.20.0:688 2014-08-11 18:44:31.634297 2014-08-13 17:38:02.732692 7fe9bae887a0 20 read_log 193305'15009 (193305'15008) modify c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.951337.0:32369 2014-08-11 18:44:31.626227 2014-08-13 17:38:02.732704 7fe9bae887a0 20 read_log 193305'15010 (193305'14931) modify 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.951337.0:32766 2014-08-11 18:44:33.845773 2014-08-13 17:38:02.732716 7fe9bae887a0 20 read_log 193305'15011 (193305'14933) modify fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.951337.0:33187 2014-08-11 18:44:34.224507 2014-08-13 17:38:02.732727 7fe9bae887a0 20 read_log 193305'15012 (193305'14935) modify 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.951337.0:33487 2014-08-11 18:44:34.450137 2014-08-13 17:38:02.732739 7fe9bae887a0 20 read_log 193305'15013 (193305'14937) modify d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.951337.0:34927 2014-08-11 18:44:35.262336 2014-08-13 17:38:02.732751 7fe9bae887a0 20 read_log 193305'15014 (193305'14939) modify 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.951337.0:35679 2014-08-11 18:44:35.700702 2014-08-13 17:38:02.732763 7fe9bae887a0 20 read_log 193305'15015 (193305'14941) modify 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by client.951337.0:36232 2014-08-11 18:44:36.006679 2014-08-13 17:38:02.732775 7fe9bae887a0 20 read_log 193305'15016 (193305'14943) modify dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by client.951337.0:37553 2014-08-11 18:44:36.967667 2014-08-13 17:38:02.732787 7fe9bae887a0 20 read_log 193305'15017 (193305'14945) modify 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.951337.0:40855 2014-08-11 18:44:38.557334 2014-08-13 17:38:02.732799 7fe9bae887a0 20 read_log 193305'15018 (193305'14947) modify 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.951337.0:42949 2014-08-11 18:44:39.432583 2014-08-13 17:38:02.732821 7fe9bae887a0 20 read_log 193305'15019 (193305'14949) modify 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by client.951337.0:43664 2014-08-11 18:44:39.659310 2014-08-13 17:38:02.732834 7fe9bae887a0 20 read_log 193305'15020 (193305'14951) modify 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by client.951337.0:44018 2014-08-11 18:44:39.770485 2014-08-13 17:38:02.732846 7fe9bae887a0 20 read_log 193305'15021 (193305'14953) modify 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by client.951337.0:45031 2014-08-11 18:44:40.072384 2014-08-13 17:38:02.732858 7fe9bae887a0 20 read_log 193305'15022 (193305'14955) modify 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by client.951337.0:46006 2014-08-11 18:44:40.363967 2014-08-13 17:38:02.732870 7fe9bae887a0 20 read_log 193305'15023 (193305'14957) modify b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by client.951337.0:48134 2014-08-11 18:44:40.962087 2014-08-13 17:38:02.732882 7fe9bae887a0 20 read_log 193305'15024 (193305'14959) modify 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.951337.0:48706 2014-08-11 18:44:41.130597 2014-08-13 17:38:02.732895 7fe9bae887a0 20 read_log 193305'15025 (193305'14961) modify 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.951337.0:51084 2014-08-11 18:44:41.846686 2014-08-13 17:38:02.732907 7fe9bae887a0 20 read_log 193305'15026 (193305'14963) modify b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by client.951337.0:54074 2014-08-11 18:44:42.723905 2014-08-13 17:38:02.732919 7fe9bae887a0 20 read_log 193305'15027 (193305'14965) modify bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.951337.0:54103 2014-08-11 18:44:42.732046 2014-08-13 17:38:02.732931 7fe9bae887a0 20 read_log 193305'15028 (193305'14967) modify 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.951337.0:54780 2014-08-11 18:44:42.936662 2014-08-13 17:38:02.732942 7fe9bae887a0 20 read_log 193305'15029 (193305'14969) modify e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.951337.0:55807 2014-08-11 18:44:43.237489 2014-08-13 17:38:02.732954 7fe9bae887a0 20 read_log 193305'15030 (193305'14971) modify 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by client.951337.0:57970 2014-08-11 18:44:43.908532 2014-08-13 17:38:02.732966 7fe9bae887a0 20 read_log 193305'15031 (193305'14973) modify 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.951337.0:57984 2014-08-11 18:44:43.913513 2014-08-13 17:38:02.732978 7fe9bae887a0 20 read_log 193305'15032 (193305'14975) modify 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by client.951337.0:58660 2014-08-11 18:44:44.105953 2014-08-13 17:38:02.732990 7fe9bae887a0 20 read_log 193305'15033 (193305'14977) modify aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by client.951337.0:58915 2014-08-11 18:44:44.178908 2014-08-13 17:38:02.733002 7fe9bae887a0 20 read_log 193305'15034 (193305'14979) modify a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.951337.0:58952 2014-08-11 18:44:44.193036 2014-08-13 17:38:02.733014 7fe9bae887a0 20 read_log 193305'15035 (193305'14981) modify b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.951337.0:59331 2014-08-11 18:44:44.296093 2014-08-13 17:38:02.733027 7fe9bae887a0 20 read_log 193305'15036 (193305'14983) modify 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.951337.0:59555 2014-08-11 18:44:44.360932 2014-08-13 17:38:02.733039 7fe9bae887a0 20 read_log 193305'15037 (193305'14985) modify 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.951337.0:59953 2014-08-11 18:44:44.475825 2014-08-13 17:38:02.733051 7fe9bae887a0 20 read_log 193305'15038 (193305'14987) modify 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.951337.0:60305 2014-08-11 18:44:44.592501 2014-08-13 17:38:02.733070 7fe9bae887a0 20 read_log 193305'15039 (193305'14989) modify 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.951337.0:60516 2014-08-11 18:44:44.653153 2014-08-13 17:38:02.733083 7fe9bae887a0 20 read_log 193305'15040 (193305'14991) modify 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by client.951337.0:61627 2014-08-11 18:44:44.980858 2014-08-13 17:38:02.733095 7fe9bae887a0 20 read_log 193305'15041 (193305'14993) modify ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by client.951337.0:62081 2014-08-11 18:44:45.120566 2014-08-13 17:38:02.733107 7fe9bae887a0 20 read_log 193305'15042 (193305'14995) modify 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.951337.0:62214 2014-08-11 18:44:45.158887 2014-08-13 17:38:02.733119 7fe9bae887a0 20 read_log 193305'15043 (193305'14997) modify 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by client.951337.0:62646 2014-08-11 18:44:45.287766 2014-08-13 17:38:02.733131 7fe9bae887a0 20 read_log 193305'15044 (193305'14999) modify b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.951337.0:62762 2014-08-11 18:44:45.319749 2014-08-13 17:38:02.733143 7fe9bae887a0 20 read_log 193305'15045 (193305'15001) modify 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.951337.0:62978 2014-08-11 18:44:45.378017 2014-08-13 17:38:02.733160 7fe9bae887a0 20 read_log 193305'15046 (193305'15003) modify 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.951337.0:64121 2014-08-11 18:44:45.709650 2014-08-13 17:38:02.733172 7fe9bae887a0 20 read_log 193305'15047 (193305'15005) modify a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.951337.0:64312 2014-08-11 18:44:45.764384 2014-08-13 17:38:02.733185 7fe9bae887a0 20 read_log 193305'15048 (193305'15007) modify e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.951337.0:64962 2014-08-11 18:44:45.975401 2014-08-13 17:38:02.733197 7fe9bae887a0 20 read_log 193305'15049 (193305'15009) modify c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.951337.0:65080 2014-08-11 18:44:46.010291 2014-08-13 17:38:02.733209 7fe9bae887a0 20 read_log 194209'15050 (192888'14912) clean 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:1204 2014-08-12 14:05:46.720579 2014-08-13 17:38:02.733221 7fe9bae887a0 20 read_log 194209'15051 (193305'15019) clean 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:1205 2014-08-12 14:05:46.721139 2014-08-13 17:38:02.733233 7fe9bae887a0 20 read_log 194209'15052 (193305'15023) clean b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:1206 2014-08-12 14:05:46.721727 2014-08-13 17:38:02.733245 7fe9bae887a0 20 read_log 194209'15053 (192888'14856) clean 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:1207 2014-08-12 14:05:46.722180 2014-08-13 17:38:02.733257 7fe9bae887a0 20 read_log 194209'15054 (193305'15040) clean 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:1208 2014-08-12 14:05:46.793692 2014-08-13 17:38:02.733269 7fe9bae887a0 20 read_log 194209'15055 (193305'15041) clean ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:1209 2014-08-12 14:05:46.800826 2014-08-13 17:38:02.733282 7fe9bae887a0 20 read_log 194209'15056 (192834'14513) clean ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:1210 2014-08-12 14:05:46.801106 2014-08-13 17:38:02.733296 7fe9bae887a0 20 read_log 194209'15057 (192888'14905) clean 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:1211 2014-08-12 14:05:46.805219 2014-08-13 17:38:02.733337 7fe9bae887a0 20 read_log 194209'15058 (192888'14926) clean 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:1212 2014-08-12 14:05:46.808809 2014-08-13 17:38:02.733353 7fe9bae887a0 20 read_log 194209'15059 (192888'14922) clean 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:1213 2014-08-12 14:05:46.812479 2014-08-13 17:38:02.733365 7fe9bae887a0 20 read_log 194209'15060 (192888'14865) clean c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:1214 2014-08-12 14:05:46.816442 2014-08-13 17:38:02.733377 7fe9bae887a0 20 read_log 194209'15061 (193305'15033) clean aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:1215 2014-08-12 14:05:46.826229 2014-08-13 17:38:02.733394 7fe9bae887a0 20 read_log 194209'15062 (192834'14381) clean c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:1216 2014-08-12 14:05:46.826758 2014-08-13 17:38:02.733406 7fe9bae887a0 20 read_log 194209'15063 (193305'15043) clean 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:1217 2014-08-12 14:05:46.827203 2014-08-13 17:38:02.733418 7fe9bae887a0 20 read_log 194209'15064 (192888'14665) clean 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:1218 2014-08-12 14:05:46.827979 2014-08-13 17:38:02.733430 7fe9bae887a0 20 read_log 194209'15065 (192888'14648) clean 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:1219 2014-08-12 14:05:46.838912 2014-08-13 17:38:02.733442 7fe9bae887a0 20 read_log 194209'15066 (192888'14650) clean 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:1220 2014-08-12 14:05:46.839186 2014-08-13 17:38:02.733454 7fe9bae887a0 20 read_log 194209'15067 (192888'14853) clean b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:1221 2014-08-12 14:05:46.842597 2014-08-13 17:38:02.733466 7fe9bae887a0 20 read_log 194209'15068 (192834'14498) clean 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:1222 2014-08-12 14:05:46.846671 2014-08-13 17:38:02.733494 7fe9bae887a0 20 read_log 194209'15069 (192834'14346) clean f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:1223 2014-08-12 14:05:46.853816 2014-08-13 17:38:02.733510 7fe9bae887a0 20 read_log 194209'15070 (192834'14511) clean 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:1224 2014-08-12 14:05:46.854455 2014-08-13 17:38:02.733539 7fe9bae887a0 20 read_log 194209'15071 (192888'14674) clean fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:1225 2014-08-12 14:05:46.858203 2014-08-13 17:38:02.733554 7fe9bae887a0 20 read_log 194209'15072 (192888'14885) clean 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:1226 2014-08-12 14:05:46.861963 2014-08-13 17:38:02.733566 7fe9bae887a0 20 read_log 194209'15073 (192888'14898) clean ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:1227 2014-08-12 14:05:46.865993 2014-08-13 17:38:02.733578 7fe9bae887a0 20 read_log 194209'15074 (192888'14654) clean f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:1228 2014-08-12 14:05:46.869735 2014-08-13 17:38:02.733592 7fe9bae887a0 20 read_log 194209'15075 (193305'15026) clean b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:1229 2014-08-12 14:05:46.873378 2014-08-13 17:38:02.733605 7fe9bae887a0 20 read_log 194209'15076 (192888'14688) clean a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:1230 2014-08-12 14:05:46.876907 2014-08-13 17:38:02.733619 7fe9bae887a0 20 read_log 194209'15077 (192888'14897) clean e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:1231 2014-08-12 14:05:46.880566 2014-08-13 17:38:02.733641 7fe9bae887a0 20 read_log 194209'15078 (192888'14877) clean 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:1232 2014-08-12 14:05:46.884093 2014-08-13 17:38:02.733656 7fe9bae887a0 20 read_log 194209'15079 (192834'14437) clean 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:1233 2014-08-12 14:05:46.891178 2014-08-13 17:38:02.733668 7fe9bae887a0 20 read_log 194209'15080 (193305'15020) clean 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:1234 2014-08-12 14:05:46.891437 2014-08-13 17:38:02.733680 7fe9bae887a0 20 read_log 194209'15081 (193305'15016) clean dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:1235 2014-08-12 14:05:46.895172 2014-08-13 17:38:02.733692 7fe9bae887a0 20 read_log 194209'15082 (193305'15022) clean 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:1236 2014-08-12 14:05:46.899065 2014-08-13 17:38:02.733704 7fe9bae887a0 20 read_log 194209'15083 (192888'14651) clean 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:1237 2014-08-12 14:05:46.902823 2014-08-13 17:38:02.733716 7fe9bae887a0 20 read_log 194209'15084 (192888'14916) clean 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:1238 2014-08-12 14:05:46.906332 2014-08-13 17:38:02.733728 7fe9bae887a0 20 read_log 194209'15085 (192888'14876) clean f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:1239 2014-08-12 14:05:46.912937 2014-08-13 17:38:02.733741 7fe9bae887a0 20 read_log 194209'15086 (193305'15032) clean 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:1240 2014-08-12 14:05:46.913193 2014-08-13 17:38:02.733753 7fe9bae887a0 20 read_log 194209'15087 (192834'14354) clean 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:1241 2014-08-12 14:05:46.917355 2014-08-13 17:38:02.733765 7fe9bae887a0 20 read_log 194209'15088 (193305'15021) clean 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:1242 2014-08-12 14:05:46.920766 2014-08-13 17:38:02.733777 7fe9bae887a0 20 read_log 194209'15089 (192888'14691) clean a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:1243 2014-08-12 14:05:46.924335 2014-08-13 17:38:02.733789 7fe9bae887a0 20 read_log 194209'15090 (192834'14525) clean a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:1244 2014-08-12 14:05:46.928141 2014-08-13 17:38:02.733801 7fe9bae887a0 20 read_log 194209'15091 (193305'15015) clean 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:1245 2014-08-12 14:05:46.931829 2014-08-13 17:38:02.733814 7fe9bae887a0 20 read_log 194209'15092 (192888'14694) clean 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:1246 2014-08-12 14:05:46.935220 2014-08-13 17:38:02.733826 7fe9bae887a0 20 read_log 194209'15093 (192888'14890) clean 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by osd.20.0:1247 2014-08-12 14:05:46.939051 2014-08-13 17:38:02.733838 7fe9bae887a0 20 read_log 194209'15094 (192834'13831) clean 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:1248 2014-08-12 14:05:46.942719 2014-08-13 17:38:02.733850 7fe9bae887a0 20 read_log 194209'15095 (193305'15030) clean 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:1249 2014-08-12 14:05:46.949217 2014-08-13 17:38:02.733863 7fe9bae887a0 20 read_log 194209'15096 (192888'14896) clean 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by osd.20.0:1250 2014-08-12 14:05:46.949495 2014-08-13 17:38:02.733877 7fe9bae887a0 20 read_log 194209'15097 (192834'13871) clean 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:1251 2014-08-12 14:05:46.953026 2014-08-13 17:38:02.733916 7fe9bae887a0 20 read_log 194209'15098 (194209'15050) delete 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:1252 2014-08-12 14:05:46.992192 2014-08-13 17:38:02.733932 7fe9bae887a0 20 read_log 194209'15099 (194209'15051) delete 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:1253 2014-08-12 14:05:46.995516 2014-08-13 17:38:02.733944 7fe9bae887a0 20 read_log 194209'15100 (194209'15052) delete b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:1254 2014-08-12 14:05:46.998899 2014-08-13 17:38:02.733958 7fe9bae887a0 20 read_log 194209'15101 (194209'15054) delete 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:1255 2014-08-12 14:05:47.002200 2014-08-13 17:38:02.733972 7fe9bae887a0 20 read_log 194209'15102 (194209'15055) delete ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:1256 2014-08-12 14:05:47.005881 2014-08-13 17:38:02.733984 7fe9bae887a0 20 read_log 194209'15103 (194209'15056) delete ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:1257 2014-08-12 14:05:47.009432 2014-08-13 17:38:02.733996 7fe9bae887a0 20 read_log 194209'15104 (194209'15062) delete c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:1258 2014-08-12 14:05:47.013594 2014-08-13 17:38:02.734011 7fe9bae887a0 20 read_log 194209'15105 (194209'15061) delete aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:1259 2014-08-12 14:05:47.016646 2014-08-13 17:38:02.734023 7fe9bae887a0 20 read_log 194209'15106 (194209'15063) delete 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:1260 2014-08-12 14:05:47.020476 2014-08-13 17:38:02.734035 7fe9bae887a0 20 read_log 194209'15107 (194209'15064) delete 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:1261 2014-08-12 14:05:47.024320 2014-08-13 17:38:02.734047 7fe9bae887a0 20 read_log 194209'15108 (194209'15065) delete 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:1262 2014-08-12 14:05:47.028325 2014-08-13 17:38:02.734061 7fe9bae887a0 20 read_log 194209'15109 (192888'14889) clean 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:1263 2014-08-12 14:05:47.028782 2014-08-13 17:38:02.734074 7fe9bae887a0 20 read_log 194209'15110 (194209'15066) delete 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:1264 2014-08-12 14:05:47.032468 2014-08-13 17:38:02.734088 7fe9bae887a0 20 read_log 194209'15111 (194209'15068) delete 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:1265 2014-08-12 14:05:47.035706 2014-08-13 17:38:02.734102 7fe9bae887a0 20 read_log 194209'15112 (194209'15069) delete f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:1266 2014-08-12 14:05:47.036107 2014-08-13 17:38:02.734114 7fe9bae887a0 20 read_log 194209'15113 (194209'15070) delete 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:1267 2014-08-12 14:05:47.039721 2014-08-13 17:38:02.734128 7fe9bae887a0 20 read_log 194209'15114 (194209'15071) delete fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:1268 2014-08-12 14:05:47.040093 2014-08-13 17:38:02.734142 7fe9bae887a0 20 read_log 194209'15115 (194209'15075) delete b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:1269 2014-08-12 14:05:47.043939 2014-08-13 17:38:02.734161 7fe9bae887a0 20 read_log 194209'15116 (194209'15076) delete a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:1270 2014-08-12 14:05:47.044241 2014-08-13 17:38:02.734174 7fe9bae887a0 20 read_log 194209'15117 (194209'15079) delete 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:1271 2014-08-12 14:05:47.047886 2014-08-13 17:38:02.734196 7fe9bae887a0 20 read_log 194209'15118 (194209'15080) delete 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:1272 2014-08-12 14:05:47.048209 2014-08-13 17:38:02.734211 7fe9bae887a0 20 read_log 194209'15119 (194209'15081) delete dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:1273 2014-08-12 14:05:47.051660 2014-08-13 17:38:02.734223 7fe9bae887a0 20 read_log 194209'15120 (194209'15082) delete 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:1274 2014-08-12 14:05:47.051986 2014-08-13 17:38:02.734237 7fe9bae887a0 20 read_log 194209'15121 (194209'15083) delete 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:1275 2014-08-12 14:05:47.055494 2014-08-13 17:38:02.734251 7fe9bae887a0 20 read_log 194209'15122 (194209'15086) delete 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:1276 2014-08-12 14:05:47.055992 2014-08-13 17:38:02.734264 7fe9bae887a0 20 read_log 194209'15123 (194209'15087) delete 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:1277 2014-08-12 14:05:47.059487 2014-08-13 17:38:02.734278 7fe9bae887a0 20 read_log 194209'15124 (194209'15088) delete 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:1278 2014-08-12 14:05:47.059815 2014-08-13 17:38:02.734290 7fe9bae887a0 20 read_log 194209'15125 (194209'15089) delete a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:1279 2014-08-12 14:05:47.063392 2014-08-13 17:38:02.734302 7fe9bae887a0 20 read_log 194209'15126 (194209'15090) delete a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:1280 2014-08-12 14:05:47.063713 2014-08-13 17:38:02.734314 7fe9bae887a0 20 read_log 194209'15127 (194209'15091) delete 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:1281 2014-08-12 14:05:47.067094 2014-08-13 17:38:02.734326 7fe9bae887a0 20 read_log 194209'15128 (194209'15092) delete 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:1282 2014-08-12 14:05:47.067436 2014-08-13 17:38:02.734340 7fe9bae887a0 20 read_log 194209'15129 (194209'15094) delete 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:1283 2014-08-12 14:05:47.068215 2014-08-13 17:38:02.734353 7fe9bae887a0 20 read_log 194209'15130 (194209'15095) delete 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:1284 2014-08-12 14:05:47.072043 2014-08-13 17:38:02.734365 7fe9bae887a0 20 read_log 194209'15131 (194209'15097) delete 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:1285 2014-08-12 14:05:47.072595 2014-08-13 17:38:02.734379 7fe9bae887a0 20 read_log 194209'15132 (194209'15109) delete 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:1457 2014-08-12 14:07:07.667995 2014-08-13 17:38:02.734391 7fe9bae887a0 20 read_log 194209'15133 (194209'15053) delete 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:1458 2014-08-12 14:07:07.699362 2014-08-13 17:38:02.734405 7fe9bae887a0 20 read_log 194209'15134 (194209'15057) delete 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:1459 2014-08-12 14:07:07.702478 2014-08-13 17:38:02.734418 7fe9bae887a0 20 read_log 194209'15135 (194209'15058) delete 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:1460 2014-08-12 14:07:07.702804 2014-08-13 17:38:02.734432 7fe9bae887a0 20 read_log 194209'15136 (194209'15059) delete 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:1461 2014-08-12 14:07:07.703220 2014-08-13 17:38:02.734444 7fe9bae887a0 20 read_log 194209'15137 (194209'15060) delete c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:1462 2014-08-12 14:07:07.703508 2014-08-13 17:38:02.734467 7fe9bae887a0 20 read_log 194209'15138 (194209'15067) delete b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:1463 2014-08-12 14:07:07.706541 2014-08-13 17:38:02.734481 7fe9bae887a0 20 read_log 194209'15139 (194209'15072) delete 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:1464 2014-08-12 14:07:07.706997 2014-08-13 17:38:02.734494 7fe9bae887a0 20 read_log 194209'15140 (194209'15073) delete ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:1465 2014-08-12 14:07:07.707629 2014-08-13 17:38:02.734508 7fe9bae887a0 20 read_log 194209'15141 (194209'15074) delete f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:1466 2014-08-12 14:07:07.707919 2014-08-13 17:38:02.734520 7fe9bae887a0 20 read_log 194209'15142 (194209'15077) delete e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:1467 2014-08-12 14:07:07.711092 2014-08-13 17:38:02.734534 7fe9bae887a0 20 read_log 194209'15143 (194209'15078) delete 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:1468 2014-08-12 14:07:07.711438 2014-08-13 17:38:02.734546 7fe9bae887a0 20 read_log 194209'15144 (194209'15084) delete 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:1469 2014-08-12 14:07:07.712111 2014-08-13 17:38:02.734560 7fe9bae887a0 20 read_log 194209'15145 (194209'15085) delete f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:1470 2014-08-12 14:07:07.712397 2014-08-13 17:38:02.734572 7fe9bae887a0 20 read_log 194215'15146 (194209'15093) delete 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.2174595.0:8655 0.000000 2014-08-13 17:38:02.734583 7fe9bae887a0 20 read_log 194215'15147 (194209'15096) delete 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.2174595.0:8657 0.000000 2014-08-13 17:38:02.734594 7fe9bae887a0 20 read_log 194215'15148 (192888'14671) clean 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by osd.20.0:1908 2014-08-12 15:36:44.627459 2014-08-13 17:38:02.734606 7fe9bae887a0 20 read_log 194215'15149 (194215'15148) delete 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.2174595.0:8659 0.000000 2014-08-13 17:38:02.734616 7fe9bae887a0 20 read_log 194215'15150 (192888'14887) clean 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by osd.20.0:1910 2014-08-12 15:36:44.635361 2014-08-13 17:38:02.734630 7fe9bae887a0 20 read_log 194215'15151 (194215'15150) delete 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.2174595.0:8661 0.000000 2014-08-13 17:38:02.734641 7fe9bae887a0 20 read_log 194215'15152 (193305'15036) clean 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by osd.20.0:1912 2014-08-12 15:36:44.642858 2014-08-13 17:38:02.734653 7fe9bae887a0 20 read_log 194215'15153 (194215'15152) delete 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.2174595.0:8663 0.000000 2014-08-13 17:38:02.734666 7fe9bae887a0 20 read_log 194215'15154 (192888'14906) clean 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by osd.20.0:1914 2014-08-12 15:36:44.650687 2014-08-13 17:38:02.734678 7fe9bae887a0 20 read_log 194215'15155 (194215'15154) delete 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.2174595.0:8665 0.000000 2014-08-13 17:38:02.734690 7fe9bae887a0 20 read_log 194215'15156 (193305'15049) clean c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by osd.20.0:1916 2014-08-12 15:36:44.657298 2014-08-13 17:38:02.734704 7fe9bae887a0 20 read_log 194215'15157 (194215'15156) delete c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.2174595.0:8667 0.000000 2014-08-13 17:38:02.734721 7fe9bae887a0 20 read_log 194215'15158 (193305'15018) clean 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by osd.20.0:1918 2014-08-12 15:36:44.664504 2014-08-13 17:38:02.734735 7fe9bae887a0 20 read_log 194215'15159 (194215'15158) delete 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.2174595.0:8669 0.000000 2014-08-13 17:38:02.734746 7fe9bae887a0 20 read_log 194215'15160 (193305'15027) clean bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by osd.20.0:1920 2014-08-12 15:36:44.672105 2014-08-13 17:38:02.734758 7fe9bae887a0 20 read_log 194215'15161 (194215'15160) delete bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.2174595.0:8671 0.000000 2014-08-13 17:38:02.734769 7fe9bae887a0 20 read_log 194215'15162 (192834'13894) clean 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by osd.20.0:1922 2014-08-12 15:36:44.678986 2014-08-13 17:38:02.734781 7fe9bae887a0 20 read_log 194215'15163 (194215'15162) delete 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.2174595.0:8673 0.000000 2014-08-13 17:38:02.734809 7fe9bae887a0 20 read_log 194215'15164 (192834'13847) clean d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by osd.20.0:1924 2014-08-12 15:36:44.686694 2014-08-13 17:38:02.734824 7fe9bae887a0 20 read_log 194215'15165 (194215'15164) delete d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.2174595.0:8675 0.000000 2014-08-13 17:38:02.734835 7fe9bae887a0 20 read_log 194215'15166 (192834'13812) clean 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by osd.20.0:1926 2014-08-12 15:36:44.694470 2014-08-13 17:38:02.734847 7fe9bae887a0 20 read_log 194215'15167 (194215'15166) delete 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.2174595.0:8677 0.000000 2014-08-13 17:38:02.734858 7fe9bae887a0 20 read_log 194215'15168 (192834'14519) clean a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by osd.20.0:1928 2014-08-12 15:36:44.702010 2014-08-13 17:38:02.734870 7fe9bae887a0 20 read_log 194215'15169 (194215'15168) delete a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.2174595.0:8679 0.000000 2014-08-13 17:38:02.734881 7fe9bae887a0 20 read_log 194215'15170 (192888'14673) clean 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by osd.20.0:1930 2014-08-12 15:36:44.710029 2014-08-13 17:38:02.734895 7fe9bae887a0 20 read_log 194215'15171 (194215'15170) delete 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.2174595.0:8681 0.000000 2014-08-13 17:38:02.734906 7fe9bae887a0 20 read_log 194215'15172 (192834'14548) clean a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by osd.20.0:1932 2014-08-12 15:36:44.717021 2014-08-13 17:38:02.734918 7fe9bae887a0 20 read_log 194215'15173 (194215'15172) delete a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.2174595.0:8683 0.000000 2014-08-13 17:38:02.734929 7fe9bae887a0 20 read_log 194215'15174 (192888'14689) clean 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by osd.20.0:1934 2014-08-12 15:36:44.724679 2014-08-13 17:38:02.734941 7fe9bae887a0 20 read_log 194215'15175 (194215'15174) delete 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.2174595.0:8685 0.000000 2014-08-13 17:38:02.734951 7fe9bae887a0 20 read_log 194215'15176 (192834'14486) clean 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by osd.20.0:1936 2014-08-12 15:36:44.731917 2014-08-13 17:38:02.734963 7fe9bae887a0 20 read_log 194215'15177 (194215'15176) delete 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.2174595.0:8687 0.000000 2014-08-13 17:38:02.734982 7fe9bae887a0 20 read_log 194215'15178 (193305'15035) clean b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by osd.20.0:1938 2014-08-12 15:36:44.739372 2014-08-13 17:38:02.734995 7fe9bae887a0 20 read_log 194215'15179 (194215'15178) delete b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.2174595.0:8689 0.000000 2014-08-13 17:38:02.735005 7fe9bae887a0 20 read_log 194215'15180 (192834'13827) clean cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by osd.20.0:1940 2014-08-12 15:36:44.747117 2014-08-13 17:38:02.735017 7fe9bae887a0 20 read_log 194215'15181 (194215'15180) delete cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.2174595.0:8691 0.000000 2014-08-13 17:38:02.735028 7fe9bae887a0 20 read_log 194215'15182 (192888'14869) clean 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by osd.20.0:1942 2014-08-12 15:36:44.755253 2014-08-13 17:38:02.735040 7fe9bae887a0 20 read_log 194215'15183 (194215'15182) delete 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.2174595.0:8693 0.000000 2014-08-13 17:38:02.735051 7fe9bae887a0 20 read_log 194215'15184 (192834'13775) clean 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by osd.20.0:1944 2014-08-12 15:36:44.763213 2014-08-13 17:38:02.735063 7fe9bae887a0 20 read_log 194215'15185 (194215'15184) delete 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.2174595.0:8695 0.000000 2014-08-13 17:38:02.735074 7fe9bae887a0 20 read_log 194215'15186 (192834'14482) clean 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by osd.20.0:1946 2014-08-12 15:36:44.770514 2014-08-13 17:38:02.735086 7fe9bae887a0 20 read_log 194215'15187 (194215'15186) delete 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.2174595.0:8697 0.000000 2014-08-13 17:38:02.735096 7fe9bae887a0 20 read_log 194215'15188 (192834'14457) clean 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by osd.20.0:1948 2014-08-12 15:36:44.778547 2014-08-13 17:38:02.735108 7fe9bae887a0 20 read_log 194215'15189 (194215'15188) delete 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.2174595.0:8699 0.000000 2014-08-13 17:38:02.735119 7fe9bae887a0 20 read_log 194215'15190 (192834'14537) clean 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by osd.20.0:1950 2014-08-12 15:36:44.785597 2014-08-13 17:38:02.735131 7fe9bae887a0 20 read_log 194215'15191 (194215'15190) delete 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.2174595.0:8701 0.000000 2014-08-13 17:38:02.735141 7fe9bae887a0 20 read_log 194215'15192 (192888'14687) clean ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by osd.20.0:1952 2014-08-12 15:36:44.794152 2014-08-13 17:38:02.735162 7fe9bae887a0 20 read_log 194215'15193 (194215'15192) delete ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.2174595.0:8703 0.000000 2014-08-13 17:38:02.735174 7fe9bae887a0 20 read_log 194215'15194 (192834'13868) clean 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by osd.20.0:1954 2014-08-12 15:36:44.802563 2014-08-13 17:38:02.735186 7fe9bae887a0 20 read_log 194215'15195 (194215'15194) delete 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.2174595.0:8705 0.000000 2014-08-13 17:38:02.735197 7fe9bae887a0 20 read_log 194215'15196 (192888'14883) clean 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by osd.20.0:1956 2014-08-12 15:36:44.810450 2014-08-13 17:38:02.735209 7fe9bae887a0 20 read_log 194215'15197 (194215'15196) delete 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.2174595.0:8707 0.000000 2014-08-13 17:38:02.735228 7fe9bae887a0 20 read_log 194215'15198 (193305'15025) clean 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by osd.20.0:1958 2014-08-12 15:36:44.817737 2014-08-13 17:38:02.735244 7fe9bae887a0 20 read_log 194215'15199 (194215'15198) delete 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.2174595.0:8709 0.000000 2014-08-13 17:38:02.735256 7fe9bae887a0 20 read_log 194215'15200 (193305'15034) clean a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by osd.20.0:1960 2014-08-12 15:36:44.825157 2014-08-13 17:38:02.735268 7fe9bae887a0 20 read_log 194215'15201 (194215'15200) delete a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.2174595.0:8711 0.000000 2014-08-13 17:38:02.735279 7fe9bae887a0 20 read_log 194215'15202 (192834'13759) clean b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by osd.20.0:1962 2014-08-12 15:36:44.832647 2014-08-13 17:38:02.735291 7fe9bae887a0 20 read_log 194215'15203 (194215'15202) delete b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.2174595.0:8713 0.000000 2014-08-13 17:38:02.735302 7fe9bae887a0 20 read_log 194215'15204 (193305'15046) clean 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by osd.20.0:1964 2014-08-12 15:36:44.841069 2014-08-13 17:38:02.735314 7fe9bae887a0 20 read_log 194215'15205 (194215'15204) delete 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.2174595.0:8715 0.000000 2014-08-13 17:38:02.735325 7fe9bae887a0 20 read_log 194215'15206 (192888'14888) clean 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by osd.20.0:1966 2014-08-12 15:36:44.848993 2014-08-13 17:38:02.735337 7fe9bae887a0 20 read_log 194215'15207 (194215'15206) delete 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.2174595.0:8717 0.000000 2014-08-13 17:38:02.735347 7fe9bae887a0 20 read_log 194215'15208 (192834'13764) clean b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by osd.20.0:1968 2014-08-12 15:36:44.857737 2014-08-13 17:38:02.735359 7fe9bae887a0 20 read_log 194215'15209 (194215'15208) delete b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.2174595.0:8719 0.000000 2014-08-13 17:38:02.735370 7fe9bae887a0 20 read_log 194215'15210 (193305'15048) clean e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by osd.20.0:1970 2014-08-12 15:36:44.866794 2014-08-13 17:38:02.735382 7fe9bae887a0 20 read_log 194215'15211 (194215'15210) delete e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.2174595.0:8721 0.000000 2014-08-13 17:38:02.735392 7fe9bae887a0 20 read_log 194215'15212 (192888'14660) clean 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by osd.20.0:1972 2014-08-12 15:36:44.875025 2014-08-13 17:38:02.735406 7fe9bae887a0 20 read_log 194215'15213 (194215'15212) delete 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.2174595.0:8723 0.000000 2014-08-13 17:38:02.735417 7fe9bae887a0 20 read_log 194215'15214 (193305'15038) clean 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by osd.20.0:1974 2014-08-12 15:36:44.882935 2014-08-13 17:38:02.735429 7fe9bae887a0 20 read_log 194215'15215 (194215'15214) delete 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.2174595.0:8725 0.000000 2014-08-13 17:38:02.735440 7fe9bae887a0 20 read_log 194215'15216 (193305'15013) clean d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by osd.20.0:1976 2014-08-12 15:36:44.890437 2014-08-13 17:38:02.735452 7fe9bae887a0 20 read_log 194215'15217 (194215'15216) delete d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.2174595.0:8727 0.000000 2014-08-13 17:38:02.735470 7fe9bae887a0 20 read_log 194215'15218 (192888'14880) clean cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by osd.20.0:1978 2014-08-12 15:36:44.898648 2014-08-13 17:38:02.735483 7fe9bae887a0 20 read_log 194215'15219 (194215'15218) delete cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.2174595.0:8729 0.000000 2014-08-13 17:38:02.735493 7fe9bae887a0 20 read_log 194215'15220 (192888'14903) clean 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by osd.20.0:1980 2014-08-12 15:36:44.906141 2014-08-13 17:38:02.735505 7fe9bae887a0 20 read_log 194215'15221 (194215'15220) delete 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.2174595.0:8731 0.000000 2014-08-13 17:38:02.735516 7fe9bae887a0 20 read_log 194215'15222 (193305'15010) clean 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by osd.20.0:1982 2014-08-12 15:36:44.914806 2014-08-13 17:38:02.735528 7fe9bae887a0 20 read_log 194215'15223 (194215'15222) delete 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.2174595.0:8733 0.000000 2014-08-13 17:38:02.735538 7fe9bae887a0 20 read_log 194215'15224 (192834'13906) clean 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by osd.20.0:1984 2014-08-12 15:36:44.923169 2014-08-13 17:38:02.735550 7fe9bae887a0 20 read_log 194215'15225 (194215'15224) delete 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.2174595.0:8735 0.000000 2014-08-13 17:38:02.735561 7fe9bae887a0 20 read_log 194215'15226 (192888'14899) clean 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by osd.20.0:1986 2014-08-12 15:36:44.931864 2014-08-13 17:38:02.735573 7fe9bae887a0 20 read_log 194215'15227 (194215'15226) delete 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.2174595.0:8737 0.000000 2014-08-13 17:38:02.735583 7fe9bae887a0 20 read_log 194215'15228 (192834'13879) clean 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by osd.20.0:1988 2014-08-12 15:36:44.940390 2014-08-13 17:38:02.735597 7fe9bae887a0 20 read_log 194215'15229 (194215'15228) delete 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.2174595.0:8739 0.000000 2014-08-13 17:38:02.735608 7fe9bae887a0 20 read_log 194215'15230 (192888'14658) clean 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by osd.20.0:1990 2014-08-12 15:36:44.948318 2014-08-13 17:38:02.735620 7fe9bae887a0 20 read_log 194215'15231 (194215'15230) delete 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.2174595.0:8741 0.000000 2014-08-13 17:38:02.735630 7fe9bae887a0 20 read_log 194215'15232 (193305'15017) clean 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by osd.20.0:1992 2014-08-12 15:36:44.956302 2014-08-13 17:38:02.735642 7fe9bae887a0 20 read_log 194215'15233 (194215'15232) delete 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.2174595.0:8743 0.000000 2014-08-13 17:38:02.735653 7fe9bae887a0 20 read_log 194215'15234 (192834'14352) clean 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by osd.20.0:1994 2014-08-12 15:36:44.963909 2014-08-13 17:38:02.735665 7fe9bae887a0 20 read_log 194215'15235 (194215'15234) delete 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.2174595.0:8745 0.000000 2014-08-13 17:38:02.735676 7fe9bae887a0 20 read_log 194215'15236 (192834'13765) clean 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by osd.20.0:1996 2014-08-12 15:36:44.971859 2014-08-13 17:38:02.735687 7fe9bae887a0 20 read_log 194215'15237 (194215'15236) delete 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.2174595.0:8747 0.000000 2014-08-13 17:38:02.735723 7fe9bae887a0 20 read_log 194215'15238 (192834'14474) clean 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by osd.20.0:1998 2014-08-12 15:36:44.980030 2014-08-13 17:38:02.735736 7fe9bae887a0 20 read_log 194215'15239 (194215'15238) delete 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.2174595.0:8749 0.000000 2014-08-13 17:38:02.735747 7fe9bae887a0 20 read_log 194215'15240 (192834'13860) clean 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by osd.20.0:2000 2014-08-12 15:36:44.988044 2014-08-13 17:38:02.735761 7fe9bae887a0 20 read_log 194215'15241 (194215'15240) delete 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.2174595.0:8751 0.000000 2014-08-13 17:38:02.735771 7fe9bae887a0 20 read_log 194215'15242 (192834'14497) clean 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by osd.20.0:2002 2014-08-12 15:36:44.997030 2014-08-13 17:38:02.735784 7fe9bae887a0 20 read_log 194215'15243 (194215'15242) delete 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.2174595.0:8753 0.000000 2014-08-13 17:38:02.735794 7fe9bae887a0 20 read_log 194215'15244 (193305'15042) clean 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by osd.20.0:2004 2014-08-12 15:36:45.005546 2014-08-13 17:38:02.735806 7fe9bae887a0 20 read_log 194215'15245 (194215'15244) delete 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.2174595.0:8755 0.000000 2014-08-13 17:38:02.735817 7fe9bae887a0 20 read_log 194215'15246 (192888'14902) clean fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by osd.20.0:2006 2014-08-12 15:36:45.013739 2014-08-13 17:38:02.735829 7fe9bae887a0 20 read_log 194215'15247 (194215'15246) delete fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.2174595.0:8757 0.000000 2014-08-13 17:38:02.735840 7fe9bae887a0 20 read_log 194215'15248 (192888'14854) clean 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by osd.20.0:2008 2014-08-12 15:36:45.021947 2014-08-13 17:38:02.735852 7fe9bae887a0 20 read_log 194215'15249 (194215'15248) delete 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.2174595.0:8759 0.000000 2014-08-13 17:38:02.735863 7fe9bae887a0 20 read_log 194215'15250 (192834'13867) clean bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by osd.20.0:2010 2014-08-12 15:36:45.030869 2014-08-13 17:38:02.735875 7fe9bae887a0 20 read_log 194215'15251 (194215'15250) delete bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.2174595.0:8761 0.000000 2014-08-13 17:38:02.735886 7fe9bae887a0 20 read_log 194215'15252 (193305'15014) clean 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by osd.20.0:2012 2014-08-12 15:36:45.038894 2014-08-13 17:38:02.735898 7fe9bae887a0 20 read_log 194215'15253 (194215'15252) delete 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.2174595.0:8763 0.000000 2014-08-13 17:38:02.735909 7fe9bae887a0 20 read_log 194215'15254 (193305'15028) clean 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by osd.20.0:2014 2014-08-12 15:36:45.047459 2014-08-13 17:38:02.735921 7fe9bae887a0 20 read_log 194215'15255 (194215'15254) delete 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.2174595.0:8765 0.000000 2014-08-13 17:38:02.735932 7fe9bae887a0 20 read_log 194215'15256 (193305'15045) clean 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by osd.20.0:2016 2014-08-12 15:36:45.055965 2014-08-13 17:38:02.735960 7fe9bae887a0 20 read_log 194215'15257 (194215'15256) delete 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.2174595.0:8767 0.000000 2014-08-13 17:38:02.735980 7fe9bae887a0 20 read_log 194215'15258 (192888'14884) clean 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by osd.20.0:2018 2014-08-12 15:36:45.064452 2014-08-13 17:38:02.735996 7fe9bae887a0 20 read_log 194215'15259 (194215'15258) delete 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.2174595.0:8769 0.000000 2014-08-13 17:38:02.736007 7fe9bae887a0 20 read_log 194215'15260 (192834'13782) clean c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by osd.20.0:2020 2014-08-12 15:36:45.072427 2014-08-13 17:38:02.736019 7fe9bae887a0 20 read_log 194215'15261 (194215'15260) delete c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.2174595.0:8771 0.000000 2014-08-13 17:38:02.736030 7fe9bae887a0 20 read_log 194215'15262 (192834'13805) clean 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by osd.20.0:2022 2014-08-12 15:36:45.080612 2014-08-13 17:38:02.736042 7fe9bae887a0 20 read_log 194215'15263 (194215'15262) delete 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.2174595.0:8773 0.000000 2014-08-13 17:38:02.736053 7fe9bae887a0 20 read_log 194215'15264 (192834'14379) clean 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by osd.20.0:2024 2014-08-12 15:36:45.088643 2014-08-13 17:38:02.736065 7fe9bae887a0 20 read_log 194215'15265 (194215'15264) delete 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.2174595.0:8775 0.000000 2014-08-13 17:38:02.736075 7fe9bae887a0 20 read_log 194215'15266 (193305'15039) clean 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by osd.20.0:2026 2014-08-12 15:36:45.097022 2014-08-13 17:38:02.736088 7fe9bae887a0 20 read_log 194215'15267 (194215'15266) delete 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.2174595.0:8777 0.000000 2014-08-13 17:38:02.736098 7fe9bae887a0 20 read_log 194215'15268 (192834'13800) clean c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by osd.20.0:2028 2014-08-12 15:36:45.104263 2014-08-13 17:38:02.736110 7fe9bae887a0 20 read_log 194215'15269 (194215'15268) delete c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.2174595.0:8779 0.000000 2014-08-13 17:38:02.736121 7fe9bae887a0 20 read_log 194215'15270 (192888'14867) clean c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by osd.20.0:2030 2014-08-12 15:36:45.111517 2014-08-13 17:38:02.736133 7fe9bae887a0 20 read_log 194215'15271 (194215'15270) delete c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.2174595.0:8781 0.000000 2014-08-13 17:38:02.736144 7fe9bae887a0 20 read_log 194215'15272 (192888'14881) clean 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by osd.20.0:2032 2014-08-12 15:36:45.119682 2014-08-13 17:38:02.736161 7fe9bae887a0 20 read_log 194215'15273 (194215'15272) delete 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.2174595.0:8783 0.000000 2014-08-13 17:38:02.736172 7fe9bae887a0 20 read_log 194215'15274 (193305'15031) clean 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by osd.20.0:2034 2014-08-12 15:36:45.127445 2014-08-13 17:38:02.736184 7fe9bae887a0 20 read_log 194215'15275 (194215'15274) delete 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.2174595.0:8785 0.000000 2014-08-13 17:38:02.736196 7fe9bae887a0 20 read_log 194215'15276 (193305'15037) clean 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by osd.20.0:2036 2014-08-12 15:36:45.134997 2014-08-13 17:38:02.736208 7fe9bae887a0 20 read_log 194215'15277 (194215'15276) delete 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.2174595.0:8787 0.000000 2014-08-13 17:38:02.736226 7fe9bae887a0 20 read_log 194215'15278 (193305'15012) clean 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by osd.20.0:2038 2014-08-12 15:36:45.142770 2014-08-13 17:38:02.736238 7fe9bae887a0 20 read_log 194215'15279 (194215'15278) delete 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.2174595.0:8789 0.000000 2014-08-13 17:38:02.736249 7fe9bae887a0 20 read_log 194215'15280 (192888'14872) clean 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by osd.20.0:2040 2014-08-12 15:36:45.149860 2014-08-13 17:38:02.736261 7fe9bae887a0 20 read_log 194215'15281 (194215'15280) delete 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.2174595.0:8791 0.000000 2014-08-13 17:38:02.736272 7fe9bae887a0 20 read_log 194215'15282 (192888'14900) clean 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by osd.20.0:2042 2014-08-12 15:36:45.157208 2014-08-13 17:38:02.736284 7fe9bae887a0 20 read_log 194215'15283 (194215'15282) delete 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.2174595.0:8793 0.000000 2014-08-13 17:38:02.736294 7fe9bae887a0 20 read_log 194215'15284 (192888'14692) clean 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by osd.20.0:2044 2014-08-12 15:36:45.163724 2014-08-13 17:38:02.736306 7fe9bae887a0 20 read_log 194215'15285 (194215'15284) delete 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.2174595.0:8795 0.000000 2014-08-13 17:38:02.736317 7fe9bae887a0 20 read_log 194215'15286 (192888'14909) clean f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by osd.20.0:2046 2014-08-12 15:36:45.171822 2014-08-13 17:38:02.736329 7fe9bae887a0 20 read_log 194215'15287 (194215'15286) delete f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.2174595.0:8797 0.000000 2014-08-13 17:38:02.736340 7fe9bae887a0 20 read_log 194215'15288 (192888'14919) clean 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by osd.20.0:2048 2014-08-12 15:36:45.179108 2014-08-13 17:38:02.736352 7fe9bae887a0 20 read_log 194215'15289 (194215'15288) delete 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.2174595.0:8799 0.000000 2014-08-13 17:38:02.736363 7fe9bae887a0 20 read_log 194215'15290 (192834'13820) clean 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by osd.20.0:2050 2014-08-12 15:36:45.186986 2014-08-13 17:38:02.736375 7fe9bae887a0 20 read_log 194215'15291 (194215'15290) delete 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.2174595.0:8801 0.000000 2014-08-13 17:38:02.736386 7fe9bae887a0 20 read_log 194215'15292 (192888'14681) clean ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by osd.20.0:2052 2014-08-12 15:36:45.194424 2014-08-13 17:38:02.736398 7fe9bae887a0 20 read_log 194215'15293 (194215'15292) delete ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.2174595.0:8803 0.000000 2014-08-13 17:38:02.736408 7fe9bae887a0 20 read_log 194215'15294 (192888'14910) clean 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by osd.20.0:2054 2014-08-12 15:36:45.201742 2014-08-13 17:38:02.736420 7fe9bae887a0 20 read_log 194215'15295 (194215'15294) delete 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.2174595.0:8805 0.000000 2014-08-13 17:38:02.736431 7fe9bae887a0 20 read_log 194215'15296 (193305'15047) clean a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by osd.20.0:2056 2014-08-12 15:36:45.209963 2014-08-13 17:38:02.736444 7fe9bae887a0 20 read_log 194215'15297 (194215'15296) delete a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.2174595.0:8807 0.000000 2014-08-13 17:38:02.736460 7fe9bae887a0 20 read_log 194215'15298 (192888'14655) clean d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by osd.20.0:2058 2014-08-12 15:36:45.217289 2014-08-13 17:38:02.736473 7fe9bae887a0 20 read_log 194215'15299 (194215'15298) delete d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.2174595.0:8809 0.000000 2014-08-13 17:38:02.736484 7fe9bae887a0 20 read_log 194215'15300 (193305'15044) clean b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by osd.20.0:2060 2014-08-12 15:36:45.224361 2014-08-13 17:38:02.736496 7fe9bae887a0 20 read_log 194215'15301 (194215'15300) delete b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.2174595.0:8811 0.000000 2014-08-13 17:38:02.736506 7fe9bae887a0 20 read_log 194215'15302 (192888'14858) clean 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by osd.20.0:2062 2014-08-12 15:36:45.231834 2014-08-13 17:38:02.736519 7fe9bae887a0 20 read_log 194215'15303 (194215'15302) delete 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.2174595.0:8813 0.000000 2014-08-13 17:38:02.736529 7fe9bae887a0 20 read_log 194215'15304 (192888'14678) clean d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by osd.20.0:2064 2014-08-12 15:36:45.240026 2014-08-13 17:38:02.736541 7fe9bae887a0 20 read_log 194215'15305 (194215'15304) delete d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.2174595.0:8815 0.000000 2014-08-13 17:38:02.736552 7fe9bae887a0 20 read_log 194215'15306 (192834'13854) clean 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by osd.20.0:2066 2014-08-12 15:36:45.247208 2014-08-13 17:38:02.736564 7fe9bae887a0 20 read_log 194215'15307 (194215'15306) delete 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.2174595.0:8817 0.000000 2014-08-13 17:38:02.736574 7fe9bae887a0 20 read_log 194215'15308 (193305'15024) clean 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by osd.20.0:2068 2014-08-12 15:36:45.254384 2014-08-13 17:38:02.736588 7fe9bae887a0 20 read_log 194215'15309 (194215'15308) delete 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.2174595.0:8819 0.000000 2014-08-13 17:38:02.736598 7fe9bae887a0 20 read_log 194215'15310 (192888'14676) clean 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by osd.20.0:2070 2014-08-12 15:36:45.262595 2014-08-13 17:38:02.736610 7fe9bae887a0 20 read_log 194215'15311 (194215'15310) delete 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.2174595.0:8821 0.000000 2014-08-13 17:38:02.736621 7fe9bae887a0 20 read_log 194215'15312 (192834'14438) clean d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by osd.20.0:2072 2014-08-12 15:36:45.270304 2014-08-13 17:38:02.736655 7fe9bae887a0 20 read_log 194215'15313 (194215'15312) delete d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.2174595.0:8823 0.000000 2014-08-13 17:38:02.736667 7fe9bae887a0 20 read_log 194215'15314 (192888'14911) clean a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by osd.20.0:2074 2014-08-12 15:36:45.277518 2014-08-13 17:38:02.736680 7fe9bae887a0 20 read_log 194215'15315 (194215'15314) delete a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.2174595.0:8825 0.000000 2014-08-13 17:38:02.736690 7fe9bae887a0 20 read_log 194215'15316 (192834'13838) clean a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by osd.20.0:2076 2014-08-12 15:36:45.285712 2014-08-13 17:38:02.736702 7fe9bae887a0 20 read_log 194215'15317 (194215'15316) delete a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.2174595.0:8827 0.000000 2014-08-13 17:38:02.736722 7fe9bae887a0 20 read_log 194215'15318 (193305'15011) clean fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by osd.20.0:2078 2014-08-12 15:36:45.295915 2014-08-13 17:38:02.736735 7fe9bae887a0 20 read_log 194215'15319 (194215'15318) delete fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.2174595.0:8829 0.000000 2014-08-13 17:38:02.736746 7fe9bae887a0 20 read_log 194215'15320 (192888'14863) clean badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by osd.20.0:2080 2014-08-12 15:36:45.303574 2014-08-13 17:38:02.736758 7fe9bae887a0 20 read_log 194215'15321 (194215'15320) delete badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.2174595.0:8831 0.000000 2014-08-13 17:38:02.736769 7fe9bae887a0 20 read_log 194215'15322 (193305'15029) clean e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by osd.20.0:2082 2014-08-12 15:36:45.312310 2014-08-13 17:38:02.736797 7fe9bae887a0 20 read_log 194215'15323 (194215'15322) delete e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.2174595.0:8833 0.000000 2014-08-13 17:38:02.736809 7fe9bae887a0 20 read_log 194215'15324 (192888'14670) clean ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by osd.20.0:2084 2014-08-12 15:36:45.320480 2014-08-13 17:38:02.736821 7fe9bae887a0 20 read_log 194215'15325 (194215'15324) delete ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.2174595.0:8835 0.000000 2014-08-13 17:38:02.736831 7fe9bae887a0 20 read_log 194215'15326 (192888'14668) clean 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by osd.20.0:2086 2014-08-12 15:36:45.328462 2014-08-13 17:38:02.736843 7fe9bae887a0 20 read_log 194215'15327 (194215'15326) delete 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.2174595.0:8837 0.000000 2014-08-13 17:38:02.736858 7fe9bae887a0 20 read_log 194215'15328 (192834'13885) clean 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by osd.20.0:2088 2014-08-12 15:36:45.335402 2014-08-13 17:38:02.736871 7fe9bae887a0 20 read_log 194215'15329 (194215'15328) delete 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.2174595.0:8839 0.000000 2014-08-13 17:38:02.736881 7fe9bae887a0 20 read_log 194215'15330 (192888'14664) clean 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by osd.20.0:2090 2014-08-12 15:36:45.342642 2014-08-13 17:38:02.736893 7fe9bae887a0 20 read_log 194215'15331 (194215'15330) delete 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.2174595.0:8841 0.000000 2014-08-13 17:38:02.736904 7fe9bae887a0 20 read_log 194215'15332 (192834'13772) clean 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by osd.20.0:2092 2014-08-12 15:36:45.350452 2014-08-13 17:38:02.736916 7fe9bae887a0 20 read_log 194215'15333 (194215'15332) delete 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.2174595.0:8843 0.000000 2014-08-13 17:38:02.736928 7fe9bae887a0 20 read_log 194217'15334 (194217'15334) modify 2c/hit_set_24.2c_archive_2014-08-12 15:32:25.737566_2014-08-12 17:25:03.718423/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 17:38:02.736940 7fe9bae887a0 20 read_log 194217'15335 (193305'14928) delete 2c/hit_set_24.2c_archive_2014-08-11 18:37:04.911332_2014-08-11 18:42:33.028272/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 17:38:02.736953 7fe9bae887a0 20 read_log 194217'15336 (0'0) promote 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by osd.20.0:4154 2014-08-12 17:25:03.734143 2014-08-13 17:38:02.736973 7fe9bae887a0 20 read_log 194217'15337 (194217'15336) modify 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by client.1487927.0:102 2014-08-12 17:25:03.710273 2014-08-13 17:38:02.736986 7fe9bae887a0 20 read_log 194217'15338 (0'0) promote a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by osd.20.0:4158 2014-08-12 17:25:03.872051 2014-08-13 17:38:02.736998 7fe9bae887a0 20 read_log 194217'15339 (194217'15338) modify a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by client.1487927.0:146 2014-08-12 17:25:03.864452 2014-08-13 17:38:02.737010 7fe9bae887a0 20 read_log 194370'15340 (0'0) promote e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by osd.20.0:4200 2014-08-12 20:30:19.536759 2014-08-13 17:38:02.737022 7fe9bae887a0 20 read_log 194370'15341 (194370'15340) modify e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by client.1508531.0:1373 2014-08-12 20:30:19.528413 2014-08-13 17:38:02.737035 7fe9bae887a0 20 read_log 194370'15342 (0'0) promote bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by osd.20.0:4202 2014-08-12 20:30:20.278092 2014-08-13 17:38:02.737047 7fe9bae887a0 20 read_log 194370'15343 (194370'15342) modify bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by client.1508531.0:1571 2014-08-12 20:30:20.269125 2014-08-13 17:38:02.737059 7fe9bae887a0 20 read_log 194370'15344 (0'0) promote 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by osd.20.0:4216 2014-08-12 20:30:22.042177 2014-08-13 17:38:02.737071 7fe9bae887a0 20 read_log 194370'15345 (194370'15344) modify 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by client.1508531.0:2033 2014-08-12 20:30:22.032396 2014-08-13 17:38:02.737083 7fe9bae887a0 20 read_log 194370'15346 (0'0) promote ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by osd.20.0:4236 2014-08-12 20:30:24.034883 2014-08-13 17:38:02.737097 7fe9bae887a0 20 read_log 194370'15347 (194370'15346) modify ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by client.1508531.0:2583 2014-08-12 20:30:24.026611 2014-08-13 17:38:02.737110 7fe9bae887a0 20 read_log 194370'15348 (0'0) promote 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by osd.20.0:4238 2014-08-12 20:30:24.464666 2014-08-13 17:38:02.737121 7fe9bae887a0 20 read_log 194370'15349 (194370'15348) modify 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by client.1508531.0:2702 2014-08-12 20:30:24.456839 2014-08-13 17:38:02.737151 7fe9bae887a0 20 read_log 194370'15350 (0'0) promote a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by osd.20.0:4260 2014-08-12 20:30:28.717201 2014-08-13 17:38:02.737173 7fe9bae887a0 20 read_log 194370'15351 (194370'15350) modify a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by client.1508531.0:3842 2014-08-12 20:30:28.710059 2014-08-13 17:38:02.737186 7fe9bae887a0 20 read_log 194370'15352 (0'0) promote 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by osd.20.0:4272 2014-08-12 20:30:30.667400 2014-08-13 17:38:02.737198 7fe9bae887a0 20 read_log 194370'15353 (194370'15352) modify 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by client.1508531.0:4358 2014-08-12 20:30:30.658820 2014-08-13 17:38:02.737210 7fe9bae887a0 20 read_log 194370'15354 (0'0) promote b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by osd.20.0:4288 2014-08-12 20:30:35.170349 2014-08-13 17:38:02.737222 7fe9bae887a0 20 read_log 194370'15355 (194370'15354) modify b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by client.1508531.0:5446 2014-08-12 20:30:35.162206 2014-08-13 17:38:02.737234 7fe9bae887a0 20 read_log 194370'15356 (0'0) promote f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by osd.20.0:4294 2014-08-12 20:30:37.799661 2014-08-13 17:38:02.737254 7fe9bae887a0 20 read_log 194370'15357 (194370'15356) modify f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by client.1508531.0:6016 2014-08-12 20:30:37.790483 2014-08-13 17:38:02.737268 7fe9bae887a0 20 read_log 194370'15358 (0'0) promote 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by osd.20.0:4308 2014-08-12 20:30:40.601390 2014-08-13 17:38:02.737281 7fe9bae887a0 20 read_log 194370'15359 (194370'15358) modify 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by client.1508531.0:6638 2014-08-12 20:30:40.590468 2014-08-13 17:38:02.737295 7fe9bae887a0 20 read_log 194370'15360 (0'0) promote 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by osd.20.0:4322 2014-08-12 20:30:43.151420 2014-08-13 17:38:02.737307 7fe9bae887a0 20 read_log 194370'15361 (194370'15360) modify 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by client.1508531.0:7156 2014-08-12 20:30:43.142664 2014-08-13 17:38:02.737319 7fe9bae887a0 20 read_log 194370'15362 (0'0) promote b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by osd.20.0:4326 2014-08-12 20:30:43.317583 2014-08-13 17:38:02.737331 7fe9bae887a0 20 read_log 194370'15363 (194370'15362) modify b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by client.1508531.0:7190 2014-08-12 20:30:43.308490 2014-08-13 17:38:02.737344 7fe9bae887a0 20 read_log 194370'15364 (0'0) promote 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by osd.20.0:4372 2014-08-12 20:30:53.223169 2014-08-13 17:38:02.737355 7fe9bae887a0 20 read_log 194370'15365 (194370'15364) modify 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by client.1508531.0:8983 2014-08-12 20:30:53.215484 2014-08-13 17:38:02.737368 7fe9bae887a0 20 read_log 194370'15366 (0'0) promote 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by osd.20.0:4376 2014-08-12 20:30:54.587347 2014-08-13 17:38:02.737379 7fe9bae887a0 20 read_log 194370'15367 (194370'15366) modify 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by client.1508531.0:9212 2014-08-12 20:30:54.579269 2014-08-13 17:38:02.737391 7fe9bae887a0 20 read_log 194370'15368 (0'0) promote 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by osd.20.0:4382 2014-08-12 20:30:55.184328 2014-08-13 17:38:02.737403 7fe9bae887a0 20 read_log 194370'15369 (194370'15368) modify 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by client.1508531.0:9311 2014-08-12 20:30:55.173988 2014-08-13 17:38:02.737415 7fe9bae887a0 20 read_log 194370'15370 (0'0) promote e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by osd.20.0:4394 2014-08-12 20:30:59.332921 2014-08-13 17:38:02.737427 7fe9bae887a0 20 read_log 194370'15371 (194370'15370) modify e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by client.1508531.0:10013 2014-08-12 20:30:59.324394 2014-08-13 17:38:02.737440 7fe9bae887a0 20 read_log 194370'15372 (0'0) promote b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by osd.20.0:4422 2014-08-12 20:31:05.215678 2014-08-13 17:38:02.737452 7fe9bae887a0 20 read_log 194370'15373 (194370'15372) modify b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by client.1508531.0:10992 2014-08-12 20:31:05.209034 2014-08-13 17:38:02.737465 7fe9bae887a0 20 read_log 194370'15374 (0'0) promote 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by osd.20.0:4448 2014-08-12 20:31:16.974350 2014-08-13 17:38:02.737477 7fe9bae887a0 20 read_log 194370'15375 (194370'15374) modify 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by client.1508531.0:12747 2014-08-12 20:31:16.966416 2014-08-13 17:38:02.737489 7fe9bae887a0 20 read_log 194370'15376 (0'0) promote 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by osd.20.0:4456 2014-08-12 20:31:18.607649 2014-08-13 17:38:02.737527 7fe9bae887a0 20 read_log 194370'15377 (194370'15376) modify 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by client.1508531.0:12920 2014-08-12 20:31:18.599908 2014-08-13 17:38:02.737543 7fe9bae887a0 20 read_log 194370'15378 (0'0) promote 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by osd.20.0:4466 2014-08-12 20:31:32.161593 2014-08-13 17:38:02.737555 7fe9bae887a0 20 read_log 194370'15379 (194370'15378) modify 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by client.1508531.0:13998 2014-08-12 20:31:32.153489 2014-08-13 17:38:02.737567 7fe9bae887a0 20 read_log 194377'15380 (0'0) promote e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by osd.20.0:4538 2014-08-12 20:33:09.241920 2014-08-13 17:38:02.737579 7fe9bae887a0 20 read_log 194377'15381 (194377'15380) modify e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by client.2231190.0:2972 2014-08-12 20:33:09.231970 2014-08-13 17:38:02.737592 7fe9bae887a0 20 read_log 194377'15382 (0'0) promote a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by osd.20.0:4570 2014-08-12 20:33:17.692054 2014-08-13 17:38:02.737604 7fe9bae887a0 20 read_log 194377'15383 (194377'15382) modify a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by client.2231190.0:4570 2014-08-12 20:33:17.683641 2014-08-13 17:38:02.737616 7fe9bae887a0 20 read_log 194377'15384 (0'0) promote de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by osd.20.0:4578 2014-08-12 20:33:18.942540 2014-08-13 17:38:02.737627 7fe9bae887a0 20 read_log 194377'15385 (194377'15384) modify de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by client.2231190.0:4744 2014-08-12 20:33:18.933995 2014-08-13 17:38:02.737640 7fe9bae887a0 20 read_log 194393'15386 (0'0) promote e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by osd.20.0:4664 2014-08-12 20:49:25.619826 2014-08-13 17:38:02.737652 7fe9bae887a0 20 read_log 194393'15387 (194393'15386) modify e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by client.1064182.0:43 2014-08-12 20:49:25.604653 2014-08-13 17:38:02.737664 7fe9bae887a0 20 read_log 194393'15388 (0'0) promote d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by osd.20.0:4675 2014-08-12 20:49:26.956715 2014-08-13 17:38:02.737676 7fe9bae887a0 20 read_log 194393'15389 (194393'15388) modify d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by client.1064182.0:318 2014-08-12 20:49:26.944410 2014-08-13 17:38:02.737688 7fe9bae887a0 20 read_log 194416'15390 (0'0) promote 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by osd.20.0:4903 2014-08-13 15:28:36.386970 2014-08-13 17:38:02.737700 7fe9bae887a0 20 read_log 194416'15391 (194416'15390) modify 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by client.2441958.0:1494 2014-08-13 15:28:36.375536 2014-08-13 17:38:02.737713 7fe9bae887a0 20 read_log 194416'15392 (0'0) promote 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by osd.20.0:4933 2014-08-13 15:28:39.365980 2014-08-13 17:38:02.737725 7fe9bae887a0 20 read_log 194416'15393 (194416'15392) modify 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by client.2441958.0:2288 2014-08-13 15:28:39.357910 2014-08-13 17:38:02.737737 7fe9bae887a0 20 read_log 194416'15394 (0'0) promote b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by osd.20.0:4981 2014-08-13 15:28:47.372315 2014-08-13 17:38:02.737749 7fe9bae887a0 20 read_log 194416'15395 (194416'15394) modify b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by client.2441958.0:4367 2014-08-13 15:28:47.365054 2014-08-13 17:38:02.737769 7fe9bae887a0 20 read_log 194416'15396 (0'0) promote dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by osd.20.0:4999 2014-08-13 15:28:51.343018 2014-08-13 17:38:02.737781 7fe9bae887a0 20 read_log 194416'15397 (194416'15396) modify dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by client.2441958.0:5380 2014-08-13 15:28:51.335442 2014-08-13 17:38:02.737793 7fe9bae887a0 20 read_log 194416'15398 (0'0) promote a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by osd.20.0:5017 2014-08-13 15:28:54.770300 2014-08-13 17:38:02.737805 7fe9bae887a0 20 read_log 194416'15399 (194416'15398) modify a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by client.2441958.0:6222 2014-08-13 15:28:54.761782 2014-08-13 17:38:02.737817 7fe9bae887a0 20 read_log 194416'15400 (0'0) promote eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by osd.20.0:5067 2014-08-13 15:29:03.748631 2014-08-13 17:38:02.737829 7fe9bae887a0 20 read_log 194416'15401 (194416'15400) modify eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by client.2441958.0:8589 2014-08-13 15:29:03.741402 2014-08-13 17:38:02.737841 7fe9bae887a0 20 read_log 194416'15402 (0'0) promote f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by osd.20.0:5095 2014-08-13 15:29:09.079007 2014-08-13 17:38:02.737853 7fe9bae887a0 20 read_log 194416'15403 (194416'15402) modify f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by client.2441958.0:9913 2014-08-13 15:29:09.071122 2014-08-13 17:38:02.737866 7fe9bae887a0 20 read_log 194416'15404 (0'0) promote f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by osd.20.0:5099 2014-08-13 15:29:09.431879 2014-08-13 17:38:02.737877 7fe9bae887a0 20 read_log 194416'15405 (194416'15404) modify f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by client.2441958.0:10002 2014-08-13 15:29:09.422426 2014-08-13 17:38:02.737890 7fe9bae887a0 20 read_log 194416'15406 (0'0) promote 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by osd.20.0:5105 2014-08-13 15:29:09.945666 2014-08-13 17:38:02.737901 7fe9bae887a0 20 read_log 194416'15407 (194416'15406) modify 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by client.2441958.0:10133 2014-08-13 15:29:09.937904 2014-08-13 17:38:02.737913 7fe9bae887a0 20 read_log 194416'15408 (0'0) promote 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by osd.20.0:5109 2014-08-13 15:29:10.597618 2014-08-13 17:38:02.737925 7fe9bae887a0 20 read_log 194416'15409 (194416'15408) modify 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by client.2441958.0:10279 2014-08-13 15:29:10.586826 2014-08-13 17:38:02.737937 7fe9bae887a0 20 read_log 194416'15410 (0'0) promote db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by osd.20.0:5175 2014-08-13 15:29:23.387230 2014-08-13 17:38:02.737949 7fe9bae887a0 20 read_log 194416'15411 (194416'15410) modify db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by client.2441958.0:13325 2014-08-13 15:29:23.378623 2014-08-13 17:38:02.737961 7fe9bae887a0 20 read_log 194416'15412 (0'0) promote ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by osd.20.0:5179 2014-08-13 15:29:25.662102 2014-08-13 17:38:02.737973 7fe9bae887a0 20 read_log 194416'15413 (194416'15412) modify ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by client.2441958.0:13859 2014-08-13 15:29:25.652448 2014-08-13 17:38:02.737986 7fe9bae887a0 20 read_log 194416'15414 (0'0) promote 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by osd.20.0:5191 2014-08-13 15:29:27.375636 2014-08-13 17:38:02.737999 7fe9bae887a0 20 read_log 194416'15415 (194416'15414) modify 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by client.2441958.0:14303 2014-08-13 15:29:27.368250 2014-08-13 17:38:02.738018 7fe9bae887a0 20 read_log 194416'15416 (0'0) promote 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by osd.20.0:5213 2014-08-13 15:29:31.715845 2014-08-13 17:38:02.738031 7fe9bae887a0 20 read_log 194416'15417 (194416'15416) modify 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by client.2441958.0:15323 2014-08-13 15:29:31.702841 2014-08-13 17:38:02.738043 7fe9bae887a0 20 read_log 194416'15418 (0'0) promote ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by osd.20.0:5217 2014-08-13 15:29:32.635883 2014-08-13 17:38:02.738055 7fe9bae887a0 20 read_log 194416'15419 (194416'15418) modify ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by client.2441958.0:15483 2014-08-13 15:29:32.626899 2014-08-13 17:38:02.738070 7fe9bae887a0 20 read_log 194416'15420 (0'0) promote 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by osd.20.0:5233 2014-08-13 15:29:37.244553 2014-08-13 17:38:02.738082 7fe9bae887a0 20 read_log 194416'15421 (194416'15420) modify 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by client.2441958.0:16262 2014-08-13 15:29:37.233451 2014-08-13 17:38:02.738096 7fe9bae887a0 20 read_log 194416'15422 (0'0) promote 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by osd.20.0:5241 2014-08-13 15:29:45.288896 2014-08-13 17:38:02.738108 7fe9bae887a0 20 read_log 194416'15423 (194416'15422) modify 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by client.2441958.0:16857 2014-08-13 15:29:45.277737 2014-08-13 17:38:02.738121 7fe9bae887a0 20 read_log 194422'15424 (0'0) promote 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by osd.20.0:5327 2014-08-13 15:32:20.502721 2014-08-13 17:38:02.738132 7fe9bae887a0 20 read_log 194422'15425 (194422'15424) modify 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by client.1148203.0:1300 2014-08-13 15:32:20.491615 2014-08-13 17:38:02.738147 7fe9bae887a0 20 read_log 194422'15426 (0'0) promote c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by osd.20.0:5359 2014-08-13 15:32:26.966964 2014-08-13 17:38:02.738163 7fe9bae887a0 20 read_log 194422'15427 (194422'15426) modify c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by client.1148203.0:2965 2014-08-13 15:32:26.958865 2014-08-13 17:38:02.738176 7fe9bae887a0 20 read_log 194422'15428 (0'0) promote 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by osd.20.0:5383 2014-08-13 15:32:31.317855 2014-08-13 17:38:02.738188 7fe9bae887a0 20 read_log 194422'15429 (194422'15428) modify 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by client.1148203.0:4082 2014-08-13 15:32:31.307985 2014-08-13 17:38:02.738200 7fe9bae887a0 20 read_log 194422'15430 (0'0) promote 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by osd.20.0:5393 2014-08-13 15:32:32.844747 2014-08-13 17:38:02.738212 7fe9bae887a0 20 read_log 194422'15431 (194422'15430) modify 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by client.1148203.0:4481 2014-08-13 15:32:32.836750 2014-08-13 17:38:02.738224 7fe9bae887a0 20 read_log 194422'15432 (0'0) promote 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by osd.20.0:5399 2014-08-13 15:32:33.750901 2014-08-13 17:38:02.738236 7fe9bae887a0 20 read_log 194422'15433 (194422'15432) modify 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by client.1148203.0:4714 2014-08-13 15:32:33.742155 2014-08-13 17:38:02.738249 7fe9bae887a0 20 read_log 194422'15434 (0'0) promote 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by osd.20.0:5403 2014-08-13 15:32:34.807661 2014-08-13 17:38:02.738261 7fe9bae887a0 20 read_log 194422'15435 (194422'15434) modify 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by client.1148203.0:4991 2014-08-13 15:32:34.798561 2014-08-13 17:38:02.738284 7fe9bae887a0 20 read_log 194422'15436 (0'0) promote 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by osd.20.0:5449 2014-08-13 15:32:47.898546 2014-08-13 17:38:02.738299 7fe9bae887a0 20 read_log 194422'15437 (194422'15436) modify 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by client.1148203.0:7388 2014-08-13 15:32:47.889207 2014-08-13 17:38:02.738311 7fe9bae887a0 20 read_log 194422'15438 (0'0) promote ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by osd.20.0:5461 2014-08-13 15:32:55.276201 2014-08-13 17:38:02.738323 7fe9bae887a0 20 read_log 194422'15439 (194422'15438) modify ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by client.1148203.0:8523 2014-08-13 15:32:55.268622 2014-08-13 17:38:02.738335 7fe9bae887a0 20 read_log 194422'15440 (0'0) promote 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by osd.20.0:5463 2014-08-13 15:32:55.665770 2014-08-13 17:38:02.738347 7fe9bae887a0 20 read_log 194422'15441 (194422'15440) modify 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by client.1148203.0:8576 2014-08-13 15:32:55.655971 2014-08-13 17:38:02.738359 7fe9bae887a0 20 read_log 194422'15442 (0'0) promote 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by osd.20.0:5481 2014-08-13 15:33:04.337860 2014-08-13 17:38:02.738412 7fe9bae887a0 20 read_log 194422'15443 (194422'15442) modify 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by client.1148203.0:9487 2014-08-13 15:33:04.327492 2014-08-13 17:38:02.738428 7fe9bae887a0 20 read_log 194422'15444 (0'0) promote 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by osd.20.0:5497 2014-08-13 15:33:16.735678 2014-08-13 17:38:02.738441 7fe9bae887a0 20 read_log 194422'15445 (194422'15444) modify 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by client.1148203.0:10239 2014-08-13 15:33:16.727346 2014-08-13 17:38:02.738456 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.738908 7fe9bae887a0 10 read_log checking for missing items over interval (194215'15333,194422'15445] 2014-08-13 17:38:02.738964 7fe9bae887a0 15 read_log missing 194422'15445 (194422'15444) modify 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by client.1148203.0:10239 2014-08-13 15:33:16.727346 2014-08-13 17:38:02.739010 7fe9bae887a0 15 read_log missing 194422'15443 (194422'15442) modify 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by client.1148203.0:9487 2014-08-13 15:33:04.327492 2014-08-13 17:38:02.739047 7fe9bae887a0 15 read_log missing 194422'15441 (194422'15440) modify 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by client.1148203.0:8576 2014-08-13 15:32:55.655971 2014-08-13 17:38:02.739084 7fe9bae887a0 15 read_log missing 194422'15439 (194422'15438) modify ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by client.1148203.0:8523 2014-08-13 15:32:55.268622 2014-08-13 17:38:02.739120 7fe9bae887a0 15 read_log missing 194422'15437 (194422'15436) modify 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by client.1148203.0:7388 2014-08-13 15:32:47.889207 2014-08-13 17:38:02.739163 7fe9bae887a0 15 read_log missing 194422'15435 (194422'15434) modify 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by client.1148203.0:4991 2014-08-13 15:32:34.798561 2014-08-13 17:38:02.739202 7fe9bae887a0 15 read_log missing 194422'15433 (194422'15432) modify 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by client.1148203.0:4714 2014-08-13 15:32:33.742155 2014-08-13 17:38:02.739238 7fe9bae887a0 15 read_log missing 194422'15431 (194422'15430) modify 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by client.1148203.0:4481 2014-08-13 15:32:32.836750 2014-08-13 17:38:02.739275 7fe9bae887a0 15 read_log missing 194422'15429 (194422'15428) modify 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by client.1148203.0:4082 2014-08-13 15:32:31.307985 2014-08-13 17:38:02.739312 7fe9bae887a0 15 read_log missing 194422'15427 (194422'15426) modify c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by client.1148203.0:2965 2014-08-13 15:32:26.958865 2014-08-13 17:38:02.739348 7fe9bae887a0 15 read_log missing 194422'15425 (194422'15424) modify 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by client.1148203.0:1300 2014-08-13 15:32:20.491615 2014-08-13 17:38:02.739385 7fe9bae887a0 15 read_log missing 194416'15423 (194416'15422) modify 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by client.2441958.0:16857 2014-08-13 15:29:45.277737 2014-08-13 17:38:02.739421 7fe9bae887a0 15 read_log missing 194416'15421 (194416'15420) modify 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by client.2441958.0:16262 2014-08-13 15:29:37.233451 2014-08-13 17:38:02.739458 7fe9bae887a0 15 read_log missing 194416'15419 (194416'15418) modify ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by client.2441958.0:15483 2014-08-13 15:29:32.626899 2014-08-13 17:38:02.739494 7fe9bae887a0 15 read_log missing 194416'15417 (194416'15416) modify 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by client.2441958.0:15323 2014-08-13 15:29:31.702841 2014-08-13 17:38:02.739531 7fe9bae887a0 15 read_log missing 194416'15415 (194416'15414) modify 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by client.2441958.0:14303 2014-08-13 15:29:27.368250 2014-08-13 17:38:02.739568 7fe9bae887a0 15 read_log missing 194416'15413 (194416'15412) modify ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by client.2441958.0:13859 2014-08-13 15:29:25.652448 2014-08-13 17:38:02.739604 7fe9bae887a0 15 read_log missing 194416'15411 (194416'15410) modify db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by client.2441958.0:13325 2014-08-13 15:29:23.378623 2014-08-13 17:38:02.739642 7fe9bae887a0 15 read_log missing 194416'15409 (194416'15408) modify 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by client.2441958.0:10279 2014-08-13 15:29:10.586826 2014-08-13 17:38:02.739684 7fe9bae887a0 15 read_log missing 194416'15407 (194416'15406) modify 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by client.2441958.0:10133 2014-08-13 15:29:09.937904 2014-08-13 17:38:02.739722 7fe9bae887a0 15 read_log missing 194416'15405 (194416'15404) modify f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by client.2441958.0:10002 2014-08-13 15:29:09.422426 2014-08-13 17:38:02.739759 7fe9bae887a0 15 read_log missing 194416'15403 (194416'15402) modify f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by client.2441958.0:9913 2014-08-13 15:29:09.071122 2014-08-13 17:38:02.739797 7fe9bae887a0 15 read_log missing 194416'15401 (194416'15400) modify eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by client.2441958.0:8589 2014-08-13 15:29:03.741402 2014-08-13 17:38:02.739833 7fe9bae887a0 15 read_log missing 194416'15399 (194416'15398) modify a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by client.2441958.0:6222 2014-08-13 15:28:54.761782 2014-08-13 17:38:02.739870 7fe9bae887a0 15 read_log missing 194416'15397 (194416'15396) modify dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by client.2441958.0:5380 2014-08-13 15:28:51.335442 2014-08-13 17:38:02.739907 7fe9bae887a0 15 read_log missing 194416'15395 (194416'15394) modify b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by client.2441958.0:4367 2014-08-13 15:28:47.365054 2014-08-13 17:38:02.739944 7fe9bae887a0 15 read_log missing 194416'15393 (194416'15392) modify 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by client.2441958.0:2288 2014-08-13 15:28:39.357910 2014-08-13 17:38:02.739981 7fe9bae887a0 15 read_log missing 194416'15391 (194416'15390) modify 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by client.2441958.0:1494 2014-08-13 15:28:36.375536 2014-08-13 17:38:02.740018 7fe9bae887a0 15 read_log missing 194393'15389 (194393'15388) modify d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by client.1064182.0:318 2014-08-12 20:49:26.944410 2014-08-13 17:38:02.740055 7fe9bae887a0 15 read_log missing 194393'15387 (194393'15386) modify e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by client.1064182.0:43 2014-08-12 20:49:25.604653 2014-08-13 17:38:02.740092 7fe9bae887a0 15 read_log missing 194377'15385 (194377'15384) modify de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by client.2231190.0:4744 2014-08-12 20:33:18.933995 2014-08-13 17:38:02.740129 7fe9bae887a0 15 read_log missing 194377'15383 (194377'15382) modify a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by client.2231190.0:4570 2014-08-12 20:33:17.683641 2014-08-13 17:38:02.740171 7fe9bae887a0 15 read_log missing 194377'15381 (194377'15380) modify e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by client.2231190.0:2972 2014-08-12 20:33:09.231970 2014-08-13 17:38:02.740209 7fe9bae887a0 15 read_log missing 194370'15379 (194370'15378) modify 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by client.1508531.0:13998 2014-08-12 20:31:32.153489 2014-08-13 17:38:02.740246 7fe9bae887a0 15 read_log missing 194370'15377 (194370'15376) modify 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by client.1508531.0:12920 2014-08-12 20:31:18.599908 2014-08-13 17:38:02.740283 7fe9bae887a0 15 read_log missing 194370'15375 (194370'15374) modify 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by client.1508531.0:12747 2014-08-12 20:31:16.966416 2014-08-13 17:38:02.740322 7fe9bae887a0 15 read_log missing 194370'15373 (194370'15372) modify b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by client.1508531.0:10992 2014-08-12 20:31:05.209034 2014-08-13 17:38:02.740381 7fe9bae887a0 15 read_log missing 194370'15371 (194370'15370) modify e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by client.1508531.0:10013 2014-08-12 20:30:59.324394 2014-08-13 17:38:02.740420 7fe9bae887a0 15 read_log missing 194370'15369 (194370'15368) modify 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by client.1508531.0:9311 2014-08-12 20:30:55.173988 2014-08-13 17:38:02.740458 7fe9bae887a0 15 read_log missing 194370'15367 (194370'15366) modify 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by client.1508531.0:9212 2014-08-12 20:30:54.579269 2014-08-13 17:38:02.740495 7fe9bae887a0 15 read_log missing 194370'15365 (194370'15364) modify 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by client.1508531.0:8983 2014-08-12 20:30:53.215484 2014-08-13 17:38:02.740532 7fe9bae887a0 15 read_log missing 194370'15363 (194370'15362) modify b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by client.1508531.0:7190 2014-08-12 20:30:43.308490 2014-08-13 17:38:02.740569 7fe9bae887a0 15 read_log missing 194370'15361 (194370'15360) modify 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by client.1508531.0:7156 2014-08-12 20:30:43.142664 2014-08-13 17:38:02.740606 7fe9bae887a0 15 read_log missing 194370'15359 (194370'15358) modify 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by client.1508531.0:6638 2014-08-12 20:30:40.590468 2014-08-13 17:38:02.740643 7fe9bae887a0 15 read_log missing 194370'15357 (194370'15356) modify f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by client.1508531.0:6016 2014-08-12 20:30:37.790483 2014-08-13 17:38:02.740681 7fe9bae887a0 15 read_log missing 194370'15355 (194370'15354) modify b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by client.1508531.0:5446 2014-08-12 20:30:35.162206 2014-08-13 17:38:02.740718 7fe9bae887a0 15 read_log missing 194370'15353 (194370'15352) modify 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by client.1508531.0:4358 2014-08-12 20:30:30.658820 2014-08-13 17:38:02.740755 7fe9bae887a0 15 read_log missing 194370'15351 (194370'15350) modify a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by client.1508531.0:3842 2014-08-12 20:30:28.710059 2014-08-13 17:38:02.740793 7fe9bae887a0 15 read_log missing 194370'15349 (194370'15348) modify 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by client.1508531.0:2702 2014-08-12 20:30:24.456839 2014-08-13 17:38:02.740830 7fe9bae887a0 15 read_log missing 194370'15347 (194370'15346) modify ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by client.1508531.0:2583 2014-08-12 20:30:24.026611 2014-08-13 17:38:02.740867 7fe9bae887a0 15 read_log missing 194370'15345 (194370'15344) modify 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by client.1508531.0:2033 2014-08-12 20:30:22.032396 2014-08-13 17:38:02.740904 7fe9bae887a0 15 read_log missing 194370'15343 (194370'15342) modify bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by client.1508531.0:1571 2014-08-12 20:30:20.269125 2014-08-13 17:38:02.740941 7fe9bae887a0 15 read_log missing 194370'15341 (194370'15340) modify e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by client.1508531.0:1373 2014-08-12 20:30:19.528413 2014-08-13 17:38:02.740977 7fe9bae887a0 15 read_log missing 194217'15339 (194217'15338) modify a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by client.1487927.0:146 2014-08-12 17:25:03.864452 2014-08-13 17:38:02.741014 7fe9bae887a0 15 read_log missing 194217'15337 (194217'15336) modify 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by client.1487927.0:102 2014-08-12 17:25:03.710273 2014-08-13 17:38:02.741055 7fe9bae887a0 15 read_log missing 194217'15334 (194217'15334) modify 2c/hit_set_24.2c_archive_2014-08-12 15:32:25.737566_2014-08-12 17:25:03.718423/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 17:38:02.741071 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.741103 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 17:38:02.741121 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.014703 0 0.000000 2014-08-13 17:38:02.741133 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 17:38:02.741143 7fe9bae887a0 20 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194470 2014-08-13 17:38:02.741153 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] log((192718'12964,194422'15445], crt=194422'15445) 2014-08-13 17:38:02.741169 7fe9bae887a0 10 osd.4 194478 pgid 24.34 coll 24.34_head 2014-08-13 17:38:02.741295 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.34 2014-08-13 17:38:02.741308 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.741350 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.34(unlocked)] enter Initial 2014-08-13 17:38:02.741361 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.34(unlocked)] enter NotTrimming 2014-08-13 17:38:02.741540 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.741657 7fe9bae887a0 20 read_log 192718'11511 (192718'11510) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:76 2014-08-06 14:53:21.371745 2014-08-13 17:38:02.741678 7fe9bae887a0 20 read_log 192718'11512 (192718'11511) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:77 2014-08-06 14:53:21.398763 2014-08-13 17:38:02.741691 7fe9bae887a0 20 read_log 192718'11513 (192718'11512) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:78 2014-08-06 14:53:21.425979 2014-08-13 17:38:02.741704 7fe9bae887a0 20 read_log 192718'11514 (192718'11513) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:79 2014-08-06 14:53:21.454713 2014-08-13 17:38:02.741716 7fe9bae887a0 20 read_log 192718'11515 (192718'11514) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:80 2014-08-06 14:53:21.485370 2014-08-13 17:38:02.741727 7fe9bae887a0 20 read_log 192718'11516 (192718'11515) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:81 2014-08-06 14:53:21.513935 2014-08-13 17:38:02.741739 7fe9bae887a0 20 read_log 192718'11517 (192718'11516) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:82 2014-08-06 14:53:21.542619 2014-08-13 17:38:02.741751 7fe9bae887a0 20 read_log 192718'11518 (192718'11517) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:83 2014-08-06 14:53:21.570695 2014-08-13 17:38:02.741763 7fe9bae887a0 20 read_log 192718'11519 (192718'11518) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:84 2014-08-06 14:53:21.601072 2014-08-13 17:38:02.741785 7fe9bae887a0 20 read_log 192718'11520 (192718'11519) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:85 2014-08-06 14:53:21.631007 2014-08-13 17:38:02.741800 7fe9bae887a0 20 read_log 192718'11521 (192718'11520) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:86 2014-08-06 14:53:21.661151 2014-08-13 17:38:02.741812 7fe9bae887a0 20 read_log 192718'11522 (192718'11521) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:87 2014-08-06 14:53:21.690380 2014-08-13 17:38:02.741826 7fe9bae887a0 20 read_log 192718'11523 (192718'11522) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:88 2014-08-06 14:53:21.717771 2014-08-13 17:38:02.741839 7fe9bae887a0 20 read_log 192718'11524 (192718'11523) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:89 2014-08-06 14:53:21.745457 2014-08-13 17:38:02.741851 7fe9bae887a0 20 read_log 192718'11525 (192718'11524) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:90 2014-08-06 14:53:21.772985 2014-08-13 17:38:02.741863 7fe9bae887a0 20 read_log 192718'11526 (192718'11525) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:91 2014-08-06 14:53:21.801243 2014-08-13 17:38:02.741874 7fe9bae887a0 20 read_log 192718'11527 (192718'11526) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:92 2014-08-06 14:53:21.830745 2014-08-13 17:38:02.741886 7fe9bae887a0 20 read_log 192718'11528 (192718'11527) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:93 2014-08-06 14:53:21.857999 2014-08-13 17:38:02.741898 7fe9bae887a0 20 read_log 192718'11529 (192718'11528) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:94 2014-08-06 14:53:21.886302 2014-08-13 17:38:02.741910 7fe9bae887a0 20 read_log 192718'11530 (192718'11529) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:95 2014-08-06 14:53:21.913830 2014-08-13 17:38:02.741921 7fe9bae887a0 20 read_log 192718'11531 (192718'11530) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:96 2014-08-06 14:53:22.391659 2014-08-13 17:38:02.741933 7fe9bae887a0 20 read_log 192718'11532 (192718'11531) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:97 2014-08-06 14:53:22.419199 2014-08-13 17:38:02.741945 7fe9bae887a0 20 read_log 192718'11533 (192718'11532) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:98 2014-08-06 14:53:22.931082 2014-08-13 17:38:02.741956 7fe9bae887a0 20 read_log 192718'11534 (192718'11533) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:99 2014-08-06 14:53:22.961517 2014-08-13 17:38:02.741968 7fe9bae887a0 20 read_log 192718'11535 (192718'11534) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:100 2014-08-06 14:53:22.989073 2014-08-13 17:38:02.741980 7fe9bae887a0 20 read_log 192718'11536 (192718'11535) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:101 2014-08-06 14:53:23.014038 2014-08-13 17:38:02.741992 7fe9bae887a0 20 read_log 192718'11537 (192718'11536) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:102 2014-08-06 14:53:23.346989 2014-08-13 17:38:02.742003 7fe9bae887a0 20 read_log 192718'11538 (192718'11537) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:103 2014-08-06 14:53:23.374668 2014-08-13 17:38:02.742016 7fe9bae887a0 20 read_log 192718'11539 (192718'11538) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:104 2014-08-06 14:53:23.402932 2014-08-13 17:38:02.742030 7fe9bae887a0 20 read_log 192718'11540 (192718'11539) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:105 2014-08-06 14:53:23.428134 2014-08-13 17:38:02.742042 7fe9bae887a0 20 read_log 192718'11541 (192718'11540) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:106 2014-08-06 14:53:23.456545 2014-08-13 17:38:02.742054 7fe9bae887a0 20 read_log 192718'11542 (192718'11541) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:107 2014-08-06 14:53:23.488256 2014-08-13 17:38:02.742066 7fe9bae887a0 20 read_log 192718'11543 (192718'11542) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:108 2014-08-06 14:53:23.516400 2014-08-13 17:38:02.742103 7fe9bae887a0 20 read_log 192718'11544 (192718'11543) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:109 2014-08-06 14:53:23.545548 2014-08-13 17:38:02.742119 7fe9bae887a0 20 read_log 192718'11545 (192718'11544) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:110 2014-08-06 14:53:23.938902 2014-08-13 17:38:02.742131 7fe9bae887a0 20 read_log 192718'11546 (192718'11545) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:111 2014-08-06 14:53:23.963656 2014-08-13 17:38:02.742143 7fe9bae887a0 20 read_log 192718'11547 (192718'11546) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:112 2014-08-06 14:53:23.986963 2014-08-13 17:38:02.742165 7fe9bae887a0 20 read_log 192718'11548 (192718'11547) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:113 2014-08-06 14:53:24.009786 2014-08-13 17:38:02.742181 7fe9bae887a0 20 read_log 192718'11549 (192718'11548) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:114 2014-08-06 14:53:24.036954 2014-08-13 17:38:02.742193 7fe9bae887a0 20 read_log 192718'11550 (192718'11549) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:115 2014-08-06 14:53:24.064570 2014-08-13 17:38:02.742206 7fe9bae887a0 20 read_log 192718'11551 (192718'11550) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:116 2014-08-06 14:53:24.091849 2014-08-13 17:38:02.742221 7fe9bae887a0 20 read_log 192718'11552 (192718'11551) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:117 2014-08-06 14:53:24.119437 2014-08-13 17:38:02.742235 7fe9bae887a0 20 read_log 192718'11553 (192718'11552) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:118 2014-08-06 14:53:24.147002 2014-08-13 17:38:02.742248 7fe9bae887a0 20 read_log 192718'11554 (192718'11553) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:119 2014-08-06 14:53:24.173819 2014-08-13 17:38:02.742261 7fe9bae887a0 20 read_log 192718'11555 (192718'11554) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:120 2014-08-06 14:53:24.201400 2014-08-13 17:38:02.742273 7fe9bae887a0 20 read_log 192718'11556 (192718'11555) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:121 2014-08-06 14:53:24.229356 2014-08-13 17:38:02.742286 7fe9bae887a0 20 read_log 192718'11557 (192718'11556) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:122 2014-08-06 14:53:24.259697 2014-08-13 17:38:02.742300 7fe9bae887a0 20 read_log 192718'11558 (192718'11557) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:123 2014-08-06 14:53:24.292136 2014-08-13 17:38:02.742312 7fe9bae887a0 20 read_log 192718'11559 (192718'11558) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:124 2014-08-06 14:53:24.317970 2014-08-13 17:38:02.742326 7fe9bae887a0 20 read_log 192718'11560 (192718'11559) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:125 2014-08-06 14:53:24.343984 2014-08-13 17:38:02.742338 7fe9bae887a0 20 read_log 192718'11561 (192718'11560) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:126 2014-08-06 14:53:24.835101 2014-08-13 17:38:02.742352 7fe9bae887a0 20 read_log 192718'11562 (192718'11561) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:127 2014-08-06 14:53:24.860893 2014-08-13 17:38:02.742364 7fe9bae887a0 20 read_log 192718'11563 (192718'11562) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:128 2014-08-06 14:53:24.883734 2014-08-13 17:38:02.742376 7fe9bae887a0 20 read_log 192718'11564 (192718'11563) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:129 2014-08-06 14:53:24.905971 2014-08-13 17:38:02.742390 7fe9bae887a0 20 read_log 192718'11565 (192718'11564) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:130 2014-08-06 14:53:24.932079 2014-08-13 17:38:02.742402 7fe9bae887a0 20 read_log 192718'11566 (192718'11565) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:131 2014-08-06 14:53:24.959006 2014-08-13 17:38:02.742417 7fe9bae887a0 20 read_log 192718'11567 (192718'11566) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:132 2014-08-06 14:53:24.986719 2014-08-13 17:38:02.742436 7fe9bae887a0 20 read_log 192718'11568 (192718'11567) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:133 2014-08-06 14:53:25.012277 2014-08-13 17:38:02.742450 7fe9bae887a0 20 read_log 192718'11569 (192718'11568) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:134 2014-08-06 14:53:25.039512 2014-08-13 17:38:02.742463 7fe9bae887a0 20 read_log 192718'11570 (192718'11569) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:135 2014-08-06 14:53:25.065389 2014-08-13 17:38:02.742475 7fe9bae887a0 20 read_log 192718'11571 (192718'11570) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:136 2014-08-06 14:53:25.092805 2014-08-13 17:38:02.742487 7fe9bae887a0 20 read_log 192718'11572 (192718'11571) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:137 2014-08-06 14:53:25.119376 2014-08-13 17:38:02.742502 7fe9bae887a0 20 read_log 192718'11573 (192718'11572) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:138 2014-08-06 14:53:25.146917 2014-08-13 17:38:02.742514 7fe9bae887a0 20 read_log 192718'11574 (192718'11573) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:139 2014-08-06 14:53:25.173320 2014-08-13 17:38:02.742527 7fe9bae887a0 20 read_log 192718'11575 (192718'11574) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:140 2014-08-06 14:53:25.202479 2014-08-13 17:38:02.742540 7fe9bae887a0 20 read_log 192718'11576 (192718'11575) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:141 2014-08-06 14:53:25.240628 2014-08-13 17:38:02.742554 7fe9bae887a0 20 read_log 192718'11577 (192718'11576) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:142 2014-08-06 14:53:25.283176 2014-08-13 17:38:02.742566 7fe9bae887a0 20 read_log 192718'11578 (192718'11577) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:143 2014-08-06 14:53:25.328199 2014-08-13 17:38:02.742580 7fe9bae887a0 20 read_log 192718'11579 (192718'11578) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:144 2014-08-06 14:53:25.373374 2014-08-13 17:38:02.742592 7fe9bae887a0 20 read_log 192718'11580 (192718'11579) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:145 2014-08-06 14:53:25.417791 2014-08-13 17:38:02.742606 7fe9bae887a0 20 read_log 192718'11581 (192718'11580) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:146 2014-08-06 14:53:25.468000 2014-08-13 17:38:02.742620 7fe9bae887a0 20 read_log 192718'11582 (192718'11581) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:147 2014-08-06 14:53:25.719433 2014-08-13 17:38:02.742632 7fe9bae887a0 20 read_log 192718'11583 (192718'11582) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:148 2014-08-06 14:53:25.775296 2014-08-13 17:38:02.742646 7fe9bae887a0 20 read_log 192718'11584 (192718'11583) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:149 2014-08-06 14:53:25.828385 2014-08-13 17:38:02.742658 7fe9bae887a0 20 read_log 192718'11585 (192718'11584) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:150 2014-08-06 14:53:25.884760 2014-08-13 17:38:02.742671 7fe9bae887a0 20 read_log 192718'11586 (192718'11585) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:151 2014-08-06 14:53:25.931886 2014-08-13 17:38:02.742683 7fe9bae887a0 20 read_log 192718'11587 (192718'11586) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:152 2014-08-06 14:53:25.981505 2014-08-13 17:38:02.742697 7fe9bae887a0 20 read_log 192718'11588 (192718'11587) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:153 2014-08-06 14:53:26.031616 2014-08-13 17:38:02.742709 7fe9bae887a0 20 read_log 192718'11589 (192718'11588) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:154 2014-08-06 14:53:26.093416 2014-08-13 17:38:02.742722 7fe9bae887a0 20 read_log 192718'11590 (192718'11589) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:155 2014-08-06 14:53:26.175936 2014-08-13 17:38:02.742736 7fe9bae887a0 20 read_log 192718'11591 (192718'11590) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:156 2014-08-06 14:53:26.250331 2014-08-13 17:38:02.742755 7fe9bae887a0 20 read_log 192718'11592 (192718'11591) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:157 2014-08-06 14:53:26.491009 2014-08-13 17:38:02.742769 7fe9bae887a0 20 read_log 192718'11593 (192718'11592) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:158 2014-08-06 14:53:26.962507 2014-08-13 17:38:02.742781 7fe9bae887a0 20 read_log 192718'11594 (192718'11593) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:159 2014-08-06 14:53:26.991441 2014-08-13 17:38:02.742793 7fe9bae887a0 20 read_log 192718'11595 (192718'11594) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:160 2014-08-06 14:53:27.021332 2014-08-13 17:38:02.742812 7fe9bae887a0 20 read_log 192718'11596 (192718'11595) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:161 2014-08-06 14:53:27.047224 2014-08-13 17:38:02.742826 7fe9bae887a0 20 read_log 192718'11597 (192718'11596) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:162 2014-08-06 14:53:27.077768 2014-08-13 17:38:02.742839 7fe9bae887a0 20 read_log 192718'11598 (192718'11597) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:163 2014-08-06 14:53:27.107026 2014-08-13 17:38:02.742852 7fe9bae887a0 20 read_log 192718'11599 (192718'11598) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:164 2014-08-06 14:53:27.136121 2014-08-13 17:38:02.742866 7fe9bae887a0 20 read_log 192718'11600 (192718'11599) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:165 2014-08-06 14:53:27.164664 2014-08-13 17:38:02.742878 7fe9bae887a0 20 read_log 192718'11601 (192718'11600) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:166 2014-08-06 14:53:27.302901 2014-08-13 17:38:02.742892 7fe9bae887a0 20 read_log 192718'11602 (192718'11601) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:167 2014-08-06 14:53:27.412708 2014-08-13 17:38:02.742906 7fe9bae887a0 20 read_log 192718'11603 (192718'11602) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:168 2014-08-06 14:53:27.452649 2014-08-13 17:38:02.742920 7fe9bae887a0 20 read_log 192718'11604 (192718'11603) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:169 2014-08-06 14:53:27.500193 2014-08-13 17:38:02.742932 7fe9bae887a0 20 read_log 192718'11605 (192718'11604) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:170 2014-08-06 14:53:27.527560 2014-08-13 17:38:02.742946 7fe9bae887a0 20 read_log 192718'11606 (192718'11605) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:171 2014-08-06 14:53:27.554913 2014-08-13 17:38:02.742958 7fe9bae887a0 20 read_log 192718'11607 (192718'11606) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:172 2014-08-06 14:53:27.581615 2014-08-13 17:38:02.742972 7fe9bae887a0 20 read_log 192718'11608 (192718'11607) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:173 2014-08-06 14:53:27.609010 2014-08-13 17:38:02.742984 7fe9bae887a0 20 read_log 192718'11609 (192718'11608) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:174 2014-08-06 14:53:27.643073 2014-08-13 17:38:02.742997 7fe9bae887a0 20 read_log 192718'11610 (192718'11609) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:175 2014-08-06 14:53:27.669013 2014-08-13 17:38:02.743011 7fe9bae887a0 20 read_log 192718'11611 (192718'11610) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:176 2014-08-06 14:53:27.697754 2014-08-13 17:38:02.743023 7fe9bae887a0 20 read_log 192718'11612 (192718'11611) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:177 2014-08-06 14:53:27.724639 2014-08-13 17:38:02.743037 7fe9bae887a0 20 read_log 192718'11613 (192718'11612) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:178 2014-08-06 14:53:27.753862 2014-08-13 17:38:02.743049 7fe9bae887a0 20 read_log 192718'11614 (192718'11613) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:179 2014-08-06 14:53:28.082822 2014-08-13 17:38:02.743062 7fe9bae887a0 20 read_log 192718'11615 (192718'11614) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:180 2014-08-06 14:53:28.110588 2014-08-13 17:38:02.743083 7fe9bae887a0 20 read_log 192718'11616 (192718'11615) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:181 2014-08-06 14:53:28.136725 2014-08-13 17:38:02.743098 7fe9bae887a0 20 read_log 192718'11617 (192718'11616) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:182 2014-08-06 14:53:28.164101 2014-08-13 17:38:02.743111 7fe9bae887a0 20 read_log 192718'11618 (192718'11617) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:183 2014-08-06 14:53:28.189933 2014-08-13 17:38:02.743123 7fe9bae887a0 20 read_log 192718'11619 (192718'11618) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:184 2014-08-06 14:53:28.216291 2014-08-13 17:38:02.743138 7fe9bae887a0 20 read_log 192718'11620 (192718'11619) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:185 2014-08-06 14:53:28.241924 2014-08-13 17:38:02.743150 7fe9bae887a0 20 read_log 192718'11621 (192718'11620) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:186 2014-08-06 14:53:28.268270 2014-08-13 17:38:02.743171 7fe9bae887a0 20 read_log 192718'11622 (192718'11621) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:187 2014-08-06 14:53:28.294466 2014-08-13 17:38:02.743184 7fe9bae887a0 20 read_log 192718'11623 (192718'11622) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:188 2014-08-06 14:53:28.320210 2014-08-13 17:38:02.743197 7fe9bae887a0 20 read_log 192718'11624 (192718'11623) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:189 2014-08-06 14:53:28.483523 2014-08-13 17:38:02.743209 7fe9bae887a0 20 read_log 192718'11625 (192718'11624) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:190 2014-08-06 14:53:28.525776 2014-08-13 17:38:02.743221 7fe9bae887a0 20 read_log 192718'11626 (192718'11625) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:191 2014-08-06 14:53:28.571402 2014-08-13 17:38:02.743234 7fe9bae887a0 20 read_log 192718'11627 (192718'11626) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:192 2014-08-06 14:53:28.625285 2014-08-13 17:38:02.743246 7fe9bae887a0 20 read_log 192718'11628 (192718'11627) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:193 2014-08-06 14:53:28.684016 2014-08-13 17:38:02.743261 7fe9bae887a0 20 read_log 192718'11629 (192718'11628) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:194 2014-08-06 14:53:28.740778 2014-08-13 17:38:02.743295 7fe9bae887a0 20 read_log 192718'11630 (192718'11629) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:195 2014-08-06 14:53:28.784731 2014-08-13 17:38:02.743307 7fe9bae887a0 20 read_log 192718'11631 (192718'11630) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:196 2014-08-06 14:53:28.823441 2014-08-13 17:38:02.743321 7fe9bae887a0 20 read_log 192718'11632 (192718'11631) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:197 2014-08-06 14:53:28.894562 2014-08-13 17:38:02.743333 7fe9bae887a0 20 read_log 192718'11633 (192718'11632) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:198 2014-08-06 14:53:28.958436 2014-08-13 17:38:02.743351 7fe9bae887a0 20 read_log 192718'11634 (192718'11633) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:199 2014-08-06 14:53:29.352032 2014-08-13 17:38:02.743363 7fe9bae887a0 20 read_log 192718'11635 (192718'11634) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:200 2014-08-06 14:53:29.430234 2014-08-13 17:38:02.743377 7fe9bae887a0 20 read_log 192718'11636 (192718'11635) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:201 2014-08-06 14:53:29.497991 2014-08-13 17:38:02.743389 7fe9bae887a0 20 read_log 192718'11637 (192718'11636) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:202 2014-08-06 14:53:29.581245 2014-08-13 17:38:02.743403 7fe9bae887a0 20 read_log 192718'11638 (192718'11637) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:203 2014-08-06 14:53:29.642221 2014-08-13 17:38:02.743415 7fe9bae887a0 20 read_log 192718'11639 (192718'11638) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:204 2014-08-06 14:53:29.679868 2014-08-13 17:38:02.743434 7fe9bae887a0 20 read_log 192718'11640 (192718'11639) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:205 2014-08-06 14:53:29.719016 2014-08-13 17:38:02.743448 7fe9bae887a0 20 read_log 192718'11641 (192718'11640) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:206 2014-08-06 14:53:29.771344 2014-08-13 17:38:02.743460 7fe9bae887a0 20 read_log 192718'11642 (192718'11641) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:207 2014-08-06 14:53:29.820431 2014-08-13 17:38:02.743472 7fe9bae887a0 20 read_log 192718'11643 (192718'11642) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:208 2014-08-06 14:53:29.854351 2014-08-13 17:38:02.743484 7fe9bae887a0 20 read_log 192718'11644 (192718'11643) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:209 2014-08-06 14:53:30.245018 2014-08-13 17:38:02.743496 7fe9bae887a0 20 read_log 192718'11645 (192718'11644) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:210 2014-08-06 14:53:30.296335 2014-08-13 17:38:02.743508 7fe9bae887a0 20 read_log 192718'11646 (192718'11645) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:211 2014-08-06 14:53:30.342482 2014-08-13 17:38:02.743520 7fe9bae887a0 20 read_log 192718'11647 (192718'11646) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:212 2014-08-06 14:53:30.388593 2014-08-13 17:38:02.743532 7fe9bae887a0 20 read_log 192718'11648 (192718'11647) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:213 2014-08-06 14:53:30.436047 2014-08-13 17:38:02.743544 7fe9bae887a0 20 read_log 192718'11649 (192718'11648) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:214 2014-08-06 14:53:30.482345 2014-08-13 17:38:02.743558 7fe9bae887a0 20 read_log 192718'11650 (192718'11649) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:215 2014-08-06 14:53:30.532798 2014-08-13 17:38:02.743570 7fe9bae887a0 20 read_log 192718'11651 (192718'11650) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:216 2014-08-06 14:53:30.595892 2014-08-13 17:38:02.743582 7fe9bae887a0 20 read_log 192718'11652 (192718'11651) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:217 2014-08-06 14:53:30.652734 2014-08-13 17:38:02.743593 7fe9bae887a0 20 read_log 192718'11653 (192718'11652) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:218 2014-08-06 14:53:30.710371 2014-08-13 17:38:02.743605 7fe9bae887a0 20 read_log 192718'11654 (192718'11653) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:219 2014-08-06 14:53:31.015504 2014-08-13 17:38:02.743617 7fe9bae887a0 20 read_log 192718'11655 (192718'11654) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:220 2014-08-06 14:53:31.077847 2014-08-13 17:38:02.743629 7fe9bae887a0 20 read_log 192718'11656 (192718'11655) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:221 2014-08-06 14:53:31.129612 2014-08-13 17:38:02.743641 7fe9bae887a0 20 read_log 192718'11657 (192718'11656) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:222 2014-08-06 14:53:31.750931 2014-08-13 17:38:02.743653 7fe9bae887a0 20 read_log 192718'11658 (192718'11657) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:223 2014-08-06 14:53:31.778106 2014-08-13 17:38:02.743665 7fe9bae887a0 20 read_log 192718'11659 (192718'11658) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:224 2014-08-06 14:53:31.802553 2014-08-13 17:38:02.743677 7fe9bae887a0 20 read_log 192718'11660 (192718'11659) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:225 2014-08-06 14:53:31.829562 2014-08-13 17:38:02.743689 7fe9bae887a0 20 read_log 192718'11661 (192718'11660) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:226 2014-08-06 14:53:31.857361 2014-08-13 17:38:02.743701 7fe9bae887a0 20 read_log 192718'11662 (192718'11661) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:227 2014-08-06 14:53:31.882503 2014-08-13 17:38:02.743713 7fe9bae887a0 20 read_log 192718'11663 (192718'11662) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:228 2014-08-06 14:53:31.908271 2014-08-13 17:38:02.743731 7fe9bae887a0 20 read_log 192718'11664 (192718'11663) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:229 2014-08-06 14:53:32.109759 2014-08-13 17:38:02.743744 7fe9bae887a0 20 read_log 192718'11665 (192718'11664) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:230 2014-08-06 14:53:32.164501 2014-08-13 17:38:02.743755 7fe9bae887a0 20 read_log 192718'11666 (192718'11665) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:231 2014-08-06 14:53:32.222681 2014-08-13 17:38:02.743768 7fe9bae887a0 20 read_log 192718'11667 (192718'11666) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:232 2014-08-06 14:53:32.272173 2014-08-13 17:38:02.743780 7fe9bae887a0 20 read_log 192718'11668 (192718'11667) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:233 2014-08-06 14:53:32.314488 2014-08-13 17:38:02.743792 7fe9bae887a0 20 read_log 192718'11669 (192718'11668) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:234 2014-08-06 14:53:32.347183 2014-08-13 17:38:02.743803 7fe9bae887a0 20 read_log 192718'11670 (192718'11669) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:235 2014-08-06 14:53:32.373319 2014-08-13 17:38:02.743815 7fe9bae887a0 20 read_log 192718'11671 (192718'11670) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:236 2014-08-06 14:53:32.400125 2014-08-13 17:38:02.743827 7fe9bae887a0 20 read_log 192718'11672 (192718'11671) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:237 2014-08-06 14:53:32.428469 2014-08-13 17:38:02.743839 7fe9bae887a0 20 read_log 192718'11673 (192718'11672) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:238 2014-08-06 14:53:32.454484 2014-08-13 17:38:02.743850 7fe9bae887a0 20 read_log 192718'11674 (192718'11673) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:239 2014-08-06 14:53:32.707937 2014-08-13 17:38:02.743862 7fe9bae887a0 20 read_log 192718'11675 (192718'11674) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:240 2014-08-06 14:53:32.763339 2014-08-13 17:38:02.743874 7fe9bae887a0 20 read_log 192718'11676 (192718'11675) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:241 2014-08-06 14:53:32.813403 2014-08-13 17:38:02.743886 7fe9bae887a0 20 read_log 192718'11677 (192718'11676) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:242 2014-08-06 14:53:32.873666 2014-08-13 17:38:02.743897 7fe9bae887a0 20 read_log 192718'11678 (192718'11677) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:243 2014-08-06 14:53:32.931520 2014-08-13 17:38:02.743909 7fe9bae887a0 20 read_log 192718'11679 (192718'11678) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:244 2014-08-06 14:53:32.958435 2014-08-13 17:38:02.743921 7fe9bae887a0 20 read_log 192718'11680 (192718'11679) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:245 2014-08-06 14:53:33.001428 2014-08-13 17:38:02.743933 7fe9bae887a0 20 read_log 192718'11681 (192718'11680) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:246 2014-08-06 14:53:33.046407 2014-08-13 17:38:02.743945 7fe9bae887a0 20 read_log 192718'11682 (192718'11681) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:247 2014-08-06 14:53:33.087721 2014-08-13 17:38:02.743957 7fe9bae887a0 20 read_log 192718'11683 (192718'11682) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:248 2014-08-06 14:53:33.131679 2014-08-13 17:38:02.743969 7fe9bae887a0 20 read_log 192718'11684 (192718'11683) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:249 2014-08-06 14:53:33.172441 2014-08-13 17:38:02.743980 7fe9bae887a0 20 read_log 192718'11685 (192718'11684) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:250 2014-08-06 14:53:33.211658 2014-08-13 17:38:02.743992 7fe9bae887a0 20 read_log 192718'11686 (192718'11685) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:251 2014-08-06 14:53:33.459391 2014-08-13 17:38:02.744004 7fe9bae887a0 20 read_log 192718'11687 (192718'11686) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:252 2014-08-06 14:53:33.505091 2014-08-13 17:38:02.744040 7fe9bae887a0 20 read_log 192718'11688 (192718'11687) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:253 2014-08-06 14:53:33.533773 2014-08-13 17:38:02.744053 7fe9bae887a0 20 read_log 192718'11689 (192718'11688) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:254 2014-08-06 14:53:33.560693 2014-08-13 17:38:02.744065 7fe9bae887a0 20 read_log 192718'11690 (192718'11689) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:255 2014-08-06 14:53:33.586282 2014-08-13 17:38:02.744080 7fe9bae887a0 20 read_log 192718'11691 (192718'11690) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:256 2014-08-06 14:53:33.612985 2014-08-13 17:38:02.744092 7fe9bae887a0 20 read_log 192718'11692 (192718'11691) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:257 2014-08-06 14:53:33.639662 2014-08-13 17:38:02.744104 7fe9bae887a0 20 read_log 192718'11693 (192718'11692) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:258 2014-08-06 14:53:33.666363 2014-08-13 17:38:02.744116 7fe9bae887a0 20 read_log 192718'11694 (192718'11693) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:259 2014-08-06 14:53:33.695155 2014-08-13 17:38:02.744128 7fe9bae887a0 20 read_log 192718'11695 (192718'11694) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:260 2014-08-06 14:53:33.721666 2014-08-13 17:38:02.744140 7fe9bae887a0 20 read_log 192718'11696 (192718'11695) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:261 2014-08-06 14:53:34.174895 2014-08-13 17:38:02.744151 7fe9bae887a0 20 read_log 192718'11697 (192718'11696) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:262 2014-08-06 14:53:34.222590 2014-08-13 17:38:02.744169 7fe9bae887a0 20 read_log 192718'11698 (192718'11697) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:263 2014-08-06 14:53:34.266140 2014-08-13 17:38:02.744181 7fe9bae887a0 20 read_log 192718'11699 (192718'11698) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:264 2014-08-06 14:53:34.310235 2014-08-13 17:38:02.744193 7fe9bae887a0 20 read_log 192718'11700 (192718'11699) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:265 2014-08-06 14:53:34.353251 2014-08-13 17:38:02.744205 7fe9bae887a0 20 read_log 192718'11701 (192718'11700) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:266 2014-08-06 14:53:34.397088 2014-08-13 17:38:02.744216 7fe9bae887a0 20 read_log 192718'11702 (192718'11701) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:267 2014-08-06 14:53:34.440079 2014-08-13 17:38:02.744228 7fe9bae887a0 20 read_log 192718'11703 (192718'11702) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:268 2014-08-06 14:53:34.483607 2014-08-13 17:38:02.744244 7fe9bae887a0 20 read_log 192718'11704 (192718'11703) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:269 2014-08-06 14:53:34.526872 2014-08-13 17:38:02.744256 7fe9bae887a0 20 read_log 192718'11705 (192718'11704) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:270 2014-08-06 14:53:34.569721 2014-08-13 17:38:02.744268 7fe9bae887a0 20 read_log 192718'11706 (192718'11705) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:271 2014-08-06 14:53:35.000469 2014-08-13 17:38:02.744279 7fe9bae887a0 20 read_log 192718'11707 (192718'11706) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:272 2014-08-06 14:53:35.087793 2014-08-13 17:38:02.744291 7fe9bae887a0 20 read_log 192718'11708 (192718'11707) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:273 2014-08-06 14:53:35.152856 2014-08-13 17:38:02.744303 7fe9bae887a0 20 read_log 192718'11709 (192718'11708) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:274 2014-08-06 14:53:35.225882 2014-08-13 17:38:02.744315 7fe9bae887a0 20 read_log 192718'11710 (192718'11709) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:275 2014-08-06 14:53:35.302013 2014-08-13 17:38:02.744326 7fe9bae887a0 20 read_log 192718'11711 (192718'11710) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:276 2014-08-06 14:53:35.350984 2014-08-13 17:38:02.744361 7fe9bae887a0 20 read_log 192718'11712 (192718'11711) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:277 2014-08-06 14:53:35.411400 2014-08-13 17:38:02.744377 7fe9bae887a0 20 read_log 192718'11713 (192718'11712) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:278 2014-08-06 14:53:35.454367 2014-08-13 17:38:02.744389 7fe9bae887a0 20 read_log 192718'11714 (192718'11713) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:279 2014-08-06 14:53:35.497304 2014-08-13 17:38:02.744401 7fe9bae887a0 20 read_log 192718'11715 (192718'11714) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:280 2014-08-06 14:53:35.541532 2014-08-13 17:38:02.744413 7fe9bae887a0 20 read_log 192718'11716 (192718'11715) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:281 2014-08-06 14:53:36.021739 2014-08-13 17:38:02.744425 7fe9bae887a0 20 read_log 192718'11717 (192718'11716) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:282 2014-08-06 14:53:36.070341 2014-08-13 17:38:02.744437 7fe9bae887a0 20 read_log 192718'11718 (192718'11717) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:283 2014-08-06 14:53:36.113384 2014-08-13 17:38:02.744449 7fe9bae887a0 20 read_log 192718'11719 (192718'11718) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:284 2014-08-06 14:53:36.156845 2014-08-13 17:38:02.744461 7fe9bae887a0 20 read_log 192718'11720 (192718'11719) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:285 2014-08-06 14:53:36.199839 2014-08-13 17:38:02.744473 7fe9bae887a0 20 read_log 192718'11721 (192718'11720) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:286 2014-08-06 14:53:37.043944 2014-08-13 17:38:02.744488 7fe9bae887a0 20 read_log 192718'11722 (192718'11721) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:287 2014-08-06 14:53:37.069708 2014-08-13 17:38:02.744500 7fe9bae887a0 20 read_log 192718'11723 (192718'11722) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:288 2014-08-06 14:53:37.094381 2014-08-13 17:38:02.744512 7fe9bae887a0 20 read_log 192718'11724 (192718'11723) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:289 2014-08-06 14:53:37.118346 2014-08-13 17:38:02.744524 7fe9bae887a0 20 read_log 192718'11725 (192718'11724) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:290 2014-08-06 14:53:37.142055 2014-08-13 17:38:02.744536 7fe9bae887a0 20 read_log 192718'11726 (192718'11725) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:291 2014-08-06 14:53:37.167926 2014-08-13 17:38:02.744548 7fe9bae887a0 20 read_log 192718'11727 (192718'11726) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:292 2014-08-06 14:53:37.194857 2014-08-13 17:38:02.744559 7fe9bae887a0 20 read_log 192718'11728 (192718'11727) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:293 2014-08-06 14:53:37.221221 2014-08-13 17:38:02.744571 7fe9bae887a0 20 read_log 192718'11729 (192718'11728) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:294 2014-08-06 14:53:37.246904 2014-08-13 17:38:02.744583 7fe9bae887a0 20 read_log 192718'11730 (192718'11729) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:295 2014-08-06 14:53:37.272161 2014-08-13 17:38:02.744595 7fe9bae887a0 20 read_log 192718'11731 (192718'11730) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:296 2014-08-06 14:53:37.298120 2014-08-13 17:38:02.744607 7fe9bae887a0 20 read_log 192718'11732 (192718'11731) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:297 2014-08-06 14:53:37.324584 2014-08-13 17:38:02.744622 7fe9bae887a0 20 read_log 192718'11733 (192718'11732) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:298 2014-08-06 14:53:37.351272 2014-08-13 17:38:02.744634 7fe9bae887a0 20 read_log 192718'11734 (192718'11733) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:299 2014-08-06 14:53:37.377884 2014-08-13 17:38:02.744646 7fe9bae887a0 20 read_log 192718'11735 (192718'11734) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:300 2014-08-06 14:53:37.404177 2014-08-13 17:38:02.744665 7fe9bae887a0 20 read_log 192718'11736 (192718'11735) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:301 2014-08-06 14:53:37.632939 2014-08-13 17:38:02.744677 7fe9bae887a0 20 read_log 192718'11737 (192718'11736) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:302 2014-08-06 14:53:37.689419 2014-08-13 17:38:02.744689 7fe9bae887a0 20 read_log 192718'11738 (192718'11737) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:303 2014-08-06 14:53:37.717803 2014-08-13 17:38:02.744700 7fe9bae887a0 20 read_log 192718'11739 (192718'11738) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:304 2014-08-06 14:53:37.744250 2014-08-13 17:38:02.744712 7fe9bae887a0 20 read_log 192718'11740 (192718'11739) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:305 2014-08-06 14:53:37.771022 2014-08-13 17:38:02.744724 7fe9bae887a0 20 read_log 192718'11741 (192718'11740) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:306 2014-08-06 14:53:37.876730 2014-08-13 17:38:02.744736 7fe9bae887a0 20 read_log 192718'11742 (192718'11741) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:307 2014-08-06 14:53:37.956372 2014-08-13 17:38:02.744747 7fe9bae887a0 20 read_log 192718'11743 (192718'11742) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:308 2014-08-06 14:53:38.013556 2014-08-13 17:38:02.744759 7fe9bae887a0 20 read_log 192718'11744 (192718'11743) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:309 2014-08-06 14:53:38.047428 2014-08-13 17:38:02.744771 7fe9bae887a0 20 read_log 192718'11745 (192718'11744) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:310 2014-08-06 14:53:38.073303 2014-08-13 17:38:02.744783 7fe9bae887a0 20 read_log 192718'11746 (192718'11745) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:311 2014-08-06 14:53:38.099159 2014-08-13 17:38:02.744794 7fe9bae887a0 20 read_log 192718'11747 (192718'11746) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:312 2014-08-06 14:53:38.126837 2014-08-13 17:38:02.744806 7fe9bae887a0 20 read_log 192718'11748 (192718'11747) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:313 2014-08-06 14:53:38.153846 2014-08-13 17:38:02.744817 7fe9bae887a0 20 read_log 192718'11749 (192718'11748) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:314 2014-08-06 14:53:38.179355 2014-08-13 17:38:02.744829 7fe9bae887a0 20 read_log 192718'11750 (192718'11749) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:315 2014-08-06 14:53:38.204818 2014-08-13 17:38:02.744841 7fe9bae887a0 20 read_log 192718'11751 (192718'11750) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:316 2014-08-06 14:53:38.230210 2014-08-13 17:38:02.744852 7fe9bae887a0 20 read_log 192718'11752 (192718'11751) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:317 2014-08-06 14:53:38.260359 2014-08-13 17:38:02.744864 7fe9bae887a0 20 read_log 192718'11753 (192718'11752) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:318 2014-08-06 14:53:38.491939 2014-08-13 17:38:02.744877 7fe9bae887a0 20 read_log 192718'11754 (192718'11754) modify 34/hit_set_24.34_archive_2014-08-06 14:48:58.711039_2014-08-06 14:53:38.550251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 14:53:38.550776 2014-08-13 17:38:02.744889 7fe9bae887a0 20 read_log 192718'11755 (192718'11171) delete 34/hit_set_24.34_archive_2014-08-06 14:16:43.165223_2014-08-06 14:48:58.710801/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 14:53:38.550776 2014-08-13 17:38:02.744902 7fe9bae887a0 20 read_log 192718'11756 (192718'11753) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:319 2014-08-06 14:53:38.541793 2014-08-13 17:38:02.744914 7fe9bae887a0 20 read_log 192718'11757 (192718'11756) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:320 2014-08-06 14:53:38.608817 2014-08-13 17:38:02.744925 7fe9bae887a0 20 read_log 192718'11758 (192718'11757) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:321 2014-08-06 14:53:38.668325 2014-08-13 17:38:02.744943 7fe9bae887a0 20 read_log 192718'11759 (192718'11758) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:322 2014-08-06 14:53:38.694989 2014-08-13 17:38:02.744955 7fe9bae887a0 20 read_log 192718'11760 (192718'11759) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:323 2014-08-06 14:53:38.721162 2014-08-13 17:38:02.744967 7fe9bae887a0 20 read_log 192718'11761 (192718'11760) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:324 2014-08-06 14:53:38.747611 2014-08-13 17:38:02.744979 7fe9bae887a0 20 read_log 192718'11762 (192718'11761) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:325 2014-08-06 14:53:38.776637 2014-08-13 17:38:02.744990 7fe9bae887a0 20 read_log 192718'11763 (192718'11762) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:326 2014-08-06 14:53:38.802653 2014-08-13 17:38:02.745002 7fe9bae887a0 20 read_log 192718'11764 (192718'11763) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:327 2014-08-06 14:53:38.828907 2014-08-13 17:38:02.745016 7fe9bae887a0 20 read_log 192718'11765 (192718'11764) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:328 2014-08-06 14:53:39.123577 2014-08-13 17:38:02.745028 7fe9bae887a0 20 read_log 192718'11766 (192718'11765) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:329 2014-08-06 14:53:39.152900 2014-08-13 17:38:02.745039 7fe9bae887a0 20 read_log 192718'11767 (192718'11766) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:330 2014-08-06 14:53:39.181496 2014-08-13 17:38:02.745051 7fe9bae887a0 20 read_log 192718'11768 (192718'11767) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:331 2014-08-06 14:53:39.209792 2014-08-13 17:38:02.745063 7fe9bae887a0 20 read_log 192718'11769 (192718'11768) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:332 2014-08-06 14:53:39.238223 2014-08-13 17:38:02.745075 7fe9bae887a0 20 read_log 192718'11770 (192718'11769) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:333 2014-08-06 14:53:39.265568 2014-08-13 17:38:02.745087 7fe9bae887a0 20 read_log 192718'11771 (192718'11770) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:334 2014-08-06 14:53:39.292236 2014-08-13 17:38:02.745099 7fe9bae887a0 20 read_log 192718'11772 (192718'11771) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:335 2014-08-06 14:53:39.318798 2014-08-13 17:38:02.745110 7fe9bae887a0 20 read_log 192718'11773 (192718'11772) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:336 2014-08-06 14:53:39.345899 2014-08-13 17:38:02.745122 7fe9bae887a0 20 read_log 192718'11774 (192718'11773) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:337 2014-08-06 14:53:39.374743 2014-08-13 17:38:02.745133 7fe9bae887a0 20 read_log 192718'11775 (192718'11774) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:338 2014-08-06 14:53:39.661525 2014-08-13 17:38:02.745147 7fe9bae887a0 20 read_log 192718'11776 (192718'11775) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:339 2014-08-06 14:53:39.690156 2014-08-13 17:38:02.745163 7fe9bae887a0 20 read_log 192718'11777 (192718'11776) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:340 2014-08-06 14:53:39.717174 2014-08-13 17:38:02.745175 7fe9bae887a0 20 read_log 192718'11778 (192718'11777) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:341 2014-08-06 14:53:39.743553 2014-08-13 17:38:02.745187 7fe9bae887a0 20 read_log 192718'11779 (192718'11778) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:342 2014-08-06 14:53:39.771621 2014-08-13 17:38:02.745199 7fe9bae887a0 20 read_log 192718'11780 (192718'11779) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:343 2014-08-06 14:53:39.797313 2014-08-13 17:38:02.745230 7fe9bae887a0 20 read_log 192718'11781 (192718'11780) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:344 2014-08-06 14:53:39.823599 2014-08-13 17:38:02.745243 7fe9bae887a0 20 read_log 192718'11782 (192718'11781) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:345 2014-08-06 14:53:39.849737 2014-08-13 17:38:02.745263 7fe9bae887a0 20 read_log 192718'11783 (192718'11782) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:346 2014-08-06 14:53:39.875420 2014-08-13 17:38:02.745276 7fe9bae887a0 20 read_log 192718'11784 (192718'11783) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:347 2014-08-06 14:53:39.901789 2014-08-13 17:38:02.745288 7fe9bae887a0 20 read_log 192718'11785 (192718'11784) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:348 2014-08-06 14:53:39.927427 2014-08-13 17:38:02.745300 7fe9bae887a0 20 read_log 192718'11786 (192718'11785) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:349 2014-08-06 14:53:39.953894 2014-08-13 17:38:02.745312 7fe9bae887a0 20 read_log 192718'11787 (192718'11786) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:350 2014-08-06 14:53:40.365858 2014-08-13 17:38:02.745324 7fe9bae887a0 20 read_log 192718'11788 (192718'11787) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:351 2014-08-06 14:53:40.390943 2014-08-13 17:38:02.745336 7fe9bae887a0 20 read_log 192718'11789 (192718'11788) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:352 2014-08-06 14:53:40.418706 2014-08-13 17:38:02.745347 7fe9bae887a0 20 read_log 192718'11790 (192718'11789) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:353 2014-08-06 14:53:40.445374 2014-08-13 17:38:02.745359 7fe9bae887a0 20 read_log 192718'11791 (192718'11790) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:354 2014-08-06 14:53:40.473037 2014-08-13 17:38:02.745371 7fe9bae887a0 20 read_log 192718'11792 (192718'11791) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:355 2014-08-06 14:53:40.499574 2014-08-13 17:38:02.745382 7fe9bae887a0 20 read_log 192718'11793 (192718'11792) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:356 2014-08-06 14:53:40.525870 2014-08-13 17:38:02.745394 7fe9bae887a0 20 read_log 192718'11794 (192718'11793) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:357 2014-08-06 14:53:40.551530 2014-08-13 17:38:02.745406 7fe9bae887a0 20 read_log 192718'11795 (192718'11794) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:358 2014-08-06 14:53:40.579196 2014-08-13 17:38:02.745418 7fe9bae887a0 20 read_log 192718'11796 (192718'11795) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:359 2014-08-06 14:53:40.609908 2014-08-13 17:38:02.745430 7fe9bae887a0 20 read_log 192718'11797 (192718'11796) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:360 2014-08-06 14:53:40.636701 2014-08-13 17:38:02.745442 7fe9bae887a0 20 read_log 192718'11798 (192718'11797) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:361 2014-08-06 14:53:40.718449 2014-08-13 17:38:02.745454 7fe9bae887a0 20 read_log 192718'11799 (192718'11798) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:362 2014-08-06 14:53:40.799478 2014-08-13 17:38:02.745467 7fe9bae887a0 20 read_log 192718'11800 (192718'11799) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:363 2014-08-06 14:53:40.880461 2014-08-13 17:38:02.745479 7fe9bae887a0 20 read_log 192718'11801 (192718'11800) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:364 2014-08-06 14:53:40.933561 2014-08-13 17:38:02.745491 7fe9bae887a0 20 read_log 192718'11802 (192718'11801) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:365 2014-08-06 14:53:41.007887 2014-08-13 17:38:02.745503 7fe9bae887a0 20 read_log 192718'11803 (192718'11802) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:366 2014-08-06 14:53:41.089504 2014-08-13 17:38:02.745515 7fe9bae887a0 20 read_log 192718'11804 (192718'11803) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:367 2014-08-06 14:53:41.169838 2014-08-13 17:38:02.745527 7fe9bae887a0 20 read_log 192718'11805 (192718'11804) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:368 2014-08-06 14:53:41.616343 2014-08-13 17:38:02.745538 7fe9bae887a0 20 read_log 192718'11806 (192718'11805) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:369 2014-08-06 14:53:41.697452 2014-08-13 17:38:02.745558 7fe9bae887a0 20 read_log 192718'11807 (192718'11806) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:370 2014-08-06 14:53:41.778535 2014-08-13 17:38:02.745570 7fe9bae887a0 20 read_log 192718'11808 (192718'11807) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:371 2014-08-06 14:53:41.862025 2014-08-13 17:38:02.745582 7fe9bae887a0 20 read_log 192718'11809 (192718'11809) modify 34/hit_set_24.34_archive_2014-08-06 14:53:38.550781_2014-08-06 15:31:20.412990/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:31:20.413258 2014-08-13 17:38:02.745595 7fe9bae887a0 20 read_log 192718'11810 (192718'11754) delete 34/hit_set_24.34_archive_2014-08-06 14:48:58.711039_2014-08-06 14:53:38.550251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:31:20.413258 2014-08-13 17:38:02.745607 7fe9bae887a0 20 read_log 192718'11977 (192718'11808) clean 75dfec34/core.19312+32+2/head//24 by osd.325.0:33832 2014-08-06 15:31:53.476866 2014-08-13 17:38:02.745619 7fe9bae887a0 20 read_log 192718'12318 (192718'12318) modify 34/hit_set_24.34_archive_2014-08-06 15:31:20.413262_2014-08-06 15:50:05.551522/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:50:05.551751 2014-08-13 17:38:02.745632 7fe9bae887a0 20 read_log 192718'12319 (192718'11809) delete 34/hit_set_24.34_archive_2014-08-06 14:53:38.550781_2014-08-06 15:31:20.412990/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:50:05.551751 2014-08-13 17:38:02.745644 7fe9bae887a0 20 read_log 192803'12662 (192718'11977) delete 75dfec34/core.19312+32+2/head//24 by osd.325.0:35389 2014-08-11 10:27:14.259503 2014-08-13 17:38:02.745656 7fe9bae887a0 20 read_log 192834'12664 (192834'12664) modify 34/hit_set_24.34_archive_2014-08-11 10:24:43.621609_2014-08-11 15:23:36.202456/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.202664 2014-08-13 17:38:02.745668 7fe9bae887a0 20 read_log 192834'12665 (192718'12318) delete 34/hit_set_24.34_archive_2014-08-06 15:31:20.413262_2014-08-06 15:50:05.551522/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.202664 2014-08-13 17:38:02.745681 7fe9bae887a0 20 read_log 192834'12666 (0'0) promote 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by osd.325.0:36357 2014-08-11 15:23:36.206543 2014-08-13 17:38:02.745693 7fe9bae887a0 20 read_log 192834'12667 (192834'12666) modify 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.933448.0:134 2014-08-11 15:23:36.195923 2014-08-13 17:38:02.745705 7fe9bae887a0 20 read_log 192834'12668 (0'0) promote d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:36362 2014-08-11 15:23:37.163127 2014-08-13 17:38:02.745718 7fe9bae887a0 20 read_log 192834'12669 (192834'12668) modify d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by client.933448.0:395 2014-08-11 15:23:37.155417 2014-08-13 17:38:02.745730 7fe9bae887a0 20 read_log 192834'12720 (0'0) promote c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:36452 2014-08-11 15:23:51.206823 2014-08-13 17:38:02.745742 7fe9bae887a0 20 read_log 192834'12721 (192834'12720) modify c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by client.933448.0:3316 2014-08-11 15:23:51.199125 2014-08-13 17:38:02.745754 7fe9bae887a0 20 read_log 192834'12728 (0'0) promote 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:36464 2014-08-11 15:23:52.804960 2014-08-13 17:38:02.745766 7fe9bae887a0 20 read_log 192834'12729 (192834'12728) modify 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by client.933448.0:3622 2014-08-11 15:23:52.796696 2014-08-13 17:38:02.745779 7fe9bae887a0 20 read_log 192834'12738 (0'0) promote 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by osd.325.0:36476 2014-08-11 15:23:54.591706 2014-08-13 17:38:02.745798 7fe9bae887a0 20 read_log 192834'12739 (192834'12738) modify 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.933448.0:3935 2014-08-11 15:23:54.583107 2014-08-13 17:38:02.745811 7fe9bae887a0 20 read_log 192834'12750 (0'0) promote 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:36512 2014-08-11 15:24:02.823498 2014-08-13 17:38:02.745823 7fe9bae887a0 20 read_log 192834'12751 (192834'12750) modify 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by client.933448.0:5771 2014-08-11 15:24:02.816311 2014-08-13 17:38:02.745836 7fe9bae887a0 20 read_log 192834'12788 (0'0) promote 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by osd.325.0:36598 2014-08-11 15:24:16.250818 2014-08-13 17:38:02.745848 7fe9bae887a0 20 read_log 192834'12789 (192834'12788) modify 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.933448.0:8936 2014-08-11 15:24:16.242386 2014-08-13 17:38:02.745862 7fe9bae887a0 20 read_log 192834'12790 (0'0) promote 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by osd.325.0:36606 2014-08-11 15:24:18.065061 2014-08-13 17:38:02.745878 7fe9bae887a0 20 read_log 192834'12791 (192834'12790) modify 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.933448.0:9230 2014-08-11 15:24:18.057115 2014-08-13 17:38:02.745890 7fe9bae887a0 20 read_log 192834'12806 (0'0) promote e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by osd.325.0:36646 2014-08-11 15:24:25.575454 2014-08-13 17:38:02.745902 7fe9bae887a0 20 read_log 192834'12807 (192834'12806) modify e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.933448.0:10876 2014-08-11 15:24:25.567264 2014-08-13 17:38:02.745914 7fe9bae887a0 20 read_log 192834'12810 (0'0) promote c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by osd.325.0:36654 2014-08-11 15:24:26.875528 2014-08-13 17:38:02.745926 7fe9bae887a0 20 read_log 192834'12811 (192834'12810) modify c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.933448.0:11214 2014-08-11 15:24:26.867286 2014-08-13 17:38:02.745938 7fe9bae887a0 20 read_log 192834'12882 (0'0) promote c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:36788 2014-08-11 15:24:51.296830 2014-08-13 17:38:02.745950 7fe9bae887a0 20 read_log 192834'12883 (192834'12882) modify c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by client.933448.0:15788 2014-08-11 15:24:51.287928 2014-08-13 17:38:02.745963 7fe9bae887a0 20 read_log 192834'12906 (0'0) promote d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by osd.325.0:36838 2014-08-11 15:25:00.839701 2014-08-13 17:38:02.745975 7fe9bae887a0 20 read_log 192834'12907 (192834'12906) modify d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.933448.0:17636 2014-08-11 15:25:00.832090 2014-08-13 17:38:02.745987 7fe9bae887a0 20 read_log 192834'12908 (0'0) promote ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by osd.325.0:36840 2014-08-11 15:25:01.187700 2014-08-13 17:38:02.745999 7fe9bae887a0 20 read_log 192834'12909 (192834'12908) modify ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.933448.0:17719 2014-08-11 15:25:01.179217 2014-08-13 17:38:02.746011 7fe9bae887a0 20 read_log 192834'12914 (0'0) promote 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by osd.325.0:36856 2014-08-11 15:25:03.701143 2014-08-13 17:38:02.746023 7fe9bae887a0 20 read_log 192834'12915 (192834'12914) modify 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.933448.0:18108 2014-08-11 15:25:03.691216 2014-08-13 17:38:02.746035 7fe9bae887a0 20 read_log 192834'12940 (0'0) promote 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by osd.325.0:36914 2014-08-11 15:25:13.869994 2014-08-13 17:38:02.746059 7fe9bae887a0 20 read_log 192834'12941 (192834'12940) modify 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.933448.0:19895 2014-08-11 15:25:13.855426 2014-08-13 17:38:02.746074 7fe9bae887a0 20 read_log 192834'12952 (0'0) promote ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:36948 2014-08-11 15:25:23.513383 2014-08-13 17:38:02.746086 7fe9bae887a0 20 read_log 192834'12953 (192834'12952) modify ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by client.933448.0:20832 2014-08-11 15:25:23.498921 2014-08-13 17:38:02.746098 7fe9bae887a0 20 read_log 192834'12976 (192834'12667) modify 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.933448.0:22425 2014-08-11 15:25:37.855704 2014-08-13 17:38:02.746110 7fe9bae887a0 20 read_log 192834'12977 (192834'12669) modify d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by client.933448.0:22686 2014-08-11 15:25:38.467411 2014-08-13 17:38:02.746122 7fe9bae887a0 20 read_log 192834'13003 (192834'12721) modify c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by client.933448.0:25607 2014-08-11 15:25:42.261735 2014-08-13 17:38:02.746134 7fe9bae887a0 20 read_log 192834'13007 (192834'12729) modify 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by client.933448.0:25913 2014-08-11 15:25:42.713837 2014-08-13 17:38:02.746146 7fe9bae887a0 20 read_log 192834'13012 (192834'12739) modify 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.933448.0:26226 2014-08-11 15:25:43.138799 2014-08-13 17:38:02.746163 7fe9bae887a0 20 read_log 192834'13018 (192834'12751) modify 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by client.933448.0:28062 2014-08-11 15:25:45.447250 2014-08-13 17:38:02.746176 7fe9bae887a0 20 read_log 192834'13037 (192834'12789) modify 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.933448.0:31227 2014-08-11 15:25:49.816037 2014-08-13 17:38:02.746188 7fe9bae887a0 20 read_log 192834'13038 (192834'12791) modify 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.933448.0:31521 2014-08-11 15:25:50.150004 2014-08-13 17:38:02.746200 7fe9bae887a0 20 read_log 192834'13046 (192834'12807) modify e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.933448.0:33167 2014-08-11 15:25:52.932793 2014-08-13 17:38:02.746213 7fe9bae887a0 20 read_log 192834'13048 (192834'12811) modify c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.933448.0:33505 2014-08-11 15:25:53.563835 2014-08-13 17:38:02.746225 7fe9bae887a0 20 read_log 192834'13084 (192834'12883) modify c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by client.933448.0:38079 2014-08-11 15:26:00.468826 2014-08-13 17:38:02.746237 7fe9bae887a0 20 read_log 192834'13096 (192834'12907) modify d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.933448.0:39927 2014-08-11 15:26:02.170591 2014-08-13 17:38:02.746250 7fe9bae887a0 20 read_log 192834'13097 (192834'12909) modify ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.933448.0:40010 2014-08-11 15:26:02.218458 2014-08-13 17:38:02.746263 7fe9bae887a0 20 read_log 192834'13100 (192834'12915) modify 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.933448.0:40399 2014-08-11 15:26:02.599976 2014-08-13 17:38:02.746275 7fe9bae887a0 20 read_log 192834'13113 (192834'12941) modify 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.933448.0:42186 2014-08-11 15:26:03.179957 2014-08-13 17:38:02.746287 7fe9bae887a0 20 read_log 192834'13119 (192834'12953) modify ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by client.933448.0:43123 2014-08-11 15:26:03.508114 2014-08-13 17:38:02.746299 7fe9bae887a0 20 read_log 192834'13137 (0'0) promote 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by osd.325.0:37340 2014-08-11 15:26:15.620950 2014-08-13 17:38:02.746335 7fe9bae887a0 20 read_log 192834'13138 (192834'13137) modify 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.1883634.0:305 2014-08-11 15:26:15.611342 2014-08-13 17:38:02.746353 7fe9bae887a0 20 read_log 192834'13143 (0'0) promote e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by osd.325.0:37352 2014-08-11 15:26:16.653772 2014-08-13 17:38:02.746366 7fe9bae887a0 20 read_log 192834'13144 (192834'13143) modify e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.1883634.0:592 2014-08-11 15:26:16.645472 2014-08-13 17:38:02.746378 7fe9bae887a0 20 read_log 192834'13145 (0'0) promote bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by osd.325.0:37356 2014-08-11 15:26:17.523319 2014-08-13 17:38:02.746390 7fe9bae887a0 20 read_log 192834'13146 (192834'13145) modify bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.1883634.0:833 2014-08-11 15:26:17.514537 2014-08-13 17:38:02.746402 7fe9bae887a0 20 read_log 192834'13159 (0'0) promote bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:37376 2014-08-11 15:26:19.871321 2014-08-13 17:38:02.746414 7fe9bae887a0 20 read_log 192834'13160 (192834'13159) modify bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.1883634.0:1483 2014-08-11 15:26:19.863625 2014-08-13 17:38:02.746426 7fe9bae887a0 20 read_log 192834'13167 (0'0) promote 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:37386 2014-08-11 15:26:20.616570 2014-08-13 17:38:02.746455 7fe9bae887a0 20 read_log 192834'13168 (192834'13167) modify 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.1883634.0:1688 2014-08-11 15:26:20.609451 2014-08-13 17:38:02.746470 7fe9bae887a0 20 read_log 192834'13185 (0'0) promote acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by osd.325.0:37416 2014-08-11 15:26:23.058460 2014-08-13 17:38:02.746482 7fe9bae887a0 20 read_log 192834'13186 (192834'13185) modify acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.1883634.0:2322 2014-08-11 15:26:23.051509 2014-08-13 17:38:02.746497 7fe9bae887a0 20 read_log 192834'13191 (0'0) promote 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by osd.325.0:37426 2014-08-11 15:26:24.506735 2014-08-13 17:38:02.746510 7fe9bae887a0 20 read_log 192834'13192 (192834'13191) modify 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.1883634.0:2730 2014-08-11 15:26:24.497929 2014-08-13 17:38:02.746523 7fe9bae887a0 20 read_log 192834'13197 (0'0) promote 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by osd.325.0:37444 2014-08-11 15:26:26.251194 2014-08-13 17:38:02.746535 7fe9bae887a0 20 read_log 192834'13198 (192834'13197) modify 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.1883634.0:3222 2014-08-11 15:26:26.242599 2014-08-13 17:38:02.746547 7fe9bae887a0 20 read_log 192834'13207 (0'0) promote 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by osd.325.0:37470 2014-08-11 15:26:28.885502 2014-08-13 17:38:02.746559 7fe9bae887a0 20 read_log 192834'13208 (192834'13207) modify 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.1883634.0:3969 2014-08-11 15:26:28.877462 2014-08-13 17:38:02.746571 7fe9bae887a0 20 read_log 192834'13209 (0'0) promote 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by osd.325.0:37476 2014-08-11 15:26:29.088711 2014-08-13 17:38:02.746583 7fe9bae887a0 20 read_log 192834'13210 (192834'13209) modify 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.1883634.0:4026 2014-08-11 15:26:29.081371 2014-08-13 17:38:02.746595 7fe9bae887a0 20 read_log 192834'13213 (0'0) promote eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by osd.325.0:37488 2014-08-11 15:26:30.316213 2014-08-13 17:38:02.746616 7fe9bae887a0 20 read_log 192834'13214 (192834'13213) modify eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.1883634.0:4372 2014-08-11 15:26:30.306501 2014-08-13 17:38:02.746629 7fe9bae887a0 20 read_log 192834'13217 (0'0) promote 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by osd.325.0:37494 2014-08-11 15:26:30.614572 2014-08-13 17:38:02.746641 7fe9bae887a0 20 read_log 192834'13218 (192834'13217) modify 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.1883634.0:4456 2014-08-11 15:26:30.605825 2014-08-13 17:38:02.746672 7fe9bae887a0 20 read_log 192834'13221 (0'0) promote d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by osd.325.0:37520 2014-08-11 15:26:34.860915 2014-08-13 17:38:02.746685 7fe9bae887a0 20 read_log 192834'13222 (192834'13221) modify d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.1883634.0:5567 2014-08-11 15:26:34.854088 2014-08-13 17:38:02.746697 7fe9bae887a0 20 read_log 192834'13233 (0'0) promote 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by osd.325.0:37542 2014-08-11 15:26:35.893138 2014-08-13 17:38:02.746710 7fe9bae887a0 20 read_log 192834'13234 (192834'13233) modify 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.1883634.0:5857 2014-08-11 15:26:35.885475 2014-08-13 17:38:02.746722 7fe9bae887a0 20 read_log 192834'13237 (0'0) promote 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by osd.325.0:37546 2014-08-11 15:26:36.146514 2014-08-13 17:38:02.746734 7fe9bae887a0 20 read_log 192834'13238 (192834'13237) modify 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.1883634.0:5927 2014-08-11 15:26:36.138623 2014-08-13 17:38:02.746746 7fe9bae887a0 20 read_log 192834'13255 (0'0) promote e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by osd.325.0:37596 2014-08-11 15:26:40.940484 2014-08-13 17:38:02.746758 7fe9bae887a0 20 read_log 192834'13256 (192834'13255) modify e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.1883634.0:7136 2014-08-11 15:26:40.933369 2014-08-13 17:38:02.746770 7fe9bae887a0 20 read_log 192834'13267 (0'0) promote c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:37622 2014-08-11 15:26:43.777036 2014-08-13 17:38:02.746782 7fe9bae887a0 20 read_log 192834'13268 (192834'13267) modify c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.1883634.0:7807 2014-08-11 15:26:43.767811 2014-08-13 17:38:02.746796 7fe9bae887a0 20 read_log 192834'13269 (0'0) promote 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by osd.325.0:37626 2014-08-11 15:26:44.550547 2014-08-13 17:38:02.746808 7fe9bae887a0 20 read_log 192834'13270 (192834'13269) modify 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.1883634.0:7982 2014-08-11 15:26:44.541786 2014-08-13 17:38:02.746820 7fe9bae887a0 20 read_log 192834'13307 (0'0) promote b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by osd.325.0:37688 2014-08-11 15:26:54.218582 2014-08-13 17:38:02.746832 7fe9bae887a0 20 read_log 192834'13308 (192834'13307) modify b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.1883634.0:10242 2014-08-11 15:26:54.210945 2014-08-13 17:38:02.746844 7fe9bae887a0 20 read_log 192834'13313 (0'0) promote 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by osd.325.0:37698 2014-08-11 15:26:55.724019 2014-08-13 17:38:02.746856 7fe9bae887a0 20 read_log 192834'13314 (192834'13313) modify 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.1883634.0:10583 2014-08-11 15:26:55.715173 2014-08-13 17:38:02.746868 7fe9bae887a0 20 read_log 192834'13323 (0'0) promote 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:37716 2014-08-11 15:27:00.924119 2014-08-13 17:38:02.746890 7fe9bae887a0 20 read_log 192834'13324 (192834'13323) modify 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.1883634.0:11670 2014-08-11 15:27:00.915979 2014-08-13 17:38:02.746905 7fe9bae887a0 20 read_log 192834'13329 (0'0) promote 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:37722 2014-08-11 15:27:01.376545 2014-08-13 17:38:02.746917 7fe9bae887a0 20 read_log 192834'13330 (192834'13329) modify 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.1883634.0:11820 2014-08-11 15:27:01.368576 2014-08-13 17:38:02.746930 7fe9bae887a0 20 read_log 192834'13345 (0'0) promote be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by osd.325.0:37754 2014-08-11 15:27:05.273301 2014-08-13 17:38:02.746944 7fe9bae887a0 20 read_log 192834'13346 (192834'13345) modify be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.1883634.0:12654 2014-08-11 15:27:05.264502 2014-08-13 17:38:02.746956 7fe9bae887a0 20 read_log 192834'13379 (0'0) promote 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:37826 2014-08-11 15:27:13.925253 2014-08-13 17:38:02.746969 7fe9bae887a0 20 read_log 192834'13380 (192834'13379) modify 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.1883634.0:14603 2014-08-11 15:27:13.916419 2014-08-13 17:38:02.746983 7fe9bae887a0 20 read_log 192834'13417 (0'0) promote ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by osd.325.0:37916 2014-08-11 15:27:34.969228 2014-08-13 17:38:02.746995 7fe9bae887a0 20 read_log 192834'13418 (192834'13417) modify ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.1883634.0:17389 2014-08-11 15:27:34.958285 2014-08-13 17:38:02.747010 7fe9bae887a0 20 read_log 192834'13437 (0'0) promote af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by osd.325.0:37964 2014-08-11 15:27:48.268989 2014-08-13 17:38:02.747022 7fe9bae887a0 20 read_log 192834'13438 (192834'13437) modify af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.1883634.0:18917 2014-08-11 15:27:48.260158 2014-08-13 17:38:02.747036 7fe9bae887a0 20 read_log 192834'13441 (0'0) promote c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by osd.325.0:37968 2014-08-11 15:27:48.849331 2014-08-13 17:38:02.747048 7fe9bae887a0 20 read_log 192834'13442 (192834'13441) modify c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.1883634.0:19014 2014-08-11 15:27:48.840043 2014-08-13 17:38:02.747063 7fe9bae887a0 20 read_log 192834'13445 (0'0) promote 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by osd.325.0:37974 2014-08-11 15:27:50.099531 2014-08-13 17:38:02.747075 7fe9bae887a0 20 read_log 192834'13446 (192834'13445) modify 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.1883634.0:19245 2014-08-11 15:27:50.089180 2014-08-13 17:38:02.747089 7fe9bae887a0 20 read_log 192834'13463 (0'0) promote 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by osd.325.0:38008 2014-08-11 15:27:54.902252 2014-08-13 17:38:02.747101 7fe9bae887a0 20 read_log 192834'13464 (192834'13463) modify 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.1883634.0:19997 2014-08-11 15:27:54.892615 2014-08-13 17:38:02.747115 7fe9bae887a0 20 read_log 192834'13465 (0'0) promote 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by osd.325.0:38010 2014-08-11 15:27:54.916035 2014-08-13 17:38:02.747127 7fe9bae887a0 20 read_log 192834'13466 (192834'13465) modify 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.1883634.0:20002 2014-08-11 15:27:54.907899 2014-08-13 17:38:02.747139 7fe9bae887a0 20 read_log 192834'13499 (0'0) promote 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by osd.325.0:38122 2014-08-11 15:28:09.040453 2014-08-13 17:38:02.747168 7fe9bae887a0 20 read_log 192834'13500 (192834'13499) modify 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.1883634.0:22506 2014-08-11 15:28:09.030809 2014-08-13 17:38:02.747183 7fe9bae887a0 20 read_log 192834'13519 (192834'13003) clean c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:38187 2014-08-11 15:28:14.474340 2014-08-13 17:38:02.747196 7fe9bae887a0 20 read_log 192834'13520 (192834'12977) clean d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:38188 2014-08-11 15:28:14.477952 2014-08-13 17:38:02.747209 7fe9bae887a0 20 read_log 192834'13521 (192834'13119) clean ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:38189 2014-08-11 15:28:14.478298 2014-08-13 17:38:02.747222 7fe9bae887a0 20 read_log 192834'13522 (192834'13084) clean c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:38190 2014-08-11 15:28:14.491231 2014-08-13 17:38:02.747235 7fe9bae887a0 20 read_log 192834'13523 (192834'13018) clean 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:38191 2014-08-11 15:28:14.584747 2014-08-13 17:38:02.747249 7fe9bae887a0 20 read_log 192834'13524 (192834'13268) clean c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38192 2014-08-11 15:28:14.585093 2014-08-13 17:38:02.747266 7fe9bae887a0 20 read_log 192834'13525 (192834'13324) clean 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38193 2014-08-11 15:28:14.585334 2014-08-13 17:38:02.747278 7fe9bae887a0 20 read_log 192834'13526 (192834'13007) clean 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:38194 2014-08-11 15:28:14.602800 2014-08-13 17:38:02.747290 7fe9bae887a0 20 read_log 192834'13527 (192834'13330) clean 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38195 2014-08-11 15:28:14.803373 2014-08-13 17:38:02.747304 7fe9bae887a0 20 read_log 192834'13528 (192834'13380) clean 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38196 2014-08-11 15:28:14.803718 2014-08-13 17:38:02.747316 7fe9bae887a0 20 read_log 192834'13529 (192834'13160) clean bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38197 2014-08-11 15:28:14.803971 2014-08-13 17:38:02.747330 7fe9bae887a0 20 read_log 192834'13533 (192834'13168) clean 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38201 2014-08-11 15:28:15.062931 2014-08-13 17:38:02.747343 7fe9bae887a0 20 read_log 192834'13539 (192834'13519) delete c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:38207 2014-08-11 15:28:15.240711 2014-08-13 17:38:02.747357 7fe9bae887a0 20 read_log 192834'13540 (192834'13524) delete c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38208 2014-08-11 15:28:15.241046 2014-08-13 17:38:02.747369 7fe9bae887a0 20 read_log 192834'13541 (192834'13520) delete d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:38209 2014-08-11 15:28:15.243429 2014-08-13 17:38:02.747382 7fe9bae887a0 20 read_log 192834'13542 (192834'13521) delete ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:38210 2014-08-11 15:28:15.243740 2014-08-13 17:38:02.747394 7fe9bae887a0 20 read_log 192834'13544 (192834'13522) delete c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:38212 2014-08-11 15:28:15.250835 2014-08-13 17:38:02.747406 7fe9bae887a0 20 read_log 192834'13545 (192834'13525) delete 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38213 2014-08-11 15:28:15.251260 2014-08-13 17:38:02.747417 7fe9bae887a0 20 read_log 192834'13546 (192834'13527) delete 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38214 2014-08-11 15:28:15.254901 2014-08-13 17:38:02.747437 7fe9bae887a0 20 read_log 192834'13547 (192834'13523) delete 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:38215 2014-08-11 15:28:15.255145 2014-08-13 17:38:02.747452 7fe9bae887a0 20 read_log 192834'13548 (192834'13526) delete 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:38216 2014-08-11 15:28:15.261277 2014-08-13 17:38:02.747464 7fe9bae887a0 20 read_log 192834'13549 (192834'13528) delete 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38217 2014-08-11 15:28:15.261849 2014-08-13 17:38:02.747476 7fe9bae887a0 20 read_log 192834'13550 (192834'13529) delete bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38218 2014-08-11 15:28:15.265128 2014-08-13 17:38:02.747488 7fe9bae887a0 20 read_log 192834'13551 (192834'13533) delete 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38219 2014-08-11 15:28:15.265585 2014-08-13 17:38:02.747501 7fe9bae887a0 20 read_log 192834'13565 (192834'13138) modify 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.1883634.0:23629 2014-08-11 15:28:17.294559 2014-08-13 17:38:02.747513 7fe9bae887a0 20 read_log 192834'13568 (192834'13144) modify e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.1883634.0:23916 2014-08-11 15:28:18.077503 2014-08-13 17:38:02.747526 7fe9bae887a0 20 read_log 192834'13569 (192834'13146) modify bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.1883634.0:24157 2014-08-11 15:28:18.421687 2014-08-13 17:38:02.747538 7fe9bae887a0 20 read_log 192834'13576 (0'0) promote bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38256 2014-08-11 15:28:19.583085 2014-08-13 17:38:02.747550 7fe9bae887a0 20 read_log 192834'13577 (192834'13576) modify bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.1883634.0:24807 2014-08-11 15:28:19.551421 2014-08-13 17:38:02.747562 7fe9bae887a0 20 read_log 192834'13581 (0'0) promote 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38262 2014-08-11 15:28:19.954376 2014-08-13 17:38:02.747574 7fe9bae887a0 20 read_log 192834'13582 (192834'13581) modify 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.1883634.0:25012 2014-08-11 15:28:19.927770 2014-08-13 17:38:02.747586 7fe9bae887a0 20 read_log 192834'13591 (192834'13186) modify acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.1883634.0:25646 2014-08-11 15:28:21.670081 2014-08-13 17:38:02.747597 7fe9bae887a0 20 read_log 192834'13594 (192834'13192) modify 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.1883634.0:26054 2014-08-11 15:28:22.566345 2014-08-13 17:38:02.747609 7fe9bae887a0 20 read_log 192834'13597 (192834'13198) modify 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.1883634.0:26546 2014-08-11 15:28:23.244525 2014-08-13 17:38:02.747621 7fe9bae887a0 20 read_log 192834'13602 (192834'13208) modify 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.1883634.0:27293 2014-08-11 15:28:24.351028 2014-08-13 17:38:02.747633 7fe9bae887a0 20 read_log 192834'13603 (192834'13210) modify 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.1883634.0:27350 2014-08-11 15:28:24.395939 2014-08-13 17:38:02.747645 7fe9bae887a0 20 read_log 192834'13605 (192834'13214) modify eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.1883634.0:27696 2014-08-11 15:28:24.831542 2014-08-13 17:38:02.747657 7fe9bae887a0 20 read_log 192834'13607 (192834'13218) modify 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.1883634.0:27780 2014-08-11 15:28:24.970154 2014-08-13 17:38:02.747669 7fe9bae887a0 20 read_log 192834'13609 (192834'13222) modify d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.1883634.0:28891 2014-08-11 15:28:26.458499 2014-08-13 17:38:02.747691 7fe9bae887a0 20 read_log 192834'13615 (192834'13234) modify 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.1883634.0:29181 2014-08-11 15:28:26.826818 2014-08-13 17:38:02.747724 7fe9bae887a0 20 read_log 192834'13617 (192834'13238) modify 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.1883634.0:29251 2014-08-11 15:28:26.891325 2014-08-13 17:38:02.747740 7fe9bae887a0 20 read_log 192834'13626 (192834'13256) modify e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.1883634.0:30460 2014-08-11 15:28:28.658110 2014-08-13 17:38:02.747752 7fe9bae887a0 20 read_log 192834'13632 (0'0) promote c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38385 2014-08-11 15:28:29.974667 2014-08-13 17:38:02.747764 7fe9bae887a0 20 read_log 192834'13633 (192834'13632) modify c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.1883634.0:31131 2014-08-11 15:28:29.940288 2014-08-13 17:38:02.747776 7fe9bae887a0 20 read_log 192834'13634 (192834'13270) modify 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.1883634.0:31306 2014-08-11 15:28:30.255515 2014-08-13 17:38:02.747788 7fe9bae887a0 20 read_log 192834'13653 (192834'13308) modify b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.1883634.0:33566 2014-08-11 15:28:33.491079 2014-08-13 17:38:02.747800 7fe9bae887a0 20 read_log 192834'13656 (192834'13314) modify 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.1883634.0:33907 2014-08-11 15:28:33.989691 2014-08-13 17:38:02.747812 7fe9bae887a0 20 read_log 192834'13661 (0'0) promote 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38435 2014-08-11 15:28:35.803610 2014-08-13 17:38:02.747824 7fe9bae887a0 20 read_log 192834'13662 (192834'13661) modify 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.1883634.0:34994 2014-08-11 15:28:35.779699 2014-08-13 17:38:02.747836 7fe9bae887a0 20 read_log 192834'13665 (0'0) promote 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38439 2014-08-11 15:28:36.119357 2014-08-13 17:38:02.747848 7fe9bae887a0 20 read_log 192834'13666 (192834'13665) modify 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.1883634.0:35144 2014-08-11 15:28:36.092811 2014-08-13 17:38:02.747860 7fe9bae887a0 20 read_log 192834'13668 (192834'13668) modify 34/hit_set_24.34_archive_2014-08-11 15:23:36.202668_2014-08-11 15:28:36.550941/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:36.551245 2014-08-13 17:38:02.747873 7fe9bae887a0 20 read_log 192834'13669 (192834'12664) delete 34/hit_set_24.34_archive_2014-08-11 10:24:43.621609_2014-08-11 15:23:36.202456/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:36.551245 2014-08-13 17:38:02.747885 7fe9bae887a0 20 read_log 192834'13676 (192834'13346) modify be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.1883634.0:35978 2014-08-11 15:28:37.421767 2014-08-13 17:38:02.747897 7fe9bae887a0 20 read_log 192834'13695 (0'0) promote 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38501 2014-08-11 15:28:39.988239 2014-08-13 17:38:02.747909 7fe9bae887a0 20 read_log 192834'13696 (192834'13695) modify 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.1883634.0:37927 2014-08-11 15:28:39.963727 2014-08-13 17:38:02.747921 7fe9bae887a0 20 read_log 192834'13715 (192834'13418) modify ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.1883634.0:40713 2014-08-11 15:28:44.163345 2014-08-13 17:38:02.747933 7fe9bae887a0 20 read_log 192834'13725 (192834'13438) modify af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.1883634.0:42241 2014-08-11 15:28:46.179940 2014-08-13 17:38:02.747952 7fe9bae887a0 20 read_log 192834'13727 (192834'13442) modify c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.1883634.0:42338 2014-08-11 15:28:46.212260 2014-08-13 17:38:02.747965 7fe9bae887a0 20 read_log 192834'13729 (192834'13446) modify 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.1883634.0:42569 2014-08-11 15:28:46.284432 2014-08-13 17:38:02.747977 7fe9bae887a0 20 read_log 192834'13738 (192834'13464) modify 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.1883634.0:43321 2014-08-11 15:28:46.519268 2014-08-13 17:38:02.747990 7fe9bae887a0 20 read_log 192834'13739 (192834'13466) modify 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.1883634.0:43326 2014-08-11 15:28:46.521225 2014-08-13 17:38:02.748002 7fe9bae887a0 20 read_log 192834'13756 (192834'13500) modify 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.1883634.0:45830 2014-08-11 15:28:47.404291 2014-08-13 17:38:02.748014 7fe9bae887a0 20 read_log 192888'13766 (192888'13766) modify 34/hit_set_24.34_archive_2014-08-11 16:02:02.123078_2014-08-11 16:44:55.132899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.133197 2014-08-13 17:38:02.748026 7fe9bae887a0 20 read_log 192888'13767 (192834'13668) delete 34/hit_set_24.34_archive_2014-08-11 15:23:36.202668_2014-08-11 15:28:36.550941/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.133197 2014-08-13 17:38:02.748038 7fe9bae887a0 20 read_log 192888'13776 (0'0) promote ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by osd.325.0:41204 2014-08-11 16:45:03.225548 2014-08-13 17:38:02.748050 7fe9bae887a0 20 read_log 192888'13777 (192888'13776) modify ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.1315259.0:2095 2014-08-11 16:45:03.216217 2014-08-13 17:38:02.748062 7fe9bae887a0 20 read_log 192888'13780 (0'0) promote 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by osd.325.0:41220 2014-08-11 16:45:07.114793 2014-08-13 17:38:02.748075 7fe9bae887a0 20 read_log 192888'13781 (192888'13780) modify 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.1315259.0:3126 2014-08-11 16:45:07.106157 2014-08-13 17:38:02.748087 7fe9bae887a0 20 read_log 192888'13786 (0'0) promote e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by osd.325.0:41278 2014-08-11 16:45:22.725451 2014-08-13 17:38:02.748099 7fe9bae887a0 20 read_log 192888'13787 (192888'13786) modify e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.1315259.0:7367 2014-08-11 16:45:22.719005 2014-08-13 17:38:02.748111 7fe9bae887a0 20 read_log 192888'13792 (0'0) promote 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by osd.325.0:41312 2014-08-11 16:45:30.170176 2014-08-13 17:38:02.748125 7fe9bae887a0 20 read_log 192888'13793 (192888'13792) modify 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.1315259.0:9364 2014-08-11 16:45:30.162162 2014-08-13 17:38:02.748137 7fe9bae887a0 20 read_log 192888'13794 (0'0) promote f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by osd.325.0:41314 2014-08-11 16:45:30.498376 2014-08-13 17:38:02.748149 7fe9bae887a0 20 read_log 192888'13795 (192888'13794) modify f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.1315259.0:9428 2014-08-11 16:45:30.489392 2014-08-13 17:38:02.748166 7fe9bae887a0 20 read_log 192888'13800 (0'0) promote 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by osd.325.0:41328 2014-08-11 16:45:32.235728 2014-08-13 17:38:02.748178 7fe9bae887a0 20 read_log 192888'13801 (192888'13800) modify 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.1315259.0:9881 2014-08-11 16:45:32.228172 2014-08-13 17:38:02.748215 7fe9bae887a0 20 read_log 192888'13804 (0'0) promote a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by osd.325.0:41350 2014-08-11 16:45:41.176435 2014-08-13 17:38:02.748231 7fe9bae887a0 20 read_log 192888'13805 (192888'13804) modify a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.1315259.0:12125 2014-08-11 16:45:41.166092 2014-08-13 17:38:02.748244 7fe9bae887a0 20 read_log 192888'13808 (0'0) promote a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by osd.325.0:41362 2014-08-11 16:45:43.730156 2014-08-13 17:38:02.748256 7fe9bae887a0 20 read_log 192888'13809 (192888'13808) modify a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.1315259.0:12746 2014-08-11 16:45:43.721723 2014-08-13 17:38:02.748271 7fe9bae887a0 20 read_log 192888'13824 (0'0) promote 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by osd.325.0:41444 2014-08-11 16:46:03.397114 2014-08-13 17:38:02.748286 7fe9bae887a0 20 read_log 192888'13825 (192888'13824) modify 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.1315259.0:17756 2014-08-11 16:46:03.388532 2014-08-13 17:38:02.748298 7fe9bae887a0 20 read_log 192888'13826 (0'0) promote 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by osd.325.0:41470 2014-08-11 16:46:11.096695 2014-08-13 17:38:02.748310 7fe9bae887a0 20 read_log 192888'13827 (192888'13826) modify 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.1315259.0:19720 2014-08-11 16:46:11.087923 2014-08-13 17:38:02.748324 7fe9bae887a0 20 read_log 192888'13836 (0'0) promote 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by osd.325.0:41494 2014-08-11 16:46:13.923117 2014-08-13 17:38:02.748336 7fe9bae887a0 20 read_log 192888'13837 (192888'13836) modify 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.1315259.0:20487 2014-08-11 16:46:13.914844 2014-08-13 17:38:02.748348 7fe9bae887a0 20 read_log 192888'13844 (0'0) promote 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by osd.325.0:41520 2014-08-11 16:46:20.301034 2014-08-13 17:38:02.748360 7fe9bae887a0 20 read_log 192888'13845 (192888'13844) modify 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.1315259.0:22152 2014-08-11 16:46:20.292833 2014-08-13 17:38:02.748372 7fe9bae887a0 20 read_log 192888'13846 (0'0) promote ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by osd.325.0:41524 2014-08-11 16:46:20.945616 2014-08-13 17:38:02.748387 7fe9bae887a0 20 read_log 192888'13847 (192888'13846) modify ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.1315259.0:22328 2014-08-11 16:46:20.937522 2014-08-13 17:38:02.748399 7fe9bae887a0 20 read_log 192888'13848 (0'0) promote d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by osd.325.0:41526 2014-08-11 16:46:20.959256 2014-08-13 17:38:02.748411 7fe9bae887a0 20 read_log 192888'13849 (192888'13848) modify d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.1315259.0:22331 2014-08-11 16:46:20.948212 2014-08-13 17:38:02.748423 7fe9bae887a0 20 read_log 192888'13852 (0'0) promote 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by osd.325.0:41544 2014-08-11 16:46:23.958695 2014-08-13 17:38:02.748435 7fe9bae887a0 20 read_log 192888'13853 (192888'13852) modify 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.1315259.0:23065 2014-08-11 16:46:23.950295 2014-08-13 17:38:02.748447 7fe9bae887a0 20 read_log 192888'13856 (0'0) promote 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by osd.325.0:41564 2014-08-11 16:46:28.965343 2014-08-13 17:38:02.748459 7fe9bae887a0 20 read_log 192888'13857 (192888'13856) modify 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.1315259.0:24353 2014-08-11 16:46:28.956645 2014-08-13 17:38:02.748478 7fe9bae887a0 20 read_log 192888'13858 (0'0) promote 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by osd.325.0:41566 2014-08-11 16:46:28.995305 2014-08-13 17:38:02.748490 7fe9bae887a0 20 read_log 192888'13859 (192888'13858) modify 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.1315259.0:24361 2014-08-11 16:46:28.985585 2014-08-13 17:38:02.748502 7fe9bae887a0 20 read_log 192888'13860 (0'0) promote fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by osd.325.0:41570 2014-08-11 16:46:29.341322 2014-08-13 17:38:02.748514 7fe9bae887a0 20 read_log 192888'13861 (192888'13860) modify fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.1315259.0:24447 2014-08-11 16:46:29.332786 2014-08-13 17:38:02.748526 7fe9bae887a0 20 read_log 192888'13868 (0'0) promote 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by osd.325.0:41600 2014-08-11 16:46:36.822492 2014-08-13 17:38:02.748538 7fe9bae887a0 20 read_log 192888'13869 (192888'13868) modify 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.1315259.0:26254 2014-08-11 16:46:36.810318 2014-08-13 17:38:02.748550 7fe9bae887a0 20 read_log 192888'13872 (0'0) promote 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by osd.325.0:41616 2014-08-11 16:46:39.085616 2014-08-13 17:38:02.748562 7fe9bae887a0 20 read_log 192888'13873 (192888'13872) modify 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.1315259.0:26804 2014-08-11 16:46:39.077378 2014-08-13 17:38:02.748574 7fe9bae887a0 20 read_log 192888'13878 (0'0) promote da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by osd.325.0:41642 2014-08-11 16:46:44.960106 2014-08-13 17:38:02.748586 7fe9bae887a0 20 read_log 192888'13879 (192888'13878) modify da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.1315259.0:28234 2014-08-11 16:46:44.952650 2014-08-13 17:38:02.748598 7fe9bae887a0 20 read_log 192888'13882 (0'0) promote 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by osd.325.0:41660 2014-08-11 16:46:47.909597 2014-08-13 17:38:02.748614 7fe9bae887a0 20 read_log 192888'13883 (192888'13882) modify 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.1315259.0:28969 2014-08-11 16:46:47.898577 2014-08-13 17:38:02.748628 7fe9bae887a0 20 read_log 192888'13884 (0'0) promote aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by osd.325.0:41674 2014-08-11 16:46:49.926443 2014-08-13 17:38:02.748640 7fe9bae887a0 20 read_log 192888'13885 (192888'13884) modify aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.1315259.0:29450 2014-08-11 16:46:49.916371 2014-08-13 17:38:02.748652 7fe9bae887a0 20 read_log 192888'13886 (0'0) promote 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by osd.325.0:41682 2014-08-11 16:46:51.389969 2014-08-13 17:38:02.748663 7fe9bae887a0 20 read_log 192888'13887 (192888'13886) modify 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.1315259.0:29808 2014-08-11 16:46:51.382545 2014-08-13 17:38:02.748676 7fe9bae887a0 20 read_log 192888'13888 (0'0) promote aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by osd.325.0:41684 2014-08-11 16:46:52.354474 2014-08-13 17:38:02.748687 7fe9bae887a0 20 read_log 192888'13889 (192888'13888) modify aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.1315259.0:30065 2014-08-11 16:46:52.346807 2014-08-13 17:38:02.748699 7fe9bae887a0 20 read_log 192888'13894 (192888'13777) modify ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.1315259.0:32822 2014-08-11 16:46:56.479027 2014-08-13 17:38:02.748711 7fe9bae887a0 20 read_log 192888'13896 (192888'13781) modify 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.1315259.0:33853 2014-08-11 16:46:56.741739 2014-08-13 17:38:02.748730 7fe9bae887a0 20 read_log 192888'13899 (192888'13787) modify e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.1315259.0:38094 2014-08-11 16:46:57.855674 2014-08-13 17:38:02.748743 7fe9bae887a0 20 read_log 192888'13902 (192888'13793) modify 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.1315259.0:40091 2014-08-11 16:46:58.384975 2014-08-13 17:38:02.748755 7fe9bae887a0 20 read_log 192888'13903 (192888'13795) modify f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.1315259.0:40155 2014-08-11 16:46:58.404083 2014-08-13 17:38:02.748767 7fe9bae887a0 20 read_log 192888'13906 (192888'13801) modify 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.1315259.0:40608 2014-08-11 16:46:58.536116 2014-08-13 17:38:02.748779 7fe9bae887a0 20 read_log 192888'13908 (192888'13805) modify a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.1315259.0:42852 2014-08-11 16:46:59.122739 2014-08-13 17:38:02.748790 7fe9bae887a0 20 read_log 192888'13910 (192888'13809) modify a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.1315259.0:43473 2014-08-11 16:46:59.301445 2014-08-13 17:38:02.748802 7fe9bae887a0 20 read_log 192888'13918 (192888'13825) modify 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.1315259.0:48483 2014-08-11 16:47:00.775823 2014-08-13 17:38:02.748814 7fe9bae887a0 20 read_log 192888'13919 (192888'13827) modify 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.1315259.0:50447 2014-08-11 16:47:01.372700 2014-08-13 17:38:02.748827 7fe9bae887a0 20 read_log 192888'13924 (192888'13837) modify 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.1315259.0:51214 2014-08-11 16:47:01.612613 2014-08-13 17:38:02.748839 7fe9bae887a0 20 read_log 192888'13928 (192888'13845) modify 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.1315259.0:52879 2014-08-11 16:47:02.094252 2014-08-13 17:38:02.748851 7fe9bae887a0 20 read_log 192888'13929 (192888'13847) modify ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.1315259.0:53055 2014-08-11 16:47:02.142537 2014-08-13 17:38:02.748863 7fe9bae887a0 20 read_log 192888'13930 (192888'13849) modify d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.1315259.0:53058 2014-08-11 16:47:02.143193 2014-08-13 17:38:02.748875 7fe9bae887a0 20 read_log 192888'13932 (192888'13853) modify 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.1315259.0:53792 2014-08-11 16:47:02.363870 2014-08-13 17:38:02.748886 7fe9bae887a0 20 read_log 192888'13934 (192888'13857) modify 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.1315259.0:55080 2014-08-11 16:47:02.804344 2014-08-13 17:38:02.748898 7fe9bae887a0 20 read_log 192888'13935 (192888'13859) modify 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.1315259.0:55088 2014-08-11 16:47:02.807175 2014-08-13 17:38:02.748910 7fe9bae887a0 20 read_log 192888'13936 (192888'13861) modify fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.1315259.0:55174 2014-08-11 16:47:02.833347 2014-08-13 17:38:02.748940 7fe9bae887a0 20 read_log 192888'13940 (192888'13869) modify 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.1315259.0:56981 2014-08-11 16:47:03.372659 2014-08-13 17:38:02.748953 7fe9bae887a0 20 read_log 192888'13942 (192888'13873) modify 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.1315259.0:57531 2014-08-11 16:47:03.541536 2014-08-13 17:38:02.748965 7fe9bae887a0 20 read_log 192888'13945 (192888'13879) modify da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.1315259.0:58961 2014-08-11 16:47:03.958789 2014-08-13 17:38:02.748978 7fe9bae887a0 20 read_log 192888'13947 (192888'13883) modify 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.1315259.0:59696 2014-08-11 16:47:04.213409 2014-08-13 17:38:02.749000 7fe9bae887a0 20 read_log 192888'13948 (192888'13885) modify aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.1315259.0:60177 2014-08-11 16:47:04.370420 2014-08-13 17:38:02.749013 7fe9bae887a0 20 read_log 192888'13949 (192888'13887) modify 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.1315259.0:60535 2014-08-11 16:47:04.470210 2014-08-13 17:38:02.749025 7fe9bae887a0 20 read_log 192888'13950 (192888'13889) modify aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.1315259.0:60792 2014-08-11 16:47:04.542488 2014-08-13 17:38:02.749037 7fe9bae887a0 20 read_log 192888'13951 (192888'13951) modify 34/hit_set_24.34_archive_2014-08-11 16:44:55.133201_2014-08-11 17:05:14.852362/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.852638 2014-08-13 17:38:02.749053 7fe9bae887a0 20 read_log 192888'13952 (192888'13766) delete 34/hit_set_24.34_archive_2014-08-11 16:02:02.123078_2014-08-11 16:44:55.132899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.852638 2014-08-13 17:38:02.749066 7fe9bae887a0 20 read_log 192888'13955 (0'0) promote 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by osd.325.0:41968 2014-08-11 17:05:17.483027 2014-08-13 17:38:02.749078 7fe9bae887a0 20 read_log 192888'13956 (192888'13955) modify 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.942868.0:603 2014-08-11 17:05:17.474818 2014-08-13 17:38:02.749090 7fe9bae887a0 20 read_log 192888'13957 (0'0) promote aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by osd.325.0:41979 2014-08-11 17:05:18.921701 2014-08-13 17:38:02.749103 7fe9bae887a0 20 read_log 192888'13958 (192888'13957) modify aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.942868.0:957 2014-08-11 17:05:18.911644 2014-08-13 17:38:02.749116 7fe9bae887a0 20 read_log 192888'13967 (0'0) promote 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by osd.325.0:42036 2014-08-11 17:05:30.136811 2014-08-13 17:38:02.749128 7fe9bae887a0 20 read_log 192888'13968 (192888'13967) modify 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.942868.0:3837 2014-08-11 17:05:30.128369 2014-08-13 17:38:02.749140 7fe9bae887a0 20 read_log 192888'13969 (0'0) promote d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by osd.325.0:42060 2014-08-11 17:05:32.614452 2014-08-13 17:38:02.749152 7fe9bae887a0 20 read_log 192888'13970 (192888'13969) modify d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.942868.0:4508 2014-08-11 17:05:32.605964 2014-08-13 17:38:02.749178 7fe9bae887a0 20 read_log 192888'13975 (0'0) promote 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by osd.325.0:42108 2014-08-11 17:05:45.572580 2014-08-13 17:38:02.749191 7fe9bae887a0 20 read_log 192888'13976 (192888'13975) modify 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.942868.0:7929 2014-08-11 17:05:45.564407 2014-08-13 17:38:02.749203 7fe9bae887a0 20 read_log 192888'13977 (0'0) promote 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by osd.325.0:42110 2014-08-11 17:05:45.625064 2014-08-13 17:38:02.749215 7fe9bae887a0 20 read_log 192888'13978 (192888'13977) modify 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.942868.0:7944 2014-08-11 17:05:45.618097 2014-08-13 17:38:02.749227 7fe9bae887a0 20 read_log 192888'13981 (0'0) promote 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by osd.325.0:42126 2014-08-11 17:05:49.466104 2014-08-13 17:38:02.749239 7fe9bae887a0 20 read_log 192888'13982 (192888'13981) modify 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.942868.0:8984 2014-08-11 17:05:49.457056 2014-08-13 17:38:02.749258 7fe9bae887a0 20 read_log 192888'13985 (0'0) promote 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by osd.325.0:42164 2014-08-11 17:05:56.585904 2014-08-13 17:38:02.749271 7fe9bae887a0 20 read_log 192888'13986 (192888'13985) modify 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.942868.0:10941 2014-08-11 17:05:56.577607 2014-08-13 17:38:02.749283 7fe9bae887a0 20 read_log 192888'13987 (0'0) promote d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by osd.325.0:42184 2014-08-11 17:06:00.406601 2014-08-13 17:38:02.749295 7fe9bae887a0 20 read_log 192888'13988 (192888'13987) modify d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.942868.0:11971 2014-08-11 17:06:00.398551 2014-08-13 17:38:02.749307 7fe9bae887a0 20 read_log 192888'13991 (0'0) promote 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by osd.325.0:42212 2014-08-11 17:06:04.404024 2014-08-13 17:38:02.749319 7fe9bae887a0 20 read_log 192888'13992 (192888'13991) modify 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.942868.0:13059 2014-08-11 17:06:04.395783 2014-08-13 17:38:02.749331 7fe9bae887a0 20 read_log 192888'13993 (0'0) promote 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by osd.325.0:42222 2014-08-11 17:06:07.575289 2014-08-13 17:38:02.749343 7fe9bae887a0 20 read_log 192888'13994 (192888'13993) modify 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.942868.0:13853 2014-08-11 17:06:07.566720 2014-08-13 17:38:02.749355 7fe9bae887a0 20 read_log 192888'13997 (0'0) promote 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by osd.325.0:42238 2014-08-11 17:06:11.813054 2014-08-13 17:38:02.749368 7fe9bae887a0 20 read_log 192888'13998 (192888'13997) modify 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.942868.0:14895 2014-08-11 17:06:11.804229 2014-08-13 17:38:02.749380 7fe9bae887a0 20 read_log 192888'13999 (0'0) promote ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by osd.325.0:42250 2014-08-11 17:06:13.011641 2014-08-13 17:38:02.749392 7fe9bae887a0 20 read_log 192888'14000 (192888'13999) modify ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.942868.0:15202 2014-08-11 17:06:13.001241 2014-08-13 17:38:02.749404 7fe9bae887a0 20 read_log 192888'14003 (0'0) promote 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by osd.325.0:42270 2014-08-11 17:06:16.388644 2014-08-13 17:38:02.749416 7fe9bae887a0 20 read_log 192888'14004 (192888'14003) modify 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.942868.0:16072 2014-08-11 17:06:16.379378 2014-08-13 17:38:02.749428 7fe9bae887a0 20 read_log 192888'14007 (0'0) promote 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by osd.325.0:42294 2014-08-11 17:06:20.659334 2014-08-13 17:38:02.749439 7fe9bae887a0 20 read_log 192888'14008 (192888'14007) modify 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.942868.0:17158 2014-08-11 17:06:20.649920 2014-08-13 17:38:02.749451 7fe9bae887a0 20 read_log 192888'14009 (0'0) promote dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by osd.325.0:42298 2014-08-11 17:06:22.178318 2014-08-13 17:38:02.749463 7fe9bae887a0 20 read_log 192888'14010 (192888'14009) modify dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.942868.0:17560 2014-08-11 17:06:22.171320 2014-08-13 17:38:02.749475 7fe9bae887a0 20 read_log 192888'14011 (0'0) promote 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by osd.325.0:42318 2014-08-11 17:06:25.165994 2014-08-13 17:38:02.749487 7fe9bae887a0 20 read_log 192888'14012 (192888'14011) modify 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.942868.0:18352 2014-08-11 17:06:25.156221 2014-08-13 17:38:02.749505 7fe9bae887a0 20 read_log 192888'14013 (0'0) promote 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by osd.325.0:42322 2014-08-11 17:06:25.594173 2014-08-13 17:38:02.749518 7fe9bae887a0 20 read_log 192888'14014 (192888'14013) modify 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.942868.0:18467 2014-08-11 17:06:25.586179 2014-08-13 17:38:02.749530 7fe9bae887a0 20 read_log 192888'14015 (0'0) promote b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by osd.325.0:42326 2014-08-11 17:06:26.409435 2014-08-13 17:38:02.749542 7fe9bae887a0 20 read_log 192888'14016 (192888'14015) modify b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.942868.0:18682 2014-08-11 17:06:26.400579 2014-08-13 17:38:02.749554 7fe9bae887a0 20 read_log 192888'14017 (0'0) promote afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by osd.325.0:42338 2014-08-11 17:06:30.172064 2014-08-13 17:38:02.749566 7fe9bae887a0 20 read_log 192888'14018 (192888'14017) modify afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.942868.0:19686 2014-08-11 17:06:30.163483 2014-08-13 17:38:02.749600 7fe9bae887a0 20 read_log 192888'14019 (0'0) promote f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by osd.325.0:42346 2014-08-11 17:06:31.992911 2014-08-13 17:38:02.749612 7fe9bae887a0 20 read_log 192888'14020 (192888'14019) modify f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.942868.0:20169 2014-08-11 17:06:31.983404 2014-08-13 17:38:02.749624 7fe9bae887a0 20 read_log 192888'14021 (0'0) promote b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by osd.325.0:42358 2014-08-11 17:06:33.484075 2014-08-13 17:38:02.749636 7fe9bae887a0 20 read_log 192888'14022 (192888'14021) modify b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.942868.0:20547 2014-08-11 17:06:33.474245 2014-08-13 17:38:02.749648 7fe9bae887a0 20 read_log 192888'14023 (0'0) promote 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by osd.325.0:42364 2014-08-11 17:06:35.222159 2014-08-13 17:38:02.749660 7fe9bae887a0 20 read_log 192888'14024 (192888'14023) modify 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.942868.0:21006 2014-08-11 17:06:35.213284 2014-08-13 17:38:02.749672 7fe9bae887a0 20 read_log 192888'14025 (0'0) promote 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by osd.325.0:42368 2014-08-11 17:06:35.728586 2014-08-13 17:38:02.749684 7fe9bae887a0 20 read_log 192888'14026 (192888'14025) modify 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.942868.0:21137 2014-08-11 17:06:35.718903 2014-08-13 17:38:02.749696 7fe9bae887a0 20 read_log 192888'14029 (0'0) promote 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by osd.325.0:42378 2014-08-11 17:06:36.805887 2014-08-13 17:38:02.749708 7fe9bae887a0 20 read_log 192888'14030 (192888'14029) modify 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.942868.0:21417 2014-08-11 17:06:36.798097 2014-08-13 17:38:02.749721 7fe9bae887a0 20 read_log 192888'14031 (0'0) promote b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by osd.325.0:42380 2014-08-11 17:06:37.379137 2014-08-13 17:38:02.749732 7fe9bae887a0 20 read_log 192888'14032 (192888'14031) modify b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.942868.0:21571 2014-08-11 17:06:37.368920 2014-08-13 17:38:02.749744 7fe9bae887a0 20 read_log 192888'14037 (0'0) promote b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by osd.325.0:42438 2014-08-11 17:06:49.187131 2014-08-13 17:38:02.749756 7fe9bae887a0 20 read_log 192888'14038 (192888'14037) modify b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.942868.0:24668 2014-08-11 17:06:49.177825 2014-08-13 17:38:02.749778 7fe9bae887a0 20 read_log 192888'14039 (0'0) promote 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by osd.325.0:42440 2014-08-11 17:06:49.798357 2014-08-13 17:38:02.749790 7fe9bae887a0 20 read_log 192888'14040 (192888'14039) modify 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.942868.0:24834 2014-08-11 17:06:49.788888 2014-08-13 17:38:02.749803 7fe9bae887a0 20 read_log 192888'14041 (0'0) promote 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by osd.325.0:42458 2014-08-11 17:06:51.833551 2014-08-13 17:38:02.749817 7fe9bae887a0 20 read_log 192888'14042 (192888'14041) modify 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.942868.0:25374 2014-08-11 17:06:51.824651 2014-08-13 17:38:02.749829 7fe9bae887a0 20 read_log 192888'14043 (0'0) promote 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by osd.325.0:42460 2014-08-11 17:06:52.213821 2014-08-13 17:38:02.749843 7fe9bae887a0 20 read_log 192888'14044 (192888'14043) modify 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.942868.0:25476 2014-08-11 17:06:52.205502 2014-08-13 17:38:02.749855 7fe9bae887a0 20 read_log 192888'14045 (0'0) promote a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by osd.325.0:42468 2014-08-11 17:06:54.688180 2014-08-13 17:38:02.749867 7fe9bae887a0 20 read_log 192888'14046 (192888'14045) modify a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.942868.0:26066 2014-08-11 17:06:54.679117 2014-08-13 17:38:02.749879 7fe9bae887a0 20 read_log 192888'14049 (0'0) promote 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by osd.325.0:42488 2014-08-11 17:06:57.804947 2014-08-13 17:38:02.749891 7fe9bae887a0 20 read_log 192888'14050 (192888'14049) modify 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.942868.0:26884 2014-08-11 17:06:57.794638 2014-08-13 17:38:02.749903 7fe9bae887a0 20 read_log 192888'14051 (0'0) promote 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by osd.325.0:42490 2014-08-11 17:06:57.920508 2014-08-13 17:38:02.749915 7fe9bae887a0 20 read_log 192888'14052 (192888'14051) modify 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.942868.0:26916 2014-08-11 17:06:57.910888 2014-08-13 17:38:02.749928 7fe9bae887a0 20 read_log 192888'14053 (0'0) promote e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by osd.325.0:42496 2014-08-11 17:06:58.867378 2014-08-13 17:38:02.749940 7fe9bae887a0 20 read_log 192888'14054 (192888'14053) modify e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.942868.0:27174 2014-08-11 17:06:58.858282 2014-08-13 17:38:02.749952 7fe9bae887a0 20 read_log 192888'14055 (0'0) promote e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by osd.325.0:42514 2014-08-11 17:07:01.059380 2014-08-13 17:38:02.749967 7fe9bae887a0 20 read_log 192888'14056 (192888'14055) modify e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.942868.0:27749 2014-08-11 17:07:01.051225 2014-08-13 17:38:02.749979 7fe9bae887a0 20 read_log 192888'14063 (0'0) promote 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by osd.325.0:42568 2014-08-11 17:07:13.975232 2014-08-13 17:38:02.749992 7fe9bae887a0 20 read_log 192888'14064 (192888'14063) modify 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.942868.0:31044 2014-08-11 17:07:13.965349 2014-08-13 17:38:02.750004 7fe9bae887a0 20 read_log 192888'14066 (192888'13956) modify 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.942868.0:31869 2014-08-11 17:07:16.082077 2014-08-13 17:38:02.750016 7fe9bae887a0 20 read_log 192888'14067 (192888'13958) modify aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.942868.0:32223 2014-08-11 17:07:16.192158 2014-08-13 17:38:02.750055 7fe9bae887a0 20 read_log 192888'14072 (192888'13968) modify 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.942868.0:35103 2014-08-11 17:07:16.984859 2014-08-13 17:38:02.750071 7fe9bae887a0 20 read_log 192888'14073 (192888'13970) modify d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.942868.0:35774 2014-08-11 17:07:17.160851 2014-08-13 17:38:02.750103 7fe9bae887a0 20 read_log 192888'14076 (192888'13976) modify 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.942868.0:39195 2014-08-11 17:07:18.113142 2014-08-13 17:38:02.750116 7fe9bae887a0 20 read_log 192888'14077 (192888'13978) modify 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.942868.0:39210 2014-08-11 17:07:18.118746 2014-08-13 17:38:02.750128 7fe9bae887a0 20 read_log 192888'14079 (192888'13982) modify 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.942868.0:40250 2014-08-11 17:07:18.402282 2014-08-13 17:38:02.750140 7fe9bae887a0 20 read_log 192888'14081 (192888'13986) modify 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.942868.0:42207 2014-08-11 17:07:18.935502 2014-08-13 17:38:02.750152 7fe9bae887a0 20 read_log 192888'14082 (192888'13988) modify d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.942868.0:43237 2014-08-11 17:07:19.217169 2014-08-13 17:38:02.750173 7fe9bae887a0 20 read_log 192888'14084 (192888'13992) modify 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.942868.0:44325 2014-08-11 17:07:19.535749 2014-08-13 17:38:02.750187 7fe9bae887a0 20 read_log 192888'14085 (192888'13994) modify 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.942868.0:45119 2014-08-11 17:07:19.751046 2014-08-13 17:38:02.750200 7fe9bae887a0 20 read_log 192888'14087 (192888'13998) modify 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.942868.0:46161 2014-08-11 17:07:20.039116 2014-08-13 17:38:02.750228 7fe9bae887a0 20 read_log 192888'14088 (192888'14000) modify ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.942868.0:46468 2014-08-11 17:07:20.131420 2014-08-13 17:38:02.750243 7fe9bae887a0 20 read_log 192888'14090 (192888'14004) modify 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.942868.0:47338 2014-08-11 17:07:20.375451 2014-08-13 17:38:02.750255 7fe9bae887a0 20 read_log 192888'14092 (192888'14008) modify 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.942868.0:48424 2014-08-11 17:07:20.716059 2014-08-13 17:38:02.750267 7fe9bae887a0 20 read_log 192888'14093 (192888'14010) modify dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.942868.0:48826 2014-08-11 17:07:20.830387 2014-08-13 17:38:02.750279 7fe9bae887a0 20 read_log 192888'14094 (192888'14012) modify 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.942868.0:49618 2014-08-11 17:07:21.069658 2014-08-13 17:38:02.750292 7fe9bae887a0 20 read_log 192888'14095 (192888'14014) modify 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.942868.0:49733 2014-08-11 17:07:21.105046 2014-08-13 17:38:02.750304 7fe9bae887a0 20 read_log 192888'14096 (192888'14016) modify b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.942868.0:49948 2014-08-11 17:07:21.171232 2014-08-13 17:38:02.750316 7fe9bae887a0 20 read_log 192888'14097 (192888'14018) modify afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.942868.0:50952 2014-08-11 17:07:21.482590 2014-08-13 17:38:02.750328 7fe9bae887a0 20 read_log 192888'14098 (192888'14020) modify f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.942868.0:51435 2014-08-11 17:07:21.648490 2014-08-13 17:38:02.750340 7fe9bae887a0 20 read_log 192888'14099 (192888'14022) modify b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.942868.0:51813 2014-08-11 17:07:21.760172 2014-08-13 17:38:02.750359 7fe9bae887a0 20 read_log 192888'14100 (192888'14024) modify 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.942868.0:52272 2014-08-11 17:07:21.889317 2014-08-13 17:38:02.750372 7fe9bae887a0 20 read_log 192888'14101 (192888'14026) modify 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.942868.0:52403 2014-08-11 17:07:21.929550 2014-08-13 17:38:02.750384 7fe9bae887a0 20 read_log 192888'14103 (192888'14030) modify 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.942868.0:52683 2014-08-11 17:07:22.018787 2014-08-13 17:38:02.750396 7fe9bae887a0 20 read_log 192888'14104 (192888'14032) modify b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.942868.0:52837 2014-08-11 17:07:22.065225 2014-08-13 17:38:02.750408 7fe9bae887a0 20 read_log 192888'14107 (192888'14038) modify b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.942868.0:55934 2014-08-11 17:07:23.000304 2014-08-13 17:38:02.750420 7fe9bae887a0 20 read_log 192888'14108 (192888'14040) modify 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.942868.0:56100 2014-08-11 17:07:23.053218 2014-08-13 17:38:02.750432 7fe9bae887a0 20 read_log 192888'14109 (192888'14042) modify 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.942868.0:56640 2014-08-11 17:07:23.215269 2014-08-13 17:38:02.750445 7fe9bae887a0 20 read_log 192888'14110 (192888'14044) modify 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.942868.0:56742 2014-08-11 17:07:23.244998 2014-08-13 17:38:02.750456 7fe9bae887a0 20 read_log 192888'14111 (192888'14046) modify a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.942868.0:57332 2014-08-11 17:07:23.425329 2014-08-13 17:38:02.750468 7fe9bae887a0 20 read_log 192888'14113 (192888'14050) modify 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.942868.0:58150 2014-08-11 17:07:23.657057 2014-08-13 17:38:02.750480 7fe9bae887a0 20 read_log 192888'14114 (192888'14052) modify 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.942868.0:58182 2014-08-11 17:07:23.668807 2014-08-13 17:38:02.750492 7fe9bae887a0 20 read_log 192888'14115 (192888'14054) modify e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.942868.0:58440 2014-08-11 17:07:23.743641 2014-08-13 17:38:02.750504 7fe9bae887a0 20 read_log 192888'14116 (192888'14056) modify e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.942868.0:59015 2014-08-11 17:07:23.919817 2014-08-13 17:38:02.750516 7fe9bae887a0 20 read_log 192888'14120 (192888'14064) modify 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.942868.0:62310 2014-08-11 17:07:24.912843 2014-08-13 17:38:02.750528 7fe9bae887a0 20 read_log 193305'14121 (193305'14121) modify 34/hit_set_24.34_archive_2014-08-11 18:37:04.940322_2014-08-11 18:42:40.475022/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.475293 2014-08-13 17:38:02.750541 7fe9bae887a0 20 read_log 193305'14122 (192888'13951) delete 34/hit_set_24.34_archive_2014-08-11 16:44:55.133201_2014-08-11 17:05:14.852362/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.475293 2014-08-13 17:38:02.750553 7fe9bae887a0 20 read_log 193305'14123 (0'0) promote 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by osd.325.0:36 2014-08-11 18:42:40.789379 2014-08-13 17:38:02.750565 7fe9bae887a0 20 read_log 193305'14124 (193305'14123) modify 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.951337.0:1921 2014-08-11 18:42:40.470029 2014-08-13 17:38:02.750577 7fe9bae887a0 20 read_log 193305'14125 (0'0) promote 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by osd.325.0:50 2014-08-11 18:42:44.160651 2014-08-13 17:38:02.750599 7fe9bae887a0 20 read_log 193305'14126 (193305'14125) modify 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.951337.0:2937 2014-08-11 18:42:44.152798 2014-08-13 17:38:02.750611 7fe9bae887a0 20 read_log 193305'14127 (0'0) promote d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by osd.325.0:104 2014-08-11 18:42:56.364367 2014-08-13 17:38:02.750624 7fe9bae887a0 20 read_log 193305'14128 (193305'14127) modify d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.951337.0:6258 2014-08-11 18:42:56.356668 2014-08-13 17:38:02.750636 7fe9bae887a0 20 read_log 193305'14129 (0'0) promote b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by osd.325.0:110 2014-08-11 18:42:58.047160 2014-08-13 17:38:02.750648 7fe9bae887a0 20 read_log 193305'14130 (193305'14129) modify b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.951337.0:6724 2014-08-11 18:42:58.039624 2014-08-13 17:38:02.750660 7fe9bae887a0 20 read_log 193305'14131 (0'0) promote 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by osd.325.0:114 2014-08-11 18:42:58.417304 2014-08-13 17:38:02.750672 7fe9bae887a0 20 read_log 193305'14132 (193305'14131) modify 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.951337.0:6826 2014-08-11 18:42:58.408919 2014-08-13 17:38:02.750685 7fe9bae887a0 20 read_log 193305'14133 (0'0) promote 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by osd.325.0:126 2014-08-11 18:43:00.374239 2014-08-13 17:38:02.750698 7fe9bae887a0 20 read_log 193305'14134 (193305'14133) modify 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.951337.0:7369 2014-08-11 18:43:00.367151 2014-08-13 17:38:02.750710 7fe9bae887a0 20 read_log 193305'14135 (0'0) promote 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by osd.325.0:138 2014-08-11 18:43:04.626031 2014-08-13 17:38:02.750722 7fe9bae887a0 20 read_log 193305'14136 (193305'14135) modify 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.951337.0:8473 2014-08-11 18:43:04.617330 2014-08-13 17:38:02.750734 7fe9bae887a0 20 read_log 193305'14137 (0'0) promote 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by osd.325.0:146 2014-08-11 18:43:07.404195 2014-08-13 17:38:02.750748 7fe9bae887a0 20 read_log 193305'14138 (193305'14137) modify 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.951337.0:9213 2014-08-11 18:43:07.396514 2014-08-13 17:38:02.750760 7fe9bae887a0 20 read_log 193305'14139 (0'0) promote 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by osd.325.0:154 2014-08-11 18:43:08.882196 2014-08-13 17:38:02.750772 7fe9bae887a0 20 read_log 193305'14140 (193305'14139) modify 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.951337.0:9613 2014-08-11 18:43:08.872031 2014-08-13 17:38:02.750784 7fe9bae887a0 20 read_log 193305'14141 (0'0) promote cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by osd.325.0:158 2014-08-11 18:43:10.500576 2014-08-13 17:38:02.750796 7fe9bae887a0 20 read_log 193305'14142 (193305'14141) modify cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.951337.0:10057 2014-08-11 18:43:10.493514 2014-08-13 17:38:02.750808 7fe9bae887a0 20 read_log 193305'14143 (0'0) promote a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by osd.325.0:186 2014-08-11 18:43:16.619681 2014-08-13 17:38:02.750820 7fe9bae887a0 20 read_log 193305'14144 (193305'14143) modify a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.951337.0:11654 2014-08-11 18:43:16.611579 2014-08-13 17:38:02.750832 7fe9bae887a0 20 read_log 193305'14145 (0'0) promote e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by osd.325.0:204 2014-08-11 18:43:20.673946 2014-08-13 17:38:02.750851 7fe9bae887a0 20 read_log 193305'14146 (193305'14145) modify e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.951337.0:12742 2014-08-11 18:43:20.667152 2014-08-13 17:38:02.750864 7fe9bae887a0 20 read_log 193305'14147 (0'0) promote 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by osd.325.0:221 2014-08-11 18:43:23.995574 2014-08-13 17:38:02.750876 7fe9bae887a0 20 read_log 193305'14148 (193305'14147) modify 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.951337.0:13612 2014-08-11 18:43:23.984097 2014-08-13 17:38:02.750889 7fe9bae887a0 20 read_log 193305'14149 (0'0) promote 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by osd.325.0:225 2014-08-11 18:43:25.453120 2014-08-13 17:38:02.750902 7fe9bae887a0 20 read_log 193305'14150 (193305'14149) modify 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.951337.0:14020 2014-08-11 18:43:25.439836 2014-08-13 17:38:02.750914 7fe9bae887a0 20 read_log 193305'14151 (0'0) promote 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by osd.325.0:233 2014-08-11 18:43:26.668905 2014-08-13 17:38:02.750926 7fe9bae887a0 20 read_log 193305'14152 (193305'14151) modify 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.951337.0:14331 2014-08-11 18:43:26.660583 2014-08-13 17:38:02.750938 7fe9bae887a0 20 read_log 193305'14153 (0'0) promote 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by osd.325.0:256 2014-08-11 18:43:31.248568 2014-08-13 17:38:02.750949 7fe9bae887a0 20 read_log 193305'14154 (193305'14153) modify 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.951337.0:15665 2014-08-11 18:43:31.241310 2014-08-13 17:38:02.750962 7fe9bae887a0 20 read_log 193305'14155 (0'0) promote 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by osd.325.0:260 2014-08-11 18:43:31.765086 2014-08-13 17:38:02.750974 7fe9bae887a0 20 read_log 193305'14156 (193305'14155) modify 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.951337.0:15822 2014-08-11 18:43:31.754467 2014-08-13 17:38:02.750986 7fe9bae887a0 20 read_log 193305'14157 (0'0) promote 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by osd.325.0:264 2014-08-11 18:43:33.862421 2014-08-13 17:38:02.750998 7fe9bae887a0 20 read_log 193305'14158 (193305'14157) modify 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.951337.0:16403 2014-08-11 18:43:33.853089 2014-08-13 17:38:02.751010 7fe9bae887a0 20 read_log 193305'14159 (0'0) promote e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by osd.325.0:268 2014-08-11 18:43:34.496954 2014-08-13 17:38:02.751022 7fe9bae887a0 20 read_log 193305'14160 (193305'14159) modify e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.951337.0:16592 2014-08-11 18:43:34.487277 2014-08-13 17:38:02.751034 7fe9bae887a0 20 read_log 193305'14161 (0'0) promote 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by osd.325.0:272 2014-08-11 18:43:36.032652 2014-08-13 17:38:02.751046 7fe9bae887a0 20 read_log 193305'14162 (193305'14161) modify 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.951337.0:17021 2014-08-11 18:43:36.024623 2014-08-13 17:38:02.751058 7fe9bae887a0 20 read_log 193305'14163 (0'0) promote 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by osd.325.0:300 2014-08-11 18:43:44.528155 2014-08-13 17:38:02.751070 7fe9bae887a0 20 read_log 193305'14164 (193305'14163) modify 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.951337.0:19404 2014-08-11 18:43:44.520413 2014-08-13 17:38:02.751082 7fe9bae887a0 20 read_log 193305'14165 (0'0) promote dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by osd.325.0:304 2014-08-11 18:43:45.262872 2014-08-13 17:38:02.751101 7fe9bae887a0 20 read_log 193305'14166 (193305'14165) modify dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.951337.0:19616 2014-08-11 18:43:45.254685 2014-08-13 17:38:02.751113 7fe9bae887a0 20 read_log 193305'14167 (0'0) promote a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by osd.325.0:308 2014-08-11 18:43:45.730169 2014-08-13 17:38:02.751125 7fe9bae887a0 20 read_log 193305'14168 (193305'14167) modify a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.951337.0:19749 2014-08-11 18:43:45.721213 2014-08-13 17:38:02.751137 7fe9bae887a0 20 read_log 193305'14169 (0'0) promote 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by osd.325.0:314 2014-08-11 18:43:46.304030 2014-08-13 17:38:02.751149 7fe9bae887a0 20 read_log 193305'14170 (193305'14169) modify 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.951337.0:19915 2014-08-11 18:43:46.293920 2014-08-13 17:38:02.751166 7fe9bae887a0 20 read_log 193305'14171 (0'0) promote 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by osd.325.0:316 2014-08-11 18:43:47.054157 2014-08-13 17:38:02.751179 7fe9bae887a0 20 read_log 193305'14172 (193305'14171) modify 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.951337.0:20118 2014-08-11 18:43:47.045851 2014-08-13 17:38:02.751191 7fe9bae887a0 20 read_log 193305'14173 (0'0) promote ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by osd.325.0:320 2014-08-11 18:43:47.600773 2014-08-13 17:38:02.751204 7fe9bae887a0 20 read_log 193305'14174 (193305'14173) modify ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.951337.0:20255 2014-08-11 18:43:47.592872 2014-08-13 17:38:02.751216 7fe9bae887a0 20 read_log 193305'14175 (0'0) promote 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by osd.325.0:324 2014-08-11 18:43:47.863393 2014-08-13 17:38:02.751228 7fe9bae887a0 20 read_log 193305'14176 (193305'14175) modify 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.951337.0:20331 2014-08-11 18:43:47.854690 2014-08-13 17:38:02.751240 7fe9bae887a0 20 read_log 193305'14177 (0'0) promote 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by osd.325.0:340 2014-08-11 18:43:51.110074 2014-08-13 17:38:02.751252 7fe9bae887a0 20 read_log 193305'14178 (193305'14177) modify 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.951337.0:21250 2014-08-11 18:43:51.100374 2014-08-13 17:38:02.751264 7fe9bae887a0 20 read_log 193305'14179 (0'0) promote 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by osd.325.0:378 2014-08-11 18:43:58.191719 2014-08-13 17:38:02.751276 7fe9bae887a0 20 read_log 193305'14180 (193305'14179) modify 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.951337.0:23215 2014-08-11 18:43:58.183498 2014-08-13 17:38:02.751288 7fe9bae887a0 20 read_log 193305'14181 (0'0) promote 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by osd.325.0:392 2014-08-11 18:44:00.594004 2014-08-13 17:38:02.751300 7fe9bae887a0 20 read_log 193305'14182 (193305'14181) modify 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.951337.0:23880 2014-08-11 18:44:00.583948 2014-08-13 17:38:02.751314 7fe9bae887a0 20 read_log 193305'14183 (0'0) promote cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by osd.325.0:400 2014-08-11 18:44:01.533309 2014-08-13 17:38:02.751326 7fe9bae887a0 20 read_log 193305'14184 (193305'14183) modify cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.951337.0:24134 2014-08-11 18:44:01.522989 2014-08-13 17:38:02.751338 7fe9bae887a0 20 read_log 193305'14185 (0'0) promote be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by osd.325.0:426 2014-08-11 18:44:06.832000 2014-08-13 17:38:02.751360 7fe9bae887a0 20 read_log 193305'14186 (193305'14185) modify be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.951337.0:25591 2014-08-11 18:44:06.824258 2014-08-13 17:38:02.751373 7fe9bae887a0 20 read_log 193305'14187 (0'0) promote 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by osd.325.0:432 2014-08-11 18:44:09.426654 2014-08-13 17:38:02.751388 7fe9bae887a0 20 read_log 193305'14188 (193305'14187) modify 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.951337.0:26282 2014-08-11 18:44:09.416407 2014-08-13 17:38:02.751401 7fe9bae887a0 20 read_log 193305'14189 (0'0) promote 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by osd.325.0:440 2014-08-11 18:44:12.922734 2014-08-13 17:38:02.751431 7fe9bae887a0 20 read_log 193305'14190 (193305'14189) modify 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.951337.0:27232 2014-08-11 18:44:12.914438 2014-08-13 17:38:02.751446 7fe9bae887a0 20 read_log 193305'14191 (0'0) promote 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by osd.325.0:452 2014-08-11 18:44:18.340515 2014-08-13 17:38:02.751459 7fe9bae887a0 20 read_log 193305'14192 (193305'14191) modify 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.951337.0:28738 2014-08-11 18:44:18.331795 2014-08-13 17:38:02.751472 7fe9bae887a0 20 read_log 193305'14193 (0'0) promote f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by osd.325.0:460 2014-08-11 18:44:18.991635 2014-08-13 17:38:02.751484 7fe9bae887a0 20 read_log 193305'14194 (193305'14193) modify f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.951337.0:28902 2014-08-11 18:44:18.982974 2014-08-13 17:38:02.751496 7fe9bae887a0 20 read_log 193305'14195 (0'0) promote cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by osd.325.0:476 2014-08-11 18:44:20.746103 2014-08-13 17:38:02.751508 7fe9bae887a0 20 read_log 193305'14196 (193305'14195) modify cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.951337.0:29385 2014-08-11 18:44:20.737420 2014-08-13 17:38:02.751521 7fe9bae887a0 20 read_log 193305'14197 (193305'14124) modify 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.951337.0:34632 2014-08-11 18:44:35.103965 2014-08-13 17:38:02.751533 7fe9bae887a0 20 read_log 193305'14198 (193305'14126) modify 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.951337.0:35648 2014-08-11 18:44:35.689120 2014-08-13 17:38:02.751545 7fe9bae887a0 20 read_log 193305'14199 (193305'14128) modify d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.951337.0:38969 2014-08-11 18:44:37.754925 2014-08-13 17:38:02.751557 7fe9bae887a0 20 read_log 193305'14200 (193305'14130) modify b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.951337.0:39435 2014-08-11 18:44:38.005310 2014-08-13 17:38:02.751569 7fe9bae887a0 20 read_log 193305'14201 (193305'14132) modify 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.951337.0:39537 2014-08-11 18:44:38.063056 2014-08-13 17:38:02.751582 7fe9bae887a0 20 read_log 193305'14202 (193305'14134) modify 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.951337.0:40080 2014-08-11 18:44:38.278763 2014-08-13 17:38:02.751594 7fe9bae887a0 20 read_log 193305'14203 (193305'14136) modify 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.951337.0:41184 2014-08-11 18:44:38.660280 2014-08-13 17:38:02.751606 7fe9bae887a0 20 read_log 193305'14204 (193305'14138) modify 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.951337.0:41924 2014-08-11 18:44:38.927376 2014-08-13 17:38:02.751618 7fe9bae887a0 20 read_log 193305'14205 (193305'14140) modify 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.951337.0:42324 2014-08-11 18:44:39.189379 2014-08-13 17:38:02.751637 7fe9bae887a0 20 read_log 193305'14206 (193305'14142) modify cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.951337.0:42768 2014-08-11 18:44:39.373880 2014-08-13 17:38:02.751649 7fe9bae887a0 20 read_log 193305'14207 (193305'14144) modify a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.951337.0:44365 2014-08-11 18:44:39.863230 2014-08-13 17:38:02.751661 7fe9bae887a0 20 read_log 193305'14208 (193305'14146) modify e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.951337.0:45453 2014-08-11 18:44:40.205937 2014-08-13 17:38:02.751692 7fe9bae887a0 20 read_log 193305'14209 (193305'14148) modify 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.951337.0:46323 2014-08-11 18:44:40.452019 2014-08-13 17:38:02.751709 7fe9bae887a0 20 read_log 193305'14210 (193305'14150) modify 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.951337.0:46731 2014-08-11 18:44:40.573077 2014-08-13 17:38:02.751722 7fe9bae887a0 20 read_log 193305'14211 (193305'14152) modify 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.951337.0:47042 2014-08-11 18:44:40.659955 2014-08-13 17:38:02.751734 7fe9bae887a0 20 read_log 193305'14212 (193305'14154) modify 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.951337.0:48376 2014-08-11 18:44:41.037660 2014-08-13 17:38:02.751746 7fe9bae887a0 20 read_log 193305'14213 (193305'14156) modify 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.951337.0:48533 2014-08-11 18:44:41.080091 2014-08-13 17:38:02.751758 7fe9bae887a0 20 read_log 193305'14214 (193305'14158) modify 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.951337.0:49114 2014-08-11 18:44:41.257388 2014-08-13 17:38:02.751770 7fe9bae887a0 20 read_log 193305'14215 (193305'14160) modify e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.951337.0:49303 2014-08-11 18:44:41.324797 2014-08-13 17:38:02.751782 7fe9bae887a0 20 read_log 193305'14216 (193305'14162) modify 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.951337.0:49732 2014-08-11 18:44:41.452655 2014-08-13 17:38:02.751794 7fe9bae887a0 20 read_log 193305'14217 (193305'14164) modify 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.951337.0:52115 2014-08-11 18:44:42.137837 2014-08-13 17:38:02.751806 7fe9bae887a0 20 read_log 193305'14218 (193305'14166) modify dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.951337.0:52327 2014-08-11 18:44:42.203595 2014-08-13 17:38:02.751818 7fe9bae887a0 20 read_log 193305'14219 (193305'14168) modify a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.951337.0:52460 2014-08-11 18:44:42.239997 2014-08-13 17:38:02.751830 7fe9bae887a0 20 read_log 193305'14220 (193305'14170) modify 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.951337.0:52626 2014-08-11 18:44:42.287238 2014-08-13 17:38:02.751842 7fe9bae887a0 20 read_log 193305'14221 (193305'14172) modify 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.951337.0:52829 2014-08-11 18:44:42.353099 2014-08-13 17:38:02.751854 7fe9bae887a0 20 read_log 193305'14222 (193305'14174) modify ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.951337.0:52966 2014-08-11 18:44:42.392567 2014-08-13 17:38:02.751866 7fe9bae887a0 20 read_log 193305'14223 (193305'14176) modify 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.951337.0:53042 2014-08-11 18:44:42.413012 2014-08-13 17:38:02.751878 7fe9bae887a0 20 read_log 193305'14224 (193305'14178) modify 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.951337.0:53961 2014-08-11 18:44:42.684845 2014-08-13 17:38:02.751890 7fe9bae887a0 20 read_log 193305'14225 (193305'14180) modify 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.951337.0:55926 2014-08-11 18:44:43.270542 2014-08-13 17:38:02.751926 7fe9bae887a0 20 read_log 193305'14226 (193305'14182) modify 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.951337.0:56591 2014-08-11 18:44:43.463129 2014-08-13 17:38:02.751939 7fe9bae887a0 20 read_log 193305'14227 (193305'14184) modify cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.951337.0:56845 2014-08-11 18:44:43.550491 2014-08-13 17:38:02.751953 7fe9bae887a0 20 read_log 193305'14228 (193305'14186) modify be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.951337.0:58302 2014-08-11 18:44:44.002585 2014-08-13 17:38:02.751966 7fe9bae887a0 20 read_log 193305'14229 (193305'14188) modify 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.951337.0:58993 2014-08-11 18:44:44.204948 2014-08-13 17:38:02.751978 7fe9bae887a0 20 read_log 193305'14230 (193305'14190) modify 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.951337.0:59943 2014-08-11 18:44:44.472854 2014-08-13 17:38:02.751990 7fe9bae887a0 20 read_log 193305'14231 (193305'14192) modify 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.951337.0:61449 2014-08-11 18:44:44.924912 2014-08-13 17:38:02.752003 7fe9bae887a0 20 read_log 193305'14232 (193305'14194) modify f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.951337.0:61613 2014-08-11 18:44:44.977410 2014-08-13 17:38:02.752015 7fe9bae887a0 20 read_log 193305'14233 (193305'14196) modify cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.951337.0:62096 2014-08-11 18:44:45.124350 2014-08-13 17:38:02.752027 7fe9bae887a0 20 read_log 194215'14234 (192888'13947) clean 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by osd.325.0:868 2014-08-12 15:36:52.608185 2014-08-13 17:38:02.752039 7fe9bae887a0 20 read_log 194215'14235 (194215'14234) delete 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.2174595.0:10275 0.000000 2014-08-13 17:38:02.752050 7fe9bae887a0 20 read_log 194215'14236 (192834'13715) clean ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by osd.325.0:870 2014-08-12 15:36:52.620388 2014-08-13 17:38:02.752062 7fe9bae887a0 20 read_log 194215'14237 (194215'14236) delete ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.2174595.0:10277 0.000000 2014-08-13 17:38:02.752073 7fe9bae887a0 20 read_log 194215'14238 (192888'13899) clean e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by osd.325.0:872 2014-08-12 15:36:52.631030 2014-08-13 17:38:02.752085 7fe9bae887a0 20 read_log 194215'14239 (194215'14238) delete e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.2174595.0:10279 0.000000 2014-08-13 17:38:02.752096 7fe9bae887a0 20 read_log 194215'14240 (192888'13929) clean ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by osd.325.0:874 2014-08-12 15:36:52.642154 2014-08-13 17:38:02.752108 7fe9bae887a0 20 read_log 194215'14241 (194215'14240) delete ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.2174595.0:10281 0.000000 2014-08-13 17:38:02.752119 7fe9bae887a0 20 read_log 194215'14242 (192888'14097) clean afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by osd.325.0:876 2014-08-12 15:36:52.652742 2014-08-13 17:38:02.752131 7fe9bae887a0 20 read_log 194215'14243 (194215'14242) delete afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.2174595.0:10283 0.000000 2014-08-13 17:38:02.752142 7fe9bae887a0 20 read_log 194215'14244 (192888'14115) clean e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by osd.325.0:878 2014-08-12 15:36:52.662495 2014-08-13 17:38:02.752169 7fe9bae887a0 20 read_log 194215'14245 (194215'14244) delete e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.2174595.0:10285 0.000000 2014-08-13 17:38:02.752181 7fe9bae887a0 20 read_log 194215'14246 (193305'14200) clean b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by osd.325.0:880 2014-08-12 15:36:52.686621 2014-08-13 17:38:02.752194 7fe9bae887a0 20 read_log 194215'14247 (194215'14246) delete b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.2174595.0:10287 0.000000 2014-08-13 17:38:02.752205 7fe9bae887a0 20 read_log 194215'14248 (192888'14103) clean 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by osd.325.0:882 2014-08-12 15:36:52.698266 2014-08-13 17:38:02.752217 7fe9bae887a0 20 read_log 194215'14249 (194215'14248) delete 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.2174595.0:10289 0.000000 2014-08-13 17:38:02.752228 7fe9bae887a0 20 read_log 194215'14250 (192888'13949) clean 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by osd.325.0:884 2014-08-12 15:36:52.708144 2014-08-13 17:38:02.752241 7fe9bae887a0 20 read_log 194215'14251 (194215'14250) delete 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.2174595.0:10291 0.000000 2014-08-13 17:38:02.752252 7fe9bae887a0 20 read_log 194215'14252 (192834'13738) clean 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by osd.325.0:886 2014-08-12 15:36:52.718951 2014-08-13 17:38:02.752264 7fe9bae887a0 20 read_log 194215'14253 (194215'14252) delete 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.2174595.0:10293 0.000000 2014-08-13 17:38:02.752275 7fe9bae887a0 20 read_log 194215'14254 (192888'14076) clean 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by osd.325.0:888 2014-08-12 15:36:52.729964 2014-08-13 17:38:02.752287 7fe9bae887a0 20 read_log 194215'14255 (194215'14254) delete 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.2174595.0:10295 0.000000 2014-08-13 17:38:02.752298 7fe9bae887a0 20 read_log 194215'14256 (193305'14233) clean cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by osd.325.0:890 2014-08-12 15:36:52.739675 2014-08-13 17:38:02.752311 7fe9bae887a0 20 read_log 194215'14257 (194215'14256) delete cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.2174595.0:10297 0.000000 2014-08-13 17:38:02.752322 7fe9bae887a0 20 read_log 194215'14258 (192888'14108) clean 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by osd.325.0:892 2014-08-12 15:36:52.751097 2014-08-13 17:38:02.752334 7fe9bae887a0 20 read_log 194215'14259 (194215'14258) delete 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.2174595.0:10299 0.000000 2014-08-13 17:38:02.752344 7fe9bae887a0 20 read_log 194215'14260 (192888'14095) clean 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by osd.325.0:894 2014-08-12 15:36:52.763031 2014-08-13 17:38:02.752356 7fe9bae887a0 20 read_log 194215'14261 (194215'14260) delete 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.2174595.0:10301 0.000000 2014-08-13 17:38:02.752367 7fe9bae887a0 20 read_log 194215'14262 (193305'14229) clean 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by osd.325.0:896 2014-08-12 15:36:52.775679 2014-08-13 17:38:02.752379 7fe9bae887a0 20 read_log 194215'14263 (194215'14262) delete 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.2174595.0:10303 0.000000 2014-08-13 17:38:02.752390 7fe9bae887a0 20 read_log 194215'14264 (192888'13932) clean 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by osd.325.0:898 2014-08-12 15:36:52.787524 2014-08-13 17:38:02.752409 7fe9bae887a0 20 read_log 194215'14265 (194215'14264) delete 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.2174595.0:10305 0.000000 2014-08-13 17:38:02.752420 7fe9bae887a0 20 read_log 194215'14266 (192834'13633) clean c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:900 2014-08-12 15:36:52.803725 2014-08-13 17:38:02.752433 7fe9bae887a0 20 read_log 194215'14267 (194215'14266) delete c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.2174595.0:10307 0.000000 2014-08-13 17:38:02.752444 7fe9bae887a0 20 read_log 194215'14268 (192888'14104) clean b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by osd.325.0:902 2014-08-12 15:36:52.814725 2014-08-13 17:38:02.752458 7fe9bae887a0 20 read_log 194215'14269 (194215'14268) delete b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.2174595.0:10309 0.000000 2014-08-13 17:38:02.752469 7fe9bae887a0 20 read_log 194215'14270 (193305'14227) clean cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by osd.325.0:904 2014-08-12 15:36:52.826788 2014-08-13 17:38:02.752483 7fe9bae887a0 20 read_log 194215'14271 (194215'14270) delete cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.2174595.0:10311 0.000000 2014-08-13 17:38:02.752493 7fe9bae887a0 20 read_log 194215'14272 (192888'14096) clean b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by osd.325.0:906 2014-08-12 15:36:52.836933 2014-08-13 17:38:02.752506 7fe9bae887a0 20 read_log 194215'14273 (194215'14272) delete b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.2174595.0:10313 0.000000 2014-08-13 17:38:02.752517 7fe9bae887a0 20 read_log 194215'14274 (192834'13662) clean 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:908 2014-08-12 15:36:52.850085 2014-08-13 17:38:02.752529 7fe9bae887a0 20 read_log 194215'14275 (194215'14274) delete 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.2174595.0:10315 0.000000 2014-08-13 17:38:02.752540 7fe9bae887a0 20 read_log 194215'14276 (192888'13896) clean 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by osd.325.0:910 2014-08-12 15:36:52.859996 2014-08-13 17:38:02.752552 7fe9bae887a0 20 read_log 194215'14277 (194215'14276) delete 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.2174595.0:10317 0.000000 2014-08-13 17:38:02.752563 7fe9bae887a0 20 read_log 194215'14278 (192888'13910) clean a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by osd.325.0:912 2014-08-12 15:36:52.870755 2014-08-13 17:38:02.752575 7fe9bae887a0 20 read_log 194215'14279 (194215'14278) delete a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.2174595.0:10319 0.000000 2014-08-13 17:38:02.752585 7fe9bae887a0 20 read_log 194215'14280 (193305'14218) clean dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by osd.325.0:914 2014-08-12 15:36:52.882437 2014-08-13 17:38:02.752597 7fe9bae887a0 20 read_log 194215'14281 (194215'14280) delete dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.2174595.0:10321 0.000000 2014-08-13 17:38:02.752608 7fe9bae887a0 20 read_log 194215'14282 (192834'13666) clean 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:916 2014-08-12 15:36:52.896291 2014-08-13 17:38:02.752636 7fe9bae887a0 20 read_log 194215'14283 (194215'14282) delete 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.2174595.0:10323 0.000000 2014-08-13 17:38:02.752647 7fe9bae887a0 20 read_log 194215'14284 (192888'14092) clean 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by osd.325.0:918 2014-08-12 15:36:52.907716 2014-08-13 17:38:02.752667 7fe9bae887a0 20 read_log 194215'14285 (194215'14284) delete 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.2174595.0:10325 0.000000 2014-08-13 17:38:02.752678 7fe9bae887a0 20 read_log 194215'14286 (192888'13908) clean a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by osd.325.0:920 2014-08-12 15:36:52.918745 2014-08-13 17:38:02.752692 7fe9bae887a0 20 read_log 194215'14287 (194215'14286) delete a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.2174595.0:10327 0.000000 2014-08-13 17:38:02.752703 7fe9bae887a0 20 read_log 194215'14288 (193305'14232) clean f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by osd.325.0:922 2014-08-12 15:36:52.929382 2014-08-13 17:38:02.752715 7fe9bae887a0 20 read_log 194215'14289 (194215'14288) delete f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.2174595.0:10329 0.000000 2014-08-13 17:38:02.752726 7fe9bae887a0 20 read_log 194215'14290 (192834'13756) clean 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by osd.325.0:924 2014-08-12 15:36:52.940959 2014-08-13 17:38:02.752738 7fe9bae887a0 20 read_log 194215'14291 (194215'14290) delete 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.2174595.0:10331 0.000000 2014-08-13 17:38:02.752749 7fe9bae887a0 20 read_log 194215'14292 (192834'13696) clean 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:926 2014-08-12 15:36:52.954838 2014-08-13 17:38:02.752761 7fe9bae887a0 20 read_log 194215'14293 (194215'14292) delete 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.2174595.0:10333 0.000000 2014-08-13 17:38:02.752772 7fe9bae887a0 20 read_log 194215'14294 (192888'13894) clean ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by osd.325.0:928 2014-08-12 15:36:52.966175 2014-08-13 17:38:02.752788 7fe9bae887a0 20 read_log 194215'14295 (194215'14294) delete ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.2174595.0:10335 0.000000 2014-08-13 17:38:02.752800 7fe9bae887a0 20 read_log 194215'14296 (192834'13577) clean bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:930 2014-08-12 15:36:52.980118 2014-08-13 17:38:02.752812 7fe9bae887a0 20 read_log 194215'14297 (194215'14296) delete bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.2174595.0:10337 0.000000 2014-08-13 17:38:02.752822 7fe9bae887a0 20 read_log 194215'14298 (193305'14222) clean ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by osd.325.0:932 2014-08-12 15:36:52.990650 2014-08-13 17:38:02.752834 7fe9bae887a0 20 read_log 194215'14299 (194215'14298) delete ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.2174595.0:10339 0.000000 2014-08-13 17:38:02.752845 7fe9bae887a0 20 read_log 194215'14300 (192834'13582) clean 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:934 2014-08-12 15:36:53.004133 2014-08-13 17:38:02.752857 7fe9bae887a0 20 read_log 194215'14301 (194215'14300) delete 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.2174595.0:10341 0.000000 2014-08-13 17:38:02.752868 7fe9bae887a0 20 read_log 194215'14302 (192834'13565) clean 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by osd.325.0:936 2014-08-12 15:36:53.013874 2014-08-13 17:38:02.752880 7fe9bae887a0 20 read_log 194215'14303 (194215'14302) delete 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.2174595.0:10343 0.000000 2014-08-13 17:38:02.752891 7fe9bae887a0 20 read_log 194215'14304 (192888'13924) clean 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by osd.325.0:938 2014-08-12 15:36:53.022162 2014-08-13 17:38:02.752912 7fe9bae887a0 20 read_log 194215'14305 (194215'14304) delete 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.2174595.0:10345 0.000000 2014-08-13 17:38:02.752924 7fe9bae887a0 20 read_log 194215'14306 (192888'14098) clean f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by osd.325.0:940 2014-08-12 15:36:53.028542 2014-08-13 17:38:02.752936 7fe9bae887a0 20 read_log 194215'14307 (194215'14306) delete f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.2174595.0:10347 0.000000 2014-08-13 17:38:02.752947 7fe9bae887a0 20 read_log 194215'14308 (193305'14221) clean 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by osd.325.0:942 2014-08-12 15:36:53.035451 2014-08-13 17:38:02.752959 7fe9bae887a0 20 read_log 194215'14309 (194215'14308) delete 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.2174595.0:10349 0.000000 2014-08-13 17:38:02.752970 7fe9bae887a0 20 read_log 194215'14310 (193305'14210) clean 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by osd.325.0:944 2014-08-12 15:36:53.042360 2014-08-13 17:38:02.752982 7fe9bae887a0 20 read_log 194215'14311 (194215'14310) delete 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.2174595.0:10351 0.000000 2014-08-13 17:38:02.752993 7fe9bae887a0 20 read_log 194215'14312 (192834'13037) clean 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by osd.325.0:946 2014-08-12 15:36:53.048592 2014-08-13 17:38:02.753005 7fe9bae887a0 20 read_log 194215'14313 (194215'14312) delete 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.2174595.0:10353 0.000000 2014-08-13 17:38:02.753016 7fe9bae887a0 20 read_log 194215'14314 (193305'14211) clean 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by osd.325.0:948 2014-08-12 15:36:53.055461 2014-08-13 17:38:02.753028 7fe9bae887a0 20 read_log 194215'14315 (194215'14314) delete 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.2174595.0:10355 0.000000 2014-08-13 17:38:02.753039 7fe9bae887a0 20 read_log 194215'14316 (192888'14088) clean ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by osd.325.0:950 2014-08-12 15:36:53.062243 2014-08-13 17:38:02.753053 7fe9bae887a0 20 read_log 194215'14317 (194215'14316) delete ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.2174595.0:10357 0.000000 2014-08-13 17:38:02.753064 7fe9bae887a0 20 read_log 194215'14318 (192888'13919) clean 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by osd.325.0:952 2014-08-12 15:36:53.069544 2014-08-13 17:38:02.753076 7fe9bae887a0 20 read_log 194215'14319 (194215'14318) delete 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.2174595.0:10359 0.000000 2014-08-13 17:38:02.753087 7fe9bae887a0 20 read_log 194215'14320 (193305'14225) clean 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by osd.325.0:954 2014-08-12 15:36:53.077451 2014-08-13 17:38:02.753099 7fe9bae887a0 20 read_log 194215'14321 (194215'14320) delete 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.2174595.0:10361 0.000000 2014-08-13 17:38:02.753110 7fe9bae887a0 20 read_log 194215'14322 (193305'14203) clean 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by osd.325.0:956 2014-08-12 15:36:53.104038 2014-08-13 17:38:02.753122 7fe9bae887a0 20 read_log 194215'14323 (194215'14322) delete 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.2174595.0:10363 0.000000 2014-08-13 17:38:02.753133 7fe9bae887a0 20 read_log 194215'14324 (192834'13607) clean 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by osd.325.0:958 2014-08-12 15:36:53.112000 2014-08-13 17:38:02.753152 7fe9bae887a0 20 read_log 194215'14325 (194215'14324) delete 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.2174595.0:10365 0.000000 2014-08-13 17:38:02.753171 7fe9bae887a0 20 read_log 194215'14326 (192888'14100) clean 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by osd.325.0:960 2014-08-12 15:36:53.119126 2014-08-13 17:38:02.753184 7fe9bae887a0 20 read_log 194215'14327 (194215'14326) delete 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.2174595.0:10367 0.000000 2014-08-13 17:38:02.753195 7fe9bae887a0 20 read_log 194215'14328 (193305'14217) clean 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by osd.325.0:962 2014-08-12 15:36:53.127022 2014-08-13 17:38:02.753207 7fe9bae887a0 20 read_log 194215'14329 (194215'14328) delete 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.2174595.0:10369 0.000000 2014-08-13 17:38:02.753220 7fe9bae887a0 20 read_log 194215'14330 (192834'13729) clean 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by osd.325.0:964 2014-08-12 15:36:53.134759 2014-08-13 17:38:02.753233 7fe9bae887a0 20 read_log 194215'14331 (194215'14330) delete 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.2174595.0:10371 0.000000 2014-08-13 17:38:02.753244 7fe9bae887a0 20 read_log 194215'14332 (192834'13727) clean c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by osd.325.0:966 2014-08-12 15:36:53.144254 2014-08-13 17:38:02.753256 7fe9bae887a0 20 read_log 194215'14333 (194215'14332) delete c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.2174595.0:10373 0.000000 2014-08-13 17:38:02.753267 7fe9bae887a0 20 read_log 194215'14334 (193305'14220) clean 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by osd.325.0:968 2014-08-12 15:36:53.168536 2014-08-13 17:38:02.753279 7fe9bae887a0 20 read_log 194215'14335 (194215'14334) delete 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.2174595.0:10375 0.000000 2014-08-13 17:38:02.753289 7fe9bae887a0 20 read_log 194215'14336 (192834'13038) clean 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by osd.325.0:970 2014-08-12 15:36:53.180426 2014-08-13 17:38:02.753302 7fe9bae887a0 20 read_log 194215'14337 (194215'14336) delete 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.2174595.0:10377 0.000000 2014-08-13 17:38:02.753313 7fe9bae887a0 20 read_log 194215'14338 (192834'13096) clean d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by osd.325.0:972 2014-08-12 15:36:53.190656 2014-08-13 17:38:02.753325 7fe9bae887a0 20 read_log 194215'14339 (194215'14338) delete d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.2174595.0:10379 0.000000 2014-08-13 17:38:02.753336 7fe9bae887a0 20 read_log 194215'14340 (193305'14209) clean 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by osd.325.0:974 2014-08-12 15:36:53.201363 2014-08-13 17:38:02.753349 7fe9bae887a0 20 read_log 194215'14341 (194215'14340) delete 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.2174595.0:10381 0.000000 2014-08-13 17:38:02.753360 7fe9bae887a0 20 read_log 194215'14342 (192888'14077) clean 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by osd.325.0:976 2014-08-12 15:36:53.212542 2014-08-13 17:38:02.753372 7fe9bae887a0 20 read_log 194215'14343 (194215'14342) delete 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.2174595.0:10383 0.000000 2014-08-13 17:38:02.753382 7fe9bae887a0 20 read_log 194215'14344 (192888'14120) clean 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by osd.325.0:978 2014-08-12 15:36:53.222984 2014-08-13 17:38:02.753401 7fe9bae887a0 20 read_log 194215'14345 (194215'14344) delete 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.2174595.0:10385 0.000000 2014-08-13 17:38:02.753412 7fe9bae887a0 20 read_log 194215'14346 (193305'14199) clean d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by osd.325.0:980 2014-08-12 15:36:53.234248 2014-08-13 17:38:02.753425 7fe9bae887a0 20 read_log 194215'14347 (194215'14346) delete d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.2174595.0:10387 0.000000 2014-08-13 17:38:02.753435 7fe9bae887a0 20 read_log 194215'14348 (193305'14204) clean 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by osd.325.0:982 2014-08-12 15:36:53.245829 2014-08-13 17:38:02.753447 7fe9bae887a0 20 read_log 194215'14349 (194215'14348) delete 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.2174595.0:10389 0.000000 2014-08-13 17:38:02.753458 7fe9bae887a0 20 read_log 194215'14350 (192888'14111) clean a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by osd.325.0:984 2014-08-12 15:36:53.256925 2014-08-13 17:38:02.753470 7fe9bae887a0 20 read_log 194215'14351 (194215'14350) delete a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.2174595.0:10391 0.000000 2014-08-13 17:38:02.753480 7fe9bae887a0 20 read_log 194215'14352 (193305'14228) clean be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by osd.325.0:986 2014-08-12 15:36:53.268819 2014-08-13 17:38:02.753492 7fe9bae887a0 20 read_log 194215'14353 (194215'14352) delete be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.2174595.0:10393 0.000000 2014-08-13 17:38:02.753503 7fe9bae887a0 20 read_log 194215'14354 (192834'13097) clean ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by osd.325.0:988 2014-08-12 15:36:53.279323 2014-08-13 17:38:02.753515 7fe9bae887a0 20 read_log 194215'14355 (194215'14354) delete ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.2174595.0:10395 0.000000 2014-08-13 17:38:02.753525 7fe9bae887a0 20 read_log 194215'14356 (193305'14226) clean 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by osd.325.0:990 2014-08-12 15:36:53.291128 2014-08-13 17:38:02.753537 7fe9bae887a0 20 read_log 194215'14357 (194215'14356) delete 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.2174595.0:10397 0.000000 2014-08-13 17:38:02.753549 7fe9bae887a0 20 read_log 194215'14358 (192888'14087) clean 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by osd.325.0:992 2014-08-12 15:36:53.300910 2014-08-13 17:38:02.753581 7fe9bae887a0 20 read_log 194215'14359 (194215'14358) delete 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.2174595.0:10399 0.000000 2014-08-13 17:38:02.753592 7fe9bae887a0 20 read_log 194215'14360 (192834'13012) clean 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by osd.325.0:994 2014-08-12 15:36:53.311433 2014-08-13 17:38:02.753607 7fe9bae887a0 20 read_log 194215'14361 (194215'14360) delete 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.2174595.0:10401 0.000000 2014-08-13 17:38:02.753618 7fe9bae887a0 20 read_log 194215'14362 (192888'14081) clean 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by osd.325.0:996 2014-08-12 15:36:53.321308 2014-08-13 17:38:02.753630 7fe9bae887a0 20 read_log 194215'14363 (194215'14362) delete 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.2174595.0:10403 0.000000 2014-08-13 17:38:02.753641 7fe9bae887a0 20 read_log 194215'14364 (192888'13918) clean 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by osd.325.0:998 2014-08-12 15:36:53.331494 2014-08-13 17:38:02.753681 7fe9bae887a0 20 read_log 194215'14365 (194215'14364) delete 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.2174595.0:10405 0.000000 2014-08-13 17:38:02.753693 7fe9bae887a0 20 read_log 194215'14366 (193305'14207) clean a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by osd.325.0:1000 2014-08-12 15:36:53.343154 2014-08-13 17:38:02.753706 7fe9bae887a0 20 read_log 194215'14367 (194215'14366) delete a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.2174595.0:10407 0.000000 2014-08-13 17:38:02.753717 7fe9bae887a0 20 read_log 194215'14368 (193305'14205) clean 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by osd.325.0:1002 2014-08-12 15:36:53.355015 2014-08-13 17:38:02.753729 7fe9bae887a0 20 read_log 194215'14369 (194215'14368) delete 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.2174595.0:10409 0.000000 2014-08-13 17:38:02.753739 7fe9bae887a0 20 read_log 194215'14370 (192888'13934) clean 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by osd.325.0:1004 2014-08-12 15:36:53.365950 2014-08-13 17:38:02.753751 7fe9bae887a0 20 read_log 194215'14371 (194215'14370) delete 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.2174595.0:10411 0.000000 2014-08-13 17:38:02.753762 7fe9bae887a0 20 read_log 194215'14372 (192888'13902) clean 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by osd.325.0:1006 2014-08-12 15:36:53.376145 2014-08-13 17:38:02.753774 7fe9bae887a0 20 read_log 194215'14373 (194215'14372) delete 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.2174595.0:10413 0.000000 2014-08-13 17:38:02.753784 7fe9bae887a0 20 read_log 194215'14374 (193305'14212) clean 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by osd.325.0:1008 2014-08-12 15:36:53.387257 2014-08-13 17:38:02.753796 7fe9bae887a0 20 read_log 194215'14375 (194215'14374) delete 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.2174595.0:10415 0.000000 2014-08-13 17:38:02.753822 7fe9bae887a0 20 read_log 194215'14376 (193305'14224) clean 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by osd.325.0:1010 2014-08-12 15:36:53.409891 2014-08-13 17:38:02.753838 7fe9bae887a0 20 read_log 194215'14377 (194215'14376) delete 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.2174595.0:10417 0.000000 2014-08-13 17:38:02.753849 7fe9bae887a0 20 read_log 194215'14378 (192834'13569) clean bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by osd.325.0:1012 2014-08-12 15:36:53.439714 2014-08-13 17:38:02.753861 7fe9bae887a0 20 read_log 194215'14379 (194215'14378) delete bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.2174595.0:10419 0.000000 2014-08-13 17:38:02.753872 7fe9bae887a0 20 read_log 194215'14380 (193305'14214) clean 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by osd.325.0:1014 2014-08-12 15:36:53.452527 2014-08-13 17:38:02.753884 7fe9bae887a0 20 read_log 194215'14381 (194215'14380) delete 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.2174595.0:10421 0.000000 2014-08-13 17:38:02.753895 7fe9bae887a0 20 read_log 194215'14382 (193305'14213) clean 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by osd.325.0:1016 2014-08-12 15:36:53.463560 2014-08-13 17:38:02.753907 7fe9bae887a0 20 read_log 194215'14383 (194215'14382) delete 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.2174595.0:10423 0.000000 2014-08-13 17:38:02.753917 7fe9bae887a0 20 read_log 194215'14384 (192834'13615) clean 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by osd.325.0:1018 2014-08-12 15:36:53.474314 2014-08-13 17:38:02.753937 7fe9bae887a0 20 read_log 194215'14385 (194215'14384) delete 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.2174595.0:10425 0.000000 2014-08-13 17:38:02.753949 7fe9bae887a0 20 read_log 194215'14386 (192888'14116) clean e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by osd.325.0:1020 2014-08-12 15:36:53.484462 2014-08-13 17:38:02.753961 7fe9bae887a0 20 read_log 194215'14387 (194215'14386) delete e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.2174595.0:10427 0.000000 2014-08-13 17:38:02.753973 7fe9bae887a0 20 read_log 194215'14388 (192834'13626) clean e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by osd.325.0:1022 2014-08-12 15:36:53.495000 2014-08-13 17:38:02.753985 7fe9bae887a0 20 read_log 194215'14389 (194215'14388) delete e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.2174595.0:10429 0.000000 2014-08-13 17:38:02.753996 7fe9bae887a0 20 read_log 194215'14390 (193305'14198) clean 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by osd.325.0:1024 2014-08-12 15:36:53.507659 2014-08-13 17:38:02.754008 7fe9bae887a0 20 read_log 194215'14391 (194215'14390) delete 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.2174595.0:10431 0.000000 2014-08-13 17:38:02.754019 7fe9bae887a0 20 read_log 194215'14392 (192834'13605) clean eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by osd.325.0:1026 2014-08-12 15:36:53.519733 2014-08-13 17:38:02.754031 7fe9bae887a0 20 read_log 194215'14393 (194215'14392) delete eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.2174595.0:10433 0.000000 2014-08-13 17:38:02.754041 7fe9bae887a0 20 read_log 194215'14394 (192834'13603) clean 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by osd.325.0:1028 2014-08-12 15:36:53.532163 2014-08-13 17:38:02.754053 7fe9bae887a0 20 read_log 194215'14395 (194215'14394) delete 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.2174595.0:10435 0.000000 2014-08-13 17:38:02.754064 7fe9bae887a0 20 read_log 194215'14396 (192888'13940) clean 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by osd.325.0:1030 2014-08-12 15:36:53.543466 2014-08-13 17:38:02.754076 7fe9bae887a0 20 read_log 194215'14397 (194215'14396) delete 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.2174595.0:10437 0.000000 2014-08-13 17:38:02.754086 7fe9bae887a0 20 read_log 194215'14398 (192834'13568) clean e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by osd.325.0:1032 2014-08-12 15:36:53.553550 2014-08-13 17:38:02.754101 7fe9bae887a0 20 read_log 194215'14399 (194215'14398) delete e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.2174595.0:10439 0.000000 2014-08-13 17:38:02.754111 7fe9bae887a0 20 read_log 194215'14400 (192834'13048) clean c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by osd.325.0:1034 2014-08-12 15:36:53.563805 2014-08-13 17:38:02.754124 7fe9bae887a0 20 read_log 194215'14401 (194215'14400) delete c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.2174595.0:10441 0.000000 2014-08-13 17:38:02.754134 7fe9bae887a0 20 read_log 194215'14402 (192888'14107) clean b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by osd.325.0:1036 2014-08-12 15:36:53.574584 2014-08-13 17:38:02.754146 7fe9bae887a0 20 read_log 194215'14403 (194215'14402) delete b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.2174595.0:10443 0.000000 2014-08-13 17:38:02.754161 7fe9bae887a0 20 read_log 194215'14404 (192834'13609) clean d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by osd.325.0:1038 2014-08-12 15:36:53.584094 2014-08-13 17:38:02.754181 7fe9bae887a0 20 read_log 194215'14405 (194215'14404) delete d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.2174595.0:10445 0.000000 2014-08-13 17:38:02.754192 7fe9bae887a0 20 read_log 194215'14406 (193305'14201) clean 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by osd.325.0:1040 2014-08-12 15:36:53.594813 2014-08-13 17:38:02.754204 7fe9bae887a0 20 read_log 194215'14407 (194215'14406) delete 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.2174595.0:10447 0.000000 2014-08-13 17:38:02.754214 7fe9bae887a0 20 read_log 194215'14408 (192834'13676) clean be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by osd.325.0:1042 2014-08-12 15:36:53.604998 2014-08-13 17:38:02.754226 7fe9bae887a0 20 read_log 194215'14409 (194215'14408) delete be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.2174595.0:10449 0.000000 2014-08-13 17:38:02.754237 7fe9bae887a0 20 read_log 194215'14410 (192888'14090) clean 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by osd.325.0:1044 2014-08-12 15:36:53.616182 2014-08-13 17:38:02.754249 7fe9bae887a0 20 read_log 194215'14411 (194215'14410) delete 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.2174595.0:10451 0.000000 2014-08-13 17:38:02.754260 7fe9bae887a0 20 read_log 194215'14412 (192888'14099) clean b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by osd.325.0:1046 2014-08-12 15:36:53.627253 2014-08-13 17:38:02.754276 7fe9bae887a0 20 read_log 194215'14413 (194215'14412) delete b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.2174595.0:10453 0.000000 2014-08-13 17:38:02.754287 7fe9bae887a0 20 read_log 194215'14414 (192888'14082) clean d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by osd.325.0:1048 2014-08-12 15:36:53.637033 2014-08-13 17:38:02.754300 7fe9bae887a0 20 read_log 194215'14415 (194215'14414) delete d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.2174595.0:10455 0.000000 2014-08-13 17:38:02.754311 7fe9bae887a0 20 read_log 194215'14416 (192888'13928) clean 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by osd.325.0:1050 2014-08-12 15:36:53.646985 2014-08-13 17:38:02.754323 7fe9bae887a0 20 read_log 194215'14417 (194215'14416) delete 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.2174595.0:10457 0.000000 2014-08-13 17:38:02.754333 7fe9bae887a0 20 read_log 194215'14418 (193305'14215) clean e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by osd.325.0:1052 2014-08-12 15:36:53.657970 2014-08-13 17:38:02.754346 7fe9bae887a0 20 read_log 194215'14419 (194215'14418) delete e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.2174595.0:10459 0.000000 2014-08-13 17:38:02.754356 7fe9bae887a0 20 read_log 194215'14420 (192888'14094) clean 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by osd.325.0:1054 2014-08-12 15:36:53.669952 2014-08-13 17:38:02.754369 7fe9bae887a0 20 read_log 194215'14421 (194215'14420) delete 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.2174595.0:10461 0.000000 2014-08-13 17:38:02.754380 7fe9bae887a0 20 read_log 194215'14422 (192834'12976) clean 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by osd.325.0:1056 2014-08-12 15:36:53.681489 2014-08-13 17:38:02.754392 7fe9bae887a0 20 read_log 194215'14423 (194215'14422) delete 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.2174595.0:10463 0.000000 2014-08-13 17:38:02.754403 7fe9bae887a0 20 read_log 194215'14424 (192888'14066) clean 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by osd.325.0:1058 2014-08-12 15:36:53.693118 2014-08-13 17:38:02.754427 7fe9bae887a0 20 read_log 194215'14425 (194215'14424) delete 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.2174595.0:10465 0.000000 2014-08-13 17:38:02.754438 7fe9bae887a0 20 read_log 194215'14426 (192834'13656) clean 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by osd.325.0:1060 2014-08-12 15:36:53.703211 2014-08-13 17:38:02.754451 7fe9bae887a0 20 read_log 194215'14427 (194215'14426) delete 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.2174595.0:10467 0.000000 2014-08-13 17:38:02.754462 7fe9bae887a0 20 read_log 194215'14428 (193305'14206) clean cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by osd.325.0:1062 2014-08-12 15:36:53.713224 2014-08-13 17:38:02.754474 7fe9bae887a0 20 read_log 194215'14429 (194215'14428) delete cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.2174595.0:10469 0.000000 2014-08-13 17:38:02.754484 7fe9bae887a0 20 read_log 194215'14430 (192888'14072) clean 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by osd.325.0:1064 2014-08-12 15:36:53.723979 2014-08-13 17:38:02.754496 7fe9bae887a0 20 read_log 194215'14431 (194215'14430) delete 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.2174595.0:10471 0.000000 2014-08-13 17:38:02.754507 7fe9bae887a0 20 read_log 194215'14432 (192888'14110) clean 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by osd.325.0:1066 2014-08-12 15:36:53.736485 2014-08-13 17:38:02.754519 7fe9bae887a0 20 read_log 194215'14433 (194215'14432) delete 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.2174595.0:10473 0.000000 2014-08-13 17:38:02.754529 7fe9bae887a0 20 read_log 194215'14434 (192888'14079) clean 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by osd.325.0:1068 2014-08-12 15:36:53.748142 2014-08-13 17:38:02.754541 7fe9bae887a0 20 read_log 194215'14435 (194215'14434) delete 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.2174595.0:10475 0.000000 2014-08-13 17:38:02.754552 7fe9bae887a0 20 read_log 194215'14436 (192888'13936) clean fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by osd.325.0:1070 2014-08-12 15:36:53.759652 2014-08-13 17:38:02.754564 7fe9bae887a0 20 read_log 194215'14437 (194215'14436) delete fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.2174595.0:10477 0.000000 2014-08-13 17:38:02.754574 7fe9bae887a0 20 read_log 194215'14438 (192888'13930) clean d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by osd.325.0:1072 2014-08-12 15:36:53.772580 2014-08-13 17:38:02.754586 7fe9bae887a0 20 read_log 194215'14439 (194215'14438) delete d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.2174595.0:10479 0.000000 2014-08-13 17:38:02.754597 7fe9bae887a0 20 read_log 194215'14440 (192888'14113) clean 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by osd.325.0:1074 2014-08-12 15:36:53.784638 2014-08-13 17:38:02.754609 7fe9bae887a0 20 read_log 194215'14441 (194215'14440) delete 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.2174595.0:10481 0.000000 2014-08-13 17:38:02.754619 7fe9bae887a0 20 read_log 194215'14442 (192888'13906) clean 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by osd.325.0:1076 2014-08-12 15:36:53.795750 2014-08-13 17:38:02.754631 7fe9bae887a0 20 read_log 194215'14443 (194215'14442) delete 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.2174595.0:10483 0.000000 2014-08-13 17:38:02.754642 7fe9bae887a0 20 read_log 194215'14444 (192888'14114) clean 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by osd.325.0:1078 2014-08-12 15:36:53.805949 2014-08-13 17:38:02.754660 7fe9bae887a0 20 read_log 194215'14445 (194215'14444) delete 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.2174595.0:10485 0.000000 2014-08-13 17:38:02.754671 7fe9bae887a0 20 read_log 194215'14446 (192888'14073) clean d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by osd.325.0:1080 2014-08-12 15:36:53.817546 2014-08-13 17:38:02.754683 7fe9bae887a0 20 read_log 194215'14447 (194215'14446) delete d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.2174595.0:10487 0.000000 2014-08-13 17:38:02.754694 7fe9bae887a0 20 read_log 194215'14448 (192834'13046) clean e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by osd.325.0:1082 2014-08-12 15:36:53.828286 2014-08-13 17:38:02.754706 7fe9bae887a0 20 read_log 194215'14449 (194215'14448) delete e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.2174595.0:10489 0.000000 2014-08-13 17:38:02.754717 7fe9bae887a0 20 read_log 194215'14450 (193305'14208) clean e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by osd.325.0:1084 2014-08-12 15:36:53.841979 2014-08-13 17:38:02.754729 7fe9bae887a0 20 read_log 194215'14451 (194215'14450) delete e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.2174595.0:10491 0.000000 2014-08-13 17:38:02.754740 7fe9bae887a0 20 read_log 194215'14452 (192834'13725) clean af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by osd.325.0:1086 2014-08-12 15:36:53.854139 2014-08-13 17:38:02.754752 7fe9bae887a0 20 read_log 194215'14453 (194215'14452) delete af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.2174595.0:10493 0.000000 2014-08-13 17:38:02.754763 7fe9bae887a0 20 read_log 194215'14454 (192834'13617) clean 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by osd.325.0:1088 2014-08-12 15:36:53.864813 2014-08-13 17:38:02.754775 7fe9bae887a0 20 read_log 194215'14455 (194215'14454) delete 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.2174595.0:10495 0.000000 2014-08-13 17:38:02.754785 7fe9bae887a0 20 read_log 194215'14456 (192834'13602) clean 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by osd.325.0:1090 2014-08-12 15:36:53.875805 2014-08-13 17:38:02.754797 7fe9bae887a0 20 read_log 194215'14457 (194215'14456) delete 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.2174595.0:10497 0.000000 2014-08-13 17:38:02.754808 7fe9bae887a0 20 read_log 194215'14458 (192888'14084) clean 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by osd.325.0:1092 2014-08-12 15:36:53.886384 2014-08-13 17:38:02.754820 7fe9bae887a0 20 read_log 194215'14459 (194215'14458) delete 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.2174595.0:10499 0.000000 2014-08-13 17:38:02.754830 7fe9bae887a0 20 read_log 194215'14460 (192888'13948) clean aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by osd.325.0:1094 2014-08-12 15:36:53.897558 2014-08-13 17:38:02.754842 7fe9bae887a0 20 read_log 194215'14461 (194215'14460) delete aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.2174595.0:10501 0.000000 2014-08-13 17:38:02.754854 7fe9bae887a0 20 read_log 194215'14462 (193305'14197) clean 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by osd.325.0:1096 2014-08-12 15:36:53.909458 2014-08-13 17:38:02.754866 7fe9bae887a0 20 read_log 194215'14463 (194215'14462) delete 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.2174595.0:10503 0.000000 2014-08-13 17:38:02.754876 7fe9bae887a0 20 read_log 194215'14464 (193305'14216) clean 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by osd.325.0:1098 2014-08-12 15:36:53.920040 2014-08-13 17:38:02.754913 7fe9bae887a0 20 read_log 194215'14465 (194215'14464) delete 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.2174595.0:10505 0.000000 2014-08-13 17:38:02.754926 7fe9bae887a0 20 read_log 194215'14466 (192888'13935) clean 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by osd.325.0:1100 2014-08-12 15:36:53.931253 2014-08-13 17:38:02.754938 7fe9bae887a0 20 read_log 194215'14467 (194215'14466) delete 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.2174595.0:10507 0.000000 2014-08-13 17:38:02.754948 7fe9bae887a0 20 read_log 194215'14468 (192888'13903) clean f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by osd.325.0:1102 2014-08-12 15:36:53.941995 2014-08-13 17:38:02.754979 7fe9bae887a0 20 read_log 194215'14469 (194215'14468) delete f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.2174595.0:10509 0.000000 2014-08-13 17:38:02.754991 7fe9bae887a0 20 read_log 194215'14470 (193305'14231) clean 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by osd.325.0:1104 2014-08-12 15:36:53.953755 2014-08-13 17:38:02.755003 7fe9bae887a0 20 read_log 194215'14471 (194215'14470) delete 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.2174595.0:10511 0.000000 2014-08-13 17:38:02.755014 7fe9bae887a0 20 read_log 194215'14472 (193305'14219) clean a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by osd.325.0:1106 2014-08-12 15:36:53.965170 2014-08-13 17:38:02.755026 7fe9bae887a0 20 read_log 194215'14473 (194215'14472) delete a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.2174595.0:10513 0.000000 2014-08-13 17:38:02.755036 7fe9bae887a0 20 read_log 194215'14474 (192834'13739) clean 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by osd.325.0:1108 2014-08-12 15:36:53.976881 2014-08-13 17:38:02.755048 7fe9bae887a0 20 read_log 194215'14475 (194215'14474) delete 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.2174595.0:10515 0.000000 2014-08-13 17:38:02.755059 7fe9bae887a0 20 read_log 194215'14476 (192834'13634) clean 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by osd.325.0:1110 2014-08-12 15:36:53.988378 2014-08-13 17:38:02.755071 7fe9bae887a0 20 read_log 194215'14477 (194215'14476) delete 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.2174595.0:10517 0.000000 2014-08-13 17:38:02.755082 7fe9bae887a0 20 read_log 194215'14478 (192888'13945) clean da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by osd.325.0:1112 2014-08-12 15:36:53.998819 2014-08-13 17:38:02.755095 7fe9bae887a0 20 read_log 194215'14479 (194215'14478) delete da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.2174595.0:10519 0.000000 2014-08-13 17:38:02.755105 7fe9bae887a0 20 read_log 194215'14480 (192834'13653) clean b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by osd.325.0:1114 2014-08-12 15:36:54.009741 2014-08-13 17:38:02.755118 7fe9bae887a0 20 read_log 194215'14481 (194215'14480) delete b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.2174595.0:10521 0.000000 2014-08-13 17:38:02.755128 7fe9bae887a0 20 read_log 194215'14482 (192888'14067) clean aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by osd.325.0:1116 2014-08-12 15:36:54.021792 2014-08-13 17:38:02.755141 7fe9bae887a0 20 read_log 194215'14483 (194215'14482) delete aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.2174595.0:10523 0.000000 2014-08-13 17:38:02.755151 7fe9bae887a0 20 read_log 194215'14484 (192834'13113) clean 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by osd.325.0:1118 2014-08-12 15:36:54.032752 2014-08-13 17:38:02.755176 7fe9bae887a0 20 read_log 194215'14485 (194215'14484) delete 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.2174595.0:10525 0.000000 2014-08-13 17:38:02.755187 7fe9bae887a0 20 read_log 194215'14486 (193305'14223) clean 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by osd.325.0:1120 2014-08-12 15:36:54.045219 2014-08-13 17:38:02.755200 7fe9bae887a0 20 read_log 194215'14487 (194215'14486) delete 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.2174595.0:10527 0.000000 2014-08-13 17:38:02.755210 7fe9bae887a0 20 read_log 194215'14488 (193305'14202) clean 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by osd.325.0:1122 2014-08-12 15:36:54.056698 2014-08-13 17:38:02.755222 7fe9bae887a0 20 read_log 194215'14489 (194215'14488) delete 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.2174595.0:10529 0.000000 2014-08-13 17:38:02.755233 7fe9bae887a0 20 read_log 194215'14490 (192888'14093) clean dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by osd.325.0:1124 2014-08-12 15:36:54.071337 2014-08-13 17:38:02.755247 7fe9bae887a0 20 read_log 194215'14491 (194215'14490) delete dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.2174595.0:10531 0.000000 2014-08-13 17:38:02.755257 7fe9bae887a0 20 read_log 194215'14492 (192834'13597) clean 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by osd.325.0:1126 2014-08-12 15:36:54.083425 2014-08-13 17:38:02.755269 7fe9bae887a0 20 read_log 194215'14493 (194215'14492) delete 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.2174595.0:10533 0.000000 2014-08-13 17:38:02.755280 7fe9bae887a0 20 read_log 194215'14494 (192888'13950) clean aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by osd.325.0:1128 2014-08-12 15:36:54.093915 2014-08-13 17:38:02.755292 7fe9bae887a0 20 read_log 194215'14495 (194215'14494) delete aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.2174595.0:10535 0.000000 2014-08-13 17:38:02.755303 7fe9bae887a0 20 read_log 194215'14496 (192834'13591) clean acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by osd.325.0:1130 2014-08-12 15:36:54.109229 2014-08-13 17:38:02.755315 7fe9bae887a0 20 read_log 194215'14497 (194215'14496) delete acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.2174595.0:10537 0.000000 2014-08-13 17:38:02.755325 7fe9bae887a0 20 read_log 194215'14498 (192834'13100) clean 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by osd.325.0:1132 2014-08-12 15:36:54.121989 2014-08-13 17:38:02.755337 7fe9bae887a0 20 read_log 194215'14499 (194215'14498) delete 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.2174595.0:10539 0.000000 2014-08-13 17:38:02.755348 7fe9bae887a0 20 read_log 194215'14500 (192888'14101) clean 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by osd.325.0:1134 2014-08-12 15:36:54.132652 2014-08-13 17:38:02.755360 7fe9bae887a0 20 read_log 194215'14501 (194215'14500) delete 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.2174595.0:10541 0.000000 2014-08-13 17:38:02.755389 7fe9bae887a0 20 read_log 194215'14502 (192834'13594) clean 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by osd.325.0:1136 2014-08-12 15:36:54.144685 2014-08-13 17:38:02.755407 7fe9bae887a0 20 read_log 194215'14503 (194215'14502) delete 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.2174595.0:10543 0.000000 2014-08-13 17:38:02.755418 7fe9bae887a0 20 read_log 194215'14504 (193305'14230) clean 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by osd.325.0:1138 2014-08-12 15:36:54.157053 2014-08-13 17:38:02.755480 7fe9bae887a0 20 read_log 194215'14505 (194215'14504) delete 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.2174595.0:10545 0.000000 2014-08-13 17:38:02.755494 7fe9bae887a0 20 read_log 194215'14506 (192888'14085) clean 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by osd.325.0:1140 2014-08-12 15:36:54.167797 2014-08-13 17:38:02.755507 7fe9bae887a0 20 read_log 194215'14507 (194215'14506) delete 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.2174595.0:10547 0.000000 2014-08-13 17:38:02.755517 7fe9bae887a0 20 read_log 194215'14508 (192888'13942) clean 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by osd.325.0:1142 2014-08-12 15:36:54.179160 2014-08-13 17:38:02.755529 7fe9bae887a0 20 read_log 194215'14509 (194215'14508) delete 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.2174595.0:10549 0.000000 2014-08-13 17:38:02.755540 7fe9bae887a0 20 read_log 194215'14510 (192888'14109) clean 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by osd.325.0:1144 2014-08-12 15:36:54.191634 2014-08-13 17:38:02.755552 7fe9bae887a0 20 read_log 194215'14511 (194215'14510) delete 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.2174595.0:10551 0.000000 2014-08-13 17:38:02.755562 7fe9bae887a0 20 read_log 194215'14512 (194215'14512) modify 34/hit_set_24.34_archive_2014-08-12 15:32:25.735509_2014-08-12 17:24:42.814554/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.814875 2014-08-13 17:38:02.755575 7fe9bae887a0 20 read_log 194215'14513 (193305'14121) delete 34/hit_set_24.34_archive_2014-08-11 18:37:04.940322_2014-08-11 18:42:40.475022/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.814875 2014-08-13 17:38:02.755587 7fe9bae887a0 20 read_log 194215'14514 (0'0) promote 47dd8c34/benchmark_data_storage0111-ib.pocstor_34628_object717/head//24 by osd.325.0:7024 2014-08-12 17:24:42.833517 2014-08-13 17:38:02.755602 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.756055 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.756095 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=0 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.756113 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=0 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.014764 0 0.000000 2014-08-13 17:38:02.756125 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=0 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:02.756134 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=0 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194478 2014-08-13 17:38:02.756143 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] log((192718'11510,194215'14514], crt=193305'14233) 2014-08-13 17:38:02.756158 7fe9bae887a0 10 osd.4 194478 pgid 24.3d coll 24.3d_head 2014-08-13 17:38:02.756302 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.3d 2014-08-13 17:38:02.756315 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.756347 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3d(unlocked)] enter Initial 2014-08-13 17:38:02.756359 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.3d(unlocked)] enter NotTrimming 2014-08-13 17:38:02.756500 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.756624 7fe9bae887a0 20 read_log 192803'16166 (192718'15535) delete 3d/hit_set_24.3d_archive_2014-08-06 15:08:57.439794_2014-08-06 15:20:33.134523/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:52:38.941197 2014-08-13 17:38:02.756645 7fe9bae887a0 20 read_log 192803'16167 (0'0) promote 81bde03d/core.47650+10+2/head//24 by osd.4.0:18137 2014-08-11 12:53:03.341756 2014-08-13 17:38:02.756660 7fe9bae887a0 20 read_log 192803'16168 (192803'16167) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:1 2014-08-11 12:52:38.934293 2014-08-13 17:38:02.756672 7fe9bae887a0 20 read_log 192803'16169 (192803'16168) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:2 2014-08-11 12:53:09.149997 2014-08-13 17:38:02.756692 7fe9bae887a0 20 read_log 192803'16170 (192803'16169) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:3 2014-08-11 12:53:09.308293 2014-08-13 17:38:02.756710 7fe9bae887a0 20 read_log 192803'16171 (192803'16170) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:4 2014-08-11 12:53:09.429976 2014-08-13 17:38:02.756723 7fe9bae887a0 20 read_log 192803'16172 (192803'16171) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:5 2014-08-11 12:53:09.529269 2014-08-13 17:38:02.756735 7fe9bae887a0 20 read_log 192803'16173 (192803'16172) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:6 2014-08-11 12:53:09.556145 2014-08-13 17:38:02.756747 7fe9bae887a0 20 read_log 192803'16174 (192803'16173) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:7 2014-08-11 12:53:09.620832 2014-08-13 17:38:02.756759 7fe9bae887a0 20 read_log 192803'16175 (192803'16174) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:8 2014-08-11 12:53:09.647294 2014-08-13 17:38:02.756770 7fe9bae887a0 20 read_log 192803'16176 (192803'16175) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:9 2014-08-11 12:53:09.724633 2014-08-13 17:38:02.756782 7fe9bae887a0 20 read_log 192803'16177 (192803'16176) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:10 2014-08-11 12:53:09.750862 2014-08-13 17:38:02.756794 7fe9bae887a0 20 read_log 192803'16178 (192803'16177) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:11 2014-08-11 12:53:09.842022 2014-08-13 17:38:02.756806 7fe9bae887a0 20 read_log 192803'16179 (192803'16178) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:12 2014-08-11 12:53:09.868148 2014-08-13 17:38:02.756818 7fe9bae887a0 20 read_log 192803'16180 (192803'16179) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:13 2014-08-11 12:53:09.926787 2014-08-13 17:38:02.756829 7fe9bae887a0 20 read_log 192803'16181 (192803'16180) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:14 2014-08-11 12:53:09.952970 2014-08-13 17:38:02.756842 7fe9bae887a0 20 read_log 192803'16182 (192803'16181) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:15 2014-08-11 12:53:10.020818 2014-08-13 17:38:02.756854 7fe9bae887a0 20 read_log 192803'16183 (192803'16182) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:16 2014-08-11 12:53:10.048300 2014-08-13 17:38:02.756866 7fe9bae887a0 20 read_log 192803'16184 (192803'16183) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:17 2014-08-11 12:53:10.113637 2014-08-13 17:38:02.756878 7fe9bae887a0 20 read_log 192803'16185 (192803'16184) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:18 2014-08-11 12:53:10.134930 2014-08-13 17:38:02.756890 7fe9bae887a0 20 read_log 192803'16186 (192803'16185) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:19 2014-08-11 12:53:10.422526 2014-08-13 17:38:02.756902 7fe9bae887a0 20 read_log 192803'16187 (192803'16186) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:20 2014-08-11 12:53:10.450567 2014-08-13 17:38:02.756914 7fe9bae887a0 20 read_log 192803'16188 (192803'16187) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:21 2014-08-11 12:53:10.515845 2014-08-13 17:38:02.756925 7fe9bae887a0 20 read_log 192803'16189 (192803'16188) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:22 2014-08-11 12:53:10.891328 2014-08-13 17:38:02.756938 7fe9bae887a0 20 read_log 192803'16190 (192803'16189) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:23 2014-08-11 12:53:10.932960 2014-08-13 17:38:02.756950 7fe9bae887a0 20 read_log 192803'16191 (192803'16190) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:24 2014-08-11 12:53:10.970227 2014-08-13 17:38:02.756962 7fe9bae887a0 20 read_log 192803'16192 (192803'16191) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:25 2014-08-11 12:53:11.035408 2014-08-13 17:38:02.756973 7fe9bae887a0 20 read_log 192803'16193 (192803'16192) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:26 2014-08-11 12:53:11.058793 2014-08-13 17:38:02.756995 7fe9bae887a0 20 read_log 192803'16194 (192803'16193) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:27 2014-08-11 12:53:11.114810 2014-08-13 17:38:02.757008 7fe9bae887a0 20 read_log 192803'16195 (192803'16194) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:28 2014-08-11 12:53:11.137614 2014-08-13 17:38:02.757020 7fe9bae887a0 20 read_log 192803'16196 (192803'16195) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:29 2014-08-11 12:53:11.206086 2014-08-13 17:38:02.757032 7fe9bae887a0 20 read_log 192803'16197 (192803'16196) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:30 2014-08-11 12:53:11.232521 2014-08-13 17:38:02.757044 7fe9bae887a0 20 read_log 192803'16198 (192803'16197) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:31 2014-08-11 12:53:11.343572 2014-08-13 17:38:02.757055 7fe9bae887a0 20 read_log 192803'16199 (192803'16198) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:32 2014-08-11 12:53:11.617607 2014-08-13 17:38:02.757067 7fe9bae887a0 20 read_log 192803'16200 (192803'16199) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:33 2014-08-11 12:53:11.690478 2014-08-13 17:38:02.757079 7fe9bae887a0 20 read_log 192803'16201 (192803'16200) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:34 2014-08-11 12:53:11.718008 2014-08-13 17:38:02.757090 7fe9bae887a0 20 read_log 192803'16202 (192803'16201) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:35 2014-08-11 12:53:11.782179 2014-08-13 17:38:02.757102 7fe9bae887a0 20 read_log 192803'16203 (192803'16202) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:36 2014-08-11 12:53:11.808717 2014-08-13 17:38:02.757116 7fe9bae887a0 20 read_log 192803'16204 (192803'16203) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:37 2014-08-11 12:53:11.860744 2014-08-13 17:38:02.757128 7fe9bae887a0 20 read_log 192803'16205 (192803'16204) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:38 2014-08-11 12:53:11.894488 2014-08-13 17:38:02.757143 7fe9bae887a0 20 read_log 192803'16206 (192803'16205) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:39 2014-08-11 12:53:11.976918 2014-08-13 17:38:02.757161 7fe9bae887a0 20 read_log 192803'16207 (192803'16206) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:40 2014-08-11 12:53:12.002124 2014-08-13 17:38:02.757193 7fe9bae887a0 20 read_log 192803'16208 (192803'16207) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:41 2014-08-11 12:53:12.039820 2014-08-13 17:38:02.757208 7fe9bae887a0 20 read_log 192803'16209 (192803'16208) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:42 2014-08-11 12:53:12.440161 2014-08-13 17:38:02.757220 7fe9bae887a0 20 read_log 192803'16210 (192803'16209) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:43 2014-08-11 12:53:12.541883 2014-08-13 17:38:02.757232 7fe9bae887a0 20 read_log 192803'16211 (192803'16210) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:44 2014-08-11 12:53:12.566215 2014-08-13 17:38:02.757244 7fe9bae887a0 20 read_log 192803'16212 (192803'16211) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:45 2014-08-11 12:53:12.627677 2014-08-13 17:38:02.757255 7fe9bae887a0 20 read_log 192803'16213 (192803'16212) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:46 2014-08-11 12:53:12.657837 2014-08-13 17:38:02.757267 7fe9bae887a0 20 read_log 192803'16214 (192803'16213) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:47 2014-08-11 12:53:12.716735 2014-08-13 17:38:02.757279 7fe9bae887a0 20 read_log 192803'16215 (192803'16214) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:48 2014-08-11 12:53:12.742744 2014-08-13 17:38:02.757291 7fe9bae887a0 20 read_log 192803'16216 (192803'16215) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:49 2014-08-11 12:53:12.790931 2014-08-13 17:38:02.757302 7fe9bae887a0 20 read_log 192803'16217 (192803'16216) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:50 2014-08-11 12:53:12.816572 2014-08-13 17:38:02.757322 7fe9bae887a0 20 read_log 192803'16218 (192803'16217) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:51 2014-08-11 12:53:12.886053 2014-08-13 17:38:02.757337 7fe9bae887a0 20 read_log 192803'16219 (192803'16218) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:52 2014-08-11 12:53:13.216554 2014-08-13 17:38:02.757350 7fe9bae887a0 20 read_log 192803'16220 (192803'16219) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:53 2014-08-11 12:53:13.256556 2014-08-13 17:38:02.757362 7fe9bae887a0 20 read_log 192803'16221 (192803'16220) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:54 2014-08-11 12:53:13.282947 2014-08-13 17:38:02.757373 7fe9bae887a0 20 read_log 192803'16222 (192803'16221) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:55 2014-08-11 12:53:13.359808 2014-08-13 17:38:02.757385 7fe9bae887a0 20 read_log 192803'16223 (192803'16222) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:56 2014-08-11 12:53:13.385423 2014-08-13 17:38:02.757396 7fe9bae887a0 20 read_log 192803'16224 (192803'16223) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:57 2014-08-11 12:53:13.458956 2014-08-13 17:38:02.757408 7fe9bae887a0 20 read_log 192803'16225 (192803'16224) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:58 2014-08-11 12:53:13.484030 2014-08-13 17:38:02.757420 7fe9bae887a0 20 read_log 192803'16226 (192803'16225) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:59 2014-08-11 12:53:13.566591 2014-08-13 17:38:02.757431 7fe9bae887a0 20 read_log 192803'16227 (192803'16226) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:60 2014-08-11 12:53:13.593985 2014-08-13 17:38:02.757443 7fe9bae887a0 20 read_log 192803'16228 (192803'16227) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:61 2014-08-11 12:53:13.664073 2014-08-13 17:38:02.757455 7fe9bae887a0 20 read_log 192803'16229 (192803'16228) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:62 2014-08-11 12:53:13.975180 2014-08-13 17:38:02.757466 7fe9bae887a0 20 read_log 192803'16230 (192803'16229) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:63 2014-08-11 12:53:14.013106 2014-08-13 17:38:02.757478 7fe9bae887a0 20 read_log 192803'16231 (192803'16230) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:64 2014-08-11 12:53:14.055333 2014-08-13 17:38:02.757490 7fe9bae887a0 20 read_log 192803'16232 (192803'16231) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:65 2014-08-11 12:53:14.119762 2014-08-13 17:38:02.757501 7fe9bae887a0 20 read_log 192803'16233 (192803'16232) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:66 2014-08-11 12:53:14.145321 2014-08-13 17:38:02.757590 7fe9bae887a0 20 read_log 192803'16234 (192803'16233) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:67 2014-08-11 12:53:14.223820 2014-08-13 17:38:02.757603 7fe9bae887a0 20 read_log 192803'16235 (192803'16234) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:68 2014-08-11 12:53:14.250277 2014-08-13 17:38:02.757615 7fe9bae887a0 20 read_log 192803'16236 (192803'16235) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:69 2014-08-11 12:53:14.275807 2014-08-13 17:38:02.757627 7fe9bae887a0 20 read_log 192803'16237 (192803'16236) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:70 2014-08-11 12:53:14.300500 2014-08-13 17:38:02.757639 7fe9bae887a0 20 read_log 192803'16238 (192803'16237) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:71 2014-08-11 12:53:14.384316 2014-08-13 17:38:02.757651 7fe9bae887a0 20 read_log 192803'16239 (192803'16238) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:72 2014-08-11 12:53:14.753101 2014-08-13 17:38:02.757663 7fe9bae887a0 20 read_log 192803'16240 (192803'16239) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:73 2014-08-11 12:53:14.834096 2014-08-13 17:38:02.757674 7fe9bae887a0 20 read_log 192803'16241 (192803'16240) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:74 2014-08-11 12:53:14.877342 2014-08-13 17:38:02.757694 7fe9bae887a0 20 read_log 192803'16242 (192803'16241) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:75 2014-08-11 12:53:14.937237 2014-08-13 17:38:02.757707 7fe9bae887a0 20 read_log 192803'16243 (192803'16242) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:76 2014-08-11 12:53:14.963267 2014-08-13 17:38:02.757719 7fe9bae887a0 20 read_log 192803'16244 (192803'16243) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:77 2014-08-11 12:53:14.993830 2014-08-13 17:38:02.757731 7fe9bae887a0 20 read_log 192803'16245 (192803'16244) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:78 2014-08-11 12:53:15.036318 2014-08-13 17:38:02.757760 7fe9bae887a0 20 read_log 192803'16246 (192803'16245) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:79 2014-08-11 12:53:15.101187 2014-08-13 17:38:02.757776 7fe9bae887a0 20 read_log 192803'16247 (192803'16246) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:80 2014-08-11 12:53:15.128008 2014-08-13 17:38:02.757789 7fe9bae887a0 20 read_log 192803'16248 (192803'16247) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:81 2014-08-11 12:53:15.162392 2014-08-13 17:38:02.757800 7fe9bae887a0 20 read_log 192803'16249 (192803'16248) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:82 2014-08-11 12:53:15.200863 2014-08-13 17:38:02.757812 7fe9bae887a0 20 read_log 192803'16250 (192803'16249) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:83 2014-08-11 12:53:15.265261 2014-08-13 17:38:02.757824 7fe9bae887a0 20 read_log 192803'16251 (192803'16250) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:84 2014-08-11 12:53:15.703763 2014-08-13 17:38:02.757837 7fe9bae887a0 20 read_log 192803'16252 (192803'16251) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:85 2014-08-11 12:53:15.741336 2014-08-13 17:38:02.757849 7fe9bae887a0 20 read_log 192803'16253 (192803'16252) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:86 2014-08-11 12:53:15.780803 2014-08-13 17:38:02.757860 7fe9bae887a0 20 read_log 192803'16254 (192803'16253) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:87 2014-08-11 12:53:15.848785 2014-08-13 17:38:02.757872 7fe9bae887a0 20 read_log 192803'16255 (192803'16254) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:88 2014-08-11 12:53:15.876047 2014-08-13 17:38:02.757885 7fe9bae887a0 20 read_log 192803'16256 (192803'16255) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:89 2014-08-11 12:53:15.939770 2014-08-13 17:38:02.757896 7fe9bae887a0 20 read_log 192803'16257 (192803'16256) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:90 2014-08-11 12:53:15.966874 2014-08-13 17:38:02.757908 7fe9bae887a0 20 read_log 192803'16258 (192803'16257) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:91 2014-08-11 12:53:16.021513 2014-08-13 17:38:02.757920 7fe9bae887a0 20 read_log 192803'16259 (192803'16258) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:92 2014-08-11 12:53:16.058161 2014-08-13 17:38:02.757932 7fe9bae887a0 20 read_log 192803'16260 (192803'16259) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:93 2014-08-11 12:53:16.116873 2014-08-13 17:38:02.757944 7fe9bae887a0 20 read_log 192803'16261 (192803'16260) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:94 2014-08-11 12:53:16.430376 2014-08-13 17:38:02.757956 7fe9bae887a0 20 read_log 192803'16262 (192803'16261) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:95 2014-08-11 12:53:16.500234 2014-08-13 17:38:02.757968 7fe9bae887a0 20 read_log 192803'16263 (192803'16262) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:96 2014-08-11 12:53:16.528880 2014-08-13 17:38:02.757979 7fe9bae887a0 20 read_log 192803'16264 (192803'16263) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:97 2014-08-11 12:53:16.555832 2014-08-13 17:38:02.757991 7fe9bae887a0 20 read_log 192803'16265 (192803'16264) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:98 2014-08-11 12:53:16.577323 2014-08-13 17:38:02.758010 7fe9bae887a0 20 read_log 192803'16266 (192803'16265) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:99 2014-08-11 12:53:16.636980 2014-08-13 17:38:02.758022 7fe9bae887a0 20 read_log 192803'16267 (192803'16266) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:100 2014-08-11 12:53:16.658951 2014-08-13 17:38:02.758034 7fe9bae887a0 20 read_log 192803'16268 (192803'16267) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:101 2014-08-11 12:53:16.721705 2014-08-13 17:38:02.758046 7fe9bae887a0 20 read_log 192803'16269 (192803'16268) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:102 2014-08-11 12:53:16.748156 2014-08-13 17:38:02.758058 7fe9bae887a0 20 read_log 192803'16270 (192803'16269) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:103 2014-08-11 12:53:16.774213 2014-08-13 17:38:02.758071 7fe9bae887a0 20 read_log 192803'16271 (192803'16270) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:104 2014-08-11 12:53:17.013259 2014-08-13 17:38:02.758084 7fe9bae887a0 20 read_log 192803'16272 (192803'16271) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:105 2014-08-11 12:53:17.076765 2014-08-13 17:38:02.758096 7fe9bae887a0 20 read_log 192803'16273 (192803'16272) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:106 2014-08-11 12:53:17.100864 2014-08-13 17:38:02.758108 7fe9bae887a0 20 read_log 192803'16274 (192803'16273) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:107 2014-08-11 12:53:17.161869 2014-08-13 17:38:02.758119 7fe9bae887a0 20 read_log 192803'16275 (192803'16274) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:108 2014-08-11 12:53:17.187720 2014-08-13 17:38:02.758133 7fe9bae887a0 20 read_log 192803'16276 (192803'16275) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:109 2014-08-11 12:53:17.214317 2014-08-13 17:38:02.758147 7fe9bae887a0 20 read_log 192803'16277 (192803'16276) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:110 2014-08-11 12:53:17.238684 2014-08-13 17:38:02.758165 7fe9bae887a0 20 read_log 192803'16278 (192803'16277) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:111 2014-08-11 12:53:17.282322 2014-08-13 17:38:02.758179 7fe9bae887a0 20 read_log 192803'16279 (192803'16278) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:112 2014-08-11 12:53:17.307557 2014-08-13 17:38:02.758191 7fe9bae887a0 20 read_log 192803'16280 (192803'16279) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:113 2014-08-11 12:53:17.334339 2014-08-13 17:38:02.758203 7fe9bae887a0 20 read_log 192803'16281 (192803'16280) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:114 2014-08-11 12:53:17.595136 2014-08-13 17:38:02.758215 7fe9bae887a0 20 read_log 192803'16282 (192803'16281) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:115 2014-08-11 12:53:17.652062 2014-08-13 17:38:02.758227 7fe9bae887a0 20 read_log 192803'16283 (192803'16282) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:116 2014-08-11 12:53:17.678893 2014-08-13 17:38:02.758239 7fe9bae887a0 20 read_log 192803'16284 (192803'16283) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:117 2014-08-11 12:53:17.743581 2014-08-13 17:38:02.758250 7fe9bae887a0 20 read_log 192803'16285 (192803'16284) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:118 2014-08-11 12:53:17.769609 2014-08-13 17:38:02.758262 7fe9bae887a0 20 read_log 192803'16286 (192803'16285) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:119 2014-08-11 12:53:17.828212 2014-08-13 17:38:02.758274 7fe9bae887a0 20 read_log 192803'16287 (192803'16286) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:120 2014-08-11 12:53:17.855241 2014-08-13 17:38:02.758286 7fe9bae887a0 20 read_log 192803'16288 (192803'16287) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:121 2014-08-11 12:53:17.919891 2014-08-13 17:38:02.758297 7fe9bae887a0 20 read_log 192803'16289 (192803'16288) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:122 2014-08-11 12:53:17.946061 2014-08-13 17:38:02.758335 7fe9bae887a0 20 read_log 192803'16290 (192803'16289) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:123 2014-08-11 12:53:18.011313 2014-08-13 17:38:02.758355 7fe9bae887a0 20 read_log 192803'16291 (192803'16290) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:124 2014-08-11 12:53:18.128711 2014-08-13 17:38:02.758368 7fe9bae887a0 20 read_log 192803'16292 (192803'16291) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:125 2014-08-11 12:53:18.214666 2014-08-13 17:38:02.758380 7fe9bae887a0 20 read_log 192803'16293 (192803'16292) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:126 2014-08-11 12:53:18.241425 2014-08-13 17:38:02.758392 7fe9bae887a0 20 read_log 192803'16294 (192803'16293) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:127 2014-08-11 12:53:18.329826 2014-08-13 17:38:02.758403 7fe9bae887a0 20 read_log 192803'16295 (192803'16294) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:128 2014-08-11 12:53:18.359328 2014-08-13 17:38:02.758415 7fe9bae887a0 20 read_log 192803'16296 (192803'16295) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:129 2014-08-11 12:53:18.425544 2014-08-13 17:38:02.758427 7fe9bae887a0 20 read_log 192803'16297 (192803'16296) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:130 2014-08-11 12:53:18.447216 2014-08-13 17:38:02.758439 7fe9bae887a0 20 read_log 192803'16298 (192803'16297) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:131 2014-08-11 12:53:18.516961 2014-08-13 17:38:02.758451 7fe9bae887a0 20 read_log 192803'16299 (192803'16298) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:132 2014-08-11 12:53:18.542532 2014-08-13 17:38:02.758464 7fe9bae887a0 20 read_log 192803'16300 (192803'16299) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:133 2014-08-11 12:53:18.639185 2014-08-13 17:38:02.758492 7fe9bae887a0 20 read_log 192803'16301 (192803'16300) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:134 2014-08-11 12:53:18.696616 2014-08-13 17:38:02.758507 7fe9bae887a0 20 read_log 192803'16302 (192803'16301) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:135 2014-08-11 12:53:18.775680 2014-08-13 17:38:02.758519 7fe9bae887a0 20 read_log 192803'16303 (192803'16302) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:136 2014-08-11 12:53:18.801705 2014-08-13 17:38:02.758531 7fe9bae887a0 20 read_log 192803'16304 (192803'16303) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:137 2014-08-11 12:53:18.878857 2014-08-13 17:38:02.758543 7fe9bae887a0 20 read_log 192803'16305 (192803'16304) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:138 2014-08-11 12:53:18.905542 2014-08-13 17:38:02.758556 7fe9bae887a0 20 read_log 192803'16306 (192803'16305) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:139 2014-08-11 12:53:18.962980 2014-08-13 17:38:02.758568 7fe9bae887a0 20 read_log 192803'16307 (192803'16306) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:140 2014-08-11 12:53:18.986933 2014-08-13 17:38:02.758579 7fe9bae887a0 20 read_log 192803'16308 (192803'16307) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:141 2014-08-11 12:53:19.048188 2014-08-13 17:38:02.758591 7fe9bae887a0 20 read_log 192803'16309 (192803'16308) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:142 2014-08-11 12:53:19.074853 2014-08-13 17:38:02.758603 7fe9bae887a0 20 read_log 192803'16310 (192803'16309) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:143 2014-08-11 12:53:19.101265 2014-08-13 17:38:02.758615 7fe9bae887a0 20 read_log 192803'16311 (192803'16310) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:144 2014-08-11 12:53:19.207160 2014-08-13 17:38:02.758626 7fe9bae887a0 20 read_log 192803'16312 (192803'16311) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:145 2014-08-11 12:53:19.233707 2014-08-13 17:38:02.758638 7fe9bae887a0 20 read_log 192803'16313 (192803'16312) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:146 2014-08-11 12:53:19.259798 2014-08-13 17:38:02.758658 7fe9bae887a0 20 read_log 192803'16314 (192803'16313) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:147 2014-08-11 12:53:19.343621 2014-08-13 17:38:02.758670 7fe9bae887a0 20 read_log 192803'16315 (192803'16314) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:148 2014-08-11 12:53:19.370268 2014-08-13 17:38:02.758683 7fe9bae887a0 20 read_log 192803'16316 (192803'16315) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:149 2014-08-11 12:53:19.431809 2014-08-13 17:38:02.758695 7fe9bae887a0 20 read_log 192803'16317 (192803'16316) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:150 2014-08-11 12:53:19.452499 2014-08-13 17:38:02.758706 7fe9bae887a0 20 read_log 192803'16318 (192803'16317) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:151 2014-08-11 12:53:19.492107 2014-08-13 17:38:02.758718 7fe9bae887a0 20 read_log 192803'16319 (192803'16318) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:152 2014-08-11 12:53:19.516496 2014-08-13 17:38:02.758729 7fe9bae887a0 20 read_log 192803'16320 (192803'16319) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:153 2014-08-11 12:53:19.570492 2014-08-13 17:38:02.758741 7fe9bae887a0 20 read_log 192803'16321 (192803'16320) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:154 2014-08-11 12:53:19.733109 2014-08-13 17:38:02.758753 7fe9bae887a0 20 read_log 192803'16322 (192803'16321) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:155 2014-08-11 12:53:19.764513 2014-08-13 17:38:02.758765 7fe9bae887a0 20 read_log 192803'16323 (192803'16322) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:156 2014-08-11 12:53:19.792803 2014-08-13 17:38:02.758776 7fe9bae887a0 20 read_log 192803'16324 (192803'16323) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:157 2014-08-11 12:53:19.821841 2014-08-13 17:38:02.758788 7fe9bae887a0 20 read_log 192803'16325 (192803'16324) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:158 2014-08-11 12:53:19.919592 2014-08-13 17:38:02.758800 7fe9bae887a0 20 read_log 192803'16326 (192803'16325) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:159 2014-08-11 12:53:20.037234 2014-08-13 17:38:02.758811 7fe9bae887a0 20 read_log 192803'16327 (192803'16326) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:160 2014-08-11 12:53:20.070366 2014-08-13 17:38:02.758823 7fe9bae887a0 20 read_log 192803'16328 (192803'16327) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:161 2014-08-11 12:53:20.133265 2014-08-13 17:38:02.758834 7fe9bae887a0 20 read_log 192803'16329 (192803'16328) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:162 2014-08-11 12:53:20.159627 2014-08-13 17:38:02.758846 7fe9bae887a0 20 read_log 192803'16330 (192803'16329) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:163 2014-08-11 12:53:20.234424 2014-08-13 17:38:02.758858 7fe9bae887a0 20 read_log 192803'16331 (192803'16330) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:164 2014-08-11 12:53:20.260085 2014-08-13 17:38:02.758870 7fe9bae887a0 20 read_log 192803'16332 (192803'16331) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:165 2014-08-11 12:53:20.306447 2014-08-13 17:38:02.758881 7fe9bae887a0 20 read_log 192803'16333 (192803'16332) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:166 2014-08-11 12:53:20.332201 2014-08-13 17:38:02.758894 7fe9bae887a0 20 read_log 192803'16334 (192803'16333) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:167 2014-08-11 12:53:20.409692 2014-08-13 17:38:02.758906 7fe9bae887a0 20 read_log 192803'16335 (192803'16334) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:168 2014-08-11 12:53:20.437062 2014-08-13 17:38:02.758920 7fe9bae887a0 20 read_log 192803'16336 (192803'16335) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:169 2014-08-11 12:53:20.501814 2014-08-13 17:38:02.758932 7fe9bae887a0 20 read_log 192803'16337 (192803'16336) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:170 2014-08-11 12:53:20.668789 2014-08-13 17:38:02.758953 7fe9bae887a0 20 read_log 192803'16338 (192803'16337) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:171 2014-08-11 12:53:20.739359 2014-08-13 17:38:02.758968 7fe9bae887a0 20 read_log 192803'16339 (192803'16338) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:172 2014-08-11 12:53:20.783028 2014-08-13 17:38:02.758980 7fe9bae887a0 20 read_log 192803'16340 (192803'16339) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:173 2014-08-11 12:53:20.816340 2014-08-13 17:38:02.758992 7fe9bae887a0 20 read_log 192803'16341 (192803'16340) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:174 2014-08-11 12:53:20.842641 2014-08-13 17:38:02.759006 7fe9bae887a0 20 read_log 192803'16342 (192803'16341) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:175 2014-08-11 12:53:20.912013 2014-08-13 17:38:02.759018 7fe9bae887a0 20 read_log 192803'16343 (192803'16342) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:176 2014-08-11 12:53:20.934022 2014-08-13 17:38:02.759030 7fe9bae887a0 20 read_log 192803'16344 (192803'16343) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:177 2014-08-11 12:53:20.968716 2014-08-13 17:38:02.759042 7fe9bae887a0 20 read_log 192803'16345 (192803'16344) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:178 2014-08-11 12:53:20.993636 2014-08-13 17:38:02.759054 7fe9bae887a0 20 read_log 192803'16346 (192803'16345) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:179 2014-08-11 12:53:21.048268 2014-08-13 17:38:02.759068 7fe9bae887a0 20 read_log 192803'16347 (192803'16346) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:180 2014-08-11 12:53:21.241107 2014-08-13 17:38:02.759080 7fe9bae887a0 20 read_log 192803'16348 (192803'16347) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:181 2014-08-11 12:53:21.329314 2014-08-13 17:38:02.759094 7fe9bae887a0 20 read_log 192803'16349 (192803'16348) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:182 2014-08-11 12:53:21.356763 2014-08-13 17:38:02.759106 7fe9bae887a0 20 read_log 192803'16350 (192803'16349) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:183 2014-08-11 12:53:21.428007 2014-08-13 17:38:02.759118 7fe9bae887a0 20 read_log 192803'16351 (192803'16350) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:184 2014-08-11 12:53:21.450847 2014-08-13 17:38:02.759130 7fe9bae887a0 20 read_log 192803'16352 (192803'16351) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:185 2014-08-11 12:53:21.516743 2014-08-13 17:38:02.759142 7fe9bae887a0 20 read_log 192803'16353 (192803'16352) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:186 2014-08-11 12:53:21.542466 2014-08-13 17:38:02.759159 7fe9bae887a0 20 read_log 192803'16354 (192803'16353) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:187 2014-08-11 12:53:21.605362 2014-08-13 17:38:02.759174 7fe9bae887a0 20 read_log 192803'16355 (192803'16354) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:188 2014-08-11 12:53:21.631143 2014-08-13 17:38:02.759186 7fe9bae887a0 20 read_log 192803'16356 (192803'16355) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:189 2014-08-11 12:53:21.659546 2014-08-13 17:38:02.759198 7fe9bae887a0 20 read_log 192803'16357 (192803'16356) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:190 2014-08-11 12:53:21.752157 2014-08-13 17:38:02.759210 7fe9bae887a0 20 read_log 192803'16358 (192803'16357) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:191 2014-08-11 12:53:21.778473 2014-08-13 17:38:02.759221 7fe9bae887a0 20 read_log 192803'16359 (192803'16358) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:192 2014-08-11 12:53:21.804135 2014-08-13 17:38:02.759235 7fe9bae887a0 20 read_log 192803'16360 (192803'16359) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:193 2014-08-11 12:53:21.863720 2014-08-13 17:38:02.759247 7fe9bae887a0 20 read_log 192803'16361 (192803'16360) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:194 2014-08-11 12:53:21.889897 2014-08-13 17:38:02.759270 7fe9bae887a0 20 read_log 192803'16362 (192803'16361) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:195 2014-08-11 12:53:21.965185 2014-08-13 17:38:02.759285 7fe9bae887a0 20 read_log 192803'16363 (192803'16362) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:196 2014-08-11 12:53:21.990976 2014-08-13 17:38:02.759297 7fe9bae887a0 20 read_log 192803'16364 (192803'16363) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:197 2014-08-11 12:53:22.066394 2014-08-13 17:38:02.759309 7fe9bae887a0 20 read_log 192803'16365 (192803'16364) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:198 2014-08-11 12:53:22.094184 2014-08-13 17:38:02.759321 7fe9bae887a0 20 read_log 192803'16366 (192803'16365) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:199 2014-08-11 12:53:22.170469 2014-08-13 17:38:02.759333 7fe9bae887a0 20 read_log 192803'16367 (192803'16366) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:200 2014-08-11 12:53:22.274931 2014-08-13 17:38:02.759345 7fe9bae887a0 20 read_log 192803'16368 (192803'16367) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:201 2014-08-11 12:53:22.327497 2014-08-13 17:38:02.759356 7fe9bae887a0 20 read_log 192803'16369 (192803'16368) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:202 2014-08-11 12:53:22.353873 2014-08-13 17:38:02.759368 7fe9bae887a0 20 read_log 192803'16370 (192803'16369) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:203 2014-08-11 12:53:22.424998 2014-08-13 17:38:02.759380 7fe9bae887a0 20 read_log 192803'16371 (192803'16370) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:204 2014-08-11 12:53:22.451050 2014-08-13 17:38:02.759392 7fe9bae887a0 20 read_log 192803'16372 (192803'16371) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:205 2014-08-11 12:53:22.471589 2014-08-13 17:38:02.759404 7fe9bae887a0 20 read_log 192803'16373 (192803'16372) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:206 2014-08-11 12:53:22.494330 2014-08-13 17:38:02.759415 7fe9bae887a0 20 read_log 192803'16374 (192803'16373) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:207 2014-08-11 12:53:22.540204 2014-08-13 17:38:02.759429 7fe9bae887a0 20 read_log 192803'16375 (192803'16374) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:208 2014-08-11 12:53:22.566975 2014-08-13 17:38:02.759441 7fe9bae887a0 20 read_log 192803'16376 (192803'16375) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:209 2014-08-11 12:53:22.630812 2014-08-13 17:38:02.759453 7fe9bae887a0 20 read_log 192803'16377 (192803'16376) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:210 2014-08-11 12:53:22.748070 2014-08-13 17:38:02.759465 7fe9bae887a0 20 read_log 192803'16378 (192803'16377) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:211 2014-08-11 12:53:22.773561 2014-08-13 17:38:02.759477 7fe9bae887a0 20 read_log 192803'16379 (192803'16378) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:212 2014-08-11 12:53:22.798412 2014-08-13 17:38:02.759488 7fe9bae887a0 20 read_log 192803'16380 (192803'16379) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:213 2014-08-11 12:53:22.860739 2014-08-13 17:38:02.759500 7fe9bae887a0 20 read_log 192803'16381 (192803'16380) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:214 2014-08-11 12:53:22.886460 2014-08-13 17:38:02.759512 7fe9bae887a0 20 read_log 192803'16382 (192803'16381) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:215 2014-08-11 12:53:22.911615 2014-08-13 17:38:02.759524 7fe9bae887a0 20 read_log 192803'16383 (192803'16382) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:216 2014-08-11 12:53:22.937607 2014-08-13 17:38:02.759535 7fe9bae887a0 20 read_log 192803'16384 (192803'16383) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:217 2014-08-11 12:53:22.994578 2014-08-13 17:38:02.759547 7fe9bae887a0 20 read_log 192803'16385 (192803'16384) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:218 2014-08-11 12:53:23.020473 2014-08-13 17:38:02.759565 7fe9bae887a0 20 read_log 192803'16386 (192803'16385) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:219 2014-08-11 12:53:23.102186 2014-08-13 17:38:02.759578 7fe9bae887a0 20 read_log 192803'16387 (192803'16386) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:220 2014-08-11 12:53:23.266340 2014-08-13 17:38:02.759592 7fe9bae887a0 20 read_log 192803'16388 (192803'16387) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:221 2014-08-11 12:53:23.301336 2014-08-13 17:38:02.759604 7fe9bae887a0 20 read_log 192803'16389 (192803'16388) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:222 2014-08-11 12:53:23.327520 2014-08-13 17:38:02.759615 7fe9bae887a0 20 read_log 192803'16390 (192803'16389) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:223 2014-08-11 12:53:23.394259 2014-08-13 17:38:02.759627 7fe9bae887a0 20 read_log 192803'16391 (192803'16390) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:224 2014-08-11 12:53:23.420533 2014-08-13 17:38:02.759639 7fe9bae887a0 20 read_log 192803'16392 (192803'16391) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:225 2014-08-11 12:53:23.468345 2014-08-13 17:38:02.759651 7fe9bae887a0 20 read_log 192803'16393 (192803'16392) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:226 2014-08-11 12:53:23.494040 2014-08-13 17:38:02.759683 7fe9bae887a0 20 read_log 192803'16394 (192803'16393) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:227 2014-08-11 12:53:23.543339 2014-08-13 17:38:02.759699 7fe9bae887a0 20 read_log 192803'16395 (192803'16394) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:228 2014-08-11 12:53:23.569467 2014-08-13 17:38:02.759711 7fe9bae887a0 20 read_log 192803'16396 (192803'16395) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:229 2014-08-11 12:53:23.604468 2014-08-13 17:38:02.759723 7fe9bae887a0 20 read_log 192803'16397 (192803'16396) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:230 2014-08-11 12:53:23.789532 2014-08-13 17:38:02.759735 7fe9bae887a0 20 read_log 192803'16398 (192803'16397) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:231 2014-08-11 12:53:23.874588 2014-08-13 17:38:02.759746 7fe9bae887a0 20 read_log 192803'16399 (192803'16398) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:232 2014-08-11 12:53:23.900001 2014-08-13 17:38:02.759758 7fe9bae887a0 20 read_log 192803'16400 (192803'16399) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:233 2014-08-11 12:53:23.968171 2014-08-13 17:38:02.759770 7fe9bae887a0 20 read_log 192803'16401 (192803'16400) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:234 2014-08-11 12:53:23.994698 2014-08-13 17:38:02.759782 7fe9bae887a0 20 read_log 192803'16402 (192803'16401) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:235 2014-08-11 12:53:24.019855 2014-08-13 17:38:02.759794 7fe9bae887a0 20 read_log 192803'16403 (192803'16402) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:236 2014-08-11 12:53:24.065672 2014-08-13 17:38:02.759805 7fe9bae887a0 20 read_log 192803'16404 (192803'16403) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:237 2014-08-11 12:53:24.110490 2014-08-13 17:38:02.759817 7fe9bae887a0 20 read_log 192803'16405 (192803'16404) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:238 2014-08-11 12:53:24.136122 2014-08-13 17:38:02.759829 7fe9bae887a0 20 read_log 192803'16406 (192803'16405) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:239 2014-08-11 12:53:24.213446 2014-08-13 17:38:02.759841 7fe9bae887a0 20 read_log 192803'16407 (192803'16406) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:240 2014-08-11 12:53:24.385694 2014-08-13 17:38:02.759852 7fe9bae887a0 20 read_log 192803'16408 (192803'16407) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:241 2014-08-11 12:53:24.428666 2014-08-13 17:38:02.759864 7fe9bae887a0 20 read_log 192803'16409 (192803'16408) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:242 2014-08-11 12:53:24.463413 2014-08-13 17:38:02.759882 7fe9bae887a0 20 read_log 192803'16410 (192803'16409) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:243 2014-08-11 12:53:24.502416 2014-08-13 17:38:02.759895 7fe9bae887a0 20 read_log 192803'16411 (192803'16410) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:244 2014-08-11 12:53:24.541034 2014-08-13 17:38:02.759910 7fe9bae887a0 20 read_log 192803'16412 (192803'16411) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:245 2014-08-11 12:53:24.606516 2014-08-13 17:38:02.759924 7fe9bae887a0 20 read_log 192803'16413 (192803'16412) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:246 2014-08-11 12:53:24.631675 2014-08-13 17:38:02.759936 7fe9bae887a0 20 read_log 192803'16414 (192803'16413) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:247 2014-08-11 12:53:24.676639 2014-08-13 17:38:02.759948 7fe9bae887a0 20 read_log 192803'16415 (192803'16414) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:248 2014-08-11 12:53:24.739787 2014-08-13 17:38:02.759960 7fe9bae887a0 20 read_log 192803'16416 (192803'16415) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:249 2014-08-11 12:53:24.807420 2014-08-13 17:38:02.759971 7fe9bae887a0 20 read_log 192803'16417 (192803'16416) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:250 2014-08-11 12:53:25.151658 2014-08-13 17:38:02.759983 7fe9bae887a0 20 read_log 192803'16418 (192803'16417) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:251 2014-08-11 12:53:25.224836 2014-08-13 17:38:02.759995 7fe9bae887a0 20 read_log 192803'16419 (192803'16418) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:252 2014-08-11 12:53:25.267682 2014-08-13 17:38:02.760007 7fe9bae887a0 20 read_log 192803'16420 (192803'16419) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:253 2014-08-11 12:53:25.301831 2014-08-13 17:38:02.760018 7fe9bae887a0 20 read_log 192803'16421 (192803'16420) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:254 2014-08-11 12:53:25.336099 2014-08-13 17:38:02.760030 7fe9bae887a0 20 read_log 192803'16422 (192803'16421) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:255 2014-08-11 12:53:25.399491 2014-08-13 17:38:02.760042 7fe9bae887a0 20 read_log 192803'16423 (192803'16422) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:256 2014-08-11 12:53:25.427045 2014-08-13 17:38:02.760054 7fe9bae887a0 20 read_log 192803'16424 (192803'16423) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:257 2014-08-11 12:53:25.473947 2014-08-13 17:38:02.760065 7fe9bae887a0 20 read_log 192803'16425 (192803'16424) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:258 2014-08-11 12:53:25.502137 2014-08-13 17:38:02.760079 7fe9bae887a0 20 read_log 192803'16426 (192803'16425) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:259 2014-08-11 12:53:25.543867 2014-08-13 17:38:02.760091 7fe9bae887a0 20 read_log 192803'16427 (192803'16426) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:260 2014-08-11 12:53:25.569796 2014-08-13 17:38:02.760104 7fe9bae887a0 20 read_log 192803'16428 (192803'16427) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:261 2014-08-11 12:53:25.641692 2014-08-13 17:38:02.760116 7fe9bae887a0 20 read_log 192803'16429 (192803'16428) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:262 2014-08-11 12:53:26.046736 2014-08-13 17:38:02.760128 7fe9bae887a0 20 read_log 192803'16430 (192803'16429) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:263 2014-08-11 12:53:26.084894 2014-08-13 17:38:02.760140 7fe9bae887a0 20 read_log 192803'16431 (192803'16430) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:264 2014-08-11 12:53:26.119832 2014-08-13 17:38:02.760161 7fe9bae887a0 20 read_log 192803'16432 (192803'16431) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:265 2014-08-11 12:53:26.185824 2014-08-13 17:38:02.760175 7fe9bae887a0 20 read_log 192803'16433 (192803'16432) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:266 2014-08-11 12:53:26.213560 2014-08-13 17:38:02.760197 7fe9bae887a0 20 read_log 192803'16434 (192803'16433) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:267 2014-08-11 12:53:26.242913 2014-08-13 17:38:02.760210 7fe9bae887a0 20 read_log 192803'16435 (192803'16434) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:268 2014-08-11 12:53:26.270172 2014-08-13 17:38:02.760222 7fe9bae887a0 20 read_log 192803'16436 (192803'16435) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:269 2014-08-11 12:53:26.299784 2014-08-13 17:38:02.760235 7fe9bae887a0 20 read_log 192803'16437 (192803'16436) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:270 2014-08-11 12:53:26.330345 2014-08-13 17:38:02.760247 7fe9bae887a0 20 read_log 192803'16438 (192803'16437) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:271 2014-08-11 12:53:26.360397 2014-08-13 17:38:02.760259 7fe9bae887a0 20 read_log 192803'16439 (192803'16438) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:272 2014-08-11 12:53:26.739140 2014-08-13 17:38:02.760270 7fe9bae887a0 20 read_log 192803'16440 (192803'16439) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:273 2014-08-11 12:53:26.773128 2014-08-13 17:38:02.760282 7fe9bae887a0 20 read_log 192803'16441 (192803'16440) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:274 2014-08-11 12:53:26.808074 2014-08-13 17:38:02.760294 7fe9bae887a0 20 read_log 192803'16442 (192803'16441) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:275 2014-08-11 12:53:26.868398 2014-08-13 17:38:02.760306 7fe9bae887a0 20 read_log 192803'16443 (192803'16442) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:276 2014-08-11 12:53:26.895765 2014-08-13 17:38:02.760317 7fe9bae887a0 20 read_log 192803'16444 (192803'16443) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:277 2014-08-11 12:53:26.962335 2014-08-13 17:38:02.760329 7fe9bae887a0 20 read_log 192803'16445 (192803'16444) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:278 2014-08-11 12:53:26.989190 2014-08-13 17:38:02.760341 7fe9bae887a0 20 read_log 192803'16446 (192803'16445) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:279 2014-08-11 12:53:27.077046 2014-08-13 17:38:02.760352 7fe9bae887a0 20 read_log 192803'16447 (192803'16446) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:280 2014-08-11 12:53:27.101465 2014-08-13 17:38:02.760364 7fe9bae887a0 20 read_log 192803'16448 (192803'16447) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:281 2014-08-11 12:53:27.125234 2014-08-13 17:38:02.760377 7fe9bae887a0 20 read_log 192803'16449 (192803'16448) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:282 2014-08-11 12:53:27.479988 2014-08-13 17:38:02.760388 7fe9bae887a0 20 read_log 192803'16450 (192803'16449) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:283 2014-08-11 12:53:27.538797 2014-08-13 17:38:02.760400 7fe9bae887a0 20 read_log 192803'16451 (192803'16450) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:284 2014-08-11 12:53:27.567548 2014-08-13 17:38:02.760412 7fe9bae887a0 20 read_log 192803'16452 (192803'16451) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:285 2014-08-11 12:53:27.597271 2014-08-13 17:38:02.760424 7fe9bae887a0 20 read_log 192803'16453 (192803'16452) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:286 2014-08-11 12:53:27.634784 2014-08-13 17:38:02.760436 7fe9bae887a0 20 read_log 192803'16454 (192803'16453) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:287 2014-08-11 12:53:27.672494 2014-08-13 17:38:02.760447 7fe9bae887a0 20 read_log 192803'16455 (192803'16454) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:288 2014-08-11 12:53:27.710214 2014-08-13 17:38:02.760459 7fe9bae887a0 20 read_log 192803'16456 (192803'16455) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:289 2014-08-11 12:53:27.747569 2014-08-13 17:38:02.760471 7fe9bae887a0 20 read_log 192803'16457 (192803'16456) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:290 2014-08-11 12:53:27.785167 2014-08-13 17:38:02.760506 7fe9bae887a0 20 read_log 192803'16458 (192803'16457) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:291 2014-08-11 12:53:27.837986 2014-08-13 17:38:02.760523 7fe9bae887a0 20 read_log 192803'16459 (192803'16458) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:292 2014-08-11 12:53:28.236914 2014-08-13 17:38:02.760535 7fe9bae887a0 20 read_log 192803'16460 (192803'16459) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:293 2014-08-11 12:53:28.305931 2014-08-13 17:38:02.760547 7fe9bae887a0 20 read_log 192803'16461 (192803'16460) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:294 2014-08-11 12:53:28.332493 2014-08-13 17:38:02.760559 7fe9bae887a0 20 read_log 192803'16462 (192803'16461) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:295 2014-08-11 12:53:28.398614 2014-08-13 17:38:02.760570 7fe9bae887a0 20 read_log 192803'16463 (192803'16462) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:296 2014-08-11 12:53:28.425509 2014-08-13 17:38:02.760582 7fe9bae887a0 20 read_log 192803'16464 (192803'16463) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:297 2014-08-11 12:53:28.483435 2014-08-13 17:38:02.760594 7fe9bae887a0 20 read_log 192803'16465 (192803'16464) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:298 2014-08-11 12:53:28.509832 2014-08-13 17:38:02.760606 7fe9bae887a0 20 read_log 192803'16466 (192803'16465) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:299 2014-08-11 12:53:28.540004 2014-08-13 17:38:02.760618 7fe9bae887a0 20 read_log 192803'16467 (192803'16466) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:300 2014-08-11 12:53:28.588369 2014-08-13 17:38:02.760629 7fe9bae887a0 20 read_log 192803'16468 (192803'16467) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:301 2014-08-11 12:53:28.625029 2014-08-13 17:38:02.760643 7fe9bae887a0 20 read_log 192803'16469 (192803'16468) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:302 2014-08-11 12:53:28.971942 2014-08-13 17:38:02.760654 7fe9bae887a0 20 read_log 192803'16470 (192803'16469) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:303 2014-08-11 12:53:29.028031 2014-08-13 17:38:02.760666 7fe9bae887a0 20 read_log 192803'16471 (192803'16470) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:304 2014-08-11 12:53:29.055078 2014-08-13 17:38:02.760678 7fe9bae887a0 20 read_log 192803'16472 (192803'16471) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:305 2014-08-11 12:53:29.084226 2014-08-13 17:38:02.760690 7fe9bae887a0 20 read_log 192803'16473 (192803'16472) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:306 2014-08-11 12:53:29.121412 2014-08-13 17:38:02.760702 7fe9bae887a0 20 read_log 192803'16474 (192803'16473) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:307 2014-08-11 12:53:29.177879 2014-08-13 17:38:02.760714 7fe9bae887a0 20 read_log 192803'16475 (192803'16474) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:308 2014-08-11 12:53:29.204719 2014-08-13 17:38:02.760725 7fe9bae887a0 20 read_log 192803'16476 (192803'16475) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:309 2014-08-11 12:53:29.265051 2014-08-13 17:38:02.760737 7fe9bae887a0 20 read_log 192803'16477 (192803'16476) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:310 2014-08-11 12:53:29.291941 2014-08-13 17:38:02.760749 7fe9bae887a0 20 read_log 192803'16478 (192803'16477) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:311 2014-08-11 12:53:29.349156 2014-08-13 17:38:02.760760 7fe9bae887a0 20 read_log 192803'16479 (192803'16478) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:312 2014-08-11 12:53:29.666842 2014-08-13 17:38:02.760772 7fe9bae887a0 20 read_log 192803'16480 (192803'16479) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:313 2014-08-11 12:53:29.706069 2014-08-13 17:38:02.760784 7fe9bae887a0 20 read_log 192803'16481 (192803'16480) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:314 2014-08-11 12:53:29.740756 2014-08-13 17:38:02.760801 7fe9bae887a0 20 read_log 192803'16482 (192803'16481) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:315 2014-08-11 12:53:29.775359 2014-08-13 17:38:02.760814 7fe9bae887a0 20 read_log 192803'16483 (192803'16482) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:316 2014-08-11 12:53:29.810787 2014-08-13 17:38:02.760826 7fe9bae887a0 20 read_log 192803'16484 (192803'16483) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:317 2014-08-11 12:53:29.845282 2014-08-13 17:38:02.760837 7fe9bae887a0 20 read_log 192803'16485 (192803'16484) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:318 2014-08-11 12:53:29.879908 2014-08-13 17:38:02.760849 7fe9bae887a0 20 read_log 192803'16486 (192803'16485) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:319 2014-08-11 12:53:29.936687 2014-08-13 17:38:02.760877 7fe9bae887a0 20 read_log 192803'16487 (192803'16486) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:320 2014-08-11 12:53:29.961242 2014-08-13 17:38:02.760892 7fe9bae887a0 20 read_log 192803'16488 (192803'16487) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:321 2014-08-11 12:53:30.024930 2014-08-13 17:38:02.760905 7fe9bae887a0 20 read_log 192803'16489 (192803'16488) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:322 2014-08-11 12:53:30.367427 2014-08-13 17:38:02.760916 7fe9bae887a0 20 read_log 192803'16490 (192803'16489) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:323 2014-08-11 12:53:30.484253 2014-08-13 17:38:02.760928 7fe9bae887a0 20 read_log 192803'16491 (192803'16490) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:324 2014-08-11 12:53:30.513044 2014-08-13 17:38:02.760940 7fe9bae887a0 20 read_log 192803'16492 (192803'16491) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:325 2014-08-11 12:53:30.539571 2014-08-13 17:38:02.760952 7fe9bae887a0 20 read_log 192803'16493 (192803'16492) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:326 2014-08-11 12:53:30.566766 2014-08-13 17:38:02.760964 7fe9bae887a0 20 read_log 192803'16494 (192803'16493) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:327 2014-08-11 12:53:30.599092 2014-08-13 17:38:02.760976 7fe9bae887a0 20 read_log 192803'16495 (192803'16494) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:328 2014-08-11 12:53:30.633685 2014-08-13 17:38:02.760987 7fe9bae887a0 20 read_log 192803'16496 (192803'16495) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:329 2014-08-11 12:53:30.696074 2014-08-13 17:38:02.761000 7fe9bae887a0 20 read_log 192803'16497 (192803'16496) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:330 2014-08-11 12:53:30.722874 2014-08-13 17:38:02.761012 7fe9bae887a0 20 read_log 192803'16498 (192803'16497) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:331 2014-08-11 12:53:30.753715 2014-08-13 17:38:02.761024 7fe9bae887a0 20 read_log 192803'16499 (192803'16498) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:332 2014-08-11 12:53:30.778953 2014-08-13 17:38:02.761036 7fe9bae887a0 20 read_log 192803'16500 (192803'16499) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:333 2014-08-11 12:53:30.807231 2014-08-13 17:38:02.761048 7fe9bae887a0 20 read_log 192803'16501 (192803'16500) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:334 2014-08-11 12:53:31.224180 2014-08-13 17:38:02.761059 7fe9bae887a0 20 read_log 192803'16502 (192803'16501) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:335 2014-08-11 12:53:31.292045 2014-08-13 17:38:02.761071 7fe9bae887a0 20 read_log 192803'16503 (192803'16502) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:336 2014-08-11 12:53:31.319679 2014-08-13 17:38:02.761083 7fe9bae887a0 20 read_log 192803'16504 (192803'16503) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:337 2014-08-11 12:53:31.346848 2014-08-13 17:38:02.761095 7fe9bae887a0 20 read_log 192803'16505 (192803'16504) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:338 2014-08-11 12:53:31.374212 2014-08-13 17:38:02.761113 7fe9bae887a0 20 read_log 192803'16506 (192803'16505) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:339 2014-08-11 12:53:31.402027 2014-08-13 17:38:02.761125 7fe9bae887a0 20 read_log 192803'16507 (192803'16506) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:340 2014-08-11 12:53:31.433271 2014-08-13 17:38:02.761138 7fe9bae887a0 20 read_log 192803'16508 (192803'16507) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:341 2014-08-11 12:53:31.469874 2014-08-13 17:38:02.761149 7fe9bae887a0 20 read_log 192803'16509 (192803'16508) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:342 2014-08-11 12:53:31.507228 2014-08-13 17:38:02.761169 7fe9bae887a0 20 read_log 192803'16510 (192803'16509) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:343 2014-08-11 12:53:31.541046 2014-08-13 17:38:02.761181 7fe9bae887a0 20 read_log 192803'16511 (192803'16510) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:344 2014-08-11 12:53:31.923304 2014-08-13 17:38:02.761193 7fe9bae887a0 20 read_log 192803'16512 (192803'16511) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:345 2014-08-11 12:53:31.957073 2014-08-13 17:38:02.761205 7fe9bae887a0 20 read_log 192803'16513 (192803'16512) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:346 2014-08-11 12:53:31.993315 2014-08-13 17:38:02.761217 7fe9bae887a0 20 read_log 192803'16514 (192803'16513) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:347 2014-08-11 12:53:32.028278 2014-08-13 17:38:02.761229 7fe9bae887a0 20 read_log 192803'16515 (192803'16514) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:348 2014-08-11 12:53:32.061342 2014-08-13 17:38:02.761241 7fe9bae887a0 20 read_log 192803'16516 (192803'16515) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:349 2014-08-11 12:53:32.123898 2014-08-13 17:38:02.761254 7fe9bae887a0 20 read_log 192803'16517 (192803'16516) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:350 2014-08-11 12:53:32.150519 2014-08-13 17:38:02.761266 7fe9bae887a0 20 read_log 192803'16518 (192803'16517) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:351 2014-08-11 12:53:32.210117 2014-08-13 17:38:02.761278 7fe9bae887a0 20 read_log 192803'16519 (192803'16518) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:352 2014-08-11 12:53:32.282192 2014-08-13 17:38:02.761290 7fe9bae887a0 20 read_log 192803'16520 (192803'16519) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:353 2014-08-11 12:53:32.361118 2014-08-13 17:38:02.761302 7fe9bae887a0 20 read_log 192803'16521 (192803'16520) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:354 2014-08-11 12:53:32.615312 2014-08-13 17:38:02.761313 7fe9bae887a0 20 read_log 192803'16522 (192803'16521) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:355 2014-08-11 12:53:32.709928 2014-08-13 17:38:02.761325 7fe9bae887a0 20 read_log 192803'16523 (192803'16522) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:356 2014-08-11 12:53:32.737643 2014-08-13 17:38:02.761337 7fe9bae887a0 20 read_log 192803'16524 (192803'16523) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:357 2014-08-11 12:53:32.768143 2014-08-13 17:38:02.761349 7fe9bae887a0 20 read_log 192803'16525 (192803'16524) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:358 2014-08-11 12:53:32.794562 2014-08-13 17:38:02.761383 7fe9bae887a0 20 read_log 192803'16526 (192803'16525) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:359 2014-08-11 12:53:32.832581 2014-08-13 17:38:02.761395 7fe9bae887a0 20 read_log 192803'16527 (192803'16526) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:360 2014-08-11 12:53:32.862235 2014-08-13 17:38:02.761407 7fe9bae887a0 20 read_log 192803'16528 (192803'16527) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:361 2014-08-11 12:53:32.919707 2014-08-13 17:38:02.761419 7fe9bae887a0 20 read_log 192803'16529 (192803'16528) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:362 2014-08-11 12:53:32.942030 2014-08-13 17:38:02.761440 7fe9bae887a0 20 read_log 192803'16530 (192803'16529) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:363 2014-08-11 12:53:33.010769 2014-08-13 17:38:02.761455 7fe9bae887a0 20 read_log 192803'16531 (192803'16530) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:364 2014-08-11 12:53:33.312443 2014-08-13 17:38:02.761467 7fe9bae887a0 20 read_log 192803'16532 (192803'16531) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:365 2014-08-11 12:53:33.383666 2014-08-13 17:38:02.761479 7fe9bae887a0 20 read_log 192803'16533 (192803'16532) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:366 2014-08-11 12:53:33.410718 2014-08-13 17:38:02.761495 7fe9bae887a0 20 read_log 192803'16534 (192803'16533) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:367 2014-08-11 12:53:33.434195 2014-08-13 17:38:02.761506 7fe9bae887a0 20 read_log 192803'16535 (192803'16534) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:368 2014-08-11 12:53:33.458830 2014-08-13 17:38:02.761518 7fe9bae887a0 20 read_log 192803'16536 (192803'16535) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:369 2014-08-11 12:53:33.511937 2014-08-13 17:38:02.761530 7fe9bae887a0 20 read_log 192803'16537 (192803'16536) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:370 2014-08-11 12:53:33.538529 2014-08-13 17:38:02.761542 7fe9bae887a0 20 read_log 192803'16538 (192803'16537) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:371 2014-08-11 12:53:33.567269 2014-08-13 17:38:02.761553 7fe9bae887a0 20 read_log 192803'16539 (192803'16538) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:372 2014-08-11 12:53:33.596348 2014-08-13 17:38:02.761565 7fe9bae887a0 20 read_log 192803'16540 (192803'16539) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:373 2014-08-11 12:53:33.631163 2014-08-13 17:38:02.761577 7fe9bae887a0 20 read_log 192803'16541 (192803'16540) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:374 2014-08-11 12:53:34.049335 2014-08-13 17:38:02.761589 7fe9bae887a0 20 read_log 192803'16542 (192803'16541) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:375 2014-08-11 12:53:34.095147 2014-08-13 17:38:02.761601 7fe9bae887a0 20 read_log 192803'16543 (192803'16542) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:376 2014-08-11 12:53:34.130600 2014-08-13 17:38:02.761612 7fe9bae887a0 20 read_log 192803'16544 (192803'16543) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:377 2014-08-11 12:53:34.172082 2014-08-13 17:38:02.761624 7fe9bae887a0 20 read_log 192803'16545 (192803'16544) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:378 2014-08-11 12:53:34.212153 2014-08-13 17:38:02.761636 7fe9bae887a0 20 read_log 192803'16546 (192803'16545) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:379 2014-08-11 12:53:34.253384 2014-08-13 17:38:02.761648 7fe9bae887a0 20 read_log 192803'16547 (192803'16546) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:380 2014-08-11 12:53:34.299015 2014-08-13 17:38:02.761660 7fe9bae887a0 20 read_log 192803'16548 (192803'16547) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:381 2014-08-11 12:53:34.334693 2014-08-13 17:38:02.761672 7fe9bae887a0 20 read_log 192803'16549 (192803'16548) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:382 2014-08-11 12:53:34.376149 2014-08-13 17:38:02.761684 7fe9bae887a0 20 read_log 192803'16550 (192803'16549) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:383 2014-08-11 12:53:34.432228 2014-08-13 17:38:02.761695 7fe9bae887a0 20 read_log 192803'16551 (192803'16550) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:384 2014-08-11 12:53:34.881279 2014-08-13 17:38:02.761707 7fe9bae887a0 20 read_log 192803'16552 (192803'16551) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:385 2014-08-11 12:53:34.931583 2014-08-13 17:38:02.761719 7fe9bae887a0 20 read_log 192803'16553 (192803'16552) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:386 2014-08-11 12:53:34.975576 2014-08-13 17:38:02.761737 7fe9bae887a0 20 read_log 192803'16554 (192803'16553) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:387 2014-08-11 12:53:35.022712 2014-08-13 17:38:02.761749 7fe9bae887a0 20 read_log 192803'16555 (192803'16554) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:388 2014-08-11 12:53:35.073581 2014-08-13 17:38:02.761761 7fe9bae887a0 20 read_log 192803'16556 (192803'16555) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:389 2014-08-11 12:53:35.125382 2014-08-13 17:38:02.761773 7fe9bae887a0 20 read_log 192803'16557 (192803'16556) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:390 2014-08-11 12:53:35.176404 2014-08-13 17:38:02.761785 7fe9bae887a0 20 read_log 192803'16558 (192803'16557) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:391 2014-08-11 12:53:35.228628 2014-08-13 17:38:02.761797 7fe9bae887a0 20 read_log 192803'16559 (192803'16558) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:392 2014-08-11 12:53:35.279046 2014-08-13 17:38:02.761809 7fe9bae887a0 20 read_log 192803'16560 (192803'16559) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:393 2014-08-11 12:53:35.326187 2014-08-13 17:38:02.761821 7fe9bae887a0 20 read_log 192803'16561 (192803'16560) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:394 2014-08-11 12:53:35.526613 2014-08-13 17:38:02.761833 7fe9bae887a0 20 read_log 192803'16562 (192803'16561) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:395 2014-08-11 12:53:35.581129 2014-08-13 17:38:02.761844 7fe9bae887a0 20 read_log 192803'16563 (192803'16562) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:396 2014-08-11 12:53:35.632092 2014-08-13 17:38:02.761856 7fe9bae887a0 20 read_log 192803'16564 (192803'16563) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:397 2014-08-11 12:53:35.679020 2014-08-13 17:38:02.761868 7fe9bae887a0 20 read_log 192803'16565 (192803'16564) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:398 2014-08-11 12:53:35.729642 2014-08-13 17:38:02.761880 7fe9bae887a0 20 read_log 192803'16566 (192803'16565) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:399 2014-08-11 12:53:35.778043 2014-08-13 17:38:02.761891 7fe9bae887a0 20 read_log 192803'16567 (192803'16566) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:400 2014-08-11 12:53:35.828791 2014-08-13 17:38:02.761903 7fe9bae887a0 20 read_log 192803'16568 (192803'16567) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:401 2014-08-11 12:53:35.881548 2014-08-13 17:38:02.761917 7fe9bae887a0 20 read_log 192803'16569 (192803'16568) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:402 2014-08-11 12:53:35.930785 2014-08-13 17:38:02.761929 7fe9bae887a0 20 read_log 192803'16570 (192803'16569) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:403 2014-08-11 12:53:35.974872 2014-08-13 17:38:02.761941 7fe9bae887a0 20 read_log 192803'16571 (192803'16570) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:404 2014-08-11 12:53:36.024446 2014-08-13 17:38:02.761953 7fe9bae887a0 20 read_log 192803'16572 (192803'16571) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:405 2014-08-11 12:53:36.115408 2014-08-13 17:38:02.761965 7fe9bae887a0 20 read_log 192803'16573 (192803'16572) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:406 2014-08-11 12:53:36.160387 2014-08-13 17:38:02.761977 7fe9bae887a0 20 read_log 192803'16574 (192803'16573) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:407 2014-08-11 12:53:36.188270 2014-08-13 17:38:02.761989 7fe9bae887a0 20 read_log 192803'16575 (192803'16574) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:408 2014-08-11 12:53:36.216079 2014-08-13 17:38:02.762000 7fe9bae887a0 20 read_log 192803'16576 (192803'16575) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:409 2014-08-11 12:53:36.249294 2014-08-13 17:38:02.762012 7fe9bae887a0 20 read_log 192803'16577 (192803'16576) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:410 2014-08-11 12:53:36.276497 2014-08-13 17:38:02.762029 7fe9bae887a0 20 read_log 192803'16578 (192803'16577) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:411 2014-08-11 12:53:36.311052 2014-08-13 17:38:02.762042 7fe9bae887a0 20 read_log 192803'16579 (192803'16578) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:412 2014-08-11 12:53:36.361850 2014-08-13 17:38:02.762072 7fe9bae887a0 20 read_log 192803'16580 (192803'16579) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:413 2014-08-11 12:53:36.410681 2014-08-13 17:38:02.762088 7fe9bae887a0 20 read_log 192803'16581 (192803'16580) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:414 2014-08-11 12:53:36.459123 2014-08-13 17:38:02.762100 7fe9bae887a0 20 read_log 192803'16582 (192803'16581) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:415 2014-08-11 12:53:36.642107 2014-08-13 17:38:02.762112 7fe9bae887a0 20 read_log 192803'16583 (192803'16582) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:416 2014-08-11 12:53:36.692681 2014-08-13 17:38:02.762123 7fe9bae887a0 20 read_log 192803'16584 (192803'16583) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:417 2014-08-11 12:53:36.744354 2014-08-13 17:38:02.762135 7fe9bae887a0 20 read_log 192803'16585 (192803'16584) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:418 2014-08-11 12:53:36.794536 2014-08-13 17:38:02.762147 7fe9bae887a0 20 read_log 192803'16586 (192803'16585) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:419 2014-08-11 12:53:36.842157 2014-08-13 17:38:02.762164 7fe9bae887a0 20 read_log 192803'16587 (192803'16586) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:420 2014-08-11 12:53:36.892743 2014-08-13 17:38:02.762176 7fe9bae887a0 20 read_log 192803'16588 (192803'16587) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:421 2014-08-11 12:53:36.943028 2014-08-13 17:38:02.762189 7fe9bae887a0 20 read_log 192803'16589 (192803'16588) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:422 2014-08-11 12:53:36.989402 2014-08-13 17:38:02.762201 7fe9bae887a0 20 read_log 192803'16590 (192803'16589) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:423 2014-08-11 12:53:37.037547 2014-08-13 17:38:02.762212 7fe9bae887a0 20 read_log 192803'16591 (192803'16590) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:424 2014-08-11 12:53:37.105272 2014-08-13 17:38:02.762224 7fe9bae887a0 20 read_log 192803'16592 (192803'16591) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:425 2014-08-11 12:53:37.164887 2014-08-13 17:38:02.762237 7fe9bae887a0 20 read_log 192803'16593 (192803'16592) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:426 2014-08-11 12:53:37.217583 2014-08-13 17:38:02.762249 7fe9bae887a0 20 read_log 192803'16594 (192803'16593) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:427 2014-08-11 12:53:37.269170 2014-08-13 17:38:02.762261 7fe9bae887a0 20 read_log 192803'16595 (192803'16594) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:428 2014-08-11 12:53:37.323100 2014-08-13 17:38:02.762273 7fe9bae887a0 20 read_log 192803'16596 (192803'16595) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:429 2014-08-11 12:53:37.372437 2014-08-13 17:38:02.762285 7fe9bae887a0 20 read_log 192803'16597 (192803'16596) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:430 2014-08-11 12:53:37.424077 2014-08-13 17:38:02.762296 7fe9bae887a0 20 read_log 192803'16598 (192803'16597) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:431 2014-08-11 12:53:37.470770 2014-08-13 17:38:02.762308 7fe9bae887a0 20 read_log 192803'16599 (192803'16598) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:432 2014-08-11 12:53:37.513613 2014-08-13 17:38:02.762320 7fe9bae887a0 20 read_log 192803'16600 (192803'16599) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:433 2014-08-11 12:53:37.557722 2014-08-13 17:38:02.762331 7fe9bae887a0 20 read_log 192803'16601 (192803'16600) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:434 2014-08-11 12:53:37.601142 2014-08-13 17:38:02.762352 7fe9bae887a0 20 read_log 192803'16602 (192803'16601) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:435 2014-08-11 12:53:37.776787 2014-08-13 17:38:02.762364 7fe9bae887a0 20 read_log 192803'16603 (192803'16602) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:436 2014-08-11 12:53:37.826003 2014-08-13 17:38:02.762376 7fe9bae887a0 20 read_log 192803'16604 (192803'16603) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:437 2014-08-11 12:53:37.848408 2014-08-13 17:38:02.762388 7fe9bae887a0 20 read_log 192803'16605 (192803'16604) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:438 2014-08-11 12:53:37.872853 2014-08-13 17:38:02.762400 7fe9bae887a0 20 read_log 192803'16606 (192803'16605) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:439 2014-08-11 12:53:37.907468 2014-08-13 17:38:02.762412 7fe9bae887a0 20 read_log 192803'16607 (192803'16606) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:440 2014-08-11 12:53:37.935821 2014-08-13 17:38:02.762424 7fe9bae887a0 20 read_log 192803'16608 (192803'16607) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:441 2014-08-11 12:53:37.970043 2014-08-13 17:38:02.762435 7fe9bae887a0 20 read_log 192803'16609 (192803'16608) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:442 2014-08-11 12:53:37.998061 2014-08-13 17:38:02.762447 7fe9bae887a0 20 read_log 192803'16610 (192803'16609) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:443 2014-08-11 12:53:38.025500 2014-08-13 17:38:02.762459 7fe9bae887a0 20 read_log 192803'16611 (192803'16610) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:444 2014-08-11 12:53:38.060105 2014-08-13 17:38:02.762471 7fe9bae887a0 20 read_log 192803'16612 (192803'16611) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:445 2014-08-11 12:53:38.371026 2014-08-13 17:38:02.762482 7fe9bae887a0 20 read_log 192803'16613 (192803'16612) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:446 2014-08-11 12:53:38.402621 2014-08-13 17:38:02.762494 7fe9bae887a0 20 read_log 192803'16614 (192803'16613) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:447 2014-08-11 12:53:38.440522 2014-08-13 17:38:02.762506 7fe9bae887a0 20 read_log 192803'16615 (192803'16614) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:448 2014-08-11 12:53:38.468552 2014-08-13 17:38:02.762518 7fe9bae887a0 20 read_log 192803'16616 (192803'16615) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:449 2014-08-11 12:53:38.500067 2014-08-13 17:38:02.762529 7fe9bae887a0 20 read_log 192803'16617 (192803'16616) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:450 2014-08-11 12:53:38.526997 2014-08-13 17:38:02.762541 7fe9bae887a0 20 read_log 192803'16618 (192803'16617) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:451 2014-08-11 12:53:38.555151 2014-08-13 17:38:02.762555 7fe9bae887a0 20 read_log 192803'16619 (192803'16618) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:452 2014-08-11 12:53:38.589299 2014-08-13 17:38:02.762567 7fe9bae887a0 20 read_log 192803'16620 (192803'16619) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:453 2014-08-11 12:53:38.617638 2014-08-13 17:38:02.762579 7fe9bae887a0 20 read_log 192803'16621 (192803'16620) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:454 2014-08-11 12:53:38.652390 2014-08-13 17:38:02.762591 7fe9bae887a0 20 read_log 192803'16622 (192803'16621) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:455 2014-08-11 12:53:38.976666 2014-08-13 17:38:02.762602 7fe9bae887a0 20 read_log 192803'16623 (192803'16622) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:456 2014-08-11 12:53:39.006223 2014-08-13 17:38:02.762614 7fe9bae887a0 20 read_log 192803'16624 (192803'16623) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:457 2014-08-11 12:53:39.043265 2014-08-13 17:38:02.762626 7fe9bae887a0 20 read_log 192803'16625 (192803'16624) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:458 2014-08-11 12:53:39.071519 2014-08-13 17:38:02.762661 7fe9bae887a0 20 read_log 192803'16626 (192803'16625) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:459 2014-08-11 12:53:39.104475 2014-08-13 17:38:02.762678 7fe9bae887a0 20 read_log 192803'16627 (192803'16626) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:460 2014-08-11 12:53:39.132011 2014-08-13 17:38:02.762690 7fe9bae887a0 20 read_log 192803'16628 (192803'16627) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:461 2014-08-11 12:53:39.159441 2014-08-13 17:38:02.762702 7fe9bae887a0 20 read_log 192803'16629 (192803'16628) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:462 2014-08-11 12:53:39.195114 2014-08-13 17:38:02.762715 7fe9bae887a0 20 read_log 192803'16630 (192803'16629) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:463 2014-08-11 12:53:39.223018 2014-08-13 17:38:02.762727 7fe9bae887a0 20 read_log 192803'16631 (192803'16630) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:464 2014-08-11 12:53:39.260784 2014-08-13 17:38:02.762742 7fe9bae887a0 20 read_log 192803'16632 (192803'16631) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:465 2014-08-11 12:53:39.514692 2014-08-13 17:38:02.762754 7fe9bae887a0 20 read_log 192803'16633 (192803'16632) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:466 2014-08-11 12:53:39.544621 2014-08-13 17:38:02.762766 7fe9bae887a0 20 read_log 192803'16634 (192803'16633) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:467 2014-08-11 12:53:39.576998 2014-08-13 17:38:02.762778 7fe9bae887a0 20 read_log 192803'16635 (192803'16634) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:468 2014-08-11 12:53:39.606095 2014-08-13 17:38:02.762789 7fe9bae887a0 20 read_log 192803'16636 (192803'16635) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:469 2014-08-11 12:53:39.639074 2014-08-13 17:38:02.762803 7fe9bae887a0 20 read_log 192803'16637 (192803'16636) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:470 2014-08-11 12:53:39.666701 2014-08-13 17:38:02.762815 7fe9bae887a0 20 read_log 192803'16638 (192803'16637) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:471 2014-08-11 12:53:39.694107 2014-08-13 17:38:02.762829 7fe9bae887a0 20 read_log 192803'16639 (192803'16638) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:472 2014-08-11 12:53:39.723358 2014-08-13 17:38:02.762841 7fe9bae887a0 20 read_log 192803'16640 (192803'16639) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:473 2014-08-11 12:53:39.750470 2014-08-13 17:38:02.762853 7fe9bae887a0 20 read_log 192803'16641 (192803'16640) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:474 2014-08-11 12:53:39.784669 2014-08-13 17:38:02.762867 7fe9bae887a0 20 read_log 192803'16642 (192803'16641) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:475 2014-08-11 12:53:40.062147 2014-08-13 17:38:02.762879 7fe9bae887a0 20 read_log 192803'16643 (192803'16642) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:476 2014-08-11 12:53:40.091191 2014-08-13 17:38:02.762893 7fe9bae887a0 20 read_log 192803'16644 (192803'16643) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:477 2014-08-11 12:53:40.138435 2014-08-13 17:38:02.762905 7fe9bae887a0 20 read_log 192803'16645 (192803'16644) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:478 2014-08-11 12:53:40.166306 2014-08-13 17:38:02.762919 7fe9bae887a0 20 read_log 192803'16646 (192803'16645) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:479 2014-08-11 12:53:40.201809 2014-08-13 17:38:02.762931 7fe9bae887a0 20 read_log 192803'16647 (192803'16646) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:480 2014-08-11 12:53:40.230629 2014-08-13 17:38:02.762943 7fe9bae887a0 20 read_log 192803'16648 (192803'16647) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:481 2014-08-11 12:53:40.259115 2014-08-13 17:38:02.762957 7fe9bae887a0 20 read_log 192803'16649 (192803'16648) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:482 2014-08-11 12:53:40.288605 2014-08-13 17:38:02.762975 7fe9bae887a0 20 read_log 192803'16650 (192803'16649) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:483 2014-08-11 12:53:40.311543 2014-08-13 17:38:02.762989 7fe9bae887a0 20 read_log 192803'16651 (192803'16650) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:484 2014-08-11 12:53:40.337840 2014-08-13 17:38:02.763001 7fe9bae887a0 20 read_log 192803'16652 (192803'16651) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:485 2014-08-11 12:53:40.604597 2014-08-13 17:38:02.763013 7fe9bae887a0 20 read_log 192803'16653 (192803'16652) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:486 2014-08-11 12:53:40.665086 2014-08-13 17:38:02.763027 7fe9bae887a0 20 read_log 192803'16654 (192803'16653) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:487 2014-08-11 12:53:40.702420 2014-08-13 17:38:02.763039 7fe9bae887a0 20 read_log 192803'16655 (192803'16654) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:488 2014-08-11 12:53:40.731522 2014-08-13 17:38:02.763051 7fe9bae887a0 20 read_log 192803'16656 (192803'16655) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:489 2014-08-11 12:53:40.760481 2014-08-13 17:38:02.763066 7fe9bae887a0 20 read_log 192803'16657 (192803'16656) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:490 2014-08-11 12:53:40.788894 2014-08-13 17:38:02.763081 7fe9bae887a0 20 read_log 192803'16658 (192803'16657) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:491 2014-08-11 12:53:40.811793 2014-08-13 17:38:02.763093 7fe9bae887a0 20 read_log 192803'16659 (192803'16658) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:492 2014-08-11 12:53:40.882334 2014-08-13 17:38:02.763105 7fe9bae887a0 20 read_log 192803'16660 (192803'16659) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:493 2014-08-11 12:53:40.922434 2014-08-13 17:38:02.763120 7fe9bae887a0 20 read_log 192803'16661 (192803'16660) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:494 2014-08-11 12:53:40.960210 2014-08-13 17:38:02.763131 7fe9bae887a0 20 read_log 192803'16662 (192803'16661) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:495 2014-08-11 12:53:41.019327 2014-08-13 17:38:02.763144 7fe9bae887a0 20 read_log 192803'16663 (192803'16662) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:496 2014-08-11 12:53:41.061599 2014-08-13 17:38:02.763161 7fe9bae887a0 20 read_log 192803'16664 (192803'16663) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:497 2014-08-11 12:53:41.560077 2014-08-13 17:38:02.763175 7fe9bae887a0 20 read_log 192803'16665 (192803'16664) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:498 2014-08-11 12:53:41.590114 2014-08-13 17:38:02.763188 7fe9bae887a0 20 read_log 192803'16666 (192803'16665) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:499 2014-08-11 12:53:41.627305 2014-08-13 17:38:02.763199 7fe9bae887a0 20 read_log 192803'16667 (192803'16666) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:500 2014-08-11 12:53:41.654869 2014-08-13 17:38:02.763211 7fe9bae887a0 20 read_log 192803'16668 (192803'16667) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:501 2014-08-11 12:53:41.687069 2014-08-13 17:38:02.763223 7fe9bae887a0 20 read_log 192803'16669 (192803'16668) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:502 2014-08-11 12:53:41.732012 2014-08-13 17:38:02.763238 7fe9bae887a0 20 read_log 192803'16670 (192803'16669) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:503 2014-08-11 12:53:41.759707 2014-08-13 17:38:02.763250 7fe9bae887a0 20 read_log 192803'16671 (192803'16670) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:504 2014-08-11 12:53:41.804005 2014-08-13 17:38:02.763263 7fe9bae887a0 20 read_log 192803'16672 (192803'16671) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:505 2014-08-11 12:53:41.831109 2014-08-13 17:38:02.763292 7fe9bae887a0 20 read_log 192803'16673 (192803'16672) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:506 2014-08-11 12:53:41.864509 2014-08-13 17:38:02.763314 7fe9bae887a0 20 read_log 192803'16674 (192803'16673) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:507 2014-08-11 12:53:42.273231 2014-08-13 17:38:02.763328 7fe9bae887a0 20 read_log 192803'16675 (192803'16674) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:508 2014-08-11 12:53:42.307259 2014-08-13 17:38:02.763340 7fe9bae887a0 20 read_log 192803'16676 (192803'16675) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:509 2014-08-11 12:53:42.346470 2014-08-13 17:38:02.763352 7fe9bae887a0 20 read_log 192803'16677 (192803'16676) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:510 2014-08-11 12:53:42.377963 2014-08-13 17:38:02.763364 7fe9bae887a0 20 read_log 192803'16678 (192803'16677) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:511 2014-08-11 12:53:42.413786 2014-08-13 17:38:02.763376 7fe9bae887a0 20 read_log 192803'16679 (192803'16678) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:512 2014-08-11 12:53:42.509080 2014-08-13 17:38:02.763388 7fe9bae887a0 20 read_log 192803'16680 (192803'16679) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:513 2014-08-11 12:53:42.562471 2014-08-13 17:38:02.763399 7fe9bae887a0 20 read_log 192803'16681 (192803'16680) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:514 2014-08-11 12:53:42.600365 2014-08-13 17:38:02.763411 7fe9bae887a0 20 read_log 192803'16682 (192803'16681) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:515 2014-08-11 12:53:42.635350 2014-08-13 17:38:02.763423 7fe9bae887a0 20 read_log 192803'16683 (192803'16682) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:516 2014-08-11 12:53:42.672246 2014-08-13 17:38:02.763437 7fe9bae887a0 20 read_log 192803'16684 (192803'16683) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:517 2014-08-11 12:53:43.082119 2014-08-13 17:38:02.763450 7fe9bae887a0 20 read_log 192803'16685 (192803'16684) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:518 2014-08-11 12:53:43.110917 2014-08-13 17:38:02.763462 7fe9bae887a0 20 read_log 192803'16686 (192803'16685) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:519 2014-08-11 12:53:43.150596 2014-08-13 17:38:02.763474 7fe9bae887a0 20 read_log 192803'16687 (192803'16686) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:520 2014-08-11 12:53:43.180390 2014-08-13 17:38:02.763487 7fe9bae887a0 20 read_log 192803'16688 (192803'16687) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:521 2014-08-11 12:53:43.216557 2014-08-13 17:38:02.763499 7fe9bae887a0 20 read_log 192803'16689 (192803'16688) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:522 2014-08-11 12:53:43.255843 2014-08-13 17:38:02.763511 7fe9bae887a0 20 read_log 192803'16690 (192803'16689) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:523 2014-08-11 12:53:43.297249 2014-08-13 17:38:02.763522 7fe9bae887a0 20 read_log 192803'16691 (192803'16690) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:524 2014-08-11 12:53:43.337742 2014-08-13 17:38:02.763534 7fe9bae887a0 20 read_log 192803'16692 (192803'16691) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:525 2014-08-11 12:53:43.372497 2014-08-13 17:38:02.763546 7fe9bae887a0 20 read_log 192803'16693 (192803'16692) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:526 2014-08-11 12:53:43.409087 2014-08-13 17:38:02.763557 7fe9bae887a0 20 read_log 192803'16694 (192803'16693) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:527 2014-08-11 12:53:43.819985 2014-08-13 17:38:02.763569 7fe9bae887a0 20 read_log 192803'16695 (192803'16694) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:528 2014-08-11 12:53:43.848431 2014-08-13 17:38:02.763581 7fe9bae887a0 20 read_log 192803'16696 (192803'16695) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:529 2014-08-11 12:53:43.898321 2014-08-13 17:38:02.763592 7fe9bae887a0 20 read_log 192803'16697 (192803'16696) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:530 2014-08-11 12:53:43.929545 2014-08-13 17:38:02.763612 7fe9bae887a0 20 read_log 192803'16698 (192803'16697) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:531 2014-08-11 12:53:43.966036 2014-08-13 17:38:02.763625 7fe9bae887a0 20 read_log 192803'16699 (192803'16698) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:532 2014-08-11 12:53:44.011612 2014-08-13 17:38:02.763637 7fe9bae887a0 20 read_log 192803'16700 (192803'16699) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:533 2014-08-11 12:53:44.039472 2014-08-13 17:38:02.763649 7fe9bae887a0 20 read_log 192803'16701 (192803'16700) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:534 2014-08-11 12:53:44.092048 2014-08-13 17:38:02.763661 7fe9bae887a0 20 read_log 192803'16702 (192803'16701) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:535 2014-08-11 12:53:44.121326 2014-08-13 17:38:02.763672 7fe9bae887a0 20 read_log 192803'16703 (192803'16702) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:536 2014-08-11 12:53:44.149746 2014-08-13 17:38:02.763684 7fe9bae887a0 20 read_log 192803'16704 (192803'16703) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:537 2014-08-11 12:53:44.566869 2014-08-13 17:38:02.763696 7fe9bae887a0 20 read_log 192803'16705 (192803'16704) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:538 2014-08-11 12:53:44.595685 2014-08-13 17:38:02.763708 7fe9bae887a0 20 read_log 192803'16706 (192803'16705) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:539 2014-08-11 12:53:44.635157 2014-08-13 17:38:02.763719 7fe9bae887a0 20 read_log 192803'16707 (192803'16706) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:540 2014-08-11 12:53:44.674059 2014-08-13 17:38:02.763731 7fe9bae887a0 20 read_log 192803'16708 (192803'16707) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:541 2014-08-11 12:53:44.703278 2014-08-13 17:38:02.763743 7fe9bae887a0 20 read_log 192803'16709 (192803'16708) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:542 2014-08-11 12:53:44.742132 2014-08-13 17:38:02.763755 7fe9bae887a0 20 read_log 192803'16710 (192803'16709) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:543 2014-08-11 12:53:44.775718 2014-08-13 17:38:02.763767 7fe9bae887a0 20 read_log 192803'16711 (192803'16710) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:544 2014-08-11 12:53:44.808810 2014-08-13 17:38:02.763779 7fe9bae887a0 20 read_log 192803'16712 (192803'16711) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:545 2014-08-11 12:53:44.851986 2014-08-13 17:38:02.763790 7fe9bae887a0 20 read_log 192803'16713 (192803'16712) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:546 2014-08-11 12:53:44.893668 2014-08-13 17:38:02.763802 7fe9bae887a0 20 read_log 192803'16714 (192803'16713) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:547 2014-08-11 12:53:45.298983 2014-08-13 17:38:02.763814 7fe9bae887a0 20 read_log 192803'16715 (192803'16714) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:548 2014-08-11 12:53:45.327193 2014-08-13 17:38:02.763827 7fe9bae887a0 20 read_log 192803'16716 (192803'16715) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:549 2014-08-11 12:53:45.357600 2014-08-13 17:38:02.763838 7fe9bae887a0 20 read_log 192803'16717 (192803'16716) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:550 2014-08-11 12:53:45.396183 2014-08-13 17:38:02.763850 7fe9bae887a0 20 read_log 192803'16718 (192803'16717) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:551 2014-08-11 12:53:45.424669 2014-08-13 17:38:02.763862 7fe9bae887a0 20 read_log 192803'16719 (192803'16718) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:552 2014-08-11 12:53:45.469147 2014-08-13 17:38:02.763873 7fe9bae887a0 20 read_log 192803'16720 (192803'16719) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:553 2014-08-11 12:53:45.493673 2014-08-13 17:38:02.763885 7fe9bae887a0 20 read_log 192803'16721 (192803'16720) clean 81bde03d/core.47650+10+2/head//24 by osd.4.0:18691 2014-08-11 12:55:06.554113 2014-08-13 17:38:02.763905 7fe9bae887a0 20 read_log 192803'16722 (192803'16721) delete 81bde03d/core.47650+10+2/head//24 by osd.4.0:18692 2014-08-11 12:55:09.349518 2014-08-13 17:38:02.763918 7fe9bae887a0 20 read_log 192834'16723 (192834'16723) modify 3d/hit_set_24.3d_archive_2014-08-11 12:52:38.941261_2014-08-11 15:23:36.570365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.570592 2014-08-13 17:38:02.763930 7fe9bae887a0 20 read_log 192834'16724 (192803'16165) delete 3d/hit_set_24.3d_archive_2014-08-11 10:24:43.584057_2014-08-11 12:52:38.940934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.570592 2014-08-13 17:38:02.763942 7fe9bae887a0 20 read_log 192834'16741 (0'0) promote 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by osd.4.0:18720 2014-08-11 15:23:39.328522 2014-08-13 17:38:02.763955 7fe9bae887a0 20 read_log 192834'16742 (192834'16741) modify 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.933448.0:958 2014-08-11 15:23:39.320690 2014-08-13 17:38:02.763967 7fe9bae887a0 20 read_log 192834'16787 (0'0) promote 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by osd.4.0:18766 2014-08-11 15:23:54.571565 2014-08-13 17:38:02.763979 7fe9bae887a0 20 read_log 192834'16788 (192834'16787) modify 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.933448.0:3932 2014-08-11 15:23:54.562641 2014-08-13 17:38:02.763993 7fe9bae887a0 20 read_log 192834'16791 (0'0) promote 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:18770 2014-08-11 15:23:54.945744 2014-08-13 17:38:02.764005 7fe9bae887a0 20 read_log 192834'16792 (192834'16791) modify 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by client.933448.0:3999 2014-08-11 15:23:54.938859 2014-08-13 17:38:02.764017 7fe9bae887a0 20 read_log 192834'16807 (0'0) promote 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:18786 2014-08-11 15:24:02.166372 2014-08-13 17:38:02.764029 7fe9bae887a0 20 read_log 192834'16808 (192834'16807) modify 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by client.933448.0:5597 2014-08-11 15:24:02.159339 2014-08-13 17:38:02.764042 7fe9bae887a0 20 read_log 192834'16829 (0'0) promote 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by osd.4.0:18808 2014-08-11 15:24:07.334638 2014-08-13 17:38:02.764054 7fe9bae887a0 20 read_log 192834'16830 (192834'16829) modify 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.933448.0:6927 2014-08-11 15:24:07.325617 2014-08-13 17:38:02.764066 7fe9bae887a0 20 read_log 192834'16839 (0'0) promote f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by osd.4.0:18818 2014-08-11 15:24:11.339202 2014-08-13 17:38:02.764078 7fe9bae887a0 20 read_log 192834'16840 (192834'16839) modify f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.933448.0:7770 2014-08-11 15:24:11.331914 2014-08-13 17:38:02.764090 7fe9bae887a0 20 read_log 192834'16885 (0'0) promote 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by osd.4.0:18864 2014-08-11 15:24:21.423659 2014-08-13 17:38:02.764102 7fe9bae887a0 20 read_log 192834'16886 (192834'16885) modify 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.933448.0:10047 2014-08-11 15:24:21.413849 2014-08-13 17:38:02.764114 7fe9bae887a0 20 read_log 192834'16887 (0'0) promote 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by osd.4.0:18866 2014-08-11 15:24:22.691548 2014-08-13 17:38:02.764126 7fe9bae887a0 20 read_log 192834'16888 (192834'16887) modify 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.933448.0:10332 2014-08-11 15:24:22.680116 2014-08-13 17:38:02.764145 7fe9bae887a0 20 read_log 192834'16889 (0'0) promote 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by osd.4.0:18868 2014-08-11 15:24:23.610327 2014-08-13 17:38:02.764163 7fe9bae887a0 20 read_log 192834'16890 (192834'16889) modify 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.933448.0:10526 2014-08-11 15:24:23.601067 2014-08-13 17:38:02.764176 7fe9bae887a0 20 read_log 192834'16901 (0'0) promote 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by osd.4.0:18880 2014-08-11 15:24:28.254812 2014-08-13 17:38:02.764188 7fe9bae887a0 20 read_log 192834'16902 (192834'16901) modify 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.933448.0:11548 2014-08-11 15:24:28.247538 2014-08-13 17:38:02.764200 7fe9bae887a0 20 read_log 192834'16959 (0'0) promote 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by osd.4.0:18938 2014-08-11 15:24:51.521230 2014-08-13 17:38:02.764212 7fe9bae887a0 20 read_log 192834'16960 (192834'16959) modify 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.933448.0:15838 2014-08-11 15:24:51.513156 2014-08-13 17:38:02.764224 7fe9bae887a0 20 read_log 192834'16979 (0'0) promote c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:18958 2014-08-11 15:24:59.312062 2014-08-13 17:38:02.764236 7fe9bae887a0 20 read_log 192834'16980 (192834'16979) modify c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by client.933448.0:17334 2014-08-11 15:24:59.304221 2014-08-13 17:38:02.764248 7fe9bae887a0 20 read_log 192834'16999 (0'0) promote 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by osd.4.0:18978 2014-08-11 15:25:06.272830 2014-08-13 17:38:02.764260 7fe9bae887a0 20 read_log 192834'17000 (192834'16999) modify 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.933448.0:18669 2014-08-11 15:25:06.263358 2014-08-13 17:38:02.764272 7fe9bae887a0 20 read_log 192834'17017 (0'0) promote ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:18996 2014-08-11 15:25:13.517677 2014-08-13 17:38:02.764284 7fe9bae887a0 20 read_log 192834'17018 (192834'17017) modify ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by client.933448.0:19809 2014-08-11 15:25:13.507884 2014-08-13 17:38:02.764296 7fe9bae887a0 20 read_log 192834'17025 (0'0) promote 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19004 2014-08-11 15:25:16.787107 2014-08-13 17:38:02.764307 7fe9bae887a0 20 read_log 192834'17026 (192834'17025) modify 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by client.933448.0:20139 2014-08-11 15:25:16.778486 2014-08-13 17:38:02.764319 7fe9bae887a0 20 read_log 192834'17063 (192834'16742) modify 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.933448.0:23249 2014-08-11 15:25:39.148826 2014-08-13 17:38:02.764331 7fe9bae887a0 20 read_log 192834'17086 (192834'16788) modify 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.933448.0:26223 2014-08-11 15:25:43.131286 2014-08-13 17:38:02.764343 7fe9bae887a0 20 read_log 192834'17088 (192834'16792) modify 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by client.933448.0:26290 2014-08-11 15:25:43.229811 2014-08-13 17:38:02.764355 7fe9bae887a0 20 read_log 192834'17096 (192834'16808) modify 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by client.933448.0:27888 2014-08-11 15:25:45.229875 2014-08-13 17:38:02.764367 7fe9bae887a0 20 read_log 192834'17107 (192834'16830) modify 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.933448.0:29218 2014-08-11 15:25:47.121765 2014-08-13 17:38:02.764380 7fe9bae887a0 20 read_log 192834'17112 (192834'16840) modify f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.933448.0:30061 2014-08-11 15:25:48.316419 2014-08-13 17:38:02.764402 7fe9bae887a0 20 read_log 192834'17135 (192834'16886) modify 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.933448.0:32338 2014-08-11 15:25:51.512288 2014-08-13 17:38:02.764415 7fe9bae887a0 20 read_log 192834'17136 (192834'16888) modify 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.933448.0:32623 2014-08-11 15:25:52.116867 2014-08-13 17:38:02.764429 7fe9bae887a0 20 read_log 192834'17137 (192834'16890) modify 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.933448.0:32817 2014-08-11 15:25:52.402448 2014-08-13 17:38:02.764441 7fe9bae887a0 20 read_log 192834'17143 (192834'16902) modify 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.933448.0:33839 2014-08-11 15:25:54.262153 2014-08-13 17:38:02.764453 7fe9bae887a0 20 read_log 192834'17172 (192834'16960) modify 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.933448.0:38129 2014-08-11 15:26:00.542281 2014-08-13 17:38:02.764482 7fe9bae887a0 20 read_log 192834'17182 (192834'16980) modify c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by client.933448.0:39625 2014-08-11 15:26:02.000371 2014-08-13 17:38:02.764495 7fe9bae887a0 20 read_log 192834'17192 (192834'17000) modify 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.933448.0:40960 2014-08-11 15:26:02.799318 2014-08-13 17:38:02.764507 7fe9bae887a0 20 read_log 192834'17201 (192834'17018) modify ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by client.933448.0:42100 2014-08-11 15:26:03.153625 2014-08-13 17:38:02.764521 7fe9bae887a0 20 read_log 192834'17205 (192834'17026) modify 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by client.933448.0:42430 2014-08-11 15:26:03.277304 2014-08-13 17:38:02.764533 7fe9bae887a0 20 read_log 192834'17222 (0'0) promote 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by osd.4.0:19201 2014-08-11 15:26:14.658560 2014-08-13 17:38:02.764545 7fe9bae887a0 20 read_log 192834'17223 (192834'17222) modify 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.1883634.0:97 2014-08-11 15:26:14.650082 2014-08-13 17:38:02.764558 7fe9bae887a0 20 read_log 192834'17226 (0'0) promote 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by osd.4.0:19205 2014-08-11 15:26:16.831773 2014-08-13 17:38:02.764570 7fe9bae887a0 20 read_log 192834'17227 (192834'17226) modify 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.1883634.0:642 2014-08-11 15:26:16.824046 2014-08-13 17:38:02.764583 7fe9bae887a0 20 read_log 192834'17258 (0'0) promote 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by osd.4.0:19237 2014-08-11 15:26:24.635586 2014-08-13 17:38:02.764595 7fe9bae887a0 20 read_log 192834'17259 (192834'17258) modify 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.1883634.0:2767 2014-08-11 15:26:24.626372 2014-08-13 17:38:02.764607 7fe9bae887a0 20 read_log 192834'17262 (0'0) promote d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by osd.4.0:19241 2014-08-11 15:26:26.394856 2014-08-13 17:38:02.764619 7fe9bae887a0 20 read_log 192834'17263 (192834'17262) modify d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.1883634.0:3264 2014-08-11 15:26:26.387203 2014-08-13 17:38:02.764631 7fe9bae887a0 20 read_log 192834'17274 (0'0) promote 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by osd.4.0:19253 2014-08-11 15:26:27.823878 2014-08-13 17:38:02.764643 7fe9bae887a0 20 read_log 192834'17275 (192834'17274) modify 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.1883634.0:3667 2014-08-11 15:26:27.814802 2014-08-13 17:38:02.764659 7fe9bae887a0 20 read_log 192834'17292 (0'0) promote 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19271 2014-08-11 15:26:32.750178 2014-08-13 17:38:02.764697 7fe9bae887a0 20 read_log 192834'17293 (192834'17292) modify 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.1883634.0:4968 2014-08-11 15:26:32.742056 2014-08-13 17:38:02.764714 7fe9bae887a0 20 read_log 192834'17318 (0'0) promote 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by osd.4.0:19297 2014-08-11 15:26:39.831754 2014-08-13 17:38:02.764726 7fe9bae887a0 20 read_log 192834'17319 (192834'17318) modify 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.1883634.0:6880 2014-08-11 15:26:39.820795 2014-08-13 17:38:02.764738 7fe9bae887a0 20 read_log 192834'17328 (0'0) promote 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by osd.4.0:19307 2014-08-11 15:26:41.446539 2014-08-13 17:38:02.764750 7fe9bae887a0 20 read_log 192834'17329 (192834'17328) modify 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.1883634.0:7270 2014-08-11 15:26:41.438433 2014-08-13 17:38:02.764762 7fe9bae887a0 20 read_log 192834'17332 (0'0) promote 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by osd.4.0:19311 2014-08-11 15:26:41.568606 2014-08-13 17:38:02.764774 7fe9bae887a0 20 read_log 192834'17333 (192834'17332) modify 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.1883634.0:7299 2014-08-11 15:26:41.560553 2014-08-13 17:38:02.764786 7fe9bae887a0 20 read_log 192834'17334 (0'0) promote 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by osd.4.0:19313 2014-08-11 15:26:41.769729 2014-08-13 17:38:02.764798 7fe9bae887a0 20 read_log 192834'17335 (192834'17334) modify 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.1883634.0:7341 2014-08-11 15:26:41.759457 2014-08-13 17:38:02.764810 7fe9bae887a0 20 read_log 192834'17352 (0'0) promote f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19331 2014-08-11 15:26:45.177189 2014-08-13 17:38:02.764823 7fe9bae887a0 20 read_log 192834'17353 (192834'17352) modify f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.1883634.0:8151 2014-08-11 15:26:45.165684 2014-08-13 17:38:02.764835 7fe9bae887a0 20 read_log 192834'17390 (0'0) promote 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19369 2014-08-11 15:26:52.691382 2014-08-13 17:38:02.764848 7fe9bae887a0 20 read_log 192834'17391 (192834'17390) modify 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.1883634.0:9853 2014-08-11 15:26:52.682402 2014-08-13 17:38:02.764860 7fe9bae887a0 20 read_log 192834'17456 (0'0) promote 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19435 2014-08-11 15:27:06.493918 2014-08-13 17:38:02.764872 7fe9bae887a0 20 read_log 192834'17457 (192834'17456) modify 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.1883634.0:12865 2014-08-11 15:27:06.484000 2014-08-13 17:38:02.764886 7fe9bae887a0 20 read_log 192834'17474 (0'0) promote 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by osd.4.0:19453 2014-08-11 15:27:09.886482 2014-08-13 17:38:02.764898 7fe9bae887a0 20 read_log 192834'17475 (192834'17474) modify 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.1883634.0:13497 2014-08-11 15:27:09.877448 2014-08-13 17:38:02.764910 7fe9bae887a0 20 read_log 192834'17486 (0'0) promote 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by osd.4.0:19465 2014-08-11 15:27:15.640401 2014-08-13 17:38:02.764922 7fe9bae887a0 20 read_log 192834'17487 (192834'17486) modify 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.1883634.0:14844 2014-08-11 15:27:15.632524 2014-08-13 17:38:02.764934 7fe9bae887a0 20 read_log 192834'17488 (0'0) promote 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by osd.4.0:19467 2014-08-11 15:27:16.884441 2014-08-13 17:38:02.764954 7fe9bae887a0 20 read_log 192834'17489 (192834'17488) modify 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.1883634.0:15065 2014-08-11 15:27:16.876357 2014-08-13 17:38:02.764967 7fe9bae887a0 20 read_log 192834'17504 (0'0) promote 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by osd.4.0:19483 2014-08-11 15:27:25.739966 2014-08-13 17:38:02.764979 7fe9bae887a0 20 read_log 192834'17505 (192834'17504) modify 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.1883634.0:16062 2014-08-11 15:27:25.730843 2014-08-13 17:38:02.764991 7fe9bae887a0 20 read_log 192834'17552 (0'0) promote e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by osd.4.0:19531 2014-08-11 15:27:46.594190 2014-08-13 17:38:02.765003 7fe9bae887a0 20 read_log 192834'17553 (192834'17552) modify e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.1883634.0:18587 2014-08-11 15:27:46.584688 2014-08-13 17:38:02.765015 7fe9bae887a0 20 read_log 192834'17588 (0'0) promote f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by osd.4.0:19567 2014-08-11 15:27:57.198999 2014-08-13 17:38:02.765027 7fe9bae887a0 20 read_log 192834'17589 (192834'17588) modify f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.1883634.0:20449 2014-08-11 15:27:57.190971 2014-08-13 17:38:02.765039 7fe9bae887a0 20 read_log 192834'17596 (0'0) promote 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by osd.4.0:19575 2014-08-11 15:28:01.957683 2014-08-13 17:38:02.765051 7fe9bae887a0 20 read_log 192834'17597 (192834'17596) modify 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.1883634.0:20904 2014-08-11 15:28:01.949410 2014-08-13 17:38:02.765063 7fe9bae887a0 20 read_log 192834'17604 (192834'17201) clean ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:19583 2014-08-11 15:28:04.379181 2014-08-13 17:38:02.765075 7fe9bae887a0 20 read_log 192834'17605 (192834'17182) clean c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:19584 2014-08-11 15:28:04.380338 2014-08-13 17:38:02.765088 7fe9bae887a0 20 read_log 192834'17606 (192834'17096) clean 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:19585 2014-08-11 15:28:04.386074 2014-08-13 17:38:02.765100 7fe9bae887a0 20 read_log 192834'17607 (192834'17205) clean 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19586 2014-08-11 15:28:04.414328 2014-08-13 17:38:02.765112 7fe9bae887a0 20 read_log 192834'17608 (192834'17088) clean 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:19587 2014-08-11 15:28:04.416848 2014-08-13 17:38:02.765125 7fe9bae887a0 20 read_log 192834'17609 (192834'17293) clean 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19588 2014-08-11 15:28:04.489517 2014-08-13 17:38:02.765137 7fe9bae887a0 20 read_log 192834'17610 (192834'17391) clean 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19589 2014-08-11 15:28:04.489825 2014-08-13 17:38:02.765150 7fe9bae887a0 20 read_log 192834'17611 (192834'17353) clean f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19590 2014-08-11 15:28:04.490938 2014-08-13 17:38:02.765167 7fe9bae887a0 20 read_log 192834'17624 (192834'17604) delete ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:19603 2014-08-11 15:28:05.387392 2014-08-13 17:38:02.765179 7fe9bae887a0 20 read_log 192834'17625 (192834'17605) delete c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:19604 2014-08-11 15:28:05.392149 2014-08-13 17:38:02.765191 7fe9bae887a0 20 read_log 192834'17626 (192834'17609) delete 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19605 2014-08-11 15:28:05.394309 2014-08-13 17:38:02.765213 7fe9bae887a0 20 read_log 192834'17629 (192834'17606) delete 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:19608 2014-08-11 15:28:05.399035 2014-08-13 17:38:02.765227 7fe9bae887a0 20 read_log 192834'17630 (192834'17607) delete 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19609 2014-08-11 15:28:05.401373 2014-08-13 17:38:02.765239 7fe9bae887a0 20 read_log 192834'17631 (192834'17610) delete 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19610 2014-08-11 15:28:05.401622 2014-08-13 17:38:02.765252 7fe9bae887a0 20 read_log 192834'17632 (192834'17608) delete 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:19611 2014-08-11 15:28:05.403853 2014-08-13 17:38:02.765264 7fe9bae887a0 20 read_log 192834'17633 (192834'17611) delete f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19612 2014-08-11 15:28:05.404107 2014-08-13 17:38:02.765276 7fe9bae887a0 20 read_log 192834'17650 (0'0) promote 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by osd.4.0:19629 2014-08-11 15:28:08.044681 2014-08-13 17:38:02.765305 7fe9bae887a0 20 read_log 192834'17651 (192834'17650) modify 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.1883634.0:22318 2014-08-11 15:28:08.035723 2014-08-13 17:38:02.765319 7fe9bae887a0 20 read_log 192834'17654 (0'0) promote 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by osd.4.0:19633 2014-08-11 15:28:11.779377 2014-08-13 17:38:02.765332 7fe9bae887a0 20 read_log 192834'17655 (192834'17654) modify 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.1883634.0:22932 2014-08-11 15:28:11.771636 2014-08-13 17:38:02.765345 7fe9bae887a0 20 read_log 192834'17660 (192834'17223) modify 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.1883634.0:23421 2014-08-11 15:28:16.311949 2014-08-13 17:38:02.765357 7fe9bae887a0 20 read_log 192834'17662 (192834'17457) clean 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19641 2014-08-11 15:28:16.395692 2014-08-13 17:38:02.765369 7fe9bae887a0 20 read_log 192834'17674 (192834'17662) delete 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19653 2014-08-11 15:28:16.821845 2014-08-13 17:38:02.765381 7fe9bae887a0 20 read_log 192834'17689 (192834'17227) modify 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.1883634.0:23966 2014-08-11 15:28:18.150896 2014-08-13 17:38:02.765393 7fe9bae887a0 20 read_log 192834'17706 (192834'17259) modify 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.1883634.0:26091 2014-08-11 15:28:22.601610 2014-08-13 17:38:02.765404 7fe9bae887a0 20 read_log 192834'17708 (192834'17263) modify d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.1883634.0:26588 2014-08-11 15:28:23.289083 2014-08-13 17:38:02.765416 7fe9bae887a0 20 read_log 192834'17716 (192834'17275) modify 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.1883634.0:26991 2014-08-11 15:28:23.871261 2014-08-13 17:38:02.765429 7fe9bae887a0 20 read_log 192834'17726 (0'0) promote 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19705 2014-08-11 15:28:25.643844 2014-08-13 17:38:02.765441 7fe9bae887a0 20 read_log 192834'17727 (192834'17726) modify 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.1883634.0:28292 2014-08-11 15:28:25.613858 2014-08-13 17:38:02.765453 7fe9bae887a0 20 read_log 192834'17741 (192834'17319) modify 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.1883634.0:30204 2014-08-11 15:28:28.223943 2014-08-13 17:38:02.765465 7fe9bae887a0 20 read_log 192834'17746 (192834'17329) modify 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.1883634.0:30594 2014-08-11 15:28:28.915971 2014-08-13 17:38:02.765484 7fe9bae887a0 20 read_log 192834'17748 (192834'17333) modify 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.1883634.0:30623 2014-08-11 15:28:28.994963 2014-08-13 17:38:02.765497 7fe9bae887a0 20 read_log 192834'17749 (192834'17335) modify 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.1883634.0:30665 2014-08-11 15:28:29.056985 2014-08-13 17:38:02.765509 7fe9bae887a0 20 read_log 192834'17760 (0'0) promote f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19739 2014-08-11 15:28:30.608070 2014-08-13 17:38:02.765521 7fe9bae887a0 20 read_log 192834'17761 (192834'17760) modify f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.1883634.0:31475 2014-08-11 15:28:30.581590 2014-08-13 17:38:02.765533 7fe9bae887a0 20 read_log 192834'17778 (192834'17778) modify 3d/hit_set_24.3d_archive_2014-08-11 15:23:36.570597_2014-08-11 15:28:32.651919/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:32.652610 2014-08-13 17:38:02.765545 7fe9bae887a0 20 read_log 192834'17779 (192834'16723) delete 3d/hit_set_24.3d_archive_2014-08-11 12:52:38.941261_2014-08-11 15:23:36.570365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:32.652610 2014-08-13 17:38:02.765558 7fe9bae887a0 20 read_log 192834'17783 (0'0) promote 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19761 2014-08-11 15:28:32.883091 2014-08-13 17:38:02.765570 7fe9bae887a0 20 read_log 192834'17784 (192834'17783) modify 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.1883634.0:33177 2014-08-11 15:28:32.855093 2014-08-13 17:38:02.765582 7fe9bae887a0 20 read_log 192834'17817 (0'0) promote 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19795 2014-08-11 15:28:37.730044 2014-08-13 17:38:02.765594 7fe9bae887a0 20 read_log 192834'17818 (192834'17817) modify 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.1883634.0:36189 2014-08-11 15:28:37.704734 2014-08-13 17:38:02.765606 7fe9bae887a0 20 read_log 192834'17827 (192834'17475) modify 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.1883634.0:36821 2014-08-11 15:28:38.451599 2014-08-13 17:38:02.765618 7fe9bae887a0 20 read_log 192834'17835 (192834'17487) modify 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.1883634.0:38168 2014-08-11 15:28:40.313681 2014-08-13 17:38:02.765630 7fe9bae887a0 20 read_log 192834'17836 (192834'17489) modify 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.1883634.0:38389 2014-08-11 15:28:40.572007 2014-08-13 17:38:02.765642 7fe9bae887a0 20 read_log 192834'17844 (192834'17505) modify 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.1883634.0:39386 2014-08-11 15:28:42.119497 2014-08-13 17:38:02.765673 7fe9bae887a0 20 read_log 192834'17868 (192834'17553) modify e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.1883634.0:41911 2014-08-11 15:28:45.573306 2014-08-13 17:38:02.765685 7fe9bae887a0 20 read_log 192834'17886 (192834'17589) modify f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.1883634.0:43773 2014-08-11 15:28:46.658664 2014-08-13 17:38:02.765698 7fe9bae887a0 20 read_log 192834'17890 (192834'17597) modify 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.1883634.0:44228 2014-08-11 15:28:46.799960 2014-08-13 17:38:02.765710 7fe9bae887a0 20 read_log 192834'17897 (192834'17651) modify 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.1883634.0:45642 2014-08-11 15:28:47.341577 2014-08-13 17:38:02.765741 7fe9bae887a0 20 read_log 192834'17899 (192834'17655) modify 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.1883634.0:46256 2014-08-11 15:28:47.538339 2014-08-13 17:38:02.765764 7fe9bae887a0 20 read_log 192888'17901 (192888'17901) modify 3d/hit_set_24.3d_archive_2014-08-11 16:02:02.114927_2014-08-11 16:44:56.253783/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.254066 2014-08-13 17:38:02.765780 7fe9bae887a0 20 read_log 192888'17902 (192834'17778) delete 3d/hit_set_24.3d_archive_2014-08-11 15:23:36.570597_2014-08-11 15:28:32.651919/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.254066 2014-08-13 17:38:02.765792 7fe9bae887a0 20 read_log 192888'17905 (0'0) promote c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by osd.4.0:21386 2014-08-11 16:44:56.463021 2014-08-13 17:38:02.765805 7fe9bae887a0 20 read_log 192888'17906 (192888'17905) modify c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.1315259.0:368 2014-08-11 16:44:56.454984 2014-08-13 17:38:02.765817 7fe9bae887a0 20 read_log 192888'17913 (0'0) promote 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by osd.4.0:21415 2014-08-11 16:45:00.433228 2014-08-13 17:38:02.765829 7fe9bae887a0 20 read_log 192888'17914 (192888'17913) modify 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.1315259.0:1355 2014-08-11 16:45:00.424399 2014-08-13 17:38:02.765841 7fe9bae887a0 20 read_log 192888'17915 (0'0) promote fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by osd.4.0:21419 2014-08-11 16:45:00.792426 2014-08-13 17:38:02.765854 7fe9bae887a0 20 read_log 192888'17916 (192888'17915) modify fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.1315259.0:1437 2014-08-11 16:45:00.781710 2014-08-13 17:38:02.765867 7fe9bae887a0 20 read_log 192888'17919 (0'0) promote 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by osd.4.0:21472 2014-08-11 16:45:06.785590 2014-08-13 17:38:02.765879 7fe9bae887a0 20 read_log 192888'17920 (192888'17919) modify 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.1315259.0:3051 2014-08-11 16:45:06.776409 2014-08-13 17:38:02.765891 7fe9bae887a0 20 read_log 192888'17925 (0'0) promote c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by osd.4.0:21524 2014-08-11 16:45:14.147077 2014-08-13 17:38:02.765903 7fe9bae887a0 20 read_log 192888'17926 (192888'17925) modify c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.1315259.0:5015 2014-08-11 16:45:14.139506 2014-08-13 17:38:02.765915 7fe9bae887a0 20 read_log 192888'17927 (0'0) promote 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by osd.4.0:21556 2014-08-11 16:45:17.240328 2014-08-13 17:38:02.765927 7fe9bae887a0 20 read_log 192888'17928 (192888'17927) modify 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.1315259.0:5848 2014-08-11 16:45:17.231647 2014-08-13 17:38:02.765939 7fe9bae887a0 20 read_log 192888'17933 (0'0) promote bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by osd.4.0:21594 2014-08-11 16:45:23.125534 2014-08-13 17:38:02.765951 7fe9bae887a0 20 read_log 192888'17934 (192888'17933) modify bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.1315259.0:7478 2014-08-11 16:45:23.117962 2014-08-13 17:38:02.765964 7fe9bae887a0 20 read_log 192888'17935 (0'0) promote d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by osd.4.0:21606 2014-08-11 16:45:26.040988 2014-08-13 17:38:02.765977 7fe9bae887a0 20 read_log 192888'17936 (192888'17935) modify d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.1315259.0:8276 2014-08-11 16:45:26.031534 2014-08-13 17:38:02.765989 7fe9bae887a0 20 read_log 192888'17941 (0'0) promote eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by osd.4.0:21658 2014-08-11 16:45:33.761590 2014-08-13 17:38:02.766010 7fe9bae887a0 20 read_log 192888'17942 (192888'17941) modify eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.1315259.0:10295 2014-08-11 16:45:33.752567 2014-08-13 17:38:02.766023 7fe9bae887a0 20 read_log 192888'17943 (0'0) promote b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by osd.4.0:21688 2014-08-11 16:45:36.712979 2014-08-13 17:38:02.766035 7fe9bae887a0 20 read_log 192888'17944 (192888'17943) modify b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.1315259.0:11024 2014-08-11 16:45:36.703544 2014-08-13 17:38:02.766047 7fe9bae887a0 20 read_log 192888'17945 (0'0) promote 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by osd.4.0:21692 2014-08-11 16:45:37.985590 2014-08-13 17:38:02.766059 7fe9bae887a0 20 read_log 192888'17946 (192888'17945) modify 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.1315259.0:11343 2014-08-11 16:45:37.977470 2014-08-13 17:38:02.766071 7fe9bae887a0 20 read_log 192888'17955 (0'0) promote 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by osd.4.0:21718 2014-08-11 16:45:41.594582 2014-08-13 17:38:02.766083 7fe9bae887a0 20 read_log 192888'17956 (192888'17955) modify 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.1315259.0:12232 2014-08-11 16:45:41.586100 2014-08-13 17:38:02.766095 7fe9bae887a0 20 read_log 192888'17957 (0'0) promote a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by osd.4.0:21740 2014-08-11 16:45:45.214273 2014-08-13 17:38:02.766107 7fe9bae887a0 20 read_log 192888'17958 (192888'17957) modify a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.1315259.0:13088 2014-08-11 16:45:45.204842 2014-08-13 17:38:02.766119 7fe9bae887a0 20 read_log 192888'17959 (0'0) promote 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by osd.4.0:21756 2014-08-11 16:45:48.138326 2014-08-13 17:38:02.766131 7fe9bae887a0 20 read_log 192888'17960 (192888'17959) modify 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.1315259.0:13829 2014-08-11 16:45:48.129634 2014-08-13 17:38:02.766143 7fe9bae887a0 20 read_log 192888'17961 (0'0) promote 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by osd.4.0:21758 2014-08-11 16:45:48.239976 2014-08-13 17:38:02.766160 7fe9bae887a0 20 read_log 192888'17962 (192888'17961) modify 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.1315259.0:13857 2014-08-11 16:45:48.233909 2014-08-13 17:38:02.766173 7fe9bae887a0 20 read_log 192888'17971 (0'0) promote 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by osd.4.0:21812 2014-08-11 16:45:56.239117 2014-08-13 17:38:02.766185 7fe9bae887a0 20 read_log 192888'17972 (192888'17971) modify 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.1315259.0:15949 2014-08-11 16:45:56.230164 2014-08-13 17:38:02.766197 7fe9bae887a0 20 read_log 192888'17973 (0'0) promote 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by osd.4.0:21814 2014-08-11 16:45:56.449710 2014-08-13 17:38:02.766209 7fe9bae887a0 20 read_log 192888'17974 (192888'17973) modify 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.1315259.0:16005 2014-08-11 16:45:56.440644 2014-08-13 17:38:02.766221 7fe9bae887a0 20 read_log 192888'17975 (0'0) promote 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by osd.4.0:21822 2014-08-11 16:45:57.098295 2014-08-13 17:38:02.766233 7fe9bae887a0 20 read_log 192888'17976 (192888'17975) modify 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.1315259.0:16176 2014-08-11 16:45:57.090281 2014-08-13 17:38:02.766245 7fe9bae887a0 20 read_log 192888'17977 (0'0) promote e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by osd.4.0:21826 2014-08-11 16:45:57.470932 2014-08-13 17:38:02.766264 7fe9bae887a0 20 read_log 192888'17978 (192888'17977) modify e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.1315259.0:16277 2014-08-11 16:45:57.462358 2014-08-13 17:38:02.766277 7fe9bae887a0 20 read_log 192888'17979 (0'0) promote 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by osd.4.0:21840 2014-08-11 16:45:59.898073 2014-08-13 17:38:02.766289 7fe9bae887a0 20 read_log 192888'17980 (192888'17979) modify 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.1315259.0:16893 2014-08-11 16:45:59.889699 2014-08-13 17:38:02.766301 7fe9bae887a0 20 read_log 192888'17981 (0'0) promote 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by osd.4.0:21850 2014-08-11 16:46:01.029567 2014-08-13 17:38:02.766313 7fe9bae887a0 20 read_log 192888'17982 (192888'17981) modify 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.1315259.0:17175 2014-08-11 16:46:01.020485 2014-08-13 17:38:02.766325 7fe9bae887a0 20 read_log 192888'17983 (0'0) promote ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by osd.4.0:21862 2014-08-11 16:46:02.242915 2014-08-13 17:38:02.766341 7fe9bae887a0 20 read_log 192888'17984 (192888'17983) modify ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.1315259.0:17482 2014-08-11 16:46:02.234908 2014-08-13 17:38:02.766353 7fe9bae887a0 20 read_log 192888'17985 (0'0) promote fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by osd.4.0:21914 2014-08-11 16:46:08.824924 2014-08-13 17:38:02.766365 7fe9bae887a0 20 read_log 192888'17986 (192888'17985) modify fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.1315259.0:19126 2014-08-11 16:46:08.817115 2014-08-13 17:38:02.766377 7fe9bae887a0 20 read_log 192888'17989 (0'0) promote 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by osd.4.0:21930 2014-08-11 16:46:11.785903 2014-08-13 17:38:02.766389 7fe9bae887a0 20 read_log 192888'17990 (192888'17989) modify 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.1315259.0:19907 2014-08-11 16:46:11.774640 2014-08-13 17:38:02.766401 7fe9bae887a0 20 read_log 192888'17991 (0'0) promote ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by osd.4.0:21934 2014-08-11 16:46:12.469506 2014-08-13 17:38:02.766413 7fe9bae887a0 20 read_log 192888'17992 (192888'17991) modify ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.1315259.0:20095 2014-08-11 16:46:12.460641 2014-08-13 17:38:02.766425 7fe9bae887a0 20 read_log 192888'17993 (0'0) promote fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by osd.4.0:21946 2014-08-11 16:46:14.375459 2014-08-13 17:38:02.766436 7fe9bae887a0 20 read_log 192888'17994 (192888'17993) modify fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.1315259.0:20591 2014-08-11 16:46:14.366699 2014-08-13 17:38:02.766448 7fe9bae887a0 20 read_log 192888'17995 (0'0) promote 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by osd.4.0:21960 2014-08-11 16:46:16.060062 2014-08-13 17:38:02.766461 7fe9bae887a0 20 read_log 192888'17996 (192888'17995) modify 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.1315259.0:21050 2014-08-11 16:46:16.051486 2014-08-13 17:38:02.766473 7fe9bae887a0 20 read_log 192888'18005 (0'0) promote 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by osd.4.0:22042 2014-08-11 16:46:28.070533 2014-08-13 17:38:02.766485 7fe9bae887a0 20 read_log 192888'18006 (192888'18005) modify 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.1315259.0:24112 2014-08-11 16:46:28.062675 2014-08-13 17:38:02.766498 7fe9bae887a0 20 read_log 192888'18007 (0'0) promote 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by osd.4.0:22050 2014-08-11 16:46:28.978097 2014-08-13 17:38:02.766535 7fe9bae887a0 20 read_log 192888'18008 (192888'18007) modify 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.1315259.0:24356 2014-08-11 16:46:28.967606 2014-08-13 17:38:02.766551 7fe9bae887a0 20 read_log 192888'18009 (0'0) promote 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by osd.4.0:22060 2014-08-11 16:46:29.926340 2014-08-13 17:38:02.766563 7fe9bae887a0 20 read_log 192888'18010 (192888'18009) modify 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.1315259.0:24601 2014-08-11 16:46:29.914132 2014-08-13 17:38:02.766575 7fe9bae887a0 20 read_log 192888'18011 (0'0) promote a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by osd.4.0:22070 2014-08-11 16:46:30.226812 2014-08-13 17:38:02.766587 7fe9bae887a0 20 read_log 192888'18012 (192888'18011) modify a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.1315259.0:24677 2014-08-11 16:46:30.216105 2014-08-13 17:38:02.766599 7fe9bae887a0 20 read_log 192888'18015 (0'0) promote 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by osd.4.0:22154 2014-08-11 16:46:45.081318 2014-08-13 17:38:02.766611 7fe9bae887a0 20 read_log 192888'18016 (192888'18015) modify 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.1315259.0:28265 2014-08-11 16:46:45.072892 2014-08-13 17:38:02.766623 7fe9bae887a0 20 read_log 192888'18017 (0'0) promote 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by osd.4.0:22162 2014-08-11 16:46:45.827070 2014-08-13 17:38:02.766635 7fe9bae887a0 20 read_log 192888'18018 (192888'18017) modify 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.1315259.0:28440 2014-08-11 16:46:45.817154 2014-08-13 17:38:02.766647 7fe9bae887a0 20 read_log 192888'18021 (0'0) promote 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by osd.4.0:22216 2014-08-11 16:46:53.841043 2014-08-13 17:38:02.766659 7fe9bae887a0 20 read_log 192888'18022 (192888'18021) modify 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.1315259.0:30461 2014-08-11 16:46:53.831802 2014-08-13 17:38:02.766671 7fe9bae887a0 20 read_log 192888'18024 (192888'17906) modify c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.1315259.0:31095 2014-08-11 16:46:55.995049 2014-08-13 17:38:02.766683 7fe9bae887a0 20 read_log 192888'18028 (192888'17914) modify 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.1315259.0:32082 2014-08-11 16:46:56.284807 2014-08-13 17:38:02.766695 7fe9bae887a0 20 read_log 192888'18029 (192888'17916) modify fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.1315259.0:32164 2014-08-11 16:46:56.304728 2014-08-13 17:38:02.766707 7fe9bae887a0 20 read_log 192888'18031 (192888'17920) modify 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.1315259.0:33778 2014-08-11 16:46:56.722757 2014-08-13 17:38:02.766719 7fe9bae887a0 20 read_log 192888'18034 (192888'17926) modify c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.1315259.0:35742 2014-08-11 16:46:57.235011 2014-08-13 17:38:02.766731 7fe9bae887a0 20 read_log 192888'18035 (192888'17928) modify 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.1315259.0:36575 2014-08-11 16:46:57.459284 2014-08-13 17:38:02.766743 7fe9bae887a0 20 read_log 192888'18038 (192888'17934) modify bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.1315259.0:38205 2014-08-11 16:46:57.885154 2014-08-13 17:38:02.766755 7fe9bae887a0 20 read_log 192888'18039 (192888'17936) modify d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.1315259.0:39003 2014-08-11 16:46:58.100923 2014-08-13 17:38:02.766767 7fe9bae887a0 20 read_log 192888'18042 (192888'17942) modify eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.1315259.0:41022 2014-08-11 16:46:58.639786 2014-08-13 17:38:02.766788 7fe9bae887a0 20 read_log 192888'18043 (192888'17944) modify b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.1315259.0:41751 2014-08-11 16:46:58.827161 2014-08-13 17:38:02.766801 7fe9bae887a0 20 read_log 192888'18044 (192888'17946) modify 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.1315259.0:42070 2014-08-11 16:46:58.911227 2014-08-13 17:38:02.766813 7fe9bae887a0 20 read_log 192888'18049 (192888'17956) modify 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.1315259.0:42959 2014-08-11 16:46:59.152633 2014-08-13 17:38:02.766825 7fe9bae887a0 20 read_log 192888'18050 (192888'17958) modify a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.1315259.0:43815 2014-08-11 16:46:59.396377 2014-08-13 17:38:02.766837 7fe9bae887a0 20 read_log 192888'18051 (192888'17960) modify 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.1315259.0:44556 2014-08-11 16:46:59.611618 2014-08-13 17:38:02.766849 7fe9bae887a0 20 read_log 192888'18052 (192888'17962) modify 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.1315259.0:44584 2014-08-11 16:46:59.619801 2014-08-13 17:38:02.766861 7fe9bae887a0 20 read_log 192888'18057 (192888'17972) modify 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.1315259.0:46676 2014-08-11 16:47:00.227419 2014-08-13 17:38:02.766873 7fe9bae887a0 20 read_log 192888'18058 (192888'17974) modify 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.1315259.0:46732 2014-08-11 16:47:00.243215 2014-08-13 17:38:02.766885 7fe9bae887a0 20 read_log 192888'18059 (192888'17976) modify 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.1315259.0:46903 2014-08-11 16:47:00.290863 2014-08-13 17:38:02.766897 7fe9bae887a0 20 read_log 192888'18060 (192888'17978) modify e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.1315259.0:47004 2014-08-11 16:47:00.319579 2014-08-13 17:38:02.766909 7fe9bae887a0 20 read_log 192888'18061 (192888'17980) modify 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.1315259.0:47620 2014-08-11 16:47:00.512319 2014-08-13 17:38:02.766921 7fe9bae887a0 20 read_log 192888'18062 (192888'17982) modify 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.1315259.0:47902 2014-08-11 16:47:00.602253 2014-08-13 17:38:02.766933 7fe9bae887a0 20 read_log 192888'18063 (192888'17984) modify ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.1315259.0:48209 2014-08-11 16:47:00.698020 2014-08-13 17:38:02.766961 7fe9bae887a0 20 read_log 192888'18064 (192888'17986) modify fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.1315259.0:49853 2014-08-11 16:47:01.191968 2014-08-13 17:38:02.766973 7fe9bae887a0 20 read_log 192888'18066 (192888'17990) modify 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.1315259.0:50634 2014-08-11 16:47:01.430670 2014-08-13 17:38:02.766985 7fe9bae887a0 20 read_log 192888'18067 (192888'17992) modify ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.1315259.0:50822 2014-08-11 16:47:01.488361 2014-08-13 17:38:02.766998 7fe9bae887a0 20 read_log 192888'18068 (192888'17994) modify fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.1315259.0:51318 2014-08-11 16:47:01.645821 2014-08-13 17:38:02.767010 7fe9bae887a0 20 read_log 192888'18069 (192888'17996) modify 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.1315259.0:51777 2014-08-11 16:47:01.779723 2014-08-13 17:38:02.767022 7fe9bae887a0 20 read_log 192888'18074 (192888'18006) modify 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.1315259.0:54839 2014-08-11 16:47:02.733103 2014-08-13 17:38:02.767034 7fe9bae887a0 20 read_log 192888'18075 (192888'18008) modify 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.1315259.0:55083 2014-08-11 16:47:02.805974 2014-08-13 17:38:02.767053 7fe9bae887a0 20 read_log 192888'18076 (192888'18010) modify 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.1315259.0:55328 2014-08-11 16:47:02.879450 2014-08-13 17:38:02.767069 7fe9bae887a0 20 read_log 192888'18077 (192888'18012) modify a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.1315259.0:55404 2014-08-11 16:47:02.902119 2014-08-13 17:38:02.767081 7fe9bae887a0 20 read_log 192888'18079 (192888'18016) modify 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.1315259.0:58992 2014-08-11 16:47:03.967250 2014-08-13 17:38:02.767094 7fe9bae887a0 20 read_log 192888'18080 (192888'18018) modify 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.1315259.0:59167 2014-08-11 16:47:04.024201 2014-08-13 17:38:02.767107 7fe9bae887a0 20 read_log 192888'18082 (192888'18022) modify 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.1315259.0:61188 2014-08-11 16:47:04.661871 2014-08-13 17:38:02.767119 7fe9bae887a0 20 read_log 192888'18083 (192888'18083) modify 3d/hit_set_24.3d_archive_2014-08-11 16:44:56.254254_2014-08-11 17:05:17.262043/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.262449 2014-08-13 17:38:02.767132 7fe9bae887a0 20 read_log 192888'18084 (192888'17901) delete 3d/hit_set_24.3d_archive_2014-08-11 16:02:02.114927_2014-08-11 16:44:56.253783/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.262449 2014-08-13 17:38:02.767144 7fe9bae887a0 20 read_log 192888'18085 (0'0) promote 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by osd.4.0:22660 2014-08-11 17:05:17.267300 2014-08-13 17:38:02.767161 7fe9bae887a0 20 read_log 192888'18086 (192888'18085) modify 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.942868.0:546 2014-08-11 17:05:17.256834 2014-08-13 17:38:02.767174 7fe9bae887a0 20 read_log 192888'18091 (0'0) promote 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by osd.4.0:22677 2014-08-11 17:05:19.797351 2014-08-13 17:38:02.767187 7fe9bae887a0 20 read_log 192888'18092 (192888'18091) modify 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.942868.0:1157 2014-08-11 17:05:19.789491 2014-08-13 17:38:02.767199 7fe9bae887a0 20 read_log 192888'18095 (0'0) promote 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by osd.4.0:22683 2014-08-11 17:05:20.534659 2014-08-13 17:38:02.767211 7fe9bae887a0 20 read_log 192888'18096 (192888'18095) modify 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.942868.0:1345 2014-08-11 17:05:20.525402 2014-08-13 17:38:02.767223 7fe9bae887a0 20 read_log 192888'18101 (0'0) promote dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by osd.4.0:22701 2014-08-11 17:05:24.380566 2014-08-13 17:38:02.767235 7fe9bae887a0 20 read_log 192888'18102 (192888'18101) modify dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.942868.0:2281 2014-08-11 17:05:24.371669 2014-08-13 17:38:02.767247 7fe9bae887a0 20 read_log 192888'18103 (0'0) promote b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by osd.4.0:22703 2014-08-11 17:05:24.391494 2014-08-13 17:38:02.767259 7fe9bae887a0 20 read_log 192888'18104 (192888'18103) modify b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.942868.0:2284 2014-08-11 17:05:24.382622 2014-08-13 17:38:02.767271 7fe9bae887a0 20 read_log 192888'18105 (0'0) promote db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by osd.4.0:22720 2014-08-11 17:05:26.623624 2014-08-13 17:38:02.767283 7fe9bae887a0 20 read_log 192888'18106 (192888'18105) modify db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.942868.0:2886 2014-08-11 17:05:26.614491 2014-08-13 17:38:02.767305 7fe9bae887a0 20 read_log 192888'18107 (0'0) promote a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by osd.4.0:22730 2014-08-11 17:05:28.062666 2014-08-13 17:38:02.767317 7fe9bae887a0 20 read_log 192888'18108 (192888'18107) modify a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.942868.0:3276 2014-08-11 17:05:28.054822 2014-08-13 17:38:02.767330 7fe9bae887a0 20 read_log 192888'18109 (0'0) promote 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by osd.4.0:22766 2014-08-11 17:05:31.895219 2014-08-13 17:38:02.767342 7fe9bae887a0 20 read_log 192888'18110 (192888'18109) modify 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.942868.0:4315 2014-08-11 17:05:31.886633 2014-08-13 17:38:02.767357 7fe9bae887a0 20 read_log 192888'18113 (0'0) promote 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by osd.4.0:22778 2014-08-11 17:05:33.383764 2014-08-13 17:38:02.767369 7fe9bae887a0 20 read_log 192888'18114 (192888'18113) modify 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.942868.0:4701 2014-08-11 17:05:33.375129 2014-08-13 17:38:02.767381 7fe9bae887a0 20 read_log 192888'18115 (0'0) promote 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by osd.4.0:22786 2014-08-11 17:05:33.841187 2014-08-13 17:38:02.767393 7fe9bae887a0 20 read_log 192888'18116 (192888'18115) modify 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.942868.0:4823 2014-08-11 17:05:33.833157 2014-08-13 17:38:02.767406 7fe9bae887a0 20 read_log 192888'18119 (0'0) promote 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by osd.4.0:22792 2014-08-11 17:05:34.506451 2014-08-13 17:38:02.767418 7fe9bae887a0 20 read_log 192888'18120 (192888'18119) modify 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.942868.0:4975 2014-08-11 17:05:34.497526 2014-08-13 17:38:02.767433 7fe9bae887a0 20 read_log 192888'18121 (0'0) promote cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by osd.4.0:22828 2014-08-11 17:05:39.850089 2014-08-13 17:38:02.767445 7fe9bae887a0 20 read_log 192888'18122 (192888'18121) modify cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.942868.0:6424 2014-08-11 17:05:39.841406 2014-08-13 17:38:02.767457 7fe9bae887a0 20 read_log 192888'18123 (0'0) promote 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by osd.4.0:22834 2014-08-11 17:05:40.375646 2014-08-13 17:38:02.767469 7fe9bae887a0 20 read_log 192888'18124 (192888'18123) modify 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.942868.0:6570 2014-08-11 17:05:40.365174 2014-08-13 17:38:02.767481 7fe9bae887a0 20 read_log 192888'18125 (0'0) promote e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by osd.4.0:22856 2014-08-11 17:05:43.515350 2014-08-13 17:38:02.767493 7fe9bae887a0 20 read_log 192888'18126 (192888'18125) modify e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.942868.0:7383 2014-08-11 17:05:43.506850 2014-08-13 17:38:02.767505 7fe9bae887a0 20 read_log 192888'18133 (0'0) promote 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by osd.4.0:22978 2014-08-11 17:06:03.404857 2014-08-13 17:38:02.767517 7fe9bae887a0 20 read_log 192888'18134 (192888'18133) modify 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.942868.0:12783 2014-08-11 17:06:03.394312 2014-08-13 17:38:02.767530 7fe9bae887a0 20 read_log 192888'18139 (0'0) promote 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by osd.4.0:22992 2014-08-11 17:06:04.859191 2014-08-13 17:38:02.767542 7fe9bae887a0 20 read_log 192888'18140 (192888'18139) modify 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.942868.0:13187 2014-08-11 17:06:04.851421 2014-08-13 17:38:02.767561 7fe9bae887a0 20 read_log 192888'18143 (0'0) promote 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by osd.4.0:23030 2014-08-11 17:06:11.782648 2014-08-13 17:38:02.767574 7fe9bae887a0 20 read_log 192888'18144 (192888'18143) modify 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.942868.0:14887 2014-08-11 17:06:11.773846 2014-08-13 17:38:02.767586 7fe9bae887a0 20 read_log 192888'18151 (0'0) promote db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by osd.4.0:23090 2014-08-11 17:06:20.725059 2014-08-13 17:38:02.767598 7fe9bae887a0 20 read_log 192888'18152 (192888'18151) modify db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.942868.0:17175 2014-08-11 17:06:20.714108 2014-08-13 17:38:02.767610 7fe9bae887a0 20 read_log 192888'18153 (0'0) promote f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by osd.4.0:23098 2014-08-11 17:06:21.574118 2014-08-13 17:38:02.767623 7fe9bae887a0 20 read_log 192888'18154 (192888'18153) modify f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.942868.0:17400 2014-08-11 17:06:21.566056 2014-08-13 17:38:02.767636 7fe9bae887a0 20 read_log 192888'18161 (0'0) promote 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by osd.4.0:23176 2014-08-11 17:06:31.947696 2014-08-13 17:38:02.767648 7fe9bae887a0 20 read_log 192888'18162 (192888'18161) modify 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.942868.0:20156 2014-08-11 17:06:31.936845 2014-08-13 17:38:02.767660 7fe9bae887a0 20 read_log 192888'18167 (0'0) promote 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by osd.4.0:23252 2014-08-11 17:06:43.173794 2014-08-13 17:38:02.767672 7fe9bae887a0 20 read_log 192888'18168 (192888'18167) modify 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.942868.0:23056 2014-08-11 17:06:43.164158 2014-08-13 17:38:02.767684 7fe9bae887a0 20 read_log 192888'18177 (0'0) promote eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by osd.4.0:23340 2014-08-11 17:06:52.488870 2014-08-13 17:38:02.767696 7fe9bae887a0 20 read_log 192888'18178 (192888'18177) modify eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.942868.0:25549 2014-08-11 17:06:52.480572 2014-08-13 17:38:02.767708 7fe9bae887a0 20 read_log 192888'18179 (0'0) promote 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by osd.4.0:23370 2014-08-11 17:06:58.820554 2014-08-13 17:38:02.767720 7fe9bae887a0 20 read_log 192888'18180 (192888'18179) modify 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.942868.0:27162 2014-08-11 17:06:58.812525 2014-08-13 17:38:02.767732 7fe9bae887a0 20 read_log 192888'18181 (0'0) promote c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by osd.4.0:23388 2014-08-11 17:07:00.934651 2014-08-13 17:38:02.767744 7fe9bae887a0 20 read_log 192888'18182 (192888'18181) modify c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.942868.0:27714 2014-08-11 17:07:00.926017 2014-08-13 17:38:02.767757 7fe9bae887a0 20 read_log 192888'18185 (0'0) promote a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by osd.4.0:23456 2014-08-11 17:07:12.238943 2014-08-13 17:38:02.767769 7fe9bae887a0 20 read_log 192888'18186 (192888'18185) modify a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.942868.0:30602 2014-08-11 17:07:12.228888 2014-08-13 17:38:02.767781 7fe9bae887a0 20 read_log 192888'18189 (192888'18086) modify 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.942868.0:31812 2014-08-11 17:07:16.065747 2014-08-13 17:38:02.767794 7fe9bae887a0 20 read_log 192888'18192 (192888'18092) modify 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.942868.0:32423 2014-08-11 17:07:16.249873 2014-08-13 17:38:02.767813 7fe9bae887a0 20 read_log 192888'18194 (192888'18096) modify 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.942868.0:32611 2014-08-11 17:07:16.300988 2014-08-13 17:38:02.767825 7fe9bae887a0 20 read_log 192888'18197 (192888'18102) modify dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.942868.0:33547 2014-08-11 17:07:16.546175 2014-08-13 17:38:02.767839 7fe9bae887a0 20 read_log 192888'18198 (192888'18104) modify b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.942868.0:33550 2014-08-11 17:07:16.546441 2014-08-13 17:38:02.767851 7fe9bae887a0 20 read_log 192888'18199 (192888'18106) modify db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.942868.0:34152 2014-08-11 17:07:16.712261 2014-08-13 17:38:02.767863 7fe9bae887a0 20 read_log 192888'18200 (192888'18108) modify a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.942868.0:34542 2014-08-11 17:07:16.819882 2014-08-13 17:38:02.767875 7fe9bae887a0 20 read_log 192888'18201 (192888'18110) modify 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.942868.0:35581 2014-08-11 17:07:17.110038 2014-08-13 17:38:02.767887 7fe9bae887a0 20 read_log 192888'18203 (192888'18114) modify 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.942868.0:35967 2014-08-11 17:07:17.209023 2014-08-13 17:38:02.767899 7fe9bae887a0 20 read_log 192888'18204 (192888'18116) modify 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.942868.0:36089 2014-08-11 17:07:17.238518 2014-08-13 17:38:02.767911 7fe9bae887a0 20 read_log 192888'18206 (192888'18120) modify 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.942868.0:36241 2014-08-11 17:07:17.277208 2014-08-13 17:38:02.767923 7fe9bae887a0 20 read_log 192888'18207 (192888'18122) modify cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.942868.0:37690 2014-08-11 17:07:17.680264 2014-08-13 17:38:02.767935 7fe9bae887a0 20 read_log 192888'18208 (192888'18124) modify 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.942868.0:37836 2014-08-11 17:07:17.720109 2014-08-13 17:38:02.767947 7fe9bae887a0 20 read_log 192888'18209 (192888'18126) modify e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.942868.0:38649 2014-08-11 17:07:17.950241 2014-08-13 17:38:02.767960 7fe9bae887a0 20 read_log 192888'18213 (192888'18134) modify 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.942868.0:44049 2014-08-11 17:07:19.451743 2014-08-13 17:38:02.767972 7fe9bae887a0 20 read_log 192888'18216 (192888'18140) modify 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.942868.0:44453 2014-08-11 17:07:19.569906 2014-08-13 17:38:02.767984 7fe9bae887a0 20 read_log 192888'18218 (192888'18144) modify 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.942868.0:46153 2014-08-11 17:07:20.035668 2014-08-13 17:38:02.767996 7fe9bae887a0 20 read_log 192888'18222 (192888'18152) modify db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.942868.0:48441 2014-08-11 17:07:20.719889 2014-08-13 17:38:02.768008 7fe9bae887a0 20 read_log 192888'18223 (192888'18154) modify f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.942868.0:48666 2014-08-11 17:07:20.782177 2014-08-13 17:38:02.768020 7fe9bae887a0 20 read_log 192888'18227 (192888'18162) modify 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.942868.0:51422 2014-08-11 17:07:21.645135 2014-08-13 17:38:02.768032 7fe9bae887a0 20 read_log 192888'18230 (192888'18168) modify 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.942868.0:54322 2014-08-11 17:07:22.525604 2014-08-13 17:38:02.768049 7fe9bae887a0 20 read_log 192888'18235 (192888'18178) modify eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.942868.0:56815 2014-08-11 17:07:23.267964 2014-08-13 17:38:02.768071 7fe9bae887a0 20 read_log 192888'18236 (192888'18180) modify 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.942868.0:58428 2014-08-11 17:07:23.740307 2014-08-13 17:38:02.768084 7fe9bae887a0 20 read_log 192888'18237 (192888'18182) modify c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.942868.0:58980 2014-08-11 17:07:23.908960 2014-08-13 17:38:02.768096 7fe9bae887a0 20 read_log 192888'18239 (192888'18186) modify a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.942868.0:61868 2014-08-11 17:07:24.780159 2014-08-13 17:38:02.768108 7fe9bae887a0 20 read_log 193305'18241 (193305'18241) modify 3d/hit_set_24.3d_archive_2014-08-11 18:37:04.904929_2014-08-11 18:42:35.578890/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:35.579189 2014-08-13 17:38:02.768120 7fe9bae887a0 20 read_log 193305'18242 (192888'18083) delete 3d/hit_set_24.3d_archive_2014-08-11 16:44:56.254254_2014-08-11 17:05:17.262043/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:35.579189 2014-08-13 17:38:02.768133 7fe9bae887a0 20 read_log 193305'18243 (0'0) promote 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by osd.4.0:2 2014-08-11 18:42:35.627695 2014-08-13 17:38:02.768145 7fe9bae887a0 20 read_log 193305'18244 (193305'18243) modify 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.951337.0:616 2014-08-11 18:42:35.571701 2014-08-13 17:38:02.768180 7fe9bae887a0 20 read_log 193305'18245 (0'0) promote 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by osd.4.0:4 2014-08-11 18:42:36.680419 2014-08-13 17:38:02.768196 7fe9bae887a0 20 read_log 193305'18246 (193305'18245) modify 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.951337.0:919 2014-08-11 18:42:36.673618 2014-08-13 17:38:02.768209 7fe9bae887a0 20 read_log 193305'18247 (0'0) promote 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by osd.4.0:9 2014-08-11 18:42:37.857728 2014-08-13 17:38:02.768221 7fe9bae887a0 20 read_log 193305'18248 (193305'18247) modify 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.951337.0:1245 2014-08-11 18:42:37.851448 2014-08-13 17:38:02.768233 7fe9bae887a0 20 read_log 193305'18249 (0'0) promote ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by osd.4.0:23 2014-08-11 18:42:40.452927 2014-08-13 17:38:02.768245 7fe9bae887a0 20 read_log 193305'18250 (193305'18249) modify ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.951337.0:1914 2014-08-11 18:42:40.445483 2014-08-13 17:38:02.768257 7fe9bae887a0 20 read_log 193305'18251 (0'0) promote 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by osd.4.0:51 2014-08-11 18:42:47.311767 2014-08-13 17:38:02.768269 7fe9bae887a0 20 read_log 193305'18252 (193305'18251) modify 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.951337.0:3805 2014-08-11 18:42:47.305036 2014-08-13 17:38:02.768281 7fe9bae887a0 20 read_log 193305'18253 (0'0) promote 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by osd.4.0:89 2014-08-11 18:42:55.181792 2014-08-13 17:38:02.768293 7fe9bae887a0 20 read_log 193305'18254 (193305'18253) modify 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.951337.0:5933 2014-08-11 18:42:55.173713 2014-08-13 17:38:02.768305 7fe9bae887a0 20 read_log 193305'18255 (0'0) promote bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by osd.4.0:93 2014-08-11 18:42:55.537740 2014-08-13 17:38:02.768317 7fe9bae887a0 20 read_log 193305'18256 (193305'18255) modify bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.951337.0:6029 2014-08-11 18:42:55.530490 2014-08-13 17:38:02.768356 7fe9bae887a0 20 read_log 193305'18257 (0'0) promote 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by osd.4.0:101 2014-08-11 18:42:56.741879 2014-08-13 17:38:02.768372 7fe9bae887a0 20 read_log 193305'18258 (193305'18257) modify 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.951337.0:6362 2014-08-11 18:42:56.733331 2014-08-13 17:38:02.768385 7fe9bae887a0 20 read_log 193305'18259 (0'0) promote 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by osd.4.0:107 2014-08-11 18:42:57.530495 2014-08-13 17:38:02.768397 7fe9bae887a0 20 read_log 193305'18260 (193305'18259) modify 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.951337.0:6580 2014-08-11 18:42:57.523865 2014-08-13 17:38:02.768409 7fe9bae887a0 20 read_log 193305'18261 (0'0) promote a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by osd.4.0:139 2014-08-11 18:43:04.370665 2014-08-13 17:38:02.768421 7fe9bae887a0 20 read_log 193305'18262 (193305'18261) modify a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.951337.0:8415 2014-08-11 18:43:04.360431 2014-08-13 17:38:02.768433 7fe9bae887a0 20 read_log 193305'18263 (0'0) promote 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by osd.4.0:171 2014-08-11 18:43:11.823420 2014-08-13 17:38:02.768445 7fe9bae887a0 20 read_log 193305'18264 (193305'18263) modify 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.951337.0:10413 2014-08-11 18:43:11.815148 2014-08-13 17:38:02.768458 7fe9bae887a0 20 read_log 193305'18265 (0'0) promote 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by osd.4.0:175 2014-08-11 18:43:13.758896 2014-08-13 17:38:02.768470 7fe9bae887a0 20 read_log 193305'18266 (193305'18265) modify 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.951337.0:10933 2014-08-11 18:43:13.748827 2014-08-13 17:38:02.768482 7fe9bae887a0 20 read_log 193305'18267 (0'0) promote 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by osd.4.0:181 2014-08-11 18:43:14.378604 2014-08-13 17:38:02.768494 7fe9bae887a0 20 read_log 193305'18268 (193305'18267) modify 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.951337.0:11099 2014-08-11 18:43:14.370536 2014-08-13 17:38:02.768506 7fe9bae887a0 20 read_log 193305'18269 (0'0) promote cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by osd.4.0:185 2014-08-11 18:43:14.988261 2014-08-13 17:38:02.768517 7fe9bae887a0 20 read_log 193305'18270 (193305'18269) modify cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.951337.0:11241 2014-08-11 18:43:14.980659 2014-08-13 17:38:02.768529 7fe9bae887a0 20 read_log 193305'18271 (0'0) promote ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by osd.4.0:187 2014-08-11 18:43:15.184336 2014-08-13 17:38:02.768541 7fe9bae887a0 20 read_log 193305'18272 (193305'18271) modify ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.951337.0:11277 2014-08-11 18:43:15.174869 2014-08-13 17:38:02.768553 7fe9bae887a0 20 read_log 193305'18273 (0'0) promote 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by osd.4.0:193 2014-08-11 18:43:16.492613 2014-08-13 17:38:02.768565 7fe9bae887a0 20 read_log 193305'18274 (193305'18273) modify 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.951337.0:11618 2014-08-11 18:43:16.484438 2014-08-13 17:38:02.768577 7fe9bae887a0 20 read_log 193305'18275 (0'0) promote 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by osd.4.0:215 2014-08-11 18:43:22.081417 2014-08-13 17:38:02.768589 7fe9bae887a0 20 read_log 193305'18276 (193305'18275) modify 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.951337.0:13123 2014-08-11 18:43:22.073614 2014-08-13 17:38:02.768608 7fe9bae887a0 20 read_log 193305'18277 (0'0) promote c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by osd.4.0:245 2014-08-11 18:43:29.972503 2014-08-13 17:38:02.768621 7fe9bae887a0 20 read_log 193305'18278 (193305'18277) modify c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.951337.0:15300 2014-08-11 18:43:29.964191 2014-08-13 17:38:02.768634 7fe9bae887a0 20 read_log 193305'18279 (0'0) promote 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by osd.4.0:249 2014-08-11 18:43:30.854709 2014-08-13 17:38:02.768647 7fe9bae887a0 20 read_log 193305'18280 (193305'18279) modify 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.951337.0:15561 2014-08-11 18:43:30.846758 2014-08-13 17:38:02.768659 7fe9bae887a0 20 read_log 193305'18281 (0'0) promote b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by osd.4.0:253 2014-08-11 18:43:32.067918 2014-08-13 17:38:02.768671 7fe9bae887a0 20 read_log 193305'18282 (193305'18281) modify b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.951337.0:15913 2014-08-11 18:43:32.058697 2014-08-13 17:38:02.768682 7fe9bae887a0 20 read_log 193305'18283 (0'0) promote 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by osd.4.0:265 2014-08-11 18:43:33.609276 2014-08-13 17:38:02.768694 7fe9bae887a0 20 read_log 193305'18284 (193305'18283) modify 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.951337.0:16329 2014-08-11 18:43:33.600704 2014-08-13 17:38:02.768706 7fe9bae887a0 20 read_log 193305'18285 (0'0) promote c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by osd.4.0:283 2014-08-11 18:43:36.194492 2014-08-13 17:38:02.768718 7fe9bae887a0 20 read_log 193305'18286 (193305'18285) modify c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.951337.0:17066 2014-08-11 18:43:36.186927 2014-08-13 17:38:02.768730 7fe9bae887a0 20 read_log 193305'18287 (0'0) promote a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by osd.4.0:341 2014-08-11 18:43:48.115912 2014-08-13 17:38:02.768742 7fe9bae887a0 20 read_log 193305'18288 (193305'18287) modify a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.951337.0:20405 2014-08-11 18:43:48.108000 2014-08-13 17:38:02.768754 7fe9bae887a0 20 read_log 193305'18289 (0'0) promote 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by osd.4.0:365 2014-08-11 18:43:53.694608 2014-08-13 17:38:02.768766 7fe9bae887a0 20 read_log 193305'18290 (193305'18289) modify 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.951337.0:21967 2014-08-11 18:43:53.685338 2014-08-13 17:38:02.768778 7fe9bae887a0 20 read_log 193305'18291 (0'0) promote cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by osd.4.0:371 2014-08-11 18:43:55.600623 2014-08-13 17:38:02.768791 7fe9bae887a0 20 read_log 193305'18292 (193305'18291) modify cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.951337.0:22482 2014-08-11 18:43:55.592374 2014-08-13 17:38:02.768802 7fe9bae887a0 20 read_log 193305'18293 (0'0) promote 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by osd.4.0:373 2014-08-11 18:43:55.641580 2014-08-13 17:38:02.768815 7fe9bae887a0 20 read_log 193305'18294 (193305'18293) modify 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.951337.0:22493 2014-08-11 18:43:55.630152 2014-08-13 17:38:02.768826 7fe9bae887a0 20 read_log 193305'18295 (0'0) promote 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by osd.4.0:395 2014-08-11 18:43:59.939904 2014-08-13 17:38:02.768838 7fe9bae887a0 20 read_log 193305'18296 (193305'18295) modify 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.951337.0:23700 2014-08-11 18:43:59.929720 2014-08-13 17:38:02.768860 7fe9bae887a0 20 read_log 193305'18297 (0'0) promote fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by osd.4.0:413 2014-08-11 18:44:05.147126 2014-08-13 17:38:02.768872 7fe9bae887a0 20 read_log 193305'18298 (193305'18297) modify fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.951337.0:25136 2014-08-11 18:44:05.139815 2014-08-13 17:38:02.768885 7fe9bae887a0 20 read_log 193305'18299 (0'0) promote e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by osd.4.0:415 2014-08-11 18:44:05.485896 2014-08-13 17:38:02.768897 7fe9bae887a0 20 read_log 193305'18300 (193305'18299) modify e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.951337.0:25232 2014-08-11 18:44:05.476915 2014-08-13 17:38:02.768909 7fe9bae887a0 20 read_log 193305'18301 (0'0) promote f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by osd.4.0:421 2014-08-11 18:44:07.449502 2014-08-13 17:38:02.768921 7fe9bae887a0 20 read_log 193305'18302 (193305'18301) modify f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.951337.0:25757 2014-08-11 18:44:07.439797 2014-08-13 17:38:02.768933 7fe9bae887a0 20 read_log 193305'18303 (0'0) promote 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by osd.4.0:423 2014-08-11 18:44:07.513771 2014-08-13 17:38:02.768945 7fe9bae887a0 20 read_log 193305'18304 (193305'18303) modify 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.951337.0:25773 2014-08-11 18:44:07.506058 2014-08-13 17:38:02.768957 7fe9bae887a0 20 read_log 193305'18305 (0'0) promote b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by osd.4.0:459 2014-08-11 18:44:15.711762 2014-08-13 17:38:02.768969 7fe9bae887a0 20 read_log 193305'18306 (193305'18305) modify b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.951337.0:28017 2014-08-11 18:44:15.704724 2014-08-13 17:38:02.768981 7fe9bae887a0 20 read_log 193305'18307 (0'0) promote 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by osd.4.0:465 2014-08-11 18:44:16.389850 2014-08-13 17:38:02.768994 7fe9bae887a0 20 read_log 193305'18308 (193305'18307) modify 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.951337.0:28208 2014-08-11 18:44:16.381563 2014-08-13 17:38:02.769006 7fe9bae887a0 20 read_log 193305'18309 (0'0) promote 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by osd.4.0:471 2014-08-11 18:44:18.142772 2014-08-13 17:38:02.769018 7fe9bae887a0 20 read_log 193305'18310 (193305'18309) modify 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.951337.0:28686 2014-08-11 18:44:18.135344 2014-08-13 17:38:02.769029 7fe9bae887a0 20 read_log 193305'18311 (0'0) promote 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by osd.4.0:485 2014-08-11 18:44:20.648948 2014-08-13 17:38:02.769042 7fe9bae887a0 20 read_log 193305'18312 (193305'18311) modify 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.951337.0:29359 2014-08-11 18:44:20.641816 2014-08-13 17:38:02.769054 7fe9bae887a0 20 read_log 193305'18313 (0'0) promote 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by osd.4.0:523 2014-08-11 18:44:29.794023 2014-08-13 17:38:02.769067 7fe9bae887a0 20 read_log 193305'18314 (193305'18313) modify 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.951337.0:31859 2014-08-11 18:44:29.786257 2014-08-13 17:38:02.769079 7fe9bae887a0 20 read_log 193305'18315 (0'0) promote db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by osd.4.0:531 2014-08-11 18:44:30.579872 2014-08-13 17:38:02.769091 7fe9bae887a0 20 read_log 193305'18316 (193305'18315) modify db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.951337.0:32091 2014-08-11 18:44:30.571599 2014-08-13 17:38:02.769110 7fe9bae887a0 20 read_log 193305'18317 (193305'18244) modify 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.951337.0:33327 2014-08-11 18:44:34.339876 2014-08-13 17:38:02.769122 7fe9bae887a0 20 read_log 193305'18318 (193305'18246) modify 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.951337.0:33630 2014-08-11 18:44:34.544380 2014-08-13 17:38:02.769134 7fe9bae887a0 20 read_log 193305'18319 (193305'18248) modify 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.951337.0:33956 2014-08-11 18:44:34.734283 2014-08-13 17:38:02.769147 7fe9bae887a0 20 read_log 193305'18320 (193305'18250) modify ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.951337.0:34625 2014-08-11 18:44:35.101470 2014-08-13 17:38:02.769168 7fe9bae887a0 20 read_log 193305'18321 (193305'18252) modify 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.951337.0:36516 2014-08-11 18:44:36.285520 2014-08-13 17:38:02.769180 7fe9bae887a0 20 read_log 193305'18322 (193305'18254) modify 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.951337.0:38644 2014-08-11 18:44:37.574840 2014-08-13 17:38:02.769193 7fe9bae887a0 20 read_log 193305'18323 (193305'18256) modify bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.951337.0:38740 2014-08-11 18:44:37.629460 2014-08-13 17:38:02.769205 7fe9bae887a0 20 read_log 193305'18324 (193305'18258) modify 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.951337.0:39073 2014-08-11 18:44:37.810862 2014-08-13 17:38:02.769217 7fe9bae887a0 20 read_log 193305'18325 (193305'18260) modify 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.951337.0:39291 2014-08-11 18:44:37.926628 2014-08-13 17:38:02.769230 7fe9bae887a0 20 read_log 193305'18326 (193305'18262) modify a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.951337.0:41126 2014-08-11 18:44:38.642102 2014-08-13 17:38:02.769242 7fe9bae887a0 20 read_log 193305'18327 (193305'18264) modify 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.951337.0:43124 2014-08-11 18:44:39.491754 2014-08-13 17:38:02.769254 7fe9bae887a0 20 read_log 193305'18328 (193305'18266) modify 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.951337.0:43644 2014-08-11 18:44:39.653371 2014-08-13 17:38:02.769266 7fe9bae887a0 20 read_log 193305'18329 (193305'18268) modify 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.951337.0:43810 2014-08-11 18:44:39.707556 2014-08-13 17:38:02.769278 7fe9bae887a0 20 read_log 193305'18330 (193305'18270) modify cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.951337.0:43952 2014-08-11 18:44:39.749228 2014-08-13 17:38:02.769290 7fe9bae887a0 20 read_log 193305'18331 (193305'18272) modify ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.951337.0:43988 2014-08-11 18:44:39.760620 2014-08-13 17:38:02.769302 7fe9bae887a0 20 read_log 193305'18332 (193305'18274) modify 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.951337.0:44329 2014-08-11 18:44:39.853623 2014-08-13 17:38:02.769314 7fe9bae887a0 20 read_log 193305'18333 (193305'18276) modify 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.951337.0:45834 2014-08-11 18:44:40.315073 2014-08-13 17:38:02.769326 7fe9bae887a0 20 read_log 193305'18334 (193305'18278) modify c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.951337.0:48011 2014-08-11 18:44:40.930420 2014-08-13 17:38:02.769338 7fe9bae887a0 20 read_log 193305'18335 (193305'18280) modify 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.951337.0:48272 2014-08-11 18:44:41.004591 2014-08-13 17:38:02.769350 7fe9bae887a0 20 read_log 193305'18336 (193305'18282) modify b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.951337.0:48624 2014-08-11 18:44:41.106124 2014-08-13 17:38:02.769368 7fe9bae887a0 20 read_log 193305'18337 (193305'18284) modify 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.951337.0:49040 2014-08-11 18:44:41.235068 2014-08-13 17:38:02.769398 7fe9bae887a0 20 read_log 193305'18338 (193305'18286) modify c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.951337.0:49777 2014-08-11 18:44:41.468519 2014-08-13 17:38:02.769410 7fe9bae887a0 20 read_log 193305'18339 (193305'18288) modify a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.951337.0:53116 2014-08-11 18:44:42.432183 2014-08-13 17:38:02.769423 7fe9bae887a0 20 read_log 193305'18340 (193305'18290) modify 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.951337.0:54678 2014-08-11 18:44:42.909512 2014-08-13 17:38:02.769435 7fe9bae887a0 20 read_log 193305'18341 (193305'18292) modify cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.951337.0:55193 2014-08-11 18:44:43.053403 2014-08-13 17:38:02.769447 7fe9bae887a0 20 read_log 193305'18342 (193305'18294) modify 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.951337.0:55204 2014-08-11 18:44:43.056249 2014-08-13 17:38:02.769459 7fe9bae887a0 20 read_log 193305'18343 (193305'18296) modify 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.951337.0:56411 2014-08-11 18:44:43.411100 2014-08-13 17:38:02.769471 7fe9bae887a0 20 read_log 193305'18344 (193305'18298) modify fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.951337.0:57847 2014-08-11 18:44:43.874368 2014-08-13 17:38:02.769484 7fe9bae887a0 20 read_log 193305'18345 (193305'18300) modify e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.951337.0:57943 2014-08-11 18:44:43.900621 2014-08-13 17:38:02.769497 7fe9bae887a0 20 read_log 193305'18346 (193305'18302) modify f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.951337.0:58468 2014-08-11 18:44:44.050429 2014-08-13 17:38:02.769510 7fe9bae887a0 20 read_log 193305'18347 (193305'18304) modify 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.951337.0:58484 2014-08-11 18:44:44.056072 2014-08-13 17:38:02.769522 7fe9bae887a0 20 read_log 193305'18348 (193305'18306) modify b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.951337.0:60728 2014-08-11 18:44:44.712485 2014-08-13 17:38:02.769536 7fe9bae887a0 20 read_log 193305'18349 (193305'18308) modify 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.951337.0:60919 2014-08-11 18:44:44.772383 2014-08-13 17:38:02.769548 7fe9bae887a0 20 read_log 193305'18350 (193305'18310) modify 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.951337.0:61397 2014-08-11 18:44:44.910219 2014-08-13 17:38:02.769560 7fe9bae887a0 20 read_log 193305'18351 (193305'18312) modify 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.951337.0:62070 2014-08-11 18:44:45.118106 2014-08-13 17:38:02.769573 7fe9bae887a0 20 read_log 193305'18352 (193305'18314) modify 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.951337.0:64570 2014-08-11 18:44:45.855216 2014-08-13 17:38:02.769585 7fe9bae887a0 20 read_log 193305'18353 (193305'18316) modify db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.951337.0:64802 2014-08-11 18:44:45.921775 2014-08-13 17:38:02.769597 7fe9bae887a0 20 read_log 194215'18354 (192888'18223) clean f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by osd.4.0:961 2014-08-12 15:37:01.766778 2014-08-13 17:38:02.769609 7fe9bae887a0 20 read_log 194215'18355 (194215'18354) delete f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.2174595.0:12038 0.000000 2014-08-13 17:38:02.769630 7fe9bae887a0 20 read_log 194215'18356 (192888'18051) clean 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by osd.4.0:963 2014-08-12 15:37:01.779391 2014-08-13 17:38:02.769643 7fe9bae887a0 20 read_log 194215'18357 (194215'18356) delete 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.2174595.0:12040 0.000000 2014-08-13 17:38:02.769654 7fe9bae887a0 20 read_log 194215'18358 (192834'17818) clean 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:965 2014-08-12 15:37:01.811920 2014-08-13 17:38:02.769667 7fe9bae887a0 20 read_log 194215'18359 (194215'18358) delete 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.2174595.0:12042 0.000000 2014-08-13 17:38:02.769678 7fe9bae887a0 20 read_log 194215'18360 (192888'18039) clean d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by osd.4.0:967 2014-08-12 15:37:01.828071 2014-08-13 17:38:02.769690 7fe9bae887a0 20 read_log 194215'18361 (194215'18360) delete d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.2174595.0:12044 0.000000 2014-08-13 17:38:02.769701 7fe9bae887a0 20 read_log 194215'18362 (192888'18057) clean 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by osd.4.0:969 2014-08-12 15:37:01.835683 2014-08-13 17:38:02.769713 7fe9bae887a0 20 read_log 194215'18363 (194215'18362) delete 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.2174595.0:12046 0.000000 2014-08-13 17:38:02.769723 7fe9bae887a0 20 read_log 194215'18364 (192888'18236) clean 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by osd.4.0:971 2014-08-12 15:37:01.843534 2014-08-13 17:38:02.769735 7fe9bae887a0 20 read_log 194215'18365 (194215'18364) delete 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.2174595.0:12048 0.000000 2014-08-13 17:38:02.769746 7fe9bae887a0 20 read_log 194215'18366 (193305'18323) clean bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by osd.4.0:973 2014-08-12 15:37:01.851894 2014-08-13 17:38:02.769758 7fe9bae887a0 20 read_log 194215'18367 (194215'18366) delete bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.2174595.0:12050 0.000000 2014-08-13 17:38:02.769768 7fe9bae887a0 20 read_log 194215'18368 (193305'18339) clean a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by osd.4.0:975 2014-08-12 15:37:01.870577 2014-08-13 17:38:02.769780 7fe9bae887a0 20 read_log 194215'18369 (194215'18368) delete a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.2174595.0:12052 0.000000 2014-08-13 17:38:02.769791 7fe9bae887a0 20 read_log 194215'18370 (193305'18318) clean 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by osd.4.0:977 2014-08-12 15:37:01.878712 2014-08-13 17:38:02.769803 7fe9bae887a0 20 read_log 194215'18371 (194215'18370) delete 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.2174595.0:12054 0.000000 2014-08-13 17:38:02.769814 7fe9bae887a0 20 read_log 194215'18372 (192888'18080) clean 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by osd.4.0:979 2014-08-12 15:37:01.885807 2014-08-13 17:38:02.769827 7fe9bae887a0 20 read_log 194215'18373 (194215'18372) delete 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.2174595.0:12056 0.000000 2014-08-13 17:38:02.769838 7fe9bae887a0 20 read_log 194215'18374 (192888'18050) clean a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by osd.4.0:981 2014-08-12 15:37:01.894856 2014-08-13 17:38:02.769850 7fe9bae887a0 20 read_log 194215'18375 (194215'18374) delete a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.2174595.0:12058 0.000000 2014-08-13 17:38:02.769873 7fe9bae887a0 20 read_log 194215'18376 (192888'18076) clean 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by osd.4.0:983 2014-08-12 15:37:01.902929 2014-08-13 17:38:02.769886 7fe9bae887a0 20 read_log 194215'18377 (194215'18376) delete 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.2174595.0:12060 0.000000 2014-08-13 17:38:02.769897 7fe9bae887a0 20 read_log 194215'18378 (193305'18335) clean 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by osd.4.0:985 2014-08-12 15:37:01.920171 2014-08-13 17:38:02.769909 7fe9bae887a0 20 read_log 194215'18379 (194215'18378) delete 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.2174595.0:12062 0.000000 2014-08-13 17:38:02.769920 7fe9bae887a0 20 read_log 194215'18380 (192888'18239) clean a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by osd.4.0:987 2014-08-12 15:37:01.930901 2014-08-13 17:38:02.769932 7fe9bae887a0 20 read_log 194215'18381 (194215'18380) delete a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.2174595.0:12064 0.000000 2014-08-13 17:38:02.769942 7fe9bae887a0 20 read_log 194215'18382 (192834'17727) clean 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:989 2014-08-12 15:37:01.942754 2014-08-13 17:38:02.769954 7fe9bae887a0 20 read_log 194215'18383 (194215'18382) delete 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.2174595.0:12066 0.000000 2014-08-13 17:38:02.769965 7fe9bae887a0 20 read_log 194215'18384 (193305'18350) clean 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by osd.4.0:991 2014-08-12 15:37:01.951216 2014-08-13 17:38:02.769977 7fe9bae887a0 20 read_log 194215'18385 (194215'18384) delete 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.2174595.0:12068 0.000000 2014-08-13 17:38:02.769988 7fe9bae887a0 20 read_log 194215'18386 (193305'18331) clean ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by osd.4.0:993 2014-08-12 15:37:01.958812 2014-08-13 17:38:02.770000 7fe9bae887a0 20 read_log 194215'18387 (194215'18386) delete ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.2174595.0:12070 0.000000 2014-08-13 17:38:02.770011 7fe9bae887a0 20 read_log 194215'18388 (192888'18069) clean 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by osd.4.0:995 2014-08-12 15:37:01.966694 2014-08-13 17:38:02.770023 7fe9bae887a0 20 read_log 194215'18389 (194215'18388) delete 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.2174595.0:12072 0.000000 2014-08-13 17:38:02.770033 7fe9bae887a0 20 read_log 194215'18390 (192888'18038) clean bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by osd.4.0:997 2014-08-12 15:37:01.976625 2014-08-13 17:38:02.770045 7fe9bae887a0 20 read_log 194215'18391 (194215'18390) delete bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.2174595.0:12074 0.000000 2014-08-13 17:38:02.770056 7fe9bae887a0 20 read_log 194215'18392 (192888'18197) clean dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by osd.4.0:999 2014-08-12 15:37:01.984295 2014-08-13 17:38:02.770068 7fe9bae887a0 20 read_log 194215'18393 (194215'18392) delete dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.2174595.0:12076 0.000000 2014-08-13 17:38:02.770078 7fe9bae887a0 20 read_log 194215'18394 (192834'17784) clean 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:1001 2014-08-12 15:37:01.996237 2014-08-13 17:38:02.770090 7fe9bae887a0 20 read_log 194215'18395 (194215'18394) delete 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.2174595.0:12078 0.000000 2014-08-13 17:38:02.770126 7fe9bae887a0 20 read_log 194215'18396 (192888'18200) clean a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by osd.4.0:1003 2014-08-12 15:37:02.019201 2014-08-13 17:38:02.770139 7fe9bae887a0 20 read_log 194215'18397 (194215'18396) delete a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.2174595.0:12080 0.000000 2014-08-13 17:38:02.770150 7fe9bae887a0 20 read_log 194215'18398 (192888'18222) clean db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by osd.4.0:1005 2014-08-12 15:37:02.027941 2014-08-13 17:38:02.770169 7fe9bae887a0 20 read_log 194215'18399 (194215'18398) delete db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.2174595.0:12082 0.000000 2014-08-13 17:38:02.770180 7fe9bae887a0 20 read_log 194215'18400 (193305'18347) clean 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by osd.4.0:1007 2014-08-12 15:37:02.037241 2014-08-13 17:38:02.770192 7fe9bae887a0 20 read_log 194215'18401 (194215'18400) delete 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.2174595.0:12084 0.000000 2014-08-13 17:38:02.770203 7fe9bae887a0 20 read_log 194215'18402 (192834'17761) clean f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:1009 2014-08-12 15:37:02.048324 2014-08-13 17:38:02.770215 7fe9bae887a0 20 read_log 194215'18403 (194215'18402) delete f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.2174595.0:12086 0.000000 2014-08-13 17:38:02.770226 7fe9bae887a0 20 read_log 194215'18404 (192888'18063) clean ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by osd.4.0:1011 2014-08-12 15:37:02.057052 2014-08-13 17:38:02.770238 7fe9bae887a0 20 read_log 194215'18405 (194215'18404) delete ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.2174595.0:12088 0.000000 2014-08-13 17:38:02.770248 7fe9bae887a0 20 read_log 194215'18406 (192888'18068) clean fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by osd.4.0:1013 2014-08-12 15:37:02.065804 2014-08-13 17:38:02.770261 7fe9bae887a0 20 read_log 194215'18407 (194215'18406) delete fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.2174595.0:12090 0.000000 2014-08-13 17:38:02.770271 7fe9bae887a0 20 read_log 194215'18408 (192834'17143) clean 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by osd.4.0:1015 2014-08-12 15:37:02.079168 2014-08-13 17:38:02.770283 7fe9bae887a0 20 read_log 194215'18409 (194215'18408) delete 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.2174595.0:12092 0.000000 2014-08-13 17:38:02.770294 7fe9bae887a0 20 read_log 194215'18410 (192834'17886) clean f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by osd.4.0:1017 2014-08-12 15:37:02.088136 2014-08-13 17:38:02.770306 7fe9bae887a0 20 read_log 194215'18411 (194215'18410) delete f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.2174595.0:12094 0.000000 2014-08-13 17:38:02.770318 7fe9bae887a0 20 read_log 194215'18412 (192888'18199) clean db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by osd.4.0:1019 2014-08-12 15:37:02.096196 2014-08-13 17:38:02.770330 7fe9bae887a0 20 read_log 194215'18413 (194215'18412) delete db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.2174595.0:12096 0.000000 2014-08-13 17:38:02.770341 7fe9bae887a0 20 read_log 194215'18414 (192834'17660) clean 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by osd.4.0:1021 2014-08-12 15:37:02.103915 2014-08-13 17:38:02.770353 7fe9bae887a0 20 read_log 194215'18415 (194215'18414) delete 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.2174595.0:12098 0.000000 2014-08-13 17:38:02.770373 7fe9bae887a0 20 read_log 194215'18416 (193305'18342) clean 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by osd.4.0:1023 2014-08-12 15:37:02.112581 2014-08-13 17:38:02.770386 7fe9bae887a0 20 read_log 194215'18417 (194215'18416) delete 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.2174595.0:12100 0.000000 2014-08-13 17:38:02.770397 7fe9bae887a0 20 read_log 194215'18418 (192888'18029) clean fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by osd.4.0:1025 2014-08-12 15:37:02.119912 2014-08-13 17:38:02.770409 7fe9bae887a0 20 read_log 194215'18419 (194215'18418) delete fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.2174595.0:12102 0.000000 2014-08-13 17:38:02.770420 7fe9bae887a0 20 read_log 194215'18420 (192834'17741) clean 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by osd.4.0:1027 2014-08-12 15:37:02.127355 2014-08-13 17:38:02.770432 7fe9bae887a0 20 read_log 194215'18421 (194215'18420) delete 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.2174595.0:12104 0.000000 2014-08-13 17:38:02.770442 7fe9bae887a0 20 read_log 194215'18422 (192888'18062) clean 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by osd.4.0:1029 2014-08-12 15:37:02.145610 2014-08-13 17:38:02.770454 7fe9bae887a0 20 read_log 194215'18423 (194215'18422) delete 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.2174595.0:12106 0.000000 2014-08-13 17:38:02.770465 7fe9bae887a0 20 read_log 194215'18424 (192888'18227) clean 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by osd.4.0:1031 2014-08-12 15:37:02.154869 2014-08-13 17:38:02.770477 7fe9bae887a0 20 read_log 194215'18425 (194215'18424) delete 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.2174595.0:12108 0.000000 2014-08-13 17:38:02.770488 7fe9bae887a0 20 read_log 194215'18426 (192888'18059) clean 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by osd.4.0:1033 2014-08-12 15:37:02.164544 2014-08-13 17:38:02.770500 7fe9bae887a0 20 read_log 194215'18427 (194215'18426) delete 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.2174595.0:12110 0.000000 2014-08-13 17:38:02.770510 7fe9bae887a0 20 read_log 194215'18428 (192888'18028) clean 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by osd.4.0:1035 2014-08-12 15:37:02.173307 2014-08-13 17:38:02.770522 7fe9bae887a0 20 read_log 194215'18429 (194215'18428) delete 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.2174595.0:12112 0.000000 2014-08-13 17:38:02.770533 7fe9bae887a0 20 read_log 194215'18430 (193305'18327) clean 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by osd.4.0:1037 2014-08-12 15:37:02.182168 2014-08-13 17:38:02.770561 7fe9bae887a0 20 read_log 194215'18431 (194215'18430) delete 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.2174595.0:12114 0.000000 2014-08-13 17:38:02.770572 7fe9bae887a0 20 read_log 194215'18432 (192888'18067) clean ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by osd.4.0:1039 2014-08-12 15:37:02.192336 2014-08-13 17:38:02.770587 7fe9bae887a0 20 read_log 194215'18433 (194215'18432) delete ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.2174595.0:12116 0.000000 2014-08-13 17:38:02.770598 7fe9bae887a0 20 read_log 194215'18434 (192888'18208) clean 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by osd.4.0:1041 2014-08-12 15:37:02.200741 2014-08-13 17:38:02.770610 7fe9bae887a0 20 read_log 194215'18435 (194215'18434) delete 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.2174595.0:12118 0.000000 2014-08-13 17:38:02.770628 7fe9bae887a0 20 read_log 194215'18436 (192888'18075) clean 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by osd.4.0:1043 2014-08-12 15:37:02.209144 2014-08-13 17:38:02.770640 7fe9bae887a0 20 read_log 194215'18437 (194215'18436) delete 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.2174595.0:12120 0.000000 2014-08-13 17:38:02.770651 7fe9bae887a0 20 read_log 194215'18438 (192834'17844) clean 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by osd.4.0:1045 2014-08-12 15:37:02.216903 2014-08-13 17:38:02.770663 7fe9bae887a0 20 read_log 194215'18439 (194215'18438) delete 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.2174595.0:12122 0.000000 2014-08-13 17:38:02.770674 7fe9bae887a0 20 read_log 194215'18440 (192888'18198) clean b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by osd.4.0:1047 2014-08-12 15:37:02.225557 2014-08-13 17:38:02.770686 7fe9bae887a0 20 read_log 194215'18441 (194215'18440) delete b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.2174595.0:12124 0.000000 2014-08-13 17:38:02.770696 7fe9bae887a0 20 read_log 194215'18442 (193305'18352) clean 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by osd.4.0:1049 2014-08-12 15:37:02.234344 2014-08-13 17:38:02.770708 7fe9bae887a0 20 read_log 194215'18443 (194215'18442) delete 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.2174595.0:12126 0.000000 2014-08-13 17:38:02.770739 7fe9bae887a0 20 read_log 194215'18444 (192888'18031) clean 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by osd.4.0:1051 2014-08-12 15:37:02.241818 2014-08-13 17:38:02.770753 7fe9bae887a0 20 read_log 194215'18445 (194215'18444) delete 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.2174595.0:12128 0.000000 2014-08-13 17:38:02.770764 7fe9bae887a0 20 read_log 194215'18446 (192888'18061) clean 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by osd.4.0:1053 2014-08-12 15:37:02.249278 2014-08-13 17:38:02.770776 7fe9bae887a0 20 read_log 194215'18447 (194215'18446) delete 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.2174595.0:12130 0.000000 2014-08-13 17:38:02.770787 7fe9bae887a0 20 read_log 194215'18448 (192888'18207) clean cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by osd.4.0:1055 2014-08-12 15:37:02.257179 2014-08-13 17:38:02.770799 7fe9bae887a0 20 read_log 194215'18449 (194215'18448) delete cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.2174595.0:12132 0.000000 2014-08-13 17:38:02.770809 7fe9bae887a0 20 read_log 194215'18450 (192888'18216) clean 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by osd.4.0:1057 2014-08-12 15:37:02.265073 2014-08-13 17:38:02.770822 7fe9bae887a0 20 read_log 194215'18451 (194215'18450) delete 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.2174595.0:12134 0.000000 2014-08-13 17:38:02.770833 7fe9bae887a0 20 read_log 194215'18452 (192888'18049) clean 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by osd.4.0:1059 2014-08-12 15:37:02.273190 2014-08-13 17:38:02.770845 7fe9bae887a0 20 read_log 194215'18453 (194215'18452) delete 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.2174595.0:12136 0.000000 2014-08-13 17:38:02.770855 7fe9bae887a0 20 read_log 194215'18454 (193305'18334) clean c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by osd.4.0:1061 2014-08-12 15:37:02.282158 2014-08-13 17:38:02.770867 7fe9bae887a0 20 read_log 194215'18455 (194215'18454) delete c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.2174595.0:12138 0.000000 2014-08-13 17:38:02.770885 7fe9bae887a0 20 read_log 194215'18456 (193305'18348) clean b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by osd.4.0:1063 2014-08-12 15:37:02.290968 2014-08-13 17:38:02.770897 7fe9bae887a0 20 read_log 194215'18457 (194215'18456) delete b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.2174595.0:12140 0.000000 2014-08-13 17:38:02.770908 7fe9bae887a0 20 read_log 194215'18458 (192888'18079) clean 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by osd.4.0:1065 2014-08-12 15:37:02.300164 2014-08-13 17:38:02.770920 7fe9bae887a0 20 read_log 194215'18459 (194215'18458) delete 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.2174595.0:12142 0.000000 2014-08-13 17:38:02.770931 7fe9bae887a0 20 read_log 194215'18460 (193305'18319) clean 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by osd.4.0:1067 2014-08-12 15:37:02.308636 2014-08-13 17:38:02.770943 7fe9bae887a0 20 read_log 194215'18461 (194215'18460) delete 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.2174595.0:12144 0.000000 2014-08-13 17:38:02.770953 7fe9bae887a0 20 read_log 194215'18462 (192834'17107) clean 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by osd.4.0:1069 2014-08-12 15:37:02.341275 2014-08-13 17:38:02.770965 7fe9bae887a0 20 read_log 194215'18463 (194215'18462) delete 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.2174595.0:12146 0.000000 2014-08-13 17:38:02.770976 7fe9bae887a0 20 read_log 194215'18464 (192888'18189) clean 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by osd.4.0:1071 2014-08-12 15:37:02.350146 2014-08-13 17:38:02.770989 7fe9bae887a0 20 read_log 194215'18465 (194215'18464) delete 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.2174595.0:12148 0.000000 2014-08-13 17:38:02.771000 7fe9bae887a0 20 read_log 194215'18466 (193305'18322) clean 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by osd.4.0:1073 2014-08-12 15:37:02.359231 2014-08-13 17:38:02.771013 7fe9bae887a0 20 read_log 194215'18467 (194215'18466) delete 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.2174595.0:12150 0.000000 2014-08-13 17:38:02.771024 7fe9bae887a0 20 read_log 194215'18468 (192888'18204) clean 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by osd.4.0:1075 2014-08-12 15:37:02.367070 2014-08-13 17:38:02.771036 7fe9bae887a0 20 read_log 194215'18469 (194215'18468) delete 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.2174595.0:12152 0.000000 2014-08-13 17:38:02.771046 7fe9bae887a0 20 read_log 194215'18470 (192834'17135) clean 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by osd.4.0:1077 2014-08-12 15:37:02.375240 2014-08-13 17:38:02.771059 7fe9bae887a0 20 read_log 194215'18471 (194215'18470) delete 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.2174595.0:12154 0.000000 2014-08-13 17:38:02.771069 7fe9bae887a0 20 read_log 194215'18472 (192888'18042) clean eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by osd.4.0:1079 2014-08-12 15:37:02.383461 2014-08-13 17:38:02.771081 7fe9bae887a0 20 read_log 194215'18473 (194215'18472) delete eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.2174595.0:12156 0.000000 2014-08-13 17:38:02.771092 7fe9bae887a0 20 read_log 194215'18474 (192888'18074) clean 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by osd.4.0:1081 2014-08-12 15:37:02.413267 2014-08-13 17:38:02.771103 7fe9bae887a0 20 read_log 194215'18475 (194215'18474) delete 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.2174595.0:12158 0.000000 2014-08-13 17:38:02.771120 7fe9bae887a0 20 read_log 194215'18476 (192834'17868) clean e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by osd.4.0:1083 2014-08-12 15:37:02.420938 2014-08-13 17:38:02.771133 7fe9bae887a0 20 read_log 194215'18477 (194215'18476) delete e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.2174595.0:12160 0.000000 2014-08-13 17:38:02.771144 7fe9bae887a0 20 read_log 194215'18478 (192888'18044) clean 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by osd.4.0:1085 2014-08-12 15:37:02.429860 2014-08-13 17:38:02.771160 7fe9bae887a0 20 read_log 194215'18479 (194215'18478) delete 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.2174595.0:12162 0.000000 2014-08-13 17:38:02.771171 7fe9bae887a0 20 read_log 194215'18480 (192834'17136) clean 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by osd.4.0:1087 2014-08-12 15:37:02.438798 2014-08-13 17:38:02.771184 7fe9bae887a0 20 read_log 194215'18481 (194215'18480) delete 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.2174595.0:12164 0.000000 2014-08-13 17:38:02.771194 7fe9bae887a0 20 read_log 194215'18482 (192888'18043) clean b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by osd.4.0:1089 2014-08-12 15:37:02.453149 2014-08-13 17:38:02.771206 7fe9bae887a0 20 read_log 194215'18483 (194215'18482) delete b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.2174595.0:12166 0.000000 2014-08-13 17:38:02.771217 7fe9bae887a0 20 read_log 194215'18484 (193305'18351) clean 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by osd.4.0:1091 2014-08-12 15:37:02.461437 2014-08-13 17:38:02.771228 7fe9bae887a0 20 read_log 194215'18485 (194215'18484) delete 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.2174595.0:12168 0.000000 2014-08-13 17:38:02.771239 7fe9bae887a0 20 read_log 194215'18486 (193305'18346) clean f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by osd.4.0:1093 2014-08-12 15:37:02.470504 2014-08-13 17:38:02.771251 7fe9bae887a0 20 read_log 194215'18487 (194215'18486) delete f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.2174595.0:12170 0.000000 2014-08-13 17:38:02.771261 7fe9bae887a0 20 read_log 194215'18488 (193305'18343) clean 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by osd.4.0:1095 2014-08-12 15:37:02.479878 2014-08-13 17:38:02.771275 7fe9bae887a0 20 read_log 194215'18489 (194215'18488) delete 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.2174595.0:12172 0.000000 2014-08-13 17:38:02.771285 7fe9bae887a0 20 read_log 194215'18490 (193305'18340) clean 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by osd.4.0:1097 2014-08-12 15:37:02.489734 2014-08-13 17:38:02.771297 7fe9bae887a0 20 read_log 194215'18491 (194215'18490) delete 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.2174595.0:12174 0.000000 2014-08-13 17:38:02.771308 7fe9bae887a0 20 read_log 194215'18492 (192834'17689) clean 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by osd.4.0:1099 2014-08-12 15:37:02.498004 2014-08-13 17:38:02.771320 7fe9bae887a0 20 read_log 194215'18493 (194215'18492) delete 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.2174595.0:12176 0.000000 2014-08-13 17:38:02.771331 7fe9bae887a0 20 read_log 194215'18494 (193305'18337) clean 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by osd.4.0:1101 2014-08-12 15:37:02.506158 2014-08-13 17:38:02.771343 7fe9bae887a0 20 read_log 194215'18495 (194215'18494) delete 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.2174595.0:12178 0.000000 2014-08-13 17:38:02.771366 7fe9bae887a0 20 read_log 194215'18496 (193305'18336) clean b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by osd.4.0:1103 2014-08-12 15:37:02.514067 2014-08-13 17:38:02.771379 7fe9bae887a0 20 read_log 194215'18497 (194215'18496) delete b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.2174595.0:12180 0.000000 2014-08-13 17:38:02.771390 7fe9bae887a0 20 read_log 194215'18498 (192834'17706) clean 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by osd.4.0:1105 2014-08-12 15:37:02.532379 2014-08-13 17:38:02.771402 7fe9bae887a0 20 read_log 194215'18499 (194215'18498) delete 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.2174595.0:12182 0.000000 2014-08-13 17:38:02.771413 7fe9bae887a0 20 read_log 194215'18500 (192888'18082) clean 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by osd.4.0:1107 2014-08-12 15:37:02.542050 2014-08-13 17:38:02.771425 7fe9bae887a0 20 read_log 194215'18501 (194215'18500) delete 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.2174595.0:12184 0.000000 2014-08-13 17:38:02.771435 7fe9bae887a0 20 read_log 194215'18502 (192834'17749) clean 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by osd.4.0:1109 2014-08-12 15:37:02.551334 2014-08-13 17:38:02.771447 7fe9bae887a0 20 read_log 194215'18503 (194215'18502) delete 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.2174595.0:12186 0.000000 2014-08-13 17:38:02.771458 7fe9bae887a0 20 read_log 194215'18504 (193305'18349) clean 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by osd.4.0:1111 2014-08-12 15:37:02.558801 2014-08-13 17:38:02.771470 7fe9bae887a0 20 read_log 194215'18505 (194215'18504) delete 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.2174595.0:12188 0.000000 2014-08-13 17:38:02.771480 7fe9bae887a0 20 read_log 194215'18506 (193305'18332) clean 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by osd.4.0:1113 2014-08-12 15:37:02.566321 2014-08-13 17:38:02.771492 7fe9bae887a0 20 read_log 194215'18507 (194215'18506) delete 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.2174595.0:12190 0.000000 2014-08-13 17:38:02.771502 7fe9bae887a0 20 read_log 194215'18508 (193305'18338) clean c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by osd.4.0:1115 2014-08-12 15:37:02.574227 2014-08-13 17:38:02.771514 7fe9bae887a0 20 read_log 194215'18509 (194215'18508) delete c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.2174595.0:12192 0.000000 2014-08-13 17:38:02.771525 7fe9bae887a0 20 read_log 194215'18510 (192834'17890) clean 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by osd.4.0:1117 2014-08-12 15:37:02.582277 2014-08-13 17:38:02.771536 7fe9bae887a0 20 read_log 194215'18511 (194215'18510) delete 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.2174595.0:12194 0.000000 2014-08-13 17:38:02.771547 7fe9bae887a0 20 read_log 194215'18512 (193305'18328) clean 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by osd.4.0:1119 2014-08-12 15:37:02.616040 2014-08-13 17:38:02.771559 7fe9bae887a0 20 read_log 194215'18513 (194215'18512) delete 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.2174595.0:12196 0.000000 2014-08-13 17:38:02.771570 7fe9bae887a0 20 read_log 194215'18514 (193305'18326) clean a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by osd.4.0:1121 2014-08-12 15:37:02.626012 2014-08-13 17:38:02.771582 7fe9bae887a0 20 read_log 194215'18515 (194215'18514) delete a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.2174595.0:12198 0.000000 2014-08-13 17:38:02.771603 7fe9bae887a0 20 read_log 194215'18516 (192834'17112) clean f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by osd.4.0:1123 2014-08-12 15:37:02.634701 2014-08-13 17:38:02.771616 7fe9bae887a0 20 read_log 194215'18517 (194215'18516) delete f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.2174595.0:12200 0.000000 2014-08-13 17:38:02.771626 7fe9bae887a0 20 read_log 194215'18518 (192888'18206) clean 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by osd.4.0:1125 2014-08-12 15:37:02.652792 2014-08-13 17:38:02.771639 7fe9bae887a0 20 read_log 194215'18519 (194215'18518) delete 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.2174595.0:12202 0.000000 2014-08-13 17:38:02.771649 7fe9bae887a0 20 read_log 194215'18520 (193305'18344) clean fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by osd.4.0:1127 2014-08-12 15:37:02.660799 2014-08-13 17:38:02.771661 7fe9bae887a0 20 read_log 194215'18521 (194215'18520) delete fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.2174595.0:12204 0.000000 2014-08-13 17:38:02.771672 7fe9bae887a0 20 read_log 194215'18522 (192888'18209) clean e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by osd.4.0:1129 2014-08-12 15:37:02.670602 2014-08-13 17:38:02.771684 7fe9bae887a0 20 read_log 194215'18523 (194215'18522) delete e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.2174595.0:12206 0.000000 2014-08-13 17:38:02.771710 7fe9bae887a0 20 read_log 194215'18524 (192834'17827) clean 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by osd.4.0:1131 2014-08-12 15:37:02.678315 2014-08-13 17:38:02.771723 7fe9bae887a0 20 read_log 194215'18525 (194215'18524) delete 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.2174595.0:12208 0.000000 2014-08-13 17:38:02.771734 7fe9bae887a0 20 read_log 194215'18526 (192888'18230) clean 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by osd.4.0:1133 2014-08-12 15:37:02.685349 2014-08-13 17:38:02.771746 7fe9bae887a0 20 read_log 194215'18527 (194215'18526) delete 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.2174595.0:12210 0.000000 2014-08-13 17:38:02.771757 7fe9bae887a0 20 read_log 194215'18528 (192888'18064) clean fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by osd.4.0:1135 2014-08-12 15:37:02.693546 2014-08-13 17:38:02.771769 7fe9bae887a0 20 read_log 194215'18529 (194215'18528) delete fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.2174595.0:12212 0.000000 2014-08-13 17:38:02.771779 7fe9bae887a0 20 read_log 194215'18530 (192834'17172) clean 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by osd.4.0:1137 2014-08-12 15:37:02.716647 2014-08-13 17:38:02.771792 7fe9bae887a0 20 read_log 194215'18531 (194215'18530) delete 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.2174595.0:12214 0.000000 2014-08-13 17:38:02.771802 7fe9bae887a0 20 read_log 194215'18532 (192888'18060) clean e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by osd.4.0:1139 2014-08-12 15:37:02.726140 2014-08-13 17:38:02.771814 7fe9bae887a0 20 read_log 194215'18533 (194215'18532) delete e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.2174595.0:12216 0.000000 2014-08-13 17:38:02.771825 7fe9bae887a0 20 read_log 194215'18534 (192834'17746) clean 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by osd.4.0:1141 2014-08-12 15:37:02.734727 2014-08-13 17:38:02.771837 7fe9bae887a0 20 read_log 194215'18535 (194215'18534) delete 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.2174595.0:12218 0.000000 2014-08-13 17:38:02.771870 7fe9bae887a0 20 read_log 194215'18536 (192888'18052) clean 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by osd.4.0:1143 2014-08-12 15:37:02.742385 2014-08-13 17:38:02.771883 7fe9bae887a0 20 read_log 194215'18537 (194215'18536) delete 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.2174595.0:12220 0.000000 2014-08-13 17:38:02.771894 7fe9bae887a0 20 read_log 194215'18538 (193305'18330) clean cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by osd.4.0:1145 2014-08-12 15:37:02.750312 2014-08-13 17:38:02.771907 7fe9bae887a0 20 read_log 194215'18539 (194215'18538) delete cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.2174595.0:12222 0.000000 2014-08-13 17:38:02.771917 7fe9bae887a0 20 read_log 194215'18540 (192888'18035) clean 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by osd.4.0:1147 2014-08-12 15:37:02.773783 2014-08-13 17:38:02.771930 7fe9bae887a0 20 read_log 194215'18541 (194215'18540) delete 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.2174595.0:12224 0.000000 2014-08-13 17:38:02.771940 7fe9bae887a0 20 read_log 194215'18542 (192834'17086) clean 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by osd.4.0:1149 2014-08-12 15:37:02.783017 2014-08-13 17:38:02.771952 7fe9bae887a0 20 read_log 194215'18543 (194215'18542) delete 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.2174595.0:12226 0.000000 2014-08-13 17:38:02.771963 7fe9bae887a0 20 read_log 194215'18544 (192834'17748) clean 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by osd.4.0:1151 2014-08-12 15:37:02.790729 2014-08-13 17:38:02.771975 7fe9bae887a0 20 read_log 194215'18545 (194215'18544) delete 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.2174595.0:12228 0.000000 2014-08-13 17:38:02.771986 7fe9bae887a0 20 read_log 194215'18546 (193305'18317) clean 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by osd.4.0:1153 2014-08-12 15:37:02.800528 2014-08-13 17:38:02.771998 7fe9bae887a0 20 read_log 194215'18547 (194215'18546) delete 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.2174595.0:12230 0.000000 2014-08-13 17:38:02.772008 7fe9bae887a0 20 read_log 194215'18548 (193305'18324) clean 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by osd.4.0:1155 2014-08-12 15:37:02.809078 2014-08-13 17:38:02.772020 7fe9bae887a0 20 read_log 194215'18549 (194215'18548) delete 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.2174595.0:12232 0.000000 2014-08-13 17:38:02.772031 7fe9bae887a0 20 read_log 194215'18550 (192834'17708) clean d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by osd.4.0:1157 2014-08-12 15:37:02.817771 2014-08-13 17:38:02.772043 7fe9bae887a0 20 read_log 194215'18551 (194215'18550) delete d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.2174595.0:12234 0.000000 2014-08-13 17:38:02.772053 7fe9bae887a0 20 read_log 194215'18552 (192888'18213) clean 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by osd.4.0:1159 2014-08-12 15:37:02.825476 2014-08-13 17:38:02.772065 7fe9bae887a0 20 read_log 194215'18553 (194215'18552) delete 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.2174595.0:12236 0.000000 2014-08-13 17:38:02.772076 7fe9bae887a0 20 read_log 194215'18554 (192888'18201) clean 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by osd.4.0:1161 2014-08-12 15:37:02.852203 2014-08-13 17:38:02.772087 7fe9bae887a0 20 read_log 194215'18555 (194215'18554) delete 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.2174595.0:12238 0.000000 2014-08-13 17:38:02.772107 7fe9bae887a0 20 read_log 194215'18556 (192834'17137) clean 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by osd.4.0:1163 2014-08-12 15:37:02.861156 2014-08-13 17:38:02.772120 7fe9bae887a0 20 read_log 194215'18557 (194215'18556) delete 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.2174595.0:12240 0.000000 2014-08-13 17:38:02.772130 7fe9bae887a0 20 read_log 194215'18558 (193305'18353) clean db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by osd.4.0:1165 2014-08-12 15:37:02.869845 2014-08-13 17:38:02.772142 7fe9bae887a0 20 read_log 194215'18559 (194215'18558) delete db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.2174595.0:12242 0.000000 2014-08-13 17:38:02.772157 7fe9bae887a0 20 read_log 194215'18560 (192834'17836) clean 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by osd.4.0:1167 2014-08-12 15:37:02.878329 2014-08-13 17:38:02.772173 7fe9bae887a0 20 read_log 194215'18561 (194215'18560) delete 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.2174595.0:12244 0.000000 2014-08-13 17:38:02.772184 7fe9bae887a0 20 read_log 194215'18562 (192888'18235) clean eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by osd.4.0:1169 2014-08-12 15:37:02.887101 2014-08-13 17:38:02.772196 7fe9bae887a0 20 read_log 194215'18563 (194215'18562) delete eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.2174595.0:12246 0.000000 2014-08-13 17:38:02.772206 7fe9bae887a0 20 read_log 194215'18564 (193305'18320) clean ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by osd.4.0:1171 2014-08-12 15:37:02.895564 2014-08-13 17:38:02.772219 7fe9bae887a0 20 read_log 194215'18565 (194215'18564) delete ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.2174595.0:12248 0.000000 2014-08-13 17:38:02.772229 7fe9bae887a0 20 read_log 194215'18566 (192834'17063) clean 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by osd.4.0:1173 2014-08-12 15:37:02.904688 2014-08-13 17:38:02.772241 7fe9bae887a0 20 read_log 194215'18567 (194215'18566) delete 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.2174595.0:12250 0.000000 2014-08-13 17:38:02.772252 7fe9bae887a0 20 read_log 194215'18568 (192834'17899) clean 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by osd.4.0:1175 2014-08-12 15:37:02.914372 2014-08-13 17:38:02.772264 7fe9bae887a0 20 read_log 194215'18569 (194215'18568) delete 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.2174595.0:12252 0.000000 2014-08-13 17:38:02.772274 7fe9bae887a0 20 read_log 194215'18570 (192888'18024) clean c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by osd.4.0:1177 2014-08-12 15:37:02.923266 2014-08-13 17:38:02.772286 7fe9bae887a0 20 read_log 194215'18571 (194215'18570) delete c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.2174595.0:12254 0.000000 2014-08-13 17:38:02.772297 7fe9bae887a0 20 read_log 194215'18572 (192834'17716) clean 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by osd.4.0:1179 2014-08-12 15:37:02.940487 2014-08-13 17:38:02.772309 7fe9bae887a0 20 read_log 194215'18573 (194215'18572) delete 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.2174595.0:12256 0.000000 2014-08-13 17:38:02.772319 7fe9bae887a0 20 read_log 194215'18574 (192834'17192) clean 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by osd.4.0:1181 2014-08-12 15:37:02.961715 2014-08-13 17:38:02.772331 7fe9bae887a0 20 read_log 194215'18575 (194215'18574) delete 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.2174595.0:12258 0.000000 2014-08-13 17:38:02.772349 7fe9bae887a0 20 read_log 194215'18576 (192888'18077) clean a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by osd.4.0:1183 2014-08-12 15:37:02.970770 2014-08-13 17:38:02.772361 7fe9bae887a0 20 read_log 194215'18577 (194215'18576) delete a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.2174595.0:12260 0.000000 2014-08-13 17:38:02.772372 7fe9bae887a0 20 read_log 194215'18578 (192888'18194) clean 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by osd.4.0:1185 2014-08-12 15:37:02.978963 2014-08-13 17:38:02.772384 7fe9bae887a0 20 read_log 194215'18579 (194215'18578) delete 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.2174595.0:12262 0.000000 2014-08-13 17:38:02.772394 7fe9bae887a0 20 read_log 194215'18580 (193305'18329) clean 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by osd.4.0:1187 2014-08-12 15:37:02.986258 2014-08-13 17:38:02.772407 7fe9bae887a0 20 read_log 194215'18581 (194215'18580) delete 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.2174595.0:12264 0.000000 2014-08-13 17:38:02.772417 7fe9bae887a0 20 read_log 194215'18582 (193305'18325) clean 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by osd.4.0:1189 2014-08-12 15:37:02.995171 2014-08-13 17:38:02.772429 7fe9bae887a0 20 read_log 194215'18583 (194215'18582) delete 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.2174595.0:12266 0.000000 2014-08-13 17:38:02.772440 7fe9bae887a0 20 read_log 194215'18584 (192888'18218) clean 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by osd.4.0:1191 2014-08-12 15:37:03.002944 2014-08-13 17:38:02.772452 7fe9bae887a0 20 read_log 194215'18585 (194215'18584) delete 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.2174595.0:12268 0.000000 2014-08-13 17:38:02.772463 7fe9bae887a0 20 read_log 194215'18586 (193305'18321) clean 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by osd.4.0:1193 2014-08-12 15:37:03.010354 2014-08-13 17:38:02.772475 7fe9bae887a0 20 read_log 194215'18587 (194215'18586) delete 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.2174595.0:12270 0.000000 2014-08-13 17:38:02.772485 7fe9bae887a0 20 read_log 194215'18588 (192834'17897) clean 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by osd.4.0:1195 2014-08-12 15:37:03.018437 2014-08-13 17:38:02.772497 7fe9bae887a0 20 read_log 194215'18589 (194215'18588) delete 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.2174595.0:12272 0.000000 2014-08-13 17:38:02.772508 7fe9bae887a0 20 read_log 194215'18590 (192888'18203) clean 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by osd.4.0:1197 2014-08-12 15:37:03.030592 2014-08-13 17:38:02.772520 7fe9bae887a0 20 read_log 194215'18591 (194215'18590) delete 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.2174595.0:12274 0.000000 2014-08-13 17:38:02.772531 7fe9bae887a0 20 read_log 194215'18592 (193305'18333) clean 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by osd.4.0:1199 2014-08-12 15:37:03.061077 2014-08-13 17:38:02.772543 7fe9bae887a0 20 read_log 194215'18593 (194215'18592) delete 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.2174595.0:12276 0.000000 2014-08-13 17:38:02.772553 7fe9bae887a0 20 read_log 194215'18594 (192888'18192) clean 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by osd.4.0:1201 2014-08-12 15:37:03.071722 2014-08-13 17:38:02.772565 7fe9bae887a0 20 read_log 194215'18595 (194215'18594) delete 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.2174595.0:12278 0.000000 2014-08-13 17:38:02.772582 7fe9bae887a0 20 read_log 194215'18596 (192888'18058) clean 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by osd.4.0:1203 2014-08-12 15:37:03.079988 2014-08-13 17:38:02.772595 7fe9bae887a0 20 read_log 194215'18597 (194215'18596) delete 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.2174595.0:12280 0.000000 2014-08-13 17:38:02.772606 7fe9bae887a0 20 read_log 194215'18598 (192888'18066) clean 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by osd.4.0:1205 2014-08-12 15:37:03.087543 2014-08-13 17:38:02.772618 7fe9bae887a0 20 read_log 194215'18599 (194215'18598) delete 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.2174595.0:12282 0.000000 2014-08-13 17:38:02.772628 7fe9bae887a0 20 read_log 194215'18600 (192888'18237) clean c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by osd.4.0:1207 2014-08-12 15:37:03.095270 2014-08-13 17:38:02.772640 7fe9bae887a0 20 read_log 194215'18601 (194215'18600) delete c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.2174595.0:12284 0.000000 2014-08-13 17:38:02.772653 7fe9bae887a0 20 read_log 194215'18602 (193305'18341) clean cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by osd.4.0:1209 2014-08-12 15:37:03.102213 2014-08-13 17:38:02.772665 7fe9bae887a0 20 read_log 194215'18603 (194215'18602) delete cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.2174595.0:12286 0.000000 2014-08-13 17:38:02.772676 7fe9bae887a0 20 read_log 194215'18604 (192834'17835) clean 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by osd.4.0:1211 2014-08-12 15:37:03.110132 2014-08-13 17:38:02.772688 7fe9bae887a0 20 read_log 194215'18605 (194215'18604) delete 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.2174595.0:12288 0.000000 2014-08-13 17:38:02.772698 7fe9bae887a0 20 read_log 194215'18606 (192888'18034) clean c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by osd.4.0:1213 2014-08-12 15:37:03.117352 2014-08-13 17:38:02.772711 7fe9bae887a0 20 read_log 194215'18607 (194215'18606) delete c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.2174595.0:12290 0.000000 2014-08-13 17:38:02.772721 7fe9bae887a0 20 read_log 194215'18608 (193305'18345) clean e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by osd.4.0:1215 2014-08-12 15:37:03.125730 2014-08-13 17:38:02.772733 7fe9bae887a0 20 read_log 194215'18609 (194215'18608) delete e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.2174595.0:12292 0.000000 2014-08-13 17:38:02.772744 7fe9bae887a0 20 read_log 194215'18610 (194215'18610) modify 3d/hit_set_24.3d_archive_2014-08-12 15:32:25.863067_2014-08-12 17:24:40.629052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.630222 2014-08-13 17:38:02.772756 7fe9bae887a0 20 read_log 194215'18611 (193305'18241) delete 3d/hit_set_24.3d_archive_2014-08-11 18:37:04.904929_2014-08-11 18:42:35.578890/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.630222 2014-08-13 17:38:02.772768 7fe9bae887a0 20 read_log 194215'18612 (0'0) promote 2fbabc3d/benchmark_data_storage0111-ib.pocstor_34628_object187/head//24 by osd.4.0:5229 2014-08-12 17:24:40.677366 2014-08-13 17:38:02.772781 7fe9bae887a0 20 read_log 194215'18613 (194215'18612) modify 2fbabc3d/benchmark_data_storage0111-ib.pocstor_34628_object187/head//24 by client.2193759.0:188 2014-08-12 17:24:40.620380 2014-08-13 17:38:02.772793 7fe9bae887a0 20 read_log 194215'18614 (0'0) promote ff60cc3d/benchmark_data_storage0111-ib.pocstor_34628_object1470/head//24 by osd.4.0:5248 2014-08-12 17:24:46.823842 2014-08-13 17:38:02.772814 7fe9bae887a0 20 read_log 194215'18615 (194215'18614) modify ff60cc3d/benchmark_data_storage0111-ib.pocstor_34628_object1470/head//24 by client.2193759.0:1471 2014-08-12 17:24:46.814795 2014-08-13 17:38:02.772827 7fe9bae887a0 20 read_log 194370'18616 (0'0) promote 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by osd.20.0:4204 2014-08-12 20:30:20.439055 2014-08-13 17:38:02.772857 7fe9bae887a0 20 read_log 194370'18617 (194370'18616) modify 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by client.1508531.0:1616 2014-08-12 20:30:20.430889 2014-08-13 17:38:02.772870 7fe9bae887a0 20 read_log 194370'18618 (0'0) promote cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by osd.20.0:4250 2014-08-12 20:30:27.367180 2014-08-13 17:38:02.772882 7fe9bae887a0 20 read_log 194370'18619 (194370'18618) modify cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by client.1508531.0:3488 2014-08-12 20:30:27.359030 2014-08-13 17:38:02.772895 7fe9bae887a0 20 read_log 194370'18620 (0'0) promote 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by osd.20.0:4266 2014-08-12 20:30:29.758622 2014-08-13 17:38:02.772907 7fe9bae887a0 20 read_log 194370'18621 (194370'18620) modify 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by client.1508531.0:4127 2014-08-12 20:30:29.751699 2014-08-13 17:38:02.772919 7fe9bae887a0 20 read_log 194370'18622 (0'0) promote 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by osd.20.0:4310 2014-08-12 20:30:41.002392 2014-08-13 17:38:02.772931 7fe9bae887a0 20 read_log 194370'18623 (194370'18622) modify 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by client.1508531.0:6728 2014-08-12 20:30:40.992116 2014-08-13 17:38:02.772943 7fe9bae887a0 20 read_log 194370'18624 (0'0) promote fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by osd.20.0:4358 2014-08-12 20:30:51.154468 2014-08-13 17:38:02.772955 7fe9bae887a0 20 read_log 194370'18625 (194370'18624) modify fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by client.1508531.0:8648 2014-08-12 20:30:51.141855 2014-08-13 17:38:02.772967 7fe9bae887a0 20 read_log 194370'18626 (0'0) promote cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by osd.20.0:4380 2014-08-12 20:30:55.175714 2014-08-13 17:38:02.772978 7fe9bae887a0 20 read_log 194370'18627 (194370'18626) modify cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by client.1508531.0:9310 2014-08-12 20:30:55.167609 2014-08-13 17:38:02.772990 7fe9bae887a0 20 read_log 194370'18628 (0'0) promote f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by osd.20.0:4388 2014-08-12 20:30:57.263858 2014-08-13 17:38:02.773002 7fe9bae887a0 20 read_log 194370'18629 (194370'18628) modify f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by client.1508531.0:9659 2014-08-12 20:30:57.255203 2014-08-13 17:38:02.773014 7fe9bae887a0 20 read_log 194370'18630 (0'0) promote 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by osd.20.0:4398 2014-08-12 20:31:00.147671 2014-08-13 17:38:02.773026 7fe9bae887a0 20 read_log 194370'18631 (194370'18630) modify 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by client.1508531.0:10152 2014-08-12 20:31:00.138476 2014-08-13 17:38:02.773038 7fe9bae887a0 20 read_log 194370'18632 (0'0) promote fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by osd.20.0:4404 2014-08-12 20:31:00.957385 2014-08-13 17:38:02.773050 7fe9bae887a0 20 read_log 194370'18633 (194370'18632) modify fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by client.1508531.0:10288 2014-08-12 20:31:00.946368 2014-08-13 17:38:02.773062 7fe9bae887a0 20 read_log 194370'18634 (0'0) promote d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by osd.20.0:4420 2014-08-12 20:31:04.922557 2014-08-13 17:38:02.773082 7fe9bae887a0 20 read_log 194370'18635 (194370'18634) modify d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by client.1508531.0:10941 2014-08-12 20:31:04.914356 2014-08-13 17:38:02.773096 7fe9bae887a0 20 read_log 194370'18636 (0'0) promote 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by osd.20.0:4458 2014-08-12 20:31:19.062773 2014-08-13 17:38:02.773108 7fe9bae887a0 20 read_log 194370'18637 (194370'18636) modify 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by client.1508531.0:12968 2014-08-12 20:31:19.055563 2014-08-13 17:38:02.773120 7fe9bae887a0 20 read_log 194370'18638 (0'0) promote 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by osd.20.0:4460 2014-08-12 20:31:19.808728 2014-08-13 17:38:02.773132 7fe9bae887a0 20 read_log 194370'18639 (194370'18638) modify 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by client.1508531.0:13048 2014-08-12 20:31:19.800706 2014-08-13 17:38:02.773147 7fe9bae887a0 20 read_log 194370'18640 (0'0) promote 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by osd.20.0:4462 2014-08-12 20:31:22.383879 2014-08-13 17:38:02.773183 7fe9bae887a0 20 read_log 194370'18641 (194370'18640) modify 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by client.1508531.0:13314 2014-08-12 20:31:22.374937 2014-08-13 17:38:02.773196 7fe9bae887a0 20 read_log 194377'18642 (0'0) promote 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by osd.20.0:4472 2014-08-12 20:32:58.011238 2014-08-13 17:38:02.773208 7fe9bae887a0 20 read_log 194377'18643 (194377'18642) modify 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by client.2231190.0:204 2014-08-12 20:32:58.004900 2014-08-13 17:38:02.773221 7fe9bae887a0 20 read_log 194377'18644 (0'0) promote a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by osd.20.0:4490 2014-08-12 20:33:00.913860 2014-08-13 17:38:02.773233 7fe9bae887a0 20 read_log 194377'18645 (194377'18644) modify a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by client.2231190.0:896 2014-08-12 20:33:00.907409 2014-08-13 17:38:02.773245 7fe9bae887a0 20 read_log 194377'18646 (0'0) promote cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by osd.20.0:4496 2014-08-12 20:33:02.030736 2014-08-13 17:38:02.773257 7fe9bae887a0 20 read_log 194377'18647 (194377'18646) modify cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by client.2231190.0:1192 2014-08-12 20:33:02.023014 2014-08-13 17:38:02.773272 7fe9bae887a0 20 read_log 194377'18648 (0'0) promote 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by osd.20.0:4504 2014-08-12 20:33:03.021960 2014-08-13 17:38:02.773284 7fe9bae887a0 20 read_log 194377'18649 (194377'18648) modify 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by client.2231190.0:1453 2014-08-12 20:33:03.014348 2014-08-13 17:38:02.773297 7fe9bae887a0 20 read_log 194377'18650 (0'0) promote 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by osd.20.0:4514 2014-08-12 20:33:06.215836 2014-08-13 17:38:02.773308 7fe9bae887a0 20 read_log 194377'18651 (194377'18650) modify 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by client.2231190.0:2241 2014-08-12 20:33:06.209363 2014-08-13 17:38:02.773321 7fe9bae887a0 20 read_log 194377'18652 (0'0) promote cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by osd.20.0:4536 2014-08-12 20:33:08.897041 2014-08-13 17:38:02.773333 7fe9bae887a0 20 read_log 194377'18653 (194377'18652) modify cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by client.2231190.0:2905 2014-08-12 20:33:08.890178 2014-08-13 17:38:02.773345 7fe9bae887a0 20 read_log 194377'18654 (0'0) promote 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by osd.20.0:4540 2014-08-12 20:33:09.682593 2014-08-13 17:38:02.773364 7fe9bae887a0 20 read_log 194377'18655 (194377'18654) modify 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by client.2231190.0:3092 2014-08-12 20:33:09.673777 2014-08-13 17:38:02.773377 7fe9bae887a0 20 read_log 194377'18656 (0'0) promote 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by osd.20.0:4558 2014-08-12 20:33:14.126715 2014-08-13 17:38:02.773393 7fe9bae887a0 20 read_log 194377'18657 (194377'18656) modify 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by client.2231190.0:4036 2014-08-12 20:33:14.118216 2014-08-13 17:38:02.773406 7fe9bae887a0 20 read_log 194377'18658 (0'0) promote 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by osd.20.0:4576 2014-08-12 20:33:18.891187 2014-08-13 17:38:02.773418 7fe9bae887a0 20 read_log 194377'18659 (194377'18658) modify 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by client.2231190.0:4736 2014-08-12 20:33:18.883461 2014-08-13 17:38:02.773430 7fe9bae887a0 20 read_log 194383'18660 (0'0) promote a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by osd.20.0:4614 2014-08-12 20:34:03.570988 2014-08-13 17:38:02.773442 7fe9bae887a0 20 read_log 194383'18661 (194383'18660) modify a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by client.2231190.0:6394 2014-08-12 20:34:03.564216 2014-08-13 17:38:02.773455 7fe9bae887a0 20 read_log 194383'18662 (0'0) promote abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by osd.20.0:4620 2014-08-12 20:34:17.976519 2014-08-13 17:38:02.773467 7fe9bae887a0 20 read_log 194383'18663 (194383'18662) modify abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by client.2231190.0:7104 2014-08-12 20:34:17.967809 2014-08-13 17:38:02.773479 7fe9bae887a0 20 read_log 194393'18664 (194393'18664) modify 3d/hit_set_24.3d_archive_2014-08-12 20:37:08.991137_2014-08-12 20:47:27.683414/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.684567 2014-08-13 17:38:02.773492 7fe9bae887a0 20 read_log 194393'18665 (194215'18610) delete 3d/hit_set_24.3d_archive_2014-08-12 15:32:25.863067_2014-08-12 17:24:40.629052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.684567 2014-08-13 17:38:02.773504 7fe9bae887a0 20 read_log 194393'18666 (0'0) promote 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by osd.20.0:4648 2014-08-12 20:47:27.689016 2014-08-13 17:38:02.773516 7fe9bae887a0 20 read_log 194393'18667 (194393'18666) modify 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by client.1064095.0:409 2014-08-12 20:47:27.678974 2014-08-13 17:38:02.773528 7fe9bae887a0 20 read_log 194393'18668 (0'0) promote 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by osd.20.0:4653 2014-08-12 20:47:28.148894 2014-08-13 17:38:02.773540 7fe9bae887a0 20 read_log 194393'18669 (194393'18668) modify 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by client.1064095.0:516 2014-08-12 20:47:28.139439 2014-08-13 17:38:02.773552 7fe9bae887a0 20 read_log 194393'18670 (0'0) promote d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by osd.20.0:4673 2014-08-12 20:49:26.743759 2014-08-13 17:38:02.773565 7fe9bae887a0 20 read_log 194393'18671 (194393'18670) modify d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by client.1064182.0:270 2014-08-12 20:49:26.734751 2014-08-13 17:38:02.773577 7fe9bae887a0 20 read_log 194416'18672 (194416'18672) modify 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 17:38:02.773589 7fe9bae887a0 20 read_log 194416'18673 (194393'18664) delete 3d/hit_set_24.3d_archive_2014-08-12 20:37:08.991137_2014-08-12 20:47:27.683414/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 17:38:02.773629 7fe9bae887a0 20 read_log 194416'18674 (0'0) promote 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by osd.20.0:4862 2014-08-13 15:28:30.882411 2014-08-13 17:38:02.773642 7fe9bae887a0 20 read_log 194416'18675 (194416'18674) modify 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by client.2441958.0:179 2014-08-13 15:28:30.869401 2014-08-13 17:38:02.773655 7fe9bae887a0 20 read_log 194416'18676 (0'0) promote 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by osd.20.0:4867 2014-08-13 15:28:31.482821 2014-08-13 17:38:02.773666 7fe9bae887a0 20 read_log 194416'18677 (194416'18676) modify 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by client.2441958.0:306 2014-08-13 15:28:31.473163 2014-08-13 17:38:02.773679 7fe9bae887a0 20 read_log 194416'18678 (0'0) promote d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by osd.20.0:4927 2014-08-13 15:28:39.112470 2014-08-13 17:38:02.773690 7fe9bae887a0 20 read_log 194416'18679 (194416'18678) modify d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by client.2441958.0:2221 2014-08-13 15:28:39.104322 2014-08-13 17:38:02.773702 7fe9bae887a0 20 read_log 194416'18680 (0'0) promote 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by osd.20.0:4997 2014-08-13 15:28:51.030775 2014-08-13 17:38:02.773714 7fe9bae887a0 20 read_log 194416'18681 (194416'18680) modify 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by client.2441958.0:5296 2014-08-13 15:28:51.021478 2014-08-13 17:38:02.773726 7fe9bae887a0 20 read_log 194416'18682 (0'0) promote 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by osd.20.0:5007 2014-08-13 15:28:52.934138 2014-08-13 17:38:02.773738 7fe9bae887a0 20 read_log 194416'18683 (194416'18682) modify 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by client.2441958.0:5788 2014-08-13 15:28:52.927293 2014-08-13 17:38:02.773750 7fe9bae887a0 20 read_log 194416'18684 (0'0) promote b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by osd.20.0:5027 2014-08-13 15:28:56.474887 2014-08-13 17:38:02.773762 7fe9bae887a0 20 read_log 194416'18685 (194416'18684) modify b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by client.2441958.0:6649 2014-08-13 15:28:56.466596 2014-08-13 17:38:02.773774 7fe9bae887a0 20 read_log 194416'18686 (0'0) promote edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by osd.20.0:5041 2014-08-13 15:28:59.609677 2014-08-13 17:38:02.773786 7fe9bae887a0 20 read_log 194416'18687 (194416'18686) modify edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by client.2441958.0:7488 2014-08-13 15:28:59.601755 2014-08-13 17:38:02.773798 7fe9bae887a0 20 read_log 194416'18688 (0'0) promote c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by osd.20.0:5057 2014-08-13 15:29:02.620130 2014-08-13 17:38:02.773810 7fe9bae887a0 20 read_log 194416'18689 (194416'18688) modify c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by client.2441958.0:8290 2014-08-13 15:29:02.611334 2014-08-13 17:38:02.773822 7fe9bae887a0 20 read_log 194416'18690 (0'0) promote 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by osd.20.0:5065 2014-08-13 15:29:03.156511 2014-08-13 17:38:02.773835 7fe9bae887a0 20 read_log 194416'18691 (194416'18690) modify 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by client.2441958.0:8429 2014-08-13 15:29:03.146674 2014-08-13 17:38:02.773848 7fe9bae887a0 20 read_log 194416'18692 (0'0) promote 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by osd.20.0:5093 2014-08-13 15:29:08.855277 2014-08-13 17:38:02.773860 7fe9bae887a0 20 read_log 194416'18693 (194416'18692) modify 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by client.2441958.0:9868 2014-08-13 15:29:08.847037 2014-08-13 17:38:02.773879 7fe9bae887a0 20 read_log 194416'18694 (0'0) promote ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by osd.20.0:5121 2014-08-13 15:29:12.011260 2014-08-13 17:38:02.773892 7fe9bae887a0 20 read_log 194416'18695 (194416'18694) modify ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by client.2441958.0:10653 2014-08-13 15:29:12.001679 2014-08-13 17:38:02.773904 7fe9bae887a0 20 read_log 194416'18696 (0'0) promote 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by osd.20.0:5123 2014-08-13 15:29:12.325346 2014-08-13 17:38:02.773916 7fe9bae887a0 20 read_log 194416'18697 (194416'18696) modify 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by client.2441958.0:10738 2014-08-13 15:29:12.316890 2014-08-13 17:38:02.773928 7fe9bae887a0 20 read_log 194416'18698 (0'0) promote ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by osd.20.0:5127 2014-08-13 15:29:12.981978 2014-08-13 17:38:02.773940 7fe9bae887a0 20 read_log 194416'18699 (194416'18698) modify ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by client.2441958.0:10908 2014-08-13 15:29:12.972969 2014-08-13 17:38:02.773952 7fe9bae887a0 20 read_log 194416'18700 (0'0) promote 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by osd.20.0:5147 2014-08-13 15:29:18.329536 2014-08-13 17:38:02.773963 7fe9bae887a0 20 read_log 194416'18701 (194416'18700) modify 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by client.2441958.0:12160 2014-08-13 15:29:18.318594 2014-08-13 17:38:02.773975 7fe9bae887a0 20 read_log 194416'18702 (0'0) promote 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by osd.20.0:5169 2014-08-13 15:29:22.749406 2014-08-13 17:38:02.773987 7fe9bae887a0 20 read_log 194416'18703 (194416'18702) modify 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by client.2441958.0:13182 2014-08-13 15:29:22.740393 2014-08-13 17:38:02.773999 7fe9bae887a0 20 read_log 194416'18704 (0'0) promote 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by osd.20.0:5177 2014-08-13 15:29:24.831349 2014-08-13 17:38:02.774011 7fe9bae887a0 20 read_log 194416'18705 (194416'18704) modify 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by client.2441958.0:13660 2014-08-13 15:29:24.821470 2014-08-13 17:38:02.774022 7fe9bae887a0 20 read_log 194416'18706 (0'0) promote 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by osd.20.0:5203 2014-08-13 15:29:29.423198 2014-08-13 17:38:02.774034 7fe9bae887a0 20 read_log 194416'18707 (194416'18706) modify 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by client.2441958.0:14808 2014-08-13 15:29:29.411886 2014-08-13 17:38:02.774046 7fe9bae887a0 20 read_log 194416'18708 (0'0) promote ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by osd.20.0:5229 2014-08-13 15:29:35.594271 2014-08-13 17:38:02.774058 7fe9bae887a0 20 read_log 194416'18709 (194416'18708) modify ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by client.2441958.0:16003 2014-08-13 15:29:35.584326 2014-08-13 17:38:02.774088 7fe9bae887a0 20 read_log 194416'18710 (0'0) promote 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by osd.20.0:5237 2014-08-13 15:29:40.938939 2014-08-13 17:38:02.774103 7fe9bae887a0 20 read_log 194416'18711 (194416'18710) modify 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by client.2441958.0:16550 2014-08-13 15:29:40.931938 2014-08-13 17:38:02.774115 7fe9bae887a0 20 read_log 194416'18712 (0'0) promote aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by osd.20.0:5253 2014-08-13 15:29:49.069530 2014-08-13 17:38:02.774127 7fe9bae887a0 20 read_log 194416'18713 (194416'18712) modify aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by client.2441958.0:17121 2014-08-13 15:29:49.061090 2014-08-13 17:38:02.774146 7fe9bae887a0 20 read_log 194416'18714 (0'0) promote 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by osd.20.0:5279 2014-08-13 15:29:57.943443 2014-08-13 17:38:02.774168 7fe9bae887a0 20 read_log 194416'18715 (194416'18714) modify 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by client.2441958.0:17752 2014-08-13 15:29:57.934316 2014-08-13 17:38:02.774182 7fe9bae887a0 20 read_log 194422'18716 (0'0) promote 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by osd.20.0:5295 2014-08-13 15:32:15.015699 2014-08-13 17:38:02.774194 7fe9bae887a0 20 read_log 194422'18717 (194422'18716) modify 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by client.1148203.0:34 2014-08-13 15:32:15.007722 2014-08-13 17:38:02.774206 7fe9bae887a0 20 read_log 194422'18718 (0'0) promote 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by osd.20.0:5331 2014-08-13 15:32:20.755966 2014-08-13 17:38:02.774218 7fe9bae887a0 20 read_log 194422'18719 (194422'18718) modify 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by client.1148203.0:1368 2014-08-13 15:32:20.746738 2014-08-13 17:38:02.774231 7fe9bae887a0 20 read_log 194422'18720 (0'0) promote f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by osd.20.0:5349 2014-08-13 15:32:24.537306 2014-08-13 17:38:02.774243 7fe9bae887a0 20 read_log 194422'18721 (194422'18720) modify f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by client.1148203.0:2339 2014-08-13 15:32:24.525408 2014-08-13 17:38:02.774255 7fe9bae887a0 20 read_log 194422'18722 (0'0) promote 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by osd.20.0:5367 2014-08-13 15:32:29.056340 2014-08-13 17:38:02.774267 7fe9bae887a0 20 read_log 194422'18723 (194422'18722) modify 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by client.1148203.0:3490 2014-08-13 15:32:29.047446 2014-08-13 17:38:02.774279 7fe9bae887a0 20 read_log 194422'18724 (0'0) promote e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by osd.20.0:5425 2014-08-13 15:32:40.724564 2014-08-13 17:38:02.774291 7fe9bae887a0 20 read_log 194422'18725 (194422'18724) modify e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by client.1148203.0:6106 2014-08-13 15:32:40.713670 2014-08-13 17:38:02.774304 7fe9bae887a0 20 read_log 194422'18726 (0'0) promote 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by osd.20.0:5427 2014-08-13 15:32:42.428687 2014-08-13 17:38:02.774316 7fe9bae887a0 20 read_log 194422'18727 (194422'18726) modify 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by client.1148203.0:6422 2014-08-13 15:32:42.419805 2014-08-13 17:38:02.774328 7fe9bae887a0 20 read_log 194422'18728 (0'0) promote 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by osd.20.0:5469 2014-08-13 15:32:59.205436 2014-08-13 17:38:02.774340 7fe9bae887a0 20 read_log 194422'18729 (194422'18728) modify 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by client.1148203.0:8962 2014-08-13 15:32:59.195194 2014-08-13 17:38:02.774352 7fe9bae887a0 20 read_log 194422'18730 (0'0) promote 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by osd.20.0:5505 2014-08-13 15:33:26.090665 2014-08-13 17:38:02.774364 7fe9bae887a0 20 read_log 194422'18731 (194422'18730) modify 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by client.1148203.0:10577 2014-08-13 15:33:26.082626 2014-08-13 17:38:02.775024 7fe9bae887a0 10 read_log checking for missing items over interval (194370'18616,194422'18731] 2014-08-13 17:38:02.775080 7fe9bae887a0 15 read_log missing 194422'18731 (194422'18730) modify 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by client.1148203.0:10577 2014-08-13 15:33:26.082626 2014-08-13 17:38:02.775127 7fe9bae887a0 15 read_log missing 194422'18729 (194422'18728) modify 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by client.1148203.0:8962 2014-08-13 15:32:59.195194 2014-08-13 17:38:02.775174 7fe9bae887a0 15 read_log missing 194422'18727 (194422'18726) modify 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by client.1148203.0:6422 2014-08-13 15:32:42.419805 2014-08-13 17:38:02.775216 7fe9bae887a0 15 read_log missing 194422'18725 (194422'18724) modify e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by client.1148203.0:6106 2014-08-13 15:32:40.713670 2014-08-13 17:38:02.775257 7fe9bae887a0 15 read_log missing 194422'18723 (194422'18722) modify 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by client.1148203.0:3490 2014-08-13 15:32:29.047446 2014-08-13 17:38:02.775298 7fe9bae887a0 15 read_log missing 194422'18721 (194422'18720) modify f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by client.1148203.0:2339 2014-08-13 15:32:24.525408 2014-08-13 17:38:02.775339 7fe9bae887a0 15 read_log missing 194422'18719 (194422'18718) modify 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by client.1148203.0:1368 2014-08-13 15:32:20.746738 2014-08-13 17:38:02.775380 7fe9bae887a0 15 read_log missing 194422'18717 (194422'18716) modify 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by client.1148203.0:34 2014-08-13 15:32:15.007722 2014-08-13 17:38:02.775422 7fe9bae887a0 15 read_log missing 194416'18715 (194416'18714) modify 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by client.2441958.0:17752 2014-08-13 15:29:57.934316 2014-08-13 17:38:02.775463 7fe9bae887a0 15 read_log missing 194416'18713 (194416'18712) modify aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by client.2441958.0:17121 2014-08-13 15:29:49.061090 2014-08-13 17:38:02.775504 7fe9bae887a0 15 read_log missing 194416'18711 (194416'18710) modify 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by client.2441958.0:16550 2014-08-13 15:29:40.931938 2014-08-13 17:38:02.775545 7fe9bae887a0 15 read_log missing 194416'18709 (194416'18708) modify ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by client.2441958.0:16003 2014-08-13 15:29:35.584326 2014-08-13 17:38:02.775586 7fe9bae887a0 15 read_log missing 194416'18707 (194416'18706) modify 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by client.2441958.0:14808 2014-08-13 15:29:29.411886 2014-08-13 17:38:02.775627 7fe9bae887a0 15 read_log missing 194416'18705 (194416'18704) modify 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by client.2441958.0:13660 2014-08-13 15:29:24.821470 2014-08-13 17:38:02.775667 7fe9bae887a0 15 read_log missing 194416'18703 (194416'18702) modify 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by client.2441958.0:13182 2014-08-13 15:29:22.740393 2014-08-13 17:38:02.775708 7fe9bae887a0 15 read_log missing 194416'18701 (194416'18700) modify 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by client.2441958.0:12160 2014-08-13 15:29:18.318594 2014-08-13 17:38:02.775749 7fe9bae887a0 15 read_log missing 194416'18699 (194416'18698) modify ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by client.2441958.0:10908 2014-08-13 15:29:12.972969 2014-08-13 17:38:02.775790 7fe9bae887a0 15 read_log missing 194416'18697 (194416'18696) modify 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by client.2441958.0:10738 2014-08-13 15:29:12.316890 2014-08-13 17:38:02.775831 7fe9bae887a0 15 read_log missing 194416'18695 (194416'18694) modify ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by client.2441958.0:10653 2014-08-13 15:29:12.001679 2014-08-13 17:38:02.775873 7fe9bae887a0 15 read_log missing 194416'18693 (194416'18692) modify 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by client.2441958.0:9868 2014-08-13 15:29:08.847037 2014-08-13 17:38:02.775914 7fe9bae887a0 15 read_log missing 194416'18691 (194416'18690) modify 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by client.2441958.0:8429 2014-08-13 15:29:03.146674 2014-08-13 17:38:02.775954 7fe9bae887a0 15 read_log missing 194416'18689 (194416'18688) modify c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by client.2441958.0:8290 2014-08-13 15:29:02.611334 2014-08-13 17:38:02.775996 7fe9bae887a0 15 read_log missing 194416'18687 (194416'18686) modify edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by client.2441958.0:7488 2014-08-13 15:28:59.601755 2014-08-13 17:38:02.776040 7fe9bae887a0 15 read_log missing 194416'18685 (194416'18684) modify b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by client.2441958.0:6649 2014-08-13 15:28:56.466596 2014-08-13 17:38:02.776082 7fe9bae887a0 15 read_log missing 194416'18683 (194416'18682) modify 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by client.2441958.0:5788 2014-08-13 15:28:52.927293 2014-08-13 17:38:02.776123 7fe9bae887a0 15 read_log missing 194416'18681 (194416'18680) modify 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by client.2441958.0:5296 2014-08-13 15:28:51.021478 2014-08-13 17:38:02.776169 7fe9bae887a0 15 read_log missing 194416'18679 (194416'18678) modify d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by client.2441958.0:2221 2014-08-13 15:28:39.104322 2014-08-13 17:38:02.776212 7fe9bae887a0 15 read_log missing 194416'18677 (194416'18676) modify 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by client.2441958.0:306 2014-08-13 15:28:31.473163 2014-08-13 17:38:02.776254 7fe9bae887a0 15 read_log missing 194416'18675 (194416'18674) modify 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by client.2441958.0:179 2014-08-13 15:28:30.869401 2014-08-13 17:38:02.776298 7fe9bae887a0 15 read_log missing 194416'18672 (194416'18672) modify 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 17:38:02.776339 7fe9bae887a0 15 read_log missing 194393'18671 (194393'18670) modify d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by client.1064182.0:270 2014-08-12 20:49:26.734751 2014-08-13 17:38:02.776381 7fe9bae887a0 15 read_log missing 194393'18669 (194393'18668) modify 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by client.1064095.0:516 2014-08-12 20:47:28.139439 2014-08-13 17:38:02.776422 7fe9bae887a0 15 read_log missing 194393'18667 (194393'18666) modify 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by client.1064095.0:409 2014-08-12 20:47:27.678974 2014-08-13 17:38:02.776466 7fe9bae887a0 15 read_log missing 194383'18663 (194383'18662) modify abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by client.2231190.0:7104 2014-08-12 20:34:17.967809 2014-08-13 17:38:02.776507 7fe9bae887a0 15 read_log missing 194383'18661 (194383'18660) modify a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by client.2231190.0:6394 2014-08-12 20:34:03.564216 2014-08-13 17:38:02.776550 7fe9bae887a0 15 read_log missing 194377'18659 (194377'18658) modify 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by client.2231190.0:4736 2014-08-12 20:33:18.883461 2014-08-13 17:38:02.776591 7fe9bae887a0 15 read_log missing 194377'18657 (194377'18656) modify 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by client.2231190.0:4036 2014-08-12 20:33:14.118216 2014-08-13 17:38:02.776632 7fe9bae887a0 15 read_log missing 194377'18655 (194377'18654) modify 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by client.2231190.0:3092 2014-08-12 20:33:09.673777 2014-08-13 17:38:02.776674 7fe9bae887a0 15 read_log missing 194377'18653 (194377'18652) modify cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by client.2231190.0:2905 2014-08-12 20:33:08.890178 2014-08-13 17:38:02.776715 7fe9bae887a0 15 read_log missing 194377'18651 (194377'18650) modify 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by client.2231190.0:2241 2014-08-12 20:33:06.209363 2014-08-13 17:38:02.776756 7fe9bae887a0 15 read_log missing 194377'18649 (194377'18648) modify 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by client.2231190.0:1453 2014-08-12 20:33:03.014348 2014-08-13 17:38:02.776797 7fe9bae887a0 15 read_log missing 194377'18647 (194377'18646) modify cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by client.2231190.0:1192 2014-08-12 20:33:02.023014 2014-08-13 17:38:02.776837 7fe9bae887a0 15 read_log missing 194377'18645 (194377'18644) modify a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by client.2231190.0:896 2014-08-12 20:33:00.907409 2014-08-13 17:38:02.776878 7fe9bae887a0 15 read_log missing 194377'18643 (194377'18642) modify 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by client.2231190.0:204 2014-08-12 20:32:58.004900 2014-08-13 17:38:02.776920 7fe9bae887a0 15 read_log missing 194370'18641 (194370'18640) modify 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by client.1508531.0:13314 2014-08-12 20:31:22.374937 2014-08-13 17:38:02.776961 7fe9bae887a0 15 read_log missing 194370'18639 (194370'18638) modify 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by client.1508531.0:13048 2014-08-12 20:31:19.800706 2014-08-13 17:38:02.777002 7fe9bae887a0 15 read_log missing 194370'18637 (194370'18636) modify 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by client.1508531.0:12968 2014-08-12 20:31:19.055563 2014-08-13 17:38:02.777043 7fe9bae887a0 15 read_log missing 194370'18635 (194370'18634) modify d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by client.1508531.0:10941 2014-08-12 20:31:04.914356 2014-08-13 17:38:02.777084 7fe9bae887a0 15 read_log missing 194370'18633 (194370'18632) modify fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by client.1508531.0:10288 2014-08-12 20:31:00.946368 2014-08-13 17:38:02.777125 7fe9bae887a0 15 read_log missing 194370'18631 (194370'18630) modify 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by client.1508531.0:10152 2014-08-12 20:31:00.138476 2014-08-13 17:38:02.777171 7fe9bae887a0 15 read_log missing 194370'18629 (194370'18628) modify f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by client.1508531.0:9659 2014-08-12 20:30:57.255203 2014-08-13 17:38:02.777213 7fe9bae887a0 15 read_log missing 194370'18627 (194370'18626) modify cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by client.1508531.0:9310 2014-08-12 20:30:55.167609 2014-08-13 17:38:02.777254 7fe9bae887a0 15 read_log missing 194370'18625 (194370'18624) modify fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by client.1508531.0:8648 2014-08-12 20:30:51.141855 2014-08-13 17:38:02.777295 7fe9bae887a0 15 read_log missing 194370'18623 (194370'18622) modify 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by client.1508531.0:6728 2014-08-12 20:30:40.992116 2014-08-13 17:38:02.777336 7fe9bae887a0 15 read_log missing 194370'18621 (194370'18620) modify 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by client.1508531.0:4127 2014-08-12 20:30:29.751699 2014-08-13 17:38:02.777376 7fe9bae887a0 15 read_log missing 194370'18619 (194370'18618) modify cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by client.1508531.0:3488 2014-08-12 20:30:27.359030 2014-08-13 17:38:02.777418 7fe9bae887a0 15 read_log missing 194370'18617 (194370'18616) modify 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by client.1508531.0:1616 2014-08-12 20:30:20.430889 2014-08-13 17:38:02.777429 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.777464 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_loaded 2014-08-13 17:38:02.777483 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Initial 0.021136 0 0.000000 2014-08-13 17:38:02.777495 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset 2014-08-13 17:38:02.777504 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194477 2014-08-13 17:38:02.777514 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 17:38:02.777525 7fe9bae887a0 10 osd.4 194478 pgid 24.5e coll 24.5e_head 2014-08-13 17:38:02.777653 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.5e 2014-08-13 17:38:02.777666 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.777698 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.5e(unlocked)] enter Initial 2014-08-13 17:38:02.777716 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.5e(unlocked)] enter NotTrimming 2014-08-13 17:38:02.777861 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.777990 7fe9bae887a0 20 read_log 192718'14373 (192718'13737) delete 5e/hit_set_24.5e_archive_2014-08-06 14:24:01.267758_2014-08-06 14:50:06.074726/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:00:54.701437 2014-08-13 17:38:02.778012 7fe9bae887a0 20 read_log 192803'14718 (192803'14718) modify 5e/hit_set_24.5e_archive_2014-08-11 10:24:43.601998_2014-08-11 12:48:05.517931/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:48:05.518598 2014-08-13 17:38:02.778025 7fe9bae887a0 20 read_log 192803'14719 (192718'14372) delete 5e/hit_set_24.5e_archive_2014-08-06 14:50:06.074945_2014-08-06 15:00:54.701195/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:48:05.518598 2014-08-13 17:38:02.778046 7fe9bae887a0 20 read_log 192803'15090 (0'0) promote a181605e/core.46996+10+2/head//24 by osd.289.0:51828 2014-08-11 12:51:33.017661 2014-08-13 17:38:02.778062 7fe9bae887a0 20 read_log 192803'15091 (192803'15090) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:1 2014-08-11 12:51:28.148875 2014-08-13 17:38:02.778074 7fe9bae887a0 20 read_log 192803'15092 (192803'15091) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:2 2014-08-11 12:51:33.930260 2014-08-13 17:38:02.778086 7fe9bae887a0 20 read_log 192803'15093 (192803'15092) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:3 2014-08-11 12:51:34.383482 2014-08-13 17:38:02.778098 7fe9bae887a0 20 read_log 192803'15094 (192803'15093) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:4 2014-08-11 12:51:34.413145 2014-08-13 17:38:02.778110 7fe9bae887a0 20 read_log 192803'15095 (192803'15094) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:5 2014-08-11 12:51:34.471494 2014-08-13 17:38:02.778121 7fe9bae887a0 20 read_log 192803'15096 (192803'15095) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:6 2014-08-11 12:51:34.499111 2014-08-13 17:38:02.778133 7fe9bae887a0 20 read_log 192803'15097 (192803'15096) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:7 2014-08-11 12:51:34.556731 2014-08-13 17:38:02.778145 7fe9bae887a0 20 read_log 192803'15098 (192803'15097) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:8 2014-08-11 12:51:34.583284 2014-08-13 17:38:02.778165 7fe9bae887a0 20 read_log 192803'15099 (192803'15098) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:9 2014-08-11 12:51:34.652838 2014-08-13 17:38:02.778178 7fe9bae887a0 20 read_log 192803'15100 (192803'15099) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:10 2014-08-11 12:51:34.679367 2014-08-13 17:38:02.778189 7fe9bae887a0 20 read_log 192803'15101 (192803'15100) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:11 2014-08-11 12:51:34.756048 2014-08-13 17:38:02.778201 7fe9bae887a0 20 read_log 192803'15102 (192803'15101) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:12 2014-08-11 12:51:34.886668 2014-08-13 17:38:02.778213 7fe9bae887a0 20 read_log 192803'15103 (192803'15102) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:13 2014-08-11 12:51:34.955906 2014-08-13 17:38:02.778225 7fe9bae887a0 20 read_log 192803'15104 (192803'15103) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:14 2014-08-11 12:51:34.986833 2014-08-13 17:38:02.778237 7fe9bae887a0 20 read_log 192803'15105 (192803'15104) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:15 2014-08-11 12:51:35.108219 2014-08-13 17:38:02.778249 7fe9bae887a0 20 read_log 192803'15106 (192803'15105) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:16 2014-08-11 12:51:35.129258 2014-08-13 17:38:02.778260 7fe9bae887a0 20 read_log 192803'15107 (192803'15106) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:17 2014-08-11 12:51:35.152078 2014-08-13 17:38:02.778272 7fe9bae887a0 20 read_log 192803'15108 (192803'15107) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:18 2014-08-11 12:51:35.191769 2014-08-13 17:38:02.778284 7fe9bae887a0 20 read_log 192803'15109 (192803'15108) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:19 2014-08-11 12:51:35.216401 2014-08-13 17:38:02.778296 7fe9bae887a0 20 read_log 192803'15110 (192803'15109) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:20 2014-08-11 12:51:35.240751 2014-08-13 17:38:02.778308 7fe9bae887a0 20 read_log 192803'15111 (192803'15110) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:21 2014-08-11 12:51:35.298213 2014-08-13 17:38:02.778319 7fe9bae887a0 20 read_log 192803'15112 (192803'15111) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:22 2014-08-11 12:51:35.718802 2014-08-13 17:38:02.778331 7fe9bae887a0 20 read_log 192803'15113 (192803'15112) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:23 2014-08-11 12:51:35.796988 2014-08-13 17:38:02.778350 7fe9bae887a0 20 read_log 192803'15114 (192803'15113) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:24 2014-08-11 12:51:35.823828 2014-08-13 17:38:02.778362 7fe9bae887a0 20 read_log 192803'15115 (192803'15114) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:25 2014-08-11 12:51:35.862002 2014-08-13 17:38:02.778374 7fe9bae887a0 20 read_log 192803'15116 (192803'15115) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:26 2014-08-11 12:51:35.908241 2014-08-13 17:38:02.778388 7fe9bae887a0 20 read_log 192803'15117 (192803'15116) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:27 2014-08-11 12:51:35.954386 2014-08-13 17:38:02.778400 7fe9bae887a0 20 read_log 192803'15118 (192803'15117) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:28 2014-08-11 12:51:36.000446 2014-08-13 17:38:02.778412 7fe9bae887a0 20 read_log 192803'15119 (192803'15118) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:29 2014-08-11 12:51:36.046029 2014-08-13 17:38:02.778424 7fe9bae887a0 20 read_log 192803'15120 (192803'15119) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:30 2014-08-11 12:51:36.089736 2014-08-13 17:38:02.778436 7fe9bae887a0 20 read_log 192803'15121 (192803'15120) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:31 2014-08-11 12:51:36.148976 2014-08-13 17:38:02.778448 7fe9bae887a0 20 read_log 192803'15122 (192803'15121) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:32 2014-08-11 12:51:36.666026 2014-08-13 17:38:02.778461 7fe9bae887a0 20 read_log 192803'15123 (192803'15122) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:33 2014-08-11 12:51:36.712683 2014-08-13 17:38:02.778473 7fe9bae887a0 20 read_log 192803'15124 (192803'15123) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:34 2014-08-11 12:51:36.759756 2014-08-13 17:38:02.778484 7fe9bae887a0 20 read_log 192803'15125 (192803'15124) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:35 2014-08-11 12:51:36.806560 2014-08-13 17:38:02.778496 7fe9bae887a0 20 read_log 192803'15126 (192803'15125) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:36 2014-08-11 12:51:36.859590 2014-08-13 17:38:02.778507 7fe9bae887a0 20 read_log 192803'15127 (192803'15126) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:37 2014-08-11 12:51:36.906385 2014-08-13 17:38:02.778519 7fe9bae887a0 20 read_log 192803'15128 (192803'15127) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:38 2014-08-11 12:51:36.954201 2014-08-13 17:38:02.778531 7fe9bae887a0 20 read_log 192803'15129 (192803'15128) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:39 2014-08-11 12:51:37.081535 2014-08-13 17:38:02.778543 7fe9bae887a0 20 read_log 192803'15130 (192803'15129) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:40 2014-08-11 12:51:37.230490 2014-08-13 17:38:02.778555 7fe9bae887a0 20 read_log 192803'15131 (192803'15130) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:41 2014-08-11 12:51:37.388095 2014-08-13 17:38:02.778567 7fe9bae887a0 20 read_log 192803'15132 (192803'15131) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:42 2014-08-11 12:51:37.594120 2014-08-13 17:38:02.778579 7fe9bae887a0 20 read_log 192803'15133 (192803'15132) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:43 2014-08-11 12:51:37.651853 2014-08-13 17:38:02.778590 7fe9bae887a0 20 read_log 192803'15134 (192803'15133) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:44 2014-08-11 12:51:37.695980 2014-08-13 17:38:02.778602 7fe9bae887a0 20 read_log 192803'15135 (192803'15134) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:45 2014-08-11 12:51:37.745008 2014-08-13 17:38:02.778614 7fe9bae887a0 20 read_log 192803'15136 (192803'15135) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:46 2014-08-11 12:51:37.795263 2014-08-13 17:38:02.778625 7fe9bae887a0 20 read_log 192803'15137 (192803'15136) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:47 2014-08-11 12:51:37.842348 2014-08-13 17:38:02.778646 7fe9bae887a0 20 read_log 192803'15138 (192803'15137) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:48 2014-08-11 12:51:37.883945 2014-08-13 17:38:02.778659 7fe9bae887a0 20 read_log 192803'15139 (192803'15138) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:49 2014-08-11 12:51:37.927824 2014-08-13 17:38:02.778670 7fe9bae887a0 20 read_log 192803'15140 (192803'15139) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:50 2014-08-11 12:51:37.977290 2014-08-13 17:38:02.778682 7fe9bae887a0 20 read_log 192803'15141 (192803'15140) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:51 2014-08-11 12:51:38.027134 2014-08-13 17:38:02.778694 7fe9bae887a0 20 read_log 192803'15142 (192803'15141) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:52 2014-08-11 12:51:38.075175 2014-08-13 17:38:02.778706 7fe9bae887a0 20 read_log 192803'15143 (192803'15142) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:53 2014-08-11 12:51:38.567081 2014-08-13 17:38:02.778718 7fe9bae887a0 20 read_log 192803'15144 (192803'15143) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:54 2014-08-11 12:51:38.618119 2014-08-13 17:38:02.778729 7fe9bae887a0 20 read_log 192803'15145 (192803'15144) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:55 2014-08-11 12:51:38.667519 2014-08-13 17:38:02.778741 7fe9bae887a0 20 read_log 192803'15146 (192803'15145) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:56 2014-08-11 12:51:38.717249 2014-08-13 17:38:02.778753 7fe9bae887a0 20 read_log 192803'15147 (192803'15146) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:57 2014-08-11 12:51:38.767026 2014-08-13 17:38:02.778765 7fe9bae887a0 20 read_log 192803'15148 (192803'15147) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:58 2014-08-11 12:51:38.831233 2014-08-13 17:38:02.778776 7fe9bae887a0 20 read_log 192803'15149 (192803'15148) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:59 2014-08-11 12:51:38.858206 2014-08-13 17:38:02.778788 7fe9bae887a0 20 read_log 192803'15150 (192803'15149) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:60 2014-08-11 12:51:38.891154 2014-08-13 17:38:02.778799 7fe9bae887a0 20 read_log 192803'15151 (192803'15150) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:61 2014-08-11 12:51:38.916702 2014-08-13 17:38:02.778811 7fe9bae887a0 20 read_log 192803'15152 (192803'15151) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:62 2014-08-11 12:51:38.946230 2014-08-13 17:38:02.778842 7fe9bae887a0 20 read_log 192803'15153 (192803'15152) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:63 2014-08-11 12:51:39.455207 2014-08-13 17:38:02.778854 7fe9bae887a0 20 read_log 192803'15154 (192803'15153) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:64 2014-08-11 12:51:39.538286 2014-08-13 17:38:02.778866 7fe9bae887a0 20 read_log 192803'15155 (192803'15154) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:65 2014-08-11 12:51:39.623434 2014-08-13 17:38:02.778881 7fe9bae887a0 20 read_log 192803'15156 (192803'15155) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:66 2014-08-11 12:51:39.705746 2014-08-13 17:38:02.778893 7fe9bae887a0 20 read_log 192803'15157 (192803'15156) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:67 2014-08-11 12:51:39.783124 2014-08-13 17:38:02.778904 7fe9bae887a0 20 read_log 192803'15158 (192803'15157) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:68 2014-08-11 12:51:39.874953 2014-08-13 17:38:02.778916 7fe9bae887a0 20 read_log 192803'15159 (192803'15158) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:69 2014-08-11 12:51:39.955526 2014-08-13 17:38:02.778928 7fe9bae887a0 20 read_log 192803'15160 (192803'15159) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:70 2014-08-11 12:51:40.040125 2014-08-13 17:38:02.778939 7fe9bae887a0 20 read_log 192803'15161 (192803'15160) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:71 2014-08-11 12:51:40.117788 2014-08-13 17:38:02.778973 7fe9bae887a0 20 read_log 192803'15162 (192803'15161) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:72 2014-08-11 12:51:40.200092 2014-08-13 17:38:02.778989 7fe9bae887a0 20 read_log 192803'15163 (192803'15162) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:73 2014-08-11 12:51:40.445578 2014-08-13 17:38:02.779001 7fe9bae887a0 20 read_log 192803'15164 (192803'15163) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:74 2014-08-11 12:51:40.473317 2014-08-13 17:38:02.779013 7fe9bae887a0 20 read_log 192803'15165 (192803'15164) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:75 2014-08-11 12:51:40.502496 2014-08-13 17:38:02.779025 7fe9bae887a0 20 read_log 192803'15166 (192803'15165) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:76 2014-08-11 12:51:40.529351 2014-08-13 17:38:02.779037 7fe9bae887a0 20 read_log 192803'15167 (192803'15166) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:77 2014-08-11 12:51:40.555503 2014-08-13 17:38:02.779049 7fe9bae887a0 20 read_log 192803'15168 (192803'15167) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:78 2014-08-11 12:51:40.588176 2014-08-13 17:38:02.779060 7fe9bae887a0 20 read_log 192803'15169 (192803'15168) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:79 2014-08-11 12:51:40.614409 2014-08-13 17:38:02.779072 7fe9bae887a0 20 read_log 192803'15170 (192803'15169) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:80 2014-08-11 12:51:40.645163 2014-08-13 17:38:02.779084 7fe9bae887a0 20 read_log 192803'15171 (192803'15170) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:81 2014-08-11 12:51:40.681089 2014-08-13 17:38:02.779096 7fe9bae887a0 20 read_log 192803'15172 (192803'15171) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:82 2014-08-11 12:51:40.728585 2014-08-13 17:38:02.779108 7fe9bae887a0 20 read_log 192803'15173 (192803'15172) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:83 2014-08-11 12:51:41.193614 2014-08-13 17:38:02.779119 7fe9bae887a0 20 read_log 192803'15174 (192803'15173) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:84 2014-08-11 12:51:41.223323 2014-08-13 17:38:02.779131 7fe9bae887a0 20 read_log 192803'15175 (192803'15174) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:85 2014-08-11 12:51:41.258193 2014-08-13 17:38:02.779143 7fe9bae887a0 20 read_log 192803'15176 (192803'15175) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:86 2014-08-11 12:51:41.286695 2014-08-13 17:38:02.779159 7fe9bae887a0 20 read_log 192803'15177 (192803'15176) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:87 2014-08-11 12:51:41.313936 2014-08-13 17:38:02.779173 7fe9bae887a0 20 read_log 192803'15178 (192803'15177) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:88 2014-08-11 12:51:41.344205 2014-08-13 17:38:02.779185 7fe9bae887a0 20 read_log 192803'15179 (192803'15178) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:89 2014-08-11 12:51:41.371658 2014-08-13 17:38:02.779197 7fe9bae887a0 20 read_log 192803'15180 (192803'15179) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:90 2014-08-11 12:51:41.408570 2014-08-13 17:38:02.779208 7fe9bae887a0 20 read_log 192803'15181 (192803'15180) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:91 2014-08-11 12:51:41.435537 2014-08-13 17:38:02.779220 7fe9bae887a0 20 read_log 192803'15182 (192803'15181) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:92 2014-08-11 12:51:41.461828 2014-08-13 17:38:02.779232 7fe9bae887a0 20 read_log 192803'15183 (192803'15182) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:93 2014-08-11 12:51:41.749054 2014-08-13 17:38:02.779243 7fe9bae887a0 20 read_log 192803'15184 (192803'15183) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:94 2014-08-11 12:51:41.778586 2014-08-13 17:38:02.779255 7fe9bae887a0 20 read_log 192803'15185 (192803'15184) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:95 2014-08-11 12:51:41.813766 2014-08-13 17:38:02.779273 7fe9bae887a0 20 read_log 192803'15186 (192803'15185) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:96 2014-08-11 12:51:41.841566 2014-08-13 17:38:02.779286 7fe9bae887a0 20 read_log 192803'15187 (192803'15186) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:97 2014-08-11 12:51:41.869467 2014-08-13 17:38:02.779297 7fe9bae887a0 20 read_log 192803'15188 (192803'15187) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:98 2014-08-11 12:51:41.907960 2014-08-13 17:38:02.779309 7fe9bae887a0 20 read_log 192803'15189 (192803'15188) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:99 2014-08-11 12:51:41.935927 2014-08-13 17:38:02.779321 7fe9bae887a0 20 read_log 192803'15190 (192803'15189) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:100 2014-08-11 12:51:41.964647 2014-08-13 17:38:02.779332 7fe9bae887a0 20 read_log 192803'15191 (192803'15190) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:101 2014-08-11 12:51:41.991298 2014-08-13 17:38:02.779344 7fe9bae887a0 20 read_log 192803'15192 (192803'15191) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:102 2014-08-11 12:51:42.021812 2014-08-13 17:38:02.779356 7fe9bae887a0 20 read_log 192803'15193 (192803'15192) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:103 2014-08-11 12:51:42.313044 2014-08-13 17:38:02.779367 7fe9bae887a0 20 read_log 192803'15194 (192803'15193) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:104 2014-08-11 12:51:42.344301 2014-08-13 17:38:02.779379 7fe9bae887a0 20 read_log 192803'15195 (192803'15194) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:105 2014-08-11 12:51:42.378036 2014-08-13 17:38:02.779391 7fe9bae887a0 20 read_log 192803'15196 (192803'15195) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:106 2014-08-11 12:51:42.407567 2014-08-13 17:38:02.779402 7fe9bae887a0 20 read_log 192803'15197 (192803'15196) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:107 2014-08-11 12:51:42.433160 2014-08-13 17:38:02.779414 7fe9bae887a0 20 read_log 192803'15198 (192803'15197) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:108 2014-08-11 12:51:42.463423 2014-08-13 17:38:02.779426 7fe9bae887a0 20 read_log 192803'15199 (192803'15198) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:109 2014-08-11 12:51:42.489784 2014-08-13 17:38:02.779437 7fe9bae887a0 20 read_log 192803'15200 (192803'15199) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:110 2014-08-11 12:51:42.519781 2014-08-13 17:38:02.779449 7fe9bae887a0 20 read_log 192803'15201 (192803'15200) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:111 2014-08-11 12:51:42.544088 2014-08-13 17:38:02.779461 7fe9bae887a0 20 read_log 192803'15202 (192803'15201) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:112 2014-08-11 12:51:42.569853 2014-08-13 17:38:02.779472 7fe9bae887a0 20 read_log 192803'15203 (192803'15202) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:113 2014-08-11 12:51:42.859576 2014-08-13 17:38:02.779484 7fe9bae887a0 20 read_log 192803'15204 (192803'15203) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:114 2014-08-11 12:51:42.888261 2014-08-13 17:38:02.779495 7fe9bae887a0 20 read_log 192803'15205 (192803'15204) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:115 2014-08-11 12:51:42.924610 2014-08-13 17:38:02.779507 7fe9bae887a0 20 read_log 192803'15206 (192803'15205) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:116 2014-08-11 12:51:42.951731 2014-08-13 17:38:02.779518 7fe9bae887a0 20 read_log 192803'15207 (192803'15206) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:117 2014-08-11 12:51:42.978054 2014-08-13 17:38:02.779530 7fe9bae887a0 20 read_log 192803'15208 (192803'15207) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:118 2014-08-11 12:51:43.008757 2014-08-13 17:38:02.779542 7fe9bae887a0 20 read_log 192803'15209 (192803'15208) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:119 2014-08-11 12:51:43.037252 2014-08-13 17:38:02.779562 7fe9bae887a0 20 read_log 192803'15210 (192803'15209) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:120 2014-08-11 12:51:43.067240 2014-08-13 17:38:02.779575 7fe9bae887a0 20 read_log 192803'15211 (192803'15210) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:121 2014-08-11 12:51:43.096791 2014-08-13 17:38:02.779586 7fe9bae887a0 20 read_log 192803'15212 (192803'15211) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:122 2014-08-11 12:51:43.123787 2014-08-13 17:38:02.779598 7fe9bae887a0 20 read_log 192803'15213 (192803'15212) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:123 2014-08-11 12:51:43.156897 2014-08-13 17:38:02.779610 7fe9bae887a0 20 read_log 192803'15214 (192803'15213) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:124 2014-08-11 12:51:43.427360 2014-08-13 17:38:02.779621 7fe9bae887a0 20 read_log 192803'15215 (192803'15214) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:125 2014-08-11 12:51:43.463522 2014-08-13 17:38:02.779633 7fe9bae887a0 20 read_log 192803'15216 (192803'15215) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:126 2014-08-11 12:51:43.490840 2014-08-13 17:38:02.779644 7fe9bae887a0 20 read_log 192803'15217 (192803'15216) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:127 2014-08-11 12:51:43.522163 2014-08-13 17:38:02.779657 7fe9bae887a0 20 read_log 192803'15218 (192803'15217) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:128 2014-08-11 12:51:43.556598 2014-08-13 17:38:02.779669 7fe9bae887a0 20 read_log 192803'15219 (192803'15218) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:129 2014-08-11 12:51:43.585877 2014-08-13 17:38:02.779681 7fe9bae887a0 20 read_log 192803'15220 (192803'15219) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:130 2014-08-11 12:51:43.619930 2014-08-13 17:38:02.779693 7fe9bae887a0 20 read_log 192803'15221 (192803'15220) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:131 2014-08-11 12:51:43.648088 2014-08-13 17:38:02.779705 7fe9bae887a0 20 read_log 192803'15222 (192803'15221) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:132 2014-08-11 12:51:43.674881 2014-08-13 17:38:02.779716 7fe9bae887a0 20 read_log 192803'15223 (192803'15222) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:133 2014-08-11 12:51:43.704609 2014-08-13 17:38:02.779728 7fe9bae887a0 20 read_log 192803'15224 (192803'15223) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:134 2014-08-11 12:51:43.940761 2014-08-13 17:38:02.779739 7fe9bae887a0 20 read_log 192803'15225 (192803'15224) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:135 2014-08-11 12:51:43.977275 2014-08-13 17:38:02.779751 7fe9bae887a0 20 read_log 192803'15226 (192803'15225) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:136 2014-08-11 12:51:44.004970 2014-08-13 17:38:02.779763 7fe9bae887a0 20 read_log 192803'15227 (192803'15226) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:137 2014-08-11 12:51:44.032293 2014-08-13 17:38:02.779774 7fe9bae887a0 20 read_log 192803'15228 (192803'15227) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:138 2014-08-11 12:51:44.059751 2014-08-13 17:38:02.779786 7fe9bae887a0 20 read_log 192803'15229 (192803'15228) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:139 2014-08-11 12:51:44.082608 2014-08-13 17:38:02.779798 7fe9bae887a0 20 read_log 192803'15230 (192803'15229) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:140 2014-08-11 12:51:44.132956 2014-08-13 17:38:02.779810 7fe9bae887a0 20 read_log 192803'15231 (192803'15230) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:141 2014-08-11 12:51:44.155206 2014-08-13 17:38:02.779822 7fe9bae887a0 20 read_log 192803'15232 (192803'15231) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:142 2014-08-11 12:51:44.192450 2014-08-13 17:38:02.779835 7fe9bae887a0 20 read_log 192803'15233 (192803'15232) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:143 2014-08-11 12:51:44.225561 2014-08-13 17:38:02.779853 7fe9bae887a0 20 read_log 192803'15234 (192803'15233) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:144 2014-08-11 12:51:44.462087 2014-08-13 17:38:02.779865 7fe9bae887a0 20 read_log 192803'15235 (192803'15234) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:145 2014-08-11 12:51:44.495255 2014-08-13 17:38:02.779877 7fe9bae887a0 20 read_log 192803'15236 (192803'15235) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:146 2014-08-11 12:51:44.520700 2014-08-13 17:38:02.779889 7fe9bae887a0 20 read_log 192803'15237 (192803'15236) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:147 2014-08-11 12:51:44.547456 2014-08-13 17:38:02.779900 7fe9bae887a0 20 read_log 192803'15238 (192803'15237) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:148 2014-08-11 12:51:44.583774 2014-08-13 17:38:02.779912 7fe9bae887a0 20 read_log 192803'15239 (192803'15238) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:149 2014-08-11 12:51:44.610919 2014-08-13 17:38:02.779924 7fe9bae887a0 20 read_log 192803'15240 (192803'15239) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:150 2014-08-11 12:51:44.638647 2014-08-13 17:38:02.779935 7fe9bae887a0 20 read_log 192803'15241 (192803'15240) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:151 2014-08-11 12:51:44.663876 2014-08-13 17:38:02.779947 7fe9bae887a0 20 read_log 192803'15242 (192803'15241) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:152 2014-08-11 12:51:44.695669 2014-08-13 17:38:02.779959 7fe9bae887a0 20 read_log 192803'15243 (192803'15242) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:153 2014-08-11 12:51:44.736393 2014-08-13 17:38:02.779970 7fe9bae887a0 20 read_log 192803'15244 (192803'15243) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:154 2014-08-11 12:51:44.983272 2014-08-13 17:38:02.779982 7fe9bae887a0 20 read_log 192803'15245 (192803'15244) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:155 2014-08-11 12:51:45.012355 2014-08-13 17:38:02.780015 7fe9bae887a0 20 read_log 192803'15246 (192803'15245) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:156 2014-08-11 12:51:45.045155 2014-08-13 17:38:02.780028 7fe9bae887a0 20 read_log 192803'15247 (192803'15246) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:157 2014-08-11 12:51:45.069567 2014-08-13 17:38:02.780040 7fe9bae887a0 20 read_log 192803'15248 (192803'15247) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:158 2014-08-11 12:51:45.099702 2014-08-13 17:38:02.780052 7fe9bae887a0 20 read_log 192803'15249 (192803'15248) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:159 2014-08-11 12:51:45.129360 2014-08-13 17:38:02.780063 7fe9bae887a0 20 read_log 192803'15250 (192803'15249) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:160 2014-08-11 12:51:45.156266 2014-08-13 17:38:02.780075 7fe9bae887a0 20 read_log 192803'15251 (192803'15250) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:161 2014-08-11 12:51:45.190255 2014-08-13 17:38:02.780086 7fe9bae887a0 20 read_log 192803'15252 (192803'15251) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:162 2014-08-11 12:51:45.218098 2014-08-13 17:38:02.780098 7fe9bae887a0 20 read_log 192803'15253 (192803'15252) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:163 2014-08-11 12:51:45.253114 2014-08-13 17:38:02.780109 7fe9bae887a0 20 read_log 192803'15254 (192803'15253) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:164 2014-08-11 12:51:45.495913 2014-08-13 17:38:02.780121 7fe9bae887a0 20 read_log 192803'15255 (192803'15254) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:165 2014-08-11 12:51:45.524677 2014-08-13 17:38:02.780133 7fe9bae887a0 20 read_log 192803'15256 (192803'15255) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:166 2014-08-11 12:51:45.558582 2014-08-13 17:38:02.780144 7fe9bae887a0 20 read_log 192803'15257 (192803'15256) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:167 2014-08-11 12:51:45.582147 2014-08-13 17:38:02.780172 7fe9bae887a0 20 read_log 192803'15258 (192803'15257) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:168 2014-08-11 12:51:45.614941 2014-08-13 17:38:02.780185 7fe9bae887a0 20 read_log 192803'15259 (192803'15258) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:169 2014-08-11 12:51:45.637976 2014-08-13 17:38:02.780197 7fe9bae887a0 20 read_log 192803'15260 (192803'15259) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:170 2014-08-11 12:51:45.660468 2014-08-13 17:38:02.780209 7fe9bae887a0 20 read_log 192803'15261 (192803'15260) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:171 2014-08-11 12:51:45.690353 2014-08-13 17:38:02.780221 7fe9bae887a0 20 read_log 192803'15262 (192803'15261) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:172 2014-08-11 12:51:45.717250 2014-08-13 17:38:02.780233 7fe9bae887a0 20 read_log 192803'15263 (192803'15262) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:173 2014-08-11 12:51:45.753957 2014-08-13 17:38:02.780244 7fe9bae887a0 20 read_log 192803'15264 (192803'15263) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:174 2014-08-11 12:51:46.012871 2014-08-13 17:38:02.780256 7fe9bae887a0 20 read_log 192803'15265 (192803'15264) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:175 2014-08-11 12:51:46.041959 2014-08-13 17:38:02.780268 7fe9bae887a0 20 read_log 192803'15266 (192803'15265) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:176 2014-08-11 12:51:46.082156 2014-08-13 17:38:02.780280 7fe9bae887a0 20 read_log 192834'15269 (192834'15269) modify 5e/hit_set_24.5e_archive_2014-08-11 12:48:05.518603_2014-08-11 15:23:37.081884/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:37.082085 2014-08-13 17:38:02.780300 7fe9bae887a0 20 read_log 192834'15270 (192803'14718) delete 5e/hit_set_24.5e_archive_2014-08-11 10:24:43.601998_2014-08-11 12:48:05.517931/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:37.082085 2014-08-13 17:38:02.780316 7fe9bae887a0 20 read_log 192834'15295 (0'0) promote 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by osd.289.0:52117 2014-08-11 15:23:42.581980 2014-08-13 17:38:02.780328 7fe9bae887a0 20 read_log 192834'15296 (192834'15295) modify 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.933448.0:1712 2014-08-11 15:23:42.571637 2014-08-13 17:38:02.780341 7fe9bae887a0 20 read_log 192834'15297 (0'0) promote 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:52127 2014-08-11 15:23:43.365951 2014-08-13 17:38:02.780352 7fe9bae887a0 20 read_log 192834'15298 (192834'15297) modify 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by client.933448.0:1844 2014-08-11 15:23:43.357131 2014-08-13 17:38:02.780365 7fe9bae887a0 20 read_log 192834'15305 (0'0) promote e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by osd.289.0:52149 2014-08-11 15:23:44.834375 2014-08-13 17:38:02.780376 7fe9bae887a0 20 read_log 192834'15306 (192834'15305) modify e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.933448.0:2067 2014-08-11 15:23:44.826459 2014-08-13 17:38:02.780388 7fe9bae887a0 20 read_log 192834'15327 (0'0) promote 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:52193 2014-08-11 15:23:50.068420 2014-08-13 17:38:02.780400 7fe9bae887a0 20 read_log 192834'15328 (192834'15327) modify 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by client.933448.0:3078 2014-08-11 15:23:50.047334 2014-08-13 17:38:02.780413 7fe9bae887a0 20 read_log 192834'15343 (0'0) promote fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by osd.289.0:52265 2014-08-11 15:23:57.307866 2014-08-13 17:38:02.780425 7fe9bae887a0 20 read_log 192834'15344 (192834'15343) modify fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.933448.0:4505 2014-08-11 15:23:57.299725 2014-08-13 17:38:02.780447 7fe9bae887a0 20 read_log 192834'15345 (0'0) promote 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:52271 2014-08-11 15:23:57.812384 2014-08-13 17:38:02.780460 7fe9bae887a0 20 read_log 192834'15346 (192834'15345) modify 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by client.933448.0:4595 2014-08-11 15:23:57.803189 2014-08-13 17:38:02.780493 7fe9bae887a0 20 read_log 192834'15347 (192803'15266) clean a181605e/core.46996+10+2/head//24 by osd.289.0:52279 2014-08-11 15:23:58.247387 2014-08-13 17:38:02.780505 7fe9bae887a0 20 read_log 192834'15348 (0'0) promote df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by osd.289.0:52280 2014-08-11 15:23:58.249582 2014-08-13 17:38:02.780517 7fe9bae887a0 20 read_log 192834'15349 (192834'15348) modify df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.933448.0:4703 2014-08-11 15:23:58.241533 2014-08-13 17:38:02.780530 7fe9bae887a0 20 read_log 192834'15352 (192834'15347) delete a181605e/core.46996+10+2/head//24 by osd.289.0:52288 2014-08-11 15:23:59.377241 2014-08-13 17:38:02.780541 7fe9bae887a0 20 read_log 192834'15369 (0'0) promote 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by osd.289.0:52329 2014-08-11 15:24:02.472772 2014-08-13 17:38:02.780553 7fe9bae887a0 20 read_log 192834'15370 (192834'15369) modify 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.933448.0:5678 2014-08-11 15:24:02.463426 2014-08-13 17:38:02.780566 7fe9bae887a0 20 read_log 192834'15411 (0'0) promote db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by osd.289.0:52457 2014-08-11 15:24:15.047999 2014-08-13 17:38:02.780578 7fe9bae887a0 20 read_log 192834'15412 (192834'15411) modify db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.933448.0:8688 2014-08-11 15:24:15.035509 2014-08-13 17:38:02.780590 7fe9bae887a0 20 read_log 192834'15413 (0'0) promote 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by osd.289.0:52461 2014-08-11 15:24:15.122133 2014-08-13 17:38:02.780601 7fe9bae887a0 20 read_log 192834'15414 (192834'15413) modify 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.933448.0:8708 2014-08-11 15:24:15.113940 2014-08-13 17:38:02.780613 7fe9bae887a0 20 read_log 192834'15417 (0'0) promote d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by osd.289.0:52475 2014-08-11 15:24:15.935584 2014-08-13 17:38:02.780625 7fe9bae887a0 20 read_log 192834'15418 (192834'15417) modify d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.933448.0:8906 2014-08-11 15:24:15.926246 2014-08-13 17:38:02.780637 7fe9bae887a0 20 read_log 192834'15419 (0'0) promote 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by osd.289.0:52487 2014-08-11 15:24:17.520503 2014-08-13 17:38:02.780649 7fe9bae887a0 20 read_log 192834'15420 (192834'15419) modify 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.933448.0:9090 2014-08-11 15:24:17.512392 2014-08-13 17:38:02.780661 7fe9bae887a0 20 read_log 192834'15423 (0'0) promote 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by osd.289.0:52497 2014-08-11 15:24:18.253862 2014-08-13 17:38:02.780673 7fe9bae887a0 20 read_log 192834'15424 (192834'15423) modify 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.933448.0:9279 2014-08-11 15:24:18.245606 2014-08-13 17:38:02.780685 7fe9bae887a0 20 read_log 192834'15439 (0'0) promote 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by osd.289.0:52587 2014-08-11 15:24:28.169430 2014-08-13 17:38:02.780697 7fe9bae887a0 20 read_log 192834'15440 (192834'15439) modify 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.933448.0:11525 2014-08-11 15:24:28.161528 2014-08-13 17:38:02.780717 7fe9bae887a0 20 read_log 192834'15467 (0'0) promote e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by osd.289.0:52711 2014-08-11 15:24:39.464419 2014-08-13 17:38:02.780730 7fe9bae887a0 20 read_log 192834'15468 (192834'15467) modify e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.933448.0:13651 2014-08-11 15:24:39.456155 2014-08-13 17:38:02.780742 7fe9bae887a0 20 read_log 192834'15515 (0'0) promote 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by osd.289.0:52980 2014-08-11 15:24:57.433124 2014-08-13 17:38:02.780755 7fe9bae887a0 20 read_log 192834'15516 (192834'15515) modify 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.933448.0:16853 2014-08-11 15:24:57.422510 2014-08-13 17:38:02.780767 7fe9bae887a0 20 read_log 192834'15534 (0'0) promote 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by osd.289.0:53045 2014-08-11 15:25:03.640900 2014-08-13 17:38:02.780779 7fe9bae887a0 20 read_log 192834'15535 (192834'15534) modify 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.933448.0:18059 2014-08-11 15:25:03.309225 2014-08-13 17:38:02.780791 7fe9bae887a0 20 read_log 192834'15537 (0'0) promote 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by osd.289.0:53094 2014-08-11 15:25:08.703300 2014-08-13 17:38:02.780803 7fe9bae887a0 20 read_log 192834'15538 (192834'15537) modify 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.933448.0:19059 2014-08-11 15:25:08.332658 2014-08-13 17:38:02.780815 7fe9bae887a0 20 read_log 192834'15567 (0'0) promote cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by osd.289.0:53376 2014-08-11 15:25:31.751128 2014-08-13 17:38:02.780827 7fe9bae887a0 20 read_log 192834'15568 (192834'15567) modify cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.933448.0:21602 2014-08-11 15:25:31.179036 2014-08-13 17:38:02.780839 7fe9bae887a0 20 read_log 192834'15585 (192834'15296) modify 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.933448.0:24003 2014-08-11 15:25:40.165000 2014-08-13 17:38:02.780851 7fe9bae887a0 20 read_log 192834'15586 (192834'15298) modify 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by client.933448.0:24135 2014-08-11 15:25:40.318705 2014-08-13 17:38:02.780863 7fe9bae887a0 20 read_log 192834'15590 (192834'15306) modify e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.933448.0:24358 2014-08-11 15:25:40.673433 2014-08-13 17:38:02.780875 7fe9bae887a0 20 read_log 192834'15601 (192834'15328) modify 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by client.933448.0:25369 2014-08-11 15:25:42.033345 2014-08-13 17:38:02.780887 7fe9bae887a0 20 read_log 192834'15609 (192834'15344) modify fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.933448.0:26796 2014-08-11 15:25:43.768760 2014-08-13 17:38:02.780899 7fe9bae887a0 20 read_log 192834'15610 (192834'15346) modify 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by client.933448.0:26886 2014-08-11 15:25:43.840895 2014-08-13 17:38:02.780911 7fe9bae887a0 20 read_log 192834'15611 (192834'15349) modify df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.933448.0:26994 2014-08-11 15:25:43.985609 2014-08-13 17:38:02.780923 7fe9bae887a0 20 read_log 192834'15621 (192834'15370) modify 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.933448.0:27969 2014-08-11 15:25:45.321803 2014-08-13 17:38:02.780935 7fe9bae887a0 20 read_log 192834'15642 (192834'15412) modify db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.933448.0:30979 2014-08-11 15:25:49.473939 2014-08-13 17:38:02.780947 7fe9bae887a0 20 read_log 192834'15643 (192834'15414) modify 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.933448.0:30999 2014-08-11 15:25:49.483423 2014-08-13 17:38:02.780982 7fe9bae887a0 20 read_log 192834'15645 (192834'15418) modify d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.933448.0:31197 2014-08-11 15:25:49.783756 2014-08-13 17:38:02.780998 7fe9bae887a0 20 read_log 192834'15646 (192834'15420) modify 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.933448.0:31381 2014-08-11 15:25:49.989824 2014-08-13 17:38:02.781011 7fe9bae887a0 20 read_log 192834'15648 (192834'15424) modify 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.933448.0:31570 2014-08-11 15:25:50.199046 2014-08-13 17:38:02.781023 7fe9bae887a0 20 read_log 192834'15656 (192834'15440) modify 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.933448.0:33816 2014-08-11 15:25:54.233560 2014-08-13 17:38:02.781035 7fe9bae887a0 20 read_log 192834'15670 (192834'15468) modify e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.933448.0:35942 2014-08-11 15:25:57.031249 2014-08-13 17:38:02.781047 7fe9bae887a0 20 read_log 192834'15694 (192834'15516) modify 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.933448.0:39144 2014-08-11 15:26:01.622184 2014-08-13 17:38:02.781059 7fe9bae887a0 20 read_log 192834'15704 (192834'15535) modify 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.933448.0:40350 2014-08-11 15:26:02.583450 2014-08-13 17:38:02.781071 7fe9bae887a0 20 read_log 192834'15705 (192834'15538) modify 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.933448.0:41350 2014-08-11 15:26:02.917189 2014-08-13 17:38:02.781083 7fe9bae887a0 20 read_log 192834'15720 (192834'15568) modify cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.933448.0:43893 2014-08-11 15:26:03.754790 2014-08-13 17:38:02.781095 7fe9bae887a0 20 read_log 192834'15729 (0'0) promote 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by osd.289.0:54389 2014-08-11 15:26:16.605256 2014-08-13 17:38:02.781107 7fe9bae887a0 20 read_log 192834'15730 (192834'15729) modify 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.1883634.0:579 2014-08-11 15:26:16.596541 2014-08-13 17:38:02.781119 7fe9bae887a0 20 read_log 192834'15745 (0'0) promote 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by osd.289.0:54429 2014-08-11 15:26:21.693416 2014-08-13 17:38:02.781131 7fe9bae887a0 20 read_log 192834'15746 (192834'15745) modify 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.1883634.0:1954 2014-08-11 15:26:21.686018 2014-08-13 17:38:02.781143 7fe9bae887a0 20 read_log 192834'15765 (0'0) promote cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:54489 2014-08-11 15:26:25.993049 2014-08-13 17:38:02.781159 7fe9bae887a0 20 read_log 192834'15766 (192834'15765) modify cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by client.1883634.0:3149 2014-08-11 15:26:25.984546 2014-08-13 17:38:02.781173 7fe9bae887a0 20 read_log 192834'15795 (0'0) promote a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by osd.289.0:54599 2014-08-11 15:26:34.065524 2014-08-13 17:38:02.781185 7fe9bae887a0 20 read_log 192834'15796 (192834'15795) modify a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.1883634.0:5342 2014-08-11 15:26:34.057539 2014-08-13 17:38:02.781197 7fe9bae887a0 20 read_log 192834'15815 (0'0) promote fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by osd.289.0:54677 2014-08-11 15:26:39.647566 2014-08-13 17:38:02.781209 7fe9bae887a0 20 read_log 192834'15816 (192834'15815) modify fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.1883634.0:6820 2014-08-11 15:26:39.614193 2014-08-13 17:38:02.781222 7fe9bae887a0 20 read_log 192834'15827 (0'0) promote bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by osd.289.0:54703 2014-08-11 15:26:42.061184 2014-08-13 17:38:02.781259 7fe9bae887a0 20 read_log 192834'15828 (192834'15827) modify bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.1883634.0:7408 2014-08-11 15:26:42.052017 2014-08-13 17:38:02.781275 7fe9bae887a0 20 read_log 192834'15869 (0'0) promote 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by osd.289.0:55140 2014-08-11 15:26:54.923756 2014-08-13 17:38:02.781287 7fe9bae887a0 20 read_log 192834'15870 (192834'15869) modify 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.1883634.0:10399 2014-08-11 15:26:54.915696 2014-08-13 17:38:02.781299 7fe9bae887a0 20 read_log 192834'15893 (0'0) promote 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by osd.289.0:55357 2014-08-11 15:27:01.387196 2014-08-13 17:38:02.781312 7fe9bae887a0 20 read_log 192834'15894 (192834'15893) modify 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.1883634.0:11812 2014-08-11 15:27:01.343735 2014-08-13 17:38:02.781324 7fe9bae887a0 20 read_log 192834'15903 (0'0) promote da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by osd.289.0:55415 2014-08-11 15:27:06.247124 2014-08-13 17:38:02.781336 7fe9bae887a0 20 read_log 192834'15904 (192834'15903) modify da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.1883634.0:12829 2014-08-11 15:27:06.239621 2014-08-13 17:38:02.781350 7fe9bae887a0 20 read_log 192834'15909 (0'0) promote 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:55453 2014-08-11 15:27:07.351048 2014-08-13 17:38:02.781362 7fe9bae887a0 20 read_log 192834'15910 (192834'15909) modify 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by client.1883634.0:13046 2014-08-11 15:27:07.341666 2014-08-13 17:38:02.781374 7fe9bae887a0 20 read_log 192834'15913 (0'0) promote 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:55545 2014-08-11 15:27:10.061375 2014-08-13 17:38:02.781386 7fe9bae887a0 20 read_log 192834'15914 (192834'15913) modify 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by client.1883634.0:13552 2014-08-11 15:27:10.052171 2014-08-13 17:38:02.781398 7fe9bae887a0 20 read_log 192834'15927 (0'0) promote cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by osd.289.0:55597 2014-08-11 15:27:14.175079 2014-08-13 17:38:02.781410 7fe9bae887a0 20 read_log 192834'15928 (192834'15927) modify cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.1883634.0:14635 2014-08-11 15:27:14.164837 2014-08-13 17:38:02.781423 7fe9bae887a0 20 read_log 192834'15939 (0'0) promote 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:55656 2014-08-11 15:27:20.710803 2014-08-13 17:38:02.781435 7fe9bae887a0 20 read_log 192834'15940 (192834'15939) modify 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by client.1883634.0:15568 2014-08-11 15:27:20.347562 2014-08-13 17:38:02.781447 7fe9bae887a0 20 read_log 192834'15943 (0'0) promote d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by osd.289.0:55661 2014-08-11 15:27:20.945819 2014-08-13 17:38:02.781459 7fe9bae887a0 20 read_log 192834'15944 (192834'15943) modify d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.1883634.0:15601 2014-08-11 15:27:20.796894 2014-08-13 17:38:02.781471 7fe9bae887a0 20 read_log 192834'15951 (0'0) promote 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by osd.289.0:55727 2014-08-11 15:27:28.630170 2014-08-13 17:38:02.781483 7fe9bae887a0 20 read_log 192834'15952 (192834'15951) modify 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.1883634.0:16644 2014-08-11 15:27:28.620972 2014-08-13 17:38:02.781495 7fe9bae887a0 20 read_log 192834'16007 (0'0) promote 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by osd.289.0:56234 2014-08-11 15:27:56.050634 2014-08-13 17:38:02.781514 7fe9bae887a0 20 read_log 192834'16010 (192834'16007) modify 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.1883634.0:20093 2014-08-11 15:27:55.206213 2014-08-13 17:38:02.781527 7fe9bae887a0 20 read_log 192834'16029 (0'0) promote 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by osd.289.0:56363 2014-08-11 15:28:03.795286 2014-08-13 17:38:02.781539 7fe9bae887a0 20 read_log 192834'16030 (192834'16029) modify 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.1883634.0:21323 2014-08-11 15:28:03.785039 2014-08-13 17:38:02.781551 7fe9bae887a0 20 read_log 192834'16047 (0'0) promote 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by osd.289.0:56439 2014-08-11 15:28:07.916215 2014-08-13 17:38:02.781563 7fe9bae887a0 20 read_log 192834'16048 (192834'16047) modify 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.1883634.0:22298 2014-08-11 15:28:07.906780 2014-08-13 17:38:02.781575 7fe9bae887a0 20 read_log 192834'16062 (192834'15730) modify 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.1883634.0:23903 2014-08-11 15:28:18.063093 2014-08-13 17:38:02.781588 7fe9bae887a0 20 read_log 192834'16070 (192834'15746) modify 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.1883634.0:25278 2014-08-11 15:28:20.450628 2014-08-13 17:38:02.781600 7fe9bae887a0 20 read_log 192834'16080 (192834'15766) modify cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by client.1883634.0:26473 2014-08-11 15:28:23.175466 2014-08-13 17:38:02.781612 7fe9bae887a0 20 read_log 192834'16095 (192834'15796) modify a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.1883634.0:28666 2014-08-11 15:28:26.117023 2014-08-13 17:38:02.781624 7fe9bae887a0 20 read_log 192834'16105 (192834'15816) modify fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.1883634.0:30144 2014-08-11 15:28:28.163152 2014-08-13 17:38:02.781635 7fe9bae887a0 20 read_log 192834'16111 (192834'15828) modify bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.1883634.0:30732 2014-08-11 15:28:29.229322 2014-08-13 17:38:02.781647 7fe9bae887a0 20 read_log 192834'16132 (192834'15870) modify 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.1883634.0:33723 2014-08-11 15:28:33.719119 2014-08-13 17:38:02.781659 7fe9bae887a0 20 read_log 192834'16144 (192834'15894) modify 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.1883634.0:35136 2014-08-11 15:28:36.066344 2014-08-13 17:38:02.781671 7fe9bae887a0 20 read_log 192834'16148 (192834'16148) modify 5e/hit_set_24.5e_archive_2014-08-11 15:23:37.082089_2014-08-11 15:28:37.297779/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:37.298018 2014-08-13 17:38:02.781683 7fe9bae887a0 20 read_log 192834'16149 (192834'15269) delete 5e/hit_set_24.5e_archive_2014-08-11 12:48:05.518603_2014-08-11 15:23:37.081884/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:37.298018 2014-08-13 17:38:02.781696 7fe9bae887a0 20 read_log 192834'16151 (192834'15904) modify da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.1883634.0:36153 2014-08-11 15:28:37.652886 2014-08-13 17:38:02.781708 7fe9bae887a0 20 read_log 192834'16154 (192834'15910) modify 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by client.1883634.0:36370 2014-08-11 15:28:37.952304 2014-08-13 17:38:02.781720 7fe9bae887a0 20 read_log 192834'16156 (192834'15914) modify 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by client.1883634.0:36876 2014-08-11 15:28:38.526662 2014-08-13 17:38:02.781732 7fe9bae887a0 20 read_log 192834'16163 (192834'15928) modify cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.1883634.0:37959 2014-08-11 15:28:40.020890 2014-08-13 17:38:02.781751 7fe9bae887a0 20 read_log 192834'16169 (192834'15940) modify 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by client.1883634.0:38892 2014-08-11 15:28:41.250112 2014-08-13 17:38:02.781764 7fe9bae887a0 20 read_log 192834'16171 (192834'15944) modify d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.1883634.0:38925 2014-08-11 15:28:41.284398 2014-08-13 17:38:02.781775 7fe9bae887a0 20 read_log 192834'16175 (192834'15952) modify 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.1883634.0:39968 2014-08-11 15:28:42.985654 2014-08-13 17:38:02.781787 7fe9bae887a0 20 read_log 192834'16203 (192834'16010) modify 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.1883634.0:43417 2014-08-11 15:28:46.547267 2014-08-13 17:38:02.781799 7fe9bae887a0 20 read_log 192834'16214 (192834'16030) modify 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.1883634.0:44647 2014-08-11 15:28:46.915324 2014-08-13 17:38:02.781811 7fe9bae887a0 20 read_log 192834'16223 (192834'16048) modify 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.1883634.0:45622 2014-08-11 15:28:47.335549 2014-08-13 17:38:02.781823 7fe9bae887a0 20 read_log 192888'16229 (192888'16229) modify 5e/hit_set_24.5e_archive_2014-08-11 16:02:02.174496_2014-08-11 16:44:57.949927/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.950147 2014-08-13 17:38:02.781836 7fe9bae887a0 20 read_log 192888'16230 (192834'16148) delete 5e/hit_set_24.5e_archive_2014-08-11 15:23:37.082089_2014-08-11 15:28:37.297779/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.950147 2014-08-13 17:38:02.781848 7fe9bae887a0 20 read_log 192888'16235 (0'0) promote d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by osd.289.0:58607 2014-08-11 16:45:02.224217 2014-08-13 17:38:02.781862 7fe9bae887a0 20 read_log 192888'16236 (192888'16235) modify d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.1315259.0:1824 2014-08-11 16:45:02.216620 2014-08-13 17:38:02.781874 7fe9bae887a0 20 read_log 192888'16237 (0'0) promote 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by osd.289.0:58613 2014-08-11 16:45:02.561952 2014-08-13 17:38:02.781886 7fe9bae887a0 20 read_log 192888'16238 (192888'16237) modify 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.1315259.0:1915 2014-08-11 16:45:02.553046 2014-08-13 17:38:02.781898 7fe9bae887a0 20 read_log 192888'16239 (0'0) promote 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by osd.289.0:58705 2014-08-11 16:45:16.983742 2014-08-13 17:38:02.781910 7fe9bae887a0 20 read_log 192888'16240 (192888'16239) modify 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.1315259.0:5778 2014-08-11 16:45:16.975357 2014-08-13 17:38:02.781922 7fe9bae887a0 20 read_log 192888'16247 (0'0) promote 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:58725 2014-08-11 16:45:19.366169 2014-08-13 17:38:02.781934 7fe9bae887a0 20 read_log 192888'16248 (192888'16247) modify 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by client.1315259.0:6438 2014-08-11 16:45:19.358092 2014-08-13 17:38:02.781946 7fe9bae887a0 20 read_log 192888'16257 (0'0) promote bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by osd.289.0:58785 2014-08-11 16:45:29.097856 2014-08-13 17:38:02.781958 7fe9bae887a0 20 read_log 192888'16258 (192888'16257) modify bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.1315259.0:9064 2014-08-11 16:45:29.090278 2014-08-13 17:38:02.781970 7fe9bae887a0 20 read_log 192888'16261 (0'0) promote a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:58795 2014-08-11 16:45:30.581054 2014-08-13 17:38:02.781992 7fe9bae887a0 20 read_log 192888'16262 (192888'16261) modify a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by client.1315259.0:9448 2014-08-11 16:45:30.572157 2014-08-13 17:38:02.782005 7fe9bae887a0 20 read_log 192888'16271 (0'0) promote 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by osd.289.0:58819 2014-08-11 16:45:36.344037 2014-08-13 17:38:02.782038 7fe9bae887a0 20 read_log 192888'16272 (192888'16271) modify 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.1315259.0:10931 2014-08-11 16:45:36.334786 2014-08-13 17:38:02.782056 7fe9bae887a0 20 read_log 192888'16273 (0'0) promote be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by osd.289.0:58823 2014-08-11 16:45:36.977441 2014-08-13 17:38:02.782068 7fe9bae887a0 20 read_log 192888'16274 (192888'16273) modify be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.1315259.0:11089 2014-08-11 16:45:36.970864 2014-08-13 17:38:02.782080 7fe9bae887a0 20 read_log 192888'16275 (0'0) promote 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by osd.289.0:58845 2014-08-11 16:45:41.700699 2014-08-13 17:38:02.782092 7fe9bae887a0 20 read_log 192888'16276 (192888'16275) modify 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.1315259.0:12260 2014-08-11 16:45:41.693138 2014-08-13 17:38:02.782104 7fe9bae887a0 20 read_log 192888'16283 (0'0) promote 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by osd.289.0:58859 2014-08-11 16:45:44.436103 2014-08-13 17:38:02.782116 7fe9bae887a0 20 read_log 192888'16284 (192888'16283) modify 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.1315259.0:12905 2014-08-11 16:45:44.428863 2014-08-13 17:38:02.782128 7fe9bae887a0 20 read_log 192888'16287 (0'0) promote e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by osd.289.0:58867 2014-08-11 16:45:45.122324 2014-08-13 17:38:02.782140 7fe9bae887a0 20 read_log 192888'16288 (192888'16287) modify e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.1315259.0:13065 2014-08-11 16:45:45.113113 2014-08-13 17:38:02.782158 7fe9bae887a0 20 read_log 192888'16289 (0'0) promote 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by osd.289.0:58871 2014-08-11 16:45:45.801271 2014-08-13 17:38:02.782171 7fe9bae887a0 20 read_log 192888'16290 (192888'16289) modify 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.1315259.0:13238 2014-08-11 16:45:45.790728 2014-08-13 17:38:02.782183 7fe9bae887a0 20 read_log 192888'16291 (0'0) promote d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by osd.289.0:58875 2014-08-11 16:45:45.923528 2014-08-13 17:38:02.782196 7fe9bae887a0 20 read_log 192888'16292 (192888'16291) modify d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.1315259.0:13270 2014-08-11 16:45:45.912169 2014-08-13 17:38:02.782208 7fe9bae887a0 20 read_log 192888'16295 (0'0) promote 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by osd.289.0:58881 2014-08-11 16:45:47.889717 2014-08-13 17:38:02.782220 7fe9bae887a0 20 read_log 192888'16296 (192888'16295) modify 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.1315259.0:13762 2014-08-11 16:45:47.881662 2014-08-13 17:38:02.782232 7fe9bae887a0 20 read_log 192888'16299 (0'0) promote 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by osd.289.0:58911 2014-08-11 16:45:53.883084 2014-08-13 17:38:02.782245 7fe9bae887a0 20 read_log 192888'16300 (192888'16299) modify 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.1315259.0:15345 2014-08-11 16:45:53.875124 2014-08-13 17:38:02.782258 7fe9bae887a0 20 read_log 192888'16301 (0'0) promote 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by osd.289.0:58935 2014-08-11 16:45:59.279291 2014-08-13 17:38:02.782278 7fe9bae887a0 20 read_log 192888'16302 (192888'16301) modify 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.1315259.0:16730 2014-08-11 16:45:59.270806 2014-08-13 17:38:02.782291 7fe9bae887a0 20 read_log 192888'16305 (0'0) promote 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by osd.289.0:58945 2014-08-11 16:46:00.234643 2014-08-13 17:38:02.782303 7fe9bae887a0 20 read_log 192888'16306 (192888'16305) modify 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.1315259.0:16976 2014-08-11 16:46:00.225503 2014-08-13 17:38:02.782315 7fe9bae887a0 20 read_log 192888'16307 (0'0) promote 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by osd.289.0:58947 2014-08-11 16:46:00.356730 2014-08-13 17:38:02.782327 7fe9bae887a0 20 read_log 192888'16308 (192888'16307) modify 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.1315259.0:17000 2014-08-11 16:46:00.348630 2014-08-13 17:38:02.782339 7fe9bae887a0 20 read_log 192888'16309 (0'0) promote 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by osd.289.0:58961 2014-08-11 16:46:03.904274 2014-08-13 17:38:02.782351 7fe9bae887a0 20 read_log 192888'16310 (192888'16309) modify 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.1315259.0:17879 2014-08-11 16:46:03.895214 2014-08-13 17:38:02.782365 7fe9bae887a0 20 read_log 192888'16317 (0'0) promote ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by osd.289.0:58999 2014-08-11 16:46:13.141423 2014-08-13 17:38:02.782377 7fe9bae887a0 20 read_log 192888'16318 (192888'16317) modify ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.1315259.0:20277 2014-08-11 16:46:13.132097 2014-08-13 17:38:02.782389 7fe9bae887a0 20 read_log 192888'16319 (0'0) promote 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by osd.289.0:59017 2014-08-11 16:46:16.464926 2014-08-13 17:38:02.782400 7fe9bae887a0 20 read_log 192888'16320 (192888'16319) modify 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.1315259.0:21160 2014-08-11 16:46:16.454651 2014-08-13 17:38:02.782412 7fe9bae887a0 20 read_log 192888'16327 (0'0) promote 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by osd.289.0:59065 2014-08-11 16:46:28.504274 2014-08-13 17:38:02.782424 7fe9bae887a0 20 read_log 192888'16328 (192888'16327) modify 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.1315259.0:24231 2014-08-11 16:46:28.497227 2014-08-13 17:38:02.782436 7fe9bae887a0 20 read_log 192888'16329 (0'0) promote 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:59089 2014-08-11 16:46:32.308198 2014-08-13 17:38:02.782448 7fe9bae887a0 20 read_log 192888'16330 (192888'16329) modify 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by client.1315259.0:25200 2014-08-11 16:46:32.295225 2014-08-13 17:38:02.782478 7fe9bae887a0 20 read_log 192888'16331 (0'0) promote 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:59093 2014-08-11 16:46:33.082691 2014-08-13 17:38:02.782491 7fe9bae887a0 20 read_log 192888'16332 (192888'16331) modify 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by client.1315259.0:25385 2014-08-11 16:46:33.072615 2014-08-13 17:38:02.782503 7fe9bae887a0 20 read_log 192888'16333 (0'0) promote abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by osd.289.0:59117 2014-08-11 16:46:37.643636 2014-08-13 17:38:02.782515 7fe9bae887a0 20 read_log 192888'16334 (192888'16333) modify abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.1315259.0:26436 2014-08-11 16:46:37.635532 2014-08-13 17:38:02.782528 7fe9bae887a0 20 read_log 192888'16337 (0'0) promote bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by osd.289.0:59135 2014-08-11 16:46:41.273907 2014-08-13 17:38:02.782546 7fe9bae887a0 20 read_log 192888'16338 (192888'16337) modify bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.1315259.0:27310 2014-08-11 16:46:41.265343 2014-08-13 17:38:02.782559 7fe9bae887a0 20 read_log 192888'16339 (0'0) promote 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by osd.289.0:59137 2014-08-11 16:46:41.370576 2014-08-13 17:38:02.782571 7fe9bae887a0 20 read_log 192888'16340 (192888'16339) modify 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.1315259.0:27333 2014-08-11 16:46:41.355114 2014-08-13 17:38:02.782583 7fe9bae887a0 20 read_log 192888'16341 (0'0) promote 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:59141 2014-08-11 16:46:41.768267 2014-08-13 17:38:02.782595 7fe9bae887a0 20 read_log 192888'16342 (192888'16341) modify 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by client.1315259.0:27437 2014-08-11 16:46:41.759469 2014-08-13 17:38:02.782608 7fe9bae887a0 20 read_log 192888'16343 (0'0) promote dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by osd.289.0:59155 2014-08-11 16:46:43.004666 2014-08-13 17:38:02.782620 7fe9bae887a0 20 read_log 192888'16344 (192888'16343) modify dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.1315259.0:27754 2014-08-11 16:46:42.996425 2014-08-13 17:38:02.782632 7fe9bae887a0 20 read_log 192888'16347 (0'0) promote 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:59161 2014-08-11 16:46:44.404343 2014-08-13 17:38:02.782643 7fe9bae887a0 20 read_log 192888'16348 (192888'16347) modify 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by client.1315259.0:28113 2014-08-11 16:46:44.394288 2014-08-13 17:38:02.782656 7fe9bae887a0 20 read_log 192888'16353 (0'0) promote 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by osd.289.0:59167 2014-08-11 16:46:45.964771 2014-08-13 17:38:02.782667 7fe9bae887a0 20 read_log 192888'16354 (192888'16353) modify 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.1315259.0:28464 2014-08-11 16:46:45.955841 2014-08-13 17:38:02.782679 7fe9bae887a0 20 read_log 192888'16357 (0'0) promote 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:59179 2014-08-11 16:46:47.385676 2014-08-13 17:38:02.782691 7fe9bae887a0 20 read_log 192888'16358 (192888'16357) modify 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by client.1315259.0:28832 2014-08-11 16:46:47.377417 2014-08-13 17:38:02.782703 7fe9bae887a0 20 read_log 192888'16359 (0'0) promote 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by osd.289.0:59183 2014-08-11 16:46:47.595462 2014-08-13 17:38:02.782732 7fe9bae887a0 20 read_log 192888'16360 (192888'16359) modify 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.1315259.0:28887 2014-08-11 16:46:47.588052 2014-08-13 17:38:02.782748 7fe9bae887a0 20 read_log 192888'16365 (0'0) promote 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by osd.289.0:59197 2014-08-11 16:46:51.586904 2014-08-13 17:38:02.782760 7fe9bae887a0 20 read_log 192888'16366 (192888'16365) modify 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.1315259.0:29858 2014-08-11 16:46:51.579316 2014-08-13 17:38:02.782772 7fe9bae887a0 20 read_log 192888'16369 (192888'16236) modify d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.1315259.0:32551 2014-08-11 16:46:56.408217 2014-08-13 17:38:02.782784 7fe9bae887a0 20 read_log 192888'16370 (192888'16238) modify 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.1315259.0:32642 2014-08-11 16:46:56.432730 2014-08-13 17:38:02.782796 7fe9bae887a0 20 read_log 192888'16371 (192888'16240) modify 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.1315259.0:36505 2014-08-11 16:46:57.439893 2014-08-13 17:38:02.782833 7fe9bae887a0 20 read_log 192888'16375 (192888'16248) modify 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by client.1315259.0:37165 2014-08-11 16:46:57.612278 2014-08-13 17:38:02.782847 7fe9bae887a0 20 read_log 192888'16380 (192888'16258) modify bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.1315259.0:39791 2014-08-11 16:46:58.303982 2014-08-13 17:38:02.782859 7fe9bae887a0 20 read_log 192888'16382 (192888'16262) modify a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by client.1315259.0:40175 2014-08-11 16:46:58.411187 2014-08-13 17:38:02.782871 7fe9bae887a0 20 read_log 192888'16387 (192888'16272) modify 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.1315259.0:41658 2014-08-11 16:46:58.802744 2014-08-13 17:38:02.782883 7fe9bae887a0 20 read_log 192888'16388 (192888'16274) modify be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.1315259.0:41816 2014-08-11 16:46:58.845865 2014-08-13 17:38:02.782894 7fe9bae887a0 20 read_log 192888'16389 (192888'16276) modify 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.1315259.0:42987 2014-08-11 16:46:59.159820 2014-08-13 17:38:02.782906 7fe9bae887a0 20 read_log 192888'16393 (192888'16284) modify 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.1315259.0:43632 2014-08-11 16:46:59.344544 2014-08-13 17:38:02.782918 7fe9bae887a0 20 read_log 192888'16395 (192888'16288) modify e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.1315259.0:43792 2014-08-11 16:46:59.389746 2014-08-13 17:38:02.782930 7fe9bae887a0 20 read_log 192888'16396 (192888'16290) modify 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.1315259.0:43965 2014-08-11 16:46:59.441253 2014-08-13 17:38:02.782942 7fe9bae887a0 20 read_log 192888'16397 (192888'16292) modify d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.1315259.0:43997 2014-08-11 16:46:59.451702 2014-08-13 17:38:02.782954 7fe9bae887a0 20 read_log 192888'16399 (192888'16296) modify 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.1315259.0:44489 2014-08-11 16:46:59.593022 2014-08-13 17:38:02.782966 7fe9bae887a0 20 read_log 192888'16401 (192888'16300) modify 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.1315259.0:46072 2014-08-11 16:47:00.052186 2014-08-13 17:38:02.782978 7fe9bae887a0 20 read_log 192888'16402 (192888'16302) modify 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.1315259.0:47457 2014-08-11 16:47:00.461523 2014-08-13 17:38:02.782990 7fe9bae887a0 20 read_log 192888'16404 (192888'16306) modify 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.1315259.0:47703 2014-08-11 16:47:00.541404 2014-08-13 17:38:02.783002 7fe9bae887a0 20 read_log 192888'16405 (192888'16308) modify 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.1315259.0:47727 2014-08-11 16:47:00.548841 2014-08-13 17:38:02.783013 7fe9bae887a0 20 read_log 192888'16406 (192888'16310) modify 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.1315259.0:48606 2014-08-11 16:47:00.811560 2014-08-13 17:38:02.783025 7fe9bae887a0 20 read_log 192888'16410 (192888'16318) modify ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.1315259.0:51004 2014-08-11 16:47:01.547227 2014-08-13 17:38:02.783037 7fe9bae887a0 20 read_log 192888'16411 (192888'16320) modify 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.1315259.0:51887 2014-08-11 16:47:01.813112 2014-08-13 17:38:02.783049 7fe9bae887a0 20 read_log 192888'16415 (192888'16328) modify 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.1315259.0:54958 2014-08-11 16:47:02.769139 2014-08-13 17:38:02.783061 7fe9bae887a0 20 read_log 192888'16416 (192888'16330) modify 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by client.1315259.0:55927 2014-08-11 16:47:03.054582 2014-08-13 17:38:02.783080 7fe9bae887a0 20 read_log 192888'16417 (192888'16332) modify 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by client.1315259.0:56112 2014-08-11 16:47:03.108902 2014-08-13 17:38:02.783092 7fe9bae887a0 20 read_log 192888'16418 (192888'16334) modify abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.1315259.0:57163 2014-08-11 16:47:03.429301 2014-08-13 17:38:02.783105 7fe9bae887a0 20 read_log 192888'16420 (192888'16338) modify bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.1315259.0:58037 2014-08-11 16:47:03.684938 2014-08-13 17:38:02.783116 7fe9bae887a0 20 read_log 192888'16421 (192888'16340) modify 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.1315259.0:58060 2014-08-11 16:47:03.691422 2014-08-13 17:38:02.783128 7fe9bae887a0 20 read_log 192888'16422 (192888'16342) modify 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by client.1315259.0:58164 2014-08-11 16:47:03.723457 2014-08-13 17:38:02.783140 7fe9bae887a0 20 read_log 192888'16423 (192888'16344) modify dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.1315259.0:58481 2014-08-11 16:47:03.815361 2014-08-13 17:38:02.783152 7fe9bae887a0 20 read_log 192888'16425 (192888'16348) modify 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by client.1315259.0:58840 2014-08-11 16:47:03.923358 2014-08-13 17:38:02.783170 7fe9bae887a0 20 read_log 192888'16428 (192888'16354) modify 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.1315259.0:59191 2014-08-11 16:47:04.029430 2014-08-13 17:38:02.783182 7fe9bae887a0 20 read_log 192888'16430 (192888'16358) modify 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by client.1315259.0:59559 2014-08-11 16:47:04.159786 2014-08-13 17:38:02.783194 7fe9bae887a0 20 read_log 192888'16431 (192888'16360) modify 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.1315259.0:59614 2014-08-11 16:47:04.182780 2014-08-13 17:38:02.783206 7fe9bae887a0 20 read_log 192888'16434 (192888'16366) modify 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.1315259.0:60585 2014-08-11 16:47:04.483820 2014-08-13 17:38:02.783218 7fe9bae887a0 20 read_log 192888'16435 (192888'16435) modify 5e/hit_set_24.5e_archive_2014-08-11 16:44:57.950151_2014-08-11 17:05:17.864913/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.865135 2014-08-13 17:38:02.783231 7fe9bae887a0 20 read_log 192888'16436 (192888'16229) delete 5e/hit_set_24.5e_archive_2014-08-11 16:02:02.174496_2014-08-11 16:44:57.949927/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.865135 2014-08-13 17:38:02.783243 7fe9bae887a0 20 read_log 192888'16441 (0'0) promote ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by osd.289.0:59575 2014-08-11 17:05:25.931326 2014-08-13 17:38:02.783255 7fe9bae887a0 20 read_log 192888'16442 (192888'16441) modify ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.942868.0:2699 2014-08-11 17:05:25.924315 2014-08-13 17:38:02.783267 7fe9bae887a0 20 read_log 192888'16443 (0'0) promote e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by osd.289.0:59581 2014-08-11 17:05:27.926967 2014-08-13 17:38:02.783279 7fe9bae887a0 20 read_log 192888'16444 (192888'16443) modify e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.942868.0:3239 2014-08-11 17:05:27.917895 2014-08-13 17:38:02.783291 7fe9bae887a0 20 read_log 192888'16453 (0'0) promote 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by osd.289.0:59697 2014-08-11 17:05:52.558911 2014-08-13 17:38:02.783303 7fe9bae887a0 20 read_log 192888'16454 (192888'16453) modify 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.942868.0:9821 2014-08-11 17:05:52.550432 2014-08-13 17:38:02.783322 7fe9bae887a0 20 read_log 192888'16459 (0'0) promote edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by osd.289.0:59731 2014-08-11 17:05:58.088047 2014-08-13 17:38:02.783335 7fe9bae887a0 20 read_log 192888'16460 (192888'16459) modify edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.942868.0:11339 2014-08-11 17:05:58.079898 2014-08-13 17:38:02.783347 7fe9bae887a0 20 read_log 192888'16461 (0'0) promote 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by osd.289.0:59733 2014-08-11 17:05:58.109620 2014-08-13 17:38:02.783360 7fe9bae887a0 20 read_log 192888'16462 (192888'16461) modify 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.942868.0:11345 2014-08-11 17:05:58.101944 2014-08-13 17:38:02.783372 7fe9bae887a0 20 read_log 192888'16463 (0'0) promote 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by osd.289.0:59735 2014-08-11 17:05:58.542574 2014-08-13 17:38:02.783384 7fe9bae887a0 20 read_log 192888'16464 (192888'16463) modify 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.942868.0:11464 2014-08-11 17:05:58.534488 2014-08-13 17:38:02.783396 7fe9bae887a0 20 read_log 192888'16465 (0'0) promote d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by osd.289.0:59741 2014-08-11 17:05:59.849564 2014-08-13 17:38:02.783408 7fe9bae887a0 20 read_log 192888'16466 (192888'16465) modify d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.942868.0:11817 2014-08-11 17:05:59.841072 2014-08-13 17:38:02.783420 7fe9bae887a0 20 read_log 192888'16469 (0'0) promote 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by osd.289.0:59755 2014-08-11 17:06:02.614900 2014-08-13 17:38:02.783432 7fe9bae887a0 20 read_log 192888'16470 (192888'16469) modify 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.942868.0:12589 2014-08-11 17:06:02.606382 2014-08-13 17:38:02.783444 7fe9bae887a0 20 read_log 192888'16475 (0'0) promote 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by osd.289.0:59775 2014-08-11 17:06:06.472642 2014-08-13 17:38:02.783456 7fe9bae887a0 20 read_log 192888'16476 (192888'16475) modify 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.942868.0:13577 2014-08-11 17:06:06.464147 2014-08-13 17:38:02.783468 7fe9bae887a0 20 read_log 192888'16479 (0'0) promote 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by osd.289.0:59787 2014-08-11 17:06:08.549408 2014-08-13 17:38:02.783480 7fe9bae887a0 20 read_log 192888'16480 (192888'16479) modify 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.942868.0:14078 2014-08-11 17:06:08.541220 2014-08-13 17:38:02.783492 7fe9bae887a0 20 read_log 192888'16481 (0'0) promote 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by osd.289.0:59795 2014-08-11 17:06:10.853143 2014-08-13 17:38:02.783504 7fe9bae887a0 20 read_log 192888'16482 (192888'16481) modify 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.942868.0:14647 2014-08-11 17:06:10.844443 2014-08-13 17:38:02.783516 7fe9bae887a0 20 read_log 192888'16483 (0'0) promote 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by osd.289.0:59797 2014-08-11 17:06:11.031839 2014-08-13 17:38:02.783528 7fe9bae887a0 20 read_log 192888'16484 (192888'16483) modify 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.942868.0:14694 2014-08-11 17:06:11.024312 2014-08-13 17:38:02.783540 7fe9bae887a0 20 read_log 192888'16487 (0'0) promote eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by osd.289.0:59831 2014-08-11 17:06:18.259522 2014-08-13 17:38:02.783552 7fe9bae887a0 20 read_log 192888'16488 (192888'16487) modify eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.942868.0:16536 2014-08-11 17:06:18.249107 2014-08-13 17:38:02.783573 7fe9bae887a0 20 read_log 192888'16489 (0'0) promote 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by osd.289.0:59835 2014-08-11 17:06:19.261080 2014-08-13 17:38:02.783586 7fe9bae887a0 20 read_log 192888'16490 (192888'16489) modify 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.942868.0:16800 2014-08-11 17:06:19.251260 2014-08-13 17:38:02.783598 7fe9bae887a0 20 read_log 192888'16499 (0'0) promote 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by osd.289.0:59867 2014-08-11 17:06:28.083323 2014-08-13 17:38:02.783610 7fe9bae887a0 20 read_log 192888'16500 (192888'16499) modify 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.942868.0:19127 2014-08-11 17:06:28.074467 2014-08-13 17:38:02.783622 7fe9bae887a0 20 read_log 192888'16505 (0'0) promote 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by osd.289.0:59905 2014-08-11 17:06:38.217706 2014-08-13 17:38:02.783634 7fe9bae887a0 20 read_log 192888'16506 (192888'16505) modify 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.942868.0:21797 2014-08-11 17:06:38.208130 2014-08-13 17:38:02.783646 7fe9bae887a0 20 read_log 192888'16507 (0'0) promote 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by osd.289.0:59911 2014-08-11 17:06:39.874383 2014-08-13 17:38:02.783658 7fe9bae887a0 20 read_log 192888'16508 (192888'16507) modify 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.942868.0:22229 2014-08-11 17:06:39.863297 2014-08-13 17:38:02.783670 7fe9bae887a0 20 read_log 192888'16509 (0'0) promote 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by osd.289.0:59921 2014-08-11 17:06:41.228441 2014-08-13 17:38:02.783701 7fe9bae887a0 20 read_log 192888'16510 (192888'16509) modify 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.942868.0:22559 2014-08-11 17:06:41.219089 2014-08-13 17:38:02.783714 7fe9bae887a0 20 read_log 192888'16515 (0'0) promote ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by osd.289.0:59949 2014-08-11 17:06:46.563774 2014-08-13 17:38:02.783726 7fe9bae887a0 20 read_log 192888'16516 (192888'16515) modify ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.942868.0:23960 2014-08-11 17:06:46.555674 2014-08-13 17:38:02.783738 7fe9bae887a0 20 read_log 192888'16527 (0'0) promote 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by osd.289.0:60019 2014-08-11 17:07:02.977283 2014-08-13 17:38:02.783751 7fe9bae887a0 20 read_log 192888'16528 (192888'16527) modify 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.942868.0:28269 2014-08-11 17:07:02.969843 2014-08-13 17:38:02.783763 7fe9bae887a0 20 read_log 192888'16533 (0'0) promote 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by osd.289.0:60035 2014-08-11 17:07:07.118681 2014-08-13 17:38:02.783775 7fe9bae887a0 20 read_log 192888'16534 (192888'16533) modify 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.942868.0:29339 2014-08-11 17:07:07.109202 2014-08-13 17:38:02.783787 7fe9bae887a0 20 read_log 192888'16537 (0'0) promote b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by osd.289.0:60043 2014-08-11 17:07:08.290672 2014-08-13 17:38:02.783799 7fe9bae887a0 20 read_log 192888'16538 (192888'16537) modify b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.942868.0:29618 2014-08-11 17:07:08.283107 2014-08-13 17:38:02.783811 7fe9bae887a0 20 read_log 192888'16547 (192888'16442) modify ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.942868.0:33965 2014-08-11 17:07:16.660561 2014-08-13 17:38:02.783823 7fe9bae887a0 20 read_log 192888'16548 (192888'16444) modify e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.942868.0:34505 2014-08-11 17:07:16.807626 2014-08-13 17:38:02.783842 7fe9bae887a0 20 read_log 192888'16553 (192888'16454) modify 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.942868.0:41087 2014-08-11 17:07:18.635237 2014-08-13 17:38:02.783855 7fe9bae887a0 20 read_log 192888'16556 (192888'16460) modify edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.942868.0:42605 2014-08-11 17:07:19.043003 2014-08-13 17:38:02.783867 7fe9bae887a0 20 read_log 192888'16557 (192888'16462) modify 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.942868.0:42611 2014-08-11 17:07:19.044268 2014-08-13 17:38:02.783880 7fe9bae887a0 20 read_log 192888'16558 (192888'16464) modify 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.942868.0:42730 2014-08-11 17:07:19.077358 2014-08-13 17:38:02.783892 7fe9bae887a0 20 read_log 192888'16559 (192888'16466) modify d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.942868.0:43083 2014-08-11 17:07:19.172797 2014-08-13 17:38:02.783904 7fe9bae887a0 20 read_log 192888'16561 (192888'16470) modify 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.942868.0:43855 2014-08-11 17:07:19.396078 2014-08-13 17:38:02.783916 7fe9bae887a0 20 read_log 192888'16564 (192888'16476) modify 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.942868.0:44843 2014-08-11 17:07:19.677189 2014-08-13 17:38:02.783927 7fe9bae887a0 20 read_log 192888'16566 (192888'16480) modify 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.942868.0:45344 2014-08-11 17:07:19.812314 2014-08-13 17:38:02.783939 7fe9bae887a0 20 read_log 192888'16567 (192888'16482) modify 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.942868.0:45913 2014-08-11 17:07:19.969488 2014-08-13 17:38:02.783951 7fe9bae887a0 20 read_log 192888'16568 (192888'16484) modify 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.942868.0:45960 2014-08-11 17:07:19.982180 2014-08-13 17:38:02.783963 7fe9bae887a0 20 read_log 192888'16570 (192888'16488) modify eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.942868.0:47802 2014-08-11 17:07:20.523667 2014-08-13 17:38:02.783975 7fe9bae887a0 20 read_log 192888'16571 (192888'16490) modify 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.942868.0:48066 2014-08-11 17:07:20.612008 2014-08-13 17:38:02.783987 7fe9bae887a0 20 read_log 192888'16576 (192888'16500) modify 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.942868.0:50393 2014-08-11 17:07:21.305271 2014-08-13 17:38:02.784000 7fe9bae887a0 20 read_log 192888'16579 (192888'16506) modify 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.942868.0:53063 2014-08-11 17:07:22.137300 2014-08-13 17:38:02.784012 7fe9bae887a0 20 read_log 192888'16580 (192888'16508) modify 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.942868.0:53495 2014-08-11 17:07:22.285758 2014-08-13 17:38:02.784024 7fe9bae887a0 20 read_log 192888'16581 (192888'16510) modify 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.942868.0:53825 2014-08-11 17:07:22.387263 2014-08-13 17:38:02.784036 7fe9bae887a0 20 read_log 192888'16584 (192888'16516) modify ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.942868.0:55226 2014-08-11 17:07:22.779504 2014-08-13 17:38:02.784048 7fe9bae887a0 20 read_log 192888'16590 (192888'16528) modify 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.942868.0:59535 2014-08-11 17:07:24.074025 2014-08-13 17:38:02.784060 7fe9bae887a0 20 read_log 192888'16593 (192888'16534) modify 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.942868.0:60605 2014-08-11 17:07:24.387846 2014-08-13 17:38:02.784072 7fe9bae887a0 20 read_log 192888'16595 (192888'16538) modify b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.942868.0:60884 2014-08-11 17:07:24.474119 2014-08-13 17:38:02.784091 7fe9bae887a0 20 read_log 193305'16599 (193305'16599) modify 5e/hit_set_24.5e_archive_2014-08-11 18:37:04.920337_2014-08-11 18:42:44.060681/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:44.060900 2014-08-13 17:38:02.784104 7fe9bae887a0 20 read_log 193305'16600 (192888'16435) delete 5e/hit_set_24.5e_archive_2014-08-11 16:44:57.950151_2014-08-11 17:05:17.864913/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:44.060900 2014-08-13 17:38:02.784117 7fe9bae887a0 20 read_log 193305'16601 (0'0) promote 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:63 2014-08-11 18:42:44.078599 2014-08-13 17:38:02.784129 7fe9bae887a0 20 read_log 193305'16602 (193305'16601) modify 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by client.951337.0:2910 2014-08-11 18:42:44.055502 2014-08-13 17:38:02.784141 7fe9bae887a0 20 read_log 193305'16603 (0'0) promote 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by osd.289.0:79 2014-08-11 18:42:47.212126 2014-08-13 17:38:02.784166 7fe9bae887a0 20 read_log 193305'16604 (193305'16603) modify 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.951337.0:3778 2014-08-11 18:42:47.205716 2014-08-13 17:38:02.784180 7fe9bae887a0 20 read_log 193305'16605 (0'0) promote 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by osd.289.0:81 2014-08-11 18:42:47.909133 2014-08-13 17:38:02.784192 7fe9bae887a0 20 read_log 193305'16606 (193305'16605) modify 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.951337.0:3968 2014-08-11 18:42:47.902320 2014-08-13 17:38:02.784206 7fe9bae887a0 20 read_log 193305'16607 (0'0) promote b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:83 2014-08-11 18:42:47.915014 2014-08-13 17:38:02.784218 7fe9bae887a0 20 read_log 193305'16608 (193305'16607) modify b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by client.951337.0:3969 2014-08-11 18:42:47.906916 2014-08-13 17:38:02.784230 7fe9bae887a0 20 read_log 193305'16609 (0'0) promote afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by osd.289.0:101 2014-08-11 18:42:52.696387 2014-08-13 17:38:02.784242 7fe9bae887a0 20 read_log 193305'16610 (193305'16609) modify afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.951337.0:5248 2014-08-11 18:42:52.687743 2014-08-13 17:38:02.784254 7fe9bae887a0 20 read_log 193305'16611 (0'0) promote f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by osd.289.0:109 2014-08-11 18:42:53.492965 2014-08-13 17:38:02.784266 7fe9bae887a0 20 read_log 193305'16612 (193305'16611) modify f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.951337.0:5468 2014-08-11 18:42:53.483833 2014-08-13 17:38:02.784278 7fe9bae887a0 20 read_log 193305'16613 (0'0) promote 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by osd.289.0:137 2014-08-11 18:42:59.378188 2014-08-13 17:38:02.784290 7fe9bae887a0 20 read_log 193305'16614 (193305'16613) modify 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.951337.0:7092 2014-08-11 18:42:59.370396 2014-08-13 17:38:02.784302 7fe9bae887a0 20 read_log 193305'16615 (0'0) promote 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by osd.289.0:167 2014-08-11 18:43:05.461047 2014-08-13 17:38:02.784314 7fe9bae887a0 20 read_log 193305'16616 (193305'16615) modify 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.951337.0:8701 2014-08-11 18:43:05.452908 2014-08-13 17:38:02.784326 7fe9bae887a0 20 read_log 193305'16617 (0'0) promote 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by osd.289.0:183 2014-08-11 18:43:09.187542 2014-08-13 17:38:02.784348 7fe9bae887a0 20 read_log 193305'16618 (193305'16617) modify 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.951337.0:9696 2014-08-11 18:43:09.180362 2014-08-13 17:38:02.784365 7fe9bae887a0 20 read_log 193305'16619 (0'0) promote e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by osd.289.0:199 2014-08-11 18:43:11.787802 2014-08-13 17:38:02.784377 7fe9bae887a0 20 read_log 193305'16620 (193305'16619) modify e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.951337.0:10404 2014-08-11 18:43:11.780832 2014-08-13 17:38:02.784389 7fe9bae887a0 20 read_log 193305'16621 (0'0) promote 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by osd.289.0:211 2014-08-11 18:43:13.903802 2014-08-13 17:38:02.784402 7fe9bae887a0 20 read_log 193305'16622 (193305'16621) modify 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.951337.0:10972 2014-08-11 18:43:13.895681 2014-08-13 17:38:02.784414 7fe9bae887a0 20 read_log 193305'16623 (0'0) promote cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by osd.289.0:369 2014-08-11 18:43:46.314563 2014-08-13 17:38:02.784426 7fe9bae887a0 20 read_log 193305'16624 (193305'16623) modify cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.951337.0:19919 2014-08-11 18:43:46.307102 2014-08-13 17:38:02.784438 7fe9bae887a0 20 read_log 193305'16625 (0'0) promote 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by osd.289.0:375 2014-08-11 18:43:47.650604 2014-08-13 17:38:02.784449 7fe9bae887a0 20 read_log 193305'16626 (193305'16625) modify 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.951337.0:20270 2014-08-11 18:43:47.643650 2014-08-13 17:38:02.784462 7fe9bae887a0 20 read_log 193305'16627 (0'0) promote c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:383 2014-08-11 18:43:49.388687 2014-08-13 17:38:02.784473 7fe9bae887a0 20 read_log 193305'16628 (193305'16627) modify c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by client.951337.0:20772 2014-08-11 18:43:49.379423 2014-08-13 17:38:02.784485 7fe9bae887a0 20 read_log 193305'16629 (0'0) promote 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by osd.289.0:397 2014-08-11 18:43:50.477192 2014-08-13 17:38:02.784497 7fe9bae887a0 20 read_log 193305'16630 (193305'16629) modify 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.951337.0:21069 2014-08-11 18:43:50.467674 2014-08-13 17:38:02.784509 7fe9bae887a0 20 read_log 193305'16631 (0'0) promote f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:407 2014-08-11 18:43:53.047596 2014-08-13 17:38:02.784521 7fe9bae887a0 20 read_log 193305'16632 (193305'16631) modify f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by client.951337.0:21794 2014-08-11 18:43:53.039061 2014-08-13 17:38:02.784533 7fe9bae887a0 20 read_log 193305'16633 (0'0) promote a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by osd.289.0:417 2014-08-11 18:43:55.378473 2014-08-13 17:38:02.784545 7fe9bae887a0 20 read_log 193305'16634 (193305'16633) modify a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.951337.0:22424 2014-08-11 18:43:55.367113 2014-08-13 17:38:02.784558 7fe9bae887a0 20 read_log 193305'16635 (0'0) promote b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by osd.289.0:423 2014-08-11 18:43:56.071866 2014-08-13 17:38:02.784570 7fe9bae887a0 20 read_log 193305'16636 (193305'16635) modify b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.951337.0:22617 2014-08-11 18:43:56.062406 2014-08-13 17:38:02.784582 7fe9bae887a0 20 read_log 193305'16637 (0'0) promote a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by osd.289.0:437 2014-08-11 18:43:59.929680 2014-08-13 17:38:02.784617 7fe9bae887a0 20 read_log 193305'16638 (193305'16637) modify a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.951337.0:23698 2014-08-11 18:43:59.922431 2014-08-13 17:38:02.784631 7fe9bae887a0 20 read_log 193305'16639 (0'0) promote 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by osd.289.0:439 2014-08-11 18:44:00.188408 2014-08-13 17:38:02.784643 7fe9bae887a0 20 read_log 193305'16640 (193305'16639) modify 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.951337.0:23771 2014-08-11 18:44:00.179739 2014-08-13 17:38:02.784655 7fe9bae887a0 20 read_log 193305'16641 (0'0) promote 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:539 2014-08-11 18:44:18.355045 2014-08-13 17:38:02.784667 7fe9bae887a0 20 read_log 193305'16642 (193305'16641) modify 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by client.951337.0:28741 2014-08-11 18:44:18.347341 2014-08-13 17:38:02.784679 7fe9bae887a0 20 read_log 193305'16643 (0'0) promote 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by osd.289.0:543 2014-08-11 18:44:20.225603 2014-08-13 17:38:02.784691 7fe9bae887a0 20 read_log 193305'16644 (193305'16643) modify 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.951337.0:29243 2014-08-11 18:44:20.217584 2014-08-13 17:38:02.784703 7fe9bae887a0 20 read_log 193305'16645 (0'0) promote 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by osd.289.0:549 2014-08-11 18:44:20.981093 2014-08-13 17:38:02.784715 7fe9bae887a0 20 read_log 193305'16646 (193305'16645) modify 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.951337.0:29448 2014-08-11 18:44:20.973834 2014-08-13 17:38:02.784727 7fe9bae887a0 20 read_log 193305'16647 (0'0) promote 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by osd.289.0:585 2014-08-11 18:44:28.251735 2014-08-13 17:38:02.784739 7fe9bae887a0 20 read_log 193305'16648 (193305'16647) modify 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.951337.0:31406 2014-08-11 18:44:28.244026 2014-08-13 17:38:02.784751 7fe9bae887a0 20 read_log 193305'16649 (193305'16602) modify 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by client.951337.0:35621 2014-08-11 18:44:35.679997 2014-08-13 17:38:02.784763 7fe9bae887a0 20 read_log 193305'16650 (193305'16604) modify 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.951337.0:36489 2014-08-11 18:44:36.260175 2014-08-13 17:38:02.784780 7fe9bae887a0 20 read_log 193305'16651 (193305'16606) modify 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.951337.0:36679 2014-08-11 18:44:36.470180 2014-08-13 17:38:02.784792 7fe9bae887a0 20 read_log 193305'16652 (193305'16608) modify b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by client.951337.0:36680 2014-08-11 18:44:36.470548 2014-08-13 17:38:02.784804 7fe9bae887a0 20 read_log 193305'16653 (193305'16610) modify afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.951337.0:37959 2014-08-11 18:44:37.141182 2014-08-13 17:38:02.784820 7fe9bae887a0 20 read_log 193305'16654 (193305'16612) modify f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.951337.0:38179 2014-08-11 18:44:37.274837 2014-08-13 17:38:02.784832 7fe9bae887a0 20 read_log 193305'16655 (193305'16614) modify 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.951337.0:39803 2014-08-11 18:44:38.189991 2014-08-13 17:38:02.784844 7fe9bae887a0 20 read_log 193305'16656 (193305'16616) modify 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.951337.0:41412 2014-08-11 18:44:38.745788 2014-08-13 17:38:02.784856 7fe9bae887a0 20 read_log 193305'16657 (193305'16618) modify 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.951337.0:42407 2014-08-11 18:44:39.214426 2014-08-13 17:38:02.784875 7fe9bae887a0 20 read_log 193305'16658 (193305'16620) modify e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.951337.0:43115 2014-08-11 18:44:39.487617 2014-08-13 17:38:02.784888 7fe9bae887a0 20 read_log 193305'16659 (193305'16622) modify 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.951337.0:43683 2014-08-11 18:44:39.665713 2014-08-13 17:38:02.784900 7fe9bae887a0 20 read_log 193305'16660 (193305'16624) modify cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.951337.0:52630 2014-08-11 18:44:42.288576 2014-08-13 17:38:02.784928 7fe9bae887a0 20 read_log 193305'16661 (193305'16626) modify 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.951337.0:52981 2014-08-11 18:44:42.396718 2014-08-13 17:38:02.784944 7fe9bae887a0 20 read_log 193305'16662 (193305'16628) modify c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by client.951337.0:53483 2014-08-11 18:44:42.544071 2014-08-13 17:38:02.784956 7fe9bae887a0 20 read_log 193305'16663 (193305'16630) modify 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.951337.0:53780 2014-08-11 18:44:42.634215 2014-08-13 17:38:02.784968 7fe9bae887a0 20 read_log 193305'16664 (193305'16632) modify f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by client.951337.0:54505 2014-08-11 18:44:42.858637 2014-08-13 17:38:02.784980 7fe9bae887a0 20 read_log 193305'16665 (193305'16634) modify a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.951337.0:55135 2014-08-11 18:44:43.035090 2014-08-13 17:38:02.784992 7fe9bae887a0 20 read_log 193305'16666 (193305'16636) modify b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.951337.0:55328 2014-08-11 18:44:43.094291 2014-08-13 17:38:02.785004 7fe9bae887a0 20 read_log 193305'16667 (193305'16638) modify a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.951337.0:56409 2014-08-11 18:44:43.410881 2014-08-13 17:38:02.785016 7fe9bae887a0 20 read_log 193305'16668 (193305'16640) modify 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.951337.0:56482 2014-08-11 18:44:43.431809 2014-08-13 17:38:02.785028 7fe9bae887a0 20 read_log 193305'16669 (193305'16642) modify 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by client.951337.0:61452 2014-08-11 18:44:44.925234 2014-08-13 17:38:02.785040 7fe9bae887a0 20 read_log 193305'16670 (193305'16644) modify 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.951337.0:61954 2014-08-11 18:44:45.081029 2014-08-13 17:38:02.785052 7fe9bae887a0 20 read_log 193305'16671 (193305'16646) modify 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.951337.0:62159 2014-08-11 18:44:45.142798 2014-08-13 17:38:02.785064 7fe9bae887a0 20 read_log 193305'16672 (193305'16648) modify 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.951337.0:64117 2014-08-11 18:44:45.708754 2014-08-13 17:38:02.785076 7fe9bae887a0 20 read_log 194209'16673 (192834'16169) clean 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:1303 2014-08-12 14:06:31.960131 2014-08-13 17:38:02.785088 7fe9bae887a0 20 read_log 194209'16674 (193305'16652) clean b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:1304 2014-08-12 14:06:31.960667 2014-08-13 17:38:02.785100 7fe9bae887a0 20 read_log 194209'16675 (192888'16422) clean 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:1305 2014-08-12 14:06:31.961366 2014-08-13 17:38:02.785112 7fe9bae887a0 20 read_log 194209'16676 (193305'16662) clean c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:1306 2014-08-12 14:06:31.961686 2014-08-13 17:38:02.785134 7fe9bae887a0 20 read_log 194209'16677 (193305'16669) clean 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:1307 2014-08-12 14:06:31.968627 2014-08-13 17:38:02.785147 7fe9bae887a0 20 read_log 194209'16678 (192834'15601) clean 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:1308 2014-08-12 14:06:31.971488 2014-08-13 17:38:02.785164 7fe9bae887a0 20 read_log 194209'16679 (192888'16382) clean a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:1309 2014-08-12 14:06:31.979168 2014-08-13 17:38:02.785180 7fe9bae887a0 20 read_log 194209'16680 (192834'15586) clean 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:1310 2014-08-12 14:06:31.979447 2014-08-13 17:38:02.785192 7fe9bae887a0 20 read_log 194209'16681 (192834'16156) clean 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:1311 2014-08-12 14:06:31.979724 2014-08-13 17:38:02.785204 7fe9bae887a0 20 read_log 194209'16682 (192834'16080) clean cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:1312 2014-08-12 14:06:31.980682 2014-08-13 17:38:02.785216 7fe9bae887a0 20 read_log 194209'16683 (192888'16430) clean 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:1313 2014-08-12 14:06:31.987486 2014-08-13 17:38:02.785229 7fe9bae887a0 20 read_log 194209'16684 (192888'16425) clean 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:1314 2014-08-12 14:06:31.990235 2014-08-13 17:38:02.785241 7fe9bae887a0 20 read_log 194209'16685 (192888'16553) clean 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by osd.289.0:1315 2014-08-12 14:06:31.993074 2014-08-13 17:38:02.785253 7fe9bae887a0 20 read_log 194209'16686 (192888'16417) clean 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:1316 2014-08-12 14:06:31.995865 2014-08-13 17:38:02.785265 7fe9bae887a0 20 read_log 194209'16687 (192888'16548) clean e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by osd.289.0:1317 2014-08-12 14:06:31.998749 2014-08-13 17:38:02.785278 7fe9bae887a0 20 read_log 194209'16688 (192834'16154) clean 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:1318 2014-08-12 14:06:32.001584 2014-08-13 17:38:02.785290 7fe9bae887a0 20 read_log 194209'16689 (192888'16571) clean 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by osd.289.0:1319 2014-08-12 14:06:32.004333 2014-08-13 17:38:02.785302 7fe9bae887a0 20 read_log 194209'16690 (192888'16568) clean 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by osd.289.0:1320 2014-08-12 14:06:32.009637 2014-08-13 17:38:02.785314 7fe9bae887a0 20 read_log 194209'16691 (193305'16664) clean f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:1321 2014-08-12 14:06:32.009905 2014-08-13 17:38:02.785326 7fe9bae887a0 20 read_log 194209'16692 (192888'16375) clean 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:1322 2014-08-12 14:06:32.012773 2014-08-13 17:38:02.785338 7fe9bae887a0 20 read_log 194209'16693 (193305'16649) clean 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:1323 2014-08-12 14:06:32.015825 2014-08-13 17:38:02.785350 7fe9bae887a0 20 read_log 194209'16694 (192888'16556) clean edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by osd.289.0:1324 2014-08-12 14:06:32.020858 2014-08-13 17:38:02.785362 7fe9bae887a0 20 read_log 194209'16695 (192888'16416) clean 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:1325 2014-08-12 14:06:32.021145 2014-08-13 17:38:02.785374 7fe9bae887a0 20 read_log 194209'16696 (192834'15610) clean 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:1326 2014-08-12 14:06:32.024132 2014-08-13 17:38:02.785393 7fe9bae887a0 20 read_log 194209'16697 (194209'16673) delete 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:1327 2014-08-12 14:06:32.054808 2014-08-13 17:38:02.785406 7fe9bae887a0 20 read_log 194209'16698 (194209'16674) delete b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:1328 2014-08-12 14:06:32.055191 2014-08-13 17:38:02.785418 7fe9bae887a0 20 read_log 194209'16699 (194209'16675) delete 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:1329 2014-08-12 14:06:32.057959 2014-08-13 17:38:02.785430 7fe9bae887a0 20 read_log 194209'16700 (194209'16676) delete c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:1330 2014-08-12 14:06:32.058251 2014-08-13 17:38:02.785443 7fe9bae887a0 20 read_log 194209'16701 (194209'16677) delete 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:1331 2014-08-12 14:06:32.060942 2014-08-13 17:38:02.785455 7fe9bae887a0 20 read_log 194209'16702 (194209'16678) delete 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:1332 2014-08-12 14:06:32.061233 2014-08-13 17:38:02.785467 7fe9bae887a0 20 read_log 194209'16703 (194209'16679) delete a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:1333 2014-08-12 14:06:32.064185 2014-08-13 17:38:02.785479 7fe9bae887a0 20 read_log 194209'16704 (194209'16680) delete 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:1334 2014-08-12 14:06:32.064546 2014-08-13 17:38:02.785491 7fe9bae887a0 20 read_log 194209'16705 (194209'16681) delete 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:1335 2014-08-12 14:06:32.067383 2014-08-13 17:38:02.785503 7fe9bae887a0 20 read_log 194209'16706 (194209'16682) delete cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:1336 2014-08-12 14:06:32.067725 2014-08-13 17:38:02.785515 7fe9bae887a0 20 read_log 194209'16707 (194209'16683) delete 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:1337 2014-08-12 14:06:32.070758 2014-08-13 17:38:02.785527 7fe9bae887a0 20 read_log 194209'16708 (194209'16684) delete 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:1338 2014-08-12 14:06:32.071088 2014-08-13 17:38:02.785539 7fe9bae887a0 20 read_log 194209'16709 (194209'16686) delete 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:1339 2014-08-12 14:06:32.073637 2014-08-13 17:38:02.785551 7fe9bae887a0 20 read_log 194209'16710 (194209'16688) delete 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:1340 2014-08-12 14:06:32.074264 2014-08-13 17:38:02.785562 7fe9bae887a0 20 read_log 194209'16711 (194209'16691) delete f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:1341 2014-08-12 14:06:32.075046 2014-08-13 17:38:02.785574 7fe9bae887a0 20 read_log 194209'16712 (194209'16692) delete 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:1342 2014-08-12 14:06:32.077637 2014-08-13 17:38:02.785586 7fe9bae887a0 20 read_log 194209'16713 (194209'16693) delete 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:1343 2014-08-12 14:06:32.077960 2014-08-13 17:38:02.785598 7fe9bae887a0 20 read_log 194209'16714 (194209'16695) delete 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:1344 2014-08-12 14:06:32.078751 2014-08-13 17:38:02.785610 7fe9bae887a0 20 read_log 194209'16715 (194209'16696) delete 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:1345 2014-08-12 14:06:32.081300 2014-08-13 17:38:02.785622 7fe9bae887a0 20 read_log 194209'16716 (193305'16670) clean 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by osd.289.0:1346 2014-08-12 14:06:32.128800 2014-08-13 17:38:02.785641 7fe9bae887a0 20 read_log 194215'16717 (194215'16717) modify 5e/hit_set_24.5e_archive_2014-08-12 15:32:25.855993_2014-08-12 15:37:32.210894/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:37:32.211130 2014-08-13 17:38:02.785654 7fe9bae887a0 20 read_log 194215'16718 (193305'16599) delete 5e/hit_set_24.5e_archive_2014-08-11 18:37:04.920337_2014-08-11 18:42:44.060681/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:37:32.211130 2014-08-13 17:38:02.785666 7fe9bae887a0 20 read_log 194215'16719 (192888'16558) clean 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by osd.289.0:1740 2014-08-12 15:37:32.215651 2014-08-13 17:38:02.785679 7fe9bae887a0 20 read_log 194215'16720 (194215'16719) delete 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.2174595.0:18609 0.000000 2014-08-13 17:38:02.785690 7fe9bae887a0 20 read_log 194215'16721 (193305'16658) clean e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by osd.289.0:1742 2014-08-12 15:37:32.223366 2014-08-13 17:38:02.785702 7fe9bae887a0 20 read_log 194215'16722 (194215'16721) delete e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.2174595.0:18611 0.000000 2014-08-13 17:38:02.785712 7fe9bae887a0 20 read_log 194215'16723 (192888'16566) clean 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by osd.289.0:1744 2014-08-12 15:37:32.230482 2014-08-13 17:38:02.785724 7fe9bae887a0 20 read_log 194215'16724 (194215'16723) delete 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.2174595.0:18613 0.000000 2014-08-13 17:38:02.785735 7fe9bae887a0 20 read_log 194215'16725 (192834'15643) clean 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by osd.289.0:1746 2014-08-12 15:37:32.237249 2014-08-13 17:38:02.785747 7fe9bae887a0 20 read_log 194215'16726 (194215'16725) delete 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.2174595.0:18615 0.000000 2014-08-13 17:38:02.785757 7fe9bae887a0 20 read_log 194215'16727 (192888'16580) clean 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by osd.289.0:1748 2014-08-12 15:37:32.244179 2014-08-13 17:38:02.785769 7fe9bae887a0 20 read_log 194215'16728 (194215'16727) delete 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.2174595.0:18617 0.000000 2014-08-13 17:38:02.785780 7fe9bae887a0 20 read_log 194215'16729 (192834'16151) clean da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by osd.289.0:1750 2014-08-12 15:37:32.251086 2014-08-13 17:38:02.785792 7fe9bae887a0 20 read_log 194215'16730 (194215'16729) delete da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.2174595.0:18619 0.000000 2014-08-13 17:38:02.785803 7fe9bae887a0 20 read_log 194215'16731 (193305'16666) clean b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by osd.289.0:1752 2014-08-12 15:37:32.258423 2014-08-13 17:38:02.785815 7fe9bae887a0 20 read_log 194215'16732 (194215'16731) delete b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.2174595.0:18621 0.000000 2014-08-13 17:38:02.785825 7fe9bae887a0 20 read_log 194215'16733 (192834'16062) clean 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by osd.289.0:1754 2014-08-12 15:37:32.265042 2014-08-13 17:38:02.785837 7fe9bae887a0 20 read_log 194215'16734 (194215'16733) delete 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.2174595.0:18623 0.000000 2014-08-13 17:38:02.785848 7fe9bae887a0 20 read_log 194215'16735 (193305'16667) clean a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by osd.289.0:1756 2014-08-12 15:37:32.271612 2014-08-13 17:38:02.785869 7fe9bae887a0 20 read_log 194215'16736 (194215'16735) delete a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.2174595.0:18625 0.000000 2014-08-13 17:38:02.785880 7fe9bae887a0 20 read_log 194215'16737 (192888'16564) clean 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by osd.289.0:1758 2014-08-12 15:37:32.278906 2014-08-13 17:38:02.785893 7fe9bae887a0 20 read_log 194215'16738 (194215'16737) delete 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.2174595.0:18627 0.000000 2014-08-13 17:38:02.785904 7fe9bae887a0 20 read_log 194215'16739 (193305'16672) clean 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by osd.289.0:1760 2014-08-12 15:37:32.286015 2014-08-13 17:38:02.785916 7fe9bae887a0 20 read_log 194215'16740 (194215'16739) delete 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.2174595.0:18629 0.000000 2014-08-13 17:38:02.785926 7fe9bae887a0 20 read_log 194215'16741 (192888'16389) clean 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by osd.289.0:1762 2014-08-12 15:37:32.293269 2014-08-13 17:38:02.785939 7fe9bae887a0 20 read_log 194215'16742 (194215'16741) delete 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.2174595.0:18631 0.000000 2014-08-13 17:38:02.785949 7fe9bae887a0 20 read_log 194215'16743 (192888'16401) clean 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by osd.289.0:1764 2014-08-12 15:37:32.301042 2014-08-13 17:38:02.785961 7fe9bae887a0 20 read_log 194215'16744 (194215'16743) delete 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.2174595.0:18633 0.000000 2014-08-13 17:38:02.785972 7fe9bae887a0 20 read_log 194215'16745 (192888'16388) clean be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by osd.289.0:1766 2014-08-12 15:37:32.308028 2014-08-13 17:38:02.785984 7fe9bae887a0 20 read_log 194215'16746 (194215'16745) delete be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.2174595.0:18635 0.000000 2014-08-13 17:38:02.785994 7fe9bae887a0 20 read_log 194215'16747 (192834'15585) clean 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by osd.289.0:1768 2014-08-12 15:37:32.314277 2014-08-13 17:38:02.786007 7fe9bae887a0 20 read_log 194215'16748 (194215'16747) delete 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.2174595.0:18637 0.000000 2014-08-13 17:38:02.786017 7fe9bae887a0 20 read_log 194215'16749 (192834'15704) clean 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by osd.289.0:1770 2014-08-12 15:37:32.320749 2014-08-13 17:38:02.786029 7fe9bae887a0 20 read_log 194215'16750 (194215'16749) delete 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.2174595.0:18639 0.000000 2014-08-13 17:38:02.786040 7fe9bae887a0 20 read_log 194215'16751 (192834'15611) clean df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by osd.289.0:1772 2014-08-12 15:37:32.328046 2014-08-13 17:38:02.786052 7fe9bae887a0 20 read_log 194215'16752 (194215'16751) delete df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.2174595.0:18641 0.000000 2014-08-13 17:38:02.786063 7fe9bae887a0 20 read_log 194215'16753 (192834'15705) clean 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by osd.289.0:1774 2014-08-12 15:37:32.335680 2014-08-13 17:38:02.786093 7fe9bae887a0 20 read_log 194215'16754 (194215'16753) delete 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.2174595.0:18643 0.000000 2014-08-13 17:38:02.786104 7fe9bae887a0 20 read_log 194215'16755 (192888'16428) clean 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by osd.289.0:1776 2014-08-12 15:37:32.342957 2014-08-13 17:38:02.786124 7fe9bae887a0 20 read_log 194215'16756 (194215'16755) delete 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.2174595.0:18645 0.000000 2014-08-13 17:38:02.786135 7fe9bae887a0 20 read_log 194215'16757 (192888'16581) clean 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by osd.289.0:1778 2014-08-12 15:37:32.350107 2014-08-13 17:38:02.786147 7fe9bae887a0 20 read_log 194215'16758 (194215'16757) delete 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.2174595.0:18647 0.000000 2014-08-13 17:38:02.786164 7fe9bae887a0 20 read_log 194215'16759 (192834'15656) clean 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by osd.289.0:1780 2014-08-12 15:37:32.357296 2014-08-13 17:38:02.786176 7fe9bae887a0 20 read_log 194215'16760 (194215'16759) delete 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.2174595.0:18649 0.000000 2014-08-13 17:38:02.786187 7fe9bae887a0 20 read_log 194215'16761 (192888'16434) clean 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by osd.289.0:1782 2014-08-12 15:37:32.364598 2014-08-13 17:38:02.786199 7fe9bae887a0 20 read_log 194215'16762 (194215'16761) delete 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.2174595.0:18651 0.000000 2014-08-13 17:38:02.786210 7fe9bae887a0 20 read_log 194215'16763 (193305'16661) clean 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by osd.289.0:1784 2014-08-12 15:37:32.371499 2014-08-13 17:38:02.786222 7fe9bae887a0 20 read_log 194215'16764 (194215'16763) delete 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.2174595.0:18653 0.000000 2014-08-13 17:38:02.786233 7fe9bae887a0 20 read_log 194215'16765 (192888'16418) clean abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by osd.289.0:1786 2014-08-12 15:37:32.378481 2014-08-13 17:38:02.786245 7fe9bae887a0 20 read_log 194215'16766 (194215'16765) delete abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.2174595.0:18655 0.000000 2014-08-13 17:38:02.786255 7fe9bae887a0 20 read_log 194215'16767 (192888'16371) clean 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by osd.289.0:1788 2014-08-12 15:37:32.385711 2014-08-13 17:38:02.786267 7fe9bae887a0 20 read_log 194215'16768 (194215'16767) delete 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.2174595.0:18657 0.000000 2014-08-13 17:38:02.786278 7fe9bae887a0 20 read_log 194215'16769 (192888'16557) clean 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by osd.289.0:1790 2014-08-12 15:37:32.393299 2014-08-13 17:38:02.786290 7fe9bae887a0 20 read_log 194215'16770 (194215'16769) delete 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.2174595.0:18659 0.000000 2014-08-13 17:38:02.786302 7fe9bae887a0 20 read_log 194215'16771 (192834'15645) clean d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by osd.289.0:1792 2014-08-12 15:37:32.399910 2014-08-13 17:38:02.786315 7fe9bae887a0 20 read_log 194215'16772 (194215'16771) delete d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.2174595.0:18661 0.000000 2014-08-13 17:38:02.786326 7fe9bae887a0 20 read_log 194215'16773 (192888'16547) clean ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by osd.289.0:1794 2014-08-12 15:37:32.406967 2014-08-13 17:38:02.786340 7fe9bae887a0 20 read_log 194215'16774 (194215'16773) delete ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.2174595.0:18663 0.000000 2014-08-13 17:38:02.786351 7fe9bae887a0 20 read_log 194215'16775 (192834'16105) clean fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by osd.289.0:1796 2014-08-12 15:37:32.413663 2014-08-13 17:38:02.786389 7fe9bae887a0 20 read_log 194215'16776 (194215'16775) delete fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.2174595.0:18665 0.000000 2014-08-13 17:38:02.786401 7fe9bae887a0 20 read_log 194215'16777 (192834'16144) clean 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by osd.289.0:1798 2014-08-12 15:37:32.421093 2014-08-13 17:38:02.786414 7fe9bae887a0 20 read_log 194215'16778 (194215'16777) delete 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.2174595.0:18667 0.000000 2014-08-13 17:38:02.786425 7fe9bae887a0 20 read_log 194215'16779 (192888'16402) clean 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by osd.289.0:1800 2014-08-12 15:37:32.428437 2014-08-13 17:38:02.786437 7fe9bae887a0 20 read_log 194215'16780 (194215'16779) delete 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.2174595.0:18669 0.000000 2014-08-13 17:38:02.786448 7fe9bae887a0 20 read_log 194215'16781 (192888'16567) clean 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by osd.289.0:1802 2014-08-12 15:37:32.435394 2014-08-13 17:38:02.786460 7fe9bae887a0 20 read_log 194215'16782 (194215'16781) delete 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.2174595.0:18671 0.000000 2014-08-13 17:38:02.786471 7fe9bae887a0 20 read_log 194215'16783 (192834'16223) clean 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by osd.289.0:1804 2014-08-12 15:37:32.442313 2014-08-13 17:38:02.786483 7fe9bae887a0 20 read_log 194215'16784 (194215'16783) delete 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.2174595.0:18673 0.000000 2014-08-13 17:38:02.786493 7fe9bae887a0 20 read_log 194215'16785 (193305'16671) clean 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by osd.289.0:1806 2014-08-12 15:37:32.449360 2014-08-13 17:38:02.786505 7fe9bae887a0 20 read_log 194215'16786 (194215'16785) delete 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.2174595.0:18675 0.000000 2014-08-13 17:38:02.786516 7fe9bae887a0 20 read_log 194215'16787 (192834'16175) clean 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by osd.289.0:1808 2014-08-12 15:37:32.456906 2014-08-13 17:38:02.786528 7fe9bae887a0 20 read_log 194215'16788 (194215'16787) delete 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.2174595.0:18677 0.000000 2014-08-13 17:38:02.786539 7fe9bae887a0 20 read_log 194215'16789 (193305'16651) clean 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by osd.289.0:1810 2014-08-12 15:37:32.464095 2014-08-13 17:38:02.786551 7fe9bae887a0 20 read_log 194215'16790 (194215'16789) delete 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.2174595.0:18679 0.000000 2014-08-13 17:38:02.786561 7fe9bae887a0 20 read_log 194215'16791 (192834'16171) clean d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by osd.289.0:1812 2014-08-12 15:37:32.471059 2014-08-13 17:38:02.786573 7fe9bae887a0 20 read_log 194215'16792 (194215'16791) delete d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.2174595.0:18681 0.000000 2014-08-13 17:38:02.786584 7fe9bae887a0 20 read_log 194215'16793 (192888'16579) clean 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by osd.289.0:1814 2014-08-12 15:37:32.478745 2014-08-13 17:38:02.786596 7fe9bae887a0 20 read_log 194215'16794 (194215'16793) delete 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.2174595.0:18683 0.000000 2014-08-13 17:38:02.786607 7fe9bae887a0 20 read_log 194215'16795 (192888'16405) clean 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by osd.289.0:1816 2014-08-12 15:37:32.485320 2014-08-13 17:38:02.786627 7fe9bae887a0 20 read_log 194215'16796 (194215'16795) delete 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.2174595.0:18685 0.000000 2014-08-13 17:38:02.786639 7fe9bae887a0 20 read_log 194215'16797 (192834'16132) clean 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by osd.289.0:1818 2014-08-12 15:37:32.492177 2014-08-13 17:38:02.786651 7fe9bae887a0 20 read_log 194215'16798 (194215'16797) delete 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.2174595.0:18687 0.000000 2014-08-13 17:38:02.786662 7fe9bae887a0 20 read_log 194215'16799 (192834'15646) clean 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by osd.289.0:1820 2014-08-12 15:37:32.499073 2014-08-13 17:38:02.786674 7fe9bae887a0 20 read_log 194215'16800 (194215'16799) delete 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.2174595.0:18689 0.000000 2014-08-13 17:38:02.786685 7fe9bae887a0 20 read_log 194215'16801 (192888'16393) clean 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by osd.289.0:1822 2014-08-12 15:37:32.506632 2014-08-13 17:38:02.786697 7fe9bae887a0 20 read_log 194215'16802 (194215'16801) delete 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.2174595.0:18691 0.000000 2014-08-13 17:38:02.786707 7fe9bae887a0 20 read_log 194215'16803 (192888'16396) clean 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by osd.289.0:1824 2014-08-12 15:37:32.513793 2014-08-13 17:38:02.786719 7fe9bae887a0 20 read_log 194215'16804 (194215'16803) delete 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.2174595.0:18693 0.000000 2014-08-13 17:38:02.786730 7fe9bae887a0 20 read_log 194215'16805 (193305'16653) clean afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by osd.289.0:1826 2014-08-12 15:37:32.521767 2014-08-13 17:38:02.786742 7fe9bae887a0 20 read_log 194215'16806 (194215'16805) delete afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.2174595.0:18695 0.000000 2014-08-13 17:38:02.786753 7fe9bae887a0 20 read_log 194215'16807 (193305'16654) clean f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by osd.289.0:1828 2014-08-12 15:37:32.529091 2014-08-13 17:38:02.786765 7fe9bae887a0 20 read_log 194215'16808 (194215'16807) delete f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.2174595.0:18697 0.000000 2014-08-13 17:38:02.786775 7fe9bae887a0 20 read_log 194215'16809 (192888'16561) clean 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by osd.289.0:1830 2014-08-12 15:37:32.535902 2014-08-13 17:38:02.786787 7fe9bae887a0 20 read_log 194215'16810 (194215'16809) delete 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.2174595.0:18699 0.000000 2014-08-13 17:38:02.786798 7fe9bae887a0 20 read_log 194215'16811 (192834'16163) clean cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by osd.289.0:1832 2014-08-12 15:37:32.542802 2014-08-13 17:38:02.786810 7fe9bae887a0 20 read_log 194215'16812 (194215'16811) delete cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.2174595.0:18701 0.000000 2014-08-13 17:38:02.786820 7fe9bae887a0 20 read_log 194215'16813 (192834'16203) clean 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by osd.289.0:1834 2014-08-12 15:37:32.550050 2014-08-13 17:38:02.786836 7fe9bae887a0 20 read_log 194215'16814 (194215'16813) delete 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.2174595.0:18703 0.000000 2014-08-13 17:38:02.786846 7fe9bae887a0 20 read_log 194215'16815 (193305'16659) clean 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by osd.289.0:1836 2014-08-12 15:37:32.556668 2014-08-13 17:38:02.786866 7fe9bae887a0 20 read_log 194215'16816 (194215'16815) delete 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.2174595.0:18705 0.000000 2014-08-13 17:38:02.786877 7fe9bae887a0 20 read_log 194215'16817 (192888'16593) clean 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by osd.289.0:1838 2014-08-12 15:37:32.563447 2014-08-13 17:38:02.786889 7fe9bae887a0 20 read_log 194215'16818 (194215'16817) delete 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.2174595.0:18707 0.000000 2014-08-13 17:38:02.786900 7fe9bae887a0 20 read_log 194215'16819 (192888'16576) clean 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by osd.289.0:1840 2014-08-12 15:37:32.570745 2014-08-13 17:38:02.786912 7fe9bae887a0 20 read_log 194215'16820 (194215'16819) delete 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.2174595.0:18709 0.000000 2014-08-13 17:38:02.786923 7fe9bae887a0 20 read_log 194215'16821 (192888'16370) clean 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by osd.289.0:1842 2014-08-12 15:37:32.578296 2014-08-13 17:38:02.786935 7fe9bae887a0 20 read_log 194215'16822 (194215'16821) delete 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.2174595.0:18711 0.000000 2014-08-13 17:38:02.786945 7fe9bae887a0 20 read_log 194215'16823 (193305'16655) clean 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by osd.289.0:1844 2014-08-12 15:37:32.585499 2014-08-13 17:38:02.786957 7fe9bae887a0 20 read_log 194215'16824 (194215'16823) delete 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.2174595.0:18713 0.000000 2014-08-13 17:38:02.786968 7fe9bae887a0 20 read_log 194215'16825 (192888'16431) clean 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by osd.289.0:1846 2014-08-12 15:37:32.592882 2014-08-13 17:38:02.786980 7fe9bae887a0 20 read_log 194215'16826 (194215'16825) delete 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.2174595.0:18715 0.000000 2014-08-13 17:38:02.786991 7fe9bae887a0 20 read_log 194215'16827 (194209'16685) delete 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.2174595.0:18717 0.000000 2014-08-13 17:38:02.787001 7fe9bae887a0 20 read_log 194215'16828 (194209'16687) delete e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.2174595.0:18719 0.000000 2014-08-13 17:38:02.787012 7fe9bae887a0 20 read_log 194215'16829 (194209'16689) delete 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.2174595.0:18721 0.000000 2014-08-13 17:38:02.787022 7fe9bae887a0 20 read_log 194215'16830 (194209'16690) delete 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.2174595.0:18723 0.000000 2014-08-13 17:38:02.787033 7fe9bae887a0 20 read_log 194215'16831 (194209'16694) delete edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.2174595.0:18725 0.000000 2014-08-13 17:38:02.787043 7fe9bae887a0 20 read_log 194215'16832 (194209'16716) delete 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.2174595.0:18727 0.000000 2014-08-13 17:38:02.787054 7fe9bae887a0 20 read_log 194215'16833 (192888'16406) clean 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by osd.289.0:1854 2014-08-12 15:37:32.632279 2014-08-13 17:38:02.787066 7fe9bae887a0 20 read_log 194215'16834 (194215'16833) delete 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.2174595.0:18729 0.000000 2014-08-13 17:38:02.787077 7fe9bae887a0 20 read_log 194215'16835 (192888'16423) clean dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by osd.289.0:1856 2014-08-12 15:37:32.639631 2014-08-13 17:38:02.787095 7fe9bae887a0 20 read_log 194215'16836 (194215'16835) delete dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.2174595.0:18731 0.000000 2014-08-13 17:38:02.787106 7fe9bae887a0 20 read_log 194215'16837 (192834'15670) clean e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by osd.289.0:1858 2014-08-12 15:37:32.646420 2014-08-13 17:38:02.787118 7fe9bae887a0 20 read_log 194215'16838 (194215'16837) delete e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.2174595.0:18733 0.000000 2014-08-13 17:38:02.787129 7fe9bae887a0 20 read_log 194215'16839 (192834'15648) clean 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by osd.289.0:1860 2014-08-12 15:37:32.653330 2014-08-13 17:38:02.787141 7fe9bae887a0 20 read_log 194215'16840 (194215'16839) delete 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.2174595.0:18735 0.000000 2014-08-13 17:38:02.787152 7fe9bae887a0 20 read_log 194215'16841 (192888'16415) clean 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by osd.289.0:1862 2014-08-12 15:37:32.660429 2014-08-13 17:38:02.787170 7fe9bae887a0 20 read_log 194215'16842 (194215'16841) delete 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.2174595.0:18737 0.000000 2014-08-13 17:38:02.787181 7fe9bae887a0 20 read_log 194215'16843 (193305'16668) clean 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by osd.289.0:1864 2014-08-12 15:37:32.667744 2014-08-13 17:38:02.787193 7fe9bae887a0 20 read_log 194215'16844 (194215'16843) delete 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.2174595.0:18739 0.000000 2014-08-13 17:38:02.787203 7fe9bae887a0 20 read_log 194215'16845 (192888'16559) clean d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by osd.289.0:1866 2014-08-12 15:37:32.674854 2014-08-13 17:38:02.787215 7fe9bae887a0 20 read_log 194215'16846 (194215'16845) delete d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.2174595.0:18741 0.000000 2014-08-13 17:38:02.787242 7fe9bae887a0 20 read_log 194215'16847 (192888'16584) clean ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by osd.289.0:1868 2014-08-12 15:37:32.681667 2014-08-13 17:38:02.787254 7fe9bae887a0 20 read_log 194215'16848 (194215'16847) delete ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.2174595.0:18743 0.000000 2014-08-13 17:38:02.787265 7fe9bae887a0 20 read_log 194215'16849 (193305'16665) clean a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by osd.289.0:1870 2014-08-12 15:37:32.688391 2014-08-13 17:38:02.787283 7fe9bae887a0 20 read_log 194215'16850 (194215'16849) delete a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.2174595.0:18745 0.000000 2014-08-13 17:38:02.787294 7fe9bae887a0 20 read_log 194215'16851 (192888'16395) clean e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by osd.289.0:1872 2014-08-12 15:37:32.695606 2014-08-13 17:38:02.787306 7fe9bae887a0 20 read_log 194215'16852 (194215'16851) delete e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.2174595.0:18747 0.000000 2014-08-13 17:38:02.787316 7fe9bae887a0 20 read_log 194215'16853 (192834'15642) clean db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by osd.289.0:1874 2014-08-12 15:37:32.702979 2014-08-13 17:38:02.787328 7fe9bae887a0 20 read_log 194215'16854 (194215'16853) delete db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.2174595.0:18749 0.000000 2014-08-13 17:38:02.787339 7fe9bae887a0 20 read_log 194215'16855 (192834'15694) clean 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by osd.289.0:1876 2014-08-12 15:37:32.709891 2014-08-13 17:38:02.787361 7fe9bae887a0 20 read_log 194215'16856 (194215'16855) delete 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.2174595.0:18751 0.000000 2014-08-13 17:38:02.787389 7fe9bae887a0 20 read_log 194215'16857 (192834'15590) clean e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by osd.289.0:1878 2014-08-12 15:37:32.717047 2014-08-13 17:38:02.787403 7fe9bae887a0 20 read_log 194215'16858 (194215'16857) delete e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.2174595.0:18753 0.000000 2014-08-13 17:38:02.787414 7fe9bae887a0 20 read_log 194215'16859 (193305'16660) clean cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by osd.289.0:1880 2014-08-12 15:37:32.724650 2014-08-13 17:38:02.787427 7fe9bae887a0 20 read_log 194215'16860 (194215'16859) delete cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.2174595.0:18755 0.000000 2014-08-13 17:38:02.787437 7fe9bae887a0 20 read_log 194215'16861 (192834'16111) clean bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by osd.289.0:1882 2014-08-12 15:37:32.732206 2014-08-13 17:38:02.787450 7fe9bae887a0 20 read_log 194215'16862 (194215'16861) delete bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.2174595.0:18757 0.000000 2014-08-13 17:38:02.787460 7fe9bae887a0 20 read_log 194215'16863 (192834'16070) clean 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by osd.289.0:1884 2014-08-12 15:37:32.739303 2014-08-13 17:38:02.787472 7fe9bae887a0 20 read_log 194215'16864 (194215'16863) delete 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.2174595.0:18759 0.000000 2014-08-13 17:38:02.787483 7fe9bae887a0 20 read_log 194215'16865 (192834'16214) clean 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by osd.289.0:1886 2014-08-12 15:37:32.746036 2014-08-13 17:38:02.787495 7fe9bae887a0 20 read_log 194215'16866 (194215'16865) delete 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.2174595.0:18761 0.000000 2014-08-13 17:38:02.787506 7fe9bae887a0 20 read_log 194215'16867 (193305'16650) clean 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by osd.289.0:1888 2014-08-12 15:37:32.752968 2014-08-13 17:38:02.787518 7fe9bae887a0 20 read_log 194215'16868 (194215'16867) delete 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.2174595.0:18763 0.000000 2014-08-13 17:38:02.787529 7fe9bae887a0 20 read_log 194215'16869 (192888'16404) clean 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by osd.289.0:1890 2014-08-12 15:37:32.759680 2014-08-13 17:38:02.787541 7fe9bae887a0 20 read_log 194215'16870 (194215'16869) delete 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.2174595.0:18765 0.000000 2014-08-13 17:38:02.787551 7fe9bae887a0 20 read_log 194215'16871 (192834'15621) clean 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by osd.289.0:1892 2014-08-12 15:37:32.766788 2014-08-13 17:38:02.787564 7fe9bae887a0 20 read_log 194215'16872 (194215'16871) delete 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.2174595.0:18767 0.000000 2014-08-13 17:38:02.787574 7fe9bae887a0 20 read_log 194215'16873 (192888'16397) clean d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by osd.289.0:1894 2014-08-12 15:37:32.773234 2014-08-13 17:38:02.787586 7fe9bae887a0 20 read_log 194215'16874 (194215'16873) delete d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.2174595.0:18769 0.000000 2014-08-13 17:38:02.787597 7fe9bae887a0 20 read_log 194215'16875 (192888'16399) clean 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by osd.289.0:1896 2014-08-12 15:37:32.779912 2014-08-13 17:38:02.787616 7fe9bae887a0 20 read_log 194215'16876 (194215'16875) delete 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.2174595.0:18771 0.000000 2014-08-13 17:38:02.787627 7fe9bae887a0 20 read_log 194215'16877 (192888'16369) clean d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by osd.289.0:1898 2014-08-12 15:37:32.786045 2014-08-13 17:38:02.787639 7fe9bae887a0 20 read_log 194215'16878 (194215'16877) delete d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.2174595.0:18773 0.000000 2014-08-13 17:38:02.787650 7fe9bae887a0 20 read_log 194215'16879 (192888'16387) clean 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by osd.289.0:1900 2014-08-12 15:37:32.793007 2014-08-13 17:38:02.787662 7fe9bae887a0 20 read_log 194215'16880 (194215'16879) delete 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.2174595.0:18775 0.000000 2014-08-13 17:38:02.787673 7fe9bae887a0 20 read_log 194215'16881 (192888'16411) clean 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by osd.289.0:1902 2014-08-12 15:37:32.802197 2014-08-13 17:38:02.787685 7fe9bae887a0 20 read_log 194215'16882 (194215'16881) delete 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.2174595.0:18777 0.000000 2014-08-13 17:38:02.787695 7fe9bae887a0 20 read_log 194215'16883 (192888'16420) clean bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by osd.289.0:1904 2014-08-12 15:37:32.809610 2014-08-13 17:38:02.787707 7fe9bae887a0 20 read_log 194215'16884 (194215'16883) delete bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.2174595.0:18779 0.000000 2014-08-13 17:38:02.787718 7fe9bae887a0 20 read_log 194215'16885 (192888'16410) clean ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by osd.289.0:1906 2014-08-12 15:37:32.817103 2014-08-13 17:38:02.787730 7fe9bae887a0 20 read_log 194215'16886 (194215'16885) delete ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.2174595.0:18781 0.000000 2014-08-13 17:38:02.787741 7fe9bae887a0 20 read_log 194215'16887 (193305'16656) clean 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by osd.289.0:1908 2014-08-12 15:37:32.824253 2014-08-13 17:38:02.787753 7fe9bae887a0 20 read_log 194215'16888 (194215'16887) delete 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.2174595.0:18783 0.000000 2014-08-13 17:38:02.787764 7fe9bae887a0 20 read_log 194215'16889 (193305'16657) clean 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by osd.289.0:1910 2014-08-12 15:37:32.830765 2014-08-13 17:38:02.787776 7fe9bae887a0 20 read_log 194215'16890 (194215'16889) delete 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.2174595.0:18785 0.000000 2014-08-13 17:38:02.787786 7fe9bae887a0 20 read_log 194215'16891 (192834'15720) clean cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by osd.289.0:1912 2014-08-12 15:37:32.837545 2014-08-13 17:38:02.787798 7fe9bae887a0 20 read_log 194215'16892 (194215'16891) delete cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.2174595.0:18787 0.000000 2014-08-13 17:38:02.787809 7fe9bae887a0 20 read_log 194215'16893 (193305'16663) clean 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by osd.289.0:1914 2014-08-12 15:37:32.844717 2014-08-13 17:38:02.787821 7fe9bae887a0 20 read_log 194215'16894 (194215'16893) delete 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.2174595.0:18789 0.000000 2014-08-13 17:38:02.787831 7fe9bae887a0 20 read_log 194215'16895 (192888'16421) clean 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by osd.289.0:1916 2014-08-12 15:37:32.851990 2014-08-13 17:38:02.787851 7fe9bae887a0 20 read_log 194215'16896 (194215'16895) delete 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.2174595.0:18791 0.000000 2014-08-13 17:38:02.787862 7fe9bae887a0 20 read_log 194215'16897 (192834'16095) clean a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by osd.289.0:1918 2014-08-12 15:37:32.859422 2014-08-13 17:38:02.787874 7fe9bae887a0 20 read_log 194215'16898 (194215'16897) delete a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.2174595.0:18793 0.000000 2014-08-13 17:38:02.787885 7fe9bae887a0 20 read_log 194215'16899 (192888'16570) clean eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by osd.289.0:1920 2014-08-12 15:37:32.866299 2014-08-13 17:38:02.787897 7fe9bae887a0 20 read_log 194215'16900 (194215'16899) delete eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.2174595.0:18795 0.000000 2014-08-13 17:38:02.787907 7fe9bae887a0 20 read_log 194215'16901 (192888'16380) clean bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by osd.289.0:1922 2014-08-12 15:37:32.874164 2014-08-13 17:38:02.787920 7fe9bae887a0 20 read_log 194215'16902 (194215'16901) delete bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.2174595.0:18797 0.000000 2014-08-13 17:38:02.787930 7fe9bae887a0 20 read_log 194215'16903 (192834'15609) clean fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by osd.289.0:1924 2014-08-12 15:37:32.881603 2014-08-13 17:38:02.787942 7fe9bae887a0 20 read_log 194215'16904 (194215'16903) delete fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.2174595.0:18799 0.000000 2014-08-13 17:38:02.787953 7fe9bae887a0 20 read_log 194215'16905 (192888'16590) clean 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by osd.289.0:1926 2014-08-12 15:37:32.889643 2014-08-13 17:38:02.787965 7fe9bae887a0 20 read_log 194215'16906 (194215'16905) delete 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.2174595.0:18801 0.000000 2014-08-13 17:38:02.787976 7fe9bae887a0 20 read_log 194215'16907 (192888'16595) clean b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by osd.289.0:1928 2014-08-12 15:37:32.897113 2014-08-13 17:38:02.787988 7fe9bae887a0 20 read_log 194215'16908 (194215'16907) delete b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.2174595.0:18803 0.000000 2014-08-13 17:38:02.787999 7fe9bae887a0 20 read_log 194217'16909 (194217'16909) modify 5e/hit_set_24.5e_archive_2014-08-12 15:37:32.211135_2014-08-12 17:25:03.987810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 17:38:02.788011 7fe9bae887a0 20 read_log 194217'16910 (194215'16717) delete 5e/hit_set_24.5e_archive_2014-08-12 15:32:25.855993_2014-08-12 15:37:32.210894/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 17:38:02.788024 7fe9bae887a0 20 read_log 194217'16911 (0'0) promote 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by osd.289.0:3614 2014-08-12 17:25:03.991195 2014-08-13 17:38:02.788036 7fe9bae887a0 20 read_log 194217'16912 (194217'16911) modify 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by client.1487927.0:177 2014-08-12 17:25:03.982296 2014-08-13 17:38:02.788048 7fe9bae887a0 20 read_log 194370'16913 (0'0) promote 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by osd.289.0:3630 2014-08-12 20:30:15.306267 2014-08-13 17:38:02.788060 7fe9bae887a0 20 read_log 194370'16914 (194370'16913) modify 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by client.1508531.0:276 2014-08-12 20:30:15.299339 2014-08-13 17:38:02.788100 7fe9bae887a0 20 read_log 194370'16915 (0'0) promote 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by osd.289.0:3632 2014-08-12 20:30:15.412195 2014-08-13 17:38:02.788114 7fe9bae887a0 20 read_log 194370'16916 (194370'16915) modify 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by client.1508531.0:303 2014-08-12 20:30:15.404611 2014-08-13 17:38:02.788126 7fe9bae887a0 20 read_log 194370'16917 (0'0) promote 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by osd.289.0:3644 2014-08-12 20:30:16.477939 2014-08-13 17:38:02.788138 7fe9bae887a0 20 read_log 194370'16918 (194370'16917) modify 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by client.1508531.0:545 2014-08-12 20:30:16.471772 2014-08-13 17:38:02.788151 7fe9bae887a0 20 read_log 194370'16919 (0'0) promote 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by osd.289.0:3662 2014-08-12 20:30:18.701982 2014-08-13 17:38:02.788167 7fe9bae887a0 20 read_log 194370'16920 (194370'16919) modify 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by client.1508531.0:1158 2014-08-12 20:30:18.692375 2014-08-13 17:38:02.788191 7fe9bae887a0 20 read_log 194370'16921 (0'0) promote 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by osd.289.0:3666 2014-08-12 20:30:19.445130 2014-08-13 17:38:02.788203 7fe9bae887a0 20 read_log 194370'16922 (194370'16921) modify 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by client.1508531.0:1349 2014-08-12 20:30:19.437629 2014-08-13 17:38:02.788216 7fe9bae887a0 20 read_log 194370'16923 (0'0) promote c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by osd.289.0:3674 2014-08-12 20:30:21.042607 2014-08-13 17:38:02.788228 7fe9bae887a0 20 read_log 194370'16924 (194370'16923) modify c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by client.1508531.0:1783 2014-08-12 20:30:21.032416 2014-08-13 17:38:02.788240 7fe9bae887a0 20 read_log 194370'16925 (0'0) promote c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by osd.289.0:3676 2014-08-12 20:30:21.088164 2014-08-13 17:38:02.788252 7fe9bae887a0 20 read_log 194370'16926 (194370'16925) modify c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by client.1508531.0:1795 2014-08-12 20:30:21.077929 2014-08-13 17:38:02.788265 7fe9bae887a0 20 read_log 194370'16927 (0'0) promote 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by osd.289.0:3750 2014-08-12 20:30:39.341360 2014-08-13 17:38:02.788277 7fe9bae887a0 20 read_log 194370'16928 (194370'16927) modify 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by client.1508531.0:6370 2014-08-12 20:30:39.333782 2014-08-13 17:38:02.788289 7fe9bae887a0 20 read_log 194370'16929 (0'0) promote 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by osd.289.0:3770 2014-08-12 20:30:43.532904 2014-08-13 17:38:02.788301 7fe9bae887a0 20 read_log 194370'16930 (194370'16929) modify 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by client.1508531.0:7235 2014-08-12 20:30:43.520542 2014-08-13 17:38:02.788314 7fe9bae887a0 20 read_log 194370'16931 (0'0) promote d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by osd.289.0:3776 2014-08-12 20:30:44.573781 2014-08-13 17:38:02.788326 7fe9bae887a0 20 read_log 194370'16932 (194370'16931) modify d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by client.1508531.0:7443 2014-08-12 20:30:44.563223 2014-08-13 17:38:02.788338 7fe9bae887a0 20 read_log 194370'16933 (0'0) promote d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by osd.289.0:3792 2014-08-12 20:30:50.836699 2014-08-13 17:38:02.788350 7fe9bae887a0 20 read_log 194370'16934 (194370'16933) modify d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by client.1508531.0:8596 2014-08-12 20:30:50.829702 2014-08-13 17:38:02.788370 7fe9bae887a0 20 read_log 194370'16935 (0'0) promote eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by osd.289.0:3816 2014-08-12 20:30:58.707828 2014-08-13 17:38:02.788383 7fe9bae887a0 20 read_log 194370'16936 (194370'16935) modify eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by client.1508531.0:9903 2014-08-12 20:30:58.699187 2014-08-13 17:38:02.788395 7fe9bae887a0 20 read_log 194370'16937 (0'0) promote aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by osd.289.0:3824 2014-08-12 20:31:00.847698 2014-08-13 17:38:02.788407 7fe9bae887a0 20 read_log 194370'16938 (194370'16937) modify aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by client.1508531.0:10270 2014-08-12 20:31:00.838695 2014-08-13 17:38:02.788419 7fe9bae887a0 20 read_log 194370'16939 (0'0) promote 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by osd.289.0:3830 2014-08-12 20:31:02.605182 2014-08-13 17:38:02.788449 7fe9bae887a0 20 read_log 194370'16940 (194370'16939) modify 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by client.1508531.0:10561 2014-08-12 20:31:02.595811 2014-08-13 17:38:02.788465 7fe9bae887a0 20 read_log 194370'16941 (0'0) promote 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by osd.289.0:3876 2014-08-12 20:31:19.931553 2014-08-13 17:38:02.788477 7fe9bae887a0 20 read_log 194370'16942 (194370'16941) modify 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by client.1508531.0:13061 2014-08-12 20:31:19.922980 2014-08-13 17:38:02.788490 7fe9bae887a0 20 read_log 194370'16943 (0'0) promote 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by osd.289.0:3894 2014-08-12 20:31:30.063409 2014-08-13 17:38:02.788502 7fe9bae887a0 20 read_log 194370'16944 (194370'16943) modify 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by client.1508531.0:13921 2014-08-12 20:31:30.054866 2014-08-13 17:38:02.788515 7fe9bae887a0 20 read_log 194377'16945 (0'0) promote da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by osd.289.0:3898 2014-08-12 20:32:57.928655 2014-08-13 17:38:02.788527 7fe9bae887a0 20 read_log 194377'16946 (194377'16945) modify da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by client.2231190.0:182 2014-08-12 20:32:57.922328 2014-08-13 17:38:02.788539 7fe9bae887a0 20 read_log 194377'16947 (0'0) promote 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by osd.289.0:3902 2014-08-12 20:32:58.158659 2014-08-13 17:38:02.788552 7fe9bae887a0 20 read_log 194377'16948 (194377'16947) modify 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by client.2231190.0:243 2014-08-12 20:32:58.153044 2014-08-13 17:38:02.788564 7fe9bae887a0 20 read_log 194377'16949 (0'0) promote cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by osd.289.0:3932 2014-08-12 20:33:04.336264 2014-08-13 17:38:02.788579 7fe9bae887a0 20 read_log 194377'16950 (194377'16949) modify cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by client.2231190.0:1778 2014-08-12 20:33:04.328244 2014-08-13 17:38:02.788592 7fe9bae887a0 20 read_log 194377'16951 (0'0) promote 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by osd.289.0:3948 2014-08-12 20:33:06.994988 2014-08-13 17:38:02.788604 7fe9bae887a0 20 read_log 194377'16952 (194377'16951) modify 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by client.2231190.0:2443 2014-08-12 20:33:06.984933 2014-08-13 17:38:02.788617 7fe9bae887a0 20 read_log 194377'16953 (0'0) promote 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by osd.289.0:3958 2014-08-12 20:33:08.787056 2014-08-13 17:38:02.788629 7fe9bae887a0 20 read_log 194377'16954 (194377'16953) modify 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by client.2231190.0:2875 2014-08-12 20:33:08.779446 2014-08-13 17:38:02.788648 7fe9bae887a0 20 read_log 194377'16955 (0'0) promote 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by osd.289.0:3978 2014-08-12 20:33:12.084534 2014-08-13 17:38:02.788661 7fe9bae887a0 20 read_log 194377'16956 (194377'16955) modify 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by client.2231190.0:3644 2014-08-12 20:33:12.077222 2014-08-13 17:38:02.788673 7fe9bae887a0 20 read_log 194377'16957 (0'0) promote 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by osd.289.0:3984 2014-08-12 20:33:13.879580 2014-08-13 17:38:02.788685 7fe9bae887a0 20 read_log 194377'16958 (194377'16957) modify 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by client.2231190.0:3993 2014-08-12 20:33:13.871436 2014-08-13 17:38:02.788697 7fe9bae887a0 20 read_log 194383'16959 (0'0) promote b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by osd.289.0:4044 2014-08-12 20:34:30.647679 2014-08-13 17:38:02.788709 7fe9bae887a0 20 read_log 194383'16960 (194383'16959) modify b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by client.2231190.0:7569 2014-08-12 20:34:30.638584 2014-08-13 17:38:02.788721 7fe9bae887a0 20 read_log 194393'16961 (0'0) promote 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by osd.289.0:4072 2014-08-12 20:49:26.719414 2014-08-13 17:38:02.788733 7fe9bae887a0 20 read_log 194393'16962 (194393'16961) modify 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by client.1064182.0:265 2014-08-12 20:49:26.711262 2014-08-13 17:38:02.788745 7fe9bae887a0 20 read_log 194393'16963 (0'0) promote c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by osd.289.0:4093 2014-08-12 20:52:56.726134 2014-08-13 17:38:02.788758 7fe9bae887a0 20 read_log 194393'16964 (194393'16963) modify c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by client.2235069.0:85 2014-08-12 20:52:56.719683 2014-08-13 17:38:02.788770 7fe9bae887a0 20 read_log 194416'16965 (0'0) promote 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by osd.289.0:4281 2014-08-13 15:28:35.502228 2014-08-13 17:38:02.788781 7fe9bae887a0 20 read_log 194416'16966 (194416'16965) modify 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by client.2441958.0:1260 2014-08-13 15:28:35.492974 2014-08-13 17:38:02.788793 7fe9bae887a0 20 read_log 194416'16967 (0'0) promote 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by osd.289.0:4314 2014-08-13 15:28:42.456735 2014-08-13 17:38:02.788805 7fe9bae887a0 20 read_log 194416'16968 (194416'16967) modify 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by client.2441958.0:3101 2014-08-13 15:28:42.448145 2014-08-13 17:38:02.788817 7fe9bae887a0 20 read_log 194416'16969 (0'0) promote 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by osd.289.0:4334 2014-08-13 15:28:46.553242 2014-08-13 17:38:02.788829 7fe9bae887a0 20 read_log 194416'16970 (194416'16969) modify 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by client.2441958.0:4159 2014-08-13 15:28:46.543697 2014-08-13 17:38:02.788841 7fe9bae887a0 20 read_log 194416'16971 (0'0) promote 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by osd.289.0:4340 2014-08-13 15:28:47.641557 2014-08-13 17:38:02.788853 7fe9bae887a0 20 read_log 194416'16972 (194416'16971) modify 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by client.2441958.0:4439 2014-08-13 15:28:47.634245 2014-08-13 17:38:02.788865 7fe9bae887a0 20 read_log 194416'16973 (0'0) promote 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by osd.289.0:4342 2014-08-13 15:28:48.797392 2014-08-13 17:38:02.788887 7fe9bae887a0 20 read_log 194416'16974 (194416'16973) modify 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by client.2441958.0:4710 2014-08-13 15:28:48.788974 2014-08-13 17:38:02.788900 7fe9bae887a0 20 read_log 194416'16975 (0'0) promote 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by osd.289.0:4348 2014-08-13 15:28:49.959265 2014-08-13 17:38:02.788913 7fe9bae887a0 20 read_log 194416'16976 (194416'16975) modify 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by client.2441958.0:5018 2014-08-13 15:28:49.950732 2014-08-13 17:38:02.788925 7fe9bae887a0 20 read_log 194416'16977 (0'0) promote 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by osd.289.0:4358 2014-08-13 15:28:51.486788 2014-08-13 17:38:02.788937 7fe9bae887a0 20 read_log 194416'16978 (194416'16977) modify 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by client.2441958.0:5418 2014-08-13 15:28:51.478489 2014-08-13 17:38:02.788949 7fe9bae887a0 20 read_log 194416'16979 (0'0) promote 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by osd.289.0:4365 2014-08-13 15:28:52.833508 2014-08-13 17:38:02.788961 7fe9bae887a0 20 read_log 194416'16980 (194416'16979) modify 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by client.2441958.0:5763 2014-08-13 15:28:52.824627 2014-08-13 17:38:02.788973 7fe9bae887a0 20 read_log 194416'16981 (0'0) promote 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by osd.289.0:4371 2014-08-13 15:28:53.238124 2014-08-13 17:38:02.788985 7fe9bae887a0 20 read_log 194416'16982 (194416'16981) modify 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by client.2441958.0:5869 2014-08-13 15:28:53.229339 2014-08-13 17:38:02.788997 7fe9bae887a0 20 read_log 194416'16983 (0'0) promote 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by osd.289.0:4385 2014-08-13 15:28:55.628268 2014-08-13 17:38:02.789009 7fe9bae887a0 20 read_log 194416'16984 (194416'16983) modify 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by client.2441958.0:6434 2014-08-13 15:28:55.619015 2014-08-13 17:38:02.789021 7fe9bae887a0 20 read_log 194416'16985 (0'0) promote 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by osd.289.0:4399 2014-08-13 15:28:59.726840 2014-08-13 17:38:02.789034 7fe9bae887a0 20 read_log 194416'16986 (194416'16985) modify 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by client.2441958.0:7520 2014-08-13 15:28:59.718424 2014-08-13 17:38:02.789047 7fe9bae887a0 20 read_log 194416'16987 (0'0) promote 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by osd.289.0:4405 2014-08-13 15:29:00.669794 2014-08-13 17:38:02.789059 7fe9bae887a0 20 read_log 194416'16988 (194416'16987) modify 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by client.2441958.0:7771 2014-08-13 15:29:00.661735 2014-08-13 17:38:02.789071 7fe9bae887a0 20 read_log 194416'16989 (0'0) promote a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by osd.289.0:4421 2014-08-13 15:29:05.020705 2014-08-13 17:38:02.789083 7fe9bae887a0 20 read_log 194416'16990 (194416'16989) modify a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by client.2441958.0:8899 2014-08-13 15:29:05.011810 2014-08-13 17:38:02.789095 7fe9bae887a0 20 read_log 194416'16991 (0'0) promote e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by osd.289.0:4473 2014-08-13 15:29:17.814759 2014-08-13 17:38:02.789107 7fe9bae887a0 20 read_log 194416'16992 (194416'16991) modify e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by client.2441958.0:12046 2014-08-13 15:29:17.806207 2014-08-13 17:38:02.789119 7fe9bae887a0 20 read_log 194416'16993 (0'0) promote 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by osd.289.0:4475 2014-08-13 15:29:17.842191 2014-08-13 17:38:02.789138 7fe9bae887a0 20 read_log 194416'16994 (194416'16993) modify 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by client.2441958.0:12052 2014-08-13 15:29:17.834093 2014-08-13 17:38:02.789151 7fe9bae887a0 20 read_log 194416'16995 (0'0) promote 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by osd.289.0:4487 2014-08-13 15:29:20.235649 2014-08-13 17:38:02.789168 7fe9bae887a0 20 read_log 194416'16996 (194416'16995) modify 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by client.2441958.0:12621 2014-08-13 15:29:20.226221 2014-08-13 17:38:02.789180 7fe9bae887a0 20 read_log 194416'16997 (0'0) promote 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by osd.289.0:4503 2014-08-13 15:29:24.175799 2014-08-13 17:38:02.789193 7fe9bae887a0 20 read_log 194416'16998 (194416'16997) modify 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by client.2441958.0:13516 2014-08-13 15:29:24.166796 2014-08-13 17:38:02.789205 7fe9bae887a0 20 read_log 194416'16999 (0'0) promote 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by osd.289.0:4535 2014-08-13 15:29:34.392361 2014-08-13 17:38:02.789217 7fe9bae887a0 20 read_log 194416'17000 (194416'16999) modify 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by client.2441958.0:15803 2014-08-13 15:29:34.382066 2014-08-13 17:38:02.789229 7fe9bae887a0 20 read_log 194416'17001 (0'0) promote 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by osd.289.0:4543 2014-08-13 15:29:39.440499 2014-08-13 17:38:02.789241 7fe9bae887a0 20 read_log 194416'17002 (194416'17001) modify 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by client.2441958.0:16443 2014-08-13 15:29:39.431217 2014-08-13 17:38:02.789253 7fe9bae887a0 20 read_log 194416'17003 (0'0) promote 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by osd.289.0:4545 2014-08-13 15:29:40.828203 2014-08-13 17:38:02.789265 7fe9bae887a0 20 read_log 194416'17004 (194416'17003) modify 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by client.2441958.0:16542 2014-08-13 15:29:40.818618 2014-08-13 17:38:02.789278 7fe9bae887a0 20 read_log 194416'17005 (0'0) promote 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by osd.289.0:4551 2014-08-13 15:29:43.801767 2014-08-13 17:38:02.789289 7fe9bae887a0 20 read_log 194416'17006 (194416'17005) modify 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by client.2441958.0:16754 2014-08-13 15:29:43.791717 2014-08-13 17:38:02.789302 7fe9bae887a0 20 read_log 194416'17007 (0'0) promote 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by osd.289.0:4555 2014-08-13 15:29:46.223807 2014-08-13 17:38:02.789313 7fe9bae887a0 20 read_log 194416'17008 (194416'17007) modify 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by client.2441958.0:16921 2014-08-13 15:29:46.214245 2014-08-13 17:38:02.789326 7fe9bae887a0 20 read_log 194416'17009 (0'0) promote 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by osd.289.0:4559 2014-08-13 15:29:47.577791 2014-08-13 17:38:02.789338 7fe9bae887a0 20 read_log 194416'17010 (194416'17009) modify 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by client.2441958.0:17016 2014-08-13 15:29:47.568548 2014-08-13 17:38:02.789350 7fe9bae887a0 20 read_log 194416'17011 (0'0) promote d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by osd.289.0:4573 2014-08-13 15:29:56.281401 2014-08-13 17:38:02.789362 7fe9bae887a0 20 read_log 194416'17012 (194416'17011) modify d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by client.2441958.0:17630 2014-08-13 15:29:56.272694 2014-08-13 17:38:02.789375 7fe9bae887a0 20 read_log 194416'17013 (0'0) promote 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by osd.289.0:4589 2014-08-13 15:30:14.384554 2014-08-13 17:38:02.789394 7fe9bae887a0 20 read_log 194416'17014 (194416'17013) modify 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by client.2441958.0:18547 2014-08-13 15:30:14.376062 2014-08-13 17:38:02.789407 7fe9bae887a0 20 read_log 194422'17015 (0'0) promote f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by osd.289.0:4595 2014-08-13 15:32:15.554127 2014-08-13 17:38:02.789419 7fe9bae887a0 20 read_log 194422'17016 (194422'17015) modify f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by client.1148203.0:137 2014-08-13 15:32:15.545108 2014-08-13 17:38:02.789431 7fe9bae887a0 20 read_log 194422'17017 (0'0) promote 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by osd.289.0:4609 2014-08-13 15:32:21.003831 2014-08-13 17:38:02.789443 7fe9bae887a0 20 read_log 194422'17018 (194422'17017) modify 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by client.1148203.0:1435 2014-08-13 15:32:20.994549 2014-08-13 17:38:02.789456 7fe9bae887a0 20 read_log 194422'17019 (0'0) promote a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by osd.289.0:4621 2014-08-13 15:32:23.189497 2014-08-13 17:38:02.789468 7fe9bae887a0 20 read_log 194422'17020 (194422'17019) modify a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by client.1148203.0:1986 2014-08-13 15:32:23.176283 2014-08-13 17:38:02.789480 7fe9bae887a0 20 read_log 194422'17021 (0'0) promote 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by osd.289.0:4633 2014-08-13 15:32:25.011040 2014-08-13 17:38:02.789492 7fe9bae887a0 20 read_log 194422'17022 (194422'17021) modify 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by client.1148203.0:2464 2014-08-13 15:32:25.001257 2014-08-13 17:38:02.789506 7fe9bae887a0 20 read_log 194422'17023 (0'0) promote 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by osd.289.0:4655 2014-08-13 15:32:31.279919 2014-08-13 17:38:02.789519 7fe9bae887a0 20 read_log 194422'17024 (194422'17023) modify 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by client.1148203.0:4071 2014-08-13 15:32:31.269670 2014-08-13 17:38:02.789531 7fe9bae887a0 20 read_log 194422'17025 (0'0) promote e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by osd.289.0:4669 2014-08-13 15:32:36.032657 2014-08-13 17:38:02.789543 7fe9bae887a0 20 read_log 194422'17026 (194422'17025) modify e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by client.1148203.0:5235 2014-08-13 15:32:36.023870 2014-08-13 17:38:02.789555 7fe9bae887a0 20 read_log 194422'17027 (0'0) promote addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by osd.289.0:4677 2014-08-13 15:32:37.076109 2014-08-13 17:38:02.789567 7fe9bae887a0 20 read_log 194422'17028 (194422'17027) modify addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by client.1148203.0:5448 2014-08-13 15:32:37.068280 2014-08-13 17:38:02.789579 7fe9bae887a0 20 read_log 194422'17029 (0'0) promote d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by osd.289.0:4683 2014-08-13 15:32:38.282723 2014-08-13 17:38:02.789591 7fe9bae887a0 20 read_log 194422'17030 (194422'17029) modify d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by client.1148203.0:5659 2014-08-13 15:32:38.273121 2014-08-13 17:38:02.789604 7fe9bae887a0 20 read_log 194422'17031 (0'0) promote ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by osd.289.0:4721 2014-08-13 15:32:51.426396 2014-08-13 17:38:02.789615 7fe9bae887a0 20 read_log 194422'17032 (194422'17031) modify ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by client.1148203.0:7956 2014-08-13 15:32:51.417220 2014-08-13 17:38:02.789643 7fe9bae887a0 20 read_log 194422'17033 (0'0) promote 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by osd.289.0:4763 2014-08-13 15:33:17.064308 2014-08-13 17:38:02.789665 7fe9bae887a0 20 read_log 194422'17034 (194422'17033) modify 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by client.1148203.0:10251 2014-08-13 15:33:17.055164 2014-08-13 17:38:02.789681 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.790090 7fe9bae887a0 10 read_log checking for missing items over interval (194215'16908,194422'17034] 2014-08-13 17:38:02.790137 7fe9bae887a0 15 read_log missing 194422'17034 (194422'17033) modify 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by client.1148203.0:10251 2014-08-13 15:33:17.055164 2014-08-13 17:38:02.790184 7fe9bae887a0 15 read_log missing 194422'17032 (194422'17031) modify ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by client.1148203.0:7956 2014-08-13 15:32:51.417220 2014-08-13 17:38:02.790222 7fe9bae887a0 15 read_log missing 194422'17030 (194422'17029) modify d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by client.1148203.0:5659 2014-08-13 15:32:38.273121 2014-08-13 17:38:02.790259 7fe9bae887a0 15 read_log missing 194422'17028 (194422'17027) modify addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by client.1148203.0:5448 2014-08-13 15:32:37.068280 2014-08-13 17:38:02.790296 7fe9bae887a0 15 read_log missing 194422'17026 (194422'17025) modify e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by client.1148203.0:5235 2014-08-13 15:32:36.023870 2014-08-13 17:38:02.790333 7fe9bae887a0 15 read_log missing 194422'17024 (194422'17023) modify 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by client.1148203.0:4071 2014-08-13 15:32:31.269670 2014-08-13 17:38:02.790373 7fe9bae887a0 15 read_log missing 194422'17022 (194422'17021) modify 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by client.1148203.0:2464 2014-08-13 15:32:25.001257 2014-08-13 17:38:02.790410 7fe9bae887a0 15 read_log missing 194422'17020 (194422'17019) modify a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by client.1148203.0:1986 2014-08-13 15:32:23.176283 2014-08-13 17:38:02.790446 7fe9bae887a0 15 read_log missing 194422'17018 (194422'17017) modify 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by client.1148203.0:1435 2014-08-13 15:32:20.994549 2014-08-13 17:38:02.790482 7fe9bae887a0 15 read_log missing 194422'17016 (194422'17015) modify f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by client.1148203.0:137 2014-08-13 15:32:15.545108 2014-08-13 17:38:02.790518 7fe9bae887a0 15 read_log missing 194416'17014 (194416'17013) modify 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by client.2441958.0:18547 2014-08-13 15:30:14.376062 2014-08-13 17:38:02.790555 7fe9bae887a0 15 read_log missing 194416'17012 (194416'17011) modify d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by client.2441958.0:17630 2014-08-13 15:29:56.272694 2014-08-13 17:38:02.790591 7fe9bae887a0 15 read_log missing 194416'17010 (194416'17009) modify 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by client.2441958.0:17016 2014-08-13 15:29:47.568548 2014-08-13 17:38:02.790632 7fe9bae887a0 15 read_log missing 194416'17008 (194416'17007) modify 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by client.2441958.0:16921 2014-08-13 15:29:46.214245 2014-08-13 17:38:02.790669 7fe9bae887a0 15 read_log missing 194416'17006 (194416'17005) modify 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by client.2441958.0:16754 2014-08-13 15:29:43.791717 2014-08-13 17:38:02.790710 7fe9bae887a0 15 read_log missing 194416'17004 (194416'17003) modify 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by client.2441958.0:16542 2014-08-13 15:29:40.818618 2014-08-13 17:38:02.790747 7fe9bae887a0 15 read_log missing 194416'17002 (194416'17001) modify 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by client.2441958.0:16443 2014-08-13 15:29:39.431217 2014-08-13 17:38:02.790783 7fe9bae887a0 15 read_log missing 194416'17000 (194416'16999) modify 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by client.2441958.0:15803 2014-08-13 15:29:34.382066 2014-08-13 17:38:02.790819 7fe9bae887a0 15 read_log missing 194416'16998 (194416'16997) modify 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by client.2441958.0:13516 2014-08-13 15:29:24.166796 2014-08-13 17:38:02.790856 7fe9bae887a0 15 read_log missing 194416'16996 (194416'16995) modify 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by client.2441958.0:12621 2014-08-13 15:29:20.226221 2014-08-13 17:38:02.790892 7fe9bae887a0 15 read_log missing 194416'16994 (194416'16993) modify 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by client.2441958.0:12052 2014-08-13 15:29:17.834093 2014-08-13 17:38:02.790929 7fe9bae887a0 15 read_log missing 194416'16992 (194416'16991) modify e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by client.2441958.0:12046 2014-08-13 15:29:17.806207 2014-08-13 17:38:02.790965 7fe9bae887a0 15 read_log missing 194416'16990 (194416'16989) modify a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by client.2441958.0:8899 2014-08-13 15:29:05.011810 2014-08-13 17:38:02.791001 7fe9bae887a0 15 read_log missing 194416'16988 (194416'16987) modify 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by client.2441958.0:7771 2014-08-13 15:29:00.661735 2014-08-13 17:38:02.791038 7fe9bae887a0 15 read_log missing 194416'16986 (194416'16985) modify 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by client.2441958.0:7520 2014-08-13 15:28:59.718424 2014-08-13 17:38:02.791075 7fe9bae887a0 15 read_log missing 194416'16984 (194416'16983) modify 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by client.2441958.0:6434 2014-08-13 15:28:55.619015 2014-08-13 17:38:02.791111 7fe9bae887a0 15 read_log missing 194416'16982 (194416'16981) modify 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by client.2441958.0:5869 2014-08-13 15:28:53.229339 2014-08-13 17:38:02.791147 7fe9bae887a0 15 read_log missing 194416'16980 (194416'16979) modify 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by client.2441958.0:5763 2014-08-13 15:28:52.824627 2014-08-13 17:38:02.791189 7fe9bae887a0 15 read_log missing 194416'16978 (194416'16977) modify 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by client.2441958.0:5418 2014-08-13 15:28:51.478489 2014-08-13 17:38:02.791227 7fe9bae887a0 15 read_log missing 194416'16976 (194416'16975) modify 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by client.2441958.0:5018 2014-08-13 15:28:49.950732 2014-08-13 17:38:02.791263 7fe9bae887a0 15 read_log missing 194416'16974 (194416'16973) modify 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by client.2441958.0:4710 2014-08-13 15:28:48.788974 2014-08-13 17:38:02.791300 7fe9bae887a0 15 read_log missing 194416'16972 (194416'16971) modify 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by client.2441958.0:4439 2014-08-13 15:28:47.634245 2014-08-13 17:38:02.791337 7fe9bae887a0 15 read_log missing 194416'16970 (194416'16969) modify 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by client.2441958.0:4159 2014-08-13 15:28:46.543697 2014-08-13 17:38:02.791373 7fe9bae887a0 15 read_log missing 194416'16968 (194416'16967) modify 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by client.2441958.0:3101 2014-08-13 15:28:42.448145 2014-08-13 17:38:02.791410 7fe9bae887a0 15 read_log missing 194416'16966 (194416'16965) modify 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by client.2441958.0:1260 2014-08-13 15:28:35.492974 2014-08-13 17:38:02.791446 7fe9bae887a0 15 read_log missing 194393'16964 (194393'16963) modify c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by client.2235069.0:85 2014-08-12 20:52:56.719683 2014-08-13 17:38:02.791482 7fe9bae887a0 15 read_log missing 194393'16962 (194393'16961) modify 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by client.1064182.0:265 2014-08-12 20:49:26.711262 2014-08-13 17:38:02.791519 7fe9bae887a0 15 read_log missing 194383'16960 (194383'16959) modify b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by client.2231190.0:7569 2014-08-12 20:34:30.638584 2014-08-13 17:38:02.791556 7fe9bae887a0 15 read_log missing 194377'16958 (194377'16957) modify 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by client.2231190.0:3993 2014-08-12 20:33:13.871436 2014-08-13 17:38:02.791593 7fe9bae887a0 15 read_log missing 194377'16956 (194377'16955) modify 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by client.2231190.0:3644 2014-08-12 20:33:12.077222 2014-08-13 17:38:02.791629 7fe9bae887a0 15 read_log missing 194377'16954 (194377'16953) modify 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by client.2231190.0:2875 2014-08-12 20:33:08.779446 2014-08-13 17:38:02.791666 7fe9bae887a0 15 read_log missing 194377'16952 (194377'16951) modify 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by client.2231190.0:2443 2014-08-12 20:33:06.984933 2014-08-13 17:38:02.791702 7fe9bae887a0 15 read_log missing 194377'16950 (194377'16949) modify cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by client.2231190.0:1778 2014-08-12 20:33:04.328244 2014-08-13 17:38:02.791738 7fe9bae887a0 15 read_log missing 194377'16948 (194377'16947) modify 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by client.2231190.0:243 2014-08-12 20:32:58.153044 2014-08-13 17:38:02.791774 7fe9bae887a0 15 read_log missing 194377'16946 (194377'16945) modify da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by client.2231190.0:182 2014-08-12 20:32:57.922328 2014-08-13 17:38:02.791811 7fe9bae887a0 15 read_log missing 194370'16944 (194370'16943) modify 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by client.1508531.0:13921 2014-08-12 20:31:30.054866 2014-08-13 17:38:02.791847 7fe9bae887a0 15 read_log missing 194370'16942 (194370'16941) modify 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by client.1508531.0:13061 2014-08-12 20:31:19.922980 2014-08-13 17:38:02.791883 7fe9bae887a0 15 read_log missing 194370'16940 (194370'16939) modify 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by client.1508531.0:10561 2014-08-12 20:31:02.595811 2014-08-13 17:38:02.791919 7fe9bae887a0 15 read_log missing 194370'16938 (194370'16937) modify aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by client.1508531.0:10270 2014-08-12 20:31:00.838695 2014-08-13 17:38:02.791956 7fe9bae887a0 15 read_log missing 194370'16936 (194370'16935) modify eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by client.1508531.0:9903 2014-08-12 20:30:58.699187 2014-08-13 17:38:02.791992 7fe9bae887a0 15 read_log missing 194370'16934 (194370'16933) modify d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by client.1508531.0:8596 2014-08-12 20:30:50.829702 2014-08-13 17:38:02.792029 7fe9bae887a0 15 read_log missing 194370'16932 (194370'16931) modify d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by client.1508531.0:7443 2014-08-12 20:30:44.563223 2014-08-13 17:38:02.792066 7fe9bae887a0 15 read_log missing 194370'16930 (194370'16929) modify 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by client.1508531.0:7235 2014-08-12 20:30:43.520542 2014-08-13 17:38:02.792103 7fe9bae887a0 15 read_log missing 194370'16928 (194370'16927) modify 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by client.1508531.0:6370 2014-08-12 20:30:39.333782 2014-08-13 17:38:02.792139 7fe9bae887a0 15 read_log missing 194370'16926 (194370'16925) modify c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by client.1508531.0:1795 2014-08-12 20:30:21.077929 2014-08-13 17:38:02.792183 7fe9bae887a0 15 read_log missing 194370'16924 (194370'16923) modify c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by client.1508531.0:1783 2014-08-12 20:30:21.032416 2014-08-13 17:38:02.792221 7fe9bae887a0 15 read_log missing 194370'16922 (194370'16921) modify 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by client.1508531.0:1349 2014-08-12 20:30:19.437629 2014-08-13 17:38:02.792258 7fe9bae887a0 15 read_log missing 194370'16920 (194370'16919) modify 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by client.1508531.0:1158 2014-08-12 20:30:18.692375 2014-08-13 17:38:02.792296 7fe9bae887a0 15 read_log missing 194370'16918 (194370'16917) modify 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by client.1508531.0:545 2014-08-12 20:30:16.471772 2014-08-13 17:38:02.792333 7fe9bae887a0 15 read_log missing 194370'16916 (194370'16915) modify 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by client.1508531.0:303 2014-08-12 20:30:15.404611 2014-08-13 17:38:02.792369 7fe9bae887a0 15 read_log missing 194370'16914 (194370'16913) modify 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by client.1508531.0:276 2014-08-12 20:30:15.299339 2014-08-13 17:38:02.792406 7fe9bae887a0 15 read_log missing 194217'16912 (194217'16911) modify 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by client.1487927.0:177 2014-08-12 17:25:03.982296 2014-08-13 17:38:02.792446 7fe9bae887a0 15 read_log missing 194217'16909 (194217'16909) modify 5e/hit_set_24.5e_archive_2014-08-12 15:37:32.211135_2014-08-12 17:25:03.987810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 17:38:02.792458 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.792486 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=0 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive m=63] handle_loaded 2014-08-13 17:38:02.792503 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=0 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Initial 0.014805 0 0.000000 2014-08-13 17:38:02.792515 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=0 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset 2014-08-13 17:38:02.792524 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=0 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194477 2014-08-13 17:38:02.792533 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log((192718'14372,194422'17034], crt=194422'17034) 2014-08-13 17:38:02.792542 7fe9bae887a0 10 osd.4 194478 pgid 24.9b coll 24.9b_head 2014-08-13 17:38:02.792650 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.9b 2014-08-13 17:38:02.792662 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.792695 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.9b(unlocked)] enter Initial 2014-08-13 17:38:02.792706 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.9b(unlocked)] enter NotTrimming 2014-08-13 17:38:02.792827 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.792974 7fe9bae887a0 20 read_log 192834'15074 (192834'15040) delete 24e5989b/benchmark_data_storage0111-ib.pocstor_26659_object6053/head//24 by osd.321.0:54113 2014-08-11 15:26:51.117161 2014-08-13 17:38:02.793004 7fe9bae887a0 20 read_log 192834'15121 (192834'14622) clean 1f6c209b/benchmark_data_storage0111-ib.pocstor_26659_object11198/head//24 by osd.321.0:54225 2014-08-11 15:26:56.367995 2014-08-13 17:38:02.793020 7fe9bae887a0 20 read_log 192834'15125 (192834'14634) clean 53cb349b/benchmark_data_storage0111-ib.pocstor_26659_object11501/head//24 by osd.321.0:54229 2014-08-11 15:26:56.387299 2014-08-13 17:38:02.793033 7fe9bae887a0 20 read_log 192834'15144 (192834'15121) delete 1f6c209b/benchmark_data_storage0111-ib.pocstor_26659_object11198/head//24 by osd.321.0:54248 2014-08-11 15:26:56.496688 2014-08-13 17:38:02.793045 7fe9bae887a0 20 read_log 192834'15148 (192834'15125) delete 53cb349b/benchmark_data_storage0111-ib.pocstor_26659_object11501/head//24 by osd.321.0:54252 2014-08-11 15:26:56.513641 2014-08-13 17:38:02.793057 7fe9bae887a0 20 read_log 192834'15157 (0'0) promote 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:54269 2014-08-11 15:26:57.334648 2014-08-13 17:38:02.793069 7fe9bae887a0 20 read_log 192834'15158 (192834'15157) modify 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by client.1883634.0:10869 2014-08-11 15:26:57.324935 2014-08-13 17:38:02.793082 7fe9bae887a0 20 read_log 192834'15179 (0'0) promote fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:54320 2014-08-11 15:26:59.683665 2014-08-13 17:38:02.793095 7fe9bae887a0 20 read_log 192834'15180 (192834'15179) modify fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by client.1883634.0:11404 2014-08-11 15:26:59.675811 2014-08-13 17:38:02.793107 7fe9bae887a0 20 read_log 192834'15205 (192834'14700) clean 6e5d549b/benchmark_data_storage0111-ib.pocstor_26659_object14626/head//24 by osd.321.0:54376 2014-08-11 15:27:01.696299 2014-08-13 17:38:02.793120 7fe9bae887a0 20 read_log 192834'15209 (192834'14733) clean 229b1c9b/benchmark_data_storage0111-ib.pocstor_26659_object16797/head//24 by osd.321.0:54380 2014-08-11 15:27:01.717278 2014-08-13 17:38:02.793132 7fe9bae887a0 20 read_log 192834'15210 (192834'14725) clean 87ff3c9b/benchmark_data_storage0111-ib.pocstor_26659_object16245/head//24 by osd.321.0:54381 2014-08-11 15:27:01.722389 2014-08-13 17:38:02.793144 7fe9bae887a0 20 read_log 192834'15211 (192834'14719) clean 47997c9b/benchmark_data_storage0111-ib.pocstor_26659_object15459/head//24 by osd.321.0:54382 2014-08-11 15:27:01.726323 2014-08-13 17:38:02.793162 7fe9bae887a0 20 read_log 192834'15247 (192834'14748) clean 378a109b/benchmark_data_storage0111-ib.pocstor_26659_object18844/head//24 by osd.321.0:54470 2014-08-11 15:27:06.912626 2014-08-13 17:38:02.793175 7fe9bae887a0 20 read_log 192834'15251 (192834'14756) clean 1ad5249b/benchmark_data_storage0111-ib.pocstor_26659_object19362/head//24 by osd.321.0:54474 2014-08-11 15:27:06.928977 2014-08-13 17:38:02.793187 7fe9bae887a0 20 read_log 192834'15252 (192834'14772) clean d3e8e49b/benchmark_data_storage0111-ib.pocstor_26659_object21507/head//24 by osd.321.0:54475 2014-08-11 15:27:06.937264 2014-08-13 17:38:02.793199 7fe9bae887a0 20 read_log 192834'15257 (192834'14741) clean 4dc89b/benchmark_data_storage0111-ib.pocstor_26659_object18019/head//24 by osd.321.0:54480 2014-08-11 15:27:06.958398 2014-08-13 17:38:02.793212 7fe9bae887a0 20 read_log 192834'15263 (192834'14739) clean 5a133c9b/benchmark_data_storage0111-ib.pocstor_26659_object17781/head//24 by osd.321.0:54486 2014-08-11 15:27:06.978421 2014-08-13 17:38:02.793224 7fe9bae887a0 20 read_log 192834'15272 (0'0) promote c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:54511 2014-08-11 15:27:08.303890 2014-08-13 17:38:02.793236 7fe9bae887a0 20 read_log 192834'15273 (192834'15272) modify c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by client.1883634.0:13261 2014-08-11 15:27:08.294647 2014-08-13 17:38:02.793249 7fe9bae887a0 20 read_log 192834'15298 (0'0) promote 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:54609 2014-08-11 15:27:16.206527 2014-08-13 17:38:02.793269 7fe9bae887a0 20 read_log 192834'15299 (192834'15298) modify 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by client.1883634.0:14936 2014-08-11 15:27:16.197910 2014-08-13 17:38:02.793282 7fe9bae887a0 20 read_log 192834'15305 (192834'14784) clean 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:54630 2014-08-11 15:27:17.681435 2014-08-13 17:38:02.793294 7fe9bae887a0 20 read_log 192834'15313 (0'0) promote dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:54652 2014-08-11 15:27:19.874486 2014-08-13 17:38:02.793306 7fe9bae887a0 20 read_log 192834'15314 (192834'15313) modify dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by client.1883634.0:15512 2014-08-11 15:27:19.865279 2014-08-13 17:38:02.793318 7fe9bae887a0 20 read_log 192834'15315 (0'0) promote 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:54654 2014-08-11 15:27:19.914332 2014-08-13 17:38:02.793330 7fe9bae887a0 20 read_log 192834'15316 (192834'15315) modify 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by client.1883634.0:15519 2014-08-11 15:27:19.904827 2014-08-13 17:38:02.793342 7fe9bae887a0 20 read_log 192834'15338 (192834'15305) delete 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:54713 2014-08-11 15:27:24.346555 2014-08-13 17:38:02.793355 7fe9bae887a0 20 read_log 192834'15353 (192834'14788) clean 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:54728 2014-08-11 15:27:25.059012 2014-08-13 17:38:02.793368 7fe9bae887a0 20 read_log 192834'15355 (192834'15353) delete 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:54731 2014-08-11 15:27:25.250349 2014-08-13 17:38:02.793380 7fe9bae887a0 20 read_log 192834'15365 (0'0) promote b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:54754 2014-08-11 15:27:26.278155 2014-08-13 17:38:02.793392 7fe9bae887a0 20 read_log 192834'15366 (192834'15365) modify b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by client.1883634.0:16161 2014-08-11 15:27:26.266643 2014-08-13 17:38:02.793405 7fe9bae887a0 20 read_log 192834'15389 (192834'14828) clean 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:54824 2014-08-11 15:27:31.262919 2014-08-13 17:38:02.793417 7fe9bae887a0 20 read_log 192834'15402 (192834'15389) delete 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:54838 2014-08-11 15:27:31.380947 2014-08-13 17:38:02.793429 7fe9bae887a0 20 read_log 192834'15406 (192834'14842) clean 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:54843 2014-08-11 15:27:31.542523 2014-08-13 17:38:02.793441 7fe9bae887a0 20 read_log 192834'15421 (192834'15406) delete 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:54892 2014-08-11 15:27:36.826042 2014-08-13 17:38:02.793453 7fe9bae887a0 20 read_log 192834'15443 (0'0) promote 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:54924 2014-08-11 15:27:38.427173 2014-08-13 17:38:02.793465 7fe9bae887a0 20 read_log 192834'15444 (192834'15443) modify 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by client.1883634.0:17680 2014-08-11 15:27:38.417513 2014-08-13 17:38:02.793477 7fe9bae887a0 20 read_log 192834'15470 (192834'15263) delete 5a133c9b/benchmark_data_storage0111-ib.pocstor_26659_object17781/head//24 by osd.321.0:54975 2014-08-11 15:27:42.675815 2014-08-13 17:38:02.793489 7fe9bae887a0 20 read_log 192834'15471 (192834'15210) delete 87ff3c9b/benchmark_data_storage0111-ib.pocstor_26659_object16245/head//24 by osd.321.0:54976 2014-08-11 15:27:42.679700 2014-08-13 17:38:02.793501 7fe9bae887a0 20 read_log 192834'15472 (192834'15211) delete 47997c9b/benchmark_data_storage0111-ib.pocstor_26659_object15459/head//24 by osd.321.0:54977 2014-08-11 15:27:42.683576 2014-08-13 17:38:02.793524 7fe9bae887a0 20 read_log 192834'15476 (192834'14878) clean 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:54981 2014-08-11 15:27:42.695193 2014-08-13 17:38:02.793537 7fe9bae887a0 20 read_log 192834'15521 (192834'15247) delete 378a109b/benchmark_data_storage0111-ib.pocstor_26659_object18844/head//24 by osd.321.0:55026 2014-08-11 15:27:42.851990 2014-08-13 17:38:02.793549 7fe9bae887a0 20 read_log 192834'15527 (192834'15251) delete 1ad5249b/benchmark_data_storage0111-ib.pocstor_26659_object19362/head//24 by osd.321.0:55032 2014-08-11 15:27:42.871981 2014-08-13 17:38:02.793561 7fe9bae887a0 20 read_log 192834'15528 (192834'15205) delete 6e5d549b/benchmark_data_storage0111-ib.pocstor_26659_object14626/head//24 by osd.321.0:55033 2014-08-11 15:27:42.874904 2014-08-13 17:38:02.793573 7fe9bae887a0 20 read_log 192834'15529 (192834'15252) delete d3e8e49b/benchmark_data_storage0111-ib.pocstor_26659_object21507/head//24 by osd.321.0:55034 2014-08-11 15:27:42.877841 2014-08-13 17:38:02.793585 7fe9bae887a0 20 read_log 192834'15535 (192834'15257) delete 4dc89b/benchmark_data_storage0111-ib.pocstor_26659_object18019/head//24 by osd.321.0:55040 2014-08-11 15:27:42.895172 2014-08-13 17:38:02.793597 7fe9bae887a0 20 read_log 192834'15543 (192834'15209) delete 229b1c9b/benchmark_data_storage0111-ib.pocstor_26659_object16797/head//24 by osd.321.0:55048 2014-08-11 15:27:42.915419 2014-08-13 17:38:02.793609 7fe9bae887a0 20 read_log 192834'15544 (192834'15476) delete 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:55049 2014-08-11 15:27:42.918447 2014-08-13 17:38:02.793621 7fe9bae887a0 20 read_log 192834'15571 (192834'15007) clean 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:55112 2014-08-11 15:27:48.337477 2014-08-13 17:38:02.793633 7fe9bae887a0 20 read_log 192834'15589 (192834'15571) delete 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:55148 2014-08-11 15:27:53.267500 2014-08-13 17:38:02.793645 7fe9bae887a0 20 read_log 192834'15596 (0'0) promote 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:55161 2014-08-11 15:27:53.589147 2014-08-13 17:38:02.793657 7fe9bae887a0 20 read_log 192834'15597 (192834'15596) modify 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by client.1883634.0:19626 2014-08-11 15:27:53.581136 2014-08-13 17:38:02.793669 7fe9bae887a0 20 read_log 192834'15602 (0'0) promote b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:55169 2014-08-11 15:27:54.357228 2014-08-13 17:38:02.793681 7fe9bae887a0 20 read_log 192834'15603 (192834'15602) modify b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by client.1883634.0:19850 2014-08-11 15:27:54.347328 2014-08-13 17:38:02.793693 7fe9bae887a0 20 read_log 192834'15608 (0'0) promote 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:55179 2014-08-11 15:27:54.903864 2014-08-13 17:38:02.793705 7fe9bae887a0 20 read_log 192834'15609 (192834'15608) modify 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by client.1883634.0:19998 2014-08-11 15:27:54.895406 2014-08-13 17:38:02.793717 7fe9bae887a0 20 read_log 192834'15640 (192834'15158) clean 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:55239 2014-08-11 15:27:58.703983 2014-08-13 17:38:02.793729 7fe9bae887a0 20 read_log 192834'15646 (192834'15640) delete 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:55245 2014-08-11 15:27:58.735483 2014-08-13 17:38:02.793741 7fe9bae887a0 20 read_log 192834'15671 (192834'15180) clean fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:55324 2014-08-11 15:28:04.104809 2014-08-13 17:38:02.793754 7fe9bae887a0 20 read_log 192834'15682 (192834'15671) delete fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:55335 2014-08-11 15:28:04.152246 2014-08-13 17:38:02.793773 7fe9bae887a0 20 read_log 192834'15695 (0'0) promote 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:55460 2014-08-11 15:28:10.359363 2014-08-13 17:38:02.793786 7fe9bae887a0 20 read_log 192834'15696 (192834'15695) modify 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by client.1883634.0:22624 2014-08-11 15:28:09.971569 2014-08-13 17:38:02.793798 7fe9bae887a0 20 read_log 192834'15710 (192834'15273) clean c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:55573 2014-08-11 15:28:14.584971 2014-08-13 17:38:02.793810 7fe9bae887a0 20 read_log 192834'15723 (192834'15710) delete c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:55586 2014-08-11 15:28:15.715581 2014-08-13 17:38:02.793822 7fe9bae887a0 20 read_log 192834'15736 (0'0) promote 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:55624 2014-08-11 15:28:18.036412 2014-08-13 17:38:02.793834 7fe9bae887a0 20 read_log 192834'15737 (192834'15736) modify 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by client.1883634.0:23715 2014-08-11 15:28:17.548271 2014-08-13 17:38:02.793846 7fe9bae887a0 20 read_log 192834'15740 (0'0) promote 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:55654 2014-08-11 15:28:18.895727 2014-08-13 17:38:02.793858 7fe9bae887a0 20 read_log 192834'15741 (192834'15740) modify 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by client.1883634.0:24250 2014-08-11 15:28:18.648664 2014-08-13 17:38:02.793870 7fe9bae887a0 20 read_log 192834'15768 (192834'15314) clean dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:55767 2014-08-11 15:28:21.668229 2014-08-13 17:38:02.793882 7fe9bae887a0 20 read_log 192834'15770 (192834'15316) clean 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:55770 2014-08-11 15:28:21.746071 2014-08-13 17:38:02.793894 7fe9bae887a0 20 read_log 192834'15773 (192834'15299) clean 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:55773 2014-08-11 15:28:21.756277 2014-08-13 17:38:02.793909 7fe9bae887a0 20 read_log 192834'15780 (192834'15768) delete dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:55784 2014-08-11 15:28:21.812808 2014-08-13 17:38:02.793922 7fe9bae887a0 20 read_log 192834'15783 (192834'15770) delete 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:55787 2014-08-11 15:28:21.852271 2014-08-13 17:38:02.793934 7fe9bae887a0 20 read_log 192834'15784 (192834'15773) delete 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:55788 2014-08-11 15:28:21.856317 2014-08-13 17:38:02.793946 7fe9bae887a0 20 read_log 192834'15813 (0'0) promote 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:55849 2014-08-11 15:28:23.841212 2014-08-13 17:38:02.793958 7fe9bae887a0 20 read_log 192834'15814 (192834'15813) modify 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by client.1883634.0:26946 2014-08-11 15:28:23.812876 2014-08-13 17:38:02.793972 7fe9bae887a0 20 read_log 192834'15827 (0'0) promote 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:55899 2014-08-11 15:28:25.200596 2014-08-13 17:38:02.793984 7fe9bae887a0 20 read_log 192834'15828 (192834'15827) modify 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by client.1883634.0:27948 2014-08-11 15:28:25.176538 2014-08-13 17:38:02.793996 7fe9bae887a0 20 read_log 192834'15860 (192834'15366) clean b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:55965 2014-08-11 15:28:27.173206 2014-08-13 17:38:02.794008 7fe9bae887a0 20 read_log 192834'15869 (0'0) promote 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:55997 2014-08-11 15:28:28.024373 2014-08-13 17:38:02.794028 7fe9bae887a0 20 read_log 192834'15870 (192834'15869) modify 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by client.1883634.0:30038 2014-08-11 15:28:27.993618 2014-08-13 17:38:02.794041 7fe9bae887a0 20 read_log 192834'15895 (0'0) promote 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56065 2014-08-11 15:28:31.179506 2014-08-13 17:38:02.794053 7fe9bae887a0 20 read_log 192834'15896 (192834'15895) modify 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by client.1883634.0:31941 2014-08-11 15:28:31.154016 2014-08-13 17:38:02.794065 7fe9bae887a0 20 read_log 192834'15908 (192834'15860) delete b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56103 2014-08-11 15:28:32.356287 2014-08-13 17:38:02.794077 7fe9bae887a0 20 read_log 192834'15941 (0'0) promote 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56174 2014-08-11 15:28:34.378710 2014-08-13 17:38:02.794106 7fe9bae887a0 20 read_log 192834'15942 (192834'15941) modify 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by client.1883634.0:34193 2014-08-11 15:28:34.351417 2014-08-13 17:38:02.794122 7fe9bae887a0 20 read_log 192834'15961 (0'0) promote fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56209 2014-08-11 15:28:35.250255 2014-08-13 17:38:02.794134 7fe9bae887a0 20 read_log 192834'15962 (192834'15961) modify fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by client.1883634.0:34728 2014-08-11 15:28:35.227977 2014-08-13 17:38:02.794146 7fe9bae887a0 20 read_log 192834'15990 (0'0) promote c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56293 2014-08-11 15:28:38.201404 2014-08-13 17:38:02.794163 7fe9bae887a0 20 read_log 192834'15991 (192834'15990) modify c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by client.1883634.0:36585 2014-08-11 15:28:38.178737 2014-08-13 17:38:02.794177 7fe9bae887a0 20 read_log 192834'16016 (0'0) promote 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56356 2014-08-11 15:28:40.454713 2014-08-13 17:38:02.794189 7fe9bae887a0 20 read_log 192834'16017 (192834'16016) modify 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by client.1883634.0:38260 2014-08-11 15:28:40.420894 2014-08-13 17:38:02.794201 7fe9bae887a0 20 read_log 192834'16026 (0'0) promote dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56380 2014-08-11 15:28:41.219275 2014-08-13 17:38:02.794213 7fe9bae887a0 20 read_log 192834'16027 (192834'16026) modify dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by client.1883634.0:38836 2014-08-11 15:28:41.194331 2014-08-13 17:38:02.794225 7fe9bae887a0 20 read_log 192834'16028 (0'0) promote 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56383 2014-08-11 15:28:41.230935 2014-08-13 17:38:02.794237 7fe9bae887a0 20 read_log 192834'16029 (192834'16028) modify 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by client.1883634.0:38843 2014-08-11 15:28:41.202067 2014-08-13 17:38:02.794249 7fe9bae887a0 20 read_log 192834'16050 (0'0) promote b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56429 2014-08-11 15:28:42.232668 2014-08-13 17:38:02.794262 7fe9bae887a0 20 read_log 192834'16051 (192834'16050) modify b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by client.1883634.0:39485 2014-08-11 15:28:42.209952 2014-08-13 17:38:02.794274 7fe9bae887a0 20 read_log 192834'16059 (192834'15444) clean 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56451 2014-08-11 15:28:42.846587 2014-08-13 17:38:02.794286 7fe9bae887a0 20 read_log 192834'16063 (192834'16059) delete 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56455 2014-08-11 15:28:42.933504 2014-08-13 17:38:02.794308 7fe9bae887a0 20 read_log 192834'16082 (0'0) promote 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56508 2014-08-11 15:28:44.524548 2014-08-13 17:38:02.794323 7fe9bae887a0 20 read_log 192834'16083 (192834'16082) modify 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by client.1883634.0:41004 2014-08-11 15:28:44.500967 2014-08-13 17:38:02.794336 7fe9bae887a0 20 read_log 192834'16102 (192834'15597) modify 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by client.1883634.0:42950 2014-08-11 15:28:46.412808 2014-08-13 17:38:02.794348 7fe9bae887a0 20 read_log 192834'16105 (192834'15603) modify b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by client.1883634.0:43174 2014-08-11 15:28:46.477021 2014-08-13 17:38:02.794360 7fe9bae887a0 20 read_log 192834'16108 (192834'15609) modify 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by client.1883634.0:43322 2014-08-11 15:28:46.520061 2014-08-13 17:38:02.794372 7fe9bae887a0 20 read_log 192834'16124 (192834'15696) modify 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by client.1883634.0:45948 2014-08-11 15:28:47.442630 2014-08-13 17:38:02.794385 7fe9bae887a0 20 read_log 192834'16130 (192834'15737) clean 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:56695 2014-08-11 15:29:18.667655 2014-08-13 17:38:02.794397 7fe9bae887a0 20 read_log 192834'16131 (192834'15741) clean 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:56696 2014-08-11 15:29:18.678428 2014-08-13 17:38:02.794410 7fe9bae887a0 20 read_log 192834'16134 (192834'16130) delete 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:56699 2014-08-11 15:29:18.735417 2014-08-13 17:38:02.794422 7fe9bae887a0 20 read_log 192834'16135 (192834'16131) delete 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:56700 2014-08-11 15:29:18.741423 2014-08-13 17:38:02.794434 7fe9bae887a0 20 read_log 192834'16154 (192834'15814) clean 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:56719 2014-08-11 15:29:23.915302 2014-08-13 17:38:02.794446 7fe9bae887a0 20 read_log 192834'16174 (192834'16154) delete 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:56739 2014-08-11 15:29:24.033769 2014-08-13 17:38:02.794459 7fe9bae887a0 20 read_log 192834'16197 (192834'15828) clean 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:56762 2014-08-11 15:29:29.223406 2014-08-13 17:38:02.794471 7fe9bae887a0 20 read_log 192834'16204 (192834'15870) clean 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:56769 2014-08-11 15:29:29.251085 2014-08-13 17:38:02.794483 7fe9bae887a0 20 read_log 192834'16225 (192834'16197) delete 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:56790 2014-08-11 15:29:29.356293 2014-08-13 17:38:02.794495 7fe9bae887a0 20 read_log 192834'16231 (192834'16204) delete 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:56796 2014-08-11 15:29:29.380973 2014-08-13 17:38:02.794507 7fe9bae887a0 20 read_log 192834'16255 (192834'15942) clean 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56820 2014-08-11 15:29:34.532726 2014-08-13 17:38:02.794519 7fe9bae887a0 20 read_log 192834'16261 (192834'15896) clean 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56826 2014-08-11 15:29:34.551303 2014-08-13 17:38:02.794531 7fe9bae887a0 20 read_log 192834'16282 (192834'16255) delete 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56847 2014-08-11 15:29:34.616265 2014-08-13 17:38:02.794543 7fe9bae887a0 20 read_log 192834'16286 (192834'16261) delete 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56851 2014-08-11 15:29:34.624557 2014-08-13 17:38:02.794579 7fe9bae887a0 20 read_log 192834'16304 (192834'15991) clean c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56869 2014-08-11 15:29:39.727824 2014-08-13 17:38:02.794595 7fe9bae887a0 20 read_log 192834'16309 (192834'15962) clean fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56874 2014-08-11 15:29:39.746112 2014-08-13 17:38:02.794607 7fe9bae887a0 20 read_log 192834'16333 (192834'16304) delete c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56898 2014-08-11 15:29:39.809613 2014-08-13 17:38:02.794619 7fe9bae887a0 20 read_log 192834'16337 (192834'16309) delete fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56902 2014-08-11 15:29:39.817818 2014-08-13 17:38:02.794632 7fe9bae887a0 20 read_log 192834'16367 (192834'16027) clean dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56932 2014-08-11 15:29:44.919861 2014-08-13 17:38:02.794644 7fe9bae887a0 20 read_log 192834'16372 (192834'16083) clean 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56937 2014-08-11 15:29:44.936612 2014-08-13 17:38:02.794656 7fe9bae887a0 20 read_log 192834'16389 (192834'16367) delete dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56954 2014-08-11 15:29:44.964210 2014-08-13 17:38:02.794668 7fe9bae887a0 20 read_log 192834'16390 (192834'16372) delete 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56955 2014-08-11 15:29:44.964464 2014-08-13 17:38:02.794680 7fe9bae887a0 20 read_log 192834'16395 (192834'16029) clean 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56960 2014-08-11 15:29:44.980958 2014-08-13 17:38:02.794692 7fe9bae887a0 20 read_log 192834'16396 (192834'16051) clean b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56961 2014-08-11 15:29:44.986162 2014-08-13 17:38:02.794704 7fe9bae887a0 20 read_log 192834'16397 (192834'16017) clean 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56962 2014-08-11 15:29:44.991458 2014-08-13 17:38:02.794716 7fe9bae887a0 20 read_log 192834'16404 (192834'16395) delete 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56969 2014-08-11 15:29:45.012510 2014-08-13 17:38:02.794728 7fe9bae887a0 20 read_log 192834'16407 (192834'16396) delete b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56972 2014-08-11 15:29:45.015429 2014-08-13 17:38:02.794740 7fe9bae887a0 20 read_log 192834'16409 (192834'16397) delete 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56974 2014-08-11 15:29:45.017453 2014-08-13 17:38:02.794752 7fe9bae887a0 20 read_log 192834'16462 (192834'16105) clean b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:57027 2014-08-11 15:29:50.114227 2014-08-13 17:38:02.794764 7fe9bae887a0 20 read_log 192834'16463 (192834'16124) clean 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:57028 2014-08-11 15:29:50.115651 2014-08-13 17:38:02.794776 7fe9bae887a0 20 read_log 192834'16465 (192834'16102) clean 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:57030 2014-08-11 15:29:50.117334 2014-08-13 17:38:02.794788 7fe9bae887a0 20 read_log 192834'16469 (192834'16462) delete b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:57034 2014-08-11 15:29:50.123636 2014-08-13 17:38:02.794807 7fe9bae887a0 20 read_log 192834'16470 (192834'16463) delete 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:57035 2014-08-11 15:29:50.124921 2014-08-13 17:38:02.794819 7fe9bae887a0 20 read_log 192834'16471 (192834'16465) delete 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:57036 2014-08-11 15:29:50.126004 2014-08-13 17:38:02.794839 7fe9bae887a0 20 read_log 192834'16480 (192834'16108) clean 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:57045 2014-08-11 15:29:50.139357 2014-08-13 17:38:02.794851 7fe9bae887a0 20 read_log 192834'16484 (192834'16480) delete 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:57049 2014-08-11 15:29:50.143491 2014-08-13 17:38:02.794864 7fe9bae887a0 20 read_log 192888'16490 (192888'16490) modify 9b/hit_set_24.9b_archive_2014-08-11 16:02:39.145403_2014-08-11 16:44:55.575365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.575861 2014-08-13 17:38:02.794876 7fe9bae887a0 20 read_log 192888'16491 (0'0) promote 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by osd.4.0:21381 2014-08-11 16:44:55.581901 2014-08-13 17:38:02.794888 7fe9bae887a0 20 read_log 192888'16492 (192888'16491) modify 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.1315259.0:161 2014-08-11 16:44:55.569607 2014-08-13 17:38:02.794900 7fe9bae887a0 20 read_log 192888'16495 (0'0) promote 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by osd.4.0:21430 2014-08-11 16:45:01.483541 2014-08-13 17:38:02.794914 7fe9bae887a0 20 read_log 192888'16496 (192888'16495) modify 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.1315259.0:1624 2014-08-11 16:45:01.475095 2014-08-13 17:38:02.794926 7fe9bae887a0 20 read_log 192888'16497 (0'0) promote ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by osd.4.0:21454 2014-08-11 16:45:04.788555 2014-08-13 17:38:02.794938 7fe9bae887a0 20 read_log 192888'16498 (192888'16497) modify ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.1315259.0:2464 2014-08-11 16:45:04.577662 2014-08-13 17:38:02.794950 7fe9bae887a0 20 read_log 192888'16499 (0'0) promote ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by osd.4.0:21466 2014-08-11 16:45:05.560019 2014-08-13 17:38:02.794963 7fe9bae887a0 20 read_log 192888'16500 (192888'16499) modify ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.1315259.0:2723 2014-08-11 16:45:05.551958 2014-08-13 17:38:02.794975 7fe9bae887a0 20 read_log 192888'16501 (0'0) promote 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by osd.4.0:21470 2014-08-11 16:45:06.752673 2014-08-13 17:38:02.794987 7fe9bae887a0 20 read_log 192888'16502 (192888'16501) modify 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.1315259.0:3045 2014-08-11 16:45:06.743502 2014-08-13 17:38:02.794999 7fe9bae887a0 20 read_log 192888'16507 (0'0) promote 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by osd.4.0:21496 2014-08-11 16:45:11.339353 2014-08-13 17:38:02.795011 7fe9bae887a0 20 read_log 192888'16508 (192888'16507) modify 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.1315259.0:4241 2014-08-11 16:45:11.330318 2014-08-13 17:38:02.795023 7fe9bae887a0 20 read_log 192888'16509 (0'0) promote 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by osd.4.0:21520 2014-08-11 16:45:13.486295 2014-08-13 17:38:02.795035 7fe9bae887a0 20 read_log 192888'16510 (192888'16509) modify 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.1315259.0:4833 2014-08-11 16:45:13.478515 2014-08-13 17:38:02.795048 7fe9bae887a0 20 read_log 192888'16511 (0'0) promote 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by osd.4.0:21534 2014-08-11 16:45:14.809563 2014-08-13 17:38:02.795060 7fe9bae887a0 20 read_log 192888'16512 (192888'16511) modify 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.1315259.0:5179 2014-08-11 16:45:14.801701 2014-08-13 17:38:02.795081 7fe9bae887a0 20 read_log 192888'16523 (0'0) promote 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by osd.4.0:21598 2014-08-11 16:45:24.414853 2014-08-13 17:38:02.795097 7fe9bae887a0 20 read_log 192888'16524 (192888'16523) modify 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.1315259.0:7825 2014-08-11 16:45:24.401578 2014-08-13 17:38:02.795109 7fe9bae887a0 20 read_log 192888'16525 (0'0) promote 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by osd.4.0:21608 2014-08-11 16:45:26.195805 2014-08-13 17:38:02.795121 7fe9bae887a0 20 read_log 192888'16526 (192888'16525) modify 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.1315259.0:8319 2014-08-11 16:45:26.187092 2014-08-13 17:38:02.795136 7fe9bae887a0 20 read_log 192888'16527 (0'0) promote a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by osd.4.0:21616 2014-08-11 16:45:27.047706 2014-08-13 17:38:02.795148 7fe9bae887a0 20 read_log 192888'16528 (192888'16527) modify a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.1315259.0:8520 2014-08-11 16:45:27.038518 2014-08-13 17:38:02.795165 7fe9bae887a0 20 read_log 192888'16545 (0'0) promote 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by osd.4.0:21728 2014-08-11 16:45:43.052477 2014-08-13 17:38:02.795180 7fe9bae887a0 20 read_log 192888'16546 (192888'16545) modify 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.1315259.0:12572 2014-08-11 16:45:43.043414 2014-08-13 17:38:02.795193 7fe9bae887a0 20 read_log 192888'16547 (0'0) promote 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by osd.4.0:21734 2014-08-11 16:45:44.181570 2014-08-13 17:38:02.795207 7fe9bae887a0 20 read_log 192888'16548 (192888'16547) modify 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.1315259.0:12837 2014-08-11 16:45:44.172547 2014-08-13 17:38:02.795219 7fe9bae887a0 20 read_log 192888'16549 (0'0) promote def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by osd.4.0:21772 2014-08-11 16:45:50.853714 2014-08-13 17:38:02.795234 7fe9bae887a0 20 read_log 192888'16550 (192888'16549) modify def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.1315259.0:14543 2014-08-11 16:45:50.845591 2014-08-13 17:38:02.795246 7fe9bae887a0 20 read_log 192888'16551 (0'0) promote cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by osd.4.0:21796 2014-08-11 16:45:54.503206 2014-08-13 17:38:02.795260 7fe9bae887a0 20 read_log 192888'16552 (192888'16551) modify cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.1315259.0:15489 2014-08-11 16:45:54.494515 2014-08-13 17:38:02.795272 7fe9bae887a0 20 read_log 192888'16559 (0'0) promote 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by osd.4.0:21818 2014-08-11 16:45:56.945931 2014-08-13 17:38:02.795284 7fe9bae887a0 20 read_log 192888'16560 (192888'16559) modify 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.1315259.0:16136 2014-08-11 16:45:56.935113 2014-08-13 17:38:02.795296 7fe9bae887a0 20 read_log 192888'16563 (0'0) promote 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by osd.4.0:21848 2014-08-11 16:46:00.984611 2014-08-13 17:38:02.795308 7fe9bae887a0 20 read_log 192888'16564 (192888'16563) modify 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.1315259.0:17163 2014-08-11 16:46:00.977207 2014-08-13 17:38:02.795320 7fe9bae887a0 20 read_log 192888'16567 (0'0) promote fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by osd.4.0:21856 2014-08-11 16:46:01.445634 2014-08-13 17:38:02.795348 7fe9bae887a0 20 read_log 192888'16568 (192888'16567) modify fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.1315259.0:17287 2014-08-11 16:46:01.436405 2014-08-13 17:38:02.795370 7fe9bae887a0 20 read_log 192888'16573 (0'0) promote e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by osd.4.0:21950 2014-08-11 16:46:15.024795 2014-08-13 17:38:02.795385 7fe9bae887a0 20 read_log 192888'16574 (192888'16573) modify e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.1315259.0:20768 2014-08-11 16:46:15.016704 2014-08-13 17:38:02.795398 7fe9bae887a0 20 read_log 192888'16577 (0'0) promote 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by osd.4.0:21976 2014-08-11 16:46:17.701039 2014-08-13 17:38:02.795410 7fe9bae887a0 20 read_log 192888'16578 (192888'16577) modify 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.1315259.0:21473 2014-08-11 16:46:17.692741 2014-08-13 17:38:02.795422 7fe9bae887a0 20 read_log 192888'16581 (0'0) promote 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by osd.4.0:22000 2014-08-11 16:46:21.319355 2014-08-13 17:38:02.795434 7fe9bae887a0 20 read_log 192888'16582 (192888'16581) modify 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.1315259.0:22411 2014-08-11 16:46:21.310832 2014-08-13 17:38:02.795446 7fe9bae887a0 20 read_log 192888'16587 (0'0) promote e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by osd.4.0:22034 2014-08-11 16:46:27.560326 2014-08-13 17:38:02.795458 7fe9bae887a0 20 read_log 192888'16588 (192888'16587) modify e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.1315259.0:23980 2014-08-11 16:46:27.551427 2014-08-13 17:38:02.795470 7fe9bae887a0 20 read_log 192888'16589 (0'0) promote 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by osd.4.0:22046 2014-08-11 16:46:28.737472 2014-08-13 17:38:02.795482 7fe9bae887a0 20 read_log 192888'16590 (192888'16589) modify 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.1315259.0:24291 2014-08-11 16:46:28.728519 2014-08-13 17:38:02.795494 7fe9bae887a0 20 read_log 192888'16593 (0'0) promote 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by osd.4.0:22084 2014-08-11 16:46:32.378159 2014-08-13 17:38:02.795506 7fe9bae887a0 20 read_log 192888'16594 (192888'16593) modify 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.1315259.0:25220 2014-08-11 16:46:32.367678 2014-08-13 17:38:02.795518 7fe9bae887a0 20 read_log 192888'16605 (0'0) promote 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by osd.4.0:22144 2014-08-11 16:46:41.907962 2014-08-13 17:38:02.795529 7fe9bae887a0 20 read_log 192888'16606 (192888'16605) modify 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.1315259.0:27473 2014-08-11 16:46:41.900184 2014-08-13 17:38:02.795541 7fe9bae887a0 20 read_log 192888'16611 (0'0) promote bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by osd.4.0:22178 2014-08-11 16:46:50.933223 2014-08-13 17:38:02.795553 7fe9bae887a0 20 read_log 192888'16612 (192888'16611) modify bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.1315259.0:29686 2014-08-11 16:46:50.924551 2014-08-13 17:38:02.795565 7fe9bae887a0 20 read_log 192888'16617 (0'0) promote 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by osd.4.0:22220 2014-08-11 16:46:54.408647 2014-08-13 17:38:02.795577 7fe9bae887a0 20 read_log 192888'16618 (192888'16617) modify 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.1315259.0:30608 2014-08-11 16:46:54.400124 2014-08-13 17:38:02.795589 7fe9bae887a0 20 read_log 192888'16619 (192888'16492) modify 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.1315259.0:30888 2014-08-11 16:46:55.928284 2014-08-13 17:38:02.795601 7fe9bae887a0 20 read_log 192888'16621 (192888'16496) modify 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.1315259.0:32351 2014-08-11 16:46:56.355066 2014-08-13 17:38:02.795620 7fe9bae887a0 20 read_log 192888'16622 (192888'16498) modify ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.1315259.0:33191 2014-08-11 16:46:56.574598 2014-08-13 17:38:02.795633 7fe9bae887a0 20 read_log 192888'16623 (192888'16500) modify ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.1315259.0:33450 2014-08-11 16:46:56.640201 2014-08-13 17:38:02.795646 7fe9bae887a0 20 read_log 192888'16624 (192888'16502) modify 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.1315259.0:33772 2014-08-11 16:46:56.721047 2014-08-13 17:38:02.795658 7fe9bae887a0 20 read_log 192888'16627 (192888'16508) modify 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.1315259.0:34968 2014-08-11 16:46:57.033749 2014-08-13 17:38:02.795670 7fe9bae887a0 20 read_log 192888'16628 (192888'16510) modify 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.1315259.0:35560 2014-08-11 16:46:57.191644 2014-08-13 17:38:02.795682 7fe9bae887a0 20 read_log 192888'16629 (192888'16512) modify 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.1315259.0:35906 2014-08-11 16:46:57.274923 2014-08-13 17:38:02.795694 7fe9bae887a0 20 read_log 192888'16635 (192888'16524) modify 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.1315259.0:38552 2014-08-11 16:46:57.979937 2014-08-13 17:38:02.795706 7fe9bae887a0 20 read_log 192888'16636 (192888'16526) modify 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.1315259.0:39046 2014-08-11 16:46:58.111536 2014-08-13 17:38:02.795718 7fe9bae887a0 20 read_log 192888'16637 (192888'16528) modify a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.1315259.0:39247 2014-08-11 16:46:58.162473 2014-08-13 17:38:02.795730 7fe9bae887a0 20 read_log 192888'16646 (192888'16546) modify 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.1315259.0:43299 2014-08-11 16:46:59.249675 2014-08-13 17:38:02.795742 7fe9bae887a0 20 read_log 192888'16647 (192888'16548) modify 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.1315259.0:43564 2014-08-11 16:46:59.326369 2014-08-13 17:38:02.795754 7fe9bae887a0 20 read_log 192888'16648 (192888'16550) modify def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.1315259.0:45270 2014-08-11 16:46:59.820029 2014-08-13 17:38:02.795766 7fe9bae887a0 20 read_log 192888'16649 (192888'16552) modify cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.1315259.0:46216 2014-08-11 16:47:00.094619 2014-08-13 17:38:02.795778 7fe9bae887a0 20 read_log 192888'16653 (192888'16560) modify 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.1315259.0:46863 2014-08-11 16:47:00.279805 2014-08-13 17:38:02.795791 7fe9bae887a0 20 read_log 192888'16655 (192888'16564) modify 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.1315259.0:47890 2014-08-11 16:47:00.599029 2014-08-13 17:38:02.795803 7fe9bae887a0 20 read_log 192888'16657 (192888'16568) modify fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.1315259.0:48014 2014-08-11 16:47:00.645284 2014-08-13 17:38:02.795815 7fe9bae887a0 20 read_log 192888'16660 (192888'16574) modify e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.1315259.0:51495 2014-08-11 16:47:01.697480 2014-08-13 17:38:02.795827 7fe9bae887a0 20 read_log 192888'16662 (192888'16578) modify 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.1315259.0:52200 2014-08-11 16:47:01.904483 2014-08-13 17:38:02.795839 7fe9bae887a0 20 read_log 192888'16664 (192888'16582) modify 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.1315259.0:53138 2014-08-11 16:47:02.166007 2014-08-13 17:38:02.795851 7fe9bae887a0 20 read_log 192888'16667 (192888'16588) modify e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.1315259.0:54707 2014-08-11 16:47:02.659072 2014-08-13 17:38:02.795872 7fe9bae887a0 20 read_log 192888'16668 (192888'16590) modify 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.1315259.0:55018 2014-08-11 16:47:02.788508 2014-08-13 17:38:02.795885 7fe9bae887a0 20 read_log 192888'16670 (192888'16594) modify 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.1315259.0:55947 2014-08-11 16:47:03.060907 2014-08-13 17:38:02.795897 7fe9bae887a0 20 read_log 192888'16676 (192888'16606) modify 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.1315259.0:58200 2014-08-11 16:47:03.733487 2014-08-13 17:38:02.795910 7fe9bae887a0 20 read_log 192888'16679 (192888'16612) modify bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.1315259.0:60413 2014-08-11 16:47:04.437455 2014-08-13 17:38:02.795922 7fe9bae887a0 20 read_log 192888'16682 (192888'16618) modify 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.1315259.0:61335 2014-08-11 16:47:04.707176 2014-08-13 17:38:02.795934 7fe9bae887a0 20 read_log 192888'16683 (192888'16683) modify 9b/hit_set_24.9b_archive_2014-08-11 16:44:55.575969_2014-08-11 17:05:18.967052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:18.967559 2014-08-13 17:38:02.795947 7fe9bae887a0 20 read_log 192888'16684 (192888'16490) delete 9b/hit_set_24.9b_archive_2014-08-11 16:02:39.145403_2014-08-11 16:44:55.575365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:18.967559 2014-08-13 17:38:02.795962 7fe9bae887a0 20 read_log 192888'16685 (0'0) promote ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by osd.4.0:22667 2014-08-11 17:05:18.972798 2014-08-13 17:38:02.795974 7fe9bae887a0 20 read_log 192888'16686 (192888'16685) modify ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.942868.0:970 2014-08-11 17:05:18.961432 2014-08-13 17:38:02.795987 7fe9bae887a0 20 read_log 192888'16689 (0'0) promote 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by osd.4.0:22689 2014-08-11 17:05:21.620176 2014-08-13 17:38:02.795999 7fe9bae887a0 20 read_log 192888'16690 (192888'16689) modify 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.942868.0:1596 2014-08-11 17:05:21.610399 2014-08-13 17:38:02.796011 7fe9bae887a0 20 read_log 192888'16691 (0'0) promote bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by osd.4.0:22691 2014-08-11 17:05:22.075070 2014-08-13 17:38:02.796023 7fe9bae887a0 20 read_log 192888'16692 (192888'16691) modify bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.942868.0:1709 2014-08-11 17:05:22.067725 2014-08-13 17:38:02.796039 7fe9bae887a0 20 read_log 192888'16699 (0'0) promote e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by osd.4.0:22734 2014-08-11 17:05:28.610686 2014-08-13 17:38:02.796052 7fe9bae887a0 20 read_log 192888'16700 (192888'16699) modify e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.942868.0:3425 2014-08-11 17:05:28.602752 2014-08-13 17:38:02.796066 7fe9bae887a0 20 read_log 192888'16705 (0'0) promote bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by osd.4.0:22754 2014-08-11 17:05:30.566472 2014-08-13 17:38:02.796079 7fe9bae887a0 20 read_log 192888'16706 (192888'16705) modify bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.942868.0:3954 2014-08-11 17:05:30.559746 2014-08-13 17:38:02.796092 7fe9bae887a0 20 read_log 192888'16707 (0'0) promote ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by osd.4.0:22784 2014-08-11 17:05:33.758902 2014-08-13 17:38:02.796104 7fe9bae887a0 20 read_log 192888'16708 (192888'16707) modify ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.942868.0:4801 2014-08-11 17:05:33.751332 2014-08-13 17:38:02.796123 7fe9bae887a0 20 read_log 192888'16709 (0'0) promote 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by osd.4.0:22790 2014-08-11 17:05:34.208586 2014-08-13 17:38:02.796136 7fe9bae887a0 20 read_log 192888'16710 (192888'16709) modify 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.942868.0:4903 2014-08-11 17:05:34.200526 2014-08-13 17:38:02.796148 7fe9bae887a0 20 read_log 192888'16713 (0'0) promote 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by osd.4.0:22810 2014-08-11 17:05:36.930786 2014-08-13 17:38:02.796169 7fe9bae887a0 20 read_log 192888'16714 (192888'16713) modify 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.942868.0:5618 2014-08-11 17:05:36.922962 2014-08-13 17:38:02.796182 7fe9bae887a0 20 read_log 192888'16721 (0'0) promote 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by osd.4.0:22854 2014-08-11 17:05:43.415846 2014-08-13 17:38:02.796195 7fe9bae887a0 20 read_log 192888'16722 (192888'16721) modify 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.942868.0:7356 2014-08-11 17:05:43.406311 2014-08-13 17:38:02.796207 7fe9bae887a0 20 read_log 192888'16725 (0'0) promote 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by osd.4.0:22908 2014-08-11 17:05:52.716042 2014-08-13 17:38:02.796218 7fe9bae887a0 20 read_log 192888'16726 (192888'16725) modify 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.942868.0:9864 2014-08-11 17:05:52.707977 2014-08-13 17:38:02.796230 7fe9bae887a0 20 read_log 192888'16729 (0'0) promote 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by osd.4.0:22930 2014-08-11 17:05:55.322844 2014-08-13 17:38:02.796242 7fe9bae887a0 20 read_log 192888'16730 (192888'16729) modify 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.942868.0:10588 2014-08-11 17:05:55.314947 2014-08-13 17:38:02.796255 7fe9bae887a0 20 read_log 192888'16731 (0'0) promote cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by osd.4.0:22944 2014-08-11 17:05:57.720776 2014-08-13 17:38:02.796267 7fe9bae887a0 20 read_log 192888'16732 (192888'16731) modify cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.942868.0:11236 2014-08-11 17:05:57.711472 2014-08-13 17:38:02.796279 7fe9bae887a0 20 read_log 192888'16741 (0'0) promote e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by osd.4.0:23020 2014-08-11 17:06:09.910660 2014-08-13 17:38:02.796291 7fe9bae887a0 20 read_log 192888'16742 (192888'16741) modify e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.942868.0:14416 2014-08-11 17:06:09.901594 2014-08-13 17:38:02.796303 7fe9bae887a0 20 read_log 192888'16743 (0'0) promote 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by osd.4.0:23022 2014-08-11 17:06:10.640869 2014-08-13 17:38:02.796315 7fe9bae887a0 20 read_log 192888'16744 (192888'16743) modify 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.942868.0:14592 2014-08-11 17:06:10.632078 2014-08-13 17:38:02.796331 7fe9bae887a0 20 read_log 192888'16757 (0'0) promote 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by osd.4.0:23124 2014-08-11 17:06:24.041643 2014-08-13 17:38:02.796343 7fe9bae887a0 20 read_log 192888'16758 (192888'16757) modify 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.942868.0:18055 2014-08-11 17:06:24.032527 2014-08-13 17:38:02.796357 7fe9bae887a0 20 read_log 192888'16759 (0'0) promote ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by osd.4.0:23160 2014-08-11 17:06:30.456178 2014-08-13 17:38:02.796369 7fe9bae887a0 20 read_log 192888'16760 (192888'16759) modify ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.942868.0:19760 2014-08-11 17:06:30.447113 2014-08-13 17:38:02.796408 7fe9bae887a0 20 read_log 192888'16761 (0'0) promote bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by osd.4.0:23178 2014-08-11 17:06:32.565707 2014-08-13 17:38:02.796424 7fe9bae887a0 20 read_log 192888'16762 (192888'16761) modify bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.942868.0:20323 2014-08-11 17:06:32.556918 2014-08-13 17:38:02.796437 7fe9bae887a0 20 read_log 192888'16765 (0'0) promote 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by osd.4.0:23188 2014-08-11 17:06:35.412377 2014-08-13 17:38:02.796449 7fe9bae887a0 20 read_log 192888'16766 (192888'16765) modify 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.942868.0:21057 2014-08-11 17:06:35.403705 2014-08-13 17:38:02.796461 7fe9bae887a0 20 read_log 192888'16773 (0'0) promote 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by osd.4.0:23216 2014-08-11 17:06:38.276877 2014-08-13 17:38:02.796473 7fe9bae887a0 20 read_log 192888'16774 (192888'16773) modify 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.942868.0:21811 2014-08-11 17:06:38.269046 2014-08-13 17:38:02.796485 7fe9bae887a0 20 read_log 192888'16775 (0'0) promote 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by osd.4.0:23234 2014-08-11 17:06:41.431866 2014-08-13 17:38:02.796497 7fe9bae887a0 20 read_log 192888'16776 (192888'16775) modify 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.942868.0:22612 2014-08-11 17:06:41.423880 2014-08-13 17:38:02.796510 7fe9bae887a0 20 read_log 192888'16783 (0'0) promote 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by osd.4.0:23302 2014-08-11 17:06:48.387243 2014-08-13 17:38:02.796522 7fe9bae887a0 20 read_log 192888'16784 (192888'16783) modify 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.942868.0:24451 2014-08-11 17:06:48.378678 2014-08-13 17:38:02.796534 7fe9bae887a0 20 read_log 192888'16787 (0'0) promote efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by osd.4.0:23332 2014-08-11 17:06:51.522486 2014-08-13 17:38:02.796545 7fe9bae887a0 20 read_log 192888'16788 (192888'16787) modify efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.942868.0:25293 2014-08-11 17:06:51.512762 2014-08-13 17:38:02.796558 7fe9bae887a0 20 read_log 192888'16795 (0'0) promote 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by osd.4.0:23382 2014-08-11 17:07:00.611648 2014-08-13 17:38:02.796585 7fe9bae887a0 20 read_log 192888'16796 (192888'16795) modify 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.942868.0:27639 2014-08-11 17:07:00.601718 2014-08-13 17:38:02.796598 7fe9bae887a0 20 read_log 192888'16797 (0'0) promote c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by osd.4.0:23394 2014-08-11 17:07:01.701078 2014-08-13 17:38:02.796610 7fe9bae887a0 20 read_log 192888'16798 (192888'16797) modify c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.942868.0:27920 2014-08-11 17:07:01.692805 2014-08-13 17:38:02.796622 7fe9bae887a0 20 read_log 192888'16803 (0'0) promote 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by osd.4.0:23436 2014-08-11 17:07:09.757402 2014-08-13 17:38:02.796634 7fe9bae887a0 20 read_log 192888'16804 (192888'16803) modify 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.942868.0:29985 2014-08-11 17:07:09.747032 2014-08-13 17:38:02.796647 7fe9bae887a0 20 read_log 192888'16805 (0'0) promote e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by osd.4.0:23450 2014-08-11 17:07:11.402054 2014-08-13 17:38:02.796659 7fe9bae887a0 20 read_log 192888'16806 (192888'16805) modify e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.942868.0:30392 2014-08-11 17:07:11.393287 2014-08-13 17:38:02.796681 7fe9bae887a0 20 read_log 192888'16811 (192888'16686) modify ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.942868.0:32236 2014-08-11 17:07:16.195017 2014-08-13 17:38:02.796697 7fe9bae887a0 20 read_log 192888'16813 (192888'16690) modify 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.942868.0:32862 2014-08-11 17:07:16.363979 2014-08-13 17:38:02.796709 7fe9bae887a0 20 read_log 192888'16814 (192888'16692) modify bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.942868.0:32975 2014-08-11 17:07:16.392657 2014-08-13 17:38:02.796721 7fe9bae887a0 20 read_log 192888'16818 (192888'16700) modify e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.942868.0:34691 2014-08-11 17:07:16.865761 2014-08-13 17:38:02.796733 7fe9bae887a0 20 read_log 192888'16821 (192888'16706) modify bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.942868.0:35220 2014-08-11 17:07:17.018744 2014-08-13 17:38:02.796746 7fe9bae887a0 20 read_log 192888'16822 (192888'16708) modify ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.942868.0:36067 2014-08-11 17:07:17.233702 2014-08-13 17:38:02.796758 7fe9bae887a0 20 read_log 192888'16823 (192888'16710) modify 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.942868.0:36169 2014-08-11 17:07:17.260112 2014-08-13 17:38:02.796770 7fe9bae887a0 20 read_log 192888'16825 (192888'16714) modify 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.942868.0:36884 2014-08-11 17:07:17.458618 2014-08-13 17:38:02.796782 7fe9bae887a0 20 read_log 192888'16829 (192888'16722) modify 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.942868.0:38622 2014-08-11 17:07:17.940788 2014-08-13 17:38:02.796794 7fe9bae887a0 20 read_log 192888'16831 (192888'16726) modify 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.942868.0:41130 2014-08-11 17:07:18.650076 2014-08-13 17:38:02.796806 7fe9bae887a0 20 read_log 192888'16833 (192888'16730) modify 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.942868.0:41854 2014-08-11 17:07:18.840959 2014-08-13 17:38:02.796818 7fe9bae887a0 20 read_log 192888'16834 (192888'16732) modify cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.942868.0:42502 2014-08-11 17:07:19.014254 2014-08-13 17:38:02.796830 7fe9bae887a0 20 read_log 192888'16839 (192888'16742) modify e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.942868.0:45682 2014-08-11 17:07:19.905192 2014-08-13 17:38:02.796842 7fe9bae887a0 20 read_log 192888'16840 (192888'16744) modify 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.942868.0:45858 2014-08-11 17:07:19.954430 2014-08-13 17:38:02.796855 7fe9bae887a0 20 read_log 192888'16847 (192888'16758) modify 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.942868.0:49321 2014-08-11 17:07:20.977700 2014-08-13 17:38:02.796867 7fe9bae887a0 20 read_log 192888'16848 (192888'16760) modify ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.942868.0:51026 2014-08-11 17:07:21.507165 2014-08-13 17:38:02.796879 7fe9bae887a0 20 read_log 192888'16849 (192888'16762) modify bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.942868.0:51589 2014-08-11 17:07:21.690314 2014-08-13 17:38:02.796891 7fe9bae887a0 20 read_log 192888'16851 (192888'16766) modify 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.942868.0:52323 2014-08-11 17:07:21.903144 2014-08-13 17:38:02.796903 7fe9bae887a0 20 read_log 192888'16855 (192888'16774) modify 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.942868.0:53077 2014-08-11 17:07:22.140725 2014-08-13 17:38:02.796916 7fe9bae887a0 20 read_log 192888'16856 (192888'16776) modify 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.942868.0:53878 2014-08-11 17:07:22.403187 2014-08-13 17:38:02.796936 7fe9bae887a0 20 read_log 192888'16860 (192888'16784) modify 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.942868.0:55717 2014-08-11 17:07:22.932737 2014-08-13 17:38:02.796949 7fe9bae887a0 20 read_log 192888'16862 (192888'16788) modify efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.942868.0:56559 2014-08-11 17:07:23.191514 2014-08-13 17:38:02.796961 7fe9bae887a0 20 read_log 192888'16866 (192888'16796) modify 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.942868.0:58905 2014-08-11 17:07:23.886567 2014-08-13 17:38:02.796973 7fe9bae887a0 20 read_log 192888'16867 (192888'16798) modify c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.942868.0:59186 2014-08-11 17:07:23.970039 2014-08-13 17:38:02.796985 7fe9bae887a0 20 read_log 192888'16870 (192888'16804) modify 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.942868.0:61251 2014-08-11 17:07:24.584930 2014-08-13 17:38:02.796996 7fe9bae887a0 20 read_log 192888'16871 (192888'16806) modify e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.942868.0:61658 2014-08-11 17:07:24.710255 2014-08-13 17:38:02.797009 7fe9bae887a0 20 read_log 193305'16874 (193305'16874) modify 9b/hit_set_24.9b_archive_2014-08-11 18:37:04.904270_2014-08-11 18:42:39.916705/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:39.916923 2014-08-13 17:38:02.797021 7fe9bae887a0 20 read_log 193305'16875 (192888'16683) delete 9b/hit_set_24.9b_archive_2014-08-11 16:44:55.575969_2014-08-11 17:05:18.967052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:39.916923 2014-08-13 17:38:02.797033 7fe9bae887a0 20 read_log 193305'16876 (0'0) promote 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by osd.4.0:21 2014-08-11 18:42:39.945347 2014-08-13 17:38:02.797045 7fe9bae887a0 20 read_log 193305'16877 (193305'16876) modify 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.951337.0:1779 2014-08-11 18:42:39.911239 2014-08-13 17:38:02.797058 7fe9bae887a0 20 read_log 193305'16878 (0'0) promote da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by osd.4.0:28 2014-08-11 18:42:40.890732 2014-08-13 17:38:02.797070 7fe9bae887a0 20 read_log 193305'16879 (193305'16878) modify da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.951337.0:2036 2014-08-11 18:42:40.884250 2014-08-13 17:38:02.797082 7fe9bae887a0 20 read_log 193305'16880 (0'0) promote 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by osd.4.0:43 2014-08-11 18:42:43.940584 2014-08-13 17:38:02.797094 7fe9bae887a0 20 read_log 193305'16881 (193305'16880) modify 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.951337.0:2876 2014-08-11 18:42:43.933265 2014-08-13 17:38:02.797106 7fe9bae887a0 20 read_log 193305'16882 (0'0) promote 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by osd.4.0:53 2014-08-11 18:42:47.435949 2014-08-13 17:38:02.797118 7fe9bae887a0 20 read_log 193305'16883 (193305'16882) modify 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.951337.0:3838 2014-08-11 18:42:47.428679 2014-08-13 17:38:02.797130 7fe9bae887a0 20 read_log 193305'16884 (0'0) promote e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by osd.4.0:55 2014-08-11 18:42:48.232307 2014-08-13 17:38:02.797142 7fe9bae887a0 20 read_log 193305'16885 (193305'16884) modify e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.951337.0:4057 2014-08-11 18:42:48.224447 2014-08-13 17:38:02.797159 7fe9bae887a0 20 read_log 193305'16886 (0'0) promote 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by osd.4.0:57 2014-08-11 18:42:48.436760 2014-08-13 17:38:02.797182 7fe9bae887a0 20 read_log 193305'16887 (193305'16886) modify 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.951337.0:4113 2014-08-11 18:42:48.429005 2014-08-13 17:38:02.797197 7fe9bae887a0 20 read_log 193305'16888 (0'0) promote 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by osd.4.0:113 2014-08-11 18:42:59.535690 2014-08-13 17:38:02.797209 7fe9bae887a0 20 read_log 193305'16889 (193305'16888) modify 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.951337.0:7135 2014-08-11 18:42:59.525590 2014-08-13 17:38:02.797223 7fe9bae887a0 20 read_log 193305'16890 (0'0) promote 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by osd.4.0:129 2014-08-11 18:43:02.222602 2014-08-13 17:38:02.797235 7fe9bae887a0 20 read_log 193305'16891 (193305'16890) modify 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.951337.0:7879 2014-08-11 18:43:02.214874 2014-08-13 17:38:02.797247 7fe9bae887a0 20 read_log 193305'16892 (0'0) promote b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by osd.4.0:155 2014-08-11 18:43:08.551901 2014-08-13 17:38:02.797262 7fe9bae887a0 20 read_log 193305'16893 (193305'16892) modify b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.951337.0:9523 2014-08-11 18:43:08.544108 2014-08-13 17:38:02.797274 7fe9bae887a0 20 read_log 193305'16894 (0'0) promote 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by osd.4.0:165 2014-08-11 18:43:10.482916 2014-08-13 17:38:02.797286 7fe9bae887a0 20 read_log 193305'16895 (193305'16894) modify 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.951337.0:10052 2014-08-11 18:43:10.475665 2014-08-13 17:38:02.797298 7fe9bae887a0 20 read_log 193305'16896 (0'0) promote 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by osd.4.0:173 2014-08-11 18:43:13.503107 2014-08-13 17:38:02.797310 7fe9bae887a0 20 read_log 193305'16897 (193305'16896) modify 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.951337.0:10865 2014-08-11 18:43:13.494945 2014-08-13 17:38:02.797322 7fe9bae887a0 20 read_log 193305'16898 (0'0) promote 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by osd.4.0:195 2014-08-11 18:43:16.495178 2014-08-13 17:38:02.797334 7fe9bae887a0 20 read_log 193305'16899 (193305'16898) modify 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.951337.0:11619 2014-08-11 18:43:16.487933 2014-08-13 17:38:02.797346 7fe9bae887a0 20 read_log 193305'16900 (0'0) promote 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by osd.4.0:219 2014-08-11 18:43:23.568978 2014-08-13 17:38:02.797358 7fe9bae887a0 20 read_log 193305'16901 (193305'16900) modify 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.951337.0:13502 2014-08-11 18:43:23.562646 2014-08-13 17:38:02.797370 7fe9bae887a0 20 read_log 193305'16902 (0'0) promote 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by osd.4.0:233 2014-08-11 18:43:26.822692 2014-08-13 17:38:02.797382 7fe9bae887a0 20 read_log 193305'16903 (193305'16902) modify 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.951337.0:14378 2014-08-11 18:43:26.815012 2014-08-13 17:38:02.797394 7fe9bae887a0 20 read_log 193305'16904 (0'0) promote 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by osd.4.0:239 2014-08-11 18:43:29.291999 2014-08-13 17:38:02.797406 7fe9bae887a0 20 read_log 193305'16905 (193305'16904) modify 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.951337.0:15096 2014-08-11 18:43:29.284821 2014-08-13 17:38:02.797417 7fe9bae887a0 20 read_log 193305'16906 (0'0) promote 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by osd.4.0:247 2014-08-11 18:43:30.263105 2014-08-13 17:38:02.797439 7fe9bae887a0 20 read_log 193305'16907 (193305'16906) modify 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.951337.0:15381 2014-08-11 18:43:30.254664 2014-08-13 17:38:02.797452 7fe9bae887a0 20 read_log 193305'16908 (0'0) promote baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by osd.4.0:263 2014-08-11 18:43:33.548506 2014-08-13 17:38:02.797464 7fe9bae887a0 20 read_log 193305'16909 (193305'16908) modify baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.951337.0:16311 2014-08-11 18:43:33.540497 2014-08-13 17:38:02.797498 7fe9bae887a0 20 read_log 193305'16910 (0'0) promote 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by osd.4.0:289 2014-08-11 18:43:36.563396 2014-08-13 17:38:02.797511 7fe9bae887a0 20 read_log 193305'16911 (193305'16910) modify 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.951337.0:17174 2014-08-11 18:43:36.555372 2014-08-13 17:38:02.797523 7fe9bae887a0 20 read_log 193305'16912 (0'0) promote 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by osd.4.0:291 2014-08-11 18:43:36.792137 2014-08-13 17:38:02.797535 7fe9bae887a0 20 read_log 193305'16913 (193305'16912) modify 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.951337.0:17244 2014-08-11 18:43:36.785395 2014-08-13 17:38:02.797547 7fe9bae887a0 20 read_log 193305'16914 (0'0) promote ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by osd.4.0:293 2014-08-11 18:43:37.104486 2014-08-13 17:38:02.797560 7fe9bae887a0 20 read_log 193305'16915 (193305'16914) modify ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.951337.0:17339 2014-08-11 18:43:37.097077 2014-08-13 17:38:02.797572 7fe9bae887a0 20 read_log 193305'16916 (0'0) promote 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by osd.4.0:305 2014-08-11 18:43:39.599624 2014-08-13 17:38:02.797584 7fe9bae887a0 20 read_log 193305'16917 (193305'16916) modify 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.951337.0:18066 2014-08-11 18:43:39.589107 2014-08-13 17:38:02.797596 7fe9bae887a0 20 read_log 193305'16918 (0'0) promote e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by osd.4.0:327 2014-08-11 18:43:43.705573 2014-08-13 17:38:02.797609 7fe9bae887a0 20 read_log 193305'16919 (193305'16918) modify e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.951337.0:19195 2014-08-11 18:43:43.697602 2014-08-13 17:38:02.797621 7fe9bae887a0 20 read_log 193305'16920 (0'0) promote bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by osd.4.0:329 2014-08-11 18:43:44.037601 2014-08-13 17:38:02.797633 7fe9bae887a0 20 read_log 193305'16921 (193305'16920) modify bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.951337.0:19274 2014-08-11 18:43:44.030086 2014-08-13 17:38:02.797645 7fe9bae887a0 20 read_log 193305'16922 (0'0) promote 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by osd.4.0:357 2014-08-11 18:43:52.546066 2014-08-13 17:38:02.797657 7fe9bae887a0 20 read_log 193305'16923 (193305'16922) modify 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.951337.0:21661 2014-08-11 18:43:52.539035 2014-08-13 17:38:02.797669 7fe9bae887a0 20 read_log 193305'16924 (0'0) promote 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by osd.4.0:361 2014-08-11 18:43:53.172007 2014-08-13 17:38:02.797681 7fe9bae887a0 20 read_log 193305'16925 (193305'16924) modify 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.951337.0:21824 2014-08-11 18:43:53.162001 2014-08-13 17:38:02.797693 7fe9bae887a0 20 read_log 193305'16926 (0'0) promote 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by osd.4.0:375 2014-08-11 18:43:55.695314 2014-08-13 17:38:02.797713 7fe9bae887a0 20 read_log 193305'16927 (193305'16926) modify 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.951337.0:22509 2014-08-11 18:43:55.686776 2014-08-13 17:38:02.797726 7fe9bae887a0 20 read_log 193305'16928 (0'0) promote 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by osd.4.0:391 2014-08-11 18:43:57.788696 2014-08-13 17:38:02.797738 7fe9bae887a0 20 read_log 193305'16929 (193305'16928) modify 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.951337.0:23100 2014-08-11 18:43:57.781052 2014-08-13 17:38:02.797750 7fe9bae887a0 20 read_log 193305'16930 (0'0) promote bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by osd.4.0:403 2014-08-11 18:44:02.034981 2014-08-13 17:38:02.797762 7fe9bae887a0 20 read_log 193305'16931 (193305'16930) modify bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.951337.0:24275 2014-08-11 18:44:02.026404 2014-08-13 17:38:02.797775 7fe9bae887a0 20 read_log 193305'16932 (0'0) promote 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by osd.4.0:409 2014-08-11 18:44:04.369233 2014-08-13 17:38:02.797787 7fe9bae887a0 20 read_log 193305'16933 (193305'16932) modify 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.951337.0:24915 2014-08-11 18:44:04.360674 2014-08-13 17:38:02.797801 7fe9bae887a0 20 read_log 193305'16934 (0'0) promote 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by osd.4.0:425 2014-08-11 18:44:08.564579 2014-08-13 17:38:02.797813 7fe9bae887a0 20 read_log 193305'16935 (193305'16934) modify 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.951337.0:26042 2014-08-11 18:44:08.556988 2014-08-13 17:38:02.797825 7fe9bae887a0 20 read_log 193305'16936 (0'0) promote 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by osd.4.0:431 2014-08-11 18:44:09.511400 2014-08-13 17:38:02.797837 7fe9bae887a0 20 read_log 193305'16937 (193305'16936) modify 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.951337.0:26307 2014-08-11 18:44:09.503337 2014-08-13 17:38:02.797850 7fe9bae887a0 20 read_log 193305'16938 (0'0) promote bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by osd.4.0:441 2014-08-11 18:44:11.183938 2014-08-13 17:38:02.797862 7fe9bae887a0 20 read_log 193305'16939 (193305'16938) modify bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.951337.0:26776 2014-08-11 18:44:11.173117 2014-08-13 17:38:02.797874 7fe9bae887a0 20 read_log 193305'16940 (0'0) promote 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by osd.4.0:443 2014-08-11 18:44:11.368213 2014-08-13 17:38:02.797886 7fe9bae887a0 20 read_log 193305'16941 (193305'16940) modify 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.951337.0:26828 2014-08-11 18:44:11.359412 2014-08-13 17:38:02.797898 7fe9bae887a0 20 read_log 193305'16942 (0'0) promote 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by osd.4.0:445 2014-08-11 18:44:11.396123 2014-08-13 17:38:02.797910 7fe9bae887a0 20 read_log 193305'16943 (193305'16942) modify 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.951337.0:26836 2014-08-11 18:44:11.387380 2014-08-13 17:38:02.797923 7fe9bae887a0 20 read_log 193305'16944 (0'0) promote b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by osd.4.0:497 2014-08-11 18:44:23.872355 2014-08-13 17:38:02.797934 7fe9bae887a0 20 read_log 193305'16945 (193305'16944) modify b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.951337.0:30216 2014-08-11 18:44:23.863648 2014-08-13 17:38:02.797946 7fe9bae887a0 20 read_log 193305'16946 (0'0) promote 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by osd.4.0:503 2014-08-11 18:44:26.080734 2014-08-13 17:38:02.797965 7fe9bae887a0 20 read_log 193305'16947 (193305'16946) modify 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.951337.0:30791 2014-08-11 18:44:26.071039 2014-08-13 17:38:02.797978 7fe9bae887a0 20 read_log 193305'16948 (0'0) promote 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by osd.4.0:533 2014-08-11 18:44:30.697342 2014-08-13 17:38:02.797990 7fe9bae887a0 20 read_log 193305'16949 (193305'16948) modify 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.951337.0:32125 2014-08-11 18:44:30.689994 2014-08-13 17:38:02.798002 7fe9bae887a0 20 read_log 193305'16950 (0'0) promote be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by osd.4.0:535 2014-08-11 18:44:30.731453 2014-08-13 17:38:02.798015 7fe9bae887a0 20 read_log 193305'16951 (193305'16950) modify be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.951337.0:32135 2014-08-11 18:44:30.724267 2014-08-13 17:38:02.798026 7fe9bae887a0 20 read_log 193305'16952 (0'0) promote 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by osd.4.0:539 2014-08-11 18:44:31.034869 2014-08-13 17:38:02.798038 7fe9bae887a0 20 read_log 193305'16953 (193305'16952) modify 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.951337.0:32216 2014-08-11 18:44:31.026260 2014-08-13 17:38:02.798051 7fe9bae887a0 20 read_log 193305'16954 (193305'16877) modify 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.951337.0:34490 2014-08-11 18:44:35.027593 2014-08-13 17:38:02.798063 7fe9bae887a0 20 read_log 193305'16955 (193305'16879) modify da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.951337.0:34747 2014-08-11 18:44:35.166935 2014-08-13 17:38:02.798075 7fe9bae887a0 20 read_log 193305'16956 (193305'16881) modify 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.951337.0:35587 2014-08-11 18:44:35.669508 2014-08-13 17:38:02.798087 7fe9bae887a0 20 read_log 193305'16957 (193305'16883) modify 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.951337.0:36549 2014-08-11 18:44:36.318001 2014-08-13 17:38:02.798099 7fe9bae887a0 20 read_log 193305'16958 (193305'16885) modify e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.951337.0:36768 2014-08-11 18:44:36.535112 2014-08-13 17:38:02.798114 7fe9bae887a0 20 read_log 193305'16959 (193305'16887) modify 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.951337.0:36824 2014-08-11 18:44:36.557272 2014-08-13 17:38:02.798127 7fe9bae887a0 20 read_log 193305'16960 (193305'16889) modify 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.951337.0:39846 2014-08-11 18:44:38.205272 2014-08-13 17:38:02.798139 7fe9bae887a0 20 read_log 193305'16961 (193305'16891) modify 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.951337.0:40590 2014-08-11 18:44:38.456609 2014-08-13 17:38:02.798158 7fe9bae887a0 20 read_log 193305'16962 (193305'16893) modify b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.951337.0:42234 2014-08-11 18:44:39.159082 2014-08-13 17:38:02.798173 7fe9bae887a0 20 read_log 193305'16963 (193305'16895) modify 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.951337.0:42763 2014-08-11 18:44:39.371984 2014-08-13 17:38:02.798186 7fe9bae887a0 20 read_log 193305'16964 (193305'16897) modify 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.951337.0:43576 2014-08-11 18:44:39.631631 2014-08-13 17:38:02.798198 7fe9bae887a0 20 read_log 193305'16965 (193305'16899) modify 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.951337.0:44330 2014-08-11 18:44:39.853658 2014-08-13 17:38:02.798210 7fe9bae887a0 20 read_log 193305'16966 (193305'16901) modify 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.951337.0:46213 2014-08-11 18:44:40.421654 2014-08-13 17:38:02.798248 7fe9bae887a0 20 read_log 193305'16967 (193305'16903) modify 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.951337.0:47089 2014-08-11 18:44:40.671382 2014-08-13 17:38:02.798265 7fe9bae887a0 20 read_log 193305'16968 (193305'16905) modify 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.951337.0:47807 2014-08-11 18:44:40.870799 2014-08-13 17:38:02.798277 7fe9bae887a0 20 read_log 193305'16969 (193305'16907) modify 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.951337.0:48092 2014-08-11 18:44:40.950773 2014-08-13 17:38:02.798290 7fe9bae887a0 20 read_log 193305'16970 (193305'16909) modify baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.951337.0:49022 2014-08-11 18:44:41.228766 2014-08-13 17:38:02.798304 7fe9bae887a0 20 read_log 193305'16971 (193305'16911) modify 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.951337.0:49885 2014-08-11 18:44:41.506848 2014-08-13 17:38:02.798316 7fe9bae887a0 20 read_log 193305'16972 (193305'16913) modify 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.951337.0:49955 2014-08-11 18:44:41.525178 2014-08-13 17:38:02.798328 7fe9bae887a0 20 read_log 193305'16973 (193305'16915) modify ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.951337.0:50050 2014-08-11 18:44:41.551453 2014-08-13 17:38:02.798340 7fe9bae887a0 20 read_log 193305'16974 (193305'16917) modify 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.951337.0:50777 2014-08-11 18:44:41.758112 2014-08-13 17:38:02.798352 7fe9bae887a0 20 read_log 193305'16975 (193305'16919) modify e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.951337.0:51906 2014-08-11 18:44:42.083576 2014-08-13 17:38:02.798368 7fe9bae887a0 20 read_log 193305'16976 (193305'16921) modify bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.951337.0:51985 2014-08-11 18:44:42.102828 2014-08-13 17:38:02.798380 7fe9bae887a0 20 read_log 193305'16977 (193305'16923) modify 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.951337.0:54372 2014-08-11 18:44:42.813466 2014-08-13 17:38:02.798392 7fe9bae887a0 20 read_log 193305'16978 (193305'16925) modify 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.951337.0:54535 2014-08-11 18:44:42.866356 2014-08-13 17:38:02.798404 7fe9bae887a0 20 read_log 193305'16979 (193305'16927) modify 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.951337.0:55220 2014-08-11 18:44:43.060423 2014-08-13 17:38:02.798417 7fe9bae887a0 20 read_log 193305'16980 (193305'16929) modify 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.951337.0:55811 2014-08-11 18:44:43.238051 2014-08-13 17:38:02.798429 7fe9bae887a0 20 read_log 193305'16981 (193305'16931) modify bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.951337.0:56986 2014-08-11 18:44:43.593001 2014-08-13 17:38:02.798441 7fe9bae887a0 20 read_log 193305'16982 (193305'16933) modify 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.951337.0:57626 2014-08-11 18:44:43.811499 2014-08-13 17:38:02.798453 7fe9bae887a0 20 read_log 193305'16983 (193305'16935) modify 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.951337.0:58753 2014-08-11 18:44:44.132441 2014-08-13 17:38:02.798465 7fe9bae887a0 20 read_log 193305'16984 (193305'16937) modify 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.951337.0:59018 2014-08-11 18:44:44.211540 2014-08-13 17:38:02.798477 7fe9bae887a0 20 read_log 193305'16985 (193305'16939) modify bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.951337.0:59487 2014-08-11 18:44:44.340653 2014-08-13 17:38:02.798489 7fe9bae887a0 20 read_log 193305'16986 (193305'16941) modify 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.951337.0:59539 2014-08-11 18:44:44.356882 2014-08-13 17:38:02.798508 7fe9bae887a0 20 read_log 193305'16987 (193305'16943) modify 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.951337.0:59547 2014-08-11 18:44:44.358266 2014-08-13 17:38:02.798521 7fe9bae887a0 20 read_log 193305'16988 (193305'16945) modify b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.951337.0:62927 2014-08-11 18:44:45.364342 2014-08-13 17:38:02.798533 7fe9bae887a0 20 read_log 193305'16989 (193305'16947) modify 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.951337.0:63502 2014-08-11 18:44:45.526590 2014-08-13 17:38:02.798545 7fe9bae887a0 20 read_log 193305'16990 (193305'16949) modify 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.951337.0:64836 2014-08-11 18:44:45.935299 2014-08-13 17:38:02.798557 7fe9bae887a0 20 read_log 193305'16991 (193305'16951) modify be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.951337.0:64846 2014-08-11 18:44:45.936177 2014-08-13 17:38:02.798569 7fe9bae887a0 20 read_log 193305'16992 (193305'16953) modify 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.951337.0:64927 2014-08-11 18:44:45.961220 2014-08-13 17:38:02.798584 7fe9bae887a0 20 read_log 194215'16993 (194215'16993) modify 9b/hit_set_24.9b_archive_2014-08-12 15:32:25.864657_2014-08-12 15:38:31.718183/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:31.718441 2014-08-13 17:38:02.798597 7fe9bae887a0 20 read_log 194215'16994 (193305'16874) delete 9b/hit_set_24.9b_archive_2014-08-11 18:37:04.904270_2014-08-11 18:42:39.916705/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:31.718441 2014-08-13 17:38:02.798609 7fe9bae887a0 20 read_log 194215'16995 (192888'16811) clean ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by osd.4.0:1218 2014-08-12 15:38:31.723310 2014-08-13 17:38:02.798621 7fe9bae887a0 20 read_log 194215'16996 (194215'16995) delete ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.2174595.0:31352 0.000000 2014-08-13 17:38:02.798632 7fe9bae887a0 20 read_log 194215'16997 (193305'16959) clean 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by osd.4.0:1220 2014-08-12 15:38:31.765829 2014-08-13 17:38:02.798644 7fe9bae887a0 20 read_log 194215'16998 (194215'16997) delete 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.2174595.0:31354 0.000000 2014-08-13 17:38:02.798655 7fe9bae887a0 20 read_log 194215'16999 (193305'16956) clean 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by osd.4.0:1222 2014-08-12 15:38:31.774738 2014-08-13 17:38:02.798667 7fe9bae887a0 20 read_log 194215'17000 (194215'16999) delete 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.2174595.0:31356 0.000000 2014-08-13 17:38:02.798678 7fe9bae887a0 20 read_log 194215'17001 (192888'16664) clean 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by osd.4.0:1224 2014-08-12 15:38:31.783784 2014-08-13 17:38:02.798690 7fe9bae887a0 20 read_log 194215'17002 (194215'17001) delete 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.2174595.0:31358 0.000000 2014-08-13 17:38:02.798701 7fe9bae887a0 20 read_log 194215'17003 (192888'16662) clean 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by osd.4.0:1226 2014-08-12 15:38:31.791960 2014-08-13 17:38:02.798713 7fe9bae887a0 20 read_log 194215'17004 (194215'17003) delete 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.2174595.0:31360 0.000000 2014-08-13 17:38:02.798723 7fe9bae887a0 20 read_log 194215'17005 (193305'16961) clean 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by osd.4.0:1228 2014-08-12 15:38:31.801209 2014-08-13 17:38:02.798742 7fe9bae887a0 20 read_log 194215'17006 (194215'17005) delete 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.2174595.0:31362 0.000000 2014-08-13 17:38:02.798753 7fe9bae887a0 20 read_log 194215'17007 (192888'16862) clean efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by osd.4.0:1230 2014-08-12 15:38:31.826654 2014-08-13 17:38:02.798766 7fe9bae887a0 20 read_log 194215'17008 (194215'17007) delete efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.2174595.0:31364 0.000000 2014-08-13 17:38:02.798776 7fe9bae887a0 20 read_log 194215'17009 (192888'16839) clean e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by osd.4.0:1232 2014-08-12 15:38:31.836138 2014-08-13 17:38:02.798788 7fe9bae887a0 20 read_log 194215'17010 (194215'17009) delete e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.2174595.0:31366 0.000000 2014-08-13 17:38:02.798799 7fe9bae887a0 20 read_log 194215'17011 (193305'16985) clean bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by osd.4.0:1234 2014-08-12 15:38:31.844546 2014-08-13 17:38:02.798811 7fe9bae887a0 20 read_log 194215'17012 (194215'17011) delete bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.2174595.0:31368 0.000000 2014-08-13 17:38:02.798821 7fe9bae887a0 20 read_log 194215'17013 (192888'16648) clean def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by osd.4.0:1236 2014-08-12 15:38:31.852863 2014-08-13 17:38:02.798833 7fe9bae887a0 20 read_log 194215'17014 (194215'17013) delete def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.2174595.0:31370 0.000000 2014-08-13 17:38:02.798844 7fe9bae887a0 20 read_log 194215'17015 (193305'16957) clean 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by osd.4.0:1238 2014-08-12 15:38:31.861378 2014-08-13 17:38:02.798856 7fe9bae887a0 20 read_log 194215'17016 (194215'17015) delete 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.2174595.0:31372 0.000000 2014-08-13 17:38:02.798866 7fe9bae887a0 20 read_log 194215'17017 (192888'16655) clean 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by osd.4.0:1240 2014-08-12 15:38:31.869387 2014-08-13 17:38:02.798878 7fe9bae887a0 20 read_log 194215'17018 (194215'17017) delete 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.2174595.0:31374 0.000000 2014-08-13 17:38:02.798889 7fe9bae887a0 20 read_log 194215'17019 (193305'16958) clean e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by osd.4.0:1242 2014-08-12 15:38:31.877987 2014-08-13 17:38:02.798901 7fe9bae887a0 20 read_log 194215'17020 (194215'17019) delete e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.2174595.0:31376 0.000000 2014-08-13 17:38:02.798911 7fe9bae887a0 20 read_log 194215'17021 (193305'16982) clean 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by osd.4.0:1244 2014-08-12 15:38:31.886898 2014-08-13 17:38:02.798923 7fe9bae887a0 20 read_log 194215'17022 (194215'17021) delete 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.2174595.0:31378 0.000000 2014-08-13 17:38:02.798934 7fe9bae887a0 20 read_log 194215'17023 (192888'16840) clean 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by osd.4.0:1246 2014-08-12 15:38:31.895022 2014-08-13 17:38:02.798946 7fe9bae887a0 20 read_log 194215'17024 (194215'17023) delete 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.2174595.0:31380 0.000000 2014-08-13 17:38:02.798956 7fe9bae887a0 20 read_log 194215'17025 (192888'16679) clean bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by osd.4.0:1248 2014-08-12 15:38:31.915279 2014-08-13 17:38:02.798977 7fe9bae887a0 20 read_log 194215'17026 (194215'17025) delete bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.2174595.0:31382 0.000000 2014-08-13 17:38:02.799008 7fe9bae887a0 20 read_log 194215'17027 (192888'16676) clean 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by osd.4.0:1250 2014-08-12 15:38:31.923291 2014-08-13 17:38:02.799020 7fe9bae887a0 20 read_log 194215'17028 (194215'17027) delete 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.2174595.0:31384 0.000000 2014-08-13 17:38:02.799031 7fe9bae887a0 20 read_log 194215'17029 (192888'16834) clean cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by osd.4.0:1252 2014-08-12 15:38:31.930907 2014-08-13 17:38:02.799043 7fe9bae887a0 20 read_log 194215'17030 (194215'17029) delete cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.2174595.0:31386 0.000000 2014-08-13 17:38:02.799054 7fe9bae887a0 20 read_log 194215'17031 (193305'16955) clean da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by osd.4.0:1254 2014-08-12 15:38:31.940216 2014-08-13 17:38:02.799066 7fe9bae887a0 20 read_log 194215'17032 (194215'17031) delete da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.2174595.0:31388 0.000000 2014-08-13 17:38:02.799077 7fe9bae887a0 20 read_log 194215'17033 (193305'16980) clean 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by osd.4.0:1256 2014-08-12 15:38:31.948960 2014-08-13 17:38:02.799089 7fe9bae887a0 20 read_log 194215'17034 (194215'17033) delete 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.2174595.0:31390 0.000000 2014-08-13 17:38:02.799099 7fe9bae887a0 20 read_log 194215'17035 (192888'16848) clean ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by osd.4.0:1258 2014-08-12 15:38:31.957415 2014-08-13 17:38:02.799111 7fe9bae887a0 20 read_log 194215'17036 (194215'17035) delete ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.2174595.0:31392 0.000000 2014-08-13 17:38:02.799121 7fe9bae887a0 20 read_log 194215'17037 (193305'16974) clean 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by osd.4.0:1260 2014-08-12 15:38:31.965692 2014-08-13 17:38:02.799133 7fe9bae887a0 20 read_log 194215'17038 (194215'17037) delete 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.2174595.0:31394 0.000000 2014-08-13 17:38:02.799144 7fe9bae887a0 20 read_log 194215'17039 (192888'16851) clean 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by osd.4.0:1262 2014-08-12 15:38:31.975226 2014-08-13 17:38:02.799168 7fe9bae887a0 20 read_log 194215'17040 (194215'17039) delete 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.2174595.0:31396 0.000000 2014-08-13 17:38:02.799180 7fe9bae887a0 20 read_log 194215'17041 (193305'16970) clean baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by osd.4.0:1264 2014-08-12 15:38:31.982937 2014-08-13 17:38:02.799192 7fe9bae887a0 20 read_log 194215'17042 (194215'17041) delete baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.2174595.0:31398 0.000000 2014-08-13 17:38:02.799203 7fe9bae887a0 20 read_log 194215'17043 (192888'16649) clean cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by osd.4.0:1266 2014-08-12 15:38:31.991780 2014-08-13 17:38:02.799215 7fe9bae887a0 20 read_log 194215'17044 (194215'17043) delete cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.2174595.0:31400 0.000000 2014-08-13 17:38:02.799226 7fe9bae887a0 20 read_log 194215'17045 (192888'16814) clean bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by osd.4.0:1268 2014-08-12 15:38:32.014012 2014-08-13 17:38:02.799245 7fe9bae887a0 20 read_log 194215'17046 (194215'17045) delete bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.2174595.0:31402 0.000000 2014-08-13 17:38:02.799256 7fe9bae887a0 20 read_log 194215'17047 (192888'16635) clean 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by osd.4.0:1270 2014-08-12 15:38:32.030280 2014-08-13 17:38:02.799269 7fe9bae887a0 20 read_log 194215'17048 (194215'17047) delete 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.2174595.0:31404 0.000000 2014-08-13 17:38:02.799279 7fe9bae887a0 20 read_log 194215'17049 (193305'16966) clean 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by osd.4.0:1272 2014-08-12 15:38:32.038918 2014-08-13 17:38:02.799291 7fe9bae887a0 20 read_log 194215'17050 (194215'17049) delete 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.2174595.0:31406 0.000000 2014-08-13 17:38:02.799302 7fe9bae887a0 20 read_log 194215'17051 (193305'16988) clean b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by osd.4.0:1274 2014-08-12 15:38:32.047364 2014-08-13 17:38:02.799314 7fe9bae887a0 20 read_log 194215'17052 (194215'17051) delete b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.2174595.0:31408 0.000000 2014-08-13 17:38:02.799324 7fe9bae887a0 20 read_log 194215'17053 (192888'16860) clean 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by osd.4.0:1276 2014-08-12 15:38:32.055413 2014-08-13 17:38:02.799337 7fe9bae887a0 20 read_log 194215'17054 (194215'17053) delete 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.2174595.0:31410 0.000000 2014-08-13 17:38:02.799347 7fe9bae887a0 20 read_log 194215'17055 (192888'16849) clean bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by osd.4.0:1278 2014-08-12 15:38:32.063029 2014-08-13 17:38:02.799359 7fe9bae887a0 20 read_log 194215'17056 (194215'17055) delete bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.2174595.0:31412 0.000000 2014-08-13 17:38:02.799370 7fe9bae887a0 20 read_log 194215'17057 (192888'16855) clean 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by osd.4.0:1280 2014-08-12 15:38:32.070571 2014-08-13 17:38:02.799382 7fe9bae887a0 20 read_log 194215'17058 (194215'17057) delete 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.2174595.0:31414 0.000000 2014-08-13 17:38:02.799392 7fe9bae887a0 20 read_log 194215'17059 (192888'16823) clean 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by osd.4.0:1282 2014-08-12 15:38:32.078405 2014-08-13 17:38:02.799404 7fe9bae887a0 20 read_log 194215'17060 (194215'17059) delete 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.2174595.0:31416 0.000000 2014-08-13 17:38:02.799415 7fe9bae887a0 20 read_log 194215'17061 (193305'16977) clean 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by osd.4.0:1284 2014-08-12 15:38:32.085833 2014-08-13 17:38:02.799427 7fe9bae887a0 20 read_log 194215'17062 (194215'17061) delete 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.2174595.0:31418 0.000000 2014-08-13 17:38:02.799437 7fe9bae887a0 20 read_log 194215'17063 (192888'16867) clean c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by osd.4.0:1286 2014-08-12 15:38:32.094246 2014-08-13 17:38:02.799449 7fe9bae887a0 20 read_log 194215'17064 (194215'17063) delete c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.2174595.0:31420 0.000000 2014-08-13 17:38:02.799460 7fe9bae887a0 20 read_log 194215'17065 (192888'16667) clean e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by osd.4.0:1288 2014-08-12 15:38:32.101534 2014-08-13 17:38:02.799479 7fe9bae887a0 20 read_log 194215'17066 (194215'17065) delete e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.2174595.0:31422 0.000000 2014-08-13 17:38:02.799492 7fe9bae887a0 20 read_log 194215'17067 (193305'16986) clean 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by osd.4.0:1290 2014-08-12 15:38:32.109966 2014-08-13 17:38:02.799504 7fe9bae887a0 20 read_log 194215'17068 (194215'17067) delete 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.2174595.0:31424 0.000000 2014-08-13 17:38:02.799515 7fe9bae887a0 20 read_log 194215'17069 (192888'16831) clean 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by osd.4.0:1292 2014-08-12 15:38:32.118180 2014-08-13 17:38:02.799526 7fe9bae887a0 20 read_log 194215'17070 (194215'17069) delete 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.2174595.0:31426 0.000000 2014-08-13 17:38:02.799537 7fe9bae887a0 20 read_log 194215'17071 (192888'16636) clean 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by osd.4.0:1294 2014-08-12 15:38:32.126295 2014-08-13 17:38:02.799549 7fe9bae887a0 20 read_log 194215'17072 (194215'17071) delete 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.2174595.0:31428 0.000000 2014-08-13 17:38:02.799560 7fe9bae887a0 20 read_log 194215'17073 (192888'16637) clean a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by osd.4.0:1296 2014-08-12 15:38:32.135833 2014-08-13 17:38:02.799572 7fe9bae887a0 20 read_log 194215'17074 (194215'17073) delete a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.2174595.0:31430 0.000000 2014-08-13 17:38:02.799582 7fe9bae887a0 20 read_log 194215'17075 (193305'16981) clean bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by osd.4.0:1298 2014-08-12 15:38:32.144611 2014-08-13 17:38:02.799594 7fe9bae887a0 20 read_log 194215'17076 (194215'17075) delete bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.2174595.0:31432 0.000000 2014-08-13 17:38:02.799605 7fe9bae887a0 20 read_log 194215'17077 (193305'16992) clean 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by osd.4.0:1300 2014-08-12 15:38:32.164229 2014-08-13 17:38:02.799617 7fe9bae887a0 20 read_log 194215'17078 (194215'17077) delete 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.2174595.0:31434 0.000000 2014-08-13 17:38:02.799627 7fe9bae887a0 20 read_log 194215'17079 (193305'16965) clean 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by osd.4.0:1302 2014-08-12 15:38:32.172728 2014-08-13 17:38:02.799641 7fe9bae887a0 20 read_log 194215'17080 (194215'17079) delete 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.2174595.0:31436 0.000000 2014-08-13 17:38:02.799651 7fe9bae887a0 20 read_log 194215'17081 (192888'16623) clean ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by osd.4.0:1304 2014-08-12 15:38:32.180392 2014-08-13 17:38:02.799663 7fe9bae887a0 20 read_log 194215'17082 (194215'17081) delete ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.2174595.0:31438 0.000000 2014-08-13 17:38:02.799674 7fe9bae887a0 20 read_log 194215'17083 (192888'16856) clean 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by osd.4.0:1306 2014-08-12 15:38:32.189187 2014-08-13 17:38:02.799686 7fe9bae887a0 20 read_log 194215'17084 (194215'17083) delete 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.2174595.0:31440 0.000000 2014-08-13 17:38:02.799696 7fe9bae887a0 20 read_log 194215'17085 (192888'16866) clean 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by osd.4.0:1308 2014-08-12 15:38:32.197039 2014-08-13 17:38:02.799717 7fe9bae887a0 20 read_log 194215'17086 (194215'17085) delete 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.2174595.0:31442 0.000000 2014-08-13 17:38:02.799729 7fe9bae887a0 20 read_log 194215'17087 (193305'16962) clean b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by osd.4.0:1310 2014-08-12 15:38:32.204839 2014-08-13 17:38:02.799741 7fe9bae887a0 20 read_log 194215'17088 (194215'17087) delete b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.2174595.0:31444 0.000000 2014-08-13 17:38:02.799751 7fe9bae887a0 20 read_log 194215'17089 (193305'16991) clean be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by osd.4.0:1312 2014-08-12 15:38:32.212337 2014-08-13 17:38:02.799763 7fe9bae887a0 20 read_log 194215'17090 (194215'17089) delete be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.2174595.0:31446 0.000000 2014-08-13 17:38:02.799774 7fe9bae887a0 20 read_log 194215'17091 (192888'16629) clean 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by osd.4.0:1314 2014-08-12 15:38:32.221426 2014-08-13 17:38:02.799786 7fe9bae887a0 20 read_log 194215'17092 (194215'17091) delete 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.2174595.0:31448 0.000000 2014-08-13 17:38:02.799797 7fe9bae887a0 20 read_log 194215'17093 (192888'16847) clean 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by osd.4.0:1316 2014-08-12 15:38:32.229452 2014-08-13 17:38:02.799809 7fe9bae887a0 20 read_log 194215'17094 (194215'17093) delete 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.2174595.0:31450 0.000000 2014-08-13 17:38:02.799819 7fe9bae887a0 20 read_log 194215'17095 (192888'16619) clean 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by osd.4.0:1318 2014-08-12 15:38:32.238435 2014-08-13 17:38:02.799831 7fe9bae887a0 20 read_log 194215'17096 (194215'17095) delete 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.2174595.0:31452 0.000000 2014-08-13 17:38:02.799841 7fe9bae887a0 20 read_log 194215'17097 (192888'16833) clean 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by osd.4.0:1320 2014-08-12 15:38:32.246224 2014-08-13 17:38:02.799853 7fe9bae887a0 20 read_log 194215'17098 (194215'17097) delete 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.2174595.0:31454 0.000000 2014-08-13 17:38:02.799863 7fe9bae887a0 20 read_log 194215'17099 (192888'16627) clean 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by osd.4.0:1322 2014-08-12 15:38:32.254634 2014-08-13 17:38:02.799876 7fe9bae887a0 20 read_log 194215'17100 (194215'17099) delete 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.2174595.0:31456 0.000000 2014-08-13 17:38:02.799886 7fe9bae887a0 20 read_log 194215'17101 (193305'16987) clean 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by osd.4.0:1324 2014-08-12 15:38:32.261454 2014-08-13 17:38:02.799898 7fe9bae887a0 20 read_log 194215'17102 (194215'17101) delete 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.2174595.0:31458 0.000000 2014-08-13 17:38:02.799909 7fe9bae887a0 20 read_log 194215'17103 (192888'16871) clean e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by osd.4.0:1326 2014-08-12 15:38:32.268904 2014-08-13 17:38:02.799921 7fe9bae887a0 20 read_log 194215'17104 (194215'17103) delete e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.2174595.0:31460 0.000000 2014-08-13 17:38:02.799931 7fe9bae887a0 20 read_log 194215'17105 (192888'16657) clean fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by osd.4.0:1328 2014-08-12 15:38:32.285781 2014-08-13 17:38:02.799966 7fe9bae887a0 20 read_log 194215'17106 (194215'17105) delete fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.2174595.0:31462 0.000000 2014-08-13 17:38:02.799978 7fe9bae887a0 20 read_log 194215'17107 (193305'16963) clean 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by osd.4.0:1330 2014-08-12 15:38:32.294127 2014-08-13 17:38:02.799991 7fe9bae887a0 20 read_log 194215'17108 (194215'17107) delete 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.2174595.0:31464 0.000000 2014-08-13 17:38:02.800002 7fe9bae887a0 20 read_log 194215'17109 (193305'16990) clean 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by osd.4.0:1332 2014-08-12 15:38:32.301704 2014-08-13 17:38:02.800014 7fe9bae887a0 20 read_log 194215'17110 (194215'17109) delete 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.2174595.0:31466 0.000000 2014-08-13 17:38:02.800024 7fe9bae887a0 20 read_log 194215'17111 (193305'16964) clean 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by osd.4.0:1334 2014-08-12 15:38:32.309668 2014-08-13 17:38:02.800036 7fe9bae887a0 20 read_log 194215'17112 (194215'17111) delete 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.2174595.0:31468 0.000000 2014-08-13 17:38:02.800047 7fe9bae887a0 20 read_log 194215'17113 (193305'16983) clean 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by osd.4.0:1336 2014-08-12 15:38:32.316656 2014-08-13 17:38:02.800059 7fe9bae887a0 20 read_log 194215'17114 (194215'17113) delete 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.2174595.0:31470 0.000000 2014-08-13 17:38:02.800069 7fe9bae887a0 20 read_log 194215'17115 (192888'16825) clean 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by osd.4.0:1338 2014-08-12 15:38:32.324276 2014-08-13 17:38:02.800081 7fe9bae887a0 20 read_log 194215'17116 (194215'17115) delete 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.2174595.0:31472 0.000000 2014-08-13 17:38:02.800092 7fe9bae887a0 20 read_log 194215'17117 (193305'16967) clean 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by osd.4.0:1340 2014-08-12 15:38:32.331882 2014-08-13 17:38:02.800103 7fe9bae887a0 20 read_log 194215'17118 (194215'17117) delete 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.2174595.0:31474 0.000000 2014-08-13 17:38:02.800168 7fe9bae887a0 20 read_log 194215'17119 (192888'16646) clean 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by osd.4.0:1342 2014-08-12 15:38:32.340753 2014-08-13 17:38:02.800198 7fe9bae887a0 20 read_log 194215'17120 (194215'17119) delete 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.2174595.0:31476 0.000000 2014-08-13 17:38:02.800210 7fe9bae887a0 20 read_log 194215'17121 (192888'16624) clean 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by osd.4.0:1344 2014-08-12 15:38:32.349085 2014-08-13 17:38:02.800222 7fe9bae887a0 20 read_log 194215'17122 (194215'17121) delete 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.2174595.0:31478 0.000000 2014-08-13 17:38:02.800233 7fe9bae887a0 20 read_log 194215'17123 (193305'16989) clean 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by osd.4.0:1346 2014-08-12 15:38:32.356894 2014-08-13 17:38:02.800245 7fe9bae887a0 20 read_log 194215'17124 (194215'17123) delete 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.2174595.0:31480 0.000000 2014-08-13 17:38:02.800256 7fe9bae887a0 20 read_log 194215'17125 (192888'16682) clean 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by osd.4.0:1348 2014-08-12 15:38:32.397317 2014-08-13 17:38:02.800275 7fe9bae887a0 20 read_log 194215'17126 (194215'17125) delete 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.2174595.0:31482 0.000000 2014-08-13 17:38:02.800287 7fe9bae887a0 20 read_log 194215'17127 (193305'16972) clean 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by osd.4.0:1350 2014-08-12 15:38:32.405053 2014-08-13 17:38:02.800299 7fe9bae887a0 20 read_log 194215'17128 (194215'17127) delete 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.2174595.0:31484 0.000000 2014-08-13 17:38:02.800309 7fe9bae887a0 20 read_log 194215'17129 (193305'16954) clean 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by osd.4.0:1352 2014-08-12 15:38:32.413815 2014-08-13 17:38:02.800321 7fe9bae887a0 20 read_log 194215'17130 (194215'17129) delete 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.2174595.0:31486 0.000000 2014-08-13 17:38:02.800331 7fe9bae887a0 20 read_log 194215'17131 (192888'16822) clean ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by osd.4.0:1354 2014-08-12 15:38:32.422603 2014-08-13 17:38:02.800343 7fe9bae887a0 20 read_log 194215'17132 (194215'17131) delete ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.2174595.0:31488 0.000000 2014-08-13 17:38:02.800354 7fe9bae887a0 20 read_log 194215'17133 (192888'16813) clean 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by osd.4.0:1356 2014-08-12 15:38:32.430715 2014-08-13 17:38:02.800366 7fe9bae887a0 20 read_log 194215'17134 (194215'17133) delete 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.2174595.0:31490 0.000000 2014-08-13 17:38:02.800376 7fe9bae887a0 20 read_log 194215'17135 (192888'16829) clean 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by osd.4.0:1358 2014-08-12 15:38:32.438272 2014-08-13 17:38:02.800388 7fe9bae887a0 20 read_log 194215'17136 (194215'17135) delete 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.2174595.0:31492 0.000000 2014-08-13 17:38:02.800398 7fe9bae887a0 20 read_log 194215'17137 (192888'16621) clean 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by osd.4.0:1360 2014-08-12 15:38:32.463860 2014-08-13 17:38:02.800410 7fe9bae887a0 20 read_log 194215'17138 (194215'17137) delete 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.2174595.0:31494 0.000000 2014-08-13 17:38:02.800421 7fe9bae887a0 20 read_log 194215'17139 (192888'16670) clean 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by osd.4.0:1362 2014-08-12 15:38:32.472343 2014-08-13 17:38:02.800433 7fe9bae887a0 20 read_log 194215'17140 (194215'17139) delete 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.2174595.0:31496 0.000000 2014-08-13 17:38:02.800445 7fe9bae887a0 20 read_log 194215'17141 (193305'16978) clean 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by osd.4.0:1364 2014-08-12 15:38:32.479893 2014-08-13 17:38:02.800457 7fe9bae887a0 20 read_log 194215'17142 (194215'17141) delete 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.2174595.0:31498 0.000000 2014-08-13 17:38:02.800468 7fe9bae887a0 20 read_log 194215'17143 (193305'16969) clean 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by osd.4.0:1366 2014-08-12 15:38:32.488572 2014-08-13 17:38:02.800479 7fe9bae887a0 20 read_log 194215'17144 (194215'17143) delete 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.2174595.0:31500 0.000000 2014-08-13 17:38:02.800490 7fe9bae887a0 20 read_log 194215'17145 (193305'16973) clean ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by osd.4.0:1368 2014-08-12 15:38:32.496764 2014-08-13 17:38:02.800511 7fe9bae887a0 20 read_log 194215'17146 (194215'17145) delete ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.2174595.0:31502 0.000000 2014-08-13 17:38:02.800522 7fe9bae887a0 20 read_log 194215'17147 (192888'16622) clean ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by osd.4.0:1370 2014-08-12 15:38:32.504695 2014-08-13 17:38:02.800534 7fe9bae887a0 20 read_log 194215'17148 (194215'17147) delete ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.2174595.0:31504 0.000000 2014-08-13 17:38:02.800545 7fe9bae887a0 20 read_log 194215'17149 (193305'16976) clean bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by osd.4.0:1372 2014-08-12 15:38:32.511906 2014-08-13 17:38:02.800557 7fe9bae887a0 20 read_log 194215'17150 (194215'17149) delete bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.2174595.0:31506 0.000000 2014-08-13 17:38:02.800567 7fe9bae887a0 20 read_log 194215'17151 (193305'16971) clean 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by osd.4.0:1374 2014-08-12 15:38:32.520071 2014-08-13 17:38:02.800579 7fe9bae887a0 20 read_log 194215'17152 (194215'17151) delete 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.2174595.0:31508 0.000000 2014-08-13 17:38:02.800589 7fe9bae887a0 20 read_log 194215'17153 (193305'16975) clean e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by osd.4.0:1376 2014-08-12 15:38:32.529123 2014-08-13 17:38:02.800601 7fe9bae887a0 20 read_log 194215'17154 (194215'17153) delete e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.2174595.0:31510 0.000000 2014-08-13 17:38:02.800612 7fe9bae887a0 20 read_log 194215'17155 (193305'16968) clean 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by osd.4.0:1378 2014-08-12 15:38:32.538705 2014-08-13 17:38:02.800623 7fe9bae887a0 20 read_log 194215'17156 (194215'17155) delete 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.2174595.0:31512 0.000000 2014-08-13 17:38:02.800634 7fe9bae887a0 20 read_log 194215'17157 (192888'16653) clean 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by osd.4.0:1380 2014-08-12 15:38:32.546135 2014-08-13 17:38:02.800646 7fe9bae887a0 20 read_log 194215'17158 (194215'17157) delete 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.2174595.0:31514 0.000000 2014-08-13 17:38:02.800656 7fe9bae887a0 20 read_log 194215'17159 (192888'16821) clean bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by osd.4.0:1382 2014-08-12 15:38:32.555002 2014-08-13 17:38:02.800668 7fe9bae887a0 20 read_log 194215'17160 (194215'17159) delete bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.2174595.0:31516 0.000000 2014-08-13 17:38:02.800678 7fe9bae887a0 20 read_log 194215'17161 (192888'16818) clean e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by osd.4.0:1384 2014-08-12 15:38:32.563564 2014-08-13 17:38:02.800690 7fe9bae887a0 20 read_log 194215'17162 (194215'17161) delete e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.2174595.0:31518 0.000000 2014-08-13 17:38:02.800700 7fe9bae887a0 20 read_log 194215'17163 (192888'16870) clean 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by osd.4.0:1386 2014-08-12 15:38:32.571644 2014-08-13 17:38:02.800712 7fe9bae887a0 20 read_log 194215'17164 (194215'17163) delete 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.2174595.0:31520 0.000000 2014-08-13 17:38:02.800723 7fe9bae887a0 20 read_log 194215'17165 (192888'16668) clean 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by osd.4.0:1388 2014-08-12 15:38:32.578978 2014-08-13 17:38:02.800741 7fe9bae887a0 20 read_log 194215'17166 (194215'17165) delete 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.2174595.0:31522 0.000000 2014-08-13 17:38:02.800752 7fe9bae887a0 20 read_log 194215'17167 (193305'16984) clean 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by osd.4.0:1390 2014-08-12 15:38:32.586331 2014-08-13 17:38:02.800768 7fe9bae887a0 20 read_log 194215'17168 (194215'17167) delete 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.2174595.0:31524 0.000000 2014-08-13 17:38:02.800778 7fe9bae887a0 20 read_log 194215'17169 (192888'16660) clean e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by osd.4.0:1392 2014-08-12 15:38:32.594193 2014-08-13 17:38:02.800791 7fe9bae887a0 20 read_log 194215'17170 (194215'17169) delete e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.2174595.0:31526 0.000000 2014-08-13 17:38:02.800801 7fe9bae887a0 20 read_log 194215'17171 (192888'16647) clean 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by osd.4.0:1394 2014-08-12 15:38:32.601523 2014-08-13 17:38:02.800813 7fe9bae887a0 20 read_log 194215'17172 (194215'17171) delete 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.2174595.0:31528 0.000000 2014-08-13 17:38:02.800824 7fe9bae887a0 20 read_log 194215'17173 (192888'16628) clean 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by osd.4.0:1396 2014-08-12 15:38:32.609453 2014-08-13 17:38:02.800836 7fe9bae887a0 20 read_log 194215'17174 (194215'17173) delete 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.2174595.0:31530 0.000000 2014-08-13 17:38:02.800846 7fe9bae887a0 20 read_log 194215'17175 (193305'16960) clean 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by osd.4.0:1398 2014-08-12 15:38:32.621336 2014-08-13 17:38:02.800858 7fe9bae887a0 20 read_log 194215'17176 (194215'17175) delete 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.2174595.0:31532 0.000000 2014-08-13 17:38:02.800869 7fe9bae887a0 20 read_log 194215'17177 (193305'16979) clean 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by osd.4.0:1400 2014-08-12 15:38:32.629679 2014-08-13 17:38:02.800880 7fe9bae887a0 20 read_log 194215'17178 (194215'17177) delete 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.2174595.0:31534 0.000000 2014-08-13 17:38:02.800891 7fe9bae887a0 20 read_log 194215'17179 (194215'17179) modify 9b/hit_set_24.9b_archive_2014-08-12 15:38:31.718448_2014-08-12 17:24:40.634730/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.635537 2014-08-13 17:38:02.800903 7fe9bae887a0 20 read_log 194215'17180 (194215'16993) delete 9b/hit_set_24.9b_archive_2014-08-12 15:32:25.864657_2014-08-12 15:38:31.718183/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.635537 2014-08-13 17:38:02.800915 7fe9bae887a0 20 read_log 194215'17181 (0'0) promote fb5d6c9b/benchmark_data_storage0111-ib.pocstor_34628_object188/head//24 by osd.4.0:5227 2014-08-12 17:24:40.640704 2014-08-13 17:38:02.800928 7fe9bae887a0 20 read_log 194215'17182 (194215'17181) modify fb5d6c9b/benchmark_data_storage0111-ib.pocstor_34628_object188/head//24 by client.2193759.0:189 2014-08-12 17:24:40.626295 2014-08-13 17:38:02.800940 7fe9bae887a0 20 read_log 194215'17183 (0'0) promote e686789b/benchmark_data_storage0111-ib.pocstor_34628_object818/head//24 by osd.4.0:5243 2014-08-12 17:24:43.085924 2014-08-13 17:38:02.800951 7fe9bae887a0 20 read_log 194215'17184 (194215'17183) modify e686789b/benchmark_data_storage0111-ib.pocstor_34628_object818/head//24 by client.2193759.0:819 2014-08-12 17:24:43.078725 2014-08-13 17:38:02.800971 7fe9bae887a0 20 read_log 194370'17185 (0'0) promote 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by osd.365.0:3664 2014-08-12 20:30:14.749730 2014-08-13 17:38:02.800984 7fe9bae887a0 20 read_log 194370'17186 (194370'17185) modify 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by client.1508531.0:157 2014-08-12 20:30:14.740139 2014-08-13 17:38:02.800996 7fe9bae887a0 20 read_log 194370'17187 (0'0) promote 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by osd.365.0:3676 2014-08-12 20:30:18.893137 2014-08-13 17:38:02.801008 7fe9bae887a0 20 read_log 194370'17188 (194370'17187) modify 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by client.1508531.0:1199 2014-08-12 20:30:18.882929 2014-08-13 17:38:02.801020 7fe9bae887a0 20 read_log 194370'17189 (0'0) promote b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by osd.365.0:3712 2014-08-12 20:30:27.743527 2014-08-13 17:38:02.801032 7fe9bae887a0 20 read_log 194370'17190 (194370'17189) modify b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by client.1508531.0:3590 2014-08-12 20:30:27.734195 2014-08-13 17:38:02.801044 7fe9bae887a0 20 read_log 194370'17191 (0'0) promote c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by osd.365.0:3738 2014-08-12 20:30:32.110372 2014-08-13 17:38:02.801056 7fe9bae887a0 20 read_log 194370'17192 (194370'17191) modify c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by client.1508531.0:4723 2014-08-12 20:30:32.103291 2014-08-13 17:38:02.801068 7fe9bae887a0 20 read_log 194370'17193 (0'0) promote e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by osd.365.0:3762 2014-08-12 20:30:39.243150 2014-08-13 17:38:02.801080 7fe9bae887a0 20 read_log 194370'17194 (194370'17193) modify e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by client.1508531.0:6348 2014-08-12 20:30:39.236432 2014-08-13 17:38:02.801092 7fe9bae887a0 20 read_log 194370'17195 (0'0) promote 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by osd.365.0:3792 2014-08-12 20:30:49.478892 2014-08-13 17:38:02.801103 7fe9bae887a0 20 read_log 194370'17196 (194370'17195) modify 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by client.1508531.0:8355 2014-08-12 20:30:49.471690 2014-08-13 17:38:02.801115 7fe9bae887a0 20 read_log 194370'17197 (0'0) promote 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by osd.365.0:3796 2014-08-12 20:30:50.324880 2014-08-13 17:38:02.801127 7fe9bae887a0 20 read_log 194370'17198 (194370'17197) modify 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by client.1508531.0:8511 2014-08-12 20:30:50.316538 2014-08-13 17:38:02.801139 7fe9bae887a0 20 read_log 194370'17199 (0'0) promote 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by osd.365.0:3820 2014-08-12 20:30:54.681070 2014-08-13 17:38:02.801151 7fe9bae887a0 20 read_log 194370'17200 (194370'17199) modify 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by client.1508531.0:9228 2014-08-12 20:30:54.674352 2014-08-13 17:38:02.801169 7fe9bae887a0 20 read_log 194370'17201 (0'0) promote efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by osd.365.0:3838 2014-08-12 20:30:58.700578 2014-08-13 17:38:02.801181 7fe9bae887a0 20 read_log 194370'17202 (194370'17201) modify efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by client.1508531.0:9902 2014-08-12 20:30:58.691845 2014-08-13 17:38:02.801193 7fe9bae887a0 20 read_log 194370'17203 (0'0) promote 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by osd.365.0:3856 2014-08-12 20:31:03.577724 2014-08-13 17:38:02.801205 7fe9bae887a0 20 read_log 194370'17204 (194370'17203) modify 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by client.1508531.0:10711 2014-08-12 20:31:03.568643 2014-08-13 17:38:02.801227 7fe9bae887a0 20 read_log 194370'17205 (0'0) promote 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by osd.365.0:3862 2014-08-12 20:31:05.678584 2014-08-13 17:38:02.801240 7fe9bae887a0 20 read_log 194370'17206 (194370'17205) modify 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by client.1508531.0:11067 2014-08-12 20:31:05.669937 2014-08-13 17:38:02.801252 7fe9bae887a0 20 read_log 194370'17207 (0'0) promote ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by osd.365.0:3870 2014-08-12 20:31:06.442885 2014-08-13 17:38:02.801264 7fe9bae887a0 20 read_log 194370'17208 (194370'17207) modify ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by client.1508531.0:11194 2014-08-12 20:31:06.434067 2014-08-13 17:38:02.801276 7fe9bae887a0 20 read_log 194370'17209 (0'0) promote 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by osd.365.0:3884 2014-08-12 20:31:15.819769 2014-08-13 17:38:02.801288 7fe9bae887a0 20 read_log 194370'17210 (194370'17209) modify 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by client.1508531.0:12615 2014-08-12 20:31:15.811325 2014-08-13 17:38:02.801300 7fe9bae887a0 20 read_log 194370'17211 (0'0) promote ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by osd.365.0:3900 2014-08-12 20:31:21.990397 2014-08-13 17:38:02.801312 7fe9bae887a0 20 read_log 194370'17212 (194370'17211) modify ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by client.1508531.0:13272 2014-08-12 20:31:21.981379 2014-08-13 17:38:02.801343 7fe9bae887a0 20 read_log 194370'17213 (0'0) promote d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by osd.365.0:3914 2014-08-12 20:31:28.961810 2014-08-13 17:38:02.801356 7fe9bae887a0 20 read_log 194370'17214 (194370'17213) modify d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by client.1508531.0:13881 2014-08-12 20:31:28.953568 2014-08-13 17:38:02.801372 7fe9bae887a0 20 read_log 194370'17215 (0'0) promote b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by osd.365.0:3916 2014-08-12 20:31:32.051201 2014-08-13 17:38:02.801384 7fe9bae887a0 20 read_log 194370'17216 (194370'17215) modify b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by client.1508531.0:13994 2014-08-12 20:31:32.042820 2014-08-13 17:38:02.801413 7fe9bae887a0 20 read_log 194377'17217 (0'0) promote d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by osd.291.0:3899 2014-08-12 20:33:00.827904 2014-08-13 17:38:02.801427 7fe9bae887a0 20 read_log 194377'17218 (194377'17217) modify d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by client.2231190.0:872 2014-08-12 20:33:00.818408 2014-08-13 17:38:02.801440 7fe9bae887a0 20 read_log 194377'17219 (0'0) promote b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by osd.291.0:3905 2014-08-12 20:33:02.345171 2014-08-13 17:38:02.801452 7fe9bae887a0 20 read_log 194377'17220 (194377'17219) modify b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by client.2231190.0:1273 2014-08-12 20:33:02.338556 2014-08-13 17:38:02.801464 7fe9bae887a0 20 read_log 194377'17221 (0'0) promote 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by osd.291.0:3907 2014-08-12 20:33:02.642836 2014-08-13 17:38:02.801476 7fe9bae887a0 20 read_log 194377'17222 (194377'17221) modify 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by client.2231190.0:1353 2014-08-12 20:33:02.636079 2014-08-13 17:38:02.801488 7fe9bae887a0 20 read_log 194377'17223 (0'0) promote 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by osd.291.0:3911 2014-08-12 20:33:03.151302 2014-08-13 17:38:02.801508 7fe9bae887a0 20 read_log 194377'17224 (194377'17223) modify 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by client.2231190.0:1487 2014-08-12 20:33:03.144670 2014-08-13 17:38:02.801521 7fe9bae887a0 20 read_log 194377'17225 (0'0) promote b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by osd.291.0:3917 2014-08-12 20:33:05.227326 2014-08-13 17:38:02.801533 7fe9bae887a0 20 read_log 194377'17226 (194377'17225) modify b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by client.2231190.0:1996 2014-08-12 20:33:05.220164 2014-08-13 17:38:02.801545 7fe9bae887a0 20 read_log 194377'17227 (0'0) promote 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by osd.291.0:3931 2014-08-12 20:33:07.879263 2014-08-13 17:38:02.801557 7fe9bae887a0 20 read_log 194377'17228 (194377'17227) modify 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by client.2231190.0:2636 2014-08-12 20:33:07.870982 2014-08-13 17:38:02.801569 7fe9bae887a0 20 read_log 194377'17229 (0'0) promote 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by osd.291.0:3937 2014-08-12 20:33:09.638339 2014-08-13 17:38:02.801584 7fe9bae887a0 20 read_log 194377'17230 (194377'17229) modify 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by client.2231190.0:3080 2014-08-12 20:33:09.629893 2014-08-13 17:38:02.801596 7fe9bae887a0 20 read_log 194377'17231 (0'0) promote 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by osd.291.0:3959 2014-08-12 20:33:16.259252 2014-08-13 17:38:02.801608 7fe9bae887a0 20 read_log 194377'17232 (194377'17231) modify 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by client.2231190.0:4379 2014-08-12 20:33:16.251492 2014-08-13 17:38:02.801620 7fe9bae887a0 20 read_log 194377'17233 (0'0) promote ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by osd.291.0:3965 2014-08-12 20:33:16.689810 2014-08-13 17:38:02.801632 7fe9bae887a0 20 read_log 194377'17234 (194377'17233) modify ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by client.2231190.0:4436 2014-08-12 20:33:16.682348 2014-08-13 17:38:02.801644 7fe9bae887a0 20 read_log 194377'17235 (0'0) promote 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by osd.291.0:3973 2014-08-12 20:33:19.532772 2014-08-13 17:38:02.801655 7fe9bae887a0 20 read_log 194377'17236 (194377'17235) modify 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by client.2231190.0:4807 2014-08-12 20:33:19.522292 2014-08-13 17:38:02.801668 7fe9bae887a0 20 read_log 194378'17237 (0'0) promote 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by osd.291.0:3977 2014-08-12 20:33:20.480003 2014-08-13 17:38:02.801680 7fe9bae887a0 20 read_log 194378'17238 (194378'17237) modify 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by client.2231190.0:4866 2014-08-12 20:33:20.470704 2014-08-13 17:38:02.801692 7fe9bae887a0 20 read_log 194380'17239 (0'0) promote f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by osd.291.0:3987 2014-08-12 20:33:27.746925 2014-08-13 17:38:02.801704 7fe9bae887a0 20 read_log 194380'17240 (194380'17239) modify f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by client.2231190.0:5343 2014-08-12 20:33:27.738227 2014-08-13 17:38:02.801716 7fe9bae887a0 20 read_log 194383'17241 (0'0) promote ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by osd.291.0:4007 2014-08-12 20:34:03.135985 2014-08-13 17:38:02.801728 7fe9bae887a0 20 read_log 194383'17242 (194383'17241) modify ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by client.2231190.0:6319 2014-08-12 20:34:03.127803 2014-08-13 17:38:02.801747 7fe9bae887a0 20 read_log 194383'17243 (0'0) promote 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by osd.291.0:4025 2014-08-12 20:34:24.543516 2014-08-13 17:38:02.801760 7fe9bae887a0 20 read_log 194383'17244 (194383'17243) modify 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by client.2231190.0:7346 2014-08-12 20:34:24.535485 2014-08-13 17:38:02.801772 7fe9bae887a0 20 read_log 194416'17245 (194416'17245) modify 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 17:38:02.801785 7fe9bae887a0 20 read_log 194416'17246 (194215'17179) delete 9b/hit_set_24.9b_archive_2014-08-12 15:38:31.718448_2014-08-12 17:24:40.634730/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 17:38:02.801797 7fe9bae887a0 20 read_log 194416'17247 (0'0) promote 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by osd.365.0:4305 2014-08-13 15:28:33.802418 2014-08-13 17:38:02.801809 7fe9bae887a0 20 read_log 194416'17248 (194416'17247) modify 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by client.2441958.0:826 2014-08-13 15:28:33.793568 2014-08-13 17:38:02.801821 7fe9bae887a0 20 read_log 194416'17249 (0'0) promote 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by osd.365.0:4355 2014-08-13 15:28:41.298245 2014-08-13 17:38:02.801833 7fe9bae887a0 20 read_log 194416'17250 (194416'17249) modify 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by client.2441958.0:2791 2014-08-13 15:28:41.288527 2014-08-13 17:38:02.801845 7fe9bae887a0 20 read_log 194416'17251 (0'0) promote a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by osd.365.0:4367 2014-08-13 15:28:43.919462 2014-08-13 17:38:02.801857 7fe9bae887a0 20 read_log 194416'17252 (194416'17251) modify a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by client.2441958.0:3487 2014-08-13 15:28:43.908733 2014-08-13 17:38:02.801868 7fe9bae887a0 20 read_log 194416'17253 (0'0) promote 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by osd.365.0:4407 2014-08-13 15:28:53.741369 2014-08-13 17:38:02.801880 7fe9bae887a0 20 read_log 194416'17254 (194416'17253) modify 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by client.2441958.0:6000 2014-08-13 15:28:53.730275 2014-08-13 17:38:02.801892 7fe9bae887a0 20 read_log 194416'17255 (0'0) promote 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by osd.365.0:4423 2014-08-13 15:28:57.206841 2014-08-13 17:38:02.801904 7fe9bae887a0 20 read_log 194416'17256 (194416'17255) modify 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by client.2441958.0:6845 2014-08-13 15:28:57.199118 2014-08-13 17:38:02.801916 7fe9bae887a0 20 read_log 194416'17257 (0'0) promote b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by osd.365.0:4435 2014-08-13 15:29:00.948687 2014-08-13 17:38:02.801928 7fe9bae887a0 20 read_log 194416'17258 (194416'17257) modify b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by client.2441958.0:7845 2014-08-13 15:29:00.939980 2014-08-13 17:38:02.801940 7fe9bae887a0 20 read_log 194416'17259 (0'0) promote 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by osd.365.0:4439 2014-08-13 15:29:01.158329 2014-08-13 17:38:02.801952 7fe9bae887a0 20 read_log 194416'17260 (194416'17259) modify 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by client.2441958.0:7901 2014-08-13 15:29:01.151151 2014-08-13 17:38:02.801964 7fe9bae887a0 20 read_log 194416'17261 (0'0) promote 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by osd.365.0:4445 2014-08-13 15:29:02.707665 2014-08-13 17:38:02.802002 7fe9bae887a0 20 read_log 194416'17262 (194416'17261) modify 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by client.2441958.0:8310 2014-08-13 15:29:02.701038 2014-08-13 17:38:02.802016 7fe9bae887a0 20 read_log 194416'17263 (0'0) promote a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by osd.365.0:4469 2014-08-13 15:29:06.850953 2014-08-13 17:38:02.802028 7fe9bae887a0 20 read_log 194416'17264 (194416'17263) modify a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by client.2441958.0:9362 2014-08-13 15:29:06.842730 2014-08-13 17:38:02.802040 7fe9bae887a0 20 read_log 194416'17265 (0'0) promote 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by osd.365.0:4475 2014-08-13 15:29:08.380752 2014-08-13 17:38:02.802052 7fe9bae887a0 20 read_log 194416'17266 (194416'17265) modify 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by client.2441958.0:9760 2014-08-13 15:29:08.372051 2014-08-13 17:38:02.802064 7fe9bae887a0 20 read_log 194416'17267 (0'0) promote c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by osd.365.0:4489 2014-08-13 15:29:10.510787 2014-08-13 17:38:02.802076 7fe9bae887a0 20 read_log 194416'17268 (194416'17267) modify c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by client.2441958.0:10262 2014-08-13 15:29:10.502047 2014-08-13 17:38:02.802089 7fe9bae887a0 20 read_log 194416'17269 (0'0) promote f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by osd.365.0:4515 2014-08-13 15:29:16.039284 2014-08-13 17:38:02.802101 7fe9bae887a0 20 read_log 194416'17270 (194416'17269) modify f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by client.2441958.0:11634 2014-08-13 15:29:16.031715 2014-08-13 17:38:02.802113 7fe9bae887a0 20 read_log 194416'17271 (0'0) promote a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by osd.365.0:4523 2014-08-13 15:29:18.417979 2014-08-13 17:38:02.802125 7fe9bae887a0 20 read_log 194416'17272 (194416'17271) modify a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by client.2441958.0:12184 2014-08-13 15:29:18.409753 2014-08-13 17:38:02.802137 7fe9bae887a0 20 read_log 194416'17273 (0'0) promote 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by osd.365.0:4531 2014-08-13 15:29:19.203121 2014-08-13 17:38:02.802149 7fe9bae887a0 20 read_log 194416'17274 (194416'17273) modify 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by client.2441958.0:12371 2014-08-13 15:29:19.195539 2014-08-13 17:38:02.802166 7fe9bae887a0 20 read_log 194416'17275 (0'0) promote d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by osd.365.0:4539 2014-08-13 15:29:20.920212 2014-08-13 17:38:02.802184 7fe9bae887a0 20 read_log 194416'17276 (194416'17275) modify d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by client.2441958.0:12775 2014-08-13 15:29:20.910681 2014-08-13 17:38:02.802196 7fe9bae887a0 20 read_log 194416'17277 (0'0) promote 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by osd.365.0:4541 2014-08-13 15:29:21.060874 2014-08-13 17:38:02.802208 7fe9bae887a0 20 read_log 194416'17278 (194416'17277) modify 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by client.2441958.0:12802 2014-08-13 15:29:21.053764 2014-08-13 17:38:02.802220 7fe9bae887a0 20 read_log 194416'17279 (0'0) promote f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by osd.365.0:4547 2014-08-13 15:29:22.800945 2014-08-13 17:38:02.802232 7fe9bae887a0 20 read_log 194416'17280 (194416'17279) modify f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by client.2441958.0:13196 2014-08-13 15:29:22.791824 2014-08-13 17:38:02.802245 7fe9bae887a0 20 read_log 194416'17281 (0'0) promote fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by osd.365.0:4551 2014-08-13 15:29:23.506297 2014-08-13 17:38:02.802264 7fe9bae887a0 20 read_log 194416'17282 (194416'17281) modify fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by client.2441958.0:13357 2014-08-13 15:29:23.498808 2014-08-13 17:38:02.802280 7fe9bae887a0 20 read_log 194416'17283 (0'0) promote 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by osd.365.0:4625 2014-08-13 15:29:58.203831 2014-08-13 17:38:02.802292 7fe9bae887a0 20 read_log 194416'17284 (194416'17283) modify 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by client.2441958.0:17771 2014-08-13 15:29:58.194289 2014-08-13 17:38:02.802304 7fe9bae887a0 20 read_log 194422'17285 (0'0) promote 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by osd.365.0:4651 2014-08-13 15:32:15.458090 2014-08-13 17:38:02.802316 7fe9bae887a0 20 read_log 194422'17286 (194422'17285) modify 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by client.1148203.0:121 2014-08-13 15:32:15.450657 2014-08-13 17:38:02.802328 7fe9bae887a0 20 read_log 194422'17287 (0'0) promote 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by osd.365.0:4659 2014-08-13 15:32:18.970767 2014-08-13 17:38:02.802341 7fe9bae887a0 20 read_log 194422'17288 (194422'17287) modify 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by client.1148203.0:913 2014-08-13 15:32:18.960547 2014-08-13 17:38:02.802353 7fe9bae887a0 20 read_log 194422'17289 (0'0) promote 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by osd.365.0:4663 2014-08-13 15:32:19.444995 2014-08-13 17:38:02.802365 7fe9bae887a0 20 read_log 194422'17290 (194422'17289) modify 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by client.1148203.0:1039 2014-08-13 15:32:19.437462 2014-08-13 17:38:02.802377 7fe9bae887a0 20 read_log 194422'17291 (0'0) promote 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by osd.365.0:4673 2014-08-13 15:32:20.973955 2014-08-13 17:38:02.802389 7fe9bae887a0 20 read_log 194422'17292 (194422'17291) modify 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by client.1148203.0:1427 2014-08-13 15:32:20.965536 2014-08-13 17:38:02.802401 7fe9bae887a0 20 read_log 194422'17293 (0'0) promote f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by osd.365.0:4707 2014-08-13 15:32:29.249344 2014-08-13 17:38:02.802412 7fe9bae887a0 20 read_log 194422'17294 (194422'17293) modify f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by client.1148203.0:3540 2014-08-13 15:32:29.240088 2014-08-13 17:38:02.802424 7fe9bae887a0 20 read_log 194422'17295 (0'0) promote 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by osd.365.0:4737 2014-08-13 15:32:35.191923 2014-08-13 17:38:02.802436 7fe9bae887a0 20 read_log 194422'17296 (194422'17295) modify 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by client.1148203.0:5077 2014-08-13 15:32:35.182165 2014-08-13 17:38:02.802448 7fe9bae887a0 20 read_log 194422'17297 (0'0) promote 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by osd.365.0:4743 2014-08-13 15:32:37.108092 2014-08-13 17:38:02.802460 7fe9bae887a0 20 read_log 194422'17298 (194422'17297) modify 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by client.1148203.0:5453 2014-08-13 15:32:37.097117 2014-08-13 17:38:02.802472 7fe9bae887a0 20 read_log 194422'17299 (0'0) promote 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by osd.365.0:4835 2014-08-13 15:33:05.261973 2014-08-13 17:38:02.802484 7fe9bae887a0 20 read_log 194422'17300 (194422'17299) modify 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by client.1148203.0:9580 2014-08-13 15:33:05.253527 2014-08-13 17:38:02.802496 7fe9bae887a0 20 read_log 194422'17301 (0'0) promote a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by osd.365.0:4839 2014-08-13 15:33:09.485388 2014-08-13 17:38:02.802515 7fe9bae887a0 20 read_log 194422'17302 (194422'17301) modify a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by client.1148203.0:9917 2014-08-13 15:33:09.476174 2014-08-13 17:38:02.802531 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.802905 7fe9bae887a0 10 read_log checking for missing items over interval (194370'17185,194422'17302] 2014-08-13 17:38:02.802952 7fe9bae887a0 15 read_log missing 194422'17302 (194422'17301) modify a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by client.1148203.0:9917 2014-08-13 15:33:09.476174 2014-08-13 17:38:02.802993 7fe9bae887a0 15 read_log missing 194422'17300 (194422'17299) modify 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by client.1148203.0:9580 2014-08-13 15:33:05.253527 2014-08-13 17:38:02.803030 7fe9bae887a0 15 read_log missing 194422'17298 (194422'17297) modify 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by client.1148203.0:5453 2014-08-13 15:32:37.097117 2014-08-13 17:38:02.803067 7fe9bae887a0 15 read_log missing 194422'17296 (194422'17295) modify 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by client.1148203.0:5077 2014-08-13 15:32:35.182165 2014-08-13 17:38:02.803104 7fe9bae887a0 15 read_log missing 194422'17294 (194422'17293) modify f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by client.1148203.0:3540 2014-08-13 15:32:29.240088 2014-08-13 17:38:02.803140 7fe9bae887a0 15 read_log missing 194422'17292 (194422'17291) modify 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by client.1148203.0:1427 2014-08-13 15:32:20.965536 2014-08-13 17:38:02.803187 7fe9bae887a0 15 read_log missing 194422'17290 (194422'17289) modify 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by client.1148203.0:1039 2014-08-13 15:32:19.437462 2014-08-13 17:38:02.803226 7fe9bae887a0 15 read_log missing 194422'17288 (194422'17287) modify 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by client.1148203.0:913 2014-08-13 15:32:18.960547 2014-08-13 17:38:02.803263 7fe9bae887a0 15 read_log missing 194422'17286 (194422'17285) modify 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by client.1148203.0:121 2014-08-13 15:32:15.450657 2014-08-13 17:38:02.803299 7fe9bae887a0 15 read_log missing 194416'17284 (194416'17283) modify 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by client.2441958.0:17771 2014-08-13 15:29:58.194289 2014-08-13 17:38:02.803336 7fe9bae887a0 15 read_log missing 194416'17282 (194416'17281) modify fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by client.2441958.0:13357 2014-08-13 15:29:23.498808 2014-08-13 17:38:02.803372 7fe9bae887a0 15 read_log missing 194416'17280 (194416'17279) modify f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by client.2441958.0:13196 2014-08-13 15:29:22.791824 2014-08-13 17:38:02.803408 7fe9bae887a0 15 read_log missing 194416'17278 (194416'17277) modify 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by client.2441958.0:12802 2014-08-13 15:29:21.053764 2014-08-13 17:38:02.803445 7fe9bae887a0 15 read_log missing 194416'17276 (194416'17275) modify d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by client.2441958.0:12775 2014-08-13 15:29:20.910681 2014-08-13 17:38:02.803481 7fe9bae887a0 15 read_log missing 194416'17274 (194416'17273) modify 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by client.2441958.0:12371 2014-08-13 15:29:19.195539 2014-08-13 17:38:02.803517 7fe9bae887a0 15 read_log missing 194416'17272 (194416'17271) modify a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by client.2441958.0:12184 2014-08-13 15:29:18.409753 2014-08-13 17:38:02.803553 7fe9bae887a0 15 read_log missing 194416'17270 (194416'17269) modify f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by client.2441958.0:11634 2014-08-13 15:29:16.031715 2014-08-13 17:38:02.803588 7fe9bae887a0 15 read_log missing 194416'17268 (194416'17267) modify c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by client.2441958.0:10262 2014-08-13 15:29:10.502047 2014-08-13 17:38:02.803624 7fe9bae887a0 15 read_log missing 194416'17266 (194416'17265) modify 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by client.2441958.0:9760 2014-08-13 15:29:08.372051 2014-08-13 17:38:02.803660 7fe9bae887a0 15 read_log missing 194416'17264 (194416'17263) modify a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by client.2441958.0:9362 2014-08-13 15:29:06.842730 2014-08-13 17:38:02.803696 7fe9bae887a0 15 read_log missing 194416'17262 (194416'17261) modify 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by client.2441958.0:8310 2014-08-13 15:29:02.701038 2014-08-13 17:38:02.803732 7fe9bae887a0 15 read_log missing 194416'17260 (194416'17259) modify 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by client.2441958.0:7901 2014-08-13 15:29:01.151151 2014-08-13 17:38:02.803768 7fe9bae887a0 15 read_log missing 194416'17258 (194416'17257) modify b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by client.2441958.0:7845 2014-08-13 15:29:00.939980 2014-08-13 17:38:02.803804 7fe9bae887a0 15 read_log missing 194416'17256 (194416'17255) modify 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by client.2441958.0:6845 2014-08-13 15:28:57.199118 2014-08-13 17:38:02.803841 7fe9bae887a0 15 read_log missing 194416'17254 (194416'17253) modify 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by client.2441958.0:6000 2014-08-13 15:28:53.730275 2014-08-13 17:38:02.803877 7fe9bae887a0 15 read_log missing 194416'17252 (194416'17251) modify a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by client.2441958.0:3487 2014-08-13 15:28:43.908733 2014-08-13 17:38:02.803913 7fe9bae887a0 15 read_log missing 194416'17250 (194416'17249) modify 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by client.2441958.0:2791 2014-08-13 15:28:41.288527 2014-08-13 17:38:02.803951 7fe9bae887a0 15 read_log missing 194416'17248 (194416'17247) modify 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by client.2441958.0:826 2014-08-13 15:28:33.793568 2014-08-13 17:38:02.803991 7fe9bae887a0 15 read_log missing 194416'17245 (194416'17245) modify 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 17:38:02.804027 7fe9bae887a0 15 read_log missing 194383'17244 (194383'17243) modify 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by client.2231190.0:7346 2014-08-12 20:34:24.535485 2014-08-13 17:38:02.804064 7fe9bae887a0 15 read_log missing 194383'17242 (194383'17241) modify ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by client.2231190.0:6319 2014-08-12 20:34:03.127803 2014-08-13 17:38:02.804100 7fe9bae887a0 15 read_log missing 194380'17240 (194380'17239) modify f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by client.2231190.0:5343 2014-08-12 20:33:27.738227 2014-08-13 17:38:02.804138 7fe9bae887a0 15 read_log missing 194378'17238 (194378'17237) modify 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by client.2231190.0:4866 2014-08-12 20:33:20.470704 2014-08-13 17:38:02.804179 7fe9bae887a0 15 read_log missing 194377'17236 (194377'17235) modify 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by client.2231190.0:4807 2014-08-12 20:33:19.522292 2014-08-13 17:38:02.804216 7fe9bae887a0 15 read_log missing 194377'17234 (194377'17233) modify ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by client.2231190.0:4436 2014-08-12 20:33:16.682348 2014-08-13 17:38:02.804253 7fe9bae887a0 15 read_log missing 194377'17232 (194377'17231) modify 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by client.2231190.0:4379 2014-08-12 20:33:16.251492 2014-08-13 17:38:02.804289 7fe9bae887a0 15 read_log missing 194377'17230 (194377'17229) modify 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by client.2231190.0:3080 2014-08-12 20:33:09.629893 2014-08-13 17:38:02.804326 7fe9bae887a0 15 read_log missing 194377'17228 (194377'17227) modify 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by client.2231190.0:2636 2014-08-12 20:33:07.870982 2014-08-13 17:38:02.804363 7fe9bae887a0 15 read_log missing 194377'17226 (194377'17225) modify b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by client.2231190.0:1996 2014-08-12 20:33:05.220164 2014-08-13 17:38:02.804399 7fe9bae887a0 15 read_log missing 194377'17224 (194377'17223) modify 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by client.2231190.0:1487 2014-08-12 20:33:03.144670 2014-08-13 17:38:02.804436 7fe9bae887a0 15 read_log missing 194377'17222 (194377'17221) modify 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by client.2231190.0:1353 2014-08-12 20:33:02.636079 2014-08-13 17:38:02.804473 7fe9bae887a0 15 read_log missing 194377'17220 (194377'17219) modify b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by client.2231190.0:1273 2014-08-12 20:33:02.338556 2014-08-13 17:38:02.804509 7fe9bae887a0 15 read_log missing 194377'17218 (194377'17217) modify d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by client.2231190.0:872 2014-08-12 20:33:00.818408 2014-08-13 17:38:02.804546 7fe9bae887a0 15 read_log missing 194370'17216 (194370'17215) modify b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by client.1508531.0:13994 2014-08-12 20:31:32.042820 2014-08-13 17:38:02.804583 7fe9bae887a0 15 read_log missing 194370'17214 (194370'17213) modify d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by client.1508531.0:13881 2014-08-12 20:31:28.953568 2014-08-13 17:38:02.804620 7fe9bae887a0 15 read_log missing 194370'17212 (194370'17211) modify ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by client.1508531.0:13272 2014-08-12 20:31:21.981379 2014-08-13 17:38:02.804656 7fe9bae887a0 15 read_log missing 194370'17210 (194370'17209) modify 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by client.1508531.0:12615 2014-08-12 20:31:15.811325 2014-08-13 17:38:02.804693 7fe9bae887a0 15 read_log missing 194370'17208 (194370'17207) modify ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by client.1508531.0:11194 2014-08-12 20:31:06.434067 2014-08-13 17:38:02.804730 7fe9bae887a0 15 read_log missing 194370'17206 (194370'17205) modify 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by client.1508531.0:11067 2014-08-12 20:31:05.669937 2014-08-13 17:38:02.804767 7fe9bae887a0 15 read_log missing 194370'17204 (194370'17203) modify 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by client.1508531.0:10711 2014-08-12 20:31:03.568643 2014-08-13 17:38:02.804804 7fe9bae887a0 15 read_log missing 194370'17202 (194370'17201) modify efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by client.1508531.0:9902 2014-08-12 20:30:58.691845 2014-08-13 17:38:02.804841 7fe9bae887a0 15 read_log missing 194370'17200 (194370'17199) modify 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by client.1508531.0:9228 2014-08-12 20:30:54.674352 2014-08-13 17:38:02.804878 7fe9bae887a0 15 read_log missing 194370'17198 (194370'17197) modify 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by client.1508531.0:8511 2014-08-12 20:30:50.316538 2014-08-13 17:38:02.804914 7fe9bae887a0 15 read_log missing 194370'17196 (194370'17195) modify 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by client.1508531.0:8355 2014-08-12 20:30:49.471690 2014-08-13 17:38:02.804951 7fe9bae887a0 15 read_log missing 194370'17194 (194370'17193) modify e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by client.1508531.0:6348 2014-08-12 20:30:39.236432 2014-08-13 17:38:02.804988 7fe9bae887a0 15 read_log missing 194370'17192 (194370'17191) modify c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by client.1508531.0:4723 2014-08-12 20:30:32.103291 2014-08-13 17:38:02.805024 7fe9bae887a0 15 read_log missing 194370'17190 (194370'17189) modify b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by client.1508531.0:3590 2014-08-12 20:30:27.734195 2014-08-13 17:38:02.805061 7fe9bae887a0 15 read_log missing 194370'17188 (194370'17187) modify 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by client.1508531.0:1199 2014-08-12 20:30:18.882929 2014-08-13 17:38:02.805098 7fe9bae887a0 15 read_log missing 194370'17186 (194370'17185) modify 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by client.1508531.0:157 2014-08-12 20:30:14.740139 2014-08-13 17:38:02.805109 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.805135 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_loaded 2014-08-13 17:38:02.805158 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.012464 0 0.000000 2014-08-13 17:38:02.805171 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 17:38:02.805201 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194477 2014-08-13 17:38:02.805212 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 17:38:02.805222 7fe9bae887a0 10 osd.4 194478 pgid 24.b6 coll 24.b6_head 2014-08-13 17:38:02.805344 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.b6 2014-08-13 17:38:02.805357 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.805398 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.b6(unlocked)] enter Initial 2014-08-13 17:38:02.805409 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.b6(unlocked)] enter NotTrimming 2014-08-13 17:38:02.805536 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.805629 7fe9bae887a0 20 read_log 192834'13334 (192834'12844) clean 55a900b6/benchmark_data_storage0111-ib.pocstor_26659_object933/head//24 by osd.21.0:14814 2014-08-11 15:26:42.582577 2014-08-13 17:38:02.805650 7fe9bae887a0 20 read_log 192834'13335 (192834'12889) clean a64550b6/benchmark_data_storage0111-ib.pocstor_26659_object2499/head//24 by osd.21.0:14815 2014-08-11 15:26:42.593255 2014-08-13 17:38:02.805663 7fe9bae887a0 20 read_log 192834'13336 (192834'12838) clean 4cf370b6/benchmark_data_storage0111-ib.pocstor_26659_object379/head//24 by osd.21.0:14816 2014-08-11 15:26:42.593528 2014-08-13 17:38:02.805676 7fe9bae887a0 20 read_log 192834'13337 (192834'12860) clean ae2280b6/benchmark_data_storage0111-ib.pocstor_26659_object1456/head//24 by osd.21.0:14817 2014-08-11 15:26:42.598633 2014-08-13 17:38:02.805688 7fe9bae887a0 20 read_log 192834'13339 (192834'12840) clean a24c74b6/benchmark_data_storage0111-ib.pocstor_26659_object588/head//24 by osd.21.0:14819 2014-08-11 15:26:42.616367 2014-08-13 17:38:02.805700 7fe9bae887a0 20 read_log 192834'13347 (192834'12881) clean c77ddcb6/benchmark_data_storage0111-ib.pocstor_26659_object1932/head//24 by osd.21.0:14827 2014-08-11 15:26:42.673764 2014-08-13 17:38:02.805712 7fe9bae887a0 20 read_log 192834'13368 (192834'13334) delete 55a900b6/benchmark_data_storage0111-ib.pocstor_26659_object933/head//24 by osd.21.0:14848 2014-08-11 15:26:42.784469 2014-08-13 17:38:02.805724 7fe9bae887a0 20 read_log 192834'13369 (192834'13335) delete a64550b6/benchmark_data_storage0111-ib.pocstor_26659_object2499/head//24 by osd.21.0:14849 2014-08-11 15:26:42.789640 2014-08-13 17:38:02.805736 7fe9bae887a0 20 read_log 192834'13370 (192834'13336) delete 4cf370b6/benchmark_data_storage0111-ib.pocstor_26659_object379/head//24 by osd.21.0:14850 2014-08-11 15:26:42.790095 2014-08-13 17:38:02.805748 7fe9bae887a0 20 read_log 192834'13371 (192834'13337) delete ae2280b6/benchmark_data_storage0111-ib.pocstor_26659_object1456/head//24 by osd.21.0:14851 2014-08-11 15:26:42.790567 2014-08-13 17:38:02.805760 7fe9bae887a0 20 read_log 192834'13373 (192834'13339) delete a24c74b6/benchmark_data_storage0111-ib.pocstor_26659_object588/head//24 by osd.21.0:14853 2014-08-11 15:26:42.810180 2014-08-13 17:38:02.805772 7fe9bae887a0 20 read_log 192834'13380 (192834'13347) delete c77ddcb6/benchmark_data_storage0111-ib.pocstor_26659_object1932/head//24 by osd.21.0:14860 2014-08-11 15:26:42.848554 2014-08-13 17:38:02.805784 7fe9bae887a0 20 read_log 192834'13404 (192834'12986) clean 769644b6/benchmark_data_storage0111-ib.pocstor_26659_object7061/head//24 by osd.21.0:14884 2014-08-11 15:26:48.218285 2014-08-13 17:38:02.805796 7fe9bae887a0 20 read_log 192834'13405 (192834'12941) clean 82cd4b6/benchmark_data_storage0111-ib.pocstor_26659_object5672/head//24 by osd.21.0:14885 2014-08-11 15:26:48.223187 2014-08-13 17:38:02.805808 7fe9bae887a0 20 read_log 192834'13412 (192834'12937) clean 249bd8b6/benchmark_data_storage0111-ib.pocstor_26659_object5569/head//24 by osd.21.0:14892 2014-08-11 15:26:48.260309 2014-08-13 17:38:02.805823 7fe9bae887a0 20 read_log 192834'13420 (192834'12984) clean f4feecb6/benchmark_data_storage0111-ib.pocstor_26659_object7018/head//24 by osd.21.0:14900 2014-08-11 15:26:48.297357 2014-08-13 17:38:02.805835 7fe9bae887a0 20 read_log 192834'13435 (192834'13404) delete 769644b6/benchmark_data_storage0111-ib.pocstor_26659_object7061/head//24 by osd.21.0:14915 2014-08-11 15:26:48.409480 2014-08-13 17:38:02.805847 7fe9bae887a0 20 read_log 192834'13436 (192834'13405) delete 82cd4b6/benchmark_data_storage0111-ib.pocstor_26659_object5672/head//24 by osd.21.0:14916 2014-08-11 15:26:48.410134 2014-08-13 17:38:02.805872 7fe9bae887a0 20 read_log 192834'13443 (192834'13412) delete 249bd8b6/benchmark_data_storage0111-ib.pocstor_26659_object5569/head//24 by osd.21.0:14923 2014-08-11 15:26:48.436869 2014-08-13 17:38:02.805885 7fe9bae887a0 20 read_log 192834'13450 (192834'13420) delete f4feecb6/benchmark_data_storage0111-ib.pocstor_26659_object7018/head//24 by osd.21.0:14930 2014-08-11 15:26:48.459530 2014-08-13 17:38:02.805898 7fe9bae887a0 20 read_log 192834'13461 (0'0) promote d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:14941 2014-08-11 15:26:49.871391 2014-08-13 17:38:02.805910 7fe9bae887a0 20 read_log 192834'13462 (192834'13461) modify d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by client.1883634.0:9192 2014-08-11 15:26:49.862663 2014-08-13 17:38:02.805922 7fe9bae887a0 20 read_log 192834'13467 (0'0) promote a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:14947 2014-08-11 15:26:51.012300 2014-08-13 17:38:02.805934 7fe9bae887a0 20 read_log 192834'13468 (192834'13467) modify a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by client.1883634.0:9481 2014-08-11 15:26:51.004045 2014-08-13 17:38:02.805947 7fe9bae887a0 20 read_log 192834'13492 (192834'13006) clean 47c04b6/benchmark_data_storage0111-ib.pocstor_26659_object8617/head//24 by osd.21.0:14972 2014-08-11 15:26:53.902775 2014-08-13 17:38:02.805960 7fe9bae887a0 20 read_log 192834'13493 (192834'13010) clean 2d1744b6/benchmark_data_storage0111-ib.pocstor_26659_object9010/head//24 by osd.21.0:14973 2014-08-11 15:26:53.903078 2014-08-13 17:38:02.805972 7fe9bae887a0 20 read_log 192834'13494 (192834'13050) clean 499e48b6/benchmark_data_storage0111-ib.pocstor_26659_object10391/head//24 by osd.21.0:14974 2014-08-11 15:26:53.911522 2014-08-13 17:38:02.805984 7fe9bae887a0 20 read_log 192834'13498 (192834'13012) clean ebd77cb6/benchmark_data_storage0111-ib.pocstor_26659_object9086/head//24 by osd.21.0:14978 2014-08-11 15:26:53.927614 2014-08-13 17:38:02.805996 7fe9bae887a0 20 read_log 192834'13509 (192834'13062) clean 99bdfcb6/benchmark_data_storage0111-ib.pocstor_26659_object11367/head//24 by osd.21.0:14989 2014-08-11 15:26:53.988650 2014-08-13 17:38:02.806008 7fe9bae887a0 20 read_log 192834'13519 (192834'13492) delete 47c04b6/benchmark_data_storage0111-ib.pocstor_26659_object8617/head//24 by osd.21.0:14999 2014-08-11 15:26:54.024071 2014-08-13 17:38:02.806021 7fe9bae887a0 20 read_log 192834'13520 (192834'13493) delete 2d1744b6/benchmark_data_storage0111-ib.pocstor_26659_object9010/head//24 by osd.21.0:15000 2014-08-11 15:26:54.027709 2014-08-13 17:38:02.806033 7fe9bae887a0 20 read_log 192834'13521 (192834'13494) delete 499e48b6/benchmark_data_storage0111-ib.pocstor_26659_object10391/head//24 by osd.21.0:15001 2014-08-11 15:26:54.036463 2014-08-13 17:38:02.806045 7fe9bae887a0 20 read_log 192834'13525 (192834'13498) delete ebd77cb6/benchmark_data_storage0111-ib.pocstor_26659_object9086/head//24 by osd.21.0:15005 2014-08-11 15:26:54.048380 2014-08-13 17:38:02.806057 7fe9bae887a0 20 read_log 192834'13526 (192834'13509) delete 99bdfcb6/benchmark_data_storage0111-ib.pocstor_26659_object11367/head//24 by osd.21.0:15006 2014-08-11 15:26:54.051748 2014-08-13 17:38:02.806069 7fe9bae887a0 20 read_log 192834'13560 (192834'13124) clean 69ff30b6/benchmark_data_storage0111-ib.pocstor_26659_object14659/head//24 by osd.21.0:15040 2014-08-11 15:26:59.202307 2014-08-13 17:38:02.806082 7fe9bae887a0 20 read_log 192834'13565 (192834'13116) clean ec4608b6/benchmark_data_storage0111-ib.pocstor_26659_object14361/head//24 by osd.21.0:15045 2014-08-11 15:26:59.222470 2014-08-13 17:38:02.806097 7fe9bae887a0 20 read_log 192834'13566 (192834'13094) clean 44d2c8b6/benchmark_data_storage0111-ib.pocstor_26659_object13186/head//24 by osd.21.0:15046 2014-08-11 15:26:59.225839 2014-08-13 17:38:02.806109 7fe9bae887a0 20 read_log 192834'13569 (192834'13122) clean 8d588cb6/benchmark_data_storage0111-ib.pocstor_26659_object14462/head//24 by osd.21.0:15049 2014-08-11 15:26:59.234425 2014-08-13 17:38:02.806129 7fe9bae887a0 20 read_log 192834'13570 (192834'13108) clean 6868dcb6/benchmark_data_storage0111-ib.pocstor_26659_object14042/head//24 by osd.21.0:15050 2014-08-11 15:26:59.241697 2014-08-13 17:38:02.806143 7fe9bae887a0 20 read_log 192834'13587 (192834'13560) delete 69ff30b6/benchmark_data_storage0111-ib.pocstor_26659_object14659/head//24 by osd.21.0:15067 2014-08-11 15:26:59.321378 2014-08-13 17:38:02.806161 7fe9bae887a0 20 read_log 192834'13594 (192834'13565) delete ec4608b6/benchmark_data_storage0111-ib.pocstor_26659_object14361/head//24 by osd.21.0:15074 2014-08-11 15:26:59.342615 2014-08-13 17:38:02.806176 7fe9bae887a0 20 read_log 192834'13595 (192834'13566) delete 44d2c8b6/benchmark_data_storage0111-ib.pocstor_26659_object13186/head//24 by osd.21.0:15075 2014-08-11 15:26:59.343546 2014-08-13 17:38:02.806188 7fe9bae887a0 20 read_log 192834'13598 (192834'13569) delete 8d588cb6/benchmark_data_storage0111-ib.pocstor_26659_object14462/head//24 by osd.21.0:15078 2014-08-11 15:26:59.355847 2014-08-13 17:38:02.806202 7fe9bae887a0 20 read_log 192834'13599 (192834'13570) delete 6868dcb6/benchmark_data_storage0111-ib.pocstor_26659_object14042/head//24 by osd.21.0:15079 2014-08-11 15:26:59.356493 2014-08-13 17:38:02.806214 7fe9bae887a0 20 read_log 192834'13652 (192834'13169) clean 49f38b6/benchmark_data_storage0111-ib.pocstor_26659_object18009/head//24 by osd.21.0:15132 2014-08-11 15:27:04.685396 2014-08-13 17:38:02.806226 7fe9bae887a0 20 read_log 192834'13653 (192834'13175) clean a6c858b6/benchmark_data_storage0111-ib.pocstor_26659_object18518/head//24 by osd.21.0:15133 2014-08-11 15:27:04.688362 2014-08-13 17:38:02.806240 7fe9bae887a0 20 read_log 192834'13654 (192834'13190) clean bbb98b6/benchmark_data_storage0111-ib.pocstor_26659_object19705/head//24 by osd.21.0:15134 2014-08-11 15:27:04.694200 2014-08-13 17:38:02.806253 7fe9bae887a0 20 read_log 192834'13655 (192834'13138) clean 330d58b6/benchmark_data_storage0111-ib.pocstor_26659_object15824/head//24 by osd.21.0:15135 2014-08-11 15:27:04.694461 2014-08-13 17:38:02.806265 7fe9bae887a0 20 read_log 192834'13657 (192834'13164) clean 635cc8b6/benchmark_data_storage0111-ib.pocstor_26659_object17522/head//24 by osd.21.0:15137 2014-08-11 15:27:04.700445 2014-08-13 17:38:02.806277 7fe9bae887a0 20 read_log 192834'13666 (192834'13172) clean bd143cb6/benchmark_data_storage0111-ib.pocstor_26659_object18307/head//24 by osd.21.0:15146 2014-08-11 15:27:04.734066 2014-08-13 17:38:02.806289 7fe9bae887a0 20 read_log 192834'13707 (192834'13652) delete 49f38b6/benchmark_data_storage0111-ib.pocstor_26659_object18009/head//24 by osd.21.0:15187 2014-08-11 15:27:04.805702 2014-08-13 17:38:02.806300 7fe9bae887a0 20 read_log 192834'13708 (192834'13653) delete a6c858b6/benchmark_data_storage0111-ib.pocstor_26659_object18518/head//24 by osd.21.0:15188 2014-08-11 15:27:04.806033 2014-08-13 17:38:02.806312 7fe9bae887a0 20 read_log 192834'13709 (192834'13655) delete 330d58b6/benchmark_data_storage0111-ib.pocstor_26659_object15824/head//24 by osd.21.0:15189 2014-08-11 15:27:04.808886 2014-08-13 17:38:02.806324 7fe9bae887a0 20 read_log 192834'13710 (192834'13654) delete bbb98b6/benchmark_data_storage0111-ib.pocstor_26659_object19705/head//24 by osd.21.0:15190 2014-08-11 15:27:04.809231 2014-08-13 17:38:02.806336 7fe9bae887a0 20 read_log 192834'13711 (192834'13657) delete 635cc8b6/benchmark_data_storage0111-ib.pocstor_26659_object17522/head//24 by osd.21.0:15191 2014-08-11 15:27:04.809509 2014-08-13 17:38:02.806348 7fe9bae887a0 20 read_log 192834'13722 (192834'13666) delete bd143cb6/benchmark_data_storage0111-ib.pocstor_26659_object18307/head//24 by osd.21.0:15202 2014-08-11 15:27:04.820860 2014-08-13 17:38:02.806360 7fe9bae887a0 20 read_log 192834'13736 (0'0) promote ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15216 2014-08-11 15:27:09.560786 2014-08-13 17:38:02.806372 7fe9bae887a0 20 read_log 192834'13737 (192834'13736) modify ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by client.1883634.0:13429 2014-08-11 15:27:09.550352 2014-08-13 17:38:02.806409 7fe9bae887a0 20 read_log 192834'13761 (0'0) promote 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15240 2014-08-11 15:27:16.424309 2014-08-13 17:38:02.806423 7fe9bae887a0 20 read_log 192834'13762 (192834'13761) modify 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by client.1883634.0:14954 2014-08-11 15:27:16.417123 2014-08-13 17:38:02.806435 7fe9bae887a0 20 read_log 192834'13765 (0'0) promote dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15244 2014-08-11 15:27:16.964951 2014-08-13 17:38:02.806447 7fe9bae887a0 20 read_log 192834'13766 (192834'13765) modify dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by client.1883634.0:15086 2014-08-11 15:27:16.955575 2014-08-13 17:38:02.806460 7fe9bae887a0 20 read_log 192834'13782 (192834'13213) clean 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15261 2014-08-11 15:27:20.153149 2014-08-13 17:38:02.806472 7fe9bae887a0 20 read_log 192834'13798 (192834'13782) delete 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15277 2014-08-11 15:27:25.483564 2014-08-13 17:38:02.806484 7fe9bae887a0 20 read_log 192834'13815 (192834'13241) clean 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15294 2014-08-11 15:27:25.584524 2014-08-13 17:38:02.806496 7fe9bae887a0 20 read_log 192834'13818 (192834'13815) delete 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15297 2014-08-11 15:27:25.590842 2014-08-13 17:38:02.806508 7fe9bae887a0 20 read_log 192834'13830 (0'0) promote 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15309 2014-08-11 15:27:27.554556 2014-08-13 17:38:02.806521 7fe9bae887a0 20 read_log 192834'13831 (192834'13830) modify 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by client.1883634.0:16441 2014-08-11 15:27:27.545577 2014-08-13 17:38:02.806534 7fe9bae887a0 20 read_log 192834'13848 (0'0) promote d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15327 2014-08-11 15:27:30.092782 2014-08-13 17:38:02.806546 7fe9bae887a0 20 read_log 192834'13849 (192834'13848) modify d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by client.1883634.0:16955 2014-08-11 15:27:30.083388 2014-08-13 17:38:02.806558 7fe9bae887a0 20 read_log 192834'13859 (192834'13271) clean 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15338 2014-08-11 15:27:30.895242 2014-08-13 17:38:02.806570 7fe9bae887a0 20 read_log 192834'13869 (192834'13859) delete 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15348 2014-08-11 15:27:30.940519 2014-08-13 17:38:02.806582 7fe9bae887a0 20 read_log 192834'13874 (0'0) promote ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15353 2014-08-11 15:27:34.114581 2014-08-13 17:38:02.806594 7fe9bae887a0 20 read_log 192834'13875 (192834'13874) modify ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by client.1883634.0:17289 2014-08-11 15:27:34.105681 2014-08-13 17:38:02.806606 7fe9bae887a0 20 read_log 192834'13901 (0'0) promote 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:15380 2014-08-11 15:27:40.064768 2014-08-13 17:38:02.806618 7fe9bae887a0 20 read_log 192834'13902 (192834'13901) modify 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by client.1883634.0:17929 2014-08-11 15:27:40.055936 2014-08-13 17:38:02.806630 7fe9bae887a0 20 read_log 192834'13935 (0'0) promote 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:15414 2014-08-11 15:27:49.550899 2014-08-13 17:38:02.806642 7fe9bae887a0 20 read_log 192834'13936 (192834'13935) modify 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by client.1883634.0:19102 2014-08-11 15:27:49.543600 2014-08-13 17:38:02.806665 7fe9bae887a0 20 read_log 192834'13948 (192834'13462) clean d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15427 2014-08-11 15:27:51.826015 2014-08-13 17:38:02.806678 7fe9bae887a0 20 read_log 192834'13949 (192834'13948) delete d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15428 2014-08-11 15:27:51.833700 2014-08-13 17:38:02.806690 7fe9bae887a0 20 read_log 192834'13950 (192834'13468) clean a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15429 2014-08-11 15:27:51.865418 2014-08-13 17:38:02.806702 7fe9bae887a0 20 read_log 192834'13964 (0'0) promote 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:15443 2014-08-11 15:27:55.392401 2014-08-13 17:38:02.806714 7fe9bae887a0 20 read_log 192834'13965 (192834'13964) modify 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by client.1883634.0:20135 2014-08-11 15:27:55.383147 2014-08-13 17:38:02.806728 7fe9bae887a0 20 read_log 192834'13970 (0'0) promote 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:15449 2014-08-11 15:27:56.267530 2014-08-13 17:38:02.806741 7fe9bae887a0 20 read_log 192834'13971 (192834'13970) modify 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by client.1883634.0:20341 2014-08-11 15:27:56.257674 2014-08-13 17:38:02.806753 7fe9bae887a0 20 read_log 192834'13972 (192834'13950) delete a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15451 2014-08-11 15:27:56.864442 2014-08-13 17:38:02.806765 7fe9bae887a0 20 read_log 192834'14038 (192834'13737) clean ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15517 2014-08-11 15:28:12.564298 2014-08-13 17:38:02.806777 7fe9bae887a0 20 read_log 192834'14048 (192834'14038) delete ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15527 2014-08-11 15:28:12.669545 2014-08-13 17:38:02.806789 7fe9bae887a0 20 read_log 192834'14069 (192834'13766) clean dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15548 2014-08-11 15:28:17.906447 2014-08-13 17:38:02.806801 7fe9bae887a0 20 read_log 192834'14070 (0'0) promote 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15549 2014-08-11 15:28:17.938466 2014-08-13 17:38:02.806813 7fe9bae887a0 20 read_log 192834'14071 (192834'13762) clean 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15550 2014-08-11 15:28:17.938747 2014-08-13 17:38:02.806826 7fe9bae887a0 20 read_log 192834'14072 (192834'14070) modify 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by client.1883634.0:23815 2014-08-11 15:28:17.857683 2014-08-13 17:38:02.806839 7fe9bae887a0 20 read_log 192834'14099 (0'0) promote 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15578 2014-08-11 15:28:20.395840 2014-08-13 17:38:02.806851 7fe9bae887a0 20 read_log 192834'14100 (192834'14099) modify 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by client.1883634.0:25238 2014-08-11 15:28:20.366028 2014-08-13 17:38:02.806882 7fe9bae887a0 20 read_log 192834'14118 (192834'14069) delete dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15597 2014-08-11 15:28:22.892244 2014-08-13 17:38:02.806895 7fe9bae887a0 20 read_log 192834'14119 (192834'14071) delete 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15598 2014-08-11 15:28:22.896160 2014-08-13 17:38:02.806907 7fe9bae887a0 20 read_log 192834'14133 (0'0) promote 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15612 2014-08-11 15:28:24.302539 2014-08-13 17:38:02.806919 7fe9bae887a0 20 read_log 192834'14134 (192834'14133) modify 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by client.1883634.0:27242 2014-08-11 15:28:24.273240 2014-08-13 17:38:02.806953 7fe9bae887a0 20 read_log 192834'14178 (192834'13831) clean 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15657 2014-08-11 15:28:28.189577 2014-08-13 17:38:02.806969 7fe9bae887a0 20 read_log 192834'14186 (192834'14178) delete 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15665 2014-08-11 15:28:28.236474 2014-08-13 17:38:02.806981 7fe9bae887a0 20 read_log 192834'14207 (0'0) promote d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15686 2014-08-11 15:28:31.992190 2014-08-13 17:38:02.806994 7fe9bae887a0 20 read_log 192834'14208 (192834'14207) modify d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by client.1883634.0:32516 2014-08-11 15:28:31.958593 2014-08-13 17:38:02.807008 7fe9bae887a0 20 read_log 192834'14213 (0'0) promote a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15692 2014-08-11 15:28:32.368941 2014-08-13 17:38:02.807020 7fe9bae887a0 20 read_log 192834'14214 (192834'14213) modify a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by client.1883634.0:32805 2014-08-11 15:28:32.343211 2014-08-13 17:38:02.807033 7fe9bae887a0 20 read_log 192834'14242 (192834'13849) clean d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15721 2014-08-11 15:28:33.724350 2014-08-13 17:38:02.807047 7fe9bae887a0 20 read_log 192834'14272 (0'0) promote ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15751 2014-08-11 15:28:38.398943 2014-08-13 17:38:02.807066 7fe9bae887a0 20 read_log 192834'14273 (192834'14272) modify ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by client.1883634.0:36753 2014-08-11 15:28:38.367459 2014-08-13 17:38:02.807078 7fe9bae887a0 20 read_log 192834'14275 (192834'14242) delete d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15754 2014-08-11 15:28:38.677270 2014-08-13 17:38:02.807092 7fe9bae887a0 20 read_log 192834'14279 (192834'13875) clean ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15758 2014-08-11 15:28:38.781680 2014-08-13 17:38:02.807106 7fe9bae887a0 20 read_log 192834'14299 (0'0) promote 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15778 2014-08-11 15:28:40.469982 2014-08-13 17:38:02.807119 7fe9bae887a0 20 read_log 192834'14300 (192834'14299) modify 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by client.1883634.0:38278 2014-08-11 15:28:40.442266 2014-08-13 17:38:02.807131 7fe9bae887a0 20 read_log 192834'14303 (0'0) promote dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15782 2014-08-11 15:28:40.632774 2014-08-13 17:38:02.807143 7fe9bae887a0 20 read_log 192834'14304 (192834'14303) modify dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by client.1883634.0:38410 2014-08-11 15:28:40.607496 2014-08-13 17:38:02.807162 7fe9bae887a0 20 read_log 192834'14317 (0'0) promote 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15796 2014-08-11 15:28:42.564704 2014-08-13 17:38:02.807175 7fe9bae887a0 20 read_log 192834'14318 (192834'14317) modify 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by client.1883634.0:39765 2014-08-11 15:28:42.535121 2014-08-13 17:38:02.807190 7fe9bae887a0 20 read_log 192834'14335 (0'0) promote d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15814 2014-08-11 15:28:43.492699 2014-08-13 17:38:02.807202 7fe9bae887a0 20 read_log 192834'14336 (192834'14335) modify d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by client.1883634.0:40279 2014-08-11 15:28:43.468856 2014-08-13 17:38:02.807217 7fe9bae887a0 20 read_log 192834'14342 (192834'14279) delete ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15821 2014-08-11 15:28:43.774574 2014-08-13 17:38:02.807258 7fe9bae887a0 20 read_log 192834'14345 (192834'13902) clean 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:15824 2014-08-11 15:28:44.076870 2014-08-13 17:38:02.807274 7fe9bae887a0 20 read_log 192834'14346 (0'0) promote ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15825 2014-08-11 15:28:44.087715 2014-08-13 17:38:02.807287 7fe9bae887a0 20 read_log 192834'14348 (192834'14346) modify ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by client.1883634.0:40613 2014-08-11 15:28:44.050415 2014-08-13 17:38:02.807299 7fe9bae887a0 20 read_log 192834'14354 (192834'14345) modify 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by client.1883634.0:41253 2014-08-11 15:28:44.817939 2014-08-13 17:38:02.807311 7fe9bae887a0 20 read_log 192834'14359 (192834'13936) modify 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by client.1883634.0:42426 2014-08-11 15:28:46.240555 2014-08-13 17:38:02.807323 7fe9bae887a0 20 read_log 192834'14368 (192834'13965) modify 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by client.1883634.0:43459 2014-08-11 15:28:46.561255 2014-08-13 17:38:02.807335 7fe9bae887a0 20 read_log 192834'14371 (192834'13971) modify 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by client.1883634.0:43665 2014-08-11 15:28:46.624796 2014-08-13 17:38:02.807348 7fe9bae887a0 20 read_log 192834'14395 (192834'14072) clean 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15874 2014-08-11 15:29:19.316108 2014-08-13 17:38:02.807360 7fe9bae887a0 20 read_log 192834'14401 (192834'14395) delete 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15880 2014-08-11 15:29:19.364777 2014-08-13 17:38:02.807372 7fe9bae887a0 20 read_log 192834'14419 (192834'14134) clean 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15898 2014-08-11 15:29:24.508420 2014-08-13 17:38:02.807384 7fe9bae887a0 20 read_log 192834'14425 (192834'14100) clean 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15904 2014-08-11 15:29:24.528486 2014-08-13 17:38:02.807396 7fe9bae887a0 20 read_log 192834'14444 (192834'14419) delete 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15923 2014-08-11 15:29:24.599879 2014-08-13 17:38:02.807408 7fe9bae887a0 20 read_log 192834'14450 (192834'14425) delete 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15929 2014-08-11 15:29:24.613478 2014-08-13 17:38:02.807420 7fe9bae887a0 20 read_log 192834'14510 (192834'14208) clean d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15989 2014-08-11 15:29:34.926212 2014-08-13 17:38:02.807432 7fe9bae887a0 20 read_log 192834'14511 (192834'14214) clean a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15990 2014-08-11 15:29:34.926532 2014-08-13 17:38:02.807443 7fe9bae887a0 20 read_log 192834'14524 (192834'14510) delete d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:16003 2014-08-11 15:29:34.973217 2014-08-13 17:38:02.807455 7fe9bae887a0 20 read_log 192834'14525 (192834'14511) delete a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:16004 2014-08-11 15:29:34.975387 2014-08-13 17:38:02.807467 7fe9bae887a0 20 read_log 192834'14541 (192834'14273) clean ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:16020 2014-08-11 15:29:40.044619 2014-08-13 17:38:02.807479 7fe9bae887a0 20 read_log 192834'14561 (192834'14541) delete ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:16040 2014-08-11 15:29:40.099076 2014-08-13 17:38:02.807491 7fe9bae887a0 20 read_log 192834'14586 (192834'14348) clean ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:16065 2014-08-11 15:29:45.179258 2014-08-13 17:38:02.807512 7fe9bae887a0 20 read_log 192834'14592 (192834'14354) clean 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:16071 2014-08-11 15:29:45.193556 2014-08-13 17:38:02.807525 7fe9bae887a0 20 read_log 192834'14593 (192834'14336) clean d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:16072 2014-08-11 15:29:45.197052 2014-08-13 17:38:02.807538 7fe9bae887a0 20 read_log 192834'14594 (192834'14318) clean 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:16073 2014-08-11 15:29:45.200363 2014-08-13 17:38:02.807549 7fe9bae887a0 20 read_log 192834'14596 (192834'14304) clean dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:16075 2014-08-11 15:29:45.207242 2014-08-13 17:38:02.807561 7fe9bae887a0 20 read_log 192834'14597 (192834'14300) clean 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:16076 2014-08-11 15:29:45.210707 2014-08-13 17:38:02.807573 7fe9bae887a0 20 read_log 192834'14613 (192834'14586) delete ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:16092 2014-08-11 15:29:45.233557 2014-08-13 17:38:02.807585 7fe9bae887a0 20 read_log 192834'14618 (192834'14592) delete 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:16097 2014-08-11 15:29:45.238583 2014-08-13 17:38:02.807597 7fe9bae887a0 20 read_log 192834'14619 (192834'14593) delete d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:16098 2014-08-11 15:29:45.239292 2014-08-13 17:38:02.807609 7fe9bae887a0 20 read_log 192834'14620 (192834'14594) delete 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:16099 2014-08-11 15:29:45.240868 2014-08-13 17:38:02.807621 7fe9bae887a0 20 read_log 192834'14622 (192834'14596) delete dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:16101 2014-08-11 15:29:45.242039 2014-08-13 17:38:02.807633 7fe9bae887a0 20 read_log 192834'14623 (192834'14597) delete 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:16102 2014-08-11 15:29:45.244382 2014-08-13 17:38:02.807645 7fe9bae887a0 20 read_log 192834'14642 (192834'14359) clean 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:16121 2014-08-11 15:29:50.280145 2014-08-13 17:38:02.807656 7fe9bae887a0 20 read_log 192834'14660 (192834'14642) delete 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:16139 2014-08-11 15:29:50.298221 2014-08-13 17:38:02.807668 7fe9bae887a0 20 read_log 192834'14666 (192834'14368) clean 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:16145 2014-08-11 15:29:50.307407 2014-08-13 17:38:02.807680 7fe9bae887a0 20 read_log 192834'14675 (192834'14666) delete 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:16154 2014-08-11 15:29:50.314819 2014-08-13 17:38:02.807694 7fe9bae887a0 20 read_log 192834'14682 (192834'14371) clean 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:16161 2014-08-11 15:29:50.321980 2014-08-13 17:38:02.807707 7fe9bae887a0 20 read_log 192834'14687 (192834'14682) delete 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:16166 2014-08-11 15:29:50.327358 2014-08-13 17:38:02.807719 7fe9bae887a0 20 read_log 192888'14690 (192888'14690) modify b6/hit_set_24.b6_archive_2014-08-11 16:02:27.751742_2014-08-11 16:44:57.822375/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.822933 2014-08-13 17:38:02.807732 7fe9bae887a0 20 read_log 192888'14701 (0'0) promote 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by osd.4.0:21464 2014-08-11 16:45:05.409770 2014-08-13 17:38:02.807752 7fe9bae887a0 20 read_log 192888'14702 (192888'14701) modify 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.1315259.0:2683 2014-08-11 16:45:05.402670 2014-08-13 17:38:02.807765 7fe9bae887a0 20 read_log 192888'14703 (0'0) promote 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by osd.4.0:21500 2014-08-11 16:45:11.786111 2014-08-13 17:38:02.807777 7fe9bae887a0 20 read_log 192888'14704 (192888'14703) modify 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.1315259.0:4364 2014-08-11 16:45:11.777501 2014-08-13 17:38:02.807793 7fe9bae887a0 20 read_log 192888'14705 (0'0) promote 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by osd.4.0:21504 2014-08-11 16:45:12.137164 2014-08-13 17:38:02.807805 7fe9bae887a0 20 read_log 192888'14706 (192888'14705) modify 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.1315259.0:4461 2014-08-11 16:45:12.128694 2014-08-13 17:38:02.807817 7fe9bae887a0 20 read_log 192888'14709 (0'0) promote c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by osd.4.0:21508 2014-08-11 16:45:12.450511 2014-08-13 17:38:02.807829 7fe9bae887a0 20 read_log 192888'14710 (192888'14709) modify c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.1315259.0:4546 2014-08-11 16:45:12.441798 2014-08-13 17:38:02.807841 7fe9bae887a0 20 read_log 192888'14715 (0'0) promote 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by osd.4.0:21546 2014-08-11 16:45:16.045119 2014-08-13 17:38:02.807853 7fe9bae887a0 20 read_log 192888'14716 (192888'14715) modify 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.1315259.0:5518 2014-08-11 16:45:16.035342 2014-08-13 17:38:02.807865 7fe9bae887a0 20 read_log 192888'14719 (0'0) promote aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by osd.4.0:21566 2014-08-11 16:45:18.783660 2014-08-13 17:38:02.807877 7fe9bae887a0 20 read_log 192888'14720 (192888'14719) modify aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.1315259.0:6277 2014-08-11 16:45:18.775492 2014-08-13 17:38:02.807889 7fe9bae887a0 20 read_log 192888'14721 (0'0) promote f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by osd.4.0:21622 2014-08-11 16:45:27.434372 2014-08-13 17:38:02.807901 7fe9bae887a0 20 read_log 192888'14722 (192888'14721) modify f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.1315259.0:8627 2014-08-11 16:45:27.425153 2014-08-13 17:38:02.807913 7fe9bae887a0 20 read_log 192888'14725 (0'0) promote a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by osd.4.0:21652 2014-08-11 16:45:32.006393 2014-08-13 17:38:02.807925 7fe9bae887a0 20 read_log 192888'14726 (192888'14725) modify a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.1315259.0:9816 2014-08-11 16:45:31.997783 2014-08-13 17:38:02.807937 7fe9bae887a0 20 read_log 192888'14739 (0'0) promote 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by osd.4.0:21750 2014-08-11 16:45:47.014484 2014-08-13 17:38:02.807950 7fe9bae887a0 20 read_log 192888'14740 (192888'14739) modify 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.1315259.0:13530 2014-08-11 16:45:47.003185 2014-08-13 17:38:02.807962 7fe9bae887a0 20 read_log 192888'14745 (0'0) promote 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by osd.4.0:21780 2014-08-11 16:45:52.607763 2014-08-13 17:38:02.807974 7fe9bae887a0 20 read_log 192888'14746 (192888'14745) modify 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.1315259.0:15013 2014-08-11 16:45:52.597880 2014-08-13 17:38:02.807986 7fe9bae887a0 20 read_log 192888'14749 (0'0) promote 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by osd.4.0:21866 2014-08-11 16:46:03.212155 2014-08-13 17:38:02.808005 7fe9bae887a0 20 read_log 192888'14750 (192888'14749) modify 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.1315259.0:17709 2014-08-11 16:46:03.204008 2014-08-13 17:38:02.808018 7fe9bae887a0 20 read_log 192888'14751 (0'0) promote 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by osd.4.0:21868 2014-08-11 16:46:03.420732 2014-08-13 17:38:02.808030 7fe9bae887a0 20 read_log 192888'14752 (192888'14751) modify 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.1315259.0:17762 2014-08-11 16:46:03.411561 2014-08-13 17:38:02.808042 7fe9bae887a0 20 read_log 192888'14757 (0'0) promote 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by osd.4.0:21904 2014-08-11 16:46:07.952459 2014-08-13 17:38:02.808054 7fe9bae887a0 20 read_log 192888'14758 (192888'14757) modify 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.1315259.0:18942 2014-08-11 16:46:07.943567 2014-08-13 17:38:02.808067 7fe9bae887a0 20 read_log 192888'14759 (0'0) promote 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by osd.4.0:21908 2014-08-11 16:46:08.476804 2014-08-13 17:38:02.808079 7fe9bae887a0 20 read_log 192888'14760 (192888'14759) modify 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.1315259.0:19052 2014-08-11 16:46:08.466460 2014-08-13 17:38:02.808091 7fe9bae887a0 20 read_log 192888'14761 (0'0) promote d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by osd.4.0:21938 2014-08-11 16:46:13.100679 2014-08-13 17:38:02.808103 7fe9bae887a0 20 read_log 192888'14762 (192888'14761) modify d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.1315259.0:20266 2014-08-11 16:46:13.092080 2014-08-13 17:38:02.808115 7fe9bae887a0 20 read_log 192888'14763 (0'0) promote 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by osd.4.0:21942 2014-08-11 16:46:13.359338 2014-08-13 17:38:02.808143 7fe9bae887a0 20 read_log 192888'14764 (192888'14763) modify 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.1315259.0:20336 2014-08-11 16:46:13.349574 2014-08-13 17:38:02.808161 7fe9bae887a0 20 read_log 192888'14779 (0'0) promote fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by osd.4.0:22032 2014-08-11 16:46:26.959638 2014-08-13 17:38:02.808173 7fe9bae887a0 20 read_log 192888'14780 (192888'14779) modify fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.1315259.0:23822 2014-08-11 16:46:26.951072 2014-08-13 17:38:02.808186 7fe9bae887a0 20 read_log 192888'14789 (0'0) promote 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by osd.4.0:22074 2014-08-11 16:46:30.756897 2014-08-13 17:38:02.808197 7fe9bae887a0 20 read_log 192888'14790 (192888'14789) modify 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.1315259.0:24796 2014-08-11 16:46:30.750037 2014-08-13 17:38:02.808210 7fe9bae887a0 20 read_log 192888'14795 (0'0) promote 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by osd.4.0:22134 2014-08-11 16:46:39.588936 2014-08-13 17:38:02.808222 7fe9bae887a0 20 read_log 192888'14796 (192888'14795) modify 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.1315259.0:26931 2014-08-11 16:46:39.579318 2014-08-13 17:38:02.808234 7fe9bae887a0 20 read_log 192888'14801 (0'0) promote 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by osd.4.0:22170 2014-08-11 16:46:48.689171 2014-08-13 17:38:02.808245 7fe9bae887a0 20 read_log 192888'14802 (192888'14801) modify 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.1315259.0:29171 2014-08-11 16:46:48.680986 2014-08-13 17:38:02.808258 7fe9bae887a0 20 read_log 192888'14803 (0'0) promote 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by osd.4.0:22176 2014-08-11 16:46:50.271004 2014-08-13 17:38:02.808280 7fe9bae887a0 20 read_log 192888'14804 (192888'14803) modify 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.1315259.0:29533 2014-08-11 16:46:50.262555 2014-08-13 17:38:02.808293 7fe9bae887a0 20 read_log 192888'14814 (192888'14702) modify 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.1315259.0:33410 2014-08-11 16:46:56.630025 2014-08-13 17:38:02.808305 7fe9bae887a0 20 read_log 192888'14815 (192888'14704) modify 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.1315259.0:35091 2014-08-11 16:46:57.066653 2014-08-13 17:38:02.808318 7fe9bae887a0 20 read_log 192888'14816 (192888'14706) modify 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.1315259.0:35188 2014-08-11 16:46:57.095514 2014-08-13 17:38:02.808330 7fe9bae887a0 20 read_log 192888'14818 (192888'14710) modify c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.1315259.0:35273 2014-08-11 16:46:57.120762 2014-08-13 17:38:02.808341 7fe9bae887a0 20 read_log 192888'14821 (192888'14716) modify 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.1315259.0:36245 2014-08-11 16:46:57.373025 2014-08-13 17:38:02.808353 7fe9bae887a0 20 read_log 192888'14823 (192888'14720) modify aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.1315259.0:37004 2014-08-11 16:46:57.569190 2014-08-13 17:38:02.808365 7fe9bae887a0 20 read_log 192888'14824 (192888'14722) modify f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.1315259.0:39354 2014-08-11 16:46:58.189203 2014-08-13 17:38:02.808377 7fe9bae887a0 20 read_log 192888'14826 (192888'14726) modify a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.1315259.0:40543 2014-08-11 16:46:58.519129 2014-08-13 17:38:02.808389 7fe9bae887a0 20 read_log 192888'14833 (192888'14740) modify 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.1315259.0:44257 2014-08-11 16:46:59.528427 2014-08-13 17:38:02.808400 7fe9bae887a0 20 read_log 192888'14836 (192888'14746) modify 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.1315259.0:45740 2014-08-11 16:46:59.955783 2014-08-13 17:38:02.808412 7fe9bae887a0 20 read_log 192888'14838 (192888'14750) modify 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.1315259.0:48436 2014-08-11 16:47:00.763339 2014-08-13 17:38:02.808424 7fe9bae887a0 20 read_log 192888'14839 (192888'14752) modify 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.1315259.0:48489 2014-08-11 16:47:00.777604 2014-08-13 17:38:02.808436 7fe9bae887a0 20 read_log 192888'14842 (192888'14758) modify 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.1315259.0:49669 2014-08-11 16:47:01.137578 2014-08-13 17:38:02.808448 7fe9bae887a0 20 read_log 192888'14843 (192888'14760) modify 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.1315259.0:49779 2014-08-11 16:47:01.170689 2014-08-13 17:38:02.808460 7fe9bae887a0 20 read_log 192888'14844 (192888'14762) modify d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.1315259.0:50993 2014-08-11 16:47:01.543674 2014-08-13 17:38:02.808472 7fe9bae887a0 20 read_log 192888'14845 (192888'14764) modify 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.1315259.0:51063 2014-08-11 16:47:01.568141 2014-08-13 17:38:02.808484 7fe9bae887a0 20 read_log 192888'14853 (192888'14780) modify fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.1315259.0:54549 2014-08-11 16:47:02.598201 2014-08-13 17:38:02.808496 7fe9bae887a0 20 read_log 192888'14858 (192888'14790) modify 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.1315259.0:55523 2014-08-11 16:47:02.935489 2014-08-13 17:38:02.808508 7fe9bae887a0 20 read_log 192888'14861 (192888'14796) modify 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.1315259.0:57658 2014-08-11 16:47:03.576744 2014-08-13 17:38:02.808527 7fe9bae887a0 20 read_log 192888'14864 (192888'14802) modify 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.1315259.0:59898 2014-08-11 16:47:04.277608 2014-08-13 17:38:02.808539 7fe9bae887a0 20 read_log 192888'14865 (192888'14804) modify 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.1315259.0:60260 2014-08-11 16:47:04.395260 2014-08-13 17:38:02.808551 7fe9bae887a0 20 read_log 192888'14868 (192888'14868) modify b6/hit_set_24.b6_archive_2014-08-11 16:44:57.823070_2014-08-11 17:05:15.028214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.028548 2014-08-13 17:38:02.808564 7fe9bae887a0 20 read_log 192888'14869 (192888'14690) delete b6/hit_set_24.b6_archive_2014-08-11 16:02:27.751742_2014-08-11 16:44:57.822375/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.028548 2014-08-13 17:38:02.808576 7fe9bae887a0 20 read_log 192888'14874 (0'0) promote d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by osd.4.0:22712 2014-08-11 17:05:25.826181 2014-08-13 17:38:02.808588 7fe9bae887a0 20 read_log 192888'14875 (192888'14874) modify d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.942868.0:2670 2014-08-11 17:05:25.818065 2014-08-13 17:38:02.808600 7fe9bae887a0 20 read_log 192888'14878 (0'0) promote a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by osd.4.0:22728 2014-08-11 17:05:28.051620 2014-08-13 17:38:02.808612 7fe9bae887a0 20 read_log 192888'14879 (192888'14878) modify a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.942868.0:3273 2014-08-11 17:05:28.043688 2014-08-13 17:38:02.808625 7fe9bae887a0 20 read_log 192888'14884 (0'0) promote a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by osd.4.0:22776 2014-08-11 17:05:32.603579 2014-08-13 17:38:02.808637 7fe9bae887a0 20 read_log 192888'14885 (192888'14884) modify a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.942868.0:4505 2014-08-11 17:05:32.595158 2014-08-13 17:38:02.808649 7fe9bae887a0 20 read_log 192888'14886 (0'0) promote 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by osd.4.0:22806 2014-08-11 17:05:36.812424 2014-08-13 17:38:02.808661 7fe9bae887a0 20 read_log 192888'14887 (192888'14886) modify 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.942868.0:5586 2014-08-11 17:05:36.805050 2014-08-13 17:38:02.808673 7fe9bae887a0 20 read_log 192888'14888 (0'0) promote 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by osd.4.0:22824 2014-08-11 17:05:39.247033 2014-08-13 17:38:02.808685 7fe9bae887a0 20 read_log 192888'14889 (192888'14888) modify 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.942868.0:6258 2014-08-11 17:05:39.240567 2014-08-13 17:38:02.808697 7fe9bae887a0 20 read_log 192888'14890 (0'0) promote 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by osd.4.0:22840 2014-08-11 17:05:41.141836 2014-08-13 17:38:02.808708 7fe9bae887a0 20 read_log 192888'14891 (192888'14890) modify 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.942868.0:6783 2014-08-11 17:05:41.130767 2014-08-13 17:38:02.808720 7fe9bae887a0 20 read_log 192888'14894 (0'0) promote 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by osd.4.0:22886 2014-08-11 17:05:48.058684 2014-08-13 17:38:02.808732 7fe9bae887a0 20 read_log 192888'14895 (192888'14894) modify 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.942868.0:8617 2014-08-11 17:05:48.049769 2014-08-13 17:38:02.808744 7fe9bae887a0 20 read_log 192888'14898 (0'0) promote ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by osd.4.0:22926 2014-08-11 17:05:55.183907 2014-08-13 17:38:02.808780 7fe9bae887a0 20 read_log 192888'14899 (192888'14898) modify ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.942868.0:10549 2014-08-11 17:05:55.175570 2014-08-13 17:38:02.808793 7fe9bae887a0 20 read_log 192888'14900 (0'0) promote 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by osd.4.0:22948 2014-08-11 17:05:58.546578 2014-08-13 17:38:02.808805 7fe9bae887a0 20 read_log 192888'14901 (192888'14900) modify 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.942868.0:11465 2014-08-11 17:05:58.538142 2014-08-13 17:38:02.808818 7fe9bae887a0 20 read_log 192888'14902 (0'0) promote a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by osd.4.0:22962 2014-08-11 17:06:01.398686 2014-08-13 17:38:02.808830 7fe9bae887a0 20 read_log 192888'14903 (192888'14902) modify a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.942868.0:12249 2014-08-11 17:06:01.389020 2014-08-13 17:38:02.808842 7fe9bae887a0 20 read_log 192888'14904 (0'0) promote 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by osd.4.0:22966 2014-08-11 17:06:01.871922 2014-08-13 17:38:02.808854 7fe9bae887a0 20 read_log 192888'14905 (192888'14904) modify 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.942868.0:12381 2014-08-11 17:06:01.862473 2014-08-13 17:38:02.808866 7fe9bae887a0 20 read_log 192888'14912 (0'0) promote 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by osd.4.0:23026 2014-08-11 17:06:11.503401 2014-08-13 17:38:02.808878 7fe9bae887a0 20 read_log 192888'14913 (192888'14912) modify 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.942868.0:14815 2014-08-11 17:06:11.494499 2014-08-13 17:38:02.808890 7fe9bae887a0 20 read_log 192888'14914 (0'0) promote 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by osd.4.0:23078 2014-08-11 17:06:19.931983 2014-08-13 17:38:02.808902 7fe9bae887a0 20 read_log 192888'14915 (192888'14914) modify 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.942868.0:16965 2014-08-11 17:06:19.923270 2014-08-13 17:38:02.808914 7fe9bae887a0 20 read_log 192888'14916 (0'0) promote 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by osd.4.0:23096 2014-08-11 17:06:21.381673 2014-08-13 17:38:02.808926 7fe9bae887a0 20 read_log 192888'14917 (192888'14916) modify 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.942868.0:17348 2014-08-11 17:06:21.370306 2014-08-13 17:38:02.808938 7fe9bae887a0 20 read_log 192888'14920 (0'0) promote 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by osd.4.0:23138 2014-08-11 17:06:27.418074 2014-08-13 17:38:02.808950 7fe9bae887a0 20 read_log 192888'14921 (192888'14920) modify 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.942868.0:18953 2014-08-11 17:06:27.410061 2014-08-13 17:38:02.808961 7fe9bae887a0 20 read_log 192888'14926 (0'0) promote 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by osd.4.0:23150 2014-08-11 17:06:29.536553 2014-08-13 17:38:02.808973 7fe9bae887a0 20 read_log 192888'14927 (192888'14926) modify 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.942868.0:19517 2014-08-11 17:06:29.528414 2014-08-13 17:38:02.808988 7fe9bae887a0 20 read_log 192888'14928 (0'0) promote e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by osd.4.0:23152 2014-08-11 17:06:29.569157 2014-08-13 17:38:02.809000 7fe9bae887a0 20 read_log 192888'14929 (192888'14928) modify e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.942868.0:19526 2014-08-11 17:06:29.560339 2014-08-13 17:38:02.809012 7fe9bae887a0 20 read_log 192888'14930 (0'0) promote 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by osd.4.0:23162 2014-08-11 17:06:30.602831 2014-08-13 17:38:02.809033 7fe9bae887a0 20 read_log 192888'14931 (192888'14930) modify 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.942868.0:19798 2014-08-11 17:06:30.593767 2014-08-13 17:38:02.809047 7fe9bae887a0 20 read_log 192888'14934 (0'0) promote 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by osd.4.0:23194 2014-08-11 17:06:35.683952 2014-08-13 17:38:02.809060 7fe9bae887a0 20 read_log 192888'14935 (192888'14934) modify 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.942868.0:21128 2014-08-11 17:06:35.675477 2014-08-13 17:38:02.809072 7fe9bae887a0 20 read_log 192888'14938 (0'0) promote 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by osd.4.0:23214 2014-08-11 17:06:38.189695 2014-08-13 17:38:02.809084 7fe9bae887a0 20 read_log 192888'14939 (192888'14938) modify 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.942868.0:21790 2014-08-11 17:06:38.183130 2014-08-13 17:38:02.809096 7fe9bae887a0 20 read_log 192888'14944 (0'0) promote 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by osd.4.0:23236 2014-08-11 17:06:41.445686 2014-08-13 17:38:02.809108 7fe9bae887a0 20 read_log 192888'14945 (192888'14944) modify 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.942868.0:22616 2014-08-11 17:06:41.437806 2014-08-13 17:38:02.809120 7fe9bae887a0 20 read_log 192888'14948 (0'0) promote ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by osd.4.0:23280 2014-08-11 17:06:46.874368 2014-08-13 17:38:02.809132 7fe9bae887a0 20 read_log 192888'14949 (192888'14948) modify ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.942868.0:24043 2014-08-11 17:06:46.864634 2014-08-13 17:38:02.809144 7fe9bae887a0 20 read_log 192888'14950 (0'0) promote d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by osd.4.0:23310 2014-08-11 17:06:49.329177 2014-08-13 17:38:02.809162 7fe9bae887a0 20 read_log 192888'14951 (192888'14950) modify d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.942868.0:24707 2014-08-11 17:06:49.320526 2014-08-13 17:38:02.809175 7fe9bae887a0 20 read_log 192888'14954 (0'0) promote edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by osd.4.0:23334 2014-08-11 17:06:51.652565 2014-08-13 17:38:02.809187 7fe9bae887a0 20 read_log 192888'14955 (192888'14954) modify edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.942868.0:25324 2014-08-11 17:06:51.642588 2014-08-13 17:38:02.809199 7fe9bae887a0 20 read_log 192888'14958 (0'0) promote 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by osd.4.0:23362 2014-08-11 17:06:57.195033 2014-08-13 17:38:02.809211 7fe9bae887a0 20 read_log 192888'14959 (192888'14958) modify 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.942868.0:26738 2014-08-11 17:06:57.185923 2014-08-13 17:38:02.809223 7fe9bae887a0 20 read_log 192888'14960 (0'0) promote 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by osd.4.0:23366 2014-08-11 17:06:58.507967 2014-08-13 17:38:02.809235 7fe9bae887a0 20 read_log 192888'14961 (192888'14960) modify 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.942868.0:27076 2014-08-11 17:06:58.498683 2014-08-13 17:38:02.809247 7fe9bae887a0 20 read_log 192888'14962 (0'0) promote 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by osd.4.0:23376 2014-08-11 17:06:59.781997 2014-08-13 17:38:02.809259 7fe9bae887a0 20 read_log 192888'14963 (192888'14962) modify 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.942868.0:27426 2014-08-11 17:06:59.773654 2014-08-13 17:38:02.809271 7fe9bae887a0 20 read_log 192888'14966 (0'0) promote 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by osd.4.0:23392 2014-08-11 17:07:01.536730 2014-08-13 17:38:02.809292 7fe9bae887a0 20 read_log 192888'14967 (192888'14966) modify 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.942868.0:27875 2014-08-11 17:07:01.525856 2014-08-13 17:38:02.809305 7fe9bae887a0 20 read_log 192888'14968 (0'0) promote d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by osd.4.0:23396 2014-08-11 17:07:03.003942 2014-08-13 17:38:02.809317 7fe9bae887a0 20 read_log 192888'14969 (192888'14968) modify d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.942868.0:28276 2014-08-11 17:07:02.994869 2014-08-13 17:38:02.809329 7fe9bae887a0 20 read_log 192888'14970 (0'0) promote f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by osd.4.0:23404 2014-08-11 17:07:03.815431 2014-08-13 17:38:02.809342 7fe9bae887a0 20 read_log 192888'14971 (192888'14970) modify f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.942868.0:28496 2014-08-11 17:07:03.806089 2014-08-13 17:38:02.809354 7fe9bae887a0 20 read_log 192888'14974 (0'0) promote 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by osd.4.0:23420 2014-08-11 17:07:06.508847 2014-08-13 17:38:02.809366 7fe9bae887a0 20 read_log 192888'14975 (192888'14974) modify 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.942868.0:29185 2014-08-11 17:07:06.499490 2014-08-13 17:38:02.809378 7fe9bae887a0 20 read_log 192888'14988 (192888'14875) modify d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.942868.0:33936 2014-08-11 17:07:16.652094 2014-08-13 17:38:02.809408 7fe9bae887a0 20 read_log 192888'14990 (192888'14879) modify a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.942868.0:34539 2014-08-11 17:07:16.818800 2014-08-13 17:38:02.809421 7fe9bae887a0 20 read_log 192888'14993 (192888'14885) modify a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.942868.0:35771 2014-08-11 17:07:17.160060 2014-08-13 17:38:02.809433 7fe9bae887a0 20 read_log 192888'14994 (192888'14887) modify 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.942868.0:36852 2014-08-11 17:07:17.450395 2014-08-13 17:38:02.809446 7fe9bae887a0 20 read_log 192888'14995 (192888'14889) modify 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.942868.0:37524 2014-08-11 17:07:17.633137 2014-08-13 17:38:02.809457 7fe9bae887a0 20 read_log 192888'14996 (192888'14891) modify 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.942868.0:38049 2014-08-11 17:07:17.777571 2014-08-13 17:38:02.809469 7fe9bae887a0 20 read_log 192888'14998 (192888'14895) modify 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.942868.0:39883 2014-08-11 17:07:18.310216 2014-08-13 17:38:02.809481 7fe9bae887a0 20 read_log 192888'15000 (192888'14899) modify ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.942868.0:41815 2014-08-11 17:07:18.831064 2014-08-13 17:38:02.809494 7fe9bae887a0 20 read_log 192888'15001 (192888'14901) modify 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.942868.0:42731 2014-08-11 17:07:19.077431 2014-08-13 17:38:02.809506 7fe9bae887a0 20 read_log 192888'15002 (192888'14903) modify a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.942868.0:43515 2014-08-11 17:07:19.293534 2014-08-13 17:38:02.809518 7fe9bae887a0 20 read_log 192888'15003 (192888'14905) modify 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.942868.0:43647 2014-08-11 17:07:19.332183 2014-08-13 17:38:02.809530 7fe9bae887a0 20 read_log 192888'15007 (192888'14913) modify 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.942868.0:46081 2014-08-11 17:07:20.015459 2014-08-13 17:38:02.809542 7fe9bae887a0 20 read_log 192888'15008 (192888'14915) modify 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.942868.0:48231 2014-08-11 17:07:20.661825 2014-08-13 17:38:02.809561 7fe9bae887a0 20 read_log 192888'15009 (192888'14917) modify 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.942868.0:48614 2014-08-11 17:07:20.766166 2014-08-13 17:38:02.809574 7fe9bae887a0 20 read_log 192888'15011 (192888'14921) modify 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.942868.0:50219 2014-08-11 17:07:21.251884 2014-08-13 17:38:02.809586 7fe9bae887a0 20 read_log 192888'15014 (192888'14927) modify 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.942868.0:50783 2014-08-11 17:07:21.431039 2014-08-13 17:38:02.809598 7fe9bae887a0 20 read_log 192888'15015 (192888'14929) modify e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.942868.0:50792 2014-08-11 17:07:21.434279 2014-08-13 17:38:02.809609 7fe9bae887a0 20 read_log 192888'15016 (192888'14931) modify 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.942868.0:51064 2014-08-11 17:07:21.520475 2014-08-13 17:38:02.809621 7fe9bae887a0 20 read_log 192888'15018 (192888'14935) modify 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.942868.0:52394 2014-08-11 17:07:21.924644 2014-08-13 17:38:02.809633 7fe9bae887a0 20 read_log 192888'15020 (192888'14939) modify 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.942868.0:53056 2014-08-11 17:07:22.134108 2014-08-13 17:38:02.809645 7fe9bae887a0 20 read_log 192888'15023 (192888'14945) modify 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.942868.0:53882 2014-08-11 17:07:22.404192 2014-08-13 17:38:02.809657 7fe9bae887a0 20 read_log 192888'15025 (192888'14949) modify ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.942868.0:55309 2014-08-11 17:07:22.808490 2014-08-13 17:38:02.809669 7fe9bae887a0 20 read_log 192888'15026 (192888'14951) modify d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.942868.0:55973 2014-08-11 17:07:23.013407 2014-08-13 17:38:02.809681 7fe9bae887a0 20 read_log 192888'15028 (192888'14955) modify edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.942868.0:56590 2014-08-11 17:07:23.199855 2014-08-13 17:38:02.809693 7fe9bae887a0 20 read_log 192888'15030 (192888'14959) modify 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.942868.0:58004 2014-08-11 17:07:23.615104 2014-08-13 17:38:02.809705 7fe9bae887a0 20 read_log 192888'15031 (192888'14961) modify 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.942868.0:58342 2014-08-11 17:07:23.712254 2014-08-13 17:38:02.809717 7fe9bae887a0 20 read_log 192888'15032 (192888'14963) modify 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.942868.0:58692 2014-08-11 17:07:23.821281 2014-08-13 17:38:02.809729 7fe9bae887a0 20 read_log 192888'15034 (192888'14967) modify 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.942868.0:59141 2014-08-11 17:07:23.956493 2014-08-13 17:38:02.809741 7fe9bae887a0 20 read_log 192888'15035 (192888'14969) modify d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.942868.0:59542 2014-08-11 17:07:24.075182 2014-08-13 17:38:02.809752 7fe9bae887a0 20 read_log 192888'15036 (192888'14971) modify f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.942868.0:59762 2014-08-11 17:07:24.142878 2014-08-13 17:38:02.809770 7fe9bae887a0 20 read_log 192888'15038 (192888'14975) modify 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.942868.0:60451 2014-08-11 17:07:24.344284 2014-08-13 17:38:02.809782 7fe9bae887a0 20 read_log 193305'15044 (193305'15044) modify b6/hit_set_24.b6_archive_2014-08-11 18:37:04.909790_2014-08-11 18:42:37.869502/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.869737 2014-08-13 17:38:02.809804 7fe9bae887a0 20 read_log 193305'15045 (192888'14868) delete b6/hit_set_24.b6_archive_2014-08-11 16:44:57.823070_2014-08-11 17:05:15.028214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.869737 2014-08-13 17:38:02.809817 7fe9bae887a0 20 read_log 193305'15046 (0'0) promote 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by osd.4.0:12 2014-08-11 18:42:37.903343 2014-08-13 17:38:02.809829 7fe9bae887a0 20 read_log 193305'15047 (193305'15046) modify 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.951337.0:1248 2014-08-11 18:42:37.863310 2014-08-13 17:38:02.809842 7fe9bae887a0 20 read_log 193305'15048 (0'0) promote e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by osd.4.0:35 2014-08-11 18:42:42.057993 2014-08-13 17:38:02.809854 7fe9bae887a0 20 read_log 193305'15049 (193305'15048) modify e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.951337.0:2358 2014-08-11 18:42:42.051033 2014-08-13 17:38:02.809866 7fe9bae887a0 20 read_log 193305'15050 (0'0) promote d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by osd.4.0:37 2014-08-11 18:42:42.443128 2014-08-13 17:38:02.809878 7fe9bae887a0 20 read_log 193305'15051 (193305'15050) modify d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.951337.0:2463 2014-08-11 18:42:42.432532 2014-08-13 17:38:02.809890 7fe9bae887a0 20 read_log 193305'15052 (0'0) promote 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by osd.4.0:41 2014-08-11 18:42:43.792019 2014-08-13 17:38:02.809902 7fe9bae887a0 20 read_log 193305'15053 (193305'15052) modify 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.951337.0:2836 2014-08-11 18:42:43.785300 2014-08-13 17:38:02.809914 7fe9bae887a0 20 read_log 193305'15054 (0'0) promote 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by osd.4.0:45 2014-08-11 18:42:44.772532 2014-08-13 17:38:02.809926 7fe9bae887a0 20 read_log 193305'15055 (193305'15054) modify 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.951337.0:3103 2014-08-11 18:42:44.765714 2014-08-13 17:38:02.809938 7fe9bae887a0 20 read_log 193305'15056 (0'0) promote 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by osd.4.0:61 2014-08-11 18:42:49.737259 2014-08-13 17:38:02.809969 7fe9bae887a0 20 read_log 193305'15057 (193305'15056) modify 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.951337.0:4459 2014-08-11 18:42:49.729113 2014-08-13 17:38:02.809982 7fe9bae887a0 20 read_log 193305'15058 (0'0) promote 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by osd.4.0:69 2014-08-11 18:42:50.870661 2014-08-13 17:38:02.809995 7fe9bae887a0 20 read_log 193305'15059 (193305'15058) modify 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.951337.0:4770 2014-08-11 18:42:50.862483 2014-08-13 17:38:02.810007 7fe9bae887a0 20 read_log 193305'15060 (0'0) promote 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by osd.4.0:71 2014-08-11 18:42:52.107574 2014-08-13 17:38:02.810019 7fe9bae887a0 20 read_log 193305'15061 (193305'15060) modify 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.951337.0:5086 2014-08-11 18:42:52.099666 2014-08-13 17:38:02.810031 7fe9bae887a0 20 read_log 193305'15062 (0'0) promote b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by osd.4.0:85 2014-08-11 18:42:54.268125 2014-08-13 17:38:02.810043 7fe9bae887a0 20 read_log 193305'15063 (193305'15062) modify b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.951337.0:5681 2014-08-11 18:42:54.260287 2014-08-13 17:38:02.810063 7fe9bae887a0 20 read_log 193305'15064 (0'0) promote 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by osd.4.0:119 2014-08-11 18:43:00.907416 2014-08-13 17:38:02.810075 7fe9bae887a0 20 read_log 193305'15065 (193305'15064) modify 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.951337.0:7515 2014-08-11 18:43:00.900205 2014-08-13 17:38:02.810088 7fe9bae887a0 20 read_log 193305'15066 (0'0) promote 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by osd.4.0:137 2014-08-11 18:43:04.344548 2014-08-13 17:38:02.810100 7fe9bae887a0 20 read_log 193305'15067 (193305'15066) modify 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.951337.0:8410 2014-08-11 18:43:04.333582 2014-08-13 17:38:02.810112 7fe9bae887a0 20 read_log 193305'15068 (0'0) promote 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by osd.4.0:147 2014-08-11 18:43:06.786242 2014-08-13 17:38:02.810124 7fe9bae887a0 20 read_log 193305'15069 (193305'15068) modify 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.951337.0:9042 2014-08-11 18:43:06.779080 2014-08-13 17:38:02.810136 7fe9bae887a0 20 read_log 193305'15070 (0'0) promote e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by osd.4.0:157 2014-08-11 18:43:08.605928 2014-08-13 17:38:02.810148 7fe9bae887a0 20 read_log 193305'15071 (193305'15070) modify e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.951337.0:9538 2014-08-11 18:43:08.598413 2014-08-13 17:38:02.810166 7fe9bae887a0 20 read_log 193305'15072 (0'0) promote f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by osd.4.0:225 2014-08-11 18:43:25.575445 2014-08-13 17:38:02.810179 7fe9bae887a0 20 read_log 193305'15073 (193305'15072) modify f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.951337.0:14058 2014-08-11 18:43:25.567009 2014-08-13 17:38:02.810191 7fe9bae887a0 20 read_log 193305'15074 (0'0) promote f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by osd.4.0:235 2014-08-11 18:43:28.966755 2014-08-13 17:38:02.810203 7fe9bae887a0 20 read_log 193305'15075 (193305'15074) modify f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.951337.0:14996 2014-08-11 18:43:28.959057 2014-08-13 17:38:02.810218 7fe9bae887a0 20 read_log 193305'15076 (0'0) promote 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by osd.4.0:237 2014-08-11 18:43:29.100651 2014-08-13 17:38:02.810232 7fe9bae887a0 20 read_log 193305'15077 (193305'15076) modify 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.951337.0:15037 2014-08-11 18:43:29.093778 2014-08-13 17:38:02.810244 7fe9bae887a0 20 read_log 193305'15078 (0'0) promote 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by osd.4.0:257 2014-08-11 18:43:32.439417 2014-08-13 17:38:02.810256 7fe9bae887a0 20 read_log 193305'15079 (193305'15078) modify 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.951337.0:16009 2014-08-11 18:43:32.430312 2014-08-13 17:38:02.810268 7fe9bae887a0 20 read_log 193305'15080 (0'0) promote decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by osd.4.0:271 2014-08-11 18:43:34.153950 2014-08-13 17:38:02.810280 7fe9bae887a0 20 read_log 193305'15081 (193305'15080) modify decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.951337.0:16488 2014-08-11 18:43:34.145672 2014-08-13 17:38:02.810293 7fe9bae887a0 20 read_log 193305'15082 (0'0) promote fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by osd.4.0:275 2014-08-11 18:43:34.439904 2014-08-13 17:38:02.810305 7fe9bae887a0 20 read_log 193305'15083 (193305'15082) modify fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.951337.0:16575 2014-08-11 18:43:34.431607 2014-08-13 17:38:02.810324 7fe9bae887a0 20 read_log 193305'15084 (0'0) promote e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by osd.4.0:281 2014-08-11 18:43:35.749899 2014-08-13 17:38:02.810337 7fe9bae887a0 20 read_log 193305'15085 (193305'15084) modify e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.951337.0:16939 2014-08-11 18:43:35.741388 2014-08-13 17:38:02.810349 7fe9bae887a0 20 read_log 193305'15086 (0'0) promote 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by osd.4.0:287 2014-08-11 18:43:36.540813 2014-08-13 17:38:02.810361 7fe9bae887a0 20 read_log 193305'15087 (193305'15086) modify 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.951337.0:17167 2014-08-11 18:43:36.533583 2014-08-13 17:38:02.810373 7fe9bae887a0 20 read_log 193305'15088 (0'0) promote dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by osd.4.0:309 2014-08-11 18:43:40.824344 2014-08-13 17:38:02.810385 7fe9bae887a0 20 read_log 193305'15089 (193305'15088) modify dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.951337.0:18410 2014-08-11 18:43:40.814504 2014-08-13 17:38:02.810397 7fe9bae887a0 20 read_log 193305'15090 (0'0) promote 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by osd.4.0:323 2014-08-11 18:43:43.467498 2014-08-13 17:38:02.810408 7fe9bae887a0 20 read_log 193305'15091 (193305'15090) modify 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.951337.0:19129 2014-08-11 18:43:43.459675 2014-08-13 17:38:02.810420 7fe9bae887a0 20 read_log 193305'15092 (0'0) promote f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by osd.4.0:369 2014-08-11 18:43:54.405738 2014-08-13 17:38:02.810432 7fe9bae887a0 20 read_log 193305'15093 (193305'15092) modify f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.951337.0:22155 2014-08-11 18:43:54.397798 2014-08-13 17:38:02.810444 7fe9bae887a0 20 read_log 193305'15094 (0'0) promote 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by osd.4.0:377 2014-08-11 18:43:55.799629 2014-08-13 17:38:02.810456 7fe9bae887a0 20 read_log 193305'15095 (193305'15094) modify 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.951337.0:22539 2014-08-11 18:43:55.791744 2014-08-13 17:38:02.810468 7fe9bae887a0 20 read_log 193305'15096 (0'0) promote 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by osd.4.0:405 2014-08-11 18:44:02.955808 2014-08-13 17:38:02.810479 7fe9bae887a0 20 read_log 193305'15097 (193305'15096) modify 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.951337.0:24534 2014-08-11 18:44:02.948465 2014-08-13 17:38:02.810491 7fe9bae887a0 20 read_log 193305'15098 (0'0) promote 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by osd.4.0:411 2014-08-11 18:44:04.970222 2014-08-13 17:38:02.810505 7fe9bae887a0 20 read_log 193305'15099 (193305'15098) modify 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.951337.0:25085 2014-08-11 18:44:04.962756 2014-08-13 17:38:02.810517 7fe9bae887a0 20 read_log 193305'15100 (0'0) promote 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by osd.4.0:437 2014-08-11 18:44:10.781459 2014-08-13 17:38:02.810529 7fe9bae887a0 20 read_log 193305'15101 (193305'15100) modify 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.951337.0:26663 2014-08-11 18:44:10.773653 2014-08-13 17:38:02.810540 7fe9bae887a0 20 read_log 193305'15102 (0'0) promote 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by osd.4.0:461 2014-08-11 18:44:16.017878 2014-08-13 17:38:02.810552 7fe9bae887a0 20 read_log 193305'15103 (193305'15102) modify 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.951337.0:28103 2014-08-11 18:44:16.009650 2014-08-13 17:38:02.810592 7fe9bae887a0 20 read_log 193305'15104 (0'0) promote 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by osd.4.0:475 2014-08-11 18:44:18.798250 2014-08-13 17:38:02.810606 7fe9bae887a0 20 read_log 193305'15105 (193305'15104) modify 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.951337.0:28847 2014-08-11 18:44:18.790590 2014-08-13 17:38:02.810618 7fe9bae887a0 20 read_log 193305'15106 (0'0) promote fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by osd.4.0:483 2014-08-11 18:44:20.239013 2014-08-13 17:38:02.810645 7fe9bae887a0 20 read_log 193305'15107 (193305'15106) modify fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.951337.0:29247 2014-08-11 18:44:20.232618 2014-08-13 17:38:02.810657 7fe9bae887a0 20 read_log 193305'15108 (0'0) promote ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by osd.4.0:493 2014-08-11 18:44:22.926795 2014-08-13 17:38:02.810669 7fe9bae887a0 20 read_log 193305'15109 (193305'15108) modify ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.951337.0:29981 2014-08-11 18:44:22.918542 2014-08-13 17:38:02.810681 7fe9bae887a0 20 read_log 193305'15110 (0'0) promote f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by osd.4.0:501 2014-08-11 18:44:26.037398 2014-08-13 17:38:02.810693 7fe9bae887a0 20 read_log 193305'15111 (193305'15110) modify f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.951337.0:30780 2014-08-11 18:44:26.029501 2014-08-13 17:38:02.810705 7fe9bae887a0 20 read_log 193305'15112 (0'0) promote 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by osd.4.0:521 2014-08-11 18:44:29.761906 2014-08-13 17:38:02.810717 7fe9bae887a0 20 read_log 193305'15113 (193305'15112) modify 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.951337.0:31849 2014-08-11 18:44:29.754208 2014-08-13 17:38:02.810730 7fe9bae887a0 20 read_log 193305'15114 (0'0) promote f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by osd.4.0:551 2014-08-11 18:44:32.107362 2014-08-13 17:38:02.810741 7fe9bae887a0 20 read_log 193305'15115 (193305'15114) modify f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.951337.0:32507 2014-08-11 18:44:32.100040 2014-08-13 17:38:02.810753 7fe9bae887a0 20 read_log 193305'15116 (193305'15047) modify 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.951337.0:33959 2014-08-11 18:44:34.736096 2014-08-13 17:38:02.810765 7fe9bae887a0 20 read_log 193305'15117 (193305'15049) modify e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.951337.0:35069 2014-08-11 18:44:35.361204 2014-08-13 17:38:02.810778 7fe9bae887a0 20 read_log 193305'15118 (193305'15051) modify d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.951337.0:35174 2014-08-11 18:44:35.446878 2014-08-13 17:38:02.810790 7fe9bae887a0 20 read_log 193305'15119 (193305'15053) modify 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.951337.0:35547 2014-08-11 18:44:35.651710 2014-08-13 17:38:02.810802 7fe9bae887a0 20 read_log 193305'15120 (193305'15055) modify 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.951337.0:35814 2014-08-11 18:44:35.757083 2014-08-13 17:38:02.810814 7fe9bae887a0 20 read_log 193305'15121 (193305'15057) modify 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.951337.0:37170 2014-08-11 18:44:36.745932 2014-08-13 17:38:02.810826 7fe9bae887a0 20 read_log 193305'15122 (193305'15059) modify 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.951337.0:37481 2014-08-11 18:44:36.933476 2014-08-13 17:38:02.810837 7fe9bae887a0 20 read_log 193305'15123 (193305'15061) modify 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.951337.0:37797 2014-08-11 18:44:37.064962 2014-08-13 17:38:02.810859 7fe9bae887a0 20 read_log 193305'15124 (193305'15063) modify b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.951337.0:38392 2014-08-11 18:44:37.392796 2014-08-13 17:38:02.810872 7fe9bae887a0 20 read_log 193305'15125 (193305'15065) modify 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.951337.0:40226 2014-08-11 18:44:38.341507 2014-08-13 17:38:02.810884 7fe9bae887a0 20 read_log 193305'15126 (193305'15067) modify 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.951337.0:41121 2014-08-11 18:44:38.641525 2014-08-13 17:38:02.810896 7fe9bae887a0 20 read_log 193305'15127 (193305'15069) modify 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.951337.0:41753 2014-08-11 18:44:38.861102 2014-08-13 17:38:02.810908 7fe9bae887a0 20 read_log 193305'15128 (193305'15071) modify e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.951337.0:42249 2014-08-11 18:44:39.163319 2014-08-13 17:38:02.810920 7fe9bae887a0 20 read_log 193305'15129 (193305'15073) modify f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.951337.0:46769 2014-08-11 18:44:40.584033 2014-08-13 17:38:02.810932 7fe9bae887a0 20 read_log 193305'15130 (193305'15075) modify f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.951337.0:47707 2014-08-11 18:44:40.843160 2014-08-13 17:38:02.810944 7fe9bae887a0 20 read_log 193305'15131 (193305'15077) modify 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.951337.0:47748 2014-08-11 18:44:40.854987 2014-08-13 17:38:02.810956 7fe9bae887a0 20 read_log 193305'15132 (193305'15079) modify 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.951337.0:48720 2014-08-11 18:44:41.135222 2014-08-13 17:38:02.810967 7fe9bae887a0 20 read_log 193305'15133 (193305'15081) modify decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.951337.0:49199 2014-08-11 18:44:41.287105 2014-08-13 17:38:02.810979 7fe9bae887a0 20 read_log 193305'15134 (193305'15083) modify fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.951337.0:49286 2014-08-11 18:44:41.320341 2014-08-13 17:38:02.810991 7fe9bae887a0 20 read_log 193305'15135 (193305'15085) modify e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.951337.0:49650 2014-08-11 18:44:41.428024 2014-08-13 17:38:02.811003 7fe9bae887a0 20 read_log 193305'15136 (193305'15087) modify 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.951337.0:49878 2014-08-11 18:44:41.504342 2014-08-13 17:38:02.811015 7fe9bae887a0 20 read_log 193305'15137 (193305'15089) modify dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.951337.0:51121 2014-08-11 18:44:41.858596 2014-08-13 17:38:02.811027 7fe9bae887a0 20 read_log 193305'15138 (193305'15091) modify 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.951337.0:51840 2014-08-11 18:44:42.062841 2014-08-13 17:38:02.811039 7fe9bae887a0 20 read_log 193305'15139 (193305'15093) modify f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.951337.0:54866 2014-08-11 18:44:42.960800 2014-08-13 17:38:02.811050 7fe9bae887a0 20 read_log 193305'15140 (193305'15095) modify 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.951337.0:55250 2014-08-11 18:44:43.068617 2014-08-13 17:38:02.811062 7fe9bae887a0 20 read_log 193305'15141 (193305'15097) modify 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.951337.0:57245 2014-08-11 18:44:43.688975 2014-08-13 17:38:02.811074 7fe9bae887a0 20 read_log 193305'15142 (193305'15099) modify 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.951337.0:57796 2014-08-11 18:44:43.862502 2014-08-13 17:38:02.811086 7fe9bae887a0 20 read_log 193305'15143 (193305'15101) modify 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.951337.0:59374 2014-08-11 18:44:44.309973 2014-08-13 17:38:02.811105 7fe9bae887a0 20 read_log 193305'15144 (193305'15103) modify 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.951337.0:60814 2014-08-11 18:44:44.742621 2014-08-13 17:38:02.811118 7fe9bae887a0 20 read_log 193305'15145 (193305'15105) modify 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.951337.0:61558 2014-08-11 18:44:44.962707 2014-08-13 17:38:02.811129 7fe9bae887a0 20 read_log 193305'15146 (193305'15107) modify fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.951337.0:61958 2014-08-11 18:44:45.085374 2014-08-13 17:38:02.811141 7fe9bae887a0 20 read_log 193305'15147 (193305'15109) modify ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.951337.0:62692 2014-08-11 18:44:45.301193 2014-08-13 17:38:02.811166 7fe9bae887a0 20 read_log 193305'15148 (193305'15111) modify f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.951337.0:63491 2014-08-11 18:44:45.522995 2014-08-13 17:38:02.811180 7fe9bae887a0 20 read_log 193305'15149 (193305'15113) modify 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.951337.0:64560 2014-08-11 18:44:45.854132 2014-08-13 17:38:02.811192 7fe9bae887a0 20 read_log 193305'15150 (193305'15115) modify f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.951337.0:65218 2014-08-11 18:44:46.053440 2014-08-13 17:38:02.811204 7fe9bae887a0 20 read_log 194215'15151 (194215'15151) modify b6/hit_set_24.b6_archive_2014-08-12 15:32:25.864388_2014-08-12 15:38:57.656047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:57.656323 2014-08-13 17:38:02.811217 7fe9bae887a0 20 read_log 194215'15152 (193305'15044) delete b6/hit_set_24.b6_archive_2014-08-11 18:37:04.909790_2014-08-11 18:42:37.869502/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:57.656323 2014-08-13 17:38:02.811229 7fe9bae887a0 20 read_log 194215'15153 (192888'15028) clean edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by osd.4.0:1403 2014-08-12 15:38:57.668821 2014-08-13 17:38:02.811241 7fe9bae887a0 20 read_log 194215'15154 (194215'15153) delete edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.2174595.0:37141 0.000000 2014-08-13 17:38:02.811252 7fe9bae887a0 20 read_log 194215'15155 (192888'15015) clean e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by osd.4.0:1405 2014-08-12 15:38:57.678186 2014-08-13 17:38:02.811264 7fe9bae887a0 20 read_log 194215'15156 (194215'15155) delete e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.2174595.0:37143 0.000000 2014-08-13 17:38:02.811275 7fe9bae887a0 20 read_log 194215'15157 (193305'15117) clean e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by osd.4.0:1407 2014-08-12 15:38:57.686239 2014-08-13 17:38:02.811288 7fe9bae887a0 20 read_log 194215'15158 (194215'15157) delete e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.2174595.0:37145 0.000000 2014-08-13 17:38:02.811299 7fe9bae887a0 20 read_log 194215'15159 (193305'15118) clean d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by osd.4.0:1409 2014-08-12 15:38:57.695490 2014-08-13 17:38:02.811311 7fe9bae887a0 20 read_log 194215'15160 (194215'15159) delete d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.2174595.0:37147 0.000000 2014-08-13 17:38:02.811322 7fe9bae887a0 20 read_log 194215'15161 (192888'14864) clean 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by osd.4.0:1411 2014-08-12 15:38:57.703466 2014-08-13 17:38:02.811334 7fe9bae887a0 20 read_log 194215'15162 (194215'15161) delete 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.2174595.0:37149 0.000000 2014-08-13 17:38:02.811355 7fe9bae887a0 20 read_log 194215'15163 (192888'15002) clean a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by osd.4.0:1413 2014-08-12 15:38:57.711308 2014-08-13 17:38:02.811368 7fe9bae887a0 20 read_log 194215'15164 (194215'15163) delete a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.2174595.0:37151 0.000000 2014-08-13 17:38:02.811378 7fe9bae887a0 20 read_log 194215'15165 (192888'14994) clean 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by osd.4.0:1415 2014-08-12 15:38:57.719804 2014-08-13 17:38:02.811391 7fe9bae887a0 20 read_log 194215'15166 (194215'15165) delete 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.2174595.0:37153 0.000000 2014-08-13 17:38:02.811401 7fe9bae887a0 20 read_log 194215'15167 (193305'15133) clean decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by osd.4.0:1417 2014-08-12 15:38:57.747336 2014-08-13 17:38:02.811414 7fe9bae887a0 20 read_log 194215'15168 (194215'15167) delete decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.2174595.0:37155 0.000000 2014-08-13 17:38:02.811424 7fe9bae887a0 20 read_log 194215'15169 (192888'14824) clean f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by osd.4.0:1419 2014-08-12 15:38:57.755711 2014-08-13 17:38:02.811436 7fe9bae887a0 20 read_log 194215'15170 (194215'15169) delete f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.2174595.0:37157 0.000000 2014-08-13 17:38:02.811447 7fe9bae887a0 20 read_log 194215'15171 (193305'15136) clean 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by osd.4.0:1421 2014-08-12 15:38:57.765143 2014-08-13 17:38:02.811459 7fe9bae887a0 20 read_log 194215'15172 (194215'15171) delete 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.2174595.0:37159 0.000000 2014-08-13 17:38:02.811470 7fe9bae887a0 20 read_log 194215'15173 (192888'15030) clean 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by osd.4.0:1423 2014-08-12 15:38:57.774522 2014-08-13 17:38:02.811482 7fe9bae887a0 20 read_log 194215'15174 (194215'15173) delete 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.2174595.0:37161 0.000000 2014-08-13 17:38:02.811492 7fe9bae887a0 20 read_log 194215'15175 (193305'15123) clean 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by osd.4.0:1425 2014-08-12 15:38:57.783729 2014-08-13 17:38:02.811504 7fe9bae887a0 20 read_log 194215'15176 (194215'15175) delete 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.2174595.0:37163 0.000000 2014-08-13 17:38:02.811515 7fe9bae887a0 20 read_log 194215'15177 (192888'15034) clean 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by osd.4.0:1427 2014-08-12 15:38:57.792408 2014-08-13 17:38:02.811527 7fe9bae887a0 20 read_log 194215'15178 (194215'15177) delete 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.2174595.0:37165 0.000000 2014-08-13 17:38:02.811537 7fe9bae887a0 20 read_log 194215'15179 (192888'14823) clean aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by osd.4.0:1429 2014-08-12 15:38:57.801706 2014-08-13 17:38:02.811550 7fe9bae887a0 20 read_log 194215'15180 (194215'15179) delete aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.2174595.0:37167 0.000000 2014-08-13 17:38:02.811560 7fe9bae887a0 20 read_log 194215'15181 (192888'14833) clean 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by osd.4.0:1431 2014-08-12 15:38:57.809961 2014-08-13 17:38:02.811572 7fe9bae887a0 20 read_log 194215'15182 (194215'15181) delete 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.2174595.0:37169 0.000000 2014-08-13 17:38:02.811590 7fe9bae887a0 20 read_log 194215'15183 (193305'15131) clean 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by osd.4.0:1433 2014-08-12 15:38:57.818730 2014-08-13 17:38:02.811603 7fe9bae887a0 20 read_log 194215'15184 (194215'15183) delete 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.2174595.0:37171 0.000000 2014-08-13 17:38:02.811613 7fe9bae887a0 20 read_log 194215'15185 (193305'15138) clean 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by osd.4.0:1435 2014-08-12 15:38:57.827425 2014-08-13 17:38:02.811628 7fe9bae887a0 20 read_log 194215'15186 (194215'15185) delete 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.2174595.0:37173 0.000000 2014-08-13 17:38:02.811638 7fe9bae887a0 20 read_log 194215'15187 (192888'15007) clean 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by osd.4.0:1437 2014-08-12 15:38:57.835508 2014-08-13 17:38:02.811650 7fe9bae887a0 20 read_log 194215'15188 (194215'15187) delete 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.2174595.0:37175 0.000000 2014-08-13 17:38:02.811661 7fe9bae887a0 20 read_log 194215'15189 (192888'14865) clean 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by osd.4.0:1439 2014-08-12 15:38:57.876942 2014-08-13 17:38:02.811673 7fe9bae887a0 20 read_log 194215'15190 (194215'15189) delete 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.2174595.0:37177 0.000000 2014-08-13 17:38:02.811683 7fe9bae887a0 20 read_log 194215'15191 (193305'15129) clean f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by osd.4.0:1441 2014-08-12 15:38:57.886338 2014-08-13 17:38:02.811695 7fe9bae887a0 20 read_log 194215'15192 (194215'15191) delete f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.2174595.0:37179 0.000000 2014-08-13 17:38:02.811706 7fe9bae887a0 20 read_log 194215'15193 (192888'14988) clean d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by osd.4.0:1443 2014-08-12 15:38:57.895832 2014-08-13 17:38:02.811718 7fe9bae887a0 20 read_log 194215'15194 (194215'15193) delete d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.2174595.0:37181 0.000000 2014-08-13 17:38:02.811728 7fe9bae887a0 20 read_log 194215'15195 (193305'15124) clean b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by osd.4.0:1445 2014-08-12 15:38:57.904759 2014-08-13 17:38:02.811740 7fe9bae887a0 20 read_log 194215'15196 (194215'15195) delete b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.2174595.0:37183 0.000000 2014-08-13 17:38:02.811750 7fe9bae887a0 20 read_log 194215'15197 (192888'14814) clean 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by osd.4.0:1447 2014-08-12 15:38:57.914716 2014-08-13 17:38:02.811762 7fe9bae887a0 20 read_log 194215'15198 (194215'15197) delete 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.2174595.0:37185 0.000000 2014-08-13 17:38:02.811773 7fe9bae887a0 20 read_log 194215'15199 (193305'15127) clean 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by osd.4.0:1449 2014-08-12 15:38:57.923559 2014-08-13 17:38:02.811786 7fe9bae887a0 20 read_log 194215'15200 (194215'15199) delete 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.2174595.0:37187 0.000000 2014-08-13 17:38:02.811797 7fe9bae887a0 20 read_log 194215'15201 (192888'15001) clean 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by osd.4.0:1451 2014-08-12 15:38:57.932098 2014-08-13 17:38:02.811809 7fe9bae887a0 20 read_log 194215'15202 (194215'15201) delete 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.2174595.0:37189 0.000000 2014-08-13 17:38:02.811827 7fe9bae887a0 20 read_log 194215'15203 (192888'14995) clean 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by osd.4.0:1453 2014-08-12 15:38:57.941184 2014-08-13 17:38:02.811839 7fe9bae887a0 20 read_log 194215'15204 (194215'15203) delete 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.2174595.0:37191 0.000000 2014-08-13 17:38:02.811868 7fe9bae887a0 20 read_log 194215'15205 (192888'15032) clean 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by osd.4.0:1455 2014-08-12 15:38:57.951395 2014-08-13 17:38:02.811880 7fe9bae887a0 20 read_log 194215'15206 (194215'15205) delete 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.2174595.0:37193 0.000000 2014-08-13 17:38:02.811891 7fe9bae887a0 20 read_log 194215'15207 (192888'14839) clean 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by osd.4.0:1457 2014-08-12 15:38:57.960463 2014-08-13 17:38:02.811903 7fe9bae887a0 20 read_log 194215'15208 (194215'15207) delete 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.2174595.0:37195 0.000000 2014-08-13 17:38:02.811914 7fe9bae887a0 20 read_log 194215'15209 (193305'15119) clean 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by osd.4.0:1459 2014-08-12 15:38:57.970213 2014-08-13 17:38:02.811926 7fe9bae887a0 20 read_log 194215'15210 (194215'15209) delete 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.2174595.0:37197 0.000000 2014-08-13 17:38:02.811936 7fe9bae887a0 20 read_log 194215'15211 (192888'14853) clean fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by osd.4.0:1461 2014-08-12 15:38:57.978808 2014-08-13 17:38:02.811948 7fe9bae887a0 20 read_log 194215'15212 (194215'15211) delete fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.2174595.0:37199 0.000000 2014-08-13 17:38:02.811959 7fe9bae887a0 20 read_log 194215'15213 (192888'15026) clean d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by osd.4.0:1463 2014-08-12 15:38:57.985882 2014-08-13 17:38:02.811971 7fe9bae887a0 20 read_log 194215'15214 (194215'15213) delete d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.2174595.0:37201 0.000000 2014-08-13 17:38:02.811981 7fe9bae887a0 20 read_log 194215'15215 (193305'15145) clean 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by osd.4.0:1465 2014-08-12 15:38:57.993755 2014-08-13 17:38:02.811994 7fe9bae887a0 20 read_log 194215'15216 (194215'15215) delete 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.2174595.0:37203 0.000000 2014-08-13 17:38:02.812004 7fe9bae887a0 20 read_log 194215'15217 (192888'15009) clean 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by osd.4.0:1467 2014-08-12 15:38:58.002083 2014-08-13 17:38:02.812016 7fe9bae887a0 20 read_log 194215'15218 (194215'15217) delete 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.2174595.0:37205 0.000000 2014-08-13 17:38:02.812027 7fe9bae887a0 20 read_log 194215'15219 (193305'15140) clean 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by osd.4.0:1469 2014-08-12 15:38:58.009976 2014-08-13 17:38:02.812039 7fe9bae887a0 20 read_log 194215'15220 (194215'15219) delete 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.2174595.0:37207 0.000000 2014-08-13 17:38:02.812049 7fe9bae887a0 20 read_log 194215'15221 (193305'15134) clean fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by osd.4.0:1471 2014-08-12 15:38:58.018429 2014-08-13 17:38:02.812061 7fe9bae887a0 20 read_log 194215'15222 (194215'15221) delete fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.2174595.0:37209 0.000000 2014-08-13 17:38:02.812081 7fe9bae887a0 20 read_log 194215'15223 (193305'15150) clean f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by osd.4.0:1473 2014-08-12 15:38:58.026679 2014-08-13 17:38:02.812094 7fe9bae887a0 20 read_log 194215'15224 (194215'15223) delete f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.2174595.0:37211 0.000000 2014-08-13 17:38:02.812105 7fe9bae887a0 20 read_log 194215'15225 (192888'15020) clean 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by osd.4.0:1475 2014-08-12 15:38:58.035374 2014-08-13 17:38:02.812117 7fe9bae887a0 20 read_log 194215'15226 (194215'15225) delete 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.2174595.0:37213 0.000000 2014-08-13 17:38:02.812128 7fe9bae887a0 20 read_log 194215'15227 (192888'15031) clean 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by osd.4.0:1477 2014-08-12 15:38:58.043817 2014-08-13 17:38:02.812140 7fe9bae887a0 20 read_log 194215'15228 (194215'15227) delete 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.2174595.0:37215 0.000000 2014-08-13 17:38:02.812150 7fe9bae887a0 20 read_log 194215'15229 (192888'15018) clean 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by osd.4.0:1479 2014-08-12 15:38:58.053127 2014-08-13 17:38:02.812167 7fe9bae887a0 20 read_log 194215'15230 (194215'15229) delete 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.2174595.0:37217 0.000000 2014-08-13 17:38:02.812178 7fe9bae887a0 20 read_log 194215'15231 (193305'15143) clean 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by osd.4.0:1481 2014-08-12 15:38:58.063224 2014-08-13 17:38:02.812193 7fe9bae887a0 20 read_log 194215'15232 (194215'15231) delete 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.2174595.0:37219 0.000000 2014-08-13 17:38:02.812204 7fe9bae887a0 20 read_log 194215'15233 (193305'15148) clean f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by osd.4.0:1483 2014-08-12 15:38:58.071336 2014-08-13 17:38:02.812216 7fe9bae887a0 20 read_log 194215'15234 (194215'15233) delete f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.2174595.0:37221 0.000000 2014-08-13 17:38:02.812226 7fe9bae887a0 20 read_log 194215'15235 (193305'15142) clean 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by osd.4.0:1485 2014-08-12 15:38:58.079702 2014-08-13 17:38:02.812238 7fe9bae887a0 20 read_log 194215'15236 (194215'15235) delete 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.2174595.0:37223 0.000000 2014-08-13 17:38:02.812249 7fe9bae887a0 20 read_log 194215'15237 (192888'15025) clean ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by osd.4.0:1487 2014-08-12 15:38:58.087507 2014-08-13 17:38:02.812261 7fe9bae887a0 20 read_log 194215'15238 (194215'15237) delete ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.2174595.0:37225 0.000000 2014-08-13 17:38:02.812271 7fe9bae887a0 20 read_log 194215'15239 (192888'15008) clean 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by osd.4.0:1489 2014-08-12 15:38:58.096135 2014-08-13 17:38:02.812283 7fe9bae887a0 20 read_log 194215'15240 (194215'15239) delete 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.2174595.0:37227 0.000000 2014-08-13 17:38:02.812294 7fe9bae887a0 20 read_log 194215'15241 (192888'14861) clean 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by osd.4.0:1491 2014-08-12 15:38:58.104188 2014-08-13 17:38:02.812306 7fe9bae887a0 20 read_log 194215'15242 (194215'15241) delete 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.2174595.0:37229 0.000000 2014-08-13 17:38:02.812341 7fe9bae887a0 20 read_log 194215'15243 (193305'15139) clean f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by osd.4.0:1493 2014-08-12 15:38:58.113005 2014-08-13 17:38:02.812355 7fe9bae887a0 20 read_log 194215'15244 (194215'15243) delete f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.2174595.0:37231 0.000000 2014-08-13 17:38:02.812366 7fe9bae887a0 20 read_log 194215'15245 (192888'14843) clean 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by osd.4.0:1495 2014-08-12 15:38:58.122612 2014-08-13 17:38:02.812378 7fe9bae887a0 20 read_log 194215'15246 (194215'15245) delete 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.2174595.0:37233 0.000000 2014-08-13 17:38:02.812389 7fe9bae887a0 20 read_log 194215'15247 (193305'15137) clean dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by osd.4.0:1497 2014-08-12 15:38:58.131445 2014-08-13 17:38:02.812401 7fe9bae887a0 20 read_log 194215'15248 (194215'15247) delete dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.2174595.0:37235 0.000000 2014-08-13 17:38:02.812411 7fe9bae887a0 20 read_log 194215'15249 (193305'15125) clean 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by osd.4.0:1499 2014-08-12 15:38:58.139631 2014-08-13 17:38:02.812424 7fe9bae887a0 20 read_log 194215'15250 (194215'15249) delete 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.2174595.0:37237 0.000000 2014-08-13 17:38:02.812439 7fe9bae887a0 20 read_log 194215'15251 (193305'15128) clean e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by osd.4.0:1501 2014-08-12 15:38:58.174456 2014-08-13 17:38:02.812451 7fe9bae887a0 20 read_log 194215'15252 (194215'15251) delete e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.2174595.0:37239 0.000000 2014-08-13 17:38:02.812462 7fe9bae887a0 20 read_log 194215'15253 (192888'15016) clean 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by osd.4.0:1503 2014-08-12 15:38:58.182506 2014-08-13 17:38:02.812474 7fe9bae887a0 20 read_log 194215'15254 (194215'15253) delete 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.2174595.0:37241 0.000000 2014-08-13 17:38:02.812485 7fe9bae887a0 20 read_log 194215'15255 (193305'15135) clean e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by osd.4.0:1505 2014-08-12 15:38:58.191665 2014-08-13 17:38:02.812497 7fe9bae887a0 20 read_log 194215'15256 (194215'15255) delete e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.2174595.0:37243 0.000000 2014-08-13 17:38:02.812507 7fe9bae887a0 20 read_log 194215'15257 (193305'15126) clean 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by osd.4.0:1507 2014-08-12 15:38:58.216603 2014-08-13 17:38:02.812519 7fe9bae887a0 20 read_log 194215'15258 (194215'15257) delete 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.2174595.0:37245 0.000000 2014-08-13 17:38:02.812530 7fe9bae887a0 20 read_log 194215'15259 (193305'15122) clean 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by osd.4.0:1509 2014-08-12 15:38:58.226718 2014-08-13 17:38:02.812542 7fe9bae887a0 20 read_log 194215'15260 (194215'15259) delete 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.2174595.0:37247 0.000000 2014-08-13 17:38:02.812552 7fe9bae887a0 20 read_log 194215'15261 (193305'15146) clean fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by osd.4.0:1511 2014-08-12 15:38:58.236759 2014-08-13 17:38:02.812564 7fe9bae887a0 20 read_log 194215'15262 (194215'15261) delete fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.2174595.0:37249 0.000000 2014-08-13 17:38:02.812581 7fe9bae887a0 20 read_log 194215'15263 (193305'15130) clean f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by osd.4.0:1513 2014-08-12 15:38:58.245829 2014-08-13 17:38:02.812594 7fe9bae887a0 20 read_log 194215'15264 (194215'15263) delete f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.2174595.0:37251 0.000000 2014-08-13 17:38:02.812605 7fe9bae887a0 20 read_log 194215'15265 (192888'14993) clean a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by osd.4.0:1515 2014-08-12 15:38:58.254554 2014-08-13 17:38:02.812617 7fe9bae887a0 20 read_log 194215'15266 (194215'15265) delete a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.2174595.0:37253 0.000000 2014-08-13 17:38:02.812627 7fe9bae887a0 20 read_log 194215'15267 (192888'14826) clean a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by osd.4.0:1517 2014-08-12 15:38:58.262537 2014-08-13 17:38:02.812639 7fe9bae887a0 20 read_log 194215'15268 (194215'15267) delete a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.2174595.0:37255 0.000000 2014-08-13 17:38:02.812650 7fe9bae887a0 20 read_log 194215'15269 (192888'14990) clean a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by osd.4.0:1519 2014-08-12 15:38:58.270226 2014-08-13 17:38:02.812662 7fe9bae887a0 20 read_log 194215'15270 (194215'15269) delete a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.2174595.0:37257 0.000000 2014-08-13 17:38:02.812672 7fe9bae887a0 20 read_log 194215'15271 (192888'14998) clean 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by osd.4.0:1521 2014-08-12 15:38:58.286524 2014-08-13 17:38:02.812684 7fe9bae887a0 20 read_log 194215'15272 (194215'15271) delete 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.2174595.0:37259 0.000000 2014-08-13 17:38:02.812694 7fe9bae887a0 20 read_log 194215'15273 (192888'14816) clean 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by osd.4.0:1523 2014-08-12 15:38:58.294823 2014-08-13 17:38:02.812706 7fe9bae887a0 20 read_log 194215'15274 (194215'15273) delete 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.2174595.0:37261 0.000000 2014-08-13 17:38:02.812717 7fe9bae887a0 20 read_log 194215'15275 (193305'15116) clean 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by osd.4.0:1525 2014-08-12 15:38:58.303280 2014-08-13 17:38:02.812729 7fe9bae887a0 20 read_log 194215'15276 (194215'15275) delete 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.2174595.0:37263 0.000000 2014-08-13 17:38:02.812739 7fe9bae887a0 20 read_log 194215'15277 (192888'14836) clean 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by osd.4.0:1527 2014-08-12 15:38:58.311520 2014-08-13 17:38:02.812751 7fe9bae887a0 20 read_log 194215'15278 (194215'15277) delete 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.2174595.0:37265 0.000000 2014-08-13 17:38:02.812762 7fe9bae887a0 20 read_log 194215'15279 (193305'15120) clean 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by osd.4.0:1529 2014-08-12 15:38:58.320115 2014-08-13 17:38:02.812774 7fe9bae887a0 20 read_log 194215'15280 (194215'15279) delete 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.2174595.0:37267 0.000000 2014-08-13 17:38:02.812784 7fe9bae887a0 20 read_log 194215'15281 (192888'14842) clean 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by osd.4.0:1531 2014-08-12 15:38:58.329161 2014-08-13 17:38:02.812796 7fe9bae887a0 20 read_log 194215'15282 (194215'15281) delete 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.2174595.0:37269 0.000000 2014-08-13 17:38:02.812813 7fe9bae887a0 20 read_log 194215'15283 (192888'14815) clean 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by osd.4.0:1533 2014-08-12 15:38:58.338138 2014-08-13 17:38:02.812826 7fe9bae887a0 20 read_log 194215'15284 (194215'15283) delete 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.2174595.0:37271 0.000000 2014-08-13 17:38:02.812837 7fe9bae887a0 20 read_log 194215'15285 (192888'14818) clean c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by osd.4.0:1535 2014-08-12 15:38:58.347261 2014-08-13 17:38:02.812849 7fe9bae887a0 20 read_log 194215'15286 (194215'15285) delete c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.2174595.0:37273 0.000000 2014-08-13 17:38:02.812859 7fe9bae887a0 20 read_log 194215'15287 (192888'14821) clean 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by osd.4.0:1537 2014-08-12 15:38:58.356845 2014-08-13 17:38:02.812871 7fe9bae887a0 20 read_log 194215'15288 (194215'15287) delete 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.2174595.0:37275 0.000000 2014-08-13 17:38:02.812882 7fe9bae887a0 20 read_log 194215'15289 (192888'14845) clean 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by osd.4.0:1539 2014-08-12 15:38:58.364730 2014-08-13 17:38:02.812894 7fe9bae887a0 20 read_log 194215'15290 (194215'15289) delete 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.2174595.0:37277 0.000000 2014-08-13 17:38:02.812904 7fe9bae887a0 20 read_log 194215'15291 (192888'15003) clean 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by osd.4.0:1541 2014-08-12 15:38:58.372851 2014-08-13 17:38:02.812916 7fe9bae887a0 20 read_log 194215'15292 (194215'15291) delete 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.2174595.0:37279 0.000000 2014-08-13 17:38:02.812926 7fe9bae887a0 20 read_log 194215'15293 (192888'14838) clean 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by osd.4.0:1543 2014-08-12 15:38:58.380390 2014-08-13 17:38:02.812938 7fe9bae887a0 20 read_log 194215'15294 (194215'15293) delete 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.2174595.0:37281 0.000000 2014-08-13 17:38:02.812948 7fe9bae887a0 20 read_log 194215'15295 (192888'15000) clean ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by osd.4.0:1545 2014-08-12 15:38:58.390547 2014-08-13 17:38:02.812960 7fe9bae887a0 20 read_log 194215'15296 (194215'15295) delete ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.2174595.0:37283 0.000000 2014-08-13 17:38:02.812970 7fe9bae887a0 20 read_log 194215'15297 (193305'15144) clean 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by osd.4.0:1547 2014-08-12 15:38:58.399686 2014-08-13 17:38:02.813002 7fe9bae887a0 20 read_log 194215'15298 (194215'15297) delete 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.2174595.0:37285 0.000000 2014-08-13 17:38:02.813013 7fe9bae887a0 20 read_log 194215'15299 (192888'14858) clean 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by osd.4.0:1549 2014-08-12 15:38:58.427096 2014-08-13 17:38:02.813025 7fe9bae887a0 20 read_log 194215'15300 (194215'15299) delete 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.2174595.0:37287 0.000000 2014-08-13 17:38:02.813036 7fe9bae887a0 20 read_log 194215'15301 (193305'15121) clean 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by osd.4.0:1551 2014-08-12 15:38:58.436635 2014-08-13 17:38:02.813049 7fe9bae887a0 20 read_log 194215'15302 (194215'15301) delete 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.2174595.0:37289 0.000000 2014-08-13 17:38:02.813069 7fe9bae887a0 20 read_log 194215'15303 (192888'15023) clean 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by osd.4.0:1553 2014-08-12 15:38:58.445032 2014-08-13 17:38:02.813082 7fe9bae887a0 20 read_log 194215'15304 (194215'15303) delete 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.2174595.0:37291 0.000000 2014-08-13 17:38:02.813093 7fe9bae887a0 20 read_log 194215'15305 (193305'15149) clean 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by osd.4.0:1555 2014-08-12 15:38:58.453379 2014-08-13 17:38:02.813105 7fe9bae887a0 20 read_log 194215'15306 (194215'15305) delete 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.2174595.0:37293 0.000000 2014-08-13 17:38:02.813115 7fe9bae887a0 20 read_log 194215'15307 (192888'15014) clean 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by osd.4.0:1557 2014-08-12 15:38:58.461365 2014-08-13 17:38:02.813127 7fe9bae887a0 20 read_log 194215'15308 (194215'15307) delete 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.2174595.0:37295 0.000000 2014-08-13 17:38:02.813138 7fe9bae887a0 20 read_log 194215'15309 (192888'15011) clean 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by osd.4.0:1559 2014-08-12 15:38:58.468593 2014-08-13 17:38:02.813150 7fe9bae887a0 20 read_log 194215'15310 (194215'15309) delete 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.2174595.0:37297 0.000000 2014-08-13 17:38:02.813166 7fe9bae887a0 20 read_log 194215'15311 (192888'15036) clean f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by osd.4.0:1561 2014-08-12 15:38:58.477665 2014-08-13 17:38:02.813178 7fe9bae887a0 20 read_log 194215'15312 (194215'15311) delete f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.2174595.0:37299 0.000000 2014-08-13 17:38:02.813189 7fe9bae887a0 20 read_log 194215'15313 (192888'15035) clean d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by osd.4.0:1563 2014-08-12 15:38:58.485360 2014-08-13 17:38:02.813201 7fe9bae887a0 20 read_log 194215'15314 (194215'15313) delete d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.2174595.0:37301 0.000000 2014-08-13 17:38:02.813211 7fe9bae887a0 20 read_log 194215'15315 (193305'15132) clean 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by osd.4.0:1565 2014-08-12 15:38:58.495111 2014-08-13 17:38:02.813223 7fe9bae887a0 20 read_log 194215'15316 (194215'15315) delete 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.2174595.0:37303 0.000000 2014-08-13 17:38:02.813234 7fe9bae887a0 20 read_log 194215'15317 (192888'14996) clean 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by osd.4.0:1567 2014-08-12 15:38:58.504474 2014-08-13 17:38:02.813246 7fe9bae887a0 20 read_log 194215'15318 (194215'15317) delete 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.2174595.0:37305 0.000000 2014-08-13 17:38:02.813256 7fe9bae887a0 20 read_log 194215'15319 (193305'15147) clean ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by osd.4.0:1569 2014-08-12 15:38:58.513240 2014-08-13 17:38:02.813268 7fe9bae887a0 20 read_log 194215'15320 (194215'15319) delete ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.2174595.0:37307 0.000000 2014-08-13 17:38:02.813279 7fe9bae887a0 20 read_log 194215'15321 (192888'15038) clean 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by osd.4.0:1571 2014-08-12 15:38:58.520583 2014-08-13 17:38:02.813291 7fe9bae887a0 20 read_log 194215'15322 (194215'15321) delete 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.2174595.0:37309 0.000000 2014-08-13 17:38:02.813309 7fe9bae887a0 20 read_log 194215'15323 (192888'14844) clean d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by osd.4.0:1573 2014-08-12 15:38:58.529404 2014-08-13 17:38:02.813322 7fe9bae887a0 20 read_log 194215'15324 (194215'15323) delete d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.2174595.0:37311 0.000000 2014-08-13 17:38:02.813333 7fe9bae887a0 20 read_log 194215'15325 (193305'15141) clean 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by osd.4.0:1575 2014-08-12 15:38:58.538415 2014-08-13 17:38:02.813345 7fe9bae887a0 20 read_log 194215'15326 (194215'15325) delete 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.2174595.0:37313 0.000000 2014-08-13 17:38:02.813355 7fe9bae887a0 20 read_log 194215'15327 (194215'15327) modify b6/hit_set_24.b6_archive_2014-08-12 15:38:57.656327_2014-08-12 17:24:42.218256/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.219403 2014-08-13 17:38:02.813368 7fe9bae887a0 20 read_log 194215'15328 (194215'15151) delete b6/hit_set_24.b6_archive_2014-08-12 15:32:25.864388_2014-08-12 15:38:57.656047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.219403 2014-08-13 17:38:02.813383 7fe9bae887a0 20 read_log 194215'15329 (0'0) promote e189c8b6/benchmark_data_storage0111-ib.pocstor_34628_object591/head//24 by osd.4.0:5238 2014-08-12 17:24:42.225151 2014-08-13 17:38:02.813396 7fe9bae887a0 20 read_log 194215'15330 (194215'15329) modify e189c8b6/benchmark_data_storage0111-ib.pocstor_34628_object591/head//24 by client.2193759.0:592 2014-08-12 17:24:42.212099 2014-08-13 17:38:02.813408 7fe9bae887a0 20 read_log 194216'15331 (0'0) promote 16e6e8b6/benchmark_data_storage0111-ib.pocstor_39693_object25/head//24 by osd.4.0:5252 2014-08-12 17:25:03.272073 2014-08-13 17:38:02.813420 7fe9bae887a0 20 read_log 194216'15332 (194216'15331) modify 16e6e8b6/benchmark_data_storage0111-ib.pocstor_39693_object25/head//24 by client.1487927.0:26 2014-08-12 17:25:03.260972 2014-08-13 17:38:02.813432 7fe9bae887a0 20 read_log 194216'15333 (0'0) promote 36319cb6/benchmark_data_storage0111-ib.pocstor_39693_object30/head//24 by osd.4.0:5254 2014-08-12 17:25:03.300761 2014-08-13 17:38:02.813444 7fe9bae887a0 20 read_log 194216'15334 (194216'15333) modify 36319cb6/benchmark_data_storage0111-ib.pocstor_39693_object30/head//24 by client.1487927.0:31 2014-08-12 17:25:03.293269 2014-08-13 17:38:02.813457 7fe9bae887a0 20 read_log 194370'15335 (0'0) promote 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by osd.327.0:1623 2014-08-12 20:30:15.938917 2014-08-13 17:38:02.813469 7fe9bae887a0 20 read_log 194370'15336 (194370'15335) modify 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by client.1508531.0:410 2014-08-12 20:30:15.927529 2014-08-13 17:38:02.813481 7fe9bae887a0 20 read_log 194370'15337 (0'0) promote fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by osd.327.0:1631 2014-08-12 20:30:18.646776 2014-08-13 17:38:02.813492 7fe9bae887a0 20 read_log 194370'15338 (194370'15337) modify fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by client.1508531.0:1144 2014-08-12 20:30:18.640063 2014-08-13 17:38:02.813504 7fe9bae887a0 20 read_log 194370'15339 (0'0) promote e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by osd.327.0:1635 2014-08-12 20:30:19.404330 2014-08-13 17:38:02.813516 7fe9bae887a0 20 read_log 194370'15340 (194370'15339) modify e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by client.1508531.0:1338 2014-08-12 20:30:19.397975 2014-08-13 17:38:02.813528 7fe9bae887a0 20 read_log 194370'15341 (0'0) promote 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by osd.327.0:1637 2014-08-12 20:30:19.453885 2014-08-13 17:38:02.813547 7fe9bae887a0 20 read_log 194370'15342 (194370'15341) modify 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by client.1508531.0:1352 2014-08-12 20:30:19.448773 2014-08-13 17:38:02.813560 7fe9bae887a0 20 read_log 194370'15343 (0'0) promote 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by osd.327.0:1647 2014-08-12 20:30:22.819535 2014-08-13 17:38:02.813572 7fe9bae887a0 20 read_log 194370'15344 (194370'15343) modify 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by client.1508531.0:2249 2014-08-12 20:30:22.811865 2014-08-13 17:38:02.813584 7fe9bae887a0 20 read_log 194370'15345 (0'0) promote b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by osd.327.0:1655 2014-08-12 20:30:28.364565 2014-08-13 17:38:02.813596 7fe9bae887a0 20 read_log 194370'15346 (194370'15345) modify b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by client.1508531.0:3744 2014-08-12 20:30:28.357002 2014-08-13 17:38:02.813608 7fe9bae887a0 20 read_log 194370'15347 (0'0) promote 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by osd.327.0:1657 2014-08-12 20:30:29.595741 2014-08-13 17:38:02.813619 7fe9bae887a0 20 read_log 194370'15348 (194370'15347) modify 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by client.1508531.0:4081 2014-08-12 20:30:29.589242 2014-08-13 17:38:02.813631 7fe9bae887a0 20 read_log 194370'15349 (0'0) promote 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by osd.327.0:1659 2014-08-12 20:30:29.601178 2014-08-13 17:38:02.813645 7fe9bae887a0 20 read_log 194370'15350 (194370'15349) modify 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by client.1508531.0:4082 2014-08-12 20:30:29.592706 2014-08-13 17:38:02.813657 7fe9bae887a0 20 read_log 194370'15351 (0'0) promote c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by osd.327.0:1661 2014-08-12 20:30:29.740578 2014-08-13 17:38:02.813669 7fe9bae887a0 20 read_log 194370'15352 (194370'15351) modify c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by client.1508531.0:4122 2014-08-12 20:30:29.734240 2014-08-13 17:38:02.813681 7fe9bae887a0 20 read_log 194370'15353 (0'0) promote 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by osd.327.0:1685 2014-08-12 20:30:40.074130 2014-08-13 17:38:02.813693 7fe9bae887a0 20 read_log 194370'15354 (194370'15353) modify 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by client.1508531.0:6520 2014-08-12 20:30:40.062956 2014-08-13 17:38:02.813705 7fe9bae887a0 20 read_log 194370'15355 (0'0) promote 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by osd.327.0:1691 2014-08-12 20:30:44.837155 2014-08-13 17:38:02.813717 7fe9bae887a0 20 read_log 194370'15356 (194370'15355) modify 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by client.1508531.0:7494 2014-08-12 20:30:44.827180 2014-08-13 17:38:02.813729 7fe9bae887a0 20 read_log 194370'15357 (0'0) promote 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by osd.327.0:1697 2014-08-12 20:30:50.203149 2014-08-13 17:38:02.813740 7fe9bae887a0 20 read_log 194370'15358 (194370'15357) modify 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by client.1508531.0:8494 2014-08-12 20:30:50.192733 2014-08-13 17:38:02.813752 7fe9bae887a0 20 read_log 194370'15359 (0'0) promote b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by osd.327.0:1715 2014-08-12 20:31:02.215713 2014-08-13 17:38:02.813764 7fe9bae887a0 20 read_log 194370'15360 (194370'15359) modify b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by client.1508531.0:10498 2014-08-12 20:31:02.203948 2014-08-13 17:38:02.813776 7fe9bae887a0 20 read_log 194370'15361 (0'0) promote dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by osd.327.0:1719 2014-08-12 20:31:04.537008 2014-08-13 17:38:02.813798 7fe9bae887a0 20 read_log 194370'15362 (194370'15361) modify dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by client.1508531.0:10874 2014-08-12 20:31:04.526909 2014-08-13 17:38:02.813811 7fe9bae887a0 20 read_log 194370'15363 (0'0) promote e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by osd.327.0:1733 2014-08-12 20:31:21.631971 2014-08-13 17:38:02.813823 7fe9bae887a0 20 read_log 194370'15364 (194370'15363) modify e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by client.1508531.0:13235 2014-08-12 20:31:21.622526 2014-08-13 17:38:02.813835 7fe9bae887a0 20 read_log 194370'15365 (0'0) promote b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by osd.327.0:1739 2014-08-12 20:31:24.446726 2014-08-13 17:38:02.813847 7fe9bae887a0 20 read_log 194370'15366 (194370'15365) modify b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by client.1508531.0:13533 2014-08-12 20:31:24.438250 2014-08-13 17:38:02.813859 7fe9bae887a0 20 read_log 194370'15367 (0'0) promote fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by osd.327.0:1747 2014-08-12 20:31:27.309446 2014-08-13 17:38:02.813871 7fe9bae887a0 20 read_log 194370'15368 (194370'15367) modify fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by client.1508531.0:13802 2014-08-12 20:31:27.300048 2014-08-13 17:38:02.813883 7fe9bae887a0 20 read_log 194377'15369 (0'0) promote cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by osd.17.0:5412 2014-08-12 20:32:59.757231 2014-08-13 17:38:02.813895 7fe9bae887a0 20 read_log 194377'15370 (194377'15369) modify cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by client.2231190.0:592 2014-08-12 20:32:59.748956 2014-08-13 17:38:02.813907 7fe9bae887a0 20 read_log 194377'15371 (0'0) promote a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by osd.17.0:5422 2014-08-12 20:33:01.245813 2014-08-13 17:38:02.813919 7fe9bae887a0 20 read_log 194377'15372 (194377'15371) modify a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by client.2231190.0:985 2014-08-12 20:33:01.237895 2014-08-13 17:38:02.813931 7fe9bae887a0 20 read_log 194377'15373 (0'0) promote 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by osd.17.0:5426 2014-08-12 20:33:02.370901 2014-08-13 17:38:02.813943 7fe9bae887a0 20 read_log 194377'15374 (194377'15373) modify 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by client.2231190.0:1280 2014-08-12 20:33:02.364322 2014-08-13 17:38:02.813955 7fe9bae887a0 20 read_log 194377'15375 (0'0) promote 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by osd.17.0:5452 2014-08-12 20:33:06.970592 2014-08-13 17:38:02.813967 7fe9bae887a0 20 read_log 194377'15376 (194377'15375) modify 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by client.2231190.0:2437 2014-08-12 20:33:06.962690 2014-08-13 17:38:02.813979 7fe9bae887a0 20 read_log 194377'15377 (0'0) promote 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by osd.17.0:5482 2014-08-12 20:33:13.599024 2014-08-13 17:38:02.813991 7fe9bae887a0 20 read_log 194377'15378 (194377'15377) modify 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by client.2231190.0:3944 2014-08-12 20:33:13.590004 2014-08-13 17:38:02.814003 7fe9bae887a0 20 read_log 194377'15379 (0'0) promote 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by osd.17.0:5506 2014-08-12 20:33:19.504028 2014-08-13 17:38:02.814015 7fe9bae887a0 20 read_log 194377'15380 (194377'15379) modify 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by client.2231190.0:4805 2014-08-12 20:33:19.495440 2014-08-13 17:38:02.814027 7fe9bae887a0 20 read_log 194380'15381 (0'0) promote 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by osd.17.0:5510 2014-08-12 20:33:27.400731 2014-08-13 17:38:02.814063 7fe9bae887a0 20 read_log 194380'15382 (194380'15381) modify 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by client.2231190.0:5318 2014-08-12 20:33:27.391945 2014-08-13 17:38:02.814077 7fe9bae887a0 20 read_log 194383'15383 (0'0) promote 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by osd.17.0:5526 2014-08-12 20:34:00.632129 2014-08-13 17:38:02.814089 7fe9bae887a0 20 read_log 194383'15384 (194383'15383) modify 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by client.2231190.0:5832 2014-08-12 20:34:00.622639 2014-08-13 17:38:02.814102 7fe9bae887a0 20 read_log 194393'15385 (194393'15385) modify b6/hit_set_24.b6_archive_2014-08-12 20:37:09.067202_2014-08-12 20:52:32.940397/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.941630 2014-08-13 17:38:02.814114 7fe9bae887a0 20 read_log 194393'15386 (194215'15327) delete b6/hit_set_24.b6_archive_2014-08-12 15:38:57.656327_2014-08-12 17:24:42.218256/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.941630 2014-08-13 17:38:02.814126 7fe9bae887a0 20 read_log 194393'15387 (0'0) promote 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by osd.327.0:1789 2014-08-12 20:52:32.947640 2014-08-13 17:38:02.814139 7fe9bae887a0 20 read_log 194393'15388 (194393'15387) modify 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by client.1064422.0:105 2014-08-12 20:52:32.928439 2014-08-13 17:38:02.814151 7fe9bae887a0 20 read_log 194393'15389 (0'0) promote 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by osd.327.0:1793 2014-08-12 20:52:33.176956 2014-08-13 17:38:02.814168 7fe9bae887a0 20 read_log 194393'15390 (194393'15389) modify 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by client.1064422.0:163 2014-08-12 20:52:33.171180 2014-08-13 17:38:02.814199 7fe9bae887a0 20 read_log 194396'15391 (0'0) promote 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by osd.327.0:1801 2014-08-12 20:53:00.273218 2014-08-13 17:38:02.814211 7fe9bae887a0 20 read_log 194396'15392 (194396'15391) modify 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by client.2235069.0:278 2014-08-12 20:53:00.005661 2014-08-13 17:38:02.814224 7fe9bae887a0 20 read_log 194416'15393 (194416'15393) modify b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 17:38:02.814236 7fe9bae887a0 20 read_log 194416'15394 (194393'15385) delete b6/hit_set_24.b6_archive_2014-08-12 20:37:09.067202_2014-08-12 20:52:32.940397/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 17:38:02.814249 7fe9bae887a0 20 read_log 194416'15395 (0'0) promote 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by osd.327.0:1929 2014-08-13 15:28:32.216485 2014-08-13 17:38:02.814261 7fe9bae887a0 20 read_log 194416'15396 (194416'15395) modify 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by client.2441958.0:456 2014-08-13 15:28:32.200243 2014-08-13 17:38:02.814273 7fe9bae887a0 20 read_log 194416'15397 (0'0) promote 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by osd.327.0:1934 2014-08-13 15:28:37.790278 2014-08-13 17:38:02.814285 7fe9bae887a0 20 read_log 194416'15398 (194416'15397) modify 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by client.2441958.0:1866 2014-08-13 15:28:37.782074 2014-08-13 17:38:02.814297 7fe9bae887a0 20 read_log 194416'15399 (0'0) promote c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by osd.327.0:1944 2014-08-13 15:28:39.841187 2014-08-13 17:38:02.814309 7fe9bae887a0 20 read_log 194416'15400 (194416'15399) modify c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by client.2441958.0:2410 2014-08-13 15:28:39.832496 2014-08-13 17:38:02.814331 7fe9bae887a0 20 read_log 194416'15401 (0'0) promote 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by osd.327.0:1946 2014-08-13 15:28:41.727376 2014-08-13 17:38:02.814344 7fe9bae887a0 20 read_log 194416'15402 (194416'15401) modify 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by client.2441958.0:2906 2014-08-13 15:28:41.719393 2014-08-13 17:38:02.814356 7fe9bae887a0 20 read_log 194416'15403 (0'0) promote d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by osd.327.0:1952 2014-08-13 15:28:45.807321 2014-08-13 17:38:02.814368 7fe9bae887a0 20 read_log 194416'15404 (194416'15403) modify d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by client.2441958.0:3967 2014-08-13 15:28:45.798504 2014-08-13 17:38:02.814380 7fe9bae887a0 20 read_log 194416'15405 (0'0) promote 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by osd.327.0:1956 2014-08-13 15:28:51.717904 2014-08-13 17:38:02.814391 7fe9bae887a0 20 read_log 194416'15406 (194416'15405) modify 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by client.2441958.0:5479 2014-08-13 15:28:51.708570 2014-08-13 17:38:02.814404 7fe9bae887a0 20 read_log 194416'15407 (0'0) promote 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by osd.327.0:1958 2014-08-13 15:28:52.976478 2014-08-13 17:38:02.814416 7fe9bae887a0 20 read_log 194416'15408 (194416'15407) modify 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by client.2441958.0:5799 2014-08-13 15:28:52.967429 2014-08-13 17:38:02.814428 7fe9bae887a0 20 read_log 194416'15409 (0'0) promote bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by osd.327.0:1966 2014-08-13 15:28:59.632837 2014-08-13 17:38:02.814439 7fe9bae887a0 20 read_log 194416'15410 (194416'15409) modify bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by client.2441958.0:7494 2014-08-13 15:28:59.623501 2014-08-13 17:38:02.814451 7fe9bae887a0 20 read_log 194416'15411 (0'0) promote aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by osd.327.0:1970 2014-08-13 15:29:01.575004 2014-08-13 17:38:02.814464 7fe9bae887a0 20 read_log 194416'15412 (194416'15411) modify aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by client.2441958.0:8011 2014-08-13 15:29:01.566132 2014-08-13 17:38:02.814477 7fe9bae887a0 20 read_log 194416'15413 (0'0) promote 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by osd.327.0:1974 2014-08-13 15:29:02.993201 2014-08-13 17:38:02.814489 7fe9bae887a0 20 read_log 194416'15414 (194416'15413) modify 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by client.2441958.0:8386 2014-08-13 15:29:02.986273 2014-08-13 17:38:02.814501 7fe9bae887a0 20 read_log 194416'15415 (0'0) promote 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by osd.327.0:1980 2014-08-13 15:29:06.514683 2014-08-13 17:38:02.814513 7fe9bae887a0 20 read_log 194416'15416 (194416'15415) modify 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by client.2441958.0:9272 2014-08-13 15:29:06.507029 2014-08-13 17:38:02.814525 7fe9bae887a0 20 read_log 194416'15417 (0'0) promote ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by osd.327.0:1982 2014-08-13 15:29:06.962728 2014-08-13 17:38:02.814537 7fe9bae887a0 20 read_log 194416'15418 (194416'15417) modify ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by client.2441958.0:9391 2014-08-13 15:29:06.953835 2014-08-13 17:38:02.814549 7fe9bae887a0 20 read_log 194416'15419 (0'0) promote 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by osd.327.0:1992 2014-08-13 15:29:11.741815 2014-08-13 17:38:02.814561 7fe9bae887a0 20 read_log 194416'15420 (194416'15419) modify 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by client.2441958.0:10581 2014-08-13 15:29:11.731226 2014-08-13 17:38:02.814580 7fe9bae887a0 20 read_log 194416'15421 (0'0) promote 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by osd.327.0:2000 2014-08-13 15:29:13.525661 2014-08-13 17:38:02.814592 7fe9bae887a0 20 read_log 194416'15422 (194416'15421) modify 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by client.2441958.0:11054 2014-08-13 15:29:13.516980 2014-08-13 17:38:02.814605 7fe9bae887a0 20 read_log 194416'15423 (0'0) promote a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by osd.327.0:2002 2014-08-13 15:29:15.854398 2014-08-13 17:38:02.814616 7fe9bae887a0 20 read_log 194416'15424 (194416'15423) modify a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by client.2441958.0:11584 2014-08-13 15:29:15.843703 2014-08-13 17:38:02.814628 7fe9bae887a0 20 read_log 194416'15425 (0'0) promote 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by osd.327.0:2004 2014-08-13 15:29:17.088086 2014-08-13 17:38:02.814641 7fe9bae887a0 20 read_log 194416'15426 (194416'15425) modify 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by client.2441958.0:11893 2014-08-13 15:29:17.077699 2014-08-13 17:38:02.814653 7fe9bae887a0 20 read_log 194416'15427 (0'0) promote 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by osd.327.0:2014 2014-08-13 15:29:25.123245 2014-08-13 17:38:02.814665 7fe9bae887a0 20 read_log 194416'15428 (194416'15427) modify 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by client.2441958.0:13729 2014-08-13 15:29:25.112446 2014-08-13 17:38:02.814677 7fe9bae887a0 20 read_log 194416'15429 (0'0) promote d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by osd.327.0:2018 2014-08-13 15:29:26.626288 2014-08-13 17:38:02.814689 7fe9bae887a0 20 read_log 194416'15430 (194416'15429) modify d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by client.2441958.0:14110 2014-08-13 15:29:26.616854 2014-08-13 17:38:02.814701 7fe9bae887a0 20 read_log 194416'15431 (0'0) promote ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by osd.327.0:2022 2014-08-13 15:29:30.499997 2014-08-13 17:38:02.814712 7fe9bae887a0 20 read_log 194416'15432 (194416'15431) modify ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by client.2441958.0:15047 2014-08-13 15:29:30.488617 2014-08-13 17:38:02.814724 7fe9bae887a0 20 read_log 194416'15433 (0'0) promote e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by osd.327.0:2024 2014-08-13 15:29:30.755553 2014-08-13 17:38:02.814736 7fe9bae887a0 20 read_log 194416'15434 (194416'15433) modify e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by client.2441958.0:15104 2014-08-13 15:29:30.745232 2014-08-13 17:38:02.814748 7fe9bae887a0 20 read_log 194416'15435 (0'0) promote 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by osd.327.0:2026 2014-08-13 15:29:32.497149 2014-08-13 17:38:02.814760 7fe9bae887a0 20 read_log 194416'15436 (194416'15435) modify 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by client.2441958.0:15457 2014-08-13 15:29:32.489527 2014-08-13 17:38:02.814772 7fe9bae887a0 20 read_log 194416'15437 (0'0) promote 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by osd.327.0:2040 2014-08-13 15:30:00.320034 2014-08-13 17:38:02.814784 7fe9bae887a0 20 read_log 194416'15438 (194416'15437) modify 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by client.2441958.0:17918 2014-08-13 15:30:00.308877 2014-08-13 17:38:02.814796 7fe9bae887a0 20 read_log 194416'15439 (0'0) promote 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by osd.327.0:2042 2014-08-13 15:30:00.483514 2014-08-13 17:38:02.814808 7fe9bae887a0 20 read_log 194416'15440 (194416'15439) modify 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by client.2441958.0:17930 2014-08-13 15:30:00.474134 2014-08-13 17:38:02.814827 7fe9bae887a0 20 read_log 194422'15441 (0'0) promote 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by osd.327.0:2066 2014-08-13 15:32:26.590505 2014-08-13 17:38:02.814840 7fe9bae887a0 20 read_log 194422'15442 (194422'15441) modify 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by client.1148203.0:2867 2014-08-13 15:32:26.581398 2014-08-13 17:38:02.814852 7fe9bae887a0 20 read_log 194422'15443 (0'0) promote b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by osd.327.0:2072 2014-08-13 15:32:30.001397 2014-08-13 17:38:02.814864 7fe9bae887a0 20 read_log 194422'15444 (194422'15443) modify b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by client.1148203.0:3736 2014-08-13 15:32:29.992429 2014-08-13 17:38:02.814876 7fe9bae887a0 20 read_log 194422'15445 (0'0) promote be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by osd.327.0:2074 2014-08-13 15:32:30.689989 2014-08-13 17:38:02.814888 7fe9bae887a0 20 read_log 194422'15446 (194422'15445) modify be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by client.1148203.0:3922 2014-08-13 15:32:30.682233 2014-08-13 17:38:02.814900 7fe9bae887a0 20 read_log 194422'15447 (0'0) promote bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by osd.327.0:2088 2014-08-13 15:32:50.050377 2014-08-13 17:38:02.814913 7fe9bae887a0 20 read_log 194422'15448 (194422'15447) modify bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by client.1148203.0:7727 2014-08-13 15:32:50.037724 2014-08-13 17:38:02.814925 7fe9bae887a0 20 read_log 194422'15449 (0'0) promote 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by osd.327.0:2100 2014-08-13 15:33:05.440521 2014-08-13 17:38:02.814937 7fe9bae887a0 20 read_log 194422'15450 (194422'15449) modify 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by client.1148203.0:9599 2014-08-13 15:33:05.430555 2014-08-13 17:38:02.814949 7fe9bae887a0 20 read_log 194422'15451 (0'0) promote c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by osd.327.0:2102 2014-08-13 15:33:07.223246 2014-08-13 17:38:02.814961 7fe9bae887a0 20 read_log 194422'15452 (194422'15451) modify c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by client.1148203.0:9755 2014-08-13 15:33:07.212684 2014-08-13 17:38:02.814973 7fe9bae887a0 20 read_log 194422'15453 (0'0) promote 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by osd.327.0:2106 2014-08-13 15:33:26.017816 2014-08-13 17:38:02.814985 7fe9bae887a0 20 read_log 194422'15454 (194422'15453) modify 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by client.1148203.0:10574 2014-08-13 15:33:26.009369 2014-08-13 17:38:02.815000 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.815362 7fe9bae887a0 10 read_log checking for missing items over interval (194370'15335,194422'15454] 2014-08-13 17:38:02.815406 7fe9bae887a0 15 read_log missing 194422'15454 (194422'15453) modify 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by client.1148203.0:10574 2014-08-13 15:33:26.009369 2014-08-13 17:38:02.815449 7fe9bae887a0 15 read_log missing 194422'15452 (194422'15451) modify c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by client.1148203.0:9755 2014-08-13 15:33:07.212684 2014-08-13 17:38:02.815485 7fe9bae887a0 15 read_log missing 194422'15450 (194422'15449) modify 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by client.1148203.0:9599 2014-08-13 15:33:05.430555 2014-08-13 17:38:02.815524 7fe9bae887a0 15 read_log missing 194422'15448 (194422'15447) modify bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by client.1148203.0:7727 2014-08-13 15:32:50.037724 2014-08-13 17:38:02.815559 7fe9bae887a0 15 read_log missing 194422'15446 (194422'15445) modify be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by client.1148203.0:3922 2014-08-13 15:32:30.682233 2014-08-13 17:38:02.815593 7fe9bae887a0 15 read_log missing 194422'15444 (194422'15443) modify b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by client.1148203.0:3736 2014-08-13 15:32:29.992429 2014-08-13 17:38:02.815628 7fe9bae887a0 15 read_log missing 194422'15442 (194422'15441) modify 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by client.1148203.0:2867 2014-08-13 15:32:26.581398 2014-08-13 17:38:02.815662 7fe9bae887a0 15 read_log missing 194416'15440 (194416'15439) modify 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by client.2441958.0:17930 2014-08-13 15:30:00.474134 2014-08-13 17:38:02.815696 7fe9bae887a0 15 read_log missing 194416'15438 (194416'15437) modify 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by client.2441958.0:17918 2014-08-13 15:30:00.308877 2014-08-13 17:38:02.815731 7fe9bae887a0 15 read_log missing 194416'15436 (194416'15435) modify 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by client.2441958.0:15457 2014-08-13 15:29:32.489527 2014-08-13 17:38:02.815765 7fe9bae887a0 15 read_log missing 194416'15434 (194416'15433) modify e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by client.2441958.0:15104 2014-08-13 15:29:30.745232 2014-08-13 17:38:02.815800 7fe9bae887a0 15 read_log missing 194416'15432 (194416'15431) modify ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by client.2441958.0:15047 2014-08-13 15:29:30.488617 2014-08-13 17:38:02.815834 7fe9bae887a0 15 read_log missing 194416'15430 (194416'15429) modify d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by client.2441958.0:14110 2014-08-13 15:29:26.616854 2014-08-13 17:38:02.815868 7fe9bae887a0 15 read_log missing 194416'15428 (194416'15427) modify 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by client.2441958.0:13729 2014-08-13 15:29:25.112446 2014-08-13 17:38:02.815902 7fe9bae887a0 15 read_log missing 194416'15426 (194416'15425) modify 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by client.2441958.0:11893 2014-08-13 15:29:17.077699 2014-08-13 17:38:02.815937 7fe9bae887a0 15 read_log missing 194416'15424 (194416'15423) modify a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by client.2441958.0:11584 2014-08-13 15:29:15.843703 2014-08-13 17:38:02.815971 7fe9bae887a0 15 read_log missing 194416'15422 (194416'15421) modify 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by client.2441958.0:11054 2014-08-13 15:29:13.516980 2014-08-13 17:38:02.816008 7fe9bae887a0 15 read_log missing 194416'15420 (194416'15419) modify 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by client.2441958.0:10581 2014-08-13 15:29:11.731226 2014-08-13 17:38:02.816045 7fe9bae887a0 15 read_log missing 194416'15418 (194416'15417) modify ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by client.2441958.0:9391 2014-08-13 15:29:06.953835 2014-08-13 17:38:02.816082 7fe9bae887a0 15 read_log missing 194416'15416 (194416'15415) modify 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by client.2441958.0:9272 2014-08-13 15:29:06.507029 2014-08-13 17:38:02.816118 7fe9bae887a0 15 read_log missing 194416'15414 (194416'15413) modify 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by client.2441958.0:8386 2014-08-13 15:29:02.986273 2014-08-13 17:38:02.816160 7fe9bae887a0 15 read_log missing 194416'15412 (194416'15411) modify aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by client.2441958.0:8011 2014-08-13 15:29:01.566132 2014-08-13 17:38:02.816204 7fe9bae887a0 15 read_log missing 194416'15410 (194416'15409) modify bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by client.2441958.0:7494 2014-08-13 15:28:59.623501 2014-08-13 17:38:02.816240 7fe9bae887a0 15 read_log missing 194416'15408 (194416'15407) modify 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by client.2441958.0:5799 2014-08-13 15:28:52.967429 2014-08-13 17:38:02.816274 7fe9bae887a0 15 read_log missing 194416'15406 (194416'15405) modify 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by client.2441958.0:5479 2014-08-13 15:28:51.708570 2014-08-13 17:38:02.816308 7fe9bae887a0 15 read_log missing 194416'15404 (194416'15403) modify d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by client.2441958.0:3967 2014-08-13 15:28:45.798504 2014-08-13 17:38:02.816342 7fe9bae887a0 15 read_log missing 194416'15402 (194416'15401) modify 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by client.2441958.0:2906 2014-08-13 15:28:41.719393 2014-08-13 17:38:02.816377 7fe9bae887a0 15 read_log missing 194416'15400 (194416'15399) modify c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by client.2441958.0:2410 2014-08-13 15:28:39.832496 2014-08-13 17:38:02.816413 7fe9bae887a0 15 read_log missing 194416'15398 (194416'15397) modify 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by client.2441958.0:1866 2014-08-13 15:28:37.782074 2014-08-13 17:38:02.816450 7fe9bae887a0 15 read_log missing 194416'15396 (194416'15395) modify 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by client.2441958.0:456 2014-08-13 15:28:32.200243 2014-08-13 17:38:02.816491 7fe9bae887a0 15 read_log missing 194416'15393 (194416'15393) modify b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 17:38:02.816526 7fe9bae887a0 15 read_log missing 194396'15392 (194396'15391) modify 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by client.2235069.0:278 2014-08-12 20:53:00.005661 2014-08-13 17:38:02.816563 7fe9bae887a0 15 read_log missing 194393'15390 (194393'15389) modify 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by client.1064422.0:163 2014-08-12 20:52:33.171180 2014-08-13 17:38:02.816600 7fe9bae887a0 15 read_log missing 194393'15388 (194393'15387) modify 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by client.1064422.0:105 2014-08-12 20:52:32.928439 2014-08-13 17:38:02.816640 7fe9bae887a0 15 read_log missing 194383'15384 (194383'15383) modify 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by client.2231190.0:5832 2014-08-12 20:34:00.622639 2014-08-13 17:38:02.816677 7fe9bae887a0 15 read_log missing 194380'15382 (194380'15381) modify 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by client.2231190.0:5318 2014-08-12 20:33:27.391945 2014-08-13 17:38:02.816973 7fe9bae887a0 15 read_log missing 194377'15380 (194377'15379) modify 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by client.2231190.0:4805 2014-08-12 20:33:19.495440 2014-08-13 17:38:02.817010 7fe9bae887a0 15 read_log missing 194377'15378 (194377'15377) modify 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by client.2231190.0:3944 2014-08-12 20:33:13.590004 2014-08-13 17:38:02.817047 7fe9bae887a0 15 read_log missing 194377'15376 (194377'15375) modify 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by client.2231190.0:2437 2014-08-12 20:33:06.962690 2014-08-13 17:38:02.817084 7fe9bae887a0 15 read_log missing 194377'15374 (194377'15373) modify 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by client.2231190.0:1280 2014-08-12 20:33:02.364322 2014-08-13 17:38:02.817122 7fe9bae887a0 15 read_log missing 194377'15372 (194377'15371) modify a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by client.2231190.0:985 2014-08-12 20:33:01.237895 2014-08-13 17:38:02.817161 7fe9bae887a0 15 read_log missing 194377'15370 (194377'15369) modify cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by client.2231190.0:592 2014-08-12 20:32:59.748956 2014-08-13 17:38:02.817196 7fe9bae887a0 15 read_log missing 194370'15368 (194370'15367) modify fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by client.1508531.0:13802 2014-08-12 20:31:27.300048 2014-08-13 17:38:02.817237 7fe9bae887a0 15 read_log missing 194370'15366 (194370'15365) modify b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by client.1508531.0:13533 2014-08-12 20:31:24.438250 2014-08-13 17:38:02.817274 7fe9bae887a0 15 read_log missing 194370'15364 (194370'15363) modify e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by client.1508531.0:13235 2014-08-12 20:31:21.622526 2014-08-13 17:38:02.817311 7fe9bae887a0 15 read_log missing 194370'15362 (194370'15361) modify dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by client.1508531.0:10874 2014-08-12 20:31:04.526909 2014-08-13 17:38:02.817348 7fe9bae887a0 15 read_log missing 194370'15360 (194370'15359) modify b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by client.1508531.0:10498 2014-08-12 20:31:02.203948 2014-08-13 17:38:02.817386 7fe9bae887a0 15 read_log missing 194370'15358 (194370'15357) modify 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by client.1508531.0:8494 2014-08-12 20:30:50.192733 2014-08-13 17:38:02.817423 7fe9bae887a0 15 read_log missing 194370'15356 (194370'15355) modify 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by client.1508531.0:7494 2014-08-12 20:30:44.827180 2014-08-13 17:38:02.817460 7fe9bae887a0 15 read_log missing 194370'15354 (194370'15353) modify 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by client.1508531.0:6520 2014-08-12 20:30:40.062956 2014-08-13 17:38:02.817497 7fe9bae887a0 15 read_log missing 194370'15352 (194370'15351) modify c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by client.1508531.0:4122 2014-08-12 20:30:29.734240 2014-08-13 17:38:02.817535 7fe9bae887a0 15 read_log missing 194370'15350 (194370'15349) modify 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by client.1508531.0:4082 2014-08-12 20:30:29.592706 2014-08-13 17:38:02.817572 7fe9bae887a0 15 read_log missing 194370'15348 (194370'15347) modify 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by client.1508531.0:4081 2014-08-12 20:30:29.589242 2014-08-13 17:38:02.817609 7fe9bae887a0 15 read_log missing 194370'15346 (194370'15345) modify b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by client.1508531.0:3744 2014-08-12 20:30:28.357002 2014-08-13 17:38:02.817646 7fe9bae887a0 15 read_log missing 194370'15344 (194370'15343) modify 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by client.1508531.0:2249 2014-08-12 20:30:22.811865 2014-08-13 17:38:02.817683 7fe9bae887a0 15 read_log missing 194370'15342 (194370'15341) modify 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by client.1508531.0:1352 2014-08-12 20:30:19.448773 2014-08-13 17:38:02.817722 7fe9bae887a0 15 read_log missing 194370'15340 (194370'15339) modify e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by client.1508531.0:1338 2014-08-12 20:30:19.397975 2014-08-13 17:38:02.817759 7fe9bae887a0 15 read_log missing 194370'15338 (194370'15337) modify fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by client.1508531.0:1144 2014-08-12 20:30:18.640063 2014-08-13 17:38:02.817796 7fe9bae887a0 15 read_log missing 194370'15336 (194370'15335) modify 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by client.1508531.0:410 2014-08-12 20:30:15.927529 2014-08-13 17:38:02.817811 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.817838 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_loaded 2014-08-13 17:38:02.817856 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.012458 0 0.000000 2014-08-13 17:38:02.817868 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 17:38:02.817878 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194478 2014-08-13 17:38:02.817887 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 17:38:02.817898 7fe9bae887a0 10 osd.4 194478 pgid 24.cd coll 24.cd_head 2014-08-13 17:38:02.818008 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.cd 2014-08-13 17:38:02.818021 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.818054 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.cd(unlocked)] enter Initial 2014-08-13 17:38:02.818065 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.cd(unlocked)] enter NotTrimming 2014-08-13 17:38:02.818193 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.818317 7fe9bae887a0 20 read_log 192718'12886 (192718'12885) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:250 2014-08-06 14:19:56.760918 2014-08-13 17:38:02.818337 7fe9bae887a0 20 read_log 192718'12887 (192718'12886) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:251 2014-08-06 14:19:56.791441 2014-08-13 17:38:02.818351 7fe9bae887a0 20 read_log 192718'12888 (192718'12887) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:251 2014-08-06 14:19:56.807918 2014-08-13 17:38:02.818365 7fe9bae887a0 20 read_log 192718'12889 (192718'12888) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:252 2014-08-06 14:19:56.875582 2014-08-13 17:38:02.818382 7fe9bae887a0 20 read_log 192718'12890 (192718'12889) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:252 2014-08-06 14:19:56.962008 2014-08-13 17:38:02.818395 7fe9bae887a0 20 read_log 192718'12891 (192718'12890) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:253 2014-08-06 14:19:56.980626 2014-08-13 17:38:02.818407 7fe9bae887a0 20 read_log 192718'12892 (192718'12891) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:253 2014-08-06 14:19:56.994968 2014-08-13 17:38:02.818419 7fe9bae887a0 20 read_log 192718'12893 (192718'12892) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:254 2014-08-06 14:19:57.010129 2014-08-13 17:38:02.818431 7fe9bae887a0 20 read_log 192718'12894 (192718'12893) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:254 2014-08-06 14:19:57.027442 2014-08-13 17:38:02.818442 7fe9bae887a0 20 read_log 192718'12895 (192718'12894) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:255 2014-08-06 14:19:57.098108 2014-08-13 17:38:02.818454 7fe9bae887a0 20 read_log 192718'12896 (192718'12895) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:255 2014-08-06 14:19:57.098331 2014-08-13 17:38:02.818477 7fe9bae887a0 20 read_log 192718'12897 (192718'12896) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:256 2014-08-06 14:19:57.151004 2014-08-13 17:38:02.818493 7fe9bae887a0 20 read_log 192718'12898 (192718'12897) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:256 2014-08-06 14:19:57.203773 2014-08-13 17:38:02.818505 7fe9bae887a0 20 read_log 192718'12899 (192718'12898) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:257 2014-08-06 14:19:57.413574 2014-08-13 17:38:02.818517 7fe9bae887a0 20 read_log 192718'12900 (192718'12899) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:257 2014-08-06 14:19:57.505038 2014-08-13 17:38:02.818529 7fe9bae887a0 20 read_log 192718'12901 (192718'12900) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:258 2014-08-06 14:19:57.525061 2014-08-13 17:38:02.818541 7fe9bae887a0 20 read_log 192718'12902 (192718'12901) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:258 2014-08-06 14:19:57.551645 2014-08-13 17:38:02.818552 7fe9bae887a0 20 read_log 192718'12903 (192718'12902) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:259 2014-08-06 14:19:57.593093 2014-08-13 17:38:02.818564 7fe9bae887a0 20 read_log 192718'12904 (192718'12903) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:259 2014-08-06 14:19:57.593061 2014-08-13 17:38:02.818576 7fe9bae887a0 20 read_log 192718'12905 (192718'12904) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:260 2014-08-06 14:19:57.825570 2014-08-13 17:38:02.818588 7fe9bae887a0 20 read_log 192718'12906 (192718'12905) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:260 2014-08-06 14:19:57.886668 2014-08-13 17:38:02.818600 7fe9bae887a0 20 read_log 192718'12907 (192718'12906) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:261 2014-08-06 14:19:57.963221 2014-08-13 17:38:02.818611 7fe9bae887a0 20 read_log 192718'12908 (192718'12907) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:261 2014-08-06 14:19:57.963338 2014-08-13 17:38:02.818624 7fe9bae887a0 20 read_log 192718'12909 (192718'12908) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:262 2014-08-06 14:19:58.061780 2014-08-13 17:38:02.818638 7fe9bae887a0 20 read_log 192718'12910 (192718'12909) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:262 2014-08-06 14:19:58.127951 2014-08-13 17:38:02.818651 7fe9bae887a0 20 read_log 192718'12911 (192718'12910) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:263 2014-08-06 14:19:58.161768 2014-08-13 17:38:02.818663 7fe9bae887a0 20 read_log 192718'12912 (192718'12911) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:263 2014-08-06 14:19:58.193512 2014-08-13 17:38:02.818675 7fe9bae887a0 20 read_log 192718'12913 (192718'12912) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:264 2014-08-06 14:19:58.243489 2014-08-13 17:38:02.818687 7fe9bae887a0 20 read_log 192718'12914 (192718'12913) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:264 2014-08-06 14:19:58.337256 2014-08-13 17:38:02.818699 7fe9bae887a0 20 read_log 192718'12915 (192718'12914) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:265 2014-08-06 14:19:58.600951 2014-08-13 17:38:02.818728 7fe9bae887a0 20 read_log 192718'12916 (192718'12915) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:265 2014-08-06 14:19:58.641740 2014-08-13 17:38:02.818740 7fe9bae887a0 20 read_log 192718'12917 (192718'12916) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:266 2014-08-06 14:19:58.751607 2014-08-13 17:38:02.818752 7fe9bae887a0 20 read_log 192718'12918 (192718'12917) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:266 2014-08-06 14:19:59.207061 2014-08-13 17:38:02.818764 7fe9bae887a0 20 read_log 192718'12919 (192718'12918) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:267 2014-08-06 14:19:59.360766 2014-08-13 17:38:02.818776 7fe9bae887a0 20 read_log 192718'12920 (192718'12919) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:267 2014-08-06 14:19:59.695766 2014-08-13 17:38:02.818796 7fe9bae887a0 20 read_log 192718'12921 (192718'12920) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:268 2014-08-06 14:19:59.786818 2014-08-13 17:38:02.818811 7fe9bae887a0 20 read_log 192718'12922 (192718'12921) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:268 2014-08-06 14:19:59.992924 2014-08-13 17:38:02.818824 7fe9bae887a0 20 read_log 192718'12923 (192718'12922) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:269 2014-08-06 14:20:00.148926 2014-08-13 17:38:02.818835 7fe9bae887a0 20 read_log 192718'12924 (192718'12923) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:269 2014-08-06 14:20:00.259181 2014-08-13 17:38:02.818850 7fe9bae887a0 20 read_log 192718'12925 (192718'12924) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:270 2014-08-06 14:20:00.337829 2014-08-13 17:38:02.818862 7fe9bae887a0 20 read_log 192718'12926 (192718'12925) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:270 2014-08-06 14:20:00.672571 2014-08-13 17:38:02.818873 7fe9bae887a0 20 read_log 192718'12927 (192718'12926) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:271 2014-08-06 14:20:00.751163 2014-08-13 17:38:02.818886 7fe9bae887a0 20 read_log 192718'12928 (192718'12927) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:271 2014-08-06 14:20:00.863751 2014-08-13 17:38:02.818899 7fe9bae887a0 20 read_log 192718'12929 (192718'12928) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:272 2014-08-06 14:20:00.923558 2014-08-13 17:38:02.818912 7fe9bae887a0 20 read_log 192718'12930 (192718'12929) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:272 2014-08-06 14:20:00.943084 2014-08-13 17:38:02.818923 7fe9bae887a0 20 read_log 192718'12931 (192718'12930) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:273 2014-08-06 14:20:00.980367 2014-08-13 17:38:02.818937 7fe9bae887a0 20 read_log 192718'12932 (192718'12931) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:273 2014-08-06 14:20:01.010928 2014-08-13 17:38:02.818950 7fe9bae887a0 20 read_log 192718'12933 (192718'12932) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:274 2014-08-06 14:20:01.038947 2014-08-13 17:38:02.818962 7fe9bae887a0 20 read_log 192718'12934 (192718'12933) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:274 2014-08-06 14:20:01.073429 2014-08-13 17:38:02.818976 7fe9bae887a0 20 read_log 192718'12935 (192718'12934) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:275 2014-08-06 14:20:01.135949 2014-08-13 17:38:02.818988 7fe9bae887a0 20 read_log 192718'12936 (192718'12935) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:275 2014-08-06 14:20:01.363661 2014-08-13 17:38:02.819000 7fe9bae887a0 20 read_log 192718'12937 (192718'12936) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:276 2014-08-06 14:20:01.425602 2014-08-13 17:38:02.819011 7fe9bae887a0 20 read_log 192718'12938 (192718'12937) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:276 2014-08-06 14:20:01.503083 2014-08-13 17:38:02.819023 7fe9bae887a0 20 read_log 192718'12939 (192718'12938) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:277 2014-08-06 14:20:01.618705 2014-08-13 17:38:02.819035 7fe9bae887a0 20 read_log 192718'12940 (192718'12939) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:277 2014-08-06 14:20:01.657661 2014-08-13 17:38:02.819049 7fe9bae887a0 20 read_log 192718'12941 (192718'12940) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:278 2014-08-06 14:20:01.765604 2014-08-13 17:38:02.819061 7fe9bae887a0 20 read_log 192718'12942 (192718'12941) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:278 2014-08-06 14:20:01.890004 2014-08-13 17:38:02.819074 7fe9bae887a0 20 read_log 192718'12943 (192718'12942) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:279 2014-08-06 14:20:02.003530 2014-08-13 17:38:02.819086 7fe9bae887a0 20 read_log 192718'12944 (192718'12943) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:279 2014-08-06 14:20:02.031348 2014-08-13 17:38:02.819104 7fe9bae887a0 20 read_log 192718'12945 (192718'12944) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:280 2014-08-06 14:20:02.083653 2014-08-13 17:38:02.819119 7fe9bae887a0 20 read_log 192718'12946 (192718'12945) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:280 2014-08-06 14:20:02.123033 2014-08-13 17:38:02.819131 7fe9bae887a0 20 read_log 192718'12947 (192718'12946) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:281 2014-08-06 14:20:02.182567 2014-08-13 17:38:02.819143 7fe9bae887a0 20 read_log 192718'12948 (192718'12947) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:281 2014-08-06 14:20:02.404137 2014-08-13 17:38:02.819164 7fe9bae887a0 20 read_log 192718'12949 (192718'12948) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:282 2014-08-06 14:20:02.448903 2014-08-13 17:38:02.819179 7fe9bae887a0 20 read_log 192718'12950 (192718'12949) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:282 2014-08-06 14:20:02.529782 2014-08-13 17:38:02.819191 7fe9bae887a0 20 read_log 192718'12951 (192718'12950) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:283 2014-08-06 14:20:02.613534 2014-08-13 17:38:02.819203 7fe9bae887a0 20 read_log 192718'12952 (192718'12951) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:283 2014-08-06 14:20:02.722147 2014-08-13 17:38:02.819215 7fe9bae887a0 20 read_log 192718'12953 (192718'12952) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:284 2014-08-06 14:20:02.794140 2014-08-13 17:38:02.819227 7fe9bae887a0 20 read_log 192718'12954 (192718'12953) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:284 2014-08-06 14:20:02.866153 2014-08-13 17:38:02.819239 7fe9bae887a0 20 read_log 192718'12955 (192718'12954) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:285 2014-08-06 14:20:02.930733 2014-08-13 17:38:02.819273 7fe9bae887a0 20 read_log 192718'12956 (192718'12955) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:285 2014-08-06 14:20:03.001145 2014-08-13 17:38:02.819286 7fe9bae887a0 20 read_log 192718'12957 (192718'12956) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:286 2014-08-06 14:20:03.067173 2014-08-13 17:38:02.819298 7fe9bae887a0 20 read_log 192718'12958 (192718'12957) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:286 2014-08-06 14:20:03.296994 2014-08-13 17:38:02.819309 7fe9bae887a0 20 read_log 192718'12959 (192718'12958) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:287 2014-08-06 14:20:03.402762 2014-08-13 17:38:02.819321 7fe9bae887a0 20 read_log 192718'12960 (192718'12959) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:287 2014-08-06 14:20:03.645512 2014-08-13 17:38:02.819333 7fe9bae887a0 20 read_log 192718'12961 (192718'12960) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:288 2014-08-06 14:20:04.476787 2014-08-13 17:38:02.819345 7fe9bae887a0 20 read_log 192718'12962 (192718'12961) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:288 2014-08-06 14:20:04.699644 2014-08-13 17:38:02.819358 7fe9bae887a0 20 read_log 192718'12963 (192718'12962) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:289 2014-08-06 14:20:04.741850 2014-08-13 17:38:02.819370 7fe9bae887a0 20 read_log 192718'12964 (192718'12963) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:289 2014-08-06 14:20:04.793174 2014-08-13 17:38:02.819381 7fe9bae887a0 20 read_log 192718'12965 (192718'12964) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:290 2014-08-06 14:20:04.813337 2014-08-13 17:38:02.819393 7fe9bae887a0 20 read_log 192718'12966 (192718'12965) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:290 2014-08-06 14:20:04.837382 2014-08-13 17:38:02.819404 7fe9bae887a0 20 read_log 192718'12967 (192718'12966) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:291 2014-08-06 14:20:04.895681 2014-08-13 17:38:02.819416 7fe9bae887a0 20 read_log 192718'12968 (192718'12967) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:291 2014-08-06 14:20:04.903477 2014-08-13 17:38:02.819456 7fe9bae887a0 20 read_log 192718'12969 (192718'12968) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:292 2014-08-06 14:20:04.942194 2014-08-13 17:38:02.819469 7fe9bae887a0 20 read_log 192718'12970 (192718'12969) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:292 2014-08-06 14:20:05.007546 2014-08-13 17:38:02.819481 7fe9bae887a0 20 read_log 192718'12971 (192718'12970) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:293 2014-08-06 14:20:05.075085 2014-08-13 17:38:02.819493 7fe9bae887a0 20 read_log 192718'12972 (192718'12971) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:293 2014-08-06 14:20:05.175712 2014-08-13 17:38:02.819505 7fe9bae887a0 20 read_log 192718'12973 (192718'12972) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:294 2014-08-06 14:20:05.190112 2014-08-13 17:38:02.819517 7fe9bae887a0 20 read_log 192718'12974 (192718'12973) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:294 2014-08-06 14:20:05.216057 2014-08-13 17:38:02.819529 7fe9bae887a0 20 read_log 192718'12975 (192718'12974) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:295 2014-08-06 14:20:05.256400 2014-08-13 17:38:02.819541 7fe9bae887a0 20 read_log 192718'12976 (192718'12975) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:295 2014-08-06 14:20:05.256547 2014-08-13 17:38:02.819553 7fe9bae887a0 20 read_log 192718'12977 (192718'12976) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:296 2014-08-06 14:20:05.290774 2014-08-13 17:38:02.819564 7fe9bae887a0 20 read_log 192718'12978 (192718'12977) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:296 2014-08-06 14:20:05.304407 2014-08-13 17:38:02.819576 7fe9bae887a0 20 read_log 192718'12979 (192718'12978) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:297 2014-08-06 14:20:05.351273 2014-08-13 17:38:02.819588 7fe9bae887a0 20 read_log 192718'12980 (192718'12979) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:297 2014-08-06 14:20:05.354027 2014-08-13 17:38:02.819599 7fe9bae887a0 20 read_log 192718'12981 (192718'12980) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:298 2014-08-06 14:20:05.468130 2014-08-13 17:38:02.819611 7fe9bae887a0 20 read_log 192718'12982 (192718'12981) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:298 2014-08-06 14:20:05.487720 2014-08-13 17:38:02.819623 7fe9bae887a0 20 read_log 192718'12983 (192718'12982) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:299 2014-08-06 14:20:05.501101 2014-08-13 17:38:02.819634 7fe9bae887a0 20 read_log 192718'12984 (192718'12983) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:299 2014-08-06 14:20:05.530302 2014-08-13 17:38:02.819646 7fe9bae887a0 20 read_log 192718'12985 (192718'12984) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:300 2014-08-06 14:20:05.590970 2014-08-13 17:38:02.819658 7fe9bae887a0 20 read_log 192718'12986 (192718'12985) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:300 2014-08-06 14:20:05.590677 2014-08-13 17:38:02.819669 7fe9bae887a0 20 read_log 192718'12987 (192718'12986) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:301 2014-08-06 14:20:05.633807 2014-08-13 17:38:02.819681 7fe9bae887a0 20 read_log 192718'12988 (192718'12987) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:301 2014-08-06 14:20:05.633807 2014-08-13 17:38:02.819693 7fe9bae887a0 20 read_log 192718'12989 (192718'12988) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:302 2014-08-06 14:20:05.660625 2014-08-13 17:38:02.819704 7fe9bae887a0 20 read_log 192718'12990 (192718'12989) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:302 2014-08-06 14:20:05.669423 2014-08-13 17:38:02.819716 7fe9bae887a0 20 read_log 192718'12991 (192718'12990) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:303 2014-08-06 14:20:05.936009 2014-08-13 17:38:02.819728 7fe9bae887a0 20 read_log 192718'12992 (192718'12991) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:303 2014-08-06 14:20:05.941043 2014-08-13 17:38:02.819746 7fe9bae887a0 20 read_log 192718'12993 (192718'12992) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:304 2014-08-06 14:20:05.980207 2014-08-13 17:38:02.819758 7fe9bae887a0 20 read_log 192718'12994 (192718'12993) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:304 2014-08-06 14:20:06.006070 2014-08-13 17:38:02.819770 7fe9bae887a0 20 read_log 192718'12995 (192718'12994) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:305 2014-08-06 14:20:06.050766 2014-08-13 17:38:02.819782 7fe9bae887a0 20 read_log 192718'12996 (192718'12995) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:305 2014-08-06 14:20:06.076547 2014-08-13 17:38:02.819794 7fe9bae887a0 20 read_log 192718'12997 (192718'12996) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:306 2014-08-06 14:20:06.095923 2014-08-13 17:38:02.819806 7fe9bae887a0 20 read_log 192718'12998 (192718'12997) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:306 2014-08-06 14:20:06.120395 2014-08-13 17:38:02.819817 7fe9bae887a0 20 read_log 192718'12999 (192718'12998) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:307 2014-08-06 14:20:06.174717 2014-08-13 17:38:02.819829 7fe9bae887a0 20 read_log 192718'13000 (192718'12999) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:307 2014-08-06 14:20:06.174949 2014-08-13 17:38:02.819841 7fe9bae887a0 20 read_log 192718'13001 (192718'13000) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:308 2014-08-06 14:20:06.200841 2014-08-13 17:38:02.819853 7fe9bae887a0 20 read_log 192718'13002 (192718'13001) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:308 2014-08-06 14:20:06.217992 2014-08-13 17:38:02.819865 7fe9bae887a0 20 read_log 192718'13003 (192718'13002) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:309 2014-08-06 14:20:06.276018 2014-08-13 17:38:02.819876 7fe9bae887a0 20 read_log 192718'13004 (192718'13003) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:309 2014-08-06 14:20:06.276051 2014-08-13 17:38:02.819888 7fe9bae887a0 20 read_log 192718'13005 (192718'13004) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:310 2014-08-06 14:20:06.309307 2014-08-13 17:38:02.819900 7fe9bae887a0 20 read_log 192718'13006 (192718'13005) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:310 2014-08-06 14:20:06.399140 2014-08-13 17:38:02.819912 7fe9bae887a0 20 read_log 192718'13007 (192718'13006) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:311 2014-08-06 14:20:06.489160 2014-08-13 17:38:02.819923 7fe9bae887a0 20 read_log 192718'13008 (192718'13007) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:311 2014-08-06 14:20:06.554912 2014-08-13 17:38:02.819951 7fe9bae887a0 20 read_log 192718'13009 (192718'13008) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:312 2014-08-06 14:20:06.574163 2014-08-13 17:38:02.819963 7fe9bae887a0 20 read_log 192718'13010 (192718'13009) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:312 2014-08-06 14:20:06.600473 2014-08-13 17:38:02.819975 7fe9bae887a0 20 read_log 192718'13011 (192718'13010) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:313 2014-08-06 14:20:06.609504 2014-08-13 17:38:02.819987 7fe9bae887a0 20 read_log 192718'13012 (192718'13011) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:313 2014-08-06 14:20:06.627689 2014-08-13 17:38:02.820001 7fe9bae887a0 20 read_log 192718'13013 (192718'13012) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:314 2014-08-06 14:20:06.636894 2014-08-13 17:38:02.820014 7fe9bae887a0 20 read_log 192718'13014 (192718'13013) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:314 2014-08-06 14:20:06.658284 2014-08-13 17:38:02.820026 7fe9bae887a0 20 read_log 192718'13015 (192718'13014) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:315 2014-08-06 14:20:06.668907 2014-08-13 17:38:02.820038 7fe9bae887a0 20 read_log 192718'13016 (192718'13015) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:315 2014-08-06 14:20:06.693763 2014-08-13 17:38:02.820055 7fe9bae887a0 20 read_log 192718'13017 (192718'13016) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:316 2014-08-06 14:20:06.703144 2014-08-13 17:38:02.820068 7fe9bae887a0 20 read_log 192718'13018 (192718'13017) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:316 2014-08-06 14:20:06.780033 2014-08-13 17:38:02.820080 7fe9bae887a0 20 read_log 192718'13019 (192718'13018) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:317 2014-08-06 14:20:06.793939 2014-08-13 17:38:02.820092 7fe9bae887a0 20 read_log 192718'13020 (192718'13019) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:317 2014-08-06 14:20:06.815555 2014-08-13 17:38:02.820103 7fe9bae887a0 20 read_log 192718'13021 (192718'13020) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:318 2014-08-06 14:20:06.828865 2014-08-13 17:38:02.820115 7fe9bae887a0 20 read_log 192718'13022 (192718'13021) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:318 2014-08-06 14:20:06.846423 2014-08-13 17:38:02.820127 7fe9bae887a0 20 read_log 192718'13023 (192718'13022) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:319 2014-08-06 14:20:06.901604 2014-08-13 17:38:02.820139 7fe9bae887a0 20 read_log 192718'13024 (192718'13023) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:319 2014-08-06 14:20:06.901684 2014-08-13 17:38:02.820150 7fe9bae887a0 20 read_log 192718'13025 (192718'13024) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:320 2014-08-06 14:20:07.057175 2014-08-13 17:38:02.820169 7fe9bae887a0 20 read_log 192718'13026 (192718'13025) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:320 2014-08-06 14:20:07.089741 2014-08-13 17:38:02.820181 7fe9bae887a0 20 read_log 192718'13027 (192718'13026) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:321 2014-08-06 14:20:07.144198 2014-08-13 17:38:02.820193 7fe9bae887a0 20 read_log 192718'13028 (192718'13027) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:321 2014-08-06 14:20:07.273928 2014-08-13 17:38:02.820205 7fe9bae887a0 20 read_log 192718'13029 (192718'13028) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:322 2014-08-06 14:20:07.291482 2014-08-13 17:38:02.820217 7fe9bae887a0 20 read_log 192718'13030 (192718'13029) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:322 2014-08-06 14:20:07.317834 2014-08-13 17:38:02.820229 7fe9bae887a0 20 read_log 192718'13031 (192718'13030) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:323 2014-08-06 14:20:07.388201 2014-08-13 17:38:02.820240 7fe9bae887a0 20 read_log 192718'13032 (192718'13031) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:323 2014-08-06 14:20:07.388663 2014-08-13 17:38:02.820252 7fe9bae887a0 20 read_log 192718'13033 (192718'13032) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:324 2014-08-06 14:20:07.419091 2014-08-13 17:38:02.820264 7fe9bae887a0 20 read_log 192718'13034 (192718'13033) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:324 2014-08-06 14:20:07.426311 2014-08-13 17:38:02.820276 7fe9bae887a0 20 read_log 192718'13035 (192718'13034) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:325 2014-08-06 14:20:07.465989 2014-08-13 17:38:02.820288 7fe9bae887a0 20 read_log 192718'13036 (192718'13035) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:325 2014-08-06 14:20:07.477567 2014-08-13 17:38:02.820300 7fe9bae887a0 20 read_log 192718'13037 (192718'13036) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:326 2014-08-06 14:20:07.579013 2014-08-13 17:38:02.820311 7fe9bae887a0 20 read_log 192718'13038 (192718'13037) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:326 2014-08-06 14:20:07.589471 2014-08-13 17:38:02.820323 7fe9bae887a0 20 read_log 192718'13039 (192718'13038) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:327 2014-08-06 14:20:07.611334 2014-08-13 17:38:02.820335 7fe9bae887a0 20 read_log 192718'13040 (192718'13039) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:327 2014-08-06 14:20:07.619952 2014-08-13 17:38:02.820355 7fe9bae887a0 20 read_log 192718'13041 (192718'13040) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:328 2014-08-06 14:20:07.739858 2014-08-13 17:38:02.820367 7fe9bae887a0 20 read_log 192718'13042 (192718'13041) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:328 2014-08-06 14:20:07.751268 2014-08-13 17:38:02.820379 7fe9bae887a0 20 read_log 192718'13043 (192718'13042) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:329 2014-08-06 14:20:07.806577 2014-08-13 17:38:02.820392 7fe9bae887a0 20 read_log 192718'13044 (192718'13043) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:329 2014-08-06 14:20:07.806586 2014-08-13 17:38:02.820404 7fe9bae887a0 20 read_log 192718'13045 (192718'13044) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:330 2014-08-06 14:20:07.838454 2014-08-13 17:38:02.820415 7fe9bae887a0 20 read_log 192718'13046 (192718'13045) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:330 2014-08-06 14:20:07.845244 2014-08-13 17:38:02.820427 7fe9bae887a0 20 read_log 192718'13047 (192718'13046) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:331 2014-08-06 14:20:07.901697 2014-08-13 17:38:02.820440 7fe9bae887a0 20 read_log 192718'13048 (192718'13047) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:331 2014-08-06 14:20:07.923359 2014-08-13 17:38:02.820452 7fe9bae887a0 20 read_log 192718'13049 (192718'13048) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:332 2014-08-06 14:20:07.937178 2014-08-13 17:38:02.820464 7fe9bae887a0 20 read_log 192718'13050 (192718'13049) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:332 2014-08-06 14:20:07.959890 2014-08-13 17:38:02.820476 7fe9bae887a0 20 read_log 192718'13051 (192718'13050) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:333 2014-08-06 14:20:07.971420 2014-08-13 17:38:02.820510 7fe9bae887a0 20 read_log 192718'13052 (192718'13051) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:333 2014-08-06 14:20:07.993926 2014-08-13 17:38:02.820523 7fe9bae887a0 20 read_log 192718'13053 (192718'13052) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:334 2014-08-06 14:20:08.007654 2014-08-13 17:38:02.820535 7fe9bae887a0 20 read_log 192718'13054 (192718'13053) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:334 2014-08-06 14:20:08.028620 2014-08-13 17:38:02.820547 7fe9bae887a0 20 read_log 192718'13055 (192718'13054) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:335 2014-08-06 14:20:08.075338 2014-08-13 17:38:02.820559 7fe9bae887a0 20 read_log 192718'13056 (192718'13055) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:335 2014-08-06 14:20:08.075446 2014-08-13 17:38:02.820571 7fe9bae887a0 20 read_log 192718'13057 (192718'13056) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:336 2014-08-06 14:20:08.165676 2014-08-13 17:38:02.820583 7fe9bae887a0 20 read_log 192718'13058 (192718'13057) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:336 2014-08-06 14:20:08.189365 2014-08-13 17:38:02.820595 7fe9bae887a0 20 read_log 192718'13059 (192718'13058) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:337 2014-08-06 14:20:08.264223 2014-08-13 17:38:02.820606 7fe9bae887a0 20 read_log 192718'13060 (192718'13059) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:337 2014-08-06 14:20:08.339552 2014-08-13 17:38:02.820618 7fe9bae887a0 20 read_log 192718'13061 (192718'13060) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:338 2014-08-06 14:20:08.362492 2014-08-13 17:38:02.820630 7fe9bae887a0 20 read_log 192718'13062 (192718'13061) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:338 2014-08-06 14:20:08.380778 2014-08-13 17:38:02.820642 7fe9bae887a0 20 read_log 192718'13063 (192718'13062) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:339 2014-08-06 14:20:08.417732 2014-08-13 17:38:02.820653 7fe9bae887a0 20 read_log 192718'13064 (192718'13063) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:339 2014-08-06 14:20:08.434376 2014-08-13 17:38:02.820671 7fe9bae887a0 20 read_log 192718'13065 (192718'13064) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:340 2014-08-06 14:20:08.461221 2014-08-13 17:38:02.820684 7fe9bae887a0 20 read_log 192718'13066 (192718'13065) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:340 2014-08-06 14:20:08.477700 2014-08-13 17:38:02.820696 7fe9bae887a0 20 read_log 192718'13067 (192718'13066) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:341 2014-08-06 14:20:08.564697 2014-08-13 17:38:02.820707 7fe9bae887a0 20 read_log 192718'13068 (192718'13067) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:341 2014-08-06 14:20:08.590690 2014-08-13 17:38:02.820719 7fe9bae887a0 20 read_log 192718'13069 (192718'13068) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:342 2014-08-06 14:20:08.602875 2014-08-13 17:38:02.820731 7fe9bae887a0 20 read_log 192718'13070 (192718'13069) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:342 2014-08-06 14:20:08.624540 2014-08-13 17:38:02.820743 7fe9bae887a0 20 read_log 192718'13071 (192718'13070) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:343 2014-08-06 14:20:08.635698 2014-08-13 17:38:02.820755 7fe9bae887a0 20 read_log 192718'13072 (192718'13071) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:343 2014-08-06 14:20:08.659770 2014-08-13 17:38:02.820768 7fe9bae887a0 20 read_log 192718'13073 (192718'13072) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:344 2014-08-06 14:20:08.676278 2014-08-13 17:38:02.820780 7fe9bae887a0 20 read_log 192718'13074 (192718'13073) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:344 2014-08-06 14:20:08.699706 2014-08-13 17:38:02.820792 7fe9bae887a0 20 read_log 192718'13075 (192718'13074) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:345 2014-08-06 14:20:08.711488 2014-08-13 17:38:02.820804 7fe9bae887a0 20 read_log 192718'13076 (192718'13075) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:345 2014-08-06 14:20:08.730302 2014-08-13 17:38:02.820815 7fe9bae887a0 20 read_log 192718'13077 (192718'13076) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:346 2014-08-06 14:20:08.749711 2014-08-13 17:38:02.820828 7fe9bae887a0 20 read_log 192718'13078 (192718'13077) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:346 2014-08-06 14:20:08.811847 2014-08-13 17:38:02.820840 7fe9bae887a0 20 read_log 192718'13079 (192718'13078) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:347 2014-08-06 14:20:08.825709 2014-08-13 17:38:02.820851 7fe9bae887a0 20 read_log 192718'13080 (192718'13079) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:347 2014-08-06 14:20:08.953242 2014-08-13 17:38:02.820863 7fe9bae887a0 20 read_log 192718'13081 (192718'13080) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:348 2014-08-06 14:20:08.964048 2014-08-13 17:38:02.820875 7fe9bae887a0 20 read_log 192718'13082 (192718'13081) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:348 2014-08-06 14:20:08.984023 2014-08-13 17:38:02.820887 7fe9bae887a0 20 read_log 192718'13083 (192718'13082) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:349 2014-08-06 14:20:09.039562 2014-08-13 17:38:02.820898 7fe9bae887a0 20 read_log 192718'13084 (192718'13083) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:349 2014-08-06 14:20:09.039713 2014-08-13 17:38:02.820910 7fe9bae887a0 20 read_log 192718'13085 (192718'13084) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:350 2014-08-06 14:20:09.079962 2014-08-13 17:38:02.820922 7fe9bae887a0 20 read_log 192718'13086 (192718'13085) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:350 2014-08-06 14:20:09.080224 2014-08-13 17:38:02.820950 7fe9bae887a0 20 read_log 192718'13087 (192718'13086) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:351 2014-08-06 14:20:09.140757 2014-08-13 17:38:02.820967 7fe9bae887a0 20 read_log 192718'13088 (192718'13087) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:351 2014-08-06 14:20:09.140779 2014-08-13 17:38:02.820985 7fe9bae887a0 20 read_log 192718'13089 (192718'13088) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:352 2014-08-06 14:20:09.204070 2014-08-13 17:38:02.821000 7fe9bae887a0 20 read_log 192718'13090 (192718'13089) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:352 2014-08-06 14:20:09.204127 2014-08-13 17:38:02.821012 7fe9bae887a0 20 read_log 192718'13091 (192718'13090) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:353 2014-08-06 14:20:09.269091 2014-08-13 17:38:02.821023 7fe9bae887a0 20 read_log 192718'13092 (192718'13091) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:353 2014-08-06 14:20:09.269219 2014-08-13 17:38:02.821035 7fe9bae887a0 20 read_log 192718'13093 (192718'13092) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:354 2014-08-06 14:20:09.299962 2014-08-13 17:38:02.821047 7fe9bae887a0 20 read_log 192718'13094 (192718'13093) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:354 2014-08-06 14:20:09.301975 2014-08-13 17:38:02.821059 7fe9bae887a0 20 read_log 192718'13095 (192718'13094) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:355 2014-08-06 14:20:09.395279 2014-08-13 17:38:02.821071 7fe9bae887a0 20 read_log 192718'13096 (192718'13095) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:355 2014-08-06 14:20:09.395285 2014-08-13 17:38:02.821082 7fe9bae887a0 20 read_log 192718'13097 (192718'13096) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:356 2014-08-06 14:20:09.425471 2014-08-13 17:38:02.821094 7fe9bae887a0 20 read_log 192718'13098 (192718'13097) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:356 2014-08-06 14:20:09.428378 2014-08-13 17:38:02.821106 7fe9bae887a0 20 read_log 192718'13099 (192718'13098) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:357 2014-08-06 14:20:09.490485 2014-08-13 17:38:02.821117 7fe9bae887a0 20 read_log 192718'13100 (192718'13099) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:357 2014-08-06 14:20:09.490592 2014-08-13 17:38:02.821129 7fe9bae887a0 20 read_log 192718'13101 (192718'13100) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:358 2014-08-06 14:20:09.536306 2014-08-13 17:38:02.821142 7fe9bae887a0 20 read_log 192718'13102 (192718'13101) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:358 2014-08-06 14:20:09.537096 2014-08-13 17:38:02.821159 7fe9bae887a0 20 read_log 192718'13103 (192718'13102) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:359 2014-08-06 14:20:09.571163 2014-08-13 17:38:02.821172 7fe9bae887a0 20 read_log 192718'13104 (192718'13103) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:359 2014-08-06 14:20:09.572033 2014-08-13 17:38:02.821184 7fe9bae887a0 20 read_log 192718'13105 (192718'13104) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:360 2014-08-06 14:20:09.611825 2014-08-13 17:38:02.821196 7fe9bae887a0 20 read_log 192718'13106 (192718'13105) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:360 2014-08-06 14:20:09.611729 2014-08-13 17:38:02.821208 7fe9bae887a0 20 read_log 192718'13107 (192718'13106) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:361 2014-08-06 14:20:10.000184 2014-08-13 17:38:02.821220 7fe9bae887a0 20 read_log 192718'13108 (192718'13107) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:361 2014-08-06 14:20:10.000178 2014-08-13 17:38:02.821252 7fe9bae887a0 20 read_log 192718'13109 (192718'13108) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:362 2014-08-06 14:20:10.094805 2014-08-13 17:38:02.821267 7fe9bae887a0 20 read_log 192718'13110 (192718'13109) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:362 2014-08-06 14:20:10.094808 2014-08-13 17:38:02.821279 7fe9bae887a0 20 read_log 192718'13111 (192718'13110) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:363 2014-08-06 14:20:10.351913 2014-08-13 17:38:02.821291 7fe9bae887a0 20 read_log 192718'13112 (192718'13111) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:363 2014-08-06 14:20:10.351991 2014-08-13 17:38:02.821317 7fe9bae887a0 20 read_log 192718'13113 (192718'13112) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:364 2014-08-06 14:20:10.431603 2014-08-13 17:38:02.821332 7fe9bae887a0 20 read_log 192718'13114 (192718'13113) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:364 2014-08-06 14:20:10.431432 2014-08-13 17:38:02.821344 7fe9bae887a0 20 read_log 192718'13115 (192718'13114) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:365 2014-08-06 14:20:10.582076 2014-08-13 17:38:02.821356 7fe9bae887a0 20 read_log 192718'13116 (192718'13115) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:365 2014-08-06 14:20:10.582588 2014-08-13 17:38:02.821368 7fe9bae887a0 20 read_log 192718'13117 (192718'13116) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:366 2014-08-06 14:20:10.637194 2014-08-13 17:38:02.821379 7fe9bae887a0 20 read_log 192718'13118 (192718'13117) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:366 2014-08-06 14:20:10.637194 2014-08-13 17:38:02.821391 7fe9bae887a0 20 read_log 192718'13119 (192718'13118) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:367 2014-08-06 14:20:10.713664 2014-08-13 17:38:02.821403 7fe9bae887a0 20 read_log 192718'13120 (192718'13119) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:367 2014-08-06 14:20:10.713433 2014-08-13 17:38:02.821415 7fe9bae887a0 20 read_log 192718'13121 (192718'13120) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:368 2014-08-06 14:20:10.743622 2014-08-13 17:38:02.821427 7fe9bae887a0 20 read_log 192718'13122 (192718'13121) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:368 2014-08-06 14:20:10.745499 2014-08-13 17:38:02.821439 7fe9bae887a0 20 read_log 192718'13123 (192718'13122) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:369 2014-08-06 14:20:10.914536 2014-08-13 17:38:02.821451 7fe9bae887a0 20 read_log 192718'13124 (192718'13123) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:369 2014-08-06 14:20:10.914468 2014-08-13 17:38:02.821463 7fe9bae887a0 20 read_log 192718'13125 (192718'13124) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:370 2014-08-06 14:20:10.947506 2014-08-13 17:38:02.821474 7fe9bae887a0 20 read_log 192718'13126 (192718'13125) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:370 2014-08-06 14:20:10.948018 2014-08-13 17:38:02.821486 7fe9bae887a0 20 read_log 192718'13127 (192718'13126) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:371 2014-08-06 14:20:10.980271 2014-08-13 17:38:02.821498 7fe9bae887a0 20 read_log 192718'13128 (192718'13127) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:371 2014-08-06 14:20:10.982385 2014-08-13 17:38:02.821509 7fe9bae887a0 20 read_log 192718'13129 (192718'13128) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:372 2014-08-06 14:20:11.011108 2014-08-13 17:38:02.821521 7fe9bae887a0 20 read_log 192718'13130 (192718'13129) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:372 2014-08-06 14:20:11.013235 2014-08-13 17:38:02.821533 7fe9bae887a0 20 read_log 192718'13131 (192718'13130) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:373 2014-08-06 14:20:11.069912 2014-08-13 17:38:02.821545 7fe9bae887a0 20 read_log 192718'13132 (192718'13131) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:373 2014-08-06 14:20:11.069767 2014-08-13 17:38:02.821557 7fe9bae887a0 20 read_log 192718'13133 (192718'13132) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:374 2014-08-06 14:20:11.100851 2014-08-13 17:38:02.821568 7fe9bae887a0 20 read_log 192718'13134 (192718'13133) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:374 2014-08-06 14:20:11.103083 2014-08-13 17:38:02.821580 7fe9bae887a0 20 read_log 192718'13135 (192718'13134) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:375 2014-08-06 14:20:11.128235 2014-08-13 17:38:02.821592 7fe9bae887a0 20 read_log 192718'13136 (192718'13135) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:375 2014-08-06 14:20:11.129689 2014-08-13 17:38:02.821627 7fe9bae887a0 20 read_log 192718'13137 (192718'13136) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:376 2014-08-06 14:20:11.154357 2014-08-13 17:38:02.821642 7fe9bae887a0 20 read_log 192718'13138 (192718'13137) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:376 2014-08-06 14:20:11.159111 2014-08-13 17:38:02.821655 7fe9bae887a0 20 read_log 192718'13139 (192718'13138) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:377 2014-08-06 14:20:11.190607 2014-08-13 17:38:02.821667 7fe9bae887a0 20 read_log 192718'13140 (192718'13139) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:377 2014-08-06 14:20:11.190954 2014-08-13 17:38:02.821678 7fe9bae887a0 20 read_log 192718'13141 (192718'13140) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:378 2014-08-06 14:20:11.220630 2014-08-13 17:38:02.821690 7fe9bae887a0 20 read_log 192718'13142 (192718'13141) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:378 2014-08-06 14:20:11.222915 2014-08-13 17:38:02.821702 7fe9bae887a0 20 read_log 192718'13143 (192718'13142) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:379 2014-08-06 14:20:11.251209 2014-08-13 17:38:02.821714 7fe9bae887a0 20 read_log 192718'13144 (192718'13143) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:379 2014-08-06 14:20:11.253554 2014-08-13 17:38:02.821728 7fe9bae887a0 20 read_log 192718'13145 (192718'13144) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:380 2014-08-06 14:20:11.281940 2014-08-13 17:38:02.821740 7fe9bae887a0 20 read_log 192718'13146 (192718'13145) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:380 2014-08-06 14:20:11.284026 2014-08-13 17:38:02.821754 7fe9bae887a0 20 read_log 192718'13147 (192718'13146) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:381 2014-08-06 14:20:11.315082 2014-08-13 17:38:02.821766 7fe9bae887a0 20 read_log 192718'13148 (192718'13147) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:381 2014-08-06 14:20:11.317394 2014-08-13 17:38:02.821780 7fe9bae887a0 20 read_log 192718'13149 (192718'13148) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:382 2014-08-06 14:20:11.346358 2014-08-13 17:38:02.821792 7fe9bae887a0 20 read_log 192718'13150 (192718'13149) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:382 2014-08-06 14:20:11.348366 2014-08-13 17:38:02.821803 7fe9bae887a0 20 read_log 192718'13151 (192718'13150) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:383 2014-08-06 14:20:11.402720 2014-08-13 17:38:02.821815 7fe9bae887a0 20 read_log 192718'13152 (192718'13151) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:383 2014-08-06 14:20:11.401728 2014-08-13 17:38:02.821827 7fe9bae887a0 20 read_log 192718'13153 (192718'13152) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:384 2014-08-06 14:20:11.451756 2014-08-13 17:38:02.821839 7fe9bae887a0 20 read_log 192718'13154 (192718'13153) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:384 2014-08-06 14:20:11.453180 2014-08-13 17:38:02.821851 7fe9bae887a0 20 read_log 192718'13155 (192718'13154) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:385 2014-08-06 14:20:11.508806 2014-08-13 17:38:02.821862 7fe9bae887a0 20 read_log 192718'13156 (192718'13155) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:385 2014-08-06 14:20:11.514143 2014-08-13 17:38:02.821874 7fe9bae887a0 20 read_log 192718'13157 (192718'13156) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:386 2014-08-06 14:20:11.567447 2014-08-13 17:38:02.821886 7fe9bae887a0 20 read_log 192718'13158 (192718'13157) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:386 2014-08-06 14:20:11.567521 2014-08-13 17:38:02.821897 7fe9bae887a0 20 read_log 192718'13159 (192718'13158) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:387 2014-08-06 14:20:11.615442 2014-08-13 17:38:02.821909 7fe9bae887a0 20 read_log 192718'13160 (192718'13159) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:387 2014-08-06 14:20:11.614314 2014-08-13 17:38:02.821927 7fe9bae887a0 20 read_log 192718'13161 (192718'13160) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:388 2014-08-06 14:20:11.662576 2014-08-13 17:38:02.821942 7fe9bae887a0 20 read_log 192718'13162 (192718'13161) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:388 2014-08-06 14:20:11.662637 2014-08-13 17:38:02.821954 7fe9bae887a0 20 read_log 192718'13163 (192718'13162) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:389 2014-08-06 14:20:11.739290 2014-08-13 17:38:02.821966 7fe9bae887a0 20 read_log 192718'13164 (192718'13163) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:389 2014-08-06 14:20:11.739303 2014-08-13 17:38:02.821977 7fe9bae887a0 20 read_log 192718'13165 (192718'13164) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:390 2014-08-06 14:20:11.792856 2014-08-13 17:38:02.821989 7fe9bae887a0 20 read_log 192718'13166 (192718'13165) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:390 2014-08-06 14:20:11.795751 2014-08-13 17:38:02.822001 7fe9bae887a0 20 read_log 192718'13167 (192718'13166) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:391 2014-08-06 14:20:11.846207 2014-08-13 17:38:02.822012 7fe9bae887a0 20 read_log 192718'13168 (192718'13167) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:391 2014-08-06 14:20:11.848697 2014-08-13 17:38:02.822024 7fe9bae887a0 20 read_log 192718'13169 (192718'13168) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:392 2014-08-06 14:20:12.103843 2014-08-13 17:38:02.822036 7fe9bae887a0 20 read_log 192718'13170 (192718'13169) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:392 2014-08-06 14:20:12.103852 2014-08-13 17:38:02.822047 7fe9bae887a0 20 read_log 192718'13171 (192718'13170) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:393 2014-08-06 14:20:12.156596 2014-08-13 17:38:02.822059 7fe9bae887a0 20 read_log 192718'13172 (192718'13171) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:393 2014-08-06 14:20:12.156630 2014-08-13 17:38:02.822071 7fe9bae887a0 20 read_log 192718'13173 (192718'13172) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:394 2014-08-06 14:20:12.206735 2014-08-13 17:38:02.822082 7fe9bae887a0 20 read_log 192718'13174 (192718'13173) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:394 2014-08-06 14:20:12.235698 2014-08-13 17:38:02.822094 7fe9bae887a0 20 read_log 192718'13175 (192718'13174) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:395 2014-08-06 14:20:12.438917 2014-08-13 17:38:02.822106 7fe9bae887a0 20 read_log 192718'13176 (192718'13175) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:395 2014-08-06 14:20:12.467530 2014-08-13 17:38:02.822117 7fe9bae887a0 20 read_log 192718'13177 (192718'13176) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:396 2014-08-06 14:20:12.547791 2014-08-13 17:38:02.822129 7fe9bae887a0 20 read_log 192718'13178 (192718'13177) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:396 2014-08-06 14:20:12.607150 2014-08-13 17:38:02.822141 7fe9bae887a0 20 read_log 192718'13179 (192718'13178) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:397 2014-08-06 14:20:12.668453 2014-08-13 17:38:02.822157 7fe9bae887a0 20 read_log 192718'13180 (192718'13179) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:397 2014-08-06 14:20:12.678935 2014-08-13 17:38:02.822172 7fe9bae887a0 20 read_log 192718'13181 (192718'13180) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:398 2014-08-06 14:20:12.745267 2014-08-13 17:38:02.822184 7fe9bae887a0 20 read_log 192718'13182 (192718'13181) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:398 2014-08-06 14:20:12.751988 2014-08-13 17:38:02.822195 7fe9bae887a0 20 read_log 192718'13183 (192718'13182) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:399 2014-08-06 14:20:13.044516 2014-08-13 17:38:02.822207 7fe9bae887a0 20 read_log 192718'13184 (192718'13183) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:399 2014-08-06 14:20:13.106547 2014-08-13 17:38:02.822228 7fe9bae887a0 20 read_log 192718'13185 (192718'13184) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:400 2014-08-06 14:20:13.186056 2014-08-13 17:38:02.822243 7fe9bae887a0 20 read_log 192718'13186 (192718'13185) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:400 2014-08-06 14:20:13.425274 2014-08-13 17:38:02.822255 7fe9bae887a0 20 read_log 192718'13187 (192718'13186) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:401 2014-08-06 14:20:13.615007 2014-08-13 17:38:02.822267 7fe9bae887a0 20 read_log 192718'13188 (192718'13187) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:401 2014-08-06 14:20:13.618620 2014-08-13 17:38:02.822281 7fe9bae887a0 20 read_log 192718'13189 (192718'13188) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:402 2014-08-06 14:20:13.697778 2014-08-13 17:38:02.822292 7fe9bae887a0 20 read_log 192718'13190 (192718'13189) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:402 2014-08-06 14:20:13.737969 2014-08-13 17:38:02.822304 7fe9bae887a0 20 read_log 192718'13191 (192718'13190) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:403 2014-08-06 14:20:13.808267 2014-08-13 17:38:02.822316 7fe9bae887a0 20 read_log 192718'13192 (192718'13191) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:403 2014-08-06 14:20:13.890650 2014-08-13 17:38:02.822327 7fe9bae887a0 20 read_log 192718'13193 (192718'13192) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:404 2014-08-06 14:20:13.975790 2014-08-13 17:38:02.822339 7fe9bae887a0 20 read_log 192718'13194 (192718'13193) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:404 2014-08-06 14:20:14.079804 2014-08-13 17:38:02.822351 7fe9bae887a0 20 read_log 192718'13195 (192718'13194) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:405 2014-08-06 14:20:14.127946 2014-08-13 17:38:02.822362 7fe9bae887a0 20 read_log 192718'13196 (192718'13195) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:405 2014-08-06 14:20:14.127851 2014-08-13 17:38:02.822374 7fe9bae887a0 20 read_log 192718'13197 (192718'13196) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:406 2014-08-06 14:20:14.172724 2014-08-13 17:38:02.822386 7fe9bae887a0 20 read_log 192718'13198 (192718'13197) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:406 2014-08-06 14:20:14.172866 2014-08-13 17:38:02.822397 7fe9bae887a0 20 read_log 192718'13199 (192718'13198) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:407 2014-08-06 14:20:14.199763 2014-08-13 17:38:02.822409 7fe9bae887a0 20 read_log 192718'13200 (192718'13199) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:407 2014-08-06 14:20:14.201338 2014-08-13 17:38:02.822421 7fe9bae887a0 20 read_log 192718'13201 (192718'13200) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:408 2014-08-06 14:20:14.229077 2014-08-13 17:38:02.822450 7fe9bae887a0 20 read_log 192718'13202 (192718'13201) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:408 2014-08-06 14:20:14.247520 2014-08-13 17:38:02.822462 7fe9bae887a0 20 read_log 192718'13203 (192718'13202) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:409 2014-08-06 14:20:14.272784 2014-08-13 17:38:02.822474 7fe9bae887a0 20 read_log 192718'13204 (192718'13203) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:409 2014-08-06 14:20:14.365455 2014-08-13 17:38:02.822486 7fe9bae887a0 20 read_log 192718'13205 (192718'13204) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:410 2014-08-06 14:20:14.385654 2014-08-13 17:38:02.822498 7fe9bae887a0 20 read_log 192718'13206 (192718'13205) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:410 2014-08-06 14:20:14.407133 2014-08-13 17:38:02.822509 7fe9bae887a0 20 read_log 192718'13207 (192718'13206) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:411 2014-08-06 14:20:14.444147 2014-08-13 17:38:02.822521 7fe9bae887a0 20 read_log 192718'13208 (192718'13207) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:411 2014-08-06 14:20:14.448299 2014-08-13 17:38:02.822541 7fe9bae887a0 20 read_log 192718'13209 (192718'13208) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:412 2014-08-06 14:20:14.494147 2014-08-13 17:38:02.822556 7fe9bae887a0 20 read_log 192718'13210 (192718'13209) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:412 2014-08-06 14:20:14.494176 2014-08-13 17:38:02.822568 7fe9bae887a0 20 read_log 192718'13211 (192718'13210) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:413 2014-08-06 14:20:14.551139 2014-08-13 17:38:02.822587 7fe9bae887a0 20 read_log 192718'13212 (192718'13211) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:413 2014-08-06 14:20:14.565780 2014-08-13 17:38:02.822599 7fe9bae887a0 20 read_log 192718'13213 (192718'13212) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:414 2014-08-06 14:20:14.632375 2014-08-13 17:38:02.822611 7fe9bae887a0 20 read_log 192718'13214 (192718'13213) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:414 2014-08-06 14:20:14.632434 2014-08-13 17:38:02.822623 7fe9bae887a0 20 read_log 192718'13215 (192718'13214) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:415 2014-08-06 14:20:14.680961 2014-08-13 17:38:02.822634 7fe9bae887a0 20 read_log 192718'13216 (192718'13215) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:415 2014-08-06 14:20:14.681021 2014-08-13 17:38:02.822646 7fe9bae887a0 20 read_log 192718'13217 (192718'13216) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:416 2014-08-06 14:20:14.731058 2014-08-13 17:38:02.822658 7fe9bae887a0 20 read_log 192718'13218 (192718'13217) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:416 2014-08-06 14:20:14.731074 2014-08-13 17:38:02.822670 7fe9bae887a0 20 read_log 192718'13219 (192718'13218) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:417 2014-08-06 14:20:14.782515 2014-08-13 17:38:02.822681 7fe9bae887a0 20 read_log 192718'13220 (192718'13219) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:417 2014-08-06 14:20:14.781787 2014-08-13 17:38:02.822693 7fe9bae887a0 20 read_log 192718'13221 (192718'13220) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:418 2014-08-06 14:20:14.832517 2014-08-13 17:38:02.822705 7fe9bae887a0 20 read_log 192718'13222 (192718'13221) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:418 2014-08-06 14:20:14.832561 2014-08-13 17:38:02.822716 7fe9bae887a0 20 read_log 192718'13223 (192718'13222) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:419 2014-08-06 14:20:14.882486 2014-08-13 17:38:02.822728 7fe9bae887a0 20 read_log 192718'13224 (192718'13223) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:419 2014-08-06 14:20:14.882751 2014-08-13 17:38:02.822740 7fe9bae887a0 20 read_log 192718'13225 (192718'13224) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:420 2014-08-06 14:20:14.941376 2014-08-13 17:38:02.822752 7fe9bae887a0 20 read_log 192718'13226 (192718'13225) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:420 2014-08-06 14:20:14.941317 2014-08-13 17:38:02.822763 7fe9bae887a0 20 read_log 192718'13227 (192718'13226) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:421 2014-08-06 14:20:14.994070 2014-08-13 17:38:02.822775 7fe9bae887a0 20 read_log 192718'13228 (192718'13227) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:421 2014-08-06 14:20:14.994051 2014-08-13 17:38:02.822787 7fe9bae887a0 20 read_log 192718'13229 (192718'13228) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:422 2014-08-06 14:20:15.046136 2014-08-13 17:38:02.822798 7fe9bae887a0 20 read_log 192718'13230 (192718'13229) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:422 2014-08-06 14:20:15.046169 2014-08-13 17:38:02.822810 7fe9bae887a0 20 read_log 192718'13231 (192718'13230) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:423 2014-08-06 14:20:15.094654 2014-08-13 17:38:02.822822 7fe9bae887a0 20 read_log 192718'13232 (192718'13231) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:423 2014-08-06 14:20:15.094553 2014-08-13 17:38:02.822840 7fe9bae887a0 20 read_log 192718'13233 (192718'13232) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:424 2014-08-06 14:20:15.145974 2014-08-13 17:38:02.822852 7fe9bae887a0 20 read_log 192718'13234 (192718'13233) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:424 2014-08-06 14:20:15.145912 2014-08-13 17:38:02.822864 7fe9bae887a0 20 read_log 192718'13235 (192718'13234) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:425 2014-08-06 14:20:15.195629 2014-08-13 17:38:02.822875 7fe9bae887a0 20 read_log 192718'13236 (192718'13235) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:425 2014-08-06 14:20:15.195589 2014-08-13 17:38:02.822887 7fe9bae887a0 20 read_log 192718'13237 (192718'13236) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:426 2014-08-06 14:20:15.239049 2014-08-13 17:38:02.822899 7fe9bae887a0 20 read_log 192718'13238 (192718'13237) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:426 2014-08-06 14:20:15.239021 2014-08-13 17:38:02.822910 7fe9bae887a0 20 read_log 192718'13239 (192718'13238) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:427 2014-08-06 14:20:15.293310 2014-08-13 17:38:02.822922 7fe9bae887a0 20 read_log 192718'13240 (192718'13239) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:427 2014-08-06 14:20:15.293310 2014-08-13 17:38:02.822934 7fe9bae887a0 20 read_log 192718'13241 (192718'13240) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:428 2014-08-06 14:20:15.345473 2014-08-13 17:38:02.822946 7fe9bae887a0 20 read_log 192718'13242 (192718'13241) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:428 2014-08-06 14:20:15.345644 2014-08-13 17:38:02.822957 7fe9bae887a0 20 read_log 192718'13243 (192718'13242) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:429 2014-08-06 14:20:15.404278 2014-08-13 17:38:02.822969 7fe9bae887a0 20 read_log 192718'13244 (192718'13243) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:429 2014-08-06 14:20:15.404545 2014-08-13 17:38:02.822981 7fe9bae887a0 20 read_log 192718'13245 (192718'13244) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:430 2014-08-06 14:20:15.456003 2014-08-13 17:38:02.822992 7fe9bae887a0 20 read_log 192718'13246 (192718'13245) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:430 2014-08-06 14:20:15.456027 2014-08-13 17:38:02.823004 7fe9bae887a0 20 read_log 192718'13247 (192718'13246) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:431 2014-08-06 14:20:15.512751 2014-08-13 17:38:02.823015 7fe9bae887a0 20 read_log 192718'13248 (192718'13247) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:431 2014-08-06 14:20:15.513907 2014-08-13 17:38:02.823027 7fe9bae887a0 20 read_log 192718'13249 (192718'13248) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:432 2014-08-06 14:20:15.564486 2014-08-13 17:38:02.823039 7fe9bae887a0 20 read_log 192718'13250 (192718'13249) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:432 2014-08-06 14:20:15.564445 2014-08-13 17:38:02.823050 7fe9bae887a0 20 read_log 192718'13251 (192718'13250) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:433 2014-08-06 14:20:15.616748 2014-08-13 17:38:02.823062 7fe9bae887a0 20 read_log 192718'13252 (192718'13251) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:433 2014-08-06 14:20:15.630699 2014-08-13 17:38:02.823074 7fe9bae887a0 20 read_log 192718'13253 (192718'13252) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:434 2014-08-06 14:20:15.665899 2014-08-13 17:38:02.823085 7fe9bae887a0 20 read_log 192718'13254 (192718'13253) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:434 2014-08-06 14:20:15.665848 2014-08-13 17:38:02.823097 7fe9bae887a0 20 read_log 192718'13255 (192718'13254) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:435 2014-08-06 14:20:15.717277 2014-08-13 17:38:02.823109 7fe9bae887a0 20 read_log 192718'13256 (192718'13255) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:435 2014-08-06 14:20:15.717310 2014-08-13 17:38:02.823126 7fe9bae887a0 20 read_log 192718'13257 (192718'13256) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:436 2014-08-06 14:20:15.768892 2014-08-13 17:38:02.823140 7fe9bae887a0 20 read_log 192718'13258 (192718'13257) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:436 2014-08-06 14:20:15.769144 2014-08-13 17:38:02.823152 7fe9bae887a0 20 read_log 192718'13259 (192718'13258) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:437 2014-08-06 14:20:15.798405 2014-08-13 17:38:02.823180 7fe9bae887a0 20 read_log 192718'13260 (192718'13259) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:437 2014-08-06 14:20:15.800263 2014-08-13 17:38:02.823192 7fe9bae887a0 20 read_log 192718'13261 (192718'13260) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:438 2014-08-06 14:20:15.828327 2014-08-13 17:38:02.823204 7fe9bae887a0 20 read_log 192718'13262 (192718'13261) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:438 2014-08-06 14:20:15.829748 2014-08-13 17:38:02.823218 7fe9bae887a0 20 read_log 192718'13263 (192718'13262) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:439 2014-08-06 14:20:15.938156 2014-08-13 17:38:02.823229 7fe9bae887a0 20 read_log 192718'13264 (192718'13263) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:439 2014-08-06 14:20:15.985137 2014-08-13 17:38:02.823241 7fe9bae887a0 20 read_log 192718'13265 (192718'13264) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:440 2014-08-06 14:20:16.014234 2014-08-13 17:38:02.823253 7fe9bae887a0 20 read_log 192718'13266 (192718'13265) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:440 2014-08-06 14:20:16.038986 2014-08-13 17:38:02.823265 7fe9bae887a0 20 read_log 192718'13267 (192718'13266) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:441 2014-08-06 14:20:16.076662 2014-08-13 17:38:02.823277 7fe9bae887a0 20 read_log 192718'13268 (192718'13267) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:441 2014-08-06 14:20:16.090390 2014-08-13 17:38:02.823289 7fe9bae887a0 20 read_log 192718'13269 (192718'13268) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:442 2014-08-06 14:20:16.139959 2014-08-13 17:38:02.823301 7fe9bae887a0 20 read_log 192718'13270 (192718'13269) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:442 2014-08-06 14:20:16.225427 2014-08-13 17:38:02.823313 7fe9bae887a0 20 read_log 192718'13271 (192718'13270) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:443 2014-08-06 14:20:16.302484 2014-08-13 17:38:02.823324 7fe9bae887a0 20 read_log 192718'13272 (192718'13271) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:443 2014-08-06 14:20:16.328123 2014-08-13 17:38:02.823336 7fe9bae887a0 20 read_log 192718'13273 (192718'13272) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:444 2014-08-06 14:20:16.391595 2014-08-13 17:38:02.823348 7fe9bae887a0 20 read_log 192718'13274 (192718'13273) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:444 2014-08-06 14:20:16.410360 2014-08-13 17:38:02.823359 7fe9bae887a0 20 read_log 192718'13275 (192718'13274) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:445 2014-08-06 14:20:16.430529 2014-08-13 17:38:02.823371 7fe9bae887a0 20 read_log 192718'13276 (192718'13275) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:445 2014-08-06 14:20:16.443579 2014-08-13 17:38:02.823383 7fe9bae887a0 20 read_log 192718'13277 (192718'13276) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:446 2014-08-06 14:20:16.460460 2014-08-13 17:38:02.823394 7fe9bae887a0 20 read_log 192718'13278 (192718'13277) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:446 2014-08-06 14:20:16.474248 2014-08-13 17:38:02.823406 7fe9bae887a0 20 read_log 192718'13279 (192718'13278) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:447 2014-08-06 14:20:16.514000 2014-08-13 17:38:02.823418 7fe9bae887a0 20 read_log 192718'13280 (192718'13279) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:447 2014-08-06 14:20:16.537587 2014-08-13 17:38:02.823436 7fe9bae887a0 20 read_log 192718'13281 (192718'13280) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:448 2014-08-06 14:20:16.565310 2014-08-13 17:38:02.823448 7fe9bae887a0 20 read_log 192718'13282 (192718'13281) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:448 2014-08-06 14:20:16.590346 2014-08-13 17:38:02.823460 7fe9bae887a0 20 read_log 192718'13283 (192718'13282) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:449 2014-08-06 14:20:16.623875 2014-08-13 17:38:02.823472 7fe9bae887a0 20 read_log 192718'13284 (192718'13283) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:449 2014-08-06 14:20:16.779209 2014-08-13 17:38:02.823484 7fe9bae887a0 20 read_log 192718'13285 (192718'13284) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:450 2014-08-06 14:20:16.881824 2014-08-13 17:38:02.823495 7fe9bae887a0 20 read_log 192718'13286 (192718'13285) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:450 2014-08-06 14:20:17.005386 2014-08-13 17:38:02.823507 7fe9bae887a0 20 read_log 192718'13287 (192718'13286) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:451 2014-08-06 14:20:17.119132 2014-08-13 17:38:02.823519 7fe9bae887a0 20 read_log 192718'13288 (192718'13287) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:451 2014-08-06 14:20:17.165051 2014-08-13 17:38:02.823530 7fe9bae887a0 20 read_log 192718'13289 (192718'13288) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:452 2014-08-06 14:20:17.209092 2014-08-13 17:38:02.823542 7fe9bae887a0 20 read_log 192718'13290 (192718'13289) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:452 2014-08-06 14:20:17.238485 2014-08-13 17:38:02.823554 7fe9bae887a0 20 read_log 192718'13291 (192718'13290) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:453 2014-08-06 14:20:17.279006 2014-08-13 17:38:02.823566 7fe9bae887a0 20 read_log 192718'13292 (192718'13291) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:453 2014-08-06 14:20:17.319177 2014-08-13 17:38:02.823579 7fe9bae887a0 20 read_log 192718'13293 (192718'13292) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:454 2014-08-06 14:20:17.433741 2014-08-13 17:38:02.823592 7fe9bae887a0 20 read_log 192718'13294 (192718'13293) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:454 2014-08-06 14:20:17.639748 2014-08-13 17:38:02.823606 7fe9bae887a0 20 read_log 192718'13295 (192718'13294) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:455 2014-08-06 14:20:17.730929 2014-08-13 17:38:02.823617 7fe9bae887a0 20 read_log 192718'13296 (192718'13295) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:455 2014-08-06 14:20:17.893304 2014-08-13 17:38:02.823629 7fe9bae887a0 20 read_log 192718'13297 (192718'13296) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:456 2014-08-06 14:20:18.021243 2014-08-13 17:38:02.823641 7fe9bae887a0 20 read_log 192718'13298 (192718'13297) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:456 2014-08-06 14:20:18.091822 2014-08-13 17:38:02.823653 7fe9bae887a0 20 read_log 192718'13299 (192718'13298) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:457 2014-08-06 14:20:18.138525 2014-08-13 17:38:02.823666 7fe9bae887a0 20 read_log 192718'13300 (192718'13299) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:457 2014-08-06 14:20:18.198240 2014-08-13 17:38:02.823677 7fe9bae887a0 20 read_log 192718'13301 (192718'13300) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:458 2014-08-06 14:20:18.312164 2014-08-13 17:38:02.823689 7fe9bae887a0 20 read_log 192718'13302 (192718'13301) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:458 2014-08-06 14:20:18.423236 2014-08-13 17:38:02.823701 7fe9bae887a0 20 read_log 192718'13303 (192718'13302) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:459 2014-08-06 14:20:18.866107 2014-08-13 17:38:02.823713 7fe9bae887a0 20 read_log 192718'13304 (192718'13303) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:459 2014-08-06 14:20:19.084168 2014-08-13 17:38:02.823814 7fe9bae887a0 20 read_log 192718'13305 (192718'13304) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:460 2014-08-06 14:20:19.154382 2014-08-13 17:38:02.823832 7fe9bae887a0 20 read_log 192718'13306 (192718'13305) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:460 2014-08-06 14:20:19.236461 2014-08-13 17:38:02.823844 7fe9bae887a0 20 read_log 192718'13307 (192718'13306) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:461 2014-08-06 14:20:19.290831 2014-08-13 17:38:02.823856 7fe9bae887a0 20 read_log 192718'13308 (192718'13307) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:461 2014-08-06 14:20:19.314630 2014-08-13 17:38:02.823870 7fe9bae887a0 20 read_log 192718'13309 (192718'13308) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:462 2014-08-06 14:20:19.350155 2014-08-13 17:38:02.823884 7fe9bae887a0 20 read_log 192718'13310 (192718'13309) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:462 2014-08-06 14:20:19.368028 2014-08-13 17:38:02.823896 7fe9bae887a0 20 read_log 192718'13311 (192718'13310) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:463 2014-08-06 14:20:19.394650 2014-08-13 17:38:02.823911 7fe9bae887a0 20 read_log 192718'13312 (192718'13311) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:463 2014-08-06 14:20:19.418823 2014-08-13 17:38:02.823923 7fe9bae887a0 20 read_log 192718'13313 (192718'13312) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:464 2014-08-06 14:20:19.449666 2014-08-13 17:38:02.823937 7fe9bae887a0 20 read_log 192718'13314 (192718'13313) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:464 2014-08-06 14:20:19.600916 2014-08-13 17:38:02.823949 7fe9bae887a0 20 read_log 192718'13315 (192718'13314) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:465 2014-08-06 14:20:19.657144 2014-08-13 17:38:02.823961 7fe9bae887a0 20 read_log 192718'13316 (192718'13315) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:465 2014-08-06 14:20:19.708572 2014-08-13 17:38:02.823973 7fe9bae887a0 20 read_log 192718'13317 (192718'13316) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:466 2014-08-06 14:20:19.763741 2014-08-13 17:38:02.823984 7fe9bae887a0 20 read_log 192718'13318 (192718'13317) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:466 2014-08-06 14:20:19.787162 2014-08-13 17:38:02.823996 7fe9bae887a0 20 read_log 192718'13319 (192718'13318) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:467 2014-08-06 14:20:19.823020 2014-08-13 17:38:02.824010 7fe9bae887a0 20 read_log 192718'13320 (192718'13319) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:467 2014-08-06 14:20:19.840519 2014-08-13 17:38:02.824022 7fe9bae887a0 20 read_log 192718'13321 (192718'13320) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:468 2014-08-06 14:20:19.866867 2014-08-13 17:38:02.824037 7fe9bae887a0 20 read_log 192718'13322 (192718'13321) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:468 2014-08-06 14:20:19.891767 2014-08-13 17:38:02.824049 7fe9bae887a0 20 read_log 192718'13323 (192718'13322) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:469 2014-08-06 14:20:19.927191 2014-08-13 17:38:02.824063 7fe9bae887a0 20 read_log 192718'13324 (192718'13323) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:469 2014-08-06 14:20:20.064167 2014-08-13 17:38:02.824079 7fe9bae887a0 20 read_log 192718'13325 (192718'13324) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:470 2014-08-06 14:20:20.142122 2014-08-13 17:38:02.824092 7fe9bae887a0 20 read_log 192718'13326 (192718'13325) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:470 2014-08-06 14:20:20.315482 2014-08-13 17:38:02.824103 7fe9bae887a0 20 read_log 192718'13327 (192718'13326) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:471 2014-08-06 14:20:20.363447 2014-08-13 17:38:02.824117 7fe9bae887a0 20 read_log 192718'13328 (192718'13327) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:471 2014-08-06 14:20:20.413264 2014-08-13 17:38:02.824135 7fe9bae887a0 20 read_log 192718'13329 (192718'13328) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:472 2014-08-06 14:20:20.532506 2014-08-13 17:38:02.824150 7fe9bae887a0 20 read_log 192718'13330 (192718'13329) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:472 2014-08-06 14:20:20.587426 2014-08-13 17:38:02.824170 7fe9bae887a0 20 read_log 192718'13331 (192718'13330) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:473 2014-08-06 14:20:20.621479 2014-08-13 17:38:02.824182 7fe9bae887a0 20 read_log 192718'13332 (192718'13331) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:473 2014-08-06 14:20:20.660494 2014-08-13 17:38:02.824194 7fe9bae887a0 20 read_log 192718'13333 (192718'13332) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:474 2014-08-06 14:20:20.813856 2014-08-13 17:38:02.824206 7fe9bae887a0 20 read_log 192718'13334 (192718'13333) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:474 2014-08-06 14:20:20.940826 2014-08-13 17:38:02.824218 7fe9bae887a0 20 read_log 192718'13335 (192718'13334) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:475 2014-08-06 14:20:20.962441 2014-08-13 17:38:02.824232 7fe9bae887a0 20 read_log 192718'13336 (192718'13335) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:475 2014-08-06 14:20:20.981178 2014-08-13 17:38:02.824244 7fe9bae887a0 20 read_log 192718'13337 (192718'13336) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:476 2014-08-06 14:20:21.001847 2014-08-13 17:38:02.824258 7fe9bae887a0 20 read_log 192718'13338 (192718'13337) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:476 2014-08-06 14:20:21.031990 2014-08-13 17:38:02.824270 7fe9bae887a0 20 read_log 192718'13339 (192718'13338) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:477 2014-08-06 14:20:21.068641 2014-08-13 17:38:02.824282 7fe9bae887a0 20 read_log 192718'13340 (192718'13339) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:477 2014-08-06 14:20:21.078172 2014-08-13 17:38:02.824294 7fe9bae887a0 20 read_log 192718'13341 (192718'13340) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:478 2014-08-06 14:20:21.115904 2014-08-13 17:38:02.824306 7fe9bae887a0 20 read_log 192718'13342 (192718'13341) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:478 2014-08-06 14:20:21.136795 2014-08-13 17:38:02.824320 7fe9bae887a0 20 read_log 192718'13343 (192718'13342) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:479 2014-08-06 14:20:21.172279 2014-08-13 17:38:02.824332 7fe9bae887a0 20 read_log 192718'13344 (192718'13343) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:479 2014-08-06 14:20:21.281631 2014-08-13 17:38:02.824346 7fe9bae887a0 20 read_log 192718'13345 (192718'13344) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:480 2014-08-06 14:20:21.303459 2014-08-13 17:38:02.824359 7fe9bae887a0 20 read_log 192718'13346 (192718'13345) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:480 2014-08-06 14:20:21.322075 2014-08-13 17:38:02.824372 7fe9bae887a0 20 read_log 192718'13347 (192718'13346) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:481 2014-08-06 14:20:21.343701 2014-08-13 17:38:02.824384 7fe9bae887a0 20 read_log 192718'13348 (192718'13347) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:481 2014-08-06 14:20:21.362298 2014-08-13 17:38:02.824398 7fe9bae887a0 20 read_log 192718'13349 (192718'13348) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:482 2014-08-06 14:20:21.434772 2014-08-13 17:38:02.824410 7fe9bae887a0 20 read_log 192718'13350 (192718'13349) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:482 2014-08-06 14:20:21.456347 2014-08-13 17:38:02.824423 7fe9bae887a0 20 read_log 192718'13351 (192718'13350) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:483 2014-08-06 14:20:21.494203 2014-08-13 17:38:02.824435 7fe9bae887a0 20 read_log 192718'13352 (192718'13351) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:483 2014-08-06 14:20:21.514921 2014-08-13 17:38:02.824455 7fe9bae887a0 20 read_log 192718'13353 (192718'13352) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:484 2014-08-06 14:20:21.543789 2014-08-13 17:38:02.824469 7fe9bae887a0 20 read_log 192718'13354 (192718'13353) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:484 2014-08-06 14:20:21.576556 2014-08-13 17:38:02.824481 7fe9bae887a0 20 read_log 192718'13355 (192718'13354) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:485 2014-08-06 14:20:21.598401 2014-08-13 17:38:02.824493 7fe9bae887a0 20 read_log 192718'13356 (192718'13355) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:485 2014-08-06 14:20:21.629285 2014-08-13 17:38:02.824507 7fe9bae887a0 20 read_log 192718'13357 (192718'13356) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:486 2014-08-06 14:20:21.669828 2014-08-13 17:38:02.824519 7fe9bae887a0 20 read_log 192718'13358 (192718'13357) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:486 2014-08-06 14:20:21.722981 2014-08-13 17:38:02.824533 7fe9bae887a0 20 read_log 192718'13359 (192718'13358) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:487 2014-08-06 14:20:21.756947 2014-08-13 17:38:02.824545 7fe9bae887a0 20 read_log 192718'13360 (192718'13359) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:487 2014-08-06 14:20:21.765643 2014-08-13 17:38:02.824559 7fe9bae887a0 20 read_log 192718'13361 (192718'13360) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:488 2014-08-06 14:20:21.795073 2014-08-13 17:38:02.824571 7fe9bae887a0 20 read_log 192718'13362 (192718'13361) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:488 2014-08-06 14:20:21.818161 2014-08-13 17:38:02.824582 7fe9bae887a0 20 read_log 192718'13363 (192718'13362) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:489 2014-08-06 14:20:21.881334 2014-08-13 17:38:02.824594 7fe9bae887a0 20 read_log 192718'13364 (192718'13363) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:489 2014-08-06 14:20:22.048758 2014-08-13 17:38:02.824606 7fe9bae887a0 20 read_log 192718'13365 (192718'13364) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:490 2014-08-06 14:20:22.109540 2014-08-13 17:38:02.824619 7fe9bae887a0 20 read_log 192718'13366 (192718'13365) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:490 2014-08-06 14:20:22.160904 2014-08-13 17:38:02.824631 7fe9bae887a0 20 read_log 192718'13367 (192718'13366) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:491 2014-08-06 14:20:22.207335 2014-08-13 17:38:02.824645 7fe9bae887a0 20 read_log 192718'13368 (192718'13367) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:491 2014-08-06 14:20:22.250512 2014-08-13 17:38:02.824657 7fe9bae887a0 20 read_log 192718'13369 (192718'13368) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:492 2014-08-06 14:20:22.286245 2014-08-13 17:38:02.824669 7fe9bae887a0 20 read_log 192718'13370 (192718'13369) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:492 2014-08-06 14:20:22.302424 2014-08-13 17:38:02.824683 7fe9bae887a0 20 read_log 192718'13371 (192718'13370) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:493 2014-08-06 14:20:22.325248 2014-08-13 17:38:02.824695 7fe9bae887a0 20 read_log 192718'13372 (192718'13371) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:493 2014-08-06 14:20:22.348834 2014-08-13 17:38:02.824711 7fe9bae887a0 20 read_log 192718'13373 (192718'13372) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:494 2014-08-06 14:20:22.405785 2014-08-13 17:38:02.824724 7fe9bae887a0 20 read_log 192718'13374 (192718'13373) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:494 2014-08-06 14:20:22.541010 2014-08-13 17:38:02.824735 7fe9bae887a0 20 read_log 192718'13375 (192718'13374) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:495 2014-08-06 14:20:22.595949 2014-08-13 17:38:02.824749 7fe9bae887a0 20 read_log 192718'13376 (192718'13375) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:495 2014-08-06 14:20:22.777152 2014-08-13 17:38:02.824770 7fe9bae887a0 20 read_log 192718'13377 (192718'13376) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:496 2014-08-06 14:20:22.870953 2014-08-13 17:38:02.824784 7fe9bae887a0 20 read_log 192718'13378 (192718'13377) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:496 2014-08-06 14:20:22.922780 2014-08-13 17:38:02.824796 7fe9bae887a0 20 read_log 192718'13379 (192718'13378) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:497 2014-08-06 14:20:22.972179 2014-08-13 17:38:02.824808 7fe9bae887a0 20 read_log 192718'13380 (192718'13379) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:497 2014-08-06 14:20:22.994578 2014-08-13 17:38:02.824822 7fe9bae887a0 20 read_log 192718'13381 (192718'13380) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:498 2014-08-06 14:20:23.067651 2014-08-13 17:38:02.824834 7fe9bae887a0 20 read_log 192718'13382 (192718'13381) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:498 2014-08-06 14:20:23.101849 2014-08-13 17:38:02.824848 7fe9bae887a0 20 read_log 192718'13383 (192718'13382) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:499 2014-08-06 14:20:23.147542 2014-08-13 17:38:02.824860 7fe9bae887a0 20 read_log 192718'13384 (192718'13383) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:499 2014-08-06 14:20:23.372579 2014-08-13 17:38:02.824874 7fe9bae887a0 20 read_log 192718'13385 (192718'13384) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:500 2014-08-06 14:20:23.520053 2014-08-13 17:38:02.824886 7fe9bae887a0 20 read_log 192718'13388 (192718'13385) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:500 2014-08-06 14:20:23.593520 2014-08-13 17:38:02.824898 7fe9bae887a0 20 read_log 192718'13389 (192718'13388) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:501 2014-08-06 14:20:23.714941 2014-08-13 17:38:02.824911 7fe9bae887a0 20 read_log 192718'13390 (192718'13389) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:501 2014-08-06 14:20:23.769108 2014-08-13 17:38:02.824923 7fe9bae887a0 20 read_log 192718'13391 (192718'13390) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:502 2014-08-06 14:20:24.210977 2014-08-13 17:38:02.824936 7fe9bae887a0 20 read_log 192718'13392 (192718'13391) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:502 2014-08-06 14:20:24.252633 2014-08-13 17:38:02.824951 7fe9bae887a0 20 read_log 192718'13393 (192718'13392) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:503 2014-08-06 14:20:24.280064 2014-08-13 17:38:02.824962 7fe9bae887a0 20 read_log 192718'13394 (192718'13393) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:503 2014-08-06 14:20:24.334842 2014-08-13 17:38:02.824977 7fe9bae887a0 20 read_log 192718'13395 (192718'13394) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:504 2014-08-06 14:20:24.449686 2014-08-13 17:38:02.824988 7fe9bae887a0 20 read_log 192718'13396 (192718'13395) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:504 2014-08-06 14:20:24.511926 2014-08-13 17:38:02.825001 7fe9bae887a0 20 read_log 192718'13397 (192718'13396) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:505 2014-08-06 14:20:24.753746 2014-08-13 17:38:02.825034 7fe9bae887a0 20 read_log 192718'13398 (192718'13397) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:505 2014-08-06 14:20:24.791125 2014-08-13 17:38:02.825048 7fe9bae887a0 20 read_log 192718'13399 (192718'13398) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:506 2014-08-06 14:20:24.836681 2014-08-13 17:38:02.825060 7fe9bae887a0 20 read_log 192718'13400 (192718'13399) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:506 2014-08-06 14:20:24.906573 2014-08-13 17:38:02.825072 7fe9bae887a0 20 read_log 192718'13401 (192718'13400) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:507 2014-08-06 14:20:25.075479 2014-08-13 17:38:02.825086 7fe9bae887a0 20 read_log 192718'13402 (192718'13401) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:507 2014-08-06 14:20:25.117971 2014-08-13 17:38:02.825104 7fe9bae887a0 20 read_log 192718'13403 (192718'13402) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:508 2014-08-06 14:20:25.167049 2014-08-13 17:38:02.825119 7fe9bae887a0 20 read_log 192718'13404 (192718'13403) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:508 2014-08-06 14:20:25.191495 2014-08-13 17:38:02.825131 7fe9bae887a0 20 read_log 192718'13405 (192718'13404) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:509 2014-08-06 14:20:25.226747 2014-08-13 17:38:02.825142 7fe9bae887a0 20 read_log 192718'13406 (192718'13405) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:509 2014-08-06 14:20:25.348127 2014-08-13 17:38:02.825160 7fe9bae887a0 20 read_log 192718'13407 (192718'13406) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:510 2014-08-06 14:20:25.397085 2014-08-13 17:38:02.825175 7fe9bae887a0 20 read_log 192718'13408 (192718'13407) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:510 2014-08-06 14:20:25.421465 2014-08-13 17:38:02.825186 7fe9bae887a0 20 read_log 192718'13409 (192718'13408) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:511 2014-08-06 14:20:25.476340 2014-08-13 17:38:02.825198 7fe9bae887a0 20 read_log 192718'13410 (192718'13409) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:511 2014-08-06 14:20:25.672362 2014-08-13 17:38:02.825210 7fe9bae887a0 20 read_log 192718'13411 (192718'13410) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:512 2014-08-06 14:20:25.719235 2014-08-13 17:38:02.825222 7fe9bae887a0 20 read_log 192718'13412 (192718'13411) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:512 2014-08-06 14:20:25.738373 2014-08-13 17:38:02.825234 7fe9bae887a0 20 read_log 192718'13413 (192718'13412) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:513 2014-08-06 14:20:25.777508 2014-08-13 17:38:02.825245 7fe9bae887a0 20 read_log 192718'13414 (192718'13413) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:513 2014-08-06 14:20:25.804636 2014-08-13 17:38:02.825257 7fe9bae887a0 20 read_log 192718'13415 (192718'13414) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:514 2014-08-06 14:20:25.860788 2014-08-13 17:38:02.825268 7fe9bae887a0 20 read_log 192718'13416 (192718'13415) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:514 2014-08-06 14:20:25.975697 2014-08-13 17:38:02.825282 7fe9bae887a0 20 read_log 192718'13417 (192718'13416) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:515 2014-08-06 14:20:26.009905 2014-08-13 17:38:02.825294 7fe9bae887a0 20 read_log 192718'13418 (192718'13417) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:515 2014-08-06 14:20:26.036843 2014-08-13 17:38:02.825308 7fe9bae887a0 20 read_log 192718'13419 (192718'13418) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:516 2014-08-06 14:20:26.081636 2014-08-13 17:38:02.825320 7fe9bae887a0 20 read_log 192718'13420 (192718'13419) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:516 2014-08-06 14:20:26.094363 2014-08-13 17:38:02.825353 7fe9bae887a0 20 read_log 192718'13421 (192718'13420) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:517 2014-08-06 14:20:26.134549 2014-08-13 17:38:02.825365 7fe9bae887a0 20 read_log 192718'13422 (192718'13421) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:517 2014-08-06 14:20:26.144087 2014-08-13 17:38:02.825379 7fe9bae887a0 20 read_log 192718'13423 (192718'13422) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:518 2014-08-06 14:20:26.179724 2014-08-13 17:38:02.825391 7fe9bae887a0 20 read_log 192718'13424 (192718'13423) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:518 2014-08-06 14:20:26.393002 2014-08-13 17:38:02.825405 7fe9bae887a0 20 read_log 192718'13425 (192718'13424) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:519 2014-08-06 14:20:26.436136 2014-08-13 17:38:02.825417 7fe9bae887a0 20 read_log 192718'13426 (192718'13425) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:519 2014-08-06 14:20:26.563209 2014-08-13 17:38:02.825437 7fe9bae887a0 20 read_log 192718'13427 (192718'13426) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:520 2014-08-06 14:20:26.664051 2014-08-13 17:38:02.825452 7fe9bae887a0 20 read_log 192718'13428 (192718'13427) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:520 2014-08-06 14:20:26.712815 2014-08-13 17:38:02.825464 7fe9bae887a0 20 read_log 192718'13429 (192718'13428) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:521 2014-08-06 14:20:26.750518 2014-08-13 17:38:02.825475 7fe9bae887a0 20 read_log 192718'13430 (192718'13429) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:521 2014-08-06 14:20:26.769964 2014-08-13 17:38:02.825487 7fe9bae887a0 20 read_log 192718'13431 (192718'13430) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:522 2014-08-06 14:20:26.816315 2014-08-13 17:38:02.825499 7fe9bae887a0 20 read_log 192718'13432 (192718'13431) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:522 2014-08-06 14:20:26.830586 2014-08-13 17:38:02.825511 7fe9bae887a0 20 read_log 192718'13433 (192718'13432) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:523 2014-08-06 14:20:26.861963 2014-08-13 17:38:02.825523 7fe9bae887a0 20 read_log 192718'13434 (192718'13433) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:523 2014-08-06 14:20:26.880525 2014-08-13 17:38:02.825534 7fe9bae887a0 20 read_log 192718'13435 (192718'13434) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:524 2014-08-06 14:20:26.950470 2014-08-13 17:38:02.825546 7fe9bae887a0 20 read_log 192718'13436 (192718'13435) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:524 2014-08-06 14:20:26.991472 2014-08-13 17:38:02.825558 7fe9bae887a0 20 read_log 192718'13437 (192718'13436) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:525 2014-08-06 14:20:27.009449 2014-08-13 17:38:02.825570 7fe9bae887a0 20 read_log 192718'13438 (192718'13437) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:525 2014-08-06 14:20:27.030752 2014-08-13 17:38:02.825581 7fe9bae887a0 20 read_log 192718'13439 (192718'13438) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:526 2014-08-06 14:20:27.067960 2014-08-13 17:38:02.825593 7fe9bae887a0 20 read_log 192718'13440 (192718'13439) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:526 2014-08-06 14:20:27.102008 2014-08-13 17:38:02.825605 7fe9bae887a0 20 read_log 192718'13441 (192718'13440) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:527 2014-08-06 14:20:27.207336 2014-08-13 17:38:02.825617 7fe9bae887a0 20 read_log 192718'13442 (192718'13441) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:527 2014-08-06 14:20:27.446459 2014-08-13 17:38:02.825629 7fe9bae887a0 20 read_log 192718'13443 (192718'13442) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:528 2014-08-06 14:20:27.485087 2014-08-13 17:38:02.825641 7fe9bae887a0 20 read_log 192718'13444 (192718'13443) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:528 2014-08-06 14:20:27.522120 2014-08-13 17:38:02.825652 7fe9bae887a0 20 read_log 192718'13445 (192718'13444) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:529 2014-08-06 14:20:27.569212 2014-08-13 17:38:02.825664 7fe9bae887a0 20 read_log 192718'13446 (192718'13445) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:529 2014-08-06 14:20:27.599378 2014-08-13 17:38:02.825676 7fe9bae887a0 20 read_log 192718'13447 (192718'13446) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:530 2014-08-06 14:20:27.637596 2014-08-13 17:38:02.825688 7fe9bae887a0 20 read_log 192718'13448 (192718'13447) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:530 2014-08-06 14:20:27.674712 2014-08-13 17:38:02.825700 7fe9bae887a0 20 read_log 192718'13449 (192718'13448) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:531 2014-08-06 14:20:27.714108 2014-08-13 17:38:02.825711 7fe9bae887a0 20 read_log 192718'13450 (192718'13449) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:531 2014-08-06 14:20:27.748999 2014-08-13 17:38:02.825729 7fe9bae887a0 20 read_log 192718'13451 (192718'13450) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:532 2014-08-06 14:20:27.796504 2014-08-13 17:38:02.825742 7fe9bae887a0 20 read_log 192718'13452 (192718'13451) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:532 2014-08-06 14:20:27.919445 2014-08-13 17:38:02.825754 7fe9bae887a0 20 read_log 192718'13453 (192718'13452) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:533 2014-08-06 14:20:27.961700 2014-08-13 17:38:02.825765 7fe9bae887a0 20 read_log 192718'13454 (192718'13453) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:533 2014-08-06 14:20:27.995143 2014-08-13 17:38:02.825777 7fe9bae887a0 20 read_log 192718'13455 (192718'13454) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:534 2014-08-06 14:20:28.053182 2014-08-13 17:38:02.825789 7fe9bae887a0 20 read_log 192718'13456 (192718'13455) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:534 2014-08-06 14:20:28.268286 2014-08-13 17:38:02.825801 7fe9bae887a0 20 read_log 192718'13457 (192718'13456) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:535 2014-08-06 14:20:28.290926 2014-08-13 17:38:02.825813 7fe9bae887a0 20 read_log 192718'13458 (192718'13457) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:535 2014-08-06 14:20:28.341313 2014-08-13 17:38:02.825824 7fe9bae887a0 20 read_log 192718'13459 (192718'13458) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:536 2014-08-06 14:20:28.384376 2014-08-13 17:38:02.825836 7fe9bae887a0 20 read_log 192718'13460 (192718'13459) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:536 2014-08-06 14:20:28.419676 2014-08-13 17:38:02.825848 7fe9bae887a0 20 read_log 192718'13461 (192718'13460) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:537 2014-08-06 14:20:28.463673 2014-08-13 17:38:02.825859 7fe9bae887a0 20 read_log 192718'13462 (192718'13461) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:537 2014-08-06 14:20:28.551931 2014-08-13 17:38:02.825871 7fe9bae887a0 20 read_log 192718'13463 (192718'13462) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:538 2014-08-06 14:20:28.566900 2014-08-13 17:38:02.825883 7fe9bae887a0 20 read_log 192718'13464 (192718'13463) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:538 2014-08-06 14:20:28.592885 2014-08-13 17:38:02.825894 7fe9bae887a0 20 read_log 192718'13465 (192718'13464) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:539 2014-08-06 14:20:28.614370 2014-08-13 17:38:02.825906 7fe9bae887a0 20 read_log 192718'13466 (192718'13465) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:539 2014-08-06 14:20:28.635217 2014-08-13 17:38:02.825918 7fe9bae887a0 20 read_log 192718'13467 (192718'13466) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:540 2014-08-06 14:20:28.679220 2014-08-13 17:38:02.825930 7fe9bae887a0 20 read_log 192718'13468 (192718'13467) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:540 2014-08-06 14:20:28.820308 2014-08-13 17:38:02.825941 7fe9bae887a0 20 read_log 192718'13469 (192718'13468) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:541 2014-08-06 14:20:28.841805 2014-08-13 17:38:02.825953 7fe9bae887a0 20 read_log 192718'13470 (192718'13469) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:541 2014-08-06 14:20:28.863588 2014-08-13 17:38:02.825965 7fe9bae887a0 20 read_log 192718'13471 (192718'13470) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:542 2014-08-06 14:20:28.885568 2014-08-13 17:38:02.825976 7fe9bae887a0 20 read_log 192718'13472 (192718'13471) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:542 2014-08-06 14:20:29.026316 2014-08-13 17:38:02.825988 7fe9bae887a0 20 read_log 192718'13473 (192718'13472) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:543 2014-08-06 14:20:29.056754 2014-08-13 17:38:02.825999 7fe9bae887a0 20 read_log 192718'13474 (192718'13473) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:543 2014-08-06 14:20:29.073678 2014-08-13 17:38:02.826035 7fe9bae887a0 20 read_log 192718'13475 (192718'13474) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:544 2014-08-06 14:20:29.099106 2014-08-13 17:38:02.826051 7fe9bae887a0 20 read_log 192718'13476 (192718'13475) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:544 2014-08-06 14:20:29.127153 2014-08-13 17:38:02.826063 7fe9bae887a0 20 read_log 192718'13477 (192718'13476) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:545 2014-08-06 14:20:29.183520 2014-08-13 17:38:02.826075 7fe9bae887a0 20 read_log 192718'13478 (192718'13477) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:545 2014-08-06 14:20:29.185933 2014-08-13 17:38:02.826087 7fe9bae887a0 20 read_log 192718'13479 (192718'13478) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:546 2014-08-06 14:20:29.215487 2014-08-13 17:38:02.826099 7fe9bae887a0 20 read_log 192718'13480 (192718'13479) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:546 2014-08-06 14:20:29.238911 2014-08-13 17:38:02.826111 7fe9bae887a0 20 read_log 192718'13481 (192718'13480) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:547 2014-08-06 14:20:29.272901 2014-08-13 17:38:02.826122 7fe9bae887a0 20 read_log 192718'13482 (192718'13481) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:547 2014-08-06 14:20:29.442239 2014-08-13 17:38:02.826134 7fe9bae887a0 20 read_log 192718'13483 (192718'13482) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:548 2014-08-06 14:20:29.580341 2014-08-13 17:38:02.826146 7fe9bae887a0 20 read_log 192718'13484 (192718'13483) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:548 2014-08-06 14:20:29.593043 2014-08-13 17:38:02.826163 7fe9bae887a0 20 read_log 192718'13485 (192718'13484) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:549 2014-08-06 14:20:29.617563 2014-08-13 17:38:02.826175 7fe9bae887a0 20 read_log 192718'13486 (192718'13485) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:549 2014-08-06 14:20:29.635197 2014-08-13 17:38:02.826187 7fe9bae887a0 20 read_log 192718'13487 (192718'13486) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:550 2014-08-06 14:20:29.664027 2014-08-13 17:38:02.826199 7fe9bae887a0 20 read_log 192718'13488 (192718'13487) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:550 2014-08-06 14:20:29.671844 2014-08-13 17:38:02.826210 7fe9bae887a0 20 read_log 192718'13489 (192718'13488) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:551 2014-08-06 14:20:29.698319 2014-08-13 17:38:02.826222 7fe9bae887a0 20 read_log 192718'13490 (192718'13489) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:551 2014-08-06 14:20:29.717411 2014-08-13 17:38:02.826251 7fe9bae887a0 20 read_log 192718'13491 (192718'13490) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:552 2014-08-06 14:20:29.761434 2014-08-13 17:38:02.826264 7fe9bae887a0 20 read_log 192718'13492 (192718'13491) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:552 2014-08-06 14:20:29.839956 2014-08-13 17:38:02.826276 7fe9bae887a0 20 read_log 192718'13493 (192718'13492) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:553 2014-08-06 14:20:29.886066 2014-08-13 17:38:02.826288 7fe9bae887a0 20 read_log 192718'13494 (192718'13493) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:553 2014-08-06 14:20:29.912573 2014-08-13 17:38:02.826300 7fe9bae887a0 20 read_log 192718'13837 (192718'13494) clean 407860cd/core.47650+11+2/head//24 by osd.371.0:25752 2014-08-06 14:52:03.375393 2014-08-13 17:38:02.826312 7fe9bae887a0 20 read_log 192798'14100 (192718'13837) delete 407860cd/core.47650+11+2/head//24 by osd.371.0:26877 2014-08-11 10:24:43.645971 2014-08-13 17:38:02.826324 7fe9bae887a0 20 read_log 192834'14113 (0'0) promote adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.371.0:26970 2014-08-11 15:23:36.476590 2014-08-13 17:38:02.826336 7fe9bae887a0 20 read_log 192834'14114 (192834'14113) modify adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by client.933448.0:207 2014-08-11 15:23:36.470563 2014-08-13 17:38:02.826357 7fe9bae887a0 20 read_log 192834'14125 (0'0) promote fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by osd.371.0:26994 2014-08-11 15:23:40.726370 2014-08-13 17:38:02.826370 7fe9bae887a0 20 read_log 192834'14126 (192834'14125) modify fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.933448.0:1305 2014-08-11 15:23:40.718631 2014-08-13 17:38:02.826382 7fe9bae887a0 20 read_log 192834'14209 (0'0) promote 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by osd.371.0:27134 2014-08-11 15:24:04.371951 2014-08-13 17:38:02.826394 7fe9bae887a0 20 read_log 192834'14210 (192834'14209) modify 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.933448.0:6165 2014-08-11 15:24:04.363883 2014-08-13 17:38:02.826406 7fe9bae887a0 20 read_log 192834'14233 (0'0) promote 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by osd.371.0:27178 2014-08-11 15:24:10.446720 2014-08-13 17:38:02.826418 7fe9bae887a0 20 read_log 192834'14234 (192834'14233) modify 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.933448.0:7535 2014-08-11 15:24:10.438788 2014-08-13 17:38:02.826430 7fe9bae887a0 20 read_log 192834'14239 (0'0) promote dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.371.0:27192 2014-08-11 15:24:12.261211 2014-08-13 17:38:02.826442 7fe9bae887a0 20 read_log 192834'14240 (192834'14239) modify dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by client.933448.0:8008 2014-08-11 15:24:12.252875 2014-08-13 17:38:02.826456 7fe9bae887a0 20 read_log 192834'14255 (0'0) promote 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by osd.371.0:27224 2014-08-11 15:24:15.205984 2014-08-13 17:38:02.826468 7fe9bae887a0 20 read_log 192834'14256 (192834'14255) modify 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.933448.0:8731 2014-08-11 15:24:15.198975 2014-08-13 17:38:02.826480 7fe9bae887a0 20 read_log 192834'14257 (0'0) promote d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by osd.371.0:27230 2014-08-11 15:24:15.946565 2014-08-13 17:38:02.826492 7fe9bae887a0 20 read_log 192834'14258 (192834'14257) modify d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.933448.0:8908 2014-08-11 15:24:15.936147 2014-08-13 17:38:02.826507 7fe9bae887a0 20 read_log 192834'14279 (0'0) promote 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by osd.371.0:27284 2014-08-11 15:24:22.602297 2014-08-13 17:38:02.826519 7fe9bae887a0 20 read_log 192834'14280 (192834'14279) modify 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.933448.0:10326 2014-08-11 15:24:22.594703 2014-08-13 17:38:02.826534 7fe9bae887a0 20 read_log 192834'14327 (0'0) promote 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.371.0:27374 2014-08-11 15:24:41.001314 2014-08-13 17:38:02.826546 7fe9bae887a0 20 read_log 192834'14328 (192834'14327) modify 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by client.933448.0:14065 2014-08-11 15:24:40.991189 2014-08-13 17:38:02.826558 7fe9bae887a0 20 read_log 192834'14359 (0'0) promote 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by osd.371.0:27422 2014-08-11 15:24:48.398025 2014-08-13 17:38:02.826570 7fe9bae887a0 20 read_log 192834'14360 (192834'14359) modify 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.933448.0:15096 2014-08-11 15:24:48.390776 2014-08-13 17:38:02.826584 7fe9bae887a0 20 read_log 192834'14369 (0'0) promote ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by osd.371.0:27436 2014-08-11 15:24:50.397593 2014-08-13 17:38:02.826597 7fe9bae887a0 20 read_log 192834'14370 (192834'14369) modify ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.933448.0:15618 2014-08-11 15:24:50.390068 2014-08-13 17:38:02.826617 7fe9bae887a0 20 read_log 192834'14393 (0'0) promote f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by osd.371.0:27478 2014-08-11 15:24:58.470355 2014-08-13 17:38:02.826632 7fe9bae887a0 20 read_log 192834'14394 (192834'14393) modify f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.933448.0:17107 2014-08-11 15:24:58.461362 2014-08-13 17:38:02.826644 7fe9bae887a0 20 read_log 192834'14397 (0'0) promote cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.371.0:27482 2014-08-11 15:24:58.782364 2014-08-13 17:38:02.826656 7fe9bae887a0 20 read_log 192834'14398 (192834'14397) modify cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by client.933448.0:17191 2014-08-11 15:24:58.773499 2014-08-13 17:38:02.826675 7fe9bae887a0 20 read_log 192834'14401 (0'0) promote 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by osd.371.0:27490 2014-08-11 15:24:59.964564 2014-08-13 17:38:02.826687 7fe9bae887a0 20 read_log 192834'14402 (192834'14401) modify 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.933448.0:17506 2014-08-11 15:24:59.956609 2014-08-13 17:38:02.826699 7fe9bae887a0 20 read_log 192834'14407 (0'0) promote a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.371.0:27502 2014-08-11 15:25:04.546574 2014-08-13 17:38:02.826711 7fe9bae887a0 20 read_log 192834'14408 (192834'14407) modify a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by client.933448.0:18272 2014-08-11 15:25:04.538140 2014-08-13 17:38:02.826723 7fe9bae887a0 20 read_log 192834'14437 (0'0) promote 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by osd.371.0:27564 2014-08-11 15:25:20.554251 2014-08-13 17:38:02.826735 7fe9bae887a0 20 read_log 192834'14438 (192834'14437) modify 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.933448.0:20354 2014-08-11 15:25:20.545362 2014-08-13 17:38:02.826747 7fe9bae887a0 20 read_log 192834'14447 (0'0) promote e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by osd.371.0:27578 2014-08-11 15:25:25.469700 2014-08-13 17:38:02.826759 7fe9bae887a0 20 read_log 192834'14448 (192834'14447) modify e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.933448.0:20965 2014-08-11 15:25:25.461251 2014-08-13 17:38:02.826771 7fe9bae887a0 20 read_log 192834'14467 (0'0) promote 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by osd.371.0:27614 2014-08-11 15:25:31.784867 2014-08-13 17:38:02.826783 7fe9bae887a0 20 read_log 192834'14468 (192834'14467) modify 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.933448.0:21759 2014-08-11 15:25:31.776610 2014-08-13 17:38:02.826795 7fe9bae887a0 20 read_log 192834'14474 (192834'14114) modify adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by client.933448.0:22498 2014-08-11 15:25:37.979493 2014-08-13 17:38:02.826807 7fe9bae887a0 20 read_log 192834'14480 (192834'14126) modify fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.933448.0:23596 2014-08-11 15:25:39.536561 2014-08-13 17:38:02.826819 7fe9bae887a0 20 read_log 192834'14522 (192834'14210) modify 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.933448.0:28456 2014-08-11 15:25:46.255252 2014-08-13 17:38:02.826831 7fe9bae887a0 20 read_log 192834'14534 (192834'14234) modify 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.933448.0:29826 2014-08-11 15:25:47.953637 2014-08-13 17:38:02.826843 7fe9bae887a0 20 read_log 192834'14537 (192834'14240) modify dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by client.933448.0:30299 2014-08-11 15:25:48.698855 2014-08-13 17:38:02.826855 7fe9bae887a0 20 read_log 192834'14545 (192834'14256) modify 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.933448.0:31022 2014-08-11 15:25:49.502666 2014-08-13 17:38:02.826877 7fe9bae887a0 20 read_log 192834'14546 (192834'14258) modify d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.933448.0:31199 2014-08-11 15:25:49.785049 2014-08-13 17:38:02.826893 7fe9bae887a0 20 read_log 192834'14557 (192834'14280) modify 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.933448.0:32617 2014-08-11 15:25:52.107426 2014-08-13 17:38:02.826905 7fe9bae887a0 20 read_log 192834'14581 (192834'14328) modify 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by client.933448.0:36356 2014-08-11 15:25:57.605793 2014-08-13 17:38:02.826917 7fe9bae887a0 20 read_log 192834'14597 (192834'14360) modify 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.933448.0:37387 2014-08-11 15:25:59.496529 2014-08-13 17:38:02.826931 7fe9bae887a0 20 read_log 192834'14602 (192834'14370) modify ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.933448.0:37909 2014-08-11 15:26:00.261738 2014-08-13 17:38:02.826943 7fe9bae887a0 20 read_log 192834'14614 (192834'14394) modify f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.933448.0:39398 2014-08-11 15:26:01.845574 2014-08-13 17:38:02.826955 7fe9bae887a0 20 read_log 192834'14616 (192834'14398) modify cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by client.933448.0:39482 2014-08-11 15:26:01.914512 2014-08-13 17:38:02.826967 7fe9bae887a0 20 read_log 192834'14618 (192834'14402) modify 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.933448.0:39797 2014-08-11 15:26:02.089110 2014-08-13 17:38:02.826979 7fe9bae887a0 20 read_log 192834'14621 (192834'14408) modify a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by client.933448.0:40563 2014-08-11 15:26:02.653475 2014-08-13 17:38:02.826994 7fe9bae887a0 20 read_log 192834'14636 (192834'14438) modify 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.933448.0:42645 2014-08-11 15:26:03.341725 2014-08-13 17:38:02.827006 7fe9bae887a0 20 read_log 192834'14641 (192834'14448) modify e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.933448.0:43256 2014-08-11 15:26:03.552578 2014-08-13 17:38:02.827018 7fe9bae887a0 20 read_log 192834'14651 (192834'14468) modify 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.933448.0:44050 2014-08-11 15:26:03.801023 2014-08-13 17:38:02.827032 7fe9bae887a0 20 read_log 192834'14661 (0'0) promote 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by osd.371.0:27983 2014-08-11 15:26:16.512241 2014-08-13 17:38:02.827045 7fe9bae887a0 20 read_log 192834'14662 (192834'14661) modify 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.1883634.0:553 2014-08-11 15:26:16.504642 2014-08-13 17:38:02.827059 7fe9bae887a0 20 read_log 192834'14691 (0'0) promote b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.371.0:28029 2014-08-11 15:26:21.917766 2014-08-13 17:38:02.827071 7fe9bae887a0 20 read_log 192834'14692 (192834'14691) modify b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by client.1883634.0:2015 2014-08-11 15:26:21.910447 2014-08-13 17:38:02.827086 7fe9bae887a0 20 read_log 192834'14693 (0'0) promote 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by osd.371.0:28031 2014-08-11 15:26:21.973118 2014-08-13 17:38:02.827098 7fe9bae887a0 20 read_log 192834'14694 (192834'14693) modify 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.1883634.0:2030 2014-08-11 15:26:21.964709 2014-08-13 17:38:02.827112 7fe9bae887a0 20 read_log 192834'14713 (0'0) promote 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by osd.371.0:28077 2014-08-11 15:26:28.877830 2014-08-13 17:38:02.827124 7fe9bae887a0 20 read_log 192834'14714 (192834'14713) modify 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.1883634.0:3967 2014-08-11 15:26:28.870023 2014-08-13 17:38:02.827143 7fe9bae887a0 20 read_log 192834'14719 (0'0) promote fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by osd.371.0:28085 2014-08-11 15:26:30.336520 2014-08-13 17:38:02.827162 7fe9bae887a0 20 read_log 192834'14720 (192834'14719) modify fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.1883634.0:4378 2014-08-11 15:26:30.328730 2014-08-13 17:38:02.827176 7fe9bae887a0 20 read_log 192834'14723 (0'0) promote daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by osd.371.0:28089 2014-08-11 15:26:31.568873 2014-08-13 17:38:02.827189 7fe9bae887a0 20 read_log 192834'14724 (192834'14723) modify daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.1883634.0:4696 2014-08-11 15:26:31.559954 2014-08-13 17:38:02.827201 7fe9bae887a0 20 read_log 192834'14731 (0'0) promote 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by osd.371.0:28101 2014-08-11 15:26:32.687351 2014-08-13 17:38:02.827213 7fe9bae887a0 20 read_log 192834'14732 (192834'14731) modify 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.1883634.0:4950 2014-08-11 15:26:32.679765 2014-08-13 17:38:02.827225 7fe9bae887a0 20 read_log 192834'14737 (0'0) promote f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by osd.371.0:28111 2014-08-11 15:26:34.006556 2014-08-13 17:38:02.827237 7fe9bae887a0 20 read_log 192834'14738 (192834'14737) modify f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.1883634.0:5325 2014-08-11 15:26:33.998672 2014-08-13 17:38:02.827249 7fe9bae887a0 20 read_log 192834'14739 (0'0) promote f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.371.0:28117 2014-08-11 15:26:34.268277 2014-08-13 17:38:02.827260 7fe9bae887a0 20 read_log 192834'14740 (192834'14739) modify f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by client.1883634.0:5399 2014-08-11 15:26:34.259521 2014-08-13 17:38:02.827275 7fe9bae887a0 20 read_log 192834'14753 (0'0) promote 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.371.0:28141 2014-08-11 15:26:36.424734 2014-08-13 17:38:02.827287 7fe9bae887a0 20 read_log 192834'14754 (192834'14753) modify 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by client.1883634.0:6004 2014-08-11 15:26:36.416947 2014-08-13 17:38:02.827299 7fe9bae887a0 20 read_log 192834'14757 (0'0) promote 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by osd.371.0:28149 2014-08-11 15:26:36.775215 2014-08-13 17:38:02.827313 7fe9bae887a0 20 read_log 192834'14758 (192834'14757) modify 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.1883634.0:6102 2014-08-11 15:26:36.767279 2014-08-13 17:38:02.827325 7fe9bae887a0 20 read_log 192834'14775 (0'0) promote 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.371.0:28191 2014-08-11 15:26:41.754236 2014-08-13 17:38:02.827339 7fe9bae887a0 20 read_log 192834'14776 (192834'14775) modify 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by client.1883634.0:7338 2014-08-11 15:26:41.745566 2014-08-13 17:38:02.827351 7fe9bae887a0 20 read_log 192834'14779 (0'0) promote 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.371.0:28209 2014-08-11 15:26:44.748599 2014-08-13 17:38:02.827364 7fe9bae887a0 20 read_log 192834'14780 (192834'14779) modify 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by client.1883634.0:7985 2014-08-11 15:26:44.552450 2014-08-13 17:38:02.827383 7fe9bae887a0 20 read_log 192834'14781 (0'0) promote e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by osd.371.0:28213 2014-08-11 15:26:44.761462 2014-08-13 17:38:02.827396 7fe9bae887a0 20 read_log 192834'14782 (192834'14781) modify e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.1883634.0:8042 2014-08-11 15:26:44.751319 2014-08-13 17:38:02.827415 7fe9bae887a0 20 read_log 192834'14785 (0'0) promote 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by osd.371.0:28227 2014-08-11 15:26:47.108199 2014-08-13 17:38:02.827429 7fe9bae887a0 20 read_log 192834'14786 (192834'14785) modify 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.1883634.0:8613 2014-08-11 15:26:47.100531 2014-08-13 17:38:02.827442 7fe9bae887a0 20 read_log 192834'14789 (0'0) promote 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by osd.371.0:28231 2014-08-11 15:26:47.834976 2014-08-13 17:38:02.827454 7fe9bae887a0 20 read_log 192834'14790 (192834'14789) modify 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.1883634.0:8774 2014-08-11 15:26:47.826458 2014-08-13 17:38:02.827466 7fe9bae887a0 20 read_log 192834'14795 (0'0) promote 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by osd.371.0:28247 2014-08-11 15:26:50.264600 2014-08-13 17:38:02.827497 7fe9bae887a0 20 read_log 192834'14796 (192834'14795) modify 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.1883634.0:9302 2014-08-11 15:26:50.256237 2014-08-13 17:38:02.827512 7fe9bae887a0 20 read_log 192834'14799 (0'0) promote b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by osd.371.0:28253 2014-08-11 15:26:50.839723 2014-08-13 17:38:02.827524 7fe9bae887a0 20 read_log 192834'14800 (192834'14799) modify b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.1883634.0:9431 2014-08-11 15:26:50.831895 2014-08-13 17:38:02.827536 7fe9bae887a0 20 read_log 192834'14807 (0'0) promote 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by osd.371.0:28267 2014-08-11 15:26:53.635092 2014-08-13 17:38:02.827548 7fe9bae887a0 20 read_log 192834'14808 (192834'14807) modify 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.1883634.0:10084 2014-08-11 15:26:53.625139 2014-08-13 17:38:02.827560 7fe9bae887a0 20 read_log 192834'14819 (0'0) promote bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by osd.371.0:28283 2014-08-11 15:26:58.891794 2014-08-13 17:38:02.827572 7fe9bae887a0 20 read_log 192834'14820 (192834'14819) modify bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.1883634.0:11221 2014-08-11 15:26:58.882409 2014-08-13 17:38:02.827584 7fe9bae887a0 20 read_log 192834'14835 (0'0) promote d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by osd.371.0:28333 2014-08-11 15:27:10.056405 2014-08-13 17:38:02.827596 7fe9bae887a0 20 read_log 192834'14836 (192834'14835) modify d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.1883634.0:13550 2014-08-11 15:27:10.046300 2014-08-13 17:38:02.827609 7fe9bae887a0 20 read_log 192834'14847 (0'0) promote d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by osd.371.0:28351 2014-08-11 15:27:12.004729 2014-08-13 17:38:02.827621 7fe9bae887a0 20 read_log 192834'14848 (192834'14847) modify d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.1883634.0:14118 2014-08-11 15:27:11.996057 2014-08-13 17:38:02.827636 7fe9bae887a0 20 read_log 192834'14853 (0'0) promote baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.371.0:28367 2014-08-11 15:27:15.160422 2014-08-13 17:38:02.827649 7fe9bae887a0 20 read_log 192834'14854 (192834'14853) modify baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by client.1883634.0:14732 2014-08-11 15:27:15.151222 2014-08-13 17:38:02.827661 7fe9bae887a0 20 read_log 192834'14865 (0'0) promote ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by osd.371.0:28389 2014-08-11 15:27:19.239125 2014-08-13 17:38:02.827673 7fe9bae887a0 20 read_log 192834'14866 (192834'14865) modify ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.1883634.0:15356 2014-08-11 15:27:19.231026 2014-08-13 17:38:02.827694 7fe9bae887a0 20 read_log 192834'14883 (0'0) promote 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by osd.371.0:28417 2014-08-11 15:27:27.404182 2014-08-13 17:38:02.827709 7fe9bae887a0 20 read_log 192834'14884 (192834'14883) modify 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.1883634.0:16406 2014-08-11 15:27:27.394412 2014-08-13 17:38:02.827722 7fe9bae887a0 20 read_log 192834'14899 (0'0) promote e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by osd.371.0:28439 2014-08-11 15:27:30.235251 2014-08-13 17:38:02.827733 7fe9bae887a0 20 read_log 192834'14900 (192834'14899) modify e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.1883634.0:16985 2014-08-11 15:27:30.225156 2014-08-13 17:38:02.827746 7fe9bae887a0 20 read_log 192834'14915 (0'0) promote bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.371.0:28463 2014-08-11 15:27:39.441241 2014-08-13 17:38:02.827757 7fe9bae887a0 20 read_log 192834'14916 (192834'14915) modify bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by client.1883634.0:17872 2014-08-11 15:27:39.431291 2014-08-13 17:38:02.827769 7fe9bae887a0 20 read_log 192834'14945 (0'0) promote 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by osd.371.0:28521 2014-08-11 15:27:53.418387 2014-08-13 17:38:02.827781 7fe9bae887a0 20 read_log 192834'14946 (192834'14945) modify 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.1883634.0:19563 2014-08-11 15:27:53.303507 2014-08-13 17:38:02.827793 7fe9bae887a0 20 read_log 192834'14981 (0'0) promote 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by osd.371.0:28597 2014-08-11 15:28:11.737448 2014-08-13 17:38:02.827805 7fe9bae887a0 20 read_log 192834'14982 (192834'14981) modify 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.1883634.0:22919 2014-08-11 15:28:11.726077 2014-08-13 17:38:02.827817 7fe9bae887a0 20 read_log 192834'14995 (192834'14662) modify 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.1883634.0:23877 2014-08-11 15:28:18.037685 2014-08-13 17:38:02.827829 7fe9bae887a0 20 read_log 192834'15010 (192834'14692) modify b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by client.1883634.0:25339 2014-08-11 15:28:20.653685 2014-08-13 17:38:02.827841 7fe9bae887a0 20 read_log 192834'15011 (192834'14694) modify 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.1883634.0:25354 2014-08-11 15:28:20.792278 2014-08-13 17:38:02.827853 7fe9bae887a0 20 read_log 192834'15021 (192834'14714) modify 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.1883634.0:27291 2014-08-11 15:28:24.350807 2014-08-13 17:38:02.827865 7fe9bae887a0 20 read_log 192834'15024 (192834'14720) modify fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.1883634.0:27702 2014-08-11 15:28:24.842691 2014-08-13 17:38:02.827877 7fe9bae887a0 20 read_log 192834'15026 (192834'14724) modify daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.1883634.0:28020 2014-08-11 15:28:25.275118 2014-08-13 17:38:02.827889 7fe9bae887a0 20 read_log 192834'15030 (192834'14732) modify 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.1883634.0:28274 2014-08-11 15:28:25.592652 2014-08-13 17:38:02.827901 7fe9bae887a0 20 read_log 192834'15033 (192834'14738) modify f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.1883634.0:28649 2014-08-11 15:28:26.092327 2014-08-13 17:38:02.827913 7fe9bae887a0 20 read_log 192834'15034 (192834'14740) modify f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by client.1883634.0:28723 2014-08-11 15:28:26.189028 2014-08-13 17:38:02.827925 7fe9bae887a0 20 read_log 192834'15041 (192834'14754) modify 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by client.1883634.0:29328 2014-08-11 15:28:26.985587 2014-08-13 17:38:02.827961 7fe9bae887a0 20 read_log 192834'15043 (192834'14758) modify 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.1883634.0:29426 2014-08-11 15:28:27.232894 2014-08-13 17:38:02.827977 7fe9bae887a0 20 read_log 192834'15052 (192834'14776) modify 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by client.1883634.0:30662 2014-08-11 15:28:29.052680 2014-08-13 17:38:02.827989 7fe9bae887a0 20 read_log 192834'15054 (192834'14780) modify 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by client.1883634.0:31309 2014-08-11 15:28:30.259843 2014-08-13 17:38:02.828001 7fe9bae887a0 20 read_log 192834'15055 (192834'14782) modify e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.1883634.0:31366 2014-08-11 15:28:30.377789 2014-08-13 17:38:02.828013 7fe9bae887a0 20 read_log 192834'15057 (192834'14786) modify 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.1883634.0:31937 2014-08-11 15:28:31.151168 2014-08-13 17:38:02.828026 7fe9bae887a0 20 read_log 192834'15059 (192834'14790) modify 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.1883634.0:32098 2014-08-11 15:28:31.349478 2014-08-13 17:38:02.828038 7fe9bae887a0 20 read_log 192834'15062 (192834'14796) modify 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.1883634.0:32626 2014-08-11 15:28:32.082331 2014-08-13 17:38:02.828051 7fe9bae887a0 20 read_log 192834'15064 (192834'14800) modify b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.1883634.0:32755 2014-08-11 15:28:32.271261 2014-08-13 17:38:02.828063 7fe9bae887a0 20 read_log 192834'15068 (192834'14808) modify 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.1883634.0:33408 2014-08-11 15:28:33.131967 2014-08-13 17:38:02.828075 7fe9bae887a0 20 read_log 192834'15074 (192834'14820) modify bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.1883634.0:34545 2014-08-11 15:28:34.982526 2014-08-13 17:38:02.828087 7fe9bae887a0 20 read_log 192834'15084 (192834'14836) modify d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.1883634.0:36874 2014-08-11 15:28:38.521178 2014-08-13 17:38:02.828100 7fe9bae887a0 20 read_log 192834'15090 (192834'14848) modify d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.1883634.0:37442 2014-08-11 15:28:39.361362 2014-08-13 17:38:02.828112 7fe9bae887a0 20 read_log 192834'15093 (192834'14854) modify baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by client.1883634.0:38056 2014-08-11 15:28:40.164630 2014-08-13 17:38:02.828124 7fe9bae887a0 20 read_log 192834'15099 (192834'14866) modify ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.1883634.0:38680 2014-08-11 15:28:41.019987 2014-08-13 17:38:02.828136 7fe9bae887a0 20 read_log 192834'15108 (192834'14884) modify 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.1883634.0:39730 2014-08-11 15:28:42.488426 2014-08-13 17:38:02.828148 7fe9bae887a0 20 read_log 192834'15116 (192834'14900) modify e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.1883634.0:40309 2014-08-11 15:28:43.506503 2014-08-13 17:38:02.828167 7fe9bae887a0 20 read_log 192834'15124 (192834'14916) modify bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by client.1883634.0:41196 2014-08-11 15:28:44.740042 2014-08-13 17:38:02.828180 7fe9bae887a0 20 read_log 192834'15139 (192834'14946) modify 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.1883634.0:42887 2014-08-11 15:28:46.390930 2014-08-13 17:38:02.828192 7fe9bae887a0 20 read_log 192834'15157 (192834'14982) modify 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.1883634.0:46243 2014-08-11 15:28:47.533949 2014-08-13 17:38:02.828205 7fe9bae887a0 20 read_log 192888'15162 (192888'15162) modify cd/hit_set_24.cd_archive_2014-08-11 16:02:34.118089_2014-08-11 16:44:57.980567/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.980782 2014-08-13 17:38:02.828225 7fe9bae887a0 20 read_log 192888'15163 (0'0) promote 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by osd.6.0:1291 2014-08-11 16:44:57.984730 2014-08-13 17:38:02.828237 7fe9bae887a0 20 read_log 192888'15164 (192888'15163) modify 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.1315259.0:710 2014-08-11 16:44:57.976007 2014-08-13 17:38:02.828250 7fe9bae887a0 20 read_log 192888'15165 (0'0) promote a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by osd.6.0:1303 2014-08-11 16:45:02.811331 2014-08-13 17:38:02.828262 7fe9bae887a0 20 read_log 192888'15166 (192888'15165) modify a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.1315259.0:1983 2014-08-11 16:45:02.804616 2014-08-13 17:38:02.828274 7fe9bae887a0 20 read_log 192888'15167 (0'0) promote 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:1307 2014-08-11 16:45:04.887503 2014-08-13 17:38:02.828286 7fe9bae887a0 20 read_log 192888'15168 (192888'15167) modify 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by client.1315259.0:2546 2014-08-11 16:45:04.881128 2014-08-13 17:38:02.828298 7fe9bae887a0 20 read_log 192888'15169 (0'0) promote dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by osd.6.0:1311 2014-08-11 16:45:05.043607 2014-08-13 17:38:02.828310 7fe9bae887a0 20 read_log 192888'15170 (192888'15169) modify dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.1315259.0:2587 2014-08-11 16:45:05.037134 2014-08-13 17:38:02.828322 7fe9bae887a0 20 read_log 192888'15175 (0'0) promote a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by osd.6.0:1321 2014-08-11 16:45:12.427412 2014-08-13 17:38:02.828334 7fe9bae887a0 20 read_log 192888'15176 (192888'15175) modify a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.1315259.0:4540 2014-08-11 16:45:12.419249 2014-08-13 17:38:02.828346 7fe9bae887a0 20 read_log 192888'15177 (0'0) promote d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by osd.6.0:1323 2014-08-11 16:45:14.564319 2014-08-13 17:38:02.828358 7fe9bae887a0 20 read_log 192888'15178 (192888'15177) modify d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.1315259.0:5112 2014-08-11 16:45:14.556390 2014-08-13 17:38:02.828370 7fe9bae887a0 20 read_log 192888'15191 (0'0) promote e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by osd.6.0:1367 2014-08-11 16:45:39.527180 2014-08-13 17:38:02.828382 7fe9bae887a0 20 read_log 192888'15192 (192888'15191) modify e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.1315259.0:11739 2014-08-11 16:45:39.519574 2014-08-13 17:38:02.828395 7fe9bae887a0 20 read_log 192888'15203 (0'0) promote 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by osd.6.0:1399 2014-08-11 16:45:53.171214 2014-08-13 17:38:02.828406 7fe9bae887a0 20 read_log 192888'15204 (192888'15203) modify 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.1315259.0:15156 2014-08-11 16:45:53.163519 2014-08-13 17:38:02.828418 7fe9bae887a0 20 read_log 192888'15213 (0'0) promote f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by osd.6.0:1409 2014-08-11 16:45:59.015641 2014-08-13 17:38:02.828430 7fe9bae887a0 20 read_log 192888'15214 (192888'15213) modify f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.1315259.0:16659 2014-08-11 16:45:59.008496 2014-08-13 17:38:02.828442 7fe9bae887a0 20 read_log 192888'15221 (0'0) promote 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:1419 2014-08-11 16:46:07.962131 2014-08-13 17:38:02.828454 7fe9bae887a0 20 read_log 192888'15222 (192888'15221) modify 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by client.1315259.0:18945 2014-08-11 16:46:07.954327 2014-08-13 17:38:02.828476 7fe9bae887a0 20 read_log 192888'15227 (0'0) promote a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by osd.6.0:1425 2014-08-11 16:46:11.558924 2014-08-13 17:38:02.828489 7fe9bae887a0 20 read_log 192888'15228 (192888'15227) modify a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.1315259.0:19846 2014-08-11 16:46:11.551765 2014-08-13 17:38:02.828501 7fe9bae887a0 20 read_log 192888'15235 (0'0) promote 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by osd.6.0:1433 2014-08-11 16:46:18.526363 2014-08-13 17:38:02.828513 7fe9bae887a0 20 read_log 192888'15236 (192888'15235) modify 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.1315259.0:21696 2014-08-11 16:46:18.518513 2014-08-13 17:38:02.828525 7fe9bae887a0 20 read_log 192888'15239 (0'0) promote e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by osd.6.0:1437 2014-08-11 16:46:19.167623 2014-08-13 17:38:02.828537 7fe9bae887a0 20 read_log 192888'15240 (192888'15239) modify e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.1315259.0:21869 2014-08-11 16:46:19.160102 2014-08-13 17:38:02.828552 7fe9bae887a0 20 read_log 192888'15249 (0'0) promote 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:1457 2014-08-11 16:46:34.133558 2014-08-13 17:38:02.828564 7fe9bae887a0 20 read_log 192888'15250 (192888'15249) modify 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by client.1315259.0:25633 2014-08-11 16:46:34.124109 2014-08-13 17:38:02.828576 7fe9bae887a0 20 read_log 192888'15251 (0'0) promote 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:1465 2014-08-11 16:46:39.111666 2014-08-13 17:38:02.828588 7fe9bae887a0 20 read_log 192888'15252 (192888'15251) modify 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by client.1315259.0:26811 2014-08-11 16:46:39.103959 2014-08-13 17:38:02.828600 7fe9bae887a0 20 read_log 192888'15253 (0'0) promote 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by osd.6.0:1469 2014-08-11 16:46:42.295977 2014-08-13 17:38:02.828612 7fe9bae887a0 20 read_log 192888'15254 (192888'15253) modify 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.1315259.0:27573 2014-08-11 16:46:42.288439 2014-08-13 17:38:02.828624 7fe9bae887a0 20 read_log 192888'15255 (0'0) promote 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by osd.6.0:1471 2014-08-11 16:46:43.513034 2014-08-13 17:38:02.828636 7fe9bae887a0 20 read_log 192888'15256 (192888'15255) modify 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.1315259.0:27885 2014-08-11 16:46:43.505253 2014-08-13 17:38:02.828649 7fe9bae887a0 20 read_log 192888'15257 (0'0) promote d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by osd.6.0:1475 2014-08-11 16:46:46.348974 2014-08-13 17:38:02.828661 7fe9bae887a0 20 read_log 192888'15258 (192888'15257) modify d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.1315259.0:28561 2014-08-11 16:46:46.342019 2014-08-13 17:38:02.828674 7fe9bae887a0 20 read_log 192888'15259 (0'0) promote ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by osd.6.0:1485 2014-08-11 16:46:50.146443 2014-08-13 17:38:02.828686 7fe9bae887a0 20 read_log 192888'15260 (192888'15259) modify ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.1315259.0:29500 2014-08-11 16:46:50.140126 2014-08-13 17:38:02.828699 7fe9bae887a0 20 read_log 192888'15261 (0'0) promote 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by osd.6.0:1487 2014-08-11 16:46:50.236077 2014-08-13 17:38:02.828711 7fe9bae887a0 20 read_log 192888'15262 (192888'15261) modify 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.1315259.0:29524 2014-08-11 16:46:50.229242 2014-08-13 17:38:02.828730 7fe9bae887a0 20 read_log 192888'15263 (0'0) promote 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by osd.6.0:1489 2014-08-11 16:46:50.462485 2014-08-13 17:38:02.828743 7fe9bae887a0 20 read_log 192888'15264 (192888'15263) modify 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.1315259.0:29583 2014-08-11 16:46:50.454838 2014-08-13 17:38:02.828755 7fe9bae887a0 20 read_log 192888'15267 (0'0) promote 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:1497 2014-08-11 16:46:54.526811 2014-08-13 17:38:02.828767 7fe9bae887a0 20 read_log 192888'15268 (192888'15267) modify 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by client.1315259.0:30640 2014-08-11 16:46:54.519778 2014-08-13 17:38:02.828779 7fe9bae887a0 20 read_log 192888'15269 (192888'15164) modify 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.1315259.0:31437 2014-08-11 16:46:56.095665 2014-08-13 17:38:02.828791 7fe9bae887a0 20 read_log 192888'15270 (192888'15166) modify a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.1315259.0:32710 2014-08-11 16:46:56.449283 2014-08-13 17:38:02.828804 7fe9bae887a0 20 read_log 192888'15271 (192888'15168) modify 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by client.1315259.0:33273 2014-08-11 16:46:56.594069 2014-08-13 17:38:02.828818 7fe9bae887a0 20 read_log 192888'15272 (192888'15170) modify dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.1315259.0:33314 2014-08-11 16:46:56.605325 2014-08-13 17:38:02.828830 7fe9bae887a0 20 read_log 192888'15275 (192888'15176) modify a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.1315259.0:35267 2014-08-11 16:46:57.118314 2014-08-13 17:38:02.828842 7fe9bae887a0 20 read_log 192888'15276 (192888'15178) modify d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.1315259.0:35839 2014-08-11 16:46:57.258935 2014-08-13 17:38:02.828854 7fe9bae887a0 20 read_log 192888'15283 (192888'15192) modify e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.1315259.0:42466 2014-08-11 16:46:59.020163 2014-08-13 17:38:02.828868 7fe9bae887a0 20 read_log 192888'15289 (192888'15204) modify 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.1315259.0:45883 2014-08-11 16:46:59.999416 2014-08-13 17:38:02.828881 7fe9bae887a0 20 read_log 192888'15294 (192888'15214) modify f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.1315259.0:47386 2014-08-11 16:47:00.441597 2014-08-13 17:38:02.828893 7fe9bae887a0 20 read_log 192888'15298 (192888'15222) modify 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by client.1315259.0:49672 2014-08-11 16:47:01.139619 2014-08-13 17:38:02.828905 7fe9bae887a0 20 read_log 192888'15301 (192888'15228) modify a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.1315259.0:50573 2014-08-11 16:47:01.412235 2014-08-13 17:38:02.828917 7fe9bae887a0 20 read_log 192888'15305 (192888'15236) modify 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.1315259.0:52423 2014-08-11 16:47:01.969318 2014-08-13 17:38:02.828929 7fe9bae887a0 20 read_log 192888'15307 (192888'15240) modify e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.1315259.0:52596 2014-08-11 16:47:02.017567 2014-08-13 17:38:02.828941 7fe9bae887a0 20 read_log 192888'15312 (192888'15250) modify 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by client.1315259.0:56360 2014-08-11 16:47:03.181680 2014-08-13 17:38:02.828953 7fe9bae887a0 20 read_log 192888'15313 (192888'15252) modify 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by client.1315259.0:57538 2014-08-11 16:47:03.542420 2014-08-13 17:38:02.828964 7fe9bae887a0 20 read_log 192888'15314 (192888'15254) modify 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.1315259.0:58300 2014-08-11 16:47:03.761194 2014-08-13 17:38:02.828984 7fe9bae887a0 20 read_log 192888'15315 (192888'15256) modify 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.1315259.0:58612 2014-08-11 16:47:03.853339 2014-08-13 17:38:02.828996 7fe9bae887a0 20 read_log 192888'15316 (192888'15258) modify d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.1315259.0:59288 2014-08-11 16:47:04.066933 2014-08-13 17:38:02.829008 7fe9bae887a0 20 read_log 192888'15317 (192888'15260) modify ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.1315259.0:60227 2014-08-11 16:47:04.386566 2014-08-13 17:38:02.829020 7fe9bae887a0 20 read_log 192888'15318 (192888'15262) modify 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.1315259.0:60251 2014-08-11 16:47:04.392415 2014-08-13 17:38:02.829032 7fe9bae887a0 20 read_log 192888'15319 (192888'15264) modify 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.1315259.0:60310 2014-08-11 16:47:04.408163 2014-08-13 17:38:02.829044 7fe9bae887a0 20 read_log 192888'15321 (192888'15268) modify 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by client.1315259.0:61367 2014-08-11 16:47:04.717233 2014-08-13 17:38:02.829056 7fe9bae887a0 20 read_log 192888'15322 (192888'15322) modify cd/hit_set_24.cd_archive_2014-08-11 16:44:57.980787_2014-08-11 17:05:16.882120/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.882347 2014-08-13 17:38:02.829068 7fe9bae887a0 20 read_log 192888'15323 (192888'15162) delete cd/hit_set_24.cd_archive_2014-08-11 16:02:34.118089_2014-08-11 16:44:57.980567/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.882347 2014-08-13 17:38:02.829080 7fe9bae887a0 20 read_log 192888'15324 (0'0) promote d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by osd.6.0:1614 2014-08-11 17:05:16.885330 2014-08-13 17:38:02.829093 7fe9bae887a0 20 read_log 192888'15325 (192888'15324) modify d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.942868.0:447 2014-08-11 17:05:16.876716 2014-08-13 17:38:02.829105 7fe9bae887a0 20 read_log 192888'15334 (0'0) promote 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by osd.6.0:1634 2014-08-11 17:05:31.810239 2014-08-13 17:38:02.829117 7fe9bae887a0 20 read_log 192888'15335 (192888'15334) modify 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.942868.0:4292 2014-08-11 17:05:31.802605 2014-08-13 17:38:02.829129 7fe9bae887a0 20 read_log 192888'15336 (0'0) promote 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by osd.6.0:1638 2014-08-11 17:05:33.354405 2014-08-13 17:38:02.829141 7fe9bae887a0 20 read_log 192888'15337 (192888'15336) modify 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.942868.0:4695 2014-08-11 17:05:33.347048 2014-08-13 17:38:02.829157 7fe9bae887a0 20 read_log 192888'15342 (0'0) promote 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by osd.6.0:1656 2014-08-11 17:05:41.667635 2014-08-13 17:38:02.829170 7fe9bae887a0 20 read_log 192888'15343 (192888'15342) modify 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.942868.0:6917 2014-08-11 17:05:41.656361 2014-08-13 17:38:02.829182 7fe9bae887a0 20 read_log 192888'15346 (0'0) promote 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:1666 2014-08-11 17:05:45.873070 2014-08-13 17:38:02.829194 7fe9bae887a0 20 read_log 192888'15347 (192888'15346) modify 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by client.942868.0:8013 2014-08-11 17:05:45.865075 2014-08-13 17:38:02.829206 7fe9bae887a0 20 read_log 192888'15354 (0'0) promote 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by osd.6.0:1674 2014-08-11 17:05:49.858991 2014-08-13 17:38:02.829228 7fe9bae887a0 20 read_log 192888'15355 (192888'15354) modify 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.942868.0:9094 2014-08-11 17:05:49.853198 2014-08-13 17:38:02.829240 7fe9bae887a0 20 read_log 192888'15358 (0'0) promote bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by osd.6.0:1682 2014-08-11 17:05:51.572770 2014-08-13 17:38:02.829252 7fe9bae887a0 20 read_log 192888'15359 (192888'15358) modify bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.942868.0:9549 2014-08-11 17:05:51.565029 2014-08-13 17:38:02.829264 7fe9bae887a0 20 read_log 192888'15360 (0'0) promote 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by osd.6.0:1684 2014-08-11 17:05:51.854970 2014-08-13 17:38:02.829276 7fe9bae887a0 20 read_log 192888'15361 (192888'15360) modify 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.942868.0:9627 2014-08-11 17:05:51.847632 2014-08-13 17:38:02.829288 7fe9bae887a0 20 read_log 192888'15364 (0'0) promote 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by osd.6.0:1694 2014-08-11 17:05:56.202866 2014-08-13 17:38:02.829300 7fe9bae887a0 20 read_log 192888'15365 (192888'15364) modify 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.942868.0:10834 2014-08-11 17:05:56.195643 2014-08-13 17:38:02.829312 7fe9bae887a0 20 read_log 192888'15366 (0'0) promote 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by osd.6.0:1698 2014-08-11 17:05:59.325041 2014-08-13 17:38:02.829325 7fe9bae887a0 20 read_log 192888'15367 (192888'15366) modify 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.942868.0:11683 2014-08-11 17:05:59.318291 2014-08-13 17:38:02.829337 7fe9bae887a0 20 read_log 192888'15368 (0'0) promote 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by osd.6.0:1700 2014-08-11 17:06:00.128563 2014-08-13 17:38:02.829349 7fe9bae887a0 20 read_log 192888'15369 (192888'15368) modify 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.942868.0:11893 2014-08-11 17:06:00.121250 2014-08-13 17:38:02.829361 7fe9bae887a0 20 read_log 192888'15374 (0'0) promote f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:1708 2014-08-11 17:06:03.808380 2014-08-13 17:38:02.829373 7fe9bae887a0 20 read_log 192888'15375 (192888'15374) modify f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by client.942868.0:12895 2014-08-11 17:06:03.801492 2014-08-13 17:38:02.829385 7fe9bae887a0 20 read_log 192888'15376 (0'0) promote da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by osd.6.0:1710 2014-08-11 17:06:04.705753 2014-08-13 17:38:02.829397 7fe9bae887a0 20 read_log 192888'15377 (192888'15376) modify da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.942868.0:13144 2014-08-11 17:06:04.698517 2014-08-13 17:38:02.829410 7fe9bae887a0 20 read_log 192888'15378 (0'0) promote 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by osd.6.0:1724 2014-08-11 17:06:12.130556 2014-08-13 17:38:02.829422 7fe9bae887a0 20 read_log 192888'15379 (192888'15378) modify 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.942868.0:14977 2014-08-11 17:06:12.123719 2014-08-13 17:38:02.829434 7fe9bae887a0 20 read_log 192888'15380 (0'0) promote 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by osd.6.0:1726 2014-08-11 17:06:13.246123 2014-08-13 17:38:02.829446 7fe9bae887a0 20 read_log 192888'15381 (192888'15380) modify 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.942868.0:15262 2014-08-11 17:06:13.238221 2014-08-13 17:38:02.829458 7fe9bae887a0 20 read_log 192888'15382 (0'0) promote 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by osd.6.0:1736 2014-08-11 17:06:22.056339 2014-08-13 17:38:02.829479 7fe9bae887a0 20 read_log 192888'15383 (192888'15382) modify 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.942868.0:17528 2014-08-11 17:06:22.049245 2014-08-13 17:38:02.829492 7fe9bae887a0 20 read_log 192888'15384 (0'0) promote 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:1738 2014-08-11 17:06:25.821010 2014-08-13 17:38:02.829504 7fe9bae887a0 20 read_log 192888'15385 (192888'15384) modify 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by client.942868.0:18526 2014-08-11 17:06:25.813580 2014-08-13 17:38:02.829516 7fe9bae887a0 20 read_log 192888'15388 (0'0) promote 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:1742 2014-08-11 17:06:27.613351 2014-08-13 17:38:02.829528 7fe9bae887a0 20 read_log 192888'15389 (192888'15388) modify 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by client.942868.0:19003 2014-08-11 17:06:27.604916 2014-08-13 17:38:02.829540 7fe9bae887a0 20 read_log 192888'15390 (0'0) promote 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by osd.6.0:1744 2014-08-11 17:06:30.695742 2014-08-13 17:38:02.829555 7fe9bae887a0 20 read_log 192888'15391 (192888'15390) modify 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.942868.0:19824 2014-08-11 17:06:30.688216 2014-08-13 17:38:02.829567 7fe9bae887a0 20 read_log 192888'15402 (0'0) promote e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by osd.6.0:1768 2014-08-11 17:06:38.903230 2014-08-13 17:38:02.829579 7fe9bae887a0 20 read_log 192888'15403 (192888'15402) modify e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.942868.0:21980 2014-08-11 17:06:38.896300 2014-08-13 17:38:02.829592 7fe9bae887a0 20 read_log 192888'15404 (0'0) promote d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by osd.6.0:1770 2014-08-11 17:06:39.426853 2014-08-13 17:38:02.829604 7fe9bae887a0 20 read_log 192888'15405 (192888'15404) modify d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.942868.0:22123 2014-08-11 17:06:39.419437 2014-08-13 17:38:02.829616 7fe9bae887a0 20 read_log 192888'15408 (0'0) promote ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by osd.6.0:1776 2014-08-11 17:06:42.211245 2014-08-13 17:38:02.829628 7fe9bae887a0 20 read_log 192888'15409 (192888'15408) modify ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.942868.0:22825 2014-08-11 17:06:42.203981 2014-08-13 17:38:02.829640 7fe9bae887a0 20 read_log 192888'15412 (0'0) promote ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:1780 2014-08-11 17:06:43.580701 2014-08-13 17:38:02.829652 7fe9bae887a0 20 read_log 192888'15413 (192888'15412) modify ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by client.942868.0:23164 2014-08-11 17:06:43.574231 2014-08-13 17:38:02.829664 7fe9bae887a0 20 read_log 192888'15416 (0'0) promote 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:1784 2014-08-11 17:06:45.608406 2014-08-13 17:38:02.829676 7fe9bae887a0 20 read_log 192888'15417 (192888'15416) modify 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by client.942868.0:23702 2014-08-11 17:06:45.601126 2014-08-13 17:38:02.829688 7fe9bae887a0 20 read_log 192888'15422 (0'0) promote 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by osd.6.0:1792 2014-08-11 17:06:48.842229 2014-08-13 17:38:02.829700 7fe9bae887a0 20 read_log 192888'15423 (192888'15422) modify 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.942868.0:24575 2014-08-11 17:06:48.837014 2014-08-13 17:38:02.829712 7fe9bae887a0 20 read_log 192888'15424 (0'0) promote 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by osd.6.0:1798 2014-08-11 17:06:56.808283 2014-08-13 17:38:02.829732 7fe9bae887a0 20 read_log 192888'15425 (192888'15424) modify 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.942868.0:26632 2014-08-11 17:06:56.801293 2014-08-13 17:38:02.829745 7fe9bae887a0 20 read_log 192888'15426 (0'0) promote dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by osd.6.0:1808 2014-08-11 17:06:59.633368 2014-08-13 17:38:02.829757 7fe9bae887a0 20 read_log 192888'15427 (192888'15426) modify dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.942868.0:27385 2014-08-11 17:06:59.625493 2014-08-13 17:38:02.829769 7fe9bae887a0 20 read_log 192888'15432 (0'0) promote 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by osd.6.0:1834 2014-08-11 17:07:11.019150 2014-08-13 17:38:02.829781 7fe9bae887a0 20 read_log 192888'15433 (192888'15432) modify 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.942868.0:30295 2014-08-11 17:07:11.011931 2014-08-13 17:38:02.829794 7fe9bae887a0 20 read_log 192888'15434 (0'0) promote d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:1838 2014-08-11 17:07:14.279657 2014-08-13 17:38:02.829808 7fe9bae887a0 20 read_log 192888'15435 (192888'15434) modify d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by client.942868.0:31124 2014-08-11 17:07:14.271810 2014-08-13 17:38:02.829821 7fe9bae887a0 20 read_log 192888'15436 (192888'15325) modify d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.942868.0:31713 2014-08-11 17:07:16.036133 2014-08-13 17:38:02.829833 7fe9bae887a0 20 read_log 192888'15441 (192888'15335) modify 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.942868.0:35558 2014-08-11 17:07:17.103824 2014-08-13 17:38:02.829846 7fe9bae887a0 20 read_log 192888'15442 (192888'15337) modify 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.942868.0:35961 2014-08-11 17:07:17.206816 2014-08-13 17:38:02.829859 7fe9bae887a0 20 read_log 192888'15445 (192888'15343) modify 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.942868.0:38183 2014-08-11 17:07:17.816231 2014-08-13 17:38:02.829871 7fe9bae887a0 20 read_log 192888'15447 (192888'15347) modify 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by client.942868.0:39279 2014-08-11 17:07:18.137715 2014-08-13 17:38:02.829883 7fe9bae887a0 20 read_log 192888'15451 (192888'15355) modify 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.942868.0:40360 2014-08-11 17:07:18.429592 2014-08-13 17:38:02.829896 7fe9bae887a0 20 read_log 192888'15453 (192888'15359) modify bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.942868.0:40815 2014-08-11 17:07:18.554453 2014-08-13 17:38:02.829908 7fe9bae887a0 20 read_log 192888'15454 (192888'15361) modify 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.942868.0:40893 2014-08-11 17:07:18.574844 2014-08-13 17:38:02.829920 7fe9bae887a0 20 read_log 192888'15456 (192888'15365) modify 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.942868.0:42100 2014-08-11 17:07:18.905739 2014-08-13 17:38:02.829932 7fe9bae887a0 20 read_log 192888'15457 (192888'15367) modify 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.942868.0:42949 2014-08-11 17:07:19.136104 2014-08-13 17:38:02.829944 7fe9bae887a0 20 read_log 192888'15458 (192888'15369) modify 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.942868.0:43159 2014-08-11 17:07:19.196037 2014-08-13 17:38:02.829958 7fe9bae887a0 20 read_log 192888'15461 (192888'15375) modify f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by client.942868.0:44161 2014-08-11 17:07:19.487549 2014-08-13 17:38:02.829970 7fe9bae887a0 20 read_log 192888'15462 (192888'15377) modify da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.942868.0:44410 2014-08-11 17:07:19.559049 2014-08-13 17:38:02.829992 7fe9bae887a0 20 read_log 192888'15463 (192888'15379) modify 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.942868.0:46243 2014-08-11 17:07:20.065633 2014-08-13 17:38:02.830022 7fe9bae887a0 20 read_log 192888'15464 (192888'15381) modify 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.942868.0:46528 2014-08-11 17:07:20.149099 2014-08-13 17:38:02.830038 7fe9bae887a0 20 read_log 192888'15465 (192888'15383) modify 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.942868.0:48794 2014-08-11 17:07:20.819233 2014-08-13 17:38:02.830050 7fe9bae887a0 20 read_log 192888'15466 (192888'15385) modify 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by client.942868.0:49792 2014-08-11 17:07:21.123059 2014-08-13 17:38:02.830062 7fe9bae887a0 20 read_log 192888'15468 (192888'15389) modify 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by client.942868.0:50269 2014-08-11 17:07:21.265447 2014-08-13 17:38:02.830074 7fe9bae887a0 20 read_log 192888'15469 (192888'15391) modify 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.942868.0:51090 2014-08-11 17:07:21.528505 2014-08-13 17:38:02.830086 7fe9bae887a0 20 read_log 192888'15475 (192888'15403) modify e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.942868.0:53246 2014-08-11 17:07:22.197644 2014-08-13 17:38:02.830098 7fe9bae887a0 20 read_log 192888'15476 (192888'15405) modify d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.942868.0:53389 2014-08-11 17:07:22.247965 2014-08-13 17:38:02.830110 7fe9bae887a0 20 read_log 192888'15478 (192888'15409) modify ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.942868.0:54091 2014-08-11 17:07:22.461397 2014-08-13 17:38:02.830123 7fe9bae887a0 20 read_log 192888'15480 (192888'15413) modify ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by client.942868.0:54430 2014-08-11 17:07:22.554534 2014-08-13 17:38:02.830135 7fe9bae887a0 20 read_log 192888'15482 (192888'15417) modify 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by client.942868.0:54968 2014-08-11 17:07:22.706277 2014-08-13 17:38:02.830147 7fe9bae887a0 20 read_log 192888'15485 (192888'15423) modify 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.942868.0:55841 2014-08-11 17:07:22.972775 2014-08-13 17:38:02.830172 7fe9bae887a0 20 read_log 192888'15486 (192888'15425) modify 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.942868.0:57898 2014-08-11 17:07:23.586382 2014-08-13 17:38:02.830188 7fe9bae887a0 20 read_log 192888'15487 (192888'15427) modify dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.942868.0:58651 2014-08-11 17:07:23.809555 2014-08-13 17:38:02.830220 7fe9bae887a0 20 read_log 192888'15490 (192888'15433) modify 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.942868.0:61561 2014-08-11 17:07:24.680622 2014-08-13 17:38:02.830232 7fe9bae887a0 20 read_log 192888'15491 (192888'15435) modify d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by client.942868.0:62390 2014-08-11 17:07:24.934421 2014-08-13 17:38:02.830245 7fe9bae887a0 20 read_log 193305'15492 (193305'15492) modify cd/hit_set_24.cd_archive_2014-08-11 18:37:04.943507_2014-08-11 18:42:36.902876/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.903321 2014-08-13 17:38:02.830257 7fe9bae887a0 20 read_log 193305'15493 (192888'15322) delete cd/hit_set_24.cd_archive_2014-08-11 16:44:57.980787_2014-08-11 17:05:16.882120/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.903321 2014-08-13 17:38:02.830269 7fe9bae887a0 20 read_log 193305'15494 (0'0) promote e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:4 2014-08-11 18:42:36.926482 2014-08-13 17:38:02.830289 7fe9bae887a0 20 read_log 193305'15495 (193305'15494) modify e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by client.951337.0:981 2014-08-11 18:42:36.898841 2014-08-13 17:38:02.830304 7fe9bae887a0 20 read_log 193305'15496 (0'0) promote e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by osd.6.0:6 2014-08-11 18:42:36.927330 2014-08-13 17:38:02.830317 7fe9bae887a0 20 read_log 193305'15497 (193305'15496) modify e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.951337.0:983 2014-08-11 18:42:36.905811 2014-08-13 17:38:02.830330 7fe9bae887a0 20 read_log 193305'15498 (0'0) promote 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by osd.6.0:10 2014-08-11 18:42:40.448617 2014-08-13 17:38:02.830342 7fe9bae887a0 20 read_log 193305'15499 (193305'15498) modify 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.951337.0:1913 2014-08-11 18:42:40.442025 2014-08-13 17:38:02.830354 7fe9bae887a0 20 read_log 193305'15500 (0'0) promote d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by osd.6.0:17 2014-08-11 18:42:43.061959 2014-08-13 17:38:02.830366 7fe9bae887a0 20 read_log 193305'15501 (193305'15500) modify d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.951337.0:2635 2014-08-11 18:42:43.055621 2014-08-13 17:38:02.830378 7fe9bae887a0 20 read_log 193305'15502 (0'0) promote f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by osd.6.0:21 2014-08-11 18:42:44.301399 2014-08-13 17:38:02.830390 7fe9bae887a0 20 read_log 193305'15503 (193305'15502) modify f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.951337.0:2976 2014-08-11 18:42:44.294613 2014-08-13 17:38:02.830402 7fe9bae887a0 20 read_log 193305'15504 (0'0) promote 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:35 2014-08-11 18:42:55.827476 2014-08-13 17:38:02.830414 7fe9bae887a0 20 read_log 193305'15505 (193305'15504) modify 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by client.951337.0:6109 2014-08-11 18:42:55.818242 2014-08-13 17:38:02.830426 7fe9bae887a0 20 read_log 193305'15506 (0'0) promote 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:39 2014-08-11 18:42:57.498207 2014-08-13 17:38:02.830438 7fe9bae887a0 20 read_log 193305'15507 (193305'15506) modify 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by client.951337.0:6571 2014-08-11 18:42:57.490010 2014-08-13 17:38:02.830450 7fe9bae887a0 20 read_log 193305'15508 (0'0) promote 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by osd.6.0:47 2014-08-11 18:43:03.313376 2014-08-13 17:38:02.830462 7fe9bae887a0 20 read_log 193305'15509 (193305'15508) modify 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.951337.0:8175 2014-08-11 18:43:03.303280 2014-08-13 17:38:02.830474 7fe9bae887a0 20 read_log 193305'15510 (0'0) promote da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:51 2014-08-11 18:43:05.772640 2014-08-13 17:38:02.830486 7fe9bae887a0 20 read_log 193305'15511 (193305'15510) modify da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by client.951337.0:8779 2014-08-11 18:43:05.765411 2014-08-13 17:38:02.830498 7fe9bae887a0 20 read_log 193305'15512 (0'0) promote 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:53 2014-08-11 18:43:07.663883 2014-08-13 17:38:02.830510 7fe9bae887a0 20 read_log 193305'15513 (193305'15512) modify 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by client.951337.0:9285 2014-08-11 18:43:07.657511 2014-08-13 17:38:02.830522 7fe9bae887a0 20 read_log 193305'15514 (0'0) promote 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:57 2014-08-11 18:43:08.349300 2014-08-13 17:38:02.830541 7fe9bae887a0 20 read_log 193305'15515 (193305'15514) modify 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by client.951337.0:9468 2014-08-11 18:43:08.342094 2014-08-13 17:38:02.830554 7fe9bae887a0 20 read_log 193305'15516 (0'0) promote a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by osd.6.0:65 2014-08-11 18:43:12.602268 2014-08-13 17:38:02.830566 7fe9bae887a0 20 read_log 193305'15517 (193305'15516) modify a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.951337.0:10622 2014-08-11 18:43:12.595214 2014-08-13 17:38:02.830578 7fe9bae887a0 20 read_log 193305'15518 (0'0) promote dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by osd.6.0:69 2014-08-11 18:43:14.085572 2014-08-13 17:38:02.830590 7fe9bae887a0 20 read_log 193305'15519 (193305'15518) modify dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.951337.0:11020 2014-08-11 18:43:14.075760 2014-08-13 17:38:02.830602 7fe9bae887a0 20 read_log 193305'15520 (0'0) promote 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:71 2014-08-11 18:43:15.841663 2014-08-13 17:38:02.830614 7fe9bae887a0 20 read_log 193305'15521 (193305'15520) modify 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by client.951337.0:11443 2014-08-11 18:43:15.833237 2014-08-13 17:38:02.830626 7fe9bae887a0 20 read_log 193305'15522 (0'0) promote 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by osd.6.0:80 2014-08-11 18:43:17.958735 2014-08-13 17:38:02.830638 7fe9bae887a0 20 read_log 193305'15523 (193305'15522) modify 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.951337.0:12018 2014-08-11 18:43:17.951291 2014-08-13 17:38:02.830650 7fe9bae887a0 20 read_log 193305'15524 (0'0) promote 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by osd.6.0:82 2014-08-11 18:43:21.110679 2014-08-13 17:38:02.830662 7fe9bae887a0 20 read_log 193305'15525 (193305'15524) modify 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.951337.0:12859 2014-08-11 18:43:21.103976 2014-08-13 17:38:02.830675 7fe9bae887a0 20 read_log 193305'15526 (0'0) promote da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by osd.6.0:84 2014-08-11 18:43:25.150463 2014-08-13 17:38:02.830687 7fe9bae887a0 20 read_log 193305'15527 (193305'15526) modify da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.951337.0:13934 2014-08-11 18:43:25.140910 2014-08-13 17:38:02.830699 7fe9bae887a0 20 read_log 193305'15528 (0'0) promote eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:92 2014-08-11 18:43:27.094127 2014-08-13 17:38:02.830711 7fe9bae887a0 20 read_log 193305'15529 (193305'15528) modify eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by client.951337.0:14461 2014-08-11 18:43:27.087528 2014-08-13 17:38:02.830723 7fe9bae887a0 20 read_log 193305'15530 (0'0) promote db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by osd.6.0:96 2014-08-11 18:43:29.017437 2014-08-13 17:38:02.830735 7fe9bae887a0 20 read_log 193305'15531 (193305'15530) modify db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.951337.0:15010 2014-08-11 18:43:29.004947 2014-08-13 17:38:02.830747 7fe9bae887a0 20 read_log 193305'15532 (0'0) promote 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:98 2014-08-11 18:43:33.516706 2014-08-13 17:38:02.830759 7fe9bae887a0 20 read_log 193305'15533 (193305'15532) modify 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by client.951337.0:16301 2014-08-11 18:43:33.508755 2014-08-13 17:38:02.830771 7fe9bae887a0 20 read_log 193305'15534 (0'0) promote 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:100 2014-08-11 18:43:35.665089 2014-08-13 17:38:02.830792 7fe9bae887a0 20 read_log 193305'15535 (193305'15534) modify 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by client.951337.0:16920 2014-08-11 18:43:35.657157 2014-08-13 17:38:02.830805 7fe9bae887a0 20 read_log 193305'15536 (0'0) promote 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by osd.6.0:104 2014-08-11 18:43:36.034807 2014-08-13 17:38:02.830817 7fe9bae887a0 20 read_log 193305'15537 (193305'15536) modify 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.951337.0:17022 2014-08-11 18:43:36.027478 2014-08-13 17:38:02.830830 7fe9bae887a0 20 read_log 193305'15538 (0'0) promote 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by osd.6.0:108 2014-08-11 18:43:38.236503 2014-08-13 17:38:02.830842 7fe9bae887a0 20 read_log 193305'15539 (193305'15538) modify 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.951337.0:17684 2014-08-11 18:43:38.229708 2014-08-13 17:38:02.830854 7fe9bae887a0 20 read_log 193305'15540 (0'0) promote 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:112 2014-08-11 18:43:42.504735 2014-08-13 17:38:02.830866 7fe9bae887a0 20 read_log 193305'15541 (193305'15540) modify 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by client.951337.0:18885 2014-08-11 18:43:42.496612 2014-08-13 17:38:02.830878 7fe9bae887a0 20 read_log 193305'15542 (0'0) promote 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:118 2014-08-11 18:43:47.741915 2014-08-13 17:38:02.830890 7fe9bae887a0 20 read_log 193305'15543 (193305'15542) modify 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by client.951337.0:20296 2014-08-11 18:43:47.732414 2014-08-13 17:38:02.830902 7fe9bae887a0 20 read_log 193305'15544 (0'0) promote 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:120 2014-08-11 18:43:51.888641 2014-08-13 17:38:02.830913 7fe9bae887a0 20 read_log 193305'15545 (193305'15544) modify 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by client.951337.0:21467 2014-08-11 18:43:51.881425 2014-08-13 17:38:02.830925 7fe9bae887a0 20 read_log 193305'15546 (0'0) promote e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by osd.6.0:128 2014-08-11 18:43:57.805777 2014-08-13 17:38:02.830939 7fe9bae887a0 20 read_log 193305'15547 (193305'15546) modify e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.951337.0:23105 2014-08-11 18:43:57.798458 2014-08-13 17:38:02.830954 7fe9bae887a0 20 read_log 193305'15548 (0'0) promote 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by osd.6.0:138 2014-08-11 18:44:06.822851 2014-08-13 17:38:02.830966 7fe9bae887a0 20 read_log 193305'15549 (193305'15548) modify 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.951337.0:25589 2014-08-11 18:44:06.815563 2014-08-13 17:38:02.830980 7fe9bae887a0 20 read_log 193305'15550 (0'0) promote 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by osd.6.0:144 2014-08-11 18:44:10.392901 2014-08-13 17:38:02.830992 7fe9bae887a0 20 read_log 193305'15551 (193305'15550) modify 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.951337.0:26557 2014-08-11 18:44:10.385017 2014-08-13 17:38:02.831007 7fe9bae887a0 20 read_log 193305'15552 (0'0) promote aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by osd.6.0:146 2014-08-11 18:44:10.732854 2014-08-13 17:38:02.831019 7fe9bae887a0 20 read_log 193305'15553 (193305'15552) modify aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.951337.0:26649 2014-08-11 18:44:10.724564 2014-08-13 17:38:02.831033 7fe9bae887a0 20 read_log 193305'15554 (0'0) promote 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:150 2014-08-11 18:44:11.355989 2014-08-13 17:38:02.831052 7fe9bae887a0 20 read_log 193305'15555 (193305'15554) modify 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by client.951337.0:26824 2014-08-11 18:44:11.345742 2014-08-13 17:38:02.831067 7fe9bae887a0 20 read_log 193305'15556 (0'0) promote 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by osd.6.0:152 2014-08-11 18:44:12.869699 2014-08-13 17:38:02.831079 7fe9bae887a0 20 read_log 193305'15557 (193305'15556) modify 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.951337.0:27218 2014-08-11 18:44:12.859720 2014-08-13 17:38:02.831092 7fe9bae887a0 20 read_log 193305'15558 (0'0) promote b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:154 2014-08-11 18:44:13.025345 2014-08-13 17:38:02.831104 7fe9bae887a0 20 read_log 193305'15559 (193305'15558) modify b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by client.951337.0:27260 2014-08-11 18:44:13.017520 2014-08-13 17:38:02.831116 7fe9bae887a0 20 read_log 193305'15560 (0'0) promote 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by osd.6.0:156 2014-08-11 18:44:14.399127 2014-08-13 17:38:02.831130 7fe9bae887a0 20 read_log 193305'15561 (193305'15560) modify 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.951337.0:27646 2014-08-11 18:44:14.390173 2014-08-13 17:38:02.831142 7fe9bae887a0 20 read_log 193305'15562 (0'0) promote e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:162 2014-08-11 18:44:21.820513 2014-08-13 17:38:02.831161 7fe9bae887a0 20 read_log 193305'15563 (193305'15562) modify e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by client.951337.0:29684 2014-08-11 18:44:21.814306 2014-08-13 17:38:02.831176 7fe9bae887a0 20 read_log 193305'15564 (0'0) promote ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:164 2014-08-11 18:44:22.094535 2014-08-13 17:38:02.831188 7fe9bae887a0 20 read_log 193305'15565 (193305'15564) modify ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by client.951337.0:29753 2014-08-11 18:44:22.088313 2014-08-13 17:38:02.831201 7fe9bae887a0 20 read_log 193305'15566 (0'0) promote a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:166 2014-08-11 18:44:24.092571 2014-08-13 17:38:02.831213 7fe9bae887a0 20 read_log 193305'15567 (193305'15566) modify a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by client.951337.0:30270 2014-08-11 18:44:24.084289 2014-08-13 17:38:02.831225 7fe9bae887a0 20 read_log 193305'15568 (0'0) promote 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by osd.6.0:168 2014-08-11 18:44:25.377964 2014-08-13 17:38:02.831239 7fe9bae887a0 20 read_log 193305'15569 (193305'15568) modify 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.951337.0:30598 2014-08-11 18:44:25.369933 2014-08-13 17:38:02.831272 7fe9bae887a0 20 read_log 193305'15570 (0'0) promote 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:170 2014-08-11 18:44:28.561574 2014-08-13 17:38:02.831288 7fe9bae887a0 20 read_log 193305'15571 (193305'15570) modify 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by client.951337.0:31499 2014-08-11 18:44:28.554908 2014-08-13 17:38:02.831301 7fe9bae887a0 20 read_log 193305'15572 (193305'15495) modify e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by client.951337.0:33692 2014-08-11 18:44:34.583624 2014-08-13 17:38:02.831315 7fe9bae887a0 20 read_log 193305'15573 (193305'15497) modify e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.951337.0:33694 2014-08-11 18:44:34.584468 2014-08-13 17:38:02.831345 7fe9bae887a0 20 read_log 193305'15574 (193305'15499) modify 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.951337.0:34624 2014-08-11 18:44:35.099666 2014-08-13 17:38:02.831367 7fe9bae887a0 20 read_log 193305'15575 (193305'15501) modify d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.951337.0:35346 2014-08-11 18:44:35.521853 2014-08-13 17:38:02.831382 7fe9bae887a0 20 read_log 193305'15576 (193305'15503) modify f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.951337.0:35687 2014-08-11 18:44:35.703275 2014-08-13 17:38:02.831394 7fe9bae887a0 20 read_log 193305'15577 (193305'15505) modify 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by client.951337.0:38820 2014-08-11 18:44:37.677494 2014-08-13 17:38:02.831408 7fe9bae887a0 20 read_log 193305'15578 (193305'15507) modify 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by client.951337.0:39282 2014-08-11 18:44:37.921499 2014-08-13 17:38:02.831420 7fe9bae887a0 20 read_log 193305'15579 (193305'15509) modify 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.951337.0:40886 2014-08-11 18:44:38.567551 2014-08-13 17:38:02.831432 7fe9bae887a0 20 read_log 193305'15580 (193305'15511) modify da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by client.951337.0:41490 2014-08-11 18:44:38.769094 2014-08-13 17:38:02.831444 7fe9bae887a0 20 read_log 193305'15581 (193305'15513) modify 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by client.951337.0:41996 2014-08-11 18:44:38.976321 2014-08-13 17:38:02.831457 7fe9bae887a0 20 read_log 193305'15582 (193305'15515) modify 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by client.951337.0:42179 2014-08-11 18:44:39.136016 2014-08-13 17:38:02.831469 7fe9bae887a0 20 read_log 193305'15583 (193305'15517) modify a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.951337.0:43333 2014-08-11 18:44:39.557821 2014-08-13 17:38:02.831481 7fe9bae887a0 20 read_log 193305'15584 (193305'15519) modify dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.951337.0:43731 2014-08-11 18:44:39.680661 2014-08-13 17:38:02.831493 7fe9bae887a0 20 read_log 193305'15585 (193305'15521) modify 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by client.951337.0:44154 2014-08-11 18:44:39.807167 2014-08-13 17:38:02.831505 7fe9bae887a0 20 read_log 193305'15586 (193305'15523) modify 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.951337.0:44729 2014-08-11 18:44:39.971916 2014-08-13 17:38:02.831517 7fe9bae887a0 20 read_log 193305'15587 (193305'15525) modify 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.951337.0:45570 2014-08-11 18:44:40.237983 2014-08-13 17:38:02.831528 7fe9bae887a0 20 read_log 193305'15588 (193305'15527) modify da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.951337.0:46645 2014-08-11 18:44:40.546293 2014-08-13 17:38:02.831540 7fe9bae887a0 20 read_log 193305'15589 (193305'15529) modify eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by client.951337.0:47172 2014-08-11 18:44:40.693345 2014-08-13 17:38:02.831552 7fe9bae887a0 20 read_log 193305'15590 (193305'15531) modify db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.951337.0:47721 2014-08-11 18:44:40.847707 2014-08-13 17:38:02.831564 7fe9bae887a0 20 read_log 193305'15591 (193305'15533) modify 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by client.951337.0:49012 2014-08-11 18:44:41.227499 2014-08-13 17:38:02.831576 7fe9bae887a0 20 read_log 193305'15592 (193305'15535) modify 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by client.951337.0:49631 2014-08-11 18:44:41.423723 2014-08-13 17:38:02.831587 7fe9bae887a0 20 read_log 193305'15593 (193305'15537) modify 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.951337.0:49733 2014-08-11 18:44:41.452764 2014-08-13 17:38:02.831599 7fe9bae887a0 20 read_log 193305'15594 (193305'15539) modify 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.951337.0:50395 2014-08-11 18:44:41.652887 2014-08-13 17:38:02.831635 7fe9bae887a0 20 read_log 193305'15595 (193305'15541) modify 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by client.951337.0:51596 2014-08-11 18:44:41.996706 2014-08-13 17:38:02.831651 7fe9bae887a0 20 read_log 193305'15596 (193305'15543) modify 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by client.951337.0:53007 2014-08-11 18:44:42.403396 2014-08-13 17:38:02.831663 7fe9bae887a0 20 read_log 193305'15597 (193305'15545) modify 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by client.951337.0:54178 2014-08-11 18:44:42.751202 2014-08-13 17:38:02.831675 7fe9bae887a0 20 read_log 193305'15598 (193305'15547) modify e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.951337.0:55816 2014-08-11 18:44:43.241411 2014-08-13 17:38:02.831687 7fe9bae887a0 20 read_log 193305'15599 (193305'15549) modify 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.951337.0:58300 2014-08-11 18:44:44.000080 2014-08-13 17:38:02.831699 7fe9bae887a0 20 read_log 193305'15600 (193305'15551) modify 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.951337.0:59268 2014-08-11 18:44:44.277736 2014-08-13 17:38:02.831711 7fe9bae887a0 20 read_log 193305'15601 (193305'15553) modify aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.951337.0:59360 2014-08-11 18:44:44.304020 2014-08-13 17:38:02.831723 7fe9bae887a0 20 read_log 193305'15602 (193305'15555) modify 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by client.951337.0:59535 2014-08-11 18:44:44.354491 2014-08-13 17:38:02.831735 7fe9bae887a0 20 read_log 193305'15603 (193305'15557) modify 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.951337.0:59929 2014-08-11 18:44:44.469215 2014-08-13 17:38:02.831747 7fe9bae887a0 20 read_log 193305'15604 (193305'15559) modify b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by client.951337.0:59971 2014-08-11 18:44:44.481152 2014-08-13 17:38:02.831758 7fe9bae887a0 20 read_log 193305'15605 (193305'15561) modify 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.951337.0:60357 2014-08-11 18:44:44.605610 2014-08-13 17:38:02.831770 7fe9bae887a0 20 read_log 193305'15606 (193305'15563) modify e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by client.951337.0:62395 2014-08-11 18:44:45.214982 2014-08-13 17:38:02.831782 7fe9bae887a0 20 read_log 193305'15607 (193305'15565) modify ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by client.951337.0:62464 2014-08-11 18:44:45.234587 2014-08-13 17:38:02.831794 7fe9bae887a0 20 read_log 193305'15608 (193305'15567) modify a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by client.951337.0:62981 2014-08-11 18:44:45.378711 2014-08-13 17:38:02.831806 7fe9bae887a0 20 read_log 193305'15609 (193305'15569) modify 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.951337.0:63309 2014-08-11 18:44:45.467753 2014-08-13 17:38:02.831818 7fe9bae887a0 20 read_log 193305'15610 (193305'15571) modify 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by client.951337.0:64210 2014-08-11 18:44:45.736754 2014-08-13 17:38:02.831830 7fe9bae887a0 20 read_log 194209'15611 (193305'15596) clean 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:458 2014-08-12 14:07:27.199692 2014-08-13 17:38:02.831842 7fe9bae887a0 20 read_log 194209'15612 (192888'15461) clean f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:459 2014-08-12 14:07:27.199982 2014-08-13 17:38:02.831854 7fe9bae887a0 20 read_log 194209'15613 (193305'15610) clean 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:460 2014-08-12 14:07:27.200546 2014-08-13 17:38:02.831866 7fe9bae887a0 20 read_log 194209'15614 (192888'15466) clean 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:461 2014-08-12 14:07:27.200954 2014-08-13 17:38:02.831885 7fe9bae887a0 20 read_log 194209'15615 (192888'15271) clean 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:462 2014-08-12 14:07:27.270171 2014-08-13 17:38:02.831898 7fe9bae887a0 20 read_log 194209'15616 (193305'15589) clean eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:463 2014-08-12 14:07:27.273295 2014-08-13 17:38:02.831910 7fe9bae887a0 20 read_log 194209'15617 (193305'15608) clean a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:464 2014-08-12 14:07:27.276442 2014-08-13 17:38:02.831922 7fe9bae887a0 20 read_log 194209'15618 (192834'14537) clean dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.6.0:465 2014-08-12 14:07:27.279717 2014-08-13 17:38:02.831934 7fe9bae887a0 20 read_log 194209'15619 (192888'15480) clean ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:466 2014-08-12 14:07:27.284230 2014-08-13 17:38:02.831945 7fe9bae887a0 20 read_log 194209'15620 (192834'14616) clean cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.6.0:467 2014-08-12 14:07:27.287455 2014-08-13 17:38:02.831957 7fe9bae887a0 20 read_log 194209'15621 (193305'15606) clean e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:468 2014-08-12 14:07:27.293128 2014-08-13 17:38:02.831969 7fe9bae887a0 20 read_log 194209'15622 (192834'14621) clean a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.6.0:469 2014-08-12 14:07:27.293470 2014-08-13 17:38:02.831981 7fe9bae887a0 20 read_log 194209'15623 (193305'15585) clean 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:470 2014-08-12 14:07:27.296594 2014-08-13 17:38:02.831993 7fe9bae887a0 20 read_log 194209'15624 (192834'15010) clean b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.6.0:471 2014-08-12 14:07:27.299767 2014-08-13 17:38:02.832021 7fe9bae887a0 20 read_log 194209'15625 (192834'15034) clean f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.6.0:472 2014-08-12 14:07:27.303198 2014-08-13 17:38:02.832037 7fe9bae887a0 20 read_log 194209'15626 (193305'15572) clean e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:473 2014-08-12 14:07:27.306155 2014-08-13 17:38:02.832050 7fe9bae887a0 20 read_log 194209'15627 (192888'15447) clean 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:474 2014-08-12 14:07:27.309348 2014-08-13 17:38:02.832062 7fe9bae887a0 20 read_log 194209'15628 (193305'15597) clean 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:475 2014-08-12 14:07:27.312460 2014-08-13 17:38:02.832074 7fe9bae887a0 20 read_log 194209'15629 (192834'14581) clean 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.6.0:476 2014-08-12 14:07:27.315579 2014-08-13 17:38:02.832085 7fe9bae887a0 20 read_log 194209'15630 (192834'14474) clean adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.6.0:477 2014-08-12 14:07:27.318771 2014-08-13 17:38:02.832097 7fe9bae887a0 20 read_log 194209'15631 (193305'15591) clean 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:478 2014-08-12 14:07:27.321922 2014-08-13 17:38:02.832109 7fe9bae887a0 20 read_log 194209'15632 (192834'15093) clean baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.6.0:479 2014-08-12 14:07:27.325105 2014-08-13 17:38:02.832121 7fe9bae887a0 20 read_log 194209'15633 (193305'15607) clean ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:480 2014-08-12 14:07:27.328278 2014-08-13 17:38:02.832132 7fe9bae887a0 20 read_log 194209'15634 (193305'15581) clean 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:481 2014-08-12 14:07:27.331506 2014-08-13 17:38:02.832158 7fe9bae887a0 20 read_log 194209'15635 (192834'15054) clean 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.6.0:482 2014-08-12 14:07:27.334847 2014-08-13 17:38:02.832172 7fe9bae887a0 20 read_log 194209'15636 (193305'15592) clean 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:483 2014-08-12 14:07:27.338026 2014-08-13 17:38:02.832187 7fe9bae887a0 20 read_log 194209'15637 (193305'15602) clean 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:484 2014-08-12 14:07:27.341146 2014-08-13 17:38:02.832199 7fe9bae887a0 20 read_log 194209'15638 (193305'15577) clean 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:485 2014-08-12 14:07:27.343956 2014-08-13 17:38:02.832211 7fe9bae887a0 20 read_log 194209'15639 (192834'15124) clean bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.6.0:486 2014-08-12 14:07:27.346983 2014-08-13 17:38:02.832223 7fe9bae887a0 20 read_log 194209'15640 (193305'15604) clean b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:487 2014-08-12 14:07:27.350026 2014-08-13 17:38:02.832235 7fe9bae887a0 20 read_log 194209'15641 (192888'15298) clean 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:488 2014-08-12 14:07:27.353029 2014-08-13 17:38:02.832247 7fe9bae887a0 20 read_log 194209'15642 (192888'15321) clean 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:489 2014-08-12 14:07:27.356458 2014-08-13 17:38:02.832259 7fe9bae887a0 20 read_log 194209'15643 (193305'15582) clean 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:490 2014-08-12 14:07:27.359584 2014-08-13 17:38:02.832271 7fe9bae887a0 20 read_log 194209'15644 (192834'15041) clean 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.6.0:491 2014-08-12 14:07:27.362524 2014-08-13 17:38:02.832283 7fe9bae887a0 20 read_log 194209'15645 (192834'15052) clean 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.6.0:492 2014-08-12 14:07:27.365474 2014-08-13 17:38:02.832296 7fe9bae887a0 20 read_log 194209'15646 (192888'15482) clean 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:493 2014-08-12 14:07:27.368538 2014-08-13 17:38:02.832313 7fe9bae887a0 20 read_log 194209'15647 (193305'15578) clean 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:494 2014-08-12 14:07:27.371691 2014-08-13 17:38:02.832325 7fe9bae887a0 20 read_log 194209'15648 (192888'15313) clean 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:495 2014-08-12 14:07:27.377180 2014-08-13 17:38:02.832337 7fe9bae887a0 20 read_log 194209'15649 (192888'15312) clean 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:496 2014-08-12 14:07:27.377440 2014-08-13 17:38:02.832349 7fe9bae887a0 20 read_log 194209'15650 (193305'15580) clean da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:497 2014-08-12 14:07:27.380385 2014-08-13 17:38:02.832360 7fe9bae887a0 20 read_log 194209'15651 (192888'15491) clean d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:498 2014-08-12 14:07:27.383447 2014-08-13 17:38:02.832372 7fe9bae887a0 20 read_log 194209'15652 (192888'15468) clean 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:499 2014-08-12 14:07:27.386736 2014-08-13 17:38:02.832384 7fe9bae887a0 20 read_log 194209'15653 (192888'15456) clean 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by osd.6.0:500 2014-08-12 14:07:27.389799 2014-08-13 17:38:02.832396 7fe9bae887a0 20 read_log 194209'15654 (194209'15611) delete 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:501 2014-08-12 14:07:27.422055 2014-08-13 17:38:02.832415 7fe9bae887a0 20 read_log 194209'15655 (194209'15613) delete 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:502 2014-08-12 14:07:27.425434 2014-08-13 17:38:02.832428 7fe9bae887a0 20 read_log 194209'15656 (194209'15615) delete 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:503 2014-08-12 14:07:27.428743 2014-08-13 17:38:02.832440 7fe9bae887a0 20 read_log 194209'15657 (194209'15616) delete eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:504 2014-08-12 14:07:27.431936 2014-08-13 17:38:02.832452 7fe9bae887a0 20 read_log 194209'15658 (194209'15617) delete a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:505 2014-08-12 14:07:27.435070 2014-08-13 17:38:02.832464 7fe9bae887a0 20 read_log 194209'15659 (193305'15595) clean 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:506 2014-08-12 14:07:27.435396 2014-08-13 17:38:02.832476 7fe9bae887a0 20 read_log 194209'15660 (194209'15618) delete dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.6.0:507 2014-08-12 14:07:27.438798 2014-08-13 17:38:02.832488 7fe9bae887a0 20 read_log 194209'15661 (194209'15620) delete cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.6.0:508 2014-08-12 14:07:27.441683 2014-08-13 17:38:02.832500 7fe9bae887a0 20 read_log 194209'15662 (194209'15621) delete e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:509 2014-08-12 14:07:27.442059 2014-08-13 17:38:02.832532 7fe9bae887a0 20 read_log 194209'15663 (194209'15622) delete a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.6.0:510 2014-08-12 14:07:27.442395 2014-08-13 17:38:02.832548 7fe9bae887a0 20 read_log 194209'15664 (194209'15623) delete 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:511 2014-08-12 14:07:27.445487 2014-08-13 17:38:02.832560 7fe9bae887a0 20 read_log 194209'15665 (194209'15624) delete b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.6.0:512 2014-08-12 14:07:27.445850 2014-08-13 17:38:02.832572 7fe9bae887a0 20 read_log 194209'15666 (194209'15625) delete f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.6.0:513 2014-08-12 14:07:27.446728 2014-08-13 17:38:02.832584 7fe9bae887a0 20 read_log 194209'15667 (194209'15626) delete e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:514 2014-08-12 14:07:27.450837 2014-08-13 17:38:02.832596 7fe9bae887a0 20 read_log 194209'15668 (194209'15628) delete 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:515 2014-08-12 14:07:27.451290 2014-08-13 17:38:02.832608 7fe9bae887a0 20 read_log 194209'15669 (194209'15629) delete 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.6.0:516 2014-08-12 14:07:27.451814 2014-08-13 17:38:02.832620 7fe9bae887a0 20 read_log 194209'15670 (194209'15630) delete adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.6.0:517 2014-08-12 14:07:27.454929 2014-08-13 17:38:02.832631 7fe9bae887a0 20 read_log 194209'15671 (194209'15631) delete 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:518 2014-08-12 14:07:27.455234 2014-08-13 17:38:02.832643 7fe9bae887a0 20 read_log 194209'15672 (194209'15632) delete baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.6.0:519 2014-08-12 14:07:27.455512 2014-08-13 17:38:02.832655 7fe9bae887a0 20 read_log 194209'15673 (194209'15633) delete ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:520 2014-08-12 14:07:27.458778 2014-08-13 17:38:02.832667 7fe9bae887a0 20 read_log 194209'15674 (194209'15634) delete 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:521 2014-08-12 14:07:27.459086 2014-08-13 17:38:02.832686 7fe9bae887a0 20 read_log 194209'15675 (194209'15635) delete 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.6.0:522 2014-08-12 14:07:27.459396 2014-08-13 17:38:02.832699 7fe9bae887a0 20 read_log 194209'15676 (194209'15636) delete 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:523 2014-08-12 14:07:27.462795 2014-08-13 17:38:02.832712 7fe9bae887a0 20 read_log 194209'15677 (194209'15637) delete 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:524 2014-08-12 14:07:27.463119 2014-08-13 17:38:02.832724 7fe9bae887a0 20 read_log 194209'15678 (194209'15638) delete 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:525 2014-08-12 14:07:27.463407 2014-08-13 17:38:02.832735 7fe9bae887a0 20 read_log 194209'15679 (194209'15639) delete bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.6.0:526 2014-08-12 14:07:27.466570 2014-08-13 17:38:02.832747 7fe9bae887a0 20 read_log 194209'15680 (194209'15640) delete b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:527 2014-08-12 14:07:27.466868 2014-08-13 17:38:02.832759 7fe9bae887a0 20 read_log 194209'15681 (194209'15641) delete 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:528 2014-08-12 14:07:27.467143 2014-08-13 17:38:02.832772 7fe9bae887a0 20 read_log 194209'15682 (194209'15643) delete 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:529 2014-08-12 14:07:27.470476 2014-08-13 17:38:02.832784 7fe9bae887a0 20 read_log 194209'15683 (194209'15644) delete 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.6.0:530 2014-08-12 14:07:27.470793 2014-08-13 17:38:02.832796 7fe9bae887a0 20 read_log 194209'15684 (194209'15645) delete 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.6.0:531 2014-08-12 14:07:27.471407 2014-08-13 17:38:02.832808 7fe9bae887a0 20 read_log 194209'15685 (194209'15647) delete 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:532 2014-08-12 14:07:27.474590 2014-08-13 17:38:02.832823 7fe9bae887a0 20 read_log 194209'15686 (194209'15648) delete 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:533 2014-08-12 14:07:27.474946 2014-08-13 17:38:02.832835 7fe9bae887a0 20 read_log 194209'15687 (194209'15649) delete 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:534 2014-08-12 14:07:27.475489 2014-08-13 17:38:02.832847 7fe9bae887a0 20 read_log 194209'15688 (194209'15650) delete da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:535 2014-08-12 14:07:27.478252 2014-08-13 17:38:02.832859 7fe9bae887a0 20 read_log 194209'15689 (194209'15659) delete 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:536 2014-08-12 14:07:27.478729 2014-08-13 17:38:02.832870 7fe9bae887a0 20 read_log 194209'15690 (194209'15612) delete f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:599 2014-08-12 14:10:13.376915 2014-08-13 17:38:02.832882 7fe9bae887a0 20 read_log 194209'15691 (194209'15614) delete 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:600 2014-08-12 14:10:13.377288 2014-08-13 17:38:02.832894 7fe9bae887a0 20 read_log 194209'15692 (194209'15619) delete ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:601 2014-08-12 14:10:13.377596 2014-08-13 17:38:02.832906 7fe9bae887a0 20 read_log 194209'15693 (194209'15627) delete 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:602 2014-08-12 14:10:13.377859 2014-08-13 17:38:02.832918 7fe9bae887a0 20 read_log 194209'15694 (194209'15642) delete 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:603 2014-08-12 14:10:13.380553 2014-08-13 17:38:02.832940 7fe9bae887a0 20 read_log 194209'15695 (194209'15646) delete 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:604 2014-08-12 14:10:13.380906 2014-08-13 17:38:02.832953 7fe9bae887a0 20 read_log 194209'15696 (194209'15651) delete d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:605 2014-08-12 14:10:13.381699 2014-08-13 17:38:02.832964 7fe9bae887a0 20 read_log 194209'15697 (194209'15652) delete 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:606 2014-08-12 14:10:13.381995 2014-08-13 17:38:02.832979 7fe9bae887a0 20 read_log 194215'15698 (194215'15698) modify cd/hit_set_24.cd_archive_2014-08-12 15:32:25.874029_2014-08-12 15:39:20.336162/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:20.336405 2014-08-13 17:38:02.832991 7fe9bae887a0 20 read_log 194215'15699 (193305'15492) delete cd/hit_set_24.cd_archive_2014-08-11 18:37:04.943507_2014-08-11 18:42:36.902876/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:20.336405 2014-08-13 17:38:02.833004 7fe9bae887a0 20 read_log 194215'15700 (193305'15598) clean e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by osd.6.0:815 2014-08-12 15:39:20.340162 2014-08-13 17:38:02.833016 7fe9bae887a0 20 read_log 194215'15701 (194215'15700) delete e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.2174595.0:41908 0.000000 2014-08-13 17:38:02.833027 7fe9bae887a0 20 read_log 194215'15702 (192888'15475) clean e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by osd.6.0:817 2014-08-12 15:39:20.347804 2014-08-13 17:38:02.833039 7fe9bae887a0 20 read_log 194215'15703 (194215'15702) delete e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.2174595.0:41910 0.000000 2014-08-13 17:38:02.833049 7fe9bae887a0 20 read_log 194215'15704 (192834'14636) clean 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by osd.6.0:819 2014-08-12 15:39:20.354460 2014-08-13 17:38:02.833061 7fe9bae887a0 20 read_log 194215'15705 (194215'15704) delete 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.2174595.0:41912 0.000000 2014-08-13 17:38:02.833072 7fe9bae887a0 20 read_log 194215'15706 (193305'15605) clean 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by osd.6.0:821 2014-08-12 15:39:20.360528 2014-08-13 17:38:02.833084 7fe9bae887a0 20 read_log 194215'15707 (194215'15706) delete 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.2174595.0:41914 0.000000 2014-08-13 17:38:02.833095 7fe9bae887a0 20 read_log 194215'15708 (192888'15318) clean 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by osd.6.0:823 2014-08-12 15:39:20.367308 2014-08-13 17:38:02.833107 7fe9bae887a0 20 read_log 194215'15709 (194215'15708) delete 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.2174595.0:41916 0.000000 2014-08-13 17:38:02.833117 7fe9bae887a0 20 read_log 194215'15710 (192888'15272) clean dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by osd.6.0:825 2014-08-12 15:39:20.373354 2014-08-13 17:38:02.833129 7fe9bae887a0 20 read_log 194215'15711 (194215'15710) delete dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.2174595.0:41918 0.000000 2014-08-13 17:38:02.833140 7fe9bae887a0 20 read_log 194215'15712 (192888'15275) clean a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by osd.6.0:827 2014-08-12 15:39:20.379222 2014-08-13 17:38:02.833151 7fe9bae887a0 20 read_log 194215'15713 (194215'15712) delete a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.2174595.0:41920 0.000000 2014-08-13 17:38:02.833175 7fe9bae887a0 20 read_log 194215'15714 (192888'15276) clean d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by osd.6.0:829 2014-08-12 15:39:20.385350 2014-08-13 17:38:02.833188 7fe9bae887a0 20 read_log 194215'15715 (194215'15714) delete d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.2174595.0:41922 0.000000 2014-08-13 17:38:02.833198 7fe9bae887a0 20 read_log 194215'15716 (192834'14546) clean d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by osd.6.0:831 2014-08-12 15:39:20.391286 2014-08-13 17:38:02.833210 7fe9bae887a0 20 read_log 194215'15717 (194215'15716) delete d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.2174595.0:41924 0.000000 2014-08-13 17:38:02.833221 7fe9bae887a0 20 read_log 194215'15718 (192888'15462) clean da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by osd.6.0:833 2014-08-12 15:39:20.397438 2014-08-13 17:38:02.833233 7fe9bae887a0 20 read_log 194215'15719 (194215'15718) delete da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.2174595.0:41926 0.000000 2014-08-13 17:38:02.833243 7fe9bae887a0 20 read_log 194215'15720 (192888'15307) clean e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by osd.6.0:835 2014-08-12 15:39:20.405318 2014-08-13 17:38:02.833255 7fe9bae887a0 20 read_log 194215'15721 (194215'15720) delete e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.2174595.0:41928 0.000000 2014-08-13 17:38:02.833266 7fe9bae887a0 20 read_log 194215'15722 (192834'14557) clean 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by osd.6.0:837 2014-08-12 15:39:20.411992 2014-08-13 17:38:02.833278 7fe9bae887a0 20 read_log 194215'15723 (194215'15722) delete 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.2174595.0:41930 0.000000 2014-08-13 17:38:02.833288 7fe9bae887a0 20 read_log 194215'15724 (192834'15084) clean d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by osd.6.0:839 2014-08-12 15:39:20.417777 2014-08-13 17:38:02.833300 7fe9bae887a0 20 read_log 194215'15725 (194215'15724) delete d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.2174595.0:41932 0.000000 2014-08-13 17:38:02.833311 7fe9bae887a0 20 read_log 194215'15726 (192834'15030) clean 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by osd.6.0:841 2014-08-12 15:39:20.423798 2014-08-13 17:38:02.833322 7fe9bae887a0 20 read_log 194215'15727 (194215'15726) delete 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.2174595.0:41934 0.000000 2014-08-13 17:38:02.833333 7fe9bae887a0 20 read_log 194215'15728 (192834'14534) clean 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by osd.6.0:843 2014-08-12 15:39:20.430699 2014-08-13 17:38:02.833344 7fe9bae887a0 20 read_log 194215'15729 (194215'15728) delete 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.2174595.0:41936 0.000000 2014-08-13 17:38:02.833355 7fe9bae887a0 20 read_log 194215'15730 (192834'14651) clean 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by osd.6.0:845 2014-08-12 15:39:20.436890 2014-08-13 17:38:02.833367 7fe9bae887a0 20 read_log 194215'15731 (194215'15730) delete 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.2174595.0:41938 0.000000 2014-08-13 17:38:02.833378 7fe9bae887a0 20 read_log 194215'15732 (193305'15586) clean 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by osd.6.0:847 2014-08-12 15:39:20.443093 2014-08-13 17:38:02.833390 7fe9bae887a0 20 read_log 194215'15733 (194215'15732) delete 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.2174595.0:41940 0.000000 2014-08-13 17:38:02.833424 7fe9bae887a0 20 read_log 194215'15734 (193305'15590) clean db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by osd.6.0:849 2014-08-12 15:39:20.450186 2014-08-13 17:38:02.833438 7fe9bae887a0 20 read_log 194215'15735 (194215'15734) delete db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.2174595.0:41942 0.000000 2014-08-13 17:38:02.833449 7fe9bae887a0 20 read_log 194215'15736 (193305'15579) clean 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by osd.6.0:851 2014-08-12 15:39:20.456641 2014-08-13 17:38:02.833461 7fe9bae887a0 20 read_log 194215'15737 (194215'15736) delete 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.2174595.0:41944 0.000000 2014-08-13 17:38:02.833471 7fe9bae887a0 20 read_log 194215'15738 (192888'15289) clean 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by osd.6.0:853 2014-08-12 15:39:20.462259 2014-08-13 17:38:02.833483 7fe9bae887a0 20 read_log 194215'15739 (194215'15738) delete 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.2174595.0:41946 0.000000 2014-08-13 17:38:02.833494 7fe9bae887a0 20 read_log 194215'15740 (194209'15653) delete 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.2174595.0:41948 0.000000 2014-08-13 17:38:02.833505 7fe9bae887a0 20 read_log 194215'15741 (192834'15055) clean e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by osd.6.0:856 2014-08-12 15:39:20.472296 2014-08-13 17:38:02.833516 7fe9bae887a0 20 read_log 194215'15742 (194215'15741) delete e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.2174595.0:41950 0.000000 2014-08-13 17:38:02.833527 7fe9bae887a0 20 read_log 194215'15743 (193305'15600) clean 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by osd.6.0:858 2014-08-12 15:39:20.478681 2014-08-13 17:38:02.833539 7fe9bae887a0 20 read_log 194215'15744 (194215'15743) delete 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.2174595.0:41952 0.000000 2014-08-13 17:38:02.833550 7fe9bae887a0 20 read_log 194215'15745 (192888'15315) clean 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by osd.6.0:860 2014-08-12 15:39:20.485451 2014-08-13 17:38:02.833561 7fe9bae887a0 20 read_log 194215'15746 (194215'15745) delete 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.2174595.0:41954 0.000000 2014-08-13 17:38:02.833572 7fe9bae887a0 20 read_log 194215'15747 (192888'15442) clean 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by osd.6.0:862 2014-08-12 15:39:20.492013 2014-08-13 17:38:02.833584 7fe9bae887a0 20 read_log 194215'15748 (194215'15747) delete 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.2174595.0:41956 0.000000 2014-08-13 17:38:02.833594 7fe9bae887a0 20 read_log 194215'15749 (193305'15583) clean a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by osd.6.0:864 2014-08-12 15:39:20.498474 2014-08-13 17:38:02.833606 7fe9bae887a0 20 read_log 194215'15750 (194215'15749) delete a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.2174595.0:41958 0.000000 2014-08-13 17:38:02.833617 7fe9bae887a0 20 read_log 194215'15751 (192888'15319) clean 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by osd.6.0:866 2014-08-12 15:39:20.505107 2014-08-13 17:38:02.833629 7fe9bae887a0 20 read_log 194215'15752 (194215'15751) delete 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.2174595.0:41960 0.000000 2014-08-13 17:38:02.833639 7fe9bae887a0 20 read_log 194215'15753 (192888'15478) clean ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by osd.6.0:868 2014-08-12 15:39:20.511583 2014-08-13 17:38:02.833663 7fe9bae887a0 20 read_log 194215'15754 (194215'15753) delete ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.2174595.0:41962 0.000000 2014-08-13 17:38:02.833674 7fe9bae887a0 20 read_log 194215'15755 (192888'15454) clean 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by osd.6.0:870 2014-08-12 15:39:20.518087 2014-08-13 17:38:02.833702 7fe9bae887a0 20 read_log 194215'15756 (194215'15755) delete 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.2174595.0:41964 0.000000 2014-08-13 17:38:02.833713 7fe9bae887a0 20 read_log 194215'15757 (192888'15463) clean 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by osd.6.0:872 2014-08-12 15:39:20.524304 2014-08-13 17:38:02.833725 7fe9bae887a0 20 read_log 194215'15758 (194215'15757) delete 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.2174595.0:41966 0.000000 2014-08-13 17:38:02.833736 7fe9bae887a0 20 read_log 194215'15759 (192834'15116) clean e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by osd.6.0:874 2014-08-12 15:39:20.530543 2014-08-13 17:38:02.833748 7fe9bae887a0 20 read_log 194215'15760 (194215'15759) delete e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.2174595.0:41968 0.000000 2014-08-13 17:38:02.833759 7fe9bae887a0 20 read_log 194215'15761 (193305'15603) clean 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by osd.6.0:876 2014-08-12 15:39:20.537407 2014-08-13 17:38:02.833771 7fe9bae887a0 20 read_log 194215'15762 (194215'15761) delete 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.2174595.0:41970 0.000000 2014-08-13 17:38:02.833781 7fe9bae887a0 20 read_log 194215'15763 (192834'14480) clean fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by osd.6.0:878 2014-08-12 15:39:20.544151 2014-08-13 17:38:02.833793 7fe9bae887a0 20 read_log 194215'15764 (194215'15763) delete fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.2174595.0:41972 0.000000 2014-08-13 17:38:02.833804 7fe9bae887a0 20 read_log 194215'15765 (192834'15090) clean d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by osd.6.0:880 2014-08-12 15:39:20.550638 2014-08-13 17:38:02.833816 7fe9bae887a0 20 read_log 194215'15766 (194215'15765) delete d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.2174595.0:41974 0.000000 2014-08-13 17:38:02.833828 7fe9bae887a0 20 read_log 194215'15767 (192834'14641) clean e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by osd.6.0:882 2014-08-12 15:39:20.556965 2014-08-13 17:38:02.833840 7fe9bae887a0 20 read_log 194215'15768 (194215'15767) delete e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.2174595.0:41976 0.000000 2014-08-13 17:38:02.833851 7fe9bae887a0 20 read_log 194215'15769 (192888'15465) clean 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by osd.6.0:884 2014-08-12 15:39:20.563203 2014-08-13 17:38:02.833863 7fe9bae887a0 20 read_log 194215'15770 (194215'15769) delete 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.2174595.0:41978 0.000000 2014-08-13 17:38:02.833873 7fe9bae887a0 20 read_log 194215'15771 (193305'15573) clean e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by osd.6.0:886 2014-08-12 15:39:20.569936 2014-08-13 17:38:02.833885 7fe9bae887a0 20 read_log 194215'15772 (194215'15771) delete e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.2174595.0:41980 0.000000 2014-08-13 17:38:02.833896 7fe9bae887a0 20 read_log 194215'15773 (192888'15314) clean 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by osd.6.0:888 2014-08-12 15:39:20.576288 2014-08-13 17:38:02.833917 7fe9bae887a0 20 read_log 194215'15774 (194215'15773) delete 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.2174595.0:41982 0.000000 2014-08-13 17:38:02.833928 7fe9bae887a0 20 read_log 194215'15775 (192834'15064) clean b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by osd.6.0:890 2014-08-12 15:39:20.582848 2014-08-13 17:38:02.833941 7fe9bae887a0 20 read_log 194215'15776 (194215'15775) delete b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.2174595.0:41984 0.000000 2014-08-13 17:38:02.833951 7fe9bae887a0 20 read_log 194215'15777 (192834'15139) clean 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by osd.6.0:892 2014-08-12 15:39:20.589326 2014-08-13 17:38:02.833963 7fe9bae887a0 20 read_log 194215'15778 (194215'15777) delete 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.2174595.0:41986 0.000000 2014-08-13 17:38:02.833974 7fe9bae887a0 20 read_log 194215'15779 (192888'15476) clean d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by osd.6.0:894 2014-08-12 15:39:20.595548 2014-08-13 17:38:02.833986 7fe9bae887a0 20 read_log 194215'15780 (194215'15779) delete d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.2174595.0:41988 0.000000 2014-08-13 17:38:02.833996 7fe9bae887a0 20 read_log 194215'15781 (192834'15043) clean 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by osd.6.0:896 2014-08-12 15:39:20.602638 2014-08-13 17:38:02.834008 7fe9bae887a0 20 read_log 194215'15782 (194215'15781) delete 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.2174595.0:41990 0.000000 2014-08-13 17:38:02.834018 7fe9bae887a0 20 read_log 194215'15783 (193305'15574) clean 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by osd.6.0:898 2014-08-12 15:39:20.609498 2014-08-13 17:38:02.834030 7fe9bae887a0 20 read_log 194215'15784 (194215'15783) delete 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.2174595.0:41992 0.000000 2014-08-13 17:38:02.834040 7fe9bae887a0 20 read_log 194215'15785 (193305'15599) clean 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by osd.6.0:900 2014-08-12 15:39:20.615731 2014-08-13 17:38:02.834052 7fe9bae887a0 20 read_log 194215'15786 (194215'15785) delete 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.2174595.0:41994 0.000000 2014-08-13 17:38:02.834063 7fe9bae887a0 20 read_log 194215'15787 (193305'15594) clean 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by osd.6.0:902 2014-08-12 15:39:20.621844 2014-08-13 17:38:02.834074 7fe9bae887a0 20 read_log 194215'15788 (194215'15787) delete 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.2174595.0:41996 0.000000 2014-08-13 17:38:02.834085 7fe9bae887a0 20 read_log 194215'15789 (192834'15026) clean daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by osd.6.0:904 2014-08-12 15:39:20.628147 2014-08-13 17:38:02.834097 7fe9bae887a0 20 read_log 194215'15790 (194215'15789) delete daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.2174595.0:41998 0.000000 2014-08-13 17:38:02.834107 7fe9bae887a0 20 read_log 194215'15791 (192888'15445) clean 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by osd.6.0:906 2014-08-12 15:39:20.635094 2014-08-13 17:38:02.834120 7fe9bae887a0 20 read_log 194215'15792 (194215'15791) delete 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.2174595.0:42000 0.000000 2014-08-13 17:38:02.834131 7fe9bae887a0 20 read_log 194215'15793 (192834'15074) clean bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by osd.6.0:908 2014-08-12 15:39:20.641975 2014-08-13 17:38:02.834149 7fe9bae887a0 20 read_log 194215'15794 (194215'15793) delete bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.2174595.0:42002 0.000000 2014-08-13 17:38:02.834166 7fe9bae887a0 20 read_log 194215'15795 (192834'14597) clean 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by osd.6.0:910 2014-08-12 15:39:20.648350 2014-08-13 17:38:02.834178 7fe9bae887a0 20 read_log 194215'15796 (194215'15795) delete 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.2174595.0:42004 0.000000 2014-08-13 17:38:02.834189 7fe9bae887a0 20 read_log 194215'15797 (192888'15451) clean 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by osd.6.0:912 2014-08-12 15:39:20.654642 2014-08-13 17:38:02.834201 7fe9bae887a0 20 read_log 194215'15798 (194215'15797) delete 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.2174595.0:42006 0.000000 2014-08-13 17:38:02.834212 7fe9bae887a0 20 read_log 194215'15799 (192834'14995) clean 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by osd.6.0:914 2014-08-12 15:39:20.661160 2014-08-13 17:38:02.834223 7fe9bae887a0 20 read_log 194215'15800 (194215'15799) delete 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.2174595.0:42008 0.000000 2014-08-13 17:38:02.834234 7fe9bae887a0 20 read_log 194215'15801 (193305'15588) clean da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by osd.6.0:916 2014-08-12 15:39:20.667342 2014-08-13 17:38:02.834246 7fe9bae887a0 20 read_log 194215'15802 (194215'15801) delete da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.2174595.0:42010 0.000000 2014-08-13 17:38:02.834256 7fe9bae887a0 20 read_log 194215'15803 (192834'15108) clean 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by osd.6.0:918 2014-08-12 15:39:20.674483 2014-08-13 17:38:02.834268 7fe9bae887a0 20 read_log 194215'15804 (194215'15803) delete 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.2174595.0:42012 0.000000 2014-08-13 17:38:02.834279 7fe9bae887a0 20 read_log 194215'15805 (192834'14522) clean 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by osd.6.0:920 2014-08-12 15:39:20.681126 2014-08-13 17:38:02.834290 7fe9bae887a0 20 read_log 194215'15806 (194215'15805) delete 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.2174595.0:42014 0.000000 2014-08-13 17:38:02.834301 7fe9bae887a0 20 read_log 194215'15807 (193305'15593) clean 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by osd.6.0:922 2014-08-12 15:39:20.687532 2014-08-13 17:38:02.834315 7fe9bae887a0 20 read_log 194215'15808 (194215'15807) delete 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.2174595.0:42016 0.000000 2014-08-13 17:38:02.834326 7fe9bae887a0 20 read_log 194215'15809 (192834'15024) clean fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by osd.6.0:924 2014-08-12 15:39:20.694381 2014-08-13 17:38:02.834338 7fe9bae887a0 20 read_log 194215'15810 (194215'15809) delete fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.2174595.0:42018 0.000000 2014-08-13 17:38:02.834348 7fe9bae887a0 20 read_log 194215'15811 (192888'15458) clean 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by osd.6.0:926 2014-08-12 15:39:20.701257 2014-08-13 17:38:02.834360 7fe9bae887a0 20 read_log 194215'15812 (194215'15811) delete 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.2174595.0:42020 0.000000 2014-08-13 17:38:02.834371 7fe9bae887a0 20 read_log 194215'15813 (192834'15033) clean f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by osd.6.0:928 2014-08-12 15:39:20.708080 2014-08-13 17:38:02.834389 7fe9bae887a0 20 read_log 194215'15814 (194215'15813) delete f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.2174595.0:42022 0.000000 2014-08-13 17:38:02.834400 7fe9bae887a0 20 read_log 194215'15815 (192834'15021) clean 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by osd.6.0:930 2014-08-12 15:39:20.715286 2014-08-13 17:38:02.834412 7fe9bae887a0 20 read_log 194215'15816 (194215'15815) delete 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.2174595.0:42024 0.000000 2014-08-13 17:38:02.834423 7fe9bae887a0 20 read_log 194215'15817 (192888'15486) clean 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by osd.6.0:932 2014-08-12 15:39:20.721851 2014-08-13 17:38:02.834434 7fe9bae887a0 20 read_log 194215'15818 (194215'15817) delete 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.2174595.0:42026 0.000000 2014-08-13 17:38:02.834445 7fe9bae887a0 20 read_log 194215'15819 (192888'15441) clean 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by osd.6.0:934 2014-08-12 15:39:20.728952 2014-08-13 17:38:02.834456 7fe9bae887a0 20 read_log 194215'15820 (194215'15819) delete 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.2174595.0:42028 0.000000 2014-08-13 17:38:02.834467 7fe9bae887a0 20 read_log 194215'15821 (192888'15269) clean 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by osd.6.0:936 2014-08-12 15:39:20.735612 2014-08-13 17:38:02.834478 7fe9bae887a0 20 read_log 194215'15822 (194215'15821) delete 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.2174595.0:42030 0.000000 2014-08-13 17:38:02.834489 7fe9bae887a0 20 read_log 194215'15823 (192888'15436) clean d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by osd.6.0:938 2014-08-12 15:39:20.742274 2014-08-13 17:38:02.834501 7fe9bae887a0 20 read_log 194215'15824 (194215'15823) delete d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.2174595.0:42032 0.000000 2014-08-13 17:38:02.834511 7fe9bae887a0 20 read_log 194215'15825 (192888'15316) clean d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by osd.6.0:940 2014-08-12 15:39:20.749539 2014-08-13 17:38:02.834523 7fe9bae887a0 20 read_log 194215'15826 (194215'15825) delete d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.2174595.0:42034 0.000000 2014-08-13 17:38:02.834533 7fe9bae887a0 20 read_log 194215'15827 (192834'15062) clean 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by osd.6.0:942 2014-08-12 15:39:20.756527 2014-08-13 17:38:02.834545 7fe9bae887a0 20 read_log 194215'15828 (194215'15827) delete 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.2174595.0:42036 0.000000 2014-08-13 17:38:02.834556 7fe9bae887a0 20 read_log 194215'15829 (192834'14614) clean f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by osd.6.0:944 2014-08-12 15:39:20.763069 2014-08-13 17:38:02.834568 7fe9bae887a0 20 read_log 194215'15830 (194215'15829) delete f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.2174595.0:42038 0.000000 2014-08-13 17:38:02.834579 7fe9bae887a0 20 read_log 194215'15831 (192888'15305) clean 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by osd.6.0:946 2014-08-12 15:39:20.769957 2014-08-13 17:38:02.834591 7fe9bae887a0 20 read_log 194215'15832 (194215'15831) delete 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.2174595.0:42040 0.000000 2014-08-13 17:38:02.834601 7fe9bae887a0 20 read_log 194215'15833 (193305'15609) clean 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by osd.6.0:948 2014-08-12 15:39:20.777056 2014-08-13 17:38:02.834623 7fe9bae887a0 20 read_log 194215'15834 (194215'15833) delete 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.2174595.0:42042 0.000000 2014-08-13 17:38:02.834635 7fe9bae887a0 20 read_log 194215'15835 (192834'15099) clean ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by osd.6.0:950 2014-08-12 15:39:20.784025 2014-08-13 17:38:02.834647 7fe9bae887a0 20 read_log 194215'15836 (194215'15835) delete ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.2174595.0:42044 0.000000 2014-08-13 17:38:02.834658 7fe9bae887a0 20 read_log 194215'15837 (192888'15270) clean a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by osd.6.0:952 2014-08-12 15:39:20.806152 2014-08-13 17:38:02.834669 7fe9bae887a0 20 read_log 194215'15838 (194215'15837) delete a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.2174595.0:42046 0.000000 2014-08-13 17:38:02.834680 7fe9bae887a0 20 read_log 194215'15839 (192888'15294) clean f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by osd.6.0:954 2014-08-12 15:39:20.813777 2014-08-13 17:38:02.834692 7fe9bae887a0 20 read_log 194215'15840 (194215'15839) delete f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.2174595.0:42048 0.000000 2014-08-13 17:38:02.834702 7fe9bae887a0 20 read_log 194215'15841 (192834'15059) clean 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by osd.6.0:956 2014-08-12 15:39:20.820536 2014-08-13 17:38:02.834714 7fe9bae887a0 20 read_log 194215'15842 (194215'15841) delete 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.2174595.0:42050 0.000000 2014-08-13 17:38:02.834725 7fe9bae887a0 20 read_log 194215'15843 (192888'15490) clean 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by osd.6.0:958 2014-08-12 15:39:20.827925 2014-08-13 17:38:02.834737 7fe9bae887a0 20 read_log 194215'15844 (194215'15843) delete 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.2174595.0:42052 0.000000 2014-08-13 17:38:02.834747 7fe9bae887a0 20 read_log 194215'15845 (192834'15068) clean 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by osd.6.0:960 2014-08-12 15:39:20.834851 2014-08-13 17:38:02.834759 7fe9bae887a0 20 read_log 194215'15846 (194215'15845) delete 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.2174595.0:42054 0.000000 2014-08-13 17:38:02.834770 7fe9bae887a0 20 read_log 194215'15847 (192888'15464) clean 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by osd.6.0:962 2014-08-12 15:39:20.841695 2014-08-13 17:38:02.834781 7fe9bae887a0 20 read_log 194215'15848 (194215'15847) delete 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.2174595.0:42056 0.000000 2014-08-13 17:38:02.834793 7fe9bae887a0 20 read_log 194215'15849 (192834'14618) clean 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by osd.6.0:964 2014-08-12 15:39:20.848365 2014-08-13 17:38:02.834806 7fe9bae887a0 20 read_log 194215'15850 (194215'15849) delete 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.2174595.0:42058 0.000000 2014-08-13 17:38:02.834816 7fe9bae887a0 20 read_log 194215'15851 (192888'15469) clean 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by osd.6.0:966 2014-08-12 15:39:20.854815 2014-08-13 17:38:02.834828 7fe9bae887a0 20 read_log 194215'15852 (194215'15851) delete 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.2174595.0:42060 0.000000 2014-08-13 17:38:02.834839 7fe9bae887a0 20 read_log 194215'15853 (192888'15457) clean 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by osd.6.0:968 2014-08-12 15:39:20.861458 2014-08-13 17:38:02.834857 7fe9bae887a0 20 read_log 194215'15854 (194215'15853) delete 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.2174595.0:42062 0.000000 2014-08-13 17:38:02.834868 7fe9bae887a0 20 read_log 194215'15855 (193305'15587) clean 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by osd.6.0:970 2014-08-12 15:39:20.868723 2014-08-13 17:38:02.834882 7fe9bae887a0 20 read_log 194215'15856 (194215'15855) delete 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.2174595.0:42064 0.000000 2014-08-13 17:38:02.834892 7fe9bae887a0 20 read_log 194215'15857 (192888'15487) clean dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by osd.6.0:972 2014-08-12 15:39:20.875611 2014-08-13 17:38:02.834904 7fe9bae887a0 20 read_log 194215'15858 (194215'15857) delete dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.2174595.0:42066 0.000000 2014-08-13 17:38:02.834935 7fe9bae887a0 20 read_log 194215'15859 (192888'15317) clean ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by osd.6.0:974 2014-08-12 15:39:20.882409 2014-08-13 17:38:02.834948 7fe9bae887a0 20 read_log 194215'15860 (194215'15859) delete ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.2174595.0:42068 0.000000 2014-08-13 17:38:02.834959 7fe9bae887a0 20 read_log 194215'15861 (192888'15301) clean a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by osd.6.0:976 2014-08-12 15:39:20.888805 2014-08-13 17:38:02.834971 7fe9bae887a0 20 read_log 194215'15862 (194215'15861) delete a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.2174595.0:42070 0.000000 2014-08-13 17:38:02.834981 7fe9bae887a0 20 read_log 194215'15863 (193305'15576) clean f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by osd.6.0:978 2014-08-12 15:39:20.895309 2014-08-13 17:38:02.834999 7fe9bae887a0 20 read_log 194215'15864 (194215'15863) delete f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.2174595.0:42072 0.000000 2014-08-13 17:38:02.835010 7fe9bae887a0 20 read_log 194215'15865 (192888'15453) clean bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by osd.6.0:980 2014-08-12 15:39:20.902144 2014-08-13 17:38:02.835022 7fe9bae887a0 20 read_log 194215'15866 (194215'15865) delete bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.2174595.0:42074 0.000000 2014-08-13 17:38:02.835032 7fe9bae887a0 20 read_log 194215'15867 (193305'15584) clean dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by osd.6.0:982 2014-08-12 15:39:20.908753 2014-08-13 17:38:02.835044 7fe9bae887a0 20 read_log 194215'15868 (194215'15867) delete dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.2174595.0:42076 0.000000 2014-08-13 17:38:02.835055 7fe9bae887a0 20 read_log 194215'15869 (192834'15011) clean 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by osd.6.0:984 2014-08-12 15:39:20.915616 2014-08-13 17:38:02.835067 7fe9bae887a0 20 read_log 194215'15870 (194215'15869) delete 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.2174595.0:42078 0.000000 2014-08-13 17:38:02.835077 7fe9bae887a0 20 read_log 194215'15871 (192888'15485) clean 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by osd.6.0:986 2014-08-12 15:39:20.922440 2014-08-13 17:38:02.835091 7fe9bae887a0 20 read_log 194215'15872 (194215'15871) delete 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.2174595.0:42080 0.000000 2014-08-13 17:38:02.835102 7fe9bae887a0 20 read_log 194215'15873 (192834'15157) clean 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by osd.6.0:988 2014-08-12 15:39:20.929179 2014-08-13 17:38:02.835139 7fe9bae887a0 20 read_log 194215'15874 (194215'15873) delete 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.2174595.0:42082 0.000000 2014-08-13 17:38:02.835151 7fe9bae887a0 20 read_log 194215'15875 (192888'15283) clean e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by osd.6.0:990 2014-08-12 15:39:20.936228 2014-08-13 17:38:02.835168 7fe9bae887a0 20 read_log 194215'15876 (194215'15875) delete e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.2174595.0:42084 0.000000 2014-08-13 17:38:02.835179 7fe9bae887a0 20 read_log 194215'15877 (192834'14602) clean ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by osd.6.0:992 2014-08-12 15:39:20.942864 2014-08-13 17:38:02.835191 7fe9bae887a0 20 read_log 194215'15878 (194215'15877) delete ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.2174595.0:42086 0.000000 2014-08-13 17:38:02.835202 7fe9bae887a0 20 read_log 194215'15879 (193305'15601) clean aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by osd.6.0:994 2014-08-12 15:39:20.950045 2014-08-13 17:38:02.835214 7fe9bae887a0 20 read_log 194215'15880 (194215'15879) delete aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.2174595.0:42088 0.000000 2014-08-13 17:38:02.835224 7fe9bae887a0 20 read_log 194215'15881 (193305'15575) clean d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by osd.6.0:996 2014-08-12 15:39:20.957126 2014-08-13 17:38:02.835236 7fe9bae887a0 20 read_log 194215'15882 (194215'15881) delete d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.2174595.0:42090 0.000000 2014-08-13 17:38:02.835247 7fe9bae887a0 20 read_log 194215'15883 (192834'15057) clean 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by osd.6.0:998 2014-08-12 15:39:20.963782 2014-08-13 17:38:02.835259 7fe9bae887a0 20 read_log 194215'15884 (194215'15883) delete 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.2174595.0:42092 0.000000 2014-08-13 17:38:02.835269 7fe9bae887a0 20 read_log 194215'15885 (192834'14545) clean 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by osd.6.0:1000 2014-08-12 15:39:20.970054 2014-08-13 17:38:02.835284 7fe9bae887a0 20 read_log 194215'15886 (194215'15885) delete 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.2174595.0:42094 0.000000 2014-08-13 17:38:02.835295 7fe9bae887a0 20 read_log 194217'15887 (194217'15887) modify cd/hit_set_24.cd_archive_2014-08-12 15:39:20.336577_2014-08-12 17:25:05.320436/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 17:38:02.835308 7fe9bae887a0 20 read_log 194217'15888 (194215'15698) delete cd/hit_set_24.cd_archive_2014-08-12 15:32:25.874029_2014-08-12 15:39:20.336162/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 17:38:02.835320 7fe9bae887a0 20 read_log 194217'15889 (0'0) promote 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by osd.6.0:1568 2014-08-12 17:25:05.324213 2014-08-13 17:38:02.835332 7fe9bae887a0 20 read_log 194217'15890 (194217'15889) modify 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by client.1487927.0:412 2014-08-12 17:25:05.313391 2014-08-13 17:38:02.835344 7fe9bae887a0 20 read_log 194370'15891 (0'0) promote 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by osd.6.0:1599 2014-08-12 20:30:25.525467 2014-08-13 17:38:02.835356 7fe9bae887a0 20 read_log 194370'15892 (194370'15891) modify 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by client.1508531.0:2994 2014-08-12 20:30:25.517860 2014-08-13 17:38:02.835378 7fe9bae887a0 20 read_log 194370'15893 (0'0) promote f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by osd.6.0:1605 2014-08-12 20:30:28.438433 2014-08-13 17:38:02.835391 7fe9bae887a0 20 read_log 194370'15894 (194370'15893) modify f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by client.1508531.0:3765 2014-08-12 20:30:28.431666 2014-08-13 17:38:02.835403 7fe9bae887a0 20 read_log 194370'15895 (0'0) promote 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by osd.6.0:1623 2014-08-12 20:30:41.566511 2014-08-13 17:38:02.835415 7fe9bae887a0 20 read_log 194370'15896 (194370'15895) modify 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by client.1508531.0:6839 2014-08-12 20:30:41.557825 2014-08-13 17:38:02.835427 7fe9bae887a0 20 read_log 194370'15897 (0'0) promote 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by osd.6.0:1643 2014-08-12 20:30:58.540988 2014-08-13 17:38:02.835439 7fe9bae887a0 20 read_log 194370'15898 (194370'15897) modify 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by client.1508531.0:9874 2014-08-12 20:30:58.532474 2014-08-13 17:38:02.835451 7fe9bae887a0 20 read_log 194370'15899 (0'0) promote a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by osd.6.0:1647 2014-08-12 20:31:06.764327 2014-08-13 17:38:02.835463 7fe9bae887a0 20 read_log 194370'15900 (194370'15899) modify a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by client.1508531.0:11245 2014-08-12 20:31:06.755136 2014-08-13 17:38:02.835475 7fe9bae887a0 20 read_log 194370'15901 (0'0) promote a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by osd.6.0:1651 2014-08-12 20:31:10.396334 2014-08-13 17:38:02.835487 7fe9bae887a0 20 read_log 194370'15902 (194370'15901) modify a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by client.1508531.0:11854 2014-08-12 20:31:10.388241 2014-08-13 17:38:02.835499 7fe9bae887a0 20 read_log 194370'15903 (0'0) promote 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by osd.6.0:1665 2014-08-12 20:31:23.235796 2014-08-13 17:38:02.835510 7fe9bae887a0 20 read_log 194370'15904 (194370'15903) modify 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by client.1508531.0:13405 2014-08-12 20:31:23.228485 2014-08-13 17:38:02.835527 7fe9bae887a0 20 read_log 194377'15905 (0'0) promote e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by osd.6.0:1671 2014-08-12 20:32:58.710442 2014-08-13 17:38:02.835539 7fe9bae887a0 20 read_log 194377'15906 (194377'15905) modify e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by client.2231190.0:361 2014-08-12 20:32:58.702360 2014-08-13 17:38:02.835551 7fe9bae887a0 20 read_log 194377'15907 (0'0) promote 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by osd.6.0:1673 2014-08-12 20:33:00.037487 2014-08-13 17:38:02.835563 7fe9bae887a0 20 read_log 194377'15908 (194377'15907) modify 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by client.2231190.0:666 2014-08-12 20:33:00.029776 2014-08-13 17:38:02.835575 7fe9bae887a0 20 read_log 194377'15909 (0'0) promote b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by osd.6.0:1683 2014-08-12 20:33:06.009473 2014-08-13 17:38:02.835588 7fe9bae887a0 20 read_log 194377'15910 (194377'15909) modify b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by client.2231190.0:2186 2014-08-12 20:33:06.003220 2014-08-13 17:38:02.835600 7fe9bae887a0 20 read_log 194377'15911 (0'0) promote 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by osd.6.0:1685 2014-08-12 20:33:06.471091 2014-08-13 17:38:02.835612 7fe9bae887a0 20 read_log 194377'15912 (194377'15911) modify 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by client.2231190.0:2302 2014-08-12 20:33:06.464047 2014-08-13 17:38:02.835631 7fe9bae887a0 20 read_log 194377'15913 (0'0) promote 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by osd.6.0:1701 2014-08-12 20:33:09.365716 2014-08-13 17:38:02.835643 7fe9bae887a0 20 read_log 194377'15914 (194377'15913) modify 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by client.2231190.0:3007 2014-08-12 20:33:09.359555 2014-08-13 17:38:02.835655 7fe9bae887a0 20 read_log 194377'15915 (0'0) promote a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by osd.6.0:1711 2014-08-12 20:33:11.084556 2014-08-13 17:38:02.835667 7fe9bae887a0 20 read_log 194377'15916 (194377'15915) modify a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by client.2231190.0:3446 2014-08-12 20:33:11.076919 2014-08-13 17:38:02.835680 7fe9bae887a0 20 read_log 194377'15917 (0'0) promote 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by osd.6.0:1713 2014-08-12 20:33:11.599861 2014-08-13 17:38:02.835691 7fe9bae887a0 20 read_log 194377'15918 (194377'15917) modify 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by client.2231190.0:3547 2014-08-12 20:33:11.591730 2014-08-13 17:38:02.835704 7fe9bae887a0 20 read_log 194377'15919 (0'0) promote 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by osd.6.0:1721 2014-08-12 20:33:16.240465 2014-08-13 17:38:02.835715 7fe9bae887a0 20 read_log 194377'15920 (194377'15919) modify 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by client.2231190.0:4376 2014-08-12 20:33:16.232582 2014-08-13 17:38:02.835728 7fe9bae887a0 20 read_log 194380'15921 (0'0) promote 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by osd.6.0:1729 2014-08-12 20:33:27.663787 2014-08-13 17:38:02.835741 7fe9bae887a0 20 read_log 194380'15922 (194380'15921) modify 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by client.2231190.0:5337 2014-08-12 20:33:27.655777 2014-08-13 17:38:02.835753 7fe9bae887a0 20 read_log 194383'15923 (0'0) promote bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by osd.6.0:1735 2014-08-12 20:34:06.936028 2014-08-13 17:38:02.835765 7fe9bae887a0 20 read_log 194383'15924 (194383'15923) modify bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by client.2231190.0:6691 2014-08-12 20:34:06.927281 2014-08-13 17:38:02.835777 7fe9bae887a0 20 read_log 194383'15925 (0'0) promote 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by osd.6.0:1737 2014-08-12 20:34:07.672346 2014-08-13 17:38:02.835789 7fe9bae887a0 20 read_log 194383'15926 (194383'15925) modify 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by client.2231190.0:6724 2014-08-12 20:34:07.664179 2014-08-13 17:38:02.835801 7fe9bae887a0 20 read_log 194393'15927 (0'0) promote 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by osd.6.0:1751 2014-08-12 20:49:26.130157 2014-08-13 17:38:02.835813 7fe9bae887a0 20 read_log 194393'15928 (194393'15927) modify 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by client.1064182.0:140 2014-08-12 20:49:26.124270 2014-08-13 17:38:02.835825 7fe9bae887a0 20 read_log 194416'15929 (0'0) promote ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by osd.6.0:1880 2014-08-13 15:28:31.344921 2014-08-13 17:38:02.835837 7fe9bae887a0 20 read_log 194416'15930 (194416'15929) modify ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by client.2441958.0:279 2014-08-13 15:28:31.335616 2014-08-13 17:38:02.835849 7fe9bae887a0 20 read_log 194416'15931 (0'0) promote 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by osd.6.0:1885 2014-08-13 15:28:33.027558 2014-08-13 17:38:02.835872 7fe9bae887a0 20 read_log 194416'15932 (194416'15931) modify 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by client.2441958.0:639 2014-08-13 15:28:33.018056 2014-08-13 17:38:02.835886 7fe9bae887a0 20 read_log 194416'15933 (0'0) promote 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by osd.6.0:1887 2014-08-13 15:28:34.899061 2014-08-13 17:38:02.835898 7fe9bae887a0 20 read_log 194416'15934 (194416'15933) modify 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by client.2441958.0:1101 2014-08-13 15:28:34.891565 2014-08-13 17:38:02.835910 7fe9bae887a0 20 read_log 194416'15935 (0'0) promote affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by osd.6.0:1897 2014-08-13 15:28:42.137168 2014-08-13 17:38:02.835922 7fe9bae887a0 20 read_log 194416'15936 (194416'15935) modify affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by client.2441958.0:3017 2014-08-13 15:28:42.129743 2014-08-13 17:38:02.835935 7fe9bae887a0 20 read_log 194416'15937 (0'0) promote 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by osd.6.0:1909 2014-08-13 15:28:49.814240 2014-08-13 17:38:02.835947 7fe9bae887a0 20 read_log 194416'15938 (194416'15937) modify 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by client.2441958.0:4978 2014-08-13 15:28:49.805250 2014-08-13 17:38:02.835959 7fe9bae887a0 20 read_log 194416'15939 (0'0) promote e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by osd.6.0:1913 2014-08-13 15:28:51.454527 2014-08-13 17:38:02.835971 7fe9bae887a0 20 read_log 194416'15940 (194416'15939) modify e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by client.2441958.0:5409 2014-08-13 15:28:51.445994 2014-08-13 17:38:02.835983 7fe9bae887a0 20 read_log 194416'15941 (0'0) promote abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by osd.6.0:1917 2014-08-13 15:28:55.148821 2014-08-13 17:38:02.835994 7fe9bae887a0 20 read_log 194416'15942 (194416'15941) modify abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by client.2441958.0:6304 2014-08-13 15:28:55.140350 2014-08-13 17:38:02.836006 7fe9bae887a0 20 read_log 194416'15943 (0'0) promote aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by osd.6.0:1929 2014-08-13 15:29:02.033402 2014-08-13 17:38:02.836018 7fe9bae887a0 20 read_log 194416'15944 (194416'15943) modify aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by client.2441958.0:8132 2014-08-13 15:29:02.024936 2014-08-13 17:38:02.836030 7fe9bae887a0 20 read_log 194416'15945 (0'0) promote b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by osd.6.0:1933 2014-08-13 15:29:03.462587 2014-08-13 17:38:02.836042 7fe9bae887a0 20 read_log 194416'15946 (194416'15945) modify b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by client.2441958.0:8512 2014-08-13 15:29:03.456402 2014-08-13 17:38:02.836054 7fe9bae887a0 20 read_log 194416'15947 (0'0) promote a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by osd.6.0:1935 2014-08-13 15:29:03.494696 2014-08-13 17:38:02.836066 7fe9bae887a0 20 read_log 194416'15948 (194416'15947) modify a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by client.2441958.0:8521 2014-08-13 15:29:03.488161 2014-08-13 17:38:02.836078 7fe9bae887a0 20 read_log 194416'15949 (0'0) promote 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by osd.6.0:1937 2014-08-13 15:29:03.782091 2014-08-13 17:38:02.836090 7fe9bae887a0 20 read_log 194416'15950 (194416'15949) modify 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by client.2441958.0:8598 2014-08-13 15:29:03.773943 2014-08-13 17:38:02.836102 7fe9bae887a0 20 read_log 194416'15951 (0'0) promote dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by osd.6.0:1941 2014-08-13 15:29:06.950672 2014-08-13 17:38:02.836141 7fe9bae887a0 20 read_log 194416'15952 (194416'15951) modify dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by client.2441958.0:9388 2014-08-13 15:29:06.942961 2014-08-13 17:38:02.836162 7fe9bae887a0 20 read_log 194416'15953 (0'0) promote 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by osd.6.0:1943 2014-08-13 15:29:10.732496 2014-08-13 17:38:02.836175 7fe9bae887a0 20 read_log 194416'15954 (194416'15953) modify 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by client.2441958.0:10313 2014-08-13 15:29:10.724222 2014-08-13 17:38:02.836188 7fe9bae887a0 20 read_log 194416'15955 (0'0) promote 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by osd.6.0:1945 2014-08-13 15:29:10.911215 2014-08-13 17:38:02.836201 7fe9bae887a0 20 read_log 194416'15956 (194416'15955) modify 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by client.2441958.0:10359 2014-08-13 15:29:10.901520 2014-08-13 17:38:02.836212 7fe9bae887a0 20 read_log 194416'15957 (0'0) promote d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by osd.6.0:1947 2014-08-13 15:29:11.147372 2014-08-13 17:38:02.836224 7fe9bae887a0 20 read_log 194416'15958 (194416'15957) modify d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by client.2441958.0:10422 2014-08-13 15:29:11.139455 2014-08-13 17:38:02.836236 7fe9bae887a0 20 read_log 194416'15959 (0'0) promote 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by osd.6.0:1953 2014-08-13 15:29:16.798127 2014-08-13 17:38:02.836248 7fe9bae887a0 20 read_log 194416'15960 (194416'15959) modify 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by client.2441958.0:11823 2014-08-13 15:29:16.790522 2014-08-13 17:38:02.836260 7fe9bae887a0 20 read_log 194416'15961 (0'0) promote 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by osd.6.0:1975 2014-08-13 15:29:34.201290 2014-08-13 17:38:02.836272 7fe9bae887a0 20 read_log 194416'15962 (194416'15961) modify 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by client.2441958.0:15771 2014-08-13 15:29:34.192971 2014-08-13 17:38:02.836284 7fe9bae887a0 20 read_log 194416'15963 (0'0) promote dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by osd.6.0:1977 2014-08-13 15:29:34.431122 2014-08-13 17:38:02.836295 7fe9bae887a0 20 read_log 194416'15964 (194416'15963) modify dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by client.2441958.0:15810 2014-08-13 15:29:34.422385 2014-08-13 17:38:02.836308 7fe9bae887a0 20 read_log 194416'15965 (0'0) promote 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by osd.6.0:1987 2014-08-13 15:29:58.847379 2014-08-13 17:38:02.836319 7fe9bae887a0 20 read_log 194416'15966 (194416'15965) modify 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by client.2441958.0:17816 2014-08-13 15:29:58.837982 2014-08-13 17:38:02.836331 7fe9bae887a0 20 read_log 194416'15967 (0'0) promote 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by osd.6.0:1989 2014-08-13 15:30:01.417168 2014-08-13 17:38:02.836343 7fe9bae887a0 20 read_log 194416'15968 (194416'15967) modify 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by client.2441958.0:17996 2014-08-13 15:30:01.408574 2014-08-13 17:38:02.836356 7fe9bae887a0 20 read_log 194422'15969 (0'0) promote 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by osd.6.0:1993 2014-08-13 15:32:20.951313 2014-08-13 17:38:02.836368 7fe9bae887a0 20 read_log 194422'15970 (194422'15969) modify 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by client.1148203.0:1421 2014-08-13 15:32:20.943748 2014-08-13 17:38:02.836386 7fe9bae887a0 20 read_log 194422'15971 (0'0) promote bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by osd.6.0:1999 2014-08-13 15:32:25.701996 2014-08-13 17:38:02.836399 7fe9bae887a0 20 read_log 194422'15972 (194422'15971) modify bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by client.1148203.0:2648 2014-08-13 15:32:25.694476 2014-08-13 17:38:02.836411 7fe9bae887a0 20 read_log 194422'15973 (0'0) promote b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by osd.6.0:2003 2014-08-13 15:32:26.834606 2014-08-13 17:38:02.836423 7fe9bae887a0 20 read_log 194422'15974 (194422'15973) modify b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by client.1148203.0:2868 2014-08-13 15:32:26.586165 2014-08-13 17:38:02.836435 7fe9bae887a0 20 read_log 194422'15975 (0'0) promote a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by osd.6.0:2009 2014-08-13 15:32:34.957558 2014-08-13 17:38:02.836447 7fe9bae887a0 20 read_log 194422'15976 (194422'15975) modify a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by client.1148203.0:5028 2014-08-13 15:32:34.948621 2014-08-13 17:38:02.836460 7fe9bae887a0 20 read_log 194422'15977 (0'0) promote 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by osd.6.0:2013 2014-08-13 15:32:38.313723 2014-08-13 17:38:02.836473 7fe9bae887a0 20 read_log 194422'15978 (194422'15977) modify 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by client.1148203.0:5665 2014-08-13 15:32:38.306136 2014-08-13 17:38:02.836485 7fe9bae887a0 20 read_log 194422'15979 (0'0) promote f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by osd.6.0:2025 2014-08-13 15:32:49.374130 2014-08-13 17:38:02.836497 7fe9bae887a0 20 read_log 194422'15980 (194422'15979) modify f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by client.1148203.0:7622 2014-08-13 15:32:49.364629 2014-08-13 17:38:02.836509 7fe9bae887a0 20 read_log 194422'15981 (0'0) promote 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by osd.6.0:2031 2014-08-13 15:32:52.463487 2014-08-13 17:38:02.836521 7fe9bae887a0 20 read_log 194422'15982 (194422'15981) modify 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by client.1148203.0:8117 2014-08-13 15:32:52.454672 2014-08-13 17:38:02.836533 7fe9bae887a0 20 read_log 194422'15983 (0'0) promote bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by osd.6.0:2035 2014-08-13 15:32:57.693618 2014-08-13 17:38:02.836544 7fe9bae887a0 20 read_log 194422'15984 (194422'15983) modify bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by client.1148203.0:8807 2014-08-13 15:32:57.684987 2014-08-13 17:38:02.836556 7fe9bae887a0 20 read_log 194422'15985 (0'0) promote 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by osd.6.0:2037 2014-08-13 15:33:01.164622 2014-08-13 17:38:02.836568 7fe9bae887a0 20 read_log 194422'15986 (194422'15985) modify 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by client.1148203.0:9164 2014-08-13 15:33:01.155232 2014-08-13 17:38:02.836581 7fe9bae887a0 20 read_log 194422'15987 (0'0) promote 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by osd.6.0:2039 2014-08-13 15:33:07.403937 2014-08-13 17:38:02.836592 7fe9bae887a0 20 read_log 194422'15988 (194422'15987) modify 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by client.1148203.0:9769 2014-08-13 15:33:07.394203 2014-08-13 17:38:02.836605 7fe9bae887a0 20 read_log 194422'15989 (0'0) promote acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by osd.6.0:2041 2014-08-13 15:33:12.330934 2014-08-13 17:38:02.836616 7fe9bae887a0 20 read_log 194422'15990 (194422'15989) modify acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by client.1148203.0:10083 2014-08-13 15:33:12.322811 2014-08-13 17:38:02.836636 7fe9bae887a0 20 read_log 194422'15991 (0'0) promote ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by osd.6.0:2043 2014-08-13 15:33:19.817059 2014-08-13 17:38:02.836650 7fe9bae887a0 20 read_log 194422'15992 (194422'15991) modify ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by client.1148203.0:10351 2014-08-13 15:33:19.809562 2014-08-13 17:38:02.836666 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.837255 7fe9bae887a0 10 read_log checking for missing items over interval (194215'15886,194422'15992] 2014-08-13 17:38:02.837312 7fe9bae887a0 15 read_log missing 194422'15992 (194422'15991) modify ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by client.1148203.0:10351 2014-08-13 15:33:19.809562 2014-08-13 17:38:02.837360 7fe9bae887a0 15 read_log missing 194422'15990 (194422'15989) modify acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by client.1148203.0:10083 2014-08-13 15:33:12.322811 2014-08-13 17:38:02.837404 7fe9bae887a0 15 read_log missing 194422'15988 (194422'15987) modify 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by client.1148203.0:9769 2014-08-13 15:33:07.394203 2014-08-13 17:38:02.837442 7fe9bae887a0 15 read_log missing 194422'15986 (194422'15985) modify 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by client.1148203.0:9164 2014-08-13 15:33:01.155232 2014-08-13 17:38:02.837478 7fe9bae887a0 15 read_log missing 194422'15984 (194422'15983) modify bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by client.1148203.0:8807 2014-08-13 15:32:57.684987 2014-08-13 17:38:02.837514 7fe9bae887a0 15 read_log missing 194422'15982 (194422'15981) modify 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by client.1148203.0:8117 2014-08-13 15:32:52.454672 2014-08-13 17:38:02.837550 7fe9bae887a0 15 read_log missing 194422'15980 (194422'15979) modify f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by client.1148203.0:7622 2014-08-13 15:32:49.364629 2014-08-13 17:38:02.837587 7fe9bae887a0 15 read_log missing 194422'15978 (194422'15977) modify 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by client.1148203.0:5665 2014-08-13 15:32:38.306136 2014-08-13 17:38:02.837623 7fe9bae887a0 15 read_log missing 194422'15976 (194422'15975) modify a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by client.1148203.0:5028 2014-08-13 15:32:34.948621 2014-08-13 17:38:02.837660 7fe9bae887a0 15 read_log missing 194422'15974 (194422'15973) modify b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by client.1148203.0:2868 2014-08-13 15:32:26.586165 2014-08-13 17:38:02.837697 7fe9bae887a0 15 read_log missing 194422'15972 (194422'15971) modify bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by client.1148203.0:2648 2014-08-13 15:32:25.694476 2014-08-13 17:38:02.837734 7fe9bae887a0 15 read_log missing 194422'15970 (194422'15969) modify 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by client.1148203.0:1421 2014-08-13 15:32:20.943748 2014-08-13 17:38:02.837770 7fe9bae887a0 15 read_log missing 194416'15968 (194416'15967) modify 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by client.2441958.0:17996 2014-08-13 15:30:01.408574 2014-08-13 17:38:02.837807 7fe9bae887a0 15 read_log missing 194416'15966 (194416'15965) modify 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by client.2441958.0:17816 2014-08-13 15:29:58.837982 2014-08-13 17:38:02.837843 7fe9bae887a0 15 read_log missing 194416'15964 (194416'15963) modify dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by client.2441958.0:15810 2014-08-13 15:29:34.422385 2014-08-13 17:38:02.837879 7fe9bae887a0 15 read_log missing 194416'15962 (194416'15961) modify 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by client.2441958.0:15771 2014-08-13 15:29:34.192971 2014-08-13 17:38:02.837916 7fe9bae887a0 15 read_log missing 194416'15960 (194416'15959) modify 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by client.2441958.0:11823 2014-08-13 15:29:16.790522 2014-08-13 17:38:02.837952 7fe9bae887a0 15 read_log missing 194416'15958 (194416'15957) modify d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by client.2441958.0:10422 2014-08-13 15:29:11.139455 2014-08-13 17:38:02.837989 7fe9bae887a0 15 read_log missing 194416'15956 (194416'15955) modify 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by client.2441958.0:10359 2014-08-13 15:29:10.901520 2014-08-13 17:38:02.838025 7fe9bae887a0 15 read_log missing 194416'15954 (194416'15953) modify 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by client.2441958.0:10313 2014-08-13 15:29:10.724222 2014-08-13 17:38:02.838062 7fe9bae887a0 15 read_log missing 194416'15952 (194416'15951) modify dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by client.2441958.0:9388 2014-08-13 15:29:06.942961 2014-08-13 17:38:02.838098 7fe9bae887a0 15 read_log missing 194416'15950 (194416'15949) modify 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by client.2441958.0:8598 2014-08-13 15:29:03.773943 2014-08-13 17:38:02.838134 7fe9bae887a0 15 read_log missing 194416'15948 (194416'15947) modify a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by client.2441958.0:8521 2014-08-13 15:29:03.488161 2014-08-13 17:38:02.838176 7fe9bae887a0 15 read_log missing 194416'15946 (194416'15945) modify b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by client.2441958.0:8512 2014-08-13 15:29:03.456402 2014-08-13 17:38:02.838213 7fe9bae887a0 15 read_log missing 194416'15944 (194416'15943) modify aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by client.2441958.0:8132 2014-08-13 15:29:02.024936 2014-08-13 17:38:02.838250 7fe9bae887a0 15 read_log missing 194416'15942 (194416'15941) modify abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by client.2441958.0:6304 2014-08-13 15:28:55.140350 2014-08-13 17:38:02.838286 7fe9bae887a0 15 read_log missing 194416'15940 (194416'15939) modify e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by client.2441958.0:5409 2014-08-13 15:28:51.445994 2014-08-13 17:38:02.838322 7fe9bae887a0 15 read_log missing 194416'15938 (194416'15937) modify 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by client.2441958.0:4978 2014-08-13 15:28:49.805250 2014-08-13 17:38:02.838358 7fe9bae887a0 15 read_log missing 194416'15936 (194416'15935) modify affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by client.2441958.0:3017 2014-08-13 15:28:42.129743 2014-08-13 17:38:02.838394 7fe9bae887a0 15 read_log missing 194416'15934 (194416'15933) modify 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by client.2441958.0:1101 2014-08-13 15:28:34.891565 2014-08-13 17:38:02.838431 7fe9bae887a0 15 read_log missing 194416'15932 (194416'15931) modify 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by client.2441958.0:639 2014-08-13 15:28:33.018056 2014-08-13 17:38:02.838467 7fe9bae887a0 15 read_log missing 194416'15930 (194416'15929) modify ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by client.2441958.0:279 2014-08-13 15:28:31.335616 2014-08-13 17:38:02.838503 7fe9bae887a0 15 read_log missing 194393'15928 (194393'15927) modify 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by client.1064182.0:140 2014-08-12 20:49:26.124270 2014-08-13 17:38:02.838540 7fe9bae887a0 15 read_log missing 194383'15926 (194383'15925) modify 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by client.2231190.0:6724 2014-08-12 20:34:07.664179 2014-08-13 17:38:02.838576 7fe9bae887a0 15 read_log missing 194383'15924 (194383'15923) modify bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by client.2231190.0:6691 2014-08-12 20:34:06.927281 2014-08-13 17:38:02.838612 7fe9bae887a0 15 read_log missing 194380'15922 (194380'15921) modify 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by client.2231190.0:5337 2014-08-12 20:33:27.655777 2014-08-13 17:38:02.838649 7fe9bae887a0 15 read_log missing 194377'15920 (194377'15919) modify 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by client.2231190.0:4376 2014-08-12 20:33:16.232582 2014-08-13 17:38:02.838685 7fe9bae887a0 15 read_log missing 194377'15918 (194377'15917) modify 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by client.2231190.0:3547 2014-08-12 20:33:11.591730 2014-08-13 17:38:02.838721 7fe9bae887a0 15 read_log missing 194377'15916 (194377'15915) modify a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by client.2231190.0:3446 2014-08-12 20:33:11.076919 2014-08-13 17:38:02.838757 7fe9bae887a0 15 read_log missing 194377'15914 (194377'15913) modify 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by client.2231190.0:3007 2014-08-12 20:33:09.359555 2014-08-13 17:38:02.838793 7fe9bae887a0 15 read_log missing 194377'15912 (194377'15911) modify 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by client.2231190.0:2302 2014-08-12 20:33:06.464047 2014-08-13 17:38:02.838829 7fe9bae887a0 15 read_log missing 194377'15910 (194377'15909) modify b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by client.2231190.0:2186 2014-08-12 20:33:06.003220 2014-08-13 17:38:02.838865 7fe9bae887a0 15 read_log missing 194377'15908 (194377'15907) modify 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by client.2231190.0:666 2014-08-12 20:33:00.029776 2014-08-13 17:38:02.838902 7fe9bae887a0 15 read_log missing 194377'15906 (194377'15905) modify e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by client.2231190.0:361 2014-08-12 20:32:58.702360 2014-08-13 17:38:02.838938 7fe9bae887a0 15 read_log missing 194370'15904 (194370'15903) modify 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by client.1508531.0:13405 2014-08-12 20:31:23.228485 2014-08-13 17:38:02.838974 7fe9bae887a0 15 read_log missing 194370'15902 (194370'15901) modify a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by client.1508531.0:11854 2014-08-12 20:31:10.388241 2014-08-13 17:38:02.839011 7fe9bae887a0 15 read_log missing 194370'15900 (194370'15899) modify a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by client.1508531.0:11245 2014-08-12 20:31:06.755136 2014-08-13 17:38:02.839047 7fe9bae887a0 15 read_log missing 194370'15898 (194370'15897) modify 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by client.1508531.0:9874 2014-08-12 20:30:58.532474 2014-08-13 17:38:02.839090 7fe9bae887a0 15 read_log missing 194370'15896 (194370'15895) modify 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by client.1508531.0:6839 2014-08-12 20:30:41.557825 2014-08-13 17:38:02.839127 7fe9bae887a0 15 read_log missing 194370'15894 (194370'15893) modify f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by client.1508531.0:3765 2014-08-12 20:30:28.431666 2014-08-13 17:38:02.839168 7fe9bae887a0 15 read_log missing 194370'15892 (194370'15891) modify 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by client.1508531.0:2994 2014-08-12 20:30:25.517860 2014-08-13 17:38:02.839206 7fe9bae887a0 15 read_log missing 194217'15890 (194217'15889) modify 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by client.1487927.0:412 2014-08-12 17:25:05.313391 2014-08-13 17:38:02.839247 7fe9bae887a0 15 read_log missing 194217'15887 (194217'15887) modify cd/hit_set_24.cd_archive_2014-08-12 15:39:20.336577_2014-08-12 17:25:05.320436/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 17:38:02.839262 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.839298 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive m=53] handle_loaded 2014-08-13 17:38:02.839316 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Initial 0.021262 0 0.000000 2014-08-13 17:38:02.839328 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset 2014-08-13 17:38:02.839336 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194477 2014-08-13 17:38:02.839346 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log((192718'12885,194422'15992], crt=194422'15992) 2014-08-13 17:38:02.839360 7fe9bae887a0 10 osd.4 194478 pgid 24.e2 coll 24.e2_head 2014-08-13 17:38:02.839502 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.e2 2014-08-13 17:38:02.839516 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.839548 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.e2(unlocked)] enter Initial 2014-08-13 17:38:02.839559 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.e2(unlocked)] enter NotTrimming 2014-08-13 17:38:02.839695 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.839805 7fe9bae887a0 20 read_log 192834'11894 (192834'11881) delete 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:12720 2014-08-11 15:24:42.480271 2014-08-13 17:38:02.839826 7fe9bae887a0 20 read_log 192834'11895 (192834'11882) delete 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:12721 2014-08-11 15:24:42.481508 2014-08-13 17:38:02.839839 7fe9bae887a0 20 read_log 192834'11901 (0'0) promote f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:12727 2014-08-11 15:24:46.244603 2014-08-13 17:38:02.839852 7fe9bae887a0 20 read_log 192834'11902 (192834'11901) modify f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by client.933448.0:14578 2014-08-11 15:24:46.234876 2014-08-13 17:38:02.839864 7fe9bae887a0 20 read_log 192834'11920 (192834'11669) clean 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:12746 2014-08-11 15:24:48.012738 2014-08-13 17:38:02.839876 7fe9bae887a0 20 read_log 192834'11922 (192834'11675) clean f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:12748 2014-08-11 15:24:48.337400 2014-08-13 17:38:02.839889 7fe9bae887a0 20 read_log 192834'11923 (192834'11677) clean ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:12749 2014-08-11 15:24:48.337629 2014-08-13 17:38:02.839910 7fe9bae887a0 20 read_log 192834'11933 (192834'11920) delete 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:12759 2014-08-11 15:24:48.365401 2014-08-13 17:38:02.839925 7fe9bae887a0 20 read_log 192834'11934 (192834'11922) delete f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:12760 2014-08-11 15:24:48.367939 2014-08-13 17:38:02.839937 7fe9bae887a0 20 read_log 192834'11935 (192834'11923) delete ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:12761 2014-08-11 15:24:48.370644 2014-08-13 17:38:02.839949 7fe9bae887a0 20 read_log 192834'11948 (0'0) promote 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:12774 2014-08-11 15:24:51.148696 2014-08-13 17:38:02.839962 7fe9bae887a0 20 read_log 192834'11949 (192834'11948) modify 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by client.933448.0:15763 2014-08-11 15:24:51.138951 2014-08-13 17:38:02.839974 7fe9bae887a0 20 read_log 192834'11965 (192834'11685) clean 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:12791 2014-08-11 15:24:53.645520 2014-08-13 17:38:02.839986 7fe9bae887a0 20 read_log 192834'11967 (192834'11693) clean 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:12793 2014-08-11 15:24:53.720878 2014-08-13 17:38:02.839998 7fe9bae887a0 20 read_log 192834'11975 (192834'11965) delete 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:12801 2014-08-11 15:24:53.747375 2014-08-13 17:38:02.840010 7fe9bae887a0 20 read_log 192834'11977 (192834'11967) delete 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:12803 2014-08-11 15:24:53.752224 2014-08-13 17:38:02.840022 7fe9bae887a0 20 read_log 192834'11981 (192834'11691) clean 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:12807 2014-08-11 15:24:53.775538 2014-08-13 17:38:02.840034 7fe9bae887a0 20 read_log 192834'11989 (0'0) promote cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:12815 2014-08-11 15:24:56.112339 2014-08-13 17:38:02.840046 7fe9bae887a0 20 read_log 192834'11990 (192834'11989) modify cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by client.933448.0:16529 2014-08-11 15:24:56.102816 2014-08-13 17:38:02.840058 7fe9bae887a0 20 read_log 192834'12008 (192834'11981) delete 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:12834 2014-08-11 15:24:59.169597 2014-08-13 17:38:02.840070 7fe9bae887a0 20 read_log 192834'12026 (0'0) promote e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:12852 2014-08-11 15:25:01.626757 2014-08-13 17:38:02.840082 7fe9bae887a0 20 read_log 192834'12027 (192834'12026) modify e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by client.933448.0:17788 2014-08-11 15:25:01.617803 2014-08-13 17:38:02.840118 7fe9bae887a0 20 read_log 192834'12098 (192834'11777) clean b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:12924 2014-08-11 15:25:15.169008 2014-08-13 17:38:02.840130 7fe9bae887a0 20 read_log 192834'12106 (192834'12098) delete b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:12932 2014-08-11 15:25:15.196894 2014-08-13 17:38:02.840142 7fe9bae887a0 20 read_log 192834'12110 (192834'11771) clean c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:12936 2014-08-11 15:25:15.233747 2014-08-13 17:38:02.840163 7fe9bae887a0 20 read_log 192834'12111 (192834'11767) clean 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:12937 2014-08-11 15:25:15.242215 2014-08-13 17:38:02.840178 7fe9bae887a0 20 read_log 192834'12127 (192834'12110) delete c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:12953 2014-08-11 15:25:20.607061 2014-08-13 17:38:02.840201 7fe9bae887a0 20 read_log 192834'12128 (192834'12111) delete 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:12954 2014-08-11 15:25:20.609450 2014-08-13 17:38:02.840216 7fe9bae887a0 20 read_log 192834'12138 (0'0) promote 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:12964 2014-08-11 15:25:20.825324 2014-08-13 17:38:02.840229 7fe9bae887a0 20 read_log 192834'12139 (192834'12138) modify 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by client.933448.0:20412 2014-08-11 15:25:20.815098 2014-08-13 17:38:02.840241 7fe9bae887a0 20 read_log 192834'12162 (192834'11811) clean 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:12988 2014-08-11 15:25:26.032514 2014-08-13 17:38:02.840253 7fe9bae887a0 20 read_log 192834'12164 (192834'11809) clean dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:12990 2014-08-11 15:25:26.225669 2014-08-13 17:38:02.840265 7fe9bae887a0 20 read_log 192834'12173 (192834'12162) delete 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:12999 2014-08-11 15:25:31.141974 2014-08-13 17:38:02.840277 7fe9bae887a0 20 read_log 192834'12175 (0'0) promote 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:13001 2014-08-11 15:25:31.277248 2014-08-13 17:38:02.840289 7fe9bae887a0 20 read_log 192834'12176 (192834'12175) modify 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by client.933448.0:21626 2014-08-11 15:25:31.268683 2014-08-13 17:38:02.840300 7fe9bae887a0 20 read_log 192834'12178 (192834'12164) delete dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13004 2014-08-11 15:25:31.335438 2014-08-13 17:38:02.840312 7fe9bae887a0 20 read_log 192834'12182 (192834'11813) clean de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13008 2014-08-11 15:25:31.349715 2014-08-13 17:38:02.840324 7fe9bae887a0 20 read_log 192834'12187 (192834'12182) delete de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13013 2014-08-11 15:25:31.365224 2014-08-13 17:38:02.840337 7fe9bae887a0 20 read_log 192834'12188 (192834'11823) clean 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13014 2014-08-11 15:25:31.404855 2014-08-13 17:38:02.840349 7fe9bae887a0 20 read_log 192834'12211 (192834'12188) delete 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13037 2014-08-11 15:25:36.754293 2014-08-13 17:38:02.840363 7fe9bae887a0 20 read_log 192834'12244 (0'0) promote 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13070 2014-08-11 15:25:38.730752 2014-08-13 17:38:02.840376 7fe9bae887a0 20 read_log 192834'12245 (192834'12244) modify 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by client.933448.0:22846 2014-08-11 15:25:38.701922 2014-08-13 17:38:02.840388 7fe9bae887a0 20 read_log 192834'12248 (0'0) promote 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13074 2014-08-11 15:25:38.773959 2014-08-13 17:38:02.840400 7fe9bae887a0 20 read_log 192834'12249 (192834'12248) modify 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by client.933448.0:22874 2014-08-11 15:25:38.737517 2014-08-13 17:38:02.840413 7fe9bae887a0 20 read_log 192834'12282 (0'0) promote 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13108 2014-08-11 15:25:40.959716 2014-08-13 17:38:02.840424 7fe9bae887a0 20 read_log 192834'12283 (192834'12282) modify 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by client.933448.0:24481 2014-08-11 15:25:40.923294 2014-08-13 17:38:02.840436 7fe9bae887a0 20 read_log 192834'12288 (0'0) promote f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13114 2014-08-11 15:25:41.241345 2014-08-13 17:38:02.840476 7fe9bae887a0 20 read_log 192834'12289 (192834'12288) modify f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by client.933448.0:24728 2014-08-11 15:25:41.223914 2014-08-13 17:38:02.840490 7fe9bae887a0 20 read_log 192834'12290 (0'0) promote ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13116 2014-08-11 15:25:41.664430 2014-08-13 17:38:02.840502 7fe9bae887a0 20 read_log 192834'12291 (192834'12290) modify ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by client.933448.0:24998 2014-08-11 15:25:41.634649 2014-08-13 17:38:02.840514 7fe9bae887a0 20 read_log 192834'12302 (0'0) promote 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13128 2014-08-11 15:25:42.193066 2014-08-13 17:38:02.840526 7fe9bae887a0 20 read_log 192834'12303 (192834'12302) modify 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by client.933448.0:25525 2014-08-11 15:25:42.173695 2014-08-13 17:38:02.840542 7fe9bae887a0 20 read_log 192834'12308 (0'0) promote 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13134 2014-08-11 15:25:42.317501 2014-08-13 17:38:02.840554 7fe9bae887a0 20 read_log 192834'12309 (192834'12308) modify 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by client.933448.0:25640 2014-08-11 15:25:42.296339 2014-08-13 17:38:02.840566 7fe9bae887a0 20 read_log 192834'12310 (0'0) promote 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13136 2014-08-11 15:25:42.335327 2014-08-13 17:38:02.840579 7fe9bae887a0 20 read_log 192834'12311 (192834'12310) modify 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by client.933448.0:25654 2014-08-11 15:25:42.313722 2014-08-13 17:38:02.840591 7fe9bae887a0 20 read_log 192834'12364 (192834'11902) clean f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13190 2014-08-11 15:25:47.012290 2014-08-13 17:38:02.840603 7fe9bae887a0 20 read_log 192834'12369 (192834'12364) delete f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13195 2014-08-11 15:25:47.044538 2014-08-13 17:38:02.840615 7fe9bae887a0 20 read_log 192834'12393 (0'0) promote 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13219 2014-08-11 15:25:48.826772 2014-08-13 17:38:02.840627 7fe9bae887a0 20 read_log 192834'12394 (192834'12393) modify 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by client.933448.0:30391 2014-08-11 15:25:48.805086 2014-08-13 17:38:02.840639 7fe9bae887a0 20 read_log 192834'12397 (0'0) promote c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13223 2014-08-11 15:25:49.119938 2014-08-13 17:38:02.840651 7fe9bae887a0 20 read_log 192834'12398 (192834'12397) modify c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by client.933448.0:30654 2014-08-11 15:25:49.094761 2014-08-13 17:38:02.840664 7fe9bae887a0 20 read_log 192834'12403 (0'0) promote b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13229 2014-08-11 15:25:49.346101 2014-08-13 17:38:02.840676 7fe9bae887a0 20 read_log 192834'12404 (192834'12403) modify b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by client.933448.0:30854 2014-08-11 15:25:49.313997 2014-08-13 17:38:02.840688 7fe9bae887a0 20 read_log 192834'12450 (192834'11949) clean 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13276 2014-08-11 15:25:52.579690 2014-08-13 17:38:02.840700 7fe9bae887a0 20 read_log 192834'12452 (192834'12450) delete 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13278 2014-08-11 15:25:52.598666 2014-08-13 17:38:02.840712 7fe9bae887a0 20 read_log 192834'12458 (0'0) promote dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13284 2014-08-11 15:25:52.840064 2014-08-13 17:38:02.840732 7fe9bae887a0 20 read_log 192834'12459 (192834'12458) modify dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by client.933448.0:33067 2014-08-11 15:25:52.809547 2014-08-13 17:38:02.840745 7fe9bae887a0 20 read_log 192834'12460 (0'0) promote 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:13286 2014-08-11 15:25:53.100320 2014-08-13 17:38:02.840757 7fe9bae887a0 20 read_log 192834'12461 (192834'12460) modify 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by client.933448.0:33230 2014-08-11 15:25:53.060737 2014-08-13 17:38:02.840769 7fe9bae887a0 20 read_log 192834'12462 (0'0) promote de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13288 2014-08-11 15:25:53.443742 2014-08-13 17:38:02.840781 7fe9bae887a0 20 read_log 192834'12463 (192834'12462) modify de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by client.933448.0:33376 2014-08-11 15:25:53.415317 2014-08-13 17:38:02.840793 7fe9bae887a0 20 read_log 192834'12472 (0'0) promote 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13298 2014-08-11 15:25:54.453692 2014-08-13 17:38:02.840805 7fe9bae887a0 20 read_log 192834'12473 (192834'12472) modify 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by client.933448.0:33964 2014-08-11 15:25:54.427518 2014-08-13 17:38:02.840816 7fe9bae887a0 20 read_log 192834'12523 (192834'11990) clean cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13349 2014-08-11 15:25:57.912690 2014-08-13 17:38:02.840828 7fe9bae887a0 20 read_log 192834'12530 (192834'12523) delete cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13356 2014-08-11 15:25:57.968613 2014-08-13 17:38:02.840840 7fe9bae887a0 20 read_log 192834'12536 (0'0) promote f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13362 2014-08-11 15:25:58.409647 2014-08-13 17:38:02.840852 7fe9bae887a0 20 read_log 192834'12537 (192834'12536) modify f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by client.933448.0:36869 2014-08-11 15:25:58.384189 2014-08-13 17:38:02.840865 7fe9bae887a0 20 read_log 192834'12556 (0'0) promote 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13382 2014-08-11 15:26:00.462414 2014-08-13 17:38:02.840877 7fe9bae887a0 20 read_log 192834'12557 (192834'12556) modify 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by client.933448.0:38054 2014-08-11 15:26:00.438527 2014-08-13 17:38:02.840889 7fe9bae887a0 20 read_log 192834'12573 (0'0) promote cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13399 2014-08-11 15:26:01.350230 2014-08-13 17:38:02.840901 7fe9bae887a0 20 read_log 192834'12574 (192834'12573) modify cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by client.933448.0:38820 2014-08-11 15:26:01.303761 2014-08-13 17:38:02.840913 7fe9bae887a0 20 read_log 192834'12586 (192834'12027) modify e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by client.933448.0:40079 2014-08-11 15:26:02.274540 2014-08-13 17:38:02.840926 7fe9bae887a0 20 read_log 192834'12605 (192834'12139) modify 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by client.933448.0:42703 2014-08-11 15:26:03.363313 2014-08-13 17:38:02.840939 7fe9bae887a0 20 read_log 192834'12614 (192834'12176) modify 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by client.933448.0:43917 2014-08-11 15:26:03.761254 2014-08-13 17:38:02.840951 7fe9bae887a0 20 read_log 192834'12667 (0'0) promote 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:13493 2014-08-11 15:26:23.572465 2014-08-13 17:38:02.840963 7fe9bae887a0 20 read_log 192834'12668 (192834'12667) modify 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by client.1883634.0:2466 2014-08-11 15:26:23.564624 2014-08-13 17:38:02.840988 7fe9bae887a0 20 read_log 192834'12693 (0'0) promote 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:13519 2014-08-11 15:26:29.356691 2014-08-13 17:38:02.841004 7fe9bae887a0 20 read_log 192834'12694 (192834'12693) modify 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by client.1883634.0:4101 2014-08-11 15:26:29.347366 2014-08-13 17:38:02.841017 7fe9bae887a0 20 read_log 192834'12697 (0'0) promote 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:13523 2014-08-11 15:26:29.914589 2014-08-13 17:38:02.841029 7fe9bae887a0 20 read_log 192834'12698 (192834'12697) modify 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by client.1883634.0:4259 2014-08-11 15:26:29.907247 2014-08-13 17:38:02.841043 7fe9bae887a0 20 read_log 192834'12719 (0'0) promote e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:13545 2014-08-11 15:26:36.067449 2014-08-13 17:38:02.841056 7fe9bae887a0 20 read_log 192834'12720 (192834'12719) modify e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by client.1883634.0:5904 2014-08-11 15:26:36.059974 2014-08-13 17:38:02.841068 7fe9bae887a0 20 read_log 192834'12734 (192834'12245) clean 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13560 2014-08-11 15:26:38.782265 2014-08-13 17:38:02.841082 7fe9bae887a0 20 read_log 192834'12735 (192834'12249) clean 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13561 2014-08-11 15:26:38.782532 2014-08-13 17:38:02.841094 7fe9bae887a0 20 read_log 192834'12736 (0'0) promote 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:13562 2014-08-11 15:26:38.828569 2014-08-13 17:38:02.841106 7fe9bae887a0 20 read_log 192834'12738 (192834'12736) modify 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by client.1883634.0:6546 2014-08-11 15:26:38.661999 2014-08-13 17:38:02.841118 7fe9bae887a0 20 read_log 192834'12742 (192834'12734) delete 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13568 2014-08-11 15:26:38.847355 2014-08-13 17:38:02.841130 7fe9bae887a0 20 read_log 192834'12743 (192834'12735) delete 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13569 2014-08-11 15:26:38.849750 2014-08-13 17:38:02.841144 7fe9bae887a0 20 read_log 192834'12785 (192834'12303) clean 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13611 2014-08-11 15:26:44.121150 2014-08-13 17:38:02.841164 7fe9bae887a0 20 read_log 192834'12786 (192834'12283) clean 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13612 2014-08-11 15:26:44.121373 2014-08-13 17:38:02.841180 7fe9bae887a0 20 read_log 192834'12789 (192834'12289) clean f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13615 2014-08-11 15:26:44.132877 2014-08-13 17:38:02.841192 7fe9bae887a0 20 read_log 192834'12790 (192834'12311) clean 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13616 2014-08-11 15:26:44.138194 2014-08-13 17:38:02.841204 7fe9bae887a0 20 read_log 192834'12795 (192834'12291) clean ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13621 2014-08-11 15:26:44.159677 2014-08-13 17:38:02.841216 7fe9bae887a0 20 read_log 192834'12823 (192834'12785) delete 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13649 2014-08-11 15:26:44.272782 2014-08-13 17:38:02.841228 7fe9bae887a0 20 read_log 192834'12824 (192834'12786) delete 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13650 2014-08-11 15:26:44.275607 2014-08-13 17:38:02.841242 7fe9bae887a0 20 read_log 192834'12827 (192834'12789) delete f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13653 2014-08-11 15:26:44.288312 2014-08-13 17:38:02.841267 7fe9bae887a0 20 read_log 192834'12828 (192834'12790) delete 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13654 2014-08-11 15:26:44.291609 2014-08-13 17:38:02.841282 7fe9bae887a0 20 read_log 192834'12832 (192834'12309) clean 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13658 2014-08-11 15:26:44.301067 2014-08-13 17:38:02.841295 7fe9bae887a0 20 read_log 192834'12833 (192834'12795) delete ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13659 2014-08-11 15:26:44.307789 2014-08-13 17:38:02.841309 7fe9bae887a0 20 read_log 192834'12838 (192834'12832) delete 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13664 2014-08-11 15:26:44.324022 2014-08-13 17:38:02.841321 7fe9bae887a0 20 read_log 192834'12843 (0'0) promote 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:13669 2014-08-11 15:26:45.030039 2014-08-13 17:38:02.841335 7fe9bae887a0 20 read_log 192834'12844 (192834'12843) modify 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by client.1883634.0:8120 2014-08-11 15:26:45.021769 2014-08-13 17:38:02.841347 7fe9bae887a0 20 read_log 192834'12871 (192834'12404) clean b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13697 2014-08-11 15:26:49.471035 2014-08-13 17:38:02.841359 7fe9bae887a0 20 read_log 192834'12881 (192834'12398) clean c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13707 2014-08-11 15:26:49.507064 2014-08-13 17:38:02.841371 7fe9bae887a0 20 read_log 192834'12882 (192834'12394) clean 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13708 2014-08-11 15:26:49.512141 2014-08-13 17:38:02.841383 7fe9bae887a0 20 read_log 192834'12899 (192834'12871) delete b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13725 2014-08-11 15:26:49.577015 2014-08-13 17:38:02.841395 7fe9bae887a0 20 read_log 192834'12907 (192834'12881) delete c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13733 2014-08-11 15:26:49.606447 2014-08-13 17:38:02.841407 7fe9bae887a0 20 read_log 192834'12908 (192834'12882) delete 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13734 2014-08-11 15:26:49.608334 2014-08-13 17:38:02.841419 7fe9bae887a0 20 read_log 192834'12921 (0'0) promote b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:13747 2014-08-11 15:26:53.423263 2014-08-13 17:38:02.841433 7fe9bae887a0 20 read_log 192834'12922 (192834'12921) modify b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by client.1883634.0:10024 2014-08-11 15:26:53.412919 2014-08-13 17:38:02.841445 7fe9bae887a0 20 read_log 192834'12946 (192834'12461) clean 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:13771 2014-08-11 15:26:54.721063 2014-08-13 17:38:02.841460 7fe9bae887a0 20 read_log 192834'12949 (192834'12463) clean de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13774 2014-08-11 15:26:54.735825 2014-08-13 17:38:02.841478 7fe9bae887a0 20 read_log 192834'12950 (192834'12459) clean dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13775 2014-08-11 15:26:54.736034 2014-08-13 17:38:02.841491 7fe9bae887a0 20 read_log 192834'12954 (192834'12473) clean 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13779 2014-08-11 15:26:54.799138 2014-08-13 17:38:02.841505 7fe9bae887a0 20 read_log 192834'12958 (0'0) promote edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:13783 2014-08-11 15:26:57.390145 2014-08-13 17:38:02.841517 7fe9bae887a0 20 read_log 192834'12959 (192834'12958) modify edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by client.1883634.0:10880 2014-08-11 15:26:57.379281 2014-08-13 17:38:02.841538 7fe9bae887a0 20 read_log 192834'12985 (192834'12537) clean f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13810 2014-08-11 15:26:59.885306 2014-08-13 17:38:02.841552 7fe9bae887a0 20 read_log 192834'13004 (0'0) promote e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:13829 2014-08-11 15:27:03.896579 2014-08-13 17:38:02.841564 7fe9bae887a0 20 read_log 192834'13005 (192834'13004) modify e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by client.1883634.0:12363 2014-08-11 15:27:03.684932 2014-08-13 17:38:02.841578 7fe9bae887a0 20 read_log 192834'13038 (192834'12557) clean 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13863 2014-08-11 15:27:05.052012 2014-08-13 17:38:02.841592 7fe9bae887a0 20 read_log 192834'13043 (192834'12614) clean 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:13868 2014-08-11 15:27:05.065041 2014-08-13 17:38:02.841604 7fe9bae887a0 20 read_log 192834'13045 (192834'12586) clean e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:13870 2014-08-11 15:27:05.066382 2014-08-13 17:38:02.841618 7fe9bae887a0 20 read_log 192834'13050 (192834'12605) clean 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:13875 2014-08-11 15:27:05.087787 2014-08-13 17:38:02.841630 7fe9bae887a0 20 read_log 192834'13051 (192834'12574) clean cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13876 2014-08-11 15:27:05.087943 2014-08-13 17:38:02.841642 7fe9bae887a0 20 read_log 192834'13102 (0'0) promote f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:13927 2014-08-11 15:27:15.417551 2014-08-13 17:38:02.841655 7fe9bae887a0 20 read_log 192834'13103 (192834'13102) modify f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by client.1883634.0:14797 2014-08-11 15:27:15.409345 2014-08-13 17:38:02.841667 7fe9bae887a0 20 read_log 192834'13104 (0'0) promote 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:13929 2014-08-11 15:27:16.054483 2014-08-13 17:38:02.841681 7fe9bae887a0 20 read_log 192834'13105 (192834'13104) modify 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by client.1883634.0:14915 2014-08-11 15:27:16.045312 2014-08-13 17:38:02.841693 7fe9bae887a0 20 read_log 192834'13106 (0'0) promote de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:13931 2014-08-11 15:27:17.353349 2014-08-13 17:38:02.841708 7fe9bae887a0 20 read_log 192834'13107 (192834'13106) modify de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by client.1883634.0:15160 2014-08-11 15:27:17.345621 2014-08-13 17:38:02.841722 7fe9bae887a0 20 read_log 192834'13179 (192834'12946) delete 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:14004 2014-08-11 15:27:26.380290 2014-08-13 17:38:02.841735 7fe9bae887a0 20 read_log 192834'13180 (192834'13038) delete 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:14005 2014-08-11 15:27:26.385049 2014-08-13 17:38:02.841749 7fe9bae887a0 20 read_log 192834'13189 (192834'12985) delete f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:14014 2014-08-11 15:27:26.406977 2014-08-13 17:38:02.841762 7fe9bae887a0 20 read_log 192834'13190 (192834'13043) delete 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:14015 2014-08-11 15:27:26.409873 2014-08-13 17:38:02.841776 7fe9bae887a0 20 read_log 192834'13191 (192834'13045) delete e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:14016 2014-08-11 15:27:26.412625 2014-08-13 17:38:02.841788 7fe9bae887a0 20 read_log 192834'13199 (192834'12668) clean 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14024 2014-08-11 15:27:26.529200 2014-08-13 17:38:02.841811 7fe9bae887a0 20 read_log 192834'13202 (192834'12949) delete de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:14027 2014-08-11 15:27:26.537119 2014-08-13 17:38:02.841826 7fe9bae887a0 20 read_log 192834'13203 (192834'12950) delete dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:14028 2014-08-11 15:27:26.539946 2014-08-13 17:38:02.841838 7fe9bae887a0 20 read_log 192834'13204 (192834'13051) delete cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:14029 2014-08-11 15:27:26.540191 2014-08-13 17:38:02.841854 7fe9bae887a0 20 read_log 192834'13205 (192834'13050) delete 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:14030 2014-08-11 15:27:26.542645 2014-08-13 17:38:02.841886 7fe9bae887a0 20 read_log 192834'13222 (192834'12954) delete 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:14047 2014-08-11 15:27:26.562893 2014-08-13 17:38:02.841901 7fe9bae887a0 20 read_log 192834'13230 (0'0) promote 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14055 2014-08-11 15:27:26.693357 2014-08-13 17:38:02.841913 7fe9bae887a0 20 read_log 192834'13231 (192834'13230) modify 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by client.1883634.0:16254 2014-08-11 15:27:26.644300 2014-08-13 17:38:02.841925 7fe9bae887a0 20 read_log 192834'13280 (192834'13199) delete 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14105 2014-08-11 15:27:26.888992 2014-08-13 17:38:02.841937 7fe9bae887a0 20 read_log 192834'13319 (192834'12694) clean 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14144 2014-08-11 15:27:32.292572 2014-08-13 17:38:02.841949 7fe9bae887a0 20 read_log 192834'13326 (192834'12698) clean 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14151 2014-08-11 15:27:32.336205 2014-08-13 17:38:02.841964 7fe9bae887a0 20 read_log 192834'13327 (192834'13319) delete 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14152 2014-08-11 15:27:32.340983 2014-08-13 17:38:02.841976 7fe9bae887a0 20 read_log 192834'13328 (192834'13326) delete 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14153 2014-08-11 15:27:32.343117 2014-08-13 17:38:02.841988 7fe9bae887a0 20 read_log 192834'13332 (0'0) promote db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14157 2014-08-11 15:27:32.946951 2014-08-13 17:38:02.842002 7fe9bae887a0 20 read_log 192834'13333 (192834'13332) modify db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by client.1883634.0:17185 2014-08-11 15:27:32.935181 2014-08-13 17:38:02.842015 7fe9bae887a0 20 read_log 192834'13345 (192834'12720) clean e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14170 2014-08-11 15:27:37.483313 2014-08-13 17:38:02.842029 7fe9bae887a0 20 read_log 192834'13353 (192834'13345) delete e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14178 2014-08-11 15:27:37.626690 2014-08-13 17:38:02.842041 7fe9bae887a0 20 read_log 192834'13361 (0'0) promote a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:14186 2014-08-11 15:27:39.257397 2014-08-13 17:38:02.842055 7fe9bae887a0 20 read_log 192834'13362 (192834'13361) modify a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by client.1883634.0:17833 2014-08-11 15:27:39.247974 2014-08-13 17:38:02.842069 7fe9bae887a0 20 read_log 192834'13378 (192834'12738) clean 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14203 2014-08-11 15:27:42.840550 2014-08-13 17:38:02.842081 7fe9bae887a0 20 read_log 192834'13383 (0'0) promote 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14208 2014-08-11 15:27:44.900332 2014-08-13 17:38:02.842104 7fe9bae887a0 20 read_log 192834'13384 (192834'13383) modify 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by client.1883634.0:18505 2014-08-11 15:27:44.889797 2014-08-13 17:38:02.842118 7fe9bae887a0 20 read_log 192834'13395 (192834'13378) delete 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14220 2014-08-11 15:27:47.887184 2014-08-13 17:38:02.842131 7fe9bae887a0 20 read_log 192834'13397 (192834'12844) clean 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14222 2014-08-11 15:27:47.952664 2014-08-13 17:38:02.842145 7fe9bae887a0 20 read_log 192834'13403 (192834'13397) delete 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14228 2014-08-11 15:27:47.970500 2014-08-13 17:38:02.842163 7fe9bae887a0 20 read_log 192834'13416 (0'0) promote 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:14241 2014-08-11 15:27:51.415615 2014-08-13 17:38:02.842178 7fe9bae887a0 20 read_log 192834'13417 (192834'13416) modify 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by client.1883634.0:19343 2014-08-11 15:27:51.400874 2014-08-13 17:38:02.842191 7fe9bae887a0 20 read_log 192834'13434 (0'0) promote 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:14259 2014-08-11 15:27:54.933338 2014-08-13 17:38:02.842203 7fe9bae887a0 20 read_log 192834'13435 (192834'13434) modify 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by client.1883634.0:20008 2014-08-11 15:27:54.926814 2014-08-13 17:38:02.842215 7fe9bae887a0 20 read_log 192834'13449 (192834'12959) clean edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14274 2014-08-11 15:27:58.362391 2014-08-13 17:38:02.842227 7fe9bae887a0 20 read_log 192834'13451 (192834'12922) clean b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14276 2014-08-11 15:27:58.375850 2014-08-13 17:38:02.842239 7fe9bae887a0 20 read_log 192834'13458 (192834'13449) delete edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14283 2014-08-11 15:28:03.326806 2014-08-13 17:38:02.842251 7fe9bae887a0 20 read_log 192834'13463 (192834'13451) delete b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14288 2014-08-11 15:28:03.439580 2014-08-13 17:38:02.842269 7fe9bae887a0 20 read_log 192834'13478 (0'0) promote db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:14303 2014-08-11 15:28:06.962276 2014-08-13 17:38:02.842281 7fe9bae887a0 20 read_log 192834'13479 (192834'13478) modify db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by client.1883634.0:22159 2014-08-11 15:28:06.953090 2014-08-13 17:38:02.842293 7fe9bae887a0 20 read_log 192834'13501 (192834'13005) clean e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14326 2014-08-11 15:28:08.749010 2014-08-13 17:38:02.842307 7fe9bae887a0 20 read_log 192834'13514 (0'0) promote cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:14339 2014-08-11 15:28:12.845985 2014-08-13 17:38:02.842319 7fe9bae887a0 20 read_log 192834'13515 (192834'13514) modify cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by client.1883634.0:23125 2014-08-11 15:28:12.835770 2014-08-13 17:38:02.842331 7fe9bae887a0 20 read_log 192834'13523 (192834'13501) delete e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14348 2014-08-11 15:28:13.937216 2014-08-13 17:38:02.842345 7fe9bae887a0 20 read_log 192834'13564 (192834'13105) clean 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14389 2014-08-11 15:28:19.162060 2014-08-13 17:38:02.842357 7fe9bae887a0 20 read_log 192834'13568 (192834'13564) delete 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14393 2014-08-11 15:28:19.178197 2014-08-13 17:38:02.842379 7fe9bae887a0 20 read_log 192834'13569 (192834'13107) clean de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14394 2014-08-11 15:28:19.189537 2014-08-13 17:38:02.842394 7fe9bae887a0 20 read_log 192834'13570 (192834'13103) clean f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14395 2014-08-11 15:28:19.199741 2014-08-13 17:38:02.842406 7fe9bae887a0 20 read_log 192834'13599 (0'0) promote 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14424 2014-08-11 15:28:21.901978 2014-08-13 17:38:02.842420 7fe9bae887a0 20 read_log 192834'13600 (192834'13599) modify 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by client.1883634.0:25790 2014-08-11 15:28:21.881267 2014-08-13 17:38:02.842432 7fe9bae887a0 20 read_log 192834'13622 (192834'13569) delete de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14447 2014-08-11 15:28:24.308379 2014-08-13 17:38:02.842446 7fe9bae887a0 20 read_log 192834'13623 (192834'13570) delete f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14448 2014-08-11 15:28:24.310555 2014-08-13 17:38:02.842458 7fe9bae887a0 20 read_log 192834'13635 (0'0) promote 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14460 2014-08-11 15:28:24.488334 2014-08-13 17:38:02.842470 7fe9bae887a0 20 read_log 192834'13636 (192834'13635) modify 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by client.1883634.0:27425 2014-08-11 15:28:24.459483 2014-08-13 17:38:02.842482 7fe9bae887a0 20 read_log 192834'13639 (0'0) promote 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14464 2014-08-11 15:28:24.676334 2014-08-13 17:38:02.842494 7fe9bae887a0 20 read_log 192834'13640 (192834'13639) modify 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by client.1883634.0:27583 2014-08-11 15:28:24.644949 2014-08-13 17:38:02.842509 7fe9bae887a0 20 read_log 192834'13661 (0'0) promote e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14486 2014-08-11 15:28:26.899026 2014-08-13 17:38:02.842521 7fe9bae887a0 20 read_log 192834'13662 (192834'13661) modify e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by client.1883634.0:29228 2014-08-11 15:28:26.873620 2014-08-13 17:38:02.842533 7fe9bae887a0 20 read_log 192834'13665 (0'0) promote 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14490 2014-08-11 15:28:27.808809 2014-08-13 17:38:02.842545 7fe9bae887a0 20 read_log 192834'13666 (192834'13665) modify 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by client.1883634.0:29870 2014-08-11 15:28:27.774317 2014-08-13 17:38:02.842557 7fe9bae887a0 20 read_log 192834'13681 (192834'13231) clean 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14506 2014-08-11 15:28:29.558507 2014-08-13 17:38:02.842569 7fe9bae887a0 20 read_log 192834'13687 (192834'13681) delete 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14512 2014-08-11 15:28:29.583201 2014-08-13 17:38:02.842581 7fe9bae887a0 20 read_log 192834'13700 (0'0) promote 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14525 2014-08-11 15:28:30.548722 2014-08-13 17:38:02.842593 7fe9bae887a0 20 read_log 192834'13701 (192834'13700) modify 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by client.1883634.0:31444 2014-08-11 15:28:30.519282 2014-08-13 17:38:02.842605 7fe9bae887a0 20 read_log 192834'13722 (0'0) promote b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14547 2014-08-11 15:28:33.071199 2014-08-13 17:38:02.842617 7fe9bae887a0 20 read_log 192834'13723 (192834'13722) modify b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by client.1883634.0:33348 2014-08-11 15:28:33.043002 2014-08-13 17:38:02.842636 7fe9bae887a0 20 read_log 192834'13732 (0'0) promote edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14557 2014-08-11 15:28:34.387218 2014-08-13 17:38:02.842650 7fe9bae887a0 20 read_log 192834'13733 (192834'13732) modify edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by client.1883634.0:34204 2014-08-11 15:28:34.359742 2014-08-13 17:38:02.842662 7fe9bae887a0 20 read_log 192834'13740 (192834'13333) clean db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14565 2014-08-11 15:28:34.715826 2014-08-13 17:38:02.842674 7fe9bae887a0 20 read_log 192834'13743 (192834'13740) delete db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14568 2014-08-11 15:28:34.730358 2014-08-13 17:38:02.842686 7fe9bae887a0 20 read_log 192834'13760 (0'0) promote e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14585 2014-08-11 15:28:37.027270 2014-08-13 17:38:02.842698 7fe9bae887a0 20 read_log 192834'13761 (192834'13760) modify e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by client.1883634.0:35687 2014-08-11 15:28:37.002594 2014-08-13 17:38:02.842710 7fe9bae887a0 20 read_log 192834'13808 (192834'13362) clean a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:14633 2014-08-11 15:28:39.988510 2014-08-13 17:38:02.842721 7fe9bae887a0 20 read_log 192834'13813 (0'0) promote f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14638 2014-08-11 15:28:40.270373 2014-08-13 17:38:02.842733 7fe9bae887a0 20 read_log 192834'13814 (192834'13813) modify f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by client.1883634.0:38121 2014-08-11 15:28:40.241382 2014-08-13 17:38:02.842745 7fe9bae887a0 20 read_log 192834'13815 (0'0) promote 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14640 2014-08-11 15:28:40.423257 2014-08-13 17:38:02.842757 7fe9bae887a0 20 read_log 192834'13816 (192834'13815) modify 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by client.1883634.0:38239 2014-08-11 15:28:40.399029 2014-08-13 17:38:02.842769 7fe9bae887a0 20 read_log 192834'13817 (0'0) promote de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14642 2014-08-11 15:28:40.726272 2014-08-13 17:38:02.842780 7fe9bae887a0 20 read_log 192834'13818 (192834'13817) modify de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by client.1883634.0:38484 2014-08-11 15:28:40.693349 2014-08-13 17:38:02.842792 7fe9bae887a0 20 read_log 192834'13831 (0'0) promote 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14656 2014-08-11 15:28:42.354107 2014-08-13 17:38:02.842804 7fe9bae887a0 20 read_log 192834'13832 (192834'13831) modify 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by client.1883634.0:39578 2014-08-11 15:28:42.323350 2014-08-13 17:38:02.842816 7fe9bae887a0 20 read_log 192834'13853 (0'0) promote db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14678 2014-08-11 15:28:43.887974 2014-08-13 17:38:02.842828 7fe9bae887a0 20 read_log 192834'13854 (192834'13853) modify db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by client.1883634.0:40509 2014-08-11 15:28:43.852982 2014-08-13 17:38:02.842840 7fe9bae887a0 20 read_log 192834'13862 (192834'13808) modify a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by client.1883634.0:41157 2014-08-11 15:28:44.696809 2014-08-13 17:38:02.842852 7fe9bae887a0 20 read_log 192834'13868 (192834'13384) clean 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14693 2014-08-11 15:28:45.133723 2014-08-13 17:38:02.842866 7fe9bae887a0 20 read_log 192834'13871 (192834'13868) delete 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14696 2014-08-11 15:28:45.175179 2014-08-13 17:38:02.842891 7fe9bae887a0 20 read_log 192834'13878 (0'0) promote 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14703 2014-08-11 15:28:45.545404 2014-08-13 17:38:02.842907 7fe9bae887a0 20 read_log 192834'13879 (192834'13878) modify 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by client.1883634.0:41829 2014-08-11 15:28:45.523032 2014-08-13 17:38:02.842919 7fe9bae887a0 20 read_log 192834'13887 (192834'13417) modify 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by client.1883634.0:42667 2014-08-11 15:28:46.317169 2014-08-13 17:38:02.842932 7fe9bae887a0 20 read_log 192834'13890 (192834'13435) modify 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by client.1883634.0:43332 2014-08-11 15:28:46.522659 2014-08-13 17:38:02.842944 7fe9bae887a0 20 read_log 192834'13899 (192834'13479) modify db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by client.1883634.0:45483 2014-08-11 15:28:47.292523 2014-08-13 17:38:02.842956 7fe9bae887a0 20 read_log 192834'13907 (192834'13515) modify cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by client.1883634.0:46449 2014-08-11 15:28:47.596292 2014-08-13 17:38:02.842968 7fe9bae887a0 20 read_log 192834'13957 (192834'13600) clean 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14782 2014-08-11 15:29:25.765155 2014-08-13 17:38:02.842980 7fe9bae887a0 20 read_log 192834'13963 (192834'13636) clean 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14788 2014-08-11 15:29:25.786234 2014-08-13 17:38:02.842992 7fe9bae887a0 20 read_log 192834'13964 (192834'13640) clean 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14789 2014-08-11 15:29:25.791380 2014-08-13 17:38:02.843004 7fe9bae887a0 20 read_log 192834'13985 (192834'13957) delete 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14810 2014-08-11 15:29:25.836936 2014-08-13 17:38:02.843016 7fe9bae887a0 20 read_log 192834'13991 (192834'13963) delete 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14816 2014-08-11 15:29:25.854789 2014-08-13 17:38:02.843028 7fe9bae887a0 20 read_log 192834'13992 (192834'13964) delete 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14817 2014-08-11 15:29:25.856470 2014-08-13 17:38:02.843040 7fe9bae887a0 20 read_log 192834'14004 (192834'13701) clean 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14829 2014-08-11 15:29:30.921610 2014-08-13 17:38:02.843053 7fe9bae887a0 20 read_log 192834'14009 (192834'13666) clean 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14834 2014-08-11 15:29:30.937720 2014-08-13 17:38:02.843065 7fe9bae887a0 20 read_log 192834'14010 (192834'13662) clean e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14835 2014-08-11 15:29:30.943217 2014-08-13 17:38:02.843077 7fe9bae887a0 20 read_log 192834'14020 (192834'14004) delete 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14845 2014-08-11 15:29:30.967513 2014-08-13 17:38:02.843089 7fe9bae887a0 20 read_log 192834'14025 (192834'14009) delete 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14850 2014-08-11 15:29:30.977244 2014-08-13 17:38:02.843118 7fe9bae887a0 20 read_log 192834'14026 (192834'14010) delete e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14851 2014-08-11 15:29:30.978731 2014-08-13 17:38:02.843133 7fe9bae887a0 20 read_log 192834'14044 (192834'13733) clean edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14869 2014-08-11 15:29:36.050181 2014-08-13 17:38:02.843146 7fe9bae887a0 20 read_log 192834'14050 (192834'13723) clean b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14875 2014-08-11 15:29:36.067538 2014-08-13 17:38:02.843187 7fe9bae887a0 20 read_log 192834'14064 (192834'14044) delete edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14889 2014-08-11 15:29:36.091967 2014-08-13 17:38:02.843204 7fe9bae887a0 20 read_log 192834'14069 (192834'14050) delete b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14894 2014-08-11 15:29:36.101936 2014-08-13 17:38:02.843216 7fe9bae887a0 20 read_log 192834'14091 (192834'13761) clean e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14916 2014-08-11 15:29:41.186439 2014-08-13 17:38:02.843228 7fe9bae887a0 20 read_log 192834'14092 (192834'13816) clean 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14917 2014-08-11 15:29:41.189771 2014-08-13 17:38:02.843240 7fe9bae887a0 20 read_log 192834'14099 (192834'13818) clean de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14924 2014-08-11 15:29:41.208156 2014-08-13 17:38:02.843252 7fe9bae887a0 20 read_log 192834'14100 (192834'13814) clean f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14925 2014-08-11 15:29:41.208354 2014-08-13 17:38:02.843267 7fe9bae887a0 20 read_log 192834'14122 (192834'14091) delete e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14947 2014-08-11 15:29:41.237411 2014-08-13 17:38:02.843279 7fe9bae887a0 20 read_log 192834'14123 (192834'14092) delete 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14948 2014-08-11 15:29:41.238847 2014-08-13 17:38:02.843291 7fe9bae887a0 20 read_log 192834'14128 (192834'14099) delete de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14953 2014-08-11 15:29:41.244491 2014-08-13 17:38:02.843304 7fe9bae887a0 20 read_log 192834'14129 (192834'14100) delete f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14954 2014-08-11 15:29:41.245520 2014-08-13 17:38:02.843316 7fe9bae887a0 20 read_log 192834'14147 (192834'13854) clean db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14972 2014-08-11 15:29:46.293628 2014-08-13 17:38:02.843328 7fe9bae887a0 20 read_log 192834'14148 (192834'13879) clean 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14973 2014-08-11 15:29:46.294164 2014-08-13 17:38:02.843345 7fe9bae887a0 20 read_log 192834'14166 (192834'14147) delete db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14991 2014-08-11 15:29:46.320026 2014-08-13 17:38:02.843357 7fe9bae887a0 20 read_log 192834'14167 (192834'14148) delete 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14992 2014-08-11 15:29:46.320906 2014-08-13 17:38:02.843369 7fe9bae887a0 20 read_log 192834'14172 (192834'13832) clean 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14997 2014-08-11 15:29:46.325109 2014-08-13 17:38:02.843381 7fe9bae887a0 20 read_log 192834'14179 (192834'13862) clean a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:15004 2014-08-11 15:29:46.335057 2014-08-13 17:38:02.843394 7fe9bae887a0 20 read_log 192834'14184 (192834'14172) delete 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:15009 2014-08-11 15:29:46.344826 2014-08-13 17:38:02.843406 7fe9bae887a0 20 read_log 192834'14190 (192834'14179) delete a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:15015 2014-08-11 15:29:46.349471 2014-08-13 17:38:02.843418 7fe9bae887a0 20 read_log 192834'14194 (192834'13887) clean 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:15019 2014-08-11 15:29:46.352122 2014-08-13 17:38:02.843430 7fe9bae887a0 20 read_log 192834'14199 (192834'14194) delete 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:15024 2014-08-11 15:29:46.354349 2014-08-13 17:38:02.843450 7fe9bae887a0 20 read_log 192834'14222 (192834'13890) clean 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:15047 2014-08-11 15:29:51.378668 2014-08-13 17:38:02.843465 7fe9bae887a0 20 read_log 192834'14226 (192834'14222) delete 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:15051 2014-08-11 15:29:51.382346 2014-08-13 17:38:02.843477 7fe9bae887a0 20 read_log 192834'14230 (192834'13907) clean cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:15055 2014-08-11 15:29:51.384125 2014-08-13 17:38:02.843489 7fe9bae887a0 20 read_log 192834'14234 (192834'14230) delete cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:15059 2014-08-11 15:29:51.387927 2014-08-13 17:38:02.843501 7fe9bae887a0 20 read_log 192834'14238 (192834'13899) clean db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:15063 2014-08-11 15:29:51.389911 2014-08-13 17:38:02.843513 7fe9bae887a0 20 read_log 192834'14239 (192834'14238) delete db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:15064 2014-08-11 15:29:56.390254 2014-08-13 17:38:02.843527 7fe9bae887a0 20 read_log 192888'14240 (192888'14240) modify e2/hit_set_24.e2_archive_2014-08-11 16:02:27.891633_2014-08-11 16:44:58.212047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.212323 2014-08-13 17:38:02.843558 7fe9bae887a0 20 read_log 192888'14241 (0'0) promote 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by osd.346.0:15307 2014-08-11 16:44:58.220761 2014-08-13 17:38:02.843573 7fe9bae887a0 20 read_log 192888'14242 (192888'14241) modify 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.1315259.0:772 2014-08-11 16:44:58.208320 2014-08-13 17:38:02.843586 7fe9bae887a0 20 read_log 192888'14243 (0'0) promote 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by osd.346.0:15321 2014-08-11 16:45:01.382128 2014-08-13 17:38:02.843598 7fe9bae887a0 20 read_log 192888'14244 (192888'14243) modify 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.1315259.0:1597 2014-08-11 16:45:01.375256 2014-08-13 17:38:02.843610 7fe9bae887a0 20 read_log 192888'14249 (0'0) promote e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by osd.346.0:15341 2014-08-11 16:45:07.924623 2014-08-13 17:38:02.843622 7fe9bae887a0 20 read_log 192888'14250 (192888'14249) modify e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.1315259.0:3320 2014-08-11 16:45:07.916878 2014-08-13 17:38:02.843634 7fe9bae887a0 20 read_log 192888'14253 (0'0) promote ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by osd.346.0:15347 2014-08-11 16:45:09.148299 2014-08-13 17:38:02.843646 7fe9bae887a0 20 read_log 192888'14254 (192888'14253) modify ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.1315259.0:3645 2014-08-11 16:45:09.140922 2014-08-13 17:38:02.843659 7fe9bae887a0 20 read_log 192888'14259 (0'0) promote 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by osd.346.0:15357 2014-08-11 16:45:13.563006 2014-08-13 17:38:02.843670 7fe9bae887a0 20 read_log 192888'14260 (192888'14259) modify 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.1315259.0:4854 2014-08-11 16:45:13.555397 2014-08-13 17:38:02.843685 7fe9bae887a0 20 read_log 192888'14265 (0'0) promote 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by osd.346.0:15377 2014-08-11 16:45:22.666173 2014-08-13 17:38:02.843697 7fe9bae887a0 20 read_log 192888'14266 (192888'14265) modify 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.1315259.0:7350 2014-08-11 16:45:22.657483 2014-08-13 17:38:02.843721 7fe9bae887a0 20 read_log 192888'14275 (0'0) promote 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by osd.346.0:15405 2014-08-11 16:45:33.022348 2014-08-13 17:38:02.843736 7fe9bae887a0 20 read_log 192888'14276 (192888'14275) modify 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.1315259.0:10101 2014-08-11 16:45:33.014965 2014-08-13 17:38:02.843750 7fe9bae887a0 20 read_log 192888'14279 (0'0) promote 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by osd.346.0:15409 2014-08-11 16:45:33.833103 2014-08-13 17:38:02.843764 7fe9bae887a0 20 read_log 192888'14280 (192888'14279) modify 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.1315259.0:10313 2014-08-11 16:45:33.823651 2014-08-13 17:38:02.843776 7fe9bae887a0 20 read_log 192888'14281 (0'0) promote 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by osd.346.0:15421 2014-08-11 16:45:36.493103 2014-08-13 17:38:02.843788 7fe9bae887a0 20 read_log 192888'14282 (192888'14281) modify 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.1315259.0:10969 2014-08-11 16:45:36.486433 2014-08-13 17:38:02.843800 7fe9bae887a0 20 read_log 192888'14287 (0'0) promote f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by osd.346.0:15445 2014-08-11 16:45:41.424337 2014-08-13 17:38:02.843812 7fe9bae887a0 20 read_log 192888'14288 (192888'14287) modify f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.1315259.0:12189 2014-08-11 16:45:41.416352 2014-08-13 17:38:02.843826 7fe9bae887a0 20 read_log 192888'14289 (0'0) promote f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by osd.346.0:15451 2014-08-11 16:45:45.526859 2014-08-13 17:38:02.843838 7fe9bae887a0 20 read_log 192888'14290 (192888'14289) modify f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.1315259.0:13170 2014-08-11 16:45:45.519395 2014-08-13 17:38:02.843850 7fe9bae887a0 20 read_log 192888'14293 (0'0) promote 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by osd.346.0:15473 2014-08-11 16:45:51.146361 2014-08-13 17:38:02.843864 7fe9bae887a0 20 read_log 192888'14294 (192888'14293) modify 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.1315259.0:14624 2014-08-11 16:45:51.139475 2014-08-13 17:38:02.843877 7fe9bae887a0 20 read_log 192888'14297 (0'0) promote adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by osd.346.0:15479 2014-08-11 16:45:52.323637 2014-08-13 17:38:02.843889 7fe9bae887a0 20 read_log 192888'14298 (192888'14297) modify adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.1315259.0:14940 2014-08-11 16:45:52.317241 2014-08-13 17:38:02.843903 7fe9bae887a0 20 read_log 192888'14299 (0'0) promote 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by osd.346.0:15485 2014-08-11 16:45:54.116869 2014-08-13 17:38:02.843915 7fe9bae887a0 20 read_log 192888'14300 (192888'14299) modify 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.1315259.0:15397 2014-08-11 16:45:54.109177 2014-08-13 17:38:02.843927 7fe9bae887a0 20 read_log 192888'14301 (0'0) promote d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by osd.346.0:15497 2014-08-11 16:46:00.547141 2014-08-13 17:38:02.843941 7fe9bae887a0 20 read_log 192888'14302 (192888'14301) modify d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.1315259.0:17045 2014-08-11 16:46:00.539501 2014-08-13 17:38:02.843958 7fe9bae887a0 20 read_log 192888'14303 (0'0) promote 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by osd.346.0:15501 2014-08-11 16:46:01.643980 2014-08-13 17:38:02.843970 7fe9bae887a0 20 read_log 192888'14304 (192888'14303) modify 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.1315259.0:17333 2014-08-11 16:46:01.635925 2014-08-13 17:38:02.843990 7fe9bae887a0 20 read_log 192888'14309 (0'0) promote 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by osd.346.0:15527 2014-08-11 16:46:12.187606 2014-08-13 17:38:02.844004 7fe9bae887a0 20 read_log 192888'14310 (192888'14309) modify 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.1315259.0:20017 2014-08-11 16:46:12.179805 2014-08-13 17:38:02.844016 7fe9bae887a0 20 read_log 192888'14311 (0'0) promote 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by osd.346.0:15529 2014-08-11 16:46:12.493679 2014-08-13 17:38:02.844028 7fe9bae887a0 20 read_log 192888'14312 (192888'14311) modify 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.1315259.0:20102 2014-08-11 16:46:12.485655 2014-08-13 17:38:02.844040 7fe9bae887a0 20 read_log 192888'14313 (0'0) promote 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by osd.346.0:15537 2014-08-11 16:46:14.218231 2014-08-13 17:38:02.844052 7fe9bae887a0 20 read_log 192888'14314 (192888'14313) modify 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.1315259.0:20551 2014-08-11 16:46:14.210052 2014-08-13 17:38:02.844064 7fe9bae887a0 20 read_log 192888'14317 (0'0) promote 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by osd.346.0:15551 2014-08-11 16:46:16.565124 2014-08-13 17:38:02.844077 7fe9bae887a0 20 read_log 192888'14318 (192888'14317) modify 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.1315259.0:21185 2014-08-11 16:46:16.558657 2014-08-13 17:38:02.844089 7fe9bae887a0 20 read_log 192888'14321 (0'0) promote 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by osd.346.0:15577 2014-08-11 16:46:22.099091 2014-08-13 17:38:02.844101 7fe9bae887a0 20 read_log 192888'14322 (192888'14321) modify 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.1315259.0:22613 2014-08-11 16:46:22.091347 2014-08-13 17:38:02.844113 7fe9bae887a0 20 read_log 192888'14323 (0'0) promote 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by osd.346.0:15579 2014-08-11 16:46:22.230226 2014-08-13 17:38:02.844125 7fe9bae887a0 20 read_log 192888'14324 (192888'14323) modify 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.1315259.0:22648 2014-08-11 16:46:22.223382 2014-08-13 17:38:02.844140 7fe9bae887a0 20 read_log 192888'14325 (0'0) promote 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by osd.346.0:15589 2014-08-11 16:46:25.340402 2014-08-13 17:38:02.844152 7fe9bae887a0 20 read_log 192888'14326 (192888'14325) modify 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.1315259.0:23433 2014-08-11 16:46:25.332969 2014-08-13 17:38:02.844170 7fe9bae887a0 20 read_log 192888'14327 (0'0) promote 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by osd.346.0:15593 2014-08-11 16:46:26.964728 2014-08-13 17:38:02.844183 7fe9bae887a0 20 read_log 192888'14328 (192888'14327) modify 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.1315259.0:23824 2014-08-11 16:46:26.958339 2014-08-13 17:38:02.844195 7fe9bae887a0 20 read_log 192888'14329 (0'0) promote 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by osd.346.0:15603 2014-08-11 16:46:28.923960 2014-08-13 17:38:02.844207 7fe9bae887a0 20 read_log 192888'14330 (192888'14329) modify 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.1315259.0:24343 2014-08-11 16:46:28.916433 2014-08-13 17:38:02.844219 7fe9bae887a0 20 read_log 192888'14343 (0'0) promote 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by osd.346.0:15641 2014-08-11 16:46:38.921893 2014-08-13 17:38:02.844231 7fe9bae887a0 20 read_log 192888'14344 (192888'14343) modify 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.1315259.0:26763 2014-08-11 16:46:38.913145 2014-08-13 17:38:02.844250 7fe9bae887a0 20 read_log 192888'14345 (0'0) promote 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by osd.346.0:15649 2014-08-11 16:46:43.418819 2014-08-13 17:38:02.844263 7fe9bae887a0 20 read_log 192888'14346 (192888'14345) modify 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.1315259.0:27861 2014-08-11 16:46:43.411757 2014-08-13 17:38:02.844275 7fe9bae887a0 20 read_log 192888'14353 (0'0) promote e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by osd.346.0:15663 2014-08-11 16:46:47.791083 2014-08-13 17:38:02.844287 7fe9bae887a0 20 read_log 192888'14354 (192888'14353) modify e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.1315259.0:28939 2014-08-11 16:46:47.783903 2014-08-13 17:38:02.844299 7fe9bae887a0 20 read_log 192888'14355 (0'0) promote e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by osd.346.0:15665 2014-08-11 16:46:48.098995 2014-08-13 17:38:02.844310 7fe9bae887a0 20 read_log 192888'14356 (192888'14355) modify e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.1315259.0:29020 2014-08-11 16:46:48.091124 2014-08-13 17:38:02.844324 7fe9bae887a0 20 read_log 192888'14357 (0'0) promote be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by osd.346.0:15667 2014-08-11 16:46:48.535653 2014-08-13 17:38:02.844335 7fe9bae887a0 20 read_log 192888'14358 (192888'14357) modify be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.1315259.0:29135 2014-08-11 16:46:48.528348 2014-08-13 17:38:02.844347 7fe9bae887a0 20 read_log 192888'14359 (0'0) promote 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by osd.346.0:15671 2014-08-11 16:46:51.597242 2014-08-13 17:38:02.844359 7fe9bae887a0 20 read_log 192888'14360 (192888'14359) modify 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.1315259.0:29861 2014-08-11 16:46:51.590119 2014-08-13 17:38:02.844371 7fe9bae887a0 20 read_log 192888'14363 (192888'14242) modify 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.1315259.0:31499 2014-08-11 16:46:56.112604 2014-08-13 17:38:02.844401 7fe9bae887a0 20 read_log 192888'14364 (192888'14244) modify 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.1315259.0:32324 2014-08-11 16:46:56.347881 2014-08-13 17:38:02.844414 7fe9bae887a0 20 read_log 192888'14367 (192888'14250) modify e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.1315259.0:34047 2014-08-11 16:46:56.791363 2014-08-13 17:38:02.844426 7fe9bae887a0 20 read_log 192888'14369 (192888'14254) modify ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.1315259.0:34372 2014-08-11 16:46:56.876738 2014-08-13 17:38:02.844439 7fe9bae887a0 20 read_log 192888'14372 (192888'14260) modify 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.1315259.0:35581 2014-08-11 16:46:57.195973 2014-08-13 17:38:02.844451 7fe9bae887a0 20 read_log 192888'14375 (192888'14266) modify 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.1315259.0:38077 2014-08-11 16:46:57.851185 2014-08-13 17:38:02.844462 7fe9bae887a0 20 read_log 192888'14380 (192888'14276) modify 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.1315259.0:40828 2014-08-11 16:46:58.591115 2014-08-13 17:38:02.844474 7fe9bae887a0 20 read_log 192888'14382 (192888'14280) modify 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.1315259.0:41040 2014-08-11 16:46:58.643844 2014-08-13 17:38:02.844486 7fe9bae887a0 20 read_log 192888'14383 (192888'14282) modify 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.1315259.0:41696 2014-08-11 16:46:58.812860 2014-08-13 17:38:02.844498 7fe9bae887a0 20 read_log 192888'14386 (192888'14288) modify f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.1315259.0:42916 2014-08-11 16:46:59.140411 2014-08-13 17:38:02.844519 7fe9bae887a0 20 read_log 192888'14387 (192888'14290) modify f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.1315259.0:43897 2014-08-11 16:46:59.421663 2014-08-13 17:38:02.844553 7fe9bae887a0 20 read_log 192888'14389 (192888'14294) modify 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.1315259.0:45351 2014-08-11 16:46:59.844300 2014-08-13 17:38:02.844565 7fe9bae887a0 20 read_log 192888'14391 (192888'14298) modify adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.1315259.0:45667 2014-08-11 16:46:59.936500 2014-08-13 17:38:02.844577 7fe9bae887a0 20 read_log 192888'14392 (192888'14300) modify 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.1315259.0:46124 2014-08-11 16:47:00.066566 2014-08-13 17:38:02.844590 7fe9bae887a0 20 read_log 192888'14393 (192888'14302) modify d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.1315259.0:47772 2014-08-11 16:47:00.561556 2014-08-13 17:38:02.844602 7fe9bae887a0 20 read_log 192888'14394 (192888'14304) modify 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.1315259.0:48060 2014-08-11 16:47:00.657103 2014-08-13 17:38:02.844616 7fe9bae887a0 20 read_log 192888'14397 (192888'14310) modify 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.1315259.0:50744 2014-08-11 16:47:01.463338 2014-08-13 17:38:02.844628 7fe9bae887a0 20 read_log 192888'14398 (192888'14312) modify 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.1315259.0:50829 2014-08-11 16:47:01.492323 2014-08-13 17:38:02.844640 7fe9bae887a0 20 read_log 192888'14399 (192888'14314) modify 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.1315259.0:51278 2014-08-11 16:47:01.632857 2014-08-13 17:38:02.844652 7fe9bae887a0 20 read_log 192888'14401 (192888'14318) modify 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.1315259.0:51912 2014-08-11 16:47:01.818802 2014-08-13 17:38:02.844664 7fe9bae887a0 20 read_log 192888'14403 (192888'14322) modify 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.1315259.0:53340 2014-08-11 16:47:02.224993 2014-08-13 17:38:02.844676 7fe9bae887a0 20 read_log 192888'14404 (192888'14324) modify 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.1315259.0:53375 2014-08-11 16:47:02.236239 2014-08-13 17:38:02.844688 7fe9bae887a0 20 read_log 192888'14405 (192888'14326) modify 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.1315259.0:54160 2014-08-11 16:47:02.484343 2014-08-13 17:38:02.844700 7fe9bae887a0 20 read_log 192888'14406 (192888'14328) modify 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.1315259.0:54551 2014-08-11 16:47:02.598721 2014-08-13 17:38:02.844711 7fe9bae887a0 20 read_log 192888'14407 (192888'14330) modify 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.1315259.0:55070 2014-08-11 16:47:02.802046 2014-08-13 17:38:02.844723 7fe9bae887a0 20 read_log 192888'14414 (192888'14344) modify 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.1315259.0:57490 2014-08-11 16:47:03.528529 2014-08-13 17:38:02.844735 7fe9bae887a0 20 read_log 192888'14415 (192888'14346) modify 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.1315259.0:58588 2014-08-11 16:47:03.847109 2014-08-13 17:38:02.844747 7fe9bae887a0 20 read_log 192888'14419 (192888'14354) modify e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.1315259.0:59666 2014-08-11 16:47:04.200732 2014-08-13 17:38:02.844759 7fe9bae887a0 20 read_log 192888'14420 (192888'14356) modify e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.1315259.0:59747 2014-08-11 16:47:04.229343 2014-08-13 17:38:02.844770 7fe9bae887a0 20 read_log 192888'14421 (192888'14358) modify be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.1315259.0:59862 2014-08-11 16:47:04.264084 2014-08-13 17:38:02.844790 7fe9bae887a0 20 read_log 192888'14422 (192888'14360) modify 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.1315259.0:60588 2014-08-11 16:47:04.484567 2014-08-13 17:38:02.844803 7fe9bae887a0 20 read_log 192888'14424 (192888'14424) modify e2/hit_set_24.e2_archive_2014-08-11 16:44:58.212327_2014-08-11 17:05:19.923836/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:19.924066 2014-08-13 17:38:02.844815 7fe9bae887a0 20 read_log 192888'14425 (192888'14240) delete e2/hit_set_24.e2_archive_2014-08-11 16:02:27.891633_2014-08-11 16:44:58.212047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:19.924066 2014-08-13 17:38:02.844827 7fe9bae887a0 20 read_log 192888'14426 (0'0) promote cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by osd.346.0:15884 2014-08-11 17:05:19.927328 2014-08-13 17:38:02.844839 7fe9bae887a0 20 read_log 192888'14427 (192888'14426) modify cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.942868.0:1189 2014-08-11 17:05:19.916121 2014-08-13 17:38:02.844851 7fe9bae887a0 20 read_log 192888'14428 (0'0) promote 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by osd.346.0:15900 2014-08-11 17:05:25.816168 2014-08-13 17:38:02.844864 7fe9bae887a0 20 read_log 192888'14429 (192888'14428) modify 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.942868.0:2668 2014-08-11 17:05:25.810896 2014-08-13 17:38:02.844876 7fe9bae887a0 20 read_log 192888'14430 (0'0) promote c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by osd.346.0:15902 2014-08-11 17:05:26.175331 2014-08-13 17:38:02.844888 7fe9bae887a0 20 read_log 192888'14431 (192888'14430) modify c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.942868.0:2765 2014-08-11 17:05:26.169086 2014-08-13 17:38:02.844900 7fe9bae887a0 20 read_log 192888'14434 (0'0) promote fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by osd.346.0:15930 2014-08-11 17:05:35.185393 2014-08-13 17:38:02.844912 7fe9bae887a0 20 read_log 192888'14435 (192888'14434) modify fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.942868.0:5149 2014-08-11 17:05:35.175804 2014-08-13 17:38:02.844924 7fe9bae887a0 20 read_log 192888'14436 (0'0) promote 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by osd.346.0:15932 2014-08-11 17:05:36.714196 2014-08-13 17:38:02.844936 7fe9bae887a0 20 read_log 192888'14437 (192888'14436) modify 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.942868.0:5559 2014-08-11 17:05:36.704368 2014-08-13 17:38:02.844948 7fe9bae887a0 20 read_log 192888'14442 (0'0) promote e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by osd.346.0:15942 2014-08-11 17:05:44.678615 2014-08-13 17:38:02.844960 7fe9bae887a0 20 read_log 192888'14443 (192888'14442) modify e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.942868.0:7682 2014-08-11 17:05:44.671600 2014-08-13 17:38:02.844972 7fe9bae887a0 20 read_log 192888'14444 (0'0) promote d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by osd.346.0:15944 2014-08-11 17:05:45.365194 2014-08-13 17:38:02.844984 7fe9bae887a0 20 read_log 192888'14445 (192888'14444) modify d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.942868.0:7872 2014-08-11 17:05:45.359365 2014-08-13 17:38:02.844996 7fe9bae887a0 20 read_log 192888'14446 (0'0) promote 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by osd.346.0:15948 2014-08-11 17:05:47.548916 2014-08-13 17:38:02.845007 7fe9bae887a0 20 read_log 192888'14447 (192888'14446) modify 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.942868.0:8477 2014-08-11 17:05:47.542121 2014-08-13 17:38:02.845041 7fe9bae887a0 20 read_log 192888'14448 (0'0) promote 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by osd.346.0:15968 2014-08-11 17:05:55.414664 2014-08-13 17:38:02.845057 7fe9bae887a0 20 read_log 192888'14449 (192888'14448) modify 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.942868.0:10614 2014-08-11 17:05:55.407532 2014-08-13 17:38:02.845070 7fe9bae887a0 20 read_log 192888'14450 (0'0) promote 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by osd.346.0:15972 2014-08-11 17:05:57.183003 2014-08-13 17:38:02.845082 7fe9bae887a0 20 read_log 192888'14451 (192888'14450) modify 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.942868.0:11108 2014-08-11 17:05:57.175916 2014-08-13 17:38:02.845094 7fe9bae887a0 20 read_log 192888'14454 (0'0) promote 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by osd.346.0:15990 2014-08-11 17:06:04.384060 2014-08-13 17:38:02.845106 7fe9bae887a0 20 read_log 192888'14455 (192888'14454) modify 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.942868.0:13054 2014-08-11 17:06:04.377479 2014-08-13 17:38:02.845118 7fe9bae887a0 20 read_log 192888'14456 (0'0) promote be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by osd.346.0:16012 2014-08-11 17:06:08.689581 2014-08-13 17:38:02.845130 7fe9bae887a0 20 read_log 192888'14457 (192888'14456) modify be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.942868.0:14113 2014-08-11 17:06:08.681754 2014-08-13 17:38:02.845142 7fe9bae887a0 20 read_log 192888'14458 (0'0) promote 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by osd.346.0:16014 2014-08-11 17:06:08.748006 2014-08-13 17:38:02.845158 7fe9bae887a0 20 read_log 192888'14459 (192888'14458) modify 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.942868.0:14127 2014-08-11 17:06:08.740653 2014-08-13 17:38:02.845171 7fe9bae887a0 20 read_log 192888'14460 (0'0) promote 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by osd.346.0:16020 2014-08-11 17:06:10.215158 2014-08-13 17:38:02.845183 7fe9bae887a0 20 read_log 192888'14461 (192888'14460) modify 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.942868.0:14492 2014-08-11 17:06:10.207604 2014-08-13 17:38:02.845195 7fe9bae887a0 20 read_log 192888'14466 (0'0) promote dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by osd.346.0:16032 2014-08-11 17:06:13.254358 2014-08-13 17:38:02.845207 7fe9bae887a0 20 read_log 192888'14467 (192888'14466) modify dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.942868.0:15264 2014-08-11 17:06:13.246262 2014-08-13 17:38:02.845219 7fe9bae887a0 20 read_log 192888'14468 (0'0) promote 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by osd.346.0:16034 2014-08-11 17:06:13.311533 2014-08-13 17:38:02.845232 7fe9bae887a0 20 read_log 192888'14469 (192888'14468) modify 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.942868.0:15279 2014-08-11 17:06:13.304203 2014-08-13 17:38:02.845244 7fe9bae887a0 20 read_log 192888'14480 (0'0) promote 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by osd.346.0:16086 2014-08-11 17:06:30.339435 2014-08-13 17:38:02.845256 7fe9bae887a0 20 read_log 192888'14481 (192888'14480) modify 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.942868.0:19729 2014-08-11 17:06:30.332446 2014-08-13 17:38:02.845268 7fe9bae887a0 20 read_log 192888'14484 (0'0) promote 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by osd.346.0:16090 2014-08-11 17:06:31.578336 2014-08-13 17:38:02.845280 7fe9bae887a0 20 read_log 192888'14485 (192888'14484) modify 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.942868.0:20058 2014-08-11 17:06:31.571838 2014-08-13 17:38:02.845301 7fe9bae887a0 20 read_log 192888'14488 (0'0) promote 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by osd.346.0:16096 2014-08-11 17:06:34.447418 2014-08-13 17:38:02.845314 7fe9bae887a0 20 read_log 192888'14489 (192888'14488) modify 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.942868.0:20800 2014-08-11 17:06:34.437522 2014-08-13 17:38:02.845326 7fe9bae887a0 20 read_log 192888'14490 (0'0) promote 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by osd.346.0:16104 2014-08-11 17:06:36.138613 2014-08-13 17:38:02.845338 7fe9bae887a0 20 read_log 192888'14491 (192888'14490) modify 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.942868.0:21238 2014-08-11 17:06:36.131166 2014-08-13 17:38:02.845350 7fe9bae887a0 20 read_log 192888'14492 (0'0) promote d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by osd.346.0:16112 2014-08-11 17:06:37.549262 2014-08-13 17:38:02.845362 7fe9bae887a0 20 read_log 192888'14493 (192888'14492) modify d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.942868.0:21618 2014-08-11 17:06:37.541317 2014-08-13 17:38:02.845375 7fe9bae887a0 20 read_log 192888'14498 (0'0) promote c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by osd.346.0:16136 2014-08-11 17:06:42.384692 2014-08-13 17:38:02.845387 7fe9bae887a0 20 read_log 192888'14499 (192888'14498) modify c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.942868.0:22873 2014-08-11 17:06:42.378050 2014-08-13 17:38:02.845399 7fe9bae887a0 20 read_log 192888'14500 (0'0) promote 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by osd.346.0:16138 2014-08-11 17:06:44.798455 2014-08-13 17:38:02.845411 7fe9bae887a0 20 read_log 192888'14501 (192888'14500) modify 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.942868.0:23481 2014-08-11 17:06:44.791283 2014-08-13 17:38:02.845423 7fe9bae887a0 20 read_log 192888'14502 (0'0) promote d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by osd.346.0:16140 2014-08-11 17:06:45.184296 2014-08-13 17:38:02.845435 7fe9bae887a0 20 read_log 192888'14503 (192888'14502) modify d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.942868.0:23587 2014-08-11 17:06:45.177357 2014-08-13 17:38:02.845448 7fe9bae887a0 20 read_log 192888'14508 (0'0) promote 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by osd.346.0:16158 2014-08-11 17:06:49.433511 2014-08-13 17:38:02.845460 7fe9bae887a0 20 read_log 192888'14509 (192888'14508) modify 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.942868.0:24735 2014-08-11 17:06:49.425851 2014-08-13 17:38:02.845472 7fe9bae887a0 20 read_log 192888'14516 (0'0) promote 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by osd.346.0:16184 2014-08-11 17:06:54.705552 2014-08-13 17:38:02.845483 7fe9bae887a0 20 read_log 192888'14517 (192888'14516) modify 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.942868.0:26070 2014-08-11 17:06:54.697484 2014-08-13 17:38:02.845495 7fe9bae887a0 20 read_log 192888'14522 (0'0) promote 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by osd.346.0:16208 2014-08-11 17:07:05.853970 2014-08-13 17:38:02.845508 7fe9bae887a0 20 read_log 192888'14523 (192888'14522) modify 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.942868.0:29048 2014-08-11 17:07:05.847289 2014-08-13 17:38:02.845520 7fe9bae887a0 20 read_log 192888'14526 (0'0) promote 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by osd.346.0:16228 2014-08-11 17:07:12.030187 2014-08-13 17:38:02.845532 7fe9bae887a0 20 read_log 192888'14527 (192888'14526) modify 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.942868.0:30553 2014-08-11 17:07:12.023268 2014-08-13 17:38:02.845551 7fe9bae887a0 20 read_log 192888'14528 (0'0) promote 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by osd.346.0:16234 2014-08-11 17:07:14.438998 2014-08-13 17:38:02.845563 7fe9bae887a0 20 read_log 192888'14529 (192888'14528) modify 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.942868.0:31166 2014-08-11 17:07:14.432020 2014-08-13 17:38:02.845575 7fe9bae887a0 20 read_log 192888'14530 (192888'14427) modify cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.942868.0:32455 2014-08-11 17:07:16.259453 2014-08-13 17:38:02.845587 7fe9bae887a0 20 read_log 192888'14531 (192888'14429) modify 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.942868.0:33934 2014-08-11 17:07:16.651909 2014-08-13 17:38:02.845600 7fe9bae887a0 20 read_log 192888'14532 (192888'14431) modify c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.942868.0:34031 2014-08-11 17:07:16.679184 2014-08-13 17:38:02.845631 7fe9bae887a0 20 read_log 192888'14534 (192888'14435) modify fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.942868.0:36415 2014-08-11 17:07:17.321712 2014-08-13 17:38:02.845653 7fe9bae887a0 20 read_log 192888'14535 (192888'14437) modify 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.942868.0:36825 2014-08-11 17:07:17.443173 2014-08-13 17:38:02.845666 7fe9bae887a0 20 read_log 192888'14538 (192888'14443) modify e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.942868.0:38948 2014-08-11 17:07:18.036663 2014-08-13 17:38:02.845678 7fe9bae887a0 20 read_log 192888'14539 (192888'14445) modify d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.942868.0:39138 2014-08-11 17:07:18.094245 2014-08-13 17:38:02.845690 7fe9bae887a0 20 read_log 192888'14540 (192888'14447) modify 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.942868.0:39743 2014-08-11 17:07:18.270873 2014-08-13 17:38:02.845702 7fe9bae887a0 20 read_log 192888'14541 (192888'14449) modify 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.942868.0:41880 2014-08-11 17:07:18.847459 2014-08-13 17:38:02.845714 7fe9bae887a0 20 read_log 192888'14542 (192888'14451) modify 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.942868.0:42374 2014-08-11 17:07:18.981708 2014-08-13 17:38:02.845726 7fe9bae887a0 20 read_log 192888'14544 (192888'14455) modify 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.942868.0:44320 2014-08-11 17:07:19.534162 2014-08-13 17:38:02.845738 7fe9bae887a0 20 read_log 192888'14545 (192888'14457) modify be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.942868.0:45379 2014-08-11 17:07:19.821254 2014-08-13 17:38:02.845750 7fe9bae887a0 20 read_log 192888'14546 (192888'14459) modify 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.942868.0:45393 2014-08-11 17:07:19.826472 2014-08-13 17:38:02.845762 7fe9bae887a0 20 read_log 192888'14547 (192888'14461) modify 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.942868.0:45758 2014-08-11 17:07:19.926863 2014-08-13 17:38:02.845774 7fe9bae887a0 20 read_log 192888'14550 (192888'14467) modify dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.942868.0:46530 2014-08-11 17:07:20.149754 2014-08-13 17:38:02.845786 7fe9bae887a0 20 read_log 192888'14551 (192888'14469) modify 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.942868.0:46545 2014-08-11 17:07:20.153322 2014-08-13 17:38:02.845798 7fe9bae887a0 20 read_log 192888'14557 (192888'14481) modify 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.942868.0:50995 2014-08-11 17:07:21.496413 2014-08-13 17:38:02.845810 7fe9bae887a0 20 read_log 192888'14559 (192888'14485) modify 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.942868.0:51324 2014-08-11 17:07:21.608988 2014-08-13 17:38:02.845832 7fe9bae887a0 20 read_log 192888'14561 (192888'14489) modify 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.942868.0:52066 2014-08-11 17:07:21.831703 2014-08-13 17:38:02.845847 7fe9bae887a0 20 read_log 192888'14562 (192888'14491) modify 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.942868.0:52504 2014-08-11 17:07:21.960744 2014-08-13 17:38:02.845860 7fe9bae887a0 20 read_log 192888'14563 (192888'14493) modify d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.942868.0:52884 2014-08-11 17:07:22.079657 2014-08-13 17:38:02.845872 7fe9bae887a0 20 read_log 192888'14566 (192888'14499) modify c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.942868.0:54139 2014-08-11 17:07:22.473398 2014-08-13 17:38:02.845884 7fe9bae887a0 20 read_log 192888'14567 (192888'14501) modify 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.942868.0:54747 2014-08-11 17:07:22.645641 2014-08-13 17:38:02.845896 7fe9bae887a0 20 read_log 192888'14568 (192888'14503) modify d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.942868.0:54853 2014-08-11 17:07:22.675361 2014-08-13 17:38:02.845908 7fe9bae887a0 20 read_log 192888'14571 (192888'14509) modify 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.942868.0:56001 2014-08-11 17:07:23.021556 2014-08-13 17:38:02.845920 7fe9bae887a0 20 read_log 192888'14575 (192888'14517) modify 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.942868.0:57336 2014-08-11 17:07:23.426189 2014-08-13 17:38:02.845932 7fe9bae887a0 20 read_log 192888'14578 (192888'14523) modify 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.942868.0:60314 2014-08-11 17:07:24.305112 2014-08-13 17:38:02.845986 7fe9bae887a0 20 read_log 192888'14580 (192888'14527) modify 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.942868.0:61819 2014-08-11 17:07:24.757652 2014-08-13 17:38:02.846004 7fe9bae887a0 20 read_log 192888'14581 (192888'14529) modify 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.942868.0:62432 2014-08-11 17:07:24.945282 2014-08-13 17:38:02.846017 7fe9bae887a0 20 read_log 193305'14582 (193305'14582) modify e2/hit_set_24.e2_archive_2014-08-11 18:37:04.930706_2014-08-11 18:42:38.411584/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.411808 2014-08-13 17:38:02.846030 7fe9bae887a0 20 read_log 193305'14583 (192888'14424) delete e2/hit_set_24.e2_archive_2014-08-11 16:44:58.212327_2014-08-11 17:05:19.923836/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.411808 2014-08-13 17:38:02.846042 7fe9bae887a0 20 read_log 193305'14584 (0'0) promote cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by osd.346.0:7 2014-08-11 18:42:38.423756 2014-08-13 17:38:02.846055 7fe9bae887a0 20 read_log 193305'14585 (193305'14584) modify cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.951337.0:1397 2014-08-11 18:42:38.405792 2014-08-13 17:38:02.846067 7fe9bae887a0 20 read_log 193305'14586 (0'0) promote ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by osd.346.0:21 2014-08-11 18:42:45.510208 2014-08-13 17:38:02.846079 7fe9bae887a0 20 read_log 193305'14587 (193305'14586) modify ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.951337.0:3307 2014-08-11 18:42:45.504144 2014-08-13 17:38:02.846091 7fe9bae887a0 20 read_log 193305'14588 (0'0) promote 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by osd.346.0:30 2014-08-11 18:42:50.212274 2014-08-13 17:38:02.846103 7fe9bae887a0 20 read_log 193305'14589 (193305'14588) modify 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.951337.0:4588 2014-08-11 18:42:50.203978 2014-08-13 17:38:02.846122 7fe9bae887a0 20 read_log 193305'14590 (0'0) promote 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by osd.346.0:46 2014-08-11 18:42:57.688680 2014-08-13 17:38:02.846137 7fe9bae887a0 20 read_log 193305'14591 (193305'14590) modify 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.951337.0:6624 2014-08-11 18:42:57.681845 2014-08-13 17:38:02.846149 7fe9bae887a0 20 read_log 193305'14592 (0'0) promote 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by osd.346.0:50 2014-08-11 18:42:58.551518 2014-08-13 17:38:02.846171 7fe9bae887a0 20 read_log 193305'14593 (193305'14592) modify 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.951337.0:6864 2014-08-11 18:42:58.544688 2014-08-13 17:38:02.846183 7fe9bae887a0 20 read_log 193305'14594 (0'0) promote 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by osd.346.0:64 2014-08-11 18:43:05.758005 2014-08-13 17:38:02.846195 7fe9bae887a0 20 read_log 193305'14595 (193305'14594) modify 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.951337.0:8775 2014-08-11 18:43:05.750436 2014-08-13 17:38:02.846208 7fe9bae887a0 20 read_log 193305'14596 (0'0) promote 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by osd.346.0:68 2014-08-11 18:43:11.275728 2014-08-13 17:38:02.846220 7fe9bae887a0 20 read_log 193305'14597 (193305'14596) modify 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.951337.0:10272 2014-08-11 18:43:11.267336 2014-08-13 17:38:02.846234 7fe9bae887a0 20 read_log 193305'14598 (0'0) promote 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by osd.346.0:74 2014-08-11 18:43:13.866365 2014-08-13 17:38:02.846247 7fe9bae887a0 20 read_log 193305'14599 (193305'14598) modify 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.951337.0:10962 2014-08-11 18:43:13.859088 2014-08-13 17:38:02.846259 7fe9bae887a0 20 read_log 193305'14600 (0'0) promote b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by osd.346.0:76 2014-08-11 18:43:14.595096 2014-08-13 17:38:02.846271 7fe9bae887a0 20 read_log 193305'14601 (193305'14600) modify b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.951337.0:11141 2014-08-11 18:43:14.587300 2014-08-13 17:38:02.846283 7fe9bae887a0 20 read_log 193305'14602 (0'0) promote 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by osd.346.0:80 2014-08-11 18:43:16.010454 2014-08-13 17:38:02.846295 7fe9bae887a0 20 read_log 193305'14603 (193305'14602) modify 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.951337.0:11490 2014-08-11 18:43:16.004045 2014-08-13 17:38:02.846307 7fe9bae887a0 20 read_log 193305'14604 (0'0) promote b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by osd.346.0:86 2014-08-11 18:43:19.304544 2014-08-13 17:38:02.846319 7fe9bae887a0 20 read_log 193305'14605 (193305'14604) modify b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.951337.0:12387 2014-08-11 18:43:19.297877 2014-08-13 17:38:02.846331 7fe9bae887a0 20 read_log 193305'14606 (0'0) promote 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by osd.346.0:96 2014-08-11 18:43:21.761682 2014-08-13 17:38:02.846343 7fe9bae887a0 20 read_log 193305'14607 (193305'14606) modify 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.951337.0:13035 2014-08-11 18:43:21.753652 2014-08-13 17:38:02.846355 7fe9bae887a0 20 read_log 193305'14608 (0'0) promote 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by osd.346.0:102 2014-08-11 18:43:22.758827 2014-08-13 17:38:02.846367 7fe9bae887a0 20 read_log 193305'14609 (193305'14608) modify 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.951337.0:13304 2014-08-11 18:43:22.751632 2014-08-13 17:38:02.846386 7fe9bae887a0 20 read_log 193305'14610 (0'0) promote d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by osd.346.0:116 2014-08-11 18:43:29.178749 2014-08-13 17:38:02.846398 7fe9bae887a0 20 read_log 193305'14611 (193305'14610) modify d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.951337.0:15061 2014-08-11 18:43:29.170910 2014-08-13 17:38:02.846410 7fe9bae887a0 20 read_log 193305'14612 (0'0) promote e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by osd.346.0:122 2014-08-11 18:43:29.937057 2014-08-13 17:38:02.846422 7fe9bae887a0 20 read_log 193305'14613 (193305'14612) modify e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.951337.0:15289 2014-08-11 18:43:29.929949 2014-08-13 17:38:02.846434 7fe9bae887a0 20 read_log 193305'14614 (0'0) promote b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by osd.346.0:136 2014-08-11 18:43:37.267541 2014-08-13 17:38:02.846446 7fe9bae887a0 20 read_log 193305'14615 (193305'14614) modify b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.951337.0:17388 2014-08-11 18:43:37.260852 2014-08-13 17:38:02.846458 7fe9bae887a0 20 read_log 193305'14616 (0'0) promote 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by osd.346.0:150 2014-08-11 18:43:43.085639 2014-08-13 17:38:02.846470 7fe9bae887a0 20 read_log 193305'14617 (193305'14616) modify 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.951337.0:19034 2014-08-11 18:43:43.077642 2014-08-13 17:38:02.846482 7fe9bae887a0 20 read_log 193305'14618 (0'0) promote 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by osd.346.0:154 2014-08-11 18:43:46.844480 2014-08-13 17:38:02.846494 7fe9bae887a0 20 read_log 193305'14619 (193305'14618) modify 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.951337.0:20062 2014-08-11 18:43:46.836360 2014-08-13 17:38:02.846505 7fe9bae887a0 20 read_log 193305'14620 (0'0) promote eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by osd.346.0:160 2014-08-11 18:43:47.959317 2014-08-13 17:38:02.846517 7fe9bae887a0 20 read_log 193305'14621 (193305'14620) modify eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.951337.0:20360 2014-08-11 18:43:47.952888 2014-08-13 17:38:02.846529 7fe9bae887a0 20 read_log 193305'14622 (0'0) promote 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by osd.346.0:162 2014-08-11 18:43:50.209000 2014-08-13 17:38:02.846541 7fe9bae887a0 20 read_log 193305'14623 (193305'14622) modify 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.951337.0:21002 2014-08-11 18:43:50.201439 2014-08-13 17:38:02.846553 7fe9bae887a0 20 read_log 193305'14624 (0'0) promote a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by osd.346.0:164 2014-08-11 18:43:50.828111 2014-08-13 17:38:02.846565 7fe9bae887a0 20 read_log 193305'14625 (193305'14624) modify a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.951337.0:21170 2014-08-11 18:43:50.819497 2014-08-13 17:38:02.846577 7fe9bae887a0 20 read_log 193305'14626 (0'0) promote e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by osd.346.0:166 2014-08-11 18:43:51.017953 2014-08-13 17:38:02.846589 7fe9bae887a0 20 read_log 193305'14627 (193305'14626) modify e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.951337.0:21224 2014-08-11 18:43:51.010872 2014-08-13 17:38:02.846601 7fe9bae887a0 20 read_log 193305'14628 (0'0) promote d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by osd.346.0:168 2014-08-11 18:43:52.095892 2014-08-13 17:38:02.846613 7fe9bae887a0 20 read_log 193305'14629 (193305'14628) modify d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.951337.0:21527 2014-08-11 18:43:52.088055 2014-08-13 17:38:02.846632 7fe9bae887a0 20 read_log 193305'14630 (0'0) promote ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by osd.346.0:170 2014-08-11 18:43:53.414794 2014-08-13 17:38:02.846644 7fe9bae887a0 20 read_log 193305'14631 (193305'14630) modify ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.951337.0:21896 2014-08-11 18:43:53.408259 2014-08-13 17:38:02.846656 7fe9bae887a0 20 read_log 193305'14632 (0'0) promote 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by osd.346.0:172 2014-08-11 18:43:55.229361 2014-08-13 17:38:02.846668 7fe9bae887a0 20 read_log 193305'14633 (193305'14632) modify 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.951337.0:22388 2014-08-11 18:43:55.222188 2014-08-13 17:38:02.846682 7fe9bae887a0 20 read_log 193305'14634 (0'0) promote cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by osd.346.0:174 2014-08-11 18:43:57.395887 2014-08-13 17:38:02.846694 7fe9bae887a0 20 read_log 193305'14635 (193305'14634) modify cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.951337.0:22987 2014-08-11 18:43:57.388819 2014-08-13 17:38:02.846706 7fe9bae887a0 20 read_log 193305'14636 (0'0) promote 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by osd.346.0:178 2014-08-11 18:44:01.287948 2014-08-13 17:38:02.846718 7fe9bae887a0 20 read_log 193305'14637 (193305'14636) modify 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.951337.0:24066 2014-08-11 18:44:01.280030 2014-08-13 17:38:02.846730 7fe9bae887a0 20 read_log 193305'14638 (0'0) promote 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by osd.346.0:188 2014-08-11 18:44:04.156740 2014-08-13 17:38:02.846742 7fe9bae887a0 20 read_log 193305'14639 (193305'14638) modify 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.951337.0:24869 2014-08-11 18:44:04.148178 2014-08-13 17:38:02.846756 7fe9bae887a0 20 read_log 193305'14640 (0'0) promote 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by osd.346.0:198 2014-08-11 18:44:12.424088 2014-08-13 17:38:02.846767 7fe9bae887a0 20 read_log 193305'14641 (193305'14640) modify 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.951337.0:27097 2014-08-11 18:44:12.415841 2014-08-13 17:38:02.846779 7fe9bae887a0 20 read_log 193305'14642 (0'0) promote 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by osd.346.0:202 2014-08-11 18:44:13.148869 2014-08-13 17:38:02.846791 7fe9bae887a0 20 read_log 193305'14643 (193305'14642) modify 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.951337.0:27295 2014-08-11 18:44:13.141203 2014-08-13 17:38:02.846803 7fe9bae887a0 20 read_log 193305'14644 (0'0) promote b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by osd.346.0:206 2014-08-11 18:44:16.038688 2014-08-13 17:38:02.846815 7fe9bae887a0 20 read_log 193305'14645 (193305'14644) modify b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.951337.0:28108 2014-08-11 18:44:16.028298 2014-08-13 17:38:02.846827 7fe9bae887a0 20 read_log 193305'14646 (0'0) promote 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by osd.346.0:239 2014-08-11 18:44:30.506208 2014-08-13 17:38:02.846839 7fe9bae887a0 20 read_log 193305'14647 (193305'14646) modify 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.951337.0:32069 2014-08-11 18:44:30.499427 2014-08-13 17:38:02.846851 7fe9bae887a0 20 read_log 193305'14648 (0'0) promote fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by osd.346.0:245 2014-08-11 18:44:32.281068 2014-08-13 17:38:02.846882 7fe9bae887a0 20 read_log 193305'14649 (193305'14648) modify fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.951337.0:32556 2014-08-11 18:44:32.273712 2014-08-13 17:38:02.846920 7fe9bae887a0 20 read_log 193305'14650 (193305'14585) modify cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.951337.0:34108 2014-08-11 18:44:34.798588 2014-08-13 17:38:02.846936 7fe9bae887a0 20 read_log 193305'14651 (193305'14587) modify ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.951337.0:36018 2014-08-11 18:44:35.876263 2014-08-13 17:38:02.846949 7fe9bae887a0 20 read_log 193305'14652 (193305'14589) modify 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.951337.0:37299 2014-08-11 18:44:36.844671 2014-08-13 17:38:02.846962 7fe9bae887a0 20 read_log 193305'14653 (193305'14591) modify 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.951337.0:39335 2014-08-11 18:44:37.952822 2014-08-13 17:38:02.846974 7fe9bae887a0 20 read_log 193305'14654 (193305'14593) modify 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.951337.0:39575 2014-08-11 18:44:38.086264 2014-08-13 17:38:02.846986 7fe9bae887a0 20 read_log 193305'14655 (193305'14595) modify 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.951337.0:41486 2014-08-11 18:44:38.768691 2014-08-13 17:38:02.846999 7fe9bae887a0 20 read_log 193305'14656 (193305'14597) modify 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.951337.0:42983 2014-08-11 18:44:39.444175 2014-08-13 17:38:02.847011 7fe9bae887a0 20 read_log 193305'14657 (193305'14599) modify 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.951337.0:43673 2014-08-11 18:44:39.661467 2014-08-13 17:38:02.847023 7fe9bae887a0 20 read_log 193305'14658 (193305'14601) modify b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.951337.0:43852 2014-08-11 18:44:39.718105 2014-08-13 17:38:02.847037 7fe9bae887a0 20 read_log 193305'14659 (193305'14603) modify 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.951337.0:44201 2014-08-11 18:44:39.820587 2014-08-13 17:38:02.847049 7fe9bae887a0 20 read_log 193305'14660 (193305'14605) modify b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.951337.0:45098 2014-08-11 18:44:40.089011 2014-08-13 17:38:02.847061 7fe9bae887a0 20 read_log 193305'14661 (193305'14607) modify 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.951337.0:45746 2014-08-11 18:44:40.292293 2014-08-13 17:38:02.847073 7fe9bae887a0 20 read_log 193305'14662 (193305'14609) modify 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.951337.0:46015 2014-08-11 18:44:40.366697 2014-08-13 17:38:02.847085 7fe9bae887a0 20 read_log 193305'14663 (193305'14611) modify d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.951337.0:47772 2014-08-11 18:44:40.861713 2014-08-13 17:38:02.847097 7fe9bae887a0 20 read_log 193305'14664 (193305'14613) modify e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.951337.0:48000 2014-08-11 18:44:40.926778 2014-08-13 17:38:02.847108 7fe9bae887a0 20 read_log 193305'14665 (193305'14615) modify b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.951337.0:50099 2014-08-11 18:44:41.565226 2014-08-13 17:38:02.847120 7fe9bae887a0 20 read_log 193305'14666 (193305'14617) modify 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.951337.0:51745 2014-08-11 18:44:42.038572 2014-08-13 17:38:02.847136 7fe9bae887a0 20 read_log 193305'14667 (193305'14619) modify 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.951337.0:52773 2014-08-11 18:44:42.333648 2014-08-13 17:38:02.847148 7fe9bae887a0 20 read_log 193305'14668 (193305'14621) modify eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.951337.0:53071 2014-08-11 18:44:42.420768 2014-08-13 17:38:02.847173 7fe9bae887a0 20 read_log 193305'14669 (193305'14623) modify 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.951337.0:53713 2014-08-11 18:44:42.610951 2014-08-13 17:38:02.847194 7fe9bae887a0 20 read_log 193305'14670 (193305'14625) modify a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.951337.0:53881 2014-08-11 18:44:42.663724 2014-08-13 17:38:02.847207 7fe9bae887a0 20 read_log 193305'14671 (193305'14627) modify e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.951337.0:53935 2014-08-11 18:44:42.678222 2014-08-13 17:38:02.847219 7fe9bae887a0 20 read_log 193305'14672 (193305'14629) modify d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.951337.0:54238 2014-08-11 18:44:42.772739 2014-08-13 17:38:02.847231 7fe9bae887a0 20 read_log 193305'14673 (193305'14631) modify ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.951337.0:54607 2014-08-11 18:44:42.887853 2014-08-13 17:38:02.847243 7fe9bae887a0 20 read_log 193305'14674 (193305'14633) modify 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.951337.0:55099 2014-08-11 18:44:43.026159 2014-08-13 17:38:02.847254 7fe9bae887a0 20 read_log 193305'14675 (193305'14635) modify cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.951337.0:55698 2014-08-11 18:44:43.204627 2014-08-13 17:38:02.847266 7fe9bae887a0 20 read_log 193305'14676 (193305'14637) modify 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.951337.0:56777 2014-08-11 18:44:43.529085 2014-08-13 17:38:02.847278 7fe9bae887a0 20 read_log 193305'14677 (193305'14639) modify 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.951337.0:57580 2014-08-11 18:44:43.798718 2014-08-13 17:38:02.847290 7fe9bae887a0 20 read_log 193305'14678 (193305'14641) modify 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.951337.0:59808 2014-08-11 18:44:44.432841 2014-08-13 17:38:02.847302 7fe9bae887a0 20 read_log 193305'14679 (193305'14643) modify 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.951337.0:60006 2014-08-11 18:44:44.492307 2014-08-13 17:38:02.847314 7fe9bae887a0 20 read_log 193305'14680 (193305'14645) modify b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.951337.0:60819 2014-08-11 18:44:44.744444 2014-08-13 17:38:02.847326 7fe9bae887a0 20 read_log 193305'14681 (193305'14647) modify 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.951337.0:64780 2014-08-11 18:44:45.915738 2014-08-13 17:38:02.847337 7fe9bae887a0 20 read_log 193305'14682 (193305'14649) modify fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.951337.0:65267 2014-08-11 18:44:46.069420 2014-08-13 17:38:02.847349 7fe9bae887a0 20 read_log 194215'14683 (194215'14683) modify e2/hit_set_24.e2_archive_2014-08-12 15:32:25.865165_2014-08-12 15:39:39.946124/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:39.946382 2014-08-13 17:38:02.847362 7fe9bae887a0 20 read_log 194215'14684 (193305'14582) delete e2/hit_set_24.e2_archive_2014-08-11 18:37:04.930706_2014-08-11 18:42:38.411584/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:39.946382 2014-08-13 17:38:02.847374 7fe9bae887a0 20 read_log 194215'14685 (192888'14398) clean 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by osd.346.0:958 2014-08-12 15:39:39.951822 2014-08-13 17:38:02.847386 7fe9bae887a0 20 read_log 194215'14686 (194215'14685) delete 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.2174595.0:46101 0.000000 2014-08-13 17:38:02.847397 7fe9bae887a0 20 read_log 194215'14687 (192888'14563) clean d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by osd.346.0:960 2014-08-12 15:39:39.972593 2014-08-13 17:38:02.847409 7fe9bae887a0 20 read_log 194215'14688 (194215'14687) delete d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.2174595.0:46103 0.000000 2014-08-13 17:38:02.847430 7fe9bae887a0 20 read_log 194215'14689 (193305'14672) clean d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by osd.346.0:962 2014-08-12 15:39:39.980161 2014-08-13 17:38:02.847443 7fe9bae887a0 20 read_log 194215'14690 (194215'14689) delete d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.2174595.0:46105 0.000000 2014-08-13 17:38:02.847454 7fe9bae887a0 20 read_log 194215'14691 (192888'14530) clean cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by osd.346.0:964 2014-08-12 15:39:39.987619 2014-08-13 17:38:02.847466 7fe9bae887a0 20 read_log 194215'14692 (194215'14691) delete cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.2174595.0:46107 0.000000 2014-08-13 17:38:02.847476 7fe9bae887a0 20 read_log 194215'14693 (193305'14659) clean 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by osd.346.0:966 2014-08-12 15:39:39.994868 2014-08-13 17:38:02.847488 7fe9bae887a0 20 read_log 194215'14694 (194215'14693) delete 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.2174595.0:46109 0.000000 2014-08-13 17:38:02.847499 7fe9bae887a0 20 read_log 194215'14695 (193305'14671) clean e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by osd.346.0:968 2014-08-12 15:39:40.001584 2014-08-13 17:38:02.847511 7fe9bae887a0 20 read_log 194215'14696 (194215'14695) delete e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.2174595.0:46111 0.000000 2014-08-13 17:38:02.847522 7fe9bae887a0 20 read_log 194215'14697 (192888'14389) clean 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by osd.346.0:970 2014-08-12 15:39:40.008683 2014-08-13 17:38:02.847534 7fe9bae887a0 20 read_log 194215'14698 (194215'14697) delete 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.2174595.0:46113 0.000000 2014-08-13 17:38:02.847544 7fe9bae887a0 20 read_log 194215'14699 (193305'14675) clean cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by osd.346.0:972 2014-08-12 15:39:40.015678 2014-08-13 17:38:02.847556 7fe9bae887a0 20 read_log 194215'14700 (194215'14699) delete cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.2174595.0:46115 0.000000 2014-08-13 17:38:02.847567 7fe9bae887a0 20 read_log 194215'14701 (192888'14534) clean fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by osd.346.0:974 2014-08-12 15:39:40.022506 2014-08-13 17:38:02.847579 7fe9bae887a0 20 read_log 194215'14702 (194215'14701) delete fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.2174595.0:46117 0.000000 2014-08-13 17:38:02.847589 7fe9bae887a0 20 read_log 194215'14703 (193305'14670) clean a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by osd.346.0:976 2014-08-12 15:39:40.029553 2014-08-13 17:38:02.847601 7fe9bae887a0 20 read_log 194215'14704 (194215'14703) delete a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.2174595.0:46119 0.000000 2014-08-13 17:38:02.847612 7fe9bae887a0 20 read_log 194215'14705 (193305'14652) clean 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by osd.346.0:978 2014-08-12 15:39:40.036619 2014-08-13 17:38:02.847624 7fe9bae887a0 20 read_log 194215'14706 (194215'14705) delete 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.2174595.0:46121 0.000000 2014-08-13 17:38:02.847635 7fe9bae887a0 20 read_log 194215'14707 (192888'14567) clean 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by osd.346.0:980 2014-08-12 15:39:40.044419 2014-08-13 17:38:02.847647 7fe9bae887a0 20 read_log 194215'14708 (194215'14707) delete 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.2174595.0:46123 0.000000 2014-08-13 17:38:02.847665 7fe9bae887a0 20 read_log 194215'14709 (192888'14561) clean 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by osd.346.0:982 2014-08-12 15:39:40.051223 2014-08-13 17:38:02.847677 7fe9bae887a0 20 read_log 194215'14710 (194215'14709) delete 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.2174595.0:46125 0.000000 2014-08-13 17:38:02.847688 7fe9bae887a0 20 read_log 194215'14711 (192888'14420) clean e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by osd.346.0:984 2014-08-12 15:39:40.058362 2014-08-13 17:38:02.847700 7fe9bae887a0 20 read_log 194215'14712 (194215'14711) delete e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.2174595.0:46127 0.000000 2014-08-13 17:38:02.847710 7fe9bae887a0 20 read_log 194215'14713 (193305'14676) clean 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by osd.346.0:986 2014-08-12 15:39:40.065305 2014-08-13 17:38:02.847722 7fe9bae887a0 20 read_log 194215'14714 (194215'14713) delete 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.2174595.0:46129 0.000000 2014-08-13 17:38:02.847733 7fe9bae887a0 20 read_log 194215'14715 (192888'14380) clean 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by osd.346.0:988 2014-08-12 15:39:40.072547 2014-08-13 17:38:02.847745 7fe9bae887a0 20 read_log 194215'14716 (194215'14715) delete 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.2174595.0:46131 0.000000 2014-08-13 17:38:02.847755 7fe9bae887a0 20 read_log 194215'14717 (192888'14535) clean 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by osd.346.0:990 2014-08-12 15:39:40.079567 2014-08-13 17:38:02.847767 7fe9bae887a0 20 read_log 194215'14718 (194215'14717) delete 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.2174595.0:46133 0.000000 2014-08-13 17:38:02.847778 7fe9bae887a0 20 read_log 194215'14719 (192888'14397) clean 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by osd.346.0:992 2014-08-12 15:39:40.086495 2014-08-13 17:38:02.847790 7fe9bae887a0 20 read_log 194215'14720 (194215'14719) delete 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.2174595.0:46135 0.000000 2014-08-13 17:38:02.847801 7fe9bae887a0 20 read_log 194215'14721 (192888'14394) clean 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by osd.346.0:994 2014-08-12 15:39:40.093456 2014-08-13 17:38:02.847813 7fe9bae887a0 20 read_log 194215'14722 (194215'14721) delete 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.2174595.0:46137 0.000000 2014-08-13 17:38:02.847823 7fe9bae887a0 20 read_log 194215'14723 (192888'14551) clean 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by osd.346.0:996 2014-08-12 15:39:40.100841 2014-08-13 17:38:02.847835 7fe9bae887a0 20 read_log 194215'14724 (194215'14723) delete 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.2174595.0:46139 0.000000 2014-08-13 17:38:02.847846 7fe9bae887a0 20 read_log 194215'14725 (193305'14674) clean 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by osd.346.0:998 2014-08-12 15:39:40.107798 2014-08-13 17:38:02.847858 7fe9bae887a0 20 read_log 194215'14726 (194215'14725) delete 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.2174595.0:46141 0.000000 2014-08-13 17:38:02.847869 7fe9bae887a0 20 read_log 194215'14727 (192888'14566) clean c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by osd.346.0:1000 2014-08-12 15:39:40.114965 2014-08-13 17:38:02.847881 7fe9bae887a0 20 read_log 194215'14728 (194215'14727) delete c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.2174595.0:46143 0.000000 2014-08-13 17:38:02.847898 7fe9bae887a0 20 read_log 194215'14729 (192888'14575) clean 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by osd.346.0:1002 2014-08-12 15:39:40.122085 2014-08-13 17:38:02.847911 7fe9bae887a0 20 read_log 194215'14730 (194215'14729) delete 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.2174595.0:46145 0.000000 2014-08-13 17:38:02.847921 7fe9bae887a0 20 read_log 194215'14731 (192888'14550) clean dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by osd.346.0:1004 2014-08-12 15:39:40.128853 2014-08-13 17:38:02.847933 7fe9bae887a0 20 read_log 194215'14732 (194215'14731) delete dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.2174595.0:46147 0.000000 2014-08-13 17:38:02.847944 7fe9bae887a0 20 read_log 194215'14733 (192888'14578) clean 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by osd.346.0:1006 2014-08-12 15:39:40.136034 2014-08-13 17:38:02.847956 7fe9bae887a0 20 read_log 194215'14734 (194215'14733) delete 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.2174595.0:46149 0.000000 2014-08-13 17:38:02.847966 7fe9bae887a0 20 read_log 194215'14735 (193305'14680) clean b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by osd.346.0:1008 2014-08-12 15:39:40.143639 2014-08-13 17:38:02.847978 7fe9bae887a0 20 read_log 194215'14736 (194215'14735) delete b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.2174595.0:46151 0.000000 2014-08-13 17:38:02.847989 7fe9bae887a0 20 read_log 194215'14737 (193305'14656) clean 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by osd.346.0:1010 2014-08-12 15:39:40.151150 2014-08-13 17:38:02.848001 7fe9bae887a0 20 read_log 194215'14738 (194215'14737) delete 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.2174595.0:46153 0.000000 2014-08-13 17:38:02.848011 7fe9bae887a0 20 read_log 194215'14739 (192888'14363) clean 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by osd.346.0:1012 2014-08-12 15:39:40.158691 2014-08-13 17:38:02.848023 7fe9bae887a0 20 read_log 194215'14740 (194215'14739) delete 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.2174595.0:46155 0.000000 2014-08-13 17:38:02.848034 7fe9bae887a0 20 read_log 194215'14741 (192888'14532) clean c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by osd.346.0:1014 2014-08-12 15:39:40.166311 2014-08-13 17:38:02.848066 7fe9bae887a0 20 read_log 194215'14742 (194215'14741) delete c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.2174595.0:46157 0.000000 2014-08-13 17:38:02.848078 7fe9bae887a0 20 read_log 194215'14743 (192888'14415) clean 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by osd.346.0:1016 2014-08-12 15:39:40.173040 2014-08-13 17:38:02.848091 7fe9bae887a0 20 read_log 194215'14744 (194215'14743) delete 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.2174595.0:46159 0.000000 2014-08-13 17:38:02.848101 7fe9bae887a0 20 read_log 194215'14745 (192888'14387) clean f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by osd.346.0:1018 2014-08-12 15:39:40.180374 2014-08-13 17:38:02.848113 7fe9bae887a0 20 read_log 194215'14746 (194215'14745) delete f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.2174595.0:46161 0.000000 2014-08-13 17:38:02.848124 7fe9bae887a0 20 read_log 194215'14747 (192888'14557) clean 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by osd.346.0:1020 2014-08-12 15:39:40.186798 2014-08-13 17:38:02.848136 7fe9bae887a0 20 read_log 194215'14748 (194215'14747) delete 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.2174595.0:46163 0.000000 2014-08-13 17:38:02.848157 7fe9bae887a0 20 read_log 194215'14749 (193305'14662) clean 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by osd.346.0:1022 2014-08-12 15:39:40.193776 2014-08-13 17:38:02.848171 7fe9bae887a0 20 read_log 194215'14750 (194215'14749) delete 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.2174595.0:46165 0.000000 2014-08-13 17:38:02.848181 7fe9bae887a0 20 read_log 194215'14751 (192888'14546) clean 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by osd.346.0:1024 2014-08-12 15:39:40.200998 2014-08-13 17:38:02.848194 7fe9bae887a0 20 read_log 194215'14752 (194215'14751) delete 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.2174595.0:46167 0.000000 2014-08-13 17:38:02.848204 7fe9bae887a0 20 read_log 194215'14753 (193305'14658) clean b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by osd.346.0:1026 2014-08-12 15:39:40.207451 2014-08-13 17:38:02.848216 7fe9bae887a0 20 read_log 194215'14754 (194215'14753) delete b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.2174595.0:46169 0.000000 2014-08-13 17:38:02.848227 7fe9bae887a0 20 read_log 194215'14755 (193305'14661) clean 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by osd.346.0:1028 2014-08-12 15:39:40.214060 2014-08-13 17:38:02.848239 7fe9bae887a0 20 read_log 194215'14756 (194215'14755) delete 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.2174595.0:46171 0.000000 2014-08-13 17:38:02.848249 7fe9bae887a0 20 read_log 194215'14757 (193305'14663) clean d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by osd.346.0:1030 2014-08-12 15:39:40.220535 2014-08-13 17:38:02.848262 7fe9bae887a0 20 read_log 194215'14758 (194215'14757) delete d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.2174595.0:46173 0.000000 2014-08-13 17:38:02.848272 7fe9bae887a0 20 read_log 194215'14759 (192888'14414) clean 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by osd.346.0:1032 2014-08-12 15:39:40.226981 2014-08-13 17:38:02.848285 7fe9bae887a0 20 read_log 194215'14760 (194215'14759) delete 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.2174595.0:46175 0.000000 2014-08-13 17:38:02.848296 7fe9bae887a0 20 read_log 194215'14761 (192888'14386) clean f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by osd.346.0:1034 2014-08-12 15:39:40.233319 2014-08-13 17:38:02.848308 7fe9bae887a0 20 read_log 194215'14762 (194215'14761) delete f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.2174595.0:46177 0.000000 2014-08-13 17:38:02.848319 7fe9bae887a0 20 read_log 194215'14763 (193305'14678) clean 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by osd.346.0:1036 2014-08-12 15:39:40.240258 2014-08-13 17:38:02.848333 7fe9bae887a0 20 read_log 194215'14764 (194215'14763) delete 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.2174595.0:46179 0.000000 2014-08-13 17:38:02.848343 7fe9bae887a0 20 read_log 194215'14765 (192888'14405) clean 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by osd.346.0:1038 2014-08-12 15:39:40.246728 2014-08-13 17:38:02.848356 7fe9bae887a0 20 read_log 194215'14766 (194215'14765) delete 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.2174595.0:46181 0.000000 2014-08-13 17:38:02.848366 7fe9bae887a0 20 read_log 194215'14767 (192888'14580) clean 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by osd.346.0:1040 2014-08-12 15:39:40.253264 2014-08-13 17:38:02.848378 7fe9bae887a0 20 read_log 194215'14768 (194215'14767) delete 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.2174595.0:46183 0.000000 2014-08-13 17:38:02.848398 7fe9bae887a0 20 read_log 194215'14769 (192888'14369) clean ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by osd.346.0:1042 2014-08-12 15:39:40.259955 2014-08-13 17:38:02.848412 7fe9bae887a0 20 read_log 194215'14770 (194215'14769) delete ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.2174595.0:46185 0.000000 2014-08-13 17:38:02.848422 7fe9bae887a0 20 read_log 194215'14771 (193305'14660) clean b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by osd.346.0:1044 2014-08-12 15:39:40.267492 2014-08-13 17:38:02.848435 7fe9bae887a0 20 read_log 194215'14772 (194215'14771) delete b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.2174595.0:46187 0.000000 2014-08-13 17:38:02.848447 7fe9bae887a0 20 read_log 194215'14773 (192888'14372) clean 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by osd.346.0:1046 2014-08-12 15:39:40.275202 2014-08-13 17:38:02.848460 7fe9bae887a0 20 read_log 194215'14774 (194215'14773) delete 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.2174595.0:46189 0.000000 2014-08-13 17:38:02.848471 7fe9bae887a0 20 read_log 194215'14775 (193305'14657) clean 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by osd.346.0:1048 2014-08-12 15:39:40.283051 2014-08-13 17:38:02.848483 7fe9bae887a0 20 read_log 194215'14776 (194215'14775) delete 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.2174595.0:46191 0.000000 2014-08-13 17:38:02.848493 7fe9bae887a0 20 read_log 194215'14777 (192888'14531) clean 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by osd.346.0:1050 2014-08-12 15:39:40.289920 2014-08-13 17:38:02.848505 7fe9bae887a0 20 read_log 194215'14778 (194215'14777) delete 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.2174595.0:46193 0.000000 2014-08-13 17:38:02.848522 7fe9bae887a0 20 read_log 194215'14779 (193305'14669) clean 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by osd.346.0:1052 2014-08-12 15:39:40.297244 2014-08-13 17:38:02.848536 7fe9bae887a0 20 read_log 194215'14780 (194215'14779) delete 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.2174595.0:46195 0.000000 2014-08-13 17:38:02.848547 7fe9bae887a0 20 read_log 194215'14781 (192888'14403) clean 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by osd.346.0:1054 2014-08-12 15:39:40.304969 2014-08-13 17:38:02.848559 7fe9bae887a0 20 read_log 194215'14782 (194215'14781) delete 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.2174595.0:46197 0.000000 2014-08-13 17:38:02.848570 7fe9bae887a0 20 read_log 194215'14783 (192888'14382) clean 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by osd.346.0:1056 2014-08-12 15:39:40.312577 2014-08-13 17:38:02.848584 7fe9bae887a0 20 read_log 194215'14784 (194215'14783) delete 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.2174595.0:46199 0.000000 2014-08-13 17:38:02.848595 7fe9bae887a0 20 read_log 194215'14785 (192888'14540) clean 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by osd.346.0:1058 2014-08-12 15:39:40.319943 2014-08-13 17:38:02.848608 7fe9bae887a0 20 read_log 194215'14786 (194215'14785) delete 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.2174595.0:46201 0.000000 2014-08-13 17:38:02.848619 7fe9bae887a0 20 read_log 194215'14787 (192888'14547) clean 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by osd.346.0:1060 2014-08-12 15:39:40.326706 2014-08-13 17:38:02.848631 7fe9bae887a0 20 read_log 194215'14788 (194215'14787) delete 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.2174595.0:46203 0.000000 2014-08-13 17:38:02.848666 7fe9bae887a0 20 read_log 194215'14789 (192888'14367) clean e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by osd.346.0:1062 2014-08-12 15:39:40.333434 2014-08-13 17:38:02.848683 7fe9bae887a0 20 read_log 194215'14790 (194215'14789) delete e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.2174595.0:46205 0.000000 2014-08-13 17:38:02.848693 7fe9bae887a0 20 read_log 194215'14791 (193305'14679) clean 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by osd.346.0:1064 2014-08-12 15:39:40.339890 2014-08-13 17:38:02.848706 7fe9bae887a0 20 read_log 194215'14792 (194215'14791) delete 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.2174595.0:46207 0.000000 2014-08-13 17:38:02.848716 7fe9bae887a0 20 read_log 194215'14793 (193305'14667) clean 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by osd.346.0:1066 2014-08-12 15:39:40.346960 2014-08-13 17:38:02.848728 7fe9bae887a0 20 read_log 194215'14794 (194215'14793) delete 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.2174595.0:46209 0.000000 2014-08-13 17:38:02.848739 7fe9bae887a0 20 read_log 194215'14795 (192888'14421) clean be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by osd.346.0:1068 2014-08-12 15:39:40.354496 2014-08-13 17:38:02.848751 7fe9bae887a0 20 read_log 194215'14796 (194215'14795) delete be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.2174595.0:46211 0.000000 2014-08-13 17:38:02.848761 7fe9bae887a0 20 read_log 194215'14797 (192888'14545) clean be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by osd.346.0:1070 2014-08-12 15:39:40.360736 2014-08-13 17:38:02.848773 7fe9bae887a0 20 read_log 194215'14798 (194215'14797) delete be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.2174595.0:46213 0.000000 2014-08-13 17:38:02.848784 7fe9bae887a0 20 read_log 194215'14799 (192888'14404) clean 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by osd.346.0:1072 2014-08-12 15:39:40.367221 2014-08-13 17:38:02.848796 7fe9bae887a0 20 read_log 194215'14800 (194215'14799) delete 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.2174595.0:46215 0.000000 2014-08-13 17:38:02.848806 7fe9bae887a0 20 read_log 194215'14801 (192888'14375) clean 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by osd.346.0:1074 2014-08-12 15:39:40.374250 2014-08-13 17:38:02.848818 7fe9bae887a0 20 read_log 194215'14802 (194215'14801) delete 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.2174595.0:46217 0.000000 2014-08-13 17:38:02.848829 7fe9bae887a0 20 read_log 194215'14803 (192888'14407) clean 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by osd.346.0:1076 2014-08-12 15:39:40.382136 2014-08-13 17:38:02.848841 7fe9bae887a0 20 read_log 194215'14804 (194215'14803) delete 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.2174595.0:46219 0.000000 2014-08-13 17:38:02.848851 7fe9bae887a0 20 read_log 194215'14805 (193305'14664) clean e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by osd.346.0:1078 2014-08-12 15:39:40.389037 2014-08-13 17:38:02.848864 7fe9bae887a0 20 read_log 194215'14806 (194215'14805) delete e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.2174595.0:46221 0.000000 2014-08-13 17:38:02.848874 7fe9bae887a0 20 read_log 194215'14807 (192888'14401) clean 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by osd.346.0:1080 2014-08-12 15:39:40.396351 2014-08-13 17:38:02.848886 7fe9bae887a0 20 read_log 194215'14808 (194215'14807) delete 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.2174595.0:46223 0.000000 2014-08-13 17:38:02.848903 7fe9bae887a0 20 read_log 194215'14809 (192888'14406) clean 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by osd.346.0:1082 2014-08-12 15:39:40.402863 2014-08-13 17:38:02.848915 7fe9bae887a0 20 read_log 194215'14810 (194215'14809) delete 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.2174595.0:46225 0.000000 2014-08-13 17:38:02.848926 7fe9bae887a0 20 read_log 194215'14811 (192888'14393) clean d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by osd.346.0:1084 2014-08-12 15:39:40.409621 2014-08-13 17:38:02.848938 7fe9bae887a0 20 read_log 194215'14812 (194215'14811) delete d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.2174595.0:46227 0.000000 2014-08-13 17:38:02.848948 7fe9bae887a0 20 read_log 194215'14813 (193305'14653) clean 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by osd.346.0:1086 2014-08-12 15:39:40.416490 2014-08-13 17:38:02.848960 7fe9bae887a0 20 read_log 194215'14814 (194215'14813) delete 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.2174595.0:46229 0.000000 2014-08-13 17:38:02.848971 7fe9bae887a0 20 read_log 194215'14815 (192888'14383) clean 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by osd.346.0:1088 2014-08-12 15:39:40.423247 2014-08-13 17:38:02.848983 7fe9bae887a0 20 read_log 194215'14816 (194215'14815) delete 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.2174595.0:46231 0.000000 2014-08-13 17:38:02.848993 7fe9bae887a0 20 read_log 194215'14817 (193305'14665) clean b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by osd.346.0:1090 2014-08-12 15:39:40.430513 2014-08-13 17:38:02.849005 7fe9bae887a0 20 read_log 194215'14818 (194215'14817) delete b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.2174595.0:46233 0.000000 2014-08-13 17:38:02.849016 7fe9bae887a0 20 read_log 194215'14819 (193305'14651) clean ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by osd.346.0:1092 2014-08-12 15:39:40.437357 2014-08-13 17:38:02.849028 7fe9bae887a0 20 read_log 194215'14820 (194215'14819) delete ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.2174595.0:46235 0.000000 2014-08-13 17:38:02.849038 7fe9bae887a0 20 read_log 194215'14821 (192888'14559) clean 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by osd.346.0:1094 2014-08-12 15:39:40.444192 2014-08-13 17:38:02.849050 7fe9bae887a0 20 read_log 194215'14822 (194215'14821) delete 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.2174595.0:46237 0.000000 2014-08-13 17:38:02.849060 7fe9bae887a0 20 read_log 194215'14823 (193305'14682) clean fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by osd.346.0:1096 2014-08-12 15:39:40.450910 2014-08-13 17:38:02.849072 7fe9bae887a0 20 read_log 194215'14824 (194215'14823) delete fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.2174595.0:46239 0.000000 2014-08-13 17:38:02.849083 7fe9bae887a0 20 read_log 194215'14825 (192888'14581) clean 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by osd.346.0:1098 2014-08-12 15:39:40.457344 2014-08-13 17:38:02.849095 7fe9bae887a0 20 read_log 194215'14826 (194215'14825) delete 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.2174595.0:46241 0.000000 2014-08-13 17:38:02.849106 7fe9bae887a0 20 read_log 194215'14827 (192888'14392) clean 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by osd.346.0:1100 2014-08-12 15:39:40.463974 2014-08-13 17:38:02.849117 7fe9bae887a0 20 read_log 194215'14828 (194215'14827) delete 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.2174595.0:46243 0.000000 2014-08-13 17:38:02.849137 7fe9bae887a0 20 read_log 194215'14829 (192888'14541) clean 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by osd.346.0:1102 2014-08-12 15:39:40.470955 2014-08-13 17:38:02.849149 7fe9bae887a0 20 read_log 194215'14830 (194215'14829) delete 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.2174595.0:46245 0.000000 2014-08-13 17:38:02.849165 7fe9bae887a0 20 read_log 194215'14831 (192888'14571) clean 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by osd.346.0:1104 2014-08-12 15:39:40.478529 2014-08-13 17:38:02.849178 7fe9bae887a0 20 read_log 194215'14832 (194215'14831) delete 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.2174595.0:46247 0.000000 2014-08-13 17:38:02.849189 7fe9bae887a0 20 read_log 194215'14833 (193305'14677) clean 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by osd.346.0:1106 2014-08-12 15:39:40.485891 2014-08-13 17:38:02.849201 7fe9bae887a0 20 read_log 194215'14834 (194215'14833) delete 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.2174595.0:46249 0.000000 2014-08-13 17:38:02.849213 7fe9bae887a0 20 read_log 194215'14835 (193305'14681) clean 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by osd.346.0:1108 2014-08-12 15:39:40.493040 2014-08-13 17:38:02.849225 7fe9bae887a0 20 read_log 194215'14836 (194215'14835) delete 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.2174595.0:46251 0.000000 2014-08-13 17:38:02.849236 7fe9bae887a0 20 read_log 194215'14837 (192888'14539) clean d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by osd.346.0:1110 2014-08-12 15:39:40.500779 2014-08-13 17:38:02.849248 7fe9bae887a0 20 read_log 194215'14838 (194215'14837) delete d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.2174595.0:46253 0.000000 2014-08-13 17:38:02.849259 7fe9bae887a0 20 read_log 194215'14839 (193305'14655) clean 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by osd.346.0:1112 2014-08-12 15:39:40.508536 2014-08-13 17:38:02.849271 7fe9bae887a0 20 read_log 194215'14840 (194215'14839) delete 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.2174595.0:46255 0.000000 2014-08-13 17:38:02.849281 7fe9bae887a0 20 read_log 194215'14841 (193305'14654) clean 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by osd.346.0:1114 2014-08-12 15:39:40.515965 2014-08-13 17:38:02.849293 7fe9bae887a0 20 read_log 194215'14842 (194215'14841) delete 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.2174595.0:46257 0.000000 2014-08-13 17:38:02.849303 7fe9bae887a0 20 read_log 194215'14843 (192888'14391) clean adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by osd.346.0:1116 2014-08-12 15:39:40.523549 2014-08-13 17:38:02.849315 7fe9bae887a0 20 read_log 194215'14844 (194215'14843) delete adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.2174595.0:46259 0.000000 2014-08-13 17:38:02.849326 7fe9bae887a0 20 read_log 194215'14845 (193305'14650) clean cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by osd.346.0:1118 2014-08-12 15:39:40.530934 2014-08-13 17:38:02.849338 7fe9bae887a0 20 read_log 194215'14846 (194215'14845) delete cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.2174595.0:46261 0.000000 2014-08-13 17:38:02.849364 7fe9bae887a0 20 read_log 194215'14847 (192888'14562) clean 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by osd.346.0:1120 2014-08-12 15:39:40.538766 2014-08-13 17:38:02.849377 7fe9bae887a0 20 read_log 194215'14848 (194215'14847) delete 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.2174595.0:46263 0.000000 2014-08-13 17:38:02.849394 7fe9bae887a0 20 read_log 194215'14849 (192888'14568) clean d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by osd.346.0:1122 2014-08-12 15:39:40.546397 2014-08-13 17:38:02.849407 7fe9bae887a0 20 read_log 194215'14850 (194215'14849) delete d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.2174595.0:46265 0.000000 2014-08-13 17:38:02.849417 7fe9bae887a0 20 read_log 194215'14851 (192888'14399) clean 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by osd.346.0:1124 2014-08-12 15:39:40.553187 2014-08-13 17:38:02.849433 7fe9bae887a0 20 read_log 194215'14852 (194215'14851) delete 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.2174595.0:46267 0.000000 2014-08-13 17:38:02.849444 7fe9bae887a0 20 read_log 194215'14853 (192888'14419) clean e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by osd.346.0:1126 2014-08-12 15:39:40.560420 2014-08-13 17:38:02.849456 7fe9bae887a0 20 read_log 194215'14854 (194215'14853) delete e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.2174595.0:46269 0.000000 2014-08-13 17:38:02.849466 7fe9bae887a0 20 read_log 194215'14855 (192888'14422) clean 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by osd.346.0:1128 2014-08-12 15:39:40.568555 2014-08-13 17:38:02.849478 7fe9bae887a0 20 read_log 194215'14856 (194215'14855) delete 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.2174595.0:46271 0.000000 2014-08-13 17:38:02.849489 7fe9bae887a0 20 read_log 194215'14857 (193305'14666) clean 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by osd.346.0:1130 2014-08-12 15:39:40.576183 2014-08-13 17:38:02.849501 7fe9bae887a0 20 read_log 194215'14858 (194215'14857) delete 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.2174595.0:46273 0.000000 2014-08-13 17:38:02.849511 7fe9bae887a0 20 read_log 194215'14859 (193305'14668) clean eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by osd.346.0:1132 2014-08-12 15:39:40.583411 2014-08-13 17:38:02.849523 7fe9bae887a0 20 read_log 194215'14860 (194215'14859) delete eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.2174595.0:46275 0.000000 2014-08-13 17:38:02.849534 7fe9bae887a0 20 read_log 194215'14861 (192888'14538) clean e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by osd.346.0:1134 2014-08-12 15:39:40.590973 2014-08-13 17:38:02.849546 7fe9bae887a0 20 read_log 194215'14862 (194215'14861) delete e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.2174595.0:46277 0.000000 2014-08-13 17:38:02.849558 7fe9bae887a0 20 read_log 194215'14863 (193305'14673) clean ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by osd.346.0:1136 2014-08-12 15:39:40.597702 2014-08-13 17:38:02.849570 7fe9bae887a0 20 read_log 194215'14864 (194215'14863) delete ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.2174595.0:46279 0.000000 2014-08-13 17:38:02.849580 7fe9bae887a0 20 read_log 194215'14865 (192888'14544) clean 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by osd.346.0:1138 2014-08-12 15:39:40.604898 2014-08-13 17:38:02.849592 7fe9bae887a0 20 read_log 194215'14866 (194215'14865) delete 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.2174595.0:46281 0.000000 2014-08-13 17:38:02.849603 7fe9bae887a0 20 read_log 194215'14867 (192888'14542) clean 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by osd.346.0:1140 2014-08-12 15:39:40.611349 2014-08-13 17:38:02.849615 7fe9bae887a0 20 read_log 194215'14868 (194215'14867) delete 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.2174595.0:46283 0.000000 2014-08-13 17:38:02.849633 7fe9bae887a0 20 read_log 194215'14869 (192888'14364) clean 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by osd.346.0:1142 2014-08-12 15:39:40.618546 2014-08-13 17:38:02.849646 7fe9bae887a0 20 read_log 194215'14870 (194215'14869) delete 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.2174595.0:46285 0.000000 2014-08-13 17:38:02.849657 7fe9bae887a0 20 read_log 194215'14871 (194215'14871) modify e2/hit_set_24.e2_archive_2014-08-12 15:39:39.946574_2014-08-12 17:24:43.501290/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.501524 2014-08-13 17:38:02.849669 7fe9bae887a0 20 read_log 194215'14872 (194215'14683) delete e2/hit_set_24.e2_archive_2014-08-12 15:32:25.865165_2014-08-12 15:39:39.946124/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.501524 2014-08-13 17:38:02.849681 7fe9bae887a0 20 read_log 194215'14873 (0'0) promote 2d679ce2/benchmark_data_storage0111-ib.pocstor_34628_object920/head//24 by osd.346.0:3550 2014-08-12 17:24:43.506498 2014-08-13 17:38:02.849693 7fe9bae887a0 20 read_log 194215'14874 (194215'14873) modify 2d679ce2/benchmark_data_storage0111-ib.pocstor_34628_object920/head//24 by client.2193759.0:921 2014-08-12 17:24:43.495413 2014-08-13 17:38:02.849706 7fe9bae887a0 20 read_log 194215'14875 (0'0) promote e999dce2/benchmark_data_storage0111-ib.pocstor_34628_object973/head//24 by osd.346.0:3552 2014-08-12 17:24:43.695936 2014-08-13 17:38:02.849717 7fe9bae887a0 20 read_log 194215'14876 (194215'14875) modify e999dce2/benchmark_data_storage0111-ib.pocstor_34628_object973/head//24 by client.2193759.0:974 2014-08-12 17:24:43.687533 2014-08-13 17:38:02.849729 7fe9bae887a0 20 read_log 194215'14877 (0'0) promote fb8c64e2/benchmark_data_storage0111-ib.pocstor_34628_object1367/head//24 by osd.346.0:3556 2014-08-12 17:24:45.817529 2014-08-13 17:38:02.849741 7fe9bae887a0 20 read_log 194215'14878 (194215'14877) modify fb8c64e2/benchmark_data_storage0111-ib.pocstor_34628_object1367/head//24 by client.2193759.0:1368 2014-08-12 17:24:45.808540 2014-08-13 17:38:02.849754 7fe9bae887a0 20 read_log 194370'14879 (0'0) promote 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by osd.346.0:3582 2014-08-12 20:30:17.686159 2014-08-13 17:38:02.849766 7fe9bae887a0 20 read_log 194370'14880 (194370'14879) modify 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by client.1508531.0:878 2014-08-12 20:30:17.678096 2014-08-13 17:38:02.849778 7fe9bae887a0 20 read_log 194370'14881 (0'0) promote b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by osd.346.0:3602 2014-08-12 20:30:27.647062 2014-08-13 17:38:02.849789 7fe9bae887a0 20 read_log 194370'14882 (194370'14881) modify b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by client.1508531.0:3564 2014-08-12 20:30:27.639554 2014-08-13 17:38:02.849801 7fe9bae887a0 20 read_log 194370'14883 (0'0) promote 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by osd.346.0:3614 2014-08-12 20:30:33.167472 2014-08-13 17:38:02.849813 7fe9bae887a0 20 read_log 194370'14884 (194370'14883) modify 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by client.1508531.0:4984 2014-08-12 20:30:33.159380 2014-08-13 17:38:02.849825 7fe9bae887a0 20 read_log 194370'14885 (0'0) promote c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by osd.346.0:3616 2014-08-12 20:30:33.509835 2014-08-13 17:38:02.849837 7fe9bae887a0 20 read_log 194370'14886 (194370'14885) modify c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by client.1508531.0:5058 2014-08-12 20:30:33.502529 2014-08-13 17:38:02.849849 7fe9bae887a0 20 read_log 194370'14887 (0'0) promote db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by osd.346.0:3618 2014-08-12 20:30:33.970323 2014-08-13 17:38:02.849870 7fe9bae887a0 20 read_log 194370'14888 (194370'14887) modify db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by client.1508531.0:5176 2014-08-12 20:30:33.962698 2014-08-13 17:38:02.849883 7fe9bae887a0 20 read_log 194370'14889 (0'0) promote 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by osd.346.0:3620 2014-08-12 20:30:36.792082 2014-08-13 17:38:02.849895 7fe9bae887a0 20 read_log 194370'14890 (194370'14889) modify 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by client.1508531.0:5802 2014-08-12 20:30:36.783911 2014-08-13 17:38:02.849908 7fe9bae887a0 20 read_log 194370'14891 (0'0) promote aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by osd.346.0:3624 2014-08-12 20:30:38.611728 2014-08-13 17:38:02.849919 7fe9bae887a0 20 read_log 194370'14892 (194370'14891) modify aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by client.1508531.0:6194 2014-08-12 20:30:38.604425 2014-08-13 17:38:02.849931 7fe9bae887a0 20 read_log 194370'14893 (0'0) promote 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by osd.346.0:3626 2014-08-12 20:30:40.408272 2014-08-13 17:38:02.849943 7fe9bae887a0 20 read_log 194370'14894 (194370'14893) modify 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by client.1508531.0:6591 2014-08-12 20:30:40.400131 2014-08-13 17:38:02.849955 7fe9bae887a0 20 read_log 194370'14895 (0'0) promote 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by osd.346.0:3630 2014-08-12 20:30:43.446901 2014-08-13 17:38:02.849967 7fe9bae887a0 20 read_log 194370'14896 (194370'14895) modify 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by client.1508531.0:7217 2014-08-12 20:30:43.438717 2014-08-13 17:38:02.849979 7fe9bae887a0 20 read_log 194370'14897 (0'0) promote 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by osd.346.0:3640 2014-08-12 20:30:50.674363 2014-08-13 17:38:02.849991 7fe9bae887a0 20 read_log 194370'14898 (194370'14897) modify 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by client.1508531.0:8568 2014-08-12 20:30:50.666014 2014-08-13 17:38:02.850003 7fe9bae887a0 20 read_log 194370'14899 (0'0) promote bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by osd.346.0:3650 2014-08-12 20:31:00.519839 2014-08-13 17:38:02.850016 7fe9bae887a0 20 read_log 194370'14900 (194370'14899) modify bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by client.1508531.0:10214 2014-08-12 20:31:00.508225 2014-08-13 17:38:02.850028 7fe9bae887a0 20 read_log 194370'14901 (0'0) promote b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by osd.346.0:3654 2014-08-12 20:31:02.313507 2014-08-13 17:38:02.850040 7fe9bae887a0 20 read_log 194370'14902 (194370'14901) modify b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by client.1508531.0:10516 2014-08-12 20:31:02.307184 2014-08-13 17:38:02.850053 7fe9bae887a0 20 read_log 194370'14903 (0'0) promote b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by osd.346.0:3658 2014-08-12 20:31:03.603179 2014-08-13 17:38:02.850065 7fe9bae887a0 20 read_log 194370'14904 (194370'14903) modify b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by client.1508531.0:10715 2014-08-12 20:31:03.591547 2014-08-13 17:38:02.850078 7fe9bae887a0 20 read_log 194370'14905 (0'0) promote b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by osd.346.0:3660 2014-08-12 20:31:04.879706 2014-08-13 17:38:02.850089 7fe9bae887a0 20 read_log 194370'14906 (194370'14905) modify b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by client.1508531.0:10933 2014-08-12 20:31:04.869721 2014-08-13 17:38:02.850101 7fe9bae887a0 20 read_log 194370'14907 (0'0) promote 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by osd.346.0:3672 2014-08-12 20:31:11.122409 2014-08-13 17:38:02.850120 7fe9bae887a0 20 read_log 194370'14908 (194370'14907) modify 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by client.1508531.0:11967 2014-08-12 20:31:11.111281 2014-08-13 17:38:02.850133 7fe9bae887a0 20 read_log 194370'14909 (0'0) promote 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by osd.346.0:3674 2014-08-12 20:31:14.392210 2014-08-13 17:38:02.850145 7fe9bae887a0 20 read_log 194370'14910 (194370'14909) modify 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by client.1508531.0:12416 2014-08-12 20:31:14.383025 2014-08-13 17:38:02.850162 7fe9bae887a0 20 read_log 194370'14911 (0'0) promote 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by osd.346.0:3678 2014-08-12 20:31:15.089493 2014-08-13 17:38:02.850175 7fe9bae887a0 20 read_log 194370'14912 (194370'14911) modify 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by client.1508531.0:12511 2014-08-12 20:31:15.080922 2014-08-13 17:38:02.850189 7fe9bae887a0 20 read_log 194370'14913 (0'0) promote e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by osd.346.0:3680 2014-08-12 20:31:21.062559 2014-08-13 17:38:02.850202 7fe9bae887a0 20 read_log 194370'14914 (194370'14913) modify e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by client.1508531.0:13176 2014-08-12 20:31:21.054330 2014-08-13 17:38:02.850214 7fe9bae887a0 20 read_log 194370'14915 (0'0) promote 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by osd.346.0:3682 2014-08-12 20:31:21.839139 2014-08-13 17:38:02.850226 7fe9bae887a0 20 read_log 194370'14916 (194370'14915) modify 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by client.1508531.0:13258 2014-08-12 20:31:21.830476 2014-08-13 17:38:02.850238 7fe9bae887a0 20 read_log 194377'14917 (0'0) promote 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by osd.346.0:3686 2014-08-12 20:32:57.802897 2014-08-13 17:38:02.850250 7fe9bae887a0 20 read_log 194377'14918 (194377'14917) modify 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by client.2231190.0:153 2014-08-12 20:32:57.792894 2014-08-13 17:38:02.850262 7fe9bae887a0 20 read_log 194377'14919 (0'0) promote c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by osd.346.0:3702 2014-08-12 20:33:02.103922 2014-08-13 17:38:02.850274 7fe9bae887a0 20 read_log 194377'14920 (194377'14919) modify c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by client.2231190.0:1209 2014-08-12 20:33:02.094827 2014-08-13 17:38:02.850286 7fe9bae887a0 20 read_log 194377'14921 (0'0) promote 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by osd.346.0:3710 2014-08-12 20:33:08.015672 2014-08-13 17:38:02.850298 7fe9bae887a0 20 read_log 194377'14922 (194377'14921) modify 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by client.2231190.0:2671 2014-08-12 20:33:08.007800 2014-08-13 17:38:02.850310 7fe9bae887a0 20 read_log 194377'14923 (0'0) promote 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by osd.346.0:3714 2014-08-12 20:33:08.889973 2014-08-13 17:38:02.850322 7fe9bae887a0 20 read_log 194377'14924 (194377'14923) modify 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by client.2231190.0:2903 2014-08-12 20:33:08.882724 2014-08-13 17:38:02.850334 7fe9bae887a0 20 read_log 194377'14925 (0'0) promote cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by osd.346.0:3716 2014-08-12 20:33:09.489008 2014-08-13 17:38:02.850347 7fe9bae887a0 20 read_log 194377'14926 (194377'14925) modify cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by client.2231190.0:3040 2014-08-12 20:33:09.482049 2014-08-13 17:38:02.850360 7fe9bae887a0 20 read_log 194377'14927 (0'0) promote f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by osd.346.0:3722 2014-08-12 20:33:10.673358 2014-08-13 17:38:02.850397 7fe9bae887a0 20 read_log 194377'14928 (194377'14927) modify f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by client.2231190.0:3360 2014-08-12 20:33:10.665803 2014-08-13 17:38:02.850413 7fe9bae887a0 20 read_log 194380'14929 (0'0) promote 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by osd.346.0:3736 2014-08-12 20:33:23.603589 2014-08-13 17:38:02.850425 7fe9bae887a0 20 read_log 194380'14930 (194380'14929) modify 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by client.2231190.0:5043 2014-08-12 20:33:23.594993 2014-08-13 17:38:02.850438 7fe9bae887a0 20 read_log 194380'14931 (0'0) promote bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by osd.346.0:3738 2014-08-12 20:33:24.371968 2014-08-13 17:38:02.850450 7fe9bae887a0 20 read_log 194380'14932 (194380'14931) modify bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by client.2231190.0:5097 2014-08-12 20:33:24.362870 2014-08-13 17:38:02.850462 7fe9bae887a0 20 read_log 194383'14933 (0'0) promote 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by osd.346.0:3754 2014-08-12 20:34:33.235487 2014-08-13 17:38:02.850474 7fe9bae887a0 20 read_log 194383'14934 (194383'14933) modify 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by client.2231190.0:7665 2014-08-12 20:34:33.226508 2014-08-13 17:38:02.850486 7fe9bae887a0 20 read_log 194393'14935 (194393'14935) modify e2/hit_set_24.e2_archive_2014-08-12 20:37:07.990544_2014-08-12 20:47:26.960810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.961032 2014-08-13 17:38:02.850498 7fe9bae887a0 20 read_log 194393'14936 (194215'14871) delete e2/hit_set_24.e2_archive_2014-08-12 15:39:39.946574_2014-08-12 17:24:43.501290/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.961032 2014-08-13 17:38:02.850510 7fe9bae887a0 20 read_log 194393'14937 (0'0) promote 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by osd.346.0:3763 2014-08-12 20:47:26.964219 2014-08-13 17:38:02.850522 7fe9bae887a0 20 read_log 194393'14938 (194393'14937) modify 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by client.1064095.0:249 2014-08-12 20:47:26.955489 2014-08-13 17:38:02.850535 7fe9bae887a0 20 read_log 194393'14939 (0'0) promote 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by osd.346.0:3765 2014-08-12 20:47:28.709802 2014-08-13 17:38:02.850566 7fe9bae887a0 20 read_log 194393'14940 (194393'14939) modify 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by client.1064095.0:615 2014-08-12 20:47:28.700564 2014-08-13 17:38:02.850581 7fe9bae887a0 20 read_log 194393'14941 (194393'14941) modify e2/hit_set_24.e2_archive_2014-08-12 20:47:26.961037_2014-08-12 20:52:32.946845/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.947226 2014-08-13 17:38:02.850594 7fe9bae887a0 20 read_log 194393'14942 (194393'14935) delete e2/hit_set_24.e2_archive_2014-08-12 20:37:07.990544_2014-08-12 20:47:26.960810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.947226 2014-08-13 17:38:02.850606 7fe9bae887a0 20 read_log 194393'14943 (0'0) promote 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by osd.346.0:3776 2014-08-12 20:52:32.951655 2014-08-13 17:38:02.850618 7fe9bae887a0 20 read_log 194393'14944 (194393'14943) modify 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by client.1064422.0:108 2014-08-12 20:52:32.942063 2014-08-13 17:38:02.850630 7fe9bae887a0 20 read_log 194393'14945 (0'0) promote 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by osd.346.0:3778 2014-08-12 20:52:33.018064 2014-08-13 17:38:02.850642 7fe9bae887a0 20 read_log 194393'14946 (194393'14945) modify 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by client.1064422.0:125 2014-08-12 20:52:33.010604 2014-08-13 17:38:02.850664 7fe9bae887a0 20 read_log 194393'14947 (0'0) promote c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by osd.346.0:3780 2014-08-12 20:52:56.623445 2014-08-13 17:38:02.850677 7fe9bae887a0 20 read_log 194393'14948 (194393'14947) modify c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by client.2235069.0:61 2014-08-12 20:52:56.613072 2014-08-13 17:38:02.850691 7fe9bae887a0 20 read_log 194393'14949 (0'0) promote ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by osd.346.0:3782 2014-08-12 20:52:56.989152 2014-08-13 17:38:02.850703 7fe9bae887a0 20 read_log 194393'14950 (194393'14949) modify ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by client.2235069.0:148 2014-08-12 20:52:56.980170 2014-08-13 17:38:02.850715 7fe9bae887a0 20 read_log 194393'14951 (0'0) promote 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by osd.346.0:3786 2014-08-12 20:52:57.712450 2014-08-13 17:38:02.850727 7fe9bae887a0 20 read_log 194393'14952 (194393'14951) modify 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by client.2235069.0:260 2014-08-12 20:52:57.702244 2014-08-13 17:38:02.850739 7fe9bae887a0 20 read_log 194416'14953 (194416'14953) modify e2/hit_set_24.e2_archive_2014-08-12 20:52:32.947277_2014-08-13 15:28:33.527799/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 17:38:02.850751 7fe9bae887a0 20 read_log 194416'14954 (194393'14941) delete e2/hit_set_24.e2_archive_2014-08-12 20:47:26.961037_2014-08-12 20:52:32.946845/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 17:38:02.850763 7fe9bae887a0 20 read_log 194416'14955 (0'0) promote 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by osd.346.0:3935 2014-08-13 15:28:33.530674 2014-08-13 17:38:02.850775 7fe9bae887a0 20 read_log 194416'14956 (194416'14955) modify 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by client.2441958.0:765 2014-08-13 15:28:33.521857 2014-08-13 17:38:02.850788 7fe9bae887a0 20 read_log 194416'14957 (0'0) promote 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by osd.346.0:3939 2014-08-13 15:28:35.607515 2014-08-13 17:38:02.850799 7fe9bae887a0 20 read_log 194416'14958 (194416'14957) modify 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by client.2441958.0:1289 2014-08-13 15:28:35.598104 2014-08-13 17:38:02.850811 7fe9bae887a0 20 read_log 194416'14959 (0'0) promote 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by osd.346.0:3944 2014-08-13 15:28:36.844152 2014-08-13 17:38:02.850825 7fe9bae887a0 20 read_log 194416'14960 (194416'14959) modify 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by client.2441958.0:1616 2014-08-13 15:28:36.837034 2014-08-13 17:38:02.850838 7fe9bae887a0 20 read_log 194416'14961 (0'0) promote 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by osd.346.0:3954 2014-08-13 15:28:42.638007 2014-08-13 17:38:02.850849 7fe9bae887a0 20 read_log 194416'14962 (194416'14961) modify 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by client.2441958.0:3150 2014-08-13 15:28:42.629635 2014-08-13 17:38:02.850862 7fe9bae887a0 20 read_log 194416'14963 (0'0) promote 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by osd.346.0:3956 2014-08-13 15:28:42.651643 2014-08-13 17:38:02.850874 7fe9bae887a0 20 read_log 194416'14964 (194416'14963) modify 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by client.2441958.0:3154 2014-08-13 15:28:42.643803 2014-08-13 17:38:02.850886 7fe9bae887a0 20 read_log 194416'14965 (0'0) promote f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by osd.346.0:3964 2014-08-13 15:28:45.677995 2014-08-13 17:38:02.850904 7fe9bae887a0 20 read_log 194416'14966 (194416'14965) modify f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by client.2441958.0:3932 2014-08-13 15:28:45.670425 2014-08-13 17:38:02.850920 7fe9bae887a0 20 read_log 194416'14967 (0'0) promote 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by osd.346.0:3966 2014-08-13 15:28:46.835923 2014-08-13 17:38:02.850932 7fe9bae887a0 20 read_log 194416'14968 (194416'14967) modify 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by client.2441958.0:4223 2014-08-13 15:28:46.828611 2014-08-13 17:38:02.850944 7fe9bae887a0 20 read_log 194416'14969 (0'0) promote 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by osd.346.0:3970 2014-08-13 15:28:47.334968 2014-08-13 17:38:02.850955 7fe9bae887a0 20 read_log 194416'14970 (194416'14969) modify 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by client.2441958.0:4357 2014-08-13 15:28:47.328858 2014-08-13 17:38:02.850968 7fe9bae887a0 20 read_log 194416'14971 (0'0) promote 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by osd.346.0:3980 2014-08-13 15:28:51.005013 2014-08-13 17:38:02.850980 7fe9bae887a0 20 read_log 194416'14972 (194416'14971) modify 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by client.2441958.0:5289 2014-08-13 15:28:50.995585 2014-08-13 17:38:02.850992 7fe9bae887a0 20 read_log 194416'14973 (0'0) promote 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by osd.346.0:3998 2014-08-13 15:28:57.693112 2014-08-13 17:38:02.851004 7fe9bae887a0 20 read_log 194416'14974 (194416'14973) modify 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by client.2441958.0:6976 2014-08-13 15:28:57.682311 2014-08-13 17:38:02.851016 7fe9bae887a0 20 read_log 194416'14975 (0'0) promote c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by osd.346.0:4004 2014-08-13 15:29:01.052099 2014-08-13 17:38:02.851028 7fe9bae887a0 20 read_log 194416'14976 (194416'14975) modify c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by client.2441958.0:7872 2014-08-13 15:29:01.043368 2014-08-13 17:38:02.851040 7fe9bae887a0 20 read_log 194416'14977 (0'0) promote a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by osd.346.0:4008 2014-08-13 15:29:02.435944 2014-08-13 17:38:02.851052 7fe9bae887a0 20 read_log 194416'14978 (194416'14977) modify a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by client.2441958.0:8240 2014-08-13 15:29:02.426744 2014-08-13 17:38:02.851064 7fe9bae887a0 20 read_log 194416'14979 (0'0) promote ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by osd.346.0:4016 2014-08-13 15:29:07.697311 2014-08-13 17:38:02.851076 7fe9bae887a0 20 read_log 194416'14980 (194416'14979) modify ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by client.2441958.0:9582 2014-08-13 15:29:07.689454 2014-08-13 17:38:02.851088 7fe9bae887a0 20 read_log 194416'14981 (0'0) promote 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by osd.346.0:4018 2014-08-13 15:29:08.082009 2014-08-13 17:38:02.851100 7fe9bae887a0 20 read_log 194416'14982 (194416'14981) modify 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by client.2441958.0:9681 2014-08-13 15:29:08.072489 2014-08-13 17:38:02.851112 7fe9bae887a0 20 read_log 194416'14983 (0'0) promote 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by osd.346.0:4020 2014-08-13 15:29:14.578390 2014-08-13 17:38:02.851124 7fe9bae887a0 20 read_log 194416'14984 (194416'14983) modify 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by client.2441958.0:11271 2014-08-13 15:29:14.563606 2014-08-13 17:38:02.851136 7fe9bae887a0 20 read_log 194416'14985 (0'0) promote baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by osd.346.0:4022 2014-08-13 15:29:15.199098 2014-08-13 17:38:02.851160 7fe9bae887a0 20 read_log 194416'14986 (194416'14985) modify baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by client.2441958.0:11427 2014-08-13 15:29:15.190979 2014-08-13 17:38:02.851177 7fe9bae887a0 20 read_log 194416'14987 (0'0) promote f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by osd.346.0:4032 2014-08-13 15:29:20.960503 2014-08-13 17:38:02.851189 7fe9bae887a0 20 read_log 194416'14988 (194416'14987) modify f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by client.2441958.0:12783 2014-08-13 15:29:20.951520 2014-08-13 17:38:02.851201 7fe9bae887a0 20 read_log 194416'14989 (0'0) promote b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by osd.346.0:4036 2014-08-13 15:29:26.478944 2014-08-13 17:38:02.851213 7fe9bae887a0 20 read_log 194416'14990 (194416'14989) modify b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by client.2441958.0:14070 2014-08-13 15:29:26.470333 2014-08-13 17:38:02.851225 7fe9bae887a0 20 read_log 194416'14991 (0'0) promote 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by osd.346.0:4040 2014-08-13 15:29:26.815135 2014-08-13 17:38:02.851237 7fe9bae887a0 20 read_log 194416'14992 (194416'14991) modify 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by client.2441958.0:14159 2014-08-13 15:29:26.807165 2014-08-13 17:38:02.851249 7fe9bae887a0 20 read_log 194416'14993 (0'0) promote 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by osd.346.0:4042 2014-08-13 15:29:27.024088 2014-08-13 17:38:02.851261 7fe9bae887a0 20 read_log 194416'14994 (194416'14993) modify 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by client.2441958.0:14212 2014-08-13 15:29:27.015024 2014-08-13 17:38:02.851273 7fe9bae887a0 20 read_log 194416'14995 (0'0) promote 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by osd.346.0:4060 2014-08-13 15:29:43.046581 2014-08-13 17:38:02.851285 7fe9bae887a0 20 read_log 194416'14996 (194416'14995) modify 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by client.2441958.0:16700 2014-08-13 15:29:43.037136 2014-08-13 17:38:02.851297 7fe9bae887a0 20 read_log 194422'14997 (0'0) promote 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by osd.346.0:4076 2014-08-13 15:32:15.767448 2014-08-13 17:38:02.851309 7fe9bae887a0 20 read_log 194422'14998 (194422'14997) modify 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by client.1148203.0:172 2014-08-13 15:32:15.757634 2014-08-13 17:38:02.851320 7fe9bae887a0 20 read_log 194422'14999 (0'0) promote 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by osd.346.0:4082 2014-08-13 15:32:18.409546 2014-08-13 17:38:02.851332 7fe9bae887a0 20 read_log 194422'15000 (194422'14999) modify 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by client.1148203.0:766 2014-08-13 15:32:18.402124 2014-08-13 17:38:02.851363 7fe9bae887a0 20 read_log 194422'15001 (0'0) promote 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by osd.346.0:4090 2014-08-13 15:32:21.744796 2014-08-13 17:38:02.851387 7fe9bae887a0 20 read_log 194422'15002 (194422'15001) modify 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by client.1148203.0:1621 2014-08-13 15:32:21.736020 2014-08-13 17:38:02.851400 7fe9bae887a0 20 read_log 194422'15003 (0'0) promote deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by osd.346.0:4104 2014-08-13 15:32:26.264497 2014-08-13 17:38:02.851412 7fe9bae887a0 20 read_log 194422'15004 (194422'15003) modify deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by client.1148203.0:2794 2014-08-13 15:32:26.256164 2014-08-13 17:38:02.851434 7fe9bae887a0 20 read_log 194422'15005 (0'0) promote 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by osd.346.0:4106 2014-08-13 15:32:26.283030 2014-08-13 17:38:02.851449 7fe9bae887a0 20 read_log 194422'15006 (194422'15005) modify 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by client.1148203.0:2799 2014-08-13 15:32:26.273946 2014-08-13 17:38:02.851462 7fe9bae887a0 20 read_log 194422'15007 (0'0) promote 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by osd.346.0:4114 2014-08-13 15:32:30.977799 2014-08-13 17:38:02.851473 7fe9bae887a0 20 read_log 194422'15008 (194422'15007) modify 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by client.1148203.0:3996 2014-08-13 15:32:30.971523 2014-08-13 17:38:02.851486 7fe9bae887a0 20 read_log 194422'15009 (0'0) promote f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by osd.346.0:4118 2014-08-13 15:32:34.425789 2014-08-13 17:38:02.851497 7fe9bae887a0 20 read_log 194422'15010 (194422'15009) modify f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by client.1148203.0:4891 2014-08-13 15:32:34.417029 2014-08-13 17:38:02.851510 7fe9bae887a0 20 read_log 194422'15011 (0'0) promote b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by osd.346.0:4156 2014-08-13 15:33:02.939383 2014-08-13 17:38:02.851522 7fe9bae887a0 20 read_log 194422'15012 (194422'15011) modify b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by client.1148203.0:9345 2014-08-13 15:33:02.932357 2014-08-13 17:38:02.851533 7fe9bae887a0 20 read_log 194422'15013 (0'0) promote d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by osd.346.0:4162 2014-08-13 15:33:04.859400 2014-08-13 17:38:02.851545 7fe9bae887a0 20 read_log 194422'15014 (194422'15013) modify d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by client.1148203.0:9539 2014-08-13 15:33:04.848878 2014-08-13 17:38:02.851560 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.852030 7fe9bae887a0 10 read_log checking for missing items over interval (194370'14879,194422'15014] 2014-08-13 17:38:02.852079 7fe9bae887a0 15 read_log missing 194422'15014 (194422'15013) modify d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by client.1148203.0:9539 2014-08-13 15:33:04.848878 2014-08-13 17:38:02.852120 7fe9bae887a0 15 read_log missing 194422'15012 (194422'15011) modify b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by client.1148203.0:9345 2014-08-13 15:33:02.932357 2014-08-13 17:38:02.852162 7fe9bae887a0 15 read_log missing 194422'15010 (194422'15009) modify f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by client.1148203.0:4891 2014-08-13 15:32:34.417029 2014-08-13 17:38:02.852199 7fe9bae887a0 15 read_log missing 194422'15008 (194422'15007) modify 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by client.1148203.0:3996 2014-08-13 15:32:30.971523 2014-08-13 17:38:02.852235 7fe9bae887a0 15 read_log missing 194422'15006 (194422'15005) modify 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by client.1148203.0:2799 2014-08-13 15:32:26.273946 2014-08-13 17:38:02.852271 7fe9bae887a0 15 read_log missing 194422'15004 (194422'15003) modify deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by client.1148203.0:2794 2014-08-13 15:32:26.256164 2014-08-13 17:38:02.852307 7fe9bae887a0 15 read_log missing 194422'15002 (194422'15001) modify 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by client.1148203.0:1621 2014-08-13 15:32:21.736020 2014-08-13 17:38:02.852343 7fe9bae887a0 15 read_log missing 194422'15000 (194422'14999) modify 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by client.1148203.0:766 2014-08-13 15:32:18.402124 2014-08-13 17:38:02.852379 7fe9bae887a0 15 read_log missing 194422'14998 (194422'14997) modify 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by client.1148203.0:172 2014-08-13 15:32:15.757634 2014-08-13 17:38:02.852415 7fe9bae887a0 15 read_log missing 194416'14996 (194416'14995) modify 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by client.2441958.0:16700 2014-08-13 15:29:43.037136 2014-08-13 17:38:02.852452 7fe9bae887a0 15 read_log missing 194416'14994 (194416'14993) modify 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by client.2441958.0:14212 2014-08-13 15:29:27.015024 2014-08-13 17:38:02.852488 7fe9bae887a0 15 read_log missing 194416'14992 (194416'14991) modify 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by client.2441958.0:14159 2014-08-13 15:29:26.807165 2014-08-13 17:38:02.852525 7fe9bae887a0 15 read_log missing 194416'14990 (194416'14989) modify b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by client.2441958.0:14070 2014-08-13 15:29:26.470333 2014-08-13 17:38:02.852561 7fe9bae887a0 15 read_log missing 194416'14988 (194416'14987) modify f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by client.2441958.0:12783 2014-08-13 15:29:20.951520 2014-08-13 17:38:02.852598 7fe9bae887a0 15 read_log missing 194416'14986 (194416'14985) modify baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by client.2441958.0:11427 2014-08-13 15:29:15.190979 2014-08-13 17:38:02.852634 7fe9bae887a0 15 read_log missing 194416'14984 (194416'14983) modify 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by client.2441958.0:11271 2014-08-13 15:29:14.563606 2014-08-13 17:38:02.852671 7fe9bae887a0 15 read_log missing 194416'14982 (194416'14981) modify 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by client.2441958.0:9681 2014-08-13 15:29:08.072489 2014-08-13 17:38:02.852708 7fe9bae887a0 15 read_log missing 194416'14980 (194416'14979) modify ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by client.2441958.0:9582 2014-08-13 15:29:07.689454 2014-08-13 17:38:02.852744 7fe9bae887a0 15 read_log missing 194416'14978 (194416'14977) modify a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by client.2441958.0:8240 2014-08-13 15:29:02.426744 2014-08-13 17:38:02.852781 7fe9bae887a0 15 read_log missing 194416'14976 (194416'14975) modify c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by client.2441958.0:7872 2014-08-13 15:29:01.043368 2014-08-13 17:38:02.852817 7fe9bae887a0 15 read_log missing 194416'14974 (194416'14973) modify 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by client.2441958.0:6976 2014-08-13 15:28:57.682311 2014-08-13 17:38:02.852853 7fe9bae887a0 15 read_log missing 194416'14972 (194416'14971) modify 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by client.2441958.0:5289 2014-08-13 15:28:50.995585 2014-08-13 17:38:02.852891 7fe9bae887a0 15 read_log missing 194416'14970 (194416'14969) modify 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by client.2441958.0:4357 2014-08-13 15:28:47.328858 2014-08-13 17:38:02.852928 7fe9bae887a0 15 read_log missing 194416'14968 (194416'14967) modify 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by client.2441958.0:4223 2014-08-13 15:28:46.828611 2014-08-13 17:38:02.852964 7fe9bae887a0 15 read_log missing 194416'14966 (194416'14965) modify f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by client.2441958.0:3932 2014-08-13 15:28:45.670425 2014-08-13 17:38:02.853001 7fe9bae887a0 15 read_log missing 194416'14964 (194416'14963) modify 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by client.2441958.0:3154 2014-08-13 15:28:42.643803 2014-08-13 17:38:02.853038 7fe9bae887a0 15 read_log missing 194416'14962 (194416'14961) modify 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by client.2441958.0:3150 2014-08-13 15:28:42.629635 2014-08-13 17:38:02.853075 7fe9bae887a0 15 read_log missing 194416'14960 (194416'14959) modify 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by client.2441958.0:1616 2014-08-13 15:28:36.837034 2014-08-13 17:38:02.853112 7fe9bae887a0 15 read_log missing 194416'14958 (194416'14957) modify 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by client.2441958.0:1289 2014-08-13 15:28:35.598104 2014-08-13 17:38:02.853149 7fe9bae887a0 15 read_log missing 194416'14956 (194416'14955) modify 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by client.2441958.0:765 2014-08-13 15:28:33.521857 2014-08-13 17:38:02.853196 7fe9bae887a0 15 read_log missing 194416'14953 (194416'14953) modify e2/hit_set_24.e2_archive_2014-08-12 20:52:32.947277_2014-08-13 15:28:33.527799/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 17:38:02.853232 7fe9bae887a0 15 read_log missing 194393'14952 (194393'14951) modify 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by client.2235069.0:260 2014-08-12 20:52:57.702244 2014-08-13 17:38:02.853269 7fe9bae887a0 15 read_log missing 194393'14950 (194393'14949) modify ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by client.2235069.0:148 2014-08-12 20:52:56.980170 2014-08-13 17:38:02.853326 7fe9bae887a0 15 read_log missing 194393'14948 (194393'14947) modify c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by client.2235069.0:61 2014-08-12 20:52:56.613072 2014-08-13 17:38:02.853364 7fe9bae887a0 15 read_log missing 194393'14946 (194393'14945) modify 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by client.1064422.0:125 2014-08-12 20:52:33.010604 2014-08-13 17:38:02.853400 7fe9bae887a0 15 read_log missing 194393'14944 (194393'14943) modify 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by client.1064422.0:108 2014-08-12 20:52:32.942063 2014-08-13 17:38:02.853441 7fe9bae887a0 15 read_log missing 194393'14940 (194393'14939) modify 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by client.1064095.0:615 2014-08-12 20:47:28.700564 2014-08-13 17:38:02.853477 7fe9bae887a0 15 read_log missing 194393'14938 (194393'14937) modify 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by client.1064095.0:249 2014-08-12 20:47:26.955489 2014-08-13 17:38:02.853518 7fe9bae887a0 15 read_log missing 194383'14934 (194383'14933) modify 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by client.2231190.0:7665 2014-08-12 20:34:33.226508 2014-08-13 17:38:02.853555 7fe9bae887a0 15 read_log missing 194380'14932 (194380'14931) modify bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by client.2231190.0:5097 2014-08-12 20:33:24.362870 2014-08-13 17:38:02.853591 7fe9bae887a0 15 read_log missing 194380'14930 (194380'14929) modify 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by client.2231190.0:5043 2014-08-12 20:33:23.594993 2014-08-13 17:38:02.853628 7fe9bae887a0 15 read_log missing 194377'14928 (194377'14927) modify f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by client.2231190.0:3360 2014-08-12 20:33:10.665803 2014-08-13 17:38:02.853664 7fe9bae887a0 15 read_log missing 194377'14926 (194377'14925) modify cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by client.2231190.0:3040 2014-08-12 20:33:09.482049 2014-08-13 17:38:02.853701 7fe9bae887a0 15 read_log missing 194377'14924 (194377'14923) modify 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by client.2231190.0:2903 2014-08-12 20:33:08.882724 2014-08-13 17:38:02.853737 7fe9bae887a0 15 read_log missing 194377'14922 (194377'14921) modify 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by client.2231190.0:2671 2014-08-12 20:33:08.007800 2014-08-13 17:38:02.853774 7fe9bae887a0 15 read_log missing 194377'14920 (194377'14919) modify c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by client.2231190.0:1209 2014-08-12 20:33:02.094827 2014-08-13 17:38:02.853811 7fe9bae887a0 15 read_log missing 194377'14918 (194377'14917) modify 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by client.2231190.0:153 2014-08-12 20:32:57.792894 2014-08-13 17:38:02.853848 7fe9bae887a0 15 read_log missing 194370'14916 (194370'14915) modify 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by client.1508531.0:13258 2014-08-12 20:31:21.830476 2014-08-13 17:38:02.853885 7fe9bae887a0 15 read_log missing 194370'14914 (194370'14913) modify e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by client.1508531.0:13176 2014-08-12 20:31:21.054330 2014-08-13 17:38:02.853922 7fe9bae887a0 15 read_log missing 194370'14912 (194370'14911) modify 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by client.1508531.0:12511 2014-08-12 20:31:15.080922 2014-08-13 17:38:02.853959 7fe9bae887a0 15 read_log missing 194370'14910 (194370'14909) modify 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by client.1508531.0:12416 2014-08-12 20:31:14.383025 2014-08-13 17:38:02.853995 7fe9bae887a0 15 read_log missing 194370'14908 (194370'14907) modify 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by client.1508531.0:11967 2014-08-12 20:31:11.111281 2014-08-13 17:38:02.854032 7fe9bae887a0 15 read_log missing 194370'14906 (194370'14905) modify b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by client.1508531.0:10933 2014-08-12 20:31:04.869721 2014-08-13 17:38:02.854069 7fe9bae887a0 15 read_log missing 194370'14904 (194370'14903) modify b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by client.1508531.0:10715 2014-08-12 20:31:03.591547 2014-08-13 17:38:02.854106 7fe9bae887a0 15 read_log missing 194370'14902 (194370'14901) modify b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by client.1508531.0:10516 2014-08-12 20:31:02.307184 2014-08-13 17:38:02.854143 7fe9bae887a0 15 read_log missing 194370'14900 (194370'14899) modify bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by client.1508531.0:10214 2014-08-12 20:31:00.508225 2014-08-13 17:38:02.854185 7fe9bae887a0 15 read_log missing 194370'14898 (194370'14897) modify 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by client.1508531.0:8568 2014-08-12 20:30:50.666014 2014-08-13 17:38:02.854222 7fe9bae887a0 15 read_log missing 194370'14896 (194370'14895) modify 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by client.1508531.0:7217 2014-08-12 20:30:43.438717 2014-08-13 17:38:02.854259 7fe9bae887a0 15 read_log missing 194370'14894 (194370'14893) modify 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by client.1508531.0:6591 2014-08-12 20:30:40.400131 2014-08-13 17:38:02.854296 7fe9bae887a0 15 read_log missing 194370'14892 (194370'14891) modify aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by client.1508531.0:6194 2014-08-12 20:30:38.604425 2014-08-13 17:38:02.854332 7fe9bae887a0 15 read_log missing 194370'14890 (194370'14889) modify 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by client.1508531.0:5802 2014-08-12 20:30:36.783911 2014-08-13 17:38:02.854369 7fe9bae887a0 15 read_log missing 194370'14888 (194370'14887) modify db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by client.1508531.0:5176 2014-08-12 20:30:33.962698 2014-08-13 17:38:02.854406 7fe9bae887a0 15 read_log missing 194370'14886 (194370'14885) modify c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by client.1508531.0:5058 2014-08-12 20:30:33.502529 2014-08-13 17:38:02.854442 7fe9bae887a0 15 read_log missing 194370'14884 (194370'14883) modify 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by client.1508531.0:4984 2014-08-12 20:30:33.159380 2014-08-13 17:38:02.854479 7fe9bae887a0 15 read_log missing 194370'14882 (194370'14881) modify b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by client.1508531.0:3564 2014-08-12 20:30:27.639554 2014-08-13 17:38:02.854517 7fe9bae887a0 15 read_log missing 194370'14880 (194370'14879) modify 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by client.1508531.0:878 2014-08-12 20:30:17.678096 2014-08-13 17:38:02.854528 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.854556 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive m=66] handle_loaded 2014-08-13 17:38:02.854574 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Initial 0.015025 0 0.000000 2014-08-13 17:38:02.854593 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset 2014-08-13 17:38:02.854602 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194477 2014-08-13 17:38:02.854611 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] log((192834'11882,194422'15014], crt=194422'15014) 2014-08-13 17:38:02.854621 7fe9bae887a0 10 osd.4 194478 pgid 24.136 coll 24.136_head 2014-08-13 17:38:02.854733 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.136 2014-08-13 17:38:02.854746 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.854778 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.136(unlocked)] enter Initial 2014-08-13 17:38:02.854789 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.136(unlocked)] enter NotTrimming 2014-08-13 17:38:02.854915 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.855040 7fe9bae887a0 20 read_log 192834'12418 (0'0) promote 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:13898 2014-08-11 15:24:19.282007 2014-08-13 17:38:02.855061 7fe9bae887a0 20 read_log 192834'12419 (192834'12418) modify 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by client.933448.0:9549 2014-08-11 15:24:19.274646 2014-08-13 17:38:02.855075 7fe9bae887a0 20 read_log 192834'12428 (0'0) promote bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:13908 2014-08-11 15:24:20.927952 2014-08-13 17:38:02.855087 7fe9bae887a0 20 read_log 192834'12429 (192834'12428) modify bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by client.933448.0:9935 2014-08-11 15:24:20.919848 2014-08-13 17:38:02.855099 7fe9bae887a0 20 read_log 192834'12430 (0'0) promote 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:13910 2014-08-11 15:24:22.347154 2014-08-13 17:38:02.855111 7fe9bae887a0 20 read_log 192834'12431 (192834'12430) modify 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by client.933448.0:10270 2014-08-11 15:24:22.338905 2014-08-13 17:38:02.855124 7fe9bae887a0 20 read_log 192834'12452 (0'0) promote aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:13932 2014-08-11 15:24:29.226630 2014-08-13 17:38:02.855136 7fe9bae887a0 20 read_log 192834'12453 (192834'12452) modify aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by client.933448.0:11750 2014-08-11 15:24:29.217974 2014-08-13 17:38:02.855148 7fe9bae887a0 20 read_log 192834'12474 (192834'12263) clean fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:13954 2014-08-11 15:24:36.059608 2014-08-13 17:38:02.855166 7fe9bae887a0 20 read_log 192834'12481 (0'0) promote dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:13961 2014-08-11 15:24:40.092709 2014-08-13 17:38:02.855180 7fe9bae887a0 20 read_log 192834'12482 (192834'12481) modify dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by client.933448.0:13824 2014-08-11 15:24:40.084916 2014-08-13 17:38:02.855193 7fe9bae887a0 20 read_log 192834'12489 (192834'12474) delete fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:13969 2014-08-11 15:24:40.998768 2014-08-13 17:38:02.855205 7fe9bae887a0 20 read_log 192834'12491 (192834'12283) clean 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:13971 2014-08-11 15:24:41.156576 2014-08-13 17:38:02.855239 7fe9bae887a0 20 read_log 192834'12495 (192834'12281) clean fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:13975 2014-08-11 15:24:41.254199 2014-08-13 17:38:02.855255 7fe9bae887a0 20 read_log 192834'12503 (192834'12491) delete 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:13983 2014-08-11 15:24:41.550408 2014-08-13 17:38:02.855267 7fe9bae887a0 20 read_log 192834'12507 (192834'12495) delete fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:13987 2014-08-11 15:24:41.572638 2014-08-13 17:38:02.855279 7fe9bae887a0 20 read_log 192834'12523 (0'0) promote 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14003 2014-08-11 15:24:44.720195 2014-08-13 17:38:02.855291 7fe9bae887a0 20 read_log 192834'12524 (192834'12523) modify 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by client.933448.0:14321 2014-08-11 15:24:44.710863 2014-08-13 17:38:02.855303 7fe9bae887a0 20 read_log 192834'12537 (192834'12306) clean 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14017 2014-08-11 15:24:46.951872 2014-08-13 17:38:02.855325 7fe9bae887a0 20 read_log 192834'12544 (192834'12537) delete 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14024 2014-08-11 15:24:47.000408 2014-08-13 17:38:02.855338 7fe9bae887a0 20 read_log 192834'12582 (192834'12325) clean 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14062 2014-08-11 15:24:52.500534 2014-08-13 17:38:02.855350 7fe9bae887a0 20 read_log 192834'12591 (192834'12582) delete 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14071 2014-08-11 15:24:57.438039 2014-08-13 17:38:02.855361 7fe9bae887a0 20 read_log 192834'12593 (192834'12337) clean 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14073 2014-08-11 15:24:57.590221 2014-08-13 17:38:02.855373 7fe9bae887a0 20 read_log 192834'12600 (192834'12593) delete 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14080 2014-08-11 15:24:57.623851 2014-08-13 17:38:02.855385 7fe9bae887a0 20 read_log 192834'12617 (0'0) promote 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14097 2014-08-11 15:25:01.307384 2014-08-13 17:38:02.855397 7fe9bae887a0 20 read_log 192834'12618 (192834'12617) modify 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by client.933448.0:17741 2014-08-11 15:25:01.299749 2014-08-13 17:38:02.855409 7fe9bae887a0 20 read_log 192834'12651 (0'0) promote b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:14131 2014-08-11 15:25:04.680329 2014-08-13 17:38:02.855421 7fe9bae887a0 20 read_log 192834'12652 (192834'12651) modify b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by client.933448.0:18303 2014-08-11 15:25:04.672494 2014-08-13 17:38:02.855433 7fe9bae887a0 20 read_log 192834'12675 (192834'12379) clean 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14155 2014-08-11 15:25:08.396549 2014-08-13 17:38:02.855445 7fe9bae887a0 20 read_log 192834'12689 (192834'12675) delete 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14169 2014-08-11 15:25:08.589376 2014-08-13 17:38:02.855457 7fe9bae887a0 20 read_log 192834'12705 (0'0) promote 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:14185 2014-08-11 15:25:11.750881 2014-08-13 17:38:02.855469 7fe9bae887a0 20 read_log 192834'12706 (192834'12705) modify 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by client.933448.0:19387 2014-08-11 15:25:11.742661 2014-08-13 17:38:02.855481 7fe9bae887a0 20 read_log 192834'12755 (0'0) promote 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:14235 2014-08-11 15:25:23.435859 2014-08-13 17:38:02.855493 7fe9bae887a0 20 read_log 192834'12756 (192834'12755) modify 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by client.933448.0:20813 2014-08-11 15:25:23.428206 2014-08-13 17:38:02.855505 7fe9bae887a0 20 read_log 192834'12757 (192834'12419) clean 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14237 2014-08-11 15:25:24.363178 2014-08-13 17:38:02.855517 7fe9bae887a0 20 read_log 192834'12761 (192834'12429) clean bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14241 2014-08-11 15:25:24.424223 2014-08-13 17:38:02.855529 7fe9bae887a0 20 read_log 192834'12762 (192834'12431) clean 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14242 2014-08-11 15:25:24.473416 2014-08-13 17:38:02.855541 7fe9bae887a0 20 read_log 192834'12767 (192834'12757) delete 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14247 2014-08-11 15:25:24.542586 2014-08-13 17:38:02.855553 7fe9bae887a0 20 read_log 192834'12771 (192834'12761) delete bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14251 2014-08-11 15:25:24.555241 2014-08-13 17:38:02.855573 7fe9bae887a0 20 read_log 192834'12772 (192834'12762) delete 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14252 2014-08-11 15:25:24.559863 2014-08-13 17:38:02.855586 7fe9bae887a0 20 read_log 192834'12785 (192834'12453) clean aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14265 2014-08-11 15:25:29.755103 2014-08-13 17:38:02.855599 7fe9bae887a0 20 read_log 192834'12792 (192834'12785) delete aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14272 2014-08-11 15:25:29.874932 2014-08-13 17:38:02.855611 7fe9bae887a0 20 read_log 192834'12812 (0'0) promote ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:14292 2014-08-11 15:25:35.049658 2014-08-13 17:38:02.855623 7fe9bae887a0 20 read_log 192834'12813 (192834'12812) modify ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by client.933448.0:22199 2014-08-11 15:25:34.961761 2014-08-13 17:38:02.855635 7fe9bae887a0 20 read_log 192834'12829 (0'0) promote fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14309 2014-08-11 15:25:37.691044 2014-08-13 17:38:02.855647 7fe9bae887a0 20 read_log 192834'12830 (192834'12829) modify fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by client.933448.0:22341 2014-08-11 15:25:37.662942 2014-08-13 17:38:02.855659 7fe9bae887a0 20 read_log 192834'12847 (0'0) promote fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14327 2014-08-11 15:25:39.264994 2014-08-13 17:38:02.855671 7fe9bae887a0 20 read_log 192834'12848 (192834'12847) modify fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by client.933448.0:23337 2014-08-11 15:25:39.234974 2014-08-13 17:38:02.855683 7fe9bae887a0 20 read_log 192834'12849 (0'0) promote 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14329 2014-08-11 15:25:39.397138 2014-08-13 17:38:02.855694 7fe9bae887a0 20 read_log 192834'12850 (192834'12849) modify 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by client.933448.0:23453 2014-08-11 15:25:39.367061 2014-08-13 17:38:02.855706 7fe9bae887a0 20 read_log 192834'12868 (192834'12482) clean dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14348 2014-08-11 15:25:40.254649 2014-08-13 17:38:02.855718 7fe9bae887a0 20 read_log 192834'12872 (192834'12868) delete dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14352 2014-08-11 15:25:40.280984 2014-08-13 17:38:02.855730 7fe9bae887a0 20 read_log 192834'12884 (0'0) promote 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14364 2014-08-11 15:25:40.551240 2014-08-13 17:38:02.855742 7fe9bae887a0 20 read_log 192834'12885 (192834'12884) modify 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by client.933448.0:24283 2014-08-11 15:25:40.522655 2014-08-13 17:38:02.855754 7fe9bae887a0 20 read_log 192834'12902 (0'0) promote 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14382 2014-08-11 15:25:42.633086 2014-08-13 17:38:02.855765 7fe9bae887a0 20 read_log 192834'12903 (192834'12902) modify 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by client.933448.0:25838 2014-08-11 15:25:42.604701 2014-08-13 17:38:02.855777 7fe9bae887a0 20 read_log 192834'12914 (0'0) promote 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14394 2014-08-11 15:25:43.796880 2014-08-13 17:38:02.855789 7fe9bae887a0 20 read_log 192834'12915 (192834'12914) modify 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by client.933448.0:26799 2014-08-11 15:25:43.770287 2014-08-13 17:38:02.855801 7fe9bae887a0 20 read_log 192834'12946 (192834'12524) clean 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14426 2014-08-11 15:25:45.467919 2014-08-13 17:38:02.855838 7fe9bae887a0 20 read_log 192834'12954 (192834'12946) delete 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14434 2014-08-11 15:25:45.609143 2014-08-13 17:38:02.855852 7fe9bae887a0 20 read_log 192834'12979 (0'0) promote 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14459 2014-08-11 15:25:47.139797 2014-08-13 17:38:02.855864 7fe9bae887a0 20 read_log 192834'12980 (192834'12979) modify 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by client.933448.0:29207 2014-08-11 15:25:47.114359 2014-08-13 17:38:02.855878 7fe9bae887a0 20 read_log 192834'13019 (0'0) promote 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14499 2014-08-11 15:25:50.708999 2014-08-13 17:38:02.855890 7fe9bae887a0 20 read_log 192834'13020 (192834'13019) modify 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by client.933448.0:31840 2014-08-11 15:25:50.682700 2014-08-13 17:38:02.855902 7fe9bae887a0 20 read_log 192834'13041 (0'0) promote bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14521 2014-08-11 15:25:51.311295 2014-08-13 17:38:02.855914 7fe9bae887a0 20 read_log 192834'13042 (192834'13041) modify bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by client.933448.0:32226 2014-08-11 15:25:51.279533 2014-08-13 17:38:02.855926 7fe9bae887a0 20 read_log 192834'13043 (0'0) promote 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14523 2014-08-11 15:25:52.047611 2014-08-13 17:38:02.855938 7fe9bae887a0 20 read_log 192834'13044 (192834'13043) modify 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by client.933448.0:32561 2014-08-11 15:25:52.017984 2014-08-13 17:38:02.855950 7fe9bae887a0 20 read_log 192834'13065 (0'0) promote aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14545 2014-08-11 15:25:54.548809 2014-08-13 17:38:02.855962 7fe9bae887a0 20 read_log 192834'13066 (192834'13065) modify aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by client.933448.0:34041 2014-08-11 15:25:54.510434 2014-08-13 17:38:02.855974 7fe9bae887a0 20 read_log 192834'13101 (0'0) promote dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14581 2014-08-11 15:25:57.281638 2014-08-13 17:38:02.855986 7fe9bae887a0 20 read_log 192834'13102 (192834'13101) modify dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by client.933448.0:36115 2014-08-11 15:25:57.256477 2014-08-13 17:38:02.855998 7fe9bae887a0 20 read_log 192834'13113 (0'0) promote 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14593 2014-08-11 15:25:58.091419 2014-08-13 17:38:02.856009 7fe9bae887a0 20 read_log 192834'13114 (192834'13113) modify 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by client.933448.0:36612 2014-08-11 15:25:58.067626 2014-08-13 17:38:02.856021 7fe9bae887a0 20 read_log 192834'13145 (192834'12618) clean 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14625 2014-08-11 15:26:01.532619 2014-08-13 17:38:02.856033 7fe9bae887a0 20 read_log 192834'13149 (192834'13145) delete 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14629 2014-08-11 15:26:01.652295 2014-08-13 17:38:02.856045 7fe9bae887a0 20 read_log 192834'13167 (0'0) promote 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14647 2014-08-11 15:26:02.257841 2014-08-13 17:38:02.856057 7fe9bae887a0 20 read_log 192834'13168 (192834'13167) modify 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by client.933448.0:40032 2014-08-11 15:26:02.231771 2014-08-13 17:38:02.856069 7fe9bae887a0 20 read_log 192834'13171 (192834'12652) modify b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by client.933448.0:40594 2014-08-11 15:26:02.662421 2014-08-13 17:38:02.856089 7fe9bae887a0 20 read_log 192834'13188 (192834'12706) modify 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by client.933448.0:41678 2014-08-11 15:26:03.018534 2014-08-13 17:38:02.856101 7fe9bae887a0 20 read_log 192834'13197 (192834'12756) modify 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by client.933448.0:43104 2014-08-11 15:26:03.503064 2014-08-13 17:38:02.856114 7fe9bae887a0 20 read_log 192834'13204 (192834'12813) modify ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by client.933448.0:44490 2014-08-11 15:26:03.960541 2014-08-13 17:38:02.856126 7fe9bae887a0 20 read_log 192834'13216 (0'0) promote 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:14696 2014-08-11 15:26:16.863024 2014-08-13 17:38:02.856138 7fe9bae887a0 20 read_log 192834'13217 (192834'13216) modify 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by client.1883634.0:651 2014-08-11 15:26:16.855277 2014-08-13 17:38:02.856150 7fe9bae887a0 20 read_log 192834'13228 (0'0) promote 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:14708 2014-08-11 15:26:20.208238 2014-08-13 17:38:02.856193 7fe9bae887a0 20 read_log 192834'13229 (192834'13228) modify 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by client.1883634.0:1577 2014-08-11 15:26:20.200892 2014-08-13 17:38:02.856207 7fe9bae887a0 20 read_log 192834'13230 (0'0) promote bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:14710 2014-08-11 15:26:20.527134 2014-08-13 17:38:02.856219 7fe9bae887a0 20 read_log 192834'13231 (192834'13230) modify bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by client.1883634.0:1663 2014-08-11 15:26:20.518306 2014-08-13 17:38:02.856231 7fe9bae887a0 20 read_log 192834'13272 (0'0) promote c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:14752 2014-08-11 15:26:29.154756 2014-08-13 17:38:02.856243 7fe9bae887a0 20 read_log 192834'13273 (192834'13272) modify c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by client.1883634.0:4045 2014-08-11 15:26:29.147716 2014-08-13 17:38:02.856255 7fe9bae887a0 20 read_log 192834'13276 (0'0) promote 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:14756 2014-08-11 15:26:29.970140 2014-08-13 17:38:02.856267 7fe9bae887a0 20 read_log 192834'13277 (192834'13276) modify 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by client.1883634.0:4275 2014-08-11 15:26:29.962730 2014-08-13 17:38:02.856279 7fe9bae887a0 20 read_log 192834'13278 (0'0) promote 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:14758 2014-08-11 15:26:30.194159 2014-08-13 17:38:02.856291 7fe9bae887a0 20 read_log 192834'13279 (192834'13278) modify 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by client.1883634.0:4339 2014-08-11 15:26:30.186725 2014-08-13 17:38:02.856303 7fe9bae887a0 20 read_log 192834'13306 (0'0) promote 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:14786 2014-08-11 15:26:38.540099 2014-08-13 17:38:02.856315 7fe9bae887a0 20 read_log 192834'13307 (192834'13306) modify 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by client.1883634.0:6508 2014-08-11 15:26:38.531222 2014-08-13 17:38:02.856327 7fe9bae887a0 20 read_log 192834'13310 (0'0) promote 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:14790 2014-08-11 15:26:40.837927 2014-08-13 17:38:02.856339 7fe9bae887a0 20 read_log 192834'13311 (192834'13310) modify 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by client.1883634.0:7106 2014-08-11 15:26:40.830906 2014-08-13 17:38:02.856351 7fe9bae887a0 20 read_log 192834'13318 (192834'12885) clean 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14798 2014-08-11 15:26:42.486454 2014-08-13 17:38:02.856371 7fe9bae887a0 20 read_log 192834'13322 (192834'12830) clean fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14802 2014-08-11 15:26:42.520966 2014-08-13 17:38:02.856384 7fe9bae887a0 20 read_log 192834'13323 (192834'12850) clean 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14803 2014-08-11 15:26:42.521248 2014-08-13 17:38:02.856396 7fe9bae887a0 20 read_log 192834'13329 (192834'12848) clean fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14809 2014-08-11 15:26:42.556159 2014-08-13 17:38:02.856407 7fe9bae887a0 20 read_log 192834'13349 (192834'13318) delete 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14829 2014-08-11 15:26:42.693153 2014-08-13 17:38:02.856419 7fe9bae887a0 20 read_log 192834'13355 (192834'13322) delete fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14835 2014-08-11 15:26:42.728305 2014-08-13 17:38:02.856431 7fe9bae887a0 20 read_log 192834'13356 (192834'13323) delete 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14836 2014-08-11 15:26:42.734380 2014-08-13 17:38:02.856443 7fe9bae887a0 20 read_log 192834'13361 (192834'12903) clean 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14841 2014-08-11 15:26:42.764257 2014-08-13 17:38:02.856456 7fe9bae887a0 20 read_log 192834'13363 (192834'13329) delete fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14843 2014-08-11 15:26:42.775718 2014-08-13 17:38:02.856469 7fe9bae887a0 20 read_log 192834'13384 (192834'13361) delete 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14864 2014-08-11 15:26:42.884276 2014-08-13 17:38:02.856480 7fe9bae887a0 20 read_log 192834'13387 (0'0) promote 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:14867 2014-08-11 15:26:43.277616 2014-08-13 17:38:02.856492 7fe9bae887a0 20 read_log 192834'13388 (192834'13387) modify 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by client.1883634.0:7684 2014-08-11 15:26:43.269813 2014-08-13 17:38:02.856505 7fe9bae887a0 20 read_log 192834'13398 (192834'12915) clean 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14878 2014-08-11 15:26:48.169887 2014-08-13 17:38:02.856517 7fe9bae887a0 20 read_log 192834'13399 (192834'12980) clean 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14879 2014-08-11 15:26:48.170231 2014-08-13 17:38:02.856529 7fe9bae887a0 20 read_log 192834'13427 (192834'13398) delete 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14907 2014-08-11 15:26:48.363275 2014-08-13 17:38:02.856540 7fe9bae887a0 20 read_log 192834'13430 (192834'13399) delete 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14910 2014-08-11 15:26:48.369185 2014-08-13 17:38:02.856552 7fe9bae887a0 20 read_log 192834'13474 (192834'13020) clean 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14954 2014-08-11 15:26:53.799572 2014-08-13 17:38:02.856565 7fe9bae887a0 20 read_log 192834'13479 (192834'13042) clean bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14959 2014-08-11 15:26:53.820989 2014-08-13 17:38:02.856576 7fe9bae887a0 20 read_log 192834'13481 (192834'13044) clean 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14961 2014-08-11 15:26:53.834148 2014-08-13 17:38:02.856588 7fe9bae887a0 20 read_log 192834'13500 (192834'13474) delete 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14980 2014-08-11 15:26:53.944512 2014-08-13 17:38:02.856600 7fe9bae887a0 20 read_log 192834'13505 (192834'13479) delete bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14985 2014-08-11 15:26:53.970510 2014-08-13 17:38:02.856640 7fe9bae887a0 20 read_log 192834'13507 (192834'13481) delete 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14987 2014-08-11 15:26:53.983309 2014-08-13 17:38:02.856653 7fe9bae887a0 20 read_log 192834'13540 (0'0) promote b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15020 2014-08-11 15:26:58.233964 2014-08-13 17:38:02.856665 7fe9bae887a0 20 read_log 192834'13541 (192834'13540) modify b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by client.1883634.0:11085 2014-08-11 15:26:58.226196 2014-08-13 17:38:02.856681 7fe9bae887a0 20 read_log 192834'13551 (192834'13102) clean dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:15031 2014-08-11 15:26:59.158127 2014-08-13 17:38:02.856693 7fe9bae887a0 20 read_log 192834'13555 (192834'13066) clean aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:15035 2014-08-11 15:26:59.179556 2014-08-13 17:38:02.856705 7fe9bae887a0 20 read_log 192834'13556 (192834'13114) clean 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:15036 2014-08-11 15:26:59.183337 2014-08-13 17:38:02.856717 7fe9bae887a0 20 read_log 192834'13578 (192834'13551) delete dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:15058 2014-08-11 15:26:59.285659 2014-08-13 17:38:02.856729 7fe9bae887a0 20 read_log 192834'13582 (192834'13555) delete aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:15062 2014-08-11 15:26:59.302575 2014-08-13 17:38:02.856740 7fe9bae887a0 20 read_log 192834'13583 (192834'13556) delete 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:15063 2014-08-11 15:26:59.306342 2014-08-13 17:38:02.856752 7fe9bae887a0 20 read_log 192834'13624 (192834'13188) clean 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:15104 2014-08-11 15:27:04.592638 2014-08-13 17:38:02.856764 7fe9bae887a0 20 read_log 192834'13632 (192834'13168) clean 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:15112 2014-08-11 15:27:04.616969 2014-08-13 17:38:02.856776 7fe9bae887a0 20 read_log 192834'13636 (192834'13204) clean ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:15116 2014-08-11 15:27:04.632024 2014-08-13 17:38:02.856791 7fe9bae887a0 20 read_log 192834'13637 (192834'13197) clean 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:15117 2014-08-11 15:27:04.635321 2014-08-13 17:38:02.856803 7fe9bae887a0 20 read_log 192834'13638 (192834'13171) clean b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:15118 2014-08-11 15:27:04.639248 2014-08-13 17:38:02.856815 7fe9bae887a0 20 read_log 192834'13680 (192834'13624) delete 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:15160 2014-08-11 15:27:04.763924 2014-08-13 17:38:02.856828 7fe9bae887a0 20 read_log 192834'13687 (192834'13632) delete 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:15167 2014-08-11 15:27:04.777246 2014-08-13 17:38:02.856840 7fe9bae887a0 20 read_log 192834'13692 (192834'13636) delete ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:15172 2014-08-11 15:27:04.786504 2014-08-13 17:38:02.856852 7fe9bae887a0 20 read_log 192834'13693 (192834'13637) delete 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:15173 2014-08-11 15:27:04.786878 2014-08-13 17:38:02.856867 7fe9bae887a0 20 read_log 192834'13694 (192834'13638) delete b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:15174 2014-08-11 15:27:04.787217 2014-08-13 17:38:02.856879 7fe9bae887a0 20 read_log 192834'13732 (0'0) promote 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15212 2014-08-11 15:27:07.014357 2014-08-13 17:38:02.856898 7fe9bae887a0 20 read_log 192834'13733 (192834'13732) modify 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by client.1883634.0:12962 2014-08-11 15:27:07.005624 2014-08-13 17:38:02.856911 7fe9bae887a0 20 read_log 192834'13748 (0'0) promote fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15228 2014-08-11 15:27:12.205432 2014-08-13 17:38:02.856923 7fe9bae887a0 20 read_log 192834'13749 (192834'13748) modify fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by client.1883634.0:14181 2014-08-11 15:27:12.198117 2014-08-13 17:38:02.856935 7fe9bae887a0 20 read_log 192834'13769 (192834'13217) clean 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15248 2014-08-11 15:27:20.009986 2014-08-13 17:38:02.856947 7fe9bae887a0 20 read_log 192834'13775 (192834'13769) delete 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15254 2014-08-11 15:27:20.094644 2014-08-13 17:38:02.856959 7fe9bae887a0 20 read_log 192834'13787 (192834'13229) clean 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15266 2014-08-11 15:27:25.270105 2014-08-13 17:38:02.856971 7fe9bae887a0 20 read_log 192834'13788 (192834'13231) clean bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15267 2014-08-11 15:27:25.270796 2014-08-13 17:38:02.856983 7fe9bae887a0 20 read_log 192834'13802 (192834'13787) delete 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15281 2014-08-11 15:27:25.535635 2014-08-13 17:38:02.856995 7fe9bae887a0 20 read_log 192834'13803 (192834'13788) delete bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15282 2014-08-11 15:27:25.541276 2014-08-13 17:38:02.857007 7fe9bae887a0 20 read_log 192834'13822 (0'0) promote 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15301 2014-08-11 15:27:26.657363 2014-08-13 17:38:02.857019 7fe9bae887a0 20 read_log 192834'13823 (192834'13822) modify 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by client.1883634.0:16256 2014-08-11 15:27:26.650085 2014-08-13 17:38:02.857031 7fe9bae887a0 20 read_log 192834'13826 (0'0) promote 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15305 2014-08-11 15:27:27.130044 2014-08-13 17:38:02.857042 7fe9bae887a0 20 read_log 192834'13827 (192834'13826) modify 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by client.1883634.0:16341 2014-08-11 15:27:27.121203 2014-08-13 17:38:02.857054 7fe9bae887a0 20 read_log 192834'13840 (0'0) promote cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15319 2014-08-11 15:27:29.220152 2014-08-13 17:38:02.857066 7fe9bae887a0 20 read_log 192834'13841 (192834'13840) modify cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by client.1883634.0:16764 2014-08-11 15:27:29.212198 2014-08-13 17:38:02.857078 7fe9bae887a0 20 read_log 192834'13842 (0'0) promote f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15321 2014-08-11 15:27:29.460252 2014-08-13 17:38:02.857090 7fe9bae887a0 20 read_log 192834'13843 (192834'13842) modify f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by client.1883634.0:16816 2014-08-11 15:27:29.451646 2014-08-13 17:38:02.857102 7fe9bae887a0 20 read_log 192834'13844 (0'0) promote f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15323 2014-08-11 15:27:29.599433 2014-08-13 17:38:02.857113 7fe9bae887a0 20 read_log 192834'13845 (192834'13844) modify f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by client.1883634.0:16842 2014-08-11 15:27:29.589944 2014-08-13 17:38:02.857125 7fe9bae887a0 20 read_log 192834'13853 (192834'13273) clean c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15332 2014-08-11 15:27:30.724336 2014-08-13 17:38:02.857148 7fe9bae887a0 20 read_log 192834'13854 (192834'13279) clean 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15333 2014-08-11 15:27:30.725216 2014-08-13 17:38:02.857173 7fe9bae887a0 20 read_log 192834'13857 (192834'13277) clean 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15336 2014-08-11 15:27:30.816144 2014-08-13 17:38:02.857189 7fe9bae887a0 20 read_log 192834'13861 (192834'13853) delete c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15340 2014-08-11 15:27:30.908256 2014-08-13 17:38:02.857201 7fe9bae887a0 20 read_log 192834'13862 (192834'13854) delete 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15341 2014-08-11 15:27:30.915000 2014-08-13 17:38:02.857213 7fe9bae887a0 20 read_log 192834'13865 (192834'13857) delete 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15344 2014-08-11 15:27:30.929356 2014-08-13 17:38:02.857225 7fe9bae887a0 20 read_log 192834'13905 (192834'13307) clean 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15384 2014-08-11 15:27:41.416738 2014-08-13 17:38:02.857237 7fe9bae887a0 20 read_log 192834'13910 (192834'13905) delete 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15389 2014-08-11 15:27:41.454668 2014-08-13 17:38:02.857249 7fe9bae887a0 20 read_log 192834'13913 (192834'13311) clean 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15392 2014-08-11 15:27:41.525475 2014-08-13 17:38:02.857261 7fe9bae887a0 20 read_log 192834'13919 (0'0) promote 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:15398 2014-08-11 15:27:46.410427 2014-08-13 17:38:02.857274 7fe9bae887a0 20 read_log 192834'13920 (192834'13919) modify 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by client.1883634.0:18550 2014-08-11 15:27:46.400894 2014-08-13 17:38:02.857286 7fe9bae887a0 20 read_log 192834'13925 (192834'13913) delete 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15404 2014-08-11 15:27:46.616644 2014-08-13 17:38:02.857298 7fe9bae887a0 20 read_log 192834'13927 (192834'13388) clean 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15406 2014-08-11 15:27:46.630123 2014-08-13 17:38:02.857310 7fe9bae887a0 20 read_log 192834'13931 (192834'13927) delete 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15410 2014-08-11 15:27:46.643973 2014-08-13 17:38:02.857322 7fe9bae887a0 20 read_log 192834'13933 (0'0) promote e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:15412 2014-08-11 15:27:47.887198 2014-08-13 17:38:02.857333 7fe9bae887a0 20 read_log 192834'13934 (192834'13933) modify e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by client.1883634.0:18856 2014-08-11 15:27:47.878517 2014-08-13 17:38:02.857346 7fe9bae887a0 20 read_log 192834'13952 (0'0) promote 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:15431 2014-08-11 15:27:52.963088 2014-08-13 17:38:02.857360 7fe9bae887a0 20 read_log 192834'13953 (192834'13952) modify 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by client.1883634.0:19497 2014-08-11 15:27:52.954620 2014-08-13 17:38:02.857373 7fe9bae887a0 20 read_log 192834'13993 (192834'13541) clean b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15472 2014-08-11 15:28:02.230971 2014-08-13 17:38:02.857385 7fe9bae887a0 20 read_log 192834'13999 (192834'13993) delete b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15478 2014-08-11 15:28:02.257656 2014-08-13 17:38:02.857397 7fe9bae887a0 20 read_log 192834'14004 (0'0) promote dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:15483 2014-08-11 15:28:03.803402 2014-08-13 17:38:02.857419 7fe9bae887a0 20 read_log 192834'14005 (192834'14004) modify dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by client.1883634.0:21326 2014-08-11 15:28:03.794049 2014-08-13 17:38:02.857432 7fe9bae887a0 20 read_log 192834'14016 (192834'13733) clean 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15495 2014-08-11 15:28:07.412325 2014-08-13 17:38:02.857444 7fe9bae887a0 20 read_log 192834'14019 (192834'14016) delete 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15498 2014-08-11 15:28:07.422644 2014-08-13 17:38:02.857456 7fe9bae887a0 20 read_log 192834'14037 (192834'13749) clean fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15516 2014-08-11 15:28:12.550306 2014-08-13 17:38:02.857468 7fe9bae887a0 20 read_log 192834'14047 (192834'14037) delete fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15526 2014-08-11 15:28:12.665014 2014-08-13 17:38:02.857480 7fe9bae887a0 20 read_log 192834'14075 (0'0) promote 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15554 2014-08-11 15:28:18.188358 2014-08-13 17:38:02.857492 7fe9bae887a0 20 read_log 192834'14076 (192834'14075) modify 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by client.1883634.0:23975 2014-08-11 15:28:18.160840 2014-08-13 17:38:02.857504 7fe9bae887a0 20 read_log 192834'14087 (0'0) promote 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15566 2014-08-11 15:28:19.759050 2014-08-13 17:38:02.857516 7fe9bae887a0 20 read_log 192834'14088 (192834'14087) modify 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by client.1883634.0:24901 2014-08-11 15:28:19.732481 2014-08-13 17:38:02.857527 7fe9bae887a0 20 read_log 192834'14089 (0'0) promote bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15568 2014-08-11 15:28:19.890344 2014-08-13 17:38:02.857539 7fe9bae887a0 20 read_log 192834'14090 (192834'14089) modify bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by client.1883634.0:24987 2014-08-11 15:28:19.865440 2014-08-13 17:38:02.857551 7fe9bae887a0 20 read_log 192834'14135 (0'0) promote c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15614 2014-08-11 15:28:24.446003 2014-08-13 17:38:02.857563 7fe9bae887a0 20 read_log 192834'14136 (192834'14135) modify c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by client.1883634.0:27369 2014-08-11 15:28:24.419778 2014-08-13 17:38:02.857575 7fe9bae887a0 20 read_log 192834'14139 (0'0) promote 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15618 2014-08-11 15:28:24.702334 2014-08-13 17:38:02.857587 7fe9bae887a0 20 read_log 192834'14140 (192834'14139) modify 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by client.1883634.0:27599 2014-08-11 15:28:24.673644 2014-08-13 17:38:02.857599 7fe9bae887a0 20 read_log 192834'14141 (0'0) promote 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15620 2014-08-11 15:28:24.799560 2014-08-13 17:38:02.857611 7fe9bae887a0 20 read_log 192834'14142 (192834'14141) modify 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by client.1883634.0:27663 2014-08-11 15:28:24.768689 2014-08-13 17:38:02.857623 7fe9bae887a0 20 read_log 192834'14169 (0'0) promote 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15648 2014-08-11 15:28:27.760475 2014-08-13 17:38:02.857635 7fe9bae887a0 20 read_log 192834'14170 (192834'14169) modify 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by client.1883634.0:29832 2014-08-11 15:28:27.731672 2014-08-13 17:38:02.857647 7fe9bae887a0 20 read_log 192834'14175 (192834'13823) clean 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15654 2014-08-11 15:28:28.057740 2014-08-13 17:38:02.857684 7fe9bae887a0 20 read_log 192834'14176 (192834'13827) clean 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15655 2014-08-11 15:28:28.085509 2014-08-13 17:38:02.857701 7fe9bae887a0 20 read_log 192834'14183 (192834'14175) delete 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15662 2014-08-11 15:28:28.216863 2014-08-13 17:38:02.857713 7fe9bae887a0 20 read_log 192834'14184 (192834'14176) delete 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15663 2014-08-11 15:28:28.221008 2014-08-13 17:38:02.857725 7fe9bae887a0 20 read_log 192834'14189 (0'0) promote 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15668 2014-08-11 15:28:28.655567 2014-08-13 17:38:02.857737 7fe9bae887a0 20 read_log 192834'14190 (192834'14189) modify 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by client.1883634.0:30430 2014-08-11 15:28:28.627326 2014-08-13 17:38:02.857749 7fe9bae887a0 20 read_log 192834'14199 (0'0) promote 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15678 2014-08-11 15:28:29.742267 2014-08-13 17:38:02.857761 7fe9bae887a0 20 read_log 192834'14200 (192834'14199) modify 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by client.1883634.0:31008 2014-08-11 15:28:29.715699 2014-08-13 17:38:02.857773 7fe9bae887a0 20 read_log 192834'14220 (192834'13843) clean f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15699 2014-08-11 15:28:33.450009 2014-08-13 17:38:02.857785 7fe9bae887a0 20 read_log 192834'14222 (192834'13841) clean cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15701 2014-08-11 15:28:33.451178 2014-08-13 17:38:02.857796 7fe9bae887a0 20 read_log 192834'14225 (192834'13845) clean f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15704 2014-08-11 15:28:33.546151 2014-08-13 17:38:02.857808 7fe9bae887a0 20 read_log 192834'14230 (192834'14220) delete f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15709 2014-08-11 15:28:33.571736 2014-08-13 17:38:02.857820 7fe9bae887a0 20 read_log 192834'14232 (192834'14222) delete cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15711 2014-08-11 15:28:33.583530 2014-08-13 17:38:02.857832 7fe9bae887a0 20 read_log 192834'14233 (192834'14225) delete f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15712 2014-08-11 15:28:33.593194 2014-08-13 17:38:02.857862 7fe9bae887a0 20 read_log 192834'14248 (0'0) promote b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15727 2014-08-11 15:28:34.679520 2014-08-13 17:38:02.857875 7fe9bae887a0 20 read_log 192834'14249 (192834'14248) modify b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by client.1883634.0:34409 2014-08-11 15:28:34.652350 2014-08-13 17:38:02.857887 7fe9bae887a0 20 read_log 192834'14268 (0'0) promote 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15747 2014-08-11 15:28:37.862958 2014-08-13 17:38:02.857899 7fe9bae887a0 20 read_log 192834'14269 (192834'14268) modify 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by client.1883634.0:36286 2014-08-11 15:28:37.833319 2014-08-13 17:38:02.857912 7fe9bae887a0 20 read_log 192834'14289 (0'0) promote fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15768 2014-08-11 15:28:39.483446 2014-08-13 17:38:02.857924 7fe9bae887a0 20 read_log 192834'14290 (192834'14289) modify fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by client.1883634.0:37505 2014-08-11 15:28:39.446720 2014-08-13 17:38:02.857936 7fe9bae887a0 20 read_log 192834'14309 (0'0) promote 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15788 2014-08-11 15:28:42.356531 2014-08-13 17:38:02.857955 7fe9bae887a0 20 read_log 192834'14310 (192834'14309) modify 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by client.1883634.0:39580 2014-08-11 15:28:42.329208 2014-08-13 17:38:02.857970 7fe9bae887a0 20 read_log 192834'14313 (0'0) promote 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15792 2014-08-11 15:28:42.452662 2014-08-13 17:38:02.857983 7fe9bae887a0 20 read_log 192834'14314 (192834'14313) modify 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by client.1883634.0:39665 2014-08-11 15:28:42.414195 2014-08-13 17:38:02.857995 7fe9bae887a0 20 read_log 192834'14327 (0'0) promote cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15806 2014-08-11 15:28:43.176310 2014-08-13 17:38:02.858006 7fe9bae887a0 20 read_log 192834'14328 (192834'14327) modify cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by client.1883634.0:40088 2014-08-11 15:28:43.146358 2014-08-13 17:38:02.858018 7fe9bae887a0 20 read_log 192834'14329 (0'0) promote f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15808 2014-08-11 15:28:43.262364 2014-08-13 17:38:02.858030 7fe9bae887a0 20 read_log 192834'14330 (192834'14329) modify f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by client.1883634.0:40140 2014-08-11 15:28:43.233934 2014-08-13 17:38:02.858042 7fe9bae887a0 20 read_log 192834'14331 (0'0) promote f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15810 2014-08-11 15:28:43.307737 2014-08-13 17:38:02.858054 7fe9bae887a0 20 read_log 192834'14332 (192834'14331) modify f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by client.1883634.0:40166 2014-08-11 15:28:43.283009 2014-08-13 17:38:02.858066 7fe9bae887a0 20 read_log 192834'14357 (192834'13920) modify 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by client.1883634.0:41874 2014-08-11 15:28:45.558637 2014-08-13 17:38:02.858078 7fe9bae887a0 20 read_log 192834'14358 (192834'13934) modify e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by client.1883634.0:42180 2014-08-11 15:28:46.162057 2014-08-13 17:38:02.858090 7fe9bae887a0 20 read_log 192834'14362 (192834'13953) modify 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by client.1883634.0:42821 2014-08-11 15:28:46.368950 2014-08-13 17:38:02.858102 7fe9bae887a0 20 read_log 192834'14375 (192834'14005) modify dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by client.1883634.0:44650 2014-08-11 15:28:46.916100 2014-08-13 17:38:02.858114 7fe9bae887a0 20 read_log 192834'14388 (192834'14076) clean 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15867 2014-08-11 15:29:19.254297 2014-08-13 17:38:02.858126 7fe9bae887a0 20 read_log 192834'14396 (192834'14388) delete 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15875 2014-08-11 15:29:19.320673 2014-08-13 17:38:02.858138 7fe9bae887a0 20 read_log 192834'14402 (192834'14136) clean c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15881 2014-08-11 15:29:24.435246 2014-08-13 17:38:02.858150 7fe9bae887a0 20 read_log 192834'14404 (192834'14088) clean 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15883 2014-08-11 15:29:24.442074 2014-08-13 17:38:02.858173 7fe9bae887a0 20 read_log 192834'14405 (192834'14090) clean bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15884 2014-08-11 15:29:24.442301 2014-08-13 17:38:02.858185 7fe9bae887a0 20 read_log 192834'14427 (192834'14402) delete c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15906 2014-08-11 15:29:24.539314 2014-08-13 17:38:02.858197 7fe9bae887a0 20 read_log 192834'14429 (192834'14404) delete 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15908 2014-08-11 15:29:24.550825 2014-08-13 17:38:02.858221 7fe9bae887a0 20 read_log 192834'14430 (192834'14405) delete bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15909 2014-08-11 15:29:24.554786 2014-08-13 17:38:02.858237 7fe9bae887a0 20 read_log 192834'14460 (192834'14170) clean 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15939 2014-08-11 15:29:29.719598 2014-08-13 17:38:02.858250 7fe9bae887a0 20 read_log 192834'14461 (192834'14142) clean 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15940 2014-08-11 15:29:29.725283 2014-08-13 17:38:02.858261 7fe9bae887a0 20 read_log 192834'14468 (192834'14200) clean 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15947 2014-08-11 15:29:29.748375 2014-08-13 17:38:02.858273 7fe9bae887a0 20 read_log 192834'14469 (192834'14140) clean 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15948 2014-08-11 15:29:29.754633 2014-08-13 17:38:02.858285 7fe9bae887a0 20 read_log 192834'14470 (192834'14190) clean 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15949 2014-08-11 15:29:29.755020 2014-08-13 17:38:02.858297 7fe9bae887a0 20 read_log 192834'14483 (192834'14460) delete 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15962 2014-08-11 15:29:29.798417 2014-08-13 17:38:02.858309 7fe9bae887a0 20 read_log 192834'14484 (192834'14461) delete 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15963 2014-08-11 15:29:29.801766 2014-08-13 17:38:02.858321 7fe9bae887a0 20 read_log 192834'14491 (192834'14468) delete 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15970 2014-08-11 15:29:29.822667 2014-08-13 17:38:02.858333 7fe9bae887a0 20 read_log 192834'14492 (192834'14469) delete 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15971 2014-08-11 15:29:29.825377 2014-08-13 17:38:02.858345 7fe9bae887a0 20 read_log 192834'14493 (192834'14470) delete 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15972 2014-08-11 15:29:29.828145 2014-08-13 17:38:02.858356 7fe9bae887a0 20 read_log 192834'14506 (192834'14249) clean b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15985 2014-08-11 15:29:34.911828 2014-08-13 17:38:02.858368 7fe9bae887a0 20 read_log 192834'14520 (192834'14506) delete b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15999 2014-08-11 15:29:34.961135 2014-08-13 17:38:02.858380 7fe9bae887a0 20 read_log 192834'14532 (192834'14269) clean 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:16011 2014-08-11 15:29:40.016106 2014-08-13 17:38:02.858392 7fe9bae887a0 20 read_log 192834'14537 (192834'14290) clean fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:16016 2014-08-11 15:29:40.032764 2014-08-13 17:38:02.858404 7fe9bae887a0 20 read_log 192834'14552 (192834'14532) delete 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:16031 2014-08-11 15:29:40.075336 2014-08-13 17:38:02.858416 7fe9bae887a0 20 read_log 192834'14557 (192834'14537) delete fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:16036 2014-08-11 15:29:40.090168 2014-08-13 17:38:02.858428 7fe9bae887a0 20 read_log 192834'14576 (192834'14330) clean f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:16055 2014-08-11 15:29:45.153743 2014-08-13 17:38:02.858440 7fe9bae887a0 20 read_log 192834'14579 (192834'14328) clean cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:16058 2014-08-11 15:29:45.161286 2014-08-13 17:38:02.858452 7fe9bae887a0 20 read_log 192834'14581 (192834'14310) clean 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:16060 2014-08-11 15:29:45.163109 2014-08-13 17:38:02.858471 7fe9bae887a0 20 read_log 192834'14582 (192834'14314) clean 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:16061 2014-08-11 15:29:45.165147 2014-08-13 17:38:02.858484 7fe9bae887a0 20 read_log 192834'14583 (192834'14332) clean f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:16062 2014-08-11 15:29:45.170133 2014-08-13 17:38:02.858496 7fe9bae887a0 20 read_log 192834'14602 (192834'14576) delete f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:16081 2014-08-11 15:29:45.220465 2014-08-13 17:38:02.858507 7fe9bae887a0 20 read_log 192834'14606 (192834'14579) delete cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:16085 2014-08-11 15:29:45.225400 2014-08-13 17:38:02.858519 7fe9bae887a0 20 read_log 192834'14608 (192834'14581) delete 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:16087 2014-08-11 15:29:45.227852 2014-08-13 17:38:02.858531 7fe9bae887a0 20 read_log 192834'14609 (192834'14582) delete 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:16088 2014-08-11 15:29:45.228280 2014-08-13 17:38:02.858543 7fe9bae887a0 20 read_log 192834'14610 (192834'14583) delete f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:16089 2014-08-11 15:29:45.230406 2014-08-13 17:38:02.858555 7fe9bae887a0 20 read_log 192834'14628 (192834'14358) clean e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:16107 2014-08-11 15:29:50.259508 2014-08-13 17:38:02.858567 7fe9bae887a0 20 read_log 192834'14629 (192834'14357) clean 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:16108 2014-08-11 15:29:50.260151 2014-08-13 17:38:02.858578 7fe9bae887a0 20 read_log 192834'14634 (192834'14362) clean 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:16113 2014-08-11 15:29:50.267181 2014-08-13 17:38:02.858590 7fe9bae887a0 20 read_log 192834'14635 (192834'14375) clean dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:16114 2014-08-11 15:29:50.268776 2014-08-13 17:38:02.858602 7fe9bae887a0 20 read_log 192834'14643 (192834'14628) delete e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:16122 2014-08-11 15:29:50.283268 2014-08-13 17:38:02.858614 7fe9bae887a0 20 read_log 192834'14646 (192834'14629) delete 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:16125 2014-08-11 15:29:50.285639 2014-08-13 17:38:02.858625 7fe9bae887a0 20 read_log 192834'14651 (192834'14634) delete 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:16130 2014-08-11 15:29:50.291084 2014-08-13 17:38:02.858637 7fe9bae887a0 20 read_log 192834'14652 (192834'14635) delete dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:16131 2014-08-11 15:29:50.291353 2014-08-13 17:38:02.858649 7fe9bae887a0 20 read_log 192888'14690 (192888'14690) modify 136/hit_set_24.136_archive_2014-08-11 16:02:49.190600_2014-08-11 16:44:58.717710/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.718577 2014-08-13 17:38:02.858662 7fe9bae887a0 20 read_log 192888'14691 (0'0) promote d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by osd.4.0:21405 2014-08-11 16:44:58.722761 2014-08-13 17:38:02.858674 7fe9bae887a0 20 read_log 192888'14692 (192888'14691) modify d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.1315259.0:907 2014-08-11 16:44:58.712700 2014-08-13 17:38:02.858687 7fe9bae887a0 20 read_log 192888'14699 (0'0) promote 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by osd.4.0:21490 2014-08-11 16:45:10.414474 2014-08-13 17:38:02.858707 7fe9bae887a0 20 read_log 192888'14700 (192888'14699) modify 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.1315259.0:3986 2014-08-11 16:45:10.402820 2014-08-13 17:38:02.858719 7fe9bae887a0 20 read_log 192888'14709 (0'0) promote d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by osd.4.0:21530 2014-08-11 16:45:14.586515 2014-08-13 17:38:02.858731 7fe9bae887a0 20 read_log 192888'14710 (192888'14709) modify d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.1315259.0:5118 2014-08-11 16:45:14.578130 2014-08-13 17:38:02.858743 7fe9bae887a0 20 read_log 192888'14711 (0'0) promote 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by osd.4.0:21532 2014-08-11 16:45:14.705578 2014-08-13 17:38:02.858755 7fe9bae887a0 20 read_log 192888'14712 (192888'14711) modify 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.1315259.0:5150 2014-08-11 16:45:14.697865 2014-08-13 17:38:02.858767 7fe9bae887a0 20 read_log 192888'14715 (0'0) promote 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by osd.4.0:21554 2014-08-11 16:45:17.045726 2014-08-13 17:38:02.858779 7fe9bae887a0 20 read_log 192888'14716 (192888'14715) modify 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.1315259.0:5794 2014-08-11 16:45:17.033330 2014-08-13 17:38:02.858791 7fe9bae887a0 20 read_log 192888'14717 (0'0) promote 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by osd.4.0:21570 2014-08-11 16:45:19.261820 2014-08-13 17:38:02.858803 7fe9bae887a0 20 read_log 192888'14718 (192888'14717) modify 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.1315259.0:6409 2014-08-11 16:45:19.252967 2014-08-13 17:38:02.858815 7fe9bae887a0 20 read_log 192888'14723 (0'0) promote f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by osd.4.0:21596 2014-08-11 16:45:23.961112 2014-08-13 17:38:02.858827 7fe9bae887a0 20 read_log 192888'14724 (192888'14723) modify f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.1315259.0:7710 2014-08-11 16:45:23.950175 2014-08-13 17:38:02.858839 7fe9bae887a0 20 read_log 192888'14725 (0'0) promote fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by osd.4.0:21618 2014-08-11 16:45:27.126301 2014-08-13 17:38:02.858851 7fe9bae887a0 20 read_log 192888'14726 (192888'14725) modify fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.1315259.0:8542 2014-08-11 16:45:27.117895 2014-08-13 17:38:02.858863 7fe9bae887a0 20 read_log 192888'14735 (0'0) promote bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by osd.4.0:21678 2014-08-11 16:45:35.747447 2014-08-13 17:38:02.858875 7fe9bae887a0 20 read_log 192888'14736 (192888'14735) modify bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.1315259.0:10782 2014-08-11 16:45:35.738505 2014-08-13 17:38:02.858887 7fe9bae887a0 20 read_log 192888'14741 (0'0) promote b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by osd.4.0:21738 2014-08-11 16:45:44.956458 2014-08-13 17:38:02.858899 7fe9bae887a0 20 read_log 192888'14742 (192888'14741) modify b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.1315259.0:13022 2014-08-11 16:45:44.947879 2014-08-13 17:38:02.858911 7fe9bae887a0 20 read_log 192888'14743 (0'0) promote 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by osd.4.0:21762 2014-08-11 16:45:49.158999 2014-08-13 17:38:02.858922 7fe9bae887a0 20 read_log 192888'14744 (192888'14743) modify 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.1315259.0:14101 2014-08-11 16:45:49.149678 2014-08-13 17:38:02.858934 7fe9bae887a0 20 read_log 192888'14745 (0'0) promote 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by osd.4.0:21770 2014-08-11 16:45:50.824551 2014-08-13 17:38:02.858954 7fe9bae887a0 20 read_log 192888'14746 (192888'14745) modify 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.1315259.0:14535 2014-08-11 16:45:50.816381 2014-08-13 17:38:02.858967 7fe9bae887a0 20 read_log 192888'14751 (0'0) promote 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by osd.4.0:21802 2014-08-11 16:45:55.654601 2014-08-13 17:38:02.858979 7fe9bae887a0 20 read_log 192888'14752 (192888'14751) modify 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.1315259.0:15795 2014-08-11 16:45:55.644691 2014-08-13 17:38:02.858991 7fe9bae887a0 20 read_log 192888'14753 (0'0) promote de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by osd.4.0:21844 2014-08-11 16:46:00.692846 2014-08-13 17:38:02.859003 7fe9bae887a0 20 read_log 192888'14754 (192888'14753) modify de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.1315259.0:17084 2014-08-11 16:46:00.684857 2014-08-13 17:38:02.859015 7fe9bae887a0 20 read_log 192888'14755 (0'0) promote 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by osd.4.0:21878 2014-08-11 16:46:04.754933 2014-08-13 17:38:02.859027 7fe9bae887a0 20 read_log 192888'14756 (192888'14755) modify 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.1315259.0:18088 2014-08-11 16:46:04.746767 2014-08-13 17:38:02.859057 7fe9bae887a0 20 read_log 192888'14761 (0'0) promote f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by osd.4.0:21910 2014-08-11 16:46:08.512692 2014-08-13 17:38:02.859070 7fe9bae887a0 20 read_log 192888'14762 (192888'14761) modify f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.1315259.0:19058 2014-08-11 16:46:08.502735 2014-08-13 17:38:02.859082 7fe9bae887a0 20 read_log 192888'14765 (0'0) promote f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by osd.4.0:21920 2014-08-11 16:46:10.436109 2014-08-13 17:38:02.859094 7fe9bae887a0 20 read_log 192888'14766 (192888'14765) modify f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.1315259.0:19541 2014-08-11 16:46:10.426708 2014-08-13 17:38:02.859106 7fe9bae887a0 20 read_log 192888'14767 (0'0) promote 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by osd.4.0:21932 2014-08-11 16:46:12.044100 2014-08-13 17:38:02.859118 7fe9bae887a0 20 read_log 192888'14768 (192888'14767) modify 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.1315259.0:19978 2014-08-11 16:46:12.035433 2014-08-13 17:38:02.859130 7fe9bae887a0 20 read_log 192888'14771 (0'0) promote 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by osd.4.0:21948 2014-08-11 16:46:14.966086 2014-08-13 17:38:02.859142 7fe9bae887a0 20 read_log 192888'14772 (192888'14771) modify 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.1315259.0:20752 2014-08-11 16:46:14.957070 2014-08-13 17:38:02.859160 7fe9bae887a0 20 read_log 192888'14773 (0'0) promote cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by osd.4.0:21964 2014-08-11 16:46:16.388696 2014-08-13 17:38:02.859172 7fe9bae887a0 20 read_log 192888'14774 (192888'14773) modify cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.1315259.0:21139 2014-08-11 16:46:16.378753 2014-08-13 17:38:02.859185 7fe9bae887a0 20 read_log 192888'14775 (0'0) promote d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by osd.4.0:21982 2014-08-11 16:46:19.267371 2014-08-13 17:38:02.859196 7fe9bae887a0 20 read_log 192888'14776 (192888'14775) modify d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.1315259.0:21896 2014-08-11 16:46:19.259496 2014-08-13 17:38:02.859208 7fe9bae887a0 20 read_log 192888'14777 (0'0) promote 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by osd.4.0:22026 2014-08-11 16:46:26.229908 2014-08-13 17:38:02.859227 7fe9bae887a0 20 read_log 192888'14778 (192888'14777) modify 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.1315259.0:23626 2014-08-11 16:46:26.221699 2014-08-13 17:38:02.859240 7fe9bae887a0 20 read_log 192888'14781 (0'0) promote ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by osd.4.0:22094 2014-08-11 16:46:33.590742 2014-08-13 17:38:02.859252 7fe9bae887a0 20 read_log 192888'14782 (192888'14781) modify ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.1315259.0:25515 2014-08-11 16:46:33.582714 2014-08-13 17:38:02.859264 7fe9bae887a0 20 read_log 192888'14785 (0'0) promote 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by osd.4.0:22112 2014-08-11 16:46:37.009214 2014-08-13 17:38:02.859276 7fe9bae887a0 20 read_log 192888'14786 (192888'14785) modify 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.1315259.0:26303 2014-08-11 16:46:37.000753 2014-08-13 17:38:02.859288 7fe9bae887a0 20 read_log 192888'14789 (0'0) promote 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by osd.4.0:22130 2014-08-11 16:46:39.288439 2014-08-13 17:38:02.859299 7fe9bae887a0 20 read_log 192888'14790 (192888'14789) modify 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.1315259.0:26856 2014-08-11 16:46:39.280206 2014-08-13 17:38:02.859312 7fe9bae887a0 20 read_log 192888'14791 (0'0) promote 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by osd.4.0:22138 2014-08-11 16:46:40.808934 2014-08-13 17:38:02.859324 7fe9bae887a0 20 read_log 192888'14792 (192888'14791) modify 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.1315259.0:27190 2014-08-11 16:46:40.800424 2014-08-13 17:38:02.859339 7fe9bae887a0 20 read_log 192888'14797 (0'0) promote de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by osd.4.0:22160 2014-08-11 16:46:45.435962 2014-08-13 17:38:02.859351 7fe9bae887a0 20 read_log 192888'14798 (192888'14797) modify de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.1315259.0:28354 2014-08-11 16:46:45.428112 2014-08-13 17:38:02.859362 7fe9bae887a0 20 read_log 192888'14801 (0'0) promote 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by osd.4.0:22206 2014-08-11 16:46:53.077404 2014-08-13 17:38:02.859374 7fe9bae887a0 20 read_log 192888'14802 (192888'14801) modify 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.1315259.0:30259 2014-08-11 16:46:53.069164 2014-08-13 17:38:02.859386 7fe9bae887a0 20 read_log 192888'14805 (192888'14692) modify d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.1315259.0:31634 2014-08-11 16:46:56.152826 2014-08-13 17:38:02.859398 7fe9bae887a0 20 read_log 192888'14809 (192888'14700) modify 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.1315259.0:34713 2014-08-11 16:46:56.967895 2014-08-13 17:38:02.859410 7fe9bae887a0 20 read_log 192888'14814 (192888'14710) modify d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.1315259.0:35845 2014-08-11 16:46:57.261069 2014-08-13 17:38:02.859422 7fe9bae887a0 20 read_log 192888'14815 (192888'14712) modify 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.1315259.0:35877 2014-08-11 16:46:57.268588 2014-08-13 17:38:02.859434 7fe9bae887a0 20 read_log 192888'14817 (192888'14716) modify 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.1315259.0:36521 2014-08-11 16:46:57.444707 2014-08-13 17:38:02.859445 7fe9bae887a0 20 read_log 192888'14818 (192888'14718) modify 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.1315259.0:37136 2014-08-11 16:46:57.603716 2014-08-13 17:38:02.859458 7fe9bae887a0 20 read_log 192888'14821 (192888'14724) modify f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.1315259.0:38437 2014-08-11 16:46:57.947169 2014-08-13 17:38:02.859493 7fe9bae887a0 20 read_log 192888'14822 (192888'14726) modify fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.1315259.0:39269 2014-08-11 16:46:58.168996 2014-08-13 17:38:02.859506 7fe9bae887a0 20 read_log 192888'14827 (192888'14736) modify bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.1315259.0:41509 2014-08-11 16:46:58.763963 2014-08-13 17:38:02.859519 7fe9bae887a0 20 read_log 192888'14830 (192888'14742) modify b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.1315259.0:43749 2014-08-11 16:46:59.378358 2014-08-13 17:38:02.859531 7fe9bae887a0 20 read_log 192888'14831 (192888'14744) modify 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.1315259.0:44828 2014-08-11 16:46:59.688620 2014-08-13 17:38:02.859543 7fe9bae887a0 20 read_log 192888'14832 (192888'14746) modify 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.1315259.0:45262 2014-08-11 16:46:59.816251 2014-08-13 17:38:02.859554 7fe9bae887a0 20 read_log 192888'14835 (192888'14752) modify 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.1315259.0:46522 2014-08-11 16:47:00.183303 2014-08-13 17:38:02.859566 7fe9bae887a0 20 read_log 192888'14836 (192888'14754) modify de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.1315259.0:47811 2014-08-11 16:47:00.577208 2014-08-13 17:38:02.859578 7fe9bae887a0 20 read_log 192888'14837 (192888'14756) modify 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.1315259.0:48815 2014-08-11 16:47:00.873874 2014-08-13 17:38:02.859590 7fe9bae887a0 20 read_log 192888'14840 (192888'14762) modify f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.1315259.0:49785 2014-08-11 16:47:01.172149 2014-08-13 17:38:02.859602 7fe9bae887a0 20 read_log 192888'14842 (192888'14766) modify f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.1315259.0:50268 2014-08-11 16:47:01.315666 2014-08-13 17:38:02.859615 7fe9bae887a0 20 read_log 192888'14843 (192888'14768) modify 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.1315259.0:50705 2014-08-11 16:47:01.452292 2014-08-13 17:38:02.859627 7fe9bae887a0 20 read_log 192888'14845 (192888'14772) modify 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.1315259.0:51479 2014-08-11 16:47:01.691909 2014-08-13 17:38:02.859639 7fe9bae887a0 20 read_log 192888'14846 (192888'14774) modify cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.1315259.0:51866 2014-08-11 16:47:01.805766 2014-08-13 17:38:02.859651 7fe9bae887a0 20 read_log 192888'14847 (192888'14776) modify d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.1315259.0:52623 2014-08-11 16:47:02.025953 2014-08-13 17:38:02.859663 7fe9bae887a0 20 read_log 192888'14848 (192888'14778) modify 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.1315259.0:54353 2014-08-11 16:47:02.543323 2014-08-13 17:38:02.859675 7fe9bae887a0 20 read_log 192888'14850 (192888'14782) modify ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.1315259.0:56242 2014-08-11 16:47:03.144404 2014-08-13 17:38:02.859687 7fe9bae887a0 20 read_log 192888'14852 (192888'14786) modify 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.1315259.0:57030 2014-08-11 16:47:03.390023 2014-08-13 17:38:02.859699 7fe9bae887a0 20 read_log 192888'14854 (192888'14790) modify 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.1315259.0:57583 2014-08-11 16:47:03.554640 2014-08-13 17:38:02.859711 7fe9bae887a0 20 read_log 192888'14855 (192888'14792) modify 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.1315259.0:57917 2014-08-11 16:47:03.649092 2014-08-13 17:38:02.859722 7fe9bae887a0 20 read_log 192888'14858 (192888'14798) modify de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.1315259.0:59081 2014-08-11 16:47:03.993833 2014-08-13 17:38:02.859744 7fe9bae887a0 20 read_log 192888'14860 (192888'14802) modify 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.1315259.0:60986 2014-08-11 16:47:04.599846 2014-08-13 17:38:02.859757 7fe9bae887a0 20 read_log 192888'14862 (192888'14862) modify 136/hit_set_24.136_archive_2014-08-11 16:44:58.718710_2014-08-11 17:05:15.819389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.819628 2014-08-13 17:38:02.859769 7fe9bae887a0 20 read_log 192888'14863 (192888'14690) delete 136/hit_set_24.136_archive_2014-08-11 16:02:49.190600_2014-08-11 16:44:58.717710/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.819628 2014-08-13 17:38:02.859782 7fe9bae887a0 20 read_log 192888'14864 (0'0) promote 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by osd.4.0:22651 2014-08-11 17:05:15.824490 2014-08-13 17:38:02.859794 7fe9bae887a0 20 read_log 192888'14865 (192888'14864) modify 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.942868.0:201 2014-08-11 17:05:15.813467 2014-08-13 17:38:02.859806 7fe9bae887a0 20 read_log 192888'14870 (0'0) promote ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by osd.4.0:22716 2014-08-11 17:05:25.976498 2014-08-13 17:38:02.859818 7fe9bae887a0 20 read_log 192888'14871 (192888'14870) modify ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.942868.0:2711 2014-08-11 17:05:25.967623 2014-08-13 17:38:02.859830 7fe9bae887a0 20 read_log 192888'14874 (0'0) promote cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by osd.4.0:22752 2014-08-11 17:05:30.458283 2014-08-13 17:38:02.859842 7fe9bae887a0 20 read_log 192888'14875 (192888'14874) modify cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.942868.0:3924 2014-08-11 17:05:30.448644 2014-08-13 17:38:02.859854 7fe9bae887a0 20 read_log 192888'14876 (0'0) promote d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by osd.4.0:22756 2014-08-11 17:05:31.292454 2014-08-13 17:38:02.859865 7fe9bae887a0 20 read_log 192888'14877 (192888'14876) modify d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.942868.0:4151 2014-08-11 17:05:31.284720 2014-08-13 17:38:02.859877 7fe9bae887a0 20 read_log 192888'14878 (0'0) promote 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by osd.4.0:22770 2014-08-11 17:05:32.042389 2014-08-13 17:38:02.859889 7fe9bae887a0 20 read_log 192888'14879 (192888'14878) modify 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.942868.0:4355 2014-08-11 17:05:32.034783 2014-08-13 17:38:02.859902 7fe9bae887a0 20 read_log 192888'14882 (0'0) promote 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by osd.4.0:22796 2014-08-11 17:05:36.126236 2014-08-13 17:38:02.859914 7fe9bae887a0 20 read_log 192888'14883 (192888'14882) modify 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.942868.0:5402 2014-08-11 17:05:36.118119 2014-08-13 17:38:02.859926 7fe9bae887a0 20 read_log 192888'14890 (0'0) promote b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by osd.4.0:22836 2014-08-11 17:05:40.585835 2014-08-13 17:38:02.859938 7fe9bae887a0 20 read_log 192888'14891 (192888'14890) modify b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.942868.0:6628 2014-08-11 17:05:40.575227 2014-08-13 17:38:02.859950 7fe9bae887a0 20 read_log 192888'14898 (0'0) promote 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by osd.4.0:22896 2014-08-11 17:05:50.050596 2014-08-13 17:38:02.859962 7fe9bae887a0 20 read_log 192888'14899 (192888'14898) modify 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.942868.0:9147 2014-08-11 17:05:50.041369 2014-08-13 17:38:02.859981 7fe9bae887a0 20 read_log 192888'14900 (0'0) promote 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by osd.4.0:22900 2014-08-11 17:05:50.739871 2014-08-13 17:38:02.859994 7fe9bae887a0 20 read_log 192888'14901 (192888'14900) modify 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.942868.0:9330 2014-08-11 17:05:50.729715 2014-08-13 17:38:02.860006 7fe9bae887a0 20 read_log 192888'14902 (0'0) promote b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by osd.4.0:22918 2014-08-11 17:05:54.495245 2014-08-13 17:38:02.860018 7fe9bae887a0 20 read_log 192888'14903 (192888'14902) modify b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.942868.0:10357 2014-08-11 17:05:54.484795 2014-08-13 17:38:02.860030 7fe9bae887a0 20 read_log 192888'14908 (0'0) promote b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by osd.4.0:22928 2014-08-11 17:05:55.293341 2014-08-13 17:38:02.860042 7fe9bae887a0 20 read_log 192888'14909 (192888'14908) modify b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.942868.0:10580 2014-08-11 17:05:55.285062 2014-08-13 17:38:02.860054 7fe9bae887a0 20 read_log 192888'14912 (0'0) promote 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by osd.4.0:22974 2014-08-11 17:06:03.090657 2014-08-13 17:38:02.860066 7fe9bae887a0 20 read_log 192888'14913 (192888'14912) modify 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.942868.0:12721 2014-08-11 17:06:03.079712 2014-08-13 17:38:02.860078 7fe9bae887a0 20 read_log 192888'14916 (0'0) promote 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by osd.4.0:23036 2014-08-11 17:06:13.589716 2014-08-13 17:38:02.860090 7fe9bae887a0 20 read_log 192888'14917 (192888'14916) modify 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.942868.0:15350 2014-08-11 17:06:13.578192 2014-08-13 17:38:02.860102 7fe9bae887a0 20 read_log 192888'14922 (0'0) promote d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by osd.4.0:23064 2014-08-11 17:06:17.643145 2014-08-13 17:38:02.860113 7fe9bae887a0 20 read_log 192888'14923 (192888'14922) modify d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.942868.0:16398 2014-08-11 17:06:17.632836 2014-08-13 17:38:02.860126 7fe9bae887a0 20 read_log 192888'14924 (0'0) promote 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by osd.4.0:23068 2014-08-11 17:06:18.733220 2014-08-13 17:38:02.860138 7fe9bae887a0 20 read_log 192888'14925 (192888'14924) modify 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.942868.0:16661 2014-08-11 17:06:18.725521 2014-08-13 17:38:02.860150 7fe9bae887a0 20 read_log 192888'14926 (0'0) promote 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by osd.4.0:23080 2014-08-11 17:06:20.054930 2014-08-13 17:38:02.860171 7fe9bae887a0 20 read_log 192888'14927 (192888'14926) modify 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.942868.0:16998 2014-08-11 17:06:20.046703 2014-08-13 17:38:02.860184 7fe9bae887a0 20 read_log 192888'14928 (0'0) promote 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by osd.4.0:23082 2014-08-11 17:06:20.320454 2014-08-13 17:38:02.860196 7fe9bae887a0 20 read_log 192888'14929 (192888'14928) modify 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.942868.0:17068 2014-08-11 17:06:20.311629 2014-08-13 17:38:02.860229 7fe9bae887a0 20 read_log 192888'14930 (0'0) promote 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by osd.4.0:23102 2014-08-11 17:06:22.293012 2014-08-13 17:38:02.860246 7fe9bae887a0 20 read_log 192888'14931 (192888'14930) modify 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.942868.0:17590 2014-08-11 17:06:22.283891 2014-08-13 17:38:02.860281 7fe9bae887a0 20 read_log 192888'14932 (0'0) promote 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by osd.4.0:23106 2014-08-11 17:06:22.737968 2014-08-13 17:38:02.860313 7fe9bae887a0 20 read_log 192888'14933 (192888'14932) modify 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.942868.0:17708 2014-08-11 17:06:22.729526 2014-08-13 17:38:02.860328 7fe9bae887a0 20 read_log 192888'14934 (0'0) promote f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by osd.4.0:23126 2014-08-11 17:06:25.907190 2014-08-13 17:38:02.860340 7fe9bae887a0 20 read_log 192888'14935 (192888'14934) modify f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.942868.0:18549 2014-08-11 17:06:25.897516 2014-08-13 17:38:02.860352 7fe9bae887a0 20 read_log 192888'14936 (0'0) promote cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by osd.4.0:23146 2014-08-11 17:06:28.687524 2014-08-13 17:38:02.860364 7fe9bae887a0 20 read_log 192888'14937 (192888'14936) modify cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.942868.0:19288 2014-08-11 17:06:28.678528 2014-08-13 17:38:02.860376 7fe9bae887a0 20 read_log 192888'14940 (0'0) promote 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by osd.4.0:23166 2014-08-11 17:06:31.093834 2014-08-13 17:38:02.860388 7fe9bae887a0 20 read_log 192888'14941 (192888'14940) modify 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.942868.0:19927 2014-08-11 17:06:31.085201 2014-08-13 17:38:02.860401 7fe9bae887a0 20 read_log 192888'14944 (0'0) promote 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by osd.4.0:23184 2014-08-11 17:06:34.051456 2014-08-13 17:38:02.860413 7fe9bae887a0 20 read_log 192888'14945 (192888'14944) modify 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.942868.0:20698 2014-08-11 17:06:34.042873 2014-08-13 17:38:02.860425 7fe9bae887a0 20 read_log 192888'14950 (0'0) promote e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by osd.4.0:23218 2014-08-11 17:06:38.660949 2014-08-13 17:38:02.860437 7fe9bae887a0 20 read_log 192888'14951 (192888'14950) modify e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.942868.0:21915 2014-08-11 17:06:38.653278 2014-08-13 17:38:02.860449 7fe9bae887a0 20 read_log 192888'14952 (0'0) promote f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by osd.4.0:23232 2014-08-11 17:06:41.072538 2014-08-13 17:38:02.860461 7fe9bae887a0 20 read_log 192888'14953 (192888'14952) modify f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.942868.0:22516 2014-08-11 17:06:41.064647 2014-08-13 17:38:02.860473 7fe9bae887a0 20 read_log 192888'14956 (0'0) promote fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by osd.4.0:23242 2014-08-11 17:06:41.993908 2014-08-13 17:38:02.860485 7fe9bae887a0 20 read_log 192888'14957 (192888'14956) modify fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.942868.0:22765 2014-08-11 17:06:41.985848 2014-08-13 17:38:02.860497 7fe9bae887a0 20 read_log 192888'14962 (0'0) promote 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by osd.4.0:23278 2014-08-11 17:06:46.569583 2014-08-13 17:38:02.860509 7fe9bae887a0 20 read_log 192888'14963 (192888'14962) modify 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.942868.0:23961 2014-08-11 17:06:46.560526 2014-08-13 17:38:02.860521 7fe9bae887a0 20 read_log 192888'14968 (0'0) promote f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by osd.4.0:23372 2014-08-11 17:06:59.026632 2014-08-13 17:38:02.860533 7fe9bae887a0 20 read_log 192888'14969 (192888'14968) modify f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.942868.0:27218 2014-08-11 17:06:59.016692 2014-08-13 17:38:02.860554 7fe9bae887a0 20 read_log 192888'14970 (0'0) promote 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by osd.4.0:23380 2014-08-11 17:07:00.411505 2014-08-13 17:38:02.860567 7fe9bae887a0 20 read_log 192888'14971 (192888'14970) modify 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.942868.0:27596 2014-08-11 17:07:00.402458 2014-08-13 17:38:02.860579 7fe9bae887a0 20 read_log 192888'14972 (0'0) promote 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by osd.4.0:23416 2014-08-11 17:07:06.304442 2014-08-13 17:38:02.860591 7fe9bae887a0 20 read_log 192888'14973 (192888'14972) modify 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.942868.0:29146 2014-08-11 17:07:06.294439 2014-08-13 17:38:02.860604 7fe9bae887a0 20 read_log 192888'14978 (0'0) promote 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by osd.4.0:23452 2014-08-11 17:07:11.741174 2014-08-13 17:38:02.860616 7fe9bae887a0 20 read_log 192888'14979 (192888'14978) modify 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.942868.0:30478 2014-08-11 17:07:11.732510 2014-08-13 17:38:02.860628 7fe9bae887a0 20 read_log 192888'14982 (192888'14865) modify 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.942868.0:31467 2014-08-11 17:07:15.934797 2014-08-13 17:38:02.860640 7fe9bae887a0 20 read_log 192888'14985 (192888'14871) modify ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.942868.0:33977 2014-08-11 17:07:16.663792 2014-08-13 17:38:02.860652 7fe9bae887a0 20 read_log 192888'14987 (192888'14875) modify cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.942868.0:35190 2014-08-11 17:07:17.009973 2014-08-13 17:38:02.860664 7fe9bae887a0 20 read_log 192888'14988 (192888'14877) modify d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.942868.0:35417 2014-08-11 17:07:17.069364 2014-08-13 17:38:02.860676 7fe9bae887a0 20 read_log 192888'14989 (192888'14879) modify 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.942868.0:35621 2014-08-11 17:07:17.121603 2014-08-13 17:38:02.860688 7fe9bae887a0 20 read_log 192888'14991 (192888'14883) modify 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.942868.0:36668 2014-08-11 17:07:17.403369 2014-08-13 17:38:02.860700 7fe9bae887a0 20 read_log 192888'14995 (192888'14891) modify b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.942868.0:37894 2014-08-11 17:07:17.737166 2014-08-13 17:38:02.860712 7fe9bae887a0 20 read_log 192888'14999 (192888'14899) modify 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.942868.0:40413 2014-08-11 17:07:18.444316 2014-08-13 17:38:02.860724 7fe9bae887a0 20 read_log 192888'15000 (192888'14901) modify 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.942868.0:40596 2014-08-11 17:07:18.495439 2014-08-13 17:38:02.860736 7fe9bae887a0 20 read_log 192888'15001 (192888'14903) modify b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.942868.0:41623 2014-08-11 17:07:18.778204 2014-08-13 17:38:02.860748 7fe9bae887a0 20 read_log 192888'15004 (192888'14909) modify b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.942868.0:41846 2014-08-11 17:07:18.838175 2014-08-13 17:38:02.860760 7fe9bae887a0 20 read_log 192888'15006 (192888'14913) modify 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.942868.0:43987 2014-08-11 17:07:19.435520 2014-08-13 17:38:02.860772 7fe9bae887a0 20 read_log 192888'15008 (192888'14917) modify 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.942868.0:46616 2014-08-11 17:07:20.172366 2014-08-13 17:38:02.860784 7fe9bae887a0 20 read_log 192888'15011 (192888'14923) modify d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.942868.0:47664 2014-08-11 17:07:20.481595 2014-08-13 17:38:02.860804 7fe9bae887a0 20 read_log 192888'15012 (192888'14925) modify 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.942868.0:47927 2014-08-11 17:07:20.568369 2014-08-13 17:38:02.860818 7fe9bae887a0 20 read_log 192888'15013 (192888'14927) modify 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.942868.0:48264 2014-08-11 17:07:20.670579 2014-08-13 17:38:02.860830 7fe9bae887a0 20 read_log 192888'15014 (192888'14929) modify 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.942868.0:48334 2014-08-11 17:07:20.690068 2014-08-13 17:38:02.860842 7fe9bae887a0 20 read_log 192888'15015 (192888'14931) modify 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.942868.0:48856 2014-08-11 17:07:20.837791 2014-08-13 17:38:02.860855 7fe9bae887a0 20 read_log 192888'15016 (192888'14933) modify 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.942868.0:48974 2014-08-11 17:07:20.871996 2014-08-13 17:38:02.860871 7fe9bae887a0 20 read_log 192888'15017 (192888'14935) modify f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.942868.0:49815 2014-08-11 17:07:21.132037 2014-08-13 17:38:02.860884 7fe9bae887a0 20 read_log 192888'15018 (192888'14937) modify cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.942868.0:50554 2014-08-11 17:07:21.360446 2014-08-13 17:38:02.860896 7fe9bae887a0 20 read_log 192888'15020 (192888'14941) modify 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.942868.0:51193 2014-08-11 17:07:21.569287 2014-08-13 17:38:02.860908 7fe9bae887a0 20 read_log 192888'15022 (192888'14945) modify 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.942868.0:51964 2014-08-11 17:07:21.802840 2014-08-13 17:38:02.860920 7fe9bae887a0 20 read_log 192888'15025 (192888'14951) modify e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.942868.0:53181 2014-08-11 17:07:22.175450 2014-08-13 17:38:02.860932 7fe9bae887a0 20 read_log 192888'15026 (192888'14953) modify f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.942868.0:53782 2014-08-11 17:07:22.371808 2014-08-13 17:38:02.860944 7fe9bae887a0 20 read_log 192888'15028 (192888'14957) modify fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.942868.0:54031 2014-08-11 17:07:22.445687 2014-08-13 17:38:02.860957 7fe9bae887a0 20 read_log 192888'15031 (192888'14963) modify 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.942868.0:55227 2014-08-11 17:07:22.779573 2014-08-13 17:38:02.860970 7fe9bae887a0 20 read_log 192888'15034 (192888'14969) modify f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.942868.0:58484 2014-08-11 17:07:23.757261 2014-08-13 17:38:02.860982 7fe9bae887a0 20 read_log 192888'15035 (192888'14971) modify 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.942868.0:58862 2014-08-11 17:07:23.872381 2014-08-13 17:38:02.860994 7fe9bae887a0 20 read_log 192888'15036 (192888'14973) modify 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.942868.0:60412 2014-08-11 17:07:24.332756 2014-08-13 17:38:02.861006 7fe9bae887a0 20 read_log 192888'15039 (192888'14979) modify 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.942868.0:61744 2014-08-11 17:07:24.733870 2014-08-13 17:38:02.861018 7fe9bae887a0 20 read_log 193305'15041 (193305'15041) modify 136/hit_set_24.136_archive_2014-08-11 18:37:04.911155_2014-08-11 18:42:38.276778/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.276999 2014-08-13 17:38:02.861031 7fe9bae887a0 20 read_log 193305'15042 (192888'14862) delete 136/hit_set_24.136_archive_2014-08-11 16:44:58.718710_2014-08-11 17:05:15.819389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.276999 2014-08-13 17:38:02.861051 7fe9bae887a0 20 read_log 193305'15043 (0'0) promote 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by osd.4.0:18 2014-08-11 18:42:38.311783 2014-08-13 17:38:02.861064 7fe9bae887a0 20 read_log 193305'15044 (193305'15043) modify 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.951337.0:1360 2014-08-11 18:42:38.270697 2014-08-13 17:38:02.861076 7fe9bae887a0 20 read_log 193305'15045 (0'0) promote fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by osd.4.0:39 2014-08-11 18:42:42.796079 2014-08-13 17:38:02.861088 7fe9bae887a0 20 read_log 193305'15046 (193305'15045) modify fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.951337.0:2561 2014-08-11 18:42:42.788439 2014-08-13 17:38:02.861103 7fe9bae887a0 20 read_log 193305'15047 (0'0) promote 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by osd.4.0:49 2014-08-11 18:42:47.155282 2014-08-13 17:38:02.861115 7fe9bae887a0 20 read_log 193305'15048 (193305'15047) modify 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.951337.0:3762 2014-08-11 18:42:47.147819 2014-08-13 17:38:02.861134 7fe9bae887a0 20 read_log 193305'15049 (0'0) promote eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by osd.4.0:59 2014-08-11 18:42:49.602003 2014-08-13 17:38:02.861149 7fe9bae887a0 20 read_log 193305'15050 (193305'15049) modify eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.951337.0:4423 2014-08-11 18:42:49.594371 2014-08-13 17:38:02.861174 7fe9bae887a0 20 read_log 193305'15051 (0'0) promote 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by osd.4.0:77 2014-08-11 18:42:52.819201 2014-08-13 17:38:02.861186 7fe9bae887a0 20 read_log 193305'15052 (193305'15051) modify 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.951337.0:5281 2014-08-11 18:42:52.809676 2014-08-13 17:38:02.861198 7fe9bae887a0 20 read_log 193305'15053 (0'0) promote 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by osd.4.0:81 2014-08-11 18:42:53.477155 2014-08-13 17:38:02.861210 7fe9bae887a0 20 read_log 193305'15054 (193305'15053) modify 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.951337.0:5464 2014-08-11 18:42:53.469492 2014-08-13 17:38:02.861222 7fe9bae887a0 20 read_log 193305'15055 (0'0) promote cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by osd.4.0:83 2014-08-11 18:42:53.603123 2014-08-13 17:38:02.861234 7fe9bae887a0 20 read_log 193305'15056 (193305'15055) modify cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.951337.0:5499 2014-08-11 18:42:53.595935 2014-08-13 17:38:02.861247 7fe9bae887a0 20 read_log 193305'15057 (0'0) promote 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by osd.4.0:91 2014-08-11 18:42:55.500508 2014-08-13 17:38:02.861259 7fe9bae887a0 20 read_log 193305'15058 (193305'15057) modify 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.951337.0:6019 2014-08-11 18:42:55.492431 2014-08-13 17:38:02.861271 7fe9bae887a0 20 read_log 193305'15059 (0'0) promote fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by osd.4.0:95 2014-08-11 18:42:56.017071 2014-08-13 17:38:02.861283 7fe9bae887a0 20 read_log 193305'15060 (193305'15059) modify fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.951337.0:6162 2014-08-11 18:42:56.008809 2014-08-13 17:38:02.861295 7fe9bae887a0 20 read_log 193305'15061 (0'0) promote 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by osd.4.0:97 2014-08-11 18:42:56.074783 2014-08-13 17:38:02.861307 7fe9bae887a0 20 read_log 193305'15062 (193305'15061) modify 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.951337.0:6178 2014-08-11 18:42:56.067236 2014-08-13 17:38:02.861348 7fe9bae887a0 20 read_log 193305'15063 (0'0) promote 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by osd.4.0:105 2014-08-11 18:42:57.125679 2014-08-13 17:38:02.861365 7fe9bae887a0 20 read_log 193305'15064 (193305'15063) modify 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.951337.0:6468 2014-08-11 18:42:57.118956 2014-08-13 17:38:02.861377 7fe9bae887a0 20 read_log 193305'15065 (0'0) promote 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by osd.4.0:115 2014-08-11 18:42:59.604539 2014-08-13 17:38:02.861389 7fe9bae887a0 20 read_log 193305'15066 (193305'15065) modify 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.951337.0:7155 2014-08-11 18:42:59.596647 2014-08-13 17:38:02.861402 7fe9bae887a0 20 read_log 193305'15067 (0'0) promote fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by osd.4.0:121 2014-08-11 18:43:01.108747 2014-08-13 17:38:02.861414 7fe9bae887a0 20 read_log 193305'15068 (193305'15067) modify fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.951337.0:7570 2014-08-11 18:43:01.101756 2014-08-13 17:38:02.861426 7fe9bae887a0 20 read_log 193305'15069 (0'0) promote 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by osd.4.0:151 2014-08-11 18:43:07.278217 2014-08-13 17:38:02.861438 7fe9bae887a0 20 read_log 193305'15070 (193305'15069) modify 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.951337.0:9178 2014-08-11 18:43:07.270127 2014-08-13 17:38:02.861451 7fe9bae887a0 20 read_log 193305'15071 (0'0) promote 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by osd.4.0:177 2014-08-11 18:43:13.799041 2014-08-13 17:38:02.861463 7fe9bae887a0 20 read_log 193305'15072 (193305'15071) modify 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.951337.0:10944 2014-08-11 18:43:13.791219 2014-08-13 17:38:02.861474 7fe9bae887a0 20 read_log 193305'15073 (0'0) promote 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by osd.4.0:183 2014-08-11 18:43:14.942553 2014-08-13 17:38:02.861487 7fe9bae887a0 20 read_log 193305'15074 (193305'15073) modify 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.951337.0:11230 2014-08-11 18:43:14.935644 2014-08-13 17:38:02.861499 7fe9bae887a0 20 read_log 193305'15075 (0'0) promote e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by osd.4.0:191 2014-08-11 18:43:15.947021 2014-08-13 17:38:02.861511 7fe9bae887a0 20 read_log 193305'15076 (193305'15075) modify e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.951337.0:11472 2014-08-11 18:43:15.937013 2014-08-13 17:38:02.861523 7fe9bae887a0 20 read_log 193305'15077 (0'0) promote a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by osd.4.0:207 2014-08-11 18:43:19.365193 2014-08-13 17:38:02.861554 7fe9bae887a0 20 read_log 193305'15078 (193305'15077) modify a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.951337.0:12403 2014-08-11 18:43:19.357697 2014-08-13 17:38:02.861570 7fe9bae887a0 20 read_log 193305'15079 (0'0) promote 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by osd.4.0:209 2014-08-11 18:43:20.198221 2014-08-13 17:38:02.861583 7fe9bae887a0 20 read_log 193305'15080 (193305'15079) modify 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.951337.0:12627 2014-08-11 18:43:20.189502 2014-08-13 17:38:02.861595 7fe9bae887a0 20 read_log 193305'15081 (0'0) promote 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by osd.4.0:267 2014-08-11 18:43:33.691573 2014-08-13 17:38:02.861607 7fe9bae887a0 20 read_log 193305'15082 (193305'15081) modify 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.951337.0:16353 2014-08-11 18:43:33.683619 2014-08-13 17:38:02.861627 7fe9bae887a0 20 read_log 193305'15083 (0'0) promote b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by osd.4.0:269 2014-08-11 18:43:33.771749 2014-08-13 17:38:02.861642 7fe9bae887a0 20 read_log 193305'15084 (193305'15083) modify b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.951337.0:16377 2014-08-11 18:43:33.764292 2014-08-13 17:38:02.861654 7fe9bae887a0 20 read_log 193305'15085 (0'0) promote 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by osd.4.0:297 2014-08-11 18:43:38.341555 2014-08-13 17:38:02.861666 7fe9bae887a0 20 read_log 193305'15086 (193305'15085) modify 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.951337.0:17716 2014-08-11 18:43:38.333431 2014-08-13 17:38:02.861678 7fe9bae887a0 20 read_log 193305'15087 (0'0) promote 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by osd.4.0:299 2014-08-11 18:43:38.975639 2014-08-13 17:38:02.861690 7fe9bae887a0 20 read_log 193305'15088 (193305'15087) modify 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.951337.0:17888 2014-08-11 18:43:38.967446 2014-08-13 17:38:02.861702 7fe9bae887a0 20 read_log 193305'15089 (0'0) promote 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by osd.4.0:313 2014-08-11 18:43:41.408822 2014-08-13 17:38:02.861714 7fe9bae887a0 20 read_log 193305'15090 (193305'15089) modify 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.951337.0:18582 2014-08-11 18:43:41.400157 2014-08-13 17:38:02.861726 7fe9bae887a0 20 read_log 193305'15091 (0'0) promote 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by osd.4.0:317 2014-08-11 18:43:41.787535 2014-08-13 17:38:02.861738 7fe9bae887a0 20 read_log 193305'15092 (193305'15091) modify 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.951337.0:18693 2014-08-11 18:43:41.779202 2014-08-13 17:38:02.861750 7fe9bae887a0 20 read_log 193305'15093 (0'0) promote c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by osd.4.0:347 2014-08-11 18:43:49.073360 2014-08-13 17:38:02.861762 7fe9bae887a0 20 read_log 193305'15094 (193305'15093) modify c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.951337.0:20681 2014-08-11 18:43:49.064897 2014-08-13 17:38:02.861774 7fe9bae887a0 20 read_log 193305'15095 (0'0) promote d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by osd.4.0:349 2014-08-11 18:43:49.594168 2014-08-13 17:38:02.861786 7fe9bae887a0 20 read_log 193305'15096 (193305'15095) modify d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.951337.0:20830 2014-08-11 18:43:49.585499 2014-08-13 17:38:02.861798 7fe9bae887a0 20 read_log 193305'15097 (0'0) promote 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by osd.4.0:363 2014-08-11 18:43:53.641904 2014-08-13 17:38:02.861810 7fe9bae887a0 20 read_log 193305'15098 (193305'15097) modify 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.951337.0:21953 2014-08-11 18:43:53.633662 2014-08-13 17:38:02.861822 7fe9bae887a0 20 read_log 193305'15099 (0'0) promote a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by osd.4.0:387 2014-08-11 18:43:57.040427 2014-08-13 17:38:02.861834 7fe9bae887a0 20 read_log 193305'15100 (193305'15099) modify a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.951337.0:22883 2014-08-11 18:43:57.029244 2014-08-13 17:38:02.861846 7fe9bae887a0 20 read_log 193305'15101 (0'0) promote f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by osd.4.0:389 2014-08-11 18:43:57.157043 2014-08-13 17:38:02.861858 7fe9bae887a0 20 read_log 193305'15102 (193305'15101) modify f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.951337.0:22918 2014-08-11 18:43:57.149422 2014-08-13 17:38:02.861877 7fe9bae887a0 20 read_log 193305'15103 (0'0) promote 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by osd.4.0:393 2014-08-11 18:43:58.516762 2014-08-13 17:38:02.861890 7fe9bae887a0 20 read_log 193305'15104 (193305'15103) modify 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.951337.0:23308 2014-08-11 18:43:58.508196 2014-08-13 17:38:02.861904 7fe9bae887a0 20 read_log 193305'15105 (0'0) promote 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by osd.4.0:433 2014-08-11 18:44:10.243562 2014-08-13 17:38:02.861917 7fe9bae887a0 20 read_log 193305'15106 (193305'15105) modify 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.951337.0:26516 2014-08-11 18:44:10.237426 2014-08-13 17:38:02.861929 7fe9bae887a0 20 read_log 193305'15107 (0'0) promote b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by osd.4.0:467 2014-08-11 18:44:17.099629 2014-08-13 17:38:02.861941 7fe9bae887a0 20 read_log 193305'15108 (193305'15107) modify b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.951337.0:28403 2014-08-11 18:44:17.091711 2014-08-13 17:38:02.861953 7fe9bae887a0 20 read_log 193305'15109 (0'0) promote daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by osd.4.0:489 2014-08-11 18:44:21.790134 2014-08-13 17:38:02.861965 7fe9bae887a0 20 read_log 193305'15110 (193305'15109) modify daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.951337.0:29675 2014-08-11 18:44:21.782752 2014-08-13 17:38:02.861977 7fe9bae887a0 20 read_log 193305'15111 (0'0) promote cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by osd.4.0:505 2014-08-11 18:44:26.451931 2014-08-13 17:38:02.861989 7fe9bae887a0 20 read_log 193305'15112 (193305'15111) modify cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.951337.0:30892 2014-08-11 18:44:26.444605 2014-08-13 17:38:02.862001 7fe9bae887a0 20 read_log 193305'15113 (0'0) promote 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by osd.4.0:507 2014-08-11 18:44:26.883037 2014-08-13 17:38:02.862013 7fe9bae887a0 20 read_log 193305'15114 (193305'15113) modify 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.951337.0:31017 2014-08-11 18:44:26.874272 2014-08-13 17:38:02.862025 7fe9bae887a0 20 read_log 193305'15115 (0'0) promote 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by osd.4.0:527 2014-08-11 18:44:30.089672 2014-08-13 17:38:02.862037 7fe9bae887a0 20 read_log 193305'15116 (193305'15115) modify 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.951337.0:31946 2014-08-11 18:44:30.081335 2014-08-13 17:38:02.862049 7fe9bae887a0 20 read_log 193305'15117 (0'0) promote 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by osd.4.0:537 2014-08-11 18:44:30.738098 2014-08-13 17:38:02.862061 7fe9bae887a0 20 read_log 193305'15118 (193305'15117) modify 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.951337.0:32137 2014-08-11 18:44:30.731458 2014-08-13 17:38:02.862073 7fe9bae887a0 20 read_log 193305'15119 (0'0) promote 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by osd.4.0:545 2014-08-11 18:44:31.643084 2014-08-13 17:38:02.862085 7fe9bae887a0 20 read_log 193305'15120 (193305'15119) modify 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.951337.0:32370 2014-08-11 18:44:31.630100 2014-08-13 17:38:02.862098 7fe9bae887a0 20 read_log 193305'15121 (0'0) promote ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by osd.4.0:547 2014-08-11 18:44:31.954494 2014-08-13 17:38:02.862110 7fe9bae887a0 20 read_log 193305'15122 (193305'15121) modify ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.951337.0:32461 2014-08-11 18:44:31.946280 2014-08-13 17:38:02.862132 7fe9bae887a0 20 read_log 193305'15123 (193305'15044) modify 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.951337.0:34071 2014-08-11 18:44:34.783420 2014-08-13 17:38:02.862145 7fe9bae887a0 20 read_log 193305'15124 (193305'15046) modify fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.951337.0:35272 2014-08-11 18:44:35.486049 2014-08-13 17:38:02.862161 7fe9bae887a0 20 read_log 193305'15125 (193305'15048) modify 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.951337.0:36473 2014-08-11 18:44:36.246961 2014-08-13 17:38:02.862174 7fe9bae887a0 20 read_log 193305'15126 (193305'15050) modify eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.951337.0:37134 2014-08-11 18:44:36.727062 2014-08-13 17:38:02.862187 7fe9bae887a0 20 read_log 193305'15127 (193305'15052) modify 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.951337.0:37992 2014-08-11 18:44:37.186043 2014-08-13 17:38:02.862199 7fe9bae887a0 20 read_log 193305'15128 (193305'15054) modify 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.951337.0:38175 2014-08-11 18:44:37.274138 2014-08-13 17:38:02.862211 7fe9bae887a0 20 read_log 193305'15129 (193305'15056) modify cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.951337.0:38210 2014-08-11 18:44:37.288348 2014-08-13 17:38:02.862223 7fe9bae887a0 20 read_log 193305'15130 (193305'15058) modify 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.951337.0:38730 2014-08-11 18:44:37.623799 2014-08-13 17:38:02.862235 7fe9bae887a0 20 read_log 193305'15131 (193305'15060) modify fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.951337.0:38873 2014-08-11 18:44:37.705993 2014-08-13 17:38:02.862247 7fe9bae887a0 20 read_log 193305'15132 (193305'15062) modify 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.951337.0:38889 2014-08-11 18:44:37.716624 2014-08-13 17:38:02.862259 7fe9bae887a0 20 read_log 193305'15133 (193305'15064) modify 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.951337.0:39179 2014-08-11 18:44:37.867600 2014-08-13 17:38:02.862270 7fe9bae887a0 20 read_log 193305'15134 (193305'15066) modify 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.951337.0:39866 2014-08-11 18:44:38.209860 2014-08-13 17:38:02.862282 7fe9bae887a0 20 read_log 193305'15135 (193305'15068) modify fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.951337.0:40281 2014-08-11 18:44:38.361379 2014-08-13 17:38:02.862294 7fe9bae887a0 20 read_log 193305'15136 (193305'15070) modify 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.951337.0:41889 2014-08-11 18:44:38.911536 2014-08-13 17:38:02.862307 7fe9bae887a0 20 read_log 193305'15137 (193305'15072) modify 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.951337.0:43655 2014-08-11 18:44:39.656646 2014-08-13 17:38:02.862318 7fe9bae887a0 20 read_log 193305'15138 (193305'15074) modify 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.951337.0:43941 2014-08-11 18:44:39.743464 2014-08-13 17:38:02.862332 7fe9bae887a0 20 read_log 193305'15139 (193305'15076) modify e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.951337.0:44183 2014-08-11 18:44:39.815446 2014-08-13 17:38:02.862344 7fe9bae887a0 20 read_log 193305'15140 (193305'15078) modify a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.951337.0:45114 2014-08-11 18:44:40.093470 2014-08-13 17:38:02.862356 7fe9bae887a0 20 read_log 193305'15141 (193305'15080) modify 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.951337.0:45338 2014-08-11 18:44:40.166973 2014-08-13 17:38:02.862368 7fe9bae887a0 20 read_log 193305'15142 (193305'15082) modify 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.951337.0:49064 2014-08-11 18:44:41.239633 2014-08-13 17:38:02.862387 7fe9bae887a0 20 read_log 193305'15143 (193305'15084) modify b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.951337.0:49088 2014-08-11 18:44:41.251711 2014-08-13 17:38:02.862399 7fe9bae887a0 20 read_log 193305'15144 (193305'15086) modify 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.951337.0:50427 2014-08-11 18:44:41.661417 2014-08-13 17:38:02.862411 7fe9bae887a0 20 read_log 193305'15145 (193305'15088) modify 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.951337.0:50599 2014-08-11 18:44:41.711520 2014-08-13 17:38:02.862423 7fe9bae887a0 20 read_log 193305'15146 (193305'15090) modify 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.951337.0:51293 2014-08-11 18:44:41.907724 2014-08-13 17:38:02.862435 7fe9bae887a0 20 read_log 193305'15147 (193305'15092) modify 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.951337.0:51404 2014-08-11 18:44:41.941828 2014-08-13 17:38:02.862447 7fe9bae887a0 20 read_log 193305'15148 (193305'15094) modify c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.951337.0:53392 2014-08-11 18:44:42.517377 2014-08-13 17:38:02.862459 7fe9bae887a0 20 read_log 193305'15149 (193305'15096) modify d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.951337.0:53541 2014-08-11 18:44:42.559971 2014-08-13 17:38:02.862471 7fe9bae887a0 20 read_log 193305'15150 (193305'15098) modify 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.951337.0:54664 2014-08-11 18:44:42.905459 2014-08-13 17:38:02.862483 7fe9bae887a0 20 read_log 193305'15151 (193305'15100) modify a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.951337.0:55594 2014-08-11 18:44:43.172363 2014-08-13 17:38:02.862494 7fe9bae887a0 20 read_log 193305'15152 (193305'15102) modify f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.951337.0:55629 2014-08-11 18:44:43.183070 2014-08-13 17:38:02.862506 7fe9bae887a0 20 read_log 193305'15153 (193305'15104) modify 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.951337.0:56019 2014-08-11 18:44:43.295230 2014-08-13 17:38:02.862518 7fe9bae887a0 20 read_log 193305'15154 (193305'15106) modify 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.951337.0:59227 2014-08-11 18:44:44.267428 2014-08-13 17:38:02.862529 7fe9bae887a0 20 read_log 193305'15155 (193305'15108) modify b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.951337.0:61114 2014-08-11 18:44:44.827932 2014-08-13 17:38:02.862541 7fe9bae887a0 20 read_log 193305'15156 (193305'15110) modify daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.951337.0:62386 2014-08-11 18:44:45.212195 2014-08-13 17:38:02.862553 7fe9bae887a0 20 read_log 193305'15157 (193305'15112) modify cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.951337.0:63603 2014-08-11 18:44:45.557607 2014-08-13 17:38:02.862564 7fe9bae887a0 20 read_log 193305'15158 (193305'15114) modify 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.951337.0:63728 2014-08-11 18:44:45.593564 2014-08-13 17:38:02.862576 7fe9bae887a0 20 read_log 193305'15159 (193305'15116) modify 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.951337.0:64657 2014-08-11 18:44:45.881088 2014-08-13 17:38:02.862588 7fe9bae887a0 20 read_log 193305'15160 (193305'15118) modify 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.951337.0:64848 2014-08-11 18:44:45.937664 2014-08-13 17:38:02.862600 7fe9bae887a0 20 read_log 193305'15161 (193305'15120) modify 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.951337.0:65081 2014-08-11 18:44:46.011027 2014-08-13 17:38:02.862612 7fe9bae887a0 20 read_log 193305'15162 (193305'15122) modify ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.951337.0:65172 2014-08-11 18:44:46.041880 2014-08-13 17:38:02.862631 7fe9bae887a0 20 read_log 194215'15163 (194215'15163) modify 136/hit_set_24.136_archive_2014-08-12 15:32:25.865410_2014-08-12 15:40:54.805843/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:40:54.806500 2014-08-13 17:38:02.862644 7fe9bae887a0 20 read_log 194215'15164 (193305'15041) delete 136/hit_set_24.136_archive_2014-08-11 18:37:04.911155_2014-08-11 18:42:38.276778/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:40:54.806500 2014-08-13 17:38:02.862656 7fe9bae887a0 20 read_log 194215'15165 (192888'15020) clean 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by osd.4.0:1578 2014-08-12 15:40:54.812453 2014-08-13 17:38:02.862669 7fe9bae887a0 20 read_log 194215'15166 (194215'15165) delete 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.2174595.0:63211 0.000000 2014-08-13 17:38:02.862679 7fe9bae887a0 20 read_log 194215'15167 (193305'15158) clean 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by osd.4.0:1580 2014-08-12 15:40:54.860249 2014-08-13 17:38:02.862692 7fe9bae887a0 20 read_log 194215'15168 (194215'15167) delete 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.2174595.0:63213 0.000000 2014-08-13 17:38:02.862702 7fe9bae887a0 20 read_log 194215'15169 (192888'14814) clean d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by osd.4.0:1582 2014-08-12 15:40:54.868405 2014-08-13 17:38:02.862714 7fe9bae887a0 20 read_log 194215'15170 (194215'15169) delete d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.2174595.0:63215 0.000000 2014-08-13 17:38:02.862742 7fe9bae887a0 20 read_log 194215'15171 (193305'15143) clean b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by osd.4.0:1584 2014-08-12 15:40:54.875727 2014-08-13 17:38:02.862755 7fe9bae887a0 20 read_log 194215'15172 (194215'15171) delete b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.2174595.0:63217 0.000000 2014-08-13 17:38:02.862766 7fe9bae887a0 20 read_log 194215'15173 (193305'15156) clean daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by osd.4.0:1586 2014-08-12 15:40:54.915635 2014-08-13 17:38:02.862778 7fe9bae887a0 20 read_log 194215'15174 (194215'15173) delete daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.2174595.0:63219 0.000000 2014-08-13 17:38:02.862789 7fe9bae887a0 20 read_log 194215'15175 (192888'14818) clean 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by osd.4.0:1588 2014-08-12 15:40:54.924333 2014-08-13 17:38:02.862801 7fe9bae887a0 20 read_log 194215'15176 (194215'15175) delete 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.2174595.0:63221 0.000000 2014-08-13 17:38:02.862811 7fe9bae887a0 20 read_log 194215'15177 (192888'14999) clean 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by osd.4.0:1590 2014-08-12 15:40:54.933248 2014-08-13 17:38:02.862823 7fe9bae887a0 20 read_log 194215'15178 (194215'15177) delete 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.2174595.0:63223 0.000000 2014-08-13 17:38:02.862833 7fe9bae887a0 20 read_log 194215'15179 (192888'14832) clean 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by osd.4.0:1592 2014-08-12 15:40:54.941943 2014-08-13 17:38:02.862845 7fe9bae887a0 20 read_log 194215'15180 (194215'15179) delete 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.2174595.0:63225 0.000000 2014-08-13 17:38:02.862858 7fe9bae887a0 20 read_log 194215'15181 (193305'15135) clean fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by osd.4.0:1594 2014-08-12 15:40:54.950533 2014-08-13 17:38:02.862880 7fe9bae887a0 20 read_log 194215'15182 (194215'15181) delete fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.2174595.0:63227 0.000000 2014-08-13 17:38:02.862892 7fe9bae887a0 20 read_log 194215'15183 (193305'15141) clean 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by osd.4.0:1596 2014-08-12 15:40:54.959264 2014-08-13 17:38:02.862904 7fe9bae887a0 20 read_log 194215'15184 (194215'15183) delete 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.2174595.0:63229 0.000000 2014-08-13 17:38:02.862915 7fe9bae887a0 20 read_log 194215'15185 (193305'15138) clean 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by osd.4.0:1598 2014-08-12 15:40:54.967567 2014-08-13 17:38:02.862927 7fe9bae887a0 20 read_log 194215'15186 (194215'15185) delete 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.2174595.0:63231 0.000000 2014-08-13 17:38:02.862938 7fe9bae887a0 20 read_log 194215'15187 (193305'15159) clean 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by osd.4.0:1600 2014-08-12 15:40:54.993535 2014-08-13 17:38:02.862950 7fe9bae887a0 20 read_log 194215'15188 (194215'15187) delete 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.2174595.0:63233 0.000000 2014-08-13 17:38:02.862960 7fe9bae887a0 20 read_log 194215'15189 (193305'15154) clean 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by osd.4.0:1602 2014-08-12 15:40:55.032202 2014-08-13 17:38:02.862973 7fe9bae887a0 20 read_log 194215'15190 (194215'15189) delete 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.2174595.0:63235 0.000000 2014-08-13 17:38:02.862983 7fe9bae887a0 20 read_log 194215'15191 (192888'14843) clean 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by osd.4.0:1604 2014-08-12 15:40:55.041324 2014-08-13 17:38:02.862995 7fe9bae887a0 20 read_log 194215'15192 (194215'15191) delete 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.2174595.0:63237 0.000000 2014-08-13 17:38:02.863006 7fe9bae887a0 20 read_log 194215'15193 (192888'15035) clean 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by osd.4.0:1606 2014-08-12 15:40:55.049360 2014-08-13 17:38:02.863018 7fe9bae887a0 20 read_log 194215'15194 (194215'15193) delete 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.2174595.0:63239 0.000000 2014-08-13 17:38:02.863028 7fe9bae887a0 20 read_log 194215'15195 (192888'15013) clean 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by osd.4.0:1608 2014-08-12 15:40:55.057066 2014-08-13 17:38:02.863040 7fe9bae887a0 20 read_log 194215'15196 (194215'15195) delete 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.2174595.0:63241 0.000000 2014-08-13 17:38:02.863051 7fe9bae887a0 20 read_log 194215'15197 (193305'15144) clean 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by osd.4.0:1610 2014-08-12 15:40:55.065117 2014-08-13 17:38:02.863063 7fe9bae887a0 20 read_log 194215'15198 (194215'15197) delete 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.2174595.0:63243 0.000000 2014-08-13 17:38:02.863073 7fe9bae887a0 20 read_log 194215'15199 (192888'15001) clean b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by osd.4.0:1612 2014-08-12 15:40:55.074588 2014-08-13 17:38:02.863085 7fe9bae887a0 20 read_log 194215'15200 (194215'15199) delete b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.2174595.0:63245 0.000000 2014-08-13 17:38:02.863096 7fe9bae887a0 20 read_log 194215'15201 (193305'15152) clean f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by osd.4.0:1614 2014-08-12 15:40:55.106909 2014-08-13 17:38:02.863130 7fe9bae887a0 20 read_log 194215'15202 (194215'15201) delete f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.2174595.0:63247 0.000000 2014-08-13 17:38:02.863142 7fe9bae887a0 20 read_log 194215'15203 (192888'14860) clean 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by osd.4.0:1616 2014-08-12 15:40:55.114586 2014-08-13 17:38:02.863162 7fe9bae887a0 20 read_log 194215'15204 (194215'15203) delete 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.2174595.0:63249 0.000000 2014-08-13 17:38:02.863173 7fe9bae887a0 20 read_log 194215'15205 (193305'15130) clean 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by osd.4.0:1618 2014-08-12 15:40:55.169581 2014-08-13 17:38:02.863185 7fe9bae887a0 20 read_log 194215'15206 (194215'15205) delete 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.2174595.0:63251 0.000000 2014-08-13 17:38:02.863196 7fe9bae887a0 20 read_log 194215'15207 (193305'15160) clean 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by osd.4.0:1620 2014-08-12 15:40:55.178388 2014-08-13 17:38:02.863208 7fe9bae887a0 20 read_log 194215'15208 (194215'15207) delete 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.2174595.0:63253 0.000000 2014-08-13 17:38:02.863218 7fe9bae887a0 20 read_log 194215'15209 (193305'15124) clean fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by osd.4.0:1622 2014-08-12 15:40:55.226444 2014-08-13 17:38:02.863230 7fe9bae887a0 20 read_log 194215'15210 (194215'15209) delete fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.2174595.0:63255 0.000000 2014-08-13 17:38:02.863241 7fe9bae887a0 20 read_log 194215'15211 (192888'14831) clean 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by osd.4.0:1624 2014-08-12 15:40:55.235026 2014-08-13 17:38:02.863253 7fe9bae887a0 20 read_log 194215'15212 (194215'15211) delete 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.2174595.0:63257 0.000000 2014-08-13 17:38:02.863264 7fe9bae887a0 20 read_log 194215'15213 (192888'15016) clean 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by osd.4.0:1626 2014-08-12 15:40:55.241989 2014-08-13 17:38:02.863276 7fe9bae887a0 20 read_log 194215'15214 (194215'15213) delete 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.2174595.0:63259 0.000000 2014-08-13 17:38:02.863287 7fe9bae887a0 20 read_log 194215'15215 (193305'15127) clean 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by osd.4.0:1628 2014-08-12 15:40:55.249841 2014-08-13 17:38:02.863300 7fe9bae887a0 20 read_log 194215'15216 (194215'15215) delete 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.2174595.0:63261 0.000000 2014-08-13 17:38:02.863311 7fe9bae887a0 20 read_log 194215'15217 (193305'15157) clean cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by osd.4.0:1630 2014-08-12 15:40:55.257513 2014-08-13 17:38:02.863323 7fe9bae887a0 20 read_log 194215'15218 (194215'15217) delete cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.2174595.0:63263 0.000000 2014-08-13 17:38:02.863334 7fe9bae887a0 20 read_log 194215'15219 (192888'14842) clean f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by osd.4.0:1632 2014-08-12 15:40:55.264721 2014-08-13 17:38:02.863346 7fe9bae887a0 20 read_log 194215'15220 (194215'15219) delete f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.2174595.0:63265 0.000000 2014-08-13 17:38:02.863356 7fe9bae887a0 20 read_log 194215'15221 (193305'15145) clean 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by osd.4.0:1634 2014-08-12 15:40:55.272342 2014-08-13 17:38:02.863375 7fe9bae887a0 20 read_log 194215'15222 (194215'15221) delete 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.2174595.0:63267 0.000000 2014-08-13 17:38:02.863387 7fe9bae887a0 20 read_log 194215'15223 (192888'14815) clean 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by osd.4.0:1636 2014-08-12 15:40:55.279940 2014-08-13 17:38:02.863399 7fe9bae887a0 20 read_log 194215'15224 (194215'15223) delete 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.2174595.0:63269 0.000000 2014-08-13 17:38:02.863410 7fe9bae887a0 20 read_log 194215'15225 (192888'15017) clean f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by osd.4.0:1638 2014-08-12 15:40:55.288071 2014-08-13 17:38:02.863422 7fe9bae887a0 20 read_log 194215'15226 (194215'15225) delete f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.2174595.0:63271 0.000000 2014-08-13 17:38:02.863432 7fe9bae887a0 20 read_log 194215'15227 (193305'15132) clean 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by osd.4.0:1640 2014-08-12 15:40:55.296326 2014-08-13 17:38:02.863444 7fe9bae887a0 20 read_log 194215'15228 (194215'15227) delete 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.2174595.0:63273 0.000000 2014-08-13 17:38:02.863455 7fe9bae887a0 20 read_log 194215'15229 (192888'15039) clean 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by osd.4.0:1642 2014-08-12 15:40:55.303906 2014-08-13 17:38:02.863467 7fe9bae887a0 20 read_log 194215'15230 (194215'15229) delete 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.2174595.0:63275 0.000000 2014-08-13 17:38:02.863478 7fe9bae887a0 20 read_log 194215'15231 (192888'14827) clean bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by osd.4.0:1644 2014-08-12 15:40:55.312714 2014-08-13 17:38:02.863493 7fe9bae887a0 20 read_log 194215'15232 (194215'15231) delete bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.2174595.0:63277 0.000000 2014-08-13 17:38:02.863504 7fe9bae887a0 20 read_log 194215'15233 (192888'14840) clean f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by osd.4.0:1646 2014-08-12 15:40:55.320697 2014-08-13 17:38:02.863516 7fe9bae887a0 20 read_log 194215'15234 (194215'15233) delete f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.2174595.0:63279 0.000000 2014-08-13 17:38:02.863526 7fe9bae887a0 20 read_log 194215'15235 (192888'14836) clean de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by osd.4.0:1648 2014-08-12 15:40:55.330822 2014-08-13 17:38:02.863538 7fe9bae887a0 20 read_log 194215'15236 (194215'15235) delete de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.2174595.0:63281 0.000000 2014-08-13 17:38:02.863549 7fe9bae887a0 20 read_log 194215'15237 (193305'15162) clean ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by osd.4.0:1650 2014-08-12 15:40:55.340018 2014-08-13 17:38:02.863561 7fe9bae887a0 20 read_log 194215'15238 (194215'15237) delete ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.2174595.0:63283 0.000000 2014-08-13 17:38:02.863571 7fe9bae887a0 20 read_log 194215'15239 (193305'15123) clean 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by osd.4.0:1652 2014-08-12 15:40:55.348547 2014-08-13 17:38:02.863583 7fe9bae887a0 20 read_log 194215'15240 (194215'15239) delete 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.2174595.0:63285 0.000000 2014-08-13 17:38:02.863594 7fe9bae887a0 20 read_log 194215'15241 (193305'15131) clean fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by osd.4.0:1654 2014-08-12 15:40:55.357787 2014-08-13 17:38:02.863615 7fe9bae887a0 20 read_log 194215'15242 (194215'15241) delete fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.2174595.0:63287 0.000000 2014-08-13 17:38:02.863626 7fe9bae887a0 20 read_log 194215'15243 (192888'14817) clean 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by osd.4.0:1656 2014-08-12 15:40:55.366050 2014-08-13 17:38:02.863639 7fe9bae887a0 20 read_log 194215'15244 (194215'15243) delete 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.2174595.0:63289 0.000000 2014-08-13 17:38:02.863650 7fe9bae887a0 20 read_log 194215'15245 (192888'15022) clean 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by osd.4.0:1658 2014-08-12 15:40:55.407521 2014-08-13 17:38:02.863662 7fe9bae887a0 20 read_log 194215'15246 (194215'15245) delete 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.2174595.0:63291 0.000000 2014-08-13 17:38:02.863672 7fe9bae887a0 20 read_log 194215'15247 (192888'14995) clean b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by osd.4.0:1660 2014-08-12 15:40:55.421818 2014-08-13 17:38:02.863684 7fe9bae887a0 20 read_log 194215'15248 (194215'15247) delete b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.2174595.0:63293 0.000000 2014-08-13 17:38:02.863695 7fe9bae887a0 20 read_log 194215'15249 (192888'15014) clean 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by osd.4.0:1662 2014-08-12 15:40:55.429479 2014-08-13 17:38:02.863707 7fe9bae887a0 20 read_log 194215'15250 (194215'15249) delete 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.2174595.0:63295 0.000000 2014-08-13 17:38:02.863717 7fe9bae887a0 20 read_log 194215'15251 (193305'15126) clean eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by osd.4.0:1664 2014-08-12 15:40:55.436833 2014-08-13 17:38:02.863729 7fe9bae887a0 20 read_log 194215'15252 (194215'15251) delete eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.2174595.0:63297 0.000000 2014-08-13 17:38:02.863739 7fe9bae887a0 20 read_log 194215'15253 (192888'14805) clean d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by osd.4.0:1666 2014-08-12 15:40:55.444535 2014-08-13 17:38:02.863751 7fe9bae887a0 20 read_log 194215'15254 (194215'15253) delete d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.2174595.0:63299 0.000000 2014-08-13 17:38:02.863762 7fe9bae887a0 20 read_log 194215'15255 (192888'15036) clean 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by osd.4.0:1668 2014-08-12 15:40:55.451769 2014-08-13 17:38:02.863774 7fe9bae887a0 20 read_log 194215'15256 (194215'15255) delete 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.2174595.0:63301 0.000000 2014-08-13 17:38:02.863785 7fe9bae887a0 20 read_log 194215'15257 (193305'15134) clean 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by osd.4.0:1670 2014-08-12 15:40:55.459166 2014-08-13 17:38:02.863798 7fe9bae887a0 20 read_log 194215'15258 (194215'15257) delete 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.2174595.0:63303 0.000000 2014-08-13 17:38:02.863809 7fe9bae887a0 20 read_log 194215'15259 (192888'15034) clean f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by osd.4.0:1672 2014-08-12 15:40:55.466283 2014-08-13 17:38:02.863822 7fe9bae887a0 20 read_log 194215'15260 (194215'15259) delete f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.2174595.0:63305 0.000000 2014-08-13 17:38:02.863833 7fe9bae887a0 20 read_log 194215'15261 (192888'14982) clean 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by osd.4.0:1674 2014-08-12 15:40:55.473935 2014-08-13 17:38:02.863864 7fe9bae887a0 20 read_log 194215'15262 (194215'15261) delete 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.2174595.0:63307 0.000000 2014-08-13 17:38:02.863876 7fe9bae887a0 20 read_log 194215'15263 (193305'15142) clean 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by osd.4.0:1676 2014-08-12 15:40:55.482340 2014-08-13 17:38:02.863890 7fe9bae887a0 20 read_log 194215'15264 (194215'15263) delete 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.2174595.0:63309 0.000000 2014-08-13 17:38:02.863900 7fe9bae887a0 20 read_log 194215'15265 (192888'15028) clean fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by osd.4.0:1678 2014-08-12 15:40:55.497953 2014-08-13 17:38:02.863912 7fe9bae887a0 20 read_log 194215'15266 (194215'15265) delete fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.2174595.0:63311 0.000000 2014-08-13 17:38:02.863923 7fe9bae887a0 20 read_log 194215'15267 (192888'15031) clean 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by osd.4.0:1680 2014-08-12 15:40:55.506684 2014-08-13 17:38:02.863935 7fe9bae887a0 20 read_log 194215'15268 (194215'15267) delete 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.2174595.0:63313 0.000000 2014-08-13 17:38:02.863946 7fe9bae887a0 20 read_log 194215'15269 (193305'15149) clean d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by osd.4.0:1682 2014-08-12 15:40:55.523238 2014-08-13 17:38:02.863958 7fe9bae887a0 20 read_log 194215'15270 (194215'15269) delete d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.2174595.0:63315 0.000000 2014-08-13 17:38:02.863968 7fe9bae887a0 20 read_log 194215'15271 (193305'15133) clean 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by osd.4.0:1684 2014-08-12 15:40:55.561811 2014-08-13 17:38:02.863980 7fe9bae887a0 20 read_log 194215'15272 (194215'15271) delete 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.2174595.0:63317 0.000000 2014-08-13 17:38:02.863991 7fe9bae887a0 20 read_log 194215'15273 (192888'14988) clean d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by osd.4.0:1686 2014-08-12 15:40:55.571371 2014-08-13 17:38:02.864003 7fe9bae887a0 20 read_log 194215'15274 (194215'15273) delete d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.2174595.0:63319 0.000000 2014-08-13 17:38:02.864014 7fe9bae887a0 20 read_log 194215'15275 (192888'14822) clean fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by osd.4.0:1688 2014-08-12 15:40:55.601344 2014-08-13 17:38:02.864034 7fe9bae887a0 20 read_log 194215'15276 (194215'15275) delete fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.2174595.0:63321 0.000000 2014-08-13 17:38:02.864045 7fe9bae887a0 20 read_log 194215'15277 (192888'14987) clean cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by osd.4.0:1690 2014-08-12 15:40:55.610900 2014-08-13 17:38:02.864077 7fe9bae887a0 20 read_log 194215'15278 (194215'15277) delete cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.2174595.0:63323 0.000000 2014-08-13 17:38:02.864089 7fe9bae887a0 20 read_log 194215'15279 (193305'15148) clean c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by osd.4.0:1692 2014-08-12 15:40:55.634120 2014-08-13 17:38:02.864101 7fe9bae887a0 20 read_log 194215'15280 (194215'15279) delete c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.2174595.0:63325 0.000000 2014-08-13 17:38:02.864112 7fe9bae887a0 20 read_log 194215'15281 (192888'14991) clean 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by osd.4.0:1694 2014-08-12 15:40:55.642346 2014-08-13 17:38:02.864131 7fe9bae887a0 20 read_log 194215'15282 (194215'15281) delete 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.2174595.0:63327 0.000000 2014-08-13 17:38:02.864142 7fe9bae887a0 20 read_log 194215'15283 (192888'15015) clean 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by osd.4.0:1696 2014-08-12 15:40:55.651416 2014-08-13 17:38:02.864161 7fe9bae887a0 20 read_log 194215'15284 (194215'15283) delete 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.2174595.0:63329 0.000000 2014-08-13 17:38:02.864172 7fe9bae887a0 20 read_log 194215'15285 (193305'15153) clean 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by osd.4.0:1698 2014-08-12 15:40:55.661040 2014-08-13 17:38:02.864185 7fe9bae887a0 20 read_log 194215'15286 (194215'15285) delete 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.2174595.0:63331 0.000000 2014-08-13 17:38:02.864195 7fe9bae887a0 20 read_log 194215'15287 (193305'15139) clean e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by osd.4.0:1700 2014-08-12 15:40:55.669062 2014-08-13 17:38:02.864207 7fe9bae887a0 20 read_log 194215'15288 (194215'15287) delete e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.2174595.0:63333 0.000000 2014-08-13 17:38:02.864218 7fe9bae887a0 20 read_log 194215'15289 (192888'15008) clean 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by osd.4.0:1702 2014-08-12 15:40:55.677340 2014-08-13 17:38:02.864230 7fe9bae887a0 20 read_log 194215'15290 (194215'15289) delete 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.2174595.0:63335 0.000000 2014-08-13 17:38:02.864240 7fe9bae887a0 20 read_log 194215'15291 (192888'14821) clean f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by osd.4.0:1704 2014-08-12 15:40:55.686000 2014-08-13 17:38:02.864252 7fe9bae887a0 20 read_log 194215'15292 (194215'15291) delete f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.2174595.0:63337 0.000000 2014-08-13 17:38:02.864263 7fe9bae887a0 20 read_log 194215'15293 (192888'15006) clean 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by osd.4.0:1706 2014-08-12 15:40:55.695103 2014-08-13 17:38:02.864275 7fe9bae887a0 20 read_log 194215'15294 (194215'15293) delete 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.2174595.0:63339 0.000000 2014-08-13 17:38:02.864285 7fe9bae887a0 20 read_log 194215'15295 (193305'15125) clean 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by osd.4.0:1708 2014-08-12 15:40:55.704519 2014-08-13 17:38:02.864297 7fe9bae887a0 20 read_log 194215'15296 (194215'15295) delete 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.2174595.0:63341 0.000000 2014-08-13 17:38:02.864308 7fe9bae887a0 20 read_log 194215'15297 (193305'15136) clean 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by osd.4.0:1710 2014-08-12 15:40:55.713219 2014-08-13 17:38:02.864319 7fe9bae887a0 20 read_log 194215'15298 (194215'15297) delete 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.2174595.0:63343 0.000000 2014-08-13 17:38:02.864330 7fe9bae887a0 20 read_log 194215'15299 (193305'15146) clean 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by osd.4.0:1712 2014-08-12 15:40:55.721495 2014-08-13 17:38:02.864342 7fe9bae887a0 20 read_log 194215'15300 (194215'15299) delete 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.2174595.0:63345 0.000000 2014-08-13 17:38:02.864352 7fe9bae887a0 20 read_log 194215'15301 (192888'14855) clean 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by osd.4.0:1714 2014-08-12 15:40:55.729061 2014-08-13 17:38:02.864373 7fe9bae887a0 20 read_log 194215'15302 (194215'15301) delete 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.2174595.0:63347 0.000000 2014-08-13 17:38:02.864385 7fe9bae887a0 20 read_log 194215'15303 (192888'14845) clean 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by osd.4.0:1716 2014-08-12 15:40:55.735852 2014-08-13 17:38:02.864397 7fe9bae887a0 20 read_log 194215'15304 (194215'15303) delete 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.2174595.0:63349 0.000000 2014-08-13 17:38:02.864408 7fe9bae887a0 20 read_log 194215'15305 (192888'15004) clean b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by osd.4.0:1718 2014-08-12 15:40:55.744503 2014-08-13 17:38:02.864420 7fe9bae887a0 20 read_log 194215'15306 (194215'15305) delete b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.2174595.0:63351 0.000000 2014-08-13 17:38:02.864430 7fe9bae887a0 20 read_log 194215'15307 (193305'15129) clean cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by osd.4.0:1720 2014-08-12 15:40:55.752179 2014-08-13 17:38:02.864442 7fe9bae887a0 20 read_log 194215'15308 (194215'15307) delete cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.2174595.0:63353 0.000000 2014-08-13 17:38:02.864453 7fe9bae887a0 20 read_log 194215'15309 (192888'15011) clean d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by osd.4.0:1722 2014-08-12 15:40:55.760216 2014-08-13 17:38:02.864465 7fe9bae887a0 20 read_log 194215'15310 (194215'15309) delete d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.2174595.0:63355 0.000000 2014-08-13 17:38:02.864476 7fe9bae887a0 20 read_log 194215'15311 (193305'15161) clean 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by osd.4.0:1724 2014-08-12 15:40:55.767523 2014-08-13 17:38:02.864488 7fe9bae887a0 20 read_log 194215'15312 (194215'15311) delete 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.2174595.0:63357 0.000000 2014-08-13 17:38:02.864498 7fe9bae887a0 20 read_log 194215'15313 (192888'14809) clean 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by osd.4.0:1726 2014-08-12 15:40:55.792504 2014-08-13 17:38:02.864510 7fe9bae887a0 20 read_log 194215'15314 (194215'15313) delete 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.2174595.0:63359 0.000000 2014-08-13 17:38:02.864520 7fe9bae887a0 20 read_log 194215'15315 (192888'15025) clean e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by osd.4.0:1728 2014-08-12 15:40:55.801252 2014-08-13 17:38:02.864532 7fe9bae887a0 20 read_log 194215'15316 (194215'15315) delete e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.2174595.0:63361 0.000000 2014-08-13 17:38:02.864543 7fe9bae887a0 20 read_log 194215'15317 (193305'15147) clean 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by osd.4.0:1730 2014-08-12 15:40:55.809369 2014-08-13 17:38:02.864555 7fe9bae887a0 20 read_log 194215'15318 (194215'15317) delete 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.2174595.0:63363 0.000000 2014-08-13 17:38:02.864565 7fe9bae887a0 20 read_log 194215'15319 (192888'15012) clean 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by osd.4.0:1732 2014-08-12 15:40:55.818123 2014-08-13 17:38:02.864577 7fe9bae887a0 20 read_log 194215'15320 (194215'15319) delete 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.2174595.0:63365 0.000000 2014-08-13 17:38:02.864588 7fe9bae887a0 20 read_log 194215'15321 (192888'15026) clean f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by osd.4.0:1734 2014-08-12 15:40:55.826204 2014-08-13 17:38:02.864607 7fe9bae887a0 20 read_log 194215'15322 (194215'15321) delete f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.2174595.0:63367 0.000000 2014-08-13 17:38:02.864618 7fe9bae887a0 20 read_log 194215'15323 (192888'14985) clean ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by osd.4.0:1736 2014-08-12 15:40:55.834074 2014-08-13 17:38:02.864630 7fe9bae887a0 20 read_log 194215'15324 (194215'15323) delete ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.2174595.0:63369 0.000000 2014-08-13 17:38:02.864641 7fe9bae887a0 20 read_log 194215'15325 (192888'14852) clean 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by osd.4.0:1738 2014-08-12 15:40:55.842274 2014-08-13 17:38:02.864653 7fe9bae887a0 20 read_log 194215'15326 (194215'15325) delete 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.2174595.0:63371 0.000000 2014-08-13 17:38:02.864664 7fe9bae887a0 20 read_log 194215'15327 (192888'15018) clean cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by osd.4.0:1740 2014-08-12 15:40:55.849672 2014-08-13 17:38:02.864675 7fe9bae887a0 20 read_log 194215'15328 (194215'15327) delete cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.2174595.0:63373 0.000000 2014-08-13 17:38:02.864686 7fe9bae887a0 20 read_log 194215'15329 (192888'14846) clean cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by osd.4.0:1742 2014-08-12 15:40:55.859192 2014-08-13 17:38:02.864698 7fe9bae887a0 20 read_log 194215'15330 (194215'15329) delete cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.2174595.0:63375 0.000000 2014-08-13 17:38:02.864709 7fe9bae887a0 20 read_log 194215'15331 (193305'15137) clean 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by osd.4.0:1744 2014-08-12 15:40:55.868271 2014-08-13 17:38:02.864721 7fe9bae887a0 20 read_log 194215'15332 (194215'15331) delete 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.2174595.0:63377 0.000000 2014-08-13 17:38:02.864731 7fe9bae887a0 20 read_log 194215'15333 (192888'14837) clean 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by osd.4.0:1746 2014-08-12 15:40:55.876768 2014-08-13 17:38:02.864744 7fe9bae887a0 20 read_log 194215'15334 (194215'15333) delete 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.2174595.0:63379 0.000000 2014-08-13 17:38:02.864754 7fe9bae887a0 20 read_log 194215'15335 (192888'14848) clean 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by osd.4.0:1748 2014-08-12 15:40:55.885182 2014-08-13 17:38:02.864768 7fe9bae887a0 20 read_log 194215'15336 (194215'15335) delete 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.2174595.0:63381 0.000000 2014-08-13 17:38:02.864779 7fe9bae887a0 20 read_log 194215'15337 (193305'15151) clean a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by osd.4.0:1750 2014-08-12 15:40:55.894693 2014-08-13 17:38:02.864791 7fe9bae887a0 20 read_log 194215'15338 (194215'15337) delete a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.2174595.0:63383 0.000000 2014-08-13 17:38:02.864801 7fe9bae887a0 20 read_log 194215'15339 (192888'15000) clean 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by osd.4.0:1752 2014-08-12 15:40:55.903896 2014-08-13 17:38:02.864813 7fe9bae887a0 20 read_log 194215'15340 (194215'15339) delete 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.2174595.0:63385 0.000000 2014-08-13 17:38:02.864823 7fe9bae887a0 20 read_log 194215'15341 (192888'14847) clean d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by osd.4.0:1754 2014-08-12 15:40:55.911668 2014-08-13 17:38:02.864860 7fe9bae887a0 20 read_log 194215'15342 (194215'15341) delete d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.2174595.0:63387 0.000000 2014-08-13 17:38:02.864872 7fe9bae887a0 20 read_log 194215'15343 (193305'15140) clean a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by osd.4.0:1756 2014-08-12 15:40:55.918997 2014-08-13 17:38:02.864884 7fe9bae887a0 20 read_log 194215'15344 (194215'15343) delete a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.2174595.0:63389 0.000000 2014-08-13 17:38:02.864895 7fe9bae887a0 20 read_log 194215'15345 (193305'15128) clean 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by osd.4.0:1758 2014-08-12 15:40:55.926143 2014-08-13 17:38:02.864907 7fe9bae887a0 20 read_log 194215'15346 (194215'15345) delete 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.2174595.0:63391 0.000000 2014-08-13 17:38:02.864917 7fe9bae887a0 20 read_log 194215'15347 (192888'14835) clean 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by osd.4.0:1760 2014-08-12 15:40:55.933898 2014-08-13 17:38:02.864929 7fe9bae887a0 20 read_log 194215'15348 (194215'15347) delete 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.2174595.0:63393 0.000000 2014-08-13 17:38:02.864940 7fe9bae887a0 20 read_log 194215'15349 (192888'14989) clean 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by osd.4.0:1762 2014-08-12 15:40:55.942124 2014-08-13 17:38:02.864952 7fe9bae887a0 20 read_log 194215'15350 (194215'15349) delete 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.2174595.0:63395 0.000000 2014-08-13 17:38:02.864962 7fe9bae887a0 20 read_log 194215'15351 (192888'14858) clean de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by osd.4.0:1764 2014-08-12 15:40:55.950572 2014-08-13 17:38:02.864974 7fe9bae887a0 20 read_log 194215'15352 (194215'15351) delete de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.2174595.0:63397 0.000000 2014-08-13 17:38:02.864985 7fe9bae887a0 20 read_log 194215'15353 (192888'14854) clean 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by osd.4.0:1766 2014-08-12 15:40:55.959839 2014-08-13 17:38:02.864998 7fe9bae887a0 20 read_log 194215'15354 (194215'15353) delete 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.2174595.0:63399 0.000000 2014-08-13 17:38:02.865008 7fe9bae887a0 20 read_log 194215'15355 (193305'15155) clean b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by osd.4.0:1768 2014-08-12 15:40:55.968808 2014-08-13 17:38:02.865020 7fe9bae887a0 20 read_log 194215'15356 (194215'15355) delete b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.2174595.0:63401 0.000000 2014-08-13 17:38:02.865031 7fe9bae887a0 20 read_log 194215'15357 (192888'14850) clean ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by osd.4.0:1770 2014-08-12 15:40:55.977509 2014-08-13 17:38:02.865043 7fe9bae887a0 20 read_log 194215'15358 (194215'15357) delete ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.2174595.0:63403 0.000000 2014-08-13 17:38:02.865053 7fe9bae887a0 20 read_log 194215'15359 (192888'14830) clean b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by osd.4.0:1772 2014-08-12 15:40:55.986061 2014-08-13 17:38:02.865065 7fe9bae887a0 20 read_log 194215'15360 (194215'15359) delete b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.2174595.0:63405 0.000000 2014-08-13 17:38:02.865076 7fe9bae887a0 20 read_log 194215'15361 (193305'15150) clean 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by osd.4.0:1774 2014-08-12 15:40:55.994146 2014-08-13 17:38:02.865095 7fe9bae887a0 20 read_log 194215'15362 (194215'15361) delete 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.2174595.0:63407 0.000000 2014-08-13 17:38:02.865107 7fe9bae887a0 20 read_log 194215'15363 (194215'15363) modify 136/hit_set_24.136_archive_2014-08-12 15:40:54.806616_2014-08-12 17:24:43.007231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.007988 2014-08-13 17:38:02.865119 7fe9bae887a0 20 read_log 194215'15364 (194215'15163) delete 136/hit_set_24.136_archive_2014-08-12 15:32:25.865410_2014-08-12 15:40:54.805843/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.007988 2014-08-13 17:38:02.865131 7fe9bae887a0 20 read_log 194215'15365 (0'0) promote ed8e8936/benchmark_data_storage0111-ib.pocstor_34628_object796/head//24 by osd.4.0:5241 2014-08-12 17:24:43.013546 2014-08-13 17:38:02.865144 7fe9bae887a0 20 read_log 194215'15366 (194215'15365) modify ed8e8936/benchmark_data_storage0111-ib.pocstor_34628_object796/head//24 by client.2193759.0:797 2014-08-12 17:24:43.002165 2014-08-13 17:38:02.865180 7fe9bae887a0 20 read_log 194370'15367 (0'0) promote d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by osd.50.0:4028 2014-08-12 20:30:14.686260 2014-08-13 17:38:02.865193 7fe9bae887a0 20 read_log 194370'15368 (194370'15367) modify d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by client.1508531.0:142 2014-08-12 20:30:14.678612 2014-08-13 17:38:02.865205 7fe9bae887a0 20 read_log 194370'15369 (0'0) promote e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by osd.50.0:4056 2014-08-12 20:30:18.139744 2014-08-13 17:38:02.865217 7fe9bae887a0 20 read_log 194370'15370 (194370'15369) modify e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by client.1508531.0:1005 2014-08-12 20:30:18.134242 2014-08-13 17:38:02.865244 7fe9bae887a0 20 read_log 194370'15371 (0'0) promote f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by osd.50.0:4078 2014-08-12 20:30:23.559797 2014-08-13 17:38:02.865257 7fe9bae887a0 20 read_log 194370'15372 (194370'15371) modify f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by client.1508531.0:2453 2014-08-12 20:30:23.553305 2014-08-13 17:38:02.865269 7fe9bae887a0 20 read_log 194370'15373 (0'0) promote 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by osd.50.0:4080 2014-08-12 20:30:23.658210 2014-08-13 17:38:02.865281 7fe9bae887a0 20 read_log 194370'15374 (194370'15373) modify 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by client.1508531.0:2480 2014-08-12 20:30:23.652101 2014-08-13 17:38:02.865293 7fe9bae887a0 20 read_log 194370'15375 (0'0) promote 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by osd.50.0:4136 2014-08-12 20:30:35.020903 2014-08-13 17:38:02.865305 7fe9bae887a0 20 read_log 194370'15376 (194370'15375) modify 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by client.1508531.0:5413 2014-08-12 20:30:35.010202 2014-08-13 17:38:02.865317 7fe9bae887a0 20 read_log 194370'15377 (0'0) promote 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by osd.50.0:4138 2014-08-12 20:30:35.088069 2014-08-13 17:38:02.865331 7fe9bae887a0 20 read_log 194370'15378 (194370'15377) modify 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by client.1508531.0:5429 2014-08-12 20:30:35.078738 2014-08-13 17:38:02.865344 7fe9bae887a0 20 read_log 194370'15379 (0'0) promote 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by osd.50.0:4140 2014-08-12 20:30:35.293102 2014-08-13 17:38:02.865356 7fe9bae887a0 20 read_log 194370'15380 (194370'15379) modify 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by client.1508531.0:5473 2014-08-12 20:30:35.284477 2014-08-13 17:38:02.865379 7fe9bae887a0 20 read_log 194370'15381 (0'0) promote ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by osd.50.0:4164 2014-08-12 20:30:41.421064 2014-08-13 17:38:02.865391 7fe9bae887a0 20 read_log 194370'15382 (194370'15381) modify ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by client.1508531.0:6812 2014-08-12 20:30:41.412373 2014-08-13 17:38:02.865403 7fe9bae887a0 20 read_log 194370'15383 (0'0) promote 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by osd.50.0:4192 2014-08-12 20:30:50.128567 2014-08-13 17:38:02.865415 7fe9bae887a0 20 read_log 194370'15384 (194370'15383) modify 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by client.1508531.0:8483 2014-08-12 20:30:50.118720 2014-08-13 17:38:02.865427 7fe9bae887a0 20 read_log 194370'15385 (0'0) promote 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by osd.50.0:4214 2014-08-12 20:30:57.232882 2014-08-13 17:38:02.865439 7fe9bae887a0 20 read_log 194370'15386 (194370'15385) modify 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by client.1508531.0:9654 2014-08-12 20:30:57.223661 2014-08-13 17:38:02.865451 7fe9bae887a0 20 read_log 194370'15387 (0'0) promote 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by osd.50.0:4232 2014-08-12 20:31:01.288857 2014-08-13 17:38:02.865463 7fe9bae887a0 20 read_log 194370'15388 (194370'15387) modify 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by client.1508531.0:10344 2014-08-12 20:31:01.281710 2014-08-13 17:38:02.865475 7fe9bae887a0 20 read_log 194370'15389 (0'0) promote cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by osd.50.0:4238 2014-08-12 20:31:02.622586 2014-08-13 17:38:02.865487 7fe9bae887a0 20 read_log 194370'15390 (194370'15389) modify cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by client.1508531.0:10564 2014-08-12 20:31:02.614352 2014-08-13 17:38:02.865499 7fe9bae887a0 20 read_log 194370'15391 (0'0) promote c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by osd.50.0:4262 2014-08-12 20:31:09.245325 2014-08-13 17:38:02.865511 7fe9bae887a0 20 read_log 194370'15392 (194370'15391) modify c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by client.1508531.0:11654 2014-08-12 20:31:09.235996 2014-08-13 17:38:02.865523 7fe9bae887a0 20 read_log 194377'15393 (0'0) promote e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by osd.50.0:4333 2014-08-12 20:33:00.397153 2014-08-13 17:38:02.865535 7fe9bae887a0 20 read_log 194377'15394 (194377'15393) modify e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by client.2231190.0:757 2014-08-12 20:33:00.390011 2014-08-13 17:38:02.865547 7fe9bae887a0 20 read_log 194377'15395 (0'0) promote 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by osd.50.0:4336 2014-08-12 20:33:00.548732 2014-08-13 17:38:02.865559 7fe9bae887a0 20 read_log 194377'15396 (194377'15395) modify 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by client.2231190.0:780 2014-08-12 20:33:00.477364 2014-08-13 17:38:02.865589 7fe9bae887a0 20 read_log 194377'15397 (0'0) promote 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by osd.50.0:4370 2014-08-12 20:33:06.822164 2014-08-13 17:38:02.865603 7fe9bae887a0 20 read_log 194377'15398 (194377'15397) modify 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by client.2231190.0:2397 2014-08-12 20:33:06.812923 2014-08-13 17:38:02.865616 7fe9bae887a0 20 read_log 194377'15399 (0'0) promote 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by osd.50.0:4414 2014-08-12 20:33:15.143469 2014-08-13 17:38:02.865628 7fe9bae887a0 20 read_log 194377'15400 (194377'15399) modify 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by client.2231190.0:4206 2014-08-12 20:33:15.133839 2014-08-13 17:38:02.865648 7fe9bae887a0 20 read_log 194377'15401 (0'0) promote 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by osd.50.0:4430 2014-08-12 20:33:18.845231 2014-08-13 17:38:02.865664 7fe9bae887a0 20 read_log 194377'15402 (194377'15401) modify 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by client.2231190.0:4729 2014-08-12 20:33:18.837895 2014-08-13 17:38:02.865676 7fe9bae887a0 20 read_log 194378'15403 (0'0) promote d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by osd.50.0:4434 2014-08-12 20:33:20.387373 2014-08-13 17:38:02.865688 7fe9bae887a0 20 read_log 194378'15404 (194378'15403) modify d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by client.2231190.0:4860 2014-08-12 20:33:20.379484 2014-08-13 17:38:02.865700 7fe9bae887a0 20 read_log 194380'15405 (0'0) promote d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by osd.50.0:4438 2014-08-12 20:33:23.997984 2014-08-13 17:38:02.865712 7fe9bae887a0 20 read_log 194380'15406 (194380'15405) modify d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by client.2231190.0:5071 2014-08-12 20:33:23.989565 2014-08-13 17:38:02.865724 7fe9bae887a0 20 read_log 194380'15407 (0'0) promote 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by osd.50.0:4440 2014-08-12 20:33:24.053255 2014-08-13 17:38:02.865736 7fe9bae887a0 20 read_log 194380'15408 (194380'15407) modify 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by client.2231190.0:5075 2014-08-12 20:33:24.042859 2014-08-13 17:38:02.865749 7fe9bae887a0 20 read_log 194380'15409 (0'0) promote 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by osd.50.0:4442 2014-08-12 20:33:24.585352 2014-08-13 17:38:02.865761 7fe9bae887a0 20 read_log 194380'15410 (194380'15409) modify 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by client.2231190.0:5113 2014-08-12 20:33:24.579796 2014-08-13 17:38:02.865773 7fe9bae887a0 20 read_log 194393'15411 (194393'15411) modify 136/hit_set_24.136_archive_2014-08-12 20:37:08.028004_2014-08-12 20:49:27.348465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.349262 2014-08-13 17:38:02.865785 7fe9bae887a0 20 read_log 194393'15412 (194215'15363) delete 136/hit_set_24.136_archive_2014-08-12 15:40:54.806616_2014-08-12 17:24:43.007231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.349262 2014-08-13 17:38:02.865797 7fe9bae887a0 20 read_log 194393'15413 (0'0) promote 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by osd.50.0:4544 2014-08-12 20:49:27.354418 2014-08-13 17:38:02.865809 7fe9bae887a0 20 read_log 194393'15414 (194393'15413) modify 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by client.1064182.0:397 2014-08-12 20:49:27.342558 2014-08-13 17:38:02.865821 7fe9bae887a0 20 read_log 194416'15415 (194416'15415) modify 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 17:38:02.865833 7fe9bae887a0 20 read_log 194416'15416 (194393'15411) delete 136/hit_set_24.136_archive_2014-08-12 20:37:08.028004_2014-08-12 20:49:27.348465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 17:38:02.865845 7fe9bae887a0 20 read_log 194416'15417 (0'0) promote f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by osd.50.0:4759 2014-08-13 15:28:40.337941 2014-08-13 17:38:02.865857 7fe9bae887a0 20 read_log 194416'15418 (194416'15417) modify f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by client.2441958.0:2541 2014-08-13 15:28:40.328510 2014-08-13 17:38:02.865869 7fe9bae887a0 20 read_log 194416'15419 (0'0) promote aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by osd.50.0:4789 2014-08-13 15:28:48.305669 2014-08-13 17:38:02.865890 7fe9bae887a0 20 read_log 194416'15420 (194416'15419) modify aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by client.2441958.0:4581 2014-08-13 15:28:48.295597 2014-08-13 17:38:02.865904 7fe9bae887a0 20 read_log 194416'15421 (0'0) promote 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by osd.50.0:4819 2014-08-13 15:28:54.031854 2014-08-13 17:38:02.865916 7fe9bae887a0 20 read_log 194416'15422 (194416'15421) modify 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by client.2441958.0:6064 2014-08-13 15:28:54.020440 2014-08-13 17:38:02.865928 7fe9bae887a0 20 read_log 194416'15423 (0'0) promote 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by osd.50.0:4827 2014-08-13 15:28:56.025679 2014-08-13 17:38:02.865940 7fe9bae887a0 20 read_log 194416'15424 (194416'15423) modify 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by client.2441958.0:6543 2014-08-13 15:28:56.016868 2014-08-13 17:38:02.865952 7fe9bae887a0 20 read_log 194416'15425 (0'0) promote 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by osd.50.0:4833 2014-08-13 15:28:58.647096 2014-08-13 17:38:02.865964 7fe9bae887a0 20 read_log 194416'15426 (194416'15425) modify 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by client.2441958.0:7233 2014-08-13 15:28:58.639937 2014-08-13 17:38:02.865976 7fe9bae887a0 20 read_log 194416'15427 (0'0) promote 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by osd.50.0:4853 2014-08-13 15:29:02.666952 2014-08-13 17:38:02.865988 7fe9bae887a0 20 read_log 194416'15428 (194416'15427) modify 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by client.2441958.0:8300 2014-08-13 15:29:02.659806 2014-08-13 17:38:02.866000 7fe9bae887a0 20 read_log 194416'15429 (0'0) promote af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by osd.50.0:4867 2014-08-13 15:29:06.450451 2014-08-13 17:38:02.866012 7fe9bae887a0 20 read_log 194416'15430 (194416'15429) modify af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by client.2441958.0:9254 2014-08-13 15:29:06.441323 2014-08-13 17:38:02.866024 7fe9bae887a0 20 read_log 194416'15431 (0'0) promote 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by osd.50.0:4877 2014-08-13 15:29:10.265851 2014-08-13 17:38:02.866036 7fe9bae887a0 20 read_log 194416'15432 (194416'15431) modify 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by client.2441958.0:10207 2014-08-13 15:29:10.255722 2014-08-13 17:38:02.866048 7fe9bae887a0 20 read_log 194416'15433 (0'0) promote 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by osd.50.0:4903 2014-08-13 15:29:14.775895 2014-08-13 17:38:02.866060 7fe9bae887a0 20 read_log 194416'15434 (194416'15433) modify 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by client.2441958.0:11325 2014-08-13 15:29:14.766401 2014-08-13 17:38:02.866072 7fe9bae887a0 20 read_log 194416'15435 (0'0) promote b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by osd.50.0:4907 2014-08-13 15:29:15.843710 2014-08-13 17:38:02.866083 7fe9bae887a0 20 read_log 194416'15436 (194416'15435) modify b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by client.2441958.0:11582 2014-08-13 15:29:15.836349 2014-08-13 17:38:02.866095 7fe9bae887a0 20 read_log 194416'15437 (0'0) promote 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by osd.50.0:4919 2014-08-13 15:29:17.552781 2014-08-13 17:38:02.866107 7fe9bae887a0 20 read_log 194416'15438 (194416'15437) modify 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by client.2441958.0:11991 2014-08-13 15:29:17.545175 2014-08-13 17:38:02.866119 7fe9bae887a0 20 read_log 194416'15439 (0'0) promote fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by osd.50.0:4953 2014-08-13 15:29:24.480841 2014-08-13 17:38:02.866138 7fe9bae887a0 20 read_log 194416'15440 (194416'15439) modify fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by client.2441958.0:13587 2014-08-13 15:29:24.471552 2014-08-13 17:38:02.866150 7fe9bae887a0 20 read_log 194416'15441 (0'0) promote 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by osd.50.0:4979 2014-08-13 15:29:31.665871 2014-08-13 17:38:02.866168 7fe9bae887a0 20 read_log 194416'15442 (194416'15441) modify 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by client.2441958.0:15311 2014-08-13 15:29:31.656206 2014-08-13 17:38:02.866180 7fe9bae887a0 20 read_log 194416'15443 (0'0) promote 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by osd.50.0:4983 2014-08-13 15:29:32.279595 2014-08-13 17:38:02.866192 7fe9bae887a0 20 read_log 194416'15444 (194416'15443) modify 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by client.2441958.0:15419 2014-08-13 15:29:32.269452 2014-08-13 17:38:02.866204 7fe9bae887a0 20 read_log 194416'15445 (0'0) promote a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by osd.50.0:5019 2014-08-13 15:29:46.036094 2014-08-13 17:38:02.866216 7fe9bae887a0 20 read_log 194416'15446 (194416'15445) modify a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by client.2441958.0:16908 2014-08-13 15:29:46.025271 2014-08-13 17:38:02.866229 7fe9bae887a0 20 read_log 194422'15447 (0'0) promote 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by osd.50.0:5053 2014-08-13 15:32:19.133039 2014-08-13 17:38:02.866240 7fe9bae887a0 20 read_log 194422'15448 (194422'15447) modify 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by client.1148203.0:956 2014-08-13 15:32:19.125424 2014-08-13 17:38:02.866253 7fe9bae887a0 20 read_log 194422'15449 (0'0) promote 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by osd.50.0:5061 2014-08-13 15:32:19.718056 2014-08-13 17:38:02.866265 7fe9bae887a0 20 read_log 194422'15450 (194422'15449) modify 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by client.1148203.0:1112 2014-08-13 15:32:19.709026 2014-08-13 17:38:02.866277 7fe9bae887a0 20 read_log 194422'15451 (0'0) promote 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by osd.50.0:5073 2014-08-13 15:32:23.772106 2014-08-13 17:38:02.866289 7fe9bae887a0 20 read_log 194422'15452 (194422'15451) modify 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by client.1148203.0:2140 2014-08-13 15:32:23.763153 2014-08-13 17:38:02.866301 7fe9bae887a0 20 read_log 194422'15453 (0'0) promote 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by osd.50.0:5079 2014-08-13 15:32:25.821181 2014-08-13 17:38:02.866313 7fe9bae887a0 20 read_log 194422'15454 (194422'15453) modify 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by client.1148203.0:2680 2014-08-13 15:32:25.813684 2014-08-13 17:38:02.866325 7fe9bae887a0 20 read_log 194422'15455 (0'0) promote 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by osd.50.0:5097 2014-08-13 15:32:29.907712 2014-08-13 17:38:02.866336 7fe9bae887a0 20 read_log 194422'15456 (194422'15455) modify 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by client.1148203.0:3710 2014-08-13 15:32:29.898750 2014-08-13 17:38:02.866348 7fe9bae887a0 20 read_log 194422'15457 (0'0) promote 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by osd.50.0:5101 2014-08-13 15:32:30.200218 2014-08-13 17:38:02.866360 7fe9bae887a0 20 read_log 194422'15458 (194422'15457) modify 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by client.1148203.0:3790 2014-08-13 15:32:30.192225 2014-08-13 17:38:02.866372 7fe9bae887a0 20 read_log 194422'15459 (0'0) promote e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by osd.50.0:5113 2014-08-13 15:32:31.725484 2014-08-13 17:38:02.866391 7fe9bae887a0 20 read_log 194422'15460 (194422'15459) modify e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by client.1148203.0:4192 2014-08-13 15:32:31.717484 2014-08-13 17:38:02.866404 7fe9bae887a0 20 read_log 194422'15461 (0'0) promote 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by osd.50.0:5115 2014-08-13 15:32:31.791282 2014-08-13 17:38:02.866416 7fe9bae887a0 20 read_log 194422'15462 (194422'15461) modify 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by client.1148203.0:4197 2014-08-13 15:32:31.736979 2014-08-13 17:38:02.866430 7fe9bae887a0 20 read_log 194422'15463 (0'0) promote 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by osd.50.0:5157 2014-08-13 15:32:48.143485 2014-08-13 17:38:02.866462 7fe9bae887a0 20 read_log 194422'15464 (194422'15463) modify 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by client.1148203.0:7426 2014-08-13 15:32:48.134944 2014-08-13 17:38:02.866475 7fe9bae887a0 20 read_log 194422'15465 (0'0) promote e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by osd.50.0:5179 2014-08-13 15:32:55.378322 2014-08-13 17:38:02.866487 7fe9bae887a0 20 read_log 194422'15466 (194422'15465) modify e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by client.1148203.0:8538 2014-08-13 15:32:55.369835 2014-08-13 17:38:02.866499 7fe9bae887a0 20 read_log 194422'15467 (0'0) promote 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by osd.50.0:5185 2014-08-13 15:32:57.125714 2014-08-13 17:38:02.866511 7fe9bae887a0 20 read_log 194422'15468 (194422'15467) modify 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by client.1148203.0:8749 2014-08-13 15:32:57.115446 2014-08-13 17:38:02.866523 7fe9bae887a0 20 read_log 194422'15469 (0'0) promote 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by osd.50.0:5199 2014-08-13 15:33:04.743917 2014-08-13 17:38:02.866535 7fe9bae887a0 20 read_log 194422'15470 (194422'15469) modify 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by client.1148203.0:9527 2014-08-13 15:33:04.733257 2014-08-13 17:38:02.866550 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.866982 7fe9bae887a0 10 read_log checking for missing items over interval (194370'15367,194422'15470] 2014-08-13 17:38:02.867029 7fe9bae887a0 15 read_log missing 194422'15470 (194422'15469) modify 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by client.1148203.0:9527 2014-08-13 15:33:04.733257 2014-08-13 17:38:02.867070 7fe9bae887a0 15 read_log missing 194422'15468 (194422'15467) modify 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by client.1148203.0:8749 2014-08-13 15:32:57.115446 2014-08-13 17:38:02.867108 7fe9bae887a0 15 read_log missing 194422'15466 (194422'15465) modify e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by client.1148203.0:8538 2014-08-13 15:32:55.369835 2014-08-13 17:38:02.867145 7fe9bae887a0 15 read_log missing 194422'15464 (194422'15463) modify 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by client.1148203.0:7426 2014-08-13 15:32:48.134944 2014-08-13 17:38:02.867188 7fe9bae887a0 15 read_log missing 194422'15462 (194422'15461) modify 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by client.1148203.0:4197 2014-08-13 15:32:31.736979 2014-08-13 17:38:02.867225 7fe9bae887a0 15 read_log missing 194422'15460 (194422'15459) modify e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by client.1148203.0:4192 2014-08-13 15:32:31.717484 2014-08-13 17:38:02.867262 7fe9bae887a0 15 read_log missing 194422'15458 (194422'15457) modify 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by client.1148203.0:3790 2014-08-13 15:32:30.192225 2014-08-13 17:38:02.867299 7fe9bae887a0 15 read_log missing 194422'15456 (194422'15455) modify 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by client.1148203.0:3710 2014-08-13 15:32:29.898750 2014-08-13 17:38:02.867336 7fe9bae887a0 15 read_log missing 194422'15454 (194422'15453) modify 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by client.1148203.0:2680 2014-08-13 15:32:25.813684 2014-08-13 17:38:02.867373 7fe9bae887a0 15 read_log missing 194422'15452 (194422'15451) modify 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by client.1148203.0:2140 2014-08-13 15:32:23.763153 2014-08-13 17:38:02.867410 7fe9bae887a0 15 read_log missing 194422'15450 (194422'15449) modify 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by client.1148203.0:1112 2014-08-13 15:32:19.709026 2014-08-13 17:38:02.867447 7fe9bae887a0 15 read_log missing 194422'15448 (194422'15447) modify 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by client.1148203.0:956 2014-08-13 15:32:19.125424 2014-08-13 17:38:02.867484 7fe9bae887a0 15 read_log missing 194416'15446 (194416'15445) modify a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by client.2441958.0:16908 2014-08-13 15:29:46.025271 2014-08-13 17:38:02.867521 7fe9bae887a0 15 read_log missing 194416'15444 (194416'15443) modify 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by client.2441958.0:15419 2014-08-13 15:29:32.269452 2014-08-13 17:38:02.867557 7fe9bae887a0 15 read_log missing 194416'15442 (194416'15441) modify 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by client.2441958.0:15311 2014-08-13 15:29:31.656206 2014-08-13 17:38:02.867593 7fe9bae887a0 15 read_log missing 194416'15440 (194416'15439) modify fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by client.2441958.0:13587 2014-08-13 15:29:24.471552 2014-08-13 17:38:02.867629 7fe9bae887a0 15 read_log missing 194416'15438 (194416'15437) modify 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by client.2441958.0:11991 2014-08-13 15:29:17.545175 2014-08-13 17:38:02.867665 7fe9bae887a0 15 read_log missing 194416'15436 (194416'15435) modify b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by client.2441958.0:11582 2014-08-13 15:29:15.836349 2014-08-13 17:38:02.867701 7fe9bae887a0 15 read_log missing 194416'15434 (194416'15433) modify 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by client.2441958.0:11325 2014-08-13 15:29:14.766401 2014-08-13 17:38:02.867737 7fe9bae887a0 15 read_log missing 194416'15432 (194416'15431) modify 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by client.2441958.0:10207 2014-08-13 15:29:10.255722 2014-08-13 17:38:02.867772 7fe9bae887a0 15 read_log missing 194416'15430 (194416'15429) modify af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by client.2441958.0:9254 2014-08-13 15:29:06.441323 2014-08-13 17:38:02.867808 7fe9bae887a0 15 read_log missing 194416'15428 (194416'15427) modify 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by client.2441958.0:8300 2014-08-13 15:29:02.659806 2014-08-13 17:38:02.867844 7fe9bae887a0 15 read_log missing 194416'15426 (194416'15425) modify 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by client.2441958.0:7233 2014-08-13 15:28:58.639937 2014-08-13 17:38:02.867880 7fe9bae887a0 15 read_log missing 194416'15424 (194416'15423) modify 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by client.2441958.0:6543 2014-08-13 15:28:56.016868 2014-08-13 17:38:02.867915 7fe9bae887a0 15 read_log missing 194416'15422 (194416'15421) modify 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by client.2441958.0:6064 2014-08-13 15:28:54.020440 2014-08-13 17:38:02.867951 7fe9bae887a0 15 read_log missing 194416'15420 (194416'15419) modify aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by client.2441958.0:4581 2014-08-13 15:28:48.295597 2014-08-13 17:38:02.867987 7fe9bae887a0 15 read_log missing 194416'15418 (194416'15417) modify f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by client.2441958.0:2541 2014-08-13 15:28:40.328510 2014-08-13 17:38:02.868027 7fe9bae887a0 15 read_log missing 194416'15415 (194416'15415) modify 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 17:38:02.868063 7fe9bae887a0 15 read_log missing 194393'15414 (194393'15413) modify 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by client.1064182.0:397 2014-08-12 20:49:27.342558 2014-08-13 17:38:02.868104 7fe9bae887a0 15 read_log missing 194380'15410 (194380'15409) modify 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by client.2231190.0:5113 2014-08-12 20:33:24.579796 2014-08-13 17:38:02.868140 7fe9bae887a0 15 read_log missing 194380'15408 (194380'15407) modify 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by client.2231190.0:5075 2014-08-12 20:33:24.042859 2014-08-13 17:38:02.868189 7fe9bae887a0 15 read_log missing 194380'15406 (194380'15405) modify d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by client.2231190.0:5071 2014-08-12 20:33:23.989565 2014-08-13 17:38:02.868227 7fe9bae887a0 15 read_log missing 194378'15404 (194378'15403) modify d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by client.2231190.0:4860 2014-08-12 20:33:20.379484 2014-08-13 17:38:02.868264 7fe9bae887a0 15 read_log missing 194377'15402 (194377'15401) modify 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by client.2231190.0:4729 2014-08-12 20:33:18.837895 2014-08-13 17:38:02.868301 7fe9bae887a0 15 read_log missing 194377'15400 (194377'15399) modify 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by client.2231190.0:4206 2014-08-12 20:33:15.133839 2014-08-13 17:38:02.868338 7fe9bae887a0 15 read_log missing 194377'15398 (194377'15397) modify 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by client.2231190.0:2397 2014-08-12 20:33:06.812923 2014-08-13 17:38:02.868375 7fe9bae887a0 15 read_log missing 194377'15396 (194377'15395) modify 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by client.2231190.0:780 2014-08-12 20:33:00.477364 2014-08-13 17:38:02.868411 7fe9bae887a0 15 read_log missing 194377'15394 (194377'15393) modify e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by client.2231190.0:757 2014-08-12 20:33:00.390011 2014-08-13 17:38:02.868448 7fe9bae887a0 15 read_log missing 194370'15392 (194370'15391) modify c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by client.1508531.0:11654 2014-08-12 20:31:09.235996 2014-08-13 17:38:02.868485 7fe9bae887a0 15 read_log missing 194370'15390 (194370'15389) modify cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by client.1508531.0:10564 2014-08-12 20:31:02.614352 2014-08-13 17:38:02.868522 7fe9bae887a0 15 read_log missing 194370'15388 (194370'15387) modify 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by client.1508531.0:10344 2014-08-12 20:31:01.281710 2014-08-13 17:38:02.868559 7fe9bae887a0 15 read_log missing 194370'15386 (194370'15385) modify 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by client.1508531.0:9654 2014-08-12 20:30:57.223661 2014-08-13 17:38:02.868596 7fe9bae887a0 15 read_log missing 194370'15384 (194370'15383) modify 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by client.1508531.0:8483 2014-08-12 20:30:50.118720 2014-08-13 17:38:02.868632 7fe9bae887a0 15 read_log missing 194370'15382 (194370'15381) modify ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by client.1508531.0:6812 2014-08-12 20:30:41.412373 2014-08-13 17:38:02.868669 7fe9bae887a0 15 read_log missing 194370'15380 (194370'15379) modify 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by client.1508531.0:5473 2014-08-12 20:30:35.284477 2014-08-13 17:38:02.868706 7fe9bae887a0 15 read_log missing 194370'15378 (194370'15377) modify 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by client.1508531.0:5429 2014-08-12 20:30:35.078738 2014-08-13 17:38:02.868746 7fe9bae887a0 15 read_log missing 194370'15376 (194370'15375) modify 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by client.1508531.0:5413 2014-08-12 20:30:35.010202 2014-08-13 17:38:02.868783 7fe9bae887a0 15 read_log missing 194370'15374 (194370'15373) modify 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by client.1508531.0:2480 2014-08-12 20:30:23.652101 2014-08-13 17:38:02.868820 7fe9bae887a0 15 read_log missing 194370'15372 (194370'15371) modify f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by client.1508531.0:2453 2014-08-12 20:30:23.553305 2014-08-13 17:38:02.868858 7fe9bae887a0 15 read_log missing 194370'15370 (194370'15369) modify e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by client.1508531.0:1005 2014-08-12 20:30:18.134242 2014-08-13 17:38:02.868895 7fe9bae887a0 15 read_log missing 194370'15368 (194370'15367) modify d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by client.1508531.0:142 2014-08-12 20:30:14.678612 2014-08-13 17:38:02.868905 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.868934 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_loaded 2014-08-13 17:38:02.868952 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Initial 0.014174 0 0.000000 2014-08-13 17:38:02.868964 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset 2014-08-13 17:38:02.868973 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194478 2014-08-13 17:38:02.868983 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 17:38:02.868994 7fe9bae887a0 10 osd.4 194478 pgid 24.181 coll 24.181_head 2014-08-13 17:38:02.869117 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.181 2014-08-13 17:38:02.869130 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.869206 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.181(unlocked)] enter Initial 2014-08-13 17:38:02.869218 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.181(unlocked)] enter NotTrimming 2014-08-13 17:38:02.869348 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.869461 7fe9bae887a0 20 read_log 192834'17290 (0'0) promote 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18226 2014-08-11 15:26:56.142374 2014-08-13 17:38:02.869482 7fe9bae887a0 20 read_log 192834'17291 (192834'17290) modify 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by client.1883634.0:10673 2014-08-11 15:26:56.133982 2014-08-13 17:38:02.869495 7fe9bae887a0 20 read_log 192834'17292 (0'0) promote 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18228 2014-08-11 15:26:56.271737 2014-08-13 17:38:02.869508 7fe9bae887a0 20 read_log 192834'17293 (192834'17292) modify 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by client.1883634.0:10701 2014-08-11 15:26:56.263816 2014-08-13 17:38:02.869520 7fe9bae887a0 20 read_log 192834'17324 (192834'16876) clean 491e0581/benchmark_data_storage0111-ib.pocstor_26659_object12533/head//24 by osd.13.0:18260 2014-08-11 15:26:57.995293 2014-08-13 17:38:02.869532 7fe9bae887a0 20 read_log 192834'17325 (192834'16839) clean 3b031581/benchmark_data_storage0111-ib.pocstor_26659_object11165/head//24 by osd.13.0:18261 2014-08-11 15:26:58.015503 2014-08-13 17:38:02.869545 7fe9bae887a0 20 read_log 192834'17326 (192834'16878) clean ee95581/benchmark_data_storage0111-ib.pocstor_26659_object12648/head//24 by osd.13.0:18262 2014-08-11 15:26:58.016079 2014-08-13 17:38:02.869557 7fe9bae887a0 20 read_log 192834'17330 (192834'16802) clean 593e2981/benchmark_data_storage0111-ib.pocstor_26659_object10540/head//24 by osd.13.0:18266 2014-08-11 15:26:58.063987 2014-08-13 17:38:02.869600 7fe9bae887a0 20 read_log 192834'17331 (192834'16904) clean c8cfa981/benchmark_data_storage0111-ib.pocstor_26659_object14036/head//24 by osd.13.0:18267 2014-08-11 15:26:58.064306 2014-08-13 17:38:02.869616 7fe9bae887a0 20 read_log 192834'17333 (192834'16896) clean a1011d81/benchmark_data_storage0111-ib.pocstor_26659_object13209/head//24 by osd.13.0:18269 2014-08-11 15:26:58.089682 2014-08-13 17:38:02.869628 7fe9bae887a0 20 read_log 192834'17334 (192834'16804) clean 8185dd81/benchmark_data_storage0111-ib.pocstor_26659_object10573/head//24 by osd.13.0:18270 2014-08-11 15:26:58.090253 2014-08-13 17:38:02.869640 7fe9bae887a0 20 read_log 192834'17336 (192834'16910) clean 11593181/benchmark_data_storage0111-ib.pocstor_26659_object14300/head//24 by osd.13.0:18272 2014-08-11 15:26:58.328812 2014-08-13 17:38:02.869653 7fe9bae887a0 20 read_log 192834'17346 (0'0) promote d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18282 2014-08-11 15:26:59.899134 2014-08-13 17:38:02.869665 7fe9bae887a0 20 read_log 192834'17347 (192834'17346) modify d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by client.1883634.0:11454 2014-08-11 15:26:59.889020 2014-08-13 17:38:02.869677 7fe9bae887a0 20 read_log 192834'17350 (0'0) promote dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18286 2014-08-11 15:27:03.202913 2014-08-13 17:38:02.869689 7fe9bae887a0 20 read_log 192834'17351 (192834'17350) modify dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by client.1883634.0:12251 2014-08-11 15:27:03.194397 2014-08-13 17:38:02.869702 7fe9bae887a0 20 read_log 192834'17414 (192834'17336) delete 11593181/benchmark_data_storage0111-ib.pocstor_26659_object14300/head//24 by osd.13.0:18350 2014-08-11 15:27:05.574640 2014-08-13 17:38:02.869714 7fe9bae887a0 20 read_log 192834'17415 (192834'17155) delete fd12b181/benchmark_data_storage0111-ib.pocstor_26659_object323/head//24 by osd.13.0:18351 2014-08-11 15:27:05.581366 2014-08-13 17:38:02.869726 7fe9bae887a0 20 read_log 192834'17417 (192834'16992) clean 329f4181/benchmark_data_storage0111-ib.pocstor_26659_object19253/head//24 by osd.13.0:18353 2014-08-11 15:27:05.587839 2014-08-13 17:38:02.869738 7fe9bae887a0 20 read_log 192834'17418 (192834'16941) clean 7de6b181/benchmark_data_storage0111-ib.pocstor_26659_object15116/head//24 by osd.13.0:18354 2014-08-11 15:27:05.594408 2014-08-13 17:38:02.869750 7fe9bae887a0 20 read_log 192834'17437 (192834'17324) delete 491e0581/benchmark_data_storage0111-ib.pocstor_26659_object12533/head//24 by osd.13.0:18373 2014-08-11 15:27:05.886660 2014-08-13 17:38:02.869762 7fe9bae887a0 20 read_log 192834'17438 (192834'17325) delete 3b031581/benchmark_data_storage0111-ib.pocstor_26659_object11165/head//24 by osd.13.0:18374 2014-08-11 15:27:05.892558 2014-08-13 17:38:02.869774 7fe9bae887a0 20 read_log 192834'17439 (192834'17253) delete 706e1581/benchmark_data_storage0111-ib.pocstor_26659_object7704/head//24 by osd.13.0:18375 2014-08-11 15:27:05.898428 2014-08-13 17:38:02.869786 7fe9bae887a0 20 read_log 192834'17440 (192834'17268) delete 8e503581/benchmark_data_storage0111-ib.pocstor_26659_object10189/head//24 by osd.13.0:18376 2014-08-11 15:27:05.903809 2014-08-13 17:38:02.869799 7fe9bae887a0 20 read_log 192834'17441 (192834'17202) delete 31575581/benchmark_data_storage0111-ib.pocstor_26659_object5439/head//24 by osd.13.0:18377 2014-08-11 15:27:05.908642 2014-08-13 17:38:02.869811 7fe9bae887a0 20 read_log 192834'17442 (192834'17326) delete ee95581/benchmark_data_storage0111-ib.pocstor_26659_object12648/head//24 by osd.13.0:18378 2014-08-11 15:27:05.914337 2014-08-13 17:38:02.869823 7fe9bae887a0 20 read_log 192834'17443 (192834'17254) delete e0a48581/benchmark_data_storage0111-ib.pocstor_26659_object9186/head//24 by osd.13.0:18379 2014-08-11 15:27:05.925461 2014-08-13 17:38:02.869835 7fe9bae887a0 20 read_log 192834'17444 (192834'16968) clean 67cc5581/benchmark_data_storage0111-ib.pocstor_26659_object16450/head//24 by osd.13.0:18380 2014-08-11 15:27:05.931672 2014-08-13 17:38:02.869854 7fe9bae887a0 20 read_log 192834'17446 (192834'17003) clean 3968d581/benchmark_data_storage0111-ib.pocstor_26659_object20233/head//24 by osd.13.0:18382 2014-08-11 15:27:05.947068 2014-08-13 17:38:02.869867 7fe9bae887a0 20 read_log 192834'17447 (192834'16945) clean 49b7e581/benchmark_data_storage0111-ib.pocstor_26659_object15256/head//24 by osd.13.0:18383 2014-08-11 15:27:05.947539 2014-08-13 17:38:02.869880 7fe9bae887a0 20 read_log 192834'17468 (192834'17159) delete aa372981/benchmark_data_storage0111-ib.pocstor_26659_object1476/head//24 by osd.13.0:18404 2014-08-11 15:27:06.141493 2014-08-13 17:38:02.869892 7fe9bae887a0 20 read_log 192834'17469 (192834'17330) delete 593e2981/benchmark_data_storage0111-ib.pocstor_26659_object10540/head//24 by osd.13.0:18405 2014-08-11 15:27:06.148804 2014-08-13 17:38:02.869904 7fe9bae887a0 20 read_log 192834'17470 (192834'17260) delete 19476981/benchmark_data_storage0111-ib.pocstor_26659_object9234/head//24 by osd.13.0:18406 2014-08-11 15:27:06.154281 2014-08-13 17:38:02.869916 7fe9bae887a0 20 read_log 192834'17471 (192834'17331) delete c8cfa981/benchmark_data_storage0111-ib.pocstor_26659_object14036/head//24 by osd.13.0:18407 2014-08-11 15:27:06.154787 2014-08-13 17:38:02.869927 7fe9bae887a0 20 read_log 192834'17474 (192834'17021) clean 78ef3981/benchmark_data_storage0111-ib.pocstor_26659_object22036/head//24 by osd.13.0:18410 2014-08-11 15:27:06.171822 2014-08-13 17:38:02.869940 7fe9bae887a0 20 read_log 192834'17475 (192834'16949) clean 1cc2b981/benchmark_data_storage0111-ib.pocstor_26659_object15351/head//24 by osd.13.0:18411 2014-08-11 15:27:06.172418 2014-08-13 17:38:02.869951 7fe9bae887a0 20 read_log 192834'17476 (192834'16994) clean 4f75d981/benchmark_data_storage0111-ib.pocstor_26659_object19627/head//24 by osd.13.0:18412 2014-08-11 15:27:06.172747 2014-08-13 17:38:02.869963 7fe9bae887a0 20 read_log 192834'17492 (192834'17333) delete a1011d81/benchmark_data_storage0111-ib.pocstor_26659_object13209/head//24 by osd.13.0:18428 2014-08-11 15:27:06.660662 2014-08-13 17:38:02.869975 7fe9bae887a0 20 read_log 192834'17493 (192834'17210) delete d19a2d81/benchmark_data_storage0111-ib.pocstor_26659_object5812/head//24 by osd.13.0:18429 2014-08-11 15:27:06.666226 2014-08-13 17:38:02.869991 7fe9bae887a0 20 read_log 192834'17494 (192834'17211) delete 6c4d5d81/benchmark_data_storage0111-ib.pocstor_26659_object5334/head//24 by osd.13.0:18430 2014-08-11 15:27:06.668752 2014-08-13 17:38:02.870003 7fe9bae887a0 20 read_log 192834'17495 (192834'17263) delete 1e677d81/benchmark_data_storage0111-ib.pocstor_26659_object9002/head//24 by osd.13.0:18431 2014-08-11 15:27:06.669104 2014-08-13 17:38:02.870015 7fe9bae887a0 20 read_log 192834'17496 (192834'17023) clean e2272d81/benchmark_data_storage0111-ib.pocstor_26659_object22061/head//24 by osd.13.0:18432 2014-08-11 15:27:06.672119 2014-08-13 17:38:02.870027 7fe9bae887a0 20 read_log 192834'17497 (192834'17212) delete 9deccd81/benchmark_data_storage0111-ib.pocstor_26659_object3029/head//24 by osd.13.0:18433 2014-08-11 15:27:06.674717 2014-08-13 17:38:02.870039 7fe9bae887a0 20 read_log 192834'17498 (192834'16981) clean c1638d81/benchmark_data_storage0111-ib.pocstor_26659_object17673/head//24 by osd.13.0:18434 2014-08-11 15:27:06.674982 2014-08-13 17:38:02.870052 7fe9bae887a0 20 read_log 192834'17499 (192834'17018) clean 14cbbd81/benchmark_data_storage0111-ib.pocstor_26659_object21700/head//24 by osd.13.0:18435 2014-08-11 15:27:06.675226 2014-08-13 17:38:02.870064 7fe9bae887a0 20 read_log 192834'17500 (192834'17334) delete 8185dd81/benchmark_data_storage0111-ib.pocstor_26659_object10573/head//24 by osd.13.0:18436 2014-08-11 15:27:06.679826 2014-08-13 17:38:02.870078 7fe9bae887a0 20 read_log 192834'17501 (192834'17213) delete f2b7fd81/benchmark_data_storage0111-ib.pocstor_26659_object4829/head//24 by osd.13.0:18437 2014-08-11 15:27:06.684253 2014-08-13 17:38:02.870091 7fe9bae887a0 20 read_log 192834'17502 (192834'17214) delete f6aefd81/benchmark_data_storage0111-ib.pocstor_26659_object4912/head//24 by osd.13.0:18438 2014-08-11 15:27:06.688303 2014-08-13 17:38:02.870114 7fe9bae887a0 20 read_log 192834'17597 (192834'17417) delete 329f4181/benchmark_data_storage0111-ib.pocstor_26659_object19253/head//24 by osd.13.0:18533 2014-08-11 15:27:07.288394 2014-08-13 17:38:02.870129 7fe9bae887a0 20 read_log 192834'17598 (192834'17418) delete 7de6b181/benchmark_data_storage0111-ib.pocstor_26659_object15116/head//24 by osd.13.0:18534 2014-08-11 15:27:07.292042 2014-08-13 17:38:02.870141 7fe9bae887a0 20 read_log 192834'17604 (192834'17444) delete 67cc5581/benchmark_data_storage0111-ib.pocstor_26659_object16450/head//24 by osd.13.0:18540 2014-08-11 15:27:07.315677 2014-08-13 17:38:02.870159 7fe9bae887a0 20 read_log 192834'17605 (192834'17446) delete 3968d581/benchmark_data_storage0111-ib.pocstor_26659_object20233/head//24 by osd.13.0:18541 2014-08-11 15:27:07.319783 2014-08-13 17:38:02.870175 7fe9bae887a0 20 read_log 192834'17606 (192834'17447) delete 49b7e581/benchmark_data_storage0111-ib.pocstor_26659_object15256/head//24 by osd.13.0:18542 2014-08-11 15:27:07.323144 2014-08-13 17:38:02.870187 7fe9bae887a0 20 read_log 192834'17615 (192834'17474) delete 78ef3981/benchmark_data_storage0111-ib.pocstor_26659_object22036/head//24 by osd.13.0:18551 2014-08-11 15:27:07.353216 2014-08-13 17:38:02.870200 7fe9bae887a0 20 read_log 192834'17616 (192834'17475) delete 1cc2b981/benchmark_data_storage0111-ib.pocstor_26659_object15351/head//24 by osd.13.0:18552 2014-08-11 15:27:07.357016 2014-08-13 17:38:02.870212 7fe9bae887a0 20 read_log 192834'17617 (192834'17476) delete 4f75d981/benchmark_data_storage0111-ib.pocstor_26659_object19627/head//24 by osd.13.0:18553 2014-08-11 15:27:07.360260 2014-08-13 17:38:02.870223 7fe9bae887a0 20 read_log 192834'17627 (192834'17496) delete e2272d81/benchmark_data_storage0111-ib.pocstor_26659_object22061/head//24 by osd.13.0:18563 2014-08-11 15:27:07.373066 2014-08-13 17:38:02.870238 7fe9bae887a0 20 read_log 192834'17628 (192834'17498) delete c1638d81/benchmark_data_storage0111-ib.pocstor_26659_object17673/head//24 by osd.13.0:18564 2014-08-11 15:27:07.373878 2014-08-13 17:38:02.870250 7fe9bae887a0 20 read_log 192834'17629 (192834'17499) delete 14cbbd81/benchmark_data_storage0111-ib.pocstor_26659_object21700/head//24 by osd.13.0:18565 2014-08-11 15:27:07.376663 2014-08-13 17:38:02.870262 7fe9bae887a0 20 read_log 192834'17649 (0'0) promote b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18585 2014-08-11 15:27:10.249546 2014-08-13 17:38:02.870274 7fe9bae887a0 20 read_log 192834'17650 (192834'17649) modify b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by client.1883634.0:13611 2014-08-11 15:27:10.240825 2014-08-13 17:38:02.870286 7fe9bae887a0 20 read_log 192834'17669 (0'0) promote f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18605 2014-08-11 15:27:14.852985 2014-08-13 17:38:02.870298 7fe9bae887a0 20 read_log 192834'17670 (192834'17669) modify f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by client.1883634.0:14710 2014-08-11 15:27:14.844768 2014-08-13 17:38:02.870310 7fe9bae887a0 20 read_log 192834'17691 (192834'17030) clean 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:18627 2014-08-11 15:27:19.111902 2014-08-13 17:38:02.870322 7fe9bae887a0 20 read_log 192834'17699 (192834'17691) delete 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:18635 2014-08-11 15:27:19.178885 2014-08-13 17:38:02.870334 7fe9bae887a0 20 read_log 192834'17711 (0'0) promote f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:18647 2014-08-11 15:27:22.198168 2014-08-13 17:38:02.870346 7fe9bae887a0 20 read_log 192834'17712 (192834'17711) modify f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by client.1883634.0:15891 2014-08-11 15:27:22.189467 2014-08-13 17:38:02.870358 7fe9bae887a0 20 read_log 192834'17743 (192834'17070) clean d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:18679 2014-08-11 15:27:26.378659 2014-08-13 17:38:02.870380 7fe9bae887a0 20 read_log 192834'17744 (0'0) promote 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:18680 2014-08-11 15:27:26.612915 2014-08-13 17:38:02.870396 7fe9bae887a0 20 read_log 192834'17745 (192834'17744) modify 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by client.1883634.0:16241 2014-08-11 15:27:26.604648 2014-08-13 17:38:02.870408 7fe9bae887a0 20 read_log 192834'17748 (0'0) promote 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:18684 2014-08-11 15:27:27.334259 2014-08-13 17:38:02.870420 7fe9bae887a0 20 read_log 192834'17749 (192834'17748) modify 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by client.1883634.0:16383 2014-08-11 15:27:27.326581 2014-08-13 17:38:02.870432 7fe9bae887a0 20 read_log 192834'17754 (0'0) promote 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:18690 2014-08-11 15:27:28.448576 2014-08-13 17:38:02.870444 7fe9bae887a0 20 read_log 192834'17755 (192834'17754) modify 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by client.1883634.0:16612 2014-08-11 15:27:28.440537 2014-08-13 17:38:02.870456 7fe9bae887a0 20 read_log 192834'17760 (0'0) promote 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:18696 2014-08-11 15:27:30.275302 2014-08-13 17:38:02.870468 7fe9bae887a0 20 read_log 192834'17761 (192834'17760) modify 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by client.1883634.0:16994 2014-08-11 15:27:30.267363 2014-08-13 17:38:02.870480 7fe9bae887a0 20 read_log 192834'17771 (192834'17072) clean 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:18707 2014-08-11 15:27:32.925390 2014-08-13 17:38:02.870492 7fe9bae887a0 20 read_log 192834'17772 (192834'17098) clean ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:18708 2014-08-11 15:27:32.926582 2014-08-13 17:38:02.870512 7fe9bae887a0 20 read_log 192834'17773 (192834'17092) clean 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:18709 2014-08-11 15:27:32.936848 2014-08-13 17:38:02.870525 7fe9bae887a0 20 read_log 192834'17774 (192834'17743) delete d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:18710 2014-08-11 15:27:33.896114 2014-08-13 17:38:02.870537 7fe9bae887a0 20 read_log 192834'17775 (192834'17086) clean e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:18711 2014-08-11 15:27:33.974366 2014-08-13 17:38:02.870571 7fe9bae887a0 20 read_log 192834'17776 (192834'17076) clean 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:18712 2014-08-11 15:27:33.975119 2014-08-13 17:38:02.870586 7fe9bae887a0 20 read_log 192834'17796 (192834'17771) delete 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:18732 2014-08-11 15:27:35.493146 2014-08-13 17:38:02.870599 7fe9bae887a0 20 read_log 192834'17797 (192834'17772) delete ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:18733 2014-08-11 15:27:35.497531 2014-08-13 17:38:02.870611 7fe9bae887a0 20 read_log 192834'17798 (192834'17773) delete 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:18734 2014-08-11 15:27:35.498141 2014-08-13 17:38:02.870623 7fe9bae887a0 20 read_log 192834'17799 (192834'17775) delete e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:18735 2014-08-11 15:27:35.505568 2014-08-13 17:38:02.870635 7fe9bae887a0 20 read_log 192834'17800 (192834'17776) delete 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:18736 2014-08-11 15:27:35.509900 2014-08-13 17:38:02.870647 7fe9bae887a0 20 read_log 192834'17825 (0'0) promote 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:18761 2014-08-11 15:27:39.222838 2014-08-13 17:38:02.870666 7fe9bae887a0 20 read_log 192834'17826 (192834'17825) modify 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by client.1883634.0:17825 2014-08-11 15:27:39.214203 2014-08-13 17:38:02.870681 7fe9bae887a0 20 read_log 192834'17852 (192834'17138) clean 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:18788 2014-08-11 15:27:41.906046 2014-08-13 17:38:02.870693 7fe9bae887a0 20 read_log 192834'17853 (192834'17146) clean 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:18789 2014-08-11 15:27:41.906313 2014-08-13 17:38:02.870705 7fe9bae887a0 20 read_log 192834'17869 (192834'17852) delete 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:18805 2014-08-11 15:27:47.158942 2014-08-13 17:38:02.870717 7fe9bae887a0 20 read_log 192834'17870 (192834'17853) delete 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:18806 2014-08-11 15:27:47.167264 2014-08-13 17:38:02.870729 7fe9bae887a0 20 read_log 192834'17879 (192834'17218) clean b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:18815 2014-08-11 15:27:47.308165 2014-08-13 17:38:02.870741 7fe9bae887a0 20 read_log 192834'17890 (0'0) promote 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:18826 2014-08-11 15:27:51.423418 2014-08-13 17:38:02.870753 7fe9bae887a0 20 read_log 192834'17891 (192834'17890) modify 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by client.1883634.0:19346 2014-08-11 15:27:51.414708 2014-08-13 17:38:02.870765 7fe9bae887a0 20 read_log 192834'17898 (192834'17879) delete b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:18834 2014-08-11 15:27:52.884975 2014-08-13 17:38:02.870777 7fe9bae887a0 20 read_log 192834'17913 (192834'17228) clean 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:18849 2014-08-11 15:27:53.105535 2014-08-13 17:38:02.870789 7fe9bae887a0 20 read_log 192834'17917 (192834'17913) delete 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:18853 2014-08-11 15:27:53.318603 2014-08-13 17:38:02.870801 7fe9bae887a0 20 read_log 192834'17927 (0'0) promote 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:18863 2014-08-11 15:27:54.229847 2014-08-13 17:38:02.870812 7fe9bae887a0 20 read_log 192834'17928 (192834'17927) modify 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by client.1883634.0:19815 2014-08-11 15:27:54.220537 2014-08-13 17:38:02.870825 7fe9bae887a0 20 read_log 192834'17941 (192834'17293) clean 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18877 2014-08-11 15:28:00.536037 2014-08-13 17:38:02.870837 7fe9bae887a0 20 read_log 192834'17942 (192834'17347) clean d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18878 2014-08-11 15:28:00.760494 2014-08-13 17:38:02.870848 7fe9bae887a0 20 read_log 192834'17947 (192834'17291) clean 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18883 2014-08-11 15:28:00.888031 2014-08-13 17:38:02.870860 7fe9bae887a0 20 read_log 192834'17955 (192834'17941) delete 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18891 2014-08-11 15:28:00.930925 2014-08-13 17:38:02.870872 7fe9bae887a0 20 read_log 192834'17956 (192834'17942) delete d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18892 2014-08-11 15:28:00.934581 2014-08-13 17:38:02.870884 7fe9bae887a0 20 read_log 192834'17963 (192834'17947) delete 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18899 2014-08-11 15:28:00.958447 2014-08-13 17:38:02.870896 7fe9bae887a0 20 read_log 192834'17974 (0'0) promote 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:18910 2014-08-11 15:28:01.506362 2014-08-13 17:38:02.870917 7fe9bae887a0 20 read_log 192834'17975 (192834'17974) modify 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by client.1883634.0:20787 2014-08-11 15:28:01.498456 2014-08-13 17:38:02.870930 7fe9bae887a0 20 read_log 192834'17988 (0'0) promote 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:18924 2014-08-11 15:28:05.402856 2014-08-13 17:38:02.870942 7fe9bae887a0 20 read_log 192834'17989 (192834'17988) modify 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by client.1883634.0:21825 2014-08-11 15:28:05.395611 2014-08-13 17:38:02.870954 7fe9bae887a0 20 read_log 192834'18007 (192834'17351) clean dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18943 2014-08-11 15:28:06.592044 2014-08-13 17:38:02.870966 7fe9bae887a0 20 read_log 192834'18023 (192834'18007) delete dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18959 2014-08-11 15:28:11.679808 2014-08-13 17:38:02.870978 7fe9bae887a0 20 read_log 192834'18026 (192834'17650) clean b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18962 2014-08-11 15:28:11.785582 2014-08-13 17:38:02.870990 7fe9bae887a0 20 read_log 192834'18033 (192834'18026) delete b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18969 2014-08-11 15:28:11.829757 2014-08-13 17:38:02.871002 7fe9bae887a0 20 read_log 192834'18046 (192834'17670) clean f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18982 2014-08-11 15:28:17.147079 2014-08-13 17:38:02.871014 7fe9bae887a0 20 read_log 192834'18060 (192834'18046) delete f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18996 2014-08-11 15:28:17.476690 2014-08-13 17:38:02.871026 7fe9bae887a0 20 read_log 192834'18069 (0'0) promote 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19005 2014-08-11 15:28:17.535118 2014-08-13 17:38:02.871038 7fe9bae887a0 20 read_log 192834'18071 (192834'18069) modify 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by client.1883634.0:23683 2014-08-11 15:28:17.443004 2014-08-13 17:38:02.871050 7fe9bae887a0 20 read_log 192834'18118 (192834'17712) clean f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19054 2014-08-11 15:28:22.809678 2014-08-13 17:38:02.871062 7fe9bae887a0 20 read_log 192834'18123 (0'0) promote d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19059 2014-08-11 15:28:23.248839 2014-08-13 17:38:02.871074 7fe9bae887a0 20 read_log 192834'18124 (192834'18123) modify d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by client.1883634.0:26520 2014-08-11 15:28:23.209939 2014-08-13 17:38:02.871086 7fe9bae887a0 20 read_log 192834'18125 (0'0) promote 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19061 2014-08-11 15:28:23.367455 2014-08-13 17:38:02.871098 7fe9bae887a0 20 read_log 192834'18126 (192834'18125) modify 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by client.1883634.0:26627 2014-08-11 15:28:23.340438 2014-08-13 17:38:02.871110 7fe9bae887a0 20 read_log 192834'18129 (0'0) promote 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19065 2014-08-11 15:28:23.604963 2014-08-13 17:38:02.871124 7fe9bae887a0 20 read_log 192834'18130 (192834'18129) modify 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by client.1883634.0:26777 2014-08-11 15:28:23.569751 2014-08-13 17:38:02.871137 7fe9bae887a0 20 read_log 192834'18139 (0'0) promote e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19075 2014-08-11 15:28:24.861779 2014-08-13 17:38:02.871149 7fe9bae887a0 20 read_log 192834'18140 (192834'18139) modify e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by client.1883634.0:27699 2014-08-11 15:28:24.836221 2014-08-13 17:38:02.871174 7fe9bae887a0 20 read_log 192834'18145 (0'0) promote 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19081 2014-08-11 15:28:25.498048 2014-08-13 17:38:02.871188 7fe9bae887a0 20 read_log 192834'18146 (192834'18145) modify 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by client.1883634.0:28174 2014-08-11 15:28:25.470648 2014-08-13 17:38:02.871201 7fe9bae887a0 20 read_log 192834'18151 (0'0) promote ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19087 2014-08-11 15:28:25.663805 2014-08-13 17:38:02.871213 7fe9bae887a0 20 read_log 192834'18152 (192834'18151) modify ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by client.1883634.0:28310 2014-08-11 15:28:25.632503 2014-08-13 17:38:02.871225 7fe9bae887a0 20 read_log 192834'18189 (0'0) promote 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19125 2014-08-11 15:28:27.540624 2014-08-13 17:38:02.871237 7fe9bae887a0 20 read_log 192834'18190 (192834'18189) modify 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by client.1883634.0:29615 2014-08-11 15:28:27.506757 2014-08-13 17:38:02.871249 7fe9bae887a0 20 read_log 192834'18195 (192834'18118) delete f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19131 2014-08-11 15:28:27.940960 2014-08-13 17:38:02.871260 7fe9bae887a0 20 read_log 192834'18196 (192834'17745) clean 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19132 2014-08-11 15:28:27.949986 2014-08-13 17:38:02.871272 7fe9bae887a0 20 read_log 192834'18200 (192834'18196) delete 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19136 2014-08-11 15:28:28.001102 2014-08-13 17:38:02.871285 7fe9bae887a0 20 read_log 192834'18202 (192834'17749) clean 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19138 2014-08-11 15:28:28.052582 2014-08-13 17:38:02.871297 7fe9bae887a0 20 read_log 192834'18210 (0'0) promote 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19146 2014-08-11 15:28:28.293825 2014-08-13 17:38:02.871309 7fe9bae887a0 20 read_log 192834'18211 (192834'18210) modify 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by client.1883634.0:30232 2014-08-11 15:28:28.263090 2014-08-13 17:38:02.871321 7fe9bae887a0 20 read_log 192834'18226 (0'0) promote b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19162 2014-08-11 15:28:31.174202 2014-08-13 17:38:02.871333 7fe9bae887a0 20 read_log 192834'18227 (192834'18226) modify b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by client.1883634.0:31932 2014-08-11 15:28:31.146398 2014-08-13 17:38:02.871345 7fe9bae887a0 20 read_log 192834'18236 (0'0) promote 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19172 2014-08-11 15:28:31.467115 2014-08-13 17:38:02.871357 7fe9bae887a0 20 read_log 192834'18237 (192834'18236) modify 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by client.1883634.0:32161 2014-08-11 15:28:31.430490 2014-08-13 17:38:02.871369 7fe9bae887a0 20 read_log 192834'18258 (192834'17761) clean 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19194 2014-08-11 15:28:33.203353 2014-08-13 17:38:02.871381 7fe9bae887a0 20 read_log 192834'18260 (192834'18202) delete 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19196 2014-08-11 15:28:33.217425 2014-08-13 17:38:02.871393 7fe9bae887a0 20 read_log 192834'18264 (192834'18258) delete 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19200 2014-08-11 15:28:33.265469 2014-08-13 17:38:02.871405 7fe9bae887a0 20 read_log 192834'18265 (192834'17755) clean 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19201 2014-08-11 15:28:33.336936 2014-08-13 17:38:02.871442 7fe9bae887a0 20 read_log 192834'18279 (0'0) promote 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19215 2014-08-11 15:28:34.142627 2014-08-13 17:38:02.871458 7fe9bae887a0 20 read_log 192834'18280 (192834'18279) modify 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by client.1883634.0:33997 2014-08-11 15:28:34.104537 2014-08-13 17:38:02.871470 7fe9bae887a0 20 read_log 192834'18281 (0'0) promote 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19217 2014-08-11 15:28:34.370684 2014-08-13 17:38:02.871482 7fe9bae887a0 20 read_log 192834'18282 (192834'18281) modify 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by client.1883634.0:34025 2014-08-11 15:28:34.138651 2014-08-13 17:38:02.871494 7fe9bae887a0 20 read_log 192834'18293 (0'0) promote d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19229 2014-08-11 15:28:35.321339 2014-08-13 17:38:02.871508 7fe9bae887a0 20 read_log 192834'18294 (192834'18293) modify d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by client.1883634.0:34778 2014-08-11 15:28:35.287194 2014-08-13 17:38:02.871520 7fe9bae887a0 20 read_log 192834'18297 (0'0) promote dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19233 2014-08-11 15:28:36.896926 2014-08-13 17:38:02.871532 7fe9bae887a0 20 read_log 192834'18298 (192834'18297) modify dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by client.1883634.0:35575 2014-08-11 15:28:36.863215 2014-08-13 17:38:02.871544 7fe9bae887a0 20 read_log 192834'18322 (192834'18265) delete 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19258 2014-08-11 15:28:38.481727 2014-08-13 17:38:02.871556 7fe9bae887a0 20 read_log 192834'18328 (0'0) promote b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19264 2014-08-11 15:28:38.681633 2014-08-13 17:38:02.871568 7fe9bae887a0 20 read_log 192834'18329 (192834'18328) modify b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by client.1883634.0:36935 2014-08-11 15:28:38.610177 2014-08-13 17:38:02.871584 7fe9bae887a0 20 read_log 192834'18348 (0'0) promote f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19284 2014-08-11 15:28:40.160628 2014-08-13 17:38:02.871596 7fe9bae887a0 20 read_log 192834'18349 (192834'18348) modify f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by client.1883634.0:38034 2014-08-11 15:28:40.127642 2014-08-13 17:38:02.871608 7fe9bae887a0 20 read_log 192834'18372 (0'0) promote f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19308 2014-08-11 15:28:41.974147 2014-08-13 17:38:02.871620 7fe9bae887a0 20 read_log 192834'18373 (192834'18372) modify f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by client.1883634.0:39215 2014-08-11 15:28:41.772740 2014-08-13 17:38:02.871632 7fe9bae887a0 20 read_log 192834'18378 (0'0) promote 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19314 2014-08-11 15:28:42.341059 2014-08-13 17:38:02.871644 7fe9bae887a0 20 read_log 192834'18379 (192834'18378) modify 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by client.1883634.0:39565 2014-08-11 15:28:42.308984 2014-08-13 17:38:02.871657 7fe9bae887a0 20 read_log 192834'18382 (0'0) promote 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19318 2014-08-11 15:28:42.495623 2014-08-13 17:38:02.871670 7fe9bae887a0 20 read_log 192834'18383 (192834'18382) modify 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by client.1883634.0:39707 2014-08-11 15:28:42.458028 2014-08-13 17:38:02.871682 7fe9bae887a0 20 read_log 192834'18388 (0'0) promote 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19324 2014-08-11 15:28:42.864371 2014-08-13 17:38:02.871701 7fe9bae887a0 20 read_log 192834'18389 (192834'18388) modify 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by client.1883634.0:39936 2014-08-11 15:28:42.838307 2014-08-13 17:38:02.871714 7fe9bae887a0 20 read_log 192834'18394 (0'0) promote 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19330 2014-08-11 15:28:43.552174 2014-08-13 17:38:02.871726 7fe9bae887a0 20 read_log 192834'18395 (192834'18394) modify 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by client.1883634.0:40318 2014-08-11 15:28:43.519407 2014-08-13 17:38:02.871738 7fe9bae887a0 20 read_log 192834'18404 (192834'17826) clean 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19340 2014-08-11 15:28:43.849176 2014-08-13 17:38:02.871750 7fe9bae887a0 20 read_log 192834'18408 (192834'18404) delete 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19344 2014-08-11 15:28:43.936807 2014-08-13 17:38:02.871762 7fe9bae887a0 20 read_log 192834'18419 (0'0) promote 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19355 2014-08-11 15:28:44.706532 2014-08-13 17:38:02.871791 7fe9bae887a0 20 read_log 192834'18420 (192834'18419) modify 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by client.1883634.0:41149 2014-08-11 15:28:44.680902 2014-08-13 17:38:02.871807 7fe9bae887a0 20 read_log 192834'18437 (192834'17891) modify 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by client.1883634.0:42670 2014-08-11 15:28:46.317733 2014-08-13 17:38:02.871819 7fe9bae887a0 20 read_log 192834'18443 (192834'17928) modify 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by client.1883634.0:43139 2014-08-11 15:28:46.465951 2014-08-13 17:38:02.871831 7fe9bae887a0 20 read_log 192834'18449 (192834'17975) modify 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by client.1883634.0:44111 2014-08-11 15:28:46.765006 2014-08-13 17:38:02.871843 7fe9bae887a0 20 read_log 192834'18456 (192834'17989) modify 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by client.1883634.0:45149 2014-08-11 15:28:47.174714 2014-08-13 17:38:02.871855 7fe9bae887a0 20 read_log 192834'18486 (192834'18071) clean 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19422 2014-08-11 15:29:19.742758 2014-08-13 17:38:02.871867 7fe9bae887a0 20 read_log 192834'18492 (192834'18486) delete 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19428 2014-08-11 15:29:19.780150 2014-08-13 17:38:02.871879 7fe9bae887a0 20 read_log 192834'18503 (192834'18126) clean 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19439 2014-08-11 15:29:24.942869 2014-08-13 17:38:02.871891 7fe9bae887a0 20 read_log 192834'18506 (192834'18140) clean e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19442 2014-08-11 15:29:24.955659 2014-08-13 17:38:02.871903 7fe9bae887a0 20 read_log 192834'18507 (192834'18124) clean d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19443 2014-08-11 15:29:24.959602 2014-08-13 17:38:02.871915 7fe9bae887a0 20 read_log 192834'18508 (192834'18130) clean 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19444 2014-08-11 15:29:24.968301 2014-08-13 17:38:02.871927 7fe9bae887a0 20 read_log 192834'18522 (192834'18503) delete 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19458 2014-08-11 15:29:25.043841 2014-08-13 17:38:02.871939 7fe9bae887a0 20 read_log 192834'18525 (192834'18506) delete e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19461 2014-08-11 15:29:25.058735 2014-08-13 17:38:02.871951 7fe9bae887a0 20 read_log 192834'18526 (192834'18507) delete d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19462 2014-08-11 15:29:25.059094 2014-08-13 17:38:02.871970 7fe9bae887a0 20 read_log 192834'18527 (192834'18508) delete 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19463 2014-08-11 15:29:25.062848 2014-08-13 17:38:02.871985 7fe9bae887a0 20 read_log 192834'18547 (192834'18152) clean ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19483 2014-08-11 15:29:30.301602 2014-08-13 17:38:02.871997 7fe9bae887a0 20 read_log 192834'18548 (192834'18146) clean 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19484 2014-08-11 15:29:30.304081 2014-08-13 17:38:02.872009 7fe9bae887a0 20 read_log 192834'18557 (192834'18190) clean 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19493 2014-08-11 15:29:30.350732 2014-08-13 17:38:02.872021 7fe9bae887a0 20 read_log 192834'18575 (192834'18547) delete ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19511 2014-08-11 15:29:30.446223 2014-08-13 17:38:02.872034 7fe9bae887a0 20 read_log 192834'18576 (192834'18548) delete 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19512 2014-08-11 15:29:30.450294 2014-08-13 17:38:02.872046 7fe9bae887a0 20 read_log 192834'18578 (192834'18211) clean 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19514 2014-08-11 15:29:30.454205 2014-08-13 17:38:02.872058 7fe9bae887a0 20 read_log 192834'18586 (192834'18557) delete 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19522 2014-08-11 15:29:30.488665 2014-08-13 17:38:02.872070 7fe9bae887a0 20 read_log 192834'18590 (192834'18578) delete 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19526 2014-08-11 15:29:30.502121 2014-08-13 17:38:02.872082 7fe9bae887a0 20 read_log 192834'18609 (192834'18227) clean b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19545 2014-08-11 15:29:35.635765 2014-08-13 17:38:02.872094 7fe9bae887a0 20 read_log 192834'18610 (192834'18282) clean 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19546 2014-08-11 15:29:35.641672 2014-08-13 17:38:02.872106 7fe9bae887a0 20 read_log 192834'18611 (192834'18294) clean d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19547 2014-08-11 15:29:35.643307 2014-08-13 17:38:02.872118 7fe9bae887a0 20 read_log 192834'18618 (192834'18237) clean 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19554 2014-08-11 15:29:35.669634 2014-08-13 17:38:02.872130 7fe9bae887a0 20 read_log 192834'18619 (192834'18280) clean 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19555 2014-08-11 15:29:35.675810 2014-08-13 17:38:02.872142 7fe9bae887a0 20 read_log 192834'18642 (192834'18609) delete b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19578 2014-08-11 15:29:35.740101 2014-08-13 17:38:02.872160 7fe9bae887a0 20 read_log 192834'18643 (192834'18610) delete 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19579 2014-08-11 15:29:35.740471 2014-08-13 17:38:02.872174 7fe9bae887a0 20 read_log 192834'18644 (192834'18611) delete d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19580 2014-08-11 15:29:35.743664 2014-08-13 17:38:02.872186 7fe9bae887a0 20 read_log 192834'18651 (192834'18618) delete 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19587 2014-08-11 15:29:35.755881 2014-08-13 17:38:02.872198 7fe9bae887a0 20 read_log 192834'18652 (192834'18619) delete 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19588 2014-08-11 15:29:35.758390 2014-08-13 17:38:02.872210 7fe9bae887a0 20 read_log 192834'18675 (192834'18349) clean f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19611 2014-08-11 15:29:40.869422 2014-08-13 17:38:02.872231 7fe9bae887a0 20 read_log 192834'18680 (192834'18329) clean b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19616 2014-08-11 15:29:40.885978 2014-08-13 17:38:02.872244 7fe9bae887a0 20 read_log 192834'18681 (192834'18298) clean dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19617 2014-08-11 15:29:40.888777 2014-08-13 17:38:02.872256 7fe9bae887a0 20 read_log 192834'18704 (192834'18675) delete f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19640 2014-08-11 15:29:40.950666 2014-08-13 17:38:02.872268 7fe9bae887a0 20 read_log 192834'18709 (192834'18680) delete b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19645 2014-08-11 15:29:40.963561 2014-08-13 17:38:02.872280 7fe9bae887a0 20 read_log 192834'18710 (192834'18681) delete dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19646 2014-08-11 15:29:40.965832 2014-08-13 17:38:02.872292 7fe9bae887a0 20 read_log 192834'18736 (192834'18395) clean 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19672 2014-08-11 15:29:46.062186 2014-08-13 17:38:02.872304 7fe9bae887a0 20 read_log 192834'18737 (192834'18379) clean 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19673 2014-08-11 15:29:46.062942 2014-08-13 17:38:02.872318 7fe9bae887a0 20 read_log 192834'18738 (192834'18389) clean 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19674 2014-08-11 15:29:46.063783 2014-08-13 17:38:02.872330 7fe9bae887a0 20 read_log 192834'18739 (192834'18373) clean f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19675 2014-08-11 15:29:46.069817 2014-08-13 17:38:02.872342 7fe9bae887a0 20 read_log 192834'18744 (192834'18420) clean 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19680 2014-08-11 15:29:46.087995 2014-08-13 17:38:02.872354 7fe9bae887a0 20 read_log 192834'18745 (192834'18383) clean 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19681 2014-08-11 15:29:46.089234 2014-08-13 17:38:02.872366 7fe9bae887a0 20 read_log 192834'18766 (192834'18736) delete 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19702 2014-08-11 15:29:46.116656 2014-08-13 17:38:02.872378 7fe9bae887a0 20 read_log 192834'18767 (192834'18737) delete 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19703 2014-08-11 15:29:46.118608 2014-08-13 17:38:02.872390 7fe9bae887a0 20 read_log 192834'18768 (192834'18738) delete 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19704 2014-08-11 15:29:46.118910 2014-08-13 17:38:02.872402 7fe9bae887a0 20 read_log 192834'18769 (192834'18739) delete f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19705 2014-08-11 15:29:46.119858 2014-08-13 17:38:02.872414 7fe9bae887a0 20 read_log 192834'18774 (192834'18744) delete 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19710 2014-08-11 15:29:46.124957 2014-08-13 17:38:02.872425 7fe9bae887a0 20 read_log 192834'18775 (192834'18745) delete 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19711 2014-08-11 15:29:46.125486 2014-08-13 17:38:02.872437 7fe9bae887a0 20 read_log 192834'18797 (192834'18437) clean 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:19733 2014-08-11 15:29:51.177950 2014-08-13 17:38:02.872449 7fe9bae887a0 20 read_log 192834'18815 (192834'18797) delete 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:19751 2014-08-11 15:29:51.198713 2014-08-13 17:38:02.872461 7fe9bae887a0 20 read_log 192834'18819 (192834'18443) clean 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:19755 2014-08-11 15:29:51.203207 2014-08-13 17:38:02.872480 7fe9bae887a0 20 read_log 192834'18820 (192834'18456) clean 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:19756 2014-08-11 15:29:51.203548 2014-08-13 17:38:02.872494 7fe9bae887a0 20 read_log 192834'18823 (192834'18449) clean 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:19759 2014-08-11 15:29:51.211103 2014-08-13 17:38:02.872506 7fe9bae887a0 20 read_log 192834'18827 (192834'18819) delete 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:19763 2014-08-11 15:29:51.216504 2014-08-13 17:38:02.872518 7fe9bae887a0 20 read_log 192834'18828 (192834'18820) delete 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:19764 2014-08-11 15:29:51.217678 2014-08-13 17:38:02.872529 7fe9bae887a0 20 read_log 192834'18831 (192834'18823) delete 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:19767 2014-08-11 15:29:51.220434 2014-08-13 17:38:02.872542 7fe9bae887a0 20 read_log 192888'18852 (192888'18852) modify 181/hit_set_24.181_archive_2014-08-11 16:02:27.752096_2014-08-11 16:44:54.931211/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:54.931521 2014-08-13 17:38:02.872554 7fe9bae887a0 20 read_log 192888'18853 (0'0) promote 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by osd.4.0:21378 2014-08-11 16:44:54.937105 2014-08-13 17:38:02.872566 7fe9bae887a0 20 read_log 192888'18854 (192888'18853) modify 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.1315259.0:22 2014-08-11 16:44:54.922103 2014-08-13 17:38:02.872579 7fe9bae887a0 20 read_log 192888'18855 (0'0) promote 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by osd.4.0:21391 2014-08-11 16:44:57.877717 2014-08-13 17:38:02.872590 7fe9bae887a0 20 read_log 192888'18856 (192888'18855) modify 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.1315259.0:682 2014-08-11 16:44:57.867837 2014-08-13 17:38:02.872602 7fe9bae887a0 20 read_log 192888'18859 (0'0) promote 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by osd.4.0:21409 2014-08-11 16:44:59.717900 2014-08-13 17:38:02.872614 7fe9bae887a0 20 read_log 192888'18860 (192888'18859) modify 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.1315259.0:1178 2014-08-11 16:44:59.711004 2014-08-13 17:38:02.872626 7fe9bae887a0 20 read_log 192888'18865 (0'0) promote 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by osd.4.0:21448 2014-08-11 16:45:03.750369 2014-08-13 17:38:02.872638 7fe9bae887a0 20 read_log 192888'18866 (192888'18865) modify 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.1315259.0:2238 2014-08-11 16:45:03.742783 2014-08-13 17:38:02.872650 7fe9bae887a0 20 read_log 192888'18867 (0'0) promote 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by osd.4.0:21462 2014-08-11 16:45:05.394582 2014-08-13 17:38:02.872662 7fe9bae887a0 20 read_log 192888'18868 (192888'18867) modify 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.1315259.0:2679 2014-08-11 16:45:05.387181 2014-08-13 17:38:02.872675 7fe9bae887a0 20 read_log 192888'18871 (0'0) promote 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by osd.4.0:21492 2014-08-11 16:45:10.870313 2014-08-13 17:38:02.872687 7fe9bae887a0 20 read_log 192888'18872 (192888'18871) modify 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.1315259.0:4112 2014-08-11 16:45:10.861190 2014-08-13 17:38:02.872699 7fe9bae887a0 20 read_log 192888'18873 (0'0) promote 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by osd.4.0:21514 2014-08-11 16:45:13.039211 2014-08-13 17:38:02.872718 7fe9bae887a0 20 read_log 192888'18874 (192888'18873) modify 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.1315259.0:4709 2014-08-11 16:45:13.029352 2014-08-13 17:38:02.872730 7fe9bae887a0 20 read_log 192888'18877 (0'0) promote b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by osd.4.0:21540 2014-08-11 16:45:15.437137 2014-08-13 17:38:02.872742 7fe9bae887a0 20 read_log 192888'18878 (192888'18877) modify b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.1315259.0:5353 2014-08-11 16:45:15.429633 2014-08-13 17:38:02.872754 7fe9bae887a0 20 read_log 192888'18879 (0'0) promote 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by osd.4.0:21552 2014-08-11 16:45:16.761594 2014-08-13 17:38:02.872766 7fe9bae887a0 20 read_log 192888'18880 (192888'18879) modify 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.1315259.0:5717 2014-08-11 16:45:16.754026 2014-08-13 17:38:02.872779 7fe9bae887a0 20 read_log 192888'18881 (0'0) promote 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by osd.4.0:21564 2014-08-11 16:45:18.427841 2014-08-13 17:38:02.872791 7fe9bae887a0 20 read_log 192888'18882 (192888'18881) modify 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.1315259.0:6178 2014-08-11 16:45:18.418877 2014-08-13 17:38:02.872803 7fe9bae887a0 20 read_log 192888'18893 (0'0) promote 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by osd.4.0:21668 2014-08-11 16:45:34.518316 2014-08-13 17:38:02.872815 7fe9bae887a0 20 read_log 192888'18894 (192888'18893) modify 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.1315259.0:10476 2014-08-11 16:45:34.507904 2014-08-13 17:38:02.872827 7fe9bae887a0 20 read_log 192888'18897 (0'0) promote e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by osd.4.0:21710 2014-08-11 16:45:40.073139 2014-08-13 17:38:02.872839 7fe9bae887a0 20 read_log 192888'18898 (192888'18897) modify e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.1315259.0:11844 2014-08-11 16:45:40.064915 2014-08-13 17:38:02.872851 7fe9bae887a0 20 read_log 192888'18901 (0'0) promote 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by osd.4.0:21744 2014-08-11 16:45:45.846724 2014-08-13 17:38:02.872863 7fe9bae887a0 20 read_log 192888'18902 (192888'18901) modify 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.1315259.0:13250 2014-08-11 16:45:45.837072 2014-08-13 17:38:02.872874 7fe9bae887a0 20 read_log 192888'18903 (0'0) promote 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by osd.4.0:21768 2014-08-11 16:45:50.726893 2014-08-13 17:38:02.872886 7fe9bae887a0 20 read_log 192888'18904 (192888'18903) modify 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.1315259.0:14508 2014-08-11 16:45:50.717693 2014-08-13 17:38:02.872898 7fe9bae887a0 20 read_log 192888'18907 (0'0) promote 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by osd.4.0:21838 2014-08-11 16:45:59.297974 2014-08-13 17:38:02.872910 7fe9bae887a0 20 read_log 192888'18908 (192888'18907) modify 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.1315259.0:16735 2014-08-11 16:45:59.289249 2014-08-13 17:38:02.872922 7fe9bae887a0 20 read_log 192888'18909 (0'0) promote f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by osd.4.0:21854 2014-08-11 16:46:01.192888 2014-08-13 17:38:02.872933 7fe9bae887a0 20 read_log 192888'18910 (192888'18909) modify f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.1315259.0:17219 2014-08-11 16:46:01.183945 2014-08-13 17:38:02.872945 7fe9bae887a0 20 read_log 192888'18913 (0'0) promote f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by osd.4.0:21864 2014-08-11 16:46:02.793027 2014-08-13 17:38:02.872982 7fe9bae887a0 20 read_log 192888'18914 (192888'18913) modify f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.1315259.0:17603 2014-08-11 16:46:02.781789 2014-08-13 17:38:02.872998 7fe9bae887a0 20 read_log 192888'18915 (0'0) promote 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by osd.4.0:21870 2014-08-11 16:46:03.854827 2014-08-13 17:38:02.873010 7fe9bae887a0 20 read_log 192888'18916 (192888'18915) modify 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.1315259.0:17869 2014-08-11 16:46:03.845744 2014-08-13 17:38:02.873022 7fe9bae887a0 20 read_log 192888'18917 (0'0) promote e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by osd.4.0:21886 2014-08-11 16:46:06.470385 2014-08-13 17:38:02.873034 7fe9bae887a0 20 read_log 192888'18918 (192888'18917) modify e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.1315259.0:18550 2014-08-11 16:46:06.462494 2014-08-13 17:38:02.873046 7fe9bae887a0 20 read_log 192888'18919 (0'0) promote 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by osd.4.0:21890 2014-08-11 16:46:06.607946 2014-08-13 17:38:02.873058 7fe9bae887a0 20 read_log 192888'18920 (192888'18919) modify 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.1315259.0:18585 2014-08-11 16:46:06.600866 2014-08-13 17:38:02.873069 7fe9bae887a0 20 read_log 192888'18921 (0'0) promote ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by osd.4.0:21924 2014-08-11 16:46:10.797472 2014-08-13 17:38:02.873081 7fe9bae887a0 20 read_log 192888'18922 (192888'18921) modify ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.1315259.0:19639 2014-08-11 16:46:10.789479 2014-08-13 17:38:02.873093 7fe9bae887a0 20 read_log 192888'18925 (0'0) promote 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by osd.4.0:21956 2014-08-11 16:46:15.730649 2014-08-13 17:38:02.873105 7fe9bae887a0 20 read_log 192888'18926 (192888'18925) modify 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.1315259.0:20960 2014-08-11 16:46:15.722267 2014-08-13 17:38:02.873117 7fe9bae887a0 20 read_log 192888'18929 (0'0) promote 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by osd.4.0:21970 2014-08-11 16:46:17.118564 2014-08-13 17:38:02.873129 7fe9bae887a0 20 read_log 192888'18930 (192888'18929) modify 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.1315259.0:21333 2014-08-11 16:46:17.110201 2014-08-13 17:38:02.873140 7fe9bae887a0 20 read_log 192888'18935 (0'0) promote d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by osd.4.0:22004 2014-08-11 16:46:21.855538 2014-08-13 17:38:02.873152 7fe9bae887a0 20 read_log 192888'18936 (192888'18935) modify d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.1315259.0:22548 2014-08-11 16:46:21.846745 2014-08-13 17:38:02.873170 7fe9bae887a0 20 read_log 192888'18937 (0'0) promote 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by osd.4.0:22024 2014-08-11 16:46:26.140211 2014-08-13 17:38:02.873182 7fe9bae887a0 20 read_log 192888'18938 (192888'18937) modify 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.1315259.0:23601 2014-08-11 16:46:26.130444 2014-08-13 17:38:02.873194 7fe9bae887a0 20 read_log 192888'18939 (0'0) promote 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by osd.4.0:22028 2014-08-11 16:46:26.643186 2014-08-13 17:38:02.873206 7fe9bae887a0 20 read_log 192888'18940 (192888'18939) modify 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.1315259.0:23736 2014-08-11 16:46:26.632987 2014-08-13 17:38:02.873218 7fe9bae887a0 20 read_log 192888'18943 (0'0) promote 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by osd.4.0:22040 2014-08-11 16:46:28.051821 2014-08-13 17:38:02.873238 7fe9bae887a0 20 read_log 192888'18944 (192888'18943) modify 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.1315259.0:24107 2014-08-11 16:46:28.044596 2014-08-13 17:38:02.873251 7fe9bae887a0 20 read_log 192888'18945 (0'0) promote 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by osd.4.0:22054 2014-08-11 16:46:29.204523 2014-08-13 17:38:02.873263 7fe9bae887a0 20 read_log 192888'18946 (192888'18945) modify 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.1315259.0:24417 2014-08-11 16:46:29.195317 2014-08-13 17:38:02.873276 7fe9bae887a0 20 read_log 192888'18949 (0'0) promote b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by osd.4.0:22080 2014-08-11 16:46:32.005925 2014-08-13 17:38:02.873289 7fe9bae887a0 20 read_log 192888'18950 (192888'18949) modify b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.1315259.0:25130 2014-08-11 16:46:31.996994 2014-08-13 17:38:02.873301 7fe9bae887a0 20 read_log 192888'18953 (0'0) promote 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by osd.4.0:22086 2014-08-11 16:46:32.539382 2014-08-13 17:38:02.873313 7fe9bae887a0 20 read_log 192888'18954 (192888'18953) modify 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.1315259.0:25254 2014-08-11 16:46:32.530900 2014-08-13 17:38:02.873325 7fe9bae887a0 20 read_log 192888'18955 (0'0) promote f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by osd.4.0:22118 2014-08-11 16:46:37.821290 2014-08-13 17:38:02.873337 7fe9bae887a0 20 read_log 192888'18956 (192888'18955) modify f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.1315259.0:26482 2014-08-11 16:46:37.813093 2014-08-13 17:38:02.873349 7fe9bae887a0 20 read_log 192888'18957 (0'0) promote 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by osd.4.0:22126 2014-08-11 16:46:38.452089 2014-08-13 17:38:02.873360 7fe9bae887a0 20 read_log 192888'18958 (192888'18957) modify 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.1315259.0:26643 2014-08-11 16:46:38.442666 2014-08-13 17:38:02.873372 7fe9bae887a0 20 read_log 192888'18959 (0'0) promote 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by osd.4.0:22140 2014-08-11 16:46:41.469562 2014-08-13 17:38:02.873384 7fe9bae887a0 20 read_log 192888'18960 (192888'18959) modify 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.1315259.0:27361 2014-08-11 16:46:41.462625 2014-08-13 17:38:02.873396 7fe9bae887a0 20 read_log 192888'18971 (192888'18854) modify 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.1315259.0:30749 2014-08-11 16:46:55.876395 2014-08-13 17:38:02.873408 7fe9bae887a0 20 read_log 192888'18972 (192888'18856) modify 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.1315259.0:31409 2014-08-11 16:46:56.084993 2014-08-13 17:38:02.873420 7fe9bae887a0 20 read_log 192888'18974 (192888'18860) modify 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.1315259.0:31905 2014-08-11 16:46:56.232884 2014-08-13 17:38:02.873432 7fe9bae887a0 20 read_log 192888'18977 (192888'18866) modify 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.1315259.0:32965 2014-08-11 16:46:56.519248 2014-08-13 17:38:02.873444 7fe9bae887a0 20 read_log 192888'18978 (192888'18868) modify 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.1315259.0:33406 2014-08-11 16:46:56.628896 2014-08-13 17:38:02.873456 7fe9bae887a0 20 read_log 192888'18980 (192888'18872) modify 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.1315259.0:34839 2014-08-11 16:46:56.998787 2014-08-13 17:38:02.873468 7fe9bae887a0 20 read_log 192888'18981 (192888'18874) modify 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.1315259.0:35436 2014-08-11 16:46:57.163755 2014-08-13 17:38:02.873487 7fe9bae887a0 20 read_log 192888'18983 (192888'18878) modify b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.1315259.0:36080 2014-08-11 16:46:57.330117 2014-08-13 17:38:02.873524 7fe9bae887a0 20 read_log 192888'18984 (192888'18880) modify 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.1315259.0:36444 2014-08-11 16:46:57.424640 2014-08-13 17:38:02.873537 7fe9bae887a0 20 read_log 192888'18985 (192888'18882) modify 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.1315259.0:36905 2014-08-11 16:46:57.542387 2014-08-13 17:38:02.873549 7fe9bae887a0 20 read_log 192888'18991 (192888'18894) modify 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.1315259.0:41203 2014-08-11 16:46:58.686202 2014-08-13 17:38:02.873561 7fe9bae887a0 20 read_log 192888'18993 (192888'18898) modify e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.1315259.0:42571 2014-08-11 16:46:59.046820 2014-08-13 17:38:02.873574 7fe9bae887a0 20 read_log 192888'18995 (192888'18902) modify 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.1315259.0:43977 2014-08-11 16:46:59.446683 2014-08-13 17:38:02.873586 7fe9bae887a0 20 read_log 192888'18996 (192888'18904) modify 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.1315259.0:45235 2014-08-11 16:46:59.808334 2014-08-13 17:38:02.873598 7fe9bae887a0 20 read_log 192888'18998 (192888'18908) modify 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.1315259.0:47462 2014-08-11 16:47:00.466027 2014-08-13 17:38:02.873610 7fe9bae887a0 20 read_log 192888'18999 (192888'18910) modify f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.1315259.0:47946 2014-08-11 16:47:00.618540 2014-08-13 17:38:02.873622 7fe9bae887a0 20 read_log 192888'19001 (192888'18914) modify f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.1315259.0:48330 2014-08-11 16:47:00.732050 2014-08-13 17:38:02.873634 7fe9bae887a0 20 read_log 192888'19002 (192888'18916) modify 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.1315259.0:48596 2014-08-11 16:47:00.808301 2014-08-13 17:38:02.873648 7fe9bae887a0 20 read_log 192888'19003 (192888'18918) modify e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.1315259.0:49277 2014-08-11 16:47:01.013446 2014-08-13 17:38:02.873661 7fe9bae887a0 20 read_log 192888'19004 (192888'18920) modify 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.1315259.0:49312 2014-08-11 16:47:01.024216 2014-08-13 17:38:02.873675 7fe9bae887a0 20 read_log 192888'19005 (192888'18922) modify ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.1315259.0:50366 2014-08-11 16:47:01.347761 2014-08-13 17:38:02.873687 7fe9bae887a0 20 read_log 192888'19007 (192888'18926) modify 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.1315259.0:51687 2014-08-11 16:47:01.754335 2014-08-13 17:38:02.873718 7fe9bae887a0 20 read_log 192888'19009 (192888'18930) modify 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.1315259.0:52060 2014-08-11 16:47:01.860336 2014-08-13 17:38:02.873730 7fe9bae887a0 20 read_log 192888'19012 (192888'18936) modify d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.1315259.0:53275 2014-08-11 16:47:02.206715 2014-08-13 17:38:02.873745 7fe9bae887a0 20 read_log 192888'19013 (192888'18938) modify 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.1315259.0:54328 2014-08-11 16:47:02.537066 2014-08-13 17:38:02.873757 7fe9bae887a0 20 read_log 192888'19014 (192888'18940) modify 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.1315259.0:54463 2014-08-11 16:47:02.574040 2014-08-13 17:38:02.873771 7fe9bae887a0 20 read_log 192888'19016 (192888'18944) modify 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.1315259.0:54834 2014-08-11 16:47:02.729468 2014-08-13 17:38:02.873794 7fe9bae887a0 20 read_log 192888'19017 (192888'18946) modify 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.1315259.0:55144 2014-08-11 16:47:02.824600 2014-08-13 17:38:02.873808 7fe9bae887a0 20 read_log 192888'19019 (192888'18950) modify b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.1315259.0:55857 2014-08-11 16:47:03.035764 2014-08-13 17:38:02.873821 7fe9bae887a0 20 read_log 192888'19021 (192888'18954) modify 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.1315259.0:55981 2014-08-11 16:47:03.072397 2014-08-13 17:38:02.873835 7fe9bae887a0 20 read_log 192888'19022 (192888'18956) modify f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.1315259.0:57209 2014-08-11 16:47:03.442791 2014-08-13 17:38:02.873847 7fe9bae887a0 20 read_log 192888'19023 (192888'18958) modify 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.1315259.0:57370 2014-08-11 16:47:03.491842 2014-08-13 17:38:02.873860 7fe9bae887a0 20 read_log 192888'19024 (192888'18960) modify 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.1315259.0:58088 2014-08-11 16:47:03.698029 2014-08-13 17:38:02.873874 7fe9bae887a0 20 read_log 192888'19030 (192888'19030) modify 181/hit_set_24.181_archive_2014-08-11 16:44:54.931606_2014-08-11 17:05:14.856635/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.856830 2014-08-13 17:38:02.873887 7fe9bae887a0 20 read_log 192888'19031 (192888'18852) delete 181/hit_set_24.181_archive_2014-08-11 16:02:27.752096_2014-08-11 16:44:54.931211/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.856830 2014-08-13 17:38:02.873901 7fe9bae887a0 20 read_log 192888'19034 (0'0) promote 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by osd.4.0:22653 2014-08-11 17:05:16.000765 2014-08-13 17:38:02.873914 7fe9bae887a0 20 read_log 192888'19035 (192888'19034) modify 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.942868.0:245 2014-08-11 17:05:15.993154 2014-08-13 17:38:02.873928 7fe9bae887a0 20 read_log 192888'19036 (0'0) promote e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by osd.4.0:22655 2014-08-11 17:05:17.182423 2014-08-13 17:38:02.873940 7fe9bae887a0 20 read_log 192888'19037 (192888'19036) modify e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.942868.0:524 2014-08-11 17:05:17.172611 2014-08-13 17:38:02.873973 7fe9bae887a0 20 read_log 192888'19040 (0'0) promote 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by osd.4.0:22681 2014-08-11 17:05:20.290443 2014-08-13 17:38:02.873989 7fe9bae887a0 20 read_log 192888'19041 (192888'19040) modify 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.942868.0:1283 2014-08-11 17:05:20.282570 2014-08-13 17:38:02.874001 7fe9bae887a0 20 read_log 192888'19042 (0'0) promote 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by osd.4.0:22687 2014-08-11 17:05:21.442398 2014-08-13 17:38:02.874015 7fe9bae887a0 20 read_log 192888'19043 (192888'19042) modify 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.942868.0:1564 2014-08-11 17:05:21.434083 2014-08-13 17:38:02.874028 7fe9bae887a0 20 read_log 192888'19046 (0'0) promote dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by osd.4.0:22726 2014-08-11 17:05:27.832090 2014-08-13 17:38:02.874039 7fe9bae887a0 20 read_log 192888'19047 (192888'19046) modify dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.942868.0:3213 2014-08-11 17:05:27.823106 2014-08-13 17:38:02.874053 7fe9bae887a0 20 read_log 192888'19048 (0'0) promote a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by osd.4.0:22742 2014-08-11 17:05:29.289712 2014-08-13 17:38:02.874073 7fe9bae887a0 20 read_log 192888'19049 (192888'19048) modify a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.942868.0:3607 2014-08-11 17:05:29.280803 2014-08-13 17:38:02.874088 7fe9bae887a0 20 read_log 192888'19050 (0'0) promote 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by osd.4.0:22750 2014-08-11 17:05:29.994874 2014-08-13 17:38:02.874100 7fe9bae887a0 20 read_log 192888'19051 (192888'19050) modify 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.942868.0:3798 2014-08-11 17:05:29.986171 2014-08-13 17:38:02.874112 7fe9bae887a0 20 read_log 192888'19060 (0'0) promote 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by osd.4.0:22844 2014-08-11 17:05:41.863144 2014-08-13 17:38:02.874126 7fe9bae887a0 20 read_log 192888'19061 (192888'19060) modify 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.942868.0:6959 2014-08-11 17:05:41.853656 2014-08-13 17:38:02.874138 7fe9bae887a0 20 read_log 192888'19066 (0'0) promote e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by osd.4.0:22862 2014-08-11 17:05:44.915316 2014-08-13 17:38:02.874151 7fe9bae887a0 20 read_log 192888'19067 (192888'19066) modify e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.942868.0:7747 2014-08-11 17:05:44.906210 2014-08-13 17:38:02.874171 7fe9bae887a0 20 read_log 192888'19070 (0'0) promote 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by osd.4.0:22882 2014-08-11 17:05:47.640604 2014-08-13 17:38:02.874183 7fe9bae887a0 20 read_log 192888'19071 (192888'19070) modify 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.942868.0:8502 2014-08-11 17:05:47.632050 2014-08-13 17:38:02.874195 7fe9bae887a0 20 read_log 192888'19072 (0'0) promote 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by osd.4.0:22894 2014-08-11 17:05:49.989404 2014-08-13 17:38:02.874207 7fe9bae887a0 20 read_log 192888'19073 (192888'19072) modify 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.942868.0:9130 2014-08-11 17:05:49.980541 2014-08-13 17:38:02.874220 7fe9bae887a0 20 read_log 192888'19074 (0'0) promote fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by osd.4.0:22904 2014-08-11 17:05:52.544037 2014-08-13 17:38:02.874234 7fe9bae887a0 20 read_log 192888'19075 (192888'19074) modify fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.942868.0:9817 2014-08-11 17:05:52.535766 2014-08-13 17:38:02.874246 7fe9bae887a0 20 read_log 192888'19080 (0'0) promote fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by osd.4.0:22916 2014-08-11 17:05:54.105014 2014-08-13 17:38:02.874262 7fe9bae887a0 20 read_log 192888'19081 (192888'19080) modify fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.942868.0:10249 2014-08-11 17:05:54.095545 2014-08-13 17:38:02.874276 7fe9bae887a0 20 read_log 192888'19088 (0'0) promote 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by osd.4.0:22956 2014-08-11 17:06:00.324855 2014-08-13 17:38:02.874289 7fe9bae887a0 20 read_log 192888'19089 (192888'19088) modify 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.942868.0:11947 2014-08-11 17:06:00.312616 2014-08-13 17:38:02.874301 7fe9bae887a0 20 read_log 192888'19092 (0'0) promote 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by osd.4.0:22982 2014-08-11 17:06:04.329890 2014-08-13 17:38:02.874315 7fe9bae887a0 20 read_log 192888'19093 (192888'19092) modify 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.942868.0:13038 2014-08-11 17:06:04.320975 2014-08-13 17:38:02.874327 7fe9bae887a0 20 read_log 192888'19094 (0'0) promote 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by osd.4.0:22996 2014-08-11 17:06:05.543846 2014-08-13 17:38:02.874346 7fe9bae887a0 20 read_log 192888'19095 (192888'19094) modify 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.942868.0:13358 2014-08-11 17:06:05.534696 2014-08-13 17:38:02.874361 7fe9bae887a0 20 read_log 192888'19096 (0'0) promote 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by osd.4.0:23002 2014-08-11 17:06:06.175031 2014-08-13 17:38:02.874374 7fe9bae887a0 20 read_log 192888'19097 (192888'19096) modify 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.942868.0:13502 2014-08-11 17:06:06.165502 2014-08-13 17:38:02.874386 7fe9bae887a0 20 read_log 192888'19098 (0'0) promote 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by osd.4.0:23004 2014-08-11 17:06:06.332082 2014-08-13 17:38:02.874398 7fe9bae887a0 20 read_log 192888'19099 (192888'19098) modify 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.942868.0:13542 2014-08-11 17:06:06.323761 2014-08-13 17:38:02.874410 7fe9bae887a0 20 read_log 192888'19102 (0'0) promote 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by osd.4.0:23042 2014-08-11 17:06:14.095190 2014-08-13 17:38:02.874422 7fe9bae887a0 20 read_log 192888'19103 (192888'19102) modify 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.942868.0:15462 2014-08-11 17:06:14.015989 2014-08-13 17:38:02.874434 7fe9bae887a0 20 read_log 192888'19104 (0'0) promote e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by osd.4.0:23046 2014-08-11 17:06:14.230488 2014-08-13 17:38:02.874446 7fe9bae887a0 20 read_log 192888'19105 (192888'19104) modify e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.942868.0:15514 2014-08-11 17:06:14.219954 2014-08-13 17:38:02.874475 7fe9bae887a0 20 read_log 192888'19106 (0'0) promote 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by osd.4.0:23050 2014-08-11 17:06:14.619314 2014-08-13 17:38:02.874490 7fe9bae887a0 20 read_log 192888'19107 (192888'19106) modify 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.942868.0:15615 2014-08-11 17:06:14.609285 2014-08-13 17:38:02.874503 7fe9bae887a0 20 read_log 192888'19110 (0'0) promote ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by osd.4.0:23084 2014-08-11 17:06:20.522440 2014-08-13 17:38:02.874515 7fe9bae887a0 20 read_log 192888'19111 (192888'19110) modify ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.942868.0:17122 2014-08-11 17:06:20.513004 2014-08-13 17:38:02.874527 7fe9bae887a0 20 read_log 192888'19114 (0'0) promote 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by osd.4.0:23118 2014-08-11 17:06:23.715555 2014-08-13 17:38:02.874538 7fe9bae887a0 20 read_log 192888'19115 (192888'19114) modify 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.942868.0:17969 2014-08-11 17:06:23.707330 2014-08-13 17:38:02.874551 7fe9bae887a0 20 read_log 192888'19116 (0'0) promote 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by osd.4.0:23128 2014-08-11 17:06:26.323103 2014-08-13 17:38:02.874563 7fe9bae887a0 20 read_log 192888'19117 (192888'19116) modify 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.942868.0:18659 2014-08-11 17:06:26.315017 2014-08-13 17:38:02.874575 7fe9bae887a0 20 read_log 192888'19118 (0'0) promote efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by osd.4.0:23132 2014-08-11 17:06:26.803553 2014-08-13 17:38:02.874587 7fe9bae887a0 20 read_log 192888'19119 (192888'19118) modify efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.942868.0:18788 2014-08-11 17:06:26.794709 2014-08-13 17:38:02.874599 7fe9bae887a0 20 read_log 192888'19120 (0'0) promote e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by osd.4.0:23148 2014-08-11 17:06:29.495322 2014-08-13 17:38:02.874621 7fe9bae887a0 20 read_log 192888'19121 (192888'19120) modify e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.942868.0:19506 2014-08-11 17:06:29.487098 2014-08-13 17:38:02.874633 7fe9bae887a0 20 read_log 192888'19122 (0'0) promote 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by osd.4.0:23190 2014-08-11 17:06:35.477560 2014-08-13 17:38:02.874646 7fe9bae887a0 20 read_log 192888'19123 (192888'19122) modify 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.942868.0:21075 2014-08-11 17:06:35.469246 2014-08-13 17:38:02.874659 7fe9bae887a0 20 read_log 192888'19126 (0'0) promote 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by osd.4.0:23204 2014-08-11 17:06:37.303695 2014-08-13 17:38:02.874671 7fe9bae887a0 20 read_log 192888'19127 (192888'19126) modify 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.942868.0:21551 2014-08-11 17:06:37.293758 2014-08-13 17:38:02.874683 7fe9bae887a0 20 read_log 192888'19130 (0'0) promote 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by osd.4.0:23254 2014-08-11 17:06:43.234463 2014-08-13 17:38:02.874695 7fe9bae887a0 20 read_log 192888'19131 (192888'19130) modify 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.942868.0:23073 2014-08-11 17:06:43.225830 2014-08-13 17:38:02.874707 7fe9bae887a0 20 read_log 192888'19140 (0'0) promote e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by osd.4.0:23308 2014-08-11 17:06:49.278156 2014-08-13 17:38:02.874719 7fe9bae887a0 20 read_log 192888'19141 (192888'19140) modify e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.942868.0:24693 2014-08-11 17:06:49.269414 2014-08-13 17:38:02.874731 7fe9bae887a0 20 read_log 192888'19146 (0'0) promote d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by osd.4.0:23352 2014-08-11 17:06:55.227311 2014-08-13 17:38:02.874743 7fe9bae887a0 20 read_log 192888'19147 (192888'19146) modify d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.942868.0:26205 2014-08-11 17:06:55.218805 2014-08-13 17:38:02.874755 7fe9bae887a0 20 read_log 192888'19148 (0'0) promote 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by osd.4.0:23354 2014-08-11 17:06:56.487206 2014-08-13 17:38:02.874767 7fe9bae887a0 20 read_log 192888'19149 (192888'19148) modify 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.942868.0:26546 2014-08-11 17:06:56.477792 2014-08-13 17:38:02.874779 7fe9bae887a0 20 read_log 192888'19152 (0'0) promote d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by osd.4.0:23402 2014-08-11 17:07:03.633151 2014-08-13 17:38:02.874791 7fe9bae887a0 20 read_log 192888'19153 (192888'19152) modify d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.942868.0:28448 2014-08-11 17:07:03.625720 2014-08-13 17:38:02.874803 7fe9bae887a0 20 read_log 192888'19154 (0'0) promote 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by osd.4.0:23418 2014-08-11 17:07:06.402141 2014-08-13 17:38:02.874814 7fe9bae887a0 20 read_log 192888'19155 (192888'19154) modify 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.942868.0:29164 2014-08-11 17:07:06.391625 2014-08-13 17:38:02.874826 7fe9bae887a0 20 read_log 192888'19156 (0'0) promote be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by osd.4.0:23440 2014-08-11 17:07:10.267519 2014-08-13 17:38:02.874838 7fe9bae887a0 20 read_log 192888'19157 (192888'19156) modify be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.942868.0:30107 2014-08-11 17:07:10.256861 2014-08-13 17:38:02.874850 7fe9bae887a0 20 read_log 192888'19160 (0'0) promote e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by osd.4.0:23478 2014-08-11 17:07:13.780629 2014-08-13 17:38:02.874869 7fe9bae887a0 20 read_log 192888'19161 (192888'19160) modify e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.942868.0:30994 2014-08-11 17:07:13.772385 2014-08-13 17:38:02.874882 7fe9bae887a0 20 read_log 192888'19163 (192888'19035) modify 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.942868.0:31511 2014-08-11 17:07:15.949001 2014-08-13 17:38:02.874894 7fe9bae887a0 20 read_log 192888'19164 (192888'19037) modify e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.942868.0:31790 2014-08-11 17:07:16.058110 2014-08-13 17:38:02.874906 7fe9bae887a0 20 read_log 192888'19166 (192888'19041) modify 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.942868.0:32549 2014-08-11 17:07:16.284983 2014-08-13 17:38:02.874918 7fe9bae887a0 20 read_log 192888'19167 (192888'19043) modify 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.942868.0:32830 2014-08-11 17:07:16.355709 2014-08-13 17:38:02.874930 7fe9bae887a0 20 read_log 192888'19169 (192888'19047) modify dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.942868.0:34479 2014-08-11 17:07:16.801692 2014-08-13 17:38:02.874942 7fe9bae887a0 20 read_log 192888'19170 (192888'19049) modify a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.942868.0:34873 2014-08-11 17:07:16.919664 2014-08-13 17:38:02.874954 7fe9bae887a0 20 read_log 192888'19171 (192888'19051) modify 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.942868.0:35064 2014-08-11 17:07:16.975715 2014-08-13 17:38:02.874966 7fe9bae887a0 20 read_log 192888'19176 (192888'19061) modify 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.942868.0:38225 2014-08-11 17:07:17.826568 2014-08-13 17:38:02.874977 7fe9bae887a0 20 read_log 192888'19179 (192888'19067) modify e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.942868.0:39013 2014-08-11 17:07:18.055422 2014-08-13 17:38:02.874991 7fe9bae887a0 20 read_log 192888'19181 (192888'19071) modify 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.942868.0:39768 2014-08-11 17:07:18.277904 2014-08-13 17:38:02.875003 7fe9bae887a0 20 read_log 192888'19182 (192888'19073) modify 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.942868.0:40396 2014-08-11 17:07:18.439382 2014-08-13 17:38:02.875015 7fe9bae887a0 20 read_log 192888'19183 (192888'19075) modify fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.942868.0:41083 2014-08-11 17:07:18.634662 2014-08-13 17:38:02.875027 7fe9bae887a0 20 read_log 192888'19186 (192888'19081) modify fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.942868.0:41515 2014-08-11 17:07:18.749169 2014-08-13 17:38:02.875039 7fe9bae887a0 20 read_log 192888'19190 (192888'19089) modify 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.942868.0:43213 2014-08-11 17:07:19.209757 2014-08-13 17:38:02.875051 7fe9bae887a0 20 read_log 192888'19192 (192888'19093) modify 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.942868.0:44304 2014-08-11 17:07:19.529156 2014-08-13 17:38:02.875063 7fe9bae887a0 20 read_log 192888'19193 (192888'19095) modify 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.942868.0:44624 2014-08-11 17:07:19.618360 2014-08-13 17:38:02.875076 7fe9bae887a0 20 read_log 192888'19194 (192888'19097) modify 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.942868.0:44768 2014-08-11 17:07:19.657375 2014-08-13 17:38:02.875088 7fe9bae887a0 20 read_log 192888'19195 (192888'19099) modify 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.942868.0:44808 2014-08-11 17:07:19.668315 2014-08-13 17:38:02.875100 7fe9bae887a0 20 read_log 192888'19197 (192888'19103) modify 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.942868.0:46728 2014-08-11 17:07:20.203299 2014-08-13 17:38:02.875122 7fe9bae887a0 20 read_log 192888'19198 (192888'19105) modify e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.942868.0:46780 2014-08-11 17:07:20.216609 2014-08-13 17:38:02.875135 7fe9bae887a0 20 read_log 192888'19199 (192888'19107) modify 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.942868.0:46881 2014-08-11 17:07:20.243783 2014-08-13 17:38:02.875146 7fe9bae887a0 20 read_log 192888'19201 (192888'19111) modify ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.942868.0:48388 2014-08-11 17:07:20.705306 2014-08-13 17:38:02.875163 7fe9bae887a0 20 read_log 192888'19203 (192888'19115) modify 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.942868.0:49235 2014-08-11 17:07:20.950348 2014-08-13 17:38:02.875176 7fe9bae887a0 20 read_log 192888'19204 (192888'19117) modify 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.942868.0:49925 2014-08-11 17:07:21.164554 2014-08-13 17:38:02.875188 7fe9bae887a0 20 read_log 192888'19205 (192888'19119) modify efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.942868.0:50054 2014-08-11 17:07:21.201692 2014-08-13 17:38:02.875200 7fe9bae887a0 20 read_log 192888'19206 (192888'19121) modify e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.942868.0:50772 2014-08-11 17:07:21.426146 2014-08-13 17:38:02.875212 7fe9bae887a0 20 read_log 192888'19207 (192888'19123) modify 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.942868.0:52341 2014-08-11 17:07:21.908353 2014-08-13 17:38:02.875224 7fe9bae887a0 20 read_log 192888'19209 (192888'19127) modify 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.942868.0:52817 2014-08-11 17:07:22.059200 2014-08-13 17:38:02.875236 7fe9bae887a0 20 read_log 192888'19211 (192888'19131) modify 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.942868.0:54339 2014-08-11 17:07:22.529625 2014-08-13 17:38:02.875248 7fe9bae887a0 20 read_log 192888'19216 (192888'19141) modify e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.942868.0:55959 2014-08-11 17:07:23.009373 2014-08-13 17:38:02.875260 7fe9bae887a0 20 read_log 192888'19219 (192888'19147) modify d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.942868.0:57471 2014-08-11 17:07:23.465636 2014-08-13 17:38:02.875272 7fe9bae887a0 20 read_log 192888'19220 (192888'19149) modify 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.942868.0:57812 2014-08-11 17:07:23.561913 2014-08-13 17:38:02.875284 7fe9bae887a0 20 read_log 192888'19222 (192888'19153) modify d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.942868.0:59714 2014-08-11 17:07:24.127237 2014-08-13 17:38:02.875296 7fe9bae887a0 20 read_log 192888'19223 (192888'19155) modify 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.942868.0:60430 2014-08-11 17:07:24.337643 2014-08-13 17:38:02.875308 7fe9bae887a0 20 read_log 192888'19224 (192888'19157) modify be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.942868.0:61373 2014-08-11 17:07:24.620557 2014-08-13 17:38:02.875320 7fe9bae887a0 20 read_log 192888'19226 (192888'19161) modify e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.942868.0:62260 2014-08-11 17:07:24.897906 2014-08-13 17:38:02.875333 7fe9bae887a0 20 read_log 193305'19227 (193305'19227) modify 181/hit_set_24.181_archive_2014-08-11 18:37:04.912085_2014-08-11 18:42:37.391795/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.392158 2014-08-13 17:38:02.875345 7fe9bae887a0 20 read_log 193305'19228 (192888'19030) delete 181/hit_set_24.181_archive_2014-08-11 16:44:54.931606_2014-08-11 17:05:14.856635/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.392158 2014-08-13 17:38:02.875364 7fe9bae887a0 20 read_log 193305'19229 (0'0) promote 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by osd.4.0:7 2014-08-11 18:42:37.414235 2014-08-13 17:38:02.875377 7fe9bae887a0 20 read_log 193305'19230 (193305'19229) modify 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.951337.0:1116 2014-08-11 18:42:37.386510 2014-08-13 17:38:02.875389 7fe9bae887a0 20 read_log 193305'19231 (0'0) promote 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by osd.4.0:65 2014-08-11 18:42:50.623266 2014-08-13 17:38:02.875401 7fe9bae887a0 20 read_log 193305'19232 (193305'19231) modify 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.951337.0:4701 2014-08-11 18:42:50.615171 2014-08-13 17:38:02.875413 7fe9bae887a0 20 read_log 193305'19233 (0'0) promote 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by osd.4.0:103 2014-08-11 18:42:56.888876 2014-08-13 17:38:02.875425 7fe9bae887a0 20 read_log 193305'19234 (193305'19233) modify 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.951337.0:6403 2014-08-11 18:42:56.880978 2014-08-13 17:38:02.875438 7fe9bae887a0 20 read_log 193305'19235 (0'0) promote be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by osd.4.0:117 2014-08-11 18:42:59.752968 2014-08-13 17:38:02.875450 7fe9bae887a0 20 read_log 193305'19236 (193305'19235) modify be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.951337.0:7196 2014-08-11 18:42:59.745109 2014-08-13 17:38:02.875464 7fe9bae887a0 20 read_log 193305'19237 (0'0) promote 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by osd.4.0:141 2014-08-11 18:43:04.570027 2014-08-13 17:38:02.875476 7fe9bae887a0 20 read_log 193305'19238 (193305'19237) modify 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.951337.0:8456 2014-08-11 18:43:04.557561 2014-08-13 17:38:02.875488 7fe9bae887a0 20 read_log 193305'19239 (0'0) promote 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by osd.4.0:143 2014-08-11 18:43:04.647499 2014-08-13 17:38:02.875501 7fe9bae887a0 20 read_log 193305'19240 (193305'19239) modify 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.951337.0:8479 2014-08-11 18:43:04.639167 2014-08-13 17:38:02.875513 7fe9bae887a0 20 read_log 193305'19241 (0'0) promote 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by osd.4.0:145 2014-08-11 18:43:06.730763 2014-08-13 17:38:02.875525 7fe9bae887a0 20 read_log 193305'19242 (193305'19241) modify 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.951337.0:9026 2014-08-11 18:43:06.722341 2014-08-13 17:38:02.875537 7fe9bae887a0 20 read_log 193305'19243 (0'0) promote 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by osd.4.0:149 2014-08-11 18:43:07.117839 2014-08-13 17:38:02.875549 7fe9bae887a0 20 read_log 193305'19244 (193305'19243) modify 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.951337.0:9134 2014-08-11 18:43:07.109874 2014-08-13 17:38:02.875561 7fe9bae887a0 20 read_log 193305'19245 (0'0) promote 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by osd.4.0:163 2014-08-11 18:43:10.317581 2014-08-13 17:38:02.875573 7fe9bae887a0 20 read_log 193305'19246 (193305'19245) modify 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.951337.0:10006 2014-08-11 18:43:10.309378 2014-08-13 17:38:02.875585 7fe9bae887a0 20 read_log 193305'19247 (0'0) promote 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by osd.4.0:179 2014-08-11 18:43:13.923431 2014-08-13 17:38:02.875596 7fe9bae887a0 20 read_log 193305'19248 (193305'19247) modify 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.951337.0:10977 2014-08-11 18:43:13.914869 2014-08-13 17:38:02.875619 7fe9bae887a0 20 read_log 193305'19249 (0'0) promote eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by osd.4.0:197 2014-08-11 18:43:16.816858 2014-08-13 17:38:02.875632 7fe9bae887a0 20 read_log 193305'19250 (193305'19249) modify eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.951337.0:11709 2014-08-11 18:43:16.809120 2014-08-13 17:38:02.875646 7fe9bae887a0 20 read_log 193305'19251 (0'0) promote 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by osd.4.0:243 2014-08-11 18:43:29.536038 2014-08-13 17:38:02.875677 7fe9bae887a0 20 read_log 193305'19252 (193305'19251) modify 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.951337.0:15169 2014-08-11 18:43:29.529550 2014-08-13 17:38:02.875690 7fe9bae887a0 20 read_log 193305'19253 (0'0) promote 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by osd.4.0:255 2014-08-11 18:43:32.251220 2014-08-13 17:38:02.875702 7fe9bae887a0 20 read_log 193305'19254 (193305'19253) modify 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.951337.0:15965 2014-08-11 18:43:32.242778 2014-08-13 17:38:02.875714 7fe9bae887a0 20 read_log 193305'19255 (0'0) promote 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by osd.4.0:259 2014-08-11 18:43:32.910096 2014-08-13 17:38:02.875726 7fe9bae887a0 20 read_log 193305'19256 (193305'19255) modify 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.951337.0:16148 2014-08-11 18:43:32.902502 2014-08-13 17:38:02.875738 7fe9bae887a0 20 read_log 193305'19257 (0'0) promote 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by osd.4.0:273 2014-08-11 18:43:34.231520 2014-08-13 17:38:02.875750 7fe9bae887a0 20 read_log 193305'19258 (193305'19257) modify 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.951337.0:16511 2014-08-11 18:43:34.222243 2014-08-13 17:38:02.875762 7fe9bae887a0 20 read_log 193305'19259 (0'0) promote 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by osd.4.0:277 2014-08-11 18:43:35.004791 2014-08-13 17:38:02.875774 7fe9bae887a0 20 read_log 193305'19260 (193305'19259) modify 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.951337.0:16744 2014-08-11 18:43:34.996459 2014-08-13 17:38:02.875786 7fe9bae887a0 20 read_log 193305'19261 (0'0) promote c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by osd.4.0:295 2014-08-11 18:43:37.947641 2014-08-13 17:38:02.875797 7fe9bae887a0 20 read_log 193305'19262 (193305'19261) modify c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.951337.0:17597 2014-08-11 18:43:37.940973 2014-08-13 17:38:02.875809 7fe9bae887a0 20 read_log 193305'19263 (0'0) promote fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by osd.4.0:335 2014-08-11 18:43:46.701167 2014-08-13 17:38:02.875821 7fe9bae887a0 20 read_log 193305'19264 (193305'19263) modify fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.951337.0:20021 2014-08-11 18:43:46.692250 2014-08-13 17:38:02.875833 7fe9bae887a0 20 read_log 193305'19265 (0'0) promote 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by osd.4.0:351 2014-08-11 18:43:49.877673 2014-08-13 17:38:02.875845 7fe9bae887a0 20 read_log 193305'19266 (193305'19265) modify 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.951337.0:20911 2014-08-11 18:43:49.869894 2014-08-13 17:38:02.875857 7fe9bae887a0 20 read_log 193305'19267 (0'0) promote 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by osd.4.0:359 2014-08-11 18:43:52.549222 2014-08-13 17:38:02.875869 7fe9bae887a0 20 read_log 193305'19268 (193305'19267) modify 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.951337.0:21662 2014-08-11 18:43:52.541745 2014-08-13 17:38:02.875891 7fe9bae887a0 20 read_log 193305'19269 (0'0) promote 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by osd.4.0:381 2014-08-11 18:43:55.936000 2014-08-13 17:38:02.875904 7fe9bae887a0 20 read_log 193305'19270 (193305'19269) modify 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.951337.0:22578 2014-08-11 18:43:55.925642 2014-08-13 17:38:02.875916 7fe9bae887a0 20 read_log 193305'19271 (0'0) promote 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by osd.4.0:407 2014-08-11 18:44:03.019451 2014-08-13 17:38:02.875928 7fe9bae887a0 20 read_log 193305'19272 (193305'19271) modify 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.951337.0:24549 2014-08-11 18:44:03.011034 2014-08-13 17:38:02.875940 7fe9bae887a0 20 read_log 193305'19273 (0'0) promote b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by osd.4.0:427 2014-08-11 18:44:08.640304 2014-08-13 17:38:02.875952 7fe9bae887a0 20 read_log 193305'19274 (193305'19273) modify b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.951337.0:26063 2014-08-11 18:44:08.632247 2014-08-13 17:38:02.875964 7fe9bae887a0 20 read_log 193305'19275 (0'0) promote c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by osd.4.0:435 2014-08-11 18:44:10.359018 2014-08-13 17:38:02.875975 7fe9bae887a0 20 read_log 193305'19276 (193305'19275) modify c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.951337.0:26548 2014-08-11 18:44:10.352627 2014-08-13 17:38:02.875987 7fe9bae887a0 20 read_log 193305'19277 (0'0) promote 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by osd.4.0:449 2014-08-11 18:44:11.539208 2014-08-13 17:38:02.875999 7fe9bae887a0 20 read_log 193305'19278 (193305'19277) modify 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.951337.0:26875 2014-08-11 18:44:11.528771 2014-08-13 17:38:02.876012 7fe9bae887a0 20 read_log 193305'19279 (0'0) promote c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by osd.4.0:451 2014-08-11 18:44:12.518839 2014-08-13 17:38:02.876024 7fe9bae887a0 20 read_log 193305'19280 (193305'19279) modify c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.951337.0:27124 2014-08-11 18:44:12.510131 2014-08-13 17:38:02.876036 7fe9bae887a0 20 read_log 193305'19281 (0'0) promote 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by osd.4.0:513 2014-08-11 18:44:28.776138 2014-08-13 17:38:02.876048 7fe9bae887a0 20 read_log 193305'19282 (193305'19281) modify 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.951337.0:31562 2014-08-11 18:44:28.766694 2014-08-13 17:38:02.876060 7fe9bae887a0 20 read_log 193305'19283 (0'0) promote d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by osd.4.0:515 2014-08-11 18:44:29.296022 2014-08-13 17:38:02.876072 7fe9bae887a0 20 read_log 193305'19284 (193305'19283) modify d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.951337.0:31712 2014-08-11 18:44:29.287665 2014-08-13 17:38:02.876083 7fe9bae887a0 20 read_log 193305'19285 (0'0) promote 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by osd.4.0:525 2014-08-11 18:44:30.070715 2014-08-13 17:38:02.876095 7fe9bae887a0 20 read_log 193305'19286 (193305'19285) modify 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.951337.0:31940 2014-08-11 18:44:30.061779 2014-08-13 17:38:02.876107 7fe9bae887a0 20 read_log 193305'19287 (193305'19230) modify 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.951337.0:33827 2014-08-11 18:44:34.672154 2014-08-13 17:38:02.876119 7fe9bae887a0 20 read_log 193305'19288 (193305'19232) modify 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.951337.0:37412 2014-08-11 18:44:36.899211 2014-08-13 17:38:02.876138 7fe9bae887a0 20 read_log 193305'19289 (193305'19234) modify 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.951337.0:39114 2014-08-11 18:44:37.831940 2014-08-13 17:38:02.876151 7fe9bae887a0 20 read_log 193305'19290 (193305'19236) modify be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.951337.0:39907 2014-08-11 18:44:38.223729 2014-08-13 17:38:02.876176 7fe9bae887a0 20 read_log 193305'19291 (193305'19238) modify 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.951337.0:41167 2014-08-11 18:44:38.655216 2014-08-13 17:38:02.876190 7fe9bae887a0 20 read_log 193305'19292 (193305'19240) modify 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.951337.0:41190 2014-08-11 18:44:38.662732 2014-08-13 17:38:02.876202 7fe9bae887a0 20 read_log 193305'19293 (193305'19242) modify 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.951337.0:41737 2014-08-11 18:44:38.853048 2014-08-13 17:38:02.876214 7fe9bae887a0 20 read_log 193305'19294 (193305'19244) modify 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.951337.0:41845 2014-08-11 18:44:38.891962 2014-08-13 17:38:02.876226 7fe9bae887a0 20 read_log 193305'19295 (193305'19246) modify 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.951337.0:42717 2014-08-11 18:44:39.360400 2014-08-13 17:38:02.876238 7fe9bae887a0 20 read_log 193305'19296 (193305'19248) modify 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.951337.0:43688 2014-08-11 18:44:39.666300 2014-08-13 17:38:02.876250 7fe9bae887a0 20 read_log 193305'19297 (193305'19250) modify eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.951337.0:44420 2014-08-11 18:44:39.876758 2014-08-13 17:38:02.876262 7fe9bae887a0 20 read_log 193305'19298 (193305'19252) modify 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.951337.0:47880 2014-08-11 18:44:40.891796 2014-08-13 17:38:02.876274 7fe9bae887a0 20 read_log 193305'19299 (193305'19254) modify 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.951337.0:48676 2014-08-11 18:44:41.121410 2014-08-13 17:38:02.876286 7fe9bae887a0 20 read_log 193305'19300 (193305'19256) modify 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.951337.0:48859 2014-08-11 18:44:41.177409 2014-08-13 17:38:02.876298 7fe9bae887a0 20 read_log 193305'19301 (193305'19258) modify 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.951337.0:49222 2014-08-11 18:44:41.298585 2014-08-13 17:38:02.876310 7fe9bae887a0 20 read_log 193305'19302 (193305'19260) modify 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.951337.0:49455 2014-08-11 18:44:41.370780 2014-08-13 17:38:02.876321 7fe9bae887a0 20 read_log 193305'19303 (193305'19262) modify c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.951337.0:50308 2014-08-11 18:44:41.628174 2014-08-13 17:38:02.876333 7fe9bae887a0 20 read_log 193305'19304 (193305'19264) modify fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.951337.0:52732 2014-08-11 18:44:42.321549 2014-08-13 17:38:02.876345 7fe9bae887a0 20 read_log 193305'19305 (193305'19266) modify 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.951337.0:53622 2014-08-11 18:44:42.584211 2014-08-13 17:38:02.876357 7fe9bae887a0 20 read_log 193305'19306 (193305'19268) modify 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.951337.0:54373 2014-08-11 18:44:42.813535 2014-08-13 17:38:02.876368 7fe9bae887a0 20 read_log 193305'19307 (193305'19270) modify 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.951337.0:55289 2014-08-11 18:44:43.079263 2014-08-13 17:38:02.876380 7fe9bae887a0 20 read_log 193305'19308 (193305'19272) modify 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.951337.0:57260 2014-08-11 18:44:43.693702 2014-08-13 17:38:02.876400 7fe9bae887a0 20 read_log 193305'19309 (193305'19274) modify b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.951337.0:58774 2014-08-11 18:44:44.138463 2014-08-13 17:38:02.876412 7fe9bae887a0 20 read_log 193305'19310 (193305'19276) modify c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.951337.0:59259 2014-08-11 18:44:44.275811 2014-08-13 17:38:02.876424 7fe9bae887a0 20 read_log 193305'19311 (193305'19278) modify 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.951337.0:59586 2014-08-11 18:44:44.370047 2014-08-13 17:38:02.876436 7fe9bae887a0 20 read_log 193305'19312 (193305'19280) modify c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.951337.0:59835 2014-08-11 18:44:44.440651 2014-08-13 17:38:02.876448 7fe9bae887a0 20 read_log 193305'19313 (193305'19282) modify 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.951337.0:64273 2014-08-11 18:44:45.753606 2014-08-13 17:38:02.876463 7fe9bae887a0 20 read_log 193305'19314 (193305'19284) modify d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.951337.0:64423 2014-08-11 18:44:45.802345 2014-08-13 17:38:02.876475 7fe9bae887a0 20 read_log 193305'19315 (193305'19286) modify 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.951337.0:64651 2014-08-11 18:44:45.880380 2014-08-13 17:38:02.876487 7fe9bae887a0 20 read_log 194215'19316 (194215'19316) modify 181/hit_set_24.181_archive_2014-08-12 15:32:25.863362_2014-08-12 15:42:02.778991/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:42:02.779435 2014-08-13 17:38:02.876500 7fe9bae887a0 20 read_log 194215'19317 (193305'19227) delete 181/hit_set_24.181_archive_2014-08-11 18:37:04.912085_2014-08-11 18:42:37.391795/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:42:02.779435 2014-08-13 17:38:02.876512 7fe9bae887a0 20 read_log 194215'19318 (192888'19016) clean 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by osd.4.0:1777 2014-08-12 15:42:02.792250 2014-08-13 17:38:02.876524 7fe9bae887a0 20 read_log 194215'19319 (194215'19318) delete 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.2174595.0:78382 0.000000 2014-08-13 17:38:02.876535 7fe9bae887a0 20 read_log 194215'19320 (192888'19176) clean 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by osd.4.0:1779 2014-08-12 15:42:02.817495 2014-08-13 17:38:02.876547 7fe9bae887a0 20 read_log 194215'19321 (194215'19320) delete 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.2174595.0:78384 0.000000 2014-08-13 17:38:02.876557 7fe9bae887a0 20 read_log 194215'19322 (192888'19005) clean ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by osd.4.0:1781 2014-08-12 15:42:02.825705 2014-08-13 17:38:02.876569 7fe9bae887a0 20 read_log 194215'19323 (194215'19322) delete ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.2174595.0:78386 0.000000 2014-08-13 17:38:02.876580 7fe9bae887a0 20 read_log 194215'19324 (193305'19292) clean 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by osd.4.0:1783 2014-08-12 15:42:02.834177 2014-08-13 17:38:02.876592 7fe9bae887a0 20 read_log 194215'19325 (194215'19324) delete 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.2174595.0:78388 0.000000 2014-08-13 17:38:02.876603 7fe9bae887a0 20 read_log 194215'19326 (193305'19288) clean 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by osd.4.0:1785 2014-08-12 15:42:02.842492 2014-08-13 17:38:02.876615 7fe9bae887a0 20 read_log 194215'19327 (194215'19326) delete 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.2174595.0:78390 0.000000 2014-08-13 17:38:02.876637 7fe9bae887a0 20 read_log 194215'19328 (192888'19186) clean fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by osd.4.0:1787 2014-08-12 15:42:02.850866 2014-08-13 17:38:02.876651 7fe9bae887a0 20 read_log 194215'19329 (194215'19328) delete fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.2174595.0:78392 0.000000 2014-08-13 17:38:02.876661 7fe9bae887a0 20 read_log 194215'19330 (193305'19308) clean 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by osd.4.0:1789 2014-08-12 15:42:02.860751 2014-08-13 17:38:02.876674 7fe9bae887a0 20 read_log 194215'19331 (194215'19330) delete 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.2174595.0:78394 0.000000 2014-08-13 17:38:02.876684 7fe9bae887a0 20 read_log 194215'19332 (193305'19310) clean c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by osd.4.0:1791 2014-08-12 15:42:02.869496 2014-08-13 17:38:02.876696 7fe9bae887a0 20 read_log 194215'19333 (194215'19332) delete c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.2174595.0:78396 0.000000 2014-08-13 17:38:02.876707 7fe9bae887a0 20 read_log 194215'19334 (192888'19198) clean e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by osd.4.0:1793 2014-08-12 15:42:02.878153 2014-08-13 17:38:02.876719 7fe9bae887a0 20 read_log 194215'19335 (194215'19334) delete e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.2174595.0:78398 0.000000 2014-08-13 17:38:02.876729 7fe9bae887a0 20 read_log 194215'19336 (192888'19224) clean be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by osd.4.0:1795 2014-08-12 15:42:02.886428 2014-08-13 17:38:02.876742 7fe9bae887a0 20 read_log 194215'19337 (194215'19336) delete be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.2174595.0:78400 0.000000 2014-08-13 17:38:02.876752 7fe9bae887a0 20 read_log 194215'19338 (192888'19017) clean 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by osd.4.0:1797 2014-08-12 15:42:02.895484 2014-08-13 17:38:02.876764 7fe9bae887a0 20 read_log 194215'19339 (194215'19338) delete 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.2174595.0:78402 0.000000 2014-08-13 17:38:02.876775 7fe9bae887a0 20 read_log 194215'19340 (192888'19201) clean ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by osd.4.0:1799 2014-08-12 15:42:02.904116 2014-08-13 17:38:02.876789 7fe9bae887a0 20 read_log 194215'19341 (194215'19340) delete ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.2174595.0:78404 0.000000 2014-08-13 17:38:02.876800 7fe9bae887a0 20 read_log 194215'19342 (192888'19021) clean 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by osd.4.0:1801 2014-08-12 15:42:02.912300 2014-08-13 17:38:02.876812 7fe9bae887a0 20 read_log 194215'19343 (194215'19342) delete 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.2174595.0:78406 0.000000 2014-08-13 17:38:02.876823 7fe9bae887a0 20 read_log 194215'19344 (193305'19291) clean 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by osd.4.0:1803 2014-08-12 15:42:02.947296 2014-08-13 17:38:02.876853 7fe9bae887a0 20 read_log 194215'19345 (194215'19344) delete 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.2174595.0:78408 0.000000 2014-08-13 17:38:02.876864 7fe9bae887a0 20 read_log 194215'19346 (192888'19003) clean e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by osd.4.0:1805 2014-08-12 15:42:02.956194 2014-08-13 17:38:02.876876 7fe9bae887a0 20 read_log 194215'19347 (194215'19346) delete e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.2174595.0:78410 0.000000 2014-08-13 17:38:02.876908 7fe9bae887a0 20 read_log 194215'19348 (192888'18974) clean 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by osd.4.0:1807 2014-08-12 15:42:02.964745 2014-08-13 17:38:02.876922 7fe9bae887a0 20 read_log 194215'19349 (194215'19348) delete 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.2174595.0:78412 0.000000 2014-08-13 17:38:02.876933 7fe9bae887a0 20 read_log 194215'19350 (192888'19013) clean 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by osd.4.0:1809 2014-08-12 15:42:02.973050 2014-08-13 17:38:02.876945 7fe9bae887a0 20 read_log 194215'19351 (194215'19350) delete 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.2174595.0:78414 0.000000 2014-08-13 17:38:02.876955 7fe9bae887a0 20 read_log 194215'19352 (192888'19220) clean 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by osd.4.0:1811 2014-08-12 15:42:02.983226 2014-08-13 17:38:02.876968 7fe9bae887a0 20 read_log 194215'19353 (194215'19352) delete 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.2174595.0:78416 0.000000 2014-08-13 17:38:02.876978 7fe9bae887a0 20 read_log 194215'19354 (192888'19024) clean 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by osd.4.0:1813 2014-08-12 15:42:02.991751 2014-08-13 17:38:02.876990 7fe9bae887a0 20 read_log 194215'19355 (194215'19354) delete 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.2174595.0:78418 0.000000 2014-08-13 17:38:02.877001 7fe9bae887a0 20 read_log 194215'19356 (192888'18985) clean 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by osd.4.0:1815 2014-08-12 15:42:02.999471 2014-08-13 17:38:02.877013 7fe9bae887a0 20 read_log 194215'19357 (194215'19356) delete 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.2174595.0:78420 0.000000 2014-08-13 17:38:02.877024 7fe9bae887a0 20 read_log 194215'19358 (192888'19203) clean 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by osd.4.0:1817 2014-08-12 15:42:03.007328 2014-08-13 17:38:02.877036 7fe9bae887a0 20 read_log 194215'19359 (194215'19358) delete 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.2174595.0:78422 0.000000 2014-08-13 17:38:02.877047 7fe9bae887a0 20 read_log 194215'19360 (192888'19211) clean 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by osd.4.0:1819 2014-08-12 15:42:03.014677 2014-08-13 17:38:02.877059 7fe9bae887a0 20 read_log 194215'19361 (194215'19360) delete 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.2174595.0:78424 0.000000 2014-08-13 17:38:02.877091 7fe9bae887a0 20 read_log 194215'19362 (192888'19009) clean 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by osd.4.0:1821 2014-08-12 15:42:03.024077 2014-08-13 17:38:02.877104 7fe9bae887a0 20 read_log 194215'19363 (194215'19362) delete 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.2174595.0:78426 0.000000 2014-08-13 17:38:02.877114 7fe9bae887a0 20 read_log 194215'19364 (192888'18971) clean 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by osd.4.0:1823 2014-08-12 15:42:03.032097 2014-08-13 17:38:02.877126 7fe9bae887a0 20 read_log 194215'19365 (194215'19364) delete 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.2174595.0:78428 0.000000 2014-08-13 17:38:02.877137 7fe9bae887a0 20 read_log 194215'19366 (193305'19314) clean d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by osd.4.0:1825 2014-08-12 15:42:03.055264 2014-08-13 17:38:02.877149 7fe9bae887a0 20 read_log 194215'19367 (194215'19366) delete d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.2174595.0:78430 0.000000 2014-08-13 17:38:02.877172 7fe9bae887a0 20 read_log 194215'19368 (192888'19022) clean f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by osd.4.0:1827 2014-08-12 15:42:03.064395 2014-08-13 17:38:02.877186 7fe9bae887a0 20 read_log 194215'19369 (194215'19368) delete f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.2174595.0:78432 0.000000 2014-08-13 17:38:02.877196 7fe9bae887a0 20 read_log 194215'19370 (193305'19293) clean 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by osd.4.0:1829 2014-08-12 15:42:03.089076 2014-08-13 17:38:02.877209 7fe9bae887a0 20 read_log 194215'19371 (194215'19370) delete 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.2174595.0:78434 0.000000 2014-08-13 17:38:02.877220 7fe9bae887a0 20 read_log 194215'19372 (192888'19219) clean d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by osd.4.0:1831 2014-08-12 15:42:03.097755 2014-08-13 17:38:02.877233 7fe9bae887a0 20 read_log 194215'19373 (194215'19372) delete d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.2174595.0:78436 0.000000 2014-08-13 17:38:02.877244 7fe9bae887a0 20 read_log 194215'19374 (192888'18972) clean 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by osd.4.0:1833 2014-08-12 15:42:03.106868 2014-08-13 17:38:02.877256 7fe9bae887a0 20 read_log 194215'19375 (194215'19374) delete 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.2174595.0:78438 0.000000 2014-08-13 17:38:02.877266 7fe9bae887a0 20 read_log 194215'19376 (193305'19294) clean 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by osd.4.0:1835 2014-08-12 15:42:03.116274 2014-08-13 17:38:02.877278 7fe9bae887a0 20 read_log 194215'19377 (194215'19376) delete 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.2174595.0:78440 0.000000 2014-08-13 17:38:02.877289 7fe9bae887a0 20 read_log 194215'19378 (192888'19199) clean 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by osd.4.0:1837 2014-08-12 15:42:03.125354 2014-08-13 17:38:02.877301 7fe9bae887a0 20 read_log 194215'19379 (194215'19378) delete 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.2174595.0:78442 0.000000 2014-08-13 17:38:02.877311 7fe9bae887a0 20 read_log 194215'19380 (192888'19014) clean 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by osd.4.0:1839 2014-08-12 15:42:03.134064 2014-08-13 17:38:02.877324 7fe9bae887a0 20 read_log 194215'19381 (194215'19380) delete 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.2174595.0:78444 0.000000 2014-08-13 17:38:02.877334 7fe9bae887a0 20 read_log 194215'19382 (193305'19301) clean 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by osd.4.0:1841 2014-08-12 15:42:03.143215 2014-08-13 17:38:02.877346 7fe9bae887a0 20 read_log 194215'19383 (194215'19382) delete 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.2174595.0:78446 0.000000 2014-08-13 17:38:02.877357 7fe9bae887a0 20 read_log 194215'19384 (192888'18991) clean 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by osd.4.0:1843 2014-08-12 15:42:03.151261 2014-08-13 17:38:02.877369 7fe9bae887a0 20 read_log 194215'19385 (194215'19384) delete 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.2174595.0:78448 0.000000 2014-08-13 17:38:02.877379 7fe9bae887a0 20 read_log 194215'19386 (192888'19183) clean fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by osd.4.0:1845 2014-08-12 15:42:03.161390 2014-08-13 17:38:02.877391 7fe9bae887a0 20 read_log 194215'19387 (194215'19386) delete fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.2174595.0:78450 0.000000 2014-08-13 17:38:02.877408 7fe9bae887a0 20 read_log 194215'19388 (192888'19023) clean 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by osd.4.0:1847 2014-08-12 15:42:03.169991 2014-08-13 17:38:02.877421 7fe9bae887a0 20 read_log 194215'19389 (194215'19388) delete 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.2174595.0:78452 0.000000 2014-08-13 17:38:02.877431 7fe9bae887a0 20 read_log 194215'19390 (193305'19298) clean 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by osd.4.0:1849 2014-08-12 15:42:03.190911 2014-08-13 17:38:02.877443 7fe9bae887a0 20 read_log 194215'19391 (194215'19390) delete 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.2174595.0:78454 0.000000 2014-08-13 17:38:02.877454 7fe9bae887a0 20 read_log 194215'19392 (192888'19216) clean e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by osd.4.0:1851 2014-08-12 15:42:03.200498 2014-08-13 17:38:02.877466 7fe9bae887a0 20 read_log 194215'19393 (194215'19392) delete e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.2174595.0:78456 0.000000 2014-08-13 17:38:02.877477 7fe9bae887a0 20 read_log 194215'19394 (192888'18999) clean f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by osd.4.0:1853 2014-08-12 15:42:03.209635 2014-08-13 17:38:02.877489 7fe9bae887a0 20 read_log 194215'19395 (194215'19394) delete f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.2174595.0:78458 0.000000 2014-08-13 17:38:02.877499 7fe9bae887a0 20 read_log 194215'19396 (193305'19289) clean 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by osd.4.0:1855 2014-08-12 15:42:03.218587 2014-08-13 17:38:02.877511 7fe9bae887a0 20 read_log 194215'19397 (194215'19396) delete 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.2174595.0:78460 0.000000 2014-08-13 17:38:02.877522 7fe9bae887a0 20 read_log 194215'19398 (192888'18977) clean 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by osd.4.0:1857 2014-08-12 15:42:03.227078 2014-08-13 17:38:02.877534 7fe9bae887a0 20 read_log 194215'19399 (194215'19398) delete 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.2174595.0:78462 0.000000 2014-08-13 17:38:02.877544 7fe9bae887a0 20 read_log 194215'19400 (193305'19309) clean b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by osd.4.0:1859 2014-08-12 15:42:03.236441 2014-08-13 17:38:02.877556 7fe9bae887a0 20 read_log 194215'19401 (194215'19400) delete b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.2174595.0:78464 0.000000 2014-08-13 17:38:02.877567 7fe9bae887a0 20 read_log 194215'19402 (192888'19194) clean 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by osd.4.0:1861 2014-08-12 15:42:03.244814 2014-08-13 17:38:02.877578 7fe9bae887a0 20 read_log 194215'19403 (194215'19402) delete 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.2174595.0:78466 0.000000 2014-08-13 17:38:02.877589 7fe9bae887a0 20 read_log 194215'19404 (192888'19204) clean 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by osd.4.0:1863 2014-08-12 15:42:03.252965 2014-08-13 17:38:02.877603 7fe9bae887a0 20 read_log 194215'19405 (194215'19404) delete 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.2174595.0:78468 0.000000 2014-08-13 17:38:02.877614 7fe9bae887a0 20 read_log 194215'19406 (192888'18981) clean 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by osd.4.0:1865 2014-08-12 15:42:03.262330 2014-08-13 17:38:02.877626 7fe9bae887a0 20 read_log 194215'19407 (194215'19406) delete 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.2174595.0:78470 0.000000 2014-08-13 17:38:02.877646 7fe9bae887a0 20 read_log 194215'19408 (192888'19169) clean dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by osd.4.0:1867 2014-08-12 15:42:03.270176 2014-08-13 17:38:02.877658 7fe9bae887a0 20 read_log 194215'19409 (194215'19408) delete dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.2174595.0:78472 0.000000 2014-08-13 17:38:02.877669 7fe9bae887a0 20 read_log 194215'19410 (193305'19306) clean 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by osd.4.0:1869 2014-08-12 15:42:03.278244 2014-08-13 17:38:02.877681 7fe9bae887a0 20 read_log 194215'19411 (194215'19410) delete 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.2174595.0:78474 0.000000 2014-08-13 17:38:02.877692 7fe9bae887a0 20 read_log 194215'19412 (192888'19193) clean 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by osd.4.0:1871 2014-08-12 15:42:03.287850 2014-08-13 17:38:02.877704 7fe9bae887a0 20 read_log 194215'19413 (194215'19412) delete 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.2174595.0:78476 0.000000 2014-08-13 17:38:02.877715 7fe9bae887a0 20 read_log 194215'19414 (192888'18983) clean b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by osd.4.0:1873 2014-08-12 15:42:03.333740 2014-08-13 17:38:02.877726 7fe9bae887a0 20 read_log 194215'19415 (194215'19414) delete b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.2174595.0:78478 0.000000 2014-08-13 17:38:02.877737 7fe9bae887a0 20 read_log 194215'19416 (192888'19019) clean b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by osd.4.0:1875 2014-08-12 15:42:03.342318 2014-08-13 17:38:02.877749 7fe9bae887a0 20 read_log 194215'19417 (194215'19416) delete b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.2174595.0:78480 0.000000 2014-08-13 17:38:02.877760 7fe9bae887a0 20 read_log 194215'19418 (192888'19170) clean a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by osd.4.0:1877 2014-08-12 15:42:03.350536 2014-08-13 17:38:02.877771 7fe9bae887a0 20 read_log 194215'19419 (194215'19418) delete a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.2174595.0:78482 0.000000 2014-08-13 17:38:02.877782 7fe9bae887a0 20 read_log 194215'19420 (192888'19001) clean f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by osd.4.0:1879 2014-08-12 15:42:03.391147 2014-08-13 17:38:02.877794 7fe9bae887a0 20 read_log 194215'19421 (194215'19420) delete f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.2174595.0:78484 0.000000 2014-08-13 17:38:02.877804 7fe9bae887a0 20 read_log 194215'19422 (193305'19307) clean 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by osd.4.0:1881 2014-08-12 15:42:03.399966 2014-08-13 17:38:02.877816 7fe9bae887a0 20 read_log 194215'19423 (194215'19422) delete 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.2174595.0:78486 0.000000 2014-08-13 17:38:02.877827 7fe9bae887a0 20 read_log 194215'19424 (193305'19287) clean 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by osd.4.0:1883 2014-08-12 15:42:03.408110 2014-08-13 17:38:02.877839 7fe9bae887a0 20 read_log 194215'19425 (194215'19424) delete 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.2174595.0:78488 0.000000 2014-08-13 17:38:02.877849 7fe9bae887a0 20 read_log 194215'19426 (192888'19167) clean 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by osd.4.0:1885 2014-08-12 15:42:03.416773 2014-08-13 17:38:02.877865 7fe9bae887a0 20 read_log 194215'19427 (194215'19426) delete 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.2174595.0:78490 0.000000 2014-08-13 17:38:02.877882 7fe9bae887a0 20 read_log 194215'19428 (192888'18998) clean 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by osd.4.0:1887 2014-08-12 15:42:03.424982 2014-08-13 17:38:02.877895 7fe9bae887a0 20 read_log 194215'19429 (194215'19428) delete 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.2174595.0:78492 0.000000 2014-08-13 17:38:02.877906 7fe9bae887a0 20 read_log 194215'19430 (192888'19207) clean 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by osd.4.0:1889 2014-08-12 15:42:03.433536 2014-08-13 17:38:02.877918 7fe9bae887a0 20 read_log 194215'19431 (194215'19430) delete 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.2174595.0:78494 0.000000 2014-08-13 17:38:02.877928 7fe9bae887a0 20 read_log 194215'19432 (192888'19195) clean 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by osd.4.0:1891 2014-08-12 15:42:03.442337 2014-08-13 17:38:02.877940 7fe9bae887a0 20 read_log 194215'19433 (194215'19432) delete 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.2174595.0:78496 0.000000 2014-08-13 17:38:02.877951 7fe9bae887a0 20 read_log 194215'19434 (192888'19007) clean 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by osd.4.0:1893 2014-08-12 15:42:03.452247 2014-08-13 17:38:02.877963 7fe9bae887a0 20 read_log 194215'19435 (194215'19434) delete 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.2174595.0:78498 0.000000 2014-08-13 17:38:02.877973 7fe9bae887a0 20 read_log 194215'19436 (193305'19312) clean c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by osd.4.0:1895 2014-08-12 15:42:03.461070 2014-08-13 17:38:02.877985 7fe9bae887a0 20 read_log 194215'19437 (194215'19436) delete c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.2174595.0:78500 0.000000 2014-08-13 17:38:02.878015 7fe9bae887a0 20 read_log 194215'19438 (192888'19163) clean 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by osd.4.0:1897 2014-08-12 15:42:03.469493 2014-08-13 17:38:02.878027 7fe9bae887a0 20 read_log 194215'19439 (194215'19438) delete 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.2174595.0:78502 0.000000 2014-08-13 17:38:02.878038 7fe9bae887a0 20 read_log 194215'19440 (193305'19296) clean 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by osd.4.0:1899 2014-08-12 15:42:03.479493 2014-08-13 17:38:02.878050 7fe9bae887a0 20 read_log 194215'19441 (194215'19440) delete 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.2174595.0:78504 0.000000 2014-08-13 17:38:02.878061 7fe9bae887a0 20 read_log 194215'19442 (192888'19209) clean 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by osd.4.0:1901 2014-08-12 15:42:03.487344 2014-08-13 17:38:02.878073 7fe9bae887a0 20 read_log 194215'19443 (194215'19442) delete 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.2174595.0:78506 0.000000 2014-08-13 17:38:02.878083 7fe9bae887a0 20 read_log 194215'19444 (193305'19295) clean 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by osd.4.0:1903 2014-08-12 15:42:03.495520 2014-08-13 17:38:02.878096 7fe9bae887a0 20 read_log 194215'19445 (194215'19444) delete 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.2174595.0:78508 0.000000 2014-08-13 17:38:02.878106 7fe9bae887a0 20 read_log 194215'19446 (192888'18984) clean 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by osd.4.0:1905 2014-08-12 15:42:03.503831 2014-08-13 17:38:02.878120 7fe9bae887a0 20 read_log 194215'19447 (194215'19446) delete 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.2174595.0:78510 0.000000 2014-08-13 17:38:02.878142 7fe9bae887a0 20 read_log 194215'19448 (193305'19315) clean 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by osd.4.0:1907 2014-08-12 15:42:03.511551 2014-08-13 17:38:02.878160 7fe9bae887a0 20 read_log 194215'19449 (194215'19448) delete 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.2174595.0:78512 0.000000 2014-08-13 17:38:02.878171 7fe9bae887a0 20 read_log 194215'19450 (193305'19311) clean 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by osd.4.0:1909 2014-08-12 15:42:03.520627 2014-08-13 17:38:02.878183 7fe9bae887a0 20 read_log 194215'19451 (194215'19450) delete 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.2174595.0:78514 0.000000 2014-08-13 17:38:02.878194 7fe9bae887a0 20 read_log 194215'19452 (192888'18995) clean 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by osd.4.0:1911 2014-08-12 15:42:03.528525 2014-08-13 17:38:02.878206 7fe9bae887a0 20 read_log 194215'19453 (194215'19452) delete 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.2174595.0:78516 0.000000 2014-08-13 17:38:02.878217 7fe9bae887a0 20 read_log 194215'19454 (192888'18978) clean 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by osd.4.0:1913 2014-08-12 15:42:03.549893 2014-08-13 17:38:02.878229 7fe9bae887a0 20 read_log 194215'19455 (194215'19454) delete 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.2174595.0:78518 0.000000 2014-08-13 17:38:02.878239 7fe9bae887a0 20 read_log 194215'19456 (192888'19192) clean 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by osd.4.0:1915 2014-08-12 15:42:03.558894 2014-08-13 17:38:02.878251 7fe9bae887a0 20 read_log 194215'19457 (194215'19456) delete 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.2174595.0:78520 0.000000 2014-08-13 17:38:02.878262 7fe9bae887a0 20 read_log 194215'19458 (193305'19290) clean be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by osd.4.0:1917 2014-08-12 15:42:03.567317 2014-08-13 17:38:02.878274 7fe9bae887a0 20 read_log 194215'19459 (194215'19458) delete be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.2174595.0:78522 0.000000 2014-08-13 17:38:02.878284 7fe9bae887a0 20 read_log 194215'19460 (192888'18993) clean e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by osd.4.0:1919 2014-08-12 15:42:03.583268 2014-08-13 17:38:02.878296 7fe9bae887a0 20 read_log 194215'19461 (194215'19460) delete e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.2174595.0:78524 0.000000 2014-08-13 17:38:02.878307 7fe9bae887a0 20 read_log 194215'19462 (193305'19313) clean 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by osd.4.0:1921 2014-08-12 15:42:03.621444 2014-08-13 17:38:02.878319 7fe9bae887a0 20 read_log 194215'19463 (194215'19462) delete 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.2174595.0:78526 0.000000 2014-08-13 17:38:02.878329 7fe9bae887a0 20 read_log 194215'19464 (192888'19223) clean 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by osd.4.0:1923 2014-08-12 15:42:03.632567 2014-08-13 17:38:02.878341 7fe9bae887a0 20 read_log 194215'19465 (194215'19464) delete 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.2174595.0:78528 0.000000 2014-08-13 17:38:02.878354 7fe9bae887a0 20 read_log 194215'19466 (192888'18996) clean 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by osd.4.0:1925 2014-08-12 15:42:03.642046 2014-08-13 17:38:02.878367 7fe9bae887a0 20 read_log 194215'19467 (194215'19466) delete 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.2174595.0:78530 0.000000 2014-08-13 17:38:02.878386 7fe9bae887a0 20 read_log 194215'19468 (192888'19222) clean d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by osd.4.0:1927 2014-08-12 15:42:03.650019 2014-08-13 17:38:02.878399 7fe9bae887a0 20 read_log 194215'19469 (194215'19468) delete d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.2174595.0:78532 0.000000 2014-08-13 17:38:02.878410 7fe9bae887a0 20 read_log 194215'19470 (192888'19181) clean 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by osd.4.0:1929 2014-08-12 15:42:03.657986 2014-08-13 17:38:02.878422 7fe9bae887a0 20 read_log 194215'19471 (194215'19470) delete 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.2174595.0:78534 0.000000 2014-08-13 17:38:02.878432 7fe9bae887a0 20 read_log 194215'19472 (193305'19304) clean fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by osd.4.0:1931 2014-08-12 15:42:03.666503 2014-08-13 17:38:02.878445 7fe9bae887a0 20 read_log 194215'19473 (194215'19472) delete fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.2174595.0:78536 0.000000 2014-08-13 17:38:02.878455 7fe9bae887a0 20 read_log 194215'19474 (193305'19303) clean c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by osd.4.0:1933 2014-08-12 15:42:03.674105 2014-08-13 17:38:02.878467 7fe9bae887a0 20 read_log 194215'19475 (194215'19474) delete c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.2174595.0:78538 0.000000 2014-08-13 17:38:02.878478 7fe9bae887a0 20 read_log 194215'19476 (193305'19297) clean eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by osd.4.0:1935 2014-08-12 15:42:03.682807 2014-08-13 17:38:02.878490 7fe9bae887a0 20 read_log 194215'19477 (194215'19476) delete eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.2174595.0:78540 0.000000 2014-08-13 17:38:02.878500 7fe9bae887a0 20 read_log 194215'19478 (192888'19179) clean e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by osd.4.0:1937 2014-08-12 15:42:03.691401 2014-08-13 17:38:02.878512 7fe9bae887a0 20 read_log 194215'19479 (194215'19478) delete e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.2174595.0:78542 0.000000 2014-08-13 17:38:02.878523 7fe9bae887a0 20 read_log 194215'19480 (192888'19012) clean d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by osd.4.0:1939 2014-08-12 15:42:03.699012 2014-08-13 17:38:02.878534 7fe9bae887a0 20 read_log 194215'19481 (194215'19480) delete d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.2174595.0:78544 0.000000 2014-08-13 17:38:02.878545 7fe9bae887a0 20 read_log 194215'19482 (192888'19205) clean efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by osd.4.0:1941 2014-08-12 15:42:03.707183 2014-08-13 17:38:02.878557 7fe9bae887a0 20 read_log 194215'19483 (194215'19482) delete efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.2174595.0:78546 0.000000 2014-08-13 17:38:02.878568 7fe9bae887a0 20 read_log 194215'19484 (192888'19190) clean 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by osd.4.0:1943 2014-08-12 15:42:03.715814 2014-08-13 17:38:02.878580 7fe9bae887a0 20 read_log 194215'19485 (194215'19484) delete 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.2174595.0:78548 0.000000 2014-08-13 17:38:02.878590 7fe9bae887a0 20 read_log 194215'19486 (192888'19182) clean 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by osd.4.0:1945 2014-08-12 15:42:03.723991 2014-08-13 17:38:02.878602 7fe9bae887a0 20 read_log 194215'19487 (194215'19486) delete 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.2174595.0:78550 0.000000 2014-08-13 17:38:02.878635 7fe9bae887a0 20 read_log 194215'19488 (193305'19305) clean 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by osd.4.0:1947 2014-08-12 15:42:03.733040 2014-08-13 17:38:02.878649 7fe9bae887a0 20 read_log 194215'19489 (194215'19488) delete 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.2174595.0:78552 0.000000 2014-08-13 17:38:02.878660 7fe9bae887a0 20 read_log 194215'19490 (192888'19002) clean 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by osd.4.0:1949 2014-08-12 15:42:03.741537 2014-08-13 17:38:02.878672 7fe9bae887a0 20 read_log 194215'19491 (194215'19490) delete 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.2174595.0:78554 0.000000 2014-08-13 17:38:02.878683 7fe9bae887a0 20 read_log 194215'19492 (192888'19171) clean 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by osd.4.0:1951 2014-08-12 15:42:03.749987 2014-08-13 17:38:02.878695 7fe9bae887a0 20 read_log 194215'19493 (194215'19492) delete 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.2174595.0:78556 0.000000 2014-08-13 17:38:02.878705 7fe9bae887a0 20 read_log 194215'19494 (192888'19226) clean e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by osd.4.0:1953 2014-08-12 15:42:03.781877 2014-08-13 17:38:02.878718 7fe9bae887a0 20 read_log 194215'19495 (194215'19494) delete e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.2174595.0:78558 0.000000 2014-08-13 17:38:02.878728 7fe9bae887a0 20 read_log 194215'19496 (193305'19299) clean 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by osd.4.0:1955 2014-08-12 15:42:03.790909 2014-08-13 17:38:02.878740 7fe9bae887a0 20 read_log 194215'19497 (194215'19496) delete 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.2174595.0:78560 0.000000 2014-08-13 17:38:02.878751 7fe9bae887a0 20 read_log 194215'19498 (192888'19004) clean 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by osd.4.0:1957 2014-08-12 15:42:03.798879 2014-08-13 17:38:02.878763 7fe9bae887a0 20 read_log 194215'19499 (194215'19498) delete 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.2174595.0:78562 0.000000 2014-08-13 17:38:02.878773 7fe9bae887a0 20 read_log 194215'19500 (192888'19166) clean 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by osd.4.0:1959 2014-08-12 15:42:03.807284 2014-08-13 17:38:02.878785 7fe9bae887a0 20 read_log 194215'19501 (194215'19500) delete 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.2174595.0:78564 0.000000 2014-08-13 17:38:02.878796 7fe9bae887a0 20 read_log 194215'19502 (193305'19300) clean 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by osd.4.0:1961 2014-08-12 15:42:03.815401 2014-08-13 17:38:02.878808 7fe9bae887a0 20 read_log 194215'19503 (194215'19502) delete 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.2174595.0:78566 0.000000 2014-08-13 17:38:02.878818 7fe9bae887a0 20 read_log 194215'19504 (193305'19302) clean 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by osd.4.0:1963 2014-08-12 15:42:03.825342 2014-08-13 17:38:02.878830 7fe9bae887a0 20 read_log 194215'19505 (194215'19504) delete 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.2174595.0:78568 0.000000 2014-08-13 17:38:02.878841 7fe9bae887a0 20 read_log 194215'19506 (192888'19206) clean e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by osd.4.0:1965 2014-08-12 15:42:03.833221 2014-08-13 17:38:02.878853 7fe9bae887a0 20 read_log 194215'19507 (194215'19506) delete e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.2174595.0:78570 0.000000 2014-08-13 17:38:02.878871 7fe9bae887a0 20 read_log 194215'19508 (192888'18980) clean 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by osd.4.0:1967 2014-08-12 15:42:03.842093 2014-08-13 17:38:02.878883 7fe9bae887a0 20 read_log 194215'19509 (194215'19508) delete 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.2174595.0:78572 0.000000 2014-08-13 17:38:02.878894 7fe9bae887a0 20 read_log 194215'19510 (192888'19197) clean 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by osd.4.0:1969 2014-08-12 15:42:03.850658 2014-08-13 17:38:02.878906 7fe9bae887a0 20 read_log 194215'19511 (194215'19510) delete 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.2174595.0:78574 0.000000 2014-08-13 17:38:02.878916 7fe9bae887a0 20 read_log 194215'19512 (192888'19164) clean e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by osd.4.0:1971 2014-08-12 15:42:03.858539 2014-08-13 17:38:02.878928 7fe9bae887a0 20 read_log 194215'19513 (194215'19512) delete e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.2174595.0:78576 0.000000 2014-08-13 17:38:02.878939 7fe9bae887a0 20 read_log 194215'19514 (194215'19514) modify 181/hit_set_24.181_archive_2014-08-12 15:42:02.779513_2014-08-12 17:24:44.874495/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.874798 2014-08-13 17:38:02.878951 7fe9bae887a0 20 read_log 194215'19515 (194215'19316) delete 181/hit_set_24.181_archive_2014-08-12 15:32:25.863362_2014-08-12 15:42:02.778991/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.874798 2014-08-13 17:38:02.878963 7fe9bae887a0 20 read_log 194215'19516 (0'0) promote ebdcd581/benchmark_data_storage0111-ib.pocstor_34628_object1261/head//24 by osd.4.0:5246 2014-08-12 17:24:44.880964 2014-08-13 17:38:02.878976 7fe9bae887a0 20 read_log 194215'19517 (194215'19516) modify ebdcd581/benchmark_data_storage0111-ib.pocstor_34628_object1261/head//24 by client.2193759.0:1262 2014-08-12 17:24:44.864634 2014-08-13 17:38:02.878988 7fe9bae887a0 20 read_log 194215'19518 (0'0) promote baad81/benchmark_data_storage0111-ib.pocstor_34628_object1494/head//24 by osd.4.0:5250 2014-08-12 17:24:47.139114 2014-08-13 17:38:02.879006 7fe9bae887a0 20 read_log 194215'19519 (194215'19518) modify baad81/benchmark_data_storage0111-ib.pocstor_34628_object1494/head//24 by client.2193759.0:1495 2014-08-12 17:24:47.129419 2014-08-13 17:38:02.879018 7fe9bae887a0 20 read_log 194370'19520 (0'0) promote c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by osd.320.0:1625 2014-08-12 20:30:17.915230 2014-08-13 17:38:02.879030 7fe9bae887a0 20 read_log 194370'19521 (194370'19520) modify c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by client.1508531.0:942 2014-08-12 20:30:17.906537 2014-08-13 17:38:02.879042 7fe9bae887a0 20 read_log 194370'19522 (0'0) promote 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by osd.320.0:1631 2014-08-12 20:30:20.614233 2014-08-13 17:38:02.879054 7fe9bae887a0 20 read_log 194370'19523 (194370'19522) modify 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by client.1508531.0:1665 2014-08-12 20:30:20.606550 2014-08-13 17:38:02.879066 7fe9bae887a0 20 read_log 194370'19524 (0'0) promote b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by osd.320.0:1639 2014-08-12 20:30:22.470096 2014-08-13 17:38:02.879078 7fe9bae887a0 20 read_log 194370'19525 (194370'19524) modify b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by client.1508531.0:2152 2014-08-12 20:30:22.461892 2014-08-13 17:38:02.879090 7fe9bae887a0 20 read_log 194370'19526 (0'0) promote 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by osd.320.0:1647 2014-08-12 20:30:24.118172 2014-08-13 17:38:02.879111 7fe9bae887a0 20 read_log 194370'19527 (194370'19526) modify 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by client.1508531.0:2607 2014-08-12 20:30:24.111564 2014-08-13 17:38:02.879124 7fe9bae887a0 20 read_log 194370'19528 (0'0) promote 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by osd.320.0:1651 2014-08-12 20:30:24.482966 2014-08-13 17:38:02.879136 7fe9bae887a0 20 read_log 194370'19529 (194370'19528) modify 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by client.1508531.0:2707 2014-08-12 20:30:24.474364 2014-08-13 17:38:02.879148 7fe9bae887a0 20 read_log 194370'19530 (0'0) promote 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by osd.320.0:1653 2014-08-12 20:30:24.773221 2014-08-13 17:38:02.879183 7fe9bae887a0 20 read_log 194370'19531 (194370'19530) modify 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by client.1508531.0:2787 2014-08-12 20:30:24.764784 2014-08-13 17:38:02.879197 7fe9bae887a0 20 read_log 194370'19532 (0'0) promote 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by osd.320.0:1663 2014-08-12 20:30:27.238370 2014-08-13 17:38:02.879210 7fe9bae887a0 20 read_log 194370'19533 (194370'19532) modify 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by client.1508531.0:3456 2014-08-12 20:30:27.230589 2014-08-13 17:38:02.879222 7fe9bae887a0 20 read_log 194370'19534 (0'0) promote e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by osd.320.0:1665 2014-08-12 20:30:28.763841 2014-08-13 17:38:02.879234 7fe9bae887a0 20 read_log 194370'19535 (194370'19534) modify e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by client.1508531.0:3854 2014-08-12 20:30:28.755859 2014-08-13 17:38:02.879246 7fe9bae887a0 20 read_log 194370'19536 (0'0) promote b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by osd.320.0:1673 2014-08-12 20:30:30.376805 2014-08-13 17:38:02.879258 7fe9bae887a0 20 read_log 194370'19537 (194370'19536) modify b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by client.1508531.0:4277 2014-08-12 20:30:30.369189 2014-08-13 17:38:02.879285 7fe9bae887a0 20 read_log 194370'19538 (0'0) promote d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by osd.320.0:1675 2014-08-12 20:30:30.577055 2014-08-13 17:38:02.879298 7fe9bae887a0 20 read_log 194370'19539 (194370'19538) modify d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by client.1508531.0:4333 2014-08-12 20:30:30.570437 2014-08-13 17:38:02.879311 7fe9bae887a0 20 read_log 194370'19540 (0'0) promote 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by osd.320.0:1695 2014-08-12 20:30:35.976511 2014-08-13 17:38:02.879323 7fe9bae887a0 20 read_log 194370'19541 (194370'19540) modify 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by client.1508531.0:5623 2014-08-12 20:30:35.968041 2014-08-13 17:38:02.879335 7fe9bae887a0 20 read_log 194370'19542 (0'0) promote 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by osd.320.0:1733 2014-08-12 20:31:08.641394 2014-08-13 17:38:02.879347 7fe9bae887a0 20 read_log 194370'19543 (194370'19542) modify 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by client.1508531.0:11554 2014-08-12 20:31:08.632409 2014-08-13 17:38:02.879360 7fe9bae887a0 20 read_log 194377'19544 (0'0) promote 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by osd.320.0:1753 2014-08-12 20:32:57.876969 2014-08-13 17:38:02.879371 7fe9bae887a0 20 read_log 194377'19545 (194377'19544) modify 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by client.2231190.0:170 2014-08-12 20:32:57.868261 2014-08-13 17:38:02.879384 7fe9bae887a0 20 read_log 194377'19546 (0'0) promote a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by osd.320.0:1755 2014-08-12 20:32:58.064153 2014-08-13 17:38:02.879403 7fe9bae887a0 20 read_log 194377'19547 (194377'19546) modify a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by client.2231190.0:218 2014-08-12 20:32:58.056121 2014-08-13 17:38:02.879420 7fe9bae887a0 20 read_log 194377'19548 (0'0) promote 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by osd.320.0:1759 2014-08-12 20:32:59.189241 2014-08-13 17:38:02.879432 7fe9bae887a0 20 read_log 194377'19549 (194377'19548) modify 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by client.2231190.0:463 2014-08-12 20:32:59.179609 2014-08-13 17:38:02.879444 7fe9bae887a0 20 read_log 194377'19550 (0'0) promote 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by osd.320.0:1761 2014-08-12 20:32:59.678978 2014-08-13 17:38:02.879456 7fe9bae887a0 20 read_log 194377'19551 (194377'19550) modify 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by client.2231190.0:575 2014-08-12 20:32:59.671033 2014-08-13 17:38:02.879468 7fe9bae887a0 20 read_log 194377'19552 (0'0) promote 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by osd.320.0:1783 2014-08-12 20:33:07.125458 2014-08-13 17:38:02.879480 7fe9bae887a0 20 read_log 194377'19553 (194377'19552) modify 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by client.2231190.0:2479 2014-08-12 20:33:07.118227 2014-08-13 17:38:02.879492 7fe9bae887a0 20 read_log 194383'19554 (0'0) promote d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by osd.320.0:1821 2014-08-12 20:34:01.175476 2014-08-13 17:38:02.879504 7fe9bae887a0 20 read_log 194383'19555 (194383'19554) modify d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by client.2231190.0:5953 2014-08-12 20:34:01.166996 2014-08-13 17:38:02.879516 7fe9bae887a0 20 read_log 194383'19556 (0'0) promote 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by osd.320.0:1825 2014-08-12 20:34:02.428053 2014-08-13 17:38:02.879527 7fe9bae887a0 20 read_log 194383'19557 (194383'19556) modify 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by client.2231190.0:6194 2014-08-12 20:34:02.418244 2014-08-13 17:38:02.879539 7fe9bae887a0 20 read_log 194383'19558 (0'0) promote a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by osd.320.0:1829 2014-08-12 20:34:06.630301 2014-08-13 17:38:02.879551 7fe9bae887a0 20 read_log 194383'19559 (194383'19558) modify a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by client.2231190.0:6670 2014-08-12 20:34:06.621019 2014-08-13 17:38:02.879563 7fe9bae887a0 20 read_log 194393'19560 (0'0) promote 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by osd.320.0:1856 2014-08-12 20:52:32.756836 2014-08-13 17:38:02.879575 7fe9bae887a0 20 read_log 194393'19561 (194393'19560) modify 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by client.1064422.0:64 2014-08-12 20:52:32.746227 2014-08-13 17:38:02.879587 7fe9bae887a0 20 read_log 194416'19562 (0'0) promote cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by osd.320.0:1982 2014-08-13 15:28:31.486966 2014-08-13 17:38:02.879599 7fe9bae887a0 20 read_log 194416'19563 (194416'19562) modify cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by client.2441958.0:307 2014-08-13 15:28:31.477555 2014-08-13 17:38:02.879611 7fe9bae887a0 20 read_log 194416'19564 (0'0) promote 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by osd.320.0:1984 2014-08-13 15:28:32.180083 2014-08-13 17:38:02.879623 7fe9bae887a0 20 read_log 194416'19565 (194416'19564) modify 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by client.2441958.0:450 2014-08-13 15:28:32.171608 2014-08-13 17:38:02.879643 7fe9bae887a0 20 read_log 194416'19566 (0'0) promote 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by osd.320.0:1993 2014-08-13 15:28:34.173447 2014-08-13 17:38:02.879656 7fe9bae887a0 20 read_log 194416'19567 (194416'19566) modify 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by client.2441958.0:924 2014-08-13 15:28:34.164486 2014-08-13 17:38:02.879668 7fe9bae887a0 20 read_log 194416'19568 (0'0) promote 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by osd.320.0:1999 2014-08-13 15:28:36.812163 2014-08-13 17:38:02.879680 7fe9bae887a0 20 read_log 194416'19569 (194416'19568) modify 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by client.2441958.0:1607 2014-08-13 15:28:36.804176 2014-08-13 17:38:02.879692 7fe9bae887a0 20 read_log 194416'19570 (0'0) promote 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by osd.320.0:2001 2014-08-13 15:28:38.765628 2014-08-13 17:38:02.879704 7fe9bae887a0 20 read_log 194416'19571 (194416'19570) modify 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by client.2441958.0:2128 2014-08-13 15:28:38.757016 2014-08-13 17:38:02.879716 7fe9bae887a0 20 read_log 194416'19572 (0'0) promote c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by osd.320.0:2018 2014-08-13 15:28:48.445764 2014-08-13 17:38:02.879728 7fe9bae887a0 20 read_log 194416'19573 (194416'19572) modify c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by client.2441958.0:4616 2014-08-13 15:28:48.437627 2014-08-13 17:38:02.879740 7fe9bae887a0 20 read_log 194416'19574 (0'0) promote ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by osd.320.0:2032 2014-08-13 15:28:56.047454 2014-08-13 17:38:02.879752 7fe9bae887a0 20 read_log 194416'19575 (194416'19574) modify ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by client.2441958.0:6548 2014-08-13 15:28:56.037366 2014-08-13 17:38:02.879765 7fe9bae887a0 20 read_log 194416'19576 (0'0) promote b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by osd.320.0:2040 2014-08-13 15:29:01.049211 2014-08-13 17:38:02.879777 7fe9bae887a0 20 read_log 194416'19577 (194416'19576) modify b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by client.2441958.0:7871 2014-08-13 15:29:01.039824 2014-08-13 17:38:02.879790 7fe9bae887a0 20 read_log 194416'19578 (0'0) promote 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by osd.320.0:2042 2014-08-13 15:29:10.111280 2014-08-13 17:38:02.879802 7fe9bae887a0 20 read_log 194416'19579 (194416'19578) modify 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by client.2441958.0:10175 2014-08-13 15:29:10.101106 2014-08-13 17:38:02.879814 7fe9bae887a0 20 read_log 194416'19580 (0'0) promote 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by osd.320.0:2064 2014-08-13 15:29:15.188096 2014-08-13 17:38:02.879826 7fe9bae887a0 20 read_log 194416'19581 (194416'19580) modify 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by client.2441958.0:11424 2014-08-13 15:29:15.179344 2014-08-13 17:38:02.879838 7fe9bae887a0 20 read_log 194416'19582 (0'0) promote b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by osd.320.0:2070 2014-08-13 15:29:18.333781 2014-08-13 17:38:02.879850 7fe9bae887a0 20 read_log 194416'19583 (194416'19582) modify b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by client.2441958.0:12162 2014-08-13 15:29:18.326088 2014-08-13 17:38:02.879862 7fe9bae887a0 20 read_log 194416'19584 (0'0) promote 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by osd.320.0:2072 2014-08-13 15:29:19.052488 2014-08-13 17:38:02.879874 7fe9bae887a0 20 read_log 194416'19585 (194416'19584) modify 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by client.2441958.0:12334 2014-08-13 15:29:19.044709 2014-08-13 17:38:02.879898 7fe9bae887a0 20 read_log 194416'19586 (0'0) promote 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by osd.320.0:2074 2014-08-13 15:29:19.453734 2014-08-13 17:38:02.879913 7fe9bae887a0 20 read_log 194416'19587 (194416'19586) modify 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by client.2441958.0:12437 2014-08-13 15:29:19.445291 2014-08-13 17:38:02.879925 7fe9bae887a0 20 read_log 194416'19588 (0'0) promote 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by osd.320.0:2076 2014-08-13 15:29:21.833855 2014-08-13 17:38:02.879937 7fe9bae887a0 20 read_log 194416'19589 (194416'19588) modify 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by client.2441958.0:12987 2014-08-13 15:29:21.825229 2014-08-13 17:38:02.879949 7fe9bae887a0 20 read_log 194416'19590 (0'0) promote 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by osd.320.0:2078 2014-08-13 15:29:22.879104 2014-08-13 17:38:02.879961 7fe9bae887a0 20 read_log 194416'19591 (194416'19590) modify 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by client.2441958.0:13216 2014-08-13 15:29:22.868567 2014-08-13 17:38:02.879973 7fe9bae887a0 20 read_log 194416'19592 (0'0) promote 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by osd.320.0:2084 2014-08-13 15:29:25.487783 2014-08-13 17:38:02.879987 7fe9bae887a0 20 read_log 194416'19593 (194416'19592) modify 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by client.2441958.0:13813 2014-08-13 15:29:25.476784 2014-08-13 17:38:02.880000 7fe9bae887a0 20 read_log 194416'19594 (0'0) promote a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by osd.320.0:2100 2014-08-13 15:29:41.211718 2014-08-13 17:38:02.880013 7fe9bae887a0 20 read_log 194416'19595 (194416'19594) modify a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by client.2441958.0:16568 2014-08-13 15:29:41.201401 2014-08-13 17:38:02.880025 7fe9bae887a0 20 read_log 194416'19596 (0'0) promote 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by osd.320.0:2102 2014-08-13 15:29:46.866231 2014-08-13 17:38:02.880040 7fe9bae887a0 20 read_log 194416'19597 (194416'19596) modify 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by client.2441958.0:16965 2014-08-13 15:29:46.855751 2014-08-13 17:38:02.880052 7fe9bae887a0 20 read_log 194416'19598 (0'0) promote 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by osd.320.0:2116 2014-08-13 15:30:08.730007 2014-08-13 17:38:02.880065 7fe9bae887a0 20 read_log 194416'19599 (194416'19598) modify 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by client.2441958.0:18342 2014-08-13 15:30:08.722319 2014-08-13 17:38:02.880077 7fe9bae887a0 20 read_log 194422'19600 (0'0) promote de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by osd.320.0:2130 2014-08-13 15:32:26.228968 2014-08-13 17:38:02.880091 7fe9bae887a0 20 read_log 194422'19601 (194422'19600) modify de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by client.1148203.0:2786 2014-08-13 15:32:26.218465 2014-08-13 17:38:02.880104 7fe9bae887a0 20 read_log 194422'19602 (0'0) promote 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by osd.320.0:2140 2014-08-13 15:32:32.608460 2014-08-13 17:38:02.880118 7fe9bae887a0 20 read_log 194422'19603 (194422'19602) modify 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by client.1148203.0:4421 2014-08-13 15:32:32.599714 2014-08-13 17:38:02.880130 7fe9bae887a0 20 read_log 194422'19604 (0'0) promote 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by osd.320.0:2142 2014-08-13 15:32:35.559977 2014-08-13 17:38:02.880161 7fe9bae887a0 20 read_log 194422'19605 (194422'19604) modify 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by client.1148203.0:5146 2014-08-13 15:32:35.550402 2014-08-13 17:38:02.880177 7fe9bae887a0 20 read_log 194422'19606 (0'0) promote c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by osd.320.0:2148 2014-08-13 15:32:42.224499 2014-08-13 17:38:02.880189 7fe9bae887a0 20 read_log 194422'19607 (194422'19606) modify c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by client.1148203.0:6381 2014-08-13 15:32:42.215412 2014-08-13 17:38:02.880202 7fe9bae887a0 20 read_log 194422'19608 (0'0) promote 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by osd.320.0:2160 2014-08-13 15:32:50.176861 2014-08-13 17:38:02.880214 7fe9bae887a0 20 read_log 194422'19609 (194422'19608) modify 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by client.1148203.0:7748 2014-08-13 15:32:50.168241 2014-08-13 17:38:02.880226 7fe9bae887a0 20 read_log 194422'19610 (0'0) promote ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by osd.320.0:2162 2014-08-13 15:32:51.917737 2014-08-13 17:38:02.880238 7fe9bae887a0 20 read_log 194422'19611 (194422'19610) modify ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by client.1148203.0:8035 2014-08-13 15:32:51.909955 2014-08-13 17:38:02.880250 7fe9bae887a0 20 read_log 194422'19612 (0'0) promote dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by osd.320.0:2168 2014-08-13 15:32:54.836968 2014-08-13 17:38:02.880262 7fe9bae887a0 20 read_log 194422'19613 (194422'19612) modify dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by client.1148203.0:8463 2014-08-13 15:32:54.827337 2014-08-13 17:38:02.880280 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.880733 7fe9bae887a0 10 read_log checking for missing items over interval (194370'19520,194422'19613] 2014-08-13 17:38:02.880779 7fe9bae887a0 15 read_log missing 194422'19613 (194422'19612) modify dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by client.1148203.0:8463 2014-08-13 15:32:54.827337 2014-08-13 17:38:02.880821 7fe9bae887a0 15 read_log missing 194422'19611 (194422'19610) modify ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by client.1148203.0:8035 2014-08-13 15:32:51.909955 2014-08-13 17:38:02.880858 7fe9bae887a0 15 read_log missing 194422'19609 (194422'19608) modify 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by client.1148203.0:7748 2014-08-13 15:32:50.168241 2014-08-13 17:38:02.880895 7fe9bae887a0 15 read_log missing 194422'19607 (194422'19606) modify c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by client.1148203.0:6381 2014-08-13 15:32:42.215412 2014-08-13 17:38:02.880932 7fe9bae887a0 15 read_log missing 194422'19605 (194422'19604) modify 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by client.1148203.0:5146 2014-08-13 15:32:35.550402 2014-08-13 17:38:02.880969 7fe9bae887a0 15 read_log missing 194422'19603 (194422'19602) modify 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by client.1148203.0:4421 2014-08-13 15:32:32.599714 2014-08-13 17:38:02.881010 7fe9bae887a0 15 read_log missing 194422'19601 (194422'19600) modify de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by client.1148203.0:2786 2014-08-13 15:32:26.218465 2014-08-13 17:38:02.881047 7fe9bae887a0 15 read_log missing 194416'19599 (194416'19598) modify 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by client.2441958.0:18342 2014-08-13 15:30:08.722319 2014-08-13 17:38:02.881084 7fe9bae887a0 15 read_log missing 194416'19597 (194416'19596) modify 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by client.2441958.0:16965 2014-08-13 15:29:46.855751 2014-08-13 17:38:02.881120 7fe9bae887a0 15 read_log missing 194416'19595 (194416'19594) modify a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by client.2441958.0:16568 2014-08-13 15:29:41.201401 2014-08-13 17:38:02.881162 7fe9bae887a0 15 read_log missing 194416'19593 (194416'19592) modify 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by client.2441958.0:13813 2014-08-13 15:29:25.476784 2014-08-13 17:38:02.881203 7fe9bae887a0 15 read_log missing 194416'19591 (194416'19590) modify 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by client.2441958.0:13216 2014-08-13 15:29:22.868567 2014-08-13 17:38:02.881240 7fe9bae887a0 15 read_log missing 194416'19589 (194416'19588) modify 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by client.2441958.0:12987 2014-08-13 15:29:21.825229 2014-08-13 17:38:02.881277 7fe9bae887a0 15 read_log missing 194416'19587 (194416'19586) modify 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by client.2441958.0:12437 2014-08-13 15:29:19.445291 2014-08-13 17:38:02.881314 7fe9bae887a0 15 read_log missing 194416'19585 (194416'19584) modify 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by client.2441958.0:12334 2014-08-13 15:29:19.044709 2014-08-13 17:38:02.881351 7fe9bae887a0 15 read_log missing 194416'19583 (194416'19582) modify b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by client.2441958.0:12162 2014-08-13 15:29:18.326088 2014-08-13 17:38:02.881388 7fe9bae887a0 15 read_log missing 194416'19581 (194416'19580) modify 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by client.2441958.0:11424 2014-08-13 15:29:15.179344 2014-08-13 17:38:02.881425 7fe9bae887a0 15 read_log missing 194416'19579 (194416'19578) modify 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by client.2441958.0:10175 2014-08-13 15:29:10.101106 2014-08-13 17:38:02.881462 7fe9bae887a0 15 read_log missing 194416'19577 (194416'19576) modify b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by client.2441958.0:7871 2014-08-13 15:29:01.039824 2014-08-13 17:38:02.881500 7fe9bae887a0 15 read_log missing 194416'19575 (194416'19574) modify ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by client.2441958.0:6548 2014-08-13 15:28:56.037366 2014-08-13 17:38:02.881537 7fe9bae887a0 15 read_log missing 194416'19573 (194416'19572) modify c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by client.2441958.0:4616 2014-08-13 15:28:48.437627 2014-08-13 17:38:02.881574 7fe9bae887a0 15 read_log missing 194416'19571 (194416'19570) modify 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by client.2441958.0:2128 2014-08-13 15:28:38.757016 2014-08-13 17:38:02.881610 7fe9bae887a0 15 read_log missing 194416'19569 (194416'19568) modify 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by client.2441958.0:1607 2014-08-13 15:28:36.804176 2014-08-13 17:38:02.881647 7fe9bae887a0 15 read_log missing 194416'19567 (194416'19566) modify 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by client.2441958.0:924 2014-08-13 15:28:34.164486 2014-08-13 17:38:02.881684 7fe9bae887a0 15 read_log missing 194416'19565 (194416'19564) modify 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by client.2441958.0:450 2014-08-13 15:28:32.171608 2014-08-13 17:38:02.881720 7fe9bae887a0 15 read_log missing 194416'19563 (194416'19562) modify cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by client.2441958.0:307 2014-08-13 15:28:31.477555 2014-08-13 17:38:02.881755 7fe9bae887a0 15 read_log missing 194393'19561 (194393'19560) modify 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by client.1064422.0:64 2014-08-12 20:52:32.746227 2014-08-13 17:38:02.881791 7fe9bae887a0 15 read_log missing 194383'19559 (194383'19558) modify a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by client.2231190.0:6670 2014-08-12 20:34:06.621019 2014-08-13 17:38:02.881827 7fe9bae887a0 15 read_log missing 194383'19557 (194383'19556) modify 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by client.2231190.0:6194 2014-08-12 20:34:02.418244 2014-08-13 17:38:02.881864 7fe9bae887a0 15 read_log missing 194383'19555 (194383'19554) modify d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by client.2231190.0:5953 2014-08-12 20:34:01.166996 2014-08-13 17:38:02.881900 7fe9bae887a0 15 read_log missing 194377'19553 (194377'19552) modify 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by client.2231190.0:2479 2014-08-12 20:33:07.118227 2014-08-13 17:38:02.881937 7fe9bae887a0 15 read_log missing 194377'19551 (194377'19550) modify 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by client.2231190.0:575 2014-08-12 20:32:59.671033 2014-08-13 17:38:02.881974 7fe9bae887a0 15 read_log missing 194377'19549 (194377'19548) modify 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by client.2231190.0:463 2014-08-12 20:32:59.179609 2014-08-13 17:38:02.882010 7fe9bae887a0 15 read_log missing 194377'19547 (194377'19546) modify a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by client.2231190.0:218 2014-08-12 20:32:58.056121 2014-08-13 17:38:02.882046 7fe9bae887a0 15 read_log missing 194377'19545 (194377'19544) modify 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by client.2231190.0:170 2014-08-12 20:32:57.868261 2014-08-13 17:38:02.882083 7fe9bae887a0 15 read_log missing 194370'19543 (194370'19542) modify 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by client.1508531.0:11554 2014-08-12 20:31:08.632409 2014-08-13 17:38:02.882121 7fe9bae887a0 15 read_log missing 194370'19541 (194370'19540) modify 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by client.1508531.0:5623 2014-08-12 20:30:35.968041 2014-08-13 17:38:02.882162 7fe9bae887a0 15 read_log missing 194370'19539 (194370'19538) modify d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by client.1508531.0:4333 2014-08-12 20:30:30.570437 2014-08-13 17:38:02.882200 7fe9bae887a0 15 read_log missing 194370'19537 (194370'19536) modify b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by client.1508531.0:4277 2014-08-12 20:30:30.369189 2014-08-13 17:38:02.882238 7fe9bae887a0 15 read_log missing 194370'19535 (194370'19534) modify e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by client.1508531.0:3854 2014-08-12 20:30:28.755859 2014-08-13 17:38:02.882275 7fe9bae887a0 15 read_log missing 194370'19533 (194370'19532) modify 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by client.1508531.0:3456 2014-08-12 20:30:27.230589 2014-08-13 17:38:02.882312 7fe9bae887a0 15 read_log missing 194370'19531 (194370'19530) modify 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by client.1508531.0:2787 2014-08-12 20:30:24.764784 2014-08-13 17:38:02.882350 7fe9bae887a0 15 read_log missing 194370'19529 (194370'19528) modify 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by client.1508531.0:2707 2014-08-12 20:30:24.474364 2014-08-13 17:38:02.882387 7fe9bae887a0 15 read_log missing 194370'19527 (194370'19526) modify 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by client.1508531.0:2607 2014-08-12 20:30:24.111564 2014-08-13 17:38:02.882424 7fe9bae887a0 15 read_log missing 194370'19525 (194370'19524) modify b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by client.1508531.0:2152 2014-08-12 20:30:22.461892 2014-08-13 17:38:02.882461 7fe9bae887a0 15 read_log missing 194370'19523 (194370'19522) modify 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by client.1508531.0:1665 2014-08-12 20:30:20.606550 2014-08-13 17:38:02.882498 7fe9bae887a0 15 read_log missing 194370'19521 (194370'19520) modify c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by client.1508531.0:942 2014-08-12 20:30:17.906537 2014-08-13 17:38:02.882509 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.882537 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_loaded 2014-08-13 17:38:02.882555 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Initial 0.013349 0 0.000000 2014-08-13 17:38:02.882567 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset 2014-08-13 17:38:02.882577 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] set_last_peering_reset 194478 2014-08-13 17:38:02.882586 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] log((192834'17288,194422'19613], crt=194422'19613) 2014-08-13 17:38:02.882597 7fe9bae887a0 10 osd.4 194478 pgid 24.1bb coll 24.1bb_head 2014-08-13 17:38:02.882706 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.1bb 2014-08-13 17:38:02.882718 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.882751 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1bb(unlocked)] enter Initial 2014-08-13 17:38:02.882762 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1bb(unlocked)] enter NotTrimming 2014-08-13 17:38:02.882884 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.882992 7fe9bae887a0 20 read_log 192718'9696 (192718'9695) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:5 2014-08-06 14:51:11.570229 2014-08-13 17:38:02.883024 7fe9bae887a0 20 read_log 192718'9697 (192718'9696) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:6 2014-08-06 14:51:11.597604 2014-08-13 17:38:02.883040 7fe9bae887a0 20 read_log 192718'9698 (192718'9697) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:7 2014-08-06 14:51:11.653681 2014-08-13 17:38:02.883053 7fe9bae887a0 20 read_log 192718'9699 (192718'9698) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:8 2014-08-06 14:51:11.680474 2014-08-13 17:38:02.883065 7fe9bae887a0 20 read_log 192718'9700 (192718'9699) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:9 2014-08-06 14:51:11.775367 2014-08-13 17:38:02.883093 7fe9bae887a0 20 read_log 192718'9701 (192718'9700) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:10 2014-08-06 14:51:11.801363 2014-08-13 17:38:02.883108 7fe9bae887a0 20 read_log 192718'9702 (192718'9701) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:11 2014-08-06 14:51:11.826722 2014-08-13 17:38:02.883120 7fe9bae887a0 20 read_log 192718'9703 (192718'9702) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:12 2014-08-06 14:51:11.979621 2014-08-13 17:38:02.883132 7fe9bae887a0 20 read_log 192718'9704 (192718'9703) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:13 2014-08-06 14:51:12.044660 2014-08-13 17:38:02.883144 7fe9bae887a0 20 read_log 192718'9705 (192718'9704) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:14 2014-08-06 14:51:12.069439 2014-08-13 17:38:02.883163 7fe9bae887a0 20 read_log 192718'9706 (192718'9705) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:15 2014-08-06 14:51:12.129877 2014-08-13 17:38:02.883175 7fe9bae887a0 20 read_log 192718'9707 (192718'9706) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:16 2014-08-06 14:51:12.152051 2014-08-13 17:38:02.883187 7fe9bae887a0 20 read_log 192718'9708 (192718'9707) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:17 2014-08-06 14:51:12.212285 2014-08-13 17:38:02.883199 7fe9bae887a0 20 read_log 192718'9709 (192718'9708) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:18 2014-08-06 14:51:12.235832 2014-08-13 17:38:02.883211 7fe9bae887a0 20 read_log 192718'9710 (192718'9709) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:19 2014-08-06 14:51:12.303282 2014-08-13 17:38:02.883223 7fe9bae887a0 20 read_log 192718'9711 (192718'9710) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:20 2014-08-06 14:51:12.327205 2014-08-13 17:38:02.883234 7fe9bae887a0 20 read_log 192718'9712 (192718'9711) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:21 2014-08-06 14:51:12.391765 2014-08-13 17:38:02.883246 7fe9bae887a0 20 read_log 192718'9713 (192718'9712) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:22 2014-08-06 14:51:12.416635 2014-08-13 17:38:02.883258 7fe9bae887a0 20 read_log 192718'9714 (192718'9713) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:23 2014-08-06 14:51:12.477402 2014-08-13 17:38:02.883270 7fe9bae887a0 20 read_log 192718'9715 (192718'9714) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:24 2014-08-06 14:51:12.499198 2014-08-13 17:38:02.883282 7fe9bae887a0 20 read_log 192718'9716 (192718'9715) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:25 2014-08-06 14:51:12.589531 2014-08-13 17:38:02.883293 7fe9bae887a0 20 read_log 192718'9717 (192718'9716) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:26 2014-08-06 14:51:12.625294 2014-08-13 17:38:02.883305 7fe9bae887a0 20 read_log 192718'9718 (192718'9717) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:27 2014-08-06 14:51:12.647996 2014-08-13 17:38:02.883316 7fe9bae887a0 20 read_log 192718'9719 (192718'9718) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:28 2014-08-06 14:51:12.672217 2014-08-13 17:38:02.883328 7fe9bae887a0 20 read_log 192718'9720 (192718'9719) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:29 2014-08-06 14:51:12.726388 2014-08-13 17:38:02.883364 7fe9bae887a0 20 read_log 192718'9721 (192718'9720) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:30 2014-08-06 14:51:12.751381 2014-08-13 17:38:02.883381 7fe9bae887a0 20 read_log 192718'9722 (192718'9721) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:31 2014-08-06 14:51:12.841734 2014-08-13 17:38:02.883393 7fe9bae887a0 20 read_log 192718'9723 (192718'9722) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:32 2014-08-06 14:51:12.867699 2014-08-13 17:38:02.883405 7fe9bae887a0 20 read_log 192718'9724 (192718'9723) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:33 2014-08-06 14:51:12.952203 2014-08-13 17:38:02.883416 7fe9bae887a0 20 read_log 192718'9725 (192718'9724) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:34 2014-08-06 14:51:12.978052 2014-08-13 17:38:02.883428 7fe9bae887a0 20 read_log 192718'9726 (192718'9725) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:35 2014-08-06 14:51:13.001808 2014-08-13 17:38:02.883440 7fe9bae887a0 20 read_log 192718'9727 (192718'9726) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:36 2014-08-06 14:51:13.025358 2014-08-13 17:38:02.883452 7fe9bae887a0 20 read_log 192718'9728 (192718'9727) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:37 2014-08-06 14:51:13.206163 2014-08-13 17:38:02.883464 7fe9bae887a0 20 read_log 192718'9729 (192718'9728) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:38 2014-08-06 14:51:13.229839 2014-08-13 17:38:02.883476 7fe9bae887a0 20 read_log 192718'9730 (192718'9729) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:39 2014-08-06 14:51:13.317738 2014-08-13 17:38:02.883488 7fe9bae887a0 20 read_log 192718'9731 (192718'9730) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:40 2014-08-06 14:51:13.343139 2014-08-13 17:38:02.883500 7fe9bae887a0 20 read_log 192718'9732 (192718'9731) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:41 2014-08-06 14:51:13.418396 2014-08-13 17:38:02.883512 7fe9bae887a0 20 read_log 192718'9733 (192718'9732) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:42 2014-08-06 14:51:13.442477 2014-08-13 17:38:02.883525 7fe9bae887a0 20 read_log 192718'9734 (192718'9733) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:43 2014-08-06 14:51:13.465916 2014-08-13 17:38:02.883536 7fe9bae887a0 20 read_log 192718'9735 (192718'9734) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:44 2014-08-06 14:51:13.489250 2014-08-13 17:38:02.883548 7fe9bae887a0 20 read_log 192718'9736 (192718'9735) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:45 2014-08-06 14:51:13.545183 2014-08-13 17:38:02.883560 7fe9bae887a0 20 read_log 192718'9737 (192718'9736) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:46 2014-08-06 14:51:13.569762 2014-08-13 17:38:02.883572 7fe9bae887a0 20 read_log 192718'9738 (192718'9737) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:47 2014-08-06 14:51:13.592102 2014-08-13 17:38:02.883583 7fe9bae887a0 20 read_log 192718'9739 (192718'9738) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:48 2014-08-06 14:51:13.614915 2014-08-13 17:38:02.883595 7fe9bae887a0 20 read_log 192718'9740 (192718'9739) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:49 2014-08-06 14:51:13.695425 2014-08-13 17:38:02.883607 7fe9bae887a0 20 read_log 192718'9741 (192718'9740) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:50 2014-08-06 14:51:13.718871 2014-08-13 17:38:02.883619 7fe9bae887a0 20 read_log 192718'9742 (192718'9741) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:51 2014-08-06 14:51:13.911742 2014-08-13 17:38:02.883630 7fe9bae887a0 20 read_log 192718'9743 (192718'9742) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:52 2014-08-06 14:51:13.937172 2014-08-13 17:38:02.883642 7fe9bae887a0 20 read_log 192718'9744 (192718'9743) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:53 2014-08-06 14:51:13.961359 2014-08-13 17:38:02.883661 7fe9bae887a0 20 read_log 192718'9745 (192718'9744) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:54 2014-08-06 14:51:13.983699 2014-08-13 17:38:02.883673 7fe9bae887a0 20 read_log 192718'9746 (192718'9745) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:55 2014-08-06 14:51:14.061329 2014-08-13 17:38:02.883685 7fe9bae887a0 20 read_log 192718'9747 (192718'9746) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:56 2014-08-06 14:51:14.133308 2014-08-13 17:38:02.883697 7fe9bae887a0 20 read_log 192718'9748 (192718'9747) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:57 2014-08-06 14:51:14.201953 2014-08-13 17:38:02.883709 7fe9bae887a0 20 read_log 192718'9749 (192718'9748) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:58 2014-08-06 14:51:14.224902 2014-08-13 17:38:02.883721 7fe9bae887a0 20 read_log 192718'9750 (192718'9749) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:59 2014-08-06 14:51:14.248547 2014-08-13 17:38:02.883732 7fe9bae887a0 20 read_log 192718'9751 (192718'9750) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:60 2014-08-06 14:51:14.271249 2014-08-13 17:38:02.883744 7fe9bae887a0 20 read_log 192718'9752 (192718'9751) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:61 2014-08-06 14:51:14.358673 2014-08-13 17:38:02.883756 7fe9bae887a0 20 read_log 192718'9753 (192718'9752) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:62 2014-08-06 14:51:14.385336 2014-08-13 17:38:02.883767 7fe9bae887a0 20 read_log 192718'9754 (192718'9753) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:63 2014-08-06 14:51:14.467799 2014-08-13 17:38:02.883779 7fe9bae887a0 20 read_log 192718'9755 (192718'9754) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:64 2014-08-06 14:51:14.494113 2014-08-13 17:38:02.883790 7fe9bae887a0 20 read_log 192718'9756 (192718'9755) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:65 2014-08-06 14:51:14.595811 2014-08-13 17:38:02.883802 7fe9bae887a0 20 read_log 192718'9757 (192718'9756) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:66 2014-08-06 14:51:14.621650 2014-08-13 17:38:02.883814 7fe9bae887a0 20 read_log 192718'9758 (192718'9757) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:67 2014-08-06 14:51:14.687018 2014-08-13 17:38:02.883825 7fe9bae887a0 20 read_log 192718'9759 (192718'9758) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:68 2014-08-06 14:51:14.733342 2014-08-13 17:38:02.883837 7fe9bae887a0 20 read_log 192718'9760 (192718'9759) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:69 2014-08-06 14:51:14.757059 2014-08-13 17:38:02.883849 7fe9bae887a0 20 read_log 192718'9761 (192718'9760) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:70 2014-08-06 14:51:14.782451 2014-08-13 17:38:02.883861 7fe9bae887a0 20 read_log 192718'9762 (192718'9761) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:71 2014-08-06 14:51:14.805746 2014-08-13 17:38:02.883872 7fe9bae887a0 20 read_log 192718'9763 (192718'9762) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:72 2014-08-06 14:51:14.832474 2014-08-13 17:38:02.883886 7fe9bae887a0 20 read_log 192718'9764 (192718'9763) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:73 2014-08-06 14:51:14.883320 2014-08-13 17:38:02.883897 7fe9bae887a0 20 read_log 192718'9765 (192718'9764) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:74 2014-08-06 14:51:14.935358 2014-08-13 17:38:02.883909 7fe9bae887a0 20 read_log 192718'9766 (192718'9765) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:75 2014-08-06 14:51:15.009004 2014-08-13 17:38:02.883921 7fe9bae887a0 20 read_log 192718'9767 (192718'9766) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:76 2014-08-06 14:51:15.068489 2014-08-13 17:38:02.883932 7fe9bae887a0 20 read_log 192718'9768 (192718'9767) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:77 2014-08-06 14:51:15.154761 2014-08-13 17:38:02.883953 7fe9bae887a0 20 read_log 192718'9769 (192718'9768) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:78 2014-08-06 14:51:15.205767 2014-08-13 17:38:02.883966 7fe9bae887a0 20 read_log 192718'9770 (192718'9769) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:79 2014-08-06 14:51:15.248762 2014-08-13 17:38:02.883978 7fe9bae887a0 20 read_log 192718'9771 (192718'9770) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:80 2014-08-06 14:51:15.290874 2014-08-13 17:38:02.883989 7fe9bae887a0 20 read_log 192718'9772 (192718'9771) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:81 2014-08-06 14:51:15.341078 2014-08-13 17:38:02.884001 7fe9bae887a0 20 read_log 192718'9773 (192718'9772) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:82 2014-08-06 14:51:15.391499 2014-08-13 17:38:02.884013 7fe9bae887a0 20 read_log 192718'9774 (192718'9773) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:83 2014-08-06 14:51:15.452023 2014-08-13 17:38:02.884030 7fe9bae887a0 20 read_log 192718'9775 (192718'9774) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:84 2014-08-06 14:51:15.499944 2014-08-13 17:38:02.884042 7fe9bae887a0 20 read_log 192718'9776 (192718'9775) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:85 2014-08-06 14:51:15.561114 2014-08-13 17:38:02.884053 7fe9bae887a0 20 read_log 192718'9777 (192718'9776) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:86 2014-08-06 14:51:15.605101 2014-08-13 17:38:02.884065 7fe9bae887a0 20 read_log 192718'9778 (192718'9777) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:87 2014-08-06 14:51:15.656542 2014-08-13 17:38:02.884077 7fe9bae887a0 20 read_log 192718'9779 (192718'9778) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:88 2014-08-06 14:51:15.708122 2014-08-13 17:38:02.884089 7fe9bae887a0 20 read_log 192718'9780 (192718'9779) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:89 2014-08-06 14:51:15.772837 2014-08-13 17:38:02.884100 7fe9bae887a0 20 read_log 192718'9781 (192718'9780) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:90 2014-08-06 14:51:15.856918 2014-08-13 17:38:02.884112 7fe9bae887a0 20 read_log 192718'9782 (192718'9781) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:91 2014-08-06 14:51:19.303214 2014-08-13 17:38:02.884124 7fe9bae887a0 20 read_log 192718'9783 (192718'9782) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:92 2014-08-06 14:51:19.331714 2014-08-13 17:38:02.884136 7fe9bae887a0 20 read_log 192718'9784 (192718'9783) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:93 2014-08-06 14:51:19.366245 2014-08-13 17:38:02.884147 7fe9bae887a0 20 read_log 192718'9785 (192718'9784) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:94 2014-08-06 14:51:19.393851 2014-08-13 17:38:02.884165 7fe9bae887a0 20 read_log 192718'9786 (192718'9785) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:95 2014-08-06 14:51:19.525371 2014-08-13 17:38:02.884177 7fe9bae887a0 20 read_log 192718'9787 (192718'9786) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:96 2014-08-06 14:51:19.576047 2014-08-13 17:38:02.884189 7fe9bae887a0 20 read_log 192718'9788 (192718'9787) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:97 2014-08-06 14:51:19.603030 2014-08-13 17:38:02.884200 7fe9bae887a0 20 read_log 192718'9789 (192718'9788) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:98 2014-08-06 14:51:19.796585 2014-08-13 17:38:02.884212 7fe9bae887a0 20 read_log 192718'9790 (192718'9789) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:99 2014-08-06 14:51:19.970715 2014-08-13 17:38:02.884224 7fe9bae887a0 20 read_log 192718'9791 (192718'9790) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:100 2014-08-06 14:51:19.998178 2014-08-13 17:38:02.884236 7fe9bae887a0 20 read_log 192718'9792 (192718'9791) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:101 2014-08-06 14:51:20.044751 2014-08-13 17:38:02.884254 7fe9bae887a0 20 read_log 192718'9793 (192718'9792) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:102 2014-08-06 14:51:20.111210 2014-08-13 17:38:02.884269 7fe9bae887a0 20 read_log 192718'9794 (192718'9793) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:103 2014-08-06 14:51:20.281296 2014-08-13 17:38:02.884281 7fe9bae887a0 20 read_log 192718'9795 (192718'9794) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:104 2014-08-06 14:51:20.306003 2014-08-13 17:38:02.884293 7fe9bae887a0 20 read_log 192718'9796 (192718'9795) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:105 2014-08-06 14:51:20.336247 2014-08-13 17:38:02.884305 7fe9bae887a0 20 read_log 192718'9797 (192718'9796) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:106 2014-08-06 14:51:20.371802 2014-08-13 17:38:02.884316 7fe9bae887a0 20 read_log 192718'9798 (192718'9797) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:107 2014-08-06 14:51:20.767198 2014-08-13 17:38:02.884328 7fe9bae887a0 20 read_log 192718'9799 (192718'9798) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:108 2014-08-06 14:51:20.813758 2014-08-13 17:38:02.884340 7fe9bae887a0 20 read_log 192718'9800 (192718'9799) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:109 2014-08-06 14:51:20.839343 2014-08-13 17:38:02.884352 7fe9bae887a0 20 read_log 192718'9801 (192718'9800) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:110 2014-08-06 14:51:20.978255 2014-08-13 17:38:02.884363 7fe9bae887a0 20 read_log 192718'9802 (192718'9801) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:111 2014-08-06 14:51:21.012120 2014-08-13 17:38:02.884375 7fe9bae887a0 20 read_log 192718'9803 (192718'9802) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:112 2014-08-06 14:51:21.147593 2014-08-13 17:38:02.884387 7fe9bae887a0 20 read_log 192718'9804 (192718'9803) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:113 2014-08-06 14:51:21.184801 2014-08-13 17:38:02.884402 7fe9bae887a0 20 read_log 192718'9805 (192718'9804) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:114 2014-08-06 14:51:21.211724 2014-08-13 17:38:02.884415 7fe9bae887a0 20 read_log 192718'9806 (192718'9805) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:115 2014-08-06 14:51:21.241501 2014-08-13 17:38:02.884426 7fe9bae887a0 20 read_log 192718'9807 (192718'9806) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:116 2014-08-06 14:51:21.468656 2014-08-13 17:38:02.884440 7fe9bae887a0 20 read_log 192718'9808 (192718'9807) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:117 2014-08-06 14:51:21.498800 2014-08-13 17:38:02.884452 7fe9bae887a0 20 read_log 192718'9809 (192718'9808) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:118 2014-08-06 14:51:21.539990 2014-08-13 17:38:02.884464 7fe9bae887a0 20 read_log 192718'9810 (192718'9809) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:119 2014-08-06 14:51:21.566708 2014-08-13 17:38:02.884475 7fe9bae887a0 20 read_log 192718'9811 (192718'9810) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:120 2014-08-06 14:51:21.606489 2014-08-13 17:38:02.884487 7fe9bae887a0 20 read_log 192718'9812 (192718'9811) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:121 2014-08-06 14:51:21.796414 2014-08-13 17:38:02.884499 7fe9bae887a0 20 read_log 192718'9813 (192718'9812) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:122 2014-08-06 14:51:22.161288 2014-08-13 17:38:02.884510 7fe9bae887a0 20 read_log 192718'9814 (192718'9813) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:123 2014-08-06 14:51:22.199283 2014-08-13 17:38:02.884522 7fe9bae887a0 20 read_log 192718'9815 (192718'9814) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:124 2014-08-06 14:51:22.231580 2014-08-13 17:38:02.884534 7fe9bae887a0 20 read_log 192718'9816 (192718'9815) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:125 2014-08-06 14:51:22.274521 2014-08-13 17:38:02.884552 7fe9bae887a0 20 read_log 192718'9817 (192718'9816) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:126 2014-08-06 14:51:22.393088 2014-08-13 17:38:02.884564 7fe9bae887a0 20 read_log 192718'9818 (192718'9817) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:127 2014-08-06 14:51:22.421795 2014-08-13 17:38:02.884578 7fe9bae887a0 20 read_log 192718'9819 (192718'9818) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:128 2014-08-06 14:51:22.551853 2014-08-13 17:38:02.884591 7fe9bae887a0 20 read_log 192718'9820 (192718'9819) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:129 2014-08-06 14:51:22.576463 2014-08-13 17:38:02.884603 7fe9bae887a0 20 read_log 192718'9821 (192718'9820) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:130 2014-08-06 14:51:22.602377 2014-08-13 17:38:02.884615 7fe9bae887a0 20 read_log 192718'9822 (192718'9821) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:131 2014-08-06 14:51:22.715852 2014-08-13 17:38:02.884626 7fe9bae887a0 20 read_log 192718'9823 (192718'9822) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:132 2014-08-06 14:51:22.863429 2014-08-13 17:38:02.884638 7fe9bae887a0 20 read_log 192718'9824 (192718'9823) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:133 2014-08-06 14:51:22.900727 2014-08-13 17:38:02.884650 7fe9bae887a0 20 read_log 192718'9825 (192718'9824) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:134 2014-08-06 14:51:22.937819 2014-08-13 17:38:02.884686 7fe9bae887a0 20 read_log 192718'9826 (192718'9825) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:135 2014-08-06 14:51:22.997844 2014-08-13 17:38:02.884698 7fe9bae887a0 20 read_log 192718'9827 (192718'9826) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:136 2014-08-06 14:51:23.157507 2014-08-13 17:38:02.884710 7fe9bae887a0 20 read_log 192718'9828 (192718'9827) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:137 2014-08-06 14:51:23.183973 2014-08-13 17:38:02.884722 7fe9bae887a0 20 read_log 192718'9829 (192718'9828) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:138 2014-08-06 14:51:23.306132 2014-08-13 17:38:02.884734 7fe9bae887a0 20 read_log 192718'9830 (192718'9829) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:139 2014-08-06 14:51:23.335649 2014-08-13 17:38:02.884748 7fe9bae887a0 20 read_log 192718'9831 (192718'9830) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:140 2014-08-06 14:51:23.366506 2014-08-13 17:38:02.884760 7fe9bae887a0 20 read_log 192718'9832 (192718'9831) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:141 2014-08-06 14:51:23.453847 2014-08-13 17:38:02.884772 7fe9bae887a0 20 read_log 192718'9833 (192718'9832) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:142 2014-08-06 14:51:23.600013 2014-08-13 17:38:02.884784 7fe9bae887a0 20 read_log 192718'9834 (192718'9833) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:143 2014-08-06 14:51:23.640648 2014-08-13 17:38:02.884795 7fe9bae887a0 20 read_log 192718'9835 (192718'9834) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:144 2014-08-06 14:51:23.669549 2014-08-13 17:38:02.884807 7fe9bae887a0 20 read_log 192718'9836 (192718'9835) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:145 2014-08-06 14:51:23.838736 2014-08-13 17:38:02.884819 7fe9bae887a0 20 read_log 192718'9837 (192718'9836) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:146 2014-08-06 14:51:24.039390 2014-08-13 17:38:02.884833 7fe9bae887a0 20 read_log 192718'9838 (192718'9837) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:147 2014-08-06 14:51:24.066889 2014-08-13 17:38:02.884845 7fe9bae887a0 20 read_log 192718'9839 (192718'9838) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:148 2014-08-06 14:51:24.109291 2014-08-13 17:38:02.884857 7fe9bae887a0 20 read_log 192718'9840 (192718'9839) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:149 2014-08-06 14:51:24.166112 2014-08-13 17:38:02.884875 7fe9bae887a0 20 read_log 192718'9841 (192718'9840) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:150 2014-08-06 14:51:24.324455 2014-08-13 17:38:02.884888 7fe9bae887a0 20 read_log 192718'9842 (192718'9841) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:151 2014-08-06 14:51:24.358059 2014-08-13 17:38:02.884899 7fe9bae887a0 20 read_log 192718'9843 (192718'9842) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:152 2014-08-06 14:51:24.549623 2014-08-13 17:38:02.884911 7fe9bae887a0 20 read_log 192718'9844 (192718'9843) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:153 2014-08-06 14:51:24.583303 2014-08-13 17:38:02.884923 7fe9bae887a0 20 read_log 192718'9845 (192718'9844) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:154 2014-08-06 14:51:24.629149 2014-08-13 17:38:02.884934 7fe9bae887a0 20 read_log 192718'9846 (192718'9845) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:155 2014-08-06 14:51:24.855637 2014-08-13 17:38:02.884946 7fe9bae887a0 20 read_log 192718'9847 (192718'9846) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:156 2014-08-06 14:51:24.887613 2014-08-13 17:38:02.884958 7fe9bae887a0 20 read_log 192718'9848 (192718'9847) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:157 2014-08-06 14:51:24.915650 2014-08-13 17:38:02.884970 7fe9bae887a0 20 read_log 192718'9849 (192718'9848) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:158 2014-08-06 14:51:24.945462 2014-08-13 17:38:02.884981 7fe9bae887a0 20 read_log 192718'9850 (192718'9849) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:159 2014-08-06 14:51:24.971432 2014-08-13 17:38:02.884993 7fe9bae887a0 20 read_log 192718'9851 (192718'9850) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:160 2014-08-06 14:51:24.995565 2014-08-13 17:38:02.885005 7fe9bae887a0 20 read_log 192718'9852 (192718'9851) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:161 2014-08-06 14:51:25.025328 2014-08-13 17:38:02.885017 7fe9bae887a0 20 read_log 192718'9853 (192718'9852) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:162 2014-08-06 14:51:25.153566 2014-08-13 17:38:02.885029 7fe9bae887a0 20 read_log 192718'9854 (192718'9853) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:163 2014-08-06 14:51:25.300566 2014-08-13 17:38:02.885040 7fe9bae887a0 20 read_log 192718'9855 (192718'9854) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:164 2014-08-06 14:51:25.329505 2014-08-13 17:38:02.885052 7fe9bae887a0 20 read_log 192718'9856 (192718'9855) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:165 2014-08-06 14:51:25.360517 2014-08-13 17:38:02.885064 7fe9bae887a0 20 read_log 192718'9857 (192718'9856) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:166 2014-08-06 14:51:25.391367 2014-08-13 17:38:02.885075 7fe9bae887a0 20 read_log 192718'9858 (192718'9857) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:167 2014-08-06 14:51:25.543055 2014-08-13 17:38:02.885087 7fe9bae887a0 20 read_log 192718'9859 (192718'9858) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:168 2014-08-06 14:51:25.723032 2014-08-13 17:38:02.885099 7fe9bae887a0 20 read_log 192718'9860 (192718'9859) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:169 2014-08-06 14:51:25.750977 2014-08-13 17:38:02.885110 7fe9bae887a0 20 read_log 192718'9861 (192718'9860) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:170 2014-08-06 14:51:25.776152 2014-08-13 17:38:02.885122 7fe9bae887a0 20 read_log 192718'9862 (192718'9861) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:171 2014-08-06 14:51:25.806821 2014-08-13 17:38:02.885134 7fe9bae887a0 20 read_log 192718'9863 (192718'9862) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:172 2014-08-06 14:51:25.847958 2014-08-13 17:38:02.885146 7fe9bae887a0 20 read_log 192718'9864 (192718'9863) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:173 2014-08-06 14:51:26.006887 2014-08-13 17:38:02.885173 7fe9bae887a0 20 read_log 192718'9865 (192718'9864) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:174 2014-08-06 14:51:26.033012 2014-08-13 17:38:02.885186 7fe9bae887a0 20 read_log 192718'9866 (192718'9865) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:175 2014-08-06 14:51:26.058666 2014-08-13 17:38:02.885198 7fe9bae887a0 20 read_log 192718'9867 (192718'9866) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:176 2014-08-06 14:51:26.093614 2014-08-13 17:38:02.885210 7fe9bae887a0 20 read_log 192718'9868 (192718'9867) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:177 2014-08-06 14:51:26.117815 2014-08-13 17:38:02.885222 7fe9bae887a0 20 read_log 192718'9869 (192718'9868) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:178 2014-08-06 14:51:26.160042 2014-08-13 17:38:02.885234 7fe9bae887a0 20 read_log 192718'9870 (192718'9869) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:179 2014-08-06 14:51:26.185401 2014-08-13 17:38:02.885246 7fe9bae887a0 20 read_log 192718'9871 (192718'9870) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:180 2014-08-06 14:51:26.406770 2014-08-13 17:38:02.885257 7fe9bae887a0 20 read_log 192718'9872 (192718'9871) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:181 2014-08-06 14:51:26.444029 2014-08-13 17:38:02.885269 7fe9bae887a0 20 read_log 192718'9873 (192718'9872) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:182 2014-08-06 14:51:26.476147 2014-08-13 17:38:02.885281 7fe9bae887a0 20 read_log 192718'9874 (192718'9873) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:183 2014-08-06 14:51:26.536014 2014-08-13 17:38:02.885292 7fe9bae887a0 20 read_log 192718'9875 (192718'9874) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:184 2014-08-06 14:51:26.715145 2014-08-13 17:38:02.885304 7fe9bae887a0 20 read_log 192718'9876 (192718'9875) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:185 2014-08-06 14:51:26.833460 2014-08-13 17:38:02.885315 7fe9bae887a0 20 read_log 192718'9877 (192718'9876) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:186 2014-08-06 14:51:26.882576 2014-08-13 17:38:02.885327 7fe9bae887a0 20 read_log 192718'9878 (192718'9877) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:187 2014-08-06 14:51:26.923674 2014-08-13 17:38:02.885339 7fe9bae887a0 20 read_log 192718'9879 (192718'9878) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:188 2014-08-06 14:51:26.963566 2014-08-13 17:38:02.885351 7fe9bae887a0 20 read_log 192718'9880 (192718'9879) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:189 2014-08-06 14:51:27.473340 2014-08-13 17:38:02.885362 7fe9bae887a0 20 read_log 192718'9881 (192718'9880) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:190 2014-08-06 14:51:27.637534 2014-08-13 17:38:02.885374 7fe9bae887a0 20 read_log 192718'9882 (192718'9881) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:191 2014-08-06 14:51:28.188279 2014-08-13 17:38:02.885386 7fe9bae887a0 20 read_log 192718'9883 (192718'9882) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:192 2014-08-06 14:51:28.962986 2014-08-13 17:38:02.885398 7fe9bae887a0 20 read_log 192718'9884 (192718'9883) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:193 2014-08-06 14:51:29.149277 2014-08-13 17:38:02.885409 7fe9bae887a0 20 read_log 192718'9885 (192718'9884) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:194 2014-08-06 14:51:29.183877 2014-08-13 17:38:02.885421 7fe9bae887a0 20 read_log 192718'9886 (192718'9885) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:195 2014-08-06 14:51:29.712400 2014-08-13 17:38:02.885448 7fe9bae887a0 20 read_log 192718'9887 (192718'9886) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:196 2014-08-06 14:51:30.333811 2014-08-13 17:38:02.885461 7fe9bae887a0 20 read_log 192718'9888 (192718'9887) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:197 2014-08-06 14:51:30.492833 2014-08-13 17:38:02.885496 7fe9bae887a0 20 read_log 192718'9889 (192718'9888) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:198 2014-08-06 14:51:30.651491 2014-08-13 17:38:02.885512 7fe9bae887a0 20 read_log 192718'9890 (192718'9889) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:199 2014-08-06 14:51:31.183686 2014-08-13 17:38:02.885524 7fe9bae887a0 20 read_log 192718'9891 (192718'9890) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:200 2014-08-06 14:51:31.669534 2014-08-13 17:38:02.885536 7fe9bae887a0 20 read_log 192718'9892 (192718'9891) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:201 2014-08-06 14:51:31.868342 2014-08-13 17:38:02.885548 7fe9bae887a0 20 read_log 192718'9893 (192718'9892) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:202 2014-08-06 14:51:32.072946 2014-08-13 17:38:02.885560 7fe9bae887a0 20 read_log 192718'9894 (192718'9893) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:203 2014-08-06 14:51:32.666981 2014-08-13 17:38:02.885572 7fe9bae887a0 20 read_log 192718'9895 (192718'9894) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:204 2014-08-06 14:51:34.486489 2014-08-13 17:38:02.885584 7fe9bae887a0 20 read_log 192718'9896 (192718'9895) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:205 2014-08-06 14:51:34.604728 2014-08-13 17:38:02.885595 7fe9bae887a0 20 read_log 192718'9897 (192718'9896) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:206 2014-08-06 14:51:34.965504 2014-08-13 17:38:02.885607 7fe9bae887a0 20 read_log 192718'9898 (192718'9897) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:207 2014-08-06 14:51:35.233551 2014-08-13 17:38:02.885619 7fe9bae887a0 20 read_log 192802'10243 (192718'9898) clean 32c071bb/core.58593+95+2/head//24 by osd.373.0:11971 2014-08-11 10:25:00.978538 2014-08-13 17:38:02.885631 7fe9bae887a0 20 read_log 192802'10244 (192802'10243) delete 32c071bb/core.58593+95+2/head//24 by osd.373.0:11972 2014-08-11 10:25:02.862111 2014-08-13 17:38:02.885643 7fe9bae887a0 20 read_log 192834'10518 (0'0) promote d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by osd.373.0:12325 2014-08-11 15:23:36.173587 2014-08-13 17:38:02.885656 7fe9bae887a0 20 read_log 192834'10519 (192834'10518) modify d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.933448.0:125 2014-08-11 15:23:36.161343 2014-08-13 17:38:02.885668 7fe9bae887a0 20 read_log 192834'10538 (0'0) promote c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by osd.373.0:12345 2014-08-11 15:23:38.870934 2014-08-13 17:38:02.885680 7fe9bae887a0 20 read_log 192834'10539 (192834'10538) modify c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.933448.0:839 2014-08-11 15:23:38.863269 2014-08-13 17:38:02.885692 7fe9bae887a0 20 read_log 192834'10544 (0'0) promote 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by osd.373.0:12351 2014-08-11 15:23:39.594380 2014-08-13 17:38:02.885704 7fe9bae887a0 20 read_log 192834'10545 (192834'10544) modify 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.933448.0:1028 2014-08-11 15:23:39.587821 2014-08-13 17:38:02.885716 7fe9bae887a0 20 read_log 192834'10577 (0'0) promote e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by osd.373.0:12384 2014-08-11 15:23:46.813383 2014-08-13 17:38:02.885728 7fe9bae887a0 20 read_log 192834'10578 (192834'10577) modify e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.933448.0:2438 2014-08-11 15:23:46.786896 2014-08-13 17:38:02.885740 7fe9bae887a0 20 read_log 192834'10596 (0'0) promote bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by osd.373.0:12403 2014-08-11 15:23:53.786649 2014-08-13 17:38:02.885752 7fe9bae887a0 20 read_log 192834'10597 (192834'10596) modify bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.933448.0:3827 2014-08-11 15:23:53.753419 2014-08-13 17:38:02.885771 7fe9bae887a0 20 read_log 192834'10608 (0'0) promote ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by osd.373.0:12415 2014-08-11 15:23:59.176341 2014-08-13 17:38:02.885784 7fe9bae887a0 20 read_log 192834'10609 (192834'10608) modify ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.933448.0:4878 2014-08-11 15:23:59.167274 2014-08-13 17:38:02.885796 7fe9bae887a0 20 read_log 192834'10642 (0'0) promote b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by osd.373.0:12449 2014-08-11 15:24:06.249942 2014-08-13 17:38:02.885808 7fe9bae887a0 20 read_log 192834'10643 (192834'10642) modify b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.933448.0:6654 2014-08-11 15:24:06.242303 2014-08-13 17:38:02.885820 7fe9bae887a0 20 read_log 192834'10646 (0'0) promote ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by osd.373.0:12453 2014-08-11 15:24:06.700016 2014-08-13 17:38:02.885832 7fe9bae887a0 20 read_log 192834'10647 (192834'10646) modify ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.933448.0:6772 2014-08-11 15:24:06.691857 2014-08-13 17:38:02.885844 7fe9bae887a0 20 read_log 192834'10668 (0'0) promote f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by osd.373.0:12475 2014-08-11 15:24:11.481453 2014-08-13 17:38:02.885856 7fe9bae887a0 20 read_log 192834'10669 (192834'10668) modify f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.933448.0:7808 2014-08-11 15:24:11.474783 2014-08-13 17:38:02.885868 7fe9bae887a0 20 read_log 192834'10684 (0'0) promote c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by osd.373.0:12491 2014-08-11 15:24:15.606739 2014-08-13 17:38:02.885880 7fe9bae887a0 20 read_log 192834'10685 (192834'10684) modify c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.933448.0:8837 2014-08-11 15:24:15.598834 2014-08-13 17:38:02.885892 7fe9bae887a0 20 read_log 192834'10688 (0'0) promote cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by osd.373.0:12495 2014-08-11 15:24:17.461211 2014-08-13 17:38:02.885904 7fe9bae887a0 20 read_log 192834'10689 (192834'10688) modify cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.933448.0:9074 2014-08-11 15:24:17.454593 2014-08-13 17:38:02.885915 7fe9bae887a0 20 read_log 192834'10698 (0'0) promote e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by osd.373.0:12505 2014-08-11 15:24:18.759082 2014-08-13 17:38:02.885927 7fe9bae887a0 20 read_log 192834'10699 (192834'10698) modify e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.933448.0:9419 2014-08-11 15:24:18.751768 2014-08-13 17:38:02.885939 7fe9bae887a0 20 read_log 192834'10704 (0'0) promote b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by osd.373.0:12511 2014-08-11 15:24:19.734582 2014-08-13 17:38:02.885971 7fe9bae887a0 20 read_log 192834'10705 (192834'10704) modify b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.933448.0:9663 2014-08-11 15:24:19.728104 2014-08-13 17:38:02.885984 7fe9bae887a0 20 read_log 192834'10708 (0'0) promote 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by osd.373.0:12515 2014-08-11 15:24:20.909771 2014-08-13 17:38:02.885996 7fe9bae887a0 20 read_log 192834'10709 (192834'10708) modify 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.933448.0:9930 2014-08-11 15:24:20.900925 2014-08-13 17:38:02.886008 7fe9bae887a0 20 read_log 192834'10712 (0'0) promote c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by osd.373.0:12519 2014-08-11 15:24:22.050722 2014-08-13 17:38:02.886020 7fe9bae887a0 20 read_log 192834'10713 (192834'10712) modify c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.933448.0:10194 2014-08-11 15:24:22.043450 2014-08-13 17:38:02.886042 7fe9bae887a0 20 read_log 192834'10714 (0'0) promote 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by osd.373.0:12521 2014-08-11 15:24:22.764161 2014-08-13 17:38:02.886057 7fe9bae887a0 20 read_log 192834'10715 (192834'10714) modify 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.933448.0:10342 2014-08-11 15:24:22.755024 2014-08-13 17:38:02.886069 7fe9bae887a0 20 read_log 192834'10744 (0'0) promote 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by osd.373.0:12551 2014-08-11 15:24:31.217190 2014-08-13 17:38:02.886081 7fe9bae887a0 20 read_log 192834'10745 (192834'10744) modify 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.933448.0:12134 2014-08-11 15:24:31.209838 2014-08-13 17:38:02.886094 7fe9bae887a0 20 read_log 192834'10778 (0'0) promote 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by osd.373.0:12585 2014-08-11 15:24:47.920516 2014-08-13 17:38:02.886106 7fe9bae887a0 20 read_log 192834'10779 (192834'10778) modify 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.933448.0:14979 2014-08-11 15:24:47.911925 2014-08-13 17:38:02.886118 7fe9bae887a0 20 read_log 192834'10788 (0'0) promote 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by osd.373.0:12595 2014-08-11 15:24:49.911588 2014-08-13 17:38:02.886130 7fe9bae887a0 20 read_log 192834'10789 (192834'10788) modify 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.933448.0:15507 2014-08-11 15:24:49.903752 2014-08-13 17:38:02.886142 7fe9bae887a0 20 read_log 192834'10802 (0'0) promote 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by osd.373.0:12609 2014-08-11 15:24:52.575819 2014-08-13 17:38:02.886159 7fe9bae887a0 20 read_log 192834'10803 (192834'10802) modify 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.933448.0:16033 2014-08-11 15:24:52.358432 2014-08-13 17:38:02.886174 7fe9bae887a0 20 read_log 192834'10882 (192834'10519) modify d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.933448.0:22416 2014-08-11 15:25:37.837977 2014-08-13 17:38:02.886189 7fe9bae887a0 20 read_log 192834'10892 (192834'10539) modify c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.933448.0:23130 2014-08-11 15:25:39.041513 2014-08-13 17:38:02.886202 7fe9bae887a0 20 read_log 192834'10895 (192834'10545) modify 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.933448.0:23319 2014-08-11 15:25:39.223281 2014-08-13 17:38:02.886214 7fe9bae887a0 20 read_log 192834'10912 (192834'10578) modify e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.933448.0:24729 2014-08-11 15:25:41.224272 2014-08-13 17:38:02.886226 7fe9bae887a0 20 read_log 192834'10921 (192834'10597) modify bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.933448.0:26118 2014-08-11 15:25:42.989934 2014-08-13 17:38:02.886238 7fe9bae887a0 20 read_log 192834'10926 (192834'10609) modify ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.933448.0:27169 2014-08-11 15:25:44.181224 2014-08-13 17:38:02.886250 7fe9bae887a0 20 read_log 192834'10943 (192834'10643) modify b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.933448.0:28945 2014-08-11 15:25:46.866149 2014-08-13 17:38:02.886262 7fe9bae887a0 20 read_log 192834'10945 (192834'10647) modify ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.933448.0:29063 2014-08-11 15:25:46.965168 2014-08-13 17:38:02.886274 7fe9bae887a0 20 read_log 192834'10956 (192834'10669) modify f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.933448.0:30099 2014-08-11 15:25:48.370737 2014-08-13 17:38:02.886285 7fe9bae887a0 20 read_log 192834'10964 (192834'10685) modify c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.933448.0:31128 2014-08-11 15:25:49.680249 2014-08-13 17:38:02.886305 7fe9bae887a0 20 read_log 192834'10966 (192834'10689) modify cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.933448.0:31365 2014-08-11 15:25:49.974738 2014-08-13 17:38:02.886318 7fe9bae887a0 20 read_log 192834'10971 (192834'10699) modify e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.933448.0:31710 2014-08-11 15:25:50.419626 2014-08-13 17:38:02.886330 7fe9bae887a0 20 read_log 192834'10974 (192834'10705) modify b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.933448.0:31954 2014-08-11 15:25:50.872526 2014-08-13 17:38:02.886342 7fe9bae887a0 20 read_log 192834'10976 (192834'10709) modify 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.933448.0:32221 2014-08-11 15:25:51.263213 2014-08-13 17:38:02.886354 7fe9bae887a0 20 read_log 192834'10978 (192834'10713) modify c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.933448.0:32485 2014-08-11 15:25:51.895350 2014-08-13 17:38:02.886366 7fe9bae887a0 20 read_log 192834'10979 (192834'10715) modify 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.933448.0:32633 2014-08-11 15:25:52.136414 2014-08-13 17:38:02.886378 7fe9bae887a0 20 read_log 192834'10994 (192834'10745) modify 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.933448.0:34425 2014-08-11 15:25:54.995856 2014-08-13 17:38:02.886390 7fe9bae887a0 20 read_log 192834'11011 (192834'10779) modify 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.933448.0:37270 2014-08-11 15:25:59.241877 2014-08-13 17:38:02.886402 7fe9bae887a0 20 read_log 192834'11016 (192834'10789) modify 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.933448.0:37798 2014-08-11 15:26:00.072755 2014-08-13 17:38:02.886414 7fe9bae887a0 20 read_log 192834'11023 (192834'10803) modify 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.933448.0:38324 2014-08-11 15:26:00.758785 2014-08-13 17:38:02.886426 7fe9bae887a0 20 read_log 192834'11063 (0'0) promote f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by osd.373.0:12870 2014-08-11 15:26:14.531449 2014-08-13 17:38:02.886438 7fe9bae887a0 20 read_log 192834'11064 (192834'11063) modify f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.1883634.0:74 2014-08-11 15:26:14.521605 2014-08-13 17:38:02.886450 7fe9bae887a0 20 read_log 192834'11075 (0'0) promote a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by osd.373.0:12882 2014-08-11 15:26:18.759636 2014-08-13 17:38:02.886462 7fe9bae887a0 20 read_log 192834'11076 (192834'11075) modify a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.1883634.0:1175 2014-08-11 15:26:18.750764 2014-08-13 17:38:02.886474 7fe9bae887a0 20 read_log 192834'11093 (0'0) promote 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by osd.373.0:12900 2014-08-11 15:26:24.859237 2014-08-13 17:38:02.886486 7fe9bae887a0 20 read_log 192834'11094 (192834'11093) modify 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.1883634.0:2831 2014-08-11 15:26:24.850752 2014-08-13 17:38:02.886498 7fe9bae887a0 20 read_log 192834'11103 (0'0) promote ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.373.0:12910 2014-08-11 15:26:29.423452 2014-08-13 17:38:02.886510 7fe9bae887a0 20 read_log 192834'11104 (192834'11103) modify ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.1883634.0:4121 2014-08-11 15:26:29.415740 2014-08-13 17:38:02.886522 7fe9bae887a0 20 read_log 192834'11111 (0'0) promote 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by osd.373.0:12918 2014-08-11 15:26:31.666131 2014-08-13 17:38:02.886534 7fe9bae887a0 20 read_log 192834'11112 (192834'11111) modify 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.1883634.0:4721 2014-08-11 15:26:31.659131 2014-08-13 17:38:02.886553 7fe9bae887a0 20 read_log 192834'11129 (0'0) promote bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by osd.373.0:12936 2014-08-11 15:26:34.672265 2014-08-13 17:38:02.886565 7fe9bae887a0 20 read_log 192834'11130 (192834'11129) modify bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.1883634.0:5513 2014-08-11 15:26:34.664094 2014-08-13 17:38:02.886578 7fe9bae887a0 20 read_log 192834'11147 (0'0) promote 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:12954 2014-08-11 15:26:37.945386 2014-08-13 17:38:02.886589 7fe9bae887a0 20 read_log 192834'11148 (192834'11147) modify 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.1883634.0:6393 2014-08-11 15:26:37.938043 2014-08-13 17:38:02.886602 7fe9bae887a0 20 read_log 192834'11199 (0'0) promote a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by osd.373.0:13006 2014-08-11 15:26:46.760513 2014-08-13 17:38:02.886614 7fe9bae887a0 20 read_log 192834'11200 (192834'11199) modify a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.1883634.0:8536 2014-08-11 15:26:46.753206 2014-08-13 17:38:02.886625 7fe9bae887a0 20 read_log 192834'11207 (0'0) promote 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by osd.373.0:13014 2014-08-11 15:26:49.096819 2014-08-13 17:38:02.886637 7fe9bae887a0 20 read_log 192834'11208 (192834'11207) modify 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.1883634.0:9053 2014-08-11 15:26:49.089815 2014-08-13 17:38:02.886649 7fe9bae887a0 20 read_log 192834'11225 (0'0) promote 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13032 2014-08-11 15:26:52.515584 2014-08-13 17:38:02.886681 7fe9bae887a0 20 read_log 192834'11226 (192834'11225) modify 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.1883634.0:9821 2014-08-11 15:26:52.506138 2014-08-13 17:38:02.886697 7fe9bae887a0 20 read_log 192834'11231 (0'0) promote 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13038 2014-08-11 15:26:56.371467 2014-08-13 17:38:02.886710 7fe9bae887a0 20 read_log 192834'11232 (192834'11231) modify 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.1883634.0:10722 2014-08-11 15:26:56.362788 2014-08-13 17:38:02.886722 7fe9bae887a0 20 read_log 192834'11241 (0'0) promote 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13048 2014-08-11 15:26:59.394020 2014-08-13 17:38:02.886734 7fe9bae887a0 20 read_log 192834'11242 (192834'11241) modify 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.1883634.0:11328 2014-08-11 15:26:59.385357 2014-08-13 17:38:02.886746 7fe9bae887a0 20 read_log 192834'11263 (0'0) promote 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by osd.373.0:13070 2014-08-11 15:27:06.169443 2014-08-13 17:38:02.886758 7fe9bae887a0 20 read_log 192834'11264 (192834'11263) modify 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.1883634.0:12803 2014-08-11 15:27:06.161641 2014-08-13 17:38:02.886771 7fe9bae887a0 20 read_log 192834'11273 (0'0) promote 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13080 2014-08-11 15:27:09.995907 2014-08-13 17:38:02.886783 7fe9bae887a0 20 read_log 192834'11274 (192834'11273) modify 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.1883634.0:13532 2014-08-11 15:27:09.987777 2014-08-13 17:38:02.886795 7fe9bae887a0 20 read_log 192834'11305 (0'0) promote 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by osd.373.0:13112 2014-08-11 15:27:19.988389 2014-08-13 17:38:02.886807 7fe9bae887a0 20 read_log 192834'11306 (192834'11305) modify 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.1883634.0:15531 2014-08-11 15:27:19.979134 2014-08-13 17:38:02.886829 7fe9bae887a0 20 read_log 192834'11309 (0'0) promote 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by osd.373.0:13116 2014-08-11 15:27:21.443821 2014-08-13 17:38:02.886841 7fe9bae887a0 20 read_log 192834'11310 (192834'11309) modify 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.1883634.0:15750 2014-08-11 15:27:21.434918 2014-08-13 17:38:02.886853 7fe9bae887a0 20 read_log 192834'11323 (0'0) promote ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by osd.373.0:13130 2014-08-11 15:27:27.101938 2014-08-13 17:38:02.886865 7fe9bae887a0 20 read_log 192834'11324 (192834'11323) modify ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.1883634.0:16336 2014-08-11 15:27:27.092626 2014-08-13 17:38:02.886877 7fe9bae887a0 20 read_log 192834'11347 (0'0) promote f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by osd.373.0:13154 2014-08-11 15:27:36.760480 2014-08-13 17:38:02.886889 7fe9bae887a0 20 read_log 192834'11348 (192834'11347) modify f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.1883634.0:17582 2014-08-11 15:27:36.750764 2014-08-13 17:38:02.886901 7fe9bae887a0 20 read_log 192834'11361 (0'0) promote e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by osd.373.0:13168 2014-08-11 15:27:44.822786 2014-08-13 17:38:02.886913 7fe9bae887a0 20 read_log 192834'11362 (192834'11361) modify e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.1883634.0:18488 2014-08-11 15:27:44.815393 2014-08-13 17:38:02.886925 7fe9bae887a0 20 read_log 192834'11371 (0'0) promote 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by osd.373.0:13178 2014-08-11 15:27:47.991888 2014-08-13 17:38:02.886937 7fe9bae887a0 20 read_log 192834'11372 (192834'11371) modify 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.1883634.0:18880 2014-08-11 15:27:47.980740 2014-08-13 17:38:02.886949 7fe9bae887a0 20 read_log 192834'11375 (0'0) promote eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by osd.373.0:13182 2014-08-11 15:27:49.976816 2014-08-13 17:38:02.886960 7fe9bae887a0 20 read_log 192834'11376 (192834'11375) modify eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.1883634.0:19206 2014-08-11 15:27:49.966867 2014-08-13 17:38:02.886972 7fe9bae887a0 20 read_log 192834'11391 (0'0) promote e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by osd.373.0:13197 2014-08-11 15:27:55.322916 2014-08-13 17:38:02.886984 7fe9bae887a0 20 read_log 192834'11392 (192834'11391) modify e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.1883634.0:20118 2014-08-11 15:27:55.314967 2014-08-13 17:38:02.886996 7fe9bae887a0 20 read_log 192834'11403 (0'0) promote b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by osd.373.0:13209 2014-08-11 15:27:57.009185 2014-08-13 17:38:02.887008 7fe9bae887a0 20 read_log 192834'11404 (192834'11403) modify b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.1883634.0:20439 2014-08-11 15:27:56.999045 2014-08-13 17:38:02.887020 7fe9bae887a0 20 read_log 192834'11439 (0'0) promote 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by osd.373.0:13245 2014-08-11 15:28:10.436394 2014-08-13 17:38:02.887032 7fe9bae887a0 20 read_log 192834'11440 (192834'11439) modify 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.1883634.0:22667 2014-08-11 15:28:10.426978 2014-08-13 17:38:02.887044 7fe9bae887a0 20 read_log 192834'11447 (192834'11064) modify f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.1883634.0:23398 2014-08-11 15:28:16.267977 2014-08-13 17:38:02.887056 7fe9bae887a0 20 read_log 192834'11448 (192834'10892) clean c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by osd.373.0:13254 2014-08-11 15:28:16.302954 2014-08-13 17:38:02.887075 7fe9bae887a0 20 read_log 192834'11449 (192834'11011) clean 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by osd.373.0:13255 2014-08-11 15:28:16.304265 2014-08-13 17:38:02.887088 7fe9bae887a0 20 read_log 192834'11451 (192834'11232) clean 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13257 2014-08-11 15:28:16.421765 2014-08-13 17:38:02.887100 7fe9bae887a0 20 read_log 192834'11452 (192834'11242) clean 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13258 2014-08-11 15:28:16.422215 2014-08-13 17:38:02.887113 7fe9bae887a0 20 read_log 192834'11453 (192834'11274) clean 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13259 2014-08-11 15:28:16.423152 2014-08-13 17:38:02.887125 7fe9bae887a0 20 read_log 192834'11454 (192834'10943) clean b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by osd.373.0:13260 2014-08-11 15:28:16.432021 2014-08-13 17:38:02.887137 7fe9bae887a0 20 read_log 192834'11455 (192834'11226) clean 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13261 2014-08-11 15:28:16.432323 2014-08-13 17:38:02.887149 7fe9bae887a0 20 read_log 192834'11456 (192834'10964) clean c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by osd.373.0:13262 2014-08-11 15:28:16.436294 2014-08-13 17:38:02.887170 7fe9bae887a0 20 read_log 192834'11457 (192834'10921) clean bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by osd.373.0:13263 2014-08-11 15:28:16.440424 2014-08-13 17:38:02.887182 7fe9bae887a0 20 read_log 192834'11458 (192834'11148) clean 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13264 2014-08-11 15:28:16.510075 2014-08-13 17:38:02.887194 7fe9bae887a0 20 read_log 192834'11461 (192834'11451) delete 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13267 2014-08-11 15:28:16.753297 2014-08-13 17:38:02.887207 7fe9bae887a0 20 read_log 192834'11462 (192834'11452) delete 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13268 2014-08-11 15:28:16.759060 2014-08-13 17:38:02.887219 7fe9bae887a0 20 read_log 192834'11463 (192834'11453) delete 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13269 2014-08-11 15:28:16.762956 2014-08-13 17:38:02.887231 7fe9bae887a0 20 read_log 192834'11464 (192834'11455) delete 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13270 2014-08-11 15:28:16.766905 2014-08-13 17:38:02.887243 7fe9bae887a0 20 read_log 192834'11465 (192834'11458) delete 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13271 2014-08-11 15:28:16.771113 2014-08-13 17:38:02.887254 7fe9bae887a0 20 read_log 192834'11466 (192834'11104) clean ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.373.0:13272 2014-08-11 15:28:16.948197 2014-08-13 17:38:02.887269 7fe9bae887a0 20 read_log 192834'11473 (192834'11076) modify a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.1883634.0:24499 2014-08-11 15:28:19.142943 2014-08-13 17:38:02.887282 7fe9bae887a0 20 read_log 192834'11482 (192834'11094) modify 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.1883634.0:26155 2014-08-11 15:28:22.684851 2014-08-13 17:38:02.887294 7fe9bae887a0 20 read_log 192834'11487 (192834'11466) modify ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.1883634.0:27445 2014-08-11 15:28:24.476961 2014-08-13 17:38:02.887306 7fe9bae887a0 20 read_log 192834'11491 (192834'11112) modify 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.1883634.0:28045 2014-08-11 15:28:25.301791 2014-08-13 17:38:02.887318 7fe9bae887a0 20 read_log 192834'11500 (192834'11130) modify bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.1883634.0:28837 2014-08-11 15:28:26.360935 2014-08-13 17:38:02.887339 7fe9bae887a0 20 read_log 192834'11509 (0'0) promote 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13315 2014-08-11 15:28:27.636238 2014-08-13 17:38:02.887355 7fe9bae887a0 20 read_log 192834'11510 (192834'11509) modify 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.1883634.0:29717 2014-08-11 15:28:27.603446 2014-08-13 17:38:02.887367 7fe9bae887a0 20 read_log 192834'11536 (192834'11200) modify a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.1883634.0:31860 2014-08-11 15:28:31.050127 2014-08-13 17:38:02.887379 7fe9bae887a0 20 read_log 192834'11540 (192834'11208) modify 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.1883634.0:32377 2014-08-11 15:28:31.747207 2014-08-13 17:38:02.887391 7fe9bae887a0 20 read_log 192834'11549 (0'0) promote 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13355 2014-08-11 15:28:32.850543 2014-08-13 17:38:02.887403 7fe9bae887a0 20 read_log 192834'11550 (192834'11549) modify 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.1883634.0:33145 2014-08-11 15:28:32.823058 2014-08-13 17:38:02.887415 7fe9bae887a0 20 read_log 192834'11553 (0'0) promote 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13359 2014-08-11 15:28:34.195532 2014-08-13 17:38:02.887427 7fe9bae887a0 20 read_log 192834'11554 (192834'11553) modify 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.1883634.0:34046 2014-08-11 15:28:34.166416 2014-08-13 17:38:02.887439 7fe9bae887a0 20 read_log 192834'11559 (0'0) promote 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13365 2014-08-11 15:28:35.152350 2014-08-13 17:38:02.887451 7fe9bae887a0 20 read_log 192834'11560 (192834'11559) modify 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.1883634.0:34652 2014-08-11 15:28:35.122947 2014-08-13 17:38:02.887464 7fe9bae887a0 20 read_log 192834'11571 (192834'11264) modify 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.1883634.0:36127 2014-08-11 15:28:37.591873 2014-08-13 17:38:02.887476 7fe9bae887a0 20 read_log 192834'11576 (0'0) promote 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13382 2014-08-11 15:28:38.527256 2014-08-13 17:38:02.887488 7fe9bae887a0 20 read_log 192834'11577 (192834'11576) modify 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.1883634.0:36856 2014-08-11 15:28:38.493179 2014-08-13 17:38:02.887500 7fe9bae887a0 20 read_log 192834'11593 (192834'11306) modify 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.1883634.0:38855 2014-08-11 15:28:41.215681 2014-08-13 17:38:02.887512 7fe9bae887a0 20 read_log 192834'11595 (192834'11310) modify 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.1883634.0:39074 2014-08-11 15:28:41.532334 2014-08-13 17:38:02.887524 7fe9bae887a0 20 read_log 192834'11602 (192834'11324) modify ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.1883634.0:39660 2014-08-11 15:28:42.409587 2014-08-13 17:38:02.887536 7fe9bae887a0 20 read_log 192834'11614 (192834'11348) modify f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.1883634.0:40906 2014-08-11 15:28:44.416508 2014-08-13 17:38:02.887548 7fe9bae887a0 20 read_log 192834'11621 (192834'11362) modify e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.1883634.0:41812 2014-08-11 15:28:45.510004 2014-08-13 17:38:02.887560 7fe9bae887a0 20 read_log 192834'11626 (192834'11372) modify 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.1883634.0:42204 2014-08-11 15:28:46.169697 2014-08-13 17:38:02.887572 7fe9bae887a0 20 read_log 192834'11628 (192834'11376) modify eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.1883634.0:42530 2014-08-11 15:28:46.273333 2014-08-13 17:38:02.887593 7fe9bae887a0 20 read_log 192834'11635 (192834'11392) modify e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.1883634.0:43442 2014-08-11 15:28:46.555327 2014-08-13 17:38:02.887606 7fe9bae887a0 20 read_log 192834'11641 (192834'11404) modify b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.1883634.0:43763 2014-08-11 15:28:46.655912 2014-08-13 17:38:02.887619 7fe9bae887a0 20 read_log 192834'11659 (192834'11440) modify 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.1883634.0:45991 2014-08-11 15:28:47.458842 2014-08-13 17:38:02.887638 7fe9bae887a0 20 read_log 192888'11663 (192888'11663) modify 1bb/hit_set_24.1bb_archive_2014-08-11 16:02:29.007052_2014-08-11 16:44:58.224868/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.225627 2014-08-13 17:38:02.887651 7fe9bae887a0 20 read_log 192888'11664 (0'0) promote 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by osd.4.0:21394 2014-08-11 16:44:58.231342 2014-08-13 17:38:02.887663 7fe9bae887a0 20 read_log 192888'11665 (192888'11664) modify 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.1315259.0:775 2014-08-11 16:44:58.219246 2014-08-13 17:38:02.887675 7fe9bae887a0 20 read_log 192888'11672 (0'0) promote 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by osd.4.0:21426 2014-08-11 16:45:01.307538 2014-08-13 17:38:02.887687 7fe9bae887a0 20 read_log 192888'11673 (192888'11672) modify 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.1315259.0:1576 2014-08-11 16:45:01.298901 2014-08-13 17:38:02.887699 7fe9bae887a0 20 read_log 192888'11674 (0'0) promote 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by osd.4.0:21436 2014-08-11 16:45:02.238717 2014-08-13 17:38:02.887711 7fe9bae887a0 20 read_log 192888'11675 (192888'11674) modify 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.1315259.0:1828 2014-08-11 16:45:02.230872 2014-08-13 17:38:02.887723 7fe9bae887a0 20 read_log 192888'11680 (0'0) promote 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by osd.4.0:21468 2014-08-11 16:45:06.718512 2014-08-13 17:38:02.887735 7fe9bae887a0 20 read_log 192888'11681 (192888'11680) modify 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.1315259.0:3036 2014-08-11 16:45:06.710251 2014-08-13 17:38:02.887747 7fe9bae887a0 20 read_log 192888'11682 (0'0) promote a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by osd.4.0:21522 2014-08-11 16:45:13.950025 2014-08-13 17:38:02.887759 7fe9bae887a0 20 read_log 192888'11683 (192888'11682) modify a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.1315259.0:4960 2014-08-11 16:45:13.941066 2014-08-13 17:38:02.887771 7fe9bae887a0 20 read_log 192888'11686 (0'0) promote cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by osd.4.0:21550 2014-08-11 16:45:16.549559 2014-08-13 17:38:02.887783 7fe9bae887a0 20 read_log 192888'11687 (192888'11686) modify cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.1315259.0:5658 2014-08-11 16:45:16.541187 2014-08-13 17:38:02.887796 7fe9bae887a0 20 read_log 192888'11698 (0'0) promote f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by osd.4.0:21610 2014-08-11 16:45:26.526727 2014-08-13 17:38:02.887808 7fe9bae887a0 20 read_log 192888'11699 (192888'11698) modify f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.1315259.0:8402 2014-08-11 16:45:26.517483 2014-08-13 17:38:02.887821 7fe9bae887a0 20 read_log 192888'11704 (0'0) promote 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by osd.4.0:21684 2014-08-11 16:45:36.207886 2014-08-13 17:38:02.887840 7fe9bae887a0 20 read_log 192888'11705 (192888'11704) modify 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.1315259.0:10897 2014-08-11 16:45:36.199399 2014-08-13 17:38:02.887853 7fe9bae887a0 20 read_log 192888'11714 (0'0) promote cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by osd.4.0:21742 2014-08-11 16:45:45.448358 2014-08-13 17:38:02.887865 7fe9bae887a0 20 read_log 192888'11715 (192888'11714) modify cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.1315259.0:13149 2014-08-11 16:45:45.439154 2014-08-13 17:38:02.887893 7fe9bae887a0 20 read_log 192888'11716 (0'0) promote 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by osd.4.0:21746 2014-08-11 16:45:46.123528 2014-08-13 17:38:02.887908 7fe9bae887a0 20 read_log 192888'11717 (192888'11716) modify 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.1315259.0:13324 2014-08-11 16:45:46.115310 2014-08-13 17:38:02.887920 7fe9bae887a0 20 read_log 192888'11718 (0'0) promote 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by osd.4.0:21752 2014-08-11 16:45:47.112025 2014-08-13 17:38:02.887932 7fe9bae887a0 20 read_log 192888'11719 (192888'11718) modify 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.1315259.0:13557 2014-08-11 16:45:47.104722 2014-08-13 17:38:02.887944 7fe9bae887a0 20 read_log 192888'11726 (0'0) promote 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by osd.4.0:21792 2014-08-11 16:45:54.038131 2014-08-13 17:38:02.887956 7fe9bae887a0 20 read_log 192888'11727 (192888'11726) modify 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.1315259.0:15381 2014-08-11 16:45:54.027412 2014-08-13 17:38:02.887968 7fe9bae887a0 20 read_log 192888'11734 (0'0) promote b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by osd.4.0:21830 2014-08-11 16:45:57.591447 2014-08-13 17:38:02.887980 7fe9bae887a0 20 read_log 192888'11735 (192888'11734) modify b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.1315259.0:16309 2014-08-11 16:45:57.583229 2014-08-13 17:38:02.887992 7fe9bae887a0 20 read_log 192888'11736 (0'0) promote 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by osd.4.0:21832 2014-08-11 16:45:57.905721 2014-08-13 17:38:02.888004 7fe9bae887a0 20 read_log 192888'11737 (192888'11736) modify 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.1315259.0:16393 2014-08-11 16:45:57.896978 2014-08-13 17:38:02.888020 7fe9bae887a0 20 read_log 192888'11738 (0'0) promote b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by osd.4.0:21842 2014-08-11 16:46:00.254837 2014-08-13 17:38:02.888031 7fe9bae887a0 20 read_log 192888'11739 (192888'11738) modify b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.1315259.0:16980 2014-08-11 16:46:00.246782 2014-08-13 17:38:02.888044 7fe9bae887a0 20 read_log 192888'11740 (0'0) promote 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by osd.4.0:21858 2014-08-11 16:46:01.481287 2014-08-13 17:38:02.888056 7fe9bae887a0 20 read_log 192888'11741 (192888'11740) modify 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.1315259.0:17297 2014-08-11 16:46:01.473429 2014-08-13 17:38:02.888068 7fe9bae887a0 20 read_log 192888'11746 (0'0) promote f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by osd.4.0:21884 2014-08-11 16:46:06.440322 2014-08-13 17:38:02.888080 7fe9bae887a0 20 read_log 192888'11747 (192888'11746) modify f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.1315259.0:18542 2014-08-11 16:46:06.432691 2014-08-13 17:38:02.888092 7fe9bae887a0 20 read_log 192888'11748 (0'0) promote f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by osd.4.0:21888 2014-08-11 16:46:06.503060 2014-08-13 17:38:02.888111 7fe9bae887a0 20 read_log 192888'11749 (192888'11748) modify f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.1315259.0:18556 2014-08-11 16:46:06.495623 2014-08-13 17:38:02.888123 7fe9bae887a0 20 read_log 192888'11750 (0'0) promote 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by osd.4.0:21906 2014-08-11 16:46:08.047075 2014-08-13 17:38:02.888135 7fe9bae887a0 20 read_log 192888'11751 (192888'11750) modify 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.1315259.0:18968 2014-08-11 16:46:08.038898 2014-08-13 17:38:02.888147 7fe9bae887a0 20 read_log 192888'11752 (0'0) promote 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by osd.4.0:21912 2014-08-11 16:46:08.583468 2014-08-13 17:38:02.888165 7fe9bae887a0 20 read_log 192888'11753 (192888'11752) modify 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.1315259.0:19071 2014-08-11 16:46:08.573989 2014-08-13 17:38:02.888178 7fe9bae887a0 20 read_log 192888'11758 (0'0) promote ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by osd.4.0:21994 2014-08-11 16:46:20.096803 2014-08-13 17:38:02.888190 7fe9bae887a0 20 read_log 192888'11759 (192888'11758) modify ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.1315259.0:22102 2014-08-11 16:46:20.089659 2014-08-13 17:38:02.888202 7fe9bae887a0 20 read_log 192888'11760 (0'0) promote 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by osd.4.0:22008 2014-08-11 16:46:22.103186 2014-08-13 17:38:02.888214 7fe9bae887a0 20 read_log 192888'11761 (192888'11760) modify 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.1315259.0:22614 2014-08-11 16:46:22.094941 2014-08-13 17:38:02.888226 7fe9bae887a0 20 read_log 192888'11764 (0'0) promote be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by osd.4.0:22014 2014-08-11 16:46:22.767415 2014-08-13 17:38:02.888237 7fe9bae887a0 20 read_log 192888'11765 (192888'11764) modify be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.1315259.0:22792 2014-08-11 16:46:22.757663 2014-08-13 17:38:02.888249 7fe9bae887a0 20 read_log 192888'11766 (0'0) promote b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by osd.4.0:22044 2014-08-11 16:46:28.198353 2014-08-13 17:38:02.888261 7fe9bae887a0 20 read_log 192888'11767 (192888'11766) modify b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.1315259.0:24147 2014-08-11 16:46:28.188138 2014-08-13 17:38:02.888273 7fe9bae887a0 20 read_log 192888'11774 (0'0) promote 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by osd.4.0:22096 2014-08-11 16:46:34.767534 2014-08-13 17:38:02.888284 7fe9bae887a0 20 read_log 192888'11775 (192888'11774) modify 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.1315259.0:25788 2014-08-11 16:46:34.759072 2014-08-13 17:38:02.888296 7fe9bae887a0 20 read_log 192888'11784 (0'0) promote 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by osd.4.0:22168 2014-08-11 16:46:48.420798 2014-08-13 17:38:02.888308 7fe9bae887a0 20 read_log 192888'11785 (192888'11784) modify 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.1315259.0:29105 2014-08-11 16:46:48.413155 2014-08-13 17:38:02.888320 7fe9bae887a0 20 read_log 192888'11788 (0'0) promote 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by osd.4.0:22184 2014-08-11 16:46:51.267681 2014-08-13 17:38:02.888331 7fe9bae887a0 20 read_log 192888'11789 (192888'11788) modify 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.1315259.0:29776 2014-08-11 16:46:51.258168 2014-08-13 17:38:02.888343 7fe9bae887a0 20 read_log 192888'11792 (0'0) promote f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by osd.4.0:22202 2014-08-11 16:46:53.026474 2014-08-13 17:38:02.888364 7fe9bae887a0 20 read_log 192888'11793 (192888'11792) modify f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.1315259.0:30247 2014-08-11 16:46:53.016351 2014-08-13 17:38:02.888378 7fe9bae887a0 20 read_log 192888'11796 (192888'11665) modify 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.1315259.0:31502 2014-08-11 16:46:56.113980 2014-08-13 17:38:02.888390 7fe9bae887a0 20 read_log 192888'11800 (192888'11673) modify 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.1315259.0:32303 2014-08-11 16:46:56.342675 2014-08-13 17:38:02.888402 7fe9bae887a0 20 read_log 192888'11801 (192888'11675) modify 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.1315259.0:32555 2014-08-11 16:46:56.409038 2014-08-13 17:38:02.888413 7fe9bae887a0 20 read_log 192888'11804 (192888'11681) modify 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.1315259.0:33763 2014-08-11 16:46:56.718564 2014-08-13 17:38:02.888425 7fe9bae887a0 20 read_log 192888'11805 (192888'11683) modify a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.1315259.0:35687 2014-08-11 16:46:57.222547 2014-08-13 17:38:02.888437 7fe9bae887a0 20 read_log 192888'11807 (192888'11687) modify cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.1315259.0:36385 2014-08-11 16:46:57.409456 2014-08-13 17:38:02.888449 7fe9bae887a0 20 read_log 192888'11813 (192888'11699) modify f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.1315259.0:39129 2014-08-11 16:46:58.133015 2014-08-13 17:38:02.888461 7fe9bae887a0 20 read_log 192888'11816 (192888'11705) modify 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.1315259.0:41624 2014-08-11 16:46:58.794369 2014-08-13 17:38:02.888473 7fe9bae887a0 20 read_log 192888'11821 (192888'11715) modify cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.1315259.0:43876 2014-08-11 16:46:59.415895 2014-08-13 17:38:02.888485 7fe9bae887a0 20 read_log 192888'11822 (192888'11717) modify 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.1315259.0:44051 2014-08-11 16:46:59.466981 2014-08-13 17:38:02.888497 7fe9bae887a0 20 read_log 192888'11823 (192888'11719) modify 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.1315259.0:44284 2014-08-11 16:46:59.535014 2014-08-13 17:38:02.888509 7fe9bae887a0 20 read_log 192888'11827 (192888'11727) modify 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.1315259.0:46108 2014-08-11 16:47:00.062461 2014-08-13 17:38:02.888520 7fe9bae887a0 20 read_log 192888'11831 (192888'11735) modify b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.1315259.0:47036 2014-08-11 16:47:00.328690 2014-08-13 17:38:02.888532 7fe9bae887a0 20 read_log 192888'11832 (192888'11737) modify 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.1315259.0:47120 2014-08-11 16:47:00.356372 2014-08-13 17:38:02.888544 7fe9bae887a0 20 read_log 192888'11833 (192888'11739) modify b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.1315259.0:47707 2014-08-11 16:47:00.541775 2014-08-13 17:38:02.888556 7fe9bae887a0 20 read_log 192888'11834 (192888'11741) modify 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.1315259.0:48024 2014-08-11 16:47:00.647057 2014-08-13 17:38:02.888568 7fe9bae887a0 20 read_log 192888'11837 (192888'11747) modify f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.1315259.0:49269 2014-08-11 16:47:01.008684 2014-08-13 17:38:02.888580 7fe9bae887a0 20 read_log 192888'11838 (192888'11749) modify f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.1315259.0:49283 2014-08-11 16:47:01.014019 2014-08-13 17:38:02.888591 7fe9bae887a0 20 read_log 192888'11839 (192888'11751) modify 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.1315259.0:49695 2014-08-11 16:47:01.146123 2014-08-13 17:38:02.888610 7fe9bae887a0 20 read_log 192888'11840 (192888'11753) modify 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.1315259.0:49798 2014-08-11 16:47:01.176641 2014-08-13 17:38:02.888623 7fe9bae887a0 20 read_log 192888'11843 (192888'11759) modify ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.1315259.0:52829 2014-08-11 16:47:02.081540 2014-08-13 17:38:02.888635 7fe9bae887a0 20 read_log 192888'11844 (192888'11761) modify 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.1315259.0:53341 2014-08-11 16:47:02.225747 2014-08-13 17:38:02.888647 7fe9bae887a0 20 read_log 192888'11846 (192888'11765) modify be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.1315259.0:53519 2014-08-11 16:47:02.277176 2014-08-13 17:38:02.888659 7fe9bae887a0 20 read_log 192888'11847 (192888'11767) modify b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.1315259.0:54874 2014-08-11 16:47:02.743164 2014-08-13 17:38:02.888671 7fe9bae887a0 20 read_log 192888'11851 (192888'11775) modify 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.1315259.0:56515 2014-08-11 16:47:03.231029 2014-08-13 17:38:02.888683 7fe9bae887a0 20 read_log 192888'11856 (192888'11785) modify 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.1315259.0:59832 2014-08-11 16:47:04.253844 2014-08-13 17:38:02.888695 7fe9bae887a0 20 read_log 192888'11858 (192888'11789) modify 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.1315259.0:60503 2014-08-11 16:47:04.461426 2014-08-13 17:38:02.888707 7fe9bae887a0 20 read_log 192888'11860 (192888'11793) modify f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.1315259.0:60974 2014-08-11 16:47:04.596900 2014-08-13 17:38:02.888720 7fe9bae887a0 20 read_log 192888'11862 (192888'11862) modify 1bb/hit_set_24.1bb_archive_2014-08-11 16:44:58.225727_2014-08-11 17:05:25.545358/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:25.545591 2014-08-13 17:38:02.888733 7fe9bae887a0 20 read_log 192888'11863 (192888'11663) delete 1bb/hit_set_24.1bb_archive_2014-08-11 16:02:29.007052_2014-08-11 16:44:58.224868/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:25.545591 2014-08-13 17:38:02.888746 7fe9bae887a0 20 read_log 192888'11864 (0'0) promote 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by osd.4.0:22708 2014-08-11 17:05:25.550802 2014-08-13 17:38:02.888762 7fe9bae887a0 20 read_log 192888'11865 (192888'11864) modify 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.942868.0:2595 2014-08-11 17:05:25.539243 2014-08-13 17:38:02.888774 7fe9bae887a0 20 read_log 192888'11866 (0'0) promote e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by osd.4.0:22736 2014-08-11 17:05:28.658274 2014-08-13 17:38:02.888786 7fe9bae887a0 20 read_log 192888'11867 (192888'11866) modify e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.942868.0:3437 2014-08-11 17:05:28.647956 2014-08-13 17:38:02.888798 7fe9bae887a0 20 read_log 192888'11870 (0'0) promote c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by osd.4.0:22768 2014-08-11 17:05:32.011179 2014-08-13 17:38:02.888810 7fe9bae887a0 20 read_log 192888'11871 (192888'11870) modify c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.942868.0:4346 2014-08-11 17:05:32.002248 2014-08-13 17:38:02.888822 7fe9bae887a0 20 read_log 192888'11872 (0'0) promote 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by osd.4.0:22820 2014-08-11 17:05:38.738237 2014-08-13 17:38:02.888834 7fe9bae887a0 20 read_log 192888'11873 (192888'11872) modify 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.942868.0:6116 2014-08-11 17:05:38.729381 2014-08-13 17:38:02.888855 7fe9bae887a0 20 read_log 192888'11880 (0'0) promote 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by osd.4.0:22884 2014-08-11 17:05:47.672866 2014-08-13 17:38:02.888868 7fe9bae887a0 20 read_log 192888'11881 (192888'11880) modify 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.942868.0:8510 2014-08-11 17:05:47.661201 2014-08-13 17:38:02.888880 7fe9bae887a0 20 read_log 192888'11882 (0'0) promote 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by osd.4.0:22892 2014-08-11 17:05:49.916267 2014-08-13 17:38:02.888892 7fe9bae887a0 20 read_log 192888'11883 (192888'11882) modify 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.942868.0:9109 2014-08-11 17:05:49.906604 2014-08-13 17:38:02.888904 7fe9bae887a0 20 read_log 192888'11884 (0'0) promote b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by osd.4.0:22912 2014-08-11 17:05:52.903118 2014-08-13 17:38:02.888916 7fe9bae887a0 20 read_log 192888'11885 (192888'11884) modify b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.942868.0:9916 2014-08-11 17:05:52.895634 2014-08-13 17:38:02.888928 7fe9bae887a0 20 read_log 192888'11886 (0'0) promote bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by osd.4.0:22932 2014-08-11 17:05:55.660735 2014-08-13 17:38:02.888940 7fe9bae887a0 20 read_log 192888'11887 (192888'11886) modify bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.942868.0:10683 2014-08-11 17:05:55.653661 2014-08-13 17:38:02.888952 7fe9bae887a0 20 read_log 192888'11892 (0'0) promote cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by osd.4.0:22942 2014-08-11 17:05:57.414968 2014-08-13 17:38:02.888964 7fe9bae887a0 20 read_log 192888'11893 (192888'11892) modify cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.942868.0:11156 2014-08-11 17:05:57.406340 2014-08-13 17:38:02.888977 7fe9bae887a0 20 read_log 192888'11896 (0'0) promote 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by osd.4.0:22958 2014-08-11 17:06:00.973274 2014-08-13 17:38:02.888989 7fe9bae887a0 20 read_log 192888'11897 (192888'11896) modify 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.942868.0:12130 2014-08-11 17:06:00.964049 2014-08-13 17:38:02.889001 7fe9bae887a0 20 read_log 192888'11898 (0'0) promote 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by osd.4.0:22964 2014-08-11 17:06:01.692019 2014-08-13 17:38:02.889013 7fe9bae887a0 20 read_log 192888'11899 (192888'11898) modify 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.942868.0:12331 2014-08-11 17:06:01.683659 2014-08-13 17:38:02.889025 7fe9bae887a0 20 read_log 192888'11900 (0'0) promote 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by osd.4.0:22968 2014-08-11 17:06:02.223124 2014-08-13 17:38:02.889037 7fe9bae887a0 20 read_log 192888'11901 (192888'11900) modify 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.942868.0:12480 2014-08-11 17:06:02.215147 2014-08-13 17:38:02.889049 7fe9bae887a0 20 read_log 192888'11902 (0'0) promote 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by osd.4.0:23000 2014-08-11 17:06:06.139079 2014-08-13 17:38:02.889078 7fe9bae887a0 20 read_log 192888'11903 (192888'11902) modify 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.942868.0:13493 2014-08-11 17:06:06.130281 2014-08-13 17:38:02.889091 7fe9bae887a0 20 read_log 192888'11904 (0'0) promote 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by osd.4.0:23006 2014-08-11 17:06:07.852919 2014-08-13 17:38:02.889103 7fe9bae887a0 20 read_log 192888'11905 (192888'11904) modify 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.942868.0:13922 2014-08-11 17:06:07.843027 2014-08-13 17:38:02.889138 7fe9bae887a0 20 read_log 192888'11906 (0'0) promote 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by osd.4.0:23010 2014-08-11 17:06:08.651023 2014-08-13 17:38:02.889151 7fe9bae887a0 20 read_log 192888'11907 (192888'11906) modify 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.942868.0:14103 2014-08-11 17:06:08.641314 2014-08-13 17:38:02.889169 7fe9bae887a0 20 read_log 192888'11912 (0'0) promote 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by osd.4.0:23028 2014-08-11 17:06:11.525726 2014-08-13 17:38:02.889181 7fe9bae887a0 20 read_log 192888'11913 (192888'11912) modify 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.942868.0:14821 2014-08-11 17:06:11.517433 2014-08-13 17:38:02.889193 7fe9bae887a0 20 read_log 192888'11918 (0'0) promote 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by osd.4.0:23088 2014-08-11 17:06:20.636199 2014-08-13 17:38:02.889205 7fe9bae887a0 20 read_log 192888'11919 (192888'11918) modify 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.942868.0:17152 2014-08-11 17:06:20.627339 2014-08-13 17:38:02.889217 7fe9bae887a0 20 read_log 192888'11920 (0'0) promote 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by osd.4.0:23094 2014-08-11 17:06:21.274779 2014-08-13 17:38:02.889229 7fe9bae887a0 20 read_log 192888'11921 (192888'11920) modify 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.942868.0:17320 2014-08-11 17:06:21.265953 2014-08-13 17:38:02.889241 7fe9bae887a0 20 read_log 192888'11924 (0'0) promote 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by osd.4.0:23108 2014-08-11 17:06:22.868704 2014-08-13 17:38:02.889253 7fe9bae887a0 20 read_log 192888'11925 (192888'11924) modify 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.942868.0:17742 2014-08-11 17:06:22.858455 2014-08-13 17:38:02.889265 7fe9bae887a0 20 read_log 192888'11928 (0'0) promote c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by osd.4.0:23136 2014-08-11 17:06:27.338192 2014-08-13 17:38:02.889277 7fe9bae887a0 20 read_log 192888'11929 (192888'11928) modify c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.942868.0:18931 2014-08-11 17:06:27.329604 2014-08-13 17:38:02.889294 7fe9bae887a0 20 read_log 192888'11932 (0'0) promote d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by osd.4.0:23186 2014-08-11 17:06:34.416083 2014-08-13 17:38:02.889307 7fe9bae887a0 20 read_log 192888'11933 (192888'11932) modify d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.942868.0:20792 2014-08-11 17:06:34.407975 2014-08-13 17:38:02.889319 7fe9bae887a0 20 read_log 192888'11938 (0'0) promote d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by osd.4.0:23266 2014-08-11 17:06:45.128382 2014-08-13 17:38:02.889331 7fe9bae887a0 20 read_log 192888'11939 (192888'11938) modify d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.942868.0:23571 2014-08-11 17:06:45.119954 2014-08-13 17:38:02.889343 7fe9bae887a0 20 read_log 192888'11944 (0'0) promote 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by osd.4.0:23298 2014-08-11 17:06:48.052290 2014-08-13 17:38:02.889355 7fe9bae887a0 20 read_log 192888'11945 (192888'11944) modify 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.942868.0:24364 2014-08-11 17:06:48.042837 2014-08-13 17:38:02.889368 7fe9bae887a0 20 read_log 192888'11946 (0'0) promote a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by osd.4.0:23300 2014-08-11 17:06:48.300162 2014-08-13 17:38:02.889379 7fe9bae887a0 20 read_log 192888'11947 (192888'11946) modify a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.942868.0:24428 2014-08-11 17:06:48.290724 2014-08-13 17:38:02.889398 7fe9bae887a0 20 read_log 192888'11948 (0'0) promote fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by osd.4.0:23318 2014-08-11 17:06:49.746408 2014-08-13 17:38:02.889411 7fe9bae887a0 20 read_log 192888'11949 (192888'11948) modify fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.942868.0:24820 2014-08-11 17:06:49.737654 2014-08-13 17:38:02.889423 7fe9bae887a0 20 read_log 192888'11950 (0'0) promote a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by osd.4.0:23320 2014-08-11 17:06:49.784824 2014-08-13 17:38:02.889435 7fe9bae887a0 20 read_log 192888'11951 (192888'11950) modify a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.942868.0:24830 2014-08-11 17:06:49.774436 2014-08-13 17:38:02.889447 7fe9bae887a0 20 read_log 192888'11952 (0'0) promote bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by osd.4.0:23324 2014-08-11 17:06:50.693715 2014-08-13 17:38:02.889459 7fe9bae887a0 20 read_log 192888'11953 (192888'11952) modify bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.942868.0:25075 2014-08-11 17:06:50.681555 2014-08-13 17:38:02.889471 7fe9bae887a0 20 read_log 192888'11956 (0'0) promote 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by osd.4.0:23338 2014-08-11 17:06:52.370971 2014-08-13 17:38:02.889483 7fe9bae887a0 20 read_log 192888'11957 (192888'11956) modify 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.942868.0:25517 2014-08-11 17:06:52.362503 2014-08-13 17:38:02.889495 7fe9bae887a0 20 read_log 192888'11958 (0'0) promote ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by osd.4.0:23346 2014-08-11 17:06:54.094439 2014-08-13 17:38:02.889507 7fe9bae887a0 20 read_log 192888'11959 (192888'11958) modify ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.942868.0:25916 2014-08-11 17:06:54.085777 2014-08-13 17:38:02.889540 7fe9bae887a0 20 read_log 192888'11960 (0'0) promote 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by osd.4.0:23348 2014-08-11 17:06:54.905964 2014-08-13 17:38:02.889552 7fe9bae887a0 20 read_log 192888'11961 (192888'11960) modify 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.942868.0:26118 2014-08-11 17:06:54.897100 2014-08-13 17:38:02.889564 7fe9bae887a0 20 read_log 192888'11964 (0'0) promote 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by osd.4.0:23368 2014-08-11 17:06:58.769938 2014-08-13 17:38:02.889576 7fe9bae887a0 20 read_log 192888'11965 (192888'11964) modify 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.942868.0:27148 2014-08-11 17:06:58.762138 2014-08-13 17:38:02.889588 7fe9bae887a0 20 read_log 192888'11966 (0'0) promote dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by osd.4.0:23374 2014-08-11 17:06:59.687533 2014-08-13 17:38:02.889600 7fe9bae887a0 20 read_log 192888'11967 (192888'11966) modify dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.942868.0:27400 2014-08-11 17:06:59.679140 2014-08-13 17:38:02.889612 7fe9bae887a0 20 read_log 192888'11970 (0'0) promote 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by osd.4.0:23386 2014-08-11 17:07:00.673588 2014-08-13 17:38:02.889624 7fe9bae887a0 20 read_log 192888'11971 (192888'11970) modify 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.942868.0:27650 2014-08-11 17:07:00.664200 2014-08-13 17:38:02.889636 7fe9bae887a0 20 read_log 192888'11972 (0'0) promote d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by osd.4.0:23400 2014-08-11 17:07:03.630731 2014-08-13 17:38:02.889648 7fe9bae887a0 20 read_log 192888'11973 (192888'11972) modify d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.942868.0:28447 2014-08-11 17:07:03.622238 2014-08-13 17:38:02.889670 7fe9bae887a0 20 read_log 192888'11974 (0'0) promote d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by osd.4.0:23406 2014-08-11 17:07:04.359977 2014-08-13 17:38:02.889683 7fe9bae887a0 20 read_log 192888'11975 (192888'11974) modify d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.942868.0:28643 2014-08-11 17:07:04.350520 2014-08-13 17:38:02.889695 7fe9bae887a0 20 read_log 192888'11976 (0'0) promote 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by osd.4.0:23414 2014-08-11 17:07:05.126053 2014-08-13 17:38:02.889707 7fe9bae887a0 20 read_log 192888'11977 (192888'11976) modify 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.942868.0:28852 2014-08-11 17:07:05.117512 2014-08-13 17:38:02.889720 7fe9bae887a0 20 read_log 192888'11980 (0'0) promote 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by osd.4.0:23438 2014-08-11 17:07:09.794505 2014-08-13 17:38:02.889731 7fe9bae887a0 20 read_log 192888'11981 (192888'11980) modify 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.942868.0:29995 2014-08-11 17:07:09.785352 2014-08-13 17:38:02.889743 7fe9bae887a0 20 read_log 192888'11986 (192888'11865) modify 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.942868.0:33861 2014-08-11 17:07:16.632374 2014-08-13 17:38:02.889755 7fe9bae887a0 20 read_log 192888'11987 (192888'11867) modify e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.942868.0:34703 2014-08-11 17:07:16.869072 2014-08-13 17:38:02.889767 7fe9bae887a0 20 read_log 192888'11989 (192888'11871) modify c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.942868.0:35612 2014-08-11 17:07:17.118579 2014-08-13 17:38:02.889779 7fe9bae887a0 20 read_log 192888'11990 (192888'11873) modify 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.942868.0:37382 2014-08-11 17:07:17.593795 2014-08-13 17:38:02.889791 7fe9bae887a0 20 read_log 192888'11994 (192888'11881) modify 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.942868.0:39776 2014-08-11 17:07:18.279856 2014-08-13 17:38:02.889803 7fe9bae887a0 20 read_log 192888'11995 (192888'11883) modify 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.942868.0:40375 2014-08-11 17:07:18.433050 2014-08-13 17:38:02.889815 7fe9bae887a0 20 read_log 192888'11996 (192888'11885) modify b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.942868.0:41182 2014-08-11 17:07:18.663115 2014-08-13 17:38:02.889827 7fe9bae887a0 20 read_log 192888'11997 (192888'11887) modify bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.942868.0:41949 2014-08-11 17:07:18.864825 2014-08-13 17:38:02.889839 7fe9bae887a0 20 read_log 192888'12000 (192888'11893) modify cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.942868.0:42422 2014-08-11 17:07:18.992873 2014-08-13 17:38:02.889851 7fe9bae887a0 20 read_log 192888'12002 (192888'11897) modify 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.942868.0:43396 2014-08-11 17:07:19.260866 2014-08-13 17:38:02.889863 7fe9bae887a0 20 read_log 192888'12003 (192888'11899) modify 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.942868.0:43597 2014-08-11 17:07:19.318313 2014-08-13 17:38:02.889875 7fe9bae887a0 20 read_log 192888'12004 (192888'11901) modify 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.942868.0:43746 2014-08-11 17:07:19.362158 2014-08-13 17:38:02.889887 7fe9bae887a0 20 read_log 192888'12005 (192888'11903) modify 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.942868.0:44759 2014-08-11 17:07:19.655477 2014-08-13 17:38:02.889899 7fe9bae887a0 20 read_log 192888'12006 (192888'11905) modify 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.942868.0:45188 2014-08-11 17:07:19.769490 2014-08-13 17:38:02.889918 7fe9bae887a0 20 read_log 192888'12007 (192888'11907) modify 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.942868.0:45369 2014-08-11 17:07:19.818310 2014-08-13 17:38:02.889931 7fe9bae887a0 20 read_log 192888'12010 (192888'11913) modify 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.942868.0:46087 2014-08-11 17:07:20.016559 2014-08-13 17:38:02.889943 7fe9bae887a0 20 read_log 192888'12013 (192888'11919) modify 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.942868.0:48418 2014-08-11 17:07:20.715055 2014-08-13 17:38:02.889955 7fe9bae887a0 20 read_log 192888'12014 (192888'11921) modify 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.942868.0:48586 2014-08-11 17:07:20.758583 2014-08-13 17:38:02.889968 7fe9bae887a0 20 read_log 192888'12016 (192888'11925) modify 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.942868.0:49008 2014-08-11 17:07:20.881773 2014-08-13 17:38:02.889998 7fe9bae887a0 20 read_log 192888'12018 (192888'11929) modify c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.942868.0:50197 2014-08-11 17:07:21.244821 2014-08-13 17:38:02.890015 7fe9bae887a0 20 read_log 192888'12020 (192888'11933) modify d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.942868.0:52058 2014-08-11 17:07:21.828846 2014-08-13 17:38:02.890027 7fe9bae887a0 20 read_log 192888'12023 (192888'11939) modify d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.942868.0:54837 2014-08-11 17:07:22.671208 2014-08-13 17:38:02.890040 7fe9bae887a0 20 read_log 192888'12026 (192888'11945) modify 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.942868.0:55630 2014-08-11 17:07:22.905405 2014-08-13 17:38:02.890052 7fe9bae887a0 20 read_log 192888'12027 (192888'11947) modify a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.942868.0:55694 2014-08-11 17:07:22.925739 2014-08-13 17:38:02.890065 7fe9bae887a0 20 read_log 192888'12028 (192888'11949) modify fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.942868.0:56086 2014-08-11 17:07:23.045357 2014-08-13 17:38:02.890077 7fe9bae887a0 20 read_log 192888'12029 (192888'11951) modify a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.942868.0:56096 2014-08-11 17:07:23.049234 2014-08-13 17:38:02.890090 7fe9bae887a0 20 read_log 192888'12030 (192888'11953) modify bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.942868.0:56341 2014-08-11 17:07:23.121807 2014-08-13 17:38:02.890102 7fe9bae887a0 20 read_log 192888'12032 (192888'11957) modify 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.942868.0:56783 2014-08-11 17:07:23.257591 2014-08-13 17:38:02.890114 7fe9bae887a0 20 read_log 192888'12033 (192888'11959) modify ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.942868.0:57182 2014-08-11 17:07:23.380512 2014-08-13 17:38:02.890126 7fe9bae887a0 20 read_log 192888'12034 (192888'11961) modify 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.942868.0:57384 2014-08-11 17:07:23.440310 2014-08-13 17:38:02.890138 7fe9bae887a0 20 read_log 192888'12036 (192888'11965) modify 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.942868.0:58414 2014-08-11 17:07:23.736170 2014-08-13 17:38:02.890150 7fe9bae887a0 20 read_log 192888'12037 (192888'11967) modify dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.942868.0:58666 2014-08-11 17:07:23.812909 2014-08-13 17:38:02.890170 7fe9bae887a0 20 read_log 192888'12039 (192888'11971) modify 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.942868.0:58916 2014-08-11 17:07:23.889480 2014-08-13 17:38:02.890183 7fe9bae887a0 20 read_log 192888'12040 (192888'11973) modify d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.942868.0:59713 2014-08-11 17:07:24.127168 2014-08-13 17:38:02.890203 7fe9bae887a0 20 read_log 192888'12041 (192888'11975) modify d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.942868.0:59909 2014-08-11 17:07:24.184316 2014-08-13 17:38:02.890215 7fe9bae887a0 20 read_log 192888'12042 (192888'11977) modify 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.942868.0:60118 2014-08-11 17:07:24.246018 2014-08-13 17:38:02.890227 7fe9bae887a0 20 read_log 192888'12044 (192888'11981) modify 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.942868.0:61261 2014-08-11 17:07:24.587748 2014-08-13 17:38:02.890240 7fe9bae887a0 20 read_log 193305'12047 (193305'12047) modify 1bb/hit_set_24.1bb_archive_2014-08-11 18:37:04.914122_2014-08-11 18:42:37.986871/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.987093 2014-08-13 17:38:02.890252 7fe9bae887a0 20 read_log 193305'12048 (192888'11862) delete 1bb/hit_set_24.1bb_archive_2014-08-11 16:44:58.225727_2014-08-11 17:05:25.545358/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.987093 2014-08-13 17:38:02.890266 7fe9bae887a0 20 read_log 193305'12049 (0'0) promote 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by osd.4.0:15 2014-08-11 18:42:38.026009 2014-08-13 17:38:02.890278 7fe9bae887a0 20 read_log 193305'12050 (193305'12049) modify 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.951337.0:1280 2014-08-11 18:42:37.980089 2014-08-13 17:38:02.890290 7fe9bae887a0 20 read_log 193305'12051 (0'0) promote 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by osd.4.0:33 2014-08-11 18:42:41.229264 2014-08-13 17:38:02.890302 7fe9bae887a0 20 read_log 193305'12052 (193305'12051) modify 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.951337.0:2129 2014-08-11 18:42:41.221561 2014-08-13 17:38:02.890314 7fe9bae887a0 20 read_log 193305'12053 (0'0) promote 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by osd.4.0:63 2014-08-11 18:42:49.809111 2014-08-13 17:38:02.890342 7fe9bae887a0 20 read_log 193305'12054 (193305'12053) modify 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.951337.0:4479 2014-08-11 18:42:49.802550 2014-08-13 17:38:02.890354 7fe9bae887a0 20 read_log 193305'12055 (0'0) promote cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by osd.4.0:67 2014-08-11 18:42:50.636458 2014-08-13 17:38:02.890366 7fe9bae887a0 20 read_log 193305'12056 (193305'12055) modify cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.951337.0:4705 2014-08-11 18:42:50.629019 2014-08-13 17:38:02.890378 7fe9bae887a0 20 read_log 193305'12057 (0'0) promote 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by osd.4.0:111 2014-08-11 18:42:58.663345 2014-08-13 17:38:02.890390 7fe9bae887a0 20 read_log 193305'12058 (193305'12057) modify 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.951337.0:6894 2014-08-11 18:42:58.653891 2014-08-13 17:38:02.890402 7fe9bae887a0 20 read_log 193305'12059 (0'0) promote b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by osd.4.0:123 2014-08-11 18:43:01.155315 2014-08-13 17:38:02.890414 7fe9bae887a0 20 read_log 193305'12060 (193305'12059) modify b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.951337.0:7583 2014-08-11 18:43:01.148102 2014-08-13 17:38:02.890426 7fe9bae887a0 20 read_log 193305'12061 (0'0) promote 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by osd.4.0:167 2014-08-11 18:43:10.670107 2014-08-13 17:38:02.890438 7fe9bae887a0 20 read_log 193305'12062 (193305'12061) modify 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.951337.0:10104 2014-08-11 18:43:10.662692 2014-08-13 17:38:02.890460 7fe9bae887a0 20 read_log 193305'12063 (0'0) promote 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by osd.4.0:211 2014-08-11 18:43:21.566919 2014-08-13 17:38:02.890476 7fe9bae887a0 20 read_log 193305'12064 (193305'12063) modify 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.951337.0:12982 2014-08-11 18:43:21.558135 2014-08-13 17:38:02.890488 7fe9bae887a0 20 read_log 193305'12065 (0'0) promote 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by osd.4.0:217 2014-08-11 18:43:22.828418 2014-08-13 17:38:02.890500 7fe9bae887a0 20 read_log 193305'12066 (193305'12065) modify 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.951337.0:13320 2014-08-11 18:43:22.820344 2014-08-13 17:38:02.890512 7fe9bae887a0 20 read_log 193305'12067 (0'0) promote d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by osd.4.0:223 2014-08-11 18:43:24.855939 2014-08-13 17:38:02.890524 7fe9bae887a0 20 read_log 193305'12068 (193305'12067) modify d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.951337.0:13849 2014-08-11 18:43:24.848444 2014-08-13 17:38:02.890536 7fe9bae887a0 20 read_log 193305'12069 (0'0) promote 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by osd.4.0:229 2014-08-11 18:43:26.370081 2014-08-13 17:38:02.890548 7fe9bae887a0 20 read_log 193305'12070 (193305'12069) modify 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.951337.0:14240 2014-08-11 18:43:26.362734 2014-08-13 17:38:02.890564 7fe9bae887a0 20 read_log 193305'12071 (0'0) promote 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by osd.4.0:231 2014-08-11 18:43:26.526152 2014-08-13 17:38:02.890576 7fe9bae887a0 20 read_log 193305'12072 (193305'12071) modify 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.951337.0:14288 2014-08-11 18:43:26.518369 2014-08-13 17:38:02.890588 7fe9bae887a0 20 read_log 193305'12073 (0'0) promote c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by osd.4.0:261 2014-08-11 18:43:33.227509 2014-08-13 17:38:02.890600 7fe9bae887a0 20 read_log 193305'12074 (193305'12073) modify c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.951337.0:16227 2014-08-11 18:43:33.217510 2014-08-13 17:38:02.890612 7fe9bae887a0 20 read_log 193305'12075 (0'0) promote e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by osd.4.0:285 2014-08-11 18:43:36.452413 2014-08-13 17:38:02.890624 7fe9bae887a0 20 read_log 193305'12076 (193305'12075) modify e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.951337.0:17140 2014-08-11 18:43:36.442932 2014-08-13 17:38:02.890635 7fe9bae887a0 20 read_log 193305'12077 (0'0) promote 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by osd.4.0:307 2014-08-11 18:43:39.825570 2014-08-13 17:38:02.890647 7fe9bae887a0 20 read_log 193305'12078 (193305'12077) modify 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.951337.0:18130 2014-08-11 18:43:39.814482 2014-08-13 17:38:02.890659 7fe9bae887a0 20 read_log 193305'12079 (0'0) promote a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by osd.4.0:315 2014-08-11 18:43:41.733560 2014-08-13 17:38:02.890671 7fe9bae887a0 20 read_log 193305'12080 (193305'12079) modify a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.951337.0:18677 2014-08-11 18:43:41.724114 2014-08-13 17:38:02.890683 7fe9bae887a0 20 read_log 193305'12081 (0'0) promote 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by osd.4.0:325 2014-08-11 18:43:43.624138 2014-08-13 17:38:02.890695 7fe9bae887a0 20 read_log 193305'12082 (193305'12081) modify 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.951337.0:19171 2014-08-11 18:43:43.617250 2014-08-13 17:38:02.890714 7fe9bae887a0 20 read_log 193305'12083 (0'0) promote 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by osd.4.0:331 2014-08-11 18:43:45.894510 2014-08-13 17:38:02.890727 7fe9bae887a0 20 read_log 193305'12084 (193305'12083) modify 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.951337.0:19797 2014-08-11 18:43:45.886968 2014-08-13 17:38:02.890739 7fe9bae887a0 20 read_log 193305'12085 (0'0) promote a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by osd.4.0:333 2014-08-11 18:43:46.139474 2014-08-13 17:38:02.890751 7fe9bae887a0 20 read_log 193305'12086 (193305'12085) modify a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.951337.0:19868 2014-08-11 18:43:46.132956 2014-08-13 17:38:02.890763 7fe9bae887a0 20 read_log 193305'12087 (0'0) promote 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by osd.4.0:345 2014-08-11 18:43:49.066255 2014-08-13 17:38:02.890775 7fe9bae887a0 20 read_log 193305'12088 (193305'12087) modify 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.951337.0:20679 2014-08-11 18:43:49.058618 2014-08-13 17:38:02.890787 7fe9bae887a0 20 read_log 193305'12089 (0'0) promote 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by osd.4.0:385 2014-08-11 18:43:56.506626 2014-08-13 17:38:02.890799 7fe9bae887a0 20 read_log 193305'12090 (193305'12089) modify 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.951337.0:22732 2014-08-11 18:43:56.497823 2014-08-13 17:38:02.890811 7fe9bae887a0 20 read_log 193305'12091 (0'0) promote 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by osd.4.0:447 2014-08-11 18:44:11.416562 2014-08-13 17:38:02.890823 7fe9bae887a0 20 read_log 193305'12092 (193305'12091) modify 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.951337.0:26841 2014-08-11 18:44:11.406100 2014-08-13 17:38:02.890835 7fe9bae887a0 20 read_log 193305'12093 (0'0) promote 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by osd.4.0:457 2014-08-11 18:44:14.003452 2014-08-13 17:38:02.890849 7fe9bae887a0 20 read_log 193305'12094 (193305'12093) modify 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.951337.0:27533 2014-08-11 18:44:13.996081 2014-08-13 17:38:02.890861 7fe9bae887a0 20 read_log 193305'12095 (0'0) promote 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by osd.4.0:463 2014-08-11 18:44:16.180664 2014-08-13 17:38:02.890873 7fe9bae887a0 20 read_log 193305'12096 (193305'12095) modify 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.951337.0:28149 2014-08-11 18:44:16.173356 2014-08-13 17:38:02.890885 7fe9bae887a0 20 read_log 193305'12097 (0'0) promote 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by osd.4.0:477 2014-08-11 18:44:19.315825 2014-08-13 17:38:02.890897 7fe9bae887a0 20 read_log 193305'12098 (193305'12097) modify 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.951337.0:28992 2014-08-11 18:44:19.307168 2014-08-13 17:38:02.890909 7fe9bae887a0 20 read_log 193305'12099 (0'0) promote 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by osd.4.0:481 2014-08-11 18:44:20.123729 2014-08-13 17:38:02.890921 7fe9bae887a0 20 read_log 193305'12100 (193305'12099) modify 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.951337.0:29215 2014-08-11 18:44:20.115801 2014-08-13 17:38:02.890933 7fe9bae887a0 20 read_log 193305'12101 (0'0) promote eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by osd.4.0:491 2014-08-11 18:44:22.748683 2014-08-13 17:38:02.890945 7fe9bae887a0 20 read_log 193305'12102 (193305'12101) modify eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.951337.0:29932 2014-08-11 18:44:22.741284 2014-08-13 17:38:02.890968 7fe9bae887a0 20 read_log 193305'12103 (0'0) promote fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by osd.4.0:499 2014-08-11 18:44:24.257088 2014-08-13 17:38:02.890981 7fe9bae887a0 20 read_log 193305'12104 (193305'12103) modify fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.951337.0:30315 2014-08-11 18:44:24.249432 2014-08-13 17:38:02.890994 7fe9bae887a0 20 read_log 193305'12105 (0'0) promote f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by osd.4.0:509 2014-08-11 18:44:28.123078 2014-08-13 17:38:02.891006 7fe9bae887a0 20 read_log 193305'12106 (193305'12105) modify f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.951337.0:31367 2014-08-11 18:44:28.114410 2014-08-13 17:38:02.891018 7fe9bae887a0 20 read_log 193305'12107 (0'0) promote ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by osd.4.0:517 2014-08-11 18:44:29.336942 2014-08-13 17:38:02.891030 7fe9bae887a0 20 read_log 193305'12108 (193305'12107) modify ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.951337.0:31725 2014-08-11 18:44:29.329961 2014-08-13 17:38:02.891042 7fe9bae887a0 20 read_log 193305'12109 (193305'12050) modify 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.951337.0:33991 2014-08-11 18:44:34.750830 2014-08-13 17:38:02.891054 7fe9bae887a0 20 read_log 193305'12110 (193305'12052) modify 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.951337.0:34840 2014-08-11 18:44:35.216510 2014-08-13 17:38:02.891066 7fe9bae887a0 20 read_log 193305'12111 (193305'12054) modify 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.951337.0:37190 2014-08-11 18:44:36.755779 2014-08-13 17:38:02.891078 7fe9bae887a0 20 read_log 193305'12112 (193305'12056) modify cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.951337.0:37416 2014-08-11 18:44:36.900744 2014-08-13 17:38:02.891089 7fe9bae887a0 20 read_log 193305'12113 (193305'12058) modify 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.951337.0:39605 2014-08-11 18:44:38.108753 2014-08-13 17:38:02.891101 7fe9bae887a0 20 read_log 193305'12114 (193305'12060) modify b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.951337.0:40294 2014-08-11 18:44:38.365393 2014-08-13 17:38:02.891113 7fe9bae887a0 20 read_log 193305'12115 (193305'12062) modify 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.951337.0:42815 2014-08-11 18:44:39.389359 2014-08-13 17:38:02.891125 7fe9bae887a0 20 read_log 193305'12116 (193305'12064) modify 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.951337.0:45693 2014-08-11 18:44:40.273962 2014-08-13 17:38:02.891137 7fe9bae887a0 20 read_log 193305'12117 (193305'12066) modify 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.951337.0:46031 2014-08-11 18:44:40.370671 2014-08-13 17:38:02.891149 7fe9bae887a0 20 read_log 193305'12118 (193305'12068) modify d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.951337.0:46560 2014-08-11 18:44:40.514455 2014-08-13 17:38:02.891172 7fe9bae887a0 20 read_log 193305'12119 (193305'12070) modify 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.951337.0:46951 2014-08-11 18:44:40.635424 2014-08-13 17:38:02.891184 7fe9bae887a0 20 read_log 193305'12120 (193305'12072) modify 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.951337.0:46999 2014-08-11 18:44:40.647881 2014-08-13 17:38:02.891197 7fe9bae887a0 20 read_log 193305'12121 (193305'12074) modify c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.951337.0:48938 2014-08-11 18:44:41.201616 2014-08-13 17:38:02.891209 7fe9bae887a0 20 read_log 193305'12122 (193305'12076) modify e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.951337.0:49851 2014-08-11 18:44:41.493099 2014-08-13 17:38:02.891230 7fe9bae887a0 20 read_log 193305'12123 (193305'12078) modify 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.951337.0:50841 2014-08-11 18:44:41.775246 2014-08-13 17:38:02.891243 7fe9bae887a0 20 read_log 193305'12124 (193305'12080) modify a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.951337.0:51388 2014-08-11 18:44:41.938036 2014-08-13 17:38:02.891256 7fe9bae887a0 20 read_log 193305'12125 (193305'12082) modify 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.951337.0:51882 2014-08-11 18:44:42.077316 2014-08-13 17:38:02.891268 7fe9bae887a0 20 read_log 193305'12126 (193305'12084) modify 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.951337.0:52508 2014-08-11 18:44:42.253145 2014-08-13 17:38:02.891280 7fe9bae887a0 20 read_log 193305'12127 (193305'12086) modify a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.951337.0:52579 2014-08-11 18:44:42.275137 2014-08-13 17:38:02.891292 7fe9bae887a0 20 read_log 193305'12128 (193305'12088) modify 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.951337.0:53390 2014-08-11 18:44:42.517158 2014-08-13 17:38:02.891304 7fe9bae887a0 20 read_log 193305'12129 (193305'12090) modify 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.951337.0:55443 2014-08-11 18:44:43.124528 2014-08-13 17:38:02.891316 7fe9bae887a0 20 read_log 193305'12130 (193305'12092) modify 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.951337.0:59552 2014-08-11 18:44:44.360577 2014-08-13 17:38:02.891328 7fe9bae887a0 20 read_log 193305'12131 (193305'12094) modify 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.951337.0:60244 2014-08-11 18:44:44.569371 2014-08-13 17:38:02.891340 7fe9bae887a0 20 read_log 193305'12132 (193305'12096) modify 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.951337.0:60860 2014-08-11 18:44:44.755311 2014-08-13 17:38:02.891352 7fe9bae887a0 20 read_log 193305'12133 (193305'12098) modify 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.951337.0:61703 2014-08-11 18:44:45.006724 2014-08-13 17:38:02.891364 7fe9bae887a0 20 read_log 193305'12134 (193305'12100) modify 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.951337.0:61926 2014-08-11 18:44:45.074299 2014-08-13 17:38:02.891377 7fe9bae887a0 20 read_log 193305'12135 (193305'12102) modify eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.951337.0:62643 2014-08-11 18:44:45.287657 2014-08-13 17:38:02.891388 7fe9bae887a0 20 read_log 193305'12136 (193305'12104) modify fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.951337.0:63026 2014-08-11 18:44:45.391044 2014-08-13 17:38:02.891400 7fe9bae887a0 20 read_log 193305'12137 (193305'12106) modify f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.951337.0:64078 2014-08-11 18:44:45.696891 2014-08-13 17:38:02.891412 7fe9bae887a0 20 read_log 193305'12138 (193305'12108) modify ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.951337.0:64436 2014-08-11 18:44:45.807003 2014-08-13 17:38:02.891424 7fe9bae887a0 20 read_log 194215'12139 (194215'12139) modify 1bb/hit_set_24.1bb_archive_2014-08-12 15:32:25.863957_2014-08-12 15:43:01.047258/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:01.047846 2014-08-13 17:38:02.891437 7fe9bae887a0 20 read_log 194215'12140 (193305'12047) delete 1bb/hit_set_24.1bb_archive_2014-08-11 18:37:04.914122_2014-08-11 18:42:37.986871/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:01.047846 2014-08-13 17:38:02.891449 7fe9bae887a0 20 read_log 194215'12141 (192834'11016) clean 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by osd.4.0:1974 2014-08-12 15:43:01.062473 2014-08-13 17:38:02.891469 7fe9bae887a0 20 read_log 194215'12142 (194215'12141) delete 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.2174595.0:91136 0.000000 2014-08-13 17:38:02.891480 7fe9bae887a0 20 read_log 194215'12143 (192888'12003) clean 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by osd.4.0:1976 2014-08-12 15:43:01.082963 2014-08-13 17:38:02.891493 7fe9bae887a0 20 read_log 194215'12144 (194215'12143) delete 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.2174595.0:91138 0.000000 2014-08-13 17:38:02.891503 7fe9bae887a0 20 read_log 194215'12145 (192834'11626) clean 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by osd.4.0:1978 2014-08-12 15:43:01.092092 2014-08-13 17:38:02.891515 7fe9bae887a0 20 read_log 194215'12146 (194215'12145) delete 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.2174595.0:91140 0.000000 2014-08-13 17:38:02.891565 7fe9bae887a0 20 read_log 194215'12147 (193305'12129) clean 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by osd.4.0:1980 2014-08-12 15:43:01.101072 2014-08-13 17:38:02.891582 7fe9bae887a0 20 read_log 194215'12148 (194215'12147) delete 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.2174595.0:91142 0.000000 2014-08-13 17:38:02.891593 7fe9bae887a0 20 read_log 194215'12149 (192888'12018) clean c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by osd.4.0:1982 2014-08-12 15:43:01.125229 2014-08-13 17:38:02.891605 7fe9bae887a0 20 read_log 194215'12150 (194215'12149) delete c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.2174595.0:91144 0.000000 2014-08-13 17:38:02.891620 7fe9bae887a0 20 read_log 194215'12151 (192834'11500) clean bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by osd.4.0:1984 2014-08-12 15:43:01.132822 2014-08-13 17:38:02.891632 7fe9bae887a0 20 read_log 194215'12152 (194215'12151) delete bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.2174595.0:91146 0.000000 2014-08-13 17:38:02.891643 7fe9bae887a0 20 read_log 194215'12153 (192888'12036) clean 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by osd.4.0:1986 2014-08-12 15:43:01.159084 2014-08-13 17:38:02.891654 7fe9bae887a0 20 read_log 194215'12154 (194215'12153) delete 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.2174595.0:91148 0.000000 2014-08-13 17:38:02.891665 7fe9bae887a0 20 read_log 194215'12155 (193305'12131) clean 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by osd.4.0:1988 2014-08-12 15:43:01.169163 2014-08-13 17:38:02.891677 7fe9bae887a0 20 read_log 194215'12156 (194215'12155) delete 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.2174595.0:91150 0.000000 2014-08-13 17:38:02.891687 7fe9bae887a0 20 read_log 194215'12157 (192834'10956) clean f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by osd.4.0:1990 2014-08-12 15:43:01.177916 2014-08-13 17:38:02.891699 7fe9bae887a0 20 read_log 194215'12158 (194215'12157) delete f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.2174595.0:91152 0.000000 2014-08-13 17:38:02.891710 7fe9bae887a0 20 read_log 194215'12159 (192888'12013) clean 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by osd.4.0:1992 2014-08-12 15:43:01.199038 2014-08-13 17:38:02.891722 7fe9bae887a0 20 read_log 194215'12160 (194215'12159) delete 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.2174595.0:91154 0.000000 2014-08-13 17:38:02.891733 7fe9bae887a0 20 read_log 194215'12161 (193305'12112) clean cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by osd.4.0:1994 2014-08-12 15:43:01.207468 2014-08-13 17:38:02.891753 7fe9bae887a0 20 read_log 194215'12162 (194215'12161) delete cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.2174595.0:91156 0.000000 2014-08-13 17:38:02.891764 7fe9bae887a0 20 read_log 194215'12163 (193305'12119) clean 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by osd.4.0:1996 2014-08-12 15:43:01.215417 2014-08-13 17:38:02.891776 7fe9bae887a0 20 read_log 194215'12164 (194215'12163) delete 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.2174595.0:91158 0.000000 2014-08-13 17:38:02.891787 7fe9bae887a0 20 read_log 194215'12165 (192888'11796) clean 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by osd.4.0:1998 2014-08-12 15:43:01.224201 2014-08-13 17:38:02.891799 7fe9bae887a0 20 read_log 194215'12166 (194215'12165) delete 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.2174595.0:91160 0.000000 2014-08-13 17:38:02.891809 7fe9bae887a0 20 read_log 194215'12167 (192888'11833) clean b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by osd.4.0:2000 2014-08-12 15:43:01.232796 2014-08-13 17:38:02.891821 7fe9bae887a0 20 read_log 194215'12168 (194215'12167) delete b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.2174595.0:91162 0.000000 2014-08-13 17:38:02.891832 7fe9bae887a0 20 read_log 194215'12169 (193305'12125) clean 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by osd.4.0:2002 2014-08-12 15:43:01.271760 2014-08-13 17:38:02.891844 7fe9bae887a0 20 read_log 194215'12170 (194215'12169) delete 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.2174595.0:91164 0.000000 2014-08-13 17:38:02.891855 7fe9bae887a0 20 read_log 194215'12171 (192888'11989) clean c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by osd.4.0:2004 2014-08-12 15:43:01.280633 2014-08-13 17:38:02.891866 7fe9bae887a0 20 read_log 194215'12172 (194215'12171) delete c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.2174595.0:91166 0.000000 2014-08-13 17:38:02.891877 7fe9bae887a0 20 read_log 194215'12173 (192888'11800) clean 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by osd.4.0:2006 2014-08-12 15:43:01.289852 2014-08-13 17:38:02.891889 7fe9bae887a0 20 read_log 194215'12174 (194215'12173) delete 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.2174595.0:91168 0.000000 2014-08-13 17:38:02.891899 7fe9bae887a0 20 read_log 194215'12175 (192834'10882) clean d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by osd.4.0:2008 2014-08-12 15:43:01.317002 2014-08-13 17:38:02.891911 7fe9bae887a0 20 read_log 194215'12176 (194215'12175) delete d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.2174595.0:91170 0.000000 2014-08-13 17:38:02.891922 7fe9bae887a0 20 read_log 194215'12177 (193305'12114) clean b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by osd.4.0:2010 2014-08-12 15:43:01.325368 2014-08-13 17:38:02.891934 7fe9bae887a0 20 read_log 194215'12178 (194215'12177) delete b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.2174595.0:91172 0.000000 2014-08-13 17:38:02.891944 7fe9bae887a0 20 read_log 194215'12179 (192834'10912) clean e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by osd.4.0:2012 2014-08-12 15:43:01.358444 2014-08-13 17:38:02.891956 7fe9bae887a0 20 read_log 194215'12180 (194215'12179) delete e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.2174595.0:91174 0.000000 2014-08-13 17:38:02.891966 7fe9bae887a0 20 read_log 194215'12181 (192834'11536) clean a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by osd.4.0:2014 2014-08-12 15:43:01.366447 2014-08-13 17:38:02.891987 7fe9bae887a0 20 read_log 194215'12182 (194215'12181) delete a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.2174595.0:91176 0.000000 2014-08-13 17:38:02.891999 7fe9bae887a0 20 read_log 194215'12183 (193305'12124) clean a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by osd.4.0:2016 2014-08-12 15:43:01.375336 2014-08-13 17:38:02.892011 7fe9bae887a0 20 read_log 194215'12184 (194215'12183) delete a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.2174595.0:91178 0.000000 2014-08-13 17:38:02.892022 7fe9bae887a0 20 read_log 194215'12185 (192888'11996) clean b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by osd.4.0:2018 2014-08-12 15:43:01.383416 2014-08-13 17:38:02.892034 7fe9bae887a0 20 read_log 194215'12186 (194215'12185) delete b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.2174595.0:91180 0.000000 2014-08-13 17:38:02.892045 7fe9bae887a0 20 read_log 194215'12187 (192888'11839) clean 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by osd.4.0:2020 2014-08-12 15:43:01.390428 2014-08-13 17:38:02.892057 7fe9bae887a0 20 read_log 194215'12188 (194215'12187) delete 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.2174595.0:91182 0.000000 2014-08-13 17:38:02.892067 7fe9bae887a0 20 read_log 194215'12189 (192834'11641) clean b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by osd.4.0:2022 2014-08-12 15:43:01.399080 2014-08-13 17:38:02.892079 7fe9bae887a0 20 read_log 194215'12190 (194215'12189) delete b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.2174595.0:91184 0.000000 2014-08-13 17:38:02.892090 7fe9bae887a0 20 read_log 194215'12191 (192834'11635) clean e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by osd.4.0:2024 2014-08-12 15:43:01.415487 2014-08-13 17:38:02.892102 7fe9bae887a0 20 read_log 194215'12192 (194215'12191) delete e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.2174595.0:91186 0.000000 2014-08-13 17:38:02.892113 7fe9bae887a0 20 read_log 194215'12193 (192888'11816) clean 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by osd.4.0:2026 2014-08-12 15:43:01.423103 2014-08-13 17:38:02.892125 7fe9bae887a0 20 read_log 194215'12194 (194215'12193) delete 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.2174595.0:91188 0.000000 2014-08-13 17:38:02.892135 7fe9bae887a0 20 read_log 194215'12195 (192834'10974) clean b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by osd.4.0:2028 2014-08-12 15:43:01.431392 2014-08-13 17:38:02.892147 7fe9bae887a0 20 read_log 194215'12196 (194215'12195) delete b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.2174595.0:91190 0.000000 2014-08-13 17:38:02.892163 7fe9bae887a0 20 read_log 194215'12197 (192834'10966) clean cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by osd.4.0:2030 2014-08-12 15:43:01.439942 2014-08-13 17:38:02.892175 7fe9bae887a0 20 read_log 194215'12198 (194215'12197) delete cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.2174595.0:91192 0.000000 2014-08-13 17:38:02.892186 7fe9bae887a0 20 read_log 194215'12199 (192834'10976) clean 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by osd.4.0:2032 2014-08-12 15:43:01.447709 2014-08-13 17:38:02.892198 7fe9bae887a0 20 read_log 194215'12200 (194215'12199) delete 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.2174595.0:91194 0.000000 2014-08-13 17:38:02.892209 7fe9bae887a0 20 read_log 194215'12201 (193305'12109) clean 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by osd.4.0:2034 2014-08-12 15:43:01.475831 2014-08-13 17:38:02.892227 7fe9bae887a0 20 read_log 194215'12202 (194215'12201) delete 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.2174595.0:91196 0.000000 2014-08-13 17:38:02.892241 7fe9bae887a0 20 read_log 194215'12203 (193305'12134) clean 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by osd.4.0:2036 2014-08-12 15:43:01.484596 2014-08-13 17:38:02.892254 7fe9bae887a0 20 read_log 194215'12204 (194215'12203) delete 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.2174595.0:91198 0.000000 2014-08-13 17:38:02.892264 7fe9bae887a0 20 read_log 194215'12205 (193305'12118) clean d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by osd.4.0:2038 2014-08-12 15:43:01.493062 2014-08-13 17:38:02.892276 7fe9bae887a0 20 read_log 194215'12206 (194215'12205) delete d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.2174595.0:91200 0.000000 2014-08-13 17:38:02.892287 7fe9bae887a0 20 read_log 194215'12207 (192834'10979) clean 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by osd.4.0:2040 2014-08-12 15:43:01.502501 2014-08-13 17:38:02.892299 7fe9bae887a0 20 read_log 194215'12208 (194215'12207) delete 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.2174595.0:91202 0.000000 2014-08-13 17:38:02.892309 7fe9bae887a0 20 read_log 194215'12209 (192888'11856) clean 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by osd.4.0:2042 2014-08-12 15:43:01.510973 2014-08-13 17:38:02.892322 7fe9bae887a0 20 read_log 194215'12210 (194215'12209) delete 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.2174595.0:91204 0.000000 2014-08-13 17:38:02.892332 7fe9bae887a0 20 read_log 194215'12211 (192888'11805) clean a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by osd.4.0:2044 2014-08-12 15:43:01.520792 2014-08-13 17:38:02.892344 7fe9bae887a0 20 read_log 194215'12212 (194215'12211) delete a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.2174595.0:91206 0.000000 2014-08-13 17:38:02.892354 7fe9bae887a0 20 read_log 194215'12213 (192888'11846) clean be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by osd.4.0:2046 2014-08-12 15:43:01.529006 2014-08-13 17:38:02.892367 7fe9bae887a0 20 read_log 194215'12214 (194215'12213) delete be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.2174595.0:91208 0.000000 2014-08-13 17:38:02.892377 7fe9bae887a0 20 read_log 194215'12215 (192834'10895) clean 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by osd.4.0:2048 2014-08-12 15:43:01.537214 2014-08-13 17:38:02.892389 7fe9bae887a0 20 read_log 194215'12216 (194215'12215) delete 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.2174595.0:91210 0.000000 2014-08-13 17:38:02.892399 7fe9bae887a0 20 read_log 194215'12217 (193305'12117) clean 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by osd.4.0:2050 2014-08-12 15:43:01.546200 2014-08-13 17:38:02.892411 7fe9bae887a0 20 read_log 194215'12218 (194215'12217) delete 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.2174595.0:91212 0.000000 2014-08-13 17:38:02.892422 7fe9bae887a0 20 read_log 194215'12219 (192888'12002) clean 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by osd.4.0:2052 2014-08-12 15:43:01.554017 2014-08-13 17:38:02.892434 7fe9bae887a0 20 read_log 194215'12220 (194215'12219) delete 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.2174595.0:91214 0.000000 2014-08-13 17:38:02.892444 7fe9bae887a0 20 read_log 194215'12221 (193305'12127) clean a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by osd.4.0:2054 2014-08-12 15:43:01.562008 2014-08-13 17:38:02.892463 7fe9bae887a0 20 read_log 194215'12222 (194215'12221) delete a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.2174595.0:91216 0.000000 2014-08-13 17:38:02.892474 7fe9bae887a0 20 read_log 194215'12223 (192888'11832) clean 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by osd.4.0:2056 2014-08-12 15:43:01.570499 2014-08-13 17:38:02.892486 7fe9bae887a0 20 read_log 194215'12224 (194215'12223) delete 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.2174595.0:91218 0.000000 2014-08-13 17:38:02.892497 7fe9bae887a0 20 read_log 194215'12225 (192888'11843) clean ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by osd.4.0:2058 2014-08-12 15:43:01.578588 2014-08-13 17:38:02.892509 7fe9bae887a0 20 read_log 194215'12226 (194215'12225) delete ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.2174595.0:91220 0.000000 2014-08-13 17:38:02.892519 7fe9bae887a0 20 read_log 194215'12227 (192834'11447) clean f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by osd.4.0:2060 2014-08-12 15:43:01.586578 2014-08-13 17:38:02.892531 7fe9bae887a0 20 read_log 194215'12228 (194215'12227) delete f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.2174595.0:91222 0.000000 2014-08-13 17:38:02.892541 7fe9bae887a0 20 read_log 194215'12229 (193305'12137) clean f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by osd.4.0:2062 2014-08-12 15:43:01.594942 2014-08-13 17:38:02.892553 7fe9bae887a0 20 read_log 194215'12230 (194215'12229) delete f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.2174595.0:91224 0.000000 2014-08-13 17:38:02.892564 7fe9bae887a0 20 read_log 194215'12231 (192888'12005) clean 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by osd.4.0:2064 2014-08-12 15:43:01.603052 2014-08-13 17:38:02.892576 7fe9bae887a0 20 read_log 194215'12232 (194215'12231) delete 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.2174595.0:91226 0.000000 2014-08-13 17:38:02.892588 7fe9bae887a0 20 read_log 194215'12233 (192834'10945) clean ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by osd.4.0:2066 2014-08-12 15:43:01.610619 2014-08-13 17:38:02.892600 7fe9bae887a0 20 read_log 194215'12234 (194215'12233) delete ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.2174595.0:91228 0.000000 2014-08-13 17:38:02.892610 7fe9bae887a0 20 read_log 194215'12235 (193305'12138) clean ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by osd.4.0:2068 2014-08-12 15:43:01.618313 2014-08-13 17:38:02.892622 7fe9bae887a0 20 read_log 194215'12236 (194215'12235) delete ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.2174595.0:91230 0.000000 2014-08-13 17:38:02.892633 7fe9bae887a0 20 read_log 194215'12237 (192888'12026) clean 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by osd.4.0:2070 2014-08-12 15:43:01.627948 2014-08-13 17:38:02.892645 7fe9bae887a0 20 read_log 194215'12238 (194215'12237) delete 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.2174595.0:91232 0.000000 2014-08-13 17:38:02.892658 7fe9bae887a0 20 read_log 194215'12239 (192834'11448) delete c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.2174595.0:91234 0.000000 2014-08-13 17:38:02.892688 7fe9bae887a0 20 read_log 194215'12240 (192834'11449) delete 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.2174595.0:91236 0.000000 2014-08-13 17:38:02.892700 7fe9bae887a0 20 read_log 194215'12241 (193305'12136) clean fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by osd.4.0:2074 2014-08-12 15:43:01.649557 2014-08-13 17:38:02.892734 7fe9bae887a0 20 read_log 194215'12242 (194215'12241) delete fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.2174595.0:91238 0.000000 2014-08-13 17:38:02.892749 7fe9bae887a0 20 read_log 194215'12243 (192888'12042) clean 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by osd.4.0:2076 2014-08-12 15:43:01.657739 2014-08-13 17:38:02.892762 7fe9bae887a0 20 read_log 194215'12244 (194215'12243) delete 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.2174595.0:91240 0.000000 2014-08-13 17:38:02.892773 7fe9bae887a0 20 read_log 194215'12245 (192834'11595) clean 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by osd.4.0:2078 2014-08-12 15:43:01.666949 2014-08-13 17:38:02.892785 7fe9bae887a0 20 read_log 194215'12246 (194215'12245) delete 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.2174595.0:91242 0.000000 2014-08-13 17:38:02.892795 7fe9bae887a0 20 read_log 194215'12247 (192834'11593) clean 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by osd.4.0:2080 2014-08-12 15:43:01.692863 2014-08-13 17:38:02.892807 7fe9bae887a0 20 read_log 194215'12248 (194215'12247) delete 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.2174595.0:91244 0.000000 2014-08-13 17:38:02.892818 7fe9bae887a0 20 read_log 194215'12249 (192834'11554) clean 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.4.0:2082 2014-08-12 15:43:01.709869 2014-08-13 17:38:02.892830 7fe9bae887a0 20 read_log 194215'12250 (194215'12249) delete 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.2174595.0:91246 0.000000 2014-08-13 17:38:02.892841 7fe9bae887a0 20 read_log 194215'12251 (192834'11560) clean 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.4.0:2084 2014-08-12 15:43:01.721246 2014-08-13 17:38:02.892852 7fe9bae887a0 20 read_log 194215'12252 (194215'12251) delete 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.2174595.0:91248 0.000000 2014-08-13 17:38:02.892863 7fe9bae887a0 20 read_log 194215'12253 (192888'11838) clean f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by osd.4.0:2086 2014-08-12 15:43:01.729438 2014-08-13 17:38:02.892875 7fe9bae887a0 20 read_log 194215'12254 (194215'12253) delete f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.2174595.0:91250 0.000000 2014-08-13 17:38:02.892885 7fe9bae887a0 20 read_log 194215'12255 (192888'11807) clean cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by osd.4.0:2088 2014-08-12 15:43:01.737263 2014-08-13 17:38:02.892897 7fe9bae887a0 20 read_log 194215'12256 (194215'12255) delete cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.2174595.0:91252 0.000000 2014-08-13 17:38:02.892908 7fe9bae887a0 20 read_log 194215'12257 (192888'12020) clean d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by osd.4.0:2090 2014-08-12 15:43:01.745848 2014-08-13 17:38:02.892920 7fe9bae887a0 20 read_log 194215'12258 (194215'12257) delete d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.2174595.0:91254 0.000000 2014-08-13 17:38:02.892930 7fe9bae887a0 20 read_log 194215'12259 (192888'12033) clean ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by osd.4.0:2092 2014-08-12 15:43:01.754735 2014-08-13 17:38:02.892942 7fe9bae887a0 20 read_log 194215'12260 (194215'12259) delete ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.2174595.0:91256 0.000000 2014-08-13 17:38:02.892953 7fe9bae887a0 20 read_log 194215'12261 (192834'11577) clean 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.4.0:2094 2014-08-12 15:43:01.766498 2014-08-13 17:38:02.892974 7fe9bae887a0 20 read_log 194215'12262 (194215'12261) delete 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.2174595.0:91258 0.000000 2014-08-13 17:38:02.892985 7fe9bae887a0 20 read_log 194215'12263 (192888'12044) clean 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by osd.4.0:2096 2014-08-12 15:43:01.775332 2014-08-13 17:38:02.892997 7fe9bae887a0 20 read_log 194215'12264 (194215'12263) delete 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.2174595.0:91260 0.000000 2014-08-13 17:38:02.893008 7fe9bae887a0 20 read_log 194215'12265 (193305'12126) clean 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by osd.4.0:2098 2014-08-12 15:43:01.782695 2014-08-13 17:38:02.893020 7fe9bae887a0 20 read_log 194215'12266 (194215'12265) delete 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.2174595.0:91262 0.000000 2014-08-13 17:38:02.893031 7fe9bae887a0 20 read_log 194215'12267 (192888'11801) clean 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by osd.4.0:2100 2014-08-12 15:43:01.791244 2014-08-13 17:38:02.893062 7fe9bae887a0 20 read_log 194215'12268 (194215'12267) delete 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.2174595.0:91264 0.000000 2014-08-13 17:38:02.893073 7fe9bae887a0 20 read_log 194215'12269 (192888'12010) clean 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by osd.4.0:2102 2014-08-12 15:43:01.800773 2014-08-13 17:38:02.893085 7fe9bae887a0 20 read_log 194215'12270 (194215'12269) delete 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.2174595.0:91266 0.000000 2014-08-13 17:38:02.893096 7fe9bae887a0 20 read_log 194215'12271 (192888'11823) clean 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by osd.4.0:2104 2014-08-12 15:43:01.809029 2014-08-13 17:38:02.893109 7fe9bae887a0 20 read_log 194215'12272 (194215'12271) delete 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.2174595.0:91268 0.000000 2014-08-13 17:38:02.893120 7fe9bae887a0 20 read_log 194215'12273 (192888'11990) clean 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by osd.4.0:2106 2014-08-12 15:43:01.818077 2014-08-13 17:38:02.893132 7fe9bae887a0 20 read_log 194215'12274 (194215'12273) delete 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.2174595.0:91270 0.000000 2014-08-13 17:38:02.893142 7fe9bae887a0 20 read_log 194215'12275 (192834'11550) clean 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.4.0:2108 2014-08-12 15:43:01.831228 2014-08-13 17:38:02.893159 7fe9bae887a0 20 read_log 194215'12276 (194215'12275) delete 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.2174595.0:91272 0.000000 2014-08-13 17:38:02.893171 7fe9bae887a0 20 read_log 194215'12277 (192834'11659) clean 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by osd.4.0:2110 2014-08-12 15:43:01.839735 2014-08-13 17:38:02.893184 7fe9bae887a0 20 read_log 194215'12278 (194215'12277) delete 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.2174595.0:91274 0.000000 2014-08-13 17:38:02.893195 7fe9bae887a0 20 read_log 194215'12279 (192888'12028) clean fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by osd.4.0:2112 2014-08-12 15:43:01.847838 2014-08-13 17:38:02.893207 7fe9bae887a0 20 read_log 194215'12280 (194215'12279) delete fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.2174595.0:91276 0.000000 2014-08-13 17:38:02.893217 7fe9bae887a0 20 read_log 194215'12281 (192888'12016) clean 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by osd.4.0:2114 2014-08-12 15:43:01.856134 2014-08-13 17:38:02.893236 7fe9bae887a0 20 read_log 194215'12282 (194215'12281) delete 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.2174595.0:91278 0.000000 2014-08-13 17:38:02.893248 7fe9bae887a0 20 read_log 194215'12283 (192834'11454) delete b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.2174595.0:91280 0.000000 2014-08-13 17:38:02.893258 7fe9bae887a0 20 read_log 194215'12284 (193305'12120) clean 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by osd.4.0:2117 2014-08-12 15:43:01.868452 2014-08-13 17:38:02.893271 7fe9bae887a0 20 read_log 194215'12285 (194215'12284) delete 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.2174595.0:91282 0.000000 2014-08-13 17:38:02.893281 7fe9bae887a0 20 read_log 194215'12286 (192834'11456) delete c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.2174595.0:91284 0.000000 2014-08-13 17:38:02.893292 7fe9bae887a0 20 read_log 194215'12287 (193305'12133) clean 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by osd.4.0:2120 2014-08-12 15:43:01.881741 2014-08-13 17:38:02.893304 7fe9bae887a0 20 read_log 194215'12288 (194215'12287) delete 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.2174595.0:91286 0.000000 2014-08-13 17:38:02.893314 7fe9bae887a0 20 read_log 194215'12289 (192888'11987) clean e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by osd.4.0:2122 2014-08-12 15:43:01.890084 2014-08-13 17:38:02.893326 7fe9bae887a0 20 read_log 194215'12290 (194215'12289) delete e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.2174595.0:91288 0.000000 2014-08-13 17:38:02.893337 7fe9bae887a0 20 read_log 194215'12291 (193305'12116) clean 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by osd.4.0:2124 2014-08-12 15:43:01.898213 2014-08-13 17:38:02.893349 7fe9bae887a0 20 read_log 194215'12292 (194215'12291) delete 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.2174595.0:91290 0.000000 2014-08-13 17:38:02.893360 7fe9bae887a0 20 read_log 194215'12293 (193305'12111) clean 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by osd.4.0:2126 2014-08-12 15:43:01.906248 2014-08-13 17:38:02.893372 7fe9bae887a0 20 read_log 194215'12294 (194215'12293) delete 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.2174595.0:91292 0.000000 2014-08-13 17:38:02.893382 7fe9bae887a0 20 read_log 194215'12295 (192834'11457) delete bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.2174595.0:91294 0.000000 2014-08-13 17:38:02.893393 7fe9bae887a0 20 read_log 194215'12296 (192834'11510) clean 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.4.0:2129 2014-08-12 15:43:01.923826 2014-08-13 17:38:02.893404 7fe9bae887a0 20 read_log 194215'12297 (194215'12296) delete 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.2174595.0:91296 0.000000 2014-08-13 17:38:02.893415 7fe9bae887a0 20 read_log 194215'12298 (192888'12014) clean 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by osd.4.0:2131 2014-08-12 15:43:01.933176 2014-08-13 17:38:02.893427 7fe9bae887a0 20 read_log 194215'12299 (194215'12298) delete 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.2174595.0:91298 0.000000 2014-08-13 17:38:02.893437 7fe9bae887a0 20 read_log 194215'12300 (192834'11487) clean ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.4.0:2133 2014-08-12 15:43:01.944654 2014-08-13 17:38:02.893449 7fe9bae887a0 20 read_log 194215'12301 (194215'12300) delete ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.2174595.0:91300 0.000000 2014-08-13 17:38:02.893467 7fe9bae887a0 20 read_log 194215'12302 (192888'11837) clean f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by osd.4.0:2135 2014-08-12 15:43:01.952827 2014-08-13 17:38:02.893479 7fe9bae887a0 20 read_log 194215'12303 (194215'12302) delete f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.2174595.0:91302 0.000000 2014-08-13 17:38:02.893490 7fe9bae887a0 20 read_log 194215'12304 (192888'11834) clean 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by osd.4.0:2137 2014-08-12 15:43:01.960696 2014-08-13 17:38:02.893502 7fe9bae887a0 20 read_log 194215'12305 (194215'12304) delete 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.2174595.0:91304 0.000000 2014-08-13 17:38:02.893513 7fe9bae887a0 20 read_log 194215'12306 (192888'11994) clean 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by osd.4.0:2139 2014-08-12 15:43:01.987108 2014-08-13 17:38:02.893525 7fe9bae887a0 20 read_log 194215'12307 (194215'12306) delete 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.2174595.0:91306 0.000000 2014-08-13 17:38:02.893536 7fe9bae887a0 20 read_log 194215'12308 (192888'12041) clean d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by osd.4.0:2141 2014-08-12 15:43:01.995639 2014-08-13 17:38:02.893548 7fe9bae887a0 20 read_log 194215'12309 (194215'12308) delete d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.2174595.0:91308 0.000000 2014-08-13 17:38:02.893558 7fe9bae887a0 20 read_log 194215'12310 (193305'12115) clean 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by osd.4.0:2143 2014-08-12 15:43:02.003868 2014-08-13 17:38:02.893572 7fe9bae887a0 20 read_log 194215'12311 (194215'12310) delete 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.2174595.0:91310 0.000000 2014-08-13 17:38:02.893583 7fe9bae887a0 20 read_log 194215'12312 (192834'11628) clean eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by osd.4.0:2145 2014-08-12 15:43:02.012919 2014-08-13 17:38:02.893595 7fe9bae887a0 20 read_log 194215'12313 (194215'12312) delete eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.2174595.0:91312 0.000000 2014-08-13 17:38:02.893606 7fe9bae887a0 20 read_log 194215'12314 (193305'12123) clean 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by osd.4.0:2147 2014-08-12 15:43:02.021342 2014-08-13 17:38:02.893618 7fe9bae887a0 20 read_log 194215'12315 (194215'12314) delete 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.2174595.0:91314 0.000000 2014-08-13 17:38:02.893628 7fe9bae887a0 20 read_log 194215'12316 (192888'11997) clean bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by osd.4.0:2149 2014-08-12 15:43:02.029783 2014-08-13 17:38:02.893641 7fe9bae887a0 20 read_log 194215'12317 (194215'12316) delete bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.2174595.0:91316 0.000000 2014-08-13 17:38:02.893651 7fe9bae887a0 20 read_log 194215'12318 (192888'11851) clean 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by osd.4.0:2151 2014-08-12 15:43:02.037265 2014-08-13 17:38:02.893664 7fe9bae887a0 20 read_log 194215'12319 (194215'12318) delete 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.2174595.0:91318 0.000000 2014-08-13 17:38:02.893674 7fe9bae887a0 20 read_log 194215'12320 (193305'12122) clean e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by osd.4.0:2153 2014-08-12 15:43:02.046180 2014-08-13 17:38:02.893687 7fe9bae887a0 20 read_log 194215'12321 (194215'12320) delete e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.2174595.0:91320 0.000000 2014-08-13 17:38:02.893706 7fe9bae887a0 20 read_log 194215'12322 (192888'11860) clean f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by osd.4.0:2155 2014-08-12 15:43:02.054124 2014-08-13 17:38:02.893719 7fe9bae887a0 20 read_log 194215'12323 (194215'12322) delete f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.2174595.0:91322 0.000000 2014-08-13 17:38:02.893730 7fe9bae887a0 20 read_log 194215'12324 (192834'10994) clean 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by osd.4.0:2157 2014-08-12 15:43:02.063668 2014-08-13 17:38:02.893742 7fe9bae887a0 20 read_log 194215'12325 (194215'12324) delete 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.2174595.0:91324 0.000000 2014-08-13 17:38:02.893752 7fe9bae887a0 20 read_log 194215'12326 (192888'11822) clean 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by osd.4.0:2159 2014-08-12 15:43:02.072405 2014-08-13 17:38:02.893764 7fe9bae887a0 20 read_log 194215'12327 (194215'12326) delete 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.2174595.0:91326 0.000000 2014-08-13 17:38:02.893775 7fe9bae887a0 20 read_log 194215'12328 (192834'11482) clean 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by osd.4.0:2161 2014-08-12 15:43:02.081532 2014-08-13 17:38:02.893787 7fe9bae887a0 20 read_log 194215'12329 (194215'12328) delete 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.2174595.0:91328 0.000000 2014-08-13 17:38:02.893798 7fe9bae887a0 20 read_log 194215'12330 (193305'12113) clean 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by osd.4.0:2163 2014-08-12 15:43:02.089847 2014-08-13 17:38:02.893810 7fe9bae887a0 20 read_log 194215'12331 (194215'12330) delete 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.2174595.0:91330 0.000000 2014-08-13 17:38:02.893821 7fe9bae887a0 20 read_log 194215'12332 (193305'12132) clean 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by osd.4.0:2165 2014-08-12 15:43:02.098139 2014-08-13 17:38:02.893849 7fe9bae887a0 20 read_log 194215'12333 (194215'12332) delete 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.2174595.0:91332 0.000000 2014-08-13 17:38:02.893860 7fe9bae887a0 20 read_log 194215'12334 (192888'12000) clean cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by osd.4.0:2167 2014-08-12 15:43:02.108194 2014-08-13 17:38:02.893872 7fe9bae887a0 20 read_log 194215'12335 (194215'12334) delete cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.2174595.0:91334 0.000000 2014-08-13 17:38:02.893883 7fe9bae887a0 20 read_log 194215'12336 (192888'12030) clean bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by osd.4.0:2169 2014-08-12 15:43:02.116334 2014-08-13 17:38:02.893896 7fe9bae887a0 20 read_log 194215'12337 (194215'12336) delete bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.2174595.0:91336 0.000000 2014-08-13 17:38:02.893906 7fe9bae887a0 20 read_log 194215'12338 (192834'10978) clean c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by osd.4.0:2171 2014-08-12 15:43:02.124821 2014-08-13 17:38:02.893918 7fe9bae887a0 20 read_log 194215'12339 (194215'12338) delete c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.2174595.0:91338 0.000000 2014-08-13 17:38:02.893929 7fe9bae887a0 20 read_log 194215'12340 (192888'12037) clean dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by osd.4.0:2173 2014-08-12 15:43:02.133316 2014-08-13 17:38:02.893943 7fe9bae887a0 20 read_log 194215'12341 (194215'12340) delete dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.2174595.0:91340 0.000000 2014-08-13 17:38:02.893960 7fe9bae887a0 20 read_log 194215'12342 (192888'12006) clean 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by osd.4.0:2175 2014-08-12 15:43:02.141708 2014-08-13 17:38:02.893973 7fe9bae887a0 20 read_log 194215'12343 (194215'12342) delete 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.2174595.0:91342 0.000000 2014-08-13 17:38:02.893984 7fe9bae887a0 20 read_log 194215'12344 (192888'12032) clean 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by osd.4.0:2177 2014-08-12 15:43:02.182758 2014-08-13 17:38:02.893996 7fe9bae887a0 20 read_log 194215'12345 (194215'12344) delete 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.2174595.0:91344 0.000000 2014-08-13 17:38:02.894007 7fe9bae887a0 20 read_log 194215'12346 (192888'12029) clean a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by osd.4.0:2179 2014-08-12 15:43:02.191047 2014-08-13 17:38:02.894019 7fe9bae887a0 20 read_log 194215'12347 (194215'12346) delete a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.2174595.0:91346 0.000000 2014-08-13 17:38:02.894029 7fe9bae887a0 20 read_log 194215'12348 (193305'12128) clean 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by osd.4.0:2181 2014-08-12 15:43:02.199031 2014-08-13 17:38:02.894041 7fe9bae887a0 20 read_log 194215'12349 (194215'12348) delete 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.2174595.0:91348 0.000000 2014-08-13 17:38:02.894052 7fe9bae887a0 20 read_log 194215'12350 (192888'12023) clean d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by osd.4.0:2183 2014-08-12 15:43:02.208199 2014-08-13 17:38:02.894064 7fe9bae887a0 20 read_log 194215'12351 (194215'12350) delete d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.2174595.0:91350 0.000000 2014-08-13 17:38:02.894075 7fe9bae887a0 20 read_log 194215'12352 (192888'11840) clean 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by osd.4.0:2185 2014-08-12 15:43:02.216071 2014-08-13 17:38:02.894087 7fe9bae887a0 20 read_log 194215'12353 (194215'12352) delete 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.2174595.0:91352 0.000000 2014-08-13 17:38:02.894098 7fe9bae887a0 20 read_log 194215'12354 (192888'12007) clean 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by osd.4.0:2187 2014-08-12 15:43:02.224138 2014-08-13 17:38:02.894109 7fe9bae887a0 20 read_log 194215'12355 (194215'12354) delete 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.2174595.0:91354 0.000000 2014-08-13 17:38:02.894120 7fe9bae887a0 20 read_log 194215'12356 (192888'12027) clean a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by osd.4.0:2189 2014-08-12 15:43:02.231932 2014-08-13 17:38:02.894136 7fe9bae887a0 20 read_log 194215'12357 (194215'12356) delete a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.2174595.0:91356 0.000000 2014-08-13 17:38:02.894146 7fe9bae887a0 20 read_log 194215'12358 (192888'11995) clean 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by osd.4.0:2191 2014-08-12 15:43:02.240202 2014-08-13 17:38:02.894164 7fe9bae887a0 20 read_log 194215'12359 (194215'12358) delete 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.2174595.0:91358 0.000000 2014-08-13 17:38:02.894175 7fe9bae887a0 20 read_log 194215'12360 (192834'11571) clean 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by osd.4.0:2193 2014-08-12 15:43:02.248134 2014-08-13 17:38:02.894187 7fe9bae887a0 20 read_log 194215'12361 (194215'12360) delete 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.2174595.0:91360 0.000000 2014-08-13 17:38:02.894210 7fe9bae887a0 20 read_log 194215'12362 (192834'11023) clean 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by osd.4.0:2195 2014-08-12 15:43:02.257437 2014-08-13 17:38:02.894223 7fe9bae887a0 20 read_log 194215'12363 (194215'12362) delete 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.2174595.0:91362 0.000000 2014-08-13 17:38:02.894234 7fe9bae887a0 20 read_log 194215'12364 (192888'11847) clean b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by osd.4.0:2197 2014-08-12 15:43:02.266524 2014-08-13 17:38:02.894246 7fe9bae887a0 20 read_log 194215'12365 (194215'12364) delete b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.2174595.0:91364 0.000000 2014-08-13 17:38:02.894256 7fe9bae887a0 20 read_log 194215'12366 (192888'12034) clean 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by osd.4.0:2199 2014-08-12 15:43:02.276059 2014-08-13 17:38:02.894272 7fe9bae887a0 20 read_log 194215'12367 (194215'12366) delete 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.2174595.0:91366 0.000000 2014-08-13 17:38:02.894282 7fe9bae887a0 20 read_log 194215'12368 (192834'11540) clean 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by osd.4.0:2201 2014-08-12 15:43:02.284961 2014-08-13 17:38:02.894295 7fe9bae887a0 20 read_log 194215'12369 (194215'12368) delete 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.2174595.0:91368 0.000000 2014-08-13 17:38:02.894305 7fe9bae887a0 20 read_log 194215'12370 (192888'12004) clean 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by osd.4.0:2203 2014-08-12 15:43:02.294104 2014-08-13 17:38:02.894317 7fe9bae887a0 20 read_log 194215'12371 (194215'12370) delete 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.2174595.0:91370 0.000000 2014-08-13 17:38:02.894328 7fe9bae887a0 20 read_log 194215'12372 (192888'11827) clean 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by osd.4.0:2205 2014-08-12 15:43:02.301929 2014-08-13 17:38:02.894340 7fe9bae887a0 20 read_log 194215'12373 (194215'12372) delete 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.2174595.0:91372 0.000000 2014-08-13 17:38:02.894351 7fe9bae887a0 20 read_log 194215'12374 (192888'11986) clean 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by osd.4.0:2207 2014-08-12 15:43:02.310145 2014-08-13 17:38:02.894363 7fe9bae887a0 20 read_log 194215'12375 (194215'12374) delete 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.2174595.0:91374 0.000000 2014-08-13 17:38:02.894373 7fe9bae887a0 20 read_log 194215'12376 (192888'11844) clean 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by osd.4.0:2209 2014-08-12 15:43:02.318327 2014-08-13 17:38:02.894387 7fe9bae887a0 20 read_log 194215'12377 (194215'12376) delete 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.2174595.0:91376 0.000000 2014-08-13 17:38:02.894398 7fe9bae887a0 20 read_log 194215'12378 (192888'11831) clean b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by osd.4.0:2211 2014-08-12 15:43:02.325768 2014-08-13 17:38:02.894410 7fe9bae887a0 20 read_log 194215'12379 (194215'12378) delete b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.2174595.0:91378 0.000000 2014-08-13 17:38:02.894421 7fe9bae887a0 20 read_log 194215'12380 (193305'12130) clean 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by osd.4.0:2213 2014-08-12 15:43:02.335031 2014-08-13 17:38:02.894433 7fe9bae887a0 20 read_log 194215'12381 (194215'12380) delete 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.2174595.0:91380 0.000000 2014-08-13 17:38:02.894472 7fe9bae887a0 20 read_log 194215'12382 (192834'10971) clean e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by osd.4.0:2215 2014-08-12 15:43:02.343191 2014-08-13 17:38:02.894487 7fe9bae887a0 20 read_log 194215'12383 (194215'12382) delete e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.2174595.0:91382 0.000000 2014-08-13 17:38:02.894497 7fe9bae887a0 20 read_log 194215'12384 (192834'11602) clean ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by osd.4.0:2217 2014-08-12 15:43:02.350462 2014-08-13 17:38:02.894510 7fe9bae887a0 20 read_log 194215'12385 (194215'12384) delete ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.2174595.0:91384 0.000000 2014-08-13 17:38:02.894520 7fe9bae887a0 20 read_log 194215'12386 (192888'11821) clean cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by osd.4.0:2219 2014-08-12 15:43:02.357831 2014-08-13 17:38:02.894532 7fe9bae887a0 20 read_log 194215'12387 (194215'12386) delete cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.2174595.0:91386 0.000000 2014-08-13 17:38:02.894543 7fe9bae887a0 20 read_log 194215'12388 (192888'11813) clean f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by osd.4.0:2221 2014-08-12 15:43:02.366070 2014-08-13 17:38:02.894555 7fe9bae887a0 20 read_log 194215'12389 (194215'12388) delete f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.2174595.0:91388 0.000000 2014-08-13 17:38:02.894566 7fe9bae887a0 20 read_log 194215'12390 (192834'10926) clean ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by osd.4.0:2223 2014-08-12 15:43:02.373453 2014-08-13 17:38:02.894578 7fe9bae887a0 20 read_log 194215'12391 (194215'12390) delete ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.2174595.0:91390 0.000000 2014-08-13 17:38:02.894589 7fe9bae887a0 20 read_log 194215'12392 (192834'11614) clean f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by osd.4.0:2225 2014-08-12 15:43:02.382388 2014-08-13 17:38:02.894602 7fe9bae887a0 20 read_log 194215'12393 (194215'12392) delete f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.2174595.0:91392 0.000000 2014-08-13 17:38:02.894613 7fe9bae887a0 20 read_log 194215'12394 (192834'11621) clean e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by osd.4.0:2227 2014-08-12 15:43:02.391049 2014-08-13 17:38:02.894625 7fe9bae887a0 20 read_log 194215'12395 (194215'12394) delete e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.2174595.0:91394 0.000000 2014-08-13 17:38:02.894636 7fe9bae887a0 20 read_log 194215'12396 (192834'11473) clean a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by osd.4.0:2229 2014-08-12 15:43:02.399533 2014-08-13 17:38:02.894648 7fe9bae887a0 20 read_log 194215'12397 (194215'12396) delete a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.2174595.0:91396 0.000000 2014-08-13 17:38:02.894659 7fe9bae887a0 20 read_log 194215'12398 (193305'12110) clean 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by osd.4.0:2231 2014-08-12 15:43:02.407094 2014-08-13 17:38:02.894671 7fe9bae887a0 20 read_log 194215'12399 (194215'12398) delete 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.2174595.0:91398 0.000000 2014-08-13 17:38:02.894681 7fe9bae887a0 20 read_log 194215'12400 (192888'11804) clean 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by osd.4.0:2233 2014-08-12 15:43:02.430858 2014-08-13 17:38:02.894693 7fe9bae887a0 20 read_log 194215'12401 (194215'12400) delete 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.2174595.0:91400 0.000000 2014-08-13 17:38:02.894711 7fe9bae887a0 20 read_log 194215'12402 (193305'12121) clean c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by osd.4.0:2235 2014-08-12 15:43:02.438996 2014-08-13 17:38:02.894724 7fe9bae887a0 20 read_log 194215'12403 (194215'12402) delete c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.2174595.0:91402 0.000000 2014-08-13 17:38:02.894735 7fe9bae887a0 20 read_log 194215'12404 (193305'12135) clean eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by osd.4.0:2237 2014-08-12 15:43:02.447111 2014-08-13 17:38:02.894747 7fe9bae887a0 20 read_log 194215'12405 (194215'12404) delete eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.2174595.0:91404 0.000000 2014-08-13 17:38:02.894758 7fe9bae887a0 20 read_log 194215'12406 (192834'11491) clean 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by osd.4.0:2239 2014-08-12 15:43:02.455272 2014-08-13 17:38:02.894770 7fe9bae887a0 20 read_log 194215'12407 (194215'12406) delete 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.2174595.0:91406 0.000000 2014-08-13 17:38:02.894781 7fe9bae887a0 20 read_log 194215'12408 (192888'12040) clean d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by osd.4.0:2241 2014-08-12 15:43:02.464606 2014-08-13 17:38:02.894793 7fe9bae887a0 20 read_log 194215'12409 (194215'12408) delete d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.2174595.0:91408 0.000000 2014-08-13 17:38:02.894804 7fe9bae887a0 20 read_log 194215'12410 (192888'12039) clean 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by osd.4.0:2243 2014-08-12 15:43:02.473186 2014-08-13 17:38:02.894816 7fe9bae887a0 20 read_log 194215'12411 (194215'12410) delete 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.2174595.0:91410 0.000000 2014-08-13 17:38:02.894827 7fe9bae887a0 20 read_log 194215'12412 (192888'11858) clean 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by osd.4.0:2245 2014-08-12 15:43:02.481162 2014-08-13 17:38:02.894839 7fe9bae887a0 20 read_log 194215'12413 (194215'12412) delete 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.2174595.0:91412 0.000000 2014-08-13 17:38:02.894849 7fe9bae887a0 20 read_log 194215'12414 (194215'12414) modify 1bb/hit_set_24.1bb_archive_2014-08-12 15:43:01.048059_2014-08-12 17:24:40.865594/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.866303 2014-08-13 17:38:02.894862 7fe9bae887a0 20 read_log 194215'12415 (194215'12139) delete 1bb/hit_set_24.1bb_archive_2014-08-12 15:32:25.863957_2014-08-12 15:43:01.047258/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.866303 2014-08-13 17:38:02.894874 7fe9bae887a0 20 read_log 194215'12416 (0'0) promote 323fcdbb/benchmark_data_storage0111-ib.pocstor_34628_object247/head//24 by osd.4.0:5232 2014-08-12 17:24:40.871311 2014-08-13 17:38:02.894886 7fe9bae887a0 20 read_log 194215'12417 (194215'12416) modify 323fcdbb/benchmark_data_storage0111-ib.pocstor_34628_object247/head//24 by client.2193759.0:248 2014-08-12 17:24:40.858439 2014-08-13 17:38:02.894899 7fe9bae887a0 20 read_log 194217'12418 (0'0) promote be386dbb/benchmark_data_storage0111-ib.pocstor_39693_object71/head//24 by osd.4.0:5256 2014-08-12 17:25:03.578296 2014-08-13 17:38:02.894911 7fe9bae887a0 20 read_log 194217'12419 (194217'12418) modify be386dbb/benchmark_data_storage0111-ib.pocstor_39693_object71/head//24 by client.1487927.0:72 2014-08-12 17:25:03.562697 2014-08-13 17:38:02.894926 7fe9bae887a0 20 read_log 194370'12420 (0'0) promote 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by osd.332.0:3733 2014-08-12 20:30:17.571032 2014-08-13 17:38:02.894947 7fe9bae887a0 20 read_log 194370'12421 (194370'12420) modify 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by client.1508531.0:846 2014-08-12 20:30:17.563335 2014-08-13 17:38:02.894960 7fe9bae887a0 20 read_log 194370'12422 (0'0) promote fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by osd.332.0:3735 2014-08-12 20:30:17.738491 2014-08-13 17:38:02.894972 7fe9bae887a0 20 read_log 194370'12423 (194370'12422) modify fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by client.1508531.0:893 2014-08-12 20:30:17.732009 2014-08-13 17:38:02.894985 7fe9bae887a0 20 read_log 194370'12424 (0'0) promote 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by osd.332.0:3739 2014-08-12 20:30:18.221906 2014-08-13 17:38:02.895003 7fe9bae887a0 20 read_log 194370'12425 (194370'12424) modify 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by client.1508531.0:1027 2014-08-12 20:30:18.215574 2014-08-13 17:38:02.895033 7fe9bae887a0 20 read_log 194370'12426 (0'0) promote 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by osd.332.0:3743 2014-08-12 20:30:18.815967 2014-08-13 17:38:02.895046 7fe9bae887a0 20 read_log 194370'12427 (194370'12426) modify 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by client.1508531.0:1182 2014-08-12 20:30:18.808413 2014-08-13 17:38:02.895058 7fe9bae887a0 20 read_log 194370'12428 (0'0) promote 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by osd.332.0:3781 2014-08-12 20:30:24.474696 2014-08-13 17:38:02.895070 7fe9bae887a0 20 read_log 194370'12429 (194370'12428) modify 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by client.1508531.0:2705 2014-08-12 20:30:24.467345 2014-08-13 17:38:02.895082 7fe9bae887a0 20 read_log 194370'12430 (0'0) promote aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by osd.332.0:3809 2014-08-12 20:30:27.725673 2014-08-13 17:38:02.895095 7fe9bae887a0 20 read_log 194370'12431 (194370'12430) modify aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by client.1508531.0:3585 2014-08-12 20:30:27.716306 2014-08-13 17:38:02.895107 7fe9bae887a0 20 read_log 194370'12432 (0'0) promote 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by osd.332.0:3879 2014-08-12 20:30:38.978130 2014-08-13 17:38:02.895119 7fe9bae887a0 20 read_log 194370'12433 (194370'12432) modify 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by client.1508531.0:6282 2014-08-12 20:30:38.971071 2014-08-13 17:38:02.895131 7fe9bae887a0 20 read_log 194370'12434 (0'0) promote 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by osd.332.0:3889 2014-08-12 20:30:42.847116 2014-08-13 17:38:02.895144 7fe9bae887a0 20 read_log 194370'12435 (194370'12434) modify 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by client.1508531.0:7087 2014-08-12 20:30:42.838696 2014-08-13 17:38:02.895161 7fe9bae887a0 20 read_log 194370'12436 (0'0) promote 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by osd.332.0:3899 2014-08-12 20:30:44.638187 2014-08-13 17:38:02.895174 7fe9bae887a0 20 read_log 194370'12437 (194370'12436) modify 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by client.1508531.0:7455 2014-08-12 20:30:44.631213 2014-08-13 17:38:02.895186 7fe9bae887a0 20 read_log 194370'12438 (0'0) promote f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by osd.332.0:3913 2014-08-12 20:30:49.212601 2014-08-13 17:38:02.895198 7fe9bae887a0 20 read_log 194370'12439 (194370'12438) modify f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by client.1508531.0:8308 2014-08-12 20:30:49.206145 2014-08-13 17:38:02.895210 7fe9bae887a0 20 read_log 194370'12440 (0'0) promote 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by osd.332.0:3925 2014-08-12 20:30:52.140583 2014-08-13 17:38:02.895230 7fe9bae887a0 20 read_log 194370'12441 (194370'12440) modify 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by client.1508531.0:8814 2014-08-12 20:30:52.131128 2014-08-13 17:38:02.895243 7fe9bae887a0 20 read_log 194370'12442 (0'0) promote 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by osd.332.0:3965 2014-08-12 20:31:01.991803 2014-08-13 17:38:02.895255 7fe9bae887a0 20 read_log 194370'12443 (194370'12442) modify 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by client.1508531.0:10461 2014-08-12 20:31:01.983217 2014-08-13 17:38:02.895285 7fe9bae887a0 20 read_log 194370'12444 (0'0) promote eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by osd.332.0:3969 2014-08-12 20:31:02.883903 2014-08-13 17:38:02.895302 7fe9bae887a0 20 read_log 194370'12445 (194370'12444) modify eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by client.1508531.0:10604 2014-08-12 20:31:02.874965 2014-08-13 17:38:02.895315 7fe9bae887a0 20 read_log 194370'12446 (0'0) promote 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by osd.332.0:3989 2014-08-12 20:31:11.747534 2014-08-13 17:38:02.895327 7fe9bae887a0 20 read_log 194370'12447 (194370'12446) modify 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by client.1508531.0:12053 2014-08-12 20:31:11.739138 2014-08-13 17:38:02.895340 7fe9bae887a0 20 read_log 194370'12448 (0'0) promote 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by osd.332.0:3999 2014-08-12 20:31:14.764086 2014-08-13 17:38:02.895351 7fe9bae887a0 20 read_log 194370'12449 (194370'12448) modify 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by client.1508531.0:12467 2014-08-12 20:31:14.755486 2014-08-13 17:38:02.895364 7fe9bae887a0 20 read_log 194370'12450 (0'0) promote 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by osd.332.0:4011 2014-08-12 20:31:20.422594 2014-08-13 17:38:02.895375 7fe9bae887a0 20 read_log 194370'12451 (194370'12450) modify 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by client.1508531.0:13107 2014-08-12 20:31:20.413293 2014-08-13 17:38:02.895388 7fe9bae887a0 20 read_log 194377'12452 (0'0) promote efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by osd.332.0:4049 2014-08-12 20:33:00.376451 2014-08-13 17:38:02.895400 7fe9bae887a0 20 read_log 194377'12453 (194377'12452) modify efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by client.2231190.0:751 2014-08-12 20:33:00.367237 2014-08-13 17:38:02.895412 7fe9bae887a0 20 read_log 194377'12454 (0'0) promote ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by osd.332.0:4059 2014-08-12 20:33:02.566410 2014-08-13 17:38:02.895424 7fe9bae887a0 20 read_log 194377'12455 (194377'12454) modify ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by client.2231190.0:1331 2014-08-12 20:33:02.557160 2014-08-13 17:38:02.895436 7fe9bae887a0 20 read_log 194377'12456 (0'0) promote b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by osd.332.0:4071 2014-08-12 20:33:06.779043 2014-08-13 17:38:02.895447 7fe9bae887a0 20 read_log 194377'12457 (194377'12456) modify b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by client.2231190.0:2386 2014-08-12 20:33:06.773113 2014-08-13 17:38:02.895459 7fe9bae887a0 20 read_log 194377'12458 (0'0) promote c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by osd.332.0:4085 2014-08-12 20:33:09.554340 2014-08-13 17:38:02.895471 7fe9bae887a0 20 read_log 194377'12459 (194377'12458) modify c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by client.2231190.0:3058 2014-08-12 20:33:09.547018 2014-08-13 17:38:02.895483 7fe9bae887a0 20 read_log 194377'12460 (0'0) promote ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by osd.332.0:4097 2014-08-12 20:33:11.455609 2014-08-13 17:38:02.895503 7fe9bae887a0 20 read_log 194377'12461 (194377'12460) modify ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by client.2231190.0:3520 2014-08-12 20:33:11.447247 2014-08-13 17:38:02.895516 7fe9bae887a0 20 read_log 194397'12462 (194397'12462) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:37:08.016110_2014-08-12 20:53:01.910165/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:01.910318 2014-08-13 17:38:02.895528 7fe9bae887a0 20 read_log 194397'12463 (194215'12414) delete 1bb/hit_set_24.1bb_archive_2014-08-12 15:43:01.048059_2014-08-12 17:24:40.865594/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:01.910318 2014-08-13 17:38:02.895540 7fe9bae887a0 20 read_log 194397'12464 (0'0) promote b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by osd.332.0:4247 2014-08-12 20:53:01.913586 2014-08-13 17:38:02.895553 7fe9bae887a0 20 read_log 194397'12465 (194397'12464) modify b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by client.2235069.0:565 2014-08-12 20:53:01.904209 2014-08-13 17:38:02.895565 7fe9bae887a0 20 read_log 194416'12466 (194416'12466) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 17:38:02.895578 7fe9bae887a0 20 read_log 194416'12467 (194397'12462) delete 1bb/hit_set_24.1bb_archive_2014-08-12 20:37:08.016110_2014-08-12 20:53:01.910165/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 17:38:02.895590 7fe9bae887a0 20 read_log 194416'12468 (0'0) promote 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by osd.332.0:4385 2014-08-13 15:28:36.887696 2014-08-13 17:38:02.895602 7fe9bae887a0 20 read_log 194416'12469 (194416'12468) modify 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by client.2441958.0:1627 2014-08-13 15:28:36.876856 2014-08-13 17:38:02.895614 7fe9bae887a0 20 read_log 194416'12470 (0'0) promote 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by osd.332.0:4409 2014-08-13 15:28:41.435570 2014-08-13 17:38:02.895626 7fe9bae887a0 20 read_log 194416'12471 (194416'12470) modify 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by client.2441958.0:2829 2014-08-13 15:28:41.426800 2014-08-13 17:38:02.895638 7fe9bae887a0 20 read_log 194416'12472 (0'0) promote 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by osd.332.0:4419 2014-08-13 15:28:42.410760 2014-08-13 17:38:02.895650 7fe9bae887a0 20 read_log 194416'12473 (194416'12472) modify 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by client.2441958.0:3088 2014-08-13 15:28:42.401137 2014-08-13 17:38:02.895662 7fe9bae887a0 20 read_log 194416'12474 (0'0) promote 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by osd.332.0:4423 2014-08-13 15:28:42.509249 2014-08-13 17:38:02.895674 7fe9bae887a0 20 read_log 194416'12475 (194416'12474) modify 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by client.2441958.0:3115 2014-08-13 15:28:42.499330 2014-08-13 17:38:02.895686 7fe9bae887a0 20 read_log 194416'12476 (0'0) promote 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by osd.332.0:4515 2014-08-13 15:28:58.599923 2014-08-13 17:38:02.895697 7fe9bae887a0 20 read_log 194416'12477 (194416'12476) modify 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by client.2441958.0:7219 2014-08-13 15:28:58.589366 2014-08-13 17:38:02.895710 7fe9bae887a0 20 read_log 194416'12478 (0'0) promote d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by osd.332.0:4545 2014-08-13 15:29:03.283742 2014-08-13 17:38:02.895721 7fe9bae887a0 20 read_log 194416'12479 (194416'12478) modify d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by client.2441958.0:8464 2014-08-13 15:29:03.275090 2014-08-13 17:38:02.895740 7fe9bae887a0 20 read_log 194416'12480 (0'0) promote 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by osd.332.0:4561 2014-08-13 15:29:04.641188 2014-08-13 17:38:02.895753 7fe9bae887a0 20 read_log 194416'12481 (194416'12480) modify 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by client.2441958.0:8807 2014-08-13 15:29:04.626173 2014-08-13 17:38:02.895765 7fe9bae887a0 20 read_log 194416'12482 (0'0) promote 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by osd.332.0:4597 2014-08-13 15:29:14.568670 2014-08-13 17:38:02.895777 7fe9bae887a0 20 read_log 194416'12483 (194416'12482) modify 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by client.2441958.0:11269 2014-08-13 15:29:14.556475 2014-08-13 17:38:02.895789 7fe9bae887a0 20 read_log 194416'12484 (0'0) promote f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by osd.332.0:4625 2014-08-13 15:29:19.200191 2014-08-13 17:38:02.895801 7fe9bae887a0 20 read_log 194416'12485 (194416'12484) modify f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by client.2441958.0:12370 2014-08-13 15:29:19.191812 2014-08-13 17:38:02.895813 7fe9bae887a0 20 read_log 194416'12486 (0'0) promote 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by osd.332.0:4631 2014-08-13 15:29:21.418124 2014-08-13 17:38:02.895825 7fe9bae887a0 20 read_log 194416'12487 (194416'12486) modify 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by client.2441958.0:12895 2014-08-13 15:29:21.410797 2014-08-13 17:38:02.895838 7fe9bae887a0 20 read_log 194416'12488 (0'0) promote bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by osd.332.0:4643 2014-08-13 15:29:23.128925 2014-08-13 17:38:02.895849 7fe9bae887a0 20 read_log 194416'12489 (194416'12488) modify bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by client.2441958.0:13269 2014-08-13 15:29:23.120531 2014-08-13 17:38:02.895862 7fe9bae887a0 20 read_log 194416'12490 (0'0) promote 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by osd.332.0:4673 2014-08-13 15:29:27.741021 2014-08-13 17:38:02.895874 7fe9bae887a0 20 read_log 194416'12491 (194416'12490) modify 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by client.2441958.0:14397 2014-08-13 15:29:27.730476 2014-08-13 17:38:02.895886 7fe9bae887a0 20 read_log 194416'12492 (0'0) promote 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by osd.332.0:4679 2014-08-13 15:29:29.604493 2014-08-13 17:38:02.895898 7fe9bae887a0 20 read_log 194416'12493 (194416'12492) modify 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by client.2441958.0:14844 2014-08-13 15:29:29.596571 2014-08-13 17:38:02.895910 7fe9bae887a0 20 read_log 194416'12494 (0'0) promote 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by osd.332.0:4691 2014-08-13 15:29:32.421036 2014-08-13 17:38:02.895922 7fe9bae887a0 20 read_log 194416'12495 (194416'12494) modify 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by client.2441958.0:15445 2014-08-13 15:29:32.411875 2014-08-13 17:38:02.895934 7fe9bae887a0 20 read_log 194416'12496 (0'0) promote cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by osd.332.0:4699 2014-08-13 15:29:33.263855 2014-08-13 17:38:02.895946 7fe9bae887a0 20 read_log 194416'12497 (194416'12496) modify cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by client.2441958.0:15605 2014-08-13 15:29:33.255247 2014-08-13 17:38:02.895958 7fe9bae887a0 20 read_log 194416'12498 (0'0) promote c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by osd.332.0:4749 2014-08-13 15:30:00.202521 2014-08-13 17:38:02.895970 7fe9bae887a0 20 read_log 194416'12499 (194416'12498) modify c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by client.2441958.0:17910 2014-08-13 15:30:00.193255 2014-08-13 17:38:02.895983 7fe9bae887a0 20 read_log 194422'12500 (0'0) promote 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by osd.332.0:4791 2014-08-13 15:32:19.666593 2014-08-13 17:38:02.895995 7fe9bae887a0 20 read_log 194422'12501 (194422'12500) modify 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by client.1148203.0:1098 2014-08-13 15:32:19.658149 2014-08-13 17:38:02.896007 7fe9bae887a0 20 read_log 194422'12502 (0'0) promote adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by osd.332.0:4799 2014-08-13 15:32:21.421131 2014-08-13 17:38:02.896019 7fe9bae887a0 20 read_log 194422'12503 (194422'12502) modify adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by client.1148203.0:1548 2014-08-13 15:32:21.413428 2014-08-13 17:38:02.896032 7fe9bae887a0 20 read_log 194422'12504 (0'0) promote 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by osd.332.0:4855 2014-08-13 15:32:33.463926 2014-08-13 17:38:02.896044 7fe9bae887a0 20 read_log 194422'12505 (194422'12504) modify 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by client.1148203.0:4640 2014-08-13 15:32:33.455244 2014-08-13 17:38:02.896056 7fe9bae887a0 20 read_log 194422'12506 (0'0) promote edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by osd.332.0:4871 2014-08-13 15:32:37.126742 2014-08-13 17:38:02.896068 7fe9bae887a0 20 read_log 194422'12507 (194422'12506) modify edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by client.1148203.0:5457 2014-08-13 15:32:37.118627 2014-08-13 17:38:02.896080 7fe9bae887a0 20 read_log 194422'12508 (0'0) promote 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by osd.332.0:4889 2014-08-13 15:32:41.239656 2014-08-13 17:38:02.896092 7fe9bae887a0 20 read_log 194422'12509 (194422'12508) modify 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by client.1148203.0:6204 2014-08-13 15:32:41.230821 2014-08-13 17:38:02.896104 7fe9bae887a0 20 read_log 194422'12510 (0'0) promote 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by osd.332.0:4893 2014-08-13 15:32:42.035695 2014-08-13 17:38:02.896116 7fe9bae887a0 20 read_log 194422'12511 (194422'12510) modify 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by client.1148203.0:6344 2014-08-13 15:32:42.025183 2014-08-13 17:38:02.896128 7fe9bae887a0 20 read_log 194422'12512 (0'0) promote f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by osd.332.0:4905 2014-08-13 15:32:46.041594 2014-08-13 17:38:02.896140 7fe9bae887a0 20 read_log 194422'12513 (194422'12512) modify f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by client.1148203.0:7082 2014-08-13 15:32:46.034022 2014-08-13 17:38:02.896152 7fe9bae887a0 20 read_log 194422'12514 (0'0) promote 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by osd.332.0:4961 2014-08-13 15:33:09.893219 2014-08-13 17:38:02.896169 7fe9bae887a0 20 read_log 194422'12515 (194422'12514) modify 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by client.1148203.0:9945 2014-08-13 15:33:09.884120 2014-08-13 17:38:02.896182 7fe9bae887a0 20 read_log 194422'12516 (0'0) promote a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by osd.332.0:4965 2014-08-13 15:33:22.152590 2014-08-13 17:38:02.896194 7fe9bae887a0 20 read_log 194422'12517 (194422'12516) modify a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by client.1148203.0:10435 2014-08-13 15:33:22.144771 2014-08-13 17:38:02.896209 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.896678 7fe9bae887a0 10 read_log checking for missing items over interval (194370'12420,194422'12517] 2014-08-13 17:38:02.896731 7fe9bae887a0 15 read_log missing 194422'12517 (194422'12516) modify a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by client.1148203.0:10435 2014-08-13 15:33:22.144771 2014-08-13 17:38:02.896780 7fe9bae887a0 15 read_log missing 194422'12515 (194422'12514) modify 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by client.1148203.0:9945 2014-08-13 15:33:09.884120 2014-08-13 17:38:02.896819 7fe9bae887a0 15 read_log missing 194422'12513 (194422'12512) modify f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by client.1148203.0:7082 2014-08-13 15:32:46.034022 2014-08-13 17:38:02.896856 7fe9bae887a0 15 read_log missing 194422'12511 (194422'12510) modify 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by client.1148203.0:6344 2014-08-13 15:32:42.025183 2014-08-13 17:38:02.896892 7fe9bae887a0 15 read_log missing 194422'12509 (194422'12508) modify 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by client.1148203.0:6204 2014-08-13 15:32:41.230821 2014-08-13 17:38:02.896928 7fe9bae887a0 15 read_log missing 194422'12507 (194422'12506) modify edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by client.1148203.0:5457 2014-08-13 15:32:37.118627 2014-08-13 17:38:02.896964 7fe9bae887a0 15 read_log missing 194422'12505 (194422'12504) modify 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by client.1148203.0:4640 2014-08-13 15:32:33.455244 2014-08-13 17:38:02.897000 7fe9bae887a0 15 read_log missing 194422'12503 (194422'12502) modify adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by client.1148203.0:1548 2014-08-13 15:32:21.413428 2014-08-13 17:38:02.897037 7fe9bae887a0 15 read_log missing 194422'12501 (194422'12500) modify 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by client.1148203.0:1098 2014-08-13 15:32:19.658149 2014-08-13 17:38:02.897073 7fe9bae887a0 15 read_log missing 194416'12499 (194416'12498) modify c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by client.2441958.0:17910 2014-08-13 15:30:00.193255 2014-08-13 17:38:02.897110 7fe9bae887a0 15 read_log missing 194416'12497 (194416'12496) modify cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by client.2441958.0:15605 2014-08-13 15:29:33.255247 2014-08-13 17:38:02.897146 7fe9bae887a0 15 read_log missing 194416'12495 (194416'12494) modify 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by client.2441958.0:15445 2014-08-13 15:29:32.411875 2014-08-13 17:38:02.897191 7fe9bae887a0 15 read_log missing 194416'12493 (194416'12492) modify 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by client.2441958.0:14844 2014-08-13 15:29:29.596571 2014-08-13 17:38:02.897228 7fe9bae887a0 15 read_log missing 194416'12491 (194416'12490) modify 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by client.2441958.0:14397 2014-08-13 15:29:27.730476 2014-08-13 17:38:02.897264 7fe9bae887a0 15 read_log missing 194416'12489 (194416'12488) modify bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by client.2441958.0:13269 2014-08-13 15:29:23.120531 2014-08-13 17:38:02.897301 7fe9bae887a0 15 read_log missing 194416'12487 (194416'12486) modify 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by client.2441958.0:12895 2014-08-13 15:29:21.410797 2014-08-13 17:38:02.897338 7fe9bae887a0 15 read_log missing 194416'12485 (194416'12484) modify f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by client.2441958.0:12370 2014-08-13 15:29:19.191812 2014-08-13 17:38:02.897374 7fe9bae887a0 15 read_log missing 194416'12483 (194416'12482) modify 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by client.2441958.0:11269 2014-08-13 15:29:14.556475 2014-08-13 17:38:02.897411 7fe9bae887a0 15 read_log missing 194416'12481 (194416'12480) modify 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by client.2441958.0:8807 2014-08-13 15:29:04.626173 2014-08-13 17:38:02.897447 7fe9bae887a0 15 read_log missing 194416'12479 (194416'12478) modify d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by client.2441958.0:8464 2014-08-13 15:29:03.275090 2014-08-13 17:38:02.897483 7fe9bae887a0 15 read_log missing 194416'12477 (194416'12476) modify 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by client.2441958.0:7219 2014-08-13 15:28:58.589366 2014-08-13 17:38:02.897520 7fe9bae887a0 15 read_log missing 194416'12475 (194416'12474) modify 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by client.2441958.0:3115 2014-08-13 15:28:42.499330 2014-08-13 17:38:02.897557 7fe9bae887a0 15 read_log missing 194416'12473 (194416'12472) modify 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by client.2441958.0:3088 2014-08-13 15:28:42.401137 2014-08-13 17:38:02.897594 7fe9bae887a0 15 read_log missing 194416'12471 (194416'12470) modify 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by client.2441958.0:2829 2014-08-13 15:28:41.426800 2014-08-13 17:38:02.897630 7fe9bae887a0 15 read_log missing 194416'12469 (194416'12468) modify 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by client.2441958.0:1627 2014-08-13 15:28:36.876856 2014-08-13 17:38:02.897671 7fe9bae887a0 15 read_log missing 194416'12466 (194416'12466) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 17:38:02.897707 7fe9bae887a0 15 read_log missing 194397'12465 (194397'12464) modify b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by client.2235069.0:565 2014-08-12 20:53:01.904209 2014-08-13 17:38:02.897748 7fe9bae887a0 15 read_log missing 194377'12461 (194377'12460) modify ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by client.2231190.0:3520 2014-08-12 20:33:11.447247 2014-08-13 17:38:02.897785 7fe9bae887a0 15 read_log missing 194377'12459 (194377'12458) modify c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by client.2231190.0:3058 2014-08-12 20:33:09.547018 2014-08-13 17:38:02.897821 7fe9bae887a0 15 read_log missing 194377'12457 (194377'12456) modify b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by client.2231190.0:2386 2014-08-12 20:33:06.773113 2014-08-13 17:38:02.897858 7fe9bae887a0 15 read_log missing 194377'12455 (194377'12454) modify ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by client.2231190.0:1331 2014-08-12 20:33:02.557160 2014-08-13 17:38:02.897894 7fe9bae887a0 15 read_log missing 194377'12453 (194377'12452) modify efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by client.2231190.0:751 2014-08-12 20:33:00.367237 2014-08-13 17:38:02.897931 7fe9bae887a0 15 read_log missing 194370'12451 (194370'12450) modify 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by client.1508531.0:13107 2014-08-12 20:31:20.413293 2014-08-13 17:38:02.897966 7fe9bae887a0 15 read_log missing 194370'12449 (194370'12448) modify 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by client.1508531.0:12467 2014-08-12 20:31:14.755486 2014-08-13 17:38:02.898002 7fe9bae887a0 15 read_log missing 194370'12447 (194370'12446) modify 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by client.1508531.0:12053 2014-08-12 20:31:11.739138 2014-08-13 17:38:02.898039 7fe9bae887a0 15 read_log missing 194370'12445 (194370'12444) modify eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by client.1508531.0:10604 2014-08-12 20:31:02.874965 2014-08-13 17:38:02.898076 7fe9bae887a0 15 read_log missing 194370'12443 (194370'12442) modify 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by client.1508531.0:10461 2014-08-12 20:31:01.983217 2014-08-13 17:38:02.898112 7fe9bae887a0 15 read_log missing 194370'12441 (194370'12440) modify 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by client.1508531.0:8814 2014-08-12 20:30:52.131128 2014-08-13 17:38:02.898149 7fe9bae887a0 15 read_log missing 194370'12439 (194370'12438) modify f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by client.1508531.0:8308 2014-08-12 20:30:49.206145 2014-08-13 17:38:02.898191 7fe9bae887a0 15 read_log missing 194370'12437 (194370'12436) modify 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by client.1508531.0:7455 2014-08-12 20:30:44.631213 2014-08-13 17:38:02.898228 7fe9bae887a0 15 read_log missing 194370'12435 (194370'12434) modify 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by client.1508531.0:7087 2014-08-12 20:30:42.838696 2014-08-13 17:38:02.898265 7fe9bae887a0 15 read_log missing 194370'12433 (194370'12432) modify 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by client.1508531.0:6282 2014-08-12 20:30:38.971071 2014-08-13 17:38:02.898302 7fe9bae887a0 15 read_log missing 194370'12431 (194370'12430) modify aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by client.1508531.0:3585 2014-08-12 20:30:27.716306 2014-08-13 17:38:02.898339 7fe9bae887a0 15 read_log missing 194370'12429 (194370'12428) modify 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by client.1508531.0:2705 2014-08-12 20:30:24.467345 2014-08-13 17:38:02.898375 7fe9bae887a0 15 read_log missing 194370'12427 (194370'12426) modify 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by client.1508531.0:1182 2014-08-12 20:30:18.808413 2014-08-13 17:38:02.898414 7fe9bae887a0 15 read_log missing 194370'12425 (194370'12424) modify 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by client.1508531.0:1027 2014-08-12 20:30:18.215574 2014-08-13 17:38:02.898451 7fe9bae887a0 15 read_log missing 194370'12423 (194370'12422) modify fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by client.1508531.0:893 2014-08-12 20:30:17.732009 2014-08-13 17:38:02.898487 7fe9bae887a0 15 read_log missing 194370'12421 (194370'12420) modify 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by client.1508531.0:846 2014-08-12 20:30:17.563335 2014-08-13 17:38:02.898498 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.898529 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_loaded 2014-08-13 17:38:02.898549 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Initial 0.015799 0 0.000000 2014-08-13 17:38:02.898561 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset 2014-08-13 17:38:02.898571 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194478 2014-08-13 17:38:02.898580 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 17:38:02.898591 7fe9bae887a0 10 osd.4 194478 pgid 24.1de coll 24.1de_head 2014-08-13 17:38:02.898719 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.1de 2014-08-13 17:38:02.898732 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.898765 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1de(unlocked)] enter Initial 2014-08-13 17:38:02.898776 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.1de(unlocked)] enter NotTrimming 2014-08-13 17:38:02.898925 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.899043 7fe9bae887a0 20 read_log 192834'15291 (0'0) promote b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by osd.289.0:52105 2014-08-11 15:23:40.933638 2014-08-13 17:38:02.899064 7fe9bae887a0 20 read_log 192834'15292 (192834'15291) modify b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.933448.0:1358 2014-08-11 15:23:40.925381 2014-08-13 17:38:02.899078 7fe9bae887a0 20 read_log 192834'15299 (0'0) promote 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.289.0:52129 2014-08-11 15:23:43.401617 2014-08-13 17:38:02.899090 7fe9bae887a0 20 read_log 192834'15300 (192834'15299) modify 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by client.933448.0:1853 2014-08-11 15:23:43.393271 2014-08-13 17:38:02.899103 7fe9bae887a0 20 read_log 192834'15317 (0'0) promote 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by osd.289.0:52181 2014-08-11 15:23:49.371321 2014-08-13 17:38:02.899115 7fe9bae887a0 20 read_log 192834'15318 (192834'15317) modify 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.933448.0:2925 2014-08-11 15:23:49.356984 2014-08-13 17:38:02.899127 7fe9bae887a0 20 read_log 192834'15333 (0'0) promote 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by osd.289.0:52225 2014-08-11 15:23:53.304362 2014-08-13 17:38:02.899139 7fe9bae887a0 20 read_log 192834'15334 (192834'15333) modify 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.933448.0:3715 2014-08-11 15:23:53.272115 2014-08-13 17:38:02.899151 7fe9bae887a0 20 read_log 192834'15339 (0'0) promote 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by osd.289.0:52257 2014-08-11 15:23:56.052408 2014-08-13 17:38:02.899171 7fe9bae887a0 20 read_log 192834'15340 (192834'15339) modify 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.933448.0:4249 2014-08-11 15:23:56.045664 2014-08-13 17:38:02.899184 7fe9bae887a0 20 read_log 192834'15365 (0'0) promote 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.289.0:52319 2014-08-11 15:24:01.102588 2014-08-13 17:38:02.899196 7fe9bae887a0 20 read_log 192834'15366 (192834'15365) modify 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by client.933448.0:5353 2014-08-11 15:24:01.091198 2014-08-13 17:38:02.899208 7fe9bae887a0 20 read_log 192834'15377 (0'0) promote 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by osd.289.0:52359 2014-08-11 15:24:05.279182 2014-08-13 17:38:02.899220 7fe9bae887a0 20 read_log 192834'15378 (192834'15377) modify 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.933448.0:6401 2014-08-11 15:24:05.271713 2014-08-13 17:38:02.899233 7fe9bae887a0 20 read_log 192834'15381 (0'0) promote e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.289.0:52371 2014-08-11 15:24:06.061720 2014-08-13 17:38:02.899245 7fe9bae887a0 20 read_log 192834'15382 (192834'15381) modify e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by client.933448.0:6605 2014-08-11 15:24:06.053539 2014-08-13 17:38:02.899257 7fe9bae887a0 20 read_log 192834'15415 (0'0) promote d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by osd.289.0:52467 2014-08-11 15:24:15.658697 2014-08-13 17:38:02.899269 7fe9bae887a0 20 read_log 192834'15416 (192834'15415) modify d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.933448.0:8851 2014-08-11 15:24:15.650934 2014-08-13 17:38:02.899307 7fe9bae887a0 20 read_log 192834'15447 (0'0) promote 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by osd.289.0:52603 2014-08-11 15:24:29.705542 2014-08-13 17:38:02.899323 7fe9bae887a0 20 read_log 192834'15448 (192834'15447) modify 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.933448.0:11868 2014-08-11 15:24:29.698018 2014-08-13 17:38:02.899336 7fe9bae887a0 20 read_log 192834'15455 (0'0) promote 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.289.0:52665 2014-08-11 15:24:35.385250 2014-08-13 17:38:02.899348 7fe9bae887a0 20 read_log 192834'15456 (192834'15455) modify 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by client.933448.0:13104 2014-08-11 15:24:35.375933 2014-08-13 17:38:02.899360 7fe9bae887a0 20 read_log 192834'15465 (0'0) promote 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.289.0:52699 2014-08-11 15:24:39.017046 2014-08-13 17:38:02.899372 7fe9bae887a0 20 read_log 192834'15466 (192834'15465) modify 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by client.933448.0:13532 2014-08-11 15:24:39.007817 2014-08-13 17:38:02.899385 7fe9bae887a0 20 read_log 192834'15473 (0'0) promote 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.289.0:52727 2014-08-11 15:24:41.179691 2014-08-13 17:38:02.899397 7fe9bae887a0 20 read_log 192834'15474 (192834'15473) modify 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by client.933448.0:14088 2014-08-11 15:24:41.170226 2014-08-13 17:38:02.899409 7fe9bae887a0 20 read_log 192834'15481 (0'0) promote 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.289.0:52802 2014-08-11 15:24:46.475202 2014-08-13 17:38:02.899421 7fe9bae887a0 20 read_log 192834'15482 (192834'15481) modify 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by client.933448.0:14533 2014-08-11 15:24:46.052447 2014-08-13 17:38:02.899433 7fe9bae887a0 20 read_log 192834'15483 (0'0) promote ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by osd.289.0:52844 2014-08-11 15:24:48.403098 2014-08-13 17:38:02.899445 7fe9bae887a0 20 read_log 192834'15484 (192834'15483) modify ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.933448.0:15097 2014-08-11 15:24:48.394353 2014-08-13 17:38:02.899457 7fe9bae887a0 20 read_log 192834'15485 (0'0) promote de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by osd.289.0:52846 2014-08-11 15:24:48.647186 2014-08-13 17:38:02.899469 7fe9bae887a0 20 read_log 192834'15486 (192834'15485) modify de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.933448.0:15165 2014-08-11 15:24:48.638711 2014-08-13 17:38:02.899481 7fe9bae887a0 20 read_log 192834'15487 (0'0) promote 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by osd.289.0:52852 2014-08-11 15:24:48.916548 2014-08-13 17:38:02.899493 7fe9bae887a0 20 read_log 192834'15488 (192834'15487) modify 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.933448.0:15239 2014-08-11 15:24:48.908341 2014-08-13 17:38:02.899506 7fe9bae887a0 20 read_log 192834'15491 (0'0) promote 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by osd.289.0:52858 2014-08-11 15:24:49.131206 2014-08-13 17:38:02.899518 7fe9bae887a0 20 read_log 192834'15492 (192834'15491) modify 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.933448.0:15297 2014-08-11 15:24:49.122458 2014-08-13 17:38:02.899531 7fe9bae887a0 20 read_log 192834'15497 (0'0) promote cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by osd.289.0:52890 2014-08-11 15:24:50.970062 2014-08-13 17:38:02.899542 7fe9bae887a0 20 read_log 192834'15498 (192834'15497) modify cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.933448.0:15659 2014-08-11 15:24:50.549880 2014-08-13 17:38:02.899562 7fe9bae887a0 20 read_log 192834'15499 (0'0) promote 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by osd.289.0:52908 2014-08-11 15:24:52.853969 2014-08-13 17:38:02.899575 7fe9bae887a0 20 read_log 192834'15500 (192834'15499) modify 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.933448.0:16029 2014-08-11 15:24:52.342486 2014-08-13 17:38:02.899597 7fe9bae887a0 20 read_log 192834'15527 (0'0) promote 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.289.0:53002 2014-08-11 15:25:00.611616 2014-08-13 17:38:02.899610 7fe9bae887a0 20 read_log 192834'15528 (192834'15527) modify 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by client.933448.0:17525 2014-08-11 15:25:00.045602 2014-08-13 17:38:02.899622 7fe9bae887a0 20 read_log 192834'15547 (0'0) promote ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.289.0:53186 2014-08-11 15:25:14.020232 2014-08-13 17:38:02.899634 7fe9bae887a0 20 read_log 192834'15548 (192834'15547) modify ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by client.933448.0:19926 2014-08-11 15:25:14.011015 2014-08-13 17:38:02.899646 7fe9bae887a0 20 read_log 192834'15551 (0'0) promote 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by osd.289.0:53262 2014-08-11 15:25:20.897336 2014-08-13 17:38:02.899658 7fe9bae887a0 20 read_log 192834'15552 (192834'15551) modify 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.933448.0:20431 2014-08-11 15:25:20.888994 2014-08-13 17:38:02.899670 7fe9bae887a0 20 read_log 192834'15582 (192834'15290) modify 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by client.933448.0:23601 2014-08-11 15:25:39.541328 2014-08-13 17:38:02.899682 7fe9bae887a0 20 read_log 192834'15583 (192834'15292) modify b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.933448.0:23649 2014-08-11 15:25:39.621900 2014-08-13 17:38:02.899694 7fe9bae887a0 20 read_log 192834'15587 (192834'15300) modify 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by client.933448.0:24144 2014-08-11 15:25:40.326923 2014-08-13 17:38:02.899706 7fe9bae887a0 20 read_log 192834'15596 (192834'15318) modify 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.933448.0:25216 2014-08-11 15:25:41.892075 2014-08-13 17:38:02.899717 7fe9bae887a0 20 read_log 192834'15604 (192834'15334) modify 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.933448.0:26006 2014-08-11 15:25:42.839602 2014-08-13 17:38:02.899729 7fe9bae887a0 20 read_log 192834'15607 (192834'15340) modify 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.933448.0:26540 2014-08-11 15:25:43.519558 2014-08-13 17:38:02.899741 7fe9bae887a0 20 read_log 192834'15619 (192834'15366) modify 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by client.933448.0:27644 2014-08-11 15:25:44.744662 2014-08-13 17:38:02.899753 7fe9bae887a0 20 read_log 192834'15625 (192834'15378) modify 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.933448.0:28692 2014-08-11 15:25:46.599687 2014-08-13 17:38:02.899766 7fe9bae887a0 20 read_log 192834'15627 (192834'15382) modify e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by client.933448.0:28896 2014-08-11 15:25:46.811940 2014-08-13 17:38:02.899778 7fe9bae887a0 20 read_log 192834'15644 (192834'15416) modify d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.933448.0:31142 2014-08-11 15:25:49.691424 2014-08-13 17:38:02.899790 7fe9bae887a0 20 read_log 192834'15660 (192834'15448) modify 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.933448.0:34159 2014-08-11 15:25:54.655997 2014-08-13 17:38:02.899802 7fe9bae887a0 20 read_log 192834'15664 (192834'15456) modify 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by client.933448.0:35395 2014-08-11 15:25:56.198227 2014-08-13 17:38:02.899824 7fe9bae887a0 20 read_log 192834'15669 (192834'15466) modify 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by client.933448.0:35823 2014-08-11 15:25:56.818352 2014-08-13 17:38:02.899838 7fe9bae887a0 20 read_log 192834'15673 (192834'15474) modify 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by client.933448.0:36379 2014-08-11 15:25:57.632340 2014-08-13 17:38:02.899852 7fe9bae887a0 20 read_log 192834'15677 (192834'15482) modify 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by client.933448.0:36824 2014-08-11 15:25:58.344725 2014-08-13 17:38:02.899864 7fe9bae887a0 20 read_log 192834'15678 (192834'15484) modify ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.933448.0:37388 2014-08-11 15:25:59.496820 2014-08-13 17:38:02.899879 7fe9bae887a0 20 read_log 192834'15679 (192834'15486) modify de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.933448.0:37456 2014-08-11 15:25:59.580782 2014-08-13 17:38:02.899891 7fe9bae887a0 20 read_log 192834'15680 (192834'15488) modify 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.933448.0:37530 2014-08-11 15:25:59.662431 2014-08-13 17:38:02.899905 7fe9bae887a0 20 read_log 192834'15682 (192834'15492) modify 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.933448.0:37588 2014-08-11 15:25:59.725507 2014-08-13 17:38:02.899920 7fe9bae887a0 20 read_log 192834'15685 (192834'15498) modify cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.933448.0:37950 2014-08-11 15:26:00.302554 2014-08-13 17:38:02.899932 7fe9bae887a0 20 read_log 192834'15686 (192834'15500) modify 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.933448.0:38320 2014-08-11 15:26:00.753748 2014-08-13 17:38:02.899946 7fe9bae887a0 20 read_log 192834'15700 (192834'15528) modify 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by client.933448.0:39816 2014-08-11 15:26:02.102046 2014-08-13 17:38:02.899958 7fe9bae887a0 20 read_log 192834'15710 (192834'15548) modify ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by client.933448.0:42217 2014-08-11 15:26:03.191883 2014-08-13 17:38:02.899970 7fe9bae887a0 20 read_log 192834'15712 (192834'15552) modify 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.933448.0:42722 2014-08-11 15:26:03.367333 2014-08-13 17:38:02.899982 7fe9bae887a0 20 read_log 192834'15733 (0'0) promote 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by osd.289.0:54405 2014-08-11 15:26:18.707854 2014-08-13 17:38:02.899994 7fe9bae887a0 20 read_log 192834'15734 (192834'15733) modify 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.1883634.0:1161 2014-08-11 15:26:18.699367 2014-08-13 17:38:02.900006 7fe9bae887a0 20 read_log 192834'15739 (0'0) promote 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by osd.289.0:54419 2014-08-11 15:26:20.650244 2014-08-13 17:38:02.900021 7fe9bae887a0 20 read_log 192834'15740 (192834'15739) modify 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.1883634.0:1697 2014-08-11 15:26:20.641990 2014-08-13 17:38:02.900033 7fe9bae887a0 20 read_log 192834'15743 (0'0) promote c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by osd.289.0:54425 2014-08-11 15:26:21.311131 2014-08-13 17:38:02.900048 7fe9bae887a0 20 read_log 192834'15744 (192834'15743) modify c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.1883634.0:1851 2014-08-11 15:26:21.302753 2014-08-13 17:38:02.900060 7fe9bae887a0 20 read_log 192834'15775 (0'0) promote 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by osd.289.0:54529 2014-08-11 15:26:29.458366 2014-08-13 17:38:02.900074 7fe9bae887a0 20 read_log 192834'15776 (192834'15775) modify 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.1883634.0:4130 2014-08-11 15:26:29.447995 2014-08-13 17:38:02.900093 7fe9bae887a0 20 read_log 192834'15777 (0'0) promote 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by osd.289.0:54547 2014-08-11 15:26:30.344448 2014-08-13 17:38:02.900107 7fe9bae887a0 20 read_log 192834'15778 (192834'15777) modify 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.1883634.0:4380 2014-08-11 15:26:30.335888 2014-08-13 17:38:02.900120 7fe9bae887a0 20 read_log 192834'15789 (0'0) promote 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.289.0:54591 2014-08-11 15:26:33.267772 2014-08-13 17:38:02.900132 7fe9bae887a0 20 read_log 192834'15790 (192834'15789) modify 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by client.1883634.0:5114 2014-08-11 15:26:33.257770 2014-08-13 17:38:02.900146 7fe9bae887a0 20 read_log 192834'15797 (0'0) promote bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by osd.289.0:54601 2014-08-11 15:26:34.101653 2014-08-13 17:38:02.900164 7fe9bae887a0 20 read_log 192834'15798 (192834'15797) modify bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.1883634.0:5352 2014-08-11 15:26:34.093094 2014-08-13 17:38:02.900179 7fe9bae887a0 20 read_log 192834'15803 (0'0) promote 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by osd.289.0:54621 2014-08-11 15:26:36.635543 2014-08-13 17:38:02.900192 7fe9bae887a0 20 read_log 192834'15804 (192834'15803) modify 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.1883634.0:6063 2014-08-11 15:26:36.626941 2014-08-13 17:38:02.900211 7fe9bae887a0 20 read_log 192834'15805 (0'0) promote 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by osd.289.0:54623 2014-08-11 15:26:36.844058 2014-08-13 17:38:02.900226 7fe9bae887a0 20 read_log 192834'15806 (192834'15805) modify 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.1883634.0:6121 2014-08-11 15:26:36.836093 2014-08-13 17:38:02.900239 7fe9bae887a0 20 read_log 192834'15807 (0'0) promote e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by osd.289.0:54627 2014-08-11 15:26:37.227569 2014-08-13 17:38:02.900272 7fe9bae887a0 20 read_log 192834'15808 (192834'15807) modify e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.1883634.0:6228 2014-08-11 15:26:37.218600 2014-08-13 17:38:02.900287 7fe9bae887a0 20 read_log 192834'15811 (0'0) promote 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by osd.289.0:54635 2014-08-11 15:26:38.764384 2014-08-13 17:38:02.900299 7fe9bae887a0 20 read_log 192834'15812 (192834'15811) modify 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.1883634.0:6573 2014-08-11 15:26:38.754359 2014-08-13 17:38:02.900311 7fe9bae887a0 20 read_log 192834'15813 (0'0) promote e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by osd.289.0:54674 2014-08-11 15:26:39.503976 2014-08-13 17:38:02.900325 7fe9bae887a0 20 read_log 192834'15814 (192834'15813) modify e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.1883634.0:6718 2014-08-11 15:26:39.259959 2014-08-13 17:38:02.900338 7fe9bae887a0 20 read_log 192834'15837 (0'0) promote b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by osd.289.0:54816 2014-08-11 15:26:44.183129 2014-08-13 17:38:02.900352 7fe9bae887a0 20 read_log 192834'15838 (192834'15837) modify b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.1883634.0:7878 2014-08-11 15:26:44.174527 2014-08-13 17:38:02.900364 7fe9bae887a0 20 read_log 192834'15849 (0'0) promote e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by osd.289.0:54910 2014-08-11 15:26:47.921115 2014-08-13 17:38:02.900378 7fe9bae887a0 20 read_log 192834'15850 (192834'15849) modify e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.1883634.0:8797 2014-08-11 15:26:47.912751 2014-08-13 17:38:02.900398 7fe9bae887a0 20 read_log 192834'15861 (0'0) promote 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.289.0:55074 2014-08-11 15:26:51.727226 2014-08-13 17:38:02.900413 7fe9bae887a0 20 read_log 192834'15862 (192834'15861) modify 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by client.1883634.0:9643 2014-08-11 15:26:51.718381 2014-08-13 17:38:02.900425 7fe9bae887a0 20 read_log 192834'15867 (0'0) promote 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.289.0:55095 2014-08-11 15:26:53.373731 2014-08-13 17:38:02.900440 7fe9bae887a0 20 read_log 192834'15868 (192834'15867) modify 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by client.1883634.0:10010 2014-08-11 15:26:53.365862 2014-08-13 17:38:02.900452 7fe9bae887a0 20 read_log 192834'15871 (0'0) promote 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by osd.289.0:55145 2014-08-11 15:26:55.152694 2014-08-13 17:38:02.900466 7fe9bae887a0 20 read_log 192834'15872 (192834'15871) modify 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.1883634.0:10465 2014-08-11 15:26:55.144383 2014-08-13 17:38:02.900478 7fe9bae887a0 20 read_log 192834'15891 (0'0) promote 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.289.0:55354 2014-08-11 15:27:01.385257 2014-08-13 17:38:02.900492 7fe9bae887a0 20 read_log 192834'15892 (192834'15891) modify 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by client.1883634.0:11808 2014-08-11 15:27:01.333206 2014-08-13 17:38:02.900504 7fe9bae887a0 20 read_log 192834'15897 (0'0) promote 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.289.0:55365 2014-08-11 15:27:01.655703 2014-08-13 17:38:02.900517 7fe9bae887a0 20 read_log 192834'15898 (192834'15897) modify 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by client.1883634.0:11903 2014-08-11 15:27:01.647448 2014-08-13 17:38:02.900529 7fe9bae887a0 20 read_log 192834'15901 (0'0) promote 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by osd.289.0:55401 2014-08-11 15:27:04.633621 2014-08-13 17:38:02.900541 7fe9bae887a0 20 read_log 192834'15902 (192834'15901) modify 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.1883634.0:12580 2014-08-11 15:27:04.625607 2014-08-13 17:38:02.900553 7fe9bae887a0 20 read_log 192834'15929 (0'0) promote b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.289.0:55609 2014-08-11 15:27:16.283381 2014-08-13 17:38:02.900565 7fe9bae887a0 20 read_log 192834'15930 (192834'15929) modify b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by client.1883634.0:14942 2014-08-11 15:27:16.273615 2014-08-13 17:38:02.900579 7fe9bae887a0 20 read_log 192834'15969 (0'0) promote ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by osd.289.0:55933 2014-08-11 15:27:36.815571 2014-08-13 17:38:02.900591 7fe9bae887a0 20 read_log 192834'15970 (192834'15969) modify ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.1883634.0:17531 2014-08-11 15:27:36.140755 2014-08-13 17:38:02.900605 7fe9bae887a0 20 read_log 192834'15981 (0'0) promote c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.289.0:56062 2014-08-11 15:27:42.484935 2014-08-13 17:38:02.900617 7fe9bae887a0 20 read_log 192834'15982 (192834'15981) modify c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by client.1883634.0:18212 2014-08-11 15:27:42.189347 2014-08-13 17:38:02.900631 7fe9bae887a0 20 read_log 192834'15983 (0'0) promote 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by osd.289.0:56066 2014-08-11 15:27:42.485947 2014-08-13 17:38:02.900643 7fe9bae887a0 20 read_log 192834'15986 (192834'15983) modify 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.1883634.0:18226 2014-08-11 15:27:42.292021 2014-08-13 17:38:02.900665 7fe9bae887a0 20 read_log 192834'16015 (0'0) promote 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.289.0:56320 2014-08-11 15:28:01.006273 2014-08-13 17:38:02.900680 7fe9bae887a0 20 read_log 192834'16016 (192834'16015) modify 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by client.1883634.0:20663 2014-08-11 15:28:00.998106 2014-08-13 17:38:02.900692 7fe9bae887a0 20 read_log 192834'16021 (0'0) promote bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by osd.289.0:56332 2014-08-11 15:28:02.220078 2014-08-13 17:38:02.900706 7fe9bae887a0 20 read_log 192834'16022 (192834'16021) modify bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.1883634.0:20985 2014-08-11 15:28:02.209818 2014-08-13 17:38:02.900718 7fe9bae887a0 20 read_log 192834'16023 (0'0) promote 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by osd.289.0:56351 2014-08-11 15:28:03.128530 2014-08-13 17:38:02.900733 7fe9bae887a0 20 read_log 192834'16024 (192834'16023) modify 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.1883634.0:21170 2014-08-11 15:28:03.119360 2014-08-13 17:38:02.900746 7fe9bae887a0 20 read_log 192834'16035 (0'0) promote f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by osd.289.0:56408 2014-08-11 15:28:05.110596 2014-08-13 17:38:02.900758 7fe9bae887a0 20 read_log 192834'16036 (192834'16035) modify f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.1883634.0:21731 2014-08-11 15:28:05.100524 2014-08-13 17:38:02.900772 7fe9bae887a0 20 read_log 192834'16053 (0'0) promote dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.289.0:56497 2014-08-11 15:28:10.995115 2014-08-13 17:38:02.900784 7fe9bae887a0 20 read_log 192834'16054 (192834'16053) modify dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by client.1883634.0:22771 2014-08-11 15:28:10.984468 2014-08-13 17:38:02.900796 7fe9bae887a0 20 read_log 192834'16057 (0'0) promote 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by osd.289.0:56501 2014-08-11 15:28:11.404695 2014-08-13 17:38:02.900808 7fe9bae887a0 20 read_log 192834'16058 (192834'16057) modify 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.1883634.0:22854 2014-08-11 15:28:11.395067 2014-08-13 17:38:02.900820 7fe9bae887a0 20 read_log 192834'16064 (192834'15734) modify 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.1883634.0:24485 2014-08-11 15:28:19.117282 2014-08-13 17:38:02.900832 7fe9bae887a0 20 read_log 192834'16067 (192834'15740) modify 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.1883634.0:25021 2014-08-11 15:28:19.948513 2014-08-13 17:38:02.900847 7fe9bae887a0 20 read_log 192834'16069 (192834'15744) modify c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.1883634.0:25175 2014-08-11 15:28:20.295779 2014-08-13 17:38:02.900859 7fe9bae887a0 20 read_log 192834'16085 (192834'15776) modify 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.1883634.0:27454 2014-08-11 15:28:24.481588 2014-08-13 17:38:02.900871 7fe9bae887a0 20 read_log 192834'16086 (192834'15778) modify 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.1883634.0:27704 2014-08-11 15:28:24.844068 2014-08-13 17:38:02.900888 7fe9bae887a0 20 read_log 192834'16092 (192834'15790) modify 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by client.1883634.0:28438 2014-08-11 15:28:25.810373 2014-08-13 17:38:02.900903 7fe9bae887a0 20 read_log 192834'16096 (192834'15798) modify bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.1883634.0:28676 2014-08-11 15:28:26.128929 2014-08-13 17:38:02.900915 7fe9bae887a0 20 read_log 192834'16099 (192834'15804) modify 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.1883634.0:29387 2014-08-11 15:28:27.080615 2014-08-13 17:38:02.900936 7fe9bae887a0 20 read_log 192834'16100 (192834'15806) modify 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.1883634.0:29445 2014-08-11 15:28:27.280511 2014-08-13 17:38:02.900951 7fe9bae887a0 20 read_log 192834'16101 (192834'15808) modify e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.1883634.0:29552 2014-08-11 15:28:27.419922 2014-08-13 17:38:02.900963 7fe9bae887a0 20 read_log 192834'16103 (192834'15812) modify 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.1883634.0:29897 2014-08-11 15:28:27.805577 2014-08-13 17:38:02.900975 7fe9bae887a0 20 read_log 192834'16104 (192834'15814) modify e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.1883634.0:30042 2014-08-11 15:28:27.997594 2014-08-13 17:38:02.900989 7fe9bae887a0 20 read_log 192834'16116 (192834'15838) modify b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.1883634.0:31202 2014-08-11 15:28:30.092629 2014-08-13 17:38:02.901002 7fe9bae887a0 20 read_log 192834'16122 (192834'15850) modify e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.1883634.0:32121 2014-08-11 15:28:31.370096 2014-08-13 17:38:02.901016 7fe9bae887a0 20 read_log 192834'16128 (192834'15862) modify 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by client.1883634.0:32967 2014-08-11 15:28:32.611718 2014-08-13 17:38:02.901028 7fe9bae887a0 20 read_log 192834'16131 (192834'15868) modify 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by client.1883634.0:33334 2014-08-11 15:28:33.034439 2014-08-13 17:38:02.901040 7fe9bae887a0 20 read_log 192834'16133 (192834'15872) modify 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.1883634.0:33789 2014-08-11 15:28:33.801448 2014-08-13 17:38:02.901052 7fe9bae887a0 20 read_log 192834'16143 (192834'15892) modify 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by client.1883634.0:35132 2014-08-11 15:28:36.060916 2014-08-13 17:38:02.901064 7fe9bae887a0 20 read_log 192834'16146 (192834'15898) modify 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by client.1883634.0:35227 2014-08-11 15:28:36.283478 2014-08-13 17:38:02.901079 7fe9bae887a0 20 read_log 192834'16150 (192834'15902) modify 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.1883634.0:35904 2014-08-11 15:28:37.295930 2014-08-13 17:38:02.901091 7fe9bae887a0 20 read_log 192834'16164 (192834'15930) modify b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by client.1883634.0:38266 2014-08-11 15:28:40.426091 2014-08-13 17:38:02.901105 7fe9bae887a0 20 read_log 192834'16184 (192834'15970) modify ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.1883634.0:40855 2014-08-11 15:28:44.336970 2014-08-13 17:38:02.901117 7fe9bae887a0 20 read_log 192834'16191 (192834'15982) modify c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by client.1883634.0:41536 2014-08-11 15:28:45.248191 2014-08-13 17:38:02.901131 7fe9bae887a0 20 read_log 192834'16192 (192834'15986) modify 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.1883634.0:41550 2014-08-11 15:28:45.266531 2014-08-13 17:38:02.901143 7fe9bae887a0 20 read_log 192834'16207 (192834'16016) modify 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by client.1883634.0:43987 2014-08-11 15:28:46.724669 2014-08-13 17:38:02.901163 7fe9bae887a0 20 read_log 192834'16210 (192834'16022) modify bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.1883634.0:44309 2014-08-11 15:28:46.824341 2014-08-13 17:38:02.901178 7fe9bae887a0 20 read_log 192834'16211 (192834'16024) modify 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.1883634.0:44494 2014-08-11 15:28:46.873704 2014-08-13 17:38:02.901190 7fe9bae887a0 20 read_log 192834'16217 (192834'16036) modify f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.1883634.0:45055 2014-08-11 15:28:47.135120 2014-08-13 17:38:02.901226 7fe9bae887a0 20 read_log 192834'16226 (192834'16054) modify dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by client.1883634.0:46095 2014-08-11 15:28:47.489765 2014-08-13 17:38:02.901241 7fe9bae887a0 20 read_log 192834'16228 (192834'16058) modify 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.1883634.0:46178 2014-08-11 15:28:47.513293 2014-08-13 17:38:02.901254 7fe9bae887a0 20 read_log 192888'16229 (192888'16229) modify 1de/hit_set_24.1de_archive_2014-08-11 16:02:27.862431_2014-08-11 16:44:55.175023/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.175327 2014-08-13 17:38:02.901269 7fe9bae887a0 20 read_log 192888'16234 (0'0) promote 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by osd.364.0:19362 2014-08-11 16:45:02.365991 2014-08-13 17:38:02.901281 7fe9bae887a0 20 read_log 192888'16235 (192888'16234) modify 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.1315259.0:1862 2014-08-11 16:45:02.357604 2014-08-13 17:38:02.901296 7fe9bae887a0 20 read_log 192888'16244 (0'0) promote 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by osd.364.0:19402 2014-08-11 16:45:10.217293 2014-08-13 17:38:02.901308 7fe9bae887a0 20 read_log 192888'16245 (192888'16244) modify 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.1315259.0:3934 2014-08-11 16:45:10.209837 2014-08-13 17:38:02.901320 7fe9bae887a0 20 read_log 192888'16248 (0'0) promote e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by osd.364.0:19416 2014-08-11 16:45:12.506789 2014-08-13 17:38:02.901332 7fe9bae887a0 20 read_log 192888'16249 (192888'16248) modify e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.1315259.0:4562 2014-08-11 16:45:12.499663 2014-08-13 17:38:02.901344 7fe9bae887a0 20 read_log 192888'16250 (0'0) promote 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:19430 2014-08-11 16:45:14.484377 2014-08-13 17:38:02.901356 7fe9bae887a0 20 read_log 192888'16251 (192888'16250) modify 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by client.1315259.0:5090 2014-08-11 16:45:14.476281 2014-08-13 17:38:02.901370 7fe9bae887a0 20 read_log 192888'16252 (0'0) promote 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by osd.364.0:19434 2014-08-11 16:45:15.153767 2014-08-13 17:38:02.901382 7fe9bae887a0 20 read_log 192888'16253 (192888'16252) modify 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.1315259.0:5274 2014-08-11 16:45:15.146353 2014-08-13 17:38:02.901395 7fe9bae887a0 20 read_log 192888'16254 (0'0) promote 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:19440 2014-08-11 16:45:15.643385 2014-08-13 17:38:02.901409 7fe9bae887a0 20 read_log 192888'16255 (192888'16254) modify 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by client.1315259.0:5409 2014-08-11 16:45:15.635592 2014-08-13 17:38:02.901421 7fe9bae887a0 20 read_log 192888'16258 (0'0) promote 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:19452 2014-08-11 16:45:17.030728 2014-08-13 17:38:02.901433 7fe9bae887a0 20 read_log 192888'16259 (192888'16258) modify 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by client.1315259.0:5791 2014-08-11 16:45:17.022785 2014-08-13 17:38:02.901445 7fe9bae887a0 20 read_log 192888'16262 (0'0) promote b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by osd.364.0:19502 2014-08-11 16:45:25.488980 2014-08-13 17:38:02.901457 7fe9bae887a0 20 read_log 192888'16263 (192888'16262) modify b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.1315259.0:8124 2014-08-11 16:45:25.481414 2014-08-13 17:38:02.901479 7fe9bae887a0 20 read_log 192888'16266 (0'0) promote 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by osd.364.0:19532 2014-08-11 16:45:30.002986 2014-08-13 17:38:02.901494 7fe9bae887a0 20 read_log 192888'16267 (192888'16266) modify 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.1315259.0:9317 2014-08-11 16:45:29.995421 2014-08-13 17:38:02.901506 7fe9bae887a0 20 read_log 192888'16268 (0'0) promote 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by osd.364.0:19558 2014-08-11 16:45:32.552281 2014-08-13 17:38:02.901518 7fe9bae887a0 20 read_log 192888'16269 (192888'16268) modify 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.1315259.0:9970 2014-08-11 16:45:32.545851 2014-08-13 17:38:02.901532 7fe9bae887a0 20 read_log 192888'16270 (0'0) promote 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by osd.364.0:19574 2014-08-11 16:45:36.817974 2014-08-13 17:38:02.901547 7fe9bae887a0 20 read_log 192888'16271 (192888'16270) modify 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.1315259.0:11050 2014-08-11 16:45:36.807954 2014-08-13 17:38:02.901560 7fe9bae887a0 20 read_log 192888'16272 (0'0) promote e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by osd.364.0:19580 2014-08-11 16:45:38.510500 2014-08-13 17:38:02.901572 7fe9bae887a0 20 read_log 192888'16273 (192888'16272) modify e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.1315259.0:11478 2014-08-11 16:45:38.502880 2014-08-13 17:38:02.901584 7fe9bae887a0 20 read_log 192888'16280 (0'0) promote bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:19630 2014-08-11 16:45:44.929039 2014-08-13 17:38:02.901596 7fe9bae887a0 20 read_log 192888'16281 (192888'16280) modify bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by client.1315259.0:13015 2014-08-11 16:45:44.921303 2014-08-13 17:38:02.901608 7fe9bae887a0 20 read_log 192888'16282 (0'0) promote 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by osd.364.0:19642 2014-08-11 16:45:46.677924 2014-08-13 17:38:02.901619 7fe9bae887a0 20 read_log 192888'16283 (192888'16282) modify 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.1315259.0:13450 2014-08-11 16:45:46.669664 2014-08-13 17:38:02.901632 7fe9bae887a0 20 read_log 192888'16292 (0'0) promote 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by osd.364.0:19700 2014-08-11 16:45:55.975387 2014-08-13 17:38:02.901643 7fe9bae887a0 20 read_log 192888'16293 (192888'16292) modify 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.1315259.0:15879 2014-08-11 16:45:55.965966 2014-08-13 17:38:02.901656 7fe9bae887a0 20 read_log 192888'16294 (0'0) promote 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by osd.364.0:19720 2014-08-11 16:45:59.912751 2014-08-13 17:38:02.901668 7fe9bae887a0 20 read_log 192888'16295 (192888'16294) modify 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.1315259.0:16897 2014-08-11 16:45:59.904944 2014-08-13 17:38:02.901680 7fe9bae887a0 20 read_log 192888'16298 (0'0) promote 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by osd.364.0:19752 2014-08-11 16:46:04.229801 2014-08-13 17:38:02.901692 7fe9bae887a0 20 read_log 192888'16299 (192888'16298) modify 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.1315259.0:17949 2014-08-11 16:46:04.222145 2014-08-13 17:38:02.901725 7fe9bae887a0 20 read_log 192888'16300 (0'0) promote 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by osd.364.0:19770 2014-08-11 16:46:08.009256 2014-08-13 17:38:02.901738 7fe9bae887a0 20 read_log 192888'16301 (192888'16300) modify 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.1315259.0:18958 2014-08-11 16:46:08.002035 2014-08-13 17:38:02.901757 7fe9bae887a0 20 read_log 192888'16302 (0'0) promote 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by osd.364.0:19786 2014-08-11 16:46:10.260014 2014-08-13 17:38:02.901772 7fe9bae887a0 20 read_log 192888'16303 (192888'16302) modify 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.1315259.0:19494 2014-08-11 16:46:10.252744 2014-08-13 17:38:02.901785 7fe9bae887a0 20 read_log 192888'16310 (0'0) promote 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:19830 2014-08-11 16:46:18.621241 2014-08-13 17:38:02.901797 7fe9bae887a0 20 read_log 192888'16311 (192888'16310) modify 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by client.1315259.0:21722 2014-08-11 16:46:18.613444 2014-08-13 17:38:02.901809 7fe9bae887a0 20 read_log 192888'16312 (0'0) promote de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by osd.364.0:19838 2014-08-11 16:46:20.689649 2014-08-13 17:38:02.901821 7fe9bae887a0 20 read_log 192888'16313 (192888'16312) modify de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.1315259.0:22258 2014-08-11 16:46:20.681703 2014-08-13 17:38:02.901833 7fe9bae887a0 20 read_log 192888'16318 (0'0) promote 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:19862 2014-08-11 16:46:24.434089 2014-08-13 17:38:02.901845 7fe9bae887a0 20 read_log 192888'16319 (192888'16318) modify 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by client.1315259.0:23191 2014-08-11 16:46:24.425606 2014-08-13 17:38:02.901857 7fe9bae887a0 20 read_log 192888'16320 (0'0) promote e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by osd.364.0:19876 2014-08-11 16:46:27.010744 2014-08-13 17:38:02.901869 7fe9bae887a0 20 read_log 192888'16321 (192888'16320) modify e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.1315259.0:23836 2014-08-11 16:46:27.001698 2014-08-13 17:38:02.901881 7fe9bae887a0 20 read_log 192888'16326 (0'0) promote c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:19914 2014-08-11 16:46:33.075963 2014-08-13 17:38:02.901893 7fe9bae887a0 20 read_log 192888'16327 (192888'16326) modify c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by client.1315259.0:25384 2014-08-11 16:46:33.069018 2014-08-13 17:38:02.901904 7fe9bae887a0 20 read_log 192888'16328 (0'0) promote 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:19916 2014-08-11 16:46:33.097385 2014-08-13 17:38:02.901916 7fe9bae887a0 20 read_log 192888'16329 (192888'16328) modify 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by client.1315259.0:25390 2014-08-11 16:46:33.090563 2014-08-13 17:38:02.901929 7fe9bae887a0 20 read_log 192888'16334 (0'0) promote 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by osd.364.0:19982 2014-08-11 16:46:44.641020 2014-08-13 17:38:02.901940 7fe9bae887a0 20 read_log 192888'16335 (192888'16334) modify 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.1315259.0:28172 2014-08-11 16:46:44.632411 2014-08-13 17:38:02.901952 7fe9bae887a0 20 read_log 192888'16336 (0'0) promote 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by osd.364.0:20026 2014-08-11 16:46:52.636669 2014-08-13 17:38:02.901964 7fe9bae887a0 20 read_log 192888'16337 (192888'16336) modify 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.1315259.0:30141 2014-08-11 16:46:52.628733 2014-08-13 17:38:02.901976 7fe9bae887a0 20 read_log 192888'16344 (192888'16235) modify 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.1315259.0:32589 2014-08-11 16:46:56.418727 2014-08-13 17:38:02.901988 7fe9bae887a0 20 read_log 192888'16349 (192888'16245) modify 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.1315259.0:34661 2014-08-11 16:46:56.952571 2014-08-13 17:38:02.902007 7fe9bae887a0 20 read_log 192888'16351 (192888'16249) modify e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.1315259.0:35289 2014-08-11 16:46:57.125816 2014-08-13 17:38:02.902023 7fe9bae887a0 20 read_log 192888'16352 (192888'16251) modify 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by client.1315259.0:35817 2014-08-11 16:46:57.254062 2014-08-13 17:38:02.902035 7fe9bae887a0 20 read_log 192888'16353 (192888'16253) modify 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.1315259.0:36001 2014-08-11 16:46:57.300781 2014-08-13 17:38:02.902047 7fe9bae887a0 20 read_log 192888'16354 (192888'16255) modify 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by client.1315259.0:36136 2014-08-11 16:46:57.345860 2014-08-13 17:38:02.902059 7fe9bae887a0 20 read_log 192888'16356 (192888'16259) modify 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by client.1315259.0:36518 2014-08-11 16:46:57.444282 2014-08-13 17:38:02.902071 7fe9bae887a0 20 read_log 192888'16358 (192888'16263) modify b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.1315259.0:38851 2014-08-11 16:46:58.061259 2014-08-13 17:38:02.902083 7fe9bae887a0 20 read_log 192888'16360 (192888'16267) modify 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.1315259.0:40044 2014-08-11 16:46:58.372533 2014-08-13 17:38:02.902095 7fe9bae887a0 20 read_log 192888'16361 (192888'16269) modify 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.1315259.0:40697 2014-08-11 16:46:58.556751 2014-08-13 17:38:02.902107 7fe9bae887a0 20 read_log 192888'16362 (192888'16271) modify 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.1315259.0:41777 2014-08-11 16:46:58.834777 2014-08-13 17:38:02.902119 7fe9bae887a0 20 read_log 192888'16363 (192888'16273) modify e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.1315259.0:42205 2014-08-11 16:46:58.946852 2014-08-13 17:38:02.902131 7fe9bae887a0 20 read_log 192888'16367 (192888'16281) modify bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by client.1315259.0:43742 2014-08-11 16:46:59.375212 2014-08-13 17:38:02.902144 7fe9bae887a0 20 read_log 192888'16368 (192888'16283) modify 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.1315259.0:44177 2014-08-11 16:46:59.503315 2014-08-13 17:38:02.902160 7fe9bae887a0 20 read_log 192888'16373 (192888'16293) modify 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.1315259.0:46606 2014-08-11 16:47:00.206167 2014-08-13 17:38:02.902173 7fe9bae887a0 20 read_log 192888'16374 (192888'16295) modify 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.1315259.0:47624 2014-08-11 16:47:00.516747 2014-08-13 17:38:02.902185 7fe9bae887a0 20 read_log 192888'16376 (192888'16299) modify 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.1315259.0:48676 2014-08-11 16:47:00.833176 2014-08-13 17:38:02.902197 7fe9bae887a0 20 read_log 192888'16377 (192888'16301) modify 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.1315259.0:49685 2014-08-11 16:47:01.142954 2014-08-13 17:38:02.902209 7fe9bae887a0 20 read_log 192888'16378 (192888'16303) modify 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.1315259.0:50221 2014-08-11 16:47:01.300075 2014-08-13 17:38:02.902221 7fe9bae887a0 20 read_log 192888'16382 (192888'16311) modify 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by client.1315259.0:52449 2014-08-11 16:47:01.976544 2014-08-13 17:38:02.902233 7fe9bae887a0 20 read_log 192888'16383 (192888'16313) modify de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.1315259.0:52985 2014-08-11 16:47:02.123022 2014-08-13 17:38:02.902245 7fe9bae887a0 20 read_log 192888'16386 (192888'16319) modify 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by client.1315259.0:53918 2014-08-11 16:47:02.402943 2014-08-13 17:38:02.902267 7fe9bae887a0 20 read_log 192888'16387 (192888'16321) modify e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.1315259.0:54563 2014-08-11 16:47:02.602145 2014-08-13 17:38:02.902280 7fe9bae887a0 20 read_log 192888'16390 (192888'16327) modify c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by client.1315259.0:56111 2014-08-11 16:47:03.108800 2014-08-13 17:38:02.902292 7fe9bae887a0 20 read_log 192888'16391 (192888'16329) modify 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by client.1315259.0:56117 2014-08-11 16:47:03.110276 2014-08-13 17:38:02.902304 7fe9bae887a0 20 read_log 192888'16394 (192888'16335) modify 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.1315259.0:58899 2014-08-11 16:47:03.941157 2014-08-13 17:38:02.902316 7fe9bae887a0 20 read_log 192888'16395 (192888'16337) modify 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.1315259.0:60868 2014-08-11 16:47:04.564442 2014-08-13 17:38:02.902328 7fe9bae887a0 20 read_log 192888'16398 (192888'16398) modify 1de/hit_set_24.1de_archive_2014-08-11 16:44:55.175331_2014-08-11 17:05:15.139431/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.139657 2014-08-13 17:38:02.902341 7fe9bae887a0 20 read_log 192888'16399 (192888'16229) delete 1de/hit_set_24.1de_archive_2014-08-11 16:02:27.862431_2014-08-11 16:44:55.175023/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.139657 2014-08-13 17:38:02.902357 7fe9bae887a0 20 read_log 192888'16406 (0'0) promote 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:20439 2014-08-11 17:05:18.937678 2014-08-13 17:38:02.902370 7fe9bae887a0 20 read_log 192888'16407 (192888'16406) modify 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by client.942868.0:962 2014-08-11 17:05:18.930939 2014-08-13 17:38:02.902382 7fe9bae887a0 20 read_log 192888'16422 (0'0) promote ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by osd.364.0:20495 2014-08-11 17:05:24.797291 2014-08-13 17:38:02.902394 7fe9bae887a0 20 read_log 192888'16423 (192888'16422) modify ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.942868.0:2393 2014-08-11 17:05:24.790409 2014-08-13 17:38:02.902406 7fe9bae887a0 20 read_log 192888'16428 (0'0) promote cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by osd.364.0:20548 2014-08-11 17:05:33.528128 2014-08-13 17:38:02.902418 7fe9bae887a0 20 read_log 192888'16429 (192888'16428) modify cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.942868.0:4739 2014-08-11 17:05:33.520182 2014-08-13 17:38:02.902431 7fe9bae887a0 20 read_log 192888'16430 (0'0) promote 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by osd.364.0:20558 2014-08-11 17:05:34.246892 2014-08-13 17:38:02.902443 7fe9bae887a0 20 read_log 192888'16431 (192888'16430) modify 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.942868.0:4913 2014-08-11 17:05:34.238843 2014-08-13 17:38:02.902455 7fe9bae887a0 20 read_log 192888'16432 (0'0) promote 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by osd.364.0:20560 2014-08-11 17:05:35.011449 2014-08-13 17:38:02.902467 7fe9bae887a0 20 read_log 192888'16433 (192888'16432) modify 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.942868.0:5103 2014-08-11 17:05:35.004378 2014-08-13 17:38:02.902479 7fe9bae887a0 20 read_log 192888'16440 (0'0) promote 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by osd.364.0:20584 2014-08-11 17:05:37.764615 2014-08-13 17:38:02.902491 7fe9bae887a0 20 read_log 192888'16441 (192888'16440) modify 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.942868.0:5846 2014-08-11 17:05:37.755472 2014-08-13 17:38:02.902510 7fe9bae887a0 20 read_log 192888'16444 (0'0) promote 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by osd.364.0:20626 2014-08-11 17:05:46.758850 2014-08-13 17:38:02.902523 7fe9bae887a0 20 read_log 192888'16445 (192888'16444) modify 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.942868.0:8258 2014-08-11 17:05:46.751066 2014-08-13 17:38:02.902535 7fe9bae887a0 20 read_log 192888'16448 (0'0) promote 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:20634 2014-08-11 17:05:48.042259 2014-08-13 17:38:02.902550 7fe9bae887a0 20 read_log 192888'16449 (192888'16448) modify 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by client.942868.0:8613 2014-08-11 17:05:48.033997 2014-08-13 17:38:02.902562 7fe9bae887a0 20 read_log 192888'16454 (0'0) promote b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:20690 2014-08-11 17:05:56.269991 2014-08-13 17:38:02.902574 7fe9bae887a0 20 read_log 192888'16455 (192888'16454) modify b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by client.942868.0:10853 2014-08-11 17:05:56.263272 2014-08-13 17:38:02.902586 7fe9bae887a0 20 read_log 192888'16460 (0'0) promote 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:20720 2014-08-11 17:06:00.982692 2014-08-13 17:38:02.902598 7fe9bae887a0 20 read_log 192888'16461 (192888'16460) modify 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by client.942868.0:12133 2014-08-11 17:06:00.975755 2014-08-13 17:38:02.902610 7fe9bae887a0 20 read_log 192888'16468 (0'0) promote ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by osd.364.0:20780 2014-08-11 17:06:13.826025 2014-08-13 17:38:02.902622 7fe9bae887a0 20 read_log 192888'16469 (192888'16468) modify ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.942868.0:15411 2014-08-11 17:06:13.816897 2014-08-13 17:38:02.902635 7fe9bae887a0 20 read_log 192888'16470 (0'0) promote 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:20784 2014-08-11 17:06:14.540096 2014-08-13 17:38:02.902647 7fe9bae887a0 20 read_log 192888'16471 (192888'16470) modify 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by client.942868.0:15595 2014-08-11 17:06:14.532350 2014-08-13 17:38:02.902659 7fe9bae887a0 20 read_log 192888'16474 (0'0) promote 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by osd.364.0:20820 2014-08-11 17:06:20.165524 2014-08-13 17:38:02.902671 7fe9bae887a0 20 read_log 192888'16475 (192888'16474) modify 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.942868.0:17027 2014-08-11 17:06:20.157502 2014-08-13 17:38:02.902683 7fe9bae887a0 20 read_log 192888'16476 (0'0) promote c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by osd.364.0:20838 2014-08-11 17:06:22.533295 2014-08-13 17:38:02.902695 7fe9bae887a0 20 read_log 192888'16477 (192888'16476) modify c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.942868.0:17654 2014-08-11 17:06:22.525712 2014-08-13 17:38:02.902707 7fe9bae887a0 20 read_log 192888'16478 (0'0) promote a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by osd.364.0:20854 2014-08-11 17:06:24.656726 2014-08-13 17:38:02.902719 7fe9bae887a0 20 read_log 192888'16479 (192888'16478) modify a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.942868.0:18216 2014-08-11 17:06:24.648836 2014-08-13 17:38:02.902731 7fe9bae887a0 20 read_log 192888'16486 (0'0) promote 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:20874 2014-08-11 17:06:26.991702 2014-08-13 17:38:02.902762 7fe9bae887a0 20 read_log 192888'16487 (192888'16486) modify 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by client.942868.0:18839 2014-08-11 17:06:26.984106 2014-08-13 17:38:02.902785 7fe9bae887a0 20 read_log 192888'16490 (0'0) promote d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by osd.364.0:20900 2014-08-11 17:06:31.560696 2014-08-13 17:38:02.902800 7fe9bae887a0 20 read_log 192888'16491 (192888'16490) modify d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.942868.0:20053 2014-08-11 17:06:31.553712 2014-08-13 17:38:02.902812 7fe9bae887a0 20 read_log 192888'16492 (0'0) promote 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by osd.364.0:20908 2014-08-11 17:06:32.446494 2014-08-13 17:38:02.902824 7fe9bae887a0 20 read_log 192888'16493 (192888'16492) modify 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.942868.0:20291 2014-08-11 17:06:32.439294 2014-08-13 17:38:02.902836 7fe9bae887a0 20 read_log 192888'16502 (0'0) promote a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:20954 2014-08-11 17:06:40.072326 2014-08-13 17:38:02.902848 7fe9bae887a0 20 read_log 192888'16503 (192888'16502) modify a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by client.942868.0:22282 2014-08-11 17:06:40.065628 2014-08-13 17:38:02.902860 7fe9bae887a0 20 read_log 192888'16504 (0'0) promote 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by osd.364.0:20964 2014-08-11 17:06:40.571476 2014-08-13 17:38:02.902872 7fe9bae887a0 20 read_log 192888'16505 (192888'16504) modify 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.942868.0:22403 2014-08-11 17:06:40.563489 2014-08-13 17:38:02.902885 7fe9bae887a0 20 read_log 192888'16508 (0'0) promote 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:21002 2014-08-11 17:06:46.309230 2014-08-13 17:38:02.902899 7fe9bae887a0 20 read_log 192888'16509 (192888'16508) modify 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by client.942868.0:23890 2014-08-11 17:06:46.301650 2014-08-13 17:38:02.902912 7fe9bae887a0 20 read_log 192888'16510 (0'0) promote 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:21014 2014-08-11 17:06:48.132069 2014-08-13 17:38:02.902924 7fe9bae887a0 20 read_log 192888'16511 (192888'16510) modify 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by client.942868.0:24387 2014-08-11 17:06:48.124661 2014-08-13 17:38:02.902937 7fe9bae887a0 20 read_log 192888'16514 (0'0) promote 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by osd.364.0:21022 2014-08-11 17:06:49.291121 2014-08-13 17:38:02.902949 7fe9bae887a0 20 read_log 192888'16515 (192888'16514) modify 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.942868.0:24697 2014-08-11 17:06:49.284245 2014-08-13 17:38:02.902961 7fe9bae887a0 20 read_log 192888'16516 (0'0) promote eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by osd.364.0:21028 2014-08-11 17:06:49.436781 2014-08-13 17:38:02.902972 7fe9bae887a0 20 read_log 192888'16517 (192888'16516) modify eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.942868.0:24736 2014-08-11 17:06:49.429485 2014-08-13 17:38:02.902984 7fe9bae887a0 20 read_log 192888'16518 (0'0) promote 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by osd.364.0:21066 2014-08-11 17:06:55.593921 2014-08-13 17:38:02.902996 7fe9bae887a0 20 read_log 192888'16519 (192888'16518) modify 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.942868.0:26306 2014-08-11 17:06:55.587243 2014-08-13 17:38:02.903008 7fe9bae887a0 20 read_log 192888'16522 (0'0) promote 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by osd.364.0:21094 2014-08-11 17:07:01.623097 2014-08-13 17:38:02.903020 7fe9bae887a0 20 read_log 192888'16523 (192888'16522) modify 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.942868.0:27899 2014-08-11 17:07:01.615714 2014-08-13 17:38:02.903065 7fe9bae887a0 20 read_log 192888'16526 (0'0) promote 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by osd.364.0:21136 2014-08-11 17:07:08.980189 2014-08-13 17:38:02.903081 7fe9bae887a0 20 read_log 192888'16527 (192888'16526) modify 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.942868.0:29793 2014-08-11 17:07:08.973343 2014-08-13 17:38:02.903093 7fe9bae887a0 20 read_log 192888'16535 (192888'16407) modify 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by client.942868.0:32228 2014-08-11 17:07:16.194477 2014-08-13 17:38:02.903106 7fe9bae887a0 20 read_log 192888'16543 (192888'16423) modify ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.942868.0:33659 2014-08-11 17:07:16.575315 2014-08-13 17:38:02.903118 7fe9bae887a0 20 read_log 192888'16546 (192888'16429) modify cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.942868.0:36005 2014-08-11 17:07:17.218100 2014-08-13 17:38:02.903130 7fe9bae887a0 20 read_log 192888'16547 (192888'16431) modify 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.942868.0:36179 2014-08-11 17:07:17.260815 2014-08-13 17:38:02.903142 7fe9bae887a0 20 read_log 192888'16548 (192888'16433) modify 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.942868.0:36369 2014-08-11 17:07:17.310466 2014-08-13 17:38:02.903162 7fe9bae887a0 20 read_log 192888'16552 (192888'16441) modify 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.942868.0:37112 2014-08-11 17:07:17.518995 2014-08-13 17:38:02.903175 7fe9bae887a0 20 read_log 192888'16554 (192888'16445) modify 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.942868.0:39524 2014-08-11 17:07:18.205816 2014-08-13 17:38:02.903188 7fe9bae887a0 20 read_log 192888'16556 (192888'16449) modify 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by client.942868.0:39879 2014-08-11 17:07:18.307556 2014-08-13 17:38:02.903200 7fe9bae887a0 20 read_log 192888'16559 (192888'16455) modify b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by client.942868.0:42119 2014-08-11 17:07:18.910739 2014-08-13 17:38:02.903212 7fe9bae887a0 20 read_log 192888'16562 (192888'16461) modify 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by client.942868.0:43399 2014-08-11 17:07:19.262028 2014-08-13 17:38:02.903224 7fe9bae887a0 20 read_log 192888'16566 (192888'16469) modify ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.942868.0:46677 2014-08-11 17:07:20.188625 2014-08-13 17:38:02.903236 7fe9bae887a0 20 read_log 192888'16567 (192888'16471) modify 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by client.942868.0:46861 2014-08-11 17:07:20.238902 2014-08-13 17:38:02.903248 7fe9bae887a0 20 read_log 192888'16569 (192888'16475) modify 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.942868.0:48293 2014-08-11 17:07:20.678616 2014-08-13 17:38:02.903260 7fe9bae887a0 20 read_log 192888'16570 (192888'16477) modify c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.942868.0:48920 2014-08-11 17:07:20.857758 2014-08-13 17:38:02.903272 7fe9bae887a0 20 read_log 192888'16571 (192888'16479) modify a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.942868.0:49482 2014-08-11 17:07:21.026204 2014-08-13 17:38:02.903285 7fe9bae887a0 20 read_log 192888'16575 (192888'16487) modify 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by client.942868.0:50105 2014-08-11 17:07:21.218616 2014-08-13 17:38:02.903297 7fe9bae887a0 20 read_log 192888'16577 (192888'16491) modify d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.942868.0:51319 2014-08-11 17:07:21.607241 2014-08-13 17:38:02.903309 7fe9bae887a0 20 read_log 192888'16578 (192888'16493) modify 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.942868.0:51557 2014-08-11 17:07:21.681049 2014-08-13 17:38:02.903331 7fe9bae887a0 20 read_log 192888'16583 (192888'16503) modify a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by client.942868.0:53548 2014-08-11 17:07:22.301462 2014-08-13 17:38:02.903344 7fe9bae887a0 20 read_log 192888'16584 (192888'16505) modify 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.942868.0:53669 2014-08-11 17:07:22.338824 2014-08-13 17:38:02.903356 7fe9bae887a0 20 read_log 192888'16586 (192888'16509) modify 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by client.942868.0:55156 2014-08-11 17:07:22.760166 2014-08-13 17:38:02.903371 7fe9bae887a0 20 read_log 192888'16587 (192888'16511) modify 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by client.942868.0:55653 2014-08-11 17:07:22.912653 2014-08-13 17:38:02.903383 7fe9bae887a0 20 read_log 192888'16589 (192888'16515) modify 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.942868.0:55963 2014-08-11 17:07:23.009726 2014-08-13 17:38:02.903395 7fe9bae887a0 20 read_log 192888'16590 (192888'16517) modify eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.942868.0:56002 2014-08-11 17:07:23.021591 2014-08-13 17:38:02.903407 7fe9bae887a0 20 read_log 192888'16591 (192888'16519) modify 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.942868.0:57572 2014-08-11 17:07:23.493667 2014-08-13 17:38:02.903419 7fe9bae887a0 20 read_log 192888'16593 (192888'16523) modify 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.942868.0:59165 2014-08-11 17:07:23.964641 2014-08-13 17:38:02.903431 7fe9bae887a0 20 read_log 192888'16595 (192888'16527) modify 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.942868.0:61059 2014-08-11 17:07:24.526657 2014-08-13 17:38:02.903444 7fe9bae887a0 20 read_log 193305'16598 (193305'16598) modify 1de/hit_set_24.1de_archive_2014-08-11 18:37:04.915039_2014-08-11 18:42:33.427516/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.427747 2014-08-13 17:38:02.903456 7fe9bae887a0 20 read_log 193305'16599 (192888'16398) delete 1de/hit_set_24.1de_archive_2014-08-11 16:44:55.175331_2014-08-11 17:05:15.139431/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.427747 2014-08-13 17:38:02.903468 7fe9bae887a0 20 read_log 193305'16600 (0'0) promote 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by osd.364.0:5 2014-08-11 18:42:33.565205 2014-08-13 17:38:02.903482 7fe9bae887a0 20 read_log 193305'16601 (193305'16600) modify 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.951337.0:130 2014-08-11 18:42:33.420297 2014-08-13 17:38:02.903494 7fe9bae887a0 20 read_log 193305'16602 (0'0) promote b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:24 2014-08-11 18:42:36.888556 2014-08-13 17:38:02.903506 7fe9bae887a0 20 read_log 193305'16603 (193305'16602) modify b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by client.951337.0:976 2014-08-11 18:42:36.880780 2014-08-13 17:38:02.903519 7fe9bae887a0 20 read_log 193305'16604 (0'0) promote 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:28 2014-08-11 18:42:37.125821 2014-08-13 17:38:02.903531 7fe9bae887a0 20 read_log 193305'16605 (193305'16604) modify 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by client.951337.0:1042 2014-08-11 18:42:37.118434 2014-08-13 17:38:02.903543 7fe9bae887a0 20 read_log 193305'16606 (0'0) promote 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by osd.364.0:38 2014-08-11 18:42:39.259338 2014-08-13 17:38:02.903555 7fe9bae887a0 20 read_log 193305'16607 (193305'16606) modify 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.951337.0:1597 2014-08-11 18:42:39.251620 2014-08-13 17:38:02.903575 7fe9bae887a0 20 read_log 193305'16608 (0'0) promote a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by osd.364.0:50 2014-08-11 18:42:40.724812 2014-08-13 17:38:02.903588 7fe9bae887a0 20 read_log 193305'16609 (193305'16608) modify a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.951337.0:1990 2014-08-11 18:42:40.717969 2014-08-13 17:38:02.903600 7fe9bae887a0 20 read_log 193305'16610 (0'0) promote 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by osd.364.0:91 2014-08-11 18:42:56.237810 2014-08-13 17:38:02.903612 7fe9bae887a0 20 read_log 193305'16611 (193305'16610) modify 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.951337.0:6223 2014-08-11 18:42:56.230859 2014-08-13 17:38:02.903624 7fe9bae887a0 20 read_log 193305'16612 (0'0) promote af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:109 2014-08-11 18:43:00.118164 2014-08-13 17:38:02.903636 7fe9bae887a0 20 read_log 193305'16613 (193305'16612) modify af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by client.951337.0:7298 2014-08-11 18:43:00.110326 2014-08-13 17:38:02.903667 7fe9bae887a0 20 read_log 193305'16614 (0'0) promote 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by osd.364.0:111 2014-08-11 18:43:01.064481 2014-08-13 17:38:02.903681 7fe9bae887a0 20 read_log 193305'16615 (193305'16614) modify 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.951337.0:7557 2014-08-11 18:43:01.055850 2014-08-13 17:38:02.903694 7fe9bae887a0 20 read_log 193305'16616 (0'0) promote 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by osd.364.0:159 2014-08-11 18:43:14.321888 2014-08-13 17:38:02.903706 7fe9bae887a0 20 read_log 193305'16617 (193305'16616) modify 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.951337.0:11084 2014-08-11 18:43:14.314494 2014-08-13 17:38:02.903718 7fe9bae887a0 20 read_log 193305'16618 (0'0) promote 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by osd.364.0:183 2014-08-11 18:43:21.326837 2014-08-13 17:38:02.903730 7fe9bae887a0 20 read_log 193305'16619 (193305'16618) modify 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.951337.0:12919 2014-08-11 18:43:21.318193 2014-08-13 17:38:02.903742 7fe9bae887a0 20 read_log 193305'16620 (0'0) promote d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by osd.364.0:185 2014-08-11 18:43:22.042479 2014-08-13 17:38:02.903754 7fe9bae887a0 20 read_log 193305'16621 (193305'16620) modify d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.951337.0:13112 2014-08-11 18:43:22.033021 2014-08-13 17:38:02.903766 7fe9bae887a0 20 read_log 193305'16622 (0'0) promote 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by osd.364.0:191 2014-08-11 18:43:23.621399 2014-08-13 17:38:02.903779 7fe9bae887a0 20 read_log 193305'16623 (193305'16622) modify 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.951337.0:13515 2014-08-11 18:43:23.608926 2014-08-13 17:38:02.903791 7fe9bae887a0 20 read_log 193305'16624 (0'0) promote 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by osd.364.0:209 2014-08-11 18:43:26.828802 2014-08-13 17:38:02.903803 7fe9bae887a0 20 read_log 193305'16625 (193305'16624) modify 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.951337.0:14380 2014-08-11 18:43:26.821989 2014-08-13 17:38:02.903815 7fe9bae887a0 20 read_log 193305'16626 (0'0) promote 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:252 2014-08-11 18:43:39.782750 2014-08-13 17:38:02.903827 7fe9bae887a0 20 read_log 193305'16627 (193305'16626) modify 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by client.951337.0:18118 2014-08-11 18:43:39.773301 2014-08-13 17:38:02.903849 7fe9bae887a0 20 read_log 193305'16628 (0'0) promote 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:256 2014-08-11 18:43:40.896165 2014-08-13 17:38:02.903862 7fe9bae887a0 20 read_log 193305'16629 (193305'16628) modify 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by client.951337.0:18431 2014-08-11 18:43:40.888150 2014-08-13 17:38:02.903874 7fe9bae887a0 20 read_log 193305'16630 (0'0) promote 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by osd.364.0:260 2014-08-11 18:43:40.992499 2014-08-13 17:38:02.903886 7fe9bae887a0 20 read_log 193305'16631 (193305'16630) modify 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.951337.0:18460 2014-08-11 18:43:40.984957 2014-08-13 17:38:02.903899 7fe9bae887a0 20 read_log 193305'16632 (0'0) promote d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:286 2014-08-11 18:43:49.918793 2014-08-13 17:38:02.903910 7fe9bae887a0 20 read_log 193305'16633 (193305'16632) modify d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by client.951337.0:20923 2014-08-11 18:43:49.912169 2014-08-13 17:38:02.903922 7fe9bae887a0 20 read_log 193305'16634 (0'0) promote f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:322 2014-08-11 18:43:58.359432 2014-08-13 17:38:02.903934 7fe9bae887a0 20 read_log 193305'16635 (193305'16634) modify f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by client.951337.0:23264 2014-08-11 18:43:58.351129 2014-08-13 17:38:02.903946 7fe9bae887a0 20 read_log 193305'16636 (0'0) promote 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by osd.364.0:340 2014-08-11 18:44:01.044268 2014-08-13 17:38:02.903978 7fe9bae887a0 20 read_log 193305'16637 (193305'16636) modify 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.951337.0:24004 2014-08-11 18:44:01.034545 2014-08-13 17:38:02.903991 7fe9bae887a0 20 read_log 193305'16638 (0'0) promote 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:344 2014-08-11 18:44:01.637172 2014-08-13 17:38:02.904003 7fe9bae887a0 20 read_log 193305'16639 (193305'16638) modify 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by client.951337.0:24162 2014-08-11 18:44:01.627871 2014-08-13 17:38:02.904016 7fe9bae887a0 20 read_log 193305'16640 (0'0) promote 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by osd.364.0:390 2014-08-11 18:44:09.407700 2014-08-13 17:38:02.904045 7fe9bae887a0 20 read_log 193305'16641 (193305'16640) modify 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.951337.0:26277 2014-08-11 18:44:09.399033 2014-08-13 17:38:02.904060 7fe9bae887a0 20 read_log 193305'16642 (0'0) promote 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:398 2014-08-11 18:44:10.407722 2014-08-13 17:38:02.904073 7fe9bae887a0 20 read_log 193305'16643 (193305'16642) modify 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by client.951337.0:26561 2014-08-11 18:44:10.399675 2014-08-13 17:38:02.904085 7fe9bae887a0 20 read_log 193305'16644 (0'0) promote 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by osd.364.0:404 2014-08-11 18:44:14.032514 2014-08-13 17:38:02.904097 7fe9bae887a0 20 read_log 193305'16645 (193305'16644) modify 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.951337.0:27541 2014-08-11 18:44:14.024928 2014-08-13 17:38:02.904109 7fe9bae887a0 20 read_log 193305'16646 (0'0) promote 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by osd.364.0:406 2014-08-11 18:44:14.583690 2014-08-13 17:38:02.904121 7fe9bae887a0 20 read_log 193305'16647 (193305'16646) modify 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.951337.0:27698 2014-08-11 18:44:14.575030 2014-08-13 17:38:02.904141 7fe9bae887a0 20 read_log 193305'16648 (0'0) promote d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:424 2014-08-11 18:44:17.990246 2014-08-13 17:38:02.904166 7fe9bae887a0 20 read_log 193305'16649 (193305'16648) modify d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by client.951337.0:28642 2014-08-11 18:44:17.982711 2014-08-13 17:38:02.904182 7fe9bae887a0 20 read_log 193305'16650 (0'0) promote e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by osd.364.0:430 2014-08-11 18:44:18.563664 2014-08-13 17:38:02.904194 7fe9bae887a0 20 read_log 193305'16651 (193305'16650) modify e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.951337.0:28786 2014-08-11 18:44:18.553845 2014-08-13 17:38:02.904206 7fe9bae887a0 20 read_log 193305'16652 (0'0) promote b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:442 2014-08-11 18:44:20.594027 2014-08-13 17:38:02.904218 7fe9bae887a0 20 read_log 193305'16653 (193305'16652) modify b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by client.951337.0:29343 2014-08-11 18:44:20.586383 2014-08-13 17:38:02.904230 7fe9bae887a0 20 read_log 193305'16654 (0'0) promote 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by osd.364.0:444 2014-08-11 18:44:20.819641 2014-08-13 17:38:02.904242 7fe9bae887a0 20 read_log 193305'16655 (193305'16654) modify 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.951337.0:29406 2014-08-11 18:44:20.813127 2014-08-13 17:38:02.904254 7fe9bae887a0 20 read_log 193305'16656 (0'0) promote 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by osd.364.0:452 2014-08-11 18:44:22.431884 2014-08-13 17:38:02.904266 7fe9bae887a0 20 read_log 193305'16657 (193305'16656) modify 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.951337.0:29844 2014-08-11 18:44:22.425079 2014-08-13 17:38:02.904279 7fe9bae887a0 20 read_log 193305'16658 (0'0) promote 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by osd.364.0:480 2014-08-11 18:44:28.376804 2014-08-13 17:38:02.904291 7fe9bae887a0 20 read_log 193305'16659 (193305'16658) modify 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.951337.0:31444 2014-08-11 18:44:28.369690 2014-08-13 17:38:02.904302 7fe9bae887a0 20 read_log 193305'16660 (0'0) promote d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by osd.364.0:486 2014-08-11 18:44:30.591503 2014-08-13 17:38:02.904314 7fe9bae887a0 20 read_log 193305'16661 (193305'16660) modify d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.951337.0:32094 2014-08-11 18:44:30.583611 2014-08-13 17:38:02.904326 7fe9bae887a0 20 read_log 193305'16662 (193305'16601) modify 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.951337.0:32841 2014-08-11 18:44:33.898900 2014-08-13 17:38:02.904338 7fe9bae887a0 20 read_log 193305'16663 (193305'16603) modify b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by client.951337.0:33687 2014-08-11 18:44:34.580089 2014-08-13 17:38:02.904350 7fe9bae887a0 20 read_log 193305'16664 (193305'16605) modify 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by client.951337.0:33753 2014-08-11 18:44:34.625191 2014-08-13 17:38:02.904362 7fe9bae887a0 20 read_log 193305'16665 (193305'16607) modify 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.951337.0:34308 2014-08-11 18:44:34.916223 2014-08-13 17:38:02.904375 7fe9bae887a0 20 read_log 193305'16666 (193305'16609) modify a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.951337.0:34701 2014-08-11 18:44:35.142659 2014-08-13 17:38:02.904387 7fe9bae887a0 20 read_log 193305'16667 (193305'16611) modify 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.951337.0:38934 2014-08-11 18:44:37.741047 2014-08-13 17:38:02.904406 7fe9bae887a0 20 read_log 193305'16668 (193305'16613) modify af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by client.951337.0:40009 2014-08-11 18:44:38.260123 2014-08-13 17:38:02.904419 7fe9bae887a0 20 read_log 193305'16669 (193305'16615) modify 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.951337.0:40268 2014-08-11 18:44:38.355307 2014-08-13 17:38:02.904431 7fe9bae887a0 20 read_log 193305'16670 (193305'16617) modify 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.951337.0:43795 2014-08-11 18:44:39.700289 2014-08-13 17:38:02.904443 7fe9bae887a0 20 read_log 193305'16671 (193305'16619) modify 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.951337.0:45630 2014-08-11 18:44:40.255101 2014-08-13 17:38:02.904455 7fe9bae887a0 20 read_log 193305'16672 (193305'16621) modify d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.951337.0:45823 2014-08-11 18:44:40.312112 2014-08-13 17:38:02.904466 7fe9bae887a0 20 read_log 193305'16673 (193305'16623) modify 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.951337.0:46226 2014-08-11 18:44:40.424384 2014-08-13 17:38:02.904478 7fe9bae887a0 20 read_log 193305'16674 (193305'16625) modify 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.951337.0:47091 2014-08-11 18:44:40.671485 2014-08-13 17:38:02.904490 7fe9bae887a0 20 read_log 193305'16675 (193305'16627) modify 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by client.951337.0:50829 2014-08-11 18:44:41.773591 2014-08-13 17:38:02.904502 7fe9bae887a0 20 read_log 193305'16676 (193305'16629) modify 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by client.951337.0:51142 2014-08-11 18:44:41.865926 2014-08-13 17:38:02.904514 7fe9bae887a0 20 read_log 193305'16677 (193305'16631) modify 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.951337.0:51171 2014-08-11 18:44:41.873828 2014-08-13 17:38:02.904527 7fe9bae887a0 20 read_log 193305'16678 (193305'16633) modify d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by client.951337.0:53634 2014-08-11 18:44:42.587204 2014-08-13 17:38:02.904539 7fe9bae887a0 20 read_log 193305'16679 (193305'16635) modify f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by client.951337.0:55975 2014-08-11 18:44:43.283626 2014-08-13 17:38:02.904551 7fe9bae887a0 20 read_log 193305'16680 (193305'16637) modify 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.951337.0:56715 2014-08-11 18:44:43.507995 2014-08-13 17:38:02.904563 7fe9bae887a0 20 read_log 193305'16681 (193305'16639) modify 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by client.951337.0:56873 2014-08-11 18:44:43.558522 2014-08-13 17:38:02.904575 7fe9bae887a0 20 read_log 193305'16682 (193305'16641) modify 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.951337.0:58988 2014-08-11 18:44:44.204622 2014-08-13 17:38:02.904587 7fe9bae887a0 20 read_log 193305'16683 (193305'16643) modify 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by client.951337.0:59272 2014-08-11 18:44:44.278097 2014-08-13 17:38:02.904600 7fe9bae887a0 20 read_log 193305'16684 (193305'16645) modify 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.951337.0:60252 2014-08-11 18:44:44.573356 2014-08-13 17:38:02.904613 7fe9bae887a0 20 read_log 193305'16685 (193305'16647) modify 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.951337.0:60409 2014-08-11 18:44:44.622451 2014-08-13 17:38:02.904628 7fe9bae887a0 20 read_log 193305'16686 (193305'16649) modify d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by client.951337.0:61353 2014-08-11 18:44:44.900210 2014-08-13 17:38:02.904640 7fe9bae887a0 20 read_log 193305'16687 (193305'16651) modify e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.951337.0:61497 2014-08-11 18:44:44.938268 2014-08-13 17:38:02.904662 7fe9bae887a0 20 read_log 193305'16688 (193305'16653) modify b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by client.951337.0:62054 2014-08-11 18:44:45.114203 2014-08-13 17:38:02.904679 7fe9bae887a0 20 read_log 193305'16689 (193305'16655) modify 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.951337.0:62117 2014-08-11 18:44:45.130191 2014-08-13 17:38:02.904691 7fe9bae887a0 20 read_log 193305'16690 (193305'16657) modify 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.951337.0:62555 2014-08-11 18:44:45.261835 2014-08-13 17:38:02.904703 7fe9bae887a0 20 read_log 193305'16691 (193305'16659) modify 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.951337.0:64155 2014-08-11 18:44:45.721145 2014-08-13 17:38:02.904715 7fe9bae887a0 20 read_log 193305'16692 (193305'16661) modify d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.951337.0:64805 2014-08-11 18:44:45.924653 2014-08-13 17:38:02.904727 7fe9bae887a0 20 read_log 194209'16693 (192834'16164) clean b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.364.0:1161 2014-08-12 14:06:37.171341 2014-08-13 17:38:02.904739 7fe9bae887a0 20 read_log 194209'16694 (192888'16587) clean 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:1162 2014-08-12 14:06:37.172011 2014-08-13 17:38:02.904752 7fe9bae887a0 20 read_log 194209'16695 (192888'16556) clean 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:1163 2014-08-12 14:06:37.172433 2014-08-13 17:38:02.904764 7fe9bae887a0 20 read_log 194209'16696 (192834'15677) clean 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.364.0:1164 2014-08-12 14:06:37.172924 2014-08-13 17:38:02.904776 7fe9bae887a0 20 read_log 194209'16697 (192888'16562) clean 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:1165 2014-08-12 14:06:37.183262 2014-08-13 17:38:02.904788 7fe9bae887a0 20 read_log 194209'16698 (192834'15619) clean 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.364.0:1166 2014-08-12 14:06:37.183548 2014-08-13 17:38:02.904800 7fe9bae887a0 20 read_log 194209'16699 (192834'15673) clean 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.364.0:1167 2014-08-12 14:06:37.186788 2014-08-13 17:38:02.904812 7fe9bae887a0 20 read_log 194209'16700 (193305'16668) clean af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:1168 2014-08-12 14:06:37.190384 2014-08-13 17:38:02.904824 7fe9bae887a0 20 read_log 194209'16701 (192888'16390) clean c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:1169 2014-08-12 14:06:37.193657 2014-08-13 17:38:02.904836 7fe9bae887a0 20 read_log 194209'16702 (192888'16386) clean 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:1170 2014-08-12 14:06:37.196992 2014-08-13 17:38:02.904848 7fe9bae887a0 20 read_log 194209'16703 (192888'16583) clean a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:1171 2014-08-12 14:06:37.200305 2014-08-13 17:38:02.904860 7fe9bae887a0 20 read_log 194209'16704 (192834'16131) clean 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.364.0:1172 2014-08-12 14:06:37.206235 2014-08-13 17:38:02.904872 7fe9bae887a0 20 read_log 194209'16705 (192834'15587) clean 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.364.0:1173 2014-08-12 14:06:37.206514 2014-08-13 17:38:02.904884 7fe9bae887a0 20 read_log 194209'16706 (192834'16146) clean 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.364.0:1174 2014-08-12 14:06:37.209720 2014-08-13 17:38:02.904929 7fe9bae887a0 20 read_log 194209'16707 (193305'16679) clean f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:1175 2014-08-12 14:06:37.213057 2014-08-13 17:38:02.904945 7fe9bae887a0 20 read_log 194209'16708 (192888'16367) clean bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:1176 2014-08-12 14:06:37.216381 2014-08-13 17:38:02.904958 7fe9bae887a0 20 read_log 194209'16709 (192888'16352) clean 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:1177 2014-08-12 14:06:37.219708 2014-08-13 17:38:02.904970 7fe9bae887a0 20 read_log 194209'16710 (193305'16663) clean b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:1178 2014-08-12 14:06:37.223069 2014-08-13 17:38:02.904982 7fe9bae887a0 20 read_log 194209'16711 (193305'16676) clean 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:1179 2014-08-12 14:06:37.226308 2014-08-13 17:38:02.904994 7fe9bae887a0 20 read_log 194209'16712 (193305'16688) clean b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:1180 2014-08-12 14:06:37.229514 2014-08-13 17:38:02.905006 7fe9bae887a0 20 read_log 194209'16713 (192834'16191) clean c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.364.0:1181 2014-08-12 14:06:37.235306 2014-08-13 17:38:02.905018 7fe9bae887a0 20 read_log 194209'16714 (192888'16586) clean 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:1182 2014-08-12 14:06:37.235818 2014-08-13 17:38:02.905030 7fe9bae887a0 20 read_log 194209'16715 (193305'16664) clean 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:1183 2014-08-12 14:06:37.239009 2014-08-13 17:38:02.905042 7fe9bae887a0 20 read_log 194209'16716 (192888'16382) clean 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:1184 2014-08-12 14:06:37.242159 2014-08-13 17:38:02.905054 7fe9bae887a0 20 read_log 194209'16717 (192834'16207) clean 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.364.0:1185 2014-08-12 14:06:37.245363 2014-08-13 17:38:02.905065 7fe9bae887a0 20 read_log 194209'16718 (193305'16686) clean d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:1186 2014-08-12 14:06:37.248618 2014-08-13 17:38:02.905077 7fe9bae887a0 20 read_log 194209'16719 (192834'16226) clean dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.364.0:1187 2014-08-12 14:06:37.251854 2014-08-13 17:38:02.905089 7fe9bae887a0 20 read_log 194209'16720 (192834'15627) clean e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.364.0:1188 2014-08-12 14:06:37.257606 2014-08-13 17:38:02.905101 7fe9bae887a0 20 read_log 194209'16721 (192834'15700) clean 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.364.0:1189 2014-08-12 14:06:37.257876 2014-08-13 17:38:02.905113 7fe9bae887a0 20 read_log 194209'16722 (192888'16559) clean b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:1190 2014-08-12 14:06:37.261061 2014-08-13 17:38:02.905125 7fe9bae887a0 20 read_log 194209'16723 (192834'16092) clean 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.364.0:1191 2014-08-12 14:06:37.267324 2014-08-13 17:38:02.905137 7fe9bae887a0 20 read_log 194209'16724 (193305'16675) clean 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:1192 2014-08-12 14:06:37.267571 2014-08-13 17:38:02.905149 7fe9bae887a0 20 read_log 194209'16725 (192888'16356) clean 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:1193 2014-08-12 14:06:37.270763 2014-08-13 17:38:02.905165 7fe9bae887a0 20 read_log 194209'16726 (192834'16143) clean 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.364.0:1194 2014-08-12 14:06:37.274031 2014-08-13 17:38:02.905186 7fe9bae887a0 20 read_log 194209'16727 (192888'16575) clean 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:1195 2014-08-12 14:06:37.277331 2014-08-13 17:38:02.905199 7fe9bae887a0 20 read_log 194209'16728 (193305'16678) clean d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:1196 2014-08-12 14:06:37.280497 2014-08-13 17:38:02.905211 7fe9bae887a0 20 read_log 194209'16729 (193305'16681) clean 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:1197 2014-08-12 14:06:37.283796 2014-08-13 17:38:02.905223 7fe9bae887a0 20 read_log 194209'16730 (192834'15710) clean ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.364.0:1198 2014-08-12 14:06:37.287339 2014-08-13 17:38:02.905235 7fe9bae887a0 20 read_log 194209'16731 (193305'16683) clean 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:1199 2014-08-12 14:06:37.293105 2014-08-13 17:38:02.905247 7fe9bae887a0 20 read_log 194209'16732 (192834'15669) clean 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.364.0:1200 2014-08-12 14:06:37.296336 2014-08-13 17:38:02.905259 7fe9bae887a0 20 read_log 194209'16733 (192888'16535) clean 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:1201 2014-08-12 14:06:37.296646 2014-08-13 17:38:02.905289 7fe9bae887a0 20 read_log 194209'16734 (192888'16354) clean 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:1202 2014-08-12 14:06:37.300218 2014-08-13 17:38:02.905302 7fe9bae887a0 20 read_log 194209'16735 (192834'15664) clean 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.364.0:1203 2014-08-12 14:06:37.303917 2014-08-13 17:38:02.905314 7fe9bae887a0 20 read_log 194209'16736 (192888'16567) clean 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:1204 2014-08-12 14:06:37.307110 2014-08-13 17:38:02.905326 7fe9bae887a0 20 read_log 194209'16737 (192888'16391) clean 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:1205 2014-08-12 14:06:37.310394 2014-08-13 17:38:02.905338 7fe9bae887a0 20 read_log 194209'16738 (194209'16693) delete b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.364.0:1206 2014-08-12 14:06:37.343051 2014-08-13 17:38:02.905350 7fe9bae887a0 20 read_log 194209'16739 (194209'16698) delete 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.364.0:1207 2014-08-12 14:06:37.343934 2014-08-13 17:38:02.905362 7fe9bae887a0 20 read_log 194209'16740 (194209'16699) delete 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.364.0:1208 2014-08-12 14:06:37.347381 2014-08-13 17:38:02.905374 7fe9bae887a0 20 read_log 194209'16741 (194209'16700) delete af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:1209 2014-08-12 14:06:37.347988 2014-08-13 17:38:02.905386 7fe9bae887a0 20 read_log 194209'16742 (194209'16701) delete c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:1210 2014-08-12 14:06:37.351416 2014-08-13 17:38:02.905398 7fe9bae887a0 20 read_log 194209'16743 (194209'16702) delete 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:1211 2014-08-12 14:06:37.351777 2014-08-13 17:38:02.905410 7fe9bae887a0 20 read_log 194209'16744 (192834'16128) clean 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.364.0:1212 2014-08-12 14:06:37.352008 2014-08-13 17:38:02.905422 7fe9bae887a0 20 read_log 194209'16745 (194209'16704) delete 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.364.0:1213 2014-08-12 14:06:37.370454 2014-08-13 17:38:02.905434 7fe9bae887a0 20 read_log 194209'16746 (194209'16705) delete 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.364.0:1214 2014-08-12 14:06:37.371196 2014-08-13 17:38:02.905455 7fe9bae887a0 20 read_log 194209'16747 (194209'16706) delete 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.364.0:1215 2014-08-12 14:06:37.374025 2014-08-13 17:38:02.905468 7fe9bae887a0 20 read_log 194209'16748 (194209'16707) delete f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:1216 2014-08-12 14:06:37.374401 2014-08-13 17:38:02.905481 7fe9bae887a0 20 read_log 194209'16749 (194209'16708) delete bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:1217 2014-08-12 14:06:37.375255 2014-08-13 17:38:02.905493 7fe9bae887a0 20 read_log 194209'16750 (194209'16709) delete 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:1218 2014-08-12 14:06:37.378531 2014-08-13 17:38:02.905505 7fe9bae887a0 20 read_log 194209'16751 (194209'16710) delete b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:1219 2014-08-12 14:06:37.378953 2014-08-13 17:38:02.905517 7fe9bae887a0 20 read_log 194209'16752 (194209'16711) delete 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:1220 2014-08-12 14:06:37.379584 2014-08-13 17:38:02.905529 7fe9bae887a0 20 read_log 194209'16753 (194209'16712) delete b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:1221 2014-08-12 14:06:37.406219 2014-08-13 17:38:02.905541 7fe9bae887a0 20 read_log 194209'16754 (194209'16713) delete c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.364.0:1222 2014-08-12 14:06:37.406939 2014-08-13 17:38:02.905552 7fe9bae887a0 20 read_log 194209'16755 (194209'16715) delete 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:1223 2014-08-12 14:06:37.407381 2014-08-13 17:38:02.905564 7fe9bae887a0 20 read_log 194209'16756 (194209'16716) delete 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:1224 2014-08-12 14:06:37.411157 2014-08-13 17:38:02.905576 7fe9bae887a0 20 read_log 194209'16757 (194209'16718) delete d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:1225 2014-08-12 14:06:37.411905 2014-08-13 17:38:02.905588 7fe9bae887a0 20 read_log 194209'16758 (194209'16719) delete dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.364.0:1226 2014-08-12 14:06:37.412222 2014-08-13 17:38:02.905600 7fe9bae887a0 20 read_log 194209'16759 (194209'16720) delete e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.364.0:1227 2014-08-12 14:06:37.415587 2014-08-13 17:38:02.905612 7fe9bae887a0 20 read_log 194209'16760 (194209'16721) delete 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.364.0:1228 2014-08-12 14:06:37.415907 2014-08-13 17:38:02.905624 7fe9bae887a0 20 read_log 194209'16761 (194209'16723) delete 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.364.0:1229 2014-08-12 14:06:37.416600 2014-08-13 17:38:02.905636 7fe9bae887a0 20 read_log 194209'16762 (194209'16724) delete 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:1230 2014-08-12 14:06:37.419609 2014-08-13 17:38:02.905648 7fe9bae887a0 20 read_log 194209'16763 (194209'16725) delete 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:1231 2014-08-12 14:06:37.419916 2014-08-13 17:38:02.905660 7fe9bae887a0 20 read_log 194209'16764 (194209'16726) delete 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.364.0:1232 2014-08-12 14:06:37.420789 2014-08-13 17:38:02.905672 7fe9bae887a0 20 read_log 194209'16765 (194209'16728) delete d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:1233 2014-08-12 14:06:37.423854 2014-08-13 17:38:02.905684 7fe9bae887a0 20 read_log 194209'16766 (194209'16729) delete 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:1234 2014-08-12 14:06:37.424157 2014-08-13 17:38:02.905703 7fe9bae887a0 20 read_log 194209'16767 (194209'16730) delete ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.364.0:1235 2014-08-12 14:06:37.424892 2014-08-13 17:38:02.905715 7fe9bae887a0 20 read_log 194209'16768 (194209'16731) delete 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:1236 2014-08-12 14:06:37.428018 2014-08-13 17:38:02.905727 7fe9bae887a0 20 read_log 194209'16769 (194209'16732) delete 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.364.0:1237 2014-08-12 14:06:37.428338 2014-08-13 17:38:02.905739 7fe9bae887a0 20 read_log 194209'16770 (194209'16734) delete 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:1238 2014-08-12 14:06:37.429069 2014-08-13 17:38:02.905751 7fe9bae887a0 20 read_log 194209'16771 (194209'16735) delete 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.364.0:1239 2014-08-12 14:06:37.432132 2014-08-13 17:38:02.905763 7fe9bae887a0 20 read_log 194209'16772 (194209'16737) delete 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:1240 2014-08-12 14:06:37.432537 2014-08-13 17:38:02.905775 7fe9bae887a0 20 read_log 194209'16773 (194209'16744) delete 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.364.0:1241 2014-08-12 14:06:37.432751 2014-08-13 17:38:02.905787 7fe9bae887a0 20 read_log 194209'16774 (194209'16722) delete b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:1583 2014-08-12 14:08:53.345943 2014-08-13 17:38:02.905799 7fe9bae887a0 20 read_log 194209'16775 (194209'16727) delete 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:1584 2014-08-12 14:08:53.346392 2014-08-13 17:38:02.905811 7fe9bae887a0 20 read_log 194209'16776 (194209'16733) delete 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:1585 2014-08-12 14:08:53.346829 2014-08-13 17:38:02.905823 7fe9bae887a0 20 read_log 194209'16777 (194209'16736) delete 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:1586 2014-08-12 14:08:53.347190 2014-08-13 17:38:02.905835 7fe9bae887a0 20 read_log 194209'16778 (194209'16694) delete 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:1587 2014-08-12 14:08:53.377509 2014-08-13 17:38:02.905847 7fe9bae887a0 20 read_log 194209'16779 (194209'16695) delete 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:1588 2014-08-12 14:08:53.377867 2014-08-13 17:38:02.905859 7fe9bae887a0 20 read_log 194209'16780 (194209'16696) delete 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.364.0:1589 2014-08-12 14:08:53.378234 2014-08-13 17:38:02.905870 7fe9bae887a0 20 read_log 194209'16781 (194209'16697) delete 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:1590 2014-08-12 14:08:53.378943 2014-08-13 17:38:02.905884 7fe9bae887a0 20 read_log 194209'16782 (194209'16703) delete a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:1591 2014-08-12 14:08:53.381510 2014-08-13 17:38:02.905896 7fe9bae887a0 20 read_log 194209'16783 (194209'16714) delete 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:1592 2014-08-12 14:08:53.382410 2014-08-13 17:38:02.905909 7fe9bae887a0 20 read_log 194209'16784 (194209'16717) delete 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.364.0:1593 2014-08-12 14:08:53.382739 2014-08-13 17:38:02.905921 7fe9bae887a0 20 read_log 194215'16785 (194215'16785) modify 1de/hit_set_24.1de_archive_2014-08-12 15:32:25.736820_2014-08-12 15:43:36.831465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:36.831705 2014-08-13 17:38:02.905941 7fe9bae887a0 20 read_log 194215'16786 (193305'16598) delete 1de/hit_set_24.1de_archive_2014-08-11 18:37:04.915039_2014-08-11 18:42:33.427516/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:36.831705 2014-08-13 17:38:02.905954 7fe9bae887a0 20 read_log 194215'16787 (192834'15607) clean 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by osd.364.0:2385 2014-08-12 15:43:36.834694 2014-08-13 17:38:02.905966 7fe9bae887a0 20 read_log 194215'16788 (194215'16787) delete 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.2174595.0:98983 0.000000 2014-08-13 17:38:02.905977 7fe9bae887a0 20 read_log 194215'16789 (192888'16571) clean a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by osd.364.0:2387 2014-08-12 15:43:36.852620 2014-08-13 17:38:02.905990 7fe9bae887a0 20 read_log 194215'16790 (194215'16789) delete a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.2174595.0:98985 0.000000 2014-08-13 17:38:02.906000 7fe9bae887a0 20 read_log 194215'16791 (192834'16210) clean bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by osd.364.0:2389 2014-08-12 15:43:36.861140 2014-08-13 17:38:02.906012 7fe9bae887a0 20 read_log 194215'16792 (194215'16791) delete bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.2174595.0:98987 0.000000 2014-08-13 17:38:02.906023 7fe9bae887a0 20 read_log 194215'16793 (192888'16361) clean 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by osd.364.0:2391 2014-08-12 15:43:36.867906 2014-08-13 17:38:02.906042 7fe9bae887a0 20 read_log 194215'16794 (194215'16793) delete 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.2174595.0:98989 0.000000 2014-08-13 17:38:02.906053 7fe9bae887a0 20 read_log 194215'16795 (193305'16673) clean 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by osd.364.0:2393 2014-08-12 15:43:36.874370 2014-08-13 17:38:02.906065 7fe9bae887a0 20 read_log 194215'16796 (194215'16795) delete 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.2174595.0:98991 0.000000 2014-08-13 17:38:02.906076 7fe9bae887a0 20 read_log 194215'16797 (192888'16591) clean 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by osd.364.0:2395 2014-08-12 15:43:36.882843 2014-08-13 17:38:02.906088 7fe9bae887a0 20 read_log 194215'16798 (194215'16797) delete 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.2174595.0:98993 0.000000 2014-08-13 17:38:02.906098 7fe9bae887a0 20 read_log 194215'16799 (192834'16100) clean 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by osd.364.0:2397 2014-08-12 15:43:36.889168 2014-08-13 17:38:02.906110 7fe9bae887a0 20 read_log 194215'16800 (194215'16799) delete 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.2174595.0:98995 0.000000 2014-08-13 17:38:02.906121 7fe9bae887a0 20 read_log 194215'16801 (192888'16569) clean 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by osd.364.0:2399 2014-08-12 15:43:36.895786 2014-08-13 17:38:02.906134 7fe9bae887a0 20 read_log 194215'16802 (194215'16801) delete 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.2174595.0:98997 0.000000 2014-08-13 17:38:02.906145 7fe9bae887a0 20 read_log 194215'16803 (193305'16682) clean 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by osd.364.0:2401 2014-08-12 15:43:36.902434 2014-08-13 17:38:02.906162 7fe9bae887a0 20 read_log 194215'16804 (194215'16803) delete 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.2174595.0:98999 0.000000 2014-08-13 17:38:02.906183 7fe9bae887a0 20 read_log 194215'16805 (192888'16570) clean c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by osd.364.0:2403 2014-08-12 15:43:36.908718 2014-08-13 17:38:02.906196 7fe9bae887a0 20 read_log 194215'16806 (194215'16805) delete c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.2174595.0:99001 0.000000 2014-08-13 17:38:02.906207 7fe9bae887a0 20 read_log 194215'16807 (192888'16395) clean 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by osd.364.0:2405 2014-08-12 15:43:36.915065 2014-08-13 17:38:02.906219 7fe9bae887a0 20 read_log 194215'16808 (194215'16807) delete 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.2174595.0:99003 0.000000 2014-08-13 17:38:02.906230 7fe9bae887a0 20 read_log 194215'16809 (193305'16667) clean 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by osd.364.0:2407 2014-08-12 15:43:36.921107 2014-08-13 17:38:02.906242 7fe9bae887a0 20 read_log 194215'16810 (194215'16809) delete 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.2174595.0:99005 0.000000 2014-08-13 17:38:02.906252 7fe9bae887a0 20 read_log 194215'16811 (192888'16368) clean 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by osd.364.0:2409 2014-08-12 15:43:36.927084 2014-08-13 17:38:02.906264 7fe9bae887a0 20 read_log 194215'16812 (194215'16811) delete 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.2174595.0:99007 0.000000 2014-08-13 17:38:02.906275 7fe9bae887a0 20 read_log 194215'16813 (193305'16670) clean 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by osd.364.0:2411 2014-08-12 15:43:36.933348 2014-08-13 17:38:02.906287 7fe9bae887a0 20 read_log 194215'16814 (194215'16813) delete 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.2174595.0:99009 0.000000 2014-08-13 17:38:02.906297 7fe9bae887a0 20 read_log 194215'16815 (192888'16353) clean 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by osd.364.0:2413 2014-08-12 15:43:36.939717 2014-08-13 17:38:02.906309 7fe9bae887a0 20 read_log 194215'16816 (194215'16815) delete 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.2174595.0:99011 0.000000 2014-08-13 17:38:02.906320 7fe9bae887a0 20 read_log 194215'16817 (192834'15660) clean 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by osd.364.0:2415 2014-08-12 15:43:36.945734 2014-08-13 17:38:02.906332 7fe9bae887a0 20 read_log 194215'16818 (194215'16817) delete 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.2174595.0:99013 0.000000 2014-08-13 17:38:02.906343 7fe9bae887a0 20 read_log 194215'16819 (192834'15604) clean 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by osd.364.0:2417 2014-08-12 15:43:36.952111 2014-08-13 17:38:02.906355 7fe9bae887a0 20 read_log 194215'16820 (194215'16819) delete 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.2174595.0:99015 0.000000 2014-08-13 17:38:02.906366 7fe9bae887a0 20 read_log 194215'16821 (192888'16547) clean 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by osd.364.0:2419 2014-08-12 15:43:36.958420 2014-08-13 17:38:02.906378 7fe9bae887a0 20 read_log 194215'16822 (194215'16821) delete 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.2174595.0:99017 0.000000 2014-08-13 17:38:02.906388 7fe9bae887a0 20 read_log 194215'16823 (192834'16217) clean f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by osd.364.0:2421 2014-08-12 15:43:36.964318 2014-08-13 17:38:02.906400 7fe9bae887a0 20 read_log 194215'16824 (194215'16823) delete f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.2174595.0:99019 0.000000 2014-08-13 17:38:02.906418 7fe9bae887a0 20 read_log 194215'16825 (192888'16589) clean 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by osd.364.0:2423 2014-08-12 15:43:36.970492 2014-08-13 17:38:02.906433 7fe9bae887a0 20 read_log 194215'16826 (194215'16825) delete 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.2174595.0:99021 0.000000 2014-08-13 17:38:02.906463 7fe9bae887a0 20 read_log 194215'16827 (192834'15644) clean d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by osd.364.0:2425 2014-08-12 15:43:36.976413 2014-08-13 17:38:02.906479 7fe9bae887a0 20 read_log 194215'16828 (194215'16827) delete d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.2174595.0:99023 0.000000 2014-08-13 17:38:02.906490 7fe9bae887a0 20 read_log 194215'16829 (192834'15686) clean 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by osd.364.0:2427 2014-08-12 15:43:36.982404 2014-08-13 17:38:02.906503 7fe9bae887a0 20 read_log 194215'16830 (194215'16829) delete 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.2174595.0:99025 0.000000 2014-08-13 17:38:02.906513 7fe9bae887a0 20 read_log 194215'16831 (192834'16116) clean b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by osd.364.0:2429 2014-08-12 15:43:36.989617 2014-08-13 17:38:02.906525 7fe9bae887a0 20 read_log 194215'16832 (194215'16831) delete b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.2174595.0:99027 0.000000 2014-08-13 17:38:02.906536 7fe9bae887a0 20 read_log 194215'16833 (192834'16086) clean 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by osd.364.0:2431 2014-08-12 15:43:36.997175 2014-08-13 17:38:02.906548 7fe9bae887a0 20 read_log 194215'16834 (194215'16833) delete 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.2174595.0:99029 0.000000 2014-08-13 17:38:02.906559 7fe9bae887a0 20 read_log 194215'16835 (192834'15583) clean b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by osd.364.0:2433 2014-08-12 15:43:37.003248 2014-08-13 17:38:02.906571 7fe9bae887a0 20 read_log 194215'16836 (194215'16835) delete b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.2174595.0:99031 0.000000 2014-08-13 17:38:02.906582 7fe9bae887a0 20 read_log 194215'16837 (193305'16680) clean 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by osd.364.0:2435 2014-08-12 15:43:37.009512 2014-08-13 17:38:02.906594 7fe9bae887a0 20 read_log 194215'16838 (194215'16837) delete 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.2174595.0:99033 0.000000 2014-08-13 17:38:02.906605 7fe9bae887a0 20 read_log 194215'16839 (192888'16578) clean 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by osd.364.0:2437 2014-08-12 15:43:37.026721 2014-08-13 17:38:02.906617 7fe9bae887a0 20 read_log 194215'16840 (194215'16839) delete 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.2174595.0:99035 0.000000 2014-08-13 17:38:02.906628 7fe9bae887a0 20 read_log 194215'16841 (192888'16344) clean 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by osd.364.0:2439 2014-08-12 15:43:37.050496 2014-08-13 17:38:02.906639 7fe9bae887a0 20 read_log 194215'16842 (194215'16841) delete 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.2174595.0:99037 0.000000 2014-08-13 17:38:02.906650 7fe9bae887a0 20 read_log 194215'16843 (192834'15625) clean 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by osd.364.0:2441 2014-08-12 15:43:37.057871 2014-08-13 17:38:02.906662 7fe9bae887a0 20 read_log 194215'16844 (194215'16843) delete 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.2174595.0:99039 0.000000 2014-08-13 17:38:02.906696 7fe9bae887a0 20 read_log 194215'16845 (192834'16085) clean 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by osd.364.0:2443 2014-08-12 15:43:37.065393 2014-08-13 17:38:02.906712 7fe9bae887a0 20 read_log 194215'16846 (194215'16845) delete 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.2174595.0:99041 0.000000 2014-08-13 17:38:02.906723 7fe9bae887a0 20 read_log 194215'16847 (192888'16351) clean e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by osd.364.0:2445 2014-08-12 15:43:37.072400 2014-08-13 17:38:02.906735 7fe9bae887a0 20 read_log 194215'16848 (194215'16847) delete e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.2174595.0:99043 0.000000 2014-08-13 17:38:02.906746 7fe9bae887a0 20 read_log 194215'16849 (193305'16665) clean 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by osd.364.0:2447 2014-08-12 15:43:37.079409 2014-08-13 17:38:02.906758 7fe9bae887a0 20 read_log 194215'16850 (194215'16849) delete 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.2174595.0:99045 0.000000 2014-08-13 17:38:02.906768 7fe9bae887a0 20 read_log 194215'16851 (193305'16691) clean 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by osd.364.0:2449 2014-08-12 15:43:37.086554 2014-08-13 17:38:02.906781 7fe9bae887a0 20 read_log 194215'16852 (194215'16851) delete 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.2174595.0:99047 0.000000 2014-08-13 17:38:02.906791 7fe9bae887a0 20 read_log 194215'16853 (192834'16064) clean 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by osd.364.0:2451 2014-08-12 15:43:37.093816 2014-08-13 17:38:02.906803 7fe9bae887a0 20 read_log 194215'16854 (194215'16853) delete 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.2174595.0:99049 0.000000 2014-08-13 17:38:02.906814 7fe9bae887a0 20 read_log 194215'16855 (192888'16548) clean 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by osd.364.0:2453 2014-08-12 15:43:37.112037 2014-08-13 17:38:02.906826 7fe9bae887a0 20 read_log 194215'16856 (194215'16855) delete 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.2174595.0:99051 0.000000 2014-08-13 17:38:02.906837 7fe9bae887a0 20 read_log 194215'16857 (192834'15712) clean 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by osd.364.0:2455 2014-08-12 15:43:37.119283 2014-08-13 17:38:02.906849 7fe9bae887a0 20 read_log 194215'16858 (194215'16857) delete 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.2174595.0:99053 0.000000 2014-08-13 17:38:02.906859 7fe9bae887a0 20 read_log 194215'16859 (192834'16228) clean 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by osd.364.0:2457 2014-08-12 15:43:37.126415 2014-08-13 17:38:02.906871 7fe9bae887a0 20 read_log 194215'16860 (194215'16859) delete 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.2174595.0:99055 0.000000 2014-08-13 17:38:02.906882 7fe9bae887a0 20 read_log 194215'16861 (193305'16677) clean 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by osd.364.0:2459 2014-08-12 15:43:37.133499 2014-08-13 17:38:02.906894 7fe9bae887a0 20 read_log 194215'16862 (194215'16861) delete 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.2174595.0:99057 0.000000 2014-08-13 17:38:02.906904 7fe9bae887a0 20 read_log 194215'16863 (192888'16378) clean 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by osd.364.0:2461 2014-08-12 15:43:37.166335 2014-08-13 17:38:02.906916 7fe9bae887a0 20 read_log 194215'16864 (194215'16863) delete 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.2174595.0:99059 0.000000 2014-08-13 17:38:02.906936 7fe9bae887a0 20 read_log 194215'16865 (192834'16192) clean 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by osd.364.0:2463 2014-08-12 15:43:37.173462 2014-08-13 17:38:02.906949 7fe9bae887a0 20 read_log 194215'16866 (194215'16865) delete 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.2174595.0:99061 0.000000 2014-08-13 17:38:02.906960 7fe9bae887a0 20 read_log 194215'16867 (192888'16374) clean 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by osd.364.0:2465 2014-08-12 15:43:37.180551 2014-08-13 17:38:02.906972 7fe9bae887a0 20 read_log 194215'16868 (194215'16867) delete 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.2174595.0:99063 0.000000 2014-08-13 17:38:02.906982 7fe9bae887a0 20 read_log 194215'16869 (192888'16554) clean 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by osd.364.0:2467 2014-08-12 15:43:37.208816 2014-08-13 17:38:02.906995 7fe9bae887a0 20 read_log 194215'16870 (194215'16869) delete 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.2174595.0:99065 0.000000 2014-08-13 17:38:02.907005 7fe9bae887a0 20 read_log 194215'16871 (193305'16666) clean a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by osd.364.0:2469 2014-08-12 15:43:37.216012 2014-08-13 17:38:02.907017 7fe9bae887a0 20 read_log 194215'16872 (194215'16871) delete a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.2174595.0:99067 0.000000 2014-08-13 17:38:02.907027 7fe9bae887a0 20 read_log 194215'16873 (192888'16349) clean 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by osd.364.0:2471 2014-08-12 15:43:37.223279 2014-08-13 17:38:02.907039 7fe9bae887a0 20 read_log 194215'16874 (194215'16873) delete 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.2174595.0:99069 0.000000 2014-08-13 17:38:02.907050 7fe9bae887a0 20 read_log 194215'16875 (192888'16362) clean 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by osd.364.0:2473 2014-08-12 15:43:37.230084 2014-08-13 17:38:02.907061 7fe9bae887a0 20 read_log 194215'16876 (194215'16875) delete 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.2174595.0:99071 0.000000 2014-08-13 17:38:02.907072 7fe9bae887a0 20 read_log 194215'16877 (192888'16595) clean 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by osd.364.0:2475 2014-08-12 15:43:37.237193 2014-08-13 17:38:02.907084 7fe9bae887a0 20 read_log 194215'16878 (194215'16877) delete 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.2174595.0:99073 0.000000 2014-08-13 17:38:02.907094 7fe9bae887a0 20 read_log 194215'16879 (192834'15582) clean 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by osd.364.0:2477 2014-08-12 15:43:37.244398 2014-08-13 17:38:02.907106 7fe9bae887a0 20 read_log 194215'16880 (194215'16879) delete 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by client.2174595.0:99075 0.000000 2014-08-13 17:38:02.907117 7fe9bae887a0 20 read_log 194215'16881 (193305'16669) clean 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by osd.364.0:2479 2014-08-12 15:43:37.250970 2014-08-13 17:38:02.907129 7fe9bae887a0 20 read_log 194215'16882 (194215'16881) delete 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.2174595.0:99077 0.000000 2014-08-13 17:38:02.907139 7fe9bae887a0 20 read_log 194215'16883 (192834'16104) clean e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by osd.364.0:2481 2014-08-12 15:43:37.258693 2014-08-13 17:38:02.907151 7fe9bae887a0 20 read_log 194215'16884 (194215'16883) delete e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.2174595.0:99079 0.000000 2014-08-13 17:38:02.907174 7fe9bae887a0 20 read_log 194215'16885 (193305'16687) clean e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by osd.364.0:2483 2014-08-12 15:43:37.265824 2014-08-13 17:38:02.907187 7fe9bae887a0 20 read_log 194215'16886 (194215'16885) delete e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.2174595.0:99081 0.000000 2014-08-13 17:38:02.907198 7fe9bae887a0 20 read_log 194215'16887 (193305'16685) clean 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by osd.364.0:2485 2014-08-12 15:43:37.273453 2014-08-13 17:38:02.907210 7fe9bae887a0 20 read_log 194215'16888 (194215'16887) delete 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.2174595.0:99083 0.000000 2014-08-13 17:38:02.907221 7fe9bae887a0 20 read_log 194215'16889 (192888'16577) clean d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by osd.364.0:2487 2014-08-12 15:43:37.281077 2014-08-13 17:38:02.907235 7fe9bae887a0 20 read_log 194215'16890 (194215'16889) delete d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.2174595.0:99085 0.000000 2014-08-13 17:38:02.907245 7fe9bae887a0 20 read_log 194215'16891 (192834'16069) clean c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by osd.364.0:2489 2014-08-12 15:43:37.288161 2014-08-13 17:38:02.907261 7fe9bae887a0 20 read_log 194215'16892 (194215'16891) delete c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.2174595.0:99087 0.000000 2014-08-13 17:38:02.907271 7fe9bae887a0 20 read_log 194215'16893 (192834'16096) clean bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by osd.364.0:2491 2014-08-12 15:43:37.295189 2014-08-13 17:38:02.907284 7fe9bae887a0 20 read_log 194215'16894 (194215'16893) delete bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.2174595.0:99089 0.000000 2014-08-13 17:38:02.907295 7fe9bae887a0 20 read_log 194215'16895 (193305'16684) clean 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by osd.364.0:2493 2014-08-12 15:43:37.302203 2014-08-13 17:38:02.907307 7fe9bae887a0 20 read_log 194215'16896 (194215'16895) delete 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.2174595.0:99091 0.000000 2014-08-13 17:38:02.907317 7fe9bae887a0 20 read_log 194215'16897 (192888'16584) clean 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by osd.364.0:2495 2014-08-12 15:43:37.309418 2014-08-13 17:38:02.907329 7fe9bae887a0 20 read_log 194215'16898 (194215'16897) delete 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.2174595.0:99093 0.000000 2014-08-13 17:38:02.907340 7fe9bae887a0 20 read_log 194215'16899 (192888'16552) clean 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by osd.364.0:2497 2014-08-12 15:43:37.316332 2014-08-13 17:38:02.907352 7fe9bae887a0 20 read_log 194215'16900 (194215'16899) delete 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.2174595.0:99095 0.000000 2014-08-13 17:38:02.907363 7fe9bae887a0 20 read_log 194215'16901 (192888'16363) clean e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by osd.364.0:2499 2014-08-12 15:43:37.323240 2014-08-13 17:38:02.907375 7fe9bae887a0 20 read_log 194215'16902 (194215'16901) delete e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.2174595.0:99097 0.000000 2014-08-13 17:38:02.907385 7fe9bae887a0 20 read_log 194215'16903 (193305'16692) clean d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by osd.364.0:2501 2014-08-12 15:43:37.330019 2014-08-13 17:38:02.907399 7fe9bae887a0 20 read_log 194215'16904 (194215'16903) delete d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.2174595.0:99099 0.000000 2014-08-13 17:38:02.907416 7fe9bae887a0 20 read_log 194215'16905 (192834'16103) clean 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by osd.364.0:2503 2014-08-12 15:43:37.337442 2014-08-13 17:38:02.907429 7fe9bae887a0 20 read_log 194215'16906 (194215'16905) delete 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.2174595.0:99101 0.000000 2014-08-13 17:38:02.907440 7fe9bae887a0 20 read_log 194215'16907 (192888'16590) clean eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by osd.364.0:2505 2014-08-12 15:43:37.344871 2014-08-13 17:38:02.907451 7fe9bae887a0 20 read_log 194215'16908 (194215'16907) delete eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.2174595.0:99103 0.000000 2014-08-13 17:38:02.907462 7fe9bae887a0 20 read_log 194215'16909 (192834'15682) clean 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by osd.364.0:2507 2014-08-12 15:43:37.352010 2014-08-13 17:38:02.907474 7fe9bae887a0 20 read_log 194215'16910 (194215'16909) delete 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.2174595.0:99105 0.000000 2014-08-13 17:38:02.907484 7fe9bae887a0 20 read_log 194215'16911 (192834'16211) clean 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by osd.364.0:2509 2014-08-12 15:43:37.358892 2014-08-13 17:38:02.907496 7fe9bae887a0 20 read_log 194215'16912 (194215'16911) delete 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.2174595.0:99107 0.000000 2014-08-13 17:38:02.907507 7fe9bae887a0 20 read_log 194215'16913 (192888'16387) clean e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by osd.364.0:2511 2014-08-12 15:43:37.365836 2014-08-13 17:38:02.907520 7fe9bae887a0 20 read_log 194215'16914 (194215'16913) delete e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.2174595.0:99109 0.000000 2014-08-13 17:38:02.907530 7fe9bae887a0 20 read_log 194215'16915 (192888'16383) clean de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by osd.364.0:2513 2014-08-12 15:43:37.372994 2014-08-13 17:38:02.907542 7fe9bae887a0 20 read_log 194215'16916 (194215'16915) delete de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.2174595.0:99111 0.000000 2014-08-13 17:38:02.907553 7fe9bae887a0 20 read_log 194215'16917 (192834'16099) clean 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by osd.364.0:2515 2014-08-12 15:43:37.380476 2014-08-13 17:38:02.907565 7fe9bae887a0 20 read_log 194215'16918 (194215'16917) delete 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.2174595.0:99113 0.000000 2014-08-13 17:38:02.907575 7fe9bae887a0 20 read_log 194215'16919 (192834'15685) clean cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by osd.364.0:2517 2014-08-12 15:43:37.402787 2014-08-13 17:38:02.907589 7fe9bae887a0 20 read_log 194215'16920 (194215'16919) delete cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.2174595.0:99115 0.000000 2014-08-13 17:38:02.907600 7fe9bae887a0 20 read_log 194215'16921 (192888'16360) clean 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by osd.364.0:2519 2014-08-12 15:43:37.410434 2014-08-13 17:38:02.907612 7fe9bae887a0 20 read_log 194215'16922 (194215'16921) delete 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.2174595.0:99117 0.000000 2014-08-13 17:38:02.907622 7fe9bae887a0 20 read_log 194215'16923 (193305'16689) clean 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by osd.364.0:2521 2014-08-12 15:43:37.417720 2014-08-13 17:38:02.907634 7fe9bae887a0 20 read_log 194215'16924 (194215'16923) delete 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.2174595.0:99119 0.000000 2014-08-13 17:38:02.907654 7fe9bae887a0 20 read_log 194215'16925 (193305'16672) clean d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by osd.364.0:2523 2014-08-12 15:43:37.424741 2014-08-13 17:38:02.907667 7fe9bae887a0 20 read_log 194215'16926 (194215'16925) delete d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.2174595.0:99121 0.000000 2014-08-13 17:38:02.907677 7fe9bae887a0 20 read_log 194215'16927 (193305'16662) clean 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by osd.364.0:2525 2014-08-12 15:43:37.431284 2014-08-13 17:38:02.907689 7fe9bae887a0 20 read_log 194215'16928 (194215'16927) delete 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.2174595.0:99123 0.000000 2014-08-13 17:38:02.907700 7fe9bae887a0 20 read_log 194215'16929 (192888'16593) clean 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by osd.364.0:2527 2014-08-12 15:43:37.437420 2014-08-13 17:38:02.907712 7fe9bae887a0 20 read_log 194215'16930 (194215'16929) delete 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.2174595.0:99125 0.000000 2014-08-13 17:38:02.907723 7fe9bae887a0 20 read_log 194215'16931 (193305'16671) clean 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by osd.364.0:2529 2014-08-12 15:43:37.443931 2014-08-13 17:38:02.907735 7fe9bae887a0 20 read_log 194215'16932 (194215'16931) delete 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.2174595.0:99127 0.000000 2014-08-13 17:38:02.907746 7fe9bae887a0 20 read_log 194215'16933 (192834'16067) clean 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by osd.364.0:2531 2014-08-12 15:43:37.451078 2014-08-13 17:38:02.907758 7fe9bae887a0 20 read_log 194215'16934 (194215'16933) delete 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.2174595.0:99129 0.000000 2014-08-13 17:38:02.907768 7fe9bae887a0 20 read_log 194215'16935 (192888'16543) clean ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by osd.364.0:2533 2014-08-12 15:43:37.458389 2014-08-13 17:38:02.907809 7fe9bae887a0 20 read_log 194215'16936 (194215'16935) delete ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.2174595.0:99131 0.000000 2014-08-13 17:38:02.907823 7fe9bae887a0 20 read_log 194215'16937 (192834'15596) clean 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by osd.364.0:2535 2014-08-12 15:43:37.464702 2014-08-13 17:38:02.907836 7fe9bae887a0 20 read_log 194215'16938 (194215'16937) delete 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.2174595.0:99133 0.000000 2014-08-13 17:38:02.907846 7fe9bae887a0 20 read_log 194215'16939 (192888'16358) clean b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by osd.364.0:2537 2014-08-12 15:43:37.471314 2014-08-13 17:38:02.907858 7fe9bae887a0 20 read_log 194215'16940 (194215'16939) delete b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.2174595.0:99135 0.000000 2014-08-13 17:38:02.907869 7fe9bae887a0 20 read_log 194215'16941 (192888'16546) clean cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by osd.364.0:2539 2014-08-12 15:43:37.478570 2014-08-13 17:38:02.907881 7fe9bae887a0 20 read_log 194215'16942 (194215'16941) delete cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.2174595.0:99137 0.000000 2014-08-13 17:38:02.907892 7fe9bae887a0 20 read_log 194215'16943 (193305'16690) clean 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by osd.364.0:2541 2014-08-12 15:43:37.485702 2014-08-13 17:38:02.907904 7fe9bae887a0 20 read_log 194215'16944 (194215'16943) delete 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.2174595.0:99139 0.000000 2014-08-13 17:38:02.907922 7fe9bae887a0 20 read_log 194215'16945 (192888'16394) clean 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by osd.364.0:2543 2014-08-12 15:43:37.493129 2014-08-13 17:38:02.907934 7fe9bae887a0 20 read_log 194215'16946 (194215'16945) delete 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.2174595.0:99141 0.000000 2014-08-13 17:38:02.907945 7fe9bae887a0 20 read_log 194215'16947 (192888'16373) clean 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by osd.364.0:2545 2014-08-12 15:43:37.499872 2014-08-13 17:38:02.907957 7fe9bae887a0 20 read_log 194215'16948 (194215'16947) delete 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.2174595.0:99143 0.000000 2014-08-13 17:38:02.907968 7fe9bae887a0 20 read_log 194215'16949 (192888'16566) clean ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by osd.364.0:2547 2014-08-12 15:43:37.525331 2014-08-13 17:38:02.907980 7fe9bae887a0 20 read_log 194215'16950 (194215'16949) delete ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.2174595.0:99145 0.000000 2014-08-13 17:38:02.907991 7fe9bae887a0 20 read_log 194215'16951 (192834'15678) clean ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by osd.364.0:2549 2014-08-12 15:43:37.533470 2014-08-13 17:38:02.908003 7fe9bae887a0 20 read_log 194215'16952 (194215'16951) delete ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.2174595.0:99147 0.000000 2014-08-13 17:38:02.908014 7fe9bae887a0 20 read_log 194215'16953 (192834'16133) clean 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by osd.364.0:2551 2014-08-12 15:43:37.540724 2014-08-13 17:38:02.908025 7fe9bae887a0 20 read_log 194215'16954 (194215'16953) delete 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.2174595.0:99149 0.000000 2014-08-13 17:38:02.908036 7fe9bae887a0 20 read_log 194215'16955 (192888'16376) clean 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by osd.364.0:2553 2014-08-12 15:43:37.547755 2014-08-13 17:38:02.908048 7fe9bae887a0 20 read_log 194215'16956 (194215'16955) delete 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.2174595.0:99151 0.000000 2014-08-13 17:38:02.908058 7fe9bae887a0 20 read_log 194215'16957 (192888'16377) clean 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by osd.364.0:2555 2014-08-12 15:43:37.555248 2014-08-13 17:38:02.908070 7fe9bae887a0 20 read_log 194215'16958 (194215'16957) delete 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.2174595.0:99153 0.000000 2014-08-13 17:38:02.908083 7fe9bae887a0 20 read_log 194215'16959 (193305'16674) clean 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by osd.364.0:2557 2014-08-12 15:43:37.562846 2014-08-13 17:38:02.908095 7fe9bae887a0 20 read_log 194215'16960 (194215'16959) delete 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.2174595.0:99155 0.000000 2014-08-13 17:38:02.908106 7fe9bae887a0 20 read_log 194215'16961 (192834'16101) clean e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by osd.364.0:2559 2014-08-12 15:43:37.570539 2014-08-13 17:38:02.908118 7fe9bae887a0 20 read_log 194215'16962 (194215'16961) delete e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.2174595.0:99157 0.000000 2014-08-13 17:38:02.908129 7fe9bae887a0 20 read_log 194215'16963 (192834'16150) clean 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by osd.364.0:2561 2014-08-12 15:43:37.579861 2014-08-13 17:38:02.908141 7fe9bae887a0 20 read_log 194215'16964 (194215'16963) delete 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.2174595.0:99159 0.000000 2014-08-13 17:38:02.908165 7fe9bae887a0 20 read_log 194215'16965 (192834'15680) clean 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by osd.364.0:2563 2014-08-12 15:43:37.586744 2014-08-13 17:38:02.908178 7fe9bae887a0 20 read_log 194215'16966 (194215'16965) delete 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.2174595.0:99161 0.000000 2014-08-13 17:38:02.908189 7fe9bae887a0 20 read_log 194215'16967 (192834'15679) clean de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by osd.364.0:2565 2014-08-12 15:43:37.593397 2014-08-13 17:38:02.908201 7fe9bae887a0 20 read_log 194215'16968 (194215'16967) delete de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.2174595.0:99163 0.000000 2014-08-13 17:38:02.908213 7fe9bae887a0 20 read_log 194215'16969 (192834'16184) clean ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by osd.364.0:2567 2014-08-12 15:43:37.601086 2014-08-13 17:38:02.908225 7fe9bae887a0 20 read_log 194215'16970 (194215'16969) delete ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.2174595.0:99165 0.000000 2014-08-13 17:38:02.908236 7fe9bae887a0 20 read_log 194215'16971 (192834'16122) clean e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by osd.364.0:2569 2014-08-12 15:43:37.608288 2014-08-13 17:38:02.908248 7fe9bae887a0 20 read_log 194215'16972 (194215'16971) delete e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.2174595.0:99167 0.000000 2014-08-13 17:38:02.908259 7fe9bae887a0 20 read_log 194215'16973 (194215'16973) modify 1de/hit_set_24.1de_archive_2014-08-12 15:43:36.831710_2014-08-12 17:24:44.125385/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.125667 2014-08-13 17:38:02.908271 7fe9bae887a0 20 read_log 194215'16974 (194215'16785) delete 1de/hit_set_24.1de_archive_2014-08-12 15:32:25.736820_2014-08-12 15:43:36.831465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.125667 2014-08-13 17:38:02.908283 7fe9bae887a0 20 read_log 194215'16975 (0'0) promote f036cdde/benchmark_data_storage0111-ib.pocstor_34628_object1089/head//24 by osd.364.0:3492 2014-08-12 17:24:44.131079 2014-08-13 17:38:02.908295 7fe9bae887a0 20 read_log 194215'16976 (194215'16975) modify f036cdde/benchmark_data_storage0111-ib.pocstor_34628_object1089/head//24 by client.2193759.0:1090 2014-08-12 17:24:44.118362 2014-08-13 17:38:02.908307 7fe9bae887a0 20 read_log 194370'16977 (0'0) promote 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by osd.364.0:3558 2014-08-12 20:30:28.204970 2014-08-13 17:38:02.908319 7fe9bae887a0 20 read_log 194370'16978 (194370'16977) modify 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by client.1508531.0:3700 2014-08-12 20:30:28.195281 2014-08-13 17:38:02.908333 7fe9bae887a0 20 read_log 194370'16979 (0'0) promote db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by osd.364.0:3566 2014-08-12 20:30:29.883521 2014-08-13 17:38:02.908345 7fe9bae887a0 20 read_log 194370'16980 (194370'16979) modify db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by client.1508531.0:4154 2014-08-12 20:30:29.875338 2014-08-13 17:38:02.908358 7fe9bae887a0 20 read_log 194370'16981 (0'0) promote d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by osd.364.0:3574 2014-08-12 20:30:31.498401 2014-08-13 17:38:02.908370 7fe9bae887a0 20 read_log 194370'16982 (194370'16981) modify d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by client.1508531.0:4570 2014-08-12 20:30:31.488804 2014-08-13 17:38:02.908382 7fe9bae887a0 20 read_log 194370'16983 (0'0) promote 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by osd.364.0:3576 2014-08-12 20:30:31.875612 2014-08-13 17:38:02.908419 7fe9bae887a0 20 read_log 194370'16984 (194370'16983) modify 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by client.1508531.0:4665 2014-08-12 20:30:31.867906 2014-08-13 17:38:02.908433 7fe9bae887a0 20 read_log 194370'16985 (0'0) promote b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by osd.364.0:3578 2014-08-12 20:30:32.470251 2014-08-13 17:38:02.908445 7fe9bae887a0 20 read_log 194370'16986 (194370'16985) modify b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by client.1508531.0:4822 2014-08-12 20:30:32.463242 2014-08-13 17:38:02.908457 7fe9bae887a0 20 read_log 194370'16987 (0'0) promote df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by osd.364.0:3678 2014-08-12 20:31:08.658202 2014-08-13 17:38:02.908470 7fe9bae887a0 20 read_log 194370'16988 (194370'16987) modify df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by client.1508531.0:11557 2014-08-12 20:31:08.650240 2014-08-13 17:38:02.908482 7fe9bae887a0 20 read_log 194370'16989 (0'0) promote 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by osd.364.0:3682 2014-08-12 20:31:09.954591 2014-08-13 17:38:02.908494 7fe9bae887a0 20 read_log 194370'16990 (194370'16989) modify 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by client.1508531.0:11778 2014-08-12 20:31:09.947609 2014-08-13 17:38:02.908506 7fe9bae887a0 20 read_log 194370'16991 (0'0) promote 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by osd.364.0:3692 2014-08-12 20:31:11.065603 2014-08-13 17:38:02.908518 7fe9bae887a0 20 read_log 194370'16992 (194370'16991) modify 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by client.1508531.0:11959 2014-08-12 20:31:11.056711 2014-08-13 17:38:02.908530 7fe9bae887a0 20 read_log 194370'16993 (0'0) promote f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by osd.364.0:3696 2014-08-12 20:31:11.584470 2014-08-13 17:38:02.908545 7fe9bae887a0 20 read_log 194370'16994 (194370'16993) modify f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by client.1508531.0:12031 2014-08-12 20:31:11.574016 2014-08-13 17:38:02.908557 7fe9bae887a0 20 read_log 194370'16995 (0'0) promote 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by osd.364.0:3698 2014-08-12 20:31:11.605710 2014-08-13 17:38:02.908569 7fe9bae887a0 20 read_log 194370'16996 (194370'16995) modify 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by client.1508531.0:12034 2014-08-12 20:31:11.595130 2014-08-13 17:38:02.908581 7fe9bae887a0 20 read_log 194370'16997 (0'0) promote 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by osd.364.0:3708 2014-08-12 20:31:16.871592 2014-08-13 17:38:02.908593 7fe9bae887a0 20 read_log 194370'16998 (194370'16997) modify 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by client.1508531.0:12736 2014-08-12 20:31:16.863845 2014-08-13 17:38:02.908605 7fe9bae887a0 20 read_log 194370'16999 (0'0) promote 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by osd.364.0:3710 2014-08-12 20:31:22.856582 2014-08-13 17:38:02.908617 7fe9bae887a0 20 read_log 194370'17000 (194370'16999) modify 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by client.1508531.0:13364 2014-08-12 20:31:22.848815 2014-08-13 17:38:02.908629 7fe9bae887a0 20 read_log 194370'17001 (0'0) promote 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by osd.364.0:3716 2014-08-12 20:31:25.947380 2014-08-13 17:38:02.908641 7fe9bae887a0 20 read_log 194370'17002 (194370'17001) modify 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by client.1508531.0:13689 2014-08-12 20:31:25.940307 2014-08-13 17:38:02.908653 7fe9bae887a0 20 read_log 194370'17003 (0'0) promote 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by osd.364.0:3722 2014-08-12 20:31:32.135148 2014-08-13 17:38:02.908672 7fe9bae887a0 20 read_log 194370'17004 (194370'17003) modify 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by client.1508531.0:13997 2014-08-12 20:31:32.126438 2014-08-13 17:38:02.908685 7fe9bae887a0 20 read_log 194377'17005 (0'0) promote 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by osd.364.0:3750 2014-08-12 20:33:02.971956 2014-08-13 17:38:02.908697 7fe9bae887a0 20 read_log 194377'17006 (194377'17005) modify 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by client.2231190.0:1440 2014-08-12 20:33:02.964206 2014-08-13 17:38:02.908709 7fe9bae887a0 20 read_log 194383'17007 (0'0) promote 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by osd.364.0:3846 2014-08-12 20:34:24.795228 2014-08-13 17:38:02.908721 7fe9bae887a0 20 read_log 194383'17008 (194383'17007) modify 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by client.2231190.0:7355 2014-08-12 20:34:24.786699 2014-08-13 17:38:02.908733 7fe9bae887a0 20 read_log 194383'17009 (0'0) promote 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by osd.364.0:3850 2014-08-12 20:34:29.642453 2014-08-13 17:38:02.908745 7fe9bae887a0 20 read_log 194383'17010 (194383'17009) modify 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by client.2231190.0:7532 2014-08-12 20:34:29.634334 2014-08-13 17:38:02.908757 7fe9bae887a0 20 read_log 194393'17011 (194393'17011) modify 1de/hit_set_24.1de_archive_2014-08-12 20:37:09.033964_2014-08-12 20:49:27.577081/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.577309 2014-08-13 17:38:02.908769 7fe9bae887a0 20 read_log 194393'17012 (194215'16973) delete 1de/hit_set_24.1de_archive_2014-08-12 15:43:36.831710_2014-08-12 17:24:44.125385/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.577309 2014-08-13 17:38:02.908782 7fe9bae887a0 20 read_log 194393'17013 (0'0) promote ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by osd.364.0:3856 2014-08-12 20:49:27.580084 2014-08-13 17:38:02.908794 7fe9bae887a0 20 read_log 194393'17014 (194393'17013) modify ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by client.1064182.0:425 2014-08-12 20:49:27.571281 2014-08-13 17:38:02.908806 7fe9bae887a0 20 read_log 194416'17015 (194416'17015) modify 1de/hit_set_24.1de_archive_2014-08-12 20:49:27.577314_2014-08-13 15:28:31.498257/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 17:38:02.908818 7fe9bae887a0 20 read_log 194416'17016 (194393'17011) delete 1de/hit_set_24.1de_archive_2014-08-12 20:37:09.033964_2014-08-12 20:49:27.577081/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 17:38:02.908830 7fe9bae887a0 20 read_log 194416'17017 (0'0) promote 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by osd.364.0:4024 2014-08-13 15:28:31.501740 2014-08-13 17:38:02.908842 7fe9bae887a0 20 read_log 194416'17018 (194416'17017) modify 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by client.2441958.0:310 2014-08-13 15:28:31.491025 2014-08-13 17:38:02.908854 7fe9bae887a0 20 read_log 194416'17019 (0'0) promote 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by osd.364.0:4043 2014-08-13 15:28:35.426964 2014-08-13 17:38:02.908866 7fe9bae887a0 20 read_log 194416'17020 (194416'17019) modify 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by client.2441958.0:1240 2014-08-13 15:28:35.419433 2014-08-13 17:38:02.908878 7fe9bae887a0 20 read_log 194416'17021 (0'0) promote b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by osd.364.0:4070 2014-08-13 15:28:41.285953 2014-08-13 17:38:02.908890 7fe9bae887a0 20 read_log 194416'17022 (194416'17021) modify b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by client.2441958.0:2788 2014-08-13 15:28:41.277463 2014-08-13 17:38:02.908909 7fe9bae887a0 20 read_log 194416'17023 (0'0) promote da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by osd.364.0:4074 2014-08-13 15:28:41.565171 2014-08-13 17:38:02.908922 7fe9bae887a0 20 read_log 194416'17024 (194416'17023) modify da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by client.2441958.0:2863 2014-08-13 15:28:41.558080 2014-08-13 17:38:02.908934 7fe9bae887a0 20 read_log 194416'17025 (0'0) promote 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by osd.364.0:4086 2014-08-13 15:28:43.716695 2014-08-13 17:38:02.908946 7fe9bae887a0 20 read_log 194416'17026 (194416'17025) modify 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by client.2441958.0:3435 2014-08-13 15:28:43.708507 2014-08-13 17:38:02.908958 7fe9bae887a0 20 read_log 194416'17027 (0'0) promote 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by osd.364.0:4126 2014-08-13 15:28:53.917553 2014-08-13 17:38:02.908970 7fe9bae887a0 20 read_log 194416'17028 (194416'17027) modify 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by client.2441958.0:6041 2014-08-13 15:28:53.907907 2014-08-13 17:38:02.908985 7fe9bae887a0 20 read_log 194416'17029 (0'0) promote 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by osd.364.0:4128 2014-08-13 15:28:54.748039 2014-08-13 17:38:02.908997 7fe9bae887a0 20 read_log 194416'17030 (194416'17029) modify 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by client.2441958.0:6218 2014-08-13 15:28:54.739638 2014-08-13 17:38:02.909028 7fe9bae887a0 20 read_log 194416'17031 (0'0) promote 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by osd.364.0:4158 2014-08-13 15:29:04.781669 2014-08-13 17:38:02.909042 7fe9bae887a0 20 read_log 194416'17032 (194416'17031) modify 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by client.2441958.0:8846 2014-08-13 15:29:04.772064 2014-08-13 17:38:02.909071 7fe9bae887a0 20 read_log 194416'17033 (0'0) promote 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by osd.364.0:4162 2014-08-13 15:29:05.686961 2014-08-13 17:38:02.909084 7fe9bae887a0 20 read_log 194416'17034 (194416'17033) modify 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by client.2441958.0:9054 2014-08-13 15:29:05.677607 2014-08-13 17:38:02.909096 7fe9bae887a0 20 read_log 194416'17035 (0'0) promote 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by osd.364.0:4184 2014-08-13 15:29:13.941911 2014-08-13 17:38:02.909108 7fe9bae887a0 20 read_log 194416'17036 (194416'17035) modify 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by client.2441958.0:11141 2014-08-13 15:29:13.933397 2014-08-13 17:38:02.909120 7fe9bae887a0 20 read_log 194416'17037 (0'0) promote aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by osd.364.0:4190 2014-08-13 15:29:14.613548 2014-08-13 17:38:02.909132 7fe9bae887a0 20 read_log 194416'17038 (194416'17037) modify aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by client.2441958.0:11283 2014-08-13 15:29:14.606987 2014-08-13 17:38:02.909182 7fe9bae887a0 20 read_log 194416'17039 (0'0) promote a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by osd.364.0:4196 2014-08-13 15:29:16.675756 2014-08-13 17:38:02.909194 7fe9bae887a0 20 read_log 194416'17040 (194416'17039) modify a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by client.2441958.0:11789 2014-08-13 15:29:16.666335 2014-08-13 17:38:02.909207 7fe9bae887a0 20 read_log 194416'17041 (0'0) promote c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by osd.364.0:4208 2014-08-13 15:29:19.591320 2014-08-13 17:38:02.909219 7fe9bae887a0 20 read_log 194416'17042 (194416'17041) modify c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by client.2441958.0:12471 2014-08-13 15:29:19.578362 2014-08-13 17:38:02.909239 7fe9bae887a0 20 read_log 194416'17043 (0'0) promote 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by osd.364.0:4226 2014-08-13 15:29:25.085099 2014-08-13 17:38:02.909251 7fe9bae887a0 20 read_log 194416'17044 (194416'17043) modify 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by client.2441958.0:13719 2014-08-13 15:29:25.075531 2014-08-13 17:38:02.909264 7fe9bae887a0 20 read_log 194416'17045 (0'0) promote 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by osd.364.0:4290 2014-08-13 15:30:13.915299 2014-08-13 17:38:02.909276 7fe9bae887a0 20 read_log 194416'17046 (194416'17045) modify 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by client.2441958.0:18529 2014-08-13 15:30:13.906249 2014-08-13 17:38:02.909288 7fe9bae887a0 20 read_log 194422'17047 (0'0) promote 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by osd.364.0:4292 2014-08-13 15:32:15.581218 2014-08-13 17:38:02.909300 7fe9bae887a0 20 read_log 194422'17048 (194422'17047) modify 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by client.1148203.0:142 2014-08-13 15:32:15.571168 2014-08-13 17:38:02.909312 7fe9bae887a0 20 read_log 194422'17049 (0'0) promote 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by osd.364.0:4296 2014-08-13 15:32:16.377171 2014-08-13 17:38:02.909324 7fe9bae887a0 20 read_log 194422'17050 (194422'17049) modify 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by client.1148203.0:300 2014-08-13 15:32:16.369060 2014-08-13 17:38:02.909336 7fe9bae887a0 20 read_log 194422'17051 (0'0) promote 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by osd.364.0:4308 2014-08-13 15:32:18.166647 2014-08-13 17:38:02.909348 7fe9bae887a0 20 read_log 194422'17052 (194422'17051) modify 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by client.1148203.0:702 2014-08-13 15:32:18.157676 2014-08-13 17:38:02.909360 7fe9bae887a0 20 read_log 194422'17053 (0'0) promote d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by osd.364.0:4314 2014-08-13 15:32:20.015280 2014-08-13 17:38:02.909374 7fe9bae887a0 20 read_log 194422'17054 (194422'17053) modify d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by client.1148203.0:1189 2014-08-13 15:32:20.006353 2014-08-13 17:38:02.909388 7fe9bae887a0 20 read_log 194422'17055 (0'0) promote 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by osd.364.0:4334 2014-08-13 15:32:25.908685 2014-08-13 17:38:02.909400 7fe9bae887a0 20 read_log 194422'17056 (194422'17055) modify 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by client.1148203.0:2704 2014-08-13 15:32:25.901830 2014-08-13 17:38:02.909413 7fe9bae887a0 20 read_log 194422'17057 (0'0) promote d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by osd.364.0:4348 2014-08-13 15:32:29.709011 2014-08-13 17:38:02.909425 7fe9bae887a0 20 read_log 194422'17058 (194422'17057) modify d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by client.1148203.0:3659 2014-08-13 15:32:29.701406 2014-08-13 17:38:02.909437 7fe9bae887a0 20 read_log 194422'17059 (0'0) promote 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by osd.364.0:4350 2014-08-13 15:32:29.902858 2014-08-13 17:38:02.909449 7fe9bae887a0 20 read_log 194422'17060 (194422'17059) modify 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by client.1148203.0:3709 2014-08-13 15:32:29.894894 2014-08-13 17:38:02.909461 7fe9bae887a0 20 read_log 194422'17061 (0'0) promote 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by osd.364.0:4372 2014-08-13 15:32:38.401185 2014-08-13 17:38:02.909473 7fe9bae887a0 20 read_log 194422'17062 (194422'17061) modify 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by client.1148203.0:5681 2014-08-13 15:32:38.391351 2014-08-13 17:38:02.909486 7fe9bae887a0 20 read_log 194422'17063 (0'0) promote 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by osd.364.0:4386 2014-08-13 15:32:43.094141 2014-08-13 17:38:02.909499 7fe9bae887a0 20 read_log 194422'17064 (194422'17063) modify 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by client.1148203.0:6548 2014-08-13 15:32:43.085134 2014-08-13 17:38:02.909511 7fe9bae887a0 20 read_log 194422'17065 (0'0) promote dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by osd.364.0:4400 2014-08-13 15:32:48.919635 2014-08-13 17:38:02.909523 7fe9bae887a0 20 read_log 194422'17066 (194422'17065) modify dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by client.1148203.0:7549 2014-08-13 15:32:48.910443 2014-08-13 17:38:02.909535 7fe9bae887a0 20 read_log 194422'17067 (0'0) promote 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by osd.364.0:4406 2014-08-13 15:32:51.270747 2014-08-13 17:38:02.909547 7fe9bae887a0 20 read_log 194422'17068 (194422'17067) modify 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by client.1148203.0:7929 2014-08-13 15:32:51.261634 2014-08-13 17:38:02.909559 7fe9bae887a0 20 read_log 194422'17069 (0'0) promote a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by osd.364.0:4414 2014-08-13 15:32:52.703908 2014-08-13 17:38:02.909571 7fe9bae887a0 20 read_log 194422'17070 (194422'17069) modify a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by client.1148203.0:8156 2014-08-13 15:32:52.695197 2014-08-13 17:38:02.909583 7fe9bae887a0 20 read_log 194422'17071 (0'0) promote e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by osd.364.0:4416 2014-08-13 15:32:52.819821 2014-08-13 17:38:02.909594 7fe9bae887a0 20 read_log 194422'17072 (194422'17071) modify e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by client.1148203.0:8174 2014-08-13 15:32:52.812513 2014-08-13 17:38:02.909607 7fe9bae887a0 20 read_log 194422'17073 (0'0) promote 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by osd.364.0:4437 2014-08-13 15:33:21.760279 2014-08-13 17:38:02.909618 7fe9bae887a0 20 read_log 194422'17074 (194422'17073) modify 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by client.1148203.0:10420 2014-08-13 15:33:21.751200 2014-08-13 17:38:02.909633 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.910049 7fe9bae887a0 10 read_log checking for missing items over interval (194370'16977,194422'17074] 2014-08-13 17:38:02.910098 7fe9bae887a0 15 read_log missing 194422'17074 (194422'17073) modify 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by client.1148203.0:10420 2014-08-13 15:33:21.751200 2014-08-13 17:38:02.910139 7fe9bae887a0 15 read_log missing 194422'17072 (194422'17071) modify e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by client.1148203.0:8174 2014-08-13 15:32:52.812513 2014-08-13 17:38:02.910182 7fe9bae887a0 15 read_log missing 194422'17070 (194422'17069) modify a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by client.1148203.0:8156 2014-08-13 15:32:52.695197 2014-08-13 17:38:02.910220 7fe9bae887a0 15 read_log missing 194422'17068 (194422'17067) modify 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by client.1148203.0:7929 2014-08-13 15:32:51.261634 2014-08-13 17:38:02.910256 7fe9bae887a0 15 read_log missing 194422'17066 (194422'17065) modify dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by client.1148203.0:7549 2014-08-13 15:32:48.910443 2014-08-13 17:38:02.910292 7fe9bae887a0 15 read_log missing 194422'17064 (194422'17063) modify 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by client.1148203.0:6548 2014-08-13 15:32:43.085134 2014-08-13 17:38:02.910329 7fe9bae887a0 15 read_log missing 194422'17062 (194422'17061) modify 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by client.1148203.0:5681 2014-08-13 15:32:38.391351 2014-08-13 17:38:02.910365 7fe9bae887a0 15 read_log missing 194422'17060 (194422'17059) modify 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by client.1148203.0:3709 2014-08-13 15:32:29.894894 2014-08-13 17:38:02.910402 7fe9bae887a0 15 read_log missing 194422'17058 (194422'17057) modify d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by client.1148203.0:3659 2014-08-13 15:32:29.701406 2014-08-13 17:38:02.910438 7fe9bae887a0 15 read_log missing 194422'17056 (194422'17055) modify 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by client.1148203.0:2704 2014-08-13 15:32:25.901830 2014-08-13 17:38:02.910475 7fe9bae887a0 15 read_log missing 194422'17054 (194422'17053) modify d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by client.1148203.0:1189 2014-08-13 15:32:20.006353 2014-08-13 17:38:02.910512 7fe9bae887a0 15 read_log missing 194422'17052 (194422'17051) modify 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by client.1148203.0:702 2014-08-13 15:32:18.157676 2014-08-13 17:38:02.910548 7fe9bae887a0 15 read_log missing 194422'17050 (194422'17049) modify 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by client.1148203.0:300 2014-08-13 15:32:16.369060 2014-08-13 17:38:02.910585 7fe9bae887a0 15 read_log missing 194422'17048 (194422'17047) modify 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by client.1148203.0:142 2014-08-13 15:32:15.571168 2014-08-13 17:38:02.910622 7fe9bae887a0 15 read_log missing 194416'17046 (194416'17045) modify 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by client.2441958.0:18529 2014-08-13 15:30:13.906249 2014-08-13 17:38:02.910658 7fe9bae887a0 15 read_log missing 194416'17044 (194416'17043) modify 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by client.2441958.0:13719 2014-08-13 15:29:25.075531 2014-08-13 17:38:02.910695 7fe9bae887a0 15 read_log missing 194416'17042 (194416'17041) modify c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by client.2441958.0:12471 2014-08-13 15:29:19.578362 2014-08-13 17:38:02.910732 7fe9bae887a0 15 read_log missing 194416'17040 (194416'17039) modify a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by client.2441958.0:11789 2014-08-13 15:29:16.666335 2014-08-13 17:38:02.910769 7fe9bae887a0 15 read_log missing 194416'17038 (194416'17037) modify aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by client.2441958.0:11283 2014-08-13 15:29:14.606987 2014-08-13 17:38:02.910806 7fe9bae887a0 15 read_log missing 194416'17036 (194416'17035) modify 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by client.2441958.0:11141 2014-08-13 15:29:13.933397 2014-08-13 17:38:02.910843 7fe9bae887a0 15 read_log missing 194416'17034 (194416'17033) modify 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by client.2441958.0:9054 2014-08-13 15:29:05.677607 2014-08-13 17:38:02.910879 7fe9bae887a0 15 read_log missing 194416'17032 (194416'17031) modify 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by client.2441958.0:8846 2014-08-13 15:29:04.772064 2014-08-13 17:38:02.910916 7fe9bae887a0 15 read_log missing 194416'17030 (194416'17029) modify 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by client.2441958.0:6218 2014-08-13 15:28:54.739638 2014-08-13 17:38:02.910952 7fe9bae887a0 15 read_log missing 194416'17028 (194416'17027) modify 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by client.2441958.0:6041 2014-08-13 15:28:53.907907 2014-08-13 17:38:02.910989 7fe9bae887a0 15 read_log missing 194416'17026 (194416'17025) modify 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by client.2441958.0:3435 2014-08-13 15:28:43.708507 2014-08-13 17:38:02.911025 7fe9bae887a0 15 read_log missing 194416'17024 (194416'17023) modify da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by client.2441958.0:2863 2014-08-13 15:28:41.558080 2014-08-13 17:38:02.911062 7fe9bae887a0 15 read_log missing 194416'17022 (194416'17021) modify b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by client.2441958.0:2788 2014-08-13 15:28:41.277463 2014-08-13 17:38:02.911098 7fe9bae887a0 15 read_log missing 194416'17020 (194416'17019) modify 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by client.2441958.0:1240 2014-08-13 15:28:35.419433 2014-08-13 17:38:02.911135 7fe9bae887a0 15 read_log missing 194416'17018 (194416'17017) modify 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by client.2441958.0:310 2014-08-13 15:28:31.491025 2014-08-13 17:38:02.911181 7fe9bae887a0 15 read_log missing 194416'17015 (194416'17015) modify 1de/hit_set_24.1de_archive_2014-08-12 20:49:27.577314_2014-08-13 15:28:31.498257/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 17:38:02.911219 7fe9bae887a0 15 read_log missing 194393'17014 (194393'17013) modify ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by client.1064182.0:425 2014-08-12 20:49:27.571281 2014-08-13 17:38:02.911258 7fe9bae887a0 15 read_log missing 194383'17010 (194383'17009) modify 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by client.2231190.0:7532 2014-08-12 20:34:29.634334 2014-08-13 17:38:02.911295 7fe9bae887a0 15 read_log missing 194383'17008 (194383'17007) modify 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by client.2231190.0:7355 2014-08-12 20:34:24.786699 2014-08-13 17:38:02.911331 7fe9bae887a0 15 read_log missing 194377'17006 (194377'17005) modify 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by client.2231190.0:1440 2014-08-12 20:33:02.964206 2014-08-13 17:38:02.911367 7fe9bae887a0 15 read_log missing 194370'17004 (194370'17003) modify 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by client.1508531.0:13997 2014-08-12 20:31:32.126438 2014-08-13 17:38:02.911404 7fe9bae887a0 15 read_log missing 194370'17002 (194370'17001) modify 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by client.1508531.0:13689 2014-08-12 20:31:25.940307 2014-08-13 17:38:02.911441 7fe9bae887a0 15 read_log missing 194370'17000 (194370'16999) modify 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by client.1508531.0:13364 2014-08-12 20:31:22.848815 2014-08-13 17:38:02.911478 7fe9bae887a0 15 read_log missing 194370'16998 (194370'16997) modify 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by client.1508531.0:12736 2014-08-12 20:31:16.863845 2014-08-13 17:38:02.911515 7fe9bae887a0 15 read_log missing 194370'16996 (194370'16995) modify 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by client.1508531.0:12034 2014-08-12 20:31:11.595130 2014-08-13 17:38:02.911552 7fe9bae887a0 15 read_log missing 194370'16994 (194370'16993) modify f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by client.1508531.0:12031 2014-08-12 20:31:11.574016 2014-08-13 17:38:02.911590 7fe9bae887a0 15 read_log missing 194370'16992 (194370'16991) modify 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by client.1508531.0:11959 2014-08-12 20:31:11.056711 2014-08-13 17:38:02.911627 7fe9bae887a0 15 read_log missing 194370'16990 (194370'16989) modify 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by client.1508531.0:11778 2014-08-12 20:31:09.947609 2014-08-13 17:38:02.911665 7fe9bae887a0 15 read_log missing 194370'16988 (194370'16987) modify df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by client.1508531.0:11557 2014-08-12 20:31:08.650240 2014-08-13 17:38:02.911702 7fe9bae887a0 15 read_log missing 194370'16986 (194370'16985) modify b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by client.1508531.0:4822 2014-08-12 20:30:32.463242 2014-08-13 17:38:02.911739 7fe9bae887a0 15 read_log missing 194370'16984 (194370'16983) modify 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by client.1508531.0:4665 2014-08-12 20:30:31.867906 2014-08-13 17:38:02.911867 7fe9bae887a0 15 read_log missing 194370'16982 (194370'16981) modify d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by client.1508531.0:4570 2014-08-12 20:30:31.488804 2014-08-13 17:38:02.911910 7fe9bae887a0 15 read_log missing 194370'16980 (194370'16979) modify db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by client.1508531.0:4154 2014-08-12 20:30:29.875338 2014-08-13 17:38:02.911948 7fe9bae887a0 15 read_log missing 194370'16978 (194370'16977) modify 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by client.1508531.0:3700 2014-08-12 20:30:28.195281 2014-08-13 17:38:02.911959 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.911986 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive m=48] handle_loaded 2014-08-13 17:38:02.912003 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Initial 0.013239 0 0.000000 2014-08-13 17:38:02.912016 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset 2014-08-13 17:38:02.912025 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194477 2014-08-13 17:38:02.912034 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] log((192834'15290,194422'17074], crt=194422'17074) 2014-08-13 17:38:02.912045 7fe9bae887a0 10 osd.4 194478 pgid 24.1e4 coll 24.1e4_head 2014-08-13 17:38:02.912160 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.1e4 2014-08-13 17:38:02.912173 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.912205 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1e4(unlocked)] enter Initial 2014-08-13 17:38:02.912217 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1e4(unlocked)] enter NotTrimming 2014-08-13 17:38:02.912347 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.912487 7fe9bae887a0 20 read_log 192834'16490 (192834'15999) clean 8b8129e4/benchmark_data_storage0111-ib.pocstor_26659_object7495/head//24 by osd.376.0:18837 2014-08-11 15:26:47.861429 2014-08-13 17:38:02.912508 7fe9bae887a0 20 read_log 192834'16493 (192834'16490) delete 8b8129e4/benchmark_data_storage0111-ib.pocstor_26659_object7495/head//24 by osd.376.0:18840 2014-08-11 15:26:47.930074 2014-08-13 17:38:02.912522 7fe9bae887a0 20 read_log 192834'16494 (192834'16001) clean 652a51e4/benchmark_data_storage0111-ib.pocstor_26659_object7550/head//24 by osd.376.0:18841 2014-08-11 15:26:48.059873 2014-08-13 17:38:02.912535 7fe9bae887a0 20 read_log 192834'16528 (192834'16494) delete 652a51e4/benchmark_data_storage0111-ib.pocstor_26659_object7550/head//24 by osd.376.0:18875 2014-08-11 15:26:53.162875 2014-08-13 17:38:02.912547 7fe9bae887a0 20 read_log 192834'16529 (192834'16088) clean 8f1151e4/benchmark_data_storage0111-ib.pocstor_26659_object10963/head//24 by osd.376.0:18876 2014-08-11 15:26:53.163334 2014-08-13 17:38:02.912560 7fe9bae887a0 20 read_log 192834'16530 (192834'16070) clean 5e9ab1e4/benchmark_data_storage0111-ib.pocstor_26659_object9782/head//24 by osd.376.0:18877 2014-08-11 15:26:53.175662 2014-08-13 17:38:02.912572 7fe9bae887a0 20 read_log 192834'16532 (192834'16033) clean 4e24d5e4/benchmark_data_storage0111-ib.pocstor_26659_object8815/head//24 by osd.376.0:18879 2014-08-11 15:26:53.184051 2014-08-13 17:38:02.912584 7fe9bae887a0 20 read_log 192834'16540 (192834'16011) clean b49e0de4/benchmark_data_storage0111-ib.pocstor_26659_object7890/head//24 by osd.376.0:18887 2014-08-11 15:26:53.227715 2014-08-13 17:38:02.912597 7fe9bae887a0 20 read_log 192834'16541 (192834'16017) clean 6a5f6de4/benchmark_data_storage0111-ib.pocstor_26659_object8143/head//24 by osd.376.0:18888 2014-08-11 15:26:53.232422 2014-08-13 17:38:02.912620 7fe9bae887a0 20 read_log 192834'16563 (192834'16529) delete 8f1151e4/benchmark_data_storage0111-ib.pocstor_26659_object10963/head//24 by osd.376.0:18910 2014-08-11 15:26:53.314990 2014-08-13 17:38:02.912635 7fe9bae887a0 20 read_log 192834'16564 (192834'16530) delete 5e9ab1e4/benchmark_data_storage0111-ib.pocstor_26659_object9782/head//24 by osd.376.0:18911 2014-08-11 15:26:53.318901 2014-08-13 17:38:02.912647 7fe9bae887a0 20 read_log 192834'16566 (192834'16532) delete 4e24d5e4/benchmark_data_storage0111-ib.pocstor_26659_object8815/head//24 by osd.376.0:18913 2014-08-11 15:26:53.324690 2014-08-13 17:38:02.912660 7fe9bae887a0 20 read_log 192834'16574 (192834'16540) delete b49e0de4/benchmark_data_storage0111-ib.pocstor_26659_object7890/head//24 by osd.376.0:18921 2014-08-11 15:26:53.347774 2014-08-13 17:38:02.912674 7fe9bae887a0 20 read_log 192834'16575 (192834'16541) delete 6a5f6de4/benchmark_data_storage0111-ib.pocstor_26659_object8143/head//24 by osd.376.0:18922 2014-08-11 15:26:53.348696 2014-08-13 17:38:02.912686 7fe9bae887a0 20 read_log 192834'16609 (192834'16094) clean 7dd549e4/benchmark_data_storage0111-ib.pocstor_26659_object11222/head//24 by osd.376.0:18956 2014-08-11 15:26:58.507301 2014-08-13 17:38:02.912699 7fe9bae887a0 20 read_log 192834'16610 (192834'16151) clean 6fb079e4/benchmark_data_storage0111-ib.pocstor_26659_object14301/head//24 by osd.376.0:18957 2014-08-11 15:26:58.513974 2014-08-13 17:38:02.912711 7fe9bae887a0 20 read_log 192834'16611 (192834'16135) clean 4e95d9e4/benchmark_data_storage0111-ib.pocstor_26659_object13710/head//24 by osd.376.0:18958 2014-08-11 15:26:58.517977 2014-08-13 17:38:02.912724 7fe9bae887a0 20 read_log 192834'16632 (192834'16609) delete 7dd549e4/benchmark_data_storage0111-ib.pocstor_26659_object11222/head//24 by osd.376.0:18979 2014-08-11 15:26:58.623016 2014-08-13 17:38:02.912737 7fe9bae887a0 20 read_log 192834'16633 (192834'16610) delete 6fb079e4/benchmark_data_storage0111-ib.pocstor_26659_object14301/head//24 by osd.376.0:18980 2014-08-11 15:26:58.626477 2014-08-13 17:38:02.912750 7fe9bae887a0 20 read_log 192834'16634 (192834'16611) delete 4e95d9e4/benchmark_data_storage0111-ib.pocstor_26659_object13710/head//24 by osd.376.0:18981 2014-08-11 15:26:58.631088 2014-08-13 17:38:02.912762 7fe9bae887a0 20 read_log 192834'16644 (0'0) promote ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:18991 2014-08-11 15:27:01.312408 2014-08-13 17:38:02.912774 7fe9bae887a0 20 read_log 192834'16645 (192834'16644) modify ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by client.1883634.0:11798 2014-08-11 15:27:01.303391 2014-08-13 17:38:02.912787 7fe9bae887a0 20 read_log 192834'16656 (0'0) promote ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19003 2014-08-11 15:27:03.576334 2014-08-13 17:38:02.912799 7fe9bae887a0 20 read_log 192834'16657 (192834'16656) modify ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by client.1883634.0:12332 2014-08-11 15:27:03.568932 2014-08-13 17:38:02.912811 7fe9bae887a0 20 read_log 192834'16693 (192834'16242) clean 3aa285e4/benchmark_data_storage0111-ib.pocstor_26659_object20665/head//24 by osd.376.0:19040 2014-08-11 15:27:03.871504 2014-08-13 17:38:02.912824 7fe9bae887a0 20 read_log 192834'16705 (192834'16230) clean 43f20de4/benchmark_data_storage0111-ib.pocstor_26659_object19129/head//24 by osd.376.0:19052 2014-08-11 15:27:03.920400 2014-08-13 17:38:02.912836 7fe9bae887a0 20 read_log 192834'16706 (192834'16247) clean 710f1de4/benchmark_data_storage0111-ib.pocstor_26659_object21474/head//24 by osd.376.0:19053 2014-08-11 15:27:03.923690 2014-08-13 17:38:02.912848 7fe9bae887a0 20 read_log 192834'16707 (192834'16172) clean e9685de4/benchmark_data_storage0111-ib.pocstor_26659_object15613/head//24 by osd.376.0:19054 2014-08-11 15:27:03.932435 2014-08-13 17:38:02.912860 7fe9bae887a0 20 read_log 192834'16708 (192834'16217) clean d41a5de4/benchmark_data_storage0111-ib.pocstor_26659_object17013/head//24 by osd.376.0:19055 2014-08-11 15:27:03.934378 2014-08-13 17:38:02.912880 7fe9bae887a0 20 read_log 192834'16748 (192834'16693) delete 3aa285e4/benchmark_data_storage0111-ib.pocstor_26659_object20665/head//24 by osd.376.0:19095 2014-08-11 15:27:04.029255 2014-08-13 17:38:02.912893 7fe9bae887a0 20 read_log 192834'16760 (192834'16705) delete 43f20de4/benchmark_data_storage0111-ib.pocstor_26659_object19129/head//24 by osd.376.0:19107 2014-08-11 15:27:04.048439 2014-08-13 17:38:02.912905 7fe9bae887a0 20 read_log 192834'16761 (192834'16706) delete 710f1de4/benchmark_data_storage0111-ib.pocstor_26659_object21474/head//24 by osd.376.0:19108 2014-08-11 15:27:04.048798 2014-08-13 17:38:02.912917 7fe9bae887a0 20 read_log 192834'16762 (192834'16707) delete e9685de4/benchmark_data_storage0111-ib.pocstor_26659_object15613/head//24 by osd.376.0:19109 2014-08-11 15:27:04.049381 2014-08-13 17:38:02.912929 7fe9bae887a0 20 read_log 192834'16763 (192834'16708) delete d41a5de4/benchmark_data_storage0111-ib.pocstor_26659_object17013/head//24 by osd.376.0:19110 2014-08-11 15:27:04.052067 2014-08-13 17:38:02.912941 7fe9bae887a0 20 read_log 192834'16772 (0'0) promote b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19119 2014-08-11 15:27:04.630613 2014-08-13 17:38:02.912953 7fe9bae887a0 20 read_log 192834'16773 (192834'16772) modify b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by client.1883634.0:12579 2014-08-11 15:27:04.621518 2014-08-13 17:38:02.912965 7fe9bae887a0 20 read_log 192834'16784 (0'0) promote c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19131 2014-08-11 15:27:09.893547 2014-08-13 17:38:02.912977 7fe9bae887a0 20 read_log 192834'16785 (192834'16784) modify c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by client.1883634.0:13500 2014-08-11 15:27:09.886556 2014-08-13 17:38:02.912989 7fe9bae887a0 20 read_log 192834'16796 (0'0) promote 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19142 2014-08-11 15:27:10.822345 2014-08-13 17:38:02.913004 7fe9bae887a0 20 read_log 192834'16797 (192834'16796) modify 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by client.1883634.0:13794 2014-08-11 15:27:10.814860 2014-08-13 17:38:02.913017 7fe9bae887a0 20 read_log 192834'16820 (192834'16253) clean 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19166 2014-08-11 15:27:19.409687 2014-08-13 17:38:02.913031 7fe9bae887a0 20 read_log 192834'16833 (192834'16255) clean cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19179 2014-08-11 15:27:19.723091 2014-08-13 17:38:02.913063 7fe9bae887a0 20 read_log 192834'16836 (192834'16820) delete 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19182 2014-08-11 15:27:19.741014 2014-08-13 17:38:02.913079 7fe9bae887a0 20 read_log 192834'16841 (192834'16833) delete cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19187 2014-08-11 15:27:19.767551 2014-08-13 17:38:02.913091 7fe9bae887a0 20 read_log 192834'16867 (0'0) promote faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19213 2014-08-11 15:27:27.709173 2014-08-13 17:38:02.913106 7fe9bae887a0 20 read_log 192834'16868 (192834'16867) modify faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by client.1883634.0:16468 2014-08-11 15:27:27.687708 2014-08-13 17:38:02.913118 7fe9bae887a0 20 read_log 192834'16871 (0'0) promote b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19217 2014-08-11 15:27:28.422589 2014-08-13 17:38:02.913132 7fe9bae887a0 20 read_log 192834'16872 (192834'16871) modify b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by client.1883634.0:16603 2014-08-11 15:27:28.414830 2014-08-13 17:38:02.913145 7fe9bae887a0 20 read_log 192834'16907 (192834'16317) clean 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19253 2014-08-11 15:27:35.686832 2014-08-13 17:38:02.913192 7fe9bae887a0 20 read_log 192834'16909 (192834'16907) delete 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19255 2014-08-11 15:27:35.702949 2014-08-13 17:38:02.913209 7fe9bae887a0 20 read_log 192834'16915 (0'0) promote c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19261 2014-08-11 15:27:38.785079 2014-08-13 17:38:02.913221 7fe9bae887a0 20 read_log 192834'16916 (192834'16915) modify c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by client.1883634.0:17735 2014-08-11 15:27:38.776108 2014-08-13 17:38:02.913234 7fe9bae887a0 20 read_log 192834'16954 (192834'16418) clean a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19300 2014-08-11 15:27:46.171596 2014-08-13 17:38:02.913246 7fe9bae887a0 20 read_log 192834'16955 (192834'16351) clean 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19301 2014-08-11 15:27:46.187454 2014-08-13 17:38:02.913260 7fe9bae887a0 20 read_log 192834'16958 (192834'16353) clean 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19304 2014-08-11 15:27:46.262566 2014-08-13 17:38:02.913273 7fe9bae887a0 20 read_log 192834'16961 (192834'16414) clean a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19307 2014-08-11 15:27:46.314392 2014-08-13 17:38:02.913287 7fe9bae887a0 20 read_log 192834'16964 (192834'16954) delete a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19310 2014-08-11 15:27:46.324663 2014-08-13 17:38:02.913299 7fe9bae887a0 20 read_log 192834'16965 (192834'16955) delete 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19311 2014-08-11 15:27:46.328271 2014-08-13 17:38:02.913311 7fe9bae887a0 20 read_log 192834'16968 (192834'16958) delete 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19314 2014-08-11 15:27:46.339635 2014-08-13 17:38:02.913332 7fe9bae887a0 20 read_log 192834'16969 (192834'16961) delete a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19315 2014-08-11 15:27:46.343372 2014-08-13 17:38:02.913344 7fe9bae887a0 20 read_log 192834'16979 (0'0) promote ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:19325 2014-08-11 15:27:48.637354 2014-08-13 17:38:02.913357 7fe9bae887a0 20 read_log 192834'16980 (192834'16979) modify ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by client.1883634.0:18985 2014-08-11 15:27:48.628369 2014-08-13 17:38:02.913369 7fe9bae887a0 20 read_log 192834'17001 (0'0) promote 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:19347 2014-08-11 15:27:53.723432 2014-08-13 17:38:02.913381 7fe9bae887a0 20 read_log 192834'17002 (192834'17001) modify 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by client.1883634.0:19662 2014-08-11 15:27:53.715724 2014-08-13 17:38:02.913393 7fe9bae887a0 20 read_log 192834'17019 (0'0) promote 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by osd.376.0:19365 2014-08-11 15:28:01.637810 2014-08-13 17:38:02.913405 7fe9bae887a0 20 read_log 192834'17020 (192834'17019) modify 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.1883634.0:20814 2014-08-11 15:28:01.628647 2014-08-13 17:38:02.913417 7fe9bae887a0 20 read_log 192834'17026 (192834'16645) clean ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19372 2014-08-11 15:28:01.930424 2014-08-13 17:38:02.913430 7fe9bae887a0 20 read_log 192834'17037 (192834'17026) delete ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19383 2014-08-11 15:28:02.201856 2014-08-13 17:38:02.913442 7fe9bae887a0 20 read_log 192834'17052 (0'0) promote 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:19398 2014-08-11 15:28:04.148687 2014-08-13 17:38:02.913464 7fe9bae887a0 20 read_log 192834'17053 (192834'17052) modify 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by client.1883634.0:21426 2014-08-11 15:28:04.139645 2014-08-13 17:38:02.913480 7fe9bae887a0 20 read_log 192834'17065 (192834'16773) clean b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19411 2014-08-11 15:28:07.412361 2014-08-13 17:38:02.913492 7fe9bae887a0 20 read_log 192834'17066 (192834'16657) clean ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19412 2014-08-11 15:28:07.431144 2014-08-13 17:38:02.913504 7fe9bae887a0 20 read_log 192834'17079 (0'0) promote ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:19425 2014-08-11 15:28:11.376531 2014-08-13 17:38:02.913516 7fe9bae887a0 20 read_log 192834'17080 (192834'17079) modify ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by client.1883634.0:22850 2014-08-11 15:28:11.367862 2014-08-13 17:38:02.913529 7fe9bae887a0 20 read_log 192834'17091 (192834'17065) delete b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19437 2014-08-11 15:28:12.791715 2014-08-13 17:38:02.913542 7fe9bae887a0 20 read_log 192834'17092 (192834'17066) delete ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19438 2014-08-11 15:28:12.798702 2014-08-13 17:38:02.913554 7fe9bae887a0 20 read_log 192834'17093 (192834'16797) clean 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19439 2014-08-11 15:28:12.802150 2014-08-13 17:38:02.913566 7fe9bae887a0 20 read_log 192834'17101 (192834'17093) delete 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19447 2014-08-11 15:28:12.838726 2014-08-13 17:38:02.913578 7fe9bae887a0 20 read_log 192834'17102 (192834'16785) clean c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19448 2014-08-11 15:28:12.887330 2014-08-13 17:38:02.913590 7fe9bae887a0 20 read_log 192834'17105 (0'0) promote 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:19451 2014-08-11 15:28:13.252449 2014-08-13 17:38:02.913602 7fe9bae887a0 20 read_log 192834'17106 (192834'17105) modify 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by client.1883634.0:23171 2014-08-11 15:28:13.244038 2014-08-13 17:38:02.913614 7fe9bae887a0 20 read_log 192834'17107 (0'0) promote 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:19453 2014-08-11 15:28:13.300441 2014-08-13 17:38:02.913626 7fe9bae887a0 20 read_log 192834'17108 (192834'17107) modify 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by client.1883634.0:23179 2014-08-11 15:28:13.291872 2014-08-13 17:38:02.913638 7fe9bae887a0 20 read_log 192834'17113 (0'0) promote 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19459 2014-08-11 15:28:16.236487 2014-08-13 17:38:02.913650 7fe9bae887a0 20 read_log 192834'17114 (192834'17113) modify 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by client.1883634.0:23343 2014-08-11 15:28:16.204144 2014-08-13 17:38:02.913662 7fe9bae887a0 20 read_log 192834'17115 (0'0) promote cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19461 2014-08-11 15:28:16.384055 2014-08-13 17:38:02.913674 7fe9bae887a0 20 read_log 192834'17116 (192834'17115) modify cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by client.1883634.0:23440 2014-08-11 15:28:16.360524 2014-08-13 17:38:02.913686 7fe9bae887a0 20 read_log 192834'17130 (192834'17102) delete c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19476 2014-08-11 15:28:17.981088 2014-08-13 17:38:02.913698 7fe9bae887a0 20 read_log 192834'17201 (0'0) promote 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19547 2014-08-11 15:28:26.103379 2014-08-13 17:38:02.913717 7fe9bae887a0 20 read_log 192834'17202 (192834'17201) modify 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by client.1883634.0:28611 2014-08-11 15:28:26.035642 2014-08-13 17:38:02.913730 7fe9bae887a0 20 read_log 192834'17240 (192834'16868) clean faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19586 2014-08-11 15:28:28.711273 2014-08-13 17:38:02.913742 7fe9bae887a0 20 read_log 192834'17242 (192834'17240) delete faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19588 2014-08-11 15:28:28.719370 2014-08-13 17:38:02.913755 7fe9bae887a0 20 read_log 192834'17246 (192834'16872) clean b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19592 2014-08-11 15:28:28.778307 2014-08-13 17:38:02.913767 7fe9bae887a0 20 read_log 192834'17247 (192834'17246) delete b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19593 2014-08-11 15:28:28.780856 2014-08-13 17:38:02.913779 7fe9bae887a0 20 read_log 192834'17249 (0'0) promote 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19595 2014-08-11 15:28:28.984745 2014-08-13 17:38:02.913791 7fe9bae887a0 20 read_log 192834'17250 (192834'17249) modify 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by client.1883634.0:30612 2014-08-11 15:28:28.952860 2014-08-13 17:38:02.913803 7fe9bae887a0 20 read_log 192834'17251 (0'0) promote 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19597 2014-08-11 15:28:29.092782 2014-08-13 17:38:02.913815 7fe9bae887a0 20 read_log 192834'17252 (192834'17251) modify 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by client.1883634.0:30672 2014-08-11 15:28:29.064254 2014-08-13 17:38:02.913828 7fe9bae887a0 20 read_log 192834'17261 (0'0) promote a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19607 2014-08-11 15:28:30.420688 2014-08-13 17:38:02.913840 7fe9bae887a0 20 read_log 192834'17262 (192834'17261) modify a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by client.1883634.0:31380 2014-08-11 15:28:30.395882 2014-08-13 17:38:02.913851 7fe9bae887a0 20 read_log 192834'17265 (0'0) promote a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19611 2014-08-11 15:28:30.857820 2014-08-13 17:38:02.913864 7fe9bae887a0 20 read_log 192834'17266 (192834'17265) modify a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by client.1883634.0:31677 2014-08-11 15:28:30.826313 2014-08-13 17:38:02.913876 7fe9bae887a0 20 read_log 192834'17310 (0'0) promote ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19656 2014-08-11 15:28:36.087509 2014-08-13 17:38:02.913888 7fe9bae887a0 20 read_log 192834'17311 (192834'17310) modify ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by client.1883634.0:35122 2014-08-11 15:28:36.047511 2014-08-13 17:38:02.913900 7fe9bae887a0 20 read_log 192834'17322 (0'0) promote ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19668 2014-08-11 15:28:36.985354 2014-08-13 17:38:02.913912 7fe9bae887a0 20 read_log 192834'17323 (192834'17322) modify ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by client.1883634.0:35656 2014-08-11 15:28:36.961937 2014-08-13 17:38:02.913923 7fe9bae887a0 20 read_log 192834'17328 (0'0) promote b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19674 2014-08-11 15:28:37.326179 2014-08-13 17:38:02.913935 7fe9bae887a0 20 read_log 192834'17329 (192834'17328) modify b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by client.1883634.0:35903 2014-08-11 15:28:37.295325 2014-08-13 17:38:02.913947 7fe9bae887a0 20 read_log 192834'17340 (0'0) promote c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19686 2014-08-11 15:28:38.484617 2014-08-13 17:38:02.913966 7fe9bae887a0 20 read_log 192834'17341 (192834'17340) modify c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by client.1883634.0:36824 2014-08-11 15:28:38.456979 2014-08-13 17:38:02.913980 7fe9bae887a0 20 read_log 192834'17352 (0'0) promote 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19698 2014-08-11 15:28:38.992190 2014-08-13 17:38:02.913992 7fe9bae887a0 20 read_log 192834'17353 (192834'17352) modify 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by client.1883634.0:37118 2014-08-11 15:28:38.860677 2014-08-13 17:38:02.914005 7fe9bae887a0 20 read_log 192834'17355 (192834'16916) clean c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19701 2014-08-11 15:28:39.065968 2014-08-13 17:38:02.914017 7fe9bae887a0 20 read_log 192834'17386 (0'0) promote faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19732 2014-08-11 15:28:42.605117 2014-08-13 17:38:02.914029 7fe9bae887a0 20 read_log 192834'17387 (192834'17386) modify faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by client.1883634.0:39792 2014-08-11 15:28:42.576655 2014-08-13 17:38:02.914041 7fe9bae887a0 20 read_log 192834'17390 (0'0) promote b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19736 2014-08-11 15:28:42.825990 2014-08-13 17:38:02.914054 7fe9bae887a0 20 read_log 192834'17391 (192834'17390) modify b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by client.1883634.0:39927 2014-08-11 15:28:42.800645 2014-08-13 17:38:02.914066 7fe9bae887a0 20 read_log 192834'17402 (192834'17355) delete c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19748 2014-08-11 15:28:44.197904 2014-08-13 17:38:02.914078 7fe9bae887a0 20 read_log 192834'17411 (0'0) promote c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19757 2014-08-11 15:28:44.594850 2014-08-13 17:38:02.914090 7fe9bae887a0 20 read_log 192834'17412 (192834'17411) modify c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by client.1883634.0:41059 2014-08-11 15:28:44.568519 2014-08-13 17:38:02.914102 7fe9bae887a0 20 read_log 192834'17425 (192834'16980) modify ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by client.1883634.0:42309 2014-08-11 15:28:46.204070 2014-08-13 17:38:02.914114 7fe9bae887a0 20 read_log 192834'17429 (192834'17002) modify 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by client.1883634.0:42986 2014-08-11 15:28:46.423089 2014-08-13 17:38:02.914125 7fe9bae887a0 20 read_log 192834'17433 (192834'17020) modify 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.1883634.0:44138 2014-08-11 15:28:46.773129 2014-08-13 17:38:02.914137 7fe9bae887a0 20 read_log 192834'17438 (192834'17053) modify 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by client.1883634.0:44750 2014-08-11 15:28:46.945642 2014-08-13 17:38:02.914149 7fe9bae887a0 20 read_log 192834'17446 (192834'17080) modify ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by client.1883634.0:46174 2014-08-11 15:28:47.512546 2014-08-13 17:38:02.914166 7fe9bae887a0 20 read_log 192834'17448 (192834'17106) modify 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by client.1883634.0:46495 2014-08-11 15:28:47.610881 2014-08-13 17:38:02.914181 7fe9bae887a0 20 read_log 192834'17449 (192834'17108) modify 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by client.1883634.0:46503 2014-08-11 15:28:47.613451 2014-08-13 17:38:02.914193 7fe9bae887a0 20 read_log 192834'17460 (192834'17114) clean 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19806 2014-08-11 15:29:19.774214 2014-08-13 17:38:02.914205 7fe9bae887a0 20 read_log 192834'17470 (192834'17116) clean cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19816 2014-08-11 15:29:19.845534 2014-08-13 17:38:02.914226 7fe9bae887a0 20 read_log 192834'17475 (192834'17460) delete 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19821 2014-08-11 15:29:19.861850 2014-08-13 17:38:02.914263 7fe9bae887a0 20 read_log 192834'17480 (192834'17470) delete cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19826 2014-08-11 15:29:19.891183 2014-08-13 17:38:02.914275 7fe9bae887a0 20 read_log 192834'17524 (192834'17250) clean 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19870 2014-08-11 15:29:30.224456 2014-08-13 17:38:02.914288 7fe9bae887a0 20 read_log 192834'17527 (192834'17252) clean 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19873 2014-08-11 15:29:30.238163 2014-08-13 17:38:02.914300 7fe9bae887a0 20 read_log 192834'17528 (192834'17202) clean 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19874 2014-08-11 15:29:30.243113 2014-08-13 17:38:02.914312 7fe9bae887a0 20 read_log 192834'17552 (192834'17524) delete 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19898 2014-08-11 15:29:30.314713 2014-08-13 17:38:02.914350 7fe9bae887a0 20 read_log 192834'17555 (192834'17527) delete 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19901 2014-08-11 15:29:30.326343 2014-08-13 17:38:02.914365 7fe9bae887a0 20 read_log 192834'17556 (192834'17528) delete 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19902 2014-08-11 15:29:30.329214 2014-08-13 17:38:02.914377 7fe9bae887a0 20 read_log 192834'17574 (192834'17266) clean a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19920 2014-08-11 15:29:35.426326 2014-08-13 17:38:02.914390 7fe9bae887a0 20 read_log 192834'17582 (192834'17262) clean a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19928 2014-08-11 15:29:35.453736 2014-08-13 17:38:02.914402 7fe9bae887a0 20 read_log 192834'17596 (192834'17574) delete a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19942 2014-08-11 15:29:35.489699 2014-08-13 17:38:02.914414 7fe9bae887a0 20 read_log 192834'17603 (192834'17582) delete a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19949 2014-08-11 15:29:35.502589 2014-08-13 17:38:02.914426 7fe9bae887a0 20 read_log 192834'17619 (192834'17353) clean 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19965 2014-08-11 15:29:40.569744 2014-08-13 17:38:02.914437 7fe9bae887a0 20 read_log 192834'17620 (192834'17311) clean ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19966 2014-08-11 15:29:40.573455 2014-08-13 17:38:02.914449 7fe9bae887a0 20 read_log 192834'17628 (192834'17329) clean b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19974 2014-08-11 15:29:40.593251 2014-08-13 17:38:02.914461 7fe9bae887a0 20 read_log 192834'17629 (192834'17341) clean c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19975 2014-08-11 15:29:40.597189 2014-08-13 17:38:02.914473 7fe9bae887a0 20 read_log 192834'17640 (192834'17323) clean ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19986 2014-08-11 15:29:40.619878 2014-08-13 17:38:02.914485 7fe9bae887a0 20 read_log 192834'17647 (192834'17619) delete 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19993 2014-08-11 15:29:40.632314 2014-08-13 17:38:02.914497 7fe9bae887a0 20 read_log 192834'17648 (192834'17620) delete ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19994 2014-08-11 15:29:40.632801 2014-08-13 17:38:02.914509 7fe9bae887a0 20 read_log 192834'17656 (192834'17628) delete b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:20002 2014-08-11 15:29:40.644802 2014-08-13 17:38:02.914528 7fe9bae887a0 20 read_log 192834'17657 (192834'17629) delete c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:20003 2014-08-11 15:29:40.646693 2014-08-13 17:38:02.914541 7fe9bae887a0 20 read_log 192834'17660 (192834'17640) delete ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:20006 2014-08-11 15:29:40.652904 2014-08-13 17:38:02.914553 7fe9bae887a0 20 read_log 192834'17681 (192834'17391) clean b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:20027 2014-08-11 15:29:45.724742 2014-08-13 17:38:02.914565 7fe9bae887a0 20 read_log 192834'17689 (192834'17387) clean faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:20035 2014-08-11 15:29:45.738487 2014-08-13 17:38:02.914577 7fe9bae887a0 20 read_log 192834'17695 (192834'17412) clean c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:20041 2014-08-11 15:29:45.745526 2014-08-13 17:38:02.914589 7fe9bae887a0 20 read_log 192834'17703 (192834'17681) delete b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:20049 2014-08-11 15:29:45.755200 2014-08-13 17:38:02.914601 7fe9bae887a0 20 read_log 192834'17706 (192834'17689) delete faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:20052 2014-08-11 15:29:45.759468 2014-08-13 17:38:02.914613 7fe9bae887a0 20 read_log 192834'17707 (192834'17695) delete c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:20053 2014-08-11 15:29:45.759827 2014-08-13 17:38:02.914625 7fe9bae887a0 20 read_log 192834'17739 (192834'17449) clean 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:20085 2014-08-11 15:29:50.810472 2014-08-13 17:38:02.914637 7fe9bae887a0 20 read_log 192834'17740 (192834'17438) clean 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:20086 2014-08-11 15:29:50.810917 2014-08-13 17:38:02.914648 7fe9bae887a0 20 read_log 192834'17748 (192834'17739) delete 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:20094 2014-08-11 15:29:50.822155 2014-08-13 17:38:02.914660 7fe9bae887a0 20 read_log 192834'17749 (192834'17740) delete 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:20095 2014-08-11 15:29:50.822456 2014-08-13 17:38:02.914672 7fe9bae887a0 20 read_log 192834'17757 (192834'17446) clean ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:20103 2014-08-11 15:29:50.829348 2014-08-13 17:38:02.914684 7fe9bae887a0 20 read_log 192834'17758 (192834'17448) clean 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:20104 2014-08-11 15:29:50.829786 2014-08-13 17:38:02.914696 7fe9bae887a0 20 read_log 192834'17759 (192834'17429) clean 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:20105 2014-08-11 15:29:50.830411 2014-08-13 17:38:02.914708 7fe9bae887a0 20 read_log 192834'17761 (192834'17757) delete ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:20107 2014-08-11 15:29:50.834783 2014-08-13 17:38:02.914722 7fe9bae887a0 20 read_log 192834'17762 (192834'17425) clean ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:20108 2014-08-11 15:29:50.835046 2014-08-13 17:38:02.914735 7fe9bae887a0 20 read_log 192834'17763 (192834'17758) delete 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:20109 2014-08-11 15:29:50.835668 2014-08-13 17:38:02.914747 7fe9bae887a0 20 read_log 192834'17764 (192834'17759) delete 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:20110 2014-08-11 15:29:50.836194 2014-08-13 17:38:02.914759 7fe9bae887a0 20 read_log 192834'17765 (192834'17762) delete ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:20111 2014-08-11 15:29:50.837287 2014-08-13 17:38:02.914779 7fe9bae887a0 20 read_log 192888'17766 (192888'17766) modify 1e4/hit_set_24.1e4_archive_2014-08-11 16:03:08.430974_2014-08-11 16:45:02.244560/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:45:02.244796 2014-08-13 17:38:02.914792 7fe9bae887a0 20 read_log 192888'17767 (0'0) promote a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by osd.31.0:2131 2014-08-11 16:45:02.249826 2014-08-13 17:38:02.914804 7fe9bae887a0 20 read_log 192888'17768 (192888'17767) modify a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.1315259.0:1830 2014-08-11 16:45:02.238156 2014-08-13 17:38:02.914816 7fe9bae887a0 20 read_log 192888'17769 (0'0) promote a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by osd.31.0:2135 2014-08-11 16:45:03.925028 2014-08-13 17:38:02.914828 7fe9bae887a0 20 read_log 192888'17770 (192888'17769) modify a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.1315259.0:2285 2014-08-11 16:45:03.918533 2014-08-13 17:38:02.914840 7fe9bae887a0 20 read_log 192888'17781 (0'0) promote 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by osd.31.0:2197 2014-08-11 16:45:15.701034 2014-08-13 17:38:02.914852 7fe9bae887a0 20 read_log 192888'17782 (192888'17781) modify 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.1315259.0:5425 2014-08-11 16:45:15.693713 2014-08-13 17:38:02.914864 7fe9bae887a0 20 read_log 192888'17785 (0'0) promote 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by osd.31.0:2213 2014-08-11 16:45:20.945382 2014-08-13 17:38:02.914876 7fe9bae887a0 20 read_log 192888'17786 (192888'17785) modify 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.1315259.0:6875 2014-08-11 16:45:20.937715 2014-08-13 17:38:02.914888 7fe9bae887a0 20 read_log 192888'17795 (0'0) promote b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by osd.31.0:2241 2014-08-11 16:45:27.396639 2014-08-13 17:38:02.914900 7fe9bae887a0 20 read_log 192888'17796 (192888'17795) modify b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.1315259.0:8617 2014-08-11 16:45:27.389129 2014-08-13 17:38:02.914912 7fe9bae887a0 20 read_log 192888'17797 (0'0) promote e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by osd.31.0:2243 2014-08-11 16:45:27.486089 2014-08-13 17:38:02.914924 7fe9bae887a0 20 read_log 192888'17798 (192888'17797) modify e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.1315259.0:8642 2014-08-11 16:45:27.478748 2014-08-13 17:38:02.914936 7fe9bae887a0 20 read_log 192888'17811 (0'0) promote b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by osd.31.0:2277 2014-08-11 16:45:35.297960 2014-08-13 17:38:02.914948 7fe9bae887a0 20 read_log 192888'17812 (192888'17811) modify b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.1315259.0:10671 2014-08-11 16:45:35.290568 2014-08-13 17:38:02.914960 7fe9bae887a0 20 read_log 192888'17813 (0'0) promote 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by osd.31.0:2283 2014-08-11 16:45:36.150203 2014-08-13 17:38:02.914972 7fe9bae887a0 20 read_log 192888'17814 (192888'17813) modify 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.1315259.0:10883 2014-08-11 16:45:36.143304 2014-08-13 17:38:02.914984 7fe9bae887a0 20 read_log 192888'17827 (0'0) promote a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by osd.31.0:2333 2014-08-11 16:45:46.545990 2014-08-13 17:38:02.914996 7fe9bae887a0 20 read_log 192888'17828 (192888'17827) modify a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.1315259.0:13426 2014-08-11 16:45:46.538181 2014-08-13 17:38:02.915033 7fe9bae887a0 20 read_log 192888'17831 (0'0) promote 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by osd.31.0:2355 2014-08-11 16:45:52.679911 2014-08-13 17:38:02.915048 7fe9bae887a0 20 read_log 192888'17832 (192888'17831) modify 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.1315259.0:15033 2014-08-11 16:45:52.671670 2014-08-13 17:38:02.915060 7fe9bae887a0 20 read_log 192888'17835 (0'0) promote 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by osd.31.0:2381 2014-08-11 16:45:56.717866 2014-08-13 17:38:02.915072 7fe9bae887a0 20 read_log 192888'17836 (192888'17835) modify 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.1315259.0:16075 2014-08-11 16:45:56.706773 2014-08-13 17:38:02.915084 7fe9bae887a0 20 read_log 192888'17837 (0'0) promote 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by osd.31.0:2387 2014-08-11 16:45:58.608924 2014-08-13 17:38:02.915096 7fe9bae887a0 20 read_log 192888'17838 (192888'17837) modify 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.1315259.0:16551 2014-08-11 16:45:58.600675 2014-08-13 17:38:02.915108 7fe9bae887a0 20 read_log 192888'17839 (0'0) promote e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by osd.31.0:2391 2014-08-11 16:45:59.907606 2014-08-13 17:38:02.915119 7fe9bae887a0 20 read_log 192888'17840 (192888'17839) modify e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.1315259.0:16895 2014-08-11 16:45:59.897207 2014-08-13 17:38:02.915132 7fe9bae887a0 20 read_log 192888'17841 (0'0) promote bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by osd.31.0:2393 2014-08-11 16:46:00.681743 2014-08-13 17:38:02.915143 7fe9bae887a0 20 read_log 192888'17842 (192888'17841) modify bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.1315259.0:17081 2014-08-11 16:46:00.674152 2014-08-13 17:38:02.915162 7fe9bae887a0 20 read_log 192888'17843 (0'0) promote 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by osd.31.0:2395 2014-08-11 16:46:00.953151 2014-08-13 17:38:02.915175 7fe9bae887a0 20 read_log 192888'17844 (192888'17843) modify 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.1315259.0:17154 2014-08-11 16:46:00.945287 2014-08-13 17:38:02.915187 7fe9bae887a0 20 read_log 192888'17849 (0'0) promote f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by osd.31.0:2431 2014-08-11 16:46:11.056852 2014-08-13 17:38:02.915199 7fe9bae887a0 20 read_log 192888'17850 (192888'17849) modify f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.1315259.0:19709 2014-08-11 16:46:11.048223 2014-08-13 17:38:02.915211 7fe9bae887a0 20 read_log 192888'17851 (0'0) promote e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by osd.31.0:2443 2014-08-11 16:46:13.260717 2014-08-13 17:38:02.915223 7fe9bae887a0 20 read_log 192888'17852 (192888'17851) modify e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.1315259.0:20309 2014-08-11 16:46:13.251914 2014-08-13 17:38:02.915235 7fe9bae887a0 20 read_log 192888'17865 (0'0) promote 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by osd.31.0:2487 2014-08-11 16:46:18.767931 2014-08-13 17:38:02.915247 7fe9bae887a0 20 read_log 192888'17866 (192888'17865) modify 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.1315259.0:21760 2014-08-11 16:46:18.759811 2014-08-13 17:38:02.915259 7fe9bae887a0 20 read_log 192888'17867 (0'0) promote 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by osd.31.0:2489 2014-08-11 16:46:18.771884 2014-08-13 17:38:02.915271 7fe9bae887a0 20 read_log 192888'17868 (192888'17867) modify 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.1315259.0:21761 2014-08-11 16:46:18.763293 2014-08-13 17:38:02.915292 7fe9bae887a0 20 read_log 192888'17871 (0'0) promote d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by osd.31.0:2499 2014-08-11 16:46:22.435297 2014-08-13 17:38:02.915305 7fe9bae887a0 20 read_log 192888'17872 (192888'17871) modify d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.1315259.0:22704 2014-08-11 16:46:22.427213 2014-08-13 17:38:02.915318 7fe9bae887a0 20 read_log 192888'17873 (0'0) promote 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by osd.31.0:2509 2014-08-11 16:46:26.731426 2014-08-13 17:38:02.915329 7fe9bae887a0 20 read_log 192888'17874 (192888'17873) modify 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.1315259.0:23760 2014-08-11 16:46:26.722464 2014-08-13 17:38:02.915342 7fe9bae887a0 20 read_log 192888'17875 (0'0) promote 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by osd.31.0:2511 2014-08-11 16:46:27.611115 2014-08-13 17:38:02.915354 7fe9bae887a0 20 read_log 192888'17876 (192888'17875) modify 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.1315259.0:23993 2014-08-11 16:46:27.601262 2014-08-13 17:38:02.915366 7fe9bae887a0 20 read_log 192888'17877 (0'0) promote c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by osd.31.0:2521 2014-08-11 16:46:28.912507 2014-08-13 17:38:02.915378 7fe9bae887a0 20 read_log 192888'17878 (192888'17877) modify c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.1315259.0:24340 2014-08-11 16:46:28.905836 2014-08-13 17:38:02.915390 7fe9bae887a0 20 read_log 192888'17879 (0'0) promote 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by osd.31.0:2523 2014-08-11 16:46:30.408840 2014-08-13 17:38:02.915402 7fe9bae887a0 20 read_log 192888'17880 (192888'17879) modify 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.1315259.0:24711 2014-08-11 16:46:30.401385 2014-08-13 17:38:02.915414 7fe9bae887a0 20 read_log 192888'17883 (0'0) promote 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by osd.31.0:2539 2014-08-11 16:46:35.933006 2014-08-13 17:38:02.915425 7fe9bae887a0 20 read_log 192888'17884 (192888'17883) modify 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.1315259.0:26054 2014-08-11 16:46:35.923801 2014-08-13 17:38:02.915438 7fe9bae887a0 20 read_log 192888'17885 (0'0) promote 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by osd.31.0:2553 2014-08-11 16:46:38.241385 2014-08-13 17:38:02.915449 7fe9bae887a0 20 read_log 192888'17886 (192888'17885) modify 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.1315259.0:26590 2014-08-11 16:46:38.233633 2014-08-13 17:38:02.915461 7fe9bae887a0 20 read_log 192888'17893 (0'0) promote 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by osd.31.0:2569 2014-08-11 16:46:43.733048 2014-08-13 17:38:02.915473 7fe9bae887a0 20 read_log 192888'17894 (192888'17893) modify 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.1315259.0:27942 2014-08-11 16:46:43.726080 2014-08-13 17:38:02.915485 7fe9bae887a0 20 read_log 192888'17897 (0'0) promote 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by osd.31.0:2585 2014-08-11 16:46:45.749028 2014-08-13 17:38:02.915497 7fe9bae887a0 20 read_log 192888'17898 (192888'17897) modify 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.1315259.0:28426 2014-08-11 16:46:45.740544 2014-08-13 17:38:02.915509 7fe9bae887a0 20 read_log 192888'17901 (0'0) promote 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by osd.31.0:2593 2014-08-11 16:46:46.828513 2014-08-13 17:38:02.915521 7fe9bae887a0 20 read_log 192888'17902 (192888'17901) modify 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.1315259.0:28687 2014-08-11 16:46:46.820907 2014-08-13 17:38:02.915557 7fe9bae887a0 20 read_log 192888'17903 (0'0) promote bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by osd.31.0:2601 2014-08-11 16:46:50.252011 2014-08-13 17:38:02.915570 7fe9bae887a0 20 read_log 192888'17904 (192888'17903) modify bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.1315259.0:29528 2014-08-11 16:46:50.244491 2014-08-13 17:38:02.915583 7fe9bae887a0 20 read_log 192888'17911 (192888'17768) modify a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.1315259.0:32557 2014-08-11 16:46:56.409629 2014-08-13 17:38:02.915595 7fe9bae887a0 20 read_log 192888'17912 (192888'17770) modify a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.1315259.0:33012 2014-08-11 16:46:56.531923 2014-08-13 17:38:02.915607 7fe9bae887a0 20 read_log 192888'17918 (192888'17782) modify 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.1315259.0:36152 2014-08-11 16:46:57.350510 2014-08-13 17:38:02.915619 7fe9bae887a0 20 read_log 192888'17920 (192888'17786) modify 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.1315259.0:37602 2014-08-11 16:46:57.727081 2014-08-13 17:38:02.915634 7fe9bae887a0 20 read_log 192888'17925 (192888'17796) modify b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.1315259.0:39344 2014-08-11 16:46:58.187124 2014-08-13 17:38:02.915647 7fe9bae887a0 20 read_log 192888'17926 (192888'17798) modify e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.1315259.0:39369 2014-08-11 16:46:58.192959 2014-08-13 17:38:02.915661 7fe9bae887a0 20 read_log 192888'17933 (192888'17812) modify b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.1315259.0:41398 2014-08-11 16:46:58.735798 2014-08-13 17:38:02.915673 7fe9bae887a0 20 read_log 192888'17934 (192888'17814) modify 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.1315259.0:41610 2014-08-11 16:46:58.790061 2014-08-13 17:38:02.915687 7fe9bae887a0 20 read_log 192888'17941 (192888'17828) modify a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.1315259.0:44153 2014-08-11 16:46:59.496286 2014-08-13 17:38:02.915699 7fe9bae887a0 20 read_log 192888'17943 (192888'17832) modify 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.1315259.0:45760 2014-08-11 16:46:59.961449 2014-08-13 17:38:02.915713 7fe9bae887a0 20 read_log 192888'17945 (192888'17836) modify 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.1315259.0:46802 2014-08-11 16:47:00.262295 2014-08-13 17:38:02.915725 7fe9bae887a0 20 read_log 192888'17946 (192888'17838) modify 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.1315259.0:47278 2014-08-11 16:47:00.403793 2014-08-13 17:38:02.915739 7fe9bae887a0 20 read_log 192888'17947 (192888'17840) modify e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.1315259.0:47622 2014-08-11 16:47:00.513336 2014-08-13 17:38:02.915751 7fe9bae887a0 20 read_log 192888'17948 (192888'17842) modify bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.1315259.0:47808 2014-08-11 16:47:00.572772 2014-08-13 17:38:02.915765 7fe9bae887a0 20 read_log 192888'17949 (192888'17844) modify 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.1315259.0:47881 2014-08-11 16:47:00.597560 2014-08-13 17:38:02.915777 7fe9bae887a0 20 read_log 192888'17952 (192888'17850) modify f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.1315259.0:50436 2014-08-11 16:47:01.369122 2014-08-13 17:38:02.915791 7fe9bae887a0 20 read_log 192888'17953 (192888'17852) modify e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.1315259.0:51036 2014-08-11 16:47:01.559397 2014-08-13 17:38:02.915803 7fe9bae887a0 20 read_log 192888'17960 (192888'17866) modify 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.1315259.0:52487 2014-08-11 16:47:01.987003 2014-08-13 17:38:02.915825 7fe9bae887a0 20 read_log 192888'17961 (192888'17868) modify 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.1315259.0:52488 2014-08-11 16:47:01.987109 2014-08-13 17:38:02.915839 7fe9bae887a0 20 read_log 192888'17963 (192888'17872) modify d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.1315259.0:53431 2014-08-11 16:47:02.249621 2014-08-13 17:38:02.915852 7fe9bae887a0 20 read_log 192888'17964 (192888'17874) modify 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.1315259.0:54487 2014-08-11 16:47:02.579560 2014-08-13 17:38:02.915867 7fe9bae887a0 20 read_log 192888'17965 (192888'17876) modify 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.1315259.0:54720 2014-08-11 16:47:02.665717 2014-08-13 17:38:02.915884 7fe9bae887a0 20 read_log 192888'17966 (192888'17878) modify c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.1315259.0:55067 2014-08-11 16:47:02.801493 2014-08-13 17:38:02.915899 7fe9bae887a0 20 read_log 192888'17967 (192888'17880) modify 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.1315259.0:55438 2014-08-11 16:47:02.912795 2014-08-13 17:38:02.915911 7fe9bae887a0 20 read_log 192888'17969 (192888'17884) modify 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.1315259.0:56781 2014-08-11 16:47:03.310461 2014-08-13 17:38:02.915925 7fe9bae887a0 20 read_log 192888'17970 (192888'17886) modify 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.1315259.0:57317 2014-08-11 16:47:03.474301 2014-08-13 17:38:02.915937 7fe9bae887a0 20 read_log 192888'17974 (192888'17894) modify 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.1315259.0:58669 2014-08-11 16:47:03.869734 2014-08-13 17:38:02.915952 7fe9bae887a0 20 read_log 192888'17976 (192888'17898) modify 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.1315259.0:59153 2014-08-11 16:47:04.017876 2014-08-13 17:38:02.915964 7fe9bae887a0 20 read_log 192888'17978 (192888'17902) modify 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.1315259.0:59414 2014-08-11 16:47:04.107668 2014-08-13 17:38:02.915977 7fe9bae887a0 20 read_log 192888'17979 (192888'17904) modify bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.1315259.0:60255 2014-08-11 16:47:04.394547 2014-08-13 17:38:02.915990 7fe9bae887a0 20 read_log 192888'17983 (192888'17983) modify 1e4/hit_set_24.1e4_archive_2014-08-11 16:45:02.244800_2014-08-11 17:05:16.516717/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.516943 2014-08-13 17:38:02.916005 7fe9bae887a0 20 read_log 192888'17984 (192888'17766) delete 1e4/hit_set_24.1e4_archive_2014-08-11 16:03:08.430974_2014-08-11 16:45:02.244560/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.516943 2014-08-13 17:38:02.916017 7fe9bae887a0 20 read_log 192888'17985 (0'0) promote c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by osd.31.0:2894 2014-08-11 17:05:16.520366 2014-08-13 17:38:02.916031 7fe9bae887a0 20 read_log 192888'17986 (192888'17985) modify c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.942868.0:359 2014-08-11 17:05:16.509882 2014-08-13 17:38:02.916044 7fe9bae887a0 20 read_log 192888'17987 (0'0) promote 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by osd.31.0:2899 2014-08-11 17:05:17.931406 2014-08-13 17:38:02.916058 7fe9bae887a0 20 read_log 192888'17988 (192888'17987) modify 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.942868.0:722 2014-08-11 17:05:17.923502 2014-08-13 17:38:02.916070 7fe9bae887a0 20 read_log 192888'17989 (0'0) promote 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by osd.31.0:2901 2014-08-11 17:05:18.704982 2014-08-13 17:38:02.916090 7fe9bae887a0 20 read_log 192888'17990 (192888'17989) modify 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.942868.0:901 2014-08-11 17:05:18.697764 2014-08-13 17:38:02.916105 7fe9bae887a0 20 read_log 192888'17993 (0'0) promote d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by osd.31.0:2914 2014-08-11 17:05:20.035892 2014-08-13 17:38:02.916117 7fe9bae887a0 20 read_log 192888'17994 (192888'17993) modify d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.942868.0:1218 2014-08-11 17:05:20.027053 2014-08-13 17:38:02.916132 7fe9bae887a0 20 read_log 192888'17997 (0'0) promote abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by osd.31.0:2944 2014-08-11 17:05:31.250967 2014-08-13 17:38:02.916145 7fe9bae887a0 20 read_log 192888'17998 (192888'17997) modify abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.942868.0:4140 2014-08-11 17:05:31.243738 2014-08-13 17:38:02.916164 7fe9bae887a0 20 read_log 192888'18003 (0'0) promote d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by osd.31.0:2972 2014-08-11 17:05:39.982775 2014-08-13 17:38:02.916178 7fe9bae887a0 20 read_log 192888'18004 (192888'18003) modify d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.942868.0:6462 2014-08-11 17:05:39.977011 2014-08-13 17:38:02.916190 7fe9bae887a0 20 read_log 192888'18009 (0'0) promote b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by osd.31.0:2998 2014-08-11 17:05:46.794550 2014-08-13 17:38:02.916202 7fe9bae887a0 20 read_log 192888'18010 (192888'18009) modify b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.942868.0:8267 2014-08-11 17:05:46.783250 2014-08-13 17:38:02.916214 7fe9bae887a0 20 read_log 192888'18015 (0'0) promote 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by osd.31.0:3026 2014-08-11 17:05:51.874354 2014-08-13 17:38:02.916226 7fe9bae887a0 20 read_log 192888'18016 (192888'18015) modify 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.942868.0:9632 2014-08-11 17:05:51.866563 2014-08-13 17:38:02.916238 7fe9bae887a0 20 read_log 192888'18017 (0'0) promote 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by osd.31.0:3028 2014-08-11 17:05:52.189616 2014-08-13 17:38:02.916250 7fe9bae887a0 20 read_log 192888'18018 (192888'18017) modify 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.942868.0:9720 2014-08-11 17:05:52.182881 2014-08-13 17:38:02.916262 7fe9bae887a0 20 read_log 192888'18019 (0'0) promote 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by osd.31.0:3032 2014-08-11 17:05:53.004278 2014-08-13 17:38:02.916274 7fe9bae887a0 20 read_log 192888'18020 (192888'18019) modify 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.942868.0:9944 2014-08-11 17:05:52.996689 2014-08-13 17:38:02.916288 7fe9bae887a0 20 read_log 192888'18021 (0'0) promote 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by osd.31.0:3056 2014-08-11 17:05:58.906902 2014-08-13 17:38:02.916300 7fe9bae887a0 20 read_log 192888'18022 (192888'18021) modify 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.942868.0:11566 2014-08-11 17:05:58.900323 2014-08-13 17:38:02.916315 7fe9bae887a0 20 read_log 192888'18025 (0'0) promote e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by osd.31.0:3066 2014-08-11 17:06:02.031120 2014-08-13 17:38:02.916327 7fe9bae887a0 20 read_log 192888'18026 (192888'18025) modify e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.942868.0:12426 2014-08-11 17:06:02.023702 2014-08-13 17:38:02.916341 7fe9bae887a0 20 read_log 192888'18027 (0'0) promote db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by osd.31.0:3068 2014-08-11 17:06:02.190415 2014-08-13 17:38:02.916360 7fe9bae887a0 20 read_log 192888'18028 (192888'18027) modify db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.942868.0:12471 2014-08-11 17:06:02.183824 2014-08-13 17:38:02.916376 7fe9bae887a0 20 read_log 192888'18033 (0'0) promote e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by osd.31.0:3078 2014-08-11 17:06:04.506098 2014-08-13 17:38:02.916388 7fe9bae887a0 20 read_log 192888'18034 (192888'18033) modify e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.942868.0:13088 2014-08-11 17:06:04.498535 2014-08-13 17:38:02.916400 7fe9bae887a0 20 read_log 192888'18037 (0'0) promote c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by osd.31.0:3090 2014-08-11 17:06:08.232513 2014-08-13 17:38:02.916412 7fe9bae887a0 20 read_log 192888'18038 (192888'18037) modify c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.942868.0:14009 2014-08-11 17:06:08.224609 2014-08-13 17:38:02.916424 7fe9bae887a0 20 read_log 192888'18041 (0'0) promote 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by osd.31.0:3118 2014-08-11 17:06:15.584015 2014-08-13 17:38:02.916436 7fe9bae887a0 20 read_log 192888'18042 (192888'18041) modify 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.942868.0:15864 2014-08-11 17:06:15.576443 2014-08-13 17:38:02.916448 7fe9bae887a0 20 read_log 192888'18043 (0'0) promote c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by osd.31.0:3122 2014-08-11 17:06:16.620605 2014-08-13 17:38:02.916460 7fe9bae887a0 20 read_log 192888'18044 (192888'18043) modify c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.942868.0:16133 2014-08-11 17:06:16.613363 2014-08-13 17:38:02.916472 7fe9bae887a0 20 read_log 192888'18047 (0'0) promote 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by osd.31.0:3140 2014-08-11 17:06:19.850972 2014-08-13 17:38:02.916484 7fe9bae887a0 20 read_log 192888'18048 (192888'18047) modify 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.942868.0:16944 2014-08-11 17:06:19.843399 2014-08-13 17:38:02.916496 7fe9bae887a0 20 read_log 192888'18049 (0'0) promote 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by osd.31.0:3144 2014-08-11 17:06:20.941598 2014-08-13 17:38:02.916508 7fe9bae887a0 20 read_log 192888'18050 (192888'18049) modify 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.942868.0:17232 2014-08-11 17:06:20.933591 2014-08-13 17:38:02.916520 7fe9bae887a0 20 read_log 192888'18053 (0'0) promote e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by osd.31.0:3152 2014-08-11 17:06:22.978611 2014-08-13 17:38:02.916532 7fe9bae887a0 20 read_log 192888'18054 (192888'18053) modify e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.942868.0:17772 2014-08-11 17:06:22.970337 2014-08-13 17:38:02.916544 7fe9bae887a0 20 read_log 192888'18057 (0'0) promote 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by osd.31.0:3172 2014-08-11 17:06:28.344584 2014-08-13 17:38:02.916556 7fe9bae887a0 20 read_log 192888'18058 (192888'18057) modify 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.942868.0:19197 2014-08-11 17:06:28.336943 2014-08-13 17:38:02.916569 7fe9bae887a0 20 read_log 192888'18059 (0'0) promote 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by osd.31.0:3192 2014-08-11 17:06:31.183532 2014-08-13 17:38:02.916581 7fe9bae887a0 20 read_log 192888'18060 (192888'18059) modify 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.942868.0:19950 2014-08-11 17:06:31.172853 2014-08-13 17:38:02.916598 7fe9bae887a0 20 read_log 192888'18071 (0'0) promote ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by osd.31.0:3246 2014-08-11 17:06:51.076772 2014-08-13 17:38:02.916619 7fe9bae887a0 20 read_log 192888'18072 (192888'18071) modify ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.942868.0:25178 2014-08-11 17:06:51.069971 2014-08-13 17:38:02.916635 7fe9bae887a0 20 read_log 192888'18075 (0'0) promote 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by osd.31.0:3252 2014-08-11 17:06:52.606360 2014-08-13 17:38:02.916647 7fe9bae887a0 20 read_log 192888'18076 (192888'18075) modify 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.942868.0:25581 2014-08-11 17:06:52.599199 2014-08-13 17:38:02.916659 7fe9bae887a0 20 read_log 192888'18077 (0'0) promote 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by osd.31.0:3260 2014-08-11 17:06:54.846843 2014-08-13 17:38:02.916671 7fe9bae887a0 20 read_log 192888'18078 (192888'18077) modify 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.942868.0:26102 2014-08-11 17:06:54.838639 2014-08-13 17:38:02.916683 7fe9bae887a0 20 read_log 192888'18079 (0'0) promote 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by osd.31.0:3264 2014-08-11 17:06:55.006914 2014-08-13 17:38:02.916695 7fe9bae887a0 20 read_log 192888'18080 (192888'18079) modify 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.942868.0:26147 2014-08-11 17:06:54.999961 2014-08-13 17:38:02.916707 7fe9bae887a0 20 read_log 192888'18081 (0'0) promote 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by osd.31.0:3268 2014-08-11 17:06:56.109580 2014-08-13 17:38:02.916720 7fe9bae887a0 20 read_log 192888'18082 (192888'18081) modify 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.942868.0:26443 2014-08-11 17:06:56.101921 2014-08-13 17:38:02.916732 7fe9bae887a0 20 read_log 192888'18089 (0'0) promote 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by osd.31.0:3294 2014-08-11 17:07:01.184082 2014-08-13 17:38:02.916744 7fe9bae887a0 20 read_log 192888'18090 (192888'18089) modify 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.942868.0:27779 2014-08-11 17:07:01.174097 2014-08-13 17:38:02.916756 7fe9bae887a0 20 read_log 192888'18099 (0'0) promote 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by osd.31.0:3324 2014-08-11 17:07:08.453502 2014-08-13 17:38:02.916768 7fe9bae887a0 20 read_log 192888'18100 (192888'18099) modify 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.942868.0:29660 2014-08-11 17:07:08.445512 2014-08-13 17:38:02.916780 7fe9bae887a0 20 read_log 192888'18101 (0'0) promote e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by osd.31.0:3330 2014-08-11 17:07:09.821007 2014-08-13 17:38:02.916795 7fe9bae887a0 20 read_log 192888'18102 (192888'18101) modify e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.942868.0:30002 2014-08-11 17:07:09.811834 2014-08-13 17:38:02.916807 7fe9bae887a0 20 read_log 192888'18103 (0'0) promote b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by osd.31.0:3332 2014-08-11 17:07:10.853302 2014-08-13 17:38:02.916819 7fe9bae887a0 20 read_log 192888'18104 (192888'18103) modify b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.942868.0:30252 2014-08-11 17:07:10.845105 2014-08-13 17:38:02.916832 7fe9bae887a0 20 read_log 192888'18105 (192888'17986) modify c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.942868.0:31625 2014-08-11 17:07:16.001800 2014-08-13 17:38:02.916844 7fe9bae887a0 20 read_log 192888'18106 (192888'17988) modify 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.942868.0:31988 2014-08-11 17:07:16.118175 2014-08-13 17:38:02.916856 7fe9bae887a0 20 read_log 192888'18107 (192888'17990) modify 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.942868.0:32167 2014-08-11 17:07:16.172482 2014-08-13 17:38:02.916894 7fe9bae887a0 20 read_log 192888'18109 (192888'17994) modify d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.942868.0:32484 2014-08-11 17:07:16.267295 2014-08-13 17:38:02.916910 7fe9bae887a0 20 read_log 192888'18111 (192888'17998) modify abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.942868.0:35406 2014-08-11 17:07:17.065860 2014-08-13 17:38:02.916922 7fe9bae887a0 20 read_log 192888'18114 (192888'18004) modify d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.942868.0:37728 2014-08-11 17:07:17.690511 2014-08-13 17:38:02.916935 7fe9bae887a0 20 read_log 192888'18117 (192888'18010) modify b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.942868.0:39533 2014-08-11 17:07:18.209032 2014-08-13 17:38:02.916947 7fe9bae887a0 20 read_log 192888'18120 (192888'18016) modify 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.942868.0:40898 2014-08-11 17:07:18.576245 2014-08-13 17:38:02.916959 7fe9bae887a0 20 read_log 192888'18121 (192888'18018) modify 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.942868.0:40986 2014-08-11 17:07:18.600033 2014-08-13 17:38:02.916971 7fe9bae887a0 20 read_log 192888'18122 (192888'18020) modify 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.942868.0:41210 2014-08-11 17:07:18.671024 2014-08-13 17:38:02.916983 7fe9bae887a0 20 read_log 192888'18123 (192888'18022) modify 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.942868.0:42832 2014-08-11 17:07:19.104278 2014-08-13 17:38:02.916995 7fe9bae887a0 20 read_log 192888'18125 (192888'18026) modify e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.942868.0:43692 2014-08-11 17:07:19.348155 2014-08-13 17:38:02.917007 7fe9bae887a0 20 read_log 192888'18126 (192888'18028) modify db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.942868.0:43737 2014-08-11 17:07:19.360506 2014-08-13 17:38:02.917020 7fe9bae887a0 20 read_log 192888'18129 (192888'18034) modify e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.942868.0:44354 2014-08-11 17:07:19.544499 2014-08-13 17:38:02.917032 7fe9bae887a0 20 read_log 192888'18131 (192888'18038) modify c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.942868.0:45275 2014-08-11 17:07:19.793275 2014-08-13 17:38:02.917044 7fe9bae887a0 20 read_log 192888'18133 (192888'18042) modify 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.942868.0:47130 2014-08-11 17:07:20.317891 2014-08-13 17:38:02.917056 7fe9bae887a0 20 read_log 192888'18134 (192888'18044) modify c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.942868.0:47399 2014-08-11 17:07:20.393492 2014-08-13 17:38:02.917068 7fe9bae887a0 20 read_log 192888'18136 (192888'18048) modify 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.942868.0:48210 2014-08-11 17:07:20.655744 2014-08-13 17:38:02.917080 7fe9bae887a0 20 read_log 192888'18137 (192888'18050) modify 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.942868.0:48498 2014-08-11 17:07:20.735290 2014-08-13 17:38:02.917092 7fe9bae887a0 20 read_log 192888'18139 (192888'18054) modify e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.942868.0:49038 2014-08-11 17:07:20.890049 2014-08-13 17:38:02.917104 7fe9bae887a0 20 read_log 192888'18141 (192888'18058) modify 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.942868.0:50463 2014-08-11 17:07:21.326204 2014-08-13 17:38:02.917116 7fe9bae887a0 20 read_log 192888'18142 (192888'18060) modify 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.942868.0:51216 2014-08-11 17:07:21.574852 2014-08-13 17:38:02.917128 7fe9bae887a0 20 read_log 192888'18148 (192888'18072) modify ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.942868.0:56444 2014-08-11 17:07:23.150459 2014-08-13 17:38:02.917158 7fe9bae887a0 20 read_log 192888'18150 (192888'18076) modify 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.942868.0:56847 2014-08-11 17:07:23.277623 2014-08-13 17:38:02.917175 7fe9bae887a0 20 read_log 192888'18151 (192888'18078) modify 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.942868.0:57368 2014-08-11 17:07:23.435994 2014-08-13 17:38:02.917187 7fe9bae887a0 20 read_log 192888'18152 (192888'18080) modify 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.942868.0:57413 2014-08-11 17:07:23.448283 2014-08-13 17:38:02.917199 7fe9bae887a0 20 read_log 192888'18153 (192888'18082) modify 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.942868.0:57709 2014-08-11 17:07:23.534515 2014-08-13 17:38:02.917211 7fe9bae887a0 20 read_log 192888'18157 (192888'18090) modify 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.942868.0:59045 2014-08-11 17:07:23.927890 2014-08-13 17:38:02.917224 7fe9bae887a0 20 read_log 192888'18162 (192888'18100) modify 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.942868.0:60926 2014-08-11 17:07:24.485568 2014-08-13 17:38:02.917236 7fe9bae887a0 20 read_log 192888'18163 (192888'18102) modify e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.942868.0:61268 2014-08-11 17:07:24.589518 2014-08-13 17:38:02.917248 7fe9bae887a0 20 read_log 192888'18164 (192888'18104) modify b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.942868.0:61518 2014-08-11 17:07:24.668425 2014-08-13 17:38:02.917261 7fe9bae887a0 20 read_log 193305'18165 (193305'18165) modify 1e4/hit_set_24.1e4_archive_2014-08-11 18:37:04.917636_2014-08-11 18:42:34.237231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:34.237452 2014-08-13 17:38:02.917273 7fe9bae887a0 20 read_log 193305'18166 (192888'17983) delete 1e4/hit_set_24.1e4_archive_2014-08-11 16:45:02.244800_2014-08-11 17:05:16.516717/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:34.237452 2014-08-13 17:38:02.917286 7fe9bae887a0 20 read_log 193305'18167 (0'0) promote 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by osd.31.0:7 2014-08-11 18:42:34.267899 2014-08-13 17:38:02.917298 7fe9bae887a0 20 read_log 193305'18168 (193305'18167) modify 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.951337.0:331 2014-08-11 18:42:34.231959 2014-08-13 17:38:02.917310 7fe9bae887a0 20 read_log 193305'18169 (0'0) promote c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by osd.31.0:16 2014-08-11 18:42:37.010307 2014-08-13 17:38:02.917323 7fe9bae887a0 20 read_log 193305'18170 (193305'18169) modify c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.951337.0:1010 2014-08-11 18:42:37.003295 2014-08-13 17:38:02.917335 7fe9bae887a0 20 read_log 193305'18171 (0'0) promote d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by osd.31.0:58 2014-08-11 18:42:46.630320 2014-08-13 17:38:02.917347 7fe9bae887a0 20 read_log 193305'18172 (193305'18171) modify d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.951337.0:3618 2014-08-11 18:42:46.623841 2014-08-13 17:38:02.917359 7fe9bae887a0 20 read_log 193305'18173 (0'0) promote 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by osd.31.0:64 2014-08-11 18:42:47.990527 2014-08-13 17:38:02.917371 7fe9bae887a0 20 read_log 193305'18174 (193305'18173) modify 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.951337.0:3990 2014-08-11 18:42:47.982858 2014-08-13 17:38:02.917383 7fe9bae887a0 20 read_log 193305'18175 (0'0) promote 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by osd.31.0:66 2014-08-11 18:42:48.559598 2014-08-13 17:38:02.917405 7fe9bae887a0 20 read_log 193305'18176 (193305'18175) modify 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.951337.0:4147 2014-08-11 18:42:48.551890 2014-08-13 17:38:02.917448 7fe9bae887a0 20 read_log 193305'18177 (0'0) promote d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by osd.31.0:70 2014-08-11 18:42:48.912991 2014-08-13 17:38:02.917465 7fe9bae887a0 20 read_log 193305'18178 (193305'18177) modify d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.951337.0:4244 2014-08-11 18:42:48.905748 2014-08-13 17:38:02.917478 7fe9bae887a0 20 read_log 193305'18179 (0'0) promote f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by osd.31.0:74 2014-08-11 18:42:49.095467 2014-08-13 17:38:02.917490 7fe9bae887a0 20 read_log 193305'18180 (193305'18179) modify f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.951337.0:4294 2014-08-11 18:42:49.086941 2014-08-13 17:38:02.917502 7fe9bae887a0 20 read_log 193305'18181 (0'0) promote 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by osd.31.0:106 2014-08-11 18:42:56.453014 2014-08-13 17:38:02.917514 7fe9bae887a0 20 read_log 193305'18182 (193305'18181) modify 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.951337.0:6282 2014-08-11 18:42:56.445764 2014-08-13 17:38:02.917526 7fe9bae887a0 20 read_log 193305'18183 (0'0) promote d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by osd.31.0:138 2014-08-11 18:43:11.196706 2014-08-13 17:38:02.917538 7fe9bae887a0 20 read_log 193305'18184 (193305'18183) modify d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.951337.0:10250 2014-08-11 18:43:11.188237 2014-08-13 17:38:02.917550 7fe9bae887a0 20 read_log 193305'18185 (0'0) promote 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by osd.31.0:158 2014-08-11 18:43:18.505721 2014-08-13 17:38:02.917562 7fe9bae887a0 20 read_log 193305'18186 (193305'18185) modify 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.951337.0:12168 2014-08-11 18:43:18.498819 2014-08-13 17:38:02.917575 7fe9bae887a0 20 read_log 193305'18187 (0'0) promote bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by osd.31.0:160 2014-08-11 18:43:20.297074 2014-08-13 17:38:02.917586 7fe9bae887a0 20 read_log 193305'18188 (193305'18187) modify bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.951337.0:12654 2014-08-11 18:43:20.288746 2014-08-13 17:38:02.917598 7fe9bae887a0 20 read_log 193305'18189 (0'0) promote 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by osd.31.0:162 2014-08-11 18:43:22.848666 2014-08-13 17:38:02.917611 7fe9bae887a0 20 read_log 193305'18190 (193305'18189) modify 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.951337.0:13326 2014-08-11 18:43:22.841204 2014-08-13 17:38:02.917623 7fe9bae887a0 20 read_log 193305'18191 (0'0) promote 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by osd.31.0:164 2014-08-11 18:43:22.978077 2014-08-13 17:38:02.917635 7fe9bae887a0 20 read_log 193305'18192 (193305'18191) modify 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.951337.0:13362 2014-08-11 18:43:22.969970 2014-08-13 17:38:02.917647 7fe9bae887a0 20 read_log 193305'18193 (0'0) promote 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by osd.31.0:170 2014-08-11 18:43:23.432693 2014-08-13 17:38:02.917659 7fe9bae887a0 20 read_log 193305'18194 (193305'18193) modify 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.951337.0:13462 2014-08-11 18:43:23.425495 2014-08-13 17:38:02.917674 7fe9bae887a0 20 read_log 193305'18195 (0'0) promote 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by osd.31.0:174 2014-08-11 18:43:24.301995 2014-08-13 17:38:02.917693 7fe9bae887a0 20 read_log 193305'18196 (193305'18195) modify 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.951337.0:13694 2014-08-11 18:43:24.293864 2014-08-13 17:38:02.917705 7fe9bae887a0 20 read_log 193305'18197 (0'0) promote c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by osd.31.0:188 2014-08-11 18:43:27.459047 2014-08-13 17:38:02.917718 7fe9bae887a0 20 read_log 193305'18198 (193305'18197) modify c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.951337.0:14567 2014-08-11 18:43:27.450155 2014-08-13 17:38:02.917730 7fe9bae887a0 20 read_log 193305'18199 (0'0) promote c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by osd.31.0:192 2014-08-11 18:43:29.690539 2014-08-13 17:38:02.917742 7fe9bae887a0 20 read_log 193305'18200 (193305'18199) modify c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.951337.0:15216 2014-08-11 18:43:29.682105 2014-08-13 17:38:02.917754 7fe9bae887a0 20 read_log 193305'18201 (0'0) promote 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by osd.31.0:204 2014-08-11 18:43:35.132495 2014-08-13 17:38:02.917768 7fe9bae887a0 20 read_log 193305'18202 (193305'18201) modify 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.951337.0:16783 2014-08-11 18:43:35.122772 2014-08-13 17:38:02.917780 7fe9bae887a0 20 read_log 193305'18203 (0'0) promote 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by osd.31.0:208 2014-08-11 18:43:38.257614 2014-08-13 17:38:02.917792 7fe9bae887a0 20 read_log 193305'18204 (193305'18203) modify 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.951337.0:17691 2014-08-11 18:43:38.251229 2014-08-13 17:38:02.917805 7fe9bae887a0 20 read_log 193305'18205 (0'0) promote d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by osd.31.0:210 2014-08-11 18:43:39.344924 2014-08-13 17:38:02.917817 7fe9bae887a0 20 read_log 193305'18206 (193305'18205) modify d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.951337.0:17989 2014-08-11 18:43:39.336680 2014-08-13 17:38:02.917829 7fe9bae887a0 20 read_log 193305'18207 (0'0) promote f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by osd.31.0:232 2014-08-11 18:43:44.460325 2014-08-13 17:38:02.917841 7fe9bae887a0 20 read_log 193305'18208 (193305'18207) modify f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.951337.0:19385 2014-08-11 18:43:44.452360 2014-08-13 17:38:02.917853 7fe9bae887a0 20 read_log 193305'18209 (0'0) promote a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by osd.31.0:246 2014-08-11 18:43:45.924327 2014-08-13 17:38:02.917865 7fe9bae887a0 20 read_log 193305'18210 (193305'18209) modify a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.951337.0:19806 2014-08-11 18:43:45.917125 2014-08-13 17:38:02.917877 7fe9bae887a0 20 read_log 193305'18211 (0'0) promote 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by osd.31.0:250 2014-08-11 18:43:47.442440 2014-08-13 17:38:02.917889 7fe9bae887a0 20 read_log 193305'18212 (193305'18211) modify 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.951337.0:20210 2014-08-11 18:43:47.435108 2014-08-13 17:38:02.917902 7fe9bae887a0 20 read_log 193305'18213 (0'0) promote 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by osd.31.0:262 2014-08-11 18:43:49.693622 2014-08-13 17:38:02.917914 7fe9bae887a0 20 read_log 193305'18214 (193305'18213) modify 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.951337.0:20858 2014-08-11 18:43:49.686252 2014-08-13 17:38:02.917926 7fe9bae887a0 20 read_log 193305'18215 (0'0) promote d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by osd.31.0:272 2014-08-11 18:43:50.976101 2014-08-13 17:38:02.917945 7fe9bae887a0 20 read_log 193305'18216 (193305'18215) modify d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.951337.0:21211 2014-08-11 18:43:50.966081 2014-08-13 17:38:02.917959 7fe9bae887a0 20 read_log 193305'18217 (0'0) promote 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by osd.31.0:276 2014-08-11 18:43:51.579189 2014-08-13 17:38:02.917971 7fe9bae887a0 20 read_log 193305'18218 (193305'18217) modify 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.951337.0:21385 2014-08-11 18:43:51.571974 2014-08-13 17:38:02.917983 7fe9bae887a0 20 read_log 193305'18219 (0'0) promote 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by osd.31.0:290 2014-08-11 18:43:54.288271 2014-08-13 17:38:02.917995 7fe9bae887a0 20 read_log 193305'18220 (193305'18219) modify 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.951337.0:22121 2014-08-11 18:43:54.280878 2014-08-13 17:38:02.918007 7fe9bae887a0 20 read_log 193305'18221 (0'0) promote 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by osd.31.0:306 2014-08-11 18:43:59.586444 2014-08-13 17:38:02.918019 7fe9bae887a0 20 read_log 193305'18222 (193305'18221) modify 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.951337.0:23600 2014-08-11 18:43:59.579087 2014-08-13 17:38:02.918031 7fe9bae887a0 20 read_log 193305'18223 (0'0) promote 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by osd.31.0:319 2014-08-11 18:44:04.114112 2014-08-13 17:38:02.918061 7fe9bae887a0 20 read_log 193305'18224 (193305'18223) modify 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.951337.0:24860 2014-08-11 18:44:04.106803 2014-08-13 17:38:02.918076 7fe9bae887a0 20 read_log 193305'18225 (0'0) promote 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by osd.31.0:351 2014-08-11 18:44:14.198462 2014-08-13 17:38:02.918088 7fe9bae887a0 20 read_log 193305'18226 (193305'18225) modify 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.951337.0:27589 2014-08-11 18:44:14.188725 2014-08-13 17:38:02.918100 7fe9bae887a0 20 read_log 193305'18227 (0'0) promote 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by osd.31.0:369 2014-08-11 18:44:18.849172 2014-08-13 17:38:02.918113 7fe9bae887a0 20 read_log 193305'18228 (193305'18227) modify 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.951337.0:28861 2014-08-11 18:44:18.841566 2014-08-13 17:38:02.918125 7fe9bae887a0 20 read_log 193305'18229 (0'0) promote d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by osd.31.0:377 2014-08-11 18:44:24.711304 2014-08-13 17:38:02.918137 7fe9bae887a0 20 read_log 193305'18230 (193305'18229) modify d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.951337.0:30428 2014-08-11 18:44:24.703177 2014-08-13 17:38:02.918148 7fe9bae887a0 20 read_log 193305'18231 (0'0) promote 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by osd.31.0:383 2014-08-11 18:44:25.710961 2014-08-13 17:38:02.918165 7fe9bae887a0 20 read_log 193305'18232 (193305'18231) modify 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.951337.0:30685 2014-08-11 18:44:25.701617 2014-08-13 17:38:02.918177 7fe9bae887a0 20 read_log 193305'18233 (0'0) promote b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by osd.31.0:391 2014-08-11 18:44:28.184473 2014-08-13 17:38:02.918190 7fe9bae887a0 20 read_log 193305'18234 (193305'18233) modify b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.951337.0:31386 2014-08-11 18:44:28.177650 2014-08-13 17:38:02.918201 7fe9bae887a0 20 read_log 193305'18235 (0'0) promote d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by osd.31.0:405 2014-08-11 18:44:32.237113 2014-08-13 17:38:02.918223 7fe9bae887a0 20 read_log 193305'18236 (193305'18235) modify d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.951337.0:32543 2014-08-11 18:44:32.227612 2014-08-13 17:38:02.918238 7fe9bae887a0 20 read_log 193305'18237 (193305'18168) modify 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.951337.0:33042 2014-08-11 18:44:34.098242 2014-08-13 17:38:02.918251 7fe9bae887a0 20 read_log 193305'18238 (193305'18170) modify c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.951337.0:33721 2014-08-11 18:44:34.605577 2014-08-13 17:38:02.918263 7fe9bae887a0 20 read_log 193305'18239 (193305'18172) modify d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.951337.0:36329 2014-08-11 18:44:36.133685 2014-08-13 17:38:02.918275 7fe9bae887a0 20 read_log 193305'18240 (193305'18174) modify 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.951337.0:36701 2014-08-11 18:44:36.482159 2014-08-13 17:38:02.918287 7fe9bae887a0 20 read_log 193305'18241 (193305'18176) modify 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.951337.0:36858 2014-08-11 18:44:36.571751 2014-08-13 17:38:02.918299 7fe9bae887a0 20 read_log 193305'18242 (193305'18178) modify d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.951337.0:36955 2014-08-11 18:44:36.620095 2014-08-13 17:38:02.918311 7fe9bae887a0 20 read_log 193305'18243 (193305'18180) modify f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.951337.0:37005 2014-08-11 18:44:36.648648 2014-08-13 17:38:02.918323 7fe9bae887a0 20 read_log 193305'18244 (193305'18182) modify 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.951337.0:38993 2014-08-11 18:44:37.766648 2014-08-13 17:38:02.918335 7fe9bae887a0 20 read_log 193305'18245 (193305'18184) modify d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.951337.0:42961 2014-08-11 18:44:39.436466 2014-08-13 17:38:02.918347 7fe9bae887a0 20 read_log 193305'18246 (193305'18186) modify 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.951337.0:44879 2014-08-11 18:44:40.022832 2014-08-13 17:38:02.918359 7fe9bae887a0 20 read_log 193305'18247 (193305'18188) modify bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.951337.0:45365 2014-08-11 18:44:40.176410 2014-08-13 17:38:02.918371 7fe9bae887a0 20 read_log 193305'18248 (193305'18190) modify 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.951337.0:46037 2014-08-11 18:44:40.371712 2014-08-13 17:38:02.918383 7fe9bae887a0 20 read_log 193305'18249 (193305'18192) modify 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.951337.0:46073 2014-08-11 18:44:40.381311 2014-08-13 17:38:02.918395 7fe9bae887a0 20 read_log 193305'18250 (193305'18194) modify 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.951337.0:46173 2014-08-11 18:44:40.410404 2014-08-13 17:38:02.918407 7fe9bae887a0 20 read_log 193305'18251 (193305'18196) modify 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.951337.0:46405 2014-08-11 18:44:40.473923 2014-08-13 17:38:02.918419 7fe9bae887a0 20 read_log 193305'18252 (193305'18198) modify c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.951337.0:47278 2014-08-11 18:44:40.723650 2014-08-13 17:38:02.918431 7fe9bae887a0 20 read_log 193305'18253 (193305'18200) modify c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.951337.0:47927 2014-08-11 18:44:40.905370 2014-08-13 17:38:02.918442 7fe9bae887a0 20 read_log 193305'18254 (193305'18202) modify 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.951337.0:49494 2014-08-11 18:44:41.386948 2014-08-13 17:38:02.918454 7fe9bae887a0 20 read_log 193305'18255 (193305'18204) modify 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.951337.0:50402 2014-08-11 18:44:41.654749 2014-08-13 17:38:02.918473 7fe9bae887a0 20 read_log 193305'18256 (193305'18206) modify d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.951337.0:50700 2014-08-11 18:44:41.738138 2014-08-13 17:38:02.918487 7fe9bae887a0 20 read_log 193305'18257 (193305'18208) modify f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.951337.0:52096 2014-08-11 18:44:42.132883 2014-08-13 17:38:02.918499 7fe9bae887a0 20 read_log 193305'18258 (193305'18210) modify a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.951337.0:52517 2014-08-11 18:44:42.254449 2014-08-13 17:38:02.918511 7fe9bae887a0 20 read_log 193305'18259 (193305'18212) modify 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.951337.0:52921 2014-08-11 18:44:42.380469 2014-08-13 17:38:02.918523 7fe9bae887a0 20 read_log 193305'18260 (193305'18214) modify 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.951337.0:53569 2014-08-11 18:44:42.566863 2014-08-13 17:38:02.918535 7fe9bae887a0 20 read_log 193305'18261 (193305'18216) modify d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.951337.0:53922 2014-08-11 18:44:42.674633 2014-08-13 17:38:02.918547 7fe9bae887a0 20 read_log 193305'18262 (193305'18218) modify 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.951337.0:54096 2014-08-11 18:44:42.729849 2014-08-13 17:38:02.918559 7fe9bae887a0 20 read_log 193305'18263 (193305'18220) modify 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.951337.0:54832 2014-08-11 18:44:42.951656 2014-08-13 17:38:02.918571 7fe9bae887a0 20 read_log 193305'18264 (193305'18222) modify 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.951337.0:56311 2014-08-11 18:44:43.381186 2014-08-13 17:38:02.918583 7fe9bae887a0 20 read_log 193305'18265 (193305'18224) modify 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.951337.0:57571 2014-08-11 18:44:43.796585 2014-08-13 17:38:02.918595 7fe9bae887a0 20 read_log 193305'18266 (193305'18226) modify 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.951337.0:60300 2014-08-11 18:44:44.589001 2014-08-13 17:38:02.918607 7fe9bae887a0 20 read_log 193305'18267 (193305'18228) modify 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.951337.0:61572 2014-08-11 18:44:44.964334 2014-08-13 17:38:02.918619 7fe9bae887a0 20 read_log 193305'18268 (193305'18230) modify d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.951337.0:63139 2014-08-11 18:44:45.420569 2014-08-13 17:38:02.918631 7fe9bae887a0 20 read_log 193305'18269 (193305'18232) modify 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.951337.0:63396 2014-08-11 18:44:45.497304 2014-08-13 17:38:02.918643 7fe9bae887a0 20 read_log 193305'18270 (193305'18234) modify b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.951337.0:64097 2014-08-11 18:44:45.703309 2014-08-13 17:38:02.918654 7fe9bae887a0 20 read_log 193305'18271 (193305'18236) modify d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.951337.0:65254 2014-08-11 18:44:46.063902 2014-08-13 17:38:02.918667 7fe9bae887a0 20 read_log 194215'18272 (194215'18272) modify 1e4/hit_set_24.1e4_archive_2014-08-12 15:32:25.782637_2014-08-12 15:43:43.172386/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:43.172618 2014-08-13 17:38:02.918679 7fe9bae887a0 20 read_log 194215'18273 (193305'18165) delete 1e4/hit_set_24.1e4_archive_2014-08-11 18:37:04.917636_2014-08-11 18:42:34.237231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:43.172618 2014-08-13 17:38:02.918692 7fe9bae887a0 20 read_log 194215'18274 (192888'17969) clean 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by osd.31.0:1598 2014-08-12 15:43:43.176821 2014-08-13 17:38:02.918730 7fe9bae887a0 20 read_log 194215'18275 (194215'18274) delete 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.2174595.0:100171 0.000000 2014-08-13 17:38:02.918744 7fe9bae887a0 20 read_log 194215'18276 (192888'18142) clean 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by osd.31.0:1600 2014-08-12 15:43:43.249820 2014-08-13 17:38:02.918777 7fe9bae887a0 20 read_log 194215'18277 (194215'18276) delete 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.2174595.0:100173 0.000000 2014-08-13 17:38:02.918788 7fe9bae887a0 20 read_log 194215'18278 (193305'18250) clean 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by osd.31.0:1602 2014-08-12 15:43:43.258606 2014-08-13 17:38:02.918801 7fe9bae887a0 20 read_log 194215'18279 (194215'18278) delete 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.2174595.0:100175 0.000000 2014-08-13 17:38:02.918811 7fe9bae887a0 20 read_log 194215'18280 (193305'18240) clean 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by osd.31.0:1604 2014-08-12 15:43:43.267221 2014-08-13 17:38:02.918823 7fe9bae887a0 20 read_log 194215'18281 (194215'18280) delete 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.2174595.0:100177 0.000000 2014-08-13 17:38:02.918834 7fe9bae887a0 20 read_log 194215'18282 (192888'17934) clean 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by osd.31.0:1606 2014-08-12 15:43:43.274964 2014-08-13 17:38:02.918846 7fe9bae887a0 20 read_log 194215'18283 (194215'18282) delete 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.2174595.0:100179 0.000000 2014-08-13 17:38:02.918857 7fe9bae887a0 20 read_log 194215'18284 (193305'18258) clean a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by osd.31.0:1608 2014-08-12 15:43:43.282043 2014-08-13 17:38:02.918871 7fe9bae887a0 20 read_log 194215'18285 (194215'18284) delete a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.2174595.0:100181 0.000000 2014-08-13 17:38:02.918882 7fe9bae887a0 20 read_log 194215'18286 (193305'18242) clean d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by osd.31.0:1610 2014-08-12 15:43:43.289905 2014-08-13 17:38:02.918895 7fe9bae887a0 20 read_log 194215'18287 (194215'18286) delete d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.2174595.0:100183 0.000000 2014-08-13 17:38:02.918906 7fe9bae887a0 20 read_log 194215'18288 (192888'18120) clean 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by osd.31.0:1612 2014-08-12 15:43:43.297640 2014-08-13 17:38:02.918918 7fe9bae887a0 20 read_log 194215'18289 (194215'18288) delete 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.2174595.0:100185 0.000000 2014-08-13 17:38:02.918929 7fe9bae887a0 20 read_log 194215'18290 (192888'18111) clean abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by osd.31.0:1614 2014-08-12 15:43:43.304954 2014-08-13 17:38:02.918941 7fe9bae887a0 20 read_log 194215'18291 (194215'18290) delete abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.2174595.0:100187 0.000000 2014-08-13 17:38:02.918951 7fe9bae887a0 20 read_log 194215'18292 (192888'18162) clean 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by osd.31.0:1616 2014-08-12 15:43:43.312458 2014-08-13 17:38:02.918963 7fe9bae887a0 20 read_log 194215'18293 (194215'18292) delete 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.2174595.0:100189 0.000000 2014-08-13 17:38:02.918974 7fe9bae887a0 20 read_log 194215'18294 (192888'18122) clean 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by osd.31.0:1618 2014-08-12 15:43:43.319512 2014-08-13 17:38:02.918995 7fe9bae887a0 20 read_log 194215'18295 (194215'18294) delete 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.2174595.0:100191 0.000000 2014-08-13 17:38:02.919006 7fe9bae887a0 20 read_log 194215'18296 (192888'17970) clean 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by osd.31.0:1620 2014-08-12 15:43:43.327227 2014-08-13 17:38:02.919018 7fe9bae887a0 20 read_log 194215'18297 (194215'18296) delete 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.2174595.0:100193 0.000000 2014-08-13 17:38:02.919029 7fe9bae887a0 20 read_log 194215'18298 (192888'17964) clean 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by osd.31.0:1622 2014-08-12 15:43:43.334500 2014-08-13 17:38:02.919041 7fe9bae887a0 20 read_log 194215'18299 (194215'18298) delete 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.2174595.0:100195 0.000000 2014-08-13 17:38:02.919052 7fe9bae887a0 20 read_log 194215'18300 (192888'18133) clean 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by osd.31.0:1624 2014-08-12 15:43:43.341493 2014-08-13 17:38:02.919064 7fe9bae887a0 20 read_log 194215'18301 (194215'18300) delete 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.2174595.0:100197 0.000000 2014-08-13 17:38:02.919075 7fe9bae887a0 20 read_log 194215'18302 (192888'18131) clean c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by osd.31.0:1626 2014-08-12 15:43:43.348859 2014-08-13 17:38:02.919088 7fe9bae887a0 20 read_log 194215'18303 (194215'18302) delete c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.2174595.0:100199 0.000000 2014-08-13 17:38:02.919099 7fe9bae887a0 20 read_log 194215'18304 (192888'17952) clean f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by osd.31.0:1628 2014-08-12 15:43:43.355994 2014-08-13 17:38:02.919111 7fe9bae887a0 20 read_log 194215'18305 (194215'18304) delete f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.2174595.0:100201 0.000000 2014-08-13 17:38:02.919121 7fe9bae887a0 20 read_log 194215'18306 (192888'18125) clean e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by osd.31.0:1630 2014-08-12 15:43:43.362854 2014-08-13 17:38:02.919133 7fe9bae887a0 20 read_log 194215'18307 (194215'18306) delete e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.2174595.0:100203 0.000000 2014-08-13 17:38:02.919144 7fe9bae887a0 20 read_log 194215'18308 (193305'18257) clean f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by osd.31.0:1632 2014-08-12 15:43:43.370039 2014-08-13 17:38:02.919160 7fe9bae887a0 20 read_log 194215'18309 (194215'18308) delete f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.2174595.0:100205 0.000000 2014-08-13 17:38:02.919171 7fe9bae887a0 20 read_log 194215'18310 (193305'18269) clean 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by osd.31.0:1634 2014-08-12 15:43:43.377110 2014-08-13 17:38:02.919183 7fe9bae887a0 20 read_log 194215'18311 (194215'18310) delete 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.2174595.0:100207 0.000000 2014-08-13 17:38:02.919194 7fe9bae887a0 20 read_log 194215'18312 (193305'18267) clean 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by osd.31.0:1636 2014-08-12 15:43:43.384371 2014-08-13 17:38:02.919206 7fe9bae887a0 20 read_log 194215'18313 (194215'18312) delete 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.2174595.0:100209 0.000000 2014-08-13 17:38:02.919217 7fe9bae887a0 20 read_log 194215'18314 (192888'18134) clean c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by osd.31.0:1638 2014-08-12 15:43:43.391807 2014-08-13 17:38:02.919236 7fe9bae887a0 20 read_log 194215'18315 (194215'18314) delete c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.2174595.0:100211 0.000000 2014-08-13 17:38:02.919247 7fe9bae887a0 20 read_log 194215'18316 (193305'18251) clean 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by osd.31.0:1640 2014-08-12 15:43:43.399533 2014-08-13 17:38:02.919281 7fe9bae887a0 20 read_log 194215'18317 (194215'18316) delete 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.2174595.0:100213 0.000000 2014-08-13 17:38:02.919293 7fe9bae887a0 20 read_log 194215'18318 (192888'18106) clean 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by osd.31.0:1642 2014-08-12 15:43:43.407404 2014-08-13 17:38:02.919305 7fe9bae887a0 20 read_log 194215'18319 (194215'18318) delete 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.2174595.0:100215 0.000000 2014-08-13 17:38:02.919316 7fe9bae887a0 20 read_log 194215'18320 (193305'18261) clean d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by osd.31.0:1644 2014-08-12 15:43:43.414842 2014-08-13 17:38:02.919328 7fe9bae887a0 20 read_log 194215'18321 (194215'18320) delete d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.2174595.0:100217 0.000000 2014-08-13 17:38:02.919339 7fe9bae887a0 20 read_log 194215'18322 (192888'17948) clean bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by osd.31.0:1646 2014-08-12 15:43:43.421997 2014-08-13 17:38:02.919351 7fe9bae887a0 20 read_log 194215'18323 (194215'18322) delete bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.2174595.0:100219 0.000000 2014-08-13 17:38:02.919362 7fe9bae887a0 20 read_log 194215'18324 (193305'18239) clean d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by osd.31.0:1648 2014-08-12 15:43:43.428559 2014-08-13 17:38:02.919374 7fe9bae887a0 20 read_log 194215'18325 (194215'18324) delete d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.2174595.0:100221 0.000000 2014-08-13 17:38:02.919385 7fe9bae887a0 20 read_log 194215'18326 (192888'18136) clean 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by osd.31.0:1650 2014-08-12 15:43:43.436226 2014-08-13 17:38:02.919396 7fe9bae887a0 20 read_log 194215'18327 (194215'18326) delete 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.2174595.0:100223 0.000000 2014-08-13 17:38:02.919407 7fe9bae887a0 20 read_log 194215'18328 (193305'18271) clean d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by osd.31.0:1652 2014-08-12 15:43:43.443653 2014-08-13 17:38:02.919419 7fe9bae887a0 20 read_log 194215'18329 (194215'18328) delete d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.2174595.0:100225 0.000000 2014-08-13 17:38:02.919430 7fe9bae887a0 20 read_log 194215'18330 (193305'18266) clean 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by osd.31.0:1654 2014-08-12 15:43:43.450780 2014-08-13 17:38:02.919442 7fe9bae887a0 20 read_log 194215'18331 (194215'18330) delete 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.2174595.0:100227 0.000000 2014-08-13 17:38:02.919452 7fe9bae887a0 20 read_log 194215'18332 (192888'18121) clean 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by osd.31.0:1656 2014-08-12 15:43:43.458096 2014-08-13 17:38:02.919464 7fe9bae887a0 20 read_log 194215'18333 (194215'18332) delete 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.2174595.0:100229 0.000000 2014-08-13 17:38:02.919475 7fe9bae887a0 20 read_log 194215'18334 (193305'18253) clean c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by osd.31.0:1658 2014-08-12 15:43:43.465662 2014-08-13 17:38:02.919496 7fe9bae887a0 20 read_log 194215'18335 (194215'18334) delete c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.2174595.0:100231 0.000000 2014-08-13 17:38:02.919508 7fe9bae887a0 20 read_log 194215'18336 (192888'17965) clean 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by osd.31.0:1660 2014-08-12 15:43:43.473350 2014-08-13 17:38:02.919520 7fe9bae887a0 20 read_log 194215'18337 (194215'18336) delete 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.2174595.0:100233 0.000000 2014-08-13 17:38:02.919530 7fe9bae887a0 20 read_log 194215'18338 (192888'18126) clean db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by osd.31.0:1662 2014-08-12 15:43:43.482005 2014-08-13 17:38:02.919542 7fe9bae887a0 20 read_log 194215'18339 (194215'18338) delete db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.2174595.0:100235 0.000000 2014-08-13 17:38:02.919553 7fe9bae887a0 20 read_log 194215'18340 (192888'17976) clean 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by osd.31.0:1664 2014-08-12 15:43:43.490042 2014-08-13 17:38:02.919565 7fe9bae887a0 20 read_log 194215'18341 (194215'18340) delete 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.2174595.0:100237 0.000000 2014-08-13 17:38:02.919575 7fe9bae887a0 20 read_log 194215'18342 (192888'17978) clean 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by osd.31.0:1666 2014-08-12 15:43:43.497442 2014-08-13 17:38:02.919587 7fe9bae887a0 20 read_log 194215'18343 (194215'18342) delete 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.2174595.0:100239 0.000000 2014-08-13 17:38:02.919597 7fe9bae887a0 20 read_log 194215'18344 (192888'17912) clean a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by osd.31.0:1668 2014-08-12 15:43:43.504933 2014-08-13 17:38:02.919609 7fe9bae887a0 20 read_log 194215'18345 (194215'18344) delete a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.2174595.0:100241 0.000000 2014-08-13 17:38:02.919620 7fe9bae887a0 20 read_log 194215'18346 (193305'18243) clean f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by osd.31.0:1670 2014-08-12 15:43:43.512493 2014-08-13 17:38:02.919632 7fe9bae887a0 20 read_log 194215'18347 (194215'18346) delete f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.2174595.0:100243 0.000000 2014-08-13 17:38:02.919643 7fe9bae887a0 20 read_log 194215'18348 (192888'17947) clean e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by osd.31.0:1672 2014-08-12 15:43:43.519915 2014-08-13 17:38:02.919655 7fe9bae887a0 20 read_log 194215'18349 (194215'18348) delete e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.2174595.0:100245 0.000000 2014-08-13 17:38:02.919665 7fe9bae887a0 20 read_log 194215'18350 (192888'18141) clean 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by osd.31.0:1674 2014-08-12 15:43:43.527567 2014-08-13 17:38:02.919677 7fe9bae887a0 20 read_log 194215'18351 (194215'18350) delete 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.2174595.0:100247 0.000000 2014-08-13 17:38:02.919688 7fe9bae887a0 20 read_log 194215'18352 (193305'18245) clean d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by osd.31.0:1676 2014-08-12 15:43:43.534506 2014-08-13 17:38:02.919700 7fe9bae887a0 20 read_log 194215'18353 (194215'18352) delete d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.2174595.0:100249 0.000000 2014-08-13 17:38:02.919711 7fe9bae887a0 20 read_log 194215'18354 (193305'18248) clean 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by osd.31.0:1678 2014-08-12 15:43:43.541650 2014-08-13 17:38:02.919732 7fe9bae887a0 20 read_log 194215'18355 (194215'18354) delete 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.2174595.0:100251 0.000000 2014-08-13 17:38:02.919743 7fe9bae887a0 20 read_log 194215'18356 (193305'18260) clean 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by osd.31.0:1680 2014-08-12 15:43:43.549083 2014-08-13 17:38:02.919755 7fe9bae887a0 20 read_log 194215'18357 (194215'18356) delete 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.2174595.0:100253 0.000000 2014-08-13 17:38:02.919765 7fe9bae887a0 20 read_log 194215'18358 (192888'17961) clean 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by osd.31.0:1682 2014-08-12 15:43:43.556931 2014-08-13 17:38:02.919777 7fe9bae887a0 20 read_log 194215'18359 (194215'18358) delete 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.2174595.0:100255 0.000000 2014-08-13 17:38:02.919788 7fe9bae887a0 20 read_log 194215'18360 (193305'18264) clean 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by osd.31.0:1684 2014-08-12 15:43:43.565284 2014-08-13 17:38:02.919800 7fe9bae887a0 20 read_log 194215'18361 (194215'18360) delete 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.2174595.0:100257 0.000000 2014-08-13 17:38:02.919811 7fe9bae887a0 20 read_log 194215'18362 (192888'17925) clean b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by osd.31.0:1686 2014-08-12 15:43:43.573307 2014-08-13 17:38:02.919822 7fe9bae887a0 20 read_log 194215'18363 (194215'18362) delete b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.2174595.0:100259 0.000000 2014-08-13 17:38:02.919833 7fe9bae887a0 20 read_log 194215'18364 (193305'18265) clean 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by osd.31.0:1688 2014-08-12 15:43:43.580553 2014-08-13 17:38:02.919848 7fe9bae887a0 20 read_log 194215'18365 (194215'18364) delete 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.2174595.0:100261 0.000000 2014-08-13 17:38:02.919859 7fe9bae887a0 20 read_log 194215'18366 (192888'18157) clean 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by osd.31.0:1690 2014-08-12 15:43:43.587991 2014-08-13 17:38:02.919871 7fe9bae887a0 20 read_log 194215'18367 (194215'18366) delete 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.2174595.0:100263 0.000000 2014-08-13 17:38:02.919882 7fe9bae887a0 20 read_log 194215'18368 (192888'18139) clean e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by osd.31.0:1692 2014-08-12 15:43:43.595155 2014-08-13 17:38:02.919894 7fe9bae887a0 20 read_log 194215'18369 (194215'18368) delete e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.2174595.0:100265 0.000000 2014-08-13 17:38:02.919904 7fe9bae887a0 20 read_log 194215'18370 (193305'18256) clean d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by osd.31.0:1694 2014-08-12 15:43:43.602326 2014-08-13 17:38:02.919916 7fe9bae887a0 20 read_log 194215'18371 (194215'18370) delete d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.2174595.0:100267 0.000000 2014-08-13 17:38:02.919927 7fe9bae887a0 20 read_log 194215'18372 (192888'17960) clean 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by osd.31.0:1696 2014-08-12 15:43:43.609468 2014-08-13 17:38:02.919939 7fe9bae887a0 20 read_log 194215'18373 (194215'18372) delete 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.2174595.0:100269 0.000000 2014-08-13 17:38:02.919949 7fe9bae887a0 20 read_log 194215'18374 (192888'17949) clean 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by osd.31.0:1698 2014-08-12 15:43:43.616898 2014-08-13 17:38:02.919968 7fe9bae887a0 20 read_log 194215'18375 (194215'18374) delete 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.2174595.0:100271 0.000000 2014-08-13 17:38:02.919979 7fe9bae887a0 20 read_log 194215'18376 (193305'18241) clean 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by osd.31.0:1700 2014-08-12 15:43:43.623453 2014-08-13 17:38:02.919991 7fe9bae887a0 20 read_log 194215'18377 (194215'18376) delete 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.2174595.0:100273 0.000000 2014-08-13 17:38:02.920002 7fe9bae887a0 20 read_log 194215'18378 (193305'18244) clean 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by osd.31.0:1702 2014-08-12 15:43:43.630486 2014-08-13 17:38:02.920014 7fe9bae887a0 20 read_log 194215'18379 (194215'18378) delete 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.2174595.0:100275 0.000000 2014-08-13 17:38:02.920024 7fe9bae887a0 20 read_log 194215'18380 (192888'18107) clean 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by osd.31.0:1704 2014-08-12 15:43:43.637886 2014-08-13 17:38:02.920036 7fe9bae887a0 20 read_log 194215'18381 (194215'18380) delete 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.2174595.0:100277 0.000000 2014-08-13 17:38:02.920047 7fe9bae887a0 20 read_log 194215'18382 (192888'18151) clean 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by osd.31.0:1706 2014-08-12 15:43:43.644750 2014-08-13 17:38:02.920059 7fe9bae887a0 20 read_log 194215'18383 (194215'18382) delete 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.2174595.0:100279 0.000000 2014-08-13 17:38:02.920069 7fe9bae887a0 20 read_log 194215'18384 (192888'17945) clean 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by osd.31.0:1708 2014-08-12 15:43:43.651904 2014-08-13 17:38:02.920081 7fe9bae887a0 20 read_log 194215'18385 (194215'18384) delete 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.2174595.0:100281 0.000000 2014-08-13 17:38:02.920092 7fe9bae887a0 20 read_log 194215'18386 (193305'18249) clean 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by osd.31.0:1710 2014-08-12 15:43:43.659442 2014-08-13 17:38:02.920103 7fe9bae887a0 20 read_log 194215'18387 (194215'18386) delete 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.2174595.0:100283 0.000000 2014-08-13 17:38:02.920114 7fe9bae887a0 20 read_log 194215'18388 (192888'17979) clean bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by osd.31.0:1712 2014-08-12 15:43:43.666799 2014-08-13 17:38:02.920126 7fe9bae887a0 20 read_log 194215'18389 (194215'18388) delete bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.2174595.0:100285 0.000000 2014-08-13 17:38:02.920137 7fe9bae887a0 20 read_log 194215'18390 (192888'17966) clean c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by osd.31.0:1714 2014-08-12 15:43:43.674568 2014-08-13 17:38:02.920150 7fe9bae887a0 20 read_log 194215'18391 (194215'18390) delete c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.2174595.0:100287 0.000000 2014-08-13 17:38:02.920166 7fe9bae887a0 20 read_log 194215'18392 (192888'18164) clean b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by osd.31.0:1716 2014-08-12 15:43:43.682380 2014-08-13 17:38:02.920181 7fe9bae887a0 20 read_log 194215'18393 (194215'18392) delete b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.2174595.0:100289 0.000000 2014-08-13 17:38:02.920193 7fe9bae887a0 20 read_log 194215'18394 (193305'18246) clean 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by osd.31.0:1718 2014-08-12 15:43:43.689808 2014-08-13 17:38:02.920212 7fe9bae887a0 20 read_log 194215'18395 (194215'18394) delete 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.2174595.0:100291 0.000000 2014-08-13 17:38:02.920223 7fe9bae887a0 20 read_log 194215'18396 (192888'18153) clean 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by osd.31.0:1720 2014-08-12 15:43:43.697003 2014-08-13 17:38:02.920235 7fe9bae887a0 20 read_log 194215'18397 (194215'18396) delete 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.2174595.0:100293 0.000000 2014-08-13 17:38:02.920246 7fe9bae887a0 20 read_log 194215'18398 (192888'17920) clean 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by osd.31.0:1722 2014-08-12 15:43:43.704647 2014-08-13 17:38:02.920258 7fe9bae887a0 20 read_log 194215'18399 (194215'18398) delete 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.2174595.0:100295 0.000000 2014-08-13 17:38:02.920269 7fe9bae887a0 20 read_log 194215'18400 (192888'17911) clean a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by osd.31.0:1724 2014-08-12 15:43:43.712202 2014-08-13 17:38:02.920281 7fe9bae887a0 20 read_log 194215'18401 (194215'18400) delete a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.2174595.0:100297 0.000000 2014-08-13 17:38:02.920291 7fe9bae887a0 20 read_log 194215'18402 (193305'18252) clean c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by osd.31.0:1726 2014-08-12 15:43:43.718863 2014-08-13 17:38:02.920303 7fe9bae887a0 20 read_log 194215'18403 (194215'18402) delete c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.2174595.0:100299 0.000000 2014-08-13 17:38:02.920314 7fe9bae887a0 20 read_log 194215'18404 (193305'18238) clean c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by osd.31.0:1728 2014-08-12 15:43:43.725832 2014-08-13 17:38:02.920327 7fe9bae887a0 20 read_log 194215'18405 (194215'18404) delete c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.2174595.0:100301 0.000000 2014-08-13 17:38:02.920338 7fe9bae887a0 20 read_log 194215'18406 (193305'18270) clean b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by osd.31.0:1730 2014-08-12 15:43:43.733562 2014-08-13 17:38:02.920349 7fe9bae887a0 20 read_log 194215'18407 (194215'18406) delete b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.2174595.0:100303 0.000000 2014-08-13 17:38:02.920360 7fe9bae887a0 20 read_log 194215'18408 (193305'18255) clean 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by osd.31.0:1732 2014-08-12 15:43:43.740834 2014-08-13 17:38:02.920373 7fe9bae887a0 20 read_log 194215'18409 (194215'18408) delete 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.2174595.0:100305 0.000000 2014-08-13 17:38:02.920400 7fe9bae887a0 20 read_log 194215'18410 (192888'18163) clean e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by osd.31.0:1734 2014-08-12 15:43:43.748073 2014-08-13 17:38:02.920413 7fe9bae887a0 20 read_log 194215'18411 (194215'18410) delete e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.2174595.0:100307 0.000000 2014-08-13 17:38:02.920424 7fe9bae887a0 20 read_log 194215'18412 (192888'17967) clean 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by osd.31.0:1736 2014-08-12 15:43:43.755565 2014-08-13 17:38:02.920436 7fe9bae887a0 20 read_log 194215'18413 (194215'18412) delete 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.2174595.0:100309 0.000000 2014-08-13 17:38:02.920446 7fe9bae887a0 20 read_log 194215'18414 (193305'18259) clean 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by osd.31.0:1738 2014-08-12 15:43:43.762529 2014-08-13 17:38:02.920482 7fe9bae887a0 20 read_log 194215'18415 (194215'18414) delete 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.2174595.0:100311 0.000000 2014-08-13 17:38:02.920493 7fe9bae887a0 20 read_log 194215'18416 (192888'18109) clean d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by osd.31.0:1740 2014-08-12 15:43:43.769492 2014-08-13 17:38:02.920506 7fe9bae887a0 20 read_log 194215'18417 (194215'18416) delete d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.2174595.0:100313 0.000000 2014-08-13 17:38:02.920517 7fe9bae887a0 20 read_log 194215'18418 (192888'17974) clean 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by osd.31.0:1742 2014-08-12 15:43:43.776037 2014-08-13 17:38:02.920529 7fe9bae887a0 20 read_log 194215'18419 (194215'18418) delete 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.2174595.0:100315 0.000000 2014-08-13 17:38:02.920539 7fe9bae887a0 20 read_log 194215'18420 (192888'18148) clean ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by osd.31.0:1744 2014-08-12 15:43:43.783018 2014-08-13 17:38:02.920551 7fe9bae887a0 20 read_log 194215'18421 (194215'18420) delete ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.2174595.0:100317 0.000000 2014-08-13 17:38:02.920562 7fe9bae887a0 20 read_log 194215'18422 (193305'18262) clean 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by osd.31.0:1746 2014-08-12 15:43:43.790266 2014-08-13 17:38:02.920574 7fe9bae887a0 20 read_log 194215'18423 (194215'18422) delete 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.2174595.0:100319 0.000000 2014-08-13 17:38:02.920585 7fe9bae887a0 20 read_log 194215'18424 (192888'17926) clean e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by osd.31.0:1748 2014-08-12 15:43:43.797872 2014-08-13 17:38:02.920597 7fe9bae887a0 20 read_log 194215'18425 (194215'18424) delete e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.2174595.0:100321 0.000000 2014-08-13 17:38:02.920608 7fe9bae887a0 20 read_log 194215'18426 (192888'18114) clean d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by osd.31.0:1750 2014-08-12 15:43:43.804848 2014-08-13 17:38:02.920622 7fe9bae887a0 20 read_log 194215'18427 (194215'18426) delete d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.2174595.0:100323 0.000000 2014-08-13 17:38:02.920633 7fe9bae887a0 20 read_log 194215'18428 (192888'17953) clean e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by osd.31.0:1752 2014-08-12 15:43:43.812866 2014-08-13 17:38:02.920645 7fe9bae887a0 20 read_log 194215'18429 (194215'18428) delete e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.2174595.0:100325 0.000000 2014-08-13 17:38:02.920656 7fe9bae887a0 20 read_log 194215'18430 (192888'17943) clean 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by osd.31.0:1754 2014-08-12 15:43:43.820773 2014-08-13 17:38:02.920667 7fe9bae887a0 20 read_log 194215'18431 (194215'18430) delete 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.2174595.0:100327 0.000000 2014-08-13 17:38:02.920678 7fe9bae887a0 20 read_log 194215'18432 (193305'18247) clean bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by osd.31.0:1756 2014-08-12 15:43:43.828648 2014-08-13 17:38:02.920690 7fe9bae887a0 20 read_log 194215'18433 (194215'18432) delete bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.2174595.0:100329 0.000000 2014-08-13 17:38:02.920708 7fe9bae887a0 20 read_log 194215'18434 (192888'17918) clean 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by osd.31.0:1758 2014-08-12 15:43:43.835375 2014-08-13 17:38:02.920730 7fe9bae887a0 20 read_log 194215'18435 (194215'18434) delete 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.2174595.0:100331 0.000000 2014-08-13 17:38:02.920741 7fe9bae887a0 20 read_log 194215'18436 (192888'18137) clean 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by osd.31.0:1760 2014-08-12 15:43:43.842121 2014-08-13 17:38:02.920754 7fe9bae887a0 20 read_log 194215'18437 (194215'18436) delete 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.2174595.0:100333 0.000000 2014-08-13 17:38:02.920764 7fe9bae887a0 20 read_log 194215'18438 (192834'17433) clean 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by osd.31.0:1762 2014-08-12 15:43:43.848946 2014-08-13 17:38:02.920776 7fe9bae887a0 20 read_log 194215'18439 (194215'18438) delete 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.2174595.0:100335 0.000000 2014-08-13 17:38:02.920787 7fe9bae887a0 20 read_log 194215'18440 (192888'18123) clean 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by osd.31.0:1764 2014-08-12 15:43:43.856049 2014-08-13 17:38:02.920799 7fe9bae887a0 20 read_log 194215'18441 (194215'18440) delete 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.2174595.0:100337 0.000000 2014-08-13 17:38:02.920810 7fe9bae887a0 20 read_log 194215'18442 (192888'18117) clean b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by osd.31.0:1766 2014-08-12 15:43:43.862953 2014-08-13 17:38:02.920822 7fe9bae887a0 20 read_log 194215'18443 (194215'18442) delete b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.2174595.0:100339 0.000000 2014-08-13 17:38:02.920833 7fe9bae887a0 20 read_log 194215'18444 (192888'17933) clean b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by osd.31.0:1768 2014-08-12 15:43:43.869724 2014-08-13 17:38:02.920845 7fe9bae887a0 20 read_log 194215'18445 (194215'18444) delete b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.2174595.0:100341 0.000000 2014-08-13 17:38:02.920855 7fe9bae887a0 20 read_log 194215'18446 (192888'17963) clean d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by osd.31.0:1770 2014-08-12 15:43:43.876164 2014-08-13 17:38:02.920867 7fe9bae887a0 20 read_log 194215'18447 (194215'18446) delete d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.2174595.0:100343 0.000000 2014-08-13 17:38:02.920878 7fe9bae887a0 20 read_log 194215'18448 (192888'18105) clean c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by osd.31.0:1772 2014-08-12 15:43:43.883207 2014-08-13 17:38:02.920890 7fe9bae887a0 20 read_log 194215'18449 (194215'18448) delete c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.2174595.0:100345 0.000000 2014-08-13 17:38:02.920901 7fe9bae887a0 20 read_log 194215'18450 (193305'18268) clean d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by osd.31.0:1774 2014-08-12 15:43:43.890484 2014-08-13 17:38:02.920912 7fe9bae887a0 20 read_log 194215'18451 (194215'18450) delete d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.2174595.0:100347 0.000000 2014-08-13 17:38:02.920923 7fe9bae887a0 20 read_log 194215'18452 (192888'17946) clean 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by osd.31.0:1776 2014-08-12 15:43:43.897520 2014-08-13 17:38:02.920935 7fe9bae887a0 20 read_log 194215'18453 (194215'18452) delete 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.2174595.0:100349 0.000000 2014-08-13 17:38:02.920945 7fe9bae887a0 20 read_log 194215'18454 (192888'18129) clean e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by osd.31.0:1778 2014-08-12 15:43:43.904426 2014-08-13 17:38:02.920964 7fe9bae887a0 20 read_log 194215'18455 (194215'18454) delete e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.2174595.0:100351 0.000000 2014-08-13 17:38:02.920976 7fe9bae887a0 20 read_log 194215'18456 (192888'18152) clean 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by osd.31.0:1780 2014-08-12 15:43:43.911985 2014-08-13 17:38:02.920988 7fe9bae887a0 20 read_log 194215'18457 (194215'18456) delete 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.2174595.0:100353 0.000000 2014-08-13 17:38:02.920999 7fe9bae887a0 20 read_log 194215'18458 (192888'18150) clean 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by osd.31.0:1782 2014-08-12 15:43:43.919809 2014-08-13 17:38:02.921011 7fe9bae887a0 20 read_log 194215'18459 (194215'18458) delete 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.2174595.0:100355 0.000000 2014-08-13 17:38:02.921022 7fe9bae887a0 20 read_log 194215'18460 (193305'18254) clean 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by osd.31.0:1784 2014-08-12 15:43:43.927228 2014-08-13 17:38:02.921034 7fe9bae887a0 20 read_log 194215'18461 (194215'18460) delete 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.2174595.0:100357 0.000000 2014-08-13 17:38:02.921044 7fe9bae887a0 20 read_log 194215'18462 (193305'18237) clean 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by osd.31.0:1786 2014-08-12 15:43:43.934613 2014-08-13 17:38:02.921056 7fe9bae887a0 20 read_log 194215'18463 (194215'18462) delete 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.2174595.0:100359 0.000000 2014-08-13 17:38:02.921067 7fe9bae887a0 20 read_log 194215'18464 (193305'18263) clean 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by osd.31.0:1788 2014-08-12 15:43:43.941918 2014-08-13 17:38:02.921079 7fe9bae887a0 20 read_log 194215'18465 (194215'18464) delete 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.2174595.0:100361 0.000000 2014-08-13 17:38:02.921090 7fe9bae887a0 20 read_log 194215'18466 (192888'17941) clean a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by osd.31.0:1790 2014-08-12 15:43:43.948996 2014-08-13 17:38:02.921102 7fe9bae887a0 20 read_log 194215'18467 (194215'18466) delete a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.2174595.0:100363 0.000000 2014-08-13 17:38:02.921113 7fe9bae887a0 20 read_log 194215'18468 (194215'18468) modify 1e4/hit_set_24.1e4_archive_2014-08-12 15:43:43.172622_2014-08-12 17:24:41.973907/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.974133 2014-08-13 17:38:02.921125 7fe9bae887a0 20 read_log 194215'18469 (194215'18272) delete 1e4/hit_set_24.1e4_archive_2014-08-12 15:32:25.782637_2014-08-12 15:43:43.172386/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.974133 2014-08-13 17:38:02.921137 7fe9bae887a0 20 read_log 194215'18470 (0'0) promote b9d7cde4/benchmark_data_storage0111-ib.pocstor_34628_object522/head//24 by osd.31.0:2686 2014-08-12 17:24:41.978763 2014-08-13 17:38:02.921150 7fe9bae887a0 20 read_log 194215'18471 (194215'18470) modify b9d7cde4/benchmark_data_storage0111-ib.pocstor_34628_object522/head//24 by client.2193759.0:523 2014-08-12 17:24:41.968629 2014-08-13 17:38:02.921168 7fe9bae887a0 20 read_log 194215'18472 (0'0) promote b0c31de4/benchmark_data_storage0111-ib.pocstor_34628_object1046/head//24 by osd.31.0:2699 2014-08-12 17:24:43.968374 2014-08-13 17:38:02.921181 7fe9bae887a0 20 read_log 194215'18473 (194215'18472) modify b0c31de4/benchmark_data_storage0111-ib.pocstor_34628_object1046/head//24 by client.2193759.0:1047 2014-08-12 17:24:43.957085 2014-08-13 17:38:02.921201 7fe9bae887a0 20 read_log 194217'18474 (0'0) promote 878cf9e4/benchmark_data_storage0111-ib.pocstor_39693_object68/head//24 by osd.31.0:2703 2014-08-12 17:25:03.584198 2014-08-13 17:38:02.921214 7fe9bae887a0 20 read_log 194217'18475 (194217'18474) modify 878cf9e4/benchmark_data_storage0111-ib.pocstor_39693_object68/head//24 by client.1487927.0:69 2014-08-12 17:25:03.546057 2014-08-13 17:38:02.921226 7fe9bae887a0 20 read_log 194370'18476 (0'0) promote e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by osd.31.0:2719 2014-08-12 20:30:14.450742 2014-08-13 17:38:02.921238 7fe9bae887a0 20 read_log 194370'18477 (194370'18476) modify e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by client.1508531.0:93 2014-08-12 20:30:14.442892 2014-08-13 17:38:02.921250 7fe9bae887a0 20 read_log 194370'18478 (0'0) promote d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by osd.31.0:2723 2014-08-12 20:30:15.427107 2014-08-13 17:38:02.921262 7fe9bae887a0 20 read_log 194370'18479 (194370'18478) modify d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by client.1508531.0:305 2014-08-12 20:30:15.416579 2014-08-13 17:38:02.921274 7fe9bae887a0 20 read_log 194370'18480 (0'0) promote ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by osd.31.0:2751 2014-08-12 20:30:21.605382 2014-08-13 17:38:02.921286 7fe9bae887a0 20 read_log 194370'18481 (194370'18480) modify ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by client.1508531.0:1938 2014-08-12 20:30:21.598856 2014-08-13 17:38:02.921298 7fe9bae887a0 20 read_log 194370'18482 (0'0) promote 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by osd.31.0:2765 2014-08-12 20:30:27.705091 2014-08-13 17:38:02.921310 7fe9bae887a0 20 read_log 194370'18483 (194370'18482) modify 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by client.1508531.0:3580 2014-08-12 20:30:27.698415 2014-08-13 17:38:02.921322 7fe9bae887a0 20 read_log 194370'18484 (0'0) promote e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by osd.31.0:2787 2014-08-12 20:30:33.356088 2014-08-13 17:38:02.921334 7fe9bae887a0 20 read_log 194370'18485 (194370'18484) modify e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by client.1508531.0:5021 2014-08-12 20:30:33.348586 2014-08-13 17:38:02.921346 7fe9bae887a0 20 read_log 194370'18486 (0'0) promote 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by osd.31.0:2807 2014-08-12 20:30:39.224805 2014-08-13 17:38:02.921357 7fe9bae887a0 20 read_log 194370'18487 (194370'18486) modify 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by client.1508531.0:6343 2014-08-12 20:30:39.217114 2014-08-13 17:38:02.921369 7fe9bae887a0 20 read_log 194370'18488 (0'0) promote 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by osd.31.0:2833 2014-08-12 20:30:52.362569 2014-08-13 17:38:02.921381 7fe9bae887a0 20 read_log 194370'18489 (194370'18488) modify 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by client.1508531.0:8851 2014-08-12 20:30:52.354717 2014-08-13 17:38:02.921393 7fe9bae887a0 20 read_log 194370'18490 (0'0) promote d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by osd.31.0:2835 2014-08-12 20:30:55.329290 2014-08-13 17:38:02.921405 7fe9bae887a0 20 read_log 194370'18491 (194370'18490) modify d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by client.1508531.0:9336 2014-08-12 20:30:55.321023 2014-08-13 17:38:02.921417 7fe9bae887a0 20 read_log 194370'18492 (0'0) promote 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by osd.31.0:2859 2014-08-12 20:31:11.150528 2014-08-13 17:38:02.921429 7fe9bae887a0 20 read_log 194370'18493 (194370'18492) modify 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by client.1508531.0:11971 2014-08-12 20:31:11.141221 2014-08-13 17:38:02.921448 7fe9bae887a0 20 read_log 194377'18494 (0'0) promote 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by osd.31.0:2883 2014-08-12 20:32:57.195553 2014-08-13 17:38:02.921461 7fe9bae887a0 20 read_log 194377'18495 (194377'18494) modify 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by client.2231190.0:23 2014-08-12 20:32:57.185894 2014-08-13 17:38:02.921473 7fe9bae887a0 20 read_log 194377'18496 (0'0) promote f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by osd.31.0:2917 2014-08-12 20:33:05.880103 2014-08-13 17:38:02.921485 7fe9bae887a0 20 read_log 194377'18497 (194377'18496) modify f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by client.2231190.0:2150 2014-08-12 20:33:05.869661 2014-08-13 17:38:02.921497 7fe9bae887a0 20 read_log 194380'18498 (0'0) promote 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by osd.31.0:2935 2014-08-12 20:33:23.490700 2014-08-13 17:38:02.921509 7fe9bae887a0 20 read_log 194380'18499 (194380'18498) modify 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by client.2231190.0:5035 2014-08-12 20:33:23.482176 2014-08-13 17:38:02.921522 7fe9bae887a0 20 read_log 194382'18500 (0'0) promote cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by osd.31.0:2951 2014-08-12 20:33:59.544580 2014-08-13 17:38:02.921533 7fe9bae887a0 20 read_log 194382'18501 (194382'18500) modify cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by client.2231190.0:5682 2014-08-12 20:33:59.536043 2014-08-13 17:38:02.921546 7fe9bae887a0 20 read_log 194383'18502 (0'0) promote bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by osd.31.0:2963 2014-08-12 20:34:03.551306 2014-08-13 17:38:02.921577 7fe9bae887a0 20 read_log 194383'18503 (194383'18502) modify bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by client.2231190.0:6390 2014-08-12 20:34:03.543343 2014-08-13 17:38:02.921590 7fe9bae887a0 20 read_log 194383'18504 (0'0) promote fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by osd.31.0:2977 2014-08-12 20:34:23.099986 2014-08-13 17:38:02.921602 7fe9bae887a0 20 read_log 194383'18505 (194383'18504) modify fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by client.2231190.0:7292 2014-08-12 20:34:23.091293 2014-08-13 17:38:02.921614 7fe9bae887a0 20 read_log 194393'18506 (0'0) promote 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by osd.31.0:2999 2014-08-12 20:49:25.782689 2014-08-13 17:38:02.921626 7fe9bae887a0 20 read_log 194393'18507 (194393'18506) modify 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by client.1064182.0:37 2014-08-12 20:49:25.574065 2014-08-13 17:38:02.921639 7fe9bae887a0 20 read_log 194416'18508 (0'0) promote 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by osd.31.0:3151 2014-08-13 15:28:33.652423 2014-08-13 17:38:02.921650 7fe9bae887a0 20 read_log 194416'18509 (194416'18508) modify 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by client.2441958.0:788 2014-08-13 15:28:33.643812 2014-08-13 17:38:02.921663 7fe9bae887a0 20 read_log 194416'18510 (0'0) promote fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by osd.31.0:3160 2014-08-13 15:28:36.604011 2014-08-13 17:38:02.921675 7fe9bae887a0 20 read_log 194416'18511 (194416'18510) modify fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by client.2441958.0:1552 2014-08-13 15:28:36.596389 2014-08-13 17:38:02.921687 7fe9bae887a0 20 read_log 194416'18512 (0'0) promote 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by osd.31.0:3166 2014-08-13 15:28:39.783912 2014-08-13 17:38:02.921706 7fe9bae887a0 20 read_log 194416'18513 (194416'18512) modify 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by client.2441958.0:2395 2014-08-13 15:28:39.775215 2014-08-13 17:38:02.921719 7fe9bae887a0 20 read_log 194416'18514 (0'0) promote ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by osd.31.0:3191 2014-08-13 15:28:49.476368 2014-08-13 17:38:02.921731 7fe9bae887a0 20 read_log 194416'18515 (194416'18514) modify ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by client.2441958.0:4890 2014-08-13 15:28:49.468609 2014-08-13 17:38:02.921743 7fe9bae887a0 20 read_log 194416'18516 (0'0) promote a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by osd.31.0:3229 2014-08-13 15:29:00.207103 2014-08-13 17:38:02.921755 7fe9bae887a0 20 read_log 194416'18517 (194416'18516) modify a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by client.2441958.0:7648 2014-08-13 15:29:00.199648 2014-08-13 17:38:02.921767 7fe9bae887a0 20 read_log 194416'18518 (0'0) promote 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by osd.31.0:3231 2014-08-13 15:29:00.319737 2014-08-13 17:38:02.921779 7fe9bae887a0 20 read_log 194416'18519 (194416'18518) modify 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by client.2441958.0:7678 2014-08-13 15:29:00.311236 2014-08-13 17:38:02.921793 7fe9bae887a0 20 read_log 194416'18520 (0'0) promote 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by osd.31.0:3239 2014-08-13 15:29:03.131875 2014-08-13 17:38:02.921806 7fe9bae887a0 20 read_log 194416'18521 (194416'18520) modify 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by client.2441958.0:8424 2014-08-13 15:29:03.124637 2014-08-13 17:38:02.921819 7fe9bae887a0 20 read_log 194416'18522 (0'0) promote 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by osd.31.0:3263 2014-08-13 15:29:10.631590 2014-08-13 17:38:02.921831 7fe9bae887a0 20 read_log 194416'18523 (194416'18522) modify 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by client.2441958.0:10287 2014-08-13 15:29:10.623621 2014-08-13 17:38:02.921843 7fe9bae887a0 20 read_log 194416'18524 (0'0) promote a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by osd.31.0:3271 2014-08-13 15:29:14.538227 2014-08-13 17:38:02.921855 7fe9bae887a0 20 read_log 194416'18525 (194416'18524) modify a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by client.2441958.0:11262 2014-08-13 15:29:14.530778 2014-08-13 17:38:02.921866 7fe9bae887a0 20 read_log 194416'18526 (0'0) promote 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by osd.31.0:3291 2014-08-13 15:29:19.404865 2014-08-13 17:38:02.921878 7fe9bae887a0 20 read_log 194416'18527 (194416'18526) modify 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by client.2441958.0:12424 2014-08-13 15:29:19.397388 2014-08-13 17:38:02.921890 7fe9bae887a0 20 read_log 194416'18528 (0'0) promote 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by osd.31.0:3293 2014-08-13 15:29:20.407581 2014-08-13 17:38:02.921902 7fe9bae887a0 20 read_log 194416'18529 (194416'18528) modify 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by client.2441958.0:12661 2014-08-13 15:29:20.398665 2014-08-13 17:38:02.921914 7fe9bae887a0 20 read_log 194416'18530 (0'0) promote c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by osd.31.0:3299 2014-08-13 15:29:21.286211 2014-08-13 17:38:02.921926 7fe9bae887a0 20 read_log 194416'18531 (194416'18530) modify c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by client.2441958.0:12860 2014-08-13 15:29:21.278560 2014-08-13 17:38:02.921938 7fe9bae887a0 20 read_log 194416'18532 (0'0) promote a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by osd.31.0:3301 2014-08-13 15:29:21.520683 2014-08-13 17:38:02.921957 7fe9bae887a0 20 read_log 194416'18533 (194416'18532) modify a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by client.2441958.0:12921 2014-08-13 15:29:21.511554 2014-08-13 17:38:02.921969 7fe9bae887a0 20 read_log 194416'18534 (0'0) promote 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by osd.31.0:3303 2014-08-13 15:29:22.370597 2014-08-13 17:38:02.921981 7fe9bae887a0 20 read_log 194416'18535 (194416'18534) modify 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by client.2441958.0:13087 2014-08-13 15:29:22.362326 2014-08-13 17:38:02.921994 7fe9bae887a0 20 read_log 194416'18536 (0'0) promote 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by osd.31.0:3319 2014-08-13 15:29:26.351879 2014-08-13 17:38:02.922007 7fe9bae887a0 20 read_log 194416'18537 (194416'18536) modify 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by client.2441958.0:14037 2014-08-13 15:29:26.343605 2014-08-13 17:38:02.922019 7fe9bae887a0 20 read_log 194416'18538 (0'0) promote cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by osd.31.0:3323 2014-08-13 15:29:27.696510 2014-08-13 17:38:02.922031 7fe9bae887a0 20 read_log 194416'18539 (194416'18538) modify cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by client.2441958.0:14386 2014-08-13 15:29:27.688209 2014-08-13 17:38:02.922043 7fe9bae887a0 20 read_log 194416'18540 (0'0) promote 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by osd.31.0:3325 2014-08-13 15:29:27.720906 2014-08-13 17:38:02.922055 7fe9bae887a0 20 read_log 194416'18541 (194416'18540) modify 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by client.2441958.0:14390 2014-08-13 15:29:27.703558 2014-08-13 17:38:02.922067 7fe9bae887a0 20 read_log 194416'18542 (0'0) promote 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by osd.31.0:3335 2014-08-13 15:29:28.840896 2014-08-13 17:38:02.922079 7fe9bae887a0 20 read_log 194416'18543 (194416'18542) modify 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by client.2441958.0:14663 2014-08-13 15:29:28.831766 2014-08-13 17:38:02.922091 7fe9bae887a0 20 read_log 194416'18544 (0'0) promote dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by osd.31.0:3341 2014-08-13 15:29:29.768284 2014-08-13 17:38:02.922103 7fe9bae887a0 20 read_log 194416'18545 (194416'18544) modify dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by client.2441958.0:14881 2014-08-13 15:29:29.757749 2014-08-13 17:38:02.922115 7fe9bae887a0 20 read_log 194422'18546 (0'0) promote a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by osd.31.0:3379 2014-08-13 15:32:18.068140 2014-08-13 17:38:02.922127 7fe9bae887a0 20 read_log 194422'18547 (194422'18546) modify a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by client.1148203.0:677 2014-08-13 15:32:18.061160 2014-08-13 17:38:02.922139 7fe9bae887a0 20 read_log 194422'18548 (0'0) promote fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by osd.31.0:3383 2014-08-13 15:32:19.172976 2014-08-13 17:38:02.922151 7fe9bae887a0 20 read_log 194422'18549 (194422'18548) modify fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by client.1148203.0:967 2014-08-13 15:32:19.166187 2014-08-13 17:38:02.922168 7fe9bae887a0 20 read_log 194422'18550 (0'0) promote 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by osd.31.0:3421 2014-08-13 15:32:27.870777 2014-08-13 17:38:02.922180 7fe9bae887a0 20 read_log 194422'18551 (194422'18550) modify 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by client.1148203.0:3204 2014-08-13 15:32:27.863636 2014-08-13 17:38:02.922194 7fe9bae887a0 20 read_log 194422'18552 (0'0) promote c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by osd.31.0:3423 2014-08-13 15:32:29.573280 2014-08-13 17:38:02.922207 7fe9bae887a0 20 read_log 194422'18553 (194422'18552) modify c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by client.1148203.0:3622 2014-08-13 15:32:29.564248 2014-08-13 17:38:02.922220 7fe9bae887a0 20 read_log 194422'18554 (0'0) promote 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by osd.31.0:3447 2014-08-13 15:32:39.301084 2014-08-13 17:38:02.922231 7fe9bae887a0 20 read_log 194422'18555 (194422'18554) modify 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by client.1148203.0:5849 2014-08-13 15:32:39.293513 2014-08-13 17:38:02.922244 7fe9bae887a0 20 read_log 194422'18556 (0'0) promote 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by osd.31.0:3451 2014-08-13 15:32:40.821318 2014-08-13 17:38:02.922256 7fe9bae887a0 20 read_log 194422'18557 (194422'18556) modify 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by client.1148203.0:6128 2014-08-13 15:32:40.813701 2014-08-13 17:38:02.922268 7fe9bae887a0 20 read_log 194422'18558 (0'0) promote d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by osd.31.0:3455 2014-08-13 15:32:45.294557 2014-08-13 17:38:02.922279 7fe9bae887a0 20 read_log 194422'18559 (194422'18558) modify d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by client.1148203.0:6965 2014-08-13 15:32:45.286396 2014-08-13 17:38:02.922292 7fe9bae887a0 20 read_log 194422'18560 (0'0) promote a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by osd.31.0:3461 2014-08-13 15:32:47.070933 2014-08-13 17:38:02.922303 7fe9bae887a0 20 read_log 194422'18561 (194422'18560) modify a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by client.1148203.0:7254 2014-08-13 15:32:47.062071 2014-08-13 17:38:02.922320 7fe9bae887a0 20 read_log 194422'18562 (0'0) promote c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by osd.31.0:3475 2014-08-13 15:32:57.016344 2014-08-13 17:38:02.922332 7fe9bae887a0 20 read_log 194422'18563 (194422'18562) modify c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by client.1148203.0:8737 2014-08-13 15:32:57.007687 2014-08-13 17:38:02.922344 7fe9bae887a0 20 read_log 194422'18564 (0'0) promote 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by osd.31.0:3485 2014-08-13 15:33:00.349920 2014-08-13 17:38:02.922356 7fe9bae887a0 20 read_log 194422'18565 (194422'18564) modify 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by client.1148203.0:9077 2014-08-13 15:33:00.341640 2014-08-13 17:38:02.922368 7fe9bae887a0 20 read_log 194422'18566 (0'0) promote 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by osd.31.0:3503 2014-08-13 15:33:23.376678 2014-08-13 17:38:02.922381 7fe9bae887a0 20 read_log 194422'18567 (194422'18566) modify 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by client.1148203.0:10479 2014-08-13 15:33:23.366178 2014-08-13 17:38:02.922396 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.922802 7fe9bae887a0 10 read_log checking for missing items over interval (194370'18476,194422'18567] 2014-08-13 17:38:02.922848 7fe9bae887a0 15 read_log missing 194422'18567 (194422'18566) modify 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by client.1148203.0:10479 2014-08-13 15:33:23.366178 2014-08-13 17:38:02.922889 7fe9bae887a0 15 read_log missing 194422'18565 (194422'18564) modify 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by client.1148203.0:9077 2014-08-13 15:33:00.341640 2014-08-13 17:38:02.922927 7fe9bae887a0 15 read_log missing 194422'18563 (194422'18562) modify c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by client.1148203.0:8737 2014-08-13 15:32:57.007687 2014-08-13 17:38:02.922964 7fe9bae887a0 15 read_log missing 194422'18561 (194422'18560) modify a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by client.1148203.0:7254 2014-08-13 15:32:47.062071 2014-08-13 17:38:02.923001 7fe9bae887a0 15 read_log missing 194422'18559 (194422'18558) modify d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by client.1148203.0:6965 2014-08-13 15:32:45.286396 2014-08-13 17:38:02.923038 7fe9bae887a0 15 read_log missing 194422'18557 (194422'18556) modify 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by client.1148203.0:6128 2014-08-13 15:32:40.813701 2014-08-13 17:38:02.923075 7fe9bae887a0 15 read_log missing 194422'18555 (194422'18554) modify 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by client.1148203.0:5849 2014-08-13 15:32:39.293513 2014-08-13 17:38:02.923112 7fe9bae887a0 15 read_log missing 194422'18553 (194422'18552) modify c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by client.1148203.0:3622 2014-08-13 15:32:29.564248 2014-08-13 17:38:02.923148 7fe9bae887a0 15 read_log missing 194422'18551 (194422'18550) modify 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by client.1148203.0:3204 2014-08-13 15:32:27.863636 2014-08-13 17:38:02.923199 7fe9bae887a0 15 read_log missing 194422'18549 (194422'18548) modify fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by client.1148203.0:967 2014-08-13 15:32:19.166187 2014-08-13 17:38:02.923236 7fe9bae887a0 15 read_log missing 194422'18547 (194422'18546) modify a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by client.1148203.0:677 2014-08-13 15:32:18.061160 2014-08-13 17:38:02.923273 7fe9bae887a0 15 read_log missing 194416'18545 (194416'18544) modify dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by client.2441958.0:14881 2014-08-13 15:29:29.757749 2014-08-13 17:38:02.923310 7fe9bae887a0 15 read_log missing 194416'18543 (194416'18542) modify 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by client.2441958.0:14663 2014-08-13 15:29:28.831766 2014-08-13 17:38:02.923347 7fe9bae887a0 15 read_log missing 194416'18541 (194416'18540) modify 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by client.2441958.0:14390 2014-08-13 15:29:27.703558 2014-08-13 17:38:02.923383 7fe9bae887a0 15 read_log missing 194416'18539 (194416'18538) modify cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by client.2441958.0:14386 2014-08-13 15:29:27.688209 2014-08-13 17:38:02.923420 7fe9bae887a0 15 read_log missing 194416'18537 (194416'18536) modify 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by client.2441958.0:14037 2014-08-13 15:29:26.343605 2014-08-13 17:38:02.923456 7fe9bae887a0 15 read_log missing 194416'18535 (194416'18534) modify 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by client.2441958.0:13087 2014-08-13 15:29:22.362326 2014-08-13 17:38:02.923492 7fe9bae887a0 15 read_log missing 194416'18533 (194416'18532) modify a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by client.2441958.0:12921 2014-08-13 15:29:21.511554 2014-08-13 17:38:02.923530 7fe9bae887a0 15 read_log missing 194416'18531 (194416'18530) modify c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by client.2441958.0:12860 2014-08-13 15:29:21.278560 2014-08-13 17:38:02.923566 7fe9bae887a0 15 read_log missing 194416'18529 (194416'18528) modify 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by client.2441958.0:12661 2014-08-13 15:29:20.398665 2014-08-13 17:38:02.923602 7fe9bae887a0 15 read_log missing 194416'18527 (194416'18526) modify 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by client.2441958.0:12424 2014-08-13 15:29:19.397388 2014-08-13 17:38:02.923639 7fe9bae887a0 15 read_log missing 194416'18525 (194416'18524) modify a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by client.2441958.0:11262 2014-08-13 15:29:14.530778 2014-08-13 17:38:02.923675 7fe9bae887a0 15 read_log missing 194416'18523 (194416'18522) modify 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by client.2441958.0:10287 2014-08-13 15:29:10.623621 2014-08-13 17:38:02.923712 7fe9bae887a0 15 read_log missing 194416'18521 (194416'18520) modify 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by client.2441958.0:8424 2014-08-13 15:29:03.124637 2014-08-13 17:38:02.923749 7fe9bae887a0 15 read_log missing 194416'18519 (194416'18518) modify 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by client.2441958.0:7678 2014-08-13 15:29:00.311236 2014-08-13 17:38:02.923806 7fe9bae887a0 15 read_log missing 194416'18517 (194416'18516) modify a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by client.2441958.0:7648 2014-08-13 15:29:00.199648 2014-08-13 17:38:02.923844 7fe9bae887a0 15 read_log missing 194416'18515 (194416'18514) modify ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by client.2441958.0:4890 2014-08-13 15:28:49.468609 2014-08-13 17:38:02.923882 7fe9bae887a0 15 read_log missing 194416'18513 (194416'18512) modify 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by client.2441958.0:2395 2014-08-13 15:28:39.775215 2014-08-13 17:38:02.923918 7fe9bae887a0 15 read_log missing 194416'18511 (194416'18510) modify fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by client.2441958.0:1552 2014-08-13 15:28:36.596389 2014-08-13 17:38:02.923955 7fe9bae887a0 15 read_log missing 194416'18509 (194416'18508) modify 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by client.2441958.0:788 2014-08-13 15:28:33.643812 2014-08-13 17:38:02.923993 7fe9bae887a0 15 read_log missing 194393'18507 (194393'18506) modify 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by client.1064182.0:37 2014-08-12 20:49:25.574065 2014-08-13 17:38:02.924029 7fe9bae887a0 15 read_log missing 194383'18505 (194383'18504) modify fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by client.2231190.0:7292 2014-08-12 20:34:23.091293 2014-08-13 17:38:02.924066 7fe9bae887a0 15 read_log missing 194383'18503 (194383'18502) modify bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by client.2231190.0:6390 2014-08-12 20:34:03.543343 2014-08-13 17:38:02.924102 7fe9bae887a0 15 read_log missing 194382'18501 (194382'18500) modify cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by client.2231190.0:5682 2014-08-12 20:33:59.536043 2014-08-13 17:38:02.924139 7fe9bae887a0 15 read_log missing 194380'18499 (194380'18498) modify 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by client.2231190.0:5035 2014-08-12 20:33:23.482176 2014-08-13 17:38:02.924181 7fe9bae887a0 15 read_log missing 194377'18497 (194377'18496) modify f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by client.2231190.0:2150 2014-08-12 20:33:05.869661 2014-08-13 17:38:02.924218 7fe9bae887a0 15 read_log missing 194377'18495 (194377'18494) modify 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by client.2231190.0:23 2014-08-12 20:32:57.185894 2014-08-13 17:38:02.924255 7fe9bae887a0 15 read_log missing 194370'18493 (194370'18492) modify 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by client.1508531.0:11971 2014-08-12 20:31:11.141221 2014-08-13 17:38:02.924292 7fe9bae887a0 15 read_log missing 194370'18491 (194370'18490) modify d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by client.1508531.0:9336 2014-08-12 20:30:55.321023 2014-08-13 17:38:02.924329 7fe9bae887a0 15 read_log missing 194370'18489 (194370'18488) modify 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by client.1508531.0:8851 2014-08-12 20:30:52.354717 2014-08-13 17:38:02.924365 7fe9bae887a0 15 read_log missing 194370'18487 (194370'18486) modify 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by client.1508531.0:6343 2014-08-12 20:30:39.217114 2014-08-13 17:38:02.924402 7fe9bae887a0 15 read_log missing 194370'18485 (194370'18484) modify e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by client.1508531.0:5021 2014-08-12 20:30:33.348586 2014-08-13 17:38:02.924438 7fe9bae887a0 15 read_log missing 194370'18483 (194370'18482) modify 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by client.1508531.0:3580 2014-08-12 20:30:27.698415 2014-08-13 17:38:02.924475 7fe9bae887a0 15 read_log missing 194370'18481 (194370'18480) modify ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by client.1508531.0:1938 2014-08-12 20:30:21.598856 2014-08-13 17:38:02.924511 7fe9bae887a0 15 read_log missing 194370'18479 (194370'18478) modify d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by client.1508531.0:305 2014-08-12 20:30:15.416579 2014-08-13 17:38:02.924548 7fe9bae887a0 15 read_log missing 194370'18477 (194370'18476) modify e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by client.1508531.0:93 2014-08-12 20:30:14.442892 2014-08-13 17:38:02.924564 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.924589 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=0 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive m=46] handle_loaded 2014-08-13 17:38:02.924606 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=0 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Initial 0.012401 0 0.000000 2014-08-13 17:38:02.924618 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=0 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset 2014-08-13 17:38:02.924627 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=0 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194478 2014-08-13 17:38:02.924636 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log((192834'16484,194422'18567], crt=194422'18567) 2014-08-13 17:38:02.924645 7fe9bae887a0 10 osd.4 194478 pgid 24.1e6 coll 24.1e6_head 2014-08-13 17:38:02.924785 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.1e6 2014-08-13 17:38:02.924798 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.924846 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1e6(unlocked)] enter Initial 2014-08-13 17:38:02.924857 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.1e6(unlocked)] enter NotTrimming 2014-08-13 17:38:02.924982 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.925106 7fe9bae887a0 20 read_log 192834'15882 (0'0) promote 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by osd.364.0:17853 2014-08-11 15:23:39.586904 2014-08-13 17:38:02.925126 7fe9bae887a0 20 read_log 192834'15883 (192834'15882) modify 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.933448.0:1026 2014-08-11 15:23:39.580290 2014-08-13 17:38:02.925140 7fe9bae887a0 20 read_log 192834'15894 (0'0) promote 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by osd.364.0:17865 2014-08-11 15:23:43.322273 2014-08-13 17:38:02.925152 7fe9bae887a0 20 read_log 192834'15895 (192834'15894) modify 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.933448.0:1832 2014-08-11 15:23:43.312343 2014-08-13 17:38:02.925173 7fe9bae887a0 20 read_log 192834'15902 (0'0) promote 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by osd.364.0:17873 2014-08-11 15:23:49.780370 2014-08-13 17:38:02.925185 7fe9bae887a0 20 read_log 192834'15903 (192834'15902) modify 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.933448.0:3010 2014-08-11 15:23:49.772085 2014-08-13 17:38:02.925198 7fe9bae887a0 20 read_log 192834'15918 (0'0) promote 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by osd.364.0:17889 2014-08-11 15:23:55.304268 2014-08-13 17:38:02.925210 7fe9bae887a0 20 read_log 192834'15919 (192834'15918) modify 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.933448.0:4059 2014-08-11 15:23:55.296455 2014-08-13 17:38:02.925222 7fe9bae887a0 20 read_log 192834'15920 (0'0) promote 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by osd.364.0:17891 2014-08-11 15:23:55.464693 2014-08-13 17:38:02.925234 7fe9bae887a0 20 read_log 192834'15921 (192834'15920) modify 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.933448.0:4100 2014-08-11 15:23:55.456664 2014-08-13 17:38:02.925250 7fe9bae887a0 20 read_log 192834'15922 (0'0) promote 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by osd.364.0:17893 2014-08-11 15:23:56.084202 2014-08-13 17:38:02.925263 7fe9bae887a0 20 read_log 192834'15923 (192834'15922) modify 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.933448.0:4254 2014-08-11 15:23:56.077638 2014-08-13 17:38:02.925275 7fe9bae887a0 20 read_log 192834'15930 (0'0) promote b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by osd.364.0:17901 2014-08-11 15:24:02.129142 2014-08-13 17:38:02.925287 7fe9bae887a0 20 read_log 192834'15931 (192834'15930) modify b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.933448.0:5590 2014-08-11 15:24:02.120743 2014-08-13 17:38:02.925300 7fe9bae887a0 20 read_log 192834'15934 (0'0) promote f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by osd.364.0:17905 2014-08-11 15:24:02.670617 2014-08-13 17:38:02.925312 7fe9bae887a0 20 read_log 192834'15935 (192834'15934) modify f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.933448.0:5730 2014-08-11 15:24:02.661350 2014-08-13 17:38:02.925324 7fe9bae887a0 20 read_log 192834'15952 (0'0) promote 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by osd.364.0:17923 2014-08-11 15:24:09.267619 2014-08-13 17:38:02.925352 7fe9bae887a0 20 read_log 192834'15953 (192834'15952) modify 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.933448.0:7234 2014-08-11 15:24:09.260092 2014-08-13 17:38:02.925369 7fe9bae887a0 20 read_log 192834'15984 (0'0) promote dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by osd.364.0:17955 2014-08-11 15:24:16.262325 2014-08-13 17:38:02.925382 7fe9bae887a0 20 read_log 192834'15985 (192834'15984) modify dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.933448.0:8938 2014-08-11 15:24:16.254647 2014-08-13 17:38:02.925394 7fe9bae887a0 20 read_log 192834'15992 (0'0) promote 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by osd.364.0:17963 2014-08-11 15:24:17.397424 2014-08-13 17:38:02.925406 7fe9bae887a0 20 read_log 192834'15993 (192834'15992) modify 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.933448.0:9059 2014-08-11 15:24:17.389655 2014-08-13 17:38:02.925418 7fe9bae887a0 20 read_log 192834'16002 (0'0) promote c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by osd.364.0:17973 2014-08-11 15:24:19.384870 2014-08-13 17:38:02.925450 7fe9bae887a0 20 read_log 192834'16003 (192834'16002) modify c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.933448.0:9571 2014-08-11 15:24:19.376173 2014-08-13 17:38:02.925465 7fe9bae887a0 20 read_log 192834'16016 (0'0) promote e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by osd.364.0:17987 2014-08-11 15:24:22.418231 2014-08-13 17:38:02.925478 7fe9bae887a0 20 read_log 192834'16017 (192834'16016) modify e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.933448.0:10287 2014-08-11 15:24:22.405262 2014-08-13 17:38:02.925490 7fe9bae887a0 20 read_log 192834'16046 (0'0) promote af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by osd.364.0:18017 2014-08-11 15:24:32.428319 2014-08-13 17:38:02.925502 7fe9bae887a0 20 read_log 192834'16047 (192834'16046) modify af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.933448.0:12413 2014-08-11 15:24:32.251592 2014-08-13 17:38:02.925514 7fe9bae887a0 20 read_log 192834'16050 (0'0) promote 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by osd.364.0:18021 2014-08-11 15:24:33.842704 2014-08-13 17:38:02.925525 7fe9bae887a0 20 read_log 192834'16051 (192834'16050) modify 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.933448.0:12785 2014-08-11 15:24:33.834503 2014-08-13 17:38:02.925538 7fe9bae887a0 20 read_log 192834'16068 (0'0) promote f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by osd.364.0:18039 2014-08-11 15:24:40.242968 2014-08-13 17:38:02.925549 7fe9bae887a0 20 read_log 192834'16069 (192834'16068) modify f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.933448.0:13865 2014-08-11 15:24:40.234958 2014-08-13 17:38:02.925561 7fe9bae887a0 20 read_log 192834'16076 (0'0) promote 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by osd.364.0:18047 2014-08-11 15:24:45.542179 2014-08-13 17:38:02.925573 7fe9bae887a0 20 read_log 192834'16077 (192834'16076) modify 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.933448.0:14446 2014-08-11 15:24:45.534415 2014-08-13 17:38:02.925585 7fe9bae887a0 20 read_log 192834'16084 (0'0) promote 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by osd.364.0:18055 2014-08-11 15:24:47.863403 2014-08-13 17:38:02.925597 7fe9bae887a0 20 read_log 192834'16085 (192834'16084) modify 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.933448.0:14964 2014-08-11 15:24:47.856554 2014-08-13 17:38:02.925609 7fe9bae887a0 20 read_log 192834'16098 (0'0) promote 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by osd.364.0:18069 2014-08-11 15:24:52.941515 2014-08-13 17:38:02.925645 7fe9bae887a0 20 read_log 192834'16099 (192834'16098) modify 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.933448.0:16084 2014-08-11 15:24:52.929345 2014-08-13 17:38:02.925658 7fe9bae887a0 20 read_log 192834'16102 (0'0) promote b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by osd.364.0:18073 2014-08-11 15:24:55.790808 2014-08-13 17:38:02.925670 7fe9bae887a0 20 read_log 192834'16103 (192834'16102) modify b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.933448.0:16455 2014-08-11 15:24:55.783520 2014-08-13 17:38:02.925682 7fe9bae887a0 20 read_log 192834'16112 (0'0) promote 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by osd.364.0:18083 2014-08-11 15:24:57.721227 2014-08-13 17:38:02.925694 7fe9bae887a0 20 read_log 192834'16113 (192834'16112) modify 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.933448.0:16920 2014-08-11 15:24:57.714232 2014-08-13 17:38:02.925706 7fe9bae887a0 20 read_log 192834'16130 (0'0) promote 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by osd.364.0:18101 2014-08-11 15:25:05.292172 2014-08-13 17:38:02.925718 7fe9bae887a0 20 read_log 192834'16131 (192834'16130) modify 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.933448.0:18432 2014-08-11 15:25:05.285860 2014-08-13 17:38:02.925730 7fe9bae887a0 20 read_log 192834'16148 (0'0) promote e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by osd.364.0:18119 2014-08-11 15:25:13.939729 2014-08-13 17:38:02.925742 7fe9bae887a0 20 read_log 192834'16149 (192834'16148) modify e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.933448.0:19910 2014-08-11 15:25:13.932531 2014-08-13 17:38:02.925754 7fe9bae887a0 20 read_log 192834'16172 (0'0) promote afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by osd.364.0:18143 2014-08-11 15:25:28.699526 2014-08-13 17:38:02.925766 7fe9bae887a0 20 read_log 192834'16173 (192834'16172) modify afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.933448.0:21357 2014-08-11 15:25:28.692663 2014-08-13 17:38:02.925778 7fe9bae887a0 20 read_log 192834'16195 (192834'15883) modify 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.933448.0:23317 2014-08-11 15:25:39.220819 2014-08-13 17:38:02.925790 7fe9bae887a0 20 read_log 192834'16201 (192834'15895) modify 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.933448.0:24123 2014-08-11 15:25:40.307083 2014-08-13 17:38:02.925802 7fe9bae887a0 20 read_log 192834'16205 (192834'15903) modify 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.933448.0:25301 2014-08-11 15:25:41.980288 2014-08-13 17:38:02.925814 7fe9bae887a0 20 read_log 192834'16212 (192834'15919) modify 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.933448.0:26350 2014-08-11 15:25:43.290252 2014-08-13 17:38:02.925826 7fe9bae887a0 20 read_log 192834'16213 (192834'15921) modify 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.933448.0:26391 2014-08-11 15:25:43.341983 2014-08-13 17:38:02.925838 7fe9bae887a0 20 read_log 192834'16214 (192834'15923) modify 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.933448.0:26545 2014-08-11 15:25:43.524028 2014-08-13 17:38:02.925850 7fe9bae887a0 20 read_log 192834'16218 (192834'15931) modify b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.933448.0:27881 2014-08-11 15:25:45.223632 2014-08-13 17:38:02.925862 7fe9bae887a0 20 read_log 192834'16220 (192834'15935) modify f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.933448.0:28021 2014-08-11 15:25:45.393622 2014-08-13 17:38:02.925885 7fe9bae887a0 20 read_log 192834'16229 (192834'15953) modify 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.933448.0:29525 2014-08-11 15:25:47.429831 2014-08-13 17:38:02.925906 7fe9bae887a0 20 read_log 192834'16245 (192834'15985) modify dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.933448.0:31229 2014-08-11 15:25:49.820651 2014-08-13 17:38:02.925921 7fe9bae887a0 20 read_log 192834'16249 (192834'15993) modify 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.933448.0:31350 2014-08-11 15:25:49.955965 2014-08-13 17:38:02.925933 7fe9bae887a0 20 read_log 192834'16254 (192834'16003) modify c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.933448.0:31862 2014-08-11 15:25:50.702478 2014-08-13 17:38:02.925945 7fe9bae887a0 20 read_log 192834'16261 (192834'16017) modify e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.933448.0:32578 2014-08-11 15:25:52.040119 2014-08-13 17:38:02.925959 7fe9bae887a0 20 read_log 192834'16276 (192834'16047) modify af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.933448.0:34704 2014-08-11 15:25:55.380170 2014-08-13 17:38:02.925972 7fe9bae887a0 20 read_log 192834'16278 (192834'16051) modify 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.933448.0:35076 2014-08-11 15:25:55.776916 2014-08-13 17:38:02.925986 7fe9bae887a0 20 read_log 192834'16287 (192834'16069) modify f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.933448.0:36156 2014-08-11 15:25:57.304427 2014-08-13 17:38:02.925999 7fe9bae887a0 20 read_log 192834'16291 (192834'16077) modify 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.933448.0:36737 2014-08-11 15:25:58.255174 2014-08-13 17:38:02.926011 7fe9bae887a0 20 read_log 192834'16295 (192834'16085) modify 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.933448.0:37255 2014-08-11 15:25:59.212294 2014-08-13 17:38:02.926023 7fe9bae887a0 20 read_log 192834'16302 (192834'16099) modify 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.933448.0:38375 2014-08-11 15:26:00.827633 2014-08-13 17:38:02.926034 7fe9bae887a0 20 read_log 192834'16304 (192834'16103) modify b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.933448.0:38746 2014-08-11 15:26:01.211878 2014-08-13 17:38:02.926049 7fe9bae887a0 20 read_log 192834'16309 (192834'16113) modify 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.933448.0:39211 2014-08-11 15:26:01.672923 2014-08-13 17:38:02.926061 7fe9bae887a0 20 read_log 192834'16318 (192834'16131) modify 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.933448.0:40723 2014-08-11 15:26:02.710004 2014-08-13 17:38:02.926074 7fe9bae887a0 20 read_log 192834'16327 (192834'16149) modify e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.933448.0:42201 2014-08-11 15:26:03.187455 2014-08-13 17:38:02.926086 7fe9bae887a0 20 read_log 192834'16339 (192834'16173) modify afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.933448.0:43648 2014-08-11 15:26:03.685002 2014-08-13 17:38:02.926100 7fe9bae887a0 20 read_log 192834'16367 (0'0) promote a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by osd.364.0:18338 2014-08-11 15:26:18.652144 2014-08-13 17:38:02.926112 7fe9bae887a0 20 read_log 192834'16368 (192834'16367) modify a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.1883634.0:1146 2014-08-11 15:26:18.644718 2014-08-13 17:38:02.926124 7fe9bae887a0 20 read_log 192834'16391 (0'0) promote ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by osd.364.0:18362 2014-08-11 15:26:24.194229 2014-08-13 17:38:02.926139 7fe9bae887a0 20 read_log 192834'16392 (192834'16391) modify ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.1883634.0:2642 2014-08-11 15:26:24.186026 2014-08-13 17:38:02.926151 7fe9bae887a0 20 read_log 192834'16401 (0'0) promote 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by osd.364.0:18372 2014-08-11 15:26:27.375534 2014-08-13 17:38:02.926182 7fe9bae887a0 20 read_log 192834'16402 (192834'16401) modify 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.1883634.0:3542 2014-08-11 15:26:27.369036 2014-08-13 17:38:02.926197 7fe9bae887a0 20 read_log 192834'16413 (0'0) promote 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by osd.364.0:18384 2014-08-11 15:26:29.542361 2014-08-13 17:38:02.926209 7fe9bae887a0 20 read_log 192834'16414 (192834'16413) modify 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.1883634.0:4154 2014-08-11 15:26:29.534485 2014-08-13 17:38:02.926226 7fe9bae887a0 20 read_log 192834'16419 (0'0) promote 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by osd.364.0:18390 2014-08-11 15:26:31.070202 2014-08-13 17:38:02.926250 7fe9bae887a0 20 read_log 192834'16420 (192834'16419) modify 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.1883634.0:4562 2014-08-11 15:26:31.062838 2014-08-13 17:38:02.926265 7fe9bae887a0 20 read_log 192834'16447 (0'0) promote 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by osd.364.0:18418 2014-08-11 15:26:36.335015 2014-08-13 17:38:02.926277 7fe9bae887a0 20 read_log 192834'16448 (192834'16447) modify 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.1883634.0:5979 2014-08-11 15:26:36.325899 2014-08-13 17:38:02.926290 7fe9bae887a0 20 read_log 192834'16467 (0'0) promote f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by osd.364.0:18438 2014-08-11 15:26:42.003939 2014-08-13 17:38:02.926302 7fe9bae887a0 20 read_log 192834'16468 (192834'16467) modify f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.1883634.0:7394 2014-08-11 15:26:41.996874 2014-08-13 17:38:02.926314 7fe9bae887a0 20 read_log 192834'16471 (0'0) promote 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by osd.364.0:18442 2014-08-11 15:26:43.029553 2014-08-13 17:38:02.926328 7fe9bae887a0 20 read_log 192834'16472 (192834'16471) modify 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.1883634.0:7622 2014-08-11 15:26:43.021221 2014-08-13 17:38:02.926340 7fe9bae887a0 20 read_log 192834'16477 (0'0) promote d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by osd.364.0:18448 2014-08-11 15:26:45.635271 2014-08-13 17:38:02.926354 7fe9bae887a0 20 read_log 192834'16478 (192834'16477) modify d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.1883634.0:8276 2014-08-11 15:26:45.626360 2014-08-13 17:38:02.926366 7fe9bae887a0 20 read_log 192834'16501 (0'0) promote da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by osd.364.0:18472 2014-08-11 15:26:51.199698 2014-08-13 17:38:02.926381 7fe9bae887a0 20 read_log 192834'16502 (192834'16501) modify da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.1883634.0:9534 2014-08-11 15:26:51.191829 2014-08-13 17:38:02.926393 7fe9bae887a0 20 read_log 192834'16515 (0'0) promote c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by osd.364.0:18486 2014-08-11 15:26:57.113244 2014-08-13 17:38:02.926407 7fe9bae887a0 20 read_log 192834'16516 (192834'16515) modify c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.1883634.0:10823 2014-08-11 15:26:57.103972 2014-08-13 17:38:02.926420 7fe9bae887a0 20 read_log 192834'16547 (0'0) promote c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by osd.364.0:18518 2014-08-11 15:27:08.836329 2014-08-13 17:38:02.926434 7fe9bae887a0 20 read_log 192834'16548 (192834'16547) modify c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.1883634.0:13319 2014-08-11 15:27:08.829179 2014-08-13 17:38:02.926446 7fe9bae887a0 20 read_log 192834'16553 (0'0) promote 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by osd.364.0:18524 2014-08-11 15:27:10.324141 2014-08-13 17:38:02.926468 7fe9bae887a0 20 read_log 192834'16554 (192834'16553) modify 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.1883634.0:13635 2014-08-11 15:27:10.317673 2014-08-13 17:38:02.926482 7fe9bae887a0 20 read_log 192834'16559 (0'0) promote c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by osd.364.0:18530 2014-08-11 15:27:12.071038 2014-08-13 17:38:02.926495 7fe9bae887a0 20 read_log 192834'16560 (192834'16559) modify c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.1883634.0:14139 2014-08-11 15:27:12.064150 2014-08-13 17:38:02.926510 7fe9bae887a0 20 read_log 192834'16561 (0'0) promote 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by osd.364.0:18532 2014-08-11 15:27:12.561770 2014-08-13 17:38:02.926522 7fe9bae887a0 20 read_log 192834'16562 (192834'16561) modify 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.1883634.0:14278 2014-08-11 15:27:12.554349 2014-08-13 17:38:02.926536 7fe9bae887a0 20 read_log 192834'16587 (0'0) promote 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by osd.364.0:18558 2014-08-11 15:27:21.461374 2014-08-13 17:38:02.926548 7fe9bae887a0 20 read_log 192834'16588 (192834'16587) modify 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.1883634.0:15756 2014-08-11 15:27:21.453911 2014-08-13 17:38:02.926560 7fe9bae887a0 20 read_log 192834'16599 (0'0) promote 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by osd.364.0:18570 2014-08-11 15:27:29.935836 2014-08-13 17:38:02.926572 7fe9bae887a0 20 read_log 192834'16600 (192834'16599) modify 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.1883634.0:16922 2014-08-11 15:27:29.927352 2014-08-13 17:38:02.926584 7fe9bae887a0 20 read_log 192834'16627 (0'0) promote 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by osd.364.0:18598 2014-08-11 15:27:54.067547 2014-08-13 17:38:02.926598 7fe9bae887a0 20 read_log 192834'16628 (192834'16627) modify 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.1883634.0:19768 2014-08-11 15:27:54.060325 2014-08-13 17:38:02.926611 7fe9bae887a0 20 read_log 192834'16637 (0'0) promote 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by osd.364.0:18608 2014-08-11 15:28:02.819015 2014-08-13 17:38:02.926625 7fe9bae887a0 20 read_log 192834'16638 (192834'16637) modify 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.1883634.0:21103 2014-08-11 15:28:02.809836 2014-08-13 17:38:02.926637 7fe9bae887a0 20 read_log 192834'16639 (0'0) promote f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by osd.364.0:18610 2014-08-11 15:28:02.942581 2014-08-13 17:38:02.926652 7fe9bae887a0 20 read_log 192834'16640 (192834'16639) modify f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.1883634.0:21125 2014-08-11 15:28:02.934566 2014-08-13 17:38:02.926664 7fe9bae887a0 20 read_log 192834'16653 (0'0) promote 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by osd.364.0:18624 2014-08-11 15:28:04.329567 2014-08-13 17:38:02.926676 7fe9bae887a0 20 read_log 192834'16654 (192834'16653) modify 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.1883634.0:21487 2014-08-11 15:28:04.321602 2014-08-13 17:38:02.926690 7fe9bae887a0 20 read_log 192834'16657 (0'0) promote 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by osd.364.0:18628 2014-08-11 15:28:05.831506 2014-08-13 17:38:02.926702 7fe9bae887a0 20 read_log 192834'16658 (192834'16657) modify 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.1883634.0:21934 2014-08-11 15:28:05.822858 2014-08-13 17:38:02.926734 7fe9bae887a0 20 read_log 192834'16698 (192834'16368) modify a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.1883634.0:24470 2014-08-11 15:28:19.091751 2014-08-13 17:38:02.926756 7fe9bae887a0 20 read_log 192834'16710 (192834'16392) modify ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.1883634.0:25966 2014-08-11 15:28:22.422382 2014-08-13 17:38:02.926771 7fe9bae887a0 20 read_log 192834'16715 (192834'16402) modify 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.1883634.0:26866 2014-08-11 15:28:23.681913 2014-08-13 17:38:02.926783 7fe9bae887a0 20 read_log 192834'16721 (192834'16414) modify 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.1883634.0:27478 2014-08-11 15:28:24.506889 2014-08-13 17:38:02.926795 7fe9bae887a0 20 read_log 192834'16724 (192834'16420) modify 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.1883634.0:27886 2014-08-11 15:28:25.089688 2014-08-13 17:38:02.926808 7fe9bae887a0 20 read_log 192834'16738 (192834'16448) modify 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.1883634.0:29303 2014-08-11 15:28:26.954655 2014-08-13 17:38:02.926820 7fe9bae887a0 20 read_log 192834'16748 (192834'16468) modify f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.1883634.0:30718 2014-08-11 15:28:29.191728 2014-08-13 17:38:02.926832 7fe9bae887a0 20 read_log 192834'16750 (192834'16472) modify 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.1883634.0:30946 2014-08-11 15:28:29.623850 2014-08-13 17:38:02.926844 7fe9bae887a0 20 read_log 192834'16753 (192834'16478) modify d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.1883634.0:31600 2014-08-11 15:28:30.744187 2014-08-13 17:38:02.926858 7fe9bae887a0 20 read_log 192834'16765 (192834'16502) modify da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.1883634.0:32858 2014-08-11 15:28:32.429009 2014-08-13 17:38:02.926870 7fe9bae887a0 20 read_log 192834'16774 (192834'16516) modify c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.1883634.0:34147 2014-08-11 15:28:34.295829 2014-08-13 17:38:02.926884 7fe9bae887a0 20 read_log 192834'16790 (192834'16548) modify c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.1883634.0:36643 2014-08-11 15:28:38.245882 2014-08-13 17:38:02.926896 7fe9bae887a0 20 read_log 192834'16793 (192834'16554) modify 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.1883634.0:36959 2014-08-11 15:28:38.631608 2014-08-13 17:38:02.926910 7fe9bae887a0 20 read_log 192834'16796 (192834'16560) modify c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.1883634.0:37463 2014-08-11 15:28:39.390165 2014-08-13 17:38:02.926922 7fe9bae887a0 20 read_log 192834'16797 (192834'16562) modify 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.1883634.0:37602 2014-08-11 15:28:39.561928 2014-08-13 17:38:02.926934 7fe9bae887a0 20 read_log 192834'16810 (192834'16588) modify 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.1883634.0:39080 2014-08-11 15:28:41.540857 2014-08-13 17:38:02.926948 7fe9bae887a0 20 read_log 192834'16816 (192834'16600) modify 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.1883634.0:40246 2014-08-11 15:28:43.423233 2014-08-13 17:38:02.926960 7fe9bae887a0 20 read_log 192834'16830 (192834'16628) modify 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.1883634.0:43092 2014-08-11 15:28:46.452820 2014-08-13 17:38:02.926974 7fe9bae887a0 20 read_log 192834'16835 (192834'16638) modify 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.1883634.0:44427 2014-08-11 15:28:46.855800 2014-08-13 17:38:02.926986 7fe9bae887a0 20 read_log 192834'16836 (192834'16640) modify f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.1883634.0:44449 2014-08-11 15:28:46.860759 2014-08-13 17:38:02.927000 7fe9bae887a0 20 read_log 192834'16843 (192834'16654) modify 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.1883634.0:44811 2014-08-11 15:28:46.966212 2014-08-13 17:38:02.927022 7fe9bae887a0 20 read_log 192834'16845 (192834'16658) modify 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.1883634.0:45258 2014-08-11 15:28:47.217296 2014-08-13 17:38:02.927039 7fe9bae887a0 20 read_log 192888'16860 (192888'16860) modify 1e6/hit_set_24.1e6_archive_2014-08-11 16:02:37.098356_2014-08-11 16:44:56.372952/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.373364 2014-08-13 17:38:02.927052 7fe9bae887a0 20 read_log 192888'16861 (0'0) promote c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by osd.285.0:17347 2014-08-11 16:44:56.377297 2014-08-13 17:38:02.927064 7fe9bae887a0 20 read_log 192888'16862 (192888'16861) modify c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.1315259.0:347 2014-08-11 16:44:56.366883 2014-08-13 17:38:02.927076 7fe9bae887a0 20 read_log 192888'16867 (0'0) promote a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by osd.285.0:17483 2014-08-11 16:45:15.246403 2014-08-13 17:38:02.927088 7fe9bae887a0 20 read_log 192888'16868 (192888'16867) modify a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.1315259.0:5299 2014-08-11 16:45:15.234639 2014-08-13 17:38:02.927101 7fe9bae887a0 20 read_log 192888'16871 (0'0) promote d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by osd.285.0:17517 2014-08-11 16:45:17.601460 2014-08-13 17:38:02.927112 7fe9bae887a0 20 read_log 192888'16872 (192888'16871) modify d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.1315259.0:5949 2014-08-11 16:45:17.593514 2014-08-13 17:38:02.927124 7fe9bae887a0 20 read_log 192888'16877 (0'0) promote bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by osd.285.0:17549 2014-08-11 16:45:23.967820 2014-08-13 17:38:02.927136 7fe9bae887a0 20 read_log 192888'16878 (192888'16877) modify bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.1315259.0:7713 2014-08-11 16:45:23.960694 2014-08-13 17:38:02.927150 7fe9bae887a0 20 read_log 192888'16879 (0'0) promote c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by osd.285.0:17571 2014-08-11 16:45:27.165831 2014-08-13 17:38:02.927171 7fe9bae887a0 20 read_log 192888'16880 (192888'16879) modify c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.1315259.0:8553 2014-08-11 16:45:27.158099 2014-08-13 17:38:02.927184 7fe9bae887a0 20 read_log 192888'16889 (0'0) promote 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by osd.285.0:17665 2014-08-11 16:45:41.386895 2014-08-13 17:38:02.927196 7fe9bae887a0 20 read_log 192888'16890 (192888'16889) modify 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.1315259.0:12179 2014-08-11 16:45:41.377529 2014-08-13 17:38:02.927208 7fe9bae887a0 20 read_log 192888'16893 (0'0) promote 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by osd.285.0:17693 2014-08-11 16:45:46.025159 2014-08-13 17:38:02.927220 7fe9bae887a0 20 read_log 192888'16894 (192888'16893) modify 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.1315259.0:13298 2014-08-11 16:45:46.015949 2014-08-13 17:38:02.927232 7fe9bae887a0 20 read_log 192888'16895 (0'0) promote 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by osd.285.0:17703 2014-08-11 16:45:47.442687 2014-08-13 17:38:02.927244 7fe9bae887a0 20 read_log 192888'16896 (192888'16895) modify 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.1315259.0:13644 2014-08-11 16:45:47.432657 2014-08-13 17:38:02.927256 7fe9bae887a0 20 read_log 192888'16897 (0'0) promote eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by osd.285.0:17705 2014-08-11 16:45:47.466839 2014-08-13 17:38:02.927275 7fe9bae887a0 20 read_log 192888'16898 (192888'16897) modify eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.1315259.0:13651 2014-08-11 16:45:47.458536 2014-08-13 17:38:02.927291 7fe9bae887a0 20 read_log 192888'16899 (0'0) promote 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by osd.285.0:17725 2014-08-11 16:45:50.599573 2014-08-13 17:38:02.927303 7fe9bae887a0 20 read_log 192888'16900 (192888'16899) modify 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.1315259.0:14474 2014-08-11 16:45:50.592230 2014-08-13 17:38:02.927315 7fe9bae887a0 20 read_log 192888'16901 (0'0) promote 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by osd.285.0:17737 2014-08-11 16:45:53.764623 2014-08-13 17:38:02.927327 7fe9bae887a0 20 read_log 192888'16902 (192888'16901) modify 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.1315259.0:15314 2014-08-11 16:45:53.756762 2014-08-13 17:38:02.927339 7fe9bae887a0 20 read_log 192888'16905 (0'0) promote 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by osd.285.0:17753 2014-08-11 16:45:56.759000 2014-08-13 17:38:02.927351 7fe9bae887a0 20 read_log 192888'16906 (192888'16905) modify 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.1315259.0:16087 2014-08-11 16:45:56.751058 2014-08-13 17:38:02.927363 7fe9bae887a0 20 read_log 192888'16907 (0'0) promote 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by osd.285.0:17755 2014-08-11 16:45:56.936952 2014-08-13 17:38:02.927375 7fe9bae887a0 20 read_log 192888'16908 (192888'16907) modify 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.1315259.0:16134 2014-08-11 16:45:56.927851 2014-08-13 17:38:02.927387 7fe9bae887a0 20 read_log 192888'16911 (0'0) promote a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by osd.285.0:17771 2014-08-11 16:45:59.124546 2014-08-13 17:38:02.927399 7fe9bae887a0 20 read_log 192888'16912 (192888'16911) modify a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.1315259.0:16688 2014-08-11 16:45:59.115774 2014-08-13 17:38:02.927411 7fe9bae887a0 20 read_log 192888'16913 (0'0) promote a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by osd.285.0:17801 2014-08-11 16:46:02.056317 2014-08-13 17:38:02.927423 7fe9bae887a0 20 read_log 192888'16914 (192888'16913) modify a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.1315259.0:17435 2014-08-11 16:46:02.049291 2014-08-13 17:38:02.927435 7fe9bae887a0 20 read_log 192888'16917 (0'0) promote d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by osd.285.0:17835 2014-08-11 16:46:06.682392 2014-08-13 17:38:02.927446 7fe9bae887a0 20 read_log 192888'16918 (192888'16917) modify d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.1315259.0:18605 2014-08-11 16:46:06.673751 2014-08-13 17:38:02.927463 7fe9bae887a0 20 read_log 192888'16919 (0'0) promote 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by osd.285.0:17837 2014-08-11 16:46:07.251479 2014-08-13 17:38:02.927476 7fe9bae887a0 20 read_log 192888'16920 (192888'16919) modify 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.1315259.0:18757 2014-08-11 16:46:07.242916 2014-08-13 17:38:02.927490 7fe9bae887a0 20 read_log 192888'16921 (0'0) promote 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by osd.285.0:17845 2014-08-11 16:46:08.551554 2014-08-13 17:38:02.927502 7fe9bae887a0 20 read_log 192888'16922 (192888'16921) modify 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.1315259.0:19065 2014-08-11 16:46:08.541925 2014-08-13 17:38:02.927514 7fe9bae887a0 20 read_log 192888'16923 (0'0) promote 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by osd.285.0:17881 2014-08-11 16:46:13.784874 2014-08-13 17:38:02.927555 7fe9bae887a0 20 read_log 192888'16924 (192888'16923) modify 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.1315259.0:20452 2014-08-11 16:46:13.774763 2014-08-13 17:38:02.927571 7fe9bae887a0 20 read_log 192888'16933 (0'0) promote f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by osd.285.0:17931 2014-08-11 16:46:19.451710 2014-08-13 17:38:02.927583 7fe9bae887a0 20 read_log 192888'16934 (192888'16933) modify f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.1315259.0:21944 2014-08-11 16:46:19.441152 2014-08-13 17:38:02.927595 7fe9bae887a0 20 read_log 192888'16935 (0'0) promote 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by osd.285.0:17941 2014-08-11 16:46:20.974145 2014-08-13 17:38:02.927617 7fe9bae887a0 20 read_log 192888'16936 (192888'16935) modify 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.1315259.0:22336 2014-08-11 16:46:20.965754 2014-08-13 17:38:02.927630 7fe9bae887a0 20 read_log 192888'16937 (0'0) promote 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by osd.285.0:17943 2014-08-11 16:46:21.131233 2014-08-13 17:38:02.927642 7fe9bae887a0 20 read_log 192888'16938 (192888'16937) modify 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.1315259.0:22373 2014-08-11 16:46:21.121968 2014-08-13 17:38:02.927654 7fe9bae887a0 20 read_log 192888'16941 (0'0) promote 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by osd.285.0:17973 2014-08-11 16:46:24.556338 2014-08-13 17:38:02.927666 7fe9bae887a0 20 read_log 192888'16942 (192888'16941) modify 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.1315259.0:23223 2014-08-11 16:46:24.548392 2014-08-13 17:38:02.927678 7fe9bae887a0 20 read_log 192888'16945 (0'0) promote efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by osd.285.0:18001 2014-08-11 16:46:30.543883 2014-08-13 17:38:02.927690 7fe9bae887a0 20 read_log 192888'16946 (192888'16945) modify efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.1315259.0:24740 2014-08-11 16:46:30.536690 2014-08-13 17:38:02.927702 7fe9bae887a0 20 read_log 192888'16947 (0'0) promote ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by osd.285.0:18011 2014-08-11 16:46:31.433916 2014-08-13 17:38:02.927714 7fe9bae887a0 20 read_log 192888'16948 (192888'16947) modify ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.1315259.0:24978 2014-08-11 16:46:31.423850 2014-08-13 17:38:02.927726 7fe9bae887a0 20 read_log 192888'16951 (0'0) promote ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by osd.285.0:18063 2014-08-11 16:46:40.907289 2014-08-13 17:38:02.927738 7fe9bae887a0 20 read_log 192888'16952 (192888'16951) modify ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.1315259.0:27215 2014-08-11 16:46:40.899033 2014-08-13 17:38:02.927750 7fe9bae887a0 20 read_log 192888'16953 (0'0) promote 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by osd.285.0:18065 2014-08-11 16:46:40.966866 2014-08-13 17:38:02.927762 7fe9bae887a0 20 read_log 192888'16954 (192888'16953) modify 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.1315259.0:27231 2014-08-11 16:46:40.960169 2014-08-13 17:38:02.927774 7fe9bae887a0 20 read_log 192888'16955 (0'0) promote f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by osd.285.0:18085 2014-08-11 16:46:44.401533 2014-08-13 17:38:02.927786 7fe9bae887a0 20 read_log 192888'16956 (192888'16955) modify f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.1315259.0:28112 2014-08-11 16:46:44.390280 2014-08-13 17:38:02.927798 7fe9bae887a0 20 read_log 192888'16957 (0'0) promote 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by osd.285.0:18111 2014-08-11 16:46:47.211898 2014-08-13 17:38:02.927820 7fe9bae887a0 20 read_log 192888'16958 (192888'16957) modify 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.1315259.0:28787 2014-08-11 16:46:47.203907 2014-08-13 17:38:02.927833 7fe9bae887a0 20 read_log 192888'16961 (0'0) promote 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by osd.285.0:18119 2014-08-11 16:46:48.337251 2014-08-13 17:38:02.927845 7fe9bae887a0 20 read_log 192888'16962 (192888'16961) modify 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.1315259.0:29083 2014-08-11 16:46:48.329620 2014-08-13 17:38:02.927857 7fe9bae887a0 20 read_log 192888'16967 (192888'16862) modify c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.1315259.0:31074 2014-08-11 16:46:55.989683 2014-08-13 17:38:02.927869 7fe9bae887a0 20 read_log 192888'16970 (192888'16868) modify a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.1315259.0:36026 2014-08-11 16:46:57.306102 2014-08-13 17:38:02.927881 7fe9bae887a0 20 read_log 192888'16972 (192888'16872) modify d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.1315259.0:36676 2014-08-11 16:46:57.483766 2014-08-13 17:38:02.927893 7fe9bae887a0 20 read_log 192888'16975 (192888'16878) modify bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.1315259.0:38440 2014-08-11 16:46:57.947564 2014-08-13 17:38:02.927905 7fe9bae887a0 20 read_log 192888'16976 (192888'16880) modify c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.1315259.0:39280 2014-08-11 16:46:58.170762 2014-08-13 17:38:02.927917 7fe9bae887a0 20 read_log 192888'16981 (192888'16890) modify 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.1315259.0:42906 2014-08-11 16:46:59.137963 2014-08-13 17:38:02.927928 7fe9bae887a0 20 read_log 192888'16983 (192888'16894) modify 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.1315259.0:44025 2014-08-11 16:46:59.458355 2014-08-13 17:38:02.927940 7fe9bae887a0 20 read_log 192888'16984 (192888'16896) modify 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.1315259.0:44371 2014-08-11 16:46:59.559320 2014-08-13 17:38:02.927952 7fe9bae887a0 20 read_log 192888'16985 (192888'16898) modify eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.1315259.0:44378 2014-08-11 16:46:59.563053 2014-08-13 17:38:02.927964 7fe9bae887a0 20 read_log 192888'16986 (192888'16900) modify 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.1315259.0:45201 2014-08-11 16:46:59.798363 2014-08-13 17:38:02.927995 7fe9bae887a0 20 read_log 192888'16987 (192888'16902) modify 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.1315259.0:46041 2014-08-11 16:47:00.043752 2014-08-13 17:38:02.928008 7fe9bae887a0 20 read_log 192888'16989 (192888'16906) modify 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.1315259.0:46814 2014-08-11 16:47:00.266498 2014-08-13 17:38:02.928020 7fe9bae887a0 20 read_log 192888'16990 (192888'16908) modify 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.1315259.0:46861 2014-08-11 16:47:00.278778 2014-08-13 17:38:02.928032 7fe9bae887a0 20 read_log 192888'16992 (192888'16912) modify a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.1315259.0:47415 2014-08-11 16:47:00.448947 2014-08-13 17:38:02.928044 7fe9bae887a0 20 read_log 192888'16993 (192888'16914) modify a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.1315259.0:48162 2014-08-11 16:47:00.686068 2014-08-13 17:38:02.928056 7fe9bae887a0 20 read_log 192888'16995 (192888'16918) modify d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.1315259.0:49332 2014-08-11 16:47:01.029137 2014-08-13 17:38:02.928068 7fe9bae887a0 20 read_log 192888'16996 (192888'16920) modify 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.1315259.0:49484 2014-08-11 16:47:01.077579 2014-08-13 17:38:02.928087 7fe9bae887a0 20 read_log 192888'16997 (192888'16922) modify 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.1315259.0:49792 2014-08-11 16:47:01.174936 2014-08-13 17:38:02.928100 7fe9bae887a0 20 read_log 192888'16998 (192888'16924) modify 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.1315259.0:51179 2014-08-11 16:47:01.602397 2014-08-13 17:38:02.928112 7fe9bae887a0 20 read_log 192888'17003 (192888'16934) modify f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.1315259.0:52671 2014-08-11 16:47:02.037196 2014-08-13 17:38:02.928124 7fe9bae887a0 20 read_log 192888'17004 (192888'16936) modify 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.1315259.0:53063 2014-08-11 16:47:02.144044 2014-08-13 17:38:02.928136 7fe9bae887a0 20 read_log 192888'17005 (192888'16938) modify 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.1315259.0:53100 2014-08-11 16:47:02.155663 2014-08-13 17:38:02.928148 7fe9bae887a0 20 read_log 192888'17007 (192888'16942) modify 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.1315259.0:53950 2014-08-11 16:47:02.412788 2014-08-13 17:38:02.928166 7fe9bae887a0 20 read_log 192888'17009 (192888'16946) modify efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.1315259.0:55467 2014-08-11 16:47:02.920864 2014-08-13 17:38:02.928178 7fe9bae887a0 20 read_log 192888'17010 (192888'16948) modify ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.1315259.0:55705 2014-08-11 16:47:02.988952 2014-08-13 17:38:02.928190 7fe9bae887a0 20 read_log 192888'17012 (192888'16952) modify ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.1315259.0:57942 2014-08-11 16:47:03.657097 2014-08-13 17:38:02.928202 7fe9bae887a0 20 read_log 192888'17013 (192888'16954) modify 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.1315259.0:57958 2014-08-11 16:47:03.661641 2014-08-13 17:38:02.928214 7fe9bae887a0 20 read_log 192888'17014 (192888'16956) modify f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.1315259.0:58839 2014-08-11 16:47:03.923128 2014-08-13 17:38:02.928227 7fe9bae887a0 20 read_log 192888'17015 (192888'16958) modify 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.1315259.0:59514 2014-08-11 16:47:04.142581 2014-08-13 17:38:02.928239 7fe9bae887a0 20 read_log 192888'17017 (192888'16962) modify 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.1315259.0:59810 2014-08-11 16:47:04.247670 2014-08-13 17:38:02.928251 7fe9bae887a0 20 read_log 192888'17020 (192888'17020) modify 1e6/hit_set_24.1e6_archive_2014-08-11 16:44:56.373369_2014-08-11 17:05:20.065317/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:20.065812 2014-08-13 17:38:02.928263 7fe9bae887a0 20 read_log 192888'17021 (192888'16860) delete 1e6/hit_set_24.1e6_archive_2014-08-11 16:02:37.098356_2014-08-11 16:44:56.372952/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:20.065812 2014-08-13 17:38:02.928276 7fe9bae887a0 20 read_log 192888'17026 (0'0) promote 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by osd.285.0:18666 2014-08-11 17:05:28.309286 2014-08-13 17:38:02.928288 7fe9bae887a0 20 read_log 192888'17027 (192888'17026) modify 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.942868.0:3342 2014-08-11 17:05:28.299783 2014-08-13 17:38:02.928300 7fe9bae887a0 20 read_log 192888'17030 (0'0) promote 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by osd.285.0:18710 2014-08-11 17:05:34.674115 2014-08-13 17:38:02.928314 7fe9bae887a0 20 read_log 192888'17031 (192888'17030) modify 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.942868.0:5012 2014-08-11 17:05:34.666630 2014-08-13 17:38:02.928335 7fe9bae887a0 20 read_log 192888'17032 (0'0) promote c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by osd.285.0:18754 2014-08-11 17:05:40.829286 2014-08-13 17:38:02.928349 7fe9bae887a0 20 read_log 192888'17033 (192888'17032) modify c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.942868.0:6697 2014-08-11 17:05:40.821428 2014-08-13 17:38:02.928363 7fe9bae887a0 20 read_log 192888'17036 (0'0) promote 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by osd.285.0:18770 2014-08-11 17:05:42.845006 2014-08-13 17:38:02.928377 7fe9bae887a0 20 read_log 192888'17037 (192888'17036) modify 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.942868.0:7221 2014-08-11 17:05:42.837205 2014-08-13 17:38:02.928390 7fe9bae887a0 20 read_log 192888'17048 (0'0) promote e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by osd.285.0:18832 2014-08-11 17:05:51.139002 2014-08-13 17:38:02.928402 7fe9bae887a0 20 read_log 192888'17049 (192888'17048) modify e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.942868.0:9429 2014-08-11 17:05:51.131053 2014-08-13 17:38:02.928414 7fe9bae887a0 20 read_log 192888'17050 (0'0) promote ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by osd.285.0:18840 2014-08-11 17:05:52.017587 2014-08-13 17:38:02.928425 7fe9bae887a0 20 read_log 192888'17051 (192888'17050) modify ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.942868.0:9672 2014-08-11 17:05:52.010440 2014-08-13 17:38:02.928438 7fe9bae887a0 20 read_log 192888'17052 (0'0) promote 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by osd.285.0:18876 2014-08-11 17:05:57.055940 2014-08-13 17:38:02.928450 7fe9bae887a0 20 read_log 192888'17053 (192888'17052) modify 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.942868.0:11072 2014-08-11 17:05:57.048164 2014-08-13 17:38:02.928462 7fe9bae887a0 20 read_log 192888'17060 (0'0) promote e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by osd.285.0:18940 2014-08-11 17:06:03.410233 2014-08-13 17:38:02.928474 7fe9bae887a0 20 read_log 192888'17061 (192888'17060) modify e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.942868.0:12785 2014-08-11 17:06:03.401712 2014-08-13 17:38:02.928486 7fe9bae887a0 20 read_log 192888'17062 (0'0) promote e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by osd.285.0:18942 2014-08-11 17:06:03.511343 2014-08-13 17:38:02.928497 7fe9bae887a0 20 read_log 192888'17063 (192888'17062) modify e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.942868.0:12813 2014-08-11 17:06:03.502776 2014-08-13 17:38:02.928509 7fe9bae887a0 20 read_log 192888'17066 (0'0) promote 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by osd.285.0:18970 2014-08-11 17:06:07.419818 2014-08-13 17:38:02.928521 7fe9bae887a0 20 read_log 192888'17067 (192888'17066) modify 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.942868.0:13814 2014-08-11 17:06:07.413217 2014-08-13 17:38:02.928533 7fe9bae887a0 20 read_log 192888'17068 (0'0) promote fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by osd.285.0:18982 2014-08-11 17:06:09.285037 2014-08-13 17:38:02.928545 7fe9bae887a0 20 read_log 192888'17069 (192888'17068) modify fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.942868.0:14260 2014-08-11 17:06:09.277470 2014-08-13 17:38:02.928557 7fe9bae887a0 20 read_log 192888'17072 (0'0) promote 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by osd.285.0:19050 2014-08-11 17:06:17.130461 2014-08-13 17:38:02.928569 7fe9bae887a0 20 read_log 192888'17073 (192888'17072) modify 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.942868.0:16267 2014-08-11 17:06:17.121618 2014-08-13 17:38:02.928588 7fe9bae887a0 20 read_log 192888'17074 (0'0) promote 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by osd.285.0:19052 2014-08-11 17:06:17.717660 2014-08-13 17:38:02.928601 7fe9bae887a0 20 read_log 192888'17075 (192888'17074) modify 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.942868.0:16418 2014-08-11 17:06:17.709359 2014-08-13 17:38:02.928633 7fe9bae887a0 20 read_log 192888'17078 (0'0) promote 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by osd.285.0:19070 2014-08-11 17:06:21.027538 2014-08-13 17:38:02.928646 7fe9bae887a0 20 read_log 192888'17079 (192888'17078) modify 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.942868.0:17255 2014-08-11 17:06:21.020081 2014-08-13 17:38:02.928658 7fe9bae887a0 20 read_log 192888'17080 (0'0) promote 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by osd.285.0:19086 2014-08-11 17:06:24.642826 2014-08-13 17:38:02.928670 7fe9bae887a0 20 read_log 192888'17081 (192888'17080) modify 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.942868.0:18213 2014-08-11 17:06:24.635209 2014-08-13 17:38:02.928683 7fe9bae887a0 20 read_log 192888'17082 (0'0) promote d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by osd.285.0:19100 2014-08-11 17:06:26.584360 2014-08-13 17:38:02.928695 7fe9bae887a0 20 read_log 192888'17083 (192888'17082) modify d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.942868.0:18729 2014-08-11 17:06:26.573925 2014-08-13 17:38:02.928709 7fe9bae887a0 20 read_log 192888'17086 (0'0) promote e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by osd.285.0:19134 2014-08-11 17:06:30.564960 2014-08-13 17:38:02.928721 7fe9bae887a0 20 read_log 192888'17087 (192888'17086) modify e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.942868.0:19788 2014-08-11 17:06:30.557573 2014-08-13 17:38:02.928733 7fe9bae887a0 20 read_log 192888'17090 (0'0) promote 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by osd.285.0:19154 2014-08-11 17:06:32.280847 2014-08-13 17:38:02.928745 7fe9bae887a0 20 read_log 192888'17091 (192888'17090) modify 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.942868.0:20246 2014-08-11 17:06:32.273552 2014-08-13 17:38:02.928757 7fe9bae887a0 20 read_log 192888'17094 (0'0) promote 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by osd.285.0:19184 2014-08-11 17:06:35.997964 2014-08-13 17:38:02.928770 7fe9bae887a0 20 read_log 192888'17095 (192888'17094) modify 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.942868.0:21200 2014-08-11 17:06:35.989775 2014-08-13 17:38:02.928782 7fe9bae887a0 20 read_log 192888'17096 (0'0) promote e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by osd.285.0:19200 2014-08-11 17:06:38.600024 2014-08-13 17:38:02.928794 7fe9bae887a0 20 read_log 192888'17097 (192888'17096) modify e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.942868.0:21898 2014-08-11 17:06:38.592416 2014-08-13 17:38:02.928806 7fe9bae887a0 20 read_log 192888'17100 (0'0) promote a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by osd.285.0:19208 2014-08-11 17:06:40.429124 2014-08-13 17:38:02.928818 7fe9bae887a0 20 read_log 192888'17101 (192888'17100) modify a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.942868.0:22375 2014-08-11 17:06:40.419533 2014-08-13 17:38:02.928830 7fe9bae887a0 20 read_log 192888'17102 (0'0) promote 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by osd.285.0:19210 2014-08-11 17:06:40.854576 2014-08-13 17:38:02.928842 7fe9bae887a0 20 read_log 192888'17103 (192888'17102) modify 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.942868.0:22470 2014-08-11 17:06:40.845626 2014-08-13 17:38:02.928860 7fe9bae887a0 20 read_log 192888'17104 (0'0) promote a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by osd.285.0:19216 2014-08-11 17:06:41.147793 2014-08-13 17:38:02.928873 7fe9bae887a0 20 read_log 192888'17105 (192888'17104) modify a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.942868.0:22537 2014-08-11 17:06:41.140438 2014-08-13 17:38:02.928885 7fe9bae887a0 20 read_log 192888'17108 (0'0) promote 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by osd.285.0:19232 2014-08-11 17:06:42.206695 2014-08-13 17:38:02.928897 7fe9bae887a0 20 read_log 192888'17109 (192888'17108) modify 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.942868.0:22823 2014-08-11 17:06:42.196158 2014-08-13 17:38:02.928909 7fe9bae887a0 20 read_log 192888'17112 (0'0) promote 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by osd.285.0:19260 2014-08-11 17:06:46.715883 2014-08-13 17:38:02.928921 7fe9bae887a0 20 read_log 192888'17113 (192888'17112) modify 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.942868.0:24001 2014-08-11 17:06:46.708763 2014-08-13 17:38:02.928933 7fe9bae887a0 20 read_log 192888'17114 (0'0) promote 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by osd.285.0:19262 2014-08-11 17:06:47.203568 2014-08-13 17:38:02.928945 7fe9bae887a0 20 read_log 192888'17115 (192888'17114) modify 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.942868.0:24132 2014-08-11 17:06:47.194942 2014-08-13 17:38:02.928957 7fe9bae887a0 20 read_log 192888'17120 (0'0) promote 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by osd.285.0:19296 2014-08-11 17:06:51.320297 2014-08-13 17:38:02.928970 7fe9bae887a0 20 read_log 192888'17121 (192888'17120) modify 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.942868.0:25244 2014-08-11 17:06:51.313116 2014-08-13 17:38:02.928982 7fe9bae887a0 20 read_log 192888'17122 (0'0) promote e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by osd.285.0:19300 2014-08-11 17:06:51.407257 2014-08-13 17:38:02.928994 7fe9bae887a0 20 read_log 192888'17123 (192888'17122) modify e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.942868.0:25267 2014-08-11 17:06:51.398396 2014-08-13 17:38:02.929006 7fe9bae887a0 20 read_log 192888'17126 (0'0) promote fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by osd.285.0:19324 2014-08-11 17:06:54.545625 2014-08-13 17:38:02.929018 7fe9bae887a0 20 read_log 192888'17127 (192888'17126) modify fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.942868.0:26037 2014-08-11 17:06:54.536278 2014-08-13 17:38:02.929030 7fe9bae887a0 20 read_log 192888'17136 (0'0) promote 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by osd.285.0:19388 2014-08-11 17:07:01.169182 2014-08-13 17:38:02.929041 7fe9bae887a0 20 read_log 192888'17137 (192888'17136) modify 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.942868.0:27775 2014-08-11 17:07:01.159166 2014-08-13 17:38:02.929055 7fe9bae887a0 20 read_log 192888'17142 (0'0) promote 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by osd.285.0:19440 2014-08-11 17:07:08.092026 2014-08-13 17:38:02.929067 7fe9bae887a0 20 read_log 192888'17143 (192888'17142) modify 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.942868.0:29568 2014-08-11 17:07:08.085757 2014-08-13 17:38:02.929080 7fe9bae887a0 20 read_log 192888'17144 (0'0) promote 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by osd.285.0:19486 2014-08-11 17:07:13.182233 2014-08-13 17:38:02.929092 7fe9bae887a0 20 read_log 192888'17145 (192888'17144) modify 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.942868.0:30839 2014-08-11 17:07:13.173476 2014-08-13 17:38:02.929114 7fe9bae887a0 20 read_log 192888'17152 (192888'17027) modify 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.942868.0:34608 2014-08-11 17:07:16.841879 2014-08-13 17:38:02.929126 7fe9bae887a0 20 read_log 192888'17154 (192888'17031) modify 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.942868.0:36278 2014-08-11 17:07:17.286774 2014-08-13 17:38:02.929138 7fe9bae887a0 20 read_log 192888'17155 (192888'17033) modify c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.942868.0:37963 2014-08-11 17:07:17.755263 2014-08-13 17:38:02.929150 7fe9bae887a0 20 read_log 192888'17157 (192888'17037) modify 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.942868.0:38487 2014-08-11 17:07:17.900344 2014-08-13 17:38:02.929170 7fe9bae887a0 20 read_log 192888'17163 (192888'17049) modify e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.942868.0:40695 2014-08-11 17:07:18.522764 2014-08-13 17:38:02.929182 7fe9bae887a0 20 read_log 192888'17164 (192888'17051) modify ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.942868.0:40938 2014-08-11 17:07:18.587213 2014-08-13 17:38:02.929194 7fe9bae887a0 20 read_log 192888'17165 (192888'17053) modify 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.942868.0:42338 2014-08-11 17:07:18.972198 2014-08-13 17:38:02.929222 7fe9bae887a0 20 read_log 192888'17169 (192888'17061) modify e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.942868.0:44051 2014-08-11 17:07:19.452412 2014-08-13 17:38:02.929235 7fe9bae887a0 20 read_log 192888'17170 (192888'17063) modify e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.942868.0:44079 2014-08-11 17:07:19.461156 2014-08-13 17:38:02.929247 7fe9bae887a0 20 read_log 192888'17172 (192888'17067) modify 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.942868.0:45080 2014-08-11 17:07:19.741398 2014-08-13 17:38:02.929259 7fe9bae887a0 20 read_log 192888'17173 (192888'17069) modify fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.942868.0:45526 2014-08-11 17:07:19.862233 2014-08-13 17:38:02.929271 7fe9bae887a0 20 read_log 192888'17175 (192888'17073) modify 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.942868.0:47533 2014-08-11 17:07:20.436686 2014-08-13 17:38:02.929285 7fe9bae887a0 20 read_log 192888'17176 (192888'17075) modify 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.942868.0:47684 2014-08-11 17:07:20.486431 2014-08-13 17:38:02.929297 7fe9bae887a0 20 read_log 192888'17178 (192888'17079) modify 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.942868.0:48521 2014-08-11 17:07:20.743766 2014-08-13 17:38:02.929309 7fe9bae887a0 20 read_log 192888'17179 (192888'17081) modify 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.942868.0:49479 2014-08-11 17:07:21.025674 2014-08-13 17:38:02.929321 7fe9bae887a0 20 read_log 192888'17180 (192888'17083) modify d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.942868.0:49995 2014-08-11 17:07:21.184424 2014-08-13 17:38:02.929334 7fe9bae887a0 20 read_log 192888'17182 (192888'17087) modify e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.942868.0:51054 2014-08-11 17:07:21.517090 2014-08-13 17:38:02.929345 7fe9bae887a0 20 read_log 192888'17184 (192888'17091) modify 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.942868.0:51512 2014-08-11 17:07:21.669555 2014-08-13 17:38:02.929357 7fe9bae887a0 20 read_log 192888'17186 (192888'17095) modify 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.942868.0:52466 2014-08-11 17:07:21.950076 2014-08-13 17:38:02.929369 7fe9bae887a0 20 read_log 192888'17187 (192888'17097) modify e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.942868.0:53164 2014-08-11 17:07:22.170976 2014-08-13 17:38:02.929411 7fe9bae887a0 20 read_log 192888'17189 (192888'17101) modify a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.942868.0:53641 2014-08-11 17:07:22.329396 2014-08-13 17:38:02.929424 7fe9bae887a0 20 read_log 192888'17190 (192888'17103) modify 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.942868.0:53736 2014-08-11 17:07:22.359201 2014-08-13 17:38:02.929437 7fe9bae887a0 20 read_log 192888'17191 (192888'17105) modify a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.942868.0:53803 2014-08-11 17:07:22.378460 2014-08-13 17:38:02.929449 7fe9bae887a0 20 read_log 192888'17193 (192888'17109) modify 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.942868.0:54089 2014-08-11 17:07:22.460614 2014-08-13 17:38:02.929461 7fe9bae887a0 20 read_log 192888'17195 (192888'17113) modify 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.942868.0:55267 2014-08-11 17:07:22.794958 2014-08-13 17:38:02.929473 7fe9bae887a0 20 read_log 192888'17196 (192888'17115) modify 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.942868.0:55398 2014-08-11 17:07:22.834641 2014-08-13 17:38:02.929485 7fe9bae887a0 20 read_log 192888'17199 (192888'17121) modify 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.942868.0:56510 2014-08-11 17:07:23.174812 2014-08-13 17:38:02.929496 7fe9bae887a0 20 read_log 192888'17200 (192888'17123) modify e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.942868.0:56533 2014-08-11 17:07:23.182095 2014-08-13 17:38:02.929508 7fe9bae887a0 20 read_log 192888'17202 (192888'17127) modify fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.942868.0:57303 2014-08-11 17:07:23.417442 2014-08-13 17:38:02.929520 7fe9bae887a0 20 read_log 192888'17207 (192888'17137) modify 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.942868.0:59041 2014-08-11 17:07:23.926655 2014-08-13 17:38:02.929533 7fe9bae887a0 20 read_log 192888'17210 (192888'17143) modify 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.942868.0:60834 2014-08-11 17:07:24.459271 2014-08-13 17:38:02.929545 7fe9bae887a0 20 read_log 192888'17211 (192888'17145) modify 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.942868.0:62105 2014-08-11 17:07:24.850186 2014-08-13 17:38:02.929557 7fe9bae887a0 20 read_log 193305'17214 (193305'17214) modify 1e6/hit_set_24.1e6_archive_2014-08-11 18:37:04.946502_2014-08-11 18:42:36.225526/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.225780 2014-08-13 17:38:02.929569 7fe9bae887a0 20 read_log 193305'17215 (192888'17020) delete 1e6/hit_set_24.1e6_archive_2014-08-11 16:44:56.373369_2014-08-11 17:05:20.065317/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.225780 2014-08-13 17:38:02.929581 7fe9bae887a0 20 read_log 193305'17216 (0'0) promote 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by osd.285.0:18 2014-08-11 18:42:36.245508 2014-08-13 17:38:02.929593 7fe9bae887a0 20 read_log 193305'17217 (193305'17216) modify 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.951337.0:793 2014-08-11 18:42:36.219952 2014-08-13 17:38:02.929606 7fe9bae887a0 20 read_log 193305'17218 (0'0) promote 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by osd.285.0:20 2014-08-11 18:42:36.687986 2014-08-13 17:38:02.929618 7fe9bae887a0 20 read_log 193305'17219 (193305'17218) modify 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.951337.0:921 2014-08-11 18:42:36.680667 2014-08-13 17:38:02.929630 7fe9bae887a0 20 read_log 193305'17220 (0'0) promote c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by osd.285.0:38 2014-08-11 18:42:38.659106 2014-08-13 17:38:02.929649 7fe9bae887a0 20 read_log 193305'17221 (193305'17220) modify c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.951337.0:1443 2014-08-11 18:42:38.651305 2014-08-13 17:38:02.929662 7fe9bae887a0 20 read_log 193305'17222 (0'0) promote 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by osd.285.0:55 2014-08-11 18:42:43.267553 2014-08-13 17:38:02.929674 7fe9bae887a0 20 read_log 193305'17223 (193305'17222) modify 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.951337.0:2691 2014-08-11 18:42:43.260413 2014-08-13 17:38:02.929686 7fe9bae887a0 20 read_log 193305'17224 (0'0) promote b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by osd.285.0:97 2014-08-11 18:42:51.388150 2014-08-13 17:38:02.929698 7fe9bae887a0 20 read_log 193305'17225 (193305'17224) modify b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.951337.0:4906 2014-08-11 18:42:51.380904 2014-08-13 17:38:02.929710 7fe9bae887a0 20 read_log 193305'17226 (0'0) promote 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by osd.285.0:195 2014-08-11 18:43:07.428532 2014-08-13 17:38:02.929722 7fe9bae887a0 20 read_log 193305'17227 (193305'17226) modify 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.951337.0:9220 2014-08-11 18:43:07.421347 2014-08-13 17:38:02.929734 7fe9bae887a0 20 read_log 193305'17228 (0'0) promote e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by osd.285.0:235 2014-08-11 18:43:14.746897 2014-08-13 17:38:02.929746 7fe9bae887a0 20 read_log 193305'17229 (193305'17228) modify e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.951337.0:11177 2014-08-11 18:43:14.738344 2014-08-13 17:38:02.929758 7fe9bae887a0 20 read_log 193305'17230 (0'0) promote 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by osd.285.0:241 2014-08-11 18:43:16.682911 2014-08-13 17:38:02.929770 7fe9bae887a0 20 read_log 193305'17231 (193305'17230) modify 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.951337.0:11672 2014-08-11 18:43:16.675212 2014-08-13 17:38:02.929782 7fe9bae887a0 20 read_log 193305'17232 (0'0) promote bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by osd.285.0:297 2014-08-11 18:43:27.721113 2014-08-13 17:38:02.929794 7fe9bae887a0 20 read_log 193305'17233 (193305'17232) modify bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.951337.0:14636 2014-08-11 18:43:27.714529 2014-08-13 17:38:02.929806 7fe9bae887a0 20 read_log 193305'17234 (0'0) promote dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by osd.285.0:349 2014-08-11 18:43:35.622704 2014-08-13 17:38:02.929818 7fe9bae887a0 20 read_log 193305'17235 (193305'17234) modify dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.951337.0:16911 2014-08-11 18:43:35.614950 2014-08-13 17:38:02.929830 7fe9bae887a0 20 read_log 193305'17236 (0'0) promote 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by osd.285.0:369 2014-08-11 18:43:38.051525 2014-08-13 17:38:02.929841 7fe9bae887a0 20 read_log 193305'17237 (193305'17236) modify 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.951337.0:17627 2014-08-11 18:43:38.042598 2014-08-13 17:38:02.929853 7fe9bae887a0 20 read_log 193305'17238 (0'0) promote d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by osd.285.0:405 2014-08-11 18:43:42.926337 2014-08-13 17:38:02.929865 7fe9bae887a0 20 read_log 193305'17239 (193305'17238) modify d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.951337.0:18995 2014-08-11 18:43:42.918365 2014-08-13 17:38:02.929877 7fe9bae887a0 20 read_log 193305'17240 (0'0) promote 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by osd.285.0:455 2014-08-11 18:43:49.845211 2014-08-13 17:38:02.929898 7fe9bae887a0 20 read_log 193305'17241 (193305'17240) modify 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.951337.0:20901 2014-08-11 18:43:49.836904 2014-08-13 17:38:02.929911 7fe9bae887a0 20 read_log 193305'17242 (0'0) promote 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by osd.285.0:467 2014-08-11 18:43:51.375171 2014-08-13 17:38:02.929925 7fe9bae887a0 20 read_log 193305'17243 (193305'17242) modify 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.951337.0:21327 2014-08-11 18:43:51.367334 2014-08-13 17:38:02.929937 7fe9bae887a0 20 read_log 193305'17244 (0'0) promote e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by osd.285.0:527 2014-08-11 18:44:00.932045 2014-08-13 17:38:02.929949 7fe9bae887a0 20 read_log 193305'17245 (193305'17244) modify e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.951337.0:23975 2014-08-11 18:44:00.924981 2014-08-13 17:38:02.929961 7fe9bae887a0 20 read_log 193305'17246 (0'0) promote 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by osd.285.0:587 2014-08-11 18:44:12.355565 2014-08-13 17:38:02.929973 7fe9bae887a0 20 read_log 193305'17247 (193305'17246) modify 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.951337.0:27078 2014-08-11 18:44:12.346972 2014-08-13 17:38:02.929985 7fe9bae887a0 20 read_log 193305'17248 (0'0) promote b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by osd.285.0:648 2014-08-11 18:44:22.174244 2014-08-13 17:38:02.929997 7fe9bae887a0 20 read_log 193305'17249 (193305'17248) modify b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.951337.0:29774 2014-08-11 18:44:22.165460 2014-08-13 17:38:02.930009 7fe9bae887a0 20 read_log 193305'17250 (0'0) promote b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by osd.285.0:682 2014-08-11 18:44:28.228903 2014-08-13 17:38:02.930021 7fe9bae887a0 20 read_log 193305'17251 (193305'17250) modify b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.951337.0:31398 2014-08-11 18:44:28.216762 2014-08-13 17:38:02.930033 7fe9bae887a0 20 read_log 193305'17252 (0'0) promote ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by osd.285.0:698 2014-08-11 18:44:29.835774 2014-08-13 17:38:02.930045 7fe9bae887a0 20 read_log 193305'17253 (193305'17252) modify ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.951337.0:31871 2014-08-11 18:44:29.827640 2014-08-13 17:38:02.930057 7fe9bae887a0 20 read_log 193305'17254 (193305'17217) modify 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.951337.0:33504 2014-08-11 18:44:34.459297 2014-08-13 17:38:02.930070 7fe9bae887a0 20 read_log 193305'17255 (193305'17219) modify 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.951337.0:33632 2014-08-11 18:44:34.544580 2014-08-13 17:38:02.930082 7fe9bae887a0 20 read_log 193305'17256 (193305'17221) modify c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.951337.0:34154 2014-08-11 18:44:34.823522 2014-08-13 17:38:02.930094 7fe9bae887a0 20 read_log 193305'17257 (193305'17223) modify 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.951337.0:35402 2014-08-11 18:44:35.551024 2014-08-13 17:38:02.930106 7fe9bae887a0 20 read_log 193305'17258 (193305'17225) modify b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.951337.0:37617 2014-08-11 18:44:36.993950 2014-08-13 17:38:02.930118 7fe9bae887a0 20 read_log 193305'17259 (193305'17227) modify 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.951337.0:41931 2014-08-11 18:44:38.930732 2014-08-13 17:38:02.930131 7fe9bae887a0 20 read_log 193305'17260 (193305'17229) modify e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.951337.0:43888 2014-08-11 18:44:39.727959 2014-08-13 17:38:02.930151 7fe9bae887a0 20 read_log 193305'17261 (193305'17231) modify 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.951337.0:44383 2014-08-11 18:44:39.867808 2014-08-13 17:38:02.930178 7fe9bae887a0 20 read_log 193305'17262 (193305'17233) modify bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.951337.0:47347 2014-08-11 18:44:40.741735 2014-08-13 17:38:02.930191 7fe9bae887a0 20 read_log 193305'17263 (193305'17235) modify dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.951337.0:49622 2014-08-11 18:44:41.422275 2014-08-13 17:38:02.930203 7fe9bae887a0 20 read_log 193305'17264 (193305'17237) modify 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.951337.0:50338 2014-08-11 18:44:41.638354 2014-08-13 17:38:02.930214 7fe9bae887a0 20 read_log 193305'17265 (193305'17239) modify d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.951337.0:51706 2014-08-11 18:44:42.027698 2014-08-13 17:38:02.930227 7fe9bae887a0 20 read_log 193305'17266 (193305'17241) modify 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.951337.0:53612 2014-08-11 18:44:42.582040 2014-08-13 17:38:02.930238 7fe9bae887a0 20 read_log 193305'17267 (193305'17243) modify 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.951337.0:54038 2014-08-11 18:44:42.711547 2014-08-13 17:38:02.930250 7fe9bae887a0 20 read_log 193305'17268 (193305'17245) modify e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.951337.0:56686 2014-08-11 18:44:43.497517 2014-08-13 17:38:02.930262 7fe9bae887a0 20 read_log 193305'17269 (193305'17247) modify 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.951337.0:59789 2014-08-11 18:44:44.427621 2014-08-13 17:38:02.930274 7fe9bae887a0 20 read_log 193305'17270 (193305'17249) modify b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.951337.0:62485 2014-08-11 18:44:45.241112 2014-08-13 17:38:02.930286 7fe9bae887a0 20 read_log 193305'17271 (193305'17251) modify b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.951337.0:64109 2014-08-11 18:44:45.706482 2014-08-13 17:38:02.930298 7fe9bae887a0 20 read_log 193305'17272 (193305'17253) modify ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.951337.0:64582 2014-08-11 18:44:45.858719 2014-08-13 17:38:02.930310 7fe9bae887a0 20 read_log 194215'17273 (194215'17273) modify 1e6/hit_set_24.1e6_archive_2014-08-12 15:32:25.782585_2014-08-12 15:43:44.587553/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:44.587824 2014-08-13 17:38:02.930322 7fe9bae887a0 20 read_log 194215'17274 (193305'17214) delete 1e6/hit_set_24.1e6_archive_2014-08-11 18:37:04.946502_2014-08-11 18:42:36.225526/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:44.587824 2014-08-13 17:38:02.930334 7fe9bae887a0 20 read_log 194215'17275 (192834'16843) clean 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by osd.285.0:2823 2014-08-12 15:43:44.592339 2014-08-13 17:38:02.930346 7fe9bae887a0 20 read_log 194215'17276 (194215'17275) delete 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.2174595.0:100553 0.000000 2014-08-13 17:38:02.930357 7fe9bae887a0 20 read_log 194215'17277 (192888'16997) clean 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by osd.285.0:2825 2014-08-12 15:43:44.624433 2014-08-13 17:38:02.930369 7fe9bae887a0 20 read_log 194215'17278 (194215'17277) delete 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.2174595.0:100555 0.000000 2014-08-13 17:38:02.930379 7fe9bae887a0 20 read_log 194215'17279 (192834'16276) clean af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by osd.285.0:2827 2014-08-12 15:43:44.633023 2014-08-13 17:38:02.930399 7fe9bae887a0 20 read_log 194215'17280 (194215'17279) delete af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.2174595.0:100557 0.000000 2014-08-13 17:38:02.930410 7fe9bae887a0 20 read_log 194215'17281 (193305'17264) clean 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by osd.285.0:2829 2014-08-12 15:43:44.659064 2014-08-13 17:38:02.930441 7fe9bae887a0 20 read_log 194215'17282 (194215'17281) delete 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.2174595.0:100559 0.000000 2014-08-13 17:38:02.930455 7fe9bae887a0 20 read_log 194215'17283 (192834'16715) clean 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by osd.285.0:2831 2014-08-12 15:43:44.667039 2014-08-13 17:38:02.930468 7fe9bae887a0 20 read_log 194215'17284 (194215'17283) delete 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.2174595.0:100561 0.000000 2014-08-13 17:38:02.930478 7fe9bae887a0 20 read_log 194215'17285 (192888'17013) clean 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by osd.285.0:2833 2014-08-12 15:43:44.695886 2014-08-13 17:38:02.930491 7fe9bae887a0 20 read_log 194215'17286 (194215'17285) delete 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.2174595.0:100563 0.000000 2014-08-13 17:38:02.930501 7fe9bae887a0 20 read_log 194215'17287 (192888'17007) clean 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by osd.285.0:2835 2014-08-12 15:43:44.703714 2014-08-13 17:38:02.930513 7fe9bae887a0 20 read_log 194215'17288 (194215'17287) delete 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.2174595.0:100565 0.000000 2014-08-13 17:38:02.930524 7fe9bae887a0 20 read_log 194215'17289 (192834'16304) clean b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by osd.285.0:2837 2014-08-12 15:43:44.711223 2014-08-13 17:38:02.930536 7fe9bae887a0 20 read_log 194215'17290 (194215'17289) delete b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.2174595.0:100567 0.000000 2014-08-13 17:38:02.930547 7fe9bae887a0 20 read_log 194215'17291 (192834'16302) clean 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by osd.285.0:2839 2014-08-12 15:43:44.719591 2014-08-13 17:38:02.930559 7fe9bae887a0 20 read_log 194215'17292 (194215'17291) delete 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.2174595.0:100569 0.000000 2014-08-13 17:38:02.930569 7fe9bae887a0 20 read_log 194215'17293 (192834'16738) clean 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by osd.285.0:2841 2014-08-12 15:43:44.727403 2014-08-13 17:38:02.930583 7fe9bae887a0 20 read_log 194215'17294 (194215'17293) delete 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.2174595.0:100571 0.000000 2014-08-13 17:38:02.930593 7fe9bae887a0 20 read_log 194215'17295 (192888'16975) clean bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by osd.285.0:2843 2014-08-12 15:43:44.735691 2014-08-13 17:38:02.930605 7fe9bae887a0 20 read_log 194215'17296 (194215'17295) delete bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.2174595.0:100573 0.000000 2014-08-13 17:38:02.930616 7fe9bae887a0 20 read_log 194215'17297 (192888'17017) clean 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by osd.285.0:2845 2014-08-12 15:43:44.743902 2014-08-13 17:38:02.930628 7fe9bae887a0 20 read_log 194215'17298 (194215'17297) delete 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.2174595.0:100575 0.000000 2014-08-13 17:38:02.930638 7fe9bae887a0 20 read_log 194215'17299 (192834'16721) clean 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by osd.285.0:2847 2014-08-12 15:43:44.751565 2014-08-13 17:38:02.930660 7fe9bae887a0 20 read_log 194215'17300 (194215'17299) delete 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.2174595.0:100577 0.000000 2014-08-13 17:38:02.930671 7fe9bae887a0 20 read_log 194215'17301 (192888'17155) clean c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by osd.285.0:2849 2014-08-12 15:43:44.760010 2014-08-13 17:38:02.930684 7fe9bae887a0 20 read_log 194215'17302 (194215'17301) delete c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.2174595.0:100579 0.000000 2014-08-13 17:38:02.930695 7fe9bae887a0 20 read_log 194215'17303 (192888'16996) clean 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by osd.285.0:2851 2014-08-12 15:43:44.787605 2014-08-13 17:38:02.930707 7fe9bae887a0 20 read_log 194215'17304 (194215'17303) delete 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.2174595.0:100581 0.000000 2014-08-13 17:38:02.930718 7fe9bae887a0 20 read_log 194215'17305 (192888'16990) clean 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by osd.285.0:2853 2014-08-12 15:43:44.801514 2014-08-13 17:38:02.930730 7fe9bae887a0 20 read_log 194215'17306 (194215'17305) delete 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.2174595.0:100583 0.000000 2014-08-13 17:38:02.930741 7fe9bae887a0 20 read_log 194215'17307 (192834'16213) clean 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by osd.285.0:2855 2014-08-12 15:43:44.809686 2014-08-13 17:38:02.930753 7fe9bae887a0 20 read_log 194215'17308 (194215'17307) delete 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.2174595.0:100585 0.000000 2014-08-13 17:38:02.930764 7fe9bae887a0 20 read_log 194215'17309 (192888'17187) clean e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by osd.285.0:2857 2014-08-12 15:43:44.818120 2014-08-13 17:38:02.930776 7fe9bae887a0 20 read_log 194215'17310 (194215'17309) delete e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.2174595.0:100587 0.000000 2014-08-13 17:38:02.930787 7fe9bae887a0 20 read_log 194215'17311 (193305'17268) clean e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by osd.285.0:2859 2014-08-12 15:43:44.826307 2014-08-13 17:38:02.930798 7fe9bae887a0 20 read_log 194215'17312 (194215'17311) delete e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.2174595.0:100589 0.000000 2014-08-13 17:38:02.930809 7fe9bae887a0 20 read_log 194215'17313 (193305'17255) clean 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by osd.285.0:2861 2014-08-12 15:43:44.834048 2014-08-13 17:38:02.930821 7fe9bae887a0 20 read_log 194215'17314 (194215'17313) delete 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.2174595.0:100591 0.000000 2014-08-13 17:38:02.930832 7fe9bae887a0 20 read_log 194215'17315 (192888'16984) clean 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by osd.285.0:2863 2014-08-12 15:43:44.841604 2014-08-13 17:38:02.930844 7fe9bae887a0 20 read_log 194215'17316 (194215'17315) delete 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.2174595.0:100593 0.000000 2014-08-13 17:38:02.930854 7fe9bae887a0 20 read_log 194215'17317 (192834'16287) clean f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by osd.285.0:2865 2014-08-12 15:43:44.849506 2014-08-13 17:38:02.930866 7fe9bae887a0 20 read_log 194215'17318 (194215'17317) delete f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.2174595.0:100595 0.000000 2014-08-13 17:38:02.930877 7fe9bae887a0 20 read_log 194215'17319 (192888'17211) clean 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by osd.285.0:2867 2014-08-12 15:43:44.876595 2014-08-13 17:38:02.930896 7fe9bae887a0 20 read_log 194215'17320 (194215'17319) delete 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.2174595.0:100597 0.000000 2014-08-13 17:38:02.930909 7fe9bae887a0 20 read_log 194215'17321 (192888'17202) clean fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by osd.285.0:2869 2014-08-12 15:43:44.884458 2014-08-13 17:38:02.930940 7fe9bae887a0 20 read_log 194215'17322 (194215'17321) delete fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.2174595.0:100599 0.000000 2014-08-13 17:38:02.930953 7fe9bae887a0 20 read_log 194215'17323 (192888'17152) clean 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by osd.285.0:2871 2014-08-12 15:43:44.893249 2014-08-13 17:38:02.930965 7fe9bae887a0 20 read_log 194215'17324 (194215'17323) delete 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.2174595.0:100601 0.000000 2014-08-13 17:38:02.930976 7fe9bae887a0 20 read_log 194215'17325 (192888'17015) clean 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by osd.285.0:2873 2014-08-12 15:43:44.901339 2014-08-13 17:38:02.930988 7fe9bae887a0 20 read_log 194215'17326 (194215'17325) delete 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.2174595.0:100603 0.000000 2014-08-13 17:38:02.930999 7fe9bae887a0 20 read_log 194215'17327 (192888'16972) clean d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by osd.285.0:2875 2014-08-12 15:43:44.909553 2014-08-13 17:38:02.931011 7fe9bae887a0 20 read_log 194215'17328 (194215'17327) delete d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.2174595.0:100605 0.000000 2014-08-13 17:38:02.931022 7fe9bae887a0 20 read_log 194215'17329 (192834'16229) clean 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by osd.285.0:2877 2014-08-12 15:43:44.918659 2014-08-13 17:38:02.931034 7fe9bae887a0 20 read_log 194215'17330 (194215'17329) delete 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.2174595.0:100607 0.000000 2014-08-13 17:38:02.931045 7fe9bae887a0 20 read_log 194215'17331 (192834'16220) clean f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by osd.285.0:2879 2014-08-12 15:43:44.926908 2014-08-13 17:38:02.931057 7fe9bae887a0 20 read_log 194215'17332 (194215'17331) delete f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.2174595.0:100609 0.000000 2014-08-13 17:38:02.931067 7fe9bae887a0 20 read_log 194215'17333 (192834'16810) clean 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by osd.285.0:2881 2014-08-12 15:43:44.934470 2014-08-13 17:38:02.931079 7fe9bae887a0 20 read_log 194215'17334 (194215'17333) delete 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.2174595.0:100611 0.000000 2014-08-13 17:38:02.931090 7fe9bae887a0 20 read_log 194215'17335 (192888'17180) clean d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by osd.285.0:2883 2014-08-12 15:43:44.943467 2014-08-13 17:38:02.931102 7fe9bae887a0 20 read_log 194215'17336 (194215'17335) delete d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.2174595.0:100613 0.000000 2014-08-13 17:38:02.931113 7fe9bae887a0 20 read_log 194215'17337 (193305'17256) clean c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by osd.285.0:2885 2014-08-12 15:43:44.951227 2014-08-13 17:38:02.931125 7fe9bae887a0 20 read_log 194215'17338 (194215'17337) delete c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.2174595.0:100615 0.000000 2014-08-13 17:38:02.931136 7fe9bae887a0 20 read_log 194215'17339 (192888'17210) clean 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by osd.285.0:2887 2014-08-12 15:43:44.960497 2014-08-13 17:38:02.931176 7fe9bae887a0 20 read_log 194215'17340 (194215'17339) delete 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.2174595.0:100617 0.000000 2014-08-13 17:38:02.931188 7fe9bae887a0 20 read_log 194215'17341 (193305'17267) clean 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by osd.285.0:2889 2014-08-12 15:43:44.970372 2014-08-13 17:38:02.931201 7fe9bae887a0 20 read_log 194215'17342 (194215'17341) delete 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.2174595.0:100619 0.000000 2014-08-13 17:38:02.931212 7fe9bae887a0 20 read_log 194215'17343 (192888'17003) clean f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by osd.285.0:2891 2014-08-12 15:43:44.978818 2014-08-13 17:38:02.931224 7fe9bae887a0 20 read_log 194215'17344 (194215'17343) delete f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.2174595.0:100621 0.000000 2014-08-13 17:38:02.931234 7fe9bae887a0 20 read_log 194215'17345 (193305'17263) clean dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by osd.285.0:2893 2014-08-12 15:43:44.986267 2014-08-13 17:38:02.931246 7fe9bae887a0 20 read_log 194215'17346 (194215'17345) delete dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.2174595.0:100623 0.000000 2014-08-13 17:38:02.931257 7fe9bae887a0 20 read_log 194215'17347 (192888'17190) clean 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by osd.285.0:2895 2014-08-12 15:43:44.994718 2014-08-13 17:38:02.931269 7fe9bae887a0 20 read_log 194215'17348 (194215'17347) delete 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.2174595.0:100625 0.000000 2014-08-13 17:38:02.931279 7fe9bae887a0 20 read_log 194215'17349 (192888'17012) clean ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by osd.285.0:2897 2014-08-12 15:43:45.002941 2014-08-13 17:38:02.931294 7fe9bae887a0 20 read_log 194215'17350 (194215'17349) delete ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.2174595.0:100627 0.000000 2014-08-13 17:38:02.931305 7fe9bae887a0 20 read_log 194215'17351 (193305'17258) clean b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by osd.285.0:2899 2014-08-12 15:43:45.010691 2014-08-13 17:38:02.931318 7fe9bae887a0 20 read_log 194215'17352 (194215'17351) delete b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.2174595.0:100629 0.000000 2014-08-13 17:38:02.931329 7fe9bae887a0 20 read_log 194215'17353 (192888'16985) clean eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by osd.285.0:2901 2014-08-12 15:43:45.037182 2014-08-13 17:38:02.931341 7fe9bae887a0 20 read_log 194215'17354 (194215'17353) delete eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.2174595.0:100631 0.000000 2014-08-13 17:38:02.931351 7fe9bae887a0 20 read_log 194215'17355 (192888'17157) clean 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by osd.285.0:2903 2014-08-12 15:43:45.047036 2014-08-13 17:38:02.931363 7fe9bae887a0 20 read_log 194215'17356 (194215'17355) delete 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.2174595.0:100633 0.000000 2014-08-13 17:38:02.931374 7fe9bae887a0 20 read_log 194215'17357 (192888'17199) clean 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by osd.285.0:2905 2014-08-12 15:43:45.055226 2014-08-13 17:38:02.931386 7fe9bae887a0 20 read_log 194215'17358 (194215'17357) delete 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.2174595.0:100635 0.000000 2014-08-13 17:38:02.931397 7fe9bae887a0 20 read_log 194215'17359 (192888'17196) clean 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by osd.285.0:2907 2014-08-12 15:43:45.064125 2014-08-13 17:38:02.931418 7fe9bae887a0 20 read_log 194215'17360 (194215'17359) delete 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.2174595.0:100637 0.000000 2014-08-13 17:38:02.931429 7fe9bae887a0 20 read_log 194215'17361 (192888'16983) clean 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by osd.285.0:2909 2014-08-12 15:43:45.072834 2014-08-13 17:38:02.931441 7fe9bae887a0 20 read_log 194215'17362 (194215'17361) delete 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.2174595.0:100639 0.000000 2014-08-13 17:38:02.931452 7fe9bae887a0 20 read_log 194215'17363 (192834'16205) clean 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by osd.285.0:2911 2014-08-12 15:43:45.094029 2014-08-13 17:38:02.931464 7fe9bae887a0 20 read_log 194215'17364 (194215'17363) delete 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.2174595.0:100641 0.000000 2014-08-13 17:38:02.931475 7fe9bae887a0 20 read_log 194215'17365 (192834'16710) clean ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by osd.285.0:2913 2014-08-12 15:43:45.101252 2014-08-13 17:38:02.931487 7fe9bae887a0 20 read_log 194215'17366 (194215'17365) delete ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.2174595.0:100643 0.000000 2014-08-13 17:38:02.931497 7fe9bae887a0 20 read_log 194215'17367 (192888'16986) clean 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by osd.285.0:2915 2014-08-12 15:43:45.108970 2014-08-13 17:38:02.931509 7fe9bae887a0 20 read_log 194215'17368 (194215'17367) delete 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.2174595.0:100645 0.000000 2014-08-13 17:38:02.931520 7fe9bae887a0 20 read_log 194215'17369 (192834'16748) clean f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by osd.285.0:2917 2014-08-12 15:43:45.116722 2014-08-13 17:38:02.931532 7fe9bae887a0 20 read_log 194215'17370 (194215'17369) delete f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.2174595.0:100647 0.000000 2014-08-13 17:38:02.931542 7fe9bae887a0 20 read_log 194215'17371 (192888'17186) clean 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by osd.285.0:2919 2014-08-12 15:43:45.124996 2014-08-13 17:38:02.931554 7fe9bae887a0 20 read_log 194215'17372 (194215'17371) delete 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.2174595.0:100649 0.000000 2014-08-13 17:38:02.931565 7fe9bae887a0 20 read_log 194215'17373 (192834'16195) clean 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by osd.285.0:2921 2014-08-12 15:43:45.133195 2014-08-13 17:38:02.931577 7fe9bae887a0 20 read_log 194215'17374 (194215'17373) delete 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.2174595.0:100651 0.000000 2014-08-13 17:38:02.931649 7fe9bae887a0 20 read_log 194215'17375 (192888'16967) clean c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by osd.285.0:2923 2014-08-12 15:43:45.140885 2014-08-13 17:38:02.931667 7fe9bae887a0 20 read_log 194215'17376 (194215'17375) delete c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.2174595.0:100653 0.000000 2014-08-13 17:38:02.931678 7fe9bae887a0 20 read_log 194215'17377 (192834'16790) clean c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by osd.285.0:2925 2014-08-12 15:43:45.148400 2014-08-13 17:38:02.931690 7fe9bae887a0 20 read_log 194215'17378 (194215'17377) delete c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.2174595.0:100655 0.000000 2014-08-13 17:38:02.931701 7fe9bae887a0 20 read_log 194215'17379 (192834'16218) clean b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by osd.285.0:2927 2014-08-12 15:43:45.155865 2014-08-13 17:38:02.931720 7fe9bae887a0 20 read_log 194215'17380 (194215'17379) delete b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.2174595.0:100657 0.000000 2014-08-13 17:38:02.931732 7fe9bae887a0 20 read_log 194215'17381 (192834'16278) clean 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by osd.285.0:2929 2014-08-12 15:43:45.162904 2014-08-13 17:38:02.931744 7fe9bae887a0 20 read_log 194215'17382 (194215'17381) delete 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.2174595.0:100659 0.000000 2014-08-13 17:38:02.931754 7fe9bae887a0 20 read_log 194215'17383 (192888'17009) clean efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by osd.285.0:2931 2014-08-12 15:43:45.171731 2014-08-13 17:38:02.931771 7fe9bae887a0 20 read_log 194215'17384 (194215'17383) delete efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.2174595.0:100661 0.000000 2014-08-13 17:38:02.931781 7fe9bae887a0 20 read_log 194215'17385 (192834'16339) clean afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by osd.285.0:2933 2014-08-12 15:43:45.180409 2014-08-13 17:38:02.931794 7fe9bae887a0 20 read_log 194215'17386 (194215'17385) delete afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.2174595.0:100663 0.000000 2014-08-13 17:38:02.931804 7fe9bae887a0 20 read_log 194215'17387 (192834'16830) clean 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by osd.285.0:2935 2014-08-12 15:43:45.188219 2014-08-13 17:38:02.931816 7fe9bae887a0 20 read_log 194215'17388 (194215'17387) delete 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.2174595.0:100665 0.000000 2014-08-13 17:38:02.931827 7fe9bae887a0 20 read_log 194215'17389 (192888'17195) clean 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by osd.285.0:2937 2014-08-12 15:43:45.222729 2014-08-13 17:38:02.931839 7fe9bae887a0 20 read_log 194215'17390 (194215'17389) delete 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.2174595.0:100667 0.000000 2014-08-13 17:38:02.931849 7fe9bae887a0 20 read_log 194215'17391 (192834'16765) clean da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by osd.285.0:2939 2014-08-12 15:43:45.230738 2014-08-13 17:38:02.931861 7fe9bae887a0 20 read_log 194215'17392 (194215'17391) delete da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.2174595.0:100669 0.000000 2014-08-13 17:38:02.931872 7fe9bae887a0 20 read_log 194215'17393 (192888'17010) clean ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by osd.285.0:2941 2014-08-12 15:43:45.239239 2014-08-13 17:38:02.931884 7fe9bae887a0 20 read_log 194215'17394 (194215'17393) delete ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.2174595.0:100671 0.000000 2014-08-13 17:38:02.931894 7fe9bae887a0 20 read_log 194215'17395 (192834'16698) clean a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by osd.285.0:2943 2014-08-12 15:43:45.262532 2014-08-13 17:38:02.931906 7fe9bae887a0 20 read_log 194215'17396 (194215'17395) delete a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.2174595.0:100673 0.000000 2014-08-13 17:38:02.931917 7fe9bae887a0 20 read_log 194215'17397 (192834'16797) clean 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by osd.285.0:2945 2014-08-12 15:43:45.272139 2014-08-13 17:38:02.931929 7fe9bae887a0 20 read_log 194215'17398 (194215'17397) delete 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.2174595.0:100675 0.000000 2014-08-13 17:38:02.931939 7fe9bae887a0 20 read_log 194215'17399 (192834'16327) clean e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by osd.285.0:2947 2014-08-12 15:43:45.302952 2014-08-13 17:38:02.931958 7fe9bae887a0 20 read_log 194215'17400 (194215'17399) delete e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.2174595.0:100677 0.000000 2014-08-13 17:38:02.931969 7fe9bae887a0 20 read_log 194215'17401 (192888'17172) clean 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by osd.285.0:2949 2014-08-12 15:43:45.313335 2014-08-13 17:38:02.931981 7fe9bae887a0 20 read_log 194215'17402 (194215'17401) delete 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.2174595.0:100679 0.000000 2014-08-13 17:38:02.931992 7fe9bae887a0 20 read_log 194215'17403 (192888'17163) clean e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by osd.285.0:2951 2014-08-12 15:43:45.321425 2014-08-13 17:38:02.932004 7fe9bae887a0 20 read_log 194215'17404 (194215'17403) delete e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.2174595.0:100681 0.000000 2014-08-13 17:38:02.932015 7fe9bae887a0 20 read_log 194215'17405 (193305'17270) clean b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by osd.285.0:2953 2014-08-12 15:43:45.346963 2014-08-13 17:38:02.932027 7fe9bae887a0 20 read_log 194215'17406 (194215'17405) delete b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.2174595.0:100683 0.000000 2014-08-13 17:38:02.932037 7fe9bae887a0 20 read_log 194215'17407 (192888'16976) clean c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by osd.285.0:2955 2014-08-12 15:43:45.355489 2014-08-13 17:38:02.932053 7fe9bae887a0 20 read_log 194215'17408 (194215'17407) delete c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.2174595.0:100685 0.000000 2014-08-13 17:38:02.932064 7fe9bae887a0 20 read_log 194215'17409 (192888'16992) clean a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by osd.285.0:2957 2014-08-12 15:43:45.364635 2014-08-13 17:38:02.932076 7fe9bae887a0 20 read_log 194215'17410 (194215'17409) delete a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.2174595.0:100687 0.000000 2014-08-13 17:38:02.932087 7fe9bae887a0 20 read_log 194215'17411 (192888'17164) clean ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by osd.285.0:2959 2014-08-12 15:43:45.373698 2014-08-13 17:38:02.932099 7fe9bae887a0 20 read_log 194215'17412 (194215'17411) delete ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.2174595.0:100689 0.000000 2014-08-13 17:38:02.932110 7fe9bae887a0 20 read_log 194215'17413 (192888'16998) clean 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by osd.285.0:2961 2014-08-12 15:43:45.383128 2014-08-13 17:38:02.932122 7fe9bae887a0 20 read_log 194215'17414 (194215'17413) delete 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.2174595.0:100691 0.000000 2014-08-13 17:38:02.932132 7fe9bae887a0 20 read_log 194215'17415 (192834'16214) clean 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by osd.285.0:2963 2014-08-12 15:43:45.402305 2014-08-13 17:38:02.932144 7fe9bae887a0 20 read_log 194215'17416 (194215'17415) delete 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.2174595.0:100693 0.000000 2014-08-13 17:38:02.932160 7fe9bae887a0 20 read_log 194215'17417 (192888'17004) clean 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by osd.285.0:2965 2014-08-12 15:43:45.410408 2014-08-13 17:38:02.932173 7fe9bae887a0 20 read_log 194215'17418 (194215'17417) delete 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.2174595.0:100695 0.000000 2014-08-13 17:38:02.932183 7fe9bae887a0 20 read_log 194215'17419 (192888'16995) clean d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by osd.285.0:2967 2014-08-12 15:43:45.418929 2014-08-13 17:38:02.932202 7fe9bae887a0 20 read_log 194215'17420 (194215'17419) delete d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.2174595.0:100697 0.000000 2014-08-13 17:38:02.932213 7fe9bae887a0 20 read_log 194215'17421 (193305'17261) clean 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by osd.285.0:2969 2014-08-12 15:43:45.427273 2014-08-13 17:38:02.932226 7fe9bae887a0 20 read_log 194215'17422 (194215'17421) delete 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.2174595.0:100699 0.000000 2014-08-13 17:38:02.932236 7fe9bae887a0 20 read_log 194215'17423 (193305'17272) clean ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by osd.285.0:2971 2014-08-12 15:43:45.435639 2014-08-13 17:38:02.932248 7fe9bae887a0 20 read_log 194215'17424 (194215'17423) delete ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.2174595.0:100701 0.000000 2014-08-13 17:38:02.932259 7fe9bae887a0 20 read_log 194215'17425 (192834'16245) clean dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by osd.285.0:2973 2014-08-12 15:43:45.444189 2014-08-13 17:38:02.932271 7fe9bae887a0 20 read_log 194215'17426 (194215'17425) delete dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.2174595.0:100703 0.000000 2014-08-13 17:38:02.932281 7fe9bae887a0 20 read_log 194215'17427 (192834'16774) clean c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by osd.285.0:2975 2014-08-12 15:43:45.452405 2014-08-13 17:38:02.932293 7fe9bae887a0 20 read_log 194215'17428 (194215'17427) delete c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.2174595.0:100705 0.000000 2014-08-13 17:38:02.932303 7fe9bae887a0 20 read_log 194215'17429 (192834'16816) clean 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by osd.285.0:2977 2014-08-12 15:43:45.460567 2014-08-13 17:38:02.932315 7fe9bae887a0 20 read_log 194215'17430 (194215'17429) delete 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.2174595.0:100707 0.000000 2014-08-13 17:38:02.932326 7fe9bae887a0 20 read_log 194215'17431 (192888'17191) clean a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by osd.285.0:2979 2014-08-12 15:43:45.469302 2014-08-13 17:38:02.932338 7fe9bae887a0 20 read_log 194215'17432 (194215'17431) delete a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.2174595.0:100709 0.000000 2014-08-13 17:38:02.932348 7fe9bae887a0 20 read_log 194215'17433 (193305'17254) clean 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by osd.285.0:2981 2014-08-12 15:43:45.477308 2014-08-13 17:38:02.932360 7fe9bae887a0 20 read_log 194215'17434 (194215'17433) delete 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.2174595.0:100711 0.000000 2014-08-13 17:38:02.932371 7fe9bae887a0 20 read_log 194215'17435 (192834'16201) clean 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by osd.285.0:2983 2014-08-12 15:43:45.485662 2014-08-13 17:38:02.932383 7fe9bae887a0 20 read_log 194215'17436 (194215'17435) delete 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.2174595.0:100713 0.000000 2014-08-13 17:38:02.932393 7fe9bae887a0 20 read_log 194215'17437 (192888'17154) clean 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by osd.285.0:2985 2014-08-12 15:43:45.494587 2014-08-13 17:38:02.932405 7fe9bae887a0 20 read_log 194215'17438 (194215'17437) delete 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.2174595.0:100715 0.000000 2014-08-13 17:38:02.932416 7fe9bae887a0 20 read_log 194215'17439 (192834'16750) clean 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by osd.285.0:2987 2014-08-12 15:43:45.502572 2014-08-13 17:38:02.932437 7fe9bae887a0 20 read_log 194215'17440 (194215'17439) delete 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.2174595.0:100717 0.000000 2014-08-13 17:38:02.932449 7fe9bae887a0 20 read_log 194215'17441 (192888'16989) clean 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by osd.285.0:2989 2014-08-12 15:43:45.529043 2014-08-13 17:38:02.932461 7fe9bae887a0 20 read_log 194215'17442 (194215'17441) delete 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.2174595.0:100719 0.000000 2014-08-13 17:38:02.932471 7fe9bae887a0 20 read_log 194215'17443 (192888'17176) clean 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by osd.285.0:2991 2014-08-12 15:43:45.538065 2014-08-13 17:38:02.932485 7fe9bae887a0 20 read_log 194215'17444 (194215'17443) delete 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.2174595.0:100721 0.000000 2014-08-13 17:38:02.932495 7fe9bae887a0 20 read_log 194215'17445 (192888'17005) clean 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by osd.285.0:2993 2014-08-12 15:43:45.546055 2014-08-13 17:38:02.932508 7fe9bae887a0 20 read_log 194215'17446 (194215'17445) delete 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.2174595.0:100723 0.000000 2014-08-13 17:38:02.932518 7fe9bae887a0 20 read_log 194215'17447 (193305'17259) clean 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by osd.285.0:2995 2014-08-12 15:43:45.555250 2014-08-13 17:38:02.932530 7fe9bae887a0 20 read_log 194215'17448 (194215'17447) delete 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.2174595.0:100725 0.000000 2014-08-13 17:38:02.932541 7fe9bae887a0 20 read_log 194215'17449 (192834'16845) clean 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by osd.285.0:2997 2014-08-12 15:43:45.563625 2014-08-13 17:38:02.932552 7fe9bae887a0 20 read_log 194215'17450 (194215'17449) delete 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.2174595.0:100727 0.000000 2014-08-13 17:38:02.932563 7fe9bae887a0 20 read_log 194215'17451 (192834'16796) clean c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by osd.285.0:2999 2014-08-12 15:43:45.572256 2014-08-13 17:38:02.932574 7fe9bae887a0 20 read_log 194215'17452 (194215'17451) delete c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.2174595.0:100729 0.000000 2014-08-13 17:38:02.932585 7fe9bae887a0 20 read_log 194215'17453 (192888'16993) clean a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by osd.285.0:3001 2014-08-12 15:43:45.614869 2014-08-13 17:38:02.932597 7fe9bae887a0 20 read_log 194215'17454 (194215'17453) delete a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.2174595.0:100731 0.000000 2014-08-13 17:38:02.932607 7fe9bae887a0 20 read_log 194215'17455 (192834'16212) clean 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by osd.285.0:3003 2014-08-12 15:43:45.623073 2014-08-13 17:38:02.932619 7fe9bae887a0 20 read_log 194215'17456 (194215'17455) delete 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.2174595.0:100733 0.000000 2014-08-13 17:38:02.932630 7fe9bae887a0 20 read_log 194215'17457 (192888'16981) clean 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by osd.285.0:3005 2014-08-12 15:43:45.631157 2014-08-13 17:38:02.932641 7fe9bae887a0 20 read_log 194215'17458 (194215'17457) delete 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.2174595.0:100735 0.000000 2014-08-13 17:38:02.932652 7fe9bae887a0 20 read_log 194215'17459 (192834'16261) clean e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by osd.285.0:3007 2014-08-12 15:43:45.638643 2014-08-13 17:38:02.932670 7fe9bae887a0 20 read_log 194215'17460 (194215'17459) delete e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.2174595.0:100737 0.000000 2014-08-13 17:38:02.932681 7fe9bae887a0 20 read_log 194215'17461 (192888'17014) clean f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by osd.285.0:3009 2014-08-12 15:43:45.646167 2014-08-13 17:38:02.932693 7fe9bae887a0 20 read_log 194215'17462 (194215'17461) delete f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.2174595.0:100739 0.000000 2014-08-13 17:38:02.932704 7fe9bae887a0 20 read_log 194215'17463 (192888'16987) clean 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by osd.285.0:3011 2014-08-12 15:43:45.670883 2014-08-13 17:38:02.932716 7fe9bae887a0 20 read_log 194215'17464 (194215'17463) delete 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.2174595.0:100741 0.000000 2014-08-13 17:38:02.932726 7fe9bae887a0 20 read_log 194215'17465 (192834'16249) clean 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by osd.285.0:3013 2014-08-12 15:43:45.679581 2014-08-13 17:38:02.932738 7fe9bae887a0 20 read_log 194215'17466 (194215'17465) delete 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.2174595.0:100743 0.000000 2014-08-13 17:38:02.932749 7fe9bae887a0 20 read_log 194215'17467 (192888'17200) clean e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by osd.285.0:3015 2014-08-12 15:43:45.689120 2014-08-13 17:38:02.932780 7fe9bae887a0 20 read_log 194215'17468 (194215'17467) delete e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.2174595.0:100745 0.000000 2014-08-13 17:38:02.932791 7fe9bae887a0 20 read_log 194215'17469 (193305'17265) clean d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by osd.285.0:3017 2014-08-12 15:43:45.698038 2014-08-13 17:38:02.932804 7fe9bae887a0 20 read_log 194215'17470 (194215'17469) delete d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.2174595.0:100747 0.000000 2014-08-13 17:38:02.932814 7fe9bae887a0 20 read_log 194215'17471 (192888'17189) clean a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by osd.285.0:3019 2014-08-12 15:43:45.713776 2014-08-13 17:38:02.932826 7fe9bae887a0 20 read_log 194215'17472 (194215'17471) delete a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.2174595.0:100749 0.000000 2014-08-13 17:38:02.932837 7fe9bae887a0 20 read_log 194215'17473 (192888'17207) clean 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by osd.285.0:3021 2014-08-12 15:43:45.721341 2014-08-13 17:38:02.932849 7fe9bae887a0 20 read_log 194215'17474 (194215'17473) delete 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.2174595.0:100751 0.000000 2014-08-13 17:38:02.932860 7fe9bae887a0 20 read_log 194215'17475 (192888'17193) clean 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by osd.285.0:3023 2014-08-12 15:43:45.730042 2014-08-13 17:38:02.932872 7fe9bae887a0 20 read_log 194215'17476 (194215'17475) delete 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.2174595.0:100753 0.000000 2014-08-13 17:38:02.932882 7fe9bae887a0 20 read_log 194215'17477 (192888'17178) clean 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by osd.285.0:3025 2014-08-12 15:43:45.740294 2014-08-13 17:38:02.932894 7fe9bae887a0 20 read_log 194215'17478 (194215'17477) delete 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.2174595.0:100755 0.000000 2014-08-13 17:38:02.932905 7fe9bae887a0 20 read_log 194215'17479 (192834'16254) clean c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by osd.285.0:3027 2014-08-12 15:43:45.748782 2014-08-13 17:38:02.932939 7fe9bae887a0 20 read_log 194215'17480 (194215'17479) delete c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.2174595.0:100757 0.000000 2014-08-13 17:38:02.932950 7fe9bae887a0 20 read_log 194215'17481 (192834'16753) clean d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by osd.285.0:3029 2014-08-12 15:43:45.757452 2014-08-13 17:38:02.932962 7fe9bae887a0 20 read_log 194215'17482 (194215'17481) delete d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.2174595.0:100759 0.000000 2014-08-13 17:38:02.932973 7fe9bae887a0 20 read_log 194215'17483 (193305'17266) clean 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by osd.285.0:3031 2014-08-12 15:43:45.766117 2014-08-13 17:38:02.932985 7fe9bae887a0 20 read_log 194215'17484 (194215'17483) delete 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.2174595.0:100761 0.000000 2014-08-13 17:38:02.932996 7fe9bae887a0 20 read_log 194215'17485 (193305'17269) clean 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by osd.285.0:3033 2014-08-12 15:43:45.787496 2014-08-13 17:38:02.933007 7fe9bae887a0 20 read_log 194215'17486 (194215'17485) delete 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.2174595.0:100763 0.000000 2014-08-13 17:38:02.933018 7fe9bae887a0 20 read_log 194215'17487 (192834'16291) clean 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by osd.285.0:3035 2014-08-12 15:43:45.796683 2014-08-13 17:38:02.933030 7fe9bae887a0 20 read_log 194215'17488 (194215'17487) delete 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.2174595.0:100765 0.000000 2014-08-13 17:38:02.933041 7fe9bae887a0 20 read_log 194215'17489 (193305'17262) clean bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by osd.285.0:3037 2014-08-12 15:43:45.830400 2014-08-13 17:38:02.933052 7fe9bae887a0 20 read_log 194215'17490 (194215'17489) delete bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.2174595.0:100767 0.000000 2014-08-13 17:38:02.933063 7fe9bae887a0 20 read_log 194215'17491 (192834'16836) clean f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by osd.285.0:3039 2014-08-12 15:43:45.840311 2014-08-13 17:38:02.933075 7fe9bae887a0 20 read_log 194215'17492 (194215'17491) delete f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.2174595.0:100769 0.000000 2014-08-13 17:38:02.933085 7fe9bae887a0 20 read_log 194215'17493 (192834'16835) clean 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by osd.285.0:3041 2014-08-12 15:43:45.850390 2014-08-13 17:38:02.933097 7fe9bae887a0 20 read_log 194215'17494 (194215'17493) delete 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.2174595.0:100771 0.000000 2014-08-13 17:38:02.933108 7fe9bae887a0 20 read_log 194215'17495 (192888'17165) clean 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by osd.285.0:3043 2014-08-12 15:43:45.871633 2014-08-13 17:38:02.933120 7fe9bae887a0 20 read_log 194215'17496 (194215'17495) delete 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.2174595.0:100773 0.000000 2014-08-13 17:38:02.933130 7fe9bae887a0 20 read_log 194215'17497 (192888'17179) clean 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by osd.285.0:3045 2014-08-12 15:43:45.879855 2014-08-13 17:38:02.933142 7fe9bae887a0 20 read_log 194215'17498 (194215'17497) delete 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.2174595.0:100775 0.000000 2014-08-13 17:38:02.933163 7fe9bae887a0 20 read_log 194215'17499 (192888'17184) clean 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by osd.285.0:3047 2014-08-12 15:43:45.889702 2014-08-13 17:38:02.933186 7fe9bae887a0 20 read_log 194215'17500 (194215'17499) delete 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.2174595.0:100777 0.000000 2014-08-13 17:38:02.933198 7fe9bae887a0 20 read_log 194215'17501 (192834'16724) clean 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by osd.285.0:3049 2014-08-12 15:43:45.898121 2014-08-13 17:38:02.933210 7fe9bae887a0 20 read_log 194215'17502 (194215'17501) delete 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.2174595.0:100779 0.000000 2014-08-13 17:38:02.933221 7fe9bae887a0 20 read_log 194215'17503 (192888'17170) clean e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by osd.285.0:3051 2014-08-12 15:43:45.922795 2014-08-13 17:38:02.933233 7fe9bae887a0 20 read_log 194215'17504 (194215'17503) delete e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.2174595.0:100781 0.000000 2014-08-13 17:38:02.933244 7fe9bae887a0 20 read_log 194215'17505 (192888'17173) clean fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by osd.285.0:3053 2014-08-12 15:43:45.932047 2014-08-13 17:38:02.933256 7fe9bae887a0 20 read_log 194215'17506 (194215'17505) delete fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.2174595.0:100783 0.000000 2014-08-13 17:38:02.933266 7fe9bae887a0 20 read_log 194215'17507 (193305'17271) clean b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by osd.285.0:3055 2014-08-12 15:43:45.941086 2014-08-13 17:38:02.933278 7fe9bae887a0 20 read_log 194215'17508 (194215'17507) delete b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.2174595.0:100785 0.000000 2014-08-13 17:38:02.933289 7fe9bae887a0 20 read_log 194215'17509 (192888'17175) clean 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by osd.285.0:3057 2014-08-12 15:43:45.950348 2014-08-13 17:38:02.933301 7fe9bae887a0 20 read_log 194215'17510 (194215'17509) delete 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.2174595.0:100787 0.000000 2014-08-13 17:38:02.933311 7fe9bae887a0 20 read_log 194215'17511 (192834'16318) clean 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by osd.285.0:3059 2014-08-12 15:43:45.958667 2014-08-13 17:38:02.933323 7fe9bae887a0 20 read_log 194215'17512 (194215'17511) delete 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.2174595.0:100789 0.000000 2014-08-13 17:38:02.933334 7fe9bae887a0 20 read_log 194215'17513 (192888'17169) clean e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by osd.285.0:3061 2014-08-12 15:43:45.968100 2014-08-13 17:38:02.933345 7fe9bae887a0 20 read_log 194215'17514 (194215'17513) delete e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.2174595.0:100791 0.000000 2014-08-13 17:38:02.933364 7fe9bae887a0 20 read_log 194215'17515 (192834'16793) clean 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by osd.285.0:3063 2014-08-12 15:43:45.977317 2014-08-13 17:38:02.933376 7fe9bae887a0 20 read_log 194215'17516 (194215'17515) delete 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.2174595.0:100793 0.000000 2014-08-13 17:38:02.933387 7fe9bae887a0 20 read_log 194215'17517 (192834'16295) clean 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by osd.285.0:3065 2014-08-12 15:43:45.984879 2014-08-13 17:38:02.933399 7fe9bae887a0 20 read_log 194215'17518 (194215'17517) delete 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.2174595.0:100795 0.000000 2014-08-13 17:38:02.933409 7fe9bae887a0 20 read_log 194215'17519 (193305'17260) clean e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by osd.285.0:3067 2014-08-12 15:43:46.008737 2014-08-13 17:38:02.933429 7fe9bae887a0 20 read_log 194215'17520 (194215'17519) delete e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.2174595.0:100797 0.000000 2014-08-13 17:38:02.933440 7fe9bae887a0 20 read_log 194215'17521 (192888'17182) clean e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by osd.285.0:3069 2014-08-12 15:43:46.017163 2014-08-13 17:38:02.933452 7fe9bae887a0 20 read_log 194215'17522 (194215'17521) delete e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.2174595.0:100799 0.000000 2014-08-13 17:38:02.933463 7fe9bae887a0 20 read_log 194215'17523 (193305'17257) clean 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by osd.285.0:3071 2014-08-12 15:43:46.025336 2014-08-13 17:38:02.933475 7fe9bae887a0 20 read_log 194215'17524 (194215'17523) delete 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.2174595.0:100801 0.000000 2014-08-13 17:38:02.933486 7fe9bae887a0 20 read_log 194215'17525 (192834'16309) clean 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by osd.285.0:3073 2014-08-12 15:43:46.033855 2014-08-13 17:38:02.933498 7fe9bae887a0 20 read_log 194215'17526 (194215'17525) delete 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.2174595.0:100803 0.000000 2014-08-13 17:38:02.933508 7fe9bae887a0 20 read_log 194215'17527 (192888'16970) clean a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by osd.285.0:3075 2014-08-12 15:43:46.042207 2014-08-13 17:38:02.933520 7fe9bae887a0 20 read_log 194215'17528 (194215'17527) delete a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.2174595.0:100805 0.000000 2014-08-13 17:38:02.933531 7fe9bae887a0 20 read_log 194215'17529 (194215'17529) modify 1e6/hit_set_24.1e6_archive_2014-08-12 15:43:44.587829_2014-08-12 17:24:40.436910/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.437356 2014-08-13 17:38:02.933543 7fe9bae887a0 20 read_log 194215'17530 (194215'17273) delete 1e6/hit_set_24.1e6_archive_2014-08-12 15:32:25.782585_2014-08-12 15:43:44.587553/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.437356 2014-08-13 17:38:02.933555 7fe9bae887a0 20 read_log 194215'17531 (0'0) promote 23a635e6/benchmark_data_storage0111-ib.pocstor_34628_object151/head//24 by osd.285.0:6158 2014-08-12 17:24:40.442940 2014-08-13 17:38:02.933567 7fe9bae887a0 20 read_log 194215'17532 (194215'17531) modify 23a635e6/benchmark_data_storage0111-ib.pocstor_34628_object151/head//24 by client.2193759.0:152 2014-08-12 17:24:40.428426 2014-08-13 17:38:02.933579 7fe9bae887a0 20 read_log 194215'17533 (0'0) promote a6e755e6/benchmark_data_storage0111-ib.pocstor_34628_object861/head//24 by osd.285.0:6191 2014-08-12 17:24:43.239335 2014-08-13 17:38:02.933591 7fe9bae887a0 20 read_log 194215'17534 (194215'17533) modify a6e755e6/benchmark_data_storage0111-ib.pocstor_34628_object861/head//24 by client.2193759.0:862 2014-08-12 17:24:43.232558 2014-08-13 17:38:02.933603 7fe9bae887a0 20 read_log 194215'17535 (0'0) promote 79b31e6/benchmark_data_storage0111-ib.pocstor_34628_object1071/head//24 by osd.285.0:6195 2014-08-12 17:24:44.057625 2014-08-13 17:38:02.933615 7fe9bae887a0 20 read_log 194215'17536 (194215'17535) modify 79b31e6/benchmark_data_storage0111-ib.pocstor_34628_object1071/head//24 by client.2193759.0:1072 2014-08-12 17:24:44.051082 2014-08-13 17:38:02.933627 7fe9bae887a0 20 read_log 194215'17537 (0'0) promote 234a49e6/benchmark_data_storage0111-ib.pocstor_34628_object1143/head//24 by osd.285.0:6197 2014-08-12 17:24:44.330931 2014-08-13 17:38:02.933639 7fe9bae887a0 20 read_log 194215'17538 (194215'17537) modify 234a49e6/benchmark_data_storage0111-ib.pocstor_34628_object1143/head//24 by client.2193759.0:1144 2014-08-12 17:24:44.324387 2014-08-13 17:38:02.933658 7fe9bae887a0 20 read_log 194215'17539 (0'0) promote 739665e6/benchmark_data_storage0111-ib.pocstor_34628_object1445/head//24 by osd.285.0:6207 2014-08-12 17:24:46.551733 2014-08-13 17:38:02.933671 7fe9bae887a0 20 read_log 194215'17540 (194215'17539) modify 739665e6/benchmark_data_storage0111-ib.pocstor_34628_object1445/head//24 by client.2193759.0:1446 2014-08-12 17:24:46.538512 2014-08-13 17:38:02.933683 7fe9bae887a0 20 read_log 194216'17541 (0'0) promote 429831e6/benchmark_data_storage0111-ib.pocstor_39693_object6/head//24 by osd.285.0:6211 2014-08-12 17:25:03.177318 2014-08-13 17:38:02.933695 7fe9bae887a0 20 read_log 194216'17542 (194216'17541) modify 429831e6/benchmark_data_storage0111-ib.pocstor_39693_object6/head//24 by client.1487927.0:7 2014-08-12 17:25:03.160232 2014-08-13 17:38:02.933707 7fe9bae887a0 20 read_log 194370'17543 (0'0) promote 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by osd.285.0:6234 2014-08-12 20:30:17.198376 2014-08-13 17:38:02.933721 7fe9bae887a0 20 read_log 194370'17544 (194370'17543) modify 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by client.1508531.0:743 2014-08-12 20:30:17.192859 2014-08-13 17:38:02.933734 7fe9bae887a0 20 read_log 194370'17545 (0'0) promote 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by osd.285.0:6238 2014-08-12 20:30:17.300018 2014-08-13 17:38:02.933745 7fe9bae887a0 20 read_log 194370'17546 (194370'17545) modify 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by client.1508531.0:771 2014-08-12 20:30:17.293344 2014-08-13 17:38:02.933757 7fe9bae887a0 20 read_log 194370'17547 (0'0) promote 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by osd.285.0:6264 2014-08-12 20:30:21.630458 2014-08-13 17:38:02.933769 7fe9bae887a0 20 read_log 194370'17548 (194370'17547) modify 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by client.1508531.0:1945 2014-08-12 20:30:21.623294 2014-08-13 17:38:02.933781 7fe9bae887a0 20 read_log 194370'17549 (0'0) promote d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by osd.285.0:6294 2014-08-12 20:30:26.872631 2014-08-13 17:38:02.933793 7fe9bae887a0 20 read_log 194370'17550 (194370'17549) modify d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by client.1508531.0:3362 2014-08-12 20:30:26.866144 2014-08-13 17:38:02.933805 7fe9bae887a0 20 read_log 194370'17551 (0'0) promote 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by osd.285.0:6300 2014-08-12 20:30:28.756668 2014-08-13 17:38:02.933817 7fe9bae887a0 20 read_log 194370'17552 (194370'17551) modify 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by client.1508531.0:3852 2014-08-12 20:30:28.748728 2014-08-13 17:38:02.933830 7fe9bae887a0 20 read_log 194370'17553 (0'0) promote 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by osd.285.0:6326 2014-08-12 20:30:33.176959 2014-08-13 17:38:02.933842 7fe9bae887a0 20 read_log 194370'17554 (194370'17553) modify 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by client.1508531.0:4986 2014-08-12 20:30:33.168763 2014-08-13 17:38:02.933853 7fe9bae887a0 20 read_log 194370'17555 (0'0) promote b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by osd.285.0:6334 2014-08-12 20:30:33.999532 2014-08-13 17:38:02.933865 7fe9bae887a0 20 read_log 194370'17556 (194370'17555) modify b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by client.1508531.0:5182 2014-08-12 20:30:33.991823 2014-08-13 17:38:02.933877 7fe9bae887a0 20 read_log 194370'17557 (0'0) promote d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by osd.285.0:6358 2014-08-12 20:30:37.932283 2014-08-13 17:38:02.933898 7fe9bae887a0 20 read_log 194370'17558 (194370'17557) modify d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by client.1508531.0:6043 2014-08-12 20:30:37.924085 2014-08-13 17:38:02.933911 7fe9bae887a0 20 read_log 194370'17559 (0'0) promote 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by osd.285.0:6362 2014-08-12 20:30:39.292374 2014-08-13 17:38:02.933923 7fe9bae887a0 20 read_log 194370'17560 (194370'17559) modify 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by client.1508531.0:6359 2014-08-12 20:30:39.285755 2014-08-13 17:38:02.933963 7fe9bae887a0 20 read_log 194370'17561 (0'0) promote 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by osd.285.0:6364 2014-08-12 20:30:39.937017 2014-08-13 17:38:02.933975 7fe9bae887a0 20 read_log 194370'17562 (194370'17561) modify 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by client.1508531.0:6493 2014-08-12 20:30:39.927587 2014-08-13 17:38:02.933988 7fe9bae887a0 20 read_log 194370'17563 (0'0) promote 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by osd.285.0:6370 2014-08-12 20:30:41.251881 2014-08-13 17:38:02.933999 7fe9bae887a0 20 read_log 194370'17564 (194370'17563) modify 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by client.1508531.0:6777 2014-08-12 20:30:41.243264 2014-08-13 17:38:02.934011 7fe9bae887a0 20 read_log 194370'17565 (0'0) promote 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by osd.285.0:6426 2014-08-12 20:30:54.950682 2014-08-13 17:38:02.934023 7fe9bae887a0 20 read_log 194370'17566 (194370'17565) modify 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by client.1508531.0:9270 2014-08-12 20:30:54.943986 2014-08-13 17:38:02.934035 7fe9bae887a0 20 read_log 194377'17567 (0'0) promote baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by osd.285.0:6542 2014-08-12 20:33:00.374306 2014-08-13 17:38:02.934047 7fe9bae887a0 20 read_log 194377'17568 (194377'17567) modify baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by client.2231190.0:750 2014-08-12 20:33:00.363477 2014-08-13 17:38:02.934059 7fe9bae887a0 20 read_log 194377'17569 (0'0) promote 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by osd.285.0:6552 2014-08-12 20:33:02.432625 2014-08-13 17:38:02.934071 7fe9bae887a0 20 read_log 194377'17570 (194377'17569) modify 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by client.2231190.0:1297 2014-08-12 20:33:02.426220 2014-08-13 17:38:02.934083 7fe9bae887a0 20 read_log 194377'17571 (0'0) promote 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by osd.285.0:6572 2014-08-12 20:33:06.347750 2014-08-13 17:38:02.934095 7fe9bae887a0 20 read_log 194377'17572 (194377'17571) modify 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by client.2231190.0:2270 2014-08-12 20:33:06.340532 2014-08-13 17:38:02.934107 7fe9bae887a0 20 read_log 194380'17573 (0'0) promote 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by osd.285.0:6638 2014-08-12 20:33:22.396527 2014-08-13 17:38:02.934119 7fe9bae887a0 20 read_log 194380'17574 (194380'17573) modify 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by client.2231190.0:4956 2014-08-12 20:33:22.387767 2014-08-13 17:38:02.934130 7fe9bae887a0 20 read_log 194380'17575 (0'0) promote 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by osd.285.0:6650 2014-08-12 20:33:26.161972 2014-08-13 17:38:02.934142 7fe9bae887a0 20 read_log 194380'17576 (194380'17575) modify 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by client.2231190.0:5227 2014-08-12 20:33:26.151470 2014-08-13 17:38:02.934168 7fe9bae887a0 20 read_log 194383'17577 (0'0) promote e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by osd.285.0:6662 2014-08-12 20:34:01.239061 2014-08-13 17:38:02.934181 7fe9bae887a0 20 read_log 194383'17578 (194383'17577) modify e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by client.2231190.0:5965 2014-08-12 20:34:01.231283 2014-08-13 17:38:02.934193 7fe9bae887a0 20 read_log 194383'17579 (0'0) promote a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by osd.285.0:6678 2014-08-12 20:34:08.806925 2014-08-13 17:38:02.934205 7fe9bae887a0 20 read_log 194383'17580 (194383'17579) modify a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by client.2231190.0:6767 2014-08-12 20:34:08.797961 2014-08-13 17:38:02.934217 7fe9bae887a0 20 read_log 194383'17581 (0'0) promote 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by osd.285.0:6694 2014-08-12 20:34:32.456987 2014-08-13 17:38:02.934229 7fe9bae887a0 20 read_log 194383'17582 (194383'17581) modify 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by client.2231190.0:7636 2014-08-12 20:34:32.449118 2014-08-13 17:38:02.934241 7fe9bae887a0 20 read_log 194393'17583 (0'0) promote c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by osd.285.0:6733 2014-08-12 20:49:26.428594 2014-08-13 17:38:02.934254 7fe9bae887a0 20 read_log 194393'17584 (194393'17583) modify c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by client.1064182.0:200 2014-08-12 20:49:26.420766 2014-08-13 17:38:02.934266 7fe9bae887a0 20 read_log 194393'17585 (0'0) promote 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by osd.285.0:6742 2014-08-12 20:49:27.680263 2014-08-13 17:38:02.934278 7fe9bae887a0 20 read_log 194393'17586 (194393'17585) modify 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by client.1064182.0:434 2014-08-12 20:49:27.672142 2014-08-13 17:38:02.934289 7fe9bae887a0 20 read_log 194393'17587 (0'0) promote 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by osd.285.0:6752 2014-08-12 20:52:33.478035 2014-08-13 17:38:02.934301 7fe9bae887a0 20 read_log 194393'17588 (194393'17587) modify 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by client.1064422.0:233 2014-08-12 20:52:33.470274 2014-08-13 17:38:02.934313 7fe9bae887a0 20 read_log 194416'17589 (0'0) promote 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by osd.285.0:6881 2014-08-13 15:28:29.954653 2014-08-13 17:38:02.934325 7fe9bae887a0 20 read_log 194416'17590 (194416'17589) modify 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by client.2441958.0:4 2014-08-13 15:28:29.919716 2014-08-13 17:38:02.934337 7fe9bae887a0 20 read_log 194416'17591 (0'0) promote 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by osd.285.0:6898 2014-08-13 15:28:34.290031 2014-08-13 17:38:02.934349 7fe9bae887a0 20 read_log 194416'17592 (194416'17591) modify 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by client.2441958.0:945 2014-08-13 15:28:34.269526 2014-08-13 17:38:02.934361 7fe9bae887a0 20 read_log 194416'17593 (0'0) promote 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by osd.285.0:6942 2014-08-13 15:28:41.002911 2014-08-13 17:38:02.934373 7fe9bae887a0 20 read_log 194416'17594 (194416'17593) modify 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by client.2441958.0:2714 2014-08-13 15:28:40.995272 2014-08-13 17:38:02.934386 7fe9bae887a0 20 read_log 194416'17595 (0'0) promote 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by osd.285.0:6952 2014-08-13 15:28:41.670103 2014-08-13 17:38:02.934405 7fe9bae887a0 20 read_log 194416'17596 (194416'17595) modify 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by client.2441958.0:2891 2014-08-13 15:28:41.661418 2014-08-13 17:38:02.934417 7fe9bae887a0 20 read_log 194416'17597 (0'0) promote 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by osd.285.0:6986 2014-08-13 15:28:48.780468 2014-08-13 17:38:02.934429 7fe9bae887a0 20 read_log 194416'17598 (194416'17597) modify 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by client.2441958.0:4705 2014-08-13 15:28:48.770243 2014-08-13 17:38:02.934442 7fe9bae887a0 20 read_log 194416'17599 (0'0) promote ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by osd.285.0:7006 2014-08-13 15:28:51.781837 2014-08-13 17:38:02.934454 7fe9bae887a0 20 read_log 194416'17600 (194416'17599) modify ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by client.2441958.0:5497 2014-08-13 15:28:51.775367 2014-08-13 17:38:02.934467 7fe9bae887a0 20 read_log 194416'17601 (0'0) promote c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by osd.285.0:7010 2014-08-13 15:28:52.158566 2014-08-13 17:38:02.934480 7fe9bae887a0 20 read_log 194416'17602 (194416'17601) modify c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by client.2441958.0:5594 2014-08-13 15:28:52.148545 2014-08-13 17:38:02.934492 7fe9bae887a0 20 read_log 194416'17603 (0'0) promote 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by osd.285.0:7024 2014-08-13 15:28:56.470640 2014-08-13 17:38:02.934504 7fe9bae887a0 20 read_log 194416'17604 (194416'17603) modify 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by client.2441958.0:6648 2014-08-13 15:28:56.462954 2014-08-13 17:38:02.934516 7fe9bae887a0 20 read_log 194416'17605 (0'0) promote 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by osd.285.0:7034 2014-08-13 15:28:59.166125 2014-08-13 17:38:02.934528 7fe9bae887a0 20 read_log 194416'17606 (194416'17605) modify 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by client.2441958.0:7371 2014-08-13 15:28:59.157077 2014-08-13 17:38:02.934540 7fe9bae887a0 20 read_log 194416'17607 (0'0) promote 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by osd.285.0:7070 2014-08-13 15:29:05.568880 2014-08-13 17:38:02.934552 7fe9bae887a0 20 read_log 194416'17608 (194416'17607) modify 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by client.2441958.0:9022 2014-08-13 15:29:05.558364 2014-08-13 17:38:02.934563 7fe9bae887a0 20 read_log 194416'17609 (0'0) promote 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by osd.285.0:7096 2014-08-13 15:29:07.899630 2014-08-13 17:38:02.934575 7fe9bae887a0 20 read_log 194416'17610 (194416'17609) modify 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by client.2441958.0:9635 2014-08-13 15:29:07.892021 2014-08-13 17:38:02.934587 7fe9bae887a0 20 read_log 194416'17611 (0'0) promote 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by osd.285.0:7098 2014-08-13 15:29:08.249448 2014-08-13 17:38:02.934599 7fe9bae887a0 20 read_log 194416'17612 (194416'17611) modify 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by client.2441958.0:9727 2014-08-13 15:29:08.241468 2014-08-13 17:38:02.934611 7fe9bae887a0 20 read_log 194416'17613 (0'0) promote c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by osd.285.0:7114 2014-08-13 15:29:11.345668 2014-08-13 17:38:02.934623 7fe9bae887a0 20 read_log 194416'17614 (194416'17613) modify c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by client.2441958.0:10475 2014-08-13 15:29:11.336396 2014-08-13 17:38:02.934634 7fe9bae887a0 20 read_log 194416'17615 (0'0) promote 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by osd.285.0:7136 2014-08-13 15:29:16.427040 2014-08-13 17:38:02.934656 7fe9bae887a0 20 read_log 194416'17616 (194416'17615) modify 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by client.2441958.0:11725 2014-08-13 15:29:16.417986 2014-08-13 17:38:02.934669 7fe9bae887a0 20 read_log 194416'17617 (0'0) promote d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by osd.285.0:7146 2014-08-13 15:29:17.758852 2014-08-13 17:38:02.934681 7fe9bae887a0 20 read_log 194416'17618 (194416'17617) modify d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by client.2441958.0:12034 2014-08-13 15:29:17.749689 2014-08-13 17:38:02.934693 7fe9bae887a0 20 read_log 194416'17619 (0'0) promote 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by osd.285.0:7154 2014-08-13 15:29:19.103144 2014-08-13 17:38:02.934705 7fe9bae887a0 20 read_log 194416'17620 (194416'17619) modify 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by client.2441958.0:12344 2014-08-13 15:29:19.093009 2014-08-13 17:38:02.934717 7fe9bae887a0 20 read_log 194416'17621 (0'0) promote 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by osd.285.0:7252 2014-08-13 15:29:41.736838 2014-08-13 17:38:02.934729 7fe9bae887a0 20 read_log 194416'17622 (194416'17621) modify 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by client.2441958.0:16606 2014-08-13 15:29:41.727382 2014-08-13 17:38:02.934741 7fe9bae887a0 20 read_log 194416'17623 (0'0) promote 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by osd.285.0:7254 2014-08-13 15:29:42.050800 2014-08-13 17:38:02.934753 7fe9bae887a0 20 read_log 194416'17624 (194416'17623) modify 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by client.2441958.0:16630 2014-08-13 15:29:42.041487 2014-08-13 17:38:02.934765 7fe9bae887a0 20 read_log 194416'17625 (0'0) promote a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by osd.285.0:7276 2014-08-13 15:29:53.367892 2014-08-13 17:38:02.934777 7fe9bae887a0 20 read_log 194416'17626 (194416'17625) modify a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by client.2441958.0:17425 2014-08-13 15:29:53.358926 2014-08-13 17:38:02.934789 7fe9bae887a0 20 read_log 194416'17627 (0'0) promote 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by osd.285.0:7284 2014-08-13 15:29:58.352010 2014-08-13 17:38:02.934800 7fe9bae887a0 20 read_log 194416'17628 (194416'17627) modify 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by client.2441958.0:17781 2014-08-13 15:29:58.341408 2014-08-13 17:38:02.934812 7fe9bae887a0 20 read_log 194422'17629 (0'0) promote dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by osd.285.0:7336 2014-08-13 15:32:20.791483 2014-08-13 17:38:02.934824 7fe9bae887a0 20 read_log 194422'17630 (194422'17629) modify dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by client.1148203.0:1378 2014-08-13 15:32:20.782756 2014-08-13 17:38:02.934836 7fe9bae887a0 20 read_log 194422'17631 (0'0) promote ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by osd.285.0:7386 2014-08-13 15:32:29.962551 2014-08-13 17:38:02.934848 7fe9bae887a0 20 read_log 194422'17632 (194422'17631) modify ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by client.1148203.0:3725 2014-08-13 15:32:29.953694 2014-08-13 17:38:02.934860 7fe9bae887a0 20 read_log 194422'17633 (0'0) promote a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by osd.285.0:7406 2014-08-13 15:32:34.472931 2014-08-13 17:38:02.934872 7fe9bae887a0 20 read_log 194422'17634 (194422'17633) modify a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by client.1148203.0:4904 2014-08-13 15:32:34.465187 2014-08-13 17:38:02.934912 7fe9bae887a0 20 read_log 194422'17635 (0'0) promote 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by osd.285.0:7458 2014-08-13 15:32:46.004223 2014-08-13 17:38:02.934929 7fe9bae887a0 20 read_log 194422'17636 (194422'17635) modify 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by client.1148203.0:7076 2014-08-13 15:32:45.995195 2014-08-13 17:38:02.934941 7fe9bae887a0 20 read_log 194422'17637 (0'0) promote 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by osd.285.0:7460 2014-08-13 15:32:46.067143 2014-08-13 17:38:02.934953 7fe9bae887a0 20 read_log 194422'17638 (194422'17637) modify 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by client.1148203.0:7087 2014-08-13 15:32:46.058576 2014-08-13 17:38:02.934966 7fe9bae887a0 20 read_log 194422'17639 (0'0) promote fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by osd.285.0:7495 2014-08-13 15:32:55.403501 2014-08-13 17:38:02.934978 7fe9bae887a0 20 read_log 194422'17640 (194422'17639) modify fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by client.1148203.0:8541 2014-08-13 15:32:55.392841 2014-08-13 17:38:02.934990 7fe9bae887a0 20 read_log 194422'17641 (0'0) promote 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by osd.285.0:7517 2014-08-13 15:33:02.095563 2014-08-13 17:38:02.935002 7fe9bae887a0 20 read_log 194422'17642 (194422'17641) modify 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by client.1148203.0:9258 2014-08-13 15:33:02.084968 2014-08-13 17:38:02.935019 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.935407 7fe9bae887a0 10 read_log checking for missing items over interval (194370'17543,194422'17642] 2014-08-13 17:38:02.935454 7fe9bae887a0 15 read_log missing 194422'17642 (194422'17641) modify 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by client.1148203.0:9258 2014-08-13 15:33:02.084968 2014-08-13 17:38:02.935493 7fe9bae887a0 15 read_log missing 194422'17640 (194422'17639) modify fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by client.1148203.0:8541 2014-08-13 15:32:55.392841 2014-08-13 17:38:02.935531 7fe9bae887a0 15 read_log missing 194422'17638 (194422'17637) modify 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by client.1148203.0:7087 2014-08-13 15:32:46.058576 2014-08-13 17:38:02.935568 7fe9bae887a0 15 read_log missing 194422'17636 (194422'17635) modify 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by client.1148203.0:7076 2014-08-13 15:32:45.995195 2014-08-13 17:38:02.935604 7fe9bae887a0 15 read_log missing 194422'17634 (194422'17633) modify a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by client.1148203.0:4904 2014-08-13 15:32:34.465187 2014-08-13 17:38:02.935639 7fe9bae887a0 15 read_log missing 194422'17632 (194422'17631) modify ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by client.1148203.0:3725 2014-08-13 15:32:29.953694 2014-08-13 17:38:02.935673 7fe9bae887a0 15 read_log missing 194422'17630 (194422'17629) modify dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by client.1148203.0:1378 2014-08-13 15:32:20.782756 2014-08-13 17:38:02.935710 7fe9bae887a0 15 read_log missing 194416'17628 (194416'17627) modify 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by client.2441958.0:17781 2014-08-13 15:29:58.341408 2014-08-13 17:38:02.935748 7fe9bae887a0 15 read_log missing 194416'17626 (194416'17625) modify a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by client.2441958.0:17425 2014-08-13 15:29:53.358926 2014-08-13 17:38:02.935785 7fe9bae887a0 15 read_log missing 194416'17624 (194416'17623) modify 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by client.2441958.0:16630 2014-08-13 15:29:42.041487 2014-08-13 17:38:02.935820 7fe9bae887a0 15 read_log missing 194416'17622 (194416'17621) modify 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by client.2441958.0:16606 2014-08-13 15:29:41.727382 2014-08-13 17:38:02.935854 7fe9bae887a0 15 read_log missing 194416'17620 (194416'17619) modify 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by client.2441958.0:12344 2014-08-13 15:29:19.093009 2014-08-13 17:38:02.935890 7fe9bae887a0 15 read_log missing 194416'17618 (194416'17617) modify d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by client.2441958.0:12034 2014-08-13 15:29:17.749689 2014-08-13 17:38:02.935924 7fe9bae887a0 15 read_log missing 194416'17616 (194416'17615) modify 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by client.2441958.0:11725 2014-08-13 15:29:16.417986 2014-08-13 17:38:02.935958 7fe9bae887a0 15 read_log missing 194416'17614 (194416'17613) modify c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by client.2441958.0:10475 2014-08-13 15:29:11.336396 2014-08-13 17:38:02.935995 7fe9bae887a0 15 read_log missing 194416'17612 (194416'17611) modify 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by client.2441958.0:9727 2014-08-13 15:29:08.241468 2014-08-13 17:38:02.936032 7fe9bae887a0 15 read_log missing 194416'17610 (194416'17609) modify 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by client.2441958.0:9635 2014-08-13 15:29:07.892021 2014-08-13 17:38:02.936069 7fe9bae887a0 15 read_log missing 194416'17608 (194416'17607) modify 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by client.2441958.0:9022 2014-08-13 15:29:05.558364 2014-08-13 17:38:02.936107 7fe9bae887a0 15 read_log missing 194416'17606 (194416'17605) modify 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by client.2441958.0:7371 2014-08-13 15:28:59.157077 2014-08-13 17:38:02.936143 7fe9bae887a0 15 read_log missing 194416'17604 (194416'17603) modify 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by client.2441958.0:6648 2014-08-13 15:28:56.462954 2014-08-13 17:38:02.936186 7fe9bae887a0 15 read_log missing 194416'17602 (194416'17601) modify c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by client.2441958.0:5594 2014-08-13 15:28:52.148545 2014-08-13 17:38:02.936223 7fe9bae887a0 15 read_log missing 194416'17600 (194416'17599) modify ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by client.2441958.0:5497 2014-08-13 15:28:51.775367 2014-08-13 17:38:02.936259 7fe9bae887a0 15 read_log missing 194416'17598 (194416'17597) modify 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by client.2441958.0:4705 2014-08-13 15:28:48.770243 2014-08-13 17:38:02.936295 7fe9bae887a0 15 read_log missing 194416'17596 (194416'17595) modify 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by client.2441958.0:2891 2014-08-13 15:28:41.661418 2014-08-13 17:38:02.936331 7fe9bae887a0 15 read_log missing 194416'17594 (194416'17593) modify 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by client.2441958.0:2714 2014-08-13 15:28:40.995272 2014-08-13 17:38:02.936368 7fe9bae887a0 15 read_log missing 194416'17592 (194416'17591) modify 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by client.2441958.0:945 2014-08-13 15:28:34.269526 2014-08-13 17:38:02.936405 7fe9bae887a0 15 read_log missing 194416'17590 (194416'17589) modify 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by client.2441958.0:4 2014-08-13 15:28:29.919716 2014-08-13 17:38:02.936442 7fe9bae887a0 15 read_log missing 194393'17588 (194393'17587) modify 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by client.1064422.0:233 2014-08-12 20:52:33.470274 2014-08-13 17:38:02.936479 7fe9bae887a0 15 read_log missing 194393'17586 (194393'17585) modify 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by client.1064182.0:434 2014-08-12 20:49:27.672142 2014-08-13 17:38:02.936516 7fe9bae887a0 15 read_log missing 194393'17584 (194393'17583) modify c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by client.1064182.0:200 2014-08-12 20:49:26.420766 2014-08-13 17:38:02.936554 7fe9bae887a0 15 read_log missing 194383'17582 (194383'17581) modify 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by client.2231190.0:7636 2014-08-12 20:34:32.449118 2014-08-13 17:38:02.936591 7fe9bae887a0 15 read_log missing 194383'17580 (194383'17579) modify a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by client.2231190.0:6767 2014-08-12 20:34:08.797961 2014-08-13 17:38:02.936629 7fe9bae887a0 15 read_log missing 194383'17578 (194383'17577) modify e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by client.2231190.0:5965 2014-08-12 20:34:01.231283 2014-08-13 17:38:02.936666 7fe9bae887a0 15 read_log missing 194380'17576 (194380'17575) modify 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by client.2231190.0:5227 2014-08-12 20:33:26.151470 2014-08-13 17:38:02.936703 7fe9bae887a0 15 read_log missing 194380'17574 (194380'17573) modify 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by client.2231190.0:4956 2014-08-12 20:33:22.387767 2014-08-13 17:38:02.936740 7fe9bae887a0 15 read_log missing 194377'17572 (194377'17571) modify 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by client.2231190.0:2270 2014-08-12 20:33:06.340532 2014-08-13 17:38:02.936777 7fe9bae887a0 15 read_log missing 194377'17570 (194377'17569) modify 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by client.2231190.0:1297 2014-08-12 20:33:02.426220 2014-08-13 17:38:02.936815 7fe9bae887a0 15 read_log missing 194377'17568 (194377'17567) modify baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by client.2231190.0:750 2014-08-12 20:33:00.363477 2014-08-13 17:38:02.936852 7fe9bae887a0 15 read_log missing 194370'17566 (194370'17565) modify 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by client.1508531.0:9270 2014-08-12 20:30:54.943986 2014-08-13 17:38:02.936889 7fe9bae887a0 15 read_log missing 194370'17564 (194370'17563) modify 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by client.1508531.0:6777 2014-08-12 20:30:41.243264 2014-08-13 17:38:02.936926 7fe9bae887a0 15 read_log missing 194370'17562 (194370'17561) modify 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by client.1508531.0:6493 2014-08-12 20:30:39.927587 2014-08-13 17:38:02.936964 7fe9bae887a0 15 read_log missing 194370'17560 (194370'17559) modify 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by client.1508531.0:6359 2014-08-12 20:30:39.285755 2014-08-13 17:38:02.937001 7fe9bae887a0 15 read_log missing 194370'17558 (194370'17557) modify d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by client.1508531.0:6043 2014-08-12 20:30:37.924085 2014-08-13 17:38:02.937042 7fe9bae887a0 15 read_log missing 194370'17556 (194370'17555) modify b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by client.1508531.0:5182 2014-08-12 20:30:33.991823 2014-08-13 17:38:02.937080 7fe9bae887a0 15 read_log missing 194370'17554 (194370'17553) modify 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by client.1508531.0:4986 2014-08-12 20:30:33.168763 2014-08-13 17:38:02.937117 7fe9bae887a0 15 read_log missing 194370'17552 (194370'17551) modify 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by client.1508531.0:3852 2014-08-12 20:30:28.748728 2014-08-13 17:38:02.937161 7fe9bae887a0 15 read_log missing 194370'17550 (194370'17549) modify d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by client.1508531.0:3362 2014-08-12 20:30:26.866144 2014-08-13 17:38:02.937200 7fe9bae887a0 15 read_log missing 194370'17548 (194370'17547) modify 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by client.1508531.0:1945 2014-08-12 20:30:21.623294 2014-08-13 17:38:02.937238 7fe9bae887a0 15 read_log missing 194370'17546 (194370'17545) modify 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by client.1508531.0:771 2014-08-12 20:30:17.293344 2014-08-13 17:38:02.937275 7fe9bae887a0 15 read_log missing 194370'17544 (194370'17543) modify 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by client.1508531.0:743 2014-08-12 20:30:17.192859 2014-08-13 17:38:02.937287 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.937312 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive m=50] handle_loaded 2014-08-13 17:38:02.937329 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.012482 0 0.000000 2014-08-13 17:38:02.937341 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 17:38:02.937349 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=0 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194477 2014-08-13 17:38:02.937359 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log((192718'14692,194422'17642], crt=194422'17642) 2014-08-13 17:38:02.937369 7fe9bae887a0 10 osd.4 194478 pgid 24.1f7 coll 24.1f7_head 2014-08-13 17:38:02.937482 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.1f7 2014-08-13 17:38:02.937494 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.937528 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1f7(unlocked)] enter Initial 2014-08-13 17:38:02.937540 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1f7(unlocked)] enter NotTrimming 2014-08-13 17:38:02.937668 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.937791 7fe9bae887a0 20 read_log 192718'12873 (192718'12872) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:3 2014-08-06 15:24:47.114884 2014-08-13 17:38:02.937812 7fe9bae887a0 20 read_log 192718'12874 (192718'12873) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:4 2014-08-06 15:24:47.424123 2014-08-13 17:38:02.937825 7fe9bae887a0 20 read_log 192718'12875 (192718'12874) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:5 2014-08-06 15:24:47.595699 2014-08-13 17:38:02.937837 7fe9bae887a0 20 read_log 192718'12876 (192718'12875) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:6 2014-08-06 15:24:48.028351 2014-08-13 17:38:02.937849 7fe9bae887a0 20 read_log 192718'12877 (192718'12876) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:7 2014-08-06 15:24:48.614236 2014-08-13 17:38:02.937861 7fe9bae887a0 20 read_log 192718'12878 (192718'12877) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:8 2014-08-06 15:24:49.060913 2014-08-13 17:38:02.937904 7fe9bae887a0 20 read_log 192718'12879 (192718'12878) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:9 2014-08-06 15:24:49.507639 2014-08-13 17:38:02.937918 7fe9bae887a0 20 read_log 192718'12880 (192718'12879) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:10 2014-08-06 15:24:50.158398 2014-08-13 17:38:02.937930 7fe9bae887a0 20 read_log 192718'12881 (192718'12880) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:11 2014-08-06 15:24:50.421065 2014-08-13 17:38:02.937942 7fe9bae887a0 20 read_log 192718'12882 (192718'12881) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:12 2014-08-06 15:24:50.657631 2014-08-13 17:38:02.937954 7fe9bae887a0 20 read_log 192718'12883 (192718'12882) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:13 2014-08-06 15:24:50.836072 2014-08-13 17:38:02.937966 7fe9bae887a0 20 read_log 192718'12884 (192718'12883) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:14 2014-08-06 15:24:50.900548 2014-08-13 17:38:02.937977 7fe9bae887a0 20 read_log 192718'12885 (192718'12884) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:15 2014-08-06 15:24:50.929959 2014-08-13 17:38:02.937989 7fe9bae887a0 20 read_log 192718'12886 (192718'12885) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:16 2014-08-06 15:24:50.960414 2014-08-13 17:38:02.938001 7fe9bae887a0 20 read_log 192718'12887 (192718'12886) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:17 2014-08-06 15:24:50.985093 2014-08-13 17:38:02.938013 7fe9bae887a0 20 read_log 192718'12888 (192718'12887) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:18 2014-08-06 15:24:51.010332 2014-08-13 17:38:02.938024 7fe9bae887a0 20 read_log 192718'12889 (192718'12888) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:19 2014-08-06 15:24:51.036022 2014-08-13 17:38:02.938038 7fe9bae887a0 20 read_log 192718'12890 (192718'12889) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:20 2014-08-06 15:24:51.060228 2014-08-13 17:38:02.938050 7fe9bae887a0 20 read_log 192718'12891 (192718'12890) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:21 2014-08-06 15:24:51.084232 2014-08-13 17:38:02.938062 7fe9bae887a0 20 read_log 192718'12892 (192718'12891) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:22 2014-08-06 15:24:51.108502 2014-08-13 17:38:02.938074 7fe9bae887a0 20 read_log 192718'12893 (192718'12892) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:23 2014-08-06 15:24:51.133094 2014-08-13 17:38:02.938086 7fe9bae887a0 20 read_log 192718'12894 (192718'12893) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:24 2014-08-06 15:24:51.156710 2014-08-13 17:38:02.938098 7fe9bae887a0 20 read_log 192718'12895 (192718'12894) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:25 2014-08-06 15:24:51.230774 2014-08-13 17:38:02.938109 7fe9bae887a0 20 read_log 192718'12896 (192718'12895) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:26 2014-08-06 15:24:51.255363 2014-08-13 17:38:02.938121 7fe9bae887a0 20 read_log 192718'12897 (192718'12896) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:27 2014-08-06 15:24:51.279009 2014-08-13 17:38:02.938133 7fe9bae887a0 20 read_log 192718'12898 (192718'12897) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:28 2014-08-06 15:24:51.302621 2014-08-13 17:38:02.938144 7fe9bae887a0 20 read_log 192718'12899 (192718'12898) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:29 2014-08-06 15:24:51.326614 2014-08-13 17:38:02.938163 7fe9bae887a0 20 read_log 192718'12900 (192718'12899) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:30 2014-08-06 15:24:51.350464 2014-08-13 17:38:02.938175 7fe9bae887a0 20 read_log 192718'12901 (192718'12900) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:31 2014-08-06 15:24:51.389035 2014-08-13 17:38:02.938187 7fe9bae887a0 20 read_log 192718'12902 (192718'12901) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:32 2014-08-06 15:24:51.414094 2014-08-13 17:38:02.938210 7fe9bae887a0 20 read_log 192718'12903 (192718'12902) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:33 2014-08-06 15:24:51.437762 2014-08-13 17:38:02.938222 7fe9bae887a0 20 read_log 192718'12904 (192718'12903) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:34 2014-08-06 15:24:51.462561 2014-08-13 17:38:02.938234 7fe9bae887a0 20 read_log 192718'12905 (192718'12904) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:35 2014-08-06 15:24:51.573919 2014-08-13 17:38:02.938246 7fe9bae887a0 20 read_log 192718'12906 (192718'12905) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:36 2014-08-06 15:24:51.599477 2014-08-13 17:38:02.938258 7fe9bae887a0 20 read_log 192718'12907 (192718'12906) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:37 2014-08-06 15:24:51.643384 2014-08-13 17:38:02.938270 7fe9bae887a0 20 read_log 192718'12908 (192718'12907) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:38 2014-08-06 15:24:51.687272 2014-08-13 17:38:02.938282 7fe9bae887a0 20 read_log 192718'12909 (192718'12908) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:39 2014-08-06 15:24:51.731104 2014-08-13 17:38:02.938294 7fe9bae887a0 20 read_log 192718'12910 (192718'12909) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:40 2014-08-06 15:24:51.773411 2014-08-13 17:38:02.938305 7fe9bae887a0 20 read_log 192718'12911 (192718'12910) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:41 2014-08-06 15:24:51.818387 2014-08-13 17:38:02.938317 7fe9bae887a0 20 read_log 192718'12912 (192718'12911) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:42 2014-08-06 15:24:51.862979 2014-08-13 17:38:02.938329 7fe9bae887a0 20 read_log 192718'12913 (192718'12912) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:43 2014-08-06 15:24:51.907051 2014-08-13 17:38:02.938341 7fe9bae887a0 20 read_log 192718'12914 (192718'12913) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:44 2014-08-06 15:24:52.052791 2014-08-13 17:38:02.938352 7fe9bae887a0 20 read_log 192718'12915 (192718'12914) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:45 2014-08-06 15:24:52.104726 2014-08-13 17:38:02.938364 7fe9bae887a0 20 read_log 192718'12916 (192718'12915) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:46 2014-08-06 15:24:52.156812 2014-08-13 17:38:02.938376 7fe9bae887a0 20 read_log 192718'12917 (192718'12916) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:47 2014-08-06 15:24:52.212763 2014-08-13 17:38:02.938387 7fe9bae887a0 20 read_log 192718'12918 (192718'12917) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:48 2014-08-06 15:24:52.261099 2014-08-13 17:38:02.938399 7fe9bae887a0 20 read_log 192718'12919 (192718'12918) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:49 2014-08-06 15:24:52.312303 2014-08-13 17:38:02.938411 7fe9bae887a0 20 read_log 192718'12920 (192718'12919) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:50 2014-08-06 15:24:52.354953 2014-08-13 17:38:02.938423 7fe9bae887a0 20 read_log 192718'12921 (192718'12920) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:51 2014-08-06 15:24:52.407051 2014-08-13 17:38:02.938434 7fe9bae887a0 20 read_log 192718'12922 (192718'12921) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:52 2014-08-06 15:24:52.450846 2014-08-13 17:38:02.938446 7fe9bae887a0 20 read_log 192718'12923 (192718'12922) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:53 2014-08-06 15:24:52.502022 2014-08-13 17:38:02.938458 7fe9bae887a0 20 read_log 192718'12924 (192718'12923) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:54 2014-08-06 15:24:52.541358 2014-08-13 17:38:02.938469 7fe9bae887a0 20 read_log 192718'12925 (192718'12924) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:55 2014-08-06 15:24:52.572327 2014-08-13 17:38:02.938481 7fe9bae887a0 20 read_log 192718'12926 (192718'12925) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:56 2014-08-06 15:24:52.602817 2014-08-13 17:38:02.938499 7fe9bae887a0 20 read_log 192718'12927 (192718'12926) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:57 2014-08-06 15:24:52.645849 2014-08-13 17:38:02.938512 7fe9bae887a0 20 read_log 192718'12928 (192718'12927) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:58 2014-08-06 15:24:52.696786 2014-08-13 17:38:02.938523 7fe9bae887a0 20 read_log 192718'12929 (192718'12928) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:59 2014-08-06 15:24:52.739794 2014-08-13 17:38:02.938535 7fe9bae887a0 20 read_log 192718'12930 (192718'12929) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:60 2014-08-06 15:24:52.781021 2014-08-13 17:38:02.938547 7fe9bae887a0 20 read_log 192718'12931 (192718'12930) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:61 2014-08-06 15:24:52.823192 2014-08-13 17:38:02.938559 7fe9bae887a0 20 read_log 192718'12932 (192718'12931) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:62 2014-08-06 15:24:52.868251 2014-08-13 17:38:02.938570 7fe9bae887a0 20 read_log 192718'12933 (192718'12932) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:63 2014-08-06 15:24:52.910387 2014-08-13 17:38:02.938582 7fe9bae887a0 20 read_log 192718'12934 (192718'12933) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:64 2014-08-06 15:24:52.953257 2014-08-13 17:38:02.938593 7fe9bae887a0 20 read_log 192718'12935 (192718'12934) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:65 2014-08-06 15:24:52.994503 2014-08-13 17:38:02.938607 7fe9bae887a0 20 read_log 192718'12936 (192718'12935) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:66 2014-08-06 15:24:53.019589 2014-08-13 17:38:02.938619 7fe9bae887a0 20 read_log 192718'12937 (192718'12936) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:67 2014-08-06 15:24:53.052020 2014-08-13 17:38:02.938631 7fe9bae887a0 20 read_log 192718'12938 (192718'12937) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:68 2014-08-06 15:24:53.077756 2014-08-13 17:38:02.938643 7fe9bae887a0 20 read_log 192718'12939 (192718'12938) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:69 2014-08-06 15:24:53.109563 2014-08-13 17:38:02.938655 7fe9bae887a0 20 read_log 192718'12940 (192718'12939) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:70 2014-08-06 15:24:53.140771 2014-08-13 17:38:02.938666 7fe9bae887a0 20 read_log 192718'12941 (192718'12940) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:71 2014-08-06 15:24:53.166270 2014-08-13 17:38:02.938678 7fe9bae887a0 20 read_log 192718'12942 (192718'12941) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:72 2014-08-06 15:24:53.215080 2014-08-13 17:38:02.938690 7fe9bae887a0 20 read_log 192718'12943 (192718'12942) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:73 2014-08-06 15:24:53.245523 2014-08-13 17:38:02.938701 7fe9bae887a0 20 read_log 192718'12944 (192718'12943) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:74 2014-08-06 15:24:53.274287 2014-08-13 17:38:02.938713 7fe9bae887a0 20 read_log 192718'12945 (192718'12944) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:75 2014-08-06 15:24:53.302027 2014-08-13 17:38:02.938725 7fe9bae887a0 20 read_log 192718'12946 (192718'12945) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:76 2014-08-06 15:24:53.335025 2014-08-13 17:38:02.938736 7fe9bae887a0 20 read_log 192718'12947 (192718'12946) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:77 2014-08-06 15:24:53.373354 2014-08-13 17:38:02.938775 7fe9bae887a0 20 read_log 192718'12948 (192718'12947) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:78 2014-08-06 15:24:53.401568 2014-08-13 17:38:02.938789 7fe9bae887a0 20 read_log 192718'12949 (192718'12948) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:79 2014-08-06 15:24:53.426974 2014-08-13 17:38:02.938800 7fe9bae887a0 20 read_log 192718'12950 (192718'12949) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:80 2014-08-06 15:24:53.458959 2014-08-13 17:38:02.938820 7fe9bae887a0 20 read_log 192718'12951 (192718'12950) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:81 2014-08-06 15:24:53.483675 2014-08-13 17:38:02.938832 7fe9bae887a0 20 read_log 192718'12952 (192718'12951) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:82 2014-08-06 15:24:53.510015 2014-08-13 17:38:02.938844 7fe9bae887a0 20 read_log 192718'12953 (192718'12952) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:83 2014-08-06 15:24:53.533872 2014-08-13 17:38:02.938856 7fe9bae887a0 20 read_log 192718'12954 (192718'12953) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:84 2014-08-06 15:24:53.558366 2014-08-13 17:38:02.938868 7fe9bae887a0 20 read_log 192718'12955 (192718'12954) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:85 2014-08-06 15:24:53.591179 2014-08-13 17:38:02.938880 7fe9bae887a0 20 read_log 192718'12956 (192718'12955) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:86 2014-08-06 15:24:53.622795 2014-08-13 17:38:02.938891 7fe9bae887a0 20 read_log 192718'12957 (192718'12956) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:87 2014-08-06 15:24:53.658412 2014-08-13 17:38:02.938903 7fe9bae887a0 20 read_log 192718'12958 (192718'12957) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:88 2014-08-06 15:24:53.683738 2014-08-13 17:38:02.938915 7fe9bae887a0 20 read_log 192718'12959 (192718'12958) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:89 2014-08-06 15:24:53.712818 2014-08-13 17:38:02.938926 7fe9bae887a0 20 read_log 192718'12960 (192718'12959) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:90 2014-08-06 15:24:53.744611 2014-08-13 17:38:02.938938 7fe9bae887a0 20 read_log 192718'12961 (192718'12960) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:91 2014-08-06 15:24:53.766915 2014-08-13 17:38:02.938950 7fe9bae887a0 20 read_log 192718'12962 (192718'12961) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:92 2014-08-06 15:24:53.802135 2014-08-13 17:38:02.938962 7fe9bae887a0 20 read_log 192718'12963 (192718'12962) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:93 2014-08-06 15:24:53.837191 2014-08-13 17:38:02.938974 7fe9bae887a0 20 read_log 192718'12964 (192718'12963) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:94 2014-08-06 15:24:53.899762 2014-08-13 17:38:02.938988 7fe9bae887a0 20 read_log 192718'12965 (192718'12964) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:95 2014-08-06 15:24:53.932235 2014-08-13 17:38:02.939000 7fe9bae887a0 20 read_log 192718'12966 (192718'12965) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:96 2014-08-06 15:24:53.962983 2014-08-13 17:38:02.939012 7fe9bae887a0 20 read_log 192718'12967 (192718'12966) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:97 2014-08-06 15:24:53.995196 2014-08-13 17:38:02.939024 7fe9bae887a0 20 read_log 192718'12968 (192718'12967) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:98 2014-08-06 15:24:54.034257 2014-08-13 17:38:02.939036 7fe9bae887a0 20 read_log 192718'12969 (192718'12968) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:99 2014-08-06 15:24:54.059904 2014-08-13 17:38:02.939048 7fe9bae887a0 20 read_log 192718'12970 (192718'12969) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:100 2014-08-06 15:24:54.096752 2014-08-13 17:38:02.939060 7fe9bae887a0 20 read_log 192718'12971 (192718'12970) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:101 2014-08-06 15:24:54.124311 2014-08-13 17:38:02.939090 7fe9bae887a0 20 read_log 192718'12972 (192718'12971) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:102 2014-08-06 15:24:54.150508 2014-08-13 17:38:02.939103 7fe9bae887a0 20 read_log 192718'12973 (192718'12972) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:103 2014-08-06 15:24:54.180616 2014-08-13 17:38:02.939115 7fe9bae887a0 20 read_log 192718'12974 (192718'12973) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:104 2014-08-06 15:24:54.205055 2014-08-13 17:38:02.939137 7fe9bae887a0 20 read_log 192718'12975 (192718'12974) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:105 2014-08-06 15:24:54.239803 2014-08-13 17:38:02.939151 7fe9bae887a0 20 read_log 192718'12976 (192718'12975) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:106 2014-08-06 15:24:54.267614 2014-08-13 17:38:02.939171 7fe9bae887a0 20 read_log 192718'12977 (192718'12976) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:107 2014-08-06 15:24:54.296078 2014-08-13 17:38:02.939183 7fe9bae887a0 20 read_log 192718'12978 (192718'12977) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:108 2014-08-06 15:24:54.321972 2014-08-13 17:38:02.939195 7fe9bae887a0 20 read_log 192718'12979 (192718'12978) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:109 2014-08-06 15:24:54.353636 2014-08-13 17:38:02.939207 7fe9bae887a0 20 read_log 192718'12980 (192718'12979) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:110 2014-08-06 15:24:54.386511 2014-08-13 17:38:02.939220 7fe9bae887a0 20 read_log 192718'12981 (192718'12980) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:111 2014-08-06 15:24:54.410818 2014-08-13 17:38:02.939232 7fe9bae887a0 20 read_log 192718'12982 (192718'12981) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:112 2014-08-06 15:24:54.452686 2014-08-13 17:38:02.939244 7fe9bae887a0 20 read_log 192718'12983 (192718'12982) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:113 2014-08-06 15:24:54.479632 2014-08-13 17:38:02.939256 7fe9bae887a0 20 read_log 192718'12984 (192718'12983) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:114 2014-08-06 15:24:54.537663 2014-08-13 17:38:02.939267 7fe9bae887a0 20 read_log 192718'12985 (192718'12984) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:115 2014-08-06 15:24:54.570157 2014-08-13 17:38:02.939279 7fe9bae887a0 20 read_log 192718'12986 (192718'12985) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:116 2014-08-06 15:24:54.598680 2014-08-13 17:38:02.939291 7fe9bae887a0 20 read_log 192718'12987 (192718'12986) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:117 2014-08-06 15:24:54.639901 2014-08-13 17:38:02.939304 7fe9bae887a0 20 read_log 192718'12988 (192718'12987) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:118 2014-08-06 15:24:54.667717 2014-08-13 17:38:02.939316 7fe9bae887a0 20 read_log 192718'12989 (192718'12988) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:119 2014-08-06 15:24:54.731431 2014-08-13 17:38:02.939328 7fe9bae887a0 20 read_log 192718'12990 (192718'12989) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:120 2014-08-06 15:24:54.763905 2014-08-13 17:38:02.939340 7fe9bae887a0 20 read_log 192718'12991 (192718'12990) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:121 2014-08-06 15:24:54.784962 2014-08-13 17:38:02.939352 7fe9bae887a0 20 read_log 192718'12992 (192718'12991) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:122 2014-08-06 15:24:54.824008 2014-08-13 17:38:02.939364 7fe9bae887a0 20 read_log 192718'12993 (192718'12992) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:123 2014-08-06 15:24:54.847003 2014-08-13 17:38:02.939377 7fe9bae887a0 20 read_log 192718'12994 (192718'12993) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:124 2014-08-06 15:24:54.875523 2014-08-13 17:38:02.939389 7fe9bae887a0 20 read_log 192718'12995 (192718'12994) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:125 2014-08-06 15:24:55.241111 2014-08-13 17:38:02.939401 7fe9bae887a0 20 read_log 192718'12996 (192718'12995) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:126 2014-08-06 15:24:55.337976 2014-08-13 17:38:02.939412 7fe9bae887a0 20 read_log 192718'12997 (192718'12996) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:127 2014-08-06 15:24:55.508412 2014-08-13 17:38:02.939424 7fe9bae887a0 20 read_log 192718'12998 (192718'12997) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:128 2014-08-06 15:24:55.542707 2014-08-13 17:38:02.939442 7fe9bae887a0 20 read_log 192718'12999 (192718'12998) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:129 2014-08-06 15:24:55.577165 2014-08-13 17:38:02.939457 7fe9bae887a0 20 read_log 192718'13000 (192718'12999) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:130 2014-08-06 15:24:55.607515 2014-08-13 17:38:02.939469 7fe9bae887a0 20 read_log 192718'13001 (192718'13000) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:131 2014-08-06 15:24:55.634278 2014-08-13 17:38:02.939488 7fe9bae887a0 20 read_log 192718'13002 (192718'13001) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:132 2014-08-06 15:24:55.805528 2014-08-13 17:38:02.939500 7fe9bae887a0 20 read_log 192718'13003 (192718'13002) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:133 2014-08-06 15:24:55.947411 2014-08-13 17:38:02.939512 7fe9bae887a0 20 read_log 192718'13004 (192718'13003) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:134 2014-08-06 15:24:55.972566 2014-08-13 17:38:02.939523 7fe9bae887a0 20 read_log 192718'13005 (192718'13004) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:135 2014-08-06 15:24:56.421414 2014-08-13 17:38:02.939535 7fe9bae887a0 20 read_log 192718'13006 (192718'13005) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:136 2014-08-06 15:24:56.451611 2014-08-13 17:38:02.939547 7fe9bae887a0 20 read_log 192718'13007 (192718'13006) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:137 2014-08-06 15:24:56.702159 2014-08-13 17:38:02.939559 7fe9bae887a0 20 read_log 192718'13008 (192718'13007) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:138 2014-08-06 15:24:56.737239 2014-08-13 17:38:02.939571 7fe9bae887a0 20 read_log 192718'13009 (192718'13008) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:139 2014-08-06 15:24:56.766385 2014-08-13 17:38:02.939582 7fe9bae887a0 20 read_log 192718'13010 (192718'13009) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:140 2014-08-06 15:24:56.859552 2014-08-13 17:38:02.939594 7fe9bae887a0 20 read_log 192718'13011 (192718'13010) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:141 2014-08-06 15:24:56.969870 2014-08-13 17:38:02.939606 7fe9bae887a0 20 read_log 192718'13012 (192718'13011) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:142 2014-08-06 15:24:57.018994 2014-08-13 17:38:02.939618 7fe9bae887a0 20 read_log 192718'13013 (192718'13012) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:143 2014-08-06 15:24:57.305959 2014-08-13 17:38:02.939629 7fe9bae887a0 20 read_log 192718'13014 (192718'13013) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:144 2014-08-06 15:24:57.360675 2014-08-13 17:38:02.939641 7fe9bae887a0 20 read_log 192718'13015 (192718'13014) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:145 2014-08-06 15:24:57.679353 2014-08-13 17:38:02.939653 7fe9bae887a0 20 read_log 192718'13016 (192718'13015) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:146 2014-08-06 15:24:57.707449 2014-08-13 17:38:02.939664 7fe9bae887a0 20 read_log 192718'13017 (192718'13016) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:147 2014-08-06 15:24:57.748457 2014-08-13 17:38:02.939676 7fe9bae887a0 20 read_log 192718'13018 (192718'13017) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:148 2014-08-06 15:24:57.843826 2014-08-13 17:38:02.939688 7fe9bae887a0 20 read_log 192718'13019 (192718'13018) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:149 2014-08-06 15:24:57.880124 2014-08-13 17:38:02.939700 7fe9bae887a0 20 read_log 192718'13020 (192718'13019) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:150 2014-08-06 15:24:58.017593 2014-08-13 17:38:02.939711 7fe9bae887a0 20 read_log 192718'13021 (192718'13020) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:151 2014-08-06 15:24:58.043514 2014-08-13 17:38:02.939723 7fe9bae887a0 20 read_log 192718'13022 (192718'13021) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:152 2014-08-06 15:24:58.122123 2014-08-13 17:38:02.939760 7fe9bae887a0 20 read_log 192718'13023 (192718'13022) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:153 2014-08-06 15:24:58.238607 2014-08-13 17:38:02.939773 7fe9bae887a0 20 read_log 192718'13024 (192718'13023) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:154 2014-08-06 15:24:58.261422 2014-08-13 17:38:02.939785 7fe9bae887a0 20 read_log 192718'13025 (192718'13024) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:155 2014-08-06 15:24:58.954801 2014-08-13 17:38:02.939799 7fe9bae887a0 20 read_log 192718'13026 (192718'13025) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:156 2014-08-06 15:24:59.190463 2014-08-13 17:38:02.939811 7fe9bae887a0 20 read_log 192718'13027 (192718'13026) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:157 2014-08-06 15:24:59.237926 2014-08-13 17:38:02.939823 7fe9bae887a0 20 read_log 192718'13028 (192718'13027) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:158 2014-08-06 15:24:59.262128 2014-08-13 17:38:02.939835 7fe9bae887a0 20 read_log 192718'13029 (192718'13028) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:159 2014-08-06 15:24:59.406584 2014-08-13 17:38:02.939847 7fe9bae887a0 20 read_log 192718'13030 (192718'13029) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:160 2014-08-06 15:24:59.438109 2014-08-13 17:38:02.939859 7fe9bae887a0 20 read_log 192718'13031 (192718'13030) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:161 2014-08-06 15:24:59.466531 2014-08-13 17:38:02.939870 7fe9bae887a0 20 read_log 192718'13032 (192718'13031) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:162 2014-08-06 15:24:59.645755 2014-08-13 17:38:02.939885 7fe9bae887a0 20 read_log 192718'13033 (192718'13032) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:163 2014-08-06 15:24:59.674300 2014-08-13 17:38:02.939897 7fe9bae887a0 20 read_log 192718'13034 (192718'13033) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:164 2014-08-06 15:24:59.706738 2014-08-13 17:38:02.939910 7fe9bae887a0 20 read_log 192718'13035 (192718'13034) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:165 2014-08-06 15:25:00.385904 2014-08-13 17:38:02.939922 7fe9bae887a0 20 read_log 192718'13036 (192718'13035) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:166 2014-08-06 15:25:00.412808 2014-08-13 17:38:02.939934 7fe9bae887a0 20 read_log 192718'13037 (192718'13036) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:167 2014-08-06 15:25:00.466025 2014-08-13 17:38:02.939947 7fe9bae887a0 20 read_log 192718'13038 (192718'13037) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:168 2014-08-06 15:25:00.637563 2014-08-13 17:38:02.939959 7fe9bae887a0 20 read_log 192718'13039 (192718'13038) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:169 2014-08-06 15:25:00.665982 2014-08-13 17:38:02.939971 7fe9bae887a0 20 read_log 192718'13040 (192718'13039) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:170 2014-08-06 15:25:00.710207 2014-08-13 17:38:02.939983 7fe9bae887a0 20 read_log 192718'13041 (192718'13040) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:171 2014-08-06 15:25:00.878477 2014-08-13 17:38:02.939995 7fe9bae887a0 20 read_log 192718'13042 (192718'13041) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:172 2014-08-06 15:25:00.916571 2014-08-13 17:38:02.940007 7fe9bae887a0 20 read_log 192718'13043 (192718'13042) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:173 2014-08-06 15:25:00.947094 2014-08-13 17:38:02.940019 7fe9bae887a0 20 read_log 192718'13044 (192718'13043) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:174 2014-08-06 15:25:01.176849 2014-08-13 17:38:02.940030 7fe9bae887a0 20 read_log 192718'13045 (192718'13044) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:175 2014-08-06 15:25:01.681116 2014-08-13 17:38:02.940042 7fe9bae887a0 20 read_log 192718'13046 (192718'13045) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:176 2014-08-06 15:25:01.710062 2014-08-13 17:38:02.940064 7fe9bae887a0 20 read_log 192718'13047 (192718'13046) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:177 2014-08-06 15:25:01.810633 2014-08-13 17:38:02.940077 7fe9bae887a0 20 read_log 192718'13048 (192718'13047) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:178 2014-08-06 15:25:01.928976 2014-08-13 17:38:02.940089 7fe9bae887a0 20 read_log 192718'13051 (192718'13048) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:179 2014-08-06 15:25:01.961540 2014-08-13 17:38:02.940101 7fe9bae887a0 20 read_log 192718'13052 (192718'13051) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:180 2014-08-06 15:25:02.069954 2014-08-13 17:38:02.940113 7fe9bae887a0 20 read_log 192718'13053 (192718'13052) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:181 2014-08-06 15:25:02.172992 2014-08-13 17:38:02.940125 7fe9bae887a0 20 read_log 192718'13054 (192718'13053) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:182 2014-08-06 15:25:02.204544 2014-08-13 17:38:02.940137 7fe9bae887a0 20 read_log 192718'13055 (192718'13054) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:183 2014-08-06 15:25:02.449339 2014-08-13 17:38:02.940149 7fe9bae887a0 20 read_log 192797'13316 (192718'13055) clean 30d03df7/core.29770+97+2/head//24 by osd.306.0:27365 2014-08-11 10:24:24.831086 2014-08-13 17:38:02.940166 7fe9bae887a0 20 read_log 192798'13321 (192797'13316) delete 30d03df7/core.29770+97+2/head//24 by osd.306.0:27374 2014-08-11 10:24:45.664487 2014-08-13 17:38:02.940179 7fe9bae887a0 20 read_log 192834'13631 (0'0) promote 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by osd.306.0:27889 2014-08-11 15:23:37.056859 2014-08-13 17:38:02.940191 7fe9bae887a0 20 read_log 192834'13632 (192834'13631) modify 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.933448.0:366 2014-08-11 15:23:37.049493 2014-08-13 17:38:02.940204 7fe9bae887a0 20 read_log 192834'13641 (0'0) promote e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by osd.306.0:27905 2014-08-11 15:23:39.830975 2014-08-13 17:38:02.940216 7fe9bae887a0 20 read_log 192834'13642 (192834'13641) modify e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.933448.0:1079 2014-08-11 15:23:39.822726 2014-08-13 17:38:02.940229 7fe9bae887a0 20 read_log 192834'13657 (0'0) promote c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by osd.306.0:27947 2014-08-11 15:23:44.957210 2014-08-13 17:38:02.940243 7fe9bae887a0 20 read_log 192834'13658 (192834'13657) modify c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.933448.0:2100 2014-08-11 15:23:44.951160 2014-08-13 17:38:02.940255 7fe9bae887a0 20 read_log 192834'13663 (0'0) promote 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by osd.306.0:27961 2014-08-11 15:23:46.584785 2014-08-13 17:38:02.940268 7fe9bae887a0 20 read_log 192834'13664 (192834'13663) modify 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.933448.0:2397 2014-08-11 15:23:46.573721 2014-08-13 17:38:02.940281 7fe9bae887a0 20 read_log 192834'13677 (0'0) promote d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by osd.306.0:27993 2014-08-11 15:23:55.429322 2014-08-13 17:38:02.940310 7fe9bae887a0 20 read_log 192834'13678 (192834'13677) modify d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.933448.0:4091 2014-08-11 15:23:55.421713 2014-08-13 17:38:02.940326 7fe9bae887a0 20 read_log 192834'13683 (0'0) promote 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by osd.306.0:28005 2014-08-11 15:23:56.289758 2014-08-13 17:38:02.940338 7fe9bae887a0 20 read_log 192834'13684 (192834'13683) modify 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.933448.0:4306 2014-08-11 15:23:56.281240 2014-08-13 17:38:02.940359 7fe9bae887a0 20 read_log 192834'13687 (0'0) promote 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by osd.306.0:28013 2014-08-11 15:23:58.634559 2014-08-13 17:38:02.940374 7fe9bae887a0 20 read_log 192834'13688 (192834'13687) modify 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.933448.0:4798 2014-08-11 15:23:58.627088 2014-08-13 17:38:02.940386 7fe9bae887a0 20 read_log 192834'13747 (0'0) promote 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by osd.306.0:28133 2014-08-11 15:24:12.112130 2014-08-13 17:38:02.940398 7fe9bae887a0 20 read_log 192834'13748 (192834'13747) modify 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.933448.0:7975 2014-08-11 15:24:12.104539 2014-08-13 17:38:02.940411 7fe9bae887a0 20 read_log 192834'13751 (0'0) promote db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by osd.306.0:28137 2014-08-11 15:24:13.015061 2014-08-13 17:38:02.940423 7fe9bae887a0 20 read_log 192834'13752 (192834'13751) modify db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.933448.0:8175 2014-08-11 15:24:13.007610 2014-08-13 17:38:02.940436 7fe9bae887a0 20 read_log 192834'13753 (0'0) promote 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by osd.306.0:28141 2014-08-11 15:24:13.316405 2014-08-13 17:38:02.940448 7fe9bae887a0 20 read_log 192834'13754 (192834'13753) modify 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.933448.0:8255 2014-08-11 15:24:13.307187 2014-08-13 17:38:02.940460 7fe9bae887a0 20 read_log 192834'13785 (0'0) promote 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by osd.306.0:28211 2014-08-11 15:24:21.258568 2014-08-13 17:38:02.940472 7fe9bae887a0 20 read_log 192834'13786 (192834'13785) modify 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.933448.0:10022 2014-08-11 15:24:21.249744 2014-08-13 17:38:02.940484 7fe9bae887a0 20 read_log 192834'13809 (0'0) promote bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by osd.306.0:28265 2014-08-11 15:24:30.991308 2014-08-13 17:38:02.940496 7fe9bae887a0 20 read_log 192834'13810 (192834'13809) modify bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.933448.0:12075 2014-08-11 15:24:30.983429 2014-08-13 17:38:02.940508 7fe9bae887a0 20 read_log 192834'13811 (0'0) promote a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by osd.306.0:28273 2014-08-11 15:24:32.343374 2014-08-13 17:38:02.940520 7fe9bae887a0 20 read_log 192834'13812 (192834'13811) modify a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.933448.0:12436 2014-08-11 15:24:32.336062 2014-08-13 17:38:02.940532 7fe9bae887a0 20 read_log 192834'13813 (0'0) promote e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by osd.306.0:28275 2014-08-11 15:24:32.445051 2014-08-13 17:38:02.940544 7fe9bae887a0 20 read_log 192834'13814 (192834'13813) modify e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.933448.0:12463 2014-08-11 15:24:32.437908 2014-08-13 17:38:02.940557 7fe9bae887a0 20 read_log 192834'13825 (0'0) promote d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by osd.306.0:28295 2014-08-11 15:24:35.210764 2014-08-13 17:38:02.940569 7fe9bae887a0 20 read_log 192834'13826 (192834'13825) modify d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.933448.0:13061 2014-08-11 15:24:35.203166 2014-08-13 17:38:02.940581 7fe9bae887a0 20 read_log 192834'13853 (0'0) promote c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by osd.306.0:28341 2014-08-11 15:24:45.508319 2014-08-13 17:38:02.940593 7fe9bae887a0 20 read_log 192834'13854 (192834'13853) modify c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.933448.0:14442 2014-08-11 15:24:45.500005 2014-08-13 17:38:02.940612 7fe9bae887a0 20 read_log 192834'13855 (0'0) promote 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by osd.306.0:28343 2014-08-11 15:24:45.932779 2014-08-13 17:38:02.940625 7fe9bae887a0 20 read_log 192834'13856 (192834'13855) modify 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.933448.0:14510 2014-08-11 15:24:45.923024 2014-08-13 17:38:02.940637 7fe9bae887a0 20 read_log 192834'13861 (0'0) promote ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by osd.306.0:28353 2014-08-11 15:24:47.738909 2014-08-13 17:38:02.940649 7fe9bae887a0 20 read_log 192834'13862 (192834'13861) modify ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.933448.0:14930 2014-08-11 15:24:47.731311 2014-08-13 17:38:02.940661 7fe9bae887a0 20 read_log 192834'13863 (0'0) promote e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by osd.306.0:28355 2014-08-11 15:24:47.845686 2014-08-13 17:38:02.940674 7fe9bae887a0 20 read_log 192834'13864 (192834'13863) modify e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.933448.0:14959 2014-08-11 15:24:47.837487 2014-08-13 17:38:02.940686 7fe9bae887a0 20 read_log 192834'13877 (0'0) promote 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by osd.306.0:28371 2014-08-11 15:24:49.766481 2014-08-13 17:38:02.940698 7fe9bae887a0 20 read_log 192834'13878 (192834'13877) modify 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.933448.0:15468 2014-08-11 15:24:49.759293 2014-08-13 17:38:02.940710 7fe9bae887a0 20 read_log 192834'13899 (0'0) promote 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by osd.306.0:28427 2014-08-11 15:24:59.769526 2014-08-13 17:38:02.940722 7fe9bae887a0 20 read_log 192834'13900 (192834'13899) modify 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.933448.0:17456 2014-08-11 15:24:59.759982 2014-08-13 17:38:02.940734 7fe9bae887a0 20 read_log 192834'13901 (0'0) promote 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by osd.306.0:28429 2014-08-11 15:24:59.918308 2014-08-13 17:38:02.940746 7fe9bae887a0 20 read_log 192834'13902 (192834'13901) modify 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.933448.0:17494 2014-08-11 15:24:59.910248 2014-08-13 17:38:02.940758 7fe9bae887a0 20 read_log 192834'13909 (0'0) promote 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by osd.306.0:28455 2014-08-11 15:25:04.552690 2014-08-13 17:38:02.940770 7fe9bae887a0 20 read_log 192834'13910 (192834'13909) modify 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.933448.0:18274 2014-08-11 15:25:04.545560 2014-08-13 17:38:02.940782 7fe9bae887a0 20 read_log 192834'13915 (0'0) promote 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by osd.306.0:28461 2014-08-11 15:25:05.345427 2014-08-13 17:38:02.940794 7fe9bae887a0 20 read_log 192834'13916 (192834'13915) modify 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.933448.0:18446 2014-08-11 15:25:05.337622 2014-08-13 17:38:02.940806 7fe9bae887a0 20 read_log 192834'13919 (0'0) promote 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by osd.306.0:28467 2014-08-11 15:25:07.336858 2014-08-13 17:38:02.940818 7fe9bae887a0 20 read_log 192834'13920 (192834'13919) modify 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.933448.0:18884 2014-08-11 15:25:07.329123 2014-08-13 17:38:02.940832 7fe9bae887a0 20 read_log 192834'13927 (0'0) promote ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by osd.306.0:28475 2014-08-11 15:25:08.673661 2014-08-13 17:38:02.940844 7fe9bae887a0 20 read_log 192834'13928 (192834'13927) modify ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.933448.0:19114 2014-08-11 15:25:08.665552 2014-08-13 17:38:02.940866 7fe9bae887a0 20 read_log 192834'13937 (0'0) promote 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by osd.306.0:28491 2014-08-11 15:25:12.822583 2014-08-13 17:38:02.940879 7fe9bae887a0 20 read_log 192834'13938 (192834'13937) modify 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.933448.0:19631 2014-08-11 15:25:12.814961 2014-08-13 17:38:02.940891 7fe9bae887a0 20 read_log 192834'13965 (192834'13632) modify 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.933448.0:22657 2014-08-11 15:25:38.404194 2014-08-13 17:38:02.940904 7fe9bae887a0 20 read_log 192834'13970 (192834'13642) modify e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.933448.0:23370 2014-08-11 15:25:39.281880 2014-08-13 17:38:02.940916 7fe9bae887a0 20 read_log 192834'13978 (192834'13658) modify c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.933448.0:24391 2014-08-11 15:25:40.747225 2014-08-13 17:38:02.940928 7fe9bae887a0 20 read_log 192834'13981 (192834'13664) modify 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.933448.0:24688 2014-08-11 15:25:41.177779 2014-08-13 17:38:02.940943 7fe9bae887a0 20 read_log 192834'13988 (192834'13678) modify d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.933448.0:26382 2014-08-11 15:25:43.333975 2014-08-13 17:38:02.940955 7fe9bae887a0 20 read_log 192834'13990 (192834'13684) modify 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.933448.0:26597 2014-08-11 15:25:43.573048 2014-08-13 17:38:02.940967 7fe9bae887a0 20 read_log 192834'13992 (192834'13688) modify 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.933448.0:27089 2014-08-11 15:25:44.079622 2014-08-13 17:38:02.940979 7fe9bae887a0 20 read_log 192834'14022 (192834'13748) modify 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.933448.0:30266 2014-08-11 15:25:48.662667 2014-08-13 17:38:02.940992 7fe9bae887a0 20 read_log 192834'14024 (192834'13752) modify db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.933448.0:30466 2014-08-11 15:25:48.889852 2014-08-13 17:38:02.941004 7fe9bae887a0 20 read_log 192834'14025 (192834'13754) modify 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.933448.0:30546 2014-08-11 15:25:48.974647 2014-08-13 17:38:02.941016 7fe9bae887a0 20 read_log 192834'14041 (192834'13786) modify 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.933448.0:32313 2014-08-11 15:25:51.455784 2014-08-13 17:38:02.941028 7fe9bae887a0 20 read_log 192834'14053 (192834'13810) modify bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.933448.0:34366 2014-08-11 15:25:54.936359 2014-08-13 17:38:02.941040 7fe9bae887a0 20 read_log 192834'14054 (192834'13812) modify a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.933448.0:34727 2014-08-11 15:25:55.394304 2014-08-13 17:38:02.941052 7fe9bae887a0 20 read_log 192834'14055 (192834'13814) modify e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.933448.0:34754 2014-08-11 15:25:55.407554 2014-08-13 17:38:02.941064 7fe9bae887a0 20 read_log 192834'14061 (192834'13826) modify d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.933448.0:35352 2014-08-11 15:25:56.130057 2014-08-13 17:38:02.941076 7fe9bae887a0 20 read_log 192834'14075 (192834'13854) modify c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.933448.0:36733 2014-08-11 15:25:58.248474 2014-08-13 17:38:02.941088 7fe9bae887a0 20 read_log 192834'14076 (192834'13856) modify 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.933448.0:36801 2014-08-11 15:25:58.325606 2014-08-13 17:38:02.941100 7fe9bae887a0 20 read_log 192834'14079 (192834'13862) modify ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.933448.0:37221 2014-08-11 15:25:59.154352 2014-08-13 17:38:02.941119 7fe9bae887a0 20 read_log 192834'14080 (192834'13864) modify e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.933448.0:37250 2014-08-11 15:25:59.206499 2014-08-13 17:38:02.941132 7fe9bae887a0 20 read_log 192834'14087 (192834'13878) modify 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.933448.0:37759 2014-08-11 15:25:59.967102 2014-08-13 17:38:02.941144 7fe9bae887a0 20 read_log 192834'14098 (192834'13900) modify 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.933448.0:39747 2014-08-11 15:26:02.057163 2014-08-13 17:38:02.941166 7fe9bae887a0 20 read_log 192834'14099 (192834'13902) modify 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.933448.0:39785 2014-08-11 15:26:02.079253 2014-08-13 17:38:02.941182 7fe9bae887a0 20 read_log 192834'14103 (192834'13910) modify 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.933448.0:40565 2014-08-11 15:26:02.653932 2014-08-13 17:38:02.941194 7fe9bae887a0 20 read_log 192834'14106 (192834'13916) modify 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.933448.0:40737 2014-08-11 15:26:02.714206 2014-08-13 17:38:02.941207 7fe9bae887a0 20 read_log 192834'14108 (192834'13920) modify 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.933448.0:41175 2014-08-11 15:26:02.860236 2014-08-13 17:38:02.941219 7fe9bae887a0 20 read_log 192834'14112 (192834'13928) modify ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.933448.0:41405 2014-08-11 15:26:02.934285 2014-08-13 17:38:02.941230 7fe9bae887a0 20 read_log 192834'14117 (192834'13938) modify 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.933448.0:41922 2014-08-11 15:26:03.094669 2014-08-13 17:38:02.941243 7fe9bae887a0 20 read_log 192834'14158 (0'0) promote 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by osd.306.0:28956 2014-08-11 15:26:21.407453 2014-08-13 17:38:02.941255 7fe9bae887a0 20 read_log 192834'14159 (192834'14158) modify 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.1883634.0:1877 2014-08-11 15:26:21.399255 2014-08-13 17:38:02.941267 7fe9bae887a0 20 read_log 192834'14162 (0'0) promote f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by osd.306.0:28964 2014-08-11 15:26:23.177186 2014-08-13 17:38:02.941279 7fe9bae887a0 20 read_log 192834'14163 (192834'14162) modify f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.1883634.0:2355 2014-08-11 15:26:23.168935 2014-08-13 17:38:02.941291 7fe9bae887a0 20 read_log 192834'14174 (0'0) promote 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by osd.306.0:28978 2014-08-11 15:26:25.110169 2014-08-13 17:38:02.941303 7fe9bae887a0 20 read_log 192834'14175 (192834'14174) modify 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.1883634.0:2902 2014-08-11 15:26:25.102157 2014-08-13 17:38:02.941344 7fe9bae887a0 20 read_log 192834'14194 (0'0) promote 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by osd.306.0:29016 2014-08-11 15:26:31.099945 2014-08-13 17:38:02.941361 7fe9bae887a0 20 read_log 192834'14195 (192834'14194) modify 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.1883634.0:4570 2014-08-11 15:26:31.091595 2014-08-13 17:38:02.941374 7fe9bae887a0 20 read_log 192834'14198 (0'0) promote 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by osd.306.0:29024 2014-08-11 15:26:31.589800 2014-08-13 17:38:02.941386 7fe9bae887a0 20 read_log 192834'14199 (192834'14198) modify 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.1883634.0:4701 2014-08-11 15:26:31.580430 2014-08-13 17:38:02.941399 7fe9bae887a0 20 read_log 192834'14220 (0'0) promote 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by osd.306.0:29074 2014-08-11 15:26:37.159596 2014-08-13 17:38:02.941418 7fe9bae887a0 20 read_log 192834'14221 (192834'14220) modify 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.1883634.0:6210 2014-08-11 15:26:37.152952 2014-08-13 17:38:02.941433 7fe9bae887a0 20 read_log 192834'14222 (0'0) promote cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by osd.306.0:29078 2014-08-11 15:26:37.329855 2014-08-13 17:38:02.941446 7fe9bae887a0 20 read_log 192834'14223 (192834'14222) modify cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.1883634.0:6253 2014-08-11 15:26:37.322353 2014-08-13 17:38:02.941458 7fe9bae887a0 20 read_log 192834'14236 (0'0) promote 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by osd.306.0:29102 2014-08-11 15:26:39.809506 2014-08-13 17:38:02.941470 7fe9bae887a0 20 read_log 192834'14237 (192834'14236) modify 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.1883634.0:6874 2014-08-11 15:26:39.799952 2014-08-13 17:38:02.941482 7fe9bae887a0 20 read_log 192834'14256 (0'0) promote 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by osd.306.0:29146 2014-08-11 15:26:46.328249 2014-08-13 17:38:02.941494 7fe9bae887a0 20 read_log 192834'14257 (192834'14256) modify 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.1883634.0:8442 2014-08-11 15:26:46.319832 2014-08-13 17:38:02.941506 7fe9bae887a0 20 read_log 192834'14286 (0'0) promote 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by osd.306.0:29192 2014-08-11 15:26:50.680179 2014-08-13 17:38:02.941518 7fe9bae887a0 20 read_log 192834'14287 (192834'14286) modify 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.1883634.0:9386 2014-08-11 15:26:50.672991 2014-08-13 17:38:02.941530 7fe9bae887a0 20 read_log 192834'14304 (0'0) promote 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by osd.306.0:29224 2014-08-11 15:26:56.695920 2014-08-13 17:38:02.941560 7fe9bae887a0 20 read_log 192834'14305 (192834'14304) modify 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.1883634.0:10775 2014-08-11 15:26:56.688379 2014-08-13 17:38:02.941573 7fe9bae887a0 20 read_log 192834'14326 (0'0) promote c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by osd.306.0:29280 2014-08-11 15:27:01.680021 2014-08-13 17:38:02.941585 7fe9bae887a0 20 read_log 192834'14327 (192834'14326) modify c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.1883634.0:11909 2014-08-11 15:27:01.670971 2014-08-13 17:38:02.941597 7fe9bae887a0 20 read_log 192834'14366 (0'0) promote 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by osd.306.0:29382 2014-08-11 15:27:19.346336 2014-08-13 17:38:02.941609 7fe9bae887a0 20 read_log 192834'14367 (192834'14366) modify 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.1883634.0:15391 2014-08-11 15:27:19.337817 2014-08-13 17:38:02.941621 7fe9bae887a0 20 read_log 192834'14370 (0'0) promote 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by osd.306.0:29386 2014-08-11 15:27:19.538697 2014-08-13 17:38:02.941633 7fe9bae887a0 20 read_log 192834'14371 (192834'14370) modify 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.1883634.0:15439 2014-08-11 15:27:19.529201 2014-08-13 17:38:02.941645 7fe9bae887a0 20 read_log 192834'14380 (0'0) promote 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by osd.306.0:29412 2014-08-11 15:27:25.355117 2014-08-13 17:38:02.941657 7fe9bae887a0 20 read_log 192834'14381 (192834'14380) modify 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.1883634.0:16029 2014-08-11 15:27:25.346109 2014-08-13 17:38:02.941669 7fe9bae887a0 20 read_log 192834'14394 (0'0) promote 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by osd.306.0:29450 2014-08-11 15:27:38.676586 2014-08-13 17:38:02.941711 7fe9bae887a0 20 read_log 192834'14395 (192834'14394) modify 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.1883634.0:17715 2014-08-11 15:27:38.666866 2014-08-13 17:38:02.941990 7fe9bae887a0 20 read_log 192834'14402 (0'0) promote 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by osd.306.0:29458 2014-08-11 15:27:39.399405 2014-08-13 17:38:02.942003 7fe9bae887a0 20 read_log 192834'14403 (192834'14402) modify 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.1883634.0:17864 2014-08-11 15:27:39.388286 2014-08-13 17:38:02.942015 7fe9bae887a0 20 read_log 192834'14430 (0'0) promote 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by osd.306.0:29510 2014-08-11 15:27:54.109913 2014-08-13 17:38:02.942027 7fe9bae887a0 20 read_log 192834'14431 (192834'14430) modify 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.1883634.0:19780 2014-08-11 15:27:54.100162 2014-08-13 17:38:02.942039 7fe9bae887a0 20 read_log 192834'14442 (0'0) promote 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by osd.306.0:29528 2014-08-11 15:27:55.994888 2014-08-13 17:38:02.942051 7fe9bae887a0 20 read_log 192834'14443 (192834'14442) modify 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.1883634.0:20278 2014-08-11 15:27:55.985794 2014-08-13 17:38:02.942063 7fe9bae887a0 20 read_log 192834'14450 (0'0) promote 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by osd.306.0:29548 2014-08-11 15:28:02.509009 2014-08-13 17:38:02.942075 7fe9bae887a0 20 read_log 192834'14451 (192834'14450) modify 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.1883634.0:21047 2014-08-11 15:28:02.499500 2014-08-13 17:38:02.942087 7fe9bae887a0 20 read_log 192834'14454 (0'0) promote d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by osd.306.0:29558 2014-08-11 15:28:04.051727 2014-08-13 17:38:02.942100 7fe9bae887a0 20 read_log 192834'14455 (192834'14454) modify d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.1883634.0:21394 2014-08-11 15:28:04.043438 2014-08-13 17:38:02.942112 7fe9bae887a0 20 read_log 192834'14464 (0'0) promote 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by osd.306.0:29578 2014-08-11 15:28:07.460423 2014-08-13 17:38:02.942124 7fe9bae887a0 20 read_log 192834'14465 (192834'14464) modify 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.1883634.0:22226 2014-08-11 15:28:07.450812 2014-08-13 17:38:02.942136 7fe9bae887a0 20 read_log 192834'14492 (192834'14159) modify 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.1883634.0:25201 2014-08-11 15:28:20.332407 2014-08-13 17:38:02.942148 7fe9bae887a0 20 read_log 192834'14494 (192834'14163) modify f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.1883634.0:25679 2014-08-11 15:28:21.725131 2014-08-13 17:38:02.942166 7fe9bae887a0 20 read_log 192834'14500 (192834'14175) modify 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.1883634.0:26226 2014-08-11 15:28:22.809462 2014-08-13 17:38:02.942178 7fe9bae887a0 20 read_log 192834'14510 (192834'14195) modify 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.1883634.0:27894 2014-08-11 15:28:25.094621 2014-08-13 17:38:02.942191 7fe9bae887a0 20 read_log 192834'14512 (192834'14199) modify 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.1883634.0:28025 2014-08-11 15:28:25.280648 2014-08-13 17:38:02.942203 7fe9bae887a0 20 read_log 192834'14523 (192834'14221) modify 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.1883634.0:29534 2014-08-11 15:28:27.398918 2014-08-13 17:38:02.942214 7fe9bae887a0 20 read_log 192834'14524 (192834'14223) modify cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.1883634.0:29577 2014-08-11 15:28:27.459899 2014-08-13 17:38:02.942234 7fe9bae887a0 20 read_log 192834'14531 (192834'14237) modify 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.1883634.0:30198 2014-08-11 15:28:28.217969 2014-08-13 17:38:02.942247 7fe9bae887a0 20 read_log 192834'14541 (192834'14257) modify 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.1883634.0:31766 2014-08-11 15:28:30.935481 2014-08-13 17:38:02.942259 7fe9bae887a0 20 read_log 192834'14556 (192834'14287) modify 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.1883634.0:32710 2014-08-11 15:28:32.223735 2014-08-13 17:38:02.942271 7fe9bae887a0 20 read_log 192834'14565 (192834'14305) modify 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.1883634.0:34099 2014-08-11 15:28:34.229560 2014-08-13 17:38:02.942283 7fe9bae887a0 20 read_log 192834'14576 (192834'14327) modify c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.1883634.0:35233 2014-08-11 15:28:36.301181 2014-08-13 17:38:02.942295 7fe9bae887a0 20 read_log 192834'14596 (192834'14367) modify 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.1883634.0:38715 2014-08-11 15:28:41.050944 2014-08-13 17:38:02.942307 7fe9bae887a0 20 read_log 192834'14598 (192834'14371) modify 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.1883634.0:38763 2014-08-11 15:28:41.109922 2014-08-13 17:38:02.942319 7fe9bae887a0 20 read_log 192834'14603 (192834'14381) modify 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.1883634.0:39353 2014-08-11 15:28:42.048074 2014-08-13 17:38:02.942332 7fe9bae887a0 20 read_log 192834'14610 (192834'14395) modify 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.1883634.0:41039 2014-08-11 15:28:44.541057 2014-08-13 17:38:02.942343 7fe9bae887a0 20 read_log 192834'14614 (192834'14403) modify 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.1883634.0:41188 2014-08-11 15:28:44.728164 2014-08-13 17:38:02.942355 7fe9bae887a0 20 read_log 192834'14628 (192834'14431) modify 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.1883634.0:43104 2014-08-11 15:28:46.456383 2014-08-13 17:38:02.942367 7fe9bae887a0 20 read_log 192834'14634 (192834'14443) modify 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.1883634.0:43602 2014-08-11 15:28:46.604603 2014-08-13 17:38:02.942379 7fe9bae887a0 20 read_log 192834'14638 (192834'14451) modify 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.1883634.0:44371 2014-08-11 15:28:46.840803 2014-08-13 17:38:02.942391 7fe9bae887a0 20 read_log 192834'14640 (192834'14455) modify d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.1883634.0:44718 2014-08-11 15:28:46.937466 2014-08-13 17:38:02.942404 7fe9bae887a0 20 read_log 192834'14645 (192834'14465) modify 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.1883634.0:45550 2014-08-11 15:28:47.313851 2014-08-13 17:38:02.942416 7fe9bae887a0 20 read_log 192888'14651 (192888'14651) modify 1f7/hit_set_24.1f7_archive_2014-08-11 16:02:38.015319_2014-08-11 16:45:01.130851/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:45:01.131472 2014-08-13 17:38:02.942428 7fe9bae887a0 20 read_log 192888'14652 (0'0) promote eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by osd.4.0:21424 2014-08-11 16:45:01.137451 2014-08-13 17:38:02.942441 7fe9bae887a0 20 read_log 192888'14653 (192888'14652) modify eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.1315259.0:1529 2014-08-11 16:45:01.124452 2014-08-13 17:38:02.942453 7fe9bae887a0 20 read_log 192888'14656 (0'0) promote ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by osd.4.0:21444 2014-08-11 16:45:03.640506 2014-08-13 17:38:02.942472 7fe9bae887a0 20 read_log 192888'14657 (192888'14656) modify ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.1315259.0:2208 2014-08-11 16:45:03.631863 2014-08-13 17:38:02.942485 7fe9bae887a0 20 read_log 192888'14662 (0'0) promote 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by osd.4.0:21484 2014-08-11 16:45:08.748402 2014-08-13 17:38:02.942497 7fe9bae887a0 20 read_log 192888'14663 (192888'14662) modify 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.1315259.0:3538 2014-08-11 16:45:08.737839 2014-08-13 17:38:02.942509 7fe9bae887a0 20 read_log 192888'14666 (0'0) promote d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by osd.4.0:21494 2014-08-11 16:45:11.104741 2014-08-13 17:38:02.942523 7fe9bae887a0 20 read_log 192888'14667 (192888'14666) modify d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.1315259.0:4176 2014-08-11 16:45:11.094155 2014-08-13 17:38:02.942535 7fe9bae887a0 20 read_log 192888'14668 (0'0) promote 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by osd.4.0:21512 2014-08-11 16:45:12.989530 2014-08-13 17:38:02.942547 7fe9bae887a0 20 read_log 192888'14669 (192888'14668) modify 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.1315259.0:4696 2014-08-11 16:45:12.980696 2014-08-13 17:38:02.942560 7fe9bae887a0 20 read_log 192888'14670 (0'0) promote 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by osd.4.0:21558 2014-08-11 16:45:17.400736 2014-08-13 17:38:02.942572 7fe9bae887a0 20 read_log 192888'14671 (192888'14670) modify 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.1315259.0:5893 2014-08-11 16:45:17.392092 2014-08-13 17:38:02.942584 7fe9bae887a0 20 read_log 192888'14674 (0'0) promote e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by osd.4.0:21588 2014-08-11 16:45:22.248854 2014-08-13 17:38:02.942596 7fe9bae887a0 20 read_log 192888'14675 (192888'14674) modify e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.1315259.0:7235 2014-08-11 16:45:22.241220 2014-08-13 17:38:02.942608 7fe9bae887a0 20 read_log 192888'14682 (0'0) promote afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by osd.4.0:21634 2014-08-11 16:45:28.393684 2014-08-13 17:38:02.942620 7fe9bae887a0 20 read_log 192888'14683 (192888'14682) modify afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.1315259.0:8869 2014-08-11 16:45:28.384940 2014-08-13 17:38:02.942632 7fe9bae887a0 20 read_log 192888'14684 (0'0) promote 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by osd.4.0:21650 2014-08-11 16:45:31.671899 2014-08-13 17:38:02.942644 7fe9bae887a0 20 read_log 192888'14685 (192888'14684) modify 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.1315259.0:9723 2014-08-11 16:45:31.663840 2014-08-13 17:38:02.942657 7fe9bae887a0 20 read_log 192888'14688 (0'0) promote da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by osd.4.0:21664 2014-08-11 16:45:34.107026 2014-08-13 17:38:02.942668 7fe9bae887a0 20 read_log 192888'14689 (192888'14688) modify da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.1315259.0:10376 2014-08-11 16:45:34.098362 2014-08-13 17:38:02.942681 7fe9bae887a0 20 read_log 192888'14700 (0'0) promote 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by osd.4.0:21766 2014-08-11 16:45:49.645245 2014-08-13 17:38:02.942692 7fe9bae887a0 20 read_log 192888'14701 (192888'14700) modify 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.1315259.0:14214 2014-08-11 16:45:49.637371 2014-08-13 17:38:02.942705 7fe9bae887a0 20 read_log 192888'14706 (0'0) promote 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by osd.4.0:21834 2014-08-11 16:45:58.238915 2014-08-13 17:38:02.942726 7fe9bae887a0 20 read_log 192888'14707 (192888'14706) modify 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.1315259.0:16476 2014-08-11 16:45:58.229869 2014-08-13 17:38:02.942739 7fe9bae887a0 20 read_log 192888'14722 (0'0) promote c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by osd.4.0:21928 2014-08-11 16:46:11.232489 2014-08-13 17:38:02.942751 7fe9bae887a0 20 read_log 192888'14723 (192888'14722) modify c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.1315259.0:19757 2014-08-11 16:46:11.224215 2014-08-13 17:38:02.942763 7fe9bae887a0 20 read_log 192888'14738 (0'0) promote 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by osd.4.0:22052 2014-08-11 16:46:29.139060 2014-08-13 17:38:02.942775 7fe9bae887a0 20 read_log 192888'14739 (192888'14738) modify 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.1315259.0:24401 2014-08-11 16:46:29.129144 2014-08-13 17:38:02.942787 7fe9bae887a0 20 read_log 192888'14746 (0'0) promote 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by osd.4.0:22114 2014-08-11 16:46:37.504991 2014-08-13 17:38:02.942799 7fe9bae887a0 20 read_log 192888'14747 (192888'14746) modify 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.1315259.0:26402 2014-08-11 16:46:37.494558 2014-08-13 17:38:02.942811 7fe9bae887a0 20 read_log 192888'14748 (0'0) promote 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by osd.4.0:22116 2014-08-11 16:46:37.671491 2014-08-13 17:38:02.942823 7fe9bae887a0 20 read_log 192888'14749 (192888'14748) modify 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.1315259.0:26443 2014-08-11 16:46:37.662481 2014-08-13 17:38:02.942835 7fe9bae887a0 20 read_log 192888'14752 (0'0) promote e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by osd.4.0:22180 2014-08-11 16:46:50.944697 2014-08-13 17:38:02.942847 7fe9bae887a0 20 read_log 192888'14753 (192888'14752) modify e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.1315259.0:29689 2014-08-11 16:46:50.935440 2014-08-13 17:38:02.942859 7fe9bae887a0 20 read_log 192888'14754 (0'0) promote b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by osd.4.0:22186 2014-08-11 16:46:51.331977 2014-08-13 17:38:02.942871 7fe9bae887a0 20 read_log 192888'14755 (192888'14754) modify b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.1315259.0:29792 2014-08-11 16:46:51.323542 2014-08-13 17:38:02.942883 7fe9bae887a0 20 read_log 192888'14756 (0'0) promote 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by osd.4.0:22188 2014-08-11 16:46:51.396382 2014-08-13 17:38:02.942895 7fe9bae887a0 20 read_log 192888'14757 (192888'14756) modify 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.1315259.0:29809 2014-08-11 16:46:51.386100 2014-08-13 17:38:02.942907 7fe9bae887a0 20 read_log 192888'14760 (192888'14653) modify eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.1315259.0:32256 2014-08-11 16:46:56.329289 2014-08-13 17:38:02.942919 7fe9bae887a0 20 read_log 192888'14762 (192888'14657) modify ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.1315259.0:32935 2014-08-11 16:46:56.509221 2014-08-13 17:38:02.942931 7fe9bae887a0 20 read_log 192888'14765 (192888'14663) modify 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.1315259.0:34265 2014-08-11 16:46:56.848338 2014-08-13 17:38:02.942943 7fe9bae887a0 20 read_log 192888'14767 (192888'14667) modify d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.1315259.0:34903 2014-08-11 16:46:57.016692 2014-08-13 17:38:02.942955 7fe9bae887a0 20 read_log 192888'14768 (192888'14669) modify 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.1315259.0:35423 2014-08-11 16:46:57.160350 2014-08-13 17:38:02.942974 7fe9bae887a0 20 read_log 192888'14769 (192888'14671) modify 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.1315259.0:36620 2014-08-11 16:46:57.470481 2014-08-13 17:38:02.942987 7fe9bae887a0 20 read_log 192888'14771 (192888'14675) modify e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.1315259.0:37962 2014-08-11 16:46:57.821838 2014-08-13 17:38:02.942999 7fe9bae887a0 20 read_log 192888'14775 (192888'14683) modify afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.1315259.0:39596 2014-08-11 16:46:58.253180 2014-08-13 17:38:02.943011 7fe9bae887a0 20 read_log 192888'14776 (192888'14685) modify 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.1315259.0:40450 2014-08-11 16:46:58.489696 2014-08-13 17:38:02.943048 7fe9bae887a0 20 read_log 192888'14778 (192888'14689) modify da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.1315259.0:41103 2014-08-11 16:46:58.659832 2014-08-13 17:38:02.943061 7fe9bae887a0 20 read_log 192888'14784 (192888'14701) modify 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.1315259.0:44941 2014-08-11 16:46:59.723797 2014-08-13 17:38:02.943073 7fe9bae887a0 20 read_log 192888'14787 (192888'14707) modify 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.1315259.0:47203 2014-08-11 16:47:00.381989 2014-08-13 17:38:02.943085 7fe9bae887a0 20 read_log 192888'14795 (192888'14723) modify c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.1315259.0:50484 2014-08-11 16:47:01.383722 2014-08-13 17:38:02.943098 7fe9bae887a0 20 read_log 192888'14803 (192888'14739) modify 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.1315259.0:55128 2014-08-11 16:47:02.820551 2014-08-13 17:38:02.943110 7fe9bae887a0 20 read_log 192888'14807 (192888'14747) modify 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.1315259.0:57129 2014-08-11 16:47:03.417012 2014-08-13 17:38:02.943130 7fe9bae887a0 20 read_log 192888'14808 (192888'14749) modify 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.1315259.0:57170 2014-08-11 16:47:03.431832 2014-08-13 17:38:02.943142 7fe9bae887a0 20 read_log 192888'14810 (192888'14753) modify e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.1315259.0:60416 2014-08-11 16:47:04.437860 2014-08-13 17:38:02.943158 7fe9bae887a0 20 read_log 192888'14811 (192888'14755) modify b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.1315259.0:60519 2014-08-11 16:47:04.466312 2014-08-13 17:38:02.943171 7fe9bae887a0 20 read_log 192888'14812 (192888'14757) modify 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.1315259.0:60536 2014-08-11 16:47:04.470431 2014-08-13 17:38:02.943184 7fe9bae887a0 20 read_log 192888'14814 (192888'14814) modify 1f7/hit_set_24.1f7_archive_2014-08-11 16:45:01.131609_2014-08-11 17:05:15.751274/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.751520 2014-08-13 17:38:02.943196 7fe9bae887a0 20 read_log 192888'14815 (192888'14651) delete 1f7/hit_set_24.1f7_archive_2014-08-11 16:02:38.015319_2014-08-11 16:45:01.130851/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.751520 2014-08-13 17:38:02.943209 7fe9bae887a0 20 read_log 192888'14818 (0'0) promote 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by osd.4.0:22657 2014-08-11 17:05:17.232592 2014-08-13 17:38:02.943221 7fe9bae887a0 20 read_log 192888'14819 (192888'14818) modify 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.942868.0:538 2014-08-11 17:05:17.225955 2014-08-13 17:38:02.943234 7fe9bae887a0 20 read_log 192888'14820 (0'0) promote b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by osd.4.0:22673 2014-08-11 17:05:19.614261 2014-08-13 17:38:02.943253 7fe9bae887a0 20 read_log 192888'14821 (192888'14820) modify b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.942868.0:1113 2014-08-11 17:05:19.605145 2014-08-13 17:38:02.943266 7fe9bae887a0 20 read_log 192888'14826 (0'0) promote d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by osd.4.0:22732 2014-08-11 17:05:28.543938 2014-08-13 17:38:02.943278 7fe9bae887a0 20 read_log 192888'14827 (192888'14826) modify d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.942868.0:3406 2014-08-11 17:05:28.533546 2014-08-13 17:38:02.943290 7fe9bae887a0 20 read_log 192888'14828 (0'0) promote aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by osd.4.0:22746 2014-08-11 17:05:29.553338 2014-08-13 17:38:02.943302 7fe9bae887a0 20 read_log 192888'14829 (192888'14828) modify aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.942868.0:3679 2014-08-11 17:05:29.544629 2014-08-13 17:38:02.943315 7fe9bae887a0 20 read_log 192888'14838 (0'0) promote 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by osd.4.0:22804 2014-08-11 17:05:36.699043 2014-08-13 17:38:02.943326 7fe9bae887a0 20 read_log 192888'14839 (192888'14838) modify 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.942868.0:5555 2014-08-11 17:05:36.689188 2014-08-13 17:38:02.943339 7fe9bae887a0 20 read_log 192888'14840 (0'0) promote 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by osd.4.0:22822 2014-08-11 17:05:38.818034 2014-08-13 17:38:02.943352 7fe9bae887a0 20 read_log 192888'14841 (192888'14840) modify 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.942868.0:6139 2014-08-11 17:05:38.810729 2014-08-13 17:38:02.943365 7fe9bae887a0 20 read_log 192888'14844 (0'0) promote aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by osd.4.0:22838 2014-08-11 17:05:40.767160 2014-08-13 17:38:02.943377 7fe9bae887a0 20 read_log 192888'14845 (192888'14844) modify aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.942868.0:6679 2014-08-11 17:05:40.757606 2014-08-13 17:38:02.943390 7fe9bae887a0 20 read_log 192888'14846 (0'0) promote 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by osd.4.0:22842 2014-08-11 17:05:41.412893 2014-08-13 17:38:02.943403 7fe9bae887a0 20 read_log 192888'14847 (192888'14846) modify 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.942868.0:6858 2014-08-11 17:05:41.402121 2014-08-13 17:38:02.943416 7fe9bae887a0 20 read_log 192888'14850 (0'0) promote 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by osd.4.0:22864 2014-08-11 17:05:45.310125 2014-08-13 17:38:02.943428 7fe9bae887a0 20 read_log 192888'14851 (192888'14850) modify 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.942868.0:7856 2014-08-11 17:05:45.302618 2014-08-13 17:38:02.943440 7fe9bae887a0 20 read_log 192888'14852 (0'0) promote ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by osd.4.0:22870 2014-08-11 17:05:45.946651 2014-08-13 17:38:02.943452 7fe9bae887a0 20 read_log 192888'14853 (192888'14852) modify ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.942868.0:8033 2014-08-11 17:05:45.938110 2014-08-13 17:38:02.943464 7fe9bae887a0 20 read_log 192888'14860 (0'0) promote 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by osd.4.0:22940 2014-08-11 17:05:56.974245 2014-08-13 17:38:02.943477 7fe9bae887a0 20 read_log 192888'14861 (192888'14860) modify 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.942868.0:11050 2014-08-11 17:05:56.967386 2014-08-13 17:38:02.943489 7fe9bae887a0 20 read_log 192888'14862 (0'0) promote ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by osd.4.0:22980 2014-08-11 17:06:03.937206 2014-08-13 17:38:02.943510 7fe9bae887a0 20 read_log 192888'14863 (192888'14862) modify ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.942868.0:12930 2014-08-11 17:06:03.928616 2014-08-13 17:38:02.943523 7fe9bae887a0 20 read_log 192888'14874 (0'0) promote 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by osd.4.0:23076 2014-08-11 17:06:19.780915 2014-08-13 17:38:02.943537 7fe9bae887a0 20 read_log 192888'14875 (192888'14874) modify 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.942868.0:16925 2014-08-11 17:06:19.772447 2014-08-13 17:38:02.943549 7fe9bae887a0 20 read_log 192888'14876 (0'0) promote ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by osd.4.0:23100 2014-08-11 17:06:21.945987 2014-08-13 17:38:02.943561 7fe9bae887a0 20 read_log 192888'14877 (192888'14876) modify ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.942868.0:17498 2014-08-11 17:06:21.937416 2014-08-13 17:38:02.943573 7fe9bae887a0 20 read_log 192888'14878 (0'0) promote fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by osd.4.0:23110 2014-08-11 17:06:23.041178 2014-08-13 17:38:02.943585 7fe9bae887a0 20 read_log 192888'14879 (192888'14878) modify fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.942868.0:17789 2014-08-11 17:06:23.033636 2014-08-13 17:38:02.943597 7fe9bae887a0 20 read_log 192888'14880 (0'0) promote 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by osd.4.0:23114 2014-08-11 17:06:23.123137 2014-08-13 17:38:02.943609 7fe9bae887a0 20 read_log 192888'14881 (192888'14880) modify 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.942868.0:17802 2014-08-11 17:06:23.082988 2014-08-13 17:38:02.943621 7fe9bae887a0 20 read_log 192888'14884 (0'0) promote c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by osd.4.0:23156 2014-08-11 17:06:30.000437 2014-08-13 17:38:02.943633 7fe9bae887a0 20 read_log 192888'14885 (192888'14884) modify c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.942868.0:19640 2014-08-11 17:06:29.991526 2014-08-13 17:38:02.943645 7fe9bae887a0 20 read_log 192888'14886 (0'0) promote 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by osd.4.0:23164 2014-08-11 17:06:30.618152 2014-08-13 17:38:02.943657 7fe9bae887a0 20 read_log 192888'14887 (192888'14886) modify 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.942868.0:19802 2014-08-11 17:06:30.608262 2014-08-13 17:38:02.943669 7fe9bae887a0 20 read_log 192888'14888 (0'0) promote a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by osd.4.0:23168 2014-08-11 17:06:31.153709 2014-08-13 17:38:02.943681 7fe9bae887a0 20 read_log 192888'14889 (192888'14888) modify a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.942868.0:19943 2014-08-11 17:06:31.144512 2014-08-13 17:38:02.943693 7fe9bae887a0 20 read_log 192888'14900 (0'0) promote e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by osd.4.0:23258 2014-08-11 17:06:44.500430 2014-08-13 17:38:02.943704 7fe9bae887a0 20 read_log 192888'14901 (192888'14900) modify e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.942868.0:23400 2014-08-11 17:06:44.493770 2014-08-13 17:38:02.943717 7fe9bae887a0 20 read_log 192888'14902 (0'0) promote 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by osd.4.0:23262 2014-08-11 17:06:44.706016 2014-08-13 17:38:02.943728 7fe9bae887a0 20 read_log 192888'14903 (192888'14902) modify 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.942868.0:23456 2014-08-11 17:06:44.698521 2014-08-13 17:38:02.943740 7fe9bae887a0 20 read_log 192888'14904 (0'0) promote 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by osd.4.0:23264 2014-08-11 17:06:45.114518 2014-08-13 17:38:02.943779 7fe9bae887a0 20 read_log 192888'14905 (192888'14904) modify 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.942868.0:23567 2014-08-11 17:06:45.105658 2014-08-13 17:38:02.943796 7fe9bae887a0 20 read_log 192888'14908 (0'0) promote ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by osd.4.0:23284 2014-08-11 17:06:47.065367 2014-08-13 17:38:02.943808 7fe9bae887a0 20 read_log 192888'14909 (192888'14908) modify ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.942868.0:24096 2014-08-11 17:06:47.056802 2014-08-13 17:38:02.943820 7fe9bae887a0 20 read_log 192888'14914 (0'0) promote a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by osd.4.0:23322 2014-08-11 17:06:50.193248 2014-08-13 17:38:02.943832 7fe9bae887a0 20 read_log 192888'14915 (192888'14914) modify a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.942868.0:24942 2014-08-11 17:06:50.185171 2014-08-13 17:38:02.943845 7fe9bae887a0 20 read_log 192888'14916 (0'0) promote e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by osd.4.0:23364 2014-08-11 17:06:57.272548 2014-08-13 17:38:02.943856 7fe9bae887a0 20 read_log 192888'14917 (192888'14916) modify e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.942868.0:26759 2014-08-11 17:06:57.264487 2014-08-13 17:38:02.943868 7fe9bae887a0 20 read_log 192888'14918 (0'0) promote 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by osd.4.0:23408 2014-08-11 17:07:04.778465 2014-08-13 17:38:02.943880 7fe9bae887a0 20 read_log 192888'14919 (192888'14918) modify 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.942868.0:28758 2014-08-11 17:07:04.769415 2014-08-13 17:38:02.943893 7fe9bae887a0 20 read_log 192888'14920 (0'0) promote 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by osd.4.0:23424 2014-08-11 17:07:06.981522 2014-08-13 17:38:02.943905 7fe9bae887a0 20 read_log 192888'14921 (192888'14920) modify 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.942868.0:29306 2014-08-11 17:07:06.973191 2014-08-13 17:38:02.943917 7fe9bae887a0 20 read_log 192888'14926 (0'0) promote 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by osd.4.0:23458 2014-08-11 17:07:12.294353 2014-08-13 17:38:02.943929 7fe9bae887a0 20 read_log 192888'14927 (192888'14926) modify 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.942868.0:30616 2014-08-11 17:07:12.285702 2014-08-13 17:38:02.943941 7fe9bae887a0 20 read_log 192888'14928 (0'0) promote c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by osd.4.0:23462 2014-08-11 17:07:12.485101 2014-08-13 17:38:02.943953 7fe9bae887a0 20 read_log 192888'14929 (192888'14928) modify c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.942868.0:30662 2014-08-11 17:07:12.474532 2014-08-13 17:38:02.943965 7fe9bae887a0 20 read_log 192888'14930 (0'0) promote f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by osd.4.0:23474 2014-08-11 17:07:13.276466 2014-08-13 17:38:02.943977 7fe9bae887a0 20 read_log 192888'14931 (192888'14930) modify f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.942868.0:30864 2014-08-11 17:07:13.267493 2014-08-13 17:38:02.943989 7fe9bae887a0 20 read_log 192888'14932 (0'0) promote e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by osd.4.0:23476 2014-08-11 17:07:13.602559 2014-08-13 17:38:02.944002 7fe9bae887a0 20 read_log 192888'14933 (192888'14932) modify e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.942868.0:30948 2014-08-11 17:07:13.594572 2014-08-13 17:38:02.944015 7fe9bae887a0 20 read_log 192888'14935 (192888'14819) modify 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.942868.0:31804 2014-08-11 17:07:16.062574 2014-08-13 17:38:02.944034 7fe9bae887a0 20 read_log 192888'14936 (192888'14821) modify b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.942868.0:32379 2014-08-11 17:07:16.237557 2014-08-13 17:38:02.944047 7fe9bae887a0 20 read_log 192888'14939 (192888'14827) modify d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.942868.0:34672 2014-08-11 17:07:16.861140 2014-08-13 17:38:02.944059 7fe9bae887a0 20 read_log 192888'14940 (192888'14829) modify aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.942868.0:34945 2014-08-11 17:07:16.939951 2014-08-13 17:38:02.944072 7fe9bae887a0 20 read_log 192888'14945 (192888'14839) modify 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.942868.0:36821 2014-08-11 17:07:17.442503 2014-08-13 17:38:02.944084 7fe9bae887a0 20 read_log 192888'14946 (192888'14841) modify 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.942868.0:37405 2014-08-11 17:07:17.600445 2014-08-13 17:38:02.944096 7fe9bae887a0 20 read_log 192888'14948 (192888'14845) modify aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.942868.0:37945 2014-08-11 17:07:17.750289 2014-08-13 17:38:02.944108 7fe9bae887a0 20 read_log 192888'14949 (192888'14847) modify 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.942868.0:38124 2014-08-11 17:07:17.798448 2014-08-13 17:38:02.944120 7fe9bae887a0 20 read_log 192888'14951 (192888'14851) modify 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.942868.0:39122 2014-08-11 17:07:18.087802 2014-08-13 17:38:02.944132 7fe9bae887a0 20 read_log 192888'14952 (192888'14853) modify ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.942868.0:39299 2014-08-11 17:07:18.142708 2014-08-13 17:38:02.944144 7fe9bae887a0 20 read_log 192888'14956 (192888'14861) modify 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.942868.0:42316 2014-08-11 17:07:18.966394 2014-08-13 17:38:02.944169 7fe9bae887a0 20 read_log 192888'14957 (192888'14863) modify ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.942868.0:44196 2014-08-11 17:07:19.497197 2014-08-13 17:38:02.944186 7fe9bae887a0 20 read_log 192888'14963 (192888'14875) modify 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.942868.0:48191 2014-08-11 17:07:20.650144 2014-08-13 17:38:02.944198 7fe9bae887a0 20 read_log 192888'14964 (192888'14877) modify ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.942868.0:48764 2014-08-11 17:07:20.810594 2014-08-13 17:38:02.944210 7fe9bae887a0 20 read_log 192888'14965 (192888'14879) modify fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.942868.0:49055 2014-08-11 17:07:20.894083 2014-08-13 17:38:02.944222 7fe9bae887a0 20 read_log 192888'14966 (192888'14881) modify 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.942868.0:49068 2014-08-11 17:07:20.897643 2014-08-13 17:38:02.944235 7fe9bae887a0 20 read_log 192888'14968 (192888'14885) modify c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.942868.0:50906 2014-08-11 17:07:21.468683 2014-08-13 17:38:02.944247 7fe9bae887a0 20 read_log 192888'14969 (192888'14887) modify 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.942868.0:51068 2014-08-11 17:07:21.521317 2014-08-13 17:38:02.944259 7fe9bae887a0 20 read_log 192888'14970 (192888'14889) modify a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.942868.0:51209 2014-08-11 17:07:21.573517 2014-08-13 17:38:02.944291 7fe9bae887a0 20 read_log 192888'14976 (192888'14901) modify e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.942868.0:54666 2014-08-11 17:07:22.622521 2014-08-13 17:38:02.944306 7fe9bae887a0 20 read_log 192888'14977 (192888'14903) modify 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.942868.0:54722 2014-08-11 17:07:22.637972 2014-08-13 17:38:02.944329 7fe9bae887a0 20 read_log 192888'14978 (192888'14905) modify 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.942868.0:54833 2014-08-11 17:07:22.668688 2014-08-13 17:38:02.944344 7fe9bae887a0 20 read_log 192888'14980 (192888'14909) modify ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.942868.0:55362 2014-08-11 17:07:22.823609 2014-08-13 17:38:02.944356 7fe9bae887a0 20 read_log 192888'14983 (192888'14915) modify a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.942868.0:56208 2014-08-11 17:07:23.081292 2014-08-13 17:38:02.944369 7fe9bae887a0 20 read_log 192888'14984 (192888'14917) modify e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.942868.0:58025 2014-08-11 17:07:23.621993 2014-08-13 17:38:02.944381 7fe9bae887a0 20 read_log 192888'14985 (192888'14919) modify 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.942868.0:60024 2014-08-11 17:07:24.219467 2014-08-13 17:38:02.944394 7fe9bae887a0 20 read_log 192888'14986 (192888'14921) modify 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.942868.0:60572 2014-08-11 17:07:24.379311 2014-08-13 17:38:02.944406 7fe9bae887a0 20 read_log 192888'14989 (192888'14927) modify 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.942868.0:61882 2014-08-11 17:07:24.784729 2014-08-13 17:38:02.944418 7fe9bae887a0 20 read_log 192888'14990 (192888'14929) modify c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.942868.0:61928 2014-08-11 17:07:24.797523 2014-08-13 17:38:02.944431 7fe9bae887a0 20 read_log 192888'14991 (192888'14931) modify f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.942868.0:62130 2014-08-11 17:07:24.857221 2014-08-13 17:38:02.944443 7fe9bae887a0 20 read_log 192888'14992 (192888'14933) modify e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.942868.0:62214 2014-08-11 17:07:24.885517 2014-08-13 17:38:02.944456 7fe9bae887a0 20 read_log 193305'14993 (193305'14993) modify 1f7/hit_set_24.1f7_archive_2014-08-11 18:37:04.910739_2014-08-11 18:42:41.143618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.143869 2014-08-13 17:38:02.944468 7fe9bae887a0 20 read_log 193305'14994 (192888'14814) delete 1f7/hit_set_24.1f7_archive_2014-08-11 16:45:01.131609_2014-08-11 17:05:15.751274/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.143869 2014-08-13 17:38:02.944480 7fe9bae887a0 20 read_log 193305'14995 (0'0) promote d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by osd.4.0:31 2014-08-11 18:42:41.222357 2014-08-13 17:38:02.944492 7fe9bae887a0 20 read_log 193305'14996 (193305'14995) modify d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.951337.0:2106 2014-08-11 18:42:41.138103 2014-08-13 17:38:02.944506 7fe9bae887a0 20 read_log 193305'14997 (0'0) promote 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by osd.4.0:47 2014-08-11 18:42:45.283409 2014-08-13 17:38:02.944519 7fe9bae887a0 20 read_log 193305'14998 (193305'14997) modify 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.951337.0:3243 2014-08-11 18:42:45.273764 2014-08-13 17:38:02.944531 7fe9bae887a0 20 read_log 193305'14999 (0'0) promote ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by osd.4.0:73 2014-08-11 18:42:52.320494 2014-08-13 17:38:02.944543 7fe9bae887a0 20 read_log 193305'15000 (193305'14999) modify ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.951337.0:5144 2014-08-11 18:42:52.310978 2014-08-13 17:38:02.944555 7fe9bae887a0 20 read_log 193305'15001 (0'0) promote a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by osd.4.0:75 2014-08-11 18:42:52.499244 2014-08-13 17:38:02.944574 7fe9bae887a0 20 read_log 193305'15002 (193305'15001) modify a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.951337.0:5194 2014-08-11 18:42:52.492771 2014-08-13 17:38:02.944587 7fe9bae887a0 20 read_log 193305'15003 (0'0) promote 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by osd.4.0:79 2014-08-11 18:42:53.382798 2014-08-13 17:38:02.944599 7fe9bae887a0 20 read_log 193305'15004 (193305'15003) modify 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.951337.0:5438 2014-08-11 18:42:53.375125 2014-08-13 17:38:02.944611 7fe9bae887a0 20 read_log 193305'15005 (0'0) promote cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by osd.4.0:87 2014-08-11 18:42:54.323998 2014-08-13 17:38:02.944623 7fe9bae887a0 20 read_log 193305'15006 (193305'15005) modify cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.951337.0:5696 2014-08-11 18:42:54.317314 2014-08-13 17:38:02.944637 7fe9bae887a0 20 read_log 193305'15007 (0'0) promote 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by osd.4.0:99 2014-08-11 18:42:56.249677 2014-08-13 17:38:02.944650 7fe9bae887a0 20 read_log 193305'15008 (193305'15007) modify 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.951337.0:6226 2014-08-11 18:42:56.241508 2014-08-13 17:38:02.944662 7fe9bae887a0 20 read_log 193305'15009 (0'0) promote 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by osd.4.0:109 2014-08-11 18:42:58.169244 2014-08-13 17:38:02.944674 7fe9bae887a0 20 read_log 193305'15010 (193305'15009) modify 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.951337.0:6757 2014-08-11 18:42:58.161290 2014-08-13 17:38:02.944686 7fe9bae887a0 20 read_log 193305'15011 (0'0) promote 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by osd.4.0:125 2014-08-11 18:43:01.278500 2014-08-13 17:38:02.944698 7fe9bae887a0 20 read_log 193305'15012 (193305'15011) modify 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.951337.0:7617 2014-08-11 18:43:01.271122 2014-08-13 17:38:02.944710 7fe9bae887a0 20 read_log 193305'15013 (0'0) promote 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by osd.4.0:127 2014-08-11 18:43:01.722052 2014-08-13 17:38:02.944722 7fe9bae887a0 20 read_log 193305'15014 (193305'15013) modify 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.951337.0:7740 2014-08-11 18:43:01.714852 2014-08-13 17:38:02.944735 7fe9bae887a0 20 read_log 193305'15015 (0'0) promote 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by osd.4.0:169 2014-08-11 18:43:10.984877 2014-08-13 17:38:02.944747 7fe9bae887a0 20 read_log 193305'15016 (193305'15015) modify 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.951337.0:10191 2014-08-11 18:43:10.976643 2014-08-13 17:38:02.944759 7fe9bae887a0 20 read_log 193305'15017 (0'0) promote 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by osd.4.0:189 2014-08-11 18:43:15.418084 2014-08-13 17:38:02.944771 7fe9bae887a0 20 read_log 193305'15018 (193305'15017) modify 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.951337.0:11332 2014-08-11 18:43:15.410732 2014-08-13 17:38:02.944783 7fe9bae887a0 20 read_log 193305'15019 (0'0) promote bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by osd.4.0:251 2014-08-11 18:43:31.736696 2014-08-13 17:38:02.944795 7fe9bae887a0 20 read_log 193305'15020 (193305'15019) modify bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.951337.0:15815 2014-08-11 18:43:31.729666 2014-08-13 17:38:02.944807 7fe9bae887a0 20 read_log 193305'15021 (0'0) promote b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by osd.4.0:279 2014-08-11 18:43:35.693000 2014-08-13 17:38:02.944827 7fe9bae887a0 20 read_log 193305'15022 (193305'15021) modify b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.951337.0:16926 2014-08-11 18:43:35.683871 2014-08-13 17:38:02.944841 7fe9bae887a0 20 read_log 193305'15023 (0'0) promote f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by osd.4.0:321 2014-08-11 18:43:43.252502 2014-08-13 17:38:02.944853 7fe9bae887a0 20 read_log 193305'15024 (193305'15023) modify f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.951337.0:19072 2014-08-11 18:43:43.245115 2014-08-13 17:38:02.944866 7fe9bae887a0 20 read_log 193305'15025 (0'0) promote 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by osd.4.0:337 2014-08-11 18:43:46.952101 2014-08-13 17:38:02.944877 7fe9bae887a0 20 read_log 193305'15026 (193305'15025) modify 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.951337.0:20091 2014-08-11 18:43:46.944424 2014-08-13 17:38:02.944889 7fe9bae887a0 20 read_log 193305'15027 (0'0) promote 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by osd.4.0:353 2014-08-11 18:43:51.774124 2014-08-13 17:38:02.944901 7fe9bae887a0 20 read_log 193305'15028 (193305'15027) modify 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.951337.0:21434 2014-08-11 18:43:51.766763 2014-08-13 17:38:02.944914 7fe9bae887a0 20 read_log 193305'15029 (0'0) promote ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by osd.4.0:355 2014-08-11 18:43:52.308846 2014-08-13 17:38:02.944926 7fe9bae887a0 20 read_log 193305'15030 (193305'15029) modify ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.951337.0:21590 2014-08-11 18:43:52.302416 2014-08-13 17:38:02.944938 7fe9bae887a0 20 read_log 193305'15031 (0'0) promote 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by osd.4.0:367 2014-08-11 18:43:53.947261 2014-08-13 17:38:02.944951 7fe9bae887a0 20 read_log 193305'15032 (193305'15031) modify 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.951337.0:22040 2014-08-11 18:43:53.939735 2014-08-13 17:38:02.944963 7fe9bae887a0 20 read_log 193305'15033 (0'0) promote e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by osd.4.0:401 2014-08-11 18:44:01.496760 2014-08-13 17:38:02.944974 7fe9bae887a0 20 read_log 193305'15034 (193305'15033) modify e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.951337.0:24124 2014-08-11 18:44:01.488379 2014-08-13 17:38:02.944986 7fe9bae887a0 20 read_log 193305'15035 (0'0) promote 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by osd.4.0:417 2014-08-11 18:44:05.809263 2014-08-13 17:38:02.944998 7fe9bae887a0 20 read_log 193305'15036 (193305'15035) modify 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.951337.0:25320 2014-08-11 18:44:05.801152 2014-08-13 17:38:02.945010 7fe9bae887a0 20 read_log 193305'15037 (0'0) promote 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by osd.4.0:439 2014-08-11 18:44:10.928508 2014-08-13 17:38:02.945022 7fe9bae887a0 20 read_log 193305'15038 (193305'15037) modify 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.951337.0:26705 2014-08-11 18:44:10.920478 2014-08-13 17:38:02.945034 7fe9bae887a0 20 read_log 193305'15039 (0'0) promote e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by osd.4.0:479 2014-08-11 18:44:19.453838 2014-08-13 17:38:02.945046 7fe9bae887a0 20 read_log 193305'15040 (193305'15039) modify e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.951337.0:29031 2014-08-11 18:44:19.446713 2014-08-13 17:38:02.945058 7fe9bae887a0 20 read_log 193305'15041 (0'0) promote cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by osd.4.0:487 2014-08-11 18:44:20.941237 2014-08-13 17:38:02.945079 7fe9bae887a0 20 read_log 193305'15042 (193305'15041) modify cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.951337.0:29438 2014-08-11 18:44:20.934369 2014-08-13 17:38:02.945092 7fe9bae887a0 20 read_log 193305'15043 (0'0) promote d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by osd.4.0:519 2014-08-11 18:44:29.677422 2014-08-13 17:38:02.945104 7fe9bae887a0 20 read_log 193305'15044 (193305'15043) modify d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.951337.0:31824 2014-08-11 18:44:29.669878 2014-08-13 17:38:02.945116 7fe9bae887a0 20 read_log 193305'15045 (0'0) promote 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by osd.4.0:549 2014-08-11 18:44:32.070950 2014-08-13 17:38:02.945128 7fe9bae887a0 20 read_log 193305'15046 (193305'15045) modify 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.951337.0:32495 2014-08-11 18:44:32.059692 2014-08-13 17:38:02.945140 7fe9bae887a0 20 read_log 193305'15047 (193305'14996) modify d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.951337.0:34817 2014-08-11 18:44:35.208564 2014-08-13 17:38:02.945152 7fe9bae887a0 20 read_log 193305'15048 (193305'14998) modify 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.951337.0:35954 2014-08-11 18:44:35.827425 2014-08-13 17:38:02.945170 7fe9bae887a0 20 read_log 193305'15049 (193305'15000) modify ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.951337.0:37855 2014-08-11 18:44:37.085215 2014-08-13 17:38:02.945182 7fe9bae887a0 20 read_log 193305'15050 (193305'15002) modify a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.951337.0:37905 2014-08-11 18:44:37.109933 2014-08-13 17:38:02.945201 7fe9bae887a0 20 read_log 193305'15051 (193305'15004) modify 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.951337.0:38149 2014-08-11 18:44:37.262858 2014-08-13 17:38:02.945214 7fe9bae887a0 20 read_log 193305'15052 (193305'15006) modify cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.951337.0:38407 2014-08-11 18:44:37.410498 2014-08-13 17:38:02.945226 7fe9bae887a0 20 read_log 193305'15053 (193305'15008) modify 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.951337.0:38937 2014-08-11 18:44:37.741804 2014-08-13 17:38:02.945238 7fe9bae887a0 20 read_log 193305'15054 (193305'15010) modify 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.951337.0:39468 2014-08-11 18:44:38.027184 2014-08-13 17:38:02.945250 7fe9bae887a0 20 read_log 193305'15055 (193305'15012) modify 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.951337.0:40328 2014-08-11 18:44:38.376904 2014-08-13 17:38:02.945262 7fe9bae887a0 20 read_log 193305'15056 (193305'15014) modify 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.951337.0:40451 2014-08-11 18:44:38.418583 2014-08-13 17:38:02.945274 7fe9bae887a0 20 read_log 193305'15057 (193305'15016) modify 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.951337.0:42902 2014-08-11 18:44:39.417558 2014-08-13 17:38:02.945286 7fe9bae887a0 20 read_log 193305'15058 (193305'15018) modify 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.951337.0:44043 2014-08-11 18:44:39.777976 2014-08-13 17:38:02.945298 7fe9bae887a0 20 read_log 193305'15059 (193305'15020) modify bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.951337.0:48526 2014-08-11 18:44:41.079157 2014-08-13 17:38:02.945310 7fe9bae887a0 20 read_log 193305'15060 (193305'15022) modify b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.951337.0:49637 2014-08-11 18:44:41.425291 2014-08-13 17:38:02.945322 7fe9bae887a0 20 read_log 193305'15061 (193305'15024) modify f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.951337.0:51783 2014-08-11 18:44:42.048338 2014-08-13 17:38:02.945342 7fe9bae887a0 20 read_log 193305'15062 (193305'15026) modify 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.951337.0:52802 2014-08-11 18:44:42.345286 2014-08-13 17:38:02.945357 7fe9bae887a0 20 read_log 193305'15063 (193305'15028) modify 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.951337.0:54145 2014-08-11 18:44:42.742455 2014-08-13 17:38:02.945369 7fe9bae887a0 20 read_log 193305'15064 (193305'15030) modify ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.951337.0:54301 2014-08-11 18:44:42.791353 2014-08-13 17:38:02.945381 7fe9bae887a0 20 read_log 193305'15065 (193305'15032) modify 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.951337.0:54751 2014-08-11 18:44:42.928778 2014-08-13 17:38:02.945394 7fe9bae887a0 20 read_log 193305'15066 (193305'15034) modify e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.951337.0:56835 2014-08-11 18:44:43.545384 2014-08-13 17:38:02.945407 7fe9bae887a0 20 read_log 193305'15067 (193305'15036) modify 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.951337.0:58031 2014-08-11 18:44:43.927930 2014-08-13 17:38:02.945419 7fe9bae887a0 20 read_log 193305'15068 (193305'15038) modify 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.951337.0:59416 2014-08-11 18:44:44.321043 2014-08-13 17:38:02.945431 7fe9bae887a0 20 read_log 193305'15069 (193305'15040) modify e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.951337.0:61742 2014-08-11 18:44:45.015522 2014-08-13 17:38:02.945443 7fe9bae887a0 20 read_log 193305'15070 (193305'15042) modify cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.951337.0:62149 2014-08-11 18:44:45.140090 2014-08-13 17:38:02.945455 7fe9bae887a0 20 read_log 193305'15071 (193305'15044) modify d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.951337.0:64535 2014-08-11 18:44:45.841030 2014-08-13 17:38:02.945467 7fe9bae887a0 20 read_log 193305'15072 (193305'15046) modify 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.951337.0:65206 2014-08-11 18:44:46.050385 2014-08-13 17:38:02.945479 7fe9bae887a0 20 read_log 194215'15073 (194215'15073) modify 1f7/hit_set_24.1f7_archive_2014-08-12 15:32:25.864872_2014-08-12 15:44:02.507691/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:02.507944 2014-08-13 17:38:02.945508 7fe9bae887a0 20 read_log 194215'15074 (193305'14993) delete 1f7/hit_set_24.1f7_archive_2014-08-11 18:37:04.910739_2014-08-11 18:42:41.143618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:02.507944 2014-08-13 17:38:02.945524 7fe9bae887a0 20 read_log 194215'15075 (192888'14966) clean 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by osd.4.0:2248 2014-08-12 15:44:02.512493 2014-08-13 17:38:02.945536 7fe9bae887a0 20 read_log 194215'15076 (194215'15075) delete 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.2174595.0:104012 0.000000 2014-08-13 17:38:02.945547 7fe9bae887a0 20 read_log 194215'15077 (192834'14576) clean c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by osd.4.0:2250 2014-08-12 15:44:02.541271 2014-08-13 17:38:02.945560 7fe9bae887a0 20 read_log 194215'15078 (194215'15077) delete c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.2174595.0:104014 0.000000 2014-08-13 17:38:02.945570 7fe9bae887a0 20 read_log 194215'15079 (192834'13970) clean e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by osd.4.0:2252 2014-08-12 15:44:02.550334 2014-08-13 17:38:02.945582 7fe9bae887a0 20 read_log 194215'15080 (194215'15079) delete e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.2174595.0:104016 0.000000 2014-08-13 17:38:02.945626 7fe9bae887a0 20 read_log 194215'15081 (192888'14812) clean 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by osd.4.0:2254 2014-08-12 15:44:02.586098 2014-08-13 17:38:02.945642 7fe9bae887a0 20 read_log 194215'15082 (194215'15081) delete 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.2174595.0:104018 0.000000 2014-08-13 17:38:02.945653 7fe9bae887a0 20 read_log 194215'15083 (192834'14103) clean 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by osd.4.0:2256 2014-08-12 15:44:02.595563 2014-08-13 17:38:02.945665 7fe9bae887a0 20 read_log 194215'15084 (194215'15083) delete 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.2174595.0:104020 0.000000 2014-08-13 17:38:02.945676 7fe9bae887a0 20 read_log 194215'15085 (192834'14634) clean 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by osd.4.0:2258 2014-08-12 15:44:02.605767 2014-08-13 17:38:02.945688 7fe9bae887a0 20 read_log 194215'15086 (194215'15085) delete 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.2174595.0:104022 0.000000 2014-08-13 17:38:02.945698 7fe9bae887a0 20 read_log 194215'15087 (193305'15060) clean b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by osd.4.0:2260 2014-08-12 15:44:02.614268 2014-08-13 17:38:02.945710 7fe9bae887a0 20 read_log 194215'15088 (194215'15087) delete b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.2174595.0:104024 0.000000 2014-08-13 17:38:02.945720 7fe9bae887a0 20 read_log 194215'15089 (192834'14603) clean 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by osd.4.0:2262 2014-08-12 15:44:02.622526 2014-08-13 17:38:02.945732 7fe9bae887a0 20 read_log 194215'15090 (194215'15089) delete 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.2174595.0:104026 0.000000 2014-08-13 17:38:02.945743 7fe9bae887a0 20 read_log 194215'15091 (192888'14978) clean 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by osd.4.0:2264 2014-08-12 15:44:02.630356 2014-08-13 17:38:02.945755 7fe9bae887a0 20 read_log 194215'15092 (194215'15091) delete 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.2174595.0:104028 0.000000 2014-08-13 17:38:02.945766 7fe9bae887a0 20 read_log 194215'15093 (192888'14940) clean aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by osd.4.0:2266 2014-08-12 15:44:02.667077 2014-08-13 17:38:02.945778 7fe9bae887a0 20 read_log 194215'15094 (194215'15093) delete aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.2174595.0:104030 0.000000 2014-08-13 17:38:02.945788 7fe9bae887a0 20 read_log 194215'15095 (192888'14970) clean a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by osd.4.0:2268 2014-08-12 15:44:02.675458 2014-08-13 17:38:02.945802 7fe9bae887a0 20 read_log 194215'15096 (194215'15095) delete a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.2174595.0:104032 0.000000 2014-08-13 17:38:02.945813 7fe9bae887a0 20 read_log 194215'15097 (192834'14614) clean 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by osd.4.0:2270 2014-08-12 15:44:02.683377 2014-08-13 17:38:02.945825 7fe9bae887a0 20 read_log 194215'15098 (194215'15097) delete 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.2174595.0:104034 0.000000 2014-08-13 17:38:02.945835 7fe9bae887a0 20 read_log 194215'15099 (192834'14054) clean a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by osd.4.0:2272 2014-08-12 15:44:02.691485 2014-08-13 17:38:02.945847 7fe9bae887a0 20 read_log 194215'15100 (194215'15099) delete a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.2174595.0:104036 0.000000 2014-08-13 17:38:02.945867 7fe9bae887a0 20 read_log 194215'15101 (192888'14991) clean f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by osd.4.0:2274 2014-08-12 15:44:02.700061 2014-08-13 17:38:02.945880 7fe9bae887a0 20 read_log 194215'15102 (194215'15101) delete f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.2174595.0:104038 0.000000 2014-08-13 17:38:02.945891 7fe9bae887a0 20 read_log 194215'15103 (192888'14803) clean 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by osd.4.0:2276 2014-08-12 15:44:02.708816 2014-08-13 17:38:02.945903 7fe9bae887a0 20 read_log 194215'15104 (194215'15103) delete 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.2174595.0:104040 0.000000 2014-08-13 17:38:02.945914 7fe9bae887a0 20 read_log 194215'15105 (192834'14510) clean 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by osd.4.0:2278 2014-08-12 15:44:02.717168 2014-08-13 17:38:02.945927 7fe9bae887a0 20 read_log 194215'15106 (194215'15105) delete 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.2174595.0:104042 0.000000 2014-08-13 17:38:02.945938 7fe9bae887a0 20 read_log 194215'15107 (192834'14098) clean 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by osd.4.0:2280 2014-08-12 15:44:02.735265 2014-08-13 17:38:02.945950 7fe9bae887a0 20 read_log 194215'15108 (194215'15107) delete 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.2174595.0:104044 0.000000 2014-08-13 17:38:02.945961 7fe9bae887a0 20 read_log 194215'15109 (192888'14775) clean afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by osd.4.0:2282 2014-08-12 15:44:02.743935 2014-08-13 17:38:02.945973 7fe9bae887a0 20 read_log 194215'15110 (194215'15109) delete afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.2174595.0:104046 0.000000 2014-08-13 17:38:02.945983 7fe9bae887a0 20 read_log 194215'15111 (193305'15059) clean bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by osd.4.0:2284 2014-08-12 15:44:02.753086 2014-08-13 17:38:02.945995 7fe9bae887a0 20 read_log 194215'15112 (194215'15111) delete bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.2174595.0:104048 0.000000 2014-08-13 17:38:02.946006 7fe9bae887a0 20 read_log 194215'15113 (193305'15047) clean d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by osd.4.0:2286 2014-08-12 15:44:02.761976 2014-08-13 17:38:02.946018 7fe9bae887a0 20 read_log 194215'15114 (194215'15113) delete d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.2174595.0:104050 0.000000 2014-08-13 17:38:02.946029 7fe9bae887a0 20 read_log 194215'15115 (192888'14963) clean 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by osd.4.0:2288 2014-08-12 15:44:02.771311 2014-08-13 17:38:02.946040 7fe9bae887a0 20 read_log 194215'15116 (194215'15115) delete 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.2174595.0:104052 0.000000 2014-08-13 17:38:02.946051 7fe9bae887a0 20 read_log 194215'15117 (192888'14976) clean e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by osd.4.0:2290 2014-08-12 15:44:02.780392 2014-08-13 17:38:02.946063 7fe9bae887a0 20 read_log 194215'15118 (194215'15117) delete e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.2174595.0:104054 0.000000 2014-08-13 17:38:02.946074 7fe9bae887a0 20 read_log 194215'15119 (192834'14076) clean 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by osd.4.0:2292 2014-08-12 15:44:02.789265 2014-08-13 17:38:02.946086 7fe9bae887a0 20 read_log 194215'15120 (194215'15119) delete 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.2174595.0:104056 0.000000 2014-08-13 17:38:02.946104 7fe9bae887a0 20 read_log 194215'15121 (192834'14041) clean 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by osd.4.0:2294 2014-08-12 15:44:02.798142 2014-08-13 17:38:02.946117 7fe9bae887a0 20 read_log 194215'15122 (194215'15121) delete 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.2174595.0:104058 0.000000 2014-08-13 17:38:02.946127 7fe9bae887a0 20 read_log 194215'15123 (192834'14075) clean c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by osd.4.0:2296 2014-08-12 15:44:02.806616 2014-08-13 17:38:02.946139 7fe9bae887a0 20 read_log 194215'15124 (194215'15123) delete c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.2174595.0:104060 0.000000 2014-08-13 17:38:02.946150 7fe9bae887a0 20 read_log 194215'15125 (192834'14112) clean ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by osd.4.0:2298 2014-08-12 15:44:02.836751 2014-08-13 17:38:02.946168 7fe9bae887a0 20 read_log 194215'15126 (194215'15125) delete ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.2174595.0:104062 0.000000 2014-08-13 17:38:02.946178 7fe9bae887a0 20 read_log 194215'15127 (192834'14494) clean f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by osd.4.0:2300 2014-08-12 15:44:02.846685 2014-08-13 17:38:02.946191 7fe9bae887a0 20 read_log 194215'15128 (194215'15127) delete f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.2174595.0:104064 0.000000 2014-08-13 17:38:02.946201 7fe9bae887a0 20 read_log 194215'15129 (192834'14500) clean 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by osd.4.0:2302 2014-08-12 15:44:02.857282 2014-08-13 17:38:02.946214 7fe9bae887a0 20 read_log 194215'15130 (194215'15129) delete 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.2174595.0:104066 0.000000 2014-08-13 17:38:02.946224 7fe9bae887a0 20 read_log 194215'15131 (192888'14948) clean aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by osd.4.0:2304 2014-08-12 15:44:02.866193 2014-08-13 17:38:02.946236 7fe9bae887a0 20 read_log 194215'15132 (194215'15131) delete aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.2174595.0:104068 0.000000 2014-08-13 17:38:02.946251 7fe9bae887a0 20 read_log 194215'15133 (192834'14610) clean 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by osd.4.0:2306 2014-08-12 15:44:02.874731 2014-08-13 17:38:02.946264 7fe9bae887a0 20 read_log 194215'15134 (194215'15133) delete 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.2174595.0:104070 0.000000 2014-08-13 17:38:02.946275 7fe9bae887a0 20 read_log 194215'15135 (192888'14810) clean e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by osd.4.0:2308 2014-08-12 15:44:02.883159 2014-08-13 17:38:02.946287 7fe9bae887a0 20 read_log 194215'15136 (194215'15135) delete e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.2174595.0:104072 0.000000 2014-08-13 17:38:02.946298 7fe9bae887a0 20 read_log 194215'15137 (192834'14024) clean db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by osd.4.0:2310 2014-08-12 15:44:02.891621 2014-08-13 17:38:02.946310 7fe9bae887a0 20 read_log 194215'15138 (194215'15137) delete db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.2174595.0:104074 0.000000 2014-08-13 17:38:02.946320 7fe9bae887a0 20 read_log 194215'15139 (192834'14512) clean 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by osd.4.0:2312 2014-08-12 15:44:02.900289 2014-08-13 17:38:02.946332 7fe9bae887a0 20 read_log 194215'15140 (194215'15139) delete 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.2174595.0:104076 0.000000 2014-08-13 17:38:02.946350 7fe9bae887a0 20 read_log 194215'15141 (192888'14949) clean 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by osd.4.0:2314 2014-08-12 15:44:02.908203 2014-08-13 17:38:02.946363 7fe9bae887a0 20 read_log 194215'15142 (194215'15141) delete 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.2174595.0:104078 0.000000 2014-08-13 17:38:02.946373 7fe9bae887a0 20 read_log 194215'15143 (192834'14598) clean 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by osd.4.0:2316 2014-08-12 15:44:02.916460 2014-08-13 17:38:02.946385 7fe9bae887a0 20 read_log 194215'15144 (194215'15143) delete 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.2174595.0:104080 0.000000 2014-08-13 17:38:02.946396 7fe9bae887a0 20 read_log 194215'15145 (192834'14108) clean 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by osd.4.0:2318 2014-08-12 15:44:02.924655 2014-08-13 17:38:02.946408 7fe9bae887a0 20 read_log 194215'15146 (194215'15145) delete 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.2174595.0:104082 0.000000 2014-08-13 17:38:02.946419 7fe9bae887a0 20 read_log 194215'15147 (192834'14645) clean 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by osd.4.0:2320 2014-08-12 15:44:02.932540 2014-08-13 17:38:02.946431 7fe9bae887a0 20 read_log 194215'15148 (194215'15147) delete 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.2174595.0:104084 0.000000 2014-08-13 17:38:02.946441 7fe9bae887a0 20 read_log 194215'15149 (193305'15053) clean 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by osd.4.0:2322 2014-08-12 15:44:02.940118 2014-08-13 17:38:02.946455 7fe9bae887a0 20 read_log 194215'15150 (194215'15149) delete 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.2174595.0:104086 0.000000 2014-08-13 17:38:02.946467 7fe9bae887a0 20 read_log 194215'15151 (192834'14565) clean 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by osd.4.0:2324 2014-08-12 15:44:02.948004 2014-08-13 17:38:02.946480 7fe9bae887a0 20 read_log 194215'15152 (194215'15151) delete 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.2174595.0:104088 0.000000 2014-08-13 17:38:02.946491 7fe9bae887a0 20 read_log 194215'15153 (192888'14957) clean ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by osd.4.0:2326 2014-08-12 15:44:02.955977 2014-08-13 17:38:02.946503 7fe9bae887a0 20 read_log 194215'15154 (194215'15153) delete ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.2174595.0:104090 0.000000 2014-08-13 17:38:02.946514 7fe9bae887a0 20 read_log 194215'15155 (192888'14939) clean d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by osd.4.0:2328 2014-08-12 15:44:02.972360 2014-08-13 17:38:02.946525 7fe9bae887a0 20 read_log 194215'15156 (194215'15155) delete d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.2174595.0:104092 0.000000 2014-08-13 17:38:02.946536 7fe9bae887a0 20 read_log 194215'15157 (192888'14936) clean b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by osd.4.0:2330 2014-08-12 15:44:02.980570 2014-08-13 17:38:02.946548 7fe9bae887a0 20 read_log 194215'15158 (194215'15157) delete b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.2174595.0:104094 0.000000 2014-08-13 17:38:02.946558 7fe9bae887a0 20 read_log 194215'15159 (192834'13978) clean c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by osd.4.0:2332 2014-08-12 15:44:02.989452 2014-08-13 17:38:02.946570 7fe9bae887a0 20 read_log 194215'15160 (194215'15159) delete c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.2174595.0:104096 0.000000 2014-08-13 17:38:02.946590 7fe9bae887a0 20 read_log 194215'15161 (192834'14524) clean cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by osd.4.0:2334 2014-08-12 15:44:02.997215 2014-08-13 17:38:02.946602 7fe9bae887a0 20 read_log 194215'15162 (194215'15161) delete cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.2174595.0:104098 0.000000 2014-08-13 17:38:02.946613 7fe9bae887a0 20 read_log 194215'15163 (192888'14951) clean 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by osd.4.0:2336 2014-08-12 15:44:03.032392 2014-08-13 17:38:02.946625 7fe9bae887a0 20 read_log 194215'15164 (194215'15163) delete 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.2174595.0:104100 0.000000 2014-08-13 17:38:02.946636 7fe9bae887a0 20 read_log 194215'15165 (192834'14640) clean d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by osd.4.0:2338 2014-08-12 15:44:03.042609 2014-08-13 17:38:02.946648 7fe9bae887a0 20 read_log 194215'15166 (194215'15165) delete d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.2174595.0:104102 0.000000 2014-08-13 17:38:02.946676 7fe9bae887a0 20 read_log 194215'15167 (192834'14087) clean 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by osd.4.0:2340 2014-08-12 15:44:03.051650 2014-08-13 17:38:02.946689 7fe9bae887a0 20 read_log 194215'15168 (194215'15167) delete 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.2174595.0:104104 0.000000 2014-08-13 17:38:02.946700 7fe9bae887a0 20 read_log 194215'15169 (192888'14952) clean ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by osd.4.0:2342 2014-08-12 15:44:03.060366 2014-08-13 17:38:02.946712 7fe9bae887a0 20 read_log 194215'15170 (194215'15169) delete ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.2174595.0:104106 0.000000 2014-08-13 17:38:02.946722 7fe9bae887a0 20 read_log 194215'15171 (192888'14968) clean c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by osd.4.0:2344 2014-08-12 15:44:03.068676 2014-08-13 17:38:02.946735 7fe9bae887a0 20 read_log 194215'15172 (194215'15171) delete c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.2174595.0:104108 0.000000 2014-08-13 17:38:02.946745 7fe9bae887a0 20 read_log 194215'15173 (193305'15055) clean 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by osd.4.0:2346 2014-08-12 15:44:03.077376 2014-08-13 17:38:02.946757 7fe9bae887a0 20 read_log 194215'15174 (194215'15173) delete 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.2174595.0:104110 0.000000 2014-08-13 17:38:02.946768 7fe9bae887a0 20 read_log 194215'15175 (192834'14628) clean 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by osd.4.0:2348 2014-08-12 15:44:03.086032 2014-08-13 17:38:02.946780 7fe9bae887a0 20 read_log 194215'15176 (194215'15175) delete 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.2174595.0:104112 0.000000 2014-08-13 17:38:02.946791 7fe9bae887a0 20 read_log 194215'15177 (193305'15051) clean 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by osd.4.0:2350 2014-08-12 15:44:03.095004 2014-08-13 17:38:02.946803 7fe9bae887a0 20 read_log 194215'15178 (194215'15177) delete 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.2174595.0:104114 0.000000 2014-08-13 17:38:02.946814 7fe9bae887a0 20 read_log 194215'15179 (193305'15072) clean 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by osd.4.0:2352 2014-08-12 15:44:03.103807 2014-08-13 17:38:02.946826 7fe9bae887a0 20 read_log 194215'15180 (194215'15179) delete 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.2174595.0:104116 0.000000 2014-08-13 17:38:02.946843 7fe9bae887a0 20 read_log 194215'15181 (192834'14079) clean ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by osd.4.0:2354 2014-08-12 15:44:03.111259 2014-08-13 17:38:02.946856 7fe9bae887a0 20 read_log 194215'15182 (194215'15181) delete ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.2174595.0:104118 0.000000 2014-08-13 17:38:02.946867 7fe9bae887a0 20 read_log 194215'15183 (192834'14106) clean 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by osd.4.0:2356 2014-08-12 15:44:03.118558 2014-08-13 17:38:02.946879 7fe9bae887a0 20 read_log 194215'15184 (194215'15183) delete 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.2174595.0:104120 0.000000 2014-08-13 17:38:02.946890 7fe9bae887a0 20 read_log 194215'15185 (192888'14989) clean 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by osd.4.0:2358 2014-08-12 15:44:03.126172 2014-08-13 17:38:02.946902 7fe9bae887a0 20 read_log 194215'15186 (194215'15185) delete 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.2174595.0:104122 0.000000 2014-08-13 17:38:02.946912 7fe9bae887a0 20 read_log 194215'15187 (192834'14596) clean 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by osd.4.0:2360 2014-08-12 15:44:03.133598 2014-08-13 17:38:02.946924 7fe9bae887a0 20 read_log 194215'15188 (194215'15187) delete 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.2174595.0:104124 0.000000 2014-08-13 17:38:02.946935 7fe9bae887a0 20 read_log 194215'15189 (193305'15048) clean 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by osd.4.0:2362 2014-08-12 15:44:03.141831 2014-08-13 17:38:02.946947 7fe9bae887a0 20 read_log 194215'15190 (194215'15189) delete 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.2174595.0:104126 0.000000 2014-08-13 17:38:02.946958 7fe9bae887a0 20 read_log 194215'15191 (192888'14767) clean d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by osd.4.0:2364 2014-08-12 15:44:03.149479 2014-08-13 17:38:02.946970 7fe9bae887a0 20 read_log 194215'15192 (194215'15191) delete d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.2174595.0:104128 0.000000 2014-08-13 17:38:02.946980 7fe9bae887a0 20 read_log 194215'15193 (192888'14946) clean 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by osd.4.0:2366 2014-08-12 15:44:03.158866 2014-08-13 17:38:02.946992 7fe9bae887a0 20 read_log 194215'15194 (194215'15193) delete 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.2174595.0:104130 0.000000 2014-08-13 17:38:02.947003 7fe9bae887a0 20 read_log 194215'15195 (193305'15070) clean cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by osd.4.0:2368 2014-08-12 15:44:03.167449 2014-08-13 17:38:02.947034 7fe9bae887a0 20 read_log 194215'15196 (194215'15195) delete cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.2174595.0:104132 0.000000 2014-08-13 17:38:02.947046 7fe9bae887a0 20 read_log 194215'15197 (192834'14061) clean d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by osd.4.0:2370 2014-08-12 15:44:03.175283 2014-08-13 17:38:02.947059 7fe9bae887a0 20 read_log 194215'15198 (194215'15197) delete d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.2174595.0:104134 0.000000 2014-08-13 17:38:02.947070 7fe9bae887a0 20 read_log 194215'15199 (192834'14541) clean 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by osd.4.0:2372 2014-08-12 15:44:03.183456 2014-08-13 17:38:02.947082 7fe9bae887a0 20 read_log 194215'15200 (194215'15199) delete 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.2174595.0:104136 0.000000 2014-08-13 17:38:02.947100 7fe9bae887a0 20 read_log 194215'15201 (192888'14769) clean 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by osd.4.0:2374 2014-08-12 15:44:03.192452 2014-08-13 17:38:02.947113 7fe9bae887a0 20 read_log 194215'15202 (194215'15201) delete 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.2174595.0:104138 0.000000 2014-08-13 17:38:02.947124 7fe9bae887a0 20 read_log 194215'15203 (192888'14778) clean da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by osd.4.0:2376 2014-08-12 15:44:03.200895 2014-08-13 17:38:02.947136 7fe9bae887a0 20 read_log 194215'15204 (194215'15203) delete da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.2174595.0:104140 0.000000 2014-08-13 17:38:02.947147 7fe9bae887a0 20 read_log 194215'15205 (192888'14992) clean e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by osd.4.0:2378 2014-08-12 15:44:03.208990 2014-08-13 17:38:02.947164 7fe9bae887a0 20 read_log 194215'15206 (194215'15205) delete e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.2174595.0:104142 0.000000 2014-08-13 17:38:02.947175 7fe9bae887a0 20 read_log 194215'15207 (192834'14492) clean 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by osd.4.0:2380 2014-08-12 15:44:03.217426 2014-08-13 17:38:02.947187 7fe9bae887a0 20 read_log 194215'15208 (194215'15207) delete 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.2174595.0:104144 0.000000 2014-08-13 17:38:02.947198 7fe9bae887a0 20 read_log 194215'15209 (192888'14768) clean 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by osd.4.0:2382 2014-08-12 15:44:03.226590 2014-08-13 17:38:02.947210 7fe9bae887a0 20 read_log 194215'15210 (194215'15209) delete 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.2174595.0:104146 0.000000 2014-08-13 17:38:02.947221 7fe9bae887a0 20 read_log 194215'15211 (193305'15068) clean 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by osd.4.0:2384 2014-08-12 15:44:03.235713 2014-08-13 17:38:02.947233 7fe9bae887a0 20 read_log 194215'15212 (194215'15211) delete 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.2174595.0:104148 0.000000 2014-08-13 17:38:02.947243 7fe9bae887a0 20 read_log 194215'15213 (193305'15052) clean cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by osd.4.0:2386 2014-08-12 15:44:03.244646 2014-08-13 17:38:02.947255 7fe9bae887a0 20 read_log 194215'15214 (194215'15213) delete cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.2174595.0:104150 0.000000 2014-08-13 17:38:02.947275 7fe9bae887a0 20 read_log 194215'15215 (192888'14935) clean 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by osd.4.0:2388 2014-08-12 15:44:03.253110 2014-08-13 17:38:02.947288 7fe9bae887a0 20 read_log 194215'15216 (194215'15215) delete 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.2174595.0:104152 0.000000 2014-08-13 17:38:02.947298 7fe9bae887a0 20 read_log 194215'15217 (192888'14776) clean 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by osd.4.0:2390 2014-08-12 15:44:03.260910 2014-08-13 17:38:02.947310 7fe9bae887a0 20 read_log 194215'15218 (194215'15217) delete 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.2174595.0:104154 0.000000 2014-08-13 17:38:02.947321 7fe9bae887a0 20 read_log 194215'15219 (192888'14811) clean b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by osd.4.0:2392 2014-08-12 15:44:03.269464 2014-08-13 17:38:02.947333 7fe9bae887a0 20 read_log 194215'15220 (194215'15219) delete b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.2174595.0:104156 0.000000 2014-08-13 17:38:02.947369 7fe9bae887a0 20 read_log 194215'15221 (192888'14956) clean 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by osd.4.0:2394 2014-08-12 15:44:03.291809 2014-08-13 17:38:02.947383 7fe9bae887a0 20 read_log 194215'15222 (194215'15221) delete 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.2174595.0:104158 0.000000 2014-08-13 17:38:02.947393 7fe9bae887a0 20 read_log 194215'15223 (193305'15062) clean 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by osd.4.0:2396 2014-08-12 15:44:03.299478 2014-08-13 17:38:02.947405 7fe9bae887a0 20 read_log 194215'15224 (194215'15223) delete 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.2174595.0:104160 0.000000 2014-08-13 17:38:02.947416 7fe9bae887a0 20 read_log 194215'15225 (193305'15065) clean 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by osd.4.0:2398 2014-08-12 15:44:03.306949 2014-08-13 17:38:02.947428 7fe9bae887a0 20 read_log 194215'15226 (194215'15225) delete 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.2174595.0:104162 0.000000 2014-08-13 17:38:02.947438 7fe9bae887a0 20 read_log 194215'15227 (193305'15067) clean 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by osd.4.0:2400 2014-08-12 15:44:03.315203 2014-08-13 17:38:02.947451 7fe9bae887a0 20 read_log 194215'15228 (194215'15227) delete 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.2174595.0:104164 0.000000 2014-08-13 17:38:02.947461 7fe9bae887a0 20 read_log 194215'15229 (192834'14638) clean 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by osd.4.0:2402 2014-08-12 15:44:03.322606 2014-08-13 17:38:02.947473 7fe9bae887a0 20 read_log 194215'15230 (194215'15229) delete 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.2174595.0:104166 0.000000 2014-08-13 17:38:02.947484 7fe9bae887a0 20 read_log 194215'15231 (193305'15049) clean ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by osd.4.0:2404 2014-08-12 15:44:03.330642 2014-08-13 17:38:02.947496 7fe9bae887a0 20 read_log 194215'15232 (194215'15231) delete ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.2174595.0:104168 0.000000 2014-08-13 17:38:02.947506 7fe9bae887a0 20 read_log 194215'15233 (192834'14099) clean 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by osd.4.0:2406 2014-08-12 15:44:03.338339 2014-08-13 17:38:02.947518 7fe9bae887a0 20 read_log 194215'15234 (194215'15233) delete 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.2174595.0:104170 0.000000 2014-08-13 17:38:02.947529 7fe9bae887a0 20 read_log 194215'15235 (193305'15066) clean e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by osd.4.0:2408 2014-08-12 15:44:03.346674 2014-08-13 17:38:02.947541 7fe9bae887a0 20 read_log 194215'15236 (194215'15235) delete e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.2174595.0:104172 0.000000 2014-08-13 17:38:02.947552 7fe9bae887a0 20 read_log 194215'15237 (193305'15063) clean 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by osd.4.0:2410 2014-08-12 15:44:03.354170 2014-08-13 17:38:02.947564 7fe9bae887a0 20 read_log 194215'15238 (194215'15237) delete 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.2174595.0:104174 0.000000 2014-08-13 17:38:02.947574 7fe9bae887a0 20 read_log 194215'15239 (192888'14784) clean 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by osd.4.0:2412 2014-08-12 15:44:03.362518 2014-08-13 17:38:02.947587 7fe9bae887a0 20 read_log 194215'15240 (194215'15239) delete 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.2174595.0:104176 0.000000 2014-08-13 17:38:02.947605 7fe9bae887a0 20 read_log 194215'15241 (193305'15054) clean 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by osd.4.0:2414 2014-08-12 15:44:03.369904 2014-08-13 17:38:02.947618 7fe9bae887a0 20 read_log 194215'15242 (194215'15241) delete 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.2174595.0:104178 0.000000 2014-08-13 17:38:02.947628 7fe9bae887a0 20 read_log 194215'15243 (192888'14969) clean 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by osd.4.0:2416 2014-08-12 15:44:03.377827 2014-08-13 17:38:02.947640 7fe9bae887a0 20 read_log 194215'15244 (194215'15243) delete 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.2174595.0:104180 0.000000 2014-08-13 17:38:02.947651 7fe9bae887a0 20 read_log 194215'15245 (192834'14053) clean bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by osd.4.0:2418 2014-08-12 15:44:03.387091 2014-08-13 17:38:02.947663 7fe9bae887a0 20 read_log 194215'15246 (194215'15245) delete bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.2174595.0:104182 0.000000 2014-08-13 17:38:02.947674 7fe9bae887a0 20 read_log 194215'15247 (192834'13992) clean 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by osd.4.0:2420 2014-08-12 15:44:03.395359 2014-08-13 17:38:02.947686 7fe9bae887a0 20 read_log 194215'15248 (194215'15247) delete 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.2174595.0:104184 0.000000 2014-08-13 17:38:02.947696 7fe9bae887a0 20 read_log 194215'15249 (192834'14117) clean 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by osd.4.0:2422 2014-08-12 15:44:03.403634 2014-08-13 17:38:02.947708 7fe9bae887a0 20 read_log 194215'15250 (194215'15249) delete 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.2174595.0:104186 0.000000 2014-08-13 17:38:02.947718 7fe9bae887a0 20 read_log 194215'15251 (192834'14055) clean e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by osd.4.0:2424 2014-08-12 15:44:03.412255 2014-08-13 17:38:02.947730 7fe9bae887a0 20 read_log 194215'15252 (194215'15251) delete e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.2174595.0:104188 0.000000 2014-08-13 17:38:02.947741 7fe9bae887a0 20 read_log 194215'15253 (192888'14787) clean 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by osd.4.0:2426 2014-08-12 15:44:03.420859 2014-08-13 17:38:02.947752 7fe9bae887a0 20 read_log 194215'15254 (194215'15253) delete 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.2174595.0:104190 0.000000 2014-08-13 17:38:02.947763 7fe9bae887a0 20 read_log 194215'15255 (192888'14760) clean eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by osd.4.0:2428 2014-08-12 15:44:03.429002 2014-08-13 17:38:02.947775 7fe9bae887a0 20 read_log 194215'15256 (194215'15255) delete eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.2174595.0:104192 0.000000 2014-08-13 17:38:02.947785 7fe9bae887a0 20 read_log 194215'15257 (192888'14795) clean c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by osd.4.0:2430 2014-08-12 15:44:03.437346 2014-08-13 17:38:02.947797 7fe9bae887a0 20 read_log 194215'15258 (194215'15257) delete c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.2174595.0:104194 0.000000 2014-08-13 17:38:02.947808 7fe9bae887a0 20 read_log 194215'15259 (192888'14980) clean ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by osd.4.0:2432 2014-08-12 15:44:03.445462 2014-08-13 17:38:02.947846 7fe9bae887a0 20 read_log 194215'15260 (194215'15259) delete ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.2174595.0:104196 0.000000 2014-08-13 17:38:02.947865 7fe9bae887a0 20 read_log 194215'15261 (193305'15050) clean a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by osd.4.0:2434 2014-08-12 15:44:03.454869 2014-08-13 17:38:02.947878 7fe9bae887a0 20 read_log 194215'15262 (194215'15261) delete a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.2174595.0:104198 0.000000 2014-08-13 17:38:02.947888 7fe9bae887a0 20 read_log 194215'15263 (192834'14080) clean e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by osd.4.0:2436 2014-08-12 15:44:03.463002 2014-08-13 17:38:02.947901 7fe9bae887a0 20 read_log 194215'15264 (194215'15263) delete e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.2174595.0:104200 0.000000 2014-08-13 17:38:02.947911 7fe9bae887a0 20 read_log 194215'15265 (193305'15057) clean 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by osd.4.0:2438 2014-08-12 15:44:03.471356 2014-08-13 17:38:02.947923 7fe9bae887a0 20 read_log 194215'15266 (194215'15265) delete 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.2174595.0:104202 0.000000 2014-08-13 17:38:02.947934 7fe9bae887a0 20 read_log 194215'15267 (192888'14765) clean 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by osd.4.0:2440 2014-08-12 15:44:03.479308 2014-08-13 17:38:02.947946 7fe9bae887a0 20 read_log 194215'15268 (194215'15267) delete 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.2174595.0:104204 0.000000 2014-08-13 17:38:02.947956 7fe9bae887a0 20 read_log 194215'15269 (192888'14985) clean 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by osd.4.0:2442 2014-08-12 15:44:03.488081 2014-08-13 17:38:02.947968 7fe9bae887a0 20 read_log 194215'15270 (194215'15269) delete 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.2174595.0:104206 0.000000 2014-08-13 17:38:02.947979 7fe9bae887a0 20 read_log 194215'15271 (192834'14523) clean 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by osd.4.0:2444 2014-08-12 15:44:03.495459 2014-08-13 17:38:02.947991 7fe9bae887a0 20 read_log 194215'15272 (194215'15271) delete 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.2174595.0:104208 0.000000 2014-08-13 17:38:02.948001 7fe9bae887a0 20 read_log 194215'15273 (192834'13981) clean 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by osd.4.0:2446 2014-08-12 15:44:03.503297 2014-08-13 17:38:02.948013 7fe9bae887a0 20 read_log 194215'15274 (194215'15273) delete 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.2174595.0:104210 0.000000 2014-08-13 17:38:02.948024 7fe9bae887a0 20 read_log 194215'15275 (192888'14965) clean fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by osd.4.0:2448 2014-08-12 15:44:03.525853 2014-08-13 17:38:02.948036 7fe9bae887a0 20 read_log 194215'15276 (194215'15275) delete fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.2174595.0:104212 0.000000 2014-08-13 17:38:02.948046 7fe9bae887a0 20 read_log 194215'15277 (192888'14984) clean e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by osd.4.0:2450 2014-08-12 15:44:03.576684 2014-08-13 17:38:02.948058 7fe9bae887a0 20 read_log 194215'15278 (194215'15277) delete e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.2174595.0:104214 0.000000 2014-08-13 17:38:02.948069 7fe9bae887a0 20 read_log 194215'15279 (193305'15071) clean d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by osd.4.0:2452 2014-08-12 15:44:03.586079 2014-08-13 17:38:02.948081 7fe9bae887a0 20 read_log 194215'15280 (194215'15279) delete d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.2174595.0:104216 0.000000 2014-08-13 17:38:02.948101 7fe9bae887a0 20 read_log 194215'15281 (192834'14556) clean 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by osd.4.0:2454 2014-08-12 15:44:03.595278 2014-08-13 17:38:02.948114 7fe9bae887a0 20 read_log 194215'15282 (194215'15281) delete 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.2174595.0:104218 0.000000 2014-08-13 17:38:02.948124 7fe9bae887a0 20 read_log 194215'15283 (192888'14771) clean e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by osd.4.0:2456 2014-08-12 15:44:03.603813 2014-08-13 17:38:02.948136 7fe9bae887a0 20 read_log 194215'15284 (194215'15283) delete e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.2174595.0:104220 0.000000 2014-08-13 17:38:02.948147 7fe9bae887a0 20 read_log 194215'15285 (192888'14977) clean 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by osd.4.0:2458 2014-08-12 15:44:03.611762 2014-08-13 17:38:02.948166 7fe9bae887a0 20 read_log 194215'15286 (194215'15285) delete 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.2174595.0:104222 0.000000 2014-08-13 17:38:02.948177 7fe9bae887a0 20 read_log 194215'15287 (193305'15061) clean f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by osd.4.0:2460 2014-08-12 15:44:03.620225 2014-08-13 17:38:02.948190 7fe9bae887a0 20 read_log 194215'15288 (194215'15287) delete f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.2174595.0:104224 0.000000 2014-08-13 17:38:02.948200 7fe9bae887a0 20 read_log 194215'15289 (192834'14022) clean 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by osd.4.0:2462 2014-08-12 15:44:03.628456 2014-08-13 17:38:02.948212 7fe9bae887a0 20 read_log 194215'15290 (194215'15289) delete 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.2174595.0:104226 0.000000 2014-08-13 17:38:02.948223 7fe9bae887a0 20 read_log 194215'15291 (192888'14986) clean 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by osd.4.0:2464 2014-08-12 15:44:03.637126 2014-08-13 17:38:02.948235 7fe9bae887a0 20 read_log 194215'15292 (194215'15291) delete 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.2174595.0:104228 0.000000 2014-08-13 17:38:02.948246 7fe9bae887a0 20 read_log 194215'15293 (192888'14945) clean 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by osd.4.0:2466 2014-08-12 15:44:03.648734 2014-08-13 17:38:02.948258 7fe9bae887a0 20 read_log 194215'15294 (194215'15293) delete 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.2174595.0:104230 0.000000 2014-08-13 17:38:02.948269 7fe9bae887a0 20 read_log 194215'15295 (192888'14762) clean ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by osd.4.0:2468 2014-08-12 15:44:03.657030 2014-08-13 17:38:02.948281 7fe9bae887a0 20 read_log 194215'15296 (194215'15295) delete ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.2174595.0:104232 0.000000 2014-08-13 17:38:02.948291 7fe9bae887a0 20 read_log 194215'15297 (192888'14990) clean c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by osd.4.0:2470 2014-08-12 15:44:03.665812 2014-08-13 17:38:02.948303 7fe9bae887a0 20 read_log 194215'15298 (194215'15297) delete c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.2174595.0:104234 0.000000 2014-08-13 17:38:02.948314 7fe9bae887a0 20 read_log 194215'15299 (192888'14808) clean 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by osd.4.0:2472 2014-08-12 15:44:03.674249 2014-08-13 17:38:02.948325 7fe9bae887a0 20 read_log 194215'15300 (194215'15299) delete 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.2174595.0:104236 0.000000 2014-08-13 17:38:02.948343 7fe9bae887a0 20 read_log 194215'15301 (193305'15058) clean 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by osd.4.0:2474 2014-08-12 15:44:03.682645 2014-08-13 17:38:02.948356 7fe9bae887a0 20 read_log 194215'15302 (194215'15301) delete 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.2174595.0:104238 0.000000 2014-08-13 17:38:02.948367 7fe9bae887a0 20 read_log 194215'15303 (192834'14531) clean 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by osd.4.0:2476 2014-08-12 15:44:03.691028 2014-08-13 17:38:02.948379 7fe9bae887a0 20 read_log 194215'15304 (194215'15303) delete 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.2174595.0:104240 0.000000 2014-08-13 17:38:02.948389 7fe9bae887a0 20 read_log 194215'15305 (193305'15069) clean e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by osd.4.0:2478 2014-08-12 15:44:03.699811 2014-08-13 17:38:02.948402 7fe9bae887a0 20 read_log 194215'15306 (194215'15305) delete e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.2174595.0:104242 0.000000 2014-08-13 17:38:02.948412 7fe9bae887a0 20 read_log 194215'15307 (192834'13990) clean 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by osd.4.0:2480 2014-08-12 15:44:03.708085 2014-08-13 17:38:02.948424 7fe9bae887a0 20 read_log 194215'15308 (194215'15307) delete 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.2174595.0:104244 0.000000 2014-08-13 17:38:02.948435 7fe9bae887a0 20 read_log 194215'15309 (193305'15064) clean ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by osd.4.0:2482 2014-08-12 15:44:03.716547 2014-08-13 17:38:02.948447 7fe9bae887a0 20 read_log 194215'15310 (194215'15309) delete ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.2174595.0:104246 0.000000 2014-08-13 17:38:02.948457 7fe9bae887a0 20 read_log 194215'15311 (192834'13988) clean d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by osd.4.0:2484 2014-08-12 15:44:03.725946 2014-08-13 17:38:02.948469 7fe9bae887a0 20 read_log 194215'15312 (194215'15311) delete d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.2174595.0:104248 0.000000 2014-08-13 17:38:02.948479 7fe9bae887a0 20 read_log 194215'15313 (192888'14983) clean a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by osd.4.0:2486 2014-08-12 15:44:03.734490 2014-08-13 17:38:02.948491 7fe9bae887a0 20 read_log 194215'15314 (194215'15313) delete a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.2174595.0:104250 0.000000 2014-08-13 17:38:02.948502 7fe9bae887a0 20 read_log 194215'15315 (192834'14025) clean 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by osd.4.0:2488 2014-08-12 15:44:03.742220 2014-08-13 17:38:02.948513 7fe9bae887a0 20 read_log 194215'15316 (194215'15315) delete 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.2174595.0:104252 0.000000 2014-08-13 17:38:02.948524 7fe9bae887a0 20 read_log 194215'15317 (193305'15056) clean 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by osd.4.0:2490 2014-08-12 15:44:03.749699 2014-08-13 17:38:02.948536 7fe9bae887a0 20 read_log 194215'15318 (194215'15317) delete 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.2174595.0:104254 0.000000 2014-08-13 17:38:02.948546 7fe9bae887a0 20 read_log 194215'15319 (192888'14807) clean 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by osd.4.0:2492 2014-08-12 15:44:03.758293 2014-08-13 17:38:02.948558 7fe9bae887a0 20 read_log 194215'15320 (194215'15319) delete 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.2174595.0:104256 0.000000 2014-08-13 17:38:02.948577 7fe9bae887a0 20 read_log 194215'15321 (192888'14964) clean ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by osd.4.0:2494 2014-08-12 15:44:03.825292 2014-08-13 17:38:02.948589 7fe9bae887a0 20 read_log 194215'15322 (194215'15321) delete ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.2174595.0:104258 0.000000 2014-08-13 17:38:02.948600 7fe9bae887a0 20 read_log 194215'15323 (192834'13965) clean 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by osd.4.0:2496 2014-08-12 15:44:03.833042 2014-08-13 17:38:02.948613 7fe9bae887a0 20 read_log 194215'15324 (194215'15323) delete 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.2174595.0:104260 0.000000 2014-08-13 17:38:02.948623 7fe9bae887a0 20 read_log 194215'15325 (194215'15325) modify 1f7/hit_set_24.1f7_archive_2014-08-12 15:44:02.507949_2014-08-12 17:24:41.967015/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.967257 2014-08-13 17:38:02.948636 7fe9bae887a0 20 read_log 194215'15326 (194215'15073) delete 1f7/hit_set_24.1f7_archive_2014-08-12 15:32:25.864872_2014-08-12 15:44:02.507691/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.967257 2014-08-13 17:38:02.948649 7fe9bae887a0 20 read_log 194215'15327 (0'0) promote b9f149f7/benchmark_data_storage0111-ib.pocstor_34628_object520/head//24 by osd.4.0:5235 2014-08-12 17:24:41.971524 2014-08-13 17:38:02.948661 7fe9bae887a0 20 read_log 194215'15328 (194215'15327) modify b9f149f7/benchmark_data_storage0111-ib.pocstor_34628_object520/head//24 by client.2193759.0:521 2014-08-12 17:24:41.962038 2014-08-13 17:38:02.948673 7fe9bae887a0 20 read_log 194370'15329 (0'0) promote 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by osd.55.0:4020 2014-08-12 20:30:16.135870 2014-08-13 17:38:02.948685 7fe9bae887a0 20 read_log 194370'15330 (194370'15329) modify 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by client.1508531.0:462 2014-08-12 20:30:16.127883 2014-08-13 17:38:02.948697 7fe9bae887a0 20 read_log 194370'15331 (0'0) promote 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by osd.55.0:4026 2014-08-12 20:30:17.500712 2014-08-13 17:38:02.948709 7fe9bae887a0 20 read_log 194370'15332 (194370'15331) modify 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by client.1508531.0:827 2014-08-12 20:30:17.494531 2014-08-13 17:38:02.948721 7fe9bae887a0 20 read_log 194370'15333 (0'0) promote de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by osd.55.0:4028 2014-08-12 20:30:17.555150 2014-08-13 17:38:02.948733 7fe9bae887a0 20 read_log 194370'15334 (194370'15333) modify de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by client.1508531.0:842 2014-08-12 20:30:17.548893 2014-08-13 17:38:02.948745 7fe9bae887a0 20 read_log 194370'15335 (0'0) promote 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by osd.55.0:4038 2014-08-12 20:30:21.338572 2014-08-13 17:38:02.948756 7fe9bae887a0 20 read_log 194370'15336 (194370'15335) modify 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by client.1508531.0:1864 2014-08-12 20:30:21.330546 2014-08-13 17:38:02.948769 7fe9bae887a0 20 read_log 194370'15337 (0'0) promote 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by osd.55.0:4040 2014-08-12 20:30:23.057662 2014-08-13 17:38:02.948781 7fe9bae887a0 20 read_log 194370'15338 (194370'15337) modify 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by client.1508531.0:2315 2014-08-12 20:30:23.049895 2014-08-13 17:38:02.948793 7fe9bae887a0 20 read_log 194370'15339 (0'0) promote 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by osd.55.0:4042 2014-08-12 20:30:23.073993 2014-08-13 17:38:02.948814 7fe9bae887a0 20 read_log 194370'15340 (194370'15339) modify 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by client.1508531.0:2319 2014-08-12 20:30:23.067777 2014-08-13 17:38:02.948827 7fe9bae887a0 20 read_log 194370'15341 (0'0) promote 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by osd.55.0:4056 2014-08-12 20:30:25.398683 2014-08-13 17:38:02.948839 7fe9bae887a0 20 read_log 194370'15342 (194370'15341) modify 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by client.1508531.0:2960 2014-08-12 20:30:25.392825 2014-08-13 17:38:02.948851 7fe9bae887a0 20 read_log 194370'15343 (0'0) promote fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by osd.55.0:4072 2014-08-12 20:30:30.136613 2014-08-13 17:38:02.948863 7fe9bae887a0 20 read_log 194370'15344 (194370'15343) modify fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by client.1508531.0:4210 2014-08-12 20:30:30.127847 2014-08-13 17:38:02.948875 7fe9bae887a0 20 read_log 194370'15345 (0'0) promote 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by osd.55.0:4076 2014-08-12 20:30:31.136903 2014-08-13 17:38:02.948887 7fe9bae887a0 20 read_log 194370'15346 (194370'15345) modify 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by client.1508531.0:4487 2014-08-12 20:30:31.129603 2014-08-13 17:38:02.948899 7fe9bae887a0 20 read_log 194370'15347 (0'0) promote 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by osd.55.0:4098 2014-08-12 20:30:35.447295 2014-08-13 17:38:02.948911 7fe9bae887a0 20 read_log 194370'15348 (194370'15347) modify 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by client.1508531.0:5513 2014-08-12 20:30:35.439859 2014-08-13 17:38:02.948923 7fe9bae887a0 20 read_log 194370'15349 (0'0) promote 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by osd.55.0:4116 2014-08-12 20:30:41.806880 2014-08-13 17:38:02.948935 7fe9bae887a0 20 read_log 194370'15350 (194370'15349) modify 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by client.1508531.0:6878 2014-08-12 20:30:41.798480 2014-08-13 17:38:02.948947 7fe9bae887a0 20 read_log 194370'15351 (0'0) promote af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by osd.55.0:4170 2014-08-12 20:31:00.759786 2014-08-13 17:38:02.948959 7fe9bae887a0 20 read_log 194370'15352 (194370'15351) modify af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by client.1508531.0:10256 2014-08-12 20:31:00.751788 2014-08-13 17:38:02.948990 7fe9bae887a0 20 read_log 194370'15353 (0'0) promote 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by osd.55.0:4180 2014-08-12 20:31:06.788082 2014-08-13 17:38:02.949002 7fe9bae887a0 20 read_log 194370'15354 (194370'15353) modify 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by client.1508531.0:11249 2014-08-12 20:31:06.779426 2014-08-13 17:38:02.949015 7fe9bae887a0 20 read_log 194370'15355 (0'0) promote e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by osd.55.0:4188 2014-08-12 20:31:10.650071 2014-08-13 17:38:02.949027 7fe9bae887a0 20 read_log 194370'15356 (194370'15355) modify e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by client.1508531.0:11899 2014-08-12 20:31:10.642433 2014-08-13 17:38:02.949039 7fe9bae887a0 20 read_log 194370'15357 (0'0) promote fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by osd.55.0:4206 2014-08-12 20:31:23.668578 2014-08-13 17:38:02.949051 7fe9bae887a0 20 read_log 194370'15358 (194370'15357) modify fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by client.1508531.0:13451 2014-08-12 20:31:23.660793 2014-08-13 17:38:02.949063 7fe9bae887a0 20 read_log 194370'15359 (0'0) promote 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by osd.55.0:4208 2014-08-12 20:31:23.792998 2014-08-13 17:38:02.949125 7fe9bae887a0 20 read_log 194370'15360 (194370'15359) modify 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by client.1508531.0:13464 2014-08-12 20:31:23.785576 2014-08-13 17:38:02.949143 7fe9bae887a0 20 read_log 194377'15361 (0'0) promote de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by osd.55.0:4226 2014-08-12 20:33:01.996518 2014-08-13 17:38:02.949163 7fe9bae887a0 20 read_log 194377'15362 (194377'15361) modify de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by client.2231190.0:1185 2014-08-12 20:33:01.990904 2014-08-13 17:38:02.949176 7fe9bae887a0 20 read_log 194377'15363 (0'0) promote 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by osd.55.0:4238 2014-08-12 20:33:04.895801 2014-08-13 17:38:02.949189 7fe9bae887a0 20 read_log 194377'15364 (194377'15363) modify 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by client.2231190.0:1925 2014-08-12 20:33:04.888178 2014-08-13 17:38:02.949201 7fe9bae887a0 20 read_log 194377'15365 (0'0) promote 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by osd.55.0:4240 2014-08-12 20:33:06.121891 2014-08-13 17:38:02.949213 7fe9bae887a0 20 read_log 194377'15366 (194377'15365) modify 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by client.2231190.0:2216 2014-08-12 20:33:06.114672 2014-08-13 17:38:02.949225 7fe9bae887a0 20 read_log 194377'15367 (0'0) promote a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by osd.55.0:4272 2014-08-12 20:33:17.458328 2014-08-13 17:38:02.949236 7fe9bae887a0 20 read_log 194377'15368 (194377'15367) modify a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by client.2231190.0:4540 2014-08-12 20:33:17.449352 2014-08-13 17:38:02.949248 7fe9bae887a0 20 read_log 194377'15369 (0'0) promote d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by osd.55.0:4274 2014-08-12 20:33:17.788024 2014-08-13 17:38:02.949260 7fe9bae887a0 20 read_log 194377'15370 (194377'15369) modify d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by client.2231190.0:4583 2014-08-12 20:33:17.781221 2014-08-13 17:38:02.949272 7fe9bae887a0 20 read_log 194378'15371 (0'0) promote 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by osd.55.0:4278 2014-08-12 20:33:19.648517 2014-08-13 17:38:02.949284 7fe9bae887a0 20 read_log 194378'15372 (194378'15371) modify 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by client.2231190.0:4814 2014-08-12 20:33:19.607282 2014-08-13 17:38:02.949296 7fe9bae887a0 20 read_log 194383'15373 (0'0) promote 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by osd.55.0:4288 2014-08-12 20:34:02.044306 2014-08-13 17:38:02.949308 7fe9bae887a0 20 read_log 194383'15374 (194383'15373) modify 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by client.2231190.0:6122 2014-08-12 20:34:02.036743 2014-08-13 17:38:02.949322 7fe9bae887a0 20 read_log 194393'15375 (194393'15375) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:37:08.012293_2014-08-12 20:47:26.399042/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.399479 2014-08-13 17:38:02.949336 7fe9bae887a0 20 read_log 194393'15376 (194215'15325) delete 1f7/hit_set_24.1f7_archive_2014-08-12 15:44:02.507949_2014-08-12 17:24:41.967015/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.399479 2014-08-13 17:38:02.949348 7fe9bae887a0 20 read_log 194393'15377 (0'0) promote 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by osd.55.0:4301 2014-08-12 20:47:26.404510 2014-08-13 17:38:02.949360 7fe9bae887a0 20 read_log 194393'15378 (194393'15377) modify 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by client.1064095.0:105 2014-08-12 20:47:26.390525 2014-08-13 17:38:02.949381 7fe9bae887a0 20 read_log 194393'15379 (0'0) promote 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by osd.55.0:4309 2014-08-12 20:47:27.511552 2014-08-13 17:38:02.949393 7fe9bae887a0 20 read_log 194393'15380 (194393'15379) modify 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by client.1064095.0:364 2014-08-12 20:47:27.505831 2014-08-13 17:38:02.949406 7fe9bae887a0 20 read_log 194393'15381 (0'0) promote e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by osd.55.0:4318 2014-08-12 20:47:28.299641 2014-08-13 17:38:02.949418 7fe9bae887a0 20 read_log 194393'15382 (194393'15381) modify e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by client.1064095.0:544 2014-08-12 20:47:28.293346 2014-08-13 17:38:02.949430 7fe9bae887a0 20 read_log 194393'15383 (0'0) promote 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by osd.55.0:4327 2014-08-12 20:49:26.154719 2014-08-13 17:38:02.949441 7fe9bae887a0 20 read_log 194393'15384 (194393'15383) modify 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by client.1064182.0:146 2014-08-12 20:49:26.147080 2014-08-13 17:38:02.949453 7fe9bae887a0 20 read_log 194393'15385 (0'0) promote 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by osd.55.0:4329 2014-08-12 20:49:26.212573 2014-08-13 17:38:02.949465 7fe9bae887a0 20 read_log 194393'15386 (194393'15385) modify 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by client.1064182.0:159 2014-08-12 20:49:26.205261 2014-08-13 17:38:02.949481 7fe9bae887a0 20 read_log 194396'15387 (194396'15387) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:47:26.399484_2014-08-12 20:53:00.360133/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.360468 2014-08-13 17:38:02.949494 7fe9bae887a0 20 read_log 194396'15388 (194393'15375) delete 1f7/hit_set_24.1f7_archive_2014-08-12 20:37:08.012293_2014-08-12 20:47:26.399042/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.360468 2014-08-13 17:38:02.949506 7fe9bae887a0 20 read_log 194396'15389 (0'0) promote a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by osd.55.0:4353 2014-08-12 20:53:00.365713 2014-08-13 17:38:02.949518 7fe9bae887a0 20 read_log 194396'15390 (194396'15389) modify a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by client.2235069.0:309 2014-08-12 20:53:00.353284 2014-08-13 17:38:02.949530 7fe9bae887a0 20 read_log 194396'15391 (0'0) promote 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by osd.55.0:4355 2014-08-12 20:53:00.537643 2014-08-13 17:38:02.949542 7fe9bae887a0 20 read_log 194396'15392 (194396'15391) modify 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by client.2235069.0:341 2014-08-12 20:53:00.529259 2014-08-13 17:38:02.949554 7fe9bae887a0 20 read_log 194396'15393 (0'0) promote da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by osd.55.0:4359 2014-08-12 20:53:00.807310 2014-08-13 17:38:02.949566 7fe9bae887a0 20 read_log 194396'15394 (194396'15393) modify da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by client.2235069.0:406 2014-08-12 20:53:00.798343 2014-08-13 17:38:02.949578 7fe9bae887a0 20 read_log 194416'15395 (194416'15395) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 17:38:02.949590 7fe9bae887a0 20 read_log 194416'15396 (194396'15387) delete 1f7/hit_set_24.1f7_archive_2014-08-12 20:47:26.399484_2014-08-12 20:53:00.360133/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 17:38:02.949602 7fe9bae887a0 20 read_log 194416'15397 (0'0) promote 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by osd.55.0:4497 2014-08-13 15:28:31.203329 2014-08-13 17:38:02.949621 7fe9bae887a0 20 read_log 194416'15398 (194416'15397) modify 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by client.2441958.0:246 2014-08-13 15:28:31.194786 2014-08-13 17:38:02.949634 7fe9bae887a0 20 read_log 194416'15399 (0'0) promote 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by osd.55.0:4509 2014-08-13 15:28:34.820408 2014-08-13 17:38:02.949646 7fe9bae887a0 20 read_log 194416'15400 (194416'15399) modify 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by client.2441958.0:1080 2014-08-13 15:28:34.814563 2014-08-13 17:38:02.949658 7fe9bae887a0 20 read_log 194416'15401 (0'0) promote 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by osd.55.0:4521 2014-08-13 15:28:41.689171 2014-08-13 17:38:02.949669 7fe9bae887a0 20 read_log 194416'15402 (194416'15401) modify 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by client.2441958.0:2896 2014-08-13 15:28:41.679680 2014-08-13 17:38:02.949681 7fe9bae887a0 20 read_log 194416'15403 (0'0) promote 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by osd.55.0:4535 2014-08-13 15:28:45.234493 2014-08-13 17:38:02.949693 7fe9bae887a0 20 read_log 194416'15404 (194416'15403) modify 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by client.2441958.0:3815 2014-08-13 15:28:45.226875 2014-08-13 17:38:02.949705 7fe9bae887a0 20 read_log 194416'15405 (0'0) promote f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by osd.55.0:4539 2014-08-13 15:28:46.778918 2014-08-13 17:38:02.949717 7fe9bae887a0 20 read_log 194416'15406 (194416'15405) modify f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by client.2441958.0:4208 2014-08-13 15:28:46.771328 2014-08-13 17:38:02.949729 7fe9bae887a0 20 read_log 194416'15407 (0'0) promote 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by osd.55.0:4577 2014-08-13 15:28:56.287463 2014-08-13 17:38:02.949740 7fe9bae887a0 20 read_log 194416'15408 (194416'15407) modify 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by client.2441958.0:6599 2014-08-13 15:28:56.276634 2014-08-13 17:38:02.949752 7fe9bae887a0 20 read_log 194416'15409 (0'0) promote ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by osd.55.0:4585 2014-08-13 15:28:59.769628 2014-08-13 17:38:02.949764 7fe9bae887a0 20 read_log 194416'15410 (194416'15409) modify ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by client.2441958.0:7532 2014-08-13 15:28:59.762703 2014-08-13 17:38:02.949776 7fe9bae887a0 20 read_log 194416'15411 (0'0) promote 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by osd.55.0:4589 2014-08-13 15:29:00.834441 2014-08-13 17:38:02.949787 7fe9bae887a0 20 read_log 194416'15412 (194416'15411) modify 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by client.2441958.0:7815 2014-08-13 15:29:00.827799 2014-08-13 17:38:02.949799 7fe9bae887a0 20 read_log 194416'15413 (0'0) promote 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by osd.55.0:4595 2014-08-13 15:29:02.236837 2014-08-13 17:38:02.949811 7fe9bae887a0 20 read_log 194416'15414 (194416'15413) modify 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by client.2441958.0:8187 2014-08-13 15:29:02.229787 2014-08-13 17:38:02.949823 7fe9bae887a0 20 read_log 194416'15415 (0'0) promote 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by osd.55.0:4599 2014-08-13 15:29:03.181613 2014-08-13 17:38:02.949835 7fe9bae887a0 20 read_log 194416'15416 (194416'15415) modify 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by client.2441958.0:8436 2014-08-13 15:29:03.172525 2014-08-13 17:38:02.949847 7fe9bae887a0 20 read_log 194416'15417 (0'0) promote f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by osd.55.0:4601 2014-08-13 15:29:03.395038 2014-08-13 17:38:02.949866 7fe9bae887a0 20 read_log 194416'15418 (194416'15417) modify f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by client.2441958.0:8493 2014-08-13 15:29:03.387213 2014-08-13 17:38:02.949878 7fe9bae887a0 20 read_log 194416'15419 (0'0) promote 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by osd.55.0:4605 2014-08-13 15:29:05.181385 2014-08-13 17:38:02.949890 7fe9bae887a0 20 read_log 194416'15420 (194416'15419) modify 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by client.2441958.0:8935 2014-08-13 15:29:05.172817 2014-08-13 17:38:02.949902 7fe9bae887a0 20 read_log 194416'15421 (0'0) promote 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by osd.55.0:4607 2014-08-13 15:29:06.092684 2014-08-13 17:38:02.949914 7fe9bae887a0 20 read_log 194416'15422 (194416'15421) modify 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by client.2441958.0:9161 2014-08-13 15:29:06.085354 2014-08-13 17:38:02.949926 7fe9bae887a0 20 read_log 194416'15423 (0'0) promote 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by osd.55.0:4645 2014-08-13 15:29:15.558144 2014-08-13 17:38:02.949938 7fe9bae887a0 20 read_log 194416'15424 (194416'15423) modify 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by client.2441958.0:11513 2014-08-13 15:29:15.547610 2014-08-13 17:38:02.949950 7fe9bae887a0 20 read_log 194416'15425 (0'0) promote 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by osd.55.0:4671 2014-08-13 15:29:29.495715 2014-08-13 17:38:02.949962 7fe9bae887a0 20 read_log 194416'15426 (194416'15425) modify 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by client.2441958.0:14822 2014-08-13 15:29:29.486067 2014-08-13 17:38:02.949974 7fe9bae887a0 20 read_log 194416'15427 (0'0) promote d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by osd.55.0:4699 2014-08-13 15:29:48.541983 2014-08-13 17:38:02.949988 7fe9bae887a0 20 read_log 194416'15428 (194416'15427) modify d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by client.2441958.0:17084 2014-08-13 15:29:48.533177 2014-08-13 17:38:02.950001 7fe9bae887a0 20 read_log 194416'15429 (0'0) promote b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by osd.55.0:4701 2014-08-13 15:29:49.591754 2014-08-13 17:38:02.950013 7fe9bae887a0 20 read_log 194416'15430 (194416'15429) modify b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by client.2441958.0:17158 2014-08-13 15:29:49.582821 2014-08-13 17:38:02.950025 7fe9bae887a0 20 read_log 194416'15431 (0'0) promote 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by osd.55.0:4711 2014-08-13 15:29:58.502359 2014-08-13 17:38:02.950037 7fe9bae887a0 20 read_log 194416'15432 (194416'15431) modify 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by client.2441958.0:17792 2014-08-13 15:29:58.493248 2014-08-13 17:38:02.950049 7fe9bae887a0 20 read_log 194422'15433 (0'0) promote 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by osd.55.0:4719 2014-08-13 15:32:15.783067 2014-08-13 17:38:02.950061 7fe9bae887a0 20 read_log 194422'15434 (194422'15433) modify 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by client.1148203.0:175 2014-08-13 15:32:15.775000 2014-08-13 17:38:02.950073 7fe9bae887a0 20 read_log 194422'15435 (0'0) promote e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by osd.55.0:4721 2014-08-13 15:32:15.853831 2014-08-13 17:38:02.950085 7fe9bae887a0 20 read_log 194422'15436 (194422'15435) modify e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by client.1148203.0:188 2014-08-13 15:32:15.845579 2014-08-13 17:38:02.950097 7fe9bae887a0 20 read_log 194422'15437 (0'0) promote 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by osd.55.0:4757 2014-08-13 15:32:27.339857 2014-08-13 17:38:02.950115 7fe9bae887a0 20 read_log 194422'15438 (194422'15437) modify 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by client.1148203.0:3063 2014-08-13 15:32:27.331306 2014-08-13 17:38:02.950128 7fe9bae887a0 20 read_log 194422'15439 (0'0) promote b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by osd.55.0:4767 2014-08-13 15:32:29.204202 2014-08-13 17:38:02.950140 7fe9bae887a0 20 read_log 194422'15440 (194422'15439) modify b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by client.1148203.0:3528 2014-08-13 15:32:29.195794 2014-08-13 17:38:02.950152 7fe9bae887a0 20 read_log 194422'15441 (0'0) promote aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by osd.55.0:4773 2014-08-13 15:32:32.003065 2014-08-13 17:38:02.950170 7fe9bae887a0 20 read_log 194422'15442 (194422'15441) modify aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by client.1148203.0:4266 2014-08-13 15:32:31.993510 2014-08-13 17:38:02.950183 7fe9bae887a0 20 read_log 194422'15443 (0'0) promote 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by osd.55.0:4781 2014-08-13 15:32:39.266538 2014-08-13 17:38:02.950196 7fe9bae887a0 20 read_log 194422'15444 (194422'15443) modify 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by client.1148203.0:5841 2014-08-13 15:32:39.257570 2014-08-13 17:38:02.950208 7fe9bae887a0 20 read_log 194422'15445 (0'0) promote 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by osd.55.0:4783 2014-08-13 15:32:39.679792 2014-08-13 17:38:02.950239 7fe9bae887a0 20 read_log 194422'15446 (194422'15445) modify 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by client.1148203.0:5918 2014-08-13 15:32:39.669700 2014-08-13 17:38:02.950252 7fe9bae887a0 20 read_log 194422'15447 (0'0) promote be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by osd.55.0:4787 2014-08-13 15:32:41.686432 2014-08-13 17:38:02.950263 7fe9bae887a0 20 read_log 194422'15448 (194422'15447) modify be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by client.1148203.0:6280 2014-08-13 15:32:41.678072 2014-08-13 17:38:02.950275 7fe9bae887a0 20 read_log 194422'15449 (0'0) promote 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by osd.55.0:4789 2014-08-13 15:32:42.406474 2014-08-13 17:38:02.950287 7fe9bae887a0 20 read_log 194422'15450 (194422'15449) modify 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by client.1148203.0:6417 2014-08-13 15:32:42.397066 2014-08-13 17:38:02.950299 7fe9bae887a0 20 read_log 194422'15451 (0'0) promote ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by osd.55.0:4793 2014-08-13 15:32:44.530865 2014-08-13 17:38:02.950312 7fe9bae887a0 20 read_log 194422'15452 (194422'15451) modify ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by client.1148203.0:6823 2014-08-13 15:32:44.523271 2014-08-13 17:38:02.950325 7fe9bae887a0 20 read_log 194422'15453 (0'0) promote 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by osd.55.0:4816 2014-08-13 15:32:58.048819 2014-08-13 17:38:02.950337 7fe9bae887a0 20 read_log 194422'15454 (194422'15453) modify 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by client.1148203.0:8845 2014-08-13 15:32:58.038562 2014-08-13 17:38:02.950349 7fe9bae887a0 20 read_log 194422'15455 (0'0) promote 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by osd.55.0:4840 2014-08-13 15:33:18.693626 2014-08-13 17:38:02.950361 7fe9bae887a0 20 read_log 194422'15456 (194422'15455) modify 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by client.1148203.0:10309 2014-08-13 15:33:18.684448 2014-08-13 17:38:02.950378 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.950807 7fe9bae887a0 10 read_log checking for missing items over interval (194370'15329,194422'15456] 2014-08-13 17:38:02.950855 7fe9bae887a0 15 read_log missing 194422'15456 (194422'15455) modify 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by client.1148203.0:10309 2014-08-13 15:33:18.684448 2014-08-13 17:38:02.950896 7fe9bae887a0 15 read_log missing 194422'15454 (194422'15453) modify 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by client.1148203.0:8845 2014-08-13 15:32:58.038562 2014-08-13 17:38:02.950933 7fe9bae887a0 15 read_log missing 194422'15452 (194422'15451) modify ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by client.1148203.0:6823 2014-08-13 15:32:44.523271 2014-08-13 17:38:02.950970 7fe9bae887a0 15 read_log missing 194422'15450 (194422'15449) modify 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by client.1148203.0:6417 2014-08-13 15:32:42.397066 2014-08-13 17:38:02.951006 7fe9bae887a0 15 read_log missing 194422'15448 (194422'15447) modify be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by client.1148203.0:6280 2014-08-13 15:32:41.678072 2014-08-13 17:38:02.951043 7fe9bae887a0 15 read_log missing 194422'15446 (194422'15445) modify 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by client.1148203.0:5918 2014-08-13 15:32:39.669700 2014-08-13 17:38:02.951080 7fe9bae887a0 15 read_log missing 194422'15444 (194422'15443) modify 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by client.1148203.0:5841 2014-08-13 15:32:39.257570 2014-08-13 17:38:02.951116 7fe9bae887a0 15 read_log missing 194422'15442 (194422'15441) modify aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by client.1148203.0:4266 2014-08-13 15:32:31.993510 2014-08-13 17:38:02.951158 7fe9bae887a0 15 read_log missing 194422'15440 (194422'15439) modify b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by client.1148203.0:3528 2014-08-13 15:32:29.195794 2014-08-13 17:38:02.951197 7fe9bae887a0 15 read_log missing 194422'15438 (194422'15437) modify 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by client.1148203.0:3063 2014-08-13 15:32:27.331306 2014-08-13 17:38:02.951234 7fe9bae887a0 15 read_log missing 194422'15436 (194422'15435) modify e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by client.1148203.0:188 2014-08-13 15:32:15.845579 2014-08-13 17:38:02.951271 7fe9bae887a0 15 read_log missing 194422'15434 (194422'15433) modify 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by client.1148203.0:175 2014-08-13 15:32:15.775000 2014-08-13 17:38:02.951308 7fe9bae887a0 15 read_log missing 194416'15432 (194416'15431) modify 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by client.2441958.0:17792 2014-08-13 15:29:58.493248 2014-08-13 17:38:02.951349 7fe9bae887a0 15 read_log missing 194416'15430 (194416'15429) modify b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by client.2441958.0:17158 2014-08-13 15:29:49.582821 2014-08-13 17:38:02.951387 7fe9bae887a0 15 read_log missing 194416'15428 (194416'15427) modify d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by client.2441958.0:17084 2014-08-13 15:29:48.533177 2014-08-13 17:38:02.951432 7fe9bae887a0 15 read_log missing 194416'15426 (194416'15425) modify 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by client.2441958.0:14822 2014-08-13 15:29:29.486067 2014-08-13 17:38:02.951470 7fe9bae887a0 15 read_log missing 194416'15424 (194416'15423) modify 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by client.2441958.0:11513 2014-08-13 15:29:15.547610 2014-08-13 17:38:02.951507 7fe9bae887a0 15 read_log missing 194416'15422 (194416'15421) modify 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by client.2441958.0:9161 2014-08-13 15:29:06.085354 2014-08-13 17:38:02.951544 7fe9bae887a0 15 read_log missing 194416'15420 (194416'15419) modify 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by client.2441958.0:8935 2014-08-13 15:29:05.172817 2014-08-13 17:38:02.951581 7fe9bae887a0 15 read_log missing 194416'15418 (194416'15417) modify f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by client.2441958.0:8493 2014-08-13 15:29:03.387213 2014-08-13 17:38:02.951618 7fe9bae887a0 15 read_log missing 194416'15416 (194416'15415) modify 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by client.2441958.0:8436 2014-08-13 15:29:03.172525 2014-08-13 17:38:02.951654 7fe9bae887a0 15 read_log missing 194416'15414 (194416'15413) modify 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by client.2441958.0:8187 2014-08-13 15:29:02.229787 2014-08-13 17:38:02.951691 7fe9bae887a0 15 read_log missing 194416'15412 (194416'15411) modify 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by client.2441958.0:7815 2014-08-13 15:29:00.827799 2014-08-13 17:38:02.951728 7fe9bae887a0 15 read_log missing 194416'15410 (194416'15409) modify ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by client.2441958.0:7532 2014-08-13 15:28:59.762703 2014-08-13 17:38:02.951765 7fe9bae887a0 15 read_log missing 194416'15408 (194416'15407) modify 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by client.2441958.0:6599 2014-08-13 15:28:56.276634 2014-08-13 17:38:02.951802 7fe9bae887a0 15 read_log missing 194416'15406 (194416'15405) modify f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by client.2441958.0:4208 2014-08-13 15:28:46.771328 2014-08-13 17:38:02.951837 7fe9bae887a0 15 read_log missing 194416'15404 (194416'15403) modify 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by client.2441958.0:3815 2014-08-13 15:28:45.226875 2014-08-13 17:38:02.951871 7fe9bae887a0 15 read_log missing 194416'15402 (194416'15401) modify 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by client.2441958.0:2896 2014-08-13 15:28:41.679680 2014-08-13 17:38:02.951908 7fe9bae887a0 15 read_log missing 194416'15400 (194416'15399) modify 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by client.2441958.0:1080 2014-08-13 15:28:34.814563 2014-08-13 17:38:02.951945 7fe9bae887a0 15 read_log missing 194416'15398 (194416'15397) modify 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by client.2441958.0:246 2014-08-13 15:28:31.194786 2014-08-13 17:38:02.951985 7fe9bae887a0 15 read_log missing 194416'15395 (194416'15395) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 17:38:02.952022 7fe9bae887a0 15 read_log missing 194396'15394 (194396'15393) modify da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by client.2235069.0:406 2014-08-12 20:53:00.798343 2014-08-13 17:38:02.952057 7fe9bae887a0 15 read_log missing 194396'15392 (194396'15391) modify 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by client.2235069.0:341 2014-08-12 20:53:00.529259 2014-08-13 17:38:02.952094 7fe9bae887a0 15 read_log missing 194396'15390 (194396'15389) modify a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by client.2235069.0:309 2014-08-12 20:53:00.353284 2014-08-13 17:38:02.952133 7fe9bae887a0 15 read_log missing 194393'15386 (194393'15385) modify 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by client.1064182.0:159 2014-08-12 20:49:26.205261 2014-08-13 17:38:02.952176 7fe9bae887a0 15 read_log missing 194393'15384 (194393'15383) modify 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by client.1064182.0:146 2014-08-12 20:49:26.147080 2014-08-13 17:38:02.952214 7fe9bae887a0 15 read_log missing 194393'15382 (194393'15381) modify e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by client.1064095.0:544 2014-08-12 20:47:28.293346 2014-08-13 17:38:02.952250 7fe9bae887a0 15 read_log missing 194393'15380 (194393'15379) modify 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by client.1064095.0:364 2014-08-12 20:47:27.505831 2014-08-13 17:38:02.952287 7fe9bae887a0 15 read_log missing 194393'15378 (194393'15377) modify 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by client.1064095.0:105 2014-08-12 20:47:26.390525 2014-08-13 17:38:02.952328 7fe9bae887a0 15 read_log missing 194383'15374 (194383'15373) modify 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by client.2231190.0:6122 2014-08-12 20:34:02.036743 2014-08-13 17:38:02.952365 7fe9bae887a0 15 read_log missing 194378'15372 (194378'15371) modify 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by client.2231190.0:4814 2014-08-12 20:33:19.607282 2014-08-13 17:38:02.952401 7fe9bae887a0 15 read_log missing 194377'15370 (194377'15369) modify d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by client.2231190.0:4583 2014-08-12 20:33:17.781221 2014-08-13 17:38:02.952438 7fe9bae887a0 15 read_log missing 194377'15368 (194377'15367) modify a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by client.2231190.0:4540 2014-08-12 20:33:17.449352 2014-08-13 17:38:02.952474 7fe9bae887a0 15 read_log missing 194377'15366 (194377'15365) modify 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by client.2231190.0:2216 2014-08-12 20:33:06.114672 2014-08-13 17:38:02.952510 7fe9bae887a0 15 read_log missing 194377'15364 (194377'15363) modify 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by client.2231190.0:1925 2014-08-12 20:33:04.888178 2014-08-13 17:38:02.952548 7fe9bae887a0 15 read_log missing 194377'15362 (194377'15361) modify de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by client.2231190.0:1185 2014-08-12 20:33:01.990904 2014-08-13 17:38:02.952585 7fe9bae887a0 15 read_log missing 194370'15360 (194370'15359) modify 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by client.1508531.0:13464 2014-08-12 20:31:23.785576 2014-08-13 17:38:02.952622 7fe9bae887a0 15 read_log missing 194370'15358 (194370'15357) modify fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by client.1508531.0:13451 2014-08-12 20:31:23.660793 2014-08-13 17:38:02.952659 7fe9bae887a0 15 read_log missing 194370'15356 (194370'15355) modify e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by client.1508531.0:11899 2014-08-12 20:31:10.642433 2014-08-13 17:38:02.952695 7fe9bae887a0 15 read_log missing 194370'15354 (194370'15353) modify 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by client.1508531.0:11249 2014-08-12 20:31:06.779426 2014-08-13 17:38:02.952732 7fe9bae887a0 15 read_log missing 194370'15352 (194370'15351) modify af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by client.1508531.0:10256 2014-08-12 20:31:00.751788 2014-08-13 17:38:02.952769 7fe9bae887a0 15 read_log missing 194370'15350 (194370'15349) modify 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by client.1508531.0:6878 2014-08-12 20:30:41.798480 2014-08-13 17:38:02.952806 7fe9bae887a0 15 read_log missing 194370'15348 (194370'15347) modify 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by client.1508531.0:5513 2014-08-12 20:30:35.439859 2014-08-13 17:38:02.952843 7fe9bae887a0 15 read_log missing 194370'15346 (194370'15345) modify 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by client.1508531.0:4487 2014-08-12 20:30:31.129603 2014-08-13 17:38:02.952880 7fe9bae887a0 15 read_log missing 194370'15344 (194370'15343) modify fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by client.1508531.0:4210 2014-08-12 20:30:30.127847 2014-08-13 17:38:02.952917 7fe9bae887a0 15 read_log missing 194370'15342 (194370'15341) modify 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by client.1508531.0:2960 2014-08-12 20:30:25.392825 2014-08-13 17:38:02.952955 7fe9bae887a0 15 read_log missing 194370'15340 (194370'15339) modify 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by client.1508531.0:2319 2014-08-12 20:30:23.067777 2014-08-13 17:38:02.952991 7fe9bae887a0 15 read_log missing 194370'15338 (194370'15337) modify 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by client.1508531.0:2315 2014-08-12 20:30:23.049895 2014-08-13 17:38:02.953029 7fe9bae887a0 15 read_log missing 194370'15336 (194370'15335) modify 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by client.1508531.0:1864 2014-08-12 20:30:21.330546 2014-08-13 17:38:02.953065 7fe9bae887a0 15 read_log missing 194370'15334 (194370'15333) modify de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by client.1508531.0:842 2014-08-12 20:30:17.548893 2014-08-13 17:38:02.953102 7fe9bae887a0 15 read_log missing 194370'15332 (194370'15331) modify 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by client.1508531.0:827 2014-08-12 20:30:17.494531 2014-08-13 17:38:02.953138 7fe9bae887a0 15 read_log missing 194370'15330 (194370'15329) modify 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by client.1508531.0:462 2014-08-12 20:30:16.127883 2014-08-13 17:38:02.953150 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.953186 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_loaded 2014-08-13 17:38:02.953205 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Initial 0.015677 0 0.000000 2014-08-13 17:38:02.953218 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset 2014-08-13 17:38:02.953227 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194478 2014-08-13 17:38:02.953237 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] log((192718'12872,194422'15456], crt=194422'15456) 2014-08-13 17:38:02.953247 7fe9bae887a0 10 osd.4 194478 pgid 24.206 coll 24.206_head 2014-08-13 17:38:02.953369 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.206 2014-08-13 17:38:02.953382 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.953415 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.206(unlocked)] enter Initial 2014-08-13 17:38:02.953426 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.206(unlocked)] enter NotTrimming 2014-08-13 17:38:02.953549 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.953674 7fe9bae887a0 20 read_log 192834'12572 (0'0) promote 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.397.0:13176 2014-08-11 15:23:36.292094 2014-08-13 17:38:02.953695 7fe9bae887a0 20 read_log 192834'12573 (192834'12572) modify 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by client.933448.0:157 2014-08-11 15:23:36.283359 2014-08-13 17:38:02.953709 7fe9bae887a0 20 read_log 192834'12580 (0'0) promote 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by osd.397.0:13184 2014-08-11 15:23:38.977732 2014-08-13 17:38:02.953721 7fe9bae887a0 20 read_log 192834'12581 (192834'12580) modify 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.933448.0:867 2014-08-11 15:23:38.970238 2014-08-13 17:38:02.953733 7fe9bae887a0 20 read_log 192834'12582 (0'0) promote fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by osd.397.0:13186 2014-08-11 15:23:39.034300 2014-08-13 17:38:02.953745 7fe9bae887a0 20 read_log 192834'12583 (192834'12582) modify fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.933448.0:881 2014-08-11 15:23:39.025075 2014-08-13 17:38:02.953758 7fe9bae887a0 20 read_log 192834'12620 (0'0) promote 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by osd.397.0:13224 2014-08-11 15:23:56.441661 2014-08-13 17:38:02.953770 7fe9bae887a0 20 read_log 192834'12621 (192834'12620) modify 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.933448.0:4345 2014-08-11 15:23:56.433697 2014-08-13 17:38:02.953783 7fe9bae887a0 20 read_log 192834'12646 (0'0) promote f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.397.0:13250 2014-08-11 15:24:02.510034 2014-08-13 17:38:02.953795 7fe9bae887a0 20 read_log 192834'12647 (192834'12646) modify f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by client.933448.0:5688 2014-08-11 15:24:02.501955 2014-08-13 17:38:02.953807 7fe9bae887a0 20 read_log 192834'12654 (0'0) promote 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.397.0:13258 2014-08-11 15:24:04.016107 2014-08-13 17:38:02.953819 7fe9bae887a0 20 read_log 192834'12655 (192834'12654) modify 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by client.933448.0:6074 2014-08-11 15:24:04.008004 2014-08-13 17:38:02.953833 7fe9bae887a0 20 read_log 192834'12658 (0'0) promote ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by osd.397.0:13262 2014-08-11 15:24:05.085447 2014-08-13 17:38:02.953845 7fe9bae887a0 20 read_log 192834'12659 (192834'12658) modify ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.933448.0:6350 2014-08-11 15:24:05.076859 2014-08-13 17:38:02.953858 7fe9bae887a0 20 read_log 192834'12666 (0'0) promote ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by osd.397.0:13270 2014-08-11 15:24:06.726396 2014-08-13 17:38:02.953870 7fe9bae887a0 20 read_log 192834'12667 (192834'12666) modify ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.933448.0:6779 2014-08-11 15:24:06.718687 2014-08-13 17:38:02.953883 7fe9bae887a0 20 read_log 192834'12678 (0'0) promote ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by osd.397.0:13282 2014-08-11 15:24:09.396510 2014-08-13 17:38:02.953895 7fe9bae887a0 20 read_log 192834'12679 (192834'12678) modify ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.933448.0:7266 2014-08-11 15:24:09.388147 2014-08-13 17:38:02.953907 7fe9bae887a0 20 read_log 192834'12706 (0'0) promote b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by osd.397.0:13310 2014-08-11 15:24:17.829208 2014-08-13 17:38:02.953948 7fe9bae887a0 20 read_log 192834'12707 (192834'12706) modify b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.933448.0:9174 2014-08-11 15:24:17.821001 2014-08-13 17:38:02.953965 7fe9bae887a0 20 read_log 192834'12708 (0'0) promote f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by osd.397.0:13312 2014-08-11 15:24:18.263837 2014-08-13 17:38:02.953978 7fe9bae887a0 20 read_log 192834'12709 (192834'12708) modify f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.933448.0:9282 2014-08-11 15:24:18.255965 2014-08-13 17:38:02.953990 7fe9bae887a0 20 read_log 192834'12712 (0'0) promote c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.397.0:13316 2014-08-11 15:24:19.706646 2014-08-13 17:38:02.954002 7fe9bae887a0 20 read_log 192834'12713 (192834'12712) modify c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by client.933448.0:9658 2014-08-11 15:24:19.699716 2014-08-13 17:38:02.954014 7fe9bae887a0 20 read_log 192834'12740 (0'0) promote 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by osd.397.0:13344 2014-08-11 15:24:26.176671 2014-08-13 17:38:02.954026 7fe9bae887a0 20 read_log 192834'12741 (192834'12740) modify 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.933448.0:11027 2014-08-11 15:24:26.169559 2014-08-13 17:38:02.954039 7fe9bae887a0 20 read_log 192834'12744 (0'0) promote 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.397.0:13348 2014-08-11 15:24:27.355703 2014-08-13 17:38:02.954051 7fe9bae887a0 20 read_log 192834'12745 (192834'12744) modify 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by client.933448.0:11341 2014-08-11 15:24:27.347861 2014-08-13 17:38:02.954063 7fe9bae887a0 20 read_log 192834'12750 (0'0) promote 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by osd.397.0:13354 2014-08-11 15:24:29.279796 2014-08-13 17:38:02.954075 7fe9bae887a0 20 read_log 192834'12751 (192834'12750) modify 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.933448.0:11764 2014-08-11 15:24:29.270029 2014-08-13 17:38:02.954087 7fe9bae887a0 20 read_log 192834'12784 (0'0) promote 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.397.0:13388 2014-08-11 15:24:40.086023 2014-08-13 17:38:02.954099 7fe9bae887a0 20 read_log 192834'12785 (192834'12784) modify 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by client.933448.0:13822 2014-08-11 15:24:40.077718 2014-08-13 17:38:02.954111 7fe9bae887a0 20 read_log 192834'12820 (0'0) promote 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by osd.397.0:13424 2014-08-11 15:24:52.453670 2014-08-13 17:38:02.954123 7fe9bae887a0 20 read_log 192834'12821 (192834'12820) modify 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.933448.0:16041 2014-08-11 15:24:52.444616 2014-08-13 17:38:02.954135 7fe9bae887a0 20 read_log 192834'12824 (0'0) promote abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.397.0:13428 2014-08-11 15:24:54.442317 2014-08-13 17:38:02.954147 7fe9bae887a0 20 read_log 192834'12825 (192834'12824) modify abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by client.933448.0:16381 2014-08-11 15:24:54.431989 2014-08-13 17:38:02.954170 7fe9bae887a0 20 read_log 192834'12836 (0'0) promote 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by osd.397.0:13440 2014-08-11 15:25:00.926777 2014-08-13 17:38:02.954183 7fe9bae887a0 20 read_log 192834'12837 (192834'12836) modify 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.933448.0:17657 2014-08-11 15:25:00.919059 2014-08-13 17:38:02.954198 7fe9bae887a0 20 read_log 192834'12846 (0'0) promote 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by osd.397.0:13450 2014-08-11 15:25:03.279511 2014-08-13 17:38:02.954218 7fe9bae887a0 20 read_log 192834'12847 (192834'12846) modify 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.933448.0:18052 2014-08-11 15:25:03.270428 2014-08-13 17:38:02.954233 7fe9bae887a0 20 read_log 192834'12856 (0'0) promote b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by osd.397.0:13460 2014-08-11 15:25:12.768675 2014-08-13 17:38:02.954246 7fe9bae887a0 20 read_log 192834'12857 (192834'12856) modify b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.933448.0:19616 2014-08-11 15:25:12.758610 2014-08-13 17:38:02.954258 7fe9bae887a0 20 read_log 192834'12872 (0'0) promote f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by osd.397.0:13476 2014-08-11 15:25:16.449680 2014-08-13 17:38:02.954270 7fe9bae887a0 20 read_log 192834'12873 (192834'12872) modify f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.933448.0:20090 2014-08-11 15:25:16.442653 2014-08-13 17:38:02.954282 7fe9bae887a0 20 read_log 192834'12874 (0'0) promote 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by osd.397.0:13478 2014-08-11 15:25:20.427158 2014-08-13 17:38:02.954294 7fe9bae887a0 20 read_log 192834'12875 (192834'12874) modify 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.933448.0:20338 2014-08-11 15:25:20.416986 2014-08-13 17:38:02.954306 7fe9bae887a0 20 read_log 192834'12886 (0'0) promote f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.397.0:13490 2014-08-11 15:25:28.610796 2014-08-13 17:38:02.954318 7fe9bae887a0 20 read_log 192834'12887 (192834'12886) modify f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by client.933448.0:21341 2014-08-11 15:25:28.601922 2014-08-13 17:38:02.954330 7fe9bae887a0 20 read_log 192834'12898 (0'0) promote 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by osd.397.0:13502 2014-08-11 15:25:32.390453 2014-08-13 17:38:02.954342 7fe9bae887a0 20 read_log 192834'12899 (192834'12898) modify 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.933448.0:21859 2014-08-11 15:25:32.380445 2014-08-13 17:38:02.954354 7fe9bae887a0 20 read_log 192834'12905 (192834'12573) modify 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by client.933448.0:22448 2014-08-11 15:25:37.885867 2014-08-13 17:38:02.954366 7fe9bae887a0 20 read_log 192834'12909 (192834'12581) modify 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.933448.0:23158 2014-08-11 15:25:39.067040 2014-08-13 17:38:02.954378 7fe9bae887a0 20 read_log 192834'12910 (192834'12583) modify fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.933448.0:23172 2014-08-11 15:25:39.075022 2014-08-13 17:38:02.954390 7fe9bae887a0 20 read_log 192834'12929 (192834'12621) modify 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.933448.0:26636 2014-08-11 15:25:43.609441 2014-08-13 17:38:02.954402 7fe9bae887a0 20 read_log 192834'12942 (192834'12647) modify f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by client.933448.0:27979 2014-08-11 15:25:45.334539 2014-08-13 17:38:02.954414 7fe9bae887a0 20 read_log 192834'12946 (192834'12655) modify 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by client.933448.0:28365 2014-08-11 15:25:46.026239 2014-08-13 17:38:02.954426 7fe9bae887a0 20 read_log 192834'12948 (192834'12659) modify ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.933448.0:28641 2014-08-11 15:25:46.550910 2014-08-13 17:38:02.954439 7fe9bae887a0 20 read_log 192834'12952 (192834'12667) modify ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.933448.0:29070 2014-08-11 15:25:46.975386 2014-08-13 17:38:02.954451 7fe9bae887a0 20 read_log 192834'12958 (192834'12679) modify ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.933448.0:29557 2014-08-11 15:25:47.475069 2014-08-13 17:38:02.954473 7fe9bae887a0 20 read_log 192834'12972 (192834'12707) modify b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.933448.0:31465 2014-08-11 15:25:50.093790 2014-08-13 17:38:02.954489 7fe9bae887a0 20 read_log 192834'12973 (192834'12709) modify f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.933448.0:31573 2014-08-11 15:25:50.203460 2014-08-13 17:38:02.954501 7fe9bae887a0 20 read_log 192834'12975 (192834'12713) modify c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by client.933448.0:31949 2014-08-11 15:25:50.868576 2014-08-13 17:38:02.954513 7fe9bae887a0 20 read_log 192834'12989 (192834'12741) modify 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.933448.0:33318 2014-08-11 15:25:53.268093 2014-08-13 17:38:02.954526 7fe9bae887a0 20 read_log 192834'12991 (192834'12745) modify 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by client.933448.0:33632 2014-08-11 15:25:53.858207 2014-08-13 17:38:02.954538 7fe9bae887a0 20 read_log 192834'12994 (192834'12751) modify 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.933448.0:34055 2014-08-11 15:25:54.531476 2014-08-13 17:38:02.954550 7fe9bae887a0 20 read_log 192834'13011 (192834'12785) modify 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by client.933448.0:36113 2014-08-11 15:25:57.253491 2014-08-13 17:38:02.954562 7fe9bae887a0 20 read_log 192834'13029 (192834'12821) modify 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.933448.0:38332 2014-08-11 15:26:00.768615 2014-08-13 17:38:02.954574 7fe9bae887a0 20 read_log 192834'13031 (192834'12825) modify abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by client.933448.0:38672 2014-08-11 15:26:01.136462 2014-08-13 17:38:02.954586 7fe9bae887a0 20 read_log 192834'13037 (192834'12837) modify 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.933448.0:39948 2014-08-11 15:26:02.179821 2014-08-13 17:38:02.954598 7fe9bae887a0 20 read_log 192834'13042 (192834'12847) modify 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.933448.0:40343 2014-08-11 15:26:02.577424 2014-08-13 17:38:02.954610 7fe9bae887a0 20 read_log 192834'13047 (192834'12857) modify b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.933448.0:41907 2014-08-11 15:26:03.089624 2014-08-13 17:38:02.954622 7fe9bae887a0 20 read_log 192834'13055 (192834'12873) modify f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.933448.0:42381 2014-08-11 15:26:03.259615 2014-08-13 17:38:02.954633 7fe9bae887a0 20 read_log 192834'13056 (192834'12875) modify 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.933448.0:42629 2014-08-11 15:26:03.337575 2014-08-13 17:38:02.954645 7fe9bae887a0 20 read_log 192834'13062 (192834'12887) modify f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by client.933448.0:43632 2014-08-11 15:26:03.679997 2014-08-13 17:38:02.954657 7fe9bae887a0 20 read_log 192834'13068 (192834'12899) modify 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.933448.0:44150 2014-08-11 15:26:03.831226 2014-08-13 17:38:02.954669 7fe9bae887a0 20 read_log 192834'13087 (0'0) promote 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by osd.397.0:13691 2014-08-11 15:26:17.351914 2014-08-13 17:38:02.954682 7fe9bae887a0 20 read_log 192834'13088 (192834'13087) modify 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.1883634.0:786 2014-08-11 15:26:17.345072 2014-08-13 17:38:02.954713 7fe9bae887a0 20 read_log 192834'13107 (0'0) promote 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.397.0:13711 2014-08-11 15:26:24.429495 2014-08-13 17:38:02.954729 7fe9bae887a0 20 read_log 192834'13108 (192834'13107) modify 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by client.1883634.0:2708 2014-08-11 15:26:24.419719 2014-08-13 17:38:02.954750 7fe9bae887a0 20 read_log 192834'13113 (0'0) promote fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by osd.397.0:13717 2014-08-11 15:26:26.679019 2014-08-13 17:38:02.954764 7fe9bae887a0 20 read_log 192834'13114 (192834'13113) modify fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.1883634.0:3343 2014-08-11 15:26:26.668944 2014-08-13 17:38:02.954777 7fe9bae887a0 20 read_log 192834'13123 (0'0) promote 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.397.0:13727 2014-08-11 15:26:28.581488 2014-08-13 17:38:02.954789 7fe9bae887a0 20 read_log 192834'13124 (192834'13123) modify 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by client.1883634.0:3883 2014-08-11 15:26:28.573150 2014-08-13 17:38:02.954802 7fe9bae887a0 20 read_log 192834'13145 (0'0) promote deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by osd.397.0:13749 2014-08-11 15:26:35.340848 2014-08-13 17:38:02.954814 7fe9bae887a0 20 read_log 192834'13146 (192834'13145) modify deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.1883634.0:5701 2014-08-11 15:26:35.332635 2014-08-13 17:38:02.954826 7fe9bae887a0 20 read_log 192834'13161 (0'0) promote a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by osd.397.0:13765 2014-08-11 15:26:38.666266 2014-08-13 17:38:02.954838 7fe9bae887a0 20 read_log 192834'13162 (192834'13161) modify a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.1883634.0:6545 2014-08-11 15:26:38.658175 2014-08-13 17:38:02.954850 7fe9bae887a0 20 read_log 192834'13219 (0'0) promote ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by osd.397.0:13823 2014-08-11 15:26:50.117795 2014-08-13 17:38:02.954862 7fe9bae887a0 20 read_log 192834'13220 (192834'13219) modify ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.1883634.0:9261 2014-08-11 15:26:50.110444 2014-08-13 17:38:02.954874 7fe9bae887a0 20 read_log 192834'13225 (0'0) promote 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by osd.397.0:13829 2014-08-11 15:26:52.324894 2014-08-13 17:38:02.954886 7fe9bae887a0 20 read_log 192834'13226 (192834'13225) modify 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.1883634.0:9772 2014-08-11 15:26:52.314487 2014-08-13 17:38:02.954899 7fe9bae887a0 20 read_log 192834'13233 (0'0) promote a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.397.0:13837 2014-08-11 15:26:52.850147 2014-08-13 17:38:02.954912 7fe9bae887a0 20 read_log 192834'13234 (192834'13233) modify a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by client.1883634.0:9890 2014-08-11 15:26:52.840690 2014-08-13 17:38:02.954924 7fe9bae887a0 20 read_log 192834'13265 (0'0) promote f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by osd.397.0:13869 2014-08-11 15:27:03.128920 2014-08-13 17:38:02.954936 7fe9bae887a0 20 read_log 192834'13266 (192834'13265) modify f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.1883634.0:12233 2014-08-11 15:27:03.120081 2014-08-13 17:38:02.954948 7fe9bae887a0 20 read_log 192834'13301 (0'0) promote 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by osd.397.0:13905 2014-08-11 15:27:14.376058 2014-08-13 17:38:02.954961 7fe9bae887a0 20 read_log 192834'13302 (192834'13301) modify 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.1883634.0:14665 2014-08-11 15:27:14.366524 2014-08-13 17:38:02.954973 7fe9bae887a0 20 read_log 192834'13319 (0'0) promote bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.397.0:13923 2014-08-11 15:27:21.305538 2014-08-13 17:38:02.954985 7fe9bae887a0 20 read_log 192834'13320 (192834'13319) modify bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by client.1883634.0:15709 2014-08-11 15:27:21.296983 2014-08-13 17:38:02.955004 7fe9bae887a0 20 read_log 192834'13325 (0'0) promote f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by osd.397.0:13929 2014-08-11 15:27:22.771728 2014-08-13 17:38:02.955020 7fe9bae887a0 20 read_log 192834'13326 (192834'13325) modify f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.1883634.0:15912 2014-08-11 15:27:22.760464 2014-08-13 17:38:02.955032 7fe9bae887a0 20 read_log 192834'13335 (0'0) promote dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.397.0:13939 2014-08-11 15:27:27.720177 2014-08-13 17:38:02.955044 7fe9bae887a0 20 read_log 192834'13336 (192834'13335) modify dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by client.1883634.0:16473 2014-08-11 15:27:27.709759 2014-08-13 17:38:02.955056 7fe9bae887a0 20 read_log 192834'13337 (0'0) promote 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.397.0:13941 2014-08-11 15:27:27.957168 2014-08-13 17:38:02.955068 7fe9bae887a0 20 read_log 192834'13338 (192834'13337) modify 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by client.1883634.0:16516 2014-08-11 15:27:27.947366 2014-08-13 17:38:02.955080 7fe9bae887a0 20 read_log 192834'13353 (0'0) promote 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by osd.397.0:13957 2014-08-11 15:27:34.077274 2014-08-13 17:38:02.955092 7fe9bae887a0 20 read_log 192834'13354 (192834'13353) modify 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.1883634.0:17283 2014-08-11 15:27:34.064084 2014-08-13 17:38:02.955108 7fe9bae887a0 20 read_log 192834'13383 (0'0) promote cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.397.0:13987 2014-08-11 15:27:48.527285 2014-08-13 17:38:02.955120 7fe9bae887a0 20 read_log 192834'13384 (192834'13383) modify cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by client.1883634.0:18969 2014-08-11 15:27:48.519399 2014-08-13 17:38:02.955132 7fe9bae887a0 20 read_log 192834'13397 (0'0) promote 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.397.0:14001 2014-08-11 15:27:55.410558 2014-08-13 17:38:02.955144 7fe9bae887a0 20 read_log 192834'13398 (192834'13397) modify 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by client.1883634.0:20138 2014-08-11 15:27:55.397145 2014-08-13 17:38:02.955161 7fe9bae887a0 20 read_log 192834'13401 (0'0) promote ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.397.0:14005 2014-08-11 15:27:55.798071 2014-08-13 17:38:02.955174 7fe9bae887a0 20 read_log 192834'13402 (192834'13401) modify ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by client.1883634.0:20232 2014-08-11 15:27:55.785793 2014-08-13 17:38:02.955186 7fe9bae887a0 20 read_log 192834'13405 (0'0) promote ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.397.0:14009 2014-08-11 15:28:01.367752 2014-08-13 17:38:02.955198 7fe9bae887a0 20 read_log 192834'13406 (192834'13405) modify ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by client.1883634.0:20749 2014-08-11 15:28:01.359528 2014-08-13 17:38:02.955210 7fe9bae887a0 20 read_log 192834'13409 (0'0) promote c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by osd.397.0:14013 2014-08-11 15:28:02.729265 2014-08-13 17:38:02.955222 7fe9bae887a0 20 read_log 192834'13410 (192834'13409) modify c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.1883634.0:21087 2014-08-11 15:28:02.716417 2014-08-13 17:38:02.955234 7fe9bae887a0 20 read_log 192834'13415 (0'0) promote 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by osd.397.0:14019 2014-08-11 15:28:04.230688 2014-08-13 17:38:02.955246 7fe9bae887a0 20 read_log 192834'13416 (192834'13415) modify 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.1883634.0:21454 2014-08-11 15:28:04.219970 2014-08-13 17:38:02.955269 7fe9bae887a0 20 read_log 192834'13421 (0'0) promote d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by osd.397.0:14025 2014-08-11 15:28:04.968324 2014-08-13 17:38:02.955282 7fe9bae887a0 20 read_log 192834'13422 (192834'13421) modify d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.1883634.0:21688 2014-08-11 15:28:04.959134 2014-08-13 17:38:02.955294 7fe9bae887a0 20 read_log 192834'13433 (0'0) promote eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by osd.397.0:14037 2014-08-11 15:28:11.421057 2014-08-13 17:38:02.955306 7fe9bae887a0 20 read_log 192834'13434 (192834'13433) modify eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.1883634.0:22855 2014-08-11 15:28:11.406764 2014-08-13 17:38:02.955318 7fe9bae887a0 20 read_log 192834'13445 (192834'13088) modify 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.1883634.0:24110 2014-08-11 15:28:18.366418 2014-08-13 17:38:02.955330 7fe9bae887a0 20 read_log 192834'13455 (192834'13108) modify 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by client.1883634.0:26032 2014-08-11 15:28:22.526748 2014-08-13 17:38:02.955342 7fe9bae887a0 20 read_log 192834'13458 (192834'13114) modify fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.1883634.0:26667 2014-08-11 15:28:23.391700 2014-08-13 17:38:02.955354 7fe9bae887a0 20 read_log 192834'13463 (192834'13124) modify 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by client.1883634.0:27207 2014-08-11 15:28:24.193205 2014-08-13 17:38:02.955367 7fe9bae887a0 20 read_log 192834'13474 (192834'13146) modify deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.1883634.0:29025 2014-08-11 15:28:26.639314 2014-08-13 17:38:02.955378 7fe9bae887a0 20 read_log 192834'13482 (192834'13162) modify a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.1883634.0:29869 2014-08-11 15:28:27.773696 2014-08-13 17:38:02.955390 7fe9bae887a0 20 read_log 192834'13511 (192834'13220) modify ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.1883634.0:32585 2014-08-11 15:28:32.049640 2014-08-13 17:38:02.955402 7fe9bae887a0 20 read_log 192834'13514 (192834'13226) modify 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.1883634.0:33096 2014-08-11 15:28:32.772315 2014-08-13 17:38:02.955414 7fe9bae887a0 20 read_log 192834'13518 (192834'13234) modify a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by client.1883634.0:33214 2014-08-11 15:28:32.906792 2014-08-13 17:38:02.955426 7fe9bae887a0 20 read_log 192834'13536 (192834'13266) modify f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.1883634.0:35557 2014-08-11 15:28:36.826573 2014-08-13 17:38:02.955438 7fe9bae887a0 20 read_log 192834'13554 (192834'13302) modify 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.1883634.0:37989 2014-08-11 15:28:40.065714 2014-08-13 17:38:02.955450 7fe9bae887a0 20 read_log 192834'13563 (192834'13320) modify bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by client.1883634.0:39033 2014-08-11 15:28:41.502004 2014-08-13 17:38:02.955462 7fe9bae887a0 20 read_log 192834'13566 (192834'13326) modify f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.1883634.0:39236 2014-08-11 15:28:41.898560 2014-08-13 17:38:02.955474 7fe9bae887a0 20 read_log 192834'13571 (192834'13336) modify dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by client.1883634.0:39797 2014-08-11 15:28:42.585177 2014-08-13 17:38:02.955486 7fe9bae887a0 20 read_log 192834'13572 (192834'13338) modify 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by client.1883634.0:39840 2014-08-11 15:28:42.645135 2014-08-13 17:38:02.955498 7fe9bae887a0 20 read_log 192834'13580 (192834'13354) modify 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.1883634.0:40607 2014-08-11 15:28:44.047491 2014-08-13 17:38:02.955518 7fe9bae887a0 20 read_log 192834'13595 (192834'13384) modify cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by client.1883634.0:42293 2014-08-11 15:28:46.199551 2014-08-13 17:38:02.955530 7fe9bae887a0 20 read_log 192834'13602 (192834'13398) modify 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by client.1883634.0:43462 2014-08-11 15:28:46.562106 2014-08-13 17:38:02.955543 7fe9bae887a0 20 read_log 192834'13604 (192834'13402) modify ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by client.1883634.0:43556 2014-08-11 15:28:46.590186 2014-08-13 17:38:02.955554 7fe9bae887a0 20 read_log 192834'13606 (192834'13406) modify ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by client.1883634.0:44073 2014-08-11 15:28:46.752107 2014-08-13 17:38:02.955566 7fe9bae887a0 20 read_log 192834'13608 (192834'13410) modify c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.1883634.0:44411 2014-08-11 15:28:46.851919 2014-08-13 17:38:02.955578 7fe9bae887a0 20 read_log 192834'13611 (192834'13416) modify 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.1883634.0:44778 2014-08-11 15:28:46.954960 2014-08-13 17:38:02.955590 7fe9bae887a0 20 read_log 192834'13614 (192834'13422) modify d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.1883634.0:45012 2014-08-11 15:28:47.121415 2014-08-13 17:38:02.955602 7fe9bae887a0 20 read_log 192834'13620 (192834'13434) modify eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.1883634.0:46179 2014-08-11 15:28:47.513603 2014-08-13 17:38:02.955614 7fe9bae887a0 20 read_log 192888'13624 (0'0) promote d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.397.0:14464 2014-08-11 16:44:54.982436 2014-08-13 17:38:02.955626 7fe9bae887a0 20 read_log 192888'13625 (192888'13624) modify d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by client.1315259.0:29 2014-08-11 16:44:54.969680 2014-08-13 17:38:02.955638 7fe9bae887a0 20 read_log 192888'13626 (0'0) promote 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by osd.397.0:14466 2014-08-11 16:44:57.675315 2014-08-13 17:38:02.955650 7fe9bae887a0 20 read_log 192888'13627 (192888'13626) modify 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.1315259.0:630 2014-08-11 16:44:57.666114 2014-08-13 17:38:02.955662 7fe9bae887a0 20 read_log 192888'13630 (0'0) promote 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by osd.397.0:14470 2014-08-11 16:44:57.814678 2014-08-13 17:38:02.955673 7fe9bae887a0 20 read_log 192888'13631 (192888'13630) modify 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.1315259.0:666 2014-08-11 16:44:57.805689 2014-08-13 17:38:02.955706 7fe9bae887a0 20 read_log 192888'13636 (0'0) promote a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by osd.397.0:14476 2014-08-11 16:45:06.280489 2014-08-13 17:38:02.955720 7fe9bae887a0 20 read_log 192888'13637 (192888'13636) modify a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.1315259.0:2918 2014-08-11 16:45:06.271296 2014-08-13 17:38:02.955732 7fe9bae887a0 20 read_log 192888'13638 (0'0) promote 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by osd.397.0:14478 2014-08-11 16:45:11.281264 2014-08-13 17:38:02.955744 7fe9bae887a0 20 read_log 192888'13639 (192888'13638) modify 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.1315259.0:4225 2014-08-11 16:45:11.271335 2014-08-13 17:38:02.955756 7fe9bae887a0 20 read_log 192888'13640 (0'0) promote c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by osd.397.0:14480 2014-08-11 16:45:14.341911 2014-08-13 17:38:02.955768 7fe9bae887a0 20 read_log 192888'13641 (192888'13640) modify c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.1315259.0:5055 2014-08-11 16:45:14.332248 2014-08-13 17:38:02.955813 7fe9bae887a0 20 read_log 192888'13658 (0'0) promote 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by osd.397.0:14498 2014-08-11 16:45:33.512633 2014-08-13 17:38:02.955830 7fe9bae887a0 20 read_log 192888'13659 (192888'13658) modify 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.1315259.0:10230 2014-08-11 16:45:33.500817 2014-08-13 17:38:02.955842 7fe9bae887a0 20 read_log 192888'13662 (0'0) promote d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.397.0:14502 2014-08-11 16:45:35.311398 2014-08-13 17:38:02.955854 7fe9bae887a0 20 read_log 192888'13663 (192888'13662) modify d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by client.1315259.0:10673 2014-08-11 16:45:35.298624 2014-08-13 17:38:02.955866 7fe9bae887a0 20 read_log 192888'13664 (0'0) promote 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by osd.397.0:14504 2014-08-11 16:45:37.571848 2014-08-13 17:38:02.955878 7fe9bae887a0 20 read_log 192888'13665 (192888'13664) modify 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.1315259.0:11237 2014-08-11 16:45:37.562112 2014-08-13 17:38:02.955890 7fe9bae887a0 20 read_log 192888'13672 (0'0) promote ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by osd.397.0:14512 2014-08-11 16:45:45.355405 2014-08-13 17:38:02.955902 7fe9bae887a0 20 read_log 192888'13673 (192888'13672) modify ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.1315259.0:13125 2014-08-11 16:45:45.347675 2014-08-13 17:38:02.955915 7fe9bae887a0 20 read_log 192888'13676 (0'0) promote 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by osd.397.0:14516 2014-08-11 16:45:48.044757 2014-08-13 17:38:02.955927 7fe9bae887a0 20 read_log 192888'13677 (192888'13676) modify 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.1315259.0:13803 2014-08-11 16:45:48.033140 2014-08-13 17:38:02.955939 7fe9bae887a0 20 read_log 192888'13680 (0'0) promote 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by osd.397.0:14520 2014-08-11 16:45:50.763016 2014-08-13 17:38:02.955967 7fe9bae887a0 20 read_log 192888'13681 (192888'13680) modify 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.1315259.0:14518 2014-08-11 16:45:50.753878 2014-08-13 17:38:02.955982 7fe9bae887a0 20 read_log 192888'13682 (0'0) promote dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by osd.397.0:14522 2014-08-11 16:45:50.951650 2014-08-13 17:38:02.955994 7fe9bae887a0 20 read_log 192888'13683 (192888'13682) modify dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.1315259.0:14569 2014-08-11 16:45:50.940855 2014-08-13 17:38:02.956006 7fe9bae887a0 20 read_log 192888'13684 (0'0) promote 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by osd.397.0:14524 2014-08-11 16:45:56.834501 2014-08-13 17:38:02.956018 7fe9bae887a0 20 read_log 192888'13685 (192888'13684) modify 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.1315259.0:16106 2014-08-11 16:45:56.820910 2014-08-13 17:38:02.956030 7fe9bae887a0 20 read_log 192888'13686 (0'0) promote 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.397.0:14526 2014-08-11 16:45:58.442984 2014-08-13 17:38:02.956042 7fe9bae887a0 20 read_log 192888'13687 (192888'13686) modify 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by client.1315259.0:16516 2014-08-11 16:45:58.433557 2014-08-13 17:38:02.956055 7fe9bae887a0 20 read_log 192888'13688 (0'0) promote 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.397.0:14528 2014-08-11 16:45:59.989603 2014-08-13 17:38:02.956067 7fe9bae887a0 20 read_log 192888'13689 (192888'13688) modify 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by client.1315259.0:16918 2014-08-11 16:45:59.981347 2014-08-13 17:38:02.956088 7fe9bae887a0 20 read_log 192888'13692 (0'0) promote 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.397.0:14532 2014-08-11 16:46:01.196953 2014-08-13 17:38:02.956104 7fe9bae887a0 20 read_log 192888'13693 (192888'13692) modify 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by client.1315259.0:17220 2014-08-11 16:46:01.187898 2014-08-13 17:38:02.956116 7fe9bae887a0 20 read_log 192888'13698 (0'0) promote 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by osd.397.0:14538 2014-08-11 16:46:05.866780 2014-08-13 17:38:02.956128 7fe9bae887a0 20 read_log 192888'13699 (192888'13698) modify 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.1315259.0:18387 2014-08-11 16:46:05.857153 2014-08-13 17:38:02.956140 7fe9bae887a0 20 read_log 192888'13702 (0'0) promote e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.397.0:14542 2014-08-11 16:46:09.726901 2014-08-13 17:38:02.956152 7fe9bae887a0 20 read_log 192888'13703 (192888'13702) modify e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by client.1315259.0:19351 2014-08-11 16:46:09.717207 2014-08-13 17:38:02.956170 7fe9bae887a0 20 read_log 192888'13704 (0'0) promote 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by osd.397.0:14544 2014-08-11 16:46:10.423331 2014-08-13 17:38:02.956183 7fe9bae887a0 20 read_log 192888'13705 (192888'13704) modify 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.1315259.0:19537 2014-08-11 16:46:10.411872 2014-08-13 17:38:02.956195 7fe9bae887a0 20 read_log 192888'13706 (0'0) promote dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by osd.397.0:14546 2014-08-11 16:46:12.479490 2014-08-13 17:38:02.956207 7fe9bae887a0 20 read_log 192888'13707 (192888'13706) modify dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.1315259.0:20098 2014-08-11 16:46:12.471540 2014-08-13 17:38:02.956219 7fe9bae887a0 20 read_log 192888'13708 (0'0) promote e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.397.0:14548 2014-08-11 16:46:13.653648 2014-08-13 17:38:02.956231 7fe9bae887a0 20 read_log 192888'13709 (192888'13708) modify e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by client.1315259.0:20417 2014-08-11 16:46:13.645500 2014-08-13 17:38:02.956243 7fe9bae887a0 20 read_log 192888'13712 (0'0) promote 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by osd.397.0:14552 2014-08-11 16:46:18.608838 2014-08-13 17:38:02.956255 7fe9bae887a0 20 read_log 192888'13713 (192888'13712) modify 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.1315259.0:21718 2014-08-11 16:46:18.599272 2014-08-13 17:38:02.956267 7fe9bae887a0 20 read_log 192888'13714 (0'0) promote c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by osd.397.0:14554 2014-08-11 16:46:20.227503 2014-08-13 17:38:02.956278 7fe9bae887a0 20 read_log 192888'13715 (192888'13714) modify c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.1315259.0:22134 2014-08-11 16:46:20.218526 2014-08-13 17:38:02.956291 7fe9bae887a0 20 read_log 192888'13718 (0'0) promote da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.397.0:14558 2014-08-11 16:46:22.825005 2014-08-13 17:38:02.956303 7fe9bae887a0 20 read_log 192888'13719 (192888'13718) modify da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by client.1315259.0:22807 2014-08-11 16:46:22.814296 2014-08-13 17:38:02.956315 7fe9bae887a0 20 read_log 192888'13724 (0'0) promote a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.397.0:14564 2014-08-11 16:46:34.518209 2014-08-13 17:38:02.956327 7fe9bae887a0 20 read_log 192888'13725 (192888'13724) modify a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by client.1315259.0:25731 2014-08-11 16:46:34.507619 2014-08-13 17:38:02.956346 7fe9bae887a0 20 read_log 192888'13728 (0'0) promote 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by osd.397.0:14568 2014-08-11 16:46:43.153881 2014-08-13 17:38:02.956359 7fe9bae887a0 20 read_log 192888'13729 (192888'13728) modify 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.1315259.0:27792 2014-08-11 16:46:43.143223 2014-08-13 17:38:02.956373 7fe9bae887a0 20 read_log 192888'13730 (0'0) promote b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by osd.397.0:14570 2014-08-11 16:46:43.163899 2014-08-13 17:38:02.956385 7fe9bae887a0 20 read_log 192888'13731 (192888'13730) modify b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.1315259.0:27794 2014-08-11 16:46:43.151411 2014-08-13 17:38:02.956397 7fe9bae887a0 20 read_log 192888'13732 (0'0) promote f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by osd.397.0:14572 2014-08-11 16:46:43.270125 2014-08-13 17:38:02.956409 7fe9bae887a0 20 read_log 192888'13733 (192888'13732) modify f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.1315259.0:27822 2014-08-11 16:46:43.259760 2014-08-13 17:38:02.956421 7fe9bae887a0 20 read_log 192888'13734 (0'0) promote d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by osd.397.0:14574 2014-08-11 16:46:43.492662 2014-08-13 17:38:02.956433 7fe9bae887a0 20 read_log 192888'13735 (192888'13734) modify d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.1315259.0:27879 2014-08-11 16:46:43.481843 2014-08-13 17:38:02.956445 7fe9bae887a0 20 read_log 192888'13736 (0'0) promote bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by osd.397.0:14576 2014-08-11 16:46:44.566136 2014-08-13 17:38:02.956457 7fe9bae887a0 20 read_log 192888'13737 (192888'13736) modify bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.1315259.0:28156 2014-08-11 16:46:44.559270 2014-08-13 17:38:02.956469 7fe9bae887a0 20 read_log 192888'13740 (0'0) promote df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by osd.397.0:14580 2014-08-11 16:46:50.336045 2014-08-13 17:38:02.956481 7fe9bae887a0 20 read_log 192888'13741 (192888'13740) modify df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.1315259.0:29550 2014-08-11 16:46:50.327553 2014-08-13 17:38:02.956493 7fe9bae887a0 20 read_log 192888'13742 (0'0) promote 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by osd.397.0:14582 2014-08-11 16:46:51.076923 2014-08-13 17:38:02.956505 7fe9bae887a0 20 read_log 192888'13743 (192888'13742) modify 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.1315259.0:29725 2014-08-11 16:46:51.067525 2014-08-13 17:38:02.956517 7fe9bae887a0 20 read_log 192888'13752 (192888'13625) modify d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by client.1315259.0:30756 2014-08-11 16:46:55.877658 2014-08-13 17:38:02.956529 7fe9bae887a0 20 read_log 192888'13753 (192888'13627) modify 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.1315259.0:31357 2014-08-11 16:46:56.070590 2014-08-13 17:38:02.956541 7fe9bae887a0 20 read_log 192888'13755 (192888'13631) modify 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.1315259.0:31393 2014-08-11 16:46:56.080061 2014-08-13 17:38:02.956553 7fe9bae887a0 20 read_log 192888'13758 (192888'13637) modify a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.1315259.0:33645 2014-08-11 16:46:56.688863 2014-08-13 17:38:02.956565 7fe9bae887a0 20 read_log 192888'13759 (192888'13639) modify 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.1315259.0:34952 2014-08-11 16:46:57.028595 2014-08-13 17:38:02.956577 7fe9bae887a0 20 read_log 192888'13760 (192888'13641) modify c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.1315259.0:35782 2014-08-11 16:46:57.244721 2014-08-13 17:38:02.956596 7fe9bae887a0 20 read_log 192888'13769 (192888'13659) modify 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.1315259.0:40957 2014-08-11 16:46:58.624706 2014-08-13 17:38:02.956609 7fe9bae887a0 20 read_log 192888'13771 (192888'13663) modify d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by client.1315259.0:41400 2014-08-11 16:46:58.736545 2014-08-13 17:38:02.956621 7fe9bae887a0 20 read_log 192888'13772 (192888'13665) modify 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.1315259.0:41964 2014-08-11 16:46:58.882090 2014-08-13 17:38:02.956633 7fe9bae887a0 20 read_log 192888'13776 (192888'13673) modify ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.1315259.0:43852 2014-08-11 16:46:59.410666 2014-08-13 17:38:02.956646 7fe9bae887a0 20 read_log 192888'13778 (192888'13677) modify 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.1315259.0:44530 2014-08-11 16:46:59.603721 2014-08-13 17:38:02.956658 7fe9bae887a0 20 read_log 192888'13780 (192888'13681) modify 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.1315259.0:45245 2014-08-11 16:46:59.811513 2014-08-13 17:38:02.956670 7fe9bae887a0 20 read_log 192888'13781 (192888'13683) modify dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.1315259.0:45296 2014-08-11 16:46:59.829908 2014-08-13 17:38:02.956681 7fe9bae887a0 20 read_log 192888'13782 (192888'13685) modify 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.1315259.0:46833 2014-08-11 16:47:00.270962 2014-08-13 17:38:02.956693 7fe9bae887a0 20 read_log 192888'13783 (192888'13687) modify 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by client.1315259.0:47243 2014-08-11 16:47:00.393381 2014-08-13 17:38:02.956705 7fe9bae887a0 20 read_log 192888'13784 (192888'13689) modify 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by client.1315259.0:47645 2014-08-11 16:47:00.522645 2014-08-13 17:38:02.956717 7fe9bae887a0 20 read_log 192888'13786 (192888'13693) modify 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by client.1315259.0:47947 2014-08-11 16:47:00.618658 2014-08-13 17:38:02.956729 7fe9bae887a0 20 read_log 192888'13789 (192888'13699) modify 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.1315259.0:49114 2014-08-11 16:47:00.962539 2014-08-13 17:38:02.956740 7fe9bae887a0 20 read_log 192888'13791 (192888'13703) modify e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by client.1315259.0:50078 2014-08-11 16:47:01.256226 2014-08-13 17:38:02.956752 7fe9bae887a0 20 read_log 192888'13792 (192888'13705) modify 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.1315259.0:50264 2014-08-11 16:47:01.314121 2014-08-13 17:38:02.956764 7fe9bae887a0 20 read_log 192888'13793 (192888'13707) modify dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.1315259.0:50825 2014-08-11 16:47:01.489217 2014-08-13 17:38:02.956776 7fe9bae887a0 20 read_log 192888'13794 (192888'13709) modify e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by client.1315259.0:51144 2014-08-11 16:47:01.592417 2014-08-13 17:38:02.956788 7fe9bae887a0 20 read_log 192888'13796 (192888'13713) modify 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.1315259.0:52445 2014-08-11 16:47:01.975970 2014-08-13 17:38:02.956800 7fe9bae887a0 20 read_log 192888'13797 (192888'13715) modify c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.1315259.0:52861 2014-08-11 16:47:02.089616 2014-08-13 17:38:02.956812 7fe9bae887a0 20 read_log 192888'13799 (192888'13719) modify da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by client.1315259.0:53534 2014-08-11 16:47:02.285108 2014-08-13 17:38:02.956824 7fe9bae887a0 20 read_log 192888'13802 (192888'13725) modify a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by client.1315259.0:56458 2014-08-11 16:47:03.211073 2014-08-13 17:38:02.956845 7fe9bae887a0 20 read_log 192888'13804 (192888'13729) modify 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.1315259.0:58519 2014-08-11 16:47:03.825914 2014-08-13 17:38:02.956858 7fe9bae887a0 20 read_log 192888'13805 (192888'13731) modify b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.1315259.0:58521 2014-08-11 16:47:03.826452 2014-08-13 17:38:02.956870 7fe9bae887a0 20 read_log 192888'13806 (192888'13733) modify f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.1315259.0:58549 2014-08-11 16:47:03.834615 2014-08-13 17:38:02.956882 7fe9bae887a0 20 read_log 192888'13807 (192888'13735) modify d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.1315259.0:58606 2014-08-11 16:47:03.851781 2014-08-13 17:38:02.956894 7fe9bae887a0 20 read_log 192888'13808 (192888'13737) modify bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.1315259.0:58883 2014-08-11 16:47:03.936632 2014-08-13 17:38:02.956908 7fe9bae887a0 20 read_log 192888'13810 (192888'13741) modify df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.1315259.0:60277 2014-08-11 16:47:04.399787 2014-08-13 17:38:02.956920 7fe9bae887a0 20 read_log 192888'13811 (192888'13743) modify 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.1315259.0:60452 2014-08-11 16:47:04.447578 2014-08-13 17:38:02.956932 7fe9bae887a0 20 read_log 192888'13818 (0'0) promote 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by osd.397.0:14657 2014-08-11 17:05:17.690008 2014-08-13 17:38:02.956944 7fe9bae887a0 20 read_log 192888'13819 (192888'13818) modify 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.942868.0:657 2014-08-11 17:05:17.678545 2014-08-13 17:38:02.956956 7fe9bae887a0 20 read_log 192888'13820 (0'0) promote 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.397.0:14659 2014-08-11 17:05:21.130444 2014-08-13 17:38:02.956968 7fe9bae887a0 20 read_log 192888'13821 (192888'13820) modify 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by client.942868.0:1497 2014-08-11 17:05:21.123224 2014-08-13 17:38:02.956980 7fe9bae887a0 20 read_log 192888'13828 (0'0) promote 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by osd.397.0:14667 2014-08-11 17:05:26.815988 2014-08-13 17:38:02.956992 7fe9bae887a0 20 read_log 192888'13829 (192888'13828) modify 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.942868.0:2939 2014-08-11 17:05:26.808260 2014-08-13 17:38:02.957007 7fe9bae887a0 20 read_log 192888'13830 (0'0) promote 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by osd.397.0:14669 2014-08-11 17:05:28.482525 2014-08-13 17:38:02.957019 7fe9bae887a0 20 read_log 192888'13831 (192888'13830) modify 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.942868.0:3389 2014-08-11 17:05:28.471785 2014-08-13 17:38:02.957034 7fe9bae887a0 20 read_log 192888'13832 (0'0) promote d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by osd.397.0:14671 2014-08-11 17:05:33.263246 2014-08-13 17:38:02.957049 7fe9bae887a0 20 read_log 192888'13833 (192888'13832) modify d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.942868.0:4675 2014-08-11 17:05:33.254577 2014-08-13 17:38:02.957061 7fe9bae887a0 20 read_log 192888'13834 (0'0) promote 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.397.0:14673 2014-08-11 17:05:33.651103 2014-08-13 17:38:02.957075 7fe9bae887a0 20 read_log 192888'13835 (192888'13834) modify 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by client.942868.0:4772 2014-08-11 17:05:33.643227 2014-08-13 17:38:02.957087 7fe9bae887a0 20 read_log 192888'13838 (0'0) promote 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.397.0:14677 2014-08-11 17:05:39.302713 2014-08-13 17:38:02.957108 7fe9bae887a0 20 read_log 192888'13839 (192888'13838) modify 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by client.942868.0:6273 2014-08-11 17:05:39.294126 2014-08-13 17:38:02.957123 7fe9bae887a0 20 read_log 192888'13840 (0'0) promote 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by osd.397.0:14679 2014-08-11 17:05:39.854552 2014-08-13 17:38:02.957136 7fe9bae887a0 20 read_log 192888'13841 (192888'13840) modify 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.942868.0:6425 2014-08-11 17:05:39.844940 2014-08-13 17:38:02.957148 7fe9bae887a0 20 read_log 192888'13842 (0'0) promote 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.397.0:14681 2014-08-11 17:05:40.321285 2014-08-13 17:38:02.957217 7fe9bae887a0 20 read_log 192888'13843 (192888'13842) modify 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by client.942868.0:6555 2014-08-11 17:05:40.312320 2014-08-13 17:38:02.957234 7fe9bae887a0 20 read_log 192888'13844 (0'0) promote 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.397.0:14683 2014-08-11 17:05:47.612984 2014-08-13 17:38:02.957256 7fe9bae887a0 20 read_log 192888'13845 (192888'13844) modify 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by client.942868.0:8494 2014-08-11 17:05:47.603625 2014-08-13 17:38:02.957269 7fe9bae887a0 20 read_log 192888'13848 (0'0) promote 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.397.0:14687 2014-08-11 17:05:48.965752 2014-08-13 17:38:02.957283 7fe9bae887a0 20 read_log 192888'13849 (192888'13848) modify 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by client.942868.0:8845 2014-08-11 17:05:48.955129 2014-08-13 17:38:02.957296 7fe9bae887a0 20 read_log 192888'13850 (0'0) promote 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by osd.397.0:14689 2014-08-11 17:05:50.329993 2014-08-13 17:38:02.957310 7fe9bae887a0 20 read_log 192888'13851 (192888'13850) modify 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.942868.0:9223 2014-08-11 17:05:50.319043 2014-08-13 17:38:02.957323 7fe9bae887a0 20 read_log 192888'13854 (0'0) promote 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.397.0:14693 2014-08-11 17:05:51.967949 2014-08-13 17:38:02.957336 7fe9bae887a0 20 read_log 192888'13855 (192888'13854) modify 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by client.942868.0:9657 2014-08-11 17:05:51.956728 2014-08-13 17:38:02.957349 7fe9bae887a0 20 read_log 192888'13856 (0'0) promote ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.397.0:14695 2014-08-11 17:05:52.665385 2014-08-13 17:38:02.957363 7fe9bae887a0 20 read_log 192888'13857 (192888'13856) modify ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by client.942868.0:9849 2014-08-11 17:05:52.654162 2014-08-13 17:38:02.957375 7fe9bae887a0 20 read_log 192888'13858 (0'0) promote 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by osd.397.0:14697 2014-08-11 17:05:52.713490 2014-08-13 17:38:02.957389 7fe9bae887a0 20 read_log 192888'13859 (192888'13858) modify 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.942868.0:9863 2014-08-11 17:05:52.704271 2014-08-13 17:38:02.957401 7fe9bae887a0 20 read_log 192888'13860 (0'0) promote 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by osd.397.0:14699 2014-08-11 17:05:52.930280 2014-08-13 17:38:02.957415 7fe9bae887a0 20 read_log 192888'13861 (192888'13860) modify 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.942868.0:9923 2014-08-11 17:05:52.920296 2014-08-13 17:38:02.957428 7fe9bae887a0 20 read_log 192888'13868 (0'0) promote 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.397.0:14707 2014-08-11 17:05:55.335121 2014-08-13 17:38:02.957449 7fe9bae887a0 20 read_log 192888'13869 (192888'13868) modify 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by client.942868.0:10591 2014-08-11 17:05:55.325428 2014-08-13 17:38:02.957465 7fe9bae887a0 20 read_log 192888'13876 (0'0) promote 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by osd.397.0:14715 2014-08-11 17:06:00.419626 2014-08-13 17:38:02.957477 7fe9bae887a0 20 read_log 192888'13877 (192888'13876) modify 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.942868.0:11974 2014-08-11 17:06:00.408940 2014-08-13 17:38:02.957492 7fe9bae887a0 20 read_log 192888'13886 (0'0) promote 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.397.0:14725 2014-08-11 17:06:16.051915 2014-08-13 17:38:02.957505 7fe9bae887a0 20 read_log 192888'13887 (192888'13886) modify 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by client.942868.0:15985 2014-08-11 17:06:16.043482 2014-08-13 17:38:02.957520 7fe9bae887a0 20 read_log 192888'13890 (0'0) promote c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by osd.397.0:14729 2014-08-11 17:06:20.796566 2014-08-13 17:38:02.957532 7fe9bae887a0 20 read_log 192888'13891 (192888'13890) modify c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.942868.0:17194 2014-08-11 17:06:20.785182 2014-08-13 17:38:02.957544 7fe9bae887a0 20 read_log 192888'13892 (0'0) promote d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by osd.397.0:14731 2014-08-11 17:06:23.270681 2014-08-13 17:38:02.957556 7fe9bae887a0 20 read_log 192888'13893 (192888'13892) modify d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.942868.0:17850 2014-08-11 17:06:23.262069 2014-08-13 17:38:02.957576 7fe9bae887a0 20 read_log 192888'13896 (0'0) promote 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.397.0:14735 2014-08-11 17:06:25.353748 2014-08-13 17:38:02.957588 7fe9bae887a0 20 read_log 192888'13897 (192888'13896) modify 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by client.942868.0:18402 2014-08-11 17:06:25.345935 2014-08-13 17:38:02.957600 7fe9bae887a0 20 read_log 192888'13898 (0'0) promote caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by osd.397.0:14737 2014-08-11 17:06:26.294115 2014-08-13 17:38:02.957614 7fe9bae887a0 20 read_log 192888'13899 (192888'13898) modify caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.942868.0:18650 2014-08-11 17:06:26.281925 2014-08-13 17:38:02.957626 7fe9bae887a0 20 read_log 192888'13902 (0'0) promote 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by osd.397.0:14741 2014-08-11 17:06:28.566697 2014-08-13 17:38:02.957639 7fe9bae887a0 20 read_log 192888'13903 (192888'13902) modify 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.942868.0:19256 2014-08-11 17:06:28.557218 2014-08-13 17:38:02.957653 7fe9bae887a0 20 read_log 192888'13908 (0'0) promote f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by osd.397.0:14747 2014-08-11 17:06:34.160839 2014-08-13 17:38:02.957665 7fe9bae887a0 20 read_log 192888'13909 (192888'13908) modify f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.942868.0:20724 2014-08-11 17:06:34.149718 2014-08-13 17:38:02.957680 7fe9bae887a0 20 read_log 192888'13912 (0'0) promote 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.397.0:14751 2014-08-11 17:06:34.633066 2014-08-13 17:38:02.957692 7fe9bae887a0 20 read_log 192888'13913 (192888'13912) modify 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by client.942868.0:20850 2014-08-11 17:06:34.623585 2014-08-13 17:38:02.957704 7fe9bae887a0 20 read_log 192888'13914 (0'0) promote b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.397.0:14753 2014-08-11 17:06:37.754375 2014-08-13 17:38:02.957749 7fe9bae887a0 20 read_log 192888'13915 (192888'13914) modify b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by client.942868.0:21674 2014-08-11 17:06:37.746402 2014-08-13 17:38:02.957765 7fe9bae887a0 20 read_log 192888'13916 (0'0) promote 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.397.0:14755 2014-08-11 17:06:38.591304 2014-08-13 17:38:02.957778 7fe9bae887a0 20 read_log 192888'13917 (192888'13916) modify 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by client.942868.0:21895 2014-08-11 17:06:38.581491 2014-08-13 17:38:02.957792 7fe9bae887a0 20 read_log 192888'13918 (0'0) promote dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.397.0:14757 2014-08-11 17:06:40.282421 2014-08-13 17:38:02.957804 7fe9bae887a0 20 read_log 192888'13919 (192888'13918) modify dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by client.942868.0:22336 2014-08-11 17:06:40.273835 2014-08-13 17:38:02.957818 7fe9bae887a0 20 read_log 192888'13922 (0'0) promote 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by osd.397.0:14761 2014-08-11 17:06:50.610266 2014-08-13 17:38:02.957831 7fe9bae887a0 20 read_log 192888'13923 (192888'13922) modify 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.942868.0:25054 2014-08-11 17:06:50.601548 2014-08-13 17:38:02.957843 7fe9bae887a0 20 read_log 192888'13930 (0'0) promote c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by osd.397.0:14769 2014-08-11 17:06:56.900790 2014-08-13 17:38:02.957855 7fe9bae887a0 20 read_log 192888'13931 (192888'13930) modify c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.942868.0:26657 2014-08-11 17:06:56.891754 2014-08-13 17:38:02.957867 7fe9bae887a0 20 read_log 192888'13932 (0'0) promote 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by osd.397.0:14771 2014-08-11 17:07:03.695508 2014-08-13 17:38:02.957881 7fe9bae887a0 20 read_log 192888'13933 (192888'13932) modify 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.942868.0:28464 2014-08-11 17:07:03.684227 2014-08-13 17:38:02.957894 7fe9bae887a0 20 read_log 192888'13934 (0'0) promote 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.397.0:14773 2014-08-11 17:07:05.104880 2014-08-13 17:38:02.957906 7fe9bae887a0 20 read_log 192888'13935 (192888'13934) modify 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by client.942868.0:28846 2014-08-11 17:07:05.096653 2014-08-13 17:38:02.957920 7fe9bae887a0 20 read_log 192888'13936 (0'0) promote d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by osd.397.0:14775 2014-08-11 17:07:13.607392 2014-08-13 17:38:02.957932 7fe9bae887a0 20 read_log 192888'13937 (192888'13936) modify d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.942868.0:30949 2014-08-11 17:07:13.598312 2014-08-13 17:38:02.957944 7fe9bae887a0 20 read_log 192888'13938 (192888'13819) modify 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.942868.0:31923 2014-08-11 17:07:16.098614 2014-08-13 17:38:02.957959 7fe9bae887a0 20 read_log 192888'13939 (192888'13821) modify 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by client.942868.0:32763 2014-08-11 17:07:16.339213 2014-08-13 17:38:02.957971 7fe9bae887a0 20 read_log 192888'13943 (192888'13829) modify 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.942868.0:34205 2014-08-11 17:07:16.725700 2014-08-13 17:38:02.957983 7fe9bae887a0 20 read_log 192888'13944 (192888'13831) modify 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.942868.0:34655 2014-08-11 17:07:16.854194 2014-08-13 17:38:02.957998 7fe9bae887a0 20 read_log 192888'13945 (192888'13833) modify d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.942868.0:35941 2014-08-11 17:07:17.202434 2014-08-13 17:38:02.958017 7fe9bae887a0 20 read_log 192888'13946 (192888'13835) modify 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by client.942868.0:36038 2014-08-11 17:07:17.226719 2014-08-13 17:38:02.958032 7fe9bae887a0 20 read_log 192888'13948 (192888'13839) modify 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by client.942868.0:37539 2014-08-11 17:07:17.637342 2014-08-13 17:38:02.958044 7fe9bae887a0 20 read_log 192888'13949 (192888'13841) modify 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.942868.0:37691 2014-08-11 17:07:17.680397 2014-08-13 17:38:02.958057 7fe9bae887a0 20 read_log 192888'13950 (192888'13843) modify 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by client.942868.0:37821 2014-08-11 17:07:17.715687 2014-08-13 17:38:02.958071 7fe9bae887a0 20 read_log 192888'13951 (192888'13845) modify 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by client.942868.0:39760 2014-08-11 17:07:18.275853 2014-08-13 17:38:02.958083 7fe9bae887a0 20 read_log 192888'13953 (192888'13849) modify 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by client.942868.0:40111 2014-08-11 17:07:18.368286 2014-08-13 17:38:02.958095 7fe9bae887a0 20 read_log 192888'13954 (192888'13851) modify 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.942868.0:40489 2014-08-11 17:07:18.466233 2014-08-13 17:38:02.958107 7fe9bae887a0 20 read_log 192888'13956 (192888'13855) modify 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by client.942868.0:40923 2014-08-11 17:07:18.583317 2014-08-13 17:38:02.958119 7fe9bae887a0 20 read_log 192888'13957 (192888'13857) modify ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by client.942868.0:41115 2014-08-11 17:07:18.646246 2014-08-13 17:38:02.958131 7fe9bae887a0 20 read_log 192888'13958 (192888'13859) modify 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.942868.0:41129 2014-08-11 17:07:18.649861 2014-08-13 17:38:02.958145 7fe9bae887a0 20 read_log 192888'13959 (192888'13861) modify 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.942868.0:41189 2014-08-11 17:07:18.665294 2014-08-13 17:38:02.958163 7fe9bae887a0 20 read_log 192888'13963 (192888'13869) modify 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by client.942868.0:41857 2014-08-11 17:07:18.842741 2014-08-13 17:38:02.958178 7fe9bae887a0 20 read_log 192888'13967 (192888'13877) modify 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.942868.0:43240 2014-08-11 17:07:19.217401 2014-08-13 17:38:02.958190 7fe9bae887a0 20 read_log 192888'13972 (192888'13887) modify 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by client.942868.0:47251 2014-08-11 17:07:20.351567 2014-08-13 17:38:02.958203 7fe9bae887a0 20 read_log 192888'13974 (192888'13891) modify c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.942868.0:48460 2014-08-11 17:07:20.725612 2014-08-13 17:38:02.958215 7fe9bae887a0 20 read_log 192888'13975 (192888'13893) modify d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.942868.0:49116 2014-08-11 17:07:20.913876 2014-08-13 17:38:02.958230 7fe9bae887a0 20 read_log 192888'13977 (192888'13897) modify 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by client.942868.0:49668 2014-08-11 17:07:21.084501 2014-08-13 17:38:02.958242 7fe9bae887a0 20 read_log 192888'13978 (192888'13899) modify caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.942868.0:49916 2014-08-11 17:07:21.162265 2014-08-13 17:38:02.958256 7fe9bae887a0 20 read_log 192888'13980 (192888'13903) modify 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.942868.0:50522 2014-08-11 17:07:21.348883 2014-08-13 17:38:02.958269 7fe9bae887a0 20 read_log 192888'13983 (192888'13909) modify f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.942868.0:51990 2014-08-11 17:07:21.810880 2014-08-13 17:38:02.958288 7fe9bae887a0 20 read_log 192888'13985 (192888'13913) modify 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by client.942868.0:52116 2014-08-11 17:07:21.845706 2014-08-13 17:38:02.958303 7fe9bae887a0 20 read_log 192888'13986 (192888'13915) modify b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by client.942868.0:52940 2014-08-11 17:07:22.097157 2014-08-13 17:38:02.958315 7fe9bae887a0 20 read_log 192888'13987 (192888'13917) modify 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by client.942868.0:53161 2014-08-11 17:07:22.170492 2014-08-13 17:38:02.958327 7fe9bae887a0 20 read_log 192888'13988 (192888'13919) modify dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by client.942868.0:53602 2014-08-11 17:07:22.318416 2014-08-13 17:38:02.958341 7fe9bae887a0 20 read_log 192888'13990 (192888'13923) modify 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.942868.0:56320 2014-08-11 17:07:23.113320 2014-08-13 17:38:02.958354 7fe9bae887a0 20 read_log 192888'13994 (192888'13931) modify c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.942868.0:57923 2014-08-11 17:07:23.592548 2014-08-13 17:38:02.958366 7fe9bae887a0 20 read_log 192888'13995 (192888'13933) modify 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.942868.0:59730 2014-08-11 17:07:24.133175 2014-08-13 17:38:02.958378 7fe9bae887a0 20 read_log 192888'13996 (192888'13935) modify 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by client.942868.0:60112 2014-08-11 17:07:24.244978 2014-08-13 17:38:02.958390 7fe9bae887a0 20 read_log 192888'13997 (192888'13937) modify d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.942868.0:62215 2014-08-11 17:07:24.885609 2014-08-13 17:38:02.958409 7fe9bae887a0 20 read_log 193305'13998 (0'0) promote 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by osd.343.0:31 2014-08-11 18:42:38.048812 2014-08-13 17:38:02.958421 7fe9bae887a0 20 read_log 193305'13999 (193305'13998) modify 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.951337.0:1297 2014-08-11 18:42:38.042090 2014-08-13 17:38:02.958437 7fe9bae887a0 20 read_log 193305'14000 (0'0) promote 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:109 2014-08-11 18:42:49.141048 2014-08-13 17:38:02.958450 7fe9bae887a0 20 read_log 193305'14001 (193305'14000) modify 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by client.951337.0:4307 2014-08-11 18:42:49.134695 2014-08-13 17:38:02.958464 7fe9bae887a0 20 read_log 193305'14002 (0'0) promote 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:133 2014-08-11 18:42:52.288739 2014-08-13 17:38:02.958476 7fe9bae887a0 20 read_log 193305'14003 (193305'14002) modify 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by client.951337.0:5136 2014-08-11 18:42:52.281338 2014-08-13 17:38:02.958490 7fe9bae887a0 20 read_log 193305'14004 (0'0) promote 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by osd.343.0:141 2014-08-11 18:42:54.231465 2014-08-13 17:38:02.958502 7fe9bae887a0 20 read_log 193305'14005 (193305'14004) modify 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.951337.0:5671 2014-08-11 18:42:54.224232 2014-08-13 17:38:02.958518 7fe9bae887a0 20 read_log 193305'14006 (0'0) promote 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by osd.343.0:143 2014-08-11 18:42:54.508615 2014-08-13 17:38:02.958533 7fe9bae887a0 20 read_log 193305'14007 (193305'14006) modify 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.951337.0:5747 2014-08-11 18:42:54.501929 2014-08-13 17:38:02.958545 7fe9bae887a0 20 read_log 193305'14008 (0'0) promote 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by osd.343.0:151 2014-08-11 18:42:56.848301 2014-08-13 17:38:02.958567 7fe9bae887a0 20 read_log 193305'14009 (193305'14008) modify 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.951337.0:6392 2014-08-11 18:42:56.840863 2014-08-13 17:38:02.958582 7fe9bae887a0 20 read_log 193305'14010 (0'0) promote 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by osd.343.0:169 2014-08-11 18:43:00.981777 2014-08-13 17:38:02.958594 7fe9bae887a0 20 read_log 193305'14011 (193305'14010) modify 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.951337.0:7535 2014-08-11 18:43:00.974716 2014-08-13 17:38:02.958607 7fe9bae887a0 20 read_log 193305'14012 (0'0) promote 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:177 2014-08-11 18:43:02.541105 2014-08-13 17:38:02.958619 7fe9bae887a0 20 read_log 193305'14013 (193305'14012) modify 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by client.951337.0:7967 2014-08-11 18:43:02.533611 2014-08-13 17:38:02.958631 7fe9bae887a0 20 read_log 193305'14014 (0'0) promote 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by osd.343.0:187 2014-08-11 18:43:05.605905 2014-08-13 17:38:02.958643 7fe9bae887a0 20 read_log 193305'14015 (193305'14014) modify 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.951337.0:8735 2014-08-11 18:43:05.598818 2014-08-13 17:38:02.958655 7fe9bae887a0 20 read_log 193305'14016 (0'0) promote a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by osd.343.0:193 2014-08-11 18:43:07.328681 2014-08-13 17:38:02.958668 7fe9bae887a0 20 read_log 193305'14017 (193305'14016) modify a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.951337.0:9192 2014-08-11 18:43:07.320220 2014-08-13 17:38:02.958680 7fe9bae887a0 20 read_log 193305'14018 (0'0) promote 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:235 2014-08-11 18:43:15.369894 2014-08-13 17:38:02.958692 7fe9bae887a0 20 read_log 193305'14019 (193305'14018) modify 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by client.951337.0:11319 2014-08-11 18:43:15.363106 2014-08-13 17:38:02.958704 7fe9bae887a0 20 read_log 193305'14020 (0'0) promote de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:263 2014-08-11 18:43:20.879278 2014-08-13 17:38:02.958716 7fe9bae887a0 20 read_log 193305'14021 (193305'14020) modify de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by client.951337.0:12796 2014-08-11 18:43:20.871168 2014-08-13 17:38:02.958729 7fe9bae887a0 20 read_log 193305'14022 (0'0) promote a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:269 2014-08-11 18:43:22.984970 2014-08-13 17:38:02.958764 7fe9bae887a0 20 read_log 193305'14023 (193305'14022) modify a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by client.951337.0:13364 2014-08-11 18:43:22.977364 2014-08-13 17:38:02.958780 7fe9bae887a0 20 read_log 193305'14024 (0'0) promote f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by osd.343.0:280 2014-08-11 18:43:24.183269 2014-08-13 17:38:02.958792 7fe9bae887a0 20 read_log 193305'14025 (193305'14024) modify f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.951337.0:13661 2014-08-11 18:43:24.176422 2014-08-13 17:38:02.958804 7fe9bae887a0 20 read_log 193305'14026 (0'0) promote 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:331 2014-08-11 18:43:33.497560 2014-08-13 17:38:02.958816 7fe9bae887a0 20 read_log 193305'14027 (193305'14026) modify 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by client.951337.0:16295 2014-08-11 18:43:33.488144 2014-08-13 17:38:02.958828 7fe9bae887a0 20 read_log 193305'14028 (0'0) promote 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by osd.343.0:355 2014-08-11 18:43:37.736789 2014-08-13 17:38:02.958849 7fe9bae887a0 20 read_log 193305'14029 (193305'14028) modify 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.951337.0:17532 2014-08-11 18:43:37.730110 2014-08-13 17:38:02.958865 7fe9bae887a0 20 read_log 193305'14030 (0'0) promote 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by osd.343.0:367 2014-08-11 18:43:38.451643 2014-08-13 17:38:02.958878 7fe9bae887a0 20 read_log 193305'14031 (193305'14030) modify 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.951337.0:17750 2014-08-11 18:43:38.442836 2014-08-13 17:38:02.958890 7fe9bae887a0 20 read_log 193305'14032 (0'0) promote 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:379 2014-08-11 18:43:42.638035 2014-08-13 17:38:02.958902 7fe9bae887a0 20 read_log 193305'14033 (193305'14032) modify 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by client.951337.0:18913 2014-08-11 18:43:42.630298 2014-08-13 17:38:02.958914 7fe9bae887a0 20 read_log 193305'14034 (0'0) promote 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by osd.343.0:389 2014-08-11 18:43:45.818464 2014-08-13 17:38:02.958926 7fe9bae887a0 20 read_log 193305'14035 (193305'14034) modify 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.951337.0:19775 2014-08-11 18:43:45.810912 2014-08-13 17:38:02.958938 7fe9bae887a0 20 read_log 193305'14036 (0'0) promote 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by osd.343.0:439 2014-08-11 18:43:55.578680 2014-08-13 17:38:02.958949 7fe9bae887a0 20 read_log 193305'14037 (193305'14036) modify 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.951337.0:22476 2014-08-11 18:43:55.571408 2014-08-13 17:38:02.958961 7fe9bae887a0 20 read_log 193305'14038 (0'0) promote aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:488 2014-08-11 18:44:04.840689 2014-08-13 17:38:02.958973 7fe9bae887a0 20 read_log 193305'14039 (193305'14038) modify aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by client.951337.0:25048 2014-08-11 18:44:04.832045 2014-08-13 17:38:02.958985 7fe9bae887a0 20 read_log 193305'14040 (0'0) promote 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by osd.343.0:496 2014-08-11 18:44:06.153479 2014-08-13 17:38:02.958999 7fe9bae887a0 20 read_log 193305'14041 (193305'14040) modify 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.951337.0:25416 2014-08-11 18:44:06.143479 2014-08-13 17:38:02.959011 7fe9bae887a0 20 read_log 193305'14042 (0'0) promote 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:504 2014-08-11 18:44:07.642518 2014-08-13 17:38:02.959023 7fe9bae887a0 20 read_log 193305'14043 (193305'14042) modify 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by client.951337.0:25802 2014-08-11 18:44:07.632289 2014-08-13 17:38:02.959035 7fe9bae887a0 20 read_log 193305'14044 (0'0) promote d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:508 2014-08-11 18:44:09.479971 2014-08-13 17:38:02.959047 7fe9bae887a0 20 read_log 193305'14045 (193305'14044) modify d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by client.951337.0:26298 2014-08-11 18:44:09.472556 2014-08-13 17:38:02.959059 7fe9bae887a0 20 read_log 193305'14046 (0'0) promote 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by osd.343.0:514 2014-08-11 18:44:10.634805 2014-08-13 17:38:02.959071 7fe9bae887a0 20 read_log 193305'14047 (193305'14046) modify 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.951337.0:26622 2014-08-11 18:44:10.627683 2014-08-13 17:38:02.959083 7fe9bae887a0 20 read_log 193305'14048 (0'0) promote 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:528 2014-08-11 18:44:14.033202 2014-08-13 17:38:02.959102 7fe9bae887a0 20 read_log 193305'14049 (193305'14048) modify 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by client.951337.0:27542 2014-08-11 18:44:14.027692 2014-08-13 17:38:02.959115 7fe9bae887a0 20 read_log 193305'14050 (0'0) promote b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by osd.343.0:530 2014-08-11 18:44:14.096084 2014-08-13 17:38:02.959127 7fe9bae887a0 20 read_log 193305'14051 (193305'14050) modify b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.951337.0:27560 2014-08-11 18:44:14.089800 2014-08-13 17:38:02.959139 7fe9bae887a0 20 read_log 193305'14052 (0'0) promote 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by osd.343.0:536 2014-08-11 18:44:14.671586 2014-08-13 17:38:02.959151 7fe9bae887a0 20 read_log 193305'14053 (193305'14052) modify 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.951337.0:27724 2014-08-11 18:44:14.665610 2014-08-13 17:38:02.959168 7fe9bae887a0 20 read_log 193305'14054 (0'0) promote ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by osd.343.0:538 2014-08-11 18:44:15.335198 2014-08-13 17:38:02.959181 7fe9bae887a0 20 read_log 193305'14055 (193305'14054) modify ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.951337.0:27910 2014-08-11 18:44:15.326005 2014-08-13 17:38:02.959193 7fe9bae887a0 20 read_log 193305'14056 (0'0) promote 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by osd.343.0:540 2014-08-11 18:44:15.665202 2014-08-13 17:38:02.959205 7fe9bae887a0 20 read_log 193305'14057 (193305'14056) modify 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.951337.0:28003 2014-08-11 18:44:15.654383 2014-08-13 17:38:02.959216 7fe9bae887a0 20 read_log 193305'14058 (0'0) promote c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by osd.343.0:590 2014-08-11 18:44:27.306817 2014-08-13 17:38:02.959228 7fe9bae887a0 20 read_log 193305'14059 (193305'14058) modify c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.951337.0:31138 2014-08-11 18:44:27.299836 2014-08-13 17:38:02.959240 7fe9bae887a0 20 read_log 193305'14060 (0'0) promote faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by osd.343.0:608 2014-08-11 18:44:30.159446 2014-08-13 17:38:02.959252 7fe9bae887a0 20 read_log 193305'14061 (193305'14060) modify faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.951337.0:31966 2014-08-11 18:44:30.150806 2014-08-13 17:38:02.959264 7fe9bae887a0 20 read_log 193305'14062 (193305'13999) modify 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.951337.0:34008 2014-08-11 18:44:34.756550 2014-08-13 17:38:02.959276 7fe9bae887a0 20 read_log 193305'14063 (193305'14001) modify 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by client.951337.0:37018 2014-08-11 18:44:36.659981 2014-08-13 17:38:02.959288 7fe9bae887a0 20 read_log 193305'14064 (193305'14003) modify 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by client.951337.0:37847 2014-08-11 18:44:37.083272 2014-08-13 17:38:02.959300 7fe9bae887a0 20 read_log 193305'14065 (193305'14005) modify 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.951337.0:38382 2014-08-11 18:44:37.384776 2014-08-13 17:38:02.959313 7fe9bae887a0 20 read_log 193305'14066 (193305'14007) modify 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.951337.0:38458 2014-08-11 18:44:37.466369 2014-08-13 17:38:02.959325 7fe9bae887a0 20 read_log 193305'14067 (193305'14009) modify 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.951337.0:39103 2014-08-11 18:44:37.826849 2014-08-13 17:38:02.959337 7fe9bae887a0 20 read_log 193305'14068 (193305'14011) modify 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.951337.0:40246 2014-08-11 18:44:38.347817 2014-08-13 17:38:02.959358 7fe9bae887a0 20 read_log 193305'14069 (193305'14013) modify 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by client.951337.0:40678 2014-08-11 18:44:38.484971 2014-08-13 17:38:02.959374 7fe9bae887a0 20 read_log 193305'14070 (193305'14015) modify 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.951337.0:41446 2014-08-11 18:44:38.755030 2014-08-13 17:38:02.959386 7fe9bae887a0 20 read_log 193305'14071 (193305'14017) modify a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.951337.0:41903 2014-08-11 18:44:38.918103 2014-08-13 17:38:02.959398 7fe9bae887a0 20 read_log 193305'14072 (193305'14019) modify 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by client.951337.0:44030 2014-08-11 18:44:39.773934 2014-08-13 17:38:02.959411 7fe9bae887a0 20 read_log 193305'14073 (193305'14021) modify de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by client.951337.0:45507 2014-08-11 18:44:40.221818 2014-08-13 17:38:02.959424 7fe9bae887a0 20 read_log 193305'14074 (193305'14023) modify a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by client.951337.0:46075 2014-08-11 18:44:40.381612 2014-08-13 17:38:02.959436 7fe9bae887a0 20 read_log 193305'14075 (193305'14025) modify f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.951337.0:46372 2014-08-11 18:44:40.465456 2014-08-13 17:38:02.959448 7fe9bae887a0 20 read_log 193305'14076 (193305'14027) modify 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by client.951337.0:49006 2014-08-11 18:44:41.223786 2014-08-13 17:38:02.959460 7fe9bae887a0 20 read_log 193305'14077 (193305'14029) modify 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.951337.0:50243 2014-08-11 18:44:41.609803 2014-08-13 17:38:02.959472 7fe9bae887a0 20 read_log 193305'14078 (193305'14031) modify 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.951337.0:50461 2014-08-11 18:44:41.672337 2014-08-13 17:38:02.959484 7fe9bae887a0 20 read_log 193305'14079 (193305'14033) modify 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by client.951337.0:51624 2014-08-11 18:44:42.003577 2014-08-13 17:38:02.959496 7fe9bae887a0 20 read_log 193305'14080 (193305'14035) modify 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.951337.0:52486 2014-08-11 18:44:42.246514 2014-08-13 17:38:02.959508 7fe9bae887a0 20 read_log 193305'14081 (193305'14037) modify 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.951337.0:55187 2014-08-11 18:44:43.048592 2014-08-13 17:38:02.959520 7fe9bae887a0 20 read_log 193305'14082 (193305'14039) modify aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by client.951337.0:57759 2014-08-11 18:44:43.850887 2014-08-13 17:38:02.959533 7fe9bae887a0 20 read_log 193305'14083 (193305'14041) modify 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.951337.0:58127 2014-08-11 18:44:43.954246 2014-08-13 17:38:02.959545 7fe9bae887a0 20 read_log 193305'14084 (193305'14043) modify 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by client.951337.0:58513 2014-08-11 18:44:44.064899 2014-08-13 17:38:02.959557 7fe9bae887a0 20 read_log 193305'14085 (193305'14045) modify d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by client.951337.0:59009 2014-08-11 18:44:44.208661 2014-08-13 17:38:02.959569 7fe9bae887a0 20 read_log 193305'14086 (193305'14047) modify 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.951337.0:59333 2014-08-11 18:44:44.296418 2014-08-13 17:38:02.959581 7fe9bae887a0 20 read_log 193305'14087 (193305'14049) modify 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by client.951337.0:60253 2014-08-11 18:44:44.573408 2014-08-13 17:38:02.959595 7fe9bae887a0 20 read_log 193305'14088 (193305'14051) modify b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.951337.0:60271 2014-08-11 18:44:44.578152 2014-08-13 17:38:02.959633 7fe9bae887a0 20 read_log 193305'14089 (193305'14053) modify 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.951337.0:60435 2014-08-11 18:44:44.628605 2014-08-13 17:38:02.959649 7fe9bae887a0 20 read_log 193305'14090 (193305'14055) modify ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.951337.0:60621 2014-08-11 18:44:44.683170 2014-08-13 17:38:02.959662 7fe9bae887a0 20 read_log 193305'14091 (193305'14057) modify 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.951337.0:60714 2014-08-11 18:44:44.708688 2014-08-13 17:38:02.959674 7fe9bae887a0 20 read_log 193305'14092 (193305'14059) modify c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.951337.0:63849 2014-08-11 18:44:45.627999 2014-08-13 17:38:02.959686 7fe9bae887a0 20 read_log 193305'14093 (193305'14061) modify faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.951337.0:64677 2014-08-11 18:44:45.886776 2014-08-13 17:38:02.959697 7fe9bae887a0 20 read_log 194209'14094 (192888'13985) clean 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.343.0:1011 2014-08-12 14:05:31.063175 2014-08-13 17:38:02.959710 7fe9bae887a0 20 read_log 194209'14095 (192888'13802) clean a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.343.0:1012 2014-08-12 14:05:31.063569 2014-08-13 17:38:02.959722 7fe9bae887a0 20 read_log 194209'14096 (192834'13031) clean abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.343.0:1013 2014-08-12 14:05:31.063911 2014-08-13 17:38:02.959734 7fe9bae887a0 20 read_log 194209'14097 (192834'13062) clean f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.343.0:1014 2014-08-12 14:05:31.064416 2014-08-13 17:38:02.959746 7fe9bae887a0 20 read_log 194209'14098 (192834'13572) clean 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.343.0:1015 2014-08-12 14:05:31.087363 2014-08-13 17:38:02.959758 7fe9bae887a0 20 read_log 194209'14099 (192888'13946) clean 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.343.0:1016 2014-08-12 14:05:31.090313 2014-08-13 17:38:02.959770 7fe9bae887a0 20 read_log 194209'14100 (192888'13784) clean 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.343.0:1017 2014-08-12 14:05:31.093431 2014-08-13 17:38:02.959783 7fe9bae887a0 20 read_log 194209'14101 (192888'13799) clean da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.343.0:1018 2014-08-12 14:05:31.096545 2014-08-13 17:38:02.959795 7fe9bae887a0 20 read_log 194209'14102 (192888'13987) clean 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.343.0:1019 2014-08-12 14:05:31.099930 2014-08-13 17:38:02.959807 7fe9bae887a0 20 read_log 194209'14103 (192888'13988) clean dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.343.0:1020 2014-08-12 14:05:31.103120 2014-08-13 17:38:02.959819 7fe9bae887a0 20 read_log 194209'14104 (192834'13606) clean ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.343.0:1021 2014-08-12 14:05:31.106080 2014-08-13 17:38:02.959832 7fe9bae887a0 20 read_log 194209'14105 (193305'14087) clean 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:1022 2014-08-12 14:05:31.112078 2014-08-13 17:38:02.959844 7fe9bae887a0 20 read_log 194209'14106 (193305'14082) clean aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:1023 2014-08-12 14:05:31.112678 2014-08-13 17:38:02.959857 7fe9bae887a0 20 read_log 194209'14107 (192834'12942) clean f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.343.0:1024 2014-08-12 14:05:31.115759 2014-08-13 17:38:02.959877 7fe9bae887a0 20 read_log 194209'14108 (192834'13571) clean dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.343.0:1025 2014-08-12 14:05:31.119246 2014-08-13 17:38:02.959889 7fe9bae887a0 20 read_log 194209'14109 (192834'13518) clean a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.343.0:1026 2014-08-12 14:05:31.125111 2014-08-13 17:38:02.959901 7fe9bae887a0 20 read_log 194209'14110 (193305'14079) clean 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:1027 2014-08-12 14:05:31.125382 2014-08-13 17:38:02.959913 7fe9bae887a0 20 read_log 194209'14111 (192834'12905) clean 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.343.0:1028 2014-08-12 14:05:31.128573 2014-08-13 17:38:02.959925 7fe9bae887a0 20 read_log 194209'14112 (192888'13752) clean d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.343.0:1029 2014-08-12 14:05:31.131639 2014-08-13 17:38:02.959937 7fe9bae887a0 20 read_log 194209'14113 (192834'12946) clean 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.343.0:1030 2014-08-12 14:05:31.134704 2014-08-13 17:38:02.959950 7fe9bae887a0 20 read_log 194209'14114 (193305'14072) clean 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:1031 2014-08-12 14:05:31.137802 2014-08-13 17:38:02.959962 7fe9bae887a0 20 read_log 194209'14115 (192834'13463) clean 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.343.0:1032 2014-08-12 14:05:31.141170 2014-08-13 17:38:02.959990 7fe9bae887a0 20 read_log 194209'14116 (192834'13563) clean bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.343.0:1033 2014-08-12 14:05:31.144423 2014-08-13 17:38:02.960005 7fe9bae887a0 20 read_log 194209'14117 (192888'13986) clean b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.343.0:1034 2014-08-12 14:05:31.152694 2014-08-13 17:38:02.960017 7fe9bae887a0 20 read_log 194209'14118 (193305'14076) clean 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:1035 2014-08-12 14:05:31.152973 2014-08-13 17:38:02.960029 7fe9bae887a0 20 read_log 194209'14119 (192888'13794) clean e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.343.0:1036 2014-08-12 14:05:31.153231 2014-08-13 17:38:02.960041 7fe9bae887a0 20 read_log 194209'14120 (192888'13786) clean 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.343.0:1037 2014-08-12 14:05:31.153673 2014-08-13 17:38:02.960053 7fe9bae887a0 20 read_log 194209'14121 (193305'14074) clean a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:1038 2014-08-12 14:05:31.161812 2014-08-13 17:38:02.960065 7fe9bae887a0 20 read_log 194209'14122 (193305'14069) clean 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:1039 2014-08-12 14:05:31.167025 2014-08-13 17:38:02.960077 7fe9bae887a0 20 read_log 194209'14123 (192888'13783) clean 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.343.0:1040 2014-08-12 14:05:31.167287 2014-08-13 17:38:02.960089 7fe9bae887a0 20 read_log 194209'14124 (192888'13948) clean 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.343.0:1041 2014-08-12 14:05:31.170837 2014-08-13 17:38:02.960102 7fe9bae887a0 20 read_log 194209'14125 (192888'13791) clean e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.343.0:1042 2014-08-12 14:05:31.174667 2014-08-13 17:38:02.960114 7fe9bae887a0 20 read_log 194209'14126 (192834'13604) clean ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.343.0:1043 2014-08-12 14:05:31.178324 2014-08-13 17:38:02.960126 7fe9bae887a0 20 read_log 194209'14127 (192888'13950) clean 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.343.0:1044 2014-08-12 14:05:31.181996 2014-08-13 17:38:02.960148 7fe9bae887a0 20 read_log 194209'14128 (192888'13972) clean 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.343.0:1045 2014-08-12 14:05:31.185496 2014-08-13 17:38:02.960168 7fe9bae887a0 20 read_log 194209'14129 (192834'12991) clean 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.343.0:1046 2014-08-12 14:05:31.189047 2014-08-13 17:38:02.960181 7fe9bae887a0 20 read_log 194209'14130 (193305'14064) clean 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:1047 2014-08-12 14:05:31.192558 2014-08-13 17:38:02.960193 7fe9bae887a0 20 read_log 194209'14131 (192834'12975) clean c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.343.0:1048 2014-08-12 14:05:31.195963 2014-08-13 17:38:02.960205 7fe9bae887a0 20 read_log 194209'14132 (192888'13939) clean 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.343.0:1049 2014-08-12 14:05:31.199588 2014-08-13 17:38:02.960217 7fe9bae887a0 20 read_log 194209'14133 (193305'14085) clean d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:1050 2014-08-12 14:05:31.203079 2014-08-13 17:38:02.960229 7fe9bae887a0 20 read_log 194209'14134 (192888'13951) clean 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.343.0:1051 2014-08-12 14:05:31.206597 2014-08-13 17:38:02.960241 7fe9bae887a0 20 read_log 194209'14135 (192888'13996) clean 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.343.0:1052 2014-08-12 14:05:31.210234 2014-08-13 17:38:02.960253 7fe9bae887a0 20 read_log 194209'14136 (193305'14084) clean 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:1053 2014-08-12 14:05:31.213661 2014-08-13 17:38:02.960266 7fe9bae887a0 20 read_log 194209'14137 (192888'13771) clean d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.343.0:1054 2014-08-12 14:05:31.217139 2014-08-13 17:38:02.960279 7fe9bae887a0 20 read_log 194209'14138 (193305'14073) clean de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:1055 2014-08-12 14:05:31.220704 2014-08-13 17:38:02.960294 7fe9bae887a0 20 read_log 194209'14139 (192888'13977) clean 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.343.0:1056 2014-08-12 14:05:31.224153 2014-08-13 17:38:02.960307 7fe9bae887a0 20 read_log 194209'14140 (192888'13957) clean ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.343.0:1057 2014-08-12 14:05:31.227740 2014-08-13 17:38:02.960322 7fe9bae887a0 20 read_log 194209'14141 (192834'13595) clean cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.343.0:1058 2014-08-12 14:05:31.231372 2014-08-13 17:38:02.960334 7fe9bae887a0 20 read_log 194209'14142 (192834'13602) clean 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.343.0:1059 2014-08-12 14:05:31.234858 2014-08-13 17:38:02.960347 7fe9bae887a0 20 read_log 194209'14143 (192834'13455) clean 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.343.0:1060 2014-08-12 14:05:31.238669 2014-08-13 17:38:02.960361 7fe9bae887a0 20 read_log 194209'14144 (192834'13011) clean 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.343.0:1061 2014-08-12 14:05:31.242180 2014-08-13 17:38:02.960373 7fe9bae887a0 20 read_log 194209'14145 (192888'13953) clean 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.343.0:1062 2014-08-12 14:05:31.245793 2014-08-13 17:38:02.960387 7fe9bae887a0 20 read_log 194209'14146 (192888'13963) clean 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.343.0:1063 2014-08-12 14:05:31.249375 2014-08-13 17:38:02.960402 7fe9bae887a0 20 read_log 194209'14147 (192888'13956) clean 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.343.0:1064 2014-08-12 14:05:31.252754 2014-08-13 17:38:02.960424 7fe9bae887a0 20 read_log 194209'14148 (194209'14094) delete 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.343.0:1065 2014-08-12 14:05:31.289310 2014-08-13 17:38:02.960439 7fe9bae887a0 20 read_log 194209'14149 (194209'14095) delete a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.343.0:1066 2014-08-12 14:05:31.292673 2014-08-13 17:38:02.960451 7fe9bae887a0 20 read_log 194209'14150 (194209'14096) delete abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.343.0:1067 2014-08-12 14:05:31.295483 2014-08-13 17:38:02.960464 7fe9bae887a0 20 read_log 194209'14151 (194209'14097) delete f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.343.0:1068 2014-08-12 14:05:31.298436 2014-08-13 17:38:02.960476 7fe9bae887a0 20 read_log 194209'14152 (194209'14098) delete 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.343.0:1069 2014-08-12 14:05:31.301882 2014-08-13 17:38:02.960488 7fe9bae887a0 20 read_log 194209'14153 (194209'14099) delete 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.343.0:1070 2014-08-12 14:05:31.304523 2014-08-13 17:38:02.960500 7fe9bae887a0 20 read_log 194209'14154 (194209'14100) delete 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.343.0:1071 2014-08-12 14:05:31.307451 2014-08-13 17:38:02.960512 7fe9bae887a0 20 read_log 194209'14155 (194209'14101) delete da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.343.0:1072 2014-08-12 14:05:31.310802 2014-08-13 17:38:02.960526 7fe9bae887a0 20 read_log 194209'14156 (194209'14102) delete 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.343.0:1073 2014-08-12 14:05:31.313350 2014-08-13 17:38:02.960538 7fe9bae887a0 20 read_log 194209'14157 (194209'14103) delete dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.343.0:1074 2014-08-12 14:05:31.316731 2014-08-13 17:38:02.960550 7fe9bae887a0 20 read_log 194209'14158 (194209'14104) delete ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.343.0:1075 2014-08-12 14:05:31.319358 2014-08-13 17:38:02.960562 7fe9bae887a0 20 read_log 194209'14159 (194209'14105) delete 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:1076 2014-08-12 14:05:31.322674 2014-08-13 17:38:02.960574 7fe9bae887a0 20 read_log 194209'14160 (193305'14063) clean 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:1077 2014-08-12 14:05:31.323125 2014-08-13 17:38:02.960586 7fe9bae887a0 20 read_log 194209'14161 (194209'14106) delete aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:1078 2014-08-12 14:05:31.326190 2014-08-13 17:38:02.960598 7fe9bae887a0 20 read_log 194209'14162 (194209'14107) delete f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.343.0:1079 2014-08-12 14:05:31.328939 2014-08-13 17:38:02.960613 7fe9bae887a0 20 read_log 194209'14163 (194209'14108) delete dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.343.0:1080 2014-08-12 14:05:31.329334 2014-08-13 17:38:02.960625 7fe9bae887a0 20 read_log 194209'14164 (194209'14109) delete a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.343.0:1081 2014-08-12 14:05:31.332871 2014-08-13 17:38:02.960640 7fe9bae887a0 20 read_log 194209'14165 (194209'14110) delete 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:1082 2014-08-12 14:05:31.333277 2014-08-13 17:38:02.960652 7fe9bae887a0 20 read_log 194209'14166 (194209'14111) delete 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.343.0:1083 2014-08-12 14:05:31.336571 2014-08-13 17:38:02.960664 7fe9bae887a0 20 read_log 194209'14167 (194209'14112) delete d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.343.0:1084 2014-08-12 14:05:31.337104 2014-08-13 17:38:02.960685 7fe9bae887a0 20 read_log 194209'14168 (194209'14113) delete 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.343.0:1085 2014-08-12 14:05:31.337623 2014-08-13 17:38:02.960699 7fe9bae887a0 20 read_log 194209'14169 (194209'14114) delete 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:1086 2014-08-12 14:05:31.340657 2014-08-13 17:38:02.960712 7fe9bae887a0 20 read_log 194209'14170 (194209'14115) delete 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.343.0:1087 2014-08-12 14:05:31.341104 2014-08-13 17:38:02.960724 7fe9bae887a0 20 read_log 194209'14171 (194209'14116) delete bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.343.0:1088 2014-08-12 14:05:31.341684 2014-08-13 17:38:02.960736 7fe9bae887a0 20 read_log 194209'14172 (194209'14117) delete b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.343.0:1089 2014-08-12 14:05:31.344579 2014-08-13 17:38:02.960748 7fe9bae887a0 20 read_log 194209'14173 (194209'14118) delete 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:1090 2014-08-12 14:05:31.345007 2014-08-13 17:38:02.960760 7fe9bae887a0 20 read_log 194209'14174 (194209'14119) delete e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.343.0:1091 2014-08-12 14:05:31.345504 2014-08-13 17:38:02.960772 7fe9bae887a0 20 read_log 194209'14175 (194209'14120) delete 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.343.0:1092 2014-08-12 14:05:31.348307 2014-08-13 17:38:02.960784 7fe9bae887a0 20 read_log 194209'14176 (194209'14121) delete a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:1093 2014-08-12 14:05:31.348606 2014-08-13 17:38:02.960804 7fe9bae887a0 20 read_log 194209'14177 (194209'14122) delete 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:1094 2014-08-12 14:05:31.349386 2014-08-13 17:38:02.960816 7fe9bae887a0 20 read_log 194209'14178 (194209'14123) delete 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.343.0:1095 2014-08-12 14:05:31.352211 2014-08-13 17:38:02.960828 7fe9bae887a0 20 read_log 194209'14179 (194209'14124) delete 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.343.0:1096 2014-08-12 14:05:31.352548 2014-08-13 17:38:02.960840 7fe9bae887a0 20 read_log 194209'14180 (194209'14125) delete e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.343.0:1097 2014-08-12 14:05:31.355866 2014-08-13 17:38:02.960852 7fe9bae887a0 20 read_log 194209'14181 (194209'14126) delete ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.343.0:1098 2014-08-12 14:05:31.356181 2014-08-13 17:38:02.960864 7fe9bae887a0 20 read_log 194209'14182 (194209'14127) delete 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.343.0:1099 2014-08-12 14:05:31.356892 2014-08-13 17:38:02.960876 7fe9bae887a0 20 read_log 194209'14183 (194209'14128) delete 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.343.0:1100 2014-08-12 14:05:31.360111 2014-08-13 17:38:02.960888 7fe9bae887a0 20 read_log 194209'14184 (194209'14129) delete 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.343.0:1101 2014-08-12 14:05:31.360453 2014-08-13 17:38:02.960900 7fe9bae887a0 20 read_log 194209'14185 (194209'14130) delete 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:1102 2014-08-12 14:05:31.361020 2014-08-13 17:38:02.960912 7fe9bae887a0 20 read_log 194209'14186 (194209'14131) delete c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.343.0:1103 2014-08-12 14:05:31.364365 2014-08-13 17:38:02.960924 7fe9bae887a0 20 read_log 194209'14187 (194209'14132) delete 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.343.0:1104 2014-08-12 14:05:31.364704 2014-08-13 17:38:02.960946 7fe9bae887a0 20 read_log 194209'14188 (194209'14133) delete d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:1105 2014-08-12 14:05:31.365428 2014-08-13 17:38:02.960959 7fe9bae887a0 20 read_log 194209'14189 (194209'14134) delete 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.343.0:1106 2014-08-12 14:05:31.368978 2014-08-13 17:38:02.960971 7fe9bae887a0 20 read_log 194209'14190 (194209'14135) delete 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.343.0:1107 2014-08-12 14:05:31.369323 2014-08-13 17:38:02.960983 7fe9bae887a0 20 read_log 194209'14191 (194209'14136) delete 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:1108 2014-08-12 14:05:31.369914 2014-08-13 17:38:02.960995 7fe9bae887a0 20 read_log 194209'14192 (194209'14137) delete d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.343.0:1109 2014-08-12 14:05:31.372840 2014-08-13 17:38:02.961007 7fe9bae887a0 20 read_log 194209'14193 (194209'14138) delete de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:1110 2014-08-12 14:05:31.373155 2014-08-13 17:38:02.961019 7fe9bae887a0 20 read_log 194209'14194 (194209'14139) delete 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.343.0:1111 2014-08-12 14:05:31.373864 2014-08-13 17:38:02.961031 7fe9bae887a0 20 read_log 194209'14195 (194209'14140) delete ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.343.0:1112 2014-08-12 14:05:31.376605 2014-08-13 17:38:02.961043 7fe9bae887a0 20 read_log 194209'14196 (194209'14141) delete cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.343.0:1113 2014-08-12 14:05:31.376965 2014-08-13 17:38:02.961058 7fe9bae887a0 20 read_log 194209'14197 (194209'14142) delete 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.343.0:1114 2014-08-12 14:05:31.377625 2014-08-13 17:38:02.961070 7fe9bae887a0 20 read_log 194209'14198 (194209'14143) delete 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.343.0:1115 2014-08-12 14:05:31.380591 2014-08-13 17:38:02.961082 7fe9bae887a0 20 read_log 194209'14199 (194209'14144) delete 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.343.0:1116 2014-08-12 14:05:31.380932 2014-08-13 17:38:02.961094 7fe9bae887a0 20 read_log 194209'14200 (194209'14145) delete 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.343.0:1117 2014-08-12 14:05:31.381477 2014-08-13 17:38:02.961107 7fe9bae887a0 20 read_log 194209'14201 (194209'14146) delete 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.343.0:1118 2014-08-12 14:05:31.385079 2014-08-13 17:38:02.961119 7fe9bae887a0 20 read_log 194209'14202 (194209'14147) delete 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.343.0:1119 2014-08-12 14:05:31.385434 2014-08-13 17:38:02.961131 7fe9bae887a0 20 read_log 194209'14203 (194209'14160) delete 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:1120 2014-08-12 14:05:31.385655 2014-08-13 17:38:02.961143 7fe9bae887a0 20 read_log 194215'14204 (194215'14204) modify 206/hit_set_24.206_archive_2014-08-12 15:32:25.739433_2014-08-12 15:44:15.482903/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:15.483247 2014-08-13 17:38:02.961160 7fe9bae887a0 20 read_log 194215'14205 (192888'13810) clean df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by osd.343.0:2585 2014-08-12 15:44:15.487361 2014-08-13 17:38:02.961177 7fe9bae887a0 20 read_log 194215'14206 (194215'14205) delete df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.2174595.0:106897 0.000000 2014-08-13 17:38:02.961196 7fe9bae887a0 20 read_log 194215'14207 (192888'13944) clean 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by osd.343.0:2587 2014-08-12 15:44:15.495491 2014-08-13 17:38:02.961209 7fe9bae887a0 20 read_log 194215'14208 (194215'14207) delete 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.2174595.0:106899 0.000000 2014-08-13 17:38:02.961247 7fe9bae887a0 20 read_log 194215'14209 (192834'13566) clean f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by osd.343.0:2589 2014-08-12 15:44:15.503823 2014-08-13 17:38:02.961260 7fe9bae887a0 20 read_log 194215'14210 (194215'14209) delete f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.2174595.0:106901 0.000000 2014-08-13 17:38:02.961271 7fe9bae887a0 20 read_log 194215'14211 (192888'13796) clean 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by osd.343.0:2591 2014-08-12 15:44:15.511222 2014-08-13 17:38:02.961283 7fe9bae887a0 20 read_log 194215'14212 (194215'14211) delete 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.2174595.0:106903 0.000000 2014-08-13 17:38:02.961312 7fe9bae887a0 20 read_log 194215'14213 (192834'12958) clean ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by osd.343.0:2593 2014-08-12 15:44:15.518952 2014-08-13 17:38:02.961326 7fe9bae887a0 20 read_log 194215'14214 (194215'14213) delete ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.2174595.0:106905 0.000000 2014-08-13 17:38:02.961336 7fe9bae887a0 20 read_log 194215'14215 (192834'13554) clean 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by osd.343.0:2595 2014-08-12 15:44:15.525956 2014-08-13 17:38:02.961349 7fe9bae887a0 20 read_log 194215'14216 (194215'14215) delete 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.2174595.0:106907 0.000000 2014-08-13 17:38:02.961359 7fe9bae887a0 20 read_log 194215'14217 (193305'14066) clean 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by osd.343.0:2597 2014-08-12 15:44:15.533120 2014-08-13 17:38:02.961371 7fe9bae887a0 20 read_log 194215'14218 (194215'14217) delete 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.2174595.0:106909 0.000000 2014-08-13 17:38:02.961382 7fe9bae887a0 20 read_log 194215'14219 (192834'12952) clean ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by osd.343.0:2599 2014-08-12 15:44:15.540172 2014-08-13 17:38:02.961394 7fe9bae887a0 20 read_log 194215'14220 (194215'14219) delete ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.2174595.0:106911 0.000000 2014-08-13 17:38:02.961404 7fe9bae887a0 20 read_log 194215'14221 (192834'13445) clean 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by osd.343.0:2601 2014-08-12 15:44:15.547485 2014-08-13 17:38:02.961416 7fe9bae887a0 20 read_log 194215'14222 (194215'14221) delete 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.2174595.0:106913 0.000000 2014-08-13 17:38:02.961427 7fe9bae887a0 20 read_log 194215'14223 (192888'13990) clean 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by osd.343.0:2603 2014-08-12 15:44:15.554528 2014-08-13 17:38:02.961439 7fe9bae887a0 20 read_log 194215'14224 (194215'14223) delete 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.2174595.0:106915 0.000000 2014-08-13 17:38:02.961450 7fe9bae887a0 20 read_log 194215'14225 (192888'13954) clean 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by osd.343.0:2605 2014-08-12 15:44:15.561308 2014-08-13 17:38:02.961462 7fe9bae887a0 20 read_log 194215'14226 (194215'14225) delete 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.2174595.0:106917 0.000000 2014-08-13 17:38:02.961497 7fe9bae887a0 20 read_log 194215'14227 (192834'12929) clean 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by osd.343.0:2607 2014-08-12 15:44:15.568994 2014-08-13 17:38:02.961511 7fe9bae887a0 20 read_log 194215'14228 (194215'14227) delete 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.2174595.0:106919 0.000000 2014-08-13 17:38:02.961522 7fe9bae887a0 20 read_log 194215'14229 (192888'13943) clean 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by osd.343.0:2609 2014-08-12 15:44:15.576142 2014-08-13 17:38:02.961534 7fe9bae887a0 20 read_log 194215'14230 (194215'14229) delete 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.2174595.0:106921 0.000000 2014-08-13 17:38:02.961545 7fe9bae887a0 20 read_log 194215'14231 (193305'14062) clean 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by osd.343.0:2611 2014-08-12 15:44:15.583654 2014-08-13 17:38:02.961557 7fe9bae887a0 20 read_log 194215'14232 (194215'14231) delete 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.2174595.0:106923 0.000000 2014-08-13 17:38:02.961568 7fe9bae887a0 20 read_log 194215'14233 (192888'13945) clean d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by osd.343.0:2613 2014-08-12 15:44:15.590597 2014-08-13 17:38:02.961580 7fe9bae887a0 20 read_log 194215'14234 (194215'14233) delete d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.2174595.0:106925 0.000000 2014-08-13 17:38:02.961590 7fe9bae887a0 20 read_log 194215'14235 (193305'14078) clean 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by osd.343.0:2615 2014-08-12 15:44:15.597818 2014-08-13 17:38:02.961602 7fe9bae887a0 20 read_log 194215'14236 (194215'14235) delete 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.2174595.0:106927 0.000000 2014-08-13 17:38:02.961613 7fe9bae887a0 20 read_log 194215'14237 (192888'13782) clean 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by osd.343.0:2617 2014-08-12 15:44:15.604059 2014-08-13 17:38:02.961625 7fe9bae887a0 20 read_log 194215'14238 (194215'14237) delete 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.2174595.0:106929 0.000000 2014-08-13 17:38:02.961635 7fe9bae887a0 20 read_log 194215'14239 (192834'12994) clean 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by osd.343.0:2619 2014-08-12 15:44:15.610345 2014-08-13 17:38:02.961648 7fe9bae887a0 20 read_log 194215'14240 (194215'14239) delete 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.2174595.0:106931 0.000000 2014-08-13 17:38:02.961658 7fe9bae887a0 20 read_log 194215'14241 (193305'14092) clean c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by osd.343.0:2621 2014-08-12 15:44:15.616755 2014-08-13 17:38:02.961670 7fe9bae887a0 20 read_log 194215'14242 (194215'14241) delete c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.2174595.0:106933 0.000000 2014-08-13 17:38:02.961681 7fe9bae887a0 20 read_log 194215'14243 (192888'13975) clean d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by osd.343.0:2623 2014-08-12 15:44:15.624018 2014-08-13 17:38:02.961693 7fe9bae887a0 20 read_log 194215'14244 (194215'14243) delete d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.2174595.0:106935 0.000000 2014-08-13 17:38:02.961703 7fe9bae887a0 20 read_log 194215'14245 (192888'13759) clean 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by osd.343.0:2625 2014-08-12 15:44:15.630848 2014-08-13 17:38:02.961715 7fe9bae887a0 20 read_log 194215'14246 (194215'14245) delete 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.2174595.0:106937 0.000000 2014-08-13 17:38:02.961735 7fe9bae887a0 20 read_log 194215'14247 (192888'13781) clean dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by osd.343.0:2627 2014-08-12 15:44:15.637682 2014-08-13 17:38:02.961748 7fe9bae887a0 20 read_log 194215'14248 (194215'14247) delete dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.2174595.0:106939 0.000000 2014-08-13 17:38:02.961759 7fe9bae887a0 20 read_log 194215'14249 (192888'13958) clean 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by osd.343.0:2629 2014-08-12 15:44:15.644360 2014-08-13 17:38:02.961771 7fe9bae887a0 20 read_log 194215'14250 (194215'14249) delete 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.2174595.0:106941 0.000000 2014-08-13 17:38:02.961782 7fe9bae887a0 20 read_log 194215'14251 (193305'14065) clean 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by osd.343.0:2631 2014-08-12 15:44:15.651474 2014-08-13 17:38:02.961794 7fe9bae887a0 20 read_log 194215'14252 (194215'14251) delete 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.2174595.0:106943 0.000000 2014-08-13 17:38:02.961805 7fe9bae887a0 20 read_log 194215'14253 (193305'14088) clean b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by osd.343.0:2633 2014-08-12 15:44:15.658319 2014-08-13 17:38:02.961817 7fe9bae887a0 20 read_log 194215'14254 (194215'14253) delete b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.2174595.0:106945 0.000000 2014-08-13 17:38:02.961828 7fe9bae887a0 20 read_log 194215'14255 (193305'14070) clean 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by osd.343.0:2635 2014-08-12 15:44:15.665789 2014-08-13 17:38:02.961840 7fe9bae887a0 20 read_log 194215'14256 (194215'14255) delete 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.2174595.0:106947 0.000000 2014-08-13 17:38:02.961851 7fe9bae887a0 20 read_log 194215'14257 (192888'13980) clean 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by osd.343.0:2637 2014-08-12 15:44:15.672323 2014-08-13 17:38:02.961863 7fe9bae887a0 20 read_log 194215'14258 (194215'14257) delete 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.2174595.0:106949 0.000000 2014-08-13 17:38:02.961874 7fe9bae887a0 20 read_log 194215'14259 (193305'14075) clean f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by osd.343.0:2639 2014-08-12 15:44:15.679127 2014-08-13 17:38:02.961887 7fe9bae887a0 20 read_log 194215'14260 (194215'14259) delete f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.2174595.0:106951 0.000000 2014-08-13 17:38:02.961898 7fe9bae887a0 20 read_log 194215'14261 (192888'13959) clean 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by osd.343.0:2641 2014-08-12 15:44:15.685731 2014-08-13 17:38:02.961910 7fe9bae887a0 20 read_log 194215'14262 (194215'14261) delete 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.2174595.0:106953 0.000000 2014-08-13 17:38:02.961921 7fe9bae887a0 20 read_log 194215'14263 (192834'13536) clean f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by osd.343.0:2643 2014-08-12 15:44:15.692525 2014-08-13 17:38:02.961932 7fe9bae887a0 20 read_log 194215'14264 (194215'14263) delete f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.2174595.0:106955 0.000000 2014-08-13 17:38:02.961943 7fe9bae887a0 20 read_log 194215'14265 (192888'13772) clean 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by osd.343.0:2645 2014-08-12 15:44:15.699187 2014-08-13 17:38:02.961955 7fe9bae887a0 20 read_log 194215'14266 (194215'14265) delete 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.2174595.0:106957 0.000000 2014-08-13 17:38:02.961972 7fe9bae887a0 20 read_log 194215'14267 (192834'13056) clean 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by osd.343.0:2647 2014-08-12 15:44:15.706179 2014-08-13 17:38:02.961985 7fe9bae887a0 20 read_log 194215'14268 (194215'14267) delete 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.2174595.0:106959 0.000000 2014-08-13 17:38:02.961995 7fe9bae887a0 20 read_log 194215'14269 (192834'13029) clean 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by osd.343.0:2649 2014-08-12 15:44:15.712788 2014-08-13 17:38:02.962007 7fe9bae887a0 20 read_log 194215'14270 (194215'14269) delete 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.2174595.0:106961 0.000000 2014-08-13 17:38:02.962018 7fe9bae887a0 20 read_log 194215'14271 (192888'13793) clean dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by osd.343.0:2651 2014-08-12 15:44:15.720060 2014-08-13 17:38:02.962030 7fe9bae887a0 20 read_log 194215'14272 (194215'14271) delete dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.2174595.0:106963 0.000000 2014-08-13 17:38:02.962040 7fe9bae887a0 20 read_log 194215'14273 (192888'13789) clean 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by osd.343.0:2653 2014-08-12 15:44:15.727003 2014-08-13 17:38:02.962052 7fe9bae887a0 20 read_log 194215'14274 (194215'14273) delete 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.2174595.0:106965 0.000000 2014-08-13 17:38:02.962063 7fe9bae887a0 20 read_log 194215'14275 (192888'13805) clean b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by osd.343.0:2655 2014-08-12 15:44:15.733966 2014-08-13 17:38:02.962074 7fe9bae887a0 20 read_log 194215'14276 (194215'14275) delete b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.2174595.0:106967 0.000000 2014-08-13 17:38:02.962085 7fe9bae887a0 20 read_log 194215'14277 (192834'12989) clean 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by osd.343.0:2657 2014-08-12 15:44:15.741255 2014-08-13 17:38:02.962097 7fe9bae887a0 20 read_log 194215'14278 (194215'14277) delete 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.2174595.0:106969 0.000000 2014-08-13 17:38:02.962107 7fe9bae887a0 20 read_log 194215'14279 (192834'13068) clean 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by osd.343.0:2659 2014-08-12 15:44:15.748625 2014-08-13 17:38:02.962120 7fe9bae887a0 20 read_log 194215'14280 (194215'14279) delete 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.2174595.0:106971 0.000000 2014-08-13 17:38:02.962130 7fe9bae887a0 20 read_log 194215'14281 (192888'13806) clean f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by osd.343.0:2661 2014-08-12 15:44:15.755637 2014-08-13 17:38:02.962142 7fe9bae887a0 20 read_log 194215'14282 (194215'14281) delete f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.2174595.0:106973 0.000000 2014-08-13 17:38:02.962159 7fe9bae887a0 20 read_log 194215'14283 (192834'13042) clean 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by osd.343.0:2663 2014-08-12 15:44:15.762418 2014-08-13 17:38:02.962172 7fe9bae887a0 20 read_log 194215'14284 (194215'14283) delete 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.2174595.0:106975 0.000000 2014-08-13 17:38:02.962183 7fe9bae887a0 20 read_log 194215'14285 (192888'13755) clean 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by osd.343.0:2665 2014-08-12 15:44:15.769184 2014-08-13 17:38:02.962195 7fe9bae887a0 20 read_log 194215'14286 (194215'14285) delete 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.2174595.0:106977 0.000000 2014-08-13 17:38:02.962213 7fe9bae887a0 20 read_log 194215'14287 (192888'13807) clean d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by osd.343.0:2667 2014-08-12 15:44:15.776061 2014-08-13 17:38:02.962225 7fe9bae887a0 20 read_log 194215'14288 (194215'14287) delete d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.2174595.0:106979 0.000000 2014-08-13 17:38:02.962236 7fe9bae887a0 20 read_log 194215'14289 (192834'13055) clean f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by osd.343.0:2669 2014-08-12 15:44:15.783231 2014-08-13 17:38:02.962248 7fe9bae887a0 20 read_log 194215'14290 (194215'14289) delete f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.2174595.0:106981 0.000000 2014-08-13 17:38:02.962259 7fe9bae887a0 20 read_log 194215'14291 (192888'13967) clean 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by osd.343.0:2671 2014-08-12 15:44:15.790603 2014-08-13 17:38:02.962271 7fe9bae887a0 20 read_log 194215'14292 (194215'14291) delete 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.2174595.0:106983 0.000000 2014-08-13 17:38:02.962281 7fe9bae887a0 20 read_log 194215'14293 (193305'14071) clean a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by osd.343.0:2673 2014-08-12 15:44:15.797755 2014-08-13 17:38:02.962293 7fe9bae887a0 20 read_log 194215'14294 (194215'14293) delete a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.2174595.0:106985 0.000000 2014-08-13 17:38:02.962304 7fe9bae887a0 20 read_log 194215'14295 (192834'13047) clean b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by osd.343.0:2675 2014-08-12 15:44:15.804737 2014-08-13 17:38:02.962316 7fe9bae887a0 20 read_log 194215'14296 (194215'14295) delete b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.2174595.0:106987 0.000000 2014-08-13 17:38:02.962326 7fe9bae887a0 20 read_log 194215'14297 (193305'14093) clean faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by osd.343.0:2677 2014-08-12 15:44:15.811871 2014-08-13 17:38:02.962338 7fe9bae887a0 20 read_log 194215'14298 (194215'14297) delete faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.2174595.0:106989 0.000000 2014-08-13 17:38:02.962349 7fe9bae887a0 20 read_log 194215'14299 (192834'13482) clean a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by osd.343.0:2679 2014-08-12 15:44:15.819454 2014-08-13 17:38:02.962361 7fe9bae887a0 20 read_log 194215'14300 (194215'14299) delete a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.2174595.0:106991 0.000000 2014-08-13 17:38:02.962371 7fe9bae887a0 20 read_log 194215'14301 (192888'13769) clean 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by osd.343.0:2681 2014-08-12 15:44:15.826871 2014-08-13 17:38:02.962401 7fe9bae887a0 20 read_log 194215'14302 (194215'14301) delete 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.2174595.0:106993 0.000000 2014-08-13 17:38:02.962412 7fe9bae887a0 20 read_log 194215'14303 (192888'13792) clean 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by osd.343.0:2683 2014-08-12 15:44:15.834403 2014-08-13 17:38:02.962424 7fe9bae887a0 20 read_log 194215'14304 (194215'14303) delete 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.2174595.0:106995 0.000000 2014-08-13 17:38:02.962435 7fe9bae887a0 20 read_log 194215'14305 (192888'13938) clean 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by osd.343.0:2685 2014-08-12 15:44:15.841939 2014-08-13 17:38:02.962447 7fe9bae887a0 20 read_log 194215'14306 (194215'14305) delete 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.2174595.0:106997 0.000000 2014-08-13 17:38:02.962466 7fe9bae887a0 20 read_log 194215'14307 (193305'14081) clean 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by osd.343.0:2687 2014-08-12 15:44:15.848939 2014-08-13 17:38:02.962479 7fe9bae887a0 20 read_log 194215'14308 (194215'14307) delete 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.2174595.0:106999 0.000000 2014-08-13 17:38:02.962490 7fe9bae887a0 20 read_log 194215'14309 (192834'12972) clean b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by osd.343.0:2689 2014-08-12 15:44:15.855848 2014-08-13 17:38:02.962502 7fe9bae887a0 20 read_log 194215'14310 (194215'14309) delete b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.2174595.0:107001 0.000000 2014-08-13 17:38:02.962513 7fe9bae887a0 20 read_log 194215'14311 (192888'13797) clean c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by osd.343.0:2691 2014-08-12 15:44:15.863081 2014-08-13 17:38:02.962525 7fe9bae887a0 20 read_log 194215'14312 (194215'14311) delete c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.2174595.0:107003 0.000000 2014-08-13 17:38:02.962535 7fe9bae887a0 20 read_log 194215'14313 (192888'13753) clean 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by osd.343.0:2693 2014-08-12 15:44:15.870449 2014-08-13 17:38:02.962547 7fe9bae887a0 20 read_log 194215'14314 (194215'14313) delete 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.2174595.0:107005 0.000000 2014-08-13 17:38:02.962558 7fe9bae887a0 20 read_log 194215'14315 (192834'13580) clean 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by osd.343.0:2695 2014-08-12 15:44:15.877601 2014-08-13 17:38:02.962570 7fe9bae887a0 20 read_log 194215'14316 (194215'14315) delete 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.2174595.0:107007 0.000000 2014-08-13 17:38:02.962581 7fe9bae887a0 20 read_log 194215'14317 (192834'13608) clean c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by osd.343.0:2697 2014-08-12 15:44:15.885776 2014-08-13 17:38:02.962592 7fe9bae887a0 20 read_log 194215'14318 (194215'14317) delete c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.2174595.0:107009 0.000000 2014-08-13 17:38:02.962603 7fe9bae887a0 20 read_log 194215'14319 (192834'12909) clean 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by osd.343.0:2699 2014-08-12 15:44:15.893010 2014-08-13 17:38:02.962615 7fe9bae887a0 20 read_log 194215'14320 (194215'14319) delete 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.2174595.0:107011 0.000000 2014-08-13 17:38:02.962626 7fe9bae887a0 20 read_log 194215'14321 (192888'13995) clean 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by osd.343.0:2701 2014-08-12 15:44:15.899983 2014-08-13 17:38:02.962638 7fe9bae887a0 20 read_log 194215'14322 (194215'14321) delete 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.2174595.0:107013 0.000000 2014-08-13 17:38:02.962648 7fe9bae887a0 20 read_log 194215'14323 (192888'13811) clean 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by osd.343.0:2703 2014-08-12 15:44:15.907128 2014-08-13 17:38:02.962660 7fe9bae887a0 20 read_log 194215'14324 (194215'14323) delete 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.2174595.0:107015 0.000000 2014-08-13 17:38:02.962670 7fe9bae887a0 20 read_log 194215'14325 (192888'13780) clean 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by osd.343.0:2705 2014-08-12 15:44:15.914858 2014-08-13 17:38:02.962682 7fe9bae887a0 20 read_log 194215'14326 (194215'14325) delete 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.2174595.0:107017 0.000000 2014-08-13 17:38:02.962699 7fe9bae887a0 20 read_log 194215'14327 (192888'13760) clean c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by osd.343.0:2707 2014-08-12 15:44:15.921502 2014-08-13 17:38:02.962714 7fe9bae887a0 20 read_log 194215'14328 (194215'14327) delete c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.2174595.0:107019 0.000000 2014-08-13 17:38:02.962725 7fe9bae887a0 20 read_log 194215'14329 (192834'13511) clean ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by osd.343.0:2709 2014-08-12 15:44:15.928017 2014-08-13 17:38:02.962738 7fe9bae887a0 20 read_log 194215'14330 (194215'14329) delete ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.2174595.0:107021 0.000000 2014-08-13 17:38:02.962748 7fe9bae887a0 20 read_log 194215'14331 (192834'13620) clean eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by osd.343.0:2711 2014-08-12 15:44:15.935218 2014-08-13 17:38:02.962760 7fe9bae887a0 20 read_log 194215'14332 (194215'14331) delete eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.2174595.0:107023 0.000000 2014-08-13 17:38:02.962771 7fe9bae887a0 20 read_log 194215'14333 (192888'13997) clean d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by osd.343.0:2713 2014-08-12 15:44:15.942705 2014-08-13 17:38:02.962783 7fe9bae887a0 20 read_log 194215'14334 (194215'14333) delete d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.2174595.0:107025 0.000000 2014-08-13 17:38:02.962793 7fe9bae887a0 20 read_log 194215'14335 (193305'14086) clean 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by osd.343.0:2715 2014-08-12 15:44:15.949305 2014-08-13 17:38:02.962805 7fe9bae887a0 20 read_log 194215'14336 (194215'14335) delete 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.2174595.0:107027 0.000000 2014-08-13 17:38:02.962816 7fe9bae887a0 20 read_log 194215'14337 (192834'13474) clean deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by osd.343.0:2717 2014-08-12 15:44:15.956674 2014-08-13 17:38:02.962829 7fe9bae887a0 20 read_log 194215'14338 (194215'14337) delete deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.2174595.0:107029 0.000000 2014-08-13 17:38:02.962839 7fe9bae887a0 20 read_log 194215'14339 (193305'14091) clean 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by osd.343.0:2719 2014-08-12 15:44:15.964166 2014-08-13 17:38:02.962851 7fe9bae887a0 20 read_log 194215'14340 (194215'14339) delete 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.2174595.0:107031 0.000000 2014-08-13 17:38:02.962862 7fe9bae887a0 20 read_log 194215'14341 (192834'13037) clean 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by osd.343.0:2721 2014-08-12 15:44:15.971439 2014-08-13 17:38:02.962874 7fe9bae887a0 20 read_log 194215'14342 (194215'14341) delete 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.2174595.0:107033 0.000000 2014-08-13 17:38:02.962885 7fe9bae887a0 20 read_log 194215'14343 (192888'13983) clean f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by osd.343.0:2723 2014-08-12 15:44:15.979108 2014-08-13 17:38:02.962897 7fe9bae887a0 20 read_log 194215'14344 (194215'14343) delete f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.2174595.0:107035 0.000000 2014-08-13 17:38:02.962907 7fe9bae887a0 20 read_log 194215'14345 (192888'13978) clean caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by osd.343.0:2725 2014-08-12 15:44:15.986811 2014-08-13 17:38:02.962919 7fe9bae887a0 20 read_log 194215'14346 (194215'14345) delete caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.2174595.0:107037 0.000000 2014-08-13 17:38:02.962937 7fe9bae887a0 20 read_log 194215'14347 (192834'13458) clean fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by osd.343.0:2727 2014-08-12 15:44:15.993904 2014-08-13 17:38:02.962949 7fe9bae887a0 20 read_log 194215'14348 (194215'14347) delete fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.2174595.0:107039 0.000000 2014-08-13 17:38:02.962960 7fe9bae887a0 20 read_log 194215'14349 (193305'14067) clean 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by osd.343.0:2729 2014-08-12 15:44:16.001260 2014-08-13 17:38:02.962972 7fe9bae887a0 20 read_log 194215'14350 (194215'14349) delete 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.2174595.0:107041 0.000000 2014-08-13 17:38:02.962983 7fe9bae887a0 20 read_log 194215'14351 (192888'13778) clean 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by osd.343.0:2731 2014-08-12 15:44:16.009241 2014-08-13 17:38:02.962995 7fe9bae887a0 20 read_log 194215'14352 (194215'14351) delete 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.2174595.0:107043 0.000000 2014-08-13 17:38:02.963006 7fe9bae887a0 20 read_log 194215'14353 (193305'14083) clean 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by osd.343.0:2733 2014-08-12 15:44:16.016562 2014-08-13 17:38:02.963018 7fe9bae887a0 20 read_log 194215'14354 (194215'14353) delete 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.2174595.0:107045 0.000000 2014-08-13 17:38:02.963028 7fe9bae887a0 20 read_log 194215'14355 (193305'14077) clean 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by osd.343.0:2735 2014-08-12 15:44:16.023514 2014-08-13 17:38:02.963040 7fe9bae887a0 20 read_log 194215'14356 (194215'14355) delete 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.2174595.0:107047 0.000000 2014-08-13 17:38:02.963051 7fe9bae887a0 20 read_log 194215'14357 (192834'13614) clean d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by osd.343.0:2737 2014-08-12 15:44:16.029799 2014-08-13 17:38:02.963063 7fe9bae887a0 20 read_log 194215'14358 (194215'14357) delete d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.2174595.0:107049 0.000000 2014-08-13 17:38:02.963073 7fe9bae887a0 20 read_log 194215'14359 (192834'12948) clean ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by osd.343.0:2739 2014-08-12 15:44:16.036941 2014-08-13 17:38:02.963087 7fe9bae887a0 20 read_log 194215'14360 (194215'14359) delete ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.2174595.0:107051 0.000000 2014-08-13 17:38:02.963098 7fe9bae887a0 20 read_log 194215'14361 (193305'14080) clean 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by osd.343.0:2741 2014-08-12 15:44:16.043568 2014-08-13 17:38:02.963110 7fe9bae887a0 20 read_log 194215'14362 (194215'14361) delete 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.2174595.0:107053 0.000000 2014-08-13 17:38:02.963120 7fe9bae887a0 20 read_log 194215'14363 (192834'13514) clean 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by osd.343.0:2743 2014-08-12 15:44:16.050630 2014-08-13 17:38:02.963132 7fe9bae887a0 20 read_log 194215'14364 (194215'14363) delete 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.2174595.0:107055 0.000000 2014-08-13 17:38:02.963143 7fe9bae887a0 20 read_log 194215'14365 (192888'13949) clean 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by osd.343.0:2745 2014-08-12 15:44:16.058976 2014-08-13 17:38:02.963160 7fe9bae887a0 20 read_log 194215'14366 (194215'14365) delete 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.2174595.0:107057 0.000000 2014-08-13 17:38:02.963199 7fe9bae887a0 20 read_log 194215'14367 (192834'12973) clean f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by osd.343.0:2747 2014-08-12 15:44:16.066787 2014-08-13 17:38:02.963212 7fe9bae887a0 20 read_log 194215'14368 (194215'14367) delete f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.2174595.0:107059 0.000000 2014-08-13 17:38:02.963223 7fe9bae887a0 20 read_log 194215'14369 (192888'13808) clean bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by osd.343.0:2749 2014-08-12 15:44:16.074276 2014-08-13 17:38:02.963235 7fe9bae887a0 20 read_log 194215'14370 (194215'14369) delete bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.2174595.0:107061 0.000000 2014-08-13 17:38:02.963246 7fe9bae887a0 20 read_log 194215'14371 (192834'13611) clean 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by osd.343.0:2751 2014-08-12 15:44:16.081782 2014-08-13 17:38:02.963258 7fe9bae887a0 20 read_log 194215'14372 (194215'14371) delete 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.2174595.0:107063 0.000000 2014-08-13 17:38:02.963269 7fe9bae887a0 20 read_log 194215'14373 (192888'13776) clean ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by osd.343.0:2753 2014-08-12 15:44:16.088887 2014-08-13 17:38:02.963281 7fe9bae887a0 20 read_log 194215'14374 (194215'14373) delete ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.2174595.0:107065 0.000000 2014-08-13 17:38:02.963292 7fe9bae887a0 20 read_log 194215'14375 (193305'14090) clean ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by osd.343.0:2755 2014-08-12 15:44:16.096186 2014-08-13 17:38:02.963304 7fe9bae887a0 20 read_log 194215'14376 (194215'14375) delete ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.2174595.0:107067 0.000000 2014-08-13 17:38:02.963314 7fe9bae887a0 20 read_log 194215'14377 (192888'13974) clean c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by osd.343.0:2757 2014-08-12 15:44:16.104169 2014-08-13 17:38:02.963326 7fe9bae887a0 20 read_log 194215'14378 (194215'14377) delete c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.2174595.0:107069 0.000000 2014-08-13 17:38:02.963337 7fe9bae887a0 20 read_log 194215'14379 (192888'13804) clean 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by osd.343.0:2759 2014-08-12 15:44:16.111274 2014-08-13 17:38:02.963349 7fe9bae887a0 20 read_log 194215'14380 (194215'14379) delete 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.2174595.0:107071 0.000000 2014-08-13 17:38:02.963359 7fe9bae887a0 20 read_log 194215'14381 (193305'14068) clean 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by osd.343.0:2761 2014-08-12 15:44:16.118314 2014-08-13 17:38:02.963371 7fe9bae887a0 20 read_log 194215'14382 (194215'14381) delete 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.2174595.0:107073 0.000000 2014-08-13 17:38:02.963382 7fe9bae887a0 20 read_log 194215'14383 (192888'13994) clean c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by osd.343.0:2763 2014-08-12 15:44:16.125542 2014-08-13 17:38:02.963394 7fe9bae887a0 20 read_log 194215'14384 (194215'14383) delete c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.2174595.0:107075 0.000000 2014-08-13 17:38:02.963405 7fe9bae887a0 20 read_log 194215'14385 (193305'14089) clean 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by osd.343.0:2765 2014-08-12 15:44:16.133203 2014-08-13 17:38:02.963417 7fe9bae887a0 20 read_log 194215'14386 (194215'14385) delete 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.2174595.0:107077 0.000000 2014-08-13 17:38:02.963434 7fe9bae887a0 20 read_log 194215'14387 (192888'13758) clean a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by osd.343.0:2767 2014-08-12 15:44:16.140443 2014-08-13 17:38:02.963447 7fe9bae887a0 20 read_log 194215'14388 (194215'14387) delete a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.2174595.0:107079 0.000000 2014-08-13 17:38:02.963458 7fe9bae887a0 20 read_log 194215'14389 (192834'12910) clean fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by osd.343.0:2769 2014-08-12 15:44:16.147307 2014-08-13 17:38:02.963470 7fe9bae887a0 20 read_log 194215'14390 (194215'14389) delete fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.2174595.0:107081 0.000000 2014-08-13 17:38:02.963481 7fe9bae887a0 20 read_log 194215'14391 (194215'14391) modify 206/hit_set_24.206_archive_2014-08-12 15:44:15.483446_2014-08-12 17:24:44.732489/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.733026 2014-08-13 17:38:02.963493 7fe9bae887a0 20 read_log 194215'14392 (194215'14204) delete 206/hit_set_24.206_archive_2014-08-12 15:32:25.739433_2014-08-12 15:44:15.482903/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.733026 2014-08-13 17:38:02.963505 7fe9bae887a0 20 read_log 194215'14393 (0'0) promote cdac7606/benchmark_data_storage0111-ib.pocstor_34628_object1236/head//24 by osd.343.0:3904 2014-08-12 17:24:44.737953 2014-08-13 17:38:02.963517 7fe9bae887a0 20 read_log 194215'14394 (194215'14393) modify cdac7606/benchmark_data_storage0111-ib.pocstor_34628_object1236/head//24 by client.2193759.0:1237 2014-08-12 17:24:44.725615 2014-08-13 17:38:02.963548 7fe9bae887a0 20 read_log 194370'14395 (0'0) promote 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by osd.343.0:3933 2014-08-12 20:30:15.905568 2014-08-13 17:38:02.963561 7fe9bae887a0 20 read_log 194370'14396 (194370'14395) modify 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by client.1508531.0:402 2014-08-12 20:30:15.898415 2014-08-13 17:38:02.963573 7fe9bae887a0 20 read_log 194370'14397 (0'0) promote 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by osd.343.0:3941 2014-08-12 20:30:18.294433 2014-08-13 17:38:02.963585 7fe9bae887a0 20 read_log 194370'14398 (194370'14397) modify 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by client.1508531.0:1047 2014-08-12 20:30:18.286872 2014-08-13 17:38:02.963598 7fe9bae887a0 20 read_log 194370'14399 (0'0) promote 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by osd.343.0:3947 2014-08-12 20:30:20.242315 2014-08-13 17:38:02.963609 7fe9bae887a0 20 read_log 194370'14400 (194370'14399) modify 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by client.1508531.0:1561 2014-08-12 20:30:20.233949 2014-08-13 17:38:02.963622 7fe9bae887a0 20 read_log 194370'14401 (0'0) promote 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by osd.343.0:3957 2014-08-12 20:30:24.174933 2014-08-13 17:38:02.963634 7fe9bae887a0 20 read_log 194370'14402 (194370'14401) modify 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by client.1508531.0:2622 2014-08-12 20:30:24.166720 2014-08-13 17:38:02.963646 7fe9bae887a0 20 read_log 194370'14403 (0'0) promote d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by osd.343.0:3995 2014-08-12 20:30:28.634648 2014-08-13 17:38:02.963658 7fe9bae887a0 20 read_log 194370'14404 (194370'14403) modify d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by client.1508531.0:3819 2014-08-12 20:30:28.626869 2014-08-13 17:38:02.963670 7fe9bae887a0 20 read_log 194370'14405 (0'0) promote eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by osd.343.0:4001 2014-08-12 20:30:30.242898 2014-08-13 17:38:02.963689 7fe9bae887a0 20 read_log 194370'14406 (194370'14405) modify eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by client.1508531.0:4240 2014-08-12 20:30:30.235018 2014-08-13 17:38:02.963702 7fe9bae887a0 20 read_log 194370'14407 (0'0) promote a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by osd.343.0:4047 2014-08-12 20:30:38.825583 2014-08-13 17:38:02.963714 7fe9bae887a0 20 read_log 194370'14408 (194370'14407) modify a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by client.1508531.0:6241 2014-08-12 20:30:38.818632 2014-08-13 17:38:02.963727 7fe9bae887a0 20 read_log 194370'14409 (0'0) promote 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by osd.343.0:4055 2014-08-12 20:30:40.033704 2014-08-13 17:38:02.963739 7fe9bae887a0 20 read_log 194370'14410 (194370'14409) modify 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by client.1508531.0:6512 2014-08-12 20:30:40.024973 2014-08-13 17:38:02.963751 7fe9bae887a0 20 read_log 194370'14411 (0'0) promote 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by osd.343.0:4057 2014-08-12 20:30:40.352287 2014-08-13 17:38:02.963775 7fe9bae887a0 20 read_log 194370'14412 (194370'14411) modify 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by client.1508531.0:6579 2014-08-12 20:30:40.344182 2014-08-13 17:38:02.963788 7fe9bae887a0 20 read_log 194370'14413 (0'0) promote 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by osd.343.0:4073 2014-08-12 20:30:45.078425 2014-08-13 17:38:02.963800 7fe9bae887a0 20 read_log 194370'14414 (194370'14413) modify 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by client.1508531.0:7536 2014-08-12 20:30:45.071702 2014-08-13 17:38:02.963813 7fe9bae887a0 20 read_log 194370'14415 (0'0) promote b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by osd.343.0:4097 2014-08-12 20:30:52.051275 2014-08-13 17:38:02.963825 7fe9bae887a0 20 read_log 194370'14416 (194370'14415) modify b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by client.1508531.0:8800 2014-08-12 20:30:52.042021 2014-08-13 17:38:02.963837 7fe9bae887a0 20 read_log 194370'14417 (0'0) promote 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by osd.343.0:4101 2014-08-12 20:30:53.112165 2014-08-13 17:38:02.963849 7fe9bae887a0 20 read_log 194370'14418 (194370'14417) modify 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by client.1508531.0:8967 2014-08-12 20:30:53.102927 2014-08-13 17:38:02.963861 7fe9bae887a0 20 read_log 194370'14419 (0'0) promote 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by osd.343.0:4107 2014-08-12 20:30:53.696432 2014-08-13 17:38:02.963873 7fe9bae887a0 20 read_log 194370'14420 (194370'14419) modify 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by client.1508531.0:9061 2014-08-12 20:30:53.686310 2014-08-13 17:38:02.963885 7fe9bae887a0 20 read_log 194370'14421 (0'0) promote fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by osd.343.0:4121 2014-08-12 20:30:59.171425 2014-08-13 17:38:02.963897 7fe9bae887a0 20 read_log 194370'14422 (194370'14421) modify fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by client.1508531.0:9984 2014-08-12 20:30:59.165164 2014-08-13 17:38:02.963910 7fe9bae887a0 20 read_log 194370'14423 (0'0) promote e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by osd.343.0:4189 2014-08-12 20:31:22.729143 2014-08-13 17:38:02.963922 7fe9bae887a0 20 read_log 194370'14424 (194370'14423) modify e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by client.1508531.0:13351 2014-08-12 20:31:22.720976 2014-08-13 17:38:02.963934 7fe9bae887a0 20 read_log 194377'14425 (0'0) promote eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by osd.343.0:4231 2014-08-12 20:33:04.930562 2014-08-13 17:38:02.963954 7fe9bae887a0 20 read_log 194377'14426 (194377'14425) modify eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by client.2231190.0:1935 2014-08-12 20:33:04.923832 2014-08-13 17:38:02.963967 7fe9bae887a0 20 read_log 194377'14427 (0'0) promote 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by osd.343.0:4247 2014-08-12 20:33:07.301305 2014-08-13 17:38:02.963979 7fe9bae887a0 20 read_log 194377'14428 (194377'14427) modify 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by client.2231190.0:2514 2014-08-12 20:33:07.293079 2014-08-13 17:38:02.963991 7fe9bae887a0 20 read_log 194377'14429 (0'0) promote 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by osd.343.0:4279 2014-08-12 20:33:16.750358 2014-08-13 17:38:02.964005 7fe9bae887a0 20 read_log 194377'14430 (194377'14429) modify 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by client.2231190.0:4444 2014-08-12 20:33:16.740558 2014-08-13 17:38:02.964017 7fe9bae887a0 20 read_log 194382'14431 (0'0) promote 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by osd.343.0:4305 2014-08-12 20:33:59.332216 2014-08-13 17:38:02.964029 7fe9bae887a0 20 read_log 194382'14432 (194382'14431) modify 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by client.2231190.0:5645 2014-08-12 20:33:59.307896 2014-08-13 17:38:02.964041 7fe9bae887a0 20 read_log 194393'14433 (0'0) promote 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by osd.343.0:4340 2014-08-12 20:47:26.474881 2014-08-13 17:38:02.964053 7fe9bae887a0 20 read_log 194393'14434 (194393'14433) modify 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by client.1064095.0:124 2014-08-12 20:47:26.463639 2014-08-13 17:38:02.964065 7fe9bae887a0 20 read_log 194397'14435 (0'0) promote 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by osd.343.0:4401 2014-08-12 20:53:04.649981 2014-08-13 17:38:02.964077 7fe9bae887a0 20 read_log 194397'14436 (194397'14435) modify 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by client.2235069.0:875 2014-08-12 20:53:04.641065 2014-08-13 17:38:02.964090 7fe9bae887a0 20 read_log 194416'14437 (0'0) promote f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by osd.343.0:4571 2014-08-13 15:28:32.574102 2014-08-13 17:38:02.964102 7fe9bae887a0 20 read_log 194416'14438 (194416'14437) modify f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by client.2441958.0:530 2014-08-13 15:28:32.566858 2014-08-13 17:38:02.964114 7fe9bae887a0 20 read_log 194416'14439 (0'0) promote f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by osd.343.0:4605 2014-08-13 15:28:38.062746 2014-08-13 17:38:02.964128 7fe9bae887a0 20 read_log 194416'14440 (194416'14439) modify f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by client.2441958.0:1939 2014-08-13 15:28:38.053390 2014-08-13 17:38:02.964140 7fe9bae887a0 20 read_log 194416'14441 (0'0) promote ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by osd.343.0:4611 2014-08-13 15:28:38.398844 2014-08-13 17:38:02.964152 7fe9bae887a0 20 read_log 194416'14442 (194416'14441) modify ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by client.2441958.0:2029 2014-08-13 15:28:38.389494 2014-08-13 17:38:02.964172 7fe9bae887a0 20 read_log 194416'14443 (0'0) promote f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by osd.343.0:4655 2014-08-13 15:28:43.547333 2014-08-13 17:38:02.964185 7fe9bae887a0 20 read_log 194416'14444 (194416'14443) modify f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by client.2441958.0:3388 2014-08-13 15:28:43.533618 2014-08-13 17:38:02.964204 7fe9bae887a0 20 read_log 194416'14445 (0'0) promote 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by osd.343.0:4671 2014-08-13 15:28:46.626885 2014-08-13 17:38:02.964217 7fe9bae887a0 20 read_log 194416'14446 (194416'14445) modify 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by client.2441958.0:4170 2014-08-13 15:28:46.617969 2014-08-13 17:38:02.964229 7fe9bae887a0 20 read_log 194416'14447 (0'0) promote 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by osd.343.0:4681 2014-08-13 15:28:47.244598 2014-08-13 17:38:02.964241 7fe9bae887a0 20 read_log 194416'14448 (194416'14447) modify 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by client.2441958.0:4333 2014-08-13 15:28:47.236323 2014-08-13 17:38:02.964253 7fe9bae887a0 20 read_log 194416'14449 (0'0) promote f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by osd.343.0:4703 2014-08-13 15:28:51.259914 2014-08-13 17:38:02.964265 7fe9bae887a0 20 read_log 194416'14450 (194416'14449) modify f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by client.2441958.0:5357 2014-08-13 15:28:51.250657 2014-08-13 17:38:02.964277 7fe9bae887a0 20 read_log 194416'14451 (0'0) promote 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by osd.343.0:4781 2014-08-13 15:29:03.612007 2014-08-13 17:38:02.964289 7fe9bae887a0 20 read_log 194416'14452 (194416'14451) modify 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by client.2441958.0:8551 2014-08-13 15:29:03.602974 2014-08-13 17:38:02.964302 7fe9bae887a0 20 read_log 194416'14453 (0'0) promote 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by osd.343.0:4803 2014-08-13 15:29:09.818220 2014-08-13 17:38:02.964314 7fe9bae887a0 20 read_log 194416'14454 (194416'14453) modify 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by client.2441958.0:10101 2014-08-13 15:29:09.810333 2014-08-13 17:38:02.964326 7fe9bae887a0 20 read_log 194416'14455 (0'0) promote 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by osd.343.0:4833 2014-08-13 15:29:16.404377 2014-08-13 17:38:02.964338 7fe9bae887a0 20 read_log 194416'14456 (194416'14455) modify 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by client.2441958.0:11719 2014-08-13 15:29:16.395680 2014-08-13 17:38:02.964350 7fe9bae887a0 20 read_log 194416'14457 (0'0) promote 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by osd.343.0:4835 2014-08-13 15:29:16.454061 2014-08-13 17:38:02.964361 7fe9bae887a0 20 read_log 194416'14458 (194416'14457) modify 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by client.2441958.0:11733 2014-08-13 15:29:16.446957 2014-08-13 17:38:02.964373 7fe9bae887a0 20 read_log 194416'14459 (0'0) promote 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by osd.343.0:4883 2014-08-13 15:29:25.957169 2014-08-13 17:38:02.964385 7fe9bae887a0 20 read_log 194416'14460 (194416'14459) modify 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by client.2441958.0:13932 2014-08-13 15:29:25.945823 2014-08-13 17:38:02.964397 7fe9bae887a0 20 read_log 194416'14461 (0'0) promote d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by osd.343.0:4905 2014-08-13 15:29:29.240942 2014-08-13 17:38:02.964409 7fe9bae887a0 20 read_log 194416'14462 (194416'14461) modify d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by client.2441958.0:14761 2014-08-13 15:29:29.232650 2014-08-13 17:38:02.964429 7fe9bae887a0 20 read_log 194416'14463 (0'0) promote a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by osd.343.0:4913 2014-08-13 15:29:30.045767 2014-08-13 17:38:02.964442 7fe9bae887a0 20 read_log 194416'14464 (194416'14463) modify a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by client.2441958.0:14945 2014-08-13 15:29:30.039259 2014-08-13 17:38:02.964456 7fe9bae887a0 20 read_log 194416'14465 (0'0) promote 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by osd.343.0:4949 2014-08-13 15:29:44.726877 2014-08-13 17:38:02.964469 7fe9bae887a0 20 read_log 194416'14466 (194416'14465) modify 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by client.2441958.0:16818 2014-08-13 15:29:44.717637 2014-08-13 17:38:02.964481 7fe9bae887a0 20 read_log 194416'14467 (0'0) promote 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by osd.343.0:4961 2014-08-13 15:29:53.472737 2014-08-13 17:38:02.964493 7fe9bae887a0 20 read_log 194416'14468 (194416'14467) modify 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by client.2441958.0:17432 2014-08-13 15:29:53.463866 2014-08-13 17:38:02.964505 7fe9bae887a0 20 read_log 194416'14469 (0'0) promote 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by osd.343.0:4965 2014-08-13 15:29:54.523057 2014-08-13 17:38:02.964517 7fe9bae887a0 20 read_log 194416'14470 (194416'14469) modify 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by client.2441958.0:17505 2014-08-13 15:29:54.513049 2014-08-13 17:38:02.964529 7fe9bae887a0 20 read_log 194422'14471 (0'0) promote c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by osd.343.0:5001 2014-08-13 15:32:18.184016 2014-08-13 17:38:02.964541 7fe9bae887a0 20 read_log 194422'14472 (194422'14471) modify c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by client.1148203.0:707 2014-08-13 15:32:18.176089 2014-08-13 17:38:02.964553 7fe9bae887a0 20 read_log 194422'14473 (0'0) promote d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by osd.343.0:5035 2014-08-13 15:32:24.791144 2014-08-13 17:38:02.964565 7fe9bae887a0 20 read_log 194422'14474 (194422'14473) modify d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by client.1148203.0:2407 2014-08-13 15:32:24.783012 2014-08-13 17:38:02.964578 7fe9bae887a0 20 read_log 194422'14475 (0'0) promote ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by osd.343.0:5073 2014-08-13 15:32:29.872702 2014-08-13 17:38:02.964590 7fe9bae887a0 20 read_log 194422'14476 (194422'14475) modify ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by client.1148203.0:3701 2014-08-13 15:32:29.864799 2014-08-13 17:38:02.964605 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.965015 7fe9bae887a0 10 read_log checking for missing items over interval (194370'14395,194422'14476] 2014-08-13 17:38:02.965062 7fe9bae887a0 15 read_log missing 194422'14476 (194422'14475) modify ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by client.1148203.0:3701 2014-08-13 15:32:29.864799 2014-08-13 17:38:02.965104 7fe9bae887a0 15 read_log missing 194422'14474 (194422'14473) modify d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by client.1148203.0:2407 2014-08-13 15:32:24.783012 2014-08-13 17:38:02.965142 7fe9bae887a0 15 read_log missing 194422'14472 (194422'14471) modify c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by client.1148203.0:707 2014-08-13 15:32:18.176089 2014-08-13 17:38:02.965184 7fe9bae887a0 15 read_log missing 194416'14470 (194416'14469) modify 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by client.2441958.0:17505 2014-08-13 15:29:54.513049 2014-08-13 17:38:02.965221 7fe9bae887a0 15 read_log missing 194416'14468 (194416'14467) modify 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by client.2441958.0:17432 2014-08-13 15:29:53.463866 2014-08-13 17:38:02.965258 7fe9bae887a0 15 read_log missing 194416'14466 (194416'14465) modify 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by client.2441958.0:16818 2014-08-13 15:29:44.717637 2014-08-13 17:38:02.965295 7fe9bae887a0 15 read_log missing 194416'14464 (194416'14463) modify a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by client.2441958.0:14945 2014-08-13 15:29:30.039259 2014-08-13 17:38:02.965331 7fe9bae887a0 15 read_log missing 194416'14462 (194416'14461) modify d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by client.2441958.0:14761 2014-08-13 15:29:29.232650 2014-08-13 17:38:02.965368 7fe9bae887a0 15 read_log missing 194416'14460 (194416'14459) modify 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by client.2441958.0:13932 2014-08-13 15:29:25.945823 2014-08-13 17:38:02.965404 7fe9bae887a0 15 read_log missing 194416'14458 (194416'14457) modify 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by client.2441958.0:11733 2014-08-13 15:29:16.446957 2014-08-13 17:38:02.965441 7fe9bae887a0 15 read_log missing 194416'14456 (194416'14455) modify 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by client.2441958.0:11719 2014-08-13 15:29:16.395680 2014-08-13 17:38:02.965478 7fe9bae887a0 15 read_log missing 194416'14454 (194416'14453) modify 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by client.2441958.0:10101 2014-08-13 15:29:09.810333 2014-08-13 17:38:02.965514 7fe9bae887a0 15 read_log missing 194416'14452 (194416'14451) modify 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by client.2441958.0:8551 2014-08-13 15:29:03.602974 2014-08-13 17:38:02.965551 7fe9bae887a0 15 read_log missing 194416'14450 (194416'14449) modify f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by client.2441958.0:5357 2014-08-13 15:28:51.250657 2014-08-13 17:38:02.965587 7fe9bae887a0 15 read_log missing 194416'14448 (194416'14447) modify 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by client.2441958.0:4333 2014-08-13 15:28:47.236323 2014-08-13 17:38:02.965645 7fe9bae887a0 15 read_log missing 194416'14446 (194416'14445) modify 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by client.2441958.0:4170 2014-08-13 15:28:46.617969 2014-08-13 17:38:02.965689 7fe9bae887a0 15 read_log missing 194416'14444 (194416'14443) modify f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by client.2441958.0:3388 2014-08-13 15:28:43.533618 2014-08-13 17:38:02.965727 7fe9bae887a0 15 read_log missing 194416'14442 (194416'14441) modify ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by client.2441958.0:2029 2014-08-13 15:28:38.389494 2014-08-13 17:38:02.965763 7fe9bae887a0 15 read_log missing 194416'14440 (194416'14439) modify f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by client.2441958.0:1939 2014-08-13 15:28:38.053390 2014-08-13 17:38:02.965799 7fe9bae887a0 15 read_log missing 194416'14438 (194416'14437) modify f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by client.2441958.0:530 2014-08-13 15:28:32.566858 2014-08-13 17:38:02.965835 7fe9bae887a0 15 read_log missing 194397'14436 (194397'14435) modify 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by client.2235069.0:875 2014-08-12 20:53:04.641065 2014-08-13 17:38:02.965871 7fe9bae887a0 15 read_log missing 194393'14434 (194393'14433) modify 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by client.1064095.0:124 2014-08-12 20:47:26.463639 2014-08-13 17:38:02.965907 7fe9bae887a0 15 read_log missing 194382'14432 (194382'14431) modify 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by client.2231190.0:5645 2014-08-12 20:33:59.307896 2014-08-13 17:38:02.965948 7fe9bae887a0 15 read_log missing 194377'14430 (194377'14429) modify 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by client.2231190.0:4444 2014-08-12 20:33:16.740558 2014-08-13 17:38:02.965986 7fe9bae887a0 15 read_log missing 194377'14428 (194377'14427) modify 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by client.2231190.0:2514 2014-08-12 20:33:07.293079 2014-08-13 17:38:02.966029 7fe9bae887a0 15 read_log missing 194377'14426 (194377'14425) modify eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by client.2231190.0:1935 2014-08-12 20:33:04.923832 2014-08-13 17:38:02.966067 7fe9bae887a0 15 read_log missing 194370'14424 (194370'14423) modify e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by client.1508531.0:13351 2014-08-12 20:31:22.720976 2014-08-13 17:38:02.966104 7fe9bae887a0 15 read_log missing 194370'14422 (194370'14421) modify fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by client.1508531.0:9984 2014-08-12 20:30:59.165164 2014-08-13 17:38:02.966141 7fe9bae887a0 15 read_log missing 194370'14420 (194370'14419) modify 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by client.1508531.0:9061 2014-08-12 20:30:53.686310 2014-08-13 17:38:02.966187 7fe9bae887a0 15 read_log missing 194370'14418 (194370'14417) modify 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by client.1508531.0:8967 2014-08-12 20:30:53.102927 2014-08-13 17:38:02.966225 7fe9bae887a0 15 read_log missing 194370'14416 (194370'14415) modify b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by client.1508531.0:8800 2014-08-12 20:30:52.042021 2014-08-13 17:38:02.966261 7fe9bae887a0 15 read_log missing 194370'14414 (194370'14413) modify 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by client.1508531.0:7536 2014-08-12 20:30:45.071702 2014-08-13 17:38:02.966298 7fe9bae887a0 15 read_log missing 194370'14412 (194370'14411) modify 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by client.1508531.0:6579 2014-08-12 20:30:40.344182 2014-08-13 17:38:02.966334 7fe9bae887a0 15 read_log missing 194370'14410 (194370'14409) modify 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by client.1508531.0:6512 2014-08-12 20:30:40.024973 2014-08-13 17:38:02.966371 7fe9bae887a0 15 read_log missing 194370'14408 (194370'14407) modify a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by client.1508531.0:6241 2014-08-12 20:30:38.818632 2014-08-13 17:38:02.966408 7fe9bae887a0 15 read_log missing 194370'14406 (194370'14405) modify eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by client.1508531.0:4240 2014-08-12 20:30:30.235018 2014-08-13 17:38:02.966445 7fe9bae887a0 15 read_log missing 194370'14404 (194370'14403) modify d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by client.1508531.0:3819 2014-08-12 20:30:28.626869 2014-08-13 17:38:02.966482 7fe9bae887a0 15 read_log missing 194370'14402 (194370'14401) modify 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by client.1508531.0:2622 2014-08-12 20:30:24.166720 2014-08-13 17:38:02.966519 7fe9bae887a0 15 read_log missing 194370'14400 (194370'14399) modify 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by client.1508531.0:1561 2014-08-12 20:30:20.233949 2014-08-13 17:38:02.966556 7fe9bae887a0 15 read_log missing 194370'14398 (194370'14397) modify 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by client.1508531.0:1047 2014-08-12 20:30:18.286872 2014-08-13 17:38:02.966593 7fe9bae887a0 15 read_log missing 194370'14396 (194370'14395) modify 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by client.1508531.0:402 2014-08-12 20:30:15.898415 2014-08-13 17:38:02.966603 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.966631 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=0 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive m=41] handle_loaded 2014-08-13 17:38:02.966648 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=0 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Initial 0.013233 0 0.000000 2014-08-13 17:38:02.966660 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=0 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset 2014-08-13 17:38:02.966668 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=0 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194477 2014-08-13 17:38:02.966677 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log((192718'11573,194422'14476], crt=194422'14476) 2014-08-13 17:38:02.966687 7fe9bae887a0 10 osd.4 194478 pgid 24.20e coll 24.20e_head 2014-08-13 17:38:02.966795 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.20e 2014-08-13 17:38:02.966808 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.966840 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.20e(unlocked)] enter Initial 2014-08-13 17:38:02.966852 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.20e(unlocked)] enter NotTrimming 2014-08-13 17:38:02.966978 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.967082 7fe9bae887a0 20 read_log 192718'8902 (0'0) promote 1a453a0e/core.19866+13+2/head//24 by osd.367.0:40779 2014-08-06 15:33:39.061719 2014-08-13 17:38:02.967103 7fe9bae887a0 20 read_log 192718'8903 (192718'8902) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:1 2014-08-06 15:33:27.695325 2014-08-13 17:38:02.967116 7fe9bae887a0 20 read_log 192718'8904 (192718'8903) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:1 2014-08-06 15:33:27.688105 2014-08-13 17:38:02.967128 7fe9bae887a0 20 read_log 192718'8905 (192718'8904) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:2 2014-08-06 15:33:40.521781 2014-08-13 17:38:02.967140 7fe9bae887a0 20 read_log 192718'8906 (192718'8905) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:2 2014-08-06 15:33:40.522254 2014-08-13 17:38:02.967152 7fe9bae887a0 20 read_log 192718'8907 (192718'8906) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:3 2014-08-06 15:33:40.691386 2014-08-13 17:38:02.967177 7fe9bae887a0 20 read_log 192718'8908 (192718'8907) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:3 2014-08-06 15:33:40.693838 2014-08-13 17:38:02.967192 7fe9bae887a0 20 read_log 192718'8909 (192718'8908) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:4 2014-08-06 15:33:40.741564 2014-08-13 17:38:02.967204 7fe9bae887a0 20 read_log 192718'8910 (192718'8909) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:4 2014-08-06 15:33:40.773085 2014-08-13 17:38:02.967216 7fe9bae887a0 20 read_log 192718'8911 (192718'8910) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:5 2014-08-06 15:33:40.813135 2014-08-13 17:38:02.967228 7fe9bae887a0 20 read_log 192718'8912 (192718'8911) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:5 2014-08-06 15:33:40.843554 2014-08-13 17:38:02.967240 7fe9bae887a0 20 read_log 192718'8913 (192718'8912) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:6 2014-08-06 15:33:40.882714 2014-08-13 17:38:02.967252 7fe9bae887a0 20 read_log 192718'8914 (192718'8913) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:6 2014-08-06 15:33:40.913481 2014-08-13 17:38:02.967263 7fe9bae887a0 20 read_log 192718'8915 (192718'8914) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:7 2014-08-06 15:33:40.951275 2014-08-13 17:38:02.967278 7fe9bae887a0 20 read_log 192718'8916 (192718'8915) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:7 2014-08-06 15:33:40.982908 2014-08-13 17:38:02.967290 7fe9bae887a0 20 read_log 192718'8917 (192718'8916) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:8 2014-08-06 15:33:41.020753 2014-08-13 17:38:02.967303 7fe9bae887a0 20 read_log 192718'8918 (192718'8917) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:8 2014-08-06 15:33:41.052491 2014-08-13 17:38:02.967325 7fe9bae887a0 20 read_log 192718'8919 (192718'8918) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:9 2014-08-06 15:33:41.090937 2014-08-13 17:38:02.967338 7fe9bae887a0 20 read_log 192718'8920 (192718'8919) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:9 2014-08-06 15:33:42.178893 2014-08-13 17:38:02.967350 7fe9bae887a0 20 read_log 192718'8921 (192718'8920) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:10 2014-08-06 15:33:42.183573 2014-08-13 17:38:02.967362 7fe9bae887a0 20 read_log 192718'8922 (192718'8921) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:10 2014-08-06 15:33:42.210413 2014-08-13 17:38:02.967374 7fe9bae887a0 20 read_log 192718'8923 (192718'8922) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:11 2014-08-06 15:33:42.215797 2014-08-13 17:38:02.967385 7fe9bae887a0 20 read_log 192718'8924 (192718'8923) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:11 2014-08-06 15:33:42.237654 2014-08-13 17:38:02.967397 7fe9bae887a0 20 read_log 192718'8925 (192718'8924) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:12 2014-08-06 15:33:42.242515 2014-08-13 17:38:02.967409 7fe9bae887a0 20 read_log 192718'8926 (192718'8925) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:12 2014-08-06 15:33:42.266075 2014-08-13 17:38:02.967421 7fe9bae887a0 20 read_log 192718'8927 (192718'8926) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:13 2014-08-06 15:33:42.270236 2014-08-13 17:38:02.967434 7fe9bae887a0 20 read_log 192718'8928 (192718'8927) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:13 2014-08-06 15:33:42.288280 2014-08-13 17:38:02.967446 7fe9bae887a0 20 read_log 192718'8929 (192718'8928) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:14 2014-08-06 15:33:42.295655 2014-08-13 17:38:02.967458 7fe9bae887a0 20 read_log 192718'8930 (192718'8929) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:14 2014-08-06 15:33:42.315139 2014-08-13 17:38:02.967470 7fe9bae887a0 20 read_log 192718'8931 (192718'8930) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:15 2014-08-06 15:33:42.321287 2014-08-13 17:38:02.967481 7fe9bae887a0 20 read_log 192718'8932 (192718'8931) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:15 2014-08-06 15:33:42.340571 2014-08-13 17:38:02.967493 7fe9bae887a0 20 read_log 192718'8933 (192718'8932) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:16 2014-08-06 15:33:42.348789 2014-08-13 17:38:02.967505 7fe9bae887a0 20 read_log 192718'8934 (192718'8933) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:17 2014-08-06 15:33:42.375567 2014-08-13 17:38:02.967516 7fe9bae887a0 20 read_log 192718'8935 (192718'8934) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:16 2014-08-06 15:33:42.385460 2014-08-13 17:38:02.967528 7fe9bae887a0 20 read_log 192718'8936 (192718'8935) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:18 2014-08-06 15:33:42.400075 2014-08-13 17:38:02.967540 7fe9bae887a0 20 read_log 192718'8937 (192718'8936) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:17 2014-08-06 15:33:42.408147 2014-08-13 17:38:02.967551 7fe9bae887a0 20 read_log 192718'8938 (192718'8937) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:19 2014-08-06 15:33:42.425381 2014-08-13 17:38:02.967563 7fe9bae887a0 20 read_log 192718'8939 (192718'8938) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:18 2014-08-06 15:33:42.430404 2014-08-13 17:38:02.967574 7fe9bae887a0 20 read_log 192718'8940 (192718'8939) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:20 2014-08-06 15:33:42.454455 2014-08-13 17:38:02.967586 7fe9bae887a0 20 read_log 192718'8941 (192718'8940) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:19 2014-08-06 15:33:42.486319 2014-08-13 17:38:02.967598 7fe9bae887a0 20 read_log 192718'8942 (192718'8941) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:21 2014-08-06 15:33:42.523677 2014-08-13 17:38:02.967634 7fe9bae887a0 20 read_log 192718'8943 (192718'8942) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:20 2014-08-06 15:33:42.556224 2014-08-13 17:38:02.967648 7fe9bae887a0 20 read_log 192718'8944 (192718'8943) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:22 2014-08-06 15:33:42.593423 2014-08-13 17:38:02.967660 7fe9bae887a0 20 read_log 192718'8945 (192718'8944) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:21 2014-08-06 15:33:42.625831 2014-08-13 17:38:02.967672 7fe9bae887a0 20 read_log 192718'8946 (192718'8945) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:23 2014-08-06 15:33:42.870238 2014-08-13 17:38:02.967684 7fe9bae887a0 20 read_log 192718'8947 (192718'8946) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:22 2014-08-06 15:33:42.908163 2014-08-13 17:38:02.967695 7fe9bae887a0 20 read_log 192718'8948 (192718'8947) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:24 2014-08-06 15:33:42.944674 2014-08-13 17:38:02.967707 7fe9bae887a0 20 read_log 192718'8949 (192718'8948) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:23 2014-08-06 15:33:42.977335 2014-08-13 17:38:02.967719 7fe9bae887a0 20 read_log 192718'8950 (192718'8949) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:25 2014-08-06 15:33:43.013856 2014-08-13 17:38:02.967731 7fe9bae887a0 20 read_log 192718'8951 (192718'8950) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:24 2014-08-06 15:33:43.046956 2014-08-13 17:38:02.967743 7fe9bae887a0 20 read_log 192718'8952 (192718'8951) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:26 2014-08-06 15:33:43.083178 2014-08-13 17:38:02.967755 7fe9bae887a0 20 read_log 192718'8953 (192718'8952) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:25 2014-08-06 15:33:43.116748 2014-08-13 17:38:02.967767 7fe9bae887a0 20 read_log 192718'8954 (192718'8953) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:27 2014-08-06 15:33:43.152991 2014-08-13 17:38:02.967778 7fe9bae887a0 20 read_log 192718'8955 (192718'8954) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:26 2014-08-06 15:33:43.187284 2014-08-13 17:38:02.967790 7fe9bae887a0 20 read_log 192718'8956 (192718'8955) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:28 2014-08-06 15:33:43.401362 2014-08-13 17:38:02.967802 7fe9bae887a0 20 read_log 192718'8957 (192718'8956) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:27 2014-08-06 15:33:43.434874 2014-08-13 17:38:02.967813 7fe9bae887a0 20 read_log 192718'8958 (192718'8957) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:29 2014-08-06 15:33:43.468718 2014-08-13 17:38:02.967825 7fe9bae887a0 20 read_log 192718'8959 (192718'8958) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:28 2014-08-06 15:33:43.501726 2014-08-13 17:38:02.967837 7fe9bae887a0 20 read_log 192718'8960 (192718'8959) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:30 2014-08-06 15:33:43.534999 2014-08-13 17:38:02.967848 7fe9bae887a0 20 read_log 192718'8961 (192718'8960) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:29 2014-08-06 15:33:43.567675 2014-08-13 17:38:02.967860 7fe9bae887a0 20 read_log 192718'8962 (192718'8961) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:31 2014-08-06 15:33:43.602488 2014-08-13 17:38:02.967872 7fe9bae887a0 20 read_log 192718'8963 (192718'8962) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:30 2014-08-06 15:33:43.634610 2014-08-13 17:38:02.967883 7fe9bae887a0 20 read_log 192718'8964 (192718'8963) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:32 2014-08-06 15:33:43.669976 2014-08-13 17:38:02.967895 7fe9bae887a0 20 read_log 192718'8965 (192718'8964) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:31 2014-08-06 15:33:43.701629 2014-08-13 17:38:02.967906 7fe9bae887a0 20 read_log 192718'8966 (192718'8965) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:33 2014-08-06 15:33:43.902819 2014-08-13 17:38:02.967925 7fe9bae887a0 20 read_log 192718'8967 (192718'8966) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:32 2014-08-06 15:33:43.940740 2014-08-13 17:38:02.967937 7fe9bae887a0 20 read_log 192718'8968 (192718'8967) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:34 2014-08-06 15:33:43.973932 2014-08-13 17:38:02.967949 7fe9bae887a0 20 read_log 192718'8969 (192718'8968) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:33 2014-08-06 15:33:44.007865 2014-08-13 17:38:02.967960 7fe9bae887a0 20 read_log 192718'8970 (192718'8969) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:35 2014-08-06 15:33:44.043451 2014-08-13 17:38:02.967972 7fe9bae887a0 20 read_log 192718'8971 (192718'8970) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:34 2014-08-06 15:33:44.076726 2014-08-13 17:38:02.967984 7fe9bae887a0 20 read_log 192718'8972 (192718'8971) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:36 2014-08-06 15:33:44.109684 2014-08-13 17:38:02.967996 7fe9bae887a0 20 read_log 192718'8973 (192718'8972) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:35 2014-08-06 15:33:44.142168 2014-08-13 17:38:02.968008 7fe9bae887a0 20 read_log 192718'8974 (192718'8973) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:37 2014-08-06 15:33:44.178451 2014-08-13 17:38:02.968019 7fe9bae887a0 20 read_log 192718'8975 (192718'8974) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:36 2014-08-06 15:33:44.208912 2014-08-13 17:38:02.968031 7fe9bae887a0 20 read_log 192718'8976 (192718'8975) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:38 2014-08-06 15:33:44.413554 2014-08-13 17:38:02.968043 7fe9bae887a0 20 read_log 192718'8977 (192718'8976) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:37 2014-08-06 15:33:44.448297 2014-08-13 17:38:02.968054 7fe9bae887a0 20 read_log 192718'8978 (192718'8977) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:39 2014-08-06 15:33:44.485195 2014-08-13 17:38:02.968066 7fe9bae887a0 20 read_log 192718'8979 (192718'8978) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:38 2014-08-06 15:33:44.512953 2014-08-13 17:38:02.968077 7fe9bae887a0 20 read_log 192718'8980 (192718'8979) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:40 2014-08-06 15:33:44.551863 2014-08-13 17:38:02.968089 7fe9bae887a0 20 read_log 192718'8981 (192718'8980) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:39 2014-08-06 15:33:44.582111 2014-08-13 17:38:02.968100 7fe9bae887a0 20 read_log 192718'8982 (192718'8981) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:41 2014-08-06 15:33:44.629915 2014-08-13 17:38:02.968112 7fe9bae887a0 20 read_log 192718'8983 (192718'8982) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:40 2014-08-06 15:33:44.648730 2014-08-13 17:38:02.968142 7fe9bae887a0 20 read_log 192718'8984 (192718'8983) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:42 2014-08-06 15:33:44.685656 2014-08-13 17:38:02.968160 7fe9bae887a0 20 read_log 192718'8985 (192718'8984) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:41 2014-08-06 15:33:44.716541 2014-08-13 17:38:02.968172 7fe9bae887a0 20 read_log 192718'8986 (192718'8985) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:43 2014-08-06 15:33:44.921162 2014-08-13 17:38:02.968184 7fe9bae887a0 20 read_log 192718'8987 (192718'8986) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:42 2014-08-06 15:33:44.951653 2014-08-13 17:38:02.968196 7fe9bae887a0 20 read_log 192718'8988 (192718'8987) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:44 2014-08-06 15:33:44.989866 2014-08-13 17:38:02.968208 7fe9bae887a0 20 read_log 192718'8989 (192718'8988) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:43 2014-08-06 15:33:45.020128 2014-08-13 17:38:02.968219 7fe9bae887a0 20 read_log 192718'8990 (192718'8989) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:45 2014-08-06 15:33:45.054932 2014-08-13 17:38:02.968238 7fe9bae887a0 20 read_log 192718'8991 (192718'8990) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:44 2014-08-06 15:33:45.090444 2014-08-13 17:38:02.968250 7fe9bae887a0 20 read_log 192718'8992 (192718'8991) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:46 2014-08-06 15:33:45.132712 2014-08-13 17:38:02.968262 7fe9bae887a0 20 read_log 192718'8993 (192718'8992) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:45 2014-08-06 15:33:45.165385 2014-08-13 17:38:02.968274 7fe9bae887a0 20 read_log 192718'8994 (192718'8993) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:47 2014-08-06 15:33:45.205637 2014-08-13 17:38:02.968305 7fe9bae887a0 20 read_log 192718'8995 (192718'8994) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:46 2014-08-06 15:33:45.240480 2014-08-13 17:38:02.968318 7fe9bae887a0 20 read_log 192718'8996 (192718'8995) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:48 2014-08-06 15:33:45.468574 2014-08-13 17:38:02.968330 7fe9bae887a0 20 read_log 192718'8997 (192718'8996) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:47 2014-08-06 15:33:45.509095 2014-08-13 17:38:02.968342 7fe9bae887a0 20 read_log 192718'8998 (192718'8997) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:49 2014-08-06 15:33:45.548182 2014-08-13 17:38:02.968354 7fe9bae887a0 20 read_log 192718'8999 (192718'8998) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:48 2014-08-06 15:33:45.583813 2014-08-13 17:38:02.968366 7fe9bae887a0 20 read_log 192718'9000 (192718'8999) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:50 2014-08-06 15:33:45.627773 2014-08-13 17:38:02.968377 7fe9bae887a0 20 read_log 192718'9001 (192718'9000) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:49 2014-08-06 15:33:45.662404 2014-08-13 17:38:02.968389 7fe9bae887a0 20 read_log 192718'9002 (192718'9001) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:51 2014-08-06 15:33:45.694376 2014-08-13 17:38:02.968400 7fe9bae887a0 20 read_log 192718'9003 (192718'9002) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:50 2014-08-06 15:33:45.719720 2014-08-13 17:38:02.968412 7fe9bae887a0 20 read_log 192718'9004 (192718'9003) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:52 2014-08-06 15:33:45.788765 2014-08-13 17:38:02.968424 7fe9bae887a0 20 read_log 192718'9005 (192718'9004) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:51 2014-08-06 15:33:45.859312 2014-08-13 17:38:02.968435 7fe9bae887a0 20 read_log 192718'9006 (192718'9005) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:53 2014-08-06 15:33:45.908071 2014-08-13 17:38:02.968447 7fe9bae887a0 20 read_log 192718'9007 (192718'9006) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:52 2014-08-06 15:33:45.960220 2014-08-13 17:38:02.968459 7fe9bae887a0 20 read_log 192718'9008 (192718'9007) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:54 2014-08-06 15:33:46.014389 2014-08-13 17:38:02.968470 7fe9bae887a0 20 read_log 192718'9009 (192718'9008) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:53 2014-08-06 15:33:46.042290 2014-08-13 17:38:02.968482 7fe9bae887a0 20 read_log 192718'9010 (192718'9009) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:55 2014-08-06 15:33:46.074016 2014-08-13 17:38:02.968494 7fe9bae887a0 20 read_log 192718'9011 (192718'9010) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:54 2014-08-06 15:33:46.098642 2014-08-13 17:38:02.968507 7fe9bae887a0 20 read_log 192718'9012 (192718'9011) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:56 2014-08-06 15:33:46.127015 2014-08-13 17:38:02.968518 7fe9bae887a0 20 read_log 192718'9013 (192718'9012) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:55 2014-08-06 15:33:46.152137 2014-08-13 17:38:02.968530 7fe9bae887a0 20 read_log 192718'9014 (192718'9013) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:57 2014-08-06 15:33:46.180304 2014-08-13 17:38:02.968551 7fe9bae887a0 20 read_log 192718'9015 (192718'9014) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:56 2014-08-06 15:33:46.205526 2014-08-13 17:38:02.968563 7fe9bae887a0 20 read_log 192718'9016 (192718'9015) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:58 2014-08-06 15:33:46.233485 2014-08-13 17:38:02.968575 7fe9bae887a0 20 read_log 192718'9017 (192718'9016) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:57 2014-08-06 15:33:46.256496 2014-08-13 17:38:02.968587 7fe9bae887a0 20 read_log 192718'9018 (192718'9017) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:59 2014-08-06 15:33:46.288865 2014-08-13 17:38:02.968599 7fe9bae887a0 20 read_log 192718'9019 (192718'9018) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:58 2014-08-06 15:33:46.312524 2014-08-13 17:38:02.968610 7fe9bae887a0 20 read_log 192718'9020 (192718'9019) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:60 2014-08-06 15:33:46.341421 2014-08-13 17:38:02.968622 7fe9bae887a0 20 read_log 192718'9021 (192718'9020) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:59 2014-08-06 15:33:46.534739 2014-08-13 17:38:02.968633 7fe9bae887a0 20 read_log 192718'9022 (192718'9021) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:61 2014-08-06 15:33:46.563086 2014-08-13 17:38:02.968645 7fe9bae887a0 20 read_log 192718'9023 (192718'9022) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:60 2014-08-06 15:33:46.589751 2014-08-13 17:38:02.968657 7fe9bae887a0 20 read_log 192718'9024 (192718'9023) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:62 2014-08-06 15:33:46.616991 2014-08-13 17:38:02.968668 7fe9bae887a0 20 read_log 192718'9025 (192718'9024) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:61 2014-08-06 15:33:46.644054 2014-08-13 17:38:02.968680 7fe9bae887a0 20 read_log 192718'9026 (192718'9025) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:63 2014-08-06 15:33:46.670383 2014-08-13 17:38:02.968691 7fe9bae887a0 20 read_log 192718'9027 (192718'9026) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:62 2014-08-06 15:33:46.696713 2014-08-13 17:38:02.968703 7fe9bae887a0 20 read_log 192718'9028 (192718'9027) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:64 2014-08-06 15:33:46.723494 2014-08-13 17:38:02.968716 7fe9bae887a0 20 read_log 192718'9029 (192718'9028) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:63 2014-08-06 15:33:46.746932 2014-08-13 17:38:02.968728 7fe9bae887a0 20 read_log 192718'9030 (192718'9029) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:65 2014-08-06 15:33:46.775157 2014-08-13 17:38:02.968740 7fe9bae887a0 20 read_log 192718'9031 (192718'9030) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:64 2014-08-06 15:33:46.938135 2014-08-13 17:38:02.968751 7fe9bae887a0 20 read_log 192718'9032 (192718'9031) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:66 2014-08-06 15:33:46.967041 2014-08-13 17:38:02.968764 7fe9bae887a0 20 read_log 192718'9033 (192718'9032) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:65 2014-08-06 15:33:46.993449 2014-08-13 17:38:02.968776 7fe9bae887a0 20 read_log 192718'9034 (192718'9033) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:67 2014-08-06 15:33:47.018161 2014-08-13 17:38:02.968788 7fe9bae887a0 20 read_log 192718'9035 (192718'9034) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:66 2014-08-06 15:33:47.042690 2014-08-13 17:38:02.968799 7fe9bae887a0 20 read_log 192718'9036 (192718'9035) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:68 2014-08-06 15:33:47.069738 2014-08-13 17:38:02.968811 7fe9bae887a0 20 read_log 192718'9037 (192718'9036) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:67 2014-08-06 15:33:47.098292 2014-08-13 17:38:02.968822 7fe9bae887a0 20 read_log 192718'9038 (192718'9037) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:69 2014-08-06 15:33:47.144123 2014-08-13 17:38:02.968840 7fe9bae887a0 20 read_log 192718'9039 (192718'9038) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:68 2014-08-06 15:33:47.193446 2014-08-13 17:38:02.968853 7fe9bae887a0 20 read_log 192718'9040 (192718'9039) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:70 2014-08-06 15:33:47.220413 2014-08-13 17:38:02.968865 7fe9bae887a0 20 read_log 192718'9041 (192718'9040) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:69 2014-08-06 15:33:47.373971 2014-08-13 17:38:02.968876 7fe9bae887a0 20 read_log 192718'9042 (192718'9041) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:71 2014-08-06 15:33:47.403297 2014-08-13 17:38:02.968888 7fe9bae887a0 20 read_log 192718'9043 (192718'9042) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:70 2014-08-06 15:33:47.428720 2014-08-13 17:38:02.968900 7fe9bae887a0 20 read_log 192718'9044 (192718'9043) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:72 2014-08-06 15:33:47.453769 2014-08-13 17:38:02.968912 7fe9bae887a0 20 read_log 192718'9045 (192718'9044) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:71 2014-08-06 15:33:47.481730 2014-08-13 17:38:02.968924 7fe9bae887a0 20 read_log 192718'9046 (192718'9045) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:73 2014-08-06 15:33:47.505702 2014-08-13 17:38:02.968935 7fe9bae887a0 20 read_log 192718'9047 (192718'9046) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:72 2014-08-06 15:33:47.531510 2014-08-13 17:38:02.968947 7fe9bae887a0 20 read_log 192718'9048 (192718'9047) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:74 2014-08-06 15:33:47.556849 2014-08-13 17:38:02.968959 7fe9bae887a0 20 read_log 192718'9049 (192718'9048) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:73 2014-08-06 15:33:47.582648 2014-08-13 17:38:02.968970 7fe9bae887a0 20 read_log 192718'9050 (192718'9049) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:75 2014-08-06 15:33:47.608495 2014-08-13 17:38:02.968982 7fe9bae887a0 20 read_log 192718'9051 (192718'9050) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:74 2014-08-06 15:33:47.758962 2014-08-13 17:38:02.968994 7fe9bae887a0 20 read_log 192718'9052 (192718'9051) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:76 2014-08-06 15:33:47.784831 2014-08-13 17:38:02.969006 7fe9bae887a0 20 read_log 192718'9053 (192718'9052) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:75 2014-08-06 15:33:47.813206 2014-08-13 17:38:02.969018 7fe9bae887a0 20 read_log 192718'9054 (192718'9053) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:77 2014-08-06 15:33:47.838667 2014-08-13 17:38:02.969029 7fe9bae887a0 20 read_log 192718'9055 (192718'9054) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:76 2014-08-06 15:33:47.865093 2014-08-13 17:38:02.969041 7fe9bae887a0 20 read_log 192718'9056 (192718'9055) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:78 2014-08-06 15:33:47.890555 2014-08-13 17:38:02.969053 7fe9bae887a0 20 read_log 192718'9057 (192718'9056) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:77 2014-08-06 15:33:47.916152 2014-08-13 17:38:02.969065 7fe9bae887a0 20 read_log 192718'9058 (192718'9057) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:79 2014-08-06 15:33:47.941991 2014-08-13 17:38:02.969076 7fe9bae887a0 20 read_log 192718'9059 (192718'9058) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:78 2014-08-06 15:33:47.967509 2014-08-13 17:38:02.969088 7fe9bae887a0 20 read_log 192718'9060 (192718'9059) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:80 2014-08-06 15:33:47.992933 2014-08-13 17:38:02.969100 7fe9bae887a0 20 read_log 192718'9061 (192718'9060) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:79 2014-08-06 15:33:48.146960 2014-08-13 17:38:02.969112 7fe9bae887a0 20 read_log 192718'9062 (192718'9061) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:81 2014-08-06 15:33:48.175445 2014-08-13 17:38:02.969129 7fe9bae887a0 20 read_log 192718'9063 (192718'9062) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:80 2014-08-06 15:33:48.201339 2014-08-13 17:38:02.969142 7fe9bae887a0 20 read_log 192718'9064 (192718'9063) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:82 2014-08-06 15:33:48.226824 2014-08-13 17:38:02.969158 7fe9bae887a0 20 read_log 192718'9065 (192718'9064) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:81 2014-08-06 15:33:48.249480 2014-08-13 17:38:02.969170 7fe9bae887a0 20 read_log 192718'9066 (192718'9065) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:83 2014-08-06 15:33:48.276078 2014-08-13 17:38:02.969182 7fe9bae887a0 20 read_log 192718'9067 (192718'9066) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:82 2014-08-06 15:33:48.301039 2014-08-13 17:38:02.969194 7fe9bae887a0 20 read_log 192718'9068 (192718'9067) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:84 2014-08-06 15:33:48.327004 2014-08-13 17:38:02.969206 7fe9bae887a0 20 read_log 192718'9069 (192718'9068) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:83 2014-08-06 15:33:48.354868 2014-08-13 17:38:02.969217 7fe9bae887a0 20 read_log 192718'9070 (192718'9069) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:85 2014-08-06 15:33:48.381248 2014-08-13 17:38:02.969229 7fe9bae887a0 20 read_log 192718'9071 (192718'9070) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:84 2014-08-06 15:33:48.543411 2014-08-13 17:38:02.969241 7fe9bae887a0 20 read_log 192718'9072 (192718'9071) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:86 2014-08-06 15:33:48.563574 2014-08-13 17:38:02.969252 7fe9bae887a0 20 read_log 192718'9073 (192718'9072) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:85 2014-08-06 15:33:48.589747 2014-08-13 17:38:02.969264 7fe9bae887a0 20 read_log 192718'9074 (192718'9073) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:87 2014-08-06 15:33:48.618150 2014-08-13 17:38:02.969276 7fe9bae887a0 20 read_log 192718'9075 (192718'9074) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:86 2014-08-06 15:33:48.643471 2014-08-13 17:38:02.969288 7fe9bae887a0 20 read_log 192718'9076 (192718'9075) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:88 2014-08-06 15:33:48.669740 2014-08-13 17:38:02.969316 7fe9bae887a0 20 read_log 192718'9077 (192718'9076) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:87 2014-08-06 15:33:48.696564 2014-08-13 17:38:02.969329 7fe9bae887a0 20 read_log 192718'9078 (192718'9077) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:89 2014-08-06 15:33:48.722646 2014-08-13 17:38:02.969341 7fe9bae887a0 20 read_log 192718'9079 (192718'9078) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:88 2014-08-06 15:33:48.745807 2014-08-13 17:38:02.969352 7fe9bae887a0 20 read_log 192718'9080 (192718'9079) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:90 2014-08-06 15:33:48.775073 2014-08-13 17:38:02.969364 7fe9bae887a0 20 read_log 192718'9081 (192718'9080) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:89 2014-08-06 15:33:48.934134 2014-08-13 17:38:02.969376 7fe9bae887a0 20 read_log 192718'9082 (192718'9081) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:91 2014-08-06 15:33:48.960896 2014-08-13 17:38:02.969388 7fe9bae887a0 20 read_log 192718'9083 (192718'9082) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:90 2014-08-06 15:33:48.986938 2014-08-13 17:38:02.969399 7fe9bae887a0 20 read_log 192718'9084 (192718'9083) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:92 2014-08-06 15:33:49.013036 2014-08-13 17:38:02.969411 7fe9bae887a0 20 read_log 192718'9085 (192718'9084) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:91 2014-08-06 15:33:49.039747 2014-08-13 17:38:02.969423 7fe9bae887a0 20 read_log 192718'9086 (192718'9085) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:92 2014-08-06 15:33:49.094268 2014-08-13 17:38:02.969443 7fe9bae887a0 20 read_log 192718'9087 (192718'9086) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:93 2014-08-06 15:33:49.098281 2014-08-13 17:38:02.969456 7fe9bae887a0 20 read_log 192718'9088 (192718'9087) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:94 2014-08-06 15:33:49.146534 2014-08-13 17:38:02.969467 7fe9bae887a0 20 read_log 192718'9089 (192718'9088) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:93 2014-08-06 15:33:49.146130 2014-08-13 17:38:02.969479 7fe9bae887a0 20 read_log 192718'9090 (192718'9089) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:95 2014-08-06 15:33:49.330730 2014-08-13 17:38:02.969491 7fe9bae887a0 20 read_log 192718'9091 (192718'9090) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:94 2014-08-06 15:33:49.330871 2014-08-13 17:38:02.969502 7fe9bae887a0 20 read_log 192718'9092 (192718'9091) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:96 2014-08-06 15:33:49.387500 2014-08-13 17:38:02.969514 7fe9bae887a0 20 read_log 192718'9093 (192718'9092) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:95 2014-08-06 15:33:49.386184 2014-08-13 17:38:02.969526 7fe9bae887a0 20 read_log 192718'9094 (192718'9093) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:97 2014-08-06 15:33:49.419018 2014-08-13 17:38:02.969537 7fe9bae887a0 20 read_log 192718'9095 (192718'9094) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:96 2014-08-06 15:33:49.437329 2014-08-13 17:38:02.969549 7fe9bae887a0 20 read_log 192718'9096 (192718'9095) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:98 2014-08-06 15:33:49.466762 2014-08-13 17:38:02.969561 7fe9bae887a0 20 read_log 192718'9097 (192718'9096) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:97 2014-08-06 15:33:49.490454 2014-08-13 17:38:02.969572 7fe9bae887a0 20 read_log 192718'9098 (192718'9097) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:99 2014-08-06 15:33:49.519624 2014-08-13 17:38:02.969584 7fe9bae887a0 20 read_log 192718'9099 (192718'9098) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:98 2014-08-06 15:33:49.543130 2014-08-13 17:38:02.969596 7fe9bae887a0 20 read_log 192718'9100 (192718'9099) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:100 2014-08-06 15:33:49.575618 2014-08-13 17:38:02.969614 7fe9bae887a0 20 read_log 192718'9101 (192718'9100) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:99 2014-08-06 15:33:49.727968 2014-08-13 17:38:02.969626 7fe9bae887a0 20 read_log 192718'9102 (192718'9101) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:101 2014-08-06 15:33:49.756554 2014-08-13 17:38:02.969638 7fe9bae887a0 20 read_log 192718'9103 (192718'9102) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:100 2014-08-06 15:33:49.782048 2014-08-13 17:38:02.969650 7fe9bae887a0 20 read_log 192718'9104 (192718'9103) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:102 2014-08-06 15:33:49.811017 2014-08-13 17:38:02.969662 7fe9bae887a0 20 read_log 192718'9105 (192718'9104) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:101 2014-08-06 15:33:49.837888 2014-08-13 17:38:02.969675 7fe9bae887a0 20 read_log 192718'9106 (192718'9105) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:103 2014-08-06 15:33:49.865239 2014-08-13 17:38:02.969686 7fe9bae887a0 20 read_log 192718'9107 (192718'9106) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:102 2014-08-06 15:33:49.890008 2014-08-13 17:38:02.969698 7fe9bae887a0 20 read_log 192718'9108 (192718'9107) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:104 2014-08-06 15:33:49.919250 2014-08-13 17:38:02.969709 7fe9bae887a0 20 read_log 192718'9109 (192718'9108) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:103 2014-08-06 15:33:49.942082 2014-08-13 17:38:02.969721 7fe9bae887a0 20 read_log 192718'9110 (192718'9109) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:105 2014-08-06 15:33:49.972422 2014-08-13 17:38:02.969758 7fe9bae887a0 20 read_log 192718'9111 (192718'9110) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:104 2014-08-06 15:33:50.127074 2014-08-13 17:38:02.969771 7fe9bae887a0 20 read_log 192718'9112 (192718'9111) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:106 2014-08-06 15:33:50.156511 2014-08-13 17:38:02.969783 7fe9bae887a0 20 read_log 192718'9113 (192718'9112) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:105 2014-08-06 15:33:50.241121 2014-08-13 17:38:02.969795 7fe9bae887a0 20 read_log 192718'9114 (192718'9113) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:107 2014-08-06 15:33:50.298756 2014-08-13 17:38:02.969807 7fe9bae887a0 20 read_log 192718'9115 (192718'9114) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:106 2014-08-06 15:33:50.330806 2014-08-13 17:38:02.969818 7fe9bae887a0 20 read_log 192718'9116 (192718'9115) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:108 2014-08-06 15:33:50.364831 2014-08-13 17:38:02.969830 7fe9bae887a0 20 read_log 192718'9117 (192718'9116) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:107 2014-08-06 15:33:50.399341 2014-08-13 17:38:02.969842 7fe9bae887a0 20 read_log 192718'9118 (192718'9117) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:109 2014-08-06 15:33:50.435105 2014-08-13 17:38:02.969854 7fe9bae887a0 20 read_log 192718'9119 (192718'9118) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:108 2014-08-06 15:33:50.466830 2014-08-13 17:38:02.969865 7fe9bae887a0 20 read_log 192718'9120 (192718'9119) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:110 2014-08-06 15:33:50.502574 2014-08-13 17:38:02.969877 7fe9bae887a0 20 read_log 192718'9121 (192718'9120) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:109 2014-08-06 15:33:50.757938 2014-08-13 17:38:02.969889 7fe9bae887a0 20 read_log 192718'9122 (192718'9121) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:111 2014-08-06 15:33:50.793010 2014-08-13 17:38:02.969900 7fe9bae887a0 20 read_log 192718'9123 (192718'9122) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:110 2014-08-06 15:33:50.825902 2014-08-13 17:38:02.969912 7fe9bae887a0 20 read_log 192718'9124 (192718'9123) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:112 2014-08-06 15:33:50.860935 2014-08-13 17:38:02.969923 7fe9bae887a0 20 read_log 192718'9125 (192718'9124) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:111 2014-08-06 15:33:50.897174 2014-08-13 17:38:02.969935 7fe9bae887a0 20 read_log 192718'9126 (192718'9125) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:113 2014-08-06 15:33:50.939688 2014-08-13 17:38:02.969947 7fe9bae887a0 20 read_log 192718'9127 (192718'9126) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:112 2014-08-06 15:33:50.972702 2014-08-13 17:38:02.969958 7fe9bae887a0 20 read_log 192718'9128 (192718'9127) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:114 2014-08-06 15:33:51.008366 2014-08-13 17:38:02.969970 7fe9bae887a0 20 read_log 192718'9129 (192718'9128) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:113 2014-08-06 15:33:51.040998 2014-08-13 17:38:02.969982 7fe9bae887a0 20 read_log 192718'9130 (192718'9129) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:115 2014-08-06 15:33:51.085477 2014-08-13 17:38:02.969993 7fe9bae887a0 20 read_log 192718'9131 (192718'9130) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:114 2014-08-06 15:33:51.164684 2014-08-13 17:38:02.970005 7fe9bae887a0 20 read_log 192718'9132 (192718'9131) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:116 2014-08-06 15:33:51.324742 2014-08-13 17:38:02.970017 7fe9bae887a0 20 read_log 192718'9133 (192718'9132) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:115 2014-08-06 15:33:51.361592 2014-08-13 17:38:02.970029 7fe9bae887a0 20 read_log 192718'9134 (192718'9133) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:117 2014-08-06 15:33:51.484211 2014-08-13 17:38:02.970047 7fe9bae887a0 20 read_log 192718'9135 (192718'9134) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:116 2014-08-06 15:33:51.519890 2014-08-13 17:38:02.970059 7fe9bae887a0 20 read_log 192718'9136 (192718'9135) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:118 2014-08-06 15:33:51.562528 2014-08-13 17:38:02.970071 7fe9bae887a0 20 read_log 192718'9137 (192718'9136) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:117 2014-08-06 15:33:51.599588 2014-08-13 17:38:02.970083 7fe9bae887a0 20 read_log 192718'9138 (192718'9137) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:119 2014-08-06 15:33:51.638092 2014-08-13 17:38:02.970094 7fe9bae887a0 20 read_log 192718'9139 (192718'9138) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:118 2014-08-06 15:33:51.674076 2014-08-13 17:38:02.970117 7fe9bae887a0 20 read_log 192718'9140 (192718'9139) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:120 2014-08-06 15:33:51.712797 2014-08-13 17:38:02.970129 7fe9bae887a0 20 read_log 192718'9141 (192718'9140) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:119 2014-08-06 15:33:51.746234 2014-08-13 17:38:02.970141 7fe9bae887a0 20 read_log 192718'9142 (192718'9141) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:121 2014-08-06 15:33:51.784349 2014-08-13 17:38:02.970152 7fe9bae887a0 20 read_log 192718'9143 (192718'9142) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:120 2014-08-06 15:33:51.824167 2014-08-13 17:38:02.970170 7fe9bae887a0 20 read_log 192718'9144 (192718'9143) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:122 2014-08-06 15:33:51.867246 2014-08-13 17:38:02.970182 7fe9bae887a0 20 read_log 192718'9145 (192718'9144) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:121 2014-08-06 15:33:51.903714 2014-08-13 17:38:02.970194 7fe9bae887a0 20 read_log 192718'9146 (192718'9145) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:123 2014-08-06 15:33:51.942229 2014-08-13 17:38:02.970206 7fe9bae887a0 20 read_log 192718'9147 (192718'9146) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:122 2014-08-06 15:33:52.203799 2014-08-13 17:38:02.970218 7fe9bae887a0 20 read_log 192718'9148 (192718'9147) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:124 2014-08-06 15:33:52.241584 2014-08-13 17:38:02.970230 7fe9bae887a0 20 read_log 192718'9149 (192718'9148) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:123 2014-08-06 15:33:52.277068 2014-08-13 17:38:02.970243 7fe9bae887a0 20 read_log 192718'9150 (192718'9149) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:125 2014-08-06 15:33:52.317953 2014-08-13 17:38:02.970255 7fe9bae887a0 20 read_log 192718'9151 (192718'9150) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:124 2014-08-06 15:33:52.353796 2014-08-13 17:38:02.970266 7fe9bae887a0 20 read_log 192718'9152 (192718'9151) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:126 2014-08-06 15:33:52.393481 2014-08-13 17:38:02.970278 7fe9bae887a0 20 read_log 192718'9153 (192718'9152) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:125 2014-08-06 15:33:52.429297 2014-08-13 17:38:02.970290 7fe9bae887a0 20 read_log 192718'9154 (192718'9153) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:127 2014-08-06 15:33:52.477491 2014-08-13 17:38:02.970301 7fe9bae887a0 20 read_log 192718'9155 (192718'9154) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:126 2014-08-06 15:33:52.504186 2014-08-13 17:38:02.970313 7fe9bae887a0 20 read_log 192718'9156 (192718'9155) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:128 2014-08-06 15:33:52.543157 2014-08-13 17:38:02.970325 7fe9bae887a0 20 read_log 192718'9157 (192718'9156) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:127 2014-08-06 15:33:52.763883 2014-08-13 17:38:02.970336 7fe9bae887a0 20 read_log 192718'9158 (192718'9157) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:129 2014-08-06 15:33:52.810650 2014-08-13 17:38:02.970357 7fe9bae887a0 20 read_log 192718'9159 (192718'9158) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:128 2014-08-06 15:33:52.846641 2014-08-13 17:38:02.970370 7fe9bae887a0 20 read_log 192718'9160 (192718'9159) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:130 2014-08-06 15:33:52.885142 2014-08-13 17:38:02.970382 7fe9bae887a0 20 read_log 192718'9161 (192718'9160) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:129 2014-08-06 15:33:52.921558 2014-08-13 17:38:02.970394 7fe9bae887a0 20 read_log 192718'9162 (192718'9161) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:131 2014-08-06 15:33:52.959276 2014-08-13 17:38:02.970406 7fe9bae887a0 20 read_log 192718'9163 (192718'9162) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:130 2014-08-06 15:33:52.991247 2014-08-13 17:38:02.970432 7fe9bae887a0 20 read_log 192718'9164 (192718'9163) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:132 2014-08-06 15:33:53.026127 2014-08-13 17:38:02.970444 7fe9bae887a0 20 read_log 192718'9165 (192718'9164) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:131 2014-08-06 15:33:53.058356 2014-08-13 17:38:02.970456 7fe9bae887a0 20 read_log 192718'9166 (192718'9165) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:133 2014-08-06 15:33:53.093606 2014-08-13 17:38:02.970468 7fe9bae887a0 20 read_log 192718'9167 (192718'9166) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:132 2014-08-06 15:33:53.291865 2014-08-13 17:38:02.970480 7fe9bae887a0 20 read_log 192718'9168 (192718'9167) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:134 2014-08-06 15:33:53.324328 2014-08-13 17:38:02.970491 7fe9bae887a0 20 read_log 192718'9169 (192718'9168) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:133 2014-08-06 15:33:53.359442 2014-08-13 17:38:02.970504 7fe9bae887a0 20 read_log 192718'9170 (192718'9169) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:135 2014-08-06 15:33:53.396204 2014-08-13 17:38:02.970517 7fe9bae887a0 20 read_log 192718'9171 (192718'9170) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:134 2014-08-06 15:33:53.427731 2014-08-13 17:38:02.970528 7fe9bae887a0 20 read_log 192718'9172 (192718'9171) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:136 2014-08-06 15:33:53.462780 2014-08-13 17:38:02.970540 7fe9bae887a0 20 read_log 192718'9173 (192718'9172) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:135 2014-08-06 15:33:53.494577 2014-08-13 17:38:02.970551 7fe9bae887a0 20 read_log 192718'9174 (192718'9173) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:137 2014-08-06 15:33:53.530318 2014-08-13 17:38:02.970563 7fe9bae887a0 20 read_log 192718'9175 (192718'9174) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:136 2014-08-06 15:33:53.561819 2014-08-13 17:38:02.970575 7fe9bae887a0 20 read_log 192718'9176 (192718'9175) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:138 2014-08-06 15:33:53.600514 2014-08-13 17:38:02.970586 7fe9bae887a0 20 read_log 192718'9177 (192718'9176) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:137 2014-08-06 15:33:53.797131 2014-08-13 17:38:02.970598 7fe9bae887a0 20 read_log 192718'9178 (192718'9177) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:139 2014-08-06 15:33:53.835838 2014-08-13 17:38:02.970609 7fe9bae887a0 20 read_log 192718'9179 (192718'9178) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:138 2014-08-06 15:33:53.876048 2014-08-13 17:38:02.970621 7fe9bae887a0 20 read_log 192718'9180 (192718'9179) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:140 2014-08-06 15:33:53.910714 2014-08-13 17:38:02.970633 7fe9bae887a0 20 read_log 192718'9181 (192718'9180) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:139 2014-08-06 15:33:53.943270 2014-08-13 17:38:02.970644 7fe9bae887a0 20 read_log 192718'9182 (192718'9181) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:141 2014-08-06 15:33:53.978042 2014-08-13 17:38:02.970663 7fe9bae887a0 20 read_log 192718'9183 (192718'9182) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:140 2014-08-06 15:33:54.010280 2014-08-13 17:38:02.970676 7fe9bae887a0 20 read_log 192718'9184 (192718'9183) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:142 2014-08-06 15:33:54.045371 2014-08-13 17:38:02.970688 7fe9bae887a0 20 read_log 192718'9185 (192718'9184) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:141 2014-08-06 15:33:54.077409 2014-08-13 17:38:02.970699 7fe9bae887a0 20 read_log 192718'9186 (192718'9185) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:143 2014-08-06 15:33:54.112113 2014-08-13 17:38:02.970711 7fe9bae887a0 20 read_log 192718'9187 (192718'9186) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:142 2014-08-06 15:33:54.310712 2014-08-13 17:38:02.970723 7fe9bae887a0 20 read_log 192718'9188 (192718'9187) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:144 2014-08-06 15:33:54.351214 2014-08-13 17:38:02.970752 7fe9bae887a0 20 read_log 192718'9189 (192718'9188) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:143 2014-08-06 15:33:54.383298 2014-08-13 17:38:02.970765 7fe9bae887a0 20 read_log 192718'9190 (192718'9189) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:145 2014-08-06 15:33:54.418203 2014-08-13 17:38:02.970776 7fe9bae887a0 20 read_log 192718'9191 (192718'9190) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:144 2014-08-06 15:33:54.464300 2014-08-13 17:38:02.970788 7fe9bae887a0 20 read_log 192718'9192 (192718'9191) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:146 2014-08-06 15:33:54.491854 2014-08-13 17:38:02.970800 7fe9bae887a0 20 read_log 192718'9193 (192718'9192) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:145 2014-08-06 15:33:54.518304 2014-08-13 17:38:02.970811 7fe9bae887a0 20 read_log 192718'9194 (192718'9193) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:147 2014-08-06 15:33:54.545591 2014-08-13 17:38:02.970823 7fe9bae887a0 20 read_log 192718'9195 (192718'9194) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:146 2014-08-06 15:33:54.572640 2014-08-13 17:38:02.970835 7fe9bae887a0 20 read_log 192718'9196 (192718'9195) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:148 2014-08-06 15:33:54.601259 2014-08-13 17:38:02.970847 7fe9bae887a0 20 read_log 192718'9197 (192718'9196) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:147 2014-08-06 15:33:54.759197 2014-08-13 17:38:02.970858 7fe9bae887a0 20 read_log 192718'9198 (192718'9197) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:149 2014-08-06 15:33:54.788460 2014-08-13 17:38:02.970870 7fe9bae887a0 20 read_log 192718'9199 (192718'9198) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:148 2014-08-06 15:33:54.817174 2014-08-13 17:38:02.970882 7fe9bae887a0 20 read_log 192718'9200 (192718'9199) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:150 2014-08-06 15:33:54.855905 2014-08-13 17:38:02.970893 7fe9bae887a0 20 read_log 192718'9201 (192718'9200) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:149 2014-08-06 15:33:54.871692 2014-08-13 17:38:02.970905 7fe9bae887a0 20 read_log 192718'9202 (192718'9201) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:151 2014-08-06 15:33:54.921069 2014-08-13 17:38:02.970916 7fe9bae887a0 20 read_log 192718'9203 (192718'9202) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:150 2014-08-06 15:33:54.925942 2014-08-13 17:38:02.970928 7fe9bae887a0 20 read_log 192718'9204 (192718'9203) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:151 2014-08-06 15:33:54.981492 2014-08-13 17:38:02.970939 7fe9bae887a0 20 read_log 192718'9205 (192718'9204) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:152 2014-08-06 15:33:55.004717 2014-08-13 17:38:02.970951 7fe9bae887a0 20 read_log 192718'9206 (192718'9205) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:152 2014-08-06 15:33:55.143600 2014-08-13 17:38:02.970969 7fe9bae887a0 20 read_log 192718'9207 (192718'9206) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:153 2014-08-06 15:33:55.202419 2014-08-13 17:38:02.970982 7fe9bae887a0 20 read_log 192718'9208 (192718'9207) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:153 2014-08-06 15:33:55.214821 2014-08-13 17:38:02.970995 7fe9bae887a0 20 read_log 192718'9209 (192718'9208) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:154 2014-08-06 15:33:55.255712 2014-08-13 17:38:02.971009 7fe9bae887a0 20 read_log 192718'9210 (192718'9209) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:154 2014-08-06 15:33:55.255496 2014-08-13 17:38:02.971020 7fe9bae887a0 20 read_log 192718'9211 (192718'9210) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:155 2014-08-06 15:33:55.336803 2014-08-13 17:38:02.971032 7fe9bae887a0 20 read_log 192718'9212 (192718'9211) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:155 2014-08-06 15:33:55.336510 2014-08-13 17:38:02.971044 7fe9bae887a0 20 read_log 192718'9214 (192718'9212) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:156 2014-08-06 15:33:57.971693 2014-08-13 17:38:02.971055 7fe9bae887a0 20 read_log 192718'9215 (192718'9214) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:156 2014-08-06 15:33:57.971595 2014-08-13 17:38:02.971067 7fe9bae887a0 20 read_log 192718'9216 (192718'9215) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:157 2014-08-06 15:33:58.026236 2014-08-13 17:38:02.971079 7fe9bae887a0 20 read_log 192718'9217 (192718'9216) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:157 2014-08-06 15:33:58.077564 2014-08-13 17:38:02.971090 7fe9bae887a0 20 read_log 192718'9218 (192718'9217) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:158 2014-08-06 15:33:58.113666 2014-08-13 17:38:02.971102 7fe9bae887a0 20 read_log 192718'9219 (192718'9218) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:158 2014-08-06 15:33:58.113617 2014-08-13 17:38:02.971114 7fe9bae887a0 20 read_log 192718'9220 (192718'9219) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:159 2014-08-06 15:33:58.240974 2014-08-13 17:38:02.971125 7fe9bae887a0 20 read_log 192718'9221 (192718'9220) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:159 2014-08-06 15:33:58.241693 2014-08-13 17:38:02.971137 7fe9bae887a0 20 read_log 192718'9222 (192718'9221) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:160 2014-08-06 15:33:58.304257 2014-08-13 17:38:02.971149 7fe9bae887a0 20 read_log 192718'9223 (192718'9222) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:160 2014-08-06 15:33:58.304325 2014-08-13 17:38:02.971173 7fe9bae887a0 20 read_log 192718'9224 (192718'9223) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:161 2014-08-06 15:33:58.492123 2014-08-13 17:38:02.971185 7fe9bae887a0 20 read_log 192718'9225 (192718'9224) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:161 2014-08-06 15:33:58.492161 2014-08-13 17:38:02.971197 7fe9bae887a0 20 read_log 192718'9226 (192718'9225) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:162 2014-08-06 15:33:58.545954 2014-08-13 17:38:02.971209 7fe9bae887a0 20 read_log 192718'9227 (192718'9226) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:162 2014-08-06 15:33:58.545810 2014-08-13 17:38:02.971221 7fe9bae887a0 20 read_log 192718'9228 (192718'9227) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:163 2014-08-06 15:33:58.608016 2014-08-13 17:38:02.971232 7fe9bae887a0 20 read_log 192718'9229 (192718'9228) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:163 2014-08-06 15:33:58.608485 2014-08-13 17:38:02.971244 7fe9bae887a0 20 read_log 192718'9230 (192718'9229) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:164 2014-08-06 15:33:58.737578 2014-08-13 17:38:02.971256 7fe9bae887a0 20 read_log 192718'9231 (192718'9230) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:164 2014-08-06 15:33:58.737576 2014-08-13 17:38:02.971274 7fe9bae887a0 20 read_log 192718'9232 (192718'9231) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:165 2014-08-06 15:33:58.891433 2014-08-13 17:38:02.971286 7fe9bae887a0 20 read_log 192718'9233 (192718'9232) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:165 2014-08-06 15:33:58.891276 2014-08-13 17:38:02.971298 7fe9bae887a0 20 read_log 192718'9234 (192718'9233) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:166 2014-08-06 15:33:58.953346 2014-08-13 17:38:02.971309 7fe9bae887a0 20 read_log 192718'9235 (192718'9234) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:166 2014-08-06 15:33:58.953558 2014-08-13 17:38:02.971321 7fe9bae887a0 20 read_log 192718'9236 (192718'9235) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:167 2014-08-06 15:33:59.451856 2014-08-13 17:38:02.971333 7fe9bae887a0 20 read_log 192718'9237 (192718'9236) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:167 2014-08-06 15:33:59.451713 2014-08-13 17:38:02.971347 7fe9bae887a0 20 read_log 192718'9238 (192718'9237) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:168 2014-08-06 15:33:59.485057 2014-08-13 17:38:02.971359 7fe9bae887a0 20 read_log 192718'9239 (192718'9238) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:168 2014-08-06 15:33:59.483377 2014-08-13 17:38:02.971371 7fe9bae887a0 20 read_log 192718'9240 (192718'9239) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:169 2014-08-06 15:33:59.514184 2014-08-13 17:38:02.971383 7fe9bae887a0 20 read_log 192718'9241 (192718'9240) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:169 2014-08-06 15:33:59.592692 2014-08-13 17:38:02.971395 7fe9bae887a0 20 read_log 192718'9242 (192718'9241) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:170 2014-08-06 15:33:59.635541 2014-08-13 17:38:02.971406 7fe9bae887a0 20 read_log 192718'9243 (192718'9242) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:170 2014-08-06 15:33:59.635613 2014-08-13 17:38:02.971418 7fe9bae887a0 20 read_log 192718'9244 (192718'9243) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:171 2014-08-06 15:33:59.779157 2014-08-13 17:38:02.971430 7fe9bae887a0 20 read_log 192718'9245 (192718'9244) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:171 2014-08-06 15:33:59.779241 2014-08-13 17:38:02.971441 7fe9bae887a0 20 read_log 192718'9246 (192718'9245) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:172 2014-08-06 15:33:59.833125 2014-08-13 17:38:02.971453 7fe9bae887a0 20 read_log 192718'9247 (192718'9246) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:172 2014-08-06 15:33:59.833053 2014-08-13 17:38:02.971465 7fe9bae887a0 20 read_log 192718'9248 (192718'9247) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:173 2014-08-06 15:33:59.899740 2014-08-13 17:38:02.971476 7fe9bae887a0 20 read_log 192718'9249 (192718'9248) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:173 2014-08-06 15:33:59.900223 2014-08-13 17:38:02.971488 7fe9bae887a0 20 read_log 192718'9250 (192718'9249) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:174 2014-08-06 15:34:00.033148 2014-08-13 17:38:02.971499 7fe9bae887a0 20 read_log 192718'9251 (192718'9250) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:174 2014-08-06 15:34:00.035125 2014-08-13 17:38:02.971511 7fe9bae887a0 20 read_log 192718'9252 (192718'9251) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:175 2014-08-06 15:34:00.147843 2014-08-13 17:38:02.971523 7fe9bae887a0 20 read_log 192718'9253 (192718'9252) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:175 2014-08-06 15:34:00.147811 2014-08-13 17:38:02.971534 7fe9bae887a0 20 read_log 192718'9254 (192718'9253) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:176 2014-08-06 15:34:00.203091 2014-08-13 17:38:02.971546 7fe9bae887a0 20 read_log 192718'9255 (192718'9254) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:176 2014-08-06 15:34:00.203010 2014-08-13 17:38:02.971566 7fe9bae887a0 20 read_log 192718'9256 (192718'9255) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:177 2014-08-06 15:34:00.279904 2014-08-13 17:38:02.971579 7fe9bae887a0 20 read_log 192718'9257 (192718'9256) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:177 2014-08-06 15:34:00.279959 2014-08-13 17:38:02.971590 7fe9bae887a0 20 read_log 192718'9258 (192718'9257) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:178 2014-08-06 15:34:00.412395 2014-08-13 17:38:02.971602 7fe9bae887a0 20 read_log 192718'9259 (192718'9258) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:178 2014-08-06 15:34:00.412327 2014-08-13 17:38:02.971614 7fe9bae887a0 20 read_log 192718'9260 (192718'9259) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:179 2014-08-06 15:34:00.448703 2014-08-13 17:38:02.971626 7fe9bae887a0 20 read_log 192718'9261 (192718'9260) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:179 2014-08-06 15:34:00.448684 2014-08-13 17:38:02.971637 7fe9bae887a0 20 read_log 192718'9262 (192718'9261) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:180 2014-08-06 15:34:00.524352 2014-08-13 17:38:02.971649 7fe9bae887a0 20 read_log 192718'9263 (192718'9262) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:180 2014-08-06 15:34:00.525559 2014-08-13 17:38:02.971661 7fe9bae887a0 20 read_log 192718'9264 (192718'9263) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:181 2014-08-06 15:34:00.555553 2014-08-13 17:38:02.971673 7fe9bae887a0 20 read_log 192718'9265 (192718'9264) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:181 2014-08-06 15:34:00.558518 2014-08-13 17:38:02.971684 7fe9bae887a0 20 read_log 192718'9266 (192718'9265) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:182 2014-08-06 15:34:00.586530 2014-08-13 17:38:02.971696 7fe9bae887a0 20 read_log 192718'9267 (192718'9266) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:182 2014-08-06 15:34:00.709623 2014-08-13 17:38:02.971707 7fe9bae887a0 20 read_log 192718'9268 (192718'9267) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:183 2014-08-06 15:34:00.724023 2014-08-13 17:38:02.971719 7fe9bae887a0 20 read_log 192718'9269 (192718'9268) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:183 2014-08-06 15:34:00.738272 2014-08-13 17:38:02.971731 7fe9bae887a0 20 read_log 192718'9270 (192718'9269) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:184 2014-08-06 15:34:00.760095 2014-08-13 17:38:02.971742 7fe9bae887a0 20 read_log 192718'9271 (192718'9270) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:184 2014-08-06 15:34:00.759493 2014-08-13 17:38:02.971754 7fe9bae887a0 20 read_log 192718'9272 (192718'9271) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:185 2014-08-06 15:34:00.877716 2014-08-13 17:38:02.971766 7fe9bae887a0 20 read_log 192718'9273 (192718'9272) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:185 2014-08-06 15:34:00.881886 2014-08-13 17:38:02.971778 7fe9bae887a0 20 read_log 192718'9274 (192718'9273) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:186 2014-08-06 15:34:00.908512 2014-08-13 17:38:02.971789 7fe9bae887a0 20 read_log 192718'9275 (192718'9274) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:186 2014-08-06 15:34:00.911689 2014-08-13 17:38:02.971801 7fe9bae887a0 20 read_log 192718'9276 (192718'9275) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:187 2014-08-06 15:34:00.938439 2014-08-13 17:38:02.971813 7fe9bae887a0 20 read_log 192718'9277 (192718'9276) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:187 2014-08-06 15:34:00.942208 2014-08-13 17:38:02.971824 7fe9bae887a0 20 read_log 192718'9278 (192718'9277) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:188 2014-08-06 15:34:00.966178 2014-08-13 17:38:02.971836 7fe9bae887a0 20 read_log 192718'9279 (192718'9278) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:188 2014-08-06 15:34:00.969434 2014-08-13 17:38:02.971872 7fe9bae887a0 20 read_log 192718'9280 (192718'9279) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:189 2014-08-06 15:34:01.064579 2014-08-13 17:38:02.971885 7fe9bae887a0 20 read_log 192718'9281 (192718'9280) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:189 2014-08-06 15:34:01.065508 2014-08-13 17:38:02.971897 7fe9bae887a0 20 read_log 192718'9282 (192718'9281) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:190 2014-08-06 15:34:01.094522 2014-08-13 17:38:02.971924 7fe9bae887a0 20 read_log 192718'9283 (192718'9282) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:190 2014-08-06 15:34:01.106804 2014-08-13 17:38:02.971937 7fe9bae887a0 20 read_log 192718'9284 (192718'9283) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:191 2014-08-06 15:34:01.122505 2014-08-13 17:38:02.971949 7fe9bae887a0 20 read_log 192718'9285 (192718'9284) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:191 2014-08-06 15:34:01.132211 2014-08-13 17:38:02.971960 7fe9bae887a0 20 read_log 192718'9286 (192718'9285) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:192 2014-08-06 15:34:01.188180 2014-08-13 17:38:02.971972 7fe9bae887a0 20 read_log 192718'9287 (192718'9286) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:192 2014-08-06 15:34:01.242561 2014-08-13 17:38:02.971984 7fe9bae887a0 20 read_log 192718'9288 (192718'9287) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:193 2014-08-06 15:34:01.334920 2014-08-13 17:38:02.971996 7fe9bae887a0 20 read_log 192718'9289 (192718'9288) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:193 2014-08-06 15:34:01.486341 2014-08-13 17:38:02.972008 7fe9bae887a0 20 read_log 192718'9290 (192718'9289) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:194 2014-08-06 15:34:01.655882 2014-08-13 17:38:02.972020 7fe9bae887a0 20 read_log 192718'9291 (192718'9290) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:194 2014-08-06 15:34:01.747673 2014-08-13 17:38:02.972032 7fe9bae887a0 20 read_log 192718'9292 (192718'9291) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:195 2014-08-06 15:34:02.139908 2014-08-13 17:38:02.972044 7fe9bae887a0 20 read_log 192718'9293 (192718'9292) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:195 2014-08-06 15:34:02.141621 2014-08-13 17:38:02.972058 7fe9bae887a0 20 read_log 192718'9294 (192718'9293) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:196 2014-08-06 15:34:03.056050 2014-08-13 17:38:02.972070 7fe9bae887a0 20 read_log 192718'9295 (192718'9294) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:196 2014-08-06 15:34:03.056369 2014-08-13 17:38:02.972082 7fe9bae887a0 20 read_log 192718'9296 (192718'9295) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:197 2014-08-06 15:34:03.089419 2014-08-13 17:38:02.972094 7fe9bae887a0 20 read_log 192718'9297 (192718'9296) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:197 2014-08-06 15:34:03.089741 2014-08-13 17:38:02.972106 7fe9bae887a0 20 read_log 192718'9298 (192718'9297) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:198 2014-08-06 15:34:03.229921 2014-08-13 17:38:02.972117 7fe9bae887a0 20 read_log 192718'9299 (192718'9298) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:198 2014-08-06 15:34:03.304814 2014-08-13 17:38:02.972129 7fe9bae887a0 20 read_log 192718'9300 (192718'9299) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:199 2014-08-06 15:34:03.461725 2014-08-13 17:38:02.972142 7fe9bae887a0 20 read_log 192718'9301 (192718'9300) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:199 2014-08-06 15:34:03.679729 2014-08-13 17:38:02.972159 7fe9bae887a0 20 read_log 192718'9302 (192718'9301) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:200 2014-08-06 15:34:03.721100 2014-08-13 17:38:02.972171 7fe9bae887a0 20 read_log 192718'9303 (192718'9302) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:200 2014-08-06 15:34:03.906722 2014-08-13 17:38:02.972190 7fe9bae887a0 20 read_log 192718'9304 (192718'9303) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:201 2014-08-06 15:34:04.077007 2014-08-13 17:38:02.972203 7fe9bae887a0 20 read_log 192718'9305 (192718'9304) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:201 2014-08-06 15:34:04.157280 2014-08-13 17:38:02.972214 7fe9bae887a0 20 read_log 192718'9306 (192718'9305) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:202 2014-08-06 15:34:04.425285 2014-08-13 17:38:02.972226 7fe9bae887a0 20 read_log 192718'9307 (192718'9306) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:202 2014-08-06 15:34:04.549345 2014-08-13 17:38:02.972238 7fe9bae887a0 20 read_log 192718'9308 (192718'9307) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:203 2014-08-06 15:34:04.552950 2014-08-13 17:38:02.972249 7fe9bae887a0 20 read_log 192718'9309 (192718'9308) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:203 2014-08-06 15:34:04.941910 2014-08-13 17:38:02.972261 7fe9bae887a0 20 read_log 192718'9310 (192718'9309) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:204 2014-08-06 15:34:04.964242 2014-08-13 17:38:02.972272 7fe9bae887a0 20 read_log 192718'9311 (192718'9310) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:204 2014-08-06 15:34:05.100719 2014-08-13 17:38:02.972284 7fe9bae887a0 20 read_log 192718'9312 (192718'9311) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:205 2014-08-06 15:34:05.161560 2014-08-13 17:38:02.972296 7fe9bae887a0 20 read_log 192718'9313 (192718'9312) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:205 2014-08-06 15:34:05.329177 2014-08-13 17:38:02.972308 7fe9bae887a0 20 read_log 192718'9314 (192718'9313) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:206 2014-08-06 15:34:05.359101 2014-08-13 17:38:02.972319 7fe9bae887a0 20 read_log 192718'9315 (192718'9314) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:206 2014-08-06 15:34:05.784527 2014-08-13 17:38:02.972331 7fe9bae887a0 20 read_log 192718'9316 (192718'9315) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:207 2014-08-06 15:34:05.829646 2014-08-13 17:38:02.972342 7fe9bae887a0 20 read_log 192718'9317 (192718'9316) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:207 2014-08-06 15:34:06.162798 2014-08-13 17:38:02.972354 7fe9bae887a0 20 read_log 192718'9318 (192718'9317) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:208 2014-08-06 15:34:06.838592 2014-08-13 17:38:02.972365 7fe9bae887a0 20 read_log 192718'9319 (192718'9318) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:208 2014-08-06 15:34:06.836982 2014-08-13 17:38:02.972377 7fe9bae887a0 20 read_log 192718'9320 (192718'9319) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:209 2014-08-06 15:34:07.098794 2014-08-13 17:38:02.972388 7fe9bae887a0 20 read_log 192718'9321 (192718'9320) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:209 2014-08-06 15:34:07.293837 2014-08-13 17:38:02.972400 7fe9bae887a0 20 read_log 192718'9322 (192718'9321) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:210 2014-08-06 15:34:07.297198 2014-08-13 17:38:02.972414 7fe9bae887a0 20 read_log 192718'9323 (192718'9322) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:211 2014-08-06 15:34:08.854318 2014-08-13 17:38:02.972426 7fe9bae887a0 20 read_log 192718'9324 (192718'9323) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:210 2014-08-06 15:34:08.851889 2014-08-13 17:38:02.972437 7fe9bae887a0 20 read_log 192718'9325 (192718'9324) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:212 2014-08-06 15:34:09.210699 2014-08-13 17:38:02.972449 7fe9bae887a0 20 read_log 192718'9326 (192718'9325) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:211 2014-08-06 15:34:09.368644 2014-08-13 17:38:02.972460 7fe9bae887a0 20 read_log 192718'9327 (192718'9326) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:213 2014-08-06 15:34:09.370781 2014-08-13 17:38:02.972481 7fe9bae887a0 20 read_log 192718'9328 (192718'9327) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:212 2014-08-06 15:34:09.869788 2014-08-13 17:38:02.972510 7fe9bae887a0 20 read_log 192718'9329 (192718'9328) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:214 2014-08-06 15:34:09.938053 2014-08-13 17:38:02.972523 7fe9bae887a0 20 read_log 192718'9330 (192718'9329) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:213 2014-08-06 15:34:10.066025 2014-08-13 17:38:02.972535 7fe9bae887a0 20 read_log 192718'9331 (192718'9330) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:215 2014-08-06 15:34:10.085322 2014-08-13 17:38:02.972547 7fe9bae887a0 20 read_log 192718'9332 (192718'9331) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:214 2014-08-06 15:34:10.605639 2014-08-13 17:38:02.972559 7fe9bae887a0 20 read_log 192718'9333 (192718'9332) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:216 2014-08-06 15:34:10.606044 2014-08-13 17:38:02.972570 7fe9bae887a0 20 read_log 192718'9334 (192718'9333) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:215 2014-08-06 15:34:10.859213 2014-08-13 17:38:02.972582 7fe9bae887a0 20 read_log 192718'9335 (192718'9334) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:217 2014-08-06 15:34:10.859199 2014-08-13 17:38:02.972594 7fe9bae887a0 20 read_log 192718'9336 (192718'9335) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:216 2014-08-06 15:34:11.438948 2014-08-13 17:38:02.972605 7fe9bae887a0 20 read_log 192718'9337 (192718'9336) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:218 2014-08-06 15:34:11.458663 2014-08-13 17:38:02.972617 7fe9bae887a0 20 read_log 192718'9338 (192718'9337) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:217 2014-08-06 15:34:11.723917 2014-08-13 17:38:02.972628 7fe9bae887a0 20 read_log 192718'9339 (192718'9338) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:219 2014-08-06 15:34:11.860683 2014-08-13 17:38:02.972640 7fe9bae887a0 20 read_log 192718'9340 (192718'9339) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:218 2014-08-06 15:34:12.103531 2014-08-13 17:38:02.972652 7fe9bae887a0 20 read_log 192718'9341 (192718'9340) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:219 2014-08-06 15:34:12.336299 2014-08-13 17:38:02.972664 7fe9bae887a0 20 read_log 192718'9342 (192718'9341) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:220 2014-08-06 15:34:12.338248 2014-08-13 17:38:02.972676 7fe9bae887a0 20 read_log 192718'9343 (192718'9342) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:220 2014-08-06 15:34:12.705435 2014-08-13 17:38:02.972688 7fe9bae887a0 20 read_log 192718'9344 (192718'9343) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:221 2014-08-06 15:34:12.730209 2014-08-13 17:38:02.972700 7fe9bae887a0 20 read_log 192718'9345 (192718'9344) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:221 2014-08-06 15:34:12.803448 2014-08-13 17:38:02.972712 7fe9bae887a0 20 read_log 192718'9346 (192718'9345) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:222 2014-08-06 15:34:13.355028 2014-08-13 17:38:02.972724 7fe9bae887a0 20 read_log 192718'9347 (192718'9346) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:222 2014-08-06 15:34:13.375514 2014-08-13 17:38:02.972735 7fe9bae887a0 20 read_log 192718'9348 (192718'9347) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:223 2014-08-06 15:34:13.559349 2014-08-13 17:38:02.972747 7fe9bae887a0 20 read_log 192718'9349 (192718'9348) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:223 2014-08-06 15:34:14.537320 2014-08-13 17:38:02.972759 7fe9bae887a0 20 read_log 192718'9350 (192718'9349) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:224 2014-08-06 15:34:15.022606 2014-08-13 17:38:02.972771 7fe9bae887a0 20 read_log 192718'9351 (192718'9350) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:224 2014-08-06 15:34:15.022523 2014-08-13 17:38:02.972789 7fe9bae887a0 20 read_log 192718'9352 (192718'9351) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:225 2014-08-06 15:34:15.167395 2014-08-13 17:38:02.972803 7fe9bae887a0 20 read_log 192718'9353 (192718'9352) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:225 2014-08-06 15:34:15.167332 2014-08-13 17:38:02.972815 7fe9bae887a0 20 read_log 192718'9354 (192718'9353) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:226 2014-08-06 15:34:15.411682 2014-08-13 17:38:02.972827 7fe9bae887a0 20 read_log 192718'9355 (192718'9354) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:226 2014-08-06 15:34:15.563129 2014-08-13 17:38:02.972838 7fe9bae887a0 20 read_log 192718'9356 (192718'9355) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:227 2014-08-06 15:34:15.981223 2014-08-13 17:38:02.972850 7fe9bae887a0 20 read_log 192718'9357 (192718'9356) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:227 2014-08-06 15:34:16.035424 2014-08-13 17:38:02.972862 7fe9bae887a0 20 read_log 192718'9358 (192718'9357) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:228 2014-08-06 15:34:16.330017 2014-08-13 17:38:02.972873 7fe9bae887a0 20 read_log 192718'9359 (192718'9358) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:228 2014-08-06 15:34:16.329990 2014-08-13 17:38:02.972885 7fe9bae887a0 20 read_log 192718'9360 (192718'9359) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:229 2014-08-06 15:34:16.812061 2014-08-13 17:38:02.972897 7fe9bae887a0 20 read_log 192718'9361 (192718'9360) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:229 2014-08-06 15:34:16.812154 2014-08-13 17:38:02.972909 7fe9bae887a0 20 read_log 192718'9362 (192718'9361) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:230 2014-08-06 15:34:17.483789 2014-08-13 17:38:02.972921 7fe9bae887a0 20 read_log 192718'9363 (192718'9362) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:230 2014-08-06 15:34:17.667190 2014-08-13 17:38:02.972933 7fe9bae887a0 20 read_log 192718'9364 (192718'9363) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:231 2014-08-06 15:34:19.824887 2014-08-13 17:38:02.972944 7fe9bae887a0 20 read_log 192718'9365 (192718'9364) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:231 2014-08-06 15:34:20.308524 2014-08-13 17:38:02.972956 7fe9bae887a0 20 read_log 192718'9366 (192718'9365) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:232 2014-08-06 15:34:20.320008 2014-08-13 17:38:02.972967 7fe9bae887a0 20 read_log 192718'9367 (192718'9366) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:232 2014-08-06 15:34:20.348612 2014-08-13 17:38:02.972979 7fe9bae887a0 20 read_log 192718'9368 (192718'9367) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:233 2014-08-06 15:34:20.669785 2014-08-13 17:38:02.972991 7fe9bae887a0 20 read_log 192718'9369 (192718'9368) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:233 2014-08-06 15:34:20.671873 2014-08-13 17:38:02.973002 7fe9bae887a0 20 read_log 192718'9370 (192718'9369) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:234 2014-08-06 15:34:21.194975 2014-08-13 17:38:02.973014 7fe9bae887a0 20 read_log 192718'9371 (192718'9370) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:234 2014-08-06 15:34:21.440500 2014-08-13 17:38:02.973026 7fe9bae887a0 20 read_log 192718'9372 (192718'9371) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:235 2014-08-06 15:34:21.709092 2014-08-13 17:38:02.973037 7fe9bae887a0 20 read_log 192718'9373 (192718'9372) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:235 2014-08-06 15:34:21.707768 2014-08-13 17:38:02.973049 7fe9bae887a0 20 read_log 192718'9374 (192718'9373) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:236 2014-08-06 15:34:22.049806 2014-08-13 17:38:02.973061 7fe9bae887a0 20 read_log 192718'9375 (192718'9374) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:236 2014-08-06 15:34:22.432285 2014-08-13 17:38:02.973097 7fe9bae887a0 20 read_log 192718'9376 (192718'9375) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:237 2014-08-06 15:34:22.434508 2014-08-13 17:38:02.973111 7fe9bae887a0 20 read_log 192718'9377 (192718'9376) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:237 2014-08-06 15:34:22.678458 2014-08-13 17:38:02.973122 7fe9bae887a0 20 read_log 192718'9378 (192718'9377) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:238 2014-08-06 15:34:22.692091 2014-08-13 17:38:02.973134 7fe9bae887a0 20 read_log 192718'9379 (192718'9378) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:238 2014-08-06 15:34:24.864361 2014-08-13 17:38:02.973146 7fe9bae887a0 20 read_log 192718'9380 (192718'9379) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:239 2014-08-06 15:34:25.389579 2014-08-13 17:38:02.973163 7fe9bae887a0 20 read_log 192718'9381 (192718'9380) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:239 2014-08-06 15:34:25.402166 2014-08-13 17:38:02.973175 7fe9bae887a0 20 read_log 192718'9382 (192718'9381) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:240 2014-08-06 15:34:26.168635 2014-08-13 17:38:02.973187 7fe9bae887a0 20 read_log 192718'9383 (192718'9382) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:240 2014-08-06 15:34:26.168581 2014-08-13 17:38:02.973199 7fe9bae887a0 20 read_log 192718'9384 (192718'9383) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:241 2014-08-06 15:34:26.347523 2014-08-13 17:38:02.973211 7fe9bae887a0 20 read_log 192718'9385 (192718'9384) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:241 2014-08-06 15:34:26.347411 2014-08-13 17:38:02.973223 7fe9bae887a0 20 read_log 192718'9386 (192718'9385) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:242 2014-08-06 15:34:26.598632 2014-08-13 17:38:02.973234 7fe9bae887a0 20 read_log 192718'9388 (192718'9386) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:242 2014-08-06 15:34:26.804029 2014-08-13 17:38:02.973246 7fe9bae887a0 20 read_log 192718'9389 (192718'9388) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:243 2014-08-06 15:34:26.817738 2014-08-13 17:38:02.973258 7fe9bae887a0 20 read_log 192718'9390 (192718'9389) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:243 2014-08-06 15:34:26.845248 2014-08-13 17:38:02.973270 7fe9bae887a0 20 read_log 192718'9391 (192718'9390) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:244 2014-08-06 15:34:27.104873 2014-08-13 17:38:02.973282 7fe9bae887a0 20 read_log 192718'9392 (192718'9391) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:244 2014-08-06 15:34:27.104927 2014-08-13 17:38:02.973293 7fe9bae887a0 20 read_log 192718'9393 (192718'9392) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:245 2014-08-06 15:34:27.304309 2014-08-13 17:38:02.973305 7fe9bae887a0 20 read_log 192718'9394 (192718'9393) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:245 2014-08-06 15:34:27.304330 2014-08-13 17:38:02.973317 7fe9bae887a0 20 read_log 192718'9395 (192718'9394) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:246 2014-08-06 15:34:27.344086 2014-08-13 17:38:02.973328 7fe9bae887a0 20 read_log 192718'9396 (192718'9395) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:246 2014-08-06 15:34:27.383166 2014-08-13 17:38:02.973340 7fe9bae887a0 20 read_log 192718'9397 (192718'9396) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:247 2014-08-06 15:34:27.406323 2014-08-13 17:38:02.973351 7fe9bae887a0 20 read_log 192718'9398 (192718'9397) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:247 2014-08-06 15:34:27.787451 2014-08-13 17:38:02.973363 7fe9bae887a0 20 read_log 192718'9399 (192718'9398) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:248 2014-08-06 15:34:27.797634 2014-08-13 17:38:02.973375 7fe9bae887a0 20 read_log 192718'9400 (192718'9399) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:249 2014-08-06 15:34:28.050031 2014-08-13 17:38:02.973393 7fe9bae887a0 20 read_log 192718'9401 (192718'9400) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:248 2014-08-06 15:34:28.048855 2014-08-13 17:38:02.973405 7fe9bae887a0 20 read_log 192718'9402 (192718'9401) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:250 2014-08-06 15:34:28.159440 2014-08-13 17:38:02.973417 7fe9bae887a0 20 read_log 192718'9403 (192718'9402) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:249 2014-08-06 15:34:28.337805 2014-08-13 17:38:02.973429 7fe9bae887a0 20 read_log 192718'9404 (192718'9403) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:251 2014-08-06 15:34:28.339697 2014-08-13 17:38:02.973441 7fe9bae887a0 20 read_log 192718'9405 (192718'9404) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:250 2014-08-06 15:34:28.549405 2014-08-13 17:38:02.973452 7fe9bae887a0 20 read_log 192718'9406 (192718'9405) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:252 2014-08-06 15:34:28.550392 2014-08-13 17:38:02.973464 7fe9bae887a0 20 read_log 192718'9407 (192718'9406) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:251 2014-08-06 15:34:29.042621 2014-08-13 17:38:02.973479 7fe9bae887a0 20 read_log 192718'9408 (192718'9407) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:253 2014-08-06 15:34:29.045759 2014-08-13 17:38:02.973510 7fe9bae887a0 20 read_log 192718'9409 (192718'9408) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:252 2014-08-06 15:34:29.071606 2014-08-13 17:38:02.973523 7fe9bae887a0 20 read_log 192718'9410 (192718'9409) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:254 2014-08-06 15:34:29.098234 2014-08-13 17:38:02.973535 7fe9bae887a0 20 read_log 192718'9411 (192718'9410) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:253 2014-08-06 15:34:29.155804 2014-08-13 17:38:02.973546 7fe9bae887a0 20 read_log 192718'9412 (192718'9411) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:254 2014-08-06 15:34:29.321339 2014-08-13 17:38:02.973558 7fe9bae887a0 20 read_log 192718'9413 (192718'9412) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:255 2014-08-06 15:34:29.329220 2014-08-13 17:38:02.973586 7fe9bae887a0 20 read_log 192718'9414 (192718'9413) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:255 2014-08-06 15:34:29.526970 2014-08-13 17:38:02.973599 7fe9bae887a0 20 read_log 192718'9415 (192718'9414) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:256 2014-08-06 15:34:29.528647 2014-08-13 17:38:02.973610 7fe9bae887a0 20 read_log 192718'9416 (192718'9415) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:256 2014-08-06 15:34:30.017798 2014-08-13 17:38:02.973623 7fe9bae887a0 20 read_log 192718'9417 (192718'9416) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:257 2014-08-06 15:34:30.019104 2014-08-13 17:38:02.973634 7fe9bae887a0 20 read_log 192718'9418 (192718'9417) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:257 2014-08-06 15:34:30.060530 2014-08-13 17:38:02.973647 7fe9bae887a0 20 read_log 192718'9419 (192718'9418) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:258 2014-08-06 15:34:30.103520 2014-08-13 17:38:02.973658 7fe9bae887a0 20 read_log 192718'9420 (192718'9419) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:258 2014-08-06 15:34:30.132825 2014-08-13 17:38:02.973670 7fe9bae887a0 20 read_log 192718'9421 (192718'9420) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:259 2014-08-06 15:34:30.298001 2014-08-13 17:38:02.973682 7fe9bae887a0 20 read_log 192718'9422 (192718'9421) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:259 2014-08-06 15:34:30.299139 2014-08-13 17:38:02.973694 7fe9bae887a0 20 read_log 192718'9423 (192718'9422) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:260 2014-08-06 15:34:30.516436 2014-08-13 17:38:02.973706 7fe9bae887a0 20 read_log 192718'9424 (192718'9423) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:260 2014-08-06 15:34:30.517083 2014-08-13 17:38:02.973727 7fe9bae887a0 20 read_log 192718'9425 (192718'9424) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:261 2014-08-06 15:34:30.850372 2014-08-13 17:38:02.973739 7fe9bae887a0 20 read_log 192718'9426 (192718'9425) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:261 2014-08-06 15:34:30.850057 2014-08-13 17:38:02.973751 7fe9bae887a0 20 read_log 192718'9427 (192718'9426) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:262 2014-08-06 15:34:31.045738 2014-08-13 17:38:02.973764 7fe9bae887a0 20 read_log 192718'9428 (192718'9427) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:262 2014-08-06 15:34:31.090955 2014-08-13 17:38:02.973776 7fe9bae887a0 20 read_log 192718'9429 (192718'9428) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:263 2014-08-06 15:34:31.280619 2014-08-13 17:38:02.973788 7fe9bae887a0 20 read_log 192718'9430 (192718'9429) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:263 2014-08-06 15:34:31.280380 2014-08-13 17:38:02.973799 7fe9bae887a0 20 read_log 192718'9431 (192718'9430) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:264 2014-08-06 15:34:31.650949 2014-08-13 17:38:02.973811 7fe9bae887a0 20 read_log 192718'9432 (192718'9431) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:264 2014-08-06 15:34:31.653118 2014-08-13 17:38:02.973822 7fe9bae887a0 20 read_log 192718'9433 (192718'9432) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:265 2014-08-06 15:34:31.976817 2014-08-13 17:38:02.973834 7fe9bae887a0 20 read_log 192718'9434 (192718'9433) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:265 2014-08-06 15:34:31.976930 2014-08-13 17:38:02.973846 7fe9bae887a0 20 read_log 192718'9435 (192718'9434) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:266 2014-08-06 15:34:32.005079 2014-08-13 17:38:02.973857 7fe9bae887a0 20 read_log 192718'9436 (192718'9435) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:266 2014-08-06 15:34:32.007062 2014-08-13 17:38:02.973869 7fe9bae887a0 20 read_log 192718'9437 (192718'9436) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:267 2014-08-06 15:34:32.032314 2014-08-13 17:38:02.973881 7fe9bae887a0 20 read_log 192718'9438 (192718'9437) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:267 2014-08-06 15:34:32.033584 2014-08-13 17:38:02.973893 7fe9bae887a0 20 read_log 192718'9439 (192718'9438) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:268 2014-08-06 15:34:32.074160 2014-08-13 17:38:02.973904 7fe9bae887a0 20 read_log 192718'9440 (192718'9439) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:268 2014-08-06 15:34:32.074144 2014-08-13 17:38:02.973916 7fe9bae887a0 20 read_log 192718'9441 (192718'9440) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:269 2014-08-06 15:34:32.099869 2014-08-13 17:38:02.973928 7fe9bae887a0 20 read_log 192718'9442 (192718'9441) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:269 2014-08-06 15:34:32.101380 2014-08-13 17:38:02.973939 7fe9bae887a0 20 read_log 192718'9443 (192718'9442) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:270 2014-08-06 15:34:32.138885 2014-08-13 17:38:02.973951 7fe9bae887a0 20 read_log 192718'9444 (192718'9443) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:270 2014-08-06 15:34:32.315720 2014-08-13 17:38:02.973963 7fe9bae887a0 20 read_log 192718'9445 (192718'9444) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:271 2014-08-06 15:34:32.360034 2014-08-13 17:38:02.973974 7fe9bae887a0 20 read_log 192718'9446 (192718'9445) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:271 2014-08-06 15:34:32.430779 2014-08-13 17:38:02.973986 7fe9bae887a0 20 read_log 192718'9447 (192718'9446) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:272 2014-08-06 15:34:32.565501 2014-08-13 17:38:02.973997 7fe9bae887a0 20 read_log 192718'9448 (192718'9447) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:272 2014-08-06 15:34:32.616932 2014-08-13 17:38:02.974031 7fe9bae887a0 20 read_log 192718'9449 (192718'9448) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:273 2014-08-06 15:34:32.779821 2014-08-13 17:38:02.974044 7fe9bae887a0 20 read_log 192718'9450 (192718'9449) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:273 2014-08-06 15:34:32.810195 2014-08-13 17:38:02.974056 7fe9bae887a0 20 read_log 192718'9451 (192718'9450) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:274 2014-08-06 15:34:33.181793 2014-08-13 17:38:02.974068 7fe9bae887a0 20 read_log 192718'9452 (192718'9451) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:274 2014-08-06 15:34:33.203156 2014-08-13 17:38:02.974080 7fe9bae887a0 20 read_log 192718'9453 (192718'9452) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:275 2014-08-06 15:34:33.236098 2014-08-13 17:38:02.974092 7fe9bae887a0 20 read_log 192718'9454 (192718'9453) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:275 2014-08-06 15:34:33.243098 2014-08-13 17:38:02.974104 7fe9bae887a0 20 read_log 192718'9455 (192718'9454) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:276 2014-08-06 15:34:33.314795 2014-08-13 17:38:02.974116 7fe9bae887a0 20 read_log 192718'9456 (192718'9455) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:276 2014-08-06 15:34:33.450532 2014-08-13 17:38:02.974127 7fe9bae887a0 20 read_log 192718'9457 (192718'9456) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:277 2014-08-06 15:34:33.452455 2014-08-13 17:38:02.974139 7fe9bae887a0 20 read_log 192718'9458 (192718'9457) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:277 2014-08-06 15:34:33.669689 2014-08-13 17:38:02.974151 7fe9bae887a0 20 read_log 192718'9459 (192718'9458) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:278 2014-08-06 15:34:33.683699 2014-08-13 17:38:02.974167 7fe9bae887a0 20 read_log 192718'9460 (192718'9459) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:278 2014-08-06 15:34:33.697681 2014-08-13 17:38:02.974179 7fe9bae887a0 20 read_log 192718'9461 (192718'9460) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:279 2014-08-06 15:34:33.716216 2014-08-13 17:38:02.974191 7fe9bae887a0 20 read_log 192718'9462 (192718'9461) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:279 2014-08-06 15:34:33.721828 2014-08-13 17:38:02.974203 7fe9bae887a0 20 read_log 192718'9463 (192718'9462) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:280 2014-08-06 15:34:33.754244 2014-08-13 17:38:02.974214 7fe9bae887a0 20 read_log 192718'9464 (192718'9463) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:280 2014-08-06 15:34:33.754951 2014-08-13 17:38:02.974226 7fe9bae887a0 20 read_log 192718'9465 (192718'9464) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:281 2014-08-06 15:34:33.895576 2014-08-13 17:38:02.974238 7fe9bae887a0 20 read_log 192718'9466 (192718'9465) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:281 2014-08-06 15:34:34.134558 2014-08-13 17:38:02.974251 7fe9bae887a0 20 read_log 192718'9467 (192718'9466) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:282 2014-08-06 15:34:34.136568 2014-08-13 17:38:02.974263 7fe9bae887a0 20 read_log 192718'9468 (192718'9467) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:282 2014-08-06 15:34:34.432043 2014-08-13 17:38:02.974274 7fe9bae887a0 20 read_log 192718'9469 (192718'9468) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:283 2014-08-06 15:34:34.485561 2014-08-13 17:38:02.974320 7fe9bae887a0 20 read_log 192718'9470 (192718'9469) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:283 2014-08-06 15:34:34.635547 2014-08-13 17:38:02.974334 7fe9bae887a0 20 read_log 192718'9471 (192718'9470) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:284 2014-08-06 15:34:34.970708 2014-08-13 17:38:02.974346 7fe9bae887a0 20 read_log 192718'9472 (192718'9471) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:284 2014-08-06 15:34:35.127758 2014-08-13 17:38:02.974364 7fe9bae887a0 20 read_log 192718'9473 (192718'9472) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:285 2014-08-06 15:34:35.413390 2014-08-13 17:38:02.974377 7fe9bae887a0 20 read_log 192718'9474 (192718'9473) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:285 2014-08-06 15:34:35.413442 2014-08-13 17:38:02.974388 7fe9bae887a0 20 read_log 192718'9475 (192718'9474) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:286 2014-08-06 15:34:35.484512 2014-08-13 17:38:02.974400 7fe9bae887a0 20 read_log 192718'9476 (192718'9475) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:286 2014-08-06 15:34:35.484663 2014-08-13 17:38:02.974412 7fe9bae887a0 20 read_log 192718'9477 (192718'9476) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:287 2014-08-06 15:34:35.533701 2014-08-13 17:38:02.974423 7fe9bae887a0 20 read_log 192718'9478 (192718'9477) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:287 2014-08-06 15:34:35.703432 2014-08-13 17:38:02.974435 7fe9bae887a0 20 read_log 192718'9479 (192718'9478) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:288 2014-08-06 15:34:35.956280 2014-08-13 17:38:02.974447 7fe9bae887a0 20 read_log 192718'9480 (192718'9479) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:288 2014-08-06 15:34:36.162178 2014-08-13 17:38:02.974459 7fe9bae887a0 20 read_log 192718'9481 (192718'9480) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:289 2014-08-06 15:34:36.189673 2014-08-13 17:38:02.974472 7fe9bae887a0 20 read_log 192718'9482 (192718'9481) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:289 2014-08-06 15:34:36.201506 2014-08-13 17:38:02.974484 7fe9bae887a0 20 read_log 192718'9483 (192718'9482) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:290 2014-08-06 15:34:36.219218 2014-08-13 17:38:02.974496 7fe9bae887a0 20 read_log 192718'9484 (192718'9483) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:290 2014-08-06 15:34:36.262544 2014-08-13 17:38:02.974508 7fe9bae887a0 20 read_log 192718'9485 (192718'9484) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:291 2014-08-06 15:34:36.469422 2014-08-13 17:38:02.974519 7fe9bae887a0 20 read_log 192718'9486 (192718'9485) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:291 2014-08-06 15:34:36.468418 2014-08-13 17:38:02.974531 7fe9bae887a0 20 read_log 192718'9487 (192718'9486) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:292 2014-08-06 15:34:36.654039 2014-08-13 17:38:02.974543 7fe9bae887a0 20 read_log 192718'9488 (192718'9487) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:292 2014-08-06 15:34:36.653427 2014-08-13 17:38:02.974554 7fe9bae887a0 20 read_log 192718'9489 (192718'9488) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:293 2014-08-06 15:34:36.683891 2014-08-13 17:38:02.974566 7fe9bae887a0 20 read_log 192718'9490 (192718'9489) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:293 2014-08-06 15:34:36.687012 2014-08-13 17:38:02.974578 7fe9bae887a0 20 read_log 192718'9491 (192718'9490) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:294 2014-08-06 15:34:37.261459 2014-08-13 17:38:02.974590 7fe9bae887a0 20 read_log 192718'9492 (192718'9491) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:294 2014-08-06 15:34:37.262969 2014-08-13 17:38:02.974601 7fe9bae887a0 20 read_log 192718'9493 (192718'9492) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:295 2014-08-06 15:34:37.293606 2014-08-13 17:38:02.974613 7fe9bae887a0 20 read_log 192718'9494 (192718'9493) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:295 2014-08-06 15:34:37.296923 2014-08-13 17:38:02.974625 7fe9bae887a0 20 read_log 192718'9495 (192718'9494) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:296 2014-08-06 15:34:37.418449 2014-08-13 17:38:02.974636 7fe9bae887a0 20 read_log 192718'9496 (192718'9495) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:296 2014-08-06 15:34:37.444352 2014-08-13 17:38:02.974658 7fe9bae887a0 20 read_log 192718'9497 (192718'9496) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:297 2014-08-06 15:34:37.475696 2014-08-13 17:38:02.974671 7fe9bae887a0 20 read_log 192718'9498 (192718'9497) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:297 2014-08-06 15:34:37.493942 2014-08-13 17:38:02.974682 7fe9bae887a0 20 read_log 192718'9499 (192718'9498) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:298 2014-08-06 15:34:37.662732 2014-08-13 17:38:02.974694 7fe9bae887a0 20 read_log 192718'9500 (192718'9499) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:298 2014-08-06 15:34:37.898344 2014-08-13 17:38:02.974706 7fe9bae887a0 20 read_log 192718'9501 (192718'9500) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:299 2014-08-06 15:34:38.032839 2014-08-13 17:38:02.974718 7fe9bae887a0 20 read_log 192718'9502 (192718'9501) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:299 2014-08-06 15:34:38.033089 2014-08-13 17:38:02.974730 7fe9bae887a0 20 read_log 192788'9506 (192718'9502) clean 1a453a0e/core.19866+13+2/head//24 by osd.367.0:43043 2014-08-11 10:23:49.050751 2014-08-13 17:38:02.974742 7fe9bae887a0 20 read_log 192798'9507 (192788'9506) delete 1a453a0e/core.19866+13+2/head//24 by osd.367.0:43044 2014-08-11 10:24:43.632401 2014-08-13 17:38:02.974754 7fe9bae887a0 20 read_log 192834'9835 (0'0) promote b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by osd.367.0:43700 2014-08-11 15:23:41.427154 2014-08-13 17:38:02.974766 7fe9bae887a0 20 read_log 192834'9836 (192834'9835) modify b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.933448.0:1474 2014-08-11 15:23:41.369678 2014-08-13 17:38:02.974778 7fe9bae887a0 20 read_log 192834'9913 (0'0) promote 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.367.0:43994 2014-08-11 15:24:12.512128 2014-08-13 17:38:02.974790 7fe9bae887a0 20 read_log 192834'9914 (192834'9913) modify 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by client.933448.0:8051 2014-08-11 15:24:12.503712 2014-08-13 17:38:02.974802 7fe9bae887a0 20 read_log 192834'9915 (0'0) promote b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by osd.367.0:44002 2014-08-11 15:24:13.338244 2014-08-13 17:38:02.974814 7fe9bae887a0 20 read_log 192834'9916 (192834'9915) modify b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.933448.0:8261 2014-08-11 15:24:13.330648 2014-08-13 17:38:02.974827 7fe9bae887a0 20 read_log 192834'9945 (0'0) promote 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by osd.367.0:44110 2014-08-11 15:24:23.319699 2014-08-13 17:38:02.974840 7fe9bae887a0 20 read_log 192834'9946 (192834'9945) modify 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.933448.0:10457 2014-08-11 15:24:23.311353 2014-08-13 17:38:02.974852 7fe9bae887a0 20 read_log 192834'9951 (0'0) promote eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by osd.367.0:44128 2014-08-11 15:24:26.987126 2014-08-13 17:38:02.974864 7fe9bae887a0 20 read_log 192834'9952 (192834'9951) modify eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.933448.0:11245 2014-08-11 15:24:26.980403 2014-08-13 17:38:02.974876 7fe9bae887a0 20 read_log 192834'9959 (0'0) promote 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by osd.367.0:44146 2014-08-11 15:24:28.453759 2014-08-13 17:38:02.974888 7fe9bae887a0 20 read_log 192834'9960 (192834'9959) modify 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.933448.0:11590 2014-08-11 15:24:28.444246 2014-08-13 17:38:02.974900 7fe9bae887a0 20 read_log 192834'9969 (0'0) promote a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.367.0:44178 2014-08-11 15:24:31.639462 2014-08-13 17:38:02.974919 7fe9bae887a0 20 read_log 192834'9970 (192834'9969) modify a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by client.933448.0:12246 2014-08-11 15:24:31.632594 2014-08-13 17:38:02.974932 7fe9bae887a0 20 read_log 192834'9973 (0'0) promote 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.367.0:44186 2014-08-11 15:24:32.581542 2014-08-13 17:38:02.974944 7fe9bae887a0 20 read_log 192834'9974 (192834'9973) modify 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by client.933448.0:12497 2014-08-11 15:24:32.572392 2014-08-13 17:38:02.974956 7fe9bae887a0 20 read_log 192834'10007 (0'0) promote 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by osd.367.0:44275 2014-08-11 15:24:41.807564 2014-08-13 17:38:02.974968 7fe9bae887a0 20 read_log 192834'10008 (0'0) promote 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by osd.367.0:44276 2014-08-11 15:24:41.809934 2014-08-13 17:38:02.974979 7fe9bae887a0 20 read_log 192834'10009 (192834'10008) modify 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.933448.0:14137 2014-08-11 15:24:41.553448 2014-08-13 17:38:02.974992 7fe9bae887a0 20 read_log 192834'10010 (192834'10007) modify 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.933448.0:14074 2014-08-11 15:24:41.040219 2014-08-13 17:38:02.975004 7fe9bae887a0 20 read_log 192834'10031 (0'0) promote 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by osd.367.0:44417 2014-08-11 15:24:51.989709 2014-08-13 17:38:02.975016 7fe9bae887a0 20 read_log 192834'10032 (192834'10031) modify 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.933448.0:15949 2014-08-11 15:24:51.981475 2014-08-13 17:38:02.975028 7fe9bae887a0 20 read_log 192834'10041 (0'0) promote b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by osd.367.0:44451 2014-08-11 15:24:56.754712 2014-08-13 17:38:02.975040 7fe9bae887a0 20 read_log 192834'10042 (192834'10041) modify b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.933448.0:16700 2014-08-11 15:24:56.747312 2014-08-13 17:38:02.975052 7fe9bae887a0 20 read_log 192834'10049 (0'0) promote a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by osd.367.0:44475 2014-08-11 15:24:59.246609 2014-08-13 17:38:02.975064 7fe9bae887a0 20 read_log 192834'10050 (192834'10049) modify a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.933448.0:17317 2014-08-11 15:24:59.239737 2014-08-13 17:38:02.975076 7fe9bae887a0 20 read_log 192834'10053 (0'0) promote ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by osd.367.0:44504 2014-08-11 15:25:00.769563 2014-08-13 17:38:02.975088 7fe9bae887a0 20 read_log 192834'10054 (192834'10053) modify ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.933448.0:17619 2014-08-11 15:25:00.759783 2014-08-13 17:38:02.975100 7fe9bae887a0 20 read_log 192834'10061 (0'0) promote 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by osd.367.0:44520 2014-08-11 15:25:03.090170 2014-08-13 17:38:02.975112 7fe9bae887a0 20 read_log 192834'10062 (192834'10061) modify 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.933448.0:18021 2014-08-11 15:25:03.082182 2014-08-13 17:38:02.975124 7fe9bae887a0 20 read_log 192834'10073 (0'0) promote ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by osd.367.0:44548 2014-08-11 15:25:05.509993 2014-08-13 17:38:02.975136 7fe9bae887a0 20 read_log 192834'10074 (192834'10073) modify ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.933448.0:18490 2014-08-11 15:25:05.503239 2014-08-13 17:38:02.975148 7fe9bae887a0 20 read_log 192834'10087 (0'0) promote c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.367.0:44597 2014-08-11 15:25:09.187315 2014-08-13 17:38:02.975176 7fe9bae887a0 20 read_log 192834'10088 (192834'10087) modify c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by client.933448.0:19226 2014-08-11 15:25:09.179086 2014-08-13 17:38:02.975189 7fe9bae887a0 20 read_log 192834'10107 (0'0) promote 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by osd.367.0:44684 2014-08-11 15:25:20.308380 2014-08-13 17:38:02.975201 7fe9bae887a0 20 read_log 192834'10108 (192834'10107) modify 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.933448.0:20323 2014-08-11 15:25:20.299110 2014-08-13 17:38:02.975213 7fe9bae887a0 20 read_log 192834'10109 (0'0) promote 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.367.0:44686 2014-08-11 15:25:20.685775 2014-08-13 17:38:02.975225 7fe9bae887a0 20 read_log 192834'10110 (192834'10109) modify 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by client.933448.0:20376 2014-08-11 15:25:20.678929 2014-08-13 17:38:02.975237 7fe9bae887a0 20 read_log 192834'10127 (0'0) promote 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.367.0:44747 2014-08-11 15:25:25.904854 2014-08-13 17:38:02.975249 7fe9bae887a0 20 read_log 192834'10128 (192834'10127) modify 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by client.933448.0:21044 2014-08-11 15:25:25.896007 2014-08-13 17:38:02.975261 7fe9bae887a0 20 read_log 192834'10131 (0'0) promote b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by osd.367.0:44784 2014-08-11 15:25:31.102596 2014-08-13 17:38:02.975273 7fe9bae887a0 20 read_log 192834'10132 (192834'10131) modify b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.933448.0:21580 2014-08-11 15:25:31.095215 2014-08-13 17:38:02.975286 7fe9bae887a0 20 read_log 192834'10153 (192834'9836) modify b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.933448.0:23765 2014-08-11 15:25:39.807801 2014-08-13 17:38:02.975298 7fe9bae887a0 20 read_log 192834'10191 (192834'9914) modify 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by client.933448.0:30342 2014-08-11 15:25:48.760307 2014-08-13 17:38:02.975310 7fe9bae887a0 20 read_log 192834'10192 (192834'9916) modify b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.933448.0:30552 2014-08-11 15:25:48.980703 2014-08-13 17:38:02.975322 7fe9bae887a0 20 read_log 192834'10207 (192834'9946) modify 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.933448.0:32748 2014-08-11 15:25:52.333271 2014-08-13 17:38:02.975334 7fe9bae887a0 20 read_log 192834'10210 (192834'9952) modify eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.933448.0:33536 2014-08-11 15:25:53.645114 2014-08-13 17:38:02.975346 7fe9bae887a0 20 read_log 192834'10214 (192834'9960) modify 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.933448.0:33881 2014-08-11 15:25:54.329367 2014-08-13 17:38:02.975357 7fe9bae887a0 20 read_log 192834'10219 (192834'9970) modify a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by client.933448.0:34537 2014-08-11 15:25:55.141588 2014-08-13 17:38:02.975371 7fe9bae887a0 20 read_log 192834'10221 (192834'9974) modify 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by client.933448.0:34788 2014-08-11 15:25:55.429979 2014-08-13 17:38:02.975391 7fe9bae887a0 20 read_log 192834'10238 (192834'10010) modify 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.933448.0:36365 2014-08-11 15:25:57.612388 2014-08-13 17:38:02.975402 7fe9bae887a0 20 read_log 192834'10239 (192834'10009) modify 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.933448.0:36428 2014-08-11 15:25:57.721027 2014-08-13 17:38:02.975414 7fe9bae887a0 20 read_log 192834'10250 (192834'10032) modify 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.933448.0:38240 2014-08-11 15:26:00.688589 2014-08-13 17:38:02.975436 7fe9bae887a0 20 read_log 192834'10255 (192834'10042) modify b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.933448.0:38991 2014-08-11 15:26:01.456353 2014-08-13 17:38:02.975449 7fe9bae887a0 20 read_log 192834'10259 (192834'10050) modify a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.933448.0:39608 2014-08-11 15:26:01.993612 2014-08-13 17:38:02.975461 7fe9bae887a0 20 read_log 192834'10261 (192834'10054) modify ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.933448.0:39910 2014-08-11 15:26:02.164784 2014-08-13 17:38:02.975473 7fe9bae887a0 20 read_log 192834'10265 (192834'10062) modify 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.933448.0:40312 2014-08-11 15:26:02.564529 2014-08-13 17:38:02.975485 7fe9bae887a0 20 read_log 192834'10271 (192834'10074) modify ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.933448.0:40781 2014-08-11 15:26:02.729403 2014-08-13 17:38:02.975499 7fe9bae887a0 20 read_log 192834'10278 (192834'10088) modify c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by client.933448.0:41517 2014-08-11 15:26:02.968215 2014-08-13 17:38:02.975511 7fe9bae887a0 20 read_log 192834'10288 (192834'10108) modify 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.933448.0:42614 2014-08-11 15:26:03.332409 2014-08-13 17:38:02.975523 7fe9bae887a0 20 read_log 192834'10289 (192834'10110) modify 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by client.933448.0:42667 2014-08-11 15:26:03.347371 2014-08-13 17:38:02.975535 7fe9bae887a0 20 read_log 192834'10298 (192834'10128) modify 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by client.933448.0:43335 2014-08-11 15:26:03.581678 2014-08-13 17:38:02.975547 7fe9bae887a0 20 read_log 192834'10300 (192834'10132) modify b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.933448.0:43871 2014-08-11 15:26:03.749194 2014-08-13 17:38:02.975559 7fe9bae887a0 20 read_log 192834'10324 (0'0) promote 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by osd.367.0:45670 2014-08-11 15:26:18.230299 2014-08-13 17:38:02.975571 7fe9bae887a0 20 read_log 192834'10325 (192834'10324) modify 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.1883634.0:1028 2014-08-11 15:26:18.222005 2014-08-13 17:38:02.975583 7fe9bae887a0 20 read_log 192834'10338 (0'0) promote 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.367.0:45704 2014-08-11 15:26:21.772016 2014-08-13 17:38:02.975595 7fe9bae887a0 20 read_log 192834'10339 (192834'10338) modify 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by client.1883634.0:1976 2014-08-11 15:26:21.763612 2014-08-13 17:38:02.975607 7fe9bae887a0 20 read_log 192834'10342 (0'0) promote 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.367.0:45722 2014-08-11 15:26:22.671660 2014-08-13 17:38:02.975619 7fe9bae887a0 20 read_log 192834'10343 (192834'10342) modify 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by client.1883634.0:2218 2014-08-11 15:26:22.664375 2014-08-13 17:38:02.975632 7fe9bae887a0 20 read_log 192834'10350 (0'0) promote 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by osd.367.0:45749 2014-08-11 15:26:25.003570 2014-08-13 17:38:02.975644 7fe9bae887a0 20 read_log 192834'10351 (192834'10350) modify 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.1883634.0:2869 2014-08-11 15:26:24.986652 2014-08-13 17:38:02.975656 7fe9bae887a0 20 read_log 192834'10364 (0'0) promote fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.367.0:45812 2014-08-11 15:26:30.620693 2014-08-13 17:38:02.975668 7fe9bae887a0 20 read_log 192834'10365 (192834'10364) modify fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by client.1883634.0:4458 2014-08-11 15:26:30.613197 2014-08-13 17:38:02.975687 7fe9bae887a0 20 read_log 192834'10390 (0'0) promote 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by osd.367.0:45886 2014-08-11 15:26:36.868098 2014-08-13 17:38:02.975700 7fe9bae887a0 20 read_log 192834'10391 (192834'10390) modify 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.1883634.0:6128 2014-08-11 15:26:36.859905 2014-08-13 17:38:02.975712 7fe9bae887a0 20 read_log 192834'10410 (0'0) promote 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by osd.367.0:45981 2014-08-11 15:26:44.861499 2014-08-13 17:38:02.975724 7fe9bae887a0 20 read_log 192834'10411 (192834'10410) modify 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.1883634.0:8072 2014-08-11 15:26:44.854070 2014-08-13 17:38:02.975736 7fe9bae887a0 20 read_log 192834'10412 (0'0) promote 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by osd.367.0:45985 2014-08-11 15:26:45.140993 2014-08-13 17:38:02.975748 7fe9bae887a0 20 read_log 192834'10413 (192834'10412) modify 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.1883634.0:8145 2014-08-11 15:26:45.133317 2014-08-13 17:38:02.975760 7fe9bae887a0 20 read_log 192834'10442 (0'0) promote c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by osd.367.0:46189 2014-08-11 15:26:53.006762 2014-08-13 17:38:02.975772 7fe9bae887a0 20 read_log 192834'10443 (192834'10442) modify c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.1883634.0:9935 2014-08-11 15:26:52.998361 2014-08-13 17:38:02.975786 7fe9bae887a0 20 read_log 192834'10464 (0'0) promote c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.367.0:46331 2014-08-11 15:27:01.171999 2014-08-13 17:38:02.975798 7fe9bae887a0 20 read_log 192834'10465 (192834'10464) modify c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by client.1883634.0:11751 2014-08-11 15:27:01.163584 2014-08-13 17:38:02.975810 7fe9bae887a0 20 read_log 192834'10466 (0'0) promote 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.367.0:46333 2014-08-11 15:27:01.278361 2014-08-13 17:38:02.975823 7fe9bae887a0 20 read_log 192834'10467 (192834'10466) modify 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by client.1883634.0:11787 2014-08-11 15:27:01.271808 2014-08-13 17:38:02.975835 7fe9bae887a0 20 read_log 192834'10478 (0'0) promote d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.367.0:46434 2014-08-11 15:27:03.766805 2014-08-13 17:38:02.975847 7fe9bae887a0 20 read_log 192834'10479 (192834'10478) modify d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by client.1883634.0:12381 2014-08-11 15:27:03.759006 2014-08-13 17:38:02.975859 7fe9bae887a0 20 read_log 192834'10512 (0'0) promote 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by osd.367.0:46564 2014-08-11 15:27:09.966075 2014-08-13 17:38:02.975871 7fe9bae887a0 20 read_log 192834'10513 (192834'10512) modify 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.1883634.0:13522 2014-08-11 15:27:09.958549 2014-08-13 17:38:02.975883 7fe9bae887a0 20 read_log 192834'10532 (0'0) promote c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by osd.367.0:46616 2014-08-11 15:27:15.831369 2014-08-13 17:38:02.975895 7fe9bae887a0 20 read_log 192834'10533 (192834'10532) modify c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.1883634.0:14880 2014-08-11 15:27:15.822132 2014-08-13 17:38:02.975907 7fe9bae887a0 20 read_log 192834'10546 (0'0) promote a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by osd.367.0:46636 2014-08-11 15:27:17.403151 2014-08-13 17:38:02.975918 7fe9bae887a0 20 read_log 192834'10547 (192834'10546) modify a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.1883634.0:15165 2014-08-11 15:27:17.394485 2014-08-13 17:38:02.975956 7fe9bae887a0 20 read_log 192834'10554 (0'0) promote abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by osd.367.0:46696 2014-08-11 15:27:21.299461 2014-08-13 17:38:02.975970 7fe9bae887a0 20 read_log 192834'10555 (192834'10554) modify abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.1883634.0:15708 2014-08-11 15:27:21.292173 2014-08-13 17:38:02.975982 7fe9bae887a0 20 read_log 192834'10556 (0'0) promote 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by osd.367.0:46706 2014-08-11 15:27:24.871224 2014-08-13 17:38:02.975994 7fe9bae887a0 20 read_log 192834'10557 (192834'10556) modify 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.1883634.0:15959 2014-08-11 15:27:24.861453 2014-08-13 17:38:02.976006 7fe9bae887a0 20 read_log 192834'10578 (0'0) promote 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.367.0:46797 2014-08-11 15:27:34.317427 2014-08-13 17:38:02.976018 7fe9bae887a0 20 read_log 192834'10579 (192834'10578) modify 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by client.1883634.0:17330 2014-08-11 15:27:34.308390 2014-08-13 17:38:02.976030 7fe9bae887a0 20 read_log 192834'10594 (0'0) promote 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by osd.367.0:46861 2014-08-11 15:27:40.508954 2014-08-13 17:38:02.976042 7fe9bae887a0 20 read_log 192834'10595 (192834'10594) modify 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.1883634.0:18039 2014-08-11 15:27:40.500597 2014-08-13 17:38:02.976054 7fe9bae887a0 20 read_log 192834'10618 (0'0) promote 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.367.0:46928 2014-08-11 15:27:47.976828 2014-08-13 17:38:02.976066 7fe9bae887a0 20 read_log 192834'10619 (192834'10618) modify 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by client.1883634.0:18877 2014-08-11 15:27:47.969599 2014-08-13 17:38:02.976078 7fe9bae887a0 20 read_log 192834'10674 (0'0) promote 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.367.0:47146 2014-08-11 15:28:04.960476 2014-08-13 17:38:02.976090 7fe9bae887a0 20 read_log 192834'10675 (192834'10674) modify 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by client.1883634.0:21686 2014-08-11 15:28:04.953091 2014-08-13 17:38:02.976102 7fe9bae887a0 20 read_log 192834'10676 (0'0) promote 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by osd.367.0:47176 2014-08-11 15:28:06.660812 2014-08-13 17:38:02.976114 7fe9bae887a0 20 read_log 192834'10677 (192834'10676) modify 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.1883634.0:22092 2014-08-11 15:28:06.651135 2014-08-13 17:38:02.976126 7fe9bae887a0 20 read_log 192834'10741 (192834'10325) modify 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.1883634.0:24352 2014-08-11 15:28:18.931915 2014-08-13 17:38:02.976138 7fe9bae887a0 20 read_log 192834'10749 (192834'10339) modify 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by client.1883634.0:25300 2014-08-11 15:28:20.474491 2014-08-13 17:38:02.976150 7fe9bae887a0 20 read_log 192834'10751 (192834'10343) modify 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by client.1883634.0:25542 2014-08-11 15:28:21.518397 2014-08-13 17:38:02.976167 7fe9bae887a0 20 read_log 192834'10755 (192834'10351) modify 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.1883634.0:26193 2014-08-11 15:28:22.748557 2014-08-13 17:38:02.976179 7fe9bae887a0 20 read_log 192834'10763 (192834'10365) modify fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by client.1883634.0:27782 2014-08-11 15:28:24.980375 2014-08-13 17:38:02.976191 7fe9bae887a0 20 read_log 192834'10776 (192834'10391) modify 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.1883634.0:29452 2014-08-11 15:28:27.290531 2014-08-13 17:38:02.976213 7fe9bae887a0 20 read_log 192834'10786 (192834'10411) modify 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.1883634.0:31396 2014-08-11 15:28:30.421943 2014-08-13 17:38:02.976226 7fe9bae887a0 20 read_log 192834'10787 (192834'10413) modify 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.1883634.0:31469 2014-08-11 15:28:30.573803 2014-08-13 17:38:02.976238 7fe9bae887a0 20 read_log 192834'10804 (192834'10443) modify c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.1883634.0:33259 2014-08-11 15:28:32.949970 2014-08-13 17:38:02.976250 7fe9bae887a0 20 read_log 192834'10818 (192834'10465) modify c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by client.1883634.0:35075 2014-08-11 15:28:35.996335 2014-08-13 17:38:02.976262 7fe9bae887a0 20 read_log 192834'10819 (192834'10467) modify 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by client.1883634.0:35111 2014-08-11 15:28:36.041264 2014-08-13 17:38:02.976274 7fe9bae887a0 20 read_log 192834'10825 (192834'10479) modify d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by client.1883634.0:35705 2014-08-11 15:28:37.026666 2014-08-13 17:38:02.976286 7fe9bae887a0 20 read_log 192834'10844 (192834'10513) modify 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.1883634.0:36846 2014-08-11 15:28:38.483690 2014-08-13 17:38:02.976298 7fe9bae887a0 20 read_log 192834'10854 (192834'10533) modify c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.1883634.0:38204 2014-08-11 15:28:40.360246 2014-08-13 17:38:02.976314 7fe9bae887a0 20 read_log 192834'10861 (192834'10547) modify a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.1883634.0:38489 2014-08-11 15:28:40.699539 2014-08-13 17:38:02.976326 7fe9bae887a0 20 read_log 192834'10865 (192834'10555) modify abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.1883634.0:39032 2014-08-11 15:28:41.501660 2014-08-13 17:38:02.976338 7fe9bae887a0 20 read_log 192834'10866 (192834'10557) modify 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.1883634.0:39283 2014-08-11 15:28:41.964185 2014-08-13 17:38:02.976350 7fe9bae887a0 20 read_log 192834'10877 (192834'10579) modify 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by client.1883634.0:40654 2014-08-11 15:28:44.085797 2014-08-13 17:38:02.976362 7fe9bae887a0 20 read_log 192834'10885 (192834'10595) modify 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.1883634.0:41363 2014-08-11 15:28:44.965739 2014-08-13 17:38:02.976374 7fe9bae887a0 20 read_log 192834'10897 (192834'10619) modify 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by client.1883634.0:42201 2014-08-11 15:28:46.167720 2014-08-13 17:38:02.976386 7fe9bae887a0 20 read_log 192834'10924 (192834'10675) modify 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by client.1883634.0:45010 2014-08-11 15:28:47.121262 2014-08-13 17:38:02.976398 7fe9bae887a0 20 read_log 192834'10925 (192834'10677) modify 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.1883634.0:45416 2014-08-11 15:28:47.270369 2014-08-13 17:38:02.976410 7fe9bae887a0 20 read_log 192888'10944 (0'0) promote ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by osd.367.0:51521 2014-08-11 16:45:04.731686 2014-08-13 17:38:02.976422 7fe9bae887a0 20 read_log 192888'10945 (192888'10944) modify ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.1315259.0:2504 2014-08-11 16:45:04.724932 2014-08-13 17:38:02.976434 7fe9bae887a0 20 read_log 192888'10946 (0'0) promote 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by osd.367.0:51523 2014-08-11 16:45:05.067198 2014-08-13 17:38:02.976446 7fe9bae887a0 20 read_log 192888'10947 (192888'10946) modify 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.1315259.0:2593 2014-08-11 16:45:05.060076 2014-08-13 17:38:02.976465 7fe9bae887a0 20 read_log 192888'10950 (0'0) promote a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.367.0:51531 2014-08-11 16:45:07.185859 2014-08-13 17:38:02.976478 7fe9bae887a0 20 read_log 192888'10951 (192888'10950) modify a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by client.1315259.0:3145 2014-08-11 16:45:07.176934 2014-08-13 17:38:02.976490 7fe9bae887a0 20 read_log 192888'10954 (0'0) promote 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by osd.367.0:51577 2014-08-11 16:45:13.116289 2014-08-13 17:38:02.976502 7fe9bae887a0 20 read_log 192888'10955 (192888'10954) modify 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.1315259.0:4731 2014-08-11 16:45:13.108903 2014-08-13 17:38:02.976514 7fe9bae887a0 20 read_log 192888'10962 (0'0) promote bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by osd.367.0:51625 2014-08-11 16:45:19.634169 2014-08-13 17:38:02.976526 7fe9bae887a0 20 read_log 192888'10963 (192888'10962) modify bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.1315259.0:6512 2014-08-11 16:45:19.625808 2014-08-13 17:38:02.976538 7fe9bae887a0 20 read_log 192888'10976 (0'0) promote 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by osd.367.0:51751 2014-08-11 16:45:34.155147 2014-08-13 17:38:02.976550 7fe9bae887a0 20 read_log 192888'10977 (192888'10976) modify 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.1315259.0:10386 2014-08-11 16:45:34.148712 2014-08-13 17:38:02.976562 7fe9bae887a0 20 read_log 192888'10982 (0'0) promote a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by osd.367.0:51833 2014-08-11 16:45:47.265985 2014-08-13 17:38:02.976574 7fe9bae887a0 20 read_log 192888'10983 (192888'10982) modify a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.1315259.0:13598 2014-08-11 16:45:47.258506 2014-08-13 17:38:02.976586 7fe9bae887a0 20 read_log 192888'10984 (0'0) promote cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by osd.367.0:51851 2014-08-11 16:45:50.173287 2014-08-13 17:38:02.976616 7fe9bae887a0 20 read_log 192888'10985 (192888'10984) modify cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.1315259.0:14358 2014-08-11 16:45:50.165361 2014-08-13 17:38:02.976630 7fe9bae887a0 20 read_log 192888'10986 (0'0) promote 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by osd.367.0:51861 2014-08-11 16:45:51.629138 2014-08-13 17:38:02.976642 7fe9bae887a0 20 read_log 192888'10987 (192888'10986) modify 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.1315259.0:14757 2014-08-11 16:45:51.622514 2014-08-13 17:38:02.976655 7fe9bae887a0 20 read_log 192888'10988 (0'0) promote 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by osd.367.0:51863 2014-08-11 16:45:51.803029 2014-08-13 17:38:02.976667 7fe9bae887a0 20 read_log 192888'10989 (192888'10988) modify 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.1315259.0:14803 2014-08-11 16:45:51.794589 2014-08-13 17:38:02.976679 7fe9bae887a0 20 read_log 192888'10992 (0'0) promote 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.367.0:51891 2014-08-11 16:45:56.753371 2014-08-13 17:38:02.976691 7fe9bae887a0 20 read_log 192888'10993 (192888'10992) modify 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by client.1315259.0:16086 2014-08-11 16:45:56.747312 2014-08-13 17:38:02.976723 7fe9bae887a0 20 read_log 192888'10994 (0'0) promote 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by osd.367.0:51893 2014-08-11 16:45:57.273489 2014-08-13 17:38:02.976736 7fe9bae887a0 20 read_log 192888'10995 (192888'10994) modify 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.1315259.0:16224 2014-08-11 16:45:57.266341 2014-08-13 17:38:02.976755 7fe9bae887a0 20 read_log 192888'10996 (0'0) promote 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by osd.367.0:51907 2014-08-11 16:45:59.318397 2014-08-13 17:38:02.976767 7fe9bae887a0 20 read_log 192888'10997 (192888'10996) modify 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.1315259.0:16741 2014-08-11 16:45:59.311077 2014-08-13 17:38:02.976779 7fe9bae887a0 20 read_log 192888'11000 (0'0) promote 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by osd.367.0:51923 2014-08-11 16:46:01.895782 2014-08-13 17:38:02.976791 7fe9bae887a0 20 read_log 192888'11001 (192888'11000) modify 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.1315259.0:17392 2014-08-11 16:46:01.887096 2014-08-13 17:38:02.976803 7fe9bae887a0 20 read_log 192888'11006 (0'0) promote d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by osd.367.0:51969 2014-08-11 16:46:06.327682 2014-08-13 17:38:02.976815 7fe9bae887a0 20 read_log 192888'11007 (192888'11006) modify d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.1315259.0:18512 2014-08-11 16:46:06.321872 2014-08-13 17:38:02.976828 7fe9bae887a0 20 read_log 192888'11010 (0'0) promote 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by osd.367.0:51975 2014-08-11 16:46:06.554185 2014-08-13 17:38:02.976841 7fe9bae887a0 20 read_log 192888'11011 (192888'11010) modify 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.1315259.0:18570 2014-08-11 16:46:06.545898 2014-08-13 17:38:02.976853 7fe9bae887a0 20 read_log 192888'11014 (0'0) promote c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by osd.367.0:52013 2014-08-11 16:46:11.080518 2014-08-13 17:38:02.976866 7fe9bae887a0 20 read_log 192888'11015 (192888'11014) modify c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.1315259.0:19716 2014-08-11 16:46:11.073196 2014-08-13 17:38:02.976878 7fe9bae887a0 20 read_log 192888'11016 (0'0) promote e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by osd.367.0:52045 2014-08-11 16:46:15.053276 2014-08-13 17:38:02.976890 7fe9bae887a0 20 read_log 192888'11017 (192888'11016) modify e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.1315259.0:20776 2014-08-11 16:46:15.045952 2014-08-13 17:38:02.976902 7fe9bae887a0 20 read_log 192888'11018 (0'0) promote 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by osd.367.0:52119 2014-08-11 16:46:22.968218 2014-08-13 17:38:02.976914 7fe9bae887a0 20 read_log 192888'11019 (192888'11018) modify 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.1315259.0:22847 2014-08-11 16:46:22.960256 2014-08-13 17:38:02.976926 7fe9bae887a0 20 read_log 192888'11022 (0'0) promote 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by osd.367.0:52163 2014-08-11 16:46:30.118991 2014-08-13 17:38:02.976938 7fe9bae887a0 20 read_log 192888'11023 (192888'11022) modify 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.1315259.0:24651 2014-08-11 16:46:30.111221 2014-08-13 17:38:02.976950 7fe9bae887a0 20 read_log 192888'11024 (0'0) promote 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.367.0:52167 2014-08-11 16:46:30.438201 2014-08-13 17:38:02.976963 7fe9bae887a0 20 read_log 192888'11025 (192888'11024) modify 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by client.1315259.0:24717 2014-08-11 16:46:30.430340 2014-08-13 17:38:02.976975 7fe9bae887a0 20 read_log 192888'11026 (0'0) promote c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by osd.367.0:52177 2014-08-11 16:46:32.623601 2014-08-13 17:38:02.976987 7fe9bae887a0 20 read_log 192888'11027 (192888'11026) modify c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.1315259.0:25272 2014-08-11 16:46:32.612928 2014-08-13 17:38:02.977008 7fe9bae887a0 20 read_log 192888'11028 (0'0) promote 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.367.0:52189 2014-08-11 16:46:33.560470 2014-08-13 17:38:02.977021 7fe9bae887a0 20 read_log 192888'11029 (192888'11028) modify 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by client.1315259.0:25508 2014-08-11 16:46:33.551832 2014-08-13 17:38:02.977033 7fe9bae887a0 20 read_log 192888'11032 (0'0) promote bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by osd.367.0:52213 2014-08-11 16:46:36.286931 2014-08-13 17:38:02.977045 7fe9bae887a0 20 read_log 192888'11033 (192888'11032) modify bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.1315259.0:26129 2014-08-11 16:46:36.277747 2014-08-13 17:38:02.977057 7fe9bae887a0 20 read_log 192888'11034 (0'0) promote 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.367.0:52245 2014-08-11 16:46:40.911730 2014-08-13 17:38:02.977069 7fe9bae887a0 20 read_log 192888'11035 (192888'11034) modify 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by client.1315259.0:27216 2014-08-11 16:46:40.903077 2014-08-13 17:38:02.977081 7fe9bae887a0 20 read_log 192888'11036 (0'0) promote ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.367.0:52267 2014-08-11 16:46:42.482224 2014-08-13 17:38:02.977092 7fe9bae887a0 20 read_log 192888'11037 (192888'11036) modify ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by client.1315259.0:27621 2014-08-11 16:46:42.473539 2014-08-13 17:38:02.977104 7fe9bae887a0 20 read_log 192888'11038 (0'0) promote 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by osd.367.0:52273 2014-08-11 16:46:42.793603 2014-08-13 17:38:02.977116 7fe9bae887a0 20 read_log 192888'11039 (192888'11038) modify 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.1315259.0:27700 2014-08-11 16:46:42.785176 2014-08-13 17:38:02.977128 7fe9bae887a0 20 read_log 192888'11054 (192888'10945) modify ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.1315259.0:33231 2014-08-11 16:46:56.584382 2014-08-13 17:38:02.977140 7fe9bae887a0 20 read_log 192888'11055 (192888'10947) modify 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.1315259.0:33320 2014-08-11 16:46:56.607744 2014-08-13 17:38:02.977152 7fe9bae887a0 20 read_log 192888'11057 (192888'10951) modify a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by client.1315259.0:33872 2014-08-11 16:46:56.746650 2014-08-13 17:38:02.977169 7fe9bae887a0 20 read_log 192888'11059 (192888'10955) modify 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.1315259.0:35458 2014-08-11 16:46:57.168188 2014-08-13 17:38:02.977181 7fe9bae887a0 20 read_log 192888'11063 (192888'10963) modify bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.1315259.0:37239 2014-08-11 16:46:57.630888 2014-08-13 17:38:02.977193 7fe9bae887a0 20 read_log 192888'11070 (192888'10977) modify 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.1315259.0:41113 2014-08-11 16:46:58.661659 2014-08-13 17:38:02.977205 7fe9bae887a0 20 read_log 192888'11073 (192888'10983) modify a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.1315259.0:44325 2014-08-11 16:46:59.546653 2014-08-13 17:38:02.977217 7fe9bae887a0 20 read_log 192888'11074 (192888'10985) modify cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.1315259.0:45085 2014-08-11 16:46:59.765559 2014-08-13 17:38:02.977229 7fe9bae887a0 20 read_log 192888'11075 (192888'10987) modify 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.1315259.0:45484 2014-08-11 16:46:59.883476 2014-08-13 17:38:02.977241 7fe9bae887a0 20 read_log 192888'11076 (192888'10989) modify 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.1315259.0:45530 2014-08-11 16:46:59.895663 2014-08-13 17:38:02.977261 7fe9bae887a0 20 read_log 192888'11078 (192888'10993) modify 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by client.1315259.0:46813 2014-08-11 16:47:00.265180 2014-08-13 17:38:02.977273 7fe9bae887a0 20 read_log 192888'11079 (192888'10995) modify 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.1315259.0:46951 2014-08-11 16:47:00.305221 2014-08-13 17:38:02.977285 7fe9bae887a0 20 read_log 192888'11080 (192888'10997) modify 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.1315259.0:47468 2014-08-11 16:47:00.466660 2014-08-13 17:38:02.977297 7fe9bae887a0 20 read_log 192888'11082 (192888'11001) modify 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.1315259.0:48119 2014-08-11 16:47:00.675237 2014-08-13 17:38:02.977309 7fe9bae887a0 20 read_log 192888'11085 (192888'11007) modify d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.1315259.0:49239 2014-08-11 16:47:00.999326 2014-08-13 17:38:02.977321 7fe9bae887a0 20 read_log 192888'11087 (192888'11011) modify 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.1315259.0:49297 2014-08-11 16:47:01.018475 2014-08-13 17:38:02.977333 7fe9bae887a0 20 read_log 192888'11089 (192888'11015) modify c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.1315259.0:50443 2014-08-11 16:47:01.371381 2014-08-13 17:38:02.977345 7fe9bae887a0 20 read_log 192888'11090 (192888'11017) modify e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.1315259.0:51503 2014-08-11 16:47:01.698788 2014-08-13 17:38:02.977357 7fe9bae887a0 20 read_log 192888'11091 (192888'11019) modify 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.1315259.0:53574 2014-08-11 16:47:02.294503 2014-08-13 17:38:02.977369 7fe9bae887a0 20 read_log 192888'11093 (192888'11023) modify 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.1315259.0:55378 2014-08-11 16:47:02.894661 2014-08-13 17:38:02.977380 7fe9bae887a0 20 read_log 192888'11094 (192888'11025) modify 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by client.1315259.0:55444 2014-08-11 16:47:02.914255 2014-08-13 17:38:02.977392 7fe9bae887a0 20 read_log 192888'11095 (192888'11027) modify c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.1315259.0:55999 2014-08-11 16:47:03.076849 2014-08-13 17:38:02.977404 7fe9bae887a0 20 read_log 192888'11096 (192888'11029) modify 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by client.1315259.0:56235 2014-08-11 16:47:03.143353 2014-08-13 17:38:02.977416 7fe9bae887a0 20 read_log 192888'11098 (192888'11033) modify bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.1315259.0:56856 2014-08-11 16:47:03.334345 2014-08-13 17:38:02.977428 7fe9bae887a0 20 read_log 192888'11099 (192888'11035) modify 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by client.1315259.0:57943 2014-08-11 16:47:03.657181 2014-08-13 17:38:02.977440 7fe9bae887a0 20 read_log 192888'11100 (192888'11037) modify ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by client.1315259.0:58348 2014-08-11 16:47:03.775170 2014-08-13 17:38:02.977452 7fe9bae887a0 20 read_log 192888'11101 (192888'11039) modify 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.1315259.0:58427 2014-08-11 16:47:03.799058 2014-08-13 17:38:02.977464 7fe9bae887a0 20 read_log 192888'11109 (0'0) promote e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.367.0:52824 2014-08-11 17:05:16.231403 2014-08-13 17:38:02.977475 7fe9bae887a0 20 read_log 192888'11110 (192888'11109) modify e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by client.942868.0:302 2014-08-11 17:05:16.221594 2014-08-13 17:38:02.977488 7fe9bae887a0 20 read_log 192888'11111 (0'0) promote 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.367.0:52837 2014-08-11 17:05:17.299470 2014-08-13 17:38:02.977509 7fe9bae887a0 20 read_log 192888'11112 (192888'11111) modify 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by client.942868.0:555 2014-08-11 17:05:17.291487 2014-08-13 17:38:02.977522 7fe9bae887a0 20 read_log 192888'11113 (0'0) promote 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by osd.367.0:52844 2014-08-11 17:05:18.652499 2014-08-13 17:38:02.977534 7fe9bae887a0 20 read_log 192888'11114 (192888'11113) modify 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.942868.0:887 2014-08-11 17:05:18.645175 2014-08-13 17:38:02.977546 7fe9bae887a0 20 read_log 192888'11127 (0'0) promote fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.367.0:52981 2014-08-11 17:05:33.694708 2014-08-13 17:38:02.977558 7fe9bae887a0 20 read_log 192888'11128 (192888'11127) modify fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by client.942868.0:4784 2014-08-11 17:05:33.687569 2014-08-13 17:38:02.977570 7fe9bae887a0 20 read_log 192888'11131 (0'0) promote afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.367.0:52985 2014-08-11 17:05:33.944768 2014-08-13 17:38:02.977582 7fe9bae887a0 20 read_log 192888'11132 (192888'11131) modify afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by client.942868.0:4845 2014-08-11 17:05:33.937089 2014-08-13 17:38:02.977594 7fe9bae887a0 20 read_log 192888'11135 (0'0) promote 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by osd.367.0:53015 2014-08-11 17:05:36.841503 2014-08-13 17:38:02.977606 7fe9bae887a0 20 read_log 192888'11136 (192888'11135) modify 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.942868.0:5594 2014-08-11 17:05:36.834546 2014-08-13 17:38:02.977618 7fe9bae887a0 20 read_log 192888'11137 (0'0) promote 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.367.0:53035 2014-08-11 17:05:40.002577 2014-08-13 17:38:02.977632 7fe9bae887a0 20 read_log 192888'11138 (192888'11137) modify 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by client.942868.0:6467 2014-08-11 17:05:39.995654 2014-08-13 17:38:02.977645 7fe9bae887a0 20 read_log 192888'11139 (0'0) promote b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.367.0:53051 2014-08-11 17:05:41.085115 2014-08-13 17:38:02.977657 7fe9bae887a0 20 read_log 192888'11140 (192888'11139) modify b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by client.942868.0:6768 2014-08-11 17:05:41.078382 2014-08-13 17:38:02.977669 7fe9bae887a0 20 read_log 192888'11141 (0'0) promote 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by osd.367.0:53055 2014-08-11 17:05:42.131591 2014-08-13 17:38:02.977681 7fe9bae887a0 20 read_log 192888'11142 (192888'11141) modify 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.942868.0:7026 2014-08-11 17:05:42.122543 2014-08-13 17:38:02.977693 7fe9bae887a0 20 read_log 192888'11143 (0'0) promote 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by osd.367.0:53079 2014-08-11 17:05:46.272201 2014-08-13 17:38:02.977705 7fe9bae887a0 20 read_log 192888'11144 (192888'11143) modify 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.942868.0:8124 2014-08-11 17:05:46.265733 2014-08-13 17:38:02.977717 7fe9bae887a0 20 read_log 192888'11145 (0'0) promote 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by osd.367.0:53095 2014-08-11 17:05:48.222282 2014-08-13 17:38:02.977737 7fe9bae887a0 20 read_log 192888'11146 (192888'11145) modify 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.942868.0:8663 2014-08-11 17:05:48.214940 2014-08-13 17:38:02.977753 7fe9bae887a0 20 read_log 192888'11149 (0'0) promote 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.367.0:53135 2014-08-11 17:05:52.729959 2014-08-13 17:38:02.977794 7fe9bae887a0 20 read_log 192888'11150 (192888'11149) modify 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by client.942868.0:9868 2014-08-11 17:05:52.723736 2014-08-13 17:38:02.977808 7fe9bae887a0 20 read_log 192888'11153 (0'0) promote 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.367.0:53167 2014-08-11 17:05:57.277656 2014-08-13 17:38:02.977820 7fe9bae887a0 20 read_log 192888'11154 (192888'11153) modify 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by client.942868.0:11127 2014-08-11 17:05:57.270410 2014-08-13 17:38:02.977832 7fe9bae887a0 20 read_log 192888'11159 (0'0) promote 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.367.0:53241 2014-08-11 17:06:07.286353 2014-08-13 17:38:02.977844 7fe9bae887a0 20 read_log 192888'11160 (192888'11159) modify 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by client.942868.0:13780 2014-08-11 17:06:07.278093 2014-08-13 17:38:02.977856 7fe9bae887a0 20 read_log 192888'11161 (0'0) promote 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by osd.367.0:53253 2014-08-11 17:06:08.970062 2014-08-13 17:38:02.977868 7fe9bae887a0 20 read_log 192888'11162 (192888'11161) modify 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.942868.0:14180 2014-08-11 17:06:08.957099 2014-08-13 17:38:02.977880 7fe9bae887a0 20 read_log 192888'11175 (0'0) promote f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by osd.367.0:53383 2014-08-11 17:06:29.098136 2014-08-13 17:38:02.977892 7fe9bae887a0 20 read_log 192888'11176 (192888'11175) modify f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.942868.0:19399 2014-08-11 17:06:29.090335 2014-08-13 17:38:02.977905 7fe9bae887a0 20 read_log 192888'11177 (0'0) promote b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by osd.367.0:53395 2014-08-11 17:06:30.044232 2014-08-13 17:38:02.977916 7fe9bae887a0 20 read_log 192888'11178 (192888'11177) modify b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.942868.0:19652 2014-08-11 17:06:30.037503 2014-08-13 17:38:02.977944 7fe9bae887a0 20 read_log 192888'11179 (0'0) promote 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by osd.367.0:53411 2014-08-11 17:06:33.351894 2014-08-13 17:38:02.977957 7fe9bae887a0 20 read_log 192888'11180 (192888'11179) modify 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.942868.0:20513 2014-08-11 17:06:33.345372 2014-08-13 17:38:02.977969 7fe9bae887a0 20 read_log 192888'11183 (0'0) promote b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by osd.367.0:53439 2014-08-11 17:06:36.530468 2014-08-13 17:38:02.977981 7fe9bae887a0 20 read_log 192888'11184 (192888'11183) modify b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.942868.0:21343 2014-08-11 17:06:36.522598 2014-08-13 17:38:02.977993 7fe9bae887a0 20 read_log 192888'11185 (0'0) promote 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by osd.367.0:53449 2014-08-11 17:06:37.908131 2014-08-13 17:38:02.978004 7fe9bae887a0 20 read_log 192888'11186 (192888'11185) modify 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.942868.0:21715 2014-08-11 17:06:37.901376 2014-08-13 17:38:02.978017 7fe9bae887a0 20 read_log 192888'11191 (0'0) promote e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by osd.367.0:53463 2014-08-11 17:06:38.767926 2014-08-13 17:38:02.978028 7fe9bae887a0 20 read_log 192888'11192 (192888'11191) modify e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.942868.0:21944 2014-08-11 17:06:38.761155 2014-08-13 17:38:02.978040 7fe9bae887a0 20 read_log 192888'11197 (0'0) promote 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by osd.367.0:53499 2014-08-11 17:06:44.300325 2014-08-13 17:38:02.978059 7fe9bae887a0 20 read_log 192888'11198 (192888'11197) modify 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.942868.0:23345 2014-08-11 17:06:44.293222 2014-08-13 17:38:02.978072 7fe9bae887a0 20 read_log 192888'11199 (0'0) promote ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by osd.367.0:53521 2014-08-11 17:06:46.922675 2014-08-13 17:38:02.978084 7fe9bae887a0 20 read_log 192888'11200 (192888'11199) modify ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.942868.0:24057 2014-08-11 17:06:46.914526 2014-08-13 17:38:02.978096 7fe9bae887a0 20 read_log 192888'11201 (0'0) promote a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by osd.367.0:53527 2014-08-11 17:06:47.407458 2014-08-13 17:38:02.978108 7fe9bae887a0 20 read_log 192888'11202 (192888'11201) modify a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.942868.0:24188 2014-08-11 17:06:47.399811 2014-08-13 17:38:02.978120 7fe9bae887a0 20 read_log 192888'11205 (0'0) promote da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.367.0:53569 2014-08-11 17:06:51.614893 2014-08-13 17:38:02.978132 7fe9bae887a0 20 read_log 192888'11206 (192888'11205) modify da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by client.942868.0:25316 2014-08-11 17:06:51.607465 2014-08-13 17:38:02.978144 7fe9bae887a0 20 read_log 192888'11207 (0'0) promote 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by osd.367.0:53573 2014-08-11 17:06:51.824881 2014-08-13 17:38:02.978160 7fe9bae887a0 20 read_log 192888'11208 (192888'11207) modify 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.942868.0:25372 2014-08-11 17:06:51.817380 2014-08-13 17:38:02.978215 7fe9bae887a0 20 read_log 192888'11209 (0'0) promote d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by osd.367.0:53595 2014-08-11 17:06:53.832892 2014-08-13 17:38:02.978228 7fe9bae887a0 20 read_log 192888'11210 (192888'11209) modify d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.942868.0:25853 2014-08-11 17:06:53.825462 2014-08-13 17:38:02.978244 7fe9bae887a0 20 read_log 192888'11211 (0'0) promote 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.367.0:53613 2014-08-11 17:06:55.829649 2014-08-13 17:38:02.978256 7fe9bae887a0 20 read_log 192888'11212 (192888'11211) modify 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by client.942868.0:26369 2014-08-11 17:06:55.822495 2014-08-13 17:38:02.978269 7fe9bae887a0 20 read_log 192888'11213 (0'0) promote d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by osd.367.0:53633 2014-08-11 17:06:58.615402 2014-08-13 17:38:02.978281 7fe9bae887a0 20 read_log 192888'11214 (192888'11213) modify d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.942868.0:27106 2014-08-11 17:06:58.608483 2014-08-13 17:38:02.978293 7fe9bae887a0 20 read_log 192888'11219 (0'0) promote 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.367.0:53671 2014-08-11 17:07:02.579471 2014-08-13 17:38:02.978305 7fe9bae887a0 20 read_log 192888'11220 (192888'11219) modify 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by client.942868.0:28159 2014-08-11 17:07:02.569063 2014-08-13 17:38:02.978317 7fe9bae887a0 20 read_log 192888'11223 (0'0) promote fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.367.0:53683 2014-08-11 17:07:04.075975 2014-08-13 17:38:02.978329 7fe9bae887a0 20 read_log 192888'11224 (192888'11223) modify fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by client.942868.0:28565 2014-08-11 17:07:04.068114 2014-08-13 17:38:02.978341 7fe9bae887a0 20 read_log 192888'11225 (0'0) promote dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by osd.367.0:53711 2014-08-11 17:07:06.773078 2014-08-13 17:38:02.978362 7fe9bae887a0 20 read_log 192888'11226 (192888'11225) modify dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.942868.0:29252 2014-08-11 17:07:06.765594 2014-08-13 17:38:02.978375 7fe9bae887a0 20 read_log 192888'11227 (192888'11110) modify e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by client.942868.0:31568 2014-08-11 17:07:15.975062 2014-08-13 17:38:02.978387 7fe9bae887a0 20 read_log 192888'11228 (192888'11112) modify 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by client.942868.0:31821 2014-08-11 17:07:16.067687 2014-08-13 17:38:02.978399 7fe9bae887a0 20 read_log 192888'11229 (192888'11114) modify 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.942868.0:32153 2014-08-11 17:07:16.168980 2014-08-13 17:38:02.978411 7fe9bae887a0 20 read_log 192888'11236 (192888'11128) modify fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by client.942868.0:36050 2014-08-11 17:07:17.229791 2014-08-13 17:38:02.978423 7fe9bae887a0 20 read_log 192888'11238 (192888'11132) modify afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by client.942868.0:36111 2014-08-11 17:07:17.245184 2014-08-13 17:38:02.978435 7fe9bae887a0 20 read_log 192888'11240 (192888'11136) modify 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.942868.0:36860 2014-08-11 17:07:17.452605 2014-08-13 17:38:02.978449 7fe9bae887a0 20 read_log 192888'11241 (192888'11138) modify 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by client.942868.0:37733 2014-08-11 17:07:17.693147 2014-08-13 17:38:02.978461 7fe9bae887a0 20 read_log 192888'11242 (192888'11140) modify b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by client.942868.0:38034 2014-08-11 17:07:17.774082 2014-08-13 17:38:02.978474 7fe9bae887a0 20 read_log 192888'11243 (192888'11142) modify 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.942868.0:38292 2014-08-11 17:07:17.844766 2014-08-13 17:38:02.978486 7fe9bae887a0 20 read_log 192888'11244 (192888'11144) modify 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.942868.0:39390 2014-08-11 17:07:18.168110 2014-08-13 17:38:02.978498 7fe9bae887a0 20 read_log 192888'11245 (192888'11146) modify 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.942868.0:39929 2014-08-11 17:07:18.320540 2014-08-13 17:38:02.978510 7fe9bae887a0 20 read_log 192888'11247 (192888'11150) modify 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by client.942868.0:41134 2014-08-11 17:07:18.651489 2014-08-13 17:38:02.978522 7fe9bae887a0 20 read_log 192888'11249 (192888'11154) modify 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by client.942868.0:42393 2014-08-11 17:07:18.985753 2014-08-13 17:38:02.978534 7fe9bae887a0 20 read_log 192888'11252 (192888'11160) modify 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by client.942868.0:45046 2014-08-11 17:07:19.732475 2014-08-13 17:38:02.978546 7fe9bae887a0 20 read_log 192888'11253 (192888'11162) modify 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.942868.0:45446 2014-08-11 17:07:19.840005 2014-08-13 17:38:02.978559 7fe9bae887a0 20 read_log 192888'11260 (192888'11176) modify f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.942868.0:50665 2014-08-11 17:07:21.394326 2014-08-13 17:38:02.978570 7fe9bae887a0 20 read_log 192888'11261 (192888'11178) modify b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.942868.0:50918 2014-08-11 17:07:21.472939 2014-08-13 17:38:02.978582 7fe9bae887a0 20 read_log 192888'11262 (192888'11180) modify 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.942868.0:51779 2014-08-11 17:07:21.750058 2014-08-13 17:38:02.978594 7fe9bae887a0 20 read_log 192888'11264 (192888'11184) modify b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.942868.0:52609 2014-08-11 17:07:21.994603 2014-08-13 17:38:02.978616 7fe9bae887a0 20 read_log 192888'11265 (192888'11186) modify 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.942868.0:52981 2014-08-11 17:07:22.112001 2014-08-13 17:38:02.978629 7fe9bae887a0 20 read_log 192888'11268 (192888'11192) modify e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.942868.0:53210 2014-08-11 17:07:22.184497 2014-08-13 17:38:02.978641 7fe9bae887a0 20 read_log 192888'11271 (192888'11198) modify 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.942868.0:54611 2014-08-11 17:07:22.608337 2014-08-13 17:38:02.978653 7fe9bae887a0 20 read_log 192888'11272 (192888'11200) modify ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.942868.0:55323 2014-08-11 17:07:22.812649 2014-08-13 17:38:02.978665 7fe9bae887a0 20 read_log 192888'11273 (192888'11202) modify a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.942868.0:55454 2014-08-11 17:07:22.851943 2014-08-13 17:38:02.978677 7fe9bae887a0 20 read_log 192888'11275 (192888'11206) modify da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by client.942868.0:56582 2014-08-11 17:07:23.198619 2014-08-13 17:38:02.978689 7fe9bae887a0 20 read_log 192888'11276 (192888'11208) modify 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.942868.0:56638 2014-08-11 17:07:23.215162 2014-08-13 17:38:02.978701 7fe9bae887a0 20 read_log 192888'11277 (192888'11210) modify d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.942868.0:57119 2014-08-11 17:07:23.362588 2014-08-13 17:38:02.978713 7fe9bae887a0 20 read_log 192888'11278 (192888'11212) modify 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by client.942868.0:57635 2014-08-11 17:07:23.512804 2014-08-13 17:38:02.978725 7fe9bae887a0 20 read_log 192888'11279 (192888'11214) modify d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.942868.0:58372 2014-08-11 17:07:23.724071 2014-08-13 17:38:02.978737 7fe9bae887a0 20 read_log 192888'11282 (192888'11220) modify 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by client.942868.0:59425 2014-08-11 17:07:24.042484 2014-08-13 17:38:02.978749 7fe9bae887a0 20 read_log 192888'11284 (192888'11224) modify fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by client.942868.0:59831 2014-08-11 17:07:24.163865 2014-08-13 17:38:02.978761 7fe9bae887a0 20 read_log 192888'11285 (192888'11226) modify dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.942868.0:60518 2014-08-11 17:07:24.362534 2014-08-13 17:38:02.978773 7fe9bae887a0 20 read_log 193305'11286 (0'0) promote 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by osd.15.0:46 2014-08-11 18:42:48.054685 2014-08-13 17:38:02.978785 7fe9bae887a0 20 read_log 193305'11287 (193305'11286) modify 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.951337.0:4008 2014-08-11 18:42:48.047800 2014-08-13 17:38:02.978796 7fe9bae887a0 20 read_log 193305'11288 (0'0) promote ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by osd.15.0:50 2014-08-11 18:42:49.702963 2014-08-13 17:38:02.978808 7fe9bae887a0 20 read_log 193305'11289 (193305'11288) modify ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.951337.0:4450 2014-08-11 18:42:49.695709 2014-08-13 17:38:02.978820 7fe9bae887a0 20 read_log 193305'11290 (0'0) promote 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by osd.15.0:52 2014-08-11 18:42:50.765268 2014-08-13 17:38:02.978832 7fe9bae887a0 20 read_log 193305'11291 (193305'11290) modify 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.951337.0:4741 2014-08-11 18:42:50.758489 2014-08-13 17:38:02.978844 7fe9bae887a0 20 read_log 193305'11292 (0'0) promote 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by osd.15.0:80 2014-08-11 18:43:05.103704 2014-08-13 17:38:02.978862 7fe9bae887a0 20 read_log 193305'11293 (193305'11292) modify 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.951337.0:8606 2014-08-11 18:43:05.096233 2014-08-13 17:38:02.978875 7fe9bae887a0 20 read_log 193305'11294 (0'0) promote 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by osd.15.0:86 2014-08-11 18:43:07.446727 2014-08-13 17:38:02.978887 7fe9bae887a0 20 read_log 193305'11295 (193305'11294) modify 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.951337.0:9225 2014-08-11 18:43:07.439410 2014-08-13 17:38:02.978899 7fe9bae887a0 20 read_log 193305'11296 (0'0) promote 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:102 2014-08-11 18:43:13.926684 2014-08-13 17:38:02.978911 7fe9bae887a0 20 read_log 193305'11297 (193305'11296) modify 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by client.951337.0:10978 2014-08-11 18:43:13.918710 2014-08-13 17:38:02.978923 7fe9bae887a0 20 read_log 193305'11298 (0'0) promote e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by osd.15.0:108 2014-08-11 18:43:16.197733 2014-08-13 17:38:02.978935 7fe9bae887a0 20 read_log 193305'11299 (193305'11298) modify e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.951337.0:11540 2014-08-11 18:43:16.189296 2014-08-13 17:38:02.978947 7fe9bae887a0 20 read_log 193305'11300 (0'0) promote 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by osd.15.0:118 2014-08-11 18:43:21.372339 2014-08-13 17:38:02.978959 7fe9bae887a0 20 read_log 193305'11301 (193305'11300) modify 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.951337.0:12928 2014-08-11 18:43:21.365399 2014-08-13 17:38:02.978971 7fe9bae887a0 20 read_log 193305'11302 (0'0) promote 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by osd.15.0:120 2014-08-11 18:43:21.484409 2014-08-13 17:38:02.978983 7fe9bae887a0 20 read_log 193305'11303 (193305'11302) modify 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.951337.0:12959 2014-08-11 18:43:21.477648 2014-08-13 17:38:02.978995 7fe9bae887a0 20 read_log 193305'11304 (0'0) promote d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by osd.15.0:124 2014-08-11 18:43:21.959908 2014-08-13 17:38:02.979006 7fe9bae887a0 20 read_log 193305'11305 (193305'11304) modify d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.951337.0:13090 2014-08-11 18:43:21.952537 2014-08-13 17:38:02.979018 7fe9bae887a0 20 read_log 193305'11306 (0'0) promote d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by osd.15.0:128 2014-08-11 18:43:22.963397 2014-08-13 17:38:02.979030 7fe9bae887a0 20 read_log 193305'11307 (193305'11306) modify d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.951337.0:13358 2014-08-11 18:43:22.956467 2014-08-13 17:38:02.979042 7fe9bae887a0 20 read_log 193305'11308 (0'0) promote 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:130 2014-08-11 18:43:23.082095 2014-08-13 17:38:02.979054 7fe9bae887a0 20 read_log 193305'11309 (193305'11308) modify 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by client.951337.0:13388 2014-08-11 18:43:23.075749 2014-08-13 17:38:02.979066 7fe9bae887a0 20 read_log 193305'11310 (0'0) promote 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:134 2014-08-11 18:43:23.587497 2014-08-13 17:38:02.979078 7fe9bae887a0 20 read_log 193305'11311 (193305'11310) modify 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by client.951337.0:13507 2014-08-11 18:43:23.581360 2014-08-13 17:38:02.979090 7fe9bae887a0 20 read_log 193305'11312 (0'0) promote 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by osd.15.0:138 2014-08-11 18:43:23.786903 2014-08-13 17:38:02.979109 7fe9bae887a0 20 read_log 193305'11313 (193305'11312) modify 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.951337.0:13561 2014-08-11 18:43:23.780761 2014-08-13 17:38:02.979121 7fe9bae887a0 20 read_log 193305'11314 (0'0) promote 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by osd.15.0:142 2014-08-11 18:43:29.273991 2014-08-13 17:38:02.979133 7fe9bae887a0 20 read_log 193305'11315 (193305'11314) modify 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.951337.0:15088 2014-08-11 18:43:29.262485 2014-08-13 17:38:02.979145 7fe9bae887a0 20 read_log 193305'11316 (0'0) promote 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by osd.15.0:156 2014-08-11 18:43:31.888423 2014-08-13 17:38:02.979182 7fe9bae887a0 20 read_log 193305'11317 (193305'11316) modify 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.951337.0:15860 2014-08-11 18:43:31.880732 2014-08-13 17:38:02.979195 7fe9bae887a0 20 read_log 193305'11318 (0'0) promote 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:158 2014-08-11 18:43:32.071092 2014-08-13 17:38:02.979208 7fe9bae887a0 20 read_log 193305'11319 (193305'11318) modify 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by client.951337.0:15914 2014-08-11 18:43:32.062261 2014-08-13 17:38:02.979220 7fe9bae887a0 20 read_log 193305'11320 (0'0) promote 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by osd.15.0:164 2014-08-11 18:43:33.065977 2014-08-13 17:38:02.979232 7fe9bae887a0 20 read_log 193305'11321 (193305'11320) modify 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.951337.0:16190 2014-08-11 18:43:33.058240 2014-08-13 17:38:02.979244 7fe9bae887a0 20 read_log 193305'11322 (0'0) promote 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by osd.15.0:170 2014-08-11 18:43:34.565791 2014-08-13 17:38:02.979256 7fe9bae887a0 20 read_log 193305'11323 (193305'11322) modify 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.951337.0:16613 2014-08-11 18:43:34.557413 2014-08-13 17:38:02.979268 7fe9bae887a0 20 read_log 193305'11324 (0'0) promote 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:186 2014-08-11 18:43:41.249832 2014-08-13 17:38:02.979281 7fe9bae887a0 20 read_log 193305'11325 (193305'11324) modify 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by client.951337.0:18535 2014-08-11 18:43:41.242579 2014-08-13 17:38:02.979293 7fe9bae887a0 20 read_log 193305'11326 (0'0) promote c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:188 2014-08-11 18:43:41.517078 2014-08-13 17:38:02.979305 7fe9bae887a0 20 read_log 193305'11327 (193305'11326) modify c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by client.951337.0:18614 2014-08-11 18:43:41.509413 2014-08-13 17:38:02.979318 7fe9bae887a0 20 read_log 193305'11328 (0'0) promote b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by osd.15.0:192 2014-08-11 18:43:42.544271 2014-08-13 17:38:02.979330 7fe9bae887a0 20 read_log 193305'11329 (193305'11328) modify b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.951337.0:18893 2014-08-11 18:43:42.535377 2014-08-13 17:38:02.979342 7fe9bae887a0 20 read_log 193305'11330 (0'0) promote bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by osd.15.0:198 2014-08-11 18:43:44.018122 2014-08-13 17:38:02.979354 7fe9bae887a0 20 read_log 193305'11331 (193305'11330) modify bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.951337.0:19270 2014-08-11 18:43:44.009958 2014-08-13 17:38:02.979366 7fe9bae887a0 20 read_log 193305'11332 (0'0) promote 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:202 2014-08-11 18:43:44.214961 2014-08-13 17:38:02.979387 7fe9bae887a0 20 read_log 193305'11333 (193305'11332) modify 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by client.951337.0:19317 2014-08-11 18:43:44.209037 2014-08-13 17:38:02.979400 7fe9bae887a0 20 read_log 193305'11334 (0'0) promote ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by osd.15.0:222 2014-08-11 18:43:50.875402 2014-08-13 17:38:02.979412 7fe9bae887a0 20 read_log 193305'11335 (193305'11334) modify ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.951337.0:21183 2014-08-11 18:43:50.866318 2014-08-13 17:38:02.979424 7fe9bae887a0 20 read_log 193305'11336 (0'0) promote f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:228 2014-08-11 18:43:53.999903 2014-08-13 17:38:02.979436 7fe9bae887a0 20 read_log 193305'11337 (193305'11336) modify f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by client.951337.0:22056 2014-08-11 18:43:53.993100 2014-08-13 17:38:02.979448 7fe9bae887a0 20 read_log 193305'11338 (0'0) promote 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:230 2014-08-11 18:43:55.168467 2014-08-13 17:38:02.979462 7fe9bae887a0 20 read_log 193305'11339 (193305'11338) modify 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by client.951337.0:22369 2014-08-11 18:43:55.160101 2014-08-13 17:38:02.979474 7fe9bae887a0 20 read_log 193305'11340 (0'0) promote 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by osd.15.0:236 2014-08-11 18:43:57.041541 2014-08-13 17:38:02.979486 7fe9bae887a0 20 read_log 193305'11341 (193305'11340) modify 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.951337.0:22884 2014-08-11 18:43:57.032819 2014-08-13 17:38:02.979498 7fe9bae887a0 20 read_log 193305'11342 (0'0) promote a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:246 2014-08-11 18:43:58.953589 2014-08-13 17:38:02.979510 7fe9bae887a0 20 read_log 193305'11343 (193305'11342) modify a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by client.951337.0:23432 2014-08-11 18:43:58.944233 2014-08-13 17:38:02.979522 7fe9bae887a0 20 read_log 193305'11344 (0'0) promote 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by osd.15.0:248 2014-08-11 18:43:59.211798 2014-08-13 17:38:02.979534 7fe9bae887a0 20 read_log 193305'11345 (193305'11344) modify 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.951337.0:23500 2014-08-11 18:43:59.203263 2014-08-13 17:38:02.979546 7fe9bae887a0 20 read_log 193305'11346 (0'0) promote c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:256 2014-08-11 18:44:02.863498 2014-08-13 17:38:02.979558 7fe9bae887a0 20 read_log 193305'11347 (193305'11346) modify c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by client.951337.0:24511 2014-08-11 18:44:02.854883 2014-08-13 17:38:02.979570 7fe9bae887a0 20 read_log 193305'11348 (0'0) promote 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by osd.15.0:266 2014-08-11 18:44:05.935682 2014-08-13 17:38:02.979582 7fe9bae887a0 20 read_log 193305'11349 (193305'11348) modify 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.951337.0:25355 2014-08-11 18:44:05.925906 2014-08-13 17:38:02.979594 7fe9bae887a0 20 read_log 193305'11350 (0'0) promote ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by osd.15.0:296 2014-08-11 18:44:17.230935 2014-08-13 17:38:02.979606 7fe9bae887a0 20 read_log 193305'11351 (193305'11350) modify ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.951337.0:28434 2014-08-11 18:44:17.223909 2014-08-13 17:38:02.979618 7fe9bae887a0 20 read_log 193305'11352 (0'0) promote ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by osd.15.0:310 2014-08-11 18:44:19.448066 2014-08-13 17:38:02.979654 7fe9bae887a0 20 read_log 193305'11353 (193305'11352) modify ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.951337.0:29029 2014-08-11 18:44:19.440514 2014-08-13 17:38:02.979670 7fe9bae887a0 20 read_log 193305'11354 (0'0) promote 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:324 2014-08-11 18:44:23.592672 2014-08-13 17:38:02.979683 7fe9bae887a0 20 read_log 193305'11355 (193305'11354) modify 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by client.951337.0:30140 2014-08-11 18:44:23.584266 2014-08-13 17:38:02.979695 7fe9bae887a0 20 read_log 193305'11356 (0'0) promote 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by osd.15.0:328 2014-08-11 18:44:24.557511 2014-08-13 17:38:02.979707 7fe9bae887a0 20 read_log 193305'11357 (193305'11356) modify 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.951337.0:30386 2014-08-11 18:44:24.550381 2014-08-13 17:38:02.979718 7fe9bae887a0 20 read_log 193305'11358 (0'0) promote c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:336 2014-08-11 18:44:26.288653 2014-08-13 17:38:02.979730 7fe9bae887a0 20 read_log 193305'11359 (193305'11358) modify c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by client.951337.0:30848 2014-08-11 18:44:26.279327 2014-08-13 17:38:02.979742 7fe9bae887a0 20 read_log 193305'11360 (0'0) promote e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by osd.15.0:342 2014-08-11 18:44:26.975458 2014-08-13 17:38:02.979754 7fe9bae887a0 20 read_log 193305'11361 (193305'11360) modify e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.951337.0:31043 2014-08-11 18:44:26.966134 2014-08-13 17:38:02.979766 7fe9bae887a0 20 read_log 193305'11362 (193305'11287) modify 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.951337.0:36719 2014-08-11 18:44:36.489446 2014-08-13 17:38:02.979778 7fe9bae887a0 20 read_log 193305'11363 (193305'11289) modify ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.951337.0:37161 2014-08-11 18:44:36.742046 2014-08-13 17:38:02.979790 7fe9bae887a0 20 read_log 193305'11364 (193305'11291) modify 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.951337.0:37452 2014-08-11 18:44:36.919572 2014-08-13 17:38:02.979803 7fe9bae887a0 20 read_log 193305'11365 (193305'11293) modify 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.951337.0:41317 2014-08-11 18:44:38.716308 2014-08-13 17:38:02.979814 7fe9bae887a0 20 read_log 193305'11366 (193305'11295) modify 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.951337.0:41936 2014-08-11 18:44:38.932609 2014-08-13 17:38:02.979827 7fe9bae887a0 20 read_log 193305'11367 (193305'11297) modify 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by client.951337.0:43689 2014-08-11 18:44:39.666394 2014-08-13 17:38:02.979839 7fe9bae887a0 20 read_log 193305'11368 (193305'11299) modify e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.951337.0:44251 2014-08-11 18:44:39.832782 2014-08-13 17:38:02.979851 7fe9bae887a0 20 read_log 193305'11369 (193305'11301) modify 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.951337.0:45639 2014-08-11 18:44:40.258866 2014-08-13 17:38:02.979863 7fe9bae887a0 20 read_log 193305'11370 (193305'11303) modify 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.951337.0:45670 2014-08-11 18:44:40.267087 2014-08-13 17:38:02.979875 7fe9bae887a0 20 read_log 193305'11371 (193305'11305) modify d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.951337.0:45801 2014-08-11 18:44:40.307624 2014-08-13 17:38:02.979887 7fe9bae887a0 20 read_log 193305'11372 (193305'11307) modify d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.951337.0:46069 2014-08-11 18:44:40.380477 2014-08-13 17:38:02.979905 7fe9bae887a0 20 read_log 193305'11373 (193305'11309) modify 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by client.951337.0:46099 2014-08-11 18:44:40.388204 2014-08-13 17:38:02.979918 7fe9bae887a0 20 read_log 193305'11374 (193305'11311) modify 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by client.951337.0:46218 2014-08-11 18:44:40.422552 2014-08-13 17:38:02.979930 7fe9bae887a0 20 read_log 193305'11375 (193305'11313) modify 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.951337.0:46272 2014-08-11 18:44:40.437100 2014-08-13 17:38:02.979942 7fe9bae887a0 20 read_log 193305'11376 (193305'11315) modify 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.951337.0:47799 2014-08-11 18:44:40.869127 2014-08-13 17:38:02.979954 7fe9bae887a0 20 read_log 193305'11377 (193305'11317) modify 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.951337.0:48571 2014-08-11 18:44:41.091429 2014-08-13 17:38:02.979966 7fe9bae887a0 20 read_log 193305'11378 (193305'11378) modify 20e/hit_set_24.20e_archive_2014-08-11 18:39:56.767178_2014-08-11 18:44:41.106881/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:41.107140 2014-08-13 17:38:02.979979 7fe9bae887a0 20 read_log 193305'11379 (193305'11319) modify 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by client.951337.0:48625 2014-08-11 18:44:41.106326 2014-08-13 17:38:02.979991 7fe9bae887a0 20 read_log 193305'11380 (193305'11321) modify 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.951337.0:48901 2014-08-11 18:44:41.190339 2014-08-13 17:38:02.980003 7fe9bae887a0 20 read_log 193305'11381 (193305'11323) modify 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.951337.0:49324 2014-08-11 18:44:41.330594 2014-08-13 17:38:02.980015 7fe9bae887a0 20 read_log 193305'11382 (193305'11325) modify 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by client.951337.0:51246 2014-08-11 18:44:41.894517 2014-08-13 17:38:02.980027 7fe9bae887a0 20 read_log 193305'11383 (193305'11327) modify c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by client.951337.0:51325 2014-08-11 18:44:41.916226 2014-08-13 17:38:02.980039 7fe9bae887a0 20 read_log 193305'11384 (193305'11329) modify b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.951337.0:51604 2014-08-11 18:44:41.999347 2014-08-13 17:38:02.980050 7fe9bae887a0 20 read_log 193305'11385 (193305'11331) modify bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.951337.0:51981 2014-08-11 18:44:42.102363 2014-08-13 17:38:02.980062 7fe9bae887a0 20 read_log 193305'11386 (193305'11333) modify 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by client.951337.0:52028 2014-08-11 18:44:42.114223 2014-08-13 17:38:02.980075 7fe9bae887a0 20 read_log 193305'11387 (193305'11335) modify ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.951337.0:53894 2014-08-11 18:44:42.667192 2014-08-13 17:38:02.980087 7fe9bae887a0 20 read_log 193305'11388 (193305'11337) modify f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by client.951337.0:54767 2014-08-11 18:44:42.933118 2014-08-13 17:38:02.980099 7fe9bae887a0 20 read_log 193305'11389 (193305'11339) modify 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by client.951337.0:55080 2014-08-11 18:44:43.018625 2014-08-13 17:38:02.980111 7fe9bae887a0 20 read_log 193305'11390 (193305'11341) modify 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.951337.0:55595 2014-08-11 18:44:43.172902 2014-08-13 17:38:02.980123 7fe9bae887a0 20 read_log 193305'11391 (193305'11343) modify a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by client.951337.0:56143 2014-08-11 18:44:43.333511 2014-08-13 17:38:02.980145 7fe9bae887a0 20 read_log 193305'11392 (193305'11345) modify 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.951337.0:56211 2014-08-11 18:44:43.354579 2014-08-13 17:38:02.980163 7fe9bae887a0 20 read_log 193305'11393 (193305'11347) modify c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by client.951337.0:57222 2014-08-11 18:44:43.676753 2014-08-13 17:38:02.980177 7fe9bae887a0 20 read_log 193305'11394 (193305'11349) modify 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.951337.0:58066 2014-08-11 18:44:43.937092 2014-08-13 17:38:02.980190 7fe9bae887a0 20 read_log 193305'11395 (193305'11351) modify ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.951337.0:61145 2014-08-11 18:44:44.835223 2014-08-13 17:38:02.980202 7fe9bae887a0 20 read_log 193305'11396 (193305'11353) modify ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.951337.0:61740 2014-08-11 18:44:45.015364 2014-08-13 17:38:02.980214 7fe9bae887a0 20 read_log 193305'11397 (193305'11355) modify 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by client.951337.0:62851 2014-08-11 18:44:45.343961 2014-08-13 17:38:02.980226 7fe9bae887a0 20 read_log 193305'11398 (193305'11357) modify 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.951337.0:63097 2014-08-11 18:44:45.408425 2014-08-13 17:38:02.980244 7fe9bae887a0 20 read_log 193305'11399 (193305'11359) modify c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by client.951337.0:63559 2014-08-11 18:44:45.543292 2014-08-13 17:38:02.980256 7fe9bae887a0 20 read_log 193305'11400 (193305'11361) modify e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.951337.0:63754 2014-08-11 18:44:45.599149 2014-08-13 17:38:02.980268 7fe9bae887a0 20 read_log 194209'11401 (192888'11236) clean fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.15.0:775 2014-08-12 14:07:13.508568 2014-08-13 17:38:02.980280 7fe9bae887a0 20 read_log 194209'11402 (193305'11388) clean f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:776 2014-08-12 14:07:13.509140 2014-08-13 17:38:02.980292 7fe9bae887a0 20 read_log 194209'11403 (193305'11382) clean 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:777 2014-08-12 14:07:13.509583 2014-08-13 17:38:02.980304 7fe9bae887a0 20 read_log 194209'11404 (192834'10278) clean c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.15.0:778 2014-08-12 14:07:13.510067 2014-08-13 17:38:02.980316 7fe9bae887a0 20 read_log 194209'11405 (192834'10825) clean d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.15.0:779 2014-08-12 14:07:13.520780 2014-08-13 17:38:02.980328 7fe9bae887a0 20 read_log 194209'11406 (192888'11099) clean 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.15.0:780 2014-08-12 14:07:13.521054 2014-08-13 17:38:02.980340 7fe9bae887a0 20 read_log 194209'11407 (193305'11399) clean c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:781 2014-08-12 14:07:13.523929 2014-08-13 17:38:02.980352 7fe9bae887a0 20 read_log 194209'11408 (192888'11241) clean 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.15.0:782 2014-08-12 14:07:13.527062 2014-08-13 17:38:02.980364 7fe9bae887a0 20 read_log 194209'11409 (193305'11373) clean 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:783 2014-08-12 14:07:13.530489 2014-08-13 17:38:02.980393 7fe9bae887a0 20 read_log 194209'11410 (192834'10818) clean c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.15.0:784 2014-08-12 14:07:13.533540 2014-08-13 17:38:02.980413 7fe9bae887a0 20 read_log 194209'11411 (193305'11397) clean 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:785 2014-08-12 14:07:13.536595 2014-08-13 17:38:02.980426 7fe9bae887a0 20 read_log 194209'11412 (192888'11238) clean afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.15.0:786 2014-08-12 14:07:13.539602 2014-08-13 17:38:02.980438 7fe9bae887a0 20 read_log 194209'11413 (192834'10877) clean 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.15.0:787 2014-08-12 14:07:13.542717 2014-08-13 17:38:02.980450 7fe9bae887a0 20 read_log 194209'11414 (192834'10289) clean 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.15.0:788 2014-08-12 14:07:13.545965 2014-08-13 17:38:02.980462 7fe9bae887a0 20 read_log 194209'11415 (192888'11078) clean 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.15.0:789 2014-08-12 14:07:13.549035 2014-08-13 17:38:02.980473 7fe9bae887a0 20 read_log 194209'11416 (192834'10749) clean 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.15.0:790 2014-08-12 14:07:13.552077 2014-08-13 17:38:02.980485 7fe9bae887a0 20 read_log 194209'11417 (192888'11284) clean fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.15.0:791 2014-08-12 14:07:13.555170 2014-08-13 17:38:02.980497 7fe9bae887a0 20 read_log 194209'11418 (192888'11282) clean 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.15.0:792 2014-08-12 14:07:13.558286 2014-08-13 17:38:02.980509 7fe9bae887a0 20 read_log 194209'11419 (192888'11247) clean 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.15.0:793 2014-08-12 14:07:13.561518 2014-08-13 17:38:02.980521 7fe9bae887a0 20 read_log 194209'11420 (192834'10219) clean a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.15.0:794 2014-08-12 14:07:13.564502 2014-08-13 17:38:02.980533 7fe9bae887a0 20 read_log 194209'11421 (192888'11278) clean 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.15.0:795 2014-08-12 14:07:13.567278 2014-08-13 17:38:02.980545 7fe9bae887a0 20 read_log 194209'11422 (192888'11227) clean e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.15.0:796 2014-08-12 14:07:13.570324 2014-08-13 17:38:02.980557 7fe9bae887a0 20 read_log 194209'11423 (192834'10763) clean fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.15.0:797 2014-08-12 14:07:13.573380 2014-08-13 17:38:02.980569 7fe9bae887a0 20 read_log 194209'11424 (192888'11275) clean da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.15.0:798 2014-08-12 14:07:13.576481 2014-08-13 17:38:02.980581 7fe9bae887a0 20 read_log 194209'11425 (192834'10221) clean 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.15.0:799 2014-08-12 14:07:13.579897 2014-08-13 17:38:02.980593 7fe9bae887a0 20 read_log 194209'11426 (192834'10751) clean 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.15.0:800 2014-08-12 14:07:13.582798 2014-08-13 17:38:02.980605 7fe9bae887a0 20 read_log 194209'11427 (192888'11228) clean 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.15.0:801 2014-08-12 14:07:13.585929 2014-08-13 17:38:02.980617 7fe9bae887a0 20 read_log 194209'11428 (192834'10897) clean 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.15.0:802 2014-08-12 14:07:13.589011 2014-08-13 17:38:02.980628 7fe9bae887a0 20 read_log 194209'11429 (193305'11386) clean 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:803 2014-08-12 14:07:13.592169 2014-08-13 17:38:02.980640 7fe9bae887a0 20 read_log 194209'11430 (192834'10819) clean 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.15.0:804 2014-08-12 14:07:13.595319 2014-08-13 17:38:02.980660 7fe9bae887a0 20 read_log 194209'11431 (193305'11379) clean 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:805 2014-08-12 14:07:13.598700 2014-08-13 17:38:02.980673 7fe9bae887a0 20 read_log 194209'11432 (193305'11367) clean 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:806 2014-08-12 14:07:13.601399 2014-08-13 17:38:02.980686 7fe9bae887a0 20 read_log 194209'11433 (192888'11094) clean 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.15.0:807 2014-08-12 14:07:13.604360 2014-08-13 17:38:02.980700 7fe9bae887a0 20 read_log 194209'11434 (193305'11393) clean c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:808 2014-08-12 14:07:13.608100 2014-08-13 17:38:02.980712 7fe9bae887a0 20 read_log 194209'11435 (193305'11383) clean c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:809 2014-08-12 14:07:13.611578 2014-08-13 17:38:02.980724 7fe9bae887a0 20 read_log 194209'11436 (193305'11389) clean 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:810 2014-08-12 14:07:13.615255 2014-08-13 17:38:02.980736 7fe9bae887a0 20 read_log 194209'11437 (192888'11057) clean a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.15.0:811 2014-08-12 14:07:13.618592 2014-08-13 17:38:02.980748 7fe9bae887a0 20 read_log 194209'11438 (192888'11242) clean b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.15.0:812 2014-08-12 14:07:13.622045 2014-08-13 17:38:02.980759 7fe9bae887a0 20 read_log 194209'11439 (193305'11391) clean a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:813 2014-08-12 14:07:13.625416 2014-08-13 17:38:02.980771 7fe9bae887a0 20 read_log 194209'11440 (192834'10924) clean 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.15.0:814 2014-08-12 14:07:13.628971 2014-08-13 17:38:02.980783 7fe9bae887a0 20 read_log 194209'11441 (192888'11249) clean 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.15.0:815 2014-08-12 14:07:13.632462 2014-08-13 17:38:02.980795 7fe9bae887a0 20 read_log 194209'11442 (192834'10191) clean 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.15.0:816 2014-08-12 14:07:13.635702 2014-08-13 17:38:02.980807 7fe9bae887a0 20 read_log 194209'11443 (192834'10298) clean 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.15.0:817 2014-08-12 14:07:13.639361 2014-08-13 17:38:02.980819 7fe9bae887a0 20 read_log 194209'11444 (193305'11374) clean 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:818 2014-08-12 14:07:13.642685 2014-08-13 17:38:02.980831 7fe9bae887a0 20 read_log 194209'11445 (192888'11252) clean 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.15.0:819 2014-08-12 14:07:13.646067 2014-08-13 17:38:02.980843 7fe9bae887a0 20 read_log 194209'11446 (192888'11100) clean ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.15.0:820 2014-08-12 14:07:13.649838 2014-08-13 17:38:02.980855 7fe9bae887a0 20 read_log 194209'11447 (194209'11401) delete fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.15.0:821 2014-08-12 14:07:13.686184 2014-08-13 17:38:02.980866 7fe9bae887a0 20 read_log 194209'11448 (194209'11402) delete f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:822 2014-08-12 14:07:13.689564 2014-08-13 17:38:02.980878 7fe9bae887a0 20 read_log 194209'11449 (194209'11403) delete 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:823 2014-08-12 14:07:13.692683 2014-08-13 17:38:02.980894 7fe9bae887a0 20 read_log 194209'11450 (194209'11407) delete c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:824 2014-08-12 14:07:13.696172 2014-08-13 17:38:02.980916 7fe9bae887a0 20 read_log 194209'11451 (194209'11411) delete 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:825 2014-08-12 14:07:13.699730 2014-08-13 17:38:02.980929 7fe9bae887a0 20 read_log 194209'11452 (194209'11429) delete 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:826 2014-08-12 14:07:13.707082 2014-08-13 17:38:02.980941 7fe9bae887a0 20 read_log 194209'11453 (194209'11431) delete 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:827 2014-08-12 14:07:13.710323 2014-08-13 17:38:02.980953 7fe9bae887a0 20 read_log 194209'11454 (194209'11434) delete c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:828 2014-08-12 14:07:13.713637 2014-08-13 17:38:02.980965 7fe9bae887a0 20 read_log 194209'11455 (194209'11435) delete c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:829 2014-08-12 14:07:13.716569 2014-08-13 17:38:02.980977 7fe9bae887a0 20 read_log 194209'11456 (194209'11436) delete 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:830 2014-08-12 14:07:13.719984 2014-08-13 17:38:02.980988 7fe9bae887a0 20 read_log 194209'11457 (194209'11439) delete a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:831 2014-08-12 14:07:13.723451 2014-08-13 17:38:02.981000 7fe9bae887a0 20 read_log 194209'11458 (192888'11096) clean 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.15.0:832 2014-08-12 14:07:13.796523 2014-08-13 17:38:02.981012 7fe9bae887a0 20 read_log 194209'11459 (194209'11425) delete 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.15.0:833 2014-08-12 14:07:18.799293 2014-08-13 17:38:02.981024 7fe9bae887a0 20 read_log 194209'11460 (194209'11426) delete 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.15.0:834 2014-08-12 14:07:18.799731 2014-08-13 17:38:02.981036 7fe9bae887a0 20 read_log 194209'11461 (194209'11427) delete 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.15.0:835 2014-08-12 14:07:18.800157 2014-08-13 17:38:02.981048 7fe9bae887a0 20 read_log 194209'11462 (194209'11428) delete 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.15.0:836 2014-08-12 14:07:18.800534 2014-08-13 17:38:02.981059 7fe9bae887a0 20 read_log 194209'11463 (194209'11430) delete 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.15.0:837 2014-08-12 14:07:18.803704 2014-08-13 17:38:02.981071 7fe9bae887a0 20 read_log 194209'11464 (194209'11432) delete 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:838 2014-08-12 14:07:18.804135 2014-08-13 17:38:02.981083 7fe9bae887a0 20 read_log 194209'11465 (194209'11433) delete 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.15.0:839 2014-08-12 14:07:18.805226 2014-08-13 17:38:02.981095 7fe9bae887a0 20 read_log 194209'11466 (194209'11437) delete a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.15.0:840 2014-08-12 14:07:18.808615 2014-08-13 17:38:02.981107 7fe9bae887a0 20 read_log 194209'11467 (194209'11438) delete b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.15.0:841 2014-08-12 14:07:18.809008 2014-08-13 17:38:02.981118 7fe9bae887a0 20 read_log 194209'11468 (194209'11440) delete 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.15.0:842 2014-08-12 14:07:18.809931 2014-08-13 17:38:02.981130 7fe9bae887a0 20 read_log 194209'11469 (194209'11441) delete 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.15.0:843 2014-08-12 14:07:18.810309 2014-08-13 17:38:02.981142 7fe9bae887a0 20 read_log 194209'11470 (194209'11442) delete 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.15.0:844 2014-08-12 14:07:18.813682 2014-08-13 17:38:02.981166 7fe9bae887a0 20 read_log 194209'11471 (194209'11443) delete 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.15.0:845 2014-08-12 14:07:18.814103 2014-08-13 17:38:02.981179 7fe9bae887a0 20 read_log 194209'11472 (194209'11444) delete 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:846 2014-08-12 14:07:18.814852 2014-08-13 17:38:02.981191 7fe9bae887a0 20 read_log 194209'11473 (194209'11445) delete 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.15.0:847 2014-08-12 14:07:18.815208 2014-08-13 17:38:02.981203 7fe9bae887a0 20 read_log 194209'11474 (194209'11446) delete ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.15.0:848 2014-08-12 14:07:18.819495 2014-08-13 17:38:02.981215 7fe9bae887a0 20 read_log 194209'11475 (194209'11458) delete 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.15.0:849 2014-08-12 14:07:18.819902 2014-08-13 17:38:02.981227 7fe9bae887a0 20 read_log 194209'11476 (194209'11404) delete c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.15.0:850 2014-08-12 14:07:18.854504 2014-08-13 17:38:02.981239 7fe9bae887a0 20 read_log 194209'11477 (194209'11405) delete d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.15.0:851 2014-08-12 14:07:18.855325 2014-08-13 17:38:02.981251 7fe9bae887a0 20 read_log 194209'11478 (194209'11406) delete 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.15.0:852 2014-08-12 14:07:18.855690 2014-08-13 17:38:02.981262 7fe9bae887a0 20 read_log 194209'11479 (194209'11408) delete 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.15.0:853 2014-08-12 14:07:18.856054 2014-08-13 17:38:02.981274 7fe9bae887a0 20 read_log 194209'11480 (194209'11409) delete 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:854 2014-08-12 14:07:18.859349 2014-08-13 17:38:02.981286 7fe9bae887a0 20 read_log 194209'11481 (194209'11410) delete c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.15.0:855 2014-08-12 14:07:18.859724 2014-08-13 17:38:02.981298 7fe9bae887a0 20 read_log 194209'11482 (194209'11412) delete afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.15.0:856 2014-08-12 14:07:18.860528 2014-08-13 17:38:02.981310 7fe9bae887a0 20 read_log 194209'11483 (194209'11413) delete 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.15.0:857 2014-08-12 14:07:18.860939 2014-08-13 17:38:02.981322 7fe9bae887a0 20 read_log 194209'11484 (194209'11414) delete 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.15.0:858 2014-08-12 14:07:18.863104 2014-08-13 17:38:02.981333 7fe9bae887a0 20 read_log 194209'11485 (194209'11415) delete 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.15.0:859 2014-08-12 14:07:18.863843 2014-08-13 17:38:02.981345 7fe9bae887a0 20 read_log 194209'11486 (194209'11416) delete 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.15.0:860 2014-08-12 14:07:18.864219 2014-08-13 17:38:02.981357 7fe9bae887a0 20 read_log 194209'11487 (194209'11417) delete fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.15.0:861 2014-08-12 14:07:18.864563 2014-08-13 17:38:02.981369 7fe9bae887a0 20 read_log 194209'11488 (194209'11418) delete 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.15.0:862 2014-08-12 14:07:18.868470 2014-08-13 17:38:02.981381 7fe9bae887a0 20 read_log 194209'11489 (194209'11419) delete 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.15.0:863 2014-08-12 14:07:18.868930 2014-08-13 17:38:02.981393 7fe9bae887a0 20 read_log 194209'11490 (194209'11420) delete a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.15.0:864 2014-08-12 14:07:18.869732 2014-08-13 17:38:02.981430 7fe9bae887a0 20 read_log 194209'11491 (194209'11421) delete 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.15.0:865 2014-08-12 14:07:18.870116 2014-08-13 17:38:02.981444 7fe9bae887a0 20 read_log 194209'11492 (194209'11422) delete e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.15.0:866 2014-08-12 14:07:18.872639 2014-08-13 17:38:02.981457 7fe9bae887a0 20 read_log 194209'11493 (194209'11423) delete fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.15.0:867 2014-08-12 14:07:18.872927 2014-08-13 17:38:02.981469 7fe9bae887a0 20 read_log 194209'11494 (194209'11424) delete da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.15.0:868 2014-08-12 14:07:18.873733 2014-08-13 17:38:02.981481 7fe9bae887a0 20 read_log 194215'11495 (194215'11495) modify 20e/hit_set_24.20e_archive_2014-08-12 15:32:25.779939_2014-08-12 15:44:23.690307/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:23.690778 2014-08-13 17:38:02.981493 7fe9bae887a0 20 read_log 194215'11496 (193305'11378) delete 20e/hit_set_24.20e_archive_2014-08-11 18:39:56.767178_2014-08-11 18:44:41.106881/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:23.690778 2014-08-13 17:38:02.981506 7fe9bae887a0 20 read_log 194215'11497 (193305'11365) clean 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by osd.15.0:1653 2014-08-12 15:44:23.695696 2014-08-13 17:38:02.981518 7fe9bae887a0 20 read_log 194215'11498 (194215'11497) delete 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.2174595.0:108597 0.000000 2014-08-13 17:38:02.981529 7fe9bae887a0 20 read_log 194215'11499 (193305'11380) clean 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by osd.15.0:1655 2014-08-12 15:44:23.703442 2014-08-13 17:38:02.981541 7fe9bae887a0 20 read_log 194215'11500 (194215'11499) delete 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.2174595.0:108599 0.000000 2014-08-13 17:38:02.981552 7fe9bae887a0 20 read_log 194215'11501 (193305'11394) clean 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by osd.15.0:1657 2014-08-12 15:44:23.711235 2014-08-13 17:38:02.981564 7fe9bae887a0 20 read_log 194215'11502 (194215'11501) delete 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.2174595.0:108601 0.000000 2014-08-13 17:38:02.981595 7fe9bae887a0 20 read_log 194215'11503 (192834'10854) clean c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by osd.15.0:1659 2014-08-12 15:44:23.718791 2014-08-13 17:38:02.981608 7fe9bae887a0 20 read_log 194215'11504 (194215'11503) delete c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.2174595.0:108603 0.000000 2014-08-13 17:38:02.981619 7fe9bae887a0 20 read_log 194215'11505 (192888'11080) clean 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by osd.15.0:1661 2014-08-12 15:44:23.727698 2014-08-13 17:38:02.981631 7fe9bae887a0 20 read_log 194215'11506 (194215'11505) delete 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.2174595.0:108605 0.000000 2014-08-13 17:38:02.981642 7fe9bae887a0 20 read_log 194215'11507 (192888'11273) clean a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by osd.15.0:1663 2014-08-12 15:44:23.735669 2014-08-13 17:38:02.981654 7fe9bae887a0 20 read_log 194215'11508 (194215'11507) delete a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.2174595.0:108607 0.000000 2014-08-13 17:38:02.981664 7fe9bae887a0 20 read_log 194215'11509 (192888'11243) clean 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by osd.15.0:1665 2014-08-12 15:44:23.743510 2014-08-13 17:38:02.981686 7fe9bae887a0 20 read_log 194215'11510 (194215'11509) delete 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.2174595.0:108609 0.000000 2014-08-13 17:38:02.981697 7fe9bae887a0 20 read_log 194215'11511 (192888'11089) clean c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by osd.15.0:1667 2014-08-12 15:44:23.751776 2014-08-13 17:38:02.981710 7fe9bae887a0 20 read_log 194215'11512 (194215'11511) delete c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.2174595.0:108611 0.000000 2014-08-13 17:38:02.981720 7fe9bae887a0 20 read_log 194215'11513 (193305'11384) clean b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by osd.15.0:1669 2014-08-12 15:44:23.759007 2014-08-13 17:38:02.981732 7fe9bae887a0 20 read_log 194215'11514 (194215'11513) delete b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.2174595.0:108613 0.000000 2014-08-13 17:38:02.981743 7fe9bae887a0 20 read_log 194215'11515 (192888'11075) clean 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by osd.15.0:1671 2014-08-12 15:44:23.766640 2014-08-13 17:38:02.981755 7fe9bae887a0 20 read_log 194215'11516 (194215'11515) delete 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.2174595.0:108615 0.000000 2014-08-13 17:38:02.981766 7fe9bae887a0 20 read_log 194215'11517 (192888'11229) clean 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by osd.15.0:1673 2014-08-12 15:44:23.774186 2014-08-13 17:38:02.981779 7fe9bae887a0 20 read_log 194215'11518 (194215'11517) delete 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.2174595.0:108617 0.000000 2014-08-13 17:38:02.981790 7fe9bae887a0 20 read_log 194215'11519 (193305'11369) clean 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by osd.15.0:1675 2014-08-12 15:44:23.781559 2014-08-13 17:38:02.981802 7fe9bae887a0 20 read_log 194215'11520 (194215'11519) delete 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.2174595.0:108619 0.000000 2014-08-13 17:38:02.981812 7fe9bae887a0 20 read_log 194215'11521 (193305'11390) clean 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by osd.15.0:1677 2014-08-12 15:44:23.789860 2014-08-13 17:38:02.981824 7fe9bae887a0 20 read_log 194215'11522 (194215'11521) delete 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.2174595.0:108621 0.000000 2014-08-13 17:38:02.981835 7fe9bae887a0 20 read_log 194215'11523 (192888'11059) clean 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by osd.15.0:1679 2014-08-12 15:44:23.797328 2014-08-13 17:38:02.981847 7fe9bae887a0 20 read_log 194215'11524 (194215'11523) delete 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.2174595.0:108623 0.000000 2014-08-13 17:38:02.981857 7fe9bae887a0 20 read_log 194215'11525 (192834'10786) clean 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by osd.15.0:1681 2014-08-12 15:44:23.806196 2014-08-13 17:38:02.981869 7fe9bae887a0 20 read_log 194215'11526 (194215'11525) delete 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.2174595.0:108625 0.000000 2014-08-13 17:38:02.981880 7fe9bae887a0 20 read_log 194215'11527 (192834'10271) clean ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by osd.15.0:1683 2014-08-12 15:44:23.813212 2014-08-13 17:38:02.981892 7fe9bae887a0 20 read_log 194215'11528 (194215'11527) delete ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.2174595.0:108627 0.000000 2014-08-13 17:38:02.981902 7fe9bae887a0 20 read_log 194215'11529 (193305'11396) clean ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by osd.15.0:1685 2014-08-12 15:44:23.820878 2014-08-13 17:38:02.981921 7fe9bae887a0 20 read_log 194215'11530 (194215'11529) delete ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.2174595.0:108629 0.000000 2014-08-13 17:38:02.981932 7fe9bae887a0 20 read_log 194215'11531 (192888'11090) clean e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by osd.15.0:1687 2014-08-12 15:44:23.828834 2014-08-13 17:38:02.981944 7fe9bae887a0 20 read_log 194215'11532 (194215'11531) delete e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.2174595.0:108631 0.000000 2014-08-13 17:38:02.981955 7fe9bae887a0 20 read_log 194215'11533 (192834'10192) clean b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by osd.15.0:1689 2014-08-12 15:44:23.836042 2014-08-13 17:38:02.981967 7fe9bae887a0 20 read_log 194215'11534 (194215'11533) delete b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.2174595.0:108633 0.000000 2014-08-13 17:38:02.981977 7fe9bae887a0 20 read_log 194215'11535 (192888'11261) clean b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by osd.15.0:1691 2014-08-12 15:44:23.844244 2014-08-13 17:38:02.982007 7fe9bae887a0 20 read_log 194215'11536 (194215'11535) delete b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.2174595.0:108635 0.000000 2014-08-13 17:38:02.982019 7fe9bae887a0 20 read_log 194215'11537 (192888'11268) clean e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by osd.15.0:1693 2014-08-12 15:44:23.852194 2014-08-13 17:38:02.982032 7fe9bae887a0 20 read_log 194215'11538 (194215'11537) delete e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.2174595.0:108637 0.000000 2014-08-13 17:38:02.982042 7fe9bae887a0 20 read_log 194215'11539 (192888'11054) clean ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by osd.15.0:1695 2014-08-12 15:44:23.859365 2014-08-13 17:38:02.982054 7fe9bae887a0 20 read_log 194215'11540 (194215'11539) delete ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.2174595.0:108639 0.000000 2014-08-13 17:38:02.982065 7fe9bae887a0 20 read_log 194215'11541 (193305'11392) clean 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by osd.15.0:1697 2014-08-12 15:44:23.867263 2014-08-13 17:38:02.982077 7fe9bae887a0 20 read_log 194215'11542 (194215'11541) delete 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.2174595.0:108641 0.000000 2014-08-13 17:38:02.982087 7fe9bae887a0 20 read_log 194215'11543 (192888'11277) clean d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by osd.15.0:1699 2014-08-12 15:44:23.874398 2014-08-13 17:38:02.982099 7fe9bae887a0 20 read_log 194215'11544 (194215'11543) delete d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.2174595.0:108643 0.000000 2014-08-13 17:38:02.982109 7fe9bae887a0 20 read_log 194215'11545 (192834'10787) clean 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by osd.15.0:1701 2014-08-12 15:44:23.881815 2014-08-13 17:38:02.982121 7fe9bae887a0 20 read_log 194215'11546 (194215'11545) delete 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.2174595.0:108645 0.000000 2014-08-13 17:38:02.982132 7fe9bae887a0 20 read_log 194215'11547 (192888'11260) clean f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by osd.15.0:1703 2014-08-12 15:44:23.890787 2014-08-13 17:38:02.982144 7fe9bae887a0 20 read_log 194215'11548 (194215'11547) delete f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.2174595.0:108647 0.000000 2014-08-13 17:38:02.982159 7fe9bae887a0 20 read_log 194215'11549 (193305'11362) clean 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by osd.15.0:1705 2014-08-12 15:44:23.898094 2014-08-13 17:38:02.982179 7fe9bae887a0 20 read_log 194215'11550 (194215'11549) delete 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.2174595.0:108649 0.000000 2014-08-13 17:38:02.982190 7fe9bae887a0 20 read_log 194215'11551 (193305'11368) clean e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by osd.15.0:1707 2014-08-12 15:44:23.905932 2014-08-13 17:38:02.982202 7fe9bae887a0 20 read_log 194215'11552 (194215'11551) delete e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.2174595.0:108651 0.000000 2014-08-13 17:38:02.982213 7fe9bae887a0 20 read_log 194215'11553 (192888'11276) clean 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by osd.15.0:1709 2014-08-12 15:44:23.913994 2014-08-13 17:38:02.982225 7fe9bae887a0 20 read_log 194215'11554 (194215'11553) delete 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.2174595.0:108653 0.000000 2014-08-13 17:38:02.982235 7fe9bae887a0 20 read_log 194215'11555 (192888'11079) clean 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by osd.15.0:1711 2014-08-12 15:44:23.921971 2014-08-13 17:38:02.982247 7fe9bae887a0 20 read_log 194215'11556 (194215'11555) delete 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.2174595.0:108655 0.000000 2014-08-13 17:38:02.982258 7fe9bae887a0 20 read_log 194215'11557 (192834'10865) clean abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by osd.15.0:1713 2014-08-12 15:44:23.929997 2014-08-13 17:38:02.982269 7fe9bae887a0 20 read_log 194215'11558 (194215'11557) delete abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.2174595.0:108657 0.000000 2014-08-13 17:38:02.982280 7fe9bae887a0 20 read_log 194215'11559 (192834'10265) clean 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by osd.15.0:1715 2014-08-12 15:44:23.937568 2014-08-13 17:38:02.982292 7fe9bae887a0 20 read_log 194215'11560 (194215'11559) delete 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.2174595.0:108659 0.000000 2014-08-13 17:38:02.982302 7fe9bae887a0 20 read_log 194215'11561 (192834'10866) clean 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by osd.15.0:1717 2014-08-12 15:44:23.945625 2014-08-13 17:38:02.982314 7fe9bae887a0 20 read_log 194215'11562 (194215'11561) delete 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.2174595.0:108661 0.000000 2014-08-13 17:38:02.982325 7fe9bae887a0 20 read_log 194215'11563 (192888'11271) clean 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by osd.15.0:1719 2014-08-12 15:44:23.952559 2014-08-13 17:38:02.982339 7fe9bae887a0 20 read_log 194215'11564 (194215'11563) delete 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.2174595.0:108663 0.000000 2014-08-13 17:38:02.982349 7fe9bae887a0 20 read_log 194215'11565 (192888'11063) clean bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by osd.15.0:1721 2014-08-12 15:44:23.959569 2014-08-13 17:38:02.982361 7fe9bae887a0 20 read_log 194215'11566 (194215'11565) delete bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.2174595.0:108665 0.000000 2014-08-13 17:38:02.982372 7fe9bae887a0 20 read_log 194215'11567 (192888'11082) clean 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by osd.15.0:1723 2014-08-12 15:44:23.966555 2014-08-13 17:38:02.982385 7fe9bae887a0 20 read_log 194215'11568 (194215'11567) delete 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.2174595.0:108667 0.000000 2014-08-13 17:38:02.982396 7fe9bae887a0 20 read_log 194215'11569 (193305'11363) clean ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by osd.15.0:1725 2014-08-12 15:44:23.974109 2014-08-13 17:38:02.982417 7fe9bae887a0 20 read_log 194215'11570 (194215'11569) delete ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.2174595.0:108669 0.000000 2014-08-13 17:38:02.982429 7fe9bae887a0 20 read_log 194215'11571 (192834'10207) clean 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by osd.15.0:1727 2014-08-12 15:44:23.981880 2014-08-13 17:38:02.982441 7fe9bae887a0 20 read_log 194215'11572 (194215'11571) delete 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.2174595.0:108671 0.000000 2014-08-13 17:38:02.982451 7fe9bae887a0 20 read_log 194215'11573 (192888'11265) clean 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by osd.15.0:1729 2014-08-12 15:44:23.989563 2014-08-13 17:38:02.982463 7fe9bae887a0 20 read_log 194215'11574 (194215'11573) delete 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.2174595.0:108673 0.000000 2014-08-13 17:38:02.982474 7fe9bae887a0 20 read_log 194215'11575 (192834'10776) clean 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by osd.15.0:1731 2014-08-12 15:44:23.996440 2014-08-13 17:38:02.982486 7fe9bae887a0 20 read_log 194215'11576 (194215'11575) delete 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.2174595.0:108675 0.000000 2014-08-13 17:38:02.982496 7fe9bae887a0 20 read_log 194215'11577 (192888'11240) clean 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by osd.15.0:1733 2014-08-12 15:44:24.003833 2014-08-13 17:38:02.982508 7fe9bae887a0 20 read_log 194215'11578 (194215'11577) delete 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.2174595.0:108677 0.000000 2014-08-13 17:38:02.982519 7fe9bae887a0 20 read_log 194215'11579 (192834'10288) clean 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by osd.15.0:1735 2014-08-12 15:44:24.011536 2014-08-13 17:38:02.982531 7fe9bae887a0 20 read_log 194215'11580 (194215'11579) delete 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.2174595.0:108679 0.000000 2014-08-13 17:38:02.982541 7fe9bae887a0 20 read_log 194215'11581 (192888'11272) clean ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by osd.15.0:1737 2014-08-12 15:44:24.018670 2014-08-13 17:38:02.982554 7fe9bae887a0 20 read_log 194215'11582 (194215'11581) delete ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.2174595.0:108681 0.000000 2014-08-13 17:38:02.982564 7fe9bae887a0 20 read_log 194215'11583 (193305'11376) clean 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by osd.15.0:1739 2014-08-12 15:44:24.026864 2014-08-13 17:38:02.982576 7fe9bae887a0 20 read_log 194215'11584 (194215'11583) delete 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.2174595.0:108683 0.000000 2014-08-13 17:38:02.982587 7fe9bae887a0 20 read_log 194215'11585 (193305'11370) clean 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by osd.15.0:1741 2014-08-12 15:44:24.034673 2014-08-13 17:38:02.982599 7fe9bae887a0 20 read_log 194215'11586 (194215'11585) delete 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.2174595.0:108685 0.000000 2014-08-13 17:38:02.982609 7fe9bae887a0 20 read_log 194215'11587 (193305'11375) clean 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by osd.15.0:1743 2014-08-12 15:44:24.042017 2014-08-13 17:38:02.982621 7fe9bae887a0 20 read_log 194215'11588 (194215'11587) delete 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.2174595.0:108687 0.000000 2014-08-13 17:38:02.982631 7fe9bae887a0 20 read_log 194215'11589 (193305'11385) clean bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by osd.15.0:1745 2014-08-12 15:44:24.050215 2014-08-13 17:38:02.982650 7fe9bae887a0 20 read_log 194215'11590 (194215'11589) delete bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.2174595.0:108689 0.000000 2014-08-13 17:38:02.982661 7fe9bae887a0 20 read_log 194215'11591 (192834'10804) clean c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by osd.15.0:1747 2014-08-12 15:44:24.057930 2014-08-13 17:38:02.982673 7fe9bae887a0 20 read_log 194215'11592 (194215'11591) delete c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.2174595.0:108691 0.000000 2014-08-13 17:38:02.982684 7fe9bae887a0 20 read_log 194215'11593 (192888'11245) clean 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by osd.15.0:1749 2014-08-12 15:44:24.066067 2014-08-13 17:38:02.982696 7fe9bae887a0 20 read_log 194215'11594 (194215'11593) delete 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.2174595.0:108693 0.000000 2014-08-13 17:38:02.982707 7fe9bae887a0 20 read_log 194215'11595 (192834'10210) clean eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by osd.15.0:1751 2014-08-12 15:44:24.073248 2014-08-13 17:38:02.982737 7fe9bae887a0 20 read_log 194215'11596 (194215'11595) delete eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.2174595.0:108695 0.000000 2014-08-13 17:38:02.982749 7fe9bae887a0 20 read_log 194215'11597 (192888'11087) clean 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by osd.15.0:1753 2014-08-12 15:44:24.080719 2014-08-13 17:38:02.982762 7fe9bae887a0 20 read_log 194215'11598 (194215'11597) delete 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.2174595.0:108697 0.000000 2014-08-13 17:38:02.982773 7fe9bae887a0 20 read_log 194215'11599 (193305'11387) clean ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by osd.15.0:1755 2014-08-12 15:44:24.088721 2014-08-13 17:38:02.982785 7fe9bae887a0 20 read_log 194215'11600 (194215'11599) delete ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.2174595.0:108699 0.000000 2014-08-13 17:38:02.982795 7fe9bae887a0 20 read_log 194215'11601 (192834'10259) clean a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by osd.15.0:1757 2014-08-12 15:44:24.095933 2014-08-13 17:38:02.982807 7fe9bae887a0 20 read_log 194215'11602 (194215'11601) delete a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.2174595.0:108701 0.000000 2014-08-13 17:38:02.982819 7fe9bae887a0 20 read_log 194215'11603 (193305'11366) clean 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by osd.15.0:1759 2014-08-12 15:44:24.104288 2014-08-13 17:38:02.982832 7fe9bae887a0 20 read_log 194215'11604 (194215'11603) delete 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.2174595.0:108703 0.000000 2014-08-13 17:38:02.982842 7fe9bae887a0 20 read_log 194215'11605 (192834'10153) clean b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by osd.15.0:1761 2014-08-12 15:44:24.112055 2014-08-13 17:38:02.982854 7fe9bae887a0 20 read_log 194215'11606 (194215'11605) delete b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.2174595.0:108705 0.000000 2014-08-13 17:38:02.982865 7fe9bae887a0 20 read_log 194215'11607 (192834'10238) clean 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by osd.15.0:1763 2014-08-12 15:44:24.119164 2014-08-13 17:38:02.982877 7fe9bae887a0 20 read_log 194215'11608 (194215'11607) delete 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.2174595.0:108707 0.000000 2014-08-13 17:38:02.982887 7fe9bae887a0 20 read_log 194215'11609 (192888'11285) clean dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by osd.15.0:1765 2014-08-12 15:44:24.127231 2014-08-13 17:38:02.982906 7fe9bae887a0 20 read_log 194215'11610 (194215'11609) delete dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.2174595.0:108709 0.000000 2014-08-13 17:38:02.982917 7fe9bae887a0 20 read_log 194215'11611 (192888'11253) clean 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by osd.15.0:1767 2014-08-12 15:44:24.134463 2014-08-13 17:38:02.982929 7fe9bae887a0 20 read_log 194215'11612 (194215'11611) delete 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.2174595.0:108711 0.000000 2014-08-13 17:38:02.982940 7fe9bae887a0 20 read_log 194215'11613 (192888'11076) clean 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by osd.15.0:1769 2014-08-12 15:44:24.144339 2014-08-13 17:38:02.982952 7fe9bae887a0 20 read_log 194215'11614 (194215'11613) delete 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.2174595.0:108713 0.000000 2014-08-13 17:38:02.982962 7fe9bae887a0 20 read_log 194215'11615 (193305'11364) clean 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by osd.15.0:1771 2014-08-12 15:44:24.152555 2014-08-13 17:38:02.982974 7fe9bae887a0 20 read_log 194215'11616 (194215'11615) delete 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.2174595.0:108715 0.000000 2014-08-13 17:38:02.982984 7fe9bae887a0 20 read_log 194215'11617 (192888'11091) clean 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by osd.15.0:1773 2014-08-12 15:44:24.160405 2014-08-13 17:38:02.982997 7fe9bae887a0 20 read_log 194215'11618 (194215'11617) delete 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.2174595.0:108717 0.000000 2014-08-13 17:38:02.983007 7fe9bae887a0 20 read_log 194215'11619 (192834'10250) clean 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by osd.15.0:1775 2014-08-12 15:44:24.167468 2014-08-13 17:38:02.983019 7fe9bae887a0 20 read_log 194215'11620 (194215'11619) delete 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.2174595.0:108719 0.000000 2014-08-13 17:38:02.983030 7fe9bae887a0 20 read_log 194215'11621 (192888'11101) clean 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by osd.15.0:1777 2014-08-12 15:44:24.175134 2014-08-13 17:38:02.983041 7fe9bae887a0 20 read_log 194215'11622 (194215'11621) delete 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.2174595.0:108721 0.000000 2014-08-13 17:38:02.983052 7fe9bae887a0 20 read_log 194215'11623 (192888'11279) clean d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by osd.15.0:1779 2014-08-12 15:44:24.182625 2014-08-13 17:38:02.983064 7fe9bae887a0 20 read_log 194215'11624 (194215'11623) delete d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.2174595.0:108723 0.000000 2014-08-13 17:38:02.983074 7fe9bae887a0 20 read_log 194215'11625 (192888'11070) clean 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by osd.15.0:1781 2014-08-12 15:44:24.189427 2014-08-13 17:38:02.983086 7fe9bae887a0 20 read_log 194215'11626 (194215'11625) delete 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.2174595.0:108725 0.000000 2014-08-13 17:38:02.983096 7fe9bae887a0 20 read_log 194215'11627 (192834'10741) clean 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by osd.15.0:1783 2014-08-12 15:44:24.197253 2014-08-13 17:38:02.983108 7fe9bae887a0 20 read_log 194215'11628 (194215'11627) delete 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.2174595.0:108727 0.000000 2014-08-13 17:38:02.983119 7fe9bae887a0 20 read_log 194215'11629 (192888'11074) clean cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by osd.15.0:1785 2014-08-12 15:44:24.204418 2014-08-13 17:38:02.983158 7fe9bae887a0 20 read_log 194215'11630 (194215'11629) delete cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.2174595.0:108729 0.000000 2014-08-13 17:38:02.983171 7fe9bae887a0 20 read_log 194215'11631 (192888'11085) clean d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by osd.15.0:1787 2014-08-12 15:44:24.212449 2014-08-13 17:38:02.983183 7fe9bae887a0 20 read_log 194215'11632 (194215'11631) delete d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.2174595.0:108731 0.000000 2014-08-13 17:38:02.983194 7fe9bae887a0 20 read_log 194215'11633 (192834'10925) clean 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by osd.15.0:1789 2014-08-12 15:44:24.220157 2014-08-13 17:38:02.983206 7fe9bae887a0 20 read_log 194215'11634 (194215'11633) delete 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.2174595.0:108733 0.000000 2014-08-13 17:38:02.983217 7fe9bae887a0 20 read_log 194215'11635 (192834'10300) clean b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by osd.15.0:1791 2014-08-12 15:44:24.227125 2014-08-13 17:38:02.983229 7fe9bae887a0 20 read_log 194215'11636 (194215'11635) delete b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.2174595.0:108735 0.000000 2014-08-13 17:38:02.983240 7fe9bae887a0 20 read_log 194215'11637 (192834'10214) clean 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by osd.15.0:1793 2014-08-12 15:44:24.232087 2014-08-13 17:38:02.983252 7fe9bae887a0 20 read_log 194215'11638 (194215'11637) delete 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.2174595.0:108737 0.000000 2014-08-13 17:38:02.983262 7fe9bae887a0 20 read_log 194215'11639 (192888'11098) clean bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by osd.15.0:1795 2014-08-12 15:44:24.238256 2014-08-13 17:38:02.983274 7fe9bae887a0 20 read_log 194215'11640 (194215'11639) delete bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.2174595.0:108739 0.000000 2014-08-13 17:38:02.983284 7fe9bae887a0 20 read_log 194215'11641 (192834'10261) clean ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by osd.15.0:1797 2014-08-12 15:44:24.244038 2014-08-13 17:38:02.983296 7fe9bae887a0 20 read_log 194215'11642 (194215'11641) delete ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.2174595.0:108741 0.000000 2014-08-13 17:38:02.983307 7fe9bae887a0 20 read_log 194215'11643 (193305'11372) clean d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by osd.15.0:1799 2014-08-12 15:44:24.250228 2014-08-13 17:38:02.983319 7fe9bae887a0 20 read_log 194215'11644 (194215'11643) delete d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.2174595.0:108743 0.000000 2014-08-13 17:38:02.983329 7fe9bae887a0 20 read_log 194215'11645 (193305'11371) clean d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by osd.15.0:1801 2014-08-12 15:44:24.256859 2014-08-13 17:38:02.983341 7fe9bae887a0 20 read_log 194215'11646 (194215'11645) delete d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.2174595.0:108745 0.000000 2014-08-13 17:38:02.983352 7fe9bae887a0 20 read_log 194215'11647 (192834'10255) clean b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by osd.15.0:1803 2014-08-12 15:44:24.262535 2014-08-13 17:38:02.983364 7fe9bae887a0 20 read_log 194215'11648 (194215'11647) delete b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.2174595.0:108747 0.000000 2014-08-13 17:38:02.983374 7fe9bae887a0 20 read_log 194215'11649 (192888'11055) clean 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by osd.15.0:1805 2014-08-12 15:44:24.268366 2014-08-13 17:38:02.983394 7fe9bae887a0 20 read_log 194215'11650 (194215'11649) delete 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.2174595.0:108749 0.000000 2014-08-13 17:38:02.983405 7fe9bae887a0 20 read_log 194215'11651 (192888'11095) clean c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by osd.15.0:1807 2014-08-12 15:44:24.274211 2014-08-13 17:38:02.983417 7fe9bae887a0 20 read_log 194215'11652 (194215'11651) delete c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.2174595.0:108751 0.000000 2014-08-13 17:38:02.983428 7fe9bae887a0 20 read_log 194215'11653 (192834'10885) clean 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by osd.15.0:1809 2014-08-12 15:44:24.280454 2014-08-13 17:38:02.983441 7fe9bae887a0 20 read_log 194215'11654 (194215'11653) delete 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.2174595.0:108753 0.000000 2014-08-13 17:38:02.983452 7fe9bae887a0 20 read_log 194215'11655 (193305'11395) clean ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by osd.15.0:1811 2014-08-12 15:44:24.286026 2014-08-13 17:38:02.983464 7fe9bae887a0 20 read_log 194215'11656 (194215'11655) delete ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.2174595.0:108755 0.000000 2014-08-13 17:38:02.983474 7fe9bae887a0 20 read_log 194215'11657 (193305'11398) clean 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by osd.15.0:1813 2014-08-12 15:44:24.291561 2014-08-13 17:38:02.983486 7fe9bae887a0 20 read_log 194215'11658 (194215'11657) delete 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.2174595.0:108757 0.000000 2014-08-13 17:38:02.983497 7fe9bae887a0 20 read_log 194215'11659 (192888'11262) clean 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by osd.15.0:1815 2014-08-12 15:44:24.297457 2014-08-13 17:38:02.983509 7fe9bae887a0 20 read_log 194215'11660 (194215'11659) delete 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.2174595.0:108759 0.000000 2014-08-13 17:38:02.983519 7fe9bae887a0 20 read_log 194215'11661 (193305'11381) clean 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by osd.15.0:1817 2014-08-12 15:44:24.303972 2014-08-13 17:38:02.983531 7fe9bae887a0 20 read_log 194215'11662 (194215'11661) delete 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.2174595.0:108761 0.000000 2014-08-13 17:38:02.983542 7fe9bae887a0 20 read_log 194215'11663 (192888'11264) clean b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by osd.15.0:1819 2014-08-12 15:44:24.309941 2014-08-13 17:38:02.983554 7fe9bae887a0 20 read_log 194215'11664 (194215'11663) delete b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.2174595.0:108763 0.000000 2014-08-13 17:38:02.983564 7fe9bae887a0 20 read_log 194215'11665 (192834'10239) clean 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by osd.15.0:1821 2014-08-12 15:44:24.315737 2014-08-13 17:38:02.983576 7fe9bae887a0 20 read_log 194215'11666 (194215'11665) delete 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.2174595.0:108765 0.000000 2014-08-13 17:38:02.983587 7fe9bae887a0 20 read_log 194215'11667 (193305'11400) clean e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by osd.15.0:1823 2014-08-12 15:44:24.322302 2014-08-13 17:38:02.983599 7fe9bae887a0 20 read_log 194215'11668 (194215'11667) delete e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.2174595.0:108767 0.000000 2014-08-13 17:38:02.983609 7fe9bae887a0 20 read_log 194215'11669 (192834'10755) clean 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by osd.15.0:1825 2014-08-12 15:44:24.328074 2014-08-13 17:38:02.983635 7fe9bae887a0 20 read_log 194215'11670 (194215'11669) delete 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.2174595.0:108769 0.000000 2014-08-13 17:38:02.983647 7fe9bae887a0 20 read_log 194215'11671 (192834'10861) clean a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by osd.15.0:1827 2014-08-12 15:44:24.334123 2014-08-13 17:38:02.983659 7fe9bae887a0 20 read_log 194215'11672 (194215'11671) delete a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.2174595.0:108771 0.000000 2014-08-13 17:38:02.983669 7fe9bae887a0 20 read_log 194215'11673 (192888'11073) clean a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by osd.15.0:1829 2014-08-12 15:44:24.341270 2014-08-13 17:38:02.983681 7fe9bae887a0 20 read_log 194215'11674 (194215'11673) delete a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.2174595.0:108773 0.000000 2014-08-13 17:38:02.983692 7fe9bae887a0 20 read_log 194215'11675 (192834'10844) clean 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by osd.15.0:1831 2014-08-12 15:44:24.349447 2014-08-13 17:38:02.983704 7fe9bae887a0 20 read_log 194215'11676 (194215'11675) delete 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.2174595.0:108775 0.000000 2014-08-13 17:38:02.983714 7fe9bae887a0 20 read_log 194215'11677 (192888'11244) clean 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by osd.15.0:1833 2014-08-12 15:44:24.358034 2014-08-13 17:38:02.983726 7fe9bae887a0 20 read_log 194215'11678 (194215'11677) delete 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.2174595.0:108777 0.000000 2014-08-13 17:38:02.983737 7fe9bae887a0 20 read_log 194215'11679 (192888'11093) clean 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by osd.15.0:1835 2014-08-12 15:44:24.366154 2014-08-13 17:38:02.983749 7fe9bae887a0 20 read_log 194215'11680 (194215'11679) delete 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.2174595.0:108779 0.000000 2014-08-13 17:38:02.983759 7fe9bae887a0 20 read_log 194215'11681 (193305'11377) clean 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by osd.15.0:1837 2014-08-12 15:44:24.373913 2014-08-13 17:38:02.983771 7fe9bae887a0 20 read_log 194215'11682 (194215'11681) delete 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.2174595.0:108781 0.000000 2014-08-13 17:38:02.983782 7fe9bae887a0 20 read_log 194215'11683 (194215'11683) modify 20e/hit_set_24.20e_archive_2014-08-12 15:44:23.690803_2014-08-12 17:24:40.657618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.657832 2014-08-13 17:38:02.983794 7fe9bae887a0 20 read_log 194215'11684 (194215'11495) delete 20e/hit_set_24.20e_archive_2014-08-12 15:32:25.779939_2014-08-12 15:44:23.690307/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.657832 2014-08-13 17:38:02.983807 7fe9bae887a0 20 read_log 194215'11685 (0'0) promote 152960e/benchmark_data_storage0111-ib.pocstor_34628_object194/head//24 by osd.15.0:2353 2014-08-12 17:24:40.660810 2014-08-13 17:38:02.983819 7fe9bae887a0 20 read_log 194215'11686 (194215'11685) modify 152960e/benchmark_data_storage0111-ib.pocstor_34628_object194/head//24 by client.2193759.0:195 2014-08-12 17:24:40.652290 2014-08-13 17:38:02.983831 7fe9bae887a0 20 read_log 194215'11687 (0'0) promote 7c285a0e/benchmark_data_storage0111-ib.pocstor_34628_object1052/head//24 by osd.15.0:2366 2014-08-12 17:24:43.988272 2014-08-13 17:38:02.983843 7fe9bae887a0 20 read_log 194215'11688 (194215'11687) modify 7c285a0e/benchmark_data_storage0111-ib.pocstor_34628_object1052/head//24 by client.2193759.0:1053 2014-08-12 17:24:43.978956 2014-08-13 17:38:02.983885 7fe9bae887a0 20 read_log 194215'11689 (0'0) promote 37487a0e/benchmark_data_storage0111-ib.pocstor_34628_object1082/head//24 by osd.15.0:2368 2014-08-12 17:24:44.101624 2014-08-13 17:38:02.983899 7fe9bae887a0 20 read_log 194215'11690 (194215'11689) modify 37487a0e/benchmark_data_storage0111-ib.pocstor_34628_object1082/head//24 by client.2193759.0:1083 2014-08-12 17:24:44.092775 2014-08-13 17:38:02.983911 7fe9bae887a0 20 read_log 194370'11691 (0'0) promote 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by osd.15.0:2396 2014-08-12 20:30:18.557266 2014-08-13 17:38:02.983923 7fe9bae887a0 20 read_log 194370'11692 (194370'11691) modify 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by client.1508531.0:1119 2014-08-12 20:30:18.549021 2014-08-13 17:38:02.983935 7fe9bae887a0 20 read_log 194370'11693 (0'0) promote 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by osd.15.0:2398 2014-08-12 20:30:19.244249 2014-08-13 17:38:02.983947 7fe9bae887a0 20 read_log 194370'11694 (194370'11693) modify 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by client.1508531.0:1294 2014-08-12 20:30:19.237527 2014-08-13 17:38:02.983959 7fe9bae887a0 20 read_log 194370'11695 (0'0) promote 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by osd.15.0:2404 2014-08-12 20:30:20.404641 2014-08-13 17:38:02.983970 7fe9bae887a0 20 read_log 194370'11696 (194370'11695) modify 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by client.1508531.0:1607 2014-08-12 20:30:20.397467 2014-08-13 17:38:02.983982 7fe9bae887a0 20 read_log 194370'11697 (0'0) promote fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by osd.15.0:2412 2014-08-12 20:30:24.187761 2014-08-13 17:38:02.983994 7fe9bae887a0 20 read_log 194370'11698 (194370'11697) modify fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by client.1508531.0:2626 2014-08-12 20:30:24.180964 2014-08-13 17:38:02.984006 7fe9bae887a0 20 read_log 194370'11699 (0'0) promote e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by osd.15.0:2414 2014-08-12 20:30:24.596818 2014-08-13 17:38:02.984018 7fe9bae887a0 20 read_log 194370'11700 (194370'11699) modify e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by client.1508531.0:2739 2014-08-12 20:30:24.590302 2014-08-13 17:38:02.984030 7fe9bae887a0 20 read_log 194370'11701 (0'0) promote 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by osd.15.0:2458 2014-08-12 20:30:43.168972 2014-08-13 17:38:02.984042 7fe9bae887a0 20 read_log 194370'11702 (194370'11701) modify 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by client.1508531.0:7161 2014-08-12 20:30:43.162041 2014-08-13 17:38:02.984054 7fe9bae887a0 20 read_log 194370'11703 (0'0) promote c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by osd.15.0:2464 2014-08-12 20:30:44.418682 2014-08-13 17:38:02.984065 7fe9bae887a0 20 read_log 194370'11704 (194370'11703) modify c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by client.1508531.0:7411 2014-08-12 20:30:44.411439 2014-08-13 17:38:02.984077 7fe9bae887a0 20 read_log 194370'11705 (0'0) promote 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by osd.15.0:2468 2014-08-12 20:30:44.737234 2014-08-13 17:38:02.984089 7fe9bae887a0 20 read_log 194370'11706 (194370'11705) modify 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by client.1508531.0:7475 2014-08-12 20:30:44.729692 2014-08-13 17:38:02.984101 7fe9bae887a0 20 read_log 194370'11707 (0'0) promote f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by osd.15.0:2474 2014-08-12 20:30:48.231402 2014-08-13 17:38:02.984113 7fe9bae887a0 20 read_log 194370'11708 (194370'11707) modify f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by client.1508531.0:8121 2014-08-12 20:30:48.221225 2014-08-13 17:38:02.984132 7fe9bae887a0 20 read_log 194370'11709 (0'0) promote 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by osd.15.0:2494 2014-08-12 20:30:54.937975 2014-08-13 17:38:02.984144 7fe9bae887a0 20 read_log 194370'11710 (194370'11709) modify 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by client.1508531.0:9267 2014-08-12 20:30:54.930205 2014-08-13 17:38:02.984161 7fe9bae887a0 20 read_log 194370'11711 (0'0) promote d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by osd.15.0:2496 2014-08-12 20:30:55.070463 2014-08-13 17:38:02.984173 7fe9bae887a0 20 read_log 194370'11712 (194370'11711) modify d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by client.1508531.0:9292 2014-08-12 20:30:55.064089 2014-08-13 17:38:02.984185 7fe9bae887a0 20 read_log 194370'11713 (0'0) promote 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by osd.15.0:2498 2014-08-12 20:30:56.513009 2014-08-13 17:38:02.984197 7fe9bae887a0 20 read_log 194370'11714 (194370'11713) modify 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by client.1508531.0:9530 2014-08-12 20:30:56.505775 2014-08-13 17:38:02.984209 7fe9bae887a0 20 read_log 194370'11715 (0'0) promote 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by osd.15.0:2500 2014-08-12 20:30:57.665176 2014-08-13 17:38:02.984223 7fe9bae887a0 20 read_log 194370'11716 (194370'11715) modify 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by client.1508531.0:9723 2014-08-12 20:30:57.657537 2014-08-13 17:38:02.984235 7fe9bae887a0 20 read_log 194370'11717 (0'0) promote 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by osd.15.0:2536 2014-08-12 20:31:20.012680 2014-08-13 17:38:02.984247 7fe9bae887a0 20 read_log 194370'11718 (194370'11717) modify 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by client.1508531.0:13070 2014-08-12 20:31:20.004477 2014-08-13 17:38:02.984259 7fe9bae887a0 20 read_log 194377'11719 (0'0) promote 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by osd.15.0:2566 2014-08-12 20:33:09.516224 2014-08-13 17:38:02.984271 7fe9bae887a0 20 read_log 194377'11720 (194377'11719) modify 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by client.2231190.0:3047 2014-08-12 20:33:09.507056 2014-08-13 17:38:02.984283 7fe9bae887a0 20 read_log 194377'11721 (0'0) promote 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by osd.15.0:2570 2014-08-12 20:33:11.875729 2014-08-13 17:38:02.984295 7fe9bae887a0 20 read_log 194377'11722 (194377'11721) modify 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by client.2231190.0:3601 2014-08-12 20:33:11.866129 2014-08-13 17:38:02.984308 7fe9bae887a0 20 read_log 194377'11723 (0'0) promote c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by osd.15.0:2572 2014-08-12 20:33:13.289171 2014-08-13 17:38:02.984319 7fe9bae887a0 20 read_log 194377'11724 (194377'11723) modify c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by client.2231190.0:3886 2014-08-12 20:33:13.281188 2014-08-13 17:38:02.984331 7fe9bae887a0 20 read_log 194377'11725 (0'0) promote 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by osd.15.0:2578 2014-08-12 20:33:16.143638 2014-08-13 17:38:02.984343 7fe9bae887a0 20 read_log 194377'11726 (194377'11725) modify 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by client.2231190.0:4362 2014-08-12 20:33:16.133686 2014-08-13 17:38:02.984355 7fe9bae887a0 20 read_log 194380'11727 (0'0) promote eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by osd.15.0:2590 2014-08-12 20:33:24.395718 2014-08-13 17:38:02.984376 7fe9bae887a0 20 read_log 194380'11728 (194380'11727) modify eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by client.2231190.0:5098 2014-08-12 20:33:24.385329 2014-08-13 17:38:02.984389 7fe9bae887a0 20 read_log 194383'11729 (0'0) promote bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by osd.15.0:2606 2014-08-12 20:34:03.197149 2014-08-13 17:38:02.984401 7fe9bae887a0 20 read_log 194383'11730 (194383'11729) modify bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by client.2231190.0:6330 2014-08-12 20:34:03.189069 2014-08-13 17:38:02.984413 7fe9bae887a0 20 read_log 194383'11731 (0'0) promote 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by osd.15.0:2610 2014-08-12 20:34:07.051698 2014-08-13 17:38:02.984425 7fe9bae887a0 20 read_log 194383'11732 (194383'11731) modify 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by client.2231190.0:6699 2014-08-12 20:34:07.042697 2014-08-13 17:38:02.984437 7fe9bae887a0 20 read_log 194383'11733 (0'0) promote 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by osd.15.0:2616 2014-08-12 20:34:35.631328 2014-08-13 17:38:02.984451 7fe9bae887a0 20 read_log 194383'11734 (194383'11733) modify 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by client.2231190.0:7747 2014-08-12 20:34:35.621285 2014-08-13 17:38:02.984463 7fe9bae887a0 20 read_log 194393'11735 (194393'11735) modify 20e/hit_set_24.20e_archive_2014-08-12 20:37:08.036878_2014-08-12 20:47:30.630815/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:30.631587 2014-08-13 17:38:02.984476 7fe9bae887a0 20 read_log 194393'11736 (194215'11683) delete 20e/hit_set_24.20e_archive_2014-08-12 15:44:23.690803_2014-08-12 17:24:40.657618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:30.631587 2014-08-13 17:38:02.984488 7fe9bae887a0 20 read_log 194393'11737 (0'0) promote 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by osd.15.0:2633 2014-08-12 20:47:30.636666 2014-08-13 17:38:02.984500 7fe9bae887a0 20 read_log 194393'11738 (194393'11737) modify 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by client.1064095.0:735 2014-08-12 20:47:30.624403 2014-08-13 17:38:02.984512 7fe9bae887a0 20 read_log 194393'11739 (0'0) promote 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by osd.15.0:2637 2014-08-12 20:49:25.445822 2014-08-13 17:38:02.984524 7fe9bae887a0 20 read_log 194393'11740 (194393'11739) modify 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by client.1064182.0:12 2014-08-12 20:49:25.421095 2014-08-13 17:38:02.984536 7fe9bae887a0 20 read_log 194393'11741 (0'0) promote 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by osd.15.0:2645 2014-08-12 20:49:28.267993 2014-08-13 17:38:02.984548 7fe9bae887a0 20 read_log 194393'11742 (194393'11741) modify 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by client.1064182.0:481 2014-08-12 20:49:28.260541 2014-08-13 17:38:02.984560 7fe9bae887a0 20 read_log 194397'11743 (194397'11743) modify 20e/hit_set_24.20e_archive_2014-08-12 20:47:30.631832_2014-08-12 20:53:02.890619/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:02.891332 2014-08-13 17:38:02.984572 7fe9bae887a0 20 read_log 194397'11744 (194393'11735) delete 20e/hit_set_24.20e_archive_2014-08-12 20:37:08.036878_2014-08-12 20:47:30.630815/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:02.891332 2014-08-13 17:38:02.984585 7fe9bae887a0 20 read_log 194397'11745 (0'0) promote f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by osd.15.0:2668 2014-08-12 20:53:02.896821 2014-08-13 17:38:02.984597 7fe9bae887a0 20 read_log 194397'11746 (194397'11745) modify f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by client.2235069.0:705 2014-08-12 20:53:02.883290 2014-08-13 17:38:02.984616 7fe9bae887a0 20 read_log 194397'11747 (0'0) promote 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by osd.15.0:2670 2014-08-12 20:53:03.282378 2014-08-13 17:38:02.984628 7fe9bae887a0 20 read_log 194397'11748 (194397'11747) modify 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by client.2235069.0:755 2014-08-12 20:53:03.272166 2014-08-13 17:38:02.984641 7fe9bae887a0 20 read_log 194416'11749 (194416'11749) modify 20e/hit_set_24.20e_archive_2014-08-12 20:53:02.891413_2014-08-13 15:28:37.371929/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 17:38:02.984653 7fe9bae887a0 20 read_log 194416'11750 (194397'11743) delete 20e/hit_set_24.20e_archive_2014-08-12 20:47:30.631832_2014-08-12 20:53:02.890619/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 17:38:02.984669 7fe9bae887a0 20 read_log 194416'11751 (0'0) promote ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by osd.15.0:2819 2014-08-13 15:28:37.377849 2014-08-13 17:38:02.984681 7fe9bae887a0 20 read_log 194416'11752 (194416'11751) modify ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by client.2441958.0:1757 2014-08-13 15:28:37.366408 2014-08-13 17:38:02.984693 7fe9bae887a0 20 read_log 194416'11753 (0'0) promote eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by osd.15.0:2835 2014-08-13 15:28:42.472115 2014-08-13 17:38:02.984705 7fe9bae887a0 20 read_log 194416'11754 (194416'11753) modify eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by client.2441958.0:3105 2014-08-13 15:28:42.462454 2014-08-13 17:38:02.984719 7fe9bae887a0 20 read_log 194416'11755 (0'0) promote e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by osd.15.0:2841 2014-08-13 15:28:45.845193 2014-08-13 17:38:02.984731 7fe9bae887a0 20 read_log 194416'11756 (194416'11755) modify e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by client.2441958.0:3977 2014-08-13 15:28:45.835441 2014-08-13 17:38:02.984743 7fe9bae887a0 20 read_log 194416'11757 (0'0) promote dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by osd.15.0:2845 2014-08-13 15:28:46.740340 2014-08-13 17:38:02.984755 7fe9bae887a0 20 read_log 194416'11758 (194416'11757) modify dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by client.2441958.0:4198 2014-08-13 15:28:46.733616 2014-08-13 17:38:02.984767 7fe9bae887a0 20 read_log 194416'11759 (0'0) promote 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by osd.15.0:2847 2014-08-13 15:28:47.338938 2014-08-13 17:38:02.984779 7fe9bae887a0 20 read_log 194416'11760 (194416'11759) modify 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by client.2441958.0:4358 2014-08-13 15:28:47.332439 2014-08-13 17:38:02.984791 7fe9bae887a0 20 read_log 194416'11761 (0'0) promote cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by osd.15.0:2857 2014-08-13 15:28:52.868690 2014-08-13 17:38:02.984802 7fe9bae887a0 20 read_log 194416'11762 (194416'11761) modify cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by client.2441958.0:5773 2014-08-13 15:28:52.861760 2014-08-13 17:38:02.984815 7fe9bae887a0 20 read_log 194416'11763 (0'0) promote 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by osd.15.0:2869 2014-08-13 15:28:55.644734 2014-08-13 17:38:02.984827 7fe9bae887a0 20 read_log 194416'11764 (194416'11763) modify 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by client.2441958.0:6439 2014-08-13 15:28:55.636855 2014-08-13 17:38:02.984839 7fe9bae887a0 20 read_log 194416'11765 (0'0) promote 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by osd.15.0:2871 2014-08-13 15:28:56.314454 2014-08-13 17:38:02.984874 7fe9bae887a0 20 read_log 194416'11766 (194416'11765) modify 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by client.2441958.0:6607 2014-08-13 15:28:56.306485 2014-08-13 17:38:02.984887 7fe9bae887a0 20 read_log 194416'11767 (0'0) promote 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by osd.15.0:2897 2014-08-13 15:29:07.683717 2014-08-13 17:38:02.984900 7fe9bae887a0 20 read_log 194416'11768 (194416'11767) modify 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by client.2441958.0:9578 2014-08-13 15:29:07.675105 2014-08-13 17:38:02.984912 7fe9bae887a0 20 read_log 194416'11769 (0'0) promote a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by osd.15.0:2901 2014-08-13 15:29:09.251756 2014-08-13 17:38:02.984924 7fe9bae887a0 20 read_log 194416'11770 (194416'11769) modify a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by client.2441958.0:9956 2014-08-13 15:29:09.242751 2014-08-13 17:38:02.984936 7fe9bae887a0 20 read_log 194416'11771 (0'0) promote d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by osd.15.0:2905 2014-08-13 15:29:11.495677 2014-08-13 17:38:02.984948 7fe9bae887a0 20 read_log 194416'11772 (194416'11771) modify d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by client.2441958.0:10515 2014-08-13 15:29:11.486380 2014-08-13 17:38:02.984960 7fe9bae887a0 20 read_log 194416'11773 (0'0) promote 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by osd.15.0:2911 2014-08-13 15:29:13.891865 2014-08-13 17:38:02.984972 7fe9bae887a0 20 read_log 194416'11774 (194416'11773) modify 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by client.2441958.0:11134 2014-08-13 15:29:13.881622 2014-08-13 17:38:02.984984 7fe9bae887a0 20 read_log 194416'11775 (0'0) promote dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by osd.15.0:2927 2014-08-13 15:29:24.940835 2014-08-13 17:38:02.984999 7fe9bae887a0 20 read_log 194416'11776 (194416'11775) modify dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by client.2441958.0:13681 2014-08-13 15:29:24.932178 2014-08-13 17:38:02.985011 7fe9bae887a0 20 read_log 194416'11777 (0'0) promote e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by osd.15.0:2933 2014-08-13 15:29:25.656553 2014-08-13 17:38:02.985023 7fe9bae887a0 20 read_log 194416'11778 (194416'11777) modify e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by client.2441958.0:13858 2014-08-13 15:29:25.648625 2014-08-13 17:38:02.985035 7fe9bae887a0 20 read_log 194416'11779 (0'0) promote 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by osd.15.0:2943 2014-08-13 15:29:27.566695 2014-08-13 17:38:02.985047 7fe9bae887a0 20 read_log 194416'11780 (194416'11779) modify 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by client.2441958.0:14352 2014-08-13 15:29:27.556476 2014-08-13 17:38:02.985059 7fe9bae887a0 20 read_log 194416'11781 (0'0) promote 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by osd.15.0:2961 2014-08-13 15:29:31.860218 2014-08-13 17:38:02.985088 7fe9bae887a0 20 read_log 194416'11782 (194416'11781) modify 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by client.2441958.0:15353 2014-08-13 15:29:31.852271 2014-08-13 17:38:02.985101 7fe9bae887a0 20 read_log 194416'11783 (0'0) promote f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by osd.15.0:2975 2014-08-13 15:29:43.772509 2014-08-13 17:38:02.985113 7fe9bae887a0 20 read_log 194416'11784 (194416'11783) modify f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by client.2441958.0:16752 2014-08-13 15:29:43.762592 2014-08-13 17:38:02.985126 7fe9bae887a0 20 read_log 194416'11785 (0'0) promote b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by osd.15.0:2983 2014-08-13 15:29:54.148756 2014-08-13 17:38:02.985145 7fe9bae887a0 20 read_log 194416'11786 (194416'11785) modify b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by client.2441958.0:17480 2014-08-13 15:29:54.140182 2014-08-13 17:38:02.985162 7fe9bae887a0 20 read_log 194416'11787 (0'0) promote 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by osd.15.0:2985 2014-08-13 15:29:54.433501 2014-08-13 17:38:02.985174 7fe9bae887a0 20 read_log 194416'11788 (194416'11787) modify 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by client.2441958.0:17500 2014-08-13 15:29:54.424428 2014-08-13 17:38:02.985187 7fe9bae887a0 20 read_log 194422'11789 (0'0) promote 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by osd.15.0:3013 2014-08-13 15:32:23.894340 2014-08-13 17:38:02.985199 7fe9bae887a0 20 read_log 194422'11790 (194422'11789) modify 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by client.1148203.0:2173 2014-08-13 15:32:23.886263 2014-08-13 17:38:02.985211 7fe9bae887a0 20 read_log 194422'11791 (0'0) promote 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by osd.15.0:3021 2014-08-13 15:32:26.581047 2014-08-13 17:38:02.985223 7fe9bae887a0 20 read_log 194422'11792 (194422'11791) modify 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by client.1148203.0:2865 2014-08-13 15:32:26.572355 2014-08-13 17:38:02.985235 7fe9bae887a0 20 read_log 194422'11793 (0'0) promote 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by osd.15.0:3025 2014-08-13 15:32:27.970042 2014-08-13 17:38:02.985247 7fe9bae887a0 20 read_log 194422'11794 (194422'11793) modify 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by client.1148203.0:3230 2014-08-13 15:32:27.962899 2014-08-13 17:38:02.985259 7fe9bae887a0 20 read_log 194422'11795 (0'0) promote bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by osd.15.0:3045 2014-08-13 15:32:41.048428 2014-08-13 17:38:02.985270 7fe9bae887a0 20 read_log 194422'11796 (194422'11795) modify bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by client.1148203.0:6167 2014-08-13 15:32:41.037193 2014-08-13 17:38:02.985283 7fe9bae887a0 20 read_log 194422'11797 (0'0) promote dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by osd.15.0:3067 2014-08-13 15:32:51.368290 2014-08-13 17:38:02.985294 7fe9bae887a0 20 read_log 194422'11798 (194422'11797) modify dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by client.1148203.0:7946 2014-08-13 15:32:51.359365 2014-08-13 17:38:02.985306 7fe9bae887a0 20 read_log 194422'11799 (0'0) promote bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by osd.15.0:3071 2014-08-13 15:32:53.078937 2014-08-13 17:38:02.985318 7fe9bae887a0 20 read_log 194422'11800 (194422'11799) modify bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by client.1148203.0:8216 2014-08-13 15:32:53.069000 2014-08-13 17:38:02.985330 7fe9bae887a0 20 read_log 194422'11801 (0'0) promote 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by osd.15.0:3075 2014-08-13 15:32:58.155342 2014-08-13 17:38:02.985342 7fe9bae887a0 20 read_log 194422'11802 (194422'11801) modify 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by client.1148203.0:8855 2014-08-13 15:32:58.146666 2014-08-13 17:38:02.985354 7fe9bae887a0 20 read_log 194422'11803 (0'0) promote 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by osd.15.0:3081 2014-08-13 15:33:00.679424 2014-08-13 17:38:02.985366 7fe9bae887a0 20 read_log 194422'11804 (194422'11803) modify 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by client.1148203.0:9113 2014-08-13 15:33:00.671918 2014-08-13 17:38:02.985378 7fe9bae887a0 20 read_log 194422'11805 (0'0) promote 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by osd.15.0:3083 2014-08-13 15:33:00.906158 2014-08-13 17:38:02.985392 7fe9bae887a0 20 read_log 194422'11806 (194422'11805) modify 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by client.1148203.0:9137 2014-08-13 15:33:00.896712 2014-08-13 17:38:02.985407 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.986037 7fe9bae887a0 10 read_log checking for missing items over interval (194370'11691,194422'11806] 2014-08-13 17:38:02.986084 7fe9bae887a0 15 read_log missing 194422'11806 (194422'11805) modify 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by client.1148203.0:9137 2014-08-13 15:33:00.896712 2014-08-13 17:38:02.986124 7fe9bae887a0 15 read_log missing 194422'11804 (194422'11803) modify 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by client.1148203.0:9113 2014-08-13 15:33:00.671918 2014-08-13 17:38:02.986165 7fe9bae887a0 15 read_log missing 194422'11802 (194422'11801) modify 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by client.1148203.0:8855 2014-08-13 15:32:58.146666 2014-08-13 17:38:02.986200 7fe9bae887a0 15 read_log missing 194422'11800 (194422'11799) modify bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by client.1148203.0:8216 2014-08-13 15:32:53.069000 2014-08-13 17:38:02.986235 7fe9bae887a0 15 read_log missing 194422'11798 (194422'11797) modify dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by client.1148203.0:7946 2014-08-13 15:32:51.359365 2014-08-13 17:38:02.986269 7fe9bae887a0 15 read_log missing 194422'11796 (194422'11795) modify bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by client.1148203.0:6167 2014-08-13 15:32:41.037193 2014-08-13 17:38:02.986303 7fe9bae887a0 15 read_log missing 194422'11794 (194422'11793) modify 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by client.1148203.0:3230 2014-08-13 15:32:27.962899 2014-08-13 17:38:02.986337 7fe9bae887a0 15 read_log missing 194422'11792 (194422'11791) modify 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by client.1148203.0:2865 2014-08-13 15:32:26.572355 2014-08-13 17:38:02.986371 7fe9bae887a0 15 read_log missing 194422'11790 (194422'11789) modify 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by client.1148203.0:2173 2014-08-13 15:32:23.886263 2014-08-13 17:38:02.986404 7fe9bae887a0 15 read_log missing 194416'11788 (194416'11787) modify 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by client.2441958.0:17500 2014-08-13 15:29:54.424428 2014-08-13 17:38:02.986438 7fe9bae887a0 15 read_log missing 194416'11786 (194416'11785) modify b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by client.2441958.0:17480 2014-08-13 15:29:54.140182 2014-08-13 17:38:02.986472 7fe9bae887a0 15 read_log missing 194416'11784 (194416'11783) modify f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by client.2441958.0:16752 2014-08-13 15:29:43.762592 2014-08-13 17:38:02.986506 7fe9bae887a0 15 read_log missing 194416'11782 (194416'11781) modify 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by client.2441958.0:15353 2014-08-13 15:29:31.852271 2014-08-13 17:38:02.986539 7fe9bae887a0 15 read_log missing 194416'11780 (194416'11779) modify 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by client.2441958.0:14352 2014-08-13 15:29:27.556476 2014-08-13 17:38:02.986573 7fe9bae887a0 15 read_log missing 194416'11778 (194416'11777) modify e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by client.2441958.0:13858 2014-08-13 15:29:25.648625 2014-08-13 17:38:02.986606 7fe9bae887a0 15 read_log missing 194416'11776 (194416'11775) modify dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by client.2441958.0:13681 2014-08-13 15:29:24.932178 2014-08-13 17:38:02.986639 7fe9bae887a0 15 read_log missing 194416'11774 (194416'11773) modify 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by client.2441958.0:11134 2014-08-13 15:29:13.881622 2014-08-13 17:38:02.986673 7fe9bae887a0 15 read_log missing 194416'11772 (194416'11771) modify d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by client.2441958.0:10515 2014-08-13 15:29:11.486380 2014-08-13 17:38:02.986706 7fe9bae887a0 15 read_log missing 194416'11770 (194416'11769) modify a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by client.2441958.0:9956 2014-08-13 15:29:09.242751 2014-08-13 17:38:02.986739 7fe9bae887a0 15 read_log missing 194416'11768 (194416'11767) modify 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by client.2441958.0:9578 2014-08-13 15:29:07.675105 2014-08-13 17:38:02.986772 7fe9bae887a0 15 read_log missing 194416'11766 (194416'11765) modify 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by client.2441958.0:6607 2014-08-13 15:28:56.306485 2014-08-13 17:38:02.986806 7fe9bae887a0 15 read_log missing 194416'11764 (194416'11763) modify 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by client.2441958.0:6439 2014-08-13 15:28:55.636855 2014-08-13 17:38:02.986841 7fe9bae887a0 15 read_log missing 194416'11762 (194416'11761) modify cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by client.2441958.0:5773 2014-08-13 15:28:52.861760 2014-08-13 17:38:02.986875 7fe9bae887a0 15 read_log missing 194416'11760 (194416'11759) modify 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by client.2441958.0:4358 2014-08-13 15:28:47.332439 2014-08-13 17:38:02.986909 7fe9bae887a0 15 read_log missing 194416'11758 (194416'11757) modify dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by client.2441958.0:4198 2014-08-13 15:28:46.733616 2014-08-13 17:38:02.986943 7fe9bae887a0 15 read_log missing 194416'11756 (194416'11755) modify e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by client.2441958.0:3977 2014-08-13 15:28:45.835441 2014-08-13 17:38:02.986978 7fe9bae887a0 15 read_log missing 194416'11754 (194416'11753) modify eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by client.2441958.0:3105 2014-08-13 15:28:42.462454 2014-08-13 17:38:02.987012 7fe9bae887a0 15 read_log missing 194416'11752 (194416'11751) modify ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by client.2441958.0:1757 2014-08-13 15:28:37.366408 2014-08-13 17:38:02.987050 7fe9bae887a0 15 read_log missing 194416'11749 (194416'11749) modify 20e/hit_set_24.20e_archive_2014-08-12 20:53:02.891413_2014-08-13 15:28:37.371929/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 17:38:02.987084 7fe9bae887a0 15 read_log missing 194397'11748 (194397'11747) modify 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by client.2235069.0:755 2014-08-12 20:53:03.272166 2014-08-13 17:38:02.987118 7fe9bae887a0 15 read_log missing 194397'11746 (194397'11745) modify f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by client.2235069.0:705 2014-08-12 20:53:02.883290 2014-08-13 17:38:02.987161 7fe9bae887a0 15 read_log missing 194393'11742 (194393'11741) modify 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by client.1064182.0:481 2014-08-12 20:49:28.260541 2014-08-13 17:38:02.987196 7fe9bae887a0 15 read_log missing 194393'11740 (194393'11739) modify 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by client.1064182.0:12 2014-08-12 20:49:25.421095 2014-08-13 17:38:02.987230 7fe9bae887a0 15 read_log missing 194393'11738 (194393'11737) modify 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by client.1064095.0:735 2014-08-12 20:47:30.624403 2014-08-13 17:38:02.987269 7fe9bae887a0 15 read_log missing 194383'11734 (194383'11733) modify 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by client.2231190.0:7747 2014-08-12 20:34:35.621285 2014-08-13 17:38:02.987303 7fe9bae887a0 15 read_log missing 194383'11732 (194383'11731) modify 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by client.2231190.0:6699 2014-08-12 20:34:07.042697 2014-08-13 17:38:02.987338 7fe9bae887a0 15 read_log missing 194383'11730 (194383'11729) modify bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by client.2231190.0:6330 2014-08-12 20:34:03.189069 2014-08-13 17:38:02.987373 7fe9bae887a0 15 read_log missing 194380'11728 (194380'11727) modify eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by client.2231190.0:5098 2014-08-12 20:33:24.385329 2014-08-13 17:38:02.987407 7fe9bae887a0 15 read_log missing 194377'11726 (194377'11725) modify 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by client.2231190.0:4362 2014-08-12 20:33:16.133686 2014-08-13 17:38:02.987441 7fe9bae887a0 15 read_log missing 194377'11724 (194377'11723) modify c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by client.2231190.0:3886 2014-08-12 20:33:13.281188 2014-08-13 17:38:02.987475 7fe9bae887a0 15 read_log missing 194377'11722 (194377'11721) modify 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by client.2231190.0:3601 2014-08-12 20:33:11.866129 2014-08-13 17:38:02.987514 7fe9bae887a0 15 read_log missing 194377'11720 (194377'11719) modify 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by client.2231190.0:3047 2014-08-12 20:33:09.507056 2014-08-13 17:38:02.987548 7fe9bae887a0 15 read_log missing 194370'11718 (194370'11717) modify 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by client.1508531.0:13070 2014-08-12 20:31:20.004477 2014-08-13 17:38:02.987582 7fe9bae887a0 15 read_log missing 194370'11716 (194370'11715) modify 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by client.1508531.0:9723 2014-08-12 20:30:57.657537 2014-08-13 17:38:02.987616 7fe9bae887a0 15 read_log missing 194370'11714 (194370'11713) modify 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by client.1508531.0:9530 2014-08-12 20:30:56.505775 2014-08-13 17:38:02.987651 7fe9bae887a0 15 read_log missing 194370'11712 (194370'11711) modify d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by client.1508531.0:9292 2014-08-12 20:30:55.064089 2014-08-13 17:38:02.987688 7fe9bae887a0 15 read_log missing 194370'11710 (194370'11709) modify 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by client.1508531.0:9267 2014-08-12 20:30:54.930205 2014-08-13 17:38:02.987722 7fe9bae887a0 15 read_log missing 194370'11708 (194370'11707) modify f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by client.1508531.0:8121 2014-08-12 20:30:48.221225 2014-08-13 17:38:02.987755 7fe9bae887a0 15 read_log missing 194370'11706 (194370'11705) modify 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by client.1508531.0:7475 2014-08-12 20:30:44.729692 2014-08-13 17:38:02.987789 7fe9bae887a0 15 read_log missing 194370'11704 (194370'11703) modify c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by client.1508531.0:7411 2014-08-12 20:30:44.411439 2014-08-13 17:38:02.987823 7fe9bae887a0 15 read_log missing 194370'11702 (194370'11701) modify 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by client.1508531.0:7161 2014-08-12 20:30:43.162041 2014-08-13 17:38:02.987856 7fe9bae887a0 15 read_log missing 194370'11700 (194370'11699) modify e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by client.1508531.0:2739 2014-08-12 20:30:24.590302 2014-08-13 17:38:02.987894 7fe9bae887a0 15 read_log missing 194370'11698 (194370'11697) modify fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by client.1508531.0:2626 2014-08-12 20:30:24.180964 2014-08-13 17:38:02.987931 7fe9bae887a0 15 read_log missing 194370'11696 (194370'11695) modify 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by client.1508531.0:1607 2014-08-12 20:30:20.397467 2014-08-13 17:38:02.987967 7fe9bae887a0 15 read_log missing 194370'11694 (194370'11693) modify 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by client.1508531.0:1294 2014-08-12 20:30:19.237527 2014-08-13 17:38:02.988004 7fe9bae887a0 15 read_log missing 194370'11692 (194370'11691) modify 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by client.1508531.0:1119 2014-08-12 20:30:18.549021 2014-08-13 17:38:02.988015 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.988051 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 17:38:02.988069 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.021229 0 0.000000 2014-08-13 17:38:02.988081 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 17:38:02.988090 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194478 2014-08-13 17:38:02.988100 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] log((192718'8900,194422'11806], crt=194422'11806) 2014-08-13 17:38:02.988110 7fe9bae887a0 10 osd.4 194478 pgid 24.299 coll 24.299_head 2014-08-13 17:38:02.988230 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.299 2014-08-13 17:38:02.988242 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.988279 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.299(unlocked)] enter Initial 2014-08-13 17:38:02.988292 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.299(unlocked)] enter NotTrimming 2014-08-13 17:38:02.988403 7fe9bae887a0 10 read_log 2014-08-13 17:38:02.988528 7fe9bae887a0 20 read_log 192834'13598 (0'0) promote e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by osd.372.0:29784 2014-08-11 15:23:49.578363 2014-08-13 17:38:02.988550 7fe9bae887a0 20 read_log 192834'13599 (192834'13598) modify e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.933448.0:2966 2014-08-11 15:23:49.570293 2014-08-13 17:38:02.988563 7fe9bae887a0 20 read_log 192834'13604 (0'0) promote c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by osd.372.0:29792 2014-08-11 15:23:50.163965 2014-08-13 17:38:02.988576 7fe9bae887a0 20 read_log 192834'13605 (192834'13604) modify c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.933448.0:3065 2014-08-11 15:23:49.995911 2014-08-13 17:38:02.988588 7fe9bae887a0 20 read_log 192834'13610 (0'0) promote e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by osd.372.0:29800 2014-08-11 15:23:51.490438 2014-08-13 17:38:02.988600 7fe9bae887a0 20 read_log 192834'13611 (192834'13610) modify e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.933448.0:3377 2014-08-11 15:23:51.482370 2014-08-13 17:38:02.988612 7fe9bae887a0 20 read_log 192834'13665 (0'0) promote 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by osd.372.0:29895 2014-08-11 15:24:04.488609 2014-08-13 17:38:02.988624 7fe9bae887a0 20 read_log 192834'13666 (192834'13665) modify 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.933448.0:6196 2014-08-11 15:24:04.481779 2014-08-13 17:38:02.988637 7fe9bae887a0 20 read_log 192834'13697 (0'0) promote d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by osd.372.0:29971 2014-08-11 15:24:13.448301 2014-08-13 17:38:02.988649 7fe9bae887a0 20 read_log 192834'13698 (192834'13697) modify d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.933448.0:8290 2014-08-11 15:24:13.441461 2014-08-13 17:38:02.988661 7fe9bae887a0 20 read_log 192834'13699 (0'0) promote 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by osd.372.0:29977 2014-08-11 15:24:17.369035 2014-08-13 17:38:02.988673 7fe9bae887a0 20 read_log 192834'13700 (192834'13699) modify 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.933448.0:9052 2014-08-11 15:24:17.362409 2014-08-13 17:38:02.988685 7fe9bae887a0 20 read_log 192834'13709 (0'0) promote 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by osd.372.0:29989 2014-08-11 15:24:18.560489 2014-08-13 17:38:02.988697 7fe9bae887a0 20 read_log 192834'13710 (192834'13709) modify 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.933448.0:9364 2014-08-11 15:24:18.553459 2014-08-13 17:38:02.988717 7fe9bae887a0 20 read_log 192834'13713 (0'0) promote 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by osd.372.0:30011 2014-08-11 15:24:21.016297 2014-08-13 17:38:02.988730 7fe9bae887a0 20 read_log 192834'13714 (192834'13713) modify 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.933448.0:9960 2014-08-11 15:24:21.009005 2014-08-13 17:38:02.988742 7fe9bae887a0 20 read_log 192834'13723 (0'0) promote a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by osd.372.0:30029 2014-08-11 15:24:22.799925 2014-08-13 17:38:02.988754 7fe9bae887a0 20 read_log 192834'13724 (192834'13723) modify a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.933448.0:10348 2014-08-11 15:24:22.791544 2014-08-13 17:38:02.988766 7fe9bae887a0 20 read_log 192834'13727 (0'0) promote c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by osd.372.0:30035 2014-08-11 15:24:23.757948 2014-08-13 17:38:02.988777 7fe9bae887a0 20 read_log 192834'13728 (192834'13727) modify c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.933448.0:10565 2014-08-11 15:24:23.751870 2014-08-13 17:38:02.988789 7fe9bae887a0 20 read_log 192834'13765 (0'0) promote c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by osd.372.0:30121 2014-08-11 15:24:34.974574 2014-08-13 17:38:02.988801 7fe9bae887a0 20 read_log 192834'13766 (192834'13765) modify c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.933448.0:13014 2014-08-11 15:24:34.966502 2014-08-13 17:38:02.988813 7fe9bae887a0 20 read_log 192834'13795 (0'0) promote 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by osd.372.0:30217 2014-08-11 15:24:51.917895 2014-08-13 17:38:02.988825 7fe9bae887a0 20 read_log 192834'13796 (192834'13795) modify 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.933448.0:15937 2014-08-11 15:24:51.910959 2014-08-13 17:38:02.988837 7fe9bae887a0 20 read_log 192834'13801 (0'0) promote 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by osd.372.0:30227 2014-08-11 15:24:55.950079 2014-08-13 17:38:02.988849 7fe9bae887a0 20 read_log 192834'13802 (192834'13801) modify 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.933448.0:16489 2014-08-11 15:24:55.942940 2014-08-13 17:38:02.988860 7fe9bae887a0 20 read_log 192834'13809 (0'0) promote 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by osd.372.0:30243 2014-08-11 15:24:58.428986 2014-08-13 17:38:02.988872 7fe9bae887a0 20 read_log 192834'13810 (192834'13809) modify 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.933448.0:17096 2014-08-11 15:24:58.421764 2014-08-13 17:38:02.988884 7fe9bae887a0 20 read_log 192834'13813 (0'0) promote 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by osd.372.0:30259 2014-08-11 15:24:59.796670 2014-08-13 17:38:02.988896 7fe9bae887a0 20 read_log 192834'13814 (192834'13813) modify 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.933448.0:17463 2014-08-11 15:24:59.790107 2014-08-13 17:38:02.988908 7fe9bae887a0 20 read_log 192834'13817 (0'0) promote 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by osd.372.0:30265 2014-08-11 15:25:01.417570 2014-08-13 17:38:02.988920 7fe9bae887a0 20 read_log 192834'13818 (192834'13817) modify 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.933448.0:17760 2014-08-11 15:25:01.408718 2014-08-13 17:38:02.988932 7fe9bae887a0 20 read_log 192834'13837 (0'0) promote ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by osd.372.0:30297 2014-08-11 15:25:06.296427 2014-08-13 17:38:02.988944 7fe9bae887a0 20 read_log 192834'13838 (192834'13837) modify ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.933448.0:18675 2014-08-11 15:25:06.289483 2014-08-13 17:38:02.988963 7fe9bae887a0 20 read_log 192834'13843 (0'0) promote bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by osd.372.0:30311 2014-08-11 15:25:07.269677 2014-08-13 17:38:02.988976 7fe9bae887a0 20 read_log 192834'13844 (192834'13843) modify bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.933448.0:18869 2014-08-11 15:25:07.262374 2014-08-13 17:38:02.988988 7fe9bae887a0 20 read_log 192834'13865 (0'0) promote d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by osd.372.0:30369 2014-08-11 15:25:22.122486 2014-08-13 17:38:02.989000 7fe9bae887a0 20 read_log 192834'13866 (192834'13865) modify d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.933448.0:20604 2014-08-11 15:25:22.114568 2014-08-13 17:38:02.989012 7fe9bae887a0 20 read_log 192834'13875 (0'0) promote 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by osd.372.0:30381 2014-08-11 15:25:26.072839 2014-08-13 17:38:02.989023 7fe9bae887a0 20 read_log 192834'13876 (192834'13875) modify 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.933448.0:21078 2014-08-11 15:25:26.064541 2014-08-13 17:38:02.989035 7fe9bae887a0 20 read_log 192834'13879 (0'0) promote 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by osd.372.0:30389 2014-08-11 15:25:27.643403 2014-08-13 17:38:02.989047 7fe9bae887a0 20 read_log 192834'13880 (192834'13879) modify 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.933448.0:21166 2014-08-11 15:25:27.634566 2014-08-13 17:38:02.989059 7fe9bae887a0 20 read_log 192834'13898 (192834'13565) modify fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by client.933448.0:23518 2014-08-11 15:25:39.433726 2014-08-13 17:38:02.989071 7fe9bae887a0 20 read_log 192834'13902 (192834'13573) modify f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by client.933448.0:24030 2014-08-11 15:25:40.203989 2014-08-13 17:38:02.989083 7fe9bae887a0 20 read_log 192834'13915 (192834'13599) modify e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.933448.0:25257 2014-08-11 15:25:41.916979 2014-08-13 17:38:02.989095 7fe9bae887a0 20 read_log 192834'13918 (192834'13605) modify c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.933448.0:25356 2014-08-11 15:25:42.026094 2014-08-13 17:38:02.989107 7fe9bae887a0 20 read_log 192834'13921 (192834'13611) modify e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.933448.0:25668 2014-08-11 15:25:42.329133 2014-08-13 17:38:02.989118 7fe9bae887a0 20 read_log 192834'13948 (192834'13666) modify 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.933448.0:28487 2014-08-11 15:25:46.309038 2014-08-13 17:38:02.989130 7fe9bae887a0 20 read_log 192834'13964 (192834'13698) modify d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.933448.0:30581 2014-08-11 15:25:49.016987 2014-08-13 17:38:02.989142 7fe9bae887a0 20 read_log 192834'13965 (192834'13700) modify 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.933448.0:31343 2014-08-11 15:25:49.949046 2014-08-13 17:38:02.989160 7fe9bae887a0 20 read_log 192834'13970 (192834'13710) modify 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.933448.0:31655 2014-08-11 15:25:50.325417 2014-08-13 17:38:02.989173 7fe9bae887a0 20 read_log 192834'13972 (192834'13714) modify 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.933448.0:32251 2014-08-11 15:25:51.333815 2014-08-13 17:38:02.989186 7fe9bae887a0 20 read_log 192834'13977 (192834'13724) modify a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.933448.0:32639 2014-08-11 15:25:52.145391 2014-08-13 17:38:02.989198 7fe9bae887a0 20 read_log 192834'13979 (192834'13728) modify c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.933448.0:32856 2014-08-11 15:25:52.441126 2014-08-13 17:38:02.989221 7fe9bae887a0 20 read_log 192834'13998 (192834'13766) modify c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.933448.0:35305 2014-08-11 15:25:56.076314 2014-08-13 17:38:02.989235 7fe9bae887a0 20 read_log 192834'14013 (192834'13796) modify 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.933448.0:38228 2014-08-11 15:26:00.680245 2014-08-13 17:38:02.989247 7fe9bae887a0 20 read_log 192834'14016 (192834'13802) modify 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.933448.0:38780 2014-08-11 15:26:01.261824 2014-08-13 17:38:02.989259 7fe9bae887a0 20 read_log 192834'14020 (192834'13810) modify 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.933448.0:39387 2014-08-11 15:26:01.839792 2014-08-13 17:38:02.989271 7fe9bae887a0 20 read_log 192834'14022 (192834'13814) modify 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.933448.0:39754 2014-08-11 15:26:02.060829 2014-08-13 17:38:02.989283 7fe9bae887a0 20 read_log 192834'14024 (192834'13818) modify 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.933448.0:40051 2014-08-11 15:26:02.246605 2014-08-13 17:38:02.989295 7fe9bae887a0 20 read_log 192834'14034 (192834'13838) modify ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.933448.0:40966 2014-08-11 15:26:02.801891 2014-08-13 17:38:02.989307 7fe9bae887a0 20 read_log 192834'14037 (192834'13844) modify bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.933448.0:41160 2014-08-11 15:26:02.855920 2014-08-13 17:38:02.989318 7fe9bae887a0 20 read_log 192834'14048 (192834'13866) modify d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.933448.0:42895 2014-08-11 15:26:03.425754 2014-08-13 17:38:02.989357 7fe9bae887a0 20 read_log 192834'14053 (192834'13876) modify 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.933448.0:43369 2014-08-11 15:26:03.591326 2014-08-13 17:38:02.989370 7fe9bae887a0 20 read_log 192834'14055 (192834'13880) modify 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.933448.0:43457 2014-08-11 15:26:03.622668 2014-08-13 17:38:02.989387 7fe9bae887a0 20 read_log 192834'14074 (0'0) promote 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by osd.372.0:30844 2014-08-11 15:26:17.061546 2014-08-13 17:38:02.989400 7fe9bae887a0 20 read_log 192834'14075 (192834'14074) modify 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.1883634.0:706 2014-08-11 15:26:17.054534 2014-08-13 17:38:02.989412 7fe9bae887a0 20 read_log 192834'14098 (0'0) promote bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by osd.372.0:30884 2014-08-11 15:26:21.739061 2014-08-13 17:38:02.989424 7fe9bae887a0 20 read_log 192834'14099 (192834'14098) modify bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.1883634.0:1967 2014-08-11 15:26:21.732715 2014-08-13 17:38:02.989437 7fe9bae887a0 20 read_log 192834'14108 (0'0) promote 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by osd.372.0:30906 2014-08-11 15:26:23.537702 2014-08-13 17:38:02.989449 7fe9bae887a0 20 read_log 192834'14109 (192834'14108) modify 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.1883634.0:2456 2014-08-11 15:26:23.529686 2014-08-13 17:38:02.989461 7fe9bae887a0 20 read_log 192834'14130 (0'0) promote 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by osd.372.0:30958 2014-08-11 15:26:31.942985 2014-08-13 17:38:02.989473 7fe9bae887a0 20 read_log 192834'14131 (192834'14130) modify 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.1883634.0:4792 2014-08-11 15:26:31.935135 2014-08-13 17:38:02.989485 7fe9bae887a0 20 read_log 192834'14154 (0'0) promote ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by osd.372.0:31014 2014-08-11 15:26:39.832357 2014-08-13 17:38:02.989505 7fe9bae887a0 20 read_log 192834'14155 (192834'14154) modify ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.1883634.0:6882 2014-08-11 15:26:39.826591 2014-08-13 17:38:02.989519 7fe9bae887a0 20 read_log 192834'14164 (0'0) promote d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by osd.372.0:31024 2014-08-11 15:26:40.691780 2014-08-13 17:38:02.989531 7fe9bae887a0 20 read_log 192834'14165 (192834'14164) modify d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.1883634.0:7072 2014-08-11 15:26:40.685654 2014-08-13 17:38:02.989544 7fe9bae887a0 20 read_log 192834'14174 (0'0) promote a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by osd.372.0:31046 2014-08-11 15:26:43.679546 2014-08-13 17:38:02.989556 7fe9bae887a0 20 read_log 192834'14175 (192834'14174) modify a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.1883634.0:7795 2014-08-11 15:26:43.671750 2014-08-13 17:38:02.989568 7fe9bae887a0 20 read_log 192834'14178 (0'0) promote eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by osd.372.0:31050 2014-08-11 15:26:44.136714 2014-08-13 17:38:02.989580 7fe9bae887a0 20 read_log 192834'14179 (192834'14178) modify eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.1883634.0:7868 2014-08-11 15:26:44.127881 2014-08-13 17:38:02.989592 7fe9bae887a0 20 read_log 192834'14184 (0'0) promote 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by osd.372.0:31056 2014-08-11 15:26:44.704057 2014-08-13 17:38:02.989603 7fe9bae887a0 20 read_log 192834'14185 (192834'14184) modify 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.1883634.0:8026 2014-08-11 15:26:44.694437 2014-08-13 17:38:02.989616 7fe9bae887a0 20 read_log 192834'14200 (0'0) promote 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by osd.372.0:31102 2014-08-11 15:26:51.073804 2014-08-13 17:38:02.989627 7fe9bae887a0 20 read_log 192834'14201 (192834'14200) modify 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.1883634.0:9498 2014-08-11 15:26:51.066901 2014-08-13 17:38:02.989640 7fe9bae887a0 20 read_log 192834'14210 (0'0) promote b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by osd.372.0:31130 2014-08-11 15:26:54.944265 2014-08-13 17:38:02.989652 7fe9bae887a0 20 read_log 192834'14211 (192834'14210) modify b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.1883634.0:10405 2014-08-11 15:26:54.935611 2014-08-13 17:38:02.989664 7fe9bae887a0 20 read_log 192834'14236 (0'0) promote 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by osd.372.0:31172 2014-08-11 15:27:01.407681 2014-08-13 17:38:02.989676 7fe9bae887a0 20 read_log 192834'14237 (192834'14236) modify 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.1883634.0:11829 2014-08-11 15:27:01.399626 2014-08-13 17:38:02.989688 7fe9bae887a0 20 read_log 192834'14242 (0'0) promote 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by osd.372.0:31186 2014-08-11 15:27:02.956335 2014-08-13 17:38:02.989700 7fe9bae887a0 20 read_log 192834'14243 (192834'14242) modify 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.1883634.0:12196 2014-08-11 15:27:02.947506 2014-08-13 17:38:02.989712 7fe9bae887a0 20 read_log 192834'14248 (0'0) promote cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by osd.372.0:31200 2014-08-11 15:27:04.345674 2014-08-13 17:38:02.989724 7fe9bae887a0 20 read_log 192834'14249 (192834'14248) modify cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.1883634.0:12525 2014-08-11 15:27:04.335830 2014-08-13 17:38:02.989736 7fe9bae887a0 20 read_log 192834'14254 (0'0) promote 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by osd.372.0:31208 2014-08-11 15:27:06.213175 2014-08-13 17:38:02.989775 7fe9bae887a0 20 read_log 192834'14255 (192834'14254) modify 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.1883634.0:12818 2014-08-11 15:27:06.206725 2014-08-13 17:38:02.989789 7fe9bae887a0 20 read_log 192834'14258 (0'0) promote 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by osd.372.0:31216 2014-08-11 15:27:07.933355 2014-08-13 17:38:02.989801 7fe9bae887a0 20 read_log 192834'14259 (192834'14258) modify 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.1883634.0:13162 2014-08-11 15:27:07.924025 2014-08-13 17:38:02.989814 7fe9bae887a0 20 read_log 192834'14276 (0'0) promote 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by osd.372.0:31250 2014-08-11 15:27:11.305149 2014-08-13 17:38:02.989825 7fe9bae887a0 20 read_log 192834'14277 (192834'14276) modify 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.1883634.0:13944 2014-08-11 15:27:11.297051 2014-08-13 17:38:02.989841 7fe9bae887a0 20 read_log 192834'14304 (0'0) promote ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by osd.372.0:31298 2014-08-11 15:27:19.392236 2014-08-13 17:38:02.989853 7fe9bae887a0 20 read_log 192834'14305 (192834'14304) modify ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.1883634.0:15406 2014-08-11 15:27:19.385383 2014-08-13 17:38:02.989865 7fe9bae887a0 20 read_log 192834'14306 (0'0) promote 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by osd.372.0:31300 2014-08-11 15:27:19.906962 2014-08-13 17:38:02.989880 7fe9bae887a0 20 read_log 192834'14307 (192834'14306) modify 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.1883634.0:15518 2014-08-11 15:27:19.898786 2014-08-13 17:38:02.989892 7fe9bae887a0 20 read_log 192834'14336 (0'0) promote 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by osd.372.0:31356 2014-08-11 15:27:40.059711 2014-08-13 17:38:02.989906 7fe9bae887a0 20 read_log 192834'14337 (192834'14336) modify 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.1883634.0:17928 2014-08-11 15:27:40.049894 2014-08-13 17:38:02.989918 7fe9bae887a0 20 read_log 192834'14344 (0'0) promote 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by osd.372.0:31370 2014-08-11 15:27:47.115181 2014-08-13 17:38:02.989930 7fe9bae887a0 20 read_log 192834'14345 (192834'14344) modify 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.1883634.0:18693 2014-08-11 15:27:47.106754 2014-08-13 17:38:02.989942 7fe9bae887a0 20 read_log 192834'14352 (0'0) promote 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by osd.372.0:31390 2014-08-11 15:27:49.034253 2014-08-13 17:38:02.989954 7fe9bae887a0 20 read_log 192834'14353 (192834'14352) modify 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.1883634.0:19038 2014-08-11 15:27:49.026447 2014-08-13 17:38:02.989969 7fe9bae887a0 20 read_log 192834'14354 (0'0) promote 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by osd.372.0:31400 2014-08-11 15:27:51.310863 2014-08-13 17:38:02.989981 7fe9bae887a0 20 read_log 192834'14355 (192834'14354) modify 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.1883634.0:19329 2014-08-11 15:27:51.302086 2014-08-13 17:38:02.989995 7fe9bae887a0 20 read_log 192834'14360 (0'0) promote be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by osd.372.0:31416 2014-08-11 15:27:54.694449 2014-08-13 17:38:02.990008 7fe9bae887a0 20 read_log 192834'14361 (192834'14360) modify be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.1883634.0:19934 2014-08-11 15:27:54.685261 2014-08-13 17:38:02.990022 7fe9bae887a0 20 read_log 192834'14374 (0'0) promote e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by osd.372.0:31440 2014-08-11 15:28:01.671420 2014-08-13 17:38:02.990042 7fe9bae887a0 20 read_log 192834'14375 (192834'14374) modify e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.1883634.0:20821 2014-08-11 15:28:01.662663 2014-08-13 17:38:02.990057 7fe9bae887a0 20 read_log 192834'14376 (0'0) promote eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by osd.372.0:31442 2014-08-11 15:28:01.899250 2014-08-13 17:38:02.990069 7fe9bae887a0 20 read_log 192834'14377 (192834'14376) modify eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.1883634.0:20883 2014-08-11 15:28:01.890913 2014-08-13 17:38:02.990081 7fe9bae887a0 20 read_log 192834'14392 (0'0) promote a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by osd.372.0:31500 2014-08-11 15:28:12.155667 2014-08-13 17:38:02.990095 7fe9bae887a0 20 read_log 192834'14393 (192834'14392) modify a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.1883634.0:23029 2014-08-11 15:28:12.147537 2014-08-13 17:38:02.990107 7fe9bae887a0 20 read_log 192834'14404 (192834'14075) modify 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.1883634.0:24030 2014-08-11 15:28:18.262266 2014-08-13 17:38:02.990120 7fe9bae887a0 20 read_log 192834'14416 (192834'14099) modify bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.1883634.0:25291 2014-08-11 15:28:20.465682 2014-08-13 17:38:02.990134 7fe9bae887a0 20 read_log 192834'14421 (192834'14109) modify 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.1883634.0:25780 2014-08-11 15:28:21.861010 2014-08-13 17:38:02.990146 7fe9bae887a0 20 read_log 192834'14432 (192834'14131) modify 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.1883634.0:28116 2014-08-11 15:28:25.396671 2014-08-13 17:38:02.990164 7fe9bae887a0 20 read_log 192834'14444 (192834'14155) modify ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.1883634.0:30206 2014-08-11 15:28:28.228450 2014-08-13 17:38:02.990179 7fe9bae887a0 20 read_log 192834'14449 (192834'14165) modify d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.1883634.0:30396 2014-08-11 15:28:28.592812 2014-08-13 17:38:02.990191 7fe9bae887a0 20 read_log 192834'14454 (192834'14175) modify a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.1883634.0:31119 2014-08-11 15:28:29.932695 2014-08-13 17:38:02.990204 7fe9bae887a0 20 read_log 192834'14456 (192834'14179) modify eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.1883634.0:31192 2014-08-11 15:28:30.075352 2014-08-13 17:38:02.990216 7fe9bae887a0 20 read_log 192834'14459 (192834'14185) modify 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.1883634.0:31350 2014-08-11 15:28:30.363848 2014-08-13 17:38:02.990227 7fe9bae887a0 20 read_log 192834'14467 (192834'14201) modify 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.1883634.0:32822 2014-08-11 15:28:32.371063 2014-08-13 17:38:02.990242 7fe9bae887a0 20 read_log 192834'14472 (192834'14211) modify b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.1883634.0:33729 2014-08-11 15:28:33.737774 2014-08-13 17:38:02.990254 7fe9bae887a0 20 read_log 192834'14485 (192834'14237) modify 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.1883634.0:35153 2014-08-11 15:28:36.103936 2014-08-13 17:38:02.990269 7fe9bae887a0 20 read_log 192834'14488 (192834'14243) modify 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.1883634.0:35520 2014-08-11 15:28:36.773981 2014-08-13 17:38:02.990281 7fe9bae887a0 20 read_log 192834'14493 (192834'14249) modify cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.1883634.0:35849 2014-08-11 15:28:37.223281 2014-08-13 17:38:02.990293 7fe9bae887a0 20 read_log 192834'14496 (192834'14255) modify 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.1883634.0:36142 2014-08-11 15:28:37.621637 2014-08-13 17:38:02.990314 7fe9bae887a0 20 read_log 192834'14498 (192834'14259) modify 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.1883634.0:36486 2014-08-11 15:28:38.075389 2014-08-13 17:38:02.990329 7fe9bae887a0 20 read_log 192834'14507 (192834'14277) modify 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.1883634.0:37268 2014-08-11 15:28:39.095287 2014-08-13 17:38:02.990341 7fe9bae887a0 20 read_log 192834'14521 (192834'14305) modify ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.1883634.0:38730 2014-08-11 15:28:41.074582 2014-08-13 17:38:02.990353 7fe9bae887a0 20 read_log 192834'14522 (192834'14307) modify 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.1883634.0:38842 2014-08-11 15:28:41.199634 2014-08-13 17:38:02.990365 7fe9bae887a0 20 read_log 192834'14537 (192834'14337) modify 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.1883634.0:41252 2014-08-11 15:28:44.814170 2014-08-13 17:38:02.990377 7fe9bae887a0 20 read_log 192834'14541 (192834'14345) modify 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.1883634.0:42017 2014-08-11 15:28:45.638669 2014-08-13 17:38:02.990389 7fe9bae887a0 20 read_log 192834'14545 (192834'14353) modify 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.1883634.0:42362 2014-08-11 15:28:46.219619 2014-08-13 17:38:02.990401 7fe9bae887a0 20 read_log 192834'14546 (192834'14355) modify 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.1883634.0:42653 2014-08-11 15:28:46.313273 2014-08-13 17:38:02.990413 7fe9bae887a0 20 read_log 192834'14549 (192834'14361) modify be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.1883634.0:43258 2014-08-11 15:28:46.501937 2014-08-13 17:38:02.990443 7fe9bae887a0 20 read_log 192834'14556 (192834'14375) modify e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.1883634.0:44145 2014-08-11 15:28:46.774131 2014-08-13 17:38:02.990457 7fe9bae887a0 20 read_log 192834'14557 (192834'14377) modify eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.1883634.0:44207 2014-08-11 15:28:46.795364 2014-08-13 17:38:02.990469 7fe9bae887a0 20 read_log 192834'14565 (192834'14393) modify a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.1883634.0:46353 2014-08-11 15:28:47.566734 2014-08-13 17:38:02.990481 7fe9bae887a0 20 read_log 192888'14569 (0'0) promote f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by osd.296.0:2325 2014-08-11 16:45:00.353250 2014-08-13 17:38:02.990493 7fe9bae887a0 20 read_log 192888'14570 (192888'14569) modify f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.1315259.0:1334 2014-08-11 16:45:00.336957 2014-08-13 17:38:02.990507 7fe9bae887a0 20 read_log 192888'14575 (0'0) promote 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by osd.296.0:2331 2014-08-11 16:45:09.240695 2014-08-13 17:38:02.990519 7fe9bae887a0 20 read_log 192888'14576 (192888'14575) modify 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.1315259.0:3670 2014-08-11 16:45:09.233471 2014-08-13 17:38:02.990531 7fe9bae887a0 20 read_log 192888'14579 (0'0) promote 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by osd.296.0:2335 2014-08-11 16:45:13.293513 2014-08-13 17:38:02.990543 7fe9bae887a0 20 read_log 192888'14580 (192888'14579) modify 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.1315259.0:4780 2014-08-11 16:45:13.285590 2014-08-13 17:38:02.990555 7fe9bae887a0 20 read_log 192888'14583 (0'0) promote e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by osd.296.0:2339 2014-08-11 16:45:15.678497 2014-08-13 17:38:02.990577 7fe9bae887a0 20 read_log 192888'14584 (192888'14583) modify e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.1315259.0:5419 2014-08-11 16:45:15.671719 2014-08-13 17:38:02.990590 7fe9bae887a0 20 read_log 192888'14585 (0'0) promote 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by osd.296.0:2341 2014-08-11 16:45:16.764595 2014-08-13 17:38:02.990602 7fe9bae887a0 20 read_log 192888'14586 (192888'14585) modify 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.1315259.0:5718 2014-08-11 16:45:16.757542 2014-08-13 17:38:02.990636 7fe9bae887a0 20 read_log 192888'14587 (0'0) promote 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by osd.296.0:2343 2014-08-11 16:45:20.922722 2014-08-13 17:38:02.990649 7fe9bae887a0 20 read_log 192888'14588 (192888'14587) modify 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.1315259.0:6869 2014-08-11 16:45:20.915355 2014-08-13 17:38:02.990661 7fe9bae887a0 20 read_log 192888'14589 (0'0) promote e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by osd.296.0:2345 2014-08-11 16:45:30.233888 2014-08-13 17:38:02.990673 7fe9bae887a0 20 read_log 192888'14590 (192888'14589) modify e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.1315259.0:9379 2014-08-11 16:45:30.225308 2014-08-13 17:38:02.990685 7fe9bae887a0 20 read_log 192888'14595 (0'0) promote a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by osd.296.0:2351 2014-08-11 16:45:34.040290 2014-08-13 17:38:02.990697 7fe9bae887a0 20 read_log 192888'14596 (192888'14595) modify a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.1315259.0:10363 2014-08-11 16:45:34.033118 2014-08-13 17:38:02.990709 7fe9bae887a0 20 read_log 192888'14597 (0'0) promote d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by osd.296.0:2353 2014-08-11 16:45:35.658200 2014-08-13 17:38:02.990722 7fe9bae887a0 20 read_log 192888'14598 (192888'14597) modify d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.1315259.0:10760 2014-08-11 16:45:35.649339 2014-08-13 17:38:02.990734 7fe9bae887a0 20 read_log 192888'14607 (0'0) promote 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by osd.296.0:2363 2014-08-11 16:45:49.965880 2014-08-13 17:38:02.990745 7fe9bae887a0 20 read_log 192888'14608 (192888'14607) modify 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.1315259.0:14301 2014-08-11 16:45:49.957333 2014-08-13 17:38:02.990757 7fe9bae887a0 20 read_log 192888'14609 (0'0) promote 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by osd.296.0:2365 2014-08-11 16:45:50.653718 2014-08-13 17:38:02.990769 7fe9bae887a0 20 read_log 192888'14610 (192888'14609) modify 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.1315259.0:14489 2014-08-11 16:45:50.647225 2014-08-13 17:38:02.990781 7fe9bae887a0 20 read_log 192888'14617 (0'0) promote 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by osd.296.0:2373 2014-08-11 16:46:07.923625 2014-08-13 17:38:02.990792 7fe9bae887a0 20 read_log 192888'14618 (192888'14617) modify 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.1315259.0:18935 2014-08-11 16:46:07.917642 2014-08-13 17:38:02.990804 7fe9bae887a0 20 read_log 192888'14619 (0'0) promote 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by osd.296.0:2375 2014-08-11 16:46:08.299473 2014-08-13 17:38:02.990816 7fe9bae887a0 20 read_log 192888'14620 (192888'14619) modify 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.1315259.0:19014 2014-08-11 16:46:08.292465 2014-08-13 17:38:02.990828 7fe9bae887a0 20 read_log 192888'14627 (0'0) promote a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by osd.296.0:2383 2014-08-11 16:46:19.879634 2014-08-13 17:38:02.990847 7fe9bae887a0 20 read_log 192888'14628 (192888'14627) modify a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.1315259.0:22059 2014-08-11 16:46:19.871353 2014-08-13 17:38:02.990861 7fe9bae887a0 20 read_log 192888'14629 (0'0) promote 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by osd.296.0:2385 2014-08-11 16:46:20.263146 2014-08-13 17:38:02.990873 7fe9bae887a0 20 read_log 192888'14630 (192888'14629) modify 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.1315259.0:22142 2014-08-11 16:46:20.256768 2014-08-13 17:38:02.990885 7fe9bae887a0 20 read_log 192888'14637 (0'0) promote e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by osd.296.0:2393 2014-08-11 16:46:26.322618 2014-08-13 17:38:02.990897 7fe9bae887a0 20 read_log 192888'14638 (192888'14637) modify e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.1315259.0:23649 2014-08-11 16:46:26.315367 2014-08-13 17:38:02.990909 7fe9bae887a0 20 read_log 192888'14641 (0'0) promote 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by osd.296.0:2397 2014-08-11 16:46:27.773121 2014-08-13 17:38:02.990921 7fe9bae887a0 20 read_log 192888'14642 (192888'14641) modify 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.1315259.0:24035 2014-08-11 16:46:27.766588 2014-08-13 17:38:02.990933 7fe9bae887a0 20 read_log 192888'14647 (0'0) promote 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by osd.296.0:2403 2014-08-11 16:46:40.488543 2014-08-13 17:38:02.990944 7fe9bae887a0 20 read_log 192888'14648 (192888'14647) modify 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.1315259.0:27112 2014-08-11 16:46:40.479513 2014-08-13 17:38:02.990956 7fe9bae887a0 20 read_log 192888'14649 (0'0) promote 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by osd.296.0:2405 2014-08-11 16:46:42.764523 2014-08-13 17:38:02.990968 7fe9bae887a0 20 read_log 192888'14650 (192888'14649) modify 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.1315259.0:27693 2014-08-11 16:46:42.757427 2014-08-13 17:38:02.990980 7fe9bae887a0 20 read_log 192888'14651 (0'0) promote 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by osd.296.0:2407 2014-08-11 16:46:42.799372 2014-08-13 17:38:02.990996 7fe9bae887a0 20 read_log 192888'14652 (192888'14651) modify 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.1315259.0:27702 2014-08-11 16:46:42.792916 2014-08-13 17:38:02.991008 7fe9bae887a0 20 read_log 192888'14657 (0'0) promote 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by osd.296.0:2413 2014-08-11 16:46:49.693938 2014-08-13 17:38:02.991020 7fe9bae887a0 20 read_log 192888'14658 (192888'14657) modify 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.1315259.0:29412 2014-08-11 16:46:49.685707 2014-08-13 17:38:02.991032 7fe9bae887a0 20 read_log 192888'14663 (0'0) promote 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by osd.296.0:2419 2014-08-11 16:46:54.351232 2014-08-13 17:38:02.991044 7fe9bae887a0 20 read_log 192888'14664 (192888'14663) modify 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.1315259.0:30593 2014-08-11 16:46:54.343627 2014-08-13 17:38:02.991056 7fe9bae887a0 20 read_log 192888'14665 (192888'14570) modify f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.1315259.0:32061 2014-08-11 16:46:56.280204 2014-08-13 17:38:02.991068 7fe9bae887a0 20 read_log 192888'14668 (192888'14576) modify 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.1315259.0:34397 2014-08-11 16:46:56.882272 2014-08-13 17:38:02.991080 7fe9bae887a0 20 read_log 192888'14670 (192888'14580) modify 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.1315259.0:35507 2014-08-11 16:46:57.178795 2014-08-13 17:38:02.991099 7fe9bae887a0 20 read_log 192888'14672 (192888'14584) modify e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.1315259.0:36146 2014-08-11 16:46:57.348196 2014-08-13 17:38:02.991112 7fe9bae887a0 20 read_log 192888'14673 (192888'14586) modify 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.1315259.0:36445 2014-08-11 16:46:57.424701 2014-08-13 17:38:02.991124 7fe9bae887a0 20 read_log 192888'14674 (192888'14588) modify 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.1315259.0:37596 2014-08-11 16:46:57.725370 2014-08-13 17:38:02.991136 7fe9bae887a0 20 read_log 192888'14675 (192888'14590) modify e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.1315259.0:40106 2014-08-11 16:46:58.390803 2014-08-13 17:38:02.991148 7fe9bae887a0 20 read_log 192888'14678 (192888'14596) modify a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.1315259.0:41090 2014-08-11 16:46:58.656180 2014-08-13 17:38:02.991165 7fe9bae887a0 20 read_log 192888'14679 (192888'14598) modify d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.1315259.0:41487 2014-08-11 16:46:58.758401 2014-08-13 17:38:02.991178 7fe9bae887a0 20 read_log 192888'14684 (192888'14608) modify 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.1315259.0:45028 2014-08-11 16:46:59.748910 2014-08-13 17:38:02.991190 7fe9bae887a0 20 read_log 192888'14685 (192888'14610) modify 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.1315259.0:45216 2014-08-11 16:46:59.803345 2014-08-13 17:38:02.991201 7fe9bae887a0 20 read_log 192888'14689 (192888'14618) modify 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.1315259.0:49662 2014-08-11 16:47:01.135983 2014-08-13 17:38:02.991213 7fe9bae887a0 20 read_log 192888'14690 (192888'14620) modify 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.1315259.0:49741 2014-08-11 16:47:01.158818 2014-08-13 17:38:02.991225 7fe9bae887a0 20 read_log 192888'14694 (192888'14628) modify a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.1315259.0:52786 2014-08-11 16:47:02.068769 2014-08-13 17:38:02.991237 7fe9bae887a0 20 read_log 192888'14695 (192888'14630) modify 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.1315259.0:52869 2014-08-11 16:47:02.092484 2014-08-13 17:38:02.991249 7fe9bae887a0 20 read_log 192888'14699 (192888'14638) modify e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.1315259.0:54376 2014-08-11 16:47:02.549643 2014-08-13 17:38:02.991261 7fe9bae887a0 20 read_log 192888'14701 (192888'14642) modify 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.1315259.0:54762 2014-08-11 16:47:02.686222 2014-08-13 17:38:02.991273 7fe9bae887a0 20 read_log 192888'14704 (192888'14648) modify 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.1315259.0:57839 2014-08-11 16:47:03.628121 2014-08-13 17:38:02.991284 7fe9bae887a0 20 read_log 192888'14705 (192888'14650) modify 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.1315259.0:58420 2014-08-11 16:47:03.796443 2014-08-13 17:38:02.991297 7fe9bae887a0 20 read_log 192888'14706 (192888'14652) modify 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.1315259.0:58429 2014-08-11 16:47:03.799517 2014-08-13 17:38:02.991309 7fe9bae887a0 20 read_log 192888'14709 (192888'14658) modify 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.1315259.0:60139 2014-08-11 16:47:04.355840 2014-08-13 17:38:02.991320 7fe9bae887a0 20 read_log 192888'14712 (192888'14664) modify 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.1315259.0:61320 2014-08-11 16:47:04.703546 2014-08-13 17:38:02.991332 7fe9bae887a0 20 read_log 192888'14715 (0'0) promote aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by osd.296.0:2470 2014-08-11 17:05:15.027834 2014-08-13 17:38:02.991354 7fe9bae887a0 20 read_log 192888'14716 (192888'14715) modify aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.942868.0:50 2014-08-11 17:05:15.017412 2014-08-13 17:38:02.991368 7fe9bae887a0 20 read_log 192888'14721 (0'0) promote 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by osd.296.0:2476 2014-08-11 17:05:21.587792 2014-08-13 17:38:02.991380 7fe9bae887a0 20 read_log 192888'14722 (192888'14721) modify 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.942868.0:1590 2014-08-11 17:05:21.579251 2014-08-13 17:38:02.991392 7fe9bae887a0 20 read_log 192888'14723 (0'0) promote 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by osd.296.0:2478 2014-08-11 17:05:23.420991 2014-08-13 17:38:02.991404 7fe9bae887a0 20 read_log 192888'14724 (192888'14723) modify 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.942868.0:2059 2014-08-11 17:05:23.413425 2014-08-13 17:38:02.991416 7fe9bae887a0 20 read_log 192888'14725 (0'0) promote 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by osd.296.0:2480 2014-08-11 17:05:27.975928 2014-08-13 17:38:02.991428 7fe9bae887a0 20 read_log 192888'14726 (192888'14725) modify 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.942868.0:3252 2014-08-11 17:05:27.966253 2014-08-13 17:38:02.991440 7fe9bae887a0 20 read_log 192888'14729 (0'0) promote 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by osd.296.0:2484 2014-08-11 17:05:28.717614 2014-08-13 17:38:02.991452 7fe9bae887a0 20 read_log 192888'14730 (192888'14729) modify 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.942868.0:3454 2014-08-11 17:05:28.709618 2014-08-13 17:38:02.991464 7fe9bae887a0 20 read_log 192888'14733 (0'0) promote 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by osd.296.0:2488 2014-08-11 17:05:34.020762 2014-08-13 17:38:02.991476 7fe9bae887a0 20 read_log 192888'14734 (192888'14733) modify 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.942868.0:4860 2014-08-11 17:05:34.013231 2014-08-13 17:38:02.991488 7fe9bae887a0 20 read_log 192888'14743 (0'0) promote 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by osd.296.0:2498 2014-08-11 17:05:39.534276 2014-08-13 17:38:02.991499 7fe9bae887a0 20 read_log 192888'14744 (192888'14743) modify 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.942868.0:6337 2014-08-11 17:05:39.526134 2014-08-13 17:38:02.991511 7fe9bae887a0 20 read_log 192888'14745 (0'0) promote f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by osd.296.0:2500 2014-08-11 17:05:39.673797 2014-08-13 17:38:02.991523 7fe9bae887a0 20 read_log 192888'14746 (192888'14745) modify f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.942868.0:6376 2014-08-11 17:05:39.668171 2014-08-13 17:38:02.991535 7fe9bae887a0 20 read_log 192888'14749 (0'0) promote 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by osd.296.0:2504 2014-08-11 17:05:44.449483 2014-08-13 17:38:02.991549 7fe9bae887a0 20 read_log 192888'14750 (192888'14749) modify 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.942868.0:7618 2014-08-11 17:05:44.441850 2014-08-13 17:38:02.991561 7fe9bae887a0 20 read_log 192888'14751 (0'0) promote dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by osd.296.0:2506 2014-08-11 17:05:45.032638 2014-08-13 17:38:02.991573 7fe9bae887a0 20 read_log 192888'14752 (192888'14751) modify dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.942868.0:7780 2014-08-11 17:05:45.025885 2014-08-13 17:38:02.991585 7fe9bae887a0 20 read_log 192888'14753 (0'0) promote cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by osd.296.0:2508 2014-08-11 17:05:45.829366 2014-08-13 17:38:02.991620 7fe9bae887a0 20 read_log 192888'14754 (192888'14753) modify cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.942868.0:8001 2014-08-11 17:05:45.822674 2014-08-13 17:38:02.991633 7fe9bae887a0 20 read_log 192888'14757 (0'0) promote ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by osd.296.0:2512 2014-08-11 17:05:48.191038 2014-08-13 17:38:02.991646 7fe9bae887a0 20 read_log 192888'14758 (192888'14757) modify ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.942868.0:8654 2014-08-11 17:05:48.182545 2014-08-13 17:38:02.991658 7fe9bae887a0 20 read_log 192888'14761 (0'0) promote c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by osd.296.0:2516 2014-08-11 17:05:56.170686 2014-08-13 17:38:02.991670 7fe9bae887a0 20 read_log 192888'14762 (192888'14761) modify c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.942868.0:10825 2014-08-11 17:05:56.163465 2014-08-13 17:38:02.991684 7fe9bae887a0 20 read_log 192888'14765 (0'0) promote aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by osd.296.0:2520 2014-08-11 17:05:59.380916 2014-08-13 17:38:02.991696 7fe9bae887a0 20 read_log 192888'14766 (192888'14765) modify aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.942868.0:11699 2014-08-11 17:05:59.374644 2014-08-13 17:38:02.991708 7fe9bae887a0 20 read_log 192888'14769 (0'0) promote 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by osd.296.0:2524 2014-08-11 17:06:02.272971 2014-08-13 17:38:02.991720 7fe9bae887a0 20 read_log 192888'14770 (192888'14769) modify 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.942868.0:12494 2014-08-11 17:06:02.265943 2014-08-13 17:38:02.991732 7fe9bae887a0 20 read_log 192888'14773 (0'0) promote 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by osd.296.0:2528 2014-08-11 17:06:08.508392 2014-08-13 17:38:02.991744 7fe9bae887a0 20 read_log 192888'14774 (192888'14773) modify 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.942868.0:14068 2014-08-11 17:06:08.501394 2014-08-13 17:38:02.991759 7fe9bae887a0 20 read_log 192888'14775 (0'0) promote 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by osd.296.0:2530 2014-08-11 17:06:08.850292 2014-08-13 17:38:02.991771 7fe9bae887a0 20 read_log 192888'14776 (192888'14775) modify 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.942868.0:14152 2014-08-11 17:06:08.843598 2014-08-13 17:38:02.991783 7fe9bae887a0 20 read_log 192888'14781 (0'0) promote 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by osd.296.0:2536 2014-08-11 17:06:15.348129 2014-08-13 17:38:02.991795 7fe9bae887a0 20 read_log 192888'14782 (192888'14781) modify 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.942868.0:15803 2014-08-11 17:06:15.340211 2014-08-13 17:38:02.991807 7fe9bae887a0 20 read_log 192888'14783 (0'0) promote e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by osd.296.0:2538 2014-08-11 17:06:16.699901 2014-08-13 17:38:02.991837 7fe9bae887a0 20 read_log 192888'14784 (192888'14783) modify e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.942868.0:16154 2014-08-11 17:06:16.691337 2014-08-13 17:38:02.991850 7fe9bae887a0 20 read_log 192888'14787 (0'0) promote d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by osd.296.0:2542 2014-08-11 17:06:19.730195 2014-08-13 17:38:02.991862 7fe9bae887a0 20 read_log 192888'14788 (192888'14787) modify d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.942868.0:16911 2014-08-11 17:06:19.719683 2014-08-13 17:38:02.991874 7fe9bae887a0 20 read_log 192888'14795 (0'0) promote 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by osd.296.0:2550 2014-08-11 17:06:23.909781 2014-08-13 17:38:02.991893 7fe9bae887a0 20 read_log 192888'14796 (192888'14795) modify 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.942868.0:18021 2014-08-11 17:06:23.903005 2014-08-13 17:38:02.991906 7fe9bae887a0 20 read_log 192888'14797 (0'0) promote 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by osd.296.0:2552 2014-08-11 17:06:24.144016 2014-08-13 17:38:02.991918 7fe9bae887a0 20 read_log 192888'14798 (192888'14797) modify 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.942868.0:18083 2014-08-11 17:06:24.137346 2014-08-13 17:38:02.991929 7fe9bae887a0 20 read_log 192888'14805 (0'0) promote e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by osd.296.0:2560 2014-08-11 17:06:36.053544 2014-08-13 17:38:02.991941 7fe9bae887a0 20 read_log 192888'14806 (192888'14805) modify e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.942868.0:21215 2014-08-11 17:06:36.047096 2014-08-13 17:38:02.991953 7fe9bae887a0 20 read_log 192888'14807 (0'0) promote 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by osd.296.0:2562 2014-08-11 17:06:36.312146 2014-08-13 17:38:02.991965 7fe9bae887a0 20 read_log 192888'14808 (192888'14807) modify 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.942868.0:21285 2014-08-11 17:06:36.305555 2014-08-13 17:38:02.991977 7fe9bae887a0 20 read_log 192888'14809 (0'0) promote 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by osd.296.0:2564 2014-08-11 17:06:37.218916 2014-08-13 17:38:02.991989 7fe9bae887a0 20 read_log 192888'14810 (192888'14809) modify 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.942868.0:21529 2014-08-11 17:06:37.212501 2014-08-13 17:38:02.992001 7fe9bae887a0 20 read_log 192888'14811 (0'0) promote aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by osd.296.0:2566 2014-08-11 17:06:37.390625 2014-08-13 17:38:02.992013 7fe9bae887a0 20 read_log 192888'14812 (192888'14811) modify aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.942868.0:21575 2014-08-11 17:06:37.383476 2014-08-13 17:38:02.992025 7fe9bae887a0 20 read_log 192888'14813 (0'0) promote 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by osd.296.0:2568 2014-08-11 17:06:41.624724 2014-08-13 17:38:02.992044 7fe9bae887a0 20 read_log 192888'14814 (192888'14813) modify 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.942868.0:22665 2014-08-11 17:06:41.617733 2014-08-13 17:38:02.992056 7fe9bae887a0 20 read_log 192888'14817 (0'0) promote 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by osd.296.0:2572 2014-08-11 17:06:44.632273 2014-08-13 17:38:02.992068 7fe9bae887a0 20 read_log 192888'14818 (192888'14817) modify 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.942868.0:23436 2014-08-11 17:06:44.625170 2014-08-13 17:38:02.992080 7fe9bae887a0 20 read_log 192888'14819 (0'0) promote 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by osd.296.0:2574 2014-08-11 17:06:46.265673 2014-08-13 17:38:02.992092 7fe9bae887a0 20 read_log 192888'14820 (192888'14819) modify 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.942868.0:23879 2014-08-11 17:06:46.259401 2014-08-13 17:38:02.992104 7fe9bae887a0 20 read_log 192888'14823 (0'0) promote 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by osd.296.0:2578 2014-08-11 17:06:49.847754 2014-08-13 17:38:02.992116 7fe9bae887a0 20 read_log 192888'14824 (192888'14823) modify 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.942868.0:24847 2014-08-11 17:06:49.838674 2014-08-13 17:38:02.992128 7fe9bae887a0 20 read_log 192888'14825 (0'0) promote 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by osd.296.0:2580 2014-08-11 17:06:50.866167 2014-08-13 17:38:02.992150 7fe9bae887a0 20 read_log 192888'14826 (192888'14825) modify 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.942868.0:25122 2014-08-11 17:06:50.858902 2014-08-13 17:38:02.992168 7fe9bae887a0 20 read_log 192888'14829 (0'0) promote 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by osd.296.0:2584 2014-08-11 17:06:54.429010 2014-08-13 17:38:02.992181 7fe9bae887a0 20 read_log 192888'14830 (192888'14829) modify 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.942868.0:26008 2014-08-11 17:06:54.422492 2014-08-13 17:38:02.992193 7fe9bae887a0 20 read_log 192888'14833 (0'0) promote 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by osd.296.0:2588 2014-08-11 17:06:57.883108 2014-08-13 17:38:02.992205 7fe9bae887a0 20 read_log 192888'14834 (192888'14833) modify 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.942868.0:26906 2014-08-11 17:06:57.873918 2014-08-13 17:38:02.992217 7fe9bae887a0 20 read_log 192888'14835 (0'0) promote 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by osd.296.0:2590 2014-08-11 17:07:01.593576 2014-08-13 17:38:02.992231 7fe9bae887a0 20 read_log 192888'14836 (192888'14835) modify 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.942868.0:27891 2014-08-11 17:07:01.586538 2014-08-13 17:38:02.992243 7fe9bae887a0 20 read_log 192888'14839 (0'0) promote 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by osd.296.0:2594 2014-08-11 17:07:01.967636 2014-08-13 17:38:02.992255 7fe9bae887a0 20 read_log 192888'14840 (192888'14839) modify 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.942868.0:27992 2014-08-11 17:07:01.958056 2014-08-13 17:38:02.992267 7fe9bae887a0 20 read_log 192888'14841 (0'0) promote d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by osd.296.0:2596 2014-08-11 17:07:08.242621 2014-08-13 17:38:02.992279 7fe9bae887a0 20 read_log 192888'14842 (192888'14841) modify d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.942868.0:29606 2014-08-11 17:07:08.234868 2014-08-13 17:38:02.992292 7fe9bae887a0 20 read_log 192888'14843 (0'0) promote d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by osd.296.0:2598 2014-08-11 17:07:08.394110 2014-08-13 17:38:02.992304 7fe9bae887a0 20 read_log 192888'14844 (192888'14843) modify d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.942868.0:29645 2014-08-11 17:07:08.387361 2014-08-13 17:38:02.992320 7fe9bae887a0 20 read_log 192888'14845 (0'0) promote d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by osd.296.0:2600 2014-08-11 17:07:09.905259 2014-08-13 17:38:02.992334 7fe9bae887a0 20 read_log 192888'14846 (192888'14845) modify d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.942868.0:30024 2014-08-11 17:07:09.897719 2014-08-13 17:38:02.992347 7fe9bae887a0 20 read_log 192888'14849 (192888'14716) modify aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.942868.0:31316 2014-08-11 17:07:15.868564 2014-08-13 17:38:02.992359 7fe9bae887a0 20 read_log 192888'14852 (192888'14722) modify 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.942868.0:32856 2014-08-11 17:07:16.362010 2014-08-13 17:38:02.992371 7fe9bae887a0 20 read_log 192888'14853 (192888'14724) modify 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.942868.0:33325 2014-08-11 17:07:16.486404 2014-08-13 17:38:02.992383 7fe9bae887a0 20 read_log 192888'14854 (192888'14726) modify 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.942868.0:34518 2014-08-11 17:07:16.811412 2014-08-13 17:38:02.992395 7fe9bae887a0 20 read_log 192888'14856 (192888'14730) modify 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.942868.0:34720 2014-08-11 17:07:16.873952 2014-08-13 17:38:02.992414 7fe9bae887a0 20 read_log 192888'14858 (192888'14734) modify 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.942868.0:36126 2014-08-11 17:07:17.248739 2014-08-13 17:38:02.992427 7fe9bae887a0 20 read_log 192888'14863 (192888'14744) modify 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.942868.0:37603 2014-08-11 17:07:17.656265 2014-08-13 17:38:02.992439 7fe9bae887a0 20 read_log 192888'14864 (192888'14746) modify f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.942868.0:37642 2014-08-11 17:07:17.667265 2014-08-13 17:38:02.992451 7fe9bae887a0 20 read_log 192888'14866 (192888'14750) modify 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.942868.0:38884 2014-08-11 17:07:18.019146 2014-08-13 17:38:02.992463 7fe9bae887a0 20 read_log 192888'14867 (192888'14752) modify dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.942868.0:39046 2014-08-11 17:07:18.064668 2014-08-13 17:38:02.992474 7fe9bae887a0 20 read_log 192888'14868 (192888'14754) modify cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.942868.0:39267 2014-08-11 17:07:18.134463 2014-08-13 17:38:02.992486 7fe9bae887a0 20 read_log 192888'14870 (192888'14758) modify ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.942868.0:39920 2014-08-11 17:07:18.319867 2014-08-13 17:38:02.992507 7fe9bae887a0 20 read_log 192888'14872 (192888'14762) modify c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.942868.0:42091 2014-08-11 17:07:18.902102 2014-08-13 17:38:02.992520 7fe9bae887a0 20 read_log 192888'14874 (192888'14766) modify aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.942868.0:42965 2014-08-11 17:07:19.140046 2014-08-13 17:38:02.992532 7fe9bae887a0 20 read_log 192888'14876 (192888'14770) modify 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.942868.0:43760 2014-08-11 17:07:19.366300 2014-08-13 17:38:02.992544 7fe9bae887a0 20 read_log 192888'14878 (192888'14774) modify 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.942868.0:45334 2014-08-11 17:07:19.809199 2014-08-13 17:38:02.992556 7fe9bae887a0 20 read_log 192888'14879 (192888'14776) modify 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.942868.0:45418 2014-08-11 17:07:19.832702 2014-08-13 17:38:02.992568 7fe9bae887a0 20 read_log 192888'14882 (192888'14782) modify 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.942868.0:47069 2014-08-11 17:07:20.299038 2014-08-13 17:38:02.992580 7fe9bae887a0 20 read_log 192888'14883 (192888'14784) modify e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.942868.0:47420 2014-08-11 17:07:20.400153 2014-08-13 17:38:02.992592 7fe9bae887a0 20 read_log 192888'14885 (192888'14788) modify d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.942868.0:48177 2014-08-11 17:07:20.646214 2014-08-13 17:38:02.992604 7fe9bae887a0 20 read_log 192888'14889 (192888'14796) modify 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.942868.0:49287 2014-08-11 17:07:20.965957 2014-08-13 17:38:02.992616 7fe9bae887a0 20 read_log 192888'14890 (192888'14798) modify 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.942868.0:49349 2014-08-11 17:07:20.986954 2014-08-13 17:38:02.992628 7fe9bae887a0 20 read_log 192888'14894 (192888'14806) modify e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.942868.0:52481 2014-08-11 17:07:21.955242 2014-08-13 17:38:02.992640 7fe9bae887a0 20 read_log 192888'14895 (192888'14808) modify 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.942868.0:52551 2014-08-11 17:07:21.976572 2014-08-13 17:38:02.992652 7fe9bae887a0 20 read_log 192888'14896 (192888'14810) modify 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.942868.0:52795 2014-08-11 17:07:22.053566 2014-08-13 17:38:02.992672 7fe9bae887a0 20 read_log 192888'14897 (192888'14812) modify aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.942868.0:52841 2014-08-11 17:07:22.066314 2014-08-13 17:38:02.992685 7fe9bae887a0 20 read_log 192888'14898 (192888'14814) modify 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.942868.0:53931 2014-08-11 17:07:22.419265 2014-08-13 17:38:02.992697 7fe9bae887a0 20 read_log 192888'14900 (192888'14818) modify 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.942868.0:54702 2014-08-11 17:07:22.633185 2014-08-13 17:38:02.992709 7fe9bae887a0 20 read_log 192888'14901 (192888'14820) modify 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.942868.0:55145 2014-08-11 17:07:22.756321 2014-08-13 17:38:02.992721 7fe9bae887a0 20 read_log 192888'14903 (192888'14824) modify 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.942868.0:56113 2014-08-11 17:07:23.056590 2014-08-13 17:38:02.992733 7fe9bae887a0 20 read_log 192888'14904 (192888'14826) modify 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.942868.0:56388 2014-08-11 17:07:23.134804 2014-08-13 17:38:02.992745 7fe9bae887a0 20 read_log 192888'14906 (192888'14830) modify 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.942868.0:57274 2014-08-11 17:07:23.409198 2014-08-13 17:38:02.992757 7fe9bae887a0 20 read_log 192888'14908 (192888'14834) modify 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.942868.0:58172 2014-08-11 17:07:23.663235 2014-08-13 17:38:02.992769 7fe9bae887a0 20 read_log 192888'14909 (192888'14836) modify 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.942868.0:59157 2014-08-11 17:07:23.961610 2014-08-13 17:38:02.992781 7fe9bae887a0 20 read_log 192888'14911 (192888'14840) modify 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.942868.0:59258 2014-08-11 17:07:23.992276 2014-08-13 17:38:02.992793 7fe9bae887a0 20 read_log 192888'14912 (192888'14842) modify d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.942868.0:60872 2014-08-11 17:07:24.471393 2014-08-13 17:38:02.992805 7fe9bae887a0 20 read_log 192888'14913 (192888'14844) modify d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.942868.0:60911 2014-08-11 17:07:24.481538 2014-08-13 17:38:02.992816 7fe9bae887a0 20 read_log 192888'14914 (192888'14846) modify d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.942868.0:61290 2014-08-11 17:07:24.596675 2014-08-13 17:38:02.992828 7fe9bae887a0 20 read_log 193305'14916 (0'0) promote 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by osd.396.0:4 2014-08-11 18:42:34.185500 2014-08-13 17:38:02.992840 7fe9bae887a0 20 read_log 193305'14917 (193305'14916) modify 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.951337.0:321 2014-08-11 18:42:34.177622 2014-08-13 17:38:02.992852 7fe9bae887a0 20 read_log 193305'14918 (0'0) promote 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by osd.396.0:15 2014-08-11 18:42:40.889235 2014-08-13 17:38:02.992864 7fe9bae887a0 20 read_log 193305'14919 (193305'14918) modify 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.951337.0:2035 2014-08-11 18:42:40.880420 2014-08-13 17:38:02.992876 7fe9bae887a0 20 read_log 193305'14920 (0'0) promote 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by osd.396.0:23 2014-08-11 18:42:46.919282 2014-08-13 17:38:02.992888 7fe9bae887a0 20 read_log 193305'14921 (193305'14920) modify 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.951337.0:3697 2014-08-11 18:42:46.910275 2014-08-13 17:38:02.992900 7fe9bae887a0 20 read_log 193305'14922 (0'0) promote 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by osd.396.0:31 2014-08-11 18:42:52.450498 2014-08-13 17:38:02.992921 7fe9bae887a0 20 read_log 193305'14923 (193305'14922) modify 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.951337.0:5180 2014-08-11 18:42:52.442427 2014-08-13 17:38:02.992934 7fe9bae887a0 20 read_log 193305'14924 (0'0) promote 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by osd.396.0:33 2014-08-11 18:42:52.911166 2014-08-13 17:38:02.992946 7fe9bae887a0 20 read_log 193305'14925 (193305'14924) modify 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.951337.0:5307 2014-08-11 18:42:52.903810 2014-08-13 17:38:02.992958 7fe9bae887a0 20 read_log 193305'14926 (0'0) promote e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by osd.396.0:35 2014-08-11 18:42:55.381751 2014-08-13 17:38:02.992970 7fe9bae887a0 20 read_log 193305'14927 (193305'14926) modify e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.951337.0:5988 2014-08-11 18:42:55.375708 2014-08-13 17:38:02.992982 7fe9bae887a0 20 read_log 193305'14928 (0'0) promote b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by osd.396.0:39 2014-08-11 18:42:56.683179 2014-08-13 17:38:02.992995 7fe9bae887a0 20 read_log 193305'14929 (193305'14928) modify b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.951337.0:6346 2014-08-11 18:42:56.676584 2014-08-13 17:38:02.993008 7fe9bae887a0 20 read_log 193305'14930 (0'0) promote 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by osd.396.0:57 2014-08-11 18:43:07.393194 2014-08-13 17:38:02.993020 7fe9bae887a0 20 read_log 193305'14931 (193305'14930) modify 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.951337.0:9210 2014-08-11 18:43:07.386043 2014-08-13 17:38:02.993050 7fe9bae887a0 20 read_log 193305'14932 (0'0) promote 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by osd.396.0:73 2014-08-11 18:43:18.088415 2014-08-13 17:38:02.993063 7fe9bae887a0 20 read_log 193305'14933 (193305'14932) modify 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.951337.0:12054 2014-08-11 18:43:18.080851 2014-08-13 17:38:02.993075 7fe9bae887a0 20 read_log 193305'14934 (0'0) promote 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by osd.396.0:77 2014-08-11 18:43:20.182722 2014-08-13 17:38:02.993088 7fe9bae887a0 20 read_log 193305'14935 (193305'14934) modify 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.951337.0:12622 2014-08-11 18:43:20.171564 2014-08-13 17:38:02.993100 7fe9bae887a0 20 read_log 193305'14936 (0'0) promote e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by osd.396.0:93 2014-08-11 18:43:28.947327 2014-08-13 17:38:02.993112 7fe9bae887a0 20 read_log 193305'14937 (193305'14936) modify e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.951337.0:14989 2014-08-11 18:43:28.939175 2014-08-13 17:38:02.993124 7fe9bae887a0 20 read_log 193305'14938 (0'0) promote e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by osd.396.0:95 2014-08-11 18:43:29.215693 2014-08-13 17:38:02.993136 7fe9bae887a0 20 read_log 193305'14939 (193305'14938) modify e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.951337.0:15071 2014-08-11 18:43:29.206782 2014-08-13 17:38:02.993150 7fe9bae887a0 20 read_log 193305'14940 (0'0) promote 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by osd.396.0:97 2014-08-11 18:43:30.936131 2014-08-13 17:38:02.993169 7fe9bae887a0 20 read_log 193305'14941 (193305'14940) modify 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.951337.0:15584 2014-08-11 18:43:30.927084 2014-08-13 17:38:02.993181 7fe9bae887a0 20 read_log 193305'14942 (0'0) promote fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by osd.396.0:105 2014-08-11 18:43:32.913926 2014-08-13 17:38:02.993201 7fe9bae887a0 20 read_log 193305'14943 (193305'14942) modify fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.951337.0:16149 2014-08-11 18:43:32.906088 2014-08-13 17:38:02.993214 7fe9bae887a0 20 read_log 193305'14944 (0'0) promote c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by osd.396.0:111 2014-08-11 18:43:35.373431 2014-08-13 17:38:02.993225 7fe9bae887a0 20 read_log 193305'14945 (193305'14944) modify c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.951337.0:16852 2014-08-11 18:43:35.365248 2014-08-13 17:38:02.993237 7fe9bae887a0 20 read_log 193305'14946 (0'0) promote d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by osd.396.0:125 2014-08-11 18:43:40.113971 2014-08-13 17:38:02.993249 7fe9bae887a0 20 read_log 193305'14947 (193305'14946) modify d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.951337.0:18218 2014-08-11 18:43:40.105175 2014-08-13 17:38:02.993261 7fe9bae887a0 20 read_log 193305'14948 (0'0) promote 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by osd.396.0:129 2014-08-11 18:43:41.044716 2014-08-13 17:38:02.993273 7fe9bae887a0 20 read_log 193305'14949 (193305'14948) modify 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.951337.0:18475 2014-08-11 18:43:41.034915 2014-08-13 17:38:02.993285 7fe9bae887a0 20 read_log 193305'14950 (0'0) promote 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by osd.396.0:135 2014-08-11 18:43:45.027307 2014-08-13 17:38:02.993297 7fe9bae887a0 20 read_log 193305'14951 (193305'14950) modify 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.951337.0:19547 2014-08-11 18:43:45.018810 2014-08-13 17:38:02.993309 7fe9bae887a0 20 read_log 193305'14952 (0'0) promote d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by osd.396.0:137 2014-08-11 18:43:45.252035 2014-08-13 17:38:02.993321 7fe9bae887a0 20 read_log 193305'14953 (193305'14952) modify d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.951337.0:19612 2014-08-11 18:43:45.243841 2014-08-13 17:38:02.993333 7fe9bae887a0 20 read_log 193305'14954 (0'0) promote de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by osd.396.0:139 2014-08-11 18:43:49.321032 2014-08-13 17:38:02.993344 7fe9bae887a0 20 read_log 193305'14955 (193305'14954) modify de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.951337.0:20753 2014-08-11 18:43:49.313548 2014-08-13 17:38:02.993356 7fe9bae887a0 20 read_log 193305'14956 (0'0) promote 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by osd.396.0:141 2014-08-11 18:43:50.417737 2014-08-13 17:38:02.993368 7fe9bae887a0 20 read_log 193305'14957 (193305'14956) modify 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.951337.0:21054 2014-08-11 18:43:50.408740 2014-08-13 17:38:02.993380 7fe9bae887a0 20 read_log 193305'14958 (0'0) promote 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by osd.396.0:145 2014-08-11 18:43:51.359499 2014-08-13 17:38:02.993392 7fe9bae887a0 20 read_log 193305'14959 (193305'14958) modify 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.951337.0:21322 2014-08-11 18:43:51.350728 2014-08-13 17:38:02.993404 7fe9bae887a0 20 read_log 193305'14960 (193305'14960) modify 299/hit_set_24.299_archive_2014-08-11 18:38:51.580559_2014-08-11 18:43:55.287544/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:55.287671 2014-08-13 17:38:02.993416 7fe9bae887a0 20 read_log 193305'14961 (0'0) promote 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by osd.396.0:152 2014-08-11 18:43:55.290463 2014-08-13 17:38:02.993452 7fe9bae887a0 20 read_log 193305'14962 (193305'14961) modify 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.951337.0:22402 2014-08-11 18:43:55.279754 2014-08-13 17:38:02.993466 7fe9bae887a0 20 read_log 193305'14963 (0'0) promote 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by osd.396.0:154 2014-08-11 18:43:55.428034 2014-08-13 17:38:02.993478 7fe9bae887a0 20 read_log 193305'14964 (193305'14963) modify 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.951337.0:22435 2014-08-11 18:43:55.419081 2014-08-13 17:38:02.993490 7fe9bae887a0 20 read_log 193305'14965 (0'0) promote 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by osd.396.0:156 2014-08-11 18:43:56.579278 2014-08-13 17:38:02.993502 7fe9bae887a0 20 read_log 193305'14966 (193305'14965) modify 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.951337.0:22753 2014-08-11 18:43:56.570426 2014-08-13 17:38:02.993514 7fe9bae887a0 20 read_log 193305'14967 (0'0) promote a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by osd.396.0:158 2014-08-11 18:43:57.346635 2014-08-13 17:38:02.993528 7fe9bae887a0 20 read_log 193305'14968 (193305'14967) modify a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.951337.0:22972 2014-08-11 18:43:57.337063 2014-08-13 17:38:02.993541 7fe9bae887a0 20 read_log 193305'14969 (0'0) promote 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by osd.396.0:162 2014-08-11 18:43:58.310029 2014-08-13 17:38:02.993553 7fe9bae887a0 20 read_log 193305'14970 (193305'14969) modify 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.951337.0:23249 2014-08-11 18:43:58.299364 2014-08-13 17:38:02.993565 7fe9bae887a0 20 read_log 193305'14971 (0'0) promote b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by osd.396.0:164 2014-08-11 18:43:59.499929 2014-08-13 17:38:02.993577 7fe9bae887a0 20 read_log 193305'14972 (193305'14971) modify b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.951337.0:23575 2014-08-11 18:43:59.492172 2014-08-13 17:38:02.993589 7fe9bae887a0 20 read_log 193305'14973 (0'0) promote 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by osd.396.0:176 2014-08-11 18:44:08.828000 2014-08-13 17:38:02.993601 7fe9bae887a0 20 read_log 193305'14974 (193305'14973) modify 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.951337.0:26115 2014-08-11 18:44:08.819791 2014-08-13 17:38:02.993613 7fe9bae887a0 20 read_log 193305'14975 (0'0) promote 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by osd.396.0:180 2014-08-11 18:44:15.086166 2014-08-13 17:38:02.993625 7fe9bae887a0 20 read_log 193305'14976 (193305'14975) modify 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.951337.0:27840 2014-08-11 18:44:15.078785 2014-08-13 17:38:02.993637 7fe9bae887a0 20 read_log 193305'14977 (0'0) promote bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by osd.396.0:184 2014-08-11 18:44:18.307104 2014-08-13 17:38:02.993648 7fe9bae887a0 20 read_log 193305'14978 (193305'14977) modify bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.951337.0:28729 2014-08-11 18:44:18.298539 2014-08-13 17:38:02.993660 7fe9bae887a0 20 read_log 193305'14979 (0'0) promote b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by osd.396.0:186 2014-08-11 18:44:20.570028 2014-08-13 17:38:02.993672 7fe9bae887a0 20 read_log 193305'14980 (193305'14979) modify b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.951337.0:29336 2014-08-11 18:44:20.560423 2014-08-13 17:38:02.993684 7fe9bae887a0 20 read_log 193305'14981 (0'0) promote e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by osd.396.0:194 2014-08-11 18:44:28.996890 2014-08-13 17:38:02.993705 7fe9bae887a0 20 read_log 193305'14982 (193305'14981) modify e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.951337.0:31628 2014-08-11 18:44:28.989158 2014-08-13 17:38:02.993718 7fe9bae887a0 20 read_log 193305'14983 (193305'14917) modify 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.951337.0:33032 2014-08-11 18:44:34.080221 2014-08-13 17:38:02.993730 7fe9bae887a0 20 read_log 193305'14984 (193305'14919) modify 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.951337.0:34746 2014-08-11 18:44:35.166597 2014-08-13 17:38:02.993742 7fe9bae887a0 20 read_log 193305'14985 (193305'14921) modify 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.951337.0:36408 2014-08-11 18:44:36.179538 2014-08-13 17:38:02.993754 7fe9bae887a0 20 read_log 193305'14986 (193305'14923) modify 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.951337.0:37891 2014-08-11 18:44:37.103587 2014-08-13 17:38:02.993766 7fe9bae887a0 20 read_log 193305'14987 (193305'14925) modify 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.951337.0:38018 2014-08-11 18:44:37.200150 2014-08-13 17:38:02.993778 7fe9bae887a0 20 read_log 193305'14988 (193305'14927) modify e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.951337.0:38699 2014-08-11 18:44:37.604140 2014-08-13 17:38:02.993790 7fe9bae887a0 20 read_log 193305'14989 (193305'14929) modify b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.951337.0:39057 2014-08-11 18:44:37.803510 2014-08-13 17:38:02.993802 7fe9bae887a0 20 read_log 193305'14990 (193305'14931) modify 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.951337.0:41921 2014-08-11 18:44:38.925468 2014-08-13 17:38:02.993814 7fe9bae887a0 20 read_log 193305'14991 (193305'14933) modify 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.951337.0:44765 2014-08-11 18:44:39.983018 2014-08-13 17:38:02.993826 7fe9bae887a0 20 read_log 193305'14992 (193305'14935) modify 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.951337.0:45333 2014-08-11 18:44:40.165915 2014-08-13 17:38:02.993837 7fe9bae887a0 20 read_log 193305'14993 (193305'14937) modify e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.951337.0:47700 2014-08-11 18:44:40.841457 2014-08-13 17:38:02.993849 7fe9bae887a0 20 read_log 193305'14994 (193305'14939) modify e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.951337.0:47782 2014-08-11 18:44:40.864950 2014-08-13 17:38:02.993861 7fe9bae887a0 20 read_log 193305'14995 (193305'14941) modify 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.951337.0:48295 2014-08-11 18:44:41.011473 2014-08-13 17:38:02.993872 7fe9bae887a0 20 read_log 193305'14996 (193305'14943) modify fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.951337.0:48860 2014-08-11 18:44:41.177457 2014-08-13 17:38:02.993885 7fe9bae887a0 20 read_log 193305'14997 (193305'14945) modify c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.951337.0:49563 2014-08-11 18:44:41.404203 2014-08-13 17:38:02.993897 7fe9bae887a0 20 read_log 193305'14998 (193305'14947) modify d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.951337.0:50929 2014-08-11 18:44:41.803811 2014-08-13 17:38:02.993909 7fe9bae887a0 20 read_log 193305'14999 (193305'14949) modify 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.951337.0:51186 2014-08-11 18:44:41.879217 2014-08-13 17:38:02.993921 7fe9bae887a0 20 read_log 193305'15000 (193305'14951) modify 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.951337.0:52258 2014-08-11 18:44:42.177949 2014-08-13 17:38:02.993933 7fe9bae887a0 20 read_log 193305'15001 (193305'14953) modify d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.951337.0:52323 2014-08-11 18:44:42.202491 2014-08-13 17:38:02.993952 7fe9bae887a0 20 read_log 193305'15002 (193305'14955) modify de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.951337.0:53464 2014-08-11 18:44:42.539077 2014-08-13 17:38:02.993964 7fe9bae887a0 20 read_log 193305'15003 (193305'14957) modify 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.951337.0:53765 2014-08-11 18:44:42.626615 2014-08-13 17:38:02.993976 7fe9bae887a0 20 read_log 193305'15004 (193305'14959) modify 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.951337.0:54033 2014-08-11 18:44:42.707869 2014-08-13 17:38:02.993992 7fe9bae887a0 20 read_log 193305'15005 (193305'14962) modify 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.951337.0:55113 2014-08-11 18:44:43.028898 2014-08-13 17:38:02.994004 7fe9bae887a0 20 read_log 193305'15006 (193305'14964) modify 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.951337.0:55146 2014-08-11 18:44:43.038430 2014-08-13 17:38:02.994016 7fe9bae887a0 20 read_log 193305'15007 (193305'14966) modify 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.951337.0:55464 2014-08-11 18:44:43.131160 2014-08-13 17:38:02.994028 7fe9bae887a0 20 read_log 193305'15008 (193305'14968) modify a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.951337.0:55683 2014-08-11 18:44:43.200385 2014-08-13 17:38:02.994040 7fe9bae887a0 20 read_log 193305'15009 (193305'14970) modify 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.951337.0:55960 2014-08-11 18:44:43.279765 2014-08-13 17:38:02.994052 7fe9bae887a0 20 read_log 193305'15010 (193305'14972) modify b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.951337.0:56286 2014-08-11 18:44:43.374272 2014-08-13 17:38:02.994063 7fe9bae887a0 20 read_log 193305'15011 (193305'14974) modify 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.951337.0:58826 2014-08-11 18:44:44.152280 2014-08-13 17:38:02.994075 7fe9bae887a0 20 read_log 193305'15012 (193305'14976) modify 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.951337.0:60551 2014-08-11 18:44:44.662406 2014-08-13 17:38:02.994087 7fe9bae887a0 20 read_log 193305'15013 (193305'14978) modify bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.951337.0:61440 2014-08-11 18:44:44.921632 2014-08-13 17:38:02.994099 7fe9bae887a0 20 read_log 193305'15014 (193305'14980) modify b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.951337.0:62047 2014-08-11 18:44:45.110721 2014-08-13 17:38:02.994110 7fe9bae887a0 20 read_log 193305'15015 (193305'14982) modify e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.951337.0:64339 2014-08-11 18:44:45.772918 2014-08-13 17:38:02.994122 7fe9bae887a0 20 read_log 194215'15016 (194215'15016) modify 299/hit_set_24.299_archive_2014-08-12 15:32:25.825077_2014-08-12 15:46:22.535741/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:22.535888 2014-08-13 17:38:02.994135 7fe9bae887a0 20 read_log 194215'15017 (193305'14960) delete 299/hit_set_24.299_archive_2014-08-11 18:38:51.580559_2014-08-11 18:43:55.287544/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:22.535888 2014-08-13 17:38:02.994147 7fe9bae887a0 20 read_log 194215'15018 (192834'13948) clean 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by osd.396.0:529 2014-08-12 15:46:22.540873 2014-08-13 17:38:02.994165 7fe9bae887a0 20 read_log 194215'15019 (194215'15018) delete 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.2174595.0:136808 0.000000 2014-08-13 17:38:02.994177 7fe9bae887a0 20 read_log 194215'15020 (192834'14048) clean d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by osd.396.0:531 2014-08-12 15:46:22.546796 2014-08-13 17:38:02.994197 7fe9bae887a0 20 read_log 194215'15021 (194215'15020) delete d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.2174595.0:136810 0.000000 2014-08-13 17:38:02.994208 7fe9bae887a0 20 read_log 194215'15022 (192888'14685) clean 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by osd.396.0:533 2014-08-12 15:46:22.553182 2014-08-13 17:38:02.994220 7fe9bae887a0 20 read_log 194215'15023 (194215'15022) delete 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.2174595.0:136812 0.000000 2014-08-13 17:38:02.994231 7fe9bae887a0 20 read_log 194215'15024 (192888'14690) clean 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by osd.396.0:535 2014-08-12 15:46:22.560242 2014-08-13 17:38:02.994261 7fe9bae887a0 20 read_log 194215'15025 (194215'15024) delete 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.2174595.0:136814 0.000000 2014-08-13 17:38:02.994272 7fe9bae887a0 20 read_log 194215'15026 (193305'14993) clean e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by osd.396.0:537 2014-08-12 15:46:22.567610 2014-08-13 17:38:02.994285 7fe9bae887a0 20 read_log 194215'15027 (194215'15026) delete e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.2174595.0:136816 0.000000 2014-08-13 17:38:02.994295 7fe9bae887a0 20 read_log 194215'15028 (192834'13998) clean c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by osd.396.0:539 2014-08-12 15:46:22.574022 2014-08-13 17:38:02.994308 7fe9bae887a0 20 read_log 194215'15029 (194215'15028) delete c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.2174595.0:136818 0.000000 2014-08-13 17:38:02.994319 7fe9bae887a0 20 read_log 194215'15030 (192888'14668) clean 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by osd.396.0:541 2014-08-12 15:46:22.581071 2014-08-13 17:38:02.994331 7fe9bae887a0 20 read_log 194215'15031 (194215'15030) delete 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.2174595.0:136820 0.000000 2014-08-13 17:38:02.994342 7fe9bae887a0 20 read_log 194215'15032 (192888'14882) clean 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by osd.396.0:543 2014-08-12 15:46:22.587475 2014-08-13 17:38:02.994354 7fe9bae887a0 20 read_log 194215'15033 (194215'15032) delete 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.2174595.0:136822 0.000000 2014-08-13 17:38:02.994364 7fe9bae887a0 20 read_log 194215'15034 (192888'14874) clean aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by osd.396.0:545 2014-08-12 15:46:22.593445 2014-08-13 17:38:02.994376 7fe9bae887a0 20 read_log 194215'15035 (194215'15034) delete aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.2174595.0:136824 0.000000 2014-08-13 17:38:02.994387 7fe9bae887a0 20 read_log 194215'15036 (192834'14053) clean 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by osd.396.0:547 2014-08-12 15:46:22.599411 2014-08-13 17:38:02.994399 7fe9bae887a0 20 read_log 194215'15037 (194215'15036) delete 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.2174595.0:136826 0.000000 2014-08-13 17:38:02.994410 7fe9bae887a0 20 read_log 194215'15038 (193305'14989) clean b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by osd.396.0:549 2014-08-12 15:46:22.604929 2014-08-13 17:38:02.994422 7fe9bae887a0 20 read_log 194215'15039 (194215'15038) delete b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.2174595.0:136828 0.000000 2014-08-13 17:38:02.994432 7fe9bae887a0 20 read_log 194215'15040 (192888'14674) clean 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by osd.396.0:551 2014-08-12 15:46:22.610616 2014-08-13 17:38:02.994454 7fe9bae887a0 20 read_log 194215'15041 (194215'15040) delete 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.2174595.0:136830 0.000000 2014-08-13 17:38:02.994466 7fe9bae887a0 20 read_log 194215'15042 (192888'14897) clean aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by osd.396.0:553 2014-08-12 15:46:22.616129 2014-08-13 17:38:02.994478 7fe9bae887a0 20 read_log 194215'15043 (194215'15042) delete aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.2174595.0:136832 0.000000 2014-08-13 17:38:02.994488 7fe9bae887a0 20 read_log 194215'15044 (193305'15010) clean b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by osd.396.0:555 2014-08-12 15:46:22.621602 2014-08-13 17:38:02.994500 7fe9bae887a0 20 read_log 194215'15045 (194215'15044) delete b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.2174595.0:136834 0.000000 2014-08-13 17:38:02.994511 7fe9bae887a0 20 read_log 194215'15046 (192888'14868) clean cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by osd.396.0:557 2014-08-12 15:46:22.626887 2014-08-13 17:38:02.994523 7fe9bae887a0 20 read_log 194215'15047 (194215'15046) delete cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.2174595.0:136836 0.000000 2014-08-13 17:38:02.994533 7fe9bae887a0 20 read_log 194215'15048 (192888'14704) clean 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by osd.396.0:559 2014-08-12 15:46:22.633046 2014-08-13 17:38:02.994545 7fe9bae887a0 20 read_log 194215'15049 (194215'15048) delete 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.2174595.0:136838 0.000000 2014-08-13 17:38:02.994556 7fe9bae887a0 20 read_log 194215'15050 (193305'14996) clean fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by osd.396.0:561 2014-08-12 15:46:22.638666 2014-08-13 17:38:02.994568 7fe9bae887a0 20 read_log 194215'15051 (194215'15050) delete fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.2174595.0:136840 0.000000 2014-08-13 17:38:02.994578 7fe9bae887a0 20 read_log 194215'15052 (193305'15011) clean 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by osd.396.0:563 2014-08-12 15:46:22.643616 2014-08-13 17:38:02.994590 7fe9bae887a0 20 read_log 194215'15053 (194215'15052) delete 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.2174595.0:136842 0.000000 2014-08-13 17:38:02.994601 7fe9bae887a0 20 read_log 194215'15054 (193305'14984) clean 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by osd.396.0:565 2014-08-12 15:46:22.648837 2014-08-13 17:38:02.994613 7fe9bae887a0 20 read_log 194215'15055 (194215'15054) delete 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.2174595.0:136844 0.000000 2014-08-13 17:38:02.994623 7fe9bae887a0 20 read_log 194215'15056 (192888'14854) clean 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by osd.396.0:567 2014-08-12 15:46:22.654264 2014-08-13 17:38:02.994635 7fe9bae887a0 20 read_log 194215'15057 (194215'15056) delete 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.2174595.0:136846 0.000000 2014-08-13 17:38:02.994645 7fe9bae887a0 20 read_log 194215'15058 (192834'14522) clean 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by osd.396.0:569 2014-08-12 15:46:22.659083 2014-08-13 17:38:02.994657 7fe9bae887a0 20 read_log 194215'15059 (194215'15058) delete 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.2174595.0:136848 0.000000 2014-08-13 17:38:02.994668 7fe9bae887a0 20 read_log 194215'15060 (192888'14679) clean d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by osd.396.0:571 2014-08-12 15:46:22.664235 2014-08-13 17:38:02.994687 7fe9bae887a0 20 read_log 194215'15061 (194215'15060) delete d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.2174595.0:136850 0.000000 2014-08-13 17:38:02.994698 7fe9bae887a0 20 read_log 194215'15062 (193305'15013) clean bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by osd.396.0:573 2014-08-12 15:46:22.669762 2014-08-13 17:38:02.994710 7fe9bae887a0 20 read_log 194215'15063 (194215'15062) delete bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.2174595.0:136852 0.000000 2014-08-13 17:38:02.994721 7fe9bae887a0 20 read_log 194215'15064 (192888'14689) clean 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by osd.396.0:575 2014-08-12 15:46:22.675693 2014-08-13 17:38:02.994733 7fe9bae887a0 20 read_log 194215'15065 (194215'15064) delete 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.2174595.0:136854 0.000000 2014-08-13 17:38:02.994743 7fe9bae887a0 20 read_log 194215'15066 (193305'14983) clean 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by osd.396.0:577 2014-08-12 15:46:22.682100 2014-08-13 17:38:02.994755 7fe9bae887a0 20 read_log 194215'15067 (194215'15066) delete 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.2174595.0:136856 0.000000 2014-08-13 17:38:02.994765 7fe9bae887a0 20 read_log 194215'15068 (192834'13979) clean c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by osd.396.0:579 2014-08-12 15:46:22.687954 2014-08-13 17:38:02.994777 7fe9bae887a0 20 read_log 194215'15069 (194215'15068) delete c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.2174595.0:136858 0.000000 2014-08-13 17:38:02.994788 7fe9bae887a0 20 read_log 194215'15070 (193305'14995) clean 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by osd.396.0:581 2014-08-12 15:46:22.693529 2014-08-13 17:38:02.994800 7fe9bae887a0 20 read_log 194215'15071 (194215'15070) delete 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.2174595.0:136860 0.000000 2014-08-13 17:38:02.994810 7fe9bae887a0 20 read_log 194215'15072 (192888'14885) clean d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by osd.396.0:583 2014-08-12 15:46:22.699228 2014-08-13 17:38:02.994822 7fe9bae887a0 20 read_log 194215'15073 (194215'15072) delete d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.2174595.0:136862 0.000000 2014-08-13 17:38:02.994832 7fe9bae887a0 20 read_log 194215'15074 (192888'14684) clean 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by osd.396.0:585 2014-08-12 15:46:22.704955 2014-08-13 17:38:02.994844 7fe9bae887a0 20 read_log 194215'15075 (194215'15074) delete 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.2174595.0:136864 0.000000 2014-08-13 17:38:02.994855 7fe9bae887a0 20 read_log 194215'15076 (192834'13902) clean f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by osd.396.0:587 2014-08-12 15:46:22.711108 2014-08-13 17:38:02.994867 7fe9bae887a0 20 read_log 194215'15077 (194215'15076) delete f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by client.2174595.0:136866 0.000000 2014-08-13 17:38:02.994877 7fe9bae887a0 20 read_log 194215'15078 (192834'14545) clean 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by osd.396.0:589 2014-08-12 15:46:22.716722 2014-08-13 17:38:02.994889 7fe9bae887a0 20 read_log 194215'15079 (194215'15078) delete 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.2174595.0:136868 0.000000 2014-08-13 17:38:02.994899 7fe9bae887a0 20 read_log 194215'15080 (192834'14020) clean 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by osd.396.0:591 2014-08-12 15:46:22.721929 2014-08-13 17:38:02.994918 7fe9bae887a0 20 read_log 194215'15081 (194215'15080) delete 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.2174595.0:136870 0.000000 2014-08-13 17:38:02.994929 7fe9bae887a0 20 read_log 194215'15082 (192888'14694) clean a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by osd.396.0:593 2014-08-12 15:46:22.727303 2014-08-13 17:38:02.994941 7fe9bae887a0 20 read_log 194215'15083 (194215'15082) delete a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.2174595.0:136872 0.000000 2014-08-13 17:38:02.994951 7fe9bae887a0 20 read_log 194215'15084 (192834'14016) clean 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by osd.396.0:595 2014-08-12 15:46:22.732694 2014-08-13 17:38:02.994963 7fe9bae887a0 20 read_log 194215'15085 (194215'15084) delete 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.2174595.0:136874 0.000000 2014-08-13 17:38:02.994974 7fe9bae887a0 20 read_log 194215'15086 (192888'14858) clean 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by osd.396.0:597 2014-08-12 15:46:22.738595 2014-08-13 17:38:02.994986 7fe9bae887a0 20 read_log 194215'15087 (194215'15086) delete 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.2174595.0:136876 0.000000 2014-08-13 17:38:02.994996 7fe9bae887a0 20 read_log 194215'15088 (192888'14673) clean 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by osd.396.0:599 2014-08-12 15:46:22.744188 2014-08-13 17:38:02.995008 7fe9bae887a0 20 read_log 194215'15089 (194215'15088) delete 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.2174595.0:136878 0.000000 2014-08-13 17:38:02.995018 7fe9bae887a0 20 read_log 194215'15090 (192888'14890) clean 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by osd.396.0:601 2014-08-12 15:46:22.750069 2014-08-13 17:38:02.995030 7fe9bae887a0 20 read_log 194215'15091 (194215'15090) delete 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.2174595.0:136880 0.000000 2014-08-13 17:38:02.995041 7fe9bae887a0 20 read_log 194215'15092 (193305'15006) clean 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by osd.396.0:603 2014-08-12 15:46:22.756028 2014-08-13 17:38:02.995053 7fe9bae887a0 20 read_log 194215'15093 (194215'15092) delete 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.2174595.0:136882 0.000000 2014-08-13 17:38:02.995063 7fe9bae887a0 20 read_log 194215'15094 (192888'14912) clean d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by osd.396.0:605 2014-08-12 15:46:22.761985 2014-08-13 17:38:02.995075 7fe9bae887a0 20 read_log 194215'15095 (194215'15094) delete d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.2174595.0:136884 0.000000 2014-08-13 17:38:02.995085 7fe9bae887a0 20 read_log 194215'15096 (193305'14990) clean 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by osd.396.0:607 2014-08-12 15:46:22.767957 2014-08-13 17:38:02.995097 7fe9bae887a0 20 read_log 194215'15097 (194215'15096) delete 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.2174595.0:136886 0.000000 2014-08-13 17:38:02.995108 7fe9bae887a0 20 read_log 194215'15098 (193305'14986) clean 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by osd.396.0:609 2014-08-12 15:46:22.773441 2014-08-13 17:38:02.995120 7fe9bae887a0 20 read_log 194215'15099 (194215'15098) delete 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.2174595.0:136888 0.000000 2014-08-13 17:38:02.995130 7fe9bae887a0 20 read_log 194215'15100 (192888'14908) clean 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by osd.396.0:611 2014-08-12 15:46:22.779199 2014-08-13 17:38:02.995171 7fe9bae887a0 20 read_log 194215'15101 (194215'15100) delete 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.2174595.0:136890 0.000000 2014-08-13 17:38:02.995183 7fe9bae887a0 20 read_log 194215'15102 (193305'14987) clean 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by osd.396.0:613 2014-08-12 15:46:22.784800 2014-08-13 17:38:02.995196 7fe9bae887a0 20 read_log 194215'15103 (194215'15102) delete 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.2174595.0:136892 0.000000 2014-08-13 17:38:02.995206 7fe9bae887a0 20 read_log 194215'15104 (192834'14432) clean 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by osd.396.0:615 2014-08-12 15:46:22.790648 2014-08-13 17:38:02.995219 7fe9bae887a0 20 read_log 194215'15105 (194215'15104) delete 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.2174595.0:136894 0.000000 2014-08-13 17:38:02.995229 7fe9bae887a0 20 read_log 194215'15106 (193305'15007) clean 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by osd.396.0:617 2014-08-12 15:46:22.796592 2014-08-13 17:38:02.995241 7fe9bae887a0 20 read_log 194215'15107 (194215'15106) delete 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.2174595.0:136896 0.000000 2014-08-13 17:38:02.995252 7fe9bae887a0 20 read_log 194215'15108 (192888'14900) clean 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by osd.396.0:619 2014-08-12 15:46:22.802239 2014-08-13 17:38:02.995264 7fe9bae887a0 20 read_log 194215'15109 (194215'15108) delete 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.2174595.0:136898 0.000000 2014-08-13 17:38:02.995275 7fe9bae887a0 20 read_log 194215'15110 (192834'13965) clean 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by osd.396.0:621 2014-08-12 15:46:22.808743 2014-08-13 17:38:02.995287 7fe9bae887a0 20 read_log 194215'15111 (194215'15110) delete 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.2174595.0:136900 0.000000 2014-08-13 17:38:02.995297 7fe9bae887a0 20 read_log 194215'15112 (192834'14416) clean bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by osd.396.0:623 2014-08-12 15:46:22.814521 2014-08-13 17:38:02.995309 7fe9bae887a0 20 read_log 194215'15113 (194215'15112) delete bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.2174595.0:136902 0.000000 2014-08-13 17:38:02.995320 7fe9bae887a0 20 read_log 194215'15114 (192834'14546) clean 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by osd.396.0:625 2014-08-12 15:46:22.820612 2014-08-13 17:38:02.995332 7fe9bae887a0 20 read_log 194215'15115 (194215'15114) delete 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.2174595.0:136904 0.000000 2014-08-13 17:38:02.995342 7fe9bae887a0 20 read_log 194215'15116 (192834'14055) clean 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by osd.396.0:627 2014-08-12 15:46:22.826383 2014-08-13 17:38:02.995354 7fe9bae887a0 20 read_log 194215'15117 (194215'15116) delete 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.2174595.0:136906 0.000000 2014-08-13 17:38:02.995382 7fe9bae887a0 20 read_log 194215'15118 (192888'14849) clean aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by osd.396.0:629 2014-08-12 15:46:22.832451 2014-08-13 17:38:02.995395 7fe9bae887a0 20 read_log 194215'15119 (194215'15118) delete aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.2174595.0:136908 0.000000 2014-08-13 17:38:02.995406 7fe9bae887a0 20 read_log 194215'15120 (192888'14911) clean 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by osd.396.0:631 2014-08-12 15:46:22.838801 2014-08-13 17:38:02.995425 7fe9bae887a0 20 read_log 194215'15121 (194215'15120) delete 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.2174595.0:136910 0.000000 2014-08-13 17:38:02.995436 7fe9bae887a0 20 read_log 194215'15122 (192834'13921) clean e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by osd.396.0:633 2014-08-12 15:46:22.844230 2014-08-13 17:38:02.995450 7fe9bae887a0 20 read_log 194215'15123 (194215'15122) delete e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.2174595.0:136912 0.000000 2014-08-13 17:38:02.995460 7fe9bae887a0 20 read_log 194215'15124 (192888'14863) clean 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by osd.396.0:635 2014-08-12 15:46:22.849449 2014-08-13 17:38:02.995472 7fe9bae887a0 20 read_log 194215'15125 (194215'15124) delete 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.2174595.0:136914 0.000000 2014-08-13 17:38:02.995483 7fe9bae887a0 20 read_log 194215'15126 (193305'14994) clean e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by osd.396.0:637 2014-08-12 15:46:22.854783 2014-08-13 17:38:02.995495 7fe9bae887a0 20 read_log 194215'15127 (194215'15126) delete e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.2174595.0:136916 0.000000 2014-08-13 17:38:02.995505 7fe9bae887a0 20 read_log 194215'15128 (193305'14997) clean c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by osd.396.0:639 2014-08-12 15:46:22.860442 2014-08-13 17:38:02.995519 7fe9bae887a0 20 read_log 194215'15129 (194215'15128) delete c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.2174595.0:136918 0.000000 2014-08-13 17:38:02.995529 7fe9bae887a0 20 read_log 194215'15130 (192834'14507) clean 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by osd.396.0:641 2014-08-12 15:46:22.866012 2014-08-13 17:38:02.995541 7fe9bae887a0 20 read_log 194215'15131 (194215'15130) delete 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.2174595.0:136920 0.000000 2014-08-13 17:38:02.995552 7fe9bae887a0 20 read_log 194215'15132 (192834'14454) clean a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by osd.396.0:643 2014-08-12 15:46:22.870916 2014-08-13 17:38:02.995564 7fe9bae887a0 20 read_log 194215'15133 (194215'15132) delete a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.2174595.0:136922 0.000000 2014-08-13 17:38:02.995574 7fe9bae887a0 20 read_log 194215'15134 (192888'14712) clean 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by osd.396.0:645 2014-08-12 15:46:22.875849 2014-08-13 17:38:02.995586 7fe9bae887a0 20 read_log 194215'15135 (194215'15134) delete 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.2174595.0:136924 0.000000 2014-08-13 17:38:02.995597 7fe9bae887a0 20 read_log 194215'15136 (192834'13964) clean d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by osd.396.0:647 2014-08-12 15:46:22.880554 2014-08-13 17:38:02.995609 7fe9bae887a0 20 read_log 194215'15137 (194215'15136) delete d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.2174595.0:136926 0.000000 2014-08-13 17:38:02.995619 7fe9bae887a0 20 read_log 194215'15138 (192888'14906) clean 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by osd.396.0:649 2014-08-12 15:46:22.886022 2014-08-13 17:38:02.995631 7fe9bae887a0 20 read_log 194215'15139 (194215'15138) delete 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.2174595.0:136928 0.000000 2014-08-13 17:38:02.995641 7fe9bae887a0 20 read_log 194215'15140 (192834'14421) clean 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by osd.396.0:651 2014-08-12 15:46:22.891379 2014-08-13 17:38:02.995662 7fe9bae887a0 20 read_log 194215'15141 (194215'15140) delete 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.2174595.0:136930 0.000000 2014-08-13 17:38:02.995673 7fe9bae887a0 20 read_log 194215'15142 (192834'14521) clean ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by osd.396.0:653 2014-08-12 15:46:22.897006 2014-08-13 17:38:02.995685 7fe9bae887a0 20 read_log 194215'15143 (194215'15142) delete ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.2174595.0:136932 0.000000 2014-08-13 17:38:02.995696 7fe9bae887a0 20 read_log 194215'15144 (192834'14493) clean cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by osd.396.0:655 2014-08-12 15:46:22.902608 2014-08-13 17:38:02.995728 7fe9bae887a0 20 read_log 194215'15145 (194215'15144) delete cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.2174595.0:136934 0.000000 2014-08-13 17:38:02.995740 7fe9bae887a0 20 read_log 194215'15146 (192888'14901) clean 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by osd.396.0:657 2014-08-12 15:46:22.908070 2014-08-13 17:38:02.995753 7fe9bae887a0 20 read_log 194215'15147 (194215'15146) delete 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.2174595.0:136936 0.000000 2014-08-13 17:38:02.995764 7fe9bae887a0 20 read_log 194215'15148 (192888'14914) clean d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by osd.396.0:659 2014-08-12 15:46:22.913677 2014-08-13 17:38:02.995776 7fe9bae887a0 20 read_log 194215'15149 (194215'15148) delete d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.2174595.0:136938 0.000000 2014-08-13 17:38:02.995787 7fe9bae887a0 20 read_log 194215'15150 (192888'14706) clean 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by osd.396.0:661 2014-08-12 15:46:22.918972 2014-08-13 17:38:02.995799 7fe9bae887a0 20 read_log 194215'15151 (194215'15150) delete 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.2174595.0:136940 0.000000 2014-08-13 17:38:02.995809 7fe9bae887a0 20 read_log 194215'15152 (192834'14541) clean 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by osd.396.0:663 2014-08-12 15:46:22.924525 2014-08-13 17:38:02.995822 7fe9bae887a0 20 read_log 194215'15153 (194215'15152) delete 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.2174595.0:136942 0.000000 2014-08-13 17:38:02.995832 7fe9bae887a0 20 read_log 194215'15154 (192888'14701) clean 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by osd.396.0:665 2014-08-12 15:46:22.930799 2014-08-13 17:38:02.995844 7fe9bae887a0 20 read_log 194215'15155 (194215'15154) delete 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.2174595.0:136944 0.000000 2014-08-13 17:38:02.995855 7fe9bae887a0 20 read_log 194215'15156 (192834'13972) clean 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by osd.396.0:667 2014-08-12 15:46:22.937451 2014-08-13 17:38:02.995886 7fe9bae887a0 20 read_log 194215'15157 (194215'15156) delete 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.2174595.0:136946 0.000000 2014-08-13 17:38:02.995896 7fe9bae887a0 20 read_log 194215'15158 (192888'14672) clean e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by osd.396.0:669 2014-08-12 15:46:22.945406 2014-08-13 17:38:02.995908 7fe9bae887a0 20 read_log 194215'15159 (194215'15158) delete e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.2174595.0:136948 0.000000 2014-08-13 17:38:02.995919 7fe9bae887a0 20 read_log 194215'15160 (192834'14449) clean d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by osd.396.0:671 2014-08-12 15:46:22.951819 2014-08-13 17:38:02.995938 7fe9bae887a0 20 read_log 194215'15161 (194215'15160) delete d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.2174595.0:136950 0.000000 2014-08-13 17:38:02.995950 7fe9bae887a0 20 read_log 194215'15162 (192834'14496) clean 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by osd.396.0:673 2014-08-12 15:46:22.957587 2014-08-13 17:38:02.995962 7fe9bae887a0 20 read_log 194215'15163 (194215'15162) delete 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.2174595.0:136952 0.000000 2014-08-13 17:38:02.995972 7fe9bae887a0 20 read_log 194215'15164 (192888'14876) clean 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by osd.396.0:675 2014-08-12 15:46:22.963596 2014-08-13 17:38:02.995985 7fe9bae887a0 20 read_log 194215'15165 (194215'15164) delete 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.2174595.0:136954 0.000000 2014-08-13 17:38:02.995995 7fe9bae887a0 20 read_log 194215'15166 (192888'14856) clean 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by osd.396.0:677 2014-08-12 15:46:22.969334 2014-08-13 17:38:02.996007 7fe9bae887a0 20 read_log 194215'15167 (194215'15166) delete 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.2174595.0:136956 0.000000 2014-08-13 17:38:02.996018 7fe9bae887a0 20 read_log 194215'15168 (192834'13898) clean fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by osd.396.0:679 2014-08-12 15:46:22.974878 2014-08-13 17:38:02.996030 7fe9bae887a0 20 read_log 194215'15169 (194215'15168) delete fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by client.2174595.0:136958 0.000000 2014-08-13 17:38:02.996041 7fe9bae887a0 20 read_log 194215'15170 (192888'14896) clean 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by osd.396.0:681 2014-08-12 15:46:22.980752 2014-08-13 17:38:02.996052 7fe9bae887a0 20 read_log 194215'15171 (194215'15170) delete 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.2174595.0:136960 0.000000 2014-08-13 17:38:02.996063 7fe9bae887a0 20 read_log 194215'15172 (192834'14467) clean 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by osd.396.0:683 2014-08-12 15:46:22.987141 2014-08-13 17:38:02.996075 7fe9bae887a0 20 read_log 194215'15173 (194215'15172) delete 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.2174595.0:136962 0.000000 2014-08-13 17:38:02.996086 7fe9bae887a0 20 read_log 194215'15174 (193305'15012) clean 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by osd.396.0:685 2014-08-12 15:46:22.993084 2014-08-13 17:38:02.996097 7fe9bae887a0 20 read_log 194215'15175 (194215'15174) delete 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.2174595.0:136964 0.000000 2014-08-13 17:38:02.996108 7fe9bae887a0 20 read_log 194215'15176 (192888'14864) clean f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by osd.396.0:687 2014-08-12 15:46:22.999297 2014-08-13 17:38:02.996120 7fe9bae887a0 20 read_log 194215'15177 (194215'15176) delete f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.2174595.0:136966 0.000000 2014-08-13 17:38:02.996130 7fe9bae887a0 20 read_log 194215'15178 (192888'14866) clean 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by osd.396.0:689 2014-08-12 15:46:23.004785 2014-08-13 17:38:02.996142 7fe9bae887a0 20 read_log 194215'15179 (194215'15178) delete 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.2174595.0:136968 0.000000 2014-08-13 17:38:02.996157 7fe9bae887a0 20 read_log 194215'15180 (192888'14709) clean 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by osd.396.0:691 2014-08-12 15:46:23.010335 2014-08-13 17:38:02.996180 7fe9bae887a0 20 read_log 194215'15181 (194215'15180) delete 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.2174595.0:136970 0.000000 2014-08-13 17:38:02.996191 7fe9bae887a0 20 read_log 194215'15182 (192888'14904) clean 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by osd.396.0:693 2014-08-12 15:46:23.016249 2014-08-13 17:38:02.996204 7fe9bae887a0 20 read_log 194215'15183 (194215'15182) delete 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.2174595.0:136972 0.000000 2014-08-13 17:38:02.996215 7fe9bae887a0 20 read_log 194215'15184 (193305'15002) clean de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by osd.396.0:695 2014-08-12 15:46:23.021906 2014-08-13 17:38:02.996227 7fe9bae887a0 20 read_log 194215'15185 (194215'15184) delete de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.2174595.0:136974 0.000000 2014-08-13 17:38:02.996238 7fe9bae887a0 20 read_log 194215'15186 (192834'14022) clean 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by osd.396.0:697 2014-08-12 15:46:23.027691 2014-08-13 17:38:02.996250 7fe9bae887a0 20 read_log 194215'15187 (194215'15186) delete 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.2174595.0:136976 0.000000 2014-08-13 17:38:02.996260 7fe9bae887a0 20 read_log 194215'15188 (192888'14879) clean 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by osd.396.0:699 2014-08-12 15:46:23.033225 2014-08-13 17:38:02.996272 7fe9bae887a0 20 read_log 194215'15189 (194215'15188) delete 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.2174595.0:136978 0.000000 2014-08-13 17:38:02.996283 7fe9bae887a0 20 read_log 194215'15190 (193305'15009) clean 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by osd.396.0:701 2014-08-12 15:46:23.038927 2014-08-13 17:38:02.996295 7fe9bae887a0 20 read_log 194215'15191 (194215'15190) delete 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.2174595.0:136980 0.000000 2014-08-13 17:38:02.996305 7fe9bae887a0 20 read_log 194215'15192 (192888'14670) clean 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by osd.396.0:703 2014-08-12 15:46:23.044625 2014-08-13 17:38:02.996317 7fe9bae887a0 20 read_log 194215'15193 (194215'15192) delete 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.2174595.0:136982 0.000000 2014-08-13 17:38:02.996327 7fe9bae887a0 20 read_log 194215'15194 (193305'15015) clean e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by osd.396.0:705 2014-08-12 15:46:23.050635 2014-08-13 17:38:02.996339 7fe9bae887a0 20 read_log 194215'15195 (194215'15194) delete e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.2174595.0:136984 0.000000 2014-08-13 17:38:02.996350 7fe9bae887a0 20 read_log 194215'15196 (193305'15004) clean 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by osd.396.0:707 2014-08-12 15:46:23.057547 2014-08-13 17:38:02.996362 7fe9bae887a0 20 read_log 194215'15197 (194215'15196) delete 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.2174595.0:136986 0.000000 2014-08-13 17:38:02.996372 7fe9bae887a0 20 read_log 194215'15198 (192888'14909) clean 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by osd.396.0:709 2014-08-12 15:46:23.063747 2014-08-13 17:38:02.996384 7fe9bae887a0 20 read_log 194215'15199 (194215'15198) delete 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.2174595.0:136988 0.000000 2014-08-13 17:38:02.996395 7fe9bae887a0 20 read_log 194215'15200 (192888'14699) clean e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by osd.396.0:711 2014-08-12 15:46:23.070050 2014-08-13 17:38:02.996415 7fe9bae887a0 20 read_log 194215'15201 (194215'15200) delete e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.2174595.0:136990 0.000000 2014-08-13 17:38:02.996426 7fe9bae887a0 20 read_log 194215'15202 (192834'14557) clean eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by osd.396.0:713 2014-08-12 15:46:23.076399 2014-08-13 17:38:02.996438 7fe9bae887a0 20 read_log 194215'15203 (194215'15202) delete eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.2174595.0:136992 0.000000 2014-08-13 17:38:02.996449 7fe9bae887a0 20 read_log 194215'15204 (193305'15003) clean 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by osd.396.0:715 2014-08-12 15:46:23.082885 2014-08-13 17:38:02.996461 7fe9bae887a0 20 read_log 194215'15205 (194215'15204) delete 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.2174595.0:136994 0.000000 2014-08-13 17:38:02.996471 7fe9bae887a0 20 read_log 194215'15206 (193305'15000) clean 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by osd.396.0:717 2014-08-12 15:46:23.088612 2014-08-13 17:38:02.996483 7fe9bae887a0 20 read_log 194215'15207 (194215'15206) delete 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.2174595.0:136996 0.000000 2014-08-13 17:38:02.996494 7fe9bae887a0 20 read_log 194215'15208 (192888'14852) clean 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by osd.396.0:719 2014-08-12 15:46:23.094394 2014-08-13 17:38:02.996506 7fe9bae887a0 20 read_log 194215'15209 (194215'15208) delete 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.2174595.0:136998 0.000000 2014-08-13 17:38:02.996517 7fe9bae887a0 20 read_log 194215'15210 (193305'14998) clean d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by osd.396.0:721 2014-08-12 15:46:23.100438 2014-08-13 17:38:02.996547 7fe9bae887a0 20 read_log 194215'15211 (194215'15210) delete d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.2174595.0:137000 0.000000 2014-08-13 17:38:02.996559 7fe9bae887a0 20 read_log 194215'15212 (192888'14898) clean 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by osd.396.0:723 2014-08-12 15:46:23.106130 2014-08-13 17:38:02.996571 7fe9bae887a0 20 read_log 194215'15213 (194215'15212) delete 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.2174595.0:137002 0.000000 2014-08-13 17:38:02.996582 7fe9bae887a0 20 read_log 194215'15214 (192834'14013) clean 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by osd.396.0:725 2014-08-12 15:46:23.111152 2014-08-13 17:38:02.996594 7fe9bae887a0 20 read_log 194215'15215 (194215'15214) delete 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.2174595.0:137004 0.000000 2014-08-13 17:38:02.996604 7fe9bae887a0 20 read_log 194215'15216 (192834'14037) clean bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by osd.396.0:727 2014-08-12 15:46:23.116174 2014-08-13 17:38:02.996617 7fe9bae887a0 20 read_log 194215'15217 (194215'15216) delete bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.2174595.0:137006 0.000000 2014-08-13 17:38:02.996628 7fe9bae887a0 20 read_log 194215'15218 (192834'14485) clean 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by osd.396.0:729 2014-08-12 15:46:23.121523 2014-08-13 17:38:02.996640 7fe9bae887a0 20 read_log 194215'15219 (194215'15218) delete 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.2174595.0:137008 0.000000 2014-08-13 17:38:02.996650 7fe9bae887a0 20 read_log 194215'15220 (192834'14034) clean ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by osd.396.0:731 2014-08-12 15:46:23.127300 2014-08-13 17:38:02.996669 7fe9bae887a0 20 read_log 194215'15221 (194215'15220) delete ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.2174595.0:137010 0.000000 2014-08-13 17:38:02.996680 7fe9bae887a0 20 read_log 194215'15222 (192834'14456) clean eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by osd.396.0:733 2014-08-12 15:46:23.133100 2014-08-13 17:38:02.996693 7fe9bae887a0 20 read_log 194215'15223 (194215'15222) delete eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.2174595.0:137012 0.000000 2014-08-13 17:38:02.996703 7fe9bae887a0 20 read_log 194215'15224 (192888'14894) clean e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by osd.396.0:735 2014-08-12 15:46:23.138956 2014-08-13 17:38:02.996715 7fe9bae887a0 20 read_log 194215'15225 (194215'15224) delete e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.2174595.0:137014 0.000000 2014-08-13 17:38:02.996726 7fe9bae887a0 20 read_log 194215'15226 (192888'14665) clean f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by osd.396.0:737 2014-08-12 15:46:23.144657 2014-08-13 17:38:02.996740 7fe9bae887a0 20 read_log 194215'15227 (194215'15226) delete f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.2174595.0:137016 0.000000 2014-08-13 17:38:02.996750 7fe9bae887a0 20 read_log 194215'15228 (192888'14878) clean 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by osd.396.0:739 2014-08-12 15:46:23.150380 2014-08-13 17:38:02.996763 7fe9bae887a0 20 read_log 194215'15229 (194215'15228) delete 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.2174595.0:137018 0.000000 2014-08-13 17:38:02.996773 7fe9bae887a0 20 read_log 194215'15230 (192834'14549) clean be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by osd.396.0:741 2014-08-12 15:46:23.156271 2014-08-13 17:38:02.996785 7fe9bae887a0 20 read_log 194215'15231 (194215'15230) delete be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.2174595.0:137020 0.000000 2014-08-13 17:38:02.996796 7fe9bae887a0 20 read_log 194215'15232 (192834'14404) clean 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by osd.396.0:743 2014-08-12 15:46:23.162486 2014-08-13 17:38:02.996808 7fe9bae887a0 20 read_log 194215'15233 (194215'15232) delete 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.2174595.0:137022 0.000000 2014-08-13 17:38:02.996819 7fe9bae887a0 20 read_log 194215'15234 (192834'13977) clean a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by osd.396.0:745 2014-08-12 15:46:23.167716 2014-08-13 17:38:02.996831 7fe9bae887a0 20 read_log 194215'15235 (194215'15234) delete a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.2174595.0:137024 0.000000 2014-08-13 17:38:02.996842 7fe9bae887a0 20 read_log 194215'15236 (192888'14883) clean e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by osd.396.0:747 2014-08-12 15:46:23.174057 2014-08-13 17:38:02.996854 7fe9bae887a0 20 read_log 194215'15237 (194215'15236) delete e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.2174595.0:137026 0.000000 2014-08-13 17:38:02.996865 7fe9bae887a0 20 read_log 194215'15238 (193305'14988) clean e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by osd.396.0:749 2014-08-12 15:46:23.180767 2014-08-13 17:38:02.996877 7fe9bae887a0 20 read_log 194215'15239 (194215'15238) delete e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.2174595.0:137028 0.000000 2014-08-13 17:38:02.996887 7fe9bae887a0 20 read_log 194215'15240 (193305'14992) clean 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by osd.396.0:751 2014-08-12 15:46:23.187338 2014-08-13 17:38:02.996923 7fe9bae887a0 20 read_log 194215'15241 (194215'15240) delete 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.2174595.0:137030 0.000000 2014-08-13 17:38:02.996934 7fe9bae887a0 20 read_log 194215'15242 (192834'13918) clean c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by osd.396.0:753 2014-08-12 15:46:23.194150 2014-08-13 17:38:02.996947 7fe9bae887a0 20 read_log 194215'15243 (194215'15242) delete c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.2174595.0:137032 0.000000 2014-08-13 17:38:02.996957 7fe9bae887a0 20 read_log 194215'15244 (192834'14565) clean a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by osd.396.0:755 2014-08-12 15:46:23.199891 2014-08-13 17:38:02.996969 7fe9bae887a0 20 read_log 194215'15245 (194215'15244) delete a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.2174595.0:137034 0.000000 2014-08-13 17:38:02.996980 7fe9bae887a0 20 read_log 194215'15246 (192888'14872) clean c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by osd.396.0:757 2014-08-12 15:46:23.205659 2014-08-13 17:38:02.996992 7fe9bae887a0 20 read_log 194215'15247 (194215'15246) delete c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.2174595.0:137036 0.000000 2014-08-13 17:38:02.997002 7fe9bae887a0 20 read_log 194215'15248 (192834'14444) clean ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by osd.396.0:759 2014-08-12 15:46:23.211354 2014-08-13 17:38:02.997014 7fe9bae887a0 20 read_log 194215'15249 (194215'15248) delete ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.2174595.0:137038 0.000000 2014-08-13 17:38:02.997025 7fe9bae887a0 20 read_log 194215'15250 (192834'13915) clean e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by osd.396.0:761 2014-08-12 15:46:23.217129 2014-08-13 17:38:02.997037 7fe9bae887a0 20 read_log 194215'15251 (194215'15250) delete e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.2174595.0:137040 0.000000 2014-08-13 17:38:02.997048 7fe9bae887a0 20 read_log 194215'15252 (193305'15008) clean a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by osd.396.0:763 2014-08-12 15:46:23.222865 2014-08-13 17:38:02.997060 7fe9bae887a0 20 read_log 194215'15253 (194215'15252) delete a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.2174595.0:137042 0.000000 2014-08-13 17:38:02.997070 7fe9bae887a0 20 read_log 194215'15254 (192888'14705) clean 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by osd.396.0:765 2014-08-12 15:46:23.228534 2014-08-13 17:38:02.997082 7fe9bae887a0 20 read_log 194215'15255 (194215'15254) delete 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.2174595.0:137044 0.000000 2014-08-13 17:38:02.997093 7fe9bae887a0 20 read_log 194215'15256 (192888'14678) clean a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by osd.396.0:767 2014-08-12 15:46:23.233671 2014-08-13 17:38:02.997105 7fe9bae887a0 20 read_log 194215'15257 (194215'15256) delete a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.2174595.0:137046 0.000000 2014-08-13 17:38:02.997115 7fe9bae887a0 20 read_log 194215'15258 (192888'14675) clean e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by osd.396.0:769 2014-08-12 15:46:23.239022 2014-08-13 17:38:02.997127 7fe9bae887a0 20 read_log 194215'15259 (194215'15258) delete e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.2174595.0:137048 0.000000 2014-08-13 17:38:02.997138 7fe9bae887a0 20 read_log 194215'15260 (192888'14853) clean 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by osd.396.0:771 2014-08-12 15:46:23.244571 2014-08-13 17:38:02.997169 7fe9bae887a0 20 read_log 194215'15261 (194215'15260) delete 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.2174595.0:137050 0.000000 2014-08-13 17:38:02.997181 7fe9bae887a0 20 read_log 194215'15262 (193305'14985) clean 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by osd.396.0:773 2014-08-12 15:46:23.250581 2014-08-13 17:38:02.997194 7fe9bae887a0 20 read_log 194215'15263 (194215'15262) delete 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.2174595.0:137052 0.000000 2014-08-13 17:38:02.997204 7fe9bae887a0 20 read_log 194215'15264 (192888'14889) clean 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by osd.396.0:775 2014-08-12 15:46:23.255906 2014-08-13 17:38:02.997216 7fe9bae887a0 20 read_log 194215'15265 (194215'15264) delete 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.2174595.0:137054 0.000000 2014-08-13 17:38:02.997227 7fe9bae887a0 20 read_log 194215'15266 (192834'14556) clean e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by osd.396.0:777 2014-08-12 15:46:23.261388 2014-08-13 17:38:02.997239 7fe9bae887a0 20 read_log 194215'15267 (194215'15266) delete e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.2174595.0:137056 0.000000 2014-08-13 17:38:02.997250 7fe9bae887a0 20 read_log 194215'15268 (193305'14999) clean 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by osd.396.0:779 2014-08-12 15:46:23.267332 2014-08-13 17:38:02.997261 7fe9bae887a0 20 read_log 194215'15269 (194215'15268) delete 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.2174595.0:137058 0.000000 2014-08-13 17:38:02.997272 7fe9bae887a0 20 read_log 194215'15270 (192888'14870) clean ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by osd.396.0:781 2014-08-12 15:46:23.273057 2014-08-13 17:38:02.997284 7fe9bae887a0 20 read_log 194215'15271 (194215'15270) delete ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.2174595.0:137060 0.000000 2014-08-13 17:38:02.997294 7fe9bae887a0 20 read_log 194215'15272 (193305'15001) clean d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by osd.396.0:783 2014-08-12 15:46:23.278679 2014-08-13 17:38:02.997306 7fe9bae887a0 20 read_log 194215'15273 (194215'15272) delete d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.2174595.0:137062 0.000000 2014-08-13 17:38:02.997317 7fe9bae887a0 20 read_log 194215'15274 (192888'14913) clean d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by osd.396.0:785 2014-08-12 15:46:23.283678 2014-08-13 17:38:02.997329 7fe9bae887a0 20 read_log 194215'15275 (194215'15274) delete d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.2174595.0:137064 0.000000 2014-08-13 17:38:02.997339 7fe9bae887a0 20 read_log 194215'15276 (192834'14498) clean 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by osd.396.0:787 2014-08-12 15:46:23.289466 2014-08-13 17:38:02.997351 7fe9bae887a0 20 read_log 194215'15277 (194215'15276) delete 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.2174595.0:137066 0.000000 2014-08-13 17:38:02.997362 7fe9bae887a0 20 read_log 194215'15278 (192834'14488) clean 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by osd.396.0:789 2014-08-12 15:46:23.295103 2014-08-13 17:38:02.997373 7fe9bae887a0 20 read_log 194215'15279 (194215'15278) delete 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.2174595.0:137068 0.000000 2014-08-13 17:38:02.997384 7fe9bae887a0 20 read_log 194215'15280 (192888'14895) clean 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by osd.396.0:791 2014-08-12 15:46:23.300957 2014-08-13 17:38:02.997403 7fe9bae887a0 20 read_log 194215'15281 (194215'15280) delete 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.2174595.0:137070 0.000000 2014-08-13 17:38:02.997414 7fe9bae887a0 20 read_log 194215'15282 (192834'14472) clean b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by osd.396.0:793 2014-08-12 15:46:23.307475 2014-08-13 17:38:02.997427 7fe9bae887a0 20 read_log 194215'15283 (194215'15282) delete b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.2174595.0:137072 0.000000 2014-08-13 17:38:02.997437 7fe9bae887a0 20 read_log 194215'15284 (192888'14903) clean 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by osd.396.0:795 2014-08-12 15:46:23.313700 2014-08-13 17:38:02.997449 7fe9bae887a0 20 read_log 194215'15285 (194215'15284) delete 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.2174595.0:137074 0.000000 2014-08-13 17:38:02.997460 7fe9bae887a0 20 read_log 194215'15286 (192834'13970) clean 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by osd.396.0:797 2014-08-12 15:46:23.319927 2014-08-13 17:38:02.997472 7fe9bae887a0 20 read_log 194215'15287 (194215'15286) delete 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.2174595.0:137076 0.000000 2014-08-13 17:38:02.997483 7fe9bae887a0 20 read_log 194215'15288 (192834'14537) clean 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by osd.396.0:799 2014-08-12 15:46:23.325965 2014-08-13 17:38:02.997495 7fe9bae887a0 20 read_log 194215'15289 (194215'15288) delete 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.2174595.0:137078 0.000000 2014-08-13 17:38:02.997505 7fe9bae887a0 20 read_log 194215'15290 (192888'14867) clean dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by osd.396.0:801 2014-08-12 15:46:23.331452 2014-08-13 17:38:02.997517 7fe9bae887a0 20 read_log 194215'15291 (194215'15290) delete dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.2174595.0:137080 0.000000 2014-08-13 17:38:02.997528 7fe9bae887a0 20 read_log 194215'15292 (192888'14695) clean 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by osd.396.0:803 2014-08-12 15:46:23.336701 2014-08-13 17:38:02.997540 7fe9bae887a0 20 read_log 194215'15293 (194215'15292) delete 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.2174595.0:137082 0.000000 2014-08-13 17:38:02.997550 7fe9bae887a0 20 read_log 194215'15294 (192834'14024) clean 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by osd.396.0:805 2014-08-12 15:46:23.342473 2014-08-13 17:38:02.997562 7fe9bae887a0 20 read_log 194215'15295 (194215'15294) delete 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.2174595.0:137084 0.000000 2014-08-13 17:38:02.997572 7fe9bae887a0 20 read_log 194215'15296 (193305'15005) clean 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by osd.396.0:807 2014-08-12 15:46:23.348316 2014-08-13 17:38:02.997584 7fe9bae887a0 20 read_log 194215'15297 (194215'15296) delete 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.2174595.0:137086 0.000000 2014-08-13 17:38:02.997595 7fe9bae887a0 20 read_log 194215'15298 (193305'15014) clean b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by osd.396.0:809 2014-08-12 15:46:23.353740 2014-08-13 17:38:02.997606 7fe9bae887a0 20 read_log 194215'15299 (194215'15298) delete b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.2174595.0:137088 0.000000 2014-08-13 17:38:02.997617 7fe9bae887a0 20 read_log 194215'15300 (193305'14991) clean 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by osd.396.0:811 2014-08-12 15:46:23.359191 2014-08-13 17:38:02.997639 7fe9bae887a0 20 read_log 194215'15301 (194215'15300) delete 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.2174595.0:137090 0.000000 2014-08-13 17:38:02.997651 7fe9bae887a0 20 read_log 194215'15302 (192834'14459) clean 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by osd.396.0:813 2014-08-12 15:46:23.364238 2014-08-13 17:38:02.997663 7fe9bae887a0 20 read_log 194215'15303 (194215'15302) delete 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.2174595.0:137092 0.000000 2014-08-13 17:38:02.997693 7fe9bae887a0 20 read_log 194215'15304 (194215'15304) modify 299/hit_set_24.299_archive_2014-08-12 15:46:22.535891_2014-08-12 17:24:42.988027/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.988140 2014-08-13 17:38:02.997706 7fe9bae887a0 20 read_log 194215'15305 (194215'15016) delete 299/hit_set_24.299_archive_2014-08-12 15:32:25.825077_2014-08-12 15:46:22.535741/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.988140 2014-08-13 17:38:02.997719 7fe9bae887a0 20 read_log 194215'15306 (0'0) promote 1b056e99/benchmark_data_storage0111-ib.pocstor_34628_object790/head//24 by osd.396.0:1210 2014-08-12 17:24:42.990773 2014-08-13 17:38:02.997731 7fe9bae887a0 20 read_log 194215'15307 (194215'15306) modify 1b056e99/benchmark_data_storage0111-ib.pocstor_34628_object790/head//24 by client.2193759.0:791 2014-08-12 17:24:42.981361 2014-08-13 17:38:02.997744 7fe9bae887a0 20 read_log 194215'15308 (0'0) promote 43f8e99/benchmark_data_storage0111-ib.pocstor_34628_object881/head//24 by osd.396.0:1212 2014-08-12 17:24:43.333341 2014-08-13 17:38:02.997756 7fe9bae887a0 20 read_log 194215'15309 (194215'15308) modify 43f8e99/benchmark_data_storage0111-ib.pocstor_34628_object881/head//24 by client.2193759.0:882 2014-08-12 17:24:43.321742 2014-08-13 17:38:02.997768 7fe9bae887a0 20 read_log 194215'15310 (0'0) promote 3b59da99/benchmark_data_storage0111-ib.pocstor_34628_object1059/head//24 by osd.396.0:1214 2014-08-12 17:24:44.013517 2014-08-13 17:38:02.997780 7fe9bae887a0 20 read_log 194215'15311 (194215'15310) modify 3b59da99/benchmark_data_storage0111-ib.pocstor_34628_object1059/head//24 by client.2193759.0:1060 2014-08-12 17:24:44.005241 2014-08-13 17:38:02.997792 7fe9bae887a0 20 read_log 194370'15312 (0'0) promote 9cae0699/benchmark_data_storage0111-ib.pocstor_12495_object139/head//24 by osd.396.0:1223 2014-08-12 20:30:14.679773 2014-08-13 17:38:02.997804 7fe9bae887a0 20 read_log 194370'15313 (194370'15312) modify 9cae0699/benchmark_data_storage0111-ib.pocstor_12495_object139/head//24 by client.1508531.0:140 2014-08-12 20:30:14.670910 2014-08-13 17:38:02.997823 7fe9bae887a0 20 read_log 194370'15314 (0'0) promote 8aa7ee99/benchmark_data_storage0111-ib.pocstor_12495_object320/head//24 by osd.396.0:1229 2014-08-12 20:30:15.509343 2014-08-13 17:38:02.997835 7fe9bae887a0 20 read_log 194370'15315 (194370'15314) modify 8aa7ee99/benchmark_data_storage0111-ib.pocstor_12495_object320/head//24 by client.1508531.0:321 2014-08-12 20:30:15.501868 2014-08-13 17:38:02.997847 7fe9bae887a0 20 read_log 194370'15316 (0'0) promote acc91a99/benchmark_data_storage0111-ib.pocstor_12495_object987/head//24 by osd.396.0:1237 2014-08-12 20:30:18.080041 2014-08-13 17:38:02.997859 7fe9bae887a0 20 read_log 194370'15317 (194370'15316) modify acc91a99/benchmark_data_storage0111-ib.pocstor_12495_object987/head//24 by client.1508531.0:988 2014-08-12 20:30:18.072641 2014-08-13 17:38:02.997871 7fe9bae887a0 20 read_log 194370'15318 (0'0) promote 96764699/benchmark_data_storage0111-ib.pocstor_12495_object2180/head//24 by osd.396.0:1247 2014-08-12 20:30:22.575736 2014-08-13 17:38:02.997883 7fe9bae887a0 20 read_log 194370'15319 (194370'15318) modify 96764699/benchmark_data_storage0111-ib.pocstor_12495_object2180/head//24 by client.1508531.0:2181 2014-08-12 20:30:22.567206 2014-08-13 17:38:02.997903 7fe9bae887a0 20 read_log 194370'15320 (0'0) promote bff57e99/benchmark_data_storage0111-ib.pocstor_12495_object2493/head//24 by osd.396.0:1249 2014-08-12 20:30:23.715490 2014-08-13 17:38:02.997915 7fe9bae887a0 20 read_log 194370'15321 (194370'15320) modify bff57e99/benchmark_data_storage0111-ib.pocstor_12495_object2493/head//24 by client.1508531.0:2494 2014-08-12 20:30:23.704600 2014-08-13 17:38:02.997927 7fe9bae887a0 20 read_log 194370'15322 (0'0) promote 8181d299/benchmark_data_storage0111-ib.pocstor_12495_object4343/head//24 by osd.396.0:1261 2014-08-12 20:30:30.617042 2014-08-13 17:38:02.997939 7fe9bae887a0 20 read_log 194370'15323 (194370'15322) modify 8181d299/benchmark_data_storage0111-ib.pocstor_12495_object4343/head//24 by client.1508531.0:4344 2014-08-12 20:30:30.608998 2014-08-13 17:38:02.997951 7fe9bae887a0 20 read_log 194370'15324 (0'0) promote 247f6e99/benchmark_data_storage0111-ib.pocstor_12495_object5228/head//24 by osd.396.0:1269 2014-08-12 20:30:34.191157 2014-08-13 17:38:02.997963 7fe9bae887a0 20 read_log 194370'15325 (194370'15324) modify 247f6e99/benchmark_data_storage0111-ib.pocstor_12495_object5228/head//24 by client.1508531.0:5229 2014-08-12 20:30:34.181438 2014-08-13 17:38:02.997975 7fe9bae887a0 20 read_log 194370'15326 (0'0) promote 220a9a99/benchmark_data_storage0111-ib.pocstor_12495_object5651/head//24 by osd.396.0:1275 2014-08-12 20:30:36.123836 2014-08-13 17:38:02.997987 7fe9bae887a0 20 read_log 194370'15327 (194370'15326) modify 220a9a99/benchmark_data_storage0111-ib.pocstor_12495_object5651/head//24 by client.1508531.0:5652 2014-08-12 20:30:36.113731 2014-08-13 17:38:02.997999 7fe9bae887a0 20 read_log 194370'15328 (0'0) promote 3ae7fa99/benchmark_data_storage0111-ib.pocstor_12495_object6649/head//24 by osd.396.0:1279 2014-08-12 20:30:40.652283 2014-08-13 17:38:02.998011 7fe9bae887a0 20 read_log 194370'15329 (194370'15328) modify 3ae7fa99/benchmark_data_storage0111-ib.pocstor_12495_object6649/head//24 by client.1508531.0:6650 2014-08-12 20:30:40.645058 2014-08-13 17:38:02.998023 7fe9bae887a0 20 read_log 194370'15330 (0'0) promote 1f91a699/benchmark_data_storage0111-ib.pocstor_12495_object6708/head//24 by osd.396.0:1281 2014-08-12 20:30:40.914268 2014-08-13 17:38:02.998035 7fe9bae887a0 20 read_log 194370'15331 (194370'15330) modify 1f91a699/benchmark_data_storage0111-ib.pocstor_12495_object6708/head//24 by client.1508531.0:6709 2014-08-12 20:30:40.908250 2014-08-13 17:38:02.998047 7fe9bae887a0 20 read_log 194370'15332 (0'0) promote d24ae99/benchmark_data_storage0111-ib.pocstor_12495_object7741/head//24 by osd.396.0:1287 2014-08-12 20:30:46.194226 2014-08-13 17:38:02.998058 7fe9bae887a0 20 read_log 194370'15333 (194370'15332) modify d24ae99/benchmark_data_storage0111-ib.pocstor_12495_object7741/head//24 by client.1508531.0:7742 2014-08-12 20:30:46.186782 2014-08-13 17:38:02.998074 7fe9bae887a0 20 read_log 194370'15334 (0'0) promote 8759299/benchmark_data_storage0111-ib.pocstor_12495_object8285/head//24 by osd.396.0:1291 2014-08-12 20:30:49.087710 2014-08-13 17:38:02.998087 7fe9bae887a0 20 read_log 194370'15335 (194370'15334) modify 8759299/benchmark_data_storage0111-ib.pocstor_12495_object8285/head//24 by client.1508531.0:8286 2014-08-12 20:30:49.079026 2014-08-13 17:38:02.998099 7fe9bae887a0 20 read_log 194370'15336 (0'0) promote 24ed1a99/benchmark_data_storage0111-ib.pocstor_12495_object9282/head//24 by osd.396.0:1297 2014-08-12 20:30:55.021664 2014-08-13 17:38:02.998111 7fe9bae887a0 20 read_log 194370'15337 (194370'15336) modify 24ed1a99/benchmark_data_storage0111-ib.pocstor_12495_object9282/head//24 by client.1508531.0:9283 2014-08-12 20:30:55.012508 2014-08-13 17:38:02.998123 7fe9bae887a0 20 read_log 194370'15338 (0'0) promote d64e6299/benchmark_data_storage0111-ib.pocstor_12495_object10206/head//24 by osd.396.0:1301 2014-08-12 20:31:00.472447 2014-08-13 17:38:02.998135 7fe9bae887a0 20 read_log 194370'15339 (194370'15338) modify d64e6299/benchmark_data_storage0111-ib.pocstor_12495_object10206/head//24 by client.1508531.0:10207 2014-08-12 20:31:00.462507 2014-08-13 17:38:02.998160 7fe9bae887a0 20 read_log 194370'15340 (0'0) promote 38862299/benchmark_data_storage0111-ib.pocstor_12495_object10505/head//24 by osd.396.0:1305 2014-08-12 20:31:02.257545 2014-08-13 17:38:02.998174 7fe9bae887a0 20 read_log 194370'15341 (194370'15340) modify 38862299/benchmark_data_storage0111-ib.pocstor_12495_object10505/head//24 by client.1508531.0:10506 2014-08-12 20:31:02.249616 2014-08-13 17:38:02.998189 7fe9bae887a0 20 read_log 194370'15342 (0'0) promote ea103699/benchmark_data_storage0111-ib.pocstor_12495_object10949/head//24 by osd.396.0:1311 2014-08-12 20:31:04.977730 2014-08-13 17:38:02.998201 7fe9bae887a0 20 read_log 194370'15343 (194370'15342) modify ea103699/benchmark_data_storage0111-ib.pocstor_12495_object10949/head//24 by client.1508531.0:10950 2014-08-12 20:31:04.967557 2014-08-13 17:38:02.998213 7fe9bae887a0 20 read_log 194370'15344 (0'0) promote ae709e99/benchmark_data_storage0111-ib.pocstor_12495_object11463/head//24 by osd.396.0:1317 2014-08-12 20:31:08.088522 2014-08-13 17:38:02.998225 7fe9bae887a0 20 read_log 194370'15345 (194370'15344) modify ae709e99/benchmark_data_storage0111-ib.pocstor_12495_object11463/head//24 by client.1508531.0:11464 2014-08-12 20:31:08.079977 2014-08-13 17:38:02.998237 7fe9bae887a0 20 read_log 194370'15346 (0'0) promote 6d3fe699/benchmark_data_storage0111-ib.pocstor_12495_object11918/head//24 by osd.396.0:1319 2014-08-12 20:31:10.778867 2014-08-13 17:38:02.998249 7fe9bae887a0 20 read_log 194370'15347 (194370'15346) modify 6d3fe699/benchmark_data_storage0111-ib.pocstor_12495_object11918/head//24 by client.1508531.0:11919 2014-08-12 20:31:10.770035 2014-08-13 17:38:02.998261 7fe9bae887a0 20 read_log 194370'15348 (0'0) promote c7c61e99/benchmark_data_storage0111-ib.pocstor_12495_object12175/head//24 by osd.396.0:1321 2014-08-12 20:31:12.666304 2014-08-13 17:38:02.998273 7fe9bae887a0 20 read_log 194370'15349 (194370'15348) modify c7c61e99/benchmark_data_storage0111-ib.pocstor_12495_object12175/head//24 by client.1508531.0:12176 2014-08-12 20:31:12.657844 2014-08-13 17:38:02.998285 7fe9bae887a0 20 read_log 194370'15350 (0'0) promote 1ff19e99/benchmark_data_storage0111-ib.pocstor_12495_object12984/head//24 by osd.396.0:1329 2014-08-12 20:31:19.217025 2014-08-13 17:38:02.998297 7fe9bae887a0 20 read_log 194370'15351 (194370'15350) modify 1ff19e99/benchmark_data_storage0111-ib.pocstor_12495_object12984/head//24 by client.1508531.0:12985 2014-08-12 20:31:19.208131 2014-08-13 17:38:02.998309 7fe9bae887a0 20 read_log 194370'15352 (0'0) promote 5020e99/benchmark_data_storage0111-ib.pocstor_12495_object13208/head//24 by osd.396.0:1333 2014-08-12 20:31:21.390019 2014-08-13 17:38:02.998321 7fe9bae887a0 20 read_log 194370'15353 (194370'15352) modify 5020e99/benchmark_data_storage0111-ib.pocstor_12495_object13208/head//24 by client.1508531.0:13209 2014-08-12 20:31:21.380233 2014-08-13 17:38:02.998333 7fe9bae887a0 20 read_log 194377'15354 (0'0) promote 8212c299/benchmark_data_storage0111-ib.pocstor_25065_object972/head//24 by osd.396.0:1339 2014-08-12 20:33:01.202646 2014-08-13 17:38:02.998345 7fe9bae887a0 20 read_log 194377'15355 (194377'15354) modify 8212c299/benchmark_data_storage0111-ib.pocstor_25065_object972/head//24 by client.2231190.0:973 2014-08-12 20:33:01.194403 2014-08-13 17:38:02.998357 7fe9bae887a0 20 read_log 194377'15356 (0'0) promote b83d699/benchmark_data_storage0111-ib.pocstor_25065_object1317/head//24 by osd.396.0:1343 2014-08-12 20:33:02.516360 2014-08-13 17:38:02.998369 7fe9bae887a0 20 read_log 194377'15357 (194377'15356) modify b83d699/benchmark_data_storage0111-ib.pocstor_25065_object1317/head//24 by client.2231190.0:1318 2014-08-12 20:33:02.508595 2014-08-13 17:38:02.998381 7fe9bae887a0 20 read_log 194377'15358 (0'0) promote 9fa81699/benchmark_data_storage0111-ib.pocstor_25065_object1483/head//24 by osd.396.0:1345 2014-08-12 20:33:03.137526 2014-08-13 17:38:02.998401 7fe9bae887a0 20 read_log 194377'15359 (194377'15358) modify 9fa81699/benchmark_data_storage0111-ib.pocstor_25065_object1483/head//24 by client.2231190.0:1484 2014-08-12 20:33:03.131429 2014-08-13 17:38:02.998414 7fe9bae887a0 20 read_log 194377'15360 (0'0) promote 742d9a99/benchmark_data_storage0111-ib.pocstor_25065_object3219/head//24 by osd.396.0:1353 2014-08-12 20:33:10.154487 2014-08-13 17:38:02.998425 7fe9bae887a0 20 read_log 194377'15361 (194377'15360) modify 742d9a99/benchmark_data_storage0111-ib.pocstor_25065_object3219/head//24 by client.2231190.0:3220 2014-08-12 20:33:10.146006 2014-08-13 17:38:02.998438 7fe9bae887a0 20 read_log 194377'15362 (0'0) promote 72c32a99/benchmark_data_storage0111-ib.pocstor_25065_object3233/head//24 by osd.396.0:1355 2014-08-12 20:33:10.205763 2014-08-13 17:38:02.998449 7fe9bae887a0 20 read_log 194377'15363 (194377'15362) modify 72c32a99/benchmark_data_storage0111-ib.pocstor_25065_object3233/head//24 by client.2231190.0:3234 2014-08-12 20:33:10.197907 2014-08-13 17:38:02.998462 7fe9bae887a0 20 read_log 194383'15364 (0'0) promote 38c90299/benchmark_data_storage0111-ib.pocstor_25065_object6446/head//24 by osd.396.0:1369 2014-08-12 20:34:03.906698 2014-08-13 17:38:02.998474 7fe9bae887a0 20 read_log 194383'15365 (194383'15364) modify 38c90299/benchmark_data_storage0111-ib.pocstor_25065_object6446/head//24 by client.2231190.0:6447 2014-08-12 20:34:03.897689 2014-08-13 17:38:02.998486 7fe9bae887a0 20 read_log 194383'15366 (0'0) promote d1656299/benchmark_data_storage0111-ib.pocstor_25065_object6747/head//24 by osd.396.0:1371 2014-08-12 20:34:08.294014 2014-08-13 17:38:02.998497 7fe9bae887a0 20 read_log 194383'15367 (194383'15366) modify d1656299/benchmark_data_storage0111-ib.pocstor_25065_object6747/head//24 by client.2231190.0:6748 2014-08-12 20:34:08.284675 2014-08-13 17:38:02.998509 7fe9bae887a0 20 read_log 194383'15368 (0'0) promote ba77ea99/benchmark_data_storage0111-ib.pocstor_25065_object7383/head//24 by osd.396.0:1377 2014-08-12 20:34:25.601299 2014-08-13 17:38:02.998521 7fe9bae887a0 20 read_log 194383'15369 (194383'15368) modify ba77ea99/benchmark_data_storage0111-ib.pocstor_25065_object7383/head//24 by client.2231190.0:7384 2014-08-12 20:34:25.592195 2014-08-13 17:38:02.998533 7fe9bae887a0 20 read_log 194383'15370 (0'0) promote 6f6aaa99/benchmark_data_storage0111-ib.pocstor_25065_object7457/head//24 by osd.396.0:1379 2014-08-12 20:34:27.606043 2014-08-13 17:38:02.998545 7fe9bae887a0 20 read_log 194383'15371 (194383'15370) modify 6f6aaa99/benchmark_data_storage0111-ib.pocstor_25065_object7457/head//24 by client.2231190.0:7458 2014-08-12 20:34:27.596612 2014-08-13 17:38:02.998557 7fe9bae887a0 20 read_log 194416'15372 (0'0) promote 3c19699/benchmark_data_storage0111-ib.pocstor_19103_object1477/head//24 by osd.396.0:1453 2014-08-13 15:28:36.326543 2014-08-13 17:38:02.998569 7fe9bae887a0 20 read_log 194416'15373 (194416'15372) modify 3c19699/benchmark_data_storage0111-ib.pocstor_19103_object1477/head//24 by client.2441958.0:1478 2014-08-13 15:28:36.317322 2014-08-13 17:38:02.998581 7fe9bae887a0 20 read_log 194416'15374 (0'0) promote 74d0d699/benchmark_data_storage0111-ib.pocstor_19103_object1698/head//24 by osd.396.0:1457 2014-08-13 15:28:37.159528 2014-08-13 17:38:02.998593 7fe9bae887a0 20 read_log 194416'15375 (194416'15374) modify 74d0d699/benchmark_data_storage0111-ib.pocstor_19103_object1698/head//24 by client.2441958.0:1699 2014-08-13 15:28:37.149912 2014-08-13 17:38:02.998605 7fe9bae887a0 20 read_log 194416'15376 (0'0) promote 8bf9f699/benchmark_data_storage0111-ib.pocstor_19103_object3913/head//24 by osd.396.0:1465 2014-08-13 15:28:45.608453 2014-08-13 17:38:02.998617 7fe9bae887a0 20 read_log 194416'15377 (194416'15376) modify 8bf9f699/benchmark_data_storage0111-ib.pocstor_19103_object3913/head//24 by client.2441958.0:3914 2014-08-13 15:28:45.598890 2014-08-13 17:38:02.998652 7fe9bae887a0 20 read_log 194416'15378 (0'0) promote c1e70e99/benchmark_data_storage0111-ib.pocstor_19103_object4064/head//24 by osd.396.0:1467 2014-08-13 15:28:46.180461 2014-08-13 17:38:02.998665 7fe9bae887a0 20 read_log 194416'15379 (194416'15378) modify c1e70e99/benchmark_data_storage0111-ib.pocstor_19103_object4064/head//24 by client.2441958.0:4065 2014-08-13 15:28:46.171096 2014-08-13 17:38:02.998677 7fe9bae887a0 20 read_log 194416'15380 (0'0) promote c1ac6e99/benchmark_data_storage0111-ib.pocstor_19103_object5160/head//24 by osd.396.0:1471 2014-08-13 15:28:50.513119 2014-08-13 17:38:02.998689 7fe9bae887a0 20 read_log 194416'15381 (194416'15380) modify c1ac6e99/benchmark_data_storage0111-ib.pocstor_19103_object5160/head//24 by client.2441958.0:5161 2014-08-13 15:28:50.503745 2014-08-13 17:38:02.998701 7fe9bae887a0 20 read_log 194416'15382 (0'0) promote 8b29a99/benchmark_data_storage0111-ib.pocstor_19103_object8488/head//24 by osd.396.0:1489 2014-08-13 15:29:03.383821 2014-08-13 17:38:02.998715 7fe9bae887a0 20 read_log 194416'15383 (194416'15382) modify 8b29a99/benchmark_data_storage0111-ib.pocstor_19103_object8488/head//24 by client.2441958.0:8489 2014-08-13 15:29:03.372840 2014-08-13 17:38:02.998728 7fe9bae887a0 20 read_log 194416'15384 (0'0) promote bd899699/benchmark_data_storage0111-ib.pocstor_19103_object9467/head//24 by osd.396.0:1493 2014-08-13 15:29:07.254506 2014-08-13 17:38:02.998740 7fe9bae887a0 20 read_log 194416'15385 (194416'15384) modify bd899699/benchmark_data_storage0111-ib.pocstor_19103_object9467/head//24 by client.2441958.0:9468 2014-08-13 15:29:07.245297 2014-08-13 17:38:02.998752 7fe9bae887a0 20 read_log 194416'15386 (0'0) promote f81cda99/benchmark_data_storage0111-ib.pocstor_19103_object10320/head//24 by osd.396.0:1505 2014-08-13 15:29:10.767106 2014-08-13 17:38:02.998764 7fe9bae887a0 20 read_log 194416'15387 (194416'15386) modify f81cda99/benchmark_data_storage0111-ib.pocstor_19103_object10320/head//24 by client.2441958.0:10321 2014-08-13 15:29:10.760119 2014-08-13 17:38:02.998776 7fe9bae887a0 20 read_log 194416'15388 (0'0) promote cd422299/benchmark_data_storage0111-ib.pocstor_19103_object13546/head//24 by osd.396.0:1517 2014-08-13 15:29:24.325197 2014-08-13 17:38:02.998787 7fe9bae887a0 20 read_log 194416'15389 (194416'15388) modify cd422299/benchmark_data_storage0111-ib.pocstor_19103_object13546/head//24 by client.2441958.0:13547 2014-08-13 15:29:24.314471 2014-08-13 17:38:02.998799 7fe9bae887a0 20 read_log 194416'15390 (0'0) promote e86a3299/benchmark_data_storage0111-ib.pocstor_19103_object15375/head//24 by osd.396.0:1529 2014-08-13 15:29:31.995192 2014-08-13 17:38:02.998812 7fe9bae887a0 20 read_log 194416'15391 (194416'15390) modify e86a3299/benchmark_data_storage0111-ib.pocstor_19103_object15375/head//24 by client.2441958.0:15376 2014-08-13 15:29:31.986210 2014-08-13 17:38:02.998823 7fe9bae887a0 20 read_log 194416'15392 (0'0) promote b4fee299/benchmark_data_storage0111-ib.pocstor_19103_object18117/head//24 by osd.396.0:1537 2014-08-13 15:30:03.118564 2014-08-13 17:38:02.998835 7fe9bae887a0 20 read_log 194416'15393 (194416'15392) modify b4fee299/benchmark_data_storage0111-ib.pocstor_19103_object18117/head//24 by client.2441958.0:18118 2014-08-13 15:30:03.107935 2014-08-13 17:38:02.998848 7fe9bae887a0 20 read_log 194422'15394 (0'0) promote 9c655e99/benchmark_data_storage0111-ib.pocstor_41231_object391/head//24 by osd.396.0:1543 2014-08-13 15:32:16.803859 2014-08-13 17:38:02.998859 7fe9bae887a0 20 read_log 194422'15395 (194422'15394) modify 9c655e99/benchmark_data_storage0111-ib.pocstor_41231_object391/head//24 by client.1148203.0:392 2014-08-13 15:32:16.794636 2014-08-13 17:38:02.998871 7fe9bae887a0 20 read_log 194422'15396 (0'0) promote 2d89ae99/benchmark_data_storage0111-ib.pocstor_41231_object1748/head//24 by osd.396.0:1553 2014-08-13 15:32:22.258762 2014-08-13 17:38:02.998923 7fe9bae887a0 20 read_log 194422'15397 (194422'15396) modify 2d89ae99/benchmark_data_storage0111-ib.pocstor_41231_object1748/head//24 by client.1148203.0:1749 2014-08-13 15:32:22.245855 2014-08-13 17:38:02.998948 7fe9bae887a0 20 read_log 194422'15398 (0'0) promote 8f21d299/benchmark_data_storage0111-ib.pocstor_41231_object2063/head//24 by osd.396.0:1555 2014-08-13 15:32:23.478200 2014-08-13 17:38:02.998961 7fe9bae887a0 20 read_log 194422'15399 (194422'15398) modify 8f21d299/benchmark_data_storage0111-ib.pocstor_41231_object2063/head//24 by client.1148203.0:2064 2014-08-13 15:32:23.469696 2014-08-13 17:38:02.998973 7fe9bae887a0 20 read_log 194422'15400 (0'0) promote a4adc699/benchmark_data_storage0111-ib.pocstor_41231_object4764/head//24 by osd.396.0:1559 2014-08-13 15:32:33.947276 2014-08-13 17:38:02.998985 7fe9bae887a0 20 read_log 194422'15401 (194422'15400) modify a4adc699/benchmark_data_storage0111-ib.pocstor_41231_object4764/head//24 by client.1148203.0:4765 2014-08-13 15:32:33.937347 2014-08-13 17:38:02.998998 7fe9bae887a0 20 read_log 194422'15402 (0'0) promote 9e665299/benchmark_data_storage0111-ib.pocstor_41231_object5122/head//24 by osd.396.0:1565 2014-08-13 15:32:35.439211 2014-08-13 17:38:02.999009 7fe9bae887a0 20 read_log 194422'15403 (194422'15402) modify 9e665299/benchmark_data_storage0111-ib.pocstor_41231_object5122/head//24 by client.1148203.0:5123 2014-08-13 15:32:35.430767 2014-08-13 17:38:02.999021 7fe9bae887a0 20 read_log 194422'15404 (0'0) promote 3ed83a99/benchmark_data_storage0111-ib.pocstor_41231_object7336/head//24 by osd.396.0:1573 2014-08-13 15:32:47.594955 2014-08-13 17:38:02.999033 7fe9bae887a0 20 read_log 194422'15405 (194422'15404) modify 3ed83a99/benchmark_data_storage0111-ib.pocstor_41231_object7336/head//24 by client.1148203.0:7337 2014-08-13 15:32:47.584290 2014-08-13 17:38:02.999045 7fe9bae887a0 20 read_log 194422'15406 (0'0) promote 24483299/benchmark_data_storage0111-ib.pocstor_41231_object8416/head//24 by osd.396.0:1579 2014-08-13 15:32:54.484376 2014-08-13 17:38:02.999057 7fe9bae887a0 20 read_log 194422'15407 (194422'15406) modify 24483299/benchmark_data_storage0111-ib.pocstor_41231_object8416/head//24 by client.1148203.0:8417 2014-08-13 15:32:54.475561 2014-08-13 17:38:02.999069 7fe9bae887a0 20 read_log 194422'15408 (0'0) promote aaac3299/benchmark_data_storage0111-ib.pocstor_41231_object8504/head//24 by osd.396.0:1581 2014-08-13 15:32:55.161577 2014-08-13 17:38:02.999081 7fe9bae887a0 20 read_log 194422'15409 (194422'15408) modify aaac3299/benchmark_data_storage0111-ib.pocstor_41231_object8504/head//24 by client.1148203.0:8505 2014-08-13 15:32:55.152028 2014-08-13 17:38:02.999093 7fe9bae887a0 20 read_log 194422'15410 (0'0) promote 1a22be99/benchmark_data_storage0111-ib.pocstor_41231_object8712/head//24 by osd.396.0:1585 2014-08-13 15:32:56.796657 2014-08-13 17:38:02.999105 7fe9bae887a0 20 read_log 194422'15411 (194422'15410) modify 1a22be99/benchmark_data_storage0111-ib.pocstor_41231_object8712/head//24 by client.1148203.0:8713 2014-08-13 15:32:56.788368 2014-08-13 17:38:02.999120 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:02.999532 7fe9bae887a0 10 read_log done 2014-08-13 17:38:02.999562 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:02.999578 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.011299 0 0.000000 2014-08-13 17:38:02.999591 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:02.999600 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:02.999610 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] log((192834'13573,194422'15411], crt=0'0) 2014-08-13 17:38:02.999620 7fe9bae887a0 10 osd.4 194478 pgid 24.2a1 coll 24.2a1_head 2014-08-13 17:38:02.999734 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.2a1 2014-08-13 17:38:02.999744 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:02.999772 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2a1(unlocked)] enter Initial 2014-08-13 17:38:02.999781 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.2a1(unlocked)] enter NotTrimming 2014-08-13 17:38:02.999906 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.000025 7fe9bae887a0 20 read_log 192834'12414 (0'0) promote 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.331.0:39406 2014-08-11 15:23:52.017585 2014-08-13 17:38:03.000042 7fe9bae887a0 20 read_log 192834'12415 (192834'12414) modify 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by client.933448.0:3488 2014-08-11 15:23:52.010687 2014-08-13 17:38:03.000055 7fe9bae887a0 20 read_log 192834'12446 (0'0) promote 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by osd.331.0:39468 2014-08-11 15:23:59.082691 2014-08-13 17:38:03.000067 7fe9bae887a0 20 read_log 192834'12447 (192834'12446) modify 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.933448.0:4859 2014-08-11 15:23:59.072933 2014-08-13 17:38:03.000080 7fe9bae887a0 20 read_log 192834'12454 (0'0) promote 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.331.0:39488 2014-08-11 15:24:00.677158 2014-08-13 17:38:03.000093 7fe9bae887a0 20 read_log 192834'12455 (192834'12454) modify 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by client.933448.0:5257 2014-08-11 15:24:00.669125 2014-08-13 17:38:03.000106 7fe9bae887a0 20 read_log 192834'12470 (0'0) promote 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by osd.331.0:39538 2014-08-11 15:24:04.449920 2014-08-13 17:38:03.000118 7fe9bae887a0 20 read_log 192834'12471 (192834'12470) modify 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.933448.0:6185 2014-08-11 15:24:04.440367 2014-08-13 17:38:03.000130 7fe9bae887a0 20 read_log 192834'12486 (0'0) promote 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by osd.331.0:39606 2014-08-11 15:24:10.641221 2014-08-13 17:38:03.000152 7fe9bae887a0 20 read_log 192834'12487 (192834'12486) modify 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.933448.0:7586 2014-08-11 15:24:10.634236 2014-08-13 17:38:03.000172 7fe9bae887a0 20 read_log 192834'12500 (0'0) promote 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by osd.331.0:39642 2014-08-11 15:24:14.214385 2014-08-13 17:38:03.000184 7fe9bae887a0 20 read_log 192834'12501 (192834'12500) modify 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.933448.0:8468 2014-08-11 15:24:14.206252 2014-08-13 17:38:03.000197 7fe9bae887a0 20 read_log 192834'12516 (0'0) promote 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by osd.331.0:39678 2014-08-11 15:24:18.394517 2014-08-13 17:38:03.000209 7fe9bae887a0 20 read_log 192834'12517 (192834'12516) modify 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.933448.0:9318 2014-08-11 15:24:18.386610 2014-08-13 17:38:03.000221 7fe9bae887a0 20 read_log 192834'12520 (0'0) promote ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by osd.331.0:39690 2014-08-11 15:24:19.031831 2014-08-13 17:38:03.000233 7fe9bae887a0 20 read_log 192834'12521 (192834'12520) modify ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.933448.0:9494 2014-08-11 15:24:19.023932 2014-08-13 17:38:03.000245 7fe9bae887a0 20 read_log 192834'12526 (0'0) promote 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by osd.331.0:39720 2014-08-11 15:24:22.703587 2014-08-13 17:38:03.000257 7fe9bae887a0 20 read_log 192834'12527 (192834'12526) modify 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.933448.0:10334 2014-08-11 15:24:22.694393 2014-08-13 17:38:03.000269 7fe9bae887a0 20 read_log 192834'12550 (0'0) promote 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by osd.331.0:39780 2014-08-11 15:24:27.073502 2014-08-13 17:38:03.000281 7fe9bae887a0 20 read_log 192834'12551 (192834'12550) modify 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.933448.0:11265 2014-08-11 15:24:27.065584 2014-08-13 17:38:03.000293 7fe9bae887a0 20 read_log 192834'12576 (0'0) promote c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by osd.331.0:39860 2014-08-11 15:24:34.127069 2014-08-13 17:38:03.000305 7fe9bae887a0 20 read_log 192834'12577 (192834'12576) modify c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.933448.0:12840 2014-08-11 15:24:34.118518 2014-08-13 17:38:03.000317 7fe9bae887a0 20 read_log 192834'12578 (0'0) promote 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by osd.331.0:39862 2014-08-11 15:24:34.211178 2014-08-13 17:38:03.000329 7fe9bae887a0 20 read_log 192834'12579 (192834'12578) modify 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.933448.0:12860 2014-08-11 15:24:34.203522 2014-08-13 17:38:03.000341 7fe9bae887a0 20 read_log 192834'12584 (0'0) promote 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.331.0:39874 2014-08-11 15:24:35.714231 2014-08-13 17:38:03.000353 7fe9bae887a0 20 read_log 192834'12585 (192834'12584) modify 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by client.933448.0:13170 2014-08-11 15:24:35.705231 2014-08-13 17:38:03.000365 7fe9bae887a0 20 read_log 192834'12586 (0'0) promote d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by osd.331.0:39876 2014-08-11 15:24:35.810678 2014-08-13 17:38:03.000377 7fe9bae887a0 20 read_log 192834'12587 (192834'12586) modify d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.933448.0:13195 2014-08-11 15:24:35.801548 2014-08-13 17:38:03.000389 7fe9bae887a0 20 read_log 192834'12606 (0'0) promote b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by osd.331.0:39931 2014-08-11 15:24:40.869111 2014-08-13 17:38:03.000408 7fe9bae887a0 20 read_log 192834'12607 (192834'12606) modify b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.933448.0:14030 2014-08-11 15:24:40.860659 2014-08-13 17:38:03.000421 7fe9bae887a0 20 read_log 192834'12610 (0'0) promote bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.331.0:39947 2014-08-11 15:24:41.958964 2014-08-13 17:38:03.000433 7fe9bae887a0 20 read_log 192834'12611 (192834'12610) modify bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by client.933448.0:14186 2014-08-11 15:24:41.950259 2014-08-13 17:38:03.000445 7fe9bae887a0 20 read_log 192834'12620 (0'0) promote f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by osd.331.0:40032 2014-08-11 15:24:49.011368 2014-08-13 17:38:03.000457 7fe9bae887a0 20 read_log 192834'12621 (192834'12620) modify f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.933448.0:15265 2014-08-11 15:24:49.003253 2014-08-13 17:38:03.000469 7fe9bae887a0 20 read_log 192834'12634 (0'0) promote d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by osd.331.0:40087 2014-08-11 15:24:56.177592 2014-08-13 17:38:03.000481 7fe9bae887a0 20 read_log 192834'12635 (192834'12634) modify d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.933448.0:16547 2014-08-11 15:24:56.169692 2014-08-13 17:38:03.000493 7fe9bae887a0 20 read_log 192834'12638 (0'0) promote 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by osd.331.0:40095 2014-08-11 15:24:56.517409 2014-08-13 17:38:03.000505 7fe9bae887a0 20 read_log 192834'12639 (192834'12638) modify 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.933448.0:16637 2014-08-11 15:24:56.508864 2014-08-13 17:38:03.000517 7fe9bae887a0 20 read_log 192834'12676 (0'0) promote eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by osd.331.0:40325 2014-08-11 15:25:16.765463 2014-08-13 17:38:03.000529 7fe9bae887a0 20 read_log 192834'12677 (192834'12676) modify eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.933448.0:20135 2014-08-11 15:25:16.756929 2014-08-13 17:38:03.000541 7fe9bae887a0 20 read_log 192834'12690 (0'0) promote 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by osd.331.0:40375 2014-08-11 15:25:21.222348 2014-08-13 17:38:03.000553 7fe9bae887a0 20 read_log 192834'12691 (192834'12690) modify 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.933448.0:20506 2014-08-11 15:25:21.213155 2014-08-13 17:38:03.000565 7fe9bae887a0 20 read_log 192834'12694 (0'0) promote c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.331.0:40412 2014-08-11 15:25:28.117708 2014-08-13 17:38:03.000577 7fe9bae887a0 20 read_log 192834'12695 (192834'12694) modify c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by client.933448.0:21246 2014-08-11 15:25:28.107590 2014-08-13 17:38:03.000589 7fe9bae887a0 20 read_log 192834'12704 (0'0) promote 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by osd.331.0:40468 2014-08-11 15:25:31.261997 2014-08-13 17:38:03.000600 7fe9bae887a0 20 read_log 192834'12705 (192834'12704) modify 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.933448.0:21622 2014-08-11 15:25:31.253902 2014-08-13 17:38:03.000613 7fe9bae887a0 20 read_log 192834'12750 (192834'12415) modify 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by client.933448.0:25779 2014-08-11 15:25:42.436494 2014-08-13 17:38:03.000625 7fe9bae887a0 20 read_log 192834'12766 (192834'12447) modify 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.933448.0:27150 2014-08-11 15:25:44.156238 2014-08-13 17:38:03.000637 7fe9bae887a0 20 read_log 192834'12770 (192834'12455) modify 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by client.933448.0:27548 2014-08-11 15:25:44.622989 2014-08-13 17:38:03.000660 7fe9bae887a0 20 read_log 192834'12778 (192834'12471) modify 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.933448.0:28476 2014-08-11 15:25:46.272787 2014-08-13 17:38:03.000673 7fe9bae887a0 20 read_log 192834'12786 (192834'12487) modify 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.933448.0:29877 2014-08-11 15:25:48.062553 2014-08-13 17:38:03.000686 7fe9bae887a0 20 read_log 192834'12793 (192834'12501) modify 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.933448.0:30759 2014-08-11 15:25:49.204087 2014-08-13 17:38:03.000698 7fe9bae887a0 20 read_log 192834'12801 (192834'12517) modify 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.933448.0:31609 2014-08-11 15:25:50.269438 2014-08-13 17:38:03.000710 7fe9bae887a0 20 read_log 192834'12803 (192834'12521) modify ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.933448.0:31785 2014-08-11 15:25:50.601978 2014-08-13 17:38:03.000723 7fe9bae887a0 20 read_log 192834'12806 (192834'12527) modify 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.933448.0:32625 2014-08-11 15:25:52.120864 2014-08-13 17:38:03.000735 7fe9bae887a0 20 read_log 192834'12818 (192834'12551) modify 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.933448.0:33556 2014-08-11 15:25:53.671213 2014-08-13 17:38:03.000747 7fe9bae887a0 20 read_log 192834'12831 (192834'12577) modify c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.933448.0:35131 2014-08-11 15:25:55.848803 2014-08-13 17:38:03.000759 7fe9bae887a0 20 read_log 192834'12832 (192834'12579) modify 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.933448.0:35151 2014-08-11 15:25:55.881915 2014-08-13 17:38:03.000771 7fe9bae887a0 20 read_log 192834'12835 (192834'12585) modify 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by client.933448.0:35461 2014-08-11 15:25:56.313517 2014-08-13 17:38:03.000782 7fe9bae887a0 20 read_log 192834'12836 (192834'12587) modify d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.933448.0:35486 2014-08-11 15:25:56.368485 2014-08-13 17:38:03.000794 7fe9bae887a0 20 read_log 192834'12846 (192834'12607) modify b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.933448.0:36321 2014-08-11 15:25:57.525740 2014-08-13 17:38:03.000806 7fe9bae887a0 20 read_log 192834'12848 (192834'12611) modify bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by client.933448.0:36477 2014-08-11 15:25:57.789503 2014-08-13 17:38:03.000818 7fe9bae887a0 20 read_log 192834'12853 (192834'12621) modify f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.933448.0:37556 2014-08-11 15:25:59.675508 2014-08-13 17:38:03.000830 7fe9bae887a0 20 read_log 192834'12860 (192834'12635) modify d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.933448.0:38838 2014-08-11 15:26:01.321465 2014-08-13 17:38:03.000842 7fe9bae887a0 20 read_log 192834'12862 (192834'12639) modify 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.933448.0:38928 2014-08-11 15:26:01.407197 2014-08-13 17:38:03.000854 7fe9bae887a0 20 read_log 192834'12881 (192834'12677) modify eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.933448.0:42426 2014-08-11 15:26:03.276578 2014-08-13 17:38:03.000866 7fe9bae887a0 20 read_log 192834'12888 (192834'12691) modify 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.933448.0:42797 2014-08-11 15:26:03.390498 2014-08-13 17:38:03.000879 7fe9bae887a0 20 read_log 192834'12890 (192834'12695) modify c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by client.933448.0:43537 2014-08-11 15:26:03.652556 2014-08-13 17:38:03.000890 7fe9bae887a0 20 read_log 192834'12895 (192834'12705) modify 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.933448.0:43913 2014-08-11 15:26:03.760571 2014-08-13 17:38:03.000910 7fe9bae887a0 20 read_log 192834'12915 (0'0) promote 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by osd.331.0:41302 2014-08-11 15:26:16.640917 2014-08-13 17:38:03.000923 7fe9bae887a0 20 read_log 192834'12916 (192834'12915) modify 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.1883634.0:589 2014-08-11 15:26:16.633316 2014-08-13 17:38:03.000936 7fe9bae887a0 20 read_log 192834'12919 (0'0) promote 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by osd.331.0:41334 2014-08-11 15:26:18.920549 2014-08-13 17:38:03.000948 7fe9bae887a0 20 read_log 192834'12920 (192834'12919) modify 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.1883634.0:1219 2014-08-11 15:26:18.912827 2014-08-13 17:38:03.000983 7fe9bae887a0 20 read_log 192834'12923 (0'0) promote 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by osd.331.0:41340 2014-08-11 15:26:19.356170 2014-08-13 17:38:03.000996 7fe9bae887a0 20 read_log 192834'12924 (192834'12923) modify 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.1883634.0:1339 2014-08-11 15:26:19.348176 2014-08-13 17:38:03.001008 7fe9bae887a0 20 read_log 192834'12939 (0'0) promote 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.331.0:41382 2014-08-11 15:26:22.077844 2014-08-13 17:38:03.001020 7fe9bae887a0 20 read_log 192834'12940 (192834'12939) modify 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by client.1883634.0:2058 2014-08-11 15:26:22.068822 2014-08-13 17:38:03.001033 7fe9bae887a0 20 read_log 192834'12975 (0'0) promote 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by osd.331.0:41518 2014-08-11 15:26:32.163631 2014-08-13 17:38:03.001045 7fe9bae887a0 20 read_log 192834'12976 (192834'12975) modify 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.1883634.0:4838 2014-08-11 15:26:32.154695 2014-08-13 17:38:03.001057 7fe9bae887a0 20 read_log 192834'12985 (0'0) promote fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.331.0:41534 2014-08-11 15:26:33.858737 2014-08-13 17:38:03.001069 7fe9bae887a0 20 read_log 192834'12986 (192834'12985) modify fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by client.1883634.0:5283 2014-08-11 15:26:33.850962 2014-08-13 17:38:03.001081 7fe9bae887a0 20 read_log 192834'12993 (0'0) promote 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by osd.331.0:41550 2014-08-11 15:26:34.483720 2014-08-13 17:38:03.001093 7fe9bae887a0 20 read_log 192834'12994 (192834'12993) modify 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.1883634.0:5461 2014-08-11 15:26:34.477407 2014-08-13 17:38:03.001105 7fe9bae887a0 20 read_log 192834'13001 (0'0) promote 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by osd.331.0:41566 2014-08-11 15:26:36.244507 2014-08-13 17:38:03.001117 7fe9bae887a0 20 read_log 192834'13002 (192834'13001) modify 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.1883634.0:5953 2014-08-11 15:26:36.235454 2014-08-13 17:38:03.001129 7fe9bae887a0 20 read_log 192834'13005 (0'0) promote 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by osd.331.0:41578 2014-08-11 15:26:36.910690 2014-08-13 17:38:03.001141 7fe9bae887a0 20 read_log 192834'13006 (192834'13005) modify 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.1883634.0:6140 2014-08-11 15:26:36.902510 2014-08-13 17:38:03.001162 7fe9bae887a0 20 read_log 192834'13013 (0'0) promote 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.331.0:41615 2014-08-11 15:26:39.365989 2014-08-13 17:38:03.001176 7fe9bae887a0 20 read_log 192834'13014 (192834'13013) modify 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by client.1883634.0:6746 2014-08-11 15:26:39.357196 2014-08-13 17:38:03.001196 7fe9bae887a0 20 read_log 192834'13015 (0'0) promote baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by osd.331.0:41621 2014-08-11 15:26:39.510893 2014-08-13 17:38:03.001209 7fe9bae887a0 20 read_log 192834'13016 (192834'13015) modify baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.1883634.0:6788 2014-08-11 15:26:39.501640 2014-08-13 17:38:03.001221 7fe9bae887a0 20 read_log 192834'13023 (0'0) promote 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.331.0:41647 2014-08-11 15:26:41.572610 2014-08-13 17:38:03.001233 7fe9bae887a0 20 read_log 192834'13024 (192834'13023) modify 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by client.1883634.0:7300 2014-08-11 15:26:41.564159 2014-08-13 17:38:03.001245 7fe9bae887a0 20 read_log 192834'13049 (0'0) promote 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by osd.331.0:41790 2014-08-11 15:26:47.159627 2014-08-13 17:38:03.001257 7fe9bae887a0 20 read_log 192834'13050 (192834'13049) modify 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.1883634.0:8627 2014-08-11 15:26:47.152674 2014-08-13 17:38:03.001269 7fe9bae887a0 20 read_log 192834'13063 (0'0) promote 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by osd.331.0:41863 2014-08-11 15:26:51.091448 2014-08-13 17:38:03.001282 7fe9bae887a0 20 read_log 192834'13064 (192834'13063) modify 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.1883634.0:9503 2014-08-11 15:26:51.083619 2014-08-13 17:38:03.001294 7fe9bae887a0 20 read_log 192834'13067 (0'0) promote e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by osd.331.0:41877 2014-08-11 15:26:52.275236 2014-08-13 17:38:03.001306 7fe9bae887a0 20 read_log 192834'13068 (192834'13067) modify e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.1883634.0:9760 2014-08-11 15:26:52.264486 2014-08-13 17:38:03.001318 7fe9bae887a0 20 read_log 192834'13079 (0'0) promote 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.331.0:41912 2014-08-11 15:26:54.859496 2014-08-13 17:38:03.001330 7fe9bae887a0 20 read_log 192834'13080 (192834'13079) modify 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by client.1883634.0:10380 2014-08-11 15:26:54.847812 2014-08-13 17:38:03.001342 7fe9bae887a0 20 read_log 192834'13083 (0'0) promote a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.331.0:41959 2014-08-11 15:26:58.104026 2014-08-13 17:38:03.001354 7fe9bae887a0 20 read_log 192834'13084 (192834'13083) modify a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by client.1883634.0:11048 2014-08-11 15:26:58.095225 2014-08-13 17:38:03.001366 7fe9bae887a0 20 read_log 192834'13105 (0'0) promote 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.331.0:42038 2014-08-11 15:27:01.711486 2014-08-13 17:38:03.001378 7fe9bae887a0 20 read_log 192834'13106 (192834'13105) modify 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by client.1883634.0:11916 2014-08-11 15:27:01.701462 2014-08-13 17:38:03.001390 7fe9bae887a0 20 read_log 192834'13111 (0'0) promote 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.331.0:42050 2014-08-11 15:27:02.757782 2014-08-13 17:38:03.001402 7fe9bae887a0 20 read_log 192834'13112 (192834'13111) modify 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by client.1883634.0:12157 2014-08-11 15:27:02.747501 2014-08-13 17:38:03.001415 7fe9bae887a0 20 read_log 192834'13119 (0'0) promote 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by osd.331.0:42068 2014-08-11 15:27:04.163667 2014-08-13 17:38:03.001427 7fe9bae887a0 20 read_log 192834'13120 (192834'13119) modify 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.1883634.0:12471 2014-08-11 15:27:04.154621 2014-08-13 17:38:03.001448 7fe9bae887a0 20 read_log 192834'13125 (0'0) promote eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.331.0:42269 2014-08-11 15:27:07.271244 2014-08-13 17:38:03.001461 7fe9bae887a0 20 read_log 192834'13126 (192834'13125) modify eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by client.1883634.0:13026 2014-08-11 15:27:07.262935 2014-08-13 17:38:03.001473 7fe9bae887a0 20 read_log 192834'13129 (0'0) promote 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by osd.331.0:42283 2014-08-11 15:27:08.401680 2014-08-13 17:38:03.001485 7fe9bae887a0 20 read_log 192834'13130 (192834'13129) modify 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.1883634.0:13283 2014-08-11 15:27:08.393268 2014-08-13 17:38:03.001497 7fe9bae887a0 20 read_log 192834'13133 (0'0) promote a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.331.0:42287 2014-08-11 15:27:09.817956 2014-08-13 17:38:03.001508 7fe9bae887a0 20 read_log 192834'13134 (192834'13133) modify a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by client.1883634.0:13476 2014-08-11 15:27:09.809735 2014-08-13 17:38:03.001520 7fe9bae887a0 20 read_log 192834'13157 (0'0) promote 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by osd.331.0:42369 2014-08-11 15:27:15.570848 2014-08-13 17:38:03.001534 7fe9bae887a0 20 read_log 192834'13158 (192834'13157) modify 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.1883634.0:14830 2014-08-11 15:27:15.561042 2014-08-13 17:38:03.001546 7fe9bae887a0 20 read_log 192834'13189 (0'0) promote c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.331.0:42546 2014-08-11 15:27:33.108829 2014-08-13 17:38:03.001558 7fe9bae887a0 20 read_log 192834'13190 (192834'13189) modify c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by client.1883634.0:17201 2014-08-11 15:27:33.098580 2014-08-13 17:38:03.001570 7fe9bae887a0 20 read_log 192834'13201 (0'0) promote 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by osd.331.0:42627 2014-08-11 15:27:41.581485 2014-08-13 17:38:03.001582 7fe9bae887a0 20 read_log 192834'13202 (192834'13201) modify 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.1883634.0:18142 2014-08-11 15:27:41.570677 2014-08-13 17:38:03.001594 7fe9bae887a0 20 read_log 192834'13205 (0'0) promote cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by osd.331.0:42666 2014-08-11 15:27:44.506514 2014-08-13 17:38:03.001606 7fe9bae887a0 20 read_log 192834'13206 (192834'13205) modify cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.1883634.0:18441 2014-08-11 15:27:44.484629 2014-08-13 17:38:03.001618 7fe9bae887a0 20 read_log 192834'13221 (0'0) promote c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.331.0:42748 2014-08-11 15:27:53.848544 2014-08-13 17:38:03.001630 7fe9bae887a0 20 read_log 192834'13222 (192834'13221) modify c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by client.1883634.0:19696 2014-08-11 15:27:53.838623 2014-08-13 17:38:03.001643 7fe9bae887a0 20 read_log 192834'13235 (0'0) promote 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.331.0:42840 2014-08-11 15:28:03.305542 2014-08-13 17:38:03.001655 7fe9bae887a0 20 read_log 192834'13236 (192834'13235) modify 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by client.1883634.0:21194 2014-08-11 15:28:03.295126 2014-08-13 17:38:03.001667 7fe9bae887a0 20 read_log 192834'13247 (0'0) promote cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by osd.331.0:42913 2014-08-11 15:28:06.166524 2014-08-13 17:38:03.001679 7fe9bae887a0 20 read_log 192834'13248 (192834'13247) modify cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.1883634.0:22001 2014-08-11 15:28:06.156066 2014-08-13 17:38:03.001716 7fe9bae887a0 20 read_log 192834'13278 (192834'12916) modify 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.1883634.0:23913 2014-08-11 15:28:18.075392 2014-08-13 17:38:03.001729 7fe9bae887a0 20 read_log 192834'13280 (192834'12920) modify 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.1883634.0:24543 2014-08-11 15:28:19.216910 2014-08-13 17:38:03.001741 7fe9bae887a0 20 read_log 192834'13282 (192834'12924) modify 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.1883634.0:24663 2014-08-11 15:28:19.338944 2014-08-13 17:38:03.001753 7fe9bae887a0 20 read_log 192834'13290 (192834'12940) modify 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by client.1883634.0:25382 2014-08-11 15:28:21.026750 2014-08-13 17:38:03.001769 7fe9bae887a0 20 read_log 192834'13308 (192834'12976) modify 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.1883634.0:28162 2014-08-11 15:28:25.456696 2014-08-13 17:38:03.001781 7fe9bae887a0 20 read_log 192834'13313 (192834'12986) modify fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by client.1883634.0:28607 2014-08-11 15:28:26.029388 2014-08-13 17:38:03.001795 7fe9bae887a0 20 read_log 192834'13317 (192834'12994) modify 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.1883634.0:28785 2014-08-11 15:28:26.287810 2014-08-13 17:38:03.001807 7fe9bae887a0 20 read_log 192834'13321 (192834'13002) modify 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.1883634.0:29277 2014-08-11 15:28:26.922283 2014-08-13 17:38:03.001819 7fe9bae887a0 20 read_log 192834'13323 (192834'13006) modify 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.1883634.0:29464 2014-08-11 15:28:27.316442 2014-08-13 17:38:03.001831 7fe9bae887a0 20 read_log 192834'13327 (192834'13014) modify 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by client.1883634.0:30070 2014-08-11 15:28:28.055881 2014-08-13 17:38:03.001845 7fe9bae887a0 20 read_log 192834'13328 (192834'13016) modify baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.1883634.0:30112 2014-08-11 15:28:28.118618 2014-08-13 17:38:03.001857 7fe9bae887a0 20 read_log 192834'13332 (192834'13024) modify 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by client.1883634.0:30624 2014-08-11 15:28:28.995063 2014-08-13 17:38:03.001886 7fe9bae887a0 20 read_log 192834'13345 (192834'13050) modify 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.1883634.0:31951 2014-08-11 15:28:31.162872 2014-08-13 17:38:03.001899 7fe9bae887a0 20 read_log 192834'13352 (192834'13064) modify 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.1883634.0:32827 2014-08-11 15:28:32.375594 2014-08-13 17:38:03.001912 7fe9bae887a0 20 read_log 192834'13354 (192834'13068) modify e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.1883634.0:33084 2014-08-11 15:28:32.760614 2014-08-13 17:38:03.001924 7fe9bae887a0 20 read_log 192834'13360 (192834'13080) modify 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by client.1883634.0:33704 2014-08-11 15:28:33.685374 2014-08-13 17:38:03.001936 7fe9bae887a0 20 read_log 192834'13364 (192834'13084) modify a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by client.1883634.0:34372 2014-08-11 15:28:34.603383 2014-08-13 17:38:03.001948 7fe9bae887a0 20 read_log 192834'13375 (192834'13106) modify 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by client.1883634.0:35240 2014-08-11 15:28:36.316157 2014-08-13 17:38:03.001960 7fe9bae887a0 20 read_log 192834'13378 (192834'13112) modify 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by client.1883634.0:35481 2014-08-11 15:28:36.709793 2014-08-13 17:38:03.001972 7fe9bae887a0 20 read_log 192834'13382 (192834'13120) modify 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.1883634.0:35795 2014-08-11 15:28:37.134655 2014-08-13 17:38:03.001992 7fe9bae887a0 20 read_log 192834'13385 (192834'13126) modify eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by client.1883634.0:36350 2014-08-11 15:28:37.916584 2014-08-13 17:38:03.002005 7fe9bae887a0 20 read_log 192834'13387 (192834'13130) modify 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.1883634.0:36607 2014-08-11 15:28:38.210715 2014-08-13 17:38:03.002017 7fe9bae887a0 20 read_log 192834'13389 (192834'13134) modify a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by client.1883634.0:36800 2014-08-11 15:28:38.423530 2014-08-13 17:38:03.002029 7fe9bae887a0 20 read_log 192834'13401 (192834'13158) modify 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.1883634.0:38154 2014-08-11 15:28:40.292568 2014-08-13 17:38:03.002041 7fe9bae887a0 20 read_log 192834'13417 (192834'13190) modify c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by client.1883634.0:40525 2014-08-11 15:28:43.874140 2014-08-13 17:38:03.002053 7fe9bae887a0 20 read_log 192834'13423 (192834'13202) modify 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.1883634.0:41466 2014-08-11 15:28:45.155021 2014-08-13 17:38:03.002064 7fe9bae887a0 20 read_log 192834'13425 (192834'13206) modify cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.1883634.0:41765 2014-08-11 15:28:45.474134 2014-08-13 17:38:03.002076 7fe9bae887a0 20 read_log 192834'13433 (192834'13222) modify c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by client.1883634.0:43020 2014-08-11 15:28:46.433949 2014-08-13 17:38:03.002089 7fe9bae887a0 20 read_log 192834'13440 (192834'13236) modify 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by client.1883634.0:44518 2014-08-11 15:28:46.879826 2014-08-13 17:38:03.002100 7fe9bae887a0 20 read_log 192834'13446 (192834'13248) modify cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.1883634.0:45325 2014-08-11 15:28:47.241326 2014-08-13 17:38:03.002113 7fe9bae887a0 20 read_log 192888'13460 (0'0) promote 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.287.0:16787 2014-08-11 16:44:55.255982 2014-08-13 17:38:03.002125 7fe9bae887a0 20 read_log 192888'13461 (192888'13460) modify 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by client.1315259.0:79 2014-08-11 16:44:55.243805 2014-08-13 17:38:03.002137 7fe9bae887a0 20 read_log 192888'13462 (0'0) promote bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.287.0:16792 2014-08-11 16:44:55.605389 2014-08-13 17:38:03.002149 7fe9bae887a0 20 read_log 192888'13463 (192888'13462) modify bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by client.1315259.0:169 2014-08-11 16:44:55.599610 2014-08-13 17:38:03.002166 7fe9bae887a0 20 read_log 192888'13470 (0'0) promote 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.287.0:16820 2014-08-11 16:45:06.117232 2014-08-13 17:38:03.002179 7fe9bae887a0 20 read_log 192888'13471 (192888'13470) modify 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by client.1315259.0:2874 2014-08-11 16:45:06.110086 2014-08-13 17:38:03.002191 7fe9bae887a0 20 read_log 192888'13474 (0'0) promote 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by osd.287.0:16844 2014-08-11 16:45:12.132645 2014-08-13 17:38:03.002228 7fe9bae887a0 20 read_log 192888'13475 (192888'13474) modify 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.1315259.0:4460 2014-08-11 16:45:12.125046 2014-08-13 17:38:03.002241 7fe9bae887a0 20 read_log 192888'13484 (0'0) promote 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.287.0:16874 2014-08-11 16:45:19.100200 2014-08-13 17:38:03.002263 7fe9bae887a0 20 read_log 192888'13485 (192888'13484) modify 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by client.1315259.0:6365 2014-08-11 16:45:19.093714 2014-08-13 17:38:03.002276 7fe9bae887a0 20 read_log 192888'13486 (0'0) promote e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by osd.287.0:16876 2014-08-11 16:45:19.213470 2014-08-13 17:38:03.002288 7fe9bae887a0 20 read_log 192888'13487 (192888'13486) modify e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.1315259.0:6396 2014-08-11 16:45:19.206766 2014-08-13 17:38:03.002300 7fe9bae887a0 20 read_log 192888'13490 (0'0) promote 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by osd.287.0:16898 2014-08-11 16:45:24.233967 2014-08-13 17:38:03.002312 7fe9bae887a0 20 read_log 192888'13491 (192888'13490) modify 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.1315259.0:7779 2014-08-11 16:45:24.225363 2014-08-13 17:38:03.002324 7fe9bae887a0 20 read_log 192888'13498 (0'0) promote 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.287.0:16914 2014-08-11 16:45:26.987971 2014-08-13 17:38:03.002336 7fe9bae887a0 20 read_log 192888'13499 (192888'13498) modify 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by client.1315259.0:8508 2014-08-11 16:45:26.981030 2014-08-13 17:38:03.002349 7fe9bae887a0 20 read_log 192888'13500 (0'0) promote 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by osd.287.0:16918 2014-08-11 16:45:27.720332 2014-08-13 17:38:03.002360 7fe9bae887a0 20 read_log 192888'13501 (192888'13500) modify 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.1315259.0:8707 2014-08-11 16:45:27.710262 2014-08-13 17:38:03.002372 7fe9bae887a0 20 read_log 192888'13502 (0'0) promote 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by osd.287.0:16920 2014-08-11 16:45:28.702369 2014-08-13 17:38:03.002384 7fe9bae887a0 20 read_log 192888'13503 (192888'13502) modify 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.1315259.0:8954 2014-08-11 16:45:28.695832 2014-08-13 17:38:03.002396 7fe9bae887a0 20 read_log 192888'13504 (0'0) promote e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by osd.287.0:16928 2014-08-11 16:45:29.995690 2014-08-13 17:38:03.002408 7fe9bae887a0 20 read_log 192888'13505 (192888'13504) modify e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.1315259.0:9315 2014-08-11 16:45:29.988327 2014-08-13 17:38:03.002420 7fe9bae887a0 20 read_log 192888'13506 (0'0) promote 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by osd.287.0:16946 2014-08-11 16:45:32.911130 2014-08-13 17:38:03.002432 7fe9bae887a0 20 read_log 192888'13507 (192888'13506) modify 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.1315259.0:10070 2014-08-11 16:45:32.903677 2014-08-13 17:38:03.002444 7fe9bae887a0 20 read_log 192888'13512 (0'0) promote 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by osd.287.0:16960 2014-08-11 16:45:36.591542 2014-08-13 17:38:03.002456 7fe9bae887a0 20 read_log 192888'13513 (192888'13512) modify 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.1315259.0:10994 2014-08-11 16:45:36.584923 2014-08-13 17:38:03.002468 7fe9bae887a0 20 read_log 192888'13516 (0'0) promote d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.287.0:16970 2014-08-11 16:45:41.043616 2014-08-13 17:38:03.002480 7fe9bae887a0 20 read_log 192888'13517 (192888'13516) modify d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by client.1315259.0:12092 2014-08-11 16:45:41.036955 2014-08-13 17:38:03.002492 7fe9bae887a0 20 read_log 192888'13518 (0'0) promote 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.287.0:16976 2014-08-11 16:45:42.579874 2014-08-13 17:38:03.002511 7fe9bae887a0 20 read_log 192888'13519 (192888'13518) modify 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by client.1315259.0:12452 2014-08-11 16:45:42.573518 2014-08-13 17:38:03.002523 7fe9bae887a0 20 read_log 192888'13522 (0'0) promote dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by osd.287.0:16988 2014-08-11 16:45:44.314036 2014-08-13 17:38:03.002535 7fe9bae887a0 20 read_log 192888'13523 (192888'13522) modify dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.1315259.0:12873 2014-08-11 16:45:44.307661 2014-08-13 17:38:03.002547 7fe9bae887a0 20 read_log 192888'13532 (0'0) promote 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by osd.287.0:17018 2014-08-11 16:45:56.409708 2014-08-13 17:38:03.002559 7fe9bae887a0 20 read_log 192888'13533 (192888'13532) modify 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.1315259.0:15995 2014-08-11 16:45:56.402496 2014-08-13 17:38:03.002572 7fe9bae887a0 20 read_log 192888'13538 (0'0) promote 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by osd.287.0:17028 2014-08-11 16:45:57.217483 2014-08-13 17:38:03.002585 7fe9bae887a0 20 read_log 192888'13539 (192888'13538) modify 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.1315259.0:16209 2014-08-11 16:45:57.209951 2014-08-13 17:38:03.002597 7fe9bae887a0 20 read_log 192888'13544 (0'0) promote 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by osd.287.0:17042 2014-08-11 16:45:59.211054 2014-08-13 17:38:03.002609 7fe9bae887a0 20 read_log 192888'13545 (192888'13544) modify 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.1315259.0:16711 2014-08-11 16:45:59.201578 2014-08-13 17:38:03.002621 7fe9bae887a0 20 read_log 192888'13546 (0'0) promote 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.287.0:17044 2014-08-11 16:45:59.825739 2014-08-13 17:38:03.002632 7fe9bae887a0 20 read_log 192888'13547 (192888'13546) modify 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by client.1315259.0:16876 2014-08-11 16:45:59.818541 2014-08-13 17:38:03.002644 7fe9bae887a0 20 read_log 192888'13548 (0'0) promote 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.287.0:17048 2014-08-11 16:46:00.911426 2014-08-13 17:38:03.002656 7fe9bae887a0 20 read_log 192888'13549 (192888'13548) modify 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by client.1315259.0:17143 2014-08-11 16:46:00.904447 2014-08-13 17:38:03.002668 7fe9bae887a0 20 read_log 192888'13550 (0'0) promote e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.287.0:17052 2014-08-11 16:46:01.833871 2014-08-13 17:38:03.002680 7fe9bae887a0 20 read_log 192888'13551 (192888'13550) modify e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by client.1315259.0:17376 2014-08-11 16:46:01.826683 2014-08-13 17:38:03.002692 7fe9bae887a0 20 read_log 192888'13552 (0'0) promote 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by osd.287.0:17072 2014-08-11 16:46:06.224722 2014-08-13 17:38:03.002704 7fe9bae887a0 20 read_log 192888'13553 (192888'13552) modify 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.1315259.0:18484 2014-08-11 16:46:06.217348 2014-08-13 17:38:03.002716 7fe9bae887a0 20 read_log 192888'13554 (0'0) promote a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by osd.287.0:17076 2014-08-11 16:46:07.397436 2014-08-13 17:38:03.002727 7fe9bae887a0 20 read_log 192888'13555 (192888'13554) modify a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.1315259.0:18796 2014-08-11 16:46:07.390571 2014-08-13 17:38:03.002739 7fe9bae887a0 20 read_log 192888'13556 (0'0) promote 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by osd.287.0:17078 2014-08-11 16:46:07.413033 2014-08-13 17:38:03.002758 7fe9bae887a0 20 read_log 192888'13557 (192888'13556) modify 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.1315259.0:18800 2014-08-11 16:46:07.405515 2014-08-13 17:38:03.002771 7fe9bae887a0 20 read_log 192888'13560 (0'0) promote 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by osd.287.0:17116 2014-08-11 16:46:19.997056 2014-08-13 17:38:03.002783 7fe9bae887a0 20 read_log 192888'13561 (192888'13560) modify 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.1315259.0:22081 2014-08-11 16:46:19.988290 2014-08-13 17:38:03.002795 7fe9bae887a0 20 read_log 192888'13566 (0'0) promote 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by osd.287.0:17128 2014-08-11 16:46:22.877814 2014-08-13 17:38:03.002807 7fe9bae887a0 20 read_log 192888'13567 (192888'13566) modify 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.1315259.0:22822 2014-08-11 16:46:22.869879 2014-08-13 17:38:03.002819 7fe9bae887a0 20 read_log 192888'13570 (0'0) promote 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by osd.287.0:17132 2014-08-11 16:46:23.310758 2014-08-13 17:38:03.002830 7fe9bae887a0 20 read_log 192888'13571 (192888'13570) modify 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.1315259.0:22922 2014-08-11 16:46:23.301617 2014-08-13 17:38:03.002842 7fe9bae887a0 20 read_log 192888'13572 (0'0) promote b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by osd.287.0:17136 2014-08-11 16:46:24.961749 2014-08-13 17:38:03.002854 7fe9bae887a0 20 read_log 192888'13573 (192888'13572) modify b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.1315259.0:23332 2014-08-11 16:46:24.954930 2014-08-13 17:38:03.002866 7fe9bae887a0 20 read_log 192888'13578 (0'0) promote 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.287.0:17172 2014-08-11 16:46:31.831699 2014-08-13 17:38:03.002878 7fe9bae887a0 20 read_log 192888'13579 (192888'13578) modify 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by client.1315259.0:25087 2014-08-11 16:46:31.824579 2014-08-13 17:38:03.002890 7fe9bae887a0 20 read_log 192888'13590 (0'0) promote 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.287.0:17194 2014-08-11 16:46:36.944757 2014-08-13 17:38:03.002902 7fe9bae887a0 20 read_log 192888'13591 (192888'13590) modify 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by client.1315259.0:26287 2014-08-11 16:46:36.938580 2014-08-13 17:38:03.002913 7fe9bae887a0 20 read_log 192888'13592 (0'0) promote 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by osd.287.0:17206 2014-08-11 16:46:37.939135 2014-08-13 17:38:03.002925 7fe9bae887a0 20 read_log 192888'13593 (192888'13592) modify 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.1315259.0:26512 2014-08-11 16:46:37.931763 2014-08-13 17:38:03.002937 7fe9bae887a0 20 read_log 192888'13594 (0'0) promote 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by osd.287.0:17232 2014-08-11 16:46:46.958687 2014-08-13 17:38:03.002949 7fe9bae887a0 20 read_log 192888'13595 (192888'13594) modify 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.1315259.0:28721 2014-08-11 16:46:46.951706 2014-08-13 17:38:03.002961 7fe9bae887a0 20 read_log 192888'13598 (0'0) promote 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by osd.287.0:17238 2014-08-11 16:46:48.291207 2014-08-13 17:38:03.002973 7fe9bae887a0 20 read_log 192888'13599 (192888'13598) modify 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.1315259.0:29071 2014-08-11 16:46:48.284221 2014-08-13 17:38:03.002984 7fe9bae887a0 20 read_log 192888'13600 (192888'13461) modify 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by client.1315259.0:30806 2014-08-11 16:46:55.896294 2014-08-13 17:38:03.003006 7fe9bae887a0 20 read_log 192888'13601 (192888'13463) modify bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by client.1315259.0:30896 2014-08-11 16:46:55.929479 2014-08-13 17:38:03.003018 7fe9bae887a0 20 read_log 192888'13605 (192888'13471) modify 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by client.1315259.0:33601 2014-08-11 16:46:56.677711 2014-08-13 17:38:03.003031 7fe9bae887a0 20 read_log 192888'13607 (192888'13475) modify 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.1315259.0:35187 2014-08-11 16:46:57.095456 2014-08-13 17:38:03.003043 7fe9bae887a0 20 read_log 192888'13612 (192888'13485) modify 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by client.1315259.0:37092 2014-08-11 16:46:57.591969 2014-08-13 17:38:03.003055 7fe9bae887a0 20 read_log 192888'13613 (192888'13487) modify e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.1315259.0:37123 2014-08-11 16:46:57.601056 2014-08-13 17:38:03.003067 7fe9bae887a0 20 read_log 192888'13615 (192888'13491) modify 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.1315259.0:38506 2014-08-11 16:46:57.968956 2014-08-13 17:38:03.003079 7fe9bae887a0 20 read_log 192888'13619 (192888'13499) modify 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by client.1315259.0:39235 2014-08-11 16:46:58.159067 2014-08-13 17:38:03.003091 7fe9bae887a0 20 read_log 192888'13620 (192888'13501) modify 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.1315259.0:39434 2014-08-11 16:46:58.210918 2014-08-13 17:38:03.003103 7fe9bae887a0 20 read_log 192888'13621 (192888'13503) modify 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.1315259.0:39681 2014-08-11 16:46:58.276248 2014-08-13 17:38:03.003114 7fe9bae887a0 20 read_log 192888'13622 (192888'13505) modify e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.1315259.0:40042 2014-08-11 16:46:58.372316 2014-08-13 17:38:03.003126 7fe9bae887a0 20 read_log 192888'13623 (192888'13507) modify 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.1315259.0:40797 2014-08-11 16:46:58.583295 2014-08-13 17:38:03.003138 7fe9bae887a0 20 read_log 192888'13626 (192888'13513) modify 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.1315259.0:41721 2014-08-11 16:46:58.819623 2014-08-13 17:38:03.003150 7fe9bae887a0 20 read_log 192888'13628 (192888'13517) modify d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by client.1315259.0:42819 2014-08-11 16:46:59.114997 2014-08-13 17:38:03.003168 7fe9bae887a0 20 read_log 192888'13629 (192888'13519) modify 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by client.1315259.0:43179 2014-08-11 16:46:59.215917 2014-08-13 17:38:03.003181 7fe9bae887a0 20 read_log 192888'13631 (192888'13523) modify dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.1315259.0:43600 2014-08-11 16:46:59.336123 2014-08-13 17:38:03.003194 7fe9bae887a0 20 read_log 192888'13636 (192888'13533) modify 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.1315259.0:46722 2014-08-11 16:47:00.239881 2014-08-13 17:38:03.003206 7fe9bae887a0 20 read_log 192888'13639 (192888'13539) modify 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.1315259.0:46936 2014-08-11 16:47:00.300911 2014-08-13 17:38:03.003218 7fe9bae887a0 20 read_log 192888'13642 (192888'13545) modify 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.1315259.0:47438 2014-08-11 16:47:00.455528 2014-08-13 17:38:03.003230 7fe9bae887a0 20 read_log 192888'13643 (192888'13547) modify 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by client.1315259.0:47603 2014-08-11 16:47:00.506762 2014-08-13 17:38:03.003242 7fe9bae887a0 20 read_log 192888'13644 (192888'13549) modify 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by client.1315259.0:47870 2014-08-11 16:47:00.593206 2014-08-13 17:38:03.003262 7fe9bae887a0 20 read_log 192888'13645 (192888'13551) modify e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by client.1315259.0:48103 2014-08-11 16:47:00.671352 2014-08-13 17:38:03.003275 7fe9bae887a0 20 read_log 192888'13646 (192888'13553) modify 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.1315259.0:49211 2014-08-11 16:47:00.991196 2014-08-13 17:38:03.003287 7fe9bae887a0 20 read_log 192888'13647 (192888'13555) modify a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.1315259.0:49523 2014-08-11 16:47:01.088888 2014-08-13 17:38:03.003299 7fe9bae887a0 20 read_log 192888'13648 (192888'13557) modify 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.1315259.0:49527 2014-08-11 16:47:01.091513 2014-08-13 17:38:03.003311 7fe9bae887a0 20 read_log 192888'13650 (192888'13561) modify 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.1315259.0:52808 2014-08-11 16:47:02.075189 2014-08-13 17:38:03.003323 7fe9bae887a0 20 read_log 192888'13653 (192888'13567) modify 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.1315259.0:53549 2014-08-11 16:47:02.288464 2014-08-13 17:38:03.003335 7fe9bae887a0 20 read_log 192888'13655 (192888'13571) modify 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.1315259.0:53649 2014-08-11 16:47:02.317430 2014-08-13 17:38:03.003347 7fe9bae887a0 20 read_log 192888'13656 (192888'13573) modify b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.1315259.0:54059 2014-08-11 16:47:02.447962 2014-08-13 17:38:03.003358 7fe9bae887a0 20 read_log 192888'13659 (192888'13579) modify 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by client.1315259.0:55814 2014-08-11 16:47:03.023592 2014-08-13 17:38:03.003370 7fe9bae887a0 20 read_log 192888'13665 (192888'13591) modify 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by client.1315259.0:57014 2014-08-11 16:47:03.385497 2014-08-13 17:38:03.003382 7fe9bae887a0 20 read_log 192888'13666 (192888'13593) modify 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.1315259.0:57239 2014-08-11 16:47:03.452406 2014-08-13 17:38:03.003413 7fe9bae887a0 20 read_log 192888'13667 (192888'13595) modify 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.1315259.0:59448 2014-08-11 16:47:04.122497 2014-08-13 17:38:03.003425 7fe9bae887a0 20 read_log 192888'13669 (192888'13599) modify 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.1315259.0:59798 2014-08-11 16:47:04.243853 2014-08-13 17:38:03.003438 7fe9bae887a0 20 read_log 192888'13674 (0'0) promote b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by osd.287.0:17548 2014-08-11 17:05:22.568616 2014-08-13 17:38:03.003449 7fe9bae887a0 20 read_log 192888'13675 (192888'13674) modify b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.942868.0:1836 2014-08-11 17:05:22.561802 2014-08-13 17:38:03.003461 7fe9bae887a0 20 read_log 192888'13676 (0'0) promote 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by osd.287.0:17554 2014-08-11 17:05:24.278921 2014-08-13 17:38:03.003473 7fe9bae887a0 20 read_log 192888'13677 (192888'13676) modify 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.942868.0:2254 2014-08-11 17:05:24.270738 2014-08-13 17:38:03.003485 7fe9bae887a0 20 read_log 192888'13680 (0'0) promote cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.287.0:17564 2014-08-11 17:05:25.637772 2014-08-13 17:38:03.003497 7fe9bae887a0 20 read_log 192888'13681 (192888'13680) modify cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by client.942868.0:2620 2014-08-11 17:05:25.629596 2014-08-13 17:38:03.003509 7fe9bae887a0 20 read_log 192888'13682 (0'0) promote af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by osd.287.0:17578 2014-08-11 17:05:29.567305 2014-08-13 17:38:03.003544 7fe9bae887a0 20 read_log 192888'13683 (192888'13682) modify af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.942868.0:3683 2014-08-11 17:05:29.559541 2014-08-13 17:38:03.003557 7fe9bae887a0 20 read_log 192888'13692 (0'0) promote b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by osd.287.0:17608 2014-08-11 17:05:35.909609 2014-08-13 17:38:03.003569 7fe9bae887a0 20 read_log 192888'13693 (192888'13692) modify b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.942868.0:5344 2014-08-11 17:05:35.902685 2014-08-13 17:38:03.003581 7fe9bae887a0 20 read_log 192888'13694 (0'0) promote 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.287.0:17620 2014-08-11 17:05:38.016853 2014-08-13 17:38:03.003593 7fe9bae887a0 20 read_log 192888'13695 (192888'13694) modify 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by client.942868.0:5917 2014-08-11 17:05:38.010395 2014-08-13 17:38:03.003606 7fe9bae887a0 20 read_log 192888'13700 (0'0) promote 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by osd.287.0:17672 2014-08-11 17:05:53.836993 2014-08-13 17:38:03.003618 7fe9bae887a0 20 read_log 192888'13701 (192888'13700) modify 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.942868.0:10175 2014-08-11 17:05:53.830808 2014-08-13 17:38:03.003630 7fe9bae887a0 20 read_log 192888'13704 (0'0) promote a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by osd.287.0:17680 2014-08-11 17:05:55.329593 2014-08-13 17:38:03.003642 7fe9bae887a0 20 read_log 192888'13705 (192888'13704) modify a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.942868.0:10590 2014-08-11 17:05:55.321917 2014-08-13 17:38:03.003654 7fe9bae887a0 20 read_log 192888'13706 (0'0) promote 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by osd.287.0:17692 2014-08-11 17:05:57.076833 2014-08-13 17:38:03.003666 7fe9bae887a0 20 read_log 192888'13707 (192888'13706) modify 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.942868.0:11078 2014-08-11 17:05:57.069576 2014-08-13 17:38:03.003678 7fe9bae887a0 20 read_log 192888'13714 (0'0) promote 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.287.0:17728 2014-08-11 17:06:07.697543 2014-08-13 17:38:03.003690 7fe9bae887a0 20 read_log 192888'13715 (192888'13714) modify 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by client.942868.0:13884 2014-08-11 17:06:07.691044 2014-08-13 17:38:03.003702 7fe9bae887a0 20 read_log 192888'13718 (0'0) promote 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by osd.287.0:17746 2014-08-11 17:06:11.508877 2014-08-13 17:38:03.003713 7fe9bae887a0 20 read_log 192888'13719 (192888'13718) modify 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.942868.0:14817 2014-08-11 17:06:11.502057 2014-08-13 17:38:03.003725 7fe9bae887a0 20 read_log 192888'13724 (0'0) promote 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.287.0:17784 2014-08-11 17:06:20.092309 2014-08-13 17:38:03.003737 7fe9bae887a0 20 read_log 192888'13725 (192888'13724) modify 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by client.942868.0:17008 2014-08-11 17:06:20.084301 2014-08-13 17:38:03.003749 7fe9bae887a0 20 read_log 192888'13726 (0'0) promote 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by osd.287.0:17796 2014-08-11 17:06:24.277319 2014-08-13 17:38:03.003761 7fe9bae887a0 20 read_log 192888'13727 (192888'13726) modify 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.942868.0:18118 2014-08-11 17:06:24.270459 2014-08-13 17:38:03.003773 7fe9bae887a0 20 read_log 192888'13730 (0'0) promote 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by osd.287.0:17806 2014-08-11 17:06:26.635647 2014-08-13 17:38:03.003794 7fe9bae887a0 20 read_log 192888'13731 (192888'13730) modify 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.942868.0:18744 2014-08-11 17:06:26.629014 2014-08-13 17:38:03.003807 7fe9bae887a0 20 read_log 192888'13738 (0'0) promote 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by osd.287.0:17832 2014-08-11 17:06:31.485260 2014-08-13 17:38:03.003819 7fe9bae887a0 20 read_log 192888'13739 (192888'13738) modify 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.942868.0:20032 2014-08-11 17:06:31.476726 2014-08-13 17:38:03.003831 7fe9bae887a0 20 read_log 192888'13740 (0'0) promote df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.287.0:17840 2014-08-11 17:06:32.217468 2014-08-13 17:38:03.003843 7fe9bae887a0 20 read_log 192888'13741 (192888'13740) modify df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by client.942868.0:20229 2014-08-11 17:06:32.208004 2014-08-13 17:38:03.003855 7fe9bae887a0 20 read_log 192888'13746 (0'0) promote c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.287.0:17870 2014-08-11 17:06:38.336781 2014-08-13 17:38:03.003866 7fe9bae887a0 20 read_log 192888'13747 (192888'13746) modify c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by client.942868.0:21828 2014-08-11 17:06:38.329830 2014-08-13 17:38:03.003878 7fe9bae887a0 20 read_log 192888'13750 (0'0) promote 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by osd.287.0:17882 2014-08-11 17:06:40.660047 2014-08-13 17:38:03.003890 7fe9bae887a0 20 read_log 192888'13751 (192888'13750) modify 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.942868.0:22423 2014-08-11 17:06:40.653422 2014-08-13 17:38:03.003902 7fe9bae887a0 20 read_log 192888'13752 (0'0) promote 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.287.0:17886 2014-08-11 17:06:45.222870 2014-08-13 17:38:03.003914 7fe9bae887a0 20 read_log 192888'13753 (192888'13752) modify 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by client.942868.0:23597 2014-08-11 17:06:45.215818 2014-08-13 17:38:03.003926 7fe9bae887a0 20 read_log 192888'13756 (0'0) promote 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by osd.287.0:17894 2014-08-11 17:06:46.430345 2014-08-13 17:38:03.003937 7fe9bae887a0 20 read_log 192888'13757 (192888'13756) modify 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.942868.0:23924 2014-08-11 17:06:46.422996 2014-08-13 17:38:03.003950 7fe9bae887a0 20 read_log 192888'13758 (0'0) promote d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by osd.287.0:17902 2014-08-11 17:06:48.492236 2014-08-13 17:38:03.003962 7fe9bae887a0 20 read_log 192888'13759 (192888'13758) modify d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.942868.0:24480 2014-08-11 17:06:48.485263 2014-08-13 17:38:03.003973 7fe9bae887a0 20 read_log 192888'13760 (0'0) promote 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.287.0:17906 2014-08-11 17:06:49.812696 2014-08-13 17:38:03.003985 7fe9bae887a0 20 read_log 192888'13761 (192888'13760) modify 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by client.942868.0:24839 2014-08-11 17:06:49.806992 2014-08-13 17:38:03.003997 7fe9bae887a0 20 read_log 192888'13766 (0'0) promote 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by osd.287.0:17936 2014-08-11 17:06:56.588130 2014-08-13 17:38:03.004009 7fe9bae887a0 20 read_log 192888'13767 (192888'13766) modify 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.942868.0:26572 2014-08-11 17:06:56.581988 2014-08-13 17:38:03.004021 7fe9bae887a0 20 read_log 192888'13768 (0'0) promote 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by osd.287.0:17942 2014-08-11 17:06:56.811537 2014-08-13 17:38:03.004039 7fe9bae887a0 20 read_log 192888'13769 (192888'13768) modify 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.942868.0:26633 2014-08-11 17:06:56.805013 2014-08-13 17:38:03.004052 7fe9bae887a0 20 read_log 192888'13770 (0'0) promote cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.287.0:17962 2014-08-11 17:07:00.866695 2014-08-13 17:38:03.004064 7fe9bae887a0 20 read_log 192888'13771 (192888'13770) modify cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by client.942868.0:27696 2014-08-11 17:07:00.858713 2014-08-13 17:38:03.004076 7fe9bae887a0 20 read_log 192888'13772 (0'0) promote a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.287.0:17970 2014-08-11 17:07:04.594589 2014-08-13 17:38:03.004088 7fe9bae887a0 20 read_log 192888'13773 (192888'13772) modify a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by client.942868.0:28708 2014-08-11 17:07:04.587972 2014-08-13 17:38:03.004100 7fe9bae887a0 20 read_log 192888'13774 (0'0) promote 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by osd.287.0:17988 2014-08-11 17:07:08.837774 2014-08-13 17:38:03.004112 7fe9bae887a0 20 read_log 192888'13775 (192888'13774) modify 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.942868.0:29756 2014-08-11 17:07:08.830178 2014-08-13 17:38:03.004124 7fe9bae887a0 20 read_log 192888'13776 (0'0) promote 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.287.0:17994 2014-08-11 17:07:09.529258 2014-08-13 17:38:03.004136 7fe9bae887a0 20 read_log 192888'13777 (192888'13776) modify 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by client.942868.0:29932 2014-08-11 17:07:09.521058 2014-08-13 17:38:03.004148 7fe9bae887a0 20 read_log 192888'13778 (0'0) promote fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by osd.287.0:18000 2014-08-11 17:07:10.127710 2014-08-13 17:38:03.004165 7fe9bae887a0 20 read_log 192888'13779 (192888'13778) modify fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.942868.0:30081 2014-08-11 17:07:10.121072 2014-08-13 17:38:03.004178 7fe9bae887a0 20 read_log 192888'13780 (0'0) promote 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by osd.287.0:18022 2014-08-11 17:07:12.782187 2014-08-13 17:38:03.004190 7fe9bae887a0 20 read_log 192888'13781 (192888'13780) modify 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.942868.0:30740 2014-08-11 17:07:12.776189 2014-08-13 17:38:03.004202 7fe9bae887a0 20 read_log 192888'13782 (0'0) promote f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.287.0:18024 2014-08-11 17:07:12.854944 2014-08-13 17:38:03.004213 7fe9bae887a0 20 read_log 192888'13783 (192888'13782) modify f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by client.942868.0:30755 2014-08-11 17:07:12.847785 2014-08-13 17:38:03.004225 7fe9bae887a0 20 read_log 192888'13789 (192888'13675) modify b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.942868.0:33102 2014-08-11 17:07:16.426581 2014-08-13 17:38:03.004237 7fe9bae887a0 20 read_log 192888'13790 (192888'13677) modify 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.942868.0:33520 2014-08-11 17:07:16.538403 2014-08-13 17:38:03.004249 7fe9bae887a0 20 read_log 192888'13792 (192888'13681) modify cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by client.942868.0:33886 2014-08-11 17:07:16.638109 2014-08-13 17:38:03.004273 7fe9bae887a0 20 read_log 192888'13793 (192888'13683) modify af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.942868.0:34949 2014-08-11 17:07:16.940638 2014-08-13 17:38:03.004286 7fe9bae887a0 20 read_log 192888'13798 (192888'13693) modify b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.942868.0:36610 2014-08-11 17:07:17.388544 2014-08-13 17:38:03.004306 7fe9bae887a0 20 read_log 192888'13799 (192888'13695) modify 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by client.942868.0:37183 2014-08-11 17:07:17.537203 2014-08-13 17:38:03.004319 7fe9bae887a0 20 read_log 192888'13802 (192888'13701) modify 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.942868.0:41441 2014-08-11 17:07:18.729336 2014-08-13 17:38:03.004331 7fe9bae887a0 20 read_log 192888'13804 (192888'13705) modify a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.942868.0:41856 2014-08-11 17:07:18.841185 2014-08-13 17:38:03.004343 7fe9bae887a0 20 read_log 192888'13805 (192888'13707) modify 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.942868.0:42344 2014-08-11 17:07:18.974047 2014-08-13 17:38:03.004355 7fe9bae887a0 20 read_log 192888'13809 (192888'13715) modify 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by client.942868.0:45150 2014-08-11 17:07:19.759466 2014-08-13 17:38:03.004367 7fe9bae887a0 20 read_log 192888'13811 (192888'13719) modify 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.942868.0:46083 2014-08-11 17:07:20.015929 2014-08-13 17:38:03.004379 7fe9bae887a0 20 read_log 192888'13814 (192888'13725) modify 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by client.942868.0:48274 2014-08-11 17:07:20.672979 2014-08-13 17:38:03.004391 7fe9bae887a0 20 read_log 192888'13815 (192888'13727) modify 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.942868.0:49384 2014-08-11 17:07:20.996007 2014-08-13 17:38:03.004403 7fe9bae887a0 20 read_log 192888'13817 (192888'13731) modify 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.942868.0:50010 2014-08-11 17:07:21.188612 2014-08-13 17:38:03.004415 7fe9bae887a0 20 read_log 192888'13821 (192888'13739) modify 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.942868.0:51298 2014-08-11 17:07:21.600410 2014-08-13 17:38:03.004427 7fe9bae887a0 20 read_log 192888'13822 (192888'13741) modify df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by client.942868.0:51495 2014-08-11 17:07:21.663258 2014-08-13 17:38:03.004439 7fe9bae887a0 20 read_log 192888'13825 (192888'13747) modify c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by client.942868.0:53094 2014-08-11 17:07:22.145522 2014-08-13 17:38:03.004451 7fe9bae887a0 20 read_log 192888'13827 (192888'13751) modify 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.942868.0:53689 2014-08-11 17:07:22.343385 2014-08-13 17:38:03.004463 7fe9bae887a0 20 read_log 192888'13828 (192888'13753) modify 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by client.942868.0:54863 2014-08-11 17:07:22.677572 2014-08-13 17:38:03.004475 7fe9bae887a0 20 read_log 192888'13830 (192888'13757) modify 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.942868.0:55190 2014-08-11 17:07:22.769185 2014-08-13 17:38:03.004487 7fe9bae887a0 20 read_log 192888'13831 (192888'13759) modify d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.942868.0:55746 2014-08-11 17:07:22.941318 2014-08-13 17:38:03.004499 7fe9bae887a0 20 read_log 192888'13832 (192888'13761) modify 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by client.942868.0:56105 2014-08-11 17:07:23.053698 2014-08-13 17:38:03.004511 7fe9bae887a0 20 read_log 192888'13835 (192888'13767) modify 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.942868.0:57838 2014-08-11 17:07:23.570648 2014-08-13 17:38:03.004523 7fe9bae887a0 20 read_log 192888'13836 (192888'13769) modify 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.942868.0:57899 2014-08-11 17:07:23.586438 2014-08-13 17:38:03.004535 7fe9bae887a0 20 read_log 192888'13837 (192888'13771) modify cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by client.942868.0:58962 2014-08-11 17:07:23.904060 2014-08-13 17:38:03.004557 7fe9bae887a0 20 read_log 192888'13838 (192888'13773) modify a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by client.942868.0:59974 2014-08-11 17:07:24.206669 2014-08-13 17:38:03.004569 7fe9bae887a0 20 read_log 192888'13839 (192888'13775) modify 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.942868.0:61022 2014-08-11 17:07:24.516104 2014-08-13 17:38:03.004581 7fe9bae887a0 20 read_log 192888'13840 (192888'13777) modify 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by client.942868.0:61198 2014-08-11 17:07:24.568633 2014-08-13 17:38:03.004593 7fe9bae887a0 20 read_log 192888'13841 (192888'13779) modify fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.942868.0:61347 2014-08-11 17:07:24.612898 2014-08-13 17:38:03.004622 7fe9bae887a0 20 read_log 192888'13842 (192888'13781) modify 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.942868.0:62006 2014-08-11 17:07:24.819529 2014-08-13 17:38:03.004634 7fe9bae887a0 20 read_log 192888'13843 (192888'13783) modify f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by client.942868.0:62021 2014-08-11 17:07:24.823999 2014-08-13 17:38:03.004647 7fe9bae887a0 20 read_log 193305'13846 (0'0) promote b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1 2014-08-11 18:42:32.814693 2014-08-13 17:38:03.004659 7fe9bae887a0 20 read_log 193305'13847 (193305'13846) modify b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by client.951337.0:2 2014-08-11 18:42:32.802553 2014-08-13 17:38:03.004671 7fe9bae887a0 20 read_log 193305'13848 (0'0) promote 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by osd.17.0:8 2014-08-11 18:42:34.934816 2014-08-13 17:38:03.004683 7fe9bae887a0 20 read_log 193305'13849 (193305'13848) modify 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.951337.0:474 2014-08-11 18:42:34.927729 2014-08-13 17:38:03.004695 7fe9bae887a0 20 read_log 193305'13850 (0'0) promote 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by osd.17.0:10 2014-08-11 18:42:35.268824 2014-08-13 17:38:03.004707 7fe9bae887a0 20 read_log 193305'13851 (193305'13850) modify 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.951337.0:548 2014-08-11 18:42:35.261381 2014-08-13 17:38:03.004719 7fe9bae887a0 20 read_log 193305'13852 (0'0) promote 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:23 2014-08-11 18:42:36.794168 2014-08-13 17:38:03.004731 7fe9bae887a0 20 read_log 193305'13853 (193305'13852) modify 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by client.951337.0:932 2014-08-11 18:42:36.720429 2014-08-13 17:38:03.004743 7fe9bae887a0 20 read_log 193305'13854 (0'0) promote 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:85 2014-08-11 18:42:51.408499 2014-08-13 17:38:03.004756 7fe9bae887a0 20 read_log 193305'13855 (193305'13854) modify 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by client.951337.0:4911 2014-08-11 18:42:51.400279 2014-08-13 17:38:03.004768 7fe9bae887a0 20 read_log 193305'13856 (0'0) promote 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:95 2014-08-11 18:42:52.899346 2014-08-13 17:38:03.004780 7fe9bae887a0 20 read_log 193305'13857 (193305'13856) modify 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by client.951337.0:5303 2014-08-11 18:42:52.889184 2014-08-13 17:38:03.004792 7fe9bae887a0 20 read_log 193305'13858 (0'0) promote 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by osd.17.0:97 2014-08-11 18:42:53.570702 2014-08-13 17:38:03.004804 7fe9bae887a0 20 read_log 193305'13859 (193305'13858) modify 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.951337.0:5490 2014-08-11 18:42:53.563157 2014-08-13 17:38:03.004823 7fe9bae887a0 20 read_log 193305'13860 (0'0) promote 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:109 2014-08-11 18:42:56.007818 2014-08-13 17:38:03.004836 7fe9bae887a0 20 read_log 193305'13861 (193305'13860) modify 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by client.951337.0:6160 2014-08-11 18:42:56.000458 2014-08-13 17:38:03.004848 7fe9bae887a0 20 read_log 193305'13862 (0'0) promote dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by osd.17.0:131 2014-08-11 18:43:00.544443 2014-08-13 17:38:03.004860 7fe9bae887a0 20 read_log 193305'13863 (193305'13862) modify dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.951337.0:7416 2014-08-11 18:43:00.536917 2014-08-13 17:38:03.004878 7fe9bae887a0 20 read_log 193305'13864 (0'0) promote 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by osd.17.0:143 2014-08-11 18:43:03.622986 2014-08-13 17:38:03.004890 7fe9bae887a0 20 read_log 193305'13865 (193305'13864) modify 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.951337.0:8242 2014-08-11 18:43:03.615856 2014-08-13 17:38:03.004902 7fe9bae887a0 20 read_log 193305'13866 (0'0) promote 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by osd.17.0:159 2014-08-11 18:43:08.597263 2014-08-13 17:38:03.004913 7fe9bae887a0 20 read_log 193305'13867 (193305'13866) modify 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.951337.0:9535 2014-08-11 18:43:08.587522 2014-08-13 17:38:03.004926 7fe9bae887a0 20 read_log 193305'13868 (0'0) promote 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by osd.17.0:161 2014-08-11 18:43:09.357897 2014-08-13 17:38:03.004938 7fe9bae887a0 20 read_log 193305'13869 (193305'13868) modify 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.951337.0:9742 2014-08-11 18:43:09.350113 2014-08-13 17:38:03.004951 7fe9bae887a0 20 read_log 193305'13870 (0'0) promote 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by osd.17.0:167 2014-08-11 18:43:10.000332 2014-08-13 17:38:03.004965 7fe9bae887a0 20 read_log 193305'13871 (193305'13870) modify 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.951337.0:9918 2014-08-11 18:43:09.990208 2014-08-13 17:38:03.004978 7fe9bae887a0 20 read_log 193305'13872 (0'0) promote ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by osd.17.0:173 2014-08-11 18:43:11.439680 2014-08-13 17:38:03.004990 7fe9bae887a0 20 read_log 193305'13873 (193305'13872) modify ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.951337.0:10317 2014-08-11 18:43:11.431564 2014-08-13 17:38:03.005001 7fe9bae887a0 20 read_log 193305'13874 (0'0) promote 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:201 2014-08-11 18:43:18.362657 2014-08-13 17:38:03.005013 7fe9bae887a0 20 read_log 193305'13875 (193305'13874) modify 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by client.951337.0:12129 2014-08-11 18:43:18.356094 2014-08-13 17:38:03.005025 7fe9bae887a0 20 read_log 193305'13876 (0'0) promote ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:214 2014-08-11 18:43:21.353237 2014-08-13 17:38:03.005037 7fe9bae887a0 20 read_log 193305'13877 (193305'13876) modify ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by client.951337.0:12923 2014-08-11 18:43:21.346442 2014-08-13 17:38:03.005049 7fe9bae887a0 20 read_log 193305'13878 (0'0) promote 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:218 2014-08-11 18:43:22.052631 2014-08-13 17:38:03.005060 7fe9bae887a0 20 read_log 193305'13879 (193305'13878) modify 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by client.951337.0:13116 2014-08-11 18:43:22.046456 2014-08-13 17:38:03.005080 7fe9bae887a0 20 read_log 193305'13880 (0'0) promote b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by osd.17.0:222 2014-08-11 18:43:22.822538 2014-08-13 17:38:03.005092 7fe9bae887a0 20 read_log 193305'13881 (193305'13880) modify b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.951337.0:13318 2014-08-11 18:43:22.813181 2014-08-13 17:38:03.005105 7fe9bae887a0 20 read_log 193305'13882 (0'0) promote 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:243 2014-08-11 18:43:29.216490 2014-08-13 17:38:03.005117 7fe9bae887a0 20 read_log 193305'13883 (193305'13882) modify 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by client.951337.0:15072 2014-08-11 18:43:29.210284 2014-08-13 17:38:03.005129 7fe9bae887a0 20 read_log 193305'13884 (0'0) promote 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by osd.17.0:251 2014-08-11 18:43:30.648736 2014-08-13 17:38:03.005141 7fe9bae887a0 20 read_log 193305'13885 (193305'13884) modify 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.951337.0:15497 2014-08-11 18:43:30.641401 2014-08-13 17:38:03.005153 7fe9bae887a0 20 read_log 193305'13886 (0'0) promote 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by osd.17.0:257 2014-08-11 18:43:31.778700 2014-08-13 17:38:03.005170 7fe9bae887a0 20 read_log 193305'13887 (193305'13886) modify 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.951337.0:15827 2014-08-11 18:43:31.771381 2014-08-13 17:38:03.005189 7fe9bae887a0 20 read_log 193305'13888 (193305'13888) modify 2a1/hit_set_24.2a1_archive_2014-08-11 18:38:33.126704_2014-08-11 18:43:37.373511/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:37.373771 2014-08-13 17:38:03.005213 7fe9bae887a0 20 read_log 193305'13889 (0'0) promote 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by osd.17.0:268 2014-08-11 18:43:37.377105 2014-08-13 17:38:03.005226 7fe9bae887a0 20 read_log 193305'13890 (193305'13889) modify 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.951337.0:17420 2014-08-11 18:43:37.366298 2014-08-13 17:38:03.005238 7fe9bae887a0 20 read_log 193305'13891 (0'0) promote 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by osd.17.0:278 2014-08-11 18:43:39.078991 2014-08-13 17:38:03.005250 7fe9bae887a0 20 read_log 193305'13892 (193305'13891) modify 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.951337.0:17912 2014-08-11 18:43:39.070327 2014-08-13 17:38:03.005262 7fe9bae887a0 20 read_log 193305'13893 (0'0) promote f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:312 2014-08-11 18:43:46.408850 2014-08-13 17:38:03.005274 7fe9bae887a0 20 read_log 193305'13894 (193305'13893) modify f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by client.951337.0:19946 2014-08-11 18:43:46.401796 2014-08-13 17:38:03.005286 7fe9bae887a0 20 read_log 193305'13895 (0'0) promote 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:356 2014-08-11 18:43:57.215262 2014-08-13 17:38:03.005298 7fe9bae887a0 20 read_log 193305'13896 (193305'13895) modify 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by client.951337.0:22935 2014-08-11 18:43:57.208525 2014-08-13 17:38:03.005310 7fe9bae887a0 20 read_log 193305'13897 (0'0) promote 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by osd.17.0:383 2014-08-11 18:44:01.985860 2014-08-13 17:38:03.005322 7fe9bae887a0 20 read_log 193305'13898 (193305'13897) modify 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.951337.0:24261 2014-08-11 18:44:01.976622 2014-08-13 17:38:03.005346 7fe9bae887a0 20 read_log 193305'13899 (0'0) promote aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by osd.17.0:397 2014-08-11 18:44:04.659279 2014-08-13 17:38:03.005359 7fe9bae887a0 20 read_log 193305'13900 (193305'13899) modify aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.951337.0:24996 2014-08-11 18:44:04.651110 2014-08-13 17:38:03.005371 7fe9bae887a0 20 read_log 193305'13901 (0'0) promote 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by osd.17.0:409 2014-08-11 18:44:07.987238 2014-08-13 17:38:03.005383 7fe9bae887a0 20 read_log 193305'13902 (193305'13901) modify 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.951337.0:25901 2014-08-11 18:44:07.980629 2014-08-13 17:38:03.005395 7fe9bae887a0 20 read_log 193305'13903 (0'0) promote 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:427 2014-08-11 18:44:13.330964 2014-08-13 17:38:03.005407 7fe9bae887a0 20 read_log 193305'13904 (193305'13903) modify 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by client.951337.0:27341 2014-08-11 18:44:13.322852 2014-08-13 17:38:03.005419 7fe9bae887a0 20 read_log 193305'13905 (0'0) promote 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:451 2014-08-11 18:44:20.909281 2014-08-13 17:38:03.005431 7fe9bae887a0 20 read_log 193305'13906 (193305'13905) modify 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by client.951337.0:29430 2014-08-11 18:44:20.900747 2014-08-13 17:38:03.005443 7fe9bae887a0 20 read_log 193305'13907 (0'0) promote 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by osd.17.0:463 2014-08-11 18:44:23.934684 2014-08-13 17:38:03.005455 7fe9bae887a0 20 read_log 193305'13908 (193305'13907) modify 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.951337.0:30233 2014-08-11 18:44:23.924983 2014-08-13 17:38:03.005468 7fe9bae887a0 20 read_log 193305'13909 (0'0) promote e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:477 2014-08-11 18:44:26.464272 2014-08-13 17:38:03.005480 7fe9bae887a0 20 read_log 193305'13910 (193305'13909) modify e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by client.951337.0:30895 2014-08-11 18:44:26.455945 2014-08-13 17:38:03.005492 7fe9bae887a0 20 read_log 193305'13911 (0'0) promote 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by osd.17.0:501 2014-08-11 18:44:32.083570 2014-08-13 17:38:03.005504 7fe9bae887a0 20 read_log 193305'13912 (193305'13911) modify 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.951337.0:32500 2014-08-11 18:44:32.075785 2014-08-13 17:38:03.005516 7fe9bae887a0 20 read_log 193305'13913 (193305'13847) modify b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by client.951337.0:32713 2014-08-11 18:44:33.823691 2014-08-13 17:38:03.005528 7fe9bae887a0 20 read_log 193305'13914 (193305'13849) modify 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.951337.0:33185 2014-08-11 18:44:34.223969 2014-08-13 17:38:03.005540 7fe9bae887a0 20 read_log 193305'13915 (193305'13851) modify 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.951337.0:33259 2014-08-11 18:44:34.281589 2014-08-13 17:38:03.005552 7fe9bae887a0 20 read_log 193305'13916 (193305'13853) modify 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by client.951337.0:33643 2014-08-11 18:44:34.552503 2014-08-13 17:38:03.005564 7fe9bae887a0 20 read_log 193305'13917 (193305'13855) modify 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by client.951337.0:37622 2014-08-11 18:44:36.995436 2014-08-13 17:38:03.005576 7fe9bae887a0 20 read_log 193305'13918 (193305'13857) modify 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by client.951337.0:38014 2014-08-11 18:44:37.197751 2014-08-13 17:38:03.005595 7fe9bae887a0 20 read_log 193305'13919 (193305'13859) modify 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.951337.0:38201 2014-08-11 18:44:37.285099 2014-08-13 17:38:03.005608 7fe9bae887a0 20 read_log 193305'13920 (193305'13861) modify 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by client.951337.0:38871 2014-08-11 18:44:37.703329 2014-08-13 17:38:03.005620 7fe9bae887a0 20 read_log 193305'13921 (193305'13863) modify dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.951337.0:40127 2014-08-11 18:44:38.298514 2014-08-13 17:38:03.005633 7fe9bae887a0 20 read_log 193305'13922 (193305'13865) modify 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.951337.0:40953 2014-08-11 18:44:38.586488 2014-08-13 17:38:03.005644 7fe9bae887a0 20 read_log 193305'13923 (193305'13867) modify 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.951337.0:42246 2014-08-11 18:44:39.162709 2014-08-13 17:38:03.005656 7fe9bae887a0 20 read_log 193305'13924 (193305'13869) modify 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.951337.0:42453 2014-08-11 18:44:39.237610 2014-08-13 17:38:03.005668 7fe9bae887a0 20 read_log 193305'13925 (193305'13871) modify 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.951337.0:42629 2014-08-11 18:44:39.320870 2014-08-13 17:38:03.005680 7fe9bae887a0 20 read_log 193305'13926 (193305'13873) modify ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.951337.0:43028 2014-08-11 18:44:39.461329 2014-08-13 17:38:03.005692 7fe9bae887a0 20 read_log 193305'13927 (193305'13875) modify 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by client.951337.0:44840 2014-08-11 18:44:40.008527 2014-08-13 17:38:03.005704 7fe9bae887a0 20 read_log 193305'13928 (193305'13877) modify ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by client.951337.0:45634 2014-08-11 18:44:40.257328 2014-08-13 17:38:03.005716 7fe9bae887a0 20 read_log 193305'13929 (193305'13879) modify 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by client.951337.0:45827 2014-08-11 18:44:40.313407 2014-08-13 17:38:03.005728 7fe9bae887a0 20 read_log 193305'13930 (193305'13881) modify b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.951337.0:46029 2014-08-11 18:44:40.370533 2014-08-13 17:38:03.005740 7fe9bae887a0 20 read_log 193305'13931 (193305'13883) modify 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by client.951337.0:47783 2014-08-11 18:44:40.865418 2014-08-13 17:38:03.005752 7fe9bae887a0 20 read_log 193305'13932 (193305'13885) modify 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.951337.0:48208 2014-08-11 18:44:40.988037 2014-08-13 17:38:03.005764 7fe9bae887a0 20 read_log 193305'13933 (193305'13887) modify 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.951337.0:48538 2014-08-11 18:44:41.082418 2014-08-13 17:38:03.005776 7fe9bae887a0 20 read_log 193305'13934 (193305'13890) modify 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.951337.0:50131 2014-08-11 18:44:41.580663 2014-08-13 17:38:03.005788 7fe9bae887a0 20 read_log 193305'13935 (193305'13892) modify 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.951337.0:50623 2014-08-11 18:44:41.719228 2014-08-13 17:38:03.005806 7fe9bae887a0 20 read_log 193305'13936 (193305'13894) modify f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by client.951337.0:52657 2014-08-11 18:44:42.297074 2014-08-13 17:38:03.005837 7fe9bae887a0 20 read_log 193305'13937 (193305'13896) modify 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by client.951337.0:55646 2014-08-11 18:44:43.188624 2014-08-13 17:38:03.005850 7fe9bae887a0 20 read_log 193305'13938 (193305'13898) modify 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.951337.0:56972 2014-08-11 18:44:43.589799 2014-08-13 17:38:03.005872 7fe9bae887a0 20 read_log 193305'13939 (193305'13900) modify aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.951337.0:57707 2014-08-11 18:44:43.836769 2014-08-13 17:38:03.005885 7fe9bae887a0 20 read_log 193305'13940 (193305'13902) modify 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.951337.0:58612 2014-08-11 18:44:44.091813 2014-08-13 17:38:03.005897 7fe9bae887a0 20 read_log 193305'13941 (193305'13904) modify 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by client.951337.0:60052 2014-08-11 18:44:44.506405 2014-08-13 17:38:03.005909 7fe9bae887a0 20 read_log 193305'13942 (193305'13906) modify 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by client.951337.0:62141 2014-08-11 18:44:45.137774 2014-08-13 17:38:03.005921 7fe9bae887a0 20 read_log 193305'13943 (193305'13908) modify 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.951337.0:62944 2014-08-11 18:44:45.368493 2014-08-13 17:38:03.005933 7fe9bae887a0 20 read_log 193305'13944 (193305'13910) modify e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by client.951337.0:63606 2014-08-11 18:44:45.557740 2014-08-13 17:38:03.005944 7fe9bae887a0 20 read_log 193305'13945 (193305'13912) modify 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.951337.0:65211 2014-08-11 18:44:46.051100 2014-08-13 17:38:03.005956 7fe9bae887a0 20 read_log 194209'13946 (192888'13628) clean d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.17.0:1312 2014-08-12 14:36:39.840975 2014-08-13 17:38:03.005969 7fe9bae887a0 20 read_log 194209'13947 (192834'12750) clean 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.17.0:1313 2014-08-12 14:36:39.841511 2014-08-13 17:38:03.005982 7fe9bae887a0 20 read_log 194209'13948 (192888'13665) clean 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.17.0:1314 2014-08-12 14:36:39.841939 2014-08-13 17:38:03.005994 7fe9bae887a0 20 read_log 194209'13949 (192834'13290) clean 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.17.0:1315 2014-08-12 14:36:39.842440 2014-08-13 17:38:03.006006 7fe9bae887a0 20 read_log 194209'13950 (192888'13659) clean 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.17.0:1316 2014-08-12 14:36:39.853153 2014-08-13 17:38:03.006018 7fe9bae887a0 20 read_log 194209'13951 (192834'13364) clean a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.17.0:1317 2014-08-12 14:36:39.853441 2014-08-13 17:38:03.006030 7fe9bae887a0 20 read_log 194209'13952 (192834'12835) clean 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.17.0:1318 2014-08-12 14:36:39.856558 2014-08-13 17:38:03.006042 7fe9bae887a0 20 read_log 194209'13953 (192888'13832) clean 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.17.0:1319 2014-08-12 14:36:39.859767 2014-08-13 17:38:03.006054 7fe9bae887a0 20 read_log 194209'13954 (192834'13389) clean a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.17.0:1320 2014-08-12 14:36:39.863056 2014-08-13 17:38:03.006065 7fe9bae887a0 20 read_log 194209'13955 (192888'13612) clean 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.17.0:1321 2014-08-12 14:36:39.866254 2014-08-13 17:38:03.006077 7fe9bae887a0 20 read_log 194209'13956 (192888'13600) clean 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.17.0:1322 2014-08-12 14:36:39.869358 2014-08-13 17:38:03.006090 7fe9bae887a0 20 read_log 194209'13957 (192834'13360) clean 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.17.0:1323 2014-08-12 14:36:39.872782 2014-08-13 17:38:03.006109 7fe9bae887a0 20 read_log 194209'13958 (192888'13605) clean 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.17.0:1324 2014-08-12 14:36:39.875720 2014-08-13 17:38:03.006122 7fe9bae887a0 20 read_log 194209'13959 (193305'13913) clean b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1325 2014-08-12 14:36:39.878882 2014-08-13 17:38:03.006134 7fe9bae887a0 20 read_log 194209'13960 (193305'13942) clean 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:1326 2014-08-12 14:36:39.882152 2014-08-13 17:38:03.006146 7fe9bae887a0 20 read_log 194209'13961 (193305'13936) clean f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:1327 2014-08-12 14:36:39.885392 2014-08-13 17:38:03.006163 7fe9bae887a0 20 read_log 194209'13962 (192888'13828) clean 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.17.0:1328 2014-08-12 14:36:39.888879 2014-08-13 17:38:03.006176 7fe9bae887a0 20 read_log 194209'13963 (192834'13313) clean fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.17.0:1329 2014-08-12 14:36:39.891830 2014-08-13 17:38:03.006188 7fe9bae887a0 20 read_log 194209'13964 (192888'13792) clean cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.17.0:1330 2014-08-12 14:36:39.897889 2014-08-13 17:38:03.006199 7fe9bae887a0 20 read_log 194209'13965 (192888'13840) clean 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.17.0:1331 2014-08-12 14:36:39.898167 2014-08-13 17:38:03.006211 7fe9bae887a0 20 read_log 194209'13966 (193305'13928) clean ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:1332 2014-08-12 14:36:39.901406 2014-08-13 17:38:03.006223 7fe9bae887a0 20 read_log 194209'13967 (192834'13378) clean 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.17.0:1333 2014-08-12 14:36:39.904842 2014-08-13 17:38:03.006235 7fe9bae887a0 20 read_log 194209'13968 (192888'13825) clean c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.17.0:1334 2014-08-12 14:36:39.908161 2014-08-13 17:38:03.006247 7fe9bae887a0 20 read_log 194209'13969 (193305'13944) clean e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:1335 2014-08-12 14:36:39.911222 2014-08-13 17:38:03.006259 7fe9bae887a0 20 read_log 194209'13970 (192888'13643) clean 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.17.0:1336 2014-08-12 14:36:39.914346 2014-08-13 17:38:03.006271 7fe9bae887a0 20 read_log 194209'13971 (192834'13440) clean 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.17.0:1337 2014-08-12 14:36:39.917712 2014-08-13 17:38:03.006283 7fe9bae887a0 20 read_log 194209'13972 (192834'13375) clean 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.17.0:1338 2014-08-12 14:36:39.921471 2014-08-13 17:38:03.006296 7fe9bae887a0 20 read_log 194209'13973 (192888'13644) clean 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.17.0:1339 2014-08-12 14:36:39.924330 2014-08-13 17:38:03.006308 7fe9bae887a0 20 read_log 194209'13974 (193305'13917) clean 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:1340 2014-08-12 14:36:39.927406 2014-08-13 17:38:03.006320 7fe9bae887a0 20 read_log 194209'13975 (192888'13814) clean 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.17.0:1341 2014-08-12 14:36:39.930786 2014-08-13 17:38:03.006332 7fe9bae887a0 20 read_log 194209'13976 (192888'13843) clean f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.17.0:1342 2014-08-12 14:36:39.934541 2014-08-13 17:38:03.006344 7fe9bae887a0 20 read_log 194209'13977 (193305'13937) clean 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:1343 2014-08-12 14:36:39.937405 2014-08-13 17:38:03.006364 7fe9bae887a0 20 read_log 194209'13978 (192888'13645) clean e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.17.0:1344 2014-08-12 14:36:39.940426 2014-08-13 17:38:03.006376 7fe9bae887a0 20 read_log 194209'13979 (193305'13927) clean 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:1345 2014-08-12 14:36:39.943623 2014-08-13 17:38:03.006388 7fe9bae887a0 20 read_log 194209'13980 (192888'13799) clean 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.17.0:1346 2014-08-12 14:36:39.946743 2014-08-13 17:38:03.006400 7fe9bae887a0 20 read_log 194209'13981 (193305'13916) clean 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:1347 2014-08-12 14:36:39.950002 2014-08-13 17:38:03.006412 7fe9bae887a0 20 read_log 194209'13982 (192834'13385) clean eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.17.0:1348 2014-08-12 14:36:39.953360 2014-08-13 17:38:03.006424 7fe9bae887a0 20 read_log 194209'13983 (192888'13619) clean 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.17.0:1349 2014-08-12 14:36:39.956540 2014-08-13 17:38:03.006435 7fe9bae887a0 20 read_log 194209'13984 (192834'12890) clean c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.17.0:1350 2014-08-12 14:36:39.959665 2014-08-13 17:38:03.006447 7fe9bae887a0 20 read_log 194209'13985 (192888'13837) clean cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.17.0:1351 2014-08-12 14:36:39.963405 2014-08-13 17:38:03.006459 7fe9bae887a0 20 read_log 194209'13986 (192888'13656) clean b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by osd.17.0:1352 2014-08-12 14:36:39.967013 2014-08-13 17:38:03.006471 7fe9bae887a0 20 read_log 194209'13987 (192888'13793) clean af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by osd.17.0:1353 2014-08-12 14:36:39.970637 2014-08-13 17:38:03.006482 7fe9bae887a0 20 read_log 194209'13988 (193305'13941) clean 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:1354 2014-08-12 14:36:39.974087 2014-08-13 17:38:03.006494 7fe9bae887a0 20 read_log 194209'13989 (193305'13918) clean 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:1355 2014-08-12 14:36:39.977730 2014-08-13 17:38:03.006506 7fe9bae887a0 20 read_log 194209'13990 (193305'13920) clean 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:1356 2014-08-12 14:36:39.981344 2014-08-13 17:38:03.006518 7fe9bae887a0 20 read_log 194209'13991 (192834'13327) clean 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.17.0:1357 2014-08-12 14:36:39.984830 2014-08-13 17:38:03.006529 7fe9bae887a0 20 read_log 194209'13992 (192834'13433) clean c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.17.0:1358 2014-08-12 14:36:39.988708 2014-08-13 17:38:03.006541 7fe9bae887a0 20 read_log 194209'13993 (192888'13629) clean 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.17.0:1359 2014-08-12 14:36:39.992137 2014-08-13 17:38:03.006553 7fe9bae887a0 20 read_log 194209'13994 (192888'13838) clean a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.17.0:1360 2014-08-12 14:36:39.995692 2014-08-13 17:38:03.006565 7fe9bae887a0 20 read_log 194209'13995 (192834'12848) clean bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.17.0:1361 2014-08-12 14:36:39.999090 2014-08-13 17:38:03.006576 7fe9bae887a0 20 read_log 194209'13996 (192888'13809) clean 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.17.0:1362 2014-08-12 14:36:40.002686 2014-08-13 17:38:03.006588 7fe9bae887a0 20 read_log 194209'13997 (193305'13931) clean 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:1363 2014-08-12 14:36:40.005797 2014-08-13 17:38:03.006610 7fe9bae887a0 20 read_log 194209'13998 (192834'13417) clean c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.17.0:1364 2014-08-12 14:36:40.009246 2014-08-13 17:38:03.006623 7fe9bae887a0 20 read_log 194209'13999 (192888'13822) clean df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.17.0:1365 2014-08-12 14:36:40.012776 2014-08-13 17:38:03.006635 7fe9bae887a0 20 read_log 194209'14000 (193305'13929) clean 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:1366 2014-08-12 14:36:40.016334 2014-08-13 17:38:03.006647 7fe9bae887a0 20 read_log 194209'14001 (192888'13601) clean bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.17.0:1367 2014-08-12 14:36:40.019766 2014-08-13 17:38:03.006659 7fe9bae887a0 20 read_log 194209'14002 (192834'12770) clean 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.17.0:1368 2014-08-12 14:36:40.023538 2014-08-13 17:38:03.006671 7fe9bae887a0 20 read_log 194209'14003 (194209'13946) delete d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.17.0:1369 2014-08-12 14:36:40.060792 2014-08-13 17:38:03.006683 7fe9bae887a0 20 read_log 194209'14004 (194209'13947) delete 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.17.0:1370 2014-08-12 14:36:40.063594 2014-08-13 17:38:03.006695 7fe9bae887a0 20 read_log 194209'14005 (192834'13332) clean 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.17.0:1371 2014-08-12 14:36:40.064040 2014-08-13 17:38:03.006707 7fe9bae887a0 20 read_log 194209'14006 (194209'13948) delete 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.17.0:1372 2014-08-12 14:36:40.083667 2014-08-13 17:38:03.006719 7fe9bae887a0 20 read_log 194209'14007 (194209'13949) delete 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.17.0:1373 2014-08-12 14:36:40.086818 2014-08-13 17:38:03.006730 7fe9bae887a0 20 read_log 194209'14008 (194209'13950) delete 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.17.0:1374 2014-08-12 14:36:40.087209 2014-08-13 17:38:03.006742 7fe9bae887a0 20 read_log 194209'14009 (194209'13951) delete a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.17.0:1375 2014-08-12 14:36:40.087609 2014-08-13 17:38:03.006754 7fe9bae887a0 20 read_log 194209'14010 (194209'13952) delete 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.17.0:1376 2014-08-12 14:36:40.091059 2014-08-13 17:38:03.006766 7fe9bae887a0 20 read_log 194209'14011 (194209'13953) delete 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.17.0:1377 2014-08-12 14:36:40.091418 2014-08-13 17:38:03.006778 7fe9bae887a0 20 read_log 194209'14012 (194209'13954) delete a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.17.0:1378 2014-08-12 14:36:40.091766 2014-08-13 17:38:03.006790 7fe9bae887a0 20 read_log 194209'14013 (194209'13955) delete 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.17.0:1379 2014-08-12 14:36:40.095118 2014-08-13 17:38:03.006802 7fe9bae887a0 20 read_log 194209'14014 (194209'13956) delete 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.17.0:1380 2014-08-12 14:36:40.095474 2014-08-13 17:38:03.006813 7fe9bae887a0 20 read_log 194209'14015 (194209'13957) delete 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.17.0:1381 2014-08-12 14:36:40.096353 2014-08-13 17:38:03.006825 7fe9bae887a0 20 read_log 194209'14016 (194209'13958) delete 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.17.0:1382 2014-08-12 14:36:40.099416 2014-08-13 17:38:03.006837 7fe9bae887a0 20 read_log 194209'14017 (194209'13959) delete b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1383 2014-08-12 14:36:40.099790 2014-08-13 17:38:03.006856 7fe9bae887a0 20 read_log 194209'14018 (194209'13960) delete 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:1384 2014-08-12 14:36:40.100593 2014-08-13 17:38:03.006868 7fe9bae887a0 20 read_log 194209'14019 (194209'13961) delete f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:1385 2014-08-12 14:36:40.103651 2014-08-13 17:38:03.006880 7fe9bae887a0 20 read_log 194209'14020 (194209'13966) delete ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:1386 2014-08-12 14:36:40.104947 2014-08-13 17:38:03.006891 7fe9bae887a0 20 read_log 194209'14021 (194209'13969) delete e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:1387 2014-08-12 14:36:40.105976 2014-08-13 17:38:03.006903 7fe9bae887a0 20 read_log 194209'14022 (194209'13974) delete 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:1388 2014-08-12 14:36:40.109593 2014-08-13 17:38:03.006915 7fe9bae887a0 20 read_log 194209'14023 (194209'13977) delete 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:1389 2014-08-12 14:36:40.110396 2014-08-13 17:38:03.006927 7fe9bae887a0 20 read_log 194209'14024 (194209'13979) delete 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:1390 2014-08-12 14:36:40.111164 2014-08-13 17:38:03.006938 7fe9bae887a0 20 read_log 194209'14025 (194209'13981) delete 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:1391 2014-08-12 14:36:40.114126 2014-08-13 17:38:03.006950 7fe9bae887a0 20 read_log 194209'14026 (194209'13988) delete 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:1392 2014-08-12 14:36:40.118331 2014-08-13 17:38:03.006962 7fe9bae887a0 20 read_log 194209'14027 (194209'13989) delete 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:1393 2014-08-12 14:36:40.118692 2014-08-13 17:38:03.006973 7fe9bae887a0 20 read_log 194209'14028 (194209'13990) delete 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:1394 2014-08-12 14:36:40.119202 2014-08-13 17:38:03.006985 7fe9bae887a0 20 read_log 194209'14029 (194209'13997) delete 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:1395 2014-08-12 14:36:40.123630 2014-08-13 17:38:03.007018 7fe9bae887a0 20 read_log 194209'14030 (194209'14000) delete 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:1396 2014-08-12 14:36:40.124602 2014-08-13 17:38:03.007030 7fe9bae887a0 20 read_log 194209'14031 (194209'13991) delete 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.17.0:1397 2014-08-12 14:36:50.230403 2014-08-13 17:38:03.007042 7fe9bae887a0 20 read_log 194209'14032 (194209'13992) delete c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.17.0:1398 2014-08-12 14:36:50.230902 2014-08-13 17:38:03.007054 7fe9bae887a0 20 read_log 194209'14033 (194209'13993) delete 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.17.0:1399 2014-08-12 14:36:50.231235 2014-08-13 17:38:03.007066 7fe9bae887a0 20 read_log 194209'14034 (194209'13994) delete a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.17.0:1400 2014-08-12 14:36:50.231504 2014-08-13 17:38:03.007079 7fe9bae887a0 20 read_log 194209'14035 (194209'13995) delete bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.17.0:1401 2014-08-12 14:36:50.234662 2014-08-13 17:38:03.007090 7fe9bae887a0 20 read_log 194209'14036 (194209'13996) delete 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.17.0:1402 2014-08-12 14:36:50.235023 2014-08-13 17:38:03.007102 7fe9bae887a0 20 read_log 194209'14037 (194209'13998) delete c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.17.0:1403 2014-08-12 14:36:50.235703 2014-08-13 17:38:03.007136 7fe9bae887a0 20 read_log 194209'14038 (194209'13999) delete df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.17.0:1404 2014-08-12 14:36:50.235987 2014-08-13 17:38:03.007149 7fe9bae887a0 20 read_log 194209'14039 (194209'14001) delete bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.17.0:1405 2014-08-12 14:36:50.239037 2014-08-13 17:38:03.007170 7fe9bae887a0 20 read_log 194209'14040 (194209'14002) delete 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.17.0:1406 2014-08-12 14:36:50.239395 2014-08-13 17:38:03.007183 7fe9bae887a0 20 read_log 194209'14041 (194209'14005) delete 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.17.0:1407 2014-08-12 14:36:50.239691 2014-08-13 17:38:03.007195 7fe9bae887a0 20 read_log 194209'14042 (194209'13962) delete 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.17.0:1408 2014-08-12 14:36:50.267483 2014-08-13 17:38:03.007225 7fe9bae887a0 20 read_log 194209'14043 (194209'13963) delete fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.17.0:1409 2014-08-12 14:36:50.267805 2014-08-13 17:38:03.007239 7fe9bae887a0 20 read_log 194209'14044 (194209'13964) delete cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.17.0:1410 2014-08-12 14:36:50.268127 2014-08-13 17:38:03.007251 7fe9bae887a0 20 read_log 194209'14045 (194209'13965) delete 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.17.0:1411 2014-08-12 14:36:50.268438 2014-08-13 17:38:03.007263 7fe9bae887a0 20 read_log 194209'14046 (194209'13967) delete 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.17.0:1412 2014-08-12 14:36:50.270990 2014-08-13 17:38:03.007276 7fe9bae887a0 20 read_log 194209'14047 (194209'13968) delete c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.17.0:1413 2014-08-12 14:36:50.271296 2014-08-13 17:38:03.007288 7fe9bae887a0 20 read_log 194209'14048 (194209'13970) delete 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.17.0:1414 2014-08-12 14:36:50.271596 2014-08-13 17:38:03.007300 7fe9bae887a0 20 read_log 194209'14049 (194209'13971) delete 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.17.0:1415 2014-08-12 14:36:50.272140 2014-08-13 17:38:03.007312 7fe9bae887a0 20 read_log 194209'14050 (194209'13972) delete 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.17.0:1416 2014-08-12 14:36:50.274663 2014-08-13 17:38:03.007324 7fe9bae887a0 20 read_log 194209'14051 (194209'13973) delete 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.17.0:1417 2014-08-12 14:36:50.274946 2014-08-13 17:38:03.007335 7fe9bae887a0 20 read_log 194209'14052 (194209'13975) delete 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.17.0:1418 2014-08-12 14:36:50.275257 2014-08-13 17:38:03.007347 7fe9bae887a0 20 read_log 194209'14053 (194209'13976) delete f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.17.0:1419 2014-08-12 14:36:50.275791 2014-08-13 17:38:03.007359 7fe9bae887a0 20 read_log 194209'14054 (194209'13978) delete e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.17.0:1420 2014-08-12 14:36:50.278244 2014-08-13 17:38:03.007371 7fe9bae887a0 20 read_log 194209'14055 (194209'13980) delete 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.17.0:1421 2014-08-12 14:36:50.278528 2014-08-13 17:38:03.007383 7fe9bae887a0 20 read_log 194209'14056 (194209'13982) delete eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.17.0:1422 2014-08-12 14:36:50.278803 2014-08-13 17:38:03.007395 7fe9bae887a0 20 read_log 194209'14057 (194209'13983) delete 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.17.0:1423 2014-08-12 14:36:50.279330 2014-08-13 17:38:03.007417 7fe9bae887a0 20 read_log 194209'14058 (194209'13984) delete c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.17.0:1424 2014-08-12 14:36:50.281811 2014-08-13 17:38:03.007430 7fe9bae887a0 20 read_log 194209'14059 (194209'13985) delete cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.17.0:1425 2014-08-12 14:36:50.282098 2014-08-13 17:38:03.007443 7fe9bae887a0 20 read_log 194215'14060 (194215'14060) modify 2a1/hit_set_24.2a1_archive_2014-08-12 15:32:25.911312_2014-08-12 15:46:28.833325/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:28.833985 2014-08-13 17:38:03.007455 7fe9bae887a0 20 read_log 194215'14061 (193305'13888) delete 2a1/hit_set_24.2a1_archive_2014-08-11 18:38:33.126704_2014-08-11 18:43:37.373511/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:28.833985 2014-08-13 17:38:03.007467 7fe9bae887a0 20 read_log 194215'14062 (194209'13986) delete b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.2174595.0:138494 0.000000 2014-08-13 17:38:03.007478 7fe9bae887a0 20 read_log 194215'14063 (194209'13987) delete af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.2174595.0:138496 0.000000 2014-08-13 17:38:03.007489 7fe9bae887a0 20 read_log 194215'14064 (192888'13631) clean dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by osd.17.0:2349 2014-08-12 15:46:28.850046 2014-08-13 17:38:03.007501 7fe9bae887a0 20 read_log 194215'14065 (194215'14064) delete dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.2174595.0:138498 0.000000 2014-08-13 17:38:03.007511 7fe9bae887a0 20 read_log 194215'14066 (192888'13811) clean 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by osd.17.0:2351 2014-08-12 15:46:28.858047 2014-08-13 17:38:03.007523 7fe9bae887a0 20 read_log 194215'14067 (194215'14066) delete 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.2174595.0:138500 0.000000 2014-08-13 17:38:03.007534 7fe9bae887a0 20 read_log 194215'14068 (192834'13354) clean e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by osd.17.0:2353 2014-08-12 15:46:28.866034 2014-08-13 17:38:03.007546 7fe9bae887a0 20 read_log 194215'14069 (194215'14068) delete e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.2174595.0:138502 0.000000 2014-08-13 17:38:03.007557 7fe9bae887a0 20 read_log 194215'14070 (192834'12895) clean 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by osd.17.0:2355 2014-08-12 15:46:28.874075 2014-08-13 17:38:03.007568 7fe9bae887a0 20 read_log 194215'14071 (194215'14070) delete 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.2174595.0:138504 0.000000 2014-08-13 17:38:03.007579 7fe9bae887a0 20 read_log 194215'14072 (193305'13926) clean ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by osd.17.0:2357 2014-08-12 15:46:28.882296 2014-08-13 17:38:03.007591 7fe9bae887a0 20 read_log 194215'14073 (194215'14072) delete ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.2174595.0:138506 0.000000 2014-08-13 17:38:03.007601 7fe9bae887a0 20 read_log 194215'14074 (193305'13921) clean dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by osd.17.0:2359 2014-08-12 15:46:28.890285 2014-08-13 17:38:03.007613 7fe9bae887a0 20 read_log 194215'14075 (194215'14074) delete dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.2174595.0:138508 0.000000 2014-08-13 17:38:03.007624 7fe9bae887a0 20 read_log 194215'14076 (192888'13667) clean 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by osd.17.0:2361 2014-08-12 15:46:28.897968 2014-08-13 17:38:03.007643 7fe9bae887a0 20 read_log 194215'14077 (194215'14076) delete 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.2174595.0:138510 0.000000 2014-08-13 17:38:03.007654 7fe9bae887a0 20 read_log 194215'14078 (192834'13382) clean 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by osd.17.0:2363 2014-08-12 15:46:28.905241 2014-08-13 17:38:03.007666 7fe9bae887a0 20 read_log 194215'14079 (194215'14078) delete 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.2174595.0:138512 0.000000 2014-08-13 17:38:03.007677 7fe9bae887a0 20 read_log 194215'14080 (192834'13308) clean 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by osd.17.0:2365 2014-08-12 15:46:28.913600 2014-08-13 17:38:03.007689 7fe9bae887a0 20 read_log 194215'14081 (194215'14080) delete 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.2174595.0:138514 0.000000 2014-08-13 17:38:03.007700 7fe9bae887a0 20 read_log 194215'14082 (192888'13789) clean b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by osd.17.0:2367 2014-08-12 15:46:28.921566 2014-08-13 17:38:03.007712 7fe9bae887a0 20 read_log 194215'14083 (194215'14082) delete b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.2174595.0:138516 0.000000 2014-08-13 17:38:03.007722 7fe9bae887a0 20 read_log 194215'14084 (192834'12862) clean 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by osd.17.0:2369 2014-08-12 15:46:28.929405 2014-08-13 17:38:03.007734 7fe9bae887a0 20 read_log 194215'14085 (194215'14084) delete 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.2174595.0:138518 0.000000 2014-08-13 17:38:03.007745 7fe9bae887a0 20 read_log 194215'14086 (193305'13933) clean 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by osd.17.0:2371 2014-08-12 15:46:28.937425 2014-08-13 17:38:03.007757 7fe9bae887a0 20 read_log 194215'14087 (194215'14086) delete 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.2174595.0:138520 0.000000 2014-08-13 17:38:03.007767 7fe9bae887a0 20 read_log 194215'14088 (192888'13650) clean 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by osd.17.0:2373 2014-08-12 15:46:28.946083 2014-08-13 17:38:03.007779 7fe9bae887a0 20 read_log 194215'14089 (194215'14088) delete 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.2174595.0:138522 0.000000 2014-08-13 17:38:03.007790 7fe9bae887a0 20 read_log 194215'14090 (192834'12846) clean b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by osd.17.0:2375 2014-08-12 15:46:28.953599 2014-08-13 17:38:03.007802 7fe9bae887a0 20 read_log 194215'14091 (194215'14090) delete b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.2174595.0:138524 0.000000 2014-08-13 17:38:03.007812 7fe9bae887a0 20 read_log 194215'14092 (192888'13666) clean 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by osd.17.0:2377 2014-08-12 15:46:28.961737 2014-08-13 17:38:03.007824 7fe9bae887a0 20 read_log 194215'14093 (194215'14092) delete 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.2174595.0:138526 0.000000 2014-08-13 17:38:03.007835 7fe9bae887a0 20 read_log 194215'14094 (192834'12803) clean ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by osd.17.0:2379 2014-08-12 15:46:28.970540 2014-08-13 17:38:03.007847 7fe9bae887a0 20 read_log 194215'14095 (194215'14094) delete ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.2174595.0:138528 0.000000 2014-08-13 17:38:03.007857 7fe9bae887a0 20 read_log 194215'14096 (192888'13622) clean e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by osd.17.0:2381 2014-08-12 15:46:28.978584 2014-08-13 17:38:03.007876 7fe9bae887a0 20 read_log 194215'14097 (194215'14096) delete e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.2174595.0:138530 0.000000 2014-08-13 17:38:03.007887 7fe9bae887a0 20 read_log 194215'14098 (192888'13827) clean 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by osd.17.0:2383 2014-08-12 15:46:28.986409 2014-08-13 17:38:03.007899 7fe9bae887a0 20 read_log 194215'14099 (194215'14098) delete 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.2174595.0:138532 0.000000 2014-08-13 17:38:03.007910 7fe9bae887a0 20 read_log 194215'14100 (192834'13401) clean 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by osd.17.0:2385 2014-08-12 15:46:28.994554 2014-08-13 17:38:03.007922 7fe9bae887a0 20 read_log 194215'14101 (194215'14100) delete 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.2174595.0:138534 0.000000 2014-08-13 17:38:03.007932 7fe9bae887a0 20 read_log 194215'14102 (192888'13655) clean 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by osd.17.0:2387 2014-08-12 15:46:29.001994 2014-08-13 17:38:03.007944 7fe9bae887a0 20 read_log 194215'14103 (194215'14102) delete 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.2174595.0:138536 0.000000 2014-08-13 17:38:03.007955 7fe9bae887a0 20 read_log 194215'14104 (193305'13923) clean 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by osd.17.0:2389 2014-08-12 15:46:29.010150 2014-08-13 17:38:03.007967 7fe9bae887a0 20 read_log 194215'14105 (194215'14104) delete 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.2174595.0:138538 0.000000 2014-08-13 17:38:03.007977 7fe9bae887a0 20 read_log 194215'14106 (193305'13924) clean 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by osd.17.0:2391 2014-08-12 15:46:29.018287 2014-08-13 17:38:03.007992 7fe9bae887a0 20 read_log 194215'14107 (194215'14106) delete 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.2174595.0:138540 0.000000 2014-08-13 17:38:03.008003 7fe9bae887a0 20 read_log 194215'14108 (192834'13328) clean baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by osd.17.0:2393 2014-08-12 15:46:29.026601 2014-08-13 17:38:03.008015 7fe9bae887a0 20 read_log 194215'14109 (194215'14108) delete baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.2174595.0:138542 0.000000 2014-08-13 17:38:03.008026 7fe9bae887a0 20 read_log 194215'14110 (193305'13922) clean 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by osd.17.0:2395 2014-08-12 15:46:29.034236 2014-08-13 17:38:03.008038 7fe9bae887a0 20 read_log 194215'14111 (194215'14110) delete 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.2174595.0:138544 0.000000 2014-08-13 17:38:03.008048 7fe9bae887a0 20 read_log 194215'14112 (192888'13653) clean 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by osd.17.0:2397 2014-08-12 15:46:29.042138 2014-08-13 17:38:03.008060 7fe9bae887a0 20 read_log 194215'14113 (194215'14112) delete 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.2174595.0:138546 0.000000 2014-08-13 17:38:03.008071 7fe9bae887a0 20 read_log 194215'14114 (192888'13842) clean 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by osd.17.0:2399 2014-08-12 15:46:29.049938 2014-08-13 17:38:03.008083 7fe9bae887a0 20 read_log 194215'14115 (194215'14114) delete 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.2174595.0:138548 0.000000 2014-08-13 17:38:03.008093 7fe9bae887a0 20 read_log 194215'14116 (192888'13615) clean 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by osd.17.0:2401 2014-08-12 15:46:29.057974 2014-08-13 17:38:03.008114 7fe9bae887a0 20 read_log 194215'14117 (194215'14116) delete 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.2174595.0:138550 0.000000 2014-08-13 17:38:03.008125 7fe9bae887a0 20 read_log 194215'14118 (192888'13647) clean a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by osd.17.0:2403 2014-08-12 15:46:29.065809 2014-08-13 17:38:03.008138 7fe9bae887a0 20 read_log 194215'14119 (194215'14118) delete a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.2174595.0:138552 0.000000 2014-08-13 17:38:03.008148 7fe9bae887a0 20 read_log 194215'14120 (192888'13639) clean 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by osd.17.0:2405 2014-08-12 15:46:29.073334 2014-08-13 17:38:03.008165 7fe9bae887a0 20 read_log 194215'14121 (194215'14120) delete 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.2174595.0:138554 0.000000 2014-08-13 17:38:03.008176 7fe9bae887a0 20 read_log 194215'14122 (192888'13839) clean 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by osd.17.0:2407 2014-08-12 15:46:29.080624 2014-08-13 17:38:03.008207 7fe9bae887a0 20 read_log 194215'14123 (194215'14122) delete 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.2174595.0:138556 0.000000 2014-08-13 17:38:03.008218 7fe9bae887a0 20 read_log 194215'14124 (192834'12793) clean 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by osd.17.0:2409 2014-08-12 15:46:29.088641 2014-08-13 17:38:03.008231 7fe9bae887a0 20 read_log 194215'14125 (194215'14124) delete 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.2174595.0:138558 0.000000 2014-08-13 17:38:03.008241 7fe9bae887a0 20 read_log 194215'14126 (192888'13835) clean 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by osd.17.0:2411 2014-08-12 15:46:29.095567 2014-08-13 17:38:03.008253 7fe9bae887a0 20 read_log 194215'14127 (194215'14126) delete 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.2174595.0:138560 0.000000 2014-08-13 17:38:03.008264 7fe9bae887a0 20 read_log 194215'14128 (192834'12786) clean 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by osd.17.0:2413 2014-08-12 15:46:29.103327 2014-08-13 17:38:03.008275 7fe9bae887a0 20 read_log 194215'14129 (194215'14128) delete 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.2174595.0:138562 0.000000 2014-08-13 17:38:03.008286 7fe9bae887a0 20 read_log 194215'14130 (192834'13278) clean 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by osd.17.0:2415 2014-08-12 15:46:29.109627 2014-08-13 17:38:03.008298 7fe9bae887a0 20 read_log 194215'14131 (194215'14130) delete 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.2174595.0:138564 0.000000 2014-08-13 17:38:03.008308 7fe9bae887a0 20 read_log 194215'14132 (192834'12881) clean eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by osd.17.0:2417 2014-08-12 15:46:29.116724 2014-08-13 17:38:03.008320 7fe9bae887a0 20 read_log 194215'14133 (194215'14132) delete eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.2174595.0:138566 0.000000 2014-08-13 17:38:03.008331 7fe9bae887a0 20 read_log 194215'14134 (192834'12831) clean c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by osd.17.0:2419 2014-08-12 15:46:29.124190 2014-08-13 17:38:03.008343 7fe9bae887a0 20 read_log 194215'14135 (194215'14134) delete c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.2174595.0:138568 0.000000 2014-08-13 17:38:03.008353 7fe9bae887a0 20 read_log 194215'14136 (193305'13935) clean 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by osd.17.0:2421 2014-08-12 15:46:29.130871 2014-08-13 17:38:03.008373 7fe9bae887a0 20 read_log 194215'14137 (194215'14136) delete 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.2174595.0:138570 0.000000 2014-08-13 17:38:03.008384 7fe9bae887a0 20 read_log 194215'14138 (192834'12853) clean f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by osd.17.0:2423 2014-08-12 15:46:29.138518 2014-08-13 17:38:03.008395 7fe9bae887a0 20 read_log 194215'14139 (194215'14138) delete f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.2174595.0:138572 0.000000 2014-08-13 17:38:03.008406 7fe9bae887a0 20 read_log 194215'14140 (193305'13915) clean 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by osd.17.0:2425 2014-08-12 15:46:29.144874 2014-08-13 17:38:03.008419 7fe9bae887a0 20 read_log 194215'14141 (194215'14140) delete 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.2174595.0:138574 0.000000 2014-08-13 17:38:03.008430 7fe9bae887a0 20 read_log 194215'14142 (192834'13317) clean 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by osd.17.0:2427 2014-08-12 15:46:29.153010 2014-08-13 17:38:03.008442 7fe9bae887a0 20 read_log 194215'14143 (194215'14142) delete 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.2174595.0:138576 0.000000 2014-08-13 17:38:03.008452 7fe9bae887a0 20 read_log 194215'14144 (192834'13323) clean 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by osd.17.0:2429 2014-08-12 15:46:29.159837 2014-08-13 17:38:03.008464 7fe9bae887a0 20 read_log 194215'14145 (194215'14144) delete 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.2174595.0:138578 0.000000 2014-08-13 17:38:03.008475 7fe9bae887a0 20 read_log 194215'14146 (192834'12888) clean 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by osd.17.0:2431 2014-08-12 15:46:29.167925 2014-08-13 17:38:03.008487 7fe9bae887a0 20 read_log 194215'14147 (194215'14146) delete 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.2174595.0:138580 0.000000 2014-08-13 17:38:03.008497 7fe9bae887a0 20 read_log 194215'14148 (193305'13945) clean 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by osd.17.0:2433 2014-08-12 15:46:29.174867 2014-08-13 17:38:03.008509 7fe9bae887a0 20 read_log 194215'14149 (194215'14148) delete 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.2174595.0:138582 0.000000 2014-08-13 17:38:03.008520 7fe9bae887a0 20 read_log 194215'14150 (192834'12836) clean d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by osd.17.0:2435 2014-08-12 15:46:29.182603 2014-08-13 17:38:03.008532 7fe9bae887a0 20 read_log 194215'14151 (194215'14150) delete d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.2174595.0:138584 0.000000 2014-08-13 17:38:03.008543 7fe9bae887a0 20 read_log 194215'14152 (193305'13943) clean 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by osd.17.0:2437 2014-08-12 15:46:29.191884 2014-08-13 17:38:03.008557 7fe9bae887a0 20 read_log 194215'14153 (194215'14152) delete 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.2174595.0:138586 0.000000 2014-08-13 17:38:03.008567 7fe9bae887a0 20 read_log 194215'14154 (192834'12778) clean 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by osd.17.0:2439 2014-08-12 15:46:29.200003 2014-08-13 17:38:03.008580 7fe9bae887a0 20 read_log 194215'14155 (194215'14154) delete 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.2174595.0:138588 0.000000 2014-08-13 17:38:03.008590 7fe9bae887a0 20 read_log 194215'14156 (192888'13817) clean 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by osd.17.0:2441 2014-08-12 15:46:29.207379 2014-08-13 17:38:03.008609 7fe9bae887a0 20 read_log 194215'14157 (194215'14156) delete 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.2174595.0:138590 0.000000 2014-08-13 17:38:03.008620 7fe9bae887a0 20 read_log 194215'14158 (193305'13919) clean 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by osd.17.0:2443 2014-08-12 15:46:29.214546 2014-08-13 17:38:03.008632 7fe9bae887a0 20 read_log 194215'14159 (194215'14158) delete 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.2174595.0:138592 0.000000 2014-08-13 17:38:03.008643 7fe9bae887a0 20 read_log 194215'14160 (192834'12818) clean 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by osd.17.0:2445 2014-08-12 15:46:29.222557 2014-08-13 17:38:03.008655 7fe9bae887a0 20 read_log 194215'14161 (194215'14160) delete 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.2174595.0:138594 0.000000 2014-08-13 17:38:03.008665 7fe9bae887a0 20 read_log 194215'14162 (192888'13613) clean e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by osd.17.0:2447 2014-08-12 15:46:29.229873 2014-08-13 17:38:03.008677 7fe9bae887a0 20 read_log 194215'14163 (194215'14162) delete e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.2174595.0:138596 0.000000 2014-08-13 17:38:03.008688 7fe9bae887a0 20 read_log 194215'14164 (192888'13623) clean 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by osd.17.0:2449 2014-08-12 15:46:29.237567 2014-08-13 17:38:03.008699 7fe9bae887a0 20 read_log 194215'14165 (194215'14164) delete 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.2174595.0:138598 0.000000 2014-08-13 17:38:03.008710 7fe9bae887a0 20 read_log 194215'14166 (192834'13352) clean 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by osd.17.0:2451 2014-08-12 15:46:29.246260 2014-08-13 17:38:03.008722 7fe9bae887a0 20 read_log 194215'14167 (194215'14166) delete 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.2174595.0:138600 0.000000 2014-08-13 17:38:03.008732 7fe9bae887a0 20 read_log 194215'14168 (192888'13836) clean 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by osd.17.0:2453 2014-08-12 15:46:29.254727 2014-08-13 17:38:03.008744 7fe9bae887a0 20 read_log 194215'14169 (194215'14168) delete 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.2174595.0:138602 0.000000 2014-08-13 17:38:03.008755 7fe9bae887a0 20 read_log 194215'14170 (192888'13636) clean 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by osd.17.0:2455 2014-08-12 15:46:29.262525 2014-08-13 17:38:03.008767 7fe9bae887a0 20 read_log 194215'14171 (194215'14170) delete 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.2174595.0:138604 0.000000 2014-08-13 17:38:03.008777 7fe9bae887a0 20 read_log 194215'14172 (192888'13642) clean 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by osd.17.0:2457 2014-08-12 15:46:29.270944 2014-08-13 17:38:03.008789 7fe9bae887a0 20 read_log 194215'14173 (194215'14172) delete 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.2174595.0:138606 0.000000 2014-08-13 17:38:03.008800 7fe9bae887a0 20 read_log 194215'14174 (192834'13282) clean 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by osd.17.0:2459 2014-08-12 15:46:29.278928 2014-08-13 17:38:03.008813 7fe9bae887a0 20 read_log 194215'14175 (194215'14174) delete 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.2174595.0:138608 0.000000 2014-08-13 17:38:03.008824 7fe9bae887a0 20 read_log 194215'14176 (192888'13646) clean 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by osd.17.0:2461 2014-08-12 15:46:29.286468 2014-08-13 17:38:03.008863 7fe9bae887a0 20 read_log 194215'14177 (194215'14176) delete 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.2174595.0:138610 0.000000 2014-08-13 17:38:03.008875 7fe9bae887a0 20 read_log 194215'14178 (192834'13387) clean 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by osd.17.0:2463 2014-08-12 15:46:29.294533 2014-08-13 17:38:03.008887 7fe9bae887a0 20 read_log 194215'14179 (194215'14178) delete 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.2174595.0:138612 0.000000 2014-08-13 17:38:03.008898 7fe9bae887a0 20 read_log 194215'14180 (192834'12801) clean 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by osd.17.0:2465 2014-08-12 15:46:29.303555 2014-08-13 17:38:03.008910 7fe9bae887a0 20 read_log 194215'14181 (194215'14180) delete 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.2174595.0:138614 0.000000 2014-08-13 17:38:03.008920 7fe9bae887a0 20 read_log 194215'14182 (192834'12832) clean 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by osd.17.0:2467 2014-08-12 15:46:29.311521 2014-08-13 17:38:03.008932 7fe9bae887a0 20 read_log 194215'14183 (194215'14182) delete 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.2174595.0:138616 0.000000 2014-08-13 17:38:03.008943 7fe9bae887a0 20 read_log 194215'14184 (192888'13626) clean 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by osd.17.0:2469 2014-08-12 15:46:29.319596 2014-08-13 17:38:03.008955 7fe9bae887a0 20 read_log 194215'14185 (194215'14184) delete 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.2174595.0:138618 0.000000 2014-08-13 17:38:03.008965 7fe9bae887a0 20 read_log 194215'14186 (192834'13423) clean 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by osd.17.0:2471 2014-08-12 15:46:29.327665 2014-08-13 17:38:03.008977 7fe9bae887a0 20 read_log 194215'14187 (194215'14186) delete 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.2174595.0:138620 0.000000 2014-08-13 17:38:03.008987 7fe9bae887a0 20 read_log 194215'14188 (192834'13345) clean 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by osd.17.0:2473 2014-08-12 15:46:29.335451 2014-08-13 17:38:03.008999 7fe9bae887a0 20 read_log 194215'14189 (194215'14188) delete 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.2174595.0:138622 0.000000 2014-08-13 17:38:03.009010 7fe9bae887a0 20 read_log 194215'14190 (192888'13607) clean 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by osd.17.0:2475 2014-08-12 15:46:29.343472 2014-08-13 17:38:03.009021 7fe9bae887a0 20 read_log 194215'14191 (194215'14190) delete 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.2174595.0:138624 0.000000 2014-08-13 17:38:03.009032 7fe9bae887a0 20 read_log 194215'14192 (192888'13648) clean 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by osd.17.0:2477 2014-08-12 15:46:29.350500 2014-08-13 17:38:03.009044 7fe9bae887a0 20 read_log 194215'14193 (194215'14192) delete 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.2174595.0:138626 0.000000 2014-08-13 17:38:03.009054 7fe9bae887a0 20 read_log 194215'14194 (192834'13425) clean cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by osd.17.0:2479 2014-08-12 15:46:29.357780 2014-08-13 17:38:03.009066 7fe9bae887a0 20 read_log 194215'14195 (194215'14194) delete cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.2174595.0:138628 0.000000 2014-08-13 17:38:03.009076 7fe9bae887a0 20 read_log 194215'14196 (192888'13790) clean 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by osd.17.0:2481 2014-08-12 15:46:29.365905 2014-08-13 17:38:03.009095 7fe9bae887a0 20 read_log 194215'14197 (194215'14196) delete 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.2174595.0:138630 0.000000 2014-08-13 17:38:03.009107 7fe9bae887a0 20 read_log 194215'14198 (192888'13831) clean d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by osd.17.0:2483 2014-08-12 15:46:29.373032 2014-08-13 17:38:03.009119 7fe9bae887a0 20 read_log 194215'14199 (194215'14198) delete d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.2174595.0:138632 0.000000 2014-08-13 17:38:03.009129 7fe9bae887a0 20 read_log 194215'14200 (192834'13446) clean cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by osd.17.0:2485 2014-08-12 15:46:29.380974 2014-08-13 17:38:03.009141 7fe9bae887a0 20 read_log 194215'14201 (194215'14200) delete cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.2174595.0:138634 0.000000 2014-08-13 17:38:03.009152 7fe9bae887a0 20 read_log 194215'14202 (193305'13930) clean b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by osd.17.0:2487 2014-08-12 15:46:29.388285 2014-08-13 17:38:03.009171 7fe9bae887a0 20 read_log 194215'14203 (194215'14202) delete b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.2174595.0:138636 0.000000 2014-08-13 17:38:03.009182 7fe9bae887a0 20 read_log 194215'14204 (192888'13805) clean 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by osd.17.0:2489 2014-08-12 15:46:29.396769 2014-08-13 17:38:03.009194 7fe9bae887a0 20 read_log 194215'14205 (194215'14204) delete 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.2174595.0:138638 0.000000 2014-08-13 17:38:03.009205 7fe9bae887a0 20 read_log 194215'14206 (192888'13821) clean 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by osd.17.0:2491 2014-08-12 15:46:29.404665 2014-08-13 17:38:03.009216 7fe9bae887a0 20 read_log 194215'14207 (194215'14206) delete 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.2174595.0:138640 0.000000 2014-08-13 17:38:03.009227 7fe9bae887a0 20 read_log 194215'14208 (192834'12860) clean d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by osd.17.0:2493 2014-08-12 15:46:29.412478 2014-08-13 17:38:03.009239 7fe9bae887a0 20 read_log 194215'14209 (194215'14208) delete d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.2174595.0:138642 0.000000 2014-08-13 17:38:03.009249 7fe9bae887a0 20 read_log 194215'14210 (192834'13280) clean 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by osd.17.0:2495 2014-08-12 15:46:29.419639 2014-08-13 17:38:03.009261 7fe9bae887a0 20 read_log 194215'14211 (194215'14210) delete 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.2174595.0:138644 0.000000 2014-08-13 17:38:03.009272 7fe9bae887a0 20 read_log 194215'14212 (192888'13841) clean fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by osd.17.0:2497 2014-08-12 15:46:29.427988 2014-08-13 17:38:03.009284 7fe9bae887a0 20 read_log 194215'14213 (194215'14212) delete fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.2174595.0:138646 0.000000 2014-08-13 17:38:03.009294 7fe9bae887a0 20 read_log 194215'14214 (193305'13938) clean 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by osd.17.0:2499 2014-08-12 15:46:29.437909 2014-08-13 17:38:03.009314 7fe9bae887a0 20 read_log 194215'14215 (194215'14214) delete 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.2174595.0:138648 0.000000 2014-08-13 17:38:03.009341 7fe9bae887a0 20 read_log 194215'14216 (192888'13830) clean 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by osd.17.0:2501 2014-08-12 15:46:29.446323 2014-08-13 17:38:03.009361 7fe9bae887a0 20 read_log 194215'14217 (194215'14216) delete 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.2174595.0:138650 0.000000 2014-08-13 17:38:03.009372 7fe9bae887a0 20 read_log 194215'14218 (192888'13621) clean 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by osd.17.0:2503 2014-08-12 15:46:29.454128 2014-08-13 17:38:03.009384 7fe9bae887a0 20 read_log 194215'14219 (194215'14218) delete 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.2174595.0:138652 0.000000 2014-08-13 17:38:03.009395 7fe9bae887a0 20 read_log 194215'14220 (192834'13321) clean 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by osd.17.0:2505 2014-08-12 15:46:29.461270 2014-08-13 17:38:03.009407 7fe9bae887a0 20 read_log 194215'14221 (194215'14220) delete 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.2174595.0:138654 0.000000 2014-08-13 17:38:03.009417 7fe9bae887a0 20 read_log 194215'14222 (193305'13934) clean 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by osd.17.0:2507 2014-08-12 15:46:29.470272 2014-08-13 17:38:03.009429 7fe9bae887a0 20 read_log 194215'14223 (194215'14222) delete 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.2174595.0:138656 0.000000 2014-08-13 17:38:03.009440 7fe9bae887a0 20 read_log 194215'14224 (192888'13804) clean a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by osd.17.0:2509 2014-08-12 15:46:29.476973 2014-08-13 17:38:03.009470 7fe9bae887a0 20 read_log 194215'14225 (194215'14224) delete a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.2174595.0:138658 0.000000 2014-08-13 17:38:03.009482 7fe9bae887a0 20 read_log 194215'14226 (192834'12806) clean 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by osd.17.0:2511 2014-08-12 15:46:29.483962 2014-08-13 17:38:03.009494 7fe9bae887a0 20 read_log 194215'14227 (194215'14226) delete 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.2174595.0:138660 0.000000 2014-08-13 17:38:03.009505 7fe9bae887a0 20 read_log 194215'14228 (193305'13932) clean 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by osd.17.0:2513 2014-08-12 15:46:29.491554 2014-08-13 17:38:03.009517 7fe9bae887a0 20 read_log 194215'14229 (194215'14228) delete 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.2174595.0:138662 0.000000 2014-08-13 17:38:03.009528 7fe9bae887a0 20 read_log 194215'14230 (192888'13802) clean 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by osd.17.0:2515 2014-08-12 15:46:29.498537 2014-08-13 17:38:03.009540 7fe9bae887a0 20 read_log 194215'14231 (194215'14230) delete 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.2174595.0:138664 0.000000 2014-08-13 17:38:03.009550 7fe9bae887a0 20 read_log 194215'14232 (192888'13798) clean b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by osd.17.0:2517 2014-08-12 15:46:29.506305 2014-08-13 17:38:03.009562 7fe9bae887a0 20 read_log 194215'14233 (194215'14232) delete b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.2174595.0:138666 0.000000 2014-08-13 17:38:03.009573 7fe9bae887a0 20 read_log 194215'14234 (193305'13925) clean 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by osd.17.0:2519 2014-08-12 15:46:29.514597 2014-08-13 17:38:03.009585 7fe9bae887a0 20 read_log 194215'14235 (194215'14234) delete 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.2174595.0:138668 0.000000 2014-08-13 17:38:03.009595 7fe9bae887a0 20 read_log 194215'14236 (192834'12766) clean 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by osd.17.0:2521 2014-08-12 15:46:29.521884 2014-08-13 17:38:03.009617 7fe9bae887a0 20 read_log 194215'14237 (194215'14236) delete 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.2174595.0:138670 0.000000 2014-08-13 17:38:03.009629 7fe9bae887a0 20 read_log 194215'14238 (193305'13914) clean 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by osd.17.0:2523 2014-08-12 15:46:29.529893 2014-08-13 17:38:03.009641 7fe9bae887a0 20 read_log 194215'14239 (194215'14238) delete 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.2174595.0:138672 0.000000 2014-08-13 17:38:03.009652 7fe9bae887a0 20 read_log 194215'14240 (192888'13669) clean 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by osd.17.0:2525 2014-08-12 15:46:29.537921 2014-08-13 17:38:03.009664 7fe9bae887a0 20 read_log 194215'14241 (194215'14240) delete 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.2174595.0:138674 0.000000 2014-08-13 17:38:03.009674 7fe9bae887a0 20 read_log 194215'14242 (193305'13940) clean 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by osd.17.0:2527 2014-08-12 15:46:29.544912 2014-08-13 17:38:03.009686 7fe9bae887a0 20 read_log 194215'14243 (194215'14242) delete 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.2174595.0:138676 0.000000 2014-08-13 17:38:03.009696 7fe9bae887a0 20 read_log 194215'14244 (192888'13815) clean 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by osd.17.0:2529 2014-08-12 15:46:29.551574 2014-08-13 17:38:03.009708 7fe9bae887a0 20 read_log 194215'14245 (194215'14244) delete 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.2174595.0:138678 0.000000 2014-08-13 17:38:03.009719 7fe9bae887a0 20 read_log 194215'14246 (193305'13939) clean aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by osd.17.0:2531 2014-08-12 15:46:29.559693 2014-08-13 17:38:03.009730 7fe9bae887a0 20 read_log 194215'14247 (194215'14246) delete aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.2174595.0:138680 0.000000 2014-08-13 17:38:03.009741 7fe9bae887a0 20 read_log 194215'14248 (192888'13620) clean 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by osd.17.0:2533 2014-08-12 15:46:29.565482 2014-08-13 17:38:03.009753 7fe9bae887a0 20 read_log 194215'14249 (194215'14248) delete 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.2174595.0:138682 0.000000 2014-08-13 17:38:03.009763 7fe9bae887a0 20 read_log 194215'14250 (194215'14250) modify 2a1/hit_set_24.2a1_archive_2014-08-12 15:46:28.834066_2014-08-12 17:24:40.103729/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.104530 2014-08-13 17:38:03.009776 7fe9bae887a0 20 read_log 194215'14251 (194215'14060) delete 2a1/hit_set_24.2a1_archive_2014-08-12 15:32:25.911312_2014-08-12 15:46:28.833325/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.104530 2014-08-13 17:38:03.009788 7fe9bae887a0 20 read_log 194215'14252 (0'0) promote 2829e2a1/benchmark_data_storage0111-ib.pocstor_34628_object82/head//24 by osd.17.0:5118 2014-08-12 17:24:40.109282 2014-08-13 17:38:03.009800 7fe9bae887a0 20 read_log 194215'14253 (194215'14252) modify 2829e2a1/benchmark_data_storage0111-ib.pocstor_34628_object82/head//24 by client.2193759.0:83 2014-08-12 17:24:40.096720 2014-08-13 17:38:03.009812 7fe9bae887a0 20 read_log 194215'14254 (0'0) promote a67eaea1/benchmark_data_storage0111-ib.pocstor_34628_object148/head//24 by osd.17.0:5120 2014-08-12 17:24:40.423123 2014-08-13 17:38:03.009826 7fe9bae887a0 20 read_log 194215'14255 (194215'14254) modify a67eaea1/benchmark_data_storage0111-ib.pocstor_34628_object148/head//24 by client.2193759.0:149 2014-08-12 17:24:40.415934 2014-08-13 17:38:03.009845 7fe9bae887a0 20 read_log 194370'14256 (0'0) promote 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by osd.17.0:5164 2014-08-12 20:30:14.431722 2014-08-13 17:38:03.009859 7fe9bae887a0 20 read_log 194370'14257 (194370'14256) modify 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by client.1508531.0:90 2014-08-12 20:30:14.423052 2014-08-13 17:38:03.009871 7fe9bae887a0 20 read_log 194370'14258 (0'0) promote e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by osd.17.0:5190 2014-08-12 20:30:19.248604 2014-08-13 17:38:03.009883 7fe9bae887a0 20 read_log 194370'14259 (194370'14258) modify e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by client.1508531.0:1295 2014-08-12 20:30:19.241079 2014-08-13 17:38:03.009895 7fe9bae887a0 20 read_log 194370'14260 (0'0) promote c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by osd.17.0:5202 2014-08-12 20:30:23.347203 2014-08-13 17:38:03.009907 7fe9bae887a0 20 read_log 194370'14261 (194370'14260) modify c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by client.1508531.0:2394 2014-08-12 20:30:23.338742 2014-08-13 17:38:03.009919 7fe9bae887a0 20 read_log 194370'14262 (0'0) promote 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by osd.17.0:5206 2014-08-12 20:30:23.625447 2014-08-13 17:38:03.009931 7fe9bae887a0 20 read_log 194370'14263 (194370'14262) modify 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by client.1508531.0:2471 2014-08-12 20:30:23.619503 2014-08-13 17:38:03.009942 7fe9bae887a0 20 read_log 194370'14264 (0'0) promote f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by osd.17.0:5212 2014-08-12 20:30:23.743929 2014-08-13 17:38:03.009954 7fe9bae887a0 20 read_log 194370'14265 (194370'14264) modify f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by client.1508531.0:2503 2014-08-12 20:30:23.736973 2014-08-13 17:38:03.009966 7fe9bae887a0 20 read_log 194370'14266 (0'0) promote 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by osd.17.0:5236 2014-08-12 20:30:27.039928 2014-08-13 17:38:03.009978 7fe9bae887a0 20 read_log 194370'14267 (194370'14266) modify 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by client.1508531.0:3407 2014-08-12 20:30:27.030616 2014-08-13 17:38:03.009990 7fe9bae887a0 20 read_log 194370'14268 (0'0) promote 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by osd.17.0:5238 2014-08-12 20:30:27.191047 2014-08-13 17:38:03.010002 7fe9bae887a0 20 read_log 194370'14269 (194370'14268) modify 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by client.1508531.0:3446 2014-08-12 20:30:27.183554 2014-08-13 17:38:03.010013 7fe9bae887a0 20 read_log 194370'14270 (0'0) promote 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by osd.17.0:5250 2014-08-12 20:30:29.308269 2014-08-13 17:38:03.010025 7fe9bae887a0 20 read_log 194370'14271 (194370'14270) modify 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by client.1508531.0:4003 2014-08-12 20:30:29.299649 2014-08-13 17:38:03.010037 7fe9bae887a0 20 read_log 194370'14272 (0'0) promote 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by osd.17.0:5252 2014-08-12 20:30:29.835072 2014-08-13 17:38:03.010049 7fe9bae887a0 20 read_log 194370'14273 (194370'14272) modify 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by client.1508531.0:4144 2014-08-12 20:30:29.826139 2014-08-13 17:38:03.010061 7fe9bae887a0 20 read_log 194370'14274 (0'0) promote c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by osd.17.0:5256 2014-08-12 20:30:30.112833 2014-08-13 17:38:03.010072 7fe9bae887a0 20 read_log 194370'14275 (194370'14274) modify c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by client.1508531.0:4204 2014-08-12 20:30:30.106220 2014-08-13 17:38:03.010092 7fe9bae887a0 20 read_log 194370'14276 (0'0) promote cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by osd.17.0:5308 2014-08-12 20:30:44.431443 2014-08-13 17:38:03.010104 7fe9bae887a0 20 read_log 194370'14277 (194370'14276) modify cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by client.1508531.0:7413 2014-08-12 20:30:44.421622 2014-08-13 17:38:03.010116 7fe9bae887a0 20 read_log 194370'14278 (0'0) promote f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by osd.17.0:5310 2014-08-12 20:30:46.538344 2014-08-13 17:38:03.010128 7fe9bae887a0 20 read_log 194370'14279 (194370'14278) modify f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by client.1508531.0:7803 2014-08-12 20:30:46.527237 2014-08-13 17:38:03.010140 7fe9bae887a0 20 read_log 194370'14280 (0'0) promote 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by osd.17.0:5338 2014-08-12 20:30:56.142438 2014-08-13 17:38:03.010151 7fe9bae887a0 20 read_log 194370'14281 (194370'14280) modify 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by client.1508531.0:9470 2014-08-12 20:30:56.134566 2014-08-13 17:38:03.010168 7fe9bae887a0 20 read_log 194370'14282 (0'0) promote d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by osd.17.0:5342 2014-08-12 20:30:56.610566 2014-08-13 17:38:03.010181 7fe9bae887a0 20 read_log 194370'14283 (194370'14282) modify d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by client.1508531.0:9549 2014-08-12 20:30:56.603705 2014-08-13 17:38:03.010193 7fe9bae887a0 20 read_log 194370'14284 (0'0) promote f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by osd.17.0:5374 2014-08-12 20:31:05.874503 2014-08-13 17:38:03.010205 7fe9bae887a0 20 read_log 194370'14285 (194370'14284) modify f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by client.1508531.0:11098 2014-08-12 20:31:05.865726 2014-08-13 17:38:03.010217 7fe9bae887a0 20 read_log 194370'14286 (0'0) promote f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by osd.17.0:5380 2014-08-12 20:31:07.367077 2014-08-13 17:38:03.010231 7fe9bae887a0 20 read_log 194370'14287 (194370'14286) modify f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by client.1508531.0:11343 2014-08-12 20:31:07.357266 2014-08-13 17:38:03.010244 7fe9bae887a0 20 read_log 194377'14288 (0'0) promote f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by osd.17.0:5410 2014-08-12 20:32:59.631872 2014-08-13 17:38:03.010256 7fe9bae887a0 20 read_log 194377'14289 (194377'14288) modify f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by client.2231190.0:562 2014-08-12 20:32:59.614726 2014-08-13 17:38:03.010268 7fe9bae887a0 20 read_log 194377'14290 (0'0) promote 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by osd.17.0:5434 2014-08-12 20:33:04.294938 2014-08-13 17:38:03.010280 7fe9bae887a0 20 read_log 194377'14291 (194377'14290) modify 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by client.2231190.0:1767 2014-08-12 20:33:04.287040 2014-08-13 17:38:03.010292 7fe9bae887a0 20 read_log 194377'14292 (0'0) promote 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by osd.17.0:5456 2014-08-12 20:33:08.917726 2014-08-13 17:38:03.010304 7fe9bae887a0 20 read_log 194377'14293 (194377'14292) modify 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by client.2231190.0:2909 2014-08-12 20:33:08.909638 2014-08-13 17:38:03.010316 7fe9bae887a0 20 read_log 194383'14294 (0'0) promote 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by osd.17.0:5536 2014-08-12 20:34:02.111147 2014-08-13 17:38:03.010336 7fe9bae887a0 20 read_log 194383'14295 (194383'14294) modify 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by client.2231190.0:6136 2014-08-12 20:34:02.103724 2014-08-13 17:38:03.010349 7fe9bae887a0 20 read_log 194393'14296 (0'0) promote 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by osd.17.0:5592 2014-08-12 20:47:27.899691 2014-08-13 17:38:03.010361 7fe9bae887a0 20 read_log 194393'14297 (194393'14296) modify 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by client.1064095.0:463 2014-08-12 20:47:27.892264 2014-08-13 17:38:03.010373 7fe9bae887a0 20 read_log 194393'14298 (0'0) promote 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by osd.17.0:5594 2014-08-12 20:49:26.683858 2014-08-13 17:38:03.010385 7fe9bae887a0 20 read_log 194393'14299 (194393'14298) modify 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by client.1064182.0:257 2014-08-12 20:49:26.673351 2014-08-13 17:38:03.010397 7fe9bae887a0 20 read_log 194393'14300 (0'0) promote 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by osd.17.0:5605 2014-08-12 20:52:33.500232 2014-08-13 17:38:03.010409 7fe9bae887a0 20 read_log 194393'14301 (194393'14300) modify 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by client.1064422.0:239 2014-08-12 20:52:33.493329 2014-08-13 17:38:03.010423 7fe9bae887a0 20 read_log 194397'14302 (0'0) promote 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by osd.17.0:5615 2014-08-12 20:53:03.782624 2014-08-13 17:38:03.010435 7fe9bae887a0 20 read_log 194397'14303 (194397'14302) modify 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by client.2235069.0:816 2014-08-12 20:53:03.772632 2014-08-13 17:38:03.010448 7fe9bae887a0 20 read_log 194416'14304 (0'0) promote 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by osd.17.0:5757 2014-08-13 15:28:31.743571 2014-08-13 17:38:03.010460 7fe9bae887a0 20 read_log 194416'14305 (194416'14304) modify 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by client.2441958.0:363 2014-08-13 15:28:31.734296 2014-08-13 17:38:03.010472 7fe9bae887a0 20 read_log 194416'14306 (0'0) promote 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by osd.17.0:5759 2014-08-13 15:28:32.066502 2014-08-13 17:38:03.010484 7fe9bae887a0 20 read_log 194416'14307 (194416'14306) modify 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by client.2441958.0:429 2014-08-13 15:28:32.058547 2014-08-13 17:38:03.010496 7fe9bae887a0 20 read_log 194416'14308 (0'0) promote bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by osd.17.0:5761 2014-08-13 15:28:32.728977 2014-08-13 17:38:03.010529 7fe9bae887a0 20 read_log 194416'14309 (194416'14308) modify bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by client.2441958.0:562 2014-08-13 15:28:32.721495 2014-08-13 17:38:03.010542 7fe9bae887a0 20 read_log 194416'14310 (0'0) promote 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by osd.17.0:5778 2014-08-13 15:28:34.932032 2014-08-13 17:38:03.010554 7fe9bae887a0 20 read_log 194416'14311 (194416'14310) modify 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by client.2441958.0:1110 2014-08-13 15:28:34.924839 2014-08-13 17:38:03.010566 7fe9bae887a0 20 read_log 194416'14312 (0'0) promote 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by osd.17.0:5791 2014-08-13 15:28:39.169035 2014-08-13 17:38:03.010578 7fe9bae887a0 20 read_log 194416'14313 (194416'14312) modify 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by client.2441958.0:2236 2014-08-13 15:28:39.160468 2014-08-13 17:38:03.010617 7fe9bae887a0 20 read_log 194416'14314 (0'0) promote b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by osd.17.0:5797 2014-08-13 15:28:40.247706 2014-08-13 17:38:03.010630 7fe9bae887a0 20 read_log 194416'14315 (194416'14314) modify b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by client.2441958.0:2517 2014-08-13 15:28:40.238988 2014-08-13 17:38:03.010642 7fe9bae887a0 20 read_log 194416'14316 (0'0) promote afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by osd.17.0:5799 2014-08-13 15:28:41.627499 2014-08-13 17:38:03.010654 7fe9bae887a0 20 read_log 194416'14317 (194416'14316) modify afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by client.2441958.0:2880 2014-08-13 15:28:41.620853 2014-08-13 17:38:03.010666 7fe9bae887a0 20 read_log 194416'14318 (0'0) promote 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by osd.17.0:5863 2014-08-13 15:29:06.699626 2014-08-13 17:38:03.010678 7fe9bae887a0 20 read_log 194416'14319 (194416'14318) modify 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by client.2441958.0:9320 2014-08-13 15:29:06.690738 2014-08-13 17:38:03.010690 7fe9bae887a0 20 read_log 194416'14320 (0'0) promote 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by osd.17.0:5885 2014-08-13 15:29:11.212498 2014-08-13 17:38:03.010702 7fe9bae887a0 20 read_log 194416'14321 (194416'14320) modify 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by client.2441958.0:10440 2014-08-13 15:29:11.205144 2014-08-13 17:38:03.010714 7fe9bae887a0 20 read_log 194416'14322 (0'0) promote b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by osd.17.0:5905 2014-08-13 15:29:16.096567 2014-08-13 17:38:03.010726 7fe9bae887a0 20 read_log 194416'14323 (194416'14322) modify b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by client.2441958.0:11646 2014-08-13 15:29:16.086396 2014-08-13 17:38:03.010738 7fe9bae887a0 20 read_log 194416'14324 (0'0) promote 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by osd.17.0:5933 2014-08-13 15:29:23.189638 2014-08-13 17:38:03.010750 7fe9bae887a0 20 read_log 194416'14325 (194416'14324) modify 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by client.2441958.0:13280 2014-08-13 15:29:23.179367 2014-08-13 17:38:03.010762 7fe9bae887a0 20 read_log 194416'14326 (0'0) promote d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by osd.17.0:5935 2014-08-13 15:29:24.233138 2014-08-13 17:38:03.010774 7fe9bae887a0 20 read_log 194416'14327 (194416'14326) modify d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by client.2441958.0:13528 2014-08-13 15:29:24.226232 2014-08-13 17:38:03.010786 7fe9bae887a0 20 read_log 194416'14328 (0'0) promote ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by osd.17.0:5947 2014-08-13 15:29:25.275428 2014-08-13 17:38:03.010798 7fe9bae887a0 20 read_log 194416'14329 (194416'14328) modify ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by client.2441958.0:13765 2014-08-13 15:29:25.265313 2014-08-13 17:38:03.010809 7fe9bae887a0 20 read_log 194416'14330 (0'0) promote e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by osd.17.0:5963 2014-08-13 15:29:28.306818 2014-08-13 17:38:03.010821 7fe9bae887a0 20 read_log 194416'14331 (194416'14330) modify e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by client.2441958.0:14523 2014-08-13 15:29:28.297506 2014-08-13 17:38:03.010834 7fe9bae887a0 20 read_log 194416'14332 (0'0) promote 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by osd.17.0:5981 2014-08-13 15:29:31.158489 2014-08-13 17:38:03.010845 7fe9bae887a0 20 read_log 194416'14333 (194416'14332) modify 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by client.2441958.0:15185 2014-08-13 15:29:31.148722 2014-08-13 17:38:03.010864 7fe9bae887a0 20 read_log 194416'14334 (0'0) promote 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by osd.17.0:6025 2014-08-13 15:30:12.366117 2014-08-13 17:38:03.010877 7fe9bae887a0 20 read_log 194416'14335 (194416'14334) modify 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by client.2441958.0:18473 2014-08-13 15:30:12.358498 2014-08-13 17:38:03.010889 7fe9bae887a0 20 read_log 194422'14336 (0'0) promote ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by osd.17.0:6047 2014-08-13 15:32:19.796684 2014-08-13 17:38:03.010901 7fe9bae887a0 20 read_log 194422'14337 (194422'14336) modify ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by client.1148203.0:1132 2014-08-13 15:32:19.788514 2014-08-13 17:38:03.010913 7fe9bae887a0 20 read_log 194422'14338 (0'0) promote baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by osd.17.0:6051 2014-08-13 15:32:20.401675 2014-08-13 17:38:03.010925 7fe9bae887a0 20 read_log 194422'14339 (194422'14338) modify baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by client.1148203.0:1278 2014-08-13 15:32:20.393588 2014-08-13 17:38:03.010937 7fe9bae887a0 20 read_log 194422'14340 (0'0) promote 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by osd.17.0:6079 2014-08-13 15:32:29.365901 2014-08-13 17:38:03.010948 7fe9bae887a0 20 read_log 194422'14341 (194422'14340) modify 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by client.1148203.0:3569 2014-08-13 15:32:29.356032 2014-08-13 17:38:03.010960 7fe9bae887a0 20 read_log 194422'14342 (0'0) promote b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by osd.17.0:6091 2014-08-13 15:32:29.903126 2014-08-13 17:38:03.010972 7fe9bae887a0 20 read_log 194422'14343 (194422'14342) modify b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by client.1148203.0:3708 2014-08-13 15:32:29.891182 2014-08-13 17:38:03.010984 7fe9bae887a0 20 read_log 194422'14344 (0'0) promote 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by osd.17.0:6115 2014-08-13 15:32:34.165788 2014-08-13 17:38:03.010995 7fe9bae887a0 20 read_log 194422'14345 (194422'14344) modify 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by client.1148203.0:4823 2014-08-13 15:32:34.155051 2014-08-13 17:38:03.011007 7fe9bae887a0 20 read_log 194422'14346 (0'0) promote 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by osd.17.0:6139 2014-08-13 15:32:40.804449 2014-08-13 17:38:03.011019 7fe9bae887a0 20 read_log 194422'14347 (194422'14346) modify 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by client.1148203.0:6124 2014-08-13 15:32:40.795858 2014-08-13 17:38:03.011031 7fe9bae887a0 20 read_log 194422'14348 (0'0) promote 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by osd.17.0:6195 2014-08-13 15:33:08.620257 2014-08-13 17:38:03.011042 7fe9bae887a0 20 read_log 194422'14349 (194422'14348) modify 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by client.1148203.0:9853 2014-08-13 15:33:08.610416 2014-08-13 17:38:03.011054 7fe9bae887a0 20 read_log 194422'14350 (0'0) promote 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by osd.17.0:6197 2014-08-13 15:33:10.060667 2014-08-13 17:38:03.011066 7fe9bae887a0 20 read_log 194422'14351 (194422'14350) modify 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by client.1148203.0:9957 2014-08-13 15:33:10.052128 2014-08-13 17:38:03.011078 7fe9bae887a0 20 read_log 194422'14352 (0'0) promote a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by osd.17.0:6209 2014-08-13 15:33:25.680868 2014-08-13 17:38:03.011090 7fe9bae887a0 20 read_log 194422'14353 (194422'14352) modify a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by client.1148203.0:10562 2014-08-13 15:33:25.671849 2014-08-13 17:38:03.011106 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.011573 7fe9bae887a0 10 read_log checking for missing items over interval (194370'14256,194422'14353] 2014-08-13 17:38:03.011619 7fe9bae887a0 15 read_log missing 194422'14353 (194422'14352) modify a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by client.1148203.0:10562 2014-08-13 15:33:25.671849 2014-08-13 17:38:03.011658 7fe9bae887a0 15 read_log missing 194422'14351 (194422'14350) modify 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by client.1148203.0:9957 2014-08-13 15:33:10.052128 2014-08-13 17:38:03.011696 7fe9bae887a0 15 read_log missing 194422'14349 (194422'14348) modify 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by client.1148203.0:9853 2014-08-13 15:33:08.610416 2014-08-13 17:38:03.011731 7fe9bae887a0 15 read_log missing 194422'14347 (194422'14346) modify 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by client.1148203.0:6124 2014-08-13 15:32:40.795858 2014-08-13 17:38:03.011766 7fe9bae887a0 15 read_log missing 194422'14345 (194422'14344) modify 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by client.1148203.0:4823 2014-08-13 15:32:34.155051 2014-08-13 17:38:03.011801 7fe9bae887a0 15 read_log missing 194422'14343 (194422'14342) modify b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by client.1148203.0:3708 2014-08-13 15:32:29.891182 2014-08-13 17:38:03.011835 7fe9bae887a0 15 read_log missing 194422'14341 (194422'14340) modify 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by client.1148203.0:3569 2014-08-13 15:32:29.356032 2014-08-13 17:38:03.011869 7fe9bae887a0 15 read_log missing 194422'14339 (194422'14338) modify baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by client.1148203.0:1278 2014-08-13 15:32:20.393588 2014-08-13 17:38:03.011903 7fe9bae887a0 15 read_log missing 194422'14337 (194422'14336) modify ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by client.1148203.0:1132 2014-08-13 15:32:19.788514 2014-08-13 17:38:03.011938 7fe9bae887a0 15 read_log missing 194416'14335 (194416'14334) modify 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by client.2441958.0:18473 2014-08-13 15:30:12.358498 2014-08-13 17:38:03.011973 7fe9bae887a0 15 read_log missing 194416'14333 (194416'14332) modify 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by client.2441958.0:15185 2014-08-13 15:29:31.148722 2014-08-13 17:38:03.012008 7fe9bae887a0 15 read_log missing 194416'14331 (194416'14330) modify e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by client.2441958.0:14523 2014-08-13 15:29:28.297506 2014-08-13 17:38:03.012045 7fe9bae887a0 15 read_log missing 194416'14329 (194416'14328) modify ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by client.2441958.0:13765 2014-08-13 15:29:25.265313 2014-08-13 17:38:03.012080 7fe9bae887a0 15 read_log missing 194416'14327 (194416'14326) modify d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by client.2441958.0:13528 2014-08-13 15:29:24.226232 2014-08-13 17:38:03.012115 7fe9bae887a0 15 read_log missing 194416'14325 (194416'14324) modify 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by client.2441958.0:13280 2014-08-13 15:29:23.179367 2014-08-13 17:38:03.012150 7fe9bae887a0 15 read_log missing 194416'14323 (194416'14322) modify b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by client.2441958.0:11646 2014-08-13 15:29:16.086396 2014-08-13 17:38:03.012191 7fe9bae887a0 15 read_log missing 194416'14321 (194416'14320) modify 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by client.2441958.0:10440 2014-08-13 15:29:11.205144 2014-08-13 17:38:03.012231 7fe9bae887a0 15 read_log missing 194416'14319 (194416'14318) modify 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by client.2441958.0:9320 2014-08-13 15:29:06.690738 2014-08-13 17:38:03.012268 7fe9bae887a0 15 read_log missing 194416'14317 (194416'14316) modify afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by client.2441958.0:2880 2014-08-13 15:28:41.620853 2014-08-13 17:38:03.012304 7fe9bae887a0 15 read_log missing 194416'14315 (194416'14314) modify b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by client.2441958.0:2517 2014-08-13 15:28:40.238988 2014-08-13 17:38:03.012341 7fe9bae887a0 15 read_log missing 194416'14313 (194416'14312) modify 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by client.2441958.0:2236 2014-08-13 15:28:39.160468 2014-08-13 17:38:03.012377 7fe9bae887a0 15 read_log missing 194416'14311 (194416'14310) modify 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by client.2441958.0:1110 2014-08-13 15:28:34.924839 2014-08-13 17:38:03.012414 7fe9bae887a0 15 read_log missing 194416'14309 (194416'14308) modify bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by client.2441958.0:562 2014-08-13 15:28:32.721495 2014-08-13 17:38:03.012451 7fe9bae887a0 15 read_log missing 194416'14307 (194416'14306) modify 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by client.2441958.0:429 2014-08-13 15:28:32.058547 2014-08-13 17:38:03.012488 7fe9bae887a0 15 read_log missing 194416'14305 (194416'14304) modify 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by client.2441958.0:363 2014-08-13 15:28:31.734296 2014-08-13 17:38:03.012525 7fe9bae887a0 15 read_log missing 194397'14303 (194397'14302) modify 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by client.2235069.0:816 2014-08-12 20:53:03.772632 2014-08-13 17:38:03.012567 7fe9bae887a0 15 read_log missing 194393'14301 (194393'14300) modify 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by client.1064422.0:239 2014-08-12 20:52:33.493329 2014-08-13 17:38:03.012604 7fe9bae887a0 15 read_log missing 194393'14299 (194393'14298) modify 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by client.1064182.0:257 2014-08-12 20:49:26.673351 2014-08-13 17:38:03.012641 7fe9bae887a0 15 read_log missing 194393'14297 (194393'14296) modify 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by client.1064095.0:463 2014-08-12 20:47:27.892264 2014-08-13 17:38:03.012677 7fe9bae887a0 15 read_log missing 194383'14295 (194383'14294) modify 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by client.2231190.0:6136 2014-08-12 20:34:02.103724 2014-08-13 17:38:03.012714 7fe9bae887a0 15 read_log missing 194377'14293 (194377'14292) modify 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by client.2231190.0:2909 2014-08-12 20:33:08.909638 2014-08-13 17:38:03.012750 7fe9bae887a0 15 read_log missing 194377'14291 (194377'14290) modify 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by client.2231190.0:1767 2014-08-12 20:33:04.287040 2014-08-13 17:38:03.012787 7fe9bae887a0 15 read_log missing 194377'14289 (194377'14288) modify f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by client.2231190.0:562 2014-08-12 20:32:59.614726 2014-08-13 17:38:03.012824 7fe9bae887a0 15 read_log missing 194370'14287 (194370'14286) modify f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by client.1508531.0:11343 2014-08-12 20:31:07.357266 2014-08-13 17:38:03.012861 7fe9bae887a0 15 read_log missing 194370'14285 (194370'14284) modify f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by client.1508531.0:11098 2014-08-12 20:31:05.865726 2014-08-13 17:38:03.012898 7fe9bae887a0 15 read_log missing 194370'14283 (194370'14282) modify d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by client.1508531.0:9549 2014-08-12 20:30:56.603705 2014-08-13 17:38:03.012935 7fe9bae887a0 15 read_log missing 194370'14281 (194370'14280) modify 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by client.1508531.0:9470 2014-08-12 20:30:56.134566 2014-08-13 17:38:03.012972 7fe9bae887a0 15 read_log missing 194370'14279 (194370'14278) modify f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by client.1508531.0:7803 2014-08-12 20:30:46.527237 2014-08-13 17:38:03.013008 7fe9bae887a0 15 read_log missing 194370'14277 (194370'14276) modify cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by client.1508531.0:7413 2014-08-12 20:30:44.421622 2014-08-13 17:38:03.013044 7fe9bae887a0 15 read_log missing 194370'14275 (194370'14274) modify c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by client.1508531.0:4204 2014-08-12 20:30:30.106220 2014-08-13 17:38:03.013081 7fe9bae887a0 15 read_log missing 194370'14273 (194370'14272) modify 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by client.1508531.0:4144 2014-08-12 20:30:29.826139 2014-08-13 17:38:03.013118 7fe9bae887a0 15 read_log missing 194370'14271 (194370'14270) modify 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by client.1508531.0:4003 2014-08-12 20:30:29.299649 2014-08-13 17:38:03.013161 7fe9bae887a0 15 read_log missing 194370'14269 (194370'14268) modify 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by client.1508531.0:3446 2014-08-12 20:30:27.183554 2014-08-13 17:38:03.013200 7fe9bae887a0 15 read_log missing 194370'14267 (194370'14266) modify 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by client.1508531.0:3407 2014-08-12 20:30:27.030616 2014-08-13 17:38:03.013237 7fe9bae887a0 15 read_log missing 194370'14265 (194370'14264) modify f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by client.1508531.0:2503 2014-08-12 20:30:23.736973 2014-08-13 17:38:03.013273 7fe9bae887a0 15 read_log missing 194370'14263 (194370'14262) modify 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by client.1508531.0:2471 2014-08-12 20:30:23.619503 2014-08-13 17:38:03.013310 7fe9bae887a0 15 read_log missing 194370'14261 (194370'14260) modify c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by client.1508531.0:2394 2014-08-12 20:30:23.338742 2014-08-13 17:38:03.013347 7fe9bae887a0 15 read_log missing 194370'14259 (194370'14258) modify e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by client.1508531.0:1295 2014-08-12 20:30:19.241079 2014-08-13 17:38:03.013383 7fe9bae887a0 15 read_log missing 194370'14257 (194370'14256) modify 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by client.1508531.0:90 2014-08-12 20:30:14.423052 2014-08-13 17:38:03.013394 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.013420 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=0 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive m=49] handle_loaded 2014-08-13 17:38:03.013435 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=0 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.013664 0 0.000000 2014-08-13 17:38:03.013447 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=0 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset 2014-08-13 17:38:03.013455 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=0 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194477 2014-08-13 17:38:03.013465 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log((192718'11272,194422'14353], crt=194422'14353) 2014-08-13 17:38:03.013474 7fe9bae887a0 10 osd.4 194478 pgid 24.2ec coll 24.2ec_head 2014-08-13 17:38:03.013585 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.2ec 2014-08-13 17:38:03.013597 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.013627 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2ec(unlocked)] enter Initial 2014-08-13 17:38:03.013638 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.2ec(unlocked)] enter NotTrimming 2014-08-13 17:38:03.013769 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.013885 7fe9bae887a0 20 read_log 192834'14496 (192834'14495) modify 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.933448.0:649 2014-08-11 15:23:38.123120 2014-08-13 17:38:03.013907 7fe9bae887a0 20 read_log 192834'14501 (0'0) promote fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by osd.361.0:31390 2014-08-11 15:23:41.382364 2014-08-13 17:38:03.013921 7fe9bae887a0 20 read_log 192834'14502 (192834'14501) modify fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.933448.0:1475 2014-08-11 15:23:41.373562 2014-08-13 17:38:03.013933 7fe9bae887a0 20 read_log 192834'14509 (0'0) promote 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by osd.361.0:31412 2014-08-11 15:23:43.535180 2014-08-13 17:38:03.013945 7fe9bae887a0 20 read_log 192834'14510 (192834'14509) modify 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.933448.0:1884 2014-08-11 15:23:43.527854 2014-08-13 17:38:03.013958 7fe9bae887a0 20 read_log 192834'14517 (0'0) promote b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.361.0:31426 2014-08-11 15:23:46.032690 2014-08-13 17:38:03.013979 7fe9bae887a0 20 read_log 192834'14518 (192834'14517) modify b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by client.933448.0:2280 2014-08-11 15:23:46.022163 2014-08-13 17:38:03.013995 7fe9bae887a0 20 read_log 192834'14519 (0'0) promote 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.361.0:31428 2014-08-11 15:23:46.269969 2014-08-13 17:38:03.014007 7fe9bae887a0 20 read_log 192834'14520 (192834'14519) modify 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by client.933448.0:2331 2014-08-11 15:23:46.262640 2014-08-13 17:38:03.014019 7fe9bae887a0 20 read_log 192834'14529 (0'0) promote 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by osd.361.0:31448 2014-08-11 15:23:48.986050 2014-08-13 17:38:03.014031 7fe9bae887a0 20 read_log 192834'14530 (192834'14529) modify 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.933448.0:2871 2014-08-11 15:23:48.976320 2014-08-13 17:38:03.014043 7fe9bae887a0 20 read_log 192834'14539 (0'0) promote 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.361.0:31462 2014-08-11 15:23:51.508348 2014-08-13 17:38:03.014055 7fe9bae887a0 20 read_log 192834'14540 (192834'14539) modify 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by client.933448.0:3380 2014-08-11 15:23:51.500196 2014-08-13 17:38:03.014068 7fe9bae887a0 20 read_log 192834'14569 (0'0) promote b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.361.0:31526 2014-08-11 15:24:02.361534 2014-08-13 17:38:03.014081 7fe9bae887a0 20 read_log 192834'14570 (192834'14569) modify b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by client.933448.0:5649 2014-08-11 15:24:02.354625 2014-08-13 17:38:03.014093 7fe9bae887a0 20 read_log 192834'14577 (0'0) promote ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by osd.361.0:31544 2014-08-11 15:24:03.765210 2014-08-13 17:38:03.014106 7fe9bae887a0 20 read_log 192834'14578 (192834'14577) modify ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.933448.0:6010 2014-08-11 15:24:03.757923 2014-08-13 17:38:03.014118 7fe9bae887a0 20 read_log 192834'14589 (0'0) promote dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by osd.361.0:31572 2014-08-11 15:24:06.342494 2014-08-13 17:38:03.014130 7fe9bae887a0 20 read_log 192834'14590 (192834'14589) modify dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.933448.0:6679 2014-08-11 15:24:06.335993 2014-08-13 17:38:03.014142 7fe9bae887a0 20 read_log 192834'14597 (0'0) promote 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.361.0:31586 2014-08-11 15:24:09.028306 2014-08-13 17:38:03.014161 7fe9bae887a0 20 read_log 192834'14598 (192834'14597) modify 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by client.933448.0:7173 2014-08-11 15:24:09.021258 2014-08-13 17:38:03.014177 7fe9bae887a0 20 read_log 192834'14617 (0'0) promote 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.361.0:31624 2014-08-11 15:24:12.599377 2014-08-13 17:38:03.014190 7fe9bae887a0 20 read_log 192834'14618 (192834'14617) modify 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by client.933448.0:8068 2014-08-11 15:24:12.589061 2014-08-13 17:38:03.014202 7fe9bae887a0 20 read_log 192834'14629 (0'0) promote b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by osd.361.0:31640 2014-08-11 15:24:14.585593 2014-08-13 17:38:03.014214 7fe9bae887a0 20 read_log 192834'14630 (192834'14629) modify b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.933448.0:8567 2014-08-11 15:24:14.578329 2014-08-13 17:38:03.014226 7fe9bae887a0 20 read_log 192834'14635 (0'0) promote 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.361.0:31652 2014-08-11 15:24:17.345073 2014-08-13 17:38:03.014246 7fe9bae887a0 20 read_log 192834'14636 (192834'14635) modify 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by client.933448.0:9046 2014-08-11 15:24:17.336967 2014-08-13 17:38:03.014259 7fe9bae887a0 20 read_log 192834'14683 (0'0) promote 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.361.0:31760 2014-08-11 15:24:31.141414 2014-08-13 17:38:03.014271 7fe9bae887a0 20 read_log 192834'14684 (192834'14683) modify 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by client.933448.0:12114 2014-08-11 15:24:31.134774 2014-08-13 17:38:03.014283 7fe9bae887a0 20 read_log 192834'14685 (0'0) promote 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by osd.361.0:31764 2014-08-11 15:24:31.321595 2014-08-13 17:38:03.014295 7fe9bae887a0 20 read_log 192834'14686 (192834'14685) modify 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.933448.0:12162 2014-08-11 15:24:31.314481 2014-08-13 17:38:03.014308 7fe9bae887a0 20 read_log 192834'14687 (0'0) promote 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.361.0:31768 2014-08-11 15:24:32.013926 2014-08-13 17:38:03.014320 7fe9bae887a0 20 read_log 192834'14688 (192834'14687) modify 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by client.933448.0:12347 2014-08-11 15:24:32.006741 2014-08-13 17:38:03.014332 7fe9bae887a0 20 read_log 192834'14709 (0'0) promote bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.361.0:31822 2014-08-11 15:24:40.428318 2014-08-13 17:38:03.014344 7fe9bae887a0 20 read_log 192834'14710 (192834'14709) modify bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by client.933448.0:13916 2014-08-11 15:24:40.420218 2014-08-13 17:38:03.014356 7fe9bae887a0 20 read_log 192834'14753 (0'0) promote 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by osd.361.0:31927 2014-08-11 15:24:53.389670 2014-08-13 17:38:03.014368 7fe9bae887a0 20 read_log 192834'14754 (192834'14753) modify 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.933448.0:16171 2014-08-11 15:24:53.381794 2014-08-13 17:38:03.014380 7fe9bae887a0 20 read_log 192834'14757 (0'0) promote 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by osd.361.0:31953 2014-08-11 15:24:55.957766 2014-08-13 17:38:03.014392 7fe9bae887a0 20 read_log 192834'14758 (192834'14757) modify 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.933448.0:16491 2014-08-11 15:24:55.950399 2014-08-13 17:38:03.014404 7fe9bae887a0 20 read_log 192834'14779 (0'0) promote 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.361.0:32001 2014-08-11 15:25:02.081447 2014-08-13 17:38:03.014416 7fe9bae887a0 20 read_log 192834'14780 (192834'14779) modify 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by client.933448.0:17860 2014-08-11 15:25:02.034769 2014-08-13 17:38:03.014428 7fe9bae887a0 20 read_log 192834'14793 (0'0) promote 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by osd.361.0:32040 2014-08-11 15:25:07.204393 2014-08-13 17:38:03.014440 7fe9bae887a0 20 read_log 192834'14794 (192834'14793) modify 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.933448.0:18854 2014-08-11 15:25:07.196859 2014-08-13 17:38:03.014471 7fe9bae887a0 20 read_log 192834'14797 (0'0) promote 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by osd.361.0:32074 2014-08-11 15:25:08.477011 2014-08-13 17:38:03.014486 7fe9bae887a0 20 read_log 192834'14798 (192834'14797) modify 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.933448.0:19065 2014-08-11 15:25:08.371525 2014-08-13 17:38:03.014499 7fe9bae887a0 20 read_log 192834'14801 (0'0) promote 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.361.0:32086 2014-08-11 15:25:11.524047 2014-08-13 17:38:03.014519 7fe9bae887a0 20 read_log 192834'14802 (192834'14801) modify 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by client.933448.0:19337 2014-08-11 15:25:11.516347 2014-08-13 17:38:03.014534 7fe9bae887a0 20 read_log 192834'14829 (0'0) promote 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by osd.361.0:32167 2014-08-11 15:25:20.070087 2014-08-13 17:38:03.014546 7fe9bae887a0 20 read_log 192834'14830 (192834'14829) modify 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.933448.0:20278 2014-08-11 15:25:20.063264 2014-08-13 17:38:03.014559 7fe9bae887a0 20 read_log 192834'14857 (0'0) promote 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by osd.361.0:32240 2014-08-11 15:25:30.822549 2014-08-13 17:38:03.014571 7fe9bae887a0 20 read_log 192834'14858 (192834'14857) modify 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.933448.0:21519 2014-08-11 15:25:30.814592 2014-08-13 17:38:03.014583 7fe9bae887a0 20 read_log 192834'14863 (0'0) promote 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by osd.361.0:32279 2014-08-11 15:25:32.850718 2014-08-13 17:38:03.014595 7fe9bae887a0 20 read_log 192834'14864 (192834'14863) modify 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.933448.0:21930 2014-08-11 15:25:32.839880 2014-08-13 17:38:03.014607 7fe9bae887a0 20 read_log 192834'14895 (192834'14496) modify 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.933448.0:22940 2014-08-11 15:25:38.830050 2014-08-13 17:38:03.014619 7fe9bae887a0 20 read_log 192834'14898 (192834'14502) modify fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.933448.0:23766 2014-08-11 15:25:39.810271 2014-08-13 17:38:03.014631 7fe9bae887a0 20 read_log 192834'14902 (192834'14510) modify 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.933448.0:24175 2014-08-11 15:25:40.363049 2014-08-13 17:38:03.014643 7fe9bae887a0 20 read_log 192834'14907 (192834'14518) modify b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by client.933448.0:24571 2014-08-11 15:25:41.035827 2014-08-13 17:38:03.014655 7fe9bae887a0 20 read_log 192834'14908 (192834'14520) modify 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by client.933448.0:24622 2014-08-11 15:25:41.111385 2014-08-13 17:38:03.014667 7fe9bae887a0 20 read_log 192834'14913 (192834'14530) modify 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.933448.0:25162 2014-08-11 15:25:41.837740 2014-08-13 17:38:03.014679 7fe9bae887a0 20 read_log 192834'14919 (192834'14540) modify 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by client.933448.0:25671 2014-08-11 15:25:42.330900 2014-08-13 17:38:03.014691 7fe9bae887a0 20 read_log 192834'14934 (192834'14570) modify b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by client.933448.0:27940 2014-08-11 15:25:45.288257 2014-08-13 17:38:03.014703 7fe9bae887a0 20 read_log 192834'14939 (192834'14578) modify ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.933448.0:28301 2014-08-11 15:25:45.891774 2014-08-13 17:38:03.014715 7fe9bae887a0 20 read_log 192834'14946 (192834'14590) modify dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.933448.0:28970 2014-08-11 15:25:46.895716 2014-08-13 17:38:03.014727 7fe9bae887a0 20 read_log 192834'14950 (192834'14598) modify 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by client.933448.0:29464 2014-08-11 15:25:47.369107 2014-08-13 17:38:03.014739 7fe9bae887a0 20 read_log 192834'14960 (192834'14618) modify 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by client.933448.0:30359 2014-08-11 15:25:48.776178 2014-08-13 17:38:03.014751 7fe9bae887a0 20 read_log 192834'14968 (192834'14630) modify b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.933448.0:30858 2014-08-11 15:25:49.318543 2014-08-13 17:38:03.014773 7fe9bae887a0 20 read_log 192834'14971 (192834'14636) modify 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by client.933448.0:31337 2014-08-11 15:25:49.941353 2014-08-13 17:38:03.014786 7fe9bae887a0 20 read_log 192834'14998 (192834'14684) modify 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by client.933448.0:34405 2014-08-11 15:25:54.972525 2014-08-13 17:38:03.014799 7fe9bae887a0 20 read_log 192834'14999 (192834'14686) modify 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.933448.0:34453 2014-08-11 15:25:55.024071 2014-08-13 17:38:03.014811 7fe9bae887a0 20 read_log 192834'15000 (192834'14688) modify 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by client.933448.0:34638 2014-08-11 15:25:55.308966 2014-08-13 17:38:03.014823 7fe9bae887a0 20 read_log 192834'15011 (192834'14710) modify bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by client.933448.0:36207 2014-08-11 15:25:57.346835 2014-08-13 17:38:03.014835 7fe9bae887a0 20 read_log 192834'15033 (192834'14754) modify 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.933448.0:38462 2014-08-11 15:26:00.893555 2014-08-13 17:38:03.014847 7fe9bae887a0 20 read_log 192834'15035 (192834'14758) modify 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.933448.0:38782 2014-08-11 15:26:01.263012 2014-08-13 17:38:03.014859 7fe9bae887a0 20 read_log 192834'15046 (192834'14780) modify 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by client.933448.0:40151 2014-08-11 15:26:02.351595 2014-08-13 17:38:03.014871 7fe9bae887a0 20 read_log 192834'15053 (192834'14794) modify 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.933448.0:41145 2014-08-11 15:26:02.851758 2014-08-13 17:38:03.014883 7fe9bae887a0 20 read_log 192834'15055 (192834'14798) modify 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.933448.0:41356 2014-08-11 15:26:02.919060 2014-08-13 17:38:03.014895 7fe9bae887a0 20 read_log 192834'15057 (192834'14802) modify 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by client.933448.0:41628 2014-08-11 15:26:03.004141 2014-08-13 17:38:03.014907 7fe9bae887a0 20 read_log 192834'15071 (192834'14830) modify 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.933448.0:42569 2014-08-11 15:26:03.318293 2014-08-13 17:38:03.014926 7fe9bae887a0 20 read_log 192834'15085 (192834'14858) modify 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.933448.0:43810 2014-08-11 15:26:03.730996 2014-08-13 17:38:03.014938 7fe9bae887a0 20 read_log 192834'15088 (192834'14864) modify 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.933448.0:44221 2014-08-11 15:26:03.853987 2014-08-13 17:38:03.014951 7fe9bae887a0 20 read_log 192834'15104 (0'0) promote d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.361.0:32911 2014-08-11 15:26:16.993682 2014-08-13 17:38:03.014963 7fe9bae887a0 20 read_log 192834'15105 (192834'15104) modify d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by client.1883634.0:687 2014-08-11 15:26:16.986686 2014-08-13 17:38:03.014977 7fe9bae887a0 20 read_log 192834'15122 (0'0) promote 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.361.0:32949 2014-08-11 15:26:22.144254 2014-08-13 17:38:03.014989 7fe9bae887a0 20 read_log 192834'15123 (192834'15122) modify 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by client.1883634.0:2077 2014-08-11 15:26:22.136952 2014-08-13 17:38:03.015005 7fe9bae887a0 20 read_log 192834'15134 (0'0) promote bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by osd.361.0:32965 2014-08-11 15:26:23.596406 2014-08-13 17:38:03.015018 7fe9bae887a0 20 read_log 192834'15135 (192834'15134) modify bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.1883634.0:2473 2014-08-11 15:26:23.588806 2014-08-13 17:38:03.015058 7fe9bae887a0 20 read_log 192834'15138 (0'0) promote 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by osd.361.0:32969 2014-08-11 15:26:23.611803 2014-08-13 17:38:03.015075 7fe9bae887a0 20 read_log 192834'15139 (192834'15138) modify 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.1883634.0:2477 2014-08-11 15:26:23.601830 2014-08-13 17:38:03.015087 7fe9bae887a0 20 read_log 192834'15144 (0'0) promote 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by osd.361.0:32975 2014-08-11 15:26:25.701871 2014-08-13 17:38:03.015100 7fe9bae887a0 20 read_log 192834'15145 (192834'15144) modify 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.1883634.0:3068 2014-08-11 15:26:25.694604 2014-08-13 17:38:03.015112 7fe9bae887a0 20 read_log 192834'15148 (0'0) promote c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by osd.361.0:32979 2014-08-11 15:26:26.183683 2014-08-13 17:38:03.015124 7fe9bae887a0 20 read_log 192834'15149 (192834'15148) modify c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.1883634.0:3204 2014-08-11 15:26:26.176264 2014-08-13 17:38:03.015136 7fe9bae887a0 20 read_log 192834'15154 (0'0) promote 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.361.0:32987 2014-08-11 15:26:27.074559 2014-08-13 17:38:03.015149 7fe9bae887a0 20 read_log 192834'15155 (192834'15154) modify 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by client.1883634.0:3456 2014-08-11 15:26:27.067324 2014-08-13 17:38:03.015166 7fe9bae887a0 20 read_log 192834'15162 (0'0) promote 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by osd.361.0:33003 2014-08-11 15:26:30.212288 2014-08-13 17:38:03.015178 7fe9bae887a0 20 read_log 192834'15163 (192834'15162) modify 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.1883634.0:4344 2014-08-11 15:26:30.204700 2014-08-13 17:38:03.015191 7fe9bae887a0 20 read_log 192834'15164 (0'0) promote 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by osd.361.0:33011 2014-08-11 15:26:30.924146 2014-08-13 17:38:03.015203 7fe9bae887a0 20 read_log 192834'15165 (192834'15164) modify 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.1883634.0:4526 2014-08-11 15:26:30.916858 2014-08-13 17:38:03.015215 7fe9bae887a0 20 read_log 192834'15174 (0'0) promote 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.361.0:33027 2014-08-11 15:26:33.383695 2014-08-13 17:38:03.015246 7fe9bae887a0 20 read_log 192834'15175 (192834'15174) modify 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by client.1883634.0:5148 2014-08-11 15:26:33.376010 2014-08-13 17:38:03.015263 7fe9bae887a0 20 read_log 192834'15216 (0'0) promote d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by osd.361.0:33198 2014-08-11 15:26:47.322218 2014-08-13 17:38:03.015276 7fe9bae887a0 20 read_log 192834'15217 (192834'15216) modify d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.1883634.0:8669 2014-08-11 15:26:47.313356 2014-08-13 17:38:03.015288 7fe9bae887a0 20 read_log 192834'15220 (0'0) promote 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by osd.361.0:33204 2014-08-11 15:26:48.147012 2014-08-13 17:38:03.015301 7fe9bae887a0 20 read_log 192834'15221 (192834'15220) modify 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.1883634.0:8844 2014-08-11 15:26:48.138119 2014-08-13 17:38:03.015313 7fe9bae887a0 20 read_log 192834'15238 (0'0) promote a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by osd.361.0:33317 2014-08-11 15:26:53.636578 2014-08-13 17:38:03.015325 7fe9bae887a0 20 read_log 192834'15239 (192834'15238) modify a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.1883634.0:10085 2014-08-11 15:26:53.628859 2014-08-13 17:38:03.015345 7fe9bae887a0 20 read_log 192834'15246 (0'0) promote a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by osd.361.0:33402 2014-08-11 15:26:57.356376 2014-08-13 17:38:03.015360 7fe9bae887a0 20 read_log 192834'15247 (192834'15246) modify a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.1883634.0:10874 2014-08-11 15:26:57.348666 2014-08-13 17:38:03.015372 7fe9bae887a0 20 read_log 192834'15264 (0'0) promote 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.361.0:33508 2014-08-11 15:27:03.999643 2014-08-13 17:38:03.015385 7fe9bae887a0 20 read_log 192834'15265 (192834'15264) modify 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by client.1883634.0:12434 2014-08-11 15:27:03.991917 2014-08-13 17:38:03.015397 7fe9bae887a0 20 read_log 192834'15274 (0'0) promote 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by osd.361.0:33593 2014-08-11 15:27:07.236032 2014-08-13 17:38:03.015409 7fe9bae887a0 20 read_log 192834'15275 (192834'15274) modify 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.1883634.0:13018 2014-08-11 15:27:07.227646 2014-08-13 17:38:03.015421 7fe9bae887a0 20 read_log 192834'15294 (0'0) promote 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.361.0:33633 2014-08-11 15:27:14.048790 2014-08-13 17:38:03.015433 7fe9bae887a0 20 read_log 192834'15295 (192834'15294) modify 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by client.1883634.0:14619 2014-08-11 15:27:14.037923 2014-08-13 17:38:03.015446 7fe9bae887a0 20 read_log 192834'15308 (0'0) promote 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by osd.361.0:33653 2014-08-11 15:27:18.987179 2014-08-13 17:38:03.015458 7fe9bae887a0 20 read_log 192834'15309 (192834'15308) modify 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.1883634.0:15297 2014-08-11 15:27:18.977724 2014-08-13 17:38:03.015470 7fe9bae887a0 20 read_log 192834'15318 (0'0) promote a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by osd.361.0:33690 2014-08-11 15:27:24.914490 2014-08-13 17:38:03.015483 7fe9bae887a0 20 read_log 192834'15319 (192834'15318) modify a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.1883634.0:15966 2014-08-11 15:27:24.905293 2014-08-13 17:38:03.015495 7fe9bae887a0 20 read_log 192834'15336 (0'0) promote 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by osd.361.0:33745 2014-08-11 15:27:33.917876 2014-08-13 17:38:03.015507 7fe9bae887a0 20 read_log 192834'15337 (192834'15336) modify 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.1883634.0:17255 2014-08-11 15:27:33.910079 2014-08-13 17:38:03.015519 7fe9bae887a0 20 read_log 192834'15344 (0'0) promote 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by osd.361.0:33814 2014-08-11 15:27:47.295439 2014-08-13 17:38:03.015531 7fe9bae887a0 20 read_log 192834'15345 (192834'15344) modify 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.1883634.0:18736 2014-08-11 15:27:47.287721 2014-08-13 17:38:03.015543 7fe9bae887a0 20 read_log 192834'15348 (0'0) promote 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by osd.361.0:33820 2014-08-11 15:27:48.995204 2014-08-13 17:38:03.015556 7fe9bae887a0 20 read_log 192834'15349 (192834'15348) modify 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.1883634.0:19032 2014-08-11 15:27:48.986754 2014-08-13 17:38:03.015568 7fe9bae887a0 20 read_log 192834'15354 (0'0) promote ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.361.0:33853 2014-08-11 15:27:51.139924 2014-08-13 17:38:03.015580 7fe9bae887a0 20 read_log 192834'15355 (192834'15354) modify ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by client.1883634.0:19307 2014-08-11 15:27:51.131704 2014-08-13 17:38:03.015602 7fe9bae887a0 20 read_log 192834'15378 (0'0) promote 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by osd.361.0:33935 2014-08-11 15:28:02.399447 2014-08-13 17:38:03.015617 7fe9bae887a0 20 read_log 192834'15379 (192834'15378) modify 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.1883634.0:21023 2014-08-11 15:28:02.390910 2014-08-13 17:38:03.015630 7fe9bae887a0 20 read_log 192834'15382 (0'0) promote 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by osd.361.0:33941 2014-08-11 15:28:02.759781 2014-08-13 17:38:03.015642 7fe9bae887a0 20 read_log 192834'15383 (192834'15382) modify 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.1883634.0:21092 2014-08-11 15:28:02.751789 2014-08-13 17:38:03.015654 7fe9bae887a0 20 read_log 192834'15388 (0'0) promote d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.361.0:33947 2014-08-11 15:28:03.258525 2014-08-13 17:38:03.015666 7fe9bae887a0 20 read_log 192834'15389 (192834'15388) modify d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by client.1883634.0:21187 2014-08-11 15:28:03.248729 2014-08-13 17:38:03.015678 7fe9bae887a0 20 read_log 192834'15410 (0'0) promote 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by osd.361.0:34013 2014-08-11 15:28:10.432366 2014-08-13 17:38:03.015693 7fe9bae887a0 20 read_log 192834'15411 (192834'15410) modify 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.1883634.0:22666 2014-08-11 15:28:10.422456 2014-08-13 17:38:03.015706 7fe9bae887a0 20 read_log 192834'15432 (192834'15105) modify d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by client.1883634.0:24011 2014-08-11 15:28:18.232309 2014-08-13 17:38:03.015718 7fe9bae887a0 20 read_log 192834'15441 (192834'15123) modify 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by client.1883634.0:25401 2014-08-11 15:28:21.153341 2014-08-13 17:38:03.015747 7fe9bae887a0 20 read_log 192834'15447 (192834'15135) modify bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.1883634.0:25797 2014-08-11 15:28:21.888982 2014-08-13 17:38:03.015762 7fe9bae887a0 20 read_log 192834'15449 (192834'15139) modify 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.1883634.0:25801 2014-08-11 15:28:21.898156 2014-08-13 17:38:03.015774 7fe9bae887a0 20 read_log 192834'15452 (192834'15145) modify 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.1883634.0:26392 2014-08-11 15:28:23.076613 2014-08-13 17:38:03.015786 7fe9bae887a0 20 read_log 192834'15454 (192834'15149) modify c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.1883634.0:26528 2014-08-11 15:28:23.221132 2014-08-13 17:38:03.015799 7fe9bae887a0 20 read_log 192834'15457 (192834'15155) modify 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by client.1883634.0:26780 2014-08-11 15:28:23.574459 2014-08-13 17:38:03.015811 7fe9bae887a0 20 read_log 192834'15461 (192834'15163) modify 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.1883634.0:27668 2014-08-11 15:28:24.775709 2014-08-13 17:38:03.015823 7fe9bae887a0 20 read_log 192834'15462 (192834'15165) modify 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.1883634.0:27850 2014-08-11 15:28:25.042698 2014-08-13 17:38:03.015836 7fe9bae887a0 20 read_log 192834'15467 (192834'15175) modify 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by client.1883634.0:28472 2014-08-11 15:28:25.856133 2014-08-13 17:38:03.015848 7fe9bae887a0 20 read_log 192834'15488 (192834'15217) modify d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.1883634.0:31993 2014-08-11 15:28:31.206074 2014-08-13 17:38:03.015860 7fe9bae887a0 20 read_log 192834'15490 (192834'15221) modify 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.1883634.0:32168 2014-08-11 15:28:31.449180 2014-08-13 17:38:03.015879 7fe9bae887a0 20 read_log 192834'15501 (192834'15239) modify a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.1883634.0:33409 2014-08-11 15:28:33.134892 2014-08-13 17:38:03.015895 7fe9bae887a0 20 read_log 192834'15505 (192834'15247) modify a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.1883634.0:34198 2014-08-11 15:28:34.356260 2014-08-13 17:38:03.015907 7fe9bae887a0 20 read_log 192834'15514 (192834'15265) modify 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by client.1883634.0:35758 2014-08-11 15:28:37.094485 2014-08-13 17:38:03.015919 7fe9bae887a0 20 read_log 192834'15519 (192834'15275) modify 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.1883634.0:36342 2014-08-11 15:28:37.910318 2014-08-13 17:38:03.015931 7fe9bae887a0 20 read_log 192834'15529 (192834'15295) modify 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by client.1883634.0:37943 2014-08-11 15:28:39.994679 2014-08-13 17:38:03.015944 7fe9bae887a0 20 read_log 192834'15536 (192834'15309) modify 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.1883634.0:38621 2014-08-11 15:28:40.941859 2014-08-13 17:38:03.015955 7fe9bae887a0 20 read_log 192834'15541 (192834'15319) modify a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.1883634.0:39290 2014-08-11 15:28:41.968621 2014-08-13 17:38:03.015968 7fe9bae887a0 20 read_log 192834'15550 (192834'15337) modify 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.1883634.0:40579 2014-08-11 15:28:44.002991 2014-08-13 17:38:03.015980 7fe9bae887a0 20 read_log 192834'15554 (192834'15345) modify 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.1883634.0:42060 2014-08-11 15:28:46.119209 2014-08-13 17:38:03.015992 7fe9bae887a0 20 read_log 192834'15556 (192834'15349) modify 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.1883634.0:42356 2014-08-11 15:28:46.217478 2014-08-13 17:38:03.016004 7fe9bae887a0 20 read_log 192834'15559 (192834'15355) modify ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by client.1883634.0:42631 2014-08-11 15:28:46.304569 2014-08-13 17:38:03.016016 7fe9bae887a0 20 read_log 192834'15571 (192834'15379) modify 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.1883634.0:44347 2014-08-11 15:28:46.834311 2014-08-13 17:38:03.016028 7fe9bae887a0 20 read_log 192834'15573 (192834'15383) modify 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.1883634.0:44416 2014-08-11 15:28:46.853378 2014-08-13 17:38:03.016040 7fe9bae887a0 20 read_log 192834'15576 (192834'15389) modify d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by client.1883634.0:44511 2014-08-11 15:28:46.878083 2014-08-13 17:38:03.016051 7fe9bae887a0 20 read_log 192834'15587 (192834'15411) modify 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.1883634.0:45990 2014-08-11 15:28:47.458413 2014-08-13 17:38:03.016063 7fe9bae887a0 20 read_log 192888'15598 (0'0) promote 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by osd.345.0:1369 2014-08-11 16:45:01.722244 2014-08-13 17:38:03.016076 7fe9bae887a0 20 read_log 192888'15599 (192888'15598) modify 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.1315259.0:1689 2014-08-11 16:45:01.715110 2014-08-13 17:38:03.016088 7fe9bae887a0 20 read_log 192888'15600 (0'0) promote 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by osd.345.0:1377 2014-08-11 16:45:05.102385 2014-08-13 17:38:03.016100 7fe9bae887a0 20 read_log 192888'15601 (192888'15600) modify 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.1315259.0:2602 2014-08-11 16:45:05.093700 2014-08-13 17:38:03.016119 7fe9bae887a0 20 read_log 192888'15602 (0'0) promote daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.345.0:1381 2014-08-11 16:45:05.784027 2014-08-13 17:38:03.016132 7fe9bae887a0 20 read_log 192888'15603 (192888'15602) modify daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by client.1315259.0:2783 2014-08-11 16:45:05.776749 2014-08-13 17:38:03.016144 7fe9bae887a0 20 read_log 192888'15604 (0'0) promote 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by osd.345.0:1383 2014-08-11 16:45:06.129703 2014-08-13 17:38:03.016162 7fe9bae887a0 20 read_log 192888'15605 (192888'15604) modify 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.1315259.0:2877 2014-08-11 16:45:06.121452 2014-08-13 17:38:03.016175 7fe9bae887a0 20 read_log 192888'15606 (0'0) promote e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by osd.345.0:1387 2014-08-11 16:45:07.651994 2014-08-13 17:38:03.016187 7fe9bae887a0 20 read_log 192888'15607 (192888'15606) modify e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.1315259.0:3252 2014-08-11 16:45:07.643104 2014-08-13 17:38:03.016199 7fe9bae887a0 20 read_log 192888'15608 (0'0) promote 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by osd.345.0:1405 2014-08-11 16:45:11.045442 2014-08-13 17:38:03.016211 7fe9bae887a0 20 read_log 192888'15609 (192888'15608) modify 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.1315259.0:4160 2014-08-11 16:45:11.037083 2014-08-13 17:38:03.016224 7fe9bae887a0 20 read_log 192888'15612 (0'0) promote 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by osd.345.0:1409 2014-08-11 16:45:11.859177 2014-08-13 17:38:03.016236 7fe9bae887a0 20 read_log 192888'15613 (192888'15612) modify 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.1315259.0:4384 2014-08-11 16:45:11.850082 2014-08-13 17:38:03.016248 7fe9bae887a0 20 read_log 192888'15614 (0'0) promote eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.345.0:1411 2014-08-11 16:45:12.787675 2014-08-13 17:38:03.016260 7fe9bae887a0 20 read_log 192888'15615 (192888'15614) modify eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by client.1315259.0:4639 2014-08-11 16:45:12.776300 2014-08-13 17:38:03.016272 7fe9bae887a0 20 read_log 192888'15622 (0'0) promote 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by osd.345.0:1433 2014-08-11 16:45:18.366298 2014-08-13 17:38:03.016284 7fe9bae887a0 20 read_log 192888'15623 (192888'15622) modify 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.1315259.0:6161 2014-08-11 16:45:18.359020 2014-08-13 17:38:03.016296 7fe9bae887a0 20 read_log 192888'15624 (0'0) promote 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.345.0:1435 2014-08-11 16:45:18.447636 2014-08-13 17:38:03.016308 7fe9bae887a0 20 read_log 192888'15625 (192888'15624) modify 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by client.1315259.0:6184 2014-08-11 16:45:18.439851 2014-08-13 17:38:03.016320 7fe9bae887a0 20 read_log 192888'15632 (0'0) promote 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by osd.345.0:1473 2014-08-11 16:45:25.610487 2014-08-13 17:38:03.016332 7fe9bae887a0 20 read_log 192888'15633 (192888'15632) modify 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.1315259.0:8158 2014-08-11 16:45:25.602967 2014-08-13 17:38:03.016344 7fe9bae887a0 20 read_log 192888'15634 (0'0) promote 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by osd.345.0:1491 2014-08-11 16:45:30.223522 2014-08-13 17:38:03.016356 7fe9bae887a0 20 read_log 192888'15635 (192888'15634) modify 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.1315259.0:9377 2014-08-11 16:45:30.213806 2014-08-13 17:38:03.016378 7fe9bae887a0 20 read_log 192888'15638 (0'0) promote cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by osd.345.0:1501 2014-08-11 16:45:32.799403 2014-08-13 17:38:03.016391 7fe9bae887a0 20 read_log 192888'15639 (192888'15638) modify cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.1315259.0:10038 2014-08-11 16:45:32.790035 2014-08-13 17:38:03.016404 7fe9bae887a0 20 read_log 192888'15640 (0'0) promote 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.345.0:1511 2014-08-11 16:45:39.345930 2014-08-13 17:38:03.016415 7fe9bae887a0 20 read_log 192888'15641 (192888'15640) modify 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by client.1315259.0:11678 2014-08-11 16:45:39.282834 2014-08-13 17:38:03.016428 7fe9bae887a0 20 read_log 192888'15642 (0'0) promote 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by osd.345.0:1515 2014-08-11 16:45:39.512309 2014-08-13 17:38:03.016440 7fe9bae887a0 20 read_log 192888'15643 (192888'15642) modify 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.1315259.0:11735 2014-08-11 16:45:39.503932 2014-08-13 17:38:03.016452 7fe9bae887a0 20 read_log 192888'15648 (0'0) promote 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by osd.345.0:1529 2014-08-11 16:45:42.293815 2014-08-13 17:38:03.016464 7fe9bae887a0 20 read_log 192888'15649 (192888'15648) modify 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.1315259.0:12379 2014-08-11 16:45:42.285804 2014-08-13 17:38:03.016476 7fe9bae887a0 20 read_log 192888'15650 (0'0) promote 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by osd.345.0:1535 2014-08-11 16:45:45.058479 2014-08-13 17:38:03.016488 7fe9bae887a0 20 read_log 192888'15651 (192888'15650) modify 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.1315259.0:13048 2014-08-11 16:45:45.049508 2014-08-13 17:38:03.016501 7fe9bae887a0 20 read_log 192888'15656 (0'0) promote b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.345.0:1549 2014-08-11 16:45:47.743477 2014-08-13 17:38:03.016513 7fe9bae887a0 20 read_log 192888'15657 (192888'15656) modify b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by client.1315259.0:13723 2014-08-11 16:45:47.736478 2014-08-13 17:38:03.016525 7fe9bae887a0 20 read_log 192888'15660 (0'0) promote d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.345.0:1565 2014-08-11 16:45:50.578659 2014-08-13 17:38:03.016537 7fe9bae887a0 20 read_log 192888'15661 (192888'15660) modify d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by client.1315259.0:14468 2014-08-11 16:45:50.570485 2014-08-13 17:38:03.016549 7fe9bae887a0 20 read_log 192888'15662 (0'0) promote 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.345.0:1569 2014-08-11 16:45:51.244444 2014-08-13 17:38:03.016561 7fe9bae887a0 20 read_log 192888'15663 (192888'15662) modify 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by client.1315259.0:14651 2014-08-11 16:45:51.236894 2014-08-13 17:38:03.016573 7fe9bae887a0 20 read_log 192888'15664 (0'0) promote 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by osd.345.0:1583 2014-08-11 16:45:55.745531 2014-08-13 17:38:03.016585 7fe9bae887a0 20 read_log 192888'15665 (192888'15664) modify 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.1315259.0:15819 2014-08-11 16:45:55.737419 2014-08-13 17:38:03.016597 7fe9bae887a0 20 read_log 192888'15666 (0'0) promote 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by osd.345.0:1585 2014-08-11 16:45:56.528214 2014-08-13 17:38:03.016609 7fe9bae887a0 20 read_log 192888'15667 (192888'15666) modify 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.1315259.0:16026 2014-08-11 16:45:56.519763 2014-08-13 17:38:03.016627 7fe9bae887a0 20 read_log 192888'15670 (0'0) promote b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by osd.345.0:1595 2014-08-11 16:45:59.246201 2014-08-13 17:38:03.016640 7fe9bae887a0 20 read_log 192888'15671 (192888'15670) modify b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.1315259.0:16721 2014-08-11 16:45:59.238007 2014-08-13 17:38:03.016653 7fe9bae887a0 20 read_log 192888'15674 (0'0) promote ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.345.0:1619 2014-08-11 16:46:04.679796 2014-08-13 17:38:03.016665 7fe9bae887a0 20 read_log 192888'15675 (192888'15674) modify ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by client.1315259.0:18070 2014-08-11 16:46:04.671303 2014-08-13 17:38:03.016677 7fe9bae887a0 20 read_log 192888'15678 (0'0) promote ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by osd.345.0:1625 2014-08-11 16:46:05.689748 2014-08-13 17:38:03.016689 7fe9bae887a0 20 read_log 192888'15679 (192888'15678) modify ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.1315259.0:18340 2014-08-11 16:46:05.682037 2014-08-13 17:38:03.016701 7fe9bae887a0 20 read_log 192888'15684 (0'0) promote 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by osd.345.0:1639 2014-08-11 16:46:07.263137 2014-08-13 17:38:03.016713 7fe9bae887a0 20 read_log 192888'15685 (192888'15684) modify 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.1315259.0:18760 2014-08-11 16:46:07.254130 2014-08-13 17:38:03.016725 7fe9bae887a0 20 read_log 192888'15690 (0'0) promote b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by osd.345.0:1675 2014-08-11 16:46:18.054131 2014-08-13 17:38:03.016737 7fe9bae887a0 20 read_log 192888'15691 (192888'15690) modify b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.1315259.0:21569 2014-08-11 16:46:18.046153 2014-08-13 17:38:03.016750 7fe9bae887a0 20 read_log 192888'15696 (0'0) promote afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.345.0:1713 2014-08-11 16:46:25.766605 2014-08-13 17:38:03.016763 7fe9bae887a0 20 read_log 192888'15697 (192888'15696) modify afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by client.1315259.0:23523 2014-08-11 16:46:25.756267 2014-08-13 17:38:03.016775 7fe9bae887a0 20 read_log 192888'15704 (0'0) promote 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by osd.345.0:1743 2014-08-11 16:46:34.412758 2014-08-13 17:38:03.016787 7fe9bae887a0 20 read_log 192888'15705 (192888'15704) modify 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.1315259.0:25706 2014-08-11 16:46:34.404412 2014-08-13 17:38:03.016799 7fe9bae887a0 20 read_log 192888'15706 (0'0) promote 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.345.0:1745 2014-08-11 16:46:34.625375 2014-08-13 17:38:03.016811 7fe9bae887a0 20 read_log 192888'15707 (192888'15706) modify 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by client.1315259.0:25755 2014-08-11 16:46:34.616758 2014-08-13 17:38:03.016823 7fe9bae887a0 20 read_log 192888'15714 (0'0) promote e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.345.0:1761 2014-08-11 16:46:42.022461 2014-08-13 17:38:03.016836 7fe9bae887a0 20 read_log 192888'15715 (192888'15714) modify e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by client.1315259.0:27503 2014-08-11 16:46:42.015164 2014-08-13 17:38:03.016848 7fe9bae887a0 20 read_log 192888'15716 (0'0) promote d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by osd.345.0:1775 2014-08-11 16:46:49.285156 2014-08-13 17:38:03.016860 7fe9bae887a0 20 read_log 192888'15717 (192888'15716) modify d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.1315259.0:29313 2014-08-11 16:46:49.277108 2014-08-13 17:38:03.016897 7fe9bae887a0 20 read_log 192888'15720 (0'0) promote dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by osd.345.0:1787 2014-08-11 16:46:52.870561 2014-08-13 17:38:03.016913 7fe9bae887a0 20 read_log 192888'15721 (192888'15720) modify dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.1315259.0:30204 2014-08-11 16:46:52.862993 2014-08-13 17:38:03.016926 7fe9bae887a0 20 read_log 192888'15722 (0'0) promote 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by osd.345.0:1789 2014-08-11 16:46:53.513921 2014-08-13 17:38:03.016938 7fe9bae887a0 20 read_log 192888'15723 (192888'15722) modify 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.1315259.0:30376 2014-08-11 16:46:53.506265 2014-08-13 17:38:03.016969 7fe9bae887a0 20 read_log 192888'15725 (192888'15599) modify 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.1315259.0:32416 2014-08-11 16:46:56.371678 2014-08-13 17:38:03.016985 7fe9bae887a0 20 read_log 192888'15726 (192888'15601) modify 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.1315259.0:33329 2014-08-11 16:46:56.609405 2014-08-13 17:38:03.016998 7fe9bae887a0 20 read_log 192888'15727 (192888'15603) modify daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by client.1315259.0:33510 2014-08-11 16:46:56.653872 2014-08-13 17:38:03.017010 7fe9bae887a0 20 read_log 192888'15728 (192888'15605) modify 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.1315259.0:33604 2014-08-11 16:46:56.678363 2014-08-13 17:38:03.017022 7fe9bae887a0 20 read_log 192888'15729 (192888'15607) modify e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.1315259.0:33979 2014-08-11 16:46:56.774026 2014-08-13 17:38:03.017034 7fe9bae887a0 20 read_log 192888'15730 (192888'15609) modify 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.1315259.0:34887 2014-08-11 16:46:57.011843 2014-08-13 17:38:03.017046 7fe9bae887a0 20 read_log 192888'15732 (192888'15613) modify 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.1315259.0:35111 2014-08-11 16:46:57.075854 2014-08-13 17:38:03.017059 7fe9bae887a0 20 read_log 192888'15733 (192888'15615) modify eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by client.1315259.0:35366 2014-08-11 16:46:57.146043 2014-08-13 17:38:03.017071 7fe9bae887a0 20 read_log 192888'15737 (192888'15623) modify 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.1315259.0:36888 2014-08-11 16:46:57.538679 2014-08-13 17:38:03.017083 7fe9bae887a0 20 read_log 192888'15738 (192888'15625) modify 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by client.1315259.0:36911 2014-08-11 16:46:57.543176 2014-08-13 17:38:03.017095 7fe9bae887a0 20 read_log 192888'15742 (192888'15633) modify 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.1315259.0:38885 2014-08-11 16:46:58.069406 2014-08-13 17:38:03.017107 7fe9bae887a0 20 read_log 192888'15743 (192888'15635) modify 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.1315259.0:40104 2014-08-11 16:46:58.390650 2014-08-13 17:38:03.017119 7fe9bae887a0 20 read_log 192888'15745 (192888'15639) modify cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.1315259.0:40765 2014-08-11 16:46:58.575784 2014-08-13 17:38:03.017131 7fe9bae887a0 20 read_log 192888'15746 (192888'15641) modify 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by client.1315259.0:42405 2014-08-11 16:46:59.002131 2014-08-13 17:38:03.017144 7fe9bae887a0 20 read_log 192888'15747 (192888'15643) modify 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.1315259.0:42462 2014-08-11 16:46:59.018106 2014-08-13 17:38:03.017167 7fe9bae887a0 20 read_log 192888'15750 (192888'15649) modify 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.1315259.0:43106 2014-08-11 16:46:59.194357 2014-08-13 17:38:03.017190 7fe9bae887a0 20 read_log 192888'15751 (192888'15651) modify 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.1315259.0:43775 2014-08-11 16:46:59.385060 2014-08-13 17:38:03.017206 7fe9bae887a0 20 read_log 192888'15754 (192888'15657) modify b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by client.1315259.0:44450 2014-08-11 16:46:59.581784 2014-08-13 17:38:03.017219 7fe9bae887a0 20 read_log 192888'15756 (192888'15661) modify d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by client.1315259.0:45195 2014-08-11 16:46:59.796362 2014-08-13 17:38:03.017231 7fe9bae887a0 20 read_log 192888'15757 (192888'15663) modify 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by client.1315259.0:45378 2014-08-11 16:46:59.851603 2014-08-13 17:38:03.017243 7fe9bae887a0 20 read_log 192888'15758 (192888'15665) modify 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.1315259.0:46546 2014-08-11 16:47:00.189603 2014-08-13 17:38:03.017255 7fe9bae887a0 20 read_log 192888'15759 (192888'15667) modify 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.1315259.0:46753 2014-08-11 16:47:00.248818 2014-08-13 17:38:03.017267 7fe9bae887a0 20 read_log 192888'15761 (192888'15671) modify b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.1315259.0:47448 2014-08-11 16:47:00.458454 2014-08-13 17:38:03.017279 7fe9bae887a0 20 read_log 192888'15763 (192888'15675) modify ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by client.1315259.0:48797 2014-08-11 16:47:00.869749 2014-08-13 17:38:03.017292 7fe9bae887a0 20 read_log 192888'15765 (192888'15679) modify ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.1315259.0:49067 2014-08-11 16:47:00.949050 2014-08-13 17:38:03.017304 7fe9bae887a0 20 read_log 192888'15768 (192888'15685) modify 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.1315259.0:49487 2014-08-11 16:47:01.079008 2014-08-13 17:38:03.017316 7fe9bae887a0 20 read_log 192888'15771 (192888'15691) modify b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.1315259.0:52296 2014-08-11 16:47:01.934692 2014-08-13 17:38:03.017328 7fe9bae887a0 20 read_log 192888'15774 (192888'15697) modify afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by client.1315259.0:54250 2014-08-11 16:47:02.510596 2014-08-13 17:38:03.017340 7fe9bae887a0 20 read_log 192888'15778 (192888'15705) modify 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.1315259.0:56433 2014-08-11 16:47:03.203281 2014-08-13 17:38:03.017352 7fe9bae887a0 20 read_log 192888'15779 (192888'15707) modify 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by client.1315259.0:56482 2014-08-11 16:47:03.218742 2014-08-13 17:38:03.017364 7fe9bae887a0 20 read_log 192888'15783 (192888'15715) modify e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by client.1315259.0:58230 2014-08-11 16:47:03.742243 2014-08-13 17:38:03.017376 7fe9bae887a0 20 read_log 192888'15784 (192888'15717) modify d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.1315259.0:60040 2014-08-11 16:47:04.322586 2014-08-13 17:38:03.017389 7fe9bae887a0 20 read_log 192888'15786 (192888'15721) modify dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.1315259.0:60931 2014-08-11 16:47:04.584399 2014-08-13 17:38:03.017401 7fe9bae887a0 20 read_log 192888'15787 (192888'15723) modify 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.1315259.0:61103 2014-08-11 16:47:04.634313 2014-08-13 17:38:03.017413 7fe9bae887a0 20 read_log 192888'15790 (0'0) promote 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.345.0:2021 2014-08-11 17:05:14.950939 2014-08-13 17:38:03.017425 7fe9bae887a0 20 read_log 192888'15791 (192888'15790) modify 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by client.942868.0:32 2014-08-11 17:05:14.939868 2014-08-13 17:38:03.017445 7fe9bae887a0 20 read_log 192888'15798 (0'0) promote a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by osd.345.0:2072 2014-08-11 17:05:27.415611 2014-08-13 17:38:03.017458 7fe9bae887a0 20 read_log 192888'15799 (192888'15798) modify a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.942868.0:3100 2014-08-11 17:05:27.405656 2014-08-13 17:38:03.017470 7fe9bae887a0 20 read_log 192888'15800 (0'0) promote 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.345.0:2084 2014-08-11 17:05:29.265988 2014-08-13 17:38:03.017482 7fe9bae887a0 20 read_log 192888'15801 (192888'15800) modify 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by client.942868.0:3601 2014-08-11 17:05:29.257204 2014-08-13 17:38:03.017495 7fe9bae887a0 20 read_log 192888'15802 (0'0) promote ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by osd.345.0:2086 2014-08-11 17:05:30.122575 2014-08-13 17:38:03.017507 7fe9bae887a0 20 read_log 192888'15803 (192888'15802) modify ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.942868.0:3833 2014-08-11 17:05:30.113920 2014-08-13 17:38:03.017519 7fe9bae887a0 20 read_log 192888'15806 (0'0) promote 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by osd.345.0:2092 2014-08-11 17:05:33.245421 2014-08-13 17:38:03.017531 7fe9bae887a0 20 read_log 192888'15807 (192888'15806) modify 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.942868.0:4672 2014-08-11 17:05:33.237251 2014-08-13 17:38:03.017543 7fe9bae887a0 20 read_log 192888'15808 (0'0) promote 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by osd.345.0:2094 2014-08-11 17:05:33.898574 2014-08-13 17:38:03.017556 7fe9bae887a0 20 read_log 192888'15809 (192888'15808) modify 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.942868.0:4836 2014-08-11 17:05:33.890276 2014-08-13 17:38:03.017570 7fe9bae887a0 20 read_log 192888'15814 (0'0) promote a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.345.0:2122 2014-08-11 17:05:42.381575 2014-08-13 17:38:03.017582 7fe9bae887a0 20 read_log 192888'15815 (192888'15814) modify a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by client.942868.0:7095 2014-08-11 17:05:42.373354 2014-08-13 17:38:03.017596 7fe9bae887a0 20 read_log 192888'15816 (0'0) promote ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by osd.345.0:2124 2014-08-11 17:05:42.801571 2014-08-13 17:38:03.017608 7fe9bae887a0 20 read_log 192888'15817 (192888'15816) modify ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.942868.0:7209 2014-08-11 17:05:42.794205 2014-08-13 17:38:03.017620 7fe9bae887a0 20 read_log 192888'15818 (0'0) promote 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by osd.345.0:2132 2014-08-11 17:05:43.980378 2014-08-13 17:38:03.017632 7fe9bae887a0 20 read_log 192888'15819 (192888'15818) modify 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.942868.0:7494 2014-08-11 17:05:43.970544 2014-08-13 17:38:03.017645 7fe9bae887a0 20 read_log 192888'15820 (0'0) promote 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by osd.345.0:2148 2014-08-11 17:05:46.187165 2014-08-13 17:38:03.017657 7fe9bae887a0 20 read_log 192888'15821 (192888'15820) modify 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.942868.0:8099 2014-08-11 17:05:46.177209 2014-08-13 17:38:03.017669 7fe9bae887a0 20 read_log 192888'15822 (0'0) promote 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by osd.345.0:2162 2014-08-11 17:05:48.890430 2014-08-13 17:38:03.017681 7fe9bae887a0 20 read_log 192888'15823 (192888'15822) modify 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.942868.0:8825 2014-08-11 17:05:48.882468 2014-08-13 17:38:03.017700 7fe9bae887a0 20 read_log 192888'15824 (0'0) promote 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.345.0:2164 2014-08-11 17:05:49.068551 2014-08-13 17:38:03.017713 7fe9bae887a0 20 read_log 192888'15825 (192888'15824) modify 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by client.942868.0:8874 2014-08-11 17:05:49.060330 2014-08-13 17:38:03.017725 7fe9bae887a0 20 read_log 192888'15828 (0'0) promote cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.345.0:2178 2014-08-11 17:05:51.606176 2014-08-13 17:38:03.017737 7fe9bae887a0 20 read_log 192888'15829 (192888'15828) modify cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by client.942868.0:9558 2014-08-11 17:05:51.597437 2014-08-13 17:38:03.017749 7fe9bae887a0 20 read_log 192888'15832 (0'0) promote 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.345.0:2186 2014-08-11 17:05:53.846327 2014-08-13 17:38:03.017761 7fe9bae887a0 20 read_log 192888'15833 (192888'15832) modify 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by client.942868.0:10177 2014-08-11 17:05:53.837724 2014-08-13 17:38:03.017773 7fe9bae887a0 20 read_log 192888'15834 (0'0) promote f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by osd.345.0:2188 2014-08-11 17:05:53.909740 2014-08-13 17:38:03.017785 7fe9bae887a0 20 read_log 192888'15835 (192888'15834) modify f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.942868.0:10195 2014-08-11 17:05:53.902240 2014-08-13 17:38:03.017798 7fe9bae887a0 20 read_log 192888'15836 (0'0) promote d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.345.0:2200 2014-08-11 17:05:56.661238 2014-08-13 17:38:03.017810 7fe9bae887a0 20 read_log 192888'15837 (192888'15836) modify d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by client.942868.0:10962 2014-08-11 17:05:56.651483 2014-08-13 17:38:03.017822 7fe9bae887a0 20 read_log 192888'15838 (0'0) promote f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by osd.345.0:2204 2014-08-11 17:05:58.645975 2014-08-13 17:38:03.017834 7fe9bae887a0 20 read_log 192888'15839 (192888'15838) modify f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.942868.0:11493 2014-08-11 17:05:58.638379 2014-08-13 17:38:03.017846 7fe9bae887a0 20 read_log 192888'15840 (0'0) promote d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.345.0:2212 2014-08-11 17:05:59.299350 2014-08-13 17:38:03.017858 7fe9bae887a0 20 read_log 192888'15841 (192888'15840) modify d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by client.942868.0:11675 2014-08-11 17:05:59.290480 2014-08-13 17:38:03.017870 7fe9bae887a0 20 read_log 192888'15850 (0'0) promote 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by osd.345.0:2240 2014-08-11 17:06:07.703040 2014-08-13 17:38:03.017882 7fe9bae887a0 20 read_log 192888'15851 (192888'15850) modify 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.942868.0:13885 2014-08-11 17:06:07.694867 2014-08-13 17:38:03.017894 7fe9bae887a0 20 read_log 192888'15856 (0'0) promote 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by osd.345.0:2260 2014-08-11 17:06:11.442130 2014-08-13 17:38:03.017906 7fe9bae887a0 20 read_log 192888'15857 (192888'15856) modify 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.942868.0:14800 2014-08-11 17:06:11.434797 2014-08-13 17:38:03.017918 7fe9bae887a0 20 read_log 192888'15872 (0'0) promote 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by osd.345.0:2384 2014-08-11 17:06:47.954521 2014-08-13 17:38:03.017930 7fe9bae887a0 20 read_log 192888'15873 (192888'15872) modify 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.942868.0:24338 2014-08-11 17:06:47.946456 2014-08-13 17:38:03.017952 7fe9bae887a0 20 read_log 192888'15874 (0'0) promote 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by osd.345.0:2390 2014-08-11 17:06:49.090638 2014-08-13 17:38:03.017965 7fe9bae887a0 20 read_log 192888'15875 (192888'15874) modify 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.942868.0:24642 2014-08-11 17:06:49.081543 2014-08-13 17:38:03.017977 7fe9bae887a0 20 read_log 192888'15876 (0'0) promote f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by osd.345.0:2394 2014-08-11 17:06:49.906735 2014-08-13 17:38:03.017989 7fe9bae887a0 20 read_log 192888'15877 (192888'15876) modify f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.942868.0:24863 2014-08-11 17:06:49.896991 2014-08-13 17:38:03.018001 7fe9bae887a0 20 read_log 192888'15878 (0'0) promote c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.345.0:2408 2014-08-11 17:06:54.416271 2014-08-13 17:38:03.018013 7fe9bae887a0 20 read_log 192888'15879 (192888'15878) modify c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by client.942868.0:26004 2014-08-11 17:06:54.408208 2014-08-13 17:38:03.018025 7fe9bae887a0 20 read_log 192888'15890 (0'0) promote 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by osd.345.0:2428 2014-08-11 17:06:57.901890 2014-08-13 17:38:03.018040 7fe9bae887a0 20 read_log 192888'15891 (192888'15890) modify 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.942868.0:26911 2014-08-11 17:06:57.892216 2014-08-13 17:38:03.018052 7fe9bae887a0 20 read_log 192888'15898 (0'0) promote e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by osd.345.0:2446 2014-08-11 17:07:02.715518 2014-08-13 17:38:03.018064 7fe9bae887a0 20 read_log 192888'15899 (192888'15898) modify e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.942868.0:28197 2014-08-11 17:07:02.707758 2014-08-13 17:38:03.018077 7fe9bae887a0 20 read_log 192888'15900 (0'0) promote b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by osd.345.0:2448 2014-08-11 17:07:03.020422 2014-08-13 17:38:03.018091 7fe9bae887a0 20 read_log 192888'15901 (192888'15900) modify b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.942868.0:28281 2014-08-11 17:07:03.012510 2014-08-13 17:38:03.018104 7fe9bae887a0 20 read_log 192888'15906 (192888'15791) modify 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by client.942868.0:31298 2014-08-11 17:07:15.860715 2014-08-13 17:38:03.018117 7fe9bae887a0 20 read_log 192888'15910 (192888'15799) modify a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.942868.0:34366 2014-08-11 17:07:16.767947 2014-08-13 17:38:03.018129 7fe9bae887a0 20 read_log 192888'15911 (192888'15801) modify 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by client.942868.0:34867 2014-08-11 17:07:16.918834 2014-08-13 17:38:03.018141 7fe9bae887a0 20 read_log 192888'15912 (192888'15803) modify ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.942868.0:35099 2014-08-11 17:07:16.984186 2014-08-13 17:38:03.018158 7fe9bae887a0 20 read_log 192888'15914 (192888'15807) modify 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.942868.0:35938 2014-08-11 17:07:17.202147 2014-08-13 17:38:03.018190 7fe9bae887a0 20 read_log 192888'15915 (192888'15809) modify 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.942868.0:36102 2014-08-11 17:07:17.242031 2014-08-13 17:38:03.018203 7fe9bae887a0 20 read_log 192888'15918 (192888'15815) modify a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by client.942868.0:38361 2014-08-11 17:07:17.864781 2014-08-13 17:38:03.018215 7fe9bae887a0 20 read_log 192888'15919 (192888'15817) modify ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.942868.0:38475 2014-08-11 17:07:17.897641 2014-08-13 17:38:03.018235 7fe9bae887a0 20 read_log 192888'15920 (192888'15819) modify 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.942868.0:38760 2014-08-11 17:07:17.980934 2014-08-13 17:38:03.018248 7fe9bae887a0 20 read_log 192888'15921 (192888'15821) modify 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.942868.0:39365 2014-08-11 17:07:18.162413 2014-08-13 17:38:03.018260 7fe9bae887a0 20 read_log 192888'15922 (192888'15823) modify 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.942868.0:40091 2014-08-11 17:07:18.362436 2014-08-13 17:38:03.018277 7fe9bae887a0 20 read_log 192888'15923 (192888'15825) modify 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by client.942868.0:40140 2014-08-11 17:07:18.374403 2014-08-13 17:38:03.018289 7fe9bae887a0 20 read_log 192888'15925 (192888'15829) modify cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by client.942868.0:40824 2014-08-11 17:07:18.557510 2014-08-13 17:38:03.018301 7fe9bae887a0 20 read_log 192888'15927 (192888'15833) modify 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by client.942868.0:41443 2014-08-11 17:07:18.729836 2014-08-13 17:38:03.018313 7fe9bae887a0 20 read_log 192888'15928 (192888'15835) modify f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.942868.0:41461 2014-08-11 17:07:18.734076 2014-08-13 17:38:03.018325 7fe9bae887a0 20 read_log 192888'15929 (192888'15837) modify d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by client.942868.0:42228 2014-08-11 17:07:18.942368 2014-08-13 17:38:03.018338 7fe9bae887a0 20 read_log 192888'15930 (192888'15839) modify f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.942868.0:42759 2014-08-11 17:07:19.084647 2014-08-13 17:38:03.018350 7fe9bae887a0 20 read_log 192888'15931 (192888'15841) modify d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by client.942868.0:42941 2014-08-11 17:07:19.134618 2014-08-13 17:38:03.018362 7fe9bae887a0 20 read_log 192888'15936 (192888'15851) modify 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.942868.0:45151 2014-08-11 17:07:19.759652 2014-08-13 17:38:03.018374 7fe9bae887a0 20 read_log 192888'15939 (192888'15857) modify 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.942868.0:46066 2014-08-11 17:07:20.011743 2014-08-13 17:38:03.018386 7fe9bae887a0 20 read_log 192888'15947 (192888'15873) modify 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.942868.0:55604 2014-08-11 17:07:22.899045 2014-08-13 17:38:03.018398 7fe9bae887a0 20 read_log 192888'15948 (192888'15875) modify 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.942868.0:55908 2014-08-11 17:07:22.992938 2014-08-13 17:38:03.018410 7fe9bae887a0 20 read_log 192888'15949 (192888'15877) modify f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.942868.0:56129 2014-08-11 17:07:23.059950 2014-08-13 17:38:03.018422 7fe9bae887a0 20 read_log 192888'15950 (192888'15879) modify c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by client.942868.0:57270 2014-08-11 17:07:23.408376 2014-08-13 17:38:03.018436 7fe9bae887a0 20 read_log 192888'15956 (192888'15891) modify 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.942868.0:58177 2014-08-11 17:07:23.664174 2014-08-13 17:38:03.018449 7fe9bae887a0 20 read_log 192888'15960 (192888'15899) modify e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.942868.0:59463 2014-08-11 17:07:24.053975 2014-08-13 17:38:03.018461 7fe9bae887a0 20 read_log 192888'15961 (192888'15901) modify b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.942868.0:59547 2014-08-11 17:07:24.077643 2014-08-13 17:38:03.018474 7fe9bae887a0 20 read_log 193305'15964 (0'0) promote 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:2 2014-08-11 18:42:33.589303 2014-08-13 17:38:03.018493 7fe9bae887a0 20 read_log 193305'15965 (193305'15964) modify 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by client.951337.0:162 2014-08-11 18:42:33.568773 2014-08-13 17:38:03.018506 7fe9bae887a0 20 read_log 193305'15966 (0'0) promote 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by osd.374.0:18 2014-08-11 18:42:36.025992 2014-08-13 17:38:03.018518 7fe9bae887a0 20 read_log 193305'15967 (193305'15966) modify 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.951337.0:737 2014-08-11 18:42:36.017516 2014-08-13 17:38:03.018530 7fe9bae887a0 20 read_log 193305'15968 (0'0) promote b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:61 2014-08-11 18:42:41.852589 2014-08-13 17:38:03.018542 7fe9bae887a0 20 read_log 193305'15969 (193305'15968) modify b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by client.951337.0:2301 2014-08-11 18:42:41.844652 2014-08-13 17:38:03.018555 7fe9bae887a0 20 read_log 193305'15970 (0'0) promote dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:69 2014-08-11 18:42:42.959895 2014-08-13 17:38:03.018567 7fe9bae887a0 20 read_log 193305'15971 (193305'15970) modify dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by client.951337.0:2606 2014-08-11 18:42:42.951618 2014-08-13 17:38:03.018579 7fe9bae887a0 20 read_log 193305'15972 (0'0) promote bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by osd.374.0:75 2014-08-11 18:42:43.913121 2014-08-13 17:38:03.018590 7fe9bae887a0 20 read_log 193305'15973 (193305'15972) modify bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.951337.0:2868 2014-08-11 18:42:43.904850 2014-08-13 17:38:03.018603 7fe9bae887a0 20 read_log 193305'15974 (0'0) promote e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by osd.374.0:169 2014-08-11 18:42:58.719861 2014-08-13 17:38:03.018615 7fe9bae887a0 20 read_log 193305'15975 (193305'15974) modify e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.951337.0:6910 2014-08-11 18:42:58.712353 2014-08-13 17:38:03.018627 7fe9bae887a0 20 read_log 193305'15976 (0'0) promote fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by osd.374.0:177 2014-08-11 18:42:59.933916 2014-08-13 17:38:03.018639 7fe9bae887a0 20 read_log 193305'15977 (193305'15976) modify fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.951337.0:7247 2014-08-11 18:42:59.926062 2014-08-13 17:38:03.018657 7fe9bae887a0 20 read_log 193305'15978 (0'0) promote c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:229 2014-08-11 18:43:08.668625 2014-08-13 17:38:03.018680 7fe9bae887a0 20 read_log 193305'15979 (193305'15978) modify c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by client.951337.0:9555 2014-08-11 18:43:08.660899 2014-08-13 17:38:03.018692 7fe9bae887a0 20 read_log 193305'15980 (0'0) promote ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by osd.374.0:233 2014-08-11 18:43:09.907456 2014-08-13 17:38:03.018705 7fe9bae887a0 20 read_log 193305'15981 (193305'15980) modify ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.951337.0:9893 2014-08-11 18:43:09.899852 2014-08-13 17:38:03.018717 7fe9bae887a0 20 read_log 193305'15982 (0'0) promote 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by osd.374.0:243 2014-08-11 18:43:12.413963 2014-08-13 17:38:03.018729 7fe9bae887a0 20 read_log 193305'15983 (193305'15982) modify 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.951337.0:10571 2014-08-11 18:43:12.406865 2014-08-13 17:38:03.018741 7fe9bae887a0 20 read_log 193305'15984 (0'0) promote 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by osd.374.0:259 2014-08-11 18:43:15.444564 2014-08-13 17:38:03.018778 7fe9bae887a0 20 read_log 193305'15985 (193305'15984) modify 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.951337.0:11339 2014-08-11 18:43:15.436638 2014-08-13 17:38:03.018794 7fe9bae887a0 20 read_log 193305'15986 (0'0) promote aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by osd.374.0:261 2014-08-11 18:43:15.971798 2014-08-13 17:38:03.018807 7fe9bae887a0 20 read_log 193305'15987 (193305'15986) modify aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.951337.0:11479 2014-08-11 18:43:15.963451 2014-08-13 17:38:03.018825 7fe9bae887a0 20 read_log 193305'15988 (0'0) promote ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:314 2014-08-11 18:43:23.728779 2014-08-13 17:38:03.018838 7fe9bae887a0 20 read_log 193305'15989 (193305'15988) modify ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by client.951337.0:13544 2014-08-11 18:43:23.721100 2014-08-13 17:38:03.018850 7fe9bae887a0 20 read_log 193305'15990 (0'0) promote fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:326 2014-08-11 18:43:25.133702 2014-08-13 17:38:03.018862 7fe9bae887a0 20 read_log 193305'15991 (193305'15990) modify fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by client.951337.0:13930 2014-08-11 18:43:25.125897 2014-08-13 17:38:03.018874 7fe9bae887a0 20 read_log 193305'15992 (0'0) promote 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by osd.374.0:418 2014-08-11 18:43:38.436804 2014-08-13 17:38:03.018886 7fe9bae887a0 20 read_log 193305'15993 (193305'15992) modify 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.951337.0:17746 2014-08-11 18:43:38.429380 2014-08-13 17:38:03.018898 7fe9bae887a0 20 read_log 193305'15994 (0'0) promote accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by osd.374.0:436 2014-08-11 18:43:40.385470 2014-08-13 17:38:03.018909 7fe9bae887a0 20 read_log 193305'15995 (193305'15994) modify accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.951337.0:18297 2014-08-11 18:43:40.378237 2014-08-13 17:38:03.018921 7fe9bae887a0 20 read_log 193305'15996 (0'0) promote c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by osd.374.0:450 2014-08-11 18:43:42.849426 2014-08-13 17:38:03.018934 7fe9bae887a0 20 read_log 193305'15997 (193305'15996) modify c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.951337.0:18973 2014-08-11 18:43:42.838944 2014-08-13 17:38:03.018946 7fe9bae887a0 20 read_log 193305'15998 (0'0) promote 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:452 2014-08-11 18:43:43.665472 2014-08-13 17:38:03.018958 7fe9bae887a0 20 read_log 193305'15999 (193305'15998) modify 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by client.951337.0:19183 2014-08-11 18:43:43.657263 2014-08-13 17:38:03.018970 7fe9bae887a0 20 read_log 193305'16000 (0'0) promote f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by osd.374.0:468 2014-08-11 18:43:46.535369 2014-08-13 17:38:03.018982 7fe9bae887a0 20 read_log 193305'16001 (193305'16000) modify f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.951337.0:19978 2014-08-11 18:43:46.526511 2014-08-13 17:38:03.018994 7fe9bae887a0 20 read_log 193305'16002 (0'0) promote 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by osd.374.0:496 2014-08-11 18:43:50.893444 2014-08-13 17:38:03.019006 7fe9bae887a0 20 read_log 193305'16003 (193305'16002) modify 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.951337.0:21188 2014-08-11 18:43:50.883961 2014-08-13 17:38:03.019018 7fe9bae887a0 20 read_log 193305'16004 (0'0) promote 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:518 2014-08-11 18:43:54.164507 2014-08-13 17:38:03.019036 7fe9bae887a0 20 read_log 193305'16005 (193305'16004) modify 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by client.951337.0:22094 2014-08-11 18:43:54.156202 2014-08-13 17:38:03.019049 7fe9bae887a0 20 read_log 193305'16006 (0'0) promote afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by osd.374.0:552 2014-08-11 18:43:59.695747 2014-08-13 17:38:03.019061 7fe9bae887a0 20 read_log 193305'16007 (193305'16006) modify afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.951337.0:23632 2014-08-11 18:43:59.688466 2014-08-13 17:38:03.019073 7fe9bae887a0 20 read_log 193305'16008 (0'0) promote fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by osd.374.0:558 2014-08-11 18:44:00.115830 2014-08-13 17:38:03.019085 7fe9bae887a0 20 read_log 193305'16009 (193305'16008) modify fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.951337.0:23751 2014-08-11 18:44:00.108332 2014-08-13 17:38:03.019097 7fe9bae887a0 20 read_log 193305'16010 (0'0) promote bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by osd.374.0:567 2014-08-11 18:44:00.914886 2014-08-13 17:38:03.019109 7fe9bae887a0 20 read_log 193305'16011 (193305'16010) modify bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.951337.0:23971 2014-08-11 18:44:00.908506 2014-08-13 17:38:03.019121 7fe9bae887a0 20 read_log 193305'16012 (0'0) promote 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by osd.374.0:573 2014-08-11 18:44:01.679936 2014-08-13 17:38:03.019133 7fe9bae887a0 20 read_log 193305'16013 (193305'16012) modify 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.951337.0:24174 2014-08-11 18:44:01.670748 2014-08-13 17:38:03.019146 7fe9bae887a0 20 read_log 193305'16014 (0'0) promote 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by osd.374.0:589 2014-08-11 18:44:03.699386 2014-08-13 17:38:03.019163 7fe9bae887a0 20 read_log 193305'16015 (193305'16014) modify 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.951337.0:24744 2014-08-11 18:44:03.690280 2014-08-13 17:38:03.019175 7fe9bae887a0 20 read_log 193305'16016 (0'0) promote f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by osd.374.0:663 2014-08-11 18:44:15.034704 2014-08-13 17:38:03.019187 7fe9bae887a0 20 read_log 193305'16017 (193305'16016) modify f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.951337.0:27825 2014-08-11 18:44:15.027941 2014-08-13 17:38:03.019200 7fe9bae887a0 20 read_log 193305'16018 (193305'16018) modify 2ec/hit_set_24.2ec_archive_2014-08-11 18:39:19.769124_2014-08-11 18:44:23.450177/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:23.450471 2014-08-13 17:38:03.019213 7fe9bae887a0 20 read_log 193305'16019 (0'0) promote d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:718 2014-08-11 18:44:23.454196 2014-08-13 17:38:03.019225 7fe9bae887a0 20 read_log 193305'16020 (193305'16019) modify d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by client.951337.0:30108 2014-08-11 18:44:23.441590 2014-08-13 17:38:03.019237 7fe9bae887a0 20 read_log 193305'16021 (0'0) promote ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by osd.374.0:724 2014-08-11 18:44:24.495194 2014-08-13 17:38:03.019249 7fe9bae887a0 20 read_log 193305'16022 (193305'16021) modify ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.951337.0:30370 2014-08-11 18:44:24.486716 2014-08-13 17:38:03.019261 7fe9bae887a0 20 read_log 193305'16023 (193305'15965) modify 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by client.951337.0:32873 2014-08-11 18:44:33.922295 2014-08-13 17:38:03.019284 7fe9bae887a0 20 read_log 193305'16024 (193305'15967) modify 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.951337.0:33448 2014-08-11 18:44:34.432454 2014-08-13 17:38:03.019297 7fe9bae887a0 20 read_log 193305'16025 (193305'15969) modify b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by client.951337.0:35012 2014-08-11 18:44:35.328807 2014-08-13 17:38:03.019309 7fe9bae887a0 20 read_log 193305'16026 (193305'15971) modify dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by client.951337.0:35317 2014-08-11 18:44:35.506860 2014-08-13 17:38:03.019321 7fe9bae887a0 20 read_log 193305'16027 (193305'15973) modify bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.951337.0:35579 2014-08-11 18:44:35.666618 2014-08-13 17:38:03.019334 7fe9bae887a0 20 read_log 193305'16028 (193305'15975) modify e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.951337.0:39621 2014-08-11 18:44:38.114615 2014-08-13 17:38:03.019346 7fe9bae887a0 20 read_log 193305'16029 (193305'15977) modify fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.951337.0:39958 2014-08-11 18:44:38.237807 2014-08-13 17:38:03.019357 7fe9bae887a0 20 read_log 193305'16030 (193305'15979) modify c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by client.951337.0:42266 2014-08-11 18:44:39.169567 2014-08-13 17:38:03.019369 7fe9bae887a0 20 read_log 193305'16031 (193305'15981) modify ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.951337.0:42604 2014-08-11 18:44:39.311978 2014-08-13 17:38:03.019381 7fe9bae887a0 20 read_log 193305'16032 (193305'15983) modify 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.951337.0:43282 2014-08-11 18:44:39.542699 2014-08-13 17:38:03.019393 7fe9bae887a0 20 read_log 193305'16033 (193305'15985) modify 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.951337.0:44050 2014-08-11 18:44:39.780677 2014-08-13 17:38:03.019405 7fe9bae887a0 20 read_log 193305'16034 (193305'15987) modify aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.951337.0:44190 2014-08-11 18:44:39.817235 2014-08-13 17:38:03.019436 7fe9bae887a0 20 read_log 193305'16035 (193305'15989) modify ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by client.951337.0:46255 2014-08-11 18:44:40.432902 2014-08-13 17:38:03.019452 7fe9bae887a0 20 read_log 193305'16036 (193305'15991) modify fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by client.951337.0:46641 2014-08-11 18:44:40.544893 2014-08-13 17:38:03.019464 7fe9bae887a0 20 read_log 193305'16037 (193305'15993) modify 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.951337.0:50457 2014-08-11 18:44:41.671977 2014-08-13 17:38:03.019476 7fe9bae887a0 20 read_log 193305'16038 (193305'15995) modify accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.951337.0:51008 2014-08-11 18:44:41.826160 2014-08-13 17:38:03.019488 7fe9bae887a0 20 read_log 193305'16039 (193305'15997) modify c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.951337.0:51684 2014-08-11 18:44:42.023248 2014-08-13 17:38:03.019501 7fe9bae887a0 20 read_log 193305'16040 (193305'15999) modify 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by client.951337.0:51894 2014-08-11 18:44:42.080729 2014-08-13 17:38:03.019513 7fe9bae887a0 20 read_log 193305'16041 (193305'16001) modify f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.951337.0:52689 2014-08-11 18:44:42.309714 2014-08-13 17:38:03.019525 7fe9bae887a0 20 read_log 193305'16042 (193305'16003) modify 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.951337.0:53899 2014-08-11 18:44:42.668428 2014-08-13 17:38:03.019537 7fe9bae887a0 20 read_log 193305'16043 (193305'16005) modify 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by client.951337.0:54805 2014-08-11 18:44:42.944183 2014-08-13 17:38:03.019559 7fe9bae887a0 20 read_log 193305'16044 (193305'16007) modify afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.951337.0:56343 2014-08-11 18:44:43.391063 2014-08-13 17:38:03.019572 7fe9bae887a0 20 read_log 193305'16045 (193305'16009) modify fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.951337.0:56462 2014-08-11 18:44:43.425998 2014-08-13 17:38:03.019584 7fe9bae887a0 20 read_log 193305'16046 (193305'16011) modify bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.951337.0:56682 2014-08-11 18:44:43.497290 2014-08-13 17:38:03.019596 7fe9bae887a0 20 read_log 193305'16047 (193305'16013) modify 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.951337.0:56885 2014-08-11 18:44:43.563611 2014-08-13 17:38:03.019608 7fe9bae887a0 20 read_log 193305'16048 (193305'16015) modify 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.951337.0:57455 2014-08-11 18:44:43.759270 2014-08-13 17:38:03.019620 7fe9bae887a0 20 read_log 193305'16049 (193305'16017) modify f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.951337.0:60536 2014-08-11 18:44:44.658970 2014-08-13 17:38:03.019632 7fe9bae887a0 20 read_log 193305'16050 (193305'16020) modify d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by client.951337.0:62819 2014-08-11 18:44:45.336165 2014-08-13 17:38:03.019644 7fe9bae887a0 20 read_log 193305'16051 (193305'16022) modify ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.951337.0:63081 2014-08-11 18:44:45.404051 2014-08-13 17:38:03.019656 7fe9bae887a0 20 read_log 194209'16052 (192888'15733) clean eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.374.0:1947 2014-08-12 14:07:58.520520 2014-08-13 17:38:03.019668 7fe9bae887a0 20 read_log 194209'16053 (192888'15756) clean d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.374.0:1948 2014-08-12 14:07:58.520950 2014-08-13 17:38:03.019680 7fe9bae887a0 20 read_log 194209'16054 (192888'15727) clean daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.374.0:1949 2014-08-12 14:07:58.521359 2014-08-13 17:38:03.019693 7fe9bae887a0 20 read_log 194209'16055 (192888'15906) clean 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.374.0:1950 2014-08-12 14:07:58.522593 2014-08-13 17:38:03.019705 7fe9bae887a0 20 read_log 194209'16056 (192888'15779) clean 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.374.0:1951 2014-08-12 14:07:58.533857 2014-08-13 17:38:03.019717 7fe9bae887a0 20 read_log 194209'16057 (192888'15763) clean ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.374.0:1952 2014-08-12 14:07:58.534304 2014-08-13 17:38:03.019729 7fe9bae887a0 20 read_log 194209'16058 (193305'16043) clean 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:1953 2014-08-12 14:07:58.538127 2014-08-13 17:38:03.019741 7fe9bae887a0 20 read_log 194209'16059 (192834'15587) clean 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by osd.374.0:1954 2014-08-12 14:07:58.541366 2014-08-13 17:38:03.019753 7fe9bae887a0 20 read_log 194209'16060 (193305'16040) clean 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:1955 2014-08-12 14:07:58.544816 2014-08-13 17:38:03.019765 7fe9bae887a0 20 read_log 194209'16061 (192834'15088) clean 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by osd.374.0:1956 2014-08-12 14:07:58.548033 2014-08-13 17:38:03.019778 7fe9bae887a0 20 read_log 194209'16062 (192888'15936) clean 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by osd.374.0:1957 2014-08-12 14:07:58.551329 2014-08-13 17:38:03.019797 7fe9bae887a0 20 read_log 194209'16063 (193305'16050) clean d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:1958 2014-08-12 14:07:58.557403 2014-08-13 17:38:03.019810 7fe9bae887a0 20 read_log 194209'16064 (192834'15057) clean 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.374.0:1959 2014-08-12 14:07:58.560464 2014-08-13 17:38:03.019822 7fe9bae887a0 20 read_log 194209'16065 (192834'15046) clean 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.374.0:1960 2014-08-12 14:07:58.560756 2014-08-13 17:38:03.019834 7fe9bae887a0 20 read_log 194209'16066 (192888'15774) clean afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.374.0:1961 2014-08-12 14:07:58.563943 2014-08-13 17:38:03.019846 7fe9bae887a0 20 read_log 194209'16067 (192834'15576) clean d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.374.0:1962 2014-08-12 14:07:58.567603 2014-08-13 17:38:03.019858 7fe9bae887a0 20 read_log 194209'16068 (192888'15911) clean 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.374.0:1963 2014-08-12 14:07:58.570907 2014-08-13 17:38:03.019870 7fe9bae887a0 20 read_log 194209'16069 (192888'15925) clean cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.374.0:1964 2014-08-12 14:07:58.574167 2014-08-13 17:38:03.019883 7fe9bae887a0 20 read_log 194209'16070 (193305'16035) clean ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:1965 2014-08-12 14:07:58.577522 2014-08-13 17:38:03.019895 7fe9bae887a0 20 read_log 194209'16071 (192888'15923) clean 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.374.0:1966 2014-08-12 14:07:58.580933 2014-08-13 17:38:03.019907 7fe9bae887a0 20 read_log 194209'16072 (193305'16023) clean 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:1967 2014-08-12 14:07:58.586957 2014-08-13 17:38:03.019922 7fe9bae887a0 20 read_log 194209'16073 (192834'14934) clean b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.374.0:1968 2014-08-12 14:07:58.587255 2014-08-13 17:38:03.019934 7fe9bae887a0 20 read_log 194209'16074 (192888'15950) clean c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.374.0:1969 2014-08-12 14:07:58.590499 2014-08-13 17:38:03.019947 7fe9bae887a0 20 read_log 194209'16075 (192888'15783) clean e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.374.0:1970 2014-08-12 14:07:58.596901 2014-08-13 17:38:03.019961 7fe9bae887a0 20 read_log 194209'16076 (192888'15918) clean a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.374.0:1971 2014-08-12 14:07:58.597266 2014-08-13 17:38:03.019973 7fe9bae887a0 20 read_log 194209'16077 (192834'14971) clean 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.374.0:1972 2014-08-12 14:07:58.600442 2014-08-13 17:38:03.019986 7fe9bae887a0 20 read_log 194209'16078 (193305'16036) clean fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:1973 2014-08-12 14:07:58.604024 2014-08-13 17:38:03.020000 7fe9bae887a0 20 read_log 194209'16079 (192888'15927) clean 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.374.0:1974 2014-08-12 14:07:58.607266 2014-08-13 17:38:03.020012 7fe9bae887a0 20 read_log 194209'16080 (193305'16030) clean c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:1975 2014-08-12 14:07:58.610585 2014-08-13 17:38:03.020027 7fe9bae887a0 20 read_log 194209'16081 (192834'15529) clean 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.374.0:1976 2014-08-12 14:07:58.613838 2014-08-13 17:38:03.020039 7fe9bae887a0 20 read_log 194209'16082 (192834'15011) clean bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.374.0:1977 2014-08-12 14:07:58.616941 2014-08-13 17:38:03.020058 7fe9bae887a0 20 read_log 194209'16083 (192834'15559) clean ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.374.0:1978 2014-08-12 14:07:58.620115 2014-08-13 17:38:03.020072 7fe9bae887a0 20 read_log 194209'16084 (192834'14950) clean 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.374.0:1979 2014-08-12 14:07:58.623165 2014-08-13 17:38:03.020085 7fe9bae887a0 20 read_log 194209'16085 (192834'15457) clean 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.374.0:1980 2014-08-12 14:07:58.626226 2014-08-13 17:38:03.020097 7fe9bae887a0 20 read_log 194209'16086 (192888'15738) clean 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.374.0:1981 2014-08-12 14:07:58.631939 2014-08-13 17:38:03.020111 7fe9bae887a0 20 read_log 194209'16087 (192834'15514) clean 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.374.0:1982 2014-08-12 14:07:58.635180 2014-08-13 17:38:03.020123 7fe9bae887a0 20 read_log 194209'16088 (192834'15000) clean 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.374.0:1983 2014-08-12 14:07:58.635506 2014-08-13 17:38:03.020135 7fe9bae887a0 20 read_log 194209'16089 (193305'16026) clean dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:1984 2014-08-12 14:07:58.638645 2014-08-13 17:38:03.020150 7fe9bae887a0 20 read_log 194209'16090 (192834'15432) clean d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.374.0:1985 2014-08-12 14:07:58.641903 2014-08-13 17:38:03.020169 7fe9bae887a0 20 read_log 194209'16091 (192834'15441) clean 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.374.0:1986 2014-08-12 14:07:58.645097 2014-08-13 17:38:03.020182 7fe9bae887a0 20 read_log 194209'16092 (192834'15467) clean 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.374.0:1987 2014-08-12 14:07:58.648203 2014-08-13 17:38:03.020194 7fe9bae887a0 20 read_log 194209'16093 (192888'15754) clean b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.374.0:1988 2014-08-12 14:07:58.652250 2014-08-13 17:38:03.020206 7fe9bae887a0 20 read_log 194209'16094 (192834'14960) clean 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.374.0:1989 2014-08-12 14:07:58.655939 2014-08-13 17:38:03.020218 7fe9bae887a0 20 read_log 194209'16095 (192888'15931) clean d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.374.0:1990 2014-08-12 14:07:58.659401 2014-08-13 17:38:03.020230 7fe9bae887a0 20 read_log 194209'16096 (193305'16025) clean b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:1991 2014-08-12 14:07:58.659662 2014-08-13 17:38:03.020242 7fe9bae887a0 20 read_log 194209'16097 (192834'14919) clean 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.374.0:1992 2014-08-12 14:07:58.663193 2014-08-13 17:38:03.020254 7fe9bae887a0 20 read_log 194209'16098 (192888'15746) clean 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.374.0:1993 2014-08-12 14:07:58.667229 2014-08-13 17:38:03.020267 7fe9bae887a0 20 read_log 194209'16099 (192888'15757) clean 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.374.0:1994 2014-08-12 14:07:58.670357 2014-08-13 17:38:03.020282 7fe9bae887a0 20 read_log 194209'16100 (192888'15929) clean d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.374.0:1995 2014-08-12 14:07:58.676080 2014-08-13 17:38:03.020294 7fe9bae887a0 20 read_log 194209'16101 (192834'14998) clean 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.374.0:1996 2014-08-12 14:07:58.676637 2014-08-13 17:38:03.020306 7fe9bae887a0 20 read_log 194209'16102 (192834'14907) clean b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.374.0:1997 2014-08-12 14:07:58.679578 2014-08-13 17:38:03.020327 7fe9bae887a0 20 read_log 194209'16103 (194209'16052) delete eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.374.0:1998 2014-08-12 14:07:58.715623 2014-08-13 17:38:03.020340 7fe9bae887a0 20 read_log 194209'16104 (194209'16053) delete d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.374.0:1999 2014-08-12 14:07:58.719320 2014-08-13 17:38:03.020353 7fe9bae887a0 20 read_log 194209'16105 (194209'16054) delete daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.374.0:2000 2014-08-12 14:07:58.722239 2014-08-13 17:38:03.020365 7fe9bae887a0 20 read_log 194209'16106 (194209'16055) delete 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.374.0:2001 2014-08-12 14:07:58.725393 2014-08-13 17:38:03.020377 7fe9bae887a0 20 read_log 194209'16107 (194209'16056) delete 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.374.0:2002 2014-08-12 14:07:58.729182 2014-08-13 17:38:03.020389 7fe9bae887a0 20 read_log 194209'16108 (194209'16057) delete ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.374.0:2003 2014-08-12 14:07:58.732796 2014-08-13 17:38:03.020401 7fe9bae887a0 20 read_log 194209'16109 (194209'16058) delete 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:2004 2014-08-12 14:07:58.736764 2014-08-13 17:38:03.020413 7fe9bae887a0 20 read_log 194209'16110 (194209'16060) delete 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:2005 2014-08-12 14:07:58.740501 2014-08-13 17:38:03.020427 7fe9bae887a0 20 read_log 194209'16111 (194209'16063) delete d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:2006 2014-08-12 14:07:58.744034 2014-08-13 17:38:03.020439 7fe9bae887a0 20 read_log 194209'16112 (194209'16070) delete ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:2007 2014-08-12 14:07:58.745651 2014-08-13 17:38:03.020451 7fe9bae887a0 20 read_log 194209'16113 (194209'16072) delete 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:2008 2014-08-12 14:07:58.749014 2014-08-13 17:38:03.020463 7fe9bae887a0 20 read_log 194209'16114 (194209'16078) delete fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:2009 2014-08-12 14:07:58.750747 2014-08-13 17:38:03.020475 7fe9bae887a0 20 read_log 194209'16115 (194209'16080) delete c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:2010 2014-08-12 14:07:58.754096 2014-08-13 17:38:03.020487 7fe9bae887a0 20 read_log 194209'16116 (194209'16089) delete dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:2011 2014-08-12 14:07:58.758915 2014-08-13 17:38:03.020499 7fe9bae887a0 20 read_log 194209'16117 (194209'16096) delete b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:2012 2014-08-12 14:07:58.760546 2014-08-13 17:38:03.020511 7fe9bae887a0 20 read_log 194209'16118 (192834'14908) clean 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.374.0:2013 2014-08-12 14:07:58.828989 2014-08-13 17:38:03.020523 7fe9bae887a0 20 read_log 194209'16119 (194209'16064) delete 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.374.0:2014 2014-08-12 14:08:03.831205 2014-08-13 17:38:03.020535 7fe9bae887a0 20 read_log 194209'16120 (194209'16065) delete 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.374.0:2015 2014-08-12 14:08:03.831662 2014-08-13 17:38:03.020547 7fe9bae887a0 20 read_log 194209'16121 (194209'16066) delete afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.374.0:2016 2014-08-12 14:08:03.832042 2014-08-13 17:38:03.020560 7fe9bae887a0 20 read_log 194209'16122 (194209'16067) delete d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.374.0:2017 2014-08-12 14:08:03.832431 2014-08-13 17:38:03.020599 7fe9bae887a0 20 read_log 194209'16123 (194209'16068) delete 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.374.0:2018 2014-08-12 14:08:03.835696 2014-08-13 17:38:03.020613 7fe9bae887a0 20 read_log 194209'16124 (194209'16069) delete cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.374.0:2019 2014-08-12 14:08:03.836546 2014-08-13 17:38:03.020625 7fe9bae887a0 20 read_log 194209'16125 (194209'16071) delete 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.374.0:2020 2014-08-12 14:08:03.836924 2014-08-13 17:38:03.020638 7fe9bae887a0 20 read_log 194209'16126 (194209'16073) delete b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.374.0:2021 2014-08-12 14:08:03.837861 2014-08-13 17:38:03.020650 7fe9bae887a0 20 read_log 194209'16127 (194209'16074) delete c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.374.0:2022 2014-08-12 14:08:03.840579 2014-08-13 17:38:03.020677 7fe9bae887a0 20 read_log 194209'16128 (194209'16075) delete e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.374.0:2023 2014-08-12 14:08:03.841327 2014-08-13 17:38:03.020707 7fe9bae887a0 20 read_log 194209'16129 (194209'16076) delete a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.374.0:2024 2014-08-12 14:08:03.841685 2014-08-13 17:38:03.020721 7fe9bae887a0 20 read_log 194209'16130 (194209'16077) delete 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.374.0:2025 2014-08-12 14:08:03.842455 2014-08-13 17:38:03.020734 7fe9bae887a0 20 read_log 194209'16131 (194209'16079) delete 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.374.0:2026 2014-08-12 14:08:03.846380 2014-08-13 17:38:03.020746 7fe9bae887a0 20 read_log 194209'16132 (194209'16081) delete 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.374.0:2027 2014-08-12 14:08:03.846757 2014-08-13 17:38:03.020758 7fe9bae887a0 20 read_log 194209'16133 (194209'16082) delete bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.374.0:2028 2014-08-12 14:08:03.847631 2014-08-13 17:38:03.020771 7fe9bae887a0 20 read_log 194209'16134 (194209'16083) delete ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.374.0:2029 2014-08-12 14:08:03.847998 2014-08-13 17:38:03.020783 7fe9bae887a0 20 read_log 194209'16135 (194209'16084) delete 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.374.0:2030 2014-08-12 14:08:03.851317 2014-08-13 17:38:03.020797 7fe9bae887a0 20 read_log 194209'16136 (194209'16085) delete 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.374.0:2031 2014-08-12 14:08:03.851686 2014-08-13 17:38:03.020809 7fe9bae887a0 20 read_log 194209'16137 (194209'16086) delete 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.374.0:2032 2014-08-12 14:08:03.852801 2014-08-13 17:38:03.020821 7fe9bae887a0 20 read_log 194209'16138 (194209'16087) delete 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.374.0:2033 2014-08-12 14:08:03.853221 2014-08-13 17:38:03.020833 7fe9bae887a0 20 read_log 194209'16139 (194209'16088) delete 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.374.0:2034 2014-08-12 14:08:03.856616 2014-08-13 17:38:03.020846 7fe9bae887a0 20 read_log 194209'16140 (194209'16090) delete d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.374.0:2035 2014-08-12 14:08:03.857036 2014-08-13 17:38:03.020858 7fe9bae887a0 20 read_log 194209'16141 (194209'16091) delete 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.374.0:2036 2014-08-12 14:08:03.857696 2014-08-13 17:38:03.020870 7fe9bae887a0 20 read_log 194209'16142 (194209'16092) delete 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.374.0:2037 2014-08-12 14:08:03.858056 2014-08-13 17:38:03.020889 7fe9bae887a0 20 read_log 194209'16143 (194209'16093) delete b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.374.0:2038 2014-08-12 14:08:03.861330 2014-08-13 17:38:03.020902 7fe9bae887a0 20 read_log 194209'16144 (194209'16094) delete 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.374.0:2039 2014-08-12 14:08:03.862198 2014-08-13 17:38:03.020914 7fe9bae887a0 20 read_log 194209'16145 (194209'16095) delete d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.374.0:2040 2014-08-12 14:08:03.862603 2014-08-13 17:38:03.020926 7fe9bae887a0 20 read_log 194209'16146 (194209'16097) delete 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.374.0:2041 2014-08-12 14:08:03.863440 2014-08-13 17:38:03.020938 7fe9bae887a0 20 read_log 194209'16147 (194209'16098) delete 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.374.0:2042 2014-08-12 14:08:03.866342 2014-08-13 17:38:03.020950 7fe9bae887a0 20 read_log 194209'16148 (194209'16099) delete 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.374.0:2043 2014-08-12 14:08:03.867225 2014-08-13 17:38:03.020962 7fe9bae887a0 20 read_log 194209'16149 (194209'16100) delete d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.374.0:2044 2014-08-12 14:08:03.867596 2014-08-13 17:38:03.020974 7fe9bae887a0 20 read_log 194209'16150 (194209'16101) delete 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.374.0:2045 2014-08-12 14:08:03.868457 2014-08-13 17:38:03.020986 7fe9bae887a0 20 read_log 194209'16151 (194209'16102) delete b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.374.0:2046 2014-08-12 14:08:03.871328 2014-08-13 17:38:03.020998 7fe9bae887a0 20 read_log 194209'16152 (194209'16118) delete 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.374.0:2047 2014-08-12 14:08:03.872198 2014-08-13 17:38:03.021011 7fe9bae887a0 20 read_log 194215'16153 (194215'16153) modify 2ec/hit_set_24.2ec_archive_2014-08-12 15:32:26.012127_2014-08-12 15:47:44.557466/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:47:44.557704 2014-08-13 17:38:03.021024 7fe9bae887a0 20 read_log 194215'16154 (193305'16018) delete 2ec/hit_set_24.2ec_archive_2014-08-11 18:39:19.769124_2014-08-11 18:44:23.450177/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:47:44.557704 2014-08-13 17:38:03.021036 7fe9bae887a0 20 read_log 194215'16155 (192888'15745) clean cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by osd.374.0:3981 2014-08-12 15:47:44.560989 2014-08-13 17:38:03.021048 7fe9bae887a0 20 read_log 194215'16156 (194215'16155) delete cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.2174595.0:154013 0.000000 2014-08-13 17:38:03.021059 7fe9bae887a0 20 read_log 194215'16157 (192888'15930) clean f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by osd.374.0:3983 2014-08-12 15:47:44.568991 2014-08-13 17:38:03.021072 7fe9bae887a0 20 read_log 194215'16158 (194215'16157) delete f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.2174595.0:154015 0.000000 2014-08-13 17:38:03.021082 7fe9bae887a0 20 read_log 194215'16159 (192888'15728) clean 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by osd.374.0:3985 2014-08-12 15:47:44.576581 2014-08-13 17:38:03.021094 7fe9bae887a0 20 read_log 194215'16160 (194215'16159) delete 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.2174595.0:154017 0.000000 2014-08-13 17:38:03.021105 7fe9bae887a0 20 read_log 194215'16161 (193305'16049) clean f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by osd.374.0:3987 2014-08-12 15:47:44.584405 2014-08-13 17:38:03.021127 7fe9bae887a0 20 read_log 194215'16162 (194215'16161) delete f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.2174595.0:154019 0.000000 2014-08-13 17:38:03.021139 7fe9bae887a0 20 read_log 194215'16163 (192834'15536) clean 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by osd.374.0:3989 2014-08-12 15:47:44.592385 2014-08-13 17:38:03.021151 7fe9bae887a0 20 read_log 194215'16164 (194215'16163) delete 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.2174595.0:154021 0.000000 2014-08-13 17:38:03.021171 7fe9bae887a0 20 read_log 194215'16165 (192888'15960) clean e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by osd.374.0:3991 2014-08-12 15:47:44.600770 2014-08-13 17:38:03.021183 7fe9bae887a0 20 read_log 194215'16166 (194215'16165) delete e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.2174595.0:154023 0.000000 2014-08-13 17:38:03.021194 7fe9bae887a0 20 read_log 194215'16167 (193305'16048) clean 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by osd.374.0:3993 2014-08-12 15:47:44.607174 2014-08-13 17:38:03.021206 7fe9bae887a0 20 read_log 194215'16168 (194215'16167) delete 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.2174595.0:154025 0.000000 2014-08-13 17:38:03.021217 7fe9bae887a0 20 read_log 194215'16169 (192888'15726) clean 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by osd.374.0:3995 2014-08-12 15:47:44.614105 2014-08-13 17:38:03.021229 7fe9bae887a0 20 read_log 194215'16170 (194215'16169) delete 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.2174595.0:154027 0.000000 2014-08-13 17:38:03.021240 7fe9bae887a0 20 read_log 194215'16171 (192834'15449) clean 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by osd.374.0:3997 2014-08-12 15:47:44.622503 2014-08-13 17:38:03.021252 7fe9bae887a0 20 read_log 194215'16172 (194215'16171) delete 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.2174595.0:154029 0.000000 2014-08-13 17:38:03.021262 7fe9bae887a0 20 read_log 194215'16173 (193305'16029) clean fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by osd.374.0:3999 2014-08-12 15:47:44.630005 2014-08-13 17:38:03.021275 7fe9bae887a0 20 read_log 194215'16174 (194215'16173) delete fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.2174595.0:154031 0.000000 2014-08-13 17:38:03.021286 7fe9bae887a0 20 read_log 194215'16175 (193305'16044) clean afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by osd.374.0:4001 2014-08-12 15:47:44.636915 2014-08-13 17:38:03.021298 7fe9bae887a0 20 read_log 194215'16176 (194215'16175) delete afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.2174595.0:154033 0.000000 2014-08-13 17:38:03.021308 7fe9bae887a0 20 read_log 194215'16177 (192888'15742) clean 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by osd.374.0:4003 2014-08-12 15:47:44.644755 2014-08-13 17:38:03.021320 7fe9bae887a0 20 read_log 194215'16178 (194215'16177) delete 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.2174595.0:154035 0.000000 2014-08-13 17:38:03.021331 7fe9bae887a0 20 read_log 194215'16179 (192888'15730) clean 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by osd.374.0:4005 2014-08-12 15:47:44.652045 2014-08-13 17:38:03.021344 7fe9bae887a0 20 read_log 194215'16180 (194215'16179) delete 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.2174595.0:154037 0.000000 2014-08-13 17:38:03.021355 7fe9bae887a0 20 read_log 194215'16181 (192834'15053) clean 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by osd.374.0:4007 2014-08-12 15:47:44.659616 2014-08-13 17:38:03.021374 7fe9bae887a0 20 read_log 194215'16182 (194215'16181) delete 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.2174595.0:154039 0.000000 2014-08-13 17:38:03.021386 7fe9bae887a0 20 read_log 194215'16183 (192834'15571) clean 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by osd.374.0:4009 2014-08-12 15:47:44.667248 2014-08-13 17:38:03.021398 7fe9bae887a0 20 read_log 194215'16184 (194215'16183) delete 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.2174595.0:154041 0.000000 2014-08-13 17:38:03.021409 7fe9bae887a0 20 read_log 194215'16185 (192834'15519) clean 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by osd.374.0:4011 2014-08-12 15:47:44.676092 2014-08-13 17:38:03.021421 7fe9bae887a0 20 read_log 194215'16186 (194215'16185) delete 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.2174595.0:154043 0.000000 2014-08-13 17:38:03.021432 7fe9bae887a0 20 read_log 194215'16187 (192888'15786) clean dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by osd.374.0:4013 2014-08-12 15:47:44.683494 2014-08-13 17:38:03.021444 7fe9bae887a0 20 read_log 194215'16188 (194215'16187) delete dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.2174595.0:154045 0.000000 2014-08-13 17:38:03.021454 7fe9bae887a0 20 read_log 194215'16189 (193305'16033) clean 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by osd.374.0:4015 2014-08-12 15:47:44.690566 2014-08-13 17:38:03.021466 7fe9bae887a0 20 read_log 194215'16190 (194215'16189) delete 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.2174595.0:154047 0.000000 2014-08-13 17:38:03.021477 7fe9bae887a0 20 read_log 194215'16191 (192834'15550) clean 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by osd.374.0:4017 2014-08-12 15:47:44.698085 2014-08-13 17:38:03.021489 7fe9bae887a0 20 read_log 194215'16192 (194215'16191) delete 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.2174595.0:154049 0.000000 2014-08-13 17:38:03.021499 7fe9bae887a0 20 read_log 194215'16193 (192888'15961) clean b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by osd.374.0:4019 2014-08-12 15:47:44.705139 2014-08-13 17:38:03.021512 7fe9bae887a0 20 read_log 194215'16194 (194215'16193) delete b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.2174595.0:154051 0.000000 2014-08-13 17:38:03.021523 7fe9bae887a0 20 read_log 194215'16195 (192834'15462) clean 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by osd.374.0:4021 2014-08-12 15:47:44.712973 2014-08-13 17:38:03.021535 7fe9bae887a0 20 read_log 194215'16196 (194215'16195) delete 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.2174595.0:154053 0.000000 2014-08-13 17:38:03.021546 7fe9bae887a0 20 read_log 194215'16197 (192834'15461) clean 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by osd.374.0:4023 2014-08-12 15:47:44.720387 2014-08-13 17:38:03.021558 7fe9bae887a0 20 read_log 194215'16198 (194215'16197) delete 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.2174595.0:154055 0.000000 2014-08-13 17:38:03.021568 7fe9bae887a0 20 read_log 194215'16199 (193305'16045) clean fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by osd.374.0:4025 2014-08-12 15:47:44.728176 2014-08-13 17:38:03.021581 7fe9bae887a0 20 read_log 194215'16200 (194215'16199) delete fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.2174595.0:154057 0.000000 2014-08-13 17:38:03.021591 7fe9bae887a0 20 read_log 194215'16201 (192888'15729) clean e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by osd.374.0:4027 2014-08-12 15:47:44.736506 2014-08-13 17:38:03.021610 7fe9bae887a0 20 read_log 194215'16202 (194215'16201) delete e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.2174595.0:154059 0.000000 2014-08-13 17:38:03.021621 7fe9bae887a0 20 read_log 194215'16203 (192888'15948) clean 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by osd.374.0:4029 2014-08-12 15:47:44.744133 2014-08-13 17:38:03.021633 7fe9bae887a0 20 read_log 194215'16204 (194215'16203) delete 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.2174595.0:154061 0.000000 2014-08-13 17:38:03.021644 7fe9bae887a0 20 read_log 194215'16205 (192834'15035) clean 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by osd.374.0:4031 2014-08-12 15:47:44.751819 2014-08-13 17:38:03.021656 7fe9bae887a0 20 read_log 194215'16206 (194215'16205) delete 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.2174595.0:154063 0.000000 2014-08-13 17:38:03.021667 7fe9bae887a0 20 read_log 194215'16207 (192888'15949) clean f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by osd.374.0:4033 2014-08-12 15:47:44.760453 2014-08-13 17:38:03.021679 7fe9bae887a0 20 read_log 194215'16208 (194215'16207) delete f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.2174595.0:154065 0.000000 2014-08-13 17:38:03.021690 7fe9bae887a0 20 read_log 194215'16209 (192834'15452) clean 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by osd.374.0:4035 2014-08-12 15:47:44.768942 2014-08-13 17:38:03.021702 7fe9bae887a0 20 read_log 194215'16210 (194215'16209) delete 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.2174595.0:154067 0.000000 2014-08-13 17:38:03.021713 7fe9bae887a0 20 read_log 194215'16211 (193305'16038) clean accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by osd.374.0:4037 2014-08-12 15:47:44.777786 2014-08-13 17:38:03.021729 7fe9bae887a0 20 read_log 194215'16212 (194215'16211) delete accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.2174595.0:154069 0.000000 2014-08-13 17:38:03.021740 7fe9bae887a0 20 read_log 194215'16213 (192834'15488) clean d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by osd.374.0:4039 2014-08-12 15:47:44.786412 2014-08-13 17:38:03.021752 7fe9bae887a0 20 read_log 194215'16214 (194215'16213) delete d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.2174595.0:154071 0.000000 2014-08-13 17:38:03.021763 7fe9bae887a0 20 read_log 194215'16215 (192834'14946) clean dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by osd.374.0:4041 2014-08-12 15:47:44.794135 2014-08-13 17:38:03.021775 7fe9bae887a0 20 read_log 194215'16216 (194215'16215) delete dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.2174595.0:154073 0.000000 2014-08-13 17:38:03.021785 7fe9bae887a0 20 read_log 194215'16217 (192888'15743) clean 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by osd.374.0:4043 2014-08-12 15:47:44.801826 2014-08-13 17:38:03.021797 7fe9bae887a0 20 read_log 194215'16218 (194215'16217) delete 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.2174595.0:154075 0.000000 2014-08-13 17:38:03.021808 7fe9bae887a0 20 read_log 194215'16219 (192888'15750) clean 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by osd.374.0:4045 2014-08-12 15:47:44.809112 2014-08-13 17:38:03.021820 7fe9bae887a0 20 read_log 194215'16220 (194215'16219) delete 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.2174595.0:154077 0.000000 2014-08-13 17:38:03.021869 7fe9bae887a0 20 read_log 194215'16221 (193305'16047) clean 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by osd.374.0:4047 2014-08-12 15:47:44.817026 2014-08-13 17:38:03.021893 7fe9bae887a0 20 read_log 194215'16222 (194215'16221) delete 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.2174595.0:154079 0.000000 2014-08-13 17:38:03.021905 7fe9bae887a0 20 read_log 194215'16223 (192834'15071) clean 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by osd.374.0:4049 2014-08-12 15:47:44.823982 2014-08-13 17:38:03.021917 7fe9bae887a0 20 read_log 194215'16224 (194215'16223) delete 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.2174595.0:154081 0.000000 2014-08-13 17:38:03.021927 7fe9bae887a0 20 read_log 194215'16225 (192834'15554) clean 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by osd.374.0:4051 2014-08-12 15:47:44.832340 2014-08-13 17:38:03.021939 7fe9bae887a0 20 read_log 194215'16226 (194215'16225) delete 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.2174595.0:154083 0.000000 2014-08-13 17:38:03.021950 7fe9bae887a0 20 read_log 194215'16227 (192888'15747) clean 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by osd.374.0:4053 2014-08-12 15:47:44.839497 2014-08-13 17:38:03.021962 7fe9bae887a0 20 read_log 194215'16228 (194215'16227) delete 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.2174595.0:154085 0.000000 2014-08-13 17:38:03.021973 7fe9bae887a0 20 read_log 194215'16229 (194209'16059) delete 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.2174595.0:154087 0.000000 2014-08-13 17:38:03.021984 7fe9bae887a0 20 read_log 194215'16230 (194209'16061) delete 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.2174595.0:154089 0.000000 2014-08-13 17:38:03.021994 7fe9bae887a0 20 read_log 194215'16231 (194209'16062) delete 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.2174595.0:154091 0.000000 2014-08-13 17:38:03.022005 7fe9bae887a0 20 read_log 194215'16232 (192888'15922) clean 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by osd.374.0:4058 2014-08-12 15:47:44.861845 2014-08-13 17:38:03.022017 7fe9bae887a0 20 read_log 194215'16233 (194215'16232) delete 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.2174595.0:154093 0.000000 2014-08-13 17:38:03.022027 7fe9bae887a0 20 read_log 194215'16234 (193305'16046) clean bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by osd.374.0:4060 2014-08-12 15:47:44.869274 2014-08-13 17:38:03.022039 7fe9bae887a0 20 read_log 194215'16235 (194215'16234) delete bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.2174595.0:154095 0.000000 2014-08-13 17:38:03.022050 7fe9bae887a0 20 read_log 194215'16236 (192888'15778) clean 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by osd.374.0:4062 2014-08-12 15:47:44.876060 2014-08-13 17:38:03.022062 7fe9bae887a0 20 read_log 194215'16237 (194215'16236) delete 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.2174595.0:154097 0.000000 2014-08-13 17:38:03.022072 7fe9bae887a0 20 read_log 194215'16238 (192888'15920) clean 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by osd.374.0:4064 2014-08-12 15:47:44.883846 2014-08-13 17:38:03.022084 7fe9bae887a0 20 read_log 194215'16239 (194215'16238) delete 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.2174595.0:154099 0.000000 2014-08-13 17:38:03.022095 7fe9bae887a0 20 read_log 194215'16240 (192888'15915) clean 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by osd.374.0:4066 2014-08-12 15:47:44.892082 2014-08-13 17:38:03.022107 7fe9bae887a0 20 read_log 194215'16241 (194215'16240) delete 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.2174595.0:154101 0.000000 2014-08-13 17:38:03.022125 7fe9bae887a0 20 read_log 194215'16242 (193305'16032) clean 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by osd.374.0:4068 2014-08-12 15:47:44.900084 2014-08-13 17:38:03.022137 7fe9bae887a0 20 read_log 194215'16243 (194215'16242) delete 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.2174595.0:154103 0.000000 2014-08-13 17:38:03.022148 7fe9bae887a0 20 read_log 194215'16244 (192888'15787) clean 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by osd.374.0:4070 2014-08-12 15:47:44.907409 2014-08-13 17:38:03.022166 7fe9bae887a0 20 read_log 194215'16245 (194215'16244) delete 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.2174595.0:154105 0.000000 2014-08-13 17:38:03.022178 7fe9bae887a0 20 read_log 194215'16246 (192888'15737) clean 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by osd.374.0:4072 2014-08-12 15:47:44.915670 2014-08-13 17:38:03.022190 7fe9bae887a0 20 read_log 194215'16247 (194215'16246) delete 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.2174595.0:154107 0.000000 2014-08-13 17:38:03.022201 7fe9bae887a0 20 read_log 194215'16248 (192888'15771) clean b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by osd.374.0:4074 2014-08-12 15:47:44.923582 2014-08-13 17:38:03.022213 7fe9bae887a0 20 read_log 194215'16249 (194215'16248) delete b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.2174595.0:154109 0.000000 2014-08-13 17:38:03.022224 7fe9bae887a0 20 read_log 194215'16250 (192834'15447) clean bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by osd.374.0:4076 2014-08-12 15:47:44.931369 2014-08-13 17:38:03.022236 7fe9bae887a0 20 read_log 194215'16251 (194215'16250) delete bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.2174595.0:154111 0.000000 2014-08-13 17:38:03.022247 7fe9bae887a0 20 read_log 194215'16252 (193305'16034) clean aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by osd.374.0:4078 2014-08-12 15:47:44.938726 2014-08-13 17:38:03.022259 7fe9bae887a0 20 read_log 194215'16253 (194215'16252) delete aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.2174595.0:154113 0.000000 2014-08-13 17:38:03.022270 7fe9bae887a0 20 read_log 194215'16254 (192888'15765) clean ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by osd.374.0:4080 2014-08-12 15:47:44.946521 2014-08-13 17:38:03.022282 7fe9bae887a0 20 read_log 194215'16255 (194215'16254) delete ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.2174595.0:154115 0.000000 2014-08-13 17:38:03.022292 7fe9bae887a0 20 read_log 194215'16256 (193305'16041) clean f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by osd.374.0:4082 2014-08-12 15:47:44.954000 2014-08-13 17:38:03.022304 7fe9bae887a0 20 read_log 194215'16257 (194215'16256) delete f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.2174595.0:154117 0.000000 2014-08-13 17:38:03.022315 7fe9bae887a0 20 read_log 194215'16258 (192834'14968) clean b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by osd.374.0:4084 2014-08-12 15:47:44.961953 2014-08-13 17:38:03.022327 7fe9bae887a0 20 read_log 194215'16259 (194215'16258) delete b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.2174595.0:154119 0.000000 2014-08-13 17:38:03.022338 7fe9bae887a0 20 read_log 194215'16260 (192888'15759) clean 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by osd.374.0:4086 2014-08-12 15:47:44.969198 2014-08-13 17:38:03.022362 7fe9bae887a0 20 read_log 194215'16261 (194215'16260) delete 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.2174595.0:154121 0.000000 2014-08-13 17:38:03.022400 7fe9bae887a0 20 read_log 194215'16262 (192834'14902) clean 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by osd.374.0:4088 2014-08-12 15:47:44.977289 2014-08-13 17:38:03.022414 7fe9bae887a0 20 read_log 194215'16263 (194215'16262) delete 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.2174595.0:154123 0.000000 2014-08-13 17:38:03.022425 7fe9bae887a0 20 read_log 194215'16264 (192888'15919) clean ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by osd.374.0:4090 2014-08-12 15:47:44.984600 2014-08-13 17:38:03.022439 7fe9bae887a0 20 read_log 194215'16265 (194215'16264) delete ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.2174595.0:154125 0.000000 2014-08-13 17:38:03.022450 7fe9bae887a0 20 read_log 194215'16266 (193305'16028) clean e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by osd.374.0:4092 2014-08-12 15:47:44.991663 2014-08-13 17:38:03.022462 7fe9bae887a0 20 read_log 194215'16267 (194215'16266) delete e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.2174595.0:154127 0.000000 2014-08-13 17:38:03.022473 7fe9bae887a0 20 read_log 194215'16268 (193305'16031) clean ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by osd.374.0:4094 2014-08-12 15:47:45.000050 2014-08-13 17:38:03.022485 7fe9bae887a0 20 read_log 194215'16269 (194215'16268) delete ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.2174595.0:154129 0.000000 2014-08-13 17:38:03.022496 7fe9bae887a0 20 read_log 194215'16270 (192888'15768) clean 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by osd.374.0:4096 2014-08-12 15:47:45.007162 2014-08-13 17:38:03.022508 7fe9bae887a0 20 read_log 194215'16271 (194215'16270) delete 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.2174595.0:154131 0.000000 2014-08-13 17:38:03.022518 7fe9bae887a0 20 read_log 194215'16272 (193305'16039) clean c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by osd.374.0:4098 2014-08-12 15:47:45.014396 2014-08-13 17:38:03.022530 7fe9bae887a0 20 read_log 194215'16273 (194215'16272) delete c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.2174595.0:154133 0.000000 2014-08-13 17:38:03.022541 7fe9bae887a0 20 read_log 194215'16274 (192834'15490) clean 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by osd.374.0:4100 2014-08-12 15:47:45.021998 2014-08-13 17:38:03.022553 7fe9bae887a0 20 read_log 194215'16275 (194215'16274) delete 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.2174595.0:154135 0.000000 2014-08-13 17:38:03.022563 7fe9bae887a0 20 read_log 194215'16276 (192888'15751) clean 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by osd.374.0:4102 2014-08-12 15:47:45.030493 2014-08-13 17:38:03.022576 7fe9bae887a0 20 read_log 194215'16277 (194215'16276) delete 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.2174595.0:154137 0.000000 2014-08-13 17:38:03.022586 7fe9bae887a0 20 read_log 194215'16278 (192834'15501) clean a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by osd.374.0:4104 2014-08-12 15:47:45.038941 2014-08-13 17:38:03.022598 7fe9bae887a0 20 read_log 194215'16279 (194215'16278) delete a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.2174595.0:154139 0.000000 2014-08-13 17:38:03.022609 7fe9bae887a0 20 read_log 194215'16280 (192888'15956) clean 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by osd.374.0:4106 2014-08-12 15:47:45.046303 2014-08-13 17:38:03.022621 7fe9bae887a0 20 read_log 194215'16281 (194215'16280) delete 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.2174595.0:154141 0.000000 2014-08-13 17:38:03.022641 7fe9bae887a0 20 read_log 194215'16282 (193305'16042) clean 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by osd.374.0:4108 2014-08-12 15:47:45.054280 2014-08-13 17:38:03.022654 7fe9bae887a0 20 read_log 194215'16283 (194215'16282) delete 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.2174595.0:154143 0.000000 2014-08-13 17:38:03.022665 7fe9bae887a0 20 read_log 194215'16284 (193305'16027) clean bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by osd.374.0:4110 2014-08-12 15:47:45.061839 2014-08-13 17:38:03.022677 7fe9bae887a0 20 read_log 194215'16285 (194215'16284) delete bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.2174595.0:154145 0.000000 2014-08-13 17:38:03.022688 7fe9bae887a0 20 read_log 194215'16286 (192888'15725) clean 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by osd.374.0:4112 2014-08-12 15:47:45.068938 2014-08-13 17:38:03.022700 7fe9bae887a0 20 read_log 194215'16287 (194215'16286) delete 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.2174595.0:154147 0.000000 2014-08-13 17:38:03.022711 7fe9bae887a0 20 read_log 194215'16288 (192888'15921) clean 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by osd.374.0:4114 2014-08-12 15:47:45.077721 2014-08-13 17:38:03.022723 7fe9bae887a0 20 read_log 194215'16289 (194215'16288) delete 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.2174595.0:154149 0.000000 2014-08-13 17:38:03.022734 7fe9bae887a0 20 read_log 194215'16290 (192888'15939) clean 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by osd.374.0:4116 2014-08-12 15:47:45.085365 2014-08-13 17:38:03.022746 7fe9bae887a0 20 read_log 194215'16291 (194215'16290) delete 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.2174595.0:154151 0.000000 2014-08-13 17:38:03.022756 7fe9bae887a0 20 read_log 194215'16292 (192834'15541) clean a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by osd.374.0:4118 2014-08-12 15:47:45.092808 2014-08-13 17:38:03.022769 7fe9bae887a0 20 read_log 194215'16293 (194215'16292) delete a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.2174595.0:154153 0.000000 2014-08-13 17:38:03.022780 7fe9bae887a0 20 read_log 194215'16294 (192834'14895) clean 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by osd.374.0:4120 2014-08-12 15:47:45.106304 2014-08-13 17:38:03.022792 7fe9bae887a0 20 read_log 194215'16295 (194215'16294) delete 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.2174595.0:154155 0.000000 2014-08-13 17:38:03.022802 7fe9bae887a0 20 read_log 194215'16296 (192834'15085) clean 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by osd.374.0:4122 2014-08-12 15:47:45.114558 2014-08-13 17:38:03.022815 7fe9bae887a0 20 read_log 194215'16297 (194215'16296) delete 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.2174595.0:154157 0.000000 2014-08-13 17:38:03.022825 7fe9bae887a0 20 read_log 194215'16298 (193305'16051) clean ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by osd.374.0:4124 2014-08-12 15:47:45.122223 2014-08-13 17:38:03.022837 7fe9bae887a0 20 read_log 194215'16299 (194215'16298) delete ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.2174595.0:154159 0.000000 2014-08-13 17:38:03.022848 7fe9bae887a0 20 read_log 194215'16300 (193305'16024) clean 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by osd.374.0:4126 2014-08-12 15:47:45.129485 2014-08-13 17:38:03.022860 7fe9bae887a0 20 read_log 194215'16301 (194215'16300) delete 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.2174595.0:154161 0.000000 2014-08-13 17:38:03.022877 7fe9bae887a0 20 read_log 194215'16302 (192834'15033) clean 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by osd.374.0:4128 2014-08-12 15:47:45.137364 2014-08-13 17:38:03.022890 7fe9bae887a0 20 read_log 194215'16303 (194215'16302) delete 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.2174595.0:154163 0.000000 2014-08-13 17:38:03.022901 7fe9bae887a0 20 read_log 194215'16304 (193305'16037) clean 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by osd.374.0:4130 2014-08-12 15:47:45.144840 2014-08-13 17:38:03.022914 7fe9bae887a0 20 read_log 194215'16305 (194215'16304) delete 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.2174595.0:154165 0.000000 2014-08-13 17:38:03.022925 7fe9bae887a0 20 read_log 194215'16306 (192834'15556) clean 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by osd.374.0:4132 2014-08-12 15:47:45.153037 2014-08-13 17:38:03.022937 7fe9bae887a0 20 read_log 194215'16307 (194215'16306) delete 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.2174595.0:154167 0.000000 2014-08-13 17:38:03.022948 7fe9bae887a0 20 read_log 194215'16308 (192888'15732) clean 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by osd.374.0:4134 2014-08-12 15:47:45.161904 2014-08-13 17:38:03.022960 7fe9bae887a0 20 read_log 194215'16309 (194215'16308) delete 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.2174595.0:154169 0.000000 2014-08-13 17:38:03.022970 7fe9bae887a0 20 read_log 194215'16310 (192888'15761) clean b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by osd.374.0:4136 2014-08-12 15:47:45.170476 2014-08-13 17:38:03.022983 7fe9bae887a0 20 read_log 194215'16311 (194215'16310) delete b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.2174595.0:154171 0.000000 2014-08-13 17:38:03.022994 7fe9bae887a0 20 read_log 194215'16312 (192888'15910) clean a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by osd.374.0:4138 2014-08-12 15:47:45.178545 2014-08-13 17:38:03.023006 7fe9bae887a0 20 read_log 194215'16313 (194215'16312) delete a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.2174595.0:154173 0.000000 2014-08-13 17:38:03.023033 7fe9bae887a0 20 read_log 194215'16314 (192834'15573) clean 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by osd.374.0:4140 2014-08-12 15:47:45.185904 2014-08-13 17:38:03.023046 7fe9bae887a0 20 read_log 194215'16315 (194215'16314) delete 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.2174595.0:154175 0.000000 2014-08-13 17:38:03.023057 7fe9bae887a0 20 read_log 194215'16316 (192888'15928) clean f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by osd.374.0:4142 2014-08-12 15:47:45.194138 2014-08-13 17:38:03.023069 7fe9bae887a0 20 read_log 194215'16317 (194215'16316) delete f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.2174595.0:154177 0.000000 2014-08-13 17:38:03.023080 7fe9bae887a0 20 read_log 194215'16318 (192834'14898) clean fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by osd.374.0:4144 2014-08-12 15:47:45.201567 2014-08-13 17:38:03.023092 7fe9bae887a0 20 read_log 194215'16319 (194215'16318) delete fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.2174595.0:154179 0.000000 2014-08-13 17:38:03.023102 7fe9bae887a0 20 read_log 194215'16320 (192834'14999) clean 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by osd.374.0:4146 2014-08-12 15:47:45.209378 2014-08-13 17:38:03.023114 7fe9bae887a0 20 read_log 194215'16321 (194215'16320) delete 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.2174595.0:154181 0.000000 2014-08-13 17:38:03.023132 7fe9bae887a0 20 read_log 194215'16322 (192834'15055) clean 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by osd.374.0:4148 2014-08-12 15:47:45.216765 2014-08-13 17:38:03.023145 7fe9bae887a0 20 read_log 194215'16323 (194215'16322) delete 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.2174595.0:154183 0.000000 2014-08-13 17:38:03.023161 7fe9bae887a0 20 read_log 194215'16324 (192888'15912) clean ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by osd.374.0:4150 2014-08-12 15:47:45.225554 2014-08-13 17:38:03.023174 7fe9bae887a0 20 read_log 194215'16325 (194215'16324) delete ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.2174595.0:154185 0.000000 2014-08-13 17:38:03.023185 7fe9bae887a0 20 read_log 194215'16326 (192888'15758) clean 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by osd.374.0:4152 2014-08-12 15:47:45.234244 2014-08-13 17:38:03.023197 7fe9bae887a0 20 read_log 194215'16327 (194215'16326) delete 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.2174595.0:154187 0.000000 2014-08-13 17:38:03.023208 7fe9bae887a0 20 read_log 194215'16328 (192834'15454) clean c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by osd.374.0:4154 2014-08-12 15:47:45.242362 2014-08-13 17:38:03.023220 7fe9bae887a0 20 read_log 194215'16329 (194215'16328) delete c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.2174595.0:154189 0.000000 2014-08-13 17:38:03.023231 7fe9bae887a0 20 read_log 194215'16330 (192834'14913) clean 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by osd.374.0:4156 2014-08-12 15:47:45.250111 2014-08-13 17:38:03.023244 7fe9bae887a0 20 read_log 194215'16331 (194215'16330) delete 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.2174595.0:154191 0.000000 2014-08-13 17:38:03.023254 7fe9bae887a0 20 read_log 194215'16332 (192888'15947) clean 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by osd.374.0:4158 2014-08-12 15:47:45.257553 2014-08-13 17:38:03.023266 7fe9bae887a0 20 read_log 194215'16333 (194215'16332) delete 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.2174595.0:154193 0.000000 2014-08-13 17:38:03.023277 7fe9bae887a0 20 read_log 194215'16334 (192834'14939) clean ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by osd.374.0:4160 2014-08-12 15:47:45.265618 2014-08-13 17:38:03.023291 7fe9bae887a0 20 read_log 194215'16335 (194215'16334) delete ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.2174595.0:154195 0.000000 2014-08-13 17:38:03.023301 7fe9bae887a0 20 read_log 194215'16336 (192888'15784) clean d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by osd.374.0:4162 2014-08-12 15:47:45.273186 2014-08-13 17:38:03.023313 7fe9bae887a0 20 read_log 194215'16337 (194215'16336) delete d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.2174595.0:154197 0.000000 2014-08-13 17:38:03.023324 7fe9bae887a0 20 read_log 194215'16338 (192888'15914) clean 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by osd.374.0:4164 2014-08-12 15:47:45.281601 2014-08-13 17:38:03.023336 7fe9bae887a0 20 read_log 194215'16339 (194215'16338) delete 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.2174595.0:154199 0.000000 2014-08-13 17:38:03.023347 7fe9bae887a0 20 read_log 194215'16340 (192834'15505) clean a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by osd.374.0:4166 2014-08-12 15:47:45.289962 2014-08-13 17:38:03.023358 7fe9bae887a0 20 read_log 194215'16341 (194215'16340) delete a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.2174595.0:154201 0.000000 2014-08-13 17:38:03.023380 7fe9bae887a0 20 read_log 194370'16342 (0'0) promote 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by osd.374.0:4900 2014-08-12 20:30:24.527969 2014-08-13 17:38:03.023393 7fe9bae887a0 20 read_log 194370'16343 (194370'16342) modify 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by client.1508531.0:2720 2014-08-12 20:30:24.520471 2014-08-13 17:38:03.023405 7fe9bae887a0 20 read_log 194370'16344 (0'0) promote 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by osd.374.0:4916 2014-08-12 20:30:26.904986 2014-08-13 17:38:03.023417 7fe9bae887a0 20 read_log 194370'16345 (194370'16344) modify 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by client.1508531.0:3371 2014-08-12 20:30:26.898087 2014-08-13 17:38:03.023429 7fe9bae887a0 20 read_log 194370'16346 (0'0) promote 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by osd.374.0:4940 2014-08-12 20:30:30.854778 2014-08-13 17:38:03.023441 7fe9bae887a0 20 read_log 194370'16347 (194370'16346) modify 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by client.1508531.0:4410 2014-08-12 20:30:30.848680 2014-08-13 17:38:03.023454 7fe9bae887a0 20 read_log 194370'16348 (0'0) promote 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by osd.374.0:4982 2014-08-12 20:30:38.552347 2014-08-13 17:38:03.023466 7fe9bae887a0 20 read_log 194370'16349 (194370'16348) modify 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by client.1508531.0:6178 2014-08-12 20:30:38.545133 2014-08-13 17:38:03.023478 7fe9bae887a0 20 read_log 194370'16350 (0'0) promote 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by osd.374.0:5010 2014-08-12 20:30:46.624293 2014-08-13 17:38:03.023490 7fe9bae887a0 20 read_log 194370'16351 (194370'16350) modify 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by client.1508531.0:7818 2014-08-12 20:30:46.615726 2014-08-13 17:38:03.023502 7fe9bae887a0 20 read_log 194370'16352 (0'0) promote 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by osd.374.0:5078 2014-08-12 20:31:07.839294 2014-08-13 17:38:03.023515 7fe9bae887a0 20 read_log 194370'16353 (194370'16352) modify 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by client.1508531.0:11419 2014-08-12 20:31:07.830173 2014-08-13 17:38:03.023527 7fe9bae887a0 20 read_log 194370'16354 (0'0) promote 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by osd.374.0:5106 2014-08-12 20:31:14.279038 2014-08-13 17:38:03.023541 7fe9bae887a0 20 read_log 194370'16355 (194370'16354) modify 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by client.1508531.0:12400 2014-08-12 20:31:14.270603 2014-08-13 17:38:03.023554 7fe9bae887a0 20 read_log 194370'16356 (0'0) promote 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by osd.374.0:5136 2014-08-12 20:31:25.690707 2014-08-13 17:38:03.023566 7fe9bae887a0 20 read_log 194370'16357 (194370'16356) modify 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by client.1508531.0:13662 2014-08-12 20:31:25.680926 2014-08-13 17:38:03.023578 7fe9bae887a0 20 read_log 194377'16358 (0'0) promote 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by osd.374.0:5172 2014-08-12 20:33:02.110149 2014-08-13 17:38:03.023591 7fe9bae887a0 20 read_log 194377'16359 (194377'16358) modify 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by client.2231190.0:1211 2014-08-12 20:33:02.102879 2014-08-13 17:38:03.023603 7fe9bae887a0 20 read_log 194377'16360 (0'0) promote 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by osd.374.0:5218 2014-08-12 20:33:07.613644 2014-08-13 17:38:03.023615 7fe9bae887a0 20 read_log 194377'16361 (194377'16360) modify 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by client.2231190.0:2577 2014-08-12 20:33:07.605310 2014-08-13 17:38:03.023635 7fe9bae887a0 20 read_log 194377'16362 (0'0) promote 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by osd.374.0:5236 2014-08-12 20:33:09.593729 2014-08-13 17:38:03.023648 7fe9bae887a0 20 read_log 194377'16363 (194377'16362) modify 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by client.2231190.0:3068 2014-08-12 20:33:09.586317 2014-08-13 17:38:03.023660 7fe9bae887a0 20 read_log 194377'16364 (0'0) promote 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by osd.374.0:5244 2014-08-12 20:33:11.004822 2014-08-13 17:38:03.023672 7fe9bae887a0 20 read_log 194377'16365 (194377'16364) modify 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by client.2231190.0:3430 2014-08-12 20:33:10.995220 2014-08-13 17:38:03.023685 7fe9bae887a0 20 read_log 194380'16366 (0'0) promote c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by osd.374.0:5284 2014-08-12 20:33:26.188953 2014-08-13 17:38:03.023697 7fe9bae887a0 20 read_log 194380'16367 (194380'16366) modify c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by client.2231190.0:5229 2014-08-12 20:33:26.179871 2014-08-13 17:38:03.023709 7fe9bae887a0 20 read_log 194383'16368 (0'0) promote 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by osd.374.0:5298 2014-08-12 20:34:00.306372 2014-08-13 17:38:03.023721 7fe9bae887a0 20 read_log 194383'16369 (194383'16368) modify 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by client.2231190.0:5795 2014-08-12 20:34:00.296784 2014-08-13 17:38:03.023733 7fe9bae887a0 20 read_log 194383'16370 (0'0) promote 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by osd.374.0:5336 2014-08-12 20:34:26.543149 2014-08-13 17:38:03.023745 7fe9bae887a0 20 read_log 194383'16371 (194383'16370) modify 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by client.2231190.0:7419 2014-08-12 20:34:26.535065 2014-08-13 17:38:03.023758 7fe9bae887a0 20 read_log 194393'16372 (194393'16372) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:37:09.049317_2014-08-12 20:47:26.077872/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.078448 2014-08-13 17:38:03.023770 7fe9bae887a0 20 read_log 194393'16373 (194215'16153) delete 2ec/hit_set_24.2ec_archive_2014-08-12 15:32:26.012127_2014-08-12 15:47:44.557466/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.078448 2014-08-13 17:38:03.023783 7fe9bae887a0 20 read_log 194393'16374 (0'0) promote fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by osd.374.0:5343 2014-08-12 20:47:26.084436 2014-08-13 17:38:03.023795 7fe9bae887a0 20 read_log 194393'16375 (194393'16374) modify fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by client.1064095.0:38 2014-08-12 20:47:26.071882 2014-08-13 17:38:03.023807 7fe9bae887a0 20 read_log 194393'16376 (0'0) promote a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by osd.374.0:5372 2014-08-12 20:49:26.301315 2014-08-13 17:38:03.023822 7fe9bae887a0 20 read_log 194393'16377 (194393'16376) modify a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by client.1064182.0:174 2014-08-12 20:49:26.293716 2014-08-13 17:38:03.023834 7fe9bae887a0 20 read_log 194393'16378 (0'0) promote 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by osd.374.0:5378 2014-08-12 20:49:27.260261 2014-08-13 17:38:03.023847 7fe9bae887a0 20 read_log 194393'16379 (194393'16378) modify 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by client.1064182.0:381 2014-08-12 20:49:27.253733 2014-08-13 17:38:03.023859 7fe9bae887a0 20 read_log 194396'16380 (194396'16380) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:47:26.078497_2014-08-12 20:53:00.018899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.019463 2014-08-13 17:38:03.023879 7fe9bae887a0 20 read_log 194396'16381 (194393'16372) delete 2ec/hit_set_24.2ec_archive_2014-08-12 20:37:09.049317_2014-08-12 20:47:26.077872/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.019463 2014-08-13 17:38:03.023892 7fe9bae887a0 20 read_log 194396'16382 (0'0) promote dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by osd.374.0:5391 2014-08-12 20:53:00.025127 2014-08-13 17:38:03.023904 7fe9bae887a0 20 read_log 194396'16383 (194396'16382) modify dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by client.2235069.0:279 2014-08-12 20:53:00.011801 2014-08-13 17:38:03.023917 7fe9bae887a0 20 read_log 194416'16384 (194416'16384) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:53:00.019540_2014-08-13 15:28:37.194823/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 17:38:03.023930 7fe9bae887a0 20 read_log 194416'16385 (194396'16380) delete 2ec/hit_set_24.2ec_archive_2014-08-12 20:47:26.078497_2014-08-12 20:53:00.018899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 17:38:03.023942 7fe9bae887a0 20 read_log 194416'16386 (0'0) promote efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by osd.374.0:5597 2014-08-13 15:28:37.198503 2014-08-13 17:38:03.023954 7fe9bae887a0 20 read_log 194416'16387 (194416'16386) modify efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by client.2441958.0:1710 2014-08-13 15:28:37.189859 2014-08-13 17:38:03.023967 7fe9bae887a0 20 read_log 194416'16388 (0'0) promote 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by osd.374.0:5623 2014-08-13 15:28:41.156396 2014-08-13 17:38:03.023979 7fe9bae887a0 20 read_log 194416'16389 (194416'16388) modify 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by client.2441958.0:2754 2014-08-13 15:28:41.147516 2014-08-13 17:38:03.023991 7fe9bae887a0 20 read_log 194416'16390 (0'0) promote c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by osd.374.0:5637 2014-08-13 15:28:43.094680 2014-08-13 17:38:03.024003 7fe9bae887a0 20 read_log 194416'16391 (194416'16390) modify c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by client.2441958.0:3268 2014-08-13 15:28:43.084947 2014-08-13 17:38:03.024015 7fe9bae887a0 20 read_log 194416'16392 (0'0) promote 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by osd.374.0:5641 2014-08-13 15:28:43.410479 2014-08-13 17:38:03.024027 7fe9bae887a0 20 read_log 194416'16393 (194416'16392) modify 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by client.2441958.0:3353 2014-08-13 15:28:43.399752 2014-08-13 17:38:03.024039 7fe9bae887a0 20 read_log 194416'16394 (0'0) promote 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by osd.374.0:5683 2014-08-13 15:28:50.940871 2014-08-13 17:38:03.024051 7fe9bae887a0 20 read_log 194416'16395 (194416'16394) modify 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by client.2441958.0:5273 2014-08-13 15:28:50.932603 2014-08-13 17:38:03.024063 7fe9bae887a0 20 read_log 194416'16396 (0'0) promote 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by osd.374.0:5709 2014-08-13 15:28:55.983030 2014-08-13 17:38:03.024075 7fe9bae887a0 20 read_log 194416'16397 (194416'16396) modify 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by client.2441958.0:6531 2014-08-13 15:28:55.974403 2014-08-13 17:38:03.024087 7fe9bae887a0 20 read_log 194416'16398 (0'0) promote 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by osd.374.0:5717 2014-08-13 15:28:56.921705 2014-08-13 17:38:03.024098 7fe9bae887a0 20 read_log 194416'16399 (194416'16398) modify 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by client.2441958.0:6770 2014-08-13 15:28:56.914226 2014-08-13 17:38:03.024135 7fe9bae887a0 20 read_log 194416'16400 (0'0) promote d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by osd.374.0:5739 2014-08-13 15:28:59.057271 2014-08-13 17:38:03.024150 7fe9bae887a0 20 read_log 194416'16401 (194416'16400) modify d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by client.2441958.0:7342 2014-08-13 15:28:59.046808 2014-08-13 17:38:03.024169 7fe9bae887a0 20 read_log 194416'16402 (0'0) promote 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by osd.374.0:5787 2014-08-13 15:29:06.946845 2014-08-13 17:38:03.024182 7fe9bae887a0 20 read_log 194416'16403 (194416'16402) modify 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by client.2441958.0:9387 2014-08-13 15:29:06.939232 2014-08-13 17:38:03.024194 7fe9bae887a0 20 read_log 194416'16404 (0'0) promote 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by osd.374.0:5803 2014-08-13 15:29:10.697215 2014-08-13 17:38:03.024206 7fe9bae887a0 20 read_log 194416'16405 (194416'16404) modify 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by client.2441958.0:10303 2014-08-13 15:29:10.688257 2014-08-13 17:38:03.024218 7fe9bae887a0 20 read_log 194416'16406 (0'0) promote b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by osd.374.0:5825 2014-08-13 15:29:15.514651 2014-08-13 17:38:03.024248 7fe9bae887a0 20 read_log 194416'16407 (194416'16406) modify b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by client.2441958.0:11505 2014-08-13 15:29:15.506072 2014-08-13 17:38:03.024261 7fe9bae887a0 20 read_log 194416'16408 (0'0) promote d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by osd.374.0:5829 2014-08-13 15:29:16.569822 2014-08-13 17:38:03.024274 7fe9bae887a0 20 read_log 194416'16409 (194416'16408) modify d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by client.2441958.0:11763 2014-08-13 15:29:16.560433 2014-08-13 17:38:03.024286 7fe9bae887a0 20 read_log 194416'16410 (0'0) promote e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by osd.374.0:5879 2014-08-13 15:29:22.903298 2014-08-13 17:38:03.024298 7fe9bae887a0 20 read_log 194416'16411 (194416'16410) modify e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by client.2441958.0:13223 2014-08-13 15:29:22.895758 2014-08-13 17:38:03.024310 7fe9bae887a0 20 read_log 194416'16412 (0'0) promote 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by osd.374.0:5897 2014-08-13 15:29:26.290554 2014-08-13 17:38:03.024322 7fe9bae887a0 20 read_log 194416'16413 (194416'16412) modify 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by client.2441958.0:14020 2014-08-13 15:29:26.280813 2014-08-13 17:38:03.024334 7fe9bae887a0 20 read_log 194416'16414 (0'0) promote f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by osd.374.0:5899 2014-08-13 15:29:26.335225 2014-08-13 17:38:03.024346 7fe9bae887a0 20 read_log 194416'16415 (194416'16414) modify f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by client.2441958.0:14032 2014-08-13 15:29:26.325083 2014-08-13 17:38:03.024358 7fe9bae887a0 20 read_log 194416'16416 (0'0) promote 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by osd.374.0:5943 2014-08-13 15:29:41.821557 2014-08-13 17:38:03.024370 7fe9bae887a0 20 read_log 194416'16417 (194416'16416) modify 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by client.2441958.0:16612 2014-08-13 15:29:41.812045 2014-08-13 17:38:03.024382 7fe9bae887a0 20 read_log 194416'16418 (0'0) promote db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by osd.374.0:5947 2014-08-13 15:29:42.875150 2014-08-13 17:38:03.024394 7fe9bae887a0 20 read_log 194416'16419 (194416'16418) modify db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by client.2441958.0:16688 2014-08-13 15:29:42.865793 2014-08-13 17:38:03.024413 7fe9bae887a0 20 read_log 194416'16420 (0'0) promote 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by osd.374.0:5969 2014-08-13 15:29:59.184442 2014-08-13 17:38:03.024426 7fe9bae887a0 20 read_log 194416'16421 (194416'16420) modify 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by client.2441958.0:17840 2014-08-13 15:29:59.173753 2014-08-13 17:38:03.024438 7fe9bae887a0 20 read_log 194416'16422 (0'0) promote bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by osd.374.0:5973 2014-08-13 15:30:03.607587 2014-08-13 17:38:03.024450 7fe9bae887a0 20 read_log 194416'16423 (194416'16422) modify bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by client.2441958.0:18154 2014-08-13 15:30:03.598192 2014-08-13 17:38:03.024462 7fe9bae887a0 20 read_log 194422'16424 (0'0) promote 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by osd.374.0:5995 2014-08-13 15:32:17.391424 2014-08-13 17:38:03.024474 7fe9bae887a0 20 read_log 194422'16425 (194422'16424) modify 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by client.1148203.0:516 2014-08-13 15:32:17.381974 2014-08-13 17:38:03.024486 7fe9bae887a0 20 read_log 194422'16426 (0'0) promote f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by osd.374.0:6031 2014-08-13 15:32:21.457303 2014-08-13 17:38:03.024498 7fe9bae887a0 20 read_log 194422'16427 (194422'16426) modify f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by client.1148203.0:1558 2014-08-13 15:32:21.449286 2014-08-13 17:38:03.024510 7fe9bae887a0 20 read_log 194422'16428 (0'0) promote f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by osd.374.0:6055 2014-08-13 15:32:26.793871 2014-08-13 17:38:03.024522 7fe9bae887a0 20 read_log 194422'16429 (194422'16428) modify f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by client.1148203.0:2918 2014-08-13 15:32:26.785090 2014-08-13 17:38:03.024535 7fe9bae887a0 20 read_log 194422'16430 (0'0) promote ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by osd.374.0:6067 2014-08-13 15:32:28.357437 2014-08-13 17:38:03.024547 7fe9bae887a0 20 read_log 194422'16431 (194422'16430) modify ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by client.1148203.0:3316 2014-08-13 15:32:28.348962 2014-08-13 17:38:03.024559 7fe9bae887a0 20 read_log 194422'16432 (0'0) promote 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by osd.374.0:6079 2014-08-13 15:32:31.313768 2014-08-13 17:38:03.024571 7fe9bae887a0 20 read_log 194422'16433 (194422'16432) modify 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by client.1148203.0:4081 2014-08-13 15:32:31.305194 2014-08-13 17:38:03.024583 7fe9bae887a0 20 read_log 194422'16434 (0'0) promote 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by osd.374.0:6089 2014-08-13 15:32:33.213894 2014-08-13 17:38:03.024595 7fe9bae887a0 20 read_log 194422'16435 (194422'16434) modify 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by client.1148203.0:4579 2014-08-13 15:32:33.206430 2014-08-13 17:38:03.024607 7fe9bae887a0 20 read_log 194422'16436 (0'0) promote e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by osd.374.0:6099 2014-08-13 15:32:34.659633 2014-08-13 17:38:03.024618 7fe9bae887a0 20 read_log 194422'16437 (194422'16436) modify e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by client.1148203.0:4953 2014-08-13 15:32:34.652060 2014-08-13 17:38:03.024633 7fe9bae887a0 20 read_log 194422'16438 (0'0) promote d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by osd.374.0:6101 2014-08-13 15:32:34.729571 2014-08-13 17:38:03.024645 7fe9bae887a0 20 read_log 194422'16439 (194422'16438) modify d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by client.1148203.0:4972 2014-08-13 15:32:34.722380 2014-08-13 17:38:03.024659 7fe9bae887a0 20 read_log 194422'16440 (0'0) promote 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by osd.374.0:6111 2014-08-13 15:32:35.576832 2014-08-13 17:38:03.024672 7fe9bae887a0 20 read_log 194422'16441 (194422'16440) modify 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by client.1148203.0:5149 2014-08-13 15:32:35.566917 2014-08-13 17:38:03.024684 7fe9bae887a0 20 read_log 194422'16442 (0'0) promote ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by osd.374.0:6143 2014-08-13 15:32:42.690107 2014-08-13 17:38:03.024696 7fe9bae887a0 20 read_log 194422'16443 (194422'16442) modify ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by client.1148203.0:6469 2014-08-13 15:32:42.681075 2014-08-13 17:38:03.024708 7fe9bae887a0 20 read_log 194422'16444 (0'0) promote 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by osd.374.0:6157 2014-08-13 15:32:46.664082 2014-08-13 17:38:03.024720 7fe9bae887a0 20 read_log 194422'16445 (194422'16444) modify 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by client.1148203.0:7183 2014-08-13 15:32:46.654917 2014-08-13 17:38:03.024732 7fe9bae887a0 20 read_log 194422'16446 (0'0) promote 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by osd.374.0:6163 2014-08-13 15:32:47.391190 2014-08-13 17:38:03.024744 7fe9bae887a0 20 read_log 194422'16447 (194422'16446) modify 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by client.1148203.0:7305 2014-08-13 15:32:47.382373 2014-08-13 17:38:03.024756 7fe9bae887a0 20 read_log 194422'16448 (0'0) promote 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by osd.374.0:6219 2014-08-13 15:33:02.415725 2014-08-13 17:38:03.024768 7fe9bae887a0 20 read_log 194422'16449 (194422'16448) modify 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by client.1148203.0:9290 2014-08-13 15:33:02.407003 2014-08-13 17:38:03.024783 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.025238 7fe9bae887a0 10 read_log checking for missing items over interval (194370'16342,194422'16449] 2014-08-13 17:38:03.025285 7fe9bae887a0 15 read_log missing 194422'16449 (194422'16448) modify 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by client.1148203.0:9290 2014-08-13 15:33:02.407003 2014-08-13 17:38:03.025325 7fe9bae887a0 15 read_log missing 194422'16447 (194422'16446) modify 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by client.1148203.0:7305 2014-08-13 15:32:47.382373 2014-08-13 17:38:03.025362 7fe9bae887a0 15 read_log missing 194422'16445 (194422'16444) modify 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by client.1148203.0:7183 2014-08-13 15:32:46.654917 2014-08-13 17:38:03.025399 7fe9bae887a0 15 read_log missing 194422'16443 (194422'16442) modify ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by client.1148203.0:6469 2014-08-13 15:32:42.681075 2014-08-13 17:38:03.025436 7fe9bae887a0 15 read_log missing 194422'16441 (194422'16440) modify 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by client.1148203.0:5149 2014-08-13 15:32:35.566917 2014-08-13 17:38:03.025473 7fe9bae887a0 15 read_log missing 194422'16439 (194422'16438) modify d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by client.1148203.0:4972 2014-08-13 15:32:34.722380 2014-08-13 17:38:03.025511 7fe9bae887a0 15 read_log missing 194422'16437 (194422'16436) modify e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by client.1148203.0:4953 2014-08-13 15:32:34.652060 2014-08-13 17:38:03.025548 7fe9bae887a0 15 read_log missing 194422'16435 (194422'16434) modify 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by client.1148203.0:4579 2014-08-13 15:32:33.206430 2014-08-13 17:38:03.025585 7fe9bae887a0 15 read_log missing 194422'16433 (194422'16432) modify 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by client.1148203.0:4081 2014-08-13 15:32:31.305194 2014-08-13 17:38:03.025622 7fe9bae887a0 15 read_log missing 194422'16431 (194422'16430) modify ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by client.1148203.0:3316 2014-08-13 15:32:28.348962 2014-08-13 17:38:03.025659 7fe9bae887a0 15 read_log missing 194422'16429 (194422'16428) modify f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by client.1148203.0:2918 2014-08-13 15:32:26.785090 2014-08-13 17:38:03.025696 7fe9bae887a0 15 read_log missing 194422'16427 (194422'16426) modify f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by client.1148203.0:1558 2014-08-13 15:32:21.449286 2014-08-13 17:38:03.025733 7fe9bae887a0 15 read_log missing 194422'16425 (194422'16424) modify 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by client.1148203.0:516 2014-08-13 15:32:17.381974 2014-08-13 17:38:03.025770 7fe9bae887a0 15 read_log missing 194416'16423 (194416'16422) modify bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by client.2441958.0:18154 2014-08-13 15:30:03.598192 2014-08-13 17:38:03.025807 7fe9bae887a0 15 read_log missing 194416'16421 (194416'16420) modify 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by client.2441958.0:17840 2014-08-13 15:29:59.173753 2014-08-13 17:38:03.025845 7fe9bae887a0 15 read_log missing 194416'16419 (194416'16418) modify db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by client.2441958.0:16688 2014-08-13 15:29:42.865793 2014-08-13 17:38:03.025902 7fe9bae887a0 15 read_log missing 194416'16417 (194416'16416) modify 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by client.2441958.0:16612 2014-08-13 15:29:41.812045 2014-08-13 17:38:03.025942 7fe9bae887a0 15 read_log missing 194416'16415 (194416'16414) modify f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by client.2441958.0:14032 2014-08-13 15:29:26.325083 2014-08-13 17:38:03.025979 7fe9bae887a0 15 read_log missing 194416'16413 (194416'16412) modify 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by client.2441958.0:14020 2014-08-13 15:29:26.280813 2014-08-13 17:38:03.026016 7fe9bae887a0 15 read_log missing 194416'16411 (194416'16410) modify e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by client.2441958.0:13223 2014-08-13 15:29:22.895758 2014-08-13 17:38:03.026053 7fe9bae887a0 15 read_log missing 194416'16409 (194416'16408) modify d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by client.2441958.0:11763 2014-08-13 15:29:16.560433 2014-08-13 17:38:03.026090 7fe9bae887a0 15 read_log missing 194416'16407 (194416'16406) modify b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by client.2441958.0:11505 2014-08-13 15:29:15.506072 2014-08-13 17:38:03.026128 7fe9bae887a0 15 read_log missing 194416'16405 (194416'16404) modify 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by client.2441958.0:10303 2014-08-13 15:29:10.688257 2014-08-13 17:38:03.026171 7fe9bae887a0 15 read_log missing 194416'16403 (194416'16402) modify 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by client.2441958.0:9387 2014-08-13 15:29:06.939232 2014-08-13 17:38:03.026210 7fe9bae887a0 15 read_log missing 194416'16401 (194416'16400) modify d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by client.2441958.0:7342 2014-08-13 15:28:59.046808 2014-08-13 17:38:03.026247 7fe9bae887a0 15 read_log missing 194416'16399 (194416'16398) modify 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by client.2441958.0:6770 2014-08-13 15:28:56.914226 2014-08-13 17:38:03.026285 7fe9bae887a0 15 read_log missing 194416'16397 (194416'16396) modify 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by client.2441958.0:6531 2014-08-13 15:28:55.974403 2014-08-13 17:38:03.026322 7fe9bae887a0 15 read_log missing 194416'16395 (194416'16394) modify 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by client.2441958.0:5273 2014-08-13 15:28:50.932603 2014-08-13 17:38:03.026358 7fe9bae887a0 15 read_log missing 194416'16393 (194416'16392) modify 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by client.2441958.0:3353 2014-08-13 15:28:43.399752 2014-08-13 17:38:03.026394 7fe9bae887a0 15 read_log missing 194416'16391 (194416'16390) modify c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by client.2441958.0:3268 2014-08-13 15:28:43.084947 2014-08-13 17:38:03.026432 7fe9bae887a0 15 read_log missing 194416'16389 (194416'16388) modify 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by client.2441958.0:2754 2014-08-13 15:28:41.147516 2014-08-13 17:38:03.026468 7fe9bae887a0 15 read_log missing 194416'16387 (194416'16386) modify efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by client.2441958.0:1710 2014-08-13 15:28:37.189859 2014-08-13 17:38:03.026509 7fe9bae887a0 15 read_log missing 194416'16384 (194416'16384) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:53:00.019540_2014-08-13 15:28:37.194823/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 17:38:03.026545 7fe9bae887a0 15 read_log missing 194396'16383 (194396'16382) modify dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by client.2235069.0:279 2014-08-12 20:53:00.011801 2014-08-13 17:38:03.026587 7fe9bae887a0 15 read_log missing 194393'16379 (194393'16378) modify 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by client.1064182.0:381 2014-08-12 20:49:27.253733 2014-08-13 17:38:03.026624 7fe9bae887a0 15 read_log missing 194393'16377 (194393'16376) modify a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by client.1064182.0:174 2014-08-12 20:49:26.293716 2014-08-13 17:38:03.026667 7fe9bae887a0 15 read_log missing 194393'16375 (194393'16374) modify fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by client.1064095.0:38 2014-08-12 20:47:26.071882 2014-08-13 17:38:03.026708 7fe9bae887a0 15 read_log missing 194383'16371 (194383'16370) modify 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by client.2231190.0:7419 2014-08-12 20:34:26.535065 2014-08-13 17:38:03.026746 7fe9bae887a0 15 read_log missing 194383'16369 (194383'16368) modify 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by client.2231190.0:5795 2014-08-12 20:34:00.296784 2014-08-13 17:38:03.026783 7fe9bae887a0 15 read_log missing 194380'16367 (194380'16366) modify c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by client.2231190.0:5229 2014-08-12 20:33:26.179871 2014-08-13 17:38:03.026821 7fe9bae887a0 15 read_log missing 194377'16365 (194377'16364) modify 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by client.2231190.0:3430 2014-08-12 20:33:10.995220 2014-08-13 17:38:03.026858 7fe9bae887a0 15 read_log missing 194377'16363 (194377'16362) modify 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by client.2231190.0:3068 2014-08-12 20:33:09.586317 2014-08-13 17:38:03.026895 7fe9bae887a0 15 read_log missing 194377'16361 (194377'16360) modify 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by client.2231190.0:2577 2014-08-12 20:33:07.605310 2014-08-13 17:38:03.026932 7fe9bae887a0 15 read_log missing 194377'16359 (194377'16358) modify 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by client.2231190.0:1211 2014-08-12 20:33:02.102879 2014-08-13 17:38:03.026969 7fe9bae887a0 15 read_log missing 194370'16357 (194370'16356) modify 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by client.1508531.0:13662 2014-08-12 20:31:25.680926 2014-08-13 17:38:03.027006 7fe9bae887a0 15 read_log missing 194370'16355 (194370'16354) modify 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by client.1508531.0:12400 2014-08-12 20:31:14.270603 2014-08-13 17:38:03.027042 7fe9bae887a0 15 read_log missing 194370'16353 (194370'16352) modify 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by client.1508531.0:11419 2014-08-12 20:31:07.830173 2014-08-13 17:38:03.027079 7fe9bae887a0 15 read_log missing 194370'16351 (194370'16350) modify 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by client.1508531.0:7818 2014-08-12 20:30:46.615726 2014-08-13 17:38:03.027116 7fe9bae887a0 15 read_log missing 194370'16349 (194370'16348) modify 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by client.1508531.0:6178 2014-08-12 20:30:38.545133 2014-08-13 17:38:03.027152 7fe9bae887a0 15 read_log missing 194370'16347 (194370'16346) modify 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by client.1508531.0:4410 2014-08-12 20:30:30.848680 2014-08-13 17:38:03.027194 7fe9bae887a0 15 read_log missing 194370'16345 (194370'16344) modify 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by client.1508531.0:3371 2014-08-12 20:30:26.898087 2014-08-13 17:38:03.027231 7fe9bae887a0 15 read_log missing 194370'16343 (194370'16342) modify 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by client.1508531.0:2720 2014-08-12 20:30:24.520471 2014-08-13 17:38:03.027242 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.027270 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive m=52] handle_loaded 2014-08-13 17:38:03.027285 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Initial 0.013659 0 0.000000 2014-08-13 17:38:03.027307 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset 2014-08-13 17:38:03.027317 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194477 2014-08-13 17:38:03.027326 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] log((192834'14495,194422'16449], crt=194422'16449) 2014-08-13 17:38:03.027337 7fe9bae887a0 10 osd.4 194478 pgid 24.30b coll 24.30b_head 2014-08-13 17:38:03.027446 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.30b 2014-08-13 17:38:03.027458 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.027487 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.30b(unlocked)] enter Initial 2014-08-13 17:38:03.027498 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.30b(unlocked)] enter NotTrimming 2014-08-13 17:38:03.027624 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.027719 7fe9bae887a0 20 read_log 192718'15574 (192718'15573) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:1 2014-08-06 15:32:19.580604 2014-08-13 17:38:03.027739 7fe9bae887a0 20 read_log 192718'15575 (192718'15574) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:2 2014-08-06 15:32:19.692600 2014-08-13 17:38:03.027752 7fe9bae887a0 20 read_log 192718'15576 (192718'15575) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:3 2014-08-06 15:32:19.743542 2014-08-13 17:38:03.027764 7fe9bae887a0 20 read_log 192718'15577 (192718'15576) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:4 2014-08-06 15:32:19.800245 2014-08-13 17:38:03.027776 7fe9bae887a0 20 read_log 192718'15578 (192718'15577) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:5 2014-08-06 15:32:19.853298 2014-08-13 17:38:03.027788 7fe9bae887a0 20 read_log 192718'15579 (192718'15578) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:6 2014-08-06 15:32:19.889022 2014-08-13 17:38:03.027800 7fe9bae887a0 20 read_log 192718'15580 (192718'15579) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:7 2014-08-06 15:32:19.913613 2014-08-13 17:38:03.027812 7fe9bae887a0 20 read_log 192718'15581 (192718'15580) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:8 2014-08-06 15:32:20.053339 2014-08-13 17:38:03.027824 7fe9bae887a0 20 read_log 192718'15582 (192718'15581) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:9 2014-08-06 15:32:20.111084 2014-08-13 17:38:03.027835 7fe9bae887a0 20 read_log 192718'15583 (192718'15582) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:10 2014-08-06 15:32:20.143403 2014-08-13 17:38:03.027848 7fe9bae887a0 20 read_log 192718'15584 (192718'15583) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:11 2014-08-06 15:32:20.177363 2014-08-13 17:38:03.027859 7fe9bae887a0 20 read_log 192718'15585 (192718'15584) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:12 2014-08-06 15:32:20.221803 2014-08-13 17:38:03.027880 7fe9bae887a0 20 read_log 192718'15586 (192718'15585) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:13 2014-08-06 15:32:20.424468 2014-08-13 17:38:03.027893 7fe9bae887a0 20 read_log 192718'15587 (192718'15586) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:14 2014-08-06 15:32:20.456094 2014-08-13 17:38:03.027905 7fe9bae887a0 20 read_log 192718'15588 (192718'15587) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:15 2014-08-06 15:32:20.486172 2014-08-13 17:38:03.027916 7fe9bae887a0 20 read_log 192718'15589 (192718'15588) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:16 2014-08-06 15:32:20.516832 2014-08-13 17:38:03.027928 7fe9bae887a0 20 read_log 192718'15590 (192718'15589) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:17 2014-08-06 15:32:21.014821 2014-08-13 17:38:03.027940 7fe9bae887a0 20 read_log 192718'15591 (192718'15590) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:18 2014-08-06 15:32:21.046317 2014-08-13 17:38:03.027952 7fe9bae887a0 20 read_log 192718'15592 (192718'15591) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:19 2014-08-06 15:32:21.498016 2014-08-13 17:38:03.027964 7fe9bae887a0 20 read_log 192718'15593 (192718'15592) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:20 2014-08-06 15:32:21.530318 2014-08-13 17:38:03.027976 7fe9bae887a0 20 read_log 192718'15594 (192718'15593) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:21 2014-08-06 15:32:21.557135 2014-08-13 17:38:03.027988 7fe9bae887a0 20 read_log 192718'15595 (192718'15594) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:22 2014-08-06 15:32:21.650377 2014-08-13 17:38:03.028000 7fe9bae887a0 20 read_log 192718'15596 (192718'15595) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:23 2014-08-06 15:32:22.089062 2014-08-13 17:38:03.028011 7fe9bae887a0 20 read_log 192718'15597 (192718'15596) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:24 2014-08-06 15:32:22.117791 2014-08-13 17:38:03.028023 7fe9bae887a0 20 read_log 192718'15598 (192718'15597) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:25 2014-08-06 15:32:22.144425 2014-08-13 17:38:03.028035 7fe9bae887a0 20 read_log 192718'15599 (192718'15598) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:26 2014-08-06 15:32:22.170562 2014-08-13 17:38:03.028047 7fe9bae887a0 20 read_log 192718'15600 (192718'15599) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:27 2014-08-06 15:32:22.343725 2014-08-13 17:38:03.028059 7fe9bae887a0 20 read_log 192718'15601 (192718'15600) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:28 2014-08-06 15:32:22.388523 2014-08-13 17:38:03.028070 7fe9bae887a0 20 read_log 192718'15602 (192718'15601) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:29 2014-08-06 15:32:22.422259 2014-08-13 17:38:03.028088 7fe9bae887a0 20 read_log 192718'15603 (192718'15602) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:30 2014-08-06 15:32:22.451117 2014-08-13 17:38:03.028100 7fe9bae887a0 20 read_log 192718'15604 (192718'15603) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:31 2014-08-06 15:32:22.912549 2014-08-13 17:38:03.028111 7fe9bae887a0 20 read_log 192718'15605 (192718'15604) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:32 2014-08-06 15:32:22.938720 2014-08-13 17:38:03.028123 7fe9bae887a0 20 read_log 192718'15606 (192718'15605) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:33 2014-08-06 15:32:22.967018 2014-08-13 17:38:03.028135 7fe9bae887a0 20 read_log 192718'15607 (192718'15606) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:34 2014-08-06 15:32:22.997888 2014-08-13 17:38:03.028147 7fe9bae887a0 20 read_log 192718'15608 (192718'15607) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:35 2014-08-06 15:32:23.025778 2014-08-13 17:38:03.028165 7fe9bae887a0 20 read_log 192718'15609 (192718'15608) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:36 2014-08-06 15:32:23.057045 2014-08-13 17:38:03.028189 7fe9bae887a0 20 read_log 192718'15610 (192718'15609) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:37 2014-08-06 15:32:23.082529 2014-08-13 17:38:03.028202 7fe9bae887a0 20 read_log 192718'15611 (192718'15610) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:38 2014-08-06 15:32:23.113545 2014-08-13 17:38:03.028214 7fe9bae887a0 20 read_log 192718'15612 (192718'15611) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:39 2014-08-06 15:32:23.140084 2014-08-13 17:38:03.028226 7fe9bae887a0 20 read_log 192718'15613 (192718'15612) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:40 2014-08-06 15:32:23.169759 2014-08-13 17:38:03.028238 7fe9bae887a0 20 read_log 192718'15614 (192718'15613) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:41 2014-08-06 15:32:23.195278 2014-08-13 17:38:03.028250 7fe9bae887a0 20 read_log 192718'15615 (192718'15614) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:42 2014-08-06 15:32:23.290058 2014-08-13 17:38:03.028262 7fe9bae887a0 20 read_log 192718'15616 (192718'15615) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:43 2014-08-06 15:32:23.488138 2014-08-13 17:38:03.028274 7fe9bae887a0 20 read_log 192718'15617 (192718'15616) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:44 2014-08-06 15:32:23.641047 2014-08-13 17:38:03.028286 7fe9bae887a0 20 read_log 192718'15618 (192718'15617) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:45 2014-08-06 15:32:23.693469 2014-08-13 17:38:03.028301 7fe9bae887a0 20 read_log 192718'15619 (192718'15618) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:46 2014-08-06 15:32:23.721590 2014-08-13 17:38:03.028317 7fe9bae887a0 20 read_log 192718'15620 (192718'15619) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:47 2014-08-06 15:32:24.400743 2014-08-13 17:38:03.028329 7fe9bae887a0 20 read_log 192718'15621 (192718'15620) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:48 2014-08-06 15:32:24.443279 2014-08-13 17:38:03.028341 7fe9bae887a0 20 read_log 192718'15622 (192718'15621) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:49 2014-08-06 15:32:24.487274 2014-08-13 17:38:03.028353 7fe9bae887a0 20 read_log 192718'15623 (192718'15622) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:50 2014-08-06 15:32:24.526395 2014-08-13 17:38:03.028364 7fe9bae887a0 20 read_log 192718'15624 (192718'15623) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:51 2014-08-06 15:32:24.552731 2014-08-13 17:38:03.028376 7fe9bae887a0 20 read_log 192718'15625 (192718'15624) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:52 2014-08-06 15:32:24.582598 2014-08-13 17:38:03.028388 7fe9bae887a0 20 read_log 192718'15626 (192718'15625) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:53 2014-08-06 15:32:24.613349 2014-08-13 17:38:03.028400 7fe9bae887a0 20 read_log 192718'15627 (192718'15626) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:54 2014-08-06 15:32:24.647504 2014-08-13 17:38:03.028412 7fe9bae887a0 20 read_log 192718'15628 (192718'15627) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:55 2014-08-06 15:32:24.849001 2014-08-13 17:38:03.028424 7fe9bae887a0 20 read_log 192718'15629 (192718'15628) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:56 2014-08-06 15:32:24.879150 2014-08-13 17:38:03.028435 7fe9bae887a0 20 read_log 192718'15630 (192718'15629) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:57 2014-08-06 15:32:25.056064 2014-08-13 17:38:03.028447 7fe9bae887a0 20 read_log 192718'15631 (192718'15630) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:58 2014-08-06 15:32:25.081502 2014-08-13 17:38:03.028459 7fe9bae887a0 20 read_log 192718'15632 (192718'15631) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:59 2014-08-06 15:32:25.108178 2014-08-13 17:38:03.028471 7fe9bae887a0 20 read_log 192718'15633 (192718'15632) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:60 2014-08-06 15:32:25.133724 2014-08-13 17:38:03.028490 7fe9bae887a0 20 read_log 192718'15634 (192718'15633) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:61 2014-08-06 15:32:25.159726 2014-08-13 17:38:03.028502 7fe9bae887a0 20 read_log 192718'15635 (192718'15634) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:62 2014-08-06 15:32:25.187642 2014-08-13 17:38:03.028514 7fe9bae887a0 20 read_log 192718'15636 (192718'15635) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:63 2014-08-06 15:32:25.213435 2014-08-13 17:38:03.028525 7fe9bae887a0 20 read_log 192718'15637 (192718'15636) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:64 2014-08-06 15:32:25.246718 2014-08-13 17:38:03.028537 7fe9bae887a0 20 read_log 192718'15638 (192718'15637) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:65 2014-08-06 15:32:25.315430 2014-08-13 17:38:03.028549 7fe9bae887a0 20 read_log 192718'15639 (192718'15638) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:66 2014-08-06 15:32:25.351597 2014-08-13 17:38:03.028561 7fe9bae887a0 20 read_log 192718'15640 (192718'15639) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:67 2014-08-06 15:32:25.454422 2014-08-13 17:38:03.028573 7fe9bae887a0 20 read_log 192718'15641 (192718'15640) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:68 2014-08-06 15:32:25.571455 2014-08-13 17:38:03.028584 7fe9bae887a0 20 read_log 192718'15642 (192718'15641) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:69 2014-08-06 15:32:25.677280 2014-08-13 17:38:03.028596 7fe9bae887a0 20 read_log 192718'15643 (192718'15642) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:70 2014-08-06 15:32:25.702622 2014-08-13 17:38:03.028608 7fe9bae887a0 20 read_log 192718'15644 (192718'15643) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:71 2014-08-06 15:32:25.727944 2014-08-13 17:38:03.028619 7fe9bae887a0 20 read_log 192718'15645 (192718'15644) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:72 2014-08-06 15:32:25.752849 2014-08-13 17:38:03.028631 7fe9bae887a0 20 read_log 192718'15646 (192718'15645) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:73 2014-08-06 15:32:25.778536 2014-08-13 17:38:03.028643 7fe9bae887a0 20 read_log 192718'15647 (192718'15646) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:74 2014-08-06 15:32:25.805235 2014-08-13 17:38:03.028654 7fe9bae887a0 20 read_log 192718'15648 (192718'15647) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:75 2014-08-06 15:32:25.844938 2014-08-13 17:38:03.028666 7fe9bae887a0 20 read_log 192718'15649 (192718'15648) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:76 2014-08-06 15:32:25.870417 2014-08-13 17:38:03.028678 7fe9bae887a0 20 read_log 192718'15650 (192718'15649) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:77 2014-08-06 15:32:25.896148 2014-08-13 17:38:03.028689 7fe9bae887a0 20 read_log 192718'15651 (192718'15650) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:78 2014-08-06 15:32:25.921919 2014-08-13 17:38:03.028701 7fe9bae887a0 20 read_log 192718'15652 (192718'15651) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:79 2014-08-06 15:32:26.358745 2014-08-13 17:38:03.028713 7fe9bae887a0 20 read_log 192718'15653 (192718'15652) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:80 2014-08-06 15:32:26.385430 2014-08-13 17:38:03.028727 7fe9bae887a0 20 read_log 192718'15654 (192718'15653) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:81 2014-08-06 15:32:26.409391 2014-08-13 17:38:03.028741 7fe9bae887a0 20 read_log 192718'15655 (192718'15654) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:82 2014-08-06 15:32:26.433938 2014-08-13 17:38:03.028753 7fe9bae887a0 20 read_log 192718'15656 (192718'15655) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:83 2014-08-06 15:32:26.457708 2014-08-13 17:38:03.028764 7fe9bae887a0 20 read_log 192718'15657 (192718'15656) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:84 2014-08-06 15:32:26.487117 2014-08-13 17:38:03.028783 7fe9bae887a0 20 read_log 192718'15658 (192718'15657) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:85 2014-08-06 15:32:26.514888 2014-08-13 17:38:03.028795 7fe9bae887a0 20 read_log 192718'15659 (192718'15658) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:86 2014-08-06 15:32:26.544492 2014-08-13 17:38:03.028807 7fe9bae887a0 20 read_log 192718'15660 (192718'15659) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:87 2014-08-06 15:32:26.573226 2014-08-13 17:38:03.028819 7fe9bae887a0 20 read_log 192718'15661 (192718'15660) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:88 2014-08-06 15:32:26.599445 2014-08-13 17:38:03.028831 7fe9bae887a0 20 read_log 192718'15662 (192718'15661) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:89 2014-08-06 15:32:26.626167 2014-08-13 17:38:03.028843 7fe9bae887a0 20 read_log 192718'15663 (192718'15662) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:90 2014-08-06 15:32:26.652695 2014-08-13 17:38:03.028854 7fe9bae887a0 20 read_log 192718'15664 (192718'15663) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:91 2014-08-06 15:32:26.678707 2014-08-13 17:38:03.028866 7fe9bae887a0 20 read_log 192718'15665 (192718'15664) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:92 2014-08-06 15:32:26.705214 2014-08-13 17:38:03.028877 7fe9bae887a0 20 read_log 192718'15666 (192718'15665) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:93 2014-08-06 15:32:26.730781 2014-08-13 17:38:03.028889 7fe9bae887a0 20 read_log 192718'15667 (192718'15666) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:94 2014-08-06 15:32:26.756977 2014-08-13 17:38:03.028900 7fe9bae887a0 20 read_log 192718'15668 (192718'15667) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:95 2014-08-06 15:32:26.785347 2014-08-13 17:38:03.028912 7fe9bae887a0 20 read_log 192718'15669 (192718'15668) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:96 2014-08-06 15:32:26.813690 2014-08-13 17:38:03.028924 7fe9bae887a0 20 read_log 192718'15670 (192718'15669) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:97 2014-08-06 15:32:26.842333 2014-08-13 17:38:03.028935 7fe9bae887a0 20 read_log 192718'15671 (192718'15670) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:98 2014-08-06 15:32:26.871918 2014-08-13 17:38:03.028947 7fe9bae887a0 20 read_log 192718'15672 (192718'15671) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:99 2014-08-06 15:32:26.900684 2014-08-13 17:38:03.028958 7fe9bae887a0 20 read_log 192718'15673 (192718'15672) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:100 2014-08-06 15:32:26.929639 2014-08-13 17:38:03.028970 7fe9bae887a0 20 read_log 192718'15674 (192718'15673) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:101 2014-08-06 15:32:26.955041 2014-08-13 17:38:03.028982 7fe9bae887a0 20 read_log 192718'15675 (192718'15674) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:102 2014-08-06 15:32:26.981658 2014-08-13 17:38:03.028993 7fe9bae887a0 20 read_log 192718'15676 (192718'15675) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:103 2014-08-06 15:32:27.007156 2014-08-13 17:38:03.029005 7fe9bae887a0 20 read_log 192718'15677 (192718'15676) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:104 2014-08-06 15:32:27.061791 2014-08-13 17:38:03.029017 7fe9bae887a0 20 read_log 192718'15678 (192718'15677) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:105 2014-08-06 15:32:27.157448 2014-08-13 17:38:03.029029 7fe9bae887a0 20 read_log 192718'15679 (192718'15678) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:106 2014-08-06 15:32:27.182423 2014-08-13 17:38:03.029041 7fe9bae887a0 20 read_log 192718'15680 (192718'15679) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:107 2014-08-06 15:32:27.206050 2014-08-13 17:38:03.029052 7fe9bae887a0 20 read_log 192718'15681 (192718'15680) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:108 2014-08-06 15:32:27.235056 2014-08-13 17:38:03.029070 7fe9bae887a0 20 read_log 192718'15682 (192718'15681) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:109 2014-08-06 15:32:27.259678 2014-08-13 17:38:03.029082 7fe9bae887a0 20 read_log 192718'15683 (192718'15682) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:110 2014-08-06 15:32:27.286879 2014-08-13 17:38:03.029094 7fe9bae887a0 20 read_log 192718'15684 (192718'15683) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:111 2014-08-06 15:32:27.312605 2014-08-13 17:38:03.029106 7fe9bae887a0 20 read_log 192718'15685 (192718'15684) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:112 2014-08-06 15:32:27.337221 2014-08-13 17:38:03.029117 7fe9bae887a0 20 read_log 192718'15686 (192718'15685) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:113 2014-08-06 15:32:27.361964 2014-08-13 17:38:03.029129 7fe9bae887a0 20 read_log 192718'15687 (192718'15686) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:114 2014-08-06 15:32:27.463402 2014-08-13 17:38:03.029141 7fe9bae887a0 20 read_log 192718'15688 (192718'15687) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:115 2014-08-06 15:32:27.531049 2014-08-13 17:38:03.029153 7fe9bae887a0 20 read_log 192718'15689 (192718'15688) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:116 2014-08-06 15:32:27.606088 2014-08-13 17:38:03.029173 7fe9bae887a0 20 read_log 192718'15690 (192718'15689) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:117 2014-08-06 15:32:27.649802 2014-08-13 17:38:03.029185 7fe9bae887a0 20 read_log 192718'15691 (192718'15690) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:118 2014-08-06 15:32:27.709757 2014-08-13 17:38:03.029197 7fe9bae887a0 20 read_log 192718'15692 (192718'15691) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:119 2014-08-06 15:32:27.736318 2014-08-13 17:38:03.029209 7fe9bae887a0 20 read_log 192718'15693 (192718'15692) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:120 2014-08-06 15:32:27.762588 2014-08-13 17:38:03.029221 7fe9bae887a0 20 read_log 192718'15694 (192718'15693) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:121 2014-08-06 15:32:27.787708 2014-08-13 17:38:03.029234 7fe9bae887a0 20 read_log 192718'15695 (192718'15694) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:122 2014-08-06 15:32:27.812620 2014-08-13 17:38:03.029246 7fe9bae887a0 20 read_log 192718'15696 (192718'15695) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:123 2014-08-06 15:32:27.839238 2014-08-13 17:38:03.029259 7fe9bae887a0 20 read_log 192718'15697 (192718'15696) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:124 2014-08-06 15:32:27.865306 2014-08-13 17:38:03.029271 7fe9bae887a0 20 read_log 192718'15698 (192718'15697) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:125 2014-08-06 15:32:27.937336 2014-08-13 17:38:03.029283 7fe9bae887a0 20 read_log 192718'15699 (192718'15698) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:126 2014-08-06 15:32:27.966533 2014-08-13 17:38:03.029295 7fe9bae887a0 20 read_log 192718'15700 (192718'15699) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:127 2014-08-06 15:32:27.993186 2014-08-13 17:38:03.029307 7fe9bae887a0 20 read_log 192718'15701 (192718'15700) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:128 2014-08-06 15:32:28.023756 2014-08-13 17:38:03.029319 7fe9bae887a0 20 read_log 192718'15702 (192718'15701) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:129 2014-08-06 15:32:28.050464 2014-08-13 17:38:03.029330 7fe9bae887a0 20 read_log 192718'15703 (192718'15702) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:130 2014-08-06 15:32:28.101485 2014-08-13 17:38:03.029342 7fe9bae887a0 20 read_log 192718'15704 (192718'15703) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:131 2014-08-06 15:32:28.207307 2014-08-13 17:38:03.029354 7fe9bae887a0 20 read_log 192718'15705 (192718'15704) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:132 2014-08-06 15:32:28.242223 2014-08-13 17:38:03.029391 7fe9bae887a0 20 read_log 192718'15706 (192718'15705) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:133 2014-08-06 15:32:28.268995 2014-08-13 17:38:03.029405 7fe9bae887a0 20 read_log 192718'15707 (192718'15706) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:134 2014-08-06 15:32:28.294469 2014-08-13 17:38:03.029417 7fe9bae887a0 20 read_log 192718'15708 (192718'15707) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:135 2014-08-06 15:32:28.413560 2014-08-13 17:38:03.029429 7fe9bae887a0 20 read_log 192718'15709 (192718'15708) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:136 2014-08-06 15:32:28.460080 2014-08-13 17:38:03.029441 7fe9bae887a0 20 read_log 192718'15710 (192718'15709) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:137 2014-08-06 15:32:28.615753 2014-08-13 17:38:03.029453 7fe9bae887a0 20 read_log 192718'15711 (192718'15710) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:138 2014-08-06 15:32:28.643496 2014-08-13 17:38:03.029484 7fe9bae887a0 20 read_log 192718'15712 (192718'15711) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:139 2014-08-06 15:32:28.684501 2014-08-13 17:38:03.029497 7fe9bae887a0 20 read_log 192718'15713 (192718'15712) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:140 2014-08-06 15:32:28.713122 2014-08-13 17:38:03.029509 7fe9bae887a0 20 read_log 192718'15714 (192718'15713) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:141 2014-08-06 15:32:28.741290 2014-08-13 17:38:03.029520 7fe9bae887a0 20 read_log 192718'15715 (192718'15714) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:142 2014-08-06 15:32:28.768859 2014-08-13 17:38:03.029532 7fe9bae887a0 20 read_log 192718'15716 (192718'15715) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:143 2014-08-06 15:32:29.285443 2014-08-13 17:38:03.029544 7fe9bae887a0 20 read_log 192718'15717 (192718'15716) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:144 2014-08-06 15:32:29.310911 2014-08-13 17:38:03.029556 7fe9bae887a0 20 read_log 192718'15718 (192718'15717) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:145 2014-08-06 15:32:29.336192 2014-08-13 17:38:03.029567 7fe9bae887a0 20 read_log 192718'15719 (192718'15718) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:146 2014-08-06 15:32:29.361498 2014-08-13 17:38:03.029579 7fe9bae887a0 20 read_log 192718'15720 (192718'15719) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:147 2014-08-06 15:32:29.389212 2014-08-13 17:38:03.029591 7fe9bae887a0 20 read_log 192718'15721 (192718'15720) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:148 2014-08-06 15:32:29.417043 2014-08-13 17:38:03.029604 7fe9bae887a0 20 read_log 192718'15722 (192718'15721) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:149 2014-08-06 15:32:29.443457 2014-08-13 17:38:03.029616 7fe9bae887a0 20 read_log 192718'15723 (192718'15722) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:150 2014-08-06 15:32:29.469226 2014-08-13 17:38:03.029628 7fe9bae887a0 20 read_log 192718'15724 (192718'15723) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:151 2014-08-06 15:32:29.503349 2014-08-13 17:38:03.029639 7fe9bae887a0 20 read_log 192718'15725 (192718'15724) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:152 2014-08-06 15:32:29.532822 2014-08-13 17:38:03.029651 7fe9bae887a0 20 read_log 192718'15726 (192718'15725) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:153 2014-08-06 15:32:29.559896 2014-08-13 17:38:03.029663 7fe9bae887a0 20 read_log 192718'15727 (192718'15726) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:154 2014-08-06 15:32:29.589702 2014-08-13 17:38:03.029674 7fe9bae887a0 20 read_log 192718'15728 (192718'15727) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:155 2014-08-06 15:32:29.616076 2014-08-13 17:38:03.029686 7fe9bae887a0 20 read_log 192718'15729 (192718'15728) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:156 2014-08-06 15:32:29.643624 2014-08-13 17:38:03.029705 7fe9bae887a0 20 read_log 192718'15730 (192718'15729) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:157 2014-08-06 15:32:29.671726 2014-08-13 17:38:03.029718 7fe9bae887a0 20 read_log 192718'15731 (192718'15730) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:158 2014-08-06 15:32:29.701999 2014-08-13 17:38:03.029730 7fe9bae887a0 20 read_log 192718'15732 (192718'15731) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:159 2014-08-06 15:32:29.729118 2014-08-13 17:38:03.029741 7fe9bae887a0 20 read_log 192718'15733 (192718'15429) delete 89fe6f0b/core.23865+28+2/head//24 by osd.329.0:16798 2014-08-06 15:32:29.755294 2014-08-13 17:38:03.029754 7fe9bae887a0 20 read_log 192718'15735 (192718'15732) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:160 2014-08-06 15:32:29.760138 2014-08-13 17:38:03.029766 7fe9bae887a0 20 read_log 192718'15736 (192718'15735) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:161 2014-08-06 15:32:29.788898 2014-08-13 17:38:03.029778 7fe9bae887a0 20 read_log 192718'15737 (192718'15736) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:162 2014-08-06 15:32:29.815989 2014-08-13 17:38:03.029789 7fe9bae887a0 20 read_log 192718'15738 (192718'15737) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:163 2014-08-06 15:32:29.844163 2014-08-13 17:38:03.029801 7fe9bae887a0 20 read_log 192718'15739 (192718'15738) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:164 2014-08-06 15:32:29.872463 2014-08-13 17:38:03.029813 7fe9bae887a0 20 read_log 192718'15740 (192718'15739) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:165 2014-08-06 15:32:29.900225 2014-08-13 17:38:03.029825 7fe9bae887a0 20 read_log 192718'15741 (192718'15740) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:166 2014-08-06 15:32:29.930765 2014-08-13 17:38:03.029837 7fe9bae887a0 20 read_log 192718'15742 (192718'15741) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:167 2014-08-06 15:32:29.957888 2014-08-13 17:38:03.029848 7fe9bae887a0 20 read_log 192718'15743 (192718'15742) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:168 2014-08-06 15:32:29.988954 2014-08-13 17:38:03.029860 7fe9bae887a0 20 read_log 192718'15744 (192718'15743) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:169 2014-08-06 15:32:30.021263 2014-08-13 17:38:03.029872 7fe9bae887a0 20 read_log 192718'15745 (192718'15744) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:170 2014-08-06 15:32:30.047725 2014-08-13 17:38:03.029884 7fe9bae887a0 20 read_log 192718'15746 (192718'15745) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:171 2014-08-06 15:32:30.075145 2014-08-13 17:38:03.029897 7fe9bae887a0 20 read_log 192718'15747 (192718'15746) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:172 2014-08-06 15:32:30.104842 2014-08-13 17:38:03.029909 7fe9bae887a0 20 read_log 192718'15748 (192718'15747) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:173 2014-08-06 15:32:30.133522 2014-08-13 17:38:03.029921 7fe9bae887a0 20 read_log 192718'15749 (192718'15748) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:174 2014-08-06 15:32:30.163963 2014-08-13 17:38:03.029933 7fe9bae887a0 20 read_log 192718'15750 (192718'15749) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:175 2014-08-06 15:32:30.192470 2014-08-13 17:38:03.029945 7fe9bae887a0 20 read_log 192718'15751 (192718'15750) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:176 2014-08-06 15:32:30.218896 2014-08-13 17:38:03.029956 7fe9bae887a0 20 read_log 192718'15752 (192718'15751) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:177 2014-08-06 15:32:30.245890 2014-08-13 17:38:03.029968 7fe9bae887a0 20 read_log 192718'15753 (192718'15752) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:178 2014-08-06 15:32:30.277306 2014-08-13 17:38:03.029980 7fe9bae887a0 20 read_log 192718'15754 (192718'15753) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:179 2014-08-06 15:32:30.309189 2014-08-13 17:38:03.029998 7fe9bae887a0 20 read_log 192718'15755 (192718'15754) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:180 2014-08-06 15:32:30.334983 2014-08-13 17:38:03.030011 7fe9bae887a0 20 read_log 192718'15756 (192718'15755) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:181 2014-08-06 15:32:30.365307 2014-08-13 17:38:03.030023 7fe9bae887a0 20 read_log 192718'15757 (192718'15756) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:182 2014-08-06 15:32:30.396031 2014-08-13 17:38:03.030035 7fe9bae887a0 20 read_log 192718'15758 (192718'15757) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:183 2014-08-06 15:32:30.425216 2014-08-13 17:38:03.030047 7fe9bae887a0 20 read_log 192718'15759 (192718'15758) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:184 2014-08-06 15:32:30.455305 2014-08-13 17:38:03.030058 7fe9bae887a0 20 read_log 192718'15760 (192718'15759) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:185 2014-08-06 15:32:30.488465 2014-08-13 17:38:03.030070 7fe9bae887a0 20 read_log 192718'15761 (192718'15760) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:186 2014-08-06 15:32:30.520055 2014-08-13 17:38:03.030082 7fe9bae887a0 20 read_log 192718'15762 (192718'15761) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:187 2014-08-06 15:32:30.761882 2014-08-13 17:38:03.030094 7fe9bae887a0 20 read_log 192718'15763 (192718'15762) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:188 2014-08-06 15:32:30.795007 2014-08-13 17:38:03.030105 7fe9bae887a0 20 read_log 192718'15764 (192718'15763) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:189 2014-08-06 15:32:30.825046 2014-08-13 17:38:03.030117 7fe9bae887a0 20 read_log 192718'15765 (192718'15764) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:190 2014-08-06 15:32:30.857840 2014-08-13 17:38:03.030129 7fe9bae887a0 20 read_log 192718'15766 (192718'15765) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:191 2014-08-06 15:32:30.888511 2014-08-13 17:38:03.030140 7fe9bae887a0 20 read_log 192718'15767 (192718'15766) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:192 2014-08-06 15:32:30.917396 2014-08-13 17:38:03.030152 7fe9bae887a0 20 read_log 192718'15768 (192718'15767) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:193 2014-08-06 15:32:30.946535 2014-08-13 17:38:03.030170 7fe9bae887a0 20 read_log 192718'15769 (192718'15768) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:194 2014-08-06 15:32:30.975519 2014-08-13 17:38:03.030182 7fe9bae887a0 20 read_log 192718'15770 (192718'15769) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:195 2014-08-06 15:32:31.006792 2014-08-13 17:38:03.030194 7fe9bae887a0 20 read_log 192718'15771 (192718'15770) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:196 2014-08-06 15:32:31.039697 2014-08-13 17:38:03.030206 7fe9bae887a0 20 read_log 192718'15772 (192718'15771) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:197 2014-08-06 15:32:31.070106 2014-08-13 17:38:03.030218 7fe9bae887a0 20 read_log 192718'15773 (192718'15772) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:198 2014-08-06 15:32:31.101274 2014-08-13 17:38:03.030229 7fe9bae887a0 20 read_log 192718'15774 (192718'15773) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:199 2014-08-06 15:32:31.130545 2014-08-13 17:38:03.030241 7fe9bae887a0 20 read_log 192718'15775 (192718'15774) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:200 2014-08-06 15:32:31.158673 2014-08-13 17:38:03.030253 7fe9bae887a0 20 read_log 192718'15776 (192718'15775) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:201 2014-08-06 15:32:31.188293 2014-08-13 17:38:03.030264 7fe9bae887a0 20 read_log 192718'15777 (192718'15776) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:202 2014-08-06 15:32:31.218281 2014-08-13 17:38:03.030276 7fe9bae887a0 20 read_log 192718'15778 (192718'15777) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:203 2014-08-06 15:32:31.249829 2014-08-13 17:38:03.030297 7fe9bae887a0 20 read_log 192718'15779 (192718'15778) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:204 2014-08-06 15:32:31.280650 2014-08-13 17:38:03.030310 7fe9bae887a0 20 read_log 192718'15780 (192718'15779) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:205 2014-08-06 15:32:31.310511 2014-08-13 17:38:03.030322 7fe9bae887a0 20 read_log 192718'15781 (192718'15780) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:206 2014-08-06 15:32:31.342265 2014-08-13 17:38:03.030333 7fe9bae887a0 20 read_log 192718'15782 (192718'15781) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:207 2014-08-06 15:32:31.808262 2014-08-13 17:38:03.030345 7fe9bae887a0 20 read_log 192718'15783 (192718'15782) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:208 2014-08-06 15:32:31.839470 2014-08-13 17:38:03.030357 7fe9bae887a0 20 read_log 192718'15784 (192718'15783) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:209 2014-08-06 15:32:31.865758 2014-08-13 17:38:03.030369 7fe9bae887a0 20 read_log 192718'15785 (192718'15784) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:210 2014-08-06 15:32:31.921105 2014-08-13 17:38:03.030380 7fe9bae887a0 20 read_log 192718'15786 (192718'15785) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:211 2014-08-06 15:32:31.953348 2014-08-13 17:38:03.030392 7fe9bae887a0 20 read_log 192718'15787 (192718'15786) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:212 2014-08-06 15:32:32.043388 2014-08-13 17:38:03.030404 7fe9bae887a0 20 read_log 192718'15788 (192718'15787) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:213 2014-08-06 15:32:32.157295 2014-08-13 17:38:03.030416 7fe9bae887a0 20 read_log 192718'15789 (192718'15788) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:214 2014-08-06 15:32:32.198584 2014-08-13 17:38:03.030428 7fe9bae887a0 20 read_log 192718'15790 (192718'15789) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:215 2014-08-06 15:32:32.229528 2014-08-13 17:38:03.030440 7fe9bae887a0 20 read_log 192718'15791 (192718'15790) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:216 2014-08-06 15:32:32.258098 2014-08-13 17:38:03.030452 7fe9bae887a0 20 read_log 192718'15792 (192718'15791) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:217 2014-08-06 15:32:32.287448 2014-08-13 17:38:03.030463 7fe9bae887a0 20 read_log 192718'15793 (192718'15792) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:218 2014-08-06 15:32:32.319038 2014-08-13 17:38:03.030475 7fe9bae887a0 20 read_log 192718'15794 (192718'15793) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:219 2014-08-06 15:32:32.351863 2014-08-13 17:38:03.030487 7fe9bae887a0 20 read_log 192718'15795 (192718'15794) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:220 2014-08-06 15:32:32.503901 2014-08-13 17:38:03.030506 7fe9bae887a0 20 read_log 192718'15796 (192718'15795) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:221 2014-08-06 15:32:32.678899 2014-08-13 17:38:03.030518 7fe9bae887a0 20 read_log 192718'15797 (192718'15796) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:222 2014-08-06 15:32:32.717315 2014-08-13 17:38:03.030534 7fe9bae887a0 20 read_log 192718'15798 (192718'15797) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:223 2014-08-06 15:32:32.746581 2014-08-13 17:38:03.030546 7fe9bae887a0 20 read_log 192718'15799 (192718'15798) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:224 2014-08-06 15:32:32.776221 2014-08-13 17:38:03.030560 7fe9bae887a0 20 read_log 192718'15800 (192718'15799) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:225 2014-08-06 15:32:32.804897 2014-08-13 17:38:03.030572 7fe9bae887a0 20 read_log 192718'15801 (192718'15800) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:226 2014-08-06 15:32:32.832389 2014-08-13 17:38:03.030584 7fe9bae887a0 20 read_log 192718'15802 (192718'15801) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:227 2014-08-06 15:32:32.862095 2014-08-13 17:38:03.030602 7fe9bae887a0 20 read_log 192718'15803 (192718'15802) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:228 2014-08-06 15:32:32.891437 2014-08-13 17:38:03.030614 7fe9bae887a0 20 read_log 192718'15804 (192718'15803) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:229 2014-08-06 15:32:32.920910 2014-08-13 17:38:03.030626 7fe9bae887a0 20 read_log 192718'15805 (192718'15804) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:230 2014-08-06 15:32:32.950345 2014-08-13 17:38:03.030655 7fe9bae887a0 20 read_log 192718'15806 (192718'15805) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:231 2014-08-06 15:32:33.060401 2014-08-13 17:38:03.030667 7fe9bae887a0 20 read_log 192718'15807 (192718'15806) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:232 2014-08-06 15:32:33.093213 2014-08-13 17:38:03.030679 7fe9bae887a0 20 read_log 192718'15808 (192718'15807) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:233 2014-08-06 15:32:33.122825 2014-08-13 17:38:03.030691 7fe9bae887a0 20 read_log 192718'15809 (192718'15808) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:234 2014-08-06 15:32:33.153468 2014-08-13 17:38:03.030703 7fe9bae887a0 20 read_log 192718'15810 (192718'15809) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:235 2014-08-06 15:32:33.184769 2014-08-13 17:38:03.030714 7fe9bae887a0 20 read_log 192718'15811 (192718'15810) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:236 2014-08-06 15:32:33.300763 2014-08-13 17:38:03.030726 7fe9bae887a0 20 read_log 192718'15812 (192718'15811) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:237 2014-08-06 15:32:33.331022 2014-08-13 17:38:03.030738 7fe9bae887a0 20 read_log 192718'15813 (192718'15812) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:238 2014-08-06 15:32:33.363149 2014-08-13 17:38:03.030749 7fe9bae887a0 20 read_log 192718'15814 (192718'15813) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:239 2014-08-06 15:32:33.391820 2014-08-13 17:38:03.030761 7fe9bae887a0 20 read_log 192718'15815 (192718'15814) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:240 2014-08-06 15:32:33.423518 2014-08-13 17:38:03.030773 7fe9bae887a0 20 read_log 192718'15816 (192718'15815) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:241 2014-08-06 15:32:33.582206 2014-08-13 17:38:03.030784 7fe9bae887a0 20 read_log 192718'15817 (192718'15816) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:242 2014-08-06 15:32:33.611622 2014-08-13 17:38:03.030796 7fe9bae887a0 20 read_log 192718'15818 (192718'15817) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:243 2014-08-06 15:32:33.641234 2014-08-13 17:38:03.030808 7fe9bae887a0 20 read_log 192718'15819 (192718'15818) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:244 2014-08-06 15:32:33.676046 2014-08-13 17:38:03.030819 7fe9bae887a0 20 read_log 192718'15820 (192718'15819) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:245 2014-08-06 15:32:33.877051 2014-08-13 17:38:03.030831 7fe9bae887a0 20 read_log 192718'15821 (192718'15820) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:246 2014-08-06 15:32:34.206285 2014-08-13 17:38:03.030843 7fe9bae887a0 20 read_log 192718'15822 (192718'15821) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:247 2014-08-06 15:32:34.301523 2014-08-13 17:38:03.030855 7fe9bae887a0 20 read_log 192718'15823 (192718'15822) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:248 2014-08-06 15:32:34.332088 2014-08-13 17:38:03.030885 7fe9bae887a0 20 read_log 192718'15824 (192718'15823) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:249 2014-08-06 15:32:34.368011 2014-08-13 17:38:03.030898 7fe9bae887a0 20 read_log 192718'15825 (192718'15824) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:250 2014-08-06 15:32:34.419966 2014-08-13 17:38:03.030910 7fe9bae887a0 20 read_log 192718'15826 (192718'15825) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:251 2014-08-06 15:32:34.605399 2014-08-13 17:38:03.030929 7fe9bae887a0 20 read_log 192718'15827 (192718'15826) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:252 2014-08-06 15:32:34.639394 2014-08-13 17:38:03.030941 7fe9bae887a0 20 read_log 192718'15828 (192718'15827) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:253 2014-08-06 15:32:34.669433 2014-08-13 17:38:03.030953 7fe9bae887a0 20 read_log 192718'15829 (192718'15828) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:254 2014-08-06 15:32:34.702937 2014-08-13 17:38:03.030965 7fe9bae887a0 20 read_log 192718'15830 (192718'15829) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:255 2014-08-06 15:32:34.803300 2014-08-13 17:38:03.030976 7fe9bae887a0 20 read_log 192718'15831 (192718'15830) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:256 2014-08-06 15:32:35.255714 2014-08-13 17:38:03.030988 7fe9bae887a0 20 read_log 192718'15832 (192718'15831) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:257 2014-08-06 15:32:35.308846 2014-08-13 17:38:03.031000 7fe9bae887a0 20 read_log 192718'15833 (192718'15832) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:258 2014-08-06 15:32:35.366357 2014-08-13 17:38:03.031012 7fe9bae887a0 20 read_log 192718'15834 (192718'15833) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:259 2014-08-06 15:32:35.415661 2014-08-13 17:38:03.031024 7fe9bae887a0 20 read_log 192718'15835 (192718'15834) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:260 2014-08-06 15:32:35.464969 2014-08-13 17:38:03.031035 7fe9bae887a0 20 read_log 192718'15836 (192718'15835) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:261 2014-08-06 15:32:35.514187 2014-08-13 17:38:03.031047 7fe9bae887a0 20 read_log 192718'15837 (192718'15836) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:262 2014-08-06 15:32:35.568987 2014-08-13 17:38:03.031059 7fe9bae887a0 20 read_log 192718'15838 (192718'15837) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:263 2014-08-06 15:32:35.601145 2014-08-13 17:38:03.031071 7fe9bae887a0 20 read_log 192718'15839 (192718'15838) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:264 2014-08-06 15:32:35.668833 2014-08-13 17:38:03.031082 7fe9bae887a0 20 read_log 192718'15840 (192718'15839) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:265 2014-08-06 15:32:35.699950 2014-08-13 17:38:03.031094 7fe9bae887a0 20 read_log 192718'15841 (192718'15840) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:266 2014-08-06 15:32:35.969883 2014-08-13 17:38:03.031106 7fe9bae887a0 20 read_log 192718'15842 (192718'15841) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:267 2014-08-06 15:32:36.023444 2014-08-13 17:38:03.031118 7fe9bae887a0 20 read_log 192718'15843 (192718'15842) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:268 2014-08-06 15:32:36.080317 2014-08-13 17:38:03.031130 7fe9bae887a0 20 read_log 192718'15844 (192718'15843) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:269 2014-08-06 15:32:36.757327 2014-08-13 17:38:03.031141 7fe9bae887a0 20 read_log 192718'15845 (192718'15844) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:270 2014-08-06 15:32:36.785950 2014-08-13 17:38:03.031158 7fe9bae887a0 20 read_log 192718'15846 (192718'15845) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:271 2014-08-06 15:32:36.937618 2014-08-13 17:38:03.031171 7fe9bae887a0 20 read_log 192718'15847 (192718'15846) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:272 2014-08-06 15:32:36.985462 2014-08-13 17:38:03.031183 7fe9bae887a0 20 read_log 192718'15848 (192718'15847) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:273 2014-08-06 15:32:37.018827 2014-08-13 17:38:03.031194 7fe9bae887a0 20 read_log 192718'15849 (192718'15848) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:274 2014-08-06 15:32:37.049528 2014-08-13 17:38:03.031206 7fe9bae887a0 20 read_log 192718'15850 (192718'15849) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:275 2014-08-06 15:32:37.083157 2014-08-13 17:38:03.031227 7fe9bae887a0 20 read_log 192718'15851 (192718'15850) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:276 2014-08-06 15:32:37.258908 2014-08-13 17:38:03.031240 7fe9bae887a0 20 read_log 192718'15852 (192718'15851) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:277 2014-08-06 15:32:37.287544 2014-08-13 17:38:03.031252 7fe9bae887a0 20 read_log 192718'15853 (192718'15852) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:278 2014-08-06 15:32:37.316662 2014-08-13 17:38:03.031264 7fe9bae887a0 20 read_log 192718'15854 (192718'15853) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:279 2014-08-06 15:32:37.345721 2014-08-13 17:38:03.031276 7fe9bae887a0 20 read_log 192718'15855 (192718'15854) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:280 2014-08-06 15:32:37.373466 2014-08-13 17:38:03.031287 7fe9bae887a0 20 read_log 192718'15856 (192718'15855) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:281 2014-08-06 15:32:37.403193 2014-08-13 17:38:03.031299 7fe9bae887a0 20 read_log 192718'15857 (192718'15856) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:282 2014-08-06 15:32:37.432064 2014-08-13 17:38:03.031311 7fe9bae887a0 20 read_log 192718'15858 (192718'15857) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:283 2014-08-06 15:32:37.591303 2014-08-13 17:38:03.031323 7fe9bae887a0 20 read_log 192718'15859 (192718'15858) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:284 2014-08-06 15:32:37.621113 2014-08-13 17:38:03.031335 7fe9bae887a0 20 read_log 192718'15860 (192718'15859) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:285 2014-08-06 15:32:37.650187 2014-08-13 17:38:03.031347 7fe9bae887a0 20 read_log 192718'15861 (192718'15860) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:286 2014-08-06 15:32:37.677911 2014-08-13 17:38:03.031359 7fe9bae887a0 20 read_log 192718'15862 (192718'15861) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:287 2014-08-06 15:32:37.706028 2014-08-13 17:38:03.031371 7fe9bae887a0 20 read_log 192718'15863 (192718'15862) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:288 2014-08-06 15:32:37.735929 2014-08-13 17:38:03.031382 7fe9bae887a0 20 read_log 192718'15864 (192718'15863) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:289 2014-08-06 15:32:37.764668 2014-08-13 17:38:03.031394 7fe9bae887a0 20 read_log 192718'15865 (192718'15864) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:290 2014-08-06 15:32:37.795652 2014-08-13 17:38:03.031406 7fe9bae887a0 20 read_log 192718'15866 (192718'15865) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:291 2014-08-06 15:32:37.823011 2014-08-13 17:38:03.031418 7fe9bae887a0 20 read_log 192718'15867 (192718'15866) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:292 2014-08-06 15:32:37.854278 2014-08-13 17:38:03.031430 7fe9bae887a0 20 read_log 192718'15868 (192718'15867) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:293 2014-08-06 15:32:37.951764 2014-08-13 17:38:03.031441 7fe9bae887a0 20 read_log 192718'15869 (192718'15868) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:294 2014-08-06 15:32:37.986952 2014-08-13 17:38:03.031453 7fe9bae887a0 20 read_log 192718'15870 (192718'15869) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:295 2014-08-06 15:32:38.018236 2014-08-13 17:38:03.031465 7fe9bae887a0 20 read_log 192718'15871 (192718'15870) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:296 2014-08-06 15:32:38.067222 2014-08-13 17:38:03.031477 7fe9bae887a0 20 read_log 192718'15872 (192718'15871) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:297 2014-08-06 15:32:38.099743 2014-08-13 17:38:03.031489 7fe9bae887a0 20 read_log 192718'15873 (192718'15872) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:298 2014-08-06 15:32:38.151085 2014-08-13 17:38:03.031501 7fe9bae887a0 20 read_log 192718'15874 (192718'15873) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:299 2014-08-06 15:32:38.346945 2014-08-13 17:38:03.031536 7fe9bae887a0 20 read_log 192718'15875 (192718'15874) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:300 2014-08-06 15:32:38.378868 2014-08-13 17:38:03.031550 7fe9bae887a0 20 read_log 192718'15876 (192718'15875) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:301 2014-08-06 15:32:38.412130 2014-08-13 17:38:03.031562 7fe9bae887a0 20 read_log 192718'15877 (192718'15876) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:302 2014-08-06 15:32:38.444119 2014-08-13 17:38:03.031574 7fe9bae887a0 20 read_log 192718'15878 (192718'15877) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:303 2014-08-06 15:32:38.680306 2014-08-13 17:38:03.031586 7fe9bae887a0 20 read_log 192718'15879 (192718'15878) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:304 2014-08-06 15:32:38.717013 2014-08-13 17:38:03.031598 7fe9bae887a0 20 read_log 192718'15880 (192718'15879) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:305 2014-08-06 15:32:38.745250 2014-08-13 17:38:03.031610 7fe9bae887a0 20 read_log 192718'15881 (192718'15880) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:306 2014-08-06 15:32:38.778823 2014-08-13 17:38:03.031622 7fe9bae887a0 20 read_log 192718'15882 (192718'15881) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:307 2014-08-06 15:32:38.807525 2014-08-13 17:38:03.031634 7fe9bae887a0 20 read_log 192718'15883 (192718'15882) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:308 2014-08-06 15:32:38.985846 2014-08-13 17:38:03.031646 7fe9bae887a0 20 read_log 192718'15884 (192718'15883) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:309 2014-08-06 15:32:39.029032 2014-08-13 17:38:03.031658 7fe9bae887a0 20 read_log 192718'15885 (192718'15884) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:310 2014-08-06 15:32:39.058417 2014-08-13 17:38:03.031670 7fe9bae887a0 20 read_log 192718'15886 (192718'15885) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:311 2014-08-06 15:32:39.090498 2014-08-13 17:38:03.031682 7fe9bae887a0 20 read_log 192718'15887 (192718'15886) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:312 2014-08-06 15:32:39.127763 2014-08-13 17:38:03.031693 7fe9bae887a0 20 read_log 192718'15888 (192718'15887) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:313 2014-08-06 15:32:39.295909 2014-08-13 17:38:03.031705 7fe9bae887a0 20 read_log 192718'15889 (192718'15888) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:314 2014-08-06 15:32:39.325375 2014-08-13 17:38:03.031717 7fe9bae887a0 20 read_log 192718'15890 (192718'15889) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:315 2014-08-06 15:32:39.352790 2014-08-13 17:38:03.031729 7fe9bae887a0 20 read_log 192718'15891 (192718'15890) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:316 2014-08-06 15:32:39.380442 2014-08-13 17:38:03.031741 7fe9bae887a0 20 read_log 192718'15892 (192718'15891) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:317 2014-08-06 15:32:39.410129 2014-08-13 17:38:03.031753 7fe9bae887a0 20 read_log 192718'15893 (192718'15892) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:318 2014-08-06 15:32:39.574470 2014-08-13 17:38:03.031767 7fe9bae887a0 20 read_log 192718'15894 (192718'15893) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:319 2014-08-06 15:32:39.604596 2014-08-13 17:38:03.031779 7fe9bae887a0 20 read_log 192718'15895 (192718'15894) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:320 2014-08-06 15:32:39.643024 2014-08-13 17:38:03.031791 7fe9bae887a0 20 read_log 192718'15896 (192718'15895) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:321 2014-08-06 15:32:39.671328 2014-08-13 17:38:03.031803 7fe9bae887a0 20 read_log 192718'15897 (192718'15896) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:322 2014-08-06 15:32:39.700551 2014-08-13 17:38:03.031815 7fe9bae887a0 20 read_log 192718'15898 (192718'15897) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:323 2014-08-06 15:32:39.854787 2014-08-13 17:38:03.031853 7fe9bae887a0 20 read_log 192718'15899 (192718'15898) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:324 2014-08-06 15:32:39.883585 2014-08-13 17:38:03.031866 7fe9bae887a0 20 read_log 192718'15900 (192718'15899) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:325 2014-08-06 15:32:39.915875 2014-08-13 17:38:03.031878 7fe9bae887a0 20 read_log 192718'15901 (192718'15900) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:326 2014-08-06 15:32:40.081389 2014-08-13 17:38:03.031890 7fe9bae887a0 20 read_log 192718'15902 (192718'15901) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:327 2014-08-06 15:32:40.111398 2014-08-13 17:38:03.031902 7fe9bae887a0 20 read_log 192718'15903 (192718'15902) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:328 2014-08-06 15:32:40.146788 2014-08-13 17:38:03.031914 7fe9bae887a0 20 read_log 192718'15904 (192718'15903) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:329 2014-08-06 15:32:40.174377 2014-08-13 17:38:03.031926 7fe9bae887a0 20 read_log 192718'15905 (192718'15904) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:330 2014-08-06 15:32:40.201925 2014-08-13 17:38:03.031940 7fe9bae887a0 20 read_log 192718'15906 (192718'15905) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:331 2014-08-06 15:32:40.229458 2014-08-13 17:38:03.031952 7fe9bae887a0 20 read_log 192718'15907 (192718'15906) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:332 2014-08-06 15:32:40.684087 2014-08-13 17:38:03.031963 7fe9bae887a0 20 read_log 192718'15908 (192718'15907) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:333 2014-08-06 15:32:40.712206 2014-08-13 17:38:03.031975 7fe9bae887a0 20 read_log 192718'15909 (192718'15908) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:334 2014-08-06 15:32:40.740754 2014-08-13 17:38:03.031987 7fe9bae887a0 20 read_log 192718'15910 (192718'15909) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:335 2014-08-06 15:32:40.767189 2014-08-13 17:38:03.031999 7fe9bae887a0 20 read_log 192718'15911 (192718'15910) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:336 2014-08-06 15:32:40.789228 2014-08-13 17:38:03.032011 7fe9bae887a0 20 read_log 192803'15912 (192718'15911) clean 168b030b/usr.tar2+63+2/head//24 by osd.329.0:17126 2014-08-11 10:25:18.791671 2014-08-13 17:38:03.032023 7fe9bae887a0 20 read_log 192803'15914 (192803'15912) delete 168b030b/usr.tar2+63+2/head//24 by osd.329.0:17128 2014-08-11 10:25:23.011486 2014-08-13 17:38:03.032034 7fe9bae887a0 20 read_log 192803'15920 (0'0) promote 38370f0b/core.31758+10+2/head//24 by osd.329.0:17308 2014-08-11 12:41:26.692787 2014-08-13 17:38:03.032047 7fe9bae887a0 20 read_log 192803'15921 (192803'15920) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:1 2014-08-11 12:41:14.584333 2014-08-13 17:38:03.032059 7fe9bae887a0 20 read_log 192803'15922 (192803'15921) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:2 2014-08-11 12:41:29.908617 2014-08-13 17:38:03.032070 7fe9bae887a0 20 read_log 192803'15923 (192803'15922) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:3 2014-08-11 12:41:30.206228 2014-08-13 17:38:03.032082 7fe9bae887a0 20 read_log 192803'15924 (192803'15923) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:4 2014-08-11 12:41:30.234759 2014-08-13 17:38:03.032096 7fe9bae887a0 20 read_log 192803'15925 (192803'15924) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:5 2014-08-11 12:41:30.275220 2014-08-13 17:38:03.032108 7fe9bae887a0 20 read_log 192803'15926 (192803'15925) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:6 2014-08-11 12:41:30.301367 2014-08-13 17:38:03.032120 7fe9bae887a0 20 read_log 192803'15927 (192803'15926) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:7 2014-08-11 12:41:30.332589 2014-08-13 17:38:03.032131 7fe9bae887a0 20 read_log 192803'15928 (192803'15927) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:8 2014-08-11 12:41:30.372722 2014-08-13 17:38:03.032150 7fe9bae887a0 20 read_log 192803'15929 (192803'15928) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:9 2014-08-11 12:41:30.446094 2014-08-13 17:38:03.032167 7fe9bae887a0 20 read_log 192803'15930 (192803'15929) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:10 2014-08-11 12:41:30.484689 2014-08-13 17:38:03.032180 7fe9bae887a0 20 read_log 192803'15931 (192803'15930) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:11 2014-08-11 12:41:30.524245 2014-08-13 17:38:03.032193 7fe9bae887a0 20 read_log 192803'15932 (192803'15931) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:12 2014-08-11 12:41:30.563772 2014-08-13 17:38:03.032205 7fe9bae887a0 20 read_log 192803'15933 (192803'15932) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:13 2014-08-11 12:41:30.602676 2014-08-13 17:38:03.032218 7fe9bae887a0 20 read_log 192803'15934 (192803'15933) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:14 2014-08-11 12:41:31.031106 2014-08-13 17:38:03.032230 7fe9bae887a0 20 read_log 192803'15935 (192803'15934) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:15 2014-08-11 12:41:31.069165 2014-08-13 17:38:03.032243 7fe9bae887a0 20 read_log 192803'15936 (192803'15935) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:16 2014-08-11 12:41:31.113055 2014-08-13 17:38:03.032255 7fe9bae887a0 20 read_log 192803'15937 (192803'15936) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:17 2014-08-11 12:41:31.158425 2014-08-13 17:38:03.032267 7fe9bae887a0 20 read_log 192803'15938 (192803'15937) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:18 2014-08-11 12:41:31.190003 2014-08-13 17:38:03.032279 7fe9bae887a0 20 read_log 192803'15939 (192803'15938) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:19 2014-08-11 12:41:31.228294 2014-08-13 17:38:03.032291 7fe9bae887a0 20 read_log 192803'15940 (192803'15939) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:20 2014-08-11 12:41:31.267155 2014-08-13 17:38:03.032303 7fe9bae887a0 20 read_log 192803'15941 (192803'15940) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:21 2014-08-11 12:41:31.305578 2014-08-13 17:38:03.032315 7fe9bae887a0 20 read_log 192803'15942 (192803'15941) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:22 2014-08-11 12:41:31.731234 2014-08-13 17:38:03.032326 7fe9bae887a0 20 read_log 192803'15943 (192803'15942) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:23 2014-08-11 12:41:31.769503 2014-08-13 17:38:03.032338 7fe9bae887a0 20 read_log 192803'15944 (192803'15943) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:24 2014-08-11 12:41:31.806968 2014-08-13 17:38:03.032350 7fe9bae887a0 20 read_log 192803'15945 (192803'15944) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:25 2014-08-11 12:41:31.843653 2014-08-13 17:38:03.032362 7fe9bae887a0 20 read_log 192803'15946 (192803'15945) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:26 2014-08-11 12:41:31.888164 2014-08-13 17:38:03.032374 7fe9bae887a0 20 read_log 192803'15947 (192803'15946) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:27 2014-08-11 12:41:31.924983 2014-08-13 17:38:03.032386 7fe9bae887a0 20 read_log 192803'15948 (192803'15947) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:28 2014-08-11 12:41:31.965521 2014-08-13 17:38:03.032397 7fe9bae887a0 20 read_log 192803'15949 (192803'15948) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:29 2014-08-11 12:41:32.003940 2014-08-13 17:38:03.032409 7fe9bae887a0 20 read_log 192803'15950 (192803'15949) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:30 2014-08-11 12:41:32.043532 2014-08-13 17:38:03.032421 7fe9bae887a0 20 read_log 192803'15951 (192803'15950) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:31 2014-08-11 12:41:32.087114 2014-08-13 17:38:03.032433 7fe9bae887a0 20 read_log 192803'15952 (192803'15951) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:32 2014-08-11 12:41:32.519446 2014-08-13 17:38:03.032455 7fe9bae887a0 20 read_log 192803'15953 (192803'15952) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:33 2014-08-11 12:41:32.603633 2014-08-13 17:38:03.032467 7fe9bae887a0 20 read_log 192803'15954 (192803'15953) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:34 2014-08-11 12:41:32.652710 2014-08-13 17:38:03.032479 7fe9bae887a0 20 read_log 192803'15955 (192803'15954) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:35 2014-08-11 12:41:32.688000 2014-08-13 17:38:03.032491 7fe9bae887a0 20 read_log 192803'15956 (192803'15955) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:36 2014-08-11 12:41:32.727064 2014-08-13 17:38:03.032503 7fe9bae887a0 20 read_log 192803'15957 (192803'15956) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:37 2014-08-11 12:41:32.767510 2014-08-13 17:38:03.032514 7fe9bae887a0 20 read_log 192803'15958 (192803'15957) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:38 2014-08-11 12:41:32.807719 2014-08-13 17:38:03.032526 7fe9bae887a0 20 read_log 192803'15959 (192803'15958) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:39 2014-08-11 12:41:32.849469 2014-08-13 17:38:03.032538 7fe9bae887a0 20 read_log 192803'15960 (192803'15959) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:40 2014-08-11 12:41:32.888526 2014-08-13 17:38:03.032570 7fe9bae887a0 20 read_log 192803'15961 (192803'15960) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:41 2014-08-11 12:41:32.928690 2014-08-13 17:38:03.032582 7fe9bae887a0 20 read_log 192803'15962 (192803'15961) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:42 2014-08-11 12:41:32.967777 2014-08-13 17:38:03.032594 7fe9bae887a0 20 read_log 192803'15963 (192803'15962) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:43 2014-08-11 12:41:33.006951 2014-08-13 17:38:03.032606 7fe9bae887a0 20 read_log 192803'15964 (192803'15963) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:44 2014-08-11 12:41:33.479763 2014-08-13 17:38:03.032618 7fe9bae887a0 20 read_log 192803'15965 (192803'15964) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:45 2014-08-11 12:41:33.523917 2014-08-13 17:38:03.032631 7fe9bae887a0 20 read_log 192803'15966 (192803'15965) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:46 2014-08-11 12:41:33.564443 2014-08-13 17:38:03.032643 7fe9bae887a0 20 read_log 192803'15967 (192803'15966) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:47 2014-08-11 12:41:33.603004 2014-08-13 17:38:03.032654 7fe9bae887a0 20 read_log 192803'15968 (192803'15967) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:48 2014-08-11 12:41:33.642330 2014-08-13 17:38:03.032666 7fe9bae887a0 20 read_log 192803'15969 (192803'15968) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:49 2014-08-11 12:41:33.680993 2014-08-13 17:38:03.032678 7fe9bae887a0 20 read_log 192803'15970 (192803'15969) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:50 2014-08-11 12:41:33.718934 2014-08-13 17:38:03.032690 7fe9bae887a0 20 read_log 192803'15971 (192803'15970) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:51 2014-08-11 12:41:33.760369 2014-08-13 17:38:03.032701 7fe9bae887a0 20 read_log 192803'15972 (192803'15971) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:52 2014-08-11 12:41:33.800733 2014-08-13 17:38:03.032713 7fe9bae887a0 20 read_log 192803'15973 (192803'15972) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:53 2014-08-11 12:41:33.839582 2014-08-13 17:38:03.032726 7fe9bae887a0 20 read_log 192803'15974 (192803'15973) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:54 2014-08-11 12:41:34.280119 2014-08-13 17:38:03.032739 7fe9bae887a0 20 read_log 192803'15975 (192803'15974) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:55 2014-08-11 12:41:34.330195 2014-08-13 17:38:03.032751 7fe9bae887a0 20 read_log 192803'15976 (192803'15975) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:56 2014-08-11 12:41:34.372370 2014-08-13 17:38:03.032769 7fe9bae887a0 20 read_log 192803'15977 (192803'15976) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:57 2014-08-11 12:41:34.412854 2014-08-13 17:38:03.032782 7fe9bae887a0 20 read_log 192803'15978 (192803'15977) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:58 2014-08-11 12:41:34.454883 2014-08-13 17:38:03.032794 7fe9bae887a0 20 read_log 192803'15979 (192803'15978) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:59 2014-08-11 12:41:34.496574 2014-08-13 17:38:03.032805 7fe9bae887a0 20 read_log 192803'15980 (192803'15979) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:60 2014-08-11 12:41:34.538729 2014-08-13 17:38:03.032817 7fe9bae887a0 20 read_log 192803'15981 (192803'15980) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:61 2014-08-11 12:41:34.581158 2014-08-13 17:38:03.032829 7fe9bae887a0 20 read_log 192803'15982 (192803'15981) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:62 2014-08-11 12:41:34.622644 2014-08-13 17:38:03.032841 7fe9bae887a0 20 read_log 192803'15983 (192803'15982) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:63 2014-08-11 12:41:34.663544 2014-08-13 17:38:03.032852 7fe9bae887a0 20 read_log 192803'15984 (192803'15983) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:64 2014-08-11 12:41:35.123383 2014-08-13 17:38:03.032864 7fe9bae887a0 20 read_log 192803'15985 (192803'15984) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:65 2014-08-11 12:41:35.164421 2014-08-13 17:38:03.032876 7fe9bae887a0 20 read_log 192803'15986 (192803'15985) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:66 2014-08-11 12:41:35.209219 2014-08-13 17:38:03.032888 7fe9bae887a0 20 read_log 192803'15987 (192803'15986) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:67 2014-08-11 12:41:35.237741 2014-08-13 17:38:03.032899 7fe9bae887a0 20 read_log 192803'15988 (192803'15987) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:68 2014-08-11 12:41:35.263654 2014-08-13 17:38:03.032911 7fe9bae887a0 20 read_log 192803'15989 (192803'15988) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:69 2014-08-11 12:41:35.291033 2014-08-13 17:38:03.032923 7fe9bae887a0 20 read_log 192803'15990 (192803'15989) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:70 2014-08-11 12:41:35.335372 2014-08-13 17:38:03.032935 7fe9bae887a0 20 read_log 192803'15991 (192803'15990) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:71 2014-08-11 12:41:35.362239 2014-08-13 17:38:03.032946 7fe9bae887a0 20 read_log 192803'15992 (192803'15991) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:72 2014-08-11 12:41:35.387605 2014-08-13 17:38:03.032958 7fe9bae887a0 20 read_log 192803'15993 (192803'15992) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:73 2014-08-11 12:41:35.412993 2014-08-13 17:38:03.032970 7fe9bae887a0 20 read_log 192803'15994 (192803'15993) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:74 2014-08-11 12:41:35.702376 2014-08-13 17:38:03.032981 7fe9bae887a0 20 read_log 192803'15995 (192803'15994) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:75 2014-08-11 12:41:35.734449 2014-08-13 17:38:03.032993 7fe9bae887a0 20 read_log 192803'15996 (192803'15995) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:76 2014-08-11 12:41:35.761356 2014-08-13 17:38:03.033004 7fe9bae887a0 20 read_log 192803'15997 (192803'15996) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:77 2014-08-11 12:41:35.788815 2014-08-13 17:38:03.033018 7fe9bae887a0 20 read_log 192803'15998 (192803'15997) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:78 2014-08-11 12:41:35.816034 2014-08-13 17:38:03.033029 7fe9bae887a0 20 read_log 192803'15999 (192803'15998) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:79 2014-08-11 12:41:35.843437 2014-08-13 17:38:03.033041 7fe9bae887a0 20 read_log 192803'16000 (192803'15999) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:80 2014-08-11 12:41:35.870187 2014-08-13 17:38:03.033060 7fe9bae887a0 20 read_log 192803'16001 (192803'16000) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:81 2014-08-11 12:41:35.898619 2014-08-13 17:38:03.033072 7fe9bae887a0 20 read_log 192803'16002 (192803'16001) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:82 2014-08-11 12:41:35.926316 2014-08-13 17:38:03.033084 7fe9bae887a0 20 read_log 192803'16003 (192803'16002) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:83 2014-08-11 12:41:35.953374 2014-08-13 17:38:03.033096 7fe9bae887a0 20 read_log 192803'16004 (192803'16003) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:84 2014-08-11 12:41:36.255486 2014-08-13 17:38:03.033107 7fe9bae887a0 20 read_log 192803'16005 (192803'16004) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:85 2014-08-11 12:41:36.286134 2014-08-13 17:38:03.033122 7fe9bae887a0 20 read_log 192803'16006 (192803'16005) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:86 2014-08-11 12:41:36.314678 2014-08-13 17:38:03.033134 7fe9bae887a0 20 read_log 192803'16007 (192803'16006) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:87 2014-08-11 12:41:36.341156 2014-08-13 17:38:03.033146 7fe9bae887a0 20 read_log 192803'16008 (192803'16007) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:88 2014-08-11 12:41:36.368535 2014-08-13 17:38:03.033167 7fe9bae887a0 20 read_log 192803'16009 (192803'16008) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:89 2014-08-11 12:41:36.395850 2014-08-13 17:38:03.033179 7fe9bae887a0 20 read_log 192803'16010 (192803'16009) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:90 2014-08-11 12:41:36.423477 2014-08-13 17:38:03.033191 7fe9bae887a0 20 read_log 192803'16011 (192803'16010) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:91 2014-08-11 12:41:36.450384 2014-08-13 17:38:03.033203 7fe9bae887a0 20 read_log 192803'16012 (192803'16011) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:92 2014-08-11 12:41:36.477995 2014-08-13 17:38:03.033215 7fe9bae887a0 20 read_log 192803'16013 (192803'16012) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:93 2014-08-11 12:41:36.506294 2014-08-13 17:38:03.033227 7fe9bae887a0 20 read_log 192803'16014 (192803'16013) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:94 2014-08-11 12:41:36.808809 2014-08-13 17:38:03.033239 7fe9bae887a0 20 read_log 192803'16015 (192803'16014) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:95 2014-08-11 12:41:36.835716 2014-08-13 17:38:03.033250 7fe9bae887a0 20 read_log 192803'16016 (192803'16015) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:96 2014-08-11 12:41:36.862845 2014-08-13 17:38:03.033262 7fe9bae887a0 20 read_log 192803'16017 (192803'16016) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:97 2014-08-11 12:41:36.891487 2014-08-13 17:38:03.033291 7fe9bae887a0 20 read_log 192803'16018 (192803'16017) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:98 2014-08-11 12:41:36.918330 2014-08-13 17:38:03.033304 7fe9bae887a0 20 read_log 192803'16019 (192803'16018) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:99 2014-08-11 12:41:36.942066 2014-08-13 17:38:03.033316 7fe9bae887a0 20 read_log 192803'16020 (192803'16019) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:100 2014-08-11 12:41:36.969858 2014-08-13 17:38:03.033328 7fe9bae887a0 20 read_log 192803'16021 (192803'16020) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:101 2014-08-11 12:41:36.997621 2014-08-13 17:38:03.033340 7fe9bae887a0 20 read_log 192803'16022 (192803'16021) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:102 2014-08-11 12:41:37.027554 2014-08-13 17:38:03.033351 7fe9bae887a0 20 read_log 192803'16023 (192803'16022) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:103 2014-08-11 12:41:37.054344 2014-08-13 17:38:03.033363 7fe9bae887a0 20 read_log 192803'16024 (192803'16023) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:104 2014-08-11 12:41:37.353887 2014-08-13 17:38:03.033384 7fe9bae887a0 20 read_log 192803'16025 (192803'16024) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:105 2014-08-11 12:41:37.384777 2014-08-13 17:38:03.033400 7fe9bae887a0 20 read_log 192803'16026 (192803'16025) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:106 2014-08-11 12:41:37.422262 2014-08-13 17:38:03.033412 7fe9bae887a0 20 read_log 192803'16027 (192803'16026) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:107 2014-08-11 12:41:37.474103 2014-08-13 17:38:03.033424 7fe9bae887a0 20 read_log 192803'16028 (192803'16027) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:108 2014-08-11 12:41:37.525652 2014-08-13 17:38:03.033436 7fe9bae887a0 20 read_log 192803'16029 (192803'16028) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:109 2014-08-11 12:41:37.569239 2014-08-13 17:38:03.033448 7fe9bae887a0 20 read_log 192803'16030 (192803'16029) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:110 2014-08-11 12:41:37.621145 2014-08-13 17:38:03.033459 7fe9bae887a0 20 read_log 192803'16031 (192803'16030) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:111 2014-08-11 12:41:37.747796 2014-08-13 17:38:03.033471 7fe9bae887a0 20 read_log 192803'16032 (192803'16031) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:112 2014-08-11 12:41:37.816216 2014-08-13 17:38:03.033483 7fe9bae887a0 20 read_log 192803'16033 (192803'16032) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:113 2014-08-11 12:41:37.859471 2014-08-13 17:38:03.033495 7fe9bae887a0 20 read_log 192803'16034 (192803'16033) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:114 2014-08-11 12:41:37.903006 2014-08-13 17:38:03.033507 7fe9bae887a0 20 read_log 192803'16035 (192803'16034) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:115 2014-08-11 12:41:37.946699 2014-08-13 17:38:03.033519 7fe9bae887a0 20 read_log 192803'16036 (192803'16035) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:116 2014-08-11 12:41:37.990123 2014-08-13 17:38:03.033531 7fe9bae887a0 20 read_log 192803'16037 (192803'16036) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:117 2014-08-11 12:41:38.033317 2014-08-13 17:38:03.033543 7fe9bae887a0 20 read_log 192803'16038 (192803'16037) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:118 2014-08-11 12:41:38.077117 2014-08-13 17:38:03.033555 7fe9bae887a0 20 read_log 192803'16039 (192803'16038) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:119 2014-08-11 12:41:38.128929 2014-08-13 17:38:03.033567 7fe9bae887a0 20 read_log 192803'16040 (192803'16039) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:120 2014-08-11 12:41:38.172172 2014-08-13 17:38:03.033579 7fe9bae887a0 20 read_log 192803'16041 (192803'16040) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:121 2014-08-11 12:41:38.214622 2014-08-13 17:38:03.033591 7fe9bae887a0 20 read_log 192803'16042 (192803'16041) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:122 2014-08-11 12:41:38.367462 2014-08-13 17:38:03.033603 7fe9bae887a0 20 read_log 192803'16043 (192803'16042) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:123 2014-08-11 12:41:38.410656 2014-08-13 17:38:03.033615 7fe9bae887a0 20 read_log 192803'16044 (192803'16043) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:124 2014-08-11 12:41:38.454365 2014-08-13 17:38:03.033627 7fe9bae887a0 20 read_log 192803'16045 (192803'16044) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:125 2014-08-11 12:41:38.497818 2014-08-13 17:38:03.033639 7fe9bae887a0 20 read_log 192803'16046 (192803'16045) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:126 2014-08-11 12:41:38.541902 2014-08-13 17:38:03.033650 7fe9bae887a0 20 read_log 192803'16047 (192803'16046) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:127 2014-08-11 12:41:38.585148 2014-08-13 17:38:03.033662 7fe9bae887a0 20 read_log 192803'16048 (192803'16047) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:128 2014-08-11 12:41:38.631007 2014-08-13 17:38:03.033703 7fe9bae887a0 20 read_log 192803'16049 (192803'16048) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:129 2014-08-11 12:41:38.674525 2014-08-13 17:38:03.033719 7fe9bae887a0 20 read_log 192803'16050 (192803'16049) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:130 2014-08-11 12:41:38.716831 2014-08-13 17:38:03.033731 7fe9bae887a0 20 read_log 192803'16051 (192803'16050) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:131 2014-08-11 12:41:38.753143 2014-08-13 17:38:03.033743 7fe9bae887a0 20 read_log 192803'16052 (192803'16051) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:132 2014-08-11 12:41:38.904926 2014-08-13 17:38:03.033757 7fe9bae887a0 20 read_log 192803'16053 (192803'16052) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:133 2014-08-11 12:41:38.948182 2014-08-13 17:38:03.033769 7fe9bae887a0 20 read_log 192803'16054 (192803'16053) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:134 2014-08-11 12:41:38.991799 2014-08-13 17:38:03.033781 7fe9bae887a0 20 read_log 192803'16055 (192803'16054) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:135 2014-08-11 12:41:39.035322 2014-08-13 17:38:03.033795 7fe9bae887a0 20 read_log 192803'16056 (192803'16055) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:136 2014-08-11 12:41:39.078760 2014-08-13 17:38:03.033808 7fe9bae887a0 20 read_log 192803'16057 (192803'16056) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:137 2014-08-11 12:41:39.122100 2014-08-13 17:38:03.033822 7fe9bae887a0 20 read_log 192803'16058 (192803'16057) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:138 2014-08-11 12:41:39.165436 2014-08-13 17:38:03.033834 7fe9bae887a0 20 read_log 192803'16059 (192803'16058) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:139 2014-08-11 12:41:39.206373 2014-08-13 17:38:03.033849 7fe9bae887a0 20 read_log 192803'16060 (192803'16059) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:140 2014-08-11 12:41:39.250283 2014-08-13 17:38:03.033861 7fe9bae887a0 20 read_log 192803'16061 (192803'16060) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:141 2014-08-11 12:41:39.293431 2014-08-13 17:38:03.033875 7fe9bae887a0 20 read_log 192803'16062 (192803'16061) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:142 2014-08-11 12:41:39.436755 2014-08-13 17:38:03.033887 7fe9bae887a0 20 read_log 192803'16063 (192803'16062) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:143 2014-08-11 12:41:39.482931 2014-08-13 17:38:03.033898 7fe9bae887a0 20 read_log 192803'16064 (192803'16063) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:144 2014-08-11 12:41:39.534794 2014-08-13 17:38:03.033910 7fe9bae887a0 20 read_log 192803'16065 (192803'16064) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:145 2014-08-11 12:41:39.578946 2014-08-13 17:38:03.033922 7fe9bae887a0 20 read_log 192803'16066 (192803'16065) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:146 2014-08-11 12:41:39.630158 2014-08-13 17:38:03.033937 7fe9bae887a0 20 read_log 192803'16067 (192803'16066) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:147 2014-08-11 12:41:39.673657 2014-08-13 17:38:03.033949 7fe9bae887a0 20 read_log 192803'16068 (192803'16067) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:148 2014-08-11 12:41:39.715631 2014-08-13 17:38:03.033961 7fe9bae887a0 20 read_log 192803'16069 (192803'16068) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:149 2014-08-11 12:41:39.758283 2014-08-13 17:38:03.033973 7fe9bae887a0 20 read_log 192803'16070 (192803'16069) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:150 2014-08-11 12:41:39.801822 2014-08-13 17:38:03.033987 7fe9bae887a0 20 read_log 192803'16071 (192803'16070) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:151 2014-08-11 12:41:39.845393 2014-08-13 17:38:03.033999 7fe9bae887a0 20 read_log 192803'16072 (192803'16071) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:152 2014-08-11 12:41:39.957475 2014-08-13 17:38:03.034018 7fe9bae887a0 20 read_log 192803'16073 (192803'16072) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:153 2014-08-11 12:41:40.000556 2014-08-13 17:38:03.034033 7fe9bae887a0 20 read_log 192803'16074 (192803'16073) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:154 2014-08-11 12:41:40.052829 2014-08-13 17:38:03.034045 7fe9bae887a0 20 read_log 192803'16075 (192803'16074) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:155 2014-08-11 12:41:40.095123 2014-08-13 17:38:03.034057 7fe9bae887a0 20 read_log 192803'16076 (192803'16075) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:156 2014-08-11 12:41:40.139972 2014-08-13 17:38:03.034071 7fe9bae887a0 20 read_log 192803'16077 (192803'16076) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:157 2014-08-11 12:41:40.182159 2014-08-13 17:38:03.034083 7fe9bae887a0 20 read_log 192803'16078 (192803'16077) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:158 2014-08-11 12:41:40.227043 2014-08-13 17:38:03.034095 7fe9bae887a0 20 read_log 192803'16079 (192803'16078) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:159 2014-08-11 12:41:40.272321 2014-08-13 17:38:03.034108 7fe9bae887a0 20 read_log 192803'16080 (192803'16079) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:160 2014-08-11 12:41:40.315953 2014-08-13 17:38:03.034121 7fe9bae887a0 20 read_log 192803'16081 (192803'16080) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:161 2014-08-11 12:41:40.359539 2014-08-13 17:38:03.034133 7fe9bae887a0 20 read_log 192803'16082 (192803'16081) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:162 2014-08-11 12:41:40.486508 2014-08-13 17:38:03.034145 7fe9bae887a0 20 read_log 192803'16083 (192803'16082) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:163 2014-08-11 12:41:40.530085 2014-08-13 17:38:03.034164 7fe9bae887a0 20 read_log 192803'16084 (192803'16083) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:164 2014-08-11 12:41:40.571413 2014-08-13 17:38:03.034179 7fe9bae887a0 20 read_log 192803'16085 (192803'16084) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:165 2014-08-11 12:41:40.618113 2014-08-13 17:38:03.034191 7fe9bae887a0 20 read_log 192803'16086 (192803'16085) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:166 2014-08-11 12:41:40.662109 2014-08-13 17:38:03.034203 7fe9bae887a0 20 read_log 192803'16087 (192803'16086) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:167 2014-08-11 12:41:40.713882 2014-08-13 17:38:03.034215 7fe9bae887a0 20 read_log 192803'16088 (192803'16087) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:168 2014-08-11 12:41:40.757398 2014-08-13 17:38:03.034229 7fe9bae887a0 20 read_log 192803'16089 (192803'16088) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:169 2014-08-11 12:41:40.801054 2014-08-13 17:38:03.034244 7fe9bae887a0 20 read_log 192803'16090 (192803'16089) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:170 2014-08-11 12:41:40.844869 2014-08-13 17:38:03.034256 7fe9bae887a0 20 read_log 192803'16091 (192803'16090) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:171 2014-08-11 12:41:40.905155 2014-08-13 17:38:03.034270 7fe9bae887a0 20 read_log 192803'16092 (192803'16091) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:172 2014-08-11 12:41:41.207003 2014-08-13 17:38:03.034282 7fe9bae887a0 20 read_log 192803'16093 (192803'16092) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:173 2014-08-11 12:41:41.259143 2014-08-13 17:38:03.034294 7fe9bae887a0 20 read_log 192803'16094 (192803'16093) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:174 2014-08-11 12:41:41.311301 2014-08-13 17:38:03.034308 7fe9bae887a0 20 read_log 192803'16095 (192803'16094) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:175 2014-08-11 12:41:41.356954 2014-08-13 17:38:03.034320 7fe9bae887a0 20 read_log 192803'16096 (192803'16095) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:176 2014-08-11 12:41:41.382747 2014-08-13 17:38:03.034341 7fe9bae887a0 20 read_log 192803'16097 (192803'16096) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:177 2014-08-11 12:41:41.409218 2014-08-13 17:38:03.034356 7fe9bae887a0 20 read_log 192803'16098 (192803'16097) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:178 2014-08-11 12:41:41.460532 2014-08-13 17:38:03.034368 7fe9bae887a0 20 read_log 192803'16099 (192803'16098) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:179 2014-08-11 12:41:41.504376 2014-08-13 17:38:03.034380 7fe9bae887a0 20 read_log 192803'16100 (192803'16099) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:180 2014-08-11 12:41:41.565073 2014-08-13 17:38:03.034392 7fe9bae887a0 20 read_log 192803'16101 (192803'16100) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:181 2014-08-11 12:41:41.608844 2014-08-13 17:38:03.034404 7fe9bae887a0 20 read_log 192803'16102 (192803'16101) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:182 2014-08-11 12:41:42.035600 2014-08-13 17:38:03.034416 7fe9bae887a0 20 read_log 192803'16103 (192803'16102) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:183 2014-08-11 12:41:42.083272 2014-08-13 17:38:03.034428 7fe9bae887a0 20 read_log 192803'16104 (192803'16103) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:184 2014-08-11 12:41:42.135249 2014-08-13 17:38:03.034439 7fe9bae887a0 20 read_log 192803'16105 (192803'16104) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:185 2014-08-11 12:41:42.187248 2014-08-13 17:38:03.034451 7fe9bae887a0 20 read_log 192803'16106 (192803'16105) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:186 2014-08-11 12:41:42.239144 2014-08-13 17:38:03.034463 7fe9bae887a0 20 read_log 192803'16107 (192803'16106) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:187 2014-08-11 12:41:42.291123 2014-08-13 17:38:03.034475 7fe9bae887a0 20 read_log 192803'16108 (192803'16107) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:188 2014-08-11 12:41:42.343167 2014-08-13 17:38:03.034487 7fe9bae887a0 20 read_log 192803'16109 (192803'16108) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:189 2014-08-11 12:41:42.395154 2014-08-13 17:38:03.034499 7fe9bae887a0 20 read_log 192803'16110 (192803'16109) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:190 2014-08-11 12:41:42.447125 2014-08-13 17:38:03.034530 7fe9bae887a0 20 read_log 192803'16111 (192803'16110) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:191 2014-08-11 12:41:42.499126 2014-08-13 17:38:03.034545 7fe9bae887a0 20 read_log 192803'16112 (192803'16111) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:192 2014-08-11 12:41:42.892760 2014-08-13 17:38:03.034557 7fe9bae887a0 20 read_log 192803'16113 (192803'16112) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:193 2014-08-11 12:41:43.020052 2014-08-13 17:38:03.034569 7fe9bae887a0 20 read_log 192803'16114 (192803'16113) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:194 2014-08-11 12:41:43.069794 2014-08-13 17:38:03.034581 7fe9bae887a0 20 read_log 192803'16115 (192803'16114) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:195 2014-08-11 12:41:43.107350 2014-08-13 17:38:03.034593 7fe9bae887a0 20 read_log 192803'16116 (192803'16115) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:196 2014-08-11 12:41:43.153469 2014-08-13 17:38:03.034605 7fe9bae887a0 20 read_log 192803'16117 (192803'16116) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:197 2014-08-11 12:41:43.197216 2014-08-13 17:38:03.034638 7fe9bae887a0 20 read_log 192803'16118 (192803'16117) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:198 2014-08-11 12:41:43.234679 2014-08-13 17:38:03.034650 7fe9bae887a0 20 read_log 192803'16119 (192803'16118) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:199 2014-08-11 12:41:43.260504 2014-08-13 17:38:03.034662 7fe9bae887a0 20 read_log 192803'16120 (192803'16119) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:200 2014-08-11 12:41:43.292357 2014-08-13 17:38:03.034681 7fe9bae887a0 20 read_log 192803'16121 (192803'16120) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:201 2014-08-11 12:41:43.333026 2014-08-13 17:38:03.034696 7fe9bae887a0 20 read_log 192803'16122 (192803'16121) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:202 2014-08-11 12:41:43.373258 2014-08-13 17:38:03.034708 7fe9bae887a0 20 read_log 192803'16123 (192803'16122) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:203 2014-08-11 12:41:43.416124 2014-08-13 17:38:03.034742 7fe9bae887a0 20 read_log 192803'16124 (192803'16123) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:204 2014-08-11 12:41:43.906477 2014-08-13 17:38:03.034755 7fe9bae887a0 20 read_log 192803'16125 (192803'16124) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:205 2014-08-11 12:41:43.947390 2014-08-13 17:38:03.034766 7fe9bae887a0 20 read_log 192803'16126 (192803'16125) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:206 2014-08-11 12:41:43.988949 2014-08-13 17:38:03.034778 7fe9bae887a0 20 read_log 192803'16127 (192803'16126) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:207 2014-08-11 12:41:44.030022 2014-08-13 17:38:03.034790 7fe9bae887a0 20 read_log 192803'16128 (192803'16127) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:208 2014-08-11 12:41:44.075665 2014-08-13 17:38:03.034802 7fe9bae887a0 20 read_log 192803'16129 (192803'16128) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:209 2014-08-11 12:41:44.111043 2014-08-13 17:38:03.034814 7fe9bae887a0 20 read_log 192803'16130 (192803'16129) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:210 2014-08-11 12:41:44.152130 2014-08-13 17:38:03.034825 7fe9bae887a0 20 read_log 192803'16131 (192803'16130) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:211 2014-08-11 12:41:44.192980 2014-08-13 17:38:03.034837 7fe9bae887a0 20 read_log 192803'16132 (192803'16131) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:212 2014-08-11 12:41:44.234677 2014-08-13 17:38:03.034849 7fe9bae887a0 20 read_log 192803'16133 (192803'16132) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:213 2014-08-11 12:41:44.281226 2014-08-13 17:38:03.034861 7fe9bae887a0 20 read_log 192803'16134 (192803'16133) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:214 2014-08-11 12:41:44.736089 2014-08-13 17:38:03.034872 7fe9bae887a0 20 read_log 192803'16135 (192803'16134) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:215 2014-08-11 12:41:44.782843 2014-08-13 17:38:03.034884 7fe9bae887a0 20 read_log 192803'16136 (192803'16135) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:216 2014-08-11 12:41:44.822866 2014-08-13 17:38:03.034896 7fe9bae887a0 20 read_log 192803'16137 (192803'16136) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:217 2014-08-11 12:41:44.863865 2014-08-13 17:38:03.034908 7fe9bae887a0 20 read_log 192803'16138 (192803'16137) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:218 2014-08-11 12:41:44.907758 2014-08-13 17:38:03.034920 7fe9bae887a0 20 read_log 192803'16139 (192803'16138) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:219 2014-08-11 12:41:44.946272 2014-08-13 17:38:03.034931 7fe9bae887a0 20 read_log 192803'16140 (192803'16139) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:220 2014-08-11 12:41:44.987150 2014-08-13 17:38:03.034943 7fe9bae887a0 20 read_log 192803'16141 (192803'16140) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:221 2014-08-11 12:41:45.028292 2014-08-13 17:38:03.034955 7fe9bae887a0 20 read_log 192803'16142 (192803'16141) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:222 2014-08-11 12:41:45.064875 2014-08-13 17:38:03.034967 7fe9bae887a0 20 read_log 192803'16143 (192803'16142) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:223 2014-08-11 12:41:45.111755 2014-08-13 17:38:03.034978 7fe9bae887a0 20 read_log 192803'16144 (192803'16143) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:224 2014-08-11 12:41:45.565107 2014-08-13 17:38:03.034997 7fe9bae887a0 20 read_log 192803'16145 (192803'16144) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:225 2014-08-11 12:41:45.614779 2014-08-13 17:38:03.035010 7fe9bae887a0 20 read_log 192803'16146 (192803'16145) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:226 2014-08-11 12:41:45.656016 2014-08-13 17:38:03.035022 7fe9bae887a0 20 read_log 192803'16147 (192803'16146) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:227 2014-08-11 12:41:45.698152 2014-08-13 17:38:03.035033 7fe9bae887a0 20 read_log 192803'16148 (192803'16147) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:228 2014-08-11 12:41:45.746862 2014-08-13 17:38:03.035045 7fe9bae887a0 20 read_log 192803'16149 (192803'16148) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:229 2014-08-11 12:41:45.781446 2014-08-13 17:38:03.035057 7fe9bae887a0 20 read_log 192803'16150 (192803'16149) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:230 2014-08-11 12:41:45.836212 2014-08-13 17:38:03.035068 7fe9bae887a0 20 read_log 192803'16151 (192803'16150) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:231 2014-08-11 12:41:45.865019 2014-08-13 17:38:03.035080 7fe9bae887a0 20 read_log 192803'16152 (192803'16151) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:232 2014-08-11 12:41:45.906990 2014-08-13 17:38:03.035092 7fe9bae887a0 20 read_log 192803'16153 (192803'16152) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:233 2014-08-11 12:41:45.956489 2014-08-13 17:38:03.035103 7fe9bae887a0 20 read_log 192803'16154 (192803'16153) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:234 2014-08-11 12:41:46.408153 2014-08-13 17:38:03.035115 7fe9bae887a0 20 read_log 192803'16155 (192803'16154) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:235 2014-08-11 12:41:46.448559 2014-08-13 17:38:03.035127 7fe9bae887a0 20 read_log 192803'16156 (192803'16155) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:236 2014-08-11 12:41:46.490895 2014-08-13 17:38:03.035138 7fe9bae887a0 20 read_log 192803'16157 (192803'16156) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:237 2014-08-11 12:41:46.532343 2014-08-13 17:38:03.035150 7fe9bae887a0 20 read_log 192803'16158 (192803'16157) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:238 2014-08-11 12:41:46.573066 2014-08-13 17:38:03.035174 7fe9bae887a0 20 read_log 192803'16159 (192803'16158) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:239 2014-08-11 12:41:46.614521 2014-08-13 17:38:03.035186 7fe9bae887a0 20 read_log 192803'16160 (192803'16159) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:240 2014-08-11 12:41:46.657857 2014-08-13 17:38:03.035198 7fe9bae887a0 20 read_log 192803'16161 (192803'16160) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:241 2014-08-11 12:41:46.699566 2014-08-13 17:38:03.035210 7fe9bae887a0 20 read_log 192803'16162 (192803'16161) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:242 2014-08-11 12:41:46.741318 2014-08-13 17:38:03.035221 7fe9bae887a0 20 read_log 192803'16163 (192803'16162) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:243 2014-08-11 12:41:46.785888 2014-08-13 17:38:03.035233 7fe9bae887a0 20 read_log 192803'16164 (192803'16163) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:244 2014-08-11 12:41:47.235111 2014-08-13 17:38:03.035245 7fe9bae887a0 20 read_log 192803'16165 (192803'16164) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:245 2014-08-11 12:41:47.278508 2014-08-13 17:38:03.035257 7fe9bae887a0 20 read_log 192803'16166 (192803'16165) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:246 2014-08-11 12:41:47.344741 2014-08-13 17:38:03.035268 7fe9bae887a0 20 read_log 192803'16167 (192803'16166) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:247 2014-08-11 12:41:47.371933 2014-08-13 17:38:03.035280 7fe9bae887a0 20 read_log 192803'16168 (192803'16167) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:248 2014-08-11 12:41:47.414230 2014-08-13 17:38:03.035298 7fe9bae887a0 20 read_log 192803'16169 (192803'16168) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:249 2014-08-11 12:41:47.444168 2014-08-13 17:38:03.035310 7fe9bae887a0 20 read_log 192803'16170 (192803'16169) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:250 2014-08-11 12:41:47.483294 2014-08-13 17:38:03.035322 7fe9bae887a0 20 read_log 192803'16171 (192803'16170) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:251 2014-08-11 12:41:47.528251 2014-08-13 17:38:03.035334 7fe9bae887a0 20 read_log 192803'16172 (192803'16171) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:252 2014-08-11 12:41:47.559492 2014-08-13 17:38:03.035346 7fe9bae887a0 20 read_log 192803'16173 (192803'16172) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:253 2014-08-11 12:41:47.607692 2014-08-13 17:38:03.035357 7fe9bae887a0 20 read_log 192803'16174 (192803'16173) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:254 2014-08-11 12:41:47.874637 2014-08-13 17:38:03.035369 7fe9bae887a0 20 read_log 192803'16175 (192803'16174) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:255 2014-08-11 12:41:47.902557 2014-08-13 17:38:03.035381 7fe9bae887a0 20 read_log 192803'16176 (192803'16175) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:256 2014-08-11 12:41:47.931902 2014-08-13 17:38:03.035392 7fe9bae887a0 20 read_log 192803'16177 (192803'16176) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:257 2014-08-11 12:41:47.958528 2014-08-13 17:38:03.035404 7fe9bae887a0 20 read_log 192803'16178 (192803'16177) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:258 2014-08-11 12:41:47.993759 2014-08-13 17:38:03.035416 7fe9bae887a0 20 read_log 192803'16179 (192803'16178) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:259 2014-08-11 12:41:48.085934 2014-08-13 17:38:03.035427 7fe9bae887a0 20 read_log 192803'16180 (192803'16179) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:260 2014-08-11 12:41:48.151118 2014-08-13 17:38:03.035439 7fe9bae887a0 20 read_log 192803'16181 (192803'16180) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:261 2014-08-11 12:41:48.209173 2014-08-13 17:38:03.035451 7fe9bae887a0 20 read_log 192803'16182 (192803'16181) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:262 2014-08-11 12:41:48.236125 2014-08-13 17:38:03.035462 7fe9bae887a0 20 read_log 192803'16183 (192803'16182) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:263 2014-08-11 12:41:48.271880 2014-08-13 17:38:03.035474 7fe9bae887a0 20 read_log 192803'16184 (192803'16183) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:264 2014-08-11 12:41:48.298021 2014-08-13 17:38:03.035485 7fe9bae887a0 20 read_log 192803'16185 (192803'16184) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:265 2014-08-11 12:41:48.324061 2014-08-13 17:38:03.035497 7fe9bae887a0 20 read_log 192803'16186 (192803'16185) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:266 2014-08-11 12:41:48.362339 2014-08-13 17:38:03.035509 7fe9bae887a0 20 read_log 192803'16187 (192803'16186) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:267 2014-08-11 12:41:48.387486 2014-08-13 17:38:03.035520 7fe9bae887a0 20 read_log 192803'16188 (192803'16187) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:268 2014-08-11 12:41:48.415772 2014-08-13 17:38:03.035532 7fe9bae887a0 20 read_log 192803'16189 (192803'16188) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:269 2014-08-11 12:41:48.440966 2014-08-13 17:38:03.035544 7fe9bae887a0 20 read_log 192803'16190 (192803'16189) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:270 2014-08-11 12:41:48.465818 2014-08-13 17:38:03.035555 7fe9bae887a0 20 read_log 192803'16191 (192803'16190) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:271 2014-08-11 12:41:48.739188 2014-08-13 17:38:03.035567 7fe9bae887a0 20 read_log 192803'16192 (192803'16191) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:272 2014-08-11 12:41:48.765576 2014-08-13 17:38:03.035587 7fe9bae887a0 20 read_log 192803'16193 (192803'16192) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:273 2014-08-11 12:41:48.801130 2014-08-13 17:38:03.035599 7fe9bae887a0 20 read_log 192803'16194 (192803'16193) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:274 2014-08-11 12:41:48.827409 2014-08-13 17:38:03.035611 7fe9bae887a0 20 read_log 192803'16195 (192803'16194) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:275 2014-08-11 12:41:48.853715 2014-08-13 17:38:03.035623 7fe9bae887a0 20 read_log 192803'16196 (192803'16195) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:276 2014-08-11 12:41:48.883573 2014-08-13 17:38:03.035635 7fe9bae887a0 20 read_log 192803'16197 (192803'16196) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:277 2014-08-11 12:41:48.909748 2014-08-13 17:38:03.035647 7fe9bae887a0 20 read_log 192803'16198 (192803'16197) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:278 2014-08-11 12:41:48.937869 2014-08-13 17:38:03.035658 7fe9bae887a0 20 read_log 192803'16199 (192803'16198) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:279 2014-08-11 12:41:48.964587 2014-08-13 17:38:03.035670 7fe9bae887a0 20 read_log 192803'16200 (192803'16199) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:280 2014-08-11 12:41:48.989247 2014-08-13 17:38:03.035682 7fe9bae887a0 20 read_log 192803'16201 (192803'16200) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:281 2014-08-11 12:41:49.252365 2014-08-13 17:38:03.035693 7fe9bae887a0 20 read_log 192803'16202 (192803'16201) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:282 2014-08-11 12:41:49.278484 2014-08-13 17:38:03.035705 7fe9bae887a0 20 read_log 192803'16203 (192803'16202) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:283 2014-08-11 12:41:49.311409 2014-08-13 17:38:03.035719 7fe9bae887a0 20 read_log 192803'16204 (192803'16203) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:284 2014-08-11 12:41:49.333092 2014-08-13 17:38:03.035731 7fe9bae887a0 20 read_log 192803'16205 (192803'16204) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:285 2014-08-11 12:41:49.354291 2014-08-13 17:38:03.035742 7fe9bae887a0 20 read_log 192803'16206 (192803'16205) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:286 2014-08-11 12:41:49.386862 2014-08-13 17:38:03.035754 7fe9bae887a0 20 read_log 192803'16207 (192803'16206) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:287 2014-08-11 12:41:49.412651 2014-08-13 17:38:03.035766 7fe9bae887a0 20 read_log 192803'16208 (192803'16207) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:288 2014-08-11 12:41:49.443169 2014-08-13 17:38:03.035777 7fe9bae887a0 20 read_log 192803'16209 (192803'16208) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:289 2014-08-11 12:41:49.469952 2014-08-13 17:38:03.035789 7fe9bae887a0 20 read_log 192803'16210 (192803'16209) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:290 2014-08-11 12:41:49.495540 2014-08-13 17:38:03.035801 7fe9bae887a0 20 read_log 192803'16211 (192803'16210) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:291 2014-08-11 12:41:49.765672 2014-08-13 17:38:03.035812 7fe9bae887a0 20 read_log 192803'16212 (192803'16211) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:292 2014-08-11 12:41:49.791810 2014-08-13 17:38:03.035824 7fe9bae887a0 20 read_log 192803'16213 (192803'16212) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:293 2014-08-11 12:41:49.819891 2014-08-13 17:38:03.035836 7fe9bae887a0 20 read_log 192803'16214 (192803'16213) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:294 2014-08-11 12:41:49.846513 2014-08-13 17:38:03.035847 7fe9bae887a0 20 read_log 192803'16215 (192803'16214) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:295 2014-08-11 12:41:49.874414 2014-08-13 17:38:03.035859 7fe9bae887a0 20 read_log 192803'16216 (192803'16215) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:296 2014-08-11 12:41:49.902413 2014-08-13 17:38:03.035893 7fe9bae887a0 20 read_log 192803'16217 (192803'16216) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:297 2014-08-11 12:41:49.929492 2014-08-13 17:38:03.035906 7fe9bae887a0 20 read_log 192803'16218 (192803'16217) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:298 2014-08-11 12:41:49.964434 2014-08-13 17:38:03.035918 7fe9bae887a0 20 read_log 192803'16219 (192803'16218) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:299 2014-08-11 12:41:49.990034 2014-08-13 17:38:03.035929 7fe9bae887a0 20 read_log 192803'16220 (192803'16219) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:300 2014-08-11 12:41:50.015121 2014-08-13 17:38:03.035942 7fe9bae887a0 20 read_log 192803'16221 (192803'16220) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:301 2014-08-11 12:41:50.287899 2014-08-13 17:38:03.035953 7fe9bae887a0 20 read_log 192803'16222 (192803'16221) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:302 2014-08-11 12:41:50.310825 2014-08-13 17:38:03.035965 7fe9bae887a0 20 read_log 192803'16223 (192803'16222) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:303 2014-08-11 12:41:50.341822 2014-08-13 17:38:03.035977 7fe9bae887a0 20 read_log 192803'16224 (192803'16223) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:304 2014-08-11 12:41:50.369068 2014-08-13 17:38:03.035988 7fe9bae887a0 20 read_log 192803'16225 (192803'16224) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:305 2014-08-11 12:41:50.395939 2014-08-13 17:38:03.036000 7fe9bae887a0 20 read_log 192803'16226 (192803'16225) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:306 2014-08-11 12:41:50.424776 2014-08-13 17:38:03.036012 7fe9bae887a0 20 read_log 192803'16227 (192803'16226) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:307 2014-08-11 12:41:50.451800 2014-08-13 17:38:03.036024 7fe9bae887a0 20 read_log 192803'16228 (192803'16227) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:308 2014-08-11 12:41:50.486703 2014-08-13 17:38:03.036035 7fe9bae887a0 20 read_log 192803'16229 (192803'16228) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:309 2014-08-11 12:41:50.514690 2014-08-13 17:38:03.036047 7fe9bae887a0 20 read_log 192803'16230 (192803'16229) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:310 2014-08-11 12:41:50.540233 2014-08-13 17:38:03.036059 7fe9bae887a0 20 read_log 192803'16231 (192803'16230) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:311 2014-08-11 12:41:50.819383 2014-08-13 17:38:03.036070 7fe9bae887a0 20 read_log 192803'16232 (192803'16231) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:312 2014-08-11 12:41:50.845963 2014-08-13 17:38:03.036082 7fe9bae887a0 20 read_log 192803'16233 (192803'16232) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:313 2014-08-11 12:41:50.893780 2014-08-13 17:38:03.036094 7fe9bae887a0 20 read_log 192803'16234 (192803'16233) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:314 2014-08-11 12:41:50.920124 2014-08-13 17:38:03.036105 7fe9bae887a0 20 read_log 192803'16235 (192803'16234) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:315 2014-08-11 12:41:50.947099 2014-08-13 17:38:03.036117 7fe9bae887a0 20 read_log 192803'16236 (192803'16235) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:316 2014-08-11 12:41:50.983102 2014-08-13 17:38:03.036128 7fe9bae887a0 20 read_log 192803'16321 (192803'16236) clean 38370f0b/core.31758+10+2/head//24 by osd.329.0:17951 2014-08-11 12:44:51.390607 2014-08-13 17:38:03.036140 7fe9bae887a0 20 read_log 192803'16322 (192803'16321) delete 38370f0b/core.31758+10+2/head//24 by osd.329.0:17952 2014-08-11 12:44:51.429624 2014-08-13 17:38:03.036152 7fe9bae887a0 20 read_log 192834'16738 (0'0) promote a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by osd.329.0:18377 2014-08-11 15:23:36.854654 2014-08-13 17:38:03.036170 7fe9bae887a0 20 read_log 192834'16739 (192834'16738) modify a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.933448.0:310 2014-08-11 15:23:36.845920 2014-08-13 17:38:03.036191 7fe9bae887a0 20 read_log 192834'16750 (0'0) promote e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by osd.329.0:18389 2014-08-11 15:23:40.006879 2014-08-13 17:38:03.036204 7fe9bae887a0 20 read_log 192834'16751 (192834'16750) modify e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.933448.0:1120 2014-08-11 15:23:39.999653 2014-08-13 17:38:03.036216 7fe9bae887a0 20 read_log 192834'16774 (0'0) promote 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by osd.329.0:18413 2014-08-11 15:23:47.193970 2014-08-13 17:38:03.036228 7fe9bae887a0 20 read_log 192834'16775 (192834'16774) modify 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.933448.0:2528 2014-08-11 15:23:47.183318 2014-08-13 17:38:03.036241 7fe9bae887a0 20 read_log 192834'16810 (0'0) promote a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18449 2014-08-11 15:24:02.922788 2014-08-13 17:38:03.036253 7fe9bae887a0 20 read_log 192834'16811 (192834'16810) modify a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.933448.0:5797 2014-08-11 15:24:02.914240 2014-08-13 17:38:03.036265 7fe9bae887a0 20 read_log 192834'16830 (0'0) promote a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by osd.329.0:18469 2014-08-11 15:24:06.485195 2014-08-13 17:38:03.036277 7fe9bae887a0 20 read_log 192834'16831 (192834'16830) modify a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.933448.0:6716 2014-08-11 15:24:06.477268 2014-08-13 17:38:03.036290 7fe9bae887a0 20 read_log 192834'16834 (0'0) promote 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by osd.329.0:18473 2014-08-11 15:24:07.574214 2014-08-13 17:38:03.036302 7fe9bae887a0 20 read_log 192834'16835 (192834'16834) modify 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.933448.0:6974 2014-08-11 15:24:07.565870 2014-08-13 17:38:03.036315 7fe9bae887a0 20 read_log 192834'16848 (0'0) promote e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by osd.329.0:18487 2014-08-11 15:24:11.032499 2014-08-13 17:38:03.036327 7fe9bae887a0 20 read_log 192834'16849 (192834'16848) modify e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.933448.0:7688 2014-08-11 15:24:11.024496 2014-08-13 17:38:03.036339 7fe9bae887a0 20 read_log 192834'16852 (0'0) promote e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by osd.329.0:18491 2014-08-11 15:24:11.193931 2014-08-13 17:38:03.036351 7fe9bae887a0 20 read_log 192834'16853 (192834'16852) modify e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.933448.0:7731 2014-08-11 15:24:11.185239 2014-08-13 17:38:03.036363 7fe9bae887a0 20 read_log 192834'16864 (0'0) promote 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by osd.329.0:18503 2014-08-11 15:24:14.751126 2014-08-13 17:38:03.036375 7fe9bae887a0 20 read_log 192834'16865 (192834'16864) modify 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.933448.0:8610 2014-08-11 15:24:14.742118 2014-08-13 17:38:03.036388 7fe9bae887a0 20 read_log 192834'16872 (0'0) promote 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by osd.329.0:18511 2014-08-11 15:24:15.247920 2014-08-13 17:38:03.036400 7fe9bae887a0 20 read_log 192834'16873 (192834'16872) modify 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.933448.0:8729 2014-08-11 15:24:15.191577 2014-08-13 17:38:03.036412 7fe9bae887a0 20 read_log 192834'16886 (0'0) promote 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by osd.329.0:18525 2014-08-11 15:24:19.256400 2014-08-13 17:38:03.036424 7fe9bae887a0 20 read_log 192834'16887 (192834'16886) modify 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.933448.0:9544 2014-08-11 15:24:19.247988 2014-08-13 17:38:03.036450 7fe9bae887a0 20 read_log 192834'16888 (0'0) promote 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by osd.329.0:18527 2014-08-11 15:24:19.357454 2014-08-13 17:38:03.036465 7fe9bae887a0 20 read_log 192834'16889 (192834'16888) modify 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.933448.0:9565 2014-08-11 15:24:19.348446 2014-08-13 17:38:03.036481 7fe9bae887a0 20 read_log 192834'16898 (0'0) promote e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by osd.329.0:18537 2014-08-11 15:24:23.503560 2014-08-13 17:38:03.036494 7fe9bae887a0 20 read_log 192834'16899 (192834'16898) modify e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.933448.0:10499 2014-08-11 15:24:23.496516 2014-08-13 17:38:03.036506 7fe9bae887a0 20 read_log 192834'16910 (0'0) promote c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by osd.329.0:18549 2014-08-11 15:24:26.019099 2014-08-13 17:38:03.036518 7fe9bae887a0 20 read_log 192834'16911 (192834'16910) modify c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.933448.0:10989 2014-08-11 15:24:26.012079 2014-08-13 17:38:03.036530 7fe9bae887a0 20 read_log 192834'16912 (0'0) promote cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by osd.329.0:18551 2014-08-11 15:24:26.971074 2014-08-13 17:38:03.036542 7fe9bae887a0 20 read_log 192834'16913 (192834'16912) modify cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.933448.0:11240 2014-08-11 15:24:26.962305 2014-08-13 17:38:03.036554 7fe9bae887a0 20 read_log 192834'16914 (0'0) promote ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by osd.329.0:18553 2014-08-11 15:24:27.378318 2014-08-13 17:38:03.036566 7fe9bae887a0 20 read_log 192834'16915 (192834'16914) modify ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.933448.0:11347 2014-08-11 15:24:27.370099 2014-08-13 17:38:03.036578 7fe9bae887a0 20 read_log 192834'16916 (0'0) promote e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by osd.329.0:18555 2014-08-11 15:24:27.468983 2014-08-13 17:38:03.036590 7fe9bae887a0 20 read_log 192834'16917 (192834'16916) modify e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.933448.0:11371 2014-08-11 15:24:27.459612 2014-08-13 17:38:03.036602 7fe9bae887a0 20 read_log 192834'16934 (0'0) promote 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by osd.329.0:18573 2014-08-11 15:24:31.289578 2014-08-13 17:38:03.036637 7fe9bae887a0 20 read_log 192834'16935 (192834'16934) modify 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.933448.0:12153 2014-08-11 15:24:31.281162 2014-08-13 17:38:03.036651 7fe9bae887a0 20 read_log 192834'16950 (0'0) promote 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by osd.329.0:18589 2014-08-11 15:24:36.211364 2014-08-13 17:38:03.036663 7fe9bae887a0 20 read_log 192834'16951 (192834'16950) modify 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.933448.0:13292 2014-08-11 15:24:36.204105 2014-08-13 17:38:03.036675 7fe9bae887a0 20 read_log 192834'16968 (0'0) promote 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by osd.329.0:18607 2014-08-11 15:24:46.266676 2014-08-13 17:38:03.036688 7fe9bae887a0 20 read_log 192834'16969 (192834'16968) modify 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.933448.0:14584 2014-08-11 15:24:46.257476 2014-08-13 17:38:03.036700 7fe9bae887a0 20 read_log 192834'17010 (0'0) promote 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by osd.329.0:18649 2014-08-11 15:24:57.166185 2014-08-13 17:38:03.036712 7fe9bae887a0 20 read_log 192834'17011 (192834'17010) modify 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.933448.0:16805 2014-08-11 15:24:57.158687 2014-08-13 17:38:03.036731 7fe9bae887a0 20 read_log 192834'17030 (0'0) promote f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by osd.329.0:18669 2014-08-11 15:25:01.699471 2014-08-13 17:38:03.036744 7fe9bae887a0 20 read_log 192834'17031 (192834'17030) modify f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.933448.0:17799 2014-08-11 15:25:01.690378 2014-08-13 17:38:03.036756 7fe9bae887a0 20 read_log 192834'17050 (0'0) promote 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by osd.329.0:18689 2014-08-11 15:25:08.435372 2014-08-13 17:38:03.036768 7fe9bae887a0 20 read_log 192834'17051 (192834'17050) modify 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.933448.0:19074 2014-08-11 15:25:08.426033 2014-08-13 17:38:03.036780 7fe9bae887a0 20 read_log 192834'17056 (0'0) promote 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by osd.329.0:18695 2014-08-11 15:25:09.173771 2014-08-13 17:38:03.036792 7fe9bae887a0 20 read_log 192834'17057 (192834'17056) modify 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.933448.0:19223 2014-08-11 15:25:09.161454 2014-08-13 17:38:03.036804 7fe9bae887a0 20 read_log 192834'17062 (0'0) promote c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by osd.329.0:18701 2014-08-11 15:25:11.704334 2014-08-13 17:38:03.036816 7fe9bae887a0 20 read_log 192834'17063 (192834'17062) modify c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.933448.0:19379 2014-08-11 15:25:11.696372 2014-08-13 17:38:03.036828 7fe9bae887a0 20 read_log 192834'17078 (0'0) promote 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by osd.329.0:18717 2014-08-11 15:25:19.671033 2014-08-13 17:38:03.036840 7fe9bae887a0 20 read_log 192834'17079 (192834'17078) modify 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.933448.0:20205 2014-08-11 15:25:19.661958 2014-08-13 17:38:03.036852 7fe9bae887a0 20 read_log 192834'17092 (0'0) promote a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by osd.329.0:18731 2014-08-11 15:25:23.135050 2014-08-13 17:38:03.036864 7fe9bae887a0 20 read_log 192834'17093 (192834'17092) modify a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.933448.0:20733 2014-08-11 15:25:23.125218 2014-08-13 17:38:03.036876 7fe9bae887a0 20 read_log 192834'17098 (0'0) promote 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by osd.329.0:18737 2014-08-11 15:25:25.682346 2014-08-13 17:38:03.036888 7fe9bae887a0 20 read_log 192834'17099 (192834'17098) modify 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.933448.0:21000 2014-08-11 15:25:25.674366 2014-08-13 17:38:03.036900 7fe9bae887a0 20 read_log 192834'17114 (0'0) promote 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by osd.329.0:18753 2014-08-11 15:25:32.149006 2014-08-13 17:38:03.036912 7fe9bae887a0 20 read_log 192834'17115 (192834'17114) modify 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.933448.0:21826 2014-08-11 15:25:32.137728 2014-08-13 17:38:03.036940 7fe9bae887a0 20 read_log 192834'17118 (0'0) promote d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by osd.329.0:18757 2014-08-11 15:25:32.659802 2014-08-13 17:38:03.036956 7fe9bae887a0 20 read_log 192834'17119 (192834'17118) modify d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.933448.0:21897 2014-08-11 15:25:32.600742 2014-08-13 17:38:03.036968 7fe9bae887a0 20 read_log 192834'17125 (0'0) promote d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by osd.329.0:18764 2014-08-11 15:25:32.850533 2014-08-13 17:38:03.036980 7fe9bae887a0 20 read_log 192834'17126 (192834'17125) modify d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.933448.0:21925 2014-08-11 15:25:32.809119 2014-08-13 17:38:03.037000 7fe9bae887a0 20 read_log 192834'17143 (192834'16811) clean a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18782 2014-08-11 15:25:33.176475 2014-08-13 17:38:03.037015 7fe9bae887a0 20 read_log 192834'17145 (192834'17143) delete a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18784 2014-08-11 15:25:33.238799 2014-08-13 17:38:03.037028 7fe9bae887a0 20 read_log 192834'17151 (192834'16739) modify a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.933448.0:22601 2014-08-11 15:25:38.274879 2014-08-13 17:38:03.037040 7fe9bae887a0 20 read_log 192834'17158 (192834'16751) modify e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.933448.0:23411 2014-08-11 15:25:39.323646 2014-08-13 17:38:03.037052 7fe9bae887a0 20 read_log 192834'17171 (192834'16775) modify 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.933448.0:24819 2014-08-11 15:25:41.379371 2014-08-13 17:38:03.037064 7fe9bae887a0 20 read_log 192834'17191 (0'0) promote a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18830 2014-08-11 15:25:45.494774 2014-08-13 17:38:03.037076 7fe9bae887a0 20 read_log 192834'17192 (192834'17191) modify a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.933448.0:28088 2014-08-11 15:25:45.470255 2014-08-13 17:38:03.037088 7fe9bae887a0 20 read_log 192834'17203 (192834'16831) modify a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.933448.0:29007 2014-08-11 15:25:46.928210 2014-08-13 17:38:03.037100 7fe9bae887a0 20 read_log 192834'17207 (192834'16835) modify 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.933448.0:29265 2014-08-11 15:25:47.166002 2014-08-13 17:38:03.037112 7fe9bae887a0 20 read_log 192834'17214 (192834'16849) modify e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.933448.0:29979 2014-08-11 15:25:48.213189 2014-08-13 17:38:03.037124 7fe9bae887a0 20 read_log 192834'17216 (192834'16853) modify e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.933448.0:30022 2014-08-11 15:25:48.265666 2014-08-13 17:38:03.037136 7fe9bae887a0 20 read_log 192834'17222 (192834'16865) modify 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.933448.0:30901 2014-08-11 15:25:49.372673 2014-08-13 17:38:03.037148 7fe9bae887a0 20 read_log 192834'17225 (192834'16873) modify 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.933448.0:31020 2014-08-11 15:25:49.499061 2014-08-13 17:38:03.037165 7fe9bae887a0 20 read_log 192834'17235 (192834'16887) modify 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.933448.0:31835 2014-08-11 15:25:50.678866 2014-08-13 17:38:03.037178 7fe9bae887a0 20 read_log 192834'17236 (192834'16889) modify 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.933448.0:31856 2014-08-11 15:25:50.697834 2014-08-13 17:38:03.037190 7fe9bae887a0 20 read_log 192834'17242 (192834'16899) modify e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.933448.0:32790 2014-08-11 15:25:52.372204 2014-08-13 17:38:03.037202 7fe9bae887a0 20 read_log 192834'17248 (192834'16911) modify c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.933448.0:33280 2014-08-11 15:25:53.190748 2014-08-13 17:38:03.037214 7fe9bae887a0 20 read_log 192834'17249 (192834'16913) modify cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.933448.0:33531 2014-08-11 15:25:53.640410 2014-08-13 17:38:03.037226 7fe9bae887a0 20 read_log 192834'17250 (192834'16915) modify ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.933448.0:33638 2014-08-11 15:25:53.864963 2014-08-13 17:38:03.037238 7fe9bae887a0 20 read_log 192834'17251 (192834'16917) modify e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.933448.0:33662 2014-08-11 15:25:53.927264 2014-08-13 17:38:03.037259 7fe9bae887a0 20 read_log 192834'17260 (192834'16935) modify 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.933448.0:34444 2014-08-11 15:25:55.019232 2014-08-13 17:38:03.037273 7fe9bae887a0 20 read_log 192834'17269 (192834'16951) modify 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.933448.0:35583 2014-08-11 15:25:56.555513 2014-08-13 17:38:03.037285 7fe9bae887a0 20 read_log 192834'17278 (192834'16969) modify 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.933448.0:36875 2014-08-11 15:25:58.424869 2014-08-13 17:38:03.037297 7fe9bae887a0 20 read_log 192834'17299 (192834'17011) modify 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.933448.0:39096 2014-08-11 15:26:01.539015 2014-08-13 17:38:03.037309 7fe9bae887a0 20 read_log 192834'17309 (192834'17031) modify f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.933448.0:40090 2014-08-11 15:26:02.309017 2014-08-13 17:38:03.037321 7fe9bae887a0 20 read_log 192834'17319 (192834'17051) modify 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.933448.0:41365 2014-08-11 15:26:02.921360 2014-08-13 17:38:03.037334 7fe9bae887a0 20 read_log 192834'17322 (192834'17057) modify 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.933448.0:41514 2014-08-11 15:26:02.967857 2014-08-13 17:38:03.037346 7fe9bae887a0 20 read_log 192834'17325 (192834'17063) modify c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.933448.0:41670 2014-08-11 15:26:03.016925 2014-08-13 17:38:03.037358 7fe9bae887a0 20 read_log 192834'17333 (192834'17079) modify 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.933448.0:42496 2014-08-11 15:26:03.298270 2014-08-13 17:38:03.037370 7fe9bae887a0 20 read_log 192834'17340 (192834'17093) modify a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.933448.0:43024 2014-08-11 15:26:03.476830 2014-08-13 17:38:03.037382 7fe9bae887a0 20 read_log 192834'17343 (192834'17099) modify 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.933448.0:43291 2014-08-11 15:26:03.563103 2014-08-13 17:38:03.037413 7fe9bae887a0 20 read_log 192834'17351 (192834'17115) modify 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.933448.0:44117 2014-08-11 15:26:03.821873 2014-08-13 17:38:03.037431 7fe9bae887a0 20 read_log 192834'17352 (192834'17119) modify d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.933448.0:44188 2014-08-11 15:26:03.843961 2014-08-13 17:38:03.037443 7fe9bae887a0 20 read_log 192834'17353 (192834'17126) modify d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.933448.0:44216 2014-08-11 15:26:03.853138 2014-08-13 17:38:03.037455 7fe9bae887a0 20 read_log 192834'17420 (0'0) promote 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by osd.329.0:19059 2014-08-11 15:26:34.808059 2014-08-13 17:38:03.037467 7fe9bae887a0 20 read_log 192834'17421 (192834'17420) modify 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.1883634.0:5551 2014-08-11 15:26:34.798486 2014-08-13 17:38:03.037480 7fe9bae887a0 20 read_log 192834'17432 (0'0) promote 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by osd.329.0:19071 2014-08-11 15:26:38.601634 2014-08-13 17:38:03.037492 7fe9bae887a0 20 read_log 192834'17433 (192834'17432) modify 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.1883634.0:6526 2014-08-11 15:26:38.593968 2014-08-13 17:38:03.037504 7fe9bae887a0 20 read_log 192834'17442 (0'0) promote 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by osd.329.0:19081 2014-08-11 15:26:40.853410 2014-08-13 17:38:03.037516 7fe9bae887a0 20 read_log 192834'17443 (192834'17442) modify 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.1883634.0:7111 2014-08-11 15:26:40.846711 2014-08-13 17:38:03.037535 7fe9bae887a0 20 read_log 192834'17478 (0'0) promote 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by osd.329.0:19117 2014-08-11 15:26:51.616483 2014-08-13 17:38:03.037549 7fe9bae887a0 20 read_log 192834'17479 (192834'17478) modify 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.1883634.0:9626 2014-08-11 15:26:51.608460 2014-08-13 17:38:03.037563 7fe9bae887a0 20 read_log 192834'17486 (0'0) promote b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by osd.329.0:19125 2014-08-11 15:26:53.788811 2014-08-13 17:38:03.037575 7fe9bae887a0 20 read_log 192834'17487 (192834'17486) modify b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.1883634.0:10127 2014-08-11 15:26:53.781163 2014-08-13 17:38:03.037587 7fe9bae887a0 20 read_log 192834'17500 (0'0) promote 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by osd.329.0:19139 2014-08-11 15:26:56.158768 2014-08-13 17:38:03.037599 7fe9bae887a0 20 read_log 192834'17501 (192834'17500) modify 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.1883634.0:10677 2014-08-11 15:26:56.149613 2014-08-13 17:38:03.037611 7fe9bae887a0 20 read_log 192834'17518 (0'0) promote 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by osd.329.0:19157 2014-08-11 15:27:01.241826 2014-08-13 17:38:03.037623 7fe9bae887a0 20 read_log 192834'17519 (192834'17518) modify 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.1883634.0:11774 2014-08-11 15:27:01.232511 2014-08-13 17:38:03.037636 7fe9bae887a0 20 read_log 192834'17520 (0'0) promote 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by osd.329.0:19159 2014-08-11 15:27:01.965895 2014-08-13 17:38:03.037648 7fe9bae887a0 20 read_log 192834'17521 (192834'17520) modify 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.1883634.0:11980 2014-08-11 15:27:01.955482 2014-08-13 17:38:03.037660 7fe9bae887a0 20 read_log 192834'17522 (0'0) promote 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by osd.329.0:19161 2014-08-11 15:27:02.052446 2014-08-13 17:38:03.037674 7fe9bae887a0 20 read_log 192834'17523 (192834'17522) modify 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.1883634.0:12007 2014-08-11 15:27:02.042572 2014-08-13 17:38:03.037687 7fe9bae887a0 20 read_log 192834'17528 (0'0) promote 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by osd.329.0:19167 2014-08-11 15:27:03.554839 2014-08-13 17:38:03.037699 7fe9bae887a0 20 read_log 192834'17529 (192834'17528) modify 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.1883634.0:12326 2014-08-11 15:27:03.545441 2014-08-13 17:38:03.037711 7fe9bae887a0 20 read_log 192834'17556 (0'0) promote 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by osd.329.0:19195 2014-08-11 15:27:11.260717 2014-08-13 17:38:03.037723 7fe9bae887a0 20 read_log 192834'17557 (192834'17556) modify 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.1883634.0:13930 2014-08-11 15:27:11.251856 2014-08-13 17:38:03.037735 7fe9bae887a0 20 read_log 192834'17558 (0'0) promote d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by osd.329.0:19197 2014-08-11 15:27:11.592111 2014-08-13 17:38:03.037747 7fe9bae887a0 20 read_log 192834'17559 (192834'17558) modify d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.1883634.0:14021 2014-08-11 15:27:11.583622 2014-08-13 17:38:03.037759 7fe9bae887a0 20 read_log 192834'17582 (0'0) promote c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by osd.329.0:19221 2014-08-11 15:27:29.694185 2014-08-13 17:38:03.037771 7fe9bae887a0 20 read_log 192834'17583 (192834'17582) modify c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.1883634.0:16861 2014-08-11 15:27:29.684112 2014-08-13 17:38:03.037805 7fe9bae887a0 20 read_log 192834'17618 (0'0) promote ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by osd.329.0:19257 2014-08-11 15:27:52.382622 2014-08-13 17:38:03.037821 7fe9bae887a0 20 read_log 192834'17619 (192834'17618) modify ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.1883634.0:19421 2014-08-11 15:27:52.351160 2014-08-13 17:38:03.037834 7fe9bae887a0 20 read_log 192834'17630 (0'0) promote 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by osd.329.0:19269 2014-08-11 15:27:55.163671 2014-08-13 17:38:03.037846 7fe9bae887a0 20 read_log 192834'17631 (192834'17630) modify 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.1883634.0:20059 2014-08-11 15:27:55.077912 2014-08-13 17:38:03.037858 7fe9bae887a0 20 read_log 192834'17646 (0'0) promote da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by osd.329.0:19285 2014-08-11 15:28:02.987115 2014-08-13 17:38:03.037870 7fe9bae887a0 20 read_log 192834'17647 (192834'17646) modify da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.1883634.0:21137 2014-08-11 15:28:02.979341 2014-08-13 17:38:03.037882 7fe9bae887a0 20 read_log 192834'17678 (0'0) promote f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by osd.329.0:19317 2014-08-11 15:28:11.026872 2014-08-13 17:38:03.037894 7fe9bae887a0 20 read_log 192834'17679 (192834'17678) modify f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.1883634.0:22778 2014-08-11 15:28:11.017984 2014-08-13 17:38:03.037907 7fe9bae887a0 20 read_log 192834'17716 (192834'17421) modify 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.1883634.0:28875 2014-08-11 15:28:26.429111 2014-08-13 17:38:03.037919 7fe9bae887a0 20 read_log 192834'17722 (192834'17433) modify 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.1883634.0:29850 2014-08-11 15:28:27.759373 2014-08-13 17:38:03.037931 7fe9bae887a0 20 read_log 192834'17727 (192834'17443) modify 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.1883634.0:30435 2014-08-11 15:28:28.634154 2014-08-13 17:38:03.037943 7fe9bae887a0 20 read_log 192834'17747 (192834'17479) modify 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.1883634.0:32950 2014-08-11 15:28:32.571817 2014-08-13 17:38:03.037955 7fe9bae887a0 20 read_log 192834'17751 (192834'17487) modify b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.1883634.0:33451 2014-08-11 15:28:33.225636 2014-08-13 17:38:03.037967 7fe9bae887a0 20 read_log 192834'17758 (192834'17501) modify 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.1883634.0:34001 2014-08-11 15:28:34.110871 2014-08-13 17:38:03.037979 7fe9bae887a0 20 read_log 192834'17767 (192834'17519) modify 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.1883634.0:35098 2014-08-11 15:28:36.021059 2014-08-13 17:38:03.037991 7fe9bae887a0 20 read_log 192834'17768 (192834'17521) modify 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.1883634.0:35304 2014-08-11 15:28:36.413465 2014-08-13 17:38:03.038003 7fe9bae887a0 20 read_log 192834'17769 (192834'17523) modify 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.1883634.0:35331 2014-08-11 15:28:36.468214 2014-08-13 17:38:03.038015 7fe9bae887a0 20 read_log 192834'17772 (192834'17529) modify 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.1883634.0:35650 2014-08-11 15:28:36.956826 2014-08-13 17:38:03.038027 7fe9bae887a0 20 read_log 192834'17786 (192834'17557) modify 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.1883634.0:37254 2014-08-11 15:28:39.067259 2014-08-13 17:38:03.038039 7fe9bae887a0 20 read_log 192834'17787 (192834'17559) modify d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.1883634.0:37345 2014-08-11 15:28:39.199626 2014-08-13 17:38:03.038061 7fe9bae887a0 20 read_log 192834'17799 (192834'17583) modify c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.1883634.0:40185 2014-08-11 15:28:43.320185 2014-08-13 17:38:03.038074 7fe9bae887a0 20 read_log 192834'17817 (192834'17619) modify ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.1883634.0:42745 2014-08-11 15:28:46.341762 2014-08-13 17:38:03.038086 7fe9bae887a0 20 read_log 192834'17823 (192834'17631) modify 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.1883634.0:43383 2014-08-11 15:28:46.536921 2014-08-13 17:38:03.038098 7fe9bae887a0 20 read_log 192834'17831 (192834'17647) modify da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.1883634.0:44461 2014-08-11 15:28:46.864706 2014-08-13 17:38:03.038110 7fe9bae887a0 20 read_log 192834'17847 (192834'17679) modify f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.1883634.0:46102 2014-08-11 15:28:47.491579 2014-08-13 17:38:03.038122 7fe9bae887a0 20 read_log 192888'17851 (0'0) promote aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by osd.320.0:1678 2014-08-11 16:44:55.449708 2014-08-13 17:38:03.038134 7fe9bae887a0 20 read_log 192888'17852 (192888'17851) modify aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.1315259.0:124 2014-08-11 16:44:55.423845 2014-08-13 17:38:03.038147 7fe9bae887a0 20 read_log 192888'17855 (0'0) promote 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by osd.320.0:1682 2014-08-11 16:45:01.181701 2014-08-13 17:38:03.038164 7fe9bae887a0 20 read_log 192888'17856 (192888'17855) modify 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.1315259.0:1542 2014-08-11 16:45:01.174411 2014-08-13 17:38:03.038198 7fe9bae887a0 20 read_log 192888'17859 (0'0) promote 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by osd.320.0:1686 2014-08-11 16:45:09.102840 2014-08-13 17:38:03.038212 7fe9bae887a0 20 read_log 192888'17860 (192888'17859) modify 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.1315259.0:3633 2014-08-11 16:45:09.095033 2014-08-13 17:38:03.038225 7fe9bae887a0 20 read_log 192888'17861 (0'0) promote a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by osd.320.0:1688 2014-08-11 16:45:09.221953 2014-08-13 17:38:03.038237 7fe9bae887a0 20 read_log 192888'17862 (192888'17861) modify a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.1315259.0:3665 2014-08-11 16:45:09.215188 2014-08-13 17:38:03.038249 7fe9bae887a0 20 read_log 192888'17867 (0'0) promote 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by osd.320.0:1694 2014-08-11 16:45:29.478499 2014-08-13 17:38:03.038261 7fe9bae887a0 20 read_log 192888'17868 (192888'17867) modify 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.1315259.0:9170 2014-08-11 16:45:29.471167 2014-08-13 17:38:03.038274 7fe9bae887a0 20 read_log 192888'17869 (0'0) promote 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by osd.320.0:1696 2014-08-11 16:45:30.303956 2014-08-13 17:38:03.038286 7fe9bae887a0 20 read_log 192888'17870 (192888'17869) modify 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.1315259.0:9392 2014-08-11 16:45:30.295664 2014-08-13 17:38:03.038298 7fe9bae887a0 20 read_log 192888'17873 (0'0) promote 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by osd.320.0:1700 2014-08-11 16:45:37.094397 2014-08-13 17:38:03.038310 7fe9bae887a0 20 read_log 192888'17874 (192888'17873) modify 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.1315259.0:11118 2014-08-11 16:45:37.087169 2014-08-13 17:38:03.038329 7fe9bae887a0 20 read_log 192888'17875 (0'0) promote 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by osd.320.0:1702 2014-08-11 16:45:41.126604 2014-08-13 17:38:03.038342 7fe9bae887a0 20 read_log 192888'17876 (192888'17875) modify 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.1315259.0:12113 2014-08-11 16:45:41.119217 2014-08-13 17:38:03.038354 7fe9bae887a0 20 read_log 192888'17879 (0'0) promote e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by osd.320.0:1706 2014-08-11 16:45:43.565827 2014-08-13 17:38:03.038366 7fe9bae887a0 20 read_log 192888'17880 (192888'17879) modify e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.1315259.0:12704 2014-08-11 16:45:43.558567 2014-08-13 17:38:03.038378 7fe9bae887a0 20 read_log 192888'17881 (0'0) promote 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by osd.320.0:1708 2014-08-11 16:45:48.949857 2014-08-13 17:38:03.038390 7fe9bae887a0 20 read_log 192888'17882 (192888'17881) modify 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.1315259.0:14050 2014-08-11 16:45:48.941918 2014-08-13 17:38:03.038403 7fe9bae887a0 20 read_log 192888'17891 (0'0) promote 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by osd.320.0:1718 2014-08-11 16:45:53.579130 2014-08-13 17:38:03.038414 7fe9bae887a0 20 read_log 192888'17892 (192888'17891) modify 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.1315259.0:15265 2014-08-11 16:45:53.572460 2014-08-13 17:38:03.038426 7fe9bae887a0 20 read_log 192888'17893 (0'0) promote 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by osd.320.0:1720 2014-08-11 16:45:56.433203 2014-08-13 17:38:03.038438 7fe9bae887a0 20 read_log 192888'17894 (192888'17893) modify 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.1315259.0:16001 2014-08-11 16:45:56.425966 2014-08-13 17:38:03.038450 7fe9bae887a0 20 read_log 192888'17899 (0'0) promote d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by osd.320.0:1726 2014-08-11 16:46:08.843061 2014-08-13 17:38:03.038462 7fe9bae887a0 20 read_log 192888'17900 (192888'17899) modify d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.1315259.0:19131 2014-08-11 16:46:08.835789 2014-08-13 17:38:03.038474 7fe9bae887a0 20 read_log 192888'17905 (0'0) promote 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by osd.320.0:1732 2014-08-11 16:46:15.016123 2014-08-13 17:38:03.038486 7fe9bae887a0 20 read_log 192888'17906 (192888'17905) modify 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.1315259.0:20766 2014-08-11 16:46:15.009616 2014-08-13 17:38:03.038498 7fe9bae887a0 20 read_log 192888'17913 (0'0) promote a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by osd.320.0:1740 2014-08-11 16:46:21.689391 2014-08-13 17:38:03.038510 7fe9bae887a0 20 read_log 192888'17914 (192888'17913) modify a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.1315259.0:22505 2014-08-11 16:46:21.683175 2014-08-13 17:38:03.038522 7fe9bae887a0 20 read_log 192888'17917 (0'0) promote 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by osd.320.0:1744 2014-08-11 16:46:28.339556 2014-08-13 17:38:03.038534 7fe9bae887a0 20 read_log 192888'17918 (192888'17917) modify 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.1315259.0:24186 2014-08-11 16:46:28.332139 2014-08-13 17:38:03.038546 7fe9bae887a0 20 read_log 192888'17919 (0'0) promote 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by osd.320.0:1746 2014-08-11 16:46:29.046989 2014-08-13 17:38:03.038558 7fe9bae887a0 20 read_log 192888'17920 (192888'17919) modify 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.1315259.0:24376 2014-08-11 16:46:29.040191 2014-08-13 17:38:03.038577 7fe9bae887a0 20 read_log 192888'17923 (0'0) promote be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by osd.320.0:1750 2014-08-11 16:46:30.616541 2014-08-13 17:38:03.038590 7fe9bae887a0 20 read_log 192888'17924 (192888'17923) modify be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.1315259.0:24760 2014-08-11 16:46:30.608690 2014-08-13 17:38:03.038602 7fe9bae887a0 20 read_log 192888'17927 (0'0) promote 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by osd.320.0:1754 2014-08-11 16:46:32.907449 2014-08-13 17:38:03.038614 7fe9bae887a0 20 read_log 192888'17928 (192888'17927) modify 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.1315259.0:25345 2014-08-11 16:46:32.899952 2014-08-13 17:38:03.038626 7fe9bae887a0 20 read_log 192888'17935 (0'0) promote 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by osd.320.0:1762 2014-08-11 16:46:48.354518 2014-08-13 17:38:03.038638 7fe9bae887a0 20 read_log 192888'17936 (192888'17935) modify 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.1315259.0:29088 2014-08-11 16:46:48.348348 2014-08-13 17:38:03.038654 7fe9bae887a0 20 read_log 192888'17937 (0'0) promote 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by osd.320.0:1764 2014-08-11 16:46:51.216313 2014-08-13 17:38:03.038666 7fe9bae887a0 20 read_log 192888'17938 (192888'17937) modify 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.1315259.0:29763 2014-08-11 16:46:51.209232 2014-08-13 17:38:03.038679 7fe9bae887a0 20 read_log 192888'17941 (192888'17852) modify aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.1315259.0:30851 2014-08-11 16:46:55.912452 2014-08-13 17:38:03.038691 7fe9bae887a0 20 read_log 192888'17943 (192888'17856) modify 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.1315259.0:32269 2014-08-11 16:46:56.332539 2014-08-13 17:38:03.038703 7fe9bae887a0 20 read_log 192888'17945 (192888'17860) modify 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.1315259.0:34360 2014-08-11 16:46:56.873094 2014-08-13 17:38:03.038735 7fe9bae887a0 20 read_log 192888'17946 (192888'17862) modify a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.1315259.0:34392 2014-08-11 16:46:56.881271 2014-08-13 17:38:03.038747 7fe9bae887a0 20 read_log 192888'17949 (192888'17868) modify 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.1315259.0:39897 2014-08-11 16:46:58.332416 2014-08-13 17:38:03.038759 7fe9bae887a0 20 read_log 192888'17950 (192888'17870) modify 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.1315259.0:40119 2014-08-11 16:46:58.394569 2014-08-13 17:38:03.038772 7fe9bae887a0 20 read_log 192888'17952 (192888'17874) modify 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.1315259.0:41845 2014-08-11 16:46:58.853254 2014-08-13 17:38:03.038784 7fe9bae887a0 20 read_log 192888'17953 (192888'17876) modify 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.1315259.0:42840 2014-08-11 16:46:59.119755 2014-08-13 17:38:03.038796 7fe9bae887a0 20 read_log 192888'17955 (192888'17880) modify e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.1315259.0:43431 2014-08-11 16:46:59.289030 2014-08-13 17:38:03.038808 7fe9bae887a0 20 read_log 192888'17956 (192888'17882) modify 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.1315259.0:44777 2014-08-11 16:46:59.673721 2014-08-13 17:38:03.038821 7fe9bae887a0 20 read_log 192888'17961 (192888'17892) modify 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.1315259.0:45992 2014-08-11 16:47:00.029303 2014-08-13 17:38:03.038833 7fe9bae887a0 20 read_log 192888'17962 (192888'17894) modify 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.1315259.0:46728 2014-08-11 16:47:00.242494 2014-08-13 17:38:03.038854 7fe9bae887a0 20 read_log 192888'17965 (192888'17900) modify d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.1315259.0:49858 2014-08-11 16:47:01.194041 2014-08-13 17:38:03.038867 7fe9bae887a0 20 read_log 192888'17968 (192888'17906) modify 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.1315259.0:51493 2014-08-11 16:47:01.697000 2014-08-13 17:38:03.038879 7fe9bae887a0 20 read_log 192888'17972 (192888'17914) modify a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.1315259.0:53232 2014-08-11 16:47:02.193519 2014-08-13 17:38:03.038891 7fe9bae887a0 20 read_log 192888'17974 (192888'17918) modify 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.1315259.0:54913 2014-08-11 16:47:02.754549 2014-08-13 17:38:03.038903 7fe9bae887a0 20 read_log 192888'17975 (192888'17920) modify 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.1315259.0:55103 2014-08-11 16:47:02.815369 2014-08-13 17:38:03.038915 7fe9bae887a0 20 read_log 192888'17977 (192888'17924) modify be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.1315259.0:55487 2014-08-11 16:47:02.926134 2014-08-13 17:38:03.038927 7fe9bae887a0 20 read_log 192888'17979 (192888'17928) modify 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.1315259.0:56072 2014-08-11 16:47:03.095984 2014-08-13 17:38:03.038939 7fe9bae887a0 20 read_log 192888'17983 (192888'17936) modify 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.1315259.0:59815 2014-08-11 16:47:04.248647 2014-08-13 17:38:03.038951 7fe9bae887a0 20 read_log 192888'17984 (192888'17938) modify 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.1315259.0:60490 2014-08-11 16:47:04.458596 2014-08-13 17:38:03.038963 7fe9bae887a0 20 read_log 192888'17990 (0'0) promote c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by osd.320.0:1816 2014-08-11 17:05:20.723171 2014-08-13 17:38:03.038975 7fe9bae887a0 20 read_log 192888'17991 (192888'17990) modify c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.942868.0:1395 2014-08-11 17:05:20.716813 2014-08-13 17:38:03.038987 7fe9bae887a0 20 read_log 192888'17992 (0'0) promote 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by osd.320.0:1818 2014-08-11 17:05:21.935641 2014-08-13 17:38:03.038999 7fe9bae887a0 20 read_log 192888'17993 (192888'17992) modify 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.942868.0:1674 2014-08-11 17:05:21.928062 2014-08-13 17:38:03.039012 7fe9bae887a0 20 read_log 192888'17994 (0'0) promote e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by osd.320.0:1820 2014-08-11 17:05:22.927420 2014-08-13 17:38:03.039024 7fe9bae887a0 20 read_log 192888'17995 (192888'17994) modify e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.942868.0:1929 2014-08-11 17:05:22.920726 2014-08-13 17:38:03.039036 7fe9bae887a0 20 read_log 192888'18000 (0'0) promote 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by osd.320.0:1826 2014-08-11 17:05:30.773990 2014-08-13 17:38:03.039048 7fe9bae887a0 20 read_log 192888'18001 (192888'18000) modify 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.942868.0:4010 2014-08-11 17:05:30.766403 2014-08-13 17:38:03.039060 7fe9bae887a0 20 read_log 192888'18004 (0'0) promote 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by osd.320.0:1830 2014-08-11 17:05:32.997773 2014-08-13 17:38:03.039072 7fe9bae887a0 20 read_log 192888'18005 (192888'18004) modify 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.942868.0:4613 2014-08-11 17:05:32.992543 2014-08-13 17:38:03.039084 7fe9bae887a0 20 read_log 192888'18006 (0'0) promote b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by osd.320.0:1832 2014-08-11 17:05:33.708532 2014-08-13 17:38:03.039103 7fe9bae887a0 20 read_log 192888'18007 (192888'18006) modify b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.942868.0:4788 2014-08-11 17:05:33.702051 2014-08-13 17:38:03.039116 7fe9bae887a0 20 read_log 192888'18010 (0'0) promote c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by osd.320.0:1836 2014-08-11 17:05:37.842812 2014-08-13 17:38:03.039128 7fe9bae887a0 20 read_log 192888'18011 (192888'18010) modify c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.942868.0:5868 2014-08-11 17:05:37.834852 2014-08-13 17:38:03.039140 7fe9bae887a0 20 read_log 192888'18020 (0'0) promote 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by osd.320.0:1846 2014-08-11 17:05:42.674396 2014-08-13 17:38:03.039152 7fe9bae887a0 20 read_log 192888'18021 (192888'18020) modify 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.942868.0:7175 2014-08-11 17:05:42.667282 2014-08-13 17:38:03.039169 7fe9bae887a0 20 read_log 192888'18024 (0'0) promote 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by osd.320.0:1850 2014-08-11 17:05:46.348391 2014-08-13 17:38:03.039181 7fe9bae887a0 20 read_log 192888'18025 (192888'18024) modify 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.942868.0:8145 2014-08-11 17:05:46.341757 2014-08-13 17:38:03.039194 7fe9bae887a0 20 read_log 192888'18026 (0'0) promote 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by osd.320.0:1852 2014-08-11 17:05:46.447805 2014-08-13 17:38:03.039205 7fe9bae887a0 20 read_log 192888'18027 (192888'18026) modify 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.942868.0:8173 2014-08-11 17:05:46.442253 2014-08-13 17:38:03.039217 7fe9bae887a0 20 read_log 192888'18028 (0'0) promote 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by osd.320.0:1854 2014-08-11 17:05:48.596413 2014-08-13 17:38:03.039229 7fe9bae887a0 20 read_log 192888'18029 (192888'18028) modify 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.942868.0:8747 2014-08-11 17:05:48.588740 2014-08-13 17:38:03.039241 7fe9bae887a0 20 read_log 192888'18034 (0'0) promote 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by osd.320.0:1860 2014-08-11 17:06:01.605270 2014-08-13 17:38:03.039253 7fe9bae887a0 20 read_log 192888'18035 (192888'18034) modify 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.942868.0:12307 2014-08-11 17:06:01.596439 2014-08-13 17:38:03.039266 7fe9bae887a0 20 read_log 192888'18038 (0'0) promote d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by osd.320.0:1864 2014-08-11 17:06:03.748518 2014-08-13 17:38:03.039277 7fe9bae887a0 20 read_log 192888'18039 (192888'18038) modify d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.942868.0:12878 2014-08-11 17:06:03.741729 2014-08-13 17:38:03.039290 7fe9bae887a0 20 read_log 192888'18040 (0'0) promote e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by osd.320.0:1866 2014-08-11 17:06:05.796056 2014-08-13 17:38:03.039302 7fe9bae887a0 20 read_log 192888'18041 (192888'18040) modify e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.942868.0:13406 2014-08-11 17:06:05.786856 2014-08-13 17:38:03.039314 7fe9bae887a0 20 read_log 192888'18042 (0'0) promote f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by osd.320.0:1868 2014-08-11 17:06:05.933077 2014-08-13 17:38:03.039326 7fe9bae887a0 20 read_log 192888'18043 (192888'18042) modify f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.942868.0:13441 2014-08-11 17:06:05.924705 2014-08-13 17:38:03.039338 7fe9bae887a0 20 read_log 192888'18044 (0'0) promote 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by osd.320.0:1870 2014-08-11 17:06:07.730122 2014-08-13 17:38:03.039357 7fe9bae887a0 20 read_log 192888'18045 (192888'18044) modify 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.942868.0:13892 2014-08-11 17:06:07.722188 2014-08-13 17:38:03.039370 7fe9bae887a0 20 read_log 192888'18048 (0'0) promote 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by osd.320.0:1874 2014-08-11 17:06:11.109526 2014-08-13 17:38:03.039382 7fe9bae887a0 20 read_log 192888'18049 (192888'18048) modify 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.942868.0:14714 2014-08-11 17:06:11.102039 2014-08-13 17:38:03.039411 7fe9bae887a0 20 read_log 192888'18056 (0'0) promote d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by osd.320.0:1882 2014-08-11 17:06:19.059635 2014-08-13 17:38:03.039427 7fe9bae887a0 20 read_log 192888'18057 (192888'18056) modify d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.942868.0:16748 2014-08-11 17:06:19.053461 2014-08-13 17:38:03.039440 7fe9bae887a0 20 read_log 192888'18058 (0'0) promote 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by osd.320.0:1884 2014-08-11 17:06:24.908608 2014-08-13 17:38:03.039453 7fe9bae887a0 20 read_log 192888'18059 (192888'18058) modify 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.942868.0:18284 2014-08-11 17:06:24.902377 2014-08-13 17:38:03.039465 7fe9bae887a0 20 read_log 192888'18060 (0'0) promote 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by osd.320.0:1886 2014-08-11 17:06:25.024103 2014-08-13 17:38:03.039477 7fe9bae887a0 20 read_log 192888'18061 (192888'18060) modify 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.942868.0:18316 2014-08-11 17:06:25.018265 2014-08-13 17:38:03.039494 7fe9bae887a0 20 read_log 192888'18066 (0'0) promote 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by osd.320.0:1892 2014-08-11 17:06:32.424671 2014-08-13 17:38:03.039506 7fe9bae887a0 20 read_log 192888'18067 (192888'18066) modify 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.942868.0:20285 2014-08-11 17:06:32.417397 2014-08-13 17:38:03.039518 7fe9bae887a0 20 read_log 192888'18068 (0'0) promote 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by osd.320.0:1894 2014-08-11 17:06:34.902789 2014-08-13 17:38:03.039530 7fe9bae887a0 20 read_log 192888'18069 (192888'18068) modify 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.942868.0:20923 2014-08-11 17:06:34.895805 2014-08-13 17:38:03.039543 7fe9bae887a0 20 read_log 192888'18074 (0'0) promote e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by osd.320.0:1900 2014-08-11 17:06:35.540149 2014-08-13 17:38:03.039555 7fe9bae887a0 20 read_log 192888'18075 (192888'18074) modify e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.942868.0:21092 2014-08-11 17:06:35.533818 2014-08-13 17:38:03.039567 7fe9bae887a0 20 read_log 192888'18076 (0'0) promote 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by osd.320.0:1902 2014-08-11 17:06:36.417365 2014-08-13 17:38:03.039580 7fe9bae887a0 20 read_log 192888'18077 (192888'18076) modify 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.942868.0:21312 2014-08-11 17:06:36.407055 2014-08-13 17:38:03.039592 7fe9bae887a0 20 read_log 192888'18078 (0'0) promote 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by osd.320.0:1904 2014-08-11 17:06:36.553513 2014-08-13 17:38:03.039604 7fe9bae887a0 20 read_log 192888'18079 (192888'18078) modify 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.942868.0:21349 2014-08-11 17:06:36.545151 2014-08-13 17:38:03.039616 7fe9bae887a0 20 read_log 192888'18086 (0'0) promote 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by osd.320.0:1912 2014-08-11 17:06:40.972929 2014-08-13 17:38:03.039638 7fe9bae887a0 20 read_log 192888'18087 (192888'18086) modify 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.942868.0:22494 2014-08-11 17:06:40.966218 2014-08-13 17:38:03.039654 7fe9bae887a0 20 read_log 192888'18090 (0'0) promote cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by osd.320.0:1916 2014-08-11 17:06:47.746107 2014-08-13 17:38:03.039666 7fe9bae887a0 20 read_log 192888'18091 (192888'18090) modify cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.942868.0:24281 2014-08-11 17:06:47.739203 2014-08-13 17:38:03.039678 7fe9bae887a0 20 read_log 192888'18100 (0'0) promote d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by osd.320.0:1926 2014-08-11 17:07:06.656943 2014-08-13 17:38:03.039691 7fe9bae887a0 20 read_log 192888'18101 (192888'18100) modify d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.942868.0:29221 2014-08-11 17:07:06.649734 2014-08-13 17:38:03.039703 7fe9bae887a0 20 read_log 192888'18102 (0'0) promote 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by osd.320.0:1928 2014-08-11 17:07:06.685316 2014-08-13 17:38:03.039715 7fe9bae887a0 20 read_log 192888'18103 (192888'18102) modify 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.942868.0:29229 2014-08-11 17:07:06.679430 2014-08-13 17:38:03.039727 7fe9bae887a0 20 read_log 192888'18109 (192888'17991) modify c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.942868.0:32661 2014-08-11 17:07:16.313140 2014-08-13 17:38:03.039739 7fe9bae887a0 20 read_log 192888'18110 (192888'17993) modify 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.942868.0:32940 2014-08-11 17:07:16.382794 2014-08-13 17:38:03.039751 7fe9bae887a0 20 read_log 192888'18111 (192888'17995) modify e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.942868.0:33195 2014-08-11 17:07:16.451339 2014-08-13 17:38:03.039763 7fe9bae887a0 20 read_log 192888'18114 (192888'18001) modify 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.942868.0:35276 2014-08-11 17:07:17.032357 2014-08-13 17:38:03.039775 7fe9bae887a0 20 read_log 192888'18116 (192888'18005) modify 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.942868.0:35879 2014-08-11 17:07:17.187537 2014-08-13 17:38:03.039788 7fe9bae887a0 20 read_log 192888'18117 (192888'18007) modify b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.942868.0:36054 2014-08-11 17:07:17.230212 2014-08-13 17:38:03.039800 7fe9bae887a0 20 read_log 192888'18119 (192888'18011) modify c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.942868.0:37134 2014-08-11 17:07:17.525020 2014-08-13 17:38:03.039812 7fe9bae887a0 20 read_log 192888'18124 (192888'18021) modify 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.942868.0:38441 2014-08-11 17:07:17.887755 2014-08-13 17:38:03.039824 7fe9bae887a0 20 read_log 192888'18126 (192888'18025) modify 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.942868.0:39411 2014-08-11 17:07:18.175535 2014-08-13 17:38:03.039836 7fe9bae887a0 20 read_log 192888'18127 (192888'18027) modify 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.942868.0:39439 2014-08-11 17:07:18.181667 2014-08-13 17:38:03.039848 7fe9bae887a0 20 read_log 192888'18128 (192888'18029) modify 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.942868.0:40013 2014-08-11 17:07:18.342138 2014-08-13 17:38:03.039860 7fe9bae887a0 20 read_log 192888'18131 (192888'18035) modify 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.942868.0:43573 2014-08-11 17:07:19.310715 2014-08-13 17:38:03.039872 7fe9bae887a0 20 read_log 192888'18133 (192888'18039) modify d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.942868.0:44144 2014-08-11 17:07:19.483064 2014-08-13 17:38:03.039908 7fe9bae887a0 20 read_log 192888'18134 (192888'18041) modify e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.942868.0:44672 2014-08-11 17:07:19.632009 2014-08-13 17:38:03.039925 7fe9bae887a0 20 read_log 192888'18135 (192888'18043) modify f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.942868.0:44707 2014-08-11 17:07:19.642635 2014-08-13 17:38:03.039937 7fe9bae887a0 20 read_log 192888'18136 (192888'18045) modify 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.942868.0:45158 2014-08-11 17:07:19.761472 2014-08-13 17:38:03.039949 7fe9bae887a0 20 read_log 192888'18138 (192888'18049) modify 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.942868.0:45980 2014-08-11 17:07:19.987064 2014-08-13 17:38:03.039961 7fe9bae887a0 20 read_log 192888'18142 (192888'18057) modify d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.942868.0:48014 2014-08-11 17:07:20.595215 2014-08-13 17:38:03.039974 7fe9bae887a0 20 read_log 192888'18143 (192888'18059) modify 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.942868.0:49550 2014-08-11 17:07:21.047019 2014-08-13 17:38:03.039988 7fe9bae887a0 20 read_log 192888'18144 (192888'18061) modify 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.942868.0:49582 2014-08-11 17:07:21.056415 2014-08-13 17:38:03.040000 7fe9bae887a0 20 read_log 192888'18147 (192888'18067) modify 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.942868.0:51551 2014-08-11 17:07:21.678656 2014-08-13 17:38:03.040012 7fe9bae887a0 20 read_log 192888'18148 (192888'18069) modify 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.942868.0:52189 2014-08-11 17:07:21.865138 2014-08-13 17:38:03.040024 7fe9bae887a0 20 read_log 192888'18151 (192888'18075) modify e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.942868.0:52358 2014-08-11 17:07:21.913564 2014-08-13 17:38:03.040038 7fe9bae887a0 20 read_log 192888'18152 (192888'18077) modify 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.942868.0:52578 2014-08-11 17:07:21.985962 2014-08-13 17:38:03.040051 7fe9bae887a0 20 read_log 192888'18153 (192888'18079) modify 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.942868.0:52615 2014-08-11 17:07:21.998310 2014-08-13 17:38:03.040063 7fe9bae887a0 20 read_log 192888'18157 (192888'18087) modify 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.942868.0:53760 2014-08-11 17:07:22.366063 2014-08-13 17:38:03.040080 7fe9bae887a0 20 read_log 192888'18159 (192888'18091) modify cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.942868.0:55547 2014-08-11 17:07:22.881690 2014-08-13 17:38:03.040092 7fe9bae887a0 20 read_log 192888'18164 (192888'18101) modify d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.942868.0:60487 2014-08-11 17:07:24.355004 2014-08-13 17:38:03.040104 7fe9bae887a0 20 read_log 192888'18165 (192888'18103) modify 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.942868.0:60495 2014-08-11 17:07:24.356795 2014-08-13 17:38:03.040117 7fe9bae887a0 20 read_log 193305'18168 (193305'18168) modify 30b/hit_set_24.30b_archive_2014-08-11 18:38:11.461328_2014-08-11 18:42:40.507251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.507471 2014-08-13 17:38:03.040130 7fe9bae887a0 20 read_log 193305'18169 (0'0) promote ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by osd.4.0:26 2014-08-11 18:42:40.511218 2014-08-13 17:38:03.040142 7fe9bae887a0 20 read_log 193305'18170 (193305'18169) modify ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.951337.0:1930 2014-08-11 18:42:40.501871 2014-08-13 17:38:03.040167 7fe9bae887a0 20 read_log 193305'18171 (0'0) promote df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by osd.4.0:131 2014-08-11 18:43:02.858078 2014-08-13 17:38:03.040183 7fe9bae887a0 20 read_log 193305'18172 (193305'18171) modify df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.951337.0:8051 2014-08-11 18:43:02.850377 2014-08-13 17:38:03.040196 7fe9bae887a0 20 read_log 193305'18173 (0'0) promote d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by osd.4.0:133 2014-08-11 18:43:03.282446 2014-08-13 17:38:03.040208 7fe9bae887a0 20 read_log 193305'18174 (193305'18173) modify d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.951337.0:8167 2014-08-11 18:43:03.274902 2014-08-13 17:38:03.040220 7fe9bae887a0 20 read_log 193305'18175 (0'0) promote 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by osd.4.0:135 2014-08-11 18:43:04.196778 2014-08-13 17:38:03.040232 7fe9bae887a0 20 read_log 193305'18176 (193305'18175) modify 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.951337.0:8376 2014-08-11 18:43:04.189559 2014-08-13 17:38:03.040244 7fe9bae887a0 20 read_log 193305'18177 (0'0) promote a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by osd.4.0:153 2014-08-11 18:43:07.654184 2014-08-13 17:38:03.040257 7fe9bae887a0 20 read_log 193305'18178 (193305'18177) modify a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.951337.0:9282 2014-08-11 18:43:07.646399 2014-08-13 17:38:03.040269 7fe9bae887a0 20 read_log 193305'18179 (0'0) promote 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by osd.4.0:159 2014-08-11 18:43:08.745876 2014-08-13 17:38:03.040281 7fe9bae887a0 20 read_log 193305'18180 (193305'18179) modify 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.951337.0:9576 2014-08-11 18:43:08.738356 2014-08-13 17:38:03.040295 7fe9bae887a0 20 read_log 193305'18181 (0'0) promote 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by osd.4.0:161 2014-08-11 18:43:10.184413 2014-08-13 17:38:03.040307 7fe9bae887a0 20 read_log 193305'18182 (193305'18181) modify 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.951337.0:9970 2014-08-11 18:43:10.176036 2014-08-13 17:38:03.040320 7fe9bae887a0 20 read_log 193305'18183 (0'0) promote 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by osd.4.0:199 2014-08-11 18:43:17.937465 2014-08-13 17:38:03.040332 7fe9bae887a0 20 read_log 193305'18184 (193305'18183) modify 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.951337.0:12012 2014-08-11 18:43:17.929806 2014-08-13 17:38:03.040344 7fe9bae887a0 20 read_log 193305'18185 (0'0) promote 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by osd.4.0:201 2014-08-11 18:43:18.746955 2014-08-13 17:38:03.040356 7fe9bae887a0 20 read_log 193305'18186 (193305'18185) modify 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.951337.0:12234 2014-08-11 18:43:18.738991 2014-08-13 17:38:03.040368 7fe9bae887a0 20 read_log 193305'18187 (0'0) promote 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by osd.4.0:203 2014-08-11 18:43:19.101383 2014-08-13 17:38:03.040380 7fe9bae887a0 20 read_log 193305'18188 (193305'18187) modify 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.951337.0:12331 2014-08-11 18:43:19.093497 2014-08-13 17:38:03.040393 7fe9bae887a0 20 read_log 193305'18189 (0'0) promote 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by osd.4.0:205 2014-08-11 18:43:19.201922 2014-08-13 17:38:03.040405 7fe9bae887a0 20 read_log 193305'18190 (193305'18189) modify 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.951337.0:12359 2014-08-11 18:43:19.194880 2014-08-13 17:38:03.040427 7fe9bae887a0 20 read_log 193305'18191 (0'0) promote a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by osd.4.0:213 2014-08-11 18:43:21.980066 2014-08-13 17:38:03.040440 7fe9bae887a0 20 read_log 193305'18192 (193305'18191) modify a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.951337.0:13096 2014-08-11 18:43:21.972908 2014-08-13 17:38:03.040452 7fe9bae887a0 20 read_log 193305'18193 (0'0) promote 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by osd.4.0:221 2014-08-11 18:43:23.969774 2014-08-13 17:38:03.040464 7fe9bae887a0 20 read_log 193305'18194 (193305'18193) modify 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.951337.0:13608 2014-08-11 18:43:23.962729 2014-08-13 17:38:03.040476 7fe9bae887a0 20 read_log 193305'18195 (0'0) promote a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by osd.4.0:227 2014-08-11 18:43:25.589217 2014-08-13 17:38:03.040488 7fe9bae887a0 20 read_log 193305'18196 (193305'18195) modify a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.951337.0:14062 2014-08-11 18:43:25.581935 2014-08-13 17:38:03.040500 7fe9bae887a0 20 read_log 193305'18197 (0'0) promote 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by osd.4.0:241 2014-08-11 18:43:29.468077 2014-08-13 17:38:03.040512 7fe9bae887a0 20 read_log 193305'18198 (193305'18197) modify 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.951337.0:15148 2014-08-11 18:43:29.458935 2014-08-13 17:38:03.040525 7fe9bae887a0 20 read_log 193305'18199 (0'0) promote 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by osd.4.0:301 2014-08-11 18:43:39.164371 2014-08-13 17:38:03.040537 7fe9bae887a0 20 read_log 193305'18200 (193305'18199) modify 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.951337.0:17933 2014-08-11 18:43:39.156527 2014-08-13 17:38:03.040549 7fe9bae887a0 20 read_log 193305'18201 (0'0) promote 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by osd.4.0:303 2014-08-11 18:43:39.396421 2014-08-13 17:38:03.040561 7fe9bae887a0 20 read_log 193305'18202 (193305'18201) modify 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.951337.0:18005 2014-08-11 18:43:39.388848 2014-08-13 17:38:03.040573 7fe9bae887a0 20 read_log 193305'18203 (0'0) promote 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by osd.4.0:311 2014-08-11 18:43:41.376248 2014-08-13 17:38:03.040585 7fe9bae887a0 20 read_log 193305'18204 (193305'18203) modify 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.951337.0:18572 2014-08-11 18:43:41.366986 2014-08-13 17:38:03.040597 7fe9bae887a0 20 read_log 193305'18205 (0'0) promote e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by osd.4.0:319 2014-08-11 18:43:43.114866 2014-08-13 17:38:03.040609 7fe9bae887a0 20 read_log 193305'18206 (193305'18205) modify e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.951337.0:19040 2014-08-11 18:43:43.104278 2014-08-13 17:38:03.040621 7fe9bae887a0 20 read_log 193305'18207 (0'0) promote c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by osd.4.0:339 2014-08-11 18:43:47.517692 2014-08-13 17:38:03.040652 7fe9bae887a0 20 read_log 193305'18208 (193305'18207) modify c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.951337.0:20231 2014-08-11 18:43:47.506883 2014-08-13 17:38:03.040668 7fe9bae887a0 20 read_log 193305'18209 (0'0) promote bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by osd.4.0:343 2014-08-11 18:43:48.572450 2014-08-13 17:38:03.040680 7fe9bae887a0 20 read_log 193305'18210 (193305'18209) modify bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.951337.0:20536 2014-08-11 18:43:48.563743 2014-08-13 17:38:03.040702 7fe9bae887a0 20 read_log 193305'18211 (0'0) promote 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by osd.4.0:379 2014-08-11 18:43:55.932728 2014-08-13 17:38:03.040718 7fe9bae887a0 20 read_log 193305'18212 (193305'18211) modify 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.951337.0:22577 2014-08-11 18:43:55.921927 2014-08-13 17:38:03.040730 7fe9bae887a0 20 read_log 193305'18213 (0'0) promote 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by osd.4.0:383 2014-08-11 18:43:56.434302 2014-08-13 17:38:03.040742 7fe9bae887a0 20 read_log 193305'18214 (193305'18213) modify 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.951337.0:22715 2014-08-11 18:43:56.427330 2014-08-13 17:38:03.040754 7fe9bae887a0 20 read_log 193305'18215 (0'0) promote 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by osd.4.0:397 2014-08-11 18:44:01.323805 2014-08-13 17:38:03.040766 7fe9bae887a0 20 read_log 193305'18216 (193305'18215) modify 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.951337.0:24076 2014-08-11 18:44:01.314588 2014-08-13 17:38:03.040780 7fe9bae887a0 20 read_log 193305'18217 (0'0) promote 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by osd.4.0:399 2014-08-11 18:44:01.399462 2014-08-13 17:38:03.040792 7fe9bae887a0 20 read_log 193305'18218 (193305'18217) modify 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.951337.0:24096 2014-08-11 18:44:01.387652 2014-08-13 17:38:03.040804 7fe9bae887a0 20 read_log 193305'18219 (0'0) promote 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by osd.4.0:419 2014-08-11 18:44:05.926555 2014-08-13 17:38:03.040838 7fe9bae887a0 20 read_log 193305'18220 (193305'18219) modify 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.951337.0:25353 2014-08-11 18:44:05.918574 2014-08-13 17:38:03.040851 7fe9bae887a0 20 read_log 193305'18221 (0'0) promote 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by osd.4.0:429 2014-08-11 18:44:08.825183 2014-08-13 17:38:03.040863 7fe9bae887a0 20 read_log 193305'18222 (193305'18221) modify 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.951337.0:26114 2014-08-11 18:44:08.817000 2014-08-13 17:38:03.040875 7fe9bae887a0 20 read_log 193305'18223 (0'0) promote 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by osd.4.0:453 2014-08-11 18:44:12.581712 2014-08-13 17:38:03.040887 7fe9bae887a0 20 read_log 193305'18224 (193305'18223) modify 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.951337.0:27142 2014-08-11 18:44:12.573845 2014-08-13 17:38:03.040899 7fe9bae887a0 20 read_log 193305'18225 (0'0) promote 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by osd.4.0:455 2014-08-11 18:44:12.781967 2014-08-13 17:38:03.040911 7fe9bae887a0 20 read_log 193305'18226 (193305'18225) modify 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.951337.0:27196 2014-08-11 18:44:12.773927 2014-08-13 17:38:03.040923 7fe9bae887a0 20 read_log 193305'18227 (0'0) promote 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by osd.4.0:469 2014-08-11 18:44:17.277740 2014-08-13 17:38:03.040935 7fe9bae887a0 20 read_log 193305'18228 (193305'18227) modify 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.951337.0:28445 2014-08-11 18:44:17.270412 2014-08-13 17:38:03.040947 7fe9bae887a0 20 read_log 193305'18229 (0'0) promote 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by osd.4.0:473 2014-08-11 18:44:18.198134 2014-08-13 17:38:03.040960 7fe9bae887a0 20 read_log 193305'18230 (193305'18229) modify 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.951337.0:28702 2014-08-11 18:44:18.190143 2014-08-13 17:38:03.040979 7fe9bae887a0 20 read_log 193305'18231 (0'0) promote bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by osd.4.0:495 2014-08-11 18:44:23.232654 2014-08-13 17:38:03.040992 7fe9bae887a0 20 read_log 193305'18232 (193305'18231) modify bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.951337.0:30065 2014-08-11 18:44:23.223808 2014-08-13 17:38:03.041004 7fe9bae887a0 20 read_log 193305'18233 (0'0) promote d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by osd.4.0:511 2014-08-11 18:44:28.704864 2014-08-13 17:38:03.041016 7fe9bae887a0 20 read_log 193305'18234 (193305'18233) modify d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.951337.0:31541 2014-08-11 18:44:28.697550 2014-08-13 17:38:03.041028 7fe9bae887a0 20 read_log 193305'18235 (0'0) promote d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by osd.4.0:529 2014-08-11 18:44:30.561501 2014-08-13 17:38:03.041039 7fe9bae887a0 20 read_log 193305'18236 (193305'18235) modify d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.951337.0:32085 2014-08-11 18:44:30.551483 2014-08-13 17:38:03.041051 7fe9bae887a0 20 read_log 193305'18237 (0'0) promote b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by osd.4.0:541 2014-08-11 18:44:31.441198 2014-08-13 17:38:03.041063 7fe9bae887a0 20 read_log 193305'18238 (193305'18237) modify b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.951337.0:32316 2014-08-11 18:44:31.431448 2014-08-13 17:38:03.041075 7fe9bae887a0 20 read_log 193305'18239 (0'0) promote 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by osd.4.0:543 2014-08-11 18:44:31.594003 2014-08-13 17:38:03.041088 7fe9bae887a0 20 read_log 193305'18240 (193305'18239) modify 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.951337.0:32357 2014-08-11 18:44:31.584382 2014-08-13 17:38:03.041100 7fe9bae887a0 20 read_log 193305'18241 (193305'18170) modify ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.951337.0:34641 2014-08-11 18:44:35.109648 2014-08-13 17:38:03.041112 7fe9bae887a0 20 read_log 193305'18242 (193305'18172) modify df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.951337.0:40762 2014-08-11 18:44:38.519031 2014-08-13 17:38:03.041126 7fe9bae887a0 20 read_log 193305'18243 (193305'18174) modify d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.951337.0:40878 2014-08-11 18:44:38.564422 2014-08-13 17:38:03.041138 7fe9bae887a0 20 read_log 193305'18244 (193305'18176) modify 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.951337.0:41087 2014-08-11 18:44:38.630794 2014-08-13 17:38:03.041150 7fe9bae887a0 20 read_log 193305'18245 (193305'18178) modify a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.951337.0:41993 2014-08-11 18:44:38.973406 2014-08-13 17:38:03.041168 7fe9bae887a0 20 read_log 193305'18246 (193305'18180) modify 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.951337.0:42287 2014-08-11 18:44:39.175560 2014-08-13 17:38:03.041180 7fe9bae887a0 20 read_log 193305'18247 (193305'18182) modify 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.951337.0:42681 2014-08-11 18:44:39.345429 2014-08-13 17:38:03.041193 7fe9bae887a0 20 read_log 193305'18248 (193305'18184) modify 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.951337.0:44723 2014-08-11 18:44:39.970216 2014-08-13 17:38:03.041208 7fe9bae887a0 20 read_log 193305'18249 (193305'18186) modify 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.951337.0:44945 2014-08-11 18:44:40.044497 2014-08-13 17:38:03.041220 7fe9bae887a0 20 read_log 193305'18250 (193305'18188) modify 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.951337.0:45042 2014-08-11 18:44:40.075987 2014-08-13 17:38:03.041244 7fe9bae887a0 20 read_log 193305'18251 (193305'18190) modify 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.951337.0:45070 2014-08-11 18:44:40.082308 2014-08-13 17:38:03.041259 7fe9bae887a0 20 read_log 193305'18252 (193305'18192) modify a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.951337.0:45807 2014-08-11 18:44:40.308289 2014-08-13 17:38:03.041272 7fe9bae887a0 20 read_log 193305'18253 (193305'18194) modify 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.951337.0:46319 2014-08-11 18:44:40.451735 2014-08-13 17:38:03.041286 7fe9bae887a0 20 read_log 193305'18254 (193305'18196) modify a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.951337.0:46773 2014-08-11 18:44:40.584809 2014-08-13 17:38:03.041299 7fe9bae887a0 20 read_log 193305'18255 (193305'18198) modify 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.951337.0:47859 2014-08-11 18:44:40.886571 2014-08-13 17:38:03.041311 7fe9bae887a0 20 read_log 193305'18256 (193305'18200) modify 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.951337.0:50644 2014-08-11 18:44:41.724179 2014-08-13 17:38:03.041323 7fe9bae887a0 20 read_log 193305'18257 (193305'18202) modify 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.951337.0:50716 2014-08-11 18:44:41.743185 2014-08-13 17:38:03.041335 7fe9bae887a0 20 read_log 193305'18258 (193305'18204) modify 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.951337.0:51283 2014-08-11 18:44:41.905944 2014-08-13 17:38:03.041347 7fe9bae887a0 20 read_log 193305'18259 (193305'18206) modify e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.951337.0:51751 2014-08-11 18:44:42.040053 2014-08-13 17:38:03.041359 7fe9bae887a0 20 read_log 193305'18260 (193305'18208) modify c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.951337.0:52942 2014-08-11 18:44:42.386029 2014-08-13 17:38:03.041371 7fe9bae887a0 20 read_log 193305'18261 (193305'18210) modify bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.951337.0:53247 2014-08-11 18:44:42.473956 2014-08-13 17:38:03.041383 7fe9bae887a0 20 read_log 193305'18262 (193305'18212) modify 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.951337.0:55288 2014-08-11 18:44:43.079223 2014-08-13 17:38:03.041397 7fe9bae887a0 20 read_log 193305'18263 (193305'18214) modify 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.951337.0:55426 2014-08-11 18:44:43.119821 2014-08-13 17:38:03.041410 7fe9bae887a0 20 read_log 193305'18264 (193305'18216) modify 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.951337.0:56787 2014-08-11 18:44:43.529643 2014-08-13 17:38:03.041422 7fe9bae887a0 20 read_log 193305'18265 (193305'18218) modify 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.951337.0:56807 2014-08-11 18:44:43.536332 2014-08-13 17:38:03.041436 7fe9bae887a0 20 read_log 193305'18266 (193305'18220) modify 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.951337.0:58064 2014-08-11 18:44:43.936225 2014-08-13 17:38:03.041448 7fe9bae887a0 20 read_log 193305'18267 (193305'18222) modify 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.951337.0:58825 2014-08-11 18:44:44.152212 2014-08-13 17:38:03.041462 7fe9bae887a0 20 read_log 193305'18268 (193305'18224) modify 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.951337.0:59853 2014-08-11 18:44:44.446654 2014-08-13 17:38:03.041474 7fe9bae887a0 20 read_log 193305'18269 (193305'18226) modify 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.951337.0:59907 2014-08-11 18:44:44.464127 2014-08-13 17:38:03.041488 7fe9bae887a0 20 read_log 193305'18270 (193305'18228) modify 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.951337.0:61156 2014-08-11 18:44:44.838716 2014-08-13 17:38:03.041508 7fe9bae887a0 20 read_log 193305'18271 (193305'18230) modify 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.951337.0:61413 2014-08-11 18:44:44.914604 2014-08-13 17:38:03.041523 7fe9bae887a0 20 read_log 193305'18272 (193305'18232) modify bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.951337.0:62776 2014-08-11 18:44:45.324090 2014-08-13 17:38:03.041535 7fe9bae887a0 20 read_log 193305'18273 (193305'18234) modify d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.951337.0:64252 2014-08-11 18:44:45.747289 2014-08-13 17:38:03.041547 7fe9bae887a0 20 read_log 193305'18274 (193305'18236) modify d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.951337.0:64796 2014-08-11 18:44:45.919881 2014-08-13 17:38:03.041559 7fe9bae887a0 20 read_log 193305'18275 (193305'18238) modify b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.951337.0:65027 2014-08-11 18:44:45.991668 2014-08-13 17:38:03.041571 7fe9bae887a0 20 read_log 193305'18276 (193305'18240) modify 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.951337.0:65068 2014-08-11 18:44:46.007445 2014-08-13 17:38:03.041583 7fe9bae887a0 20 read_log 194215'18277 (194215'18277) modify 30b/hit_set_24.30b_archive_2014-08-12 15:32:25.863655_2014-08-12 15:48:15.603804/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:15.604108 2014-08-13 17:38:03.041598 7fe9bae887a0 20 read_log 194215'18278 (193305'18168) delete 30b/hit_set_24.30b_archive_2014-08-11 18:38:11.461328_2014-08-11 18:42:40.507251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:15.604108 2014-08-13 17:38:03.041610 7fe9bae887a0 20 read_log 194215'18279 (192834'17831) clean da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by osd.4.0:2499 2014-08-12 15:48:15.614058 2014-08-13 17:38:03.041625 7fe9bae887a0 20 read_log 194215'18280 (194215'18279) delete da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.2174595.0:160490 0.000000 2014-08-13 17:38:03.041636 7fe9bae887a0 20 read_log 194215'18281 (192888'17962) clean 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by osd.4.0:2501 2014-08-12 15:48:15.622292 2014-08-13 17:38:03.041650 7fe9bae887a0 20 read_log 194215'18282 (194215'18281) delete 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.2174595.0:160492 0.000000 2014-08-13 17:38:03.041660 7fe9bae887a0 20 read_log 194215'18283 (193305'18250) clean 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by osd.4.0:2503 2014-08-12 15:48:15.675695 2014-08-13 17:38:03.041674 7fe9bae887a0 20 read_log 194215'18284 (194215'18283) delete 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.2174595.0:160494 0.000000 2014-08-13 17:38:03.041685 7fe9bae887a0 20 read_log 194215'18285 (193305'18258) clean 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by osd.4.0:2505 2014-08-12 15:48:15.692726 2014-08-13 17:38:03.041699 7fe9bae887a0 20 read_log 194215'18286 (194215'18285) delete 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.2174595.0:160496 0.000000 2014-08-13 17:38:03.041710 7fe9bae887a0 20 read_log 194215'18287 (193305'18256) clean 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by osd.4.0:2507 2014-08-12 15:48:15.700465 2014-08-13 17:38:03.041722 7fe9bae887a0 20 read_log 194215'18288 (194215'18287) delete 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.2174595.0:160498 0.000000 2014-08-13 17:38:03.041733 7fe9bae887a0 20 read_log 194215'18289 (193305'18274) clean d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by osd.4.0:2509 2014-08-12 15:48:15.708847 2014-08-13 17:38:03.041770 7fe9bae887a0 20 read_log 194215'18290 (194215'18289) delete d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.2174595.0:160500 0.000000 2014-08-13 17:38:03.041784 7fe9bae887a0 20 read_log 194215'18291 (192888'18136) clean 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by osd.4.0:2511 2014-08-12 15:48:15.719121 2014-08-13 17:38:03.041797 7fe9bae887a0 20 read_log 194215'18292 (194215'18291) delete 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.2174595.0:160502 0.000000 2014-08-13 17:38:03.041808 7fe9bae887a0 20 read_log 194215'18293 (193305'18241) clean ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by osd.4.0:2513 2014-08-12 15:48:15.728509 2014-08-13 17:38:03.041820 7fe9bae887a0 20 read_log 194215'18294 (194215'18293) delete ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.2174595.0:160504 0.000000 2014-08-13 17:38:03.041831 7fe9bae887a0 20 read_log 194215'18295 (192888'18128) clean 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by osd.4.0:2515 2014-08-12 15:48:15.738295 2014-08-13 17:38:03.041843 7fe9bae887a0 20 read_log 194215'18296 (194215'18295) delete 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.2174595.0:160506 0.000000 2014-08-13 17:38:03.041853 7fe9bae887a0 20 read_log 194215'18297 (192888'18127) clean 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by osd.4.0:2517 2014-08-12 15:48:15.748184 2014-08-13 17:38:03.041865 7fe9bae887a0 20 read_log 194215'18298 (194215'18297) delete 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.2174595.0:160508 0.000000 2014-08-13 17:38:03.041876 7fe9bae887a0 20 read_log 194215'18299 (192834'17772) clean 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by osd.4.0:2519 2014-08-12 15:48:15.758269 2014-08-13 17:38:03.041888 7fe9bae887a0 20 read_log 194215'18300 (194215'18299) delete 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.2174595.0:160510 0.000000 2014-08-13 17:38:03.041916 7fe9bae887a0 20 read_log 194215'18301 (192888'18153) clean 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by osd.4.0:2521 2014-08-12 15:48:15.767103 2014-08-13 17:38:03.041931 7fe9bae887a0 20 read_log 194215'18302 (194215'18301) delete 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.2174595.0:160512 0.000000 2014-08-13 17:38:03.041942 7fe9bae887a0 20 read_log 194215'18303 (192834'17192) clean a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.4.0:2523 2014-08-12 15:48:15.779603 2014-08-13 17:38:03.041954 7fe9bae887a0 20 read_log 194215'18304 (194215'18303) delete a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.2174595.0:160514 0.000000 2014-08-13 17:38:03.041965 7fe9bae887a0 20 read_log 194215'18305 (192834'17251) clean e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by osd.4.0:2525 2014-08-12 15:48:15.787502 2014-08-13 17:38:03.041978 7fe9bae887a0 20 read_log 194215'18306 (194215'18305) delete e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.2174595.0:160516 0.000000 2014-08-13 17:38:03.041989 7fe9bae887a0 20 read_log 194215'18307 (192888'17952) clean 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by osd.4.0:2527 2014-08-12 15:48:15.795148 2014-08-13 17:38:03.042004 7fe9bae887a0 20 read_log 194215'18308 (194215'18307) delete 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.2174595.0:160518 0.000000 2014-08-13 17:38:03.042014 7fe9bae887a0 20 read_log 194215'18309 (193305'18270) clean 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by osd.4.0:2529 2014-08-12 15:48:15.805318 2014-08-13 17:38:03.042036 7fe9bae887a0 20 read_log 194215'18310 (194215'18309) delete 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.2174595.0:160520 0.000000 2014-08-13 17:38:03.042050 7fe9bae887a0 20 read_log 194215'18311 (192888'17945) clean 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by osd.4.0:2531 2014-08-12 15:48:15.814257 2014-08-13 17:38:03.042062 7fe9bae887a0 20 read_log 194215'18312 (194215'18311) delete 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.2174595.0:160522 0.000000 2014-08-13 17:38:03.042073 7fe9bae887a0 20 read_log 194215'18313 (192834'17236) clean 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by osd.4.0:2533 2014-08-12 15:48:15.823922 2014-08-13 17:38:03.042085 7fe9bae887a0 20 read_log 194215'18314 (194215'18313) delete 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.2174595.0:160524 0.000000 2014-08-13 17:38:03.042096 7fe9bae887a0 20 read_log 194215'18315 (192888'18134) clean e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by osd.4.0:2535 2014-08-12 15:48:15.833671 2014-08-13 17:38:03.042108 7fe9bae887a0 20 read_log 194215'18316 (194215'18315) delete e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.2174595.0:160526 0.000000 2014-08-13 17:38:03.042118 7fe9bae887a0 20 read_log 194215'18317 (192834'17171) clean 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by osd.4.0:2537 2014-08-12 15:48:15.842926 2014-08-13 17:38:03.042130 7fe9bae887a0 20 read_log 194215'18318 (194215'18317) delete 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.2174595.0:160528 0.000000 2014-08-13 17:38:03.042141 7fe9bae887a0 20 read_log 194215'18319 (192888'17946) clean a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by osd.4.0:2539 2014-08-12 15:48:15.852239 2014-08-13 17:38:03.042160 7fe9bae887a0 20 read_log 194215'18320 (194215'18319) delete a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.2174595.0:160530 0.000000 2014-08-13 17:38:03.042173 7fe9bae887a0 20 read_log 194215'18321 (192834'17151) clean a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by osd.4.0:2541 2014-08-12 15:48:15.860577 2014-08-13 17:38:03.042186 7fe9bae887a0 20 read_log 194215'18322 (194215'18321) delete a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.2174595.0:160532 0.000000 2014-08-13 17:38:03.042196 7fe9bae887a0 20 read_log 194215'18323 (192834'17716) clean 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by osd.4.0:2543 2014-08-12 15:48:15.868447 2014-08-13 17:38:03.042208 7fe9bae887a0 20 read_log 194215'18324 (194215'18323) delete 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.2174595.0:160534 0.000000 2014-08-13 17:38:03.042219 7fe9bae887a0 20 read_log 194215'18325 (192888'17941) clean aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by osd.4.0:2545 2014-08-12 15:48:15.877531 2014-08-13 17:38:03.042231 7fe9bae887a0 20 read_log 194215'18326 (194215'18325) delete aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.2174595.0:160536 0.000000 2014-08-13 17:38:03.042241 7fe9bae887a0 20 read_log 194215'18327 (192888'18159) clean cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by osd.4.0:2547 2014-08-12 15:48:15.885897 2014-08-13 17:38:03.042256 7fe9bae887a0 20 read_log 194215'18328 (194215'18327) delete cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.2174595.0:160538 0.000000 2014-08-13 17:38:03.042267 7fe9bae887a0 20 read_log 194215'18329 (193305'18261) clean bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by osd.4.0:2549 2014-08-12 15:48:15.897343 2014-08-13 17:38:03.042287 7fe9bae887a0 20 read_log 194215'18330 (194215'18329) delete bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.2174595.0:160540 0.000000 2014-08-13 17:38:03.042300 7fe9bae887a0 20 read_log 194215'18331 (193305'18245) clean a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by osd.4.0:2551 2014-08-12 15:48:15.907064 2014-08-13 17:38:03.042313 7fe9bae887a0 20 read_log 194215'18332 (194215'18331) delete a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.2174595.0:160542 0.000000 2014-08-13 17:38:03.042323 7fe9bae887a0 20 read_log 194215'18333 (193305'18260) clean c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by osd.4.0:2553 2014-08-12 15:48:15.916382 2014-08-13 17:38:03.042335 7fe9bae887a0 20 read_log 194215'18334 (194215'18333) delete c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.2174595.0:160544 0.000000 2014-08-13 17:38:03.042346 7fe9bae887a0 20 read_log 194215'18335 (193305'18271) clean 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by osd.4.0:2555 2014-08-12 15:48:15.924967 2014-08-13 17:38:03.042358 7fe9bae887a0 20 read_log 194215'18336 (194215'18335) delete 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.2174595.0:160546 0.000000 2014-08-13 17:38:03.042369 7fe9bae887a0 20 read_log 194215'18337 (192888'17955) clean e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by osd.4.0:2557 2014-08-12 15:48:15.933508 2014-08-13 17:38:03.042381 7fe9bae887a0 20 read_log 194215'18338 (194215'18337) delete e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.2174595.0:160548 0.000000 2014-08-13 17:38:03.042391 7fe9bae887a0 20 read_log 194215'18339 (192834'17787) clean d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by osd.4.0:2559 2014-08-12 15:48:15.943347 2014-08-13 17:38:03.042406 7fe9bae887a0 20 read_log 194215'18340 (194215'18339) delete d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.2174595.0:160550 0.000000 2014-08-13 17:38:03.042417 7fe9bae887a0 20 read_log 194215'18341 (192834'17319) clean 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by osd.4.0:2561 2014-08-12 15:48:15.952390 2014-08-13 17:38:03.042429 7fe9bae887a0 20 read_log 194215'18342 (194215'18341) delete 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.2174595.0:160552 0.000000 2014-08-13 17:38:03.042440 7fe9bae887a0 20 read_log 194215'18343 (193305'18253) clean 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by osd.4.0:2563 2014-08-12 15:48:16.006517 2014-08-13 17:38:03.042454 7fe9bae887a0 20 read_log 194215'18344 (194215'18343) delete 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.2174595.0:160554 0.000000 2014-08-13 17:38:03.042465 7fe9bae887a0 20 read_log 194215'18345 (192888'17983) clean 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by osd.4.0:2565 2014-08-12 15:48:16.014932 2014-08-13 17:38:03.042477 7fe9bae887a0 20 read_log 194215'18346 (194215'18345) delete 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.2174595.0:160556 0.000000 2014-08-13 17:38:03.042488 7fe9bae887a0 20 read_log 194215'18347 (192834'17769) clean 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by osd.4.0:2567 2014-08-12 15:48:16.023008 2014-08-13 17:38:03.042500 7fe9bae887a0 20 read_log 194215'18348 (194215'18347) delete 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.2174595.0:160558 0.000000 2014-08-13 17:38:03.042510 7fe9bae887a0 20 read_log 194215'18349 (192888'18111) clean e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by osd.4.0:2569 2014-08-12 15:48:16.031143 2014-08-13 17:38:03.042529 7fe9bae887a0 20 read_log 194215'18350 (194215'18349) delete e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.2174595.0:160560 0.000000 2014-08-13 17:38:03.042540 7fe9bae887a0 20 read_log 194215'18351 (192834'17767) clean 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by osd.4.0:2571 2014-08-12 15:48:16.039399 2014-08-13 17:38:03.042553 7fe9bae887a0 20 read_log 194215'18352 (194215'18351) delete 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.2174595.0:160562 0.000000 2014-08-13 17:38:03.042563 7fe9bae887a0 20 read_log 194215'18353 (193305'18268) clean 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by osd.4.0:2573 2014-08-12 15:48:16.048353 2014-08-13 17:38:03.042575 7fe9bae887a0 20 read_log 194215'18354 (194215'18353) delete 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.2174595.0:160564 0.000000 2014-08-13 17:38:03.042586 7fe9bae887a0 20 read_log 194215'18355 (192834'17340) clean a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by osd.4.0:2575 2014-08-12 15:48:16.056757 2014-08-13 17:38:03.042598 7fe9bae887a0 20 read_log 194215'18356 (194215'18355) delete a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.2174595.0:160566 0.000000 2014-08-13 17:38:03.042609 7fe9bae887a0 20 read_log 194215'18357 (192834'17242) clean e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by osd.4.0:2577 2014-08-12 15:48:16.065638 2014-08-13 17:38:03.042621 7fe9bae887a0 20 read_log 194215'18358 (194215'18357) delete e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.2174595.0:160568 0.000000 2014-08-13 17:38:03.042632 7fe9bae887a0 20 read_log 194215'18359 (192834'17847) clean f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by osd.4.0:2579 2014-08-12 15:48:16.074161 2014-08-13 17:38:03.042644 7fe9bae887a0 20 read_log 194215'18360 (194215'18359) delete f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.2174595.0:160570 0.000000 2014-08-13 17:38:03.042655 7fe9bae887a0 20 read_log 194215'18361 (192834'17823) clean 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by osd.4.0:2581 2014-08-12 15:48:16.082421 2014-08-13 17:38:03.042667 7fe9bae887a0 20 read_log 194215'18362 (194215'18361) delete 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.2174595.0:160572 0.000000 2014-08-13 17:38:03.042677 7fe9bae887a0 20 read_log 194215'18363 (192888'17943) clean 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by osd.4.0:2583 2014-08-12 15:48:16.090435 2014-08-13 17:38:03.042689 7fe9bae887a0 20 read_log 194215'18364 (194215'18363) delete 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.2174595.0:160574 0.000000 2014-08-13 17:38:03.042700 7fe9bae887a0 20 read_log 194215'18365 (192834'17352) clean d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by osd.4.0:2585 2014-08-12 15:48:16.098066 2014-08-13 17:38:03.042712 7fe9bae887a0 20 read_log 194215'18366 (194215'18365) delete d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.2174595.0:160576 0.000000 2014-08-13 17:38:03.042723 7fe9bae887a0 20 read_log 194215'18367 (192888'17949) clean 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by osd.4.0:2587 2014-08-12 15:48:16.105486 2014-08-13 17:38:03.042735 7fe9bae887a0 20 read_log 194215'18368 (194215'18367) delete 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.2174595.0:160578 0.000000 2014-08-13 17:38:03.042745 7fe9bae887a0 20 read_log 194215'18369 (192834'17299) clean 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by osd.4.0:2589 2014-08-12 15:48:16.112601 2014-08-13 17:38:03.042766 7fe9bae887a0 20 read_log 194215'18370 (194215'18369) delete 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.2174595.0:160580 0.000000 2014-08-13 17:38:03.042778 7fe9bae887a0 20 read_log 194215'18371 (192834'17817) clean ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by osd.4.0:2591 2014-08-12 15:48:16.120558 2014-08-13 17:38:03.042790 7fe9bae887a0 20 read_log 194215'18372 (194215'18371) delete ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.2174595.0:160582 0.000000 2014-08-13 17:38:03.042801 7fe9bae887a0 20 read_log 194215'18373 (192834'17322) clean 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by osd.4.0:2593 2014-08-12 15:48:16.129488 2014-08-13 17:38:03.042813 7fe9bae887a0 20 read_log 194215'18374 (194215'18373) delete 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.2174595.0:160584 0.000000 2014-08-13 17:38:03.042823 7fe9bae887a0 20 read_log 194215'18375 (192834'17250) clean ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by osd.4.0:2595 2014-08-12 15:48:16.138879 2014-08-13 17:38:03.042880 7fe9bae887a0 20 read_log 194215'18376 (194215'18375) delete ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.2174595.0:160586 0.000000 2014-08-13 17:38:03.042913 7fe9bae887a0 20 read_log 194215'18377 (192834'17158) clean e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by osd.4.0:2597 2014-08-12 15:48:16.149562 2014-08-13 17:38:03.042926 7fe9bae887a0 20 read_log 194215'18378 (194215'18377) delete e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.2174595.0:160588 0.000000 2014-08-13 17:38:03.042937 7fe9bae887a0 20 read_log 194215'18379 (192888'17972) clean a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by osd.4.0:2599 2014-08-12 15:48:16.158418 2014-08-13 17:38:03.042949 7fe9bae887a0 20 read_log 194215'18380 (194215'18379) delete a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.2174595.0:160590 0.000000 2014-08-13 17:38:03.042959 7fe9bae887a0 20 read_log 194215'18381 (192888'18114) clean 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by osd.4.0:2601 2014-08-12 15:48:16.167277 2014-08-13 17:38:03.042971 7fe9bae887a0 20 read_log 194215'18382 (194215'18381) delete 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.2174595.0:160592 0.000000 2014-08-13 17:38:03.042982 7fe9bae887a0 20 read_log 194215'18383 (192888'18109) clean c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by osd.4.0:2603 2014-08-12 15:48:16.177775 2014-08-13 17:38:03.042995 7fe9bae887a0 20 read_log 194215'18384 (194215'18383) delete c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.2174595.0:160594 0.000000 2014-08-13 17:38:03.043006 7fe9bae887a0 20 read_log 194215'18385 (192888'18147) clean 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by osd.4.0:2605 2014-08-12 15:48:16.186554 2014-08-13 17:38:03.043018 7fe9bae887a0 20 read_log 194215'18386 (194215'18385) delete 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.2174595.0:160596 0.000000 2014-08-13 17:38:03.043029 7fe9bae887a0 20 read_log 194215'18387 (192834'17248) clean c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by osd.4.0:2607 2014-08-12 15:48:16.196497 2014-08-13 17:38:03.043041 7fe9bae887a0 20 read_log 194215'18388 (194215'18387) delete c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.2174595.0:160598 0.000000 2014-08-13 17:38:03.043052 7fe9bae887a0 20 read_log 194215'18389 (192834'17269) clean 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by osd.4.0:2609 2014-08-12 15:48:16.205487 2014-08-13 17:38:03.043072 7fe9bae887a0 20 read_log 194215'18390 (194215'18389) delete 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.2174595.0:160600 0.000000 2014-08-13 17:38:03.043083 7fe9bae887a0 20 read_log 194215'18391 (192888'18151) clean e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by osd.4.0:2611 2014-08-12 15:48:16.213286 2014-08-13 17:38:03.043095 7fe9bae887a0 20 read_log 194215'18392 (194215'18391) delete e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.2174595.0:160602 0.000000 2014-08-13 17:38:03.043106 7fe9bae887a0 20 read_log 194215'18393 (192888'18157) clean 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by osd.4.0:2613 2014-08-12 15:48:16.221772 2014-08-13 17:38:03.043136 7fe9bae887a0 20 read_log 194215'18394 (194215'18393) delete 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.2174595.0:160604 0.000000 2014-08-13 17:38:03.043150 7fe9bae887a0 20 read_log 194215'18395 (192834'17786) clean 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by osd.4.0:2615 2014-08-12 15:48:16.230497 2014-08-13 17:38:03.043171 7fe9bae887a0 20 read_log 194215'18396 (194215'18395) delete 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.2174595.0:160606 0.000000 2014-08-13 17:38:03.043182 7fe9bae887a0 20 read_log 194215'18397 (193305'18275) clean b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by osd.4.0:2617 2014-08-12 15:48:16.240639 2014-08-13 17:38:03.043195 7fe9bae887a0 20 read_log 194215'18398 (194215'18397) delete b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.2174595.0:160608 0.000000 2014-08-13 17:38:03.043205 7fe9bae887a0 20 read_log 194215'18399 (192834'17799) clean c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by osd.4.0:2619 2014-08-12 15:48:16.250388 2014-08-13 17:38:03.043217 7fe9bae887a0 20 read_log 194215'18400 (194215'18399) delete c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.2174595.0:160610 0.000000 2014-08-13 17:38:03.043228 7fe9bae887a0 20 read_log 194215'18401 (193305'18276) clean 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by osd.4.0:2621 2014-08-12 15:48:16.258931 2014-08-13 17:38:03.043240 7fe9bae887a0 20 read_log 194215'18402 (194215'18401) delete 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.2174595.0:160612 0.000000 2014-08-13 17:38:03.043251 7fe9bae887a0 20 read_log 194215'18403 (193305'18272) clean bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by osd.4.0:2623 2014-08-12 15:48:16.267409 2014-08-13 17:38:03.043263 7fe9bae887a0 20 read_log 194215'18404 (194215'18403) delete bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.2174595.0:160614 0.000000 2014-08-13 17:38:03.043274 7fe9bae887a0 20 read_log 194215'18405 (193305'18248) clean 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by osd.4.0:2625 2014-08-12 15:48:16.275936 2014-08-13 17:38:03.043286 7fe9bae887a0 20 read_log 194215'18406 (194215'18405) delete 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.2174595.0:160616 0.000000 2014-08-13 17:38:03.043296 7fe9bae887a0 20 read_log 194215'18407 (193305'18252) clean a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by osd.4.0:2627 2014-08-12 15:48:16.286200 2014-08-13 17:38:03.043308 7fe9bae887a0 20 read_log 194215'18408 (194215'18407) delete a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.2174595.0:160618 0.000000 2014-08-13 17:38:03.043319 7fe9bae887a0 20 read_log 194215'18409 (192888'18164) clean d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by osd.4.0:2629 2014-08-12 15:48:16.294691 2014-08-13 17:38:03.043338 7fe9bae887a0 20 read_log 194215'18410 (194215'18409) delete d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.2174595.0:160620 0.000000 2014-08-13 17:38:03.043350 7fe9bae887a0 20 read_log 194215'18411 (192888'18138) clean 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by osd.4.0:2631 2014-08-12 15:48:16.302788 2014-08-13 17:38:03.043362 7fe9bae887a0 20 read_log 194215'18412 (194215'18411) delete 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.2174595.0:160622 0.000000 2014-08-13 17:38:03.043373 7fe9bae887a0 20 read_log 194215'18413 (192834'17222) clean 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by osd.4.0:2633 2014-08-12 15:48:16.311194 2014-08-13 17:38:03.043385 7fe9bae887a0 20 read_log 194215'18414 (194215'18413) delete 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.2174595.0:160624 0.000000 2014-08-13 17:38:03.043395 7fe9bae887a0 20 read_log 194215'18415 (193305'18244) clean 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by osd.4.0:2635 2014-08-12 15:48:16.319498 2014-08-13 17:38:03.043407 7fe9bae887a0 20 read_log 194215'18416 (194215'18415) delete 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.2174595.0:160626 0.000000 2014-08-13 17:38:03.043418 7fe9bae887a0 20 read_log 194215'18417 (193305'18247) clean 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by osd.4.0:2637 2014-08-12 15:48:16.328340 2014-08-13 17:38:03.043430 7fe9bae887a0 20 read_log 194215'18418 (194215'18417) delete 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.2174595.0:160628 0.000000 2014-08-13 17:38:03.043441 7fe9bae887a0 20 read_log 194215'18419 (192834'17343) clean 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by osd.4.0:2639 2014-08-12 15:48:16.337192 2014-08-13 17:38:03.043453 7fe9bae887a0 20 read_log 194215'18420 (194215'18419) delete 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.2174595.0:160630 0.000000 2014-08-13 17:38:03.043464 7fe9bae887a0 20 read_log 194215'18421 (192834'17325) clean c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by osd.4.0:2641 2014-08-12 15:48:16.346718 2014-08-13 17:38:03.043476 7fe9bae887a0 20 read_log 194215'18422 (194215'18421) delete c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.2174595.0:160632 0.000000 2014-08-13 17:38:03.043486 7fe9bae887a0 20 read_log 194215'18423 (192888'17950) clean 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by osd.4.0:2643 2014-08-12 15:48:16.355336 2014-08-13 17:38:03.043498 7fe9bae887a0 20 read_log 194215'18424 (194215'18423) delete 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.2174595.0:160634 0.000000 2014-08-13 17:38:03.043509 7fe9bae887a0 20 read_log 194215'18425 (192888'18135) clean f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by osd.4.0:2645 2014-08-12 15:48:16.365805 2014-08-13 17:38:03.043521 7fe9bae887a0 20 read_log 194215'18426 (194215'18425) delete f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.2174595.0:160636 0.000000 2014-08-13 17:38:03.043532 7fe9bae887a0 20 read_log 194215'18427 (192834'17351) clean 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by osd.4.0:2647 2014-08-12 15:48:16.374695 2014-08-13 17:38:03.043544 7fe9bae887a0 20 read_log 194215'18428 (194215'18427) delete 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.2174595.0:160638 0.000000 2014-08-13 17:38:03.043555 7fe9bae887a0 20 read_log 194215'18429 (192834'17278) clean 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by osd.4.0:2649 2014-08-12 15:48:16.383421 2014-08-13 17:38:03.043595 7fe9bae887a0 20 read_log 194215'18430 (194215'18429) delete 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.2174595.0:160640 0.000000 2014-08-13 17:38:03.043610 7fe9bae887a0 20 read_log 194215'18431 (192888'18131) clean 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by osd.4.0:2651 2014-08-12 15:48:16.393296 2014-08-13 17:38:03.043623 7fe9bae887a0 20 read_log 194215'18432 (194215'18431) delete 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.2174595.0:160642 0.000000 2014-08-13 17:38:03.043634 7fe9bae887a0 20 read_log 194215'18433 (192834'17353) clean d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by osd.4.0:2653 2014-08-12 15:48:16.401817 2014-08-13 17:38:03.043646 7fe9bae887a0 20 read_log 194215'18434 (194215'18433) delete d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.2174595.0:160644 0.000000 2014-08-13 17:38:03.043657 7fe9bae887a0 20 read_log 194215'18435 (192834'17309) clean f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by osd.4.0:2655 2014-08-12 15:48:16.411407 2014-08-13 17:38:03.043669 7fe9bae887a0 20 read_log 194215'18436 (194215'18435) delete f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.2174595.0:160646 0.000000 2014-08-13 17:38:03.043680 7fe9bae887a0 20 read_log 194215'18437 (192888'17984) clean 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by osd.4.0:2657 2014-08-12 15:48:16.421521 2014-08-13 17:38:03.043692 7fe9bae887a0 20 read_log 194215'18438 (194215'18437) delete 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.2174595.0:160648 0.000000 2014-08-13 17:38:03.043703 7fe9bae887a0 20 read_log 194215'18439 (192834'17260) clean 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by osd.4.0:2659 2014-08-12 15:48:16.448559 2014-08-13 17:38:03.043715 7fe9bae887a0 20 read_log 194215'18440 (194215'18439) delete 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.2174595.0:160650 0.000000 2014-08-13 17:38:03.043726 7fe9bae887a0 20 read_log 194215'18441 (193305'18259) clean e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by osd.4.0:2661 2014-08-12 15:48:16.458774 2014-08-13 17:38:03.043738 7fe9bae887a0 20 read_log 194215'18442 (194215'18441) delete e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.2174595.0:160652 0.000000 2014-08-13 17:38:03.043748 7fe9bae887a0 20 read_log 194215'18443 (192888'17956) clean 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by osd.4.0:2663 2014-08-12 15:48:16.468416 2014-08-13 17:38:03.043760 7fe9bae887a0 20 read_log 194215'18444 (194215'18443) delete 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.2174595.0:160654 0.000000 2014-08-13 17:38:03.043771 7fe9bae887a0 20 read_log 194215'18445 (192888'17968) clean 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by osd.4.0:2665 2014-08-12 15:48:16.476818 2014-08-13 17:38:03.043783 7fe9bae887a0 20 read_log 194215'18446 (194215'18445) delete 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.2174595.0:160656 0.000000 2014-08-13 17:38:03.043794 7fe9bae887a0 20 read_log 194215'18447 (193305'18251) clean 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by osd.4.0:2667 2014-08-12 15:48:16.484892 2014-08-13 17:38:03.043806 7fe9bae887a0 20 read_log 194215'18448 (194215'18447) delete 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.2174595.0:160658 0.000000 2014-08-13 17:38:03.043817 7fe9bae887a0 20 read_log 194215'18449 (192888'18148) clean 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by osd.4.0:2669 2014-08-12 15:48:16.492943 2014-08-13 17:38:03.043837 7fe9bae887a0 20 read_log 194215'18450 (194215'18449) delete 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.2174595.0:160660 0.000000 2014-08-13 17:38:03.043849 7fe9bae887a0 20 read_log 194215'18451 (193305'18257) clean 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by osd.4.0:2671 2014-08-12 15:48:16.501209 2014-08-13 17:38:03.043861 7fe9bae887a0 20 read_log 194215'18452 (194215'18451) delete 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.2174595.0:160662 0.000000 2014-08-13 17:38:03.043872 7fe9bae887a0 20 read_log 194215'18453 (192834'17207) clean 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by osd.4.0:2673 2014-08-12 15:48:16.510799 2014-08-13 17:38:03.043884 7fe9bae887a0 20 read_log 194215'18454 (194215'18453) delete 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.2174595.0:160664 0.000000 2014-08-13 17:38:03.043894 7fe9bae887a0 20 read_log 194215'18455 (192834'17203) clean a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by osd.4.0:2675 2014-08-12 15:48:16.520531 2014-08-13 17:38:03.043907 7fe9bae887a0 20 read_log 194215'18456 (194215'18455) delete a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.2174595.0:160666 0.000000 2014-08-13 17:38:03.043918 7fe9bae887a0 20 read_log 194215'18457 (192834'17214) clean e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by osd.4.0:2677 2014-08-12 15:48:16.529447 2014-08-13 17:38:03.043930 7fe9bae887a0 20 read_log 194215'18458 (194215'18457) delete e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.2174595.0:160668 0.000000 2014-08-13 17:38:03.043941 7fe9bae887a0 20 read_log 194215'18459 (192888'17953) clean 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by osd.4.0:2679 2014-08-12 15:48:16.538132 2014-08-13 17:38:03.043953 7fe9bae887a0 20 read_log 194215'18460 (194215'18459) delete 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.2174595.0:160670 0.000000 2014-08-13 17:38:03.043963 7fe9bae887a0 20 read_log 194215'18461 (192888'18142) clean d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by osd.4.0:2681 2014-08-12 15:48:16.555928 2014-08-13 17:38:03.043975 7fe9bae887a0 20 read_log 194215'18462 (194215'18461) delete d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.2174595.0:160672 0.000000 2014-08-13 17:38:03.043986 7fe9bae887a0 20 read_log 194215'18463 (193305'18263) clean 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by osd.4.0:2683 2014-08-12 15:48:16.566026 2014-08-13 17:38:03.043998 7fe9bae887a0 20 read_log 194215'18464 (194215'18463) delete 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.2174595.0:160674 0.000000 2014-08-13 17:38:03.044008 7fe9bae887a0 20 read_log 194215'18465 (192888'18124) clean 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by osd.4.0:2685 2014-08-12 15:48:16.575026 2014-08-13 17:38:03.044020 7fe9bae887a0 20 read_log 194215'18466 (194215'18465) delete 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.2174595.0:160676 0.000000 2014-08-13 17:38:03.044031 7fe9bae887a0 20 read_log 194215'18467 (193305'18266) clean 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by osd.4.0:2687 2014-08-12 15:48:16.585135 2014-08-13 17:38:03.044043 7fe9bae887a0 20 read_log 194215'18468 (194215'18467) delete 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.2174595.0:160678 0.000000 2014-08-13 17:38:03.044054 7fe9bae887a0 20 read_log 194215'18469 (192888'18110) clean 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by osd.4.0:2689 2014-08-12 15:48:16.594372 2014-08-13 17:38:03.044072 7fe9bae887a0 20 read_log 194215'18470 (194215'18469) delete 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.2174595.0:160680 0.000000 2014-08-13 17:38:03.044084 7fe9bae887a0 20 read_log 194215'18471 (192834'17747) clean 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by osd.4.0:2691 2014-08-12 15:48:16.602842 2014-08-13 17:38:03.044097 7fe9bae887a0 20 read_log 194215'18472 (194215'18471) delete 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.2174595.0:160682 0.000000 2014-08-13 17:38:03.044108 7fe9bae887a0 20 read_log 194215'18473 (192888'18116) clean 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by osd.4.0:2693 2014-08-12 15:48:16.611061 2014-08-13 17:38:03.044120 7fe9bae887a0 20 read_log 194215'18474 (194215'18473) delete 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.2174595.0:160684 0.000000 2014-08-13 17:38:03.044131 7fe9bae887a0 20 read_log 194215'18475 (192834'17249) clean cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by osd.4.0:2695 2014-08-12 15:48:16.618466 2014-08-13 17:38:03.044143 7fe9bae887a0 20 read_log 194215'18476 (194215'18475) delete cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.2174595.0:160686 0.000000 2014-08-13 17:38:03.044158 7fe9bae887a0 20 read_log 194215'18477 (192888'17961) clean 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by osd.4.0:2697 2014-08-12 15:48:16.626523 2014-08-13 17:38:03.044171 7fe9bae887a0 20 read_log 194215'18478 (194215'18477) delete 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.2174595.0:160688 0.000000 2014-08-13 17:38:03.044182 7fe9bae887a0 20 read_log 194215'18479 (192888'17979) clean 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by osd.4.0:2699 2014-08-12 15:48:16.635419 2014-08-13 17:38:03.044194 7fe9bae887a0 20 read_log 194215'18480 (194215'18479) delete 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.2174595.0:160690 0.000000 2014-08-13 17:38:03.044205 7fe9bae887a0 20 read_log 194215'18481 (192888'18119) clean c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by osd.4.0:2701 2014-08-12 15:48:16.643505 2014-08-13 17:38:03.044217 7fe9bae887a0 20 read_log 194215'18482 (194215'18481) delete c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.2174595.0:160692 0.000000 2014-08-13 17:38:03.044227 7fe9bae887a0 20 read_log 194215'18483 (193305'18246) clean 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by osd.4.0:2703 2014-08-12 15:48:16.651625 2014-08-13 17:38:03.044239 7fe9bae887a0 20 read_log 194215'18484 (194215'18483) delete 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.2174595.0:160694 0.000000 2014-08-13 17:38:03.044250 7fe9bae887a0 20 read_log 194215'18485 (192888'17974) clean 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by osd.4.0:2705 2014-08-12 15:48:16.660964 2014-08-13 17:38:03.044265 7fe9bae887a0 20 read_log 194215'18486 (194215'18485) delete 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.2174595.0:160696 0.000000 2014-08-13 17:38:03.044294 7fe9bae887a0 20 read_log 194215'18487 (192834'17225) clean 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by osd.4.0:2707 2014-08-12 15:48:16.669714 2014-08-13 17:38:03.044309 7fe9bae887a0 20 read_log 194215'18488 (194215'18487) delete 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.2174595.0:160698 0.000000 2014-08-13 17:38:03.044320 7fe9bae887a0 20 read_log 194215'18489 (192834'17722) clean 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by osd.4.0:2709 2014-08-12 15:48:16.678155 2014-08-13 17:38:03.044342 7fe9bae887a0 20 read_log 194215'18490 (194215'18489) delete 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.2174595.0:160700 0.000000 2014-08-13 17:38:03.044356 7fe9bae887a0 20 read_log 194215'18491 (192834'17758) clean 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by osd.4.0:2711 2014-08-12 15:48:16.686889 2014-08-13 17:38:03.044368 7fe9bae887a0 20 read_log 194215'18492 (194215'18491) delete 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.2174595.0:160702 0.000000 2014-08-13 17:38:03.044379 7fe9bae887a0 20 read_log 194215'18493 (192888'18133) clean d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by osd.4.0:2713 2014-08-12 15:48:16.694833 2014-08-13 17:38:03.044393 7fe9bae887a0 20 read_log 194215'18494 (194215'18493) delete d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.2174595.0:160704 0.000000 2014-08-13 17:38:03.044404 7fe9bae887a0 20 read_log 194215'18495 (193305'18254) clean a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by osd.4.0:2715 2014-08-12 15:48:16.701823 2014-08-13 17:38:03.044416 7fe9bae887a0 20 read_log 194215'18496 (194215'18495) delete a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.2174595.0:160706 0.000000 2014-08-13 17:38:03.044427 7fe9bae887a0 20 read_log 194215'18497 (193305'18243) clean d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by osd.4.0:2717 2014-08-12 15:48:16.711152 2014-08-13 17:38:03.044439 7fe9bae887a0 20 read_log 194215'18498 (194215'18497) delete d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.2174595.0:160708 0.000000 2014-08-13 17:38:03.044449 7fe9bae887a0 20 read_log 194215'18499 (192834'17333) clean 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by osd.4.0:2719 2014-08-12 15:48:16.719616 2014-08-13 17:38:03.044462 7fe9bae887a0 20 read_log 194215'18500 (194215'18499) delete 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.2174595.0:160710 0.000000 2014-08-13 17:38:03.044472 7fe9bae887a0 20 read_log 194215'18501 (192834'17235) clean 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by osd.4.0:2721 2014-08-12 15:48:16.728109 2014-08-13 17:38:03.044486 7fe9bae887a0 20 read_log 194215'18502 (194215'18501) delete 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.2174595.0:160712 0.000000 2014-08-13 17:38:03.044498 7fe9bae887a0 20 read_log 194215'18503 (192888'17975) clean 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by osd.4.0:2723 2014-08-12 15:48:16.735976 2014-08-13 17:38:03.044510 7fe9bae887a0 20 read_log 194215'18504 (194215'18503) delete 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.2174595.0:160714 0.000000 2014-08-13 17:38:03.044521 7fe9bae887a0 20 read_log 194215'18505 (192888'18143) clean 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by osd.4.0:2725 2014-08-12 15:48:16.743633 2014-08-13 17:38:03.044533 7fe9bae887a0 20 read_log 194215'18506 (194215'18505) delete 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.2174595.0:160716 0.000000 2014-08-13 17:38:03.044544 7fe9bae887a0 20 read_log 194215'18507 (192834'17751) clean b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by osd.4.0:2727 2014-08-12 15:48:16.752139 2014-08-13 17:38:03.044556 7fe9bae887a0 20 read_log 194215'18508 (194215'18507) delete b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.2174595.0:160718 0.000000 2014-08-13 17:38:03.044566 7fe9bae887a0 20 read_log 194215'18509 (193305'18264) clean 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by osd.4.0:2729 2014-08-12 15:48:16.760354 2014-08-13 17:38:03.044586 7fe9bae887a0 20 read_log 194215'18510 (194215'18509) delete 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.2174595.0:160720 0.000000 2014-08-13 17:38:03.044600 7fe9bae887a0 20 read_log 194215'18511 (192888'18152) clean 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by osd.4.0:2731 2014-08-12 15:48:16.769309 2014-08-13 17:38:03.044613 7fe9bae887a0 20 read_log 194215'18512 (194215'18511) delete 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.2174595.0:160722 0.000000 2014-08-13 17:38:03.044623 7fe9bae887a0 20 read_log 194215'18513 (193305'18267) clean 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by osd.4.0:2733 2014-08-12 15:48:16.778579 2014-08-13 17:38:03.044639 7fe9bae887a0 20 read_log 194215'18514 (194215'18513) delete 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.2174595.0:160724 0.000000 2014-08-13 17:38:03.044650 7fe9bae887a0 20 read_log 194215'18515 (192834'17216) clean e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by osd.4.0:2735 2014-08-12 15:48:16.788307 2014-08-13 17:38:03.044662 7fe9bae887a0 20 read_log 194215'18516 (194215'18515) delete e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.2174595.0:160726 0.000000 2014-08-13 17:38:03.044673 7fe9bae887a0 20 read_log 194215'18517 (193305'18249) clean 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by osd.4.0:2737 2014-08-12 15:48:16.796235 2014-08-13 17:38:03.044691 7fe9bae887a0 20 read_log 194215'18518 (194215'18517) delete 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.2174595.0:160728 0.000000 2014-08-13 17:38:03.044703 7fe9bae887a0 20 read_log 194215'18519 (192888'18165) clean 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by osd.4.0:2739 2014-08-12 15:48:16.804794 2014-08-13 17:38:03.044717 7fe9bae887a0 20 read_log 194215'18520 (194215'18519) delete 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.2174595.0:160730 0.000000 2014-08-13 17:38:03.044728 7fe9bae887a0 20 read_log 194215'18521 (192834'17727) clean 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by osd.4.0:2741 2014-08-12 15:48:16.812858 2014-08-13 17:38:03.044742 7fe9bae887a0 20 read_log 194215'18522 (194215'18521) delete 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.2174595.0:160732 0.000000 2014-08-13 17:38:03.044753 7fe9bae887a0 20 read_log 194215'18523 (193305'18273) clean d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by osd.4.0:2743 2014-08-12 15:48:16.823024 2014-08-13 17:38:03.044766 7fe9bae887a0 20 read_log 194215'18524 (194215'18523) delete d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.2174595.0:160734 0.000000 2014-08-13 17:38:03.044778 7fe9bae887a0 20 read_log 194215'18525 (192834'17768) clean 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by osd.4.0:2745 2014-08-12 15:48:16.832344 2014-08-13 17:38:03.044792 7fe9bae887a0 20 read_log 194215'18526 (194215'18525) delete 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.2174595.0:160736 0.000000 2014-08-13 17:38:03.044803 7fe9bae887a0 20 read_log 194215'18527 (192888'17965) clean d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by osd.4.0:2747 2014-08-12 15:48:16.840478 2014-08-13 17:38:03.044815 7fe9bae887a0 20 read_log 194215'18528 (194215'18527) delete d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.2174595.0:160738 0.000000 2014-08-13 17:38:03.044829 7fe9bae887a0 20 read_log 194215'18529 (192888'18126) clean 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by osd.4.0:2749 2014-08-12 15:48:16.849979 2014-08-13 17:38:03.044853 7fe9bae887a0 20 read_log 194215'18530 (194215'18529) delete 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.2174595.0:160740 0.000000 2014-08-13 17:38:03.044867 7fe9bae887a0 20 read_log 194215'18531 (192888'18144) clean 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by osd.4.0:2751 2014-08-12 15:48:16.857428 2014-08-13 17:38:03.044879 7fe9bae887a0 20 read_log 194215'18532 (194215'18531) delete 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.2174595.0:160742 0.000000 2014-08-13 17:38:03.044890 7fe9bae887a0 20 read_log 194215'18533 (193305'18242) clean df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by osd.4.0:2753 2014-08-12 15:48:16.866341 2014-08-13 17:38:03.044923 7fe9bae887a0 20 read_log 194215'18534 (194215'18533) delete df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.2174595.0:160744 0.000000 2014-08-13 17:38:03.044934 7fe9bae887a0 20 read_log 194215'18535 (193305'18269) clean 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by osd.4.0:2755 2014-08-12 15:48:16.875215 2014-08-13 17:38:03.044948 7fe9bae887a0 20 read_log 194215'18536 (194215'18535) delete 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.2174595.0:160746 0.000000 2014-08-13 17:38:03.044959 7fe9bae887a0 20 read_log 194215'18537 (192888'18117) clean b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by osd.4.0:2757 2014-08-12 15:48:16.883755 2014-08-13 17:38:03.044971 7fe9bae887a0 20 read_log 194215'18538 (194215'18537) delete b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.2174595.0:160748 0.000000 2014-08-13 17:38:03.044982 7fe9bae887a0 20 read_log 194215'18539 (193305'18265) clean 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by osd.4.0:2759 2014-08-12 15:48:16.892363 2014-08-13 17:38:03.044994 7fe9bae887a0 20 read_log 194215'18540 (194215'18539) delete 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.2174595.0:160750 0.000000 2014-08-13 17:38:03.045005 7fe9bae887a0 20 read_log 194215'18541 (192888'17977) clean be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by osd.4.0:2761 2014-08-12 15:48:16.900886 2014-08-13 17:38:03.045020 7fe9bae887a0 20 read_log 194215'18542 (194215'18541) delete be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.2174595.0:160752 0.000000 2014-08-13 17:38:03.045031 7fe9bae887a0 20 read_log 194215'18543 (193305'18262) clean 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by osd.4.0:2763 2014-08-12 15:48:16.909571 2014-08-13 17:38:03.045043 7fe9bae887a0 20 read_log 194215'18544 (194215'18543) delete 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.2174595.0:160754 0.000000 2014-08-13 17:38:03.045055 7fe9bae887a0 20 read_log 194215'18545 (193305'18255) clean 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by osd.4.0:2765 2014-08-12 15:48:16.917487 2014-08-13 17:38:03.045067 7fe9bae887a0 20 read_log 194215'18546 (194215'18545) delete 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.2174595.0:160756 0.000000 2014-08-13 17:38:03.045078 7fe9bae887a0 20 read_log 194370'18547 (0'0) promote d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by osd.301.0:3202 2014-08-12 20:30:15.984800 2014-08-13 17:38:03.045093 7fe9bae887a0 20 read_log 194370'18548 (194370'18547) modify d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by client.1508531.0:423 2014-08-12 20:30:15.977446 2014-08-13 17:38:03.045105 7fe9bae887a0 20 read_log 194370'18549 (0'0) promote e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by osd.301.0:3222 2014-08-12 20:30:20.297306 2014-08-13 17:38:03.045128 7fe9bae887a0 20 read_log 194370'18550 (194370'18549) modify e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by client.1508531.0:1577 2014-08-12 20:30:20.290518 2014-08-13 17:38:03.045144 7fe9bae887a0 20 read_log 194370'18551 (0'0) promote a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by osd.301.0:3240 2014-08-12 20:30:22.812103 2014-08-13 17:38:03.045160 7fe9bae887a0 20 read_log 194370'18552 (194370'18551) modify a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by client.1508531.0:2246 2014-08-12 20:30:22.801067 2014-08-13 17:38:03.045175 7fe9bae887a0 20 read_log 194370'18553 (0'0) promote 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by osd.301.0:3294 2014-08-12 20:30:32.686355 2014-08-13 17:38:03.045187 7fe9bae887a0 20 read_log 194370'18554 (194370'18553) modify 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by client.1508531.0:4881 2014-08-12 20:30:32.677788 2014-08-13 17:38:03.045200 7fe9bae887a0 20 read_log 194370'18555 (0'0) promote d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by osd.301.0:3298 2014-08-12 20:30:33.102369 2014-08-13 17:38:03.045212 7fe9bae887a0 20 read_log 194370'18556 (194370'18555) modify d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by client.1508531.0:4972 2014-08-12 20:30:33.095955 2014-08-13 17:38:03.045224 7fe9bae887a0 20 read_log 194370'18557 (0'0) promote 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by osd.301.0:3314 2014-08-12 20:30:36.226278 2014-08-13 17:38:03.045236 7fe9bae887a0 20 read_log 194370'18558 (194370'18557) modify 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by client.1508531.0:5674 2014-08-12 20:30:36.218820 2014-08-13 17:38:03.045248 7fe9bae887a0 20 read_log 194370'18559 (0'0) promote 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by osd.301.0:3322 2014-08-12 20:30:37.835165 2014-08-13 17:38:03.045260 7fe9bae887a0 20 read_log 194370'18560 (194370'18559) modify 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by client.1508531.0:6024 2014-08-12 20:30:37.828080 2014-08-13 17:38:03.045275 7fe9bae887a0 20 read_log 194370'18561 (0'0) promote 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by osd.301.0:3328 2014-08-12 20:30:39.150319 2014-08-13 17:38:03.045287 7fe9bae887a0 20 read_log 194370'18562 (194370'18561) modify 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by client.1508531.0:6327 2014-08-12 20:30:39.141385 2014-08-13 17:38:03.045301 7fe9bae887a0 20 read_log 194370'18563 (0'0) promote 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by osd.301.0:3336 2014-08-12 20:30:40.679066 2014-08-13 17:38:03.045313 7fe9bae887a0 20 read_log 194370'18564 (194370'18563) modify 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by client.1508531.0:6655 2014-08-12 20:30:40.670736 2014-08-13 17:38:03.045325 7fe9bae887a0 20 read_log 194370'18565 (0'0) promote eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by osd.301.0:3342 2014-08-12 20:30:42.561288 2014-08-13 17:38:03.045339 7fe9bae887a0 20 read_log 194370'18566 (194370'18565) modify eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by client.1508531.0:7024 2014-08-12 20:30:42.552204 2014-08-13 17:38:03.045358 7fe9bae887a0 20 read_log 194370'18567 (0'0) promote a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by osd.301.0:3348 2014-08-12 20:30:43.648976 2014-08-13 17:38:03.045370 7fe9bae887a0 20 read_log 194370'18568 (194370'18567) modify a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by client.1508531.0:7259 2014-08-12 20:30:43.640862 2014-08-13 17:38:03.045384 7fe9bae887a0 20 read_log 194370'18569 (0'0) promote d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by osd.301.0:3376 2014-08-12 20:30:49.062684 2014-08-13 17:38:03.045422 7fe9bae887a0 20 read_log 194370'18570 (194370'18569) modify d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by client.1508531.0:8281 2014-08-12 20:30:49.053145 2014-08-13 17:38:03.045438 7fe9bae887a0 20 read_log 194370'18571 (0'0) promote 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by osd.301.0:3438 2014-08-12 20:31:06.635907 2014-08-13 17:38:03.045451 7fe9bae887a0 20 read_log 194370'18572 (194370'18571) modify 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by client.1508531.0:11225 2014-08-12 20:31:06.626292 2014-08-13 17:38:03.045463 7fe9bae887a0 20 read_log 194370'18573 (0'0) promote 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by osd.301.0:3456 2014-08-12 20:31:09.504784 2014-08-13 17:38:03.045475 7fe9bae887a0 20 read_log 194370'18574 (194370'18573) modify 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by client.1508531.0:11697 2014-08-12 20:31:09.495710 2014-08-13 17:38:03.045487 7fe9bae887a0 20 read_log 194370'18575 (0'0) promote aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by osd.301.0:3472 2014-08-12 20:31:12.334532 2014-08-13 17:38:03.045499 7fe9bae887a0 20 read_log 194370'18576 (194370'18575) modify aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by client.1508531.0:12132 2014-08-12 20:31:12.323754 2014-08-13 17:38:03.045511 7fe9bae887a0 20 read_log 194377'18577 (0'0) promote d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by osd.10.0:6686 2014-08-12 20:32:59.420399 2014-08-13 17:38:03.045523 7fe9bae887a0 20 read_log 194377'18578 (194377'18577) modify d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by client.2231190.0:510 2014-08-12 20:32:59.412977 2014-08-13 17:38:03.045535 7fe9bae887a0 20 read_log 194377'18579 (0'0) promote eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by osd.10.0:6690 2014-08-12 20:33:01.015336 2014-08-13 17:38:03.045566 7fe9bae887a0 20 read_log 194377'18580 (194377'18579) modify eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by client.2231190.0:923 2014-08-12 20:33:01.007967 2014-08-13 17:38:03.045580 7fe9bae887a0 20 read_log 194377'18581 (0'0) promote f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by osd.10.0:6692 2014-08-12 20:33:02.446757 2014-08-13 17:38:03.045592 7fe9bae887a0 20 read_log 194377'18582 (194377'18581) modify f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by client.2231190.0:1301 2014-08-12 20:33:02.440688 2014-08-13 17:38:03.045604 7fe9bae887a0 20 read_log 194377'18583 (0'0) promote a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by osd.10.0:6704 2014-08-12 20:33:05.327851 2014-08-13 17:38:03.045616 7fe9bae887a0 20 read_log 194377'18584 (194377'18583) modify a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by client.2231190.0:2017 2014-08-12 20:33:05.320173 2014-08-13 17:38:03.045628 7fe9bae887a0 20 read_log 194377'18585 (0'0) promote 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by osd.10.0:6710 2014-08-12 20:33:05.997041 2014-08-13 17:38:03.045640 7fe9bae887a0 20 read_log 194377'18586 (194377'18585) modify 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by client.2231190.0:2183 2014-08-12 20:33:05.991812 2014-08-13 17:38:03.045652 7fe9bae887a0 20 read_log 194377'18587 (0'0) promote 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by osd.10.0:6716 2014-08-12 20:33:08.985499 2014-08-13 17:38:03.045664 7fe9bae887a0 20 read_log 194377'18588 (194377'18587) modify 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by client.2231190.0:2921 2014-08-12 20:33:08.976531 2014-08-13 17:38:03.045676 7fe9bae887a0 20 read_log 194377'18589 (0'0) promote e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by osd.10.0:6724 2014-08-12 20:33:09.953748 2014-08-13 17:38:03.045696 7fe9bae887a0 20 read_log 194377'18590 (194377'18589) modify e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by client.2231190.0:3165 2014-08-12 20:33:09.942727 2014-08-13 17:38:03.045709 7fe9bae887a0 20 read_log 194378'18591 (0'0) promote 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by osd.10.0:6738 2014-08-12 20:33:20.419690 2014-08-13 17:38:03.045721 7fe9bae887a0 20 read_log 194378'18592 (194378'18591) modify 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by client.2231190.0:4862 2014-08-12 20:33:20.412102 2014-08-13 17:38:03.045733 7fe9bae887a0 20 read_log 194380'18593 (0'0) promote 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by osd.10.0:6744 2014-08-12 20:33:29.996469 2014-08-13 17:38:03.045745 7fe9bae887a0 20 read_log 194380'18594 (194380'18593) modify 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by client.2231190.0:5507 2014-08-12 20:33:29.989100 2014-08-13 17:38:03.045757 7fe9bae887a0 20 read_log 194393'18595 (194393'18595) modify 30b/hit_set_24.30b_archive_2014-08-12 20:37:09.053178_2014-08-12 20:49:27.261750/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.261985 2014-08-13 17:38:03.045770 7fe9bae887a0 20 read_log 194393'18596 (194215'18277) delete 30b/hit_set_24.30b_archive_2014-08-12 15:32:25.863655_2014-08-12 15:48:15.603804/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.261985 2014-08-13 17:38:03.045782 7fe9bae887a0 20 read_log 194393'18597 (0'0) promote ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by osd.301.0:3685 2014-08-12 20:49:27.265309 2014-08-13 17:38:03.045794 7fe9bae887a0 20 read_log 194393'18598 (194393'18597) modify ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by client.1064182.0:382 2014-08-12 20:49:27.257464 2014-08-13 17:38:03.045806 7fe9bae887a0 20 read_log 194396'18599 (0'0) promote d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by osd.301.0:3715 2014-08-12 20:53:00.628865 2014-08-13 17:38:03.045818 7fe9bae887a0 20 read_log 194396'18600 (194396'18599) modify d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by client.2235069.0:364 2014-08-12 20:53:00.620088 2014-08-13 17:38:03.045831 7fe9bae887a0 20 read_log 194416'18601 (194416'18601) modify 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 17:38:03.045843 7fe9bae887a0 20 read_log 194416'18602 (194393'18595) delete 30b/hit_set_24.30b_archive_2014-08-12 20:37:09.053178_2014-08-12 20:49:27.261750/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 17:38:03.045855 7fe9bae887a0 20 read_log 194416'18603 (0'0) promote c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by osd.301.0:3867 2014-08-13 15:28:34.596370 2014-08-13 17:38:03.045867 7fe9bae887a0 20 read_log 194416'18604 (194416'18603) modify c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by client.2441958.0:1020 2014-08-13 15:28:34.585941 2014-08-13 17:38:03.045879 7fe9bae887a0 20 read_log 194416'18605 (0'0) promote 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by osd.301.0:3897 2014-08-13 15:28:38.704066 2014-08-13 17:38:03.045891 7fe9bae887a0 20 read_log 194416'18606 (194416'18605) modify 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by client.2441958.0:2112 2014-08-13 15:28:38.696406 2014-08-13 17:38:03.045903 7fe9bae887a0 20 read_log 194416'18607 (0'0) promote ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by osd.301.0:3905 2014-08-13 15:28:40.400240 2014-08-13 17:38:03.045922 7fe9bae887a0 20 read_log 194416'18608 (194416'18607) modify ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by client.2441958.0:2555 2014-08-13 15:28:40.388566 2014-08-13 17:38:03.045935 7fe9bae887a0 20 read_log 194416'18609 (0'0) promote a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by osd.301.0:3949 2014-08-13 15:28:48.479236 2014-08-13 17:38:03.045947 7fe9bae887a0 20 read_log 194416'18610 (194416'18609) modify a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by client.2441958.0:4625 2014-08-13 15:28:48.470757 2014-08-13 17:38:03.045959 7fe9bae887a0 20 read_log 194416'18611 (0'0) promote 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by osd.301.0:3951 2014-08-13 15:28:48.606666 2014-08-13 17:38:03.045971 7fe9bae887a0 20 read_log 194416'18612 (194416'18611) modify 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by client.2441958.0:4659 2014-08-13 15:28:48.598232 2014-08-13 17:38:03.045983 7fe9bae887a0 20 read_log 194416'18613 (0'0) promote 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by osd.301.0:3991 2014-08-13 15:28:55.397809 2014-08-13 17:38:03.045995 7fe9bae887a0 20 read_log 194416'18614 (194416'18613) modify 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by client.2441958.0:6372 2014-08-13 15:28:55.388244 2014-08-13 17:38:03.046008 7fe9bae887a0 20 read_log 194416'18615 (0'0) promote 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by osd.301.0:4055 2014-08-13 15:29:07.624257 2014-08-13 17:38:03.046020 7fe9bae887a0 20 read_log 194416'18616 (194416'18615) modify 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by client.2441958.0:9562 2014-08-13 15:29:07.616896 2014-08-13 17:38:03.046032 7fe9bae887a0 20 read_log 194416'18617 (0'0) promote f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by osd.301.0:4067 2014-08-13 15:29:11.323561 2014-08-13 17:38:03.046044 7fe9bae887a0 20 read_log 194416'18618 (194416'18617) modify f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by client.2441958.0:10469 2014-08-13 15:29:11.314182 2014-08-13 17:38:03.046056 7fe9bae887a0 20 read_log 194416'18619 (0'0) promote 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by osd.301.0:4071 2014-08-13 15:29:11.821492 2014-08-13 17:38:03.046068 7fe9bae887a0 20 read_log 194416'18620 (194416'18619) modify 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by client.2441958.0:10603 2014-08-13 15:29:11.812825 2014-08-13 17:38:03.046080 7fe9bae887a0 20 read_log 194416'18621 (0'0) promote 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by osd.301.0:4093 2014-08-13 15:29:14.872061 2014-08-13 17:38:03.046092 7fe9bae887a0 20 read_log 194416'18622 (194416'18621) modify 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by client.2441958.0:11350 2014-08-13 15:29:14.864431 2014-08-13 17:38:03.046104 7fe9bae887a0 20 read_log 194416'18623 (0'0) promote d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by osd.301.0:4147 2014-08-13 15:29:29.751060 2014-08-13 17:38:03.046116 7fe9bae887a0 20 read_log 194416'18624 (194416'18623) modify d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by client.2441958.0:14878 2014-08-13 15:29:29.742414 2014-08-13 17:38:03.046128 7fe9bae887a0 20 read_log 194416'18625 (0'0) promote 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by osd.301.0:4159 2014-08-13 15:29:30.717320 2014-08-13 17:38:03.046140 7fe9bae887a0 20 read_log 194416'18626 (194416'18625) modify 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by client.2441958.0:15097 2014-08-13 15:29:30.708981 2014-08-13 17:38:03.046152 7fe9bae887a0 20 read_log 194416'18627 (0'0) promote ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by osd.301.0:4173 2014-08-13 15:29:33.685595 2014-08-13 17:38:03.046180 7fe9bae887a0 20 read_log 194416'18628 (194416'18627) modify ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by client.2441958.0:15685 2014-08-13 15:29:33.676427 2014-08-13 17:38:03.046193 7fe9bae887a0 20 read_log 194416'18629 (0'0) promote 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by osd.301.0:4189 2014-08-13 15:29:45.860462 2014-08-13 17:38:03.046205 7fe9bae887a0 20 read_log 194416'18630 (194416'18629) modify 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by client.2441958.0:16896 2014-08-13 15:29:45.851032 2014-08-13 17:38:03.046217 7fe9bae887a0 20 read_log 194416'18631 (0'0) promote 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by osd.301.0:4203 2014-08-13 15:29:55.046533 2014-08-13 17:38:03.046230 7fe9bae887a0 20 read_log 194416'18632 (194416'18631) modify 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by client.2441958.0:17543 2014-08-13 15:29:55.037173 2014-08-13 17:38:03.046242 7fe9bae887a0 20 read_log 194416'18633 (0'0) promote 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by osd.301.0:4209 2014-08-13 15:30:09.294520 2014-08-13 17:38:03.046254 7fe9bae887a0 20 read_log 194416'18634 (194416'18633) modify 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by client.2441958.0:18363 2014-08-13 15:30:09.286116 2014-08-13 17:38:03.046266 7fe9bae887a0 20 read_log 194416'18635 (0'0) promote 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by osd.301.0:4217 2014-08-13 15:30:15.202330 2014-08-13 17:38:03.046278 7fe9bae887a0 20 read_log 194416'18636 (194416'18635) modify 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by client.2441958.0:18577 2014-08-13 15:30:15.193629 2014-08-13 17:38:03.046290 7fe9bae887a0 20 read_log 194422'18637 (0'0) promote eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by osd.301.0:4233 2014-08-13 15:32:16.926044 2014-08-13 17:38:03.046302 7fe9bae887a0 20 read_log 194422'18638 (194422'18637) modify eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by client.1148203.0:420 2014-08-13 15:32:16.918548 2014-08-13 17:38:03.046314 7fe9bae887a0 20 read_log 194422'18639 (0'0) promote b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by osd.301.0:4257 2014-08-13 15:32:23.686723 2014-08-13 17:38:03.046326 7fe9bae887a0 20 read_log 194422'18640 (194422'18639) modify b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by client.1148203.0:2118 2014-08-13 15:32:23.675230 2014-08-13 17:38:03.046338 7fe9bae887a0 20 read_log 194422'18641 (0'0) promote bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by osd.301.0:4281 2014-08-13 15:32:29.929305 2014-08-13 17:38:03.046350 7fe9bae887a0 20 read_log 194422'18642 (194422'18641) modify bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by client.1148203.0:3716 2014-08-13 15:32:29.921559 2014-08-13 17:38:03.046362 7fe9bae887a0 20 read_log 194422'18643 (0'0) promote 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by osd.301.0:4315 2014-08-13 15:32:35.203109 2014-08-13 17:38:03.046374 7fe9bae887a0 20 read_log 194422'18644 (194422'18643) modify 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by client.1148203.0:5079 2014-08-13 15:32:35.194308 2014-08-13 17:38:03.046386 7fe9bae887a0 20 read_log 194422'18645 (0'0) promote 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by osd.301.0:4345 2014-08-13 15:32:44.256364 2014-08-13 17:38:03.046398 7fe9bae887a0 20 read_log 194422'18646 (194422'18645) modify 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by client.1148203.0:6766 2014-08-13 15:32:44.248958 2014-08-13 17:38:03.046410 7fe9bae887a0 20 read_log 194422'18647 (0'0) promote 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by osd.301.0:4347 2014-08-13 15:32:45.107587 2014-08-13 17:38:03.046423 7fe9bae887a0 20 read_log 194422'18648 (194422'18647) modify 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by client.1148203.0:6938 2014-08-13 15:32:45.099903 2014-08-13 17:38:03.046435 7fe9bae887a0 20 read_log 194422'18649 (0'0) promote 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by osd.301.0:4363 2014-08-13 15:32:47.484346 2014-08-13 17:38:03.046447 7fe9bae887a0 20 read_log 194422'18650 (194422'18649) modify 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by client.1148203.0:7320 2014-08-13 15:32:47.475508 2014-08-13 17:38:03.046459 7fe9bae887a0 20 read_log 194422'18651 (0'0) promote eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by osd.301.0:4383 2014-08-13 15:32:53.769159 2014-08-13 17:38:03.046471 7fe9bae887a0 20 read_log 194422'18652 (194422'18651) modify eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by client.1148203.0:8324 2014-08-13 15:32:53.760637 2014-08-13 17:38:03.046483 7fe9bae887a0 20 read_log 194422'18653 (0'0) promote 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by osd.301.0:4389 2014-08-13 15:32:54.756341 2014-08-13 17:38:03.046495 7fe9bae887a0 20 read_log 194422'18654 (194422'18653) modify 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by client.1148203.0:8452 2014-08-13 15:32:54.747584 2014-08-13 17:38:03.046507 7fe9bae887a0 20 read_log 194422'18655 (0'0) promote e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by osd.301.0:4401 2014-08-13 15:33:04.760428 2014-08-13 17:38:03.046519 7fe9bae887a0 20 read_log 194422'18656 (194422'18655) modify e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by client.1148203.0:9529 2014-08-13 15:33:04.752691 2014-08-13 17:38:03.046531 7fe9bae887a0 20 read_log 194422'18657 (0'0) promote c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by osd.301.0:4405 2014-08-13 15:33:06.256675 2014-08-13 17:38:03.046543 7fe9bae887a0 20 read_log 194422'18658 (194422'18657) modify c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by client.1148203.0:9685 2014-08-13 15:33:06.246976 2014-08-13 17:38:03.046555 7fe9bae887a0 20 read_log 194422'18659 (0'0) promote c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by osd.301.0:4413 2014-08-13 15:33:07.054088 2014-08-13 17:38:03.046567 7fe9bae887a0 20 read_log 194422'18660 (194422'18659) modify c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by client.1148203.0:9743 2014-08-13 15:33:07.044878 2014-08-13 17:38:03.046582 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.047191 7fe9bae887a0 10 read_log checking for missing items over interval (194370'18547,194422'18660] 2014-08-13 17:38:03.047240 7fe9bae887a0 15 read_log missing 194422'18660 (194422'18659) modify c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by client.1148203.0:9743 2014-08-13 15:33:07.044878 2014-08-13 17:38:03.047281 7fe9bae887a0 15 read_log missing 194422'18658 (194422'18657) modify c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by client.1148203.0:9685 2014-08-13 15:33:06.246976 2014-08-13 17:38:03.047318 7fe9bae887a0 15 read_log missing 194422'18656 (194422'18655) modify e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by client.1148203.0:9529 2014-08-13 15:33:04.752691 2014-08-13 17:38:03.047355 7fe9bae887a0 15 read_log missing 194422'18654 (194422'18653) modify 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by client.1148203.0:8452 2014-08-13 15:32:54.747584 2014-08-13 17:38:03.047391 7fe9bae887a0 15 read_log missing 194422'18652 (194422'18651) modify eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by client.1148203.0:8324 2014-08-13 15:32:53.760637 2014-08-13 17:38:03.047428 7fe9bae887a0 15 read_log missing 194422'18650 (194422'18649) modify 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by client.1148203.0:7320 2014-08-13 15:32:47.475508 2014-08-13 17:38:03.047464 7fe9bae887a0 15 read_log missing 194422'18648 (194422'18647) modify 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by client.1148203.0:6938 2014-08-13 15:32:45.099903 2014-08-13 17:38:03.047500 7fe9bae887a0 15 read_log missing 194422'18646 (194422'18645) modify 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by client.1148203.0:6766 2014-08-13 15:32:44.248958 2014-08-13 17:38:03.047536 7fe9bae887a0 15 read_log missing 194422'18644 (194422'18643) modify 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by client.1148203.0:5079 2014-08-13 15:32:35.194308 2014-08-13 17:38:03.047573 7fe9bae887a0 15 read_log missing 194422'18642 (194422'18641) modify bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by client.1148203.0:3716 2014-08-13 15:32:29.921559 2014-08-13 17:38:03.047609 7fe9bae887a0 15 read_log missing 194422'18640 (194422'18639) modify b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by client.1148203.0:2118 2014-08-13 15:32:23.675230 2014-08-13 17:38:03.047645 7fe9bae887a0 15 read_log missing 194422'18638 (194422'18637) modify eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by client.1148203.0:420 2014-08-13 15:32:16.918548 2014-08-13 17:38:03.047682 7fe9bae887a0 15 read_log missing 194416'18636 (194416'18635) modify 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by client.2441958.0:18577 2014-08-13 15:30:15.193629 2014-08-13 17:38:03.047718 7fe9bae887a0 15 read_log missing 194416'18634 (194416'18633) modify 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by client.2441958.0:18363 2014-08-13 15:30:09.286116 2014-08-13 17:38:03.047754 7fe9bae887a0 15 read_log missing 194416'18632 (194416'18631) modify 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by client.2441958.0:17543 2014-08-13 15:29:55.037173 2014-08-13 17:38:03.047792 7fe9bae887a0 15 read_log missing 194416'18630 (194416'18629) modify 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by client.2441958.0:16896 2014-08-13 15:29:45.851032 2014-08-13 17:38:03.047828 7fe9bae887a0 15 read_log missing 194416'18628 (194416'18627) modify ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by client.2441958.0:15685 2014-08-13 15:29:33.676427 2014-08-13 17:38:03.047865 7fe9bae887a0 15 read_log missing 194416'18626 (194416'18625) modify 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by client.2441958.0:15097 2014-08-13 15:29:30.708981 2014-08-13 17:38:03.047902 7fe9bae887a0 15 read_log missing 194416'18624 (194416'18623) modify d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by client.2441958.0:14878 2014-08-13 15:29:29.742414 2014-08-13 17:38:03.047938 7fe9bae887a0 15 read_log missing 194416'18622 (194416'18621) modify 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by client.2441958.0:11350 2014-08-13 15:29:14.864431 2014-08-13 17:38:03.047975 7fe9bae887a0 15 read_log missing 194416'18620 (194416'18619) modify 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by client.2441958.0:10603 2014-08-13 15:29:11.812825 2014-08-13 17:38:03.048011 7fe9bae887a0 15 read_log missing 194416'18618 (194416'18617) modify f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by client.2441958.0:10469 2014-08-13 15:29:11.314182 2014-08-13 17:38:03.048048 7fe9bae887a0 15 read_log missing 194416'18616 (194416'18615) modify 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by client.2441958.0:9562 2014-08-13 15:29:07.616896 2014-08-13 17:38:03.048084 7fe9bae887a0 15 read_log missing 194416'18614 (194416'18613) modify 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by client.2441958.0:6372 2014-08-13 15:28:55.388244 2014-08-13 17:38:03.048121 7fe9bae887a0 15 read_log missing 194416'18612 (194416'18611) modify 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by client.2441958.0:4659 2014-08-13 15:28:48.598232 2014-08-13 17:38:03.048163 7fe9bae887a0 15 read_log missing 194416'18610 (194416'18609) modify a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by client.2441958.0:4625 2014-08-13 15:28:48.470757 2014-08-13 17:38:03.048202 7fe9bae887a0 15 read_log missing 194416'18608 (194416'18607) modify ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by client.2441958.0:2555 2014-08-13 15:28:40.388566 2014-08-13 17:38:03.048239 7fe9bae887a0 15 read_log missing 194416'18606 (194416'18605) modify 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by client.2441958.0:2112 2014-08-13 15:28:38.696406 2014-08-13 17:38:03.048295 7fe9bae887a0 15 read_log missing 194416'18604 (194416'18603) modify c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by client.2441958.0:1020 2014-08-13 15:28:34.585941 2014-08-13 17:38:03.048338 7fe9bae887a0 15 read_log missing 194416'18601 (194416'18601) modify 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 17:38:03.048374 7fe9bae887a0 15 read_log missing 194396'18600 (194396'18599) modify d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by client.2235069.0:364 2014-08-12 20:53:00.620088 2014-08-13 17:38:03.048411 7fe9bae887a0 15 read_log missing 194393'18598 (194393'18597) modify ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by client.1064182.0:382 2014-08-12 20:49:27.257464 2014-08-13 17:38:03.048452 7fe9bae887a0 15 read_log missing 194380'18594 (194380'18593) modify 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by client.2231190.0:5507 2014-08-12 20:33:29.989100 2014-08-13 17:38:03.048489 7fe9bae887a0 15 read_log missing 194378'18592 (194378'18591) modify 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by client.2231190.0:4862 2014-08-12 20:33:20.412102 2014-08-13 17:38:03.048526 7fe9bae887a0 15 read_log missing 194377'18590 (194377'18589) modify e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by client.2231190.0:3165 2014-08-12 20:33:09.942727 2014-08-13 17:38:03.048563 7fe9bae887a0 15 read_log missing 194377'18588 (194377'18587) modify 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by client.2231190.0:2921 2014-08-12 20:33:08.976531 2014-08-13 17:38:03.048600 7fe9bae887a0 15 read_log missing 194377'18586 (194377'18585) modify 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by client.2231190.0:2183 2014-08-12 20:33:05.991812 2014-08-13 17:38:03.048637 7fe9bae887a0 15 read_log missing 194377'18584 (194377'18583) modify a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by client.2231190.0:2017 2014-08-12 20:33:05.320173 2014-08-13 17:38:03.048674 7fe9bae887a0 15 read_log missing 194377'18582 (194377'18581) modify f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by client.2231190.0:1301 2014-08-12 20:33:02.440688 2014-08-13 17:38:03.048710 7fe9bae887a0 15 read_log missing 194377'18580 (194377'18579) modify eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by client.2231190.0:923 2014-08-12 20:33:01.007967 2014-08-13 17:38:03.048747 7fe9bae887a0 15 read_log missing 194377'18578 (194377'18577) modify d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by client.2231190.0:510 2014-08-12 20:32:59.412977 2014-08-13 17:38:03.048783 7fe9bae887a0 15 read_log missing 194370'18576 (194370'18575) modify aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by client.1508531.0:12132 2014-08-12 20:31:12.323754 2014-08-13 17:38:03.048820 7fe9bae887a0 15 read_log missing 194370'18574 (194370'18573) modify 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by client.1508531.0:11697 2014-08-12 20:31:09.495710 2014-08-13 17:38:03.048856 7fe9bae887a0 15 read_log missing 194370'18572 (194370'18571) modify 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by client.1508531.0:11225 2014-08-12 20:31:06.626292 2014-08-13 17:38:03.048893 7fe9bae887a0 15 read_log missing 194370'18570 (194370'18569) modify d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by client.1508531.0:8281 2014-08-12 20:30:49.053145 2014-08-13 17:38:03.048929 7fe9bae887a0 15 read_log missing 194370'18568 (194370'18567) modify a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by client.1508531.0:7259 2014-08-12 20:30:43.640862 2014-08-13 17:38:03.048966 7fe9bae887a0 15 read_log missing 194370'18566 (194370'18565) modify eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by client.1508531.0:7024 2014-08-12 20:30:42.552204 2014-08-13 17:38:03.049003 7fe9bae887a0 15 read_log missing 194370'18564 (194370'18563) modify 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by client.1508531.0:6655 2014-08-12 20:30:40.670736 2014-08-13 17:38:03.049041 7fe9bae887a0 15 read_log missing 194370'18562 (194370'18561) modify 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by client.1508531.0:6327 2014-08-12 20:30:39.141385 2014-08-13 17:38:03.049077 7fe9bae887a0 15 read_log missing 194370'18560 (194370'18559) modify 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by client.1508531.0:6024 2014-08-12 20:30:37.828080 2014-08-13 17:38:03.049114 7fe9bae887a0 15 read_log missing 194370'18558 (194370'18557) modify 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by client.1508531.0:5674 2014-08-12 20:30:36.218820 2014-08-13 17:38:03.049150 7fe9bae887a0 15 read_log missing 194370'18556 (194370'18555) modify d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by client.1508531.0:4972 2014-08-12 20:30:33.095955 2014-08-13 17:38:03.049193 7fe9bae887a0 15 read_log missing 194370'18554 (194370'18553) modify 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by client.1508531.0:4881 2014-08-12 20:30:32.677788 2014-08-13 17:38:03.049230 7fe9bae887a0 15 read_log missing 194370'18552 (194370'18551) modify a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by client.1508531.0:2246 2014-08-12 20:30:22.801067 2014-08-13 17:38:03.049267 7fe9bae887a0 15 read_log missing 194370'18550 (194370'18549) modify e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by client.1508531.0:1577 2014-08-12 20:30:20.290518 2014-08-13 17:38:03.049305 7fe9bae887a0 15 read_log missing 194370'18548 (194370'18547) modify d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by client.1508531.0:423 2014-08-12 20:30:15.977446 2014-08-13 17:38:03.049316 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.049354 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_loaded 2014-08-13 17:38:03.049373 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Initial 0.021885 0 0.000000 2014-08-13 17:38:03.049385 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset 2014-08-13 17:38:03.049395 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194478 2014-08-13 17:38:03.049405 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] log((192718'15573,194422'18660], crt=194422'18660) 2014-08-13 17:38:03.049415 7fe9bae887a0 10 osd.4 194478 pgid 24.33a coll 24.33a_head 2014-08-13 17:38:03.049525 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.33a 2014-08-13 17:38:03.049538 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.049579 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.33a(unlocked)] enter Initial 2014-08-13 17:38:03.049591 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.33a(unlocked)] enter NotTrimming 2014-08-13 17:38:03.049720 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.049816 7fe9bae887a0 20 read_log 192834'14162 (192834'14161) modify 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.933448.0:1124 2014-08-11 15:23:40.014770 2014-08-13 17:38:03.049836 7fe9bae887a0 20 read_log 192834'14187 (0'0) promote 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by osd.285.0:15089 2014-08-11 15:23:45.523176 2014-08-13 17:38:03.049850 7fe9bae887a0 20 read_log 192834'14188 (192834'14187) modify 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.933448.0:2197 2014-08-11 15:23:45.514326 2014-08-13 17:38:03.049862 7fe9bae887a0 20 read_log 192834'14189 (0'0) promote fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by osd.285.0:15091 2014-08-11 15:23:45.601818 2014-08-13 17:38:03.049875 7fe9bae887a0 20 read_log 192834'14190 (192834'14189) modify fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.933448.0:2211 2014-08-11 15:23:45.592825 2014-08-13 17:38:03.049887 7fe9bae887a0 20 read_log 192834'14222 (0'0) promote 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by osd.285.0:15124 2014-08-11 15:23:59.605777 2014-08-13 17:38:03.049899 7fe9bae887a0 20 read_log 192834'14225 (192834'14222) modify 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.933448.0:4945 2014-08-11 15:23:59.429020 2014-08-13 17:38:03.049931 7fe9bae887a0 20 read_log 192834'14233 (0'0) promote 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by osd.285.0:15135 2014-08-11 15:24:02.302393 2014-08-13 17:38:03.049947 7fe9bae887a0 20 read_log 192834'14234 (192834'14233) modify 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.933448.0:5633 2014-08-11 15:24:02.294951 2014-08-13 17:38:03.049960 7fe9bae887a0 20 read_log 192834'14235 (0'0) promote f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by osd.285.0:15137 2014-08-11 15:24:03.045645 2014-08-13 17:38:03.049972 7fe9bae887a0 20 read_log 192834'14236 (192834'14235) modify f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.933448.0:5830 2014-08-11 15:24:03.037968 2014-08-13 17:38:03.049995 7fe9bae887a0 20 read_log 192834'14267 (0'0) promote 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by osd.285.0:15169 2014-08-11 15:24:12.234007 2014-08-13 17:38:03.050010 7fe9bae887a0 20 read_log 192834'14268 (192834'14267) modify 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.933448.0:8004 2014-08-11 15:24:12.226134 2014-08-13 17:38:03.050022 7fe9bae887a0 20 read_log 192834'14271 (0'0) promote 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by osd.285.0:15173 2014-08-11 15:24:13.386656 2014-08-13 17:38:03.050034 7fe9bae887a0 20 read_log 192834'14272 (192834'14271) modify 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.933448.0:8274 2014-08-11 15:24:13.380030 2014-08-13 17:38:03.050046 7fe9bae887a0 20 read_log 192834'14293 (0'0) promote 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by osd.285.0:15195 2014-08-11 15:24:18.366663 2014-08-13 17:38:03.050058 7fe9bae887a0 20 read_log 192834'14294 (192834'14293) modify 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.933448.0:9310 2014-08-11 15:24:18.359061 2014-08-13 17:38:03.050071 7fe9bae887a0 20 read_log 192834'14297 (0'0) promote f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by osd.285.0:15199 2014-08-11 15:24:19.581398 2014-08-13 17:38:03.050083 7fe9bae887a0 20 read_log 192834'14298 (192834'14297) modify f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.933448.0:9623 2014-08-11 15:24:19.573092 2014-08-13 17:38:03.050095 7fe9bae887a0 20 read_log 192834'14301 (0'0) promote 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by osd.285.0:15203 2014-08-11 15:24:21.738710 2014-08-13 17:38:03.050107 7fe9bae887a0 20 read_log 192834'14302 (192834'14301) modify 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.933448.0:10111 2014-08-11 15:24:21.730563 2014-08-13 17:38:03.050119 7fe9bae887a0 20 read_log 192834'14307 (0'0) promote d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by osd.285.0:15209 2014-08-11 15:24:24.854277 2014-08-13 17:38:03.050131 7fe9bae887a0 20 read_log 192834'14308 (192834'14307) modify d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.933448.0:10689 2014-08-11 15:24:24.846797 2014-08-13 17:38:03.050143 7fe9bae887a0 20 read_log 192834'14315 (0'0) promote f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by osd.285.0:15217 2014-08-11 15:24:25.363711 2014-08-13 17:38:03.050162 7fe9bae887a0 20 read_log 192834'14316 (192834'14315) modify f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.933448.0:10820 2014-08-11 15:24:25.356541 2014-08-13 17:38:03.050177 7fe9bae887a0 20 read_log 192834'14355 (0'0) promote 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by osd.285.0:15257 2014-08-11 15:24:35.458767 2014-08-13 17:38:03.050189 7fe9bae887a0 20 read_log 192834'14356 (192834'14355) modify 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.933448.0:13118 2014-08-11 15:24:35.449703 2014-08-13 17:38:03.050202 7fe9bae887a0 20 read_log 192834'14367 (0'0) promote c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by osd.285.0:15269 2014-08-11 15:24:46.127342 2014-08-13 17:38:03.050214 7fe9bae887a0 20 read_log 192834'14368 (192834'14367) modify c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.933448.0:14551 2014-08-11 15:24:46.119345 2014-08-13 17:38:03.050226 7fe9bae887a0 20 read_log 192834'14373 (0'0) promote 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by osd.285.0:15275 2014-08-11 15:24:46.298819 2014-08-13 17:38:03.050240 7fe9bae887a0 20 read_log 192834'14374 (192834'14373) modify 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.933448.0:14592 2014-08-11 15:24:46.288790 2014-08-13 17:38:03.050264 7fe9bae887a0 20 read_log 192834'14425 (0'0) promote 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by osd.285.0:15327 2014-08-11 15:24:58.358542 2014-08-13 17:38:03.050278 7fe9bae887a0 20 read_log 192834'14426 (192834'14425) modify 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.933448.0:17077 2014-08-11 15:24:58.350293 2014-08-13 17:38:03.050291 7fe9bae887a0 20 read_log 192834'14451 (0'0) promote b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by osd.285.0:15353 2014-08-11 15:25:06.581086 2014-08-13 17:38:03.050303 7fe9bae887a0 20 read_log 192834'14452 (192834'14451) modify b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.933448.0:18734 2014-08-11 15:25:06.570785 2014-08-13 17:38:03.050317 7fe9bae887a0 20 read_log 192834'14487 (0'0) promote 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by osd.285.0:15389 2014-08-11 15:25:21.338689 2014-08-13 17:38:03.050329 7fe9bae887a0 20 read_log 192834'14488 (192834'14487) modify 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.933448.0:20524 2014-08-11 15:25:21.329408 2014-08-13 17:38:03.050344 7fe9bae887a0 20 read_log 192834'14491 (0'0) promote 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by osd.285.0:15393 2014-08-11 15:25:22.162973 2014-08-13 17:38:03.050356 7fe9bae887a0 20 read_log 192834'14492 (192834'14491) modify 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.933448.0:20611 2014-08-11 15:25:22.152890 2014-08-13 17:38:03.050370 7fe9bae887a0 20 read_log 192834'14518 (192834'14160) modify e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by client.933448.0:23383 2014-08-11 15:25:39.294057 2014-08-13 17:38:03.050382 7fe9bae887a0 20 read_log 192834'14519 (192834'14162) modify 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.933448.0:23415 2014-08-11 15:25:39.329813 2014-08-13 17:38:03.050395 7fe9bae887a0 20 read_log 192834'14532 (192834'14188) modify 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.933448.0:24488 2014-08-11 15:25:40.937228 2014-08-13 17:38:03.050407 7fe9bae887a0 20 read_log 192834'14533 (192834'14190) modify fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.933448.0:24502 2014-08-11 15:25:40.962331 2014-08-13 17:38:03.050419 7fe9bae887a0 20 read_log 192834'14550 (192834'14225) modify 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.933448.0:27236 2014-08-11 15:25:44.261666 2014-08-13 17:38:03.050432 7fe9bae887a0 20 read_log 192834'14555 (192834'14234) modify 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.933448.0:27924 2014-08-11 15:25:45.267745 2014-08-13 17:38:03.050444 7fe9bae887a0 20 read_log 192834'14556 (192834'14236) modify f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.933448.0:28121 2014-08-11 15:25:45.527785 2014-08-13 17:38:03.050459 7fe9bae887a0 20 read_log 192834'14572 (192834'14268) modify 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.933448.0:30295 2014-08-11 15:25:48.696166 2014-08-13 17:38:03.050471 7fe9bae887a0 20 read_log 192834'14574 (192834'14272) modify 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.933448.0:30565 2014-08-11 15:25:49.000389 2014-08-13 17:38:03.050484 7fe9bae887a0 20 read_log 192834'14585 (192834'14294) modify 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.933448.0:31601 2014-08-11 15:25:50.259247 2014-08-13 17:38:03.050499 7fe9bae887a0 20 read_log 192834'14587 (192834'14298) modify f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.933448.0:31914 2014-08-11 15:25:50.792946 2014-08-13 17:38:03.050511 7fe9bae887a0 20 read_log 192834'14589 (192834'14302) modify 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.933448.0:32402 2014-08-11 15:25:51.734472 2014-08-13 17:38:03.050534 7fe9bae887a0 20 read_log 192834'14592 (192834'14308) modify d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.933448.0:32980 2014-08-11 15:25:52.710888 2014-08-13 17:38:03.050548 7fe9bae887a0 20 read_log 192834'14596 (192834'14316) modify f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.933448.0:33111 2014-08-11 15:25:52.855817 2014-08-13 17:38:03.050561 7fe9bae887a0 20 read_log 192834'14616 (192834'14356) modify 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.933448.0:35409 2014-08-11 15:25:56.219556 2014-08-13 17:38:03.050573 7fe9bae887a0 20 read_log 192834'14622 (192834'14368) modify c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.933448.0:36842 2014-08-11 15:25:58.362770 2014-08-13 17:38:03.050587 7fe9bae887a0 20 read_log 192834'14625 (192834'14374) modify 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.933448.0:36883 2014-08-11 15:25:58.435189 2014-08-13 17:38:03.050599 7fe9bae887a0 20 read_log 192834'14651 (192834'14426) modify 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.933448.0:39368 2014-08-11 15:26:01.827325 2014-08-13 17:38:03.050612 7fe9bae887a0 20 read_log 192834'14664 (192834'14452) modify b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.933448.0:41025 2014-08-11 15:26:02.818611 2014-08-13 17:38:03.050626 7fe9bae887a0 20 read_log 192834'14682 (192834'14488) modify 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.933448.0:42815 2014-08-11 15:26:03.396706 2014-08-13 17:38:03.050638 7fe9bae887a0 20 read_log 192834'14684 (192834'14492) modify 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.933448.0:42902 2014-08-11 15:26:03.428755 2014-08-13 17:38:03.050653 7fe9bae887a0 20 read_log 192834'14707 (0'0) promote 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by osd.285.0:15609 2014-08-11 15:26:16.709175 2014-08-13 17:38:03.050665 7fe9bae887a0 20 read_log 192834'14708 (192834'14707) modify 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.1883634.0:608 2014-08-11 15:26:16.702233 2014-08-13 17:38:03.050678 7fe9bae887a0 20 read_log 192834'14709 (0'0) promote 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by osd.285.0:15611 2014-08-11 15:26:17.058838 2014-08-13 17:38:03.050690 7fe9bae887a0 20 read_log 192834'14710 (192834'14709) modify 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.1883634.0:705 2014-08-11 15:26:17.050502 2014-08-13 17:38:03.050702 7fe9bae887a0 20 read_log 192834'14717 (0'0) promote 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by osd.285.0:15619 2014-08-11 15:26:20.496379 2014-08-13 17:38:03.050715 7fe9bae887a0 20 read_log 192834'14718 (192834'14717) modify 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.1883634.0:1655 2014-08-11 15:26:20.488970 2014-08-13 17:38:03.050727 7fe9bae887a0 20 read_log 192834'14719 (0'0) promote b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by osd.285.0:15621 2014-08-11 15:26:20.517988 2014-08-13 17:38:03.050742 7fe9bae887a0 20 read_log 192834'14720 (192834'14719) modify b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.1883634.0:1661 2014-08-11 15:26:20.510842 2014-08-13 17:38:03.050754 7fe9bae887a0 20 read_log 192834'14721 (0'0) promote 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by osd.285.0:15623 2014-08-11 15:26:20.634423 2014-08-13 17:38:03.050768 7fe9bae887a0 20 read_log 192834'14722 (192834'14721) modify 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.1883634.0:1693 2014-08-11 15:26:20.626192 2014-08-13 17:38:03.050781 7fe9bae887a0 20 read_log 192834'14753 (0'0) promote f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by osd.285.0:15655 2014-08-11 15:26:28.217223 2014-08-13 17:38:03.050821 7fe9bae887a0 20 read_log 192834'14754 (192834'14753) modify f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.1883634.0:3780 2014-08-11 15:26:28.209413 2014-08-13 17:38:03.050837 7fe9bae887a0 20 read_log 192834'14757 (0'0) promote 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by osd.285.0:15659 2014-08-11 15:26:29.202101 2014-08-13 17:38:03.050849 7fe9bae887a0 20 read_log 192834'14758 (192834'14757) modify 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.1883634.0:4058 2014-08-11 15:26:29.193426 2014-08-13 17:38:03.050863 7fe9bae887a0 20 read_log 192834'14763 (0'0) promote 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by osd.285.0:15665 2014-08-11 15:26:33.201305 2014-08-13 17:38:03.050875 7fe9bae887a0 20 read_log 192834'14764 (192834'14763) modify 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.1883634.0:5095 2014-08-11 15:26:33.193287 2014-08-13 17:38:03.050887 7fe9bae887a0 20 read_log 192834'14765 (0'0) promote f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by osd.285.0:15667 2014-08-11 15:26:33.733870 2014-08-13 17:38:03.050899 7fe9bae887a0 20 read_log 192834'14766 (192834'14765) modify f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.1883634.0:5247 2014-08-11 15:26:33.725826 2014-08-13 17:38:03.050911 7fe9bae887a0 20 read_log 192834'14777 (0'0) promote f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by osd.285.0:15679 2014-08-11 15:26:36.161859 2014-08-13 17:38:03.050923 7fe9bae887a0 20 read_log 192834'14778 (192834'14777) modify f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.1883634.0:5931 2014-08-11 15:26:36.153295 2014-08-13 17:38:03.050935 7fe9bae887a0 20 read_log 192834'14785 (0'0) promote 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by osd.285.0:15687 2014-08-11 15:26:39.275795 2014-08-13 17:38:03.050947 7fe9bae887a0 20 read_log 192834'14786 (192834'14785) modify 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.1883634.0:6720 2014-08-11 15:26:39.267074 2014-08-13 17:38:03.050959 7fe9bae887a0 20 read_log 192834'14791 (0'0) promote a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by osd.285.0:15693 2014-08-11 15:26:39.645606 2014-08-13 17:38:03.050971 7fe9bae887a0 20 read_log 192834'14792 (192834'14791) modify a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.1883634.0:6827 2014-08-11 15:26:39.637399 2014-08-13 17:38:03.050983 7fe9bae887a0 20 read_log 192834'14803 (0'0) promote 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by osd.285.0:15705 2014-08-11 15:26:43.581545 2014-08-13 17:38:03.050995 7fe9bae887a0 20 read_log 192834'14804 (192834'14803) modify 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.1883634.0:7770 2014-08-11 15:26:43.572710 2014-08-13 17:38:03.051007 7fe9bae887a0 20 read_log 192834'14833 (0'0) promote 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by osd.285.0:15735 2014-08-11 15:26:50.087951 2014-08-13 17:38:03.051019 7fe9bae887a0 20 read_log 192834'14834 (192834'14833) modify 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.1883634.0:9252 2014-08-11 15:26:50.079805 2014-08-13 17:38:03.051032 7fe9bae887a0 20 read_log 192834'14841 (0'0) promote 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by osd.285.0:15743 2014-08-11 15:26:51.999674 2014-08-13 17:38:03.051043 7fe9bae887a0 20 read_log 192834'14842 (192834'14841) modify 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.1883634.0:9702 2014-08-11 15:26:51.989350 2014-08-13 17:38:03.051056 7fe9bae887a0 20 read_log 192834'14849 (0'0) promote 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by osd.285.0:15751 2014-08-11 15:26:53.728506 2014-08-13 17:38:03.051077 7fe9bae887a0 20 read_log 192834'14850 (192834'14849) modify 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.1883634.0:10110 2014-08-11 15:26:53.721002 2014-08-13 17:38:03.051090 7fe9bae887a0 20 read_log 192834'14851 (0'0) promote c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by osd.285.0:15753 2014-08-11 15:26:54.205151 2014-08-13 17:38:03.051102 7fe9bae887a0 20 read_log 192834'14852 (192834'14851) modify c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.1883634.0:10239 2014-08-11 15:26:54.197679 2014-08-13 17:38:03.051114 7fe9bae887a0 20 read_log 192834'14873 (0'0) promote fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by osd.285.0:15775 2014-08-11 15:27:01.208552 2014-08-13 17:38:03.051126 7fe9bae887a0 20 read_log 192834'14874 (192834'14873) modify fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.1883634.0:11762 2014-08-11 15:27:01.197866 2014-08-13 17:38:03.051149 7fe9bae887a0 20 read_log 192834'14879 (0'0) promote 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by osd.285.0:15781 2014-08-11 15:27:02.665953 2014-08-13 17:38:03.051170 7fe9bae887a0 20 read_log 192834'14880 (192834'14879) modify 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.1883634.0:12138 2014-08-11 15:27:02.655752 2014-08-13 17:38:03.051183 7fe9bae887a0 20 read_log 192834'14885 (0'0) promote ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by osd.285.0:15787 2014-08-11 15:27:04.358375 2014-08-13 17:38:03.051195 7fe9bae887a0 20 read_log 192834'14886 (192834'14885) modify ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.1883634.0:12529 2014-08-11 15:27:04.349833 2014-08-13 17:38:03.051225 7fe9bae887a0 20 read_log 192834'14893 (0'0) promote 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by osd.285.0:15795 2014-08-11 15:27:10.125759 2014-08-13 17:38:03.051238 7fe9bae887a0 20 read_log 192834'14894 (192834'14893) modify 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.1883634.0:13572 2014-08-11 15:27:10.118053 2014-08-13 17:38:03.051250 7fe9bae887a0 20 read_log 192834'14901 (0'0) promote ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by osd.285.0:15803 2014-08-11 15:27:12.647227 2014-08-13 17:38:03.051262 7fe9bae887a0 20 read_log 192834'14902 (192834'14901) modify ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.1883634.0:14303 2014-08-11 15:27:12.638463 2014-08-13 17:38:03.051275 7fe9bae887a0 20 read_log 192834'14917 (0'0) promote 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by osd.285.0:15819 2014-08-11 15:27:15.934264 2014-08-13 17:38:03.051287 7fe9bae887a0 20 read_log 192834'14918 (192834'14917) modify 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.1883634.0:14895 2014-08-11 15:27:15.925402 2014-08-13 17:38:03.051299 7fe9bae887a0 20 read_log 192834'14975 (0'0) promote b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by osd.285.0:15877 2014-08-11 15:27:56.370505 2014-08-13 17:38:03.051311 7fe9bae887a0 20 read_log 192834'14976 (192834'14975) modify b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.1883634.0:20358 2014-08-11 15:27:56.362062 2014-08-13 17:38:03.051323 7fe9bae887a0 20 read_log 192834'14987 (0'0) promote 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by osd.285.0:15889 2014-08-11 15:28:03.035761 2014-08-13 17:38:03.051336 7fe9bae887a0 20 read_log 192834'14988 (192834'14987) modify 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.1883634.0:21149 2014-08-11 15:28:03.026835 2014-08-13 17:38:03.051348 7fe9bae887a0 20 read_log 192834'14999 (0'0) promote f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by osd.285.0:15901 2014-08-11 15:28:04.935339 2014-08-13 17:38:03.051368 7fe9bae887a0 20 read_log 192834'15000 (192834'14999) modify f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.1883634.0:21677 2014-08-11 15:28:04.926949 2014-08-13 17:38:03.051380 7fe9bae887a0 20 read_log 192834'15009 (0'0) promote 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by osd.285.0:15911 2014-08-11 15:28:06.737540 2014-08-13 17:38:03.051392 7fe9bae887a0 20 read_log 192834'15010 (192834'15009) modify 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.1883634.0:22107 2014-08-11 15:28:06.728351 2014-08-13 17:38:03.051405 7fe9bae887a0 20 read_log 192834'15011 (0'0) promote d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by osd.285.0:15913 2014-08-11 15:28:07.306261 2014-08-13 17:38:03.051417 7fe9bae887a0 20 read_log 192834'15012 (192834'15011) modify d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.1883634.0:22209 2014-08-11 15:28:07.296450 2014-08-13 17:38:03.051429 7fe9bae887a0 20 read_log 192834'15015 (0'0) promote 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by osd.285.0:15917 2014-08-11 15:28:08.528707 2014-08-13 17:38:03.051461 7fe9bae887a0 20 read_log 192834'15016 (192834'15015) modify 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.1883634.0:22417 2014-08-11 15:28:08.518458 2014-08-13 17:38:03.051474 7fe9bae887a0 20 read_log 192834'15037 (192834'14708) modify 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.1883634.0:23932 2014-08-11 15:28:18.102226 2014-08-13 17:38:03.051486 7fe9bae887a0 20 read_log 192834'15038 (192834'14710) modify 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.1883634.0:24029 2014-08-11 15:28:18.258434 2014-08-13 17:38:03.051499 7fe9bae887a0 20 read_log 192834'15042 (192834'14718) modify 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.1883634.0:24979 2014-08-11 15:28:19.853757 2014-08-13 17:38:03.051511 7fe9bae887a0 20 read_log 192834'15043 (192834'14720) modify b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.1883634.0:24985 2014-08-11 15:28:19.862282 2014-08-13 17:38:03.051523 7fe9bae887a0 20 read_log 192834'15044 (192834'14722) modify 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.1883634.0:25017 2014-08-11 15:28:19.944148 2014-08-13 17:38:03.051535 7fe9bae887a0 20 read_log 192834'15060 (192834'14754) modify f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.1883634.0:27104 2014-08-11 15:28:24.038126 2014-08-13 17:38:03.051548 7fe9bae887a0 20 read_log 192834'15062 (192834'14758) modify 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.1883634.0:27382 2014-08-11 15:28:24.429378 2014-08-13 17:38:03.051578 7fe9bae887a0 20 read_log 192834'15065 (192834'14764) modify 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.1883634.0:28419 2014-08-11 15:28:25.787274 2014-08-13 17:38:03.051592 7fe9bae887a0 20 read_log 192834'15066 (192834'14766) modify f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.1883634.0:28571 2014-08-11 15:28:25.997599 2014-08-13 17:38:03.051604 7fe9bae887a0 20 read_log 192834'15072 (192834'14778) modify f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.1883634.0:29255 2014-08-11 15:28:26.895782 2014-08-13 17:38:03.051616 7fe9bae887a0 20 read_log 192834'15076 (192834'14786) modify 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.1883634.0:30044 2014-08-11 15:28:27.999688 2014-08-13 17:38:03.051628 7fe9bae887a0 20 read_log 192834'15079 (192834'14792) modify a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.1883634.0:30151 2014-08-11 15:28:28.168122 2014-08-13 17:38:03.051640 7fe9bae887a0 20 read_log 192834'15085 (192834'14804) modify 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.1883634.0:31094 2014-08-11 15:28:29.878010 2014-08-13 17:38:03.051660 7fe9bae887a0 20 read_log 192834'15100 (192834'14834) modify 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.1883634.0:32576 2014-08-11 15:28:32.041413 2014-08-13 17:38:03.051673 7fe9bae887a0 20 read_log 192834'15104 (192834'14842) modify 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.1883634.0:33026 2014-08-11 15:28:32.686551 2014-08-13 17:38:03.051685 7fe9bae887a0 20 read_log 192834'15108 (192834'14850) modify 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.1883634.0:33434 2014-08-11 15:28:33.192983 2014-08-13 17:38:03.051697 7fe9bae887a0 20 read_log 192834'15109 (192834'14852) modify c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.1883634.0:33563 2014-08-11 15:28:33.464327 2014-08-13 17:38:03.051709 7fe9bae887a0 20 read_log 192834'15122 (192834'14874) modify fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.1883634.0:35086 2014-08-11 15:28:36.003340 2014-08-13 17:38:03.051724 7fe9bae887a0 20 read_log 192834'15125 (192834'14880) modify 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.1883634.0:35462 2014-08-11 15:28:36.667851 2014-08-13 17:38:03.051736 7fe9bae887a0 20 read_log 192834'15128 (192834'14886) modify ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.1883634.0:35853 2014-08-11 15:28:37.227144 2014-08-13 17:38:03.051748 7fe9bae887a0 20 read_log 192834'15132 (192834'14894) modify 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.1883634.0:36896 2014-08-11 15:28:38.556572 2014-08-13 17:38:03.051760 7fe9bae887a0 20 read_log 192834'15136 (192834'14902) modify ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.1883634.0:37627 2014-08-11 15:28:39.585103 2014-08-13 17:38:03.051772 7fe9bae887a0 20 read_log 192834'15144 (192834'14918) modify 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.1883634.0:38219 2014-08-11 15:28:40.372122 2014-08-13 17:38:03.051784 7fe9bae887a0 20 read_log 192834'15173 (192834'14976) modify b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.1883634.0:43682 2014-08-11 15:28:46.631171 2014-08-13 17:38:03.051796 7fe9bae887a0 20 read_log 192834'15179 (192834'14988) modify 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.1883634.0:44473 2014-08-11 15:28:46.868141 2014-08-13 17:38:03.051808 7fe9bae887a0 20 read_log 192834'15185 (192834'15000) modify f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.1883634.0:45001 2014-08-11 15:28:47.119151 2014-08-13 17:38:03.051820 7fe9bae887a0 20 read_log 192834'15190 (192834'15010) modify 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.1883634.0:45431 2014-08-11 15:28:47.275251 2014-08-13 17:38:03.051832 7fe9bae887a0 20 read_log 192834'15191 (192834'15012) modify d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.1883634.0:45533 2014-08-11 15:28:47.309162 2014-08-13 17:38:03.051844 7fe9bae887a0 20 read_log 192834'15193 (192834'15016) modify 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.1883634.0:45741 2014-08-11 15:28:47.371580 2014-08-13 17:38:03.051856 7fe9bae887a0 20 read_log 192888'15202 (0'0) promote 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by osd.368.0:33045 2014-08-11 16:44:55.898755 2014-08-13 17:38:03.051868 7fe9bae887a0 20 read_log 192888'15203 (192888'15202) modify 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.1315259.0:245 2014-08-11 16:44:55.889706 2014-08-13 17:38:03.051881 7fe9bae887a0 20 read_log 192888'15208 (0'0) promote be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by osd.368.0:33093 2014-08-11 16:45:00.853385 2014-08-13 17:38:03.051893 7fe9bae887a0 20 read_log 192888'15209 (192888'15208) modify be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.1315259.0:1454 2014-08-11 16:45:00.846015 2014-08-13 17:38:03.051915 7fe9bae887a0 20 read_log 192888'15212 (0'0) promote 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by osd.368.0:33111 2014-08-11 16:45:03.308596 2014-08-13 17:38:03.051928 7fe9bae887a0 20 read_log 192888'15213 (192888'15212) modify 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.1315259.0:2118 2014-08-11 16:45:03.301151 2014-08-13 17:38:03.051940 7fe9bae887a0 20 read_log 192888'15220 (0'0) promote eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by osd.368.0:33167 2014-08-11 16:45:12.788806 2014-08-13 17:38:03.051952 7fe9bae887a0 20 read_log 192888'15221 (192888'15220) modify eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.1315259.0:4641 2014-08-11 16:45:12.783258 2014-08-13 17:38:03.051965 7fe9bae887a0 20 read_log 192888'15224 (0'0) promote 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by osd.368.0:33189 2014-08-11 16:45:14.648375 2014-08-13 17:38:03.051977 7fe9bae887a0 20 read_log 192888'15225 (192888'15224) modify 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.1315259.0:5135 2014-08-11 16:45:14.641313 2014-08-13 17:38:03.051989 7fe9bae887a0 20 read_log 192888'15230 (0'0) promote a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by osd.368.0:33221 2014-08-11 16:45:20.269191 2014-08-13 17:38:03.052000 7fe9bae887a0 20 read_log 192888'15231 (192888'15230) modify a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.1315259.0:6688 2014-08-11 16:45:20.258889 2014-08-13 17:38:03.052013 7fe9bae887a0 20 read_log 192888'15232 (0'0) promote bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by osd.368.0:33223 2014-08-11 16:45:20.469987 2014-08-13 17:38:03.052024 7fe9bae887a0 20 read_log 192888'15233 (192888'15232) modify bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.1315259.0:6744 2014-08-11 16:45:20.461612 2014-08-13 17:38:03.052036 7fe9bae887a0 20 read_log 192888'15234 (0'0) promote b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by osd.368.0:33243 2014-08-11 16:45:21.413626 2014-08-13 17:38:03.052048 7fe9bae887a0 20 read_log 192888'15235 (192888'15234) modify b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.1315259.0:7004 2014-08-11 16:45:21.406100 2014-08-13 17:38:03.052060 7fe9bae887a0 20 read_log 192888'15236 (0'0) promote 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by osd.368.0:33251 2014-08-11 16:45:22.795822 2014-08-13 17:38:03.052072 7fe9bae887a0 20 read_log 192888'15237 (192888'15236) modify 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.1315259.0:7386 2014-08-11 16:45:22.785937 2014-08-13 17:38:03.052084 7fe9bae887a0 20 read_log 192888'15238 (0'0) promote a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by osd.368.0:33253 2014-08-11 16:45:22.878540 2014-08-13 17:38:03.052096 7fe9bae887a0 20 read_log 192888'15239 (192888'15238) modify a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.1315259.0:7409 2014-08-11 16:45:22.869413 2014-08-13 17:38:03.052108 7fe9bae887a0 20 read_log 192888'15240 (0'0) promote 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by osd.368.0:33285 2014-08-11 16:45:26.226445 2014-08-13 17:38:03.052120 7fe9bae887a0 20 read_log 192888'15241 (192888'15240) modify 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.1315259.0:8328 2014-08-11 16:45:26.219207 2014-08-13 17:38:03.052132 7fe9bae887a0 20 read_log 192888'15246 (0'0) promote de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by osd.368.0:33343 2014-08-11 16:45:33.543662 2014-08-13 17:38:03.052144 7fe9bae887a0 20 read_log 192888'15247 (192888'15246) modify de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.1315259.0:10239 2014-08-11 16:45:33.536133 2014-08-13 17:38:03.052168 7fe9bae887a0 20 read_log 192888'15250 (0'0) promote 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by osd.368.0:33351 2014-08-11 16:45:34.314300 2014-08-13 17:38:03.052185 7fe9bae887a0 20 read_log 192888'15251 (192888'15250) modify 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.1315259.0:10426 2014-08-11 16:45:34.306249 2014-08-13 17:38:03.052197 7fe9bae887a0 20 read_log 192888'15258 (0'0) promote 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by osd.368.0:33381 2014-08-11 16:45:40.951726 2014-08-13 17:38:03.052209 7fe9bae887a0 20 read_log 192888'15259 (192888'15258) modify 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.1315259.0:12068 2014-08-11 16:45:40.943960 2014-08-13 17:38:03.052221 7fe9bae887a0 20 read_log 192888'15266 (0'0) promote be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by osd.368.0:33397 2014-08-11 16:45:44.126038 2014-08-13 17:38:03.052233 7fe9bae887a0 20 read_log 192888'15267 (192888'15266) modify be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.1315259.0:12823 2014-08-11 16:45:44.117411 2014-08-13 17:38:03.052245 7fe9bae887a0 20 read_log 192888'15268 (0'0) promote 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by osd.368.0:33465 2014-08-11 16:45:52.568381 2014-08-13 17:38:03.052257 7fe9bae887a0 20 read_log 192888'15269 (192888'15268) modify 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.1315259.0:15003 2014-08-11 16:45:52.559759 2014-08-13 17:38:03.052269 7fe9bae887a0 20 read_log 192888'15270 (0'0) promote a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by osd.368.0:33467 2014-08-11 16:45:52.850646 2014-08-13 17:38:03.052281 7fe9bae887a0 20 read_log 192888'15271 (192888'15270) modify a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.1315259.0:15074 2014-08-11 16:45:52.842828 2014-08-13 17:38:03.052293 7fe9bae887a0 20 read_log 192888'15272 (0'0) promote 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by osd.368.0:33479 2014-08-11 16:45:54.838099 2014-08-13 17:38:03.052305 7fe9bae887a0 20 read_log 192888'15273 (192888'15272) modify 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.1315259.0:15578 2014-08-11 16:45:54.831057 2014-08-13 17:38:03.052317 7fe9bae887a0 20 read_log 192888'15274 (0'0) promote 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by osd.368.0:33507 2014-08-11 16:45:58.693949 2014-08-13 17:38:03.052329 7fe9bae887a0 20 read_log 192888'15275 (192888'15274) modify 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.1315259.0:16574 2014-08-11 16:45:58.686505 2014-08-13 17:38:03.052340 7fe9bae887a0 20 read_log 192888'15278 (0'0) promote c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by osd.368.0:33523 2014-08-11 16:46:02.163265 2014-08-13 17:38:03.052352 7fe9bae887a0 20 read_log 192888'15279 (192888'15278) modify c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.1315259.0:17464 2014-08-11 16:46:02.154428 2014-08-13 17:38:03.052364 7fe9bae887a0 20 read_log 192888'15284 (0'0) promote 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by osd.368.0:33597 2014-08-11 16:46:13.701128 2014-08-13 17:38:03.052376 7fe9bae887a0 20 read_log 192888'15285 (192888'15284) modify 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.1315259.0:20430 2014-08-11 16:46:13.693922 2014-08-13 17:38:03.052388 7fe9bae887a0 20 read_log 192888'15294 (0'0) promote 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by osd.368.0:33721 2014-08-11 16:46:29.003707 2014-08-13 17:38:03.052400 7fe9bae887a0 20 read_log 192888'15295 (192888'15294) modify 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.1315259.0:24364 2014-08-11 16:46:28.996545 2014-08-13 17:38:03.052419 7fe9bae887a0 20 read_log 192888'15296 (0'0) promote b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by osd.368.0:33723 2014-08-11 16:46:29.398063 2014-08-13 17:38:03.052432 7fe9bae887a0 20 read_log 192888'15297 (192888'15296) modify b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.1315259.0:24461 2014-08-11 16:46:29.390049 2014-08-13 17:38:03.052463 7fe9bae887a0 20 read_log 192888'15298 (0'0) promote 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by osd.368.0:33735 2014-08-11 16:46:30.818931 2014-08-13 17:38:03.052476 7fe9bae887a0 20 read_log 192888'15299 (192888'15298) modify 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.1315259.0:24813 2014-08-11 16:46:30.811518 2014-08-13 17:38:03.052488 7fe9bae887a0 20 read_log 192888'15302 (0'0) promote 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by osd.368.0:33739 2014-08-11 16:46:31.877404 2014-08-13 17:38:03.052502 7fe9bae887a0 20 read_log 192888'15303 (192888'15302) modify 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.1315259.0:25099 2014-08-11 16:46:31.868799 2014-08-13 17:38:03.052514 7fe9bae887a0 20 read_log 192888'15310 (0'0) promote ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by osd.368.0:33791 2014-08-11 16:46:39.631796 2014-08-13 17:38:03.052526 7fe9bae887a0 20 read_log 192888'15311 (192888'15310) modify ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.1315259.0:26942 2014-08-11 16:46:39.625045 2014-08-13 17:38:03.052538 7fe9bae887a0 20 read_log 192888'15312 (0'0) promote 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by osd.368.0:33799 2014-08-11 16:46:42.870797 2014-08-13 17:38:03.052550 7fe9bae887a0 20 read_log 192888'15313 (192888'15312) modify 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.1315259.0:27720 2014-08-11 16:46:42.863053 2014-08-13 17:38:03.052562 7fe9bae887a0 20 read_log 192888'15316 (0'0) promote d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by osd.368.0:33807 2014-08-11 16:46:44.468653 2014-08-13 17:38:03.052574 7fe9bae887a0 20 read_log 192888'15317 (192888'15316) modify d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.1315259.0:28130 2014-08-11 16:46:44.460055 2014-08-13 17:38:03.052586 7fe9bae887a0 20 read_log 192888'15318 (0'0) promote 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by osd.368.0:33817 2014-08-11 16:46:45.645608 2014-08-13 17:38:03.052598 7fe9bae887a0 20 read_log 192888'15319 (192888'15318) modify 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.1315259.0:28407 2014-08-11 16:46:45.638001 2014-08-13 17:38:03.052620 7fe9bae887a0 20 read_log 192888'15324 (192888'15203) modify 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.1315259.0:30972 2014-08-11 16:46:55.954528 2014-08-13 17:38:03.052633 7fe9bae887a0 20 read_log 192888'15327 (192888'15209) modify be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.1315259.0:32181 2014-08-11 16:46:56.308882 2014-08-13 17:38:03.052645 7fe9bae887a0 20 read_log 192888'15329 (192888'15213) modify 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.1315259.0:32845 2014-08-11 16:46:56.484376 2014-08-13 17:38:03.052657 7fe9bae887a0 20 read_log 192888'15333 (192888'15221) modify eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.1315259.0:35368 2014-08-11 16:46:57.146865 2014-08-13 17:38:03.052669 7fe9bae887a0 20 read_log 192888'15335 (192888'15225) modify 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.1315259.0:35862 2014-08-11 16:46:57.265215 2014-08-13 17:38:03.052681 7fe9bae887a0 20 read_log 192888'15338 (192888'15231) modify a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.1315259.0:37415 2014-08-11 16:46:57.677054 2014-08-13 17:38:03.052719 7fe9bae887a0 20 read_log 192888'15339 (192888'15233) modify bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.1315259.0:37471 2014-08-11 16:46:57.692255 2014-08-13 17:38:03.052733 7fe9bae887a0 20 read_log 192888'15340 (192888'15235) modify b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.1315259.0:37731 2014-08-11 16:46:57.759153 2014-08-13 17:38:03.052745 7fe9bae887a0 20 read_log 192888'15341 (192888'15237) modify 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.1315259.0:38113 2014-08-11 16:46:57.860451 2014-08-13 17:38:03.052758 7fe9bae887a0 20 read_log 192888'15342 (192888'15239) modify a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.1315259.0:38136 2014-08-11 16:46:57.865822 2014-08-13 17:38:03.052770 7fe9bae887a0 20 read_log 192888'15343 (192888'15241) modify 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.1315259.0:39055 2014-08-11 16:46:58.114634 2014-08-13 17:38:03.052782 7fe9bae887a0 20 read_log 192888'15346 (192888'15247) modify de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.1315259.0:40966 2014-08-11 16:46:58.627342 2014-08-13 17:38:03.052794 7fe9bae887a0 20 read_log 192888'15348 (192888'15251) modify 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.1315259.0:41153 2014-08-11 16:46:58.672822 2014-08-13 17:38:03.052806 7fe9bae887a0 20 read_log 192888'15352 (192888'15259) modify 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.1315259.0:42795 2014-08-11 16:46:59.108049 2014-08-13 17:38:03.052819 7fe9bae887a0 20 read_log 192888'15356 (192888'15267) modify be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.1315259.0:43550 2014-08-11 16:46:59.322360 2014-08-13 17:38:03.052831 7fe9bae887a0 20 read_log 192888'15357 (192888'15269) modify 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.1315259.0:45730 2014-08-11 16:46:59.952823 2014-08-13 17:38:03.052843 7fe9bae887a0 20 read_log 192888'15358 (192888'15271) modify a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.1315259.0:45801 2014-08-11 16:46:59.974189 2014-08-13 17:38:03.052855 7fe9bae887a0 20 read_log 192888'15359 (192888'15273) modify 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.1315259.0:46305 2014-08-11 16:47:00.120898 2014-08-13 17:38:03.052867 7fe9bae887a0 20 read_log 192888'15360 (192888'15275) modify 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.1315259.0:47301 2014-08-11 16:47:00.414315 2014-08-13 17:38:03.052879 7fe9bae887a0 20 read_log 192888'15362 (192888'15279) modify c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.1315259.0:48191 2014-08-11 16:47:00.693678 2014-08-13 17:38:03.052892 7fe9bae887a0 20 read_log 192888'15365 (192888'15285) modify 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.1315259.0:51157 2014-08-11 16:47:01.596434 2014-08-13 17:38:03.052904 7fe9bae887a0 20 read_log 192888'15370 (192888'15295) modify 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.1315259.0:55091 2014-08-11 16:47:02.810051 2014-08-13 17:38:03.052916 7fe9bae887a0 20 read_log 192888'15371 (192888'15297) modify b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.1315259.0:55188 2014-08-11 16:47:02.841108 2014-08-13 17:38:03.052930 7fe9bae887a0 20 read_log 192888'15372 (192888'15299) modify 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.1315259.0:55540 2014-08-11 16:47:02.942272 2014-08-13 17:38:03.052942 7fe9bae887a0 20 read_log 192888'15374 (192888'15303) modify 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.1315259.0:55826 2014-08-11 16:47:03.028204 2014-08-13 17:38:03.052958 7fe9bae887a0 20 read_log 192888'15378 (192888'15311) modify ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.1315259.0:57669 2014-08-11 16:47:03.578571 2014-08-13 17:38:03.052978 7fe9bae887a0 20 read_log 192888'15379 (192888'15313) modify 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.1315259.0:58447 2014-08-11 16:47:03.804436 2014-08-13 17:38:03.052992 7fe9bae887a0 20 read_log 192888'15381 (192888'15317) modify d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.1315259.0:58857 2014-08-11 16:47:03.928922 2014-08-13 17:38:03.053005 7fe9bae887a0 20 read_log 192888'15382 (192888'15319) modify 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.1315259.0:59134 2014-08-11 16:47:04.013559 2014-08-13 17:38:03.053020 7fe9bae887a0 20 read_log 192888'15395 (0'0) promote 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by osd.368.0:34361 2014-08-11 17:05:21.369147 2014-08-13 17:38:03.053032 7fe9bae887a0 20 read_log 192888'15396 (192888'15395) modify 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.942868.0:1552 2014-08-11 17:05:21.361364 2014-08-13 17:38:03.053044 7fe9bae887a0 20 read_log 192888'15397 (0'0) promote f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by osd.368.0:34373 2014-08-11 17:05:22.107220 2014-08-13 17:38:03.053056 7fe9bae887a0 20 read_log 192888'15398 (192888'15397) modify f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.942868.0:1717 2014-08-11 17:05:22.098848 2014-08-13 17:38:03.053069 7fe9bae887a0 20 read_log 192888'15399 (0'0) promote 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by osd.368.0:34399 2014-08-11 17:05:25.608119 2014-08-13 17:38:03.053083 7fe9bae887a0 20 read_log 192888'15400 (192888'15399) modify 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.942868.0:2612 2014-08-11 17:05:25.600995 2014-08-13 17:38:03.053095 7fe9bae887a0 20 read_log 192888'15405 (0'0) promote 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by osd.368.0:34427 2014-08-11 17:05:30.632236 2014-08-13 17:38:03.053109 7fe9bae887a0 20 read_log 192888'15406 (192888'15405) modify 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.942868.0:3971 2014-08-11 17:05:30.622700 2014-08-13 17:38:03.053122 7fe9bae887a0 20 read_log 192888'15407 (0'0) promote 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by osd.368.0:34431 2014-08-11 17:05:31.377944 2014-08-13 17:38:03.053136 7fe9bae887a0 20 read_log 192888'15408 (192888'15407) modify 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.942868.0:4174 2014-08-11 17:05:31.369321 2014-08-13 17:38:03.053149 7fe9bae887a0 20 read_log 192888'15409 (0'0) promote 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by osd.368.0:34455 2014-08-11 17:05:35.728450 2014-08-13 17:38:03.053169 7fe9bae887a0 20 read_log 192888'15410 (192888'15409) modify 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.942868.0:5295 2014-08-11 17:05:35.719816 2014-08-13 17:38:03.053182 7fe9bae887a0 20 read_log 192888'15413 (0'0) promote 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by osd.368.0:34463 2014-08-11 17:05:37.044725 2014-08-13 17:38:03.053194 7fe9bae887a0 20 read_log 192888'15414 (192888'15413) modify 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.942868.0:5649 2014-08-11 17:05:37.037132 2014-08-13 17:38:03.053206 7fe9bae887a0 20 read_log 192888'15427 (0'0) promote 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by osd.368.0:34583 2014-08-11 17:05:52.433754 2014-08-13 17:38:03.053218 7fe9bae887a0 20 read_log 192888'15428 (192888'15427) modify 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.942868.0:9787 2014-08-11 17:05:52.425731 2014-08-13 17:38:03.053230 7fe9bae887a0 20 read_log 192888'15433 (0'0) promote 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by osd.368.0:34649 2014-08-11 17:06:00.661890 2014-08-13 17:38:03.053249 7fe9bae887a0 20 read_log 192888'15434 (192888'15433) modify 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.942868.0:12043 2014-08-11 17:06:00.655161 2014-08-13 17:38:03.053262 7fe9bae887a0 20 read_log 192888'15443 (0'0) promote d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by osd.368.0:34693 2014-08-11 17:06:05.809441 2014-08-13 17:38:03.053274 7fe9bae887a0 20 read_log 192888'15444 (192888'15443) modify d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.942868.0:13410 2014-08-11 17:06:05.802176 2014-08-13 17:38:03.053286 7fe9bae887a0 20 read_log 192888'15445 (0'0) promote b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by osd.368.0:34733 2014-08-11 17:06:13.119785 2014-08-13 17:38:03.053299 7fe9bae887a0 20 read_log 192888'15446 (192888'15445) modify b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.942868.0:15230 2014-08-11 17:06:13.111811 2014-08-13 17:38:03.053311 7fe9bae887a0 20 read_log 192888'15451 (0'0) promote af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by osd.368.0:34755 2014-08-11 17:06:15.744963 2014-08-13 17:38:03.053323 7fe9bae887a0 20 read_log 192888'15452 (192888'15451) modify af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.942868.0:15905 2014-08-11 17:06:15.734822 2014-08-13 17:38:03.053334 7fe9bae887a0 20 read_log 192888'15455 (0'0) promote 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by osd.368.0:34779 2014-08-11 17:06:19.319683 2014-08-13 17:38:03.053346 7fe9bae887a0 20 read_log 192888'15456 (192888'15455) modify 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.942868.0:16816 2014-08-11 17:06:19.312728 2014-08-13 17:38:03.053358 7fe9bae887a0 20 read_log 192888'15457 (0'0) promote 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by osd.368.0:34785 2014-08-11 17:06:20.006342 2014-08-13 17:38:03.053370 7fe9bae887a0 20 read_log 192888'15458 (192888'15457) modify 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.942868.0:16985 2014-08-11 17:06:19.997695 2014-08-13 17:38:03.053382 7fe9bae887a0 20 read_log 192888'15461 (0'0) promote 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by osd.368.0:34803 2014-08-11 17:06:23.063876 2014-08-13 17:38:03.053394 7fe9bae887a0 20 read_log 192888'15462 (192888'15461) modify 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.942868.0:17795 2014-08-11 17:06:23.056628 2014-08-13 17:38:03.053406 7fe9bae887a0 20 read_log 192888'15467 (0'0) promote 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by osd.368.0:34863 2014-08-11 17:06:34.676219 2014-08-13 17:38:03.053418 7fe9bae887a0 20 read_log 192888'15468 (192888'15467) modify 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.942868.0:20862 2014-08-11 17:06:34.668394 2014-08-13 17:38:03.053430 7fe9bae887a0 20 read_log 192888'15473 (0'0) promote 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by osd.368.0:34893 2014-08-11 17:06:41.201593 2014-08-13 17:38:03.053442 7fe9bae887a0 20 read_log 192888'15474 (192888'15473) modify 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.942868.0:22552 2014-08-11 17:06:41.194045 2014-08-13 17:38:03.053455 7fe9bae887a0 20 read_log 192888'15475 (0'0) promote 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by osd.368.0:34947 2014-08-11 17:06:49.148182 2014-08-13 17:38:03.053467 7fe9bae887a0 20 read_log 192888'15476 (192888'15475) modify 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.942868.0:24658 2014-08-11 17:06:49.139180 2014-08-13 17:38:03.053479 7fe9bae887a0 20 read_log 192888'15477 (0'0) promote a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by osd.368.0:34961 2014-08-11 17:06:50.337174 2014-08-13 17:38:03.053500 7fe9bae887a0 20 read_log 192888'15478 (192888'15477) modify a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.942868.0:24980 2014-08-11 17:06:50.329903 2014-08-13 17:38:03.053513 7fe9bae887a0 20 read_log 192888'15481 (0'0) promote f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by osd.368.0:34983 2014-08-11 17:06:52.578208 2014-08-13 17:38:03.053525 7fe9bae887a0 20 read_log 192888'15482 (192888'15481) modify f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.942868.0:25573 2014-08-11 17:06:52.570959 2014-08-13 17:38:03.053560 7fe9bae887a0 20 read_log 192888'15485 (0'0) promote f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by osd.368.0:35067 2014-08-11 17:07:04.428393 2014-08-13 17:38:03.053572 7fe9bae887a0 20 read_log 192888'15486 (192888'15485) modify f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.942868.0:28662 2014-08-11 17:07:04.421003 2014-08-13 17:38:03.053584 7fe9bae887a0 20 read_log 192888'15487 (0'0) promote c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by osd.368.0:35073 2014-08-11 17:07:05.501492 2014-08-13 17:38:03.053596 7fe9bae887a0 20 read_log 192888'15488 (192888'15487) modify c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.942868.0:28955 2014-08-11 17:07:05.494665 2014-08-13 17:38:03.053608 7fe9bae887a0 20 read_log 192888'15489 (0'0) promote 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by osd.368.0:35077 2014-08-11 17:07:06.166724 2014-08-13 17:38:03.053620 7fe9bae887a0 20 read_log 192888'15490 (192888'15489) modify 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.942868.0:29117 2014-08-11 17:07:06.159242 2014-08-13 17:38:03.053632 7fe9bae887a0 20 read_log 192888'15491 (0'0) promote 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by osd.368.0:35081 2014-08-11 17:07:07.140676 2014-08-13 17:38:03.053644 7fe9bae887a0 20 read_log 192888'15492 (192888'15491) modify 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.942868.0:29345 2014-08-11 17:07:07.132927 2014-08-13 17:38:03.053657 7fe9bae887a0 20 read_log 192888'15499 (192888'15396) modify 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.942868.0:32818 2014-08-11 17:07:16.353307 2014-08-13 17:38:03.053669 7fe9bae887a0 20 read_log 192888'15500 (192888'15398) modify f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.942868.0:32983 2014-08-11 17:07:16.395800 2014-08-13 17:38:03.053681 7fe9bae887a0 20 read_log 192888'15501 (192888'15400) modify 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.942868.0:33878 2014-08-11 17:07:16.636734 2014-08-13 17:38:03.053693 7fe9bae887a0 20 read_log 192888'15504 (192888'15406) modify 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.942868.0:35237 2014-08-11 17:07:17.023609 2014-08-13 17:38:03.053720 7fe9bae887a0 20 read_log 192888'15505 (192888'15408) modify 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.942868.0:35440 2014-08-11 17:07:17.075124 2014-08-13 17:38:03.053733 7fe9bae887a0 20 read_log 192888'15506 (192888'15410) modify 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.942868.0:36561 2014-08-11 17:07:17.375086 2014-08-13 17:38:03.053745 7fe9bae887a0 20 read_log 192888'15508 (192888'15414) modify 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.942868.0:36915 2014-08-11 17:07:17.467253 2014-08-13 17:38:03.053757 7fe9bae887a0 20 read_log 192888'15515 (192888'15428) modify 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.942868.0:41053 2014-08-11 17:07:18.622289 2014-08-13 17:38:03.053769 7fe9bae887a0 20 read_log 192888'15518 (192888'15434) modify 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.942868.0:43309 2014-08-11 17:07:19.236761 2014-08-13 17:38:03.053789 7fe9bae887a0 20 read_log 192888'15523 (192888'15444) modify d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.942868.0:44676 2014-08-11 17:07:19.633664 2014-08-13 17:38:03.053805 7fe9bae887a0 20 read_log 192888'15524 (192888'15446) modify b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.942868.0:46496 2014-08-11 17:07:20.139540 2014-08-13 17:38:03.053817 7fe9bae887a0 20 read_log 192888'15527 (192888'15452) modify af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.942868.0:47171 2014-08-11 17:07:20.329171 2014-08-13 17:38:03.053829 7fe9bae887a0 20 read_log 192888'15529 (192888'15456) modify 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.942868.0:48082 2014-08-11 17:07:20.616123 2014-08-13 17:38:03.053841 7fe9bae887a0 20 read_log 192888'15530 (192888'15458) modify 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.942868.0:48251 2014-08-11 17:07:20.667435 2014-08-13 17:38:03.053853 7fe9bae887a0 20 read_log 192888'15532 (192888'15462) modify 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.942868.0:49061 2014-08-11 17:07:20.897075 2014-08-13 17:38:03.053865 7fe9bae887a0 20 read_log 192888'15535 (192888'15468) modify 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.942868.0:52128 2014-08-11 17:07:21.848096 2014-08-13 17:38:03.053879 7fe9bae887a0 20 read_log 192888'15538 (192888'15474) modify 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.942868.0:53818 2014-08-11 17:07:22.383746 2014-08-13 17:38:03.053891 7fe9bae887a0 20 read_log 192888'15539 (192888'15476) modify 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.942868.0:55924 2014-08-11 17:07:22.997564 2014-08-13 17:38:03.053903 7fe9bae887a0 20 read_log 192888'15540 (192888'15478) modify a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.942868.0:56246 2014-08-11 17:07:23.092675 2014-08-13 17:38:03.053915 7fe9bae887a0 20 read_log 192888'15542 (192888'15482) modify f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.942868.0:56839 2014-08-11 17:07:23.274220 2014-08-13 17:38:03.053927 7fe9bae887a0 20 read_log 192888'15544 (192888'15486) modify f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.942868.0:59928 2014-08-11 17:07:24.189898 2014-08-13 17:38:03.053939 7fe9bae887a0 20 read_log 192888'15545 (192888'15488) modify c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.942868.0:60221 2014-08-11 17:07:24.275013 2014-08-13 17:38:03.053951 7fe9bae887a0 20 read_log 192888'15546 (192888'15490) modify 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.942868.0:60383 2014-08-11 17:07:24.324648 2014-08-13 17:38:03.053962 7fe9bae887a0 20 read_log 192888'15547 (192888'15492) modify 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.942868.0:60611 2014-08-11 17:07:24.390088 2014-08-13 17:38:03.053974 7fe9bae887a0 20 read_log 193305'15549 (0'0) promote 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by osd.304.0:1 2014-08-11 18:42:33.705921 2014-08-13 17:38:03.053986 7fe9bae887a0 20 read_log 193305'15550 (193305'15549) modify 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.951337.0:194 2014-08-11 18:42:33.695470 2014-08-13 17:38:03.053999 7fe9bae887a0 20 read_log 193305'15551 (0'0) promote 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by osd.304.0:40 2014-08-11 18:42:50.521334 2014-08-13 17:38:03.054011 7fe9bae887a0 20 read_log 193305'15552 (193305'15551) modify 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.951337.0:4673 2014-08-11 18:42:50.513623 2014-08-13 17:38:03.054023 7fe9bae887a0 20 read_log 193305'15553 (0'0) promote 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by osd.304.0:44 2014-08-11 18:42:51.635201 2014-08-13 17:38:03.054042 7fe9bae887a0 20 read_log 193305'15554 (193305'15553) modify 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.951337.0:4974 2014-08-11 18:42:51.627466 2014-08-13 17:38:03.054057 7fe9bae887a0 20 read_log 193305'15555 (0'0) promote b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by osd.304.0:56 2014-08-11 18:42:58.036339 2014-08-13 17:38:03.054069 7fe9bae887a0 20 read_log 193305'15556 (193305'15555) modify b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.951337.0:6720 2014-08-11 18:42:58.025265 2014-08-13 17:38:03.054084 7fe9bae887a0 20 read_log 193305'15557 (0'0) promote fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by osd.304.0:58 2014-08-11 18:42:58.300554 2014-08-13 17:38:03.054096 7fe9bae887a0 20 read_log 193305'15558 (193305'15557) modify fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.951337.0:6793 2014-08-11 18:42:58.290875 2014-08-13 17:38:03.054108 7fe9bae887a0 20 read_log 193305'15559 (0'0) promote 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by osd.304.0:62 2014-08-11 18:43:01.945118 2014-08-13 17:38:03.054120 7fe9bae887a0 20 read_log 193305'15560 (193305'15559) modify 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.951337.0:7801 2014-08-11 18:43:01.935038 2014-08-13 17:38:03.054133 7fe9bae887a0 20 read_log 193305'15561 (0'0) promote 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by osd.304.0:64 2014-08-11 18:43:02.839760 2014-08-13 17:38:03.054147 7fe9bae887a0 20 read_log 193305'15562 (193305'15561) modify 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.951337.0:8046 2014-08-11 18:43:02.832315 2014-08-13 17:38:03.054163 7fe9bae887a0 20 read_log 193305'15563 (0'0) promote 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by osd.304.0:76 2014-08-11 18:43:07.406977 2014-08-13 17:38:03.054177 7fe9bae887a0 20 read_log 193305'15564 (193305'15563) modify 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.951337.0:9214 2014-08-11 18:43:07.399991 2014-08-13 17:38:03.054193 7fe9bae887a0 20 read_log 193305'15565 (0'0) promote 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by osd.304.0:78 2014-08-11 18:43:08.195284 2014-08-13 17:38:03.054205 7fe9bae887a0 20 read_log 193305'15566 (193305'15565) modify 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.951337.0:9427 2014-08-11 18:43:08.188814 2014-08-13 17:38:03.054217 7fe9bae887a0 20 read_log 193305'15567 (0'0) promote cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by osd.304.0:92 2014-08-11 18:43:10.634154 2014-08-13 17:38:03.054229 7fe9bae887a0 20 read_log 193305'15568 (193305'15567) modify cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.951337.0:10094 2014-08-11 18:43:10.626155 2014-08-13 17:38:03.054244 7fe9bae887a0 20 read_log 193305'15569 (0'0) promote deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by osd.304.0:94 2014-08-11 18:43:11.506550 2014-08-13 17:38:03.054256 7fe9bae887a0 20 read_log 193305'15570 (193305'15569) modify deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.951337.0:10332 2014-08-11 18:43:11.498277 2014-08-13 17:38:03.054268 7fe9bae887a0 20 read_log 193305'15571 (0'0) promote 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by osd.304.0:98 2014-08-11 18:43:12.320363 2014-08-13 17:38:03.054281 7fe9bae887a0 20 read_log 193305'15572 (193305'15571) modify 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.951337.0:10546 2014-08-11 18:43:12.311864 2014-08-13 17:38:03.054293 7fe9bae887a0 20 read_log 193305'15573 (0'0) promote e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by osd.304.0:113 2014-08-11 18:43:19.226572 2014-08-13 17:38:03.054314 7fe9bae887a0 20 read_log 193305'15574 (193305'15573) modify e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.951337.0:12366 2014-08-11 18:43:19.219127 2014-08-13 17:38:03.054329 7fe9bae887a0 20 read_log 193305'15575 (0'0) promote 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by osd.304.0:127 2014-08-11 18:43:21.897577 2014-08-13 17:38:03.054342 7fe9bae887a0 20 read_log 193305'15576 (193305'15575) modify 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.951337.0:13073 2014-08-11 18:43:21.890722 2014-08-13 17:38:03.054354 7fe9bae887a0 20 read_log 193305'15577 (0'0) promote 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by osd.304.0:133 2014-08-11 18:43:24.328436 2014-08-13 17:38:03.054366 7fe9bae887a0 20 read_log 193305'15578 (193305'15577) modify 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.951337.0:13702 2014-08-11 18:43:24.320530 2014-08-13 17:38:03.054378 7fe9bae887a0 20 read_log 193305'15579 (0'0) promote 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by osd.304.0:153 2014-08-11 18:43:32.767967 2014-08-13 17:38:03.054390 7fe9bae887a0 20 read_log 193305'15580 (193305'15579) modify 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.951337.0:16104 2014-08-11 18:43:32.758534 2014-08-13 17:38:03.054402 7fe9bae887a0 20 read_log 193305'15581 (0'0) promote 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by osd.304.0:157 2014-08-11 18:43:35.798347 2014-08-13 17:38:03.054413 7fe9bae887a0 20 read_log 193305'15582 (193305'15581) modify 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.951337.0:16952 2014-08-11 18:43:35.789792 2014-08-13 17:38:03.054425 7fe9bae887a0 20 read_log 193305'15583 (0'0) promote 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by osd.304.0:165 2014-08-11 18:43:36.775821 2014-08-13 17:38:03.054437 7fe9bae887a0 20 read_log 193305'15584 (193305'15583) modify 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.951337.0:17238 2014-08-11 18:43:36.765850 2014-08-13 17:38:03.054449 7fe9bae887a0 20 read_log 193305'15585 (0'0) promote c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by osd.304.0:171 2014-08-11 18:43:37.733100 2014-08-13 17:38:03.054461 7fe9bae887a0 20 read_log 193305'15586 (193305'15585) modify c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.951337.0:17530 2014-08-11 18:43:37.725478 2014-08-13 17:38:03.054473 7fe9bae887a0 20 read_log 193305'15587 (0'0) promote 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by osd.304.0:175 2014-08-11 18:43:38.573705 2014-08-13 17:38:03.054485 7fe9bae887a0 20 read_log 193305'15588 (193305'15587) modify 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.951337.0:17789 2014-08-11 18:43:38.565859 2014-08-13 17:38:03.054497 7fe9bae887a0 20 read_log 193305'15589 (0'0) promote 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by osd.304.0:191 2014-08-11 18:43:43.019978 2014-08-13 17:38:03.054509 7fe9bae887a0 20 read_log 193305'15590 (193305'15589) modify 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.951337.0:19019 2014-08-11 18:43:43.011850 2014-08-13 17:38:03.054521 7fe9bae887a0 20 read_log 193305'15591 (0'0) promote 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by osd.304.0:195 2014-08-11 18:43:44.023448 2014-08-13 17:38:03.054533 7fe9bae887a0 20 read_log 193305'15592 (193305'15591) modify 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.951337.0:19271 2014-08-11 18:43:44.014951 2014-08-13 17:38:03.054545 7fe9bae887a0 20 read_log 193305'15593 (0'0) promote 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by osd.304.0:205 2014-08-11 18:43:45.837641 2014-08-13 17:38:03.054584 7fe9bae887a0 20 read_log 193305'15594 (193305'15593) modify 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.951337.0:19780 2014-08-11 18:43:45.830070 2014-08-13 17:38:03.054600 7fe9bae887a0 20 read_log 193305'15595 (0'0) promote 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by osd.304.0:213 2014-08-11 18:43:50.719769 2014-08-13 17:38:03.054613 7fe9bae887a0 20 read_log 193305'15596 (193305'15595) modify 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.951337.0:21138 2014-08-11 18:43:50.711486 2014-08-13 17:38:03.054625 7fe9bae887a0 20 read_log 193305'15597 (0'0) promote c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by osd.304.0:227 2014-08-11 18:43:55.975253 2014-08-13 17:38:03.054637 7fe9bae887a0 20 read_log 193305'15598 (193305'15597) modify c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.951337.0:22588 2014-08-11 18:43:55.961892 2014-08-13 17:38:03.054649 7fe9bae887a0 20 read_log 193305'15599 (0'0) promote f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by osd.304.0:275 2014-08-11 18:44:10.048400 2014-08-13 17:38:03.054661 7fe9bae887a0 20 read_log 193305'15600 (193305'15599) modify f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.951337.0:26460 2014-08-11 18:44:10.041187 2014-08-13 17:38:03.054673 7fe9bae887a0 20 read_log 193305'15601 (0'0) promote 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by osd.304.0:283 2014-08-11 18:44:14.699398 2014-08-13 17:38:03.054685 7fe9bae887a0 20 read_log 193305'15602 (193305'15601) modify 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.951337.0:27732 2014-08-11 18:44:14.693192 2014-08-13 17:38:03.054697 7fe9bae887a0 20 read_log 193305'15603 (0'0) promote 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by osd.304.0:287 2014-08-11 18:44:19.494239 2014-08-13 17:38:03.054709 7fe9bae887a0 20 read_log 193305'15604 (193305'15603) modify 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.951337.0:29042 2014-08-11 18:44:19.487048 2014-08-13 17:38:03.054721 7fe9bae887a0 20 read_log 193305'15605 (0'0) promote 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by osd.304.0:293 2014-08-11 18:44:21.173698 2014-08-13 17:38:03.054733 7fe9bae887a0 20 read_log 193305'15606 (193305'15605) modify 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.951337.0:29502 2014-08-11 18:44:21.166549 2014-08-13 17:38:03.054745 7fe9bae887a0 20 read_log 193305'15607 (0'0) promote 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by osd.304.0:303 2014-08-11 18:44:24.162345 2014-08-13 17:38:03.054758 7fe9bae887a0 20 read_log 193305'15608 (193305'15607) modify 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.951337.0:30289 2014-08-11 18:44:24.155129 2014-08-13 17:38:03.054771 7fe9bae887a0 20 read_log 193305'15609 (0'0) promote dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by osd.304.0:309 2014-08-11 18:44:26.439910 2014-08-13 17:38:03.054783 7fe9bae887a0 20 read_log 193305'15610 (193305'15609) modify dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.951337.0:30889 2014-08-11 18:44:26.431959 2014-08-13 17:38:03.054795 7fe9bae887a0 20 read_log 193305'15611 (193305'15550) modify 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.951337.0:32905 2014-08-11 18:44:33.949112 2014-08-13 17:38:03.054807 7fe9bae887a0 20 read_log 193305'15612 (193305'15552) modify 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.951337.0:37384 2014-08-11 18:44:36.878940 2014-08-13 17:38:03.054819 7fe9bae887a0 20 read_log 193305'15613 (193305'15554) modify 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.951337.0:37685 2014-08-11 18:44:37.023487 2014-08-13 17:38:03.054841 7fe9bae887a0 20 read_log 193305'15614 (193305'15556) modify b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.951337.0:39431 2014-08-11 18:44:38.003227 2014-08-13 17:38:03.054856 7fe9bae887a0 20 read_log 193305'15615 (193305'15558) modify fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.951337.0:39504 2014-08-11 18:44:38.047885 2014-08-13 17:38:03.054868 7fe9bae887a0 20 read_log 193305'15616 (193305'15560) modify 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.951337.0:40512 2014-08-11 18:44:38.435040 2014-08-13 17:38:03.054880 7fe9bae887a0 20 read_log 193305'15617 (193305'15562) modify 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.951337.0:40757 2014-08-11 18:44:38.517026 2014-08-13 17:38:03.054895 7fe9bae887a0 20 read_log 193305'15618 (193305'15564) modify 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.951337.0:41925 2014-08-11 18:44:38.927749 2014-08-13 17:38:03.054910 7fe9bae887a0 20 read_log 193305'15619 (193305'15566) modify 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.951337.0:42138 2014-08-11 18:44:39.121265 2014-08-13 17:38:03.054923 7fe9bae887a0 20 read_log 193305'15620 (193305'15568) modify cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.951337.0:42805 2014-08-11 18:44:39.387165 2014-08-13 17:38:03.054941 7fe9bae887a0 20 read_log 193305'15621 (193305'15570) modify deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.951337.0:43043 2014-08-11 18:44:39.465777 2014-08-13 17:38:03.054970 7fe9bae887a0 20 read_log 193305'15622 (193305'15572) modify 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.951337.0:43257 2014-08-11 18:44:39.534353 2014-08-13 17:38:03.054985 7fe9bae887a0 20 read_log 193305'15623 (193305'15623) modify 33a/hit_set_24.33a_archive_2014-08-11 18:39:39.883446_2014-08-11 18:44:40.084433/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:40.084663 2014-08-13 17:38:03.054998 7fe9bae887a0 20 read_log 193305'15624 (193305'15574) modify e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.951337.0:45077 2014-08-11 18:44:40.083366 2014-08-13 17:38:03.055012 7fe9bae887a0 20 read_log 193305'15625 (193305'15576) modify 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.951337.0:45784 2014-08-11 18:44:40.303394 2014-08-13 17:38:03.055024 7fe9bae887a0 20 read_log 193305'15626 (193305'15578) modify 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.951337.0:46413 2014-08-11 18:44:40.475890 2014-08-13 17:38:03.055036 7fe9bae887a0 20 read_log 193305'15627 (193305'15580) modify 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.951337.0:48815 2014-08-11 18:44:41.163819 2014-08-13 17:38:03.055050 7fe9bae887a0 20 read_log 193305'15628 (193305'15582) modify 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.951337.0:49663 2014-08-11 18:44:41.434142 2014-08-13 17:38:03.055062 7fe9bae887a0 20 read_log 193305'15629 (193305'15584) modify 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.951337.0:49949 2014-08-11 18:44:41.524260 2014-08-13 17:38:03.055076 7fe9bae887a0 20 read_log 193305'15630 (193305'15586) modify c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.951337.0:50241 2014-08-11 18:44:41.609318 2014-08-13 17:38:03.055089 7fe9bae887a0 20 read_log 193305'15631 (193305'15588) modify 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.951337.0:50500 2014-08-11 18:44:41.682508 2014-08-13 17:38:03.055101 7fe9bae887a0 20 read_log 193305'15632 (193305'15590) modify 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.951337.0:51730 2014-08-11 18:44:42.034688 2014-08-13 17:38:03.055125 7fe9bae887a0 20 read_log 193305'15633 (193305'15592) modify 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.951337.0:51982 2014-08-11 18:44:42.102559 2014-08-13 17:38:03.055141 7fe9bae887a0 20 read_log 193305'15634 (193305'15594) modify 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.951337.0:52491 2014-08-11 18:44:42.248927 2014-08-13 17:38:03.055157 7fe9bae887a0 20 read_log 193305'15635 (193305'15596) modify 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.951337.0:53849 2014-08-11 18:44:42.656593 2014-08-13 17:38:03.055170 7fe9bae887a0 20 read_log 193305'15636 (193305'15598) modify c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.951337.0:55299 2014-08-11 18:44:43.082503 2014-08-13 17:38:03.055182 7fe9bae887a0 20 read_log 193305'15637 (193305'15600) modify f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.951337.0:59171 2014-08-11 18:44:44.252832 2014-08-13 17:38:03.055195 7fe9bae887a0 20 read_log 193305'15638 (193305'15602) modify 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.951337.0:60443 2014-08-11 18:44:44.630331 2014-08-13 17:38:03.055207 7fe9bae887a0 20 read_log 193305'15639 (193305'15604) modify 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.951337.0:61753 2014-08-11 18:44:45.019353 2014-08-13 17:38:03.055219 7fe9bae887a0 20 read_log 193305'15640 (193305'15606) modify 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.951337.0:62213 2014-08-11 18:44:45.158826 2014-08-13 17:38:03.055231 7fe9bae887a0 20 read_log 193305'15641 (193305'15608) modify 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.951337.0:63000 2014-08-11 18:44:45.383347 2014-08-13 17:38:03.055243 7fe9bae887a0 20 read_log 193305'15642 (193305'15610) modify dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.951337.0:63600 2014-08-11 18:44:45.555656 2014-08-13 17:38:03.055255 7fe9bae887a0 20 read_log 194215'15643 (194215'15643) modify 33a/hit_set_24.33a_archive_2014-08-12 15:32:25.735124_2014-08-12 15:48:57.821063/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:57.821307 2014-08-13 17:38:03.055268 7fe9bae887a0 20 read_log 194215'15644 (193305'15623) delete 33a/hit_set_24.33a_archive_2014-08-11 18:39:39.883446_2014-08-11 18:44:40.084433/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:57.821307 2014-08-13 17:38:03.055280 7fe9bae887a0 20 read_log 194215'15645 (192834'14574) clean 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by osd.304.0:1242 2014-08-12 15:48:57.828539 2014-08-13 17:38:03.055292 7fe9bae887a0 20 read_log 194215'15646 (194215'15645) delete 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.2174595.0:170253 0.000000 2014-08-13 17:38:03.055306 7fe9bae887a0 20 read_log 194215'15647 (193305'15624) clean e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by osd.304.0:1244 2014-08-12 15:48:57.841510 2014-08-13 17:38:03.055319 7fe9bae887a0 20 read_log 194215'15648 (194215'15647) delete e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.2174595.0:170255 0.000000 2014-08-13 17:38:03.055330 7fe9bae887a0 20 read_log 194215'15649 (193305'15639) clean 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by osd.304.0:1246 2014-08-12 15:48:57.849884 2014-08-13 17:38:03.055342 7fe9bae887a0 20 read_log 194215'15650 (194215'15649) delete 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.2174595.0:170257 0.000000 2014-08-13 17:38:03.055353 7fe9bae887a0 20 read_log 194215'15651 (192888'15518) clean 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by osd.304.0:1248 2014-08-12 15:48:57.856886 2014-08-13 17:38:03.055373 7fe9bae887a0 20 read_log 194215'15652 (194215'15651) delete 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.2174595.0:170259 0.000000 2014-08-13 17:38:03.055384 7fe9bae887a0 20 read_log 194215'15653 (192834'15038) clean 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by osd.304.0:1250 2014-08-12 15:48:57.863772 2014-08-13 17:38:03.055396 7fe9bae887a0 20 read_log 194215'15654 (194215'15653) delete 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.2174595.0:170261 0.000000 2014-08-13 17:38:03.055407 7fe9bae887a0 20 read_log 194215'15655 (193305'15612) clean 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by osd.304.0:1252 2014-08-12 15:48:57.870645 2014-08-13 17:38:03.055419 7fe9bae887a0 20 read_log 194215'15656 (194215'15655) delete 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.2174595.0:170263 0.000000 2014-08-13 17:38:03.055430 7fe9bae887a0 20 read_log 194215'15657 (193305'15634) clean 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by osd.304.0:1254 2014-08-12 15:48:57.877426 2014-08-13 17:38:03.055442 7fe9bae887a0 20 read_log 194215'15658 (194215'15657) delete 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.2174595.0:170265 0.000000 2014-08-13 17:38:03.055453 7fe9bae887a0 20 read_log 194215'15659 (192888'15523) clean d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by osd.304.0:1256 2014-08-12 15:48:57.884407 2014-08-13 17:38:03.055465 7fe9bae887a0 20 read_log 194215'15660 (194215'15659) delete d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.2174595.0:170267 0.000000 2014-08-13 17:38:03.055475 7fe9bae887a0 20 read_log 194215'15661 (192834'14518) clean e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by osd.304.0:1258 2014-08-12 15:48:57.907387 2014-08-13 17:38:03.055487 7fe9bae887a0 20 read_log 194215'15662 (194215'15661) delete e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by client.2174595.0:170269 0.000000 2014-08-13 17:38:03.055498 7fe9bae887a0 20 read_log 194215'15663 (192834'14664) clean b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by osd.304.0:1260 2014-08-12 15:48:57.915419 2014-08-13 17:38:03.055510 7fe9bae887a0 20 read_log 194215'15664 (194215'15663) delete b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.2174595.0:170271 0.000000 2014-08-13 17:38:03.055520 7fe9bae887a0 20 read_log 194215'15665 (192888'15379) clean 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by osd.304.0:1262 2014-08-12 15:48:57.922581 2014-08-13 17:38:03.055533 7fe9bae887a0 20 read_log 194215'15666 (194215'15665) delete 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.2174595.0:170273 0.000000 2014-08-13 17:38:03.055543 7fe9bae887a0 20 read_log 194215'15667 (193305'15621) clean deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by osd.304.0:1264 2014-08-12 15:48:57.929923 2014-08-13 17:38:03.055555 7fe9bae887a0 20 read_log 194215'15668 (194215'15667) delete deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.2174595.0:170275 0.000000 2014-08-13 17:38:03.055566 7fe9bae887a0 20 read_log 194215'15669 (192834'15190) clean 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by osd.304.0:1266 2014-08-12 15:48:57.937398 2014-08-13 17:38:03.055578 7fe9bae887a0 20 read_log 194215'15670 (194215'15669) delete 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.2174595.0:170277 0.000000 2014-08-13 17:38:03.055589 7fe9bae887a0 20 read_log 194215'15671 (193305'15614) clean b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by osd.304.0:1268 2014-08-12 15:48:57.943909 2014-08-13 17:38:03.055608 7fe9bae887a0 20 read_log 194215'15672 (194215'15671) delete b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.2174595.0:170279 0.000000 2014-08-13 17:38:03.055619 7fe9bae887a0 20 read_log 194215'15673 (192888'15346) clean de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by osd.304.0:1270 2014-08-12 15:48:57.959140 2014-08-13 17:38:03.055651 7fe9bae887a0 20 read_log 194215'15674 (194215'15673) delete de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.2174595.0:170281 0.000000 2014-08-13 17:38:03.055662 7fe9bae887a0 20 read_log 194215'15675 (192888'15535) clean 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by osd.304.0:1272 2014-08-12 15:48:57.965839 2014-08-13 17:38:03.055674 7fe9bae887a0 20 read_log 194215'15676 (194215'15675) delete 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.2174595.0:170283 0.000000 2014-08-13 17:38:03.055685 7fe9bae887a0 20 read_log 194215'15677 (193305'15615) clean fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by osd.304.0:1274 2014-08-12 15:48:57.972358 2014-08-13 17:38:03.055697 7fe9bae887a0 20 read_log 194215'15678 (194215'15677) delete fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.2174595.0:170285 0.000000 2014-08-13 17:38:03.055707 7fe9bae887a0 20 read_log 194215'15679 (192888'15348) clean 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by osd.304.0:1276 2014-08-12 15:48:57.979538 2014-08-13 17:38:03.055719 7fe9bae887a0 20 read_log 194215'15680 (194215'15679) delete 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.2174595.0:170287 0.000000 2014-08-13 17:38:03.055730 7fe9bae887a0 20 read_log 194215'15681 (192834'14625) clean 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by osd.304.0:1278 2014-08-12 15:48:57.986032 2014-08-13 17:38:03.055742 7fe9bae887a0 20 read_log 194215'15682 (194215'15681) delete 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.2174595.0:170289 0.000000 2014-08-13 17:38:03.055753 7fe9bae887a0 20 read_log 194215'15683 (192888'15539) clean 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by osd.304.0:1280 2014-08-12 15:48:57.991517 2014-08-13 17:38:03.055765 7fe9bae887a0 20 read_log 194215'15684 (194215'15683) delete 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.2174595.0:170291 0.000000 2014-08-13 17:38:03.055775 7fe9bae887a0 20 read_log 194215'15685 (192834'15122) clean fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by osd.304.0:1282 2014-08-12 15:48:57.997465 2014-08-13 17:38:03.055787 7fe9bae887a0 20 read_log 194215'15686 (194215'15685) delete fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.2174595.0:170293 0.000000 2014-08-13 17:38:03.055798 7fe9bae887a0 20 read_log 194215'15687 (192888'15360) clean 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by osd.304.0:1284 2014-08-12 15:48:58.017425 2014-08-13 17:38:03.055810 7fe9bae887a0 20 read_log 194215'15688 (194215'15687) delete 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.2174595.0:170295 0.000000 2014-08-13 17:38:03.055821 7fe9bae887a0 20 read_log 194215'15689 (193305'15617) clean 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by osd.304.0:1286 2014-08-12 15:48:58.023333 2014-08-13 17:38:03.055836 7fe9bae887a0 20 read_log 194215'15690 (194215'15689) delete 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.2174595.0:170297 0.000000 2014-08-13 17:38:03.055847 7fe9bae887a0 20 read_log 194215'15691 (192834'14589) clean 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by osd.304.0:1288 2014-08-12 15:48:58.028802 2014-08-13 17:38:03.055869 7fe9bae887a0 20 read_log 194215'15692 (194215'15691) delete 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.2174595.0:170299 0.000000 2014-08-13 17:38:03.055883 7fe9bae887a0 20 read_log 194215'15693 (192888'15499) clean 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by osd.304.0:1290 2014-08-12 15:48:58.035068 2014-08-13 17:38:03.055895 7fe9bae887a0 20 read_log 194215'15694 (194215'15693) delete 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.2174595.0:170301 0.000000 2014-08-13 17:38:03.055906 7fe9bae887a0 20 read_log 194215'15695 (192888'15372) clean 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by osd.304.0:1292 2014-08-12 15:48:58.040580 2014-08-13 17:38:03.055918 7fe9bae887a0 20 read_log 194215'15696 (194215'15695) delete 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.2174595.0:170303 0.000000 2014-08-13 17:38:03.055929 7fe9bae887a0 20 read_log 194215'15697 (192834'15144) clean 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by osd.304.0:1294 2014-08-12 15:48:58.045868 2014-08-13 17:38:03.055941 7fe9bae887a0 20 read_log 194215'15698 (194215'15697) delete 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.2174595.0:170305 0.000000 2014-08-13 17:38:03.055952 7fe9bae887a0 20 read_log 194215'15699 (192888'15501) clean 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by osd.304.0:1296 2014-08-12 15:48:58.051167 2014-08-13 17:38:03.055966 7fe9bae887a0 20 read_log 194215'15700 (194215'15699) delete 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.2174595.0:170307 0.000000 2014-08-13 17:38:03.055977 7fe9bae887a0 20 read_log 194215'15701 (192888'15506) clean 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by osd.304.0:1298 2014-08-12 15:48:58.056637 2014-08-13 17:38:03.055989 7fe9bae887a0 20 read_log 194215'15702 (194215'15701) delete 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.2174595.0:170309 0.000000 2014-08-13 17:38:03.056000 7fe9bae887a0 20 read_log 194215'15703 (192888'15352) clean 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by osd.304.0:1300 2014-08-12 15:48:58.062432 2014-08-13 17:38:03.056015 7fe9bae887a0 20 read_log 194215'15704 (194215'15703) delete 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.2174595.0:170311 0.000000 2014-08-13 17:38:03.056026 7fe9bae887a0 20 read_log 194215'15705 (192888'15508) clean 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by osd.304.0:1302 2014-08-12 15:48:58.068584 2014-08-13 17:38:03.056038 7fe9bae887a0 20 read_log 194215'15706 (194215'15705) delete 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.2174595.0:170313 0.000000 2014-08-13 17:38:03.056048 7fe9bae887a0 20 read_log 194215'15707 (192834'15062) clean 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by osd.304.0:1304 2014-08-12 15:48:58.074133 2014-08-13 17:38:03.056061 7fe9bae887a0 20 read_log 194215'15708 (194215'15707) delete 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.2174595.0:170315 0.000000 2014-08-13 17:38:03.056074 7fe9bae887a0 20 read_log 194215'15709 (192834'15043) clean b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by osd.304.0:1306 2014-08-12 15:48:58.106862 2014-08-13 17:38:03.056089 7fe9bae887a0 20 read_log 194215'15710 (194215'15709) delete b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.2174595.0:170317 0.000000 2014-08-13 17:38:03.056100 7fe9bae887a0 20 read_log 194215'15711 (192834'15109) clean c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by osd.304.0:1308 2014-08-12 15:48:58.121314 2014-08-13 17:38:03.056119 7fe9bae887a0 20 read_log 194215'15712 (194215'15711) delete c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.2174595.0:170319 0.000000 2014-08-13 17:38:03.056132 7fe9bae887a0 20 read_log 194215'15713 (192888'15544) clean f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by osd.304.0:1310 2014-08-12 15:48:58.129344 2014-08-13 17:38:03.056145 7fe9bae887a0 20 read_log 194215'15714 (194215'15713) delete f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.2174595.0:170321 0.000000 2014-08-13 17:38:03.056177 7fe9bae887a0 20 read_log 194215'15715 (192888'15339) clean bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by osd.304.0:1312 2014-08-12 15:48:58.146135 2014-08-13 17:38:03.056193 7fe9bae887a0 20 read_log 194215'15716 (194215'15715) delete bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.2174595.0:170323 0.000000 2014-08-13 17:38:03.056204 7fe9bae887a0 20 read_log 194215'15717 (192834'14556) clean f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by osd.304.0:1314 2014-08-12 15:48:58.153056 2014-08-13 17:38:03.056216 7fe9bae887a0 20 read_log 194215'15718 (194215'15717) delete f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.2174595.0:170325 0.000000 2014-08-13 17:38:03.056227 7fe9bae887a0 20 read_log 194215'15719 (192888'15530) clean 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by osd.304.0:1316 2014-08-12 15:48:58.160084 2014-08-13 17:38:03.056242 7fe9bae887a0 20 read_log 194215'15720 (194215'15719) delete 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.2174595.0:170327 0.000000 2014-08-13 17:38:03.056253 7fe9bae887a0 20 read_log 194215'15721 (192834'15136) clean ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by osd.304.0:1318 2014-08-12 15:48:58.167256 2014-08-13 17:38:03.056265 7fe9bae887a0 20 read_log 194215'15722 (194215'15721) delete ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.2174595.0:170329 0.000000 2014-08-13 17:38:03.056277 7fe9bae887a0 20 read_log 194215'15723 (193305'15613) clean 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by osd.304.0:1320 2014-08-12 15:48:58.174195 2014-08-13 17:38:03.056292 7fe9bae887a0 20 read_log 194215'15724 (194215'15723) delete 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.2174595.0:170331 0.000000 2014-08-13 17:38:03.056303 7fe9bae887a0 20 read_log 194215'15725 (193305'15627) clean 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by osd.304.0:1322 2014-08-12 15:48:58.180789 2014-08-13 17:38:03.056315 7fe9bae887a0 20 read_log 194215'15726 (194215'15725) delete 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.2174595.0:170333 0.000000 2014-08-13 17:38:03.056326 7fe9bae887a0 20 read_log 194215'15727 (193305'15628) clean 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by osd.304.0:1324 2014-08-12 15:48:58.187466 2014-08-13 17:38:03.056338 7fe9bae887a0 20 read_log 194215'15728 (194215'15727) delete 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.2174595.0:170335 0.000000 2014-08-13 17:38:03.056349 7fe9bae887a0 20 read_log 194215'15729 (192888'15538) clean 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by osd.304.0:1326 2014-08-12 15:48:58.194919 2014-08-13 17:38:03.056361 7fe9bae887a0 20 read_log 194215'15730 (194215'15729) delete 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.2174595.0:170337 0.000000 2014-08-13 17:38:03.056372 7fe9bae887a0 20 read_log 194215'15731 (192834'15191) clean d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by osd.304.0:1328 2014-08-12 15:48:58.201320 2014-08-13 17:38:03.056410 7fe9bae887a0 20 read_log 194215'15732 (194215'15731) delete d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.2174595.0:170339 0.000000 2014-08-13 17:38:03.056424 7fe9bae887a0 20 read_log 194215'15733 (192888'15504) clean 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by osd.304.0:1330 2014-08-12 15:48:58.209941 2014-08-13 17:38:03.056437 7fe9bae887a0 20 read_log 194215'15734 (194215'15733) delete 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.2174595.0:170341 0.000000 2014-08-13 17:38:03.056448 7fe9bae887a0 20 read_log 194215'15735 (193305'15620) clean cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by osd.304.0:1332 2014-08-12 15:48:58.217339 2014-08-13 17:38:03.056460 7fe9bae887a0 20 read_log 194215'15736 (194215'15735) delete cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.2174595.0:170343 0.000000 2014-08-13 17:38:03.056471 7fe9bae887a0 20 read_log 194215'15737 (192834'15100) clean 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by osd.304.0:1334 2014-08-12 15:48:58.224422 2014-08-13 17:38:03.056483 7fe9bae887a0 20 read_log 194215'15738 (194215'15737) delete 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.2174595.0:170345 0.000000 2014-08-13 17:38:03.056493 7fe9bae887a0 20 read_log 194215'15739 (192888'15359) clean 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by osd.304.0:1336 2014-08-12 15:48:58.230922 2014-08-13 17:38:03.056505 7fe9bae887a0 20 read_log 194215'15740 (194215'15739) delete 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.2174595.0:170347 0.000000 2014-08-13 17:38:03.056516 7fe9bae887a0 20 read_log 194215'15741 (192888'15343) clean 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by osd.304.0:1338 2014-08-12 15:48:58.239914 2014-08-13 17:38:03.056528 7fe9bae887a0 20 read_log 194215'15742 (194215'15741) delete 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.2174595.0:170349 0.000000 2014-08-13 17:38:03.056539 7fe9bae887a0 20 read_log 194215'15743 (193305'15630) clean c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by osd.304.0:1340 2014-08-12 15:48:58.246803 2014-08-13 17:38:03.056551 7fe9bae887a0 20 read_log 194215'15744 (194215'15743) delete c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.2174595.0:170351 0.000000 2014-08-13 17:38:03.056562 7fe9bae887a0 20 read_log 194215'15745 (192888'15357) clean 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by osd.304.0:1342 2014-08-12 15:48:58.280139 2014-08-13 17:38:03.056574 7fe9bae887a0 20 read_log 194215'15746 (194215'15745) delete 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.2174595.0:170353 0.000000 2014-08-13 17:38:03.056585 7fe9bae887a0 20 read_log 194215'15747 (193305'15633) clean 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by osd.304.0:1344 2014-08-12 15:48:58.288215 2014-08-13 17:38:03.056597 7fe9bae887a0 20 read_log 194215'15748 (194215'15747) delete 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.2174595.0:170355 0.000000 2014-08-13 17:38:03.056607 7fe9bae887a0 20 read_log 194215'15749 (192834'14519) clean 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by osd.304.0:1346 2014-08-12 15:48:58.295819 2014-08-13 17:38:03.056620 7fe9bae887a0 20 read_log 194215'15750 (194215'15749) delete 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.2174595.0:170357 0.000000 2014-08-13 17:38:03.056630 7fe9bae887a0 20 read_log 194215'15751 (192888'15342) clean a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by osd.304.0:1348 2014-08-12 15:48:58.303355 2014-08-13 17:38:03.056652 7fe9bae887a0 20 read_log 194215'15752 (194215'15751) delete a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.2174595.0:170359 0.000000 2014-08-13 17:38:03.056663 7fe9bae887a0 20 read_log 194215'15753 (192834'14596) clean f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by osd.304.0:1350 2014-08-12 15:48:58.311909 2014-08-13 17:38:03.056676 7fe9bae887a0 20 read_log 194215'15754 (194215'15753) delete f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.2174595.0:170361 0.000000 2014-08-13 17:38:03.056686 7fe9bae887a0 20 read_log 194215'15755 (192834'15179) clean 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by osd.304.0:1352 2014-08-12 15:48:58.320308 2014-08-13 17:38:03.056698 7fe9bae887a0 20 read_log 194215'15756 (194215'15755) delete 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.2174595.0:170363 0.000000 2014-08-13 17:38:03.056709 7fe9bae887a0 20 read_log 194215'15757 (192888'15540) clean a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by osd.304.0:1354 2014-08-12 15:48:58.329425 2014-08-13 17:38:03.056721 7fe9bae887a0 20 read_log 194215'15758 (194215'15757) delete a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.2174595.0:170365 0.000000 2014-08-13 17:38:03.056731 7fe9bae887a0 20 read_log 194215'15759 (193305'15625) clean 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by osd.304.0:1356 2014-08-12 15:48:58.336883 2014-08-13 17:38:03.056743 7fe9bae887a0 20 read_log 194215'15760 (194215'15759) delete 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.2174595.0:170367 0.000000 2014-08-13 17:38:03.056754 7fe9bae887a0 20 read_log 194215'15761 (192888'15527) clean af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by osd.304.0:1358 2014-08-12 15:48:58.344307 2014-08-13 17:38:03.056766 7fe9bae887a0 20 read_log 194215'15762 (194215'15761) delete af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.2174595.0:170369 0.000000 2014-08-13 17:38:03.056776 7fe9bae887a0 20 read_log 194215'15763 (192888'15532) clean 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by osd.304.0:1360 2014-08-12 15:48:58.351826 2014-08-13 17:38:03.056789 7fe9bae887a0 20 read_log 194215'15764 (194215'15763) delete 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.2174595.0:170371 0.000000 2014-08-13 17:38:03.056799 7fe9bae887a0 20 read_log 194215'15765 (192834'15072) clean f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by osd.304.0:1362 2014-08-12 15:48:58.359632 2014-08-13 17:38:03.056811 7fe9bae887a0 20 read_log 194215'15766 (194215'15765) delete f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.2174595.0:170373 0.000000 2014-08-13 17:38:03.056822 7fe9bae887a0 20 read_log 194215'15767 (193305'15622) clean 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by osd.304.0:1364 2014-08-12 15:48:58.366981 2014-08-13 17:38:03.056834 7fe9bae887a0 20 read_log 194215'15768 (194215'15767) delete 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.2174595.0:170375 0.000000 2014-08-13 17:38:03.056844 7fe9bae887a0 20 read_log 194215'15769 (192834'14532) clean 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by osd.304.0:1366 2014-08-12 15:48:58.373560 2014-08-13 17:38:03.056856 7fe9bae887a0 20 read_log 194215'15770 (194215'15769) delete 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.2174595.0:170377 0.000000 2014-08-13 17:38:03.056867 7fe9bae887a0 20 read_log 194215'15771 (193305'15616) clean 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by osd.304.0:1368 2014-08-12 15:48:58.380613 2014-08-13 17:38:03.056888 7fe9bae887a0 20 read_log 194215'15772 (194215'15771) delete 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.2174595.0:170379 0.000000 2014-08-13 17:38:03.056902 7fe9bae887a0 20 read_log 194215'15773 (192834'15076) clean 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by osd.304.0:1370 2014-08-12 15:48:58.387695 2014-08-13 17:38:03.056914 7fe9bae887a0 20 read_log 194215'15774 (194215'15773) delete 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.2174595.0:170381 0.000000 2014-08-13 17:38:03.056926 7fe9bae887a0 20 read_log 194215'15775 (192888'15333) clean eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by osd.304.0:1372 2014-08-12 15:48:58.395329 2014-08-13 17:38:03.056939 7fe9bae887a0 20 read_log 194215'15776 (194215'15775) delete eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.2174595.0:170383 0.000000 2014-08-13 17:38:03.056950 7fe9bae887a0 20 read_log 194215'15777 (192834'14533) clean fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by osd.304.0:1374 2014-08-12 15:48:58.403242 2014-08-13 17:38:03.056962 7fe9bae887a0 20 read_log 194215'15778 (194215'15777) delete fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.2174595.0:170385 0.000000 2014-08-13 17:38:03.056973 7fe9bae887a0 20 read_log 194215'15779 (193305'15635) clean 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by osd.304.0:1376 2014-08-12 15:48:58.410808 2014-08-13 17:38:03.057004 7fe9bae887a0 20 read_log 194215'15780 (194215'15779) delete 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.2174595.0:170387 0.000000 2014-08-13 17:38:03.057019 7fe9bae887a0 20 read_log 194215'15781 (192888'15362) clean c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by osd.304.0:1378 2014-08-12 15:48:58.418577 2014-08-13 17:38:03.057032 7fe9bae887a0 20 read_log 194215'15782 (194215'15781) delete c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.2174595.0:170389 0.000000 2014-08-13 17:38:03.057043 7fe9bae887a0 20 read_log 194215'15783 (192888'15329) clean 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by osd.304.0:1380 2014-08-12 15:48:58.427827 2014-08-13 17:38:03.057057 7fe9bae887a0 20 read_log 194215'15784 (194215'15783) delete 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.2174595.0:170391 0.000000 2014-08-13 17:38:03.057068 7fe9bae887a0 20 read_log 194215'15785 (192834'15108) clean 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by osd.304.0:1382 2014-08-12 15:48:58.434875 2014-08-13 17:38:03.057080 7fe9bae887a0 20 read_log 194215'15786 (194215'15785) delete 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.2174595.0:170393 0.000000 2014-08-13 17:38:03.057090 7fe9bae887a0 20 read_log 194215'15787 (192834'15085) clean 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by osd.304.0:1384 2014-08-12 15:48:58.441839 2014-08-13 17:38:03.057105 7fe9bae887a0 20 read_log 194215'15788 (194215'15787) delete 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.2174595.0:170395 0.000000 2014-08-13 17:38:03.057116 7fe9bae887a0 20 read_log 194215'15789 (192834'14572) clean 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by osd.304.0:1386 2014-08-12 15:48:58.448941 2014-08-13 17:38:03.057128 7fe9bae887a0 20 read_log 194215'15790 (194215'15789) delete 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.2174595.0:170397 0.000000 2014-08-13 17:38:03.057139 7fe9bae887a0 20 read_log 194215'15791 (193305'15626) clean 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by osd.304.0:1388 2014-08-12 15:48:58.456424 2014-08-13 17:38:03.057165 7fe9bae887a0 20 read_log 194215'15792 (194215'15791) delete 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.2174595.0:170399 0.000000 2014-08-13 17:38:03.057178 7fe9bae887a0 20 read_log 194215'15793 (193305'15611) clean 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by osd.304.0:1390 2014-08-12 15:48:58.463909 2014-08-13 17:38:03.057191 7fe9bae887a0 20 read_log 194215'15794 (194215'15793) delete 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.2174595.0:170401 0.000000 2014-08-13 17:38:03.057202 7fe9bae887a0 20 read_log 194215'15795 (192834'15125) clean 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by osd.304.0:1392 2014-08-12 15:48:58.471543 2014-08-13 17:38:03.057216 7fe9bae887a0 20 read_log 194215'15796 (194215'15795) delete 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.2174595.0:170403 0.000000 2014-08-13 17:38:03.057227 7fe9bae887a0 20 read_log 194215'15797 (192888'15324) clean 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by osd.304.0:1394 2014-08-12 15:48:58.479124 2014-08-13 17:38:03.057239 7fe9bae887a0 20 read_log 194215'15798 (194215'15797) delete 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.2174595.0:170405 0.000000 2014-08-13 17:38:03.057250 7fe9bae887a0 20 read_log 194215'15799 (192888'15500) clean f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by osd.304.0:1396 2014-08-12 15:48:58.486422 2014-08-13 17:38:03.057262 7fe9bae887a0 20 read_log 194215'15800 (194215'15799) delete f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.2174595.0:170407 0.000000 2014-08-13 17:38:03.057273 7fe9bae887a0 20 read_log 194215'15801 (192834'14651) clean 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by osd.304.0:1398 2014-08-12 15:48:58.493415 2014-08-13 17:38:03.057288 7fe9bae887a0 20 read_log 194215'15802 (194215'15801) delete 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.2174595.0:170409 0.000000 2014-08-13 17:38:03.057299 7fe9bae887a0 20 read_log 194215'15803 (192888'15529) clean 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by osd.304.0:1400 2014-08-12 15:48:58.500261 2014-08-13 17:38:03.057311 7fe9bae887a0 20 read_log 194215'15804 (194215'15803) delete 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.2174595.0:170411 0.000000 2014-08-13 17:38:03.057321 7fe9bae887a0 20 read_log 194215'15805 (192834'15060) clean f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by osd.304.0:1402 2014-08-12 15:48:58.507343 2014-08-13 17:38:03.057333 7fe9bae887a0 20 read_log 194215'15806 (194215'15805) delete f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.2174595.0:170413 0.000000 2014-08-13 17:38:03.057344 7fe9bae887a0 20 read_log 194215'15807 (193305'15640) clean 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by osd.304.0:1404 2014-08-12 15:48:58.514514 2014-08-13 17:38:03.057372 7fe9bae887a0 20 read_log 194215'15808 (194215'15807) delete 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.2174595.0:170415 0.000000 2014-08-13 17:38:03.057386 7fe9bae887a0 20 read_log 194215'15809 (192888'15370) clean 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by osd.304.0:1406 2014-08-12 15:48:58.522291 2014-08-13 17:38:03.057398 7fe9bae887a0 20 read_log 194215'15810 (194215'15809) delete 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.2174595.0:170417 0.000000 2014-08-13 17:38:03.057409 7fe9bae887a0 20 read_log 194215'15811 (192888'15335) clean 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by osd.304.0:1408 2014-08-12 15:48:58.529519 2014-08-13 17:38:03.057430 7fe9bae887a0 20 read_log 194215'15812 (194215'15811) delete 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.2174595.0:170419 0.000000 2014-08-13 17:38:03.057444 7fe9bae887a0 20 read_log 194215'15813 (192834'15079) clean a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by osd.304.0:1410 2014-08-12 15:48:58.536928 2014-08-13 17:38:03.057456 7fe9bae887a0 20 read_log 194215'15814 (194215'15813) delete a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.2174595.0:170421 0.000000 2014-08-13 17:38:03.057467 7fe9bae887a0 20 read_log 194215'15815 (192834'15104) clean 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by osd.304.0:1412 2014-08-12 15:48:58.543570 2014-08-13 17:38:03.057479 7fe9bae887a0 20 read_log 194215'15816 (194215'15815) delete 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.2174595.0:170423 0.000000 2014-08-13 17:38:03.057489 7fe9bae887a0 20 read_log 194215'15817 (192834'15037) clean 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by osd.304.0:1414 2014-08-12 15:48:58.550359 2014-08-13 17:38:03.057502 7fe9bae887a0 20 read_log 194215'15818 (194215'15817) delete 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.2174595.0:170425 0.000000 2014-08-13 17:38:03.057512 7fe9bae887a0 20 read_log 194215'15819 (192834'14550) clean 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by osd.304.0:1416 2014-08-12 15:48:58.557762 2014-08-13 17:38:03.057524 7fe9bae887a0 20 read_log 194215'15820 (194215'15819) delete 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.2174595.0:170427 0.000000 2014-08-13 17:38:03.057535 7fe9bae887a0 20 read_log 194215'15821 (193305'15619) clean 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by osd.304.0:1418 2014-08-12 15:48:58.564935 2014-08-13 17:38:03.057547 7fe9bae887a0 20 read_log 194215'15822 (194215'15821) delete 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.2174595.0:170429 0.000000 2014-08-13 17:38:03.057557 7fe9bae887a0 20 read_log 194215'15823 (193305'15618) clean 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by osd.304.0:1420 2014-08-12 15:48:58.572018 2014-08-13 17:38:03.057569 7fe9bae887a0 20 read_log 194215'15824 (194215'15823) delete 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.2174595.0:170431 0.000000 2014-08-13 17:38:03.057580 7fe9bae887a0 20 read_log 194215'15825 (192834'14616) clean 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by osd.304.0:1422 2014-08-12 15:48:58.579378 2014-08-13 17:38:03.057592 7fe9bae887a0 20 read_log 194215'15826 (194215'15825) delete 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.2174595.0:170433 0.000000 2014-08-13 17:38:03.057602 7fe9bae887a0 20 read_log 194215'15827 (192888'15327) clean be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by osd.304.0:1424 2014-08-12 15:48:58.586255 2014-08-13 17:38:03.057614 7fe9bae887a0 20 read_log 194215'15828 (194215'15827) delete be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.2174595.0:170435 0.000000 2014-08-13 17:38:03.057625 7fe9bae887a0 20 read_log 194215'15829 (192834'14587) clean f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by osd.304.0:1426 2014-08-12 15:48:58.594131 2014-08-13 17:38:03.057637 7fe9bae887a0 20 read_log 194215'15830 (194215'15829) delete f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.2174595.0:170437 0.000000 2014-08-13 17:38:03.057651 7fe9bae887a0 20 read_log 194215'15831 (192888'15371) clean b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by osd.304.0:1428 2014-08-12 15:48:58.601172 2014-08-13 17:38:03.057670 7fe9bae887a0 20 read_log 194215'15832 (194215'15831) delete b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.2174595.0:170439 0.000000 2014-08-13 17:38:03.057681 7fe9bae887a0 20 read_log 194215'15833 (192888'15505) clean 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by osd.304.0:1430 2014-08-12 15:48:58.609991 2014-08-13 17:38:03.057694 7fe9bae887a0 20 read_log 194215'15834 (194215'15833) delete 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.2174595.0:170441 0.000000 2014-08-13 17:38:03.057704 7fe9bae887a0 20 read_log 194215'15835 (193305'15638) clean 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by osd.304.0:1432 2014-08-12 15:48:58.617416 2014-08-13 17:38:03.057716 7fe9bae887a0 20 read_log 194215'15836 (194215'15835) delete 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.2174595.0:170443 0.000000 2014-08-13 17:38:03.057727 7fe9bae887a0 20 read_log 194215'15837 (193305'15637) clean f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by osd.304.0:1434 2014-08-12 15:48:58.624562 2014-08-13 17:38:03.057739 7fe9bae887a0 20 read_log 194215'15838 (194215'15837) delete f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.2174595.0:170445 0.000000 2014-08-13 17:38:03.057750 7fe9bae887a0 20 read_log 194215'15839 (192888'15374) clean 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by osd.304.0:1436 2014-08-12 15:48:58.646026 2014-08-13 17:38:03.057761 7fe9bae887a0 20 read_log 194215'15840 (194215'15839) delete 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.2174595.0:170447 0.000000 2014-08-13 17:38:03.057772 7fe9bae887a0 20 read_log 194215'15841 (193305'15631) clean 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by osd.304.0:1438 2014-08-12 15:48:58.652808 2014-08-13 17:38:03.057784 7fe9bae887a0 20 read_log 194215'15842 (194215'15841) delete 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.2174595.0:170449 0.000000 2014-08-13 17:38:03.057795 7fe9bae887a0 20 read_log 194215'15843 (193305'15641) clean 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by osd.304.0:1440 2014-08-12 15:48:58.659971 2014-08-13 17:38:03.057807 7fe9bae887a0 20 read_log 194215'15844 (194215'15843) delete 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.2174595.0:170451 0.000000 2014-08-13 17:38:03.057817 7fe9bae887a0 20 read_log 194215'15845 (192888'15546) clean 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by osd.304.0:1442 2014-08-12 15:48:58.667391 2014-08-13 17:38:03.057829 7fe9bae887a0 20 read_log 194215'15846 (194215'15845) delete 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.2174595.0:170453 0.000000 2014-08-13 17:38:03.057840 7fe9bae887a0 20 read_log 194215'15847 (192888'15340) clean b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by osd.304.0:1444 2014-08-12 15:48:58.674258 2014-08-13 17:38:03.057852 7fe9bae887a0 20 read_log 194215'15848 (194215'15847) delete b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.2174595.0:170455 0.000000 2014-08-13 17:38:03.057863 7fe9bae887a0 20 read_log 194215'15849 (192834'15065) clean 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by osd.304.0:1446 2014-08-12 15:48:58.682531 2014-08-13 17:38:03.057875 7fe9bae887a0 20 read_log 194215'15850 (194215'15849) delete 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.2174595.0:170457 0.000000 2014-08-13 17:38:03.057885 7fe9bae887a0 20 read_log 194215'15851 (192888'15356) clean be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by osd.304.0:1448 2014-08-12 15:48:58.690000 2014-08-13 17:38:03.057904 7fe9bae887a0 20 read_log 194215'15852 (194215'15851) delete be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.2174595.0:170459 0.000000 2014-08-13 17:38:03.057915 7fe9bae887a0 20 read_log 194215'15853 (192888'15524) clean b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by osd.304.0:1450 2014-08-12 15:48:58.697855 2014-08-13 17:38:03.057929 7fe9bae887a0 20 read_log 194215'15854 (194215'15853) delete b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.2174595.0:170461 0.000000 2014-08-13 17:38:03.057940 7fe9bae887a0 20 read_log 194215'15855 (192834'15044) clean 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by osd.304.0:1452 2014-08-12 15:48:58.705384 2014-08-13 17:38:03.057952 7fe9bae887a0 20 read_log 194215'15856 (194215'15855) delete 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.2174595.0:170463 0.000000 2014-08-13 17:38:03.057963 7fe9bae887a0 20 read_log 194215'15857 (192834'14622) clean c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by osd.304.0:1454 2014-08-12 15:48:58.714083 2014-08-13 17:38:03.057975 7fe9bae887a0 20 read_log 194215'15858 (194215'15857) delete c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.2174595.0:170465 0.000000 2014-08-13 17:38:03.057986 7fe9bae887a0 20 read_log 194215'15859 (192834'14592) clean d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by osd.304.0:1456 2014-08-12 15:48:58.721567 2014-08-13 17:38:03.057998 7fe9bae887a0 20 read_log 194215'15860 (194215'15859) delete d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.2174595.0:170467 0.000000 2014-08-13 17:38:03.058009 7fe9bae887a0 20 read_log 194215'15861 (193305'15642) clean dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by osd.304.0:1458 2014-08-12 15:48:58.729123 2014-08-13 17:38:03.058020 7fe9bae887a0 20 read_log 194215'15862 (194215'15861) delete dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.2174595.0:170469 0.000000 2014-08-13 17:38:03.058031 7fe9bae887a0 20 read_log 194215'15863 (192888'15378) clean ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by osd.304.0:1460 2014-08-12 15:48:58.737016 2014-08-13 17:38:03.058043 7fe9bae887a0 20 read_log 194215'15864 (194215'15863) delete ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.2174595.0:170471 0.000000 2014-08-13 17:38:03.058054 7fe9bae887a0 20 read_log 194215'15865 (192834'15128) clean ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by osd.304.0:1462 2014-08-12 15:48:58.744069 2014-08-13 17:38:03.058065 7fe9bae887a0 20 read_log 194215'15866 (194215'15865) delete ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.2174595.0:170473 0.000000 2014-08-13 17:38:03.058076 7fe9bae887a0 20 read_log 194215'15867 (192834'15066) clean f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by osd.304.0:1464 2014-08-12 15:48:58.751891 2014-08-13 17:38:03.058088 7fe9bae887a0 20 read_log 194215'15868 (194215'15867) delete f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.2174595.0:170475 0.000000 2014-08-13 17:38:03.058098 7fe9bae887a0 20 read_log 194215'15869 (192888'15338) clean a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by osd.304.0:1466 2014-08-12 15:48:58.758510 2014-08-13 17:38:03.058110 7fe9bae887a0 20 read_log 194215'15870 (194215'15869) delete a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.2174595.0:170477 0.000000 2014-08-13 17:38:03.058121 7fe9bae887a0 20 read_log 194215'15871 (192834'14555) clean 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by osd.304.0:1468 2014-08-12 15:48:58.765770 2014-08-13 17:38:03.058164 7fe9bae887a0 20 read_log 194215'15872 (194215'15871) delete 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.2174595.0:170479 0.000000 2014-08-13 17:38:03.058179 7fe9bae887a0 20 read_log 194215'15873 (193305'15632) clean 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by osd.304.0:1470 2014-08-12 15:48:58.773086 2014-08-13 17:38:03.058192 7fe9bae887a0 20 read_log 194215'15874 (194215'15873) delete 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.2174595.0:170481 0.000000 2014-08-13 17:38:03.058205 7fe9bae887a0 20 read_log 194215'15875 (192888'15547) clean 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by osd.304.0:1472 2014-08-12 15:48:58.781181 2014-08-13 17:38:03.058219 7fe9bae887a0 20 read_log 194215'15876 (194215'15875) delete 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.2174595.0:170483 0.000000 2014-08-13 17:38:03.058230 7fe9bae887a0 20 read_log 194215'15877 (192888'15381) clean d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by osd.304.0:1474 2014-08-12 15:48:58.789086 2014-08-13 17:38:03.058242 7fe9bae887a0 20 read_log 194215'15878 (194215'15877) delete d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.2174595.0:170485 0.000000 2014-08-13 17:38:03.058253 7fe9bae887a0 20 read_log 194215'15879 (192834'15042) clean 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by osd.304.0:1476 2014-08-12 15:48:58.796702 2014-08-13 17:38:03.058265 7fe9bae887a0 20 read_log 194215'15880 (194215'15879) delete 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.2174595.0:170487 0.000000 2014-08-13 17:38:03.058275 7fe9bae887a0 20 read_log 194215'15881 (192888'15358) clean a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by osd.304.0:1478 2014-08-12 15:48:58.804620 2014-08-13 17:38:03.058287 7fe9bae887a0 20 read_log 194215'15882 (194215'15881) delete a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.2174595.0:170489 0.000000 2014-08-13 17:38:03.058298 7fe9bae887a0 20 read_log 194215'15883 (192834'15185) clean f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by osd.304.0:1480 2014-08-12 15:48:58.812597 2014-08-13 17:38:03.058310 7fe9bae887a0 20 read_log 194215'15884 (194215'15883) delete f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.2174595.0:170491 0.000000 2014-08-13 17:38:03.058322 7fe9bae887a0 20 read_log 194215'15885 (192834'15173) clean b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by osd.304.0:1482 2014-08-12 15:48:58.820753 2014-08-13 17:38:03.058334 7fe9bae887a0 20 read_log 194215'15886 (194215'15885) delete b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.2174595.0:170493 0.000000 2014-08-13 17:38:03.058345 7fe9bae887a0 20 read_log 194215'15887 (192888'15341) clean 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by osd.304.0:1484 2014-08-12 15:48:58.828071 2014-08-13 17:38:03.058357 7fe9bae887a0 20 read_log 194215'15888 (194215'15887) delete 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.2174595.0:170495 0.000000 2014-08-13 17:38:03.058368 7fe9bae887a0 20 read_log 194215'15889 (192834'14585) clean 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by osd.304.0:1486 2014-08-12 15:48:58.835089 2014-08-13 17:38:03.058380 7fe9bae887a0 20 read_log 194215'15890 (194215'15889) delete 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.2174595.0:170497 0.000000 2014-08-13 17:38:03.058391 7fe9bae887a0 20 read_log 194215'15891 (193305'15636) clean c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by osd.304.0:1488 2014-08-12 15:48:58.842880 2014-08-13 17:38:03.058410 7fe9bae887a0 20 read_log 194215'15892 (194215'15891) delete c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.2174595.0:170499 0.000000 2014-08-13 17:38:03.058421 7fe9bae887a0 20 read_log 194215'15893 (192888'15365) clean 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by osd.304.0:1490 2014-08-12 15:48:58.850448 2014-08-13 17:38:03.058433 7fe9bae887a0 20 read_log 194215'15894 (194215'15893) delete 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.2174595.0:170501 0.000000 2014-08-13 17:38:03.058444 7fe9bae887a0 20 read_log 194215'15895 (192888'15382) clean 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by osd.304.0:1492 2014-08-12 15:48:58.857113 2014-08-13 17:38:03.058456 7fe9bae887a0 20 read_log 194215'15896 (194215'15895) delete 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.2174595.0:170503 0.000000 2014-08-13 17:38:03.058466 7fe9bae887a0 20 read_log 194215'15897 (193305'15629) clean 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by osd.304.0:1494 2014-08-12 15:48:58.864247 2014-08-13 17:38:03.058478 7fe9bae887a0 20 read_log 194215'15898 (194215'15897) delete 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.2174595.0:170505 0.000000 2014-08-13 17:38:03.058489 7fe9bae887a0 20 read_log 194215'15899 (192834'15193) clean 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by osd.304.0:1496 2014-08-12 15:48:58.871743 2014-08-13 17:38:03.058501 7fe9bae887a0 20 read_log 194215'15900 (194215'15899) delete 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.2174595.0:170507 0.000000 2014-08-13 17:38:03.058530 7fe9bae887a0 20 read_log 194215'15901 (192834'15132) clean 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by osd.304.0:1498 2014-08-12 15:48:58.879492 2014-08-13 17:38:03.058546 7fe9bae887a0 20 read_log 194215'15902 (194215'15901) delete 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.2174595.0:170509 0.000000 2014-08-13 17:38:03.058557 7fe9bae887a0 20 read_log 194215'15903 (192834'14684) clean 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by osd.304.0:1500 2014-08-12 15:48:58.886606 2014-08-13 17:38:03.058569 7fe9bae887a0 20 read_log 194215'15904 (194215'15903) delete 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.2174595.0:170511 0.000000 2014-08-13 17:38:03.058580 7fe9bae887a0 20 read_log 194215'15905 (192888'15545) clean c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by osd.304.0:1502 2014-08-12 15:48:58.894488 2014-08-13 17:38:03.058592 7fe9bae887a0 20 read_log 194215'15906 (194215'15905) delete c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.2174595.0:170513 0.000000 2014-08-13 17:38:03.058603 7fe9bae887a0 20 read_log 194215'15907 (192888'15515) clean 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by osd.304.0:1504 2014-08-12 15:48:58.902234 2014-08-13 17:38:03.058616 7fe9bae887a0 20 read_log 194215'15908 (194215'15907) delete 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.2174595.0:170515 0.000000 2014-08-13 17:38:03.058628 7fe9bae887a0 20 read_log 194215'15909 (192834'14682) clean 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by osd.304.0:1506 2014-08-12 15:48:58.910205 2014-08-13 17:38:03.058640 7fe9bae887a0 20 read_log 194215'15910 (194215'15909) delete 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.2174595.0:170517 0.000000 2014-08-13 17:38:03.058651 7fe9bae887a0 20 read_log 194215'15911 (192888'15542) clean f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by osd.304.0:1508 2014-08-12 15:48:58.917821 2014-08-13 17:38:03.058671 7fe9bae887a0 20 read_log 194215'15912 (194215'15911) delete f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.2174595.0:170519 0.000000 2014-08-13 17:38:03.058683 7fe9bae887a0 20 read_log 194217'15913 (194217'15913) modify 33a/hit_set_24.33a_archive_2014-08-12 15:48:57.821312_2014-08-12 17:25:04.496076/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 17:38:03.058695 7fe9bae887a0 20 read_log 194217'15914 (194215'15643) delete 33a/hit_set_24.33a_archive_2014-08-12 15:32:25.735124_2014-08-12 15:48:57.821063/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 17:38:03.058708 7fe9bae887a0 20 read_log 194217'15915 (0'0) promote d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by osd.304.0:2262 2014-08-12 17:25:04.499120 2014-08-13 17:38:03.058720 7fe9bae887a0 20 read_log 194217'15916 (194217'15915) modify d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by client.1487927.0:307 2014-08-12 17:25:04.489405 2014-08-13 17:38:03.058732 7fe9bae887a0 20 read_log 194217'15917 (0'0) promote 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by osd.304.0:2266 2014-08-12 17:25:05.614118 2014-08-13 17:38:03.058744 7fe9bae887a0 20 read_log 194217'15918 (194217'15917) modify 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by client.1487927.0:434 2014-08-12 17:25:05.605946 2014-08-13 17:38:03.058759 7fe9bae887a0 20 read_log 194370'15919 (0'0) promote 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by osd.304.0:2272 2014-08-12 20:30:15.868992 2014-08-13 17:38:03.058771 7fe9bae887a0 20 read_log 194370'15920 (194370'15919) modify 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by client.1508531.0:392 2014-08-12 20:30:15.862085 2014-08-13 17:38:03.058783 7fe9bae887a0 20 read_log 194370'15921 (0'0) promote 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by osd.304.0:2290 2014-08-12 20:30:20.189642 2014-08-13 17:38:03.058796 7fe9bae887a0 20 read_log 194370'15922 (194370'15921) modify 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by client.1508531.0:1547 2014-08-12 20:30:20.183075 2014-08-13 17:38:03.058809 7fe9bae887a0 20 read_log 194370'15923 (0'0) promote 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by osd.304.0:2318 2014-08-12 20:30:29.623516 2014-08-13 17:38:03.058824 7fe9bae887a0 20 read_log 194370'15924 (194370'15923) modify 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by client.1508531.0:4089 2014-08-12 20:30:29.617119 2014-08-13 17:38:03.058836 7fe9bae887a0 20 read_log 194370'15925 (0'0) promote 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by osd.304.0:2346 2014-08-12 20:30:36.419403 2014-08-13 17:38:03.058851 7fe9bae887a0 20 read_log 194370'15926 (194370'15925) modify 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by client.1508531.0:5714 2014-08-12 20:30:36.411349 2014-08-13 17:38:03.058863 7fe9bae887a0 20 read_log 194370'15927 (0'0) promote 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by osd.304.0:2356 2014-08-12 20:30:39.333156 2014-08-13 17:38:03.058877 7fe9bae887a0 20 read_log 194370'15928 (194370'15927) modify 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by client.1508531.0:6368 2014-08-12 20:30:39.326068 2014-08-13 17:38:03.058889 7fe9bae887a0 20 read_log 194370'15929 (0'0) promote 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by osd.304.0:2358 2014-08-12 20:30:39.903464 2014-08-13 17:38:03.058903 7fe9bae887a0 20 read_log 194370'15930 (194370'15929) modify 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by client.1508531.0:6487 2014-08-12 20:30:39.894743 2014-08-13 17:38:03.058927 7fe9bae887a0 20 read_log 194370'15931 (0'0) promote fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by osd.304.0:2374 2014-08-12 20:30:49.259262 2014-08-13 17:38:03.058942 7fe9bae887a0 20 read_log 194370'15932 (194370'15931) modify fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by client.1508531.0:8317 2014-08-12 20:30:49.252693 2014-08-13 17:38:03.058954 7fe9bae887a0 20 read_log 194370'15933 (0'0) promote 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by osd.304.0:2400 2014-08-12 20:31:07.746055 2014-08-13 17:38:03.058967 7fe9bae887a0 20 read_log 194370'15934 (194370'15933) modify 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by client.1508531.0:11403 2014-08-12 20:31:07.740071 2014-08-13 17:38:03.058982 7fe9bae887a0 20 read_log 194370'15935 (0'0) promote a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by osd.304.0:2402 2014-08-12 20:31:07.896567 2014-08-13 17:38:03.058994 7fe9bae887a0 20 read_log 194370'15936 (194370'15935) modify a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by client.1508531.0:11430 2014-08-12 20:31:07.890091 2014-08-13 17:38:03.059008 7fe9bae887a0 20 read_log 194370'15937 (0'0) promote 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by osd.304.0:2408 2014-08-12 20:31:12.899146 2014-08-13 17:38:03.059021 7fe9bae887a0 20 read_log 194370'15938 (194370'15937) modify 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by client.1508531.0:12208 2014-08-12 20:31:12.891369 2014-08-13 17:38:03.059033 7fe9bae887a0 20 read_log 194370'15939 (0'0) promote 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by osd.304.0:2414 2014-08-12 20:31:14.757246 2014-08-13 17:38:03.059045 7fe9bae887a0 20 read_log 194370'15940 (194370'15939) modify 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by client.1508531.0:12466 2014-08-12 20:31:14.749585 2014-08-13 17:38:03.059057 7fe9bae887a0 20 read_log 194377'15941 (0'0) promote 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by osd.304.0:2440 2014-08-12 20:33:04.737896 2014-08-13 17:38:03.059071 7fe9bae887a0 20 read_log 194377'15942 (194377'15941) modify 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by client.2231190.0:1883 2014-08-12 20:33:04.731658 2014-08-13 17:38:03.059084 7fe9bae887a0 20 read_log 194377'15943 (0'0) promote 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by osd.304.0:2462 2014-08-12 20:33:12.309625 2014-08-13 17:38:03.059096 7fe9bae887a0 20 read_log 194377'15944 (194377'15943) modify 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by client.2231190.0:3695 2014-08-12 20:33:12.304340 2014-08-13 17:38:03.059113 7fe9bae887a0 20 read_log 194377'15945 (0'0) promote e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by osd.304.0:2470 2014-08-12 20:33:14.997340 2014-08-13 17:38:03.059126 7fe9bae887a0 20 read_log 194377'15946 (194377'15945) modify e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by client.2231190.0:4182 2014-08-12 20:33:14.989295 2014-08-13 17:38:03.059139 7fe9bae887a0 20 read_log 194383'15947 (0'0) promote b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by osd.304.0:2488 2014-08-12 20:34:10.323439 2014-08-13 17:38:03.059165 7fe9bae887a0 20 read_log 194383'15948 (194383'15947) modify b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by client.2231190.0:6822 2014-08-12 20:34:10.315611 2014-08-13 17:38:03.059181 7fe9bae887a0 20 read_log 194383'15949 (0'0) promote 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by osd.304.0:2490 2014-08-12 20:34:10.846295 2014-08-13 17:38:03.059193 7fe9bae887a0 20 read_log 194383'15950 (194383'15949) modify 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by client.2231190.0:6841 2014-08-12 20:34:10.839540 2014-08-13 17:38:03.059215 7fe9bae887a0 20 read_log 194383'15951 (0'0) promote 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by osd.304.0:2508 2014-08-12 20:34:36.974881 2014-08-13 17:38:03.059230 7fe9bae887a0 20 read_log 194383'15952 (194383'15951) modify 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by client.2231190.0:7792 2014-08-12 20:34:36.968217 2014-08-13 17:38:03.059243 7fe9bae887a0 20 read_log 194393'15953 (0'0) promote 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by osd.304.0:2510 2014-08-12 20:47:25.958135 2014-08-13 17:38:03.059257 7fe9bae887a0 20 read_log 194393'15954 (194393'15953) modify 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by client.1064095.0:18 2014-08-12 20:47:25.949889 2014-08-13 17:38:03.059269 7fe9bae887a0 20 read_log 194397'15955 (0'0) promote 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by osd.304.0:2542 2014-08-12 20:53:06.101508 2014-08-13 17:38:03.059284 7fe9bae887a0 20 read_log 194397'15956 (194397'15955) modify 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by client.2235069.0:926 2014-08-12 20:53:06.093281 2014-08-13 17:38:03.059297 7fe9bae887a0 20 read_log 194416'15957 (0'0) promote fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by osd.304.0:2636 2014-08-13 15:28:30.857506 2014-08-13 17:38:03.059309 7fe9bae887a0 20 read_log 194416'15958 (194416'15957) modify fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by client.2441958.0:175 2014-08-13 15:28:30.846495 2014-08-13 17:38:03.059321 7fe9bae887a0 20 read_log 194416'15959 (0'0) promote bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by osd.304.0:2641 2014-08-13 15:28:34.025376 2014-08-13 17:38:03.059333 7fe9bae887a0 20 read_log 194416'15960 (194416'15959) modify bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by client.2441958.0:886 2014-08-13 15:28:34.018559 2014-08-13 17:38:03.059346 7fe9bae887a0 20 read_log 194416'15961 (0'0) promote 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by osd.304.0:2648 2014-08-13 15:28:36.112854 2014-08-13 17:38:03.059358 7fe9bae887a0 20 read_log 194416'15962 (194416'15961) modify 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by client.2441958.0:1421 2014-08-13 15:28:36.105385 2014-08-13 17:38:03.059370 7fe9bae887a0 20 read_log 194416'15963 (0'0) promote 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by osd.304.0:2650 2014-08-13 15:28:36.397928 2014-08-13 17:38:03.059382 7fe9bae887a0 20 read_log 194416'15964 (194416'15963) modify 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by client.2441958.0:1498 2014-08-13 15:28:36.390188 2014-08-13 17:38:03.059394 7fe9bae887a0 20 read_log 194416'15965 (0'0) promote e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by osd.304.0:2657 2014-08-13 15:28:40.177505 2014-08-13 17:38:03.059406 7fe9bae887a0 20 read_log 194416'15966 (194416'15965) modify e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by client.2441958.0:2498 2014-08-13 15:28:40.168526 2014-08-13 17:38:03.059418 7fe9bae887a0 20 read_log 194416'15967 (0'0) promote d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by osd.304.0:2661 2014-08-13 15:28:40.475143 2014-08-13 17:38:03.059430 7fe9bae887a0 20 read_log 194416'15968 (194416'15967) modify d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by client.2441958.0:2576 2014-08-13 15:28:40.467172 2014-08-13 17:38:03.059442 7fe9bae887a0 20 read_log 194416'15969 (0'0) promote da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by osd.304.0:2670 2014-08-13 15:28:45.984364 2014-08-13 17:38:03.059461 7fe9bae887a0 20 read_log 194416'15970 (194416'15969) modify da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by client.2441958.0:4014 2014-08-13 15:28:45.977701 2014-08-13 17:38:03.059479 7fe9bae887a0 20 read_log 194416'15971 (0'0) promote eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by osd.304.0:2676 2014-08-13 15:28:47.667845 2014-08-13 17:38:03.059491 7fe9bae887a0 20 read_log 194416'15972 (194416'15971) modify eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by client.2441958.0:4446 2014-08-13 15:28:47.660091 2014-08-13 17:38:03.059503 7fe9bae887a0 20 read_log 194416'15973 (0'0) promote 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by osd.304.0:2680 2014-08-13 15:28:48.119425 2014-08-13 17:38:03.059516 7fe9bae887a0 20 read_log 194416'15974 (194416'15973) modify 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by client.2441958.0:4542 2014-08-13 15:28:48.111350 2014-08-13 17:38:03.059528 7fe9bae887a0 20 read_log 194416'15975 (0'0) promote cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by osd.304.0:2692 2014-08-13 15:28:51.544599 2014-08-13 17:38:03.059540 7fe9bae887a0 20 read_log 194416'15976 (194416'15975) modify cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by client.2441958.0:5434 2014-08-13 15:28:51.536921 2014-08-13 17:38:03.059552 7fe9bae887a0 20 read_log 194416'15977 (0'0) promote 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by osd.304.0:2710 2014-08-13 15:28:56.459404 2014-08-13 17:38:03.059564 7fe9bae887a0 20 read_log 194416'15978 (194416'15977) modify 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by client.2441958.0:6645 2014-08-13 15:28:56.451593 2014-08-13 17:38:03.059576 7fe9bae887a0 20 read_log 194416'15979 (0'0) promote f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by osd.304.0:2720 2014-08-13 15:28:58.842643 2014-08-13 17:38:03.059588 7fe9bae887a0 20 read_log 194416'15980 (194416'15979) modify f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by client.2441958.0:7286 2014-08-13 15:28:58.835798 2014-08-13 17:38:03.059600 7fe9bae887a0 20 read_log 194416'15981 (0'0) promote b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by osd.304.0:2740 2014-08-13 15:29:05.612289 2014-08-13 17:38:03.059612 7fe9bae887a0 20 read_log 194416'15982 (194416'15981) modify b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by client.2441958.0:9035 2014-08-13 15:29:05.605167 2014-08-13 17:38:03.059624 7fe9bae887a0 20 read_log 194416'15983 (0'0) promote 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by osd.304.0:2744 2014-08-13 15:29:07.123515 2014-08-13 17:38:03.059636 7fe9bae887a0 20 read_log 194416'15984 (194416'15983) modify 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by client.2441958.0:9432 2014-08-13 15:29:07.115784 2014-08-13 17:38:03.059648 7fe9bae887a0 20 read_log 194416'15985 (0'0) promote fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by osd.304.0:2750 2014-08-13 15:29:07.440706 2014-08-13 17:38:03.059660 7fe9bae887a0 20 read_log 194416'15986 (194416'15985) modify fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by client.2441958.0:9514 2014-08-13 15:29:07.431964 2014-08-13 17:38:03.059672 7fe9bae887a0 20 read_log 194416'15987 (0'0) promote 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by osd.304.0:2756 2014-08-13 15:29:10.518230 2014-08-13 17:38:03.059728 7fe9bae887a0 20 read_log 194416'15988 (194416'15987) modify 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by client.2441958.0:10263 2014-08-13 15:29:10.507788 2014-08-13 17:38:03.059743 7fe9bae887a0 20 read_log 194416'15989 (0'0) promote d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by osd.304.0:2776 2014-08-13 15:29:19.234295 2014-08-13 17:38:03.059763 7fe9bae887a0 20 read_log 194416'15990 (194416'15989) modify d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by client.2441958.0:12379 2014-08-13 15:29:19.224710 2014-08-13 17:38:03.059778 7fe9bae887a0 20 read_log 194416'15991 (0'0) promote 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by osd.304.0:2780 2014-08-13 15:29:19.789551 2014-08-13 17:38:03.059791 7fe9bae887a0 20 read_log 194416'15992 (194416'15991) modify 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by client.2441958.0:12524 2014-08-13 15:29:19.782078 2014-08-13 17:38:03.059805 7fe9bae887a0 20 read_log 194416'15993 (0'0) promote b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by osd.304.0:2790 2014-08-13 15:29:22.498882 2014-08-13 17:38:03.059832 7fe9bae887a0 20 read_log 194416'15994 (194416'15993) modify b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by client.2441958.0:13120 2014-08-13 15:29:22.491211 2014-08-13 17:38:03.059847 7fe9bae887a0 20 read_log 194416'15995 (0'0) promote ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by osd.304.0:2796 2014-08-13 15:29:26.225346 2014-08-13 17:38:03.059860 7fe9bae887a0 20 read_log 194416'15996 (194416'15995) modify ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by client.2441958.0:14003 2014-08-13 15:29:26.217514 2014-08-13 17:38:03.059872 7fe9bae887a0 20 read_log 194416'15997 (0'0) promote eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by osd.304.0:2802 2014-08-13 15:29:28.953608 2014-08-13 17:38:03.059884 7fe9bae887a0 20 read_log 194416'15998 (194416'15997) modify eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by client.2441958.0:14695 2014-08-13 15:29:28.945683 2014-08-13 17:38:03.059896 7fe9bae887a0 20 read_log 194416'15999 (0'0) promote 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by osd.304.0:2806 2014-08-13 15:29:33.724536 2014-08-13 17:38:03.059910 7fe9bae887a0 20 read_log 194416'16000 (194416'15999) modify 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by client.2441958.0:15691 2014-08-13 15:29:33.715254 2014-08-13 17:38:03.059923 7fe9bae887a0 20 read_log 194416'16001 (0'0) promote 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by osd.304.0:2814 2014-08-13 15:29:37.910468 2014-08-13 17:38:03.059936 7fe9bae887a0 20 read_log 194416'16002 (194416'16001) modify 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by client.2441958.0:16332 2014-08-13 15:29:37.901575 2014-08-13 17:38:03.059951 7fe9bae887a0 20 read_log 194416'16003 (0'0) promote 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by osd.304.0:2818 2014-08-13 15:29:41.340684 2014-08-13 17:38:03.059963 7fe9bae887a0 20 read_log 194416'16004 (194416'16003) modify 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by client.2441958.0:16577 2014-08-13 15:29:41.332043 2014-08-13 17:38:03.059977 7fe9bae887a0 20 read_log 194416'16005 (0'0) promote eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by osd.304.0:2834 2014-08-13 15:29:54.664006 2014-08-13 17:38:03.059989 7fe9bae887a0 20 read_log 194416'16006 (194416'16005) modify eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by client.2441958.0:17516 2014-08-13 15:29:54.655401 2014-08-13 17:38:03.060004 7fe9bae887a0 20 read_log 194416'16007 (0'0) promote ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by osd.304.0:2836 2014-08-13 15:29:56.390028 2014-08-13 17:38:03.060016 7fe9bae887a0 20 read_log 194416'16008 (194416'16007) modify ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by client.2441958.0:17638 2014-08-13 15:29:56.381296 2014-08-13 17:38:03.060030 7fe9bae887a0 20 read_log 194416'16009 (0'0) promote 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by osd.304.0:2842 2014-08-13 15:30:00.877414 2014-08-13 17:38:03.060068 7fe9bae887a0 20 read_log 194416'16010 (194416'16009) modify 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by client.2441958.0:17958 2014-08-13 15:30:00.867823 2014-08-13 17:38:03.060083 7fe9bae887a0 20 read_log 194416'16011 (0'0) promote 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by osd.304.0:2848 2014-08-13 15:30:09.642168 2014-08-13 17:38:03.060096 7fe9bae887a0 20 read_log 194416'16012 (194416'16011) modify 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by client.2441958.0:18376 2014-08-13 15:30:09.633913 2014-08-13 17:38:03.060110 7fe9bae887a0 20 read_log 194422'16013 (0'0) promote 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by osd.304.0:2852 2014-08-13 15:32:16.737428 2014-08-13 17:38:03.060122 7fe9bae887a0 20 read_log 194422'16014 (194422'16013) modify 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by client.1148203.0:376 2014-08-13 15:32:16.730155 2014-08-13 17:38:03.060137 7fe9bae887a0 20 read_log 194422'16015 (0'0) promote d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by osd.304.0:2870 2014-08-13 15:32:23.702897 2014-08-13 17:38:03.060149 7fe9bae887a0 20 read_log 194422'16016 (194422'16015) modify d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by client.1148203.0:2123 2014-08-13 15:32:23.694114 2014-08-13 17:38:03.060167 7fe9bae887a0 20 read_log 194422'16017 (0'0) promote c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by osd.304.0:2888 2014-08-13 15:32:28.814325 2014-08-13 17:38:03.060180 7fe9bae887a0 20 read_log 194422'16018 (194422'16017) modify c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by client.1148203.0:3431 2014-08-13 15:32:28.807064 2014-08-13 17:38:03.060195 7fe9bae887a0 20 read_log 194422'16019 (0'0) promote 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by osd.304.0:2898 2014-08-13 15:32:34.015941 2014-08-13 17:38:03.060208 7fe9bae887a0 20 read_log 194422'16020 (194422'16019) modify 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by client.1148203.0:4784 2014-08-13 15:32:34.007596 2014-08-13 17:38:03.060220 7fe9bae887a0 20 read_log 194422'16021 (0'0) promote f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by osd.304.0:2902 2014-08-13 15:32:35.236035 2014-08-13 17:38:03.060232 7fe9bae887a0 20 read_log 194422'16022 (194422'16021) modify f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by client.1148203.0:5085 2014-08-13 15:32:35.228439 2014-08-13 17:38:03.060245 7fe9bae887a0 20 read_log 194422'16023 (0'0) promote a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by osd.304.0:2914 2014-08-13 15:32:39.906919 2014-08-13 17:38:03.060258 7fe9bae887a0 20 read_log 194422'16024 (194422'16023) modify a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by client.1148203.0:5960 2014-08-13 15:32:39.898895 2014-08-13 17:38:03.060270 7fe9bae887a0 20 read_log 194422'16025 (0'0) promote 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by osd.304.0:2922 2014-08-13 15:32:46.849289 2014-08-13 17:38:03.060282 7fe9bae887a0 20 read_log 194422'16026 (194422'16025) modify 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by client.1148203.0:7217 2014-08-13 15:32:46.840221 2014-08-13 17:38:03.060294 7fe9bae887a0 20 read_log 194422'16027 (0'0) promote 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by osd.304.0:2926 2014-08-13 15:32:47.864308 2014-08-13 17:38:03.060307 7fe9bae887a0 20 read_log 194422'16028 (194422'16027) modify 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by client.1148203.0:7382 2014-08-13 15:32:47.852956 2014-08-13 17:38:03.060319 7fe9bae887a0 20 read_log 194422'16029 (0'0) promote 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by osd.304.0:2940 2014-08-13 15:32:52.655792 2014-08-13 17:38:03.060333 7fe9bae887a0 20 read_log 194422'16030 (194422'16029) modify 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by client.1148203.0:8149 2014-08-13 15:32:52.647182 2014-08-13 17:38:03.060348 7fe9bae887a0 20 read_log 194422'16031 (0'0) promote 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by osd.304.0:2952 2014-08-13 15:33:07.870709 2014-08-13 17:38:03.060360 7fe9bae887a0 20 read_log 194422'16032 (194422'16031) modify 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by client.1148203.0:9801 2014-08-13 15:33:07.862907 2014-08-13 17:38:03.060373 7fe9bae887a0 20 read_log 194422'16033 (0'0) promote 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by osd.304.0:2958 2014-08-13 15:33:18.363531 2014-08-13 17:38:03.060387 7fe9bae887a0 20 read_log 194422'16034 (194422'16033) modify 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by client.1148203.0:10297 2014-08-13 15:33:18.354570 2014-08-13 17:38:03.060403 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.060781 7fe9bae887a0 10 read_log checking for missing items over interval (194215'15912,194422'16034] 2014-08-13 17:38:03.060828 7fe9bae887a0 15 read_log missing 194422'16034 (194422'16033) modify 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by client.1148203.0:10297 2014-08-13 15:33:18.354570 2014-08-13 17:38:03.060869 7fe9bae887a0 15 read_log missing 194422'16032 (194422'16031) modify 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by client.1148203.0:9801 2014-08-13 15:33:07.862907 2014-08-13 17:38:03.060908 7fe9bae887a0 15 read_log missing 194422'16030 (194422'16029) modify 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by client.1148203.0:8149 2014-08-13 15:32:52.647182 2014-08-13 17:38:03.060945 7fe9bae887a0 15 read_log missing 194422'16028 (194422'16027) modify 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by client.1148203.0:7382 2014-08-13 15:32:47.852956 2014-08-13 17:38:03.060981 7fe9bae887a0 15 read_log missing 194422'16026 (194422'16025) modify 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by client.1148203.0:7217 2014-08-13 15:32:46.840221 2014-08-13 17:38:03.061018 7fe9bae887a0 15 read_log missing 194422'16024 (194422'16023) modify a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by client.1148203.0:5960 2014-08-13 15:32:39.898895 2014-08-13 17:38:03.061055 7fe9bae887a0 15 read_log missing 194422'16022 (194422'16021) modify f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by client.1148203.0:5085 2014-08-13 15:32:35.228439 2014-08-13 17:38:03.061092 7fe9bae887a0 15 read_log missing 194422'16020 (194422'16019) modify 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by client.1148203.0:4784 2014-08-13 15:32:34.007596 2014-08-13 17:38:03.061128 7fe9bae887a0 15 read_log missing 194422'16018 (194422'16017) modify c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by client.1148203.0:3431 2014-08-13 15:32:28.807064 2014-08-13 17:38:03.061170 7fe9bae887a0 15 read_log missing 194422'16016 (194422'16015) modify d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by client.1148203.0:2123 2014-08-13 15:32:23.694114 2014-08-13 17:38:03.061208 7fe9bae887a0 15 read_log missing 194422'16014 (194422'16013) modify 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by client.1148203.0:376 2014-08-13 15:32:16.730155 2014-08-13 17:38:03.061245 7fe9bae887a0 15 read_log missing 194416'16012 (194416'16011) modify 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by client.2441958.0:18376 2014-08-13 15:30:09.633913 2014-08-13 17:38:03.061282 7fe9bae887a0 15 read_log missing 194416'16010 (194416'16009) modify 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by client.2441958.0:17958 2014-08-13 15:30:00.867823 2014-08-13 17:38:03.061319 7fe9bae887a0 15 read_log missing 194416'16008 (194416'16007) modify ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by client.2441958.0:17638 2014-08-13 15:29:56.381296 2014-08-13 17:38:03.061356 7fe9bae887a0 15 read_log missing 194416'16006 (194416'16005) modify eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by client.2441958.0:17516 2014-08-13 15:29:54.655401 2014-08-13 17:38:03.061393 7fe9bae887a0 15 read_log missing 194416'16004 (194416'16003) modify 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by client.2441958.0:16577 2014-08-13 15:29:41.332043 2014-08-13 17:38:03.061430 7fe9bae887a0 15 read_log missing 194416'16002 (194416'16001) modify 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by client.2441958.0:16332 2014-08-13 15:29:37.901575 2014-08-13 17:38:03.061466 7fe9bae887a0 15 read_log missing 194416'16000 (194416'15999) modify 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by client.2441958.0:15691 2014-08-13 15:29:33.715254 2014-08-13 17:38:03.061503 7fe9bae887a0 15 read_log missing 194416'15998 (194416'15997) modify eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by client.2441958.0:14695 2014-08-13 15:29:28.945683 2014-08-13 17:38:03.061540 7fe9bae887a0 15 read_log missing 194416'15996 (194416'15995) modify ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by client.2441958.0:14003 2014-08-13 15:29:26.217514 2014-08-13 17:38:03.061576 7fe9bae887a0 15 read_log missing 194416'15994 (194416'15993) modify b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by client.2441958.0:13120 2014-08-13 15:29:22.491211 2014-08-13 17:38:03.061613 7fe9bae887a0 15 read_log missing 194416'15992 (194416'15991) modify 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by client.2441958.0:12524 2014-08-13 15:29:19.782078 2014-08-13 17:38:03.061650 7fe9bae887a0 15 read_log missing 194416'15990 (194416'15989) modify d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by client.2441958.0:12379 2014-08-13 15:29:19.224710 2014-08-13 17:38:03.061685 7fe9bae887a0 15 read_log missing 194416'15988 (194416'15987) modify 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by client.2441958.0:10263 2014-08-13 15:29:10.507788 2014-08-13 17:38:03.061721 7fe9bae887a0 15 read_log missing 194416'15986 (194416'15985) modify fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by client.2441958.0:9514 2014-08-13 15:29:07.431964 2014-08-13 17:38:03.061758 7fe9bae887a0 15 read_log missing 194416'15984 (194416'15983) modify 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by client.2441958.0:9432 2014-08-13 15:29:07.115784 2014-08-13 17:38:03.061795 7fe9bae887a0 15 read_log missing 194416'15982 (194416'15981) modify b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by client.2441958.0:9035 2014-08-13 15:29:05.605167 2014-08-13 17:38:03.061831 7fe9bae887a0 15 read_log missing 194416'15980 (194416'15979) modify f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by client.2441958.0:7286 2014-08-13 15:28:58.835798 2014-08-13 17:38:03.061868 7fe9bae887a0 15 read_log missing 194416'15978 (194416'15977) modify 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by client.2441958.0:6645 2014-08-13 15:28:56.451593 2014-08-13 17:38:03.061905 7fe9bae887a0 15 read_log missing 194416'15976 (194416'15975) modify cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by client.2441958.0:5434 2014-08-13 15:28:51.536921 2014-08-13 17:38:03.061942 7fe9bae887a0 15 read_log missing 194416'15974 (194416'15973) modify 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by client.2441958.0:4542 2014-08-13 15:28:48.111350 2014-08-13 17:38:03.061978 7fe9bae887a0 15 read_log missing 194416'15972 (194416'15971) modify eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by client.2441958.0:4446 2014-08-13 15:28:47.660091 2014-08-13 17:38:03.062015 7fe9bae887a0 15 read_log missing 194416'15970 (194416'15969) modify da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by client.2441958.0:4014 2014-08-13 15:28:45.977701 2014-08-13 17:38:03.062052 7fe9bae887a0 15 read_log missing 194416'15968 (194416'15967) modify d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by client.2441958.0:2576 2014-08-13 15:28:40.467172 2014-08-13 17:38:03.062088 7fe9bae887a0 15 read_log missing 194416'15966 (194416'15965) modify e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by client.2441958.0:2498 2014-08-13 15:28:40.168526 2014-08-13 17:38:03.062124 7fe9bae887a0 15 read_log missing 194416'15964 (194416'15963) modify 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by client.2441958.0:1498 2014-08-13 15:28:36.390188 2014-08-13 17:38:03.062165 7fe9bae887a0 15 read_log missing 194416'15962 (194416'15961) modify 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by client.2441958.0:1421 2014-08-13 15:28:36.105385 2014-08-13 17:38:03.062202 7fe9bae887a0 15 read_log missing 194416'15960 (194416'15959) modify bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by client.2441958.0:886 2014-08-13 15:28:34.018559 2014-08-13 17:38:03.062238 7fe9bae887a0 15 read_log missing 194416'15958 (194416'15957) modify fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by client.2441958.0:175 2014-08-13 15:28:30.846495 2014-08-13 17:38:03.062275 7fe9bae887a0 15 read_log missing 194397'15956 (194397'15955) modify 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by client.2235069.0:926 2014-08-12 20:53:06.093281 2014-08-13 17:38:03.062311 7fe9bae887a0 15 read_log missing 194393'15954 (194393'15953) modify 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by client.1064095.0:18 2014-08-12 20:47:25.949889 2014-08-13 17:38:03.062347 7fe9bae887a0 15 read_log missing 194383'15952 (194383'15951) modify 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by client.2231190.0:7792 2014-08-12 20:34:36.968217 2014-08-13 17:38:03.062384 7fe9bae887a0 15 read_log missing 194383'15950 (194383'15949) modify 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by client.2231190.0:6841 2014-08-12 20:34:10.839540 2014-08-13 17:38:03.062420 7fe9bae887a0 15 read_log missing 194383'15948 (194383'15947) modify b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by client.2231190.0:6822 2014-08-12 20:34:10.315611 2014-08-13 17:38:03.062456 7fe9bae887a0 15 read_log missing 194377'15946 (194377'15945) modify e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by client.2231190.0:4182 2014-08-12 20:33:14.989295 2014-08-13 17:38:03.062493 7fe9bae887a0 15 read_log missing 194377'15944 (194377'15943) modify 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by client.2231190.0:3695 2014-08-12 20:33:12.304340 2014-08-13 17:38:03.062530 7fe9bae887a0 15 read_log missing 194377'15942 (194377'15941) modify 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by client.2231190.0:1883 2014-08-12 20:33:04.731658 2014-08-13 17:38:03.062566 7fe9bae887a0 15 read_log missing 194370'15940 (194370'15939) modify 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by client.1508531.0:12466 2014-08-12 20:31:14.749585 2014-08-13 17:38:03.062602 7fe9bae887a0 15 read_log missing 194370'15938 (194370'15937) modify 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by client.1508531.0:12208 2014-08-12 20:31:12.891369 2014-08-13 17:38:03.062638 7fe9bae887a0 15 read_log missing 194370'15936 (194370'15935) modify a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by client.1508531.0:11430 2014-08-12 20:31:07.890091 2014-08-13 17:38:03.062675 7fe9bae887a0 15 read_log missing 194370'15934 (194370'15933) modify 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by client.1508531.0:11403 2014-08-12 20:31:07.740071 2014-08-13 17:38:03.062711 7fe9bae887a0 15 read_log missing 194370'15932 (194370'15931) modify fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by client.1508531.0:8317 2014-08-12 20:30:49.252693 2014-08-13 17:38:03.062747 7fe9bae887a0 15 read_log missing 194370'15930 (194370'15929) modify 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by client.1508531.0:6487 2014-08-12 20:30:39.894743 2014-08-13 17:38:03.062784 7fe9bae887a0 15 read_log missing 194370'15928 (194370'15927) modify 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by client.1508531.0:6368 2014-08-12 20:30:39.326068 2014-08-13 17:38:03.062821 7fe9bae887a0 15 read_log missing 194370'15926 (194370'15925) modify 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by client.1508531.0:5714 2014-08-12 20:30:36.411349 2014-08-13 17:38:03.062859 7fe9bae887a0 15 read_log missing 194370'15924 (194370'15923) modify 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by client.1508531.0:4089 2014-08-12 20:30:29.617119 2014-08-13 17:38:03.062893 7fe9bae887a0 15 read_log missing 194370'15922 (194370'15921) modify 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by client.1508531.0:1547 2014-08-12 20:30:20.183075 2014-08-13 17:38:03.062930 7fe9bae887a0 15 read_log missing 194370'15920 (194370'15919) modify 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by client.1508531.0:392 2014-08-12 20:30:15.862085 2014-08-13 17:38:03.062967 7fe9bae887a0 15 read_log missing 194217'15918 (194217'15917) modify 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by client.1487927.0:434 2014-08-12 17:25:05.605946 2014-08-13 17:38:03.063004 7fe9bae887a0 15 read_log missing 194217'15916 (194217'15915) modify d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by client.1487927.0:307 2014-08-12 17:25:04.489405 2014-08-13 17:38:03.063045 7fe9bae887a0 15 read_log missing 194217'15913 (194217'15913) modify 33a/hit_set_24.33a_archive_2014-08-12 15:48:57.821312_2014-08-12 17:25:04.496076/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 17:38:03.063056 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.063084 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=0 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive m=61] handle_loaded 2014-08-13 17:38:03.063101 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=0 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Initial 0.013522 0 0.000000 2014-08-13 17:38:03.063113 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=0 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset 2014-08-13 17:38:03.063121 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=0 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194477 2014-08-13 17:38:03.063130 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log((192834'14161,194422'16034], crt=194422'16034) 2014-08-13 17:38:03.063140 7fe9bae887a0 10 osd.4 194478 pgid 24.350 coll 24.350_head 2014-08-13 17:38:03.063259 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.350 2014-08-13 17:38:03.063271 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.063304 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.350(unlocked)] enter Initial 2014-08-13 17:38:03.063315 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.350(unlocked)] enter NotTrimming 2014-08-13 17:38:03.063437 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.063563 7fe9bae887a0 20 read_log 192834'15731 (192834'15722) delete a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17280 2014-08-11 15:27:29.003777 2014-08-13 17:38:03.063584 7fe9bae887a0 20 read_log 192834'15732 (192834'15201) clean bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17281 2014-08-11 15:27:29.162877 2014-08-13 17:38:03.063598 7fe9bae887a0 20 read_log 192834'15740 (192834'15732) delete bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17289 2014-08-11 15:27:29.439631 2014-08-13 17:38:03.063610 7fe9bae887a0 20 read_log 192834'15748 (192834'15209) clean edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17297 2014-08-11 15:27:29.539874 2014-08-13 17:38:03.063622 7fe9bae887a0 20 read_log 192834'15752 (0'0) promote 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17301 2014-08-11 15:27:29.798676 2014-08-13 17:38:03.063634 7fe9bae887a0 20 read_log 192834'15753 (192834'15752) modify 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by client.1883634.0:16886 2014-08-11 15:27:29.788807 2014-08-13 17:38:03.063647 7fe9bae887a0 20 read_log 192834'15764 (192834'15748) delete edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17313 2014-08-11 15:27:34.507816 2014-08-13 17:38:03.063659 7fe9bae887a0 20 read_log 192834'15766 (192834'15227) clean 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17315 2014-08-11 15:27:34.606858 2014-08-13 17:38:03.063671 7fe9bae887a0 20 read_log 192834'15767 (192834'15223) clean cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17316 2014-08-11 15:27:34.611485 2014-08-13 17:38:03.063683 7fe9bae887a0 20 read_log 192834'15768 (192834'15213) clean 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17317 2014-08-11 15:27:34.611810 2014-08-13 17:38:03.063695 7fe9bae887a0 20 read_log 192834'15774 (192834'15766) delete 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17323 2014-08-11 15:27:34.734901 2014-08-13 17:38:03.063706 7fe9bae887a0 20 read_log 192834'15778 (192834'15767) delete cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17327 2014-08-11 15:27:34.833712 2014-08-13 17:38:03.063718 7fe9bae887a0 20 read_log 192834'15779 (192834'15768) delete 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17328 2014-08-11 15:27:34.835683 2014-08-13 17:38:03.063730 7fe9bae887a0 20 read_log 192834'15806 (192834'15249) clean e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17354 2014-08-11 15:27:40.022394 2014-08-13 17:38:03.063742 7fe9bae887a0 20 read_log 192834'15815 (192834'15806) delete e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17363 2014-08-11 15:27:40.244569 2014-08-13 17:38:03.063754 7fe9bae887a0 20 read_log 192834'15823 (0'0) promote f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17371 2014-08-11 15:27:40.439075 2014-08-13 17:38:03.063775 7fe9bae887a0 20 read_log 192834'15824 (192834'15823) modify f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by client.1883634.0:18019 2014-08-11 15:27:40.430814 2014-08-13 17:38:03.063788 7fe9bae887a0 20 read_log 192834'15830 (192834'15346) clean b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17378 2014-08-11 15:27:45.395855 2014-08-13 17:38:03.063800 7fe9bae887a0 20 read_log 192834'15831 (192834'15294) clean 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17379 2014-08-11 15:27:45.396452 2014-08-13 17:38:03.063813 7fe9bae887a0 20 read_log 192834'15835 (192834'15830) delete b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17383 2014-08-11 15:27:45.421435 2014-08-13 17:38:03.063825 7fe9bae887a0 20 read_log 192834'15836 (192834'15831) delete 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17384 2014-08-11 15:27:45.425752 2014-08-13 17:38:03.063837 7fe9bae887a0 20 read_log 192834'15843 (192834'15348) clean 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17391 2014-08-11 15:27:45.721258 2014-08-13 17:38:03.063850 7fe9bae887a0 20 read_log 192834'15852 (0'0) promote dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:17400 2014-08-11 15:27:48.967829 2014-08-13 17:38:03.063862 7fe9bae887a0 20 read_log 192834'15853 (192834'15852) modify dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by client.1883634.0:19027 2014-08-11 15:27:48.958878 2014-08-13 17:38:03.063876 7fe9bae887a0 20 read_log 192834'15858 (192834'15843) delete 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17406 2014-08-11 15:27:50.523494 2014-08-13 17:38:03.063888 7fe9bae887a0 20 read_log 192834'15887 (192834'15436) clean 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17435 2014-08-11 15:27:55.795501 2014-08-13 17:38:03.063900 7fe9bae887a0 20 read_log 192834'15890 (192834'15438) clean 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17438 2014-08-11 15:27:55.878505 2014-08-13 17:38:03.063912 7fe9bae887a0 20 read_log 192834'15896 (192834'15887) delete 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17444 2014-08-11 15:27:55.900172 2014-08-13 17:38:03.063923 7fe9bae887a0 20 read_log 192834'15898 (192834'15890) delete 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17446 2014-08-11 15:27:55.912818 2014-08-13 17:38:03.063936 7fe9bae887a0 20 read_log 192834'15921 (0'0) promote 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:17469 2014-08-11 15:28:02.069080 2014-08-13 17:38:03.063948 7fe9bae887a0 20 read_log 192834'15922 (192834'15921) modify 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by client.1883634.0:20943 2014-08-11 15:28:02.060855 2014-08-13 17:38:03.063960 7fe9bae887a0 20 read_log 192834'15931 (0'0) promote d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:17479 2014-08-11 15:28:04.789308 2014-08-13 17:38:03.063972 7fe9bae887a0 20 read_log 192834'15932 (192834'15931) modify d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by client.1883634.0:21627 2014-08-11 15:28:04.780152 2014-08-13 17:38:03.063984 7fe9bae887a0 20 read_log 192834'15945 (192834'15549) clean 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17493 2014-08-11 15:28:06.398159 2014-08-13 17:38:03.063996 7fe9bae887a0 20 read_log 192834'15954 (192834'15945) delete 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17502 2014-08-11 15:28:06.440828 2014-08-13 17:38:03.064034 7fe9bae887a0 20 read_log 192834'15975 (192834'15563) clean 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17523 2014-08-11 15:28:11.579495 2014-08-13 17:38:03.064054 7fe9bae887a0 20 read_log 192834'15988 (192834'15665) clean 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17536 2014-08-11 15:28:16.628059 2014-08-13 17:38:03.064069 7fe9bae887a0 20 read_log 192834'15991 (192834'15975) delete 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17539 2014-08-11 15:28:16.634404 2014-08-13 17:38:03.064081 7fe9bae887a0 20 read_log 192834'15995 (192834'15988) delete 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17543 2014-08-11 15:28:16.735101 2014-08-13 17:38:03.064094 7fe9bae887a0 20 read_log 192834'16013 (0'0) promote b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17561 2014-08-11 15:28:18.564085 2014-08-13 17:38:03.064108 7fe9bae887a0 20 read_log 192834'16014 (192834'16013) modify b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by client.1883634.0:24214 2014-08-11 15:28:18.537958 2014-08-13 17:38:03.064120 7fe9bae887a0 20 read_log 192834'16019 (0'0) promote 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17567 2014-08-11 15:28:19.123685 2014-08-13 17:38:03.064134 7fe9bae887a0 20 read_log 192834'16020 (192834'16019) modify 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by client.1883634.0:24466 2014-08-11 15:28:19.087449 2014-08-13 17:38:03.064146 7fe9bae887a0 20 read_log 192834'16023 (0'0) promote 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17571 2014-08-11 15:28:19.396264 2014-08-13 17:38:03.064165 7fe9bae887a0 20 read_log 192834'16024 (192834'16023) modify 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by client.1883634.0:24650 2014-08-11 15:28:19.327856 2014-08-13 17:38:03.064180 7fe9bae887a0 20 read_log 192834'16031 (0'0) promote 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17579 2014-08-11 15:28:20.276662 2014-08-13 17:38:03.064192 7fe9bae887a0 20 read_log 192834'16032 (192834'16031) modify 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by client.1883634.0:25138 2014-08-11 15:28:20.252347 2014-08-13 17:38:03.064223 7fe9bae887a0 20 read_log 192834'16037 (0'0) promote a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17585 2014-08-11 15:28:21.539584 2014-08-13 17:38:03.064238 7fe9bae887a0 20 read_log 192834'16038 (192834'16037) modify a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by client.1883634.0:25531 2014-08-11 15:28:21.507809 2014-08-13 17:38:03.064250 7fe9bae887a0 20 read_log 192834'16039 (0'0) promote d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17587 2014-08-11 15:28:21.692880 2014-08-13 17:38:03.064262 7fe9bae887a0 20 read_log 192834'16040 (192834'16039) modify d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by client.1883634.0:25640 2014-08-11 15:28:21.664684 2014-08-13 17:38:03.064296 7fe9bae887a0 20 read_log 192834'16051 (192834'15675) clean bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17599 2014-08-11 15:28:22.512931 2014-08-13 17:38:03.064308 7fe9bae887a0 20 read_log 192834'16059 (0'0) promote bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17607 2014-08-11 15:28:23.444160 2014-08-13 17:38:03.064320 7fe9bae887a0 20 read_log 192834'16060 (192834'16059) modify bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by client.1883634.0:26684 2014-08-11 15:28:23.415652 2014-08-13 17:38:03.064333 7fe9bae887a0 20 read_log 192834'16067 (0'0) promote edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17615 2014-08-11 15:28:24.510669 2014-08-13 17:38:03.064345 7fe9bae887a0 20 read_log 192834'16068 (192834'16067) modify edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by client.1883634.0:27440 2014-08-11 15:28:24.471386 2014-08-13 17:38:03.064368 7fe9bae887a0 20 read_log 192834'16071 (0'0) promote 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17619 2014-08-11 15:28:24.775525 2014-08-13 17:38:03.064383 7fe9bae887a0 20 read_log 192834'16072 (192834'16071) modify 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by client.1883634.0:27648 2014-08-11 15:28:24.742592 2014-08-13 17:38:03.064395 7fe9bae887a0 20 read_log 192834'16081 (0'0) promote cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17629 2014-08-11 15:28:25.665640 2014-08-13 17:38:03.064407 7fe9bae887a0 20 read_log 192834'16082 (192834'16081) modify cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by client.1883634.0:28318 2014-08-11 15:28:25.640691 2014-08-13 17:38:03.064419 7fe9bae887a0 20 read_log 192834'16085 (0'0) promote 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17633 2014-08-11 15:28:25.877379 2014-08-13 17:38:03.064431 7fe9bae887a0 20 read_log 192834'16086 (192834'16085) modify 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by client.1883634.0:28465 2014-08-11 15:28:25.846626 2014-08-13 17:38:03.064443 7fe9bae887a0 20 read_log 192834'16100 (192834'16051) delete bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17648 2014-08-11 15:28:26.850528 2014-08-13 17:38:03.064455 7fe9bae887a0 20 read_log 192834'16111 (0'0) promote e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17659 2014-08-11 15:28:27.453644 2014-08-13 17:38:03.064466 7fe9bae887a0 20 read_log 192834'16112 (192834'16111) modify e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by client.1883634.0:29562 2014-08-11 15:28:27.428185 2014-08-13 17:38:03.064478 7fe9bae887a0 20 read_log 192834'16131 (0'0) promote 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17679 2014-08-11 15:28:30.578850 2014-08-13 17:38:03.064490 7fe9bae887a0 20 read_log 192834'16132 (192834'16131) modify 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by client.1883634.0:31281 2014-08-11 15:28:30.210083 2014-08-13 17:38:03.064503 7fe9bae887a0 20 read_log 192834'16133 (0'0) promote b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17681 2014-08-11 15:28:30.608408 2014-08-13 17:38:03.064515 7fe9bae887a0 20 read_log 192834'16134 (192834'16133) modify b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by client.1883634.0:31465 2014-08-11 15:28:30.568941 2014-08-13 17:38:03.064527 7fe9bae887a0 20 read_log 192834'16135 (0'0) promote 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17683 2014-08-11 15:28:30.689935 2014-08-13 17:38:03.064538 7fe9bae887a0 20 read_log 192834'16136 (192834'16135) modify 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by client.1883634.0:31524 2014-08-11 15:28:30.660602 2014-08-13 17:38:03.064550 7fe9bae887a0 20 read_log 192834'16155 (192834'15753) clean 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17703 2014-08-11 15:28:32.015113 2014-08-13 17:38:03.064562 7fe9bae887a0 20 read_log 192834'16157 (192834'16155) delete 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17705 2014-08-11 15:28:32.056882 2014-08-13 17:38:03.064574 7fe9bae887a0 20 read_log 192834'16173 (0'0) promote 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17721 2014-08-11 15:28:33.106466 2014-08-13 17:38:03.064586 7fe9bae887a0 20 read_log 192834'16174 (192834'16173) modify 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by client.1883634.0:33375 2014-08-11 15:28:33.079063 2014-08-13 17:38:03.064598 7fe9bae887a0 20 read_log 192834'16175 (0'0) promote 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17723 2014-08-11 15:28:33.157308 2014-08-13 17:38:03.064618 7fe9bae887a0 20 read_log 192834'16176 (192834'16175) modify 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by client.1883634.0:33407 2014-08-11 15:28:33.129969 2014-08-13 17:38:03.064631 7fe9bae887a0 20 read_log 192834'16195 (0'0) promote 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17743 2014-08-11 15:28:36.714381 2014-08-13 17:38:03.064643 7fe9bae887a0 20 read_log 192834'16196 (192834'16195) modify 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by client.1883634.0:35471 2014-08-11 15:28:36.685448 2014-08-13 17:38:03.064655 7fe9bae887a0 20 read_log 192834'16222 (0'0) promote 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17770 2014-08-11 15:28:37.777287 2014-08-13 17:38:03.064666 7fe9bae887a0 20 read_log 192834'16223 (192834'16222) modify 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by client.1883634.0:36204 2014-08-11 15:28:37.730189 2014-08-13 17:38:03.064678 7fe9bae887a0 20 read_log 192834'16236 (0'0) promote 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17784 2014-08-11 15:28:39.712284 2014-08-13 17:38:03.064690 7fe9bae887a0 20 read_log 192834'16237 (192834'16236) modify 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by client.1883634.0:37732 2014-08-11 15:28:39.686029 2014-08-13 17:38:03.064702 7fe9bae887a0 20 read_log 192834'16244 (0'0) promote bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17792 2014-08-11 15:28:40.679352 2014-08-13 17:38:03.064714 7fe9bae887a0 20 read_log 192834'16245 (192834'16244) modify bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by client.1883634.0:38455 2014-08-11 15:28:40.655188 2014-08-13 17:38:03.064726 7fe9bae887a0 20 read_log 192834'16260 (192834'15824) clean f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17808 2014-08-11 15:28:42.902141 2014-08-13 17:38:03.064738 7fe9bae887a0 20 read_log 192834'16263 (192834'16260) delete f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17811 2014-08-11 15:28:42.928481 2014-08-13 17:38:03.064750 7fe9bae887a0 20 read_log 192834'16269 (0'0) promote 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17817 2014-08-11 15:28:43.395187 2014-08-13 17:38:03.064761 7fe9bae887a0 20 read_log 192834'16270 (192834'16269) modify 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by client.1883634.0:40210 2014-08-11 15:28:43.367956 2014-08-13 17:38:03.064773 7fe9bae887a0 20 read_log 192834'16291 (0'0) promote f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17839 2014-08-11 15:28:44.953948 2014-08-13 17:38:03.064785 7fe9bae887a0 20 read_log 192834'16292 (192834'16291) modify f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by client.1883634.0:41343 2014-08-11 15:28:44.926388 2014-08-13 17:38:03.064797 7fe9bae887a0 20 read_log 192834'16299 (192834'15853) modify dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by client.1883634.0:42351 2014-08-11 15:28:46.216382 2014-08-13 17:38:03.064809 7fe9bae887a0 20 read_log 192834'16312 (192834'15922) modify 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by client.1883634.0:44267 2014-08-11 15:28:46.810083 2014-08-13 17:38:03.064821 7fe9bae887a0 20 read_log 192834'16317 (192834'15932) modify d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by client.1883634.0:44951 2014-08-11 15:28:47.100579 2014-08-13 17:38:03.064833 7fe9bae887a0 20 read_log 192834'16347 (192834'16014) clean b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17895 2014-08-11 15:29:18.702713 2014-08-13 17:38:03.064845 7fe9bae887a0 20 read_log 192834'16350 (192834'16038) clean a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17898 2014-08-11 15:29:23.722519 2014-08-13 17:38:03.064864 7fe9bae887a0 20 read_log 192834'16351 (192834'16032) clean 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17899 2014-08-11 15:29:23.722816 2014-08-13 17:38:03.064876 7fe9bae887a0 20 read_log 192834'16354 (192834'16060) clean bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17902 2014-08-11 15:29:23.744743 2014-08-13 17:38:03.064888 7fe9bae887a0 20 read_log 192834'16355 (192834'16020) clean 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17903 2014-08-11 15:29:23.752500 2014-08-13 17:38:03.064900 7fe9bae887a0 20 read_log 192834'16356 (192834'16024) clean 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17904 2014-08-11 15:29:23.756378 2014-08-13 17:38:03.064912 7fe9bae887a0 20 read_log 192834'16357 (192834'16040) clean d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17905 2014-08-11 15:29:23.760451 2014-08-13 17:38:03.064924 7fe9bae887a0 20 read_log 192834'16359 (192834'16347) delete b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17907 2014-08-11 15:29:23.768252 2014-08-13 17:38:03.064936 7fe9bae887a0 20 read_log 192834'16371 (192834'16350) delete a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17919 2014-08-11 15:29:23.825422 2014-08-13 17:38:03.064948 7fe9bae887a0 20 read_log 192834'16372 (192834'16351) delete 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17920 2014-08-11 15:29:23.826554 2014-08-13 17:38:03.064959 7fe9bae887a0 20 read_log 192834'16375 (192834'16354) delete bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17923 2014-08-11 15:29:23.837205 2014-08-13 17:38:03.064971 7fe9bae887a0 20 read_log 192834'16376 (192834'16355) delete 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17924 2014-08-11 15:29:23.849995 2014-08-13 17:38:03.064983 7fe9bae887a0 20 read_log 192834'16377 (192834'16356) delete 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17925 2014-08-11 15:29:23.850494 2014-08-13 17:38:03.064994 7fe9bae887a0 20 read_log 192834'16378 (192834'16357) delete d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17926 2014-08-11 15:29:23.854583 2014-08-13 17:38:03.065006 7fe9bae887a0 20 read_log 192834'16392 (192834'16086) clean 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17940 2014-08-11 15:29:29.013570 2014-08-13 17:38:03.065018 7fe9bae887a0 20 read_log 192834'16396 (192834'16082) clean cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17944 2014-08-11 15:29:29.032013 2014-08-13 17:38:03.065030 7fe9bae887a0 20 read_log 192834'16397 (192834'16068) clean edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17945 2014-08-11 15:29:29.036095 2014-08-13 17:38:03.065042 7fe9bae887a0 20 read_log 192834'16399 (192834'16072) clean 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17947 2014-08-11 15:29:29.046468 2014-08-13 17:38:03.065054 7fe9bae887a0 20 read_log 192834'16405 (192834'16112) clean e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17953 2014-08-11 15:29:29.064153 2014-08-13 17:38:03.065065 7fe9bae887a0 20 read_log 192834'16421 (192834'16392) delete 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17969 2014-08-11 15:29:29.129482 2014-08-13 17:38:03.065078 7fe9bae887a0 20 read_log 192834'16426 (192834'16396) delete cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17974 2014-08-11 15:29:29.139237 2014-08-13 17:38:03.065090 7fe9bae887a0 20 read_log 192834'16427 (192834'16397) delete edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17975 2014-08-11 15:29:29.140315 2014-08-13 17:38:03.065129 7fe9bae887a0 20 read_log 192834'16428 (192834'16399) delete 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17976 2014-08-11 15:29:29.143984 2014-08-13 17:38:03.065143 7fe9bae887a0 20 read_log 192834'16434 (192834'16405) delete e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17982 2014-08-11 15:29:29.156572 2014-08-13 17:38:03.065160 7fe9bae887a0 20 read_log 192834'16450 (192834'16134) clean b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17998 2014-08-11 15:29:34.233221 2014-08-13 17:38:03.065173 7fe9bae887a0 20 read_log 192834'16453 (192834'16132) clean 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:18001 2014-08-11 15:29:34.243419 2014-08-13 17:38:03.065185 7fe9bae887a0 20 read_log 192834'16454 (192834'16174) clean 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:18002 2014-08-11 15:29:34.244568 2014-08-13 17:38:03.065197 7fe9bae887a0 20 read_log 192834'16457 (192834'16136) clean 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:18005 2014-08-11 15:29:34.254779 2014-08-13 17:38:03.065209 7fe9bae887a0 20 read_log 192834'16459 (192834'16176) clean 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:18007 2014-08-11 15:29:34.262108 2014-08-13 17:38:03.065221 7fe9bae887a0 20 read_log 192834'16472 (192834'16450) delete b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:18020 2014-08-11 15:29:34.304977 2014-08-13 17:38:03.065236 7fe9bae887a0 20 read_log 192834'16475 (192834'16453) delete 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:18023 2014-08-11 15:29:34.314487 2014-08-13 17:38:03.065249 7fe9bae887a0 20 read_log 192834'16476 (192834'16454) delete 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:18024 2014-08-11 15:29:34.317293 2014-08-13 17:38:03.065260 7fe9bae887a0 20 read_log 192834'16481 (192834'16457) delete 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:18029 2014-08-11 15:29:34.326493 2014-08-13 17:38:03.065272 7fe9bae887a0 20 read_log 192834'16483 (192834'16459) delete 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:18031 2014-08-11 15:29:34.332875 2014-08-13 17:38:03.065284 7fe9bae887a0 20 read_log 192834'16500 (192834'16196) clean 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:18048 2014-08-11 15:29:39.417180 2014-08-13 17:38:03.065296 7fe9bae887a0 20 read_log 192834'16503 (192834'16223) clean 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:18051 2014-08-11 15:29:39.427162 2014-08-13 17:38:03.065308 7fe9bae887a0 20 read_log 192834'16514 (192834'16500) delete 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:18062 2014-08-11 15:29:39.459588 2014-08-13 17:38:03.065320 7fe9bae887a0 20 read_log 192834'16517 (192834'16503) delete 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:18065 2014-08-11 15:29:39.469998 2014-08-13 17:38:03.065332 7fe9bae887a0 20 read_log 192834'16534 (192834'16237) clean 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:18082 2014-08-11 15:29:44.531384 2014-08-13 17:38:03.065344 7fe9bae887a0 20 read_log 192834'16539 (192834'16270) clean 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:18087 2014-08-11 15:29:44.548303 2014-08-13 17:38:03.065356 7fe9bae887a0 20 read_log 192834'16540 (192834'16245) clean bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:18088 2014-08-11 15:29:44.549171 2014-08-13 17:38:03.065368 7fe9bae887a0 20 read_log 192834'16555 (192834'16534) delete 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:18103 2014-08-11 15:29:44.594759 2014-08-13 17:38:03.065387 7fe9bae887a0 20 read_log 192834'16560 (192834'16539) delete 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:18108 2014-08-11 15:29:44.606006 2014-08-13 17:38:03.065400 7fe9bae887a0 20 read_log 192834'16561 (192834'16540) delete bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:18109 2014-08-11 15:29:44.607937 2014-08-13 17:38:03.065412 7fe9bae887a0 20 read_log 192834'16584 (192834'16317) clean d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:18132 2014-08-11 15:29:49.663337 2014-08-13 17:38:03.065424 7fe9bae887a0 20 read_log 192834'16588 (192834'16312) clean 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:18136 2014-08-11 15:29:49.670715 2014-08-13 17:38:03.065456 7fe9bae887a0 20 read_log 192834'16590 (192834'16292) clean f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:18138 2014-08-11 15:29:49.675658 2014-08-13 17:38:03.065469 7fe9bae887a0 20 read_log 192834'16602 (192834'16584) delete d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:18150 2014-08-11 15:29:49.697567 2014-08-13 17:38:03.065481 7fe9bae887a0 20 read_log 192834'16606 (192834'16588) delete 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:18154 2014-08-11 15:29:49.703364 2014-08-13 17:38:03.065493 7fe9bae887a0 20 read_log 192834'16607 (192834'16590) delete f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:18155 2014-08-11 15:29:49.704323 2014-08-13 17:38:03.065505 7fe9bae887a0 20 read_log 192834'16609 (192834'16299) clean dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:18157 2014-08-11 15:29:49.706653 2014-08-13 17:38:03.065517 7fe9bae887a0 20 read_log 192834'16614 (192834'16609) delete dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:18162 2014-08-11 15:29:49.711594 2014-08-13 17:38:03.065529 7fe9bae887a0 20 read_log 192888'16666 (0'0) promote b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by osd.51.0:1972 2014-08-11 16:45:08.002090 2014-08-13 17:38:03.065541 7fe9bae887a0 20 read_log 192888'16667 (192888'16666) modify b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.1315259.0:3340 2014-08-11 16:45:07.995057 2014-08-13 17:38:03.065553 7fe9bae887a0 20 read_log 192888'16672 (0'0) promote 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by osd.51.0:1990 2014-08-11 16:45:12.396800 2014-08-13 17:38:03.065565 7fe9bae887a0 20 read_log 192888'16673 (192888'16672) modify 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.1315259.0:4532 2014-08-11 16:45:12.388186 2014-08-13 17:38:03.065577 7fe9bae887a0 20 read_log 192888'16682 (0'0) promote c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by osd.51.0:2014 2014-08-11 16:45:21.240689 2014-08-13 17:38:03.065589 7fe9bae887a0 20 read_log 192888'16683 (192888'16682) modify c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.1315259.0:6956 2014-08-11 16:45:21.233950 2014-08-13 17:38:03.065601 7fe9bae887a0 20 read_log 192888'16684 (0'0) promote 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by osd.51.0:2022 2014-08-11 16:45:22.099679 2014-08-13 17:38:03.065632 7fe9bae887a0 20 read_log 192888'16685 (192888'16684) modify 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.1315259.0:7194 2014-08-11 16:45:22.093008 2014-08-13 17:38:03.065649 7fe9bae887a0 20 read_log 192888'16686 (0'0) promote 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by osd.51.0:2032 2014-08-11 16:45:24.280000 2014-08-13 17:38:03.065669 7fe9bae887a0 20 read_log 192888'16687 (192888'16686) modify 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.1315259.0:7789 2014-08-11 16:45:24.271155 2014-08-13 17:38:03.065684 7fe9bae887a0 20 read_log 192888'16690 (0'0) promote 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by osd.51.0:2036 2014-08-11 16:45:25.035265 2014-08-13 17:38:03.065696 7fe9bae887a0 20 read_log 192888'16691 (192888'16690) modify 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.1315259.0:7998 2014-08-11 16:45:25.026310 2014-08-13 17:38:03.065709 7fe9bae887a0 20 read_log 192888'16694 (0'0) promote 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by osd.51.0:2048 2014-08-11 16:45:28.962813 2014-08-13 17:38:03.065721 7fe9bae887a0 20 read_log 192888'16695 (192888'16694) modify 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.1315259.0:9026 2014-08-11 16:45:28.955203 2014-08-13 17:38:03.065733 7fe9bae887a0 20 read_log 192888'16696 (0'0) promote 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by osd.51.0:2056 2014-08-11 16:45:30.343818 2014-08-13 17:38:03.065745 7fe9bae887a0 20 read_log 192888'16697 (192888'16696) modify 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.1315259.0:9399 2014-08-11 16:45:30.334562 2014-08-13 17:38:03.065757 7fe9bae887a0 20 read_log 192888'16698 (0'0) promote 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by osd.51.0:2064 2014-08-11 16:45:31.963079 2014-08-13 17:38:03.065768 7fe9bae887a0 20 read_log 192888'16699 (192888'16698) modify 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.1315259.0:9804 2014-08-11 16:45:31.955732 2014-08-13 17:38:03.065780 7fe9bae887a0 20 read_log 192888'16702 (0'0) promote 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by osd.51.0:2072 2014-08-11 16:45:35.017465 2014-08-13 17:38:03.065792 7fe9bae887a0 20 read_log 192888'16703 (192888'16702) modify 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.1315259.0:10601 2014-08-11 16:45:35.010589 2014-08-13 17:38:03.065805 7fe9bae887a0 20 read_log 192888'16712 (0'0) promote 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by osd.51.0:2096 2014-08-11 16:45:41.156376 2014-08-13 17:38:03.065816 7fe9bae887a0 20 read_log 192888'16713 (192888'16712) modify 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.1315259.0:12120 2014-08-11 16:45:41.146355 2014-08-13 17:38:03.065829 7fe9bae887a0 20 read_log 192888'16718 (0'0) promote e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by osd.51.0:2140 2014-08-11 16:45:54.498907 2014-08-13 17:38:03.065840 7fe9bae887a0 20 read_log 192888'16719 (192888'16718) modify e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.1315259.0:15488 2014-08-11 16:45:54.490574 2014-08-13 17:38:03.065852 7fe9bae887a0 20 read_log 192888'16720 (0'0) promote 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by osd.51.0:2144 2014-08-11 16:45:56.369899 2014-08-13 17:38:03.065865 7fe9bae887a0 20 read_log 192888'16721 (192888'16720) modify 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.1315259.0:15985 2014-08-11 16:45:56.363154 2014-08-13 17:38:03.065877 7fe9bae887a0 20 read_log 192888'16724 (0'0) promote 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by osd.51.0:2158 2014-08-11 16:45:58.139895 2014-08-13 17:38:03.065889 7fe9bae887a0 20 read_log 192888'16725 (192888'16724) modify 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.1315259.0:16452 2014-08-11 16:45:58.131248 2014-08-13 17:38:03.065901 7fe9bae887a0 20 read_log 192888'16726 (0'0) promote 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by osd.51.0:2164 2014-08-11 16:46:02.279688 2014-08-13 17:38:03.065923 7fe9bae887a0 20 read_log 192888'16727 (192888'16726) modify 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.1315259.0:17492 2014-08-11 16:46:02.271124 2014-08-13 17:38:03.065936 7fe9bae887a0 20 read_log 192888'16734 (0'0) promote 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by osd.51.0:2202 2014-08-11 16:46:15.129255 2014-08-13 17:38:03.065948 7fe9bae887a0 20 read_log 192888'16735 (192888'16734) modify 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.1315259.0:20797 2014-08-11 16:46:15.121829 2014-08-13 17:38:03.065960 7fe9bae887a0 20 read_log 192888'16736 (0'0) promote 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by osd.51.0:2204 2014-08-11 16:46:16.425481 2014-08-13 17:38:03.065972 7fe9bae887a0 20 read_log 192888'16737 (192888'16736) modify 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.1315259.0:21150 2014-08-11 16:46:16.418800 2014-08-13 17:38:03.065984 7fe9bae887a0 20 read_log 192888'16750 (0'0) promote a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by osd.51.0:2242 2014-08-11 16:46:30.127470 2014-08-13 17:38:03.065996 7fe9bae887a0 20 read_log 192888'16751 (192888'16750) modify a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.1315259.0:24653 2014-08-11 16:46:30.119728 2014-08-13 17:38:03.066008 7fe9bae887a0 20 read_log 192888'16754 (0'0) promote f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by osd.51.0:2246 2014-08-11 16:46:31.223415 2014-08-13 17:38:03.066020 7fe9bae887a0 20 read_log 192888'16755 (192888'16754) modify f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.1315259.0:24921 2014-08-11 16:46:31.215193 2014-08-13 17:38:03.066032 7fe9bae887a0 20 read_log 192888'16758 (0'0) promote c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by osd.51.0:2258 2014-08-11 16:46:37.907598 2014-08-13 17:38:03.066044 7fe9bae887a0 20 read_log 192888'16759 (192888'16758) modify c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.1315259.0:26504 2014-08-11 16:46:37.900177 2014-08-13 17:38:03.066056 7fe9bae887a0 20 read_log 192888'16760 (0'0) promote 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by osd.51.0:2260 2014-08-11 16:46:38.714094 2014-08-13 17:38:03.066068 7fe9bae887a0 20 read_log 192888'16761 (192888'16760) modify 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.1315259.0:26710 2014-08-11 16:46:38.706678 2014-08-13 17:38:03.066080 7fe9bae887a0 20 read_log 192888'16762 (0'0) promote b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by osd.51.0:2262 2014-08-11 16:46:40.730616 2014-08-13 17:38:03.066091 7fe9bae887a0 20 read_log 192888'16763 (192888'16762) modify b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.1315259.0:27170 2014-08-11 16:46:40.722748 2014-08-13 17:38:03.066103 7fe9bae887a0 20 read_log 192888'16764 (0'0) promote 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by osd.51.0:2266 2014-08-11 16:46:43.143069 2014-08-13 17:38:03.066115 7fe9bae887a0 20 read_log 192888'16765 (192888'16764) modify 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.1315259.0:27790 2014-08-11 16:46:43.135649 2014-08-13 17:38:03.066148 7fe9bae887a0 20 read_log 192888'16766 (0'0) promote 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by osd.51.0:2284 2014-08-11 16:46:51.604745 2014-08-13 17:38:03.066166 7fe9bae887a0 20 read_log 192888'16767 (192888'16766) modify 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.1315259.0:29863 2014-08-11 16:46:51.597373 2014-08-13 17:38:03.066181 7fe9bae887a0 20 read_log 192888'16768 (0'0) promote 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by osd.51.0:2286 2014-08-11 16:46:51.793641 2014-08-13 17:38:03.066201 7fe9bae887a0 20 read_log 192888'16769 (192888'16768) modify 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.1315259.0:29913 2014-08-11 16:46:51.785909 2014-08-13 17:38:03.066216 7fe9bae887a0 20 read_log 192888'16770 (0'0) promote f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by osd.51.0:2290 2014-08-11 16:46:52.936303 2014-08-13 17:38:03.066228 7fe9bae887a0 20 read_log 192888'16771 (192888'16770) modify f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.1315259.0:30222 2014-08-11 16:46:52.927785 2014-08-13 17:38:03.066240 7fe9bae887a0 20 read_log 192888'16775 (192888'16667) modify b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.1315259.0:34067 2014-08-11 16:46:56.796437 2014-08-13 17:38:03.066252 7fe9bae887a0 20 read_log 192888'16778 (192888'16673) modify 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.1315259.0:35259 2014-08-11 16:46:57.116479 2014-08-13 17:38:03.066266 7fe9bae887a0 20 read_log 192888'16783 (192888'16683) modify c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.1315259.0:37683 2014-08-11 16:46:57.747663 2014-08-13 17:38:03.066278 7fe9bae887a0 20 read_log 192888'16784 (192888'16685) modify 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.1315259.0:37921 2014-08-11 16:46:57.810745 2014-08-13 17:38:03.066290 7fe9bae887a0 20 read_log 192888'16785 (192888'16687) modify 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.1315259.0:38516 2014-08-11 16:46:57.970062 2014-08-13 17:38:03.066302 7fe9bae887a0 20 read_log 192888'16787 (192888'16691) modify 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.1315259.0:38725 2014-08-11 16:46:58.029597 2014-08-13 17:38:03.066314 7fe9bae887a0 20 read_log 192888'16789 (192888'16695) modify 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.1315259.0:39753 2014-08-11 16:46:58.294443 2014-08-13 17:38:03.066326 7fe9bae887a0 20 read_log 192888'16790 (192888'16697) modify 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.1315259.0:40126 2014-08-11 16:46:58.395446 2014-08-13 17:38:03.066337 7fe9bae887a0 20 read_log 192888'16791 (192888'16699) modify 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.1315259.0:40531 2014-08-11 16:46:58.515144 2014-08-13 17:38:03.066349 7fe9bae887a0 20 read_log 192888'16793 (192888'16703) modify 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.1315259.0:41328 2014-08-11 16:46:58.718115 2014-08-13 17:38:03.066362 7fe9bae887a0 20 read_log 192888'16798 (192888'16713) modify 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.1315259.0:42847 2014-08-11 16:46:59.121365 2014-08-13 17:38:03.066374 7fe9bae887a0 20 read_log 192888'16801 (192888'16719) modify e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.1315259.0:46215 2014-08-11 16:47:00.094092 2014-08-13 17:38:03.066386 7fe9bae887a0 20 read_log 192888'16802 (192888'16721) modify 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.1315259.0:46712 2014-08-11 16:47:00.237145 2014-08-13 17:38:03.066398 7fe9bae887a0 20 read_log 192888'16804 (192888'16725) modify 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.1315259.0:47179 2014-08-11 16:47:00.374867 2014-08-13 17:38:03.066410 7fe9bae887a0 20 read_log 192888'16805 (192888'16727) modify 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.1315259.0:48219 2014-08-11 16:47:00.701502 2014-08-13 17:38:03.066422 7fe9bae887a0 20 read_log 192888'16809 (192888'16735) modify 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.1315259.0:51524 2014-08-11 16:47:01.705344 2014-08-13 17:38:03.066434 7fe9bae887a0 20 read_log 192888'16810 (192888'16737) modify 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.1315259.0:51877 2014-08-11 16:47:01.809213 2014-08-13 17:38:03.066454 7fe9bae887a0 20 read_log 192888'16817 (192888'16751) modify a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.1315259.0:55380 2014-08-11 16:47:02.895538 2014-08-13 17:38:03.066466 7fe9bae887a0 20 read_log 192888'16819 (192888'16755) modify f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.1315259.0:55648 2014-08-11 16:47:02.973488 2014-08-13 17:38:03.066478 7fe9bae887a0 20 read_log 192888'16821 (192888'16759) modify c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.1315259.0:57231 2014-08-11 16:47:03.448915 2014-08-13 17:38:03.066490 7fe9bae887a0 20 read_log 192888'16822 (192888'16761) modify 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.1315259.0:57437 2014-08-11 16:47:03.512625 2014-08-13 17:38:03.066502 7fe9bae887a0 20 read_log 192888'16823 (192888'16763) modify b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.1315259.0:57897 2014-08-11 16:47:03.643324 2014-08-13 17:38:03.066514 7fe9bae887a0 20 read_log 192888'16824 (192888'16765) modify 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.1315259.0:58517 2014-08-11 16:47:03.825459 2014-08-13 17:38:03.066526 7fe9bae887a0 20 read_log 192888'16825 (192888'16767) modify 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.1315259.0:60590 2014-08-11 16:47:04.484847 2014-08-13 17:38:03.066538 7fe9bae887a0 20 read_log 192888'16826 (192888'16769) modify 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.1315259.0:60640 2014-08-11 16:47:04.499732 2014-08-13 17:38:03.066550 7fe9bae887a0 20 read_log 192888'16827 (192888'16771) modify f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.1315259.0:60949 2014-08-11 16:47:04.589670 2014-08-13 17:38:03.066562 7fe9bae887a0 20 read_log 192888'16837 (0'0) promote 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by osd.51.0:2490 2014-08-11 17:05:17.443839 2014-08-13 17:38:03.066574 7fe9bae887a0 20 read_log 192888'16838 (192888'16837) modify 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.942868.0:593 2014-08-11 17:05:17.436652 2014-08-13 17:38:03.066587 7fe9bae887a0 20 read_log 192888'16843 (0'0) promote 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by osd.51.0:2496 2014-08-11 17:05:18.461911 2014-08-13 17:38:03.066599 7fe9bae887a0 20 read_log 192888'16844 (192888'16843) modify 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.942868.0:836 2014-08-11 17:05:18.453230 2014-08-13 17:38:03.066611 7fe9bae887a0 20 read_log 192888'16845 (0'0) promote 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by osd.51.0:2498 2014-08-11 17:05:19.805430 2014-08-13 17:38:03.066623 7fe9bae887a0 20 read_log 192888'16846 (192888'16845) modify 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.942868.0:1159 2014-08-11 17:05:19.797346 2014-08-13 17:38:03.066634 7fe9bae887a0 20 read_log 192888'16847 (0'0) promote 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by osd.51.0:2500 2014-08-11 17:05:19.981263 2014-08-13 17:38:03.066646 7fe9bae887a0 20 read_log 192888'16848 (192888'16847) modify 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.942868.0:1204 2014-08-11 17:05:19.972770 2014-08-13 17:38:03.066658 7fe9bae887a0 20 read_log 192888'16851 (0'0) promote e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by osd.51.0:2521 2014-08-11 17:05:26.268415 2014-08-13 17:38:03.066673 7fe9bae887a0 20 read_log 192888'16852 (192888'16851) modify e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.942868.0:2790 2014-08-11 17:05:26.261159 2014-08-13 17:38:03.066685 7fe9bae887a0 20 read_log 192888'16853 (0'0) promote 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by osd.51.0:2525 2014-08-11 17:05:27.140074 2014-08-13 17:38:03.066707 7fe9bae887a0 20 read_log 192888'16854 (192888'16853) modify 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.942868.0:3026 2014-08-11 17:05:27.132189 2014-08-13 17:38:03.066976 7fe9bae887a0 20 read_log 192888'16855 (0'0) promote 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by osd.51.0:2527 2014-08-11 17:05:30.581622 2014-08-13 17:38:03.066989 7fe9bae887a0 20 read_log 192888'16856 (192888'16855) modify 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.942868.0:3958 2014-08-11 17:05:30.574356 2014-08-13 17:38:03.067001 7fe9bae887a0 20 read_log 192888'16861 (0'0) promote a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by osd.51.0:2547 2014-08-11 17:05:35.802422 2014-08-13 17:38:03.067013 7fe9bae887a0 20 read_log 192888'16862 (192888'16861) modify a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.942868.0:5315 2014-08-11 17:05:35.794790 2014-08-13 17:38:03.067027 7fe9bae887a0 20 read_log 192888'16863 (0'0) promote 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by osd.51.0:2551 2014-08-11 17:05:36.790258 2014-08-13 17:38:03.067039 7fe9bae887a0 20 read_log 192888'16864 (192888'16863) modify 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.942868.0:5580 2014-08-11 17:05:36.782699 2014-08-13 17:38:03.067051 7fe9bae887a0 20 read_log 192888'16865 (0'0) promote 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by osd.51.0:2585 2014-08-11 17:05:46.010203 2014-08-13 17:38:03.067063 7fe9bae887a0 20 read_log 192888'16866 (192888'16865) modify 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.942868.0:8051 2014-08-11 17:05:46.002342 2014-08-13 17:38:03.067077 7fe9bae887a0 20 read_log 192888'16875 (0'0) promote c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by osd.51.0:2603 2014-08-11 17:05:51.030821 2014-08-13 17:38:03.067089 7fe9bae887a0 20 read_log 192888'16876 (192888'16875) modify c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.942868.0:9399 2014-08-11 17:05:51.021561 2014-08-13 17:38:03.067101 7fe9bae887a0 20 read_log 192888'16881 (0'0) promote db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by osd.51.0:2621 2014-08-11 17:05:56.921405 2014-08-13 17:38:03.067113 7fe9bae887a0 20 read_log 192888'16882 (192888'16881) modify db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.942868.0:11035 2014-08-11 17:05:56.914126 2014-08-13 17:38:03.067125 7fe9bae887a0 20 read_log 192888'16887 (0'0) promote 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by osd.51.0:2629 2014-08-11 17:06:00.300999 2014-08-13 17:38:03.067137 7fe9bae887a0 20 read_log 192888'16888 (192888'16887) modify 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.942868.0:11942 2014-08-11 17:06:00.295175 2014-08-13 17:38:03.067149 7fe9bae887a0 20 read_log 192888'16893 (0'0) promote f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by osd.51.0:2649 2014-08-11 17:06:04.214835 2014-08-13 17:38:03.067176 7fe9bae887a0 20 read_log 192888'16894 (192888'16893) modify f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.942868.0:13007 2014-08-11 17:06:04.207175 2014-08-13 17:38:03.067189 7fe9bae887a0 20 read_log 192888'16895 (0'0) promote d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by osd.51.0:2651 2014-08-11 17:06:04.270043 2014-08-13 17:38:03.067201 7fe9bae887a0 20 read_log 192888'16896 (192888'16895) modify d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.942868.0:13022 2014-08-11 17:06:04.261899 2014-08-13 17:38:03.067213 7fe9bae887a0 20 read_log 192888'16899 (0'0) promote 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by osd.51.0:2667 2014-08-11 17:06:07.552641 2014-08-13 17:38:03.067253 7fe9bae887a0 20 read_log 192888'16900 (192888'16899) modify 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.942868.0:13847 2014-08-11 17:06:07.542820 2014-08-13 17:38:03.067270 7fe9bae887a0 20 read_log 192888'16903 (0'0) promote deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by osd.51.0:2675 2014-08-11 17:06:09.386954 2014-08-13 17:38:03.067282 7fe9bae887a0 20 read_log 192888'16904 (192888'16903) modify deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.942868.0:14285 2014-08-11 17:06:09.379028 2014-08-13 17:38:03.067295 7fe9bae887a0 20 read_log 192888'16905 (0'0) promote ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by osd.51.0:2683 2014-08-11 17:06:14.783958 2014-08-13 17:38:03.067307 7fe9bae887a0 20 read_log 192888'16906 (192888'16905) modify ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.942868.0:15658 2014-08-11 17:06:14.776593 2014-08-13 17:38:03.067319 7fe9bae887a0 20 read_log 192888'16907 (0'0) promote 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by osd.51.0:2703 2014-08-11 17:06:18.736788 2014-08-13 17:38:03.067332 7fe9bae887a0 20 read_log 192888'16908 (192888'16907) modify 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.942868.0:16662 2014-08-11 17:06:18.729252 2014-08-13 17:38:03.067345 7fe9bae887a0 20 read_log 192888'16909 (0'0) promote 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by osd.51.0:2713 2014-08-11 17:06:21.629647 2014-08-13 17:38:03.067357 7fe9bae887a0 20 read_log 192888'16910 (192888'16909) modify 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.942868.0:17415 2014-08-11 17:06:21.622734 2014-08-13 17:38:03.067369 7fe9bae887a0 20 read_log 192888'16913 (0'0) promote 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by osd.51.0:2719 2014-08-11 17:06:23.500127 2014-08-13 17:38:03.067381 7fe9bae887a0 20 read_log 192888'16914 (192888'16913) modify 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.942868.0:17912 2014-08-11 17:06:23.492270 2014-08-13 17:38:03.067393 7fe9bae887a0 20 read_log 192888'16915 (0'0) promote e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by osd.51.0:2727 2014-08-11 17:06:27.449820 2014-08-13 17:38:03.067405 7fe9bae887a0 20 read_log 192888'16916 (192888'16915) modify e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.942868.0:18962 2014-08-11 17:06:27.442272 2014-08-13 17:38:03.067417 7fe9bae887a0 20 read_log 192888'16917 (0'0) promote d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by osd.51.0:2735 2014-08-11 17:06:32.070302 2014-08-13 17:38:03.067429 7fe9bae887a0 20 read_log 192888'16918 (192888'16917) modify d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.942868.0:20189 2014-08-11 17:06:32.062416 2014-08-13 17:38:03.067441 7fe9bae887a0 20 read_log 192888'16919 (0'0) promote a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by osd.51.0:2739 2014-08-11 17:06:32.521282 2014-08-13 17:38:03.067453 7fe9bae887a0 20 read_log 192888'16920 (192888'16919) modify a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.942868.0:20311 2014-08-11 17:06:32.513382 2014-08-13 17:38:03.067465 7fe9bae887a0 20 read_log 192888'16921 (0'0) promote 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by osd.51.0:2743 2014-08-11 17:06:32.904790 2014-08-13 17:38:03.067477 7fe9bae887a0 20 read_log 192888'16922 (192888'16921) modify 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.942868.0:20397 2014-08-11 17:06:32.897078 2014-08-13 17:38:03.067489 7fe9bae887a0 20 read_log 192888'16927 (0'0) promote b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by osd.51.0:2755 2014-08-11 17:06:35.891790 2014-08-13 17:38:03.067507 7fe9bae887a0 20 read_log 192888'16928 (192888'16927) modify b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.942868.0:21174 2014-08-11 17:06:35.884458 2014-08-13 17:38:03.067520 7fe9bae887a0 20 read_log 192888'16929 (0'0) promote 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by osd.51.0:2757 2014-08-11 17:06:36.576736 2014-08-13 17:38:03.067532 7fe9bae887a0 20 read_log 192888'16930 (192888'16929) modify 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.942868.0:21355 2014-08-11 17:06:36.567013 2014-08-13 17:38:03.067544 7fe9bae887a0 20 read_log 192888'16933 (0'0) promote 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by osd.51.0:2765 2014-08-11 17:06:38.370879 2014-08-13 17:38:03.067556 7fe9bae887a0 20 read_log 192888'16934 (192888'16933) modify 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.942868.0:21837 2014-08-11 17:06:38.363160 2014-08-13 17:38:03.067569 7fe9bae887a0 20 read_log 192888'16939 (0'0) promote a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by osd.51.0:2781 2014-08-11 17:06:43.811619 2014-08-13 17:38:03.067581 7fe9bae887a0 20 read_log 192888'16940 (192888'16939) modify a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.942868.0:23212 2014-08-11 17:06:43.803959 2014-08-13 17:38:03.067593 7fe9bae887a0 20 read_log 192888'16945 (0'0) promote 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by osd.51.0:2803 2014-08-11 17:06:48.830144 2014-08-13 17:38:03.067604 7fe9bae887a0 20 read_log 192888'16946 (192888'16945) modify 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.942868.0:24571 2014-08-11 17:06:48.822047 2014-08-13 17:38:03.067616 7fe9bae887a0 20 read_log 192888'16951 (0'0) promote 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by osd.51.0:2825 2014-08-11 17:06:54.470316 2014-08-13 17:38:03.067628 7fe9bae887a0 20 read_log 192888'16952 (192888'16951) modify 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.942868.0:26019 2014-08-11 17:06:54.462128 2014-08-13 17:38:03.067640 7fe9bae887a0 20 read_log 192888'16953 (0'0) promote 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by osd.51.0:2829 2014-08-11 17:06:55.202313 2014-08-13 17:38:03.067652 7fe9bae887a0 20 read_log 192888'16954 (192888'16953) modify 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.942868.0:26198 2014-08-11 17:06:55.194148 2014-08-13 17:38:03.067664 7fe9bae887a0 20 read_log 192888'16955 (0'0) promote 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by osd.51.0:2835 2014-08-11 17:06:56.290335 2014-08-13 17:38:03.067676 7fe9bae887a0 20 read_log 192888'16956 (192888'16955) modify 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.942868.0:26493 2014-08-11 17:06:56.282823 2014-08-13 17:38:03.067688 7fe9bae887a0 20 read_log 192888'16957 (0'0) promote eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by osd.51.0:2841 2014-08-11 17:06:58.313749 2014-08-13 17:38:03.067699 7fe9bae887a0 20 read_log 192888'16958 (192888'16957) modify eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.942868.0:27024 2014-08-11 17:06:58.305684 2014-08-13 17:38:03.067712 7fe9bae887a0 20 read_log 192888'16961 (0'0) promote f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by osd.51.0:2849 2014-08-11 17:07:02.643901 2014-08-13 17:38:03.067723 7fe9bae887a0 20 read_log 192888'16962 (192888'16961) modify f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.942868.0:28178 2014-08-11 17:07:02.636718 2014-08-13 17:38:03.067735 7fe9bae887a0 20 read_log 192888'16967 (0'0) promote dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by osd.51.0:2863 2014-08-11 17:07:07.735958 2014-08-13 17:38:03.067756 7fe9bae887a0 20 read_log 192888'16968 (192888'16967) modify dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.942868.0:29481 2014-08-11 17:07:07.727174 2014-08-13 17:38:03.067772 7fe9bae887a0 20 read_log 192888'16969 (0'0) promote e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by osd.51.0:2871 2014-08-11 17:07:12.944188 2014-08-13 17:38:03.067784 7fe9bae887a0 20 read_log 192888'16970 (192888'16969) modify e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.942868.0:30778 2014-08-11 17:07:12.936368 2014-08-13 17:38:03.067797 7fe9bae887a0 20 read_log 192888'16973 (0'0) promote 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by osd.51.0:2877 2014-08-11 17:07:14.536729 2014-08-13 17:38:03.067811 7fe9bae887a0 20 read_log 192888'16974 (192888'16973) modify 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.942868.0:31190 2014-08-11 17:07:14.526514 2014-08-13 17:38:03.067823 7fe9bae887a0 20 read_log 192888'16978 (192888'16838) modify 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.942868.0:31859 2014-08-11 17:07:16.079265 2014-08-13 17:38:03.067835 7fe9bae887a0 20 read_log 192888'16981 (192888'16844) modify 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.942868.0:32102 2014-08-11 17:07:16.155528 2014-08-13 17:38:03.067847 7fe9bae887a0 20 read_log 192888'16982 (192888'16846) modify 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.942868.0:32425 2014-08-11 17:07:16.250078 2014-08-13 17:38:03.067861 7fe9bae887a0 20 read_log 192888'16983 (192888'16848) modify 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.942868.0:32470 2014-08-11 17:07:16.263268 2014-08-13 17:38:03.067873 7fe9bae887a0 20 read_log 192888'16985 (192888'16852) modify e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.942868.0:34056 2014-08-11 17:07:16.686599 2014-08-13 17:38:03.067885 7fe9bae887a0 20 read_log 192888'16986 (192888'16854) modify 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.942868.0:34292 2014-08-11 17:07:16.748452 2014-08-13 17:38:03.067897 7fe9bae887a0 20 read_log 192888'16987 (192888'16856) modify 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.942868.0:35224 2014-08-11 17:07:17.020184 2014-08-13 17:38:03.067909 7fe9bae887a0 20 read_log 192888'16990 (192888'16862) modify a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.942868.0:36581 2014-08-11 17:07:17.379751 2014-08-13 17:38:03.067921 7fe9bae887a0 20 read_log 192888'16991 (192888'16864) modify 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.942868.0:36846 2014-08-11 17:07:17.449171 2014-08-13 17:38:03.067933 7fe9bae887a0 20 read_log 192888'16992 (192888'16866) modify 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.942868.0:39317 2014-08-11 17:07:18.147515 2014-08-13 17:38:03.067944 7fe9bae887a0 20 read_log 192888'16997 (192888'16876) modify c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.942868.0:40665 2014-08-11 17:07:18.514436 2014-08-13 17:38:03.067956 7fe9bae887a0 20 read_log 192888'17000 (192888'16882) modify db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.942868.0:42301 2014-08-11 17:07:18.962047 2014-08-13 17:38:03.067968 7fe9bae887a0 20 read_log 192888'17003 (192888'16888) modify 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.942868.0:43208 2014-08-11 17:07:19.207821 2014-08-13 17:38:03.067980 7fe9bae887a0 20 read_log 192888'17006 (192888'16894) modify f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.942868.0:44273 2014-08-11 17:07:19.519128 2014-08-13 17:38:03.067992 7fe9bae887a0 20 read_log 192888'17007 (192888'16896) modify d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.942868.0:44288 2014-08-11 17:07:19.524137 2014-08-13 17:38:03.068011 7fe9bae887a0 20 read_log 192888'17009 (192888'16900) modify 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.942868.0:45113 2014-08-11 17:07:19.750103 2014-08-13 17:38:03.068024 7fe9bae887a0 20 read_log 192888'17011 (192888'16904) modify deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.942868.0:45551 2014-08-11 17:07:19.869505 2014-08-13 17:38:03.068036 7fe9bae887a0 20 read_log 192888'17012 (192888'16906) modify ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.942868.0:46924 2014-08-11 17:07:20.256382 2014-08-13 17:38:03.068048 7fe9bae887a0 20 read_log 192888'17013 (192888'16908) modify 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.942868.0:47928 2014-08-11 17:07:20.568465 2014-08-13 17:38:03.068060 7fe9bae887a0 20 read_log 192888'17014 (192888'16910) modify 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.942868.0:48681 2014-08-11 17:07:20.785727 2014-08-13 17:38:03.068072 7fe9bae887a0 20 read_log 192888'17016 (192888'16914) modify 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.942868.0:49178 2014-08-11 17:07:20.931894 2014-08-13 17:38:03.068084 7fe9bae887a0 20 read_log 192888'17017 (192888'16916) modify e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.942868.0:50228 2014-08-11 17:07:21.253477 2014-08-13 17:38:03.068096 7fe9bae887a0 20 read_log 192888'17018 (192888'16918) modify d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.942868.0:51455 2014-08-11 17:07:21.653708 2014-08-13 17:38:03.068107 7fe9bae887a0 20 read_log 192888'17019 (192888'16920) modify a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.942868.0:51577 2014-08-11 17:07:21.687141 2014-08-13 17:38:03.068119 7fe9bae887a0 20 read_log 192888'17020 (192888'16922) modify 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.942868.0:51663 2014-08-11 17:07:21.713923 2014-08-13 17:38:03.068131 7fe9bae887a0 20 read_log 192888'17023 (192888'16928) modify b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.942868.0:52440 2014-08-11 17:07:21.940939 2014-08-13 17:38:03.068165 7fe9bae887a0 20 read_log 192888'17024 (192888'16930) modify 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.942868.0:52621 2014-08-11 17:07:21.999472 2014-08-13 17:38:03.068178 7fe9bae887a0 20 read_log 192888'17026 (192888'16934) modify 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.942868.0:53103 2014-08-11 17:07:22.148402 2014-08-13 17:38:03.068191 7fe9bae887a0 20 read_log 192888'17029 (192888'16940) modify a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.942868.0:54478 2014-08-11 17:07:22.568773 2014-08-13 17:38:03.068203 7fe9bae887a0 20 read_log 192888'17032 (192888'16946) modify 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.942868.0:55837 2014-08-11 17:07:22.972350 2014-08-13 17:38:03.068215 7fe9bae887a0 20 read_log 192888'17035 (192888'16952) modify 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.942868.0:57285 2014-08-11 17:07:23.412466 2014-08-13 17:38:03.068227 7fe9bae887a0 20 read_log 192888'17036 (192888'16954) modify 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.942868.0:57464 2014-08-11 17:07:23.463170 2014-08-13 17:38:03.068243 7fe9bae887a0 20 read_log 192888'17037 (192888'16956) modify 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.942868.0:57759 2014-08-11 17:07:23.547536 2014-08-13 17:38:03.068255 7fe9bae887a0 20 read_log 192888'17038 (192888'16958) modify eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.942868.0:58290 2014-08-11 17:07:23.697059 2014-08-13 17:38:03.068267 7fe9bae887a0 20 read_log 192888'17040 (192888'16962) modify f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.942868.0:59444 2014-08-11 17:07:24.049065 2014-08-13 17:38:03.068287 7fe9bae887a0 20 read_log 192888'17043 (192888'16968) modify dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.942868.0:60747 2014-08-11 17:07:24.431819 2014-08-13 17:38:03.068300 7fe9bae887a0 20 read_log 192888'17044 (192888'16970) modify e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.942868.0:62044 2014-08-11 17:07:24.832056 2014-08-13 17:38:03.068312 7fe9bae887a0 20 read_log 192888'17046 (192888'16974) modify 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.942868.0:62456 2014-08-11 17:07:24.951367 2014-08-13 17:38:03.068324 7fe9bae887a0 20 read_log 193305'17047 (0'0) promote 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by osd.356.0:1 2014-08-11 18:42:33.802159 2014-08-13 17:38:03.068336 7fe9bae887a0 20 read_log 193305'17048 (193305'17047) modify 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.951337.0:217 2014-08-11 18:42:33.789721 2014-08-13 17:38:03.068348 7fe9bae887a0 20 read_log 193305'17049 (0'0) promote 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by osd.356.0:3 2014-08-11 18:42:35.868439 2014-08-13 17:38:03.068360 7fe9bae887a0 20 read_log 193305'17050 (193305'17049) modify 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.951337.0:694 2014-08-11 18:42:35.861295 2014-08-13 17:38:03.068372 7fe9bae887a0 20 read_log 193305'17051 (0'0) promote d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by osd.356.0:5 2014-08-11 18:42:36.054869 2014-08-13 17:38:03.068384 7fe9bae887a0 20 read_log 193305'17052 (193305'17051) modify d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.951337.0:746 2014-08-11 18:42:36.049345 2014-08-13 17:38:03.068397 7fe9bae887a0 20 read_log 193305'17053 (193305'17053) modify 350/hit_set_24.350_archive_2014-08-11 18:37:36.154490_2014-08-11 18:42:41.505389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.505523 2014-08-13 17:38:03.068409 7fe9bae887a0 20 read_log 193305'17054 (0'0) promote fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by osd.356.0:21 2014-08-11 18:42:41.507717 2014-08-13 17:38:03.068421 7fe9bae887a0 20 read_log 193305'17055 (193305'17054) modify fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.951337.0:2206 2014-08-11 18:42:41.501187 2014-08-13 17:38:03.068433 7fe9bae887a0 20 read_log 193305'17056 (0'0) promote f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by osd.356.0:43 2014-08-11 18:42:51.738517 2014-08-13 17:38:03.068445 7fe9bae887a0 20 read_log 193305'17057 (193305'17056) modify f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.951337.0:5003 2014-08-11 18:42:51.730540 2014-08-13 17:38:03.068476 7fe9bae887a0 20 read_log 193305'17058 (0'0) promote efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by osd.356.0:45 2014-08-11 18:42:54.217394 2014-08-13 17:38:03.068488 7fe9bae887a0 20 read_log 193305'17059 (193305'17058) modify efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.951337.0:5666 2014-08-11 18:42:54.206683 2014-08-13 17:38:03.068500 7fe9bae887a0 20 read_log 193305'17060 (0'0) promote 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by osd.356.0:51 2014-08-11 18:42:56.206075 2014-08-13 17:38:03.068512 7fe9bae887a0 20 read_log 193305'17061 (193305'17060) modify 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.951337.0:6214 2014-08-11 18:42:56.198000 2014-08-13 17:38:03.068524 7fe9bae887a0 20 read_log 193305'17062 (0'0) promote 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by osd.356.0:69 2014-08-11 18:43:05.953857 2014-08-13 17:38:03.068545 7fe9bae887a0 20 read_log 193305'17063 (193305'17062) modify 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.951337.0:8826 2014-08-11 18:43:05.946396 2014-08-13 17:38:03.068558 7fe9bae887a0 20 read_log 193305'17064 (0'0) promote d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by osd.356.0:79 2014-08-11 18:43:10.265706 2014-08-13 17:38:03.068570 7fe9bae887a0 20 read_log 193305'17065 (193305'17064) modify d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.951337.0:9992 2014-08-11 18:43:10.258690 2014-08-13 17:38:03.068582 7fe9bae887a0 20 read_log 193305'17066 (0'0) promote b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by osd.356.0:81 2014-08-11 18:43:11.278661 2014-08-13 17:38:03.068594 7fe9bae887a0 20 read_log 193305'17067 (193305'17066) modify b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.951337.0:10273 2014-08-11 18:43:11.270940 2014-08-13 17:38:03.068607 7fe9bae887a0 20 read_log 193305'17068 (0'0) promote d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by osd.356.0:89 2014-08-11 18:43:13.014672 2014-08-13 17:38:03.068619 7fe9bae887a0 20 read_log 193305'17069 (193305'17068) modify d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.951337.0:10734 2014-08-11 18:43:13.007355 2014-08-13 17:38:03.068631 7fe9bae887a0 20 read_log 193305'17070 (0'0) promote 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by osd.356.0:95 2014-08-11 18:43:15.378357 2014-08-13 17:38:03.068643 7fe9bae887a0 20 read_log 193305'17071 (193305'17070) modify 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.951337.0:11321 2014-08-11 18:43:15.370364 2014-08-13 17:38:03.068655 7fe9bae887a0 20 read_log 193305'17072 (0'0) promote 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by osd.356.0:99 2014-08-11 18:43:16.521862 2014-08-13 17:38:03.068667 7fe9bae887a0 20 read_log 193305'17073 (193305'17072) modify 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.951337.0:11627 2014-08-11 18:43:16.514615 2014-08-13 17:38:03.068679 7fe9bae887a0 20 read_log 193305'17074 (0'0) promote f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by osd.356.0:111 2014-08-11 18:43:22.863547 2014-08-13 17:38:03.068691 7fe9bae887a0 20 read_log 193305'17075 (193305'17074) modify f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.951337.0:13330 2014-08-11 18:43:22.855741 2014-08-13 17:38:03.068703 7fe9bae887a0 20 read_log 193305'17076 (0'0) promote d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by osd.356.0:117 2014-08-11 18:43:24.419895 2014-08-13 17:38:03.068715 7fe9bae887a0 20 read_log 193305'17077 (193305'17076) modify d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.951337.0:13729 2014-08-11 18:43:24.413797 2014-08-13 17:38:03.068727 7fe9bae887a0 20 read_log 193305'17078 (0'0) promote 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by osd.356.0:121 2014-08-11 18:43:25.529223 2014-08-13 17:38:03.068739 7fe9bae887a0 20 read_log 193305'17079 (193305'17078) modify 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.951337.0:14044 2014-08-11 18:43:25.521270 2014-08-13 17:38:03.068751 7fe9bae887a0 20 read_log 193305'17080 (0'0) promote 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by osd.356.0:125 2014-08-11 18:43:26.193569 2014-08-13 17:38:03.068763 7fe9bae887a0 20 read_log 193305'17081 (193305'17080) modify 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.951337.0:14186 2014-08-11 18:43:26.185086 2014-08-13 17:38:03.068775 7fe9bae887a0 20 read_log 193305'17082 (0'0) promote cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by osd.356.0:145 2014-08-11 18:43:30.984457 2014-08-13 17:38:03.068793 7fe9bae887a0 20 read_log 193305'17083 (193305'17082) modify cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.951337.0:15595 2014-08-11 18:43:30.974035 2014-08-13 17:38:03.068806 7fe9bae887a0 20 read_log 193305'17084 (0'0) promote 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by osd.356.0:149 2014-08-11 18:43:34.749255 2014-08-13 17:38:03.068818 7fe9bae887a0 20 read_log 193305'17085 (193305'17084) modify 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.951337.0:16670 2014-08-11 18:43:34.742308 2014-08-13 17:38:03.068830 7fe9bae887a0 20 read_log 193305'17086 (0'0) promote 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by osd.356.0:159 2014-08-11 18:43:37.081002 2014-08-13 17:38:03.068842 7fe9bae887a0 20 read_log 193305'17087 (193305'17086) modify 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.951337.0:17332 2014-08-11 18:43:37.072943 2014-08-13 17:38:03.068854 7fe9bae887a0 20 read_log 193305'17088 (0'0) promote f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by osd.356.0:171 2014-08-11 18:43:42.713712 2014-08-13 17:38:03.068866 7fe9bae887a0 20 read_log 193305'17089 (193305'17088) modify f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.951337.0:18935 2014-08-11 18:43:42.706790 2014-08-13 17:38:03.068878 7fe9bae887a0 20 read_log 193305'17090 (0'0) promote 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by osd.356.0:177 2014-08-11 18:43:45.883199 2014-08-13 17:38:03.068890 7fe9bae887a0 20 read_log 193305'17091 (193305'17090) modify 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.951337.0:19794 2014-08-11 18:43:45.876639 2014-08-13 17:38:03.068902 7fe9bae887a0 20 read_log 193305'17092 (0'0) promote 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by osd.356.0:185 2014-08-11 18:43:49.452087 2014-08-13 17:38:03.068913 7fe9bae887a0 20 read_log 193305'17093 (193305'17092) modify 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.951337.0:20791 2014-08-11 18:43:49.444640 2014-08-13 17:38:03.068925 7fe9bae887a0 20 read_log 193305'17094 (0'0) promote a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by osd.356.0:191 2014-08-11 18:43:52.027819 2014-08-13 17:38:03.068937 7fe9bae887a0 20 read_log 193305'17095 (193305'17094) modify a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.951337.0:21507 2014-08-11 18:43:52.019578 2014-08-13 17:38:03.068949 7fe9bae887a0 20 read_log 193305'17096 (0'0) promote f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by osd.356.0:201 2014-08-11 18:43:56.648858 2014-08-13 17:38:03.068961 7fe9bae887a0 20 read_log 193305'17097 (193305'17096) modify f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.951337.0:22773 2014-08-11 18:43:56.641559 2014-08-13 17:38:03.068973 7fe9bae887a0 20 read_log 193305'17098 (0'0) promote c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by osd.356.0:209 2014-08-11 18:43:58.678251 2014-08-13 17:38:03.068985 7fe9bae887a0 20 read_log 193305'17099 (193305'17098) modify c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.951337.0:23354 2014-08-11 18:43:58.671098 2014-08-13 17:38:03.068997 7fe9bae887a0 20 read_log 193305'17100 (0'0) promote f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by osd.356.0:213 2014-08-11 18:44:00.458243 2014-08-13 17:38:03.069009 7fe9bae887a0 20 read_log 193305'17101 (193305'17100) modify f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.951337.0:23845 2014-08-11 18:44:00.449613 2014-08-13 17:38:03.069021 7fe9bae887a0 20 read_log 193305'17102 (0'0) promote 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by osd.356.0:217 2014-08-11 18:44:04.865607 2014-08-13 17:38:03.069055 7fe9bae887a0 20 read_log 193305'17103 (193305'17102) modify 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.951337.0:25056 2014-08-11 18:44:04.859683 2014-08-13 17:38:03.069068 7fe9bae887a0 20 read_log 193305'17104 (0'0) promote ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by osd.356.0:243 2014-08-11 18:44:17.193329 2014-08-13 17:38:03.069081 7fe9bae887a0 20 read_log 193305'17105 (193305'17104) modify ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.951337.0:28425 2014-08-11 18:44:17.184770 2014-08-13 17:38:03.069093 7fe9bae887a0 20 read_log 193305'17106 (0'0) promote c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by osd.356.0:253 2014-08-11 18:44:21.863605 2014-08-13 17:38:03.069105 7fe9bae887a0 20 read_log 193305'17107 (193305'17106) modify c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.951337.0:29695 2014-08-11 18:44:21.855461 2014-08-13 17:38:03.069117 7fe9bae887a0 20 read_log 193305'17108 (193305'17048) modify 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.951337.0:32928 2014-08-11 18:44:33.962268 2014-08-13 17:38:03.069129 7fe9bae887a0 20 read_log 193305'17109 (193305'17050) modify 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.951337.0:33405 2014-08-11 18:44:34.388894 2014-08-13 17:38:03.069141 7fe9bae887a0 20 read_log 193305'17110 (193305'17052) modify d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.951337.0:33457 2014-08-11 18:44:34.437696 2014-08-13 17:38:03.069153 7fe9bae887a0 20 read_log 193305'17111 (193305'17055) modify fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.951337.0:34917 2014-08-11 18:44:35.258149 2014-08-13 17:38:03.069170 7fe9bae887a0 20 read_log 193305'17112 (193305'17057) modify f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.951337.0:37714 2014-08-11 18:44:37.034529 2014-08-13 17:38:03.069182 7fe9bae887a0 20 read_log 193305'17113 (193305'17059) modify efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.951337.0:38377 2014-08-11 18:44:37.381250 2014-08-13 17:38:03.069194 7fe9bae887a0 20 read_log 193305'17114 (193305'17061) modify 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.951337.0:38925 2014-08-11 18:44:37.737430 2014-08-13 17:38:03.069206 7fe9bae887a0 20 read_log 193305'17115 (193305'17063) modify 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.951337.0:41537 2014-08-11 18:44:38.785552 2014-08-13 17:38:03.069217 7fe9bae887a0 20 read_log 193305'17116 (193305'17065) modify d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.951337.0:42703 2014-08-11 18:44:39.356617 2014-08-13 17:38:03.069232 7fe9bae887a0 20 read_log 193305'17117 (193305'17067) modify b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.951337.0:42984 2014-08-11 18:44:39.444342 2014-08-13 17:38:03.069244 7fe9bae887a0 20 read_log 193305'17118 (193305'17069) modify d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.951337.0:43445 2014-08-11 18:44:39.592492 2014-08-13 17:38:03.069256 7fe9bae887a0 20 read_log 193305'17119 (193305'17071) modify 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.951337.0:44032 2014-08-11 18:44:39.774287 2014-08-13 17:38:03.069268 7fe9bae887a0 20 read_log 193305'17120 (193305'17073) modify 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.951337.0:44338 2014-08-11 18:44:39.856607 2014-08-13 17:38:03.069280 7fe9bae887a0 20 read_log 193305'17121 (193305'17075) modify f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.951337.0:46041 2014-08-11 18:44:40.373310 2014-08-13 17:38:03.069292 7fe9bae887a0 20 read_log 193305'17122 (193305'17077) modify d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.951337.0:46440 2014-08-11 18:44:40.482582 2014-08-13 17:38:03.069313 7fe9bae887a0 20 read_log 193305'17123 (193305'17079) modify 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.951337.0:46755 2014-08-11 18:44:40.579895 2014-08-13 17:38:03.069326 7fe9bae887a0 20 read_log 193305'17124 (193305'17081) modify 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.951337.0:46897 2014-08-11 18:44:40.620662 2014-08-13 17:38:03.069338 7fe9bae887a0 20 read_log 193305'17125 (193305'17083) modify cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.951337.0:48306 2014-08-11 18:44:41.015955 2014-08-13 17:38:03.069350 7fe9bae887a0 20 read_log 193305'17126 (193305'17085) modify 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.951337.0:49381 2014-08-11 18:44:41.349136 2014-08-13 17:38:03.069362 7fe9bae887a0 20 read_log 193305'17127 (193305'17087) modify 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.951337.0:50043 2014-08-11 18:44:41.547409 2014-08-13 17:38:03.069394 7fe9bae887a0 20 read_log 193305'17128 (193305'17089) modify f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.951337.0:51646 2014-08-11 18:44:42.009807 2014-08-13 17:38:03.069407 7fe9bae887a0 20 read_log 193305'17129 (193305'17091) modify 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.951337.0:52505 2014-08-11 18:44:42.251727 2014-08-13 17:38:03.069419 7fe9bae887a0 20 read_log 193305'17130 (193305'17093) modify 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.951337.0:53502 2014-08-11 18:44:42.548126 2014-08-13 17:38:03.069431 7fe9bae887a0 20 read_log 193305'17131 (193305'17095) modify a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.951337.0:54218 2014-08-11 18:44:42.764693 2014-08-13 17:38:03.069443 7fe9bae887a0 20 read_log 193305'17132 (193305'17097) modify f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.951337.0:55484 2014-08-11 18:44:43.140563 2014-08-13 17:38:03.069455 7fe9bae887a0 20 read_log 193305'17133 (193305'17099) modify c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.951337.0:56065 2014-08-11 18:44:43.311153 2014-08-13 17:38:03.069466 7fe9bae887a0 20 read_log 193305'17134 (193305'17101) modify f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.951337.0:56556 2014-08-11 18:44:43.453315 2014-08-13 17:38:03.069478 7fe9bae887a0 20 read_log 193305'17135 (193305'17103) modify 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.951337.0:57767 2014-08-11 18:44:43.854226 2014-08-13 17:38:03.069490 7fe9bae887a0 20 read_log 193305'17136 (193305'17105) modify ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.951337.0:61136 2014-08-11 18:44:44.834288 2014-08-13 17:38:03.069502 7fe9bae887a0 20 read_log 193305'17137 (193305'17107) modify c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.951337.0:62406 2014-08-11 18:44:45.218117 2014-08-13 17:38:03.069514 7fe9bae887a0 20 read_log 194215'17138 (194215'17138) modify 350/hit_set_24.350_archive_2014-08-12 15:32:25.978048_2014-08-12 15:49:16.440311/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:16.440550 2014-08-13 17:38:03.069527 7fe9bae887a0 20 read_log 194215'17139 (193305'17053) delete 350/hit_set_24.350_archive_2014-08-11 18:37:36.154490_2014-08-11 18:42:41.505389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:16.440550 2014-08-13 17:38:03.069539 7fe9bae887a0 20 read_log 194215'17140 (192888'17016) clean 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by osd.356.0:1047 2014-08-12 15:49:16.444986 2014-08-13 17:38:03.069551 7fe9bae887a0 20 read_log 194215'17141 (194215'17140) delete 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.2174595.0:174649 0.000000 2014-08-13 17:38:03.069570 7fe9bae887a0 20 read_log 194215'17142 (192888'17000) clean db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by osd.356.0:1049 2014-08-12 15:49:16.453124 2014-08-13 17:38:03.069583 7fe9bae887a0 20 read_log 194215'17143 (194215'17142) delete db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.2174595.0:174651 0.000000 2014-08-13 17:38:03.069594 7fe9bae887a0 20 read_log 194215'17144 (193305'17135) clean 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by osd.356.0:1051 2014-08-12 15:49:16.460690 2014-08-13 17:38:03.069606 7fe9bae887a0 20 read_log 194215'17145 (194215'17144) delete 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.2174595.0:174653 0.000000 2014-08-13 17:38:03.069616 7fe9bae887a0 20 read_log 194215'17146 (192888'16981) clean 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by osd.356.0:1053 2014-08-12 15:49:16.467683 2014-08-13 17:38:03.069628 7fe9bae887a0 20 read_log 194215'17147 (194215'17146) delete 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.2174595.0:174655 0.000000 2014-08-13 17:38:03.069639 7fe9bae887a0 20 read_log 194215'17148 (192888'16987) clean 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by osd.356.0:1055 2014-08-12 15:49:16.474986 2014-08-13 17:38:03.069651 7fe9bae887a0 20 read_log 194215'17149 (194215'17148) delete 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.2174595.0:174657 0.000000 2014-08-13 17:38:03.069662 7fe9bae887a0 20 read_log 194215'17150 (193305'17127) clean 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by osd.356.0:1057 2014-08-12 15:49:16.482525 2014-08-13 17:38:03.069674 7fe9bae887a0 20 read_log 194215'17151 (194215'17150) delete 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.2174595.0:174659 0.000000 2014-08-13 17:38:03.069684 7fe9bae887a0 20 read_log 194215'17152 (192888'16784) clean 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by osd.356.0:1059 2014-08-12 15:49:16.490115 2014-08-13 17:38:03.069696 7fe9bae887a0 20 read_log 194215'17153 (194215'17152) delete 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.2174595.0:174661 0.000000 2014-08-13 17:38:03.069707 7fe9bae887a0 20 read_log 194215'17154 (192888'16783) clean c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by osd.356.0:1061 2014-08-12 15:49:16.497736 2014-08-13 17:38:03.069718 7fe9bae887a0 20 read_log 194215'17155 (194215'17154) delete c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.2174595.0:174663 0.000000 2014-08-13 17:38:03.069729 7fe9bae887a0 20 read_log 194215'17156 (192888'17023) clean b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by osd.356.0:1063 2014-08-12 15:49:16.505080 2014-08-13 17:38:03.069741 7fe9bae887a0 20 read_log 194215'17157 (194215'17156) delete b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.2174595.0:174665 0.000000 2014-08-13 17:38:03.069752 7fe9bae887a0 20 read_log 194215'17158 (192888'16824) clean 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by osd.356.0:1065 2014-08-12 15:49:16.512192 2014-08-13 17:38:03.069764 7fe9bae887a0 20 read_log 194215'17159 (194215'17158) delete 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.2174595.0:174667 0.000000 2014-08-13 17:38:03.069774 7fe9bae887a0 20 read_log 194215'17160 (193305'17120) clean 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by osd.356.0:1067 2014-08-12 15:49:16.519349 2014-08-13 17:38:03.069786 7fe9bae887a0 20 read_log 194215'17161 (194215'17160) delete 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.2174595.0:174669 0.000000 2014-08-13 17:38:03.069803 7fe9bae887a0 20 read_log 194215'17162 (193305'17134) clean f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by osd.356.0:1069 2014-08-12 15:49:16.526266 2014-08-13 17:38:03.069816 7fe9bae887a0 20 read_log 194215'17163 (194215'17162) delete f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.2174595.0:174671 0.000000 2014-08-13 17:38:03.069827 7fe9bae887a0 20 read_log 194215'17164 (192888'16985) clean e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by osd.356.0:1071 2014-08-12 15:49:16.534061 2014-08-13 17:38:03.069839 7fe9bae887a0 20 read_log 194215'17165 (194215'17164) delete e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.2174595.0:174673 0.000000 2014-08-13 17:38:03.069850 7fe9bae887a0 20 read_log 194215'17166 (193305'17124) clean 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by osd.356.0:1073 2014-08-12 15:49:16.541323 2014-08-13 17:38:03.069862 7fe9bae887a0 20 read_log 194215'17167 (194215'17166) delete 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.2174595.0:174675 0.000000 2014-08-13 17:38:03.069872 7fe9bae887a0 20 read_log 194215'17168 (193305'17108) clean 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by osd.356.0:1075 2014-08-12 15:49:16.548306 2014-08-13 17:38:03.069884 7fe9bae887a0 20 read_log 194215'17169 (194215'17168) delete 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.2174595.0:174677 0.000000 2014-08-13 17:38:03.069895 7fe9bae887a0 20 read_log 194215'17170 (193305'17129) clean 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by osd.356.0:1077 2014-08-12 15:49:16.556725 2014-08-13 17:38:03.069907 7fe9bae887a0 20 read_log 194215'17171 (194215'17170) delete 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.2174595.0:174679 0.000000 2014-08-13 17:38:03.069917 7fe9bae887a0 20 read_log 194215'17172 (192888'16785) clean 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by osd.356.0:1079 2014-08-12 15:49:16.562791 2014-08-13 17:38:03.069929 7fe9bae887a0 20 read_log 194215'17173 (194215'17172) delete 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.2174595.0:174681 0.000000 2014-08-13 17:38:03.069940 7fe9bae887a0 20 read_log 194215'17174 (192888'16821) clean c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by osd.356.0:1081 2014-08-12 15:49:16.569706 2014-08-13 17:38:03.069951 7fe9bae887a0 20 read_log 194215'17175 (194215'17174) delete c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.2174595.0:174683 0.000000 2014-08-13 17:38:03.069962 7fe9bae887a0 20 read_log 194215'17176 (193305'17121) clean f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by osd.356.0:1083 2014-08-12 15:49:16.576250 2014-08-13 17:38:03.069974 7fe9bae887a0 20 read_log 194215'17177 (194215'17176) delete f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.2174595.0:174685 0.000000 2014-08-13 17:38:03.069984 7fe9bae887a0 20 read_log 194215'17178 (192888'16810) clean 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by osd.356.0:1085 2014-08-12 15:49:16.584084 2014-08-13 17:38:03.069996 7fe9bae887a0 20 read_log 194215'17179 (194215'17178) delete 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.2174595.0:174687 0.000000 2014-08-13 17:38:03.070007 7fe9bae887a0 20 read_log 194215'17180 (193305'17131) clean a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by osd.356.0:1087 2014-08-12 15:49:16.591897 2014-08-13 17:38:03.070018 7fe9bae887a0 20 read_log 194215'17181 (194215'17180) delete a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.2174595.0:174689 0.000000 2014-08-13 17:38:03.070038 7fe9bae887a0 20 read_log 194215'17182 (192888'16802) clean 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by osd.356.0:1089 2014-08-12 15:49:16.599071 2014-08-13 17:38:03.070051 7fe9bae887a0 20 read_log 194215'17183 (194215'17182) delete 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.2174595.0:174691 0.000000 2014-08-13 17:38:03.070062 7fe9bae887a0 20 read_log 194215'17184 (193305'17132) clean f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by osd.356.0:1091 2014-08-12 15:49:16.605344 2014-08-13 17:38:03.070073 7fe9bae887a0 20 read_log 194215'17185 (194215'17184) delete f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.2174595.0:174693 0.000000 2014-08-13 17:38:03.070084 7fe9bae887a0 20 read_log 194215'17186 (192888'17018) clean d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by osd.356.0:1093 2014-08-12 15:49:16.611612 2014-08-13 17:38:03.070096 7fe9bae887a0 20 read_log 194215'17187 (194215'17186) delete d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.2174595.0:174695 0.000000 2014-08-13 17:38:03.070106 7fe9bae887a0 20 read_log 194215'17188 (192888'17006) clean f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by osd.356.0:1095 2014-08-12 15:49:16.617911 2014-08-13 17:38:03.070119 7fe9bae887a0 20 read_log 194215'17189 (194215'17188) delete f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.2174595.0:174697 0.000000 2014-08-13 17:38:03.070129 7fe9bae887a0 20 read_log 194215'17190 (193305'17118) clean d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by osd.356.0:1097 2014-08-12 15:49:16.624032 2014-08-13 17:38:03.070141 7fe9bae887a0 20 read_log 194215'17191 (194215'17190) delete d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.2174595.0:174699 0.000000 2014-08-13 17:38:03.070152 7fe9bae887a0 20 read_log 194215'17192 (192888'17040) clean f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by osd.356.0:1099 2014-08-12 15:49:16.630768 2014-08-13 17:38:03.070169 7fe9bae887a0 20 read_log 194215'17193 (194215'17192) delete f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.2174595.0:174701 0.000000 2014-08-13 17:38:03.070180 7fe9bae887a0 20 read_log 194215'17194 (192888'17017) clean e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by osd.356.0:1101 2014-08-12 15:49:16.637659 2014-08-13 17:38:03.070192 7fe9bae887a0 20 read_log 194215'17195 (194215'17194) delete e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.2174595.0:174703 0.000000 2014-08-13 17:38:03.070203 7fe9bae887a0 20 read_log 194215'17196 (192888'16804) clean 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by osd.356.0:1103 2014-08-12 15:49:16.643993 2014-08-13 17:38:03.070215 7fe9bae887a0 20 read_log 194215'17197 (194215'17196) delete 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.2174595.0:174705 0.000000 2014-08-13 17:38:03.070226 7fe9bae887a0 20 read_log 194215'17198 (192888'16791) clean 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by osd.356.0:1105 2014-08-12 15:49:16.650325 2014-08-13 17:38:03.070238 7fe9bae887a0 20 read_log 194215'17199 (194215'17198) delete 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.2174595.0:174707 0.000000 2014-08-13 17:38:03.070248 7fe9bae887a0 20 read_log 194215'17200 (192888'16822) clean 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by osd.356.0:1107 2014-08-12 15:49:16.657736 2014-08-13 17:38:03.070260 7fe9bae887a0 20 read_log 194215'17201 (194215'17200) delete 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.2174595.0:174709 0.000000 2014-08-13 17:38:03.070278 7fe9bae887a0 20 read_log 194215'17202 (192888'16801) clean e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by osd.356.0:1109 2014-08-12 15:49:16.664471 2014-08-13 17:38:03.070290 7fe9bae887a0 20 read_log 194215'17203 (194215'17202) delete e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.2174595.0:174711 0.000000 2014-08-13 17:38:03.070301 7fe9bae887a0 20 read_log 194215'17204 (192888'16825) clean 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by osd.356.0:1111 2014-08-12 15:49:16.671531 2014-08-13 17:38:03.070313 7fe9bae887a0 20 read_log 194215'17205 (194215'17204) delete 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.2174595.0:174713 0.000000 2014-08-13 17:38:03.070323 7fe9bae887a0 20 read_log 194215'17206 (193305'17130) clean 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by osd.356.0:1113 2014-08-12 15:49:16.678484 2014-08-13 17:38:03.070335 7fe9bae887a0 20 read_log 194215'17207 (194215'17206) delete 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.2174595.0:174715 0.000000 2014-08-13 17:38:03.070346 7fe9bae887a0 20 read_log 194215'17208 (192888'17009) clean 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by osd.356.0:1115 2014-08-12 15:49:16.685368 2014-08-13 17:38:03.070358 7fe9bae887a0 20 read_log 194215'17209 (194215'17208) delete 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.2174595.0:174717 0.000000 2014-08-13 17:38:03.070369 7fe9bae887a0 20 read_log 194215'17210 (192888'16991) clean 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by osd.356.0:1117 2014-08-12 15:49:16.692352 2014-08-13 17:38:03.070381 7fe9bae887a0 20 read_log 194215'17211 (194215'17210) delete 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.2174595.0:174719 0.000000 2014-08-13 17:38:03.070391 7fe9bae887a0 20 read_log 194215'17212 (193305'17126) clean 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by osd.356.0:1119 2014-08-12 15:49:16.699620 2014-08-13 17:38:03.070403 7fe9bae887a0 20 read_log 194215'17213 (194215'17212) delete 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.2174595.0:174721 0.000000 2014-08-13 17:38:03.070414 7fe9bae887a0 20 read_log 194215'17214 (192888'16805) clean 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by osd.356.0:1121 2014-08-12 15:49:16.705123 2014-08-13 17:38:03.070448 7fe9bae887a0 20 read_log 194215'17215 (194215'17214) delete 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.2174595.0:174723 0.000000 2014-08-13 17:38:03.070458 7fe9bae887a0 20 read_log 194215'17216 (192888'17011) clean deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by osd.356.0:1123 2014-08-12 15:49:16.710426 2014-08-13 17:38:03.070472 7fe9bae887a0 20 read_log 194215'17217 (194215'17216) delete deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.2174595.0:174725 0.000000 2014-08-13 17:38:03.070482 7fe9bae887a0 20 read_log 194215'17218 (192888'17046) clean 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by osd.356.0:1125 2014-08-12 15:49:16.715928 2014-08-13 17:38:03.070495 7fe9bae887a0 20 read_log 194215'17219 (194215'17218) delete 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.2174595.0:174727 0.000000 2014-08-13 17:38:03.070505 7fe9bae887a0 20 read_log 194215'17220 (192888'17026) clean 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by osd.356.0:1127 2014-08-12 15:49:16.721060 2014-08-13 17:38:03.070532 7fe9bae887a0 20 read_log 194215'17221 (194215'17220) delete 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.2174595.0:174729 0.000000 2014-08-13 17:38:03.070550 7fe9bae887a0 20 read_log 194215'17222 (192888'17014) clean 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by osd.356.0:1129 2014-08-12 15:49:16.726399 2014-08-13 17:38:03.070563 7fe9bae887a0 20 read_log 194215'17223 (194215'17222) delete 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.2174595.0:174731 0.000000 2014-08-13 17:38:03.070574 7fe9bae887a0 20 read_log 194215'17224 (193305'17113) clean efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by osd.356.0:1131 2014-08-12 15:49:16.731880 2014-08-13 17:38:03.070586 7fe9bae887a0 20 read_log 194215'17225 (194215'17224) delete efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.2174595.0:174733 0.000000 2014-08-13 17:38:03.070596 7fe9bae887a0 20 read_log 194215'17226 (192888'16798) clean 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by osd.356.0:1133 2014-08-12 15:49:16.737276 2014-08-13 17:38:03.070609 7fe9bae887a0 20 read_log 194215'17227 (194215'17226) delete 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.2174595.0:174735 0.000000 2014-08-13 17:38:03.070619 7fe9bae887a0 20 read_log 194215'17228 (192888'17013) clean 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by osd.356.0:1135 2014-08-12 15:49:16.742388 2014-08-13 17:38:03.070631 7fe9bae887a0 20 read_log 194215'17229 (194215'17228) delete 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.2174595.0:174737 0.000000 2014-08-13 17:38:03.070642 7fe9bae887a0 20 read_log 194215'17230 (192888'16819) clean f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by osd.356.0:1137 2014-08-12 15:49:16.747780 2014-08-13 17:38:03.070654 7fe9bae887a0 20 read_log 194215'17231 (194215'17230) delete f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.2174595.0:174739 0.000000 2014-08-13 17:38:03.070664 7fe9bae887a0 20 read_log 194215'17232 (192888'16778) clean 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by osd.356.0:1139 2014-08-12 15:49:16.752999 2014-08-13 17:38:03.070676 7fe9bae887a0 20 read_log 194215'17233 (194215'17232) delete 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.2174595.0:174741 0.000000 2014-08-13 17:38:03.070687 7fe9bae887a0 20 read_log 194215'17234 (193305'17109) clean 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by osd.356.0:1141 2014-08-12 15:49:16.758453 2014-08-13 17:38:03.070698 7fe9bae887a0 20 read_log 194215'17235 (194215'17234) delete 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.2174595.0:174743 0.000000 2014-08-13 17:38:03.070709 7fe9bae887a0 20 read_log 194215'17236 (193305'17117) clean b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by osd.356.0:1143 2014-08-12 15:49:16.764157 2014-08-13 17:38:03.070721 7fe9bae887a0 20 read_log 194215'17237 (194215'17236) delete b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.2174595.0:174745 0.000000 2014-08-13 17:38:03.070732 7fe9bae887a0 20 read_log 194215'17238 (192888'17037) clean 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by osd.356.0:1145 2014-08-12 15:49:16.769482 2014-08-13 17:38:03.070744 7fe9bae887a0 20 read_log 194215'17239 (194215'17238) delete 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.2174595.0:174747 0.000000 2014-08-13 17:38:03.070754 7fe9bae887a0 20 read_log 194215'17240 (192888'16809) clean 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by osd.356.0:1147 2014-08-12 15:49:16.775023 2014-08-13 17:38:03.070766 7fe9bae887a0 20 read_log 194215'17241 (194215'17240) delete 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.2174595.0:174749 0.000000 2014-08-13 17:38:03.070802 7fe9bae887a0 20 read_log 194215'17242 (192888'16775) clean b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by osd.356.0:1149 2014-08-12 15:49:16.780471 2014-08-13 17:38:03.070815 7fe9bae887a0 20 read_log 194215'17243 (194215'17242) delete b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.2174595.0:174751 0.000000 2014-08-13 17:38:03.070826 7fe9bae887a0 20 read_log 194215'17244 (192888'17020) clean 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by osd.356.0:1151 2014-08-12 15:49:16.786218 2014-08-13 17:38:03.070838 7fe9bae887a0 20 read_log 194215'17245 (194215'17244) delete 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.2174595.0:174753 0.000000 2014-08-13 17:38:03.070849 7fe9bae887a0 20 read_log 194215'17246 (192888'17019) clean a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by osd.356.0:1153 2014-08-12 15:49:16.792047 2014-08-13 17:38:03.070861 7fe9bae887a0 20 read_log 194215'17247 (194215'17246) delete a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.2174595.0:174755 0.000000 2014-08-13 17:38:03.070871 7fe9bae887a0 20 read_log 194215'17248 (192888'17012) clean ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by osd.356.0:1155 2014-08-12 15:49:16.797673 2014-08-13 17:38:03.070884 7fe9bae887a0 20 read_log 194215'17249 (194215'17248) delete ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.2174595.0:174757 0.000000 2014-08-13 17:38:03.070895 7fe9bae887a0 20 read_log 194215'17250 (192888'16982) clean 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by osd.356.0:1157 2014-08-12 15:49:16.803673 2014-08-13 17:38:03.070907 7fe9bae887a0 20 read_log 194215'17251 (194215'17250) delete 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.2174595.0:174759 0.000000 2014-08-13 17:38:03.070917 7fe9bae887a0 20 read_log 194215'17252 (193305'17115) clean 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by osd.356.0:1159 2014-08-12 15:49:16.810051 2014-08-13 17:38:03.070929 7fe9bae887a0 20 read_log 194215'17253 (194215'17252) delete 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.2174595.0:174761 0.000000 2014-08-13 17:38:03.070939 7fe9bae887a0 20 read_log 194215'17254 (192888'17032) clean 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by osd.356.0:1161 2014-08-12 15:49:16.816705 2014-08-13 17:38:03.070951 7fe9bae887a0 20 read_log 194215'17255 (194215'17254) delete 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.2174595.0:174763 0.000000 2014-08-13 17:38:03.070962 7fe9bae887a0 20 read_log 194215'17256 (193305'17136) clean ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by osd.356.0:1163 2014-08-12 15:49:16.823746 2014-08-13 17:38:03.070974 7fe9bae887a0 20 read_log 194215'17257 (194215'17256) delete ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.2174595.0:174765 0.000000 2014-08-13 17:38:03.070985 7fe9bae887a0 20 read_log 194215'17258 (193305'17114) clean 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by osd.356.0:1165 2014-08-12 15:49:16.830392 2014-08-13 17:38:03.070997 7fe9bae887a0 20 read_log 194215'17259 (194215'17258) delete 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.2174595.0:174767 0.000000 2014-08-13 17:38:03.071008 7fe9bae887a0 20 read_log 194215'17260 (193305'17110) clean d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by osd.356.0:1167 2014-08-12 15:49:16.837126 2014-08-13 17:38:03.071020 7fe9bae887a0 20 read_log 194215'17261 (194215'17260) delete d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.2174595.0:174769 0.000000 2014-08-13 17:38:03.071037 7fe9bae887a0 20 read_log 194215'17262 (192888'16990) clean a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by osd.356.0:1169 2014-08-12 15:49:16.843927 2014-08-13 17:38:03.071049 7fe9bae887a0 20 read_log 194215'17263 (194215'17262) delete a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.2174595.0:174771 0.000000 2014-08-13 17:38:03.071060 7fe9bae887a0 20 read_log 194215'17264 (192888'17044) clean e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by osd.356.0:1171 2014-08-12 15:49:16.850556 2014-08-13 17:38:03.071072 7fe9bae887a0 20 read_log 194215'17265 (194215'17264) delete e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.2174595.0:174773 0.000000 2014-08-13 17:38:03.071082 7fe9bae887a0 20 read_log 194215'17266 (192888'17036) clean 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by osd.356.0:1173 2014-08-12 15:49:16.857886 2014-08-13 17:38:03.071094 7fe9bae887a0 20 read_log 194215'17267 (194215'17266) delete 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.2174595.0:174775 0.000000 2014-08-13 17:38:03.071105 7fe9bae887a0 20 read_log 194215'17268 (192888'16790) clean 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by osd.356.0:1175 2014-08-12 15:49:16.864360 2014-08-13 17:38:03.071117 7fe9bae887a0 20 read_log 194215'17269 (194215'17268) delete 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.2174595.0:174777 0.000000 2014-08-13 17:38:03.071127 7fe9bae887a0 20 read_log 194215'17270 (192888'16787) clean 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by osd.356.0:1177 2014-08-12 15:49:16.871107 2014-08-13 17:38:03.071139 7fe9bae887a0 20 read_log 194215'17271 (194215'17270) delete 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.2174595.0:174779 0.000000 2014-08-13 17:38:03.071150 7fe9bae887a0 20 read_log 194215'17272 (192888'16789) clean 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by osd.356.0:1179 2014-08-12 15:49:16.877830 2014-08-13 17:38:03.071174 7fe9bae887a0 20 read_log 194215'17273 (194215'17272) delete 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.2174595.0:174781 0.000000 2014-08-13 17:38:03.071206 7fe9bae887a0 20 read_log 194215'17274 (193305'17123) clean 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by osd.356.0:1181 2014-08-12 15:49:16.884360 2014-08-13 17:38:03.071220 7fe9bae887a0 20 read_log 194215'17275 (194215'17274) delete 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.2174595.0:174783 0.000000 2014-08-13 17:38:03.071231 7fe9bae887a0 20 read_log 194215'17276 (192888'16986) clean 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by osd.356.0:1183 2014-08-12 15:49:16.890480 2014-08-13 17:38:03.071243 7fe9bae887a0 20 read_log 194215'17277 (194215'17276) delete 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.2174595.0:174785 0.000000 2014-08-13 17:38:03.071254 7fe9bae887a0 20 read_log 194215'17278 (193305'17119) clean 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by osd.356.0:1185 2014-08-12 15:49:16.897175 2014-08-13 17:38:03.071266 7fe9bae887a0 20 read_log 194215'17279 (194215'17278) delete 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.2174595.0:174787 0.000000 2014-08-13 17:38:03.071276 7fe9bae887a0 20 read_log 194215'17280 (192888'16826) clean 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by osd.356.0:1187 2014-08-12 15:49:16.903365 2014-08-13 17:38:03.071288 7fe9bae887a0 20 read_log 194215'17281 (194215'17280) delete 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.2174595.0:174789 0.000000 2014-08-13 17:38:03.071305 7fe9bae887a0 20 read_log 194215'17282 (193305'17133) clean c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by osd.356.0:1189 2014-08-12 15:49:16.909796 2014-08-13 17:38:03.071318 7fe9bae887a0 20 read_log 194215'17283 (194215'17282) delete c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.2174595.0:174791 0.000000 2014-08-13 17:38:03.071329 7fe9bae887a0 20 read_log 194215'17284 (192888'16983) clean 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by osd.356.0:1191 2014-08-12 15:49:16.916672 2014-08-13 17:38:03.071341 7fe9bae887a0 20 read_log 194215'17285 (194215'17284) delete 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.2174595.0:174793 0.000000 2014-08-13 17:38:03.071352 7fe9bae887a0 20 read_log 194215'17286 (192888'17007) clean d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by osd.356.0:1193 2014-08-12 15:49:16.923542 2014-08-13 17:38:03.071364 7fe9bae887a0 20 read_log 194215'17287 (194215'17286) delete d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.2174595.0:174795 0.000000 2014-08-13 17:38:03.071374 7fe9bae887a0 20 read_log 194215'17288 (193305'17128) clean f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by osd.356.0:1195 2014-08-12 15:49:16.930574 2014-08-13 17:38:03.071386 7fe9bae887a0 20 read_log 194215'17289 (194215'17288) delete f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.2174595.0:174797 0.000000 2014-08-13 17:38:03.071397 7fe9bae887a0 20 read_log 194215'17290 (192888'17038) clean eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by osd.356.0:1197 2014-08-12 15:49:16.937618 2014-08-13 17:38:03.071409 7fe9bae887a0 20 read_log 194215'17291 (194215'17290) delete eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.2174595.0:174799 0.000000 2014-08-13 17:38:03.071419 7fe9bae887a0 20 read_log 194215'17292 (193305'17125) clean cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by osd.356.0:1199 2014-08-12 15:49:16.944471 2014-08-13 17:38:03.071431 7fe9bae887a0 20 read_log 194215'17293 (194215'17292) delete cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.2174595.0:174801 0.000000 2014-08-13 17:38:03.071442 7fe9bae887a0 20 read_log 194215'17294 (192888'17029) clean a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by osd.356.0:1201 2014-08-12 15:49:16.951230 2014-08-13 17:38:03.071454 7fe9bae887a0 20 read_log 194215'17295 (194215'17294) delete a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.2174595.0:174803 0.000000 2014-08-13 17:38:03.071464 7fe9bae887a0 20 read_log 194215'17296 (192888'17024) clean 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by osd.356.0:1203 2014-08-12 15:49:16.957915 2014-08-13 17:38:03.071476 7fe9bae887a0 20 read_log 194215'17297 (194215'17296) delete 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.2174595.0:174805 0.000000 2014-08-13 17:38:03.071487 7fe9bae887a0 20 read_log 194215'17298 (193305'17112) clean f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by osd.356.0:1205 2014-08-12 15:49:16.965083 2014-08-13 17:38:03.071499 7fe9bae887a0 20 read_log 194215'17299 (194215'17298) delete f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.2174595.0:174807 0.000000 2014-08-13 17:38:03.071509 7fe9bae887a0 20 read_log 194215'17300 (193305'17122) clean d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by osd.356.0:1207 2014-08-12 15:49:16.972195 2014-08-13 17:38:03.071521 7fe9bae887a0 20 read_log 194215'17301 (194215'17300) delete d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.2174595.0:174809 0.000000 2014-08-13 17:38:03.071541 7fe9bae887a0 20 read_log 194215'17302 (192888'16817) clean a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by osd.356.0:1209 2014-08-12 15:49:16.979265 2014-08-13 17:38:03.071554 7fe9bae887a0 20 read_log 194215'17303 (194215'17302) delete a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.2174595.0:174811 0.000000 2014-08-13 17:38:03.071565 7fe9bae887a0 20 read_log 194215'17304 (192888'16793) clean 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by osd.356.0:1211 2014-08-12 15:49:16.986441 2014-08-13 17:38:03.071577 7fe9bae887a0 20 read_log 194215'17305 (194215'17304) delete 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.2174595.0:174813 0.000000 2014-08-13 17:38:03.071588 7fe9bae887a0 20 read_log 194215'17306 (192888'17043) clean dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by osd.356.0:1213 2014-08-12 15:49:16.993442 2014-08-13 17:38:03.071600 7fe9bae887a0 20 read_log 194215'17307 (194215'17306) delete dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.2174595.0:174815 0.000000 2014-08-13 17:38:03.071610 7fe9bae887a0 20 read_log 194215'17308 (193305'17111) clean fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by osd.356.0:1215 2014-08-12 15:49:17.000131 2014-08-13 17:38:03.071623 7fe9bae887a0 20 read_log 194215'17309 (194215'17308) delete fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.2174595.0:174817 0.000000 2014-08-13 17:38:03.071633 7fe9bae887a0 20 read_log 194215'17310 (192888'17035) clean 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by osd.356.0:1217 2014-08-12 15:49:17.007034 2014-08-13 17:38:03.071645 7fe9bae887a0 20 read_log 194215'17311 (194215'17310) delete 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.2174595.0:174819 0.000000 2014-08-13 17:38:03.071656 7fe9bae887a0 20 read_log 194215'17312 (193305'17116) clean d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by osd.356.0:1219 2014-08-12 15:49:17.013750 2014-08-13 17:38:03.071668 7fe9bae887a0 20 read_log 194215'17313 (194215'17312) delete d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.2174595.0:174821 0.000000 2014-08-13 17:38:03.071697 7fe9bae887a0 20 read_log 194215'17314 (192888'16992) clean 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by osd.356.0:1221 2014-08-12 15:49:17.020804 2014-08-13 17:38:03.071710 7fe9bae887a0 20 read_log 194215'17315 (194215'17314) delete 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.2174595.0:174823 0.000000 2014-08-13 17:38:03.071720 7fe9bae887a0 20 read_log 194215'17316 (192888'16997) clean c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by osd.356.0:1223 2014-08-12 15:49:17.027592 2014-08-13 17:38:03.071733 7fe9bae887a0 20 read_log 194215'17317 (194215'17316) delete c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.2174595.0:174825 0.000000 2014-08-13 17:38:03.071743 7fe9bae887a0 20 read_log 194215'17318 (192888'17003) clean 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by osd.356.0:1225 2014-08-12 15:49:17.034180 2014-08-13 17:38:03.071756 7fe9bae887a0 20 read_log 194215'17319 (194215'17318) delete 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.2174595.0:174827 0.000000 2014-08-13 17:38:03.071766 7fe9bae887a0 20 read_log 194215'17320 (193305'17137) clean c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by osd.356.0:1227 2014-08-12 15:49:17.041118 2014-08-13 17:38:03.071779 7fe9bae887a0 20 read_log 194215'17321 (194215'17320) delete c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.2174595.0:174829 0.000000 2014-08-13 17:38:03.071797 7fe9bae887a0 20 read_log 194215'17322 (192888'16978) clean 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by osd.356.0:1229 2014-08-12 15:49:17.047858 2014-08-13 17:38:03.071810 7fe9bae887a0 20 read_log 194215'17323 (194215'17322) delete 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.2174595.0:174831 0.000000 2014-08-13 17:38:03.071821 7fe9bae887a0 20 read_log 194215'17324 (192888'16823) clean b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by osd.356.0:1231 2014-08-12 15:49:17.054922 2014-08-13 17:38:03.071833 7fe9bae887a0 20 read_log 194215'17325 (194215'17324) delete b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.2174595.0:174833 0.000000 2014-08-13 17:38:03.071843 7fe9bae887a0 20 read_log 194215'17326 (192888'16827) clean f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by osd.356.0:1233 2014-08-12 15:49:17.062074 2014-08-13 17:38:03.071855 7fe9bae887a0 20 read_log 194215'17327 (194215'17326) delete f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.2174595.0:174835 0.000000 2014-08-13 17:38:03.071866 7fe9bae887a0 20 read_log 194216'17328 (194216'17328) modify 350/hit_set_24.350_archive_2014-08-12 15:49:16.440554_2014-08-12 17:25:03.186720/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.186960 2014-08-13 17:38:03.071896 7fe9bae887a0 20 read_log 194216'17329 (194215'17138) delete 350/hit_set_24.350_archive_2014-08-12 15:32:25.978048_2014-08-12 15:49:16.440311/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.186960 2014-08-13 17:38:03.071908 7fe9bae887a0 20 read_log 194216'17330 (0'0) promote 8eaf2750/benchmark_data_storage0111-ib.pocstor_39693_object14/head//24 by osd.356.0:1772 2014-08-12 17:25:03.190251 2014-08-13 17:38:03.071921 7fe9bae887a0 20 read_log 194216'17331 (194216'17330) modify 8eaf2750/benchmark_data_storage0111-ib.pocstor_39693_object14/head//24 by client.1487927.0:15 2014-08-12 17:25:03.177457 2014-08-13 17:38:03.071938 7fe9bae887a0 20 read_log 194370'17332 (0'0) promote 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by osd.356.0:1779 2014-08-12 20:30:16.257189 2014-08-13 17:38:03.071952 7fe9bae887a0 20 read_log 194370'17333 (194370'17332) modify 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by client.1508531.0:490 2014-08-12 20:30:16.248882 2014-08-13 17:38:03.071965 7fe9bae887a0 20 read_log 194370'17334 (0'0) promote 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by osd.356.0:1787 2014-08-12 20:30:21.179789 2014-08-13 17:38:03.071977 7fe9bae887a0 20 read_log 194370'17335 (194370'17334) modify 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by client.1508531.0:1821 2014-08-12 20:30:21.172164 2014-08-13 17:38:03.071989 7fe9bae887a0 20 read_log 194370'17336 (0'0) promote a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by osd.356.0:1819 2014-08-12 20:30:45.312449 2014-08-13 17:38:03.072001 7fe9bae887a0 20 read_log 194370'17337 (194370'17336) modify a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by client.1508531.0:7578 2014-08-12 20:30:45.303588 2014-08-13 17:38:03.072013 7fe9bae887a0 20 read_log 194370'17338 (0'0) promote 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by osd.356.0:1821 2014-08-12 20:30:46.293447 2014-08-13 17:38:03.072025 7fe9bae887a0 20 read_log 194370'17339 (194370'17338) modify 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by client.1508531.0:7761 2014-08-12 20:30:46.285561 2014-08-13 17:38:03.072037 7fe9bae887a0 20 read_log 194370'17340 (0'0) promote b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by osd.356.0:1823 2014-08-12 20:30:47.384244 2014-08-13 17:38:03.072057 7fe9bae887a0 20 read_log 194370'17341 (194370'17340) modify b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by client.1508531.0:7959 2014-08-12 20:30:47.377375 2014-08-13 17:38:03.072069 7fe9bae887a0 20 read_log 194370'17342 (0'0) promote 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by osd.356.0:1829 2014-08-12 20:30:51.479978 2014-08-13 17:38:03.072081 7fe9bae887a0 20 read_log 194370'17343 (194370'17342) modify 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by client.1508531.0:8704 2014-08-12 20:30:51.472682 2014-08-13 17:38:03.072093 7fe9bae887a0 20 read_log 194370'17344 (0'0) promote b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by osd.356.0:1831 2014-08-12 20:30:52.055736 2014-08-13 17:38:03.072105 7fe9bae887a0 20 read_log 194370'17345 (194370'17344) modify b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by client.1508531.0:8801 2014-08-12 20:30:52.048105 2014-08-13 17:38:03.072117 7fe9bae887a0 20 read_log 194370'17346 (0'0) promote 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by osd.356.0:1863 2014-08-12 20:31:27.913261 2014-08-13 17:38:03.072129 7fe9bae887a0 20 read_log 194370'17347 (194370'17346) modify 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by client.1508531.0:13844 2014-08-12 20:31:27.906128 2014-08-13 17:38:03.072141 7fe9bae887a0 20 read_log 194370'17348 (0'0) promote 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by osd.356.0:1869 2014-08-12 20:31:30.741976 2014-08-13 17:38:03.072159 7fe9bae887a0 20 read_log 194370'17349 (194370'17348) modify 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by client.1508531.0:13946 2014-08-12 20:31:30.733784 2014-08-13 17:38:03.072173 7fe9bae887a0 20 read_log 194370'17350 (0'0) promote 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by osd.356.0:1871 2014-08-12 20:31:32.307468 2014-08-13 17:38:03.072185 7fe9bae887a0 20 read_log 194370'17351 (194370'17350) modify 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by client.1508531.0:14004 2014-08-12 20:31:32.299744 2014-08-13 17:38:03.072197 7fe9bae887a0 20 read_log 194377'17352 (0'0) promote 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by osd.356.0:1873 2014-08-12 20:32:58.005669 2014-08-13 17:38:03.072209 7fe9bae887a0 20 read_log 194377'17353 (194377'17352) modify 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by client.2231190.0:202 2014-08-12 20:32:57.997217 2014-08-13 17:38:03.072221 7fe9bae887a0 20 read_log 194377'17354 (0'0) promote e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by osd.356.0:1885 2014-08-12 20:33:02.781514 2014-08-13 17:38:03.072233 7fe9bae887a0 20 read_log 194377'17355 (194377'17354) modify e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by client.2231190.0:1389 2014-08-12 20:33:02.774558 2014-08-13 17:38:03.072245 7fe9bae887a0 20 read_log 194377'17356 (0'0) promote 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by osd.356.0:1889 2014-08-12 20:33:03.709956 2014-08-13 17:38:03.072257 7fe9bae887a0 20 read_log 194377'17357 (194377'17356) modify 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by client.2231190.0:1638 2014-08-12 20:33:03.702259 2014-08-13 17:38:03.072269 7fe9bae887a0 20 read_log 194377'17358 (0'0) promote 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by osd.356.0:1903 2014-08-12 20:33:12.347777 2014-08-13 17:38:03.072281 7fe9bae887a0 20 read_log 194377'17359 (194377'17358) modify 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by client.2231190.0:3703 2014-08-12 20:33:12.341623 2014-08-13 17:38:03.072293 7fe9bae887a0 20 read_log 194377'17360 (0'0) promote 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by osd.356.0:1907 2014-08-12 20:33:14.272285 2014-08-13 17:38:03.072315 7fe9bae887a0 20 read_log 194377'17361 (194377'17360) modify 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by client.2231190.0:4060 2014-08-12 20:33:14.263601 2014-08-13 17:38:03.072328 7fe9bae887a0 20 read_log 194377'17362 (0'0) promote 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by osd.356.0:1909 2014-08-12 20:33:15.606095 2014-08-13 17:38:03.072340 7fe9bae887a0 20 read_log 194377'17363 (194377'17362) modify 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by client.2231190.0:4283 2014-08-12 20:33:15.597519 2014-08-13 17:38:03.072353 7fe9bae887a0 20 read_log 194379'17364 (0'0) promote 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by osd.356.0:1915 2014-08-12 20:33:20.693640 2014-08-13 17:38:03.072365 7fe9bae887a0 20 read_log 194379'17365 (194379'17364) modify 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by client.2231190.0:4878 2014-08-12 20:33:20.668730 2014-08-13 17:38:03.072377 7fe9bae887a0 20 read_log 194416'17366 (0'0) promote ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by osd.356.0:2097 2014-08-13 15:28:41.588026 2014-08-13 17:38:03.072389 7fe9bae887a0 20 read_log 194416'17367 (194416'17366) modify ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by client.2441958.0:2869 2014-08-13 15:28:41.580397 2014-08-13 17:38:03.072401 7fe9bae887a0 20 read_log 194416'17368 (0'0) promote c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by osd.356.0:2107 2014-08-13 15:28:44.431377 2014-08-13 17:38:03.072413 7fe9bae887a0 20 read_log 194416'17369 (194416'17368) modify c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by client.2441958.0:3617 2014-08-13 15:28:44.423650 2014-08-13 17:38:03.072425 7fe9bae887a0 20 read_log 194416'17370 (0'0) promote 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by osd.356.0:2109 2014-08-13 15:28:45.821212 2014-08-13 17:38:03.072437 7fe9bae887a0 20 read_log 194416'17371 (194416'17370) modify 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by client.2441958.0:3971 2014-08-13 15:28:45.813008 2014-08-13 17:38:03.072468 7fe9bae887a0 20 read_log 194416'17372 (0'0) promote ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by osd.356.0:2115 2014-08-13 15:28:49.749259 2014-08-13 17:38:03.072480 7fe9bae887a0 20 read_log 194416'17373 (194416'17372) modify ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by client.2441958.0:4962 2014-08-13 15:28:49.741277 2014-08-13 17:38:03.072492 7fe9bae887a0 20 read_log 194416'17374 (0'0) promote 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by osd.356.0:2119 2014-08-13 15:28:50.287985 2014-08-13 17:38:03.072504 7fe9bae887a0 20 read_log 194416'17375 (194416'17374) modify 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by client.2441958.0:5104 2014-08-13 15:28:50.281361 2014-08-13 17:38:03.072516 7fe9bae887a0 20 read_log 194416'17376 (0'0) promote f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by osd.356.0:2121 2014-08-13 15:28:50.649857 2014-08-13 17:38:03.072528 7fe9bae887a0 20 read_log 194416'17377 (194416'17376) modify f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by client.2441958.0:5198 2014-08-13 15:28:50.642625 2014-08-13 17:38:03.072540 7fe9bae887a0 20 read_log 194416'17378 (0'0) promote 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by osd.356.0:2139 2014-08-13 15:28:56.509710 2014-08-13 17:38:03.072552 7fe9bae887a0 20 read_log 194416'17379 (194416'17378) modify 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by client.2441958.0:6659 2014-08-13 15:28:56.502292 2014-08-13 17:38:03.072591 7fe9bae887a0 20 read_log 194416'17380 (0'0) promote d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by osd.356.0:2147 2014-08-13 15:29:00.199723 2014-08-13 17:38:03.072605 7fe9bae887a0 20 read_log 194416'17381 (194416'17380) modify d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by client.2441958.0:7646 2014-08-13 15:29:00.192240 2014-08-13 17:38:03.072617 7fe9bae887a0 20 read_log 194416'17382 (0'0) promote a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by osd.356.0:2163 2014-08-13 15:29:05.338299 2014-08-13 17:38:03.072629 7fe9bae887a0 20 read_log 194416'17383 (194416'17382) modify a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by client.2441958.0:8966 2014-08-13 15:29:05.330008 2014-08-13 17:38:03.072641 7fe9bae887a0 20 read_log 194416'17384 (0'0) promote 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by osd.356.0:2179 2014-08-13 15:29:11.139605 2014-08-13 17:38:03.072653 7fe9bae887a0 20 read_log 194416'17385 (194416'17384) modify 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by client.2441958.0:10420 2014-08-13 15:29:11.132288 2014-08-13 17:38:03.072668 7fe9bae887a0 20 read_log 194416'17386 (0'0) promote dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by osd.356.0:2187 2014-08-13 15:29:15.129261 2014-08-13 17:38:03.072680 7fe9bae887a0 20 read_log 194416'17387 (194416'17386) modify dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by client.2441958.0:11410 2014-08-13 15:29:15.119476 2014-08-13 17:38:03.072692 7fe9bae887a0 20 read_log 194416'17388 (0'0) promote 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by osd.356.0:2191 2014-08-13 15:29:19.812140 2014-08-13 17:38:03.072703 7fe9bae887a0 20 read_log 194416'17389 (194416'17388) modify 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by client.2441958.0:12530 2014-08-13 15:29:19.803948 2014-08-13 17:38:03.072715 7fe9bae887a0 20 read_log 194416'17390 (0'0) promote 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by osd.356.0:2195 2014-08-13 15:29:20.587732 2014-08-13 17:38:03.072727 7fe9bae887a0 20 read_log 194416'17391 (194416'17390) modify 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by client.2441958.0:12708 2014-08-13 15:29:20.579787 2014-08-13 17:38:03.072739 7fe9bae887a0 20 read_log 194416'17392 (0'0) promote dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by osd.356.0:2199 2014-08-13 15:29:21.236488 2014-08-13 17:38:03.072751 7fe9bae887a0 20 read_log 194416'17393 (194416'17392) modify dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by client.2441958.0:12847 2014-08-13 15:29:21.228530 2014-08-13 17:38:03.072763 7fe9bae887a0 20 read_log 194416'17394 (0'0) promote ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by osd.356.0:2203 2014-08-13 15:29:22.935524 2014-08-13 17:38:03.072775 7fe9bae887a0 20 read_log 194416'17395 (194416'17394) modify ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by client.2441958.0:13230 2014-08-13 15:29:22.928752 2014-08-13 17:38:03.072787 7fe9bae887a0 20 read_log 194416'17396 (0'0) promote 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by osd.356.0:2207 2014-08-13 15:29:23.938036 2014-08-13 17:38:03.072799 7fe9bae887a0 20 read_log 194416'17397 (194416'17396) modify 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by client.2441958.0:13466 2014-08-13 15:29:23.930792 2014-08-13 17:38:03.072811 7fe9bae887a0 20 read_log 194416'17398 (0'0) promote 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by osd.356.0:2209 2014-08-13 15:29:24.008495 2014-08-13 17:38:03.072823 7fe9bae887a0 20 read_log 194416'17399 (194416'17398) modify 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by client.2441958.0:13481 2014-08-13 15:29:24.001910 2014-08-13 17:38:03.072842 7fe9bae887a0 20 read_log 194416'17400 (0'0) promote 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by osd.356.0:2213 2014-08-13 15:29:26.682067 2014-08-13 17:38:03.072855 7fe9bae887a0 20 read_log 194416'17401 (194416'17400) modify 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by client.2441958.0:14123 2014-08-13 15:29:26.670759 2014-08-13 17:38:03.072867 7fe9bae887a0 20 read_log 194416'17402 (0'0) promote aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by osd.356.0:2225 2014-08-13 15:29:48.419803 2014-08-13 17:38:03.072878 7fe9bae887a0 20 read_log 194416'17403 (194416'17402) modify aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by client.2441958.0:17075 2014-08-13 15:29:48.410568 2014-08-13 17:38:03.072890 7fe9bae887a0 20 read_log 194416'17404 (0'0) promote 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by osd.356.0:2229 2014-08-13 15:29:50.066265 2014-08-13 17:38:03.072902 7fe9bae887a0 20 read_log 194416'17405 (194416'17404) modify 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by client.2441958.0:17192 2014-08-13 15:29:50.058856 2014-08-13 17:38:03.072914 7fe9bae887a0 20 read_log 194416'17406 (0'0) promote ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by osd.356.0:2237 2014-08-13 15:30:00.290538 2014-08-13 17:38:03.072942 7fe9bae887a0 20 read_log 194416'17407 (194416'17406) modify ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by client.2441958.0:17916 2014-08-13 15:30:00.280228 2014-08-13 17:38:03.072955 7fe9bae887a0 20 read_log 194416'17408 (0'0) promote 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by osd.356.0:2239 2014-08-13 15:30:16.335107 2014-08-13 17:38:03.072967 7fe9bae887a0 20 read_log 194416'17409 (194416'17408) modify 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by client.2441958.0:18617 2014-08-13 15:30:16.323541 2014-08-13 17:38:03.072979 7fe9bae887a0 20 read_log 194422'17410 (0'0) promote 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by osd.356.0:2257 2014-08-13 15:32:26.787872 2014-08-13 17:38:03.072990 7fe9bae887a0 20 read_log 194422'17411 (194422'17410) modify 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by client.1148203.0:2916 2014-08-13 15:32:26.777366 2014-08-13 17:38:03.073002 7fe9bae887a0 20 read_log 194422'17412 (0'0) promote 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by osd.356.0:2265 2014-08-13 15:32:27.995798 2014-08-13 17:38:03.073014 7fe9bae887a0 20 read_log 194422'17413 (194422'17412) modify 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by client.1148203.0:3236 2014-08-13 15:32:27.984825 2014-08-13 17:38:03.073026 7fe9bae887a0 20 read_log 194422'17414 (0'0) promote e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by osd.356.0:2267 2014-08-13 15:32:30.898203 2014-08-13 17:38:03.073038 7fe9bae887a0 20 read_log 194422'17415 (194422'17414) modify e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by client.1148203.0:3975 2014-08-13 15:32:30.888992 2014-08-13 17:38:03.073050 7fe9bae887a0 20 read_log 194422'17416 (0'0) promote 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by osd.356.0:2269 2014-08-13 15:32:32.797439 2014-08-13 17:38:03.073062 7fe9bae887a0 20 read_log 194422'17417 (194422'17416) modify 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by client.1148203.0:4468 2014-08-13 15:32:32.789466 2014-08-13 17:38:03.073074 7fe9bae887a0 20 read_log 194422'17418 (0'0) promote 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by osd.356.0:2275 2014-08-13 15:32:35.994587 2014-08-13 17:38:03.073085 7fe9bae887a0 20 read_log 194422'17419 (194422'17418) modify 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by client.1148203.0:5228 2014-08-13 15:32:35.986693 2014-08-13 17:38:03.073108 7fe9bae887a0 20 read_log 194422'17420 (0'0) promote d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by osd.356.0:2277 2014-08-13 15:32:37.054422 2014-08-13 17:38:03.073121 7fe9bae887a0 20 read_log 194422'17421 (194422'17420) modify d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by client.1148203.0:5444 2014-08-13 15:32:37.046145 2014-08-13 17:38:03.073133 7fe9bae887a0 20 read_log 194422'17422 (0'0) promote cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by osd.356.0:2281 2014-08-13 15:32:37.712762 2014-08-13 17:38:03.073145 7fe9bae887a0 20 read_log 194422'17423 (194422'17422) modify cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by client.1148203.0:5556 2014-08-13 15:32:37.704474 2014-08-13 17:38:03.073162 7fe9bae887a0 20 read_log 194422'17424 (0'0) promote 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by osd.356.0:2291 2014-08-13 15:32:44.209205 2014-08-13 17:38:03.073175 7fe9bae887a0 20 read_log 194422'17425 (194422'17424) modify 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by client.1148203.0:6757 2014-08-13 15:32:44.201499 2014-08-13 17:38:03.073187 7fe9bae887a0 20 read_log 194422'17426 (0'0) promote eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by osd.356.0:2293 2014-08-13 15:32:44.973381 2014-08-13 17:38:03.073199 7fe9bae887a0 20 read_log 194422'17427 (194422'17426) modify eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by client.1148203.0:6911 2014-08-13 15:32:44.966068 2014-08-13 17:38:03.073212 7fe9bae887a0 20 read_log 194422'17428 (0'0) promote 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by osd.356.0:2299 2014-08-13 15:32:46.578731 2014-08-13 17:38:03.073223 7fe9bae887a0 20 read_log 194422'17429 (194422'17428) modify 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by client.1148203.0:7169 2014-08-13 15:32:46.567134 2014-08-13 17:38:03.073235 7fe9bae887a0 20 read_log 194422'17430 (0'0) promote 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by osd.356.0:2307 2014-08-13 15:32:54.300993 2014-08-13 17:38:03.073247 7fe9bae887a0 20 read_log 194422'17431 (194422'17430) modify 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by client.1148203.0:8394 2014-08-13 15:32:54.290972 2014-08-13 17:38:03.073259 7fe9bae887a0 20 read_log 194422'17432 (0'0) promote ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by osd.356.0:2309 2014-08-13 15:32:57.088447 2014-08-13 17:38:03.073271 7fe9bae887a0 20 read_log 194422'17433 (194422'17432) modify ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by client.1148203.0:8745 2014-08-13 15:32:57.079855 2014-08-13 17:38:03.073286 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.073645 7fe9bae887a0 10 read_log checking for missing items over interval (194370'17332,194422'17433] 2014-08-13 17:38:03.073691 7fe9bae887a0 15 read_log missing 194422'17433 (194422'17432) modify ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by client.1148203.0:8745 2014-08-13 15:32:57.079855 2014-08-13 17:38:03.073733 7fe9bae887a0 15 read_log missing 194422'17431 (194422'17430) modify 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by client.1148203.0:8394 2014-08-13 15:32:54.290972 2014-08-13 17:38:03.073770 7fe9bae887a0 15 read_log missing 194422'17429 (194422'17428) modify 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by client.1148203.0:7169 2014-08-13 15:32:46.567134 2014-08-13 17:38:03.073807 7fe9bae887a0 15 read_log missing 194422'17427 (194422'17426) modify eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by client.1148203.0:6911 2014-08-13 15:32:44.966068 2014-08-13 17:38:03.073844 7fe9bae887a0 15 read_log missing 194422'17425 (194422'17424) modify 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by client.1148203.0:6757 2014-08-13 15:32:44.201499 2014-08-13 17:38:03.073880 7fe9bae887a0 15 read_log missing 194422'17423 (194422'17422) modify cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by client.1148203.0:5556 2014-08-13 15:32:37.704474 2014-08-13 17:38:03.073917 7fe9bae887a0 15 read_log missing 194422'17421 (194422'17420) modify d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by client.1148203.0:5444 2014-08-13 15:32:37.046145 2014-08-13 17:38:03.073954 7fe9bae887a0 15 read_log missing 194422'17419 (194422'17418) modify 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by client.1148203.0:5228 2014-08-13 15:32:35.986693 2014-08-13 17:38:03.073991 7fe9bae887a0 15 read_log missing 194422'17417 (194422'17416) modify 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by client.1148203.0:4468 2014-08-13 15:32:32.789466 2014-08-13 17:38:03.074027 7fe9bae887a0 15 read_log missing 194422'17415 (194422'17414) modify e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by client.1148203.0:3975 2014-08-13 15:32:30.888992 2014-08-13 17:38:03.074064 7fe9bae887a0 15 read_log missing 194422'17413 (194422'17412) modify 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by client.1148203.0:3236 2014-08-13 15:32:27.984825 2014-08-13 17:38:03.074100 7fe9bae887a0 15 read_log missing 194422'17411 (194422'17410) modify 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by client.1148203.0:2916 2014-08-13 15:32:26.777366 2014-08-13 17:38:03.074137 7fe9bae887a0 15 read_log missing 194416'17409 (194416'17408) modify 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by client.2441958.0:18617 2014-08-13 15:30:16.323541 2014-08-13 17:38:03.074180 7fe9bae887a0 15 read_log missing 194416'17407 (194416'17406) modify ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by client.2441958.0:17916 2014-08-13 15:30:00.280228 2014-08-13 17:38:03.074218 7fe9bae887a0 15 read_log missing 194416'17405 (194416'17404) modify 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by client.2441958.0:17192 2014-08-13 15:29:50.058856 2014-08-13 17:38:03.074255 7fe9bae887a0 15 read_log missing 194416'17403 (194416'17402) modify aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by client.2441958.0:17075 2014-08-13 15:29:48.410568 2014-08-13 17:38:03.074292 7fe9bae887a0 15 read_log missing 194416'17401 (194416'17400) modify 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by client.2441958.0:14123 2014-08-13 15:29:26.670759 2014-08-13 17:38:03.074328 7fe9bae887a0 15 read_log missing 194416'17399 (194416'17398) modify 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by client.2441958.0:13481 2014-08-13 15:29:24.001910 2014-08-13 17:38:03.074365 7fe9bae887a0 15 read_log missing 194416'17397 (194416'17396) modify 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by client.2441958.0:13466 2014-08-13 15:29:23.930792 2014-08-13 17:38:03.074402 7fe9bae887a0 15 read_log missing 194416'17395 (194416'17394) modify ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by client.2441958.0:13230 2014-08-13 15:29:22.928752 2014-08-13 17:38:03.074439 7fe9bae887a0 15 read_log missing 194416'17393 (194416'17392) modify dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by client.2441958.0:12847 2014-08-13 15:29:21.228530 2014-08-13 17:38:03.074476 7fe9bae887a0 15 read_log missing 194416'17391 (194416'17390) modify 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by client.2441958.0:12708 2014-08-13 15:29:20.579787 2014-08-13 17:38:03.074512 7fe9bae887a0 15 read_log missing 194416'17389 (194416'17388) modify 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by client.2441958.0:12530 2014-08-13 15:29:19.803948 2014-08-13 17:38:03.074548 7fe9bae887a0 15 read_log missing 194416'17387 (194416'17386) modify dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by client.2441958.0:11410 2014-08-13 15:29:15.119476 2014-08-13 17:38:03.074585 7fe9bae887a0 15 read_log missing 194416'17385 (194416'17384) modify 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by client.2441958.0:10420 2014-08-13 15:29:11.132288 2014-08-13 17:38:03.074621 7fe9bae887a0 15 read_log missing 194416'17383 (194416'17382) modify a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by client.2441958.0:8966 2014-08-13 15:29:05.330008 2014-08-13 17:38:03.074659 7fe9bae887a0 15 read_log missing 194416'17381 (194416'17380) modify d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by client.2441958.0:7646 2014-08-13 15:29:00.192240 2014-08-13 17:38:03.074696 7fe9bae887a0 15 read_log missing 194416'17379 (194416'17378) modify 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by client.2441958.0:6659 2014-08-13 15:28:56.502292 2014-08-13 17:38:03.074732 7fe9bae887a0 15 read_log missing 194416'17377 (194416'17376) modify f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by client.2441958.0:5198 2014-08-13 15:28:50.642625 2014-08-13 17:38:03.074769 7fe9bae887a0 15 read_log missing 194416'17375 (194416'17374) modify 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by client.2441958.0:5104 2014-08-13 15:28:50.281361 2014-08-13 17:38:03.074806 7fe9bae887a0 15 read_log missing 194416'17373 (194416'17372) modify ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by client.2441958.0:4962 2014-08-13 15:28:49.741277 2014-08-13 17:38:03.074843 7fe9bae887a0 15 read_log missing 194416'17371 (194416'17370) modify 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by client.2441958.0:3971 2014-08-13 15:28:45.813008 2014-08-13 17:38:03.074879 7fe9bae887a0 15 read_log missing 194416'17369 (194416'17368) modify c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by client.2441958.0:3617 2014-08-13 15:28:44.423650 2014-08-13 17:38:03.074916 7fe9bae887a0 15 read_log missing 194416'17367 (194416'17366) modify ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by client.2441958.0:2869 2014-08-13 15:28:41.580397 2014-08-13 17:38:03.074952 7fe9bae887a0 15 read_log missing 194379'17365 (194379'17364) modify 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by client.2231190.0:4878 2014-08-12 20:33:20.668730 2014-08-13 17:38:03.074989 7fe9bae887a0 15 read_log missing 194377'17363 (194377'17362) modify 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by client.2231190.0:4283 2014-08-12 20:33:15.597519 2014-08-13 17:38:03.075026 7fe9bae887a0 15 read_log missing 194377'17361 (194377'17360) modify 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by client.2231190.0:4060 2014-08-12 20:33:14.263601 2014-08-13 17:38:03.075062 7fe9bae887a0 15 read_log missing 194377'17359 (194377'17358) modify 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by client.2231190.0:3703 2014-08-12 20:33:12.341623 2014-08-13 17:38:03.075099 7fe9bae887a0 15 read_log missing 194377'17357 (194377'17356) modify 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by client.2231190.0:1638 2014-08-12 20:33:03.702259 2014-08-13 17:38:03.075136 7fe9bae887a0 15 read_log missing 194377'17355 (194377'17354) modify e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by client.2231190.0:1389 2014-08-12 20:33:02.774558 2014-08-13 17:38:03.075178 7fe9bae887a0 15 read_log missing 194377'17353 (194377'17352) modify 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by client.2231190.0:202 2014-08-12 20:32:57.997217 2014-08-13 17:38:03.075216 7fe9bae887a0 15 read_log missing 194370'17351 (194370'17350) modify 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by client.1508531.0:14004 2014-08-12 20:31:32.299744 2014-08-13 17:38:03.075253 7fe9bae887a0 15 read_log missing 194370'17349 (194370'17348) modify 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by client.1508531.0:13946 2014-08-12 20:31:30.733784 2014-08-13 17:38:03.075290 7fe9bae887a0 15 read_log missing 194370'17347 (194370'17346) modify 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by client.1508531.0:13844 2014-08-12 20:31:27.906128 2014-08-13 17:38:03.075327 7fe9bae887a0 15 read_log missing 194370'17345 (194370'17344) modify b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by client.1508531.0:8801 2014-08-12 20:30:52.048105 2014-08-13 17:38:03.075363 7fe9bae887a0 15 read_log missing 194370'17343 (194370'17342) modify 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by client.1508531.0:8704 2014-08-12 20:30:51.472682 2014-08-13 17:38:03.075420 7fe9bae887a0 15 read_log missing 194370'17341 (194370'17340) modify b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by client.1508531.0:7959 2014-08-12 20:30:47.377375 2014-08-13 17:38:03.075459 7fe9bae887a0 15 read_log missing 194370'17339 (194370'17338) modify 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by client.1508531.0:7761 2014-08-12 20:30:46.285561 2014-08-13 17:38:03.075497 7fe9bae887a0 15 read_log missing 194370'17337 (194370'17336) modify a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by client.1508531.0:7578 2014-08-12 20:30:45.303588 2014-08-13 17:38:03.075533 7fe9bae887a0 15 read_log missing 194370'17335 (194370'17334) modify 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by client.1508531.0:1821 2014-08-12 20:30:21.172164 2014-08-13 17:38:03.075570 7fe9bae887a0 15 read_log missing 194370'17333 (194370'17332) modify 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by client.1508531.0:490 2014-08-12 20:30:16.248882 2014-08-13 17:38:03.075581 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.075605 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=0 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive m=51] handle_loaded 2014-08-13 17:38:03.075622 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=0 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Initial 0.012318 0 0.000000 2014-08-13 17:38:03.075634 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=0 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset 2014-08-13 17:38:03.075643 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=0 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194477 2014-08-13 17:38:03.075652 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log((192834'15725,194422'17433], crt=194422'17433) 2014-08-13 17:38:03.075662 7fe9bae887a0 10 osd.4 194478 pgid 24.369 coll 24.369_head 2014-08-13 17:38:03.075784 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.369 2014-08-13 17:38:03.075797 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.075830 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.369(unlocked)] enter Initial 2014-08-13 17:38:03.075841 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.369(unlocked)] enter NotTrimming 2014-08-13 17:38:03.075965 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.076079 7fe9bae887a0 20 read_log 192834'11429 (192834'11428) modify 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.933448.0:2663 2014-08-11 15:23:47.808008 2014-08-13 17:38:03.076100 7fe9bae887a0 20 read_log 192834'11442 (0'0) promote 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.338.0:12956 2014-08-11 15:23:57.605666 2014-08-13 17:38:03.076113 7fe9bae887a0 20 read_log 192834'11443 (192834'11442) modify 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by client.933448.0:4560 2014-08-11 15:23:57.594074 2014-08-13 17:38:03.076126 7fe9bae887a0 20 read_log 192834'11450 (0'0) promote 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:12964 2014-08-11 15:23:59.286759 2014-08-13 17:38:03.076138 7fe9bae887a0 20 read_log 192834'11451 (192834'11450) modify 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by client.933448.0:4905 2014-08-11 15:23:59.277462 2014-08-13 17:38:03.076151 7fe9bae887a0 20 read_log 192834'11470 (0'0) promote 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by osd.338.0:12984 2014-08-11 15:24:03.061694 2014-08-13 17:38:03.076173 7fe9bae887a0 20 read_log 192834'11471 (192834'11470) modify 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.933448.0:5834 2014-08-11 15:24:03.051794 2014-08-13 17:38:03.076186 7fe9bae887a0 20 read_log 192834'11498 (0'0) promote 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by osd.338.0:13012 2014-08-11 15:24:07.808801 2014-08-13 17:38:03.076198 7fe9bae887a0 20 read_log 192834'11499 (192834'11498) modify 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.933448.0:7029 2014-08-11 15:24:07.800422 2014-08-13 17:38:03.076210 7fe9bae887a0 20 read_log 192834'11508 (0'0) promote 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by osd.338.0:13022 2014-08-11 15:24:10.567263 2014-08-13 17:38:03.076223 7fe9bae887a0 20 read_log 192834'11509 (192834'11508) modify 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.933448.0:7566 2014-08-11 15:24:10.558757 2014-08-13 17:38:03.076235 7fe9bae887a0 20 read_log 192834'11516 (0'0) promote e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13030 2014-08-11 15:24:12.516818 2014-08-13 17:38:03.076247 7fe9bae887a0 20 read_log 192834'11517 (192834'11516) modify e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by client.933448.0:8052 2014-08-11 15:24:12.508898 2014-08-13 17:38:03.076260 7fe9bae887a0 20 read_log 192834'11578 (0'0) promote 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:13092 2014-08-11 15:24:25.398774 2014-08-13 17:38:03.076272 7fe9bae887a0 20 read_log 192834'11579 (192834'11578) modify 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by client.933448.0:10829 2014-08-11 15:24:25.389836 2014-08-13 17:38:03.076284 7fe9bae887a0 20 read_log 192834'11580 (0'0) promote 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:13094 2014-08-11 15:24:25.877238 2014-08-13 17:38:03.076296 7fe9bae887a0 20 read_log 192834'11581 (192834'11580) modify 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by client.933448.0:10953 2014-08-11 15:24:25.868009 2014-08-13 17:38:03.076308 7fe9bae887a0 20 read_log 192834'11584 (0'0) promote a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13098 2014-08-11 15:24:26.974271 2014-08-13 17:38:03.076330 7fe9bae887a0 20 read_log 192834'11585 (192834'11584) modify a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by client.933448.0:11241 2014-08-11 15:24:26.965943 2014-08-13 17:38:03.076345 7fe9bae887a0 20 read_log 192834'11590 (0'0) promote 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13104 2014-08-11 15:24:31.241056 2014-08-13 17:38:03.076357 7fe9bae887a0 20 read_log 192834'11591 (192834'11590) modify 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by client.933448.0:12140 2014-08-11 15:24:31.232396 2014-08-13 17:38:03.076370 7fe9bae887a0 20 read_log 192834'11604 (0'0) promote 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by osd.338.0:13118 2014-08-11 15:24:39.601167 2014-08-13 17:38:03.076382 7fe9bae887a0 20 read_log 192834'11605 (192834'11604) modify 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.933448.0:13689 2014-08-11 15:24:39.594556 2014-08-13 17:38:03.076394 7fe9bae887a0 20 read_log 192834'11624 (0'0) promote 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13138 2014-08-11 15:24:47.654738 2014-08-13 17:38:03.076406 7fe9bae887a0 20 read_log 192834'11625 (192834'11624) modify 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by client.933448.0:14907 2014-08-11 15:24:47.646644 2014-08-13 17:38:03.076418 7fe9bae887a0 20 read_log 192834'11638 (0'0) promote 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by osd.338.0:13152 2014-08-11 15:24:49.377263 2014-08-13 17:38:03.076430 7fe9bae887a0 20 read_log 192834'11639 (192834'11638) modify 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.933448.0:15363 2014-08-11 15:24:49.369581 2014-08-13 17:38:03.076443 7fe9bae887a0 20 read_log 192834'11672 (0'0) promote f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13186 2014-08-11 15:24:58.657771 2014-08-13 17:38:03.076455 7fe9bae887a0 20 read_log 192834'11673 (192834'11672) modify f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by client.933448.0:17157 2014-08-11 15:24:58.649479 2014-08-13 17:38:03.076467 7fe9bae887a0 20 read_log 192834'11684 (0'0) promote 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13198 2014-08-11 15:25:05.515490 2014-08-13 17:38:03.076503 7fe9bae887a0 20 read_log 192834'11685 (192834'11684) modify 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by client.933448.0:18491 2014-08-11 15:25:05.506978 2014-08-13 17:38:03.076516 7fe9bae887a0 20 read_log 192834'11692 (0'0) promote 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by osd.338.0:13206 2014-08-11 15:25:07.965562 2014-08-13 17:38:03.076528 7fe9bae887a0 20 read_log 192834'11693 (192834'11692) modify 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.933448.0:18992 2014-08-11 15:25:07.957170 2014-08-13 17:38:03.076540 7fe9bae887a0 20 read_log 192834'11694 (0'0) promote 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by osd.338.0:13208 2014-08-11 15:25:08.697584 2014-08-13 17:38:03.076552 7fe9bae887a0 20 read_log 192834'11695 (192834'11694) modify 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.933448.0:19120 2014-08-11 15:25:08.688746 2014-08-13 17:38:03.076564 7fe9bae887a0 20 read_log 192834'11698 (0'0) promote bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:13212 2014-08-11 15:25:11.378619 2014-08-13 17:38:03.076576 7fe9bae887a0 20 read_log 192834'11699 (192834'11698) modify bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by client.933448.0:19298 2014-08-11 15:25:11.370658 2014-08-13 17:38:03.076589 7fe9bae887a0 20 read_log 192834'11704 (0'0) promote f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by osd.338.0:13218 2014-08-11 15:25:12.090591 2014-08-13 17:38:03.076609 7fe9bae887a0 20 read_log 192834'11705 (192834'11704) modify f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.933448.0:19449 2014-08-11 15:25:12.081508 2014-08-13 17:38:03.076624 7fe9bae887a0 20 read_log 192834'11720 (0'0) promote 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by osd.338.0:13234 2014-08-11 15:25:22.875983 2014-08-13 17:38:03.076637 7fe9bae887a0 20 read_log 192834'11721 (192834'11720) modify 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.933448.0:20678 2014-08-11 15:25:22.866990 2014-08-13 17:38:03.076649 7fe9bae887a0 20 read_log 192834'11749 (192834'11429) modify 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.933448.0:24954 2014-08-11 15:25:41.571332 2014-08-13 17:38:03.076662 7fe9bae887a0 20 read_log 192834'11756 (192834'11443) modify 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by client.933448.0:26851 2014-08-11 15:25:43.816649 2014-08-13 17:38:03.076674 7fe9bae887a0 20 read_log 192834'11760 (192834'11451) modify 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by client.933448.0:27196 2014-08-11 15:25:44.207179 2014-08-13 17:38:03.076686 7fe9bae887a0 20 read_log 192834'11770 (192834'11471) modify 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.933448.0:28125 2014-08-11 15:25:45.530060 2014-08-13 17:38:03.076700 7fe9bae887a0 20 read_log 192834'11784 (192834'11499) modify 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.933448.0:29320 2014-08-11 15:25:47.216754 2014-08-13 17:38:03.076712 7fe9bae887a0 20 read_log 192834'11789 (192834'11509) modify 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.933448.0:29857 2014-08-11 15:25:48.014366 2014-08-13 17:38:03.076725 7fe9bae887a0 20 read_log 192834'11793 (192834'11517) modify e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by client.933448.0:30343 2014-08-11 15:25:48.760690 2014-08-13 17:38:03.076737 7fe9bae887a0 20 read_log 192834'11824 (192834'11579) modify 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by client.933448.0:33120 2014-08-11 15:25:52.867364 2014-08-13 17:38:03.076749 7fe9bae887a0 20 read_log 192834'11825 (192834'11581) modify 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by client.933448.0:33244 2014-08-11 15:25:53.112138 2014-08-13 17:38:03.076761 7fe9bae887a0 20 read_log 192834'11827 (192834'11585) modify a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by client.933448.0:33532 2014-08-11 15:25:53.640855 2014-08-13 17:38:03.076773 7fe9bae887a0 20 read_log 192834'11830 (192834'11591) modify 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by client.933448.0:34431 2014-08-11 15:25:55.004758 2014-08-13 17:38:03.076786 7fe9bae887a0 20 read_log 192834'11837 (192834'11605) modify 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.933448.0:35980 2014-08-11 15:25:57.099456 2014-08-13 17:38:03.076799 7fe9bae887a0 20 read_log 192834'11847 (192834'11625) modify 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by client.933448.0:37198 2014-08-11 15:25:59.107351 2014-08-13 17:38:03.076811 7fe9bae887a0 20 read_log 192834'11854 (192834'11639) modify 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.933448.0:37654 2014-08-11 15:25:59.811219 2014-08-13 17:38:03.076823 7fe9bae887a0 20 read_log 192834'11871 (192834'11673) modify f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by client.933448.0:39448 2014-08-11 15:26:01.882970 2014-08-13 17:38:03.076835 7fe9bae887a0 20 read_log 192834'11877 (192834'11685) modify 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by client.933448.0:40782 2014-08-11 15:26:02.729495 2014-08-13 17:38:03.076847 7fe9bae887a0 20 read_log 192834'11881 (192834'11693) modify 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.933448.0:41283 2014-08-11 15:26:02.896172 2014-08-13 17:38:03.076870 7fe9bae887a0 20 read_log 192834'11882 (192834'11695) modify 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.933448.0:41411 2014-08-11 15:26:02.935495 2014-08-13 17:38:03.076886 7fe9bae887a0 20 read_log 192834'11884 (192834'11699) modify bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by client.933448.0:41589 2014-08-11 15:26:02.991009 2014-08-13 17:38:03.076898 7fe9bae887a0 20 read_log 192834'11887 (192834'11705) modify f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.933448.0:41740 2014-08-11 15:26:03.037888 2014-08-13 17:38:03.076929 7fe9bae887a0 20 read_log 192834'11895 (192834'11721) modify 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.933448.0:42969 2014-08-11 15:26:03.458620 2014-08-13 17:38:03.076944 7fe9bae887a0 20 read_log 192834'11916 (0'0) promote eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13430 2014-08-11 15:26:18.477909 2014-08-13 17:38:03.076956 7fe9bae887a0 20 read_log 192834'11917 (192834'11916) modify eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by client.1883634.0:1097 2014-08-11 15:26:18.469033 2014-08-13 17:38:03.076969 7fe9bae887a0 20 read_log 192834'11924 (0'0) promote d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.338.0:13438 2014-08-11 15:26:19.788157 2014-08-13 17:38:03.076981 7fe9bae887a0 20 read_log 192834'11925 (192834'11924) modify d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by client.1883634.0:1459 2014-08-11 15:26:19.778352 2014-08-13 17:38:03.076993 7fe9bae887a0 20 read_log 192834'11940 (0'0) promote 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:13454 2014-08-11 15:26:22.943425 2014-08-13 17:38:03.077005 7fe9bae887a0 20 read_log 192834'11941 (192834'11940) modify 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.1883634.0:2290 2014-08-11 15:26:22.932366 2014-08-13 17:38:03.077020 7fe9bae887a0 20 read_log 192834'11946 (0'0) promote 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:13460 2014-08-11 15:26:23.008433 2014-08-13 17:38:03.077032 7fe9bae887a0 20 read_log 192834'11947 (192834'11946) modify 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.1883634.0:2308 2014-08-11 15:26:22.999247 2014-08-13 17:38:03.077047 7fe9bae887a0 20 read_log 192834'11978 (0'0) promote 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by osd.338.0:13492 2014-08-11 15:26:31.461640 2014-08-13 17:38:03.077059 7fe9bae887a0 20 read_log 192834'11979 (192834'11978) modify 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.1883634.0:4671 2014-08-11 15:26:31.451471 2014-08-13 17:38:03.077073 7fe9bae887a0 20 read_log 192834'12002 (0'0) promote 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by osd.338.0:13516 2014-08-11 15:26:36.724030 2014-08-13 17:38:03.077085 7fe9bae887a0 20 read_log 192834'12003 (192834'12002) modify 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.1883634.0:6087 2014-08-11 15:26:36.714638 2014-08-13 17:38:03.077098 7fe9bae887a0 20 read_log 192834'12020 (0'0) promote 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by osd.338.0:13534 2014-08-11 15:26:39.451719 2014-08-13 17:38:03.077112 7fe9bae887a0 20 read_log 192834'12021 (192834'12020) modify 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.1883634.0:6771 2014-08-11 15:26:39.443036 2014-08-13 17:38:03.077124 7fe9bae887a0 20 read_log 192834'12022 (0'0) promote fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13536 2014-08-11 15:26:39.526174 2014-08-13 17:38:03.077138 7fe9bae887a0 20 read_log 192834'12023 (192834'12022) modify fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by client.1883634.0:6793 2014-08-11 15:26:39.518675 2014-08-13 17:38:03.077182 7fe9bae887a0 20 read_log 192834'12028 (0'0) promote 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:13542 2014-08-11 15:26:40.281637 2014-08-13 17:38:03.077200 7fe9bae887a0 20 read_log 192834'12029 (192834'12028) modify 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.1883634.0:6979 2014-08-11 15:26:40.272345 2014-08-13 17:38:03.077212 7fe9bae887a0 20 read_log 192834'12038 (0'0) promote 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by osd.338.0:13552 2014-08-11 15:26:40.865155 2014-08-13 17:38:03.077224 7fe9bae887a0 20 read_log 192834'12039 (192834'12038) modify 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.1883634.0:7114 2014-08-11 15:26:40.857517 2014-08-13 17:38:03.077237 7fe9bae887a0 20 read_log 192834'12042 (0'0) promote d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by osd.338.0:13556 2014-08-11 15:26:44.309377 2014-08-13 17:38:03.077249 7fe9bae887a0 20 read_log 192834'12043 (192834'12042) modify d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.1883634.0:7914 2014-08-11 15:26:44.300640 2014-08-13 17:38:03.077261 7fe9bae887a0 20 read_log 192834'12044 (0'0) promote d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:13558 2014-08-11 15:26:45.357941 2014-08-13 17:38:03.077273 7fe9bae887a0 20 read_log 192834'12045 (192834'12044) modify d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.1883634.0:8194 2014-08-11 15:26:45.347158 2014-08-13 17:38:03.077286 7fe9bae887a0 20 read_log 192834'12060 (0'0) promote 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by osd.338.0:13574 2014-08-11 15:26:49.331038 2014-08-13 17:38:03.077298 7fe9bae887a0 20 read_log 192834'12061 (192834'12060) modify 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.1883634.0:9084 2014-08-11 15:26:49.320720 2014-08-13 17:38:03.077310 7fe9bae887a0 20 read_log 192834'12094 (0'0) promote ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:13608 2014-08-11 15:26:55.528921 2014-08-13 17:38:03.077322 7fe9bae887a0 20 read_log 192834'12095 (192834'12094) modify ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.1883634.0:10536 2014-08-11 15:26:55.519729 2014-08-13 17:38:03.077334 7fe9bae887a0 20 read_log 192834'12104 (0'0) promote f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by osd.338.0:13618 2014-08-11 15:26:59.046657 2014-08-13 17:38:03.077346 7fe9bae887a0 20 read_log 192834'12105 (192834'12104) modify f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.1883634.0:11257 2014-08-11 15:26:59.037343 2014-08-13 17:38:03.077359 7fe9bae887a0 20 read_log 192834'12126 (0'0) promote b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by osd.338.0:13640 2014-08-11 15:27:05.105671 2014-08-13 17:38:03.077371 7fe9bae887a0 20 read_log 192834'12127 (192834'12126) modify b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.1883634.0:12646 2014-08-11 15:27:05.095901 2014-08-13 17:38:03.077383 7fe9bae887a0 20 read_log 192834'12156 (0'0) promote dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13670 2014-08-11 15:27:11.126437 2014-08-13 17:38:03.077398 7fe9bae887a0 20 read_log 192834'12157 (192834'12156) modify dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by client.1883634.0:13888 2014-08-11 15:27:11.117446 2014-08-13 17:38:03.077412 7fe9bae887a0 20 read_log 192834'12174 (0'0) promote c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.338.0:13688 2014-08-11 15:27:17.618008 2014-08-13 17:38:03.077424 7fe9bae887a0 20 read_log 192834'12175 (192834'12174) modify c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by client.1883634.0:15197 2014-08-11 15:27:17.607226 2014-08-13 17:38:03.077460 7fe9bae887a0 20 read_log 192834'12196 (0'0) promote c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by osd.338.0:13710 2014-08-11 15:27:28.400497 2014-08-13 17:38:03.077476 7fe9bae887a0 20 read_log 192834'12197 (192834'12196) modify c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.1883634.0:16596 2014-08-11 15:27:28.391074 2014-08-13 17:38:03.077489 7fe9bae887a0 20 read_log 192834'12198 (0'0) promote 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.338.0:13712 2014-08-11 15:27:28.591588 2014-08-13 17:38:03.077501 7fe9bae887a0 20 read_log 192834'12199 (192834'12198) modify 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by client.1883634.0:16637 2014-08-11 15:27:28.581804 2014-08-13 17:38:03.077513 7fe9bae887a0 20 read_log 192834'12216 (0'0) promote 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.338.0:13730 2014-08-11 15:27:38.818154 2014-08-13 17:38:03.077525 7fe9bae887a0 20 read_log 192834'12217 (192834'12216) modify 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by client.1883634.0:17742 2014-08-11 15:27:38.808875 2014-08-13 17:38:03.077538 7fe9bae887a0 20 read_log 192834'12256 (0'0) promote bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.338.0:13770 2014-08-11 15:27:50.136394 2014-08-13 17:38:03.077550 7fe9bae887a0 20 read_log 192834'12257 (192834'12256) modify bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by client.1883634.0:19259 2014-08-11 15:27:50.127232 2014-08-13 17:38:03.077563 7fe9bae887a0 20 read_log 192834'12272 (0'0) promote 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by osd.338.0:13786 2014-08-11 15:27:55.567259 2014-08-13 17:38:03.077576 7fe9bae887a0 20 read_log 192834'12273 (192834'12272) modify 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.1883634.0:20183 2014-08-11 15:27:55.557801 2014-08-13 17:38:03.077588 7fe9bae887a0 20 read_log 192834'12332 (0'0) promote f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.338.0:13846 2014-08-11 15:28:03.717752 2014-08-13 17:38:03.077600 7fe9bae887a0 20 read_log 192834'12333 (192834'12332) modify f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by client.1883634.0:21298 2014-08-11 15:28:03.708896 2014-08-13 17:38:03.077617 7fe9bae887a0 20 read_log 192834'12342 (0'0) promote 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.338.0:13856 2014-08-11 15:28:06.723358 2014-08-13 17:38:03.077629 7fe9bae887a0 20 read_log 192834'12343 (192834'12342) modify 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by client.1883634.0:22104 2014-08-11 15:28:06.712110 2014-08-13 17:38:03.077641 7fe9bae887a0 20 read_log 192834'12364 (0'0) promote 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by osd.338.0:13878 2014-08-11 15:28:08.856462 2014-08-13 17:38:03.077654 7fe9bae887a0 20 read_log 192834'12365 (192834'12364) modify 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.1883634.0:22473 2014-08-11 15:28:08.847311 2014-08-13 17:38:03.077666 7fe9bae887a0 20 read_log 192834'12367 (192834'11760) clean 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:13881 2014-08-11 15:28:08.882193 2014-08-13 17:38:03.077678 7fe9bae887a0 20 read_log 192834'12369 (192834'11793) clean e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13883 2014-08-11 15:28:09.467476 2014-08-13 17:38:03.077692 7fe9bae887a0 20 read_log 192834'12371 (192834'11877) clean 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13885 2014-08-11 15:28:09.814756 2014-08-13 17:38:03.077704 7fe9bae887a0 20 read_log 192834'12372 (192834'12023) clean fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13886 2014-08-11 15:28:09.815099 2014-08-13 17:38:03.077727 7fe9bae887a0 20 read_log 192834'12374 (192834'11827) clean a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13888 2014-08-11 15:28:09.832537 2014-08-13 17:38:03.077740 7fe9bae887a0 20 read_log 192834'12375 (192834'11830) clean 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13889 2014-08-11 15:28:09.832832 2014-08-13 17:38:03.077752 7fe9bae887a0 20 read_log 192834'12376 (192834'11847) clean 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13890 2014-08-11 15:28:09.851877 2014-08-13 17:38:03.077764 7fe9bae887a0 20 read_log 192834'12380 (192834'11917) clean eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13894 2014-08-11 15:28:09.867380 2014-08-13 17:38:03.077777 7fe9bae887a0 20 read_log 192834'12400 (192834'12367) delete 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:13914 2014-08-11 15:28:10.054995 2014-08-13 17:38:03.077789 7fe9bae887a0 20 read_log 192834'12401 (192834'12372) delete fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13915 2014-08-11 15:28:10.055489 2014-08-13 17:38:03.077801 7fe9bae887a0 20 read_log 192834'12402 (192834'12369) delete e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13916 2014-08-11 15:28:10.055780 2014-08-13 17:38:03.077813 7fe9bae887a0 20 read_log 192834'12403 (192834'12380) delete eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13917 2014-08-11 15:28:10.059713 2014-08-13 17:38:03.077826 7fe9bae887a0 20 read_log 192834'12404 (192834'12371) delete 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13918 2014-08-11 15:28:10.061758 2014-08-13 17:38:03.077838 7fe9bae887a0 20 read_log 192834'12405 (192834'12374) delete a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13919 2014-08-11 15:28:10.062470 2014-08-13 17:38:03.077851 7fe9bae887a0 20 read_log 192834'12406 (192834'12375) delete 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13920 2014-08-11 15:28:10.062825 2014-08-13 17:38:03.077863 7fe9bae887a0 20 read_log 192834'12407 (192834'12376) delete 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13921 2014-08-11 15:28:10.067817 2014-08-13 17:38:03.077878 7fe9bae887a0 20 read_log 192834'12428 (0'0) promote b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by osd.338.0:13942 2014-08-11 15:28:12.300144 2014-08-13 17:38:03.077893 7fe9bae887a0 20 read_log 192834'12429 (192834'12428) modify b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.1883634.0:23055 2014-08-11 15:28:12.289580 2014-08-13 17:38:03.077905 7fe9bae887a0 20 read_log 192834'12432 (0'0) promote 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by osd.338.0:13946 2014-08-11 15:28:13.602545 2014-08-13 17:38:03.077920 7fe9bae887a0 20 read_log 192834'12433 (192834'12432) modify 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.1883634.0:23243 2014-08-11 15:28:13.593326 2014-08-13 17:38:03.077932 7fe9bae887a0 20 read_log 192834'12439 (192834'12157) clean dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13953 2014-08-11 15:28:15.576065 2014-08-13 17:38:03.077946 7fe9bae887a0 20 read_log 192834'12454 (192834'11871) clean f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13968 2014-08-11 15:28:16.102118 2014-08-13 17:38:03.077959 7fe9bae887a0 20 read_log 192834'12455 (192834'11884) clean bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:13969 2014-08-11 15:28:16.110089 2014-08-13 17:38:03.077971 7fe9bae887a0 20 read_log 192834'12457 (192834'12095) clean ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:13971 2014-08-11 15:28:16.312186 2014-08-13 17:38:03.077991 7fe9bae887a0 20 read_log 192834'12458 (192834'12045) clean d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:13972 2014-08-11 15:28:16.312513 2014-08-13 17:38:03.078005 7fe9bae887a0 20 read_log 192834'12460 (192834'11941) clean 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:13974 2014-08-11 15:28:16.440614 2014-08-13 17:38:03.078018 7fe9bae887a0 20 read_log 192834'12461 (192834'11825) clean 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:13975 2014-08-11 15:28:16.554188 2014-08-13 17:38:03.078030 7fe9bae887a0 20 read_log 192834'12462 (192834'11824) clean 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:13976 2014-08-11 15:28:16.658741 2014-08-13 17:38:03.078043 7fe9bae887a0 20 read_log 192834'12463 (192834'12029) clean 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:13977 2014-08-11 15:28:16.659077 2014-08-13 17:38:03.078055 7fe9bae887a0 20 read_log 192834'12465 (192834'12439) delete dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13979 2014-08-11 15:28:16.798178 2014-08-13 17:38:03.078069 7fe9bae887a0 20 read_log 192834'12472 (192834'11947) clean 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:13986 2014-08-11 15:28:16.824111 2014-08-13 17:38:03.078081 7fe9bae887a0 20 read_log 192834'12485 (192834'12454) delete f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13999 2014-08-11 15:28:16.873434 2014-08-13 17:38:03.078096 7fe9bae887a0 20 read_log 192834'12486 (192834'12455) delete bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:14000 2014-08-11 15:28:16.876617 2014-08-13 17:38:03.078108 7fe9bae887a0 20 read_log 192834'12487 (192834'12457) delete ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:14001 2014-08-11 15:28:16.879811 2014-08-13 17:38:03.078123 7fe9bae887a0 20 read_log 192834'12488 (192834'12458) delete d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:14002 2014-08-11 15:28:16.882491 2014-08-13 17:38:03.078137 7fe9bae887a0 20 read_log 192834'12489 (192834'12460) delete 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:14003 2014-08-11 15:28:16.887606 2014-08-13 17:38:03.078149 7fe9bae887a0 20 read_log 192834'12490 (192834'12463) delete 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:14004 2014-08-11 15:28:16.893249 2014-08-13 17:38:03.078169 7fe9bae887a0 20 read_log 192834'12491 (192834'12461) delete 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:14005 2014-08-11 15:28:16.893702 2014-08-13 17:38:03.078183 7fe9bae887a0 20 read_log 192834'12492 (192834'12472) delete 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:14006 2014-08-11 15:28:16.896122 2014-08-13 17:38:03.078198 7fe9bae887a0 20 read_log 192834'12493 (192834'12462) delete 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:14007 2014-08-11 15:28:16.897830 2014-08-13 17:38:03.078230 7fe9bae887a0 20 read_log 192834'12503 (0'0) promote eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:14017 2014-08-11 15:28:19.058274 2014-08-13 17:38:03.078246 7fe9bae887a0 20 read_log 192834'12504 (192834'12503) modify eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by client.1883634.0:24421 2014-08-11 15:28:19.031741 2014-08-13 17:38:03.078258 7fe9bae887a0 20 read_log 192834'12509 (192834'11925) modify d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by client.1883634.0:24783 2014-08-11 15:28:19.519578 2014-08-13 17:38:03.078273 7fe9bae887a0 20 read_log 192834'12519 (0'0) promote 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:14033 2014-08-11 15:28:21.639334 2014-08-13 17:38:03.078294 7fe9bae887a0 20 read_log 192834'12520 (192834'12519) modify 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.1883634.0:25614 2014-08-11 15:28:21.613290 2014-08-13 17:38:03.078308 7fe9bae887a0 20 read_log 192834'12522 (0'0) promote 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:14036 2014-08-11 15:28:21.687946 2014-08-13 17:38:03.078321 7fe9bae887a0 20 read_log 192834'12523 (192834'12522) modify 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.1883634.0:25632 2014-08-11 15:28:21.658847 2014-08-13 17:38:03.078333 7fe9bae887a0 20 read_log 192834'12544 (192834'11979) modify 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.1883634.0:27995 2014-08-11 15:28:25.246055 2014-08-13 17:38:03.078345 7fe9bae887a0 20 read_log 192834'12559 (192834'12003) modify 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.1883634.0:29411 2014-08-11 15:28:27.188875 2014-08-13 17:38:03.078357 7fe9bae887a0 20 read_log 192834'12569 (192834'12021) modify 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.1883634.0:30095 2014-08-11 15:28:28.082149 2014-08-13 17:38:03.078369 7fe9bae887a0 20 read_log 192834'12570 (0'0) promote fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:14083 2014-08-11 15:28:28.154256 2014-08-13 17:38:03.078381 7fe9bae887a0 20 read_log 192834'12571 (192834'12570) modify fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by client.1883634.0:30117 2014-08-11 15:28:28.128162 2014-08-13 17:38:03.078394 7fe9bae887a0 20 read_log 192834'12574 (0'0) promote 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:14087 2014-08-11 15:28:28.460223 2014-08-13 17:38:03.078406 7fe9bae887a0 20 read_log 192834'12575 (192834'12574) modify 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.1883634.0:30303 2014-08-11 15:28:28.434262 2014-08-13 17:38:03.078418 7fe9bae887a0 20 read_log 192834'12582 (192834'12039) modify 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.1883634.0:30438 2014-08-11 15:28:28.635573 2014-08-13 17:38:03.078430 7fe9bae887a0 20 read_log 192834'12584 (192834'12043) modify d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.1883634.0:31238 2014-08-11 15:28:30.145163 2014-08-13 17:38:03.078443 7fe9bae887a0 20 read_log 192834'12586 (0'0) promote d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:14099 2014-08-11 15:28:30.681202 2014-08-13 17:38:03.078455 7fe9bae887a0 20 read_log 192834'12587 (192834'12586) modify d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.1883634.0:31518 2014-08-11 15:28:30.652881 2014-08-13 17:38:03.078467 7fe9bae887a0 20 read_log 192834'12597 (192834'12061) modify 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.1883634.0:32408 2014-08-11 15:28:31.812262 2014-08-13 17:38:03.078479 7fe9bae887a0 20 read_log 192834'12616 (0'0) promote ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:14129 2014-08-11 15:28:33.949505 2014-08-13 17:38:03.078491 7fe9bae887a0 20 read_log 192834'12617 (192834'12616) modify ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.1883634.0:33860 2014-08-11 15:28:33.916286 2014-08-13 17:38:03.078504 7fe9bae887a0 20 read_log 192834'12623 (192834'12105) modify f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.1883634.0:34581 2014-08-11 15:28:35.031808 2014-08-13 17:38:03.078516 7fe9bae887a0 20 read_log 192834'12635 (192834'12127) modify b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.1883634.0:35970 2014-08-11 15:28:37.414489 2014-08-13 17:38:03.078528 7fe9bae887a0 20 read_log 192834'12652 (0'0) promote dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:14165 2014-08-11 15:28:39.031281 2014-08-13 17:38:03.078550 7fe9bae887a0 20 read_log 192834'12653 (192834'12652) modify dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by client.1883634.0:37212 2014-08-11 15:28:39.001580 2014-08-13 17:38:03.078565 7fe9bae887a0 20 read_log 192834'12663 (192834'12175) modify c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by client.1883634.0:38521 2014-08-11 15:28:40.757255 2014-08-13 17:38:03.078578 7fe9bae887a0 20 read_log 192834'12674 (192834'12197) modify c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.1883634.0:39920 2014-08-11 15:28:42.780660 2014-08-13 17:38:03.078590 7fe9bae887a0 20 read_log 192834'12675 (192834'12199) modify 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by client.1883634.0:39961 2014-08-11 15:28:42.940909 2014-08-13 17:38:03.078602 7fe9bae887a0 20 read_log 192834'12684 (192834'12217) modify 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by client.1883634.0:41066 2014-08-11 15:28:44.577069 2014-08-13 17:38:03.078616 7fe9bae887a0 20 read_log 192834'12704 (192834'12257) modify bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by client.1883634.0:42583 2014-08-11 15:28:46.288337 2014-08-13 17:38:03.078628 7fe9bae887a0 20 read_log 192834'12712 (192834'12273) modify 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.1883634.0:43507 2014-08-11 15:28:46.575864 2014-08-13 17:38:03.078640 7fe9bae887a0 20 read_log 192834'12721 (192834'12333) modify f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by client.1883634.0:44622 2014-08-11 15:28:46.908095 2014-08-13 17:38:03.078652 7fe9bae887a0 20 read_log 192834'12726 (192834'12343) modify 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by client.1883634.0:45428 2014-08-11 15:28:47.274040 2014-08-13 17:38:03.078684 7fe9bae887a0 20 read_log 192834'12732 (192834'12365) modify 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.1883634.0:45797 2014-08-11 15:28:47.392947 2014-08-13 17:38:03.078697 7fe9bae887a0 20 read_log 192834'12739 (192834'12429) modify b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.1883634.0:46379 2014-08-11 15:28:47.573669 2014-08-13 17:38:03.078709 7fe9bae887a0 20 read_log 192834'12741 (192834'12433) modify 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.1883634.0:46567 2014-08-11 15:28:47.633148 2014-08-13 17:38:03.078721 7fe9bae887a0 20 read_log 192888'12746 (0'0) promote 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by osd.305.0:27402 2014-08-11 16:44:58.481828 2014-08-13 17:38:03.078734 7fe9bae887a0 20 read_log 192888'12747 (192888'12746) modify 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.1315259.0:843 2014-08-11 16:44:58.473411 2014-08-13 17:38:03.078746 7fe9bae887a0 20 read_log 192888'12748 (0'0) promote f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.305.0:27406 2014-08-11 16:44:58.664888 2014-08-13 17:38:03.078760 7fe9bae887a0 20 read_log 192888'12749 (192888'12748) modify f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by client.1315259.0:892 2014-08-11 16:44:58.655515 2014-08-13 17:38:03.078773 7fe9bae887a0 20 read_log 192888'12750 (0'0) promote e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by osd.305.0:27410 2014-08-11 16:44:59.005411 2014-08-13 17:38:03.078787 7fe9bae887a0 20 read_log 192888'12751 (192888'12750) modify e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.1315259.0:984 2014-08-11 16:44:58.997019 2014-08-13 17:38:03.078799 7fe9bae887a0 20 read_log 192888'12754 (0'0) promote 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by osd.305.0:27418 2014-08-11 16:45:01.546408 2014-08-13 17:38:03.078820 7fe9bae887a0 20 read_log 192888'12755 (192888'12754) modify 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.1315259.0:1641 2014-08-11 16:45:01.538449 2014-08-13 17:38:03.078835 7fe9bae887a0 20 read_log 192888'12756 (0'0) promote ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by osd.305.0:27422 2014-08-11 16:45:02.120529 2014-08-13 17:38:03.078848 7fe9bae887a0 20 read_log 192888'12757 (192888'12756) modify ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.1315259.0:1796 2014-08-11 16:45:02.112815 2014-08-13 17:38:03.078862 7fe9bae887a0 20 read_log 192888'12758 (0'0) promote 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.305.0:27426 2014-08-11 16:45:02.903642 2014-08-13 17:38:03.078875 7fe9bae887a0 20 read_log 192888'12759 (192888'12758) modify 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by client.1315259.0:2008 2014-08-11 16:45:02.895752 2014-08-13 17:38:03.078887 7fe9bae887a0 20 read_log 192888'12766 (0'0) promote e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by osd.305.0:27455 2014-08-11 16:45:06.866469 2014-08-13 17:38:03.078899 7fe9bae887a0 20 read_log 192888'12767 (192888'12766) modify e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.1315259.0:3068 2014-08-11 16:45:06.858142 2014-08-13 17:38:03.078912 7fe9bae887a0 20 read_log 192888'12768 (0'0) promote 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by osd.305.0:27471 2014-08-11 16:45:09.639102 2014-08-13 17:38:03.078926 7fe9bae887a0 20 read_log 192888'12769 (192888'12768) modify 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.1315259.0:3776 2014-08-11 16:45:09.631060 2014-08-13 17:38:03.078939 7fe9bae887a0 20 read_log 192888'12770 (0'0) promote 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by osd.305.0:27475 2014-08-11 16:45:10.072817 2014-08-13 17:38:03.078953 7fe9bae887a0 20 read_log 192888'12771 (192888'12770) modify 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.1315259.0:3894 2014-08-11 16:45:10.065369 2014-08-13 17:38:03.078965 7fe9bae887a0 20 read_log 192888'12772 (0'0) promote c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by osd.305.0:27483 2014-08-11 16:45:10.684042 2014-08-13 17:38:03.078979 7fe9bae887a0 20 read_log 192888'12773 (192888'12772) modify c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.1315259.0:4061 2014-08-11 16:45:10.676331 2014-08-13 17:38:03.078992 7fe9bae887a0 20 read_log 192888'12774 (0'0) promote d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.305.0:27501 2014-08-11 16:45:13.041530 2014-08-13 17:38:03.079006 7fe9bae887a0 20 read_log 192888'12775 (192888'12774) modify d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by client.1315259.0:4710 2014-08-11 16:45:13.032955 2014-08-13 17:38:03.079019 7fe9bae887a0 20 read_log 192888'12780 (0'0) promote afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by osd.305.0:27509 2014-08-11 16:45:14.287842 2014-08-13 17:38:03.079033 7fe9bae887a0 20 read_log 192888'12781 (192888'12780) modify afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.1315259.0:5044 2014-08-11 16:45:14.279057 2014-08-13 17:38:03.079045 7fe9bae887a0 20 read_log 192888'12784 (0'0) promote db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by osd.305.0:27539 2014-08-11 16:45:18.541294 2014-08-13 17:38:03.079059 7fe9bae887a0 20 read_log 192888'12785 (192888'12784) modify db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.1315259.0:6210 2014-08-11 16:45:18.533172 2014-08-13 17:38:03.079071 7fe9bae887a0 20 read_log 192888'12786 (0'0) promote 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by osd.305.0:27541 2014-08-11 16:45:18.781674 2014-08-13 17:38:03.079092 7fe9bae887a0 20 read_log 192888'12787 (192888'12786) modify 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.1315259.0:6276 2014-08-11 16:45:18.771615 2014-08-13 17:38:03.079107 7fe9bae887a0 20 read_log 192888'12788 (0'0) promote 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.305.0:27547 2014-08-11 16:45:19.485946 2014-08-13 17:38:03.079119 7fe9bae887a0 20 read_log 192888'12789 (192888'12788) modify 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by client.1315259.0:6471 2014-08-11 16:45:19.477888 2014-08-13 17:38:03.079132 7fe9bae887a0 20 read_log 192888'12790 (0'0) promote 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by osd.305.0:27549 2014-08-11 16:45:20.103082 2014-08-13 17:38:03.079144 7fe9bae887a0 20 read_log 192888'12791 (192888'12790) modify 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.1315259.0:6643 2014-08-11 16:45:20.095501 2014-08-13 17:38:03.079161 7fe9bae887a0 20 read_log 192888'12792 (0'0) promote c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.305.0:27557 2014-08-11 16:45:22.253642 2014-08-13 17:38:03.079176 7fe9bae887a0 20 read_log 192888'12793 (192888'12792) modify c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by client.1315259.0:7236 2014-08-11 16:45:22.244724 2014-08-13 17:38:03.079188 7fe9bae887a0 20 read_log 192888'12794 (0'0) promote 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.305.0:27567 2014-08-11 16:45:23.899001 2014-08-13 17:38:03.079202 7fe9bae887a0 20 read_log 192888'12795 (192888'12794) modify 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by client.1315259.0:7694 2014-08-11 16:45:23.891896 2014-08-13 17:38:03.079216 7fe9bae887a0 20 read_log 192888'12798 (0'0) promote f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.305.0:27607 2014-08-11 16:45:27.514750 2014-08-13 17:38:03.079228 7fe9bae887a0 20 read_log 192888'12799 (192888'12798) modify f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by client.1315259.0:8650 2014-08-11 16:45:27.506696 2014-08-13 17:38:03.079242 7fe9bae887a0 20 read_log 192888'12802 (0'0) promote 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.305.0:27651 2014-08-11 16:45:34.733452 2014-08-13 17:38:03.079254 7fe9bae887a0 20 read_log 192888'12803 (192888'12802) modify 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by client.1315259.0:10530 2014-08-11 16:45:34.725705 2014-08-13 17:38:03.079269 7fe9bae887a0 20 read_log 192888'12804 (0'0) promote f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.305.0:27665 2014-08-11 16:45:37.339602 2014-08-13 17:38:03.079281 7fe9bae887a0 20 read_log 192888'12805 (192888'12804) modify f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by client.1315259.0:11180 2014-08-11 16:45:37.331965 2014-08-13 17:38:03.079295 7fe9bae887a0 20 read_log 192888'12806 (0'0) promote d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.305.0:27679 2014-08-11 16:45:38.042929 2014-08-13 17:38:03.079307 7fe9bae887a0 20 read_log 192888'12807 (192888'12806) modify d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by client.1315259.0:11358 2014-08-11 16:45:38.035641 2014-08-13 17:38:03.079319 7fe9bae887a0 20 read_log 192888'12808 (0'0) promote 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by osd.305.0:27687 2014-08-11 16:45:39.500694 2014-08-13 17:38:03.079331 7fe9bae887a0 20 read_log 192888'12809 (192888'12808) modify 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.1315259.0:11732 2014-08-11 16:45:39.492511 2014-08-13 17:38:03.079344 7fe9bae887a0 20 read_log 192888'12810 (0'0) promote 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.305.0:27703 2014-08-11 16:45:42.624694 2014-08-13 17:38:03.079381 7fe9bae887a0 20 read_log 192888'12811 (192888'12810) modify 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by client.1315259.0:12463 2014-08-11 16:45:42.616822 2014-08-13 17:38:03.079397 7fe9bae887a0 20 read_log 192888'12818 (0'0) promote 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by osd.305.0:27745 2014-08-11 16:45:51.620835 2014-08-13 17:38:03.079409 7fe9bae887a0 20 read_log 192888'12819 (192888'12818) modify 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.1315259.0:14754 2014-08-11 16:45:51.611943 2014-08-13 17:38:03.079422 7fe9bae887a0 20 read_log 192888'12820 (0'0) promote 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by osd.305.0:27777 2014-08-11 16:45:57.492638 2014-08-13 17:38:03.079434 7fe9bae887a0 20 read_log 192888'12821 (192888'12820) modify 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.1315259.0:16283 2014-08-11 16:45:57.484663 2014-08-13 17:38:03.079448 7fe9bae887a0 20 read_log 192888'12822 (0'0) promote f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.305.0:27801 2014-08-11 16:46:00.261347 2014-08-13 17:38:03.079460 7fe9bae887a0 20 read_log 192888'12823 (192888'12822) modify f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by client.1315259.0:16981 2014-08-11 16:46:00.252542 2014-08-13 17:38:03.079473 7fe9bae887a0 20 read_log 192888'12832 (0'0) promote 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by osd.305.0:27831 2014-08-11 16:46:05.527123 2014-08-13 17:38:03.079486 7fe9bae887a0 20 read_log 192888'12833 (192888'12832) modify 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.1315259.0:18296 2014-08-11 16:46:05.517121 2014-08-13 17:38:03.079502 7fe9bae887a0 20 read_log 192888'12834 (0'0) promote c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by osd.305.0:27835 2014-08-11 16:46:06.936200 2014-08-13 17:38:03.079551 7fe9bae887a0 20 read_log 192888'12835 (192888'12834) modify c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.1315259.0:18673 2014-08-11 16:46:06.927103 2014-08-13 17:38:03.079588 7fe9bae887a0 20 read_log 192888'12836 (0'0) promote 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.305.0:27839 2014-08-11 16:46:07.412455 2014-08-13 17:38:03.079605 7fe9bae887a0 20 read_log 192888'12837 (192888'12836) modify 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by client.1315259.0:18799 2014-08-11 16:46:07.401681 2014-08-13 17:38:03.079617 7fe9bae887a0 20 read_log 192888'12838 (0'0) promote d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by osd.305.0:27843 2014-08-11 16:46:09.228380 2014-08-13 17:38:03.079630 7fe9bae887a0 20 read_log 192888'12839 (192888'12838) modify d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.1315259.0:19234 2014-08-11 16:46:09.219655 2014-08-13 17:38:03.079642 7fe9bae887a0 20 read_log 192888'12840 (0'0) promote 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by osd.305.0:27871 2014-08-11 16:46:13.465331 2014-08-13 17:38:03.079654 7fe9bae887a0 20 read_log 192888'12841 (192888'12840) modify 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.1315259.0:20365 2014-08-11 16:46:13.456177 2014-08-13 17:38:03.079667 7fe9bae887a0 20 read_log 192888'12842 (0'0) promote 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by osd.305.0:27877 2014-08-11 16:46:16.146016 2014-08-13 17:38:03.079679 7fe9bae887a0 20 read_log 192888'12843 (192888'12842) modify 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.1315259.0:21073 2014-08-11 16:46:16.136835 2014-08-13 17:38:03.079691 7fe9bae887a0 20 read_log 192888'12844 (0'0) promote a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.305.0:27895 2014-08-11 16:46:18.285906 2014-08-13 17:38:03.079711 7fe9bae887a0 20 read_log 192888'12845 (192888'12844) modify a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by client.1315259.0:21630 2014-08-11 16:46:18.276682 2014-08-13 17:38:03.079727 7fe9bae887a0 20 read_log 192888'12858 (0'0) promote 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by osd.305.0:27997 2014-08-11 16:46:34.940573 2014-08-13 17:38:03.079739 7fe9bae887a0 20 read_log 192888'12859 (192888'12858) modify 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.1315259.0:25833 2014-08-11 16:46:34.930191 2014-08-13 17:38:03.079751 7fe9bae887a0 20 read_log 192888'12862 (0'0) promote 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by osd.305.0:28045 2014-08-11 16:46:43.287565 2014-08-13 17:38:03.079763 7fe9bae887a0 20 read_log 192888'12863 (192888'12862) modify 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.1315259.0:27827 2014-08-11 16:46:43.279264 2014-08-13 17:38:03.079775 7fe9bae887a0 20 read_log 192888'12864 (0'0) promote f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by osd.305.0:28053 2014-08-11 16:46:45.115545 2014-08-13 17:38:03.079787 7fe9bae887a0 20 read_log 192888'12865 (192888'12864) modify f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.1315259.0:28274 2014-08-11 16:46:45.107281 2014-08-13 17:38:03.079799 7fe9bae887a0 20 read_log 192888'12868 (0'0) promote 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by osd.305.0:28077 2014-08-11 16:46:48.908918 2014-08-13 17:38:03.079811 7fe9bae887a0 20 read_log 192888'12869 (192888'12868) modify 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.1315259.0:29216 2014-08-11 16:46:48.899651 2014-08-13 17:38:03.079823 7fe9bae887a0 20 read_log 192888'12870 (0'0) promote e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by osd.305.0:28117 2014-08-11 16:46:54.279204 2014-08-13 17:38:03.079835 7fe9bae887a0 20 read_log 192888'12871 (192888'12870) modify e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.1315259.0:30574 2014-08-11 16:46:54.270345 2014-08-13 17:38:03.079847 7fe9bae887a0 20 read_log 192888'12873 (192888'12747) modify 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.1315259.0:31570 2014-08-11 16:46:56.132979 2014-08-13 17:38:03.079859 7fe9bae887a0 20 read_log 192888'12874 (192888'12749) modify f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by client.1315259.0:31619 2014-08-11 16:46:56.148294 2014-08-13 17:38:03.079872 7fe9bae887a0 20 read_log 192888'12875 (192888'12751) modify e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.1315259.0:31711 2014-08-11 16:46:56.178349 2014-08-13 17:38:03.079884 7fe9bae887a0 20 read_log 192888'12877 (192888'12755) modify 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.1315259.0:32368 2014-08-11 16:46:56.359187 2014-08-13 17:38:03.079896 7fe9bae887a0 20 read_log 192888'12878 (192888'12757) modify ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.1315259.0:32523 2014-08-11 16:46:56.401165 2014-08-13 17:38:03.079910 7fe9bae887a0 20 read_log 192888'12879 (192888'12759) modify 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by client.1315259.0:32735 2014-08-11 16:46:56.456087 2014-08-13 17:38:03.079923 7fe9bae887a0 20 read_log 192888'12883 (192888'12767) modify e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.1315259.0:33795 2014-08-11 16:46:56.727196 2014-08-13 17:38:03.079936 7fe9bae887a0 20 read_log 192888'12884 (192888'12769) modify 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.1315259.0:34503 2014-08-11 16:46:56.911069 2014-08-13 17:38:03.079948 7fe9bae887a0 20 read_log 192888'12885 (192888'12771) modify 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.1315259.0:34621 2014-08-11 16:46:56.943153 2014-08-13 17:38:03.079970 7fe9bae887a0 20 read_log 192888'12886 (192888'12773) modify c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.1315259.0:34788 2014-08-11 16:46:56.986354 2014-08-13 17:38:03.079985 7fe9bae887a0 20 read_log 192888'12887 (192888'12775) modify d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by client.1315259.0:35437 2014-08-11 16:46:57.164075 2014-08-13 17:38:03.079997 7fe9bae887a0 20 read_log 192888'12890 (192888'12781) modify afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.1315259.0:35771 2014-08-11 16:46:57.241531 2014-08-13 17:38:03.080009 7fe9bae887a0 20 read_log 192888'12892 (192888'12785) modify db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.1315259.0:36937 2014-08-11 16:46:57.550493 2014-08-13 17:38:03.080021 7fe9bae887a0 20 read_log 192888'12893 (192888'12787) modify 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.1315259.0:37003 2014-08-11 16:46:57.569100 2014-08-13 17:38:03.080033 7fe9bae887a0 20 read_log 192888'12894 (192888'12789) modify 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by client.1315259.0:37198 2014-08-11 16:46:57.620096 2014-08-13 17:38:03.080047 7fe9bae887a0 20 read_log 192888'12895 (192888'12791) modify 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.1315259.0:37370 2014-08-11 16:46:57.664670 2014-08-13 17:38:03.080058 7fe9bae887a0 20 read_log 192888'12896 (192888'12793) modify c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by client.1315259.0:37963 2014-08-11 16:46:57.821958 2014-08-13 17:38:03.080073 7fe9bae887a0 20 read_log 192888'12897 (192888'12795) modify 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by client.1315259.0:38421 2014-08-11 16:46:57.943109 2014-08-13 17:38:03.080085 7fe9bae887a0 20 read_log 192888'12899 (192888'12799) modify f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by client.1315259.0:39377 2014-08-11 16:46:58.195773 2014-08-13 17:38:03.080097 7fe9bae887a0 20 read_log 192888'12901 (192888'12803) modify 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by client.1315259.0:41257 2014-08-11 16:46:58.700899 2014-08-13 17:38:03.080111 7fe9bae887a0 20 read_log 192888'12902 (192888'12805) modify f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by client.1315259.0:41907 2014-08-11 16:46:58.868370 2014-08-13 17:38:03.080123 7fe9bae887a0 20 read_log 192888'12903 (192888'12807) modify d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by client.1315259.0:42085 2014-08-11 16:46:58.915331 2014-08-13 17:38:03.080135 7fe9bae887a0 20 read_log 192888'12904 (192888'12809) modify 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.1315259.0:42459 2014-08-11 16:46:59.016260 2014-08-13 17:38:03.080150 7fe9bae887a0 20 read_log 192888'12905 (192888'12811) modify 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by client.1315259.0:43190 2014-08-11 16:46:59.218752 2014-08-13 17:38:03.080177 7fe9bae887a0 20 read_log 192888'12909 (192888'12819) modify 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.1315259.0:45481 2014-08-11 16:46:59.882019 2014-08-13 17:38:03.080190 7fe9bae887a0 20 read_log 192888'12910 (192888'12821) modify 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.1315259.0:47010 2014-08-11 16:47:00.321359 2014-08-13 17:38:03.080204 7fe9bae887a0 20 read_log 192888'12911 (192888'12823) modify f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by client.1315259.0:47708 2014-08-11 16:47:00.542200 2014-08-13 17:38:03.080217 7fe9bae887a0 20 read_log 192888'12916 (192888'12833) modify 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.1315259.0:49023 2014-08-11 16:47:00.936109 2014-08-13 17:38:03.080230 7fe9bae887a0 20 read_log 192888'12917 (192888'12835) modify c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.1315259.0:49400 2014-08-11 16:47:01.048651 2014-08-13 17:38:03.080253 7fe9bae887a0 20 read_log 192888'12918 (192888'12837) modify 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by client.1315259.0:49526 2014-08-11 16:47:01.091294 2014-08-13 17:38:03.080268 7fe9bae887a0 20 read_log 192888'12919 (192888'12839) modify d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.1315259.0:49961 2014-08-11 16:47:01.222499 2014-08-13 17:38:03.080280 7fe9bae887a0 20 read_log 192888'12920 (192888'12841) modify 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.1315259.0:51092 2014-08-11 16:47:01.579019 2014-08-13 17:38:03.080292 7fe9bae887a0 20 read_log 192888'12921 (192888'12843) modify 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.1315259.0:51800 2014-08-11 16:47:01.787232 2014-08-13 17:38:03.080304 7fe9bae887a0 20 read_log 192888'12922 (192888'12845) modify a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by client.1315259.0:52357 2014-08-11 16:47:01.952121 2014-08-13 17:38:03.080316 7fe9bae887a0 20 read_log 192888'12929 (192888'12859) modify 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.1315259.0:56560 2014-08-11 16:47:03.244641 2014-08-13 17:38:03.080328 7fe9bae887a0 20 read_log 192888'12931 (192888'12863) modify 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.1315259.0:58554 2014-08-11 16:47:03.835799 2014-08-13 17:38:03.080340 7fe9bae887a0 20 read_log 192888'12932 (192888'12865) modify f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.1315259.0:59001 2014-08-11 16:47:03.971162 2014-08-13 17:38:03.080352 7fe9bae887a0 20 read_log 192888'12934 (192888'12869) modify 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.1315259.0:59943 2014-08-11 16:47:04.292852 2014-08-13 17:38:03.080364 7fe9bae887a0 20 read_log 192888'12935 (192888'12871) modify e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.1315259.0:61301 2014-08-11 16:47:04.697605 2014-08-13 17:38:03.080377 7fe9bae887a0 20 read_log 192888'12950 (0'0) promote 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by osd.305.0:28536 2014-08-11 17:05:25.641419 2014-08-13 17:38:03.080389 7fe9bae887a0 20 read_log 192888'12951 (192888'12950) modify 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.942868.0:2621 2014-08-11 17:05:25.633232 2014-08-13 17:38:03.080401 7fe9bae887a0 20 read_log 192888'12952 (0'0) promote 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.305.0:28544 2014-08-11 17:05:27.579611 2014-08-13 17:38:03.080413 7fe9bae887a0 20 read_log 192888'12953 (192888'12952) modify 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by client.942868.0:3145 2014-08-11 17:05:27.571718 2014-08-13 17:38:03.080426 7fe9bae887a0 20 read_log 192888'12954 (0'0) promote de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by osd.305.0:28548 2014-08-11 17:05:27.867341 2014-08-13 17:38:03.080438 7fe9bae887a0 20 read_log 192888'12955 (192888'12954) modify de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.942868.0:3223 2014-08-11 17:05:27.859526 2014-08-13 17:38:03.080450 7fe9bae887a0 20 read_log 192888'12956 (0'0) promote 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.305.0:28572 2014-08-11 17:05:31.644777 2014-08-13 17:38:03.080464 7fe9bae887a0 20 read_log 192888'12957 (192888'12956) modify 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by client.942868.0:4247 2014-08-11 17:05:31.638035 2014-08-13 17:38:03.080476 7fe9bae887a0 20 read_log 192888'12960 (0'0) promote ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.305.0:28610 2014-08-11 17:05:36.452783 2014-08-13 17:38:03.080489 7fe9bae887a0 20 read_log 192888'12961 (192888'12960) modify ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by client.942868.0:5489 2014-08-11 17:05:36.444042 2014-08-13 17:38:03.080508 7fe9bae887a0 20 read_log 192888'12964 (0'0) promote 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by osd.305.0:28630 2014-08-11 17:05:39.848129 2014-08-13 17:38:03.080521 7fe9bae887a0 20 read_log 192888'12965 (192888'12964) modify 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.942868.0:6423 2014-08-11 17:05:39.837896 2014-08-13 17:38:03.080533 7fe9bae887a0 20 read_log 192888'12968 (0'0) promote 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.305.0:28648 2014-08-11 17:05:41.501503 2014-08-13 17:38:03.080545 7fe9bae887a0 20 read_log 192888'12969 (192888'12968) modify 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by client.942868.0:6883 2014-08-11 17:05:41.494179 2014-08-13 17:38:03.080559 7fe9bae887a0 20 read_log 192888'12978 (0'0) promote 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by osd.305.0:28708 2014-08-11 17:05:49.071536 2014-08-13 17:38:03.080572 7fe9bae887a0 20 read_log 192888'12979 (192888'12978) modify 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.942868.0:8875 2014-08-11 17:05:49.063922 2014-08-13 17:38:03.080584 7fe9bae887a0 20 read_log 192888'12980 (0'0) promote 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by osd.305.0:28728 2014-08-11 17:05:52.369211 2014-08-13 17:38:03.080596 7fe9bae887a0 20 read_log 192888'12981 (192888'12980) modify 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.942868.0:9769 2014-08-11 17:05:52.361275 2014-08-13 17:38:03.080608 7fe9bae887a0 20 read_log 192888'12982 (0'0) promote 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by osd.305.0:28758 2014-08-11 17:05:55.808279 2014-08-13 17:38:03.080620 7fe9bae887a0 20 read_log 192888'12983 (192888'12982) modify 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.942868.0:10724 2014-08-11 17:05:55.800446 2014-08-13 17:38:03.080632 7fe9bae887a0 20 read_log 192888'12986 (0'0) promote a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.305.0:28794 2014-08-11 17:06:02.564670 2014-08-13 17:38:03.080644 7fe9bae887a0 20 read_log 192888'12987 (192888'12986) modify a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by client.942868.0:12575 2014-08-11 17:06:02.556515 2014-08-13 17:38:03.080656 7fe9bae887a0 20 read_log 192888'12990 (0'0) promote ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.305.0:28844 2014-08-11 17:06:07.336797 2014-08-13 17:38:03.080668 7fe9bae887a0 20 read_log 192888'12991 (192888'12990) modify ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by client.942868.0:13793 2014-08-11 17:06:07.329064 2014-08-13 17:38:03.080681 7fe9bae887a0 20 read_log 192888'12992 (0'0) promote 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by osd.305.0:28858 2014-08-11 17:06:08.995967 2014-08-13 17:38:03.080693 7fe9bae887a0 20 read_log 192888'12993 (192888'12992) modify 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.942868.0:14188 2014-08-11 17:06:08.988265 2014-08-13 17:38:03.080705 7fe9bae887a0 20 read_log 192888'12994 (0'0) promote c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by osd.305.0:28860 2014-08-11 17:06:09.812658 2014-08-13 17:38:03.080717 7fe9bae887a0 20 read_log 192888'12995 (192888'12994) modify c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.942868.0:14392 2014-08-11 17:06:09.804228 2014-08-13 17:38:03.080729 7fe9bae887a0 20 read_log 192888'12998 (0'0) promote fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by osd.305.0:28878 2014-08-11 17:06:12.501838 2014-08-13 17:38:03.080741 7fe9bae887a0 20 read_log 192888'12999 (192888'12998) modify fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.942868.0:15072 2014-08-11 17:06:12.493407 2014-08-13 17:38:03.080760 7fe9bae887a0 20 read_log 192888'13004 (0'0) promote 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by osd.305.0:28954 2014-08-11 17:06:28.152689 2014-08-13 17:38:03.080772 7fe9bae887a0 20 read_log 192888'13005 (192888'13004) modify 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.942868.0:19146 2014-08-11 17:06:28.145059 2014-08-13 17:38:03.080788 7fe9bae887a0 20 read_log 192888'13006 (0'0) promote 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by osd.305.0:28956 2014-08-11 17:06:28.815603 2014-08-13 17:38:03.080819 7fe9bae887a0 20 read_log 192888'13007 (192888'13006) modify 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.942868.0:19323 2014-08-11 17:06:28.807139 2014-08-13 17:38:03.080834 7fe9bae887a0 20 read_log 192888'13008 (0'0) promote 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by osd.305.0:28962 2014-08-11 17:06:29.431846 2014-08-13 17:38:03.080847 7fe9bae887a0 20 read_log 192888'13009 (192888'13008) modify 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.942868.0:19489 2014-08-11 17:06:29.424256 2014-08-13 17:38:03.080859 7fe9bae887a0 20 read_log 192888'13010 (0'0) promote e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.305.0:28966 2014-08-11 17:06:30.010752 2014-08-13 17:38:03.080890 7fe9bae887a0 20 read_log 192888'13011 (192888'13010) modify e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by client.942868.0:19643 2014-08-11 17:06:30.002215 2014-08-13 17:38:03.080903 7fe9bae887a0 20 read_log 192888'13012 (0'0) promote abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.305.0:28978 2014-08-11 17:06:31.909292 2014-08-13 17:38:03.080917 7fe9bae887a0 20 read_log 192888'13013 (192888'13012) modify abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by client.942868.0:20146 2014-08-11 17:06:31.900532 2014-08-13 17:38:03.080929 7fe9bae887a0 20 read_log 192888'13014 (0'0) promote 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by osd.305.0:28986 2014-08-11 17:06:32.844293 2014-08-13 17:38:03.080942 7fe9bae887a0 20 read_log 192888'13015 (192888'13014) modify 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.942868.0:20383 2014-08-11 17:06:32.834976 2014-08-13 17:38:03.080956 7fe9bae887a0 20 read_log 192888'13020 (0'0) promote 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by osd.305.0:29024 2014-08-11 17:06:38.200266 2014-08-13 17:38:03.080968 7fe9bae887a0 20 read_log 192888'13021 (192888'13020) modify 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.942868.0:21793 2014-08-11 17:06:38.193885 2014-08-13 17:38:03.080983 7fe9bae887a0 20 read_log 192888'13022 (0'0) promote ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by osd.305.0:29052 2014-08-11 17:06:43.449258 2014-08-13 17:38:03.080995 7fe9bae887a0 20 read_log 192888'13023 (192888'13022) modify ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.942868.0:23132 2014-08-11 17:06:43.441067 2014-08-13 17:38:03.081010 7fe9bae887a0 20 read_log 192888'13026 (0'0) promote 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by osd.305.0:29062 2014-08-11 17:06:47.049327 2014-08-13 17:38:03.081022 7fe9bae887a0 20 read_log 192888'13027 (192888'13026) modify 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.942868.0:24091 2014-08-11 17:06:47.039087 2014-08-13 17:38:03.081036 7fe9bae887a0 20 read_log 192888'13032 (0'0) promote 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by osd.305.0:29102 2014-08-11 17:06:51.734317 2014-08-13 17:38:03.081048 7fe9bae887a0 20 read_log 192888'13033 (192888'13032) modify 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.942868.0:25346 2014-08-11 17:06:51.721568 2014-08-13 17:38:03.081070 7fe9bae887a0 20 read_log 192888'13036 (0'0) promote df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by osd.305.0:29134 2014-08-11 17:06:58.324199 2014-08-13 17:38:03.081085 7fe9bae887a0 20 read_log 192888'13037 (192888'13036) modify df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.942868.0:27027 2014-08-11 17:06:58.316743 2014-08-13 17:38:03.081097 7fe9bae887a0 20 read_log 192888'13038 (0'0) promote 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by osd.305.0:29142 2014-08-11 17:06:59.884021 2014-08-13 17:38:03.081109 7fe9bae887a0 20 read_log 192888'13039 (192888'13038) modify 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.942868.0:27453 2014-08-11 17:06:59.876660 2014-08-13 17:38:03.081123 7fe9bae887a0 20 read_log 192888'13042 (0'0) promote cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by osd.305.0:29170 2014-08-11 17:07:05.443233 2014-08-13 17:38:03.081136 7fe9bae887a0 20 read_log 192888'13043 (192888'13042) modify cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.942868.0:28938 2014-08-11 17:07:05.434496 2014-08-13 17:38:03.081148 7fe9bae887a0 20 read_log 192888'13044 (0'0) promote 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by osd.305.0:29174 2014-08-11 17:07:05.477243 2014-08-13 17:38:03.081165 7fe9bae887a0 20 read_log 192888'13045 (192888'13044) modify 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.942868.0:28948 2014-08-11 17:07:05.470175 2014-08-13 17:38:03.081180 7fe9bae887a0 20 read_log 192888'13048 (0'0) promote 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by osd.305.0:29194 2014-08-11 17:07:09.144107 2014-08-13 17:38:03.081192 7fe9bae887a0 20 read_log 192888'13049 (192888'13048) modify 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.942868.0:29835 2014-08-11 17:07:09.134805 2014-08-13 17:38:03.081204 7fe9bae887a0 20 read_log 192888'13050 (0'0) promote 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by osd.305.0:29198 2014-08-11 17:07:09.194766 2014-08-13 17:38:03.081219 7fe9bae887a0 20 read_log 192888'13051 (192888'13050) modify 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.942868.0:29848 2014-08-11 17:07:09.186436 2014-08-13 17:38:03.081231 7fe9bae887a0 20 read_log 192888'13052 (0'0) promote 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.305.0:29202 2014-08-11 17:07:10.206619 2014-08-13 17:38:03.081245 7fe9bae887a0 20 read_log 192888'13053 (192888'13052) modify 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by client.942868.0:30096 2014-08-11 17:07:10.197905 2014-08-13 17:38:03.081258 7fe9bae887a0 20 read_log 192888'13068 (192888'12951) modify 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.942868.0:33887 2014-08-11 17:07:16.639184 2014-08-13 17:38:03.081272 7fe9bae887a0 20 read_log 192888'13069 (192888'12953) modify 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by client.942868.0:34411 2014-08-11 17:07:16.780699 2014-08-13 17:38:03.081285 7fe9bae887a0 20 read_log 192888'13070 (192888'12955) modify de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.942868.0:34489 2014-08-11 17:07:16.803582 2014-08-13 17:38:03.081299 7fe9bae887a0 20 read_log 192888'13071 (192888'12957) modify 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by client.942868.0:35513 2014-08-11 17:07:17.092353 2014-08-13 17:38:03.081311 7fe9bae887a0 20 read_log 192888'13073 (192888'12961) modify ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by client.942868.0:36755 2014-08-11 17:07:17.425832 2014-08-13 17:38:03.081323 7fe9bae887a0 20 read_log 192888'13075 (192888'12965) modify 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.942868.0:37689 2014-08-11 17:07:17.680174 2014-08-13 17:38:03.081360 7fe9bae887a0 20 read_log 192888'13077 (192888'12969) modify 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by client.942868.0:38149 2014-08-11 17:07:17.807532 2014-08-13 17:38:03.081376 7fe9bae887a0 20 read_log 192888'13082 (192888'12979) modify 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.942868.0:40141 2014-08-11 17:07:18.374477 2014-08-13 17:38:03.081388 7fe9bae887a0 20 read_log 192888'13083 (192888'12981) modify 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.942868.0:41035 2014-08-11 17:07:18.616035 2014-08-13 17:38:03.081400 7fe9bae887a0 20 read_log 192888'13084 (192888'12983) modify 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.942868.0:41990 2014-08-11 17:07:18.875637 2014-08-13 17:38:03.081413 7fe9bae887a0 20 read_log 192888'13086 (192888'12987) modify a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by client.942868.0:43841 2014-08-11 17:07:19.391794 2014-08-13 17:38:03.081425 7fe9bae887a0 20 read_log 192888'13088 (192888'12991) modify ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by client.942868.0:45059 2014-08-11 17:07:19.735426 2014-08-13 17:38:03.081437 7fe9bae887a0 20 read_log 192888'13089 (192888'12993) modify 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.942868.0:45454 2014-08-11 17:07:19.841867 2014-08-13 17:38:03.081449 7fe9bae887a0 20 read_log 192888'13090 (192888'12995) modify c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.942868.0:45658 2014-08-11 17:07:19.898216 2014-08-13 17:38:03.081462 7fe9bae887a0 20 read_log 192888'13092 (192888'12999) modify fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.942868.0:46338 2014-08-11 17:07:20.093354 2014-08-13 17:38:03.081477 7fe9bae887a0 20 read_log 192888'13095 (192888'13005) modify 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.942868.0:50412 2014-08-11 17:07:21.312190 2014-08-13 17:38:03.081490 7fe9bae887a0 20 read_log 192888'13096 (192888'13007) modify 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.942868.0:50589 2014-08-11 17:07:21.371077 2014-08-13 17:38:03.081502 7fe9bae887a0 20 read_log 192888'13097 (192888'13009) modify 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.942868.0:50755 2014-08-11 17:07:21.421740 2014-08-13 17:38:03.081514 7fe9bae887a0 20 read_log 192888'13098 (192888'13011) modify e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by client.942868.0:50909 2014-08-11 17:07:21.469579 2014-08-13 17:38:03.081526 7fe9bae887a0 20 read_log 192888'13099 (192888'13013) modify abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by client.942868.0:51412 2014-08-11 17:07:21.641750 2014-08-13 17:38:03.081538 7fe9bae887a0 20 read_log 192888'13100 (192888'13015) modify 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.942868.0:51649 2014-08-11 17:07:21.709709 2014-08-13 17:38:03.081550 7fe9bae887a0 20 read_log 192888'13103 (192888'13021) modify 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.942868.0:53059 2014-08-11 17:07:22.136174 2014-08-13 17:38:03.081562 7fe9bae887a0 20 read_log 192888'13104 (192888'13023) modify ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.942868.0:54398 2014-08-11 17:07:22.546559 2014-08-13 17:38:03.081574 7fe9bae887a0 20 read_log 192888'13106 (192888'13027) modify 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.942868.0:55357 2014-08-11 17:07:22.823264 2014-08-13 17:38:03.081586 7fe9bae887a0 20 read_log 192888'13109 (192888'13033) modify 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.942868.0:56612 2014-08-11 17:07:23.206976 2014-08-13 17:38:03.081599 7fe9bae887a0 20 read_log 192888'13111 (192888'13037) modify df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.942868.0:58293 2014-08-11 17:07:23.697664 2014-08-13 17:38:03.081618 7fe9bae887a0 20 read_log 192888'13112 (192888'13039) modify 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.942868.0:58719 2014-08-11 17:07:23.831699 2014-08-13 17:38:03.081631 7fe9bae887a0 20 read_log 192888'13114 (192888'13043) modify cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.942868.0:60204 2014-08-11 17:07:24.269977 2014-08-13 17:38:03.081643 7fe9bae887a0 20 read_log 192888'13115 (192888'13045) modify 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.942868.0:60214 2014-08-11 17:07:24.273250 2014-08-13 17:38:03.081655 7fe9bae887a0 20 read_log 192888'13117 (192888'13049) modify 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.942868.0:61101 2014-08-11 17:07:24.541956 2014-08-13 17:38:03.081667 7fe9bae887a0 20 read_log 192888'13118 (192888'13051) modify 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.942868.0:61114 2014-08-11 17:07:24.545268 2014-08-13 17:38:03.081679 7fe9bae887a0 20 read_log 192888'13119 (192888'13053) modify 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by client.942868.0:61362 2014-08-11 17:07:24.618109 2014-08-13 17:38:03.081692 7fe9bae887a0 20 read_log 193305'13124 (0'0) promote ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by osd.45.0:29 2014-08-11 18:42:38.303434 2014-08-13 17:38:03.081704 7fe9bae887a0 20 read_log 193305'13125 (193305'13124) modify ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.951337.0:1366 2014-08-11 18:42:38.293149 2014-08-13 17:38:03.081716 7fe9bae887a0 20 read_log 193305'13126 (0'0) promote b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by osd.45.0:31 2014-08-11 18:42:38.310384 2014-08-13 17:38:03.081728 7fe9bae887a0 20 read_log 193305'13127 (193305'13126) modify b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.951337.0:1368 2014-08-11 18:42:38.300363 2014-08-13 17:38:03.081740 7fe9bae887a0 20 read_log 193305'13128 (0'0) promote fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:39 2014-08-11 18:42:40.089141 2014-08-13 17:38:03.081752 7fe9bae887a0 20 read_log 193305'13129 (193305'13128) modify fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by client.951337.0:1820 2014-08-11 18:42:40.081185 2014-08-13 17:38:03.081765 7fe9bae887a0 20 read_log 193305'13130 (0'0) promote a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by osd.45.0:88 2014-08-11 18:42:50.487939 2014-08-13 17:38:03.081777 7fe9bae887a0 20 read_log 193305'13131 (193305'13130) modify a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.951337.0:4664 2014-08-11 18:42:50.480683 2014-08-13 17:38:03.081789 7fe9bae887a0 20 read_log 193305'13132 (0'0) promote c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:92 2014-08-11 18:42:51.290705 2014-08-13 17:38:03.081801 7fe9bae887a0 20 read_log 193305'13133 (193305'13132) modify c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by client.951337.0:4879 2014-08-11 18:42:51.283775 2014-08-13 17:38:03.081816 7fe9bae887a0 20 read_log 193305'13134 (0'0) promote f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by osd.45.0:132 2014-08-11 18:42:58.446992 2014-08-13 17:38:03.081828 7fe9bae887a0 20 read_log 193305'13135 (193305'13134) modify f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.951337.0:6834 2014-08-11 18:42:58.437393 2014-08-13 17:38:03.081843 7fe9bae887a0 20 read_log 193305'13136 (0'0) promote db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by osd.45.0:162 2014-08-11 18:43:07.189900 2014-08-13 17:38:03.081855 7fe9bae887a0 20 read_log 193305'13137 (193305'13136) modify db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.951337.0:9153 2014-08-11 18:43:07.179205 2014-08-13 17:38:03.081877 7fe9bae887a0 20 read_log 193305'13138 (0'0) promote e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:192 2014-08-11 18:43:16.461963 2014-08-13 17:38:03.081892 7fe9bae887a0 20 read_log 193305'13139 (193305'13138) modify e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by client.951337.0:11610 2014-08-11 18:43:16.454333 2014-08-13 17:38:03.081905 7fe9bae887a0 20 read_log 193305'13140 (0'0) promote c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by osd.45.0:196 2014-08-11 18:43:17.236684 2014-08-13 17:38:03.081919 7fe9bae887a0 20 read_log 193305'13141 (193305'13140) modify c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.951337.0:11818 2014-08-11 18:43:17.228946 2014-08-13 17:38:03.081933 7fe9bae887a0 20 read_log 193305'13142 (0'0) promote de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by osd.45.0:200 2014-08-11 18:43:20.733203 2014-08-13 17:38:03.081948 7fe9bae887a0 20 read_log 193305'13143 (193305'13142) modify de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.951337.0:12758 2014-08-11 18:43:20.726013 2014-08-13 17:38:03.081960 7fe9bae887a0 20 read_log 193305'13144 (193305'13144) modify 369/hit_set_24.369_archive_2014-08-11 18:38:30.736751_2014-08-11 18:43:33.856980/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:33.857214 2014-08-13 17:38:03.081975 7fe9bae887a0 20 read_log 193305'13145 (0'0) promote e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by osd.45.0:257 2014-08-11 18:43:33.860077 2014-08-13 17:38:03.081987 7fe9bae887a0 20 read_log 193305'13146 (193305'13145) modify e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.951337.0:16402 2014-08-11 18:43:33.849577 2014-08-13 17:38:03.082001 7fe9bae887a0 20 read_log 193305'13147 (0'0) promote 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by osd.45.0:259 2014-08-11 18:43:34.233886 2014-08-13 17:38:03.082014 7fe9bae887a0 20 read_log 193305'13148 (193305'13147) modify 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.951337.0:16512 2014-08-11 18:43:34.225802 2014-08-13 17:38:03.082028 7fe9bae887a0 20 read_log 193305'13149 (0'0) promote b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by osd.45.0:279 2014-08-11 18:43:37.018257 2014-08-13 17:38:03.082040 7fe9bae887a0 20 read_log 193305'13150 (193305'13149) modify b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.951337.0:17312 2014-08-11 18:43:37.008815 2014-08-13 17:38:03.082054 7fe9bae887a0 20 read_log 193305'13151 (0'0) promote bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by osd.45.0:311 2014-08-11 18:43:44.335616 2014-08-13 17:38:03.082066 7fe9bae887a0 20 read_log 193305'13152 (193305'13151) modify bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.951337.0:19349 2014-08-11 18:43:44.327489 2014-08-13 17:38:03.082079 7fe9bae887a0 20 read_log 193305'13153 (0'0) promote 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by osd.45.0:319 2014-08-11 18:43:46.882141 2014-08-13 17:38:03.082111 7fe9bae887a0 20 read_log 193305'13154 (193305'13153) modify 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.951337.0:19908 2014-08-11 18:43:46.270297 2014-08-13 17:38:03.082127 7fe9bae887a0 20 read_log 193305'13155 (0'0) promote abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:385 2014-08-11 18:44:00.112093 2014-08-13 17:38:03.082139 7fe9bae887a0 20 read_log 193305'13156 (193305'13155) modify abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by client.951337.0:23750 2014-08-11 18:44:00.104709 2014-08-13 17:38:03.082163 7fe9bae887a0 20 read_log 193305'13157 (0'0) promote cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by osd.45.0:391 2014-08-11 18:44:00.682529 2014-08-13 17:38:03.082179 7fe9bae887a0 20 read_log 193305'13158 (193305'13157) modify cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.951337.0:23906 2014-08-11 18:44:00.673963 2014-08-13 17:38:03.082192 7fe9bae887a0 20 read_log 193305'13159 (0'0) promote c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by osd.45.0:431 2014-08-11 18:44:10.643085 2014-08-13 17:38:03.082204 7fe9bae887a0 20 read_log 193305'13160 (193305'13159) modify c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.951337.0:26624 2014-08-11 18:44:10.634300 2014-08-13 17:38:03.082216 7fe9bae887a0 20 read_log 193305'13161 (0'0) promote 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:433 2014-08-11 18:44:10.921470 2014-08-13 17:38:03.082228 7fe9bae887a0 20 read_log 193305'13162 (193305'13161) modify 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by client.951337.0:26702 2014-08-11 18:44:10.910431 2014-08-13 17:38:03.082242 7fe9bae887a0 20 read_log 193305'13163 (0'0) promote d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by osd.45.0:460 2014-08-11 18:44:17.930311 2014-08-13 17:38:03.082254 7fe9bae887a0 20 read_log 193305'13164 (193305'13163) modify d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.951337.0:28625 2014-08-11 18:44:17.921716 2014-08-13 17:38:03.082267 7fe9bae887a0 20 read_log 193305'13165 (0'0) promote 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by osd.45.0:470 2014-08-11 18:44:19.412934 2014-08-13 17:38:03.082281 7fe9bae887a0 20 read_log 193305'13166 (193305'13165) modify 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.951337.0:29019 2014-08-11 18:44:19.404723 2014-08-13 17:38:03.082293 7fe9bae887a0 20 read_log 193305'13167 (0'0) promote c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by osd.45.0:474 2014-08-11 18:44:20.057795 2014-08-13 17:38:03.082305 7fe9bae887a0 20 read_log 193305'13168 (193305'13167) modify c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.951337.0:29197 2014-08-11 18:44:20.050376 2014-08-13 17:38:03.082317 7fe9bae887a0 20 read_log 193305'13169 (0'0) promote a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:520 2014-08-11 18:44:29.234075 2014-08-13 17:38:03.082329 7fe9bae887a0 20 read_log 193305'13170 (193305'13169) modify a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by client.951337.0:31693 2014-08-11 18:44:29.223561 2014-08-13 17:38:03.082341 7fe9bae887a0 20 read_log 193305'13171 (0'0) promote 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:532 2014-08-11 18:44:30.504922 2014-08-13 17:38:03.082353 7fe9bae887a0 20 read_log 193305'13172 (193305'13171) modify 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by client.951337.0:32068 2014-08-11 18:44:30.495713 2014-08-13 17:38:03.082365 7fe9bae887a0 20 read_log 193305'13173 (0'0) promote 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by osd.45.0:536 2014-08-11 18:44:30.819469 2014-08-13 17:38:03.082377 7fe9bae887a0 20 read_log 193305'13174 (193305'13173) modify 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.951337.0:32160 2014-08-11 18:44:30.808668 2014-08-13 17:38:03.082389 7fe9bae887a0 20 read_log 193305'13175 (0'0) promote 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by osd.45.0:544 2014-08-11 18:44:32.542139 2014-08-13 17:38:03.082401 7fe9bae887a0 20 read_log 193305'13176 (193305'13175) modify 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.951337.0:32631 2014-08-11 18:44:32.534292 2014-08-13 17:38:03.082421 7fe9bae887a0 20 read_log 193305'13177 (193305'13125) modify ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.951337.0:34077 2014-08-11 18:44:34.785894 2014-08-13 17:38:03.082434 7fe9bae887a0 20 read_log 193305'13178 (193305'13127) modify b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.951337.0:34079 2014-08-11 18:44:34.786537 2014-08-13 17:38:03.082446 7fe9bae887a0 20 read_log 193305'13179 (193305'13129) modify fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by client.951337.0:34531 2014-08-11 18:44:35.056791 2014-08-13 17:38:03.082458 7fe9bae887a0 20 read_log 193305'13180 (193305'13131) modify a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.951337.0:37375 2014-08-11 18:44:36.873993 2014-08-13 17:38:03.082470 7fe9bae887a0 20 read_log 193305'13181 (193305'13133) modify c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by client.951337.0:37590 2014-08-11 18:44:36.985643 2014-08-13 17:38:03.082482 7fe9bae887a0 20 read_log 193305'13182 (193305'13135) modify f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.951337.0:39545 2014-08-11 18:44:38.067003 2014-08-13 17:38:03.082494 7fe9bae887a0 20 read_log 193305'13183 (193305'13137) modify db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.951337.0:41864 2014-08-11 18:44:38.900327 2014-08-13 17:38:03.082508 7fe9bae887a0 20 read_log 193305'13184 (193305'13139) modify e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by client.951337.0:44321 2014-08-11 18:44:39.852111 2014-08-13 17:38:03.082521 7fe9bae887a0 20 read_log 193305'13185 (193305'13141) modify c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.951337.0:44529 2014-08-11 18:44:39.904970 2014-08-13 17:38:03.082533 7fe9bae887a0 20 read_log 193305'13186 (193305'13143) modify de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.951337.0:45469 2014-08-11 18:44:40.209935 2014-08-13 17:38:03.082545 7fe9bae887a0 20 read_log 193305'13187 (193305'13146) modify e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.951337.0:49113 2014-08-11 18:44:41.257210 2014-08-13 17:38:03.082558 7fe9bae887a0 20 read_log 193305'13188 (193305'13148) modify 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.951337.0:49223 2014-08-11 18:44:41.298646 2014-08-13 17:38:03.082570 7fe9bae887a0 20 read_log 193305'13189 (193305'13150) modify b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.951337.0:50023 2014-08-11 18:44:41.543018 2014-08-13 17:38:03.082582 7fe9bae887a0 20 read_log 193305'13190 (193305'13152) modify bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.951337.0:52060 2014-08-11 18:44:42.123016 2014-08-13 17:38:03.082594 7fe9bae887a0 20 read_log 193305'13191 (193305'13154) modify 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.951337.0:52619 2014-08-11 18:44:42.285745 2014-08-13 17:38:03.082606 7fe9bae887a0 20 read_log 193305'13192 (193305'13156) modify abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by client.951337.0:56461 2014-08-11 18:44:43.425941 2014-08-13 17:38:03.082618 7fe9bae887a0 20 read_log 193305'13193 (193305'13158) modify cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.951337.0:56617 2014-08-11 18:44:43.471577 2014-08-13 17:38:03.082630 7fe9bae887a0 20 read_log 193305'13194 (193305'13160) modify c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.951337.0:59335 2014-08-11 18:44:44.298773 2014-08-13 17:38:03.082642 7fe9bae887a0 20 read_log 193305'13195 (193305'13162) modify 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by client.951337.0:59413 2014-08-11 18:44:44.320332 2014-08-13 17:38:03.082654 7fe9bae887a0 20 read_log 193305'13196 (193305'13164) modify d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.951337.0:61336 2014-08-11 18:44:44.893215 2014-08-13 17:38:03.082676 7fe9bae887a0 20 read_log 193305'13197 (193305'13166) modify 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.951337.0:61730 2014-08-11 18:44:45.012543 2014-08-13 17:38:03.082689 7fe9bae887a0 20 read_log 193305'13198 (193305'13168) modify c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.951337.0:61908 2014-08-11 18:44:45.066822 2014-08-13 17:38:03.082701 7fe9bae887a0 20 read_log 193305'13199 (193305'13170) modify a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by client.951337.0:64404 2014-08-11 18:44:45.796744 2014-08-13 17:38:03.082714 7fe9bae887a0 20 read_log 193305'13200 (193305'13172) modify 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by client.951337.0:64779 2014-08-11 18:44:45.915396 2014-08-13 17:38:03.082726 7fe9bae887a0 20 read_log 193305'13201 (193305'13174) modify 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.951337.0:64871 2014-08-11 18:44:45.944862 2014-08-13 17:38:03.082738 7fe9bae887a0 20 read_log 193305'13202 (193305'13176) modify 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.951337.0:65342 2014-08-11 18:44:46.095252 2014-08-13 17:38:03.082750 7fe9bae887a0 20 read_log 194209'13203 (192888'12902) clean f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.45.0:1101 2014-08-12 14:06:05.766946 2014-08-13 17:38:03.082764 7fe9bae887a0 20 read_log 194209'13204 (192834'12509) clean d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.45.0:1102 2014-08-12 14:06:05.767344 2014-08-13 17:38:03.082779 7fe9bae887a0 20 read_log 194209'13205 (192888'12918) clean 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.45.0:1103 2014-08-12 14:06:05.767711 2014-08-13 17:38:03.082794 7fe9bae887a0 20 read_log 194209'13206 (192834'11756) clean 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.45.0:1104 2014-08-12 14:06:05.768422 2014-08-13 17:38:03.082807 7fe9bae887a0 20 read_log 194209'13207 (193305'13179) clean fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:1105 2014-08-12 14:06:05.817748 2014-08-13 17:38:03.082821 7fe9bae887a0 20 read_log 194209'13208 (192888'13073) clean ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.45.0:1106 2014-08-12 14:06:05.821198 2014-08-13 17:38:03.082833 7fe9bae887a0 20 read_log 194209'13209 (192888'13071) clean 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.45.0:1107 2014-08-12 14:06:05.824612 2014-08-13 17:38:03.082847 7fe9bae887a0 20 read_log 194209'13210 (192834'12675) clean 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.45.0:1108 2014-08-12 14:06:05.828241 2014-08-13 17:38:03.082859 7fe9bae887a0 20 read_log 194209'13211 (192834'12726) clean 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.45.0:1109 2014-08-12 14:06:05.831600 2014-08-13 17:38:03.082874 7fe9bae887a0 20 read_log 194209'13212 (192888'12887) clean d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.45.0:1110 2014-08-12 14:06:05.834856 2014-08-13 17:38:03.082886 7fe9bae887a0 20 read_log 194209'13213 (193305'13199) clean a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:1111 2014-08-12 14:06:05.841121 2014-08-13 17:38:03.082900 7fe9bae887a0 20 read_log 194209'13214 (192834'12653) clean dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.45.0:1112 2014-08-12 14:06:05.841414 2014-08-13 17:38:03.082912 7fe9bae887a0 20 read_log 194209'13215 (192888'12897) clean 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.45.0:1113 2014-08-12 14:06:05.844630 2014-08-13 17:38:03.082932 7fe9bae887a0 20 read_log 194209'13216 (192888'12894) clean 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.45.0:1114 2014-08-12 14:06:05.850834 2014-08-13 17:38:03.082946 7fe9bae887a0 20 read_log 194209'13217 (192834'12571) clean fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.45.0:1115 2014-08-12 14:06:05.851145 2014-08-13 17:38:03.082959 7fe9bae887a0 20 read_log 194209'13218 (192888'13077) clean 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.45.0:1116 2014-08-12 14:06:05.854494 2014-08-13 17:38:03.082971 7fe9bae887a0 20 read_log 194209'13219 (192888'12905) clean 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.45.0:1117 2014-08-12 14:06:05.858425 2014-08-13 17:38:03.082985 7fe9bae887a0 20 read_log 194209'13220 (192834'12704) clean bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.45.0:1118 2014-08-12 14:06:05.861672 2014-08-13 17:38:03.082997 7fe9bae887a0 20 read_log 194209'13221 (193305'13200) clean 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:1119 2014-08-12 14:06:05.865020 2014-08-13 17:38:03.083032 7fe9bae887a0 20 read_log 194209'13222 (192888'12874) clean f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.45.0:1120 2014-08-12 14:06:05.868389 2014-08-13 17:38:03.083045 7fe9bae887a0 20 read_log 194209'13223 (192888'13088) clean ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.45.0:1121 2014-08-12 14:06:05.871609 2014-08-13 17:38:03.083059 7fe9bae887a0 20 read_log 194209'13224 (192888'12911) clean f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.45.0:1122 2014-08-12 14:06:05.874907 2014-08-13 17:38:03.083071 7fe9bae887a0 20 read_log 194209'13225 (192834'12684) clean 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.45.0:1123 2014-08-12 14:06:05.878215 2014-08-13 17:38:03.083085 7fe9bae887a0 20 read_log 194209'13226 (192888'12901) clean 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.45.0:1124 2014-08-12 14:06:05.881476 2014-08-13 17:38:03.083097 7fe9bae887a0 20 read_log 194209'13227 (192888'13086) clean a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.45.0:1125 2014-08-12 14:06:05.884726 2014-08-13 17:38:03.083113 7fe9bae887a0 20 read_log 194209'13228 (193305'13184) clean e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:1126 2014-08-12 14:06:05.887956 2014-08-13 17:38:03.083126 7fe9bae887a0 20 read_log 194209'13229 (192888'13119) clean 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.45.0:1127 2014-08-12 14:06:05.891093 2014-08-13 17:38:03.083140 7fe9bae887a0 20 read_log 194209'13230 (192888'13099) clean abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.45.0:1128 2014-08-12 14:06:05.894347 2014-08-13 17:38:03.083152 7fe9bae887a0 20 read_log 194209'13231 (193305'13195) clean 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:1129 2014-08-12 14:06:05.900345 2014-08-13 17:38:03.083171 7fe9bae887a0 20 read_log 194209'13232 (192834'12504) clean eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.45.0:1130 2014-08-12 14:06:05.903592 2014-08-13 17:38:03.083184 7fe9bae887a0 20 read_log 194209'13233 (192888'12922) clean a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.45.0:1131 2014-08-12 14:06:05.903878 2014-08-13 17:38:03.083199 7fe9bae887a0 20 read_log 194209'13234 (192888'12899) clean f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.45.0:1132 2014-08-12 14:06:05.907079 2014-08-13 17:38:03.083211 7fe9bae887a0 20 read_log 194209'13235 (193305'13192) clean abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:1133 2014-08-12 14:06:05.910529 2014-08-13 17:38:03.083246 7fe9bae887a0 20 read_log 194209'13236 (192834'12663) clean c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.45.0:1134 2014-08-12 14:06:05.913839 2014-08-13 17:38:03.083261 7fe9bae887a0 20 read_log 194209'13237 (192888'12896) clean c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.45.0:1135 2014-08-12 14:06:05.917246 2014-08-13 17:38:03.083274 7fe9bae887a0 20 read_log 194209'13238 (192834'12721) clean f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.45.0:1136 2014-08-12 14:06:05.920334 2014-08-13 17:38:03.083286 7fe9bae887a0 20 read_log 194209'13239 (193305'13181) clean c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:1137 2014-08-12 14:06:05.923692 2014-08-13 17:38:03.083300 7fe9bae887a0 20 read_log 194209'13240 (192888'12879) clean 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.45.0:1138 2014-08-12 14:06:05.927071 2014-08-13 17:38:03.083312 7fe9bae887a0 20 read_log 194209'13241 (192888'12903) clean d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.45.0:1139 2014-08-12 14:06:05.930619 2014-08-13 17:38:03.083324 7fe9bae887a0 20 read_log 194209'13242 (192888'13100) clean 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by osd.45.0:1140 2014-08-12 14:06:05.933809 2014-08-13 17:38:03.083336 7fe9bae887a0 20 read_log 194209'13243 (192888'13098) clean e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.45.0:1141 2014-08-12 14:06:05.937278 2014-08-13 17:38:03.083348 7fe9bae887a0 20 read_log 194209'13244 (192888'13069) clean 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.45.0:1142 2014-08-12 14:06:05.940690 2014-08-13 17:38:03.083360 7fe9bae887a0 20 read_log 194209'13245 (194209'13203) delete f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.45.0:1143 2014-08-12 14:06:05.974803 2014-08-13 17:38:03.083372 7fe9bae887a0 20 read_log 194209'13246 (194209'13204) delete d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.45.0:1144 2014-08-12 14:06:05.977478 2014-08-13 17:38:03.083402 7fe9bae887a0 20 read_log 194209'13247 (194209'13205) delete 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.45.0:1145 2014-08-12 14:06:05.980352 2014-08-13 17:38:03.083417 7fe9bae887a0 20 read_log 194209'13248 (194209'13206) delete 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.45.0:1146 2014-08-12 14:06:05.984148 2014-08-13 17:38:03.083429 7fe9bae887a0 20 read_log 194209'13249 (194209'13207) delete fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:1147 2014-08-12 14:06:05.987807 2014-08-13 17:38:03.083441 7fe9bae887a0 20 read_log 194209'13250 (194209'13214) delete dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.45.0:1148 2014-08-12 14:06:05.992042 2014-08-13 17:38:03.083454 7fe9bae887a0 20 read_log 194209'13251 (194209'13213) delete a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:1149 2014-08-12 14:06:05.995483 2014-08-13 17:38:03.083466 7fe9bae887a0 20 read_log 194209'13252 (194209'13221) delete 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:1150 2014-08-12 14:06:05.999430 2014-08-13 17:38:03.083478 7fe9bae887a0 20 read_log 194209'13253 (194209'13228) delete e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:1151 2014-08-12 14:06:06.003103 2014-08-13 17:38:03.083490 7fe9bae887a0 20 read_log 194209'13254 (194209'13231) delete 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:1152 2014-08-12 14:06:06.006492 2014-08-13 17:38:03.083502 7fe9bae887a0 20 read_log 194209'13255 (194209'13235) delete abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:1153 2014-08-12 14:06:06.009732 2014-08-13 17:38:03.083523 7fe9bae887a0 20 read_log 194209'13256 (194209'13239) delete c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:1154 2014-08-12 14:06:06.012940 2014-08-13 17:38:03.083536 7fe9bae887a0 20 read_log 194209'13257 (194209'13243) delete e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.45.0:1155 2014-08-12 14:06:06.048493 2014-08-13 17:38:03.083548 7fe9bae887a0 20 read_log 194209'13258 (194209'13244) delete 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.45.0:1156 2014-08-12 14:06:06.051242 2014-08-13 17:38:03.083560 7fe9bae887a0 20 read_log 194209'13259 (194209'13208) delete ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.45.0:1157 2014-08-12 14:06:06.082493 2014-08-13 17:38:03.083573 7fe9bae887a0 20 read_log 194209'13260 (194209'13209) delete 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.45.0:1158 2014-08-12 14:06:06.085637 2014-08-13 17:38:03.083585 7fe9bae887a0 20 read_log 194209'13261 (194209'13210) delete 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.45.0:1159 2014-08-12 14:06:06.088717 2014-08-13 17:38:03.083597 7fe9bae887a0 20 read_log 194209'13262 (194209'13211) delete 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.45.0:1160 2014-08-12 14:06:06.091891 2014-08-13 17:38:03.083609 7fe9bae887a0 20 read_log 194209'13263 (194209'13212) delete d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.45.0:1161 2014-08-12 14:06:06.095090 2014-08-13 17:38:03.083621 7fe9bae887a0 20 read_log 194209'13264 (194209'13215) delete 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.45.0:1162 2014-08-12 14:06:06.098340 2014-08-13 17:38:03.083633 7fe9bae887a0 20 read_log 194209'13265 (194209'13217) delete fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.45.0:1163 2014-08-12 14:06:06.101657 2014-08-13 17:38:03.083645 7fe9bae887a0 20 read_log 194209'13266 (194209'13216) delete 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.45.0:1164 2014-08-12 14:06:06.104816 2014-08-13 17:38:03.083657 7fe9bae887a0 20 read_log 194209'13267 (194209'13218) delete 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.45.0:1165 2014-08-12 14:06:06.108292 2014-08-13 17:38:03.083669 7fe9bae887a0 20 read_log 194209'13268 (194209'13219) delete 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.45.0:1166 2014-08-12 14:06:06.111452 2014-08-13 17:38:03.083681 7fe9bae887a0 20 read_log 194209'13269 (194209'13220) delete bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.45.0:1167 2014-08-12 14:06:06.114655 2014-08-13 17:38:03.083693 7fe9bae887a0 20 read_log 194209'13270 (194209'13222) delete f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.45.0:1168 2014-08-12 14:06:06.117788 2014-08-13 17:38:03.083705 7fe9bae887a0 20 read_log 194209'13271 (194209'13223) delete ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.45.0:1169 2014-08-12 14:06:06.120823 2014-08-13 17:38:03.083717 7fe9bae887a0 20 read_log 194209'13272 (194209'13224) delete f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.45.0:1170 2014-08-12 14:06:06.123789 2014-08-13 17:38:03.083729 7fe9bae887a0 20 read_log 194209'13273 (194209'13225) delete 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.45.0:1171 2014-08-12 14:06:06.126737 2014-08-13 17:38:03.083740 7fe9bae887a0 20 read_log 194209'13274 (194209'13226) delete 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.45.0:1172 2014-08-12 14:06:06.129717 2014-08-13 17:38:03.083752 7fe9bae887a0 20 read_log 194209'13275 (194209'13227) delete a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.45.0:1173 2014-08-12 14:06:06.132708 2014-08-13 17:38:03.083772 7fe9bae887a0 20 read_log 194209'13276 (194209'13229) delete 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.45.0:1174 2014-08-12 14:06:06.135732 2014-08-13 17:38:03.083792 7fe9bae887a0 20 read_log 194209'13277 (194209'13230) delete abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.45.0:1175 2014-08-12 14:06:06.138775 2014-08-13 17:38:03.083804 7fe9bae887a0 20 read_log 194209'13278 (194209'13232) delete eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.45.0:1176 2014-08-12 14:06:06.141686 2014-08-13 17:38:03.083816 7fe9bae887a0 20 read_log 194209'13279 (194209'13233) delete a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.45.0:1177 2014-08-12 14:06:06.144572 2014-08-13 17:38:03.083829 7fe9bae887a0 20 read_log 194209'13280 (194209'13234) delete f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.45.0:1178 2014-08-12 14:06:06.147522 2014-08-13 17:38:03.083841 7fe9bae887a0 20 read_log 194209'13281 (194209'13236) delete c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.45.0:1179 2014-08-12 14:06:06.150493 2014-08-13 17:38:03.083853 7fe9bae887a0 20 read_log 194209'13282 (194209'13237) delete c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.45.0:1180 2014-08-12 14:06:06.153463 2014-08-13 17:38:03.083865 7fe9bae887a0 20 read_log 194209'13283 (194209'13238) delete f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.45.0:1181 2014-08-12 14:06:06.156586 2014-08-13 17:38:03.083877 7fe9bae887a0 20 read_log 194209'13284 (194209'13240) delete 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.45.0:1182 2014-08-12 14:06:06.159413 2014-08-13 17:38:03.083889 7fe9bae887a0 20 read_log 194209'13285 (194209'13241) delete d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.45.0:1183 2014-08-12 14:06:06.162395 2014-08-13 17:38:03.083901 7fe9bae887a0 20 read_log 194209'13286 (192888'13084) clean 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by osd.45.0:1184 2014-08-12 14:06:06.162739 2014-08-13 17:38:03.083914 7fe9bae887a0 20 read_log 194215'13287 (194215'13287) modify 369/hit_set_24.369_archive_2014-08-12 15:32:25.738576_2014-08-12 15:49:40.152882/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:40.153160 2014-08-13 17:38:03.083926 7fe9bae887a0 20 read_log 194215'13288 (193305'13144) delete 369/hit_set_24.369_archive_2014-08-11 18:38:30.736751_2014-08-11 18:43:33.856980/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:40.153160 2014-08-13 17:38:03.083938 7fe9bae887a0 20 read_log 194215'13289 (194209'13242) delete 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.2174595.0:180166 0.000000 2014-08-13 17:38:03.083949 7fe9bae887a0 20 read_log 194215'13290 (194209'13286) delete 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.2174595.0:180168 0.000000 2014-08-13 17:38:03.083960 7fe9bae887a0 20 read_log 194215'13291 (193305'13183) clean db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by osd.45.0:2955 2014-08-12 15:49:40.170168 2014-08-13 17:38:03.083972 7fe9bae887a0 20 read_log 194215'13292 (194215'13291) delete db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.2174595.0:180170 0.000000 2014-08-13 17:38:03.083983 7fe9bae887a0 20 read_log 194215'13293 (192888'13097) clean 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by osd.45.0:2957 2014-08-12 15:49:40.178164 2014-08-13 17:38:03.083995 7fe9bae887a0 20 read_log 194215'13294 (194215'13293) delete 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.2174595.0:180172 0.000000 2014-08-13 17:38:03.084013 7fe9bae887a0 20 read_log 194215'13295 (192888'12917) clean c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by osd.45.0:2959 2014-08-12 15:49:40.186438 2014-08-13 17:38:03.084026 7fe9bae887a0 20 read_log 194215'13296 (194215'13295) delete c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.2174595.0:180174 0.000000 2014-08-13 17:38:03.084037 7fe9bae887a0 20 read_log 194215'13297 (193305'13180) clean a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by osd.45.0:2961 2014-08-12 15:49:40.194663 2014-08-13 17:38:03.084049 7fe9bae887a0 20 read_log 194215'13298 (194215'13297) delete a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.2174595.0:180176 0.000000 2014-08-13 17:38:03.084060 7fe9bae887a0 20 read_log 194215'13299 (192834'12617) clean ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.45.0:2963 2014-08-12 15:49:40.206138 2014-08-13 17:38:03.084072 7fe9bae887a0 20 read_log 194215'13300 (194215'13299) delete ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.2174595.0:180178 0.000000 2014-08-13 17:38:03.084082 7fe9bae887a0 20 read_log 194215'13301 (192834'12587) clean d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.45.0:2965 2014-08-12 15:49:40.217078 2014-08-13 17:38:03.084094 7fe9bae887a0 20 read_log 194215'13302 (194215'13301) delete d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.2174595.0:180180 0.000000 2014-08-13 17:38:03.084105 7fe9bae887a0 20 read_log 194215'13303 (192834'12741) clean 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by osd.45.0:2967 2014-08-12 15:49:40.225443 2014-08-13 17:38:03.084117 7fe9bae887a0 20 read_log 194215'13304 (194215'13303) delete 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.2174595.0:180182 0.000000 2014-08-13 17:38:03.084128 7fe9bae887a0 20 read_log 194215'13305 (192888'13089) clean 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by osd.45.0:2969 2014-08-12 15:49:40.233165 2014-08-13 17:38:03.084140 7fe9bae887a0 20 read_log 194215'13306 (194215'13305) delete 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.2174595.0:180184 0.000000 2014-08-13 17:38:03.084150 7fe9bae887a0 20 read_log 194215'13307 (192888'13112) clean 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by osd.45.0:2971 2014-08-12 15:49:40.240812 2014-08-13 17:38:03.084169 7fe9bae887a0 20 read_log 194215'13308 (194215'13307) delete 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.2174595.0:180186 0.000000 2014-08-13 17:38:03.084180 7fe9bae887a0 20 read_log 194215'13309 (192888'13118) clean 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by osd.45.0:2973 2014-08-12 15:49:40.248699 2014-08-13 17:38:03.084193 7fe9bae887a0 20 read_log 194215'13310 (194215'13309) delete 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.2174595.0:180188 0.000000 2014-08-13 17:38:03.084203 7fe9bae887a0 20 read_log 194215'13311 (192888'13070) clean de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by osd.45.0:2975 2014-08-12 15:49:40.256105 2014-08-13 17:38:03.084216 7fe9bae887a0 20 read_log 194215'13312 (194215'13311) delete de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.2174595.0:180190 0.000000 2014-08-13 17:38:03.084226 7fe9bae887a0 20 read_log 194215'13313 (192834'12520) clean 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.45.0:2977 2014-08-12 15:49:40.270494 2014-08-13 17:38:03.084238 7fe9bae887a0 20 read_log 194215'13314 (194215'13313) delete 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.2174595.0:180192 0.000000 2014-08-13 17:38:03.084261 7fe9bae887a0 20 read_log 194215'13315 (193305'13185) clean c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by osd.45.0:2979 2014-08-12 15:49:40.278069 2014-08-13 17:38:03.084274 7fe9bae887a0 20 read_log 194215'13316 (194215'13315) delete c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.2174595.0:180194 0.000000 2014-08-13 17:38:03.084285 7fe9bae887a0 20 read_log 194215'13317 (192834'12575) clean 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.45.0:2981 2014-08-12 15:49:40.290321 2014-08-13 17:38:03.084297 7fe9bae887a0 20 read_log 194215'13318 (194215'13317) delete 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.2174595.0:180196 0.000000 2014-08-13 17:38:03.084308 7fe9bae887a0 20 read_log 194215'13319 (192888'12935) clean e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by osd.45.0:2983 2014-08-12 15:49:40.298751 2014-08-13 17:38:03.084320 7fe9bae887a0 20 read_log 194215'13320 (194215'13319) delete e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.2174595.0:180198 0.000000 2014-08-13 17:38:03.084331 7fe9bae887a0 20 read_log 194215'13321 (192888'12921) clean 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by osd.45.0:2985 2014-08-12 15:49:40.306858 2014-08-13 17:38:03.084344 7fe9bae887a0 20 read_log 194215'13322 (194215'13321) delete 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.2174595.0:180200 0.000000 2014-08-13 17:38:03.084354 7fe9bae887a0 20 read_log 194215'13323 (192888'12934) clean 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by osd.45.0:2987 2014-08-12 15:49:40.314194 2014-08-13 17:38:03.084366 7fe9bae887a0 20 read_log 194215'13324 (194215'13323) delete 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.2174595.0:180202 0.000000 2014-08-13 17:38:03.084377 7fe9bae887a0 20 read_log 194215'13325 (193305'13177) clean ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by osd.45.0:2989 2014-08-12 15:49:40.322145 2014-08-13 17:38:03.084389 7fe9bae887a0 20 read_log 194215'13326 (194215'13325) delete ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.2174595.0:180204 0.000000 2014-08-13 17:38:03.084399 7fe9bae887a0 20 read_log 194215'13327 (192888'12873) clean 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by osd.45.0:2991 2014-08-12 15:49:40.329597 2014-08-13 17:38:03.084412 7fe9bae887a0 20 read_log 194215'13328 (194215'13327) delete 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.2174595.0:180206 0.000000 2014-08-13 17:38:03.084422 7fe9bae887a0 20 read_log 194215'13329 (193305'13189) clean b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by osd.45.0:2993 2014-08-12 15:49:40.337847 2014-08-13 17:38:03.084434 7fe9bae887a0 20 read_log 194215'13330 (194215'13329) delete b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.2174595.0:180208 0.000000 2014-08-13 17:38:03.084445 7fe9bae887a0 20 read_log 194215'13331 (193305'13188) clean 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by osd.45.0:2995 2014-08-12 15:49:40.345259 2014-08-13 17:38:03.084458 7fe9bae887a0 20 read_log 194215'13332 (194215'13331) delete 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.2174595.0:180210 0.000000 2014-08-13 17:38:03.084469 7fe9bae887a0 20 read_log 194215'13333 (192834'12523) clean 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.45.0:2997 2014-08-12 15:49:40.356720 2014-08-13 17:38:03.084481 7fe9bae887a0 20 read_log 194215'13334 (194215'13333) delete 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.2174595.0:180212 0.000000 2014-08-13 17:38:03.084499 7fe9bae887a0 20 read_log 194215'13335 (192888'12904) clean 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by osd.45.0:2999 2014-08-12 15:49:40.365404 2014-08-13 17:38:03.084512 7fe9bae887a0 20 read_log 194215'13336 (194215'13335) delete 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.2174595.0:180214 0.000000 2014-08-13 17:38:03.084522 7fe9bae887a0 20 read_log 194215'13337 (192888'13095) clean 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by osd.45.0:3001 2014-08-12 15:49:40.373167 2014-08-13 17:38:03.084534 7fe9bae887a0 20 read_log 194215'13338 (194215'13337) delete 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.2174595.0:180216 0.000000 2014-08-13 17:38:03.084545 7fe9bae887a0 20 read_log 194215'13339 (192834'11882) clean 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by osd.45.0:3003 2014-08-12 15:49:40.380550 2014-08-13 17:38:03.084578 7fe9bae887a0 20 read_log 194215'13340 (194215'13339) delete 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.2174595.0:180218 0.000000 2014-08-13 17:38:03.084589 7fe9bae887a0 20 read_log 194215'13341 (192834'11837) clean 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by osd.45.0:3005 2014-08-12 15:49:40.387983 2014-08-13 17:38:03.084602 7fe9bae887a0 20 read_log 194215'13342 (194215'13341) delete 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.2174595.0:180220 0.000000 2014-08-13 17:38:03.084613 7fe9bae887a0 20 read_log 194215'13343 (192834'11895) clean 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by osd.45.0:3007 2014-08-12 15:49:40.396164 2014-08-13 17:38:03.084625 7fe9bae887a0 20 read_log 194215'13344 (194215'13343) delete 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.2174595.0:180222 0.000000 2014-08-13 17:38:03.084638 7fe9bae887a0 20 read_log 194215'13345 (192834'12623) clean f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by osd.45.0:3009 2014-08-12 15:49:40.404584 2014-08-13 17:38:03.084653 7fe9bae887a0 20 read_log 194215'13346 (194215'13345) delete f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.2174595.0:180224 0.000000 2014-08-13 17:38:03.084664 7fe9bae887a0 20 read_log 194215'13347 (192834'11784) clean 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by osd.45.0:3011 2014-08-12 15:49:40.412273 2014-08-13 17:38:03.084676 7fe9bae887a0 20 read_log 194215'13348 (194215'13347) delete 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.2174595.0:180226 0.000000 2014-08-13 17:38:03.084689 7fe9bae887a0 20 read_log 194215'13349 (192834'12544) clean 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by osd.45.0:3013 2014-08-12 15:49:40.419554 2014-08-13 17:38:03.084702 7fe9bae887a0 20 read_log 194215'13350 (194215'13349) delete 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.2174595.0:180228 0.000000 2014-08-13 17:38:03.084713 7fe9bae887a0 20 read_log 194215'13351 (192888'12883) clean e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by osd.45.0:3015 2014-08-12 15:49:40.427267 2014-08-13 17:38:03.084728 7fe9bae887a0 20 read_log 194215'13352 (194215'13351) delete e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.2174595.0:180230 0.000000 2014-08-13 17:38:03.084738 7fe9bae887a0 20 read_log 194215'13353 (192834'11770) clean 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by osd.45.0:3017 2014-08-12 15:49:40.435123 2014-08-13 17:38:03.084750 7fe9bae887a0 20 read_log 194215'13354 (194215'13353) delete 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.2174595.0:180232 0.000000 2014-08-13 17:38:03.084768 7fe9bae887a0 20 read_log 194215'13355 (192834'11887) clean f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by osd.45.0:3019 2014-08-12 15:49:40.443201 2014-08-13 17:38:03.084783 7fe9bae887a0 20 read_log 194215'13356 (194215'13355) delete f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.2174595.0:180234 0.000000 2014-08-13 17:38:03.084794 7fe9bae887a0 20 read_log 194215'13357 (193305'13191) clean 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by osd.45.0:3021 2014-08-12 15:49:40.451547 2014-08-13 17:38:03.084806 7fe9bae887a0 20 read_log 194215'13358 (194215'13357) delete 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.2174595.0:180236 0.000000 2014-08-13 17:38:03.084817 7fe9bae887a0 20 read_log 194215'13359 (192834'12584) clean d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by osd.45.0:3023 2014-08-12 15:49:40.459929 2014-08-13 17:38:03.084829 7fe9bae887a0 20 read_log 194215'13360 (194215'13359) delete d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.2174595.0:180238 0.000000 2014-08-13 17:38:03.084840 7fe9bae887a0 20 read_log 194215'13361 (193305'13186) clean de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by osd.45.0:3025 2014-08-12 15:49:40.467173 2014-08-13 17:38:03.084852 7fe9bae887a0 20 read_log 194215'13362 (194215'13361) delete de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.2174595.0:180240 0.000000 2014-08-13 17:38:03.084862 7fe9bae887a0 20 read_log 194215'13363 (192888'13111) clean df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by osd.45.0:3027 2014-08-12 15:49:40.475017 2014-08-13 17:38:03.084874 7fe9bae887a0 20 read_log 194215'13364 (194215'13363) delete df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.2174595.0:180242 0.000000 2014-08-13 17:38:03.084885 7fe9bae887a0 20 read_log 194215'13365 (192888'12892) clean db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by osd.45.0:3029 2014-08-12 15:49:40.482990 2014-08-13 17:38:03.084899 7fe9bae887a0 20 read_log 194215'13366 (194215'13365) delete db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.2174595.0:180244 0.000000 2014-08-13 17:38:03.084910 7fe9bae887a0 20 read_log 194215'13367 (192834'12597) clean 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by osd.45.0:3031 2014-08-12 15:49:40.491182 2014-08-13 17:38:03.084922 7fe9bae887a0 20 read_log 194215'13368 (194215'13367) delete 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.2174595.0:180246 0.000000 2014-08-13 17:38:03.084933 7fe9bae887a0 20 read_log 194215'13369 (192888'13114) clean cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by osd.45.0:3033 2014-08-12 15:49:40.498963 2014-08-13 17:38:03.084945 7fe9bae887a0 20 read_log 194215'13370 (194215'13369) delete cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.2174595.0:180248 0.000000 2014-08-13 17:38:03.084956 7fe9bae887a0 20 read_log 194215'13371 (192888'13115) clean 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by osd.45.0:3035 2014-08-12 15:49:40.507492 2014-08-13 17:38:03.084970 7fe9bae887a0 20 read_log 194215'13372 (194215'13371) delete 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.2174595.0:180250 0.000000 2014-08-13 17:38:03.084981 7fe9bae887a0 20 read_log 194215'13373 (192888'13082) clean 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by osd.45.0:3037 2014-08-12 15:49:40.515221 2014-08-13 17:38:03.084993 7fe9bae887a0 20 read_log 194215'13374 (194215'13373) delete 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.2174595.0:180252 0.000000 2014-08-13 17:38:03.085028 7fe9bae887a0 20 read_log 194215'13375 (192888'12893) clean 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by osd.45.0:3039 2014-08-12 15:49:40.523105 2014-08-13 17:38:03.085043 7fe9bae887a0 20 read_log 194215'13376 (194215'13375) delete 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.2174595.0:180254 0.000000 2014-08-13 17:38:03.085054 7fe9bae887a0 20 read_log 194215'13377 (192888'13068) clean 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by osd.45.0:3041 2014-08-12 15:49:40.530246 2014-08-13 17:38:03.085067 7fe9bae887a0 20 read_log 194215'13378 (194215'13377) delete 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.2174595.0:180256 0.000000 2014-08-13 17:38:03.085097 7fe9bae887a0 20 read_log 194215'13379 (192888'12909) clean 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by osd.45.0:3043 2014-08-12 15:49:40.538634 2014-08-13 17:38:03.085110 7fe9bae887a0 20 read_log 194215'13380 (194215'13379) delete 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.2174595.0:180258 0.000000 2014-08-13 17:38:03.085140 7fe9bae887a0 20 read_log 194215'13381 (193305'13197) clean 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by osd.45.0:3045 2014-08-12 15:49:40.546188 2014-08-13 17:38:03.085160 7fe9bae887a0 20 read_log 194215'13382 (194215'13381) delete 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.2174595.0:180260 0.000000 2014-08-13 17:38:03.085174 7fe9bae887a0 20 read_log 194215'13383 (192888'12886) clean c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by osd.45.0:3047 2014-08-12 15:49:40.553772 2014-08-13 17:38:03.085186 7fe9bae887a0 20 read_log 194215'13384 (194215'13383) delete c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.2174595.0:180262 0.000000 2014-08-13 17:38:03.085197 7fe9bae887a0 20 read_log 194215'13385 (193305'13194) clean c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by osd.45.0:3049 2014-08-12 15:49:40.562187 2014-08-13 17:38:03.085209 7fe9bae887a0 20 read_log 194215'13386 (194215'13385) delete c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.2174595.0:180264 0.000000 2014-08-13 17:38:03.085220 7fe9bae887a0 20 read_log 194215'13387 (192834'12712) clean 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by osd.45.0:3051 2014-08-12 15:49:40.570787 2014-08-13 17:38:03.085232 7fe9bae887a0 20 read_log 194215'13388 (194215'13387) delete 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.2174595.0:180266 0.000000 2014-08-13 17:38:03.085243 7fe9bae887a0 20 read_log 194215'13389 (192888'12895) clean 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by osd.45.0:3053 2014-08-12 15:49:40.578826 2014-08-13 17:38:03.085255 7fe9bae887a0 20 read_log 194215'13390 (194215'13389) delete 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.2174595.0:180268 0.000000 2014-08-13 17:38:03.085266 7fe9bae887a0 20 read_log 194215'13391 (192888'12877) clean 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by osd.45.0:3055 2014-08-12 15:49:40.587240 2014-08-13 17:38:03.085278 7fe9bae887a0 20 read_log 194215'13392 (194215'13391) delete 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.2174595.0:180270 0.000000 2014-08-13 17:38:03.085289 7fe9bae887a0 20 read_log 194215'13393 (193305'13193) clean cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by osd.45.0:3057 2014-08-12 15:49:40.595118 2014-08-13 17:38:03.085303 7fe9bae887a0 20 read_log 194215'13394 (194215'13393) delete cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.2174595.0:180272 0.000000 2014-08-13 17:38:03.085320 7fe9bae887a0 20 read_log 194215'13395 (192834'12635) clean b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by osd.45.0:3059 2014-08-12 15:49:40.603338 2014-08-13 17:38:03.085333 7fe9bae887a0 20 read_log 194215'13396 (194215'13395) delete b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.2174595.0:180274 0.000000 2014-08-13 17:38:03.085344 7fe9bae887a0 20 read_log 194215'13397 (192888'12885) clean 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by osd.45.0:3061 2014-08-12 15:49:40.610674 2014-08-13 17:38:03.085356 7fe9bae887a0 20 read_log 194215'13398 (194215'13397) delete 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.2174595.0:180276 0.000000 2014-08-13 17:38:03.085367 7fe9bae887a0 20 read_log 194215'13399 (192888'12929) clean 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by osd.45.0:3063 2014-08-12 15:49:40.618226 2014-08-13 17:38:03.085379 7fe9bae887a0 20 read_log 194215'13400 (194215'13399) delete 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.2174595.0:180278 0.000000 2014-08-13 17:38:03.085390 7fe9bae887a0 20 read_log 194215'13401 (192888'12910) clean 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by osd.45.0:3065 2014-08-12 15:49:40.626164 2014-08-13 17:38:03.085402 7fe9bae887a0 20 read_log 194215'13402 (194215'13401) delete 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.2174595.0:180280 0.000000 2014-08-13 17:38:03.085412 7fe9bae887a0 20 read_log 194215'13403 (192834'11749) clean 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by osd.45.0:3067 2014-08-12 15:49:40.633952 2014-08-13 17:38:03.085426 7fe9bae887a0 20 read_log 194215'13404 (194215'13403) delete 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.2174595.0:180282 0.000000 2014-08-13 17:38:03.085436 7fe9bae887a0 20 read_log 194215'13405 (192888'12890) clean afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by osd.45.0:3069 2014-08-12 15:49:40.642162 2014-08-13 17:38:03.085450 7fe9bae887a0 20 read_log 194215'13406 (194215'13405) delete afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.2174595.0:180284 0.000000 2014-08-13 17:38:03.085461 7fe9bae887a0 20 read_log 194215'13407 (193305'13190) clean bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by osd.45.0:3071 2014-08-12 15:49:40.649819 2014-08-13 17:38:03.085473 7fe9bae887a0 20 read_log 194215'13408 (194215'13407) delete bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.2174595.0:180286 0.000000 2014-08-13 17:38:03.085483 7fe9bae887a0 20 read_log 194215'13409 (192888'13096) clean 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by osd.45.0:3073 2014-08-12 15:49:40.657008 2014-08-13 17:38:03.085496 7fe9bae887a0 20 read_log 194215'13410 (194215'13409) delete 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.2174595.0:180288 0.000000 2014-08-13 17:38:03.085506 7fe9bae887a0 20 read_log 194215'13411 (193305'13182) clean f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by osd.45.0:3075 2014-08-12 15:49:40.665307 2014-08-13 17:38:03.085518 7fe9bae887a0 20 read_log 194215'13412 (194215'13411) delete f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.2174595.0:180290 0.000000 2014-08-13 17:38:03.085529 7fe9bae887a0 20 read_log 194215'13413 (192834'11854) clean 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by osd.45.0:3077 2014-08-12 15:49:40.673007 2014-08-13 17:38:03.085541 7fe9bae887a0 20 read_log 194215'13414 (194215'13413) delete 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.2174595.0:180292 0.000000 2014-08-13 17:38:03.085563 7fe9bae887a0 20 read_log 194215'13415 (192888'13092) clean fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by osd.45.0:3079 2014-08-12 15:49:40.680756 2014-08-13 17:38:03.085576 7fe9bae887a0 20 read_log 194215'13416 (194215'13415) delete fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.2174595.0:180294 0.000000 2014-08-13 17:38:03.085587 7fe9bae887a0 20 read_log 194215'13417 (192834'12732) clean 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by osd.45.0:3081 2014-08-12 15:49:40.688893 2014-08-13 17:38:03.085599 7fe9bae887a0 20 read_log 194215'13418 (194215'13417) delete 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.2174595.0:180296 0.000000 2014-08-13 17:38:03.085610 7fe9bae887a0 20 read_log 194215'13419 (192888'12931) clean 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by osd.45.0:3083 2014-08-12 15:49:40.697032 2014-08-13 17:38:03.085622 7fe9bae887a0 20 read_log 194215'13420 (194215'13419) delete 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.2174595.0:180298 0.000000 2014-08-13 17:38:03.085632 7fe9bae887a0 20 read_log 194215'13421 (192834'12569) clean 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by osd.45.0:3085 2014-08-12 15:49:40.704900 2014-08-13 17:38:03.085644 7fe9bae887a0 20 read_log 194215'13422 (194215'13421) delete 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.2174595.0:180300 0.000000 2014-08-13 17:38:03.085655 7fe9bae887a0 20 read_log 194215'13423 (192888'12920) clean 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by osd.45.0:3087 2014-08-12 15:49:40.713079 2014-08-13 17:38:03.085667 7fe9bae887a0 20 read_log 194215'13424 (194215'13423) delete 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.2174595.0:180302 0.000000 2014-08-13 17:38:03.085678 7fe9bae887a0 20 read_log 194215'13425 (192888'13103) clean 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by osd.45.0:3089 2014-08-12 15:49:40.722446 2014-08-13 17:38:03.085690 7fe9bae887a0 20 read_log 194215'13426 (194215'13425) delete 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.2174595.0:180304 0.000000 2014-08-13 17:38:03.085701 7fe9bae887a0 20 read_log 194215'13427 (192888'13109) clean 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by osd.45.0:3091 2014-08-12 15:49:40.729801 2014-08-13 17:38:03.085713 7fe9bae887a0 20 read_log 194215'13428 (194215'13427) delete 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.2174595.0:180306 0.000000 2014-08-13 17:38:03.085724 7fe9bae887a0 20 read_log 194215'13429 (192888'12875) clean e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by osd.45.0:3093 2014-08-12 15:49:40.737477 2014-08-13 17:38:03.085735 7fe9bae887a0 20 read_log 194215'13430 (194215'13429) delete e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.2174595.0:180308 0.000000 2014-08-13 17:38:03.085746 7fe9bae887a0 20 read_log 194215'13431 (193305'13201) clean 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by osd.45.0:3095 2014-08-12 15:49:40.744824 2014-08-13 17:38:03.085758 7fe9bae887a0 20 read_log 194215'13432 (194215'13431) delete 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.2174595.0:180310 0.000000 2014-08-13 17:38:03.085789 7fe9bae887a0 20 read_log 194215'13433 (193305'13202) clean 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by osd.45.0:3097 2014-08-12 15:49:40.752303 2014-08-13 17:38:03.085802 7fe9bae887a0 20 read_log 194215'13434 (194215'13433) delete 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.2174595.0:180312 0.000000 2014-08-13 17:38:03.085820 7fe9bae887a0 20 read_log 194215'13435 (192888'13090) clean c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by osd.45.0:3099 2014-08-12 15:49:40.760095 2014-08-13 17:38:03.085835 7fe9bae887a0 20 read_log 194215'13436 (194215'13435) delete c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.2174595.0:180314 0.000000 2014-08-13 17:38:03.085846 7fe9bae887a0 20 read_log 194215'13437 (192888'12932) clean f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by osd.45.0:3101 2014-08-12 15:49:40.767826 2014-08-13 17:38:03.085859 7fe9bae887a0 20 read_log 194215'13438 (194215'13437) delete f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.2174595.0:180316 0.000000 2014-08-13 17:38:03.085871 7fe9bae887a0 20 read_log 194215'13439 (192888'13106) clean 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by osd.45.0:3103 2014-08-12 15:49:40.776314 2014-08-13 17:38:03.085885 7fe9bae887a0 20 read_log 194215'13440 (194215'13439) delete 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.2174595.0:180318 0.000000 2014-08-13 17:38:03.085896 7fe9bae887a0 20 read_log 194215'13441 (192888'12884) clean 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by osd.45.0:3105 2014-08-12 15:49:40.784405 2014-08-13 17:38:03.085908 7fe9bae887a0 20 read_log 194215'13442 (194215'13441) delete 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.2174595.0:180320 0.000000 2014-08-13 17:38:03.085920 7fe9bae887a0 20 read_log 194215'13443 (192888'13117) clean 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by osd.45.0:3107 2014-08-12 15:49:40.792240 2014-08-13 17:38:03.085934 7fe9bae887a0 20 read_log 194215'13444 (194215'13443) delete 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.2174595.0:180322 0.000000 2014-08-13 17:38:03.085945 7fe9bae887a0 20 read_log 194215'13445 (192834'12739) clean b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by osd.45.0:3109 2014-08-12 15:49:40.801432 2014-08-13 17:38:03.085957 7fe9bae887a0 20 read_log 194215'13446 (194215'13445) delete b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.2174595.0:180324 0.000000 2014-08-13 17:38:03.085968 7fe9bae887a0 20 read_log 194215'13447 (192888'12916) clean 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by osd.45.0:3111 2014-08-12 15:49:40.808291 2014-08-13 17:38:03.085982 7fe9bae887a0 20 read_log 194215'13448 (194215'13447) delete 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.2174595.0:180326 0.000000 2014-08-13 17:38:03.085993 7fe9bae887a0 20 read_log 194215'13449 (192888'13083) clean 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by osd.45.0:3113 2014-08-12 15:49:40.815359 2014-08-13 17:38:03.086005 7fe9bae887a0 20 read_log 194215'13450 (194215'13449) delete 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.2174595.0:180328 0.000000 2014-08-13 17:38:03.086016 7fe9bae887a0 20 read_log 194215'13451 (193305'13196) clean d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by osd.45.0:3115 2014-08-12 15:49:40.822707 2014-08-13 17:38:03.086030 7fe9bae887a0 20 read_log 194215'13452 (194215'13451) delete d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.2174595.0:180330 0.000000 2014-08-13 17:38:03.086041 7fe9bae887a0 20 read_log 194215'13453 (193305'13198) clean c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by osd.45.0:3117 2014-08-12 15:49:40.830345 2014-08-13 17:38:03.086053 7fe9bae887a0 20 read_log 194215'13454 (194215'13453) delete c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.2174595.0:180332 0.000000 2014-08-13 17:38:03.086080 7fe9bae887a0 20 read_log 194215'13455 (193305'13187) clean e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by osd.45.0:3119 2014-08-12 15:49:40.838265 2014-08-13 17:38:03.086095 7fe9bae887a0 20 read_log 194215'13456 (194215'13455) delete e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.2174595.0:180334 0.000000 2014-08-13 17:38:03.086106 7fe9bae887a0 20 read_log 194215'13457 (192888'13075) clean 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by osd.45.0:3121 2014-08-12 15:49:40.846159 2014-08-13 17:38:03.086118 7fe9bae887a0 20 read_log 194215'13458 (194215'13457) delete 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.2174595.0:180336 0.000000 2014-08-13 17:38:03.086129 7fe9bae887a0 20 read_log 194215'13459 (193305'13178) clean b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by osd.45.0:3123 2014-08-12 15:49:40.853841 2014-08-13 17:38:03.086143 7fe9bae887a0 20 read_log 194215'13460 (194215'13459) delete b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.2174595.0:180338 0.000000 2014-08-13 17:38:03.086159 7fe9bae887a0 20 read_log 194215'13461 (192834'12559) clean 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by osd.45.0:3125 2014-08-12 15:49:40.861122 2014-08-13 17:38:03.086172 7fe9bae887a0 20 read_log 194215'13462 (194215'13461) delete 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.2174595.0:180340 0.000000 2014-08-13 17:38:03.086186 7fe9bae887a0 20 read_log 194215'13463 (192834'11789) clean 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by osd.45.0:3127 2014-08-12 15:49:40.868792 2014-08-13 17:38:03.086198 7fe9bae887a0 20 read_log 194215'13464 (194215'13463) delete 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.2174595.0:180342 0.000000 2014-08-13 17:38:03.086209 7fe9bae887a0 20 read_log 194215'13465 (192834'12582) clean 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by osd.45.0:3129 2014-08-12 15:49:40.875934 2014-08-13 17:38:03.086221 7fe9bae887a0 20 read_log 194215'13466 (194215'13465) delete 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.2174595.0:180344 0.000000 2014-08-13 17:38:03.086232 7fe9bae887a0 20 read_log 194215'13467 (192888'13104) clean ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by osd.45.0:3131 2014-08-12 15:49:40.883057 2014-08-13 17:38:03.086244 7fe9bae887a0 20 read_log 194215'13468 (194215'13467) delete ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.2174595.0:180346 0.000000 2014-08-13 17:38:03.086257 7fe9bae887a0 20 read_log 194215'13469 (192888'12919) clean d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by osd.45.0:3133 2014-08-12 15:49:40.890159 2014-08-13 17:38:03.086270 7fe9bae887a0 20 read_log 194215'13470 (194215'13469) delete d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.2174595.0:180348 0.000000 2014-08-13 17:38:03.086281 7fe9bae887a0 20 read_log 194215'13471 (192834'12674) clean c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by osd.45.0:3135 2014-08-12 15:49:40.897524 2014-08-13 17:38:03.086293 7fe9bae887a0 20 read_log 194215'13472 (194215'13471) delete c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.2174595.0:180350 0.000000 2014-08-13 17:38:03.086304 7fe9bae887a0 20 read_log 194215'13473 (192834'11881) clean 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by osd.45.0:3137 2014-08-12 15:49:40.904791 2014-08-13 17:38:03.086317 7fe9bae887a0 20 read_log 194215'13474 (194215'13473) delete 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.2174595.0:180352 0.000000 2014-08-13 17:38:03.086336 7fe9bae887a0 20 read_log 194215'13475 (192888'12878) clean ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by osd.45.0:3139 2014-08-12 15:49:40.912294 2014-08-13 17:38:03.086351 7fe9bae887a0 20 read_log 194215'13476 (194215'13475) delete ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.2174595.0:180354 0.000000 2014-08-13 17:38:03.086362 7fe9bae887a0 20 read_log 194370'13477 (0'0) promote ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by osd.45.0:3399 2014-08-12 20:30:22.021328 2014-08-13 17:38:03.086374 7fe9bae887a0 20 read_log 194370'13478 (194370'13477) modify ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by client.1508531.0:2028 2014-08-12 20:30:22.014253 2014-08-13 17:38:03.086386 7fe9bae887a0 20 read_log 194370'13479 (0'0) promote fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by osd.45.0:3405 2014-08-12 20:30:22.454132 2014-08-13 17:38:03.086398 7fe9bae887a0 20 read_log 194370'13480 (194370'13479) modify fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by client.1508531.0:2148 2014-08-12 20:30:22.447561 2014-08-13 17:38:03.086410 7fe9bae887a0 20 read_log 194370'13481 (0'0) promote 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by osd.45.0:3437 2014-08-12 20:30:28.913119 2014-08-13 17:38:03.086422 7fe9bae887a0 20 read_log 194370'13482 (194370'13481) modify 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by client.1508531.0:3896 2014-08-12 20:30:28.906404 2014-08-13 17:38:03.086434 7fe9bae887a0 20 read_log 194370'13483 (0'0) promote 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by osd.45.0:3441 2014-08-12 20:30:29.602747 2014-08-13 17:38:03.086446 7fe9bae887a0 20 read_log 194370'13484 (194370'13483) modify 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by client.1508531.0:4083 2014-08-12 20:30:29.596244 2014-08-13 17:38:03.086458 7fe9bae887a0 20 read_log 194370'13485 (0'0) promote 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by osd.45.0:3503 2014-08-12 20:30:41.005350 2014-08-13 17:38:03.086470 7fe9bae887a0 20 read_log 194370'13486 (194370'13485) modify 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by client.1508531.0:6729 2014-08-12 20:30:40.996084 2014-08-13 17:38:03.086482 7fe9bae887a0 20 read_log 194370'13487 (0'0) promote e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by osd.45.0:3523 2014-08-12 20:30:45.669858 2014-08-13 17:38:03.086494 7fe9bae887a0 20 read_log 194370'13488 (194370'13487) modify e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by client.1508531.0:7646 2014-08-12 20:30:45.662745 2014-08-13 17:38:03.086506 7fe9bae887a0 20 read_log 194370'13489 (0'0) promote 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by osd.45.0:3533 2014-08-12 20:30:48.035291 2014-08-13 17:38:03.086518 7fe9bae887a0 20 read_log 194370'13490 (194370'13489) modify 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by client.1508531.0:8084 2014-08-12 20:30:48.028390 2014-08-13 17:38:03.086531 7fe9bae887a0 20 read_log 194370'13491 (0'0) promote d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by osd.45.0:3539 2014-08-12 20:30:48.160452 2014-08-13 17:38:03.086543 7fe9bae887a0 20 read_log 194370'13492 (194370'13491) modify d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by client.1508531.0:8107 2014-08-12 20:30:48.149665 2014-08-13 17:38:03.086555 7fe9bae887a0 20 read_log 194370'13493 (0'0) promote 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by osd.45.0:3569 2014-08-12 20:31:00.741134 2014-08-13 17:38:03.086567 7fe9bae887a0 20 read_log 194370'13494 (194370'13493) modify 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by client.1508531.0:10253 2014-08-12 20:31:00.733201 2014-08-13 17:38:03.086586 7fe9bae887a0 20 read_log 194370'13495 (0'0) promote 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by osd.45.0:3575 2014-08-12 20:31:02.107572 2014-08-13 17:38:03.086599 7fe9bae887a0 20 read_log 194370'13496 (194370'13495) modify 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by client.1508531.0:10481 2014-08-12 20:31:02.100588 2014-08-13 17:38:03.086611 7fe9bae887a0 20 read_log 194370'13497 (0'0) promote d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by osd.45.0:3591 2014-08-12 20:31:06.337415 2014-08-13 17:38:03.086623 7fe9bae887a0 20 read_log 194370'13498 (194370'13497) modify d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by client.1508531.0:11177 2014-08-12 20:31:06.329571 2014-08-13 17:38:03.086635 7fe9bae887a0 20 read_log 194370'13499 (0'0) promote e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by osd.45.0:3593 2014-08-12 20:31:06.860950 2014-08-13 17:38:03.086648 7fe9bae887a0 20 read_log 194370'13500 (194370'13499) modify e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by client.1508531.0:11261 2014-08-12 20:31:06.852597 2014-08-13 17:38:03.086660 7fe9bae887a0 20 read_log 194370'13501 (0'0) promote 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by osd.45.0:3601 2014-08-12 20:31:08.926182 2014-08-13 17:38:03.086671 7fe9bae887a0 20 read_log 194370'13502 (194370'13501) modify 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by client.1508531.0:11600 2014-08-12 20:31:08.918385 2014-08-13 17:38:03.086683 7fe9bae887a0 20 read_log 194370'13503 (0'0) promote 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by osd.45.0:3605 2014-08-12 20:31:11.137458 2014-08-13 17:38:03.086695 7fe9bae887a0 20 read_log 194370'13504 (194370'13503) modify 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by client.1508531.0:11969 2014-08-12 20:31:11.129136 2014-08-13 17:38:03.086707 7fe9bae887a0 20 read_log 194370'13505 (0'0) promote 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by osd.45.0:3613 2014-08-12 20:31:23.482958 2014-08-13 17:38:03.086719 7fe9bae887a0 20 read_log 194370'13506 (194370'13505) modify 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by client.1508531.0:13431 2014-08-12 20:31:23.475320 2014-08-13 17:38:03.086731 7fe9bae887a0 20 read_log 194370'13507 (0'0) promote 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by osd.45.0:3615 2014-08-12 20:31:23.587754 2014-08-13 17:38:03.086743 7fe9bae887a0 20 read_log 194370'13508 (194370'13507) modify 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by client.1508531.0:13442 2014-08-12 20:31:23.580280 2014-08-13 17:38:03.086754 7fe9bae887a0 20 read_log 194370'13509 (0'0) promote fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by osd.45.0:3629 2014-08-12 20:31:28.499813 2014-08-13 17:38:03.086767 7fe9bae887a0 20 read_log 194370'13510 (194370'13509) modify fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by client.1508531.0:13865 2014-08-12 20:31:28.491484 2014-08-13 17:38:03.086778 7fe9bae887a0 20 read_log 194377'13511 (0'0) promote b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by osd.45.0:3645 2014-08-12 20:33:01.112480 2014-08-13 17:38:03.086790 7fe9bae887a0 20 read_log 194377'13512 (194377'13511) modify b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by client.2231190.0:950 2014-08-12 20:33:01.106449 2014-08-13 17:38:03.086802 7fe9bae887a0 20 read_log 194377'13513 (0'0) promote fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by osd.45.0:3659 2014-08-12 20:33:05.700290 2014-08-13 17:38:03.086814 7fe9bae887a0 20 read_log 194377'13514 (194377'13513) modify fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by client.2231190.0:2105 2014-08-12 20:33:05.692627 2014-08-13 17:38:03.086852 7fe9bae887a0 20 read_log 194377'13515 (0'0) promote 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by osd.45.0:3661 2014-08-12 20:33:05.714862 2014-08-13 17:38:03.086868 7fe9bae887a0 20 read_log 194377'13516 (194377'13515) modify 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by client.2231190.0:2109 2014-08-12 20:33:05.709251 2014-08-13 17:38:03.086881 7fe9bae887a0 20 read_log 194377'13517 (0'0) promote 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by osd.45.0:3707 2014-08-12 20:33:15.694535 2014-08-13 17:38:03.086893 7fe9bae887a0 20 read_log 194377'13518 (194377'13517) modify 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by client.2231190.0:4296 2014-08-12 20:33:15.688131 2014-08-13 17:38:03.086907 7fe9bae887a0 20 read_log 194382'13519 (0'0) promote 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by osd.45.0:3731 2014-08-12 20:33:59.599982 2014-08-13 17:38:03.086920 7fe9bae887a0 20 read_log 194382'13520 (194382'13519) modify 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by client.2231190.0:5693 2014-08-12 20:33:59.591804 2014-08-13 17:38:03.086934 7fe9bae887a0 20 read_log 194382'13521 (0'0) promote 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by osd.45.0:3733 2014-08-12 20:33:59.975295 2014-08-13 17:38:03.086946 7fe9bae887a0 20 read_log 194382'13522 (194382'13521) modify 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by client.2231190.0:5763 2014-08-12 20:33:59.966938 2014-08-13 17:38:03.086959 7fe9bae887a0 20 read_log 194383'13523 (0'0) promote 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by osd.45.0:3745 2014-08-12 20:34:02.795334 2014-08-13 17:38:03.086974 7fe9bae887a0 20 read_log 194383'13524 (194383'13523) modify 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by client.2231190.0:6262 2014-08-12 20:34:02.788571 2014-08-13 17:38:03.086986 7fe9bae887a0 20 read_log 194383'13525 (0'0) promote 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by osd.45.0:3751 2014-08-12 20:34:07.121273 2014-08-13 17:38:03.087002 7fe9bae887a0 20 read_log 194383'13526 (194383'13525) modify 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by client.2231190.0:6704 2014-08-12 20:34:07.114278 2014-08-13 17:38:03.087017 7fe9bae887a0 20 read_log 194393'13527 (0'0) promote 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by osd.45.0:3772 2014-08-12 20:47:25.921551 2014-08-13 17:38:03.087029 7fe9bae887a0 20 read_log 194393'13528 (194393'13527) modify 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by client.1064095.0:3 2014-08-12 20:47:25.910100 2014-08-13 17:38:03.087043 7fe9bae887a0 20 read_log 194393'13529 (0'0) promote 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by osd.45.0:3799 2014-08-12 20:52:33.410142 2014-08-13 17:38:03.087056 7fe9bae887a0 20 read_log 194393'13530 (194393'13529) modify 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by client.1064422.0:218 2014-08-12 20:52:33.403536 2014-08-13 17:38:03.087069 7fe9bae887a0 20 read_log 194397'13531 (0'0) promote 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by osd.45.0:3820 2014-08-12 20:53:03.600982 2014-08-13 17:38:03.087081 7fe9bae887a0 20 read_log 194397'13532 (194397'13531) modify 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by client.2235069.0:797 2014-08-12 20:53:03.593214 2014-08-13 17:38:03.087096 7fe9bae887a0 20 read_log 194416'13533 (0'0) promote 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by osd.45.0:3934 2014-08-13 15:28:32.170447 2014-08-13 17:38:03.087115 7fe9bae887a0 20 read_log 194416'13534 (194416'13533) modify 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by client.2441958.0:448 2014-08-13 15:28:32.162246 2014-08-13 17:38:03.087130 7fe9bae887a0 20 read_log 194416'13535 (0'0) promote b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by osd.45.0:3962 2014-08-13 15:28:39.954509 2014-08-13 17:38:03.087169 7fe9bae887a0 20 read_log 194416'13536 (194416'13535) modify b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by client.2441958.0:2439 2014-08-13 15:28:39.945728 2014-08-13 17:38:03.087182 7fe9bae887a0 20 read_log 194416'13537 (0'0) promote 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by osd.45.0:3980 2014-08-13 15:28:45.583488 2014-08-13 17:38:03.087194 7fe9bae887a0 20 read_log 194416'13538 (194416'13537) modify 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by client.2441958.0:3908 2014-08-13 15:28:45.577009 2014-08-13 17:38:03.087206 7fe9bae887a0 20 read_log 194416'13539 (0'0) promote ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by osd.45.0:3982 2014-08-13 15:28:45.959559 2014-08-13 17:38:03.087220 7fe9bae887a0 20 read_log 194416'13540 (194416'13539) modify ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by client.2441958.0:4007 2014-08-13 15:28:45.952340 2014-08-13 17:38:03.087232 7fe9bae887a0 20 read_log 194416'13541 (0'0) promote bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by osd.45.0:4037 2014-08-13 15:28:56.428969 2014-08-13 17:38:03.087247 7fe9bae887a0 20 read_log 194416'13542 (194416'13541) modify bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by client.2441958.0:6636 2014-08-13 15:28:56.420019 2014-08-13 17:38:03.087259 7fe9bae887a0 20 read_log 194416'13543 (0'0) promote 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by osd.45.0:4051 2014-08-13 15:28:57.822315 2014-08-13 17:38:03.087272 7fe9bae887a0 20 read_log 194416'13544 (194416'13543) modify 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by client.2441958.0:7012 2014-08-13 15:28:57.812126 2014-08-13 17:38:03.087286 7fe9bae887a0 20 read_log 194416'13545 (0'0) promote b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by osd.45.0:4053 2014-08-13 15:28:58.067874 2014-08-13 17:38:03.087298 7fe9bae887a0 20 read_log 194416'13546 (194416'13545) modify b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by client.2441958.0:7077 2014-08-13 15:28:58.055264 2014-08-13 17:38:03.087330 7fe9bae887a0 20 read_log 194416'13547 (0'0) promote 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by osd.45.0:4101 2014-08-13 15:29:05.158874 2014-08-13 17:38:03.087345 7fe9bae887a0 20 read_log 194416'13548 (194416'13547) modify 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by client.2441958.0:8931 2014-08-13 15:29:05.149794 2014-08-13 17:38:03.087358 7fe9bae887a0 20 read_log 194416'13549 (0'0) promote 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by osd.45.0:4157 2014-08-13 15:29:16.079732 2014-08-13 17:38:03.087370 7fe9bae887a0 20 read_log 194416'13550 (194416'13549) modify 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by client.2441958.0:11642 2014-08-13 15:29:16.070252 2014-08-13 17:38:03.087382 7fe9bae887a0 20 read_log 194416'13551 (0'0) promote f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by osd.45.0:4185 2014-08-13 15:29:24.195078 2014-08-13 17:38:03.087394 7fe9bae887a0 20 read_log 194416'13552 (194416'13551) modify f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by client.2441958.0:13520 2014-08-13 15:29:24.186479 2014-08-13 17:38:03.087407 7fe9bae887a0 20 read_log 194416'13553 (0'0) promote 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by osd.45.0:4207 2014-08-13 15:29:31.601741 2014-08-13 17:38:03.087427 7fe9bae887a0 20 read_log 194416'13554 (194416'13553) modify 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by client.2441958.0:15295 2014-08-13 15:29:31.592735 2014-08-13 17:38:03.087442 7fe9bae887a0 20 read_log 194422'13555 (0'0) promote 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by osd.45.0:4267 2014-08-13 15:32:16.666305 2014-08-13 17:38:03.087454 7fe9bae887a0 20 read_log 194422'13556 (194422'13555) modify 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by client.1148203.0:357 2014-08-13 15:32:16.659486 2014-08-13 17:38:03.087467 7fe9bae887a0 20 read_log 194422'13557 (0'0) promote b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by osd.45.0:4271 2014-08-13 15:32:18.007212 2014-08-13 17:38:03.087480 7fe9bae887a0 20 read_log 194422'13558 (194422'13557) modify b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by client.1148203.0:660 2014-08-13 15:32:17.997715 2014-08-13 17:38:03.087492 7fe9bae887a0 20 read_log 194422'13559 (0'0) promote ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by osd.45.0:4315 2014-08-13 15:32:27.587934 2014-08-13 17:38:03.087506 7fe9bae887a0 20 read_log 194422'13560 (194422'13559) modify ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by client.1148203.0:3129 2014-08-13 15:32:27.579892 2014-08-13 17:38:03.087518 7fe9bae887a0 20 read_log 194422'13561 (0'0) promote 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by osd.45.0:4317 2014-08-13 15:32:27.917519 2014-08-13 17:38:03.087530 7fe9bae887a0 20 read_log 194422'13562 (194422'13561) modify 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by client.1148203.0:3216 2014-08-13 15:32:27.910541 2014-08-13 17:38:03.087543 7fe9bae887a0 20 read_log 194422'13563 (0'0) promote 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by osd.45.0:4319 2014-08-13 15:32:28.298173 2014-08-13 17:38:03.087554 7fe9bae887a0 20 read_log 194422'13564 (194422'13563) modify 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by client.1148203.0:3301 2014-08-13 15:32:28.289735 2014-08-13 17:38:03.087572 7fe9bae887a0 20 read_log 194422'13565 (0'0) promote 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by osd.45.0:4359 2014-08-13 15:32:41.147567 2014-08-13 17:38:03.087584 7fe9bae887a0 20 read_log 194422'13566 (194422'13565) modify 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by client.1148203.0:6186 2014-08-13 15:32:41.139420 2014-08-13 17:38:03.087596 7fe9bae887a0 20 read_log 194422'13567 (0'0) promote 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by osd.45.0:4365 2014-08-13 15:32:42.381606 2014-08-13 17:38:03.087608 7fe9bae887a0 20 read_log 194422'13568 (194422'13567) modify 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by client.1148203.0:6412 2014-08-13 15:32:42.373457 2014-08-13 17:38:03.087621 7fe9bae887a0 20 read_log 194422'13569 (0'0) promote cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by osd.45.0:4425 2014-08-13 15:33:00.878658 2014-08-13 17:38:03.087633 7fe9bae887a0 20 read_log 194422'13570 (194422'13569) modify cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by client.1148203.0:9134 2014-08-13 15:33:00.869629 2014-08-13 17:38:03.087645 7fe9bae887a0 20 read_log 194422'13571 (0'0) promote 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by osd.45.0:4427 2014-08-13 15:33:01.333839 2014-08-13 17:38:03.087657 7fe9bae887a0 20 read_log 194422'13572 (194422'13571) modify 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by client.1148203.0:9180 2014-08-13 15:33:01.324967 2014-08-13 17:38:03.087669 7fe9bae887a0 20 read_log 194422'13573 (0'0) promote 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by osd.45.0:4441 2014-08-13 15:33:11.111036 2014-08-13 17:38:03.087683 7fe9bae887a0 20 read_log 194422'13574 (194422'13573) modify 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by client.1148203.0:10028 2014-08-13 15:33:11.102604 2014-08-13 17:38:03.087695 7fe9bae887a0 20 read_log 194422'13575 (0'0) promote a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by osd.45.0:4445 2014-08-13 15:33:25.819849 2014-08-13 17:38:03.087707 7fe9bae887a0 20 read_log 194422'13576 (194422'13575) modify a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by client.1148203.0:10567 2014-08-13 15:33:25.810541 2014-08-13 17:38:03.087722 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.088192 7fe9bae887a0 10 read_log checking for missing items over interval (194370'13477,194422'13576] 2014-08-13 17:38:03.088247 7fe9bae887a0 15 read_log missing 194422'13576 (194422'13575) modify a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by client.1148203.0:10567 2014-08-13 15:33:25.810541 2014-08-13 17:38:03.088302 7fe9bae887a0 15 read_log missing 194422'13574 (194422'13573) modify 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by client.1148203.0:10028 2014-08-13 15:33:11.102604 2014-08-13 17:38:03.088340 7fe9bae887a0 15 read_log missing 194422'13572 (194422'13571) modify 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by client.1148203.0:9180 2014-08-13 15:33:01.324967 2014-08-13 17:38:03.088377 7fe9bae887a0 15 read_log missing 194422'13570 (194422'13569) modify cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by client.1148203.0:9134 2014-08-13 15:33:00.869629 2014-08-13 17:38:03.088413 7fe9bae887a0 15 read_log missing 194422'13568 (194422'13567) modify 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by client.1148203.0:6412 2014-08-13 15:32:42.373457 2014-08-13 17:38:03.088450 7fe9bae887a0 15 read_log missing 194422'13566 (194422'13565) modify 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by client.1148203.0:6186 2014-08-13 15:32:41.139420 2014-08-13 17:38:03.088486 7fe9bae887a0 15 read_log missing 194422'13564 (194422'13563) modify 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by client.1148203.0:3301 2014-08-13 15:32:28.289735 2014-08-13 17:38:03.088522 7fe9bae887a0 15 read_log missing 194422'13562 (194422'13561) modify 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by client.1148203.0:3216 2014-08-13 15:32:27.910541 2014-08-13 17:38:03.088558 7fe9bae887a0 15 read_log missing 194422'13560 (194422'13559) modify ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by client.1148203.0:3129 2014-08-13 15:32:27.579892 2014-08-13 17:38:03.088596 7fe9bae887a0 15 read_log missing 194422'13558 (194422'13557) modify b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by client.1148203.0:660 2014-08-13 15:32:17.997715 2014-08-13 17:38:03.088633 7fe9bae887a0 15 read_log missing 194422'13556 (194422'13555) modify 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by client.1148203.0:357 2014-08-13 15:32:16.659486 2014-08-13 17:38:03.088669 7fe9bae887a0 15 read_log missing 194416'13554 (194416'13553) modify 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by client.2441958.0:15295 2014-08-13 15:29:31.592735 2014-08-13 17:38:03.088706 7fe9bae887a0 15 read_log missing 194416'13552 (194416'13551) modify f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by client.2441958.0:13520 2014-08-13 15:29:24.186479 2014-08-13 17:38:03.088743 7fe9bae887a0 15 read_log missing 194416'13550 (194416'13549) modify 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by client.2441958.0:11642 2014-08-13 15:29:16.070252 2014-08-13 17:38:03.088780 7fe9bae887a0 15 read_log missing 194416'13548 (194416'13547) modify 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by client.2441958.0:8931 2014-08-13 15:29:05.149794 2014-08-13 17:38:03.088817 7fe9bae887a0 15 read_log missing 194416'13546 (194416'13545) modify b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by client.2441958.0:7077 2014-08-13 15:28:58.055264 2014-08-13 17:38:03.088854 7fe9bae887a0 15 read_log missing 194416'13544 (194416'13543) modify 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by client.2441958.0:7012 2014-08-13 15:28:57.812126 2014-08-13 17:38:03.088891 7fe9bae887a0 15 read_log missing 194416'13542 (194416'13541) modify bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by client.2441958.0:6636 2014-08-13 15:28:56.420019 2014-08-13 17:38:03.088928 7fe9bae887a0 15 read_log missing 194416'13540 (194416'13539) modify ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by client.2441958.0:4007 2014-08-13 15:28:45.952340 2014-08-13 17:38:03.088965 7fe9bae887a0 15 read_log missing 194416'13538 (194416'13537) modify 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by client.2441958.0:3908 2014-08-13 15:28:45.577009 2014-08-13 17:38:03.089002 7fe9bae887a0 15 read_log missing 194416'13536 (194416'13535) modify b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by client.2441958.0:2439 2014-08-13 15:28:39.945728 2014-08-13 17:38:03.089038 7fe9bae887a0 15 read_log missing 194416'13534 (194416'13533) modify 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by client.2441958.0:448 2014-08-13 15:28:32.162246 2014-08-13 17:38:03.089075 7fe9bae887a0 15 read_log missing 194397'13532 (194397'13531) modify 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by client.2235069.0:797 2014-08-12 20:53:03.593214 2014-08-13 17:38:03.089112 7fe9bae887a0 15 read_log missing 194393'13530 (194393'13529) modify 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by client.1064422.0:218 2014-08-12 20:52:33.403536 2014-08-13 17:38:03.089149 7fe9bae887a0 15 read_log missing 194393'13528 (194393'13527) modify 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by client.1064095.0:3 2014-08-12 20:47:25.910100 2014-08-13 17:38:03.089192 7fe9bae887a0 15 read_log missing 194383'13526 (194383'13525) modify 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by client.2231190.0:6704 2014-08-12 20:34:07.114278 2014-08-13 17:38:03.089229 7fe9bae887a0 15 read_log missing 194383'13524 (194383'13523) modify 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by client.2231190.0:6262 2014-08-12 20:34:02.788571 2014-08-13 17:38:03.089266 7fe9bae887a0 15 read_log missing 194382'13522 (194382'13521) modify 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by client.2231190.0:5763 2014-08-12 20:33:59.966938 2014-08-13 17:38:03.089303 7fe9bae887a0 15 read_log missing 194382'13520 (194382'13519) modify 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by client.2231190.0:5693 2014-08-12 20:33:59.591804 2014-08-13 17:38:03.089339 7fe9bae887a0 15 read_log missing 194377'13518 (194377'13517) modify 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by client.2231190.0:4296 2014-08-12 20:33:15.688131 2014-08-13 17:38:03.089376 7fe9bae887a0 15 read_log missing 194377'13516 (194377'13515) modify 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by client.2231190.0:2109 2014-08-12 20:33:05.709251 2014-08-13 17:38:03.089412 7fe9bae887a0 15 read_log missing 194377'13514 (194377'13513) modify fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by client.2231190.0:2105 2014-08-12 20:33:05.692627 2014-08-13 17:38:03.089449 7fe9bae887a0 15 read_log missing 194377'13512 (194377'13511) modify b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by client.2231190.0:950 2014-08-12 20:33:01.106449 2014-08-13 17:38:03.089486 7fe9bae887a0 15 read_log missing 194370'13510 (194370'13509) modify fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by client.1508531.0:13865 2014-08-12 20:31:28.491484 2014-08-13 17:38:03.089523 7fe9bae887a0 15 read_log missing 194370'13508 (194370'13507) modify 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by client.1508531.0:13442 2014-08-12 20:31:23.580280 2014-08-13 17:38:03.089560 7fe9bae887a0 15 read_log missing 194370'13506 (194370'13505) modify 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by client.1508531.0:13431 2014-08-12 20:31:23.475320 2014-08-13 17:38:03.089596 7fe9bae887a0 15 read_log missing 194370'13504 (194370'13503) modify 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by client.1508531.0:11969 2014-08-12 20:31:11.129136 2014-08-13 17:38:03.089632 7fe9bae887a0 15 read_log missing 194370'13502 (194370'13501) modify 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by client.1508531.0:11600 2014-08-12 20:31:08.918385 2014-08-13 17:38:03.089669 7fe9bae887a0 15 read_log missing 194370'13500 (194370'13499) modify e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by client.1508531.0:11261 2014-08-12 20:31:06.852597 2014-08-13 17:38:03.089706 7fe9bae887a0 15 read_log missing 194370'13498 (194370'13497) modify d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by client.1508531.0:11177 2014-08-12 20:31:06.329571 2014-08-13 17:38:03.089747 7fe9bae887a0 15 read_log missing 194370'13496 (194370'13495) modify 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by client.1508531.0:10481 2014-08-12 20:31:02.100588 2014-08-13 17:38:03.089784 7fe9bae887a0 15 read_log missing 194370'13494 (194370'13493) modify 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by client.1508531.0:10253 2014-08-12 20:31:00.733201 2014-08-13 17:38:03.089821 7fe9bae887a0 15 read_log missing 194370'13492 (194370'13491) modify d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by client.1508531.0:8107 2014-08-12 20:30:48.149665 2014-08-13 17:38:03.089858 7fe9bae887a0 15 read_log missing 194370'13490 (194370'13489) modify 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by client.1508531.0:8084 2014-08-12 20:30:48.028390 2014-08-13 17:38:03.089895 7fe9bae887a0 15 read_log missing 194370'13488 (194370'13487) modify e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by client.1508531.0:7646 2014-08-12 20:30:45.662745 2014-08-13 17:38:03.089932 7fe9bae887a0 15 read_log missing 194370'13486 (194370'13485) modify 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by client.1508531.0:6729 2014-08-12 20:30:40.996084 2014-08-13 17:38:03.089968 7fe9bae887a0 15 read_log missing 194370'13484 (194370'13483) modify 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by client.1508531.0:4083 2014-08-12 20:30:29.596244 2014-08-13 17:38:03.090005 7fe9bae887a0 15 read_log missing 194370'13482 (194370'13481) modify 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by client.1508531.0:3896 2014-08-12 20:30:28.906404 2014-08-13 17:38:03.090042 7fe9bae887a0 15 read_log missing 194370'13480 (194370'13479) modify fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by client.1508531.0:2148 2014-08-12 20:30:22.447561 2014-08-13 17:38:03.090079 7fe9bae887a0 15 read_log missing 194370'13478 (194370'13477) modify ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by client.1508531.0:2028 2014-08-12 20:30:22.014253 2014-08-13 17:38:03.090090 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.090118 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=0 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive m=50] handle_loaded 2014-08-13 17:38:03.090135 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=0 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.014304 0 0.000000 2014-08-13 17:38:03.090147 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=0 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 17:38:03.090161 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=0 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194477 2014-08-13 17:38:03.090170 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log((192834'11428,194422'13576], crt=194422'13576) 2014-08-13 17:38:03.090185 7fe9bae887a0 10 osd.4 194478 pgid 24.39a coll 24.39a_head 2014-08-13 17:38:03.090304 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.39a 2014-08-13 17:38:03.090318 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.090351 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] enter Initial 2014-08-13 17:38:03.090362 7fe9bae887a0 20 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] enter NotTrimming 2014-08-13 17:38:03.090512 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.090610 7fe9bae887a0 20 read_log 192834'11375 (0'0) promote 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by osd.346.0:12983 2014-08-11 15:23:40.711310 2014-08-13 17:38:03.090631 7fe9bae887a0 20 read_log 192834'11376 (192834'11375) modify 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.933448.0:1300 2014-08-11 15:23:40.698804 2014-08-13 17:38:03.090645 7fe9bae887a0 20 read_log 192834'11425 (0'0) promote df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by osd.346.0:13033 2014-08-11 15:23:53.464058 2014-08-13 17:38:03.090657 7fe9bae887a0 20 read_log 192834'11426 (192834'11425) modify df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.933448.0:3754 2014-08-11 15:23:53.456241 2014-08-13 17:38:03.090669 7fe9bae887a0 20 read_log 192834'11427 (0'0) promote 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by osd.346.0:13035 2014-08-11 15:23:55.456867 2014-08-13 17:38:03.090681 7fe9bae887a0 20 read_log 192834'11428 (192834'11427) modify 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.933448.0:4098 2014-08-11 15:23:55.449029 2014-08-13 17:38:03.090694 7fe9bae887a0 20 read_log 192834'11453 (0'0) promote 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by osd.346.0:13061 2014-08-11 15:24:03.729715 2014-08-13 17:38:03.090706 7fe9bae887a0 20 read_log 192834'11454 (192834'11453) modify 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.933448.0:6000 2014-08-11 15:24:03.719699 2014-08-13 17:38:03.090731 7fe9bae887a0 20 read_log 192834'11495 (0'0) promote 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by osd.346.0:13103 2014-08-11 15:24:17.183348 2014-08-13 17:38:03.090744 7fe9bae887a0 20 read_log 192834'11496 (192834'11495) modify 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.933448.0:9009 2014-08-11 15:24:17.173764 2014-08-13 17:38:03.090757 7fe9bae887a0 20 read_log 192834'11505 (0'0) promote 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by osd.346.0:13113 2014-08-11 15:24:19.664646 2014-08-13 17:38:03.090769 7fe9bae887a0 20 read_log 192834'11506 (192834'11505) modify 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.933448.0:9646 2014-08-11 15:24:19.655719 2014-08-13 17:38:03.090781 7fe9bae887a0 20 read_log 192834'11525 (0'0) promote 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by osd.346.0:13133 2014-08-11 15:24:25.529612 2014-08-13 17:38:03.090794 7fe9bae887a0 20 read_log 192834'11526 (192834'11525) modify 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.933448.0:10863 2014-08-11 15:24:25.519540 2014-08-13 17:38:03.090806 7fe9bae887a0 20 read_log 192834'11527 (0'0) promote d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by osd.346.0:13135 2014-08-11 15:24:25.589303 2014-08-13 17:38:03.090818 7fe9bae887a0 20 read_log 192834'11528 (192834'11527) modify d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.933448.0:10880 2014-08-11 15:24:25.581648 2014-08-13 17:38:03.090830 7fe9bae887a0 20 read_log 192834'11529 (0'0) promote 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by osd.346.0:13137 2014-08-11 15:24:25.722823 2014-08-13 17:38:03.090842 7fe9bae887a0 20 read_log 192834'11530 (192834'11529) modify 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.933448.0:10915 2014-08-11 15:24:25.713221 2014-08-13 17:38:03.090854 7fe9bae887a0 20 read_log 192834'11533 (0'0) promote 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by osd.346.0:13141 2014-08-11 15:24:26.729678 2014-08-13 17:38:03.090866 7fe9bae887a0 20 read_log 192834'11534 (192834'11533) modify 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.933448.0:11177 2014-08-11 15:24:26.718603 2014-08-13 17:38:03.090878 7fe9bae887a0 20 read_log 192834'11543 (0'0) promote b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by osd.346.0:13151 2014-08-11 15:24:29.652401 2014-08-13 17:38:03.090890 7fe9bae887a0 20 read_log 192834'11544 (192834'11543) modify b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.933448.0:11857 2014-08-11 15:24:29.643421 2014-08-13 17:38:03.090903 7fe9bae887a0 20 read_log 192834'11565 (0'0) promote 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by osd.346.0:13173 2014-08-11 15:24:35.801867 2014-08-13 17:38:03.090915 7fe9bae887a0 20 read_log 192834'11566 (192834'11565) modify 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.933448.0:13193 2014-08-11 15:24:35.794114 2014-08-13 17:38:03.090927 7fe9bae887a0 20 read_log 192834'11567 (0'0) promote 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by osd.346.0:13175 2014-08-11 15:24:35.830566 2014-08-13 17:38:03.090940 7fe9bae887a0 20 read_log 192834'11568 (192834'11567) modify 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.933448.0:13201 2014-08-11 15:24:35.823288 2014-08-13 17:38:03.090953 7fe9bae887a0 20 read_log 192834'11575 (0'0) promote 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by osd.346.0:13183 2014-08-11 15:24:39.708983 2014-08-13 17:38:03.090965 7fe9bae887a0 20 read_log 192834'11576 (192834'11575) modify 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.933448.0:13719 2014-08-11 15:24:39.702123 2014-08-13 17:38:03.090984 7fe9bae887a0 20 read_log 192834'11581 (0'0) promote 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by osd.346.0:13189 2014-08-11 15:24:43.614896 2014-08-13 17:38:03.090997 7fe9bae887a0 20 read_log 192834'11582 (192834'11581) modify 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.933448.0:14284 2014-08-11 15:24:43.605540 2014-08-13 17:38:03.091009 7fe9bae887a0 20 read_log 192834'11595 (0'0) promote 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by osd.346.0:13203 2014-08-11 15:24:49.189960 2014-08-13 17:38:03.091021 7fe9bae887a0 20 read_log 192834'11596 (192834'11595) modify 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.933448.0:15313 2014-08-11 15:24:49.181594 2014-08-13 17:38:03.091033 7fe9bae887a0 20 read_log 192834'11599 (0'0) promote 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by osd.346.0:13207 2014-08-11 15:24:49.815373 2014-08-13 17:38:03.091045 7fe9bae887a0 20 read_log 192834'11600 (192834'11599) modify 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.933448.0:15481 2014-08-11 15:24:49.807741 2014-08-13 17:38:03.091057 7fe9bae887a0 20 read_log 192834'11603 (0'0) promote dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by osd.346.0:13211 2014-08-11 15:24:50.291943 2014-08-13 17:38:03.091069 7fe9bae887a0 20 read_log 192834'11604 (192834'11603) modify dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.933448.0:15594 2014-08-11 15:24:50.283454 2014-08-13 17:38:03.091081 7fe9bae887a0 20 read_log 192834'11647 (0'0) promote c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by osd.346.0:13255 2014-08-11 15:25:01.424029 2014-08-13 17:38:03.091093 7fe9bae887a0 20 read_log 192834'11648 (192834'11647) modify c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.933448.0:17761 2014-08-11 15:25:01.414991 2014-08-13 17:38:03.091105 7fe9bae887a0 20 read_log 192834'11659 (0'0) promote b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by osd.346.0:13267 2014-08-11 15:25:06.248281 2014-08-13 17:38:03.091117 7fe9bae887a0 20 read_log 192834'11660 (192834'11659) modify b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.933448.0:18664 2014-08-11 15:25:06.239065 2014-08-13 17:38:03.091129 7fe9bae887a0 20 read_log 192834'11699 (0'0) promote b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by osd.346.0:13307 2014-08-11 15:25:20.645598 2014-08-13 17:38:03.091141 7fe9bae887a0 20 read_log 192834'11700 (192834'11699) modify b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.933448.0:20367 2014-08-11 15:25:20.636700 2014-08-13 17:38:03.091159 7fe9bae887a0 20 read_log 192834'11703 (0'0) promote 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by osd.346.0:13311 2014-08-11 15:25:20.841904 2014-08-13 17:38:03.091173 7fe9bae887a0 20 read_log 192834'11704 (192834'11703) modify 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.933448.0:20417 2014-08-11 15:25:20.833713 2014-08-13 17:38:03.091185 7fe9bae887a0 20 read_log 192834'11717 (0'0) promote 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by osd.346.0:13325 2014-08-11 15:25:34.846995 2014-08-13 17:38:03.091197 7fe9bae887a0 20 read_log 192834'11718 (192834'11717) modify 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.933448.0:22168 2014-08-11 15:25:34.837381 2014-08-13 17:38:03.091210 7fe9bae887a0 20 read_log 192834'11731 (192834'11376) modify 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.933448.0:23591 2014-08-11 15:25:39.529856 2014-08-13 17:38:03.091223 7fe9bae887a0 20 read_log 192834'11756 (192834'11426) modify df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.933448.0:26045 2014-08-11 15:25:42.876839 2014-08-13 17:38:03.091242 7fe9bae887a0 20 read_log 192834'11757 (192834'11428) modify 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.933448.0:26389 2014-08-11 15:25:43.340436 2014-08-13 17:38:03.091255 7fe9bae887a0 20 read_log 192834'11770 (192834'11454) modify 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.933448.0:28291 2014-08-11 15:25:45.876388 2014-08-13 17:38:03.091267 7fe9bae887a0 20 read_log 192834'11791 (192834'11496) modify 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.933448.0:31300 2014-08-11 15:25:49.887739 2014-08-13 17:38:03.091279 7fe9bae887a0 20 read_log 192834'11796 (192834'11506) modify 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.933448.0:31937 2014-08-11 15:25:50.834765 2014-08-13 17:38:03.091316 7fe9bae887a0 20 read_log 192834'11806 (192834'11526) modify 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.933448.0:33154 2014-08-11 15:25:52.919630 2014-08-13 17:38:03.091329 7fe9bae887a0 20 read_log 192834'11807 (192834'11528) modify d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.933448.0:33171 2014-08-11 15:25:52.937715 2014-08-13 17:38:03.091341 7fe9bae887a0 20 read_log 192834'11808 (192834'11530) modify 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.933448.0:33206 2014-08-11 15:25:53.000575 2014-08-13 17:38:03.091353 7fe9bae887a0 20 read_log 192834'11810 (192834'11534) modify 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.933448.0:33468 2014-08-11 15:25:53.539928 2014-08-13 17:38:03.091365 7fe9bae887a0 20 read_log 192834'11815 (192834'11544) modify b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.933448.0:34148 2014-08-11 15:25:54.638271 2014-08-13 17:38:03.091377 7fe9bae887a0 20 read_log 192834'11826 (192834'11566) modify 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.933448.0:35484 2014-08-11 15:25:56.364536 2014-08-13 17:38:03.091406 7fe9bae887a0 20 read_log 192834'11827 (192834'11568) modify 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.933448.0:35492 2014-08-11 15:25:56.376410 2014-08-13 17:38:03.091418 7fe9bae887a0 20 read_log 192834'11831 (192834'11576) modify 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.933448.0:36010 2014-08-11 15:25:57.129164 2014-08-13 17:38:03.091430 7fe9bae887a0 20 read_log 192834'11834 (192834'11582) modify 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.933448.0:36575 2014-08-11 15:25:57.993090 2014-08-13 17:38:03.091444 7fe9bae887a0 20 read_log 192834'11841 (192834'11596) modify 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.933448.0:37604 2014-08-11 15:25:59.740883 2014-08-13 17:38:03.091456 7fe9bae887a0 20 read_log 192834'11843 (192834'11600) modify 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.933448.0:37772 2014-08-11 15:26:00.008805 2014-08-13 17:38:03.091469 7fe9bae887a0 20 read_log 192834'11845 (192834'11604) modify dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.933448.0:37885 2014-08-11 15:26:00.238894 2014-08-13 17:38:03.091481 7fe9bae887a0 20 read_log 192834'11867 (192834'11648) modify c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.933448.0:40052 2014-08-11 15:26:02.246842 2014-08-13 17:38:03.091493 7fe9bae887a0 20 read_log 192834'11873 (192834'11660) modify b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.933448.0:40955 2014-08-11 15:26:02.797810 2014-08-13 17:38:03.091505 7fe9bae887a0 20 read_log 192834'11893 (192834'11700) modify b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.933448.0:42658 2014-08-11 15:26:03.345963 2014-08-13 17:38:03.091517 7fe9bae887a0 20 read_log 192834'11895 (192834'11704) modify 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.933448.0:42708 2014-08-11 15:26:03.363717 2014-08-13 17:38:03.091540 7fe9bae887a0 20 read_log 192834'11902 (192834'11718) modify 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.933448.0:44459 2014-08-11 15:26:03.947531 2014-08-13 17:38:03.091553 7fe9bae887a0 20 read_log 192834'11909 (0'0) promote dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by osd.346.0:13517 2014-08-11 15:26:17.744773 2014-08-13 17:38:03.091565 7fe9bae887a0 20 read_log 192834'11910 (192834'11909) modify dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.1883634.0:893 2014-08-11 15:26:17.733910 2014-08-13 17:38:03.091578 7fe9bae887a0 20 read_log 192834'11915 (0'0) promote 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by osd.346.0:13523 2014-08-11 15:26:18.977754 2014-08-13 17:38:03.091590 7fe9bae887a0 20 read_log 192834'11916 (192834'11915) modify 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.1883634.0:1235 2014-08-11 15:26:18.970204 2014-08-13 17:38:03.091602 7fe9bae887a0 20 read_log 192834'11921 (0'0) promote fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by osd.346.0:13529 2014-08-11 15:26:19.771328 2014-08-13 17:38:03.091614 7fe9bae887a0 20 read_log 192834'11922 (192834'11921) modify fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.1883634.0:1455 2014-08-11 15:26:19.763655 2014-08-13 17:38:03.091626 7fe9bae887a0 20 read_log 192834'11923 (0'0) promote 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by osd.346.0:13531 2014-08-11 15:26:20.401680 2014-08-13 17:38:03.091638 7fe9bae887a0 20 read_log 192834'11924 (192834'11923) modify 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.1883634.0:1629 2014-08-11 15:26:20.393853 2014-08-13 17:38:03.091650 7fe9bae887a0 20 read_log 192834'11929 (0'0) promote 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by osd.346.0:13537 2014-08-11 15:26:22.737201 2014-08-13 17:38:03.091661 7fe9bae887a0 20 read_log 192834'11930 (192834'11929) modify 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.1883634.0:2235 2014-08-11 15:26:22.729464 2014-08-13 17:38:03.091674 7fe9bae887a0 20 read_log 192834'11935 (0'0) promote 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by osd.346.0:13543 2014-08-11 15:26:23.530659 2014-08-13 17:38:03.091686 7fe9bae887a0 20 read_log 192834'11936 (192834'11935) modify 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.1883634.0:2454 2014-08-11 15:26:23.522974 2014-08-13 17:38:03.091698 7fe9bae887a0 20 read_log 192834'11945 (0'0) promote 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by osd.346.0:13553 2014-08-11 15:26:25.618147 2014-08-13 17:38:03.091709 7fe9bae887a0 20 read_log 192834'11946 (192834'11945) modify 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.1883634.0:3043 2014-08-11 15:26:25.606464 2014-08-13 17:38:03.091721 7fe9bae887a0 20 read_log 192834'11955 (0'0) promote 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by osd.346.0:13563 2014-08-11 15:26:29.821144 2014-08-13 17:38:03.091733 7fe9bae887a0 20 read_log 192834'11956 (192834'11955) modify 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.1883634.0:4232 2014-08-11 15:26:29.812822 2014-08-13 17:38:03.091745 7fe9bae887a0 20 read_log 192834'11965 (0'0) promote 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by osd.346.0:13573 2014-08-11 15:26:33.056567 2014-08-13 17:38:03.091757 7fe9bae887a0 20 read_log 192834'11966 (192834'11965) modify 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.1883634.0:5054 2014-08-11 15:26:33.048315 2014-08-13 17:38:03.091769 7fe9bae887a0 20 read_log 192834'11971 (0'0) promote 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by osd.346.0:13579 2014-08-11 15:26:34.290114 2014-08-13 17:38:03.091804 7fe9bae887a0 20 read_log 192834'11972 (192834'11971) modify 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.1883634.0:5405 2014-08-11 15:26:34.279282 2014-08-13 17:38:03.091817 7fe9bae887a0 20 read_log 192834'11973 (0'0) promote 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by osd.346.0:13581 2014-08-11 15:26:35.263766 2014-08-13 17:38:03.091829 7fe9bae887a0 20 read_log 192834'11974 (192834'11973) modify 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.1883634.0:5679 2014-08-11 15:26:35.255206 2014-08-13 17:38:03.091841 7fe9bae887a0 20 read_log 192834'11985 (0'0) promote f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by osd.346.0:13593 2014-08-11 15:26:37.452258 2014-08-13 17:38:03.091853 7fe9bae887a0 20 read_log 192834'11986 (192834'11985) modify f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.1883634.0:6279 2014-08-11 15:26:37.442771 2014-08-13 17:38:03.091865 7fe9bae887a0 20 read_log 192834'11987 (0'0) promote a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by osd.346.0:13595 2014-08-11 15:26:40.775524 2014-08-13 17:38:03.091877 7fe9bae887a0 20 read_log 192834'11988 (192834'11987) modify a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.1883634.0:7089 2014-08-11 15:26:40.765805 2014-08-13 17:38:03.091889 7fe9bae887a0 20 read_log 192834'12005 (0'0) promote 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by osd.346.0:13613 2014-08-11 15:26:46.521197 2014-08-13 17:38:03.091901 7fe9bae887a0 20 read_log 192834'12006 (192834'12005) modify 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.1883634.0:8478 2014-08-11 15:26:46.512465 2014-08-13 17:38:03.091913 7fe9bae887a0 20 read_log 192834'12019 (0'0) promote f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by osd.346.0:13627 2014-08-11 15:26:50.866751 2014-08-13 17:38:03.091925 7fe9bae887a0 20 read_log 192834'12020 (192834'12019) modify f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.1883634.0:9438 2014-08-11 15:26:50.857121 2014-08-13 17:38:03.091937 7fe9bae887a0 20 read_log 192834'12023 (0'0) promote 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by osd.346.0:13631 2014-08-11 15:26:51.961872 2014-08-13 17:38:03.091949 7fe9bae887a0 20 read_log 192834'12024 (192834'12023) modify 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.1883634.0:9694 2014-08-11 15:26:51.953248 2014-08-13 17:38:03.091961 7fe9bae887a0 20 read_log 192834'12035 (0'0) promote 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by osd.346.0:13643 2014-08-11 15:26:55.599977 2014-08-13 17:38:03.091973 7fe9bae887a0 20 read_log 192834'12036 (192834'12035) modify 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.1883634.0:10552 2014-08-11 15:26:55.591405 2014-08-13 17:38:03.091985 7fe9bae887a0 20 read_log 192834'12037 (0'0) promote 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by osd.346.0:13645 2014-08-11 15:26:55.649085 2014-08-13 17:38:03.091997 7fe9bae887a0 20 read_log 192834'12038 (192834'12037) modify 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.1883634.0:10563 2014-08-11 15:26:55.641203 2014-08-13 17:38:03.092009 7fe9bae887a0 20 read_log 192834'12049 (0'0) promote 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by osd.346.0:13657 2014-08-11 15:27:00.230796 2014-08-13 17:38:03.092021 7fe9bae887a0 20 read_log 192834'12050 (192834'12049) modify 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.1883634.0:11527 2014-08-11 15:27:00.223244 2014-08-13 17:38:03.092033 7fe9bae887a0 20 read_log 192834'12063 (0'0) promote 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by osd.346.0:13671 2014-08-11 15:27:02.456859 2014-08-13 17:38:03.092052 7fe9bae887a0 20 read_log 192834'12064 (192834'12063) modify 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.1883634.0:12100 2014-08-11 15:27:02.449026 2014-08-13 17:38:03.092065 7fe9bae887a0 20 read_log 192834'12111 (0'0) promote 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by osd.346.0:13719 2014-08-11 15:27:16.840562 2014-08-13 17:38:03.092077 7fe9bae887a0 20 read_log 192834'12112 (192834'12111) modify 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.1883634.0:15051 2014-08-11 15:27:16.831873 2014-08-13 17:38:03.092089 7fe9bae887a0 20 read_log 192834'12147 (0'0) promote fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by osd.346.0:13755 2014-08-11 15:27:42.564008 2014-08-13 17:38:03.092101 7fe9bae887a0 20 read_log 192834'12148 (192834'12147) modify fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.1883634.0:18250 2014-08-11 15:27:42.553386 2014-08-13 17:38:03.092113 7fe9bae887a0 20 read_log 192834'12157 (0'0) promote 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by osd.346.0:13765 2014-08-11 15:27:51.504538 2014-08-13 17:38:03.092125 7fe9bae887a0 20 read_log 192834'12158 (192834'12157) modify 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.1883634.0:19355 2014-08-11 15:27:51.494524 2014-08-13 17:38:03.092137 7fe9bae887a0 20 read_log 192834'12175 (0'0) promote bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by osd.346.0:13783 2014-08-11 15:28:01.363956 2014-08-13 17:38:03.092149 7fe9bae887a0 20 read_log 192834'12176 (192834'12175) modify bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.1883634.0:20748 2014-08-11 15:28:01.354775 2014-08-13 17:38:03.092171 7fe9bae887a0 20 read_log 192834'12179 (0'0) promote af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by osd.346.0:13787 2014-08-11 15:28:01.680887 2014-08-13 17:38:03.092183 7fe9bae887a0 20 read_log 192834'12180 (192834'12179) modify af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.1883634.0:20823 2014-08-11 15:28:01.671719 2014-08-13 17:38:03.092195 7fe9bae887a0 20 read_log 192834'12205 (0'0) promote f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by osd.346.0:13813 2014-08-11 15:28:13.726265 2014-08-13 17:38:03.092207 7fe9bae887a0 20 read_log 192834'12206 (192834'12205) modify f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.1883634.0:23263 2014-08-11 15:28:13.717103 2014-08-13 17:38:03.092219 7fe9bae887a0 20 read_log 192834'12207 (0'0) promote 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by osd.346.0:13815 2014-08-11 15:28:13.765260 2014-08-13 17:38:03.092231 7fe9bae887a0 20 read_log 192834'12208 (192834'12207) modify 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.1883634.0:23270 2014-08-11 15:28:13.757325 2014-08-13 17:38:03.092244 7fe9bae887a0 20 read_log 192834'12214 (192834'11910) modify dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.1883634.0:24217 2014-08-11 15:28:18.554572 2014-08-13 17:38:03.092256 7fe9bae887a0 20 read_log 192834'12217 (192834'11916) modify 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.1883634.0:24559 2014-08-11 15:28:19.228978 2014-08-13 17:38:03.092268 7fe9bae887a0 20 read_log 192834'12220 (192834'11922) modify fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.1883634.0:24779 2014-08-11 15:28:19.509252 2014-08-13 17:38:03.092280 7fe9bae887a0 20 read_log 192834'12221 (192834'11924) modify 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.1883634.0:24953 2014-08-11 15:28:19.812831 2014-08-13 17:38:03.092292 7fe9bae887a0 20 read_log 192834'12224 (192834'11930) modify 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.1883634.0:25559 2014-08-11 15:28:21.541239 2014-08-13 17:38:03.092314 7fe9bae887a0 20 read_log 192834'12227 (192834'11936) modify 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.1883634.0:25778 2014-08-11 15:28:21.852665 2014-08-13 17:38:03.092327 7fe9bae887a0 20 read_log 192834'12232 (192834'11946) modify 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.1883634.0:26367 2014-08-11 15:28:23.036551 2014-08-13 17:38:03.092339 7fe9bae887a0 20 read_log 192834'12237 (192834'11956) modify 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.1883634.0:27556 2014-08-11 15:28:24.610524 2014-08-13 17:38:03.092351 7fe9bae887a0 20 read_log 192834'12242 (192834'11966) modify 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.1883634.0:28378 2014-08-11 15:28:25.738413 2014-08-13 17:38:03.092363 7fe9bae887a0 20 read_log 192834'12245 (192834'11972) modify 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.1883634.0:28729 2014-08-11 15:28:26.196393 2014-08-13 17:38:03.092375 7fe9bae887a0 20 read_log 192834'12246 (192834'11974) modify 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.1883634.0:29003 2014-08-11 15:28:26.604851 2014-08-13 17:38:03.092387 7fe9bae887a0 20 read_log 192834'12252 (192834'11986) modify f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.1883634.0:29603 2014-08-11 15:28:27.487015 2014-08-13 17:38:03.092399 7fe9bae887a0 20 read_log 192834'12253 (192834'11988) modify a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.1883634.0:30413 2014-08-11 15:28:28.609156 2014-08-13 17:38:03.092411 7fe9bae887a0 20 read_log 192834'12262 (192834'12006) modify 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.1883634.0:31802 2014-08-11 15:28:30.985331 2014-08-13 17:38:03.092423 7fe9bae887a0 20 read_log 192834'12269 (192834'12020) modify f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.1883634.0:32762 2014-08-11 15:28:32.279375 2014-08-13 17:38:03.092436 7fe9bae887a0 20 read_log 192834'12271 (192834'12024) modify 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.1883634.0:33018 2014-08-11 15:28:32.672533 2014-08-13 17:38:03.092449 7fe9bae887a0 20 read_log 192834'12277 (192834'12036) modify 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.1883634.0:33876 2014-08-11 15:28:33.932542 2014-08-13 17:38:03.092461 7fe9bae887a0 20 read_log 192834'12278 (192834'12038) modify 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.1883634.0:33887 2014-08-11 15:28:33.950175 2014-08-13 17:38:03.092472 7fe9bae887a0 20 read_log 192834'12284 (192834'12050) modify 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.1883634.0:34851 2014-08-11 15:28:35.401774 2014-08-13 17:38:03.092485 7fe9bae887a0 20 read_log 192834'12293 (192834'12064) modify 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.1883634.0:35424 2014-08-11 15:28:36.613371 2014-08-13 17:38:03.092497 7fe9bae887a0 20 read_log 192834'12317 (192834'12112) modify 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.1883634.0:38375 2014-08-11 15:28:40.557369 2014-08-13 17:38:03.092509 7fe9bae887a0 20 read_log 192834'12335 (192834'12148) modify fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.1883634.0:41574 2014-08-11 15:28:45.346539 2014-08-13 17:38:03.092521 7fe9bae887a0 20 read_log 192834'12340 (192834'12158) modify 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.1883634.0:42679 2014-08-11 15:28:46.320064 2014-08-13 17:38:03.092533 7fe9bae887a0 20 read_log 192834'12349 (192834'12176) modify bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.1883634.0:44072 2014-08-11 15:28:46.751928 2014-08-13 17:38:03.092545 7fe9bae887a0 20 read_log 192834'12351 (192834'12180) modify af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.1883634.0:44147 2014-08-11 15:28:46.774794 2014-08-13 17:38:03.092564 7fe9bae887a0 20 read_log 192834'12364 (192834'12206) modify f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.1883634.0:46587 2014-08-11 15:28:47.639470 2014-08-13 17:38:03.092581 7fe9bae887a0 20 read_log 192834'12365 (192834'12208) modify 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.1883634.0:46594 2014-08-11 15:28:47.642020 2014-08-13 17:38:03.092594 7fe9bae887a0 20 read_log 192888'12368 (0'0) promote 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by osd.53.0:1865 2014-08-11 16:44:55.383268 2014-08-13 17:38:03.092624 7fe9bae887a0 20 read_log 192888'12369 (192888'12368) modify 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.1315259.0:111 2014-08-11 16:44:55.374000 2014-08-13 17:38:03.092638 7fe9bae887a0 20 read_log 192888'12370 (0'0) promote a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by osd.53.0:1867 2014-08-11 16:44:55.400506 2014-08-13 17:38:03.092650 7fe9bae887a0 20 read_log 192888'12371 (192888'12370) modify a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.1315259.0:116 2014-08-11 16:44:55.393410 2014-08-13 17:38:03.092662 7fe9bae887a0 20 read_log 192888'12372 (0'0) promote d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by osd.53.0:1878 2014-08-11 16:44:57.833198 2014-08-13 17:38:03.092674 7fe9bae887a0 20 read_log 192888'12373 (192888'12372) modify d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.1315259.0:671 2014-08-11 16:44:57.824793 2014-08-13 17:38:03.092687 7fe9bae887a0 20 read_log 192888'12374 (0'0) promote 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by osd.53.0:1884 2014-08-11 16:44:59.015272 2014-08-13 17:38:03.092703 7fe9bae887a0 20 read_log 192888'12375 (192888'12374) modify 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.1315259.0:987 2014-08-11 16:44:59.007888 2014-08-13 17:38:03.092716 7fe9bae887a0 20 read_log 192888'12378 (0'0) promote f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by osd.53.0:1888 2014-08-11 16:44:59.342570 2014-08-13 17:38:03.092731 7fe9bae887a0 20 read_log 192888'12379 (192888'12378) modify f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.1315259.0:1075 2014-08-11 16:44:59.332433 2014-08-13 17:38:03.092744 7fe9bae887a0 20 read_log 192888'12386 (0'0) promote 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by osd.53.0:1915 2014-08-11 16:45:05.160933 2014-08-13 17:38:03.092758 7fe9bae887a0 20 read_log 192888'12387 (192888'12386) modify 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.1315259.0:2618 2014-08-11 16:45:05.153710 2014-08-13 17:38:03.092771 7fe9bae887a0 20 read_log 192888'12388 (0'0) promote d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by osd.53.0:1931 2014-08-11 16:45:10.552799 2014-08-13 17:38:03.092785 7fe9bae887a0 20 read_log 192888'12389 (192888'12388) modify d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.1315259.0:4024 2014-08-11 16:45:10.544237 2014-08-13 17:38:03.092797 7fe9bae887a0 20 read_log 192888'12392 (0'0) promote 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by osd.53.0:1949 2014-08-11 16:45:16.717728 2014-08-13 17:38:03.092811 7fe9bae887a0 20 read_log 192888'12393 (192888'12392) modify 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.1315259.0:5705 2014-08-11 16:45:16.710711 2014-08-13 17:38:03.092824 7fe9bae887a0 20 read_log 192888'12400 (0'0) promote c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by osd.53.0:1993 2014-08-11 16:45:25.889662 2014-08-13 17:38:03.092838 7fe9bae887a0 20 read_log 192888'12401 (192888'12400) modify c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.1315259.0:8235 2014-08-11 16:45:25.883037 2014-08-13 17:38:03.092859 7fe9bae887a0 20 read_log 192888'12402 (0'0) promote 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by osd.53.0:1999 2014-08-11 16:45:26.936857 2014-08-13 17:38:03.092873 7fe9bae887a0 20 read_log 192888'12403 (192888'12402) modify 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.1315259.0:8498 2014-08-11 16:45:26.928186 2014-08-13 17:38:03.092885 7fe9bae887a0 20 read_log 192888'12410 (0'0) promote 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by osd.53.0:2037 2014-08-11 16:45:34.052460 2014-08-13 17:38:03.092898 7fe9bae887a0 20 read_log 192888'12411 (192888'12410) modify 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.1315259.0:10365 2014-08-11 16:45:34.043356 2014-08-13 17:38:03.092911 7fe9bae887a0 20 read_log 192888'12416 (0'0) promote 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by osd.53.0:2059 2014-08-11 16:45:39.223933 2014-08-13 17:38:03.092923 7fe9bae887a0 20 read_log 192888'12417 (192888'12416) modify 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.1315259.0:11661 2014-08-11 16:45:39.217190 2014-08-13 17:38:03.092935 7fe9bae887a0 20 read_log 192888'12418 (0'0) promote eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by osd.53.0:2071 2014-08-11 16:45:41.471408 2014-08-13 17:38:03.092946 7fe9bae887a0 20 read_log 192888'12419 (192888'12418) modify eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.1315259.0:12201 2014-08-11 16:45:41.463546 2014-08-13 17:38:03.092959 7fe9bae887a0 20 read_log 192888'12420 (0'0) promote 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by osd.53.0:2075 2014-08-11 16:45:42.788268 2014-08-13 17:38:03.092971 7fe9bae887a0 20 read_log 192888'12421 (192888'12420) modify 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.1315259.0:12505 2014-08-11 16:45:42.781002 2014-08-13 17:38:03.092983 7fe9bae887a0 20 read_log 192888'12442 (0'0) promote 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by osd.53.0:2143 2014-08-11 16:45:58.763281 2014-08-13 17:38:03.092994 7fe9bae887a0 20 read_log 192888'12443 (192888'12442) modify 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.1315259.0:16592 2014-08-11 16:45:58.755273 2014-08-13 17:38:03.093006 7fe9bae887a0 20 read_log 192888'12444 (0'0) promote 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by osd.53.0:2147 2014-08-11 16:45:59.915992 2014-08-13 17:38:03.093018 7fe9bae887a0 20 read_log 192888'12445 (192888'12444) modify 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.1315259.0:16898 2014-08-11 16:45:59.908643 2014-08-13 17:38:03.093030 7fe9bae887a0 20 read_log 192888'12446 (0'0) promote f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by osd.53.0:2157 2014-08-11 16:46:01.580487 2014-08-13 17:38:03.093042 7fe9bae887a0 20 read_log 192888'12447 (192888'12446) modify f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.1315259.0:17320 2014-08-11 16:46:01.571507 2014-08-13 17:38:03.093054 7fe9bae887a0 20 read_log 192888'12448 (0'0) promote dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by osd.53.0:2161 2014-08-11 16:46:03.063227 2014-08-13 17:38:03.093066 7fe9bae887a0 20 read_log 192888'12449 (192888'12448) modify dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.1315259.0:17671 2014-08-11 16:46:03.055781 2014-08-13 17:38:03.093081 7fe9bae887a0 20 read_log 192888'12452 (0'0) promote e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by osd.53.0:2171 2014-08-11 16:46:03.993420 2014-08-13 17:38:03.093094 7fe9bae887a0 20 read_log 192888'12453 (192888'12452) modify e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.1315259.0:17891 2014-08-11 16:46:03.957026 2014-08-13 17:38:03.093117 7fe9bae887a0 20 read_log 192888'12454 (0'0) promote 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by osd.53.0:2173 2014-08-11 16:46:04.877779 2014-08-13 17:38:03.093132 7fe9bae887a0 20 read_log 192888'12455 (192888'12454) modify 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.1315259.0:18122 2014-08-11 16:46:04.870614 2014-08-13 17:38:03.093144 7fe9bae887a0 20 read_log 192888'12456 (0'0) promote 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by osd.53.0:2175 2014-08-11 16:46:04.903738 2014-08-13 17:38:03.093162 7fe9bae887a0 20 read_log 192888'12457 (192888'12456) modify 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.1315259.0:18129 2014-08-11 16:46:04.895979 2014-08-13 17:38:03.093177 7fe9bae887a0 20 read_log 192888'12460 (0'0) promote e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by osd.53.0:2211 2014-08-11 16:46:12.524070 2014-08-13 17:38:03.093189 7fe9bae887a0 20 read_log 192888'12461 (192888'12460) modify e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.1315259.0:20109 2014-08-11 16:46:12.512960 2014-08-13 17:38:03.093202 7fe9bae887a0 20 read_log 192888'12462 (0'0) promote 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by osd.53.0:2215 2014-08-11 16:46:13.572111 2014-08-13 17:38:03.093216 7fe9bae887a0 20 read_log 192888'12463 (192888'12462) modify 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.1315259.0:20395 2014-08-11 16:46:13.563930 2014-08-13 17:38:03.093228 7fe9bae887a0 20 read_log 192888'12466 (0'0) promote 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by osd.53.0:2225 2014-08-11 16:46:14.593998 2014-08-13 17:38:03.093240 7fe9bae887a0 20 read_log 192888'12467 (192888'12466) modify 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.1315259.0:20650 2014-08-11 16:46:14.585665 2014-08-13 17:38:03.093254 7fe9bae887a0 20 read_log 192888'12470 (0'0) promote d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by osd.53.0:2237 2014-08-11 16:46:17.978491 2014-08-13 17:38:03.093266 7fe9bae887a0 20 read_log 192888'12471 (192888'12470) modify d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.1315259.0:21548 2014-08-11 16:46:17.970388 2014-08-13 17:38:03.093278 7fe9bae887a0 20 read_log 192888'12474 (0'0) promote c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by osd.53.0:2253 2014-08-11 16:46:22.788946 2014-08-13 17:38:03.093292 7fe9bae887a0 20 read_log 192888'12475 (192888'12474) modify c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.1315259.0:22798 2014-08-11 16:46:22.781797 2014-08-13 17:38:03.093305 7fe9bae887a0 20 read_log 192888'12478 (0'0) promote b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by osd.53.0:2263 2014-08-11 16:46:25.000529 2014-08-13 17:38:03.093317 7fe9bae887a0 20 read_log 192888'12479 (192888'12478) modify b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.1315259.0:23342 2014-08-11 16:46:24.991539 2014-08-13 17:38:03.093331 7fe9bae887a0 20 read_log 192888'12482 (0'0) promote 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by osd.53.0:2279 2014-08-11 16:46:28.177916 2014-08-13 17:38:03.093343 7fe9bae887a0 20 read_log 192888'12483 (192888'12482) modify 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.1315259.0:24142 2014-08-11 16:46:28.170330 2014-08-13 17:38:03.093355 7fe9bae887a0 20 read_log 192888'12486 (0'0) promote 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by osd.53.0:2287 2014-08-11 16:46:30.158464 2014-08-13 17:38:03.093369 7fe9bae887a0 20 read_log 192888'12487 (192888'12486) modify 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.1315259.0:24660 2014-08-11 16:46:30.149948 2014-08-13 17:38:03.093389 7fe9bae887a0 20 read_log 192888'12494 (0'0) promote 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by osd.53.0:2323 2014-08-11 16:46:43.036669 2014-08-13 17:38:03.093424 7fe9bae887a0 20 read_log 192888'12495 (192888'12494) modify 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.1315259.0:27762 2014-08-11 16:46:43.027373 2014-08-13 17:38:03.093455 7fe9bae887a0 20 read_log 192888'12498 (0'0) promote 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by osd.53.0:2333 2014-08-11 16:46:46.836034 2014-08-13 17:38:03.093471 7fe9bae887a0 20 read_log 192888'12499 (192888'12498) modify 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.1315259.0:28689 2014-08-11 16:46:46.828603 2014-08-13 17:38:03.093483 7fe9bae887a0 20 read_log 192888'12502 (0'0) promote 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by osd.53.0:2359 2014-08-11 16:46:54.119376 2014-08-13 17:38:03.093495 7fe9bae887a0 20 read_log 192888'12503 (192888'12502) modify 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.1315259.0:30534 2014-08-11 16:46:54.111958 2014-08-13 17:38:03.093508 7fe9bae887a0 20 read_log 192888'12504 (192888'12369) modify 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.1315259.0:30838 2014-08-11 16:46:55.906571 2014-08-13 17:38:03.093520 7fe9bae887a0 20 read_log 192888'12505 (192888'12371) modify a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.1315259.0:30843 2014-08-11 16:46:55.908805 2014-08-13 17:38:03.093532 7fe9bae887a0 20 read_log 192888'12506 (192888'12373) modify d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.1315259.0:31398 2014-08-11 16:46:56.081015 2014-08-13 17:38:03.093564 7fe9bae887a0 20 read_log 192888'12507 (192888'12375) modify 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.1315259.0:31714 2014-08-11 16:46:56.178587 2014-08-13 17:38:03.093577 7fe9bae887a0 20 read_log 192888'12509 (192888'12379) modify f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.1315259.0:31802 2014-08-11 16:46:56.204657 2014-08-13 17:38:03.093589 7fe9bae887a0 20 read_log 192888'12513 (192888'12387) modify 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.1315259.0:33345 2014-08-11 16:46:56.613530 2014-08-13 17:38:03.093601 7fe9bae887a0 20 read_log 192888'12514 (192888'12389) modify d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.1315259.0:34751 2014-08-11 16:46:56.976364 2014-08-13 17:38:03.093613 7fe9bae887a0 20 read_log 192888'12516 (192888'12393) modify 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.1315259.0:36432 2014-08-11 16:46:57.420603 2014-08-13 17:38:03.093625 7fe9bae887a0 20 read_log 192888'12520 (192888'12401) modify c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.1315259.0:38962 2014-08-11 16:46:58.091769 2014-08-13 17:38:03.093637 7fe9bae887a0 20 read_log 192888'12521 (192888'12403) modify 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.1315259.0:39225 2014-08-11 16:46:58.156709 2014-08-13 17:38:03.093649 7fe9bae887a0 20 read_log 192888'12525 (192888'12411) modify 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.1315259.0:41092 2014-08-11 16:46:58.656324 2014-08-13 17:38:03.093661 7fe9bae887a0 20 read_log 192888'12528 (192888'12417) modify 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.1315259.0:42388 2014-08-11 16:46:58.997673 2014-08-13 17:38:03.093673 7fe9bae887a0 20 read_log 192888'12529 (192888'12419) modify eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.1315259.0:42928 2014-08-11 16:46:59.143980 2014-08-13 17:38:03.093686 7fe9bae887a0 20 read_log 192888'12530 (192888'12421) modify 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.1315259.0:43232 2014-08-11 16:46:59.229077 2014-08-13 17:38:03.093721 7fe9bae887a0 20 read_log 192888'12541 (192888'12443) modify 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.1315259.0:47319 2014-08-11 16:47:00.418691 2014-08-13 17:38:03.093737 7fe9bae887a0 20 read_log 192888'12542 (192888'12445) modify 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.1315259.0:47625 2014-08-11 16:47:00.516835 2014-08-13 17:38:03.093750 7fe9bae887a0 20 read_log 192888'12543 (192888'12447) modify f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.1315259.0:48047 2014-08-11 16:47:00.653680 2014-08-13 17:38:03.093762 7fe9bae887a0 20 read_log 192888'12544 (192888'12449) modify dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.1315259.0:48398 2014-08-11 16:47:00.753281 2014-08-13 17:38:03.093774 7fe9bae887a0 20 read_log 192888'12546 (192888'12453) modify e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.1315259.0:48618 2014-08-11 16:47:00.816175 2014-08-13 17:38:03.093786 7fe9bae887a0 20 read_log 192888'12547 (192888'12455) modify 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.1315259.0:48849 2014-08-11 16:47:00.885176 2014-08-13 17:38:03.093798 7fe9bae887a0 20 read_log 192888'12548 (192888'12457) modify 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.1315259.0:48856 2014-08-11 16:47:00.886137 2014-08-13 17:38:03.093810 7fe9bae887a0 20 read_log 192888'12550 (192888'12461) modify e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.1315259.0:50836 2014-08-11 16:47:01.493274 2014-08-13 17:38:03.093822 7fe9bae887a0 20 read_log 192888'12551 (192888'12463) modify 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.1315259.0:51122 2014-08-11 16:47:01.586388 2014-08-13 17:38:03.093834 7fe9bae887a0 20 read_log 192888'12553 (192888'12467) modify 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.1315259.0:51377 2014-08-11 16:47:01.662153 2014-08-13 17:38:03.093846 7fe9bae887a0 20 read_log 192888'12555 (192888'12471) modify d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.1315259.0:52275 2014-08-11 16:47:01.927022 2014-08-13 17:38:03.093858 7fe9bae887a0 20 read_log 192888'12557 (192888'12475) modify c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.1315259.0:53525 2014-08-11 16:47:02.277973 2014-08-13 17:38:03.093871 7fe9bae887a0 20 read_log 192888'12559 (192888'12479) modify b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.1315259.0:54069 2014-08-11 16:47:02.452238 2014-08-13 17:38:03.093883 7fe9bae887a0 20 read_log 192888'12561 (192888'12483) modify 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.1315259.0:54869 2014-08-11 16:47:02.741844 2014-08-13 17:38:03.093895 7fe9bae887a0 20 read_log 192888'12563 (192888'12487) modify 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.1315259.0:55387 2014-08-11 16:47:02.897703 2014-08-13 17:38:03.093907 7fe9bae887a0 20 read_log 192888'12567 (192888'12495) modify 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.1315259.0:58489 2014-08-11 16:47:03.817197 2014-08-13 17:38:03.093941 7fe9bae887a0 20 read_log 192888'12569 (192888'12499) modify 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.1315259.0:59416 2014-08-11 16:47:04.108973 2014-08-13 17:38:03.093956 7fe9bae887a0 20 read_log 192888'12571 (192888'12503) modify 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.1315259.0:61261 2014-08-11 16:47:04.687107 2014-08-13 17:38:03.093969 7fe9bae887a0 20 read_log 192888'12576 (0'0) promote 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by osd.53.0:2629 2014-08-11 17:05:20.442140 2014-08-13 17:38:03.093983 7fe9bae887a0 20 read_log 192888'12577 (192888'12576) modify 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.942868.0:1322 2014-08-11 17:05:20.434400 2014-08-13 17:38:03.094005 7fe9bae887a0 20 read_log 192888'12582 (0'0) promote 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by osd.53.0:2652 2014-08-11 17:05:27.824069 2014-08-13 17:38:03.094020 7fe9bae887a0 20 read_log 192888'12583 (192888'12582) modify 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.942868.0:3211 2014-08-11 17:05:27.815726 2014-08-13 17:38:03.094033 7fe9bae887a0 20 read_log 192888'12590 (0'0) promote d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by osd.53.0:2666 2014-08-11 17:05:33.055738 2014-08-13 17:38:03.094045 7fe9bae887a0 20 read_log 192888'12591 (192888'12590) modify d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.942868.0:4628 2014-08-11 17:05:33.049194 2014-08-13 17:38:03.094060 7fe9bae887a0 20 read_log 192888'12592 (0'0) promote 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by osd.53.0:2676 2014-08-11 17:05:34.826544 2014-08-13 17:38:03.094072 7fe9bae887a0 20 read_log 192888'12593 (192888'12592) modify 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.942868.0:5053 2014-08-11 17:05:34.819231 2014-08-13 17:38:03.094086 7fe9bae887a0 20 read_log 192888'12602 (0'0) promote 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by osd.53.0:2722 2014-08-11 17:05:42.594013 2014-08-13 17:38:03.094098 7fe9bae887a0 20 read_log 192888'12603 (192888'12602) modify 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.942868.0:7153 2014-08-11 17:05:42.585500 2014-08-13 17:38:03.094112 7fe9bae887a0 20 read_log 192888'12604 (0'0) promote ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by osd.53.0:2728 2014-08-11 17:05:44.379369 2014-08-13 17:38:03.094125 7fe9bae887a0 20 read_log 192888'12605 (192888'12604) modify ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.942868.0:7599 2014-08-11 17:05:44.372173 2014-08-13 17:38:03.094139 7fe9bae887a0 20 read_log 192888'12606 (0'0) promote 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by osd.53.0:2742 2014-08-11 17:05:48.884789 2014-08-13 17:38:03.094151 7fe9bae887a0 20 read_log 192888'12607 (192888'12606) modify 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.942868.0:8823 2014-08-11 17:05:48.875327 2014-08-13 17:38:03.094171 7fe9bae887a0 20 read_log 192888'12608 (0'0) promote 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by osd.53.0:2762 2014-08-11 17:05:52.292885 2014-08-13 17:38:03.094183 7fe9bae887a0 20 read_log 192888'12609 (192888'12608) modify 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.942868.0:9748 2014-08-11 17:05:52.284982 2014-08-13 17:38:03.094196 7fe9bae887a0 20 read_log 192888'12610 (0'0) promote 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by osd.53.0:2774 2014-08-11 17:05:54.794589 2014-08-13 17:38:03.094208 7fe9bae887a0 20 read_log 192888'12611 (192888'12610) modify 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.942868.0:10441 2014-08-11 17:05:54.787112 2014-08-13 17:38:03.094220 7fe9bae887a0 20 read_log 192888'12614 (0'0) promote d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by osd.53.0:2804 2014-08-11 17:06:01.913170 2014-08-13 17:38:03.094234 7fe9bae887a0 20 read_log 192888'12615 (192888'12614) modify d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.942868.0:12393 2014-08-11 17:06:01.904756 2014-08-13 17:38:03.094247 7fe9bae887a0 20 read_log 192888'12616 (0'0) promote 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by osd.53.0:2806 2014-08-11 17:06:02.871602 2014-08-13 17:38:03.094261 7fe9bae887a0 20 read_log 192888'12617 (192888'12616) modify 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.942868.0:12661 2014-08-11 17:06:02.862826 2014-08-13 17:38:03.094280 7fe9bae887a0 20 read_log 192888'12620 (0'0) promote 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by osd.53.0:2822 2014-08-11 17:06:06.222188 2014-08-13 17:38:03.094293 7fe9bae887a0 20 read_log 192888'12621 (192888'12620) modify 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.942868.0:13514 2014-08-11 17:06:06.213957 2014-08-13 17:38:03.094305 7fe9bae887a0 20 read_log 192888'12626 (0'0) promote 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by osd.53.0:2854 2014-08-11 17:06:12.924197 2014-08-13 17:38:03.094317 7fe9bae887a0 20 read_log 192888'12627 (192888'12626) modify 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.942868.0:15180 2014-08-11 17:06:12.916057 2014-08-13 17:38:03.094330 7fe9bae887a0 20 read_log 192888'12628 (0'0) promote e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by osd.53.0:2866 2014-08-11 17:06:14.561760 2014-08-13 17:38:03.094342 7fe9bae887a0 20 read_log 192888'12629 (192888'12628) modify e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.942868.0:15601 2014-08-11 17:06:14.555047 2014-08-13 17:38:03.094354 7fe9bae887a0 20 read_log 192888'12632 (0'0) promote 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by osd.53.0:2876 2014-08-11 17:06:16.480713 2014-08-13 17:38:03.094366 7fe9bae887a0 20 read_log 192888'12633 (192888'12632) modify 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.942868.0:16097 2014-08-11 17:06:16.473295 2014-08-13 17:38:03.094378 7fe9bae887a0 20 read_log 192888'12638 (0'0) promote ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by osd.53.0:2938 2014-08-11 17:06:31.295201 2014-08-13 17:38:03.094391 7fe9bae887a0 20 read_log 192888'12639 (192888'12638) modify ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.942868.0:19981 2014-08-11 17:06:31.286061 2014-08-13 17:38:03.094403 7fe9bae887a0 20 read_log 192888'12642 (0'0) promote 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by osd.53.0:2960 2014-08-11 17:06:35.019717 2014-08-13 17:38:03.094415 7fe9bae887a0 20 read_log 192888'12643 (192888'12642) modify 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.942868.0:20954 2014-08-11 17:06:35.010610 2014-08-13 17:38:03.094428 7fe9bae887a0 20 read_log 192888'12646 (0'0) promote 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by osd.53.0:2974 2014-08-11 17:06:39.333142 2014-08-13 17:38:03.094440 7fe9bae887a0 20 read_log 192888'12647 (192888'12646) modify 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.942868.0:22097 2014-08-11 17:06:39.325462 2014-08-13 17:38:03.094452 7fe9bae887a0 20 read_log 192888'12648 (0'0) promote 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by osd.53.0:2990 2014-08-11 17:06:44.015003 2014-08-13 17:38:03.094465 7fe9bae887a0 20 read_log 192888'12649 (192888'12648) modify 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.942868.0:23267 2014-08-11 17:06:44.006553 2014-08-13 17:38:03.094479 7fe9bae887a0 20 read_log 192888'12650 (0'0) promote fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by osd.53.0:3004 2014-08-11 17:06:45.912874 2014-08-13 17:38:03.094491 7fe9bae887a0 20 read_log 192888'12651 (192888'12650) modify fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.942868.0:23782 2014-08-11 17:06:45.904366 2014-08-13 17:38:03.094508 7fe9bae887a0 20 read_log 192888'12652 (0'0) promote 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by osd.53.0:3006 2014-08-11 17:06:46.120649 2014-08-13 17:38:03.094520 7fe9bae887a0 20 read_log 192888'12653 (192888'12652) modify 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.942868.0:23838 2014-08-11 17:06:46.112325 2014-08-13 17:38:03.094540 7fe9bae887a0 20 read_log 192888'12656 (0'0) promote c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by osd.53.0:3034 2014-08-11 17:06:52.462498 2014-08-13 17:38:03.094552 7fe9bae887a0 20 read_log 192888'12657 (192888'12656) modify c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.942868.0:25542 2014-08-11 17:06:52.454655 2014-08-13 17:38:03.094564 7fe9bae887a0 20 read_log 192888'12658 (0'0) promote c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by osd.53.0:3036 2014-08-11 17:06:52.477178 2014-08-13 17:38:03.094576 7fe9bae887a0 20 read_log 192888'12659 (192888'12658) modify c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.942868.0:25546 2014-08-11 17:06:52.469096 2014-08-13 17:38:03.094589 7fe9bae887a0 20 read_log 192888'12664 (0'0) promote 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by osd.53.0:3064 2014-08-11 17:07:01.564991 2014-08-13 17:38:03.094601 7fe9bae887a0 20 read_log 192888'12665 (192888'12664) modify 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.942868.0:27883 2014-08-11 17:07:01.556873 2014-08-13 17:38:03.094613 7fe9bae887a0 20 read_log 192888'12673 (192888'12577) modify 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.942868.0:32588 2014-08-11 17:07:16.294958 2014-08-13 17:38:03.094625 7fe9bae887a0 20 read_log 192888'12676 (192888'12583) modify 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.942868.0:34477 2014-08-11 17:07:16.801474 2014-08-13 17:38:03.094637 7fe9bae887a0 20 read_log 192888'12680 (192888'12591) modify d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.942868.0:35894 2014-08-11 17:07:17.191056 2014-08-13 17:38:03.094651 7fe9bae887a0 20 read_log 192888'12681 (192888'12593) modify 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.942868.0:36319 2014-08-11 17:07:17.296341 2014-08-13 17:38:03.094663 7fe9bae887a0 20 read_log 192888'12686 (192888'12603) modify 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.942868.0:38419 2014-08-11 17:07:17.882285 2014-08-13 17:38:03.094675 7fe9bae887a0 20 read_log 192888'12687 (192888'12605) modify ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.942868.0:38865 2014-08-11 17:07:18.012738 2014-08-13 17:38:03.094687 7fe9bae887a0 20 read_log 192888'12688 (192888'12607) modify 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.942868.0:40089 2014-08-11 17:07:18.361616 2014-08-13 17:38:03.094700 7fe9bae887a0 20 read_log 192888'12689 (192888'12609) modify 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.942868.0:41014 2014-08-11 17:07:18.610084 2014-08-13 17:38:03.094712 7fe9bae887a0 20 read_log 192888'12690 (192888'12611) modify 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.942868.0:41707 2014-08-11 17:07:18.802927 2014-08-13 17:38:03.094724 7fe9bae887a0 20 read_log 192888'12692 (192888'12615) modify d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.942868.0:43659 2014-08-11 17:07:19.335679 2014-08-13 17:38:03.094736 7fe9bae887a0 20 read_log 192888'12693 (192888'12617) modify 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.942868.0:43927 2014-08-11 17:07:19.416546 2014-08-13 17:38:03.094749 7fe9bae887a0 20 read_log 192888'12695 (192888'12621) modify 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.942868.0:44780 2014-08-11 17:07:19.660594 2014-08-13 17:38:03.094762 7fe9bae887a0 20 read_log 192888'12698 (192888'12627) modify 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.942868.0:46446 2014-08-11 17:07:20.123284 2014-08-13 17:38:03.094775 7fe9bae887a0 20 read_log 192888'12699 (192888'12629) modify e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.942868.0:46867 2014-08-11 17:07:20.239795 2014-08-13 17:38:03.094796 7fe9bae887a0 20 read_log 192888'12701 (192888'12633) modify 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.942868.0:47363 2014-08-11 17:07:20.382427 2014-08-13 17:38:03.094812 7fe9bae887a0 20 read_log 192888'12704 (192888'12639) modify ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.942868.0:51247 2014-08-11 17:07:21.584368 2014-08-13 17:38:03.094825 7fe9bae887a0 20 read_log 192888'12706 (192888'12643) modify 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.942868.0:52220 2014-08-11 17:07:21.874732 2014-08-13 17:38:03.094837 7fe9bae887a0 20 read_log 192888'12708 (192888'12647) modify 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.942868.0:53363 2014-08-11 17:07:22.239381 2014-08-13 17:38:03.094851 7fe9bae887a0 20 read_log 192888'12709 (192888'12649) modify 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.942868.0:54533 2014-08-11 17:07:22.584666 2014-08-13 17:38:03.094864 7fe9bae887a0 20 read_log 192888'12710 (192888'12651) modify fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.942868.0:55048 2014-08-11 17:07:22.726760 2014-08-13 17:38:03.094878 7fe9bae887a0 20 read_log 192888'12711 (192888'12653) modify 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.942868.0:55104 2014-08-11 17:07:22.744878 2014-08-13 17:38:03.094890 7fe9bae887a0 20 read_log 192888'12713 (192888'12657) modify c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.942868.0:56808 2014-08-11 17:07:23.264460 2014-08-13 17:38:03.094904 7fe9bae887a0 20 read_log 192888'12714 (192888'12659) modify c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.942868.0:56812 2014-08-11 17:07:23.266562 2014-08-13 17:38:03.094916 7fe9bae887a0 20 read_log 192888'12717 (192888'12665) modify 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.942868.0:59149 2014-08-11 17:07:23.959845 2014-08-13 17:38:03.094930 7fe9bae887a0 20 read_log 193305'12721 (0'0) promote 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by osd.325.0:19 2014-08-11 18:42:36.835277 2014-08-13 17:38:03.094943 7fe9bae887a0 20 read_log 193305'12722 (193305'12721) modify 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.951337.0:961 2014-08-11 18:42:36.826632 2014-08-13 17:38:03.094957 7fe9bae887a0 20 read_log 193305'12723 (0'0) promote a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by osd.325.0:21 2014-08-11 18:42:37.735078 2014-08-13 17:38:03.094969 7fe9bae887a0 20 read_log 193305'12724 (193305'12723) modify a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.951337.0:1210 2014-08-11 18:42:37.727210 2014-08-13 17:38:03.094981 7fe9bae887a0 20 read_log 193305'12725 (0'0) promote fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by osd.325.0:52 2014-08-11 18:42:44.294427 2014-08-13 17:38:03.094995 7fe9bae887a0 20 read_log 193305'12726 (193305'12725) modify fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.951337.0:2974 2014-08-11 18:42:44.286836 2014-08-13 17:38:03.095007 7fe9bae887a0 20 read_log 193305'12727 (0'0) promote 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by osd.325.0:64 2014-08-11 18:42:47.862278 2014-08-13 17:38:03.095022 7fe9bae887a0 20 read_log 193305'12728 (193305'12727) modify 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.951337.0:3954 2014-08-11 18:42:47.852246 2014-08-13 17:38:03.095035 7fe9bae887a0 20 read_log 193305'12729 (0'0) promote 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by osd.325.0:68 2014-08-11 18:42:48.124204 2014-08-13 17:38:03.095047 7fe9bae887a0 20 read_log 193305'12730 (193305'12729) modify 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.951337.0:4027 2014-08-11 18:42:48.115775 2014-08-13 17:38:03.095066 7fe9bae887a0 20 read_log 193305'12731 (0'0) promote 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by osd.325.0:88 2014-08-11 18:42:53.761103 2014-08-13 17:38:03.095081 7fe9bae887a0 20 read_log 193305'12732 (193305'12731) modify 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.951337.0:5542 2014-08-11 18:42:53.750412 2014-08-13 17:38:03.095093 7fe9bae887a0 20 read_log 193305'12733 (0'0) promote 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by osd.325.0:92 2014-08-11 18:42:54.152778 2014-08-13 17:38:03.095105 7fe9bae887a0 20 read_log 193305'12734 (193305'12733) modify 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.951337.0:5649 2014-08-11 18:42:54.145526 2014-08-13 17:38:03.095119 7fe9bae887a0 20 read_log 193305'12735 (0'0) promote 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by osd.325.0:108 2014-08-11 18:42:57.926559 2014-08-13 17:38:03.095132 7fe9bae887a0 20 read_log 193305'12736 (193305'12735) modify 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.951337.0:6690 2014-08-11 18:42:57.917772 2014-08-13 17:38:03.095144 7fe9bae887a0 20 read_log 193305'12737 (0'0) promote 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by osd.325.0:148 2014-08-11 18:43:07.498801 2014-08-13 17:38:03.095162 7fe9bae887a0 20 read_log 193305'12738 (193305'12737) modify 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.951337.0:9238 2014-08-11 18:43:07.486230 2014-08-13 17:38:03.095177 7fe9bae887a0 20 read_log 193305'12739 (0'0) promote 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by osd.325.0:190 2014-08-11 18:43:17.492263 2014-08-13 17:38:03.095206 7fe9bae887a0 20 read_log 193305'12740 (193305'12739) modify 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.951337.0:11889 2014-08-11 18:43:17.484481 2014-08-13 17:38:03.095221 7fe9bae887a0 20 read_log 193305'12741 (0'0) promote c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by osd.325.0:196 2014-08-11 18:43:18.906139 2014-08-13 17:38:03.095233 7fe9bae887a0 20 read_log 193305'12742 (193305'12741) modify c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.951337.0:12277 2014-08-11 18:43:18.897854 2014-08-13 17:38:03.095245 7fe9bae887a0 20 read_log 193305'12743 (0'0) promote 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by osd.325.0:198 2014-08-11 18:43:19.401694 2014-08-13 17:38:03.095257 7fe9bae887a0 20 read_log 193305'12744 (193305'12743) modify 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.951337.0:12413 2014-08-11 18:43:19.394021 2014-08-13 17:38:03.095269 7fe9bae887a0 20 read_log 193305'12745 (0'0) promote 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by osd.325.0:200 2014-08-11 18:43:20.149984 2014-08-13 17:38:03.095281 7fe9bae887a0 20 read_log 193305'12746 (193305'12745) modify 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.951337.0:12614 2014-08-11 18:43:20.142676 2014-08-13 17:38:03.095293 7fe9bae887a0 20 read_log 193305'12747 (0'0) promote f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by osd.325.0:217 2014-08-11 18:43:23.957693 2014-08-13 17:38:03.095306 7fe9bae887a0 20 read_log 193305'12748 (193305'12747) modify f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.951337.0:13604 2014-08-11 18:43:23.949435 2014-08-13 17:38:03.095318 7fe9bae887a0 20 read_log 193305'12749 (0'0) promote cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by osd.325.0:227 2014-08-11 18:43:25.855570 2014-08-13 17:38:03.095330 7fe9bae887a0 20 read_log 193305'12750 (193305'12749) modify cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.951337.0:14127 2014-08-11 18:43:25.846069 2014-08-13 17:38:03.095350 7fe9bae887a0 20 read_log 193305'12751 (0'0) promote b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by osd.325.0:362 2014-08-11 18:43:55.502763 2014-08-13 17:38:03.095363 7fe9bae887a0 20 read_log 193305'12752 (193305'12751) modify b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.951337.0:22454 2014-08-11 18:43:55.493161 2014-08-13 17:38:03.095375 7fe9bae887a0 20 read_log 193305'12753 (0'0) promote b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by osd.325.0:366 2014-08-11 18:43:56.487628 2014-08-13 17:38:03.095387 7fe9bae887a0 20 read_log 193305'12754 (193305'12753) modify b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.951337.0:22727 2014-08-11 18:43:56.479265 2014-08-13 17:38:03.095398 7fe9bae887a0 20 read_log 193305'12755 (0'0) promote 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by osd.325.0:370 2014-08-11 18:43:57.637324 2014-08-13 17:38:03.095410 7fe9bae887a0 20 read_log 193305'12756 (193305'12755) modify 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.951337.0:23056 2014-08-11 18:43:57.628972 2014-08-13 17:38:03.095422 7fe9bae887a0 20 read_log 193305'12757 (0'0) promote feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by osd.325.0:382 2014-08-11 18:43:59.095918 2014-08-13 17:38:03.095434 7fe9bae887a0 20 read_log 193305'12758 (193305'12757) modify feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.951337.0:23472 2014-08-11 18:43:59.086354 2014-08-13 17:38:03.095446 7fe9bae887a0 20 read_log 193305'12759 (0'0) promote f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by osd.325.0:420 2014-08-11 18:44:05.435034 2014-08-13 17:38:03.095458 7fe9bae887a0 20 read_log 193305'12760 (193305'12759) modify f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.951337.0:25217 2014-08-11 18:44:05.425805 2014-08-13 17:38:03.095470 7fe9bae887a0 20 read_log 193305'12761 (0'0) promote e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by osd.325.0:444 2014-08-11 18:44:15.175379 2014-08-13 17:38:03.095482 7fe9bae887a0 20 read_log 193305'12762 (193305'12761) modify e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.951337.0:27865 2014-08-11 18:44:15.166186 2014-08-13 17:38:03.095494 7fe9bae887a0 20 read_log 193305'12763 (0'0) promote 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by osd.325.0:450 2014-08-11 18:44:17.757383 2014-08-13 17:38:03.095506 7fe9bae887a0 20 read_log 193305'12764 (193305'12763) modify 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.951337.0:28576 2014-08-11 18:44:17.749603 2014-08-13 17:38:03.095518 7fe9bae887a0 20 read_log 193305'12765 (0'0) promote 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by osd.325.0:480 2014-08-11 18:44:21.215262 2014-08-13 17:38:03.095530 7fe9bae887a0 20 read_log 193305'12766 (193305'12765) modify 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.951337.0:29513 2014-08-11 18:44:21.205954 2014-08-13 17:38:03.095542 7fe9bae887a0 20 read_log 193305'12767 (0'0) promote 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by osd.325.0:490 2014-08-11 18:44:26.079118 2014-08-13 17:38:03.095554 7fe9bae887a0 20 read_log 193305'12768 (193305'12767) modify 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.951337.0:30790 2014-08-11 18:44:26.066081 2014-08-13 17:38:03.095587 7fe9bae887a0 20 read_log 193305'12769 (193305'12722) modify 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.951337.0:33672 2014-08-11 18:44:34.572021 2014-08-13 17:38:03.095599 7fe9bae887a0 20 read_log 193305'12770 (193305'12724) modify a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.951337.0:33921 2014-08-11 18:44:34.717676 2014-08-13 17:38:03.095661 7fe9bae887a0 20 read_log 193305'12771 (193305'12726) modify fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.951337.0:35685 2014-08-11 18:44:35.702476 2014-08-13 17:38:03.095678 7fe9bae887a0 20 read_log 193305'12772 (193305'12728) modify 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.951337.0:36665 2014-08-11 18:44:36.463487 2014-08-13 17:38:03.095691 7fe9bae887a0 20 read_log 193305'12773 (193305'12730) modify 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.951337.0:36738 2014-08-11 18:44:36.500962 2014-08-13 17:38:03.095703 7fe9bae887a0 20 read_log 193305'12774 (193305'12732) modify 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.951337.0:38253 2014-08-11 18:44:37.305858 2014-08-13 17:38:03.095715 7fe9bae887a0 20 read_log 193305'12775 (193305'12734) modify 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.951337.0:38360 2014-08-11 18:44:37.366464 2014-08-13 17:38:03.095728 7fe9bae887a0 20 read_log 193305'12776 (193305'12736) modify 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.951337.0:39401 2014-08-11 18:44:37.988147 2014-08-13 17:38:03.095740 7fe9bae887a0 20 read_log 193305'12777 (193305'12738) modify 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.951337.0:41949 2014-08-11 18:44:38.940231 2014-08-13 17:38:03.095752 7fe9bae887a0 20 read_log 193305'12778 (193305'12740) modify 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.951337.0:44600 2014-08-11 18:44:39.928990 2014-08-13 17:38:03.095764 7fe9bae887a0 20 read_log 193305'12779 (193305'12742) modify c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.951337.0:44988 2014-08-11 18:44:40.059738 2014-08-13 17:38:03.095776 7fe9bae887a0 20 read_log 193305'12780 (193305'12744) modify 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.951337.0:45124 2014-08-11 18:44:40.096802 2014-08-13 17:38:03.095788 7fe9bae887a0 20 read_log 193305'12781 (193305'12746) modify 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.951337.0:45325 2014-08-11 18:44:40.163551 2014-08-13 17:38:03.095801 7fe9bae887a0 20 read_log 193305'12782 (193305'12748) modify f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.951337.0:46315 2014-08-11 18:44:40.448619 2014-08-13 17:38:03.095813 7fe9bae887a0 20 read_log 193305'12783 (193305'12750) modify cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.951337.0:46838 2014-08-11 18:44:40.602189 2014-08-13 17:38:03.095825 7fe9bae887a0 20 read_log 193305'12784 (193305'12752) modify b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.951337.0:55165 2014-08-11 18:44:43.042363 2014-08-13 17:38:03.095837 7fe9bae887a0 20 read_log 193305'12785 (193305'12754) modify b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.951337.0:55438 2014-08-11 18:44:43.123289 2014-08-13 17:38:03.095849 7fe9bae887a0 20 read_log 193305'12786 (193305'12756) modify 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.951337.0:55767 2014-08-11 18:44:43.225264 2014-08-13 17:38:03.095861 7fe9bae887a0 20 read_log 193305'12787 (193305'12758) modify feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.951337.0:56183 2014-08-11 18:44:43.347657 2014-08-13 17:38:03.095875 7fe9bae887a0 20 read_log 193305'12788 (193305'12760) modify f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.951337.0:57928 2014-08-11 18:44:43.897261 2014-08-13 17:38:03.095887 7fe9bae887a0 20 read_log 193305'12789 (193305'12762) modify e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.951337.0:60576 2014-08-11 18:44:44.669363 2014-08-13 17:38:03.095899 7fe9bae887a0 20 read_log 193305'12790 (193305'12764) modify 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.951337.0:61287 2014-08-11 18:44:44.879334 2014-08-13 17:38:03.095919 7fe9bae887a0 20 read_log 193305'12791 (193305'12766) modify 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.951337.0:62224 2014-08-11 18:44:45.162413 2014-08-13 17:38:03.095932 7fe9bae887a0 20 read_log 193305'12792 (193305'12768) modify 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.951337.0:63501 2014-08-11 18:44:45.526454 2014-08-13 17:38:03.095945 7fe9bae887a0 20 read_log 194215'12793 (194215'12793) modify 39a/hit_set_24.39a_archive_2014-08-12 15:32:25.737468_2014-08-12 15:50:26.492943/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:26.493191 2014-08-13 17:38:03.095957 7fe9bae887a0 20 read_log 194215'12794 (192834'11807) clean d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by osd.325.0:2318 2014-08-12 15:50:26.498736 2014-08-13 17:38:03.095970 7fe9bae887a0 20 read_log 194215'12795 (194215'12794) delete d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.2174595.0:190441 0.000000 2014-08-13 17:38:03.095981 7fe9bae887a0 20 read_log 194215'12796 (192834'12293) clean 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by osd.325.0:2320 2014-08-12 15:50:26.530490 2014-08-13 17:38:03.095993 7fe9bae887a0 20 read_log 194215'12797 (194215'12796) delete 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.2174595.0:190443 0.000000 2014-08-13 17:38:03.096004 7fe9bae887a0 20 read_log 194215'12798 (193305'12770) clean a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by osd.325.0:2322 2014-08-12 15:50:26.560218 2014-08-13 17:38:03.096016 7fe9bae887a0 20 read_log 194215'12799 (194215'12798) delete a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.2174595.0:190445 0.000000 2014-08-13 17:38:03.096027 7fe9bae887a0 20 read_log 194215'12800 (192888'12688) clean 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by osd.325.0:2324 2014-08-12 15:50:26.573778 2014-08-13 17:38:03.096039 7fe9bae887a0 20 read_log 194215'12801 (194215'12800) delete 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.2174595.0:190447 0.000000 2014-08-13 17:38:03.096050 7fe9bae887a0 20 read_log 194215'12802 (192888'12553) clean 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by osd.325.0:2326 2014-08-12 15:50:26.587278 2014-08-13 17:38:03.096062 7fe9bae887a0 20 read_log 194215'12803 (194215'12802) delete 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.2174595.0:190449 0.000000 2014-08-13 17:38:03.096073 7fe9bae887a0 20 read_log 194215'12804 (192834'12284) clean 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by osd.325.0:2328 2014-08-12 15:50:26.602366 2014-08-13 17:38:03.096085 7fe9bae887a0 20 read_log 194215'12805 (194215'12804) delete 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.2174595.0:190451 0.000000 2014-08-13 17:38:03.096095 7fe9bae887a0 20 read_log 194215'12806 (193305'12776) clean 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by osd.325.0:2330 2014-08-12 15:50:26.616354 2014-08-13 17:38:03.096107 7fe9bae887a0 20 read_log 194215'12807 (194215'12806) delete 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.2174595.0:190453 0.000000 2014-08-13 17:38:03.096118 7fe9bae887a0 20 read_log 194215'12808 (192888'12689) clean 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by osd.325.0:2332 2014-08-12 15:50:26.630632 2014-08-13 17:38:03.096130 7fe9bae887a0 20 read_log 194215'12809 (194215'12808) delete 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.2174595.0:190455 0.000000 2014-08-13 17:38:03.096148 7fe9bae887a0 20 read_log 194215'12810 (192888'12507) clean 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by osd.325.0:2334 2014-08-12 15:50:26.646704 2014-08-13 17:38:03.096173 7fe9bae887a0 20 read_log 194215'12811 (194215'12810) delete 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.2174595.0:190457 0.000000 2014-08-13 17:38:03.096185 7fe9bae887a0 20 read_log 194215'12812 (192834'12227) clean 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by osd.325.0:2336 2014-08-12 15:50:26.659809 2014-08-13 17:38:03.096197 7fe9bae887a0 20 read_log 194215'12813 (194215'12812) delete 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.2174595.0:190459 0.000000 2014-08-13 17:38:03.096208 7fe9bae887a0 20 read_log 194215'12814 (192834'11902) clean 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by osd.325.0:2338 2014-08-12 15:50:26.673679 2014-08-13 17:38:03.096220 7fe9bae887a0 20 read_log 194215'12815 (194215'12814) delete 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.2174595.0:190461 0.000000 2014-08-13 17:38:03.096231 7fe9bae887a0 20 read_log 194215'12816 (192834'12220) clean fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by osd.325.0:2340 2014-08-12 15:50:26.688533 2014-08-13 17:38:03.096243 7fe9bae887a0 20 read_log 194215'12817 (194215'12816) delete fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.2174595.0:190463 0.000000 2014-08-13 17:38:03.096254 7fe9bae887a0 20 read_log 194215'12818 (192834'12245) clean 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by osd.325.0:2342 2014-08-12 15:50:26.703366 2014-08-13 17:38:03.096266 7fe9bae887a0 20 read_log 194215'12819 (194215'12818) delete 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.2174595.0:190465 0.000000 2014-08-13 17:38:03.096277 7fe9bae887a0 20 read_log 194215'12820 (193305'12769) clean 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by osd.325.0:2344 2014-08-12 15:50:26.718586 2014-08-13 17:38:03.096289 7fe9bae887a0 20 read_log 194215'12821 (194215'12820) delete 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.2174595.0:190467 0.000000 2014-08-13 17:38:03.096300 7fe9bae887a0 20 read_log 194215'12822 (192834'11810) clean 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by osd.325.0:2346 2014-08-12 15:50:26.731141 2014-08-13 17:38:03.096312 7fe9bae887a0 20 read_log 194215'12823 (194215'12822) delete 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.2174595.0:190469 0.000000 2014-08-13 17:38:03.096323 7fe9bae887a0 20 read_log 194215'12824 (193305'12773) clean 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by osd.325.0:2348 2014-08-12 15:50:26.743517 2014-08-13 17:38:03.096335 7fe9bae887a0 20 read_log 194215'12825 (194215'12824) delete 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.2174595.0:190471 0.000000 2014-08-13 17:38:03.096346 7fe9bae887a0 20 read_log 194215'12826 (192834'12262) clean 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by osd.325.0:2350 2014-08-12 15:50:26.757645 2014-08-13 17:38:03.096360 7fe9bae887a0 20 read_log 194215'12827 (194215'12826) delete 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.2174595.0:190473 0.000000 2014-08-13 17:38:03.096371 7fe9bae887a0 20 read_log 194215'12828 (192888'12557) clean c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by osd.325.0:2352 2014-08-12 15:50:26.773343 2014-08-13 17:38:03.096384 7fe9bae887a0 20 read_log 194215'12829 (194215'12828) delete c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.2174595.0:190475 0.000000 2014-08-13 17:38:03.096404 7fe9bae887a0 20 read_log 194215'12830 (193305'12783) clean cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by osd.325.0:2354 2014-08-12 15:50:26.788736 2014-08-13 17:38:03.096417 7fe9bae887a0 20 read_log 194215'12831 (194215'12830) delete cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.2174595.0:190477 0.000000 2014-08-13 17:38:03.096428 7fe9bae887a0 20 read_log 194215'12832 (192888'12714) clean c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by osd.325.0:2356 2014-08-12 15:50:26.801139 2014-08-13 17:38:03.096461 7fe9bae887a0 20 read_log 194215'12833 (194215'12832) delete c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.2174595.0:190479 0.000000 2014-08-13 17:38:03.096472 7fe9bae887a0 20 read_log 194215'12834 (192834'12269) clean f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by osd.325.0:2358 2014-08-12 15:50:26.815053 2014-08-13 17:38:03.096485 7fe9bae887a0 20 read_log 194215'12835 (194215'12834) delete f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.2174595.0:190481 0.000000 2014-08-13 17:38:03.096496 7fe9bae887a0 20 read_log 194215'12836 (192834'11756) clean df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by osd.325.0:2360 2014-08-12 15:50:26.829789 2014-08-13 17:38:03.096508 7fe9bae887a0 20 read_log 194215'12837 (194215'12836) delete df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.2174595.0:190483 0.000000 2014-08-13 17:38:03.096519 7fe9bae887a0 20 read_log 194215'12838 (193305'12780) clean 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by osd.325.0:2362 2014-08-12 15:50:26.844025 2014-08-13 17:38:03.096531 7fe9bae887a0 20 read_log 194215'12839 (194215'12838) delete 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.2174595.0:190485 0.000000 2014-08-13 17:38:03.096541 7fe9bae887a0 20 read_log 194215'12840 (192888'12528) clean 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by osd.325.0:2364 2014-08-12 15:50:26.889538 2014-08-13 17:38:03.096553 7fe9bae887a0 20 read_log 194215'12841 (194215'12840) delete 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.2174595.0:190487 0.000000 2014-08-13 17:38:03.096564 7fe9bae887a0 20 read_log 194215'12842 (192834'12252) clean f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by osd.325.0:2366 2014-08-12 15:50:26.904158 2014-08-13 17:38:03.096576 7fe9bae887a0 20 read_log 194215'12843 (194215'12842) delete f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.2174595.0:190489 0.000000 2014-08-13 17:38:03.096587 7fe9bae887a0 20 read_log 194215'12844 (192888'12514) clean d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by osd.325.0:2368 2014-08-12 15:50:26.919116 2014-08-13 17:38:03.096599 7fe9bae887a0 20 read_log 194215'12845 (194215'12844) delete d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.2174595.0:190491 0.000000 2014-08-13 17:38:03.096609 7fe9bae887a0 20 read_log 194215'12846 (193305'12774) clean 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by osd.325.0:2370 2014-08-12 15:50:26.932619 2014-08-13 17:38:03.096621 7fe9bae887a0 20 read_log 194215'12847 (194215'12846) delete 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.2174595.0:190493 0.000000 2014-08-13 17:38:03.096631 7fe9bae887a0 20 read_log 194215'12848 (192834'11831) clean 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by osd.325.0:2372 2014-08-12 15:50:26.945728 2014-08-13 17:38:03.096646 7fe9bae887a0 20 read_log 194215'12849 (194215'12848) delete 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.2174595.0:190495 0.000000 2014-08-13 17:38:03.096664 7fe9bae887a0 20 read_log 194215'12850 (192834'11867) clean c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by osd.325.0:2374 2014-08-12 15:50:26.960969 2014-08-13 17:38:03.096678 7fe9bae887a0 20 read_log 194215'12851 (194215'12850) delete c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.2174595.0:190497 0.000000 2014-08-13 17:38:03.096689 7fe9bae887a0 20 read_log 194215'12852 (192834'11815) clean b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by osd.325.0:2376 2014-08-12 15:50:26.974412 2014-08-13 17:38:03.096702 7fe9bae887a0 20 read_log 194215'12853 (194215'12852) delete b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.2174595.0:190499 0.000000 2014-08-13 17:38:03.096712 7fe9bae887a0 20 read_log 194215'12854 (193305'12791) clean 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by osd.325.0:2378 2014-08-12 15:50:26.988834 2014-08-13 17:38:03.096724 7fe9bae887a0 20 read_log 194215'12855 (194215'12854) delete 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.2174595.0:190501 0.000000 2014-08-13 17:38:03.096735 7fe9bae887a0 20 read_log 194215'12856 (192888'12513) clean 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by osd.325.0:2380 2014-08-12 15:50:27.001361 2014-08-13 17:38:03.096747 7fe9bae887a0 20 read_log 194215'12857 (194215'12856) delete 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.2174595.0:190503 0.000000 2014-08-13 17:38:03.096758 7fe9bae887a0 20 read_log 194215'12858 (193305'12790) clean 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by osd.325.0:2382 2014-08-12 15:50:27.015826 2014-08-13 17:38:03.096770 7fe9bae887a0 20 read_log 194215'12859 (194215'12858) delete 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.2174595.0:190505 0.000000 2014-08-13 17:38:03.096781 7fe9bae887a0 20 read_log 194215'12860 (192834'11893) clean b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by osd.325.0:2384 2014-08-12 15:50:27.030439 2014-08-13 17:38:03.096793 7fe9bae887a0 20 read_log 194215'12861 (194215'12860) delete b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.2174595.0:190507 0.000000 2014-08-13 17:38:03.096803 7fe9bae887a0 20 read_log 194215'12862 (192834'12224) clean 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by osd.325.0:2386 2014-08-12 15:50:27.043700 2014-08-13 17:38:03.096815 7fe9bae887a0 20 read_log 194215'12863 (194215'12862) delete 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.2174595.0:190509 0.000000 2014-08-13 17:38:03.096826 7fe9bae887a0 20 read_log 194215'12864 (192888'12516) clean 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by osd.325.0:2388 2014-08-12 15:50:27.058018 2014-08-13 17:38:03.096838 7fe9bae887a0 20 read_log 194215'12865 (194215'12864) delete 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.2174595.0:190511 0.000000 2014-08-13 17:38:03.096849 7fe9bae887a0 20 read_log 194215'12866 (192888'12692) clean d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by osd.325.0:2390 2014-08-12 15:50:27.071713 2014-08-13 17:38:03.096861 7fe9bae887a0 20 read_log 194215'12867 (194215'12866) delete d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.2174595.0:190513 0.000000 2014-08-13 17:38:03.096872 7fe9bae887a0 20 read_log 194215'12868 (192834'11845) clean dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by osd.325.0:2392 2014-08-12 15:50:27.087665 2014-08-13 17:38:03.096884 7fe9bae887a0 20 read_log 194215'12869 (194215'12868) delete dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.2174595.0:190515 0.000000 2014-08-13 17:38:03.096901 7fe9bae887a0 20 read_log 194215'12870 (192888'12717) clean 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by osd.325.0:2394 2014-08-12 15:50:27.100203 2014-08-13 17:38:03.096913 7fe9bae887a0 20 read_log 194215'12871 (194215'12870) delete 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.2174595.0:190517 0.000000 2014-08-13 17:38:03.096924 7fe9bae887a0 20 read_log 194215'12872 (192834'12214) clean dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by osd.325.0:2396 2014-08-12 15:50:27.114111 2014-08-13 17:38:03.096937 7fe9bae887a0 20 read_log 194215'12873 (194215'12872) delete dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.2174595.0:190519 0.000000 2014-08-13 17:38:03.096947 7fe9bae887a0 20 read_log 194215'12874 (192888'12673) clean 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by osd.325.0:2398 2014-08-12 15:50:27.128655 2014-08-13 17:38:03.096960 7fe9bae887a0 20 read_log 194215'12875 (194215'12874) delete 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.2174595.0:190521 0.000000 2014-08-13 17:38:03.096972 7fe9bae887a0 20 read_log 194215'12876 (192834'12237) clean 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by osd.325.0:2400 2014-08-12 15:50:27.143921 2014-08-13 17:38:03.096985 7fe9bae887a0 20 read_log 194215'12877 (194215'12876) delete 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.2174595.0:190523 0.000000 2014-08-13 17:38:03.096995 7fe9bae887a0 20 read_log 194215'12878 (192834'11895) clean 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by osd.325.0:2402 2014-08-12 15:50:27.156919 2014-08-13 17:38:03.097007 7fe9bae887a0 20 read_log 194215'12879 (194215'12878) delete 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.2174595.0:190525 0.000000 2014-08-13 17:38:03.097018 7fe9bae887a0 20 read_log 194215'12880 (192888'12555) clean d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by osd.325.0:2404 2014-08-12 15:50:27.171481 2014-08-13 17:38:03.097048 7fe9bae887a0 20 read_log 194215'12881 (194215'12880) delete d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.2174595.0:190527 0.000000 2014-08-13 17:38:03.097059 7fe9bae887a0 20 read_log 194215'12882 (192888'12529) clean eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by osd.325.0:2406 2014-08-12 15:50:27.186559 2014-08-13 17:38:03.097071 7fe9bae887a0 20 read_log 194215'12883 (194215'12882) delete eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.2174595.0:190529 0.000000 2014-08-13 17:38:03.097082 7fe9bae887a0 20 read_log 194215'12884 (193305'12781) clean 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by osd.325.0:2408 2014-08-12 15:50:27.201278 2014-08-13 17:38:03.097094 7fe9bae887a0 20 read_log 194215'12885 (194215'12884) delete 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.2174595.0:190531 0.000000 2014-08-13 17:38:03.097105 7fe9bae887a0 20 read_log 194215'12886 (193305'12792) clean 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by osd.325.0:2410 2014-08-12 15:50:27.214916 2014-08-13 17:38:03.097117 7fe9bae887a0 20 read_log 194215'12887 (194215'12886) delete 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.2174595.0:190533 0.000000 2014-08-13 17:38:03.097128 7fe9bae887a0 20 read_log 194215'12888 (192888'12521) clean 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by osd.325.0:2412 2014-08-12 15:50:27.228685 2014-08-13 17:38:03.097141 7fe9bae887a0 20 read_log 194215'12889 (194215'12888) delete 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.2174595.0:190535 0.000000 2014-08-13 17:38:03.097171 7fe9bae887a0 20 read_log 194215'12890 (193305'12778) clean 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by osd.325.0:2414 2014-08-12 15:50:27.244199 2014-08-13 17:38:03.097188 7fe9bae887a0 20 read_log 194215'12891 (194215'12890) delete 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.2174595.0:190537 0.000000 2014-08-13 17:38:03.097199 7fe9bae887a0 20 read_log 194215'12892 (192888'12506) clean d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by osd.325.0:2416 2014-08-12 15:50:27.259251 2014-08-13 17:38:03.097211 7fe9bae887a0 20 read_log 194215'12893 (194215'12892) delete d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.2174595.0:190539 0.000000 2014-08-13 17:38:03.097222 7fe9bae887a0 20 read_log 194215'12894 (192834'12246) clean 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by osd.325.0:2418 2014-08-12 15:50:27.271750 2014-08-13 17:38:03.097238 7fe9bae887a0 20 read_log 194215'12895 (194215'12894) delete 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.2174595.0:190541 0.000000 2014-08-13 17:38:03.097249 7fe9bae887a0 20 read_log 194215'12896 (192888'12561) clean 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by osd.325.0:2420 2014-08-12 15:50:27.284514 2014-08-13 17:38:03.097263 7fe9bae887a0 20 read_log 194215'12897 (194215'12896) delete 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.2174595.0:190543 0.000000 2014-08-13 17:38:03.097274 7fe9bae887a0 20 read_log 194215'12898 (192834'11827) clean 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by osd.325.0:2422 2014-08-12 15:50:27.300500 2014-08-13 17:38:03.097286 7fe9bae887a0 20 read_log 194215'12899 (194215'12898) delete 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.2174595.0:190545 0.000000 2014-08-13 17:38:03.097297 7fe9bae887a0 20 read_log 194215'12900 (192888'12547) clean 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by osd.325.0:2424 2014-08-12 15:50:27.327363 2014-08-13 17:38:03.097309 7fe9bae887a0 20 read_log 194215'12901 (194215'12900) delete 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.2174595.0:190547 0.000000 2014-08-13 17:38:03.097320 7fe9bae887a0 20 read_log 194215'12902 (192888'12709) clean 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by osd.325.0:2426 2014-08-12 15:50:27.342003 2014-08-13 17:38:03.097332 7fe9bae887a0 20 read_log 194215'12903 (194215'12902) delete 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.2174595.0:190549 0.000000 2014-08-13 17:38:03.097343 7fe9bae887a0 20 read_log 194215'12904 (192834'11826) clean 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by osd.325.0:2428 2014-08-12 15:50:27.357282 2014-08-13 17:38:03.097355 7fe9bae887a0 20 read_log 194215'12905 (194215'12904) delete 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.2174595.0:190551 0.000000 2014-08-13 17:38:03.097366 7fe9bae887a0 20 read_log 194215'12906 (193305'12789) clean e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by osd.325.0:2430 2014-08-12 15:50:27.373123 2014-08-13 17:38:03.097380 7fe9bae887a0 20 read_log 194215'12907 (194215'12906) delete e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.2174595.0:190553 0.000000 2014-08-13 17:38:03.097391 7fe9bae887a0 20 read_log 194215'12908 (192888'12530) clean 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by osd.325.0:2432 2014-08-12 15:50:27.387897 2014-08-13 17:38:03.097403 7fe9bae887a0 20 read_log 194215'12909 (194215'12908) delete 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.2174595.0:190555 0.000000 2014-08-13 17:38:03.097439 7fe9bae887a0 20 read_log 194215'12910 (192834'11791) clean 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by osd.325.0:2434 2014-08-12 15:50:27.401528 2014-08-13 17:38:03.097455 7fe9bae887a0 20 read_log 194215'12911 (194215'12910) delete 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.2174595.0:190557 0.000000 2014-08-13 17:38:03.097466 7fe9bae887a0 20 read_log 194215'12912 (192888'12543) clean f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by osd.325.0:2436 2014-08-12 15:50:27.416233 2014-08-13 17:38:03.097478 7fe9bae887a0 20 read_log 194215'12913 (194215'12912) delete f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.2174595.0:190559 0.000000 2014-08-13 17:38:03.097489 7fe9bae887a0 20 read_log 194215'12914 (192888'12698) clean 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by osd.325.0:2438 2014-08-12 15:50:27.428697 2014-08-13 17:38:03.097501 7fe9bae887a0 20 read_log 194215'12915 (194215'12914) delete 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.2174595.0:190561 0.000000 2014-08-13 17:38:03.097512 7fe9bae887a0 20 read_log 194215'12916 (192888'12551) clean 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by osd.325.0:2440 2014-08-12 15:50:27.442500 2014-08-13 17:38:03.097524 7fe9bae887a0 20 read_log 194215'12917 (194215'12916) delete 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.2174595.0:190563 0.000000 2014-08-13 17:38:03.097535 7fe9bae887a0 20 read_log 194215'12918 (192834'12221) clean 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by osd.325.0:2442 2014-08-12 15:50:27.470283 2014-08-13 17:38:03.097547 7fe9bae887a0 20 read_log 194215'12919 (194215'12918) delete 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.2174595.0:190565 0.000000 2014-08-13 17:38:03.097558 7fe9bae887a0 20 read_log 194215'12920 (192834'12217) clean 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by osd.325.0:2444 2014-08-12 15:50:27.485493 2014-08-13 17:38:03.097570 7fe9bae887a0 20 read_log 194215'12921 (194215'12920) delete 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.2174595.0:190567 0.000000 2014-08-13 17:38:03.097581 7fe9bae887a0 20 read_log 194215'12922 (192888'12509) clean f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by osd.325.0:2446 2014-08-12 15:50:27.498601 2014-08-13 17:38:03.097593 7fe9bae887a0 20 read_log 194215'12923 (194215'12922) delete f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.2174595.0:190569 0.000000 2014-08-13 17:38:03.097604 7fe9bae887a0 20 read_log 194215'12924 (192834'12242) clean 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by osd.325.0:2448 2014-08-12 15:50:27.512695 2014-08-13 17:38:03.097616 7fe9bae887a0 20 read_log 194215'12925 (194215'12924) delete 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.2174595.0:190571 0.000000 2014-08-13 17:38:03.097661 7fe9bae887a0 20 read_log 194215'12926 (192834'11843) clean 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by osd.325.0:2450 2014-08-12 15:50:27.526648 2014-08-13 17:38:03.097677 7fe9bae887a0 20 read_log 194215'12927 (194215'12926) delete 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.2174595.0:190573 0.000000 2014-08-13 17:38:03.097688 7fe9bae887a0 20 read_log 194215'12928 (193305'12779) clean c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by osd.325.0:2452 2014-08-12 15:50:27.540862 2014-08-13 17:38:03.097700 7fe9bae887a0 20 read_log 194215'12929 (194215'12928) delete c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.2174595.0:190575 0.000000 2014-08-13 17:38:03.097718 7fe9bae887a0 20 read_log 194215'12930 (192888'12548) clean 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by osd.325.0:2454 2014-08-12 15:50:27.555235 2014-08-13 17:38:03.097731 7fe9bae887a0 20 read_log 194215'12931 (194215'12930) delete 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.2174595.0:190577 0.000000 2014-08-13 17:38:03.097742 7fe9bae887a0 20 read_log 194215'12932 (192888'12546) clean e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by osd.325.0:2456 2014-08-12 15:50:27.569053 2014-08-13 17:38:03.097754 7fe9bae887a0 20 read_log 194215'12933 (194215'12932) delete e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.2174595.0:190579 0.000000 2014-08-13 17:38:03.097765 7fe9bae887a0 20 read_log 194215'12934 (192888'12681) clean 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by osd.325.0:2458 2014-08-12 15:50:27.582850 2014-08-13 17:38:03.097779 7fe9bae887a0 20 read_log 194215'12935 (194215'12934) delete 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.2174595.0:190581 0.000000 2014-08-13 17:38:03.097790 7fe9bae887a0 20 read_log 194215'12936 (192888'12687) clean ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by osd.325.0:2460 2014-08-12 15:50:27.606056 2014-08-13 17:38:03.097802 7fe9bae887a0 20 read_log 194215'12937 (194215'12936) delete ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.2174595.0:190583 0.000000 2014-08-13 17:38:03.097813 7fe9bae887a0 20 read_log 194215'12938 (192888'12701) clean 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by osd.325.0:2462 2014-08-12 15:50:27.619264 2014-08-13 17:38:03.097825 7fe9bae887a0 20 read_log 194215'12939 (194215'12938) delete 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.2174595.0:190585 0.000000 2014-08-13 17:38:03.097835 7fe9bae887a0 20 read_log 194215'12940 (192888'12505) clean a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by osd.325.0:2464 2014-08-12 15:50:27.633354 2014-08-13 17:38:03.097848 7fe9bae887a0 20 read_log 194215'12941 (194215'12940) delete a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.2174595.0:190587 0.000000 2014-08-13 17:38:03.097859 7fe9bae887a0 20 read_log 194215'12942 (192888'12690) clean 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by osd.325.0:2466 2014-08-12 15:50:27.649008 2014-08-13 17:38:03.097871 7fe9bae887a0 20 read_log 194215'12943 (194215'12942) delete 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.2174595.0:190589 0.000000 2014-08-13 17:38:03.097881 7fe9bae887a0 20 read_log 194215'12944 (192834'11808) clean 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by osd.325.0:2468 2014-08-12 15:50:27.665935 2014-08-13 17:38:03.097893 7fe9bae887a0 20 read_log 194215'12945 (194215'12944) delete 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.2174595.0:190591 0.000000 2014-08-13 17:38:03.097904 7fe9bae887a0 20 read_log 194215'12946 (192888'12567) clean 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by osd.325.0:2470 2014-08-12 15:50:27.681452 2014-08-13 17:38:03.097916 7fe9bae887a0 20 read_log 194215'12947 (194215'12946) delete 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.2174595.0:190593 0.000000 2014-08-13 17:38:03.097927 7fe9bae887a0 20 read_log 194215'12948 (192834'12349) clean bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by osd.325.0:2472 2014-08-12 15:50:27.694175 2014-08-13 17:38:03.097939 7fe9bae887a0 20 read_log 194215'12949 (194215'12948) delete bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.2174595.0:190595 0.000000 2014-08-13 17:38:03.097959 7fe9bae887a0 20 read_log 194215'12950 (192834'11841) clean 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by osd.325.0:2474 2014-08-12 15:50:27.708527 2014-08-13 17:38:03.097972 7fe9bae887a0 20 read_log 194215'12951 (194215'12950) delete 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.2174595.0:190597 0.000000 2014-08-13 17:38:03.097983 7fe9bae887a0 20 read_log 194215'12952 (192888'12550) clean e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by osd.325.0:2476 2014-08-12 15:50:27.722426 2014-08-13 17:38:03.097995 7fe9bae887a0 20 read_log 194215'12953 (194215'12952) delete e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.2174595.0:190599 0.000000 2014-08-13 17:38:03.098006 7fe9bae887a0 20 read_log 194215'12954 (192888'12559) clean b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by osd.325.0:2478 2014-08-12 15:50:27.737703 2014-08-13 17:38:03.098018 7fe9bae887a0 20 read_log 194215'12955 (194215'12954) delete b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.2174595.0:190601 0.000000 2014-08-13 17:38:03.098029 7fe9bae887a0 20 read_log 194215'12956 (192834'11873) clean b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by osd.325.0:2480 2014-08-12 15:50:27.752253 2014-08-13 17:38:03.098041 7fe9bae887a0 20 read_log 194215'12957 (194215'12956) delete b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.2174595.0:190603 0.000000 2014-08-13 17:38:03.098051 7fe9bae887a0 20 read_log 194215'12958 (192888'12544) clean dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by osd.325.0:2482 2014-08-12 15:50:27.766988 2014-08-13 17:38:03.098063 7fe9bae887a0 20 read_log 194215'12959 (194215'12958) delete dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.2174595.0:190605 0.000000 2014-08-13 17:38:03.098074 7fe9bae887a0 20 read_log 194215'12960 (193305'12782) clean f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by osd.325.0:2484 2014-08-12 15:50:27.779954 2014-08-13 17:38:03.098086 7fe9bae887a0 20 read_log 194215'12961 (194215'12960) delete f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.2174595.0:190607 0.000000 2014-08-13 17:38:03.098097 7fe9bae887a0 20 read_log 194215'12962 (192834'11731) clean 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by osd.325.0:2486 2014-08-12 15:50:27.793538 2014-08-13 17:38:03.098109 7fe9bae887a0 20 read_log 194215'12963 (194215'12962) delete 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.2174595.0:190609 0.000000 2014-08-13 17:38:03.098120 7fe9bae887a0 20 read_log 194215'12964 (192834'12271) clean 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by osd.325.0:2488 2014-08-12 15:50:27.808252 2014-08-13 17:38:03.098132 7fe9bae887a0 20 read_log 194215'12965 (194215'12964) delete 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.2174595.0:190611 0.000000 2014-08-13 17:38:03.098143 7fe9bae887a0 20 read_log 194215'12966 (193305'12771) clean fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by osd.325.0:2490 2014-08-12 15:50:27.822572 2014-08-13 17:38:03.098160 7fe9bae887a0 20 read_log 194215'12967 (194215'12966) delete fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.2174595.0:190613 0.000000 2014-08-13 17:38:03.098171 7fe9bae887a0 20 read_log 194215'12968 (193305'12787) clean feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by osd.325.0:2492 2014-08-12 15:50:27.835093 2014-08-13 17:38:03.098184 7fe9bae887a0 20 read_log 194215'12969 (194215'12968) delete feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.2174595.0:190615 0.000000 2014-08-13 17:38:03.098201 7fe9bae887a0 20 read_log 194215'12970 (192834'12232) clean 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by osd.325.0:2494 2014-08-12 15:50:27.848752 2014-08-13 17:38:03.098214 7fe9bae887a0 20 read_log 194215'12971 (194215'12970) delete 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.2174595.0:190617 0.000000 2014-08-13 17:38:03.098225 7fe9bae887a0 20 read_log 194215'12972 (192834'12340) clean 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by osd.325.0:2496 2014-08-12 15:50:27.862019 2014-08-13 17:38:03.098237 7fe9bae887a0 20 read_log 194215'12973 (194215'12972) delete 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.2174595.0:190619 0.000000 2014-08-13 17:38:03.098248 7fe9bae887a0 20 read_log 194215'12974 (193305'12775) clean 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by osd.325.0:2498 2014-08-12 15:50:27.875423 2014-08-13 17:38:03.098260 7fe9bae887a0 20 read_log 194215'12975 (194215'12974) delete 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.2174595.0:190621 0.000000 2014-08-13 17:38:03.098271 7fe9bae887a0 20 read_log 194215'12976 (192888'12542) clean 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by osd.325.0:2500 2014-08-12 15:50:27.888418 2014-08-13 17:38:03.098283 7fe9bae887a0 20 read_log 194215'12977 (194215'12976) delete 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.2174595.0:190623 0.000000 2014-08-13 17:38:03.098294 7fe9bae887a0 20 read_log 194215'12978 (192834'12335) clean fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by osd.325.0:2502 2014-08-12 15:50:27.904153 2014-08-13 17:38:03.098306 7fe9bae887a0 20 read_log 194215'12979 (194215'12978) delete fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.2174595.0:190625 0.000000 2014-08-13 17:38:03.098316 7fe9bae887a0 20 read_log 194215'12980 (192834'12277) clean 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by osd.325.0:2504 2014-08-12 15:50:27.918318 2014-08-13 17:38:03.098329 7fe9bae887a0 20 read_log 194215'12981 (194215'12980) delete 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.2174595.0:190627 0.000000 2014-08-13 17:38:03.098339 7fe9bae887a0 20 read_log 194215'12982 (193305'12788) clean f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by osd.325.0:2506 2014-08-12 15:50:27.933199 2014-08-13 17:38:03.098351 7fe9bae887a0 20 read_log 194215'12983 (194215'12982) delete f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.2174595.0:190629 0.000000 2014-08-13 17:38:03.098362 7fe9bae887a0 20 read_log 194215'12984 (192834'11834) clean 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by osd.325.0:2508 2014-08-12 15:50:27.945894 2014-08-13 17:38:03.098374 7fe9bae887a0 20 read_log 194215'12985 (194215'12984) delete 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.2174595.0:190631 0.000000 2014-08-13 17:38:03.098385 7fe9bae887a0 20 read_log 194215'12986 (192888'12520) clean c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by osd.325.0:2510 2014-08-12 15:50:27.959341 2014-08-13 17:38:03.098397 7fe9bae887a0 20 read_log 194215'12987 (194215'12986) delete c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.2174595.0:190633 0.000000 2014-08-13 17:38:03.098408 7fe9bae887a0 20 read_log 194215'12988 (192834'12365) clean 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by osd.325.0:2512 2014-08-12 15:50:27.973469 2014-08-13 17:38:03.098420 7fe9bae887a0 20 read_log 194215'12989 (194215'12988) delete 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.2174595.0:190635 0.000000 2014-08-13 17:38:03.098439 7fe9bae887a0 20 read_log 194215'12990 (192888'12711) clean 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by osd.325.0:2514 2014-08-12 15:50:27.986035 2014-08-13 17:38:03.098452 7fe9bae887a0 20 read_log 194215'12991 (194215'12990) delete 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.2174595.0:190637 0.000000 2014-08-13 17:38:03.098463 7fe9bae887a0 20 read_log 194215'12992 (192888'12571) clean 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by osd.325.0:2516 2014-08-12 15:50:28.000475 2014-08-13 17:38:03.098475 7fe9bae887a0 20 read_log 194215'12993 (194215'12992) delete 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.2174595.0:190639 0.000000 2014-08-13 17:38:03.098486 7fe9bae887a0 20 read_log 194215'12994 (192834'12278) clean 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by osd.325.0:2518 2014-08-12 15:50:28.016226 2014-08-13 17:38:03.098498 7fe9bae887a0 20 read_log 194215'12995 (194215'12994) delete 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.2174595.0:190641 0.000000 2014-08-13 17:38:03.098509 7fe9bae887a0 20 read_log 194215'12996 (192834'12351) clean af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by osd.325.0:2520 2014-08-12 15:50:28.023934 2014-08-13 17:38:03.098521 7fe9bae887a0 20 read_log 194215'12997 (194215'12996) delete af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.2174595.0:190643 0.000000 2014-08-13 17:38:03.098532 7fe9bae887a0 20 read_log 194215'12998 (192888'12680) clean d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by osd.325.0:2522 2014-08-12 15:50:28.032742 2014-08-13 17:38:03.098544 7fe9bae887a0 20 read_log 194215'12999 (194215'12998) delete d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.2174595.0:190645 0.000000 2014-08-13 17:38:03.098555 7fe9bae887a0 20 read_log 194215'13000 (193305'12786) clean 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by osd.325.0:2524 2014-08-12 15:50:28.042228 2014-08-13 17:38:03.098567 7fe9bae887a0 20 read_log 194215'13001 (194215'13000) delete 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.2174595.0:190647 0.000000 2014-08-13 17:38:03.098578 7fe9bae887a0 20 read_log 194215'13002 (193305'12784) clean b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by osd.325.0:2526 2014-08-12 15:50:28.050922 2014-08-13 17:38:03.098590 7fe9bae887a0 20 read_log 194215'13003 (194215'13002) delete b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.2174595.0:190649 0.000000 2014-08-13 17:38:03.098600 7fe9bae887a0 20 read_log 194215'13004 (192834'12317) clean 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by osd.325.0:2528 2014-08-12 15:50:28.058746 2014-08-13 17:38:03.098613 7fe9bae887a0 20 read_log 194215'13005 (194215'13004) delete 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.2174595.0:190651 0.000000 2014-08-13 17:38:03.098623 7fe9bae887a0 20 read_log 194215'13006 (192888'12504) clean 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by osd.325.0:2530 2014-08-12 15:50:28.066870 2014-08-13 17:38:03.098636 7fe9bae887a0 20 read_log 194215'13007 (194215'13006) delete 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.2174595.0:190653 0.000000 2014-08-13 17:38:03.098646 7fe9bae887a0 20 read_log 194215'13008 (192888'12569) clean 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by osd.325.0:2532 2014-08-12 15:50:28.075322 2014-08-13 17:38:03.098658 7fe9bae887a0 20 read_log 194215'13009 (194215'13008) delete 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.2174595.0:190655 0.000000 2014-08-13 17:38:03.098678 7fe9bae887a0 20 read_log 194215'13010 (193305'12772) clean 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by osd.325.0:2534 2014-08-12 15:50:28.084124 2014-08-13 17:38:03.098691 7fe9bae887a0 20 read_log 194215'13011 (194215'13010) delete 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.2174595.0:190657 0.000000 2014-08-13 17:38:03.098702 7fe9bae887a0 20 read_log 194215'13012 (192888'12708) clean 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by osd.325.0:2536 2014-08-12 15:50:28.093704 2014-08-13 17:38:03.098714 7fe9bae887a0 20 read_log 194215'13013 (194215'13012) delete 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.2174595.0:190659 0.000000 2014-08-13 17:38:03.098725 7fe9bae887a0 20 read_log 194215'13014 (192834'12364) clean f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by osd.325.0:2538 2014-08-12 15:50:28.105822 2014-08-13 17:38:03.098737 7fe9bae887a0 20 read_log 194215'13015 (194215'13014) delete f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.2174595.0:190661 0.000000 2014-08-13 17:38:03.098747 7fe9bae887a0 20 read_log 194215'13016 (192834'11806) clean 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by osd.325.0:2540 2014-08-12 15:50:28.117598 2014-08-13 17:38:03.098759 7fe9bae887a0 20 read_log 194215'13017 (194215'13016) delete 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.2174595.0:190663 0.000000 2014-08-13 17:38:03.098770 7fe9bae887a0 20 read_log 194215'13018 (192888'12693) clean 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by osd.325.0:2542 2014-08-12 15:50:28.130743 2014-08-13 17:38:03.098801 7fe9bae887a0 20 read_log 194215'13019 (194215'13018) delete 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.2174595.0:190665 0.000000 2014-08-13 17:38:03.098812 7fe9bae887a0 20 read_log 194215'13020 (192834'11796) clean 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by osd.325.0:2544 2014-08-12 15:50:28.142807 2014-08-13 17:38:03.098825 7fe9bae887a0 20 read_log 194215'13021 (194215'13020) delete 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.2174595.0:190667 0.000000 2014-08-13 17:38:03.098836 7fe9bae887a0 20 read_log 194215'13022 (192888'12541) clean 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by osd.325.0:2546 2014-08-12 15:50:28.157510 2014-08-13 17:38:03.098864 7fe9bae887a0 20 read_log 194215'13023 (194215'13022) delete 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.2174595.0:190669 0.000000 2014-08-13 17:38:03.098876 7fe9bae887a0 20 read_log 194215'13024 (192888'12676) clean 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by osd.325.0:2548 2014-08-12 15:50:28.171217 2014-08-13 17:38:03.098889 7fe9bae887a0 20 read_log 194215'13025 (194215'13024) delete 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.2174595.0:190671 0.000000 2014-08-13 17:38:03.098899 7fe9bae887a0 20 read_log 194215'13026 (192834'12253) clean a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by osd.325.0:2550 2014-08-12 15:50:28.186252 2014-08-13 17:38:03.098912 7fe9bae887a0 20 read_log 194215'13027 (194215'13026) delete a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.2174595.0:190673 0.000000 2014-08-13 17:38:03.098922 7fe9bae887a0 20 read_log 194215'13028 (192834'11757) clean 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by osd.325.0:2552 2014-08-12 15:50:28.199716 2014-08-13 17:38:03.098935 7fe9bae887a0 20 read_log 194215'13029 (194215'13028) delete 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.2174595.0:190675 0.000000 2014-08-13 17:38:03.098952 7fe9bae887a0 20 read_log 194215'13030 (192888'12695) clean 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by osd.325.0:2554 2014-08-12 15:50:28.212691 2014-08-13 17:38:03.098965 7fe9bae887a0 20 read_log 194215'13031 (194215'13030) delete 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.2174595.0:190677 0.000000 2014-08-13 17:38:03.098976 7fe9bae887a0 20 read_log 194215'13032 (193305'12785) clean b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by osd.325.0:2556 2014-08-12 15:50:28.227014 2014-08-13 17:38:03.098988 7fe9bae887a0 20 read_log 194215'13033 (194215'13032) delete b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.2174595.0:190679 0.000000 2014-08-13 17:38:03.098999 7fe9bae887a0 20 read_log 194215'13034 (192888'12525) clean 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by osd.325.0:2558 2014-08-12 15:50:28.240289 2014-08-13 17:38:03.099011 7fe9bae887a0 20 read_log 194215'13035 (194215'13034) delete 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.2174595.0:190681 0.000000 2014-08-13 17:38:03.099022 7fe9bae887a0 20 read_log 194215'13036 (192888'12699) clean e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by osd.325.0:2560 2014-08-12 15:50:28.253618 2014-08-13 17:38:03.099034 7fe9bae887a0 20 read_log 194215'13037 (194215'13036) delete e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.2174595.0:190683 0.000000 2014-08-13 17:38:03.099044 7fe9bae887a0 20 read_log 194215'13038 (192888'12713) clean c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by osd.325.0:2562 2014-08-12 15:50:28.269951 2014-08-13 17:38:03.099057 7fe9bae887a0 20 read_log 194215'13039 (194215'13038) delete c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.2174595.0:190685 0.000000 2014-08-13 17:38:03.099067 7fe9bae887a0 20 read_log 194215'13040 (192888'12563) clean 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by osd.325.0:2564 2014-08-12 15:50:28.282727 2014-08-13 17:38:03.099080 7fe9bae887a0 20 read_log 194215'13041 (194215'13040) delete 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.2174595.0:190687 0.000000 2014-08-13 17:38:03.099090 7fe9bae887a0 20 read_log 194215'13042 (192888'12686) clean 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by osd.325.0:2566 2014-08-12 15:50:28.296106 2014-08-13 17:38:03.099103 7fe9bae887a0 20 read_log 194215'13043 (194215'13042) delete 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.2174595.0:190689 0.000000 2014-08-13 17:38:03.099113 7fe9bae887a0 20 read_log 194215'13044 (192888'12704) clean ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by osd.325.0:2568 2014-08-12 15:50:28.308436 2014-08-13 17:38:03.099125 7fe9bae887a0 20 read_log 194215'13045 (194215'13044) delete ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.2174595.0:190691 0.000000 2014-08-13 17:38:03.099136 7fe9bae887a0 20 read_log 194215'13046 (193305'12777) clean 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by osd.325.0:2570 2014-08-12 15:50:28.322080 2014-08-13 17:38:03.099148 7fe9bae887a0 20 read_log 194215'13047 (194215'13046) delete 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.2174595.0:190693 0.000000 2014-08-13 17:38:03.099164 7fe9bae887a0 20 read_log 194215'13048 (192888'12706) clean 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by osd.325.0:2572 2014-08-12 15:50:28.336072 2014-08-13 17:38:03.099176 7fe9bae887a0 20 read_log 194215'13049 (194215'13048) delete 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.2174595.0:190695 0.000000 2014-08-13 17:38:03.099214 7fe9bae887a0 20 read_log 194215'13050 (192834'11770) clean 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by osd.325.0:2574 2014-08-12 15:50:28.349354 2014-08-13 17:38:03.099232 7fe9bae887a0 20 read_log 194215'13051 (194215'13050) delete 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.2174595.0:190697 0.000000 2014-08-13 17:38:03.099243 7fe9bae887a0 20 read_log 194215'13052 (192888'12710) clean fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by osd.325.0:2576 2014-08-12 15:50:28.363662 2014-08-13 17:38:03.099255 7fe9bae887a0 20 read_log 194215'13053 (194215'13052) delete fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.2174595.0:190699 0.000000 2014-08-13 17:38:03.099266 7fe9bae887a0 20 read_log 194370'13054 (0'0) promote 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by osd.324.0:4372 2014-08-12 20:30:17.653172 2014-08-13 17:38:03.099278 7fe9bae887a0 20 read_log 194370'13055 (194370'13054) modify 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by client.1508531.0:869 2014-08-12 20:30:17.645600 2014-08-13 17:38:03.099296 7fe9bae887a0 20 read_log 194370'13056 (0'0) promote fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by osd.324.0:4416 2014-08-12 20:30:25.651001 2014-08-13 17:38:03.099308 7fe9bae887a0 20 read_log 194370'13057 (194370'13056) modify fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by client.1508531.0:3028 2014-08-12 20:30:25.643346 2014-08-13 17:38:03.099321 7fe9bae887a0 20 read_log 194370'13058 (0'0) promote e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by osd.324.0:4440 2014-08-12 20:30:29.416303 2014-08-13 17:38:03.099333 7fe9bae887a0 20 read_log 194370'13059 (194370'13058) modify e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by client.1508531.0:4031 2014-08-12 20:30:29.406321 2014-08-13 17:38:03.099345 7fe9bae887a0 20 read_log 194370'13060 (0'0) promote b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by osd.324.0:4450 2014-08-12 20:30:30.210042 2014-08-13 17:38:03.099356 7fe9bae887a0 20 read_log 194370'13061 (194370'13060) modify b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by client.1508531.0:4231 2014-08-12 20:30:30.202760 2014-08-13 17:38:03.099369 7fe9bae887a0 20 read_log 194370'13062 (0'0) promote 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by osd.324.0:4460 2014-08-12 20:30:32.802646 2014-08-13 17:38:03.099381 7fe9bae887a0 20 read_log 194370'13063 (194370'13062) modify 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by client.1508531.0:4908 2014-08-12 20:30:32.794881 2014-08-13 17:38:03.099393 7fe9bae887a0 20 read_log 194370'13064 (0'0) promote 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by osd.324.0:4488 2014-08-12 20:30:38.782573 2014-08-13 17:38:03.099405 7fe9bae887a0 20 read_log 194370'13065 (194370'13064) modify 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by client.1508531.0:6231 2014-08-12 20:30:38.774709 2014-08-13 17:38:03.099417 7fe9bae887a0 20 read_log 194370'13066 (0'0) promote fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by osd.324.0:4498 2014-08-12 20:30:39.800147 2014-08-13 17:38:03.099428 7fe9bae887a0 20 read_log 194370'13067 (194370'13066) modify fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by client.1508531.0:6465 2014-08-12 20:30:39.791322 2014-08-13 17:38:03.099441 7fe9bae887a0 20 read_log 194370'13068 (0'0) promote 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by osd.324.0:4502 2014-08-12 20:30:40.839066 2014-08-13 17:38:03.099453 7fe9bae887a0 20 read_log 194370'13069 (194370'13068) modify 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by client.1508531.0:6688 2014-08-12 20:30:40.829729 2014-08-13 17:38:03.099475 7fe9bae887a0 20 read_log 194370'13070 (0'0) promote dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by osd.324.0:4536 2014-08-12 20:30:48.419090 2014-08-13 17:38:03.099488 7fe9bae887a0 20 read_log 194370'13071 (194370'13070) modify dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by client.1508531.0:8156 2014-08-12 20:30:48.410034 2014-08-13 17:38:03.099500 7fe9bae887a0 20 read_log 194370'13072 (0'0) promote c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by osd.324.0:4562 2014-08-12 20:30:57.167267 2014-08-13 17:38:03.099512 7fe9bae887a0 20 read_log 194370'13073 (194370'13072) modify c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by client.1508531.0:9643 2014-08-12 20:30:57.159265 2014-08-13 17:38:03.099524 7fe9bae887a0 20 read_log 194370'13074 (0'0) promote 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by osd.324.0:4654 2014-08-12 20:31:19.085945 2014-08-13 17:38:03.099536 7fe9bae887a0 20 read_log 194370'13075 (194370'13074) modify 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by client.1508531.0:12970 2014-08-12 20:31:19.077634 2014-08-13 17:38:03.099549 7fe9bae887a0 20 read_log 194370'13076 (0'0) promote e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by osd.324.0:4670 2014-08-12 20:31:26.769589 2014-08-13 17:38:03.099561 7fe9bae887a0 20 read_log 194370'13077 (194370'13076) modify e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by client.1508531.0:13763 2014-08-12 20:31:26.761274 2014-08-13 17:38:03.099573 7fe9bae887a0 20 read_log 194377'13078 (0'0) promote d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by osd.13.0:2811 2014-08-12 20:33:03.110943 2014-08-13 17:38:03.099585 7fe9bae887a0 20 read_log 194377'13079 (194377'13078) modify d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by client.2231190.0:1476 2014-08-12 20:33:03.102034 2014-08-13 17:38:03.099598 7fe9bae887a0 20 read_log 194377'13080 (0'0) promote 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by osd.13.0:2823 2014-08-12 20:33:06.459017 2014-08-13 17:38:03.099610 7fe9bae887a0 20 read_log 194377'13081 (194377'13080) modify 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by client.2231190.0:2299 2014-08-12 20:33:06.453284 2014-08-13 17:38:03.099622 7fe9bae887a0 20 read_log 194377'13082 (0'0) promote 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by osd.13.0:2825 2014-08-12 20:33:06.747346 2014-08-13 17:38:03.099652 7fe9bae887a0 20 read_log 194377'13083 (194377'13082) modify 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by client.2231190.0:2377 2014-08-12 20:33:06.741091 2014-08-13 17:38:03.099665 7fe9bae887a0 20 read_log 194377'13084 (0'0) promote b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by osd.13.0:2855 2014-08-12 20:33:17.041923 2014-08-13 17:38:03.099677 7fe9bae887a0 20 read_log 194377'13085 (194377'13084) modify b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by client.2231190.0:4484 2014-08-12 20:33:17.034718 2014-08-13 17:38:03.099689 7fe9bae887a0 20 read_log 194380'13086 (0'0) promote 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by osd.13.0:2861 2014-08-12 20:33:25.607982 2014-08-13 17:38:03.099702 7fe9bae887a0 20 read_log 194380'13087 (194380'13086) modify 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by client.2231190.0:5187 2014-08-12 20:33:25.601065 2014-08-13 17:38:03.099714 7fe9bae887a0 20 read_log 194383'13088 (0'0) promote d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by osd.325.0:7 2014-08-12 20:34:04.303075 2014-08-13 17:38:03.099735 7fe9bae887a0 20 read_log 194383'13089 (194383'13088) modify d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by client.2231190.0:6498 2014-08-12 20:34:04.278600 2014-08-13 17:38:03.099748 7fe9bae887a0 20 read_log 194383'13090 (0'0) promote fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by osd.325.0:19 2014-08-12 20:34:19.084519 2014-08-13 17:38:03.099760 7fe9bae887a0 20 read_log 194383'13091 (194383'13090) modify fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by client.2231190.0:7146 2014-08-12 20:34:19.077512 2014-08-13 17:38:03.099772 7fe9bae887a0 20 read_log 194383'13092 (0'0) promote 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by osd.325.0:21 2014-08-12 20:34:23.787632 2014-08-13 17:38:03.099784 7fe9bae887a0 20 read_log 194383'13093 (194383'13092) modify 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by client.2231190.0:7317 2014-08-12 20:34:23.779474 2014-08-13 17:38:03.099797 7fe9bae887a0 20 read_log 194416'13094 (0'0) promote 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by osd.395.0:4258 2014-08-13 15:28:44.079719 2014-08-13 17:38:03.099809 7fe9bae887a0 20 read_log 194416'13095 (194416'13094) modify 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by client.2441958.0:3528 2014-08-13 15:28:44.070814 2014-08-13 17:38:03.099821 7fe9bae887a0 20 read_log 194416'13096 (0'0) promote 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by osd.395.0:4278 2014-08-13 15:28:51.121397 2014-08-13 17:38:03.099833 7fe9bae887a0 20 read_log 194416'13097 (194416'13096) modify 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by client.2441958.0:5321 2014-08-13 15:28:51.114390 2014-08-13 17:38:03.099845 7fe9bae887a0 20 read_log 194416'13098 (0'0) promote 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by osd.395.0:4294 2014-08-13 15:28:56.070304 2014-08-13 17:38:03.099858 7fe9bae887a0 20 read_log 194416'13099 (194416'13098) modify 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by client.2441958.0:6554 2014-08-13 15:28:56.060052 2014-08-13 17:38:03.099870 7fe9bae887a0 20 read_log 194416'13100 (0'0) promote 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by osd.395.0:4310 2014-08-13 15:28:58.542579 2014-08-13 17:38:03.099882 7fe9bae887a0 20 read_log 194416'13101 (194416'13100) modify 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by client.2441958.0:7204 2014-08-13 15:28:58.533818 2014-08-13 17:38:03.099894 7fe9bae887a0 20 read_log 194416'13102 (0'0) promote c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by osd.395.0:4400 2014-08-13 15:29:19.354688 2014-08-13 17:38:03.099906 7fe9bae887a0 20 read_log 194416'13103 (194416'13102) modify c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by client.2441958.0:12410 2014-08-13 15:29:19.345491 2014-08-13 17:38:03.099920 7fe9bae887a0 20 read_log 194416'13104 (0'0) promote 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by osd.395.0:4434 2014-08-13 15:29:28.500047 2014-08-13 17:38:03.099932 7fe9bae887a0 20 read_log 194416'13105 (194416'13104) modify 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by client.2441958.0:14574 2014-08-13 15:29:28.490784 2014-08-13 17:38:03.099944 7fe9bae887a0 20 read_log 194416'13106 (0'0) promote b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by osd.395.0:4436 2014-08-13 15:29:29.693888 2014-08-13 17:38:03.099956 7fe9bae887a0 20 read_log 194416'13107 (194416'13106) modify b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by client.2441958.0:14864 2014-08-13 15:29:29.685386 2014-08-13 17:38:03.099968 7fe9bae887a0 20 read_log 194416'13108 (0'0) promote 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by osd.395.0:4444 2014-08-13 15:29:33.200498 2014-08-13 17:38:03.099987 7fe9bae887a0 20 read_log 194416'13109 (194416'13108) modify 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by client.2441958.0:15592 2014-08-13 15:29:33.188173 2014-08-13 17:38:03.100000 7fe9bae887a0 20 read_log 194416'13110 (0'0) promote f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by osd.395.0:4450 2014-08-13 15:29:48.002028 2014-08-13 17:38:03.100012 7fe9bae887a0 20 read_log 194416'13111 (194416'13110) modify f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by client.2441958.0:17046 2014-08-13 15:29:47.991603 2014-08-13 17:38:03.100041 7fe9bae887a0 20 read_log 194416'13112 (0'0) promote 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by osd.395.0:4468 2014-08-13 15:30:11.258275 2014-08-13 17:38:03.100055 7fe9bae887a0 20 read_log 194416'13113 (194416'13112) modify 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by client.2441958.0:18433 2014-08-13 15:30:11.249320 2014-08-13 17:38:03.100067 7fe9bae887a0 20 read_log 194416'13114 (0'0) promote 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by osd.395.0:4470 2014-08-13 15:30:12.135566 2014-08-13 17:38:03.100079 7fe9bae887a0 20 read_log 194416'13115 (194416'13114) modify 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by client.2441958.0:18464 2014-08-13 15:30:12.126663 2014-08-13 17:38:03.100091 7fe9bae887a0 20 read_log 194422'13116 (0'0) promote cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by osd.395.0:4480 2014-08-13 15:32:16.353064 2014-08-13 17:38:03.100103 7fe9bae887a0 20 read_log 194422'13117 (194422'13116) modify cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by client.1148203.0:295 2014-08-13 15:32:16.344464 2014-08-13 17:38:03.100115 7fe9bae887a0 20 read_log 194422'13118 (0'0) promote 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by osd.395.0:4484 2014-08-13 15:32:18.824551 2014-08-13 17:38:03.100127 7fe9bae887a0 20 read_log 194422'13119 (194422'13118) modify 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by client.1148203.0:874 2014-08-13 15:32:18.813722 2014-08-13 17:38:03.100139 7fe9bae887a0 20 read_log 194422'13120 (0'0) promote c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by osd.395.0:4498 2014-08-13 15:32:22.257929 2014-08-13 17:38:03.100151 7fe9bae887a0 20 read_log 194422'13121 (194422'13120) modify c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by client.1148203.0:1750 2014-08-13 15:32:22.249403 2014-08-13 17:38:03.100169 7fe9bae887a0 20 read_log 194422'13122 (0'0) promote de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by osd.395.0:4514 2014-08-13 15:32:25.466808 2014-08-13 17:38:03.100181 7fe9bae887a0 20 read_log 194422'13123 (194422'13122) modify de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by client.1148203.0:2585 2014-08-13 15:32:25.460245 2014-08-13 17:38:03.100193 7fe9bae887a0 20 read_log 194422'13124 (0'0) promote 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by osd.395.0:4516 2014-08-13 15:32:25.932710 2014-08-13 17:38:03.100207 7fe9bae887a0 20 read_log 194422'13125 (194422'13124) modify 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by client.1148203.0:2710 2014-08-13 15:32:25.923689 2014-08-13 17:38:03.100219 7fe9bae887a0 20 read_log 194422'13126 (0'0) promote 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by osd.395.0:4524 2014-08-13 15:32:27.674331 2014-08-13 17:38:03.100231 7fe9bae887a0 20 read_log 194422'13127 (194422'13126) modify 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by client.1148203.0:3151 2014-08-13 15:32:27.666329 2014-08-13 17:38:03.100243 7fe9bae887a0 20 read_log 194422'13128 (0'0) promote 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by osd.395.0:4540 2014-08-13 15:32:31.055137 2014-08-13 17:38:03.100264 7fe9bae887a0 20 read_log 194422'13129 (194422'13128) modify 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by client.1148203.0:4016 2014-08-13 15:32:31.046978 2014-08-13 17:38:03.100277 7fe9bae887a0 20 read_log 194422'13130 (0'0) promote ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by osd.395.0:4554 2014-08-13 15:32:35.380206 2014-08-13 17:38:03.100289 7fe9bae887a0 20 read_log 194422'13131 (194422'13130) modify ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by client.1148203.0:5110 2014-08-13 15:32:35.372868 2014-08-13 17:38:03.100301 7fe9bae887a0 20 read_log 194422'13132 (0'0) promote 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by osd.395.0:4556 2014-08-13 15:32:36.140636 2014-08-13 17:38:03.100313 7fe9bae887a0 20 read_log 194422'13133 (194422'13132) modify 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by client.1148203.0:5259 2014-08-13 15:32:36.130829 2014-08-13 17:38:03.100325 7fe9bae887a0 20 read_log 194422'13134 (0'0) promote a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by osd.395.0:4558 2014-08-13 15:32:37.003814 2014-08-13 17:38:03.100337 7fe9bae887a0 20 read_log 194422'13135 (194422'13134) modify a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by client.1148203.0:5434 2014-08-13 15:32:36.992775 2014-08-13 17:38:03.100349 7fe9bae887a0 20 read_log 194422'13136 (0'0) promote bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by osd.395.0:4568 2014-08-13 15:32:38.746016 2014-08-13 17:38:03.100361 7fe9bae887a0 20 read_log 194422'13137 (194422'13136) modify bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by client.1148203.0:5737 2014-08-13 15:32:38.736377 2014-08-13 17:38:03.100373 7fe9bae887a0 20 read_log 194422'13138 (0'0) promote bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by osd.395.0:4574 2014-08-13 15:32:39.973211 2014-08-13 17:38:03.100385 7fe9bae887a0 20 read_log 194422'13139 (194422'13138) modify bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by client.1148203.0:5971 2014-08-13 15:32:39.963483 2014-08-13 17:38:03.100397 7fe9bae887a0 20 read_log 194422'13140 (0'0) promote 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by osd.395.0:4588 2014-08-13 15:32:47.138728 2014-08-13 17:38:03.100409 7fe9bae887a0 20 read_log 194422'13141 (194422'13140) modify 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by client.1148203.0:7265 2014-08-13 15:32:47.129206 2014-08-13 17:38:03.100421 7fe9bae887a0 20 read_log 194422'13142 (0'0) promote d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by osd.395.0:4602 2014-08-13 15:32:50.373573 2014-08-13 17:38:03.100433 7fe9bae887a0 20 read_log 194422'13143 (194422'13142) modify d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by client.1148203.0:7782 2014-08-13 15:32:50.364448 2014-08-13 17:38:03.100445 7fe9bae887a0 20 read_log 194422'13144 (0'0) promote 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by osd.395.0:4612 2014-08-13 15:32:54.349970 2014-08-13 17:38:03.100457 7fe9bae887a0 20 read_log 194422'13145 (194422'13144) modify 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by client.1148203.0:8400 2014-08-13 15:32:54.340862 2014-08-13 17:38:03.100469 7fe9bae887a0 20 read_log 194422'13146 (0'0) promote b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by osd.395.0:4618 2014-08-13 15:32:55.512494 2014-08-13 17:38:03.100481 7fe9bae887a0 20 read_log 194422'13147 (194422'13146) modify b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by client.1148203.0:8556 2014-08-13 15:32:55.498245 2014-08-13 17:38:03.100493 7fe9bae887a0 20 read_log 194422'13148 (0'0) promote c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by osd.395.0:4626 2014-08-13 15:32:56.934578 2014-08-13 17:38:03.100506 7fe9bae887a0 20 read_log 194422'13149 (194422'13148) modify c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by client.1148203.0:8728 2014-08-13 15:32:56.925677 2014-08-13 17:38:03.100518 7fe9bae887a0 20 read_log 194422'13150 (0'0) promote 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by osd.395.0:4632 2014-08-13 15:32:57.924797 2014-08-13 17:38:03.100530 7fe9bae887a0 20 read_log 194422'13151 (194422'13150) modify 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by client.1148203.0:8832 2014-08-13 15:32:57.916210 2014-08-13 17:38:03.100545 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.100929 7fe9bae887a0 10 read_log checking for missing items over interval (194370'13054,194422'13151] 2014-08-13 17:38:03.100976 7fe9bae887a0 15 read_log missing 194422'13151 (194422'13150) modify 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by client.1148203.0:8832 2014-08-13 15:32:57.916210 2014-08-13 17:38:03.101017 7fe9bae887a0 15 read_log missing 194422'13149 (194422'13148) modify c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by client.1148203.0:8728 2014-08-13 15:32:56.925677 2014-08-13 17:38:03.101054 7fe9bae887a0 15 read_log missing 194422'13147 (194422'13146) modify b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by client.1148203.0:8556 2014-08-13 15:32:55.498245 2014-08-13 17:38:03.101090 7fe9bae887a0 15 read_log missing 194422'13145 (194422'13144) modify 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by client.1148203.0:8400 2014-08-13 15:32:54.340862 2014-08-13 17:38:03.101127 7fe9bae887a0 15 read_log missing 194422'13143 (194422'13142) modify d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by client.1148203.0:7782 2014-08-13 15:32:50.364448 2014-08-13 17:38:03.101175 7fe9bae887a0 15 read_log missing 194422'13141 (194422'13140) modify 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by client.1148203.0:7265 2014-08-13 15:32:47.129206 2014-08-13 17:38:03.101213 7fe9bae887a0 15 read_log missing 194422'13139 (194422'13138) modify bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by client.1148203.0:5971 2014-08-13 15:32:39.963483 2014-08-13 17:38:03.101251 7fe9bae887a0 15 read_log missing 194422'13137 (194422'13136) modify bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by client.1148203.0:5737 2014-08-13 15:32:38.736377 2014-08-13 17:38:03.101288 7fe9bae887a0 15 read_log missing 194422'13135 (194422'13134) modify a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by client.1148203.0:5434 2014-08-13 15:32:36.992775 2014-08-13 17:38:03.101324 7fe9bae887a0 15 read_log missing 194422'13133 (194422'13132) modify 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by client.1148203.0:5259 2014-08-13 15:32:36.130829 2014-08-13 17:38:03.101361 7fe9bae887a0 15 read_log missing 194422'13131 (194422'13130) modify ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by client.1148203.0:5110 2014-08-13 15:32:35.372868 2014-08-13 17:38:03.101399 7fe9bae887a0 15 read_log missing 194422'13129 (194422'13128) modify 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by client.1148203.0:4016 2014-08-13 15:32:31.046978 2014-08-13 17:38:03.101436 7fe9bae887a0 15 read_log missing 194422'13127 (194422'13126) modify 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by client.1148203.0:3151 2014-08-13 15:32:27.666329 2014-08-13 17:38:03.101472 7fe9bae887a0 15 read_log missing 194422'13125 (194422'13124) modify 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by client.1148203.0:2710 2014-08-13 15:32:25.923689 2014-08-13 17:38:03.101509 7fe9bae887a0 15 read_log missing 194422'13123 (194422'13122) modify de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by client.1148203.0:2585 2014-08-13 15:32:25.460245 2014-08-13 17:38:03.101545 7fe9bae887a0 15 read_log missing 194422'13121 (194422'13120) modify c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by client.1148203.0:1750 2014-08-13 15:32:22.249403 2014-08-13 17:38:03.101582 7fe9bae887a0 15 read_log missing 194422'13119 (194422'13118) modify 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by client.1148203.0:874 2014-08-13 15:32:18.813722 2014-08-13 17:38:03.101619 7fe9bae887a0 15 read_log missing 194422'13117 (194422'13116) modify cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by client.1148203.0:295 2014-08-13 15:32:16.344464 2014-08-13 17:38:03.101657 7fe9bae887a0 15 read_log missing 194416'13115 (194416'13114) modify 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by client.2441958.0:18464 2014-08-13 15:30:12.126663 2014-08-13 17:38:03.101694 7fe9bae887a0 15 read_log missing 194416'13113 (194416'13112) modify 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by client.2441958.0:18433 2014-08-13 15:30:11.249320 2014-08-13 17:38:03.101730 7fe9bae887a0 15 read_log missing 194416'13111 (194416'13110) modify f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by client.2441958.0:17046 2014-08-13 15:29:47.991603 2014-08-13 17:38:03.101767 7fe9bae887a0 15 read_log missing 194416'13109 (194416'13108) modify 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by client.2441958.0:15592 2014-08-13 15:29:33.188173 2014-08-13 17:38:03.101804 7fe9bae887a0 15 read_log missing 194416'13107 (194416'13106) modify b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by client.2441958.0:14864 2014-08-13 15:29:29.685386 2014-08-13 17:38:03.101841 7fe9bae887a0 15 read_log missing 194416'13105 (194416'13104) modify 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by client.2441958.0:14574 2014-08-13 15:29:28.490784 2014-08-13 17:38:03.101877 7fe9bae887a0 15 read_log missing 194416'13103 (194416'13102) modify c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by client.2441958.0:12410 2014-08-13 15:29:19.345491 2014-08-13 17:38:03.101915 7fe9bae887a0 15 read_log missing 194416'13101 (194416'13100) modify 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by client.2441958.0:7204 2014-08-13 15:28:58.533818 2014-08-13 17:38:03.101952 7fe9bae887a0 15 read_log missing 194416'13099 (194416'13098) modify 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by client.2441958.0:6554 2014-08-13 15:28:56.060052 2014-08-13 17:38:03.101989 7fe9bae887a0 15 read_log missing 194416'13097 (194416'13096) modify 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by client.2441958.0:5321 2014-08-13 15:28:51.114390 2014-08-13 17:38:03.102026 7fe9bae887a0 15 read_log missing 194416'13095 (194416'13094) modify 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by client.2441958.0:3528 2014-08-13 15:28:44.070814 2014-08-13 17:38:03.102062 7fe9bae887a0 15 read_log missing 194383'13093 (194383'13092) modify 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by client.2231190.0:7317 2014-08-12 20:34:23.779474 2014-08-13 17:38:03.102099 7fe9bae887a0 15 read_log missing 194383'13091 (194383'13090) modify fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by client.2231190.0:7146 2014-08-12 20:34:19.077512 2014-08-13 17:38:03.102136 7fe9bae887a0 15 read_log missing 194383'13089 (194383'13088) modify d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by client.2231190.0:6498 2014-08-12 20:34:04.278600 2014-08-13 17:38:03.102178 7fe9bae887a0 15 read_log missing 194380'13087 (194380'13086) modify 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by client.2231190.0:5187 2014-08-12 20:33:25.601065 2014-08-13 17:38:03.102217 7fe9bae887a0 15 read_log missing 194377'13085 (194377'13084) modify b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by client.2231190.0:4484 2014-08-12 20:33:17.034718 2014-08-13 17:38:03.102252 7fe9bae887a0 15 read_log missing 194377'13083 (194377'13082) modify 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by client.2231190.0:2377 2014-08-12 20:33:06.741091 2014-08-13 17:38:03.102290 7fe9bae887a0 15 read_log missing 194377'13081 (194377'13080) modify 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by client.2231190.0:2299 2014-08-12 20:33:06.453284 2014-08-13 17:38:03.102350 7fe9bae887a0 15 read_log missing 194377'13079 (194377'13078) modify d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by client.2231190.0:1476 2014-08-12 20:33:03.102034 2014-08-13 17:38:03.102388 7fe9bae887a0 15 read_log missing 194370'13077 (194370'13076) modify e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by client.1508531.0:13763 2014-08-12 20:31:26.761274 2014-08-13 17:38:03.102425 7fe9bae887a0 15 read_log missing 194370'13075 (194370'13074) modify 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by client.1508531.0:12970 2014-08-12 20:31:19.077634 2014-08-13 17:38:03.102462 7fe9bae887a0 15 read_log missing 194370'13073 (194370'13072) modify c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by client.1508531.0:9643 2014-08-12 20:30:57.159265 2014-08-13 17:38:03.102500 7fe9bae887a0 15 read_log missing 194370'13071 (194370'13070) modify dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by client.1508531.0:8156 2014-08-12 20:30:48.410034 2014-08-13 17:38:03.102536 7fe9bae887a0 15 read_log missing 194370'13069 (194370'13068) modify 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by client.1508531.0:6688 2014-08-12 20:30:40.829729 2014-08-13 17:38:03.102574 7fe9bae887a0 15 read_log missing 194370'13067 (194370'13066) modify fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by client.1508531.0:6465 2014-08-12 20:30:39.791322 2014-08-13 17:38:03.102611 7fe9bae887a0 15 read_log missing 194370'13065 (194370'13064) modify 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by client.1508531.0:6231 2014-08-12 20:30:38.774709 2014-08-13 17:38:03.102647 7fe9bae887a0 15 read_log missing 194370'13063 (194370'13062) modify 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by client.1508531.0:4908 2014-08-12 20:30:32.794881 2014-08-13 17:38:03.102684 7fe9bae887a0 15 read_log missing 194370'13061 (194370'13060) modify b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by client.1508531.0:4231 2014-08-12 20:30:30.202760 2014-08-13 17:38:03.102721 7fe9bae887a0 15 read_log missing 194370'13059 (194370'13058) modify e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by client.1508531.0:4031 2014-08-12 20:30:29.406321 2014-08-13 17:38:03.102758 7fe9bae887a0 15 read_log missing 194370'13057 (194370'13056) modify fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by client.1508531.0:3028 2014-08-12 20:30:25.643346 2014-08-13 17:38:03.102795 7fe9bae887a0 15 read_log missing 194370'13055 (194370'13054) modify 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by client.1508531.0:869 2014-08-12 20:30:17.645600 2014-08-13 17:38:03.102806 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.102836 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive m=49] handle_loaded 2014-08-13 17:38:03.102853 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.012503 0 0.000000 2014-08-13 17:38:03.102865 7fe9bae887a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Reset 2014-08-13 17:38:03.102874 7fe9bae887a0 20 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194470 2014-08-13 17:38:03.102883 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] log((192718'10533,194422'13151], crt=194422'13151) 2014-08-13 17:38:03.102893 7fe9bae887a0 10 osd.4 194478 pgid 24.3b7 coll 24.3b7_head 2014-08-13 17:38:03.103002 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.3b7 2014-08-13 17:38:03.103015 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.103058 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3b7(unlocked)] enter Initial 2014-08-13 17:38:03.103069 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.3b7(unlocked)] enter NotTrimming 2014-08-13 17:38:03.103209 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.103336 7fe9bae887a0 20 read_log 192718'15727 (192718'15726) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:280 2014-08-06 15:23:12.592898 2014-08-13 17:38:03.103356 7fe9bae887a0 20 read_log 192718'15728 (192718'15727) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:280 2014-08-06 15:23:12.620061 2014-08-13 17:38:03.103369 7fe9bae887a0 20 read_log 192718'15729 (192718'15728) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:281 2014-08-06 15:23:12.680961 2014-08-13 17:38:03.103382 7fe9bae887a0 20 read_log 192718'15730 (192718'15729) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:281 2014-08-06 15:23:12.706703 2014-08-13 17:38:03.103394 7fe9bae887a0 20 read_log 192718'15731 (192718'15730) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:282 2014-08-06 15:23:12.748671 2014-08-13 17:38:03.103405 7fe9bae887a0 20 read_log 192718'15732 (192718'15731) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:282 2014-08-06 15:23:12.766485 2014-08-13 17:38:03.103425 7fe9bae887a0 20 read_log 192718'15733 (192718'15732) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:283 2014-08-06 15:23:12.797400 2014-08-13 17:38:03.103438 7fe9bae887a0 20 read_log 192718'15734 (192718'15733) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:283 2014-08-06 15:23:12.823267 2014-08-13 17:38:03.103450 7fe9bae887a0 20 read_log 192718'15735 (192718'15734) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:284 2014-08-06 15:23:12.852654 2014-08-13 17:38:03.103462 7fe9bae887a0 20 read_log 192718'15736 (192718'15735) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:284 2014-08-06 15:23:12.879426 2014-08-13 17:38:03.103474 7fe9bae887a0 20 read_log 192718'15737 (192718'15736) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:285 2014-08-06 15:23:12.926570 2014-08-13 17:38:03.103485 7fe9bae887a0 20 read_log 192718'15738 (192718'15737) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:285 2014-08-06 15:23:12.934988 2014-08-13 17:38:03.103497 7fe9bae887a0 20 read_log 192718'15739 (192718'15738) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:286 2014-08-06 15:23:12.964785 2014-08-13 17:38:03.103509 7fe9bae887a0 20 read_log 192718'15740 (192718'15739) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:286 2014-08-06 15:23:12.990135 2014-08-13 17:38:03.103521 7fe9bae887a0 20 read_log 192718'15741 (192718'15740) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:287 2014-08-06 15:23:13.023338 2014-08-13 17:38:03.103533 7fe9bae887a0 20 read_log 192718'15742 (192718'15741) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:287 2014-08-06 15:23:13.218211 2014-08-13 17:38:03.103545 7fe9bae887a0 20 read_log 192718'15743 (192718'15742) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:288 2014-08-06 15:23:13.248739 2014-08-13 17:38:03.103556 7fe9bae887a0 20 read_log 192718'15744 (192718'15743) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:288 2014-08-06 15:23:13.277080 2014-08-13 17:38:03.103568 7fe9bae887a0 20 read_log 192718'15745 (192718'15744) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:289 2014-08-06 15:23:13.308129 2014-08-13 17:38:03.103579 7fe9bae887a0 20 read_log 192718'15746 (192718'15745) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:289 2014-08-06 15:23:13.334989 2014-08-13 17:38:03.103591 7fe9bae887a0 20 read_log 192718'15747 (192718'15746) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:290 2014-08-06 15:23:13.367015 2014-08-13 17:38:03.103603 7fe9bae887a0 20 read_log 192718'15748 (192718'15747) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:290 2014-08-06 15:23:13.385246 2014-08-13 17:38:03.103614 7fe9bae887a0 20 read_log 192718'15749 (192718'15748) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:291 2014-08-06 15:23:13.416462 2014-08-13 17:38:03.103626 7fe9bae887a0 20 read_log 192718'15750 (192718'15749) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:291 2014-08-06 15:23:13.438750 2014-08-13 17:38:03.103638 7fe9bae887a0 20 read_log 192718'15751 (192718'15750) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:292 2014-08-06 15:23:13.478822 2014-08-13 17:38:03.103650 7fe9bae887a0 20 read_log 192718'15752 (192718'15751) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:292 2014-08-06 15:23:13.632401 2014-08-13 17:38:03.103662 7fe9bae887a0 20 read_log 192718'15753 (192718'15752) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:293 2014-08-06 15:23:13.661490 2014-08-13 17:38:03.103674 7fe9bae887a0 20 read_log 192718'15754 (192718'15753) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:293 2014-08-06 15:23:13.686672 2014-08-13 17:38:03.103685 7fe9bae887a0 20 read_log 192718'15755 (192718'15754) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:294 2014-08-06 15:23:13.715513 2014-08-13 17:38:03.103697 7fe9bae887a0 20 read_log 192718'15756 (192718'15755) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:294 2014-08-06 15:23:13.741236 2014-08-13 17:38:03.103717 7fe9bae887a0 20 read_log 192718'15757 (192718'15756) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:295 2014-08-06 15:23:13.778692 2014-08-13 17:38:03.103730 7fe9bae887a0 20 read_log 192718'15758 (192718'15757) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:295 2014-08-06 15:23:13.795316 2014-08-13 17:38:03.103741 7fe9bae887a0 20 read_log 192718'15759 (192718'15758) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:296 2014-08-06 15:23:13.827873 2014-08-13 17:38:03.103753 7fe9bae887a0 20 read_log 192718'15760 (192718'15759) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:296 2014-08-06 15:23:13.853232 2014-08-13 17:38:03.103765 7fe9bae887a0 20 read_log 192718'15761 (192718'15760) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:297 2014-08-06 15:23:13.895526 2014-08-13 17:38:03.103777 7fe9bae887a0 20 read_log 192718'15762 (192718'15761) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:297 2014-08-06 15:23:14.039796 2014-08-13 17:38:03.103789 7fe9bae887a0 20 read_log 192718'15763 (192718'15762) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:298 2014-08-06 15:23:14.063721 2014-08-13 17:38:03.103801 7fe9bae887a0 20 read_log 192718'15764 (192718'15763) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:298 2014-08-06 15:23:14.089812 2014-08-13 17:38:03.103813 7fe9bae887a0 20 read_log 192718'15865 (192718'15558) delete a59f37b7/core.10872+97+2/head//24 by osd.317.0:17237 2014-08-06 15:31:07.383440 2014-08-13 17:38:03.103825 7fe9bae887a0 20 read_log 192718'16433 (192718'15764) clean 9148e3b7/core.50654+12+2/head//24 by osd.317.0:17806 2014-08-06 15:48:41.860477 2014-08-13 17:38:03.103837 7fe9bae887a0 20 read_log 192798'16436 (192718'16433) delete 9148e3b7/core.50654+12+2/head//24 by osd.317.0:18127 2014-08-11 10:24:43.624903 2014-08-13 17:38:03.103848 7fe9bae887a0 20 read_log 192803'16440 (192718'14158) delete 339f5bb7/usr.tar3+38+2/head//24 by osd.317.0:18131 2014-08-11 10:30:28.665138 2014-08-13 17:38:03.103861 7fe9bae887a0 20 read_log 192834'17023 (0'0) promote f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.317.0:19065 2014-08-11 15:23:40.530492 2014-08-13 17:38:03.103873 7fe9bae887a0 20 read_log 192834'17024 (192834'17023) modify f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by client.933448.0:1255 2014-08-11 15:23:40.522528 2014-08-13 17:38:03.103886 7fe9bae887a0 20 read_log 192834'17039 (0'0) promote 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.317.0:19081 2014-08-11 15:23:48.797531 2014-08-13 17:38:03.103898 7fe9bae887a0 20 read_log 192834'17040 (192834'17039) modify 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by client.933448.0:2834 2014-08-11 15:23:48.788658 2014-08-13 17:38:03.103910 7fe9bae887a0 20 read_log 192834'17055 (0'0) promote f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by osd.317.0:19097 2014-08-11 15:23:55.217781 2014-08-13 17:38:03.103922 7fe9bae887a0 20 read_log 192834'17056 (192834'17055) modify f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.933448.0:4043 2014-08-11 15:23:55.208817 2014-08-13 17:38:03.103955 7fe9bae887a0 20 read_log 192834'17097 (0'0) promote 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by osd.317.0:19139 2014-08-11 15:24:09.574408 2014-08-13 17:38:03.103970 7fe9bae887a0 20 read_log 192834'17098 (192834'17097) modify 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.933448.0:7310 2014-08-11 15:24:09.566590 2014-08-13 17:38:03.103983 7fe9bae887a0 20 read_log 192834'17155 (0'0) promote 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.317.0:19197 2014-08-11 15:24:26.675065 2014-08-13 17:38:03.103996 7fe9bae887a0 20 read_log 192834'17156 (192834'17155) modify 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by client.933448.0:11163 2014-08-11 15:24:26.667210 2014-08-13 17:38:03.104066 7fe9bae887a0 20 read_log 192834'17165 (0'0) promote b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by osd.317.0:19207 2014-08-11 15:24:32.317027 2014-08-13 17:38:03.104083 7fe9bae887a0 20 read_log 192834'17166 (192834'17165) modify b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.933448.0:12429 2014-08-11 15:24:32.310013 2014-08-13 17:38:03.104096 7fe9bae887a0 20 read_log 192834'17181 (0'0) promote 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by osd.317.0:19223 2014-08-11 15:24:34.594620 2014-08-13 17:38:03.104108 7fe9bae887a0 20 read_log 192834'17182 (192834'17181) modify 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.933448.0:12957 2014-08-11 15:24:34.583709 2014-08-13 17:38:03.104123 7fe9bae887a0 20 read_log 192834'17185 (0'0) promote 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by osd.317.0:19227 2014-08-11 15:24:35.733848 2014-08-13 17:38:03.104135 7fe9bae887a0 20 read_log 192834'17186 (192834'17185) modify 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.933448.0:13175 2014-08-11 15:24:35.724085 2014-08-13 17:38:03.104150 7fe9bae887a0 20 read_log 192834'17195 (0'0) promote 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by osd.317.0:19237 2014-08-11 15:24:40.101207 2014-08-13 17:38:03.104168 7fe9bae887a0 20 read_log 192834'17196 (192834'17195) modify 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.933448.0:13826 2014-08-11 15:24:40.092251 2014-08-13 17:38:03.104181 7fe9bae887a0 20 read_log 192834'17211 (0'0) promote abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.317.0:19253 2014-08-11 15:24:45.960382 2014-08-13 17:38:03.104193 7fe9bae887a0 20 read_log 192834'17212 (192834'17211) modify abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by client.933448.0:14515 2014-08-11 15:24:45.951967 2014-08-13 17:38:03.104208 7fe9bae887a0 20 read_log 192834'17219 (0'0) promote 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by osd.317.0:19261 2014-08-11 15:24:49.736097 2014-08-13 17:38:03.104220 7fe9bae887a0 20 read_log 192834'17220 (192834'17219) modify 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.933448.0:15458 2014-08-11 15:24:49.722546 2014-08-13 17:38:03.104234 7fe9bae887a0 20 read_log 192834'17231 (0'0) promote 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.317.0:19273 2014-08-11 15:24:53.670689 2014-08-13 17:38:03.104246 7fe9bae887a0 20 read_log 192834'17232 (192834'17231) modify 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by client.933448.0:16239 2014-08-11 15:24:53.662692 2014-08-13 17:38:03.104261 7fe9bae887a0 20 read_log 192834'17241 (0'0) promote 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by osd.317.0:19283 2014-08-11 15:24:56.539699 2014-08-13 17:38:03.104273 7fe9bae887a0 20 read_log 192834'17242 (192834'17241) modify 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.933448.0:16643 2014-08-11 15:24:56.531915 2014-08-13 17:38:03.104285 7fe9bae887a0 20 read_log 192834'17255 (0'0) promote 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.317.0:19297 2014-08-11 15:24:59.371890 2014-08-13 17:38:03.104297 7fe9bae887a0 20 read_log 192834'17256 (192834'17255) modify 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by client.933448.0:17350 2014-08-11 15:24:59.364431 2014-08-13 17:38:03.104309 7fe9bae887a0 20 read_log 192834'17259 (0'0) promote 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.317.0:19301 2014-08-11 15:25:01.082938 2014-08-13 17:38:03.104321 7fe9bae887a0 20 read_log 192834'17260 (192834'17259) modify 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by client.933448.0:17698 2014-08-11 15:25:01.074830 2014-08-13 17:38:03.104341 7fe9bae887a0 20 read_log 192834'17275 (0'0) promote f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by osd.317.0:19317 2014-08-11 15:25:05.949622 2014-08-13 17:38:03.104357 7fe9bae887a0 20 read_log 192834'17276 (192834'17275) modify f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.933448.0:18605 2014-08-11 15:25:05.940910 2014-08-13 17:38:03.104370 7fe9bae887a0 20 read_log 192834'17317 (0'0) promote 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by osd.317.0:19359 2014-08-11 15:25:30.668946 2014-08-13 17:38:03.104382 7fe9bae887a0 20 read_log 192834'17318 (192834'17317) modify 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.933448.0:21493 2014-08-11 15:25:30.658927 2014-08-13 17:38:03.104394 7fe9bae887a0 20 read_log 192834'17330 (192834'17024) modify f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by client.933448.0:23546 2014-08-11 15:25:39.464984 2014-08-13 17:38:03.104406 7fe9bae887a0 20 read_log 192834'17338 (192834'17040) modify 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by client.933448.0:25125 2014-08-11 15:25:41.807109 2014-08-13 17:38:03.104418 7fe9bae887a0 20 read_log 192834'17346 (192834'17056) modify f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.933448.0:26334 2014-08-11 15:25:43.279758 2014-08-13 17:38:03.104431 7fe9bae887a0 20 read_log 192834'17367 (192834'17098) modify 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.933448.0:29601 2014-08-11 15:25:47.512335 2014-08-13 17:38:03.104443 7fe9bae887a0 20 read_log 192834'17396 (192834'17156) modify 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by client.933448.0:33454 2014-08-11 15:25:53.524996 2014-08-13 17:38:03.104454 7fe9bae887a0 20 read_log 192834'17401 (192834'17166) modify b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.933448.0:34720 2014-08-11 15:25:55.390367 2014-08-13 17:38:03.104466 7fe9bae887a0 20 read_log 192834'17409 (192834'17182) modify 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.933448.0:35248 2014-08-11 15:25:56.004388 2014-08-13 17:38:03.104478 7fe9bae887a0 20 read_log 192834'17411 (192834'17186) modify 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.933448.0:35466 2014-08-11 15:25:56.320554 2014-08-13 17:38:03.104490 7fe9bae887a0 20 read_log 192834'17416 (192834'17196) modify 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.933448.0:36117 2014-08-11 15:25:57.257164 2014-08-13 17:38:03.104502 7fe9bae887a0 20 read_log 192834'17424 (192834'17212) modify abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by client.933448.0:36806 2014-08-11 15:25:58.330423 2014-08-13 17:38:03.104514 7fe9bae887a0 20 read_log 192834'17428 (192834'17220) modify 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.933448.0:37749 2014-08-11 15:25:59.949825 2014-08-13 17:38:03.104526 7fe9bae887a0 20 read_log 192834'17434 (192834'17232) modify 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by client.933448.0:38530 2014-08-11 15:26:00.971272 2014-08-13 17:38:03.104538 7fe9bae887a0 20 read_log 192834'17439 (192834'17242) modify 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.933448.0:38934 2014-08-11 15:26:01.412493 2014-08-13 17:38:03.104550 7fe9bae887a0 20 read_log 192834'17446 (192834'17256) modify 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by client.933448.0:39641 2014-08-11 15:26:02.007815 2014-08-13 17:38:03.104562 7fe9bae887a0 20 read_log 192834'17448 (192834'17260) modify 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by client.933448.0:39989 2014-08-11 15:26:02.205566 2014-08-13 17:38:03.104574 7fe9bae887a0 20 read_log 192834'17456 (192834'17276) modify f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.933448.0:40896 2014-08-11 15:26:02.773871 2014-08-13 17:38:03.104597 7fe9bae887a0 20 read_log 192834'17477 (192834'17318) modify 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.933448.0:43784 2014-08-11 15:26:03.723468 2014-08-13 17:38:03.104611 7fe9bae887a0 20 read_log 192834'17490 (0'0) promote 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by osd.317.0:19532 2014-08-11 15:26:18.147259 2014-08-13 17:38:03.104623 7fe9bae887a0 20 read_log 192834'17491 (192834'17490) modify 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.1883634.0:1005 2014-08-11 15:26:18.139396 2014-08-13 17:38:03.104635 7fe9bae887a0 20 read_log 192834'17542 (0'0) promote a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by osd.317.0:19584 2014-08-11 15:26:27.736745 2014-08-13 17:38:03.104647 7fe9bae887a0 20 read_log 192834'17543 (192834'17542) modify a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.1883634.0:3643 2014-08-11 15:26:27.727335 2014-08-13 17:38:03.104659 7fe9bae887a0 20 read_log 192834'17544 (0'0) promote 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by osd.317.0:19586 2014-08-11 15:26:28.512710 2014-08-13 17:38:03.104671 7fe9bae887a0 20 read_log 192834'17545 (192834'17544) modify 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.1883634.0:3863 2014-08-11 15:26:28.504815 2014-08-13 17:38:03.104683 7fe9bae887a0 20 read_log 192834'17564 (0'0) promote acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by osd.317.0:19606 2014-08-11 15:26:34.148592 2014-08-13 17:38:03.104695 7fe9bae887a0 20 read_log 192834'17565 (192834'17564) modify acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.1883634.0:5366 2014-08-11 15:26:34.141462 2014-08-13 17:38:03.104708 7fe9bae887a0 20 read_log 192834'17570 (0'0) promote adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.317.0:19612 2014-08-11 15:26:36.318268 2014-08-13 17:38:03.104720 7fe9bae887a0 20 read_log 192834'17571 (192834'17570) modify adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by client.1883634.0:5974 2014-08-11 15:26:36.310277 2014-08-13 17:38:03.104732 7fe9bae887a0 20 read_log 192834'17586 (0'0) promote d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by osd.317.0:19628 2014-08-11 15:26:40.195846 2014-08-13 17:38:03.104744 7fe9bae887a0 20 read_log 192834'17587 (192834'17586) modify d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.1883634.0:6962 2014-08-11 15:26:40.188026 2014-08-13 17:38:03.104756 7fe9bae887a0 20 read_log 192834'17596 (0'0) promote 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by osd.317.0:19638 2014-08-11 15:26:45.160281 2014-08-13 17:38:03.104768 7fe9bae887a0 20 read_log 192834'17597 (192834'17596) modify 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.1883634.0:8148 2014-08-11 15:26:45.150794 2014-08-13 17:38:03.104780 7fe9bae887a0 20 read_log 192834'17598 (0'0) promote eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by osd.317.0:19640 2014-08-11 15:26:46.077398 2014-08-13 17:38:03.104792 7fe9bae887a0 20 read_log 192834'17599 (192834'17598) modify eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.1883634.0:8391 2014-08-11 15:26:46.068076 2014-08-13 17:38:03.104804 7fe9bae887a0 20 read_log 192834'17608 (0'0) promote 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by osd.317.0:19650 2014-08-11 15:26:49.682007 2014-08-13 17:38:03.104816 7fe9bae887a0 20 read_log 192834'17609 (192834'17608) modify 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.1883634.0:9155 2014-08-11 15:26:49.670694 2014-08-13 17:38:03.104829 7fe9bae887a0 20 read_log 192834'17628 (0'0) promote 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by osd.317.0:19670 2014-08-11 15:26:53.726406 2014-08-13 17:38:03.104850 7fe9bae887a0 20 read_log 192834'17629 (192834'17628) modify 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.1883634.0:10109 2014-08-11 15:26:53.717496 2014-08-13 17:38:03.104863 7fe9bae887a0 20 read_log 192834'17644 (0'0) promote f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by osd.317.0:19686 2014-08-11 15:26:58.566572 2014-08-13 17:38:03.104876 7fe9bae887a0 20 read_log 192834'17645 (192834'17644) modify f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.1883634.0:11155 2014-08-11 15:26:58.557201 2014-08-13 17:38:03.104888 7fe9bae887a0 20 read_log 192834'17654 (0'0) promote db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by osd.317.0:19696 2014-08-11 15:27:02.761501 2014-08-13 17:38:03.104900 7fe9bae887a0 20 read_log 192834'17655 (192834'17654) modify db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.1883634.0:12158 2014-08-11 15:27:02.752094 2014-08-13 17:38:03.104912 7fe9bae887a0 20 read_log 192834'17710 (0'0) promote eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by osd.317.0:19752 2014-08-11 15:27:29.574624 2014-08-13 17:38:03.104925 7fe9bae887a0 20 read_log 192834'17711 (192834'17710) modify eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.1883634.0:16838 2014-08-11 15:27:29.564982 2014-08-13 17:38:03.104937 7fe9bae887a0 20 read_log 192834'17714 (0'0) promote 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by osd.317.0:19756 2014-08-11 15:27:34.012370 2014-08-13 17:38:03.104949 7fe9bae887a0 20 read_log 192834'17715 (192834'17714) modify 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.1883634.0:17274 2014-08-11 15:27:34.002352 2014-08-13 17:38:03.104961 7fe9bae887a0 20 read_log 192834'17728 (0'0) promote b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by osd.317.0:19770 2014-08-11 15:27:47.104103 2014-08-13 17:38:03.104973 7fe9bae887a0 20 read_log 192834'17729 (192834'17728) modify b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.1883634.0:18691 2014-08-11 15:27:47.093957 2014-08-13 17:38:03.104985 7fe9bae887a0 20 read_log 192834'17732 (0'0) promote 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by osd.317.0:19774 2014-08-11 15:27:47.773474 2014-08-13 17:38:03.104997 7fe9bae887a0 20 read_log 192834'17733 (192834'17732) modify 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.1883634.0:18834 2014-08-11 15:27:47.763955 2014-08-13 17:38:03.105009 7fe9bae887a0 20 read_log 192834'17736 (0'0) promote f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.317.0:19778 2014-08-11 15:27:48.597862 2014-08-13 17:38:03.105021 7fe9bae887a0 20 read_log 192834'17737 (192834'17736) modify f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by client.1883634.0:18978 2014-08-11 15:27:48.589213 2014-08-13 17:38:03.105033 7fe9bae887a0 20 read_log 192834'17744 (0'0) promote 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by osd.317.0:19786 2014-08-11 15:27:50.233709 2014-08-13 17:38:03.105045 7fe9bae887a0 20 read_log 192834'17745 (192834'17744) modify 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.1883634.0:19280 2014-08-11 15:27:50.225030 2014-08-13 17:38:03.105057 7fe9bae887a0 20 read_log 192834'17760 (0'0) promote e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.317.0:19802 2014-08-11 15:27:56.565945 2014-08-13 17:38:03.105068 7fe9bae887a0 20 read_log 192834'17761 (192834'17760) modify e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by client.1883634.0:20390 2014-08-11 15:27:56.555620 2014-08-13 17:38:03.105081 7fe9bae887a0 20 read_log 192834'17807 (192834'17491) modify 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.1883634.0:24329 2014-08-11 15:28:18.915962 2014-08-13 17:38:03.105100 7fe9bae887a0 20 read_log 192834'17833 (192834'17543) modify a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.1883634.0:26967 2014-08-11 15:28:23.845686 2014-08-13 17:38:03.105113 7fe9bae887a0 20 read_log 192834'17834 (192834'17545) modify 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.1883634.0:27187 2014-08-11 15:28:24.149979 2014-08-13 17:38:03.105125 7fe9bae887a0 20 read_log 192834'17844 (192834'17565) modify acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.1883634.0:28690 2014-08-11 15:28:26.144243 2014-08-13 17:38:03.105137 7fe9bae887a0 20 read_log 192834'17847 (192834'17571) modify adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by client.1883634.0:29298 2014-08-11 15:28:26.947557 2014-08-13 17:38:03.105149 7fe9bae887a0 20 read_log 192834'17855 (192834'17587) modify d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.1883634.0:30286 2014-08-11 15:28:28.401415 2014-08-13 17:38:03.105166 7fe9bae887a0 20 read_log 192834'17860 (192834'17597) modify 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.1883634.0:31472 2014-08-11 15:28:30.577931 2014-08-13 17:38:03.105178 7fe9bae887a0 20 read_log 192834'17861 (192834'17599) modify eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.1883634.0:31715 2014-08-11 15:28:30.878325 2014-08-13 17:38:03.105190 7fe9bae887a0 20 read_log 192834'17866 (192834'17609) modify 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.1883634.0:32479 2014-08-11 15:28:31.912051 2014-08-13 17:38:03.105202 7fe9bae887a0 20 read_log 192834'17876 (192834'17629) modify 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.1883634.0:33433 2014-08-11 15:28:33.190837 2014-08-13 17:38:03.105220 7fe9bae887a0 20 read_log 192834'17884 (192834'17645) modify f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.1883634.0:34479 2014-08-11 15:28:34.875792 2014-08-13 17:38:03.105233 7fe9bae887a0 20 read_log 192834'17891 (192834'17655) modify db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.1883634.0:35482 2014-08-11 15:28:36.711352 2014-08-13 17:38:03.105245 7fe9bae887a0 20 read_log 192834'17919 (192834'17711) modify eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.1883634.0:40162 2014-08-11 15:28:43.279722 2014-08-13 17:38:03.105257 7fe9bae887a0 20 read_log 192834'17921 (192834'17715) modify 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.1883634.0:40598 2014-08-11 15:28:44.041766 2014-08-13 17:38:03.105269 7fe9bae887a0 20 read_log 192834'17928 (192834'17729) modify b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.1883634.0:42015 2014-08-11 15:28:45.636176 2014-08-13 17:38:03.105281 7fe9bae887a0 20 read_log 192834'17930 (192834'17733) modify 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.1883634.0:42158 2014-08-11 15:28:46.153680 2014-08-13 17:38:03.105293 7fe9bae887a0 20 read_log 192834'17932 (192834'17737) modify f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by client.1883634.0:42302 2014-08-11 15:28:46.200853 2014-08-13 17:38:03.105305 7fe9bae887a0 20 read_log 192834'17936 (192834'17745) modify 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.1883634.0:42604 2014-08-11 15:28:46.295428 2014-08-13 17:38:03.105317 7fe9bae887a0 20 read_log 192834'17944 (192834'17761) modify e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by client.1883634.0:43714 2014-08-11 15:28:46.640688 2014-08-13 17:38:03.105329 7fe9bae887a0 20 read_log 192888'17974 (0'0) promote 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:17514 2014-08-11 16:45:12.002365 2014-08-13 17:38:03.105349 7fe9bae887a0 20 read_log 192888'17975 (192888'17974) modify 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by client.1315259.0:4424 2014-08-11 16:45:11.994368 2014-08-13 17:38:03.105361 7fe9bae887a0 20 read_log 192888'17980 (0'0) promote 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by osd.356.0:17530 2014-08-11 16:45:19.410219 2014-08-13 17:38:03.105374 7fe9bae887a0 20 read_log 192888'17981 (192888'17980) modify 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.1315259.0:6450 2014-08-11 16:45:19.401855 2014-08-13 17:38:03.105386 7fe9bae887a0 20 read_log 192888'17982 (0'0) promote 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by osd.356.0:17536 2014-08-11 16:45:20.283910 2014-08-13 17:38:03.105398 7fe9bae887a0 20 read_log 192888'17983 (192888'17982) modify 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.1315259.0:6693 2014-08-11 16:45:20.276653 2014-08-13 17:38:03.105410 7fe9bae887a0 20 read_log 192888'17992 (0'0) promote 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:17550 2014-08-11 16:45:28.324798 2014-08-13 17:38:03.105422 7fe9bae887a0 20 read_log 192888'17993 (192888'17992) modify 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by client.1315259.0:8850 2014-08-11 16:45:28.317281 2014-08-13 17:38:03.105434 7fe9bae887a0 20 read_log 192888'17994 (0'0) promote f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by osd.356.0:17554 2014-08-11 16:45:30.386087 2014-08-13 17:38:03.105446 7fe9bae887a0 20 read_log 192888'17995 (192888'17994) modify f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.1315259.0:9407 2014-08-11 16:45:30.377349 2014-08-13 17:38:03.105458 7fe9bae887a0 20 read_log 192888'17996 (0'0) promote 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by osd.356.0:17558 2014-08-11 16:45:30.937502 2014-08-13 17:38:03.105469 7fe9bae887a0 20 read_log 192888'17997 (192888'17996) modify 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.1315259.0:9548 2014-08-11 16:45:30.930144 2014-08-13 17:38:03.105482 7fe9bae887a0 20 read_log 192888'17998 (0'0) promote 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:17560 2014-08-11 16:45:32.184642 2014-08-13 17:38:03.105493 7fe9bae887a0 20 read_log 192888'17999 (192888'17998) modify 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by client.1315259.0:9866 2014-08-11 16:45:32.175982 2014-08-13 17:38:03.105506 7fe9bae887a0 20 read_log 192888'18002 (0'0) promote 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by osd.356.0:17566 2014-08-11 16:45:34.219372 2014-08-13 17:38:03.105517 7fe9bae887a0 20 read_log 192888'18003 (192888'18002) modify 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.1315259.0:10402 2014-08-11 16:45:34.211276 2014-08-13 17:38:03.105529 7fe9bae887a0 20 read_log 192888'18004 (0'0) promote b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by osd.356.0:17568 2014-08-11 16:45:36.560867 2014-08-13 17:38:03.105541 7fe9bae887a0 20 read_log 192888'18005 (192888'18004) modify b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.1315259.0:10986 2014-08-11 16:45:36.553773 2014-08-13 17:38:03.105553 7fe9bae887a0 20 read_log 192888'18006 (0'0) promote 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:17570 2014-08-11 16:45:36.568784 2014-08-13 17:38:03.105565 7fe9bae887a0 20 read_log 192888'18007 (192888'18006) modify 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by client.1315259.0:10988 2014-08-11 16:45:36.561494 2014-08-13 17:38:03.105580 7fe9bae887a0 20 read_log 192888'18008 (0'0) promote f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by osd.356.0:17574 2014-08-11 16:45:40.497053 2014-08-13 17:38:03.105601 7fe9bae887a0 20 read_log 192888'18009 (192888'18008) modify f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.1315259.0:11953 2014-08-11 16:45:40.489805 2014-08-13 17:38:03.105614 7fe9bae887a0 20 read_log 192888'18010 (0'0) promote 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by osd.356.0:17578 2014-08-11 16:45:41.704342 2014-08-13 17:38:03.105626 7fe9bae887a0 20 read_log 192888'18011 (192888'18010) modify 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.1315259.0:12261 2014-08-11 16:45:41.696943 2014-08-13 17:38:03.105641 7fe9bae887a0 20 read_log 192888'18012 (0'0) promote c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by osd.356.0:17582 2014-08-11 16:45:43.192582 2014-08-13 17:38:03.105653 7fe9bae887a0 20 read_log 192888'18013 (192888'18012) modify c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.1315259.0:12608 2014-08-11 16:45:43.185428 2014-08-13 17:38:03.105665 7fe9bae887a0 20 read_log 192888'18016 (0'0) promote 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by osd.356.0:17588 2014-08-11 16:45:46.666613 2014-08-13 17:38:03.105677 7fe9bae887a0 20 read_log 192888'18017 (192888'18016) modify 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.1315259.0:13448 2014-08-11 16:45:46.658781 2014-08-13 17:38:03.105691 7fe9bae887a0 20 read_log 192888'18018 (0'0) promote 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by osd.356.0:17592 2014-08-11 16:45:47.503893 2014-08-13 17:38:03.105703 7fe9bae887a0 20 read_log 192888'18019 (192888'18018) modify 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.1315259.0:13660 2014-08-11 16:45:47.492376 2014-08-13 17:38:03.105715 7fe9bae887a0 20 read_log 192888'18022 (0'0) promote 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by osd.356.0:17600 2014-08-11 16:45:48.487877 2014-08-13 17:38:03.105726 7fe9bae887a0 20 read_log 192888'18023 (192888'18022) modify 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.1315259.0:13924 2014-08-11 16:45:48.480282 2014-08-13 17:38:03.105739 7fe9bae887a0 20 read_log 192888'18024 (0'0) promote 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by osd.356.0:17604 2014-08-11 16:45:49.866046 2014-08-13 17:38:03.105750 7fe9bae887a0 20 read_log 192888'18025 (192888'18024) modify 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.1315259.0:14274 2014-08-11 16:45:49.857673 2014-08-13 17:38:03.105762 7fe9bae887a0 20 read_log 192888'18026 (0'0) promote f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:17606 2014-08-11 16:45:50.481070 2014-08-13 17:38:03.105774 7fe9bae887a0 20 read_log 192888'18027 (192888'18026) modify f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by client.1315259.0:14442 2014-08-11 16:45:50.475072 2014-08-13 17:38:03.105786 7fe9bae887a0 20 read_log 192888'18028 (0'0) promote 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by osd.356.0:17614 2014-08-11 16:45:52.806472 2014-08-13 17:38:03.105798 7fe9bae887a0 20 read_log 192888'18029 (192888'18028) modify 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.1315259.0:15063 2014-08-11 16:45:52.799840 2014-08-13 17:38:03.105810 7fe9bae887a0 20 read_log 192888'18030 (0'0) promote 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by osd.356.0:17618 2014-08-11 16:45:57.700734 2014-08-13 17:38:03.105822 7fe9bae887a0 20 read_log 192888'18031 (192888'18030) modify 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.1315259.0:16339 2014-08-11 16:45:57.693288 2014-08-13 17:38:03.105834 7fe9bae887a0 20 read_log 192888'18032 (0'0) promote eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by osd.356.0:17620 2014-08-11 16:45:57.824688 2014-08-13 17:38:03.105868 7fe9bae887a0 20 read_log 192888'18033 (192888'18032) modify eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.1315259.0:16372 2014-08-11 16:45:57.817071 2014-08-13 17:38:03.105882 7fe9bae887a0 20 read_log 192888'18036 (0'0) promote bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by osd.356.0:17628 2014-08-11 16:46:00.083738 2014-08-13 17:38:03.105894 7fe9bae887a0 20 read_log 192888'18037 (192888'18036) modify bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.1315259.0:16944 2014-08-11 16:46:00.076273 2014-08-13 17:38:03.105906 7fe9bae887a0 20 read_log 192888'18042 (0'0) promote c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:17636 2014-08-11 16:46:02.979924 2014-08-13 17:38:03.105918 7fe9bae887a0 20 read_log 192888'18043 (192888'18042) modify c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by client.1315259.0:17654 2014-08-11 16:46:02.972957 2014-08-13 17:38:03.105930 7fe9bae887a0 20 read_log 192888'18050 (0'0) promote 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:17660 2014-08-11 16:46:19.189029 2014-08-13 17:38:03.105942 7fe9bae887a0 20 read_log 192888'18051 (192888'18050) modify 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by client.1315259.0:21875 2014-08-11 16:46:19.181546 2014-08-13 17:38:03.105954 7fe9bae887a0 20 read_log 192888'18052 (0'0) promote 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:17670 2014-08-11 16:46:22.259285 2014-08-13 17:38:03.105966 7fe9bae887a0 20 read_log 192888'18053 (192888'18052) modify 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by client.1315259.0:22656 2014-08-11 16:46:22.252615 2014-08-13 17:38:03.105978 7fe9bae887a0 20 read_log 192888'18056 (0'0) promote 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:17682 2014-08-11 16:46:26.690193 2014-08-13 17:38:03.105990 7fe9bae887a0 20 read_log 192888'18057 (192888'18056) modify 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by client.1315259.0:23749 2014-08-11 16:46:26.682917 2014-08-13 17:38:03.106002 7fe9bae887a0 20 read_log 192888'18062 (0'0) promote 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by osd.356.0:17692 2014-08-11 16:46:31.248453 2014-08-13 17:38:03.106014 7fe9bae887a0 20 read_log 192888'18063 (192888'18062) modify 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.1315259.0:24928 2014-08-11 16:46:31.240772 2014-08-13 17:38:03.106026 7fe9bae887a0 20 read_log 192888'18066 (0'0) promote c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by osd.356.0:17698 2014-08-11 16:46:36.491539 2014-08-13 17:38:03.106038 7fe9bae887a0 20 read_log 192888'18067 (192888'18066) modify c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.1315259.0:26173 2014-08-11 16:46:36.483805 2014-08-13 17:38:03.106050 7fe9bae887a0 20 read_log 192888'18072 (0'0) promote d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by osd.356.0:17708 2014-08-11 16:46:41.218457 2014-08-13 17:38:03.106062 7fe9bae887a0 20 read_log 192888'18073 (192888'18072) modify d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.1315259.0:27296 2014-08-11 16:46:41.211637 2014-08-13 17:38:03.106074 7fe9bae887a0 20 read_log 192888'18080 (0'0) promote 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by osd.356.0:17720 2014-08-11 16:46:46.375284 2014-08-13 17:38:03.106086 7fe9bae887a0 20 read_log 192888'18081 (192888'18080) modify 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.1315259.0:28568 2014-08-11 16:46:46.368158 2014-08-13 17:38:03.106098 7fe9bae887a0 20 read_log 192888'18082 (0'0) promote e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by osd.356.0:17722 2014-08-11 16:46:48.138146 2014-08-13 17:38:03.106117 7fe9bae887a0 20 read_log 192888'18083 (192888'18082) modify e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.1315259.0:29031 2014-08-11 16:46:48.132040 2014-08-13 17:38:03.106130 7fe9bae887a0 20 read_log 192888'18084 (0'0) promote 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by osd.356.0:17724 2014-08-11 16:46:48.584329 2014-08-13 17:38:03.106142 7fe9bae887a0 20 read_log 192888'18085 (192888'18084) modify 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.1315259.0:29147 2014-08-11 16:46:48.576764 2014-08-13 17:38:03.106159 7fe9bae887a0 20 read_log 192888'18086 (0'0) promote cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by osd.356.0:17726 2014-08-11 16:46:48.983730 2014-08-13 17:38:03.106172 7fe9bae887a0 20 read_log 192888'18087 (192888'18086) modify cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.1315259.0:29236 2014-08-11 16:46:48.976894 2014-08-13 17:38:03.106185 7fe9bae887a0 20 read_log 192888'18090 (0'0) promote 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by osd.356.0:17732 2014-08-11 16:46:52.139828 2014-08-13 17:38:03.106197 7fe9bae887a0 20 read_log 192888'18091 (192888'18090) modify 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.1315259.0:30007 2014-08-11 16:46:52.132185 2014-08-13 17:38:03.106209 7fe9bae887a0 20 read_log 192888'18098 (192888'17975) modify 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by client.1315259.0:35151 2014-08-11 16:46:57.085177 2014-08-13 17:38:03.106221 7fe9bae887a0 20 read_log 192888'18101 (192888'17981) modify 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.1315259.0:37177 2014-08-11 16:46:57.614629 2014-08-13 17:38:03.106234 7fe9bae887a0 20 read_log 192888'18102 (192888'17983) modify 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.1315259.0:37420 2014-08-11 16:46:57.678696 2014-08-13 17:38:03.106246 7fe9bae887a0 20 read_log 192888'18107 (192888'17993) modify 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by client.1315259.0:39577 2014-08-11 16:46:58.248935 2014-08-13 17:38:03.106259 7fe9bae887a0 20 read_log 192888'18108 (192888'17995) modify f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.1315259.0:40134 2014-08-11 16:46:58.398546 2014-08-13 17:38:03.106272 7fe9bae887a0 20 read_log 192888'18109 (192888'17997) modify 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.1315259.0:40275 2014-08-11 16:46:58.439164 2014-08-13 17:38:03.106284 7fe9bae887a0 20 read_log 192888'18110 (192888'17999) modify 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by client.1315259.0:40593 2014-08-11 16:46:58.532314 2014-08-13 17:38:03.106297 7fe9bae887a0 20 read_log 192888'18112 (192888'18003) modify 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.1315259.0:41129 2014-08-11 16:46:58.666241 2014-08-13 17:38:03.106309 7fe9bae887a0 20 read_log 192888'18113 (192888'18005) modify b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.1315259.0:41713 2014-08-11 16:46:58.816973 2014-08-13 17:38:03.106321 7fe9bae887a0 20 read_log 192888'18114 (192888'18007) modify 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by client.1315259.0:41715 2014-08-11 16:46:58.817841 2014-08-13 17:38:03.106333 7fe9bae887a0 20 read_log 192888'18115 (192888'18009) modify f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.1315259.0:42680 2014-08-11 16:46:59.075527 2014-08-13 17:38:03.106345 7fe9bae887a0 20 read_log 192888'18116 (192888'18011) modify 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.1315259.0:42988 2014-08-11 16:46:59.159910 2014-08-13 17:38:03.106357 7fe9bae887a0 20 read_log 192888'18117 (192888'18013) modify c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.1315259.0:43335 2014-08-11 16:46:59.262756 2014-08-13 17:38:03.106378 7fe9bae887a0 20 read_log 192888'18119 (192888'18017) modify 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.1315259.0:44175 2014-08-11 16:46:59.503100 2014-08-13 17:38:03.106391 7fe9bae887a0 20 read_log 192888'18120 (192888'18019) modify 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.1315259.0:44387 2014-08-11 16:46:59.564089 2014-08-13 17:38:03.106403 7fe9bae887a0 20 read_log 192888'18122 (192888'18023) modify 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.1315259.0:44651 2014-08-11 16:46:59.639652 2014-08-13 17:38:03.106435 7fe9bae887a0 20 read_log 192888'18123 (192888'18025) modify 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.1315259.0:45001 2014-08-11 16:46:59.742512 2014-08-13 17:38:03.106447 7fe9bae887a0 20 read_log 192888'18124 (192888'18027) modify f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by client.1315259.0:45169 2014-08-11 16:46:59.790114 2014-08-13 17:38:03.106459 7fe9bae887a0 20 read_log 192888'18125 (192888'18029) modify 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.1315259.0:45790 2014-08-11 16:46:59.970401 2014-08-13 17:38:03.106471 7fe9bae887a0 20 read_log 192888'18126 (192888'18031) modify 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.1315259.0:47066 2014-08-11 16:47:00.336937 2014-08-13 17:38:03.106483 7fe9bae887a0 20 read_log 192888'18127 (192888'18033) modify eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.1315259.0:47099 2014-08-11 16:47:00.346308 2014-08-13 17:38:03.106495 7fe9bae887a0 20 read_log 192888'18129 (192888'18037) modify bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.1315259.0:47671 2014-08-11 16:47:00.529269 2014-08-13 17:38:03.106508 7fe9bae887a0 20 read_log 192888'18132 (192888'18043) modify c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by client.1315259.0:48381 2014-08-11 16:47:00.748918 2014-08-13 17:38:03.106520 7fe9bae887a0 20 read_log 192888'18136 (192888'18051) modify 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by client.1315259.0:52602 2014-08-11 16:47:02.019561 2014-08-13 17:38:03.106532 7fe9bae887a0 20 read_log 192888'18137 (192888'18053) modify 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by client.1315259.0:53383 2014-08-11 16:47:02.236567 2014-08-13 17:38:03.106544 7fe9bae887a0 20 read_log 192888'18139 (192888'18057) modify 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by client.1315259.0:54476 2014-08-11 16:47:02.576988 2014-08-13 17:38:03.106556 7fe9bae887a0 20 read_log 192888'18142 (192888'18063) modify 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.1315259.0:55655 2014-08-11 16:47:02.974342 2014-08-13 17:38:03.106568 7fe9bae887a0 20 read_log 192888'18144 (192888'18067) modify c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.1315259.0:56900 2014-08-11 16:47:03.348419 2014-08-13 17:38:03.106579 7fe9bae887a0 20 read_log 192888'18147 (192888'18073) modify d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.1315259.0:58023 2014-08-11 16:47:03.679815 2014-08-13 17:38:03.106592 7fe9bae887a0 20 read_log 192888'18151 (192888'18081) modify 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.1315259.0:59295 2014-08-11 16:47:04.067924 2014-08-13 17:38:03.106604 7fe9bae887a0 20 read_log 192888'18152 (192888'18083) modify e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.1315259.0:59758 2014-08-11 16:47:04.232717 2014-08-13 17:38:03.106615 7fe9bae887a0 20 read_log 192888'18153 (192888'18085) modify 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.1315259.0:59874 2014-08-11 16:47:04.267420 2014-08-13 17:38:03.106627 7fe9bae887a0 20 read_log 192888'18154 (192888'18087) modify cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.1315259.0:59963 2014-08-11 16:47:04.297850 2014-08-13 17:38:03.106646 7fe9bae887a0 20 read_log 192888'18156 (192888'18091) modify 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.1315259.0:60734 2014-08-11 16:47:04.524504 2014-08-13 17:38:03.106659 7fe9bae887a0 20 read_log 192888'18162 (0'0) promote e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by osd.356.0:17868 2014-08-11 17:05:27.050421 2014-08-13 17:38:03.106671 7fe9bae887a0 20 read_log 192888'18163 (192888'18162) modify e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.942868.0:3002 2014-08-11 17:05:27.043401 2014-08-13 17:38:03.106684 7fe9bae887a0 20 read_log 192888'18164 (0'0) promote 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:17880 2014-08-11 17:05:31.133597 2014-08-13 17:38:03.106696 7fe9bae887a0 20 read_log 192888'18165 (192888'18164) modify 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by client.942868.0:4108 2014-08-11 17:05:31.126492 2014-08-13 17:38:03.106708 7fe9bae887a0 20 read_log 192888'18168 (0'0) promote ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by osd.356.0:17888 2014-08-11 17:05:34.547744 2014-08-13 17:38:03.106720 7fe9bae887a0 20 read_log 192888'18169 (192888'18168) modify ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.942868.0:4984 2014-08-11 17:05:34.539875 2014-08-13 17:38:03.106732 7fe9bae887a0 20 read_log 192888'18174 (0'0) promote 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by osd.356.0:17896 2014-08-11 17:05:39.743511 2014-08-13 17:38:03.106744 7fe9bae887a0 20 read_log 192888'18175 (192888'18174) modify 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.942868.0:6395 2014-08-11 17:05:39.736743 2014-08-13 17:38:03.106756 7fe9bae887a0 20 read_log 192888'18176 (0'0) promote e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:17898 2014-08-11 17:05:40.059169 2014-08-13 17:38:03.106768 7fe9bae887a0 20 read_log 192888'18177 (192888'18176) modify e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by client.942868.0:6483 2014-08-11 17:05:40.051694 2014-08-13 17:38:03.106783 7fe9bae887a0 20 read_log 192888'18178 (0'0) promote 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by osd.356.0:17900 2014-08-11 17:05:40.401566 2014-08-13 17:38:03.106795 7fe9bae887a0 20 read_log 192888'18179 (192888'18178) modify 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.942868.0:6578 2014-08-11 17:05:40.393251 2014-08-13 17:38:03.106807 7fe9bae887a0 20 read_log 192888'18180 (0'0) promote b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by osd.356.0:17904 2014-08-11 17:05:42.777995 2014-08-13 17:38:03.106819 7fe9bae887a0 20 read_log 192888'18181 (192888'18180) modify b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.942868.0:7202 2014-08-11 17:05:42.769006 2014-08-13 17:38:03.106832 7fe9bae887a0 20 read_log 192888'18182 (0'0) promote 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by osd.356.0:17906 2014-08-11 17:05:44.974864 2014-08-13 17:38:03.106844 7fe9bae887a0 20 read_log 192888'18183 (192888'18182) modify 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.942868.0:7764 2014-08-11 17:05:44.966952 2014-08-13 17:38:03.106856 7fe9bae887a0 20 read_log 192888'18186 (0'0) promote 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by osd.356.0:17910 2014-08-11 17:05:46.111310 2014-08-13 17:38:03.106868 7fe9bae887a0 20 read_log 192888'18187 (192888'18186) modify 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.942868.0:8079 2014-08-11 17:05:46.104011 2014-08-13 17:38:03.106880 7fe9bae887a0 20 read_log 192888'18188 (0'0) promote a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by osd.356.0:17914 2014-08-11 17:05:48.718843 2014-08-13 17:38:03.106899 7fe9bae887a0 20 read_log 192888'18189 (192888'18188) modify a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.942868.0:8777 2014-08-11 17:05:48.708387 2014-08-13 17:38:03.106912 7fe9bae887a0 20 read_log 192888'18194 (0'0) promote d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by osd.356.0:17922 2014-08-11 17:05:51.465210 2014-08-13 17:38:03.106924 7fe9bae887a0 20 read_log 192888'18195 (192888'18194) modify d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.942868.0:9519 2014-08-11 17:05:51.457463 2014-08-13 17:38:03.106936 7fe9bae887a0 20 read_log 192888'18200 (0'0) promote 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:17932 2014-08-11 17:06:00.398521 2014-08-13 17:38:03.106949 7fe9bae887a0 20 read_log 192888'18201 (192888'18200) modify 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by client.942868.0:11969 2014-08-11 17:06:00.391456 2014-08-13 17:38:03.106961 7fe9bae887a0 20 read_log 192888'18208 (0'0) promote 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by osd.356.0:17944 2014-08-11 17:06:07.019455 2014-08-13 17:38:03.106973 7fe9bae887a0 20 read_log 192888'18209 (192888'18208) modify 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.942868.0:13713 2014-08-11 17:06:07.011627 2014-08-13 17:38:03.106985 7fe9bae887a0 20 read_log 192888'18212 (0'0) promote b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:17948 2014-08-11 17:06:08.408545 2014-08-13 17:38:03.106998 7fe9bae887a0 20 read_log 192888'18213 (192888'18212) modify b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by client.942868.0:14043 2014-08-11 17:06:08.401363 2014-08-13 17:38:03.107010 7fe9bae887a0 20 read_log 192888'18214 (0'0) promote ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by osd.356.0:17952 2014-08-11 17:06:14.143721 2014-08-13 17:38:03.107022 7fe9bae887a0 20 read_log 192888'18215 (192888'18214) modify ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.942868.0:15492 2014-08-11 17:06:14.135201 2014-08-13 17:38:03.107034 7fe9bae887a0 20 read_log 192888'18216 (0'0) promote c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by osd.356.0:17956 2014-08-11 17:06:14.717093 2014-08-13 17:38:03.107046 7fe9bae887a0 20 read_log 192888'18217 (192888'18216) modify c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.942868.0:15641 2014-08-11 17:06:14.709466 2014-08-13 17:38:03.107058 7fe9bae887a0 20 read_log 192888'18220 (0'0) promote a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:17964 2014-08-11 17:06:16.375005 2014-08-13 17:38:03.107070 7fe9bae887a0 20 read_log 192888'18221 (192888'18220) modify a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by client.942868.0:16069 2014-08-11 17:06:16.368059 2014-08-13 17:38:03.107102 7fe9bae887a0 20 read_log 192888'18226 (0'0) promote e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by osd.356.0:17976 2014-08-11 17:06:19.734637 2014-08-13 17:38:03.107115 7fe9bae887a0 20 read_log 192888'18227 (192888'18226) modify e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.942868.0:16913 2014-08-11 17:06:19.727190 2014-08-13 17:38:03.107127 7fe9bae887a0 20 read_log 192888'18228 (0'0) promote eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by osd.356.0:17980 2014-08-11 17:06:22.635337 2014-08-13 17:38:03.107139 7fe9bae887a0 20 read_log 192888'18229 (192888'18228) modify eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.942868.0:17681 2014-08-11 17:06:22.628455 2014-08-13 17:38:03.107152 7fe9bae887a0 20 read_log 192888'18230 (0'0) promote bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by osd.356.0:17984 2014-08-11 17:06:23.858678 2014-08-13 17:38:03.107178 7fe9bae887a0 20 read_log 192888'18231 (192888'18230) modify bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.942868.0:18007 2014-08-11 17:06:23.850755 2014-08-13 17:38:03.107192 7fe9bae887a0 20 read_log 192888'18232 (0'0) promote 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by osd.356.0:17986 2014-08-11 17:06:26.085348 2014-08-13 17:38:03.107204 7fe9bae887a0 20 read_log 192888'18233 (192888'18232) modify 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.942868.0:18597 2014-08-11 17:06:26.077379 2014-08-13 17:38:03.107216 7fe9bae887a0 20 read_log 192888'18238 (0'0) promote 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:17992 2014-08-11 17:06:28.915000 2014-08-13 17:38:03.107230 7fe9bae887a0 20 read_log 192888'18239 (192888'18238) modify 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by client.942868.0:19350 2014-08-11 17:06:28.907553 2014-08-13 17:38:03.107242 7fe9bae887a0 20 read_log 192888'18240 (0'0) promote ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:17994 2014-08-11 17:06:29.278181 2014-08-13 17:38:03.107254 7fe9bae887a0 20 read_log 192888'18241 (192888'18240) modify ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by client.942868.0:19448 2014-08-11 17:06:29.271423 2014-08-13 17:38:03.107266 7fe9bae887a0 20 read_log 192888'18252 (0'0) promote 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by osd.356.0:18020 2014-08-11 17:06:44.437087 2014-08-13 17:38:03.107279 7fe9bae887a0 20 read_log 192888'18253 (192888'18252) modify 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.942868.0:23382 2014-08-11 17:06:44.430200 2014-08-13 17:38:03.107292 7fe9bae887a0 20 read_log 192888'18256 (0'0) promote e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:18026 2014-08-11 17:06:45.447660 2014-08-13 17:38:03.107305 7fe9bae887a0 20 read_log 192888'18257 (192888'18256) modify e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by client.942868.0:23658 2014-08-11 17:06:45.440344 2014-08-13 17:38:03.107317 7fe9bae887a0 20 read_log 192888'18258 (0'0) promote b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:18028 2014-08-11 17:06:46.541032 2014-08-13 17:38:03.107329 7fe9bae887a0 20 read_log 192888'18259 (192888'18258) modify b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by client.942868.0:23954 2014-08-11 17:06:46.533672 2014-08-13 17:38:03.107341 7fe9bae887a0 20 read_log 192888'18262 (0'0) promote 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:18032 2014-08-11 17:06:48.962049 2014-08-13 17:38:03.107354 7fe9bae887a0 20 read_log 192888'18263 (192888'18262) modify 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by client.942868.0:24607 2014-08-11 17:06:48.954257 2014-08-13 17:38:03.107367 7fe9bae887a0 20 read_log 192888'18266 (0'0) promote e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:18036 2014-08-11 17:06:51.331039 2014-08-13 17:38:03.107379 7fe9bae887a0 20 read_log 192888'18267 (192888'18266) modify e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by client.942868.0:25247 2014-08-11 17:06:51.324057 2014-08-13 17:38:03.107391 7fe9bae887a0 20 read_log 192888'18268 (0'0) promote c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:18038 2014-08-11 17:06:53.243571 2014-08-13 17:38:03.107403 7fe9bae887a0 20 read_log 192888'18269 (192888'18268) modify c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by client.942868.0:25726 2014-08-11 17:06:53.236076 2014-08-13 17:38:03.107415 7fe9bae887a0 20 read_log 192888'18280 (0'0) promote 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by osd.356.0:18058 2014-08-11 17:06:57.866249 2014-08-13 17:38:03.107436 7fe9bae887a0 20 read_log 192888'18281 (192888'18280) modify 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.942868.0:26902 2014-08-11 17:06:57.859512 2014-08-13 17:38:03.107450 7fe9bae887a0 20 read_log 192888'18282 (0'0) promote c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by osd.356.0:18064 2014-08-11 17:07:02.018786 2014-08-13 17:38:03.107462 7fe9bae887a0 20 read_log 192888'18283 (192888'18282) modify c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.942868.0:28006 2014-08-11 17:07:02.011695 2014-08-13 17:38:03.107474 7fe9bae887a0 20 read_log 192888'18290 (0'0) promote ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by osd.356.0:18084 2014-08-11 17:07:11.683933 2014-08-13 17:38:03.107486 7fe9bae887a0 20 read_log 192888'18291 (192888'18290) modify ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.942868.0:30464 2014-08-11 17:07:11.676867 2014-08-13 17:38:03.107499 7fe9bae887a0 20 read_log 192888'18295 (192888'18163) modify e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.942868.0:34268 2014-08-11 17:07:16.742708 2014-08-13 17:38:03.107511 7fe9bae887a0 20 read_log 192888'18296 (192888'18165) modify 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by client.942868.0:35374 2014-08-11 17:07:17.058108 2014-08-13 17:38:03.107523 7fe9bae887a0 20 read_log 192888'18298 (192888'18169) modify ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.942868.0:36250 2014-08-11 17:07:17.280170 2014-08-13 17:38:03.107558 7fe9bae887a0 20 read_log 192888'18301 (192888'18175) modify 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.942868.0:37661 2014-08-11 17:07:17.672214 2014-08-13 17:38:03.107575 7fe9bae887a0 20 read_log 192888'18302 (192888'18177) modify e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by client.942868.0:37749 2014-08-11 17:07:17.696350 2014-08-13 17:38:03.107588 7fe9bae887a0 20 read_log 192888'18303 (192888'18179) modify 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.942868.0:37844 2014-08-11 17:07:17.722356 2014-08-13 17:38:03.107600 7fe9bae887a0 20 read_log 192888'18304 (192888'18181) modify b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.942868.0:38468 2014-08-11 17:07:17.895439 2014-08-13 17:38:03.107612 7fe9bae887a0 20 read_log 192888'18305 (192888'18183) modify 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.942868.0:39030 2014-08-11 17:07:18.061065 2014-08-13 17:38:03.107624 7fe9bae887a0 20 read_log 192888'18307 (192888'18187) modify 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.942868.0:39345 2014-08-11 17:07:18.155657 2014-08-13 17:38:03.107636 7fe9bae887a0 20 read_log 192888'18308 (192888'18189) modify a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.942868.0:40043 2014-08-11 17:07:18.350524 2014-08-13 17:38:03.107649 7fe9bae887a0 20 read_log 192888'18311 (192888'18195) modify d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.942868.0:40785 2014-08-11 17:07:18.547543 2014-08-13 17:38:03.107680 7fe9bae887a0 20 read_log 192888'18314 (192888'18201) modify 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by client.942868.0:43235 2014-08-11 17:07:19.215432 2014-08-13 17:38:03.107696 7fe9bae887a0 20 read_log 192888'18318 (192888'18209) modify 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.942868.0:44979 2014-08-11 17:07:19.714582 2014-08-13 17:38:03.107708 7fe9bae887a0 20 read_log 192888'18320 (192888'18213) modify b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by client.942868.0:45309 2014-08-11 17:07:19.802505 2014-08-13 17:38:03.107720 7fe9bae887a0 20 read_log 192888'18321 (192888'18215) modify ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.942868.0:46758 2014-08-11 17:07:20.211654 2014-08-13 17:38:03.107755 7fe9bae887a0 20 read_log 192888'18322 (192888'18217) modify c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.942868.0:46907 2014-08-11 17:07:20.251961 2014-08-13 17:38:03.107771 7fe9bae887a0 20 read_log 192888'18324 (192888'18221) modify a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by client.942868.0:47335 2014-08-11 17:07:20.375032 2014-08-13 17:38:03.107784 7fe9bae887a0 20 read_log 192888'18327 (192888'18227) modify e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.942868.0:48179 2014-08-11 17:07:20.648985 2014-08-13 17:38:03.107796 7fe9bae887a0 20 read_log 192888'18328 (192888'18229) modify eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.942868.0:48947 2014-08-11 17:07:20.864714 2014-08-13 17:38:03.107810 7fe9bae887a0 20 read_log 192888'18329 (192888'18231) modify bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.942868.0:49273 2014-08-11 17:07:20.962030 2014-08-13 17:38:03.107823 7fe9bae887a0 20 read_log 192888'18330 (192888'18233) modify 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.942868.0:49863 2014-08-11 17:07:21.147035 2014-08-13 17:38:03.107835 7fe9bae887a0 20 read_log 192888'18333 (192888'18239) modify 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by client.942868.0:50616 2014-08-11 17:07:21.377797 2014-08-13 17:38:03.107847 7fe9bae887a0 20 read_log 192888'18334 (192888'18241) modify ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by client.942868.0:50714 2014-08-11 17:07:21.408598 2014-08-13 17:38:03.107859 7fe9bae887a0 20 read_log 192888'18340 (192888'18253) modify 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.942868.0:54648 2014-08-11 17:07:22.617915 2014-08-13 17:38:03.107872 7fe9bae887a0 20 read_log 192888'18342 (192888'18257) modify e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by client.942868.0:54924 2014-08-11 17:07:22.695823 2014-08-13 17:38:03.107884 7fe9bae887a0 20 read_log 192888'18343 (192888'18259) modify b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by client.942868.0:55220 2014-08-11 17:07:22.777171 2014-08-13 17:38:03.107896 7fe9bae887a0 20 read_log 192888'18345 (192888'18263) modify 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by client.942868.0:55873 2014-08-11 17:07:22.982825 2014-08-13 17:38:03.107908 7fe9bae887a0 20 read_log 192888'18347 (192888'18267) modify e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by client.942868.0:56513 2014-08-11 17:07:23.174997 2014-08-13 17:38:03.107920 7fe9bae887a0 20 read_log 192888'18348 (192888'18269) modify c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by client.942868.0:56992 2014-08-11 17:07:23.324201 2014-08-13 17:38:03.107932 7fe9bae887a0 20 read_log 192888'18354 (192888'18281) modify 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.942868.0:58168 2014-08-11 17:07:23.662669 2014-08-13 17:38:03.107943 7fe9bae887a0 20 read_log 192888'18355 (192888'18283) modify c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.942868.0:59272 2014-08-11 17:07:23.996372 2014-08-13 17:38:03.107955 7fe9bae887a0 20 read_log 192888'18359 (192888'18291) modify ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.942868.0:61730 2014-08-11 17:07:24.729264 2014-08-13 17:38:03.107968 7fe9bae887a0 20 read_log 193305'18361 (193305'18361) modify 3b7/hit_set_24.3b7_archive_2014-08-11 18:37:35.561357_2014-08-11 18:42:37.204550/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.204792 2014-08-13 17:38:03.107980 7fe9bae887a0 20 read_log 193305'18362 (0'0) promote 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:11 2014-08-11 18:42:37.208362 2014-08-13 17:38:03.108002 7fe9bae887a0 20 read_log 193305'18363 (193305'18362) modify 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by client.951337.0:1065 2014-08-11 18:42:37.199835 2014-08-13 17:38:03.108015 7fe9bae887a0 20 read_log 193305'18364 (0'0) promote 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by osd.356.0:16 2014-08-11 18:42:38.680835 2014-08-13 17:38:03.108027 7fe9bae887a0 20 read_log 193305'18365 (193305'18364) modify 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.951337.0:1447 2014-08-11 18:42:38.672964 2014-08-13 17:38:03.108039 7fe9bae887a0 20 read_log 193305'18366 (0'0) promote 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by osd.356.0:23 2014-08-11 18:42:42.178816 2014-08-13 17:38:03.108051 7fe9bae887a0 20 read_log 193305'18367 (193305'18366) modify 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.951337.0:2391 2014-08-11 18:42:42.171349 2014-08-13 17:38:03.108063 7fe9bae887a0 20 read_log 193305'18368 (0'0) promote d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by osd.356.0:27 2014-08-11 18:42:45.126919 2014-08-13 17:38:03.108075 7fe9bae887a0 20 read_log 193305'18369 (193305'18368) modify d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.951337.0:3200 2014-08-11 18:42:45.119639 2014-08-13 17:38:03.108087 7fe9bae887a0 20 read_log 193305'18370 (0'0) promote f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by osd.356.0:29 2014-08-11 18:42:45.388065 2014-08-13 17:38:03.108099 7fe9bae887a0 20 read_log 193305'18371 (193305'18370) modify f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.951337.0:3273 2014-08-11 18:42:45.381633 2014-08-13 17:38:03.108113 7fe9bae887a0 20 read_log 193305'18372 (0'0) promote 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by osd.356.0:57 2014-08-11 18:42:58.743410 2014-08-13 17:38:03.108125 7fe9bae887a0 20 read_log 193305'18373 (193305'18372) modify 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.951337.0:6916 2014-08-11 18:42:58.734468 2014-08-13 17:38:03.108139 7fe9bae887a0 20 read_log 193305'18374 (0'0) promote 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:59 2014-08-11 18:42:59.748810 2014-08-13 17:38:03.108151 7fe9bae887a0 20 read_log 193305'18375 (193305'18374) modify 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by client.951337.0:7195 2014-08-11 18:42:59.741637 2014-08-13 17:38:03.108171 7fe9bae887a0 20 read_log 193305'18376 (0'0) promote 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:61 2014-08-11 18:43:00.690202 2014-08-13 17:38:03.108184 7fe9bae887a0 20 read_log 193305'18377 (193305'18376) modify 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by client.951337.0:7456 2014-08-11 18:43:00.682863 2014-08-13 17:38:03.108198 7fe9bae887a0 20 read_log 193305'18378 (0'0) promote 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by osd.356.0:67 2014-08-11 18:43:05.656056 2014-08-13 17:38:03.108210 7fe9bae887a0 20 read_log 193305'18379 (193305'18378) modify 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.951337.0:8747 2014-08-11 18:43:05.646812 2014-08-13 17:38:03.108223 7fe9bae887a0 20 read_log 193305'18380 (0'0) promote a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by osd.356.0:77 2014-08-11 18:43:08.134102 2014-08-13 17:38:03.108237 7fe9bae887a0 20 read_log 193305'18381 (193305'18380) modify a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.951337.0:9410 2014-08-11 18:43:08.126059 2014-08-13 17:38:03.108250 7fe9bae887a0 20 read_log 193305'18382 (0'0) promote f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:97 2014-08-11 18:43:16.158212 2014-08-13 17:38:03.108269 7fe9bae887a0 20 read_log 193305'18383 (193305'18382) modify f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by client.951337.0:11530 2014-08-11 18:43:16.151070 2014-08-13 17:38:03.108284 7fe9bae887a0 20 read_log 193305'18384 (0'0) promote 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by osd.356.0:101 2014-08-11 18:43:17.031646 2014-08-13 17:38:03.108296 7fe9bae887a0 20 read_log 193305'18385 (193305'18384) modify 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.951337.0:11763 2014-08-11 18:43:17.024298 2014-08-13 17:38:03.108309 7fe9bae887a0 20 read_log 193305'18386 (0'0) promote 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:127 2014-08-11 18:43:26.578134 2014-08-13 17:38:03.108322 7fe9bae887a0 20 read_log 193305'18387 (193305'18386) modify 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by client.951337.0:14304 2014-08-11 18:43:26.569915 2014-08-13 17:38:03.108334 7fe9bae887a0 20 read_log 193305'18388 (0'0) promote 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by osd.356.0:131 2014-08-11 18:43:26.985726 2014-08-13 17:38:03.108346 7fe9bae887a0 20 read_log 193305'18389 (193305'18388) modify 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.951337.0:14428 2014-08-11 18:43:26.977152 2014-08-13 17:38:03.108358 7fe9bae887a0 20 read_log 193305'18390 (0'0) promote 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:133 2014-08-11 18:43:27.240278 2014-08-13 17:38:03.108370 7fe9bae887a0 20 read_log 193305'18391 (193305'18390) modify 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by client.951337.0:14505 2014-08-11 18:43:27.232734 2014-08-13 17:38:03.108382 7fe9bae887a0 20 read_log 193305'18392 (0'0) promote eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by osd.356.0:151 2014-08-11 18:43:34.971042 2014-08-13 17:38:03.108394 7fe9bae887a0 20 read_log 193305'18393 (193305'18392) modify eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.951337.0:16734 2014-08-11 18:43:34.963481 2014-08-13 17:38:03.108406 7fe9bae887a0 20 read_log 193305'18394 (0'0) promote 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by osd.356.0:155 2014-08-11 18:43:36.091909 2014-08-13 17:38:03.108418 7fe9bae887a0 20 read_log 193305'18395 (193305'18394) modify 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.951337.0:17039 2014-08-11 18:43:36.082545 2014-08-13 17:38:03.108430 7fe9bae887a0 20 read_log 193305'18396 (0'0) promote 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:163 2014-08-11 18:43:38.270245 2014-08-13 17:38:03.108442 7fe9bae887a0 20 read_log 193305'18397 (193305'18396) modify 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by client.951337.0:17694 2014-08-11 18:43:38.261756 2014-08-13 17:38:03.108454 7fe9bae887a0 20 read_log 193305'18398 (0'0) promote a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by osd.356.0:165 2014-08-11 18:43:38.790420 2014-08-13 17:38:03.108467 7fe9bae887a0 20 read_log 193305'18399 (193305'18398) modify a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.951337.0:17849 2014-08-11 18:43:38.782952 2014-08-13 17:38:03.108479 7fe9bae887a0 20 read_log 193305'18400 (0'0) promote a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by osd.356.0:169 2014-08-11 18:43:39.496343 2014-08-13 17:38:03.108491 7fe9bae887a0 20 read_log 193305'18401 (193305'18400) modify a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.951337.0:18035 2014-08-11 18:43:39.487984 2014-08-13 17:38:03.108503 7fe9bae887a0 20 read_log 193305'18402 (0'0) promote 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:179 2014-08-11 18:43:46.368230 2014-08-13 17:38:03.108521 7fe9bae887a0 20 read_log 193305'18403 (193305'18402) modify 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by client.951337.0:19935 2014-08-11 18:43:46.361211 2014-08-13 17:38:03.108534 7fe9bae887a0 20 read_log 193305'18404 (0'0) promote e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by osd.356.0:183 2014-08-11 18:43:48.768594 2014-08-13 17:38:03.108546 7fe9bae887a0 20 read_log 193305'18405 (193305'18404) modify e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.951337.0:20593 2014-08-11 18:43:48.761186 2014-08-13 17:38:03.108558 7fe9bae887a0 20 read_log 193305'18406 (0'0) promote e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by osd.356.0:205 2014-08-11 18:43:57.557515 2014-08-13 17:38:03.108570 7fe9bae887a0 20 read_log 193305'18407 (193305'18406) modify e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.951337.0:23034 2014-08-11 18:43:57.550580 2014-08-13 17:38:03.108582 7fe9bae887a0 20 read_log 193305'18408 (0'0) promote 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:211 2014-08-11 18:43:58.913423 2014-08-13 17:38:03.108594 7fe9bae887a0 20 read_log 193305'18409 (193305'18408) modify 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by client.951337.0:23421 2014-08-11 18:43:58.906515 2014-08-13 17:38:03.108606 7fe9bae887a0 20 read_log 193305'18410 (0'0) promote ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by osd.356.0:215 2014-08-11 18:44:04.046591 2014-08-13 17:38:03.108618 7fe9bae887a0 20 read_log 193305'18411 (193305'18410) modify ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.951337.0:24842 2014-08-11 18:44:04.038708 2014-08-13 17:38:03.108630 7fe9bae887a0 20 read_log 193305'18412 (0'0) promote 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by osd.356.0:221 2014-08-11 18:44:06.368856 2014-08-13 17:38:03.108642 7fe9bae887a0 20 read_log 193305'18413 (193305'18412) modify 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.951337.0:25471 2014-08-11 18:44:06.360640 2014-08-13 17:38:03.108654 7fe9bae887a0 20 read_log 193305'18414 (0'0) promote 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by osd.356.0:225 2014-08-11 18:44:10.944441 2014-08-13 17:38:03.108666 7fe9bae887a0 20 read_log 193305'18415 (193305'18414) modify 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.951337.0:26710 2014-08-11 18:44:10.936117 2014-08-13 17:38:03.108678 7fe9bae887a0 20 read_log 193305'18416 (0'0) promote 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by osd.356.0:245 2014-08-11 18:44:18.716670 2014-08-13 17:38:03.108690 7fe9bae887a0 20 read_log 193305'18417 (193305'18416) modify 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.951337.0:28825 2014-08-11 18:44:18.709267 2014-08-13 17:38:03.108702 7fe9bae887a0 20 read_log 193305'18418 (0'0) promote 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:257 2014-08-11 18:44:25.943866 2014-08-13 17:38:03.108714 7fe9bae887a0 20 read_log 193305'18419 (193305'18418) modify 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by client.951337.0:30753 2014-08-11 18:44:25.935534 2014-08-13 17:38:03.108726 7fe9bae887a0 20 read_log 193305'18420 (0'0) promote 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by osd.356.0:259 2014-08-11 18:44:26.934616 2014-08-13 17:38:03.108738 7fe9bae887a0 20 read_log 193305'18421 (193305'18420) modify 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.951337.0:31032 2014-08-11 18:44:26.927766 2014-08-13 17:38:03.108750 7fe9bae887a0 20 read_log 193305'18422 (0'0) promote 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by osd.356.0:261 2014-08-11 18:44:27.422222 2014-08-13 17:38:03.108774 7fe9bae887a0 20 read_log 193305'18423 (193305'18422) modify 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.951337.0:31171 2014-08-11 18:44:27.414085 2014-08-13 17:38:03.108787 7fe9bae887a0 20 read_log 193305'18424 (0'0) promote 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by osd.356.0:271 2014-08-11 18:44:32.093220 2014-08-13 17:38:03.108799 7fe9bae887a0 20 read_log 193305'18425 (193305'18424) modify 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.951337.0:32503 2014-08-11 18:44:32.086306 2014-08-13 17:38:03.108811 7fe9bae887a0 20 read_log 193305'18426 (193305'18363) modify 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by client.951337.0:33776 2014-08-11 18:44:34.640597 2014-08-13 17:38:03.108824 7fe9bae887a0 20 read_log 193305'18427 (193305'18365) modify 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.951337.0:34158 2014-08-11 18:44:34.826402 2014-08-13 17:38:03.108836 7fe9bae887a0 20 read_log 193305'18428 (193305'18367) modify 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.951337.0:35102 2014-08-11 18:44:35.387709 2014-08-13 17:38:03.108848 7fe9bae887a0 20 read_log 193305'18429 (193305'18369) modify d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.951337.0:35911 2014-08-11 18:44:35.809106 2014-08-13 17:38:03.108860 7fe9bae887a0 20 read_log 193305'18430 (193305'18371) modify f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.951337.0:35984 2014-08-11 18:44:35.850584 2014-08-13 17:38:03.108872 7fe9bae887a0 20 read_log 193305'18431 (193305'18373) modify 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.951337.0:39627 2014-08-11 18:44:38.117119 2014-08-13 17:38:03.108884 7fe9bae887a0 20 read_log 193305'18432 (193305'18375) modify 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by client.951337.0:39906 2014-08-11 18:44:38.223650 2014-08-13 17:38:03.108912 7fe9bae887a0 20 read_log 193305'18433 (193305'18377) modify 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by client.951337.0:40167 2014-08-11 18:44:38.317545 2014-08-13 17:38:03.108925 7fe9bae887a0 20 read_log 193305'18434 (193305'18379) modify 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.951337.0:41458 2014-08-11 18:44:38.758743 2014-08-13 17:38:03.108937 7fe9bae887a0 20 read_log 193305'18435 (193305'18381) modify a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.951337.0:42121 2014-08-11 18:44:39.114149 2014-08-13 17:38:03.108949 7fe9bae887a0 20 read_log 193305'18436 (193305'18383) modify f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by client.951337.0:44241 2014-08-11 18:44:39.830075 2014-08-13 17:38:03.108961 7fe9bae887a0 20 read_log 193305'18437 (193305'18385) modify 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.951337.0:44474 2014-08-11 18:44:39.890399 2014-08-13 17:38:03.108973 7fe9bae887a0 20 read_log 193305'18438 (193305'18387) modify 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by client.951337.0:47015 2014-08-11 18:44:40.651442 2014-08-13 17:38:03.108985 7fe9bae887a0 20 read_log 193305'18439 (193305'18389) modify 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.951337.0:47139 2014-08-11 18:44:40.683615 2014-08-13 17:38:03.108997 7fe9bae887a0 20 read_log 193305'18440 (193305'18391) modify 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by client.951337.0:47216 2014-08-11 18:44:40.704675 2014-08-13 17:38:03.109009 7fe9bae887a0 20 read_log 193305'18441 (193305'18393) modify eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.951337.0:49445 2014-08-11 18:44:41.367277 2014-08-13 17:38:03.109021 7fe9bae887a0 20 read_log 193305'18442 (193305'18395) modify 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.951337.0:49750 2014-08-11 18:44:41.461083 2014-08-13 17:38:03.109040 7fe9bae887a0 20 read_log 193305'18443 (193305'18397) modify 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by client.951337.0:50405 2014-08-11 18:44:41.655479 2014-08-13 17:38:03.109053 7fe9bae887a0 20 read_log 193305'18444 (193305'18399) modify a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.951337.0:50560 2014-08-11 18:44:41.698389 2014-08-13 17:38:03.109065 7fe9bae887a0 20 read_log 193305'18445 (193305'18401) modify a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.951337.0:50746 2014-08-11 18:44:41.750475 2014-08-13 17:38:03.109077 7fe9bae887a0 20 read_log 193305'18446 (193305'18403) modify 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by client.951337.0:52646 2014-08-11 18:44:42.293977 2014-08-13 17:38:03.109089 7fe9bae887a0 20 read_log 193305'18447 (193305'18405) modify e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.951337.0:53304 2014-08-11 18:44:42.488173 2014-08-13 17:38:03.109101 7fe9bae887a0 20 read_log 193305'18448 (193305'18407) modify e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.951337.0:55745 2014-08-11 18:44:43.219896 2014-08-13 17:38:03.109113 7fe9bae887a0 20 read_log 193305'18449 (193305'18409) modify 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by client.951337.0:56132 2014-08-11 18:44:43.330009 2014-08-13 17:38:03.109125 7fe9bae887a0 20 read_log 193305'18450 (193305'18411) modify ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.951337.0:57553 2014-08-11 18:44:43.792525 2014-08-13 17:38:03.109137 7fe9bae887a0 20 read_log 193305'18451 (193305'18413) modify 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.951337.0:58182 2014-08-11 18:44:43.969106 2014-08-13 17:38:03.109149 7fe9bae887a0 20 read_log 193305'18452 (193305'18415) modify 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.951337.0:59421 2014-08-11 18:44:44.321946 2014-08-13 17:38:03.109168 7fe9bae887a0 20 read_log 193305'18453 (193305'18417) modify 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.951337.0:61536 2014-08-11 18:44:44.950318 2014-08-13 17:38:03.109181 7fe9bae887a0 20 read_log 193305'18454 (193305'18419) modify 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by client.951337.0:63464 2014-08-11 18:44:45.515408 2014-08-13 17:38:03.109193 7fe9bae887a0 20 read_log 193305'18455 (193305'18421) modify 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.951337.0:63743 2014-08-11 18:44:45.596876 2014-08-13 17:38:03.109206 7fe9bae887a0 20 read_log 193305'18456 (193305'18423) modify 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.951337.0:63882 2014-08-11 18:44:45.636167 2014-08-13 17:38:03.109219 7fe9bae887a0 20 read_log 193305'18457 (193305'18425) modify 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.951337.0:65214 2014-08-11 18:44:46.052662 2014-08-13 17:38:03.109231 7fe9bae887a0 20 read_log 194209'18458 (192888'18324) clean a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:583 2014-08-12 14:07:41.714253 2014-08-13 17:38:03.109243 7fe9bae887a0 20 read_log 194209'18459 (192834'17424) clean abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.356.0:584 2014-08-12 14:07:41.714670 2014-08-13 17:38:03.109256 7fe9bae887a0 20 read_log 194209'18460 (192888'18314) clean 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:585 2014-08-12 14:07:41.715336 2014-08-13 17:38:03.109268 7fe9bae887a0 20 read_log 194209'18461 (192888'18124) clean f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:586 2014-08-12 14:07:41.715873 2014-08-13 17:38:03.109289 7fe9bae887a0 20 read_log 194209'18462 (193305'18436) clean f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:587 2014-08-12 14:07:41.724594 2014-08-13 17:38:03.109302 7fe9bae887a0 20 read_log 194209'18463 (192888'18320) clean b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:588 2014-08-12 14:07:41.727547 2014-08-13 17:38:03.109314 7fe9bae887a0 20 read_log 194209'18464 (192888'18342) clean e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:589 2014-08-12 14:07:41.733314 2014-08-13 17:38:03.109326 7fe9bae887a0 20 read_log 194209'18465 (193305'18454) clean 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:590 2014-08-12 14:07:41.733574 2014-08-13 17:38:03.109339 7fe9bae887a0 20 read_log 194209'18466 (193305'18446) clean 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:591 2014-08-12 14:07:41.736625 2014-08-13 17:38:03.109351 7fe9bae887a0 20 read_log 194209'18467 (193305'18443) clean 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:592 2014-08-12 14:07:41.739768 2014-08-13 17:38:03.109363 7fe9bae887a0 20 read_log 194209'18468 (192888'18132) clean c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:593 2014-08-12 14:07:41.741207 2014-08-13 17:38:03.109376 7fe9bae887a0 20 read_log 194209'18469 (192834'17448) clean 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.356.0:594 2014-08-12 14:07:41.747904 2014-08-13 17:38:03.109389 7fe9bae887a0 20 read_log 194209'18470 (193305'18440) clean 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:595 2014-08-12 14:07:41.750818 2014-08-13 17:38:03.109401 7fe9bae887a0 20 read_log 194209'18471 (192888'18334) clean ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:596 2014-08-12 14:07:41.751071 2014-08-13 17:38:03.109413 7fe9bae887a0 20 read_log 194209'18472 (192834'17330) clean f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.356.0:597 2014-08-12 14:07:41.754198 2014-08-13 17:38:03.109425 7fe9bae887a0 20 read_log 194209'18473 (192888'18139) clean 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:598 2014-08-12 14:07:41.757353 2014-08-13 17:38:03.109437 7fe9bae887a0 20 read_log 194209'18474 (193305'18432) clean 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:599 2014-08-12 14:07:41.760462 2014-08-13 17:38:03.109449 7fe9bae887a0 20 read_log 194209'18475 (192888'18107) clean 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:600 2014-08-12 14:07:41.763596 2014-08-13 17:38:03.109465 7fe9bae887a0 20 read_log 194209'18476 (192888'18296) clean 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:601 2014-08-12 14:07:41.766743 2014-08-13 17:38:03.109478 7fe9bae887a0 20 read_log 194209'18477 (192888'18098) clean 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:602 2014-08-12 14:07:41.769846 2014-08-13 17:38:03.109490 7fe9bae887a0 20 read_log 194209'18478 (193305'18433) clean 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:603 2014-08-12 14:07:41.775752 2014-08-13 17:38:03.109502 7fe9bae887a0 20 read_log 194209'18479 (192834'17847) clean adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.356.0:604 2014-08-12 14:07:41.776037 2014-08-13 17:38:03.109514 7fe9bae887a0 20 read_log 194209'18480 (192888'18347) clean e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:605 2014-08-12 14:07:41.778932 2014-08-13 17:38:03.109527 7fe9bae887a0 20 read_log 194209'18481 (193305'18426) clean 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:606 2014-08-12 14:07:41.782081 2014-08-13 17:38:03.109565 7fe9bae887a0 20 read_log 194209'18482 (192834'17434) clean 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.356.0:607 2014-08-12 14:07:41.785240 2014-08-13 17:38:03.109582 7fe9bae887a0 20 read_log 194209'18483 (192888'18302) clean e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:608 2014-08-12 14:07:41.788208 2014-08-13 17:38:03.109594 7fe9bae887a0 20 read_log 194209'18484 (192888'18343) clean b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:609 2014-08-12 14:07:41.791427 2014-08-13 17:38:03.109606 7fe9bae887a0 20 read_log 194209'18485 (193305'18449) clean 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:610 2014-08-12 14:07:41.794403 2014-08-13 17:38:03.109619 7fe9bae887a0 20 read_log 194209'18486 (192888'18348) clean c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:611 2014-08-12 14:07:41.797556 2014-08-13 17:38:03.109631 7fe9bae887a0 20 read_log 194209'18487 (192888'18110) clean 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:612 2014-08-12 14:07:41.800829 2014-08-13 17:38:03.109643 7fe9bae887a0 20 read_log 194209'18488 (192834'17944) clean e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.356.0:613 2014-08-12 14:07:41.803885 2014-08-13 17:38:03.109655 7fe9bae887a0 20 read_log 194209'18489 (192888'18136) clean 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:614 2014-08-12 14:07:41.807036 2014-08-13 17:38:03.109667 7fe9bae887a0 20 read_log 194209'18490 (192888'18333) clean 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:615 2014-08-12 14:07:41.809919 2014-08-13 17:38:03.109679 7fe9bae887a0 20 read_log 194209'18491 (192834'17396) clean 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.356.0:616 2014-08-12 14:07:41.813425 2014-08-13 17:38:03.109691 7fe9bae887a0 20 read_log 194209'18492 (192888'18345) clean 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:617 2014-08-12 14:07:41.817160 2014-08-13 17:38:03.109704 7fe9bae887a0 20 read_log 194209'18493 (192888'18137) clean 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:618 2014-08-12 14:07:41.820235 2014-08-13 17:38:03.109716 7fe9bae887a0 20 read_log 194209'18494 (192834'17338) clean 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.356.0:619 2014-08-12 14:07:41.823315 2014-08-13 17:38:03.109728 7fe9bae887a0 20 read_log 194209'18495 (192834'17932) clean f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.356.0:620 2014-08-12 14:07:41.826370 2014-08-13 17:38:03.109740 7fe9bae887a0 20 read_log 194209'18496 (192888'18114) clean 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:621 2014-08-12 14:07:41.829675 2014-08-13 17:38:03.109752 7fe9bae887a0 20 read_log 194209'18497 (193305'18438) clean 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:622 2014-08-12 14:07:41.832740 2014-08-13 17:38:03.109764 7fe9bae887a0 20 read_log 194209'18498 (192834'17446) clean 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.356.0:623 2014-08-12 14:07:41.855840 2014-08-13 17:38:03.109776 7fe9bae887a0 20 read_log 194209'18499 (194209'18458) delete a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:624 2014-08-12 14:07:41.867000 2014-08-13 17:38:03.109788 7fe9bae887a0 20 read_log 194209'18500 (194209'18459) delete abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.356.0:625 2014-08-12 14:07:41.867382 2014-08-13 17:38:03.109800 7fe9bae887a0 20 read_log 194209'18501 (194209'18460) delete 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:626 2014-08-12 14:07:41.870105 2014-08-13 17:38:03.109819 7fe9bae887a0 20 read_log 194209'18502 (194209'18461) delete f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:627 2014-08-12 14:07:41.870718 2014-08-13 17:38:03.109832 7fe9bae887a0 20 read_log 194209'18503 (194209'18462) delete f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:628 2014-08-12 14:07:41.871078 2014-08-13 17:38:03.109844 7fe9bae887a0 20 read_log 194209'18504 (194209'18465) delete 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:629 2014-08-12 14:07:41.874977 2014-08-13 17:38:03.109856 7fe9bae887a0 20 read_log 194209'18505 (194209'18466) delete 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:630 2014-08-12 14:07:41.875363 2014-08-13 17:38:03.109868 7fe9bae887a0 20 read_log 194209'18506 (194209'18467) delete 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:631 2014-08-12 14:07:41.875961 2014-08-13 17:38:03.109880 7fe9bae887a0 20 read_log 194209'18507 (194209'18470) delete 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:632 2014-08-12 14:07:41.879498 2014-08-13 17:38:03.109892 7fe9bae887a0 20 read_log 194209'18508 (194209'18474) delete 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:633 2014-08-12 14:07:41.880567 2014-08-13 17:38:03.109905 7fe9bae887a0 20 read_log 194209'18509 (194209'18478) delete 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:634 2014-08-12 14:07:41.884031 2014-08-13 17:38:03.109917 7fe9bae887a0 20 read_log 194209'18510 (194209'18481) delete 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:635 2014-08-12 14:07:41.884480 2014-08-13 17:38:03.109929 7fe9bae887a0 20 read_log 194209'18511 (194209'18485) delete 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:636 2014-08-12 14:07:41.885428 2014-08-13 17:38:03.109941 7fe9bae887a0 20 read_log 194209'18512 (194209'18497) delete 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:637 2014-08-12 14:07:41.889031 2014-08-13 17:38:03.109953 7fe9bae887a0 20 read_log 194209'18513 (194209'18472) delete f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.356.0:638 2014-08-12 14:07:51.998618 2014-08-13 17:38:03.109965 7fe9bae887a0 20 read_log 194209'18514 (194209'18473) delete 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:639 2014-08-12 14:07:51.999204 2014-08-13 17:38:03.109977 7fe9bae887a0 20 read_log 194209'18515 (194209'18475) delete 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:640 2014-08-12 14:07:51.999657 2014-08-13 17:38:03.110010 7fe9bae887a0 20 read_log 194209'18516 (194209'18476) delete 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:641 2014-08-12 14:07:52.000091 2014-08-13 17:38:03.110026 7fe9bae887a0 20 read_log 194209'18517 (194209'18477) delete 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:642 2014-08-12 14:07:52.004933 2014-08-13 17:38:03.110038 7fe9bae887a0 20 read_log 194209'18518 (194209'18479) delete adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.356.0:643 2014-08-12 14:07:52.005357 2014-08-13 17:38:03.110053 7fe9bae887a0 20 read_log 194209'18519 (194209'18480) delete e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:644 2014-08-12 14:07:52.005735 2014-08-13 17:38:03.110065 7fe9bae887a0 20 read_log 194209'18520 (194209'18482) delete 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.356.0:645 2014-08-12 14:07:52.006337 2014-08-13 17:38:03.110079 7fe9bae887a0 20 read_log 194209'18521 (194209'18483) delete e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:646 2014-08-12 14:07:52.009860 2014-08-13 17:38:03.110099 7fe9bae887a0 20 read_log 194209'18522 (194209'18484) delete b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:647 2014-08-12 14:07:52.010274 2014-08-13 17:38:03.110114 7fe9bae887a0 20 read_log 194209'18523 (194209'18486) delete c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:648 2014-08-12 14:07:52.011253 2014-08-13 17:38:03.110126 7fe9bae887a0 20 read_log 194209'18524 (194209'18487) delete 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:649 2014-08-12 14:07:52.011672 2014-08-13 17:38:03.110139 7fe9bae887a0 20 read_log 194209'18525 (194209'18488) delete e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.356.0:650 2014-08-12 14:07:52.014938 2014-08-13 17:38:03.110174 7fe9bae887a0 20 read_log 194209'18526 (194209'18489) delete 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:651 2014-08-12 14:07:52.015324 2014-08-13 17:38:03.110190 7fe9bae887a0 20 read_log 194209'18527 (194209'18490) delete 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:652 2014-08-12 14:07:52.016173 2014-08-13 17:38:03.110202 7fe9bae887a0 20 read_log 194209'18528 (194209'18491) delete 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.356.0:653 2014-08-12 14:07:52.016573 2014-08-13 17:38:03.110215 7fe9bae887a0 20 read_log 194209'18529 (194209'18492) delete 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:654 2014-08-12 14:07:52.020299 2014-08-13 17:38:03.110229 7fe9bae887a0 20 read_log 194209'18530 (194209'18493) delete 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:655 2014-08-12 14:07:52.020707 2014-08-13 17:38:03.110241 7fe9bae887a0 20 read_log 194209'18531 (194209'18494) delete 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.356.0:656 2014-08-12 14:07:52.021098 2014-08-13 17:38:03.110254 7fe9bae887a0 20 read_log 194209'18532 (194209'18495) delete f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.356.0:657 2014-08-12 14:07:52.021690 2014-08-13 17:38:03.110266 7fe9bae887a0 20 read_log 194209'18533 (194209'18496) delete 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:658 2014-08-12 14:07:52.024843 2014-08-13 17:38:03.110278 7fe9bae887a0 20 read_log 194209'18534 (194209'18498) delete 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.356.0:659 2014-08-12 14:07:52.025263 2014-08-13 17:38:03.110290 7fe9bae887a0 20 read_log 194209'18535 (194209'18463) delete b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:660 2014-08-12 14:07:52.057844 2014-08-13 17:38:03.110302 7fe9bae887a0 20 read_log 194209'18536 (194209'18464) delete e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:661 2014-08-12 14:07:52.058288 2014-08-13 17:38:03.110315 7fe9bae887a0 20 read_log 194209'18537 (194209'18468) delete c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:662 2014-08-12 14:07:52.060694 2014-08-13 17:38:03.110327 7fe9bae887a0 20 read_log 194209'18538 (194209'18469) delete 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.356.0:663 2014-08-12 14:07:52.061031 2014-08-13 17:38:03.110339 7fe9bae887a0 20 read_log 194209'18539 (194209'18471) delete ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:664 2014-08-12 14:07:52.061321 2014-08-13 17:38:03.110352 7fe9bae887a0 20 read_log 194215'18540 (194215'18540) modify 3b7/hit_set_24.3b7_archive_2014-08-12 15:32:25.978488_2014-08-12 15:50:56.911217/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:56.911482 2014-08-13 17:38:03.110375 7fe9bae887a0 20 read_log 194215'18541 (193305'18361) delete 3b7/hit_set_24.3b7_archive_2014-08-11 18:37:35.561357_2014-08-11 18:42:37.204550/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:56.911482 2014-08-13 17:38:03.110390 7fe9bae887a0 20 read_log 194215'18542 (193305'18427) clean 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by osd.356.0:1236 2014-08-12 15:50:56.914547 2014-08-13 17:38:03.110403 7fe9bae887a0 20 read_log 194215'18543 (194215'18542) delete 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.2174595.0:196574 0.000000 2014-08-13 17:38:03.110414 7fe9bae887a0 20 read_log 194215'18544 (192834'17855) clean d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by osd.356.0:1238 2014-08-12 15:50:56.923968 2014-08-13 17:38:03.110426 7fe9bae887a0 20 read_log 194215'18545 (194215'18544) delete d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.2174595.0:196576 0.000000 2014-08-13 17:38:03.110437 7fe9bae887a0 20 read_log 194215'18546 (192888'18151) clean 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by osd.356.0:1240 2014-08-12 15:50:56.931770 2014-08-13 17:38:03.110449 7fe9bae887a0 20 read_log 194215'18547 (194215'18546) delete 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.2174595.0:196578 0.000000 2014-08-13 17:38:03.110460 7fe9bae887a0 20 read_log 194215'18548 (192834'17919) clean eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by osd.356.0:1242 2014-08-12 15:50:56.939662 2014-08-13 17:38:03.110472 7fe9bae887a0 20 read_log 194215'18549 (194215'18548) delete eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.2174595.0:196580 0.000000 2014-08-13 17:38:03.110483 7fe9bae887a0 20 read_log 194215'18550 (193305'18451) clean 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by osd.356.0:1244 2014-08-12 15:50:56.947360 2014-08-13 17:38:03.110495 7fe9bae887a0 20 read_log 194215'18551 (194215'18550) delete 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.2174595.0:196582 0.000000 2014-08-13 17:38:03.110505 7fe9bae887a0 20 read_log 194215'18552 (192888'18117) clean c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by osd.356.0:1246 2014-08-12 15:50:56.955114 2014-08-13 17:38:03.110517 7fe9bae887a0 20 read_log 194215'18553 (194215'18552) delete c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.2174595.0:196584 0.000000 2014-08-13 17:38:03.110528 7fe9bae887a0 20 read_log 194215'18554 (192888'18307) clean 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by osd.356.0:1248 2014-08-12 15:50:56.962192 2014-08-13 17:38:03.110540 7fe9bae887a0 20 read_log 194215'18555 (194215'18554) delete 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.2174595.0:196586 0.000000 2014-08-13 17:38:03.110551 7fe9bae887a0 20 read_log 194215'18556 (192888'18147) clean d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by osd.356.0:1250 2014-08-12 15:50:56.969710 2014-08-13 17:38:03.110563 7fe9bae887a0 20 read_log 194215'18557 (194215'18556) delete d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.2174595.0:196588 0.000000 2014-08-13 17:38:03.110573 7fe9bae887a0 20 read_log 194215'18558 (192888'18120) clean 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by osd.356.0:1252 2014-08-12 15:50:56.977487 2014-08-13 17:38:03.110586 7fe9bae887a0 20 read_log 194215'18559 (194215'18558) delete 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.2174595.0:196590 0.000000 2014-08-13 17:38:03.110604 7fe9bae887a0 20 read_log 194215'18560 (193305'18452) clean 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by osd.356.0:1254 2014-08-12 15:50:56.985314 2014-08-13 17:38:03.110616 7fe9bae887a0 20 read_log 194215'18561 (194215'18560) delete 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.2174595.0:196592 0.000000 2014-08-13 17:38:03.110627 7fe9bae887a0 20 read_log 194215'18562 (192888'18123) clean 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by osd.356.0:1256 2014-08-12 15:50:56.992600 2014-08-13 17:38:03.110639 7fe9bae887a0 20 read_log 194215'18563 (194215'18562) delete 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.2174595.0:196594 0.000000 2014-08-13 17:38:03.110651 7fe9bae887a0 20 read_log 194215'18564 (192888'18129) clean bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by osd.356.0:1258 2014-08-12 15:50:57.001002 2014-08-13 17:38:03.110664 7fe9bae887a0 20 read_log 194215'18565 (194215'18564) delete bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.2174595.0:196596 0.000000 2014-08-13 17:38:03.110674 7fe9bae887a0 20 read_log 194215'18566 (192834'17401) clean b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by osd.356.0:1260 2014-08-12 15:50:57.008322 2014-08-13 17:38:03.110686 7fe9bae887a0 20 read_log 194215'18567 (194215'18566) delete b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.2174595.0:196598 0.000000 2014-08-13 17:38:03.110697 7fe9bae887a0 20 read_log 194215'18568 (193305'18444) clean a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by osd.356.0:1262 2014-08-12 15:50:57.015506 2014-08-13 17:38:03.110709 7fe9bae887a0 20 read_log 194215'18569 (194215'18568) delete a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.2174595.0:196600 0.000000 2014-08-13 17:38:03.110720 7fe9bae887a0 20 read_log 194215'18570 (193305'18434) clean 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by osd.356.0:1264 2014-08-12 15:50:57.022820 2014-08-13 17:38:03.110733 7fe9bae887a0 20 read_log 194215'18571 (194215'18570) delete 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.2174595.0:196602 0.000000 2014-08-13 17:38:03.110744 7fe9bae887a0 20 read_log 194215'18572 (192834'17367) clean 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by osd.356.0:1266 2014-08-12 15:50:57.029742 2014-08-13 17:38:03.110756 7fe9bae887a0 20 read_log 194215'18573 (194215'18572) delete 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.2174595.0:196604 0.000000 2014-08-13 17:38:03.110766 7fe9bae887a0 20 read_log 194215'18574 (193305'18435) clean a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by osd.356.0:1268 2014-08-12 15:50:57.037354 2014-08-13 17:38:03.110778 7fe9bae887a0 20 read_log 194215'18575 (194215'18574) delete a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.2174595.0:196606 0.000000 2014-08-13 17:38:03.110789 7fe9bae887a0 20 read_log 194215'18576 (192834'17439) clean 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by osd.356.0:1270 2014-08-12 15:50:57.045101 2014-08-13 17:38:03.110801 7fe9bae887a0 20 read_log 194215'18577 (194215'18576) delete 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.2174595.0:196608 0.000000 2014-08-13 17:38:03.110812 7fe9bae887a0 20 read_log 194215'18578 (192834'17807) clean 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by osd.356.0:1272 2014-08-12 15:50:57.052539 2014-08-13 17:38:03.110824 7fe9bae887a0 20 read_log 194215'18579 (194215'18578) delete 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.2174595.0:196610 0.000000 2014-08-13 17:38:03.110842 7fe9bae887a0 20 read_log 194215'18580 (192888'18305) clean 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by osd.356.0:1274 2014-08-12 15:50:57.060019 2014-08-13 17:38:03.110855 7fe9bae887a0 20 read_log 194215'18581 (194215'18580) delete 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.2174595.0:196612 0.000000 2014-08-13 17:38:03.110865 7fe9bae887a0 20 read_log 194215'18582 (192888'18113) clean b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by osd.356.0:1276 2014-08-12 15:50:57.067458 2014-08-13 17:38:03.110878 7fe9bae887a0 20 read_log 194215'18583 (194215'18582) delete b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.2174595.0:196614 0.000000 2014-08-13 17:38:03.110888 7fe9bae887a0 20 read_log 194215'18584 (192888'18156) clean 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by osd.356.0:1278 2014-08-12 15:50:57.075594 2014-08-13 17:38:03.110900 7fe9bae887a0 20 read_log 194215'18585 (194215'18584) delete 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.2174595.0:196616 0.000000 2014-08-13 17:38:03.110911 7fe9bae887a0 20 read_log 194215'18586 (192834'17844) clean acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by osd.356.0:1280 2014-08-12 15:50:57.081661 2014-08-13 17:38:03.110923 7fe9bae887a0 20 read_log 194215'18587 (194215'18586) delete acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.2174595.0:196618 0.000000 2014-08-13 17:38:03.110934 7fe9bae887a0 20 read_log 194215'18588 (192888'18329) clean bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by osd.356.0:1282 2014-08-12 15:50:57.087674 2014-08-13 17:38:03.110946 7fe9bae887a0 20 read_log 194215'18589 (194215'18588) delete bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.2174595.0:196620 0.000000 2014-08-13 17:38:03.110957 7fe9bae887a0 20 read_log 194215'18590 (192834'17346) clean f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by osd.356.0:1284 2014-08-12 15:50:57.093834 2014-08-13 17:38:03.110969 7fe9bae887a0 20 read_log 194215'18591 (194215'18590) delete f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.2174595.0:196622 0.000000 2014-08-13 17:38:03.110980 7fe9bae887a0 20 read_log 194215'18592 (192888'18115) clean f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by osd.356.0:1286 2014-08-12 15:50:57.100230 2014-08-13 17:38:03.110995 7fe9bae887a0 20 read_log 194215'18593 (194215'18592) delete f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.2174595.0:196624 0.000000 2014-08-13 17:38:03.111005 7fe9bae887a0 20 read_log 194215'18594 (192888'18308) clean a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by osd.356.0:1288 2014-08-12 15:50:57.106344 2014-08-13 17:38:03.111020 7fe9bae887a0 20 read_log 194215'18595 (194215'18594) delete a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.2174595.0:196626 0.000000 2014-08-13 17:38:03.111031 7fe9bae887a0 20 read_log 194215'18596 (192888'18126) clean 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by osd.356.0:1290 2014-08-12 15:50:57.112373 2014-08-13 17:38:03.111046 7fe9bae887a0 20 read_log 194215'18597 (194215'18596) delete 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.2174595.0:196628 0.000000 2014-08-13 17:38:03.111057 7fe9bae887a0 20 read_log 194215'18598 (193305'18448) clean e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by osd.356.0:1292 2014-08-12 15:50:57.117962 2014-08-13 17:38:03.111069 7fe9bae887a0 20 read_log 194215'18599 (194215'18598) delete e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.2174595.0:196630 0.000000 2014-08-13 17:38:03.111091 7fe9bae887a0 20 read_log 194215'18600 (192888'18321) clean ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by osd.356.0:1294 2014-08-12 15:50:57.124127 2014-08-13 17:38:03.111108 7fe9bae887a0 20 read_log 194215'18601 (194215'18600) delete ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.2174595.0:196632 0.000000 2014-08-13 17:38:03.111119 7fe9bae887a0 20 read_log 194215'18602 (192888'18102) clean 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by osd.356.0:1296 2014-08-12 15:50:57.130579 2014-08-13 17:38:03.111132 7fe9bae887a0 20 read_log 194215'18603 (194215'18602) delete 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.2174595.0:196634 0.000000 2014-08-13 17:38:03.111143 7fe9bae887a0 20 read_log 194215'18604 (192834'17456) clean f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by osd.356.0:1298 2014-08-12 15:50:57.136881 2014-08-13 17:38:03.111160 7fe9bae887a0 20 read_log 194215'18605 (194215'18604) delete f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.2174595.0:196636 0.000000 2014-08-13 17:38:03.111174 7fe9bae887a0 20 read_log 194215'18606 (192888'18301) clean 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by osd.356.0:1300 2014-08-12 15:50:57.143069 2014-08-13 17:38:03.111186 7fe9bae887a0 20 read_log 194215'18607 (194215'18606) delete 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.2174595.0:196638 0.000000 2014-08-13 17:38:03.111200 7fe9bae887a0 20 read_log 194215'18608 (192834'17876) clean 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by osd.356.0:1302 2014-08-12 15:50:57.149343 2014-08-13 17:38:03.111214 7fe9bae887a0 20 read_log 194215'18609 (194215'18608) delete 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.2174595.0:196640 0.000000 2014-08-13 17:38:03.111225 7fe9bae887a0 20 read_log 194215'18610 (192834'17428) clean 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by osd.356.0:1304 2014-08-12 15:50:57.155240 2014-08-13 17:38:03.111237 7fe9bae887a0 20 read_log 194215'18611 (194215'18610) delete 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.2174595.0:196642 0.000000 2014-08-13 17:38:03.111250 7fe9bae887a0 20 read_log 194215'18612 (192888'18119) clean 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by osd.356.0:1306 2014-08-12 15:50:57.161080 2014-08-13 17:38:03.111264 7fe9bae887a0 20 read_log 194215'18613 (194215'18612) delete 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.2174595.0:196644 0.000000 2014-08-13 17:38:03.111279 7fe9bae887a0 20 read_log 194215'18614 (193305'18441) clean eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by osd.356.0:1308 2014-08-12 15:50:57.167313 2014-08-13 17:38:03.111291 7fe9bae887a0 20 read_log 194215'18615 (194215'18614) delete eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.2174595.0:196646 0.000000 2014-08-13 17:38:03.111303 7fe9bae887a0 20 read_log 194215'18616 (193305'18430) clean f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by osd.356.0:1310 2014-08-12 15:50:57.174337 2014-08-13 17:38:03.111317 7fe9bae887a0 20 read_log 194215'18617 (194215'18616) delete f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.2174595.0:196648 0.000000 2014-08-13 17:38:03.111328 7fe9bae887a0 20 read_log 194215'18618 (193305'18455) clean 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by osd.356.0:1312 2014-08-12 15:50:57.181825 2014-08-13 17:38:03.111358 7fe9bae887a0 20 read_log 194215'18619 (194215'18618) delete 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.2174595.0:196650 0.000000 2014-08-13 17:38:03.111393 7fe9bae887a0 20 read_log 194215'18620 (192834'17834) clean 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by osd.356.0:1314 2014-08-12 15:50:57.189768 2014-08-13 17:38:03.111409 7fe9bae887a0 20 read_log 194215'18621 (194215'18620) delete 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.2174595.0:196652 0.000000 2014-08-13 17:38:03.111420 7fe9bae887a0 20 read_log 194215'18622 (192834'17921) clean 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by osd.356.0:1316 2014-08-12 15:50:57.197224 2014-08-13 17:38:03.111432 7fe9bae887a0 20 read_log 194215'18623 (194215'18622) delete 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.2174595.0:196654 0.000000 2014-08-13 17:38:03.111443 7fe9bae887a0 20 read_log 194215'18624 (192888'18127) clean eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by osd.356.0:1318 2014-08-12 15:50:57.204537 2014-08-13 17:38:03.111455 7fe9bae887a0 20 read_log 194215'18625 (194215'18624) delete eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.2174595.0:196656 0.000000 2014-08-13 17:38:03.111466 7fe9bae887a0 20 read_log 194215'18626 (193305'18428) clean 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by osd.356.0:1320 2014-08-12 15:50:57.211494 2014-08-13 17:38:03.111478 7fe9bae887a0 20 read_log 194215'18627 (194215'18626) delete 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.2174595.0:196658 0.000000 2014-08-13 17:38:03.111488 7fe9bae887a0 20 read_log 194215'18628 (192834'17477) clean 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by osd.356.0:1322 2014-08-12 15:50:57.218445 2014-08-13 17:38:03.111501 7fe9bae887a0 20 read_log 194215'18629 (194215'18628) delete 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.2174595.0:196660 0.000000 2014-08-13 17:38:03.111512 7fe9bae887a0 20 read_log 194215'18630 (192834'17860) clean 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by osd.356.0:1324 2014-08-12 15:50:57.225519 2014-08-13 17:38:03.111524 7fe9bae887a0 20 read_log 194215'18631 (194215'18630) delete 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.2174595.0:196662 0.000000 2014-08-13 17:38:03.111534 7fe9bae887a0 20 read_log 194215'18632 (193305'18431) clean 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by osd.356.0:1326 2014-08-12 15:50:57.232637 2014-08-13 17:38:03.111547 7fe9bae887a0 20 read_log 194215'18633 (194215'18632) delete 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.2174595.0:196664 0.000000 2014-08-13 17:38:03.111557 7fe9bae887a0 20 read_log 194215'18634 (193305'18453) clean 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by osd.356.0:1328 2014-08-12 15:50:57.239435 2014-08-13 17:38:03.111570 7fe9bae887a0 20 read_log 194215'18635 (194215'18634) delete 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.2174595.0:196666 0.000000 2014-08-13 17:38:03.111580 7fe9bae887a0 20 read_log 194215'18636 (192888'18122) clean 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by osd.356.0:1330 2014-08-12 15:50:57.246442 2014-08-13 17:38:03.111592 7fe9bae887a0 20 read_log 194215'18637 (194215'18636) delete 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.2174595.0:196668 0.000000 2014-08-13 17:38:03.111603 7fe9bae887a0 20 read_log 194215'18638 (192888'18303) clean 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by osd.356.0:1332 2014-08-12 15:50:57.253740 2014-08-13 17:38:03.111615 7fe9bae887a0 20 read_log 194215'18639 (194215'18638) delete 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.2174595.0:196670 0.000000 2014-08-13 17:38:03.111633 7fe9bae887a0 20 read_log 194215'18640 (192888'18311) clean d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by osd.356.0:1334 2014-08-12 15:50:57.261632 2014-08-13 17:38:03.111645 7fe9bae887a0 20 read_log 194215'18641 (194215'18640) delete d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.2174595.0:196672 0.000000 2014-08-13 17:38:03.111656 7fe9bae887a0 20 read_log 194215'18642 (193305'18445) clean a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by osd.356.0:1336 2014-08-12 15:50:57.269743 2014-08-13 17:38:03.111668 7fe9bae887a0 20 read_log 194215'18643 (194215'18642) delete a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.2174595.0:196674 0.000000 2014-08-13 17:38:03.111679 7fe9bae887a0 20 read_log 194215'18644 (192834'17936) clean 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by osd.356.0:1338 2014-08-12 15:50:57.277575 2014-08-13 17:38:03.111691 7fe9bae887a0 20 read_log 194215'18645 (194215'18644) delete 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.2174595.0:196676 0.000000 2014-08-13 17:38:03.111701 7fe9bae887a0 20 read_log 194215'18646 (192888'18152) clean e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by osd.356.0:1340 2014-08-12 15:50:57.284595 2014-08-13 17:38:03.111714 7fe9bae887a0 20 read_log 194215'18647 (194215'18646) delete e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.2174595.0:196678 0.000000 2014-08-13 17:38:03.111725 7fe9bae887a0 20 read_log 194215'18648 (192888'18327) clean e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by osd.356.0:1342 2014-08-12 15:50:57.291978 2014-08-13 17:38:03.111737 7fe9bae887a0 20 read_log 194215'18649 (194215'18648) delete e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.2174595.0:196680 0.000000 2014-08-13 17:38:03.111748 7fe9bae887a0 20 read_log 194215'18650 (192888'18340) clean 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by osd.356.0:1344 2014-08-12 15:50:57.299059 2014-08-13 17:38:03.111760 7fe9bae887a0 20 read_log 194215'18651 (194215'18650) delete 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.2174595.0:196682 0.000000 2014-08-13 17:38:03.111771 7fe9bae887a0 20 read_log 194215'18652 (192834'17411) clean 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by osd.356.0:1346 2014-08-12 15:50:57.306782 2014-08-13 17:38:03.111783 7fe9bae887a0 20 read_log 194215'18653 (194215'18652) delete 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.2174595.0:196684 0.000000 2014-08-13 17:38:03.111794 7fe9bae887a0 20 read_log 194215'18654 (192888'18355) clean c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by osd.356.0:1348 2014-08-12 15:50:57.314761 2014-08-13 17:38:03.111806 7fe9bae887a0 20 read_log 194215'18655 (194215'18654) delete c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.2174595.0:196686 0.000000 2014-08-13 17:38:03.111817 7fe9bae887a0 20 read_log 194215'18656 (192888'18101) clean 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by osd.356.0:1350 2014-08-12 15:50:57.322144 2014-08-13 17:38:03.111829 7fe9bae887a0 20 read_log 194215'18657 (194215'18656) delete 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.2174595.0:196688 0.000000 2014-08-13 17:38:03.111839 7fe9bae887a0 20 read_log 194215'18658 (192888'18109) clean 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by osd.356.0:1352 2014-08-12 15:50:57.330181 2014-08-13 17:38:03.111851 7fe9bae887a0 20 read_log 194215'18659 (194215'18658) delete 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.2174595.0:196690 0.000000 2014-08-13 17:38:03.111870 7fe9bae887a0 20 read_log 194215'18660 (193305'18437) clean 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by osd.356.0:1354 2014-08-12 15:50:57.337730 2014-08-13 17:38:03.111883 7fe9bae887a0 20 read_log 194215'18661 (194215'18660) delete 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.2174595.0:196692 0.000000 2014-08-13 17:38:03.111894 7fe9bae887a0 20 read_log 194215'18662 (192834'17930) clean 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by osd.356.0:1356 2014-08-12 15:50:57.345169 2014-08-13 17:38:03.111907 7fe9bae887a0 20 read_log 194215'18663 (194215'18662) delete 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.2174595.0:196694 0.000000 2014-08-13 17:38:03.111917 7fe9bae887a0 20 read_log 194215'18664 (193305'18457) clean 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by osd.356.0:1358 2014-08-12 15:50:57.353138 2014-08-13 17:38:03.111930 7fe9bae887a0 20 read_log 194215'18665 (194215'18664) delete 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.2174595.0:196696 0.000000 2014-08-13 17:38:03.111941 7fe9bae887a0 20 read_log 194215'18666 (192888'18154) clean cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by osd.356.0:1360 2014-08-12 15:50:57.361130 2014-08-13 17:38:03.111953 7fe9bae887a0 20 read_log 194215'18667 (194215'18666) delete cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.2174595.0:196698 0.000000 2014-08-13 17:38:03.111964 7fe9bae887a0 20 read_log 194215'18668 (192888'18112) clean 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by osd.356.0:1362 2014-08-12 15:50:57.369486 2014-08-13 17:38:03.111977 7fe9bae887a0 20 read_log 194215'18669 (194215'18668) delete 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.2174595.0:196700 0.000000 2014-08-13 17:38:03.111988 7fe9bae887a0 20 read_log 194215'18670 (192834'17891) clean db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by osd.356.0:1364 2014-08-12 15:50:57.377587 2014-08-13 17:38:03.112003 7fe9bae887a0 20 read_log 194215'18671 (194215'18670) delete db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.2174595.0:196702 0.000000 2014-08-13 17:38:03.112014 7fe9bae887a0 20 read_log 194215'18672 (193305'18439) clean 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by osd.356.0:1366 2014-08-12 15:50:57.385858 2014-08-13 17:38:03.112026 7fe9bae887a0 20 read_log 194215'18673 (194215'18672) delete 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.2174595.0:196704 0.000000 2014-08-13 17:38:03.112037 7fe9bae887a0 20 read_log 194215'18674 (192834'17416) clean 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by osd.356.0:1368 2014-08-12 15:50:57.393776 2014-08-13 17:38:03.112051 7fe9bae887a0 20 read_log 194215'18675 (194215'18674) delete 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.2174595.0:196706 0.000000 2014-08-13 17:38:03.112062 7fe9bae887a0 20 read_log 194215'18676 (193305'18456) clean 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by osd.356.0:1370 2014-08-12 15:50:57.401412 2014-08-13 17:38:03.112074 7fe9bae887a0 20 read_log 194215'18677 (194215'18676) delete 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.2174595.0:196708 0.000000 2014-08-13 17:38:03.112085 7fe9bae887a0 20 read_log 194215'18678 (192834'17866) clean 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by osd.356.0:1372 2014-08-12 15:50:57.409105 2014-08-13 17:38:03.112099 7fe9bae887a0 20 read_log 194215'18679 (194215'18678) delete 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.2174595.0:196710 0.000000 2014-08-13 17:38:03.112117 7fe9bae887a0 20 read_log 194215'18680 (192888'18153) clean 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by osd.356.0:1374 2014-08-12 15:50:57.416926 2014-08-13 17:38:03.112131 7fe9bae887a0 20 read_log 194215'18681 (194215'18680) delete 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.2174595.0:196712 0.000000 2014-08-13 17:38:03.112142 7fe9bae887a0 20 read_log 194215'18682 (192834'17884) clean f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by osd.356.0:1376 2014-08-12 15:50:57.424114 2014-08-13 17:38:03.112184 7fe9bae887a0 20 read_log 194215'18683 (194215'18682) delete f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.2174595.0:196714 0.000000 2014-08-13 17:38:03.112195 7fe9bae887a0 20 read_log 194215'18684 (192888'18359) clean ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by osd.356.0:1378 2014-08-12 15:50:57.431334 2014-08-13 17:38:03.112208 7fe9bae887a0 20 read_log 194215'18685 (194215'18684) delete ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.2174595.0:196716 0.000000 2014-08-13 17:38:03.112218 7fe9bae887a0 20 read_log 194215'18686 (192888'18304) clean b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by osd.356.0:1380 2014-08-12 15:50:57.439067 2014-08-13 17:38:03.112231 7fe9bae887a0 20 read_log 194215'18687 (194215'18686) delete b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.2174595.0:196718 0.000000 2014-08-13 17:38:03.112242 7fe9bae887a0 20 read_log 194215'18688 (192834'17928) clean b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by osd.356.0:1382 2014-08-12 15:50:57.446565 2014-08-13 17:38:03.112255 7fe9bae887a0 20 read_log 194215'18689 (194215'18688) delete b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.2174595.0:196720 0.000000 2014-08-13 17:38:03.112267 7fe9bae887a0 20 read_log 194215'18690 (192888'18295) clean e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by osd.356.0:1384 2014-08-12 15:50:57.455099 2014-08-13 17:38:03.112282 7fe9bae887a0 20 read_log 194215'18691 (194215'18690) delete e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.2174595.0:196722 0.000000 2014-08-13 17:38:03.112293 7fe9bae887a0 20 read_log 194215'18692 (192888'18322) clean c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by osd.356.0:1386 2014-08-12 15:50:57.462982 2014-08-13 17:38:03.112305 7fe9bae887a0 20 read_log 194215'18693 (194215'18692) delete c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.2174595.0:196724 0.000000 2014-08-13 17:38:03.112316 7fe9bae887a0 20 read_log 194215'18694 (192888'18354) clean 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by osd.356.0:1388 2014-08-12 15:50:57.471038 2014-08-13 17:38:03.112329 7fe9bae887a0 20 read_log 194215'18695 (194215'18694) delete 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.2174595.0:196726 0.000000 2014-08-13 17:38:03.112339 7fe9bae887a0 20 read_log 194215'18696 (192888'18318) clean 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by osd.356.0:1390 2014-08-12 15:50:57.478951 2014-08-13 17:38:03.112351 7fe9bae887a0 20 read_log 194215'18697 (194215'18696) delete 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.2174595.0:196728 0.000000 2014-08-13 17:38:03.112362 7fe9bae887a0 20 read_log 194215'18698 (193305'18442) clean 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by osd.356.0:1392 2014-08-12 15:50:57.486856 2014-08-13 17:38:03.112374 7fe9bae887a0 20 read_log 194215'18699 (194215'18698) delete 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.2174595.0:196730 0.000000 2014-08-13 17:38:03.112392 7fe9bae887a0 20 read_log 194215'18700 (193305'18450) clean ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by osd.356.0:1394 2014-08-12 15:50:57.494599 2014-08-13 17:38:03.112404 7fe9bae887a0 20 read_log 194215'18701 (194215'18700) delete ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.2174595.0:196732 0.000000 2014-08-13 17:38:03.112415 7fe9bae887a0 20 read_log 194215'18702 (193305'18429) clean d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by osd.356.0:1396 2014-08-12 15:50:57.501848 2014-08-13 17:38:03.112427 7fe9bae887a0 20 read_log 194215'18703 (194215'18702) delete d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.2174595.0:196734 0.000000 2014-08-13 17:38:03.112438 7fe9bae887a0 20 read_log 194215'18704 (192888'18144) clean c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by osd.356.0:1398 2014-08-12 15:50:57.510487 2014-08-13 17:38:03.112450 7fe9bae887a0 20 read_log 194215'18705 (194215'18704) delete c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.2174595.0:196736 0.000000 2014-08-13 17:38:03.112460 7fe9bae887a0 20 read_log 194215'18706 (192888'18108) clean f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by osd.356.0:1400 2014-08-12 15:50:57.517915 2014-08-13 17:38:03.112472 7fe9bae887a0 20 read_log 194215'18707 (194215'18706) delete f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.2174595.0:196738 0.000000 2014-08-13 17:38:03.112483 7fe9bae887a0 20 read_log 194215'18708 (192888'18298) clean ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by osd.356.0:1402 2014-08-12 15:50:57.525098 2014-08-13 17:38:03.112495 7fe9bae887a0 20 read_log 194215'18709 (194215'18708) delete ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.2174595.0:196740 0.000000 2014-08-13 17:38:03.112505 7fe9bae887a0 20 read_log 194215'18710 (192888'18142) clean 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by osd.356.0:1404 2014-08-12 15:50:57.532776 2014-08-13 17:38:03.112517 7fe9bae887a0 20 read_log 194215'18711 (194215'18710) delete 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.2174595.0:196742 0.000000 2014-08-13 17:38:03.112544 7fe9bae887a0 20 read_log 194215'18712 (192888'18125) clean 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by osd.356.0:1406 2014-08-12 15:50:57.540472 2014-08-13 17:38:03.112556 7fe9bae887a0 20 read_log 194215'18713 (194215'18712) delete 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.2174595.0:196744 0.000000 2014-08-13 17:38:03.112567 7fe9bae887a0 20 read_log 194215'18714 (192888'18116) clean 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by osd.356.0:1408 2014-08-12 15:50:57.548312 2014-08-13 17:38:03.112579 7fe9bae887a0 20 read_log 194215'18715 (194215'18714) delete 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.2174595.0:196746 0.000000 2014-08-13 17:38:03.112590 7fe9bae887a0 20 read_log 194215'18716 (192834'17861) clean eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by osd.356.0:1410 2014-08-12 15:50:57.556333 2014-08-13 17:38:03.112602 7fe9bae887a0 20 read_log 194215'18717 (194215'18716) delete eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.2174595.0:196748 0.000000 2014-08-13 17:38:03.112613 7fe9bae887a0 20 read_log 194215'18718 (192888'18328) clean eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by osd.356.0:1412 2014-08-12 15:50:57.564498 2014-08-13 17:38:03.112625 7fe9bae887a0 20 read_log 194215'18719 (194215'18718) delete eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.2174595.0:196750 0.000000 2014-08-13 17:38:03.112645 7fe9bae887a0 20 read_log 194215'18720 (192834'17409) clean 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by osd.356.0:1414 2014-08-12 15:50:57.572345 2014-08-13 17:38:03.112658 7fe9bae887a0 20 read_log 194215'18721 (194215'18720) delete 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.2174595.0:196752 0.000000 2014-08-13 17:38:03.112669 7fe9bae887a0 20 read_log 194215'18722 (192834'17833) clean a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by osd.356.0:1416 2014-08-12 15:50:57.580396 2014-08-13 17:38:03.112681 7fe9bae887a0 20 read_log 194215'18723 (194215'18722) delete a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.2174595.0:196754 0.000000 2014-08-13 17:38:03.112692 7fe9bae887a0 20 read_log 194215'18724 (192888'18330) clean 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by osd.356.0:1418 2014-08-12 15:50:57.612878 2014-08-13 17:38:03.112703 7fe9bae887a0 20 read_log 194215'18725 (194215'18724) delete 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.2174595.0:196756 0.000000 2014-08-13 17:38:03.112714 7fe9bae887a0 20 read_log 194215'18726 (193305'18447) clean e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by osd.356.0:1420 2014-08-12 15:50:57.620632 2014-08-13 17:38:03.112726 7fe9bae887a0 20 read_log 194215'18727 (194215'18726) delete e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.2174595.0:196758 0.000000 2014-08-13 17:38:03.112737 7fe9bae887a0 20 read_log 194217'18728 (194217'18728) modify 3b7/hit_set_24.3b7_archive_2014-08-12 15:50:56.911487_2014-08-12 17:25:03.810214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.810430 2014-08-13 17:38:03.112750 7fe9bae887a0 20 read_log 194217'18729 (194215'18540) delete 3b7/hit_set_24.3b7_archive_2014-08-12 15:32:25.978488_2014-08-12 15:50:56.911217/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.810430 2014-08-13 17:38:03.112762 7fe9bae887a0 20 read_log 194217'18730 (0'0) promote 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by osd.356.0:1775 2014-08-12 17:25:03.816059 2014-08-13 17:38:03.112774 7fe9bae887a0 20 read_log 194217'18731 (194217'18730) modify 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by client.1487927.0:129 2014-08-12 17:25:03.805212 2014-08-13 17:38:03.112786 7fe9bae887a0 20 read_log 194370'18732 (0'0) promote efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by osd.356.0:1789 2014-08-12 20:30:22.056991 2014-08-13 17:38:03.112798 7fe9bae887a0 20 read_log 194370'18733 (194370'18732) modify efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by client.1508531.0:2038 2014-08-12 20:30:22.049716 2014-08-13 17:38:03.112810 7fe9bae887a0 20 read_log 194370'18734 (0'0) promote b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by osd.356.0:1803 2014-08-12 20:30:30.164477 2014-08-13 17:38:03.112823 7fe9bae887a0 20 read_log 194370'18735 (194370'18734) modify b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by client.1508531.0:4218 2014-08-12 20:30:30.156160 2014-08-13 17:38:03.112835 7fe9bae887a0 20 read_log 194370'18736 (0'0) promote 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by osd.356.0:1811 2014-08-12 20:30:35.339641 2014-08-13 17:38:03.112847 7fe9bae887a0 20 read_log 194370'18737 (194370'18736) modify 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by client.1508531.0:5485 2014-08-12 20:30:35.334220 2014-08-13 17:38:03.112859 7fe9bae887a0 20 read_log 194370'18738 (0'0) promote c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by osd.356.0:1825 2014-08-12 20:30:48.946434 2014-08-13 17:38:03.112879 7fe9bae887a0 20 read_log 194370'18739 (194370'18738) modify c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by client.1508531.0:8257 2014-08-12 20:30:48.939251 2014-08-13 17:38:03.112891 7fe9bae887a0 20 read_log 194370'18740 (0'0) promote 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by osd.356.0:1839 2014-08-12 20:30:59.726485 2014-08-13 17:38:03.112903 7fe9bae887a0 20 read_log 194370'18741 (194370'18740) modify 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by client.1508531.0:10082 2014-08-12 20:30:59.719353 2014-08-13 17:38:03.112915 7fe9bae887a0 20 read_log 194370'18742 (0'0) promote 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by osd.356.0:1841 2014-08-12 20:31:00.857931 2014-08-13 17:38:03.112927 7fe9bae887a0 20 read_log 194370'18743 (194370'18742) modify 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by client.1508531.0:10272 2014-08-12 20:31:00.849645 2014-08-13 17:38:03.112941 7fe9bae887a0 20 read_log 194370'18744 (0'0) promote 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by osd.356.0:1843 2014-08-12 20:31:01.325389 2014-08-13 17:38:03.112953 7fe9bae887a0 20 read_log 194370'18745 (194370'18744) modify 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by client.1508531.0:10351 2014-08-12 20:31:01.318249 2014-08-13 17:38:03.112984 7fe9bae887a0 20 read_log 194370'18746 (0'0) promote a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by osd.356.0:1847 2014-08-12 20:31:03.723013 2014-08-13 17:38:03.112998 7fe9bae887a0 20 read_log 194370'18747 (194370'18746) modify a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by client.1508531.0:10735 2014-08-12 20:31:03.714078 2014-08-13 17:38:03.113010 7fe9bae887a0 20 read_log 194370'18748 (0'0) promote 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by osd.356.0:1849 2014-08-12 20:31:07.515806 2014-08-13 17:38:03.113026 7fe9bae887a0 20 read_log 194370'18749 (194370'18748) modify 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by client.1508531.0:11367 2014-08-12 20:31:07.508175 2014-08-13 17:38:03.113038 7fe9bae887a0 20 read_log 194370'18750 (0'0) promote e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by osd.356.0:1857 2014-08-12 20:31:16.612356 2014-08-13 17:38:03.113053 7fe9bae887a0 20 read_log 194370'18751 (194370'18750) modify e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by client.1508531.0:12709 2014-08-12 20:31:16.602273 2014-08-13 17:38:03.113065 7fe9bae887a0 20 read_log 194370'18752 (0'0) promote 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by osd.356.0:1859 2014-08-12 20:31:17.721544 2014-08-13 17:38:03.113079 7fe9bae887a0 20 read_log 194370'18753 (194370'18752) modify 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by client.1508531.0:12825 2014-08-12 20:31:17.712236 2014-08-13 17:38:03.113093 7fe9bae887a0 20 read_log 194377'18754 (0'0) promote 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by osd.356.0:1895 2014-08-12 20:33:08.237705 2014-08-13 17:38:03.113107 7fe9bae887a0 20 read_log 194377'18755 (194377'18754) modify 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by client.2231190.0:2731 2014-08-12 20:33:08.231506 2014-08-13 17:38:03.113120 7fe9bae887a0 20 read_log 194377'18756 (0'0) promote 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by osd.356.0:1901 2014-08-12 20:33:12.086791 2014-08-13 17:38:03.113134 7fe9bae887a0 20 read_log 194377'18757 (194377'18756) modify 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by client.2231190.0:3645 2014-08-12 20:33:12.080752 2014-08-13 17:38:03.113187 7fe9bae887a0 20 read_log 194377'18758 (0'0) promote 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by osd.356.0:1913 2014-08-12 20:33:17.900671 2014-08-13 17:38:03.113204 7fe9bae887a0 20 read_log 194377'18759 (194377'18758) modify 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by client.2231190.0:4599 2014-08-12 20:33:17.892578 2014-08-13 17:38:03.113217 7fe9bae887a0 20 read_log 194383'18760 (0'0) promote 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by osd.356.0:1923 2014-08-12 20:34:00.084002 2014-08-13 17:38:03.113231 7fe9bae887a0 20 read_log 194383'18761 (194383'18760) modify 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by client.2231190.0:5769 2014-08-12 20:34:00.019150 2014-08-13 17:38:03.113244 7fe9bae887a0 20 read_log 194383'18762 (0'0) promote b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by osd.356.0:1925 2014-08-12 20:34:07.590059 2014-08-13 17:38:03.113258 7fe9bae887a0 20 read_log 194383'18763 (194383'18762) modify b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by client.2231190.0:6721 2014-08-12 20:34:07.583644 2014-08-13 17:38:03.113270 7fe9bae887a0 20 read_log 194383'18764 (0'0) promote c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by osd.356.0:1935 2014-08-12 20:34:24.401887 2014-08-13 17:38:03.113282 7fe9bae887a0 20 read_log 194383'18765 (194383'18764) modify c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by client.2231190.0:7341 2014-08-12 20:34:24.392949 2014-08-13 17:38:03.113295 7fe9bae887a0 20 read_log 194383'18766 (0'0) promote 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by osd.356.0:1937 2014-08-12 20:34:30.701553 2014-08-13 17:38:03.113309 7fe9bae887a0 20 read_log 194383'18767 (194383'18766) modify 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by client.2231190.0:7571 2014-08-12 20:34:30.693282 2014-08-13 17:38:03.113322 7fe9bae887a0 20 read_log 194393'18768 (194393'18768) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:37:09.043051_2014-08-12 20:47:27.879383/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.879597 2014-08-13 17:38:03.113337 7fe9bae887a0 20 read_log 194393'18769 (194217'18728) delete 3b7/hit_set_24.3b7_archive_2014-08-12 15:50:56.911487_2014-08-12 17:25:03.810214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.879597 2014-08-13 17:38:03.113349 7fe9bae887a0 20 read_log 194393'18770 (0'0) promote 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by osd.356.0:1943 2014-08-12 20:47:27.882962 2014-08-13 17:38:03.113362 7fe9bae887a0 20 read_log 194393'18771 (194393'18770) modify 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by client.1064095.0:459 2014-08-12 20:47:27.874894 2014-08-13 17:38:03.113374 7fe9bae887a0 20 read_log 194397'18772 (194397'18772) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:47:27.879602_2014-08-12 20:53:06.322101/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:06.322318 2014-08-13 17:38:03.113387 7fe9bae887a0 20 read_log 194397'18773 (194393'18768) delete 3b7/hit_set_24.3b7_archive_2014-08-12 20:37:09.043051_2014-08-12 20:47:27.879383/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:06.322318 2014-08-13 17:38:03.113399 7fe9bae887a0 20 read_log 194397'18774 (0'0) promote 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by osd.356.0:1962 2014-08-12 20:53:06.325818 2014-08-13 17:38:03.113411 7fe9bae887a0 20 read_log 194397'18775 (194397'18774) modify 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by client.2235069.0:933 2014-08-12 20:53:06.315038 2014-08-13 17:38:03.113424 7fe9bae887a0 20 read_log 194416'18776 (194416'18776) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:53:06.322323_2014-08-13 15:28:31.768122/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 17:38:03.113447 7fe9bae887a0 20 read_log 194416'18777 (194397'18772) delete 3b7/hit_set_24.3b7_archive_2014-08-12 20:47:27.879602_2014-08-12 20:53:06.322101/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 17:38:03.113462 7fe9bae887a0 20 read_log 194416'18778 (0'0) promote 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by osd.356.0:2077 2014-08-13 15:28:31.771636 2014-08-13 17:38:03.113474 7fe9bae887a0 20 read_log 194416'18779 (194416'18778) modify 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by client.2441958.0:369 2014-08-13 15:28:31.761856 2014-08-13 17:38:03.113487 7fe9bae887a0 20 read_log 194416'18780 (0'0) promote 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by osd.356.0:2079 2014-08-13 15:28:33.333612 2014-08-13 17:38:03.113499 7fe9bae887a0 20 read_log 194416'18781 (194416'18780) modify 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by client.2441958.0:719 2014-08-13 15:28:33.326094 2014-08-13 17:38:03.113511 7fe9bae887a0 20 read_log 194416'18782 (0'0) promote b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by osd.356.0:2085 2014-08-13 15:28:36.783438 2014-08-13 17:38:03.113523 7fe9bae887a0 20 read_log 194416'18783 (194416'18782) modify b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by client.2441958.0:1599 2014-08-13 15:28:36.774701 2014-08-13 17:38:03.113536 7fe9bae887a0 20 read_log 194416'18784 (0'0) promote 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by osd.356.0:2105 2014-08-13 15:28:43.236766 2014-08-13 17:38:03.113548 7fe9bae887a0 20 read_log 194416'18785 (194416'18784) modify 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by client.2441958.0:3307 2014-08-13 15:28:43.228885 2014-08-13 17:38:03.113562 7fe9bae887a0 20 read_log 194416'18786 (0'0) promote f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by osd.356.0:2117 2014-08-13 15:28:49.895930 2014-08-13 17:38:03.113574 7fe9bae887a0 20 read_log 194416'18787 (194416'18786) modify f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by client.2441958.0:5001 2014-08-13 15:28:49.888965 2014-08-13 17:38:03.113587 7fe9bae887a0 20 read_log 194416'18788 (0'0) promote 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by osd.356.0:2125 2014-08-13 15:28:52.067271 2014-08-13 17:38:03.113599 7fe9bae887a0 20 read_log 194416'18789 (194416'18788) modify 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by client.2441958.0:5572 2014-08-13 15:28:52.060924 2014-08-13 17:38:03.113611 7fe9bae887a0 20 read_log 194416'18790 (0'0) promote c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by osd.356.0:2135 2014-08-13 15:28:54.342025 2014-08-13 17:38:03.113623 7fe9bae887a0 20 read_log 194416'18791 (194416'18790) modify c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by client.2441958.0:6123 2014-08-13 15:28:54.334808 2014-08-13 17:38:03.113635 7fe9bae887a0 20 read_log 194416'18792 (0'0) promote 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by osd.356.0:2143 2014-08-13 15:28:56.819379 2014-08-13 17:38:03.113647 7fe9bae887a0 20 read_log 194416'18793 (194416'18792) modify 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by client.2441958.0:6743 2014-08-13 15:28:56.811884 2014-08-13 17:38:03.113659 7fe9bae887a0 20 read_log 194416'18794 (0'0) promote d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by osd.356.0:2145 2014-08-13 15:29:00.088480 2014-08-13 17:38:03.113671 7fe9bae887a0 20 read_log 194416'18795 (194416'18794) modify d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by client.2441958.0:7616 2014-08-13 15:29:00.080339 2014-08-13 17:38:03.113691 7fe9bae887a0 20 read_log 194416'18796 (0'0) promote 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by osd.356.0:2153 2014-08-13 15:29:00.974133 2014-08-13 17:38:03.113704 7fe9bae887a0 20 read_log 194416'18797 (194416'18796) modify 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by client.2441958.0:7852 2014-08-13 15:29:00.965404 2014-08-13 17:38:03.113716 7fe9bae887a0 20 read_log 194416'18798 (0'0) promote 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by osd.356.0:2155 2014-08-13 15:29:02.330139 2014-08-13 17:38:03.113728 7fe9bae887a0 20 read_log 194416'18799 (194416'18798) modify 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by client.2441958.0:8213 2014-08-13 15:29:02.324036 2014-08-13 17:38:03.113740 7fe9bae887a0 20 read_log 194416'18800 (0'0) promote fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by osd.356.0:2161 2014-08-13 15:29:05.284996 2014-08-13 17:38:03.113752 7fe9bae887a0 20 read_log 194416'18801 (194416'18800) modify fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by client.2441958.0:8955 2014-08-13 15:29:05.276987 2014-08-13 17:38:03.113764 7fe9bae887a0 20 read_log 194416'18802 (0'0) promote 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by osd.356.0:2171 2014-08-13 15:29:08.363714 2014-08-13 17:38:03.113776 7fe9bae887a0 20 read_log 194416'18803 (194416'18802) modify 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by client.2441958.0:9757 2014-08-13 15:29:08.356283 2014-08-13 17:38:03.113788 7fe9bae887a0 20 read_log 194416'18804 (0'0) promote 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by osd.356.0:2183 2014-08-13 15:29:13.558051 2014-08-13 17:38:03.113818 7fe9bae887a0 20 read_log 194416'18805 (194416'18804) modify 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by client.2441958.0:11063 2014-08-13 15:29:13.549874 2014-08-13 17:38:03.113833 7fe9bae887a0 20 read_log 194416'18806 (0'0) promote 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by osd.356.0:2189 2014-08-13 15:29:18.883994 2014-08-13 17:38:03.113845 7fe9bae887a0 20 read_log 194416'18807 (194416'18806) modify 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by client.2441958.0:12291 2014-08-13 15:29:18.874106 2014-08-13 17:38:03.113858 7fe9bae887a0 20 read_log 194416'18808 (0'0) promote e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by osd.356.0:2197 2014-08-13 15:29:20.731051 2014-08-13 17:38:03.113870 7fe9bae887a0 20 read_log 194416'18809 (194416'18808) modify e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by client.2441958.0:12738 2014-08-13 15:29:20.723333 2014-08-13 17:38:03.113882 7fe9bae887a0 20 read_log 194416'18810 (0'0) promote 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by osd.356.0:2201 2014-08-13 15:29:21.357739 2014-08-13 17:38:03.113895 7fe9bae887a0 20 read_log 194416'18811 (194416'18810) modify 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by client.2441958.0:12879 2014-08-13 15:29:21.350712 2014-08-13 17:38:03.113910 7fe9bae887a0 20 read_log 194416'18812 (0'0) promote edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by osd.356.0:2211 2014-08-13 15:29:26.439987 2014-08-13 17:38:03.113922 7fe9bae887a0 20 read_log 194416'18813 (194416'18812) modify edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by client.2441958.0:14059 2014-08-13 15:29:26.429216 2014-08-13 17:38:03.113936 7fe9bae887a0 20 read_log 194416'18814 (0'0) promote 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by osd.356.0:2215 2014-08-13 15:29:28.359569 2014-08-13 17:38:03.113950 7fe9bae887a0 20 read_log 194416'18815 (194416'18814) modify 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by client.2441958.0:14537 2014-08-13 15:29:28.349970 2014-08-13 17:38:03.113970 7fe9bae887a0 20 read_log 194416'18816 (0'0) promote 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by osd.356.0:2217 2014-08-13 15:29:32.179418 2014-08-13 17:38:03.113985 7fe9bae887a0 20 read_log 194416'18817 (194416'18816) modify 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by client.2441958.0:15404 2014-08-13 15:29:32.170808 2014-08-13 17:38:03.113997 7fe9bae887a0 20 read_log 194416'18818 (0'0) promote 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by osd.356.0:2227 2014-08-13 15:29:49.344089 2014-08-13 17:38:03.114009 7fe9bae887a0 20 read_log 194416'18819 (194416'18818) modify 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by client.2441958.0:17140 2014-08-13 15:29:49.335836 2014-08-13 17:38:03.114024 7fe9bae887a0 20 read_log 194416'18820 (0'0) promote 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by osd.356.0:2235 2014-08-13 15:30:00.051696 2014-08-13 17:38:03.114036 7fe9bae887a0 20 read_log 194416'18821 (194416'18820) modify 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by client.2441958.0:17900 2014-08-13 15:30:00.042666 2014-08-13 17:38:03.114069 7fe9bae887a0 20 read_log 194422'18822 (0'0) promote 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by osd.356.0:2247 2014-08-13 15:32:20.656769 2014-08-13 17:38:03.114082 7fe9bae887a0 20 read_log 194422'18823 (194422'18822) modify 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by client.1148203.0:1343 2014-08-13 15:32:20.648957 2014-08-13 17:38:03.114096 7fe9bae887a0 20 read_log 194422'18824 (0'0) promote a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by osd.356.0:2261 2014-08-13 15:32:27.177459 2014-08-13 17:38:03.114108 7fe9bae887a0 20 read_log 194422'18825 (194422'18824) modify a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by client.1148203.0:3021 2014-08-13 15:32:27.170065 2014-08-13 17:38:03.114122 7fe9bae887a0 20 read_log 194422'18826 (0'0) promote 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by osd.356.0:2263 2014-08-13 15:32:27.840527 2014-08-13 17:38:03.114134 7fe9bae887a0 20 read_log 194422'18827 (194422'18826) modify 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by client.1148203.0:3196 2014-08-13 15:32:27.833290 2014-08-13 17:38:03.114146 7fe9bae887a0 20 read_log 194422'18828 (0'0) promote 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by osd.356.0:2271 2014-08-13 15:32:34.011483 2014-08-13 17:38:03.114164 7fe9bae887a0 20 read_log 194422'18829 (194422'18828) modify 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by client.1148203.0:4783 2014-08-13 15:32:34.003860 2014-08-13 17:38:03.114179 7fe9bae887a0 20 read_log 194422'18830 (0'0) promote cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by osd.356.0:2273 2014-08-13 15:32:34.647899 2014-08-13 17:38:03.114191 7fe9bae887a0 20 read_log 194422'18831 (194422'18830) modify cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by client.1148203.0:4950 2014-08-13 15:32:34.641471 2014-08-13 17:38:03.114203 7fe9bae887a0 20 read_log 194422'18832 (0'0) promote e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by osd.356.0:2283 2014-08-13 15:32:38.043010 2014-08-13 17:38:03.114217 7fe9bae887a0 20 read_log 194422'18833 (194422'18832) modify e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by client.1148203.0:5620 2014-08-13 15:32:38.034770 2014-08-13 17:38:03.114230 7fe9bae887a0 20 read_log 194422'18834 (0'0) promote bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by osd.356.0:2285 2014-08-13 15:32:38.054487 2014-08-13 17:38:03.114244 7fe9bae887a0 20 read_log 194422'18835 (194422'18834) modify bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by client.1148203.0:5622 2014-08-13 15:32:38.045437 2014-08-13 17:38:03.114266 7fe9bae887a0 20 read_log 194422'18836 (0'0) promote 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by osd.356.0:2295 2014-08-13 15:32:46.093217 2014-08-13 17:38:03.114280 7fe9bae887a0 20 read_log 194422'18837 (194422'18836) modify 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by client.1148203.0:7093 2014-08-13 15:32:46.086223 2014-08-13 17:38:03.114293 7fe9bae887a0 20 read_log 194422'18838 (0'0) promote fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by osd.356.0:2297 2014-08-13 15:32:46.389557 2014-08-13 17:38:03.114305 7fe9bae887a0 20 read_log 194422'18839 (194422'18838) modify fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by client.1148203.0:7140 2014-08-13 15:32:46.380212 2014-08-13 17:38:03.114319 7fe9bae887a0 20 read_log 194422'18840 (0'0) promote e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by osd.356.0:2319 2014-08-13 15:33:06.361050 2014-08-13 17:38:03.114332 7fe9bae887a0 20 read_log 194422'18841 (194422'18840) modify e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by client.1148203.0:9693 2014-08-13 15:33:06.353096 2014-08-13 17:38:03.114344 7fe9bae887a0 20 read_log 194422'18842 (0'0) promote 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by osd.356.0:2323 2014-08-13 15:33:11.972044 2014-08-13 17:38:03.114356 7fe9bae887a0 20 read_log 194422'18843 (194422'18842) modify 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by client.1148203.0:10072 2014-08-13 15:33:11.963605 2014-08-13 17:38:03.114368 7fe9bae887a0 20 read_log 194422'18844 (0'0) promote aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by osd.356.0:2325 2014-08-13 15:33:17.566159 2014-08-13 17:38:03.114380 7fe9bae887a0 20 read_log 194422'18845 (194422'18844) modify aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by client.1148203.0:10269 2014-08-13 15:33:17.557449 2014-08-13 17:38:03.114397 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.114800 7fe9bae887a0 10 read_log checking for missing items over interval (194217'18730,194422'18845] 2014-08-13 17:38:03.114848 7fe9bae887a0 15 read_log missing 194422'18845 (194422'18844) modify aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by client.1148203.0:10269 2014-08-13 15:33:17.557449 2014-08-13 17:38:03.114890 7fe9bae887a0 15 read_log missing 194422'18843 (194422'18842) modify 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by client.1148203.0:10072 2014-08-13 15:33:11.963605 2014-08-13 17:38:03.114927 7fe9bae887a0 15 read_log missing 194422'18841 (194422'18840) modify e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by client.1148203.0:9693 2014-08-13 15:33:06.353096 2014-08-13 17:38:03.114964 7fe9bae887a0 15 read_log missing 194422'18839 (194422'18838) modify fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by client.1148203.0:7140 2014-08-13 15:32:46.380212 2014-08-13 17:38:03.115001 7fe9bae887a0 15 read_log missing 194422'18837 (194422'18836) modify 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by client.1148203.0:7093 2014-08-13 15:32:46.086223 2014-08-13 17:38:03.115037 7fe9bae887a0 15 read_log missing 194422'18835 (194422'18834) modify bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by client.1148203.0:5622 2014-08-13 15:32:38.045437 2014-08-13 17:38:03.115073 7fe9bae887a0 15 read_log missing 194422'18833 (194422'18832) modify e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by client.1148203.0:5620 2014-08-13 15:32:38.034770 2014-08-13 17:38:03.115110 7fe9bae887a0 15 read_log missing 194422'18831 (194422'18830) modify cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by client.1148203.0:4950 2014-08-13 15:32:34.641471 2014-08-13 17:38:03.115146 7fe9bae887a0 15 read_log missing 194422'18829 (194422'18828) modify 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by client.1148203.0:4783 2014-08-13 15:32:34.003860 2014-08-13 17:38:03.115190 7fe9bae887a0 15 read_log missing 194422'18827 (194422'18826) modify 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by client.1148203.0:3196 2014-08-13 15:32:27.833290 2014-08-13 17:38:03.115228 7fe9bae887a0 15 read_log missing 194422'18825 (194422'18824) modify a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by client.1148203.0:3021 2014-08-13 15:32:27.170065 2014-08-13 17:38:03.115264 7fe9bae887a0 15 read_log missing 194422'18823 (194422'18822) modify 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by client.1148203.0:1343 2014-08-13 15:32:20.648957 2014-08-13 17:38:03.115301 7fe9bae887a0 15 read_log missing 194416'18821 (194416'18820) modify 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by client.2441958.0:17900 2014-08-13 15:30:00.042666 2014-08-13 17:38:03.115338 7fe9bae887a0 15 read_log missing 194416'18819 (194416'18818) modify 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by client.2441958.0:17140 2014-08-13 15:29:49.335836 2014-08-13 17:38:03.115374 7fe9bae887a0 15 read_log missing 194416'18817 (194416'18816) modify 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by client.2441958.0:15404 2014-08-13 15:29:32.170808 2014-08-13 17:38:03.115411 7fe9bae887a0 15 read_log missing 194416'18815 (194416'18814) modify 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by client.2441958.0:14537 2014-08-13 15:29:28.349970 2014-08-13 17:38:03.115448 7fe9bae887a0 15 read_log missing 194416'18813 (194416'18812) modify edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by client.2441958.0:14059 2014-08-13 15:29:26.429216 2014-08-13 17:38:03.115485 7fe9bae887a0 15 read_log missing 194416'18811 (194416'18810) modify 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by client.2441958.0:12879 2014-08-13 15:29:21.350712 2014-08-13 17:38:03.115523 7fe9bae887a0 15 read_log missing 194416'18809 (194416'18808) modify e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by client.2441958.0:12738 2014-08-13 15:29:20.723333 2014-08-13 17:38:03.115559 7fe9bae887a0 15 read_log missing 194416'18807 (194416'18806) modify 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by client.2441958.0:12291 2014-08-13 15:29:18.874106 2014-08-13 17:38:03.115595 7fe9bae887a0 15 read_log missing 194416'18805 (194416'18804) modify 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by client.2441958.0:11063 2014-08-13 15:29:13.549874 2014-08-13 17:38:03.115631 7fe9bae887a0 15 read_log missing 194416'18803 (194416'18802) modify 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by client.2441958.0:9757 2014-08-13 15:29:08.356283 2014-08-13 17:38:03.115666 7fe9bae887a0 15 read_log missing 194416'18801 (194416'18800) modify fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by client.2441958.0:8955 2014-08-13 15:29:05.276987 2014-08-13 17:38:03.115702 7fe9bae887a0 15 read_log missing 194416'18799 (194416'18798) modify 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by client.2441958.0:8213 2014-08-13 15:29:02.324036 2014-08-13 17:38:03.115737 7fe9bae887a0 15 read_log missing 194416'18797 (194416'18796) modify 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by client.2441958.0:7852 2014-08-13 15:29:00.965404 2014-08-13 17:38:03.115773 7fe9bae887a0 15 read_log missing 194416'18795 (194416'18794) modify d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by client.2441958.0:7616 2014-08-13 15:29:00.080339 2014-08-13 17:38:03.115809 7fe9bae887a0 15 read_log missing 194416'18793 (194416'18792) modify 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by client.2441958.0:6743 2014-08-13 15:28:56.811884 2014-08-13 17:38:03.115845 7fe9bae887a0 15 read_log missing 194416'18791 (194416'18790) modify c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by client.2441958.0:6123 2014-08-13 15:28:54.334808 2014-08-13 17:38:03.115880 7fe9bae887a0 15 read_log missing 194416'18789 (194416'18788) modify 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by client.2441958.0:5572 2014-08-13 15:28:52.060924 2014-08-13 17:38:03.115917 7fe9bae887a0 15 read_log missing 194416'18787 (194416'18786) modify f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by client.2441958.0:5001 2014-08-13 15:28:49.888965 2014-08-13 17:38:03.115954 7fe9bae887a0 15 read_log missing 194416'18785 (194416'18784) modify 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by client.2441958.0:3307 2014-08-13 15:28:43.228885 2014-08-13 17:38:03.115991 7fe9bae887a0 15 read_log missing 194416'18783 (194416'18782) modify b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by client.2441958.0:1599 2014-08-13 15:28:36.774701 2014-08-13 17:38:03.116028 7fe9bae887a0 15 read_log missing 194416'18781 (194416'18780) modify 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by client.2441958.0:719 2014-08-13 15:28:33.326094 2014-08-13 17:38:03.116065 7fe9bae887a0 15 read_log missing 194416'18779 (194416'18778) modify 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by client.2441958.0:369 2014-08-13 15:28:31.761856 2014-08-13 17:38:03.116106 7fe9bae887a0 15 read_log missing 194416'18776 (194416'18776) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:53:06.322323_2014-08-13 15:28:31.768122/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 17:38:03.116142 7fe9bae887a0 15 read_log missing 194397'18775 (194397'18774) modify 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by client.2235069.0:933 2014-08-12 20:53:06.315038 2014-08-13 17:38:03.116186 7fe9bae887a0 15 read_log missing 194393'18771 (194393'18770) modify 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by client.1064095.0:459 2014-08-12 20:47:27.874894 2014-08-13 17:38:03.116229 7fe9bae887a0 15 read_log missing 194383'18767 (194383'18766) modify 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by client.2231190.0:7571 2014-08-12 20:34:30.693282 2014-08-13 17:38:03.116265 7fe9bae887a0 15 read_log missing 194383'18765 (194383'18764) modify c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by client.2231190.0:7341 2014-08-12 20:34:24.392949 2014-08-13 17:38:03.116302 7fe9bae887a0 15 read_log missing 194383'18763 (194383'18762) modify b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by client.2231190.0:6721 2014-08-12 20:34:07.583644 2014-08-13 17:38:03.116339 7fe9bae887a0 15 read_log missing 194383'18761 (194383'18760) modify 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by client.2231190.0:5769 2014-08-12 20:34:00.019150 2014-08-13 17:38:03.116377 7fe9bae887a0 15 read_log missing 194377'18759 (194377'18758) modify 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by client.2231190.0:4599 2014-08-12 20:33:17.892578 2014-08-13 17:38:03.116414 7fe9bae887a0 15 read_log missing 194377'18757 (194377'18756) modify 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by client.2231190.0:3645 2014-08-12 20:33:12.080752 2014-08-13 17:38:03.116451 7fe9bae887a0 15 read_log missing 194377'18755 (194377'18754) modify 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by client.2231190.0:2731 2014-08-12 20:33:08.231506 2014-08-13 17:38:03.116488 7fe9bae887a0 15 read_log missing 194370'18753 (194370'18752) modify 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by client.1508531.0:12825 2014-08-12 20:31:17.712236 2014-08-13 17:38:03.116525 7fe9bae887a0 15 read_log missing 194370'18751 (194370'18750) modify e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by client.1508531.0:12709 2014-08-12 20:31:16.602273 2014-08-13 17:38:03.116562 7fe9bae887a0 15 read_log missing 194370'18749 (194370'18748) modify 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by client.1508531.0:11367 2014-08-12 20:31:07.508175 2014-08-13 17:38:03.116598 7fe9bae887a0 15 read_log missing 194370'18747 (194370'18746) modify a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by client.1508531.0:10735 2014-08-12 20:31:03.714078 2014-08-13 17:38:03.116636 7fe9bae887a0 15 read_log missing 194370'18745 (194370'18744) modify 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by client.1508531.0:10351 2014-08-12 20:31:01.318249 2014-08-13 17:38:03.116673 7fe9bae887a0 15 read_log missing 194370'18743 (194370'18742) modify 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by client.1508531.0:10272 2014-08-12 20:31:00.849645 2014-08-13 17:38:03.116709 7fe9bae887a0 15 read_log missing 194370'18741 (194370'18740) modify 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by client.1508531.0:10082 2014-08-12 20:30:59.719353 2014-08-13 17:38:03.116746 7fe9bae887a0 15 read_log missing 194370'18739 (194370'18738) modify c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by client.1508531.0:8257 2014-08-12 20:30:48.939251 2014-08-13 17:38:03.116783 7fe9bae887a0 15 read_log missing 194370'18737 (194370'18736) modify 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by client.1508531.0:5485 2014-08-12 20:30:35.334220 2014-08-13 17:38:03.116820 7fe9bae887a0 15 read_log missing 194370'18735 (194370'18734) modify b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by client.1508531.0:4218 2014-08-12 20:30:30.156160 2014-08-13 17:38:03.116858 7fe9bae887a0 15 read_log missing 194370'18733 (194370'18732) modify efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by client.1508531.0:2038 2014-08-12 20:30:22.049716 2014-08-13 17:38:03.116895 7fe9bae887a0 15 read_log missing 194217'18731 (194217'18730) modify 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by client.1487927.0:129 2014-08-12 17:25:03.805212 2014-08-13 17:38:03.116906 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.116936 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 17:38:03.116953 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.013896 0 0.000000 2014-08-13 17:38:03.116965 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 17:38:03.116974 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194478 2014-08-13 17:38:03.116984 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] log((192718'15726,194422'18845], crt=194422'18845) 2014-08-13 17:38:03.116994 7fe9bae887a0 10 osd.4 194478 pgid 24.3bf coll 24.3bf_head 2014-08-13 17:38:03.117103 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.3bf 2014-08-13 17:38:03.117119 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.117162 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3bf(unlocked)] enter Initial 2014-08-13 17:38:03.117174 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.3bf(unlocked)] enter NotTrimming 2014-08-13 17:38:03.117285 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.117397 7fe9bae887a0 20 read_log 192834'12053 (0'0) promote 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.318.0:12439 2014-08-11 15:23:50.920416 2014-08-13 17:38:03.117427 7fe9bae887a0 20 read_log 192834'12054 (192834'12053) modify 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by client.933448.0:3248 2014-08-11 15:23:50.912591 2014-08-13 17:38:03.117443 7fe9bae887a0 20 read_log 192834'12055 (0'0) promote acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by osd.318.0:12441 2014-08-11 15:23:51.644583 2014-08-13 17:38:03.117456 7fe9bae887a0 20 read_log 192834'12056 (192834'12055) modify acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.933448.0:3405 2014-08-11 15:23:51.635870 2014-08-13 17:38:03.117468 7fe9bae887a0 20 read_log 192834'12073 (0'0) promote e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by osd.318.0:12459 2014-08-11 15:23:58.266628 2014-08-13 17:38:03.117480 7fe9bae887a0 20 read_log 192834'12074 (192834'12073) modify e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.933448.0:4708 2014-08-11 15:23:58.259895 2014-08-13 17:38:03.117494 7fe9bae887a0 20 read_log 192834'12081 (0'0) promote 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by osd.318.0:12467 2014-08-11 15:24:00.277004 2014-08-13 17:38:03.117506 7fe9bae887a0 20 read_log 192834'12082 (192834'12081) modify 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.933448.0:5165 2014-08-11 15:24:00.259614 2014-08-13 17:38:03.117519 7fe9bae887a0 20 read_log 192834'12089 (0'0) promote 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by osd.318.0:12475 2014-08-11 15:24:06.292097 2014-08-13 17:38:03.117531 7fe9bae887a0 20 read_log 192834'12090 (192834'12089) modify 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.933448.0:6665 2014-08-11 15:24:06.283874 2014-08-13 17:38:03.117543 7fe9bae887a0 20 read_log 192834'12135 (0'0) promote 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by osd.318.0:12521 2014-08-11 15:24:17.688344 2014-08-13 17:38:03.117555 7fe9bae887a0 20 read_log 192834'12136 (192834'12135) modify 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.933448.0:9136 2014-08-11 15:24:17.680802 2014-08-13 17:38:03.117567 7fe9bae887a0 20 read_log 192834'12143 (0'0) promote d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by osd.318.0:12529 2014-08-11 15:24:19.375199 2014-08-13 17:38:03.117579 7fe9bae887a0 20 read_log 192834'12144 (192834'12143) modify d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.933448.0:9569 2014-08-11 15:24:19.367033 2014-08-13 17:38:03.117591 7fe9bae887a0 20 read_log 192834'12153 (0'0) promote 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by osd.318.0:12539 2014-08-11 15:24:21.944902 2014-08-13 17:38:03.117603 7fe9bae887a0 20 read_log 192834'12154 (192834'12153) modify 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.933448.0:10165 2014-08-11 15:24:21.936874 2014-08-13 17:38:03.117615 7fe9bae887a0 20 read_log 192834'12157 (0'0) promote b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.318.0:12543 2014-08-11 15:24:24.512579 2014-08-13 17:38:03.117627 7fe9bae887a0 20 read_log 192834'12158 (192834'12157) modify b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by client.933448.0:10625 2014-08-11 15:24:24.503866 2014-08-13 17:38:03.117640 7fe9bae887a0 20 read_log 192834'12161 (0'0) promote 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by osd.318.0:12547 2014-08-11 15:24:25.327545 2014-08-13 17:38:03.117652 7fe9bae887a0 20 read_log 192834'12162 (192834'12161) modify 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.933448.0:10809 2014-08-11 15:24:25.315651 2014-08-13 17:38:03.117664 7fe9bae887a0 20 read_log 192834'12193 (0'0) promote c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by osd.318.0:12579 2014-08-11 15:24:35.711707 2014-08-13 17:38:03.117702 7fe9bae887a0 20 read_log 192834'12194 (192834'12193) modify c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.933448.0:13169 2014-08-11 15:24:35.701430 2014-08-13 17:38:03.117718 7fe9bae887a0 20 read_log 192834'12199 (0'0) promote 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by osd.318.0:12585 2014-08-11 15:24:36.893864 2014-08-13 17:38:03.117731 7fe9bae887a0 20 read_log 192834'12200 (192834'12199) modify 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.933448.0:13395 2014-08-11 15:24:36.883668 2014-08-13 17:38:03.117743 7fe9bae887a0 20 read_log 192834'12203 (0'0) promote e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.318.0:12589 2014-08-11 15:24:39.199826 2014-08-13 17:38:03.117755 7fe9bae887a0 20 read_log 192834'12204 (192834'12203) modify e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by client.933448.0:13579 2014-08-11 15:24:39.190927 2014-08-13 17:38:03.117767 7fe9bae887a0 20 read_log 192834'12211 (0'0) promote cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by osd.318.0:12597 2014-08-11 15:24:47.290157 2014-08-13 17:38:03.117779 7fe9bae887a0 20 read_log 192834'12212 (192834'12211) modify cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.933448.0:14816 2014-08-11 15:24:47.280651 2014-08-13 17:38:03.117791 7fe9bae887a0 20 read_log 192834'12237 (0'0) promote 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.318.0:12623 2014-08-11 15:24:59.435234 2014-08-13 17:38:03.117803 7fe9bae887a0 20 read_log 192834'12238 (192834'12237) modify 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by client.933448.0:17367 2014-08-11 15:24:59.427278 2014-08-13 17:38:03.117815 7fe9bae887a0 20 read_log 192834'12239 (0'0) promote dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.318.0:12625 2014-08-11 15:24:59.815897 2014-08-13 17:38:03.117827 7fe9bae887a0 20 read_log 192834'12240 (192834'12239) modify dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by client.933448.0:17468 2014-08-11 15:24:59.808384 2014-08-13 17:38:03.117839 7fe9bae887a0 20 read_log 192834'12251 (0'0) promote d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.318.0:12637 2014-08-11 15:25:06.224619 2014-08-13 17:38:03.117851 7fe9bae887a0 20 read_log 192834'12252 (192834'12251) modify d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by client.933448.0:18660 2014-08-11 15:25:06.216566 2014-08-13 17:38:03.117863 7fe9bae887a0 20 read_log 192834'12257 (0'0) promote 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by osd.318.0:12643 2014-08-11 15:25:07.587764 2014-08-13 17:38:03.117875 7fe9bae887a0 20 read_log 192834'12258 (192834'12257) modify 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.933448.0:18939 2014-08-11 15:25:07.578830 2014-08-13 17:38:03.117887 7fe9bae887a0 20 read_log 192834'12261 (0'0) promote f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by osd.318.0:12647 2014-08-11 15:25:08.896232 2014-08-13 17:38:03.117900 7fe9bae887a0 20 read_log 192834'12262 (192834'12261) modify f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.933448.0:19172 2014-08-11 15:25:08.888470 2014-08-13 17:38:03.117913 7fe9bae887a0 20 read_log 192834'12277 (0'0) promote 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.318.0:12663 2014-08-11 15:25:16.416550 2014-08-13 17:38:03.117924 7fe9bae887a0 20 read_log 192834'12278 (192834'12277) modify 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by client.933448.0:20083 2014-08-11 15:25:16.408423 2014-08-13 17:38:03.117936 7fe9bae887a0 20 read_log 192834'12283 (0'0) promote 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.318.0:12669 2014-08-11 15:25:21.145753 2014-08-13 17:38:03.117956 7fe9bae887a0 20 read_log 192834'12284 (192834'12283) modify 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by client.933448.0:20489 2014-08-11 15:25:21.135926 2014-08-13 17:38:03.117971 7fe9bae887a0 20 read_log 192834'12325 (192834'12012) modify e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by client.933448.0:22806 2014-08-11 15:25:38.665654 2014-08-13 17:38:03.117983 7fe9bae887a0 20 read_log 192834'12326 (192834'12014) modify 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by client.933448.0:22864 2014-08-11 15:25:38.727873 2014-08-13 17:38:03.117995 7fe9bae887a0 20 read_log 192834'12346 (192834'12054) modify 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by client.933448.0:25539 2014-08-11 15:25:42.186818 2014-08-13 17:38:03.118007 7fe9bae887a0 20 read_log 192834'12347 (192834'12056) modify acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.933448.0:25696 2014-08-11 15:25:42.346335 2014-08-13 17:38:03.118020 7fe9bae887a0 20 read_log 192834'12356 (192834'12074) modify e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.933448.0:26999 2014-08-11 15:25:43.990485 2014-08-13 17:38:03.118073 7fe9bae887a0 20 read_log 192834'12360 (192834'12082) modify 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.933448.0:27456 2014-08-11 15:25:44.517265 2014-08-13 17:38:03.118090 7fe9bae887a0 20 read_log 192834'12364 (192834'12090) modify 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.933448.0:28956 2014-08-11 15:25:46.877125 2014-08-13 17:38:03.118103 7fe9bae887a0 20 read_log 192834'12387 (192834'12136) modify 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.933448.0:31427 2014-08-11 15:25:50.050318 2014-08-13 17:38:03.118115 7fe9bae887a0 20 read_log 192834'12391 (192834'12144) modify d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.933448.0:31860 2014-08-11 15:25:50.702295 2014-08-13 17:38:03.118129 7fe9bae887a0 20 read_log 192834'12396 (192834'12154) modify 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.933448.0:32456 2014-08-11 15:25:51.836699 2014-08-13 17:38:03.118142 7fe9bae887a0 20 read_log 192834'12398 (192834'12158) modify b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by client.933448.0:32916 2014-08-11 15:25:52.564423 2014-08-13 17:38:03.118163 7fe9bae887a0 20 read_log 192834'12400 (192834'12162) modify 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.933448.0:33100 2014-08-11 15:25:52.847362 2014-08-13 17:38:03.118179 7fe9bae887a0 20 read_log 192834'12416 (192834'12194) modify c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.933448.0:35460 2014-08-11 15:25:56.311597 2014-08-13 17:38:03.118191 7fe9bae887a0 20 read_log 192834'12419 (192834'12200) modify 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.933448.0:35686 2014-08-11 15:25:56.681245 2014-08-13 17:38:03.118204 7fe9bae887a0 20 read_log 192834'12421 (192834'12204) modify e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by client.933448.0:35870 2014-08-11 15:25:56.892525 2014-08-13 17:38:03.118219 7fe9bae887a0 20 read_log 192834'12425 (192834'12212) modify cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.933448.0:37107 2014-08-11 15:25:58.943170 2014-08-13 17:38:03.118231 7fe9bae887a0 20 read_log 192834'12438 (192834'12238) modify 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by client.933448.0:39658 2014-08-11 15:26:02.013690 2014-08-13 17:38:03.118245 7fe9bae887a0 20 read_log 192834'12439 (192834'12240) modify dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by client.933448.0:39759 2014-08-11 15:26:02.064344 2014-08-13 17:38:03.118258 7fe9bae887a0 20 read_log 192834'12445 (192834'12252) modify d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by client.933448.0:40951 2014-08-11 15:26:02.796460 2014-08-13 17:38:03.118282 7fe9bae887a0 20 read_log 192834'12448 (192834'12258) modify 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.933448.0:41230 2014-08-11 15:26:02.877142 2014-08-13 17:38:03.118297 7fe9bae887a0 20 read_log 192834'12450 (192834'12262) modify f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.933448.0:41463 2014-08-11 15:26:02.949849 2014-08-13 17:38:03.118309 7fe9bae887a0 20 read_log 192834'12458 (192834'12278) modify 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by client.933448.0:42374 2014-08-11 15:26:03.258454 2014-08-13 17:38:03.118321 7fe9bae887a0 20 read_log 192834'12461 (192834'12284) modify 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by client.933448.0:42780 2014-08-11 15:26:03.384766 2014-08-13 17:38:03.118333 7fe9bae887a0 20 read_log 192834'12485 (0'0) promote f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.318.0:12871 2014-08-11 15:26:15.405606 2014-08-13 17:38:03.118345 7fe9bae887a0 20 read_log 192834'12486 (192834'12485) modify f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by client.1883634.0:247 2014-08-11 15:26:15.397583 2014-08-13 17:38:03.118358 7fe9bae887a0 20 read_log 192834'12521 (0'0) promote f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by osd.318.0:12907 2014-08-11 15:26:27.880871 2014-08-13 17:38:03.118371 7fe9bae887a0 20 read_log 192834'12522 (192834'12521) modify f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.1883634.0:3684 2014-08-11 15:26:27.873348 2014-08-13 17:38:03.118383 7fe9bae887a0 20 read_log 192834'12523 (0'0) promote c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by osd.318.0:12909 2014-08-11 15:26:28.014507 2014-08-13 17:38:03.118395 7fe9bae887a0 20 read_log 192834'12524 (192834'12523) modify c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.1883634.0:3721 2014-08-11 15:26:28.006149 2014-08-13 17:38:03.118407 7fe9bae887a0 20 read_log 192834'12527 (0'0) promote f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.318.0:12913 2014-08-11 15:26:30.069205 2014-08-13 17:38:03.118420 7fe9bae887a0 20 read_log 192834'12528 (192834'12527) modify f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by client.1883634.0:4303 2014-08-11 15:26:30.061797 2014-08-13 17:38:03.118432 7fe9bae887a0 20 read_log 192834'12547 (0'0) promote d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by osd.318.0:12933 2014-08-11 15:26:34.676141 2014-08-13 17:38:03.118444 7fe9bae887a0 20 read_log 192834'12548 (192834'12547) modify d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.1883634.0:5514 2014-08-11 15:26:34.667882 2014-08-13 17:38:03.118456 7fe9bae887a0 20 read_log 192834'12585 (0'0) promote 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by osd.318.0:12971 2014-08-11 15:26:46.686219 2014-08-13 17:38:03.118471 7fe9bae887a0 20 read_log 192834'12586 (192834'12585) modify 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.1883634.0:8514 2014-08-11 15:26:46.676044 2014-08-13 17:38:03.118484 7fe9bae887a0 20 read_log 192834'12595 (0'0) promote d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by osd.318.0:12981 2014-08-11 15:26:48.358336 2014-08-13 17:38:03.118497 7fe9bae887a0 20 read_log 192834'12596 (192834'12595) modify d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.1883634.0:8893 2014-08-11 15:26:48.347449 2014-08-13 17:38:03.118509 7fe9bae887a0 20 read_log 192834'12615 (0'0) promote a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by osd.318.0:13001 2014-08-11 15:26:55.553568 2014-08-13 17:38:03.118521 7fe9bae887a0 20 read_log 192834'12616 (192834'12615) modify a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.1883634.0:10541 2014-08-11 15:26:55.544157 2014-08-13 17:38:03.118543 7fe9bae887a0 20 read_log 192834'12619 (0'0) promote 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by osd.318.0:13005 2014-08-11 15:26:55.729145 2014-08-13 17:38:03.118556 7fe9bae887a0 20 read_log 192834'12620 (192834'12619) modify 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.1883634.0:10585 2014-08-11 15:26:55.722330 2014-08-13 17:38:03.118568 7fe9bae887a0 20 read_log 192834'12629 (0'0) promote 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.318.0:13015 2014-08-11 15:26:59.141676 2014-08-13 17:38:03.118580 7fe9bae887a0 20 read_log 192834'12630 (192834'12629) modify 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by client.1883634.0:11277 2014-08-11 15:26:59.133717 2014-08-13 17:38:03.118592 7fe9bae887a0 20 read_log 192834'12645 (0'0) promote 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by osd.318.0:13031 2014-08-11 15:27:03.092882 2014-08-13 17:38:03.118604 7fe9bae887a0 20 read_log 192834'12646 (192834'12645) modify 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.1883634.0:12225 2014-08-11 15:27:03.081015 2014-08-13 17:38:03.118617 7fe9bae887a0 20 read_log 192834'12665 (0'0) promote c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by osd.318.0:13051 2014-08-11 15:27:09.362379 2014-08-13 17:38:03.118629 7fe9bae887a0 20 read_log 192834'12666 (192834'12665) modify c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.1883634.0:13398 2014-08-11 15:27:09.352997 2014-08-13 17:38:03.118641 7fe9bae887a0 20 read_log 192834'12667 (0'0) promote ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.318.0:13053 2014-08-11 15:27:09.983996 2014-08-13 17:38:03.118653 7fe9bae887a0 20 read_log 192834'12668 (192834'12667) modify ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by client.1883634.0:13528 2014-08-11 15:27:09.975506 2014-08-13 17:38:03.118666 7fe9bae887a0 20 read_log 192834'12671 (0'0) promote 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by osd.318.0:13057 2014-08-11 15:27:11.563098 2014-08-13 17:38:03.118680 7fe9bae887a0 20 read_log 192834'12672 (192834'12671) modify 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.1883634.0:14016 2014-08-11 15:27:11.555402 2014-08-13 17:38:03.118692 7fe9bae887a0 20 read_log 192834'12681 (0'0) promote 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by osd.318.0:13067 2014-08-11 15:27:15.543281 2014-08-13 17:38:03.118704 7fe9bae887a0 20 read_log 192834'12682 (192834'12681) modify 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.1883634.0:14825 2014-08-11 15:27:15.533733 2014-08-13 17:38:03.118716 7fe9bae887a0 20 read_log 192834'12689 (0'0) promote 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.318.0:13075 2014-08-11 15:27:17.770204 2014-08-13 17:38:03.118728 7fe9bae887a0 20 read_log 192834'12690 (192834'12689) modify 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by client.1883634.0:15227 2014-08-11 15:27:17.762603 2014-08-13 17:38:03.118741 7fe9bae887a0 20 read_log 192834'12721 (0'0) promote da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.318.0:13107 2014-08-11 15:27:28.369746 2014-08-13 17:38:03.118753 7fe9bae887a0 20 read_log 192834'12722 (192834'12721) modify da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by client.1883634.0:16588 2014-08-11 15:27:28.360738 2014-08-13 17:38:03.118765 7fe9bae887a0 20 read_log 192834'12725 (0'0) promote 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.318.0:13111 2014-08-11 15:27:28.780280 2014-08-13 17:38:03.118780 7fe9bae887a0 20 read_log 192834'12726 (192834'12725) modify 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by client.1883634.0:16680 2014-08-11 15:27:28.772338 2014-08-13 17:38:03.118800 7fe9bae887a0 20 read_log 192834'12731 (0'0) promote 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by osd.318.0:13117 2014-08-11 15:27:33.942922 2014-08-13 17:38:03.118815 7fe9bae887a0 20 read_log 192834'12732 (192834'12731) modify 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.1883634.0:17261 2014-08-11 15:27:33.933177 2014-08-13 17:38:03.118827 7fe9bae887a0 20 read_log 192834'12753 (0'0) promote b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by osd.318.0:13139 2014-08-11 15:27:46.480179 2014-08-13 17:38:03.118842 7fe9bae887a0 20 read_log 192834'12754 (192834'12753) modify b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.1883634.0:18563 2014-08-11 15:27:46.470567 2014-08-13 17:38:03.118854 7fe9bae887a0 20 read_log 192834'12769 (0'0) promote ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by osd.318.0:13155 2014-08-11 15:27:52.965688 2014-08-13 17:38:03.118868 7fe9bae887a0 20 read_log 192834'12770 (192834'12769) modify ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.1883634.0:19496 2014-08-11 15:27:52.951526 2014-08-13 17:38:03.118881 7fe9bae887a0 20 read_log 192834'12789 (0'0) promote fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by osd.318.0:13175 2014-08-11 15:28:02.955588 2014-08-13 17:38:03.118895 7fe9bae887a0 20 read_log 192834'12790 (192834'12789) modify fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.1883634.0:21127 2014-08-11 15:28:02.944261 2014-08-13 17:38:03.118907 7fe9bae887a0 20 read_log 192834'12791 (0'0) promote 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by osd.318.0:13177 2014-08-11 15:28:04.991407 2014-08-13 17:38:03.118921 7fe9bae887a0 20 read_log 192834'12792 (192834'12791) modify 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.1883634.0:21696 2014-08-11 15:28:04.983250 2014-08-13 17:38:03.118935 7fe9bae887a0 20 read_log 192834'12811 (0'0) promote 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by osd.318.0:13197 2014-08-11 15:28:10.492758 2014-08-13 17:38:03.118947 7fe9bae887a0 20 read_log 192834'12812 (192834'12811) modify 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.1883634.0:22677 2014-08-11 15:28:10.482821 2014-08-13 17:38:03.118961 7fe9bae887a0 20 read_log 192834'12813 (0'0) promote f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by osd.318.0:13199 2014-08-11 15:28:10.732156 2014-08-13 17:38:03.118973 7fe9bae887a0 20 read_log 192834'12814 (192834'12813) modify f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.1883634.0:22720 2014-08-11 15:28:10.723655 2014-08-13 17:38:03.118987 7fe9bae887a0 20 read_log 192834'12817 (0'0) promote 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.318.0:13203 2014-08-11 15:28:10.750175 2014-08-13 17:38:03.118999 7fe9bae887a0 20 read_log 192834'12818 (192834'12817) modify 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by client.1883634.0:22723 2014-08-11 15:28:10.741813 2014-08-13 17:38:03.119013 7fe9bae887a0 20 read_log 192834'12838 (192834'12486) modify f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by client.1883634.0:23571 2014-08-11 15:28:17.143427 2014-08-13 17:38:03.119026 7fe9bae887a0 20 read_log 192834'12856 (192834'12522) modify f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.1883634.0:27008 2014-08-11 15:28:23.899618 2014-08-13 17:38:03.119040 7fe9bae887a0 20 read_log 192834'12857 (192834'12524) modify c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.1883634.0:27045 2014-08-11 15:28:23.935217 2014-08-13 17:38:03.119052 7fe9bae887a0 20 read_log 192834'12859 (192834'12528) modify f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by client.1883634.0:27627 2014-08-11 15:28:24.717120 2014-08-13 17:38:03.119071 7fe9bae887a0 20 read_log 192834'12869 (192834'12548) modify d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.1883634.0:28838 2014-08-11 15:28:26.361619 2014-08-13 17:38:03.119086 7fe9bae887a0 20 read_log 192834'12888 (192834'12586) modify 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.1883634.0:31838 2014-08-11 15:28:31.027454 2014-08-13 17:38:03.119098 7fe9bae887a0 20 read_log 192834'12893 (192834'12596) modify d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.1883634.0:32217 2014-08-11 15:28:31.516789 2014-08-13 17:38:03.119111 7fe9bae887a0 20 read_log 192834'12903 (192834'12616) modify a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.1883634.0:33865 2014-08-11 15:28:33.922460 2014-08-13 17:38:03.119125 7fe9bae887a0 20 read_log 192834'12905 (192834'12620) modify 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.1883634.0:33909 2014-08-11 15:28:33.994305 2014-08-13 17:38:03.119137 7fe9bae887a0 20 read_log 192834'12910 (192834'12630) modify 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by client.1883634.0:34601 2014-08-11 15:28:35.056361 2014-08-13 17:38:03.119149 7fe9bae887a0 20 read_log 192834'12920 (192834'12646) modify 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.1883634.0:35549 2014-08-11 15:28:36.821845 2014-08-13 17:38:03.119168 7fe9bae887a0 20 read_log 192834'12930 (192834'12666) modify c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.1883634.0:36722 2014-08-11 15:28:38.333998 2014-08-13 17:38:03.119181 7fe9bae887a0 20 read_log 192834'12931 (192834'12668) modify ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by client.1883634.0:36852 2014-08-11 15:28:38.491244 2014-08-13 17:38:03.119193 7fe9bae887a0 20 read_log 192834'12933 (192834'12672) modify 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.1883634.0:37340 2014-08-11 15:28:39.189103 2014-08-13 17:38:03.119205 7fe9bae887a0 20 read_log 192834'12938 (192834'12682) modify 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.1883634.0:38149 2014-08-11 15:28:40.287679 2014-08-13 17:38:03.119219 7fe9bae887a0 20 read_log 192834'12942 (192834'12690) modify 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by client.1883634.0:38551 2014-08-11 15:28:40.832874 2014-08-13 17:38:03.119232 7fe9bae887a0 20 read_log 192834'12958 (192834'12722) modify da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by client.1883634.0:39912 2014-08-11 15:28:42.750468 2014-08-13 17:38:03.119246 7fe9bae887a0 20 read_log 192834'12960 (192834'12726) modify 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by client.1883634.0:40004 2014-08-11 15:28:43.020209 2014-08-13 17:38:03.119258 7fe9bae887a0 20 read_log 192834'12963 (192834'12732) modify 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.1883634.0:40585 2014-08-11 15:28:44.023379 2014-08-13 17:38:03.119272 7fe9bae887a0 20 read_log 192834'12974 (192834'12754) modify b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.1883634.0:41887 2014-08-11 15:28:45.563270 2014-08-13 17:38:03.119284 7fe9bae887a0 20 read_log 192834'12982 (192834'12770) modify ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.1883634.0:42820 2014-08-11 15:28:46.368857 2014-08-13 17:38:03.119298 7fe9bae887a0 20 read_log 192834'12992 (192834'12790) modify fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.1883634.0:44451 2014-08-11 15:28:46.860992 2014-08-13 17:38:03.119311 7fe9bae887a0 20 read_log 192834'12993 (192834'12792) modify 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.1883634.0:45020 2014-08-11 15:28:47.124501 2014-08-13 17:38:03.119339 7fe9bae887a0 20 read_log 192834'13003 (192834'12812) modify 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.1883634.0:46001 2014-08-11 15:28:47.460485 2014-08-13 17:38:03.119364 7fe9bae887a0 20 read_log 192834'13004 (192834'12814) modify f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.1883634.0:46044 2014-08-11 15:28:47.473068 2014-08-13 17:38:03.119379 7fe9bae887a0 20 read_log 192834'13006 (192834'12818) modify 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by client.1883634.0:46047 2014-08-11 15:28:47.473955 2014-08-13 17:38:03.119391 7fe9bae887a0 20 read_log 192888'13018 (0'0) promote 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by osd.63.0:2341 2014-08-11 16:44:56.514349 2014-08-13 17:38:03.119403 7fe9bae887a0 20 read_log 192888'13019 (192888'13018) modify 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.1315259.0:381 2014-08-11 16:44:56.506779 2014-08-13 17:38:03.119415 7fe9bae887a0 20 read_log 192888'13020 (0'0) promote 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by osd.63.0:2350 2014-08-11 16:44:58.239975 2014-08-13 17:38:03.119428 7fe9bae887a0 20 read_log 192888'13021 (192888'13020) modify 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.1315259.0:779 2014-08-11 16:44:58.234225 2014-08-13 17:38:03.119440 7fe9bae887a0 20 read_log 192888'13024 (0'0) promote 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by osd.63.0:2354 2014-08-11 16:44:58.512734 2014-08-13 17:38:03.119452 7fe9bae887a0 20 read_log 192888'13025 (192888'13024) modify 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.1315259.0:852 2014-08-11 16:44:58.507292 2014-08-13 17:38:03.119464 7fe9bae887a0 20 read_log 192888'13028 (0'0) promote b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by osd.63.0:2358 2014-08-11 16:44:59.297270 2014-08-13 17:38:03.119476 7fe9bae887a0 20 read_log 192888'13029 (192888'13028) modify b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.1315259.0:1064 2014-08-11 16:44:59.291081 2014-08-13 17:38:03.119488 7fe9bae887a0 20 read_log 192888'13032 (0'0) promote dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by osd.63.0:2372 2014-08-11 16:45:03.242153 2014-08-13 17:38:03.119500 7fe9bae887a0 20 read_log 192888'13033 (192888'13032) modify dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.1315259.0:2100 2014-08-11 16:45:03.236033 2014-08-13 17:38:03.119513 7fe9bae887a0 20 read_log 192888'13038 (0'0) promote ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by osd.63.0:2380 2014-08-11 16:45:04.263051 2014-08-13 17:38:03.119525 7fe9bae887a0 20 read_log 192888'13039 (192888'13038) modify ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.1315259.0:2377 2014-08-11 16:45:04.257977 2014-08-13 17:38:03.119558 7fe9bae887a0 20 read_log 192888'13044 (0'0) promote 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by osd.63.0:2412 2014-08-11 16:45:14.210476 2014-08-13 17:38:03.119571 7fe9bae887a0 20 read_log 192888'13045 (192888'13044) modify 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.1315259.0:5030 2014-08-11 16:45:14.202884 2014-08-13 17:38:03.119583 7fe9bae887a0 20 read_log 192888'13048 (0'0) promote 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by osd.63.0:2416 2014-08-11 16:45:15.950716 2014-08-13 17:38:03.119595 7fe9bae887a0 20 read_log 192888'13049 (192888'13048) modify 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.1315259.0:5493 2014-08-11 16:45:15.944932 2014-08-13 17:38:03.119608 7fe9bae887a0 20 read_log 192888'13050 (0'0) promote 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.63.0:2418 2014-08-11 16:45:16.133226 2014-08-13 17:38:03.119621 7fe9bae887a0 20 read_log 192888'13051 (192888'13050) modify 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by client.1315259.0:5544 2014-08-11 16:45:16.126949 2014-08-13 17:38:03.119658 7fe9bae887a0 20 read_log 192888'13058 (0'0) promote 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by osd.63.0:2436 2014-08-11 16:45:21.623701 2014-08-13 17:38:03.119674 7fe9bae887a0 20 read_log 192888'13059 (192888'13058) modify 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.1315259.0:7063 2014-08-11 16:45:21.617468 2014-08-13 17:38:03.119687 7fe9bae887a0 20 read_log 192888'13066 (0'0) promote f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.63.0:2470 2014-08-11 16:45:31.268004 2014-08-13 17:38:03.119699 7fe9bae887a0 20 read_log 192888'13067 (192888'13066) modify f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by client.1315259.0:9641 2014-08-11 16:45:31.262391 2014-08-13 17:38:03.119713 7fe9bae887a0 20 read_log 192888'13072 (0'0) promote 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.63.0:2480 2014-08-11 16:45:34.485517 2014-08-13 17:38:03.119725 7fe9bae887a0 20 read_log 192888'13073 (192888'13072) modify 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by client.1315259.0:10469 2014-08-11 16:45:34.479256 2014-08-13 17:38:03.119739 7fe9bae887a0 20 read_log 192888'13080 (0'0) promote 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by osd.63.0:2496 2014-08-11 16:45:40.703724 2014-08-13 17:38:03.119752 7fe9bae887a0 20 read_log 192888'13081 (192888'13080) modify 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.1315259.0:12006 2014-08-11 16:45:40.698252 2014-08-13 17:38:03.119766 7fe9bae887a0 20 read_log 192888'13086 (0'0) promote 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by osd.63.0:2548 2014-08-11 16:45:53.959373 2014-08-13 17:38:03.119778 7fe9bae887a0 20 read_log 192888'13087 (192888'13086) modify 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.1315259.0:15366 2014-08-11 16:45:53.954108 2014-08-13 17:38:03.119794 7fe9bae887a0 20 read_log 192888'13092 (0'0) promote 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by osd.63.0:2564 2014-08-11 16:46:02.201034 2014-08-13 17:38:03.119809 7fe9bae887a0 20 read_log 192888'13093 (192888'13092) modify 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.1315259.0:17475 2014-08-11 16:46:02.195207 2014-08-13 17:38:03.119821 7fe9bae887a0 20 read_log 192888'13096 (0'0) promote 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.63.0:2568 2014-08-11 16:46:03.727654 2014-08-13 17:38:03.119835 7fe9bae887a0 20 read_log 192888'13097 (192888'13096) modify 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by client.1315259.0:17844 2014-08-11 16:46:03.721406 2014-08-13 17:38:03.119849 7fe9bae887a0 20 read_log 192888'13100 (0'0) promote f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by osd.63.0:2574 2014-08-11 16:46:05.981573 2014-08-13 17:38:03.119861 7fe9bae887a0 20 read_log 192888'13101 (192888'13100) modify f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.1315259.0:18419 2014-08-11 16:46:05.975916 2014-08-13 17:38:03.119873 7fe9bae887a0 20 read_log 192888'13102 (0'0) promote 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by osd.63.0:2576 2014-08-11 16:46:06.869613 2014-08-13 17:38:03.119888 7fe9bae887a0 20 read_log 192888'13103 (192888'13102) modify 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.1315259.0:18656 2014-08-11 16:46:06.864396 2014-08-13 17:38:03.119900 7fe9bae887a0 20 read_log 192888'13104 (0'0) promote 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by osd.63.0:2578 2014-08-11 16:46:07.459680 2014-08-13 17:38:03.119912 7fe9bae887a0 20 read_log 192888'13105 (192888'13104) modify 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.1315259.0:18813 2014-08-11 16:46:07.454120 2014-08-13 17:38:03.119931 7fe9bae887a0 20 read_log 192888'13110 (0'0) promote 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.63.0:2602 2014-08-11 16:46:18.780546 2014-08-13 17:38:03.119946 7fe9bae887a0 20 read_log 192888'13111 (192888'13110) modify 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by client.1315259.0:21764 2014-08-11 16:46:18.774353 2014-08-13 17:38:03.119959 7fe9bae887a0 20 read_log 192888'13114 (0'0) promote 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by osd.63.0:2618 2014-08-11 16:46:23.058543 2014-08-13 17:38:03.119971 7fe9bae887a0 20 read_log 192888'13115 (192888'13114) modify 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.1315259.0:22872 2014-08-11 16:46:23.052571 2014-08-13 17:38:03.119985 7fe9bae887a0 20 read_log 192888'13118 (0'0) promote e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by osd.63.0:2636 2014-08-11 16:46:28.896787 2014-08-13 17:38:03.119997 7fe9bae887a0 20 read_log 192888'13119 (192888'13118) modify e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.1315259.0:24336 2014-08-11 16:46:28.890831 2014-08-13 17:38:03.120009 7fe9bae887a0 20 read_log 192888'13122 (0'0) promote 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by osd.63.0:2648 2014-08-11 16:46:32.245392 2014-08-13 17:38:03.120023 7fe9bae887a0 20 read_log 192888'13123 (192888'13122) modify 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.1315259.0:25185 2014-08-11 16:46:32.239510 2014-08-13 17:38:03.120035 7fe9bae887a0 20 read_log 192888'13124 (0'0) promote 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by osd.63.0:2660 2014-08-11 16:46:36.396936 2014-08-13 17:38:03.120049 7fe9bae887a0 20 read_log 192888'13125 (192888'13124) modify 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.1315259.0:26149 2014-08-11 16:46:36.389546 2014-08-13 17:38:03.120061 7fe9bae887a0 20 read_log 192888'13132 (0'0) promote c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by osd.63.0:2678 2014-08-11 16:46:39.864162 2014-08-13 17:38:03.120073 7fe9bae887a0 20 read_log 192888'13133 (192888'13132) modify c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.1315259.0:26996 2014-08-11 16:46:39.858450 2014-08-13 17:38:03.120087 7fe9bae887a0 20 read_log 192888'13136 (0'0) promote adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by osd.63.0:2694 2014-08-11 16:46:47.332561 2014-08-13 17:38:03.120099 7fe9bae887a0 20 read_log 192888'13137 (192888'13136) modify adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.1315259.0:28819 2014-08-11 16:46:47.326675 2014-08-13 17:38:03.120118 7fe9bae887a0 20 read_log 192888'13140 (0'0) promote d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by osd.63.0:2700 2014-08-11 16:46:48.850958 2014-08-13 17:38:03.120131 7fe9bae887a0 20 read_log 192888'13141 (192888'13140) modify d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.1315259.0:29204 2014-08-11 16:46:48.843197 2014-08-13 17:38:03.120145 7fe9bae887a0 20 read_log 192888'13144 (0'0) promote f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by osd.63.0:2704 2014-08-11 16:46:50.194159 2014-08-13 17:38:03.120163 7fe9bae887a0 20 read_log 192888'13145 (192888'13144) modify f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.1315259.0:29513 2014-08-11 16:46:50.188457 2014-08-13 17:38:03.120176 7fe9bae887a0 20 read_log 192888'13147 (192888'13019) modify 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.1315259.0:31108 2014-08-11 16:46:56.000250 2014-08-13 17:38:03.120189 7fe9bae887a0 20 read_log 192888'13148 (192888'13021) modify 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.1315259.0:31506 2014-08-11 16:46:56.114865 2014-08-13 17:38:03.120213 7fe9bae887a0 20 read_log 192888'13150 (192888'13025) modify 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.1315259.0:31579 2014-08-11 16:46:56.135246 2014-08-13 17:38:03.120228 7fe9bae887a0 20 read_log 192888'13152 (192888'13029) modify b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.1315259.0:31791 2014-08-11 16:46:56.199828 2014-08-13 17:38:03.120241 7fe9bae887a0 20 read_log 192888'13154 (192888'13033) modify dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.1315259.0:32827 2014-08-11 16:46:56.480045 2014-08-13 17:38:03.120255 7fe9bae887a0 20 read_log 192888'13157 (192888'13039) modify ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.1315259.0:33104 2014-08-11 16:46:56.554071 2014-08-13 17:38:03.120267 7fe9bae887a0 20 read_log 192888'13160 (192888'13045) modify 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.1315259.0:35757 2014-08-11 16:46:57.238752 2014-08-13 17:38:03.120281 7fe9bae887a0 20 read_log 192888'13162 (192888'13049) modify 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.1315259.0:36220 2014-08-11 16:46:57.366910 2014-08-13 17:38:03.120293 7fe9bae887a0 20 read_log 192888'13163 (192888'13051) modify 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by client.1315259.0:36271 2014-08-11 16:46:57.378361 2014-08-13 17:38:03.120305 7fe9bae887a0 20 read_log 192888'13167 (192888'13059) modify 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.1315259.0:37790 2014-08-11 16:46:57.774425 2014-08-13 17:38:03.120317 7fe9bae887a0 20 read_log 192888'13171 (192888'13067) modify f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by client.1315259.0:40368 2014-08-11 16:46:58.464763 2014-08-13 17:38:03.120329 7fe9bae887a0 20 read_log 192888'13174 (192888'13073) modify 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by client.1315259.0:41196 2014-08-11 16:46:58.683595 2014-08-13 17:38:03.120341 7fe9bae887a0 20 read_log 192888'13178 (192888'13081) modify 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.1315259.0:42733 2014-08-11 16:46:59.089662 2014-08-13 17:38:03.120353 7fe9bae887a0 20 read_log 192888'13181 (192888'13087) modify 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.1315259.0:46093 2014-08-11 16:47:00.058626 2014-08-13 17:38:03.120365 7fe9bae887a0 20 read_log 192888'13184 (192888'13093) modify 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.1315259.0:48202 2014-08-11 16:47:00.695736 2014-08-13 17:38:03.120378 7fe9bae887a0 20 read_log 192888'13186 (192888'13097) modify 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by client.1315259.0:48571 2014-08-11 16:47:00.801205 2014-08-13 17:38:03.120390 7fe9bae887a0 20 read_log 192888'13188 (192888'13101) modify f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.1315259.0:49146 2014-08-11 16:47:00.970794 2014-08-13 17:38:03.120402 7fe9bae887a0 20 read_log 192888'13189 (192888'13103) modify 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.1315259.0:49383 2014-08-11 16:47:01.044349 2014-08-13 17:38:03.120414 7fe9bae887a0 20 read_log 192888'13190 (192888'13105) modify 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.1315259.0:49540 2014-08-11 16:47:01.095165 2014-08-13 17:38:03.120426 7fe9bae887a0 20 read_log 192888'13193 (192888'13111) modify 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by client.1315259.0:52491 2014-08-11 16:47:01.989906 2014-08-13 17:38:03.120438 7fe9bae887a0 20 read_log 192888'13195 (192888'13115) modify 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.1315259.0:53599 2014-08-11 16:47:02.302146 2014-08-13 17:38:03.120450 7fe9bae887a0 20 read_log 192888'13197 (192888'13119) modify e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.1315259.0:55063 2014-08-11 16:47:02.799704 2014-08-13 17:38:03.120469 7fe9bae887a0 20 read_log 192888'13199 (192888'13123) modify 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.1315259.0:55912 2014-08-11 16:47:03.050423 2014-08-13 17:38:03.120481 7fe9bae887a0 20 read_log 192888'13200 (192888'13125) modify 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.1315259.0:56876 2014-08-11 16:47:03.339329 2014-08-13 17:38:03.120493 7fe9bae887a0 20 read_log 192888'13204 (192888'13133) modify c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.1315259.0:57723 2014-08-11 16:47:03.593676 2014-08-13 17:38:03.120505 7fe9bae887a0 20 read_log 192888'13206 (192888'13137) modify adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.1315259.0:59546 2014-08-11 16:47:04.154934 2014-08-13 17:38:03.120517 7fe9bae887a0 20 read_log 192888'13208 (192888'13141) modify d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.1315259.0:59931 2014-08-11 16:47:04.287965 2014-08-13 17:38:03.120531 7fe9bae887a0 20 read_log 192888'13210 (192888'13145) modify f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.1315259.0:60240 2014-08-11 16:47:04.389446 2014-08-13 17:38:03.120543 7fe9bae887a0 20 read_log 192888'13213 (0'0) promote 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by osd.63.0:2908 2014-08-11 17:05:16.026506 2014-08-13 17:38:03.120555 7fe9bae887a0 20 read_log 192888'13214 (192888'13213) modify 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.942868.0:251 2014-08-11 17:05:16.020273 2014-08-13 17:38:03.120568 7fe9bae887a0 20 read_log 192888'13215 (0'0) promote bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by osd.63.0:2914 2014-08-11 17:05:17.173859 2014-08-13 17:38:03.120580 7fe9bae887a0 20 read_log 192888'13216 (192888'13215) modify bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.942868.0:523 2014-08-11 17:05:17.168743 2014-08-13 17:38:03.120592 7fe9bae887a0 20 read_log 192888'13217 (0'0) promote d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.63.0:2916 2014-08-11 17:05:18.714615 2014-08-13 17:38:03.120604 7fe9bae887a0 20 read_log 192888'13218 (192888'13217) modify d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by client.942868.0:904 2014-08-11 17:05:18.708782 2014-08-13 17:38:03.120635 7fe9bae887a0 20 read_log 192888'13225 (0'0) promote e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by osd.63.0:2929 2014-08-11 17:05:21.997270 2014-08-13 17:38:03.120650 7fe9bae887a0 20 read_log 192888'13226 (192888'13225) modify e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.942868.0:1690 2014-08-11 17:05:21.991088 2014-08-13 17:38:03.120663 7fe9bae887a0 20 read_log 192888'13231 (0'0) promote bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by osd.63.0:2953 2014-08-11 17:05:27.591804 2014-08-13 17:38:03.120675 7fe9bae887a0 20 read_log 192888'13232 (192888'13231) modify bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.942868.0:3149 2014-08-11 17:05:27.586004 2014-08-13 17:38:03.120687 7fe9bae887a0 20 read_log 192888'13237 (0'0) promote 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by osd.63.0:2995 2014-08-11 17:05:40.841671 2014-08-13 17:38:03.120699 7fe9bae887a0 20 read_log 192888'13238 (192888'13237) modify 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.942868.0:6701 2014-08-11 17:05:40.835695 2014-08-13 17:38:03.120711 7fe9bae887a0 20 read_log 192888'13241 (0'0) promote 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.63.0:3017 2014-08-11 17:05:48.814057 2014-08-13 17:38:03.120724 7fe9bae887a0 20 read_log 192888'13242 (192888'13241) modify 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by client.942868.0:8805 2014-08-11 17:05:48.808790 2014-08-13 17:38:03.120743 7fe9bae887a0 20 read_log 192888'13243 (0'0) promote c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by osd.63.0:3023 2014-08-11 17:05:51.151986 2014-08-13 17:38:03.120755 7fe9bae887a0 20 read_log 192888'13244 (192888'13243) modify c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.942868.0:9433 2014-08-11 17:05:51.146275 2014-08-13 17:38:03.120770 7fe9bae887a0 20 read_log 192888'13247 (0'0) promote ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by osd.63.0:3039 2014-08-11 17:05:53.476772 2014-08-13 17:38:03.120782 7fe9bae887a0 20 read_log 192888'13248 (192888'13247) modify ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.942868.0:10075 2014-08-11 17:05:53.471438 2014-08-13 17:38:03.120794 7fe9bae887a0 20 read_log 192888'13249 (0'0) promote 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by osd.63.0:3041 2014-08-11 17:05:53.547422 2014-08-13 17:38:03.120809 7fe9bae887a0 20 read_log 192888'13250 (192888'13249) modify 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.942868.0:10095 2014-08-11 17:05:53.542405 2014-08-13 17:38:03.120821 7fe9bae887a0 20 read_log 192888'13253 (0'0) promote 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by osd.63.0:3049 2014-08-11 17:05:56.024802 2014-08-13 17:38:03.120836 7fe9bae887a0 20 read_log 192888'13254 (192888'13253) modify 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.942868.0:10785 2014-08-11 17:05:56.018522 2014-08-13 17:38:03.120848 7fe9bae887a0 20 read_log 192888'13261 (0'0) promote 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.63.0:3085 2014-08-11 17:06:04.546055 2014-08-13 17:38:03.120862 7fe9bae887a0 20 read_log 192888'13262 (192888'13261) modify 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by client.942868.0:13100 2014-08-11 17:06:04.541004 2014-08-13 17:38:03.120875 7fe9bae887a0 20 read_log 192888'13263 (0'0) promote e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by osd.63.0:3089 2014-08-11 17:06:06.461972 2014-08-13 17:38:03.120889 7fe9bae887a0 20 read_log 192888'13264 (192888'13263) modify e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.942868.0:13575 2014-08-11 17:06:06.456216 2014-08-13 17:38:03.120903 7fe9bae887a0 20 read_log 192888'13265 (0'0) promote fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by osd.63.0:3091 2014-08-11 17:06:07.361719 2014-08-13 17:38:03.120917 7fe9bae887a0 20 read_log 192888'13266 (192888'13265) modify fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.942868.0:13800 2014-08-11 17:06:07.356195 2014-08-13 17:38:03.120929 7fe9bae887a0 20 read_log 192888'13273 (0'0) promote 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by osd.63.0:3111 2014-08-11 17:06:20.131968 2014-08-13 17:38:03.120943 7fe9bae887a0 20 read_log 192888'13274 (192888'13273) modify 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.942868.0:17019 2014-08-11 17:06:20.126452 2014-08-13 17:38:03.120956 7fe9bae887a0 20 read_log 192888'13275 (0'0) promote 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.63.0:3115 2014-08-11 17:06:20.455417 2014-08-13 17:38:03.120968 7fe9bae887a0 20 read_log 192888'13276 (192888'13275) modify 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by client.942868.0:17105 2014-08-11 17:06:20.450218 2014-08-13 17:38:03.120980 7fe9bae887a0 20 read_log 192888'13277 (0'0) promote 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.63.0:3123 2014-08-11 17:06:23.470737 2014-08-13 17:38:03.120994 7fe9bae887a0 20 read_log 192888'13278 (192888'13277) modify 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by client.942868.0:17905 2014-08-11 17:06:23.465589 2014-08-13 17:38:03.121016 7fe9bae887a0 20 read_log 192888'13281 (0'0) promote 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by osd.63.0:3147 2014-08-11 17:06:35.391110 2014-08-13 17:38:03.121030 7fe9bae887a0 20 read_log 192888'13282 (192888'13281) modify 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.942868.0:21052 2014-08-11 17:06:35.385330 2014-08-13 17:38:03.121043 7fe9bae887a0 20 read_log 192888'13287 (0'0) promote 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by osd.63.0:3159 2014-08-11 17:06:38.432324 2014-08-13 17:38:03.121055 7fe9bae887a0 20 read_log 192888'13288 (192888'13287) modify 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.942868.0:21854 2014-08-11 17:06:38.426622 2014-08-13 17:38:03.121069 7fe9bae887a0 20 read_log 192888'13289 (0'0) promote 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by osd.63.0:3163 2014-08-11 17:06:38.894415 2014-08-13 17:38:03.121082 7fe9bae887a0 20 read_log 192888'13290 (192888'13289) modify 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.942868.0:21978 2014-08-11 17:06:38.888769 2014-08-13 17:38:03.121094 7fe9bae887a0 20 read_log 192888'13291 (0'0) promote 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by osd.63.0:3165 2014-08-11 17:06:39.569837 2014-08-13 17:38:03.121108 7fe9bae887a0 20 read_log 192888'13292 (192888'13291) modify 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.942868.0:22162 2014-08-11 17:06:39.564177 2014-08-13 17:38:03.121120 7fe9bae887a0 20 read_log 192888'13293 (0'0) promote 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by osd.63.0:3175 2014-08-11 17:06:42.516783 2014-08-13 17:38:03.121133 7fe9bae887a0 20 read_log 192888'13294 (192888'13293) modify 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.942868.0:22909 2014-08-11 17:06:42.510832 2014-08-13 17:38:03.121147 7fe9bae887a0 20 read_log 192888'13295 (0'0) promote cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by osd.63.0:3177 2014-08-11 17:06:44.928057 2014-08-13 17:38:03.121165 7fe9bae887a0 20 read_log 192888'13296 (192888'13295) modify cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.942868.0:23517 2014-08-11 17:06:44.922386 2014-08-13 17:38:03.121180 7fe9bae887a0 20 read_log 192888'13297 (0'0) promote a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by osd.63.0:3183 2014-08-11 17:06:46.837513 2014-08-13 17:38:03.121192 7fe9bae887a0 20 read_log 192888'13298 (192888'13297) modify a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.942868.0:24034 2014-08-11 17:06:46.831311 2014-08-13 17:38:03.121204 7fe9bae887a0 20 read_log 192888'13305 (0'0) promote 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by osd.63.0:3193 2014-08-11 17:06:49.967136 2014-08-13 17:38:03.121218 7fe9bae887a0 20 read_log 192888'13306 (192888'13305) modify 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.942868.0:24881 2014-08-11 17:06:49.961602 2014-08-13 17:38:03.121231 7fe9bae887a0 20 read_log 192888'13309 (0'0) promote 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by osd.63.0:3197 2014-08-11 17:06:50.766202 2014-08-13 17:38:03.121243 7fe9bae887a0 20 read_log 192888'13310 (192888'13309) modify 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.942868.0:25095 2014-08-11 17:06:50.760186 2014-08-13 17:38:03.121255 7fe9bae887a0 20 read_log 192888'13311 (0'0) promote e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by osd.63.0:3201 2014-08-11 17:06:51.567773 2014-08-13 17:38:03.121267 7fe9bae887a0 20 read_log 192888'13312 (192888'13311) modify e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.942868.0:25305 2014-08-11 17:06:51.561765 2014-08-13 17:38:03.121296 7fe9bae887a0 20 read_log 192888'13315 (0'0) promote 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.63.0:3211 2014-08-11 17:06:55.163074 2014-08-13 17:38:03.121311 7fe9bae887a0 20 read_log 192888'13316 (192888'13315) modify 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by client.942868.0:26188 2014-08-11 17:06:55.157641 2014-08-13 17:38:03.121323 7fe9bae887a0 20 read_log 192888'13317 (0'0) promote 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.63.0:3213 2014-08-11 17:06:55.991229 2014-08-13 17:38:03.121335 7fe9bae887a0 20 read_log 192888'13318 (192888'13317) modify 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by client.942868.0:26411 2014-08-11 17:06:55.985065 2014-08-13 17:38:03.121349 7fe9bae887a0 20 read_log 192888'13327 (0'0) promote a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by osd.63.0:3265 2014-08-11 17:07:08.595537 2014-08-13 17:38:03.121362 7fe9bae887a0 20 read_log 192888'13328 (192888'13327) modify a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.942868.0:29694 2014-08-11 17:07:08.589508 2014-08-13 17:38:03.121374 7fe9bae887a0 20 read_log 192888'13329 (0'0) promote afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by osd.63.0:3285 2014-08-11 17:07:14.606968 2014-08-13 17:38:03.121386 7fe9bae887a0 20 read_log 192888'13330 (192888'13329) modify afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.942868.0:31209 2014-08-11 17:07:14.600171 2014-08-13 17:38:03.121398 7fe9bae887a0 20 read_log 192888'13331 (192888'13214) modify 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.942868.0:31517 2014-08-11 17:07:15.949907 2014-08-13 17:38:03.121410 7fe9bae887a0 20 read_log 192888'13332 (192888'13216) modify bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.942868.0:31789 2014-08-11 17:07:16.058045 2014-08-13 17:38:03.121422 7fe9bae887a0 20 read_log 192888'13333 (192888'13218) modify d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by client.942868.0:32170 2014-08-11 17:07:16.172900 2014-08-13 17:38:03.121435 7fe9bae887a0 20 read_log 192888'13337 (192888'13226) modify e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.942868.0:32956 2014-08-11 17:07:16.386731 2014-08-13 17:38:03.121447 7fe9bae887a0 20 read_log 192888'13340 (192888'13232) modify bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.942868.0:34415 2014-08-11 17:07:16.781081 2014-08-13 17:38:03.121459 7fe9bae887a0 20 read_log 192888'13343 (192888'13238) modify 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.942868.0:37967 2014-08-11 17:07:17.756254 2014-08-13 17:38:03.121471 7fe9bae887a0 20 read_log 192888'13345 (192888'13242) modify 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by client.942868.0:40071 2014-08-11 17:07:18.357573 2014-08-13 17:38:03.121483 7fe9bae887a0 20 read_log 192888'13346 (192888'13244) modify c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.942868.0:40699 2014-08-11 17:07:18.524456 2014-08-13 17:38:03.121495 7fe9bae887a0 20 read_log 192888'13348 (192888'13248) modify ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.942868.0:41341 2014-08-11 17:07:18.703708 2014-08-13 17:38:03.121508 7fe9bae887a0 20 read_log 192888'13349 (192888'13250) modify 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.942868.0:41361 2014-08-11 17:07:18.708485 2014-08-13 17:38:03.121520 7fe9bae887a0 20 read_log 192888'13351 (192888'13254) modify 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.942868.0:42051 2014-08-11 17:07:18.892404 2014-08-13 17:38:03.121532 7fe9bae887a0 20 read_log 192888'13355 (192888'13262) modify 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by client.942868.0:44366 2014-08-11 17:07:19.546908 2014-08-13 17:38:03.121570 7fe9bae887a0 20 read_log 192888'13356 (192888'13264) modify e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.942868.0:44841 2014-08-11 17:07:19.676915 2014-08-13 17:38:03.121587 7fe9bae887a0 20 read_log 192888'13357 (192888'13266) modify fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.942868.0:45066 2014-08-11 17:07:19.737132 2014-08-13 17:38:03.121599 7fe9bae887a0 20 read_log 192888'13361 (192888'13274) modify 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.942868.0:48285 2014-08-11 17:07:20.675970 2014-08-13 17:38:03.121611 7fe9bae887a0 20 read_log 192888'13362 (192888'13276) modify 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by client.942868.0:48371 2014-08-11 17:07:20.699647 2014-08-13 17:38:03.121623 7fe9bae887a0 20 read_log 192888'13363 (192888'13278) modify 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by client.942868.0:49171 2014-08-11 17:07:20.930478 2014-08-13 17:38:03.121635 7fe9bae887a0 20 read_log 192888'13365 (192888'13282) modify 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.942868.0:52318 2014-08-11 17:07:21.902490 2014-08-13 17:38:03.121647 7fe9bae887a0 20 read_log 192888'13368 (192888'13288) modify 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.942868.0:53120 2014-08-11 17:07:22.154653 2014-08-13 17:38:03.121659 7fe9bae887a0 20 read_log 192888'13369 (192888'13290) modify 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.942868.0:53244 2014-08-11 17:07:22.197483 2014-08-13 17:38:03.121671 7fe9bae887a0 20 read_log 192888'13370 (192888'13292) modify 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.942868.0:53428 2014-08-11 17:07:22.259563 2014-08-13 17:38:03.121684 7fe9bae887a0 20 read_log 192888'13371 (192888'13294) modify 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.942868.0:54175 2014-08-11 17:07:22.483793 2014-08-13 17:38:03.121696 7fe9bae887a0 20 read_log 192888'13372 (192888'13296) modify cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.942868.0:54783 2014-08-11 17:07:22.655131 2014-08-13 17:38:03.121725 7fe9bae887a0 20 read_log 192888'13373 (192888'13298) modify a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.942868.0:55300 2014-08-11 17:07:22.804687 2014-08-13 17:38:03.121740 7fe9bae887a0 20 read_log 192888'13377 (192888'13306) modify 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.942868.0:56147 2014-08-11 17:07:23.065609 2014-08-13 17:38:03.121753 7fe9bae887a0 20 read_log 192888'13379 (192888'13310) modify 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.942868.0:56361 2014-08-11 17:07:23.126742 2014-08-13 17:38:03.121768 7fe9bae887a0 20 read_log 192888'13380 (192888'13312) modify e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.942868.0:56571 2014-08-11 17:07:23.194927 2014-08-13 17:38:03.121780 7fe9bae887a0 20 read_log 192888'13382 (192888'13316) modify 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by client.942868.0:57454 2014-08-11 17:07:23.459797 2014-08-13 17:38:03.121792 7fe9bae887a0 20 read_log 192888'13383 (192888'13318) modify 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by client.942868.0:57677 2014-08-11 17:07:23.524921 2014-08-13 17:38:03.121804 7fe9bae887a0 20 read_log 192888'13388 (192888'13328) modify a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.942868.0:60960 2014-08-11 17:07:24.495758 2014-08-13 17:38:03.121817 7fe9bae887a0 20 read_log 192888'13389 (192888'13330) modify afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.942868.0:62475 2014-08-11 17:07:24.959035 2014-08-13 17:38:03.121832 7fe9bae887a0 20 read_log 193305'13390 (0'0) promote d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by osd.27.0:14 2014-08-11 18:42:41.077521 2014-08-13 17:38:03.121854 7fe9bae887a0 20 read_log 193305'13391 (193305'13390) modify d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.951337.0:2087 2014-08-11 18:42:41.069328 2014-08-13 17:38:03.121869 7fe9bae887a0 20 read_log 193305'13392 (0'0) promote 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by osd.27.0:24 2014-08-11 18:42:45.983295 2014-08-13 17:38:03.121882 7fe9bae887a0 20 read_log 193305'13393 (193305'13392) modify 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.951337.0:3437 2014-08-11 18:42:45.975808 2014-08-13 17:38:03.121895 7fe9bae887a0 20 read_log 193305'13394 (0'0) promote 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by osd.27.0:36 2014-08-11 18:42:47.244125 2014-08-13 17:38:03.121907 7fe9bae887a0 20 read_log 193305'13395 (193305'13394) modify 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.951337.0:3787 2014-08-11 18:42:47.237528 2014-08-13 17:38:03.121941 7fe9bae887a0 20 read_log 193305'13396 (0'0) promote a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:40 2014-08-11 18:42:47.464134 2014-08-13 17:38:03.121956 7fe9bae887a0 20 read_log 193305'13397 (193305'13396) modify a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by client.951337.0:3846 2014-08-11 18:42:47.457381 2014-08-13 17:38:03.121968 7fe9bae887a0 20 read_log 193305'13398 (0'0) promote 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:58 2014-08-11 18:42:59.280930 2014-08-13 17:38:03.121982 7fe9bae887a0 20 read_log 193305'13399 (193305'13398) modify 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by client.951337.0:7065 2014-08-11 18:42:59.274133 2014-08-13 17:38:03.121995 7fe9bae887a0 20 read_log 193305'13400 (0'0) promote df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by osd.27.0:66 2014-08-11 18:43:02.062890 2014-08-13 17:38:03.122009 7fe9bae887a0 20 read_log 193305'13401 (193305'13400) modify df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.951337.0:7835 2014-08-11 18:43:02.056854 2014-08-13 17:38:03.122022 7fe9bae887a0 20 read_log 193305'13402 (0'0) promote 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by osd.27.0:88 2014-08-11 18:43:06.690295 2014-08-13 17:38:03.122036 7fe9bae887a0 20 read_log 193305'13403 (193305'13402) modify 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.951337.0:9015 2014-08-11 18:43:06.681386 2014-08-13 17:38:03.122048 7fe9bae887a0 20 read_log 193305'13404 (0'0) promote 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:104 2014-08-11 18:43:11.101945 2014-08-13 17:38:03.122062 7fe9bae887a0 20 read_log 193305'13405 (193305'13404) modify 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by client.951337.0:10224 2014-08-11 18:43:11.094459 2014-08-13 17:38:03.122075 7fe9bae887a0 20 read_log 193305'13406 (0'0) promote b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:112 2014-08-11 18:43:13.287270 2014-08-13 17:38:03.122087 7fe9bae887a0 20 read_log 193305'13407 (193305'13406) modify b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by client.951337.0:10807 2014-08-11 18:43:13.280438 2014-08-13 17:38:03.122099 7fe9bae887a0 20 read_log 193305'13408 (0'0) promote 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by osd.27.0:116 2014-08-11 18:43:13.663828 2014-08-13 17:38:03.122111 7fe9bae887a0 20 read_log 193305'13409 (193305'13408) modify 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.951337.0:10908 2014-08-11 18:43:13.658287 2014-08-13 17:38:03.122125 7fe9bae887a0 20 read_log 193305'13410 (193305'13410) modify 3bf/hit_set_24.3bf_archive_2014-08-11 18:38:15.792949_2014-08-11 18:43:20.626482/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:20.626706 2014-08-13 17:38:03.122147 7fe9bae887a0 20 read_log 193305'13411 (0'0) promote 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by osd.27.0:137 2014-08-11 18:43:20.629186 2014-08-13 17:38:03.122178 7fe9bae887a0 20 read_log 193305'13412 (193305'13411) modify 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.951337.0:12731 2014-08-11 18:43:20.620819 2014-08-13 17:38:03.122193 7fe9bae887a0 20 read_log 193305'13413 (0'0) promote 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by osd.27.0:139 2014-08-11 18:43:21.349622 2014-08-13 17:38:03.122206 7fe9bae887a0 20 read_log 193305'13414 (193305'13413) modify 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.951337.0:12922 2014-08-11 18:43:21.342777 2014-08-13 17:38:03.122218 7fe9bae887a0 20 read_log 193305'13415 (0'0) promote 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:149 2014-08-11 18:43:24.512279 2014-08-13 17:38:03.122230 7fe9bae887a0 20 read_log 193305'13416 (193305'13415) modify 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by client.951337.0:13756 2014-08-11 18:43:24.504899 2014-08-13 17:38:03.122242 7fe9bae887a0 20 read_log 193305'13417 (0'0) promote 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by osd.27.0:161 2014-08-11 18:43:29.451095 2014-08-13 17:38:03.122254 7fe9bae887a0 20 read_log 193305'13418 (193305'13417) modify 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.951337.0:15143 2014-08-11 18:43:29.443952 2014-08-13 17:38:03.122266 7fe9bae887a0 20 read_log 193305'13419 (0'0) promote ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by osd.27.0:163 2014-08-11 18:43:29.553355 2014-08-13 17:38:03.122278 7fe9bae887a0 20 read_log 193305'13420 (193305'13419) modify ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.951337.0:15174 2014-08-11 18:43:29.547223 2014-08-13 17:38:03.122290 7fe9bae887a0 20 read_log 193305'13421 (0'0) promote f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by osd.27.0:165 2014-08-11 18:43:30.009412 2014-08-13 17:38:03.122302 7fe9bae887a0 20 read_log 193305'13422 (193305'13421) modify f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.951337.0:15311 2014-08-11 18:43:30.002185 2014-08-13 17:38:03.122314 7fe9bae887a0 20 read_log 193305'13423 (0'0) promote ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by osd.27.0:176 2014-08-11 18:43:33.828064 2014-08-13 17:38:03.122326 7fe9bae887a0 20 read_log 193305'13424 (193305'13423) modify ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.951337.0:16393 2014-08-11 18:43:33.819440 2014-08-13 17:38:03.122338 7fe9bae887a0 20 read_log 193305'13425 (0'0) promote 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:180 2014-08-11 18:43:36.402518 2014-08-13 17:38:03.122350 7fe9bae887a0 20 read_log 193305'13426 (193305'13425) modify 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by client.951337.0:17126 2014-08-11 18:43:36.395320 2014-08-13 17:38:03.122362 7fe9bae887a0 20 read_log 193305'13427 (0'0) promote ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:198 2014-08-11 18:43:43.741580 2014-08-13 17:38:03.122374 7fe9bae887a0 20 read_log 193305'13428 (193305'13427) modify ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by client.951337.0:19205 2014-08-11 18:43:43.733785 2014-08-13 17:38:03.122386 7fe9bae887a0 20 read_log 193305'13429 (0'0) promote 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by osd.27.0:240 2014-08-11 18:43:59.514352 2014-08-13 17:38:03.122398 7fe9bae887a0 20 read_log 193305'13430 (193305'13429) modify 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.951337.0:23579 2014-08-11 18:43:59.506751 2014-08-13 17:38:03.122420 7fe9bae887a0 20 read_log 193305'13431 (0'0) promote 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by osd.27.0:242 2014-08-11 18:43:59.706124 2014-08-13 17:38:03.122433 7fe9bae887a0 20 read_log 193305'13432 (193305'13431) modify 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.951337.0:23635 2014-08-11 18:43:59.699107 2014-08-13 17:38:03.122446 7fe9bae887a0 20 read_log 193305'13433 (0'0) promote 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by osd.27.0:262 2014-08-11 18:44:06.890159 2014-08-13 17:38:03.122458 7fe9bae887a0 20 read_log 193305'13434 (193305'13433) modify 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.951337.0:25605 2014-08-11 18:44:06.882262 2014-08-13 17:38:03.122470 7fe9bae887a0 20 read_log 193305'13435 (0'0) promote b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:270 2014-08-11 18:44:09.366111 2014-08-13 17:38:03.122481 7fe9bae887a0 20 read_log 193305'13436 (193305'13435) modify b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by client.951337.0:26266 2014-08-11 18:44:09.359645 2014-08-13 17:38:03.122494 7fe9bae887a0 20 read_log 193305'13437 (193305'13391) modify d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.951337.0:34798 2014-08-11 18:44:35.195917 2014-08-13 17:38:03.122506 7fe9bae887a0 20 read_log 193305'13438 (193305'13393) modify 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.951337.0:36148 2014-08-11 18:44:35.957183 2014-08-13 17:38:03.122518 7fe9bae887a0 20 read_log 193305'13439 (193305'13395) modify 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.951337.0:36498 2014-08-11 18:44:36.266506 2014-08-13 17:38:03.122530 7fe9bae887a0 20 read_log 193305'13440 (193305'13397) modify a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by client.951337.0:36557 2014-08-11 18:44:36.325854 2014-08-13 17:38:03.122542 7fe9bae887a0 20 read_log 193305'13441 (193305'13399) modify 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by client.951337.0:39776 2014-08-11 18:44:38.181871 2014-08-13 17:38:03.122554 7fe9bae887a0 20 read_log 193305'13442 (193305'13401) modify df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.951337.0:40546 2014-08-11 18:44:38.444339 2014-08-13 17:38:03.122566 7fe9bae887a0 20 read_log 193305'13443 (193305'13403) modify 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.951337.0:41726 2014-08-11 18:44:38.847398 2014-08-13 17:38:03.122578 7fe9bae887a0 20 read_log 193305'13444 (193305'13405) modify 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by client.951337.0:42935 2014-08-11 18:44:39.427803 2014-08-13 17:38:03.122590 7fe9bae887a0 20 read_log 193305'13445 (193305'13407) modify b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by client.951337.0:43518 2014-08-11 18:44:39.615884 2014-08-13 17:38:03.122602 7fe9bae887a0 20 read_log 193305'13446 (193305'13409) modify 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.951337.0:43619 2014-08-11 18:44:39.647523 2014-08-13 17:38:03.122614 7fe9bae887a0 20 read_log 193305'13447 (193305'13412) modify 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.951337.0:45442 2014-08-11 18:44:40.202377 2014-08-13 17:38:03.122626 7fe9bae887a0 20 read_log 193305'13448 (193305'13414) modify 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.951337.0:45633 2014-08-11 18:44:40.257112 2014-08-13 17:38:03.122638 7fe9bae887a0 20 read_log 193305'13449 (193305'13416) modify 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by client.951337.0:46467 2014-08-11 18:44:40.489965 2014-08-13 17:38:03.122650 7fe9bae887a0 20 read_log 193305'13450 (193305'13418) modify 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.951337.0:47854 2014-08-11 18:44:40.884192 2014-08-13 17:38:03.122669 7fe9bae887a0 20 read_log 193305'13451 (193305'13420) modify ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.951337.0:47885 2014-08-11 18:44:40.892586 2014-08-13 17:38:03.122682 7fe9bae887a0 20 read_log 193305'13452 (193305'13422) modify f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.951337.0:48022 2014-08-11 18:44:40.933522 2014-08-13 17:38:03.122694 7fe9bae887a0 20 read_log 193305'13453 (193305'13424) modify ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.951337.0:49104 2014-08-11 18:44:41.255802 2014-08-13 17:38:03.122706 7fe9bae887a0 20 read_log 193305'13454 (193305'13426) modify 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by client.951337.0:49837 2014-08-11 18:44:41.486832 2014-08-13 17:38:03.122718 7fe9bae887a0 20 read_log 193305'13455 (193305'13428) modify ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by client.951337.0:51916 2014-08-11 18:44:42.085579 2014-08-13 17:38:03.122730 7fe9bae887a0 20 read_log 193305'13456 (193305'13430) modify 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.951337.0:56290 2014-08-11 18:44:43.375049 2014-08-13 17:38:03.122742 7fe9bae887a0 20 read_log 193305'13457 (193305'13432) modify 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.951337.0:56346 2014-08-11 18:44:43.392348 2014-08-13 17:38:03.122754 7fe9bae887a0 20 read_log 193305'13458 (193305'13434) modify 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.951337.0:58316 2014-08-11 18:44:44.005193 2014-08-13 17:38:03.122768 7fe9bae887a0 20 read_log 193305'13459 (193305'13436) modify b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by client.951337.0:58977 2014-08-11 18:44:44.199123 2014-08-13 17:38:03.122781 7fe9bae887a0 20 read_log 194209'13460 (192834'12398) clean b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.27.0:771 2014-08-12 14:05:54.246391 2014-08-13 17:38:03.122793 7fe9bae887a0 20 read_log 194209'13461 (192834'12838) clean f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.27.0:772 2014-08-12 14:05:54.246718 2014-08-13 17:38:03.122805 7fe9bae887a0 20 read_log 194209'13462 (192834'12445) clean d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.27.0:773 2014-08-12 14:05:54.247370 2014-08-13 17:38:03.122819 7fe9bae887a0 20 read_log 194209'13463 (192834'12439) clean dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.27.0:774 2014-08-12 14:05:54.247865 2014-08-13 17:38:03.122831 7fe9bae887a0 20 read_log 194209'13464 (192834'12421) clean e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.27.0:775 2014-08-12 14:05:54.295144 2014-08-13 17:38:03.122843 7fe9bae887a0 20 read_log 194209'13465 (192834'12960) clean 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.27.0:776 2014-08-12 14:05:54.298212 2014-08-13 17:38:03.122855 7fe9bae887a0 20 read_log 194209'13466 (192834'12346) clean 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.27.0:777 2014-08-12 14:05:54.301447 2014-08-13 17:38:03.122867 7fe9bae887a0 20 read_log 194209'13467 (192888'13383) clean 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.27.0:778 2014-08-12 14:05:54.304311 2014-08-13 17:38:03.122879 7fe9bae887a0 20 read_log 194209'13468 (192888'13174) clean 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.27.0:779 2014-08-12 14:05:54.310633 2014-08-13 17:38:03.122891 7fe9bae887a0 20 read_log 194209'13469 (192834'12958) clean da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.27.0:780 2014-08-12 14:05:54.310996 2014-08-13 17:38:03.122915 7fe9bae887a0 20 read_log 194209'13470 (193305'13459) clean b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:781 2014-08-12 14:05:54.313938 2014-08-13 17:38:03.122930 7fe9bae887a0 20 read_log 194209'13471 (192888'13355) clean 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.27.0:782 2014-08-12 14:05:54.317455 2014-08-13 17:38:03.122962 7fe9bae887a0 20 read_log 194209'13472 (192834'12910) clean 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.27.0:783 2014-08-12 14:05:54.320441 2014-08-13 17:38:03.122977 7fe9bae887a0 20 read_log 194209'13473 (192834'12438) clean 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.27.0:784 2014-08-12 14:05:54.323820 2014-08-13 17:38:03.122991 7fe9bae887a0 20 read_log 194209'13474 (192834'12461) clean 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.27.0:785 2014-08-12 14:05:54.327003 2014-08-13 17:38:03.123003 7fe9bae887a0 20 read_log 194209'13475 (193305'13444) clean 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:786 2014-08-12 14:05:54.330516 2014-08-13 17:38:03.123015 7fe9bae887a0 20 read_log 194209'13476 (193305'13440) clean a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:787 2014-08-12 14:05:54.333560 2014-08-13 17:38:03.123030 7fe9bae887a0 20 read_log 194209'13477 (192888'13362) clean 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.27.0:788 2014-08-12 14:05:54.336644 2014-08-13 17:38:03.123042 7fe9bae887a0 20 read_log 194209'13478 (192888'13163) clean 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.27.0:789 2014-08-12 14:05:54.339647 2014-08-13 17:38:03.123057 7fe9bae887a0 20 read_log 194209'13479 (192888'13382) clean 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.27.0:790 2014-08-12 14:05:54.342704 2014-08-13 17:38:03.123069 7fe9bae887a0 20 read_log 194209'13480 (192834'13006) clean 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.27.0:791 2014-08-12 14:05:54.345694 2014-08-13 17:38:03.123083 7fe9bae887a0 20 read_log 194209'13481 (193305'13445) clean b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:792 2014-08-12 14:05:54.348740 2014-08-13 17:38:03.123095 7fe9bae887a0 20 read_log 194209'13482 (192888'13345) clean 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.27.0:793 2014-08-12 14:05:54.351817 2014-08-13 17:38:03.123109 7fe9bae887a0 20 read_log 194209'13483 (192888'13186) clean 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.27.0:794 2014-08-12 14:05:54.354813 2014-08-13 17:38:03.123121 7fe9bae887a0 20 read_log 194209'13484 (192834'12942) clean 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.27.0:795 2014-08-12 14:05:54.358239 2014-08-13 17:38:03.123133 7fe9bae887a0 20 read_log 194209'13485 (192834'12931) clean ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.27.0:796 2014-08-12 14:05:54.361292 2014-08-13 17:38:03.123146 7fe9bae887a0 20 read_log 194209'13486 (192834'12856) clean f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by osd.27.0:797 2014-08-12 14:05:54.364396 2014-08-13 17:38:03.123163 7fe9bae887a0 20 read_log 194209'13487 (193305'13441) clean 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:798 2014-08-12 14:05:54.367613 2014-08-13 17:38:03.123175 7fe9bae887a0 20 read_log 194209'13488 (192834'12425) clean cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by osd.27.0:799 2014-08-12 14:05:54.370830 2014-08-13 17:38:03.123209 7fe9bae887a0 20 read_log 194209'13489 (192888'13193) clean 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.27.0:800 2014-08-12 14:05:54.373822 2014-08-13 17:38:03.123234 7fe9bae887a0 20 read_log 194209'13490 (192888'13363) clean 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.27.0:801 2014-08-12 14:05:54.377075 2014-08-13 17:38:03.123250 7fe9bae887a0 20 read_log 194209'13491 (192834'12859) clean f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.27.0:802 2014-08-12 14:05:54.379882 2014-08-13 17:38:03.123262 7fe9bae887a0 20 read_log 194209'13492 (192834'12458) clean 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.27.0:803 2014-08-12 14:05:54.382929 2014-08-13 17:38:03.123274 7fe9bae887a0 20 read_log 194209'13493 (193305'13455) clean ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:804 2014-08-12 14:05:54.386025 2014-08-13 17:38:03.123289 7fe9bae887a0 20 read_log 194209'13494 (192888'13171) clean f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.27.0:805 2014-08-12 14:05:54.389277 2014-08-13 17:38:03.123301 7fe9bae887a0 20 read_log 194209'13495 (193305'13449) clean 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:806 2014-08-12 14:05:54.392409 2014-08-13 17:38:03.123313 7fe9bae887a0 20 read_log 194209'13496 (192888'13333) clean d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.27.0:807 2014-08-12 14:05:54.395462 2014-08-13 17:38:03.123325 7fe9bae887a0 20 read_log 194209'13497 (194209'13460) delete b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.27.0:809 2014-08-12 14:05:54.428558 2014-08-13 17:38:03.123337 7fe9bae887a0 20 read_log 194209'13498 (194209'13461) delete f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.27.0:810 2014-08-12 14:05:54.431831 2014-08-13 17:38:03.123349 7fe9bae887a0 20 read_log 194209'13499 (194209'13462) delete d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.27.0:811 2014-08-12 14:05:54.434516 2014-08-13 17:38:03.123361 7fe9bae887a0 20 read_log 194209'13500 (194209'13463) delete dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.27.0:812 2014-08-12 14:05:54.438225 2014-08-13 17:38:03.123372 7fe9bae887a0 20 read_log 194209'13501 (194209'13464) delete e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.27.0:813 2014-08-12 14:05:54.441560 2014-08-13 17:38:03.123384 7fe9bae887a0 20 read_log 194209'13502 (194209'13465) delete 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.27.0:814 2014-08-12 14:05:54.444282 2014-08-13 17:38:03.123396 7fe9bae887a0 20 read_log 194209'13503 (194209'13466) delete 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.27.0:815 2014-08-12 14:05:54.448091 2014-08-13 17:38:03.123408 7fe9bae887a0 20 read_log 194209'13504 (194209'13467) delete 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.27.0:816 2014-08-12 14:05:54.451410 2014-08-13 17:38:03.123420 7fe9bae887a0 20 read_log 194209'13505 (194209'13468) delete 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.27.0:817 2014-08-12 14:05:54.454645 2014-08-13 17:38:03.123432 7fe9bae887a0 20 read_log 194209'13506 (193305'13454) clean 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:818 2014-08-12 14:05:54.454880 2014-08-13 17:38:03.123444 7fe9bae887a0 20 read_log 194209'13507 (194209'13469) delete da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.27.0:819 2014-08-12 14:05:54.458455 2014-08-13 17:38:03.123456 7fe9bae887a0 20 read_log 194209'13508 (194209'13470) delete b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:820 2014-08-12 14:05:54.461127 2014-08-13 17:38:03.123467 7fe9bae887a0 20 read_log 194209'13509 (194209'13471) delete 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.27.0:821 2014-08-12 14:05:54.461546 2014-08-13 17:38:03.123502 7fe9bae887a0 20 read_log 194209'13510 (194209'13472) delete 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.27.0:822 2014-08-12 14:05:54.461907 2014-08-13 17:38:03.123518 7fe9bae887a0 20 read_log 194209'13511 (194209'13473) delete 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.27.0:823 2014-08-12 14:05:54.465263 2014-08-13 17:38:03.123530 7fe9bae887a0 20 read_log 194209'13512 (194209'13474) delete 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.27.0:824 2014-08-12 14:05:54.465660 2014-08-13 17:38:03.123542 7fe9bae887a0 20 read_log 194209'13513 (194209'13475) delete 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:825 2014-08-12 14:05:54.466510 2014-08-13 17:38:03.123554 7fe9bae887a0 20 read_log 194209'13514 (194209'13476) delete a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:826 2014-08-12 14:05:54.469696 2014-08-13 17:38:03.123566 7fe9bae887a0 20 read_log 194209'13515 (194209'13481) delete b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:827 2014-08-12 14:05:54.471230 2014-08-13 17:38:03.123579 7fe9bae887a0 20 read_log 194209'13516 (194209'13487) delete 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:828 2014-08-12 14:05:54.475403 2014-08-13 17:38:03.123590 7fe9bae887a0 20 read_log 194209'13517 (194209'13493) delete ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:829 2014-08-12 14:05:54.476720 2014-08-13 17:38:03.123603 7fe9bae887a0 20 read_log 194209'13518 (194209'13495) delete 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:830 2014-08-12 14:05:54.477343 2014-08-13 17:38:03.123615 7fe9bae887a0 20 read_log 194209'13519 (194209'13506) delete 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:831 2014-08-12 14:05:54.480370 2014-08-13 17:38:03.123627 7fe9bae887a0 20 read_log 194209'13520 (194209'13489) delete 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.27.0:842 2014-08-12 14:06:04.569929 2014-08-13 17:38:03.123640 7fe9bae887a0 20 read_log 194209'13521 (194209'13490) delete 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.27.0:843 2014-08-12 14:06:04.570353 2014-08-13 17:38:03.123652 7fe9bae887a0 20 read_log 194209'13522 (194209'13491) delete f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.27.0:844 2014-08-12 14:06:04.570709 2014-08-13 17:38:03.123664 7fe9bae887a0 20 read_log 194209'13523 (194209'13492) delete 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.27.0:845 2014-08-12 14:06:04.571106 2014-08-13 17:38:03.123676 7fe9bae887a0 20 read_log 194209'13524 (194209'13494) delete f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.27.0:846 2014-08-12 14:06:04.573437 2014-08-13 17:38:03.123689 7fe9bae887a0 20 read_log 194209'13525 (194209'13496) delete d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.27.0:847 2014-08-12 14:06:04.574209 2014-08-13 17:38:03.123701 7fe9bae887a0 20 read_log 194209'13526 (194209'13477) delete 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.27.0:848 2014-08-12 14:06:04.607287 2014-08-13 17:38:03.123713 7fe9bae887a0 20 read_log 194209'13527 (194209'13478) delete 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.27.0:849 2014-08-12 14:06:04.607673 2014-08-13 17:38:03.123725 7fe9bae887a0 20 read_log 194209'13528 (194209'13479) delete 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.27.0:850 2014-08-12 14:06:04.608046 2014-08-13 17:38:03.123737 7fe9bae887a0 20 read_log 194209'13529 (194209'13480) delete 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.27.0:851 2014-08-12 14:06:04.608441 2014-08-13 17:38:03.123757 7fe9bae887a0 20 read_log 194209'13530 (194209'13482) delete 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.27.0:852 2014-08-12 14:06:04.611784 2014-08-13 17:38:03.123770 7fe9bae887a0 20 read_log 194209'13531 (194209'13483) delete 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.27.0:853 2014-08-12 14:06:04.612187 2014-08-13 17:38:03.123782 7fe9bae887a0 20 read_log 194209'13532 (194209'13484) delete 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.27.0:854 2014-08-12 14:06:04.612495 2014-08-13 17:38:03.123796 7fe9bae887a0 20 read_log 194209'13533 (194209'13485) delete ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.27.0:855 2014-08-12 14:06:04.613235 2014-08-13 17:38:03.123808 7fe9bae887a0 20 read_log 194215'13534 (194215'13534) modify 3bf/hit_set_24.3bf_archive_2014-08-12 15:32:25.978491_2014-08-12 15:51:03.552127/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:03.552625 2014-08-13 17:38:03.123824 7fe9bae887a0 20 read_log 194215'13535 (193305'13410) delete 3bf/hit_set_24.3bf_archive_2014-08-11 18:38:15.792949_2014-08-11 18:43:20.626482/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:03.552625 2014-08-13 17:38:03.123836 7fe9bae887a0 20 read_log 194215'13536 (192888'13197) clean e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by osd.27.0:1826 2014-08-12 15:51:03.556557 2014-08-13 17:38:03.123850 7fe9bae887a0 20 read_log 194215'13537 (194215'13536) delete e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.2174595.0:198138 0.000000 2014-08-13 17:38:03.123862 7fe9bae887a0 20 read_log 194215'13538 (193305'13451) clean ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by osd.27.0:1828 2014-08-12 15:51:03.564477 2014-08-13 17:38:03.123874 7fe9bae887a0 20 read_log 194215'13539 (194215'13538) delete ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.2174595.0:198140 0.000000 2014-08-13 17:38:03.123884 7fe9bae887a0 20 read_log 194215'13540 (192834'12364) clean 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by osd.27.0:1830 2014-08-12 15:51:03.572221 2014-08-13 17:38:03.123899 7fe9bae887a0 20 read_log 194215'13541 (194215'13540) delete 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.2174595.0:198142 0.000000 2014-08-13 17:38:03.123909 7fe9bae887a0 20 read_log 194215'13542 (193305'13447) clean 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by osd.27.0:1832 2014-08-12 15:51:03.579839 2014-08-13 17:38:03.123921 7fe9bae887a0 20 read_log 194215'13543 (194215'13542) delete 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.2174595.0:198144 0.000000 2014-08-13 17:38:03.123932 7fe9bae887a0 20 read_log 194215'13544 (193305'13446) clean 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by osd.27.0:1834 2014-08-12 15:51:03.587534 2014-08-13 17:38:03.123944 7fe9bae887a0 20 read_log 194215'13545 (194215'13544) delete 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.2174595.0:198146 0.000000 2014-08-13 17:38:03.123955 7fe9bae887a0 20 read_log 194215'13546 (192888'13343) clean 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by osd.27.0:1836 2014-08-12 15:51:03.594892 2014-08-13 17:38:03.123967 7fe9bae887a0 20 read_log 194215'13547 (194215'13546) delete 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.2174595.0:198148 0.000000 2014-08-13 17:38:03.123977 7fe9bae887a0 20 read_log 194215'13548 (192888'13188) clean f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by osd.27.0:1838 2014-08-12 15:51:03.602046 2014-08-13 17:38:03.124001 7fe9bae887a0 20 read_log 194215'13549 (194215'13548) delete f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.2174595.0:198150 0.000000 2014-08-13 17:38:03.124014 7fe9bae887a0 20 read_log 194215'13550 (192834'12448) clean 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by osd.27.0:1840 2014-08-12 15:51:03.609361 2014-08-13 17:38:03.124026 7fe9bae887a0 20 read_log 194215'13551 (194215'13550) delete 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.2174595.0:198152 0.000000 2014-08-13 17:38:03.124037 7fe9bae887a0 20 read_log 194215'13552 (192888'13195) clean 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by osd.27.0:1842 2014-08-12 15:51:03.616578 2014-08-13 17:38:03.124051 7fe9bae887a0 20 read_log 194215'13553 (194215'13552) delete 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.2174595.0:198154 0.000000 2014-08-13 17:38:03.124062 7fe9bae887a0 20 read_log 194215'13554 (192888'13150) clean 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by osd.27.0:1844 2014-08-12 15:51:03.623710 2014-08-13 17:38:03.124074 7fe9bae887a0 20 read_log 194215'13555 (194215'13554) delete 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.2174595.0:198156 0.000000 2014-08-13 17:38:03.124087 7fe9bae887a0 20 read_log 194215'13556 (192834'12391) clean d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by osd.27.0:1846 2014-08-12 15:51:03.630814 2014-08-13 17:38:03.124100 7fe9bae887a0 20 read_log 194215'13557 (194215'13556) delete d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.2174595.0:198158 0.000000 2014-08-13 17:38:03.124110 7fe9bae887a0 20 read_log 194215'13558 (192888'13189) clean 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by osd.27.0:1848 2014-08-12 15:51:03.638301 2014-08-13 17:38:03.124124 7fe9bae887a0 20 read_log 194215'13559 (194215'13558) delete 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.2174595.0:198160 0.000000 2014-08-13 17:38:03.124135 7fe9bae887a0 20 read_log 194215'13560 (192888'13379) clean 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by osd.27.0:1850 2014-08-12 15:51:03.645409 2014-08-13 17:38:03.124147 7fe9bae887a0 20 read_log 194215'13561 (194215'13560) delete 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.2174595.0:198162 0.000000 2014-08-13 17:38:03.124164 7fe9bae887a0 20 read_log 194215'13562 (194209'13486) delete f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.2174595.0:198164 0.000000 2014-08-13 17:38:03.124177 7fe9bae887a0 20 read_log 194215'13563 (194209'13488) delete cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.2174595.0:198166 0.000000 2014-08-13 17:38:03.124188 7fe9bae887a0 20 read_log 194215'13564 (193305'13437) clean d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by osd.27.0:1854 2014-08-12 15:51:03.662111 2014-08-13 17:38:03.124200 7fe9bae887a0 20 read_log 194215'13565 (194215'13564) delete d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.2174595.0:198168 0.000000 2014-08-13 17:38:03.124211 7fe9bae887a0 20 read_log 194215'13566 (192834'12905) clean 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by osd.27.0:1856 2014-08-12 15:51:03.668634 2014-08-13 17:38:03.124223 7fe9bae887a0 20 read_log 194215'13567 (194215'13566) delete 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.2174595.0:198170 0.000000 2014-08-13 17:38:03.124233 7fe9bae887a0 20 read_log 194215'13568 (192834'12982) clean ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by osd.27.0:1858 2014-08-12 15:51:03.675781 2014-08-13 17:38:03.124252 7fe9bae887a0 20 read_log 194215'13569 (194215'13568) delete ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.2174595.0:198172 0.000000 2014-08-13 17:38:03.124266 7fe9bae887a0 20 read_log 194215'13570 (192834'12893) clean d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by osd.27.0:1860 2014-08-12 15:51:03.683061 2014-08-13 17:38:03.124278 7fe9bae887a0 20 read_log 194215'13571 (194215'13570) delete d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.2174595.0:198174 0.000000 2014-08-13 17:38:03.124289 7fe9bae887a0 20 read_log 194215'13572 (192888'13206) clean adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by osd.27.0:1862 2014-08-12 15:51:03.690399 2014-08-13 17:38:03.124304 7fe9bae887a0 20 read_log 194215'13573 (194215'13572) delete adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.2174595.0:198176 0.000000 2014-08-13 17:38:03.124315 7fe9bae887a0 20 read_log 194215'13574 (192888'13337) clean e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by osd.27.0:1864 2014-08-12 15:51:03.697243 2014-08-13 17:38:03.124327 7fe9bae887a0 20 read_log 194215'13575 (194215'13574) delete e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.2174595.0:198178 0.000000 2014-08-13 17:38:03.124337 7fe9bae887a0 20 read_log 194215'13576 (192888'13388) clean a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by osd.27.0:1866 2014-08-12 15:51:03.704200 2014-08-13 17:38:03.124349 7fe9bae887a0 20 read_log 194215'13577 (194215'13576) delete a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.2174595.0:198180 0.000000 2014-08-13 17:38:03.124360 7fe9bae887a0 20 read_log 194215'13578 (192888'13351) clean 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by osd.27.0:1868 2014-08-12 15:51:03.711403 2014-08-13 17:38:03.124372 7fe9bae887a0 20 read_log 194215'13579 (194215'13578) delete 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.2174595.0:198182 0.000000 2014-08-13 17:38:03.124382 7fe9bae887a0 20 read_log 194215'13580 (192888'13348) clean ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by osd.27.0:1870 2014-08-12 15:51:03.718289 2014-08-13 17:38:03.124394 7fe9bae887a0 20 read_log 194215'13581 (194215'13580) delete ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.2174595.0:198184 0.000000 2014-08-13 17:38:03.124425 7fe9bae887a0 20 read_log 194215'13582 (192888'13370) clean 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by osd.27.0:1872 2014-08-12 15:51:03.725262 2014-08-13 17:38:03.124440 7fe9bae887a0 20 read_log 194215'13583 (194215'13582) delete 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.2174595.0:198186 0.000000 2014-08-13 17:38:03.124451 7fe9bae887a0 20 read_log 194215'13584 (192888'13377) clean 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by osd.27.0:1874 2014-08-12 15:51:03.732332 2014-08-13 17:38:03.124463 7fe9bae887a0 20 read_log 194215'13585 (194215'13584) delete 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.2174595.0:198188 0.000000 2014-08-13 17:38:03.124474 7fe9bae887a0 20 read_log 194215'13586 (192834'12869) clean d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by osd.27.0:1876 2014-08-12 15:51:03.739291 2014-08-13 17:38:03.124486 7fe9bae887a0 20 read_log 194215'13587 (194215'13586) delete d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.2174595.0:198190 0.000000 2014-08-13 17:38:03.124497 7fe9bae887a0 20 read_log 194215'13588 (193305'13456) clean 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by osd.27.0:1878 2014-08-12 15:51:03.746374 2014-08-13 17:38:03.124516 7fe9bae887a0 20 read_log 194215'13589 (194215'13588) delete 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.2174595.0:198192 0.000000 2014-08-13 17:38:03.124527 7fe9bae887a0 20 read_log 194215'13590 (192888'13368) clean 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by osd.27.0:1880 2014-08-12 15:51:03.753037 2014-08-13 17:38:03.124541 7fe9bae887a0 20 read_log 194215'13591 (194215'13590) delete 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.2174595.0:198194 0.000000 2014-08-13 17:38:03.124552 7fe9bae887a0 20 read_log 194215'13592 (192888'13332) clean bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by osd.27.0:1882 2014-08-12 15:51:03.759920 2014-08-13 17:38:03.124564 7fe9bae887a0 20 read_log 194215'13593 (194215'13592) delete bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.2174595.0:198196 0.000000 2014-08-13 17:38:03.124575 7fe9bae887a0 20 read_log 194215'13594 (193305'13439) clean 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by osd.27.0:1884 2014-08-12 15:51:03.766340 2014-08-13 17:38:03.124587 7fe9bae887a0 20 read_log 194215'13595 (194215'13594) delete 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.2174595.0:198198 0.000000 2014-08-13 17:38:03.124597 7fe9bae887a0 20 read_log 194215'13596 (192834'12857) clean c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by osd.27.0:1886 2014-08-12 15:51:03.773666 2014-08-13 17:38:03.124609 7fe9bae887a0 20 read_log 194215'13597 (194215'13596) delete c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.2174595.0:198200 0.000000 2014-08-13 17:38:03.124620 7fe9bae887a0 20 read_log 194215'13598 (192834'12396) clean 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by osd.27.0:1888 2014-08-12 15:51:03.780778 2014-08-13 17:38:03.124632 7fe9bae887a0 20 read_log 194215'13599 (194215'13598) delete 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.2174595.0:198202 0.000000 2014-08-13 17:38:03.124642 7fe9bae887a0 20 read_log 194215'13600 (192888'13148) clean 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by osd.27.0:1890 2014-08-12 15:51:03.788052 2014-08-13 17:38:03.124654 7fe9bae887a0 20 read_log 194215'13601 (194215'13600) delete 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.2174595.0:198204 0.000000 2014-08-13 17:38:03.124665 7fe9bae887a0 20 read_log 194215'13602 (193305'13450) clean 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by osd.27.0:1892 2014-08-12 15:51:03.795059 2014-08-13 17:38:03.124677 7fe9bae887a0 20 read_log 194215'13603 (194215'13602) delete 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.2174595.0:198206 0.000000 2014-08-13 17:38:03.124687 7fe9bae887a0 20 read_log 194215'13604 (193305'13442) clean df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by osd.27.0:1894 2014-08-12 15:51:03.803358 2014-08-13 17:38:03.124699 7fe9bae887a0 20 read_log 194215'13605 (194215'13604) delete df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.2174595.0:198208 0.000000 2014-08-13 17:38:03.124710 7fe9bae887a0 20 read_log 194215'13606 (192834'12325) clean e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by osd.27.0:1896 2014-08-12 15:51:03.810112 2014-08-13 17:38:03.124722 7fe9bae887a0 20 read_log 194215'13607 (194215'13606) delete e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by client.2174595.0:198210 0.000000 2014-08-13 17:38:03.124732 7fe9bae887a0 20 read_log 194215'13608 (192888'13200) clean 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by osd.27.0:1898 2014-08-12 15:51:03.817288 2014-08-13 17:38:03.124753 7fe9bae887a0 20 read_log 194215'13609 (194215'13608) delete 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.2174595.0:198212 0.000000 2014-08-13 17:38:03.124764 7fe9bae887a0 20 read_log 194215'13610 (192834'12888) clean 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by osd.27.0:1900 2014-08-12 15:51:03.824884 2014-08-13 17:38:03.124777 7fe9bae887a0 20 read_log 194215'13611 (194215'13610) delete 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.2174595.0:198214 0.000000 2014-08-13 17:38:03.124787 7fe9bae887a0 20 read_log 194215'13612 (192834'12387) clean 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by osd.27.0:1902 2014-08-12 15:51:03.831667 2014-08-13 17:38:03.124799 7fe9bae887a0 20 read_log 194215'13613 (194215'13612) delete 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.2174595.0:198216 0.000000 2014-08-13 17:38:03.124810 7fe9bae887a0 20 read_log 194215'13614 (192834'12416) clean c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by osd.27.0:1904 2014-08-12 15:51:03.838089 2014-08-13 17:38:03.124822 7fe9bae887a0 20 read_log 194215'13615 (194215'13614) delete c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.2174595.0:198218 0.000000 2014-08-13 17:38:03.124832 7fe9bae887a0 20 read_log 194215'13616 (192888'13373) clean a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by osd.27.0:1906 2014-08-12 15:51:03.845357 2014-08-13 17:38:03.124845 7fe9bae887a0 20 read_log 194215'13617 (194215'13616) delete a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.2174595.0:198220 0.000000 2014-08-13 17:38:03.124855 7fe9bae887a0 20 read_log 194215'13618 (192888'13152) clean b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by osd.27.0:1908 2014-08-12 15:51:03.852813 2014-08-13 17:38:03.124867 7fe9bae887a0 20 read_log 194215'13619 (194215'13618) delete b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.2174595.0:198222 0.000000 2014-08-13 17:38:03.124878 7fe9bae887a0 20 read_log 194215'13620 (192888'13184) clean 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by osd.27.0:1910 2014-08-12 15:51:03.860462 2014-08-13 17:38:03.124890 7fe9bae887a0 20 read_log 194215'13621 (194215'13620) delete 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.2174595.0:198224 0.000000 2014-08-13 17:38:03.124901 7fe9bae887a0 20 read_log 194215'13622 (192834'13003) clean 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by osd.27.0:1912 2014-08-12 15:51:03.867653 2014-08-13 17:38:03.124913 7fe9bae887a0 20 read_log 194215'13623 (194215'13622) delete 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.2174595.0:198226 0.000000 2014-08-13 17:38:03.124924 7fe9bae887a0 20 read_log 194215'13624 (192888'13369) clean 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by osd.27.0:1914 2014-08-12 15:51:03.874841 2014-08-13 17:38:03.124936 7fe9bae887a0 20 read_log 194215'13625 (194215'13624) delete 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.2174595.0:198228 0.000000 2014-08-13 17:38:03.124946 7fe9bae887a0 20 read_log 194215'13626 (193305'13448) clean 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by osd.27.0:1916 2014-08-12 15:51:03.882260 2014-08-13 17:38:03.124958 7fe9bae887a0 20 read_log 194215'13627 (194215'13626) delete 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.2174595.0:198230 0.000000 2014-08-13 17:38:03.124969 7fe9bae887a0 20 read_log 194215'13628 (192888'13371) clean 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by osd.27.0:1918 2014-08-12 15:51:03.889134 2014-08-13 17:38:03.124988 7fe9bae887a0 20 read_log 194215'13629 (194215'13628) delete 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.2174595.0:198232 0.000000 2014-08-13 17:38:03.125001 7fe9bae887a0 20 read_log 194215'13630 (192888'13154) clean dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by osd.27.0:1920 2014-08-12 15:51:03.895933 2014-08-13 17:38:03.125014 7fe9bae887a0 20 read_log 194215'13631 (194215'13630) delete dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.2174595.0:198234 0.000000 2014-08-13 17:38:03.125025 7fe9bae887a0 20 read_log 194215'13632 (192888'13331) clean 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by osd.27.0:1922 2014-08-12 15:51:03.902609 2014-08-13 17:38:03.125037 7fe9bae887a0 20 read_log 194215'13633 (194215'13632) delete 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.2174595.0:198236 0.000000 2014-08-13 17:38:03.125047 7fe9bae887a0 20 read_log 194215'13634 (192834'12326) clean 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by osd.27.0:1924 2014-08-12 15:51:03.909926 2014-08-13 17:38:03.125061 7fe9bae887a0 20 read_log 194215'13635 (194215'13634) delete 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by client.2174595.0:198238 0.000000 2014-08-13 17:38:03.125072 7fe9bae887a0 20 read_log 194215'13636 (193305'13452) clean f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by osd.27.0:1926 2014-08-12 15:51:03.917399 2014-08-13 17:38:03.125084 7fe9bae887a0 20 read_log 194215'13637 (194215'13636) delete f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.2174595.0:198240 0.000000 2014-08-13 17:38:03.125095 7fe9bae887a0 20 read_log 194215'13638 (192888'13204) clean c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by osd.27.0:1928 2014-08-12 15:51:03.924637 2014-08-13 17:38:03.125109 7fe9bae887a0 20 read_log 194215'13639 (194215'13638) delete c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.2174595.0:198242 0.000000 2014-08-13 17:38:03.125120 7fe9bae887a0 20 read_log 194215'13640 (192834'12993) clean 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by osd.27.0:1930 2014-08-12 15:51:03.932527 2014-08-13 17:38:03.125132 7fe9bae887a0 20 read_log 194215'13641 (194215'13640) delete 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.2174595.0:198244 0.000000 2014-08-13 17:38:03.125146 7fe9bae887a0 20 read_log 194215'13642 (192834'12933) clean 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by osd.27.0:1932 2014-08-12 15:51:03.940050 2014-08-13 17:38:03.125162 7fe9bae887a0 20 read_log 194215'13643 (194215'13642) delete 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.2174595.0:198246 0.000000 2014-08-13 17:38:03.125173 7fe9bae887a0 20 read_log 194215'13644 (192888'13357) clean fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by osd.27.0:1934 2014-08-12 15:51:03.947391 2014-08-13 17:38:03.125188 7fe9bae887a0 20 read_log 194215'13645 (194215'13644) delete fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.2174595.0:198248 0.000000 2014-08-13 17:38:03.125199 7fe9bae887a0 20 read_log 194215'13646 (192834'12963) clean 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by osd.27.0:1936 2014-08-12 15:51:03.955321 2014-08-13 17:38:03.125212 7fe9bae887a0 20 read_log 194215'13647 (194215'13646) delete 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.2174595.0:198250 0.000000 2014-08-13 17:38:03.125222 7fe9bae887a0 20 read_log 194215'13648 (192888'13157) clean ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by osd.27.0:1938 2014-08-12 15:51:03.962503 2014-08-13 17:38:03.125258 7fe9bae887a0 20 read_log 194215'13649 (194215'13648) delete ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.2174595.0:198252 0.000000 2014-08-13 17:38:03.125273 7fe9bae887a0 20 read_log 194215'13650 (192888'13181) clean 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by osd.27.0:1940 2014-08-12 15:51:03.969296 2014-08-13 17:38:03.125285 7fe9bae887a0 20 read_log 194215'13651 (194215'13650) delete 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.2174595.0:198254 0.000000 2014-08-13 17:38:03.125296 7fe9bae887a0 20 read_log 194215'13652 (192888'13167) clean 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by osd.27.0:1942 2014-08-12 15:51:03.976613 2014-08-13 17:38:03.125311 7fe9bae887a0 20 read_log 194215'13653 (194215'13652) delete 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.2174595.0:198256 0.000000 2014-08-13 17:38:03.125322 7fe9bae887a0 20 read_log 194215'13654 (192834'12992) clean fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by osd.27.0:1944 2014-08-12 15:51:03.984186 2014-08-13 17:38:03.125336 7fe9bae887a0 20 read_log 194215'13655 (194215'13654) delete fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.2174595.0:198258 0.000000 2014-08-13 17:38:03.125347 7fe9bae887a0 20 read_log 194215'13656 (193305'13438) clean 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by osd.27.0:1946 2014-08-12 15:51:03.991423 2014-08-13 17:38:03.125359 7fe9bae887a0 20 read_log 194215'13657 (194215'13656) delete 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.2174595.0:198260 0.000000 2014-08-13 17:38:03.125370 7fe9bae887a0 20 read_log 194215'13658 (192888'13372) clean cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by osd.27.0:1948 2014-08-12 15:51:03.999509 2014-08-13 17:38:03.125382 7fe9bae887a0 20 read_log 194215'13659 (194215'13658) delete cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.2174595.0:198262 0.000000 2014-08-13 17:38:03.125393 7fe9bae887a0 20 read_log 194215'13660 (192888'13190) clean 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by osd.27.0:1950 2014-08-12 15:51:04.007296 2014-08-13 17:38:03.125406 7fe9bae887a0 20 read_log 194215'13661 (194215'13660) delete 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.2174595.0:198264 0.000000 2014-08-13 17:38:03.125416 7fe9bae887a0 20 read_log 194215'13662 (192888'13380) clean e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by osd.27.0:1952 2014-08-12 15:51:04.014366 2014-08-13 17:38:03.125429 7fe9bae887a0 20 read_log 194215'13663 (194215'13662) delete e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.2174595.0:198266 0.000000 2014-08-13 17:38:03.125439 7fe9bae887a0 20 read_log 194215'13664 (192834'12938) clean 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by osd.27.0:1954 2014-08-12 15:51:04.021270 2014-08-13 17:38:03.125452 7fe9bae887a0 20 read_log 194215'13665 (194215'13664) delete 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.2174595.0:198268 0.000000 2014-08-13 17:38:03.125463 7fe9bae887a0 20 read_log 194215'13666 (192834'12903) clean a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by osd.27.0:1956 2014-08-12 15:51:04.028683 2014-08-13 17:38:03.125475 7fe9bae887a0 20 read_log 194215'13667 (194215'13666) delete a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.2174595.0:198270 0.000000 2014-08-13 17:38:03.125486 7fe9bae887a0 20 read_log 194215'13668 (193305'13457) clean 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by osd.27.0:1958 2014-08-12 15:51:04.036325 2014-08-13 17:38:03.125505 7fe9bae887a0 20 read_log 194215'13669 (194215'13668) delete 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.2174595.0:198272 0.000000 2014-08-13 17:38:03.125517 7fe9bae887a0 20 read_log 194215'13670 (192888'13210) clean f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by osd.27.0:1960 2014-08-12 15:51:04.043476 2014-08-13 17:38:03.125529 7fe9bae887a0 20 read_log 194215'13671 (194215'13670) delete f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.2174595.0:198274 0.000000 2014-08-13 17:38:03.125540 7fe9bae887a0 20 read_log 194215'13672 (192834'12360) clean 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by osd.27.0:1962 2014-08-12 15:51:04.050644 2014-08-13 17:38:03.125552 7fe9bae887a0 20 read_log 194215'13673 (194215'13672) delete 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.2174595.0:198276 0.000000 2014-08-13 17:38:03.125563 7fe9bae887a0 20 read_log 194215'13674 (192888'13340) clean bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by osd.27.0:1964 2014-08-12 15:51:04.057338 2014-08-13 17:38:03.125590 7fe9bae887a0 20 read_log 194215'13675 (194215'13674) delete bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.2174595.0:198278 0.000000 2014-08-13 17:38:03.125601 7fe9bae887a0 20 read_log 194215'13676 (192834'12930) clean c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by osd.27.0:1966 2014-08-12 15:51:04.065172 2014-08-13 17:38:03.125614 7fe9bae887a0 20 read_log 194215'13677 (194215'13676) delete c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.2174595.0:198280 0.000000 2014-08-13 17:38:03.125624 7fe9bae887a0 20 read_log 194215'13678 (192888'13365) clean 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by osd.27.0:1968 2014-08-12 15:51:04.072076 2014-08-13 17:38:03.125637 7fe9bae887a0 20 read_log 194215'13679 (194215'13678) delete 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.2174595.0:198282 0.000000 2014-08-13 17:38:03.125647 7fe9bae887a0 20 read_log 194215'13680 (192834'13004) clean f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by osd.27.0:1970 2014-08-12 15:51:04.079229 2014-08-13 17:38:03.125659 7fe9bae887a0 20 read_log 194215'13681 (194215'13680) delete f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.2174595.0:198284 0.000000 2014-08-13 17:38:03.125670 7fe9bae887a0 20 read_log 194215'13682 (192834'12356) clean e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by osd.27.0:1972 2014-08-12 15:51:04.086664 2014-08-13 17:38:03.125682 7fe9bae887a0 20 read_log 194215'13683 (194215'13682) delete e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.2174595.0:198286 0.000000 2014-08-13 17:38:03.125693 7fe9bae887a0 20 read_log 194215'13684 (193305'13443) clean 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by osd.27.0:1974 2014-08-12 15:51:04.093874 2014-08-13 17:38:03.125706 7fe9bae887a0 20 read_log 194215'13685 (194215'13684) delete 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.2174595.0:198288 0.000000 2014-08-13 17:38:03.125716 7fe9bae887a0 20 read_log 194215'13686 (192834'12974) clean b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by osd.27.0:1976 2014-08-12 15:51:04.101702 2014-08-13 17:38:03.125728 7fe9bae887a0 20 read_log 194215'13687 (194215'13686) delete b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.2174595.0:198290 0.000000 2014-08-13 17:38:03.125739 7fe9bae887a0 20 read_log 194215'13688 (192888'13147) clean 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by osd.27.0:1978 2014-08-12 15:51:04.108349 2014-08-13 17:38:03.125762 7fe9bae887a0 20 read_log 194215'13689 (194215'13688) delete 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.2174595.0:198292 0.000000 2014-08-13 17:38:03.125773 7fe9bae887a0 20 read_log 194215'13690 (192834'12920) clean 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by osd.27.0:1980 2014-08-12 15:51:04.114882 2014-08-13 17:38:03.125786 7fe9bae887a0 20 read_log 194215'13691 (194215'13690) delete 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.2174595.0:198294 0.000000 2014-08-13 17:38:03.125797 7fe9bae887a0 20 read_log 194215'13692 (192888'13162) clean 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by osd.27.0:1982 2014-08-12 15:51:04.121746 2014-08-13 17:38:03.125809 7fe9bae887a0 20 read_log 194215'13693 (194215'13692) delete 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.2174595.0:198296 0.000000 2014-08-13 17:38:03.125819 7fe9bae887a0 20 read_log 194215'13694 (192834'12419) clean 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by osd.27.0:1984 2014-08-12 15:51:04.128895 2014-08-13 17:38:03.125831 7fe9bae887a0 20 read_log 194215'13695 (194215'13694) delete 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.2174595.0:198298 0.000000 2014-08-13 17:38:03.125842 7fe9bae887a0 20 read_log 194215'13696 (192888'13356) clean e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by osd.27.0:1986 2014-08-12 15:51:04.135952 2014-08-13 17:38:03.125854 7fe9bae887a0 20 read_log 194215'13697 (194215'13696) delete e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.2174595.0:198300 0.000000 2014-08-13 17:38:03.125865 7fe9bae887a0 20 read_log 194215'13698 (192888'13349) clean 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by osd.27.0:1988 2014-08-12 15:51:04.142672 2014-08-13 17:38:03.125877 7fe9bae887a0 20 read_log 194215'13699 (194215'13698) delete 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.2174595.0:198302 0.000000 2014-08-13 17:38:03.125887 7fe9bae887a0 20 read_log 194215'13700 (192888'13208) clean d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by osd.27.0:1990 2014-08-12 15:51:04.149916 2014-08-13 17:38:03.125899 7fe9bae887a0 20 read_log 194215'13701 (194215'13700) delete d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.2174595.0:198304 0.000000 2014-08-13 17:38:03.125910 7fe9bae887a0 20 read_log 194215'13702 (192834'12347) clean acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by osd.27.0:1992 2014-08-12 15:51:04.157011 2014-08-13 17:38:03.125922 7fe9bae887a0 20 read_log 194215'13703 (194215'13702) delete acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.2174595.0:198306 0.000000 2014-08-13 17:38:03.125933 7fe9bae887a0 20 read_log 194215'13704 (192888'13346) clean c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by osd.27.0:1994 2014-08-12 15:51:04.164787 2014-08-13 17:38:03.125945 7fe9bae887a0 20 read_log 194215'13705 (194215'13704) delete c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.2174595.0:198308 0.000000 2014-08-13 17:38:03.125956 7fe9bae887a0 20 read_log 194215'13706 (192888'13199) clean 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by osd.27.0:1996 2014-08-12 15:51:04.171223 2014-08-13 17:38:03.125968 7fe9bae887a0 20 read_log 194215'13707 (194215'13706) delete 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.2174595.0:198310 0.000000 2014-08-13 17:38:03.125978 7fe9bae887a0 20 read_log 194215'13708 (192834'12400) clean 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by osd.27.0:1998 2014-08-12 15:51:04.178011 2014-08-13 17:38:03.125997 7fe9bae887a0 20 read_log 194215'13709 (194215'13708) delete 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.2174595.0:198312 0.000000 2014-08-13 17:38:03.126013 7fe9bae887a0 20 read_log 194215'13710 (192888'13361) clean 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by osd.27.0:2000 2014-08-12 15:51:04.185073 2014-08-13 17:38:03.126025 7fe9bae887a0 20 read_log 194215'13711 (194215'13710) delete 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.2174595.0:198314 0.000000 2014-08-13 17:38:03.126036 7fe9bae887a0 20 read_log 194215'13712 (192834'12450) clean f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by osd.27.0:2002 2014-08-12 15:51:04.191644 2014-08-13 17:38:03.126051 7fe9bae887a0 20 read_log 194215'13713 (194215'13712) delete f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.2174595.0:198316 0.000000 2014-08-13 17:38:03.126062 7fe9bae887a0 20 read_log 194215'13714 (193305'13458) clean 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by osd.27.0:2004 2014-08-12 15:51:04.198377 2014-08-13 17:38:03.126074 7fe9bae887a0 20 read_log 194215'13715 (194215'13714) delete 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.2174595.0:198318 0.000000 2014-08-13 17:38:03.126085 7fe9bae887a0 20 read_log 194215'13716 (192888'13178) clean 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by osd.27.0:2006 2014-08-12 15:51:04.206475 2014-08-13 17:38:03.126099 7fe9bae887a0 20 read_log 194215'13717 (194215'13716) delete 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.2174595.0:198320 0.000000 2014-08-13 17:38:03.126110 7fe9bae887a0 20 read_log 194215'13718 (192888'13389) clean afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by osd.27.0:2008 2014-08-12 15:51:04.213946 2014-08-13 17:38:03.126122 7fe9bae887a0 20 read_log 194215'13719 (194215'13718) delete afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.2174595.0:198322 0.000000 2014-08-13 17:38:03.126133 7fe9bae887a0 20 read_log 194215'13720 (192888'13160) clean 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by osd.27.0:2010 2014-08-12 15:51:04.221130 2014-08-13 17:38:03.126147 7fe9bae887a0 20 read_log 194215'13721 (194215'13720) delete 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.2174595.0:198324 0.000000 2014-08-13 17:38:03.126164 7fe9bae887a0 20 read_log 194215'13722 (193305'13453) clean ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by osd.27.0:2012 2014-08-12 15:51:04.227484 2014-08-13 17:38:03.126179 7fe9bae887a0 20 read_log 194215'13723 (194215'13722) delete ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.2174595.0:198326 0.000000 2014-08-13 17:38:03.126190 7fe9bae887a0 20 read_log 194215'13724 (194215'13724) modify 3bf/hit_set_24.3bf_archive_2014-08-12 15:51:03.552629_2014-08-12 17:24:41.785138/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.785371 2014-08-13 17:38:03.126202 7fe9bae887a0 20 read_log 194215'13725 (194215'13534) delete 3bf/hit_set_24.3bf_archive_2014-08-12 15:32:25.978491_2014-08-12 15:51:03.552127/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.785371 2014-08-13 17:38:03.126217 7fe9bae887a0 20 read_log 194215'13726 (0'0) promote e56603bf/benchmark_data_storage0111-ib.pocstor_34628_object485/head//24 by osd.27.0:2353 2014-08-12 17:24:41.788558 2014-08-13 17:38:03.126229 7fe9bae887a0 20 read_log 194215'13727 (194215'13726) modify e56603bf/benchmark_data_storage0111-ib.pocstor_34628_object485/head//24 by client.2193759.0:486 2014-08-12 17:24:41.780036 2014-08-13 17:38:03.126249 7fe9bae887a0 20 read_log 194370'13728 (0'0) promote ffb0abbf/benchmark_data_storage0111-ib.pocstor_12495_object1602/head//24 by osd.19.0:4650 2014-08-12 20:30:20.388956 2014-08-13 17:38:03.126264 7fe9bae887a0 20 read_log 194370'13729 (194370'13728) modify ffb0abbf/benchmark_data_storage0111-ib.pocstor_12495_object1602/head//24 by client.1508531.0:1603 2014-08-12 20:30:20.382979 2014-08-13 17:38:03.126277 7fe9bae887a0 20 read_log 194370'13730 (0'0) promote 2d4c3fbf/benchmark_data_storage0111-ib.pocstor_12495_object1903/head//24 by osd.19.0:4658 2014-08-12 20:30:21.481769 2014-08-13 17:38:03.126289 7fe9bae887a0 20 read_log 194370'13731 (194370'13730) modify 2d4c3fbf/benchmark_data_storage0111-ib.pocstor_12495_object1903/head//24 by client.1508531.0:1904 2014-08-12 20:30:21.475144 2014-08-13 17:38:03.126301 7fe9bae887a0 20 read_log 194370'13732 (0'0) promote e1c40fbf/benchmark_data_storage0111-ib.pocstor_12495_object2723/head//24 by osd.19.0:4664 2014-08-12 20:30:24.540766 2014-08-13 17:38:03.126313 7fe9bae887a0 20 read_log 194370'13733 (194370'13732) modify e1c40fbf/benchmark_data_storage0111-ib.pocstor_12495_object2723/head//24 by client.1508531.0:2724 2014-08-12 20:30:24.534664 2014-08-13 17:38:03.126325 7fe9bae887a0 20 read_log 194370'13734 (0'0) promote 7028d3bf/benchmark_data_storage0111-ib.pocstor_12495_object3559/head//24 by osd.19.0:4672 2014-08-12 20:30:27.633709 2014-08-13 17:38:03.126337 7fe9bae887a0 20 read_log 194370'13735 (194370'13734) modify 7028d3bf/benchmark_data_storage0111-ib.pocstor_12495_object3559/head//24 by client.1508531.0:3560 2014-08-12 20:30:27.625198 2014-08-13 17:38:03.126349 7fe9bae887a0 20 read_log 194370'13736 (0'0) promote 40de5fbf/benchmark_data_storage0111-ib.pocstor_12495_object5663/head//24 by osd.19.0:4694 2014-08-12 20:30:36.178414 2014-08-13 17:38:03.126361 7fe9bae887a0 20 read_log 194370'13737 (194370'13736) modify 40de5fbf/benchmark_data_storage0111-ib.pocstor_12495_object5663/head//24 by client.1508531.0:5664 2014-08-12 20:30:36.170257 2014-08-13 17:38:03.126373 7fe9bae887a0 20 read_log 194370'13738 (0'0) promote fd417bbf/benchmark_data_storage0111-ib.pocstor_12495_object6099/head//24 by osd.19.0:4700 2014-08-12 20:30:38.202367 2014-08-13 17:38:03.126384 7fe9bae887a0 20 read_log 194370'13739 (194370'13738) modify fd417bbf/benchmark_data_storage0111-ib.pocstor_12495_object6099/head//24 by client.1508531.0:6100 2014-08-12 20:30:38.196246 2014-08-13 17:38:03.126396 7fe9bae887a0 20 read_log 194370'13740 (0'0) promote 700c0fbf/benchmark_data_storage0111-ib.pocstor_12495_object6411/head//24 by osd.19.0:4706 2014-08-12 20:30:39.534530 2014-08-13 17:38:03.126408 7fe9bae887a0 20 read_log 194370'13741 (194370'13740) modify 700c0fbf/benchmark_data_storage0111-ib.pocstor_12495_object6411/head//24 by client.1508531.0:6412 2014-08-12 20:30:39.527742 2014-08-13 17:38:03.126420 7fe9bae887a0 20 read_log 194370'13742 (0'0) promote b2e17bbf/benchmark_data_storage0111-ib.pocstor_12495_object7303/head//24 by osd.19.0:4724 2014-08-12 20:30:43.915880 2014-08-13 17:38:03.126432 7fe9bae887a0 20 read_log 194370'13743 (194370'13742) modify b2e17bbf/benchmark_data_storage0111-ib.pocstor_12495_object7303/head//24 by client.1508531.0:7304 2014-08-12 20:30:43.904828 2014-08-13 17:38:03.126444 7fe9bae887a0 20 read_log 194370'13744 (0'0) promote cf8933bf/benchmark_data_storage0111-ib.pocstor_12495_object8707/head//24 by osd.19.0:4756 2014-08-12 20:30:51.499238 2014-08-13 17:38:03.126456 7fe9bae887a0 20 read_log 194370'13745 (194370'13744) modify cf8933bf/benchmark_data_storage0111-ib.pocstor_12495_object8707/head//24 by client.1508531.0:8708 2014-08-12 20:30:51.493886 2014-08-13 17:38:03.126468 7fe9bae887a0 20 read_log 194370'13746 (0'0) promote 865c7bf/benchmark_data_storage0111-ib.pocstor_12495_object9836/head//24 by osd.19.0:4770 2014-08-12 20:30:58.331667 2014-08-13 17:38:03.126480 7fe9bae887a0 20 read_log 194370'13747 (194370'13746) modify 865c7bf/benchmark_data_storage0111-ib.pocstor_12495_object9836/head//24 by client.1508531.0:9837 2014-08-12 20:30:58.324036 2014-08-13 17:38:03.126499 7fe9bae887a0 20 read_log 194370'13748 (0'0) promote 7a1333bf/benchmark_data_storage0111-ib.pocstor_12495_object11725/head//24 by osd.19.0:4786 2014-08-12 20:31:09.674196 2014-08-13 17:38:03.126512 7fe9bae887a0 20 read_log 194370'13749 (194370'13748) modify 7a1333bf/benchmark_data_storage0111-ib.pocstor_12495_object11725/head//24 by client.1508531.0:11726 2014-08-12 20:31:09.667264 2014-08-13 17:38:03.126524 7fe9bae887a0 20 read_log 194370'13750 (0'0) promote d757d3bf/benchmark_data_storage0111-ib.pocstor_12495_object13273/head//24 by osd.19.0:4798 2014-08-12 20:31:22.004315 2014-08-13 17:38:03.126536 7fe9bae887a0 20 read_log 194370'13751 (194370'13750) modify d757d3bf/benchmark_data_storage0111-ib.pocstor_12495_object13273/head//24 by client.1508531.0:13274 2014-08-12 20:31:21.997243 2014-08-13 17:38:03.126549 7fe9bae887a0 20 read_log 194377'13752 (0'0) promote 79755fbf/benchmark_data_storage0111-ib.pocstor_25065_object2615/head//24 by osd.19.0:4838 2014-08-12 20:33:07.804959 2014-08-13 17:38:03.126560 7fe9bae887a0 20 read_log 194377'13753 (194377'13752) modify 79755fbf/benchmark_data_storage0111-ib.pocstor_25065_object2615/head//24 by client.2231190.0:2616 2014-08-12 20:33:07.795754 2014-08-13 17:38:03.126573 7fe9bae887a0 20 read_log 194377'13754 (0'0) promote b7d97fbf/benchmark_data_storage0111-ib.pocstor_25065_object3948/head//24 by osd.19.0:4850 2014-08-12 20:33:13.628830 2014-08-13 17:38:03.126585 7fe9bae887a0 20 read_log 194377'13755 (194377'13754) modify b7d97fbf/benchmark_data_storage0111-ib.pocstor_25065_object3948/head//24 by client.2231190.0:3949 2014-08-12 20:33:13.618902 2014-08-13 17:38:03.126597 7fe9bae887a0 20 read_log 194380'13756 (0'0) promote c538cbbf/benchmark_data_storage0111-ib.pocstor_25065_object5297/head//24 by osd.19.0:4868 2014-08-12 20:33:27.113369 2014-08-13 17:38:03.126609 7fe9bae887a0 20 read_log 194380'13757 (194380'13756) modify c538cbbf/benchmark_data_storage0111-ib.pocstor_25065_object5297/head//24 by client.2231190.0:5298 2014-08-12 20:33:27.106661 2014-08-13 17:38:03.126621 7fe9bae887a0 20 read_log 194380'13758 (0'0) promote c07903bf/benchmark_data_storage0111-ib.pocstor_25065_object5425/head//24 by osd.19.0:4874 2014-08-12 20:33:28.891875 2014-08-13 17:38:03.126632 7fe9bae887a0 20 read_log 194380'13759 (194380'13758) modify c07903bf/benchmark_data_storage0111-ib.pocstor_25065_object5425/head//24 by client.2231190.0:5426 2014-08-12 20:33:28.884787 2014-08-13 17:38:03.126645 7fe9bae887a0 20 read_log 194383'13760 (0'0) promote 41f87fbf/benchmark_data_storage0111-ib.pocstor_25065_object5885/head//24 by osd.19.0:4888 2014-08-12 20:34:00.870766 2014-08-13 17:38:03.126656 7fe9bae887a0 20 read_log 194383'13761 (194383'13760) modify 41f87fbf/benchmark_data_storage0111-ib.pocstor_25065_object5885/head//24 by client.2231190.0:5886 2014-08-12 20:34:00.862447 2014-08-13 17:38:03.126668 7fe9bae887a0 20 read_log 194383'13762 (0'0) promote a6a70bbf/benchmark_data_storage0111-ib.pocstor_25065_object5950/head//24 by osd.19.0:4890 2014-08-12 20:34:01.167736 2014-08-13 17:38:03.126680 7fe9bae887a0 20 read_log 194383'13763 (194383'13762) modify a6a70bbf/benchmark_data_storage0111-ib.pocstor_25065_object5950/head//24 by client.2231190.0:5951 2014-08-12 20:34:01.157856 2014-08-13 17:38:03.126692 7fe9bae887a0 20 read_log 194393'13764 (0'0) promote f990f3bf/benchmark_data_storage0111-ib.pocstor_48304_object150/head//24 by osd.19.0:4943 2014-08-12 20:52:33.125398 2014-08-13 17:38:03.126704 7fe9bae887a0 20 read_log 194393'13765 (194393'13764) modify f990f3bf/benchmark_data_storage0111-ib.pocstor_48304_object150/head//24 by client.1064422.0:151 2014-08-12 20:52:33.115711 2014-08-13 17:38:03.126717 7fe9bae887a0 20 read_log 194416'13766 (0'0) promote 801dfbbf/benchmark_data_storage0111-ib.pocstor_19103_object538/head//24 by osd.19.0:5091 2014-08-13 15:28:32.618697 2014-08-13 17:38:03.126736 7fe9bae887a0 20 read_log 194416'13767 (194416'13766) modify 801dfbbf/benchmark_data_storage0111-ib.pocstor_19103_object538/head//24 by client.2441958.0:539 2014-08-13 15:28:32.610887 2014-08-13 17:38:03.126768 7fe9bae887a0 20 read_log 194416'13768 (0'0) promote 44ad77bf/benchmark_data_storage0111-ib.pocstor_19103_object976/head//24 by osd.19.0:5102 2014-08-13 15:28:34.424112 2014-08-13 17:38:03.126783 7fe9bae887a0 20 read_log 194416'13769 (194416'13768) modify 44ad77bf/benchmark_data_storage0111-ib.pocstor_19103_object976/head//24 by client.2441958.0:977 2014-08-13 15:28:34.417346 2014-08-13 17:38:03.126816 7fe9bae887a0 20 read_log 194416'13770 (0'0) promote 648653bf/benchmark_data_storage0111-ib.pocstor_19103_object2959/head//24 by osd.19.0:5133 2014-08-13 15:28:41.923799 2014-08-13 17:38:03.126829 7fe9bae887a0 20 read_log 194416'13771 (194416'13770) modify 648653bf/benchmark_data_storage0111-ib.pocstor_19103_object2959/head//24 by client.2441958.0:2960 2014-08-13 15:28:41.917209 2014-08-13 17:38:03.126843 7fe9bae887a0 20 read_log 194416'13772 (0'0) promote 1736f7bf/benchmark_data_storage0111-ib.pocstor_19103_object4678/head//24 by osd.19.0:5157 2014-08-13 15:28:48.677571 2014-08-13 17:38:03.126855 7fe9bae887a0 20 read_log 194416'13773 (194416'13772) modify 1736f7bf/benchmark_data_storage0111-ib.pocstor_19103_object4678/head//24 by client.2441958.0:4679 2014-08-13 15:28:48.670385 2014-08-13 17:38:03.126870 7fe9bae887a0 20 read_log 194416'13774 (0'0) promote 127e77bf/benchmark_data_storage0111-ib.pocstor_19103_object6287/head//24 by osd.19.0:5179 2014-08-13 15:28:55.091692 2014-08-13 17:38:03.126882 7fe9bae887a0 20 read_log 194416'13775 (194416'13774) modify 127e77bf/benchmark_data_storage0111-ib.pocstor_19103_object6287/head//24 by client.2441958.0:6288 2014-08-13 15:28:55.083137 2014-08-13 17:38:03.126896 7fe9bae887a0 20 read_log 194416'13776 (0'0) promote 9e8ca3bf/benchmark_data_storage0111-ib.pocstor_19103_object6377/head//24 by osd.19.0:5183 2014-08-13 15:28:55.417795 2014-08-13 17:38:03.126908 7fe9bae887a0 20 read_log 194416'13777 (194416'13776) modify 9e8ca3bf/benchmark_data_storage0111-ib.pocstor_19103_object6377/head//24 by client.2441958.0:6378 2014-08-13 15:28:55.410346 2014-08-13 17:38:03.126922 7fe9bae887a0 20 read_log 194416'13778 (0'0) promote 95aa8fbf/benchmark_data_storage0111-ib.pocstor_19103_object7273/head//24 by osd.19.0:5191 2014-08-13 15:28:58.796500 2014-08-13 17:38:03.126934 7fe9bae887a0 20 read_log 194416'13779 (194416'13778) modify 95aa8fbf/benchmark_data_storage0111-ib.pocstor_19103_object7273/head//24 by client.2441958.0:7274 2014-08-13 15:28:58.787603 2014-08-13 17:38:03.126948 7fe9bae887a0 20 read_log 194416'13780 (0'0) promote 411383bf/benchmark_data_storage0111-ib.pocstor_19103_object7620/head//24 by osd.19.0:5197 2014-08-13 15:29:00.105735 2014-08-13 17:38:03.126962 7fe9bae887a0 20 read_log 194416'13781 (194416'13780) modify 411383bf/benchmark_data_storage0111-ib.pocstor_19103_object7620/head//24 by client.2441958.0:7621 2014-08-13 15:29:00.099008 2014-08-13 17:38:03.126975 7fe9bae887a0 20 read_log 194416'13782 (0'0) promote 78bf7fbf/benchmark_data_storage0111-ib.pocstor_19103_object12517/head//24 by osd.19.0:5251 2014-08-13 15:29:19.766645 2014-08-13 17:38:03.126989 7fe9bae887a0 20 read_log 194416'13783 (194416'13782) modify 78bf7fbf/benchmark_data_storage0111-ib.pocstor_19103_object12517/head//24 by client.2441958.0:12518 2014-08-13 15:29:19.759662 2014-08-13 17:38:03.127003 7fe9bae887a0 20 read_log 194416'13784 (0'0) promote 4142d7bf/benchmark_data_storage0111-ib.pocstor_19103_object14735/head//24 by osd.19.0:5285 2014-08-13 15:29:29.143426 2014-08-13 17:38:03.127015 7fe9bae887a0 20 read_log 194416'13785 (194416'13784) modify 4142d7bf/benchmark_data_storage0111-ib.pocstor_19103_object14735/head//24 by client.2441958.0:14736 2014-08-13 15:29:29.134575 2014-08-13 17:38:03.127028 7fe9bae887a0 20 read_log 194416'13786 (0'0) promote 9f1f5fbf/benchmark_data_storage0111-ib.pocstor_19103_object15019/head//24 by osd.19.0:5291 2014-08-13 15:29:30.383809 2014-08-13 17:38:03.127063 7fe9bae887a0 20 read_log 194416'13787 (194416'13786) modify 9f1f5fbf/benchmark_data_storage0111-ib.pocstor_19103_object15019/head//24 by client.2441958.0:15020 2014-08-13 15:29:30.376513 2014-08-13 17:38:03.127079 7fe9bae887a0 20 read_log 194416'13788 (0'0) promote 5e660fbf/benchmark_data_storage0111-ib.pocstor_19103_object16368/head//24 by osd.19.0:5307 2014-08-13 15:29:38.391449 2014-08-13 17:38:03.127092 7fe9bae887a0 20 read_log 194416'13789 (194416'13788) modify 5e660fbf/benchmark_data_storage0111-ib.pocstor_19103_object16368/head//24 by client.2441958.0:16369 2014-08-13 15:29:38.382308 2014-08-13 17:38:03.127106 7fe9bae887a0 20 read_log 194416'13790 (0'0) promote 739373bf/benchmark_data_storage0111-ib.pocstor_19103_object16613/head//24 by osd.19.0:5311 2014-08-13 15:29:41.844800 2014-08-13 17:38:03.127118 7fe9bae887a0 20 read_log 194416'13791 (194416'13790) modify 739373bf/benchmark_data_storage0111-ib.pocstor_19103_object16613/head//24 by client.2441958.0:16614 2014-08-13 15:29:41.837470 2014-08-13 17:38:03.127132 7fe9bae887a0 20 read_log 194416'13792 (0'0) promote 171767bf/benchmark_data_storage0111-ib.pocstor_19103_object17318/head//24 by osd.19.0:5325 2014-08-13 15:29:51.891613 2014-08-13 17:38:03.127144 7fe9bae887a0 20 read_log 194416'13793 (194416'13792) modify 171767bf/benchmark_data_storage0111-ib.pocstor_19103_object17318/head//24 by client.2441958.0:17319 2014-08-13 15:29:51.882464 2014-08-13 17:38:03.127161 7fe9bae887a0 20 read_log 194416'13794 (0'0) promote c22513bf/benchmark_data_storage0111-ib.pocstor_19103_object18618/head//24 by osd.19.0:5337 2014-08-13 15:30:16.388470 2014-08-13 17:38:03.127176 7fe9bae887a0 20 read_log 194416'13795 (194416'13794) modify c22513bf/benchmark_data_storage0111-ib.pocstor_19103_object18618/head//24 by client.2441958.0:18619 2014-08-13 15:30:16.380078 2014-08-13 17:38:03.127206 7fe9bae887a0 20 read_log 194422'13796 (0'0) promote 946ec7bf/benchmark_data_storage0111-ib.pocstor_41231_object998/head//24 by osd.19.0:5357 2014-08-13 15:32:19.294942 2014-08-13 17:38:03.127222 7fe9bae887a0 20 read_log 194422'13797 (194422'13796) modify 946ec7bf/benchmark_data_storage0111-ib.pocstor_41231_object998/head//24 by client.1148203.0:999 2014-08-13 15:32:19.287995 2014-08-13 17:38:03.127237 7fe9bae887a0 20 read_log 194422'13798 (0'0) promote a36a9bbf/benchmark_data_storage0111-ib.pocstor_41231_object3736/head//24 by osd.19.0:5383 2014-08-13 15:32:30.005484 2014-08-13 17:38:03.127249 7fe9bae887a0 20 read_log 194422'13799 (194422'13798) modify a36a9bbf/benchmark_data_storage0111-ib.pocstor_41231_object3736/head//24 by client.1148203.0:3737 2014-08-13 15:32:29.998002 2014-08-13 17:38:03.127263 7fe9bae887a0 20 read_log 194422'13800 (0'0) promote 2d6a1fbf/benchmark_data_storage0111-ib.pocstor_41231_object4583/head//24 by osd.19.0:5391 2014-08-13 15:32:33.243740 2014-08-13 17:38:03.127275 7fe9bae887a0 20 read_log 194422'13801 (194422'13800) modify 2d6a1fbf/benchmark_data_storage0111-ib.pocstor_41231_object4583/head//24 by client.1148203.0:4584 2014-08-13 15:32:33.234264 2014-08-13 17:38:03.127290 7fe9bae887a0 20 read_log 194422'13802 (0'0) promote 98e2dfbf/benchmark_data_storage0111-ib.pocstor_41231_object6292/head//24 by osd.19.0:5413 2014-08-13 15:32:41.752848 2014-08-13 17:38:03.127303 7fe9bae887a0 20 read_log 194422'13803 (194422'13802) modify 98e2dfbf/benchmark_data_storage0111-ib.pocstor_41231_object6292/head//24 by client.1148203.0:6293 2014-08-13 15:32:41.744371 2014-08-13 17:38:03.127317 7fe9bae887a0 20 read_log 194422'13804 (0'0) promote b4ee43bf/benchmark_data_storage0111-ib.pocstor_41231_object7436/head//24 by osd.19.0:5431 2014-08-13 15:32:48.222667 2014-08-13 17:38:03.127331 7fe9bae887a0 20 read_log 194422'13805 (194422'13804) modify b4ee43bf/benchmark_data_storage0111-ib.pocstor_41231_object7436/head//24 by client.1148203.0:7437 2014-08-13 15:32:48.214318 2014-08-13 17:38:03.127343 7fe9bae887a0 20 read_log 194422'13806 (0'0) promote fdc83fbf/benchmark_data_storage0111-ib.pocstor_41231_object8127/head//24 by osd.19.0:5439 2014-08-13 15:32:52.530922 2014-08-13 17:38:03.127357 7fe9bae887a0 20 read_log 194422'13807 (194422'13806) modify fdc83fbf/benchmark_data_storage0111-ib.pocstor_41231_object8127/head//24 by client.1148203.0:8128 2014-08-13 15:32:52.521785 2014-08-13 17:38:03.127370 7fe9bae887a0 20 read_log 194422'13808 (0'0) promote 9c81cfbf/benchmark_data_storage0111-ib.pocstor_41231_object10089/head//24 by osd.19.0:5469 2014-08-13 15:33:12.596365 2014-08-13 17:38:03.127382 7fe9bae887a0 20 read_log 194422'13809 (194422'13808) modify 9c81cfbf/benchmark_data_storage0111-ib.pocstor_41231_object10089/head//24 by client.1148203.0:10090 2014-08-13 15:33:12.589305 2014-08-13 17:38:03.127394 7fe9bae887a0 20 read_log 194422'13810 (0'0) promote ce8a4fbf/benchmark_data_storage0111-ib.pocstor_41231_object10098/head//24 by osd.19.0:5471 2014-08-13 15:33:12.841098 2014-08-13 17:38:03.127406 7fe9bae887a0 20 read_log 194422'13811 (194422'13810) modify ce8a4fbf/benchmark_data_storage0111-ib.pocstor_41231_object10098/head//24 by client.1148203.0:10099 2014-08-13 15:33:12.831737 2014-08-13 17:38:03.127421 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.127797 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.127832 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive] handle_loaded 2014-08-13 17:38:03.127851 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.010698 0 0.000000 2014-08-13 17:38:03.127863 7fe9bae887a0 5 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.127873 7fe9bae887a0 20 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194477 2014-08-13 17:38:03.127882 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] log((192834'12014,194422'13811], crt=0'0) 2014-08-13 17:38:03.127892 7fe9bae887a0 10 osd.4 194478 pgid 24.3f7 coll 24.3f7_head 2014-08-13 17:38:03.128007 7fe9bae887a0 10 osd.4 194478 _open_lock_pg 24.3f7 2014-08-13 17:38:03.128020 7fe9bae887a0 10 osd.4 194478 _get_pool 24 2014-08-13 17:38:03.128052 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3f7(unlocked)] enter Initial 2014-08-13 17:38:03.128063 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.3f7(unlocked)] enter NotTrimming 2014-08-13 17:38:03.128206 7fe9bae887a0 10 read_log 2014-08-13 17:38:03.128334 7fe9bae887a0 20 read_log 192834'13627 (0'0) promote c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by osd.306.0:27880 2014-08-11 15:23:35.826126 2014-08-13 17:38:03.128354 7fe9bae887a0 20 read_log 192834'13628 (192834'13627) modify c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.933448.0:53 2014-08-11 15:23:35.813800 2014-08-13 17:38:03.128368 7fe9bae887a0 20 read_log 192834'13665 (0'0) promote 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by osd.306.0:27971 2014-08-11 15:23:49.115086 2014-08-13 17:38:03.128380 7fe9bae887a0 20 read_log 192834'13666 (192834'13665) modify 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.933448.0:2852 2014-08-11 15:23:48.870454 2014-08-13 17:38:03.128393 7fe9bae887a0 20 read_log 192834'13689 (0'0) promote 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by osd.306.0:28015 2014-08-11 15:23:59.256466 2014-08-13 17:38:03.128405 7fe9bae887a0 20 read_log 192834'13690 (192834'13689) modify 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.933448.0:4898 2014-08-11 15:23:59.249077 2014-08-13 17:38:03.128417 7fe9bae887a0 20 read_log 192834'13719 (0'0) promote 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by osd.306.0:28089 2014-08-11 15:24:06.760031 2014-08-13 17:38:03.128429 7fe9bae887a0 20 read_log 192834'13720 (192834'13719) modify 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.933448.0:6788 2014-08-11 15:24:06.752258 2014-08-13 17:38:03.128443 7fe9bae887a0 20 read_log 192834'13739 (0'0) promote 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.306.0:28123 2014-08-11 15:24:10.515439 2014-08-13 17:38:03.128464 7fe9bae887a0 20 read_log 192834'13740 (192834'13739) modify 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by client.933448.0:7553 2014-08-11 15:24:10.507486 2014-08-13 17:38:03.128477 7fe9bae887a0 20 read_log 192834'13743 (0'0) promote f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.306.0:28129 2014-08-11 15:24:11.792918 2014-08-13 17:38:03.128489 7fe9bae887a0 20 read_log 192834'13744 (192834'13743) modify f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by client.933448.0:7890 2014-08-11 15:24:11.784675 2014-08-13 17:38:03.128501 7fe9bae887a0 20 read_log 192834'13767 (0'0) promote dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.306.0:28169 2014-08-11 15:24:17.771300 2014-08-13 17:38:03.128513 7fe9bae887a0 20 read_log 192834'13768 (192834'13767) modify dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by client.933448.0:9159 2014-08-11 15:24:17.764345 2014-08-13 17:38:03.128525 7fe9bae887a0 20 read_log 192834'13775 (0'0) promote d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.306.0:28189 2014-08-11 15:24:19.118498 2014-08-13 17:38:03.128537 7fe9bae887a0 20 read_log 192834'13776 (192834'13775) modify d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by client.933448.0:9518 2014-08-11 15:24:19.110819 2014-08-13 17:38:03.128549 7fe9bae887a0 20 read_log 192834'13777 (0'0) promote 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.306.0:28193 2014-08-11 15:24:19.347210 2014-08-13 17:38:03.128561 7fe9bae887a0 20 read_log 192834'13778 (192834'13777) modify 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by client.933448.0:9563 2014-08-11 15:24:19.339217 2014-08-13 17:38:03.128574 7fe9bae887a0 20 read_log 192834'13801 (0'0) promote 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.306.0:28253 2014-08-11 15:24:29.386308 2014-08-13 17:38:03.128586 7fe9bae887a0 20 read_log 192834'13802 (192834'13801) modify 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by client.933448.0:11793 2014-08-11 15:24:29.378570 2014-08-13 17:38:03.128598 7fe9bae887a0 20 read_log 192834'13817 (0'0) promote 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.306.0:28281 2014-08-11 15:24:33.471041 2014-08-13 17:38:03.128610 7fe9bae887a0 20 read_log 192834'13818 (192834'13817) modify 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by client.933448.0:12690 2014-08-11 15:24:33.462814 2014-08-13 17:38:03.128622 7fe9bae887a0 20 read_log 192834'13841 (0'0) promote b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by osd.306.0:28323 2014-08-11 15:24:41.519608 2014-08-13 17:38:03.128634 7fe9bae887a0 20 read_log 192834'13842 (192834'13841) modify b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.933448.0:14130 2014-08-11 15:24:41.510311 2014-08-13 17:38:03.128646 7fe9bae887a0 20 read_log 192834'13847 (0'0) promote 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by osd.306.0:28335 2014-08-11 15:24:44.823697 2014-08-13 17:38:03.128658 7fe9bae887a0 20 read_log 192834'13848 (192834'13847) modify 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.933448.0:14342 2014-08-11 15:24:44.815877 2014-08-13 17:38:03.128670 7fe9bae887a0 20 read_log 192834'13857 (0'0) promote 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by osd.306.0:28347 2014-08-11 15:24:47.167811 2014-08-13 17:38:03.128681 7fe9bae887a0 20 read_log 192834'13858 (192834'13857) modify 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.933448.0:14793 2014-08-11 15:24:47.159175 2014-08-13 17:38:03.128693 7fe9bae887a0 20 read_log 192834'13879 (0'0) promote 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.306.0:28373 2014-08-11 15:24:50.074217 2014-08-13 17:38:03.128716 7fe9bae887a0 20 read_log 192834'13880 (192834'13879) modify 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by client.933448.0:15550 2014-08-11 15:24:50.066438 2014-08-13 17:38:03.128729 7fe9bae887a0 20 read_log 192834'13883 (0'0) promote daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by osd.306.0:28377 2014-08-11 15:24:50.454041 2014-08-13 17:38:03.128741 7fe9bae887a0 20 read_log 192834'13884 (192834'13883) modify daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.933448.0:15632 2014-08-11 15:24:50.446190 2014-08-13 17:38:03.128753 7fe9bae887a0 20 read_log 192834'13895 (0'0) promote b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by osd.306.0:28419 2014-08-11 15:24:59.405618 2014-08-13 17:38:03.128765 7fe9bae887a0 20 read_log 192834'13896 (192834'13895) modify b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.933448.0:17359 2014-08-11 15:24:59.397856 2014-08-13 17:38:03.128778 7fe9bae887a0 20 read_log 192834'13897 (0'0) promote ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by osd.306.0:28423 2014-08-11 15:24:59.646939 2014-08-13 17:38:03.128789 7fe9bae887a0 20 read_log 192834'13898 (192834'13897) modify ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.933448.0:17424 2014-08-11 15:24:59.640020 2014-08-13 17:38:03.128801 7fe9bae887a0 20 read_log 192834'13903 (0'0) promote d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.306.0:28439 2014-08-11 15:25:01.143866 2014-08-13 17:38:03.128816 7fe9bae887a0 20 read_log 192834'13904 (192834'13903) modify d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by client.933448.0:17709 2014-08-11 15:25:01.130089 2014-08-13 17:38:03.128829 7fe9bae887a0 20 read_log 192834'13907 (0'0) promote fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.306.0:28451 2014-08-11 15:25:03.267502 2014-08-13 17:38:03.128843 7fe9bae887a0 20 read_log 192834'13908 (192834'13907) modify fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by client.933448.0:18050 2014-08-11 15:25:03.256054 2014-08-13 17:38:03.128856 7fe9bae887a0 20 read_log 192834'13935 (0'0) promote fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.306.0:28489 2014-08-11 15:25:12.470654 2014-08-13 17:38:03.128869 7fe9bae887a0 20 read_log 192834'13936 (192834'13935) modify fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by client.933448.0:19539 2014-08-11 15:25:12.462660 2014-08-13 17:38:03.128882 7fe9bae887a0 20 read_log 192834'13947 (0'0) promote 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by osd.306.0:28509 2014-08-11 15:25:19.915863 2014-08-13 17:38:03.128894 7fe9bae887a0 20 read_log 192834'13948 (192834'13947) modify 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.933448.0:20245 2014-08-11 15:25:19.907818 2014-08-13 17:38:03.128927 7fe9bae887a0 20 read_log 192834'13949 (0'0) promote b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by osd.306.0:28517 2014-08-11 15:25:20.183559 2014-08-13 17:38:03.128942 7fe9bae887a0 20 read_log 192834'13950 (192834'13949) modify b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.933448.0:20307 2014-08-11 15:25:20.175279 2014-08-13 17:38:03.128955 7fe9bae887a0 20 read_log 192834'13953 (0'0) promote 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by osd.306.0:28525 2014-08-11 15:25:22.051013 2014-08-13 17:38:03.128967 7fe9bae887a0 20 read_log 192834'13954 (192834'13953) modify 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.933448.0:20591 2014-08-11 15:25:22.043775 2014-08-13 17:38:03.128981 7fe9bae887a0 20 read_log 192834'13963 (192834'13628) modify c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.933448.0:22344 2014-08-11 15:25:37.667547 2014-08-13 17:38:03.129001 7fe9bae887a0 20 read_log 192834'13982 (192834'13666) modify 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.933448.0:25143 2014-08-11 15:25:41.827334 2014-08-13 17:38:03.129016 7fe9bae887a0 20 read_log 192834'13993 (192834'13690) modify 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.933448.0:27189 2014-08-11 15:25:44.197911 2014-08-13 17:38:03.129028 7fe9bae887a0 20 read_log 192834'14008 (192834'13720) modify 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.933448.0:29079 2014-08-11 15:25:46.983836 2014-08-13 17:38:03.129041 7fe9bae887a0 20 read_log 192834'14018 (192834'13740) modify 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by client.933448.0:29844 2014-08-11 15:25:47.990705 2014-08-13 17:38:03.129055 7fe9bae887a0 20 read_log 192834'14020 (192834'13744) modify f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by client.933448.0:30181 2014-08-11 15:25:48.510464 2014-08-13 17:38:03.129068 7fe9bae887a0 20 read_log 192834'14032 (192834'13768) modify dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by client.933448.0:31450 2014-08-11 15:25:50.076029 2014-08-13 17:38:03.129082 7fe9bae887a0 20 read_log 192834'14036 (192834'13776) modify d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by client.933448.0:31809 2014-08-11 15:25:50.628314 2014-08-13 17:38:03.129095 7fe9bae887a0 20 read_log 192834'14037 (192834'13778) modify 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by client.933448.0:31854 2014-08-11 15:25:50.695053 2014-08-13 17:38:03.129108 7fe9bae887a0 20 read_log 192834'14049 (192834'13802) modify 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by client.933448.0:34084 2014-08-11 15:25:54.560523 2014-08-13 17:38:03.129121 7fe9bae887a0 20 read_log 192834'14057 (192834'13818) modify 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by client.933448.0:34981 2014-08-11 15:25:55.660605 2014-08-13 17:38:03.129135 7fe9bae887a0 20 read_log 192834'14069 (192834'13842) modify b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.933448.0:36421 2014-08-11 15:25:57.712095 2014-08-13 17:38:03.129147 7fe9bae887a0 20 read_log 192834'14072 (192834'13848) modify 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.933448.0:36633 2014-08-11 15:25:58.105001 2014-08-13 17:38:03.129169 7fe9bae887a0 20 read_log 192834'14077 (192834'13858) modify 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.933448.0:37084 2014-08-11 15:25:58.899196 2014-08-13 17:38:03.129182 7fe9bae887a0 20 read_log 192834'14088 (192834'13880) modify 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by client.933448.0:37841 2014-08-11 15:26:00.169333 2014-08-13 17:38:03.129194 7fe9bae887a0 20 read_log 192834'14090 (192834'13884) modify daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.933448.0:37923 2014-08-11 15:26:00.269717 2014-08-13 17:38:03.129206 7fe9bae887a0 20 read_log 192834'14096 (192834'13896) modify b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.933448.0:39650 2014-08-11 15:26:02.009965 2014-08-13 17:38:03.129221 7fe9bae887a0 20 read_log 192834'14097 (192834'13898) modify ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.933448.0:39715 2014-08-11 15:26:02.040124 2014-08-13 17:38:03.129234 7fe9bae887a0 20 read_log 192834'14100 (192834'13904) modify d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by client.933448.0:40000 2014-08-11 15:26:02.213408 2014-08-13 17:38:03.129248 7fe9bae887a0 20 read_log 192834'14102 (192834'13908) modify fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by client.933448.0:40341 2014-08-11 15:26:02.576847 2014-08-13 17:38:03.129261 7fe9bae887a0 20 read_log 192834'14116 (192834'13936) modify fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by client.933448.0:41830 2014-08-11 15:26:03.067054 2014-08-13 17:38:03.129283 7fe9bae887a0 20 read_log 192834'14122 (192834'13948) modify 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.933448.0:42536 2014-08-11 15:26:03.309522 2014-08-13 17:38:03.129298 7fe9bae887a0 20 read_log 192834'14123 (192834'13950) modify b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.933448.0:42598 2014-08-11 15:26:03.328524 2014-08-13 17:38:03.129310 7fe9bae887a0 20 read_log 192834'14125 (192834'13954) modify 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.933448.0:42882 2014-08-11 15:26:03.420815 2014-08-13 17:38:03.129322 7fe9bae887a0 20 read_log 192834'14150 (0'0) promote 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by osd.306.0:28938 2014-08-11 15:26:19.560172 2014-08-13 17:38:03.129336 7fe9bae887a0 20 read_log 192834'14151 (192834'14150) modify 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.1883634.0:1396 2014-08-11 15:26:19.551952 2014-08-13 17:38:03.129348 7fe9bae887a0 20 read_log 192834'14170 (0'0) promote 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by osd.306.0:28974 2014-08-11 15:26:24.879518 2014-08-13 17:38:03.129360 7fe9bae887a0 20 read_log 192834'14171 (192834'14170) modify 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.1883634.0:2837 2014-08-11 15:26:24.872144 2014-08-13 17:38:03.129372 7fe9bae887a0 20 read_log 192834'14196 (0'0) promote 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by osd.306.0:29020 2014-08-11 15:26:31.483633 2014-08-13 17:38:03.129384 7fe9bae887a0 20 read_log 192834'14197 (192834'14196) modify 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.1883634.0:4678 2014-08-11 15:26:31.474601 2014-08-13 17:38:03.129396 7fe9bae887a0 20 read_log 192834'14216 (0'0) promote cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by osd.306.0:29054 2014-08-11 15:26:35.806194 2014-08-13 17:38:03.129408 7fe9bae887a0 20 read_log 192834'14217 (192834'14216) modify cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.1883634.0:5832 2014-08-11 15:26:35.798089 2014-08-13 17:38:03.129421 7fe9bae887a0 20 read_log 192834'14244 (0'0) promote df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by osd.306.0:29120 2014-08-11 15:26:43.667852 2014-08-13 17:38:03.129433 7fe9bae887a0 20 read_log 192834'14245 (192834'14244) modify df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.1883634.0:7792 2014-08-11 15:26:43.658736 2014-08-13 17:38:03.129445 7fe9bae887a0 20 read_log 192834'14254 (0'0) promote 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by osd.306.0:29144 2014-08-11 15:26:46.303517 2014-08-13 17:38:03.129457 7fe9bae887a0 20 read_log 192834'14255 (192834'14254) modify 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.1883634.0:8437 2014-08-11 15:26:46.294310 2014-08-13 17:38:03.129469 7fe9bae887a0 20 read_log 192834'14310 (0'0) promote 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by osd.306.0:29236 2014-08-11 15:26:58.651994 2014-08-13 17:38:03.129481 7fe9bae887a0 20 read_log 192834'14311 (192834'14310) modify 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.1883634.0:11174 2014-08-11 15:26:58.643100 2014-08-13 17:38:03.129493 7fe9bae887a0 20 read_log 192834'14320 (0'0) promote b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by osd.306.0:29266 2014-08-11 15:27:01.110941 2014-08-13 17:38:03.129505 7fe9bae887a0 20 read_log 192834'14321 (192834'14320) modify b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.1883634.0:11734 2014-08-11 15:27:01.103851 2014-08-13 17:38:03.129517 7fe9bae887a0 20 read_log 192834'14336 (0'0) promote f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by osd.306.0:29316 2014-08-11 15:27:07.208182 2014-08-13 17:38:03.129540 7fe9bae887a0 20 read_log 192834'14337 (192834'14336) modify f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.1883634.0:13010 2014-08-11 15:27:07.198134 2014-08-13 17:38:03.129553 7fe9bae887a0 20 read_log 192834'14364 (0'0) promote cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by osd.306.0:29378 2014-08-11 15:27:18.892648 2014-08-13 17:38:03.129565 7fe9bae887a0 20 read_log 192834'14365 (192834'14364) modify cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.1883634.0:15290 2014-08-11 15:27:18.883444 2014-08-13 17:38:03.129578 7fe9bae887a0 20 read_log 192834'14384 (0'0) promote 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by osd.306.0:29422 2014-08-11 15:27:27.303616 2014-08-13 17:38:03.129590 7fe9bae887a0 20 read_log 192834'14385 (192834'14384) modify 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.1883634.0:16375 2014-08-11 15:27:27.295529 2014-08-13 17:38:03.129602 7fe9bae887a0 20 read_log 192834'14386 (0'0) promote 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by osd.306.0:29430 2014-08-11 15:27:28.699178 2014-08-13 17:38:03.129614 7fe9bae887a0 20 read_log 192834'14387 (192834'14386) modify 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.1883634.0:16659 2014-08-11 15:27:28.690698 2014-08-13 17:38:03.129626 7fe9bae887a0 20 read_log 192834'14408 (0'0) promote 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.306.0:29468 2014-08-11 15:27:42.831667 2014-08-13 17:38:03.129638 7fe9bae887a0 20 read_log 192834'14409 (192834'14408) modify 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by client.1883634.0:18315 2014-08-11 15:27:42.821838 2014-08-13 17:38:03.129650 7fe9bae887a0 20 read_log 192834'14420 (0'0) promote 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by osd.306.0:29480 2014-08-11 15:27:47.001068 2014-08-13 17:38:03.129662 7fe9bae887a0 20 read_log 192834'14421 (192834'14420) modify 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.1883634.0:18674 2014-08-11 15:27:46.993331 2014-08-13 17:38:03.129673 7fe9bae887a0 20 read_log 192834'14452 (0'0) promote 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.306.0:29552 2014-08-11 15:28:02.982966 2014-08-13 17:38:03.129685 7fe9bae887a0 20 read_log 192834'14453 (192834'14452) modify 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by client.1883634.0:21135 2014-08-11 15:28:02.974405 2014-08-13 17:38:03.129697 7fe9bae887a0 20 read_log 192834'14470 (0'0) promote 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by osd.306.0:29590 2014-08-11 15:28:09.736738 2014-08-13 17:38:03.129709 7fe9bae887a0 20 read_log 192834'14471 (192834'14470) modify 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.1883634.0:22592 2014-08-11 15:28:09.727955 2014-08-13 17:38:03.129723 7fe9bae887a0 20 read_log 192834'14488 (192834'14151) modify 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.1883634.0:24720 2014-08-11 15:28:19.424616 2014-08-13 17:38:03.129736 7fe9bae887a0 20 read_log 192834'14498 (192834'14171) modify 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.1883634.0:26161 2014-08-11 15:28:22.694898 2014-08-13 17:38:03.129748 7fe9bae887a0 20 read_log 192834'14511 (192834'14197) modify 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.1883634.0:28002 2014-08-11 15:28:25.259435 2014-08-13 17:38:03.129760 7fe9bae887a0 20 read_log 192834'14521 (192834'14217) modify cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.1883634.0:29156 2014-08-11 15:28:26.804373 2014-08-13 17:38:03.129772 7fe9bae887a0 20 read_log 192834'14535 (192834'14245) modify df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.1883634.0:31116 2014-08-11 15:28:29.931270 2014-08-13 17:38:03.129798 7fe9bae887a0 20 read_log 192834'14540 (192834'14255) modify 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.1883634.0:31761 2014-08-11 15:28:30.931289 2014-08-13 17:38:03.129814 7fe9bae887a0 20 read_log 192834'14568 (192834'14311) modify 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.1883634.0:34498 2014-08-11 15:28:34.923419 2014-08-13 17:38:03.129826 7fe9bae887a0 20 read_log 192834'14573 (192834'14321) modify b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.1883634.0:35058 2014-08-11 15:28:35.985753 2014-08-13 17:38:03.129838 7fe9bae887a0 20 read_log 192834'14581 (192834'14337) modify f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.1883634.0:36334 2014-08-11 15:28:37.903473 2014-08-13 17:38:03.129853 7fe9bae887a0 20 read_log 192834'14595 (192834'14365) modify cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.1883634.0:38614 2014-08-11 15:28:40.936446 2014-08-13 17:38:03.129865 7fe9bae887a0 20 read_log 192834'14605 (192834'14385) modify 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.1883634.0:39699 2014-08-11 15:28:42.443577 2014-08-13 17:38:03.129877 7fe9bae887a0 20 read_log 192834'14606 (192834'14387) modify 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.1883634.0:39983 2014-08-11 15:28:43.008034 2014-08-13 17:38:03.129891 7fe9bae887a0 20 read_log 192834'14617 (192834'14409) modify 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by client.1883634.0:41639 2014-08-11 15:28:45.388854 2014-08-13 17:38:03.129903 7fe9bae887a0 20 read_log 192834'14623 (192834'14421) modify 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.1883634.0:41998 2014-08-11 15:28:45.625325 2014-08-13 17:38:03.129917 7fe9bae887a0 20 read_log 192834'14639 (192834'14453) modify 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by client.1883634.0:44459 2014-08-11 15:28:46.864570 2014-08-13 17:38:03.129929 7fe9bae887a0 20 read_log 192834'14648 (192834'14471) modify 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.1883634.0:45916 2014-08-11 15:28:47.432494 2014-08-13 17:38:03.129943 7fe9bae887a0 20 read_log 192888'14654 (0'0) promote a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.4.0:21440 2014-08-11 16:45:02.937110 2014-08-13 17:38:03.129956 7fe9bae887a0 20 read_log 192888'14655 (192888'14654) modify a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by client.1315259.0:2017 2014-08-11 16:45:02.929036 2014-08-13 17:38:03.129968 7fe9bae887a0 20 read_log 192888'14658 (0'0) promote c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by osd.4.0:21456 2014-08-11 16:45:05.014076 2014-08-13 17:38:03.129980 7fe9bae887a0 20 read_log 192888'14659 (192888'14658) modify c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.1315259.0:2578 2014-08-11 16:45:05.003359 2014-08-13 17:38:03.129994 7fe9bae887a0 20 read_log 192888'14660 (0'0) promote d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by osd.4.0:21478 2014-08-11 16:45:07.732218 2014-08-13 17:38:03.130007 7fe9bae887a0 20 read_log 192888'14661 (192888'14660) modify d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.1315259.0:3274 2014-08-11 16:45:07.725306 2014-08-13 17:38:03.130019 7fe9bae887a0 20 read_log 192888'14664 (0'0) promote b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by osd.4.0:21486 2014-08-11 16:45:09.941730 2014-08-13 17:38:03.130031 7fe9bae887a0 20 read_log 192888'14665 (192888'14664) modify b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.1315259.0:3857 2014-08-11 16:45:09.931205 2014-08-13 17:38:03.130070 7fe9bae887a0 20 read_log 192888'14672 (0'0) promote 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by osd.4.0:21582 2014-08-11 16:45:20.649586 2014-08-13 17:38:03.130086 7fe9bae887a0 20 read_log 192888'14673 (192888'14672) modify 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.1315259.0:6794 2014-08-11 16:45:20.641480 2014-08-13 17:38:03.130098 7fe9bae887a0 20 read_log 192888'14676 (0'0) promote e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by osd.4.0:21602 2014-08-11 16:45:25.258901 2014-08-13 17:38:03.130110 7fe9bae887a0 20 read_log 192888'14677 (192888'14676) modify e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.1315259.0:8060 2014-08-11 16:45:25.249117 2014-08-13 17:38:03.130125 7fe9bae887a0 20 read_log 192888'14678 (0'0) promote 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by osd.4.0:21624 2014-08-11 16:45:27.444965 2014-08-13 17:38:03.130137 7fe9bae887a0 20 read_log 192888'14679 (192888'14678) modify 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.1315259.0:8630 2014-08-11 16:45:27.435698 2014-08-13 17:38:03.130152 7fe9bae887a0 20 read_log 192888'14680 (0'0) promote e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by osd.4.0:21628 2014-08-11 16:45:27.690926 2014-08-13 17:38:03.130171 7fe9bae887a0 20 read_log 192888'14681 (192888'14680) modify e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.1315259.0:8698 2014-08-11 16:45:27.678606 2014-08-13 17:38:03.130184 7fe9bae887a0 20 read_log 192888'14686 (0'0) promote 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by osd.4.0:21660 2014-08-11 16:45:33.991986 2014-08-13 17:38:03.130215 7fe9bae887a0 20 read_log 192888'14687 (192888'14686) modify 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.1315259.0:10351 2014-08-11 16:45:33.983200 2014-08-13 17:38:03.130230 7fe9bae887a0 20 read_log 192888'14690 (0'0) promote 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by osd.4.0:21672 2014-08-11 16:45:35.287364 2014-08-13 17:38:03.130243 7fe9bae887a0 20 read_log 192888'14691 (192888'14690) modify 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.1315259.0:10668 2014-08-11 16:45:35.278289 2014-08-13 17:38:03.130257 7fe9bae887a0 20 read_log 192888'14692 (0'0) promote 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by osd.4.0:21696 2014-08-11 16:45:38.355299 2014-08-13 17:38:03.130270 7fe9bae887a0 20 read_log 192888'14693 (192888'14692) modify 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.1315259.0:11438 2014-08-11 16:45:38.347476 2014-08-13 17:38:03.130282 7fe9bae887a0 20 read_log 192888'14694 (0'0) promote 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.4.0:21700 2014-08-11 16:45:39.198432 2014-08-13 17:38:03.130294 7fe9bae887a0 20 read_log 192888'14695 (192888'14694) modify 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by client.1315259.0:11654 2014-08-11 16:45:39.190244 2014-08-13 17:38:03.130306 7fe9bae887a0 20 read_log 192888'14696 (0'0) promote ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by osd.4.0:21724 2014-08-11 16:45:42.534854 2014-08-13 17:38:03.130318 7fe9bae887a0 20 read_log 192888'14697 (192888'14696) modify ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.1315259.0:12440 2014-08-11 16:45:42.527527 2014-08-13 17:38:03.130333 7fe9bae887a0 20 read_log 192888'14698 (0'0) promote da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by osd.4.0:21730 2014-08-11 16:45:43.090714 2014-08-13 17:38:03.130345 7fe9bae887a0 20 read_log 192888'14699 (192888'14698) modify da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.1315259.0:12582 2014-08-11 16:45:43.081732 2014-08-13 17:38:03.130367 7fe9bae887a0 20 read_log 192888'14702 (0'0) promote 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by osd.4.0:21824 2014-08-11 16:45:57.119876 2014-08-13 17:38:03.130382 7fe9bae887a0 20 read_log 192888'14703 (192888'14702) modify 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.1315259.0:16182 2014-08-11 16:45:57.112089 2014-08-13 17:38:03.130394 7fe9bae887a0 20 read_log 192888'14704 (0'0) promote 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by osd.4.0:21828 2014-08-11 16:45:57.482144 2014-08-13 17:38:03.130406 7fe9bae887a0 20 read_log 192888'14705 (192888'14704) modify 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.1315259.0:16280 2014-08-11 16:45:57.473494 2014-08-13 17:38:03.130418 7fe9bae887a0 20 read_log 192888'14708 (0'0) promote ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by osd.4.0:21836 2014-08-11 16:45:58.568385 2014-08-13 17:38:03.130430 7fe9bae887a0 20 read_log 192888'14709 (192888'14708) modify ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.1315259.0:16540 2014-08-11 16:45:58.560219 2014-08-13 17:38:03.130442 7fe9bae887a0 20 read_log 192888'14710 (0'0) promote 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by osd.4.0:21846 2014-08-11 16:46:00.946618 2014-08-13 17:38:03.130454 7fe9bae887a0 20 read_log 192888'14711 (192888'14710) modify 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.1315259.0:17152 2014-08-11 16:46:00.937813 2014-08-13 17:38:03.130467 7fe9bae887a0 20 read_log 192888'14712 (0'0) promote fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.4.0:21872 2014-08-11 16:46:04.155557 2014-08-13 17:38:03.130479 7fe9bae887a0 20 read_log 192888'14713 (192888'14712) modify fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by client.1315259.0:17929 2014-08-11 16:46:04.148939 2014-08-13 17:38:03.130491 7fe9bae887a0 20 read_log 192888'14714 (0'0) promote 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by osd.4.0:21876 2014-08-11 16:46:04.555483 2014-08-13 17:38:03.130503 7fe9bae887a0 20 read_log 192888'14715 (192888'14714) modify 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.1315259.0:18036 2014-08-11 16:46:04.546308 2014-08-13 17:38:03.130515 7fe9bae887a0 20 read_log 192888'14716 (0'0) promote 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.4.0:21882 2014-08-11 16:46:06.279008 2014-08-13 17:38:03.130527 7fe9bae887a0 20 read_log 192888'14717 (192888'14716) modify 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by client.1315259.0:18498 2014-08-11 16:46:06.270706 2014-08-13 17:38:03.130539 7fe9bae887a0 20 read_log 192888'14718 (0'0) promote f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.4.0:21896 2014-08-11 16:46:07.122915 2014-08-13 17:38:03.130551 7fe9bae887a0 20 read_log 192888'14719 (192888'14718) modify f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by client.1315259.0:18723 2014-08-11 16:46:07.114287 2014-08-13 17:38:03.130563 7fe9bae887a0 20 read_log 192888'14720 (0'0) promote 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by osd.4.0:21902 2014-08-11 16:46:07.947990 2014-08-13 17:38:03.130575 7fe9bae887a0 20 read_log 192888'14721 (192888'14720) modify 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.1315259.0:18941 2014-08-11 16:46:07.939990 2014-08-13 17:38:03.130587 7fe9bae887a0 20 read_log 192888'14724 (0'0) promote 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by osd.4.0:21952 2014-08-11 16:46:15.340099 2014-08-13 17:38:03.130599 7fe9bae887a0 20 read_log 192888'14725 (192888'14724) modify 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.1315259.0:20854 2014-08-11 16:46:15.331011 2014-08-13 17:38:03.130618 7fe9bae887a0 20 read_log 192888'14726 (0'0) promote 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by osd.4.0:21966 2014-08-11 16:46:16.745419 2014-08-13 17:38:03.130630 7fe9bae887a0 20 read_log 192888'14727 (192888'14726) modify 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.1315259.0:21235 2014-08-11 16:46:16.736661 2014-08-13 17:38:03.130642 7fe9bae887a0 20 read_log 192888'14728 (0'0) promote efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by osd.4.0:21988 2014-08-11 16:46:19.578778 2014-08-13 17:38:03.130654 7fe9bae887a0 20 read_log 192888'14729 (192888'14728) modify efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.1315259.0:21979 2014-08-11 16:46:19.571035 2014-08-13 17:38:03.130667 7fe9bae887a0 20 read_log 192888'14730 (0'0) promote 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by osd.4.0:22006 2014-08-11 16:46:21.895428 2014-08-13 17:38:03.130679 7fe9bae887a0 20 read_log 192888'14731 (192888'14730) modify 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.1315259.0:22557 2014-08-11 16:46:21.887387 2014-08-13 17:38:03.130691 7fe9bae887a0 20 read_log 192888'14732 (0'0) promote be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by osd.4.0:22016 2014-08-11 16:46:22.873091 2014-08-13 17:38:03.130703 7fe9bae887a0 20 read_log 192888'14733 (192888'14732) modify be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.1315259.0:22820 2014-08-11 16:46:22.862434 2014-08-13 17:38:03.130715 7fe9bae887a0 20 read_log 192888'14734 (0'0) promote d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by osd.4.0:22018 2014-08-11 16:46:23.339711 2014-08-13 17:38:03.130727 7fe9bae887a0 20 read_log 192888'14735 (192888'14734) modify d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.1315259.0:22930 2014-08-11 16:46:23.330369 2014-08-13 17:38:03.130739 7fe9bae887a0 20 read_log 192888'14736 (0'0) promote 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by osd.4.0:22048 2014-08-11 16:46:28.853784 2014-08-13 17:38:03.130751 7fe9bae887a0 20 read_log 192888'14737 (192888'14736) modify 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.1315259.0:24323 2014-08-11 16:46:28.842990 2014-08-13 17:38:03.130763 7fe9bae887a0 20 read_log 192888'14740 (0'0) promote 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by osd.4.0:22078 2014-08-11 16:46:31.709722 2014-08-13 17:38:03.130774 7fe9bae887a0 20 read_log 192888'14741 (192888'14740) modify 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.1315259.0:25054 2014-08-11 16:46:31.701223 2014-08-13 17:38:03.130786 7fe9bae887a0 20 read_log 192888'14742 (0'0) promote 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by osd.4.0:22100 2014-08-11 16:46:34.820522 2014-08-13 17:38:03.130798 7fe9bae887a0 20 read_log 192888'14743 (192888'14742) modify 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.1315259.0:25801 2014-08-11 16:46:34.809800 2014-08-13 17:38:03.130810 7fe9bae887a0 20 read_log 192888'14744 (0'0) promote 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by osd.4.0:22104 2014-08-11 16:46:36.730503 2014-08-13 17:38:03.130822 7fe9bae887a0 20 read_log 192888'14745 (192888'14744) modify 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.1315259.0:26231 2014-08-11 16:46:36.720809 2014-08-13 17:38:03.130834 7fe9bae887a0 20 read_log 192888'14750 (0'0) promote cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by osd.4.0:22166 2014-08-11 16:46:47.937414 2014-08-13 17:38:03.130846 7fe9bae887a0 20 read_log 192888'14751 (192888'14750) modify cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.1315259.0:28977 2014-08-11 16:46:47.928446 2014-08-13 17:38:03.130865 7fe9bae887a0 20 read_log 192888'14758 (0'0) promote 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by osd.4.0:22218 2014-08-11 16:46:54.300245 2014-08-13 17:38:03.130880 7fe9bae887a0 20 read_log 192888'14759 (192888'14758) modify 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.1315259.0:30579 2014-08-11 16:46:54.290044 2014-08-13 17:38:03.130893 7fe9bae887a0 20 read_log 192888'14761 (192888'14655) modify a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by client.1315259.0:32744 2014-08-11 16:46:56.458468 2014-08-13 17:38:03.130905 7fe9bae887a0 20 read_log 192888'14763 (192888'14659) modify c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.1315259.0:33305 2014-08-11 16:46:56.604203 2014-08-13 17:38:03.130919 7fe9bae887a0 20 read_log 192888'14764 (192888'14661) modify d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.1315259.0:34001 2014-08-11 16:46:56.779032 2014-08-13 17:38:03.130932 7fe9bae887a0 20 read_log 192888'14766 (192888'14665) modify b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.1315259.0:34584 2014-08-11 16:46:56.932302 2014-08-13 17:38:03.130946 7fe9bae887a0 20 read_log 192888'14770 (192888'14673) modify 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.1315259.0:37521 2014-08-11 16:46:57.705566 2014-08-13 17:38:03.130958 7fe9bae887a0 20 read_log 192888'14772 (192888'14677) modify e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.1315259.0:38787 2014-08-11 16:46:58.045702 2014-08-13 17:38:03.130972 7fe9bae887a0 20 read_log 192888'14773 (192888'14679) modify 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.1315259.0:39357 2014-08-11 16:46:58.190224 2014-08-13 17:38:03.130984 7fe9bae887a0 20 read_log 192888'14774 (192888'14681) modify e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.1315259.0:39425 2014-08-11 16:46:58.208281 2014-08-13 17:38:03.130996 7fe9bae887a0 20 read_log 192888'14777 (192888'14687) modify 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.1315259.0:41078 2014-08-11 16:46:58.653011 2014-08-13 17:38:03.131010 7fe9bae887a0 20 read_log 192888'14779 (192888'14691) modify 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.1315259.0:41395 2014-08-11 16:46:58.735115 2014-08-13 17:38:03.131022 7fe9bae887a0 20 read_log 192888'14780 (192888'14693) modify 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.1315259.0:42165 2014-08-11 16:46:58.936456 2014-08-13 17:38:03.131035 7fe9bae887a0 20 read_log 192888'14781 (192888'14695) modify 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by client.1315259.0:42381 2014-08-11 16:46:58.995334 2014-08-13 17:38:03.131047 7fe9bae887a0 20 read_log 192888'14782 (192888'14697) modify ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.1315259.0:43167 2014-08-11 16:46:59.211841 2014-08-13 17:38:03.131059 7fe9bae887a0 20 read_log 192888'14783 (192888'14699) modify da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.1315259.0:43309 2014-08-11 16:46:59.252901 2014-08-13 17:38:03.131070 7fe9bae887a0 20 read_log 192888'14785 (192888'14703) modify 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.1315259.0:46909 2014-08-11 16:47:00.292764 2014-08-13 17:38:03.131083 7fe9bae887a0 20 read_log 192888'14786 (192888'14705) modify 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.1315259.0:47007 2014-08-11 16:47:00.320338 2014-08-13 17:38:03.131095 7fe9bae887a0 20 read_log 192888'14788 (192888'14709) modify ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.1315259.0:47267 2014-08-11 16:47:00.400567 2014-08-13 17:38:03.131107 7fe9bae887a0 20 read_log 192888'14789 (192888'14711) modify 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.1315259.0:47879 2014-08-11 16:47:00.596698 2014-08-13 17:38:03.131128 7fe9bae887a0 20 read_log 192888'14790 (192888'14713) modify fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by client.1315259.0:48656 2014-08-11 16:47:00.827139 2014-08-13 17:38:03.131140 7fe9bae887a0 20 read_log 192888'14791 (192888'14715) modify 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.1315259.0:48763 2014-08-11 16:47:00.859603 2014-08-13 17:38:03.131152 7fe9bae887a0 20 read_log 192888'14792 (192888'14717) modify 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by client.1315259.0:49225 2014-08-11 16:47:00.995000 2014-08-13 17:38:03.131176 7fe9bae887a0 20 read_log 192888'14793 (192888'14719) modify f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by client.1315259.0:49450 2014-08-11 16:47:01.066247 2014-08-13 17:38:03.131189 7fe9bae887a0 20 read_log 192888'14794 (192888'14721) modify 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.1315259.0:49668 2014-08-11 16:47:01.137496 2014-08-13 17:38:03.131201 7fe9bae887a0 20 read_log 192888'14796 (192888'14725) modify 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.1315259.0:51581 2014-08-11 16:47:01.724332 2014-08-13 17:38:03.131214 7fe9bae887a0 20 read_log 192888'14797 (192888'14727) modify 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.1315259.0:51962 2014-08-11 16:47:01.832891 2014-08-13 17:38:03.131226 7fe9bae887a0 20 read_log 192888'14798 (192888'14729) modify efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.1315259.0:52706 2014-08-11 16:47:02.047650 2014-08-13 17:38:03.131238 7fe9bae887a0 20 read_log 192888'14799 (192888'14731) modify 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.1315259.0:53284 2014-08-11 16:47:02.208030 2014-08-13 17:38:03.131250 7fe9bae887a0 20 read_log 192888'14800 (192888'14733) modify be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.1315259.0:53547 2014-08-11 16:47:02.286959 2014-08-13 17:38:03.131262 7fe9bae887a0 20 read_log 192888'14801 (192888'14735) modify d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.1315259.0:53657 2014-08-11 16:47:02.319461 2014-08-13 17:38:03.131274 7fe9bae887a0 20 read_log 192888'14802 (192888'14737) modify 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.1315259.0:55050 2014-08-11 16:47:02.796236 2014-08-13 17:38:03.131286 7fe9bae887a0 20 read_log 192888'14804 (192888'14741) modify 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.1315259.0:55781 2014-08-11 16:47:03.010939 2014-08-13 17:38:03.131298 7fe9bae887a0 20 read_log 192888'14805 (192888'14743) modify 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.1315259.0:56528 2014-08-11 16:47:03.233773 2014-08-13 17:38:03.131310 7fe9bae887a0 20 read_log 192888'14806 (192888'14745) modify 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.1315259.0:56958 2014-08-11 16:47:03.366402 2014-08-13 17:38:03.131321 7fe9bae887a0 20 read_log 192888'14809 (192888'14751) modify cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.1315259.0:59704 2014-08-11 16:47:04.215685 2014-08-13 17:38:03.131352 7fe9bae887a0 20 read_log 192888'14813 (192888'14759) modify 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.1315259.0:61306 2014-08-11 16:47:04.699315 2014-08-13 17:38:03.131364 7fe9bae887a0 20 read_log 192888'14816 (0'0) promote 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.4.0:22648 2014-08-11 17:05:15.756733 2014-08-13 17:38:03.131376 7fe9bae887a0 20 read_log 192888'14817 (192888'14816) modify 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by client.942868.0:185 2014-08-11 17:05:15.744166 2014-08-13 17:38:03.131389 7fe9bae887a0 20 read_log 192888'14822 (0'0) promote 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.4.0:22685 2014-08-11 17:05:21.171385 2014-08-13 17:38:03.131408 7fe9bae887a0 20 read_log 192888'14823 (192888'14822) modify 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by client.942868.0:1506 2014-08-11 17:05:21.161585 2014-08-13 17:38:03.131421 7fe9bae887a0 20 read_log 192888'14824 (0'0) promote e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by osd.4.0:22722 2014-08-11 17:05:26.942292 2014-08-13 17:38:03.131455 7fe9bae887a0 20 read_log 192888'14825 (192888'14824) modify e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.942868.0:2973 2014-08-11 17:05:26.933706 2014-08-13 17:38:03.131470 7fe9bae887a0 20 read_log 192888'14830 (0'0) promote dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.4.0:22760 2014-08-11 17:05:31.438896 2014-08-13 17:38:03.131482 7fe9bae887a0 20 read_log 192888'14831 (192888'14830) modify dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by client.942868.0:4191 2014-08-11 17:05:31.431316 2014-08-13 17:38:03.131495 7fe9bae887a0 20 read_log 192888'14832 (0'0) promote 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.4.0:22764 2014-08-11 17:05:31.847811 2014-08-13 17:38:03.131507 7fe9bae887a0 20 read_log 192888'14833 (192888'14832) modify 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by client.942868.0:4302 2014-08-11 17:05:31.839700 2014-08-13 17:38:03.131519 7fe9bae887a0 20 read_log 192888'14834 (0'0) promote 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by osd.4.0:22782 2014-08-11 17:05:33.606301 2014-08-13 17:38:03.131530 7fe9bae887a0 20 read_log 192888'14835 (192888'14834) modify 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.942868.0:4760 2014-08-11 17:05:33.599597 2014-08-13 17:38:03.131542 7fe9bae887a0 20 read_log 192888'14836 (0'0) promote e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.4.0:22802 2014-08-11 17:05:36.309744 2014-08-13 17:38:03.131554 7fe9bae887a0 20 read_log 192888'14837 (192888'14836) modify e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by client.942868.0:5451 2014-08-11 17:05:36.302167 2014-08-13 17:38:03.131566 7fe9bae887a0 20 read_log 192888'14842 (0'0) promote 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by osd.4.0:22826 2014-08-11 17:05:39.441896 2014-08-13 17:38:03.131578 7fe9bae887a0 20 read_log 192888'14843 (192888'14842) modify 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.942868.0:6311 2014-08-11 17:05:39.432965 2014-08-13 17:38:03.131590 7fe9bae887a0 20 read_log 192888'14848 (0'0) promote fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by osd.4.0:22860 2014-08-11 17:05:44.116272 2014-08-13 17:38:03.131602 7fe9bae887a0 20 read_log 192888'14849 (192888'14848) modify fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.942868.0:7527 2014-08-11 17:05:44.108566 2014-08-13 17:38:03.131614 7fe9bae887a0 20 read_log 192888'14854 (0'0) promote 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.4.0:22880 2014-08-11 17:05:47.165759 2014-08-13 17:38:03.131626 7fe9bae887a0 20 read_log 192888'14855 (192888'14854) modify 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by client.942868.0:8371 2014-08-11 17:05:47.156438 2014-08-13 17:38:03.131638 7fe9bae887a0 20 read_log 192888'14856 (0'0) promote 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by osd.4.0:22888 2014-08-11 17:05:48.850351 2014-08-13 17:38:03.131650 7fe9bae887a0 20 read_log 192888'14857 (192888'14856) modify 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.942868.0:8813 2014-08-11 17:05:48.839328 2014-08-13 17:38:03.131661 7fe9bae887a0 20 read_log 192888'14858 (0'0) promote abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.4.0:22890 2014-08-11 17:05:49.651794 2014-08-13 17:38:03.131696 7fe9bae887a0 20 read_log 192888'14859 (192888'14858) modify abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by client.942868.0:9036 2014-08-11 17:05:49.643654 2014-08-13 17:38:03.131711 7fe9bae887a0 20 read_log 192888'14864 (0'0) promote 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by osd.4.0:23016 2014-08-11 17:06:09.762906 2014-08-13 17:38:03.131724 7fe9bae887a0 20 read_log 192888'14865 (192888'14864) modify 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.942868.0:14379 2014-08-11 17:06:09.752978 2014-08-13 17:38:03.131737 7fe9bae887a0 20 read_log 192888'14866 (0'0) promote 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by osd.4.0:23044 2014-08-11 17:06:14.106738 2014-08-13 17:38:03.131749 7fe9bae887a0 20 read_log 192888'14867 (192888'14866) modify 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.942868.0:15480 2014-08-11 17:06:14.089192 2014-08-13 17:38:03.131764 7fe9bae887a0 20 read_log 192888'14868 (0'0) promote 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by osd.4.0:23060 2014-08-11 17:06:16.008122 2014-08-13 17:38:03.131776 7fe9bae887a0 20 read_log 192888'14869 (192888'14868) modify 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.942868.0:15973 2014-08-11 17:06:15.997883 2014-08-13 17:38:03.131788 7fe9bae887a0 20 read_log 192888'14870 (0'0) promote f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by osd.4.0:23066 2014-08-11 17:06:18.390139 2014-08-13 17:38:03.131800 7fe9bae887a0 20 read_log 192888'14871 (192888'14870) modify f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.942868.0:16571 2014-08-11 17:06:18.381535 2014-08-13 17:38:03.131814 7fe9bae887a0 20 read_log 192888'14872 (0'0) promote e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by osd.4.0:23074 2014-08-11 17:06:19.360374 2014-08-13 17:38:03.131826 7fe9bae887a0 20 read_log 192888'14873 (192888'14872) modify e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.942868.0:16826 2014-08-11 17:06:19.351425 2014-08-13 17:38:03.131838 7fe9bae887a0 20 read_log 192888'14882 (0'0) promote dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.4.0:23134 2014-08-11 17:06:26.916121 2014-08-13 17:38:03.131850 7fe9bae887a0 20 read_log 192888'14883 (192888'14882) modify dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by client.942868.0:18818 2014-08-11 17:06:26.908081 2014-08-13 17:38:03.131862 7fe9bae887a0 20 read_log 192888'14890 (0'0) promote 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.4.0:23170 2014-08-11 17:06:31.231601 2014-08-13 17:38:03.131874 7fe9bae887a0 20 read_log 192888'14891 (192888'14890) modify 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by client.942868.0:19964 2014-08-11 17:06:31.223850 2014-08-13 17:38:03.131886 7fe9bae887a0 20 read_log 192888'14892 (0'0) promote bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by osd.4.0:23210 2014-08-11 17:06:38.024771 2014-08-13 17:38:03.131898 7fe9bae887a0 20 read_log 192888'14893 (192888'14892) modify bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.942868.0:21744 2014-08-11 17:06:38.016662 2014-08-13 17:38:03.131910 7fe9bae887a0 20 read_log 192888'14894 (0'0) promote 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by osd.4.0:23220 2014-08-11 17:06:38.888915 2014-08-13 17:38:03.131922 7fe9bae887a0 20 read_log 192888'14895 (192888'14894) modify 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.942868.0:21976 2014-08-11 17:06:38.879263 2014-08-13 17:38:03.131934 7fe9bae887a0 20 read_log 192888'14896 (0'0) promote 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by osd.4.0:23226 2014-08-11 17:06:39.777720 2014-08-13 17:38:03.131972 7fe9bae887a0 20 read_log 192888'14897 (192888'14896) modify 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.942868.0:22211 2014-08-11 17:06:39.767946 2014-08-13 17:38:03.131988 7fe9bae887a0 20 read_log 192888'14898 (0'0) promote a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.4.0:23238 2014-08-11 17:06:41.504203 2014-08-13 17:38:03.132001 7fe9bae887a0 20 read_log 192888'14899 (192888'14898) modify a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by client.942868.0:22632 2014-08-11 17:06:41.495727 2014-08-13 17:38:03.132013 7fe9bae887a0 20 read_log 192888'14906 (0'0) promote 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by osd.4.0:23274 2014-08-11 17:06:46.280268 2014-08-13 17:38:03.132025 7fe9bae887a0 20 read_log 192888'14907 (192888'14906) modify 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.942868.0:23882 2014-08-11 17:06:46.270240 2014-08-13 17:38:03.132037 7fe9bae887a0 20 read_log 192888'14910 (0'0) promote f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.4.0:23288 2014-08-11 17:06:47.235678 2014-08-13 17:38:03.132049 7fe9bae887a0 20 read_log 192888'14911 (192888'14910) modify f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by client.942868.0:24141 2014-08-11 17:06:47.227811 2014-08-13 17:38:03.132061 7fe9bae887a0 20 read_log 192888'14912 (0'0) promote a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.4.0:23306 2014-08-11 17:06:48.863495 2014-08-13 17:38:03.132073 7fe9bae887a0 20 read_log 192888'14913 (192888'14912) modify a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by client.942868.0:24580 2014-08-11 17:06:48.854857 2014-08-13 17:38:03.132085 7fe9bae887a0 20 read_log 192888'14922 (0'0) promote 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.4.0:23426 2014-08-11 17:07:07.315042 2014-08-13 17:38:03.132097 7fe9bae887a0 20 read_log 192888'14923 (192888'14922) modify 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by client.942868.0:29389 2014-08-11 17:07:07.305968 2014-08-13 17:38:03.132109 7fe9bae887a0 20 read_log 192888'14924 (0'0) promote 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.4.0:23444 2014-08-11 17:07:10.351501 2014-08-13 17:38:03.132121 7fe9bae887a0 20 read_log 192888'14925 (192888'14924) modify 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by client.942868.0:30125 2014-08-11 17:07:10.339840 2014-08-13 17:38:03.132134 7fe9bae887a0 20 read_log 192888'14934 (192888'14817) modify 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by client.942868.0:31451 2014-08-11 17:07:15.928922 2014-08-13 17:38:03.132146 7fe9bae887a0 20 read_log 192888'14937 (192888'14823) modify 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by client.942868.0:32772 2014-08-11 17:07:16.341604 2014-08-13 17:38:03.132163 7fe9bae887a0 20 read_log 192888'14938 (192888'14825) modify e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.942868.0:34239 2014-08-11 17:07:16.734995 2014-08-13 17:38:03.132176 7fe9bae887a0 20 read_log 192888'14941 (192888'14831) modify dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by client.942868.0:35457 2014-08-11 17:07:17.078911 2014-08-13 17:38:03.132188 7fe9bae887a0 20 read_log 192888'14942 (192888'14833) modify 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by client.942868.0:35568 2014-08-11 17:07:17.106818 2014-08-13 17:38:03.132200 7fe9bae887a0 20 read_log 192888'14943 (192888'14835) modify 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.942868.0:36026 2014-08-11 17:07:17.223235 2014-08-13 17:38:03.132213 7fe9bae887a0 20 read_log 192888'14944 (192888'14837) modify e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by client.942868.0:36717 2014-08-11 17:07:17.416323 2014-08-13 17:38:03.132233 7fe9bae887a0 20 read_log 192888'14947 (192888'14843) modify 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.942868.0:37577 2014-08-11 17:07:17.650146 2014-08-13 17:38:03.132248 7fe9bae887a0 20 read_log 192888'14950 (192888'14849) modify fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.942868.0:38793 2014-08-11 17:07:17.989391 2014-08-13 17:38:03.132260 7fe9bae887a0 20 read_log 192888'14953 (192888'14855) modify 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by client.942868.0:39637 2014-08-11 17:07:18.240296 2014-08-13 17:38:03.132273 7fe9bae887a0 20 read_log 192888'14954 (192888'14857) modify 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.942868.0:40079 2014-08-11 17:07:18.359447 2014-08-13 17:38:03.132285 7fe9bae887a0 20 read_log 192888'14955 (192888'14859) modify abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by client.942868.0:40302 2014-08-11 17:07:18.414780 2014-08-13 17:38:03.132297 7fe9bae887a0 20 read_log 192888'14958 (192888'14865) modify 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.942868.0:45645 2014-08-11 17:07:19.895067 2014-08-13 17:38:03.132309 7fe9bae887a0 20 read_log 192888'14959 (192888'14867) modify 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.942868.0:46746 2014-08-11 17:07:20.207543 2014-08-13 17:38:03.132320 7fe9bae887a0 20 read_log 192888'14960 (192888'14869) modify 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.942868.0:47239 2014-08-11 17:07:20.347923 2014-08-13 17:38:03.132332 7fe9bae887a0 20 read_log 192888'14961 (192888'14871) modify f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.942868.0:47837 2014-08-11 17:07:20.536038 2014-08-13 17:38:03.132344 7fe9bae887a0 20 read_log 192888'14962 (192888'14873) modify e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.942868.0:48092 2014-08-11 17:07:20.620130 2014-08-13 17:38:03.132356 7fe9bae887a0 20 read_log 192888'14967 (192888'14883) modify dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by client.942868.0:50084 2014-08-11 17:07:21.213156 2014-08-13 17:38:03.132368 7fe9bae887a0 20 read_log 192888'14971 (192888'14891) modify 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by client.942868.0:51230 2014-08-11 17:07:21.579113 2014-08-13 17:38:03.132380 7fe9bae887a0 20 read_log 192888'14972 (192888'14893) modify bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.942868.0:53010 2014-08-11 17:07:22.121635 2014-08-13 17:38:03.132392 7fe9bae887a0 20 read_log 192888'14973 (192888'14895) modify 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.942868.0:53242 2014-08-11 17:07:22.193677 2014-08-13 17:38:03.132404 7fe9bae887a0 20 read_log 192888'14974 (192888'14897) modify 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.942868.0:53477 2014-08-11 17:07:22.276795 2014-08-13 17:38:03.132416 7fe9bae887a0 20 read_log 192888'14975 (192888'14899) modify a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by client.942868.0:53898 2014-08-11 17:07:22.410118 2014-08-13 17:38:03.132429 7fe9bae887a0 20 read_log 192888'14979 (192888'14907) modify 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.942868.0:55148 2014-08-11 17:07:22.758011 2014-08-13 17:38:03.132441 7fe9bae887a0 20 read_log 192888'14981 (192888'14911) modify f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by client.942868.0:55407 2014-08-11 17:07:22.838290 2014-08-13 17:38:03.132453 7fe9bae887a0 20 read_log 192888'14982 (192888'14913) modify a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by client.942868.0:55846 2014-08-11 17:07:22.974069 2014-08-13 17:38:03.132465 7fe9bae887a0 20 read_log 192888'14987 (192888'14923) modify 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by client.942868.0:60655 2014-08-11 17:07:24.406260 2014-08-13 17:38:03.132484 7fe9bae887a0 20 read_log 192888'14988 (192888'14925) modify 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by client.942868.0:61391 2014-08-11 17:07:24.626766 2014-08-13 17:38:03.132497 7fe9bae887a0 20 read_log 193305'14993 (0'0) promote 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by osd.330.0:15 2014-08-11 18:42:35.340522 2014-08-13 17:38:03.132509 7fe9bae887a0 20 read_log 193305'14994 (193305'14993) modify 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.951337.0:563 2014-08-11 18:42:35.331304 2014-08-13 17:38:03.132522 7fe9bae887a0 20 read_log 193305'14995 (0'0) promote 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:21 2014-08-11 18:42:36.670029 2014-08-13 17:38:03.132534 7fe9bae887a0 20 read_log 193305'14996 (193305'14995) modify 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by client.951337.0:916 2014-08-11 18:42:36.663123 2014-08-13 17:38:03.132546 7fe9bae887a0 20 read_log 193305'14997 (0'0) promote 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:23 2014-08-11 18:42:36.693711 2014-08-13 17:38:03.132558 7fe9bae887a0 20 read_log 193305'14998 (193305'14997) modify 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by client.951337.0:923 2014-08-11 18:42:36.687744 2014-08-13 17:38:03.132570 7fe9bae887a0 20 read_log 193305'14999 (0'0) promote f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:25 2014-08-11 18:42:38.375226 2014-08-13 17:38:03.132584 7fe9bae887a0 20 read_log 193305'15000 (193305'14999) modify f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by client.951337.0:1387 2014-08-11 18:42:38.367811 2014-08-13 17:38:03.132597 7fe9bae887a0 20 read_log 193305'15001 (0'0) promote d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by osd.330.0:43 2014-08-11 18:42:41.192695 2014-08-13 17:38:03.132608 7fe9bae887a0 20 read_log 193305'15002 (193305'15001) modify d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.951337.0:2119 2014-08-11 18:42:41.185186 2014-08-13 17:38:03.132621 7fe9bae887a0 20 read_log 193305'15003 (0'0) promote 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:57 2014-08-11 18:42:45.914253 2014-08-13 17:38:03.132633 7fe9bae887a0 20 read_log 193305'15004 (193305'15003) modify 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by client.951337.0:3417 2014-08-11 18:42:45.902501 2014-08-13 17:38:03.132645 7fe9bae887a0 20 read_log 193305'15005 (0'0) promote d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:59 2014-08-11 18:42:45.993795 2014-08-13 17:38:03.132657 7fe9bae887a0 20 read_log 193305'15006 (193305'15005) modify d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by client.951337.0:3439 2014-08-11 18:42:45.983241 2014-08-13 17:38:03.132687 7fe9bae887a0 20 read_log 193305'15007 (0'0) promote f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by osd.330.0:65 2014-08-11 18:42:46.530813 2014-08-13 17:38:03.132703 7fe9bae887a0 20 read_log 193305'15008 (193305'15007) modify f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.951337.0:3590 2014-08-11 18:42:46.523695 2014-08-13 17:38:03.132715 7fe9bae887a0 20 read_log 193305'15009 (0'0) promote eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by osd.330.0:95 2014-08-11 18:42:53.606651 2014-08-13 17:38:03.132727 7fe9bae887a0 20 read_log 193305'15010 (193305'15009) modify eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.951337.0:5500 2014-08-11 18:42:53.599454 2014-08-13 17:38:03.132739 7fe9bae887a0 20 read_log 193305'15011 (0'0) promote 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by osd.330.0:101 2014-08-11 18:42:55.925446 2014-08-13 17:38:03.132761 7fe9bae887a0 20 read_log 193305'15012 (193305'15011) modify 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.951337.0:6137 2014-08-11 18:42:55.918404 2014-08-13 17:38:03.132776 7fe9bae887a0 20 read_log 193305'15013 (0'0) promote 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:109 2014-08-11 18:42:58.182279 2014-08-13 17:38:03.132788 7fe9bae887a0 20 read_log 193305'15014 (193305'15013) modify 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by client.951337.0:6761 2014-08-11 18:42:58.175495 2014-08-13 17:38:03.132800 7fe9bae887a0 20 read_log 193305'15015 (0'0) promote ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by osd.330.0:177 2014-08-11 18:43:10.535781 2014-08-13 17:38:03.132812 7fe9bae887a0 20 read_log 193305'15016 (193305'15015) modify ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.951337.0:10067 2014-08-11 18:43:10.528566 2014-08-13 17:38:03.132824 7fe9bae887a0 20 read_log 193305'15017 (0'0) promote a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by osd.330.0:248 2014-08-11 18:43:26.437496 2014-08-13 17:38:03.132836 7fe9bae887a0 20 read_log 193305'15018 (193305'15017) modify a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.951337.0:14260 2014-08-11 18:43:26.426670 2014-08-13 17:38:03.132848 7fe9bae887a0 20 read_log 193305'15019 (0'0) promote af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by osd.330.0:250 2014-08-11 18:43:26.827260 2014-08-13 17:38:03.132860 7fe9bae887a0 20 read_log 193305'15020 (193305'15019) modify af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.951337.0:14379 2014-08-11 18:43:26.818495 2014-08-13 17:38:03.132873 7fe9bae887a0 20 read_log 193305'15021 (0'0) promote 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:254 2014-08-11 18:43:27.247088 2014-08-13 17:38:03.132884 7fe9bae887a0 20 read_log 193305'15022 (193305'15021) modify 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by client.951337.0:14507 2014-08-11 18:43:27.240157 2014-08-13 17:38:03.132896 7fe9bae887a0 20 read_log 193305'15023 (0'0) promote 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:258 2014-08-11 18:43:27.394066 2014-08-13 17:38:03.132908 7fe9bae887a0 20 read_log 193305'15024 (193305'15023) modify 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by client.951337.0:14552 2014-08-11 18:43:27.387768 2014-08-13 17:38:03.132920 7fe9bae887a0 20 read_log 193305'15025 (0'0) promote 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by osd.330.0:280 2014-08-11 18:43:32.332630 2014-08-13 17:38:03.132932 7fe9bae887a0 20 read_log 193305'15026 (193305'15025) modify 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.951337.0:15984 2014-08-11 18:43:32.324039 2014-08-13 17:38:03.132944 7fe9bae887a0 20 read_log 193305'15027 (0'0) promote e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by osd.330.0:290 2014-08-11 18:43:33.646698 2014-08-13 17:38:03.132956 7fe9bae887a0 20 read_log 193305'15028 (193305'15027) modify e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.951337.0:16339 2014-08-11 18:43:33.637715 2014-08-13 17:38:03.132968 7fe9bae887a0 20 read_log 193305'15029 (0'0) promote 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:296 2014-08-11 18:43:34.896768 2014-08-13 17:38:03.132980 7fe9bae887a0 20 read_log 193305'15030 (193305'15029) modify 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by client.951337.0:16712 2014-08-11 18:43:34.890803 2014-08-13 17:38:03.132992 7fe9bae887a0 20 read_log 193305'15031 (0'0) promote 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:316 2014-08-11 18:43:39.256170 2014-08-13 17:38:03.133012 7fe9bae887a0 20 read_log 193305'15032 (193305'15031) modify 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by client.951337.0:17961 2014-08-11 18:43:39.248285 2014-08-13 17:38:03.133026 7fe9bae887a0 20 read_log 193305'15033 (0'0) promote c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:330 2014-08-11 18:43:41.319741 2014-08-13 17:38:03.133038 7fe9bae887a0 20 read_log 193305'15034 (193305'15033) modify c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by client.951337.0:18556 2014-08-11 18:43:41.313165 2014-08-13 17:38:03.133051 7fe9bae887a0 20 read_log 193305'15035 (0'0) promote f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by osd.330.0:346 2014-08-11 18:43:45.432300 2014-08-13 17:38:03.133063 7fe9bae887a0 20 read_log 193305'15036 (193305'15035) modify f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.951337.0:19664 2014-08-11 18:43:45.424264 2014-08-13 17:38:03.133075 7fe9bae887a0 20 read_log 193305'15037 (0'0) promote d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:348 2014-08-11 18:43:45.771971 2014-08-13 17:38:03.133087 7fe9bae887a0 20 read_log 193305'15038 (193305'15037) modify d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by client.951337.0:19761 2014-08-11 18:43:45.762916 2014-08-13 17:38:03.133099 7fe9bae887a0 20 read_log 193305'15039 (0'0) promote ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by osd.330.0:354 2014-08-11 18:43:48.032963 2014-08-13 17:38:03.133111 7fe9bae887a0 20 read_log 193305'15040 (193305'15039) modify ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.951337.0:20381 2014-08-11 18:43:48.025253 2014-08-13 17:38:03.133123 7fe9bae887a0 20 read_log 193305'15041 (0'0) promote cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:356 2014-08-11 18:43:48.660399 2014-08-13 17:38:03.133135 7fe9bae887a0 20 read_log 193305'15042 (193305'15041) modify cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by client.951337.0:20560 2014-08-11 18:43:48.648947 2014-08-13 17:38:03.133148 7fe9bae887a0 20 read_log 193305'15043 (0'0) promote 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by osd.330.0:380 2014-08-11 18:43:55.986416 2014-08-13 17:38:03.133166 7fe9bae887a0 20 read_log 193305'15044 (193305'15043) modify 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.951337.0:22593 2014-08-11 18:43:55.979183 2014-08-13 17:38:03.133179 7fe9bae887a0 20 read_log 193305'15045 (0'0) promote e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by osd.330.0:412 2014-08-11 18:44:02.715279 2014-08-13 17:38:03.133191 7fe9bae887a0 20 read_log 193305'15046 (193305'15045) modify e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.951337.0:24468 2014-08-11 18:44:02.705878 2014-08-13 17:38:03.133203 7fe9bae887a0 20 read_log 193305'15047 (0'0) promote e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by osd.330.0:414 2014-08-11 18:44:03.092933 2014-08-13 17:38:03.133215 7fe9bae887a0 20 read_log 193305'15048 (193305'15047) modify e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.951337.0:24570 2014-08-11 18:44:03.084880 2014-08-13 17:38:03.133227 7fe9bae887a0 20 read_log 193305'15049 (0'0) promote cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by osd.330.0:434 2014-08-11 18:44:07.569587 2014-08-13 17:38:03.133239 7fe9bae887a0 20 read_log 193305'15050 (193305'15049) modify cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.951337.0:25786 2014-08-11 18:44:07.562019 2014-08-13 17:38:03.133251 7fe9bae887a0 20 read_log 193305'15051 (0'0) promote 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by osd.330.0:436 2014-08-11 18:44:07.578968 2014-08-13 17:38:03.133271 7fe9bae887a0 20 read_log 193305'15052 (193305'15051) modify 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.951337.0:25787 2014-08-11 18:44:07.566360 2014-08-13 17:38:03.133284 7fe9bae887a0 20 read_log 193305'15053 (0'0) promote 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by osd.330.0:442 2014-08-11 18:44:08.698966 2014-08-13 17:38:03.133296 7fe9bae887a0 20 read_log 193305'15054 (193305'15053) modify 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.951337.0:26080 2014-08-11 18:44:08.690787 2014-08-13 17:38:03.133308 7fe9bae887a0 20 read_log 193305'15055 (0'0) promote 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by osd.330.0:458 2014-08-11 18:44:11.600308 2014-08-13 17:38:03.133320 7fe9bae887a0 20 read_log 193305'15056 (193305'15055) modify 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.951337.0:26893 2014-08-11 18:44:11.591770 2014-08-13 17:38:03.133332 7fe9bae887a0 20 read_log 193305'15057 (0'0) promote 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by osd.330.0:460 2014-08-11 18:44:11.778828 2014-08-13 17:38:03.133344 7fe9bae887a0 20 read_log 193305'15058 (193305'15057) modify 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.951337.0:26935 2014-08-11 18:44:11.770765 2014-08-13 17:38:03.133356 7fe9bae887a0 20 read_log 193305'15059 (0'0) promote ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:466 2014-08-11 18:44:12.685596 2014-08-13 17:38:03.133368 7fe9bae887a0 20 read_log 193305'15060 (193305'15059) modify ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by client.951337.0:27171 2014-08-11 18:44:12.679213 2014-08-13 17:38:03.133380 7fe9bae887a0 20 read_log 193305'15061 (193305'15061) modify 3f7/hit_set_24.3f7_archive_2014-08-11 18:39:14.563860_2014-08-11 18:44:18.014758/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:18.015046 2014-08-13 17:38:03.133393 7fe9bae887a0 20 read_log 193305'15062 (0'0) promote 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:477 2014-08-11 18:44:18.018055 2014-08-13 17:38:03.133405 7fe9bae887a0 20 read_log 193305'15063 (193305'15062) modify 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by client.951337.0:28650 2014-08-11 18:44:18.009271 2014-08-13 17:38:03.133418 7fe9bae887a0 20 read_log 193305'15064 (0'0) promote ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:481 2014-08-11 18:44:18.373399 2014-08-13 17:38:03.133430 7fe9bae887a0 20 read_log 193305'15065 (193305'15064) modify ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by client.951337.0:28745 2014-08-11 18:44:18.365716 2014-08-13 17:38:03.133442 7fe9bae887a0 20 read_log 193305'15066 (0'0) promote fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by osd.330.0:491 2014-08-11 18:44:19.770796 2014-08-13 17:38:03.133454 7fe9bae887a0 20 read_log 193305'15067 (193305'15066) modify fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.951337.0:29117 2014-08-11 18:44:19.763161 2014-08-13 17:38:03.133467 7fe9bae887a0 20 read_log 193305'15068 (0'0) promote dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by osd.330.0:501 2014-08-11 18:44:21.006648 2014-08-13 17:38:03.133479 7fe9bae887a0 20 read_log 193305'15069 (193305'15068) modify dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.951337.0:29455 2014-08-11 18:44:20.999706 2014-08-13 17:38:03.133491 7fe9bae887a0 20 read_log 193305'15070 (0'0) promote 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by osd.330.0:503 2014-08-11 18:44:22.424664 2014-08-13 17:38:03.133513 7fe9bae887a0 20 read_log 193305'15071 (193305'15070) modify 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.951337.0:29841 2014-08-11 18:44:22.413648 2014-08-13 17:38:03.133525 7fe9bae887a0 20 read_log 193305'15072 (0'0) promote 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by osd.330.0:543 2014-08-11 18:44:32.651066 2014-08-13 17:38:03.133538 7fe9bae887a0 20 read_log 193305'15073 (193305'15072) modify 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.951337.0:32663 2014-08-11 18:44:32.644122 2014-08-13 17:38:03.133550 7fe9bae887a0 20 read_log 193305'15074 (193305'14994) modify 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.951337.0:33274 2014-08-11 18:44:34.291001 2014-08-13 17:38:03.133582 7fe9bae887a0 20 read_log 193305'15075 (193305'14996) modify 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by client.951337.0:33627 2014-08-11 18:44:34.543969 2014-08-13 17:38:03.133599 7fe9bae887a0 20 read_log 193305'15076 (193305'14998) modify 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by client.951337.0:33634 2014-08-11 18:44:34.547736 2014-08-13 17:38:03.133611 7fe9bae887a0 20 read_log 193305'15077 (193305'15000) modify f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by client.951337.0:34098 2014-08-11 18:44:34.794341 2014-08-13 17:38:03.133623 7fe9bae887a0 20 read_log 193305'15078 (193305'15002) modify d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.951337.0:34830 2014-08-11 18:44:35.212586 2014-08-13 17:38:03.133635 7fe9bae887a0 20 read_log 193305'15079 (193305'15004) modify 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by client.951337.0:36128 2014-08-11 18:44:35.949615 2014-08-13 17:38:03.133648 7fe9bae887a0 20 read_log 193305'15080 (193305'15006) modify d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by client.951337.0:36150 2014-08-11 18:44:35.957419 2014-08-13 17:38:03.133660 7fe9bae887a0 20 read_log 193305'15081 (193305'15008) modify f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.951337.0:36301 2014-08-11 18:44:36.122344 2014-08-13 17:38:03.133672 7fe9bae887a0 20 read_log 193305'15082 (193305'15010) modify eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.951337.0:38211 2014-08-11 18:44:37.288830 2014-08-13 17:38:03.133683 7fe9bae887a0 20 read_log 193305'15083 (193305'15012) modify 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.951337.0:38848 2014-08-11 18:44:37.689692 2014-08-13 17:38:03.133695 7fe9bae887a0 20 read_log 193305'15084 (193305'15014) modify 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by client.951337.0:39472 2014-08-11 18:44:38.030317 2014-08-13 17:38:03.133707 7fe9bae887a0 20 read_log 193305'15085 (193305'15016) modify ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.951337.0:42778 2014-08-11 18:44:39.377246 2014-08-13 17:38:03.133719 7fe9bae887a0 20 read_log 193305'15086 (193305'15018) modify a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.951337.0:46971 2014-08-11 18:44:40.639963 2014-08-13 17:38:03.133731 7fe9bae887a0 20 read_log 193305'15087 (193305'15020) modify af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.951337.0:47090 2014-08-11 18:44:40.671435 2014-08-13 17:38:03.133743 7fe9bae887a0 20 read_log 193305'15088 (193305'15022) modify 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by client.951337.0:47218 2014-08-11 18:44:40.704907 2014-08-13 17:38:03.133755 7fe9bae887a0 20 read_log 193305'15089 (193305'15024) modify 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by client.951337.0:47263 2014-08-11 18:44:40.719186 2014-08-13 17:38:03.133767 7fe9bae887a0 20 read_log 193305'15090 (193305'15026) modify 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.951337.0:48695 2014-08-11 18:44:41.129240 2014-08-13 17:38:03.133803 7fe9bae887a0 20 read_log 193305'15091 (193305'15028) modify e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.951337.0:49050 2014-08-11 18:44:41.237018 2014-08-13 17:38:03.133819 7fe9bae887a0 20 read_log 193305'15092 (193305'15030) modify 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by client.951337.0:49423 2014-08-11 18:44:41.361921 2014-08-13 17:38:03.133831 7fe9bae887a0 20 read_log 193305'15093 (193305'15032) modify 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by client.951337.0:50672 2014-08-11 18:44:41.731056 2014-08-13 17:38:03.133843 7fe9bae887a0 20 read_log 193305'15094 (193305'15034) modify c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by client.951337.0:51267 2014-08-11 18:44:41.902985 2014-08-13 17:38:03.133855 7fe9bae887a0 20 read_log 193305'15095 (193305'15036) modify f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.951337.0:52375 2014-08-11 18:44:42.216205 2014-08-13 17:38:03.133867 7fe9bae887a0 20 read_log 193305'15096 (193305'15038) modify d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by client.951337.0:52472 2014-08-11 18:44:42.243173 2014-08-13 17:38:03.133879 7fe9bae887a0 20 read_log 193305'15097 (193305'15040) modify ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.951337.0:53092 2014-08-11 18:44:42.426736 2014-08-13 17:38:03.133891 7fe9bae887a0 20 read_log 193305'15098 (193305'15042) modify cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by client.951337.0:53271 2014-08-11 18:44:42.480082 2014-08-13 17:38:03.133903 7fe9bae887a0 20 read_log 193305'15099 (193305'15044) modify 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.951337.0:55304 2014-08-11 18:44:43.083333 2014-08-13 17:38:03.133931 7fe9bae887a0 20 read_log 193305'15100 (193305'15046) modify e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.951337.0:57179 2014-08-11 18:44:43.660596 2014-08-13 17:38:03.133943 7fe9bae887a0 20 read_log 193305'15101 (193305'15048) modify e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.951337.0:57281 2014-08-11 18:44:43.700911 2014-08-13 17:38:03.133955 7fe9bae887a0 20 read_log 193305'15102 (193305'15050) modify cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.951337.0:58497 2014-08-11 18:44:44.059674 2014-08-13 17:38:03.133968 7fe9bae887a0 20 read_log 193305'15103 (193305'15052) modify 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.951337.0:58498 2014-08-11 18:44:44.059768 2014-08-13 17:38:03.133980 7fe9bae887a0 20 read_log 193305'15104 (193305'15054) modify 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.951337.0:58791 2014-08-11 18:44:44.142842 2014-08-13 17:38:03.133992 7fe9bae887a0 20 read_log 193305'15105 (193305'15056) modify 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.951337.0:59604 2014-08-11 18:44:44.374900 2014-08-13 17:38:03.134003 7fe9bae887a0 20 read_log 193305'15106 (193305'15058) modify 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.951337.0:59646 2014-08-11 18:44:44.385389 2014-08-13 17:38:03.134015 7fe9bae887a0 20 read_log 193305'15107 (193305'15060) modify ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by client.951337.0:59882 2014-08-11 18:44:44.454166 2014-08-13 17:38:03.134027 7fe9bae887a0 20 read_log 193305'15108 (193305'15063) modify 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by client.951337.0:61361 2014-08-11 18:44:44.901608 2014-08-13 17:38:03.134039 7fe9bae887a0 20 read_log 193305'15109 (193305'15065) modify ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by client.951337.0:61456 2014-08-11 18:44:44.926806 2014-08-13 17:38:03.134054 7fe9bae887a0 20 read_log 193305'15110 (193305'15067) modify fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.951337.0:61828 2014-08-11 18:44:45.043875 2014-08-13 17:38:03.134075 7fe9bae887a0 20 read_log 193305'15111 (193305'15069) modify dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.951337.0:62166 2014-08-11 18:44:45.144826 2014-08-13 17:38:03.134089 7fe9bae887a0 20 read_log 193305'15112 (193305'15071) modify 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.951337.0:62552 2014-08-11 18:44:45.261257 2014-08-13 17:38:03.134102 7fe9bae887a0 20 read_log 193305'15113 (193305'15073) modify 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.951337.0:65374 2014-08-11 18:44:46.106070 2014-08-13 17:38:03.134114 7fe9bae887a0 20 read_log 194209'15114 (192888'14942) clean 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.330.0:971 2014-08-12 14:05:29.932182 2014-08-13 17:38:03.134128 7fe9bae887a0 20 read_log 194209'15115 (192888'14967) clean dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.330.0:972 2014-08-12 14:05:29.932576 2014-08-13 17:38:03.134141 7fe9bae887a0 20 read_log 194209'15116 (193305'15076) clean 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:973 2014-08-12 14:05:29.932961 2014-08-13 17:38:03.134159 7fe9bae887a0 20 read_log 194209'15117 (192888'14987) clean 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.330.0:974 2014-08-12 14:05:29.933417 2014-08-13 17:38:03.134172 7fe9bae887a0 20 read_log 194209'15118 (192888'14944) clean e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.330.0:975 2014-08-12 14:05:29.957206 2014-08-13 17:38:03.134184 7fe9bae887a0 20 read_log 194209'15119 (192834'14018) clean 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.330.0:976 2014-08-12 14:05:29.960577 2014-08-13 17:38:03.134197 7fe9bae887a0 20 read_log 194209'15120 (192834'14037) clean 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.330.0:977 2014-08-12 14:05:29.963636 2014-08-13 17:38:03.134209 7fe9bae887a0 20 read_log 194209'15121 (193305'15108) clean 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:978 2014-08-12 14:05:29.967008 2014-08-13 17:38:03.134224 7fe9bae887a0 20 read_log 194209'15122 (193305'15098) clean cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:979 2014-08-12 14:05:29.970238 2014-08-13 17:38:03.134236 7fe9bae887a0 20 read_log 194209'15123 (192888'14941) clean dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.330.0:980 2014-08-12 14:05:29.973565 2014-08-13 17:38:03.134251 7fe9bae887a0 20 read_log 194209'15124 (192888'14988) clean 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.330.0:981 2014-08-12 14:05:29.976777 2014-08-13 17:38:03.134263 7fe9bae887a0 20 read_log 194209'15125 (193305'15096) clean d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:982 2014-08-12 14:05:29.979864 2014-08-13 17:38:03.134275 7fe9bae887a0 20 read_log 194209'15126 (192888'14781) clean 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.330.0:983 2014-08-12 14:05:29.982946 2014-08-13 17:38:03.134287 7fe9bae887a0 20 read_log 194209'15127 (192888'14975) clean a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.330.0:984 2014-08-12 14:05:29.986129 2014-08-13 17:38:03.134299 7fe9bae887a0 20 read_log 194209'15128 (192888'14792) clean 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.330.0:985 2014-08-12 14:05:29.989173 2014-08-13 17:38:03.134311 7fe9bae887a0 20 read_log 194209'15129 (192888'14761) clean a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.330.0:986 2014-08-12 14:05:29.994944 2014-08-13 17:38:03.134324 7fe9bae887a0 20 read_log 194209'15130 (193305'15109) clean ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:987 2014-08-12 14:05:29.997864 2014-08-13 17:38:03.134346 7fe9bae887a0 20 read_log 194209'15131 (193305'15079) clean 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:988 2014-08-12 14:05:29.998164 2014-08-13 17:38:03.134361 7fe9bae887a0 20 read_log 194209'15132 (192834'14102) clean fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.330.0:989 2014-08-12 14:05:30.001608 2014-08-13 17:38:03.134374 7fe9bae887a0 20 read_log 194209'15133 (192888'14981) clean f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.330.0:990 2014-08-12 14:05:30.005109 2014-08-13 17:38:03.134386 7fe9bae887a0 20 read_log 194209'15134 (192834'14090) clean daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by osd.330.0:991 2014-08-12 14:05:30.010564 2014-08-13 17:38:03.134398 7fe9bae887a0 20 read_log 194209'15135 (192888'14802) clean 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by osd.330.0:992 2014-08-12 14:05:30.010855 2014-08-13 17:38:03.134410 7fe9bae887a0 20 read_log 194209'15136 (193305'15093) clean 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:993 2014-08-12 14:05:30.014085 2014-08-13 17:38:03.134423 7fe9bae887a0 20 read_log 194209'15137 (193305'15075) clean 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:994 2014-08-12 14:05:30.017291 2014-08-13 17:38:03.134435 7fe9bae887a0 20 read_log 194209'15138 (192834'14617) clean 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.330.0:995 2014-08-12 14:05:30.020499 2014-08-13 17:38:03.134447 7fe9bae887a0 20 read_log 194209'15139 (192834'14100) clean d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.330.0:996 2014-08-12 14:05:30.023614 2014-08-13 17:38:03.134459 7fe9bae887a0 20 read_log 194209'15140 (192888'14971) clean 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.330.0:997 2014-08-12 14:05:30.026735 2014-08-13 17:38:03.134471 7fe9bae887a0 20 read_log 194209'15141 (192834'14116) clean fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.330.0:998 2014-08-12 14:05:30.029882 2014-08-13 17:38:03.134483 7fe9bae887a0 20 read_log 194209'15142 (193305'15088) clean 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:999 2014-08-12 14:05:30.033404 2014-08-13 17:38:03.134495 7fe9bae887a0 20 read_log 194209'15143 (192888'14953) clean 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.330.0:1000 2014-08-12 14:05:30.036995 2014-08-13 17:38:03.134508 7fe9bae887a0 20 read_log 194209'15144 (192834'14032) clean dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.330.0:1001 2014-08-12 14:05:30.040476 2014-08-13 17:38:03.134520 7fe9bae887a0 20 read_log 194209'15145 (193305'15094) clean c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:1002 2014-08-12 14:05:30.044185 2014-08-13 17:38:03.134532 7fe9bae887a0 20 read_log 194209'15146 (192834'14020) clean f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.330.0:1003 2014-08-12 14:05:30.047754 2014-08-13 17:38:03.134569 7fe9bae887a0 20 read_log 194209'15147 (193305'15080) clean d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:1004 2014-08-12 14:05:30.051444 2014-08-13 17:38:03.134582 7fe9bae887a0 20 read_log 194209'15148 (192834'14036) clean d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.330.0:1005 2014-08-12 14:05:30.055229 2014-08-13 17:38:03.134594 7fe9bae887a0 20 read_log 194209'15149 (192834'14088) clean 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.330.0:1006 2014-08-12 14:05:30.058559 2014-08-13 17:38:03.134606 7fe9bae887a0 20 read_log 194209'15150 (193305'15077) clean f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:1007 2014-08-12 14:05:30.062091 2014-08-13 17:38:03.134627 7fe9bae887a0 20 read_log 194209'15151 (192834'14049) clean 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.330.0:1008 2014-08-12 14:05:30.065522 2014-08-13 17:38:03.134642 7fe9bae887a0 20 read_log 194209'15152 (192888'14955) clean abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.330.0:1009 2014-08-12 14:05:30.069056 2014-08-13 17:38:03.134654 7fe9bae887a0 20 read_log 194209'15153 (192834'14057) clean 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.330.0:1010 2014-08-12 14:05:30.072596 2014-08-13 17:38:03.134667 7fe9bae887a0 20 read_log 194209'15154 (193305'15089) clean 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:1011 2014-08-12 14:05:30.075934 2014-08-13 17:38:03.134679 7fe9bae887a0 20 read_log 194209'15155 (192888'14790) clean fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.330.0:1012 2014-08-12 14:05:30.079497 2014-08-13 17:38:03.134691 7fe9bae887a0 20 read_log 194209'15156 (192888'14793) clean f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.330.0:1013 2014-08-12 14:05:30.083011 2014-08-13 17:38:03.134703 7fe9bae887a0 20 read_log 194209'15157 (192888'14937) clean 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.330.0:1014 2014-08-12 14:05:30.086492 2014-08-13 17:38:03.134715 7fe9bae887a0 20 read_log 194209'15158 (193305'15092) clean 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:1015 2014-08-12 14:05:30.089987 2014-08-13 17:38:03.134727 7fe9bae887a0 20 read_log 194209'15159 (192834'14639) clean 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.330.0:1016 2014-08-12 14:05:30.093830 2014-08-13 17:38:03.134739 7fe9bae887a0 20 read_log 194209'15160 (193305'15107) clean ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:1017 2014-08-12 14:05:30.097320 2014-08-13 17:38:03.134751 7fe9bae887a0 20 read_log 194209'15161 (192888'14982) clean a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.330.0:1018 2014-08-12 14:05:30.100814 2014-08-13 17:38:03.134764 7fe9bae887a0 20 read_log 194209'15162 (192888'14934) clean 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.330.0:1019 2014-08-12 14:05:30.104306 2014-08-13 17:38:03.134776 7fe9bae887a0 20 read_log 194209'15163 (194209'15114) delete 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.330.0:1020 2014-08-12 14:05:30.141397 2014-08-13 17:38:03.134788 7fe9bae887a0 20 read_log 194209'15164 (194209'15115) delete dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.330.0:1021 2014-08-12 14:05:30.144832 2014-08-13 17:38:03.134800 7fe9bae887a0 20 read_log 194209'15165 (194209'15116) delete 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:1022 2014-08-12 14:05:30.148145 2014-08-13 17:38:03.134812 7fe9bae887a0 20 read_log 194209'15166 (194209'15121) delete 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:1023 2014-08-12 14:05:30.151833 2014-08-13 17:38:03.134824 7fe9bae887a0 20 read_log 194209'15167 (193305'15084) clean 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:1024 2014-08-12 14:05:30.152370 2014-08-13 17:38:03.134836 7fe9bae887a0 20 read_log 194209'15168 (194209'15122) delete cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:1025 2014-08-12 14:05:30.155531 2014-08-13 17:38:03.134848 7fe9bae887a0 20 read_log 194209'15169 (194209'15125) delete d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:1026 2014-08-12 14:05:30.158555 2014-08-13 17:38:03.134860 7fe9bae887a0 20 read_log 194209'15170 (194209'15130) delete ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:1027 2014-08-12 14:05:30.159815 2014-08-13 17:38:03.134879 7fe9bae887a0 20 read_log 194209'15171 (194209'15131) delete 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:1028 2014-08-12 14:05:30.162837 2014-08-13 17:38:03.134892 7fe9bae887a0 20 read_log 194209'15172 (194209'15136) delete 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:1029 2014-08-12 14:05:30.164120 2014-08-13 17:38:03.134904 7fe9bae887a0 20 read_log 194209'15173 (194209'15137) delete 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:1030 2014-08-12 14:05:30.167091 2014-08-13 17:38:03.134916 7fe9bae887a0 20 read_log 194209'15174 (194209'15142) delete 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:1031 2014-08-12 14:05:30.168356 2014-08-13 17:38:03.134928 7fe9bae887a0 20 read_log 194209'15175 (194209'15145) delete c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:1032 2014-08-12 14:05:30.171999 2014-08-13 17:38:03.134940 7fe9bae887a0 20 read_log 194209'15176 (194209'15147) delete d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:1033 2014-08-12 14:05:30.172955 2014-08-13 17:38:03.134952 7fe9bae887a0 20 read_log 194209'15177 (194209'15150) delete f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:1034 2014-08-12 14:05:30.177253 2014-08-13 17:38:03.134964 7fe9bae887a0 20 read_log 194209'15178 (194209'15154) delete 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:1035 2014-08-12 14:05:30.178392 2014-08-13 17:38:03.134976 7fe9bae887a0 20 read_log 194209'15179 (194209'15158) delete 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:1036 2014-08-12 14:05:30.182239 2014-08-13 17:38:03.134988 7fe9bae887a0 20 read_log 194209'15180 (194209'15160) delete ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:1037 2014-08-12 14:05:30.182669 2014-08-13 17:38:03.135000 7fe9bae887a0 20 read_log 194209'15181 (194209'15167) delete 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:1038 2014-08-12 14:05:30.185713 2014-08-13 17:38:03.135012 7fe9bae887a0 20 read_log 194209'15182 (194209'15138) delete 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.330.0:1195 2014-08-12 14:05:45.334872 2014-08-13 17:38:03.135024 7fe9bae887a0 20 read_log 194209'15183 (194209'15139) delete d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.330.0:1196 2014-08-12 14:05:45.335300 2014-08-13 17:38:03.135036 7fe9bae887a0 20 read_log 194209'15184 (194209'15140) delete 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.330.0:1197 2014-08-12 14:05:45.335839 2014-08-13 17:38:03.135048 7fe9bae887a0 20 read_log 194209'15185 (194209'15141) delete fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.330.0:1198 2014-08-12 14:05:45.336770 2014-08-13 17:38:03.135060 7fe9bae887a0 20 read_log 194209'15186 (194209'15143) delete 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.330.0:1199 2014-08-12 14:05:45.340138 2014-08-13 17:38:03.135072 7fe9bae887a0 20 read_log 194209'15187 (194209'15144) delete dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.330.0:1200 2014-08-12 14:05:45.340503 2014-08-13 17:38:03.135084 7fe9bae887a0 20 read_log 194209'15188 (194209'15146) delete f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.330.0:1201 2014-08-12 14:05:45.341330 2014-08-13 17:38:03.135096 7fe9bae887a0 20 read_log 194209'15189 (194209'15148) delete d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.330.0:1202 2014-08-12 14:05:45.341690 2014-08-13 17:38:03.135108 7fe9bae887a0 20 read_log 194209'15190 (194209'15149) delete 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.330.0:1203 2014-08-12 14:05:45.345275 2014-08-13 17:38:03.135129 7fe9bae887a0 20 read_log 194209'15191 (194209'15151) delete 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.330.0:1204 2014-08-12 14:05:45.345671 2014-08-13 17:38:03.135142 7fe9bae887a0 20 read_log 194209'15192 (194209'15152) delete abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.330.0:1205 2014-08-12 14:05:45.346304 2014-08-13 17:38:03.135175 7fe9bae887a0 20 read_log 194209'15193 (194209'15153) delete 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.330.0:1206 2014-08-12 14:05:45.346656 2014-08-13 17:38:03.135191 7fe9bae887a0 20 read_log 194209'15194 (194209'15155) delete fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.330.0:1207 2014-08-12 14:05:45.350553 2014-08-13 17:38:03.135203 7fe9bae887a0 20 read_log 194209'15195 (194209'15156) delete f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.330.0:1208 2014-08-12 14:05:45.351033 2014-08-13 17:38:03.135215 7fe9bae887a0 20 read_log 194209'15196 (194209'15157) delete 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.330.0:1209 2014-08-12 14:05:45.351968 2014-08-13 17:38:03.135228 7fe9bae887a0 20 read_log 194209'15197 (194209'15159) delete 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.330.0:1210 2014-08-12 14:05:45.352386 2014-08-13 17:38:03.135243 7fe9bae887a0 20 read_log 194209'15198 (194209'15161) delete a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.330.0:1211 2014-08-12 14:05:45.355829 2014-08-13 17:38:03.135255 7fe9bae887a0 20 read_log 194209'15199 (194209'15162) delete 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.330.0:1212 2014-08-12 14:05:45.356243 2014-08-13 17:38:03.135268 7fe9bae887a0 20 read_log 194209'15200 (194209'15117) delete 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.330.0:1213 2014-08-12 14:05:45.389163 2014-08-13 17:38:03.135279 7fe9bae887a0 20 read_log 194209'15201 (194209'15118) delete e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.330.0:1214 2014-08-12 14:05:45.389589 2014-08-13 17:38:03.135291 7fe9bae887a0 20 read_log 194209'15202 (194209'15119) delete 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.330.0:1215 2014-08-12 14:05:45.389942 2014-08-13 17:38:03.135305 7fe9bae887a0 20 read_log 194209'15203 (194209'15120) delete 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.330.0:1216 2014-08-12 14:05:45.390286 2014-08-13 17:38:03.135318 7fe9bae887a0 20 read_log 194209'15204 (194209'15123) delete dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.330.0:1217 2014-08-12 14:05:45.393282 2014-08-13 17:38:03.135329 7fe9bae887a0 20 read_log 194209'15205 (194209'15124) delete 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.330.0:1218 2014-08-12 14:05:45.393710 2014-08-13 17:38:03.135342 7fe9bae887a0 20 read_log 194209'15206 (194209'15126) delete 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.330.0:1219 2014-08-12 14:05:45.394599 2014-08-13 17:38:03.135354 7fe9bae887a0 20 read_log 194209'15207 (194209'15127) delete a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.330.0:1220 2014-08-12 14:05:45.395000 2014-08-13 17:38:03.135366 7fe9bae887a0 20 read_log 194209'15208 (194209'15128) delete 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.330.0:1221 2014-08-12 14:05:45.398158 2014-08-13 17:38:03.135378 7fe9bae887a0 20 read_log 194209'15209 (194209'15129) delete a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.330.0:1222 2014-08-12 14:05:45.398557 2014-08-13 17:38:03.135390 7fe9bae887a0 20 read_log 194209'15210 (194209'15132) delete fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.330.0:1223 2014-08-12 14:05:45.399413 2014-08-13 17:38:03.135410 7fe9bae887a0 20 read_log 194209'15211 (194209'15133) delete f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.330.0:1224 2014-08-12 14:05:45.399784 2014-08-13 17:38:03.135423 7fe9bae887a0 20 read_log 194215'15212 (194215'15212) modify 3f7/hit_set_24.3f7_archive_2014-08-12 15:32:25.737625_2014-08-12 15:51:57.155796/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:57.156146 2014-08-13 17:38:03.135436 7fe9bae887a0 20 read_log 194215'15213 (193305'15061) delete 3f7/hit_set_24.3f7_archive_2014-08-11 18:39:14.563860_2014-08-11 18:44:18.014758/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:57.156146 2014-08-13 17:38:03.135448 7fe9bae887a0 20 read_log 194215'15214 (192834'14069) clean b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by osd.330.0:3092 2014-08-12 15:51:57.163015 2014-08-13 17:38:03.135460 7fe9bae887a0 20 read_log 194215'15215 (194215'15214) delete b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.2174595.0:209404 0.000000 2014-08-13 17:38:03.135471 7fe9bae887a0 20 read_log 194215'15216 (192888'14780) clean 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by osd.330.0:3094 2014-08-12 15:51:57.173454 2014-08-13 17:38:03.135484 7fe9bae887a0 20 read_log 194215'15217 (194215'15216) delete 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.2174595.0:209406 0.000000 2014-08-13 17:38:03.135494 7fe9bae887a0 20 read_log 194215'15218 (193305'15103) clean 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by osd.330.0:3096 2014-08-12 15:51:57.183348 2014-08-13 17:38:03.135506 7fe9bae887a0 20 read_log 194215'15219 (194215'15218) delete 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.2174595.0:209408 0.000000 2014-08-13 17:38:03.135517 7fe9bae887a0 20 read_log 194215'15220 (192834'14521) clean cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by osd.330.0:3098 2014-08-12 15:51:57.190832 2014-08-13 17:38:03.135529 7fe9bae887a0 20 read_log 194215'15221 (194215'15220) delete cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.2174595.0:209410 0.000000 2014-08-13 17:38:03.135539 7fe9bae887a0 20 read_log 194215'15222 (192834'14122) clean 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by osd.330.0:3100 2014-08-12 15:51:57.199665 2014-08-13 17:38:03.135551 7fe9bae887a0 20 read_log 194215'15223 (194215'15222) delete 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.2174595.0:209412 0.000000 2014-08-13 17:38:03.135562 7fe9bae887a0 20 read_log 194215'15224 (192888'14958) clean 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by osd.330.0:3102 2014-08-12 15:51:57.208976 2014-08-13 17:38:03.135574 7fe9bae887a0 20 read_log 194215'15225 (194215'15224) delete 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.2174595.0:209414 0.000000 2014-08-13 17:38:03.135584 7fe9bae887a0 20 read_log 194215'15226 (192834'14511) clean 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by osd.330.0:3104 2014-08-12 15:51:57.217245 2014-08-13 17:38:03.135596 7fe9bae887a0 20 read_log 194215'15227 (194215'15226) delete 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.2174595.0:209416 0.000000 2014-08-13 17:38:03.135607 7fe9bae887a0 20 read_log 194215'15228 (192888'14786) clean 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by osd.330.0:3106 2014-08-12 15:51:57.226430 2014-08-13 17:38:03.135619 7fe9bae887a0 20 read_log 194215'15229 (194215'15228) delete 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.2174595.0:209418 0.000000 2014-08-13 17:38:03.135678 7fe9bae887a0 20 read_log 194215'15230 (193305'15085) clean ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by osd.330.0:3108 2014-08-12 15:51:57.235148 2014-08-13 17:38:03.135696 7fe9bae887a0 20 read_log 194215'15231 (194215'15230) delete ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.2174595.0:209420 0.000000 2014-08-13 17:38:03.135707 7fe9bae887a0 20 read_log 194215'15232 (192888'14763) clean c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by osd.330.0:3110 2014-08-12 15:51:57.244290 2014-08-13 17:38:03.135720 7fe9bae887a0 20 read_log 194215'15233 (194215'15232) delete c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.2174595.0:209422 0.000000 2014-08-13 17:38:03.135730 7fe9bae887a0 20 read_log 194215'15234 (192888'14777) clean 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by osd.330.0:3112 2014-08-12 15:51:57.254100 2014-08-13 17:38:03.135742 7fe9bae887a0 20 read_log 194215'15235 (194215'15234) delete 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.2174595.0:209424 0.000000 2014-08-13 17:38:03.135753 7fe9bae887a0 20 read_log 194215'15236 (193305'15095) clean f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by osd.330.0:3114 2014-08-12 15:51:57.262152 2014-08-13 17:38:03.135765 7fe9bae887a0 20 read_log 194215'15237 (194215'15236) delete f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.2174595.0:209426 0.000000 2014-08-13 17:38:03.135776 7fe9bae887a0 20 read_log 194215'15238 (193305'15087) clean af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by osd.330.0:3116 2014-08-12 15:51:57.270789 2014-08-13 17:38:03.135788 7fe9bae887a0 20 read_log 194215'15239 (194215'15238) delete af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.2174595.0:209428 0.000000 2014-08-13 17:38:03.135799 7fe9bae887a0 20 read_log 194215'15240 (192888'14954) clean 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by osd.330.0:3118 2014-08-12 15:51:57.279301 2014-08-13 17:38:03.135811 7fe9bae887a0 20 read_log 194215'15241 (194215'15240) delete 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.2174595.0:209430 0.000000 2014-08-13 17:38:03.135822 7fe9bae887a0 20 read_log 194215'15242 (192888'14788) clean ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by osd.330.0:3120 2014-08-12 15:51:57.287883 2014-08-13 17:38:03.135836 7fe9bae887a0 20 read_log 194215'15243 (194215'15242) delete ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.2174595.0:209432 0.000000 2014-08-13 17:38:03.135846 7fe9bae887a0 20 read_log 194215'15244 (192888'14791) clean 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by osd.330.0:3122 2014-08-12 15:51:57.297959 2014-08-13 17:38:03.135859 7fe9bae887a0 20 read_log 194215'15245 (194215'15244) delete 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.2174595.0:209434 0.000000 2014-08-13 17:38:03.135869 7fe9bae887a0 20 read_log 194215'15246 (192834'13993) clean 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by osd.330.0:3124 2014-08-12 15:51:57.306825 2014-08-13 17:38:03.135881 7fe9bae887a0 20 read_log 194215'15247 (194215'15246) delete 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.2174595.0:209436 0.000000 2014-08-13 17:38:03.135892 7fe9bae887a0 20 read_log 194215'15248 (192888'14801) clean d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by osd.330.0:3126 2014-08-12 15:51:57.315275 2014-08-13 17:38:03.135904 7fe9bae887a0 20 read_log 194215'15249 (194215'15248) delete d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.2174595.0:209438 0.000000 2014-08-13 17:38:03.135924 7fe9bae887a0 20 read_log 194215'15250 (192888'14794) clean 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by osd.330.0:3128 2014-08-12 15:51:57.325646 2014-08-13 17:38:03.135937 7fe9bae887a0 20 read_log 194215'15251 (194215'15250) delete 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.2174595.0:209440 0.000000 2014-08-13 17:38:03.135948 7fe9bae887a0 20 read_log 194215'15252 (193305'15097) clean ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by osd.330.0:3130 2014-08-12 15:51:57.333537 2014-08-13 17:38:03.135962 7fe9bae887a0 20 read_log 194215'15253 (194215'15252) delete ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.2174595.0:209442 0.000000 2014-08-13 17:38:03.135973 7fe9bae887a0 20 read_log 194215'15254 (192834'14581) clean f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by osd.330.0:3132 2014-08-12 15:51:57.342202 2014-08-13 17:38:03.135985 7fe9bae887a0 20 read_log 194215'15255 (194215'15254) delete f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.2174595.0:209444 0.000000 2014-08-13 17:38:03.135996 7fe9bae887a0 20 read_log 194215'15256 (193305'15082) clean eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by osd.330.0:3134 2014-08-12 15:51:57.350514 2014-08-13 17:38:03.136008 7fe9bae887a0 20 read_log 194215'15257 (194215'15256) delete eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.2174595.0:209446 0.000000 2014-08-13 17:38:03.136019 7fe9bae887a0 20 read_log 194215'15258 (193305'15111) clean dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by osd.330.0:3136 2014-08-12 15:51:57.358340 2014-08-13 17:38:03.136030 7fe9bae887a0 20 read_log 194215'15259 (194215'15258) delete dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.2174595.0:209448 0.000000 2014-08-13 17:38:03.136041 7fe9bae887a0 20 read_log 194215'15260 (192834'14498) clean 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by osd.330.0:3138 2014-08-12 15:51:57.365159 2014-08-13 17:38:03.136054 7fe9bae887a0 20 read_log 194215'15261 (194215'15260) delete 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.2174595.0:209450 0.000000 2014-08-13 17:38:03.136065 7fe9bae887a0 20 read_log 194215'15262 (192888'14772) clean e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by osd.330.0:3140 2014-08-12 15:51:57.371557 2014-08-13 17:38:03.136077 7fe9bae887a0 20 read_log 194215'15263 (194215'15262) delete e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.2174595.0:209452 0.000000 2014-08-13 17:38:03.136088 7fe9bae887a0 20 read_log 194215'15264 (193305'15086) clean a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by osd.330.0:3142 2014-08-12 15:51:57.377277 2014-08-13 17:38:03.136100 7fe9bae887a0 20 read_log 194215'15265 (194215'15264) delete a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.2174595.0:209454 0.000000 2014-08-13 17:38:03.136110 7fe9bae887a0 20 read_log 194215'15266 (192888'14770) clean 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by osd.330.0:3144 2014-08-12 15:51:57.384205 2014-08-13 17:38:03.136123 7fe9bae887a0 20 read_log 194215'15267 (194215'15266) delete 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.2174595.0:209456 0.000000 2014-08-13 17:38:03.136134 7fe9bae887a0 20 read_log 194215'15268 (192888'14962) clean e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by osd.330.0:3146 2014-08-12 15:51:57.391008 2014-08-13 17:38:03.136146 7fe9bae887a0 20 read_log 194215'15269 (194215'15268) delete e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.2174595.0:209458 0.000000 2014-08-13 17:38:03.136169 7fe9bae887a0 20 read_log 194215'15270 (192834'14605) clean 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by osd.330.0:3148 2014-08-12 15:51:57.397179 2014-08-13 17:38:03.136183 7fe9bae887a0 20 read_log 194215'15271 (194215'15270) delete 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.2174595.0:209460 0.000000 2014-08-13 17:38:03.136194 7fe9bae887a0 20 read_log 194215'15272 (192888'14960) clean 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by osd.330.0:3150 2014-08-12 15:51:57.403346 2014-08-13 17:38:03.136206 7fe9bae887a0 20 read_log 194215'15273 (194215'15272) delete 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.2174595.0:209462 0.000000 2014-08-13 17:38:03.136218 7fe9bae887a0 20 read_log 194215'15274 (192888'14979) clean 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by osd.330.0:3152 2014-08-12 15:51:57.409477 2014-08-13 17:38:03.136231 7fe9bae887a0 20 read_log 194215'15275 (194215'15274) delete 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.2174595.0:209464 0.000000 2014-08-13 17:38:03.136242 7fe9bae887a0 20 read_log 194215'15276 (192834'14123) clean b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by osd.330.0:3154 2014-08-12 15:51:57.416222 2014-08-13 17:38:03.136254 7fe9bae887a0 20 read_log 194215'15277 (194215'15276) delete b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.2174595.0:209466 0.000000 2014-08-13 17:38:03.136264 7fe9bae887a0 20 read_log 194215'15278 (192834'14606) clean 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by osd.330.0:3156 2014-08-12 15:51:57.422245 2014-08-13 17:38:03.136276 7fe9bae887a0 20 read_log 194215'15279 (194215'15278) delete 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.2174595.0:209468 0.000000 2014-08-13 17:38:03.136287 7fe9bae887a0 20 read_log 194215'15280 (193305'15081) clean f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by osd.330.0:3158 2014-08-12 15:51:57.428740 2014-08-13 17:38:03.136299 7fe9bae887a0 20 read_log 194215'15281 (194215'15280) delete f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.2174595.0:209470 0.000000 2014-08-13 17:38:03.136311 7fe9bae887a0 20 read_log 194215'15282 (193305'15102) clean cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by osd.330.0:3160 2014-08-12 15:51:57.434806 2014-08-13 17:38:03.136323 7fe9bae887a0 20 read_log 194215'15283 (194215'15282) delete cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.2174595.0:209472 0.000000 2014-08-13 17:38:03.136333 7fe9bae887a0 20 read_log 194215'15284 (192888'14972) clean bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by osd.330.0:3162 2014-08-12 15:51:57.441555 2014-08-13 17:38:03.136345 7fe9bae887a0 20 read_log 194215'15285 (194215'15284) delete bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.2174595.0:209474 0.000000 2014-08-13 17:38:03.136374 7fe9bae887a0 20 read_log 194215'15286 (192834'14077) clean 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by osd.330.0:3164 2014-08-12 15:51:57.447930 2014-08-13 17:38:03.136387 7fe9bae887a0 20 read_log 194215'15287 (194215'15286) delete 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.2174595.0:209476 0.000000 2014-08-13 17:38:03.136398 7fe9bae887a0 20 read_log 194215'15288 (193305'15078) clean d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by osd.330.0:3166 2014-08-12 15:51:57.454510 2014-08-13 17:38:03.136410 7fe9bae887a0 20 read_log 194215'15289 (194215'15288) delete d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.2174595.0:209478 0.000000 2014-08-13 17:38:03.136428 7fe9bae887a0 20 read_log 194215'15290 (192888'14974) clean 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by osd.330.0:3168 2014-08-12 15:51:57.460955 2014-08-13 17:38:03.136440 7fe9bae887a0 20 read_log 194215'15291 (194215'15290) delete 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.2174595.0:209480 0.000000 2014-08-13 17:38:03.136451 7fe9bae887a0 20 read_log 194215'15292 (192888'14779) clean 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by osd.330.0:3170 2014-08-12 15:51:57.467655 2014-08-13 17:38:03.136463 7fe9bae887a0 20 read_log 194215'15293 (194215'15292) delete 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.2174595.0:209482 0.000000 2014-08-13 17:38:03.136474 7fe9bae887a0 20 read_log 194215'15294 (192834'13963) clean c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by osd.330.0:3172 2014-08-12 15:51:57.473987 2014-08-13 17:38:03.136486 7fe9bae887a0 20 read_log 194215'15295 (194215'15294) delete c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.2174595.0:209484 0.000000 2014-08-13 17:38:03.136497 7fe9bae887a0 20 read_log 194215'15296 (192888'14773) clean 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by osd.330.0:3174 2014-08-12 15:51:57.481246 2014-08-13 17:38:03.136511 7fe9bae887a0 20 read_log 194215'15297 (194215'15296) delete 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.2174595.0:209486 0.000000 2014-08-13 17:38:03.136521 7fe9bae887a0 20 read_log 194215'15298 (192888'14973) clean 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by osd.330.0:3176 2014-08-12 15:51:57.488747 2014-08-13 17:38:03.136534 7fe9bae887a0 20 read_log 194215'15299 (194215'15298) delete 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.2174595.0:209488 0.000000 2014-08-13 17:38:03.136545 7fe9bae887a0 20 read_log 194215'15300 (193305'15110) clean fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by osd.330.0:3178 2014-08-12 15:51:57.497669 2014-08-13 17:38:03.136557 7fe9bae887a0 20 read_log 194215'15301 (194215'15300) delete fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.2174595.0:209490 0.000000 2014-08-13 17:38:03.136567 7fe9bae887a0 20 read_log 194215'15302 (192834'14535) clean df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by osd.330.0:3180 2014-08-12 15:51:57.507016 2014-08-13 17:38:03.136580 7fe9bae887a0 20 read_log 194215'15303 (194215'15302) delete df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.2174595.0:209492 0.000000 2014-08-13 17:38:03.136590 7fe9bae887a0 20 read_log 194215'15304 (192888'14789) clean 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by osd.330.0:3182 2014-08-12 15:51:57.514363 2014-08-13 17:38:03.136603 7fe9bae887a0 20 read_log 194215'15305 (194215'15304) delete 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.2174595.0:209494 0.000000 2014-08-13 17:38:03.136614 7fe9bae887a0 20 read_log 194215'15306 (193305'15100) clean e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by osd.330.0:3184 2014-08-12 15:51:57.522708 2014-08-13 17:38:03.136626 7fe9bae887a0 20 read_log 194215'15307 (194215'15306) delete e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.2174595.0:209496 0.000000 2014-08-13 17:38:03.136637 7fe9bae887a0 20 read_log 194215'15308 (193305'15101) clean e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by osd.330.0:3186 2014-08-12 15:51:57.530423 2014-08-13 17:38:03.136649 7fe9bae887a0 20 read_log 194215'15309 (194215'15308) delete e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.2174595.0:209498 0.000000 2014-08-13 17:38:03.136669 7fe9bae887a0 20 read_log 194215'15310 (193305'15091) clean e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by osd.330.0:3188 2014-08-12 15:51:57.537765 2014-08-13 17:38:03.136681 7fe9bae887a0 20 read_log 194215'15311 (194215'15310) delete e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.2174595.0:209500 0.000000 2014-08-13 17:38:03.136692 7fe9bae887a0 20 read_log 194215'15312 (192834'14568) clean 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by osd.330.0:3190 2014-08-12 15:51:57.546360 2014-08-13 17:38:03.136704 7fe9bae887a0 20 read_log 194215'15313 (194215'15312) delete 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.2174595.0:209502 0.000000 2014-08-13 17:38:03.136715 7fe9bae887a0 20 read_log 194215'15314 (193305'15105) clean 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by osd.330.0:3192 2014-08-12 15:51:57.554766 2014-08-13 17:38:03.136727 7fe9bae887a0 20 read_log 194215'15315 (194215'15314) delete 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.2174595.0:209504 0.000000 2014-08-13 17:38:03.136737 7fe9bae887a0 20 read_log 194215'15316 (192888'14782) clean ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by osd.330.0:3194 2014-08-12 15:51:57.562109 2014-08-13 17:38:03.136750 7fe9bae887a0 20 read_log 194215'15317 (194215'15316) delete ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.2174595.0:209506 0.000000 2014-08-13 17:38:03.136761 7fe9bae887a0 20 read_log 194215'15318 (192888'14943) clean 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by osd.330.0:3196 2014-08-12 15:51:57.569806 2014-08-13 17:38:03.136776 7fe9bae887a0 20 read_log 194215'15319 (194215'15318) delete 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.2174595.0:209508 0.000000 2014-08-13 17:38:03.136787 7fe9bae887a0 20 read_log 194215'15320 (192834'14573) clean b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by osd.330.0:3198 2014-08-12 15:51:57.581068 2014-08-13 17:38:03.136799 7fe9bae887a0 20 read_log 194215'15321 (194215'15320) delete b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.2174595.0:209510 0.000000 2014-08-13 17:38:03.136810 7fe9bae887a0 20 read_log 194215'15322 (193305'15106) clean 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by osd.330.0:3200 2014-08-12 15:51:57.589873 2014-08-13 17:38:03.136822 7fe9bae887a0 20 read_log 194215'15323 (194215'15322) delete 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.2174595.0:209512 0.000000 2014-08-13 17:38:03.136833 7fe9bae887a0 20 read_log 194215'15324 (192888'14796) clean 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by osd.330.0:3202 2014-08-12 15:51:57.598898 2014-08-13 17:38:03.136845 7fe9bae887a0 20 read_log 194215'15325 (194215'15324) delete 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.2174595.0:209514 0.000000 2014-08-13 17:38:03.136855 7fe9bae887a0 20 read_log 194215'15326 (192834'14097) clean ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by osd.330.0:3204 2014-08-12 15:51:57.607086 2014-08-13 17:38:03.136867 7fe9bae887a0 20 read_log 194215'15327 (194215'15326) delete ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.2174595.0:209516 0.000000 2014-08-13 17:38:03.136878 7fe9bae887a0 20 read_log 194215'15328 (192834'14488) clean 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by osd.330.0:3206 2014-08-12 15:51:57.616334 2014-08-13 17:38:03.136890 7fe9bae887a0 20 read_log 194215'15329 (194215'15328) delete 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.2174595.0:209518 0.000000 2014-08-13 17:38:03.136907 7fe9bae887a0 20 read_log 194215'15330 (192888'14804) clean 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by osd.330.0:3208 2014-08-12 15:51:57.625101 2014-08-13 17:38:03.136920 7fe9bae887a0 20 read_log 194215'15331 (194215'15330) delete 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.2174595.0:209520 0.000000 2014-08-13 17:38:03.136931 7fe9bae887a0 20 read_log 194215'15332 (192888'14813) clean 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by osd.330.0:3210 2014-08-12 15:51:57.635725 2014-08-13 17:38:03.136943 7fe9bae887a0 20 read_log 194215'15333 (194215'15332) delete 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.2174595.0:209522 0.000000 2014-08-13 17:38:03.136954 7fe9bae887a0 20 read_log 194215'15334 (192834'14648) clean 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by osd.330.0:3212 2014-08-12 15:51:57.645636 2014-08-13 17:38:03.136966 7fe9bae887a0 20 read_log 194215'15335 (194215'15334) delete 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.2174595.0:209524 0.000000 2014-08-13 17:38:03.136976 7fe9bae887a0 20 read_log 194215'15336 (192888'14809) clean cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by osd.330.0:3214 2014-08-12 15:51:57.654479 2014-08-13 17:38:03.136988 7fe9bae887a0 20 read_log 194215'15337 (194215'15336) delete cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.2174595.0:209526 0.000000 2014-08-13 17:38:03.136999 7fe9bae887a0 20 read_log 194215'15338 (192834'14096) clean b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by osd.330.0:3216 2014-08-12 15:51:57.662568 2014-08-13 17:38:03.137011 7fe9bae887a0 20 read_log 194215'15339 (194215'15338) delete b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.2174595.0:209528 0.000000 2014-08-13 17:38:03.137021 7fe9bae887a0 20 read_log 194215'15340 (192834'14072) clean 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by osd.330.0:3218 2014-08-12 15:51:57.672342 2014-08-13 17:38:03.137033 7fe9bae887a0 20 read_log 194215'15341 (194215'15340) delete 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.2174595.0:209530 0.000000 2014-08-13 17:38:03.137044 7fe9bae887a0 20 read_log 194215'15342 (193305'15113) clean 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by osd.330.0:3220 2014-08-12 15:51:57.680876 2014-08-13 17:38:03.137056 7fe9bae887a0 20 read_log 194215'15343 (194215'15342) delete 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.2174595.0:209532 0.000000 2014-08-13 17:38:03.137066 7fe9bae887a0 20 read_log 194215'15344 (192888'14950) clean fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by osd.330.0:3222 2014-08-12 15:51:57.689587 2014-08-13 17:38:03.137078 7fe9bae887a0 20 read_log 194215'15345 (194215'15344) delete fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.2174595.0:209534 0.000000 2014-08-13 17:38:03.137089 7fe9bae887a0 20 read_log 194215'15346 (192888'14766) clean b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by osd.330.0:3224 2014-08-12 15:51:57.698015 2014-08-13 17:38:03.137101 7fe9bae887a0 20 read_log 194215'15347 (194215'15346) delete b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.2174595.0:209536 0.000000 2014-08-13 17:38:03.137112 7fe9bae887a0 20 read_log 194215'15348 (192888'14959) clean 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by osd.330.0:3226 2014-08-12 15:51:57.708252 2014-08-13 17:38:03.137123 7fe9bae887a0 20 read_log 194215'15349 (194215'15348) delete 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.2174595.0:209538 0.000000 2014-08-13 17:38:03.137141 7fe9bae887a0 20 read_log 194215'15350 (193305'15074) clean 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by osd.330.0:3228 2014-08-12 15:51:57.716005 2014-08-13 17:38:03.137161 7fe9bae887a0 20 read_log 194215'15351 (194215'15350) delete 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.2174595.0:209540 0.000000 2014-08-13 17:38:03.137174 7fe9bae887a0 20 read_log 194215'15352 (192834'14008) clean 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by osd.330.0:3230 2014-08-12 15:51:57.724370 2014-08-13 17:38:03.137186 7fe9bae887a0 20 read_log 194215'15353 (194215'15352) delete 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.2174595.0:209542 0.000000 2014-08-13 17:38:03.137197 7fe9bae887a0 20 read_log 194215'15354 (192834'13982) clean 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by osd.330.0:3232 2014-08-12 15:51:57.734127 2014-08-13 17:38:03.137209 7fe9bae887a0 20 read_log 194215'15355 (194215'15354) delete 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.2174595.0:209544 0.000000 2014-08-13 17:38:03.137219 7fe9bae887a0 20 read_log 194215'15356 (193305'15112) clean 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by osd.330.0:3234 2014-08-12 15:51:57.742749 2014-08-13 17:38:03.137231 7fe9bae887a0 20 read_log 194215'15357 (194215'15356) delete 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.2174595.0:209546 0.000000 2014-08-13 17:38:03.137242 7fe9bae887a0 20 read_log 194215'15358 (192888'14947) clean 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by osd.330.0:3236 2014-08-12 15:51:57.751812 2014-08-13 17:38:03.137254 7fe9bae887a0 20 read_log 194215'15359 (194215'15358) delete 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.2174595.0:209548 0.000000 2014-08-13 17:38:03.137265 7fe9bae887a0 20 read_log 194215'15360 (193305'15090) clean 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by osd.330.0:3238 2014-08-12 15:51:57.761110 2014-08-13 17:38:03.137277 7fe9bae887a0 20 read_log 194215'15361 (194215'15360) delete 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.2174595.0:209550 0.000000 2014-08-13 17:38:03.137287 7fe9bae887a0 20 read_log 194215'15362 (192888'14938) clean e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by osd.330.0:3240 2014-08-12 15:51:57.771432 2014-08-13 17:38:03.137299 7fe9bae887a0 20 read_log 194215'15363 (194215'15362) delete e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.2174595.0:209552 0.000000 2014-08-13 17:38:03.137310 7fe9bae887a0 20 read_log 194215'15364 (193305'15099) clean 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by osd.330.0:3242 2014-08-12 15:51:57.780434 2014-08-13 17:38:03.137322 7fe9bae887a0 20 read_log 194215'15365 (194215'15364) delete 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.2174595.0:209554 0.000000 2014-08-13 17:38:03.137332 7fe9bae887a0 20 read_log 194215'15366 (192888'14797) clean 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by osd.330.0:3244 2014-08-12 15:51:57.788552 2014-08-13 17:38:03.137345 7fe9bae887a0 20 read_log 194215'15367 (194215'15366) delete 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.2174595.0:209556 0.000000 2014-08-13 17:38:03.137355 7fe9bae887a0 20 read_log 194215'15368 (192834'14595) clean cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by osd.330.0:3246 2014-08-12 15:51:57.796469 2014-08-13 17:38:03.137367 7fe9bae887a0 20 read_log 194215'15369 (194215'15368) delete cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.2174595.0:209558 0.000000 2014-08-13 17:38:03.137404 7fe9bae887a0 20 read_log 194215'15370 (192834'14125) clean 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by osd.330.0:3248 2014-08-12 15:51:57.805533 2014-08-13 17:38:03.137417 7fe9bae887a0 20 read_log 194215'15371 (194215'15370) delete 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.2174595.0:209560 0.000000 2014-08-13 17:38:03.137428 7fe9bae887a0 20 read_log 194215'15372 (192888'14764) clean d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by osd.330.0:3250 2014-08-12 15:51:57.813563 2014-08-13 17:38:03.137441 7fe9bae887a0 20 read_log 194215'15373 (194215'15372) delete d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.2174595.0:209562 0.000000 2014-08-13 17:38:03.137451 7fe9bae887a0 20 read_log 194215'15374 (193305'15104) clean 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by osd.330.0:3252 2014-08-12 15:51:57.821066 2014-08-13 17:38:03.137465 7fe9bae887a0 20 read_log 194215'15375 (194215'15374) delete 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.2174595.0:209564 0.000000 2014-08-13 17:38:03.137476 7fe9bae887a0 20 read_log 194215'15376 (192888'14798) clean efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by osd.330.0:3254 2014-08-12 15:51:57.829427 2014-08-13 17:38:03.137488 7fe9bae887a0 20 read_log 194215'15377 (194215'15376) delete efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.2174595.0:209566 0.000000 2014-08-13 17:38:03.137498 7fe9bae887a0 20 read_log 194215'15378 (192888'14783) clean da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by osd.330.0:3256 2014-08-12 15:51:57.839143 2014-08-13 17:38:03.137527 7fe9bae887a0 20 read_log 194215'15379 (194215'15378) delete da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.2174595.0:209568 0.000000 2014-08-13 17:38:03.137538 7fe9bae887a0 20 read_log 194215'15380 (192888'14799) clean 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by osd.330.0:3258 2014-08-12 15:51:57.848891 2014-08-13 17:38:03.137554 7fe9bae887a0 20 read_log 194215'15381 (194215'15380) delete 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.2174595.0:209570 0.000000 2014-08-13 17:38:03.137565 7fe9bae887a0 20 read_log 194215'15382 (193305'15083) clean 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by osd.330.0:3260 2014-08-12 15:51:57.856768 2014-08-13 17:38:03.137578 7fe9bae887a0 20 read_log 194215'15383 (194215'15382) delete 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.2174595.0:209572 0.000000 2014-08-13 17:38:03.137589 7fe9bae887a0 20 read_log 194215'15384 (192834'14540) clean 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by osd.330.0:3262 2014-08-12 15:51:57.864561 2014-08-13 17:38:03.137602 7fe9bae887a0 20 read_log 194215'15385 (194215'15384) delete 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.2174595.0:209574 0.000000 2014-08-13 17:38:03.137613 7fe9bae887a0 20 read_log 194215'15386 (192888'14961) clean f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by osd.330.0:3264 2014-08-12 15:51:57.872584 2014-08-13 17:38:03.137625 7fe9bae887a0 20 read_log 194215'15387 (194215'15386) delete f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.2174595.0:209576 0.000000 2014-08-13 17:38:03.137636 7fe9bae887a0 20 read_log 194215'15388 (192888'14774) clean e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by osd.330.0:3266 2014-08-12 15:51:57.881014 2014-08-13 17:38:03.137648 7fe9bae887a0 20 read_log 194215'15389 (194215'15388) delete e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.2174595.0:209578 0.000000 2014-08-13 17:38:03.137666 7fe9bae887a0 20 read_log 194215'15390 (192888'14785) clean 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by osd.330.0:3268 2014-08-12 15:51:57.889693 2014-08-13 17:38:03.137679 7fe9bae887a0 20 read_log 194215'15391 (194215'15390) delete 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.2174595.0:209580 0.000000 2014-08-13 17:38:03.137690 7fe9bae887a0 20 read_log 194215'15392 (192888'14806) clean 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by osd.330.0:3270 2014-08-12 15:51:57.899196 2014-08-13 17:38:03.137702 7fe9bae887a0 20 read_log 194215'15393 (194215'15392) delete 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.2174595.0:209582 0.000000 2014-08-13 17:38:03.137713 7fe9bae887a0 20 read_log 194215'15394 (192888'14805) clean 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by osd.330.0:3272 2014-08-12 15:51:57.907103 2014-08-13 17:38:03.137725 7fe9bae887a0 20 read_log 194215'15395 (194215'15394) delete 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.2174595.0:209584 0.000000 2014-08-13 17:38:03.137735 7fe9bae887a0 20 read_log 194215'15396 (192888'14800) clean be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by osd.330.0:3274 2014-08-12 15:51:57.917426 2014-08-13 17:38:03.137748 7fe9bae887a0 20 read_log 194215'15397 (194215'15396) delete be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.2174595.0:209586 0.000000 2014-08-13 17:38:03.137758 7fe9bae887a0 20 read_log 194215'15398 (192834'14623) clean 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by osd.330.0:3276 2014-08-12 15:51:57.925992 2014-08-13 17:38:03.137770 7fe9bae887a0 20 read_log 194215'15399 (194215'15398) delete 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.2174595.0:209588 0.000000 2014-08-13 17:38:03.137781 7fe9bae887a0 20 read_log 194215'15400 (194209'15134) delete daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.2174595.0:209590 0.000000 2014-08-13 17:38:03.137791 7fe9bae887a0 20 read_log 194215'15401 (194209'15135) delete 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.2174595.0:209592 0.000000 2014-08-13 17:38:03.137802 7fe9bae887a0 20 read_log 194215'15402 (194215'15402) modify 3f7/hit_set_24.3f7_archive_2014-08-12 15:51:57.156178_2014-08-12 17:24:41.391144/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.391356 2014-08-13 17:38:03.137815 7fe9bae887a0 20 read_log 194215'15403 (194215'15212) delete 3f7/hit_set_24.3f7_archive_2014-08-12 15:32:25.737625_2014-08-12 15:51:57.155796/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.391356 2014-08-13 17:38:03.137827 7fe9bae887a0 20 read_log 194215'15404 (0'0) promote 4e0adbf7/benchmark_data_storage0111-ib.pocstor_34628_object386/head//24 by osd.330.0:3998 2014-08-12 17:24:41.394531 2014-08-13 17:38:03.137839 7fe9bae887a0 20 read_log 194215'15405 (194215'15404) modify 4e0adbf7/benchmark_data_storage0111-ib.pocstor_34628_object386/head//24 by client.2193759.0:387 2014-08-12 17:24:41.386371 2014-08-13 17:38:03.137852 7fe9bae887a0 20 read_log 194215'15406 (0'0) promote 5e186bf7/benchmark_data_storage0111-ib.pocstor_34628_object787/head//24 by osd.330.0:4008 2014-08-12 17:24:42.980428 2014-08-13 17:38:03.137864 7fe9bae887a0 20 read_log 194215'15407 (194215'15406) modify 5e186bf7/benchmark_data_storage0111-ib.pocstor_34628_object787/head//24 by client.2193759.0:788 2014-08-12 17:24:42.969146 2014-08-13 17:38:03.137876 7fe9bae887a0 20 read_log 194370'15408 (0'0) promote 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by osd.330.0:4033 2014-08-12 20:30:14.940235 2014-08-13 17:38:03.137895 7fe9bae887a0 20 read_log 194370'15409 (194370'15408) modify 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by client.1508531.0:203 2014-08-12 20:30:14.929515 2014-08-13 17:38:03.137909 7fe9bae887a0 20 read_log 194370'15410 (0'0) promote 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by osd.330.0:4059 2014-08-12 20:30:21.882955 2014-08-13 17:38:03.137921 7fe9bae887a0 20 read_log 194370'15411 (194370'15410) modify 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by client.1508531.0:1995 2014-08-12 20:30:21.874345 2014-08-13 17:38:03.137934 7fe9bae887a0 20 read_log 194370'15412 (0'0) promote 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by osd.330.0:4069 2014-08-12 20:30:24.817667 2014-08-13 17:38:03.137946 7fe9bae887a0 20 read_log 194370'15413 (194370'15412) modify 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by client.1508531.0:2799 2014-08-12 20:30:24.806873 2014-08-13 17:38:03.137958 7fe9bae887a0 20 read_log 194370'15414 (0'0) promote 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by osd.330.0:4073 2014-08-12 20:30:25.008021 2014-08-13 17:38:03.137970 7fe9bae887a0 20 read_log 194370'15415 (194370'15414) modify 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by client.1508531.0:2852 2014-08-12 20:30:25.000236 2014-08-13 17:38:03.137982 7fe9bae887a0 20 read_log 194370'15416 (0'0) promote d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by osd.330.0:4097 2014-08-12 20:30:29.474109 2014-08-13 17:38:03.137994 7fe9bae887a0 20 read_log 194370'15417 (194370'15416) modify d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by client.1508531.0:4047 2014-08-12 20:30:29.463943 2014-08-13 17:38:03.138006 7fe9bae887a0 20 read_log 194370'15418 (0'0) promote f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by osd.330.0:4107 2014-08-12 20:30:32.487020 2014-08-13 17:38:03.138018 7fe9bae887a0 20 read_log 194370'15419 (194370'15418) modify f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by client.1508531.0:4826 2014-08-12 20:30:32.478351 2014-08-13 17:38:03.138048 7fe9bae887a0 20 read_log 194370'15420 (0'0) promote c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by osd.330.0:4117 2014-08-12 20:30:35.407349 2014-08-13 17:38:03.138060 7fe9bae887a0 20 read_log 194370'15421 (194370'15420) modify c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by client.1508531.0:5502 2014-08-12 20:30:35.399089 2014-08-13 17:38:03.138072 7fe9bae887a0 20 read_log 194370'15422 (0'0) promote 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by osd.330.0:4141 2014-08-12 20:30:40.861330 2014-08-13 17:38:03.138084 7fe9bae887a0 20 read_log 194370'15423 (194370'15422) modify 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by client.1508531.0:6694 2014-08-12 20:30:40.851596 2014-08-13 17:38:03.138096 7fe9bae887a0 20 read_log 194370'15424 (0'0) promote 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by osd.330.0:4143 2014-08-12 20:30:40.880586 2014-08-13 17:38:03.138108 7fe9bae887a0 20 read_log 194370'15425 (194370'15424) modify 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by client.1508531.0:6699 2014-08-12 20:30:40.872634 2014-08-13 17:38:03.138121 7fe9bae887a0 20 read_log 194370'15426 (0'0) promote 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by osd.330.0:4159 2014-08-12 20:30:44.680911 2014-08-13 17:38:03.138133 7fe9bae887a0 20 read_log 194370'15427 (194370'15426) modify 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by client.1508531.0:7463 2014-08-12 20:30:44.671825 2014-08-13 17:38:03.138149 7fe9bae887a0 20 read_log 194370'15428 (0'0) promote 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by osd.330.0:4163 2014-08-12 20:30:44.818938 2014-08-13 17:38:03.138179 7fe9bae887a0 20 read_log 194370'15429 (194370'15428) modify 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by client.1508531.0:7491 2014-08-12 20:30:44.809735 2014-08-13 17:38:03.138194 7fe9bae887a0 20 read_log 194370'15430 (0'0) promote de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by osd.330.0:4173 2014-08-12 20:30:45.899902 2014-08-13 17:38:03.138207 7fe9bae887a0 20 read_log 194370'15431 (194370'15430) modify de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by client.1508531.0:7688 2014-08-12 20:30:45.885740 2014-08-13 17:38:03.138222 7fe9bae887a0 20 read_log 194370'15432 (0'0) promote a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by osd.330.0:4175 2014-08-12 20:30:46.029510 2014-08-13 17:38:03.138236 7fe9bae887a0 20 read_log 194370'15433 (194370'15432) modify a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by client.1508531.0:7711 2014-08-12 20:30:46.018440 2014-08-13 17:38:03.138249 7fe9bae887a0 20 read_log 194370'15434 (0'0) promote ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by osd.330.0:4179 2014-08-12 20:30:50.377398 2014-08-13 17:38:03.138263 7fe9bae887a0 20 read_log 194370'15435 (194370'15434) modify ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by client.1508531.0:8518 2014-08-12 20:30:50.366822 2014-08-13 17:38:03.138275 7fe9bae887a0 20 read_log 194370'15436 (0'0) promote ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by osd.330.0:4181 2014-08-12 20:30:51.589360 2014-08-13 17:38:03.138288 7fe9bae887a0 20 read_log 194370'15437 (194370'15436) modify ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by client.1508531.0:8722 2014-08-12 20:30:51.577344 2014-08-13 17:38:03.138300 7fe9bae887a0 20 read_log 194370'15438 (0'0) promote 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by osd.330.0:4237 2014-08-12 20:31:15.627731 2014-08-13 17:38:03.138312 7fe9bae887a0 20 read_log 194370'15439 (194370'15438) modify 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by client.1508531.0:12589 2014-08-12 20:31:15.617565 2014-08-13 17:38:03.138324 7fe9bae887a0 20 read_log 194370'15440 (0'0) promote fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by osd.330.0:4245 2014-08-12 20:31:16.689293 2014-08-13 17:38:03.138336 7fe9bae887a0 20 read_log 194370'15441 (194370'15440) modify fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by client.1508531.0:12717 2014-08-12 20:31:16.677145 2014-08-13 17:38:03.138349 7fe9bae887a0 20 read_log 194377'15442 (0'0) promote d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by osd.330.0:4273 2014-08-12 20:32:57.945917 2014-08-13 17:38:03.138361 7fe9bae887a0 20 read_log 194377'15443 (194377'15442) modify d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by client.2231190.0:185 2014-08-12 20:32:57.934951 2014-08-13 17:38:03.138373 7fe9bae887a0 20 read_log 194377'15444 (0'0) promote 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by osd.330.0:4279 2014-08-12 20:33:00.510117 2014-08-13 17:38:03.138385 7fe9bae887a0 20 read_log 194377'15445 (194377'15444) modify 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by client.2231190.0:787 2014-08-12 20:33:00.502816 2014-08-13 17:38:03.138397 7fe9bae887a0 20 read_log 194377'15446 (0'0) promote 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by osd.330.0:4295 2014-08-12 20:33:04.940914 2014-08-13 17:38:03.138410 7fe9bae887a0 20 read_log 194377'15447 (194377'15446) modify 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by client.2231190.0:1938 2014-08-12 20:33:04.934317 2014-08-13 17:38:03.138422 7fe9bae887a0 20 read_log 194377'15448 (0'0) promote 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by osd.330.0:4299 2014-08-12 20:33:05.215181 2014-08-13 17:38:03.138441 7fe9bae887a0 20 read_log 194377'15449 (194377'15448) modify 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by client.2231190.0:1993 2014-08-12 20:33:05.205475 2014-08-13 17:38:03.138454 7fe9bae887a0 20 read_log 194377'15450 (0'0) promote a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by osd.330.0:4307 2014-08-12 20:33:07.204187 2014-08-13 17:38:03.138466 7fe9bae887a0 20 read_log 194377'15451 (194377'15450) modify a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by client.2231190.0:2495 2014-08-12 20:33:07.191237 2014-08-13 17:38:03.138479 7fe9bae887a0 20 read_log 194377'15452 (0'0) promote b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by osd.330.0:4327 2014-08-12 20:33:11.508975 2014-08-13 17:38:03.138491 7fe9bae887a0 20 read_log 194377'15453 (194377'15452) modify b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by client.2231190.0:3530 2014-08-12 20:33:11.499104 2014-08-13 17:38:03.138503 7fe9bae887a0 20 read_log 194377'15454 (0'0) promote afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by osd.330.0:4333 2014-08-12 20:33:14.031056 2014-08-13 17:38:03.138515 7fe9bae887a0 20 read_log 194377'15455 (194377'15454) modify afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by client.2231190.0:4019 2014-08-12 20:33:14.019939 2014-08-13 17:38:03.138527 7fe9bae887a0 20 read_log 194377'15456 (0'0) promote a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by osd.330.0:4339 2014-08-12 20:33:14.820058 2014-08-13 17:38:03.138539 7fe9bae887a0 20 read_log 194377'15457 (194377'15456) modify a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by client.2231190.0:4155 2014-08-12 20:33:14.812906 2014-08-13 17:38:03.138551 7fe9bae887a0 20 read_log 194377'15458 (0'0) promote a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by osd.330.0:4341 2014-08-12 20:33:14.838016 2014-08-13 17:38:03.138563 7fe9bae887a0 20 read_log 194377'15459 (194377'15458) modify a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by client.2231190.0:4158 2014-08-12 20:33:14.826432 2014-08-13 17:38:03.138575 7fe9bae887a0 20 read_log 194377'15460 (0'0) promote b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by osd.330.0:4349 2014-08-12 20:33:18.628081 2014-08-13 17:38:03.138587 7fe9bae887a0 20 read_log 194377'15461 (194377'15460) modify b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by client.2231190.0:4699 2014-08-12 20:33:18.617965 2014-08-13 17:38:03.138599 7fe9bae887a0 20 read_log 194377'15462 (0'0) promote ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by osd.330.0:4351 2014-08-12 20:33:19.538997 2014-08-13 17:38:03.138611 7fe9bae887a0 20 read_log 194377'15463 (194377'15462) modify ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by client.2231190.0:4808 2014-08-12 20:33:19.529097 2014-08-13 17:38:03.138623 7fe9bae887a0 20 read_log 194383'15464 (0'0) promote 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by osd.330.0:4377 2014-08-12 20:34:01.736789 2014-08-13 17:38:03.138635 7fe9bae887a0 20 read_log 194383'15465 (194383'15464) modify 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by client.2231190.0:6063 2014-08-12 20:34:01.727784 2014-08-13 17:38:03.138647 7fe9bae887a0 20 read_log 194383'15466 (0'0) promote 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by osd.330.0:4385 2014-08-12 20:34:03.010476 2014-08-13 17:38:03.138659 7fe9bae887a0 20 read_log 194383'15467 (194383'15466) modify 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by client.2231190.0:6297 2014-08-12 20:34:03.000017 2014-08-13 17:38:03.138672 7fe9bae887a0 20 read_log 194393'15468 (194393'15468) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:37:09.025600_2014-08-12 20:52:57.054304/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:57.055717 2014-08-13 17:38:03.138692 7fe9bae887a0 20 read_log 194393'15469 (194215'15402) delete 3f7/hit_set_24.3f7_archive_2014-08-12 15:51:57.156178_2014-08-12 17:24:41.391144/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:57.055717 2014-08-13 17:38:03.138705 7fe9bae887a0 20 read_log 194393'15470 (0'0) promote e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by osd.330.0:4447 2014-08-12 20:52:57.062199 2014-08-13 17:38:03.138717 7fe9bae887a0 20 read_log 194393'15471 (194393'15470) modify e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by client.2235069.0:164 2014-08-12 20:52:57.047153 2014-08-13 17:38:03.138748 7fe9bae887a0 20 read_log 194416'15472 (194416'15472) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:52:57.055774_2014-08-13 15:28:32.051663/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 17:38:03.138764 7fe9bae887a0 20 read_log 194416'15473 (194393'15468) delete 3f7/hit_set_24.3f7_archive_2014-08-12 20:37:09.025600_2014-08-12 20:52:57.054304/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 17:38:03.138776 7fe9bae887a0 20 read_log 194416'15474 (0'0) promote 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by osd.330.0:4667 2014-08-13 15:28:32.055676 2014-08-13 17:38:03.138788 7fe9bae887a0 20 read_log 194416'15475 (194416'15474) modify 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by client.2441958.0:426 2014-08-13 15:28:32.043849 2014-08-13 17:38:03.138801 7fe9bae887a0 20 read_log 194416'15476 (0'0) promote e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by osd.330.0:4688 2014-08-13 15:28:37.908509 2014-08-13 17:38:03.138813 7fe9bae887a0 20 read_log 194416'15477 (194416'15476) modify e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by client.2441958.0:1897 2014-08-13 15:28:37.900974 2014-08-13 17:38:03.138825 7fe9bae887a0 20 read_log 194416'15478 (0'0) promote 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by osd.330.0:4695 2014-08-13 15:28:38.677634 2014-08-13 17:38:03.138836 7fe9bae887a0 20 read_log 194416'15479 (194416'15478) modify 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by client.2441958.0:2104 2014-08-13 15:28:38.667281 2014-08-13 17:38:03.138849 7fe9bae887a0 20 read_log 194416'15480 (0'0) promote 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by osd.330.0:4711 2014-08-13 15:28:40.710198 2014-08-13 17:38:03.138860 7fe9bae887a0 20 read_log 194416'15481 (194416'15480) modify 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by client.2441958.0:2635 2014-08-13 15:28:40.701579 2014-08-13 17:38:03.138873 7fe9bae887a0 20 read_log 194416'15482 (0'0) promote facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by osd.330.0:4725 2014-08-13 15:28:42.995238 2014-08-13 17:38:03.138885 7fe9bae887a0 20 read_log 194416'15483 (194416'15482) modify facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by client.2441958.0:3243 2014-08-13 15:28:42.984910 2014-08-13 17:38:03.138897 7fe9bae887a0 20 read_log 194416'15484 (0'0) promote 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by osd.330.0:4741 2014-08-13 15:28:48.997905 2014-08-13 17:38:03.138909 7fe9bae887a0 20 read_log 194416'15485 (194416'15484) modify 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by client.2441958.0:4763 2014-08-13 15:28:48.989745 2014-08-13 17:38:03.138921 7fe9bae887a0 20 read_log 194416'15486 (0'0) promote 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by osd.330.0:4743 2014-08-13 15:28:49.017045 2014-08-13 17:38:03.138933 7fe9bae887a0 20 read_log 194416'15487 (194416'15486) modify 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by client.2441958.0:4768 2014-08-13 15:28:49.007748 2014-08-13 17:38:03.138952 7fe9bae887a0 20 read_log 194416'15488 (0'0) promote 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by osd.330.0:4761 2014-08-13 15:28:54.248645 2014-08-13 17:38:03.138965 7fe9bae887a0 20 read_log 194416'15489 (194416'15488) modify 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by client.2441958.0:6101 2014-08-13 15:28:54.235380 2014-08-13 17:38:03.138977 7fe9bae887a0 20 read_log 194416'15490 (0'0) promote 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by osd.330.0:4769 2014-08-13 15:28:57.994805 2014-08-13 17:38:03.138989 7fe9bae887a0 20 read_log 194416'15491 (194416'15490) modify 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by client.2441958.0:7058 2014-08-13 15:28:57.984296 2014-08-13 17:38:03.139021 7fe9bae887a0 20 read_log 194416'15492 (0'0) promote c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by osd.330.0:4787 2014-08-13 15:29:01.074347 2014-08-13 17:38:03.139037 7fe9bae887a0 20 read_log 194416'15493 (194416'15492) modify c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by client.2441958.0:7878 2014-08-13 15:29:01.065004 2014-08-13 17:38:03.139050 7fe9bae887a0 20 read_log 194416'15494 (0'0) promote f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by osd.330.0:4835 2014-08-13 15:29:19.424557 2014-08-13 17:38:03.139063 7fe9bae887a0 20 read_log 194416'15495 (194416'15494) modify f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by client.2441958.0:12429 2014-08-13 15:29:19.416060 2014-08-13 17:38:03.139075 7fe9bae887a0 20 read_log 194416'15496 (0'0) promote 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by osd.330.0:4841 2014-08-13 15:29:19.815860 2014-08-13 17:38:03.139087 7fe9bae887a0 20 read_log 194416'15497 (194416'15496) modify 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by client.2441958.0:12531 2014-08-13 15:29:19.807708 2014-08-13 17:38:03.139099 7fe9bae887a0 20 read_log 194416'15498 (0'0) promote ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by osd.330.0:4845 2014-08-13 15:29:20.313716 2014-08-13 17:38:03.139111 7fe9bae887a0 20 read_log 194416'15499 (194416'15498) modify ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by client.2441958.0:12639 2014-08-13 15:29:20.303506 2014-08-13 17:38:03.139123 7fe9bae887a0 20 read_log 194416'15500 (0'0) promote 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by osd.330.0:4857 2014-08-13 15:29:23.535708 2014-08-13 17:38:03.139135 7fe9bae887a0 20 read_log 194416'15501 (194416'15500) modify 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by client.2441958.0:13364 2014-08-13 15:29:23.524595 2014-08-13 17:38:03.139150 7fe9bae887a0 20 read_log 194416'15502 (0'0) promote eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by osd.330.0:4859 2014-08-13 15:29:23.583424 2014-08-13 17:38:03.139167 7fe9bae887a0 20 read_log 194416'15503 (194416'15502) modify eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by client.2441958.0:13376 2014-08-13 15:29:23.572377 2014-08-13 17:38:03.139181 7fe9bae887a0 20 read_log 194416'15504 (0'0) promote cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by osd.330.0:4917 2014-08-13 15:29:53.743202 2014-08-13 17:38:03.139193 7fe9bae887a0 20 read_log 194416'15505 (194416'15504) modify cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by client.2441958.0:17451 2014-08-13 15:29:53.732383 2014-08-13 17:38:03.139205 7fe9bae887a0 20 read_log 194416'15506 (0'0) promote e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by osd.330.0:4919 2014-08-13 15:29:56.941801 2014-08-13 17:38:03.139217 7fe9bae887a0 20 read_log 194416'15507 (194416'15506) modify e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by client.2441958.0:17679 2014-08-13 15:29:56.930965 2014-08-13 17:38:03.139253 7fe9bae887a0 20 read_log 194422'15508 (0'0) promote 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by osd.330.0:4937 2014-08-13 15:32:15.288210 2014-08-13 17:38:03.139270 7fe9bae887a0 20 read_log 194422'15509 (194422'15508) modify 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by client.1148203.0:86 2014-08-13 15:32:15.277660 2014-08-13 17:38:03.139282 7fe9bae887a0 20 read_log 194422'15510 (0'0) promote 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by osd.330.0:4941 2014-08-13 15:32:15.817647 2014-08-13 17:38:03.139294 7fe9bae887a0 20 read_log 194422'15511 (194422'15510) modify 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by client.1148203.0:181 2014-08-13 15:32:15.808131 2014-08-13 17:38:03.139307 7fe9bae887a0 20 read_log 194422'15512 (0'0) promote 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by osd.330.0:4955 2014-08-13 15:32:17.902294 2014-08-13 17:38:03.139319 7fe9bae887a0 20 read_log 194422'15513 (194422'15512) modify 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by client.1148203.0:632 2014-08-13 15:32:17.893667 2014-08-13 17:38:03.139331 7fe9bae887a0 20 read_log 194422'15514 (0'0) promote a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by osd.330.0:4957 2014-08-13 15:32:18.112116 2014-08-13 17:38:03.139343 7fe9bae887a0 20 read_log 194422'15515 (194422'15514) modify a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by client.1148203.0:688 2014-08-13 15:32:18.102195 2014-08-13 17:38:03.139356 7fe9bae887a0 20 read_log 194422'15516 (0'0) promote 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by osd.330.0:4973 2014-08-13 15:32:21.622013 2014-08-13 17:38:03.139368 7fe9bae887a0 20 read_log 194422'15517 (194422'15516) modify 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by client.1148203.0:1600 2014-08-13 15:32:21.614021 2014-08-13 17:38:03.139380 7fe9bae887a0 20 read_log 194422'15518 (0'0) promote 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by osd.330.0:4981 2014-08-13 15:32:22.830385 2014-08-13 17:38:03.139392 7fe9bae887a0 20 read_log 194422'15519 (194422'15518) modify 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by client.1148203.0:1897 2014-08-13 15:32:22.815971 2014-08-13 17:38:03.139404 7fe9bae887a0 20 read_log 194422'15520 (0'0) promote 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by osd.330.0:4993 2014-08-13 15:32:25.791724 2014-08-13 17:38:03.139416 7fe9bae887a0 20 read_log 194422'15521 (194422'15520) modify 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by client.1148203.0:2671 2014-08-13 15:32:25.781809 2014-08-13 17:38:03.139428 7fe9bae887a0 20 read_log 194422'15522 (0'0) promote 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by osd.330.0:5007 2014-08-13 15:32:28.016320 2014-08-13 17:38:03.139440 7fe9bae887a0 20 read_log 194422'15523 (194422'15522) modify 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by client.1148203.0:3242 2014-08-13 15:32:28.006489 2014-08-13 17:38:03.139452 7fe9bae887a0 20 read_log 194422'15524 (0'0) promote 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by osd.330.0:5045 2014-08-13 15:32:42.914970 2014-08-13 17:38:03.139464 7fe9bae887a0 20 read_log 194422'15525 (194422'15524) modify 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by client.1148203.0:6512 2014-08-13 15:32:42.904663 2014-08-13 17:38:03.139476 7fe9bae887a0 20 read_log 194422'15526 (0'0) promote 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by osd.330.0:5067 2014-08-13 15:32:51.560170 2014-08-13 17:38:03.139488 7fe9bae887a0 20 read_log 194422'15527 (194422'15526) modify 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by client.1148203.0:7977 2014-08-13 15:32:51.551272 2014-08-13 17:38:03.139502 7fe9bae887a0 20 read_log 194422'15528 (0'0) promote 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by osd.330.0:5079 2014-08-13 15:32:54.854619 2014-08-13 17:38:03.139514 7fe9bae887a0 20 read_log 194422'15529 (194422'15528) modify 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by client.1148203.0:8465 2014-08-13 15:32:54.843415 2014-08-13 17:38:03.139526 7fe9bae887a0 20 read_log 194422'15530 (0'0) promote 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by osd.330.0:5089 2014-08-13 15:32:57.984761 2014-08-13 17:38:03.139538 7fe9bae887a0 20 read_log 194422'15531 (194422'15530) modify 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by client.1148203.0:8838 2014-08-13 15:32:57.973713 2014-08-13 17:38:03.139550 7fe9bae887a0 20 read_log 194422'15532 (0'0) promote 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by osd.330.0:5107 2014-08-13 15:33:03.202134 2014-08-13 17:38:03.139562 7fe9bae887a0 20 read_log 194422'15533 (194422'15532) modify 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by client.1148203.0:9372 2014-08-13 15:33:03.192061 2014-08-13 17:38:03.139574 7fe9bae887a0 20 read_log 194422'15534 (0'0) promote e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by osd.330.0:5111 2014-08-13 15:33:04.009393 2014-08-13 17:38:03.139586 7fe9bae887a0 20 read_log 194422'15535 (194422'15534) modify e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by client.1148203.0:9455 2014-08-13 15:33:03.998920 2014-08-13 17:38:03.139598 7fe9bae887a0 20 read_log 194422'15536 (0'0) promote e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by osd.330.0:5131 2014-08-13 15:33:23.813065 2014-08-13 17:38:03.139610 7fe9bae887a0 20 read_log 194422'15537 (194422'15536) modify e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by client.1148203.0:10495 2014-08-13 15:33:23.805389 2014-08-13 17:38:03.139625 7fe9bae887a0 20 read_log 0 divergent_priors 2014-08-13 17:38:03.140080 7fe9bae887a0 10 read_log checking for missing items over interval (194370'15408,194422'15537] 2014-08-13 17:38:03.140129 7fe9bae887a0 15 read_log missing 194422'15537 (194422'15536) modify e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by client.1148203.0:10495 2014-08-13 15:33:23.805389 2014-08-13 17:38:03.140178 7fe9bae887a0 15 read_log missing 194422'15535 (194422'15534) modify e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by client.1148203.0:9455 2014-08-13 15:33:03.998920 2014-08-13 17:38:03.140216 7fe9bae887a0 15 read_log missing 194422'15533 (194422'15532) modify 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by client.1148203.0:9372 2014-08-13 15:33:03.192061 2014-08-13 17:38:03.140254 7fe9bae887a0 15 read_log missing 194422'15531 (194422'15530) modify 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by client.1148203.0:8838 2014-08-13 15:32:57.973713 2014-08-13 17:38:03.140290 7fe9bae887a0 15 read_log missing 194422'15529 (194422'15528) modify 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by client.1148203.0:8465 2014-08-13 15:32:54.843415 2014-08-13 17:38:03.140327 7fe9bae887a0 15 read_log missing 194422'15527 (194422'15526) modify 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by client.1148203.0:7977 2014-08-13 15:32:51.551272 2014-08-13 17:38:03.140364 7fe9bae887a0 15 read_log missing 194422'15525 (194422'15524) modify 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by client.1148203.0:6512 2014-08-13 15:32:42.904663 2014-08-13 17:38:03.140401 7fe9bae887a0 15 read_log missing 194422'15523 (194422'15522) modify 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by client.1148203.0:3242 2014-08-13 15:32:28.006489 2014-08-13 17:38:03.140437 7fe9bae887a0 15 read_log missing 194422'15521 (194422'15520) modify 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by client.1148203.0:2671 2014-08-13 15:32:25.781809 2014-08-13 17:38:03.140474 7fe9bae887a0 15 read_log missing 194422'15519 (194422'15518) modify 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by client.1148203.0:1897 2014-08-13 15:32:22.815971 2014-08-13 17:38:03.140510 7fe9bae887a0 15 read_log missing 194422'15517 (194422'15516) modify 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by client.1148203.0:1600 2014-08-13 15:32:21.614021 2014-08-13 17:38:03.140547 7fe9bae887a0 15 read_log missing 194422'15515 (194422'15514) modify a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by client.1148203.0:688 2014-08-13 15:32:18.102195 2014-08-13 17:38:03.140583 7fe9bae887a0 15 read_log missing 194422'15513 (194422'15512) modify 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by client.1148203.0:632 2014-08-13 15:32:17.893667 2014-08-13 17:38:03.140620 7fe9bae887a0 15 read_log missing 194422'15511 (194422'15510) modify 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by client.1148203.0:181 2014-08-13 15:32:15.808131 2014-08-13 17:38:03.140657 7fe9bae887a0 15 read_log missing 194422'15509 (194422'15508) modify 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by client.1148203.0:86 2014-08-13 15:32:15.277660 2014-08-13 17:38:03.140694 7fe9bae887a0 15 read_log missing 194416'15507 (194416'15506) modify e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by client.2441958.0:17679 2014-08-13 15:29:56.930965 2014-08-13 17:38:03.140731 7fe9bae887a0 15 read_log missing 194416'15505 (194416'15504) modify cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by client.2441958.0:17451 2014-08-13 15:29:53.732383 2014-08-13 17:38:03.140767 7fe9bae887a0 15 read_log missing 194416'15503 (194416'15502) modify eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by client.2441958.0:13376 2014-08-13 15:29:23.572377 2014-08-13 17:38:03.140804 7fe9bae887a0 15 read_log missing 194416'15501 (194416'15500) modify 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by client.2441958.0:13364 2014-08-13 15:29:23.524595 2014-08-13 17:38:03.140841 7fe9bae887a0 15 read_log missing 194416'15499 (194416'15498) modify ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by client.2441958.0:12639 2014-08-13 15:29:20.303506 2014-08-13 17:38:03.140877 7fe9bae887a0 15 read_log missing 194416'15497 (194416'15496) modify 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by client.2441958.0:12531 2014-08-13 15:29:19.807708 2014-08-13 17:38:03.140913 7fe9bae887a0 15 read_log missing 194416'15495 (194416'15494) modify f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by client.2441958.0:12429 2014-08-13 15:29:19.416060 2014-08-13 17:38:03.140950 7fe9bae887a0 15 read_log missing 194416'15493 (194416'15492) modify c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by client.2441958.0:7878 2014-08-13 15:29:01.065004 2014-08-13 17:38:03.140987 7fe9bae887a0 15 read_log missing 194416'15491 (194416'15490) modify 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by client.2441958.0:7058 2014-08-13 15:28:57.984296 2014-08-13 17:38:03.141024 7fe9bae887a0 15 read_log missing 194416'15489 (194416'15488) modify 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by client.2441958.0:6101 2014-08-13 15:28:54.235380 2014-08-13 17:38:03.141060 7fe9bae887a0 15 read_log missing 194416'15487 (194416'15486) modify 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by client.2441958.0:4768 2014-08-13 15:28:49.007748 2014-08-13 17:38:03.141097 7fe9bae887a0 15 read_log missing 194416'15485 (194416'15484) modify 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by client.2441958.0:4763 2014-08-13 15:28:48.989745 2014-08-13 17:38:03.141133 7fe9bae887a0 15 read_log missing 194416'15483 (194416'15482) modify facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by client.2441958.0:3243 2014-08-13 15:28:42.984910 2014-08-13 17:38:03.141175 7fe9bae887a0 15 read_log missing 194416'15481 (194416'15480) modify 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by client.2441958.0:2635 2014-08-13 15:28:40.701579 2014-08-13 17:38:03.141212 7fe9bae887a0 15 read_log missing 194416'15479 (194416'15478) modify 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by client.2441958.0:2104 2014-08-13 15:28:38.667281 2014-08-13 17:38:03.141248 7fe9bae887a0 15 read_log missing 194416'15477 (194416'15476) modify e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by client.2441958.0:1897 2014-08-13 15:28:37.900974 2014-08-13 17:38:03.141285 7fe9bae887a0 15 read_log missing 194416'15475 (194416'15474) modify 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by client.2441958.0:426 2014-08-13 15:28:32.043849 2014-08-13 17:38:03.141327 7fe9bae887a0 15 read_log missing 194416'15472 (194416'15472) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:52:57.055774_2014-08-13 15:28:32.051663/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 17:38:03.141363 7fe9bae887a0 15 read_log missing 194393'15471 (194393'15470) modify e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by client.2235069.0:164 2014-08-12 20:52:57.047153 2014-08-13 17:38:03.141403 7fe9bae887a0 15 read_log missing 194383'15467 (194383'15466) modify 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by client.2231190.0:6297 2014-08-12 20:34:03.000017 2014-08-13 17:38:03.141439 7fe9bae887a0 15 read_log missing 194383'15465 (194383'15464) modify 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by client.2231190.0:6063 2014-08-12 20:34:01.727784 2014-08-13 17:38:03.141475 7fe9bae887a0 15 read_log missing 194377'15463 (194377'15462) modify ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by client.2231190.0:4808 2014-08-12 20:33:19.529097 2014-08-13 17:38:03.141511 7fe9bae887a0 15 read_log missing 194377'15461 (194377'15460) modify b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by client.2231190.0:4699 2014-08-12 20:33:18.617965 2014-08-13 17:38:03.141548 7fe9bae887a0 15 read_log missing 194377'15459 (194377'15458) modify a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by client.2231190.0:4158 2014-08-12 20:33:14.826432 2014-08-13 17:38:03.141585 7fe9bae887a0 15 read_log missing 194377'15457 (194377'15456) modify a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by client.2231190.0:4155 2014-08-12 20:33:14.812906 2014-08-13 17:38:03.141621 7fe9bae887a0 15 read_log missing 194377'15455 (194377'15454) modify afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by client.2231190.0:4019 2014-08-12 20:33:14.019939 2014-08-13 17:38:03.141658 7fe9bae887a0 15 read_log missing 194377'15453 (194377'15452) modify b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by client.2231190.0:3530 2014-08-12 20:33:11.499104 2014-08-13 17:38:03.141694 7fe9bae887a0 15 read_log missing 194377'15451 (194377'15450) modify a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by client.2231190.0:2495 2014-08-12 20:33:07.191237 2014-08-13 17:38:03.141731 7fe9bae887a0 15 read_log missing 194377'15449 (194377'15448) modify 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by client.2231190.0:1993 2014-08-12 20:33:05.205475 2014-08-13 17:38:03.141767 7fe9bae887a0 15 read_log missing 194377'15447 (194377'15446) modify 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by client.2231190.0:1938 2014-08-12 20:33:04.934317 2014-08-13 17:38:03.141803 7fe9bae887a0 15 read_log missing 194377'15445 (194377'15444) modify 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by client.2231190.0:787 2014-08-12 20:33:00.502816 2014-08-13 17:38:03.141840 7fe9bae887a0 15 read_log missing 194377'15443 (194377'15442) modify d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by client.2231190.0:185 2014-08-12 20:32:57.934951 2014-08-13 17:38:03.141877 7fe9bae887a0 15 read_log missing 194370'15441 (194370'15440) modify fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by client.1508531.0:12717 2014-08-12 20:31:16.677145 2014-08-13 17:38:03.141913 7fe9bae887a0 15 read_log missing 194370'15439 (194370'15438) modify 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by client.1508531.0:12589 2014-08-12 20:31:15.617565 2014-08-13 17:38:03.141950 7fe9bae887a0 15 read_log missing 194370'15437 (194370'15436) modify ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by client.1508531.0:8722 2014-08-12 20:30:51.577344 2014-08-13 17:38:03.141987 7fe9bae887a0 15 read_log missing 194370'15435 (194370'15434) modify ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by client.1508531.0:8518 2014-08-12 20:30:50.366822 2014-08-13 17:38:03.142023 7fe9bae887a0 15 read_log missing 194370'15433 (194370'15432) modify a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by client.1508531.0:7711 2014-08-12 20:30:46.018440 2014-08-13 17:38:03.142060 7fe9bae887a0 15 read_log missing 194370'15431 (194370'15430) modify de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by client.1508531.0:7688 2014-08-12 20:30:45.885740 2014-08-13 17:38:03.142097 7fe9bae887a0 15 read_log missing 194370'15429 (194370'15428) modify 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by client.1508531.0:7491 2014-08-12 20:30:44.809735 2014-08-13 17:38:03.142135 7fe9bae887a0 15 read_log missing 194370'15427 (194370'15426) modify 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by client.1508531.0:7463 2014-08-12 20:30:44.671825 2014-08-13 17:38:03.142177 7fe9bae887a0 15 read_log missing 194370'15425 (194370'15424) modify 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by client.1508531.0:6699 2014-08-12 20:30:40.872634 2014-08-13 17:38:03.142215 7fe9bae887a0 15 read_log missing 194370'15423 (194370'15422) modify 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by client.1508531.0:6694 2014-08-12 20:30:40.851596 2014-08-13 17:38:03.142253 7fe9bae887a0 15 read_log missing 194370'15421 (194370'15420) modify c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by client.1508531.0:5502 2014-08-12 20:30:35.399089 2014-08-13 17:38:03.142290 7fe9bae887a0 15 read_log missing 194370'15419 (194370'15418) modify f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by client.1508531.0:4826 2014-08-12 20:30:32.478351 2014-08-13 17:38:03.142327 7fe9bae887a0 15 read_log missing 194370'15417 (194370'15416) modify d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by client.1508531.0:4047 2014-08-12 20:30:29.463943 2014-08-13 17:38:03.142364 7fe9bae887a0 15 read_log missing 194370'15415 (194370'15414) modify 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by client.1508531.0:2852 2014-08-12 20:30:25.000236 2014-08-13 17:38:03.142401 7fe9bae887a0 15 read_log missing 194370'15413 (194370'15412) modify 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by client.1508531.0:2799 2014-08-12 20:30:24.806873 2014-08-13 17:38:03.142438 7fe9bae887a0 15 read_log missing 194370'15411 (194370'15410) modify 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by client.1508531.0:1995 2014-08-12 20:30:21.874345 2014-08-13 17:38:03.142476 7fe9bae887a0 15 read_log missing 194370'15409 (194370'15408) modify 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by client.1508531.0:203 2014-08-12 20:30:14.929515 2014-08-13 17:38:03.142487 7fe9bae887a0 10 read_log done 2014-08-13 17:38:03.142515 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive m=64] handle_loaded 2014-08-13 17:38:03.142532 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Initial 0.014480 0 0.000000 2014-08-13 17:38:03.142545 7fe9bae887a0 5 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset 2014-08-13 17:38:03.142554 7fe9bae887a0 20 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194478 2014-08-13 17:38:03.142563 7fe9bae887a0 10 osd.4 194478 load_pgs loaded pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] log((192718'12872,194422'15537], crt=194422'15537) 2014-08-13 17:38:03.142573 7fe9bae887a0 0 osd.4 194478 load_pgs opened 74 pgs 2014-08-13 17:38:03.142579 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.30b(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142586 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.220(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142589 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.362(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142594 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.28f(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142597 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.14a(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142601 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.22c(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142604 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.619(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142607 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.30f(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142611 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.9b(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142614 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.581(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142617 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.f3(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142621 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.116(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142624 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3d(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142627 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4aa(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142630 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1d2(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142633 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1de(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142639 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[2.2a(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142643 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.113(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142646 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.142649 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.33a(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142653 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[1.2b(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142656 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.34(unlocked)] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.142659 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.99(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142663 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.84(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142666 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4df(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142670 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.39d(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142673 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2b9(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142676 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.142679 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3f7(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142682 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.775(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142685 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.630(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142688 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.e2(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142692 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.69f(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142695 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3e4(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142698 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2a1(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142702 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.639(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142705 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.707(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142708 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.5e(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142711 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.1f1(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142714 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2d1(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142718 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.b6(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142721 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3bf(unlocked)] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.142724 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1fb(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142727 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.350(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142730 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1f7(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142733 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.136(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142736 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4fe(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142739 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.653(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142742 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3b7(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142745 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3b9(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142748 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.791(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142751 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.6bd(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142755 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.181(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142758 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.20e(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142767 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1e4(unlocked)] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.142770 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1e6(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142773 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.12a(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142777 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3af(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142780 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.481(unlocked)] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.142783 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.48e(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142786 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3ad(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142789 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.206(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142792 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1bb(unlocked)] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.142795 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.299(unlocked)] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.142799 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3d5(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142802 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.537(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142805 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1b8(unlocked)] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.142809 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.29a(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142812 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.cd(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142815 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.226(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142818 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3d9(unlocked)] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.142821 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2ec(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.142824 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1a0(unlocked)] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.142827 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.369(unlocked)] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.142830 7fe9bae887a0 10 osd.4 194478 build_past_intervals_parallel nothing to build 2014-08-13 17:38:03.150536 7fe9bae887a0 2 osd.4 194478 superblock: i am osd.4 2014-08-13 17:38:03.150553 7fe9bae887a0 10 osd.4 194478 create_logger 2014-08-13 17:38:03.151350 7fe99c3e4700 20 osd.4 194478 update_osd_stat osd_stat(8089 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) 2014-08-13 17:38:03.151381 7fe99c3e4700 5 osd.4 194478 heartbeat: osd_stat(8089 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) 2014-08-13 17:38:03.151634 7fe9986dd700 10 osd.4 194478 agent_entry start 2014-08-13 17:38:03.151672 7fe9986dd700 20 osd.4 194478 agent_entry empty queue 2014-08-13 17:38:03.151927 7fe999be0700 10 osd.4 194478 OSD::ms_get_authorizer type=mon 2014-08-13 17:38:03.152074 7fe9a77f6700 10 osd.4 194478 ms_handle_connect on mon 2014-08-13 17:38:03.152111 7fe9a77f6700 10 osd.4 194478 send_alive up_thru currently 194477 want 0 2014-08-13 17:38:03.152117 7fe9a77f6700 20 osd.4 194478 send_pg_stats 2014-08-13 17:38:03.163655 7fe9a77f6700 10 osd.4 194478 do_waiters -- start 2014-08-13 17:38:03.163659 7fe9a77f6700 10 osd.4 194478 do_waiters -- finish 2014-08-13 17:38:03.163662 7fe9a77f6700 20 osd.4 194478 _dispatch 0x2729200 osd_map(194484..194484 src has 185139..194484) v3 2014-08-13 17:38:03.163667 7fe9a77f6700 0 osd.4 194478 ignoring osdmap until we have initialized 2014-08-13 17:38:03.163669 7fe9a77f6700 10 osd.4 194478 do_waiters -- start 2014-08-13 17:38:03.163670 7fe9a77f6700 10 osd.4 194478 do_waiters -- finish 2014-08-13 17:38:03.164545 7fe9a77f6700 10 osd.4 194478 do_waiters -- start 2014-08-13 17:38:03.164548 7fe9a77f6700 10 osd.4 194478 do_waiters -- finish 2014-08-13 17:38:03.164549 7fe9a77f6700 20 osd.4 194478 _dispatch 0x272ad00 osd_map(194484..194484 src has 185139..194484) v3 2014-08-13 17:38:03.164553 7fe9a77f6700 0 osd.4 194478 ignoring osdmap until we have initialized 2014-08-13 17:38:03.164554 7fe9a77f6700 10 osd.4 194478 do_waiters -- start 2014-08-13 17:38:03.164555 7fe9a77f6700 10 osd.4 194478 do_waiters -- finish 2014-08-13 17:38:03.164706 7fe9bae887a0 10 osd.4 194478 ensuring pgs have consumed prior maps 2014-08-13 17:38:03.164714 7fe9bae887a0 7 osd.4 194478 consume_map version 194478 2014-08-13 17:38:03.164759 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 17:38:03.164792 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164812 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.164835 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164843 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.164861 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.164892 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.164892 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.164905 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164923 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.164942 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.164955 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164966 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 17:38:03.164979 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164990 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165000 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.164999 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.165012 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] null 2014-08-13 17:38:03.165019 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.165025 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165038 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165049 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 17:38:03.165062 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165076 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165087 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:03.165098 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 17:38:03.165120 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165132 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165133 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 17:38:03.165147 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165157 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.165159 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165174 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.165178 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165187 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.165204 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.165205 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165218 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165218 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.165214 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 2014-08-13 17:38:03.165232 7fe9bae887a0 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.165245 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.165244 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 17:38:03.165250 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.165256 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165261 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.165265 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.165271 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165275 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.165281 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.165285 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 17:38:03.165288 7fe997bdb700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.165288 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map 2014-08-13 17:38:03.165296 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.509233 2 0.000098 2014-08-13 17:38:03.165299 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165311 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.165312 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165325 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:03.165328 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.165337 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.165338 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165350 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165349 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] update_heartbeat_peers unchanged 2014-08-13 17:38:03.165357 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000 2014-08-13 17:38:03.165362 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters 2014-08-13 17:38:03.165364 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 17:38:03.165372 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.165373 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.102260 2 0.000069 2014-08-13 17:38:03.165377 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165388 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started 2014-08-13 17:38:03.165390 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165398 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start 2014-08-13 17:38:03.165402 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 17:38:03.165407 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray 2014-08-13 17:38:03.165416 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165419 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.165429 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165431 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray 2014-08-13 17:38:03.165440 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 17:38:03.165452 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 17:38:03.165464 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 17:38:03.165476 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165486 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165497 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.165508 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165519 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165529 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165547 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] null 2014-08-13 17:38:03.165560 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.165571 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 17:38:03.165582 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:03.165593 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165604 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165615 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165626 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165637 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165647 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 17:38:03.165659 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 17:38:03.165671 7fe997ada700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.165670 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165682 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165692 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165703 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165713 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165724 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 17:38:03.165736 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165758 7fe9bae887a0 10 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.165774 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 17:38:03.165785 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.165797 7fe9bae887a0 10 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:03.165444 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.165886 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.165905 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.165389 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.165948 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.165968 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.165991 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:03.166009 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.166013 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.166028 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.166039 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.166049 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.166059 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340 2014-08-13 17:38:03.166068 7fe99f5e9700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.166070 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.166078 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.587770 2 0.000111 2014-08-13 17:38:03.166089 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.166096 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.166104 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.166113 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.166126 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.166136 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.166146 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.166158 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.166171 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340 down blocked_by {} 2014-08-13 17:38:03.166152 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 17:38:03.166189 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.166180 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.166193 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.166204 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:177950 2014-08-13 17:38:03.166212 7fe9979d9700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.166216 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.166206 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.166229 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.166240 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.166253 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.166233 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.166260 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.166309 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.166313 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.166322 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.166332 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.166332 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.540934 2 0.000117 2014-08-13 17:38:03.166347 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.166350 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.166364 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.166357 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.166374 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.166385 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000 2014-08-13 17:38:03.166400 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.166413 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.166429 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 17:38:03.166446 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.166449 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.166460 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.166470 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.166480 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.166425 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.166490 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,28,304 2014-08-13 17:38:03.166499 7fe99f5e9700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.166501 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.166493 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.166510 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.561141 2 0.000080 2014-08-13 17:38:03.166520 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.166528 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.166536 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.166545 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.166558 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.166568 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.166576 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.166585 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.166595 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,28,304 down blocked_by {} 2014-08-13 17:38:03.166598 7fe9977c7700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.166604 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.166614 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2898 2014-08-13 17:38:03.166623 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.28 2014-08-13 17:38:03.166635 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.304 2014-08-13 17:38:03.166644 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.166654 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.166664 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.166720 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 17:38:03.166735 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.166738 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.166748 7fe9974c4700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.166746 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.166756 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.166765 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.166776 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,356,357 2014-08-13 17:38:03.166785 7fe99f5e9700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.166794 7fe9975c5700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.166786 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.166829 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.465055 2 0.000058 2014-08-13 17:38:03.166842 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.166854 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.166862 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.166873 7fe9973c3700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.166873 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.166884 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.166893 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.166901 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.166909 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.166919 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [356,357](356) acting [356,357](356) maybe_went_rw) 2014-08-13 17:38:03.166933 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 17:38:03.166943 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,356,357 down blocked_by {} 2014-08-13 17:38:03.166953 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.166964 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2654 2014-08-13 17:38:03.166973 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.356 2014-08-13 17:38:03.166984 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.357 2014-08-13 17:38:03.166993 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.167002 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.167014 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.167070 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.167087 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.167171 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 17:38:03.167193 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.167197 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.167206 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.167218 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.167208 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 17:38:03.167234 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.167229 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.167227 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] flushed 2014-08-13 17:38:03.167239 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.167251 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.167261 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.167263 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.167271 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.167274 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.167280 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.504761 2 0.000083 2014-08-13 17:38:03.167290 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,301,324 2014-08-13 17:38:03.167300 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.167294 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.167302 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.167304 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.167312 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.167312 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.514833 2 0.000088 2014-08-13 17:38:03.167323 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.167325 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.167336 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.167337 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.167346 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.167350 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.167348 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.167358 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.167360 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.167375 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.167371 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.167382 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.167390 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.167399 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.167412 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 17:38:03.167423 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 17:38:03.167433 7fe99f5e9700 20 osd.4 194478 get_map 194471 - loading and decoding 0x2746200 2014-08-13 17:38:03.167435 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,324 down blocked_by {} 2014-08-13 17:38:03.167444 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.167458 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2826 2014-08-13 17:38:03.167468 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.301 2014-08-13 17:38:03.167479 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 17:38:03.167490 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.167499 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.167510 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.167530 7fe9971c1700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.167620 7fe9970c0700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.167619 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.167642 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.167661 7fe9976c6700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.167811 7fe996cbc700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.167979 7fe99f5e9700 10 osd.4 194478 add_map_bl 194471 268447 bytes 2014-08-13 17:38:03.167983 7fe9969b9700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.168178 7fe9968b8700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.169126 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.170090 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.170478 7fe99f5e9700 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.170509 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.170515 7fe99f5e9700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.170537 7fe99f5e9700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.170548 7fe99f5e9700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.170541 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 17:38:03.170561 7fe99f5e9700 20 osd.4 194478 get_map 194472 - loading and decoding 0x2741c00 2014-08-13 17:38:03.170565 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.170570 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.170593 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.170604 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.170617 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.170654 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.170665 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.170675 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.501207 2 0.000100 2014-08-13 17:38:03.170688 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.170698 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.170708 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.170719 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.170756 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.170771 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.170781 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.170794 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.170883 7fe99f5e9700 10 osd.4 194478 add_map_bl 194472 268447 bytes 2014-08-13 17:38:03.171268 7fe9961b1700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.171334 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.173233 7fe99f5e9700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.173285 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.173248 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 2014-08-13 17:38:03.173303 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.173293 7fe99f5e9700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.173310 7fe99f5e9700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.173310 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 17:38:03.173341 7fe99f5e9700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.173343 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.173363 7fe99f5e9700 20 osd.4 194478 get_map 194473 - loading and decoding 0x2746000 2014-08-13 17:38:03.173361 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.173379 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_activate_map 2014-08-13 17:38:03.173397 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] update_heartbeat_peers -> 4,291,350,367,378 2014-08-13 17:38:03.173413 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.173416 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] take_waiters 2014-08-13 17:38:03.173431 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.368260 2 0.000128 2014-08-13 17:38:03.173450 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started 2014-08-13 17:38:03.173465 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Start 2014-08-13 17:38:03.173480 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary 2014-08-13 17:38:03.173497 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000031 0 0.000000 2014-08-13 17:38:03.173515 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 17:38:03.173531 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering 2014-08-13 17:38:03.173546 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.173575 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.173594 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194473-194475 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 17:38:03.173615 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194471-194472 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 17:38:03.173644 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,291,350,367,378 down blocked_by {} 2014-08-13 17:38:03.173660 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.173679 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] publish_stats_to_osd 194478:25283 2014-08-13 17:38:03.173697 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.291 2014-08-13 17:38:03.173715 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.350 2014-08-13 17:38:03.173732 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.367 2014-08-13 17:38:03.173754 7fe99f5e9700 10 osd.4 194478 add_map_bl 194473 268447 bytes 2014-08-13 17:38:03.173747 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.378 2014-08-13 17:38:03.173780 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.173798 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.173819 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.173936 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.173965 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.174406 7fe995faf700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.174411 7fe995eae700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.175998 7fe99f5e9700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.176039 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.176045 7fe99f5e9700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.176031 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 17:38:03.176079 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.176075 7fe99f5e9700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.176085 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.176093 7fe99f5e9700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.176100 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.176127 7fe99f5e9700 20 osd.4 194478 get_map 194474 - loading and decoding 0x2747400 2014-08-13 17:38:03.176128 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.176140 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.176152 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,285,312 2014-08-13 17:38:03.176162 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.176164 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.176175 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.564001 2 0.000117 2014-08-13 17:38:03.176195 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.176205 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.176215 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.176226 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.176238 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.176248 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.176260 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.176280 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.176290 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 17:38:03.176301 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 17:38:03.176311 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,312 down blocked_by {} 2014-08-13 17:38:03.176320 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.176331 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2856 2014-08-13 17:38:03.176340 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.285 2014-08-13 17:38:03.176350 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.312 2014-08-13 17:38:03.176359 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.176369 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.176380 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.176380 7fe99f5e9700 10 osd.4 194478 add_map_bl 194474 268950 bytes 2014-08-13 17:38:03.177238 7fe995bab700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.178565 7fe99f5e9700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.178605 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.178613 7fe99f5e9700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.178645 7fe99f5e9700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.178611 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 17:38:03.178667 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.178662 7fe99f5e9700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.178683 7fe99f5e9700 20 osd.4 194478 get_map 194475 - loading and decoding 0x2741a00 2014-08-13 17:38:03.178674 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.178701 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.178712 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.178723 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.178770 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.178784 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.178798 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.560497 2 0.000100 2014-08-13 17:38:03.178846 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.178856 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.178863 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.178874 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.178884 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.178896 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.178905 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.178916 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.179005 7fe99f5e9700 10 osd.4 194478 add_map_bl 194475 268950 bytes 2014-08-13 17:38:03.179511 7fe9959a9700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.179590 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.181672 7fe99f5e9700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.181713 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.181720 7fe99f5e9700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.181737 7fe99f5e9700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.181717 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 2014-08-13 17:38:03.181762 7fe99f5e9700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.181772 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.181775 7fe99f5e9700 20 osd.4 194478 get_map 194476 - loading and decoding 0x2741e00 2014-08-13 17:38:03.181777 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Reset advmap 2014-08-13 17:38:03.181788 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.181798 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.181810 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_activate_map 2014-08-13 17:38:03.181821 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] update_heartbeat_peers -> 4,20,25,303,362 2014-08-13 17:38:03.181831 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.181833 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] take_waiters 2014-08-13 17:38:03.181843 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Reset 0.404347 2 0.000065 2014-08-13 17:38:03.181854 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started 2014-08-13 17:38:03.181864 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Start 2014-08-13 17:38:03.181873 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: transitioning to Primary 2014-08-13 17:38:03.181885 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.181896 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary 2014-08-13 17:38:03.181906 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering 2014-08-13 17:38:03.181915 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.181925 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.181936 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194473-194475 up [25,362](25) acting [20,303](20) maybe_went_rw) 2014-08-13 17:38:03.181948 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194471-194472 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 17:38:03.181959 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362 down blocked_by {} 2014-08-13 17:38:03.181969 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.181981 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] publish_stats_to_osd 194478:29700 2014-08-13 17:38:03.181990 7fe99f5e9700 10 osd.4 194478 add_map_bl 194476 268950 bytes 2014-08-13 17:38:03.181992 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.20 2014-08-13 17:38:03.182003 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.25 2014-08-13 17:38:03.182013 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.303 2014-08-13 17:38:03.182022 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.362 2014-08-13 17:38:03.182032 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.182042 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.182055 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.182516 7fe9957a7700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.184355 7fe99f5e9700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.184394 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.184373 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 17:38:03.184408 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.184401 7fe99f5e9700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.184414 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.184419 7fe99f5e9700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.184435 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.184439 7fe99f5e9700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.184462 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.184472 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.184478 7fe99f5e9700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.184492 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.184495 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.184503 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.184504 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.184514 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.184512 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.184535 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.184541 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.484627 2 0.000088 2014-08-13 17:38:03.184550 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.184553 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.184552 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.184562 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.184563 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 17:38:03.184571 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.184573 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.184581 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.184583 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:03.184592 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.184603 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.184612 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.184622 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.184623 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 17:38:03.184633 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:03.184643 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.443510 9 0.000449 2014-08-13 17:38:03.184659 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 17:38:03.184669 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 17:38:03.184678 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 17:38:03.184688 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.184700 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 17:38:03.184711 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194478 2014-08-13 17:38:03.184721 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.184718 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 17:38:03.184732 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.184735 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.184739 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.184748 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.184757 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.184770 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.184780 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,61,349 2014-08-13 17:38:03.184789 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.184791 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.184799 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.550120 2 0.000085 2014-08-13 17:38:03.184800 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.184819 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.184811 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.184853 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.184867 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.184877 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 2014-08-13 17:38:03.184887 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.184896 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.184905 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.184913 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.184913 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.184931 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.184924 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 17:38:03.184935 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 17:38:03.184945 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,61,349 down blocked_by {} 2014-08-13 17:38:03.184956 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.184972 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:3016 2014-08-13 17:38:03.184982 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 17:38:03.184992 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 17:38:03.185002 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.185011 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.185007 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 17:38:03.185024 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.185025 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.185028 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.185037 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.185046 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.185056 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.185088 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.185098 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.185108 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.473836 2 0.000058 2014-08-13 17:38:03.185118 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.185115 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 2014-08-13 17:38:03.185127 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.185136 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.185135 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.185140 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.185146 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.185151 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.185157 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.185162 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.185152 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.185170 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.185181 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.185173 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map 2014-08-13 17:38:03.185191 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.185194 7fe9955a5700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.185223 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] update_heartbeat_peers unchanged 2014-08-13 17:38:03.185235 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] take_waiters 2014-08-13 17:38:03.185239 7fe9954a4700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.185246 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Reset 0.273229 2 0.000066 2014-08-13 17:38:03.185257 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started 2014-08-13 17:38:03.185267 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Start 2014-08-13 17:38:03.185274 7fe9953a3700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.185266 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 2014-08-13 17:38:03.185276 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: transitioning to Stray 2014-08-13 17:38:03.185288 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.185289 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Start 0.000022 0 0.000000 2014-08-13 17:38:03.185292 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.185300 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started/Stray 2014-08-13 17:38:03.185302 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.185312 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.185312 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.185323 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.185324 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map 2014-08-13 17:38:03.185334 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.185367 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] update_heartbeat_peers unchanged 2014-08-13 17:38:03.185378 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] take_waiters 2014-08-13 17:38:03.185388 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Reset 0.330796 2 0.000071 2014-08-13 17:38:03.185399 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started 2014-08-13 17:38:03.185409 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Start 2014-08-13 17:38:03.185406 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:03.185424 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.185418 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: transitioning to Stray 2014-08-13 17:38:03.185427 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.185430 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Start 0.000022 0 0.000000 2014-08-13 17:38:03.185439 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.185441 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started/Stray 2014-08-13 17:38:03.185450 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.185453 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.185460 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.185464 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.185470 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340 2014-08-13 17:38:03.185477 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.185479 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.185477 7fe9952a2700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.185481 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.185489 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.583362 2 0.000090 2014-08-13 17:38:03.185498 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.185506 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.185514 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.185512 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 17:38:03.185524 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.185534 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.185541 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.185549 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.185560 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.185570 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340 down blocked_by {} 2014-08-13 17:38:03.185578 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.185588 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:176846 2014-08-13 17:38:03.185597 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.185607 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.185615 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.185624 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.185825 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 17:38:03.185845 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.185848 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.185857 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.185849 7fe9951a1700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.185866 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.185877 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.185890 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,45,339 2014-08-13 17:38:03.185899 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.185901 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.185909 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.570538 2 0.000081 2014-08-13 17:38:03.185919 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.185927 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.185935 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.185945 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.185955 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.185964 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.186019 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 17:38:03.186035 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186035 7fe9950a0700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.186038 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.186034 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.186047 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.186053 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.186057 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186066 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.186066 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [339,45](339) acting [339,45](339) maybe_went_rw) 2014-08-13 17:38:03.186079 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 17:38:03.186091 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,339 down blocked_by {} 2014-08-13 17:38:03.186100 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.186100 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.186110 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.186113 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2929 2014-08-13 17:38:03.186119 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.469071 2 0.000058 2014-08-13 17:38:03.186125 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 17:38:03.186131 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.186136 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 17:38:03.186140 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.186148 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.186148 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.186159 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.186158 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.186173 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.186172 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.186191 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.186200 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.186210 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.186272 7fe994e9e700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.186320 7fe99ebe8700 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186342 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186346 7fe99ebe8700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186356 7fe99ebe8700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186366 7fe99ebe8700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186388 7fe99ebe8700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186401 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186403 7fe99ebe8700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186412 7fe99ebe8700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186420 7fe99ebe8700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186440 7fe99ebe8700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186453 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186455 7fe99ebe8700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186463 7fe99ebe8700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186472 7fe99ebe8700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186494 7fe99ebe8700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186507 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186509 7fe99ebe8700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186518 7fe99ebe8700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186526 7fe99ebe8700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186547 7fe99ebe8700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186559 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186547 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.186562 7fe99ebe8700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186570 7fe99ebe8700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186579 7fe99ebe8700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186599 7fe99ebe8700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186612 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186614 7fe99ebe8700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186623 7fe99ebe8700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186632 7fe99ebe8700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186632 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] flushed 2014-08-13 17:38:03.186654 7fe99ebe8700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186666 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186668 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186677 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186685 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186707 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.186719 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.186721 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.186730 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 17:38:03.186738 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.186747 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:03.186803 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 17:38:03.186814 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:03.186823 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.083958 9 0.000246 2014-08-13 17:38:03.186834 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started 2014-08-13 17:38:03.186843 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Start 2014-08-13 17:38:03.186852 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray 2014-08-13 17:38:03.186863 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.186873 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray 2014-08-13 17:38:03.186884 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194478 2014-08-13 17:38:03.186893 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.186904 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.186982 7fe994f9f700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.186987 7fe994a9a700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.187094 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187089 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] flushed 2014-08-13 17:38:03.187116 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187184 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187209 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187264 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187281 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187183 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 2014-08-13 17:38:03.187332 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.187337 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.187347 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.187344 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 17:38:03.187362 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.187358 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.187365 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.187368 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:03.187375 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.187386 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.187395 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.187409 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 17:38:03.187419 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:03.187428 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.173982 2 0.000062 2014-08-13 17:38:03.187429 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.187439 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.187440 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started 2014-08-13 17:38:03.187448 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.483657 2 0.000059 2014-08-13 17:38:03.187449 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start 2014-08-13 17:38:03.187459 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.187459 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray 2014-08-13 17:38:03.187469 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.187470 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.187480 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.187485 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray 2014-08-13 17:38:03.187492 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 2014-08-13 17:38:03.187497 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.187502 7fe994494700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.187503 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.187508 7fe994797700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.187508 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187519 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187516 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.187525 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187534 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187606 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 17:38:03.187627 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.187630 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.187639 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.187642 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187651 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.187660 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187662 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.187695 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.187706 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.187715 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.474583 2 0.000061 2014-08-13 17:38:03.187715 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187725 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.187733 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187735 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.187743 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.187752 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.187765 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.187776 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.187785 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187794 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187788 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.187792 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 17:38:03.187809 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.187812 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.187823 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.187832 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.187842 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.187850 7fe994292700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.187850 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187867 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187876 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.187886 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.187895 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.501538 2 0.000082 2014-08-13 17:38:03.187895 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 2014-08-13 17:38:03.187911 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.187907 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.187916 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.187915 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.187924 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.187925 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.187936 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.187935 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.187947 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.187947 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map 2014-08-13 17:38:03.187961 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.187970 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.187979 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.187996 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] update_heartbeat_peers unchanged 2014-08-13 17:38:03.188009 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters 2014-08-13 17:38:03.188018 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.395503 2 0.000063 2014-08-13 17:38:03.188030 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started 2014-08-13 17:38:03.188040 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start 2014-08-13 17:38:03.188049 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray 2014-08-13 17:38:03.188045 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 17:38:03.188062 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.188059 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.188065 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.188071 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray 2014-08-13 17:38:03.188075 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.188082 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.188084 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.188092 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188094 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.188102 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188126 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.188135 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.188144 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.463848 2 0.000059 2014-08-13 17:38:03.188155 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.188149 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188166 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188164 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.188171 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.188181 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.188191 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.188207 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.188216 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188228 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188265 7fe993c8c700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188273 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] flushed 2014-08-13 17:38:03.188359 7fe993a8a700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188455 7fe993d8d700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188448 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] flushed 2014-08-13 17:38:03.188494 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 17:38:03.188531 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.188536 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.188546 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.188558 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.188569 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.188579 7fe993b8b700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188579 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,50,289 2014-08-13 17:38:03.188588 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.188590 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.188598 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.469685 2 0.000061 2014-08-13 17:38:03.188608 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.188616 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.188624 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.188616 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 17:38:03.188640 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.188635 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.188629 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.188646 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.188644 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.188655 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.188659 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.188664 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.188672 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.188676 7fe993888700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188675 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.188683 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.188686 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,50,289 down blocked_by {} 2014-08-13 17:38:03.188696 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.188708 7fe994191700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188706 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2699 2014-08-13 17:38:03.188716 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.50 2014-08-13 17:38:03.188725 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 17:38:03.188735 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.188743 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188737 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.188754 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188775 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188789 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188756 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.188809 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.522368 2 0.000092 2014-08-13 17:38:03.188828 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.188840 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188857 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188846 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.188864 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.188876 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000 2014-08-13 17:38:03.188889 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.188904 7fe993686700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.188910 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188928 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188923 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.188950 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.188984 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.188985 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 2014-08-13 17:38:03.189004 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.189007 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.189017 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.189027 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.189038 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:03.189078 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:03.189092 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:03.189102 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.251762 2 0.000060 2014-08-13 17:38:03.189114 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 17:38:03.189123 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 17:38:03.189118 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.189132 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:03.189139 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.189142 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.189153 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:03.189166 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.189176 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.189186 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.189213 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 17:38:03.189249 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.189254 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.189244 7fe993383700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.189265 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.189276 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.189228 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.189287 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.189284 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 17:38:03.189302 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.189305 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.189207 7fe993282700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.189305 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.189315 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.189316 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.189325 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.189326 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.061463 2 0.000066 2014-08-13 17:38:03.189335 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.189339 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.189348 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.189357 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.189363 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.189368 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.189374 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.189380 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.189383 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.567552 2 0.000080 2014-08-13 17:38:03.189393 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.189395 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.189405 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.189406 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.189418 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.189416 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.189427 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.189437 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.189451 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.189460 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.189473 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.189841 7fe992c7c700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.189904 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.189943 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.189950 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.190150 7fe993f8f700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190153 7fe993181700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190160 7fe993484700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190132 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] flushed 2014-08-13 17:38:03.190199 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 17:38:03.190220 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.190225 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.190233 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.190246 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.190258 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.190290 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.190299 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.190308 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.548196 2 0.000085 2014-08-13 17:38:03.190319 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.190327 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.190334 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.190333 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.190345 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.190355 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.190455 7fe992f7f700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190477 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.190494 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.190505 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.190537 7fe992777700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190566 7fe992474700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.190670 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 2014-08-13 17:38:03.190718 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.190626 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 17:38:03.190728 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.190723 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.190733 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.190732 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.190743 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.190757 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map 2014-08-13 17:38:03.190753 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.190794 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] update_heartbeat_peers unchanged 2014-08-13 17:38:03.190794 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.190804 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters 2014-08-13 17:38:03.190809 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.190814 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.115180 2 0.000067 2014-08-13 17:38:03.190822 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,287,309 2014-08-13 17:38:03.190833 7fe99ebe8700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.190826 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started 2014-08-13 17:38:03.190835 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.190836 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start 2014-08-13 17:38:03.190845 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray 2014-08-13 17:38:03.190856 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.190867 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray 2014-08-13 17:38:03.190844 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.494718 2 0.000107 2014-08-13 17:38:03.190878 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.190890 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.190892 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.190902 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.190903 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.190912 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.190936 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 2014-08-13 17:38:03.190968 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.190980 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.190991 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.191001 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.191019 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 17:38:03.191033 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 17:38:03.191062 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,309 down blocked_by {} 2014-08-13 17:38:03.191074 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.191088 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2678 2014-08-13 17:38:03.191100 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.287 2014-08-13 17:38:03.191112 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.309 2014-08-13 17:38:03.191127 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.191138 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.191164 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.191595 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 17:38:03.191616 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.191620 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.191611 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.191631 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.191641 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.191651 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.191660 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,286,343 2014-08-13 17:38:03.191669 7fe99f5e9700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.191670 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.191678 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.502892 2 0.000079 2014-08-13 17:38:03.191688 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.191698 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.191705 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.191962 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000264 0 0.000000 2014-08-13 17:38:03.191977 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.191987 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.191998 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.192011 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.192025 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,286,343 down blocked_by {} 2014-08-13 17:38:03.192037 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.192050 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2645 2014-08-13 17:38:03.192062 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.286 2014-08-13 17:38:03.192074 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 17:38:03.192086 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.192097 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.192111 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.192160 7fe992373700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.192303 7fe992272700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.192304 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 17:38:03.192494 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.192542 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.192568 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 17:38:03.192599 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.192604 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.192602 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.192617 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.192631 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.192644 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.192687 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.192701 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.192687 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 17:38:03.192715 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.192712 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.487054 2 0.000069 2014-08-13 17:38:03.192720 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.192727 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.192751 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.192762 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.192771 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.192777 7fe991a6a700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.192778 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.192790 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.192810 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000047 0 0.000000 2014-08-13 17:38:03.192827 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.192847 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.192852 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.192860 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.192849 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.192867 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.192875 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.192881 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.513748 2 0.000131 2014-08-13 17:38:03.192914 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.192929 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.192948 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.192966 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.192999 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.193022 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.193022 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 17:38:03.193035 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.193045 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.193050 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.193050 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.193064 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.193076 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.193089 7fe991868700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.193089 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.193103 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340,347 2014-08-13 17:38:03.193118 7fe99f5e9700 20 osd.4 194478 need_heartbeat_peer_update 2014-08-13 17:38:03.193120 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.193131 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.561545 2 0.000088 2014-08-13 17:38:03.193145 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.193155 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.193167 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.193180 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000024 0 0.000000 2014-08-13 17:38:03.193162 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 2014-08-13 17:38:03.193185 7fe991969700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.193204 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.193193 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.193209 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.193213 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.193223 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.193225 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.193252 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.193255 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.193271 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map 2014-08-13 17:38:03.193274 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340,347 down blocked_by {} 2014-08-13 17:38:03.193303 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194477 >= same_since 194476, all is well 2014-08-13 17:38:03.193321 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194478:2992 2014-08-13 17:38:03.193333 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.193338 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] update_heartbeat_peers unchanged 2014-08-13 17:38:03.193347 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.347 2014-08-13 17:38:03.193352 7fe991666700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.193354 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters 2014-08-13 17:38:03.193361 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.193368 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.226709 2 0.000092 2014-08-13 17:38:03.193374 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.193388 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.193384 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started 2014-08-13 17:38:03.193397 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start 2014-08-13 17:38:03.193409 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray 2014-08-13 17:38:03.193422 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000026 0 0.000000 2014-08-13 17:38:03.193441 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray 2014-08-13 17:38:03.193456 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.193483 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.193500 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.193499 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 17:38:03.193530 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.193535 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.193560 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.193558 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.193577 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.193572 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.193586 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.193652 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.193673 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.193699 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.534714 2 0.000102 2014-08-13 17:38:03.193713 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.193724 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.193719 7fe99ebe8700 10 osd.4 pg_epoch: 194477 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 17:38:03.193735 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.193755 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.193752 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000027 0 0.000000 2014-08-13 17:38:03.193761 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.193766 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.193774 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.193788 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.193801 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.193822 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.193834 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.193846 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.194255 2 0.000074 2014-08-13 17:38:03.193862 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.193920 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.193929 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.193960 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.193977 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.193874 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.193973 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] flushed 2014-08-13 17:38:03.193987 7fe99ebe8700 1 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.194029 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000155 0 0.000000 2014-08-13 17:38:03.194051 7fe99ebe8700 5 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.194076 7fe99ebe8700 20 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.194094 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.194117 7fe99ebe8700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.194253 7fe991262700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.194259 7fe991161700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.194301 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 2014-08-13 17:38:03.194331 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.194335 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.194348 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.194361 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.194355 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.194374 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map 2014-08-13 17:38:03.194385 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.194417 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.194433 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] update_heartbeat_peers unchanged 2014-08-13 17:38:03.194450 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters 2014-08-13 17:38:03.194481 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.355154 2 0.000069 2014-08-13 17:38:03.194496 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started 2014-08-13 17:38:03.194491 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.194508 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start 2014-08-13 17:38:03.194511 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.194523 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.194519 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray 2014-08-13 17:38:03.194547 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000039 0 0.000000 2014-08-13 17:38:03.194570 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray 2014-08-13 17:38:03.194589 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.194582 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.194617 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.194620 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.194633 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.194631 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.194674 7fe991464700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.194680 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.194700 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.194718 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.194768 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.194799 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.194790 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.194829 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.194849 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.194919 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.194958 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.195027 7fe990a5a700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195080 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 2014-08-13 17:38:03.195109 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.195114 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.195143 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.195161 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.195204 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map 2014-08-13 17:38:03.195236 7fe990656700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195268 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] update_heartbeat_peers unchanged 2014-08-13 17:38:03.195280 7fe990d5d700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195283 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] take_waiters 2014-08-13 17:38:03.195297 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Reset 0.167989 2 0.000132 2014-08-13 17:38:03.195318 7fe990555700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195314 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started 2014-08-13 17:38:03.195327 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Start 2014-08-13 17:38:03.195339 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: transitioning to Stray 2014-08-13 17:38:03.195353 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Start 0.000026 0 0.000000 2014-08-13 17:38:03.195383 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started/Stray 2014-08-13 17:38:03.195404 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.195425 7fe990757700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195421 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.195451 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.195554 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 17:38:03.195592 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.195598 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.195614 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.195630 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.195655 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.195564 7fe990959700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195693 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.195707 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.195718 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.567117 2 0.000109 2014-08-13 17:38:03.195735 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.195760 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.195773 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.195790 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 2014-08-13 17:38:03.195819 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.195832 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.195845 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.195857 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.195925 7fe99f5e9700 10 osd.4 pg_epoch: 194477 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 2014-08-13 17:38:03.195962 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.195966 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.195979 7fe990353700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.195978 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.195991 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.196004 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:03.196050 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:03.196066 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:03.196078 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.105931 2 0.000075 2014-08-13 17:38:03.196095 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 17:38:03.196108 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 17:38:03.196119 7fe99f5e9700 1 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:03.196132 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000024 0 0.000000 2014-08-13 17:38:03.196145 7fe99f5e9700 5 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:03.196159 7fe99f5e9700 20 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194478 2014-08-13 17:38:03.196182 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 NullEvt 2014-08-13 17:38:03.196196 7fe99f5e9700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 17:38:03.195977 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196211 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196233 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196232 7fe990b5b700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.196301 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196345 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.196359 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196346 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196368 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196366 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196382 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196382 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196394 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196423 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.196460 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] flushed 2014-08-13 17:38:03.196482 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196488 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196513 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196511 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.196528 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196533 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196534 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196540 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196549 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196565 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196556 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196570 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196571 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196577 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196589 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196602 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196593 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196607 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196592 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.196618 7fe98fc4c700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.196651 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196656 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196685 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196703 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196690 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.196712 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196711 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196721 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196731 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196759 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196741 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196767 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.196802 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196825 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.196842 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196888 7fe98fb4b700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.196899 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.196938 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196944 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.196965 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196965 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.196979 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.196981 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.197020 7fe990858700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197039 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197064 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.197078 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.197107 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197121 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.197133 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.197138 7fe98f646700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197139 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197152 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.197164 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.197162 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] flushed 2014-08-13 17:38:03.197215 7fe98f343700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197234 7fe990c5c700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197252 7fe98fa4a700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197260 7fe98f747700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197362 7fe98f242700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197391 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197396 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197417 7fe994393700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.197415 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.197420 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.197432 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.197433 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.197495 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197506 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197520 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.197534 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.197534 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.197549 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.197564 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.197581 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.197594 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.197608 7fe9bae887a0 0 osd.4 194478 done with init, starting boot process 2014-08-13 17:38:03.197615 7fe9bae887a0 10 osd.4 194478 start_boot - have maps 185139..194478 2014-08-13 17:38:03.197766 7fe98f040700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.198235 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] flushed 2014-08-13 17:38:03.198275 7fe99ffea700 10 osd.4 194478 _maybe_boot mon has osdmaps 185139..194484 2014-08-13 17:38:03.198299 7fe99ffea700 10 osd.4 194478 _send_boot 2014-08-13 17:38:03.198306 7fe99ffea700 10 osd.4 194478 client_addr 192.168.100.115:6805/34805, cluster_addr 192.168.1.115:6813/34805, hb_back_addr 192.168.1.115:6817/34805, hb_front_addr 192.168.100.115:6806/34805 2014-08-13 17:38:03.198333 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.198360 7fe98ee3e700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.198362 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.198377 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.198373 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.198416 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.198444 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.198458 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.198612 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 17:38:03.198651 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.198661 7fe99ffea700 10 osd.4 194478 _collect_metadata {arch=x86_64,back_addr=192.168.1.115:6813/34805,ceph_version=ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6),cpu=Intel(R) Xeon(R) CPU E5-2450 0 @ 2.10GHz,front_addr=192.168.100.115:6805/34805,hb_back_addr=192.168.1.115:6817/34805,hb_front_addr=192.168.100.115:6806/34805,hostname=storage0115-ib.pocstor,kernel_description=#1 SMP Wed Mar 13 00:26:49 UTC 2013,kernel_version=2.6.32-358.2.1.el6.x86_64,mem_swap_kb=23446856,mem_total_kb=24568044,os=Linux,osd_data=/var/lib/ceph/osd/ceph-4,osd_journal=/var/lib/ceph/osd/ceph-4/journal} 2014-08-13 17:38:03.198676 7fe99f5e9700 15 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.198692 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.198979 7fe9aad09700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.199023 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194478 epoch_requested: 194478 FlushedEvt 2014-08-13 17:38:03.199060 7fe99ebe8700 15 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.199072 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.199467 7fe98da33700 10 osd.4 194478 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.266319 7fe9a63f4700 10 osd.4 194478 do_waiters -- start 2014-08-13 17:38:03.266323 7fe9a63f4700 10 osd.4 194478 do_waiters -- finish 2014-08-13 17:38:03.266325 7fe9a63f4700 20 osd.4 194478 _dispatch 0x4d8e9c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.266394 7fe9a63f4700 3 osd.4 194478 handle_osd_map epochs [194479,194485], i have 194478, src has [185139,194485] 2014-08-13 17:38:03.266403 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194479 2014-08-13 17:38:03.270468 7fe98d52e700 10 osd.4 194478 new session 0x2a89180 con=0x4ec8580 addr=192.168.1.110:6802/41438 2014-08-13 17:38:03.270644 7fe98d52e700 10 osd.4 194478 session 0x2a89180 osd.342 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.270903 7fe98d62f700 10 osd.4 194478 new session 0x2a89260 con=0x4ec8160 addr=192.168.1.110:6818/45268 2014-08-13 17:38:03.271016 7fe98d62f700 10 osd.4 194478 session 0x2a89260 osd.339 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.273041 7fe98d029700 10 osd.4 194478 new session 0x2a89340 con=0x4ec9b80 addr=192.168.1.106:6802/4876 2014-08-13 17:38:03.273081 7fe98d029700 10 osd.4 194478 session 0x2a89340 osd.330 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.274436 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194480 2014-08-13 17:38:03.281970 7fe98d42d700 10 osd.4 194478 new session 0x2a89420 con=0x4ec8c60 addr=192.168.1.115:6810/59615 2014-08-13 17:38:03.281987 7fe98d730700 10 osd.4 194478 new session 0x2a89500 con=0x4ec8000 addr=192.168.1.106:6820/10050 2014-08-13 17:38:03.282012 7fe98d42d700 10 osd.4 194478 session 0x2a89420 osd.0 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.282010 7fe98c720700 10 osd.4 194478 new session 0x2a895e0 con=0x4ecaec0 addr=192.168.1.103:6813/37929 2014-08-13 17:38:03.282021 7fe98d730700 10 osd.4 194478 session 0x2a89500 osd.320 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.282046 7fe98c720700 10 osd.4 194478 session 0x2a895e0 osd.11 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.282204 7fe98cb24700 10 osd.4 194478 new session 0x2a896c0 con=0x4ec98c0 addr=192.168.1.111:6832/13543 2014-08-13 17:38:03.282246 7fe98cb24700 10 osd.4 194478 session 0x2a896c0 osd.52 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.282675 7fe98bd16700 10 osd.4 194478 new session 0x2a897a0 con=0x4ecb700 addr=192.168.1.105:6814/41806 2014-08-13 17:38:03.282713 7fe98bd16700 10 osd.4 194478 session 0x2a897a0 osd.396 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.283639 7fe98c41d700 10 osd.4 194478 new session 0x2a89880 con=0x4ecac00 addr=192.168.1.102:6824/15623 2014-08-13 17:38:03.283674 7fe98c41d700 10 osd.4 194478 session 0x2a89880 osd.365 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.288446 7fe98b710700 10 osd.4 194478 new session 0x2a89960 con=0x4ecce60 addr=192.168.1.113:6800/14161 2014-08-13 17:38:03.288486 7fe98b710700 10 osd.4 194478 session 0x2a89960 osd.315 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.288482 7fe98a902700 10 osd.4 194478 new session 0x2a89a40 con=0x4ecf640 addr=192.168.1.112:6838/8300 2014-08-13 17:38:03.288487 7fe98bb14700 10 osd.4 194478 new session 0x2a89b20 con=0x4ecd120 addr=192.168.1.101:6802/22303 2014-08-13 17:38:03.288518 7fe98a902700 10 osd.4 194478 session 0x2a89a40 osd.48 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.288523 7fe98bb14700 10 osd.4 194478 session 0x2a89b20 osd.314 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.288857 7fe98b009700 10 osd.4 194478 new session 0x2a89c00 con=0x4ecc8e0 addr=192.168.1.101:6816/24270 2014-08-13 17:38:03.288909 7fe98b009700 10 osd.4 194478 session 0x2a89c00 osd.311 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.291351 7fe98b40d700 10 osd.4 194478 new session 0x2a89ce0 con=0x4eccba0 addr=192.168.1.103:6800/36920 2014-08-13 17:38:03.291414 7fe98b40d700 10 osd.4 194478 session 0x2a89ce0 osd.25 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.291541 7fe98bc15700 10 osd.4 194478 new session 0x2a89dc0 con=0x4ecb440 addr=192.168.1.107:6815/53679 2014-08-13 17:38:03.291591 7fe98bc15700 10 osd.4 194478 session 0x2a89dc0 osd.2 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.291742 7fe989ff9700 10 osd.4 194478 new session 0x2a89ea0 con=0x4ecee00 addr=192.168.1.114:6842/56512 2014-08-13 17:38:03.291804 7fe989ff9700 10 osd.4 194478 session 0x2a89ea0 osd.370 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.293081 7fe98c821700 10 osd.4 194478 new session 0x2a89f80 con=0x4ec9600 addr=192.168.1.113:6816/17521 2014-08-13 17:38:03.293120 7fe98c821700 10 osd.4 194478 session 0x2a89f80 osd.312 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.295962 7fe98a3fd700 10 osd.4 194478 new session 0x2a8a060 con=0x4ecf0c0 addr=192.168.1.101:6814/23703 2014-08-13 17:38:03.296007 7fe98a3fd700 10 osd.4 194478 session 0x2a8a060 osd.309 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.296234 7fe98c21b700 10 osd.4 194478 new session 0x2a8a140 con=0x4ecb9c0 addr=192.168.1.102:6822/15390 2014-08-13 17:38:03.296288 7fe98c21b700 10 osd.4 194478 session 0x2a8a140 osd.381 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.296728 7fe988be5700 10 osd.4 194478 new session 0x2a8a220 con=0x428e720 addr=192.168.1.112:6830/6302 2014-08-13 17:38:03.296847 7fe9a45f1700 10 osd.4 194478 note_peer_epoch osd.376 now has 194485 2014-08-13 17:38:03.296864 7fe98d32c700 10 osd.4 194478 new session 0x2a8a300 con=0x4ec89a0 addr=192.168.1.109:6816/33855 2014-08-13 17:38:03.296837 7fe988be5700 10 osd.4 194478 session 0x2a8a220 osd.60 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.296899 7fe98d32c700 10 osd.4 194478 session 0x2a8a300 osd.293 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.297250 7fe98ac05700 10 osd.4 194478 new session 0x2a8a3e0 con=0x4ecf900 addr=192.168.1.109:6812/32413 2014-08-13 17:38:03.297260 7fe98c31c700 10 osd.4 194478 new session 0x2a8a4c0 con=0x4eca940 addr=192.168.1.102:6840/19195 2014-08-13 17:38:03.297291 7fe98ac05700 10 osd.4 194478 session 0x2a8a3e0 osd.310 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.297299 7fe98c31c700 10 osd.4 194478 session 0x2a8a4c0 osd.378 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298218 7fe98a700700 10 osd.4 194478 new session 0x2a8a5a0 con=0x4ecf380 addr=192.168.1.113:6810/15731 2014-08-13 17:38:03.298274 7fe98a700700 10 osd.4 194478 session 0x2a8a5a0 osd.302 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298272 7fe9885df700 10 osd.4 194478 new session 0x2a8a680 con=0x5251080 addr=192.168.1.102:6826/15861 2014-08-13 17:38:03.298317 7fe9885df700 10 osd.4 194478 session 0x2a8a680 osd.368 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298318 7fe988de7700 10 osd.4 194478 new session 0x2a8a760 con=0x428e460 addr=192.168.1.109:6818/34549 2014-08-13 17:38:03.298364 7fe988de7700 10 osd.4 194478 session 0x2a8a760 osd.284 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298468 7fe9875cf700 10 osd.4 194478 new session 0x2a8a840 con=0x5250580 addr=192.168.1.115:6838/62713 2014-08-13 17:38:03.298508 7fe9875cf700 10 osd.4 194478 session 0x2a8a840 osd.7 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298669 7fe9889e3700 10 osd.4 194478 new session 0x2a8aa00 con=0x41e5540 addr=192.168.1.102:6832/16753 2014-08-13 17:38:03.298517 7fe9882dc700 10 osd.4 194478 new session 0x2a8a920 con=0x5250dc0 addr=192.168.1.114:6822/51844 2014-08-13 17:38:03.298708 7fe9889e3700 10 osd.4 194478 session 0x2a8aa00 osd.371 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298722 7fe989bf5700 10 osd.4 194478 new session 0x2a8aae0 con=0x4313f40 addr=192.168.1.114:6830/52879 2014-08-13 17:38:03.298725 7fe9882dc700 10 osd.4 194478 session 0x2a8a920 osd.380 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298755 7fe989bf5700 10 osd.4 194478 session 0x2a8aae0 osd.375 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298874 7fe9877d1700 10 osd.4 194478 new session 0x2a8abc0 con=0x5250840 addr=192.168.1.108:6824/4037 2014-08-13 17:38:03.298878 7fe987bd5700 10 osd.4 194478 new session 0x2a8aca0 con=0x5250b00 addr=192.168.1.102:6830/16444 2014-08-13 17:38:03.298933 7fe987bd5700 10 osd.4 194478 session 0x2a8aca0 osd.351 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.298934 7fe9877d1700 10 osd.4 194478 session 0x2a8abc0 osd.369 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.314915 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194481 2014-08-13 17:38:03.317069 7fe9874ce700 10 osd.4 194478 new session 0x2a8ad80 con=0x52502c0 addr=192.168.1.111:6830/12680 2014-08-13 17:38:03.317137 7fe9874ce700 10 osd.4 194478 session 0x2a8ad80 osd.63 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.318577 7fe9859b3700 10 osd.4 194478 new session 0x2a8ae60 con=0x53814a0 addr=192.168.1.104:6810/63264 2014-08-13 17:38:03.318637 7fe9859b3700 10 osd.4 194478 session 0x2a8ae60 osd.322 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.318660 7fe9852ac700 10 osd.4 194478 new session 0x2a8af40 con=0x5380f20 addr=192.168.1.114:6836/54427 2014-08-13 17:38:03.318711 7fe9852ac700 10 osd.4 194478 session 0x2a8af40 osd.355 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.318786 7fe9854ae700 10 osd.4 194478 new session 0x2a8b020 con=0x53811e0 addr=192.168.1.110:6810/42728 2014-08-13 17:38:03.318847 7fe9854ae700 10 osd.4 194478 session 0x2a8b020 osd.329 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.318874 7fe9849a3700 10 osd.4 194478 new session 0x2a8b100 con=0x53809a0 addr=192.168.1.114:6832/53185 2014-08-13 17:38:03.318915 7fe9849a3700 10 osd.4 194478 session 0x2a8b100 osd.350 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.318978 7fe984ca6700 10 osd.4 194478 new session 0x2a8b1e0 con=0x5380c60 addr=192.168.1.104:6800/61726 2014-08-13 17:38:03.319029 7fe984ca6700 10 osd.4 194478 session 0x2a8b1e0 osd.349 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.319220 7fe9848a2700 10 osd.4 194478 new session 0x2a8b2c0 con=0x53806e0 addr=192.168.1.101:6806/6812 2014-08-13 17:38:03.319341 7fe9848a2700 10 osd.4 194478 session 0x2a8b2c0 osd.306 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.319396 7fe98439d700 10 osd.4 194478 new session 0x2a8b3a0 con=0x53853e0 addr=192.168.1.108:6800/30527 2014-08-13 17:38:03.319447 7fe98439d700 10 osd.4 194478 session 0x2a8b3a0 osd.382 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.319969 7fe9846a0700 10 osd.4 194478 new session 0x2a8b480 con=0x53856a0 addr=192.168.1.108:6828/4534 2014-08-13 17:38:03.320021 7fe9846a0700 10 osd.4 194478 session 0x2a8b480 osd.366 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.320062 7fe9847a1700 10 osd.4 194478 new session 0x2a8b560 con=0x5385960 addr=192.168.1.108:6826/4273 2014-08-13 17:38:03.320199 7fe9847a1700 10 osd.4 194478 session 0x2a8b560 osd.352 has caps osdcap[grant(*)] 'allow *' 2014-08-13 17:38:03.322014 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194482 2014-08-13 17:38:03.327331 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194483 2014-08-13 17:38:03.332480 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194484 2014-08-13 17:38:03.334978 7fe9a45f1700 10 osd.4 194478 note_peer_epoch osd.345 now has 194485 2014-08-13 17:38:03.337575 7fe9a63f4700 10 osd.4 194478 handle_osd_map got inc map for epoch 194485 2014-08-13 17:38:03.342622 7fe9a63f4700 10 osd.4 194478 advance to epoch 194479 (<= newest 194485) 2014-08-13 17:38:03.342726 7fe9a63f4700 7 osd.4 194479 advance_map epoch 194479 74 pgs 2014-08-13 17:38:03.342757 7fe9a63f4700 10 osd.4 194479 advance to epoch 194480 (<= newest 194485) 2014-08-13 17:38:03.342836 7fe9a63f4700 7 osd.4 194480 advance_map epoch 194480 74 pgs 2014-08-13 17:38:03.342865 7fe9a63f4700 10 osd.4 194480 advance to epoch 194481 (<= newest 194485) 2014-08-13 17:38:03.342941 7fe9a63f4700 7 osd.4 194481 advance_map epoch 194481 74 pgs 2014-08-13 17:38:03.342969 7fe9a63f4700 10 osd.4 194481 advance to epoch 194482 (<= newest 194485) 2014-08-13 17:38:03.343044 7fe9a63f4700 7 osd.4 194482 advance_map epoch 194482 74 pgs 2014-08-13 17:38:03.343072 7fe9a63f4700 10 osd.4 194482 advance to epoch 194483 (<= newest 194485) 2014-08-13 17:38:03.343147 7fe9a63f4700 7 osd.4 194483 advance_map epoch 194483 74 pgs 2014-08-13 17:38:03.343190 7fe9a63f4700 10 osd.4 194483 advance to epoch 194484 (<= newest 194485) 2014-08-13 17:38:03.343269 7fe9a63f4700 7 osd.4 194484 advance_map epoch 194484 74 pgs 2014-08-13 17:38:03.343298 7fe9a63f4700 10 osd.4 194484 advance to epoch 194485 (<= newest 194485) 2014-08-13 17:38:03.343373 7fe9a63f4700 7 osd.4 194485 advance_map epoch 194485 74 pgs 2014-08-13 17:38:03.343381 7fe9a63f4700 10 osd.4 194485 up_epoch is 194485 2014-08-13 17:38:03.343383 7fe9a63f4700 10 osd.4 194485 boot_epoch is 194485 2014-08-13 17:38:03.343411 7fe9a63f4700 1 osd.4 194485 state: booting -> active 2014-08-13 17:38:03.343420 7fe9a63f4700 10 osd.4 194485 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194485 [185139,194485] lci=[194485,194485]) 2014-08-13 17:38:03.343634 7fe9a63f4700 0 osd.4 194485 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:38:03.343653 7fe9a63f4700 7 osd.4 194485 consume_map version 194485 2014-08-13 17:38:03.343719 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 17:38:03.343769 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.343821 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.343848 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.343842 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_advance_map [286,375]/[375,10] -- 286/375 2014-08-13 17:38:03.343870 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.343899 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.343890 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.343909 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Reset advmap 2014-08-13 17:38:03.343912 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.343899 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,18]/[351,18] -- 351/351 2014-08-13 17:38:03.343943 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.343930 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.343935 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.343950 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.343950 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [286,375] newacting [375,10] 2014-08-13 17:38:03.343959 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] null 2014-08-13 17:38:03.343975 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.343977 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.343984 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.343993 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,18] newacting [351,18] 2014-08-13 17:38:03.344006 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344010 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.344007 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [4,286,375]/[4,375,10] r=0 lpr=194478 pi=193640-194476/30 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194479 2014-08-13 17:38:03.344029 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,4,18] r=1 lpr=194478 pi=192970-194476/26 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.344033 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344054 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] null 2014-08-13 17:38:03.344062 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,286,375](4) acting [4,375,10](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.344065 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [351,4,18](351) acting [351,4,18](351)): not rw, up_thru 194477 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194477-194478 up [351,4,18](351) acting [351,4,18](351) maybe_went_rw) : primary up 192967-194477 includes interval 2014-08-13 17:38:03.344078 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344092 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194476/194477/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194477-194478 up [351,4,18](351) acting [351,4,18](351) maybe_went_rw) 2014-08-13 17:38:03.344089 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194476/194477/194477) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] noting past interval(194477-194478 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) 2014-08-13 17:38:03.344101 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344114 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,4,18] -> [351,18], acting [351,4,18] -> [351,18], acting_primary 351 -> 351, up_primary 351 -> 351, role 1 -> -1 2014-08-13 17:38:03.344116 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] up [4,286,375] -> [286,375], acting [4,375,10] -> [375,10], acting_primary 4 -> 375, up_primary 4 -> 286, role 0 -> -1 2014-08-13 17:38:03.344124 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 17:38:03.344139 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.344143 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] clear_primary_state 2014-08-13 17:38:03.344151 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344164 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.344186 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344193 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.344208 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:03.344220 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.344231 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 17:38:03.344243 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.344254 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344267 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.344274 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344292 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.344304 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344324 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344326 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.344345 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.344344 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344363 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.344365 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344378 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.344386 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344395 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.344409 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.344406 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.344431 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 17:38:03.344434 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.344452 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344471 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344472 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,18]/[351,18,21] -- 351/351 2014-08-13 17:38:03.344493 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.344490 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 17:38:03.344497 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.344180 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] agent_stop 2014-08-13 17:38:03.344513 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344514 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.344533 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344534 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,18] newacting [351,18,21] 2014-08-13 17:38:03.344532 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] on_change 2014-08-13 17:38:03.344552 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.344556 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:03.344562 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.344573 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18] r=-1 lpr=194479 pi=192970-194478/27 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.344581 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344582 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:03.344605 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] cancel_copy_ops 2014-08-13 17:38:03.344605 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344612 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [351,18](351) acting [351,18](351)): not rw, up_thru 194477 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194479-194479 up [351,18](351) acting [351,18](351)) : primary up 192967-194477 does not include interval 2014-08-13 17:38:03.344624 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] cancel_flush_ops 2014-08-13 17:38:03.344632 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 17:38:03.344635 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194479/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [351,18](351) acting [351,18](351)) 2014-08-13 17:38:03.344645 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] on_change_cleanup 2014-08-13 17:38:03.344654 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.344657 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] up [351,18] -> [351,18], acting [351,18] -> [351,18,21], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 17:38:03.344665 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] on_change 2014-08-13 17:38:03.344676 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344679 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.344687 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] exit NotTrimming 2014-08-13 17:38:03.344697 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.344699 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 17:38:03.344708 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped m=56] enter NotTrimming 2014-08-13 17:38:03.344714 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.344728 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_stats 2014-08-13 17:38:03.344727 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344734 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.344750 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.344748 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.344759 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344771 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.344771 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 17:38:03.344782 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 17:38:03.344791 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.344792 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.344811 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.344811 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.344810 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 17:38:03.344830 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.344830 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.344838 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 17:38:03.344855 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.344855 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.344863 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344874 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.344885 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344894 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.344899 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [286,375]/[375,10,301] -- 286/375 2014-08-13 17:38:03.344906 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.344930 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.344929 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344929 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,18]/[351,18,21] -- 351/351 2014-08-13 17:38:03.344937 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.344952 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.344951 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.344956 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.344958 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.344974 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.344977 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.344976 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [286,375] newacting [375,10,301] 2014-08-13 17:38:03.344997 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.344997 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.344996 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] null 2014-08-13 17:38:03.345017 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10] r=-1 lpr=194479 pi=193640-194478/31 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194480 2014-08-13 17:38:03.345023 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:03.345038 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,21,52]/[351,18,21] -- 351/351 2014-08-13 17:38:03.345059 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.345045 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 17:38:03.345052 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [286,375](286) acting [375,10](375)): not rw, up_thru 194476 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [286,375](286) acting [375,10](375)) : primary up 192969-194476 does not include interval 2014-08-13 17:38:03.345063 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.345070 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:03.345087 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.345085 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194479/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] noting past interval(194479-194479 up [286,375](286) acting [375,10](375)) 2014-08-13 17:38:03.345095 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345106 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] acting_up_affected newup [351,21,52] newacting [351,18,21] 2014-08-13 17:38:03.345109 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] up [286,375] -> [286,375], acting [375,10] -> [375,10,301], acting_primary 375 -> 375, up_primary 286 -> 286, role -1 -> -1 2014-08-13 17:38:03.345119 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.345124 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.345132 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.345142 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,18]/[351,18,21] r=-1 lpr=194480 pi=192970-194479/28 crt=192888'481 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.345141 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.345152 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.345170 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.345185 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.345174 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [351,18](351) acting [351,18,21](351)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194480-194481 up [351,18](351) acting [351,18,21](351) maybe_went_rw) : primary up 192967-194480 includes interval 2014-08-13 17:38:03.345192 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345205 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.345208 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194479/194480/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [351,18](351) acting [351,18,21](351) maybe_went_rw) 2014-08-13 17:38:03.345214 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 17:38:03.345223 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.345229 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] up [351,18] -> [351,21,52], acting [351,18,21] -> [351,18,21], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 17:38:03.345239 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 17:38:03.345241 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.345251 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.345262 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.345264 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345269 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.345285 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.345283 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.345289 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.345303 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.345303 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.345311 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345320 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.345323 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.345333 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:03.345339 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.345342 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.345356 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345360 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.345360 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.345381 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.345380 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 17:38:03.345401 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.345399 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [286,375]/[375,10,301] -- 286/375 2014-08-13 17:38:03.345408 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345429 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.345420 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.345430 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 17:38:03.345435 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.345441 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.345453 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.345453 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 17:38:03.345462 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.345475 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.345480 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:03.345499 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:03.345526 7fe9a63f4700 10 osd.4 194485 maybe_update_heartbeat_peers updating 2014-08-13 17:38:03.345512 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,21,52]/[351,21,52] -- 351/351 2014-08-13 17:38:03.345532 7fe9a63f4700 20 osd.4 194485 24.30b heartbeat_peers 2014-08-13 17:38:03.345536 7fe9a63f4700 20 osd.4 194485 22.220 heartbeat_peers 2014-08-13 17:38:03.345535 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.345538 7fe9a63f4700 20 osd.4 194485 22.362 heartbeat_peers 2014-08-13 17:38:03.345540 7fe9a63f4700 20 osd.4 194485 22.28f heartbeat_peers 2014-08-13 17:38:03.345544 7fe9a63f4700 20 osd.4 194485 22.14a heartbeat_peers 2014-08-13 17:38:03.345546 7fe9a63f4700 20 osd.4 194485 22.22c heartbeat_peers 4,301,324 2014-08-13 17:38:03.345540 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.345537 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [301,375,10]/[375,10,301] -- 301/375 2014-08-13 17:38:03.345568 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.345559 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.345574 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.345574 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] acting_up_affected newup [351,21,52] newacting [351,21,52] 2014-08-13 17:38:03.345593 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.345592 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.345610 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52]/[351,18,21] r=-1 lpr=194482 pi=192970-194481/29 crt=192888'481 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.345610 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [301,375,10] newacting [375,10,301] 2014-08-13 17:38:03.345634 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.345647 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [351,21,52](351) acting [351,18,21](351)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194482-194482 up [351,21,52](351) acting [351,18,21](351)) : primary up 192967-194480 does not include interval 2014-08-13 17:38:03.345652 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [286,375]/[375,10,301] r=-1 lpr=194480 pi=193640-194479/32 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194482 2014-08-13 17:38:03.345669 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194482/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [351,21,52](351) acting [351,18,21](351)) 2014-08-13 17:38:03.345686 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [286,375](286) acting [375,10,301](375)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [286,375](286) acting [375,10,301](375) maybe_went_rw) : primary up 192969-194480 includes interval 2014-08-13 17:38:03.345686 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,21,52] -> [351,21,52], acting [351,18,21] -> [351,21,52], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 17:38:03.345708 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.345707 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194479/194480/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] noting past interval(194480-194481 up [286,375](286) acting [375,10,301](375) maybe_went_rw) 2014-08-13 17:38:03.345726 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.345742 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.345735 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.301 192.168.1.113:6815/16727 192.168.100.113:6840/16727 2014-08-13 17:38:03.345730 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] up [286,375] -> [301,375,10], acting [375,10,301] -> [375,10,301], acting_primary 375 -> 375, up_primary 286 -> 301, role -1 -> -1 2014-08-13 17:38:03.345759 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.345761 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.345773 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.345780 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.345789 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.345802 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.345808 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.345820 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.345823 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.345836 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.345839 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.345854 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.345853 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.345873 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.345875 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.345886 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.324 192.168.1.104:6815/65209 192.168.100.104:6840/65209 2014-08-13 17:38:03.345890 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.345904 7fe9a63f4700 20 osd.4 194485 22.619 heartbeat_peers 2014-08-13 17:38:03.345908 7fe9a63f4700 20 osd.4 194485 22.30f heartbeat_peers 2014-08-13 17:38:03.345894 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.345914 7fe9a63f4700 20 osd.4 194485 24.9b heartbeat_peers 4,291,350,367,378 2014-08-13 17:38:03.345916 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.345929 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,21,52]/[351,21,52] -- 351/351 2014-08-13 17:38:03.345933 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.345953 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.345953 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.345959 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.345975 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.345977 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.345994 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.346006 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.291 192.168.1.101:6821/25633 192.168.100.101:6841/25633 2014-08-13 17:38:03.346030 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [301,375,10]/[301,375,10] -- 301/301 2014-08-13 17:38:03.346061 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.346046 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 17:38:03.346076 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.346068 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.346080 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.346091 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.346105 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.346113 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [301,375,10] newacting [301,375,10] 2014-08-13 17:38:03.346122 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,4,18] newacting [351,4,18] 2014-08-13 17:38:03.346125 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.350 192.168.1.114:6833/53185 192.168.100.114:6835/53185 2014-08-13 17:38:03.346132 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.346139 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.346154 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,21,52] r=-1 lpr=194483 pi=192970-194482/30 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.346150 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10]/[375,10,301] r=-1 lpr=194482 pi=193640-194481/33 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194483 2014-08-13 17:38:03.346191 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) : primary up 192969-194480 does not include interval 2014-08-13 17:38:03.346194 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [351,21,52](351) acting [351,21,52](351)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194483-194484 up [351,21,52](351) acting [351,21,52](351) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.346211 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194482/194479) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] noting past interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 17:38:03.346216 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194482/194483/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [351,21,52](351) acting [351,21,52](351) maybe_went_rw) 2014-08-13 17:38:03.346231 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] up [301,375,10] -> [301,375,10], acting [375,10,301] -> [301,375,10], acting_primary 375 -> 301, up_primary 301 -> 301, role -1 -> -1 2014-08-13 17:38:03.346235 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,21,52] -> [351,4,18], acting [351,21,52] -> [351,4,18], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> 1 2014-08-13 17:38:03.346251 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.346254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.346267 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.346270 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.346283 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.346285 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.346304 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.346305 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.346320 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.346320 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.346324 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.367 192.168.1.114:6841/55513 192.168.100.114:6847/55513 2014-08-13 17:38:03.346336 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.346337 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.346352 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.346354 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.346368 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.346371 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.346384 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.346388 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.346399 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.346413 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.346405 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] [375,10,301] -> [301,375,10], acting primary 375 -> 301 2014-08-13 17:38:03.346428 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.346432 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.346446 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.346453 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.346465 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.346470 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.346487 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.346511 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [301,375,10]/[301,375,10] -- 301/301 2014-08-13 17:38:03.346536 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.346541 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.346561 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.346561 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.346578 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.346583 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.346596 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.378 192.168.1.102:6841/19195 192.168.100.102:6844/19195 2014-08-13 17:38:03.346600 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.700850 9 0.002271 2014-08-13 17:38:03.346617 7fe9a63f4700 20 osd.4 194485 22.581 heartbeat_peers 2014-08-13 17:38:03.346621 7fe9a63f4700 20 osd.4 194485 22.f3 heartbeat_peers 4,285,312 2014-08-13 17:38:03.346611 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [4,286,375]/[4,286,375] -- 4/4 2014-08-13 17:38:03.346622 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.346636 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.346639 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.346641 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.346653 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.346658 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.346673 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [4,286,375] newacting [4,286,375] 2014-08-13 17:38:03.346675 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 2014-08-13 17:38:03.346689 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.346695 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.346705 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [301,375,10] r=-1 lpr=194483 pi=193640-194482/34 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194485 2014-08-13 17:38:03.346719 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.346736 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.346737 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [301,375,10](301) acting [301,375,10](301)): not rw, up_thru 194483 up_from 192972 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) : primary up 192972-194483 includes interval 2014-08-13 17:38:03.346760 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194482/194483/194483) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] noting past interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 17:38:03.346777 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] up [301,375,10] -> [4,286,375], acting [301,375,10] -> [4,286,375], acting_primary 301 -> 4, up_primary 301 -> 4, role -1 -> 0 2014-08-13 17:38:03.346796 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] clear_primary_state 2014-08-13 17:38:03.346812 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] agent_stop 2014-08-13 17:38:03.346831 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.346849 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd 194485:30292 2014-08-13 17:38:03.346866 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.346882 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.346896 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.346910 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.346924 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.346937 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.346952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.346966 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.346981 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.346996 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.347011 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] clear_stats 2014-08-13 17:38:03.347026 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] requeue_ops 2014-08-13 17:38:03.347040 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] on_role_change 2014-08-13 17:38:03.347054 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] requeue_ops 2014-08-13 17:38:03.347068 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] cancel_recovery 2014-08-13 17:38:03.347082 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] clear_recovery_state 2014-08-13 17:38:03.347097 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.347120 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_activate_map 2014-08-13 17:38:03.347139 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] update_heartbeat_peers -> 4,286,375 2014-08-13 17:38:03.347962 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.347946 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.285 192.168.1.101:6823/26295 192.168.100.101:6845/26295 2014-08-13 17:38:03.348021 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [58,343]/[58,343] -- 58/58 2014-08-13 17:38:03.348051 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.348058 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.348074 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.348088 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [58,343] newacting [58,343] 2014-08-13 17:38:03.348103 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.348121 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,58,343] r=0 lpr=194478 pi=193306-194476/27 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.348170 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,58,343](4) acting [4,58,343](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.348190 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] noting past interval(194477-194478 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) 2014-08-13 17:38:03.348206 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] up [4,58,343] -> [58,343], acting [4,58,343] -> [58,343], acting_primary 4 -> 58, up_primary 4 -> 58, role 0 -> -1 2014-08-13 17:38:03.348224 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.348240 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.348254 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.348267 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.348281 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.348296 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.348309 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.348327 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.348341 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.348357 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.348371 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.348385 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.348398 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.348411 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.348423 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.348435 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.348448 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.348464 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.348501 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [58,343]/[58,343,61] -- 58/58 2014-08-13 17:38:03.348520 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.348525 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.348539 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.348553 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,343] newacting [58,343,61] 2014-08-13 17:38:03.348574 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.348591 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'415 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.348627 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [58,343](58) acting [58,343](58)): not rw, up_thru 194476 up_from 192935 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [58,343](58) acting [58,343](58)) : primary up 192935-194476 does not include interval 2014-08-13 17:38:03.348655 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [58,343](58) acting [58,343](58)) 2014-08-13 17:38:03.348673 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] up [58,343] -> [58,343], acting [58,343] -> [58,343,61], acting_primary 58 -> 58, up_primary 58 -> 58, role -1 -> -1 2014-08-13 17:38:03.348692 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.348707 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.348721 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.348737 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.348751 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.348765 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.348780 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.348795 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.348814 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.348829 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.348843 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.348859 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.348897 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] handle_advance_map [58,343]/[58,343,61] -- 58/58 2014-08-13 17:38:03.348916 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.348921 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.348936 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.348952 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.348990 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] handle_advance_map [61,349,362]/[58,343,61] -- 61/58 2014-08-13 17:38:03.349008 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.349012 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.349031 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.349045 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,349,362] newacting [58,343,61] 2014-08-13 17:38:03.349061 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.349076 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [58,343]/[58,343,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'415 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.349103 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [58,343](58) acting [58,343,61](58)): not rw, up_thru 194480 up_from 192935 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [58,343](58) acting [58,343,61](58) maybe_went_rw) : primary up 192935-194480 includes interval 2014-08-13 17:38:03.349119 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [58,343](58) acting [58,343,61](58) maybe_went_rw) 2014-08-13 17:38:03.349136 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] up [58,343] -> [61,349,362], acting [58,343,61] -> [58,343,61], acting_primary 58 -> 58, up_primary 58 -> 61, role -1 -> -1 2014-08-13 17:38:03.349163 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.349179 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.349193 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.349208 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.349223 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.349237 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.349251 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.349251 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.312 192.168.1.113:6817/17521 192.168.100.113:6843/17521 2014-08-13 17:38:03.349280 7fe9a63f4700 20 osd.4 194485 22.116 heartbeat_peers 2014-08-13 17:38:03.349268 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.349288 7fe9a63f4700 20 osd.4 194485 24.3d heartbeat_peers 4,20,25,303,362 2014-08-13 17:38:03.349289 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.349310 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.349324 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.349341 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.349382 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 17:38:03.349408 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.349413 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.349429 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.349444 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,349,362] newacting [61,349,362] 2014-08-13 17:38:03.349464 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.349479 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362]/[58,343,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'415 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.349478 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.20 192.168.1.103:6816/38269 192.168.100.103:6827/38269 2014-08-13 17:38:03.349508 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)): not rw, up_thru 194480 up_from 192935 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) : primary up 192935-194480 does not include interval 2014-08-13 17:38:03.349527 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) 2014-08-13 17:38:03.349544 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] up [61,349,362] -> [61,349,362], acting [58,343,61] -> [61,349,362], acting_primary 58 -> 61, up_primary 61 -> 61, role -1 -> -1 2014-08-13 17:38:03.349569 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.349577 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.25 192.168.1.103:6801/36920 192.168.100.103:6802/36920 2014-08-13 17:38:03.349584 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.349597 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.349614 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.349629 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.349643 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.349660 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.349674 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.349693 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.303 192.168.1.109:6803/30768 192.168.100.109:6807/30768 2014-08-13 17:38:03.349695 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.349713 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] [58,343,61] -> [61,349,362], acting primary 58 -> 61 2014-08-13 17:38:03.349744 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.349758 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.349777 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.349815 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.362 192.168.1.108:6843/8170 192.168.100.108:6853/8170 2014-08-13 17:38:03.349818 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 17:38:03.349833 7fe9a63f4700 20 osd.4 194485 22.4aa heartbeat_peers 2014-08-13 17:38:03.349837 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.349838 7fe9a63f4700 20 osd.4 194485 22.1d2 heartbeat_peers 4,61,349 2014-08-13 17:38:03.349842 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.349856 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.349871 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.349905 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 17:38:03.349923 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.349927 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.349944 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.349959 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,58,343] newacting [4,58,343] 2014-08-13 17:38:03.349974 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.349965 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.61 192.168.1.111:6841/16566 192.168.100.111:6839/16566 2014-08-13 17:38:03.349988 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [61,349,362] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'415 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.350017 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [61,349,362](61) acting [61,349,362](61)): not rw, up_thru 194483 up_from 192938 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) : primary up 192938-194483 includes interval 2014-08-13 17:38:03.350043 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 17:38:03.350064 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [61,349,362] -> [4,58,343], acting [61,349,362] -> [4,58,343], acting_primary 61 -> 4, up_primary 61 -> 4, role -1 -> 0 2014-08-13 17:38:03.350083 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.350102 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.350106 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.349 192.168.1.104:6801/61726 192.168.100.104:6803/61726 2014-08-13 17:38:03.350124 7fe9a63f4700 20 osd.4 194485 24.1de heartbeat_peers 2014-08-13 17:38:03.350117 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.350129 7fe9a63f4700 20 osd.4 194485 2.2a heartbeat_peers 4,340 2014-08-13 17:38:03.350135 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2678 2014-08-13 17:38:03.350152 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.350177 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.350192 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.350206 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.350219 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.350237 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.350254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.350268 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.350284 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.350300 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.350324 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.350338 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.350334 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.340 192.168.1.104:6807/62463 192.168.100.104:6822/62463 2014-08-13 17:38:03.350358 7fe9a63f4700 20 osd.4 194485 22.113 heartbeat_peers 4,45,339 2014-08-13 17:38:03.350352 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.350366 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.350379 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.350392 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.350408 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.350424 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.350442 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,58,343 2014-08-13 17:38:03.350436 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.352446 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.45 192.168.1.112:6829/6040 192.168.100.112:6823/6040 2014-08-13 17:38:03.353238 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.339 192.168.1.110:6819/45268 192.168.100.110:6846/45268 2014-08-13 17:38:03.353261 7fe9a63f4700 20 osd.4 194485 24.39a heartbeat_peers 2014-08-13 17:38:03.353265 7fe9a63f4700 20 osd.4 194485 24.33a heartbeat_peers 2014-08-13 17:38:03.353268 7fe9a63f4700 20 osd.4 194485 1.2b heartbeat_peers 4,340 2014-08-13 17:38:03.353272 7fe9a63f4700 20 osd.4 194485 24.34 heartbeat_peers 2014-08-13 17:38:03.353278 7fe9a63f4700 20 osd.4 194485 22.99 heartbeat_peers 2014-08-13 17:38:03.353280 7fe9a63f4700 20 osd.4 194485 22.84 heartbeat_peers 4,28,304 2014-08-13 17:38:03.353427 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.28 192.168.1.103:6808/37391 192.168.100.103:6813/37391 2014-08-13 17:38:03.353531 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.304 192.168.1.101:6809/22782 192.168.100.101:6812/22782 2014-08-13 17:38:03.353549 7fe9a63f4700 20 osd.4 194485 22.4df heartbeat_peers 4,356,357 2014-08-13 17:38:03.353642 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.356 192.168.1.108:6833/5235 192.168.100.108:6838/5235 2014-08-13 17:38:03.353735 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.357 192.168.1.102:6835/17224 192.168.100.102:6835/17224 2014-08-13 17:38:03.353751 7fe9a63f4700 20 osd.4 194485 22.39d heartbeat_peers 2014-08-13 17:38:03.353754 7fe9a63f4700 20 osd.4 194485 22.2b9 heartbeat_peers 2014-08-13 17:38:03.353757 7fe9a63f4700 20 osd.4 194485 24.2c heartbeat_peers 2014-08-13 17:38:03.353759 7fe9a63f4700 20 osd.4 194485 24.3f7 heartbeat_peers 2014-08-13 17:38:03.353761 7fe9a63f4700 20 osd.4 194485 22.775 heartbeat_peers 2014-08-13 17:38:03.353764 7fe9a63f4700 20 osd.4 194485 22.630 heartbeat_peers 2014-08-13 17:38:03.353766 7fe9a63f4700 20 osd.4 194485 24.e2 heartbeat_peers 2014-08-13 17:38:03.353768 7fe9a63f4700 20 osd.4 194485 22.69f heartbeat_peers 2014-08-13 17:38:03.353770 7fe9a63f4700 20 osd.4 194485 22.3e4 heartbeat_peers 2014-08-13 17:38:03.353772 7fe9a63f4700 20 osd.4 194485 24.2a1 heartbeat_peers 2014-08-13 17:38:03.353775 7fe9a63f4700 20 osd.4 194485 22.639 heartbeat_peers 2014-08-13 17:38:03.353777 7fe9a63f4700 20 osd.4 194485 22.707 heartbeat_peers 2014-08-13 17:38:03.353779 7fe9a63f4700 20 osd.4 194485 24.5e heartbeat_peers 2014-08-13 17:38:03.353781 7fe9a63f4700 20 osd.4 194485 22.1f1 heartbeat_peers 2014-08-13 17:38:03.353783 7fe9a63f4700 20 osd.4 194485 22.2d1 heartbeat_peers 2014-08-13 17:38:03.353786 7fe9a63f4700 20 osd.4 194485 24.b6 heartbeat_peers 2014-08-13 17:38:03.353789 7fe9a63f4700 20 osd.4 194485 24.3bf heartbeat_peers 2014-08-13 17:38:03.353791 7fe9a63f4700 20 osd.4 194485 22.1fb heartbeat_peers 2014-08-13 17:38:03.353793 7fe9a63f4700 20 osd.4 194485 24.350 heartbeat_peers 2014-08-13 17:38:03.353796 7fe9a63f4700 20 osd.4 194485 24.1f7 heartbeat_peers 2014-08-13 17:38:03.353798 7fe9a63f4700 20 osd.4 194485 24.136 heartbeat_peers 2014-08-13 17:38:03.353800 7fe9a63f4700 20 osd.4 194485 22.4fe heartbeat_peers 2014-08-13 17:38:03.353802 7fe9a63f4700 20 osd.4 194485 22.653 heartbeat_peers 2014-08-13 17:38:03.353804 7fe9a63f4700 20 osd.4 194485 24.3b7 heartbeat_peers 2014-08-13 17:38:03.353806 7fe9a63f4700 20 osd.4 194485 22.3b9 heartbeat_peers 2014-08-13 17:38:03.353808 7fe9a63f4700 20 osd.4 194485 22.791 heartbeat_peers 2014-08-13 17:38:03.353811 7fe9a63f4700 20 osd.4 194485 22.6bd heartbeat_peers 4,50,289 2014-08-13 17:38:03.353914 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.50 192.168.1.112:6843/9607 192.168.100.112:6849/9607 2014-08-13 17:38:03.354040 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.289 192.168.1.113:6809/15284 192.168.100.113:6828/15284 2014-08-13 17:38:03.354058 7fe9a63f4700 20 osd.4 194485 24.181 heartbeat_peers 2014-08-13 17:38:03.354062 7fe9a63f4700 20 osd.4 194485 24.20e heartbeat_peers 2014-08-13 17:38:03.354064 7fe9a63f4700 20 osd.4 194485 24.1e4 heartbeat_peers 2014-08-13 17:38:03.354066 7fe9a63f4700 20 osd.4 194485 24.1e6 heartbeat_peers 2014-08-13 17:38:03.354069 7fe9a63f4700 20 osd.4 194485 22.12a heartbeat_peers 2014-08-13 17:38:03.354071 7fe9a63f4700 20 osd.4 194485 22.3af heartbeat_peers 2014-08-13 17:38:03.354078 7fe9a63f4700 20 osd.4 194485 22.481 heartbeat_peers 4,287,309 2014-08-13 17:38:03.355147 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.287 192.168.1.109:6815/33435 192.168.100.109:6839/33435 2014-08-13 17:38:03.355622 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.309 192.168.1.101:6815/23703 192.168.100.101:6825/23703 2014-08-13 17:38:03.355655 7fe9a63f4700 20 osd.4 194485 22.48e heartbeat_peers 2014-08-13 17:38:03.355660 7fe9a63f4700 20 osd.4 194485 22.3ad heartbeat_peers 2014-08-13 17:38:03.355663 7fe9a63f4700 20 osd.4 194485 24.206 heartbeat_peers 2014-08-13 17:38:03.355665 7fe9a63f4700 20 osd.4 194485 24.1bb heartbeat_peers 2014-08-13 17:38:03.355668 7fe9a63f4700 20 osd.4 194485 24.299 heartbeat_peers 2014-08-13 17:38:03.355670 7fe9a63f4700 20 osd.4 194485 22.3d5 heartbeat_peers 4,286,343 2014-08-13 17:38:03.355861 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.286 192.168.1.113:6821/19173 192.168.100.113:6849/19173 2014-08-13 17:38:03.355995 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.343 192.168.1.104:6803/61960 192.168.100.104:6810/61960 2014-08-13 17:38:03.356025 7fe9a63f4700 20 osd.4 194485 22.537 heartbeat_peers 2014-08-13 17:38:03.356031 7fe9a63f4700 20 osd.4 194485 22.1b8 heartbeat_peers 4,340,347 2014-08-13 17:38:03.356150 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.347 192.168.1.106:6801/4640 192.168.100.106:6803/4640 2014-08-13 17:38:03.356172 7fe9a63f4700 20 osd.4 194485 22.29a heartbeat_peers 2014-08-13 17:38:03.356184 7fe9a63f4700 20 osd.4 194485 24.cd heartbeat_peers 2014-08-13 17:38:03.356187 7fe9a63f4700 20 osd.4 194485 22.226 heartbeat_peers 2014-08-13 17:38:03.356189 7fe9a63f4700 20 osd.4 194485 22.3d9 heartbeat_peers 2014-08-13 17:38:03.356192 7fe9a63f4700 20 osd.4 194485 24.2ec heartbeat_peers 2014-08-13 17:38:03.356194 7fe9a63f4700 20 osd.4 194485 22.1a0 heartbeat_peers 2014-08-13 17:38:03.356198 7fe9a63f4700 20 osd.4 194485 24.369 heartbeat_peers 2014-08-13 17:38:03.356201 7fe9a63f4700 10 osd.4 194485 adding neighbor peer osd.3 2014-08-13 17:38:03.356283 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.3 192.168.1.107:6841/56287 192.168.100.107:6845/56287 2014-08-13 17:38:03.356299 7fe9a63f4700 10 osd.4 194485 adding neighbor peer osd.5 2014-08-13 17:38:03.356434 7fe9a63f4700 10 osd.4 194485 _add_heartbeat_peer: new peer osd.5 192.168.1.103:6830/40192 192.168.100.103:6847/40192 2014-08-13 17:38:03.356457 7fe9a63f4700 10 osd.4 194485 maybe_update_heartbeat_peers 30 peers, extras 3,5 2014-08-13 17:38:03.356474 7fe9a63f4700 7 osd.4 194485 activate_map version 194485 2014-08-13 17:38:03.356478 7fe9a63f4700 1 osd.4 194485 pausing recovery (NORECOVER flag set) 2014-08-13 17:38:03.356472 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.356490 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356493 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356497 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.356511 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356491 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] take_waiters 2014-08-13 17:38:03.356527 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356509 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.356531 7fe9a77f6700 20 osd.4 194485 _dispatch 0x272a640 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.356530 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Reset 0.307145 9 0.002886 2014-08-13 17:38:03.356539 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.683882 9 0.002149 2014-08-13 17:38:03.356556 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started 2014-08-13 17:38:03.356567 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.356577 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Start 2014-08-13 17:38:03.356586 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.356491 7fe9986dd700 20 osd.4 194485 agent_entry empty queue 2014-08-13 17:38:03.356593 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: transitioning to Primary 2014-08-13 17:38:03.356602 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.356616 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Start 0.000039 0 0.000000 2014-08-13 17:38:03.356622 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 2014-08-13 17:38:03.356635 7fe9a77f6700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.356645 7fe9a77f6700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.356651 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356654 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356637 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary 2014-08-13 17:38:03.356646 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.356661 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering 2014-08-13 17:38:03.356677 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356671 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.356686 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356689 7fe9a63f4700 20 osd.4 194485 _dispatch 0x29985a0 pg_query(22.14a epoch 194485) v3 2014-08-13 17:38:03.356681 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.356690 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.356701 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.356709 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.356723 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 17:38:03.356730 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 17:38:03.356753 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.58 epoch 194485 2014-08-13 17:38:03.356758 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x29985a0 2014-08-13 17:38:03.356751 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 17:38:03.356753 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) 2014-08-13 17:38:03.356770 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194480-194481 up [286,375](286) acting [375,10,301](375) maybe_went_rw) 2014-08-13 17:38:03.356773 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [58,343](58) acting [58,343,61](58) maybe_went_rw) 2014-08-13 17:38:03.356779 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 58 2014-08-13 17:38:03.356792 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194479-194479 up [286,375](286) acting [375,10](375)) 2014-08-13 17:38:03.356794 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [58,343](58) acting [58,343](58)) 2014-08-13 17:38:03.356815 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194477-194478 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) 2014-08-13 17:38:03.356819 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) 2014-08-13 17:38:03.356841 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356845 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356836 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194476-194476 up [4,286,375](4) acting [375,10,301](375)) 2014-08-13 17:38:03.356839 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,58,343](4) acting [58,343,61](58)) 2014-08-13 17:38:03.356858 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.356872 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.356876 7fe9a77f6700 20 osd.4 194485 _dispatch 0x5411440 osd_map(194485..194485 src has 185139..194485) v3 2014-08-13 17:38:03.356858 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194474-194475 up [286,375](286) acting [375,10,301](375) maybe_went_rw) 2014-08-13 17:38:03.356859 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [58,343](58) acting [58,343,61](58) maybe_went_rw) 2014-08-13 17:38:03.356886 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194473-194473 up [286,375](286) acting [375,10](375)) 2014-08-13 17:38:03.356890 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [58,343](58) acting [58,343](58)) 2014-08-13 17:38:03.356908 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 17:38:03.356911 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,343,349,362 down blocked_by {} 2014-08-13 17:38:03.356927 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.356930 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.356952 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] publish_stats_to_osd 194485:30293 2014-08-13 17:38:03.356956 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2679 2014-08-13 17:38:03.356973 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.10 2014-08-13 17:38:03.356975 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 17:38:03.356993 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.286 2014-08-13 17:38:03.356995 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 17:38:03.357010 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.301 2014-08-13 17:38:03.357013 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 17:38:03.357028 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.375 2014-08-13 17:38:03.357030 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 17:38:03.357052 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 17:38:03.357053 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.357071 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.357072 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.357092 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.357205 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.357223 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.357238 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru now 194485 (was 0), currently 194477 2014-08-13 17:38:03.357245 7fe99f5e9700 10 osd.4 194485 send_alive up_thru currently 194477 want 194485 2014-08-13 17:38:03.357248 7fe99f5e9700 10 osd.4 194485 send_alive want 194485 2014-08-13 17:38:03.357251 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.357265 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:03.357269 7fe99ebe8700 7 osd.4 194485 do_notify osd 351 on 1 PGs 2014-08-13 17:38:03.357284 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:03.357288 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.10 on 1 PGs 2014-08-13 17:38:03.357352 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:03.357386 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.58 on 1 PGs 2014-08-13 17:38:03.357434 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:03.357438 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.61 on 1 PGs 2014-08-13 17:38:03.357474 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 don't know epoch, doing nothing 2014-08-13 17:38:03.357477 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.343 on 1 PGs 2014-08-13 17:38:03.357498 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5380c60 don't know epoch, doing nothing 2014-08-13 17:38:03.357501 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.349 on 1 PGs 2014-08-13 17:38:03.357523 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:03.357534 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.362 on 1 PGs 2014-08-13 17:38:03.357531 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:03.357541 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.286 on 1 PGs 2014-08-13 17:38:03.357569 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing 2014-08-13 17:38:03.357572 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.301 on 1 PGs 2014-08-13 17:38:03.357590 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing 2014-08-13 17:38:03.357594 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.375 on 1 PGs 2014-08-13 17:38:03.357619 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 17:38:03.357666 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.357677 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.357675 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 17:38:03.357704 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.357695 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292] newacting [9,292] 2014-08-13 17:38:03.357712 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.357712 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.357730 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.357745 7fe99f5e9700 1 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.357763 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.191187 2 0.000157 2014-08-13 17:38:03.357739 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.192367 3 0.000173 2014-08-13 17:38:03.357787 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.357786 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.192474 0 0.000000 2014-08-13 17:38:03.357804 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.191236 0 0.000000 2014-08-13 17:38:03.357807 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.357823 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194477 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.357824 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.191266 0 0.000000 2014-08-13 17:38:03.357841 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194479 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.357844 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started 0.191324 0 0.000000 2014-08-13 17:38:03.357867 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.357857 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194479 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.357881 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194477 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.357888 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194479 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292] newacting [9,292] 2014-08-13 17:38:03.357898 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194479 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.357910 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194479 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.357917 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194479 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.357927 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292,4] r=2 lpr=194479 pi=192972-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.357935 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194479 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [28,304] newacting [28,304] 2014-08-13 17:38:03.357950 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194479 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.357964 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [4,28,304] r=0 lpr=194479 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.357962 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [9,292,4](9) acting [9,292,4](9)): not rw, up_thru 194476 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [9,292,4](9) acting [9,292,4](9) maybe_went_rw) : primary up 192972-194476 includes interval 2014-08-13 17:38:03.357990 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194476/194476/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [9,292,4](9) acting [9,292,4](9) maybe_went_rw) 2014-08-13 17:38:03.357997 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,28,304](4) acting [4,28,304](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.358009 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292,4] -> [9,292], acting [9,292,4] -> [9,292], acting_primary 9 -> 9, up_primary 9 -> 9, role 2 -> -1 2014-08-13 17:38:03.358014 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/194476) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] noting past interval(194476-194478 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 17:38:03.358030 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.358033 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] up [4,28,304] -> [28,304], acting [4,28,304] -> [28,304], acting_primary 4 -> 28, up_primary 4 -> 28, role 0 -> -1 2014-08-13 17:38:03.358052 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.358071 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.358046 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.358085 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.358096 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.358100 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.358117 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.358116 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.358138 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.358138 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.358154 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.358206 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.358215 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.358248 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.358234 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.358267 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.358282 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.358296 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.358314 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.358302 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.358328 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.358333 7fe9a77f6700 3 osd.4 194485 handle_osd_map epochs [194485,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358346 7fe9a77f6700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358351 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358341 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.358354 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358356 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.358338 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.358369 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358369 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.358382 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358385 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db3600 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358375 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.358387 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358397 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.358412 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.358429 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.358435 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 17:38:03.358457 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358446 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358463 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358478 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358487 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 17:38:03.358508 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358498 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358514 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358529 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358536 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 17:38:03.358556 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358545 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358560 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358573 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358575 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358586 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358590 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358593 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358590 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358589 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 17:38:03.358611 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358616 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358629 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358632 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358634 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9b180 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358630 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358642 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358646 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358649 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358652 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358638 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 17:38:03.358647 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.358657 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358665 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358679 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358698 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358683 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 17:38:03.358701 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358707 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.358694 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] acting_up_affected newup [28,7,304] newacting [28,7,304] 2014-08-13 17:38:03.358707 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5410480 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358714 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358719 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358722 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358723 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358712 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.358712 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.358730 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.358730 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,304] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'337 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.358750 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358753 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358755 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d99680 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358748 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292,362] newacting [9,292,362] 2014-08-13 17:38:03.358761 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358766 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358768 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358770 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358764 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.358769 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [28,304](28) acting [28,304](28)): not rw, up_thru 194479 up_from 192966 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) : primary up 192966-194479 includes interval 2014-08-13 17:38:03.358791 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358793 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358780 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292] r=-1 lpr=194479 pi=192972-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.358796 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9e780 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358801 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358789 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 17:38:03.358805 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358807 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358810 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358807 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] up [28,304] -> [28,7,304], acting [28,304] -> [28,7,304], acting_primary 28 -> 28, up_primary 28 -> 28, role -1 -> -1 2014-08-13 17:38:03.358826 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.358843 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358846 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358848 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db7980 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358832 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [9,292](9) acting [9,292](9)): not rw, up_thru 194479 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [9,292](9) acting [9,292](9) maybe_went_rw) : primary up 192972-194479 includes interval 2014-08-13 17:38:03.358853 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358842 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.358859 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358861 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358863 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358852 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194479/194479/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [9,292](9) acting [9,292](9) maybe_went_rw) 2014-08-13 17:38:03.358860 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.358884 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358886 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358878 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.358888 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9f980 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358898 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358903 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358904 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358907 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358894 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.358910 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.358928 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358930 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358932 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9b840 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358924 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.358937 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358941 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358943 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358945 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358940 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.358965 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358968 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358970 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9c140 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.358962 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.358975 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.358980 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.358982 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.358983 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358897 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292] -> [9,292,362], acting [9,292] -> [9,292,362], acting_primary 9 -> 9, up_primary 9 -> 9, role -1 -> -1 2014-08-13 17:38:03.358979 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.358993 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.359006 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359009 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.358990 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359011 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d8f080 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359017 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359023 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359010 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359025 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359027 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359046 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359048 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359050 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9e300 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359055 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359059 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359061 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359063 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359015 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.359060 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 17:38:03.359081 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359084 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359087 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359089 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d98900 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359078 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359094 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359098 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359101 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359086 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359102 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359103 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359123 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359125 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359127 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d8f2c0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359102 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.359132 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359121 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359138 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359140 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359142 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359162 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359164 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359165 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9e0c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359136 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.359185 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359190 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359194 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359195 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359169 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 17:38:03.359200 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359188 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.359219 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359204 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359225 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359206 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359230 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db1b00 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359236 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359241 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359227 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359243 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359245 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359235 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.359256 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359246 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359264 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359254 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.359267 7fe9a77f6700 20 osd.4 194485 _dispatch 0x5411200 osd_map(194485..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359277 7fe9a77f6700 3 osd.4 194485 handle_osd_map epochs [194485,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359270 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.359283 7fe9a77f6700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359288 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359291 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359288 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.359287 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 17:38:03.359309 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359306 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359318 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359307 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359313 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359321 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d92f40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359330 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359339 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359346 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359349 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359352 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359345 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,28,304] newacting [4,28,304] 2014-08-13 17:38:03.359349 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 17:38:03.359371 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359363 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.359385 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359388 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359376 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359378 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [28,7,304] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'337 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.359391 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db4c80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359397 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359403 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359406 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359393 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359408 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359410 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359412 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [28,7,304](28) acting [28,7,304](28)): not rw, up_thru 194482 up_from 192966 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) : primary up 192966-194482 includes interval 2014-08-13 17:38:03.359432 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359434 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359436 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db2880 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359441 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359429 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/194479) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 17:38:03.359445 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359450 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359452 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359444 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 17:38:03.359463 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359447 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [28,7,304] -> [4,28,304], acting [28,7,304] -> [4,28,304], acting_primary 28 -> 4, up_primary 28 -> 4, role -1 -> 0 2014-08-13 17:38:03.359474 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359476 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359478 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d986c0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359467 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359469 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.359484 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359493 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359495 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359498 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359483 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359493 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.359502 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359518 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359520 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359522 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9fbc0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359515 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359527 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359532 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359534 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359536 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359533 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2899 2014-08-13 17:38:03.359532 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 17:38:03.359556 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.359559 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359561 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359564 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d99f80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359554 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359569 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359573 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359560 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.359577 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359579 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359572 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.359579 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.359599 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359589 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.359603 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359606 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d99440 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359612 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359615 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359618 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359604 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359620 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359602 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292,4] newacting [9,292,4] 2014-08-13 17:38:03.359621 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359622 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.359640 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359643 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359644 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db72c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359636 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359649 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359639 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,362] r=-1 lpr=194482 pi=192972-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.359654 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359656 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359658 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359652 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.359679 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359667 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359682 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359684 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d99d40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359672 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [9,292,362](9) acting [9,292,362](9)): not rw, up_thru 194482 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [9,292,362](9) acting [9,292,362](9) maybe_went_rw) : primary up 192972-194482 includes interval 2014-08-13 17:38:03.359690 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359694 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359696 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359698 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359684 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.359691 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194482/194482/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [9,292,362](9) acting [9,292,362](9) maybe_went_rw) 2014-08-13 17:38:03.359703 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.359721 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359724 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359726 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db5e80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359710 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292,362] -> [9,292,4], acting [9,292,362] -> [9,292,4], acting_primary 9 -> 9, up_primary 9 -> 9, role -1 -> 2 2014-08-13 17:38:03.359731 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359719 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.359737 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359739 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359743 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359731 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359738 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.359763 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359765 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359754 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.359767 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9a1c0 osd_map(194472..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359773 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194472,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359777 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359779 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359768 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.359781 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359749 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.359783 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.359789 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359805 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359807 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359810 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9ec00 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359798 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.359816 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359820 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359823 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359825 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359817 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.359807 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.359844 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359846 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359848 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d8f980 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359835 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.359835 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.359855 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359859 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359862 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359863 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359858 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,28,304 unchanged 2014-08-13 17:38:03.359855 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.359884 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359886 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359875 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.359888 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d93180 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359876 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.359896 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359903 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359905 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359908 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359894 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002028 7 0.001799 2014-08-13 17:38:03.359898 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.359927 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359929 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359916 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.359931 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54d8900 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359936 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359940 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359942 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359944 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359933 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.359916 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.359949 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.359963 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359950 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.359966 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359968 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90fc0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.359975 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.359979 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.359981 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.359983 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359968 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.359969 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.359987 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.360003 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360005 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.359992 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.360008 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5430000 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360013 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360001 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.360018 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360020 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360022 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360011 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.360041 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360029 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.360043 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360031 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.360046 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db3cc0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360054 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360059 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360045 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.360061 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360063 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360053 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.360081 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360083 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360065 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 17:38:03.360084 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9e9c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360089 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360093 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360095 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360097 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360087 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 17:38:03.360118 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360120 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360106 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 17:38:03.360122 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9e540 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360127 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360131 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360133 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360135 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360124 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,28,304 down blocked_by {} 2014-08-13 17:38:03.360143 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.360167 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360169 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.360171 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db57c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360176 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360180 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360182 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360161 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2900 2014-08-13 17:38:03.360184 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360172 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.360189 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.7 2014-08-13 17:38:03.360216 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360201 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.002394 7 0.002101 2014-08-13 17:38:03.360219 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360208 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.28 2014-08-13 17:38:03.360221 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90240 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360230 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360234 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360236 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360239 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360228 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.304 2014-08-13 17:38:03.360234 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.360261 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360264 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360247 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.360253 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.360266 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9ba80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360272 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360277 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360279 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360280 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360269 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.360270 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.360300 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360302 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360304 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db6c00 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360310 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360314 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360316 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360294 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000042 0 0.000000 2014-08-13 17:38:03.360317 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360340 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.360341 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360328 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.360344 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360346 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db0b40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360351 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360356 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360357 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360359 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360351 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.360378 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360381 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360383 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db6540 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.360388 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360399 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360401 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360403 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360394 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_advance_map [356,357]/[356,357] -- 356/356 2014-08-13 17:38:03.360423 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360421 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.360424 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360426 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5458000 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360432 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360436 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360438 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360439 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360428 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.360458 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360460 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360450 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.360461 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5498000 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360467 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360471 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360473 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360474 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360474 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.360465 7fe99f5e9700 1 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.360498 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360484 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.193582 2 0.000220 2014-08-13 17:38:03.360500 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360502 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d918c0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360507 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360511 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360513 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360515 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360504 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.360521 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.193628 0 0.000000 2014-08-13 17:38:03.360535 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360537 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360539 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db0fc0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360544 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360548 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360550 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360552 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360540 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.193656 0 0.000000 2014-08-13 17:38:03.360572 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360559 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started 0.193718 0 0.000000 2014-08-13 17:38:03.360574 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360577 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d972c0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360583 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360587 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360589 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360577 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.360591 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360572 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 17:38:03.360602 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.360594 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194477 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.360612 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360615 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360616 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9f2c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360608 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.360621 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360611 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194479 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.360626 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360629 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360631 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360625 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28] newacting [58,28] 2014-08-13 17:38:03.360628 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194479 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.360651 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360653 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360655 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9cec0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360645 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194479 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [356,357] newacting [356,357] 2014-08-13 17:38:03.360662 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360667 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360669 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360670 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360663 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194479 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.360642 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.360690 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360692 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360679 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [4,356,357] r=0 lpr=194479 pi=193306-194475/23 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.360694 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d969c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360682 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.194282 3 0.000161 2014-08-13 17:38:03.360700 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360704 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360706 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360708 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360704 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.194357 0 0.000000 2014-08-13 17:38:03.360726 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360729 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360730 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db45c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360715 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,356,357](4) acting [4,356,357](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.360736 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360725 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.360742 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360744 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360746 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360736 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194476/194476/194476) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] noting past interval(194476-194478 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 17:38:03.360743 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.360764 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360766 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360768 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d91b00 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360756 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] up [4,356,357] -> [356,357], acting [4,356,357] -> [356,357], acting_primary 4 -> 356, up_primary 4 -> 356, role 0 -> -1 2014-08-13 17:38:03.360761 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.360774 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360780 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360782 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360784 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360776 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.360778 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.360806 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360793 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.360810 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360813 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5411680 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360800 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28] newacting [58,28] 2014-08-13 17:38:03.360818 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360823 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360825 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360812 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.360827 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360821 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.360829 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.360848 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360851 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360852 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9f080 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360845 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.360857 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360862 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360864 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360866 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360840 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.360861 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.360885 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360888 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360878 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.360890 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9ee40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360895 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360899 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360901 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360903 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360890 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [58,28,4](58) acting [58,28,4](58)): not rw, up_thru 194476 up_from 192935 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [58,28,4](58) acting [58,28,4](58) maybe_went_rw) : primary up 192935-194476 includes interval 2014-08-13 17:38:03.360892 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.360923 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360909 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.360907 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [58,28,4](58) acting [58,28,4](58) maybe_went_rw) 2014-08-13 17:38:03.360925 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360927 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d95340 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360933 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360937 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360940 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360942 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360928 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.360926 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28,4] -> [58,28], acting [58,28,4] -> [58,28], acting_primary 58 -> 58, up_primary 58 -> 58, role 2 -> -1 2014-08-13 17:38:03.360945 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.360948 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.360966 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360968 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360970 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9c5c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.360960 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.360976 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.360965 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.360981 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.360984 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.360986 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.360976 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.360982 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.360992 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.361006 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361008 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361011 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db21c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361001 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.361016 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361022 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361008 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.361024 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361030 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361017 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.361025 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.361034 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.361049 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361052 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361040 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.361054 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db0d80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361060 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361065 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361051 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.361066 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361069 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361058 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361070 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.361087 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361089 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361091 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9c380 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361096 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361100 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361102 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361104 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361086 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.361099 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [356,357]/[356,357,371] -- 356/356 2014-08-13 17:38:03.361121 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361128 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361130 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361119 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.361132 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5490480 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361138 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361143 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361144 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361146 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361130 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361134 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.361151 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.361168 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361171 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361160 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.361174 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db33c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361180 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361177 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.361184 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361193 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361169 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] acting_up_affected newup [356,357] newacting [356,357,371] 2014-08-13 17:38:03.361195 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361194 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361197 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.361215 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361217 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361219 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d91200 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361213 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'433 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.361224 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361229 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361233 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361234 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361253 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361255 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361238 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 17:38:03.361257 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9c800 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361260 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361244 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [356,357](356) acting [356,357](356)): not rw, up_thru 194477 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [356,357](356) acting [356,357](356)) : primary up 192969-194477 does not include interval 2014-08-13 17:38:03.361263 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361268 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361270 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361272 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361264 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361263 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194479/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [356,357](356) acting [356,357](356)) 2014-08-13 17:38:03.361282 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.361300 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361285 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] up [356,357] -> [356,357], acting [356,357] -> [356,357,371], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.361302 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361306 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9da00 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361297 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361313 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361317 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361319 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361320 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361305 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.361323 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.361341 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361343 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361345 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5461440 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361338 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.361351 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361334 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 17:38:03.361357 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361359 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361359 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361362 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361356 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.361364 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361380 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361383 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361385 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9cc80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361374 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.361391 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361395 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361397 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361399 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361381 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.361392 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.361404 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361421 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361423 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361411 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.361425 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90d80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361430 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361434 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361437 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361438 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361428 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.361457 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361445 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.361460 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361446 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 17:38:03.361462 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90900 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361471 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361475 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361462 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.361476 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361469 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361478 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361480 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.361482 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361499 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361502 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361505 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d91680 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361510 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361498 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.361514 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361516 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361498 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361518 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361516 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28,284] newacting [58,28,284] 2014-08-13 17:38:03.361551 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361554 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361555 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90b40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361542 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [356,357]/[356,357,371] -- 356/356 2014-08-13 17:38:03.361561 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361533 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.361564 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361566 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361568 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361570 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361568 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.361569 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361590 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361593 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361595 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54806c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361600 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361588 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.361604 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361606 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361607 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361607 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361630 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361613 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [58,28](58) acting [58,28](58)): not rw, up_thru 194480 up_from 192935 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [58,28](58) acting [58,28](58) maybe_went_rw) : primary up 192935-194480 includes interval 2014-08-13 17:38:03.361632 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361634 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d96c00 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361640 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361644 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361645 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361647 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361634 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [58,28](58) acting [58,28](58) maybe_went_rw) 2014-08-13 17:38:03.361646 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [352,371,380]/[356,357,371] -- 352/356 2014-08-13 17:38:03.361667 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361666 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361669 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361671 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d97bc0 osd_map(194472..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361677 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194472,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361682 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361663 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28] -> [58,28,284], acting [58,28] -> [58,28,284], acting_primary 58 -> 58, up_primary 58 -> 58, role -1 -> -1 2014-08-13 17:38:03.361683 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361686 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361671 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361685 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.361689 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.361708 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361710 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361712 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5481440 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361701 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.361719 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361706 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] acting_up_affected newup [352,371,380] newacting [356,357,371] 2014-08-13 17:38:03.361723 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361726 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361728 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361718 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.361725 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.361747 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361749 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361751 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5481200 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361742 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.361742 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [356,357]/[356,357,371] r=-1 lpr=194480 pi=193306-194479/25 crt=191727'433 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.361757 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361761 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361763 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361765 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361758 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.361783 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361786 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361788 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5480900 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361774 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.361775 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [356,357](356) acting [356,357,371](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [356,357](356) acting [356,357,371](356) maybe_went_rw) : primary up 192969-194480 includes interval 2014-08-13 17:38:03.361794 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361798 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361800 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361803 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361797 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194479/194480/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [356,357](356) acting [356,357,371](356) maybe_went_rw) 2014-08-13 17:38:03.361822 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361825 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361827 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5480d80 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361797 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.361832 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361818 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] up [356,357] -> [352,371,380], acting [356,357,371] -> [356,357,371], acting_primary 356 -> 356, up_primary 356 -> 352, role -1 -> -1 2014-08-13 17:38:03.361837 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361839 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361842 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361838 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.361835 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.361863 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361865 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361867 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54d86c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361856 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.361858 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.361873 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361877 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361880 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361882 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361874 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.361875 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.361901 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361903 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361905 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54d8480 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361892 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.361892 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.361911 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361916 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361918 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361919 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361910 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.361912 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.361938 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361927 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.361941 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361943 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5480fc0 osd_map(194472..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361948 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194472,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361953 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361956 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361942 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.361957 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361960 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.361958 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 17:38:03.361977 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361983 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361982 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.361985 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54daf40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.361991 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.361979 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.361995 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.361997 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.361999 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.361988 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.361996 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.362006 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.362021 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362023 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362025 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54dd580 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362016 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.362031 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362024 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362040 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362042 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362044 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362035 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362061 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362064 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362065 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54dd340 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362070 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362074 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362076 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362077 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362071 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 17:38:03.362091 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.362095 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362098 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362100 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54dd7c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362105 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362109 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362095 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.362111 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362113 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362096 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [352,371,380]/[352,371,380] -- 352/352 2014-08-13 17:38:03.362123 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.362111 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.362133 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362135 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362137 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54dec00 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362142 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362128 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.362147 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362149 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362151 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362129 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362148 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.362169 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362172 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362173 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54de9c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362179 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362183 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362164 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] acting_up_affected newup [352,371,380] newacting [352,371,380] 2014-08-13 17:38:03.362192 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362195 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362178 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 17:38:03.362199 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.362193 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.362203 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.362217 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362220 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362222 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54de300 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362211 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380]/[356,357,371] r=-1 lpr=194482 pi=193306-194481/26 crt=191727'433 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.362227 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362232 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362234 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362219 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.362236 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362237 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28,4] newacting [58,28,4] 2014-08-13 17:38:03.362260 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362244 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) : primary up 192969-194480 does not include interval 2014-08-13 17:38:03.362264 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362267 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db4380 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.362254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.362273 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.362278 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.362280 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362282 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362265 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194482/194479) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 17:38:03.362272 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,284] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.362302 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362286 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] up [352,371,380] -> [352,371,380], acting [356,357,371] -> [352,371,380], acting_primary 356 -> 352, up_primary 352 -> 352, role -1 -> -1 2014-08-13 17:38:03.362304 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362306 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260700 pg_notify(22.1d2(27) epoch 194485) v5 2014-08-13 17:38:03.362305 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.362305 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [58,28,284](58) acting [58,28,284](58)): not rw, up_thru 194482 up_from 192935 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [58,28,284](58) acting [58,28,284](58) maybe_went_rw) : primary up 192935-194482 includes interval 2014-08-13 17:38:03.362322 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.362323 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [58,28,284](58) acting [58,28,284](58) maybe_went_rw) 2014-08-13 17:38:03.362340 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.362362 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.58 2014-08-13 17:38:03.362348 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28,284] -> [58,28,4], acting [58,28,284] -> [58,28,4], acting_primary 58 -> 58, up_primary 58 -> 58, role -1 -> 2 2014-08-13 17:38:03.362366 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260700 2014-08-13 17:38:03.362357 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.362369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.362375 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.362391 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.362386 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.362408 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.362421 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362410 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.362424 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362425 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.362427 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.362451 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362453 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362456 7fe9a63f4700 20 osd.4 194485 _dispatch 0x29990e0 pg_query(24.33a epoch 194485) v3 2014-08-13 17:38:03.362441 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.362446 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.362462 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.362460 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] [356,357,371] -> [352,371,380], acting primary 356 -> 352 2014-08-13 17:38:03.362494 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.304 epoch 194485 2014-08-13 17:38:03.362479 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.362497 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x29990e0 2014-08-13 17:38:03.362490 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.362501 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.362506 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.362511 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(info 0'0) from replica 304 2014-08-13 17:38:03.362518 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.362527 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362557 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362535 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.362560 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362562 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.362582 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362585 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362569 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [352,371,380]/[352,371,380] -- 352/352 2014-08-13 17:38:03.362587 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299a760 pg_query(22.28f epoch 194485) v3 2014-08-13 17:38:03.362576 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.362590 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.362592 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.362595 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.362610 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.362609 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362627 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.9 epoch 194485 2014-08-13 17:38:03.362632 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299a760 2014-08-13 17:38:03.362632 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.362644 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 9 2014-08-13 17:38:03.362634 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.362662 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 17:38:03.362680 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362681 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.362683 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362685 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.362701 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362704 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362706 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca840 pg_notify(22.1b8(2) epoch 194485) v5 2014-08-13 17:38:03.362700 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.362716 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,356,357] newacting [4,356,357] 2014-08-13 17:38:03.362745 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.340 2014-08-13 17:38:03.362734 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.362748 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca840 2014-08-13 17:38:03.362749 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [352,371,380] r=-1 lpr=194483 pi=193306-194482/27 crt=191727'433 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.362705 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.362768 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.362789 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362793 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362778 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [352,371,380](352) acting [352,371,380](352)): not rw, up_thru 194483 up_from 192964 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) : primary up 192964-194483 includes interval 2014-08-13 17:38:03.362789 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.002065 7 0.001708 2014-08-13 17:38:03.362812 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.362797 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194482/194483/194483) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 17:38:03.362814 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.362818 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299a3a0 pg_query(24.1e4 epoch 194485) v3 2014-08-13 17:38:03.362818 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.363358 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.31 epoch 194485 2014-08-13 17:38:03.363362 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299a3a0 2014-08-13 17:38:03.363374 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(info 0'0) from replica 31 2014-08-13 17:38:03.363415 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363418 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363440 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363442 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363444 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51caf40 pg_notify(22.84(2) epoch 194485) v5 2014-08-13 17:38:03.363479 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.28 2014-08-13 17:38:03.363482 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51caf40 2014-08-13 17:38:03.363518 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363521 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363539 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363541 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363543 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265400 pg_notify(24.136(30) epoch 194485) v5 2014-08-13 17:38:03.363577 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.356 2014-08-13 17:38:03.363579 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265400 2014-08-13 17:38:03.363614 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363620 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363638 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363640 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363641 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299b660 pg_query(22.2b9 epoch 194485) v3 2014-08-13 17:38:03.363677 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.328 epoch 194485 2014-08-13 17:38:03.363679 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299b660 2014-08-13 17:38:03.363690 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 328 2014-08-13 17:38:03.363762 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363765 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363783 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363785 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363786 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5100000 pg_notify(24.30b(1) epoch 194485) v5 2014-08-13 17:38:03.362839 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.362816 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [352,371,380] -> [4,356,357], acting [352,371,380] -> [4,356,357], acting_primary 352 -> 4, up_primary 352 -> 4, role -1 -> 0 2014-08-13 17:38:03.363832 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.301 2014-08-13 17:38:03.363834 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5100000 2014-08-13 17:38:03.363819 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.363820 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.363872 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363876 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363860 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.363866 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.001027 0 0.000000 2014-08-13 17:38:03.363880 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.363894 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363898 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363900 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299ad00 pg_query(22.30f epoch 194485) v3 2014-08-13 17:38:03.363900 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2655 2014-08-13 17:38:03.363900 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.363919 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.363938 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.54 epoch 194485 2014-08-13 17:38:03.363941 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299ad00 2014-08-13 17:38:03.363930 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.363939 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.363949 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.363957 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.363953 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 54 2014-08-13 17:38:03.363973 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.363994 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.363997 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.363993 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.364020 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364008 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.364022 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364024 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42601c0 pg_notify(24.9b(33) epoch 194485) v5 2014-08-13 17:38:03.364023 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.364037 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.364040 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.364062 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.291 2014-08-13 17:38:03.364065 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42601c0 2014-08-13 17:38:03.364058 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.364075 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.364090 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.364104 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364107 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364105 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.364125 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364127 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364129 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299b480 pg_query(22.791 epoch 194485) v3 2014-08-13 17:38:03.364121 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.364136 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.364151 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.364166 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.30 epoch 194485 2014-08-13 17:38:03.364169 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299b480 2014-08-13 17:38:03.364166 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.364106 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 17:38:03.364188 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.364181 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 30 2014-08-13 17:38:03.364186 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.364195 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.364215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.364230 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364232 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364213 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.364235 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,356,357 unchanged 2014-08-13 17:38:03.364251 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364237 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.364254 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364257 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2999e00 pg_query(24.1e6 epoch 194485) v3 2014-08-13 17:38:03.364254 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.364258 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.196867 2 0.000204 2014-08-13 17:38:03.364272 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.003695 7 0.003478 2014-08-13 17:38:03.364294 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.285 epoch 194485 2014-08-13 17:38:03.364280 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.364297 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2999e00 2014-08-13 17:38:03.364294 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.364298 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.196917 0 0.000000 2014-08-13 17:38:03.364308 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(info 0'0) from replica 285 2014-08-13 17:38:03.364317 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.364320 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.196949 0 0.000000 2014-08-13 17:38:03.364333 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.364346 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364353 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364341 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started 0.197016 0 0.000000 2014-08-13 17:38:03.364355 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 2014-08-13 17:38:03.364371 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364361 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.364376 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364378 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2998000 pg_query(22.29a epoch 194485) v3 2014-08-13 17:38:03.364377 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194477 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.364375 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.364398 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.364416 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.345 epoch 194485 2014-08-13 17:38:03.364419 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2998000 2014-08-13 17:38:03.364415 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.364396 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194479 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.364432 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.364431 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 345 2014-08-13 17:38:03.364436 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194479 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.364453 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194479 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [301,324] newacting [301,324] 2014-08-13 17:38:03.364471 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364474 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364453 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 17:38:03.364473 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194479 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.364496 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364498 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364482 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 17:38:03.364500 7fe9a63f4700 20 osd.4 194485 _dispatch 0x50f8380 pg_notify(24.3d(1),22.99(27) epoch 194485) v5 2014-08-13 17:38:03.364490 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [4,301,324] r=0 lpr=194479 pi=193306-194475/23 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.364501 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [356,357](356) acting [356,357,371](356) maybe_went_rw) 2014-08-13 17:38:03.364525 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [356,357](356) acting [356,357](356)) 2014-08-13 17:38:03.364547 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.20 2014-08-13 17:38:03.364550 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x50f8380 2014-08-13 17:38:03.364541 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,301,324](4) acting [4,301,324](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194465 generate_past_intervals interval(194476-194478 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.364545 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 17:38:03.364564 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194476/194476/194476) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] noting past interval(194476-194478 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 17:38:03.364566 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [356,357](356) acting [356,357](356) maybe_went_rw) 2014-08-13 17:38:03.364584 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] up [4,301,324] -> [301,324], acting [4,301,324] -> [301,324], acting_primary 4 -> 301, up_primary 4 -> 301, role 0 -> -1 2014-08-13 17:38:03.364601 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364587 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 17:38:03.364604 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364604 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.364605 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,352,356,357,371,380 down blocked_by {} 2014-08-13 17:38:03.364624 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364627 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364629 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2998d20 pg_query(22.4aa epoch 194485) v3 2014-08-13 17:38:03.364625 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.364623 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.364647 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2656 2014-08-13 17:38:03.364664 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.321 epoch 194485 2014-08-13 17:38:03.364668 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2998d20 2014-08-13 17:38:03.364655 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.364667 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.352 2014-08-13 17:38:03.364675 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.364680 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 321 2014-08-13 17:38:03.364688 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.356 2014-08-13 17:38:03.364696 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.364717 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364706 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.357 2014-08-13 17:38:03.364720 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364721 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.364724 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.371 2014-08-13 17:38:03.364740 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364742 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364745 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299c920 pg_query(24.2ec epoch 194485) v3 2014-08-13 17:38:03.364736 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.364741 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.380 2014-08-13 17:38:03.364752 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.364761 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.364785 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.374 epoch 194485 2014-08-13 17:38:03.364769 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.364788 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299c920 2014-08-13 17:38:03.364779 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.364792 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.364800 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_query query(info 0'0) from replica 374 2014-08-13 17:38:03.364822 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.364838 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364841 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364837 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.364859 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364862 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364864 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264980 pg_notify(22.3af(1) epoch 194485) v5 2014-08-13 17:38:03.364858 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.364883 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.364872 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.364888 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.364906 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.364923 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.364928 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264980 2014-08-13 17:38:03.364919 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.364969 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364971 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364967 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5250580 don't know epoch, doing nothing 2014-08-13 17:38:03.364942 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.364979 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.7 on 1 PGs 2014-08-13 17:38:03.364991 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.364993 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.364994 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2998f00 pg_query(22.630 epoch 194485) v3 2014-08-13 17:38:03.365010 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:03.365013 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.28 on 1 PGs 2014-08-13 17:38:03.365029 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.294 epoch 194485 2014-08-13 17:38:03.365032 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2998f00 2014-08-13 17:38:03.365004 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 17:38:03.365043 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 294 2014-08-13 17:38:03.365064 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.365082 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365084 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365070 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.365102 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365091 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.365104 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365106 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2999c20 pg_query(24.1de epoch 194485) v3 2014-08-13 17:38:03.365107 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.365134 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:03.365142 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.364 epoch 194485 2014-08-13 17:38:03.365143 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.304 on 1 PGs 2014-08-13 17:38:03.365145 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2999c20 2014-08-13 17:38:03.365169 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5385960 don't know epoch, doing nothing 2014-08-13 17:38:03.365173 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.352 on 1 PGs 2014-08-13 17:38:03.365156 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_query query(info 0'0) from replica 364 2014-08-13 17:38:03.365169 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 17:38:03.365209 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365212 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365217 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:03.365222 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.356 on 1 PGs 2014-08-13 17:38:03.365226 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.365230 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365233 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365235 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c88c0 pg_notify(22.3e4(31) epoch 194485) v5 2014-08-13 17:38:03.365243 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5431600 don't know epoch, doing nothing 2014-08-13 17:38:03.365246 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.357 sending saperate messages on 1 PGs 2014-08-13 17:38:03.365264 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:03.365267 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.371 on 1 PGs 2014-08-13 17:38:03.365272 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.45 2014-08-13 17:38:03.365275 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c88c0 2014-08-13 17:38:03.365232 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.365287 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5250dc0 don't know epoch, doing nothing 2014-08-13 17:38:03.365290 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.380 on 1 PGs 2014-08-13 17:38:03.365286 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.365304 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.365318 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365321 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365339 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365341 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365343 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc980 pg_notify(22.6bd(25) epoch 194485) v5 2014-08-13 17:38:03.365378 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.289 2014-08-13 17:38:03.365380 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc980 2014-08-13 17:38:03.365364 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 17:38:03.365396 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.365417 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365420 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365409 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.365416 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [373,344]/[373,344] -- 373/373 2014-08-13 17:38:03.365431 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.365448 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365451 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365449 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.365453 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5140000 pg_notify(22.113(29) epoch 194485) v5 2014-08-13 17:38:03.365447 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] acting_up_affected newup [302,324,368] newacting [302,324,368] 2014-08-13 17:38:03.365458 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.365464 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.365477 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.365481 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [301,324] r=-1 lpr=194479 pi=193306-194478/24 crt=191727'446 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.365498 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.55 2014-08-13 17:38:03.365500 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5140000 2014-08-13 17:38:03.365494 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [373,344] newacting [373,344] 2014-08-13 17:38:03.365512 7fe97ba14700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.365511 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.365529 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [4,373,344] r=0 lpr=194478 pi=193306-194476/25 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.365541 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365544 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365567 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365512 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [301,324](301) acting [301,324](301)): not rw, up_thru 194479 up_from 192972 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) : primary up 192972-194479 includes interval 2014-08-13 17:38:03.365569 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365571 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265240 pg_notify(24.136(1) epoch 194485) v5 2014-08-13 17:38:03.365563 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,373,344](4) acting [4,373,344](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.365569 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194479/194479/194479) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 17:38:03.365581 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194476/194477/194477) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] noting past interval(194477-194478 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) 2014-08-13 17:38:03.365611 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.50 2014-08-13 17:38:03.365614 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265240 2014-08-13 17:38:03.365600 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] up [4,373,344] -> [373,344], acting [4,373,344] -> [373,344], acting_primary 4 -> 373, up_primary 4 -> 373, role 0 -> -1 2014-08-13 17:38:03.365595 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] up [301,324] -> [302,324,368], acting [301,324] -> [302,324,368], acting_primary 301 -> 302, up_primary 301 -> 302, role -1 -> -1 2014-08-13 17:38:03.365621 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.365626 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.365638 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.365654 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365658 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365646 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.365654 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.365676 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365678 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365680 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299a580 pg_query(24.2a1 epoch 194485) v3 2014-08-13 17:38:03.365668 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.365670 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.365686 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.365686 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.365721 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.17 epoch 194485 2014-08-13 17:38:03.365709 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.365724 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299a580 2014-08-13 17:38:03.365725 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.365709 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.365739 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.365735 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(info 0'0) from replica 17 2014-08-13 17:38:03.365753 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.365756 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.365778 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365780 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365770 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.365774 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.365800 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365802 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365790 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.365788 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.365804 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264d00 pg_notify(22.99(27) epoch 194485) v5 2014-08-13 17:38:03.365805 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.365810 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.365821 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.365848 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.337 2014-08-13 17:38:03.365851 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264d00 2014-08-13 17:38:03.365837 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.365828 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] [301,324] -> [302,324,368], acting primary 301 -> 302 2014-08-13 17:38:03.365855 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.365857 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.365871 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.365886 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.365905 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365873 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.365908 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365902 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.365912 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.365931 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.365933 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.365935 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265e80 pg_notify(22.6bd(2) epoch 194485) v5 2014-08-13 17:38:03.365947 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [373,344]/[373,344,320] -- 373/373 2014-08-13 17:38:03.365970 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.365978 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.32 2014-08-13 17:38:03.365980 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265e80 2014-08-13 17:38:03.365975 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.365973 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 17:38:03.365994 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.366008 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366016 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,344] newacting [373,344,320] 2014-08-13 17:38:03.366029 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366032 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366033 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366031 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.366058 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366060 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366062 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8fc0 pg_notify(22.619(1) epoch 194485) v5 2014-08-13 17:38:03.366052 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344] r=-1 lpr=194479 pi=193306-194478/26 crt=191727'416 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.366052 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.366084 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [373,344](373) acting [373,344](373)): not rw, up_thru 194476 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [373,344](373) acting [373,344](373)) : primary up 192978-194476 does not include interval 2014-08-13 17:38:03.366100 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.44 2014-08-13 17:38:03.366087 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.366104 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8fc0 2014-08-13 17:38:03.366103 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194479/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [373,344](373) acting [373,344](373)) 2014-08-13 17:38:03.366121 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] up [373,344] -> [373,344], acting [373,344] -> [373,344,320], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> -1 2014-08-13 17:38:03.366142 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366145 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366142 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.366132 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 17:38:03.366164 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366165 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366167 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366169 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42608c0 pg_notify(22.3d5(2) epoch 194485) v5 2014-08-13 17:38:03.366158 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.366174 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.366175 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366206 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.366223 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.366220 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.366243 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.343 2014-08-13 17:38:03.366247 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42608c0 2014-08-13 17:38:03.366240 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.366244 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.366257 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.366273 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.366298 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366301 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366290 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.366278 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 17:38:03.366307 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.366319 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366322 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366325 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366328 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca680 pg_notify(22.707(33) epoch 194485) v5 2014-08-13 17:38:03.366322 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.366324 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366344 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.366348 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.366369 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.15 2014-08-13 17:38:03.366371 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca680 2014-08-13 17:38:03.366364 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,301,324] newacting [4,301,324] 2014-08-13 17:38:03.366381 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [373,344]/[373,344,320] -- 373/373 2014-08-13 17:38:03.366403 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366407 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366410 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366380 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.366408 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366426 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.366423 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [302,324,368] r=-1 lpr=194482 pi=193306-194481/25 crt=191727'446 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.366452 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366454 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366443 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.366456 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cdb00 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.366494 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.338 2014-08-13 17:38:03.366463 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [302,324,368](302) acting [302,324,368](302)): not rw, up_thru 194482 up_from 192970 last_epoch_clean 194465 generate_past_intervals interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) : primary up 192970-194482 includes interval 2014-08-13 17:38:03.366497 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cdb00 2014-08-13 17:38:03.366483 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [366,320,363]/[373,344,320] -- 366/373 2014-08-13 17:38:03.366503 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366499 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194482/194482/194482) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 17:38:03.366507 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366534 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366537 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366524 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.366520 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [302,324,368] -> [4,301,324], acting [302,324,368] -> [4,301,324], acting_primary 302 -> 4, up_primary 302 -> 4, role -1 -> 0 2014-08-13 17:38:03.366543 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] acting_up_affected newup [366,320,363] newacting [373,344,320] 2014-08-13 17:38:03.366544 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.366560 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.366587 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366590 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366575 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.366576 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [373,344]/[373,344,320] r=-1 lpr=194480 pi=193306-194479/27 crt=191727'416 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.366592 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299d280 pg_query(22.2d1 epoch 194485) v3 2014-08-13 17:38:03.366594 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.366609 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [373,344](373) acting [373,344,320](373)): not rw, up_thru 194480 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [373,344](373) acting [373,344,320](373) maybe_went_rw) : primary up 192978-194480 includes interval 2014-08-13 17:38:03.366630 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.358 epoch 194485 2014-08-13 17:38:03.366633 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299d280 2014-08-13 17:38:03.366623 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2827 2014-08-13 17:38:03.366627 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194479/194480/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [373,344](373) acting [373,344,320](373) maybe_went_rw) 2014-08-13 17:38:03.366645 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 358 2014-08-13 17:38:03.366647 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] up [373,344] -> [366,320,363], acting [373,344,320] -> [373,344,320], acting_primary 373 -> 373, up_primary 373 -> 366, role -1 -> -1 2014-08-13 17:38:03.366670 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.366687 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366690 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366642 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.366685 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.366710 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366712 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366701 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.366714 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299c740 pg_query(22.12a epoch 194485) v3 2014-08-13 17:38:03.366718 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.366716 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.366735 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.366736 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.366752 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.323 epoch 194485 2014-08-13 17:38:03.366755 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299c740 2014-08-13 17:38:03.366752 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.366755 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.366769 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.366766 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 323 2014-08-13 17:38:03.366771 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.366787 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.366806 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366787 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.366810 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366806 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.366828 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366830 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366832 7fe9a63f4700 20 osd.4 194485 _dispatch 0x29992c0 pg_query(24.3bf epoch 194485) v3 2014-08-13 17:38:03.366823 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.366811 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.366840 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.366858 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.366877 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.19 epoch 194485 2014-08-13 17:38:03.366880 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x29992c0 2014-08-13 17:38:03.366841 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.366892 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 19 2014-08-13 17:38:03.366900 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [366,320,363]/[366,320,363] -- 366/366 2014-08-13 17:38:03.366889 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.366921 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.366922 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.366941 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366945 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366926 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.366938 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.366948 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.366965 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.366967 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.366970 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb2c0 pg_notify(22.362(31) epoch 194485) v5 2014-08-13 17:38:03.366957 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.366967 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] acting_up_affected newup [366,320,363] newacting [366,320,363] 2014-08-13 17:38:03.366976 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.367009 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.362 2014-08-13 17:38:03.367012 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb2c0 2014-08-13 17:38:03.366992 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.366987 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.367019 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.367047 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367051 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367036 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363]/[373,344,320] r=-1 lpr=194482 pi=193306-194481/28 crt=191727'416 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.367045 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.367065 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.367079 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367081 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367071 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)): not rw, up_thru 194480 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) : primary up 192978-194480 does not include interval 2014-08-13 17:38:03.367084 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9c00 pg_notify(22.481(46) epoch 194485) v5 2014-08-13 17:38:03.367084 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.367088 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194482/194479) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 17:38:03.367124 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.287 2014-08-13 17:38:03.367107 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] up [366,320,363] -> [366,320,363], acting [373,344,320] -> [366,320,363], acting_primary 373 -> 366, up_primary 366 -> 366, role -1 -> -1 2014-08-13 17:38:03.367126 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9c00 2014-08-13 17:38:03.367109 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,301,324 unchanged 2014-08-13 17:38:03.367127 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.367142 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.367163 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367166 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367140 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.367157 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.367173 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.367172 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002811 7 0.002378 2014-08-13 17:38:03.367206 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367188 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.367209 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367198 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.367216 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42655c0 pg_notify(22.3d5(25) epoch 194485) v5 2014-08-13 17:38:03.367215 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.367219 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.367231 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.367256 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.286 2014-08-13 17:38:03.367259 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42655c0 2014-08-13 17:38:03.367248 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.367237 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.367264 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.367276 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000057 0 0.000000 2014-08-13 17:38:03.367281 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] [373,344,320] -> [366,320,363], acting primary 373 -> 366 2014-08-13 17:38:03.367309 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367316 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367305 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.367334 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367336 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367309 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.367338 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299b2a0 pg_query(22.537 epoch 194485) v3 2014-08-13 17:38:03.367339 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.367343 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.367363 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.367378 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.51 epoch 194485 2014-08-13 17:38:03.367366 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.367381 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299b2a0 2014-08-13 17:38:03.367380 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.367393 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 51 2014-08-13 17:38:03.367409 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [366,320,363]/[366,320,363] -- 366/366 2014-08-13 17:38:03.367430 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.367431 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367433 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367405 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 17:38:03.367434 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.367452 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367455 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367457 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299ab20 pg_query(22.3ad epoch 194485) v3 2014-08-13 17:38:03.367439 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 17:38:03.367450 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.367468 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.367465 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 17:38:03.367494 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.292 epoch 194485 2014-08-13 17:38:03.367497 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299ab20 2014-08-13 17:38:03.367486 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 17:38:03.367499 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 17:38:03.367520 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.367510 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 292 2014-08-13 17:38:03.367511 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 17:38:03.367524 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.367543 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.367555 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367534 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,302,324,368 down blocked_by {} 2014-08-13 17:38:03.367562 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367561 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,373,344] newacting [4,373,344] 2014-08-13 17:38:03.367581 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367584 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367586 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265080 pg_notify(22.1b8(25) epoch 194485) v5 2014-08-13 17:38:03.367577 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.367593 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [366,320,363] r=-1 lpr=194483 pi=193306-194482/29 crt=191727'416 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.367573 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.367624 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.347 2014-08-13 17:38:03.367627 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265080 2014-08-13 17:38:03.367621 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2828 2014-08-13 17:38:03.367625 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [366,320,363](366) acting [366,320,363](366)): not rw, up_thru 194483 up_from 192968 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) : primary up 192968-194483 includes interval 2014-08-13 17:38:03.367643 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194482/194483/194483) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 17:38:03.367666 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367669 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367662 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [366,320,363] -> [4,373,344], acting [366,320,363] -> [4,373,344], acting_primary 366 -> 4, up_primary 366 -> 4, role -1 -> 0 2014-08-13 17:38:03.367687 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367689 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367692 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260c40 pg_notify(24.1bb(1) epoch 194485) v5 2014-08-13 17:38:03.367681 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.367699 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.367646 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.301 2014-08-13 17:38:03.367715 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.367731 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2610 2014-08-13 17:38:03.367760 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.332 2014-08-13 17:38:03.367734 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.302 2014-08-13 17:38:03.367749 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.367763 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260c40 2014-08-13 17:38:03.367766 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.367781 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.367773 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 17:38:03.367804 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367807 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367796 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.367810 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.367809 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 17:38:03.367826 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.367839 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367842 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367845 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264ec0 pg_notify(22.113(29) epoch 194485) v5 2014-08-13 17:38:03.367840 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.367841 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.367858 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.367885 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.24 2014-08-13 17:38:03.367887 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264ec0 2014-08-13 17:38:03.367877 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.367867 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.367894 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.367911 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.367932 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367936 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367933 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.367948 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.367960 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.367966 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.367968 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb100 pg_notify(22.99(1) epoch 194485) v5 2014-08-13 17:38:03.367964 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.367982 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.367979 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.368007 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.363 2014-08-13 17:38:03.367994 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.368009 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb100 2014-08-13 17:38:03.368012 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.368030 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.368046 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.368050 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.368031 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [65,62]/[65,62] -- 65/65 2014-08-13 17:38:03.368057 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.368050 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,344,373 2014-08-13 17:38:03.368067 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.368072 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.368074 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.368077 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299d0a0 pg_query(24.e2 epoch 194485) v3 2014-08-13 17:38:03.368063 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.368070 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.368089 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.692354 9 0.002342 2014-08-13 17:38:03.368113 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.346 epoch 194485 2014-08-13 17:38:03.368081 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.368116 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299d0a0 2014-08-13 17:38:03.368109 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.368118 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [65,62] newacting [65,62] 2014-08-13 17:38:03.368128 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.368128 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_query query(info 0'0) from replica 346 2014-08-13 17:38:03.368146 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.368138 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.368177 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.368181 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.368165 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [4,65,62] r=0 lpr=194478 pi=193306-194476/27 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.368164 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.368188 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.368212 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.368215 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.368217 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299bde0 pg_query(22.69f epoch 194485) v3 2014-08-13 17:38:03.368202 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,65,62](4) acting [4,65,62](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.368215 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.368225 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194476/194477/194477) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] noting past interval(194477-194478 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) 2014-08-13 17:38:03.368234 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.368258 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.334 epoch 194485 2014-08-13 17:38:03.368262 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299bde0 2014-08-13 17:38:03.368244 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] up [4,65,62] -> [65,62], acting [4,65,62] -> [65,62], acting_primary 4 -> 65, up_primary 4 -> 65, role 0 -> -1 2014-08-13 17:38:03.382064 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.368251 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.382096 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.382087 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 334 2014-08-13 17:38:03.382112 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.382128 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.382145 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382149 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382116 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 17:38:03.382143 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.382158 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 17:38:03.382177 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382180 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382170 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.382183 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260a80 pg_notify(22.22c(27) epoch 194485) v5 2014-08-13 17:38:03.382187 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.382178 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [373,344](373) acting [373,344,320](373) maybe_went_rw) 2014-08-13 17:38:03.382204 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.382214 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [373,344](373) acting [373,344](373)) 2014-08-13 17:38:03.382225 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.382238 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.324 2014-08-13 17:38:03.382246 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260a80 2014-08-13 17:38:03.382234 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) 2014-08-13 17:38:03.382246 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.382255 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,373,344](4) acting [373,344,320](373)) 2014-08-13 17:38:03.382264 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.382273 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [373,344](373) acting [373,344,320](373) maybe_went_rw) 2014-08-13 17:38:03.382291 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382294 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382283 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.382293 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [373,344](373) acting [373,344](373)) 2014-08-13 17:38:03.382300 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.382315 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382318 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382320 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299c1a0 pg_query(24.cd epoch 194485) v3 2014-08-13 17:38:03.382315 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.382313 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,320,344,363,366,373 down blocked_by {} 2014-08-13 17:38:03.382331 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.382334 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.382358 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.6 epoch 194485 2014-08-13 17:38:03.382347 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.382362 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299c1a0 2014-08-13 17:38:03.382358 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2611 2014-08-13 17:38:03.382364 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.382379 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.320 2014-08-13 17:38:03.382375 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(info 0'0) from replica 6 2014-08-13 17:38:03.382383 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.382398 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.344 2014-08-13 17:38:03.382416 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 17:38:03.382429 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382432 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382423 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [65,62]/[65,62,44] -- 65/65 2014-08-13 17:38:03.382444 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.382432 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.366 2014-08-13 17:38:03.382453 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382455 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382458 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9180 pg_notify(22.707(1) epoch 194485) v5 2014-08-13 17:38:03.382450 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 17:38:03.382450 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.382474 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.382473 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.382502 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.382494 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] acting_up_affected newup [65,62] newacting [65,62,44] 2014-08-13 17:38:03.382509 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9180 2014-08-13 17:38:03.382497 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.382511 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.382528 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'439 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.382549 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382552 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382570 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382573 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382575 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5138000 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.382559 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [65,62](65) acting [65,62](65)): not rw, up_thru 194476 up_from 192939 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [65,62](65) acting [65,62](65)) : primary up 192939-194476 does not include interval 2014-08-13 17:38:03.382580 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194479/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [65,62](65) acting [65,62](65)) 2014-08-13 17:38:03.382602 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.382614 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.382598 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] up [65,62] -> [65,62], acting [65,62] -> [65,62,44], acting_primary 65 -> 65, up_primary 65 -> 65, role -1 -> -1 2014-08-13 17:38:03.382616 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5138000 2014-08-13 17:38:03.382618 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.382634 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.382654 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382657 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382650 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.382677 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382665 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.382679 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382664 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 17:38:03.382685 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299b840 pg_query(22.639 epoch 194485) v3 2014-08-13 17:38:03.382684 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.382697 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.382702 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.382705 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.382723 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364] newacting [328,364] 2014-08-13 17:38:03.382737 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.367 epoch 194485 2014-08-13 17:38:03.382741 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299b840 2014-08-13 17:38:03.382740 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.382720 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.382753 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 367 2014-08-13 17:38:03.382759 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.215422 3 0.000128 2014-08-13 17:38:03.382761 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.382779 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.382780 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started 0.215486 0 0.000000 2014-08-13 17:38:03.382802 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382805 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382798 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.382801 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.382824 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382826 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382813 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.382829 7fe9a63f4700 20 osd.4 194485 _dispatch 0x29994a0 pg_query(22.3b9 epoch 194485) v3 2014-08-13 17:38:03.382816 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.382832 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.382838 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.382855 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.382868 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.397 epoch 194485 2014-08-13 17:38:03.382872 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x29994a0 2014-08-13 17:38:03.382867 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [65,62]/[65,62,44] -- 65/65 2014-08-13 17:38:03.382871 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364] newacting [328,364] 2014-08-13 17:38:03.382892 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.382884 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 397 2014-08-13 17:38:03.382891 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.382896 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.382910 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.382927 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382916 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.382931 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382935 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.382953 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.382956 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.382958 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8700 pg_notify(22.1d2(1) epoch 194485) v5 2014-08-13 17:38:03.382944 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [328,364,4](328) acting [328,364,4](328)): not rw, up_thru 194476 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [328,364,4](328) acting [328,364,4](328) maybe_went_rw) : primary up 192949-194476 includes interval 2014-08-13 17:38:03.382962 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [328,364,4](328) acting [328,364,4](328) maybe_went_rw) 2014-08-13 17:38:03.382980 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [44,48,314]/[65,62,44] -- 44/65 2014-08-13 17:38:03.382980 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364,4] -> [328,364], acting [328,364,4] -> [328,364], acting_primary 328 -> 328, up_primary 328 -> 328, role 2 -> -1 2014-08-13 17:38:03.383000 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.331 2014-08-13 17:38:03.383002 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383004 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8700 2014-08-13 17:38:03.383001 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383007 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383018 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.383026 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.383038 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.383049 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383055 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383042 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] acting_up_affected newup [44,48,314] newacting [65,62,44] 2014-08-13 17:38:03.383057 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.383060 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.383075 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383078 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383081 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc7c0 pg_notify(24.1bb(34) epoch 194485) v5 2014-08-13 17:38:03.383072 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.383076 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [65,62]/[65,62,44] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'439 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.383089 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.383107 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383121 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.379 2014-08-13 17:38:03.383124 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc7c0 2014-08-13 17:38:03.383109 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [65,62](65) acting [65,62,44](65)): not rw, up_thru 194480 up_from 192939 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [65,62](65) acting [65,62,44](65) maybe_went_rw) : primary up 192939-194480 includes interval 2014-08-13 17:38:03.383126 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.383144 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.383129 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194479/194480/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [65,62](65) acting [65,62,44](65) maybe_went_rw) 2014-08-13 17:38:03.383165 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383169 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383159 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.383163 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] up [65,62] -> [44,48,314], acting [65,62,44] -> [65,62,44], acting_primary 65 -> 65, up_primary 65 -> 44, role -1 -> -1 2014-08-13 17:38:03.383174 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.383187 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.383208 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383211 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383189 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.383213 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8a80 pg_notify(22.48e(29) epoch 194485) v5 2014-08-13 17:38:03.383203 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.383214 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.383221 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.383234 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383243 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.383257 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.327 2014-08-13 17:38:03.383262 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8a80 2014-08-13 17:38:03.383261 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.383272 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 17:38:03.383279 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.383294 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383302 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383305 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383295 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.383299 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383327 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383314 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.383329 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383331 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9dc0 pg_notify(24.136(30) epoch 194485) v5 2014-08-13 17:38:03.383313 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.383331 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383337 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.383355 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.383370 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.65 2014-08-13 17:38:03.383373 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9dc0 2014-08-13 17:38:03.383362 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 17:38:03.383382 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383371 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.383386 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383390 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383402 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.383414 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383417 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383417 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383435 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383437 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383439 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9f80 pg_notify(24.3d(31) epoch 194485) v5 2014-08-13 17:38:03.383454 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 17:38:03.383474 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383439 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [44,48,314]/[44,48,314] -- 44/44 2014-08-13 17:38:03.383482 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.373 2014-08-13 17:38:03.383485 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9f80 2014-08-13 17:38:03.383481 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383478 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383493 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.383493 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383509 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364,329] newacting [328,364,329] 2014-08-13 17:38:03.383525 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383528 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383525 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.383546 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383548 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383550 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299cb00 pg_query(22.581 epoch 194485) v3 2014-08-13 17:38:03.383540 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.383512 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.383570 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [328,364](328) acting [328,364](328)): not rw, up_thru 194479 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [328,364](328) acting [328,364](328) maybe_went_rw) : primary up 192949-194479 includes interval 2014-08-13 17:38:03.383591 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.320 epoch 194485 2014-08-13 17:38:03.383594 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299cb00 2014-08-13 17:38:03.383587 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [328,364](328) acting [328,364](328) maybe_went_rw) 2014-08-13 17:38:03.383565 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] acting_up_affected newup [44,48,314] newacting [44,48,314] 2014-08-13 17:38:03.383605 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364] -> [328,364,329], acting [328,364] -> [328,364,329], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> -1 2014-08-13 17:38:03.383606 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 320 2014-08-13 17:38:03.383611 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.383624 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383631 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314]/[65,62,44] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'439 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.383647 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383650 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383644 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.383659 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.383672 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383675 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383677 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264600 pg_notify(22.3af(31) epoch 194485) v5 2014-08-13 17:38:03.383675 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.383664 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)): not rw, up_thru 194480 up_from 192939 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) : primary up 192939-194480 does not include interval 2014-08-13 17:38:03.383691 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.383694 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194482/194479) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 17:38:03.383717 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.339 2014-08-13 17:38:03.383720 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264600 2014-08-13 17:38:03.383706 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.383714 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] up [44,48,314] -> [44,48,314], acting [65,62,44] -> [44,48,314], acting_primary 65 -> 44, up_primary 44 -> 44, role -1 -> -1 2014-08-13 17:38:03.383723 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383740 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.383736 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383757 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.383770 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383773 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383760 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.383772 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.383776 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.383792 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383794 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383796 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4263d40 pg_notify(22.113(1) epoch 194485) v5 2014-08-13 17:38:03.383787 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.383793 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.383805 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383817 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.383836 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.383839 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4263d40 2014-08-13 17:38:03.383833 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.383841 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 17:38:03.383849 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.383862 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383866 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.383869 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383891 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383894 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383887 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.383886 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.383914 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.383916 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383906 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.383918 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cbb80 pg_notify(22.775(31) epoch 194485) v5 2014-08-13 17:38:03.383907 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] [65,62,44] -> [44,48,314], acting primary 65 -> 44 2014-08-13 17:38:03.383939 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 17:38:03.383955 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.0 2014-08-13 17:38:03.383959 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.383961 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cbb80 2014-08-13 17:38:03.383953 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.383963 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.383974 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.383985 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.384003 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384006 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.383995 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384002 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384033 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384036 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384037 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cde80 pg_notify(22.3d9(49) epoch 194485) v5 2014-08-13 17:38:03.384038 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 17:38:03.384058 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.384075 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.293 2014-08-13 17:38:03.384064 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.384078 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cde80 2014-08-13 17:38:03.384063 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [44,48,314]/[44,48,314] -- 44/44 2014-08-13 17:38:03.384094 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.384080 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.384100 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364,4] newacting [328,364,4] 2014-08-13 17:38:03.384099 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.384119 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384122 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384118 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.384121 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.384134 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,329] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.384151 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384154 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384156 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2999fe0 pg_query(24.3f7 epoch 194485) v3 2014-08-13 17:38:03.384139 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384164 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [328,364,329](328) acting [328,364,329](328)): not rw, up_thru 194482 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [328,364,329](328) acting [328,364,329](328) maybe_went_rw) : primary up 192949-194482 includes interval 2014-08-13 17:38:03.384179 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [328,364,329](328) acting [328,364,329](328) maybe_went_rw) 2014-08-13 17:38:03.384208 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.330 epoch 194485 2014-08-13 17:38:03.384211 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2999fe0 2014-08-13 17:38:03.384204 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364,329] -> [328,364,4], acting [328,364,329] -> [328,364,4], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> 2 2014-08-13 17:38:03.384198 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 17:38:03.384224 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.384228 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.384224 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_query query(info 0'0) from replica 330 2014-08-13 17:38:03.384241 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.384245 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.384257 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384271 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384275 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384274 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.384271 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.384298 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384301 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384289 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.384304 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cdcc0 pg_notify(22.f3(1) epoch 194485) v5 2014-08-13 17:38:03.384307 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.384299 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,65,62] newacting [4,65,62] 2014-08-13 17:38:03.384323 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.384323 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.384344 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.312 2014-08-13 17:38:03.384348 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cdcc0 2014-08-13 17:38:03.384341 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.384342 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [44,48,314] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'439 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.384361 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.384387 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384377 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.384391 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384392 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384393 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [44,48,314](44) acting [44,48,314](44)): not rw, up_thru 194483 up_from 192926 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) : primary up 192926-194483 includes interval 2014-08-13 17:38:03.384413 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384416 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384418 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5140380 pg_notify(22.1f1(1) epoch 194485) v5 2014-08-13 17:38:03.384407 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.384412 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194482/194483/194483) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 17:38:03.384422 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.384442 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384459 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.11 2014-08-13 17:38:03.384462 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5140380 2014-08-13 17:38:03.384437 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [44,48,314] -> [4,65,62], acting [44,48,314] -> [4,65,62], acting_primary 44 -> 4, up_primary 44 -> 4, role -1 -> 0 2014-08-13 17:38:03.384461 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.384468 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.384490 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.384513 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384516 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384506 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.384519 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.384536 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384539 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384542 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb800 pg_notify(24.9b(1) epoch 194485) v5 2014-08-13 17:38:03.384527 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2730 2014-08-13 17:38:03.384537 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.384547 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384558 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.001758 7 0.001495 2014-08-13 17:38:03.384586 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.365 2014-08-13 17:38:03.384589 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb800 2014-08-13 17:38:03.384580 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.384567 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.384597 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.384600 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.384613 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.384633 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384617 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384636 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384632 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.384637 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384658 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384660 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384662 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ccec0 pg_notify(22.481(46) epoch 194485) v5 2014-08-13 17:38:03.384653 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.384653 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.384676 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.384678 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.384700 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.381 2014-08-13 17:38:03.384703 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ccec0 2014-08-13 17:38:03.384696 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.384692 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.384721 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.384743 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384747 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384741 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.384778 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384780 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384781 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.384758 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.384782 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8540 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.384786 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.384802 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.384820 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.396 2014-08-13 17:38:03.384822 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8540 2014-08-13 17:38:03.384817 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.384832 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.384836 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.384851 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.384865 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384868 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384889 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.384891 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.384893 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5140540 pg_notify(22.619(31) epoch 194485) v5 2014-08-13 17:38:03.384895 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.384869 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384903 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.384932 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.314 2014-08-13 17:38:03.384935 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5140540 2014-08-13 17:38:03.384924 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.384925 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.384961 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,62,65 2014-08-13 17:38:03.384979 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.384967 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.384991 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.384982 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.384996 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.384998 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.675644 9 0.016510 2014-08-13 17:38:03.385013 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385024 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.385052 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.385078 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385083 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.385099 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385040 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.385135 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.385157 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385117 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.385162 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.385167 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000127 0 0.000000 2014-08-13 17:38:03.385179 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385195 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.385211 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.385226 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.385230 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.385255 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385246 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.385260 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.385276 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385265 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 17:38:03.385301 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 17:38:03.385313 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.385336 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385322 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [65,62](65) acting [65,62,44](65) maybe_went_rw) 2014-08-13 17:38:03.385341 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.385342 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [65,62](65) acting [65,62](65)) 2014-08-13 17:38:03.385359 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385361 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) 2014-08-13 17:38:03.385384 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,65,62](4) acting [65,62,44](65)) 2014-08-13 17:38:03.385397 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:03.385420 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385401 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [65,62](65) acting [65,62,44](65) maybe_went_rw) 2014-08-13 17:38:03.385425 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:03.385436 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [65,62](65) acting [65,62](65)) 2014-08-13 17:38:03.385447 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.385469 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,48,62,65,314 down blocked_by {} 2014-08-13 17:38:03.385476 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:03.385489 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.385527 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2731 2014-08-13 17:38:03.385544 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 17:38:03.385551 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:03.385567 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.48 2014-08-13 17:38:03.385570 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194485 2014-08-13 17:38:03.385587 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.385606 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.385589 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.62 2014-08-13 17:38:03.385618 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.65 2014-08-13 17:38:03.385631 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,288]/[330,57] -- 330/330 2014-08-13 17:38:03.385641 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.314 2014-08-13 17:38:03.385660 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385666 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.385663 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.385683 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.385685 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.385700 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,288] newacting [330,57] 2014-08-13 17:38:03.385716 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.385732 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,4,288]/[330,4,57] r=1 lpr=194478 pi=193227-194476/33 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194479 2014-08-13 17:38:03.385763 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [330,4,288](330) acting [330,4,57](330)): not rw, up_thru 194477 up_from 192965 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [330,4,288](330) acting [330,4,57](330) maybe_went_rw) : primary up 192965-194477 includes interval 2014-08-13 17:38:03.385780 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194476/194477/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194477-194478 up [330,4,288](330) acting [330,4,57](330) maybe_went_rw) 2014-08-13 17:38:03.385797 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,4,288] -> [330,288], acting [330,4,57] -> [330,57], acting_primary 330 -> 330, up_primary 330 -> 330, role 1 -> -1 2014-08-13 17:38:03.385818 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.385816 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.385833 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.385847 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.385864 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:03.385879 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:03.385894 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:03.385920 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.385924 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.385896 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 17:38:03.385911 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.385930 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:03.385948 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.385951 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.385952 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb9c0 pg_notify(24.b6(37) epoch 194485) v5 2014-08-13 17:38:03.385946 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:03.385929 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.385965 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_role_change 2014-08-13 17:38:03.385969 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.385991 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.315 2014-08-13 17:38:03.385994 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb9c0 2014-08-13 17:38:03.385981 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.385993 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331] newacting [54,331] 2014-08-13 17:38:03.386000 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 17:38:03.386012 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.386018 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:03.386036 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386040 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386036 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386037 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.215280 3 0.000159 2014-08-13 17:38:03.386072 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386075 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386082 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca140 pg_notify(22.f3(23) epoch 194485) v5 2014-08-13 17:38:03.386070 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started 0.215382 0 0.000000 2014-08-13 17:38:03.386073 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,288]/[330,57,304] -- 330/330 2014-08-13 17:38:03.386099 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.386095 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.386104 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.386125 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.311 2014-08-13 17:38:03.386112 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194477 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.386128 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca140 2014-08-13 17:38:03.386123 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.386141 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,288] newacting [330,57,304] 2014-08-13 17:38:03.386170 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386174 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386159 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.386179 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57] r=-1 lpr=194479 pi=193227-194478/34 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194480 2014-08-13 17:38:03.386205 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386144 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194479 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.386208 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386210 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8380 pg_notify(22.619(31) epoch 194485) v5 2014-08-13 17:38:03.386219 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [330,288](330) acting [330,57](330)): not rw, up_thru 194477 up_from 192965 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [330,288](330) acting [330,57](330)) : primary up 192965-194477 does not include interval 2014-08-13 17:38:03.386247 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.48 2014-08-13 17:38:03.386250 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8380 2014-08-13 17:38:03.386209 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194479 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.386240 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194479/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194479-194479 up [330,288](330) acting [330,57](330)) 2014-08-13 17:38:03.386254 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194479 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331] newacting [54,331] 2014-08-13 17:38:03.386261 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,288] -> [330,288], acting [330,57] -> [330,57,304], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 17:38:03.386272 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194479 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.386282 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.386302 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386306 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386300 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.386293 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331,4] r=2 lpr=194479 pi=192980-194475/21 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.386328 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386316 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.386330 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386336 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd5c0 pg_notify(22.22c(1) epoch 194485) v5 2014-08-13 17:38:03.386336 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:03.386341 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [54,331,4](54) acting [54,331,4](54)): not rw, up_thru 194476 up_from 192936 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [54,331,4](54) acting [54,331,4](54) maybe_went_rw) : primary up 192936-194476 includes interval 2014-08-13 17:38:03.386353 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:03.386377 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.302 2014-08-13 17:38:03.386379 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd5c0 2014-08-13 17:38:03.386371 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:03.386360 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194476/194476/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [54,331,4](54) acting [54,331,4](54) maybe_went_rw) 2014-08-13 17:38:03.386389 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.386419 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386421 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386408 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:03.386391 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331,4] -> [54,331], acting [54,331,4] -> [54,331], acting_primary 54 -> 54, up_primary 54 -> 54, role 2 -> -1 2014-08-13 17:38:03.386428 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.386427 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:03.386446 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386449 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386451 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd400 pg_notify(22.481(39) epoch 194485) v5 2014-08-13 17:38:03.386445 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.386447 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 17:38:03.386465 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:03.386463 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.386494 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.309 2014-08-13 17:38:03.386497 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd400 2014-08-13 17:38:03.386484 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386496 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.386515 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.386536 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386539 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386521 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,288]/[330,57,304] -- 330/330 2014-08-13 17:38:03.386534 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.386551 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.386550 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.386568 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386571 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386557 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.386573 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd240 pg_notify(2.2a(2) epoch 194485) v5 2014-08-13 17:38:03.386576 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.386569 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.386594 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386614 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.370 2014-08-13 17:38:03.386602 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.386617 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd240 2014-08-13 17:38:03.386619 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.386633 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,304,57]/[330,57,304] -- 330/330 2014-08-13 17:38:03.386657 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386658 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.386660 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386638 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.386663 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.386665 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.386683 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386686 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386689 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5142bc0 pg_notify(24.30b(38) epoch 194485) v5 2014-08-13 17:38:03.386684 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.386683 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.386703 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,304,57] newacting [330,57,304] 2014-08-13 17:38:03.386705 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386727 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.375 2014-08-13 17:38:03.386730 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5142bc0 2014-08-13 17:38:03.386722 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.386738 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,288]/[330,57,304] r=-1 lpr=194480 pi=193227-194479/35 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194482 2014-08-13 17:38:03.386764 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386767 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386768 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [330,288](330) acting [330,57,304](330)): not rw, up_thru 194480 up_from 192965 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [330,288](330) acting [330,57,304](330) maybe_went_rw) : primary up 192965-194480 includes interval 2014-08-13 17:38:03.386757 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 17:38:03.386790 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.386798 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386801 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386787 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194479/194480/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194480-194481 up [330,288](330) acting [330,57,304](330) maybe_went_rw) 2014-08-13 17:38:03.386803 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5141500 pg_notify(22.4df(27) epoch 194485) v5 2014-08-13 17:38:03.386796 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.386809 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,288] -> [330,304,57], acting [330,57,304] -> [330,57,304], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 17:38:03.386827 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.386831 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.386845 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.371 2014-08-13 17:38:03.386848 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5141500 2014-08-13 17:38:03.386848 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.386849 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386868 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.386890 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386893 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386885 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:03.386887 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 17:38:03.386907 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.386903 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:03.386923 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.386925 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.386927 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce040 pg_notify(22.22c(25) epoch 194485) v5 2014-08-13 17:38:03.386911 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.386920 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:03.386938 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:03.386931 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.386966 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.368 2014-08-13 17:38:03.386968 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce040 2014-08-13 17:38:03.386956 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:03.386966 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.386975 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:03.386992 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 17:38:03.387018 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387020 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387010 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:03.387006 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 17:38:03.387031 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.387028 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.387047 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387049 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387051 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce740 pg_notify(22.4df(27) epoch 194485) v5 2014-08-13 17:38:03.387035 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.387066 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.387068 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,304,57]/[330,304,57] -- 330/330 2014-08-13 17:38:03.387088 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.380 2014-08-13 17:38:03.387091 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce740 2014-08-13 17:38:03.387094 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.387083 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331,310] newacting [54,331,310] 2014-08-13 17:38:03.387099 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.387116 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.387131 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387117 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.387134 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387136 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,304,57] newacting [330,304,57] 2014-08-13 17:38:03.387158 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387160 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387162 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299d640 pg_query(22.220 epoch 194485) v3 2014-08-13 17:38:03.387153 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.387134 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331] r=-1 lpr=194479 pi=192980-194478/22 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.387169 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57]/[330,57,304] r=-1 lpr=194482 pi=193227-194481/36 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194483 2014-08-13 17:38:03.387207 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.351 epoch 194485 2014-08-13 17:38:03.387210 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299d640 2014-08-13 17:38:03.387208 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [330,304,57](330) acting [330,57,304](330)): not rw, up_thru 194480 up_from 192965 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [330,304,57](330) acting [330,57,304](330)) : primary up 192965-194480 does not include interval 2014-08-13 17:38:03.387197 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [54,331](54) acting [54,331](54)): not rw, up_thru 194479 up_from 192936 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [54,331](54) acting [54,331](54) maybe_went_rw) : primary up 192936-194479 includes interval 2014-08-13 17:38:03.387222 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 351 2014-08-13 17:38:03.387226 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194482/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] noting past interval(194482-194482 up [330,304,57](330) acting [330,57,304](330)) 2014-08-13 17:38:03.387240 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194479/194479/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [54,331](54) acting [54,331](54) maybe_went_rw) 2014-08-13 17:38:03.387262 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387245 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] up [330,304,57] -> [330,304,57], acting [330,57,304] -> [330,304,57], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 17:38:03.387265 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387266 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 17:38:03.387264 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331] -> [54,331,310], acting [54,331] -> [54,331,310], acting_primary 54 -> 54, up_primary 54 -> 54, role -1 -> -1 2014-08-13 17:38:03.387285 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387292 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.387295 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5142f40 pg_notify(22.3af(31) epoch 194485) v5 2014-08-13 17:38:03.387283 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.387290 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.387301 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.387312 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.387320 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:03.387336 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.387339 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5142f40 2014-08-13 17:38:03.387329 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.387337 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:03.387353 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:03.387347 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.387380 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.387369 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 17:38:03.387384 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396208 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:03.387378 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.396246 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:03.396263 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_recovery 2014-08-13 17:38:03.396289 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396279 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:03.396293 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396261 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.396296 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5142d80 pg_notify(22.84(2) epoch 194485) v5 2014-08-13 17:38:03.396298 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396299 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.396319 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.396336 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.396353 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.7 2014-08-13 17:38:03.396358 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5142d80 2014-08-13 17:38:03.396348 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,304,57]/[330,304,57] -- 330/330 2014-08-13 17:38:03.396356 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.396380 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.396372 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.396386 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.396391 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396408 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396411 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396404 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.396421 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396438 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396440 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396443 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51439c0 pg_notify(22.775(31) epoch 194485) v5 2014-08-13 17:38:03.396438 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 17:38:03.396460 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.396452 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,288] -- 330/330 2014-08-13 17:38:03.396476 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.396467 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.396482 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.63 2014-08-13 17:38:03.396485 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51439c0 2014-08-13 17:38:03.396481 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:03.396484 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.396503 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396498 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.396528 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396531 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396523 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,288] 2014-08-13 17:38:03.396537 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 17:38:03.396541 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.396556 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.396561 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396564 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396566 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143d40 pg_notify(22.362(31) epoch 194485) v5 2014-08-13 17:38:03.396557 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,304,57] r=-1 lpr=194483 pi=193227-194482/37 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194485 2014-08-13 17:38:03.396561 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.396578 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.396606 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.349 2014-08-13 17:38:03.396608 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143d40 2014-08-13 17:38:03.396596 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396595 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [330,304,57](330) acting [330,304,57](330)): not rw, up_thru 194483 up_from 192965 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [330,304,57](330) acting [330,304,57](330) maybe_went_rw) : primary up 192965-194483 includes interval 2014-08-13 17:38:03.396615 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194482/194483/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] noting past interval(194483-194484 up [330,304,57](330) acting [330,304,57](330) maybe_went_rw) 2014-08-13 17:38:03.396628 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 17:38:03.396648 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.396650 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396633 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] up [330,304,57] -> [330,4,288], acting [330,304,57] -> [330,4,288], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> 1 2014-08-13 17:38:03.396653 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396651 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.396655 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 17:38:03.396678 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396680 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396669 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.396683 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143b80 pg_notify(24.9b(33) epoch 194485) v5 2014-08-13 17:38:03.396672 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.396685 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331,4] newacting [54,331,4] 2014-08-13 17:38:03.396689 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.396705 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.396709 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:03.396732 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.350 2014-08-13 17:38:03.396734 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143b80 2014-08-13 17:38:03.396723 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,310] r=-1 lpr=194482 pi=192980-194481/23 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.396725 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:03.396743 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:03.396760 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 17:38:03.396761 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [54,331,310](54) acting [54,331,310](54)): not rw, up_thru 194482 up_from 192936 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [54,331,310](54) acting [54,331,310](54) maybe_went_rw) : primary up 192936-194482 includes interval 2014-08-13 17:38:03.396776 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396779 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396778 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:03.396779 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194482/194482/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [54,331,310](54) acting [54,331,310](54) maybe_went_rw) 2014-08-13 17:38:03.396809 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396811 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396798 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:03.396814 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143f00 pg_notify(22.481(44) epoch 194485) v5 2014-08-13 17:38:03.396799 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331,310] -> [54,331,4], acting [54,331,310] -> [54,331,4], acting_primary 54 -> 54, up_primary 54 -> 54, role -1 -> 2 2014-08-13 17:38:03.396817 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_role_change 2014-08-13 17:38:03.396822 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.396834 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.396838 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.396854 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.306 2014-08-13 17:38:03.396857 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143f00 2014-08-13 17:38:03.396850 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_recovery 2014-08-13 17:38:03.396854 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.396867 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:03.396873 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.396885 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.396900 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396904 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396889 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.396903 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_activate_map 2014-08-13 17:38:03.396910 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.396925 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.396936 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.396939 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.396940 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5144440 pg_notify(22.4df(1) epoch 194485) v5 2014-08-13 17:38:03.396940 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.396956 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.396977 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.352 2014-08-13 17:38:03.396980 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5144440 2014-08-13 17:38:03.396971 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.396985 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.396983 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] update_heartbeat_peers unchanged 2014-08-13 17:38:03.397001 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.397016 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397019 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397004 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] take_waiters 2014-08-13 17:38:03.397016 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.397024 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Reset 0.254480 9 0.010957 2014-08-13 17:38:03.397043 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397045 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397035 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.397047 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5144280 pg_notify(22.39d(1) epoch 194485) v5 2014-08-13 17:38:03.397047 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started 2014-08-13 17:38:03.397054 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.397066 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Start 2014-08-13 17:38:03.397087 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.366 2014-08-13 17:38:03.397089 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5144280 2014-08-13 17:38:03.397082 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: transitioning to Stray 2014-08-13 17:38:03.397102 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.397126 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397129 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397119 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.397122 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started/Stray 2014-08-13 17:38:03.397137 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.397151 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397154 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397156 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51440c0 pg_notify(22.3d9(49) epoch 194485) v5 2014-08-13 17:38:03.397143 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.397163 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.397154 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.011060 7 0.010790 2014-08-13 17:38:03.397187 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.397208 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.382 2014-08-13 17:38:03.397211 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51440c0 2014-08-13 17:38:03.397204 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.397219 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.397250 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397238 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 2014-08-13 17:38:03.397253 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397261 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.397280 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.397291 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397294 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397282 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.397296 7fe9a63f4700 20 osd.4 194485 _dispatch 0x54dd7c0 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.397305 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.397311 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.397299 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.397313 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397315 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397340 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397342 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397343 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42647c0 pg_notify(22.3d5(2),24.30b(34) epoch 194485) v5 2014-08-13 17:38:03.397333 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [353,56]/[353,56] -- 353/353 2014-08-13 17:38:03.397361 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.397367 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.397386 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.10 2014-08-13 17:38:03.397388 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.397388 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42647c0 2014-08-13 17:38:03.397383 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.397399 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [353,56] newacting [353,56] 2014-08-13 17:38:03.397415 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.397430 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,353,56] r=0 lpr=194478 pi=193306-194476/27 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.397453 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397456 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397463 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,353,56](4) acting [4,353,56](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.397478 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397480 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397485 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102140 pg_notify(22.362 epoch 194485) v5 2014-08-13 17:38:03.397442 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 17:38:03.397479 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] noting past interval(194477-194478 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) 2014-08-13 17:38:03.397506 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.397498 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] up [4,353,56] -> [353,56], acting [4,353,56] -> [353,56], acting_primary 4 -> 353, up_primary 4 -> 353, role 0 -> -1 2014-08-13 17:38:03.397526 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.58 2014-08-13 17:38:03.397528 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102140 2014-08-13 17:38:03.397517 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.397515 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Peering advmap 2014-08-13 17:38:03.397534 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.397537 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.397550 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.397565 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397569 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397558 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.397566 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.397588 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397590 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397592 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265780 pg_notify(22.84(25) epoch 194485) v5 2014-08-13 17:38:03.397581 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.397580 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.224033 2 0.000390 2014-08-13 17:38:03.397600 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.397605 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Leaving Peering 2014-08-13 17:38:03.397615 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.397633 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.304 2014-08-13 17:38:03.397635 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265780 2014-08-13 17:38:03.397625 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.224094 0 0.000000 2014-08-13 17:38:03.397632 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.397647 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.397654 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.224140 0 0.000000 2014-08-13 17:38:03.397666 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.397681 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397684 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397678 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.224227 0 0.000000 2014-08-13 17:38:03.397684 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.397703 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397705 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397707 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c17e00 pg_notify(22.84(2) epoch 194485) v5 2014-08-13 17:38:03.397700 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 17:38:03.397699 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.397720 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.397719 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194477 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194479 2014-08-13 17:38:03.397747 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.28 2014-08-13 17:38:03.397736 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.397750 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c17e00 2014-08-13 17:38:03.397741 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194479 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 17:38:03.397750 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.397760 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194479 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.397766 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.397781 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.397793 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397779 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194479 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [378,367] newacting [291,350] 2014-08-13 17:38:03.397800 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397798 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194479 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.397801 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.397820 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397822 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397824 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2999680 pg_query(24.350 epoch 194485) v3 2014-08-13 17:38:03.397816 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [4,378,367]/[4,291,350] r=0 lpr=194479 pi=193637-194475/26 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194479 2014-08-13 17:38:03.397836 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [353,56]/[353,56,63] -- 353/353 2014-08-13 17:38:03.397858 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.397861 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.356 epoch 194485 2014-08-13 17:38:03.397865 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2999680 2014-08-13 17:38:03.397854 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,378,367](4) acting [4,291,350](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.397863 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.397874 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194476/194476/194476) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] noting past interval(194476-194478 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 17:38:03.397881 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.397877 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(info 0'0) from replica 356 2014-08-13 17:38:03.397900 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] acting_up_affected newup [353,56] newacting [353,56,63] 2014-08-13 17:38:03.397898 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] up [4,378,367] -> [378,367], acting [4,291,350] -> [291,350], acting_primary 4 -> 291, up_primary 4 -> 378, role 0 -> -1 2014-08-13 17:38:03.397924 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397927 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397918 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.397919 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] clear_primary_state 2014-08-13 17:38:03.397934 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'453 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.397950 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.397938 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] agent_stop 2014-08-13 17:38:03.397952 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.397955 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299bc00 pg_query(22.1a0 epoch 194485) v3 2014-08-13 17:38:03.397955 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] on_change 2014-08-13 17:38:03.397964 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [353,56](353) acting [353,56](353)): not rw, up_thru 194476 up_from 192973 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [353,56](353) acting [353,56](353)) : primary up 192973-194476 does not include interval 2014-08-13 17:38:03.397973 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.397983 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [353,56](353) acting [353,56](353)) 2014-08-13 17:38:03.398000 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.328 epoch 194485 2014-08-13 17:38:03.398003 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299bc00 2014-08-13 17:38:03.397991 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:03.398003 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] up [353,56] -> [353,56], acting [353,56] -> [353,56,63], acting_primary 353 -> 353, up_primary 353 -> 353, role -1 -> -1 2014-08-13 17:38:03.398011 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 17:38:03.398027 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 17:38:03.398024 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.398016 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 328 2014-08-13 17:38:03.398047 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 17:38:03.398050 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.398064 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] on_change 2014-08-13 17:38:03.398068 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.398089 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398092 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398084 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.398083 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 17:38:03.398100 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.398116 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398102 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 17:38:03.398119 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398122 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51024c0 pg_notify(24.30b(1) epoch 194485) v5 2014-08-13 17:38:03.398116 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.398120 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_stats 2014-08-13 17:38:03.398137 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.398138 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.398164 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.301 2014-08-13 17:38:03.398167 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51024c0 2014-08-13 17:38:03.398156 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.398156 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_role_change 2014-08-13 17:38:03.398175 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.398183 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.398191 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.398199 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.398217 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398220 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398215 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.398218 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.398244 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398247 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398232 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398250 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51401c0 pg_notify(22.1f1(29) epoch 194485) v5 2014-08-13 17:38:03.398238 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398289 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.54 2014-08-13 17:38:03.398273 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [353,56]/[353,56,63] -- 353/353 2014-08-13 17:38:03.398292 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51401c0 2014-08-13 17:38:03.398295 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398289 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 17:38:03.398300 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.398319 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398316 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398333 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398336 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398325 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.398334 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398344 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398359 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398363 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398366 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2998b40 pg_query(22.116 epoch 194485) v3 2014-08-13 17:38:03.398363 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398376 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [369,63,0]/[353,56,63] -- 369/353 2014-08-13 17:38:03.398396 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398402 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.345 epoch 194485 2014-08-13 17:38:03.398405 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2998b40 2014-08-13 17:38:03.398395 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 17:38:03.398401 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.398420 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398418 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398417 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 345 2014-08-13 17:38:03.398425 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.398435 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] acting_up_affected newup [369,63,0] newacting [353,56,63] 2014-08-13 17:38:03.398444 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398460 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398464 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398454 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.398463 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398482 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398485 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398471 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [353,56]/[353,56,63] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'453 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.398487 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c81c0 pg_notify(22.362(1) epoch 194485) v5 2014-08-13 17:38:03.398501 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [353,56](353) acting [353,56,63](353)): not rw, up_thru 194480 up_from 192973 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [353,56](353) acting [353,56,63](353) maybe_went_rw) : primary up 192973-194480 includes interval 2014-08-13 17:38:03.398509 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [365,350,291]/[291,350] -- 365/291 2014-08-13 17:38:03.398527 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.398530 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c81c0 2014-08-13 17:38:03.398534 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398519 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [353,56](353) acting [353,56,63](353) maybe_went_rw) 2014-08-13 17:38:03.398538 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] up [353,56] -> [369,63,0], acting [353,56,63] -> [353,56,63], acting_primary 353 -> 353, up_primary 353 -> 369, role -1 -> -1 2014-08-13 17:38:03.398538 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.398561 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.398561 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398583 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398587 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398579 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.398580 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [365,350,291] newacting [291,350] 2014-08-13 17:38:03.398596 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.398611 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398599 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.398614 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398617 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299b0c0 pg_query(24.369 epoch 194485) v3 2014-08-13 17:38:03.398613 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.398615 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [378,367]/[291,350] r=-1 lpr=194479 pi=193637-194478/27 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194482 2014-08-13 17:38:03.398629 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.398658 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.45 epoch 194485 2014-08-13 17:38:03.398646 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.398661 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299b0c0 2014-08-13 17:38:03.398648 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [378,367](378) acting [291,350](291)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [378,367](378) acting [291,350](291) maybe_went_rw) : primary up 192974-194479 includes interval 2014-08-13 17:38:03.398663 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.398667 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194479/194479/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] noting past interval(194479-194481 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 17:38:03.398673 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_query query(info 0'0) from replica 45 2014-08-13 17:38:03.398681 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.398687 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] up [378,367] -> [365,350,291], acting [291,350] -> [291,350], acting_primary 291 -> 291, up_primary 378 -> 365, role -1 -> -1 2014-08-13 17:38:03.398699 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.398710 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.398728 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398717 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.398734 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398728 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.398736 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.398746 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.398764 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398766 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398754 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398769 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299d460 pg_query(24.5e epoch 194485) v3 2014-08-13 17:38:03.398764 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.398782 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.398813 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.289 epoch 194485 2014-08-13 17:38:03.398797 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [369,63,0]/[369,63,0] -- 369/369 2014-08-13 17:38:03.398816 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299d460 2014-08-13 17:38:03.398803 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.398820 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398822 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.398824 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.398828 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_query query(info 0'0) from replica 289 2014-08-13 17:38:03.398840 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.398842 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.398876 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398860 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.398860 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] acting_up_affected newup [369,63,0] newacting [369,63,0] 2014-08-13 17:38:03.398878 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398880 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.398880 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.398908 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.398911 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.398897 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0]/[353,56,63] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'453 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.398898 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.398914 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb640 pg_notify(24.1f7(1) epoch 194485) v5 2014-08-13 17:38:03.398921 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.398931 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)): not rw, up_thru 194480 up_from 192973 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) : primary up 192973-194480 does not include interval 2014-08-13 17:38:03.398955 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.55 2014-08-13 17:38:03.398958 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb640 2014-08-13 17:38:03.398949 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 17:38:03.398966 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] up [369,63,0] -> [369,63,0], acting [353,56,63] -> [369,63,0], acting_primary 353 -> 369, up_primary 369 -> 369, role -1 -> -1 2014-08-13 17:38:03.398968 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [365,350,291]/[365,350,291] -- 365/365 2014-08-13 17:38:03.398994 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.398986 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.399000 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.399015 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399002 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.399020 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399022 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.399019 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.399050 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399052 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399039 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.399055 7fe9a63f4700 20 osd.4 194485 _dispatch 0x50f81c0 pg_notify(22.6bd(2) epoch 194485) v5 2014-08-13 17:38:03.399039 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [365,350,291] newacting [365,350,291] 2014-08-13 17:38:03.399056 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.399060 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.399072 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.399080 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291]/[291,350] r=-1 lpr=194482 pi=193637-194481/28 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194483 2014-08-13 17:38:03.399096 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.50 2014-08-13 17:38:03.399099 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x50f81c0 2014-08-13 17:38:03.399089 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.399106 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.399114 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [365,350,291](365) acting [291,350](291)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [365,350,291](365) acting [291,350](291)) : primary up 192974-194479 does not include interval 2014-08-13 17:38:03.399123 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.399134 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194482/194479) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] noting past interval(194482-194482 up [365,350,291](365) acting [291,350](291)) 2014-08-13 17:38:03.399156 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399159 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399139 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] [353,56,63] -> [369,63,0], acting primary 353 -> 369 2014-08-13 17:38:03.399153 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] up [365,350,291] -> [365,350,291], acting [291,350] -> [365,350,291], acting_primary 291 -> 365, up_primary 365 -> 365, role -1 -> -1 2014-08-13 17:38:03.399170 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.399181 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.399200 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399204 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399184 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.399206 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4267e00 pg_notify(24.b6(37) epoch 194485) v5 2014-08-13 17:38:03.399197 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.399211 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.399213 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.399232 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.399252 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.17 2014-08-13 17:38:03.399255 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4267e00 2014-08-13 17:38:03.399247 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.399253 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [369,63,0]/[369,63,0] -- 369/369 2014-08-13 17:38:03.399274 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.399267 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.399278 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.399285 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.399295 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.399314 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399303 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.399318 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399312 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.399321 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.399345 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399348 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399351 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299a1c0 pg_query(22.4fe epoch 194485) v3 2014-08-13 17:38:03.399343 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 17:38:03.399338 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] [291,350] -> [365,350,291], acting primary 291 -> 365 2014-08-13 17:38:03.399366 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.399366 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.399371 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.399392 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.337 epoch 194485 2014-08-13 17:38:03.399395 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299a1c0 2014-08-13 17:38:03.399382 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.399387 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.399402 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.399403 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,353,56] newacting [4,353,56] 2014-08-13 17:38:03.399408 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 337 2014-08-13 17:38:03.399421 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.399437 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [369,63,0] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'453 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.399447 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [365,350,291]/[365,350,291] -- 365/365 2014-08-13 17:38:03.399464 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399467 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399472 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.399467 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [369,63,0](369) acting [369,63,0](369)): not rw, up_thru 194483 up_from 192964 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) : primary up 192964-194483 includes interval 2014-08-13 17:38:03.399486 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399489 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399492 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299a940 pg_query(24.206 epoch 194485) v3 2014-08-13 17:38:03.399478 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.399485 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 17:38:03.399495 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.399505 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [369,63,0] -> [4,353,56], acting [369,63,0] -> [4,353,56], acting_primary 369 -> 4, up_primary 369 -> 4, role -1 -> 0 2014-08-13 17:38:03.399514 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.399533 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.343 epoch 194485 2014-08-13 17:38:03.399536 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299a940 2014-08-13 17:38:03.399526 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.399544 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.399547 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(info 0'0) from replica 343 2014-08-13 17:38:03.399550 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [4,378,367]/[4,378,367] -- 4/4 2014-08-13 17:38:03.399560 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.399575 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.399579 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2718 2014-08-13 17:38:03.399580 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.399598 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.399597 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.399620 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399624 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399616 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.399615 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] acting_up_affected newup [4,378,367] newacting [4,378,367] 2014-08-13 17:38:03.399647 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399635 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.399633 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.399650 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399655 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299bfc0 pg_query(24.20e epoch 194485) v3 2014-08-13 17:38:03.399651 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [365,350,291] r=-1 lpr=194483 pi=193637-194482/29 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] set_last_peering_reset 194485 2014-08-13 17:38:03.399653 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.399669 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.399692 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.15 epoch 194485 2014-08-13 17:38:03.399695 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299bfc0 2014-08-13 17:38:03.399684 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.399686 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [365,350,291](365) acting [365,350,291](365)): not rw, up_thru 194483 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) : primary up 192966-194483 includes interval 2014-08-13 17:38:03.399701 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.399705 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194482/194483/194483) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] noting past interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 17:38:03.399716 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.399714 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 15 2014-08-13 17:38:03.399726 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] up [365,350,291] -> [4,378,367], acting [365,350,291] -> [4,378,367], acting_primary 365 -> 4, up_primary 365 -> 4, role -1 -> 0 2014-08-13 17:38:03.399735 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.399761 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399749 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] clear_primary_state 2014-08-13 17:38:03.399765 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399753 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.399768 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] agent_stop 2014-08-13 17:38:03.399769 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.399788 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399791 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399793 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102bc0 pg_notify(22.362(31) epoch 194485) v5 2014-08-13 17:38:03.399786 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.399788 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.399805 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.399806 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd 194485:25284 2014-08-13 17:38:03.399834 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.362 2014-08-13 17:38:03.399821 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.399837 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102bc0 2014-08-13 17:38:03.399826 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.399837 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.399846 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.399854 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.399863 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.399879 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399882 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399872 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.399880 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.399891 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.399909 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.399897 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.399912 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399915 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102f40 pg_notify(24.30b epoch 194485) v5 2014-08-13 17:38:03.399909 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,56,353 2014-08-13 17:38:03.399914 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.399929 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.399932 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.399931 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.399956 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.286 2014-08-13 17:38:03.399959 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102f40 2014-08-13 17:38:03.399950 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.399951 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.677525 9 0.002274 2014-08-13 17:38:03.399973 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.399970 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.399998 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400002 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.399990 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.399994 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.400006 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.400025 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400028 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400031 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc600 pg_notify(22.39d(29) epoch 194485) v5 2014-08-13 17:38:03.400017 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] clear_stats 2014-08-13 17:38:03.400026 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000035 0 0.000000 2014-08-13 17:38:03.400039 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 17:38:03.400047 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.400056 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] on_role_change 2014-08-13 17:38:03.400071 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.363 2014-08-13 17:38:03.400077 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc600 2014-08-13 17:38:03.400066 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.400075 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 17:38:03.400084 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.400092 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] cancel_recovery 2014-08-13 17:38:03.400101 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.400108 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] clear_recovery_state 2014-08-13 17:38:03.400121 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 17:38:03.400132 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400140 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400126 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.400142 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 17:38:03.400159 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400146 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_activate_map 2014-08-13 17:38:03.400164 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400167 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42640c0 pg_notify(2.2a(1) epoch 194485) v5 2014-08-13 17:38:03.400161 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [353,56](353) acting [353,56,63](353) maybe_went_rw) 2014-08-13 17:38:03.400176 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] update_heartbeat_peers 4,291,350,367,378 -> 4,367,378 2014-08-13 17:38:03.400198 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.400181 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [353,56](353) acting [353,56](353)) 2014-08-13 17:38:03.400214 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.324 2014-08-13 17:38:03.400202 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] take_waiters 2014-08-13 17:38:03.400218 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42640c0 2014-08-13 17:38:03.400211 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) 2014-08-13 17:38:03.400221 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.002522 7 0.002306 2014-08-13 17:38:03.400233 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,353,56](4) acting [353,56,63](353)) 2014-08-13 17:38:03.400245 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started 2014-08-13 17:38:03.400266 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400270 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400254 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [353,56](353) acting [353,56,63](353) maybe_went_rw) 2014-08-13 17:38:03.400265 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Start 2014-08-13 17:38:03.400273 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [353,56](353) acting [353,56](353)) 2014-08-13 17:38:03.400291 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400293 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400295 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9500 pg_notify(24.3d(31) epoch 194485) v5 2014-08-13 17:38:03.400283 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary 2014-08-13 17:38:03.400292 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,4,56,63,353,369 down blocked_by {} 2014-08-13 17:38:03.400305 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000039 0 0.000000 2014-08-13 17:38:03.400314 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.400335 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.400337 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9500 2014-08-13 17:38:03.400326 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 17:38:03.400341 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2719 2014-08-13 17:38:03.400345 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering 2014-08-13 17:38:03.400361 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-13 17:38:03.400363 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.400384 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400388 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400381 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.56 2014-08-13 17:38:03.400381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.400406 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400409 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400411 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca4c0 pg_notify(22.48e(29) epoch 194485) v5 2014-08-13 17:38:03.400401 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.63 2014-08-13 17:38:03.400403 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 17:38:03.400419 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.353 2014-08-13 17:38:03.400428 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [365,350,291](365) acting [291,350](291)) 2014-08-13 17:38:03.400436 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 17:38:03.400456 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.400459 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca4c0 2014-08-13 17:38:03.400450 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194479-194481 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 17:38:03.400455 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.400473 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.400472 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194476-194478 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 17:38:03.400511 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400498 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194473-194475 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 17:38:03.400515 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400519 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194471-194472 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 17:38:03.400536 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400539 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400542 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db5100 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.400549 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.400537 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 17:38:03.400554 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.400557 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400559 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400557 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.400574 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.400584 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400586 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400588 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8e00 pg_notify(24.b6(1) epoch 194485) v5 2014-08-13 17:38:03.400580 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] publish_stats_to_osd 194485:25285 2014-08-13 17:38:03.400599 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.291 2014-08-13 17:38:03.400632 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.327 2014-08-13 17:38:03.400635 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8e00 2014-08-13 17:38:03.400625 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.350 2014-08-13 17:38:03.400644 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.365 2014-08-13 17:38:03.400657 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4c71b80 don't know epoch, doing nothing 2014-08-13 17:38:03.400667 7fe99f5e9700 7 osd.4 194485 do_notify osd 20 on 1 PGs 2014-08-13 17:38:03.400661 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.367 2014-08-13 17:38:03.400676 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400679 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400679 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.378 2014-08-13 17:38:03.400699 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400700 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:03.400701 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400704 7fe99f5e9700 7 osd.4 194485 do_notify osd 328 on 1 PGs 2014-08-13 17:38:03.400704 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299c380 pg_query(22.1fb epoch 194485) v3 2014-08-13 17:38:03.400698 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.400725 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:03.400728 7fe99f5e9700 7 osd.4 194485 do_notify osd 330 on 1 PGs 2014-08-13 17:38:03.400718 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.400743 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.373 epoch 194485 2014-08-13 17:38:03.400746 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299c380 2014-08-13 17:38:03.400748 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8c60 don't know epoch, doing nothing 2014-08-13 17:38:03.400752 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.0 on 1 PGs 2014-08-13 17:38:03.400773 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4c77640 don't know epoch, doing nothing 2014-08-13 17:38:03.400758 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 373 2014-08-13 17:38:03.400776 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.56 on 1 PGs 2014-08-13 17:38:03.400800 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x52502c0 don't know epoch, doing nothing 2014-08-13 17:38:03.400803 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.63 on 1 PGs 2014-08-13 17:38:03.400804 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400807 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400822 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.400829 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400831 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400833 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299cec0 pg_query(24.181 epoch 194485) v3 2014-08-13 17:38:03.400865 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.320 epoch 194485 2014-08-13 17:38:03.400868 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299cec0 2014-08-13 17:38:03.400884 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_query query(info 0'0) from replica 320 2014-08-13 17:38:03.400921 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400923 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400941 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.400943 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.400944 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264440 pg_notify(22.3e4(31) epoch 194485) v5 2014-08-13 17:38:03.401125 7fe97b812700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.401150 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:03.410978 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.320 on 1 PGs 2014-08-13 17:38:03.410988 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.410993 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264440 2014-08-13 17:38:03.401158 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,30]/[320,30] -- 320/320 2014-08-13 17:38:03.411023 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing 2014-08-13 17:38:03.411027 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.344 on 1 PGs 2014-08-13 17:38:03.411029 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411047 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x57570c0 don't know epoch, doing nothing 2014-08-13 17:38:03.411050 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.353 sending saperate messages on 1 PGs 2014-08-13 17:38:03.411065 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411067 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4a680 don't know epoch, doing nothing 2014-08-13 17:38:03.411069 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411070 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.363 on 1 PGs 2014-08-13 17:38:03.411047 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411074 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.411096 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x53856a0 don't know epoch, doing nothing 2014-08-13 17:38:03.411098 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411099 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.366 on 1 PGs 2014-08-13 17:38:03.411100 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411091 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,30] newacting [320,30] 2014-08-13 17:38:03.411103 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265940 pg_notify(22.1d2(27) epoch 194485) v5 2014-08-13 17:38:03.411120 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5250840 don't know epoch, doing nothing 2014-08-13 17:38:03.411124 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.369 on 1 PGs 2014-08-13 17:38:03.411110 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.411127 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,4,30] r=1 lpr=194478 pi=192979-194476/26 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.411147 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.330 2014-08-13 17:38:03.411150 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265940 2014-08-13 17:38:03.411169 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:03.411173 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.373 on 1 PGs 2014-08-13 17:38:03.411162 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [320,4,30](320) acting [320,4,30](320)): not rw, up_thru 194477 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194477-194478 up [320,4,30](320) acting [320,4,30](320) maybe_went_rw) : primary up 192979-194477 includes interval 2014-08-13 17:38:03.411192 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194476/194477/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194477-194478 up [320,4,30](320) acting [320,4,30](320) maybe_went_rw) 2014-08-13 17:38:03.411209 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411213 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411209 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,4,30] -> [320,30], acting [320,4,30] -> [320,30], acting_primary 320 -> 320, up_primary 320 -> 320, role 1 -> -1 2014-08-13 17:38:03.411229 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411243 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.411258 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.411251 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 17:38:03.411284 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411274 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.411289 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.411292 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.411306 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.411309 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327] newacting [294,327] 2014-08-13 17:38:03.411321 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411326 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.411337 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411342 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411344 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc280 pg_notify(22.481(39) epoch 194485) v5 2014-08-13 17:38:03.411339 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.411344 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.226187 3 0.000122 2014-08-13 17:38:03.411357 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.411367 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started 0.226248 0 0.000000 2014-08-13 17:38:03.411375 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.411388 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.365 2014-08-13 17:38:03.411392 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc280 2014-08-13 17:38:03.411388 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.411390 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.411406 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.411405 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194477 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.411423 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.411425 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194479 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411451 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411454 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411444 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194479 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.411446 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.411460 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194479 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327] newacting [294,327] 2014-08-13 17:38:03.411475 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411478 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411480 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299aee0 pg_query(24.299 epoch 194485) v3 2014-08-13 17:38:03.411476 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194479 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.411488 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,30]/[320,30] -- 320/320 2014-08-13 17:38:03.411494 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327,4] r=2 lpr=194479 pi=192977-194475/21 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.411510 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411522 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.396 epoch 194485 2014-08-13 17:38:03.411525 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299aee0 2014-08-13 17:38:03.411515 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411531 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.411528 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [294,327,4](294) acting [294,327,4](294)): not rw, up_thru 194476 up_from 192968 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [294,327,4](294) acting [294,327,4](294) maybe_went_rw) : primary up 192968-194476 includes interval 2014-08-13 17:38:03.411538 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 396 2014-08-13 17:38:03.411549 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.411548 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [294,327,4](294) acting [294,327,4](294) maybe_went_rw) 2014-08-13 17:38:03.411568 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327,4] -> [294,327], acting [294,327,4] -> [294,327], acting_primary 294 -> 294, up_primary 294 -> 294, role 2 -> -1 2014-08-13 17:38:03.411594 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411582 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,30]/[320,30] -- 320/320 2014-08-13 17:38:03.411598 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411587 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411601 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411617 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411603 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.411605 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411620 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411622 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd940 pg_notify(1.2b(2) epoch 194485) v5 2014-08-13 17:38:03.411620 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.411621 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.411640 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.411639 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.411663 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.370 2014-08-13 17:38:03.411666 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd940 2014-08-13 17:38:03.411657 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.411672 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.411681 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 17:38:03.411688 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411708 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411712 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.411712 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411727 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411730 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411729 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.411729 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.411747 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.411749 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,12,343] newacting [320,12,343] 2014-08-13 17:38:03.411764 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411767 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411770 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5103800 pg_notify(24.30b(38) epoch 194485) v5 2014-08-13 17:38:03.411762 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.411767 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.411778 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.411785 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,30] r=-1 lpr=194479 pi=192979-194478/27 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.411794 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.411812 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.375 2014-08-13 17:38:03.411815 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5103800 2014-08-13 17:38:03.411813 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.411819 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [320,30](320) acting [320,30](320)): not rw, up_thru 194479 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [320,30](320) acting [320,30](320) maybe_went_rw) : primary up 192979-194479 includes interval 2014-08-13 17:38:03.411839 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194479/194479/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [320,30](320) acting [320,30](320) maybe_went_rw) 2014-08-13 17:38:03.411855 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411858 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411852 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 17:38:03.411872 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411857 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,30] -> [320,12,343], acting [320,30] -> [320,12,343], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 17:38:03.411889 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.411877 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411876 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411892 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411895 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299d820 pg_query(22.653 epoch 194485) v3 2014-08-13 17:38:03.411893 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.411893 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.411911 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.411911 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.411932 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.371 epoch 194485 2014-08-13 17:38:03.411936 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299d820 2014-08-13 17:38:03.411928 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.411942 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.411950 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 17:38:03.411947 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 371 2014-08-13 17:38:03.411970 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.411958 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.411975 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.411976 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.411998 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412001 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.411991 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.411993 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.412007 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412009 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.412021 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412028 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412031 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce3c0 pg_notify(22.1f1(29) epoch 194485) v5 2014-08-13 17:38:03.412026 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.412041 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.412041 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 17:38:03.412063 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412070 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.368 2014-08-13 17:38:03.412073 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce3c0 2014-08-13 17:38:03.412061 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412068 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412089 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.412117 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412104 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327,363] newacting [294,327,363] 2014-08-13 17:38:03.412120 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412102 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 17:38:03.412125 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412121 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.412129 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412147 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412150 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412137 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327] r=-1 lpr=194479 pi=192977-194478/22 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.412152 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513de80 pg_notify(22.4df(27) epoch 194485) v5 2014-08-13 17:38:03.412145 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.412168 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [294,327](294) acting [294,327](294)): not rw, up_thru 194479 up_from 192968 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [294,327](294) acting [294,327](294) maybe_went_rw) : primary up 192968-194479 includes interval 2014-08-13 17:38:03.412164 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412201 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.380 2014-08-13 17:38:03.412204 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513de80 2014-08-13 17:38:03.412186 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [294,327](294) acting [294,327](294) maybe_went_rw) 2014-08-13 17:38:03.412209 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 17:38:03.412229 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412213 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327] -> [294,327,363], acting [294,327] -> [294,327,363], acting_primary 294 -> 294, up_primary 294 -> 294, role -1 -> -1 2014-08-13 17:38:03.412233 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412233 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412249 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412252 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412250 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.412250 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.412280 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412268 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.412283 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412269 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412287 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143480 pg_notify(24.1f7(37) epoch 194485) v5 2014-08-13 17:38:03.412285 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.412299 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.412305 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 17:38:03.412325 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412326 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.351 2014-08-13 17:38:03.412315 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.412330 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143480 2014-08-13 17:38:03.412331 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412330 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412348 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.412349 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.412367 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,4,30] newacting [320,4,30] 2014-08-13 17:38:03.412380 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412371 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.412386 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412384 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.412387 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.412401 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,12,343] r=-1 lpr=194482 pi=192979-194481/28 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.412402 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.412420 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412423 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412425 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143800 pg_notify(22.f3(23) epoch 194485) v5 2014-08-13 17:38:03.412419 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412433 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [320,12,343](320) acting [320,12,343](320)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194482-194484 up [320,12,343](320) acting [320,12,343](320) maybe_went_rw) : primary up 192979-194483 includes interval 2014-08-13 17:38:03.412465 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.412449 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194482/194482/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [320,12,343](320) acting [320,12,343](320) maybe_went_rw) 2014-08-13 17:38:03.412467 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143800 2014-08-13 17:38:03.412457 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 17:38:03.412481 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412469 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,12,343] -> [320,4,30], acting [320,12,343] -> [320,4,30], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> 1 2014-08-13 17:38:03.412486 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412492 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412507 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412511 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412502 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.412507 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.412520 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412523 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.412535 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412538 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412540 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51432c0 pg_notify(22.1b8(25) epoch 194485) v5 2014-08-13 17:38:03.412538 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.412553 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.412552 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 17:38:03.412574 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412579 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.7 2014-08-13 17:38:03.412582 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51432c0 2014-08-13 17:38:03.412571 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.412578 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412588 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412595 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.412606 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.412612 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412630 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412633 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412625 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.412640 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.412642 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 17:38:03.412661 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.412654 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.412666 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412670 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412672 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5142f40 pg_notify(22.775(31) epoch 194485) v5 2014-08-13 17:38:03.412665 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.412669 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.412682 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.412686 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.412714 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.63 2014-08-13 17:38:03.412717 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5142f40 2014-08-13 17:38:03.412704 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.412699 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327,4] newacting [294,327,4] 2014-08-13 17:38:03.412724 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.412724 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.412743 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,363] r=-1 lpr=194482 pi=192977-194481/23 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.412760 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412763 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412771 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [294,327,363](294) acting [294,327,363](294)): not rw, up_thru 194482 up_from 192968 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [294,327,363](294) acting [294,327,363](294) maybe_went_rw) : primary up 192968-194482 includes interval 2014-08-13 17:38:03.412793 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412795 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412797 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102840 pg_notify(22.362(31) epoch 194485) v5 2014-08-13 17:38:03.412787 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [294,327,363](294) acting [294,327,363](294) maybe_went_rw) 2014-08-13 17:38:03.412795 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.412807 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327,363] -> [294,327,4], acting [294,327,363] -> [294,327,4], acting_primary 294 -> 294, up_primary 294 -> 294, role -1 -> 2 2014-08-13 17:38:03.412816 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.412838 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.349 2014-08-13 17:38:03.412828 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412841 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102840 2014-08-13 17:38:03.412835 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.705387 9 0.001438 2014-08-13 17:38:03.412844 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.412858 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.412861 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.412877 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.412877 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.412890 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412897 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412895 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.412897 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.412911 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.412923 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.412926 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.412928 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513d780 pg_notify(22.4df(1) epoch 194485) v5 2014-08-13 17:38:03.412918 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 2014-08-13 17:38:03.412926 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.412939 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.412943 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.412967 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.352 2014-08-13 17:38:03.412971 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513d780 2014-08-13 17:38:03.412959 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.412966 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.412976 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.412986 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.412992 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.413006 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.413018 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413022 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413020 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.413041 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413043 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413045 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5140540 pg_notify(22.39d(1) epoch 194485) v5 2014-08-13 17:38:03.413036 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.413088 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.413053 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.413106 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.366 2014-08-13 17:38:03.413109 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5140540 2014-08-13 17:38:03.413151 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413138 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.413154 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413156 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.413174 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413143 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 17:38:03.413177 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413179 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e040 pg_notify(22.4df epoch 194485) v5 2014-08-13 17:38:03.413180 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.413173 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.001784 7 0.001505 2014-08-13 17:38:03.413188 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.413202 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.413208 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.413230 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.357 2014-08-13 17:38:03.413220 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.413233 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e040 2014-08-13 17:38:03.413227 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.413238 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.413247 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.236988 2 0.000211 2014-08-13 17:38:03.413260 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000040 0 0.000000 2014-08-13 17:38:03.413281 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413284 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413270 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.413281 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.413289 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.237041 0 0.000000 2014-08-13 17:38:03.413305 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413308 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413310 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90240 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.413305 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.413319 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.413324 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.413311 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.237072 0 0.000000 2014-08-13 17:38:03.413327 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413332 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413322 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.413333 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started 0.237139 0 0.000000 2014-08-13 17:38:03.413361 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413363 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413365 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16e40 pg_notify(24.30b(34) epoch 194485) v5 2014-08-13 17:38:03.413358 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.413373 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194477 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.413403 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.10 2014-08-13 17:38:03.413390 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194479 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.413407 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16e40 2014-08-13 17:38:03.413413 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.413408 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194479 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.413425 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194479 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [312,285] newacting [312,285] 2014-08-13 17:38:03.413448 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413451 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413440 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194479 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.413456 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [4,312,285] r=0 lpr=194479 pi=193306-194475/21 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.413462 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 17:38:03.413484 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413487 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413489 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513d080 pg_notify(22.84(25) epoch 194485) v5 2014-08-13 17:38:03.413494 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.413494 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,312,285](4) acting [4,312,285](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194465 generate_past_intervals interval(194476-194478 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.413501 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Started advmap 2014-08-13 17:38:03.413511 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194476/194476/194476) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] noting past interval(194476-194478 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 17:38:03.413532 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.304 2014-08-13 17:38:03.413535 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513d080 2014-08-13 17:38:03.413525 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,304] newacting [346,288] 2014-08-13 17:38:03.413529 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] up [4,312,285] -> [312,285], acting [4,312,285] -> [312,285], acting_primary 4 -> 312, up_primary 4 -> 312, role 0 -> -1 2014-08-13 17:38:03.413544 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.413550 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.413575 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413578 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413567 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.413563 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started/Stray 0.228122 3 0.000159 2014-08-13 17:38:03.413583 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.413601 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413603 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413586 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started 0.228186 0 0.000000 2014-08-13 17:38:03.413605 7fe9a63f4700 20 osd.4 194485 _dispatch 0x2999a40 pg_query(24.3b7 epoch 194485) v3 2014-08-13 17:38:03.413597 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.413608 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset 2014-08-13 17:38:03.413615 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.413627 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194477 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194479 2014-08-13 17:38:03.413643 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.356 epoch 194485 2014-08-13 17:38:03.413632 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.413646 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x2999a40 2014-08-13 17:38:03.413645 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194479 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.413648 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.413658 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 356 2014-08-13 17:38:03.413665 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194479 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.413667 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.413687 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.413683 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194479 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,304] newacting [346,288] 2014-08-13 17:38:03.413705 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.413704 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194479 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.413721 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.413734 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413722 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,4,304]/[346,4,288] r=1 lpr=194479 pi=193125-194475/28 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194479 2014-08-13 17:38:03.413738 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413736 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.413758 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413765 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413753 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.413768 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d918c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.413755 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [346,4,304](346) acting [346,4,288](346)): not rw, up_thru 194476 up_from 192955 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [346,4,304](346) acting [346,4,288](346) maybe_went_rw) : primary up 192955-194476 includes interval 2014-08-13 17:38:03.413775 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.413780 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.413783 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413785 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413769 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.413775 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194476/194476/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194476-194478 up [346,4,304](346) acting [346,4,288](346) maybe_went_rw) 2014-08-13 17:38:03.413787 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.413806 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413808 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413811 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d90900 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.413797 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,4,304] -> [346,304], acting [346,4,288] -> [346,288], acting_primary 346 -> 346, up_primary 346 -> 346, role 1 -> -1 2014-08-13 17:38:03.413804 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.413816 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.413821 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.413824 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413826 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413819 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.413819 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:03.413839 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.413854 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413839 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.413857 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413859 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8000 pg_notify(22.362(1) epoch 194485) v5 2014-08-13 17:38:03.413858 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.413877 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 17:38:03.413876 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 17:38:03.413898 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.413899 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.413903 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8000 2014-08-13 17:38:03.413895 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 17:38:03.413903 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.413912 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 17:38:03.413920 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.413941 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413944 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413929 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:03.413938 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.413963 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.413949 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 17:38:03.413965 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.413967 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ccd00 pg_notify(22.48e(1) epoch 194485) v5 2014-08-13 17:38:03.413968 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 17:38:03.413970 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 17:38:03.413994 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.413985 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_role_change 2014-08-13 17:38:03.413998 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.414013 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.289 2014-08-13 17:38:03.414001 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.414016 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ccd00 2014-08-13 17:38:03.414014 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.414018 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 17:38:03.414032 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414035 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 17:38:03.414054 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414057 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414053 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414075 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414077 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414079 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265cc0 pg_notify(24.1bb(34) epoch 194485) v5 2014-08-13 17:38:03.414071 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 17:38:03.414090 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414089 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 17:38:03.414095 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.414117 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.337 2014-08-13 17:38:03.414119 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265cc0 2014-08-13 17:38:03.414119 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414114 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.414125 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.414130 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] acting_up_affected newup [312,311,369] newacting [312,311,369] 2014-08-13 17:38:03.414143 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.414147 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.414167 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414170 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414162 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414162 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,285] r=-1 lpr=194479 pi=193306-194478/22 crt=191648'343 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.414195 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414197 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414199 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260540 pg_notify(22.362 epoch 194485) v5 2014-08-13 17:38:03.414200 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [312,285](312) acting [312,285](312)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) : primary up 192974-194479 includes interval 2014-08-13 17:38:03.414203 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 17:38:03.414220 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194479/194479/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 17:38:03.414234 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414239 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.343 2014-08-13 17:38:03.414242 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260540 2014-08-13 17:38:03.414239 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.414237 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] up [312,285] -> [312,311,369], acting [312,285] -> [312,311,369], acting_primary 312 -> 312, up_primary 312 -> 312, role -1 -> -1 2014-08-13 17:38:03.414259 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.414258 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.414277 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414281 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414276 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.414278 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414299 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414301 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414303 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc440 pg_notify(22.39d(29) epoch 194485) v5 2014-08-13 17:38:03.414293 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.414310 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.414318 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,288,369]/[346,288] -- 346/346 2014-08-13 17:38:03.414326 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.414343 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.363 2014-08-13 17:38:03.414344 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414347 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc440 2014-08-13 17:38:03.414341 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.414349 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.414360 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.414368 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.414385 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414389 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414378 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.414386 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] acting_up_affected newup [346,288,369] newacting [346,288] 2014-08-13 17:38:03.414396 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.414414 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414416 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414404 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.414419 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4263f00 pg_notify(1.2b(1) epoch 194485) v5 2014-08-13 17:38:03.414412 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.414422 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,304]/[346,288] r=-1 lpr=194479 pi=193125-194478/29 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] set_last_peering_reset 194482 2014-08-13 17:38:03.414429 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.414447 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414462 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.324 2014-08-13 17:38:03.414465 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4263f00 2014-08-13 17:38:03.414454 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [346,304](346) acting [346,288](346)): not rw, up_thru 194479 up_from 192955 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [346,304](346) acting [346,288](346) maybe_went_rw) : primary up 192955-194479 includes interval 2014-08-13 17:38:03.414472 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194479/194479/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194479-194481 up [346,304](346) acting [346,288](346) maybe_went_rw) 2014-08-13 17:38:03.414487 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 17:38:03.414491 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,304] -> [346,288,369], acting [346,288] -> [346,288], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> -1 2014-08-13 17:38:03.414513 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414526 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414529 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414517 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.414513 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:03.414535 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.414537 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.414554 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414557 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414559 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9340 pg_notify(22.3e4(1) epoch 194485) v5 2014-08-13 17:38:03.414552 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414553 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.414577 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 17:38:03.414599 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.414602 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9340 2014-08-13 17:38:03.414589 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 17:38:03.414593 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 17:38:03.414610 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414610 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 17:38:03.414614 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.414627 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:03.414630 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.414652 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414655 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414646 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414645 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 17:38:03.414674 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414676 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414664 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 17:38:03.414678 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca300 pg_notify(24.1f7(33) epoch 194485) v5 2014-08-13 17:38:03.414677 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 17:38:03.414680 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 17:38:03.414697 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414699 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 17:38:03.414701 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.414722 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.414725 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca300 2014-08-13 17:38:03.414722 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.414722 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.414741 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,312,285] newacting [4,312,285] 2014-08-13 17:38:03.414758 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.414775 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414779 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414765 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,288,369]/[346,288,369] -- 346/346 2014-08-13 17:38:03.414773 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [312,311,369] r=-1 lpr=194482 pi=193306-194481/23 crt=191648'343 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.414792 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.414798 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414801 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414803 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264b40 pg_notify(22.3d9(16) epoch 194485) v5 2014-08-13 17:38:03.414797 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.414806 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [312,311,369](312) acting [312,311,369](312)): not rw, up_thru 194482 up_from 192974 last_epoch_clean 194465 generate_past_intervals interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) : primary up 192974-194482 includes interval 2014-08-13 17:38:03.414818 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.414825 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194482/194482/194479) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 17:38:03.414844 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.397 2014-08-13 17:38:03.414847 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264b40 2014-08-13 17:38:03.414836 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] acting_up_affected newup [346,288,369] newacting [346,288,369] 2014-08-13 17:38:03.414844 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [312,311,369] -> [4,312,285], acting [312,311,369] -> [4,312,285], acting_primary 312 -> 4, up_primary 312 -> 4, role -1 -> 0 2014-08-13 17:38:03.414856 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.414866 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.414887 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414874 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369]/[346,288] r=-1 lpr=194482 pi=193125-194481/30 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] set_last_peering_reset 194483 2014-08-13 17:38:03.414890 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414883 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.414908 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.414911 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414900 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.414913 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cad80 pg_notify(22.3d9(42) epoch 194485) v5 2014-08-13 17:38:03.414905 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [346,288,369](346) acting [346,288](346)): not rw, up_thru 194479 up_from 192955 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [346,288,369](346) acting [346,288](346)) : primary up 192955-194479 does not include interval 2014-08-13 17:38:03.414919 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2857 2014-08-13 17:38:03.414924 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194482/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] noting past interval(194482-194482 up [346,288,369](346) acting [346,288](346)) 2014-08-13 17:38:03.414937 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.414952 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.373 2014-08-13 17:38:03.414955 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cad80 2014-08-13 17:38:03.414944 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] up [346,288,369] -> [346,288,369], acting [346,288] -> [346,288,369], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> -1 2014-08-13 17:38:03.414955 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.414965 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 17:38:03.414973 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.414983 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.414991 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.415005 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415008 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.414999 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.415006 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.415026 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415028 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415016 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_copy_ops 2014-08-13 17:38:03.415030 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cbf00 pg_notify(22.39d(29) epoch 194485) v5 2014-08-13 17:38:03.415022 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.415032 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_flush_ops 2014-08-13 17:38:03.415039 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.415048 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change_cleanup 2014-08-13 17:38:03.415056 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.415074 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.320 2014-08-13 17:38:03.415077 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cbf00 2014-08-13 17:38:03.415065 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 17:38:03.415074 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.415083 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit NotTrimming 2014-08-13 17:38:03.415092 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.415101 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter NotTrimming 2014-08-13 17:38:03.415120 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415110 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.415124 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415117 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_recovery 2014-08-13 17:38:03.415142 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415145 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415126 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.415133 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] clear_recovery_state 2014-08-13 17:38:03.415148 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264280 pg_notify(22.707(33) epoch 194485) v5 2014-08-13 17:38:03.415148 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.415152 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.415165 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.415191 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.415197 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.415210 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264280 2014-08-13 17:38:03.415189 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,288,369]/[346,288,369] -- 346/346 2014-08-13 17:38:03.415208 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.415224 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.415227 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.415230 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.415246 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.415260 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415247 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.415264 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415266 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.415283 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415285 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415268 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,285,312 unchanged 2014-08-13 17:38:03.415287 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51439c0 pg_notify(22.775(31) epoch 194485) v5 2014-08-13 17:38:03.415288 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.415297 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,304] -- 346/346 2014-08-13 17:38:03.415306 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001949 7 0.001790 2014-08-13 17:38:03.415325 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.415331 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.0 2014-08-13 17:38:03.415334 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51439c0 2014-08-13 17:38:03.415327 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.415331 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:03.415344 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.415348 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.415360 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.415374 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415379 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415368 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,304] 2014-08-13 17:38:03.415381 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 2014-08-13 17:38:03.415386 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.415403 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415406 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415408 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513db00 pg_notify(22.4df(27) epoch 194485) v5 2014-08-13 17:38:03.415401 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.415402 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,288,369] r=-1 lpr=194483 pi=193125-194482/31 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194485 2014-08-13 17:38:03.415422 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.415446 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.371 2014-08-13 17:38:03.415449 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513db00 2014-08-13 17:38:03.415436 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [346,288,369](346) acting [346,288,369](346)): not rw, up_thru 194483 up_from 192955 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [346,288,369](346) acting [346,288,369](346) maybe_went_rw) : primary up 192955-194483 includes interval 2014-08-13 17:38:03.415439 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.415453 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194482/194483/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] noting past interval(194483-194484 up [346,288,369](346) acting [346,288,369](346) maybe_went_rw) 2014-08-13 17:38:03.415457 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.415485 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415488 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415473 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] up [346,288,369] -> [346,4,304], acting [346,288,369] -> [346,4,304], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> 1 2014-08-13 17:38:03.415477 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 17:38:03.415494 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 17:38:03.415498 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 17:38:03.415516 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.415519 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.415521 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143640 pg_notify(22.775(1) epoch 194485) v5 2014-08-13 17:38:03.415514 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.415518 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 17:38:03.415530 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.415543 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 17:38:03.425191 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_copy_ops 2014-08-13 17:38:03.423553 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.425227 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.425232 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143640 2014-08-13 17:38:03.425229 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_flush_ops 2014-08-13 17:38:03.425210 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 17:38:03.425247 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change_cleanup 2014-08-13 17:38:03.425251 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,311,312,369 down blocked_by {} 2014-08-13 17:38:03.425264 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 17:38:03.425273 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.425291 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425294 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425283 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit NotTrimming 2014-08-13 17:38:03.425301 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter NotTrimming 2014-08-13 17:38:03.425299 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2858 2014-08-13 17:38:03.425319 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425322 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425324 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143100 pg_notify(22.84(2) epoch 194485) v5 2014-08-13 17:38:03.425317 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_role_change 2014-08-13 17:38:03.425320 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.285 2014-08-13 17:38:03.425334 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.425348 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.311 2014-08-13 17:38:03.425355 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_recovery 2014-08-13 17:38:03.425371 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.7 2014-08-13 17:38:03.425376 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143100 2014-08-13 17:38:03.425369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.312 2014-08-13 17:38:03.425371 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] clear_recovery_state 2014-08-13 17:38:03.425389 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 17:38:03.425391 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.425416 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425419 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425412 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map 2014-08-13 17:38:03.425411 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.425444 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425446 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425434 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.425448 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51440c0 pg_notify(22.775 epoch 194485) v5 2014-08-13 17:38:03.425483 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.353 2014-08-13 17:38:03.425486 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51440c0 2014-08-13 17:38:03.425484 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] update_heartbeat_peers unchanged 2014-08-13 17:38:03.425507 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] take_waiters 2014-08-13 17:38:03.425527 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425530 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425536 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.425528 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Reset 0.011920 7 0.011581 2014-08-13 17:38:03.425553 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425555 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425558 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513d400 pg_notify(22.4df epoch 194485) v5 2014-08-13 17:38:03.425554 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started 2014-08-13 17:38:03.425571 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Start 2014-08-13 17:38:03.425599 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.356 2014-08-13 17:38:03.425586 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: transitioning to Stray 2014-08-13 17:38:03.425602 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513d400 2014-08-13 17:38:03.425597 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 17:38:03.425607 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Start 0.000035 0 0.000000 2014-08-13 17:38:03.425627 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.425637 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425640 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425628 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started/Stray 2014-08-13 17:38:03.425635 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.425658 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425660 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425662 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5140000 pg_notify(22.39d epoch 194485) v5 2014-08-13 17:38:03.425651 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.425652 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,374] newacting [345,374] 2014-08-13 17:38:03.425671 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.425670 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.425700 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.344 2014-08-13 17:38:03.425703 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5140000 2014-08-13 17:38:03.425691 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.246807 3 0.000184 2014-08-13 17:38:03.425714 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started 0.246868 0 0.000000 2014-08-13 17:38:03.425736 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425739 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425735 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.425756 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425758 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425760 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5142bc0 pg_notify(22.775 epoch 194485) v5 2014-08-13 17:38:03.425750 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194477 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.425775 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.425767 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194479 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.425784 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194479 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.425798 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.56 2014-08-13 17:38:03.425801 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5142bc0 2014-08-13 17:38:03.425799 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194479 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,374] newacting [345,374] 2014-08-13 17:38:03.425815 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194479 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.425832 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425835 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425830 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,4,374] r=1 lpr=194479 pi=192981-194475/22 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.425826 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 17:38:03.425854 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425853 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.425856 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425858 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9a40 pg_notify(22.481(39) epoch 194485) v5 2014-08-13 17:38:03.425859 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.425864 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [345,4,374](345) acting [345,4,374](345)): not rw, up_thru 194476 up_from 192981 last_epoch_clean 194465 generate_past_intervals interval(194476-194478 up [345,4,374](345) acting [345,4,374](345) maybe_went_rw) : primary up 192981-194476 includes interval 2014-08-13 17:38:03.425877 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289] newacting [334,289] 2014-08-13 17:38:03.425881 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194476/194476/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [345,4,374](345) acting [345,4,374](345) maybe_went_rw) 2014-08-13 17:38:03.425901 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.425904 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9a40 2014-08-13 17:38:03.425894 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.425903 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,4,374] -> [345,374], acting [345,4,374] -> [345,374], acting_primary 345 -> 345, up_primary 345 -> 345, role 1 -> -1 2014-08-13 17:38:03.425916 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.239744 3 0.000129 2014-08-13 17:38:03.425925 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.425944 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425947 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425938 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started 0.239807 0 0.000000 2014-08-13 17:38:03.425940 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.425965 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.425968 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.425956 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.425957 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.425970 7fe9a63f4700 20 osd.4 194485 _dispatch 0x299c560 pg_query(22.226 epoch 194485) v3 2014-08-13 17:38:03.425974 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.425973 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194477 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.425991 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.426009 7fe9a63f4700 7 osd.4 194485 handle_pg_query from osd.376 epoch 194485 2014-08-13 17:38:03.425993 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194479 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426012 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x299c560 2014-08-13 17:38:03.426008 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.426014 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194479 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.426025 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426032 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194479 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289] newacting [334,289] 2014-08-13 17:38:03.426024 7fe9a63f4700 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 376 2014-08-13 17:38:03.426045 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.426048 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194479 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.426074 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426062 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.426077 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426065 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289,4] r=2 lpr=194479 pi=192974-194475/21 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.426079 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.426095 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426098 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426100 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cabc0 pg_notify(22.39d epoch 194485) v5 2014-08-13 17:38:03.426093 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.426097 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [334,289,4](334) acting [334,289,4](334)): not rw, up_thru 194476 up_from 192946 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [334,289,4](334) acting [334,289,4](334) maybe_went_rw) : primary up 192946-194476 includes interval 2014-08-13 17:38:03.426109 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.426114 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [334,289,4](334) acting [334,289,4](334) maybe_went_rw) 2014-08-13 17:38:03.426124 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.426139 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.373 2014-08-13 17:38:03.426142 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cabc0 2014-08-13 17:38:03.426133 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289,4] -> [334,289], acting [334,289,4] -> [334,289], acting_primary 334 -> 334, up_primary 334 -> 334, role 2 -> -1 2014-08-13 17:38:03.426144 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426154 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426170 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.426182 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426194 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426185 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.426215 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426217 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426198 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 17:38:03.426219 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5144280 pg_notify(22.39d(29) epoch 194485) v5 2014-08-13 17:38:03.426211 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.426222 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426227 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.426231 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426243 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.426259 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.320 2014-08-13 17:38:03.426248 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.426263 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5144280 2014-08-13 17:38:03.426260 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426267 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426278 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.426299 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426302 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426295 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.426300 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 17:38:03.426320 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426309 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.426325 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426328 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426331 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143f00 pg_notify(22.775(1) epoch 194485) v5 2014-08-13 17:38:03.426324 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.426324 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426342 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.426344 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.426357 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.426371 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.426373 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143f00 2014-08-13 17:38:03.426362 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426375 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426407 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.426410 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.426401 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 17:38:03.426420 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426416 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 17:38:03.426424 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426437 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426438 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.426441 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426453 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,378,304] newacting [345,378,304] 2014-08-13 17:38:03.426457 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.426470 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.426472 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426485 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,374] r=-1 lpr=194479 pi=192981-194478/23 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.426502 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 17:38:03.426520 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426512 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [345,374](345) acting [345,374](345)): not rw, up_thru 194479 up_from 192981 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [345,374](345) acting [345,374](345) maybe_went_rw) : primary up 192981-194479 includes interval 2014-08-13 17:38:03.426524 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426529 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194479/194479/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [345,374](345) acting [345,374](345) maybe_went_rw) 2014-08-13 17:38:03.426539 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.426547 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,374] -> [345,378,304], acting [345,374] -> [345,378,304], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> -1 2014-08-13 17:38:03.426555 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426566 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426580 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.426588 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 17:38:03.426607 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426595 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.426611 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426614 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.426626 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.426630 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.426641 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289,65] newacting [334,289,65] 2014-08-13 17:38:03.426646 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.426657 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.426662 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426672 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289] r=-1 lpr=194479 pi=192974-194478/22 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.426680 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.426698 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.426702 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [334,289](334) acting [334,289](334)): not rw, up_thru 194479 up_from 192946 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [334,289](334) acting [334,289](334) maybe_went_rw) : primary up 192946-194479 includes interval 2014-08-13 17:38:03.426714 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.426719 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [334,289](334) acting [334,289](334) maybe_went_rw) 2014-08-13 17:38:03.426729 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.426737 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289] -> [334,289,65], acting [334,289] -> [334,289,65], acting_primary 334 -> 334, up_primary 334 -> 334, role -1 -> -1 2014-08-13 17:38:03.426747 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426755 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426770 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.426784 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.426787 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 17:38:03.426807 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426799 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.426812 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426815 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.426828 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.426829 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.426844 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.426844 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426860 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.426875 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.426882 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 17:38:03.426890 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.426901 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426904 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.426906 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426921 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426925 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.426941 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.426956 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 17:38:03.426974 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426970 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 17:38:03.426989 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.426978 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426993 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.426998 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.427008 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.427015 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.427024 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,4,374] newacting [345,4,374] 2014-08-13 17:38:03.427040 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.427049 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 17:38:03.427055 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,378,304] r=-1 lpr=194482 pi=192981-194481/24 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.427068 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.427072 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.427083 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [345,378,304](345) acting [345,378,304](345)): not rw, up_thru 194482 up_from 192981 last_epoch_clean 194465 generate_past_intervals interval(194482-194484 up [345,378,304](345) acting [345,378,304](345) maybe_went_rw) : primary up 192981-194482 includes interval 2014-08-13 17:38:03.427087 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.427100 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194482/194482/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [345,378,304](345) acting [345,378,304](345) maybe_went_rw) 2014-08-13 17:38:03.427108 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.427118 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,378,304] -> [345,4,374], acting [345,378,304] -> [345,4,374], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> 1 2014-08-13 17:38:03.427137 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.427138 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 17:38:03.427158 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.427152 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.427162 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.427167 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.427177 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.427192 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.427199 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289,4] newacting [334,289,4] 2014-08-13 17:38:03.427207 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.427217 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.427223 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.427233 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,65] r=-1 lpr=194482 pi=192974-194481/23 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.427238 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.427254 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.427263 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [334,289,65](334) acting [334,289,65](334)): not rw, up_thru 194482 up_from 192946 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [334,289,65](334) acting [334,289,65](334) maybe_went_rw) : primary up 192946-194482 includes interval 2014-08-13 17:38:03.427271 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.427281 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [334,289,65](334) acting [334,289,65](334) maybe_went_rw) 2014-08-13 17:38:03.427287 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.427301 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.427299 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289,65] -> [334,289,4], acting [334,289,65] -> [334,289,4], acting_primary 334 -> 334, up_primary 334 -> 334, role -1 -> 2 2014-08-13 17:38:03.427315 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.427319 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.427330 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.427335 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.427347 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.427350 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.427365 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.427365 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.427381 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.427398 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.427412 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.427426 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.427431 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.427442 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.427454 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.427457 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.427471 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.427470 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.001736 7 0.001447 2014-08-13 17:38:03.427486 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.427492 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.427502 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.427509 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.427520 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.427525 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.427538 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.427544 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 2014-08-13 17:38:03.427564 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.427584 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.427592 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.427601 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.427610 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.427627 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.001671 7 0.001434 2014-08-13 17:38:03.427647 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.427662 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.427675 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.427688 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.427691 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 2014-08-13 17:38:03.427709 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.427728 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.427744 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.427739 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_advance_map [25,362]/[20,303] -- 25/20 2014-08-13 17:38:03.427776 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.427783 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Peering advmap 2014-08-13 17:38:03.427801 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.427824 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.427818 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.427837 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetInfo 0.245921 2 0.000215 2014-08-13 17:38:03.427860 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Leaving Peering 2014-08-13 17:38:03.427878 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering 0.245971 0 0.000000 2014-08-13 17:38:03.427873 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [313,318]/[313,318] -- 313/313 2014-08-13 17:38:03.427898 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.427899 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.246003 0 0.000000 2014-08-13 17:38:03.427904 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.427920 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.427920 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started 0.246065 0 0.000000 2014-08-13 17:38:03.427937 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [313,318] newacting [313,318] 2014-08-13 17:38:03.427942 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset 2014-08-13 17:38:03.427953 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.427960 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194477 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194479 2014-08-13 17:38:03.427970 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [4,313,318] r=0 lpr=194478 pi=193306-194476/27 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.427979 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194479 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Reset advmap 2014-08-13 17:38:03.427997 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194479 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428004 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,313,318](4) acting [4,313,318](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.428014 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194479 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [25,362] newacting [20,303] 2014-08-13 17:38:03.428021 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194476/194477/194477) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] noting past interval(194477-194478 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) 2014-08-13 17:38:03.428033 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194479 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.428038 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] up [4,313,318] -> [313,318], acting [4,313,318] -> [313,318], acting_primary 4 -> 313, up_primary 4 -> 313, role 0 -> -1 2014-08-13 17:38:03.428050 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [4,25,362]/[4,20,303] r=0 lpr=194479 pi=193635-194475/27 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194479 2014-08-13 17:38:03.428057 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.428072 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.428086 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.428084 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,25,362](4) acting [4,20,303](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.428100 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.428102 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194476/194476/194476) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] noting past interval(194476-194478 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 17:38:03.428116 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.428122 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] up [4,25,362] -> [25,362], acting [4,20,303] -> [20,303], acting_primary 4 -> 20, up_primary 4 -> 25, role 0 -> -1 2014-08-13 17:38:03.428132 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.428143 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] clear_primary_state 2014-08-13 17:38:03.428147 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.428161 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.428176 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.428161 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] agent_stop 2014-08-13 17:38:03.428193 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] on_change 2014-08-13 17:38:03.428191 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.428209 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.428216 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.428227 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:03.428230 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.428245 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.428245 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] cancel_copy_ops 2014-08-13 17:38:03.428260 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.428262 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] cancel_flush_ops 2014-08-13 17:38:03.428273 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.428280 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] on_change_cleanup 2014-08-13 17:38:03.428288 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.428296 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] on_change 2014-08-13 17:38:03.428303 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.428316 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] exit NotTrimming 2014-08-13 17:38:03.428319 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428333 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped m=57] enter NotTrimming 2014-08-13 17:38:03.428348 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_stats 2014-08-13 17:38:03.428353 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [313,318]/[313,318,303] -- 313/313 2014-08-13 17:38:03.428372 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.428366 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.428379 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.428381 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_role_change 2014-08-13 17:38:03.428394 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428397 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.428409 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [313,318] newacting [313,318,303] 2014-08-13 17:38:03.428413 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 17:38:03.428424 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.428428 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 17:38:03.428439 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.428447 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428469 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [313,318](313) acting [313,318](313)): not rw, up_thru 194476 up_from 192981 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [313,318](313) acting [313,318](313)) : primary up 192981-194476 does not include interval 2014-08-13 17:38:03.428485 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194479/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [313,318](313) acting [313,318](313)) 2014-08-13 17:38:03.428485 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [25,362]/[20,303,373] -- 25/20 2014-08-13 17:38:03.428513 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.428504 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] up [313,318] -> [313,318], acting [313,318] -> [313,318,303], acting_primary 313 -> 313, up_primary 313 -> 313, role -1 -> -1 2014-08-13 17:38:03.428518 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.428523 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.428538 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.428535 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428554 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.428556 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [25,362] newacting [20,303,373] 2014-08-13 17:38:03.428570 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.428574 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.428585 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.428591 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303] r=-1 lpr=194479 pi=193635-194478/28 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194480 2014-08-13 17:38:03.428601 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.428617 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.428623 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [25,362](25) acting [20,303](20)): not rw, up_thru 194473 up_from 192971 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [25,362](25) acting [20,303](20)) : primary up 192971-194473 does not include interval 2014-08-13 17:38:03.428633 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.428642 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194479/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] noting past interval(194479-194479 up [25,362](25) acting [20,303](20)) 2014-08-13 17:38:03.428650 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.428666 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.428662 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] up [25,362] -> [25,362], acting [20,303] -> [20,303,373], acting_primary 20 -> 20, up_primary 25 -> 25, role -1 -> -1 2014-08-13 17:38:03.428681 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.428683 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 17:38:03.428698 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428700 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.428717 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.428733 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_copy_ops 2014-08-13 17:38:03.428731 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [313,318]/[313,318,303] -- 313/313 2014-08-13 17:38:03.428751 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.428749 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_flush_ops 2014-08-13 17:38:03.428755 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.428765 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change_cleanup 2014-08-13 17:38:03.428776 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428782 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 17:38:03.428794 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428800 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] exit NotTrimming 2014-08-13 17:38:03.428818 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] enter NotTrimming 2014-08-13 17:38:03.428833 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [303,342,45]/[313,318,303] -- 303/313 2014-08-13 17:38:03.428838 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 17:38:03.428853 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.428854 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 17:38:03.428857 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.428872 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428872 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428889 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,342,45] newacting [313,318,303] 2014-08-13 17:38:03.428905 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.428907 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [25,362]/[20,303,373] -- 25/20 2014-08-13 17:38:03.428920 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [313,318]/[313,318,303] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'442 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.428934 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.428939 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.428948 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [313,318](313) acting [313,318,303](313)): not rw, up_thru 194480 up_from 192981 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [313,318](313) acting [313,318,303](313) maybe_went_rw) : primary up 192981-194480 includes interval 2014-08-13 17:38:03.428956 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.428967 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194479/194480/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [313,318](313) acting [313,318,303](313) maybe_went_rw) 2014-08-13 17:38:03.428975 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.428986 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] up [313,318] -> [303,342,45], acting [313,318,303] -> [313,318,303], acting_primary 313 -> 313, up_primary 313 -> 303, role -1 -> -1 2014-08-13 17:38:03.429006 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.429016 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [20,373,303]/[20,303,373] -- 20/20 2014-08-13 17:38:03.429022 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.429040 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429039 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.429046 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.429056 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.429064 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429072 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.429081 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [20,373,303] newacting [20,303,373] 2014-08-13 17:38:03.429088 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.429099 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.429105 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.429116 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [25,362]/[20,303,373] r=-1 lpr=194480 pi=193635-194479/29 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194482 2014-08-13 17:38:03.429122 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.429138 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.429151 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [25,362](25) acting [20,303,373](20)): not rw, up_thru 194480 up_from 192971 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [25,362](25) acting [20,303,373](20) maybe_went_rw) : primary up 192971-194480 includes interval 2014-08-13 17:38:03.429155 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.429181 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194479/194480/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] noting past interval(194480-194481 up [25,362](25) acting [20,303,373](20) maybe_went_rw) 2014-08-13 17:38:03.429187 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.429202 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] up [25,362] -> [20,373,303], acting [20,303,373] -> [20,303,373], acting_primary 20 -> 20, up_primary 25 -> 20, role -1 -> -1 2014-08-13 17:38:03.429215 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429223 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 17:38:03.429240 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.429254 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.429257 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [303,342,45]/[303,342,45] -- 303/303 2014-08-13 17:38:03.429278 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429272 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_copy_ops 2014-08-13 17:38:03.429282 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.429289 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_flush_ops 2014-08-13 17:38:03.429299 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429306 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change_cleanup 2014-08-13 17:38:03.429316 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,342,45] newacting [303,342,45] 2014-08-13 17:38:03.429324 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 17:38:03.429334 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.429342 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] exit NotTrimming 2014-08-13 17:38:03.429350 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45]/[313,318,303] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'442 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.429360 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] enter NotTrimming 2014-08-13 17:38:03.429377 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 17:38:03.429382 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)): not rw, up_thru 194480 up_from 192981 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) : primary up 192981-194480 does not include interval 2014-08-13 17:38:03.429394 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 17:38:03.429398 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194482/194479) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) 2014-08-13 17:38:03.429412 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429415 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] up [303,342,45] -> [303,342,45], acting [313,318,303] -> [303,342,45], acting_primary 313 -> 303, up_primary 303 -> 303, role -1 -> -1 2014-08-13 17:38:03.429434 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.429448 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.429462 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.429460 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [20,373,303]/[20,373,303] -- 20/20 2014-08-13 17:38:03.429478 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.429488 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429492 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.429494 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.429506 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.429512 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429522 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.429530 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [20,373,303] newacting [20,373,303] 2014-08-13 17:38:03.429538 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.429548 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.429554 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.429565 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303]/[20,303,373] r=-1 lpr=194482 pi=193635-194481/30 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194483 2014-08-13 17:38:03.429569 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] [313,318,303] -> [303,342,45], acting primary 313 -> 303 2014-08-13 17:38:03.429594 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.429597 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)): not rw, up_thru 194480 up_from 192971 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) : primary up 192971-194480 does not include interval 2014-08-13 17:38:03.429609 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.429615 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194482/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] noting past interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 17:38:03.429625 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429634 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] up [20,373,303] -> [20,373,303], acting [20,303,373] -> [20,373,303], acting_primary 20 -> 20, up_primary 20 -> 20, role -1 -> -1 2014-08-13 17:38:03.429654 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 17:38:03.429668 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.429664 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [303,342,45]/[303,342,45] -- 303/303 2014-08-13 17:38:03.429687 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429683 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.429692 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.429700 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_copy_ops 2014-08-13 17:38:03.429708 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429715 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_flush_ops 2014-08-13 17:38:03.429724 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429732 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change_cleanup 2014-08-13 17:38:03.429747 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 17:38:03.429754 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 17:38:03.429773 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429763 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] exit NotTrimming 2014-08-13 17:38:03.429776 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.429779 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] enter NotTrimming 2014-08-13 17:38:03.429791 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429795 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_recovery 2014-08-13 17:38:03.429806 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,313,318] newacting [4,313,318] 2014-08-13 17:38:03.429811 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] clear_recovery_state 2014-08-13 17:38:03.429823 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.429828 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429838 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [303,342,45] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.429866 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [303,342,45](303) acting [303,342,45](303)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.429869 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] handle_advance_map [20,373,303]/[20,373,303] -- 20/20 2014-08-13 17:38:03.429893 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429882 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194482/194483/194483) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 17:38:03.429898 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.429899 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,342,45] -> [4,313,318], acting [303,342,45] -> [4,313,318], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 17:38:03.429915 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.429920 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.429936 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.429932 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.429951 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.429967 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2702 2014-08-13 17:38:03.429967 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] handle_advance_map [4,25,362]/[4,25,362] -- 4/4 2014-08-13 17:38:03.429991 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.429986 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.429996 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: Reset advmap 2014-08-13 17:38:03.430002 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.430013 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.430018 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.430030 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] acting_up_affected newup [4,25,362] newacting [4,25,362] 2014-08-13 17:38:03.430036 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.430048 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.430050 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.430063 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [20,373,303] r=-1 lpr=194483 pi=193635-194482/31 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] set_last_peering_reset 194485 2014-08-13 17:38:03.430065 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.430080 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.430094 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.430095 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [20,373,303](20) acting [20,373,303](20)): not rw, up_thru 194483 up_from 192971 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) : primary up 192971-194483 includes interval 2014-08-13 17:38:03.430111 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.430113 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194482/194483/194479) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] noting past interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 17:38:03.430127 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.430132 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] up [20,373,303] -> [4,25,362], acting [20,373,303] -> [4,25,362], acting_primary 20 -> 4, up_primary 20 -> 4, role -1 -> 0 2014-08-13 17:38:03.430143 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.430158 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.430154 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] clear_primary_state 2014-08-13 17:38:03.430173 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.430183 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] agent_stop 2014-08-13 17:38:03.430187 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.430200 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 17:38:03.430209 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.430219 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] publish_stats_to_osd 194485:29701 2014-08-13 17:38:03.430225 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.430237 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.430242 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.430254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] cancel_copy_ops 2014-08-13 17:38:03.430260 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.430271 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] cancel_flush_ops 2014-08-13 17:38:03.430279 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,313,318 2014-08-13 17:38:03.430296 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.430287 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.430299 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.430303 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.430316 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.736055 9 0.002123 2014-08-13 17:38:03.430319 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 17:38:03.430335 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change_cleanup 2014-08-13 17:38:03.430336 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.430350 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 17:38:03.430358 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.430368 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] exit NotTrimming 2014-08-13 17:38:03.430373 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.430386 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] enter NotTrimming 2014-08-13 17:38:03.430391 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000033 0 0.000000 2014-08-13 17:38:03.430402 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] clear_stats 2014-08-13 17:38:03.430411 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.430420 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] requeue_ops 2014-08-13 17:38:03.430428 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.430436 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] on_role_change 2014-08-13 17:38:03.430444 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.430451 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] requeue_ops 2014-08-13 17:38:03.430461 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.430467 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] cancel_recovery 2014-08-13 17:38:03.430483 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] clear_recovery_state 2014-08-13 17:38:03.430480 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 17:38:03.430500 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.430501 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) 2014-08-13 17:38:03.430520 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_activate_map 2014-08-13 17:38:03.430520 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [313,318](313) acting [313,318,303](313) maybe_went_rw) 2014-08-13 17:38:03.430541 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [313,318](313) acting [313,318](313)) 2014-08-13 17:38:03.430540 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] update_heartbeat_peers 4,20,25,303,362 -> 4,25,362 2014-08-13 17:38:03.430561 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.430559 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) 2014-08-13 17:38:03.430564 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] take_waiters 2014-08-13 17:38:03.430578 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,313,318](4) acting [313,318,45](313)) 2014-08-13 17:38:03.430586 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Reset 0.002644 7 0.002425 2014-08-13 17:38:03.430596 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [313,318](313) acting [313,318,45](313) maybe_went_rw) 2014-08-13 17:38:03.430607 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started 2014-08-13 17:38:03.430615 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [313,318](313) acting [313,318](313)) 2014-08-13 17:38:03.430625 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Start 2014-08-13 17:38:03.430634 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,303,313,318,342 down blocked_by {} 2014-08-13 17:38:03.430641 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: transitioning to Primary 2014-08-13 17:38:03.430652 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.430661 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Start 0.000037 0 0.000000 2014-08-13 17:38:03.430673 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2703 2014-08-13 17:38:03.430682 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary 2014-08-13 17:38:03.430692 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 17:38:03.430701 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering 2014-08-13 17:38:03.430711 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 17:38:03.430720 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.430729 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.313 2014-08-13 17:38:03.430737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.430746 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.318 2014-08-13 17:38:03.430762 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 17:38:03.430757 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 17:38:03.430779 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.430779 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 17:38:03.430797 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.430800 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194480-194481 up [25,362](25) acting [20,303,373](20) maybe_went_rw) 2014-08-13 17:38:03.430820 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194479-194479 up [25,362](25) acting [20,303](20)) 2014-08-13 17:38:03.430837 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194476-194478 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 17:38:03.430856 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194473-194475 up [25,362](25) acting [20,303](20) maybe_went_rw) 2014-08-13 17:38:03.430873 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194471-194472 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 17:38:03.430893 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.430890 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 17:38:03.430908 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.430927 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] publish_stats_to_osd 194485:29702 2014-08-13 17:38:03.430951 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.20 2014-08-13 17:38:03.430971 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cfb2e0 don't know epoch, doing nothing 2014-08-13 17:38:03.430981 7fe99f5e9700 7 osd.4 194485 do_notify osd 294 on 1 PGs 2014-08-13 17:38:03.430969 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.25 2014-08-13 17:38:03.430987 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.303 2014-08-13 17:38:03.431012 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e49ce0 don't know epoch, doing nothing 2014-08-13 17:38:03.431016 7fe99f5e9700 7 osd.4 194485 do_notify osd 334 on 1 PGs 2014-08-13 17:38:03.431003 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.362 2014-08-13 17:38:03.431020 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.373 2014-08-13 17:38:03.431037 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4a100 don't know epoch, doing nothing 2014-08-13 17:38:03.431041 7fe99f5e9700 7 osd.4 194485 do_notify osd 346 on 1 PGs 2014-08-13 17:38:03.431038 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.431059 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:03.431062 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.45 on 1 PGs 2014-08-13 17:38:03.431055 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.431084 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.431087 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.303 on 1 PGs 2014-08-13 17:38:03.431108 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cfd6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.431111 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.313 on 1 PGs 2014-08-13 17:38:03.431130 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x57cfe80 don't know epoch, doing nothing 2014-08-13 17:38:03.431132 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.318 sending saperate messages on 1 PGs 2014-08-13 17:38:03.431147 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8580 don't know epoch, doing nothing 2014-08-13 17:38:03.431150 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.342 on 1 PGs 2014-08-13 17:38:03.431226 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 17:38:03.431290 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.431299 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.431308 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.431326 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395] newacting [17,395] 2014-08-13 17:38:03.431341 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.431341 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 17:38:03.431369 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.431360 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 0.243875 3 0.000138 2014-08-13 17:38:03.431375 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.431380 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started 0.243940 0 0.000000 2014-08-13 17:38:03.431392 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379] newacting [321,379] 2014-08-13 17:38:03.431400 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset 2014-08-13 17:38:03.431411 7fe97ae08700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.431408 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.431416 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194477 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194479 2014-08-13 17:38:03.431428 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.246835 3 0.000153 2014-08-13 17:38:03.431434 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194479 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.431449 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194479 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.431448 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started 0.246896 0 0.000000 2014-08-13 17:38:03.431466 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194479 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395] newacting [17,395] 2014-08-13 17:38:03.431470 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.431482 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194479 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.431487 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194477 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.431498 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,4,395] r=1 lpr=194479 pi=193188-194475/32 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194479 2014-08-13 17:38:03.431504 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194479 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.431520 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194479 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.431530 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [17,4,395](17) acting [17,4,395](17)): not rw, up_thru 194476 up_from 192970 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [17,4,395](17) acting [17,4,395](17) maybe_went_rw) : primary up 192970-194476 includes interval 2014-08-13 17:38:03.431535 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194479 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379] newacting [321,379] 2014-08-13 17:38:03.431547 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194476/194476/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194476-194478 up [17,4,395](17) acting [17,4,395](17) maybe_went_rw) 2014-08-13 17:38:03.431551 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194479 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.431566 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379,4] r=2 lpr=194479 pi=192971-194475/21 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.431566 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,4,395] -> [17,395], acting [17,4,395] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role 1 -> -1 2014-08-13 17:38:03.431588 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:03.431598 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [321,379,4](321) acting [321,379,4](321)): not rw, up_thru 194476 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [321,379,4](321) acting [321,379,4](321) maybe_went_rw) : primary up 192971-194476 includes interval 2014-08-13 17:38:03.431604 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.431615 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [321,379,4](321) acting [321,379,4](321) maybe_went_rw) 2014-08-13 17:38:03.431620 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.431634 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379,4] -> [321,379], acting [321,379,4] -> [321,379], acting_primary 321 -> 321, up_primary 321 -> 321, role 2 -> -1 2014-08-13 17:38:03.431637 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 17:38:03.431654 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 17:38:03.431652 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.431669 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 17:38:03.431675 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.431686 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:03.431691 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.431704 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 17:38:03.431707 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.431720 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 17:38:03.431721 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.431736 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_role_change 2014-08-13 17:38:03.431737 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.431754 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.431752 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.431770 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.431771 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 17:38:03.431787 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.431787 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 17:38:03.431806 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.431811 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.431821 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.431835 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.431847 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.431848 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 17:38:03.431873 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.431865 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.431879 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.431894 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.431900 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 17:38:03.431920 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.431910 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.431925 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.431939 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.431941 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 17:38:03.431955 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.431976 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.431981 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.431992 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 17:38:03.431996 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.432011 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432013 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432015 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.432031 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.432045 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432050 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395,382]/[17,395,382] -- 17/17 2014-08-13 17:38:03.432072 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432077 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.432078 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 17:38:03.432097 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432093 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.432101 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.432108 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395,382] newacting [17,395,382] 2014-08-13 17:38:03.432117 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.432125 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.432132 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379,360] newacting [321,379,360] 2014-08-13 17:38:03.432141 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395] r=-1 lpr=194479 pi=193188-194478/33 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194482 2014-08-13 17:38:03.432150 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.432172 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [17,395](17) acting [17,395](17)): not rw, up_thru 194480 up_from 192970 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [17,395](17) acting [17,395](17) maybe_went_rw) : primary up 192970-194480 includes interval 2014-08-13 17:38:03.432165 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379] r=-1 lpr=194479 pi=192971-194478/22 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.432201 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432206 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432208 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d8f2c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.432188 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194479/194479/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194479-194481 up [17,395](17) acting [17,395](17) maybe_went_rw) 2014-08-13 17:38:03.432203 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [321,379](321) acting [321,379](321)): not rw, up_thru 194479 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [321,379](321) acting [321,379](321) maybe_went_rw) : primary up 192971-194479 includes interval 2014-08-13 17:38:03.432218 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.432223 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.432226 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432228 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432214 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,395] -> [17,395,382], acting [17,395] -> [17,395,382], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 17:38:03.432220 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [321,379](321) acting [321,379](321) maybe_went_rw) 2014-08-13 17:38:03.432235 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:03.432238 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379] -> [321,379,360], acting [321,379] -> [321,379,360], acting_primary 321 -> 321, up_primary 321 -> 321, role -1 -> -1 2014-08-13 17:38:03.432251 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.432258 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.432267 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.432285 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432288 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432275 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.432290 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264980 pg_notify(22.3e4(31) epoch 194485) v5 2014-08-13 17:38:03.432286 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 17:38:03.432292 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.432303 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 17:38:03.432310 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.432334 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.45 2014-08-13 17:38:03.432320 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 17:38:03.432337 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264980 2014-08-13 17:38:03.432329 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.432339 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:03.432346 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.432360 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 17:38:03.432363 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.432385 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432388 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432377 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 17:38:03.432378 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.432394 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 17:38:03.432395 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.432420 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432409 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 17:38:03.432423 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432412 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.432426 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42655c0 pg_notify(22.3e4 epoch 194485) v5 2014-08-13 17:38:03.432428 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.432427 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432448 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432462 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.313 2014-08-13 17:38:03.432465 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42655c0 2014-08-13 17:38:03.432469 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395,382]/[17,395] -- 17/17 2014-08-13 17:38:03.432494 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432498 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432484 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 17:38:03.432501 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432504 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432500 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.432520 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432523 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432509 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.432525 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9880 pg_notify(22.3e4(1) epoch 194485) v5 2014-08-13 17:38:03.432517 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.432525 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.432534 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395,382] newacting [17,395] 2014-08-13 17:38:03.432544 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432564 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.432553 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.432567 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9880 2014-08-13 17:38:03.432569 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382] r=-1 lpr=194482 pi=193188-194481/34 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194483 2014-08-13 17:38:03.432579 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 17:38:03.432600 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.432605 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432608 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432601 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [17,395,382](17) acting [17,395,382](17)): not rw, up_thru 194480 up_from 192970 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [17,395,382](17) acting [17,395,382](17)) : primary up 192970-194480 does not include interval 2014-08-13 17:38:03.432604 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.432627 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.432629 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.432631 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513d780 pg_notify(22.3e4(31) epoch 194485) v5 2014-08-13 17:38:03.432621 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.432619 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194482/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] noting past interval(194482-194482 up [17,395,382](17) acting [17,395,382](17)) 2014-08-13 17:38:03.432638 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.432641 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] up [17,395,382] -> [17,395,382], acting [17,395,382] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 17:38:03.432670 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.432674 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513d780 2014-08-13 17:38:03.432662 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 17:38:03.432669 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 17:38:03.432690 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.439528 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.432679 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.439566 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.439579 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.439584 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.439584 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379,4] newacting [321,379,4] 2014-08-13 17:38:03.439610 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.439573 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.439612 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.439602 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.439614 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db45c0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.439624 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.439629 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.439632 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.439617 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_copy_ops 2014-08-13 17:38:03.439634 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.439621 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,360] r=-1 lpr=194482 pi=192971-194481/23 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.439635 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_flush_ops 2014-08-13 17:38:03.439653 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.439656 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.439658 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e040 pg_notify(22.3e4 epoch 194485) v5 2014-08-13 17:38:03.439653 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change_cleanup 2014-08-13 17:38:03.439658 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [321,379,360](321) acting [321,379,360](321)): not rw, up_thru 194482 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [321,379,360](321) acting [321,379,360](321) maybe_went_rw) : primary up 192971-194482 includes interval 2014-08-13 17:38:03.439672 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 17:38:03.439676 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [321,379,360](321) acting [321,379,360](321) maybe_went_rw) 2014-08-13 17:38:03.439696 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.318 2014-08-13 17:38:03.439700 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e040 2014-08-13 17:38:03.439692 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit NotTrimming 2014-08-13 17:38:03.439697 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379,360] -> [321,379,4], acting [321,379,360] -> [321,379,4], acting_primary 321 -> 321, up_primary 321 -> 321, role -1 -> 2 2014-08-13 17:38:03.439712 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter NotTrimming 2014-08-13 17:38:03.439718 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.439735 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.439739 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.439732 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_recovery 2014-08-13 17:38:03.439735 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.439748 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] clear_recovery_state 2014-08-13 17:38:03.439751 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.439769 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.439767 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.439784 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.439804 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.439818 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.439817 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,395,382]/[17,395] -- 17/17 2014-08-13 17:38:03.439835 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.439849 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.439851 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.439857 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.439870 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.439875 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.439885 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.439894 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.439901 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.439915 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.439926 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,4,395]/[17,395] -- 17/17 2014-08-13 17:38:03.439931 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.439950 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.439951 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.439956 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:03.439972 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.439988 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,395] 2014-08-13 17:38:03.440004 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.440009 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.440020 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,395,382]/[17,395] r=-1 lpr=194483 pi=193188-194482/35 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194485 2014-08-13 17:38:03.440028 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.440045 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.008574 7 0.001494 2014-08-13 17:38:03.440055 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [17,395,382](17) acting [17,395](17)): not rw, up_thru 194483 up_from 192970 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [17,395,382](17) acting [17,395](17) maybe_went_rw) : primary up 192970-194483 includes interval 2014-08-13 17:38:03.440065 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.440077 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194482/194483/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] noting past interval(194483-194484 up [17,395,382](17) acting [17,395](17) maybe_went_rw) 2014-08-13 17:38:03.440081 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.440097 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.440097 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] up [17,395,382] -> [17,4,395], acting [17,395] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 17:38:03.440119 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 17:38:03.440118 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 2014-08-13 17:38:03.440135 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.440138 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.440151 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.440169 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_copy_ops 2014-08-13 17:38:03.440160 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.440189 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.440184 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_flush_ops 2014-08-13 17:38:03.440214 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change_cleanup 2014-08-13 17:38:03.440231 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 17:38:03.440248 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit NotTrimming 2014-08-13 17:38:03.440264 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter NotTrimming 2014-08-13 17:38:03.440278 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.440280 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_recovery 2014-08-13 17:38:03.440299 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] clear_recovery_state 2014-08-13 17:38:03.440316 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.440333 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_activate_map 2014-08-13 17:38:03.440333 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 17:38:03.440362 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.440369 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.440385 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.440399 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.440406 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 17:38:03.440417 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.255512 2 0.000225 2014-08-13 17:38:03.440426 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] take_waiters 2014-08-13 17:38:03.440439 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.440444 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Reset 0.009044 7 0.008709 2014-08-13 17:38:03.440456 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.255559 0 0.000000 2014-08-13 17:38:03.440468 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started 2014-08-13 17:38:03.440477 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.255589 0 0.000000 2014-08-13 17:38:03.440485 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Start 2014-08-13 17:38:03.440496 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started 0.255685 0 0.000000 2014-08-13 17:38:03.440503 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: transitioning to Stray 2014-08-13 17:38:03.440517 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.440523 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Start 0.000037 0 0.000000 2014-08-13 17:38:03.440532 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194477 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.440544 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started/Stray 2014-08-13 17:38:03.440549 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194479 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.440564 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194479 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.440566 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.440580 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194479 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [349,61] newacting [349,61] 2014-08-13 17:38:03.440584 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.440596 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194479 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.440612 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [4,349,61] r=0 lpr=194479 pi=193306-194475/23 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.440646 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,349,61](4) acting [4,349,61](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.440661 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194476/194476/194476) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] noting past interval(194476-194478 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 17:38:03.440677 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] up [4,349,61] -> [349,61], acting [4,349,61] -> [349,61], acting_primary 4 -> 349, up_primary 4 -> 349, role 0 -> -1 2014-08-13 17:38:03.440693 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.440694 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.440709 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.440723 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.440736 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.440749 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.440750 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 17:38:03.440765 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.440778 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.440780 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.440784 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.440794 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.440804 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44] newacting [367,44] 2014-08-13 17:38:03.440811 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.440822 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.440827 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.440839 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.253074 3 0.000128 2014-08-13 17:38:03.440843 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.440858 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.440860 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.253134 0 0.000000 2014-08-13 17:38:03.440874 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.440879 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.440888 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.440895 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.440902 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.440911 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.440918 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.440928 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.440932 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.440944 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44] newacting [367,44] 2014-08-13 17:38:03.440949 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.440959 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.440975 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.440986 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 17:38:03.441006 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441005 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [367,44,4](367) acting [367,44,4](367)): not rw, up_thru 194476 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [367,44,4](367) acting [367,44,4](367) maybe_went_rw) : primary up 192976-194476 includes interval 2014-08-13 17:38:03.441011 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441022 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [367,44,4](367) acting [367,44,4](367) maybe_went_rw) 2014-08-13 17:38:03.441027 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.441043 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441040 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44,4] -> [367,44], acting [367,44,4] -> [367,44], acting_primary 367 -> 367, up_primary 367 -> 367, role 2 -> -1 2014-08-13 17:38:03.441059 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.441072 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.441073 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 17:38:03.441093 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441088 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.441096 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441105 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.441112 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.441119 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.441128 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441135 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.441149 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.441164 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.441178 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.441167 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 17:38:03.441241 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441233 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.441252 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.441253 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441267 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.441271 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.441280 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.441287 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] acting_up_affected newup [331,58,330] newacting [331,58,330] 2014-08-13 17:38:03.441299 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441303 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.441319 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [349,61] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'372 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.441336 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 17:38:03.441356 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441347 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [349,61](349) acting [349,61](349)): not rw, up_thru 194479 up_from 192942 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) : primary up 192942-194479 includes interval 2014-08-13 17:38:03.441361 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441363 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194479/194479/194479) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 17:38:03.441376 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.441380 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] up [349,61] -> [331,58,330], acting [349,61] -> [331,58,330], acting_primary 349 -> 331, up_primary 349 -> 331, role -1 -> -1 2014-08-13 17:38:03.441392 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441399 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.441414 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.441421 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 17:38:03.441429 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.441440 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441445 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441445 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.441461 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.441461 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.441477 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441477 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.441493 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.441508 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.441512 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 17:38:03.441530 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441523 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.441534 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441550 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.441539 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] [349,61] -> [331,58,330], acting primary 349 -> 331 2014-08-13 17:38:03.441564 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44,342] newacting [367,44,342] 2014-08-13 17:38:03.441567 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.441580 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.441582 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.441596 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.441599 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441625 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [367,44](367) acting [367,44](367)): not rw, up_thru 194479 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [367,44](367) acting [367,44](367) maybe_went_rw) : primary up 192976-194479 includes interval 2014-08-13 17:38:03.441640 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [367,44](367) acting [367,44](367) maybe_went_rw) 2014-08-13 17:38:03.441641 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 17:38:03.441661 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.441656 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44] -> [367,44,342], acting [367,44] -> [367,44,342], acting_primary 367 -> 367, up_primary 367 -> 367, role -1 -> -1 2014-08-13 17:38:03.441665 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.441675 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.441682 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.441691 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.441699 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.441706 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.441993 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.442001 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.442000 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 17:38:03.442013 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442010 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.442015 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442019 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.442025 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442029 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.442034 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442038 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.442050 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.442052 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 17:38:03.442064 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442058 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.442066 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442067 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442074 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442084 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,349,61] newacting [4,349,61] 2014-08-13 17:38:03.442088 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 17:38:03.442099 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442093 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.442101 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [331,58,330] r=-1 lpr=194482 pi=193306-194481/25 crt=191648'372 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.442102 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442111 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.442120 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442120 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [331,58,330](331) acting [331,58,330](331)): not rw, up_thru 194482 up_from 192951 last_epoch_clean 194466 generate_past_intervals interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) : primary up 192951-194482 includes interval 2014-08-13 17:38:03.442130 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194482/194482/194482) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 17:38:03.442139 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 17:38:03.442139 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [331,58,330] -> [4,349,61], acting [331,58,330] -> [4,349,61], acting_primary 331 -> 4, up_primary 331 -> 4, role -1 -> 0 2014-08-13 17:38:03.442151 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442152 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.442153 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442162 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.442161 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.442171 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442176 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.442187 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:3017 2014-08-13 17:38:03.442188 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 17:38:03.442196 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442203 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442206 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442206 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.442215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.442216 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.442225 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442225 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44,4] newacting [367,44,4] 2014-08-13 17:38:03.442234 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442235 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.442243 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442244 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,342] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.442253 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.442261 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.442261 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [367,44,342](367) acting [367,44,342](367)): not rw, up_thru 194482 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [367,44,342](367) acting [367,44,342](367) maybe_went_rw) : primary up 192976-194482 includes interval 2014-08-13 17:38:03.442272 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.442271 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [367,44,342](367) acting [367,44,342](367) maybe_went_rw) 2014-08-13 17:38:03.442282 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.442282 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44,342] -> [367,44,4], acting [367,44,342] -> [367,44,4], acting_primary 367 -> 367, up_primary 367 -> 367, role -1 -> 2 2014-08-13 17:38:03.442291 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.442294 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.442300 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.442303 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.442309 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.442312 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442318 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.442321 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.442327 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.442330 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.442336 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.442338 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.442345 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442347 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.442356 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.442356 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.442366 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.442369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,61,349 unchanged 2014-08-13 17:38:03.442374 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.442379 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.442383 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442391 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.442389 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001873 7 0.001700 2014-08-13 17:38:03.442399 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.442404 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.442410 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442414 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.442420 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.442423 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.442434 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.442444 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.442453 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.442455 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.442466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.442465 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.442475 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442476 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.001596 7 0.001439 2014-08-13 17:38:03.442488 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.442487 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 17:38:03.442497 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.442500 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 17:38:03.442505 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.442511 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 17:38:03.442516 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.442523 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 17:38:03.442526 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.442533 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 17:38:03.442538 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.442544 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,330,331,349 down blocked_by {} 2014-08-13 17:38:03.442548 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.442555 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.442566 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:3018 2014-08-13 17:38:03.442575 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 17:38:03.442586 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 17:38:03.442594 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.442596 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.330 2014-08-13 17:38:03.442605 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.331 2014-08-13 17:38:03.442613 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 17:38:03.442623 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.442623 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [310,21]/[310,21] -- 310/310 2014-08-13 17:38:03.442638 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442632 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.442641 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.442650 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442659 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [310,21] newacting [310,21] 2014-08-13 17:38:03.442668 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.442676 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [4,310,21] r=0 lpr=194478 pi=193306-194476/27 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.442688 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.442693 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,310,21](4) acting [4,310,21](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.442702 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194476/194477/194477) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] noting past interval(194477-194478 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) 2014-08-13 17:38:03.442711 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] up [4,310,21] -> [310,21], acting [4,310,21] -> [310,21], acting_primary 4 -> 310, up_primary 4 -> 310, role 0 -> -1 2014-08-13 17:38:03.442721 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.442721 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 17:38:03.442730 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.442741 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442739 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.442745 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Started advmap 2014-08-13 17:38:03.442748 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.442757 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.442756 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,360] newacting [364,301] 2014-08-13 17:38:03.442766 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.442767 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.442774 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.442778 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started/Stray 0.257478 3 0.000136 2014-08-13 17:38:03.442783 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.442791 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.442790 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started 0.257533 0 0.000000 2014-08-13 17:38:03.442800 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.442802 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset 2014-08-13 17:38:03.442809 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.442813 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194477 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194479 2014-08-13 17:38:03.442818 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.442823 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194479 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.442826 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442834 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.442833 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194479 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442842 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.442843 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194479 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,360] newacting [364,301] 2014-08-13 17:38:03.442850 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.442853 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194479 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.442859 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.442863 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,4,360]/[364,4,301] r=1 lpr=194479 pi=193125-194475/29 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194479 2014-08-13 17:38:03.442869 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.442882 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [364,4,360](364) acting [364,4,301](364)): not rw, up_thru 194476 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [364,4,360](364) acting [364,4,301](364) maybe_went_rw) : primary up 192966-194476 includes interval 2014-08-13 17:38:03.442888 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [310,21]/[310,21,15] -- 310/310 2014-08-13 17:38:03.442899 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.442893 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194476/194476/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194476-194478 up [364,4,360](364) acting [364,4,301](364) maybe_went_rw) 2014-08-13 17:38:03.442902 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.442904 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,4,360] -> [364,360], acting [364,4,301] -> [364,301], acting_primary 364 -> 364, up_primary 364 -> 364, role 1 -> -1 2014-08-13 17:38:03.442910 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.442916 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.442923 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] acting_up_affected newup [310,21] newacting [310,21,15] 2014-08-13 17:38:03.442927 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.442932 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.442936 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.442941 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'417 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.442947 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.442956 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.442957 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [310,21](310) acting [310,21](310)): not rw, up_thru 194476 up_from 192972 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [310,21](310) acting [310,21](310)) : primary up 192972-194476 does not include interval 2014-08-13 17:38:03.442966 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.442968 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194479/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [310,21](310) acting [310,21](310)) 2014-08-13 17:38:03.442975 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.442978 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] up [310,21] -> [310,21], acting [310,21] -> [310,21,15], acting_primary 310 -> 310, up_primary 310 -> 310, role -1 -> -1 2014-08-13 17:38:03.442985 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.442989 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.442995 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.442999 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443005 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_role_change 2014-08-13 17:38:03.443008 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.443015 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.443018 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.443024 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.443027 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.443033 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.443036 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.443045 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.443044 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443055 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.443067 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.443066 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 17:38:03.443082 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443077 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.443085 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.443085 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443095 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443095 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443106 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443115 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [310,21]/[310,21,15] -- 310/310 2014-08-13 17:38:03.443126 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443124 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 17:38:03.443128 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.443137 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443137 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443140 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443147 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443151 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443161 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443170 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [303,15,342]/[310,21,15] -- 303/310 2014-08-13 17:38:03.443180 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443182 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.443187 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,368,301]/[364,301] -- 364/364 2014-08-13 17:38:03.443201 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443196 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443204 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443206 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,15,342] newacting [310,21,15] 2014-08-13 17:38:03.443214 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443215 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443224 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [310,21]/[310,21,15] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'417 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.443223 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [364,368,301] newacting [364,301] 2014-08-13 17:38:03.443234 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443241 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [310,21](310) acting [310,21,15](310)): not rw, up_thru 194480 up_from 192972 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [310,21](310) acting [310,21,15](310) maybe_went_rw) : primary up 192972-194480 includes interval 2014-08-13 17:38:03.443243 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,360]/[364,301] r=-1 lpr=194479 pi=193125-194478/30 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194482 2014-08-13 17:38:03.443251 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194479/194480/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [310,21](310) acting [310,21,15](310) maybe_went_rw) 2014-08-13 17:38:03.443260 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [364,360](364) acting [364,301](364)): not rw, up_thru 194479 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [364,360](364) acting [364,301](364) maybe_went_rw) : primary up 192966-194479 includes interval 2014-08-13 17:38:03.443261 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] up [310,21] -> [303,15,342], acting [310,21,15] -> [310,21,15], acting_primary 310 -> 310, up_primary 310 -> 303, role -1 -> -1 2014-08-13 17:38:03.443271 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194479/194479/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194479-194481 up [364,360](364) acting [364,301](364) maybe_went_rw) 2014-08-13 17:38:03.443273 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.443282 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443282 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,360] -> [364,368,301], acting [364,301] -> [364,301], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> -1 2014-08-13 17:38:03.443292 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.443294 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.443301 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.443305 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443310 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.443315 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.443320 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.443329 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.443325 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.443338 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.443339 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.443348 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.443349 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.443358 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.443358 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.443368 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.443369 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.443377 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443379 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.443388 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.443397 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.443400 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [303,15,342]/[303,15,342] -- 303/303 2014-08-13 17:38:03.443412 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443407 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443415 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.443424 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443430 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,368,301]/[364,368,301] -- 364/364 2014-08-13 17:38:03.443432 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,15,342] newacting [303,15,342] 2014-08-13 17:38:03.443444 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443442 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443447 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443451 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342]/[310,21,15] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'417 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.443458 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443468 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)): not rw, up_thru 194480 up_from 192972 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) : primary up 192972-194480 does not include interval 2014-08-13 17:38:03.443467 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [364,368,301] newacting [364,368,301] 2014-08-13 17:38:03.443478 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443478 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194482/194479) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 17:38:03.443490 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301]/[364,301] r=-1 lpr=194482 pi=193125-194481/31 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194483 2014-08-13 17:38:03.443491 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] up [303,15,342] -> [303,15,342], acting [310,21,15] -> [303,15,342], acting_primary 310 -> 303, up_primary 303 -> 303, role -1 -> -1 2014-08-13 17:38:03.443502 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.443508 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [364,368,301](364) acting [364,301](364)): not rw, up_thru 194479 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [364,368,301](364) acting [364,301](364)) : primary up 192966-194479 does not include interval 2014-08-13 17:38:03.443510 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443518 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.443518 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194482/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] noting past interval(194482-194482 up [364,368,301](364) acting [364,301](364)) 2014-08-13 17:38:03.443528 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.443530 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] up [364,368,301] -> [364,368,301], acting [364,301] -> [364,368,301], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> -1 2014-08-13 17:38:03.443536 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.443541 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.443545 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.443551 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443554 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.443560 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.443562 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.443570 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.443572 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.443578 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.443580 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] [310,21,15] -> [303,15,342], acting primary 310 -> 303 2014-08-13 17:38:03.443587 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.443594 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.443596 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.443603 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.443606 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.443612 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443616 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.443624 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.443635 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.443634 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [303,15,342]/[303,15,342] -- 303/303 2014-08-13 17:38:03.443646 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443645 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443648 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.443657 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443665 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443668 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,368,301]/[364,368,301] -- 364/364 2014-08-13 17:38:03.443681 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443682 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 17:38:03.443684 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443693 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443694 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443695 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.443703 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443704 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443715 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,310,21] newacting [4,310,21] 2014-08-13 17:38:03.443725 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443723 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,360] -- 364/364 2014-08-13 17:38:03.443737 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.443734 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [303,15,342] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'417 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.443739 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.443749 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.443751 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [303,15,342](303) acting [303,15,342](303)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.443758 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,360] 2014-08-13 17:38:03.443761 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194482/194483/194483) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 17:38:03.443768 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.443771 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,15,342] -> [4,310,21], acting [303,15,342] -> [4,310,21], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 17:38:03.443777 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,368,301] r=-1 lpr=194483 pi=193125-194482/32 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194485 2014-08-13 17:38:03.443783 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.443793 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.443795 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [364,368,301](364) acting [364,368,301](364)): not rw, up_thru 194483 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [364,368,301](364) acting [364,368,301](364) maybe_went_rw) : primary up 192966-194483 includes interval 2014-08-13 17:38:03.443802 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.443806 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194482/194483/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] noting past interval(194483-194484 up [364,368,301](364) acting [364,368,301](364) maybe_went_rw) 2014-08-13 17:38:03.443812 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2566 2014-08-13 17:38:03.443816 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] up [364,368,301] -> [364,4,360], acting [364,368,301] -> [364,4,360], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> 1 2014-08-13 17:38:03.443822 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.443828 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.443832 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.443838 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.443841 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.443847 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.443850 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.443857 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.443860 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.443866 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.443868 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.443875 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.443876 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.443884 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.443885 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.443894 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.443894 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.443903 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.443904 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.443912 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_role_change 2014-08-13 17:38:03.443916 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.443921 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.443926 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.443930 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.443934 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.443939 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.443942 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.443949 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.443958 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map 2014-08-13 17:38:03.443961 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.443970 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.443979 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.443989 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,21,310 2014-08-13 17:38:03.443998 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.444000 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.444001 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] update_heartbeat_peers unchanged 2014-08-13 17:38:03.444008 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.723390 9 0.001220 2014-08-13 17:38:03.444012 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] take_waiters 2014-08-13 17:38:03.444020 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.444022 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Reset 0.001220 7 0.001047 2014-08-13 17:38:03.444029 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.444034 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started 2014-08-13 17:38:03.444038 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.444044 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Start 2014-08-13 17:38:03.444048 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.444053 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: transitioning to Stray 2014-08-13 17:38:03.444059 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.444064 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.444069 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.444076 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started/Stray 2014-08-13 17:38:03.444079 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.444088 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.444088 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.444099 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.444100 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 17:38:03.444112 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 17:38:03.444121 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [310,21](310) acting [310,21,15](310) maybe_went_rw) 2014-08-13 17:38:03.444131 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [310,21](310) acting [310,21](310)) 2014-08-13 17:38:03.444140 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) 2014-08-13 17:38:03.444153 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.444150 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,310,21](4) acting [310,21,15](310)) 2014-08-13 17:38:03.444160 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [310,21](310) acting [310,21,15](310) maybe_went_rw) 2014-08-13 17:38:03.444170 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [310,21](310) acting [310,21](310)) 2014-08-13 17:38:03.444179 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,15,21,303,310,342 down blocked_by {} 2014-08-13 17:38:03.444183 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.444199 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444189 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.444205 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2567 2014-08-13 17:38:03.444203 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.444215 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.444215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.15 2014-08-13 17:38:03.444224 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.444227 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.21 2014-08-13 17:38:03.444235 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.258686 2 0.000140 2014-08-13 17:38:03.444236 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 17:38:03.444245 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.310 2014-08-13 17:38:03.444246 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.444254 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 17:38:03.444255 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.258714 0 0.000000 2014-08-13 17:38:03.444267 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.444268 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.258735 0 0.000000 2014-08-13 17:38:03.444276 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.444278 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.258780 0 0.000000 2014-08-13 17:38:03.444288 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.444296 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.444304 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.444315 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.444323 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340] newacting [340] 2014-08-13 17:38:03.444334 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.444330 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.444338 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.444354 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,340](4) acting [4,340](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.444362 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] noting past interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:03.444361 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 17:38:03.444372 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] up [4,340] -> [340], acting [4,340] -> [340], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 17:38:03.444380 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444382 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.444384 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Started advmap 2014-08-13 17:38:03.444391 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.444394 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,285] newacting [289,285] 2014-08-13 17:38:03.444400 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.444404 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.444408 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.444416 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.444415 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 0.256344 3 0.000146 2014-08-13 17:38:03.444425 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.444427 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started 0.256397 0 0.000000 2014-08-13 17:38:03.444433 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.444438 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset 2014-08-13 17:38:03.444442 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.444448 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194477 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194479 2014-08-13 17:38:03.444451 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.444458 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194479 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.444460 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.444467 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194479 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.444469 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.444477 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.444476 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194479 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,285] newacting [289,285] 2014-08-13 17:38:03.444487 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.444488 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194479 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.444496 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.444497 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,4,285] r=1 lpr=194479 pi=193125-194475/39 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194479 2014-08-13 17:38:03.444503 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.444511 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.444515 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [289,4,285](289) acting [289,4,285](289)): not rw, up_thru 194476 up_from 192968 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [289,4,285](289) acting [289,4,285](289) maybe_went_rw) : primary up 192968-194476 includes interval 2014-08-13 17:38:03.444519 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.444526 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194476/194476/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194476-194478 up [289,4,285](289) acting [289,4,285](289) maybe_went_rw) 2014-08-13 17:38:03.444528 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444536 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,4,285] -> [289,285], acting [289,4,285] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role 1 -> -1 2014-08-13 17:38:03.444547 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:03.444545 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.444557 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444556 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.444559 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444565 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.444568 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444575 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 17:38:03.444577 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444583 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 17:38:03.444592 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 17:38:03.444593 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.444603 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444602 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:03.444606 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444612 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 17:38:03.444614 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444622 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 17:38:03.444622 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444631 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_role_change 2014-08-13 17:38:03.444638 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.444643 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.444647 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 17:38:03.444654 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444655 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 17:38:03.444656 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444665 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444666 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444673 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [324,370] newacting [324,370] 2014-08-13 17:38:03.444682 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.444685 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 17:38:03.444690 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.444699 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444702 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.444704 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [340](340) acting [340](340)): not rw, up_thru 194479 up_from 192944 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) : primary up 192944-194479 includes interval 2014-08-13 17:38:03.444711 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.444713 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.444721 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444722 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] up [340] -> [324,370], acting [340] -> [324,370], acting_primary 340 -> 324, up_primary 340 -> 324, role -1 -> -1 2014-08-13 17:38:03.444732 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.444740 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.444738 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 17:38:03.444751 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444748 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.444753 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.444756 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.444764 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.444762 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.444772 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.444774 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444780 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.444788 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.444796 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.444796 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,306,364]/[289,306,364] -- 289/289 2014-08-13 17:38:03.444810 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444805 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] [340] -> [324,370], acting primary 340 -> 324 2014-08-13 17:38:03.444816 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.444819 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.444825 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.444827 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.444834 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,306,364] newacting [289,306,364] 2014-08-13 17:38:03.444836 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444844 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.444852 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,285] r=-1 lpr=194479 pi=193125-194478/40 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194482 2014-08-13 17:38:03.444857 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.444868 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444868 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [289,285](289) acting [289,285](289)): not rw, up_thru 194480 up_from 192968 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [289,285](289) acting [289,285](289) maybe_went_rw) : primary up 192968-194480 includes interval 2014-08-13 17:38:03.444871 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444879 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444877 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194479/194479/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194479-194481 up [289,285](289) acting [289,285](289) maybe_went_rw) 2014-08-13 17:38:03.444888 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444888 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,285] -> [289,306,364], acting [289,285] -> [289,306,364], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.444899 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:03.444908 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.444908 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.444919 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444917 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.444925 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444926 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 17:38:03.444933 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444934 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 17:38:03.444941 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.444943 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 17:38:03.444952 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:03.444956 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:03.444967 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.444961 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 17:38:03.444969 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.444970 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 17:38:03.444978 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.444979 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 17:38:03.444987 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340] newacting [4,340] 2014-08-13 17:38:03.444988 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 17:38:03.444996 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.444997 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445004 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.445020 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,306,364]/[289,285] -- 289/289 2014-08-13 17:38:03.445019 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [324,370](324) acting [324,370](324)): not rw, up_thru 194482 up_from 192951 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) : primary up 192951-194482 includes interval 2014-08-13 17:38:03.445034 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445031 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.445037 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.445042 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [324,370] -> [4,340], acting [324,370] -> [4,340], acting_primary 324 -> 4, up_primary 324 -> 4, role -1 -> 0 2014-08-13 17:38:03.445047 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.445053 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.445056 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,306,364] newacting [289,285] 2014-08-13 17:38:03.445062 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.445066 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.445071 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.445077 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364] r=-1 lpr=194482 pi=193125-194481/41 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194483 2014-08-13 17:38:03.445082 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:176847 2014-08-13 17:38:03.445091 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445094 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [289,306,364](289) acting [289,306,364](289)): not rw, up_thru 194480 up_from 192968 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [289,306,364](289) acting [289,306,364](289)) : primary up 192968-194480 does not include interval 2014-08-13 17:38:03.445100 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.445105 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194482/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] noting past interval(194482-194482 up [289,306,364](289) acting [289,306,364](289)) 2014-08-13 17:38:03.445109 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.445117 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445116 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] up [289,306,364] -> [289,306,364], acting [289,306,364] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.445125 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445128 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 17:38:03.445133 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445137 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.445141 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.445147 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.445150 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.445157 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_copy_ops 2014-08-13 17:38:03.445159 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.445166 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_flush_ops 2014-08-13 17:38:03.445168 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.445175 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change_cleanup 2014-08-13 17:38:03.445182 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.445191 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.445185 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 17:38:03.445199 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.445201 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit NotTrimming 2014-08-13 17:38:03.445208 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.445212 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter NotTrimming 2014-08-13 17:38:03.445216 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.445221 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_recovery 2014-08-13 17:38:03.445225 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.445231 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] clear_recovery_state 2014-08-13 17:38:03.445235 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445242 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445245 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.445255 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340 unchanged 2014-08-13 17:38:03.445264 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.445265 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,306,364]/[289,285] -- 289/289 2014-08-13 17:38:03.445272 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000984 7 0.000904 2014-08-13 17:38:03.445280 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445283 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.445283 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.445292 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.445293 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.445300 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.445303 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445311 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.445321 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.445321 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,4,285]/[289,285] -- 289/289 2014-08-13 17:38:03.445335 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445330 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.445339 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.445337 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:03.445348 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.445350 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.445358 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.445359 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,285] 2014-08-13 17:38:03.445369 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.445369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.445379 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,306,364]/[289,285] r=-1 lpr=194483 pi=193125-194482/42 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194485 2014-08-13 17:38:03.445380 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:03.445391 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 17:38:03.445396 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [289,306,364](289) acting [289,285](289)): not rw, up_thru 194483 up_from 192968 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [289,306,364](289) acting [289,285](289) maybe_went_rw) : primary up 192968-194483 includes interval 2014-08-13 17:38:03.445401 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.445407 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194482/194483/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] noting past interval(194483-194484 up [289,306,364](289) acting [289,285](289) maybe_went_rw) 2014-08-13 17:38:03.445413 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:176848 2014-08-13 17:38:03.445418 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] up [289,306,364] -> [289,4,285], acting [289,285] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.445424 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 17:38:03.445430 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 17:38:03.445434 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.445440 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.445445 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.370 2014-08-13 17:38:03.445449 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.445455 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.445459 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_copy_ops 2014-08-13 17:38:03.445465 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.445469 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_flush_ops 2014-08-13 17:38:03.445478 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change_cleanup 2014-08-13 17:38:03.445487 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 17:38:03.445499 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.445496 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit NotTrimming 2014-08-13 17:38:03.445505 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter NotTrimming 2014-08-13 17:38:03.445514 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_recovery 2014-08-13 17:38:03.445522 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] clear_recovery_state 2014-08-13 17:38:03.445525 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_advance_map [339,45]/[339,45] -- 339/339 2014-08-13 17:38:03.445532 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445540 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445542 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_activate_map 2014-08-13 17:38:03.445544 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.445554 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.445562 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.445572 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.259539 2 0.000262 2014-08-13 17:38:03.445582 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.445591 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.259627 0 0.000000 2014-08-13 17:38:03.445593 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] update_heartbeat_peers unchanged 2014-08-13 17:38:03.445602 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.259647 0 0.000000 2014-08-13 17:38:03.445604 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] take_waiters 2014-08-13 17:38:03.445613 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started 0.259693 0 0.000000 2014-08-13 17:38:03.445614 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Reset 0.001176 7 0.000990 2014-08-13 17:38:03.445623 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.445626 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started 2014-08-13 17:38:03.445632 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194477 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.445637 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Start 2014-08-13 17:38:03.445642 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194479 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.445646 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: transitioning to Stray 2014-08-13 17:38:03.445651 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194479 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.445657 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.445661 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194479 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [339,45] newacting [339,45] 2014-08-13 17:38:03.445670 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194479 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.445669 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started/Stray 2014-08-13 17:38:03.445679 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [4,339,45] r=0 lpr=194479 pi=193306-194475/23 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.445682 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.445692 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.445697 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,339,45](4) acting [4,339,45](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.445710 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194476/194476/194476) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] noting past interval(194476-194478 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 17:38:03.445719 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] up [4,339,45] -> [339,45], acting [4,339,45] -> [339,45], acting_primary 4 -> 339, up_primary 4 -> 339, role 0 -> -1 2014-08-13 17:38:03.445729 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.445737 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.445745 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.445753 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.445753 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.445761 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.445769 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.445776 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.445784 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.445783 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [5,47]/[5,47] -- 5/5 2014-08-13 17:38:03.445792 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.445799 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445802 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.445802 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.445812 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.445813 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.445820 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.445822 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [5,47] newacting [5,47] 2014-08-13 17:38:03.445829 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445831 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.445837 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.445840 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [4,5,47] r=0 lpr=194478 pi=193306-194476/27 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.445845 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445853 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.445860 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.445859 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,5,47](4) acting [4,5,47](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.445869 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194476/194477/194477) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] noting past interval(194477-194478 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) 2014-08-13 17:38:03.445870 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.445878 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] up [4,5,47] -> [5,47], acting [4,5,47] -> [5,47], acting_primary 4 -> 5, up_primary 4 -> 5, role 0 -> -1 2014-08-13 17:38:03.445888 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.445890 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [339,45]/[339,45,55] -- 339/339 2014-08-13 17:38:03.445901 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.445897 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.445904 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.445906 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.445913 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.445914 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.445922 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.445921 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] acting_up_affected newup [339,45] newacting [339,45,55] 2014-08-13 17:38:03.445931 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.445931 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.445940 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.445939 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'383 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.445949 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.445957 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.445957 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [339,45](339) acting [339,45](339)): not rw, up_thru 194473 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [339,45](339) acting [339,45](339)) : primary up 192978-194473 does not include interval 2014-08-13 17:38:03.445967 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.445967 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194479/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [339,45](339) acting [339,45](339)) 2014-08-13 17:38:03.445976 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.445978 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] up [339,45] -> [339,45], acting [339,45] -> [339,45,55], acting_primary 339 -> 339, up_primary 339 -> 339, role -1 -> -1 2014-08-13 17:38:03.445984 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.445990 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.445993 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.445999 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.446001 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.446009 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.446008 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.446017 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.446019 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.446024 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.446028 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.446034 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.446038 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.446046 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.446054 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [5,47]/[5,47] -- 5/5 2014-08-13 17:38:03.446055 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.446066 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.446066 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.446069 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.446076 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.450989 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451001 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.450995 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.451019 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [5,47]/[5,47] -- 5/5 2014-08-13 17:38:03.451020 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451030 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451033 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451041 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451049 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451047 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [339,45]/[339,45,55] -- 339/339 2014-08-13 17:38:03.451060 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451064 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451070 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 17:38:03.451081 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451074 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451084 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451084 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451092 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451100 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] acting_up_affected newup [11,54,368] newacting [11,54,368] 2014-08-13 17:38:03.451108 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.451108 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [342,55,24]/[339,45,55] -- 342/339 2014-08-13 17:38:03.451120 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451117 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [5,47] r=-1 lpr=194479 pi=193306-194478/28 crt=191648'346 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.451123 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451132 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451134 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [5,47](5) acting [5,47](5)): not rw, up_thru 194479 up_from 192977 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) : primary up 192977-194479 includes interval 2014-08-13 17:38:03.451142 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] acting_up_affected newup [342,55,24] newacting [339,45,55] 2014-08-13 17:38:03.451143 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194479/194479/194479) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) 2014-08-13 17:38:03.451151 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.451153 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] up [5,47] -> [11,54,368], acting [5,47] -> [11,54,368], acting_primary 5 -> 11, up_primary 5 -> 11, role -1 -> -1 2014-08-13 17:38:03.451164 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451161 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [339,45]/[339,45,55] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'383 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.451172 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.451181 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451187 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [339,45](339) acting [339,45,55](339)): not rw, up_thru 194480 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [339,45](339) acting [339,45,55](339) maybe_went_rw) : primary up 192978-194480 includes interval 2014-08-13 17:38:03.451191 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.451197 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194479/194480/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [339,45](339) acting [339,45,55](339) maybe_went_rw) 2014-08-13 17:38:03.451204 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.451208 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] up [339,45] -> [342,55,24], acting [339,45,55] -> [339,45,55], acting_primary 339 -> 339, up_primary 339 -> 342, role -1 -> -1 2014-08-13 17:38:03.451213 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.451220 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.451221 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451229 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.451230 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.451239 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.451239 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.451249 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.451248 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] [5,47] -> [11,54,368], acting primary 5 -> 11 2014-08-13 17:38:03.451260 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.451264 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.451270 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.451272 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.451279 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.451282 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451290 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.451299 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.451304 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 17:38:03.451308 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.451316 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451318 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.451319 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451327 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451328 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451337 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451356 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [342,55,24]/[342,55,24] -- 342/342 2014-08-13 17:38:03.451368 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451370 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451379 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451379 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 17:38:03.451396 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451388 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] acting_up_affected newup [342,55,24] newacting [342,55,24] 2014-08-13 17:38:03.451398 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451398 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.451408 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24]/[339,45,55] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'383 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.451407 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451419 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451426 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)): not rw, up_thru 194480 up_from 192978 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) : primary up 192978-194480 does not include interval 2014-08-13 17:38:03.451437 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 17:38:03.451438 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194482/194479) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) 2014-08-13 17:38:03.451448 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451451 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451449 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] up [342,55,24] -> [342,55,24], acting [339,45,55] -> [342,55,24], acting_primary 339 -> 342, up_primary 342 -> 342, role -1 -> -1 2014-08-13 17:38:03.451460 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451460 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451469 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.451469 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,5,47] newacting [4,5,47] 2014-08-13 17:38:03.451480 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451480 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.451489 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [11,54,368] r=-1 lpr=194482 pi=193306-194481/29 crt=191648'346 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.451489 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.451499 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.451507 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.451507 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [11,54,368](11) acting [11,54,368](11)): not rw, up_thru 194482 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) : primary up 192970-194482 includes interval 2014-08-13 17:38:03.451515 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451521 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194482/194482/194482) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 17:38:03.451525 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.451531 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [11,54,368] -> [4,5,47], acting [11,54,368] -> [4,5,47], acting_primary 11 -> 4, up_primary 11 -> 4, role -1 -> 0 2014-08-13 17:38:03.451535 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.451543 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.451544 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] [339,45,55] -> [342,55,24], acting primary 339 -> 342 2014-08-13 17:38:03.451552 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.451559 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.451562 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451567 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.451572 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:3003 2014-08-13 17:38:03.451577 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451582 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451591 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.451599 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.451600 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [342,55,24]/[342,55,24] -- 342/342 2014-08-13 17:38:03.451612 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451608 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451616 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451614 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451624 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451626 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451632 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.451635 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451641 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451650 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.451653 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 17:38:03.451665 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.451659 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.451667 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.451668 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.451678 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451679 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.451687 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.451686 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,339,45] newacting [4,339,45] 2014-08-13 17:38:03.451695 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.451696 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.451704 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.451705 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [342,55,24] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'383 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.451712 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.451721 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451723 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [342,55,24](342) acting [342,55,24](342)): not rw, up_thru 194483 up_from 192964 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) : primary up 192964-194483 includes interval 2014-08-13 17:38:03.451731 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.451733 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194482/194483/194483) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 17:38:03.451741 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,5,47 2014-08-13 17:38:03.451750 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.451743 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [342,55,24] -> [4,339,45], acting [342,55,24] -> [4,339,45], acting_primary 342 -> 4, up_primary 342 -> 4, role -1 -> 0 2014-08-13 17:38:03.451752 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.451754 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.451761 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.813595 9 0.005841 2014-08-13 17:38:03.451764 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.451772 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.451773 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451781 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.451784 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2930 2014-08-13 17:38:03.451790 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.451796 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451800 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.451806 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.451812 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.451815 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.451821 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.451824 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451831 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.451833 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451840 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.451841 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.451850 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.451851 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 17:38:03.451859 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.451863 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) 2014-08-13 17:38:03.451868 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.451874 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) 2014-08-13 17:38:03.451878 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.451887 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.451886 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,5,47](4) acting [5,47,11](5)) 2014-08-13 17:38:03.451896 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.451896 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [5,47](5) acting [5,47,11](5) maybe_went_rw) 2014-08-13 17:38:03.451906 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.451908 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [5,47](5) acting [5,47](5)) 2014-08-13 17:38:03.451914 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.451919 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,5,11,47,54,368 down blocked_by {} 2014-08-13 17:38:03.451923 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.451929 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.451932 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.451941 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:3004 2014-08-13 17:38:03.451942 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.451951 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.5 2014-08-13 17:38:03.451952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.451961 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.11 2014-08-13 17:38:03.451963 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,45,339 unchanged 2014-08-13 17:38:03.451970 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.47 2014-08-13 17:38:03.451973 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.451980 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.54 2014-08-13 17:38:03.451983 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.006360 7 0.006243 2014-08-13 17:38:03.451989 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 17:38:03.451995 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.452000 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.452005 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.452009 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.452013 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.452024 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.452036 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.452045 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.452054 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.452064 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.452063 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.452073 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 17:38:03.452084 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) 2014-08-13 17:38:03.452097 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [339,45](339) acting [339,45,55](339) maybe_went_rw) 2014-08-13 17:38:03.452107 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [339,45](339) acting [339,45](339)) 2014-08-13 17:38:03.452116 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 17:38:03.452127 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [339,45](339) acting [339,45](339) maybe_went_rw) 2014-08-13 17:38:03.452138 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:03.452144 7fe99f5e9700 7 osd.4 194485 do_notify osd 17 on 1 PGs 2014-08-13 17:38:03.452137 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 17:38:03.452147 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,24,45,55,339,342 down blocked_by {} 2014-08-13 17:38:03.452157 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.452165 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:03.452167 7fe99f5e9700 7 osd.4 194485 do_notify osd 289 on 1 PGs 2014-08-13 17:38:03.452180 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:03.452168 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2931 2014-08-13 17:38:03.452182 7fe99f5e9700 7 osd.4 194485 do_notify osd 367 on 1 PGs 2014-08-13 17:38:03.452185 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.24 2014-08-13 17:38:03.452198 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d36b40 don't know epoch, doing nothing 2014-08-13 17:38:03.452201 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.5 on 1 PGs 2014-08-13 17:38:03.452196 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 17:38:03.452214 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecaec0 don't know epoch, doing nothing 2014-08-13 17:38:03.452205 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.55 2014-08-13 17:38:03.452216 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.11 on 1 PGs 2014-08-13 17:38:03.452217 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 17:38:03.452229 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:03.452231 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.15 on 1 PGs 2014-08-13 17:38:03.452227 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 17:38:03.452240 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.452249 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.452305 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.452327 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d335a0 don't know epoch, doing nothing 2014-08-13 17:38:03.452332 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.21 on 1 PGs 2014-08-13 17:38:03.452346 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4829b80 don't know epoch, doing nothing 2014-08-13 17:38:03.452348 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.47 sending saperate messages on 1 PGs 2014-08-13 17:38:03.452358 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a4f380 don't know epoch, doing nothing 2014-08-13 17:38:03.452359 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.54 on 1 PGs 2014-08-13 17:38:03.452371 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.452372 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.303 on 1 PGs 2014-08-13 17:38:03.452358 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452379 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452385 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecf900 don't know epoch, doing nothing 2014-08-13 17:38:03.452387 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.310 on 1 PGs 2014-08-13 17:38:03.452383 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452398 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8580 don't know epoch, doing nothing 2014-08-13 17:38:03.452400 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.342 on 1 PGs 2014-08-13 17:38:03.452394 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452410 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5251080 don't know epoch, doing nothing 2014-08-13 17:38:03.452412 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.368 on 1 PGs 2014-08-13 17:38:03.452417 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452430 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452432 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452441 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452461 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452474 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452476 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452485 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452483 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 17:38:03.452501 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452505 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.452506 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452519 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452514 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374] newacting [358,374] 2014-08-13 17:38:03.452522 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452523 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.452532 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452533 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.263644 3 0.000246 2014-08-13 17:38:03.452545 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started 0.263718 0 0.000000 2014-08-13 17:38:03.452555 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.452553 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452566 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452563 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194477 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.452569 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452573 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194479 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.452579 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452582 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194479 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.452590 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194479 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374] newacting [358,374] 2014-08-13 17:38:03.452598 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194479 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.452600 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452612 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452607 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374,4] r=2 lpr=194479 pi=192969-194475/21 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.452615 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452626 7fe97ac06700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.452624 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452624 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [358,374,4](358) acting [358,374,4](358)): not rw, up_thru 194476 up_from 192966 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [358,374,4](358) acting [358,374,4](358) maybe_went_rw) : primary up 192966-194476 includes interval 2014-08-13 17:38:03.452635 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [358,374,4](358) acting [358,374,4](358) maybe_went_rw) 2014-08-13 17:38:03.452644 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374,4] -> [358,374], acting [358,374,4] -> [358,374], acting_primary 358 -> 358, up_primary 358 -> 358, role 2 -> -1 2014-08-13 17:38:03.452646 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:03.452659 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452655 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.452662 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:03.452663 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.452671 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452672 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.452683 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:03.452684 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.452693 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.452700 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.452709 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.452717 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.452726 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.452733 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.452741 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.452745 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:03.452749 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.452757 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.452757 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194485 2014-08-13 17:38:03.452767 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.452768 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452778 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.452791 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 17:38:03.452802 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452805 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.452801 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 17:38:03.452819 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452813 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.452822 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452822 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Started advmap 2014-08-13 17:38:03.452833 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323] newacting [304,323] 2014-08-13 17:38:03.452840 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 17:38:03.452841 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.452852 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452855 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.452854 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 0.287423 3 0.000166 2014-08-13 17:38:03.452863 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.452867 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started 0.287480 0 0.000000 2014-08-13 17:38:03.452872 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.452878 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset 2014-08-13 17:38:03.452887 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194477 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194479 2014-08-13 17:38:03.452893 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 17:38:03.452904 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.452900 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194479 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.452906 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.452910 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194479 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.452915 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.452919 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194479 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323] newacting [304,323] 2014-08-13 17:38:03.452923 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374,2] newacting [358,374,2] 2014-08-13 17:38:03.452929 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194479 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.452935 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.452942 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323,4] r=2 lpr=194479 pi=193250-194475/26 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194479 2014-08-13 17:38:03.452944 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374] r=-1 lpr=194479 pi=192969-194478/22 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.452960 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [304,323,4](304) acting [304,323,4](304)): not rw, up_thru 194476 up_from 192965 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [304,323,4](304) acting [304,323,4](304) maybe_went_rw) : primary up 192965-194476 includes interval 2014-08-13 17:38:03.452960 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [358,374](358) acting [358,374](358)): not rw, up_thru 194479 up_from 192966 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [358,374](358) acting [358,374](358) maybe_went_rw) : primary up 192966-194479 includes interval 2014-08-13 17:38:03.452971 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194476/194476/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194476-194478 up [304,323,4](304) acting [304,323,4](304) maybe_went_rw) 2014-08-13 17:38:03.452972 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [358,374](358) acting [358,374](358) maybe_went_rw) 2014-08-13 17:38:03.452981 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323,4] -> [304,323], acting [304,323,4] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role 2 -> -1 2014-08-13 17:38:03.452982 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374] -> [358,374,2], acting [358,374] -> [358,374,2], acting_primary 358 -> 358, up_primary 358 -> 358, role -1 -> -1 2014-08-13 17:38:03.452994 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:03.452995 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.453004 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453004 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453013 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.453014 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.453023 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 17:38:03.453023 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.453032 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.453031 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 17:38:03.453041 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.453042 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 17:38:03.453050 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.453051 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:03.453058 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.453061 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 17:38:03.453067 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.453070 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 17:38:03.453076 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.453079 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_role_change 2014-08-13 17:38:03.453085 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.453089 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.453094 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453098 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 17:38:03.453107 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 17:38:03.453116 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453117 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 17:38:03.453128 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453131 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.453136 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 17:38:03.453139 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.453151 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453148 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453154 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453163 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453169 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 17:38:03.453180 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453178 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453182 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.453196 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 17:38:03.453199 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.453209 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453209 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453212 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453222 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453230 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453233 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453234 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c96c0 pg_notify(22.707(1) epoch 194485) v5 2014-08-13 17:38:03.453227 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 17:38:03.453231 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453239 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453244 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.453252 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.453262 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.453252 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,373]/[304,323,373] -- 304/304 2014-08-13 17:38:03.453264 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c96c0 2014-08-13 17:38:03.453263 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374,4] newacting [358,374,4] 2014-08-13 17:38:03.453267 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453272 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.453274 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453280 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,2] r=-1 lpr=194482 pi=192969-194481/23 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.453284 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453293 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453296 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453294 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323,373] newacting [304,323,373] 2014-08-13 17:38:03.453298 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [358,374,2](358) acting [358,374,2](358)): not rw, up_thru 194482 up_from 192966 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [358,374,2](358) acting [358,374,2](358) maybe_went_rw) : primary up 192966-194482 includes interval 2014-08-13 17:38:03.453304 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.453307 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [358,374,2](358) acting [358,374,2](358) maybe_went_rw) 2014-08-13 17:38:03.453313 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323] r=-1 lpr=194479 pi=193250-194478/27 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194482 2014-08-13 17:38:03.453318 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374,2] -> [358,374,4], acting [358,374,2] -> [358,374,4], acting_primary 358 -> 358, up_primary 358 -> 358, role -1 -> 2 2014-08-13 17:38:03.453329 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.453331 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [304,323](304) acting [304,323](304)): not rw, up_thru 194479 up_from 192965 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [304,323](304) acting [304,323](304) maybe_went_rw) : primary up 192965-194479 includes interval 2014-08-13 17:38:03.453337 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453341 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194479/194479/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194479-194481 up [304,323](304) acting [304,323](304) maybe_went_rw) 2014-08-13 17:38:03.453346 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.453355 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.453352 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323] -> [304,323,373], acting [304,323] -> [304,323,373], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 17:38:03.453364 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.453364 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:03.453372 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.453374 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453381 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.453383 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.453391 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453394 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453396 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265400 pg_notify(22.1f1(29) epoch 194485) v5 2014-08-13 17:38:03.453390 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.453393 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 17:38:03.453400 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.453403 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 17:38:03.453410 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.453412 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 17:38:03.453424 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.54 2014-08-13 17:38:03.453418 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.453427 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265400 2014-08-13 17:38:03.453421 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:03.453432 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 17:38:03.453427 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.453442 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 17:38:03.453457 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453450 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.453459 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453453 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 17:38:03.453461 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453463 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 17:38:03.453471 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.453478 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453480 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453473 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453481 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5480000 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.453486 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.453489 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.453491 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453492 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453503 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453504 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453495 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,373]/[304,323] -- 304/304 2014-08-13 17:38:03.453505 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db3cc0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.453508 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.453509 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453510 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.453513 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453515 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453512 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.453513 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453526 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453527 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453529 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42608c0 pg_notify(22.707(33) epoch 194485) v5 2014-08-13 17:38:03.453523 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.453524 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453534 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323,373] newacting [304,323] 2014-08-13 17:38:03.453533 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.000978 7 0.000817 2014-08-13 17:38:03.453552 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.15 2014-08-13 17:38:03.453545 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.453547 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.453554 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42608c0 2014-08-13 17:38:03.453554 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373] r=-1 lpr=194482 pi=193250-194481/28 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194483 2014-08-13 17:38:03.453556 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.453565 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.453577 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453579 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453573 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [304,323,373](304) acting [304,323,373](304)): not rw, up_thru 194479 up_from 192965 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [304,323,373](304) acting [304,323,373](304)) : primary up 192965-194479 does not include interval 2014-08-13 17:38:03.453576 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.453583 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194482/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] noting past interval(194482-194482 up [304,323,373](304) acting [304,323,373](304)) 2014-08-13 17:38:03.453588 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.453600 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453603 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453604 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5138000 pg_notify(22.1f1(1) epoch 194485) v5 2014-08-13 17:38:03.453595 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] up [304,323,373] -> [304,323,373], acting [304,323,373] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 17:38:03.453600 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.453608 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 17:38:03.453613 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.453618 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453630 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.11 2014-08-13 17:38:03.453632 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5138000 2014-08-13 17:38:03.453629 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.453640 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_copy_ops 2014-08-13 17:38:03.453649 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_flush_ops 2014-08-13 17:38:03.453663 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.453658 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change_cleanup 2014-08-13 17:38:03.453666 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453668 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453667 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 17:38:03.453678 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453680 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453681 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e200 pg_notify(22.707 epoch 194485) v5 2014-08-13 17:38:03.453677 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit NotTrimming 2014-08-13 17:38:03.453687 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter NotTrimming 2014-08-13 17:38:03.453702 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.310 2014-08-13 17:38:03.453703 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e200 2014-08-13 17:38:03.453690 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [330,28]/[17,315] -- 330/17 2014-08-13 17:38:03.453696 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_recovery 2014-08-13 17:38:03.453710 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453706 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] clear_recovery_state 2014-08-13 17:38:03.453714 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 17:38:03.453723 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453725 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453716 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453724 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.453737 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453738 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453739 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e3c0 pg_notify(22.707 epoch 194485) v5 2014-08-13 17:38:03.453735 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [330,28] newacting [17,315] 2014-08-13 17:38:03.453741 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,373]/[304,323] -- 304/304 2014-08-13 17:38:03.453747 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.453757 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453763 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.21 2014-08-13 17:38:03.453764 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e3c0 2014-08-13 17:38:03.453758 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [4,330,28]/[4,17,315] r=0 lpr=194478 pi=193748-194476/29 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194479 2014-08-13 17:38:03.453760 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453778 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,330,28](4) acting [4,17,315](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.453788 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453779 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453789 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453790 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453789 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194476/194477/194477) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] noting past interval(194477-194478 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) 2014-08-13 17:38:03.453802 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453804 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453805 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513de80 pg_notify(22.707(33) epoch 194485) v5 2014-08-13 17:38:03.453801 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] up [4,330,28] -> [330,28], acting [4,17,315] -> [17,315], acting_primary 4 -> 17, up_primary 4 -> 330, role 0 -> -1 2014-08-13 17:38:03.453809 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,4]/[304,323] -- 304/304 2014-08-13 17:38:03.453813 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] clear_primary_state 2014-08-13 17:38:03.453823 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.453828 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.453830 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513de80 2014-08-13 17:38:03.453824 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] agent_stop 2014-08-13 17:38:03.453826 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:03.453834 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] on_change 2014-08-13 17:38:03.453836 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.453851 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453852 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453845 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453848 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323] 2014-08-13 17:38:03.453855 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:03.453859 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.453870 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453872 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453865 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 17:38:03.453873 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce200 pg_notify(22.1f1(29) epoch 194485) v5 2014-08-13 17:38:03.453870 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,373]/[304,323] r=-1 lpr=194483 pi=193250-194482/29 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194485 2014-08-13 17:38:03.453875 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 17:38:03.453886 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 17:38:03.453897 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.368 2014-08-13 17:38:03.453889 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [304,323,373](304) acting [304,323](304)): not rw, up_thru 194483 up_from 192965 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [304,323,373](304) acting [304,323](304) maybe_went_rw) : primary up 192965-194483 includes interval 2014-08-13 17:38:03.453899 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce200 2014-08-13 17:38:03.453896 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] on_change 2014-08-13 17:38:03.453900 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194482/194483/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] noting past interval(194483-194484 up [304,323,373](304) acting [304,323](304) maybe_went_rw) 2014-08-13 17:38:03.453907 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 17:38:03.453920 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453921 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453912 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] up [304,323,373] -> [304,323,4], acting [304,323] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 17:38:03.453918 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 17:38:03.453925 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 17:38:03.453928 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_stats 2014-08-13 17:38:03.453935 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.453937 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.453944 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.453946 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_role_change 2014-08-13 17:38:03.453954 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_copy_ops 2014-08-13 17:38:03.453955 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.453964 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_flush_ops 2014-08-13 17:38:03.453964 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.453973 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.453977 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453978 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db1b00 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.453974 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.453973 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change_cleanup 2014-08-13 17:38:03.453984 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 17:38:03.453985 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.453993 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.453997 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.453999 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454000 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.453994 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit NotTrimming 2014-08-13 17:38:03.454004 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter NotTrimming 2014-08-13 17:38:03.454006 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [330,28]/[17,315,327] -- 330/17 2014-08-13 17:38:03.454020 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454013 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_recovery 2014-08-13 17:38:03.454020 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454022 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454023 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51439c0 pg_notify(22.1f1 epoch 194485) v5 2014-08-13 17:38:03.454023 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] clear_recovery_state 2014-08-13 17:38:03.454024 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454035 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454035 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454053 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.47 2014-08-13 17:38:03.454055 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51439c0 2014-08-13 17:38:03.454047 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [330,28] newacting [17,315,327] 2014-08-13 17:38:03.454048 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_activate_map 2014-08-13 17:38:03.454057 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454067 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315] r=-1 lpr=194479 pi=193748-194478/30 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194480 2014-08-13 17:38:03.454084 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [330,28](330) acting [17,315](17)): not rw, up_thru 194476 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [330,28](330) acting [17,315](17)) : primary up 192970-194476 does not include interval 2014-08-13 17:38:03.454094 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454097 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454092 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] update_heartbeat_peers unchanged 2014-08-13 17:38:03.454093 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194479/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] noting past interval(194479-194479 up [330,28](330) acting [17,315](17)) 2014-08-13 17:38:03.454111 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454112 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454113 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4db3600 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.454106 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] up [330,28] -> [330,28], acting [17,315] -> [17,315,327], acting_primary 17 -> 17, up_primary 330 -> 330, role -1 -> -1 2014-08-13 17:38:03.454118 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.454121 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.454122 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454123 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454119 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.454128 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.454105 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] take_waiters 2014-08-13 17:38:03.454142 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454143 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454137 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.454149 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143640 pg_notify(22.1f1 epoch 194485) v5 2014-08-13 17:38:03.454140 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Reset 0.001262 7 0.001054 2014-08-13 17:38:03.454147 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.454155 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started 2014-08-13 17:38:03.454157 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.454174 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.5 2014-08-13 17:38:03.454167 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.454176 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143640 2014-08-13 17:38:03.454166 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Start 2014-08-13 17:38:03.454177 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.454182 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: transitioning to Stray 2014-08-13 17:38:03.454188 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.454201 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.454195 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Start 0.000029 0 0.000000 2014-08-13 17:38:03.454204 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.454204 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.454207 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started/Stray 2014-08-13 17:38:03.454214 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.454221 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.454223 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.454235 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.454255 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [330,28]/[17,315,327] -- 330/17 2014-08-13 17:38:03.454272 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454275 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454291 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.454284 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454295 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454319 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [327,17,315]/[17,315,327] -- 327/17 2014-08-13 17:38:03.454333 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454329 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.454336 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454345 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454345 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454349 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.454354 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [327,17,315] newacting [17,315,327] 2014-08-13 17:38:03.454359 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.454365 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454367 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.454374 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [330,28]/[17,315,327] r=-1 lpr=194480 pi=193748-194479/31 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194482 2014-08-13 17:38:03.454378 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.288232 2 0.000174 2014-08-13 17:38:03.454389 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.454391 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [330,28](330) acting [17,315,327](17)): not rw, up_thru 194480 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [330,28](330) acting [17,315,327](17) maybe_went_rw) : primary up 192970-194480 includes interval 2014-08-13 17:38:03.454398 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.288261 0 0.000000 2014-08-13 17:38:03.454401 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194479/194480/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] noting past interval(194480-194481 up [330,28](330) acting [17,315,327](17) maybe_went_rw) 2014-08-13 17:38:03.454409 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.288283 0 0.000000 2014-08-13 17:38:03.454412 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] up [330,28] -> [327,17,315], acting [17,315,327] -> [17,315,327], acting_primary 17 -> 17, up_primary 330 -> 327, role -1 -> -1 2014-08-13 17:38:03.454419 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.288331 0 0.000000 2014-08-13 17:38:03.454424 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.454432 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.454434 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.454440 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194477 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.454443 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.454449 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.454454 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.454460 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.454464 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.454470 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340] newacting [340] 2014-08-13 17:38:03.454479 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454487 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [4,340] r=0 lpr=194479 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.454474 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.454502 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 17:38:03.454505 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,340](4) acting [4,340](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.454512 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.454514 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194476/194476/194476) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] noting past interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:03.454523 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.454524 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] up [4,340] -> [340], acting [4,340] -> [340], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 17:38:03.454534 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.454533 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.454543 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.454542 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.454552 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.454553 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454560 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.454568 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.454576 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.454577 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [327,17,315]/[327,17,315] -- 327/327 2014-08-13 17:38:03.454584 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.454592 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454593 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.454595 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454601 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.454605 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454610 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.454615 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [327,17,315] newacting [327,17,315] 2014-08-13 17:38:03.454619 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.454625 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454628 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.454635 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315]/[17,315,327] r=-1 lpr=194482 pi=193748-194481/32 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194483 2014-08-13 17:38:03.454636 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.454644 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.454651 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.454652 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)): not rw, up_thru 194480 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) : primary up 192970-194480 does not include interval 2014-08-13 17:38:03.454660 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.454662 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194482/194479) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] noting past interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 17:38:03.454668 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.454672 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] up [327,17,315] -> [327,17,315], acting [17,315,327] -> [327,17,315], acting_primary 17 -> 327, up_primary 327 -> 327, role -1 -> -1 2014-08-13 17:38:03.454677 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454684 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.454692 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.454697 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.454701 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.454708 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454710 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.454712 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.454719 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.454720 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.454727 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.454730 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454736 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.454745 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.454746 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 17:38:03.454756 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454754 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.454759 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.454768 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.454763 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] [17,315,327] -> [327,17,315], acting primary 17 -> 327 2014-08-13 17:38:03.454776 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454778 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_recovery 2014-08-13 17:38:03.454786 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] clear_recovery_state 2014-08-13 17:38:03.454795 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454796 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.454806 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454808 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.454816 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.454817 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [327,17,315]/[327,17,315] -- 327/327 2014-08-13 17:38:03.454824 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [324,370] newacting [324,370] 2014-08-13 17:38:03.454832 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454833 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454835 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454841 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [340] r=-1 lpr=194479 pi=194476-194478/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.454844 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454853 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454856 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [340](340) acting [340](340)): not rw, up_thru 194479 up_from 192944 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) : primary up 192944-194479 includes interval 2014-08-13 17:38:03.454865 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194479/194479/194479) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.454871 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [4,330,28]/[4,330,28] -- 4/4 2014-08-13 17:38:03.454874 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] up [340] -> [324,370], acting [340] -> [324,370], acting_primary 340 -> 324, up_primary 340 -> 324, role -1 -> -1 2014-08-13 17:38:03.454884 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.454884 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.454886 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 17:38:03.454892 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.454896 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.454901 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.454905 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] acting_up_affected newup [4,330,28] newacting [4,330,28] 2014-08-13 17:38:03.454910 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.454915 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.454919 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.454924 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [327,17,315] r=-1 lpr=194483 pi=193748-194482/33 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] set_last_peering_reset 194485 2014-08-13 17:38:03.454927 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.454936 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.454944 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.454942 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [327,17,315](327) acting [327,17,315](327)): not rw, up_thru 194483 up_from 192966 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) : primary up 192966-194483 includes interval 2014-08-13 17:38:03.454952 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.454952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194482/194483/194483) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] noting past interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 17:38:03.454964 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] up [327,17,315] -> [4,330,28], acting [327,17,315] -> [4,330,28], acting_primary 327 -> 4, up_primary 327 -> 4, role -1 -> 0 2014-08-13 17:38:03.454961 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] [340] -> [324,370], acting primary 340 -> 324 2014-08-13 17:38:03.454975 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] clear_primary_state 2014-08-13 17:38:03.454977 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.454985 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.454985 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] agent_stop 2014-08-13 17:38:03.454994 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.454995 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.455005 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd 194485:22345 2014-08-13 17:38:03.455015 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.455015 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.455026 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.455024 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 17:38:03.455029 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.455033 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 17:38:03.455038 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.455042 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.455047 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.455051 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.455059 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 17:38:03.455067 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 17:38:03.455066 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 17:38:03.455077 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.455076 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 17:38:03.455079 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.455086 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 17:38:03.455088 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.455095 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 17:38:03.455099 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.455105 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] clear_stats 2014-08-13 17:38:03.455114 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 17:38:03.455114 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:03.455126 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.455123 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] on_role_change 2014-08-13 17:38:03.455128 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.455132 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 17:38:03.455136 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.455141 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] cancel_recovery 2014-08-13 17:38:03.455145 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340] newacting [4,340] 2014-08-13 17:38:03.455149 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] clear_recovery_state 2014-08-13 17:38:03.455154 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.455159 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.455164 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [324,370] r=-1 lpr=194482 pi=194476-194481/2 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.455170 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_activate_map 2014-08-13 17:38:03.455182 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] update_heartbeat_peers -> 4,28,330 2014-08-13 17:38:03.455185 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [324,370](324) acting [324,370](324)): not rw, up_thru 194482 up_from 192951 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) : primary up 192951-194482 includes interval 2014-08-13 17:38:03.455192 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.455194 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194482/194482/194482) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.455198 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] take_waiters 2014-08-13 17:38:03.455204 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [324,370] -> [4,340], acting [324,370] -> [4,340], acting_primary 324 -> 4, up_primary 324 -> 4, role -1 -> 0 2014-08-13 17:38:03.455209 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.637341 9 0.001315 2014-08-13 17:38:03.455215 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.455221 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started 2014-08-13 17:38:03.455225 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.455230 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Start 2014-08-13 17:38:03.455234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.455240 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary 2014-08-13 17:38:03.455244 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:177951 2014-08-13 17:38:03.455251 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.455254 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.455263 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.455262 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 17:38:03.455272 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.455274 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering 2014-08-13 17:38:03.455280 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.455284 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.455289 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.455294 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.455297 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.455305 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.455305 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 17:38:03.455315 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.455319 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 17:38:03.455324 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.455330 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194480-194481 up [330,28](330) acting [17,315,327](17) maybe_went_rw) 2014-08-13 17:38:03.455333 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.455343 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.455341 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194479-194479 up [330,28](330) acting [17,315](17)) 2014-08-13 17:38:03.455351 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.455352 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194477-194478 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) 2014-08-13 17:38:03.455360 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.455364 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194476-194476 up [4,330,28](4) acting [17,315,327](17)) 2014-08-13 17:38:03.455368 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.455376 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.455374 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194474-194475 up [330,28](330) acting [17,315,327](17) maybe_went_rw) 2014-08-13 17:38:03.455384 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.455386 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194473-194473 up [330,28](330) acting [17,315](17)) 2014-08-13 17:38:03.455393 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.455397 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 17:38:03.455404 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.455407 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.455414 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340 unchanged 2014-08-13 17:38:03.455419 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] publish_stats_to_osd 194485:22346 2014-08-13 17:38:03.455423 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.455429 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.17 2014-08-13 17:38:03.455433 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001001 7 0.000919 2014-08-13 17:38:03.455444 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.455452 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.455460 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.455440 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.28 2014-08-13 17:38:03.455470 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.455474 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.315 2014-08-13 17:38:03.455481 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.455484 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.327 2014-08-13 17:38:03.455490 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.455494 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.330 2014-08-13 17:38:03.455500 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.455505 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.455509 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.455515 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.455520 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.455531 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.462179 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:03.455576 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.462199 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 17:38:03.462209 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.462220 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:177952 2014-08-13 17:38:03.462230 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 17:38:03.462240 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.462249 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.370 2014-08-13 17:38:03.462259 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.462252 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 17:38:03.462268 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.462282 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462287 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.462297 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [47,378] newacting [19,65] 2014-08-13 17:38:03.462306 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.462307 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.462320 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.272940 3 0.000113 2014-08-13 17:38:03.462332 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.272994 0 0.000000 2014-08-13 17:38:03.462344 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.462341 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462353 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.462360 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462363 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462364 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.462372 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.462374 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462382 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [47,378] newacting [19,65] 2014-08-13 17:38:03.462383 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [26,30,340] newacting [26,30,340] 2014-08-13 17:38:03.462392 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.462393 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.462401 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [4,47,378]/[19,65] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.462403 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340]/[4] r=0 lpr=194478 pi=193125-194476/50 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.462419 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [26,30,340](26) acting [4](4)) : acting set is too small 2014-08-13 17:38:03.462420 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,47,378](4) acting [19,65](19)): not rw, up_thru 194476 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194476-194478 up [4,47,378](4) acting [19,65](19) maybe_went_rw) : primary up 192976-194476 includes interval 2014-08-13 17:38:03.462429 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194477/194477) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] noting past interval(194477-194478 up [26,30,340](26) acting [4](4)) 2014-08-13 17:38:03.462431 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194476-194478 up [4,47,378](4) acting [19,65](19) maybe_went_rw) 2014-08-13 17:38:03.462440 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] up [26,30,340] -> [26,30,340], acting [4] -> [26,30,340], acting_primary 4 -> 26, up_primary 26 -> 26, role 0 -> -1 2014-08-13 17:38:03.462442 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] up [4,47,378] -> [47,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 4 -> 47, role -1 -> -1 2014-08-13 17:38:03.462451 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.462454 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.462461 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.462465 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.462470 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.462474 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.462479 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.462485 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.462488 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.462495 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.462498 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.462504 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.462506 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.462514 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.462515 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.462524 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.462525 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.462534 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.462536 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.462544 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.462545 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.462552 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.462554 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.462561 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.462565 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462570 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.462578 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.462586 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.462592 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 17:38:03.462596 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.462608 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462606 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462611 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462621 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.462631 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462631 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462645 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462648 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462656 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462655 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 17:38:03.462669 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462666 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462671 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462681 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.462691 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462688 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462720 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462715 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 17:38:03.462728 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462723 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462730 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462733 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462740 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.462742 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462750 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [47,19,378] newacting [19,65] 2014-08-13 17:38:03.462759 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.462763 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462768 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,378]/[19,65] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.462777 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462780 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462785 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [47,378](47) acting [19,65](19)): not rw, up_thru 194479 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194479-194481 up [47,378](47) acting [19,65](19) maybe_went_rw) : primary up 192976-194479 includes interval 2014-08-13 17:38:03.462789 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462795 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194479-194481 up [47,378](47) acting [19,65](19) maybe_went_rw) 2014-08-13 17:38:03.462798 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462806 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] up [47,378] -> [47,19,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 47 -> 47, role -1 -> -1 2014-08-13 17:38:03.462817 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.462819 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462832 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462827 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.462835 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462836 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.462844 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462846 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.462852 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462856 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.462865 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.462873 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.462874 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462887 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462884 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.462890 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462894 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.462899 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462903 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.462908 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462912 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.462922 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462929 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 17:38:03.462941 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462944 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462946 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 17:38:03.462952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:03.462961 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.462961 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.462964 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.462970 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.462975 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.462984 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.463008 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 17:38:03.463020 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.463023 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.463024 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.463032 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.463035 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.463042 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.463044 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Reset 0.706919 9 0.000451 2014-08-13 17:38:03.463055 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.463064 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.463065 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 17:38:03.463079 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.463072 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.463081 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.463083 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.463091 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.463094 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.463100 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [4,47,378] newacting [19,65] 2014-08-13 17:38:03.463105 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.463110 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.463115 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.463120 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [47,19,378]/[19,65] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.463137 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [47,19,378](47) acting [19,65](19)): not rw, up_thru 194482 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194482-194484 up [47,19,378](47) acting [19,65](19) maybe_went_rw) : primary up 192976-194482 includes interval 2014-08-13 17:38:03.463147 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194482-194484 up [47,19,378](47) acting [19,65](19) maybe_went_rw) 2014-08-13 17:38:03.463157 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] up [47,19,378] -> [4,47,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 47 -> 4, role -1 -> -1 2014-08-13 17:38:03.463167 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.463185 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.463178 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.463187 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.463204 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.463216 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.463225 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.463234 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.463243 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.463251 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e2aa0 don't know epoch, doing nothing 2014-08-13 17:38:03.463257 7fe99ebe8700 7 osd.4 194485 do_notify osd 9 on 1 PGs 2014-08-13 17:38:03.463253 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.463262 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.463277 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x592c360 don't know epoch, doing nothing 2014-08-13 17:38:03.463271 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.463279 7fe99ebe8700 7 osd.4 194485 do_notify osd 26 sending separate messages 2014-08-13 17:38:03.463281 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.463291 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 17:38:03.463302 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a4f380 don't know epoch, doing nothing 2014-08-13 17:38:03.463303 7fe99ebe8700 7 osd.4 194485 do_notify osd 54 on 1 PGs 2014-08-13 17:38:03.463314 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:03.463316 7fe99ebe8700 7 osd.4 194485 do_notify osd 58 on 1 PGs 2014-08-13 17:38:03.463313 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.463330 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:03.463332 7fe99ebe8700 7 osd.4 194485 do_notify osd 304 on 1 PGs 2014-08-13 17:38:03.463343 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:03.463345 7fe99ebe8700 7 osd.4 194485 do_notify osd 320 on 1 PGs 2014-08-13 17:38:03.463323 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 17:38:03.463452 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c73f40 don't know epoch, doing nothing 2014-08-13 17:38:03.463457 7fe99ebe8700 7 osd.4 194485 do_notify osd 321 on 1 PGs 2014-08-13 17:38:03.463456 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.001113 7 0.000816 2014-08-13 17:38:03.463472 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:03.463474 7fe99ebe8700 7 osd.4 194485 do_notify osd 324 on 1 PGs 2014-08-13 17:38:03.463470 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started 2014-08-13 17:38:03.463486 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c70f20 already has epoch 194485 2014-08-13 17:38:03.463488 7fe99ebe8700 7 osd.4 194485 do_notify osd 345 on 1 PGs 2014-08-13 17:38:03.463481 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start 2014-08-13 17:38:03.463490 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.463499 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing 2014-08-13 17:38:03.463501 7fe99ebe8700 7 osd.4 194485 do_notify osd 364 on 1 PGs 2014-08-13 17:38:03.463501 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.463511 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c71b80 don't know epoch, doing nothing 2014-08-13 17:38:03.463513 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.20 on 1 PGs 2014-08-13 17:38:03.463513 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray 2014-08-13 17:38:03.463527 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e3ef60 don't know epoch, doing nothing 2014-08-13 17:38:03.463529 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.24 on 1 PGs 2014-08-13 17:38:03.463526 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.463540 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing 2014-08-13 17:38:03.463542 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.25 on 1 PGs 2014-08-13 17:38:03.463536 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.463556 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d348e0 don't know epoch, doing nothing 2014-08-13 17:38:03.463558 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.44 on 1 PGs 2014-08-13 17:38:03.463571 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:03.463573 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.45 on 1 PGs 2014-08-13 17:38:03.463575 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.463584 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecf640 don't know epoch, doing nothing 2014-08-13 17:38:03.463586 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.48 on 1 PGs 2014-08-13 17:38:03.463599 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing 2014-08-13 17:38:03.463600 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.55 on 1 PGs 2014-08-13 17:38:03.463657 7fe97aa04700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.463687 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:03.463692 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.58 on 1 PGs 2014-08-13 17:38:03.463708 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:03.463710 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.61 on 1 PGs 2014-08-13 17:38:03.463722 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a4dd80 don't know epoch, doing nothing 2014-08-13 17:38:03.463723 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.62 on 1 PGs 2014-08-13 17:38:03.463734 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing 2014-08-13 17:38:03.463735 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.65 on 1 PGs 2014-08-13 17:38:03.463748 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:03.463749 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.285 on 1 PGs 2014-08-13 17:38:03.463760 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing 2014-08-13 17:38:03.463762 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.291 on 1 PGs 2014-08-13 17:38:03.463755 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 17:38:03.463772 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing 2014-08-13 17:38:03.463774 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.301 on 1 PGs 2014-08-13 17:38:03.463774 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.463786 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecf380 don't know epoch, doing nothing 2014-08-13 17:38:03.463788 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.302 on 1 PGs 2014-08-13 17:38:03.463799 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.463801 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.303 on 1 PGs 2014-08-13 17:38:03.463779 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.463805 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360] newacting [373,360] 2014-08-13 17:38:03.463815 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.463824 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.273469 3 0.000229 2014-08-13 17:38:03.463835 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.273517 0 0.000000 2014-08-13 17:38:03.463845 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.463853 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.463859 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecc8e0 don't know epoch, doing nothing 2014-08-13 17:38:03.463864 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.311 on 1 PGs 2014-08-13 17:38:03.463862 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.463870 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.463878 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec9600 don't know epoch, doing nothing 2014-08-13 17:38:03.463880 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.312 on 1 PGs 2014-08-13 17:38:03.463878 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360] newacting [373,360] 2014-08-13 17:38:03.463889 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.463895 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecd120 don't know epoch, doing nothing 2014-08-13 17:38:03.463897 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.314 on 1 PGs 2014-08-13 17:38:03.463897 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.463921 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:03.463914 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [373,360,4](373) acting [373,360,4](373)): not rw, up_thru 194476 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [373,360,4](373) acting [373,360,4](373) maybe_went_rw) : primary up 192978-194476 includes interval 2014-08-13 17:38:03.463923 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.324 on 3 PGs 2014-08-13 17:38:03.463924 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194476/194476/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [373,360,4](373) acting [373,360,4](373) maybe_went_rw) 2014-08-13 17:38:03.463936 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:03.463938 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.330 on 1 PGs 2014-08-13 17:38:03.463934 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360,4] -> [373,360], acting [373,360,4] -> [373,360], acting_primary 373 -> 373, up_primary 373 -> 373, role 2 -> -1 2014-08-13 17:38:03.463949 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing 2014-08-13 17:38:03.463950 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.331 on 1 PGs 2014-08-13 17:38:03.463944 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.463953 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.463964 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec8160 don't know epoch, doing nothing 2014-08-13 17:38:03.463965 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.339 on 1 PGs 2014-08-13 17:38:03.463982 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:03.463984 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.340 on 2 PGs 2014-08-13 17:38:03.463961 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.464036 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.464047 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec8580 don't know epoch, doing nothing 2014-08-13 17:38:03.464044 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.464052 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.342 on 1 PGs 2014-08-13 17:38:03.464053 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.464061 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.464071 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.464079 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.464086 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5380c60 don't know epoch, doing nothing 2014-08-13 17:38:03.464088 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.349 on 1 PGs 2014-08-13 17:38:03.464088 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.464101 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing 2014-08-13 17:38:03.464096 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.464103 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.350 on 1 PGs 2014-08-13 17:38:03.464104 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.464115 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:03.464117 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.362 on 1 PGs 2014-08-13 17:38:03.464111 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.464128 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecac00 don't know epoch, doing nothing 2014-08-13 17:38:03.464121 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.464131 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.365 on 1 PGs 2014-08-13 17:38:03.464142 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:03.464144 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.367 on 1 PGs 2014-08-13 17:38:03.464145 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 17:38:03.464156 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464158 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.464167 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.464175 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.464182 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5251080 don't know epoch, doing nothing 2014-08-13 17:38:03.464185 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.368 on 1 PGs 2014-08-13 17:38:03.464192 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 17:38:03.464208 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464210 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.464217 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464220 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464221 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5138000 pg_notify(22.619(1) epoch 194485) v5 2014-08-13 17:38:03.464218 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.464227 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.464245 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.44 2014-08-13 17:38:03.464247 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5138000 2014-08-13 17:38:03.464246 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 17:38:03.464257 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464262 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5250840 don't know epoch, doing nothing 2014-08-13 17:38:03.464260 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.464267 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.369 on 1 PGs 2014-08-13 17:38:03.464275 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464268 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.464278 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464283 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecee00 don't know epoch, doing nothing 2014-08-13 17:38:03.464285 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.370 on 2 PGs 2014-08-13 17:38:03.464279 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360,355] newacting [373,360,355] 2014-08-13 17:38:03.464288 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.464296 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:03.464299 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.373 on 1 PGs 2014-08-13 17:38:03.464296 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.464310 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing 2014-08-13 17:38:03.464312 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.378 on 1 PGs 2014-08-13 17:38:03.464313 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [373,360](373) acting [373,360](373)): not rw, up_thru 194480 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [373,360](373) acting [373,360](373) maybe_went_rw) : primary up 192978-194480 includes interval 2014-08-13 17:38:03.464322 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194479/194479/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [373,360](373) acting [373,360](373) maybe_went_rw) 2014-08-13 17:38:03.464331 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360] -> [373,360,355], acting [373,360] -> [373,360,355], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> -1 2014-08-13 17:38:03.464341 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.464349 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.464357 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.464365 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.464363 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [374,31]/[374,31] -- 374/374 2014-08-13 17:38:03.464373 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.464382 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464382 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.464385 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.464390 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.464395 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.464400 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.464405 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [374,31] newacting [374,31] 2014-08-13 17:38:03.464409 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.464416 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.464418 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.464425 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [4,374,31] r=0 lpr=194478 pi=193306-194476/23 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.464426 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.464435 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.464445 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,374,31](4) acting [4,374,31](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194465 generate_past_intervals interval(194477-194478 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.464460 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464454 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194476/194477/194477) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] noting past interval(194477-194478 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) 2014-08-13 17:38:03.464463 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464455 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 17:38:03.464464 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513de80 pg_notify(22.1d2 epoch 194485) v5 2014-08-13 17:38:03.464467 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464464 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] up [4,374,31] -> [374,31], acting [4,374,31] -> [374,31], acting_primary 4 -> 374, up_primary 4 -> 374, role 0 -> -1 2014-08-13 17:38:03.464470 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.464476 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.464482 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.464486 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.464494 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.464496 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513de80 2014-08-13 17:38:03.464493 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.464495 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.464505 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.464513 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.464524 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464526 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464523 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.464532 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.464538 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464539 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464541 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265240 pg_notify(22.113 epoch 194485) v5 2014-08-13 17:38:03.464540 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.464548 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.464562 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.45 2014-08-13 17:38:03.464564 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265240 2014-08-13 17:38:03.464558 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.464567 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.464575 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.464584 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464586 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464583 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.464596 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.464590 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.464597 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.464599 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e740 pg_notify(22.619 epoch 194485) v5 2014-08-13 17:38:03.464512 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 17:38:03.464598 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.464605 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.464606 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.465140 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465154 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.465165 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.65 2014-08-13 17:38:03.465171 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e740 2014-08-13 17:38:03.465164 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465153 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.465177 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465197 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465198 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465185 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 17:38:03.465202 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465205 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465213 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465215 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465216 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d91440 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.465209 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [374,31]/[374,31,20] -- 374/374 2014-08-13 17:38:03.465214 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.465223 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465069 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.465222 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.465230 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.465223 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360,4] newacting [373,360,4] 2014-08-13 17:38:03.465232 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465234 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465226 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465233 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.465238 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.465247 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465249 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465250 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8700 pg_notify(22.22c(1) epoch 194485) v5 2014-08-13 17:38:03.465243 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,355] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.465248 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] acting_up_affected newup [374,31] newacting [374,31,20] 2014-08-13 17:38:03.465258 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.465263 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [373,360,355](373) acting [373,360,355](373)): not rw, up_thru 194482 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [373,360,355](373) acting [373,360,355](373) maybe_went_rw) : primary up 192978-194482 includes interval 2014-08-13 17:38:03.465274 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.302 2014-08-13 17:38:03.465268 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31] r=-1 lpr=194479 pi=193306-194478/24 crt=191648'365 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.465277 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8700 2014-08-13 17:38:03.465273 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194482/194482/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [373,360,355](373) acting [373,360,355](373) maybe_went_rw) 2014-08-13 17:38:03.465285 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360,355] -> [373,360,4], acting [373,360,355] -> [373,360,4], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> 2 2014-08-13 17:38:03.465288 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [374,31](374) acting [374,31](374)): not rw, up_thru 194476 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194479-194479 up [374,31](374) acting [374,31](374)) : primary up 192967-194476 does not include interval 2014-08-13 17:38:03.465299 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465301 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465296 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.465298 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194479/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [374,31](374) acting [374,31](374)) 2014-08-13 17:38:03.465305 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.465313 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465315 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465316 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd780 pg_notify(1.2b(2),2.2a(2) epoch 194485) v5 2014-08-13 17:38:03.465310 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] up [374,31] -> [374,31], acting [374,31] -> [374,31,20], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 17:38:03.465315 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.465322 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.465325 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.465334 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.465333 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.465343 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.370 2014-08-13 17:38:03.465344 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd780 2014-08-13 17:38:03.465343 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.465343 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.465354 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.465355 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.465363 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.465365 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.465374 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465376 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465373 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.465374 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.465382 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.465391 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465392 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465384 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.465395 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102680 pg_notify(22.1d2 epoch 194485) v5 2014-08-13 17:38:03.465390 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.465397 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.465399 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.465407 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.465408 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.465417 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.349 2014-08-13 17:38:03.465419 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102680 2014-08-13 17:38:03.465418 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465418 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.465429 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.465429 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.465441 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465442 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465439 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465453 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465454 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465455 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ca4c0 pg_notify(24.9b(33) epoch 194485) v5 2014-08-13 17:38:03.465459 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] handle_advance_map [374,31]/[374,31,20] -- 374/374 2014-08-13 17:38:03.465471 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.465478 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.350 2014-08-13 17:38:03.465480 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ca4c0 2014-08-13 17:38:03.465474 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465477 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.465484 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.465488 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.001642 7 0.000883 2014-08-13 17:38:03.465502 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465503 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465495 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465500 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.465509 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.465517 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.465524 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465527 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465529 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ccec0 pg_notify(1.2b,2.2a epoch 194485) v5 2014-08-13 17:38:03.465520 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] handle_advance_map [363,20,337]/[374,31,20] -- 363/374 2014-08-13 17:38:03.465534 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465529 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.465536 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465541 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.465553 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.340 2014-08-13 17:38:03.465546 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.465555 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ccec0 2014-08-13 17:38:03.465554 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.465556 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] acting_up_affected newup [363,20,337] newacting [374,31,20] 2014-08-13 17:38:03.465564 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.465566 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.465584 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465576 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [374,31]/[374,31,20] r=-1 lpr=194480 pi=193306-194479/25 crt=191648'365 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.465586 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465599 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465601 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465602 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102300 pg_notify(22.22c epoch 194485) v5 2014-08-13 17:38:03.465596 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [374,31](374) acting [374,31,20](374)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194480-194481 up [374,31](374) acting [374,31,20](374) maybe_went_rw) : primary up 192967-194480 includes interval 2014-08-13 17:38:03.465613 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.465605 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194479/194480/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [374,31](374) acting [374,31,20](374) maybe_went_rw) 2014-08-13 17:38:03.465622 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.301 2014-08-13 17:38:03.465624 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102300 2014-08-13 17:38:03.465616 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] up [374,31] -> [363,20,337], acting [374,31,20] -> [374,31,20], acting_primary 374 -> 374, up_primary 374 -> 363, role -1 -> -1 2014-08-13 17:38:03.465628 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.465636 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.465644 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465645 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465645 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.465652 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.465657 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465659 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465658 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d7b020 don't know epoch, doing nothing 2014-08-13 17:38:03.465660 7fe99f5e9700 7 osd.4 194485 do_notify osd 19 on 1 PGs 2014-08-13 17:38:03.465660 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d99440 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.465654 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.465665 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.465668 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.465669 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465670 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465664 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.465677 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d7c8e0 don't know epoch, doing nothing 2014-08-13 17:38:03.465679 7fe99f5e9700 7 osd.4 194485 do_notify osd 358 on 1 PGs 2014-08-13 17:38:03.465674 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.465684 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465686 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465687 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8380 pg_notify(24.9b(33) epoch 194485) v5 2014-08-13 17:38:03.465690 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:03.465684 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.465692 7fe99f5e9700 7 osd.4 194485 do_notify osd 373 on 1 PGs 2014-08-13 17:38:03.465694 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.465705 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:03.465707 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.17 on 1 PGs 2014-08-13 17:38:03.465712 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.291 2014-08-13 17:38:03.465713 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8380 2014-08-13 17:38:03.465706 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.465722 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:03.465715 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.465724 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.28 on 1 PGs 2014-08-13 17:38:03.465735 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465736 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465736 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecce60 don't know epoch, doing nothing 2014-08-13 17:38:03.465738 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.315 on 1 PGs 2014-08-13 17:38:03.465724 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.465751 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465752 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465753 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e82520 don't know epoch, doing nothing 2014-08-13 17:38:03.465753 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cabc0 pg_notify(22.f3 epoch 194485) v5 2014-08-13 17:38:03.465755 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.327 on 1 PGs 2014-08-13 17:38:03.465767 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:03.465768 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.330 on 1 PGs 2014-08-13 17:38:03.465774 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.285 2014-08-13 17:38:03.465776 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cabc0 2014-08-13 17:38:03.465817 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465818 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465815 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 17:38:03.465830 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465831 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465832 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16580 pg_notify(24.3d(1) epoch 194485) v5 2014-08-13 17:38:03.465834 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465838 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Started advmap 2014-08-13 17:38:03.465746 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.465848 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364] newacting [356,364] 2014-08-13 17:38:03.465858 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.20 2014-08-13 17:38:03.465860 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16580 2014-08-13 17:38:03.465868 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 17:38:03.465881 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465881 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.465883 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465859 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.465884 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.465894 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465896 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465897 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb480 pg_notify(22.113(29) epoch 194485) v5 2014-08-13 17:38:03.465888 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 0.275021 3 0.000127 2014-08-13 17:38:03.465894 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.465901 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started 0.275075 0 0.000000 2014-08-13 17:38:03.465905 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] acting_up_affected newup [363,20,337] newacting [363,20,337] 2014-08-13 17:38:03.465914 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset 2014-08-13 17:38:03.465922 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.55 2014-08-13 17:38:03.465923 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb480 2014-08-13 17:38:03.465916 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.465923 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194477 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194479 2014-08-13 17:38:03.465927 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337]/[374,31,20] r=-1 lpr=194482 pi=193306-194481/26 crt=191648'365 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.465934 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194479 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.465945 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465946 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465943 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194479 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.465949 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194465 generate_past_intervals interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) : primary up 192967-194480 does not include interval 2014-08-13 17:38:03.465953 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194479 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364] newacting [356,364] 2014-08-13 17:38:03.465965 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.465967 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.465959 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194482/194479) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) 2014-08-13 17:38:03.465968 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102a00 pg_notify(24.3d epoch 194485) v5 2014-08-13 17:38:03.465963 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194479 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.465972 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364,4] r=2 lpr=194479 pi=193151-194475/26 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194479 2014-08-13 17:38:03.465970 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] up [363,20,337] -> [363,20,337], acting [374,31,20] -> [363,20,337], acting_primary 374 -> 363, up_primary 363 -> 363, role -1 -> -1 2014-08-13 17:38:03.465985 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.465994 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.362 2014-08-13 17:38:03.465997 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102a00 2014-08-13 17:38:03.465994 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.465993 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [356,364,4](356) acting [356,364,4](356)): not rw, up_thru 194477 up_from 192969 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [356,364,4](356) acting [356,364,4](356) maybe_went_rw) : primary up 192969-194477 includes interval 2014-08-13 17:38:03.466004 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.466005 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194476-194478 up [356,364,4](356) acting [356,364,4](356) maybe_went_rw) 2014-08-13 17:38:03.466016 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466018 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466014 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.466016 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364,4] -> [356,364], acting [356,364,4] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role 2 -> -1 2014-08-13 17:38:03.466029 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466031 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466023 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.466032 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c96c0 pg_notify(22.113(29) epoch 194485) v5 2014-08-13 17:38:03.466027 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.466033 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.466038 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.466043 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.466048 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.466056 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.24 2014-08-13 17:38:03.466058 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c96c0 2014-08-13 17:38:03.466053 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.466058 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.466078 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466069 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.466080 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466064 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.466081 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.466094 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466096 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466090 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.466097 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513db00 pg_notify(1.2b(1),2.2a(1),22.22c(27) epoch 194485) v5 2014-08-13 17:38:03.466090 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] [374,31,20] -> [363,20,337], acting primary 374 -> 363 2014-08-13 17:38:03.466102 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.466107 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.466112 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.466116 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.466126 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.324 2014-08-13 17:38:03.466127 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513db00 2014-08-13 17:38:03.466122 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_role_change 2014-08-13 17:38:03.466127 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.466131 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.466140 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.466148 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.466155 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 17:38:03.466158 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.466167 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.466167 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466169 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466170 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.466185 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466186 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466188 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8e00 pg_notify(24.3d(31) epoch 194485) v5 2014-08-13 17:38:03.466179 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 17:38:03.466178 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.466196 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.466206 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.466211 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.466212 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8e00 2014-08-13 17:38:03.466209 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.466219 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.466219 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 17:38:03.466232 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.466236 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466230 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.466238 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.466243 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.466254 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466255 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466257 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4263d40 pg_notify(24.9b epoch 194485) v5 2014-08-13 17:38:03.466248 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 17:38:03.466252 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,374,31] newacting [4,374,31] 2014-08-13 17:38:03.466262 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.466261 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.466265 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.466270 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [363,20,337] r=-1 lpr=194483 pi=193306-194482/27 crt=191648'365 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.466279 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.367 2014-08-13 17:38:03.466281 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4263d40 2014-08-13 17:38:03.466275 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.466285 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.466290 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [363,20,337](363) acting [363,20,337](363)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) : primary up 192979-194483 includes interval 2014-08-13 17:38:03.466300 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466301 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466300 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194482/194483/194483) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 17:38:03.466312 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466313 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466315 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8540 pg_notify(22.1d2(1) epoch 194485) v5 2014-08-13 17:38:03.466306 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,381]/[356,364,381] -- 356/356 2014-08-13 17:38:03.466320 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.466311 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [363,20,337] -> [4,374,31], acting [363,20,337] -> [4,374,31], acting_primary 363 -> 4, up_primary 363 -> 4, role -1 -> 0 2014-08-13 17:38:03.466323 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.466323 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.466339 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.331 2014-08-13 17:38:03.466340 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8540 2014-08-13 17:38:03.466334 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.466334 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.466344 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364,381] newacting [356,364,381] 2014-08-13 17:38:03.466344 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.466362 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466355 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.466363 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466358 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2868 2014-08-13 17:38:03.466364 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364] r=-1 lpr=194479 pi=193151-194478/27 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194482 2014-08-13 17:38:03.466369 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.466376 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466378 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466379 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd400 pg_notify(24.3d(31) epoch 194485) v5 2014-08-13 17:38:03.466379 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.466383 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [356,364](356) acting [356,364](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [356,364](356) acting [356,364](356) maybe_went_rw) : primary up 192969-194480 includes interval 2014-08-13 17:38:03.466388 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.466394 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194479-194481 up [356,364](356) acting [356,364](356) maybe_went_rw) 2014-08-13 17:38:03.466398 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.466403 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.373 2014-08-13 17:38:03.466409 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd400 2014-08-13 17:38:03.466406 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.466406 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364] -> [356,364,381], acting [356,364] -> [356,364,381], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.466416 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.466418 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.466430 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466431 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466425 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.466428 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.466435 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.466437 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.466448 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466450 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466451 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c81c0 pg_notify(22.113 epoch 194485) v5 2014-08-13 17:38:03.466444 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.466446 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.466454 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.466455 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.466463 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.466464 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.466474 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.339 2014-08-13 17:38:03.466476 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c81c0 2014-08-13 17:38:03.466474 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.466473 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.466484 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.466485 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.466496 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466498 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466493 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.466495 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.466508 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.466501 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.466510 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.466511 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264d00 pg_notify(22.113(1) epoch 194485) v5 2014-08-13 17:38:03.466505 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.471995 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.342 2014-08-13 17:38:03.471998 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264d00 2014-08-13 17:38:03.471996 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.466511 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.472020 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472032 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472034 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472031 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472053 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472055 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472056 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16200 pg_notify(22.1d2(27) epoch 194485) v5 2014-08-13 17:38:03.472048 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,381]/[356,364] -- 356/356 2014-08-13 17:38:03.472055 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.472066 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472068 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,31,374 2014-08-13 17:38:03.472079 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.472071 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.472080 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.330 2014-08-13 17:38:03.472082 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16200 2014-08-13 17:38:03.472081 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.472081 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.472092 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364,381] newacting [356,364] 2014-08-13 17:38:03.472092 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.863637 9 0.007544 2014-08-13 17:38:03.472107 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472110 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472103 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.472107 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.472113 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381] r=-1 lpr=194482 pi=193151-194481/28 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194483 2014-08-13 17:38:03.472117 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.472130 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472132 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472126 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.472133 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9500 pg_notify(22.f3(1) epoch 194485) v5 2014-08-13 17:38:03.472133 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [356,364,381](356) acting [356,364,381](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [356,364,381](356) acting [356,364,381](356)) : primary up 192969-194480 does not include interval 2014-08-13 17:38:03.472138 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.472145 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] noting past interval(194482-194482 up [356,364,381](356) acting [356,364,381](356)) 2014-08-13 17:38:03.472158 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.312 2014-08-13 17:38:03.472157 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] up [356,364,381] -> [356,364,381], acting [356,364,381] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.472151 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.472169 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9500 2014-08-13 17:38:03.472170 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.472173 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.472180 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.472193 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472195 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472190 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.472190 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.472206 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472208 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472201 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.472209 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc0c0 pg_notify(24.9b(1) epoch 194485) v5 2014-08-13 17:38:03.472209 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.472212 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 17:38:03.472219 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.472234 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.365 2014-08-13 17:38:03.472236 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc0c0 2014-08-13 17:38:03.472226 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) 2014-08-13 17:38:03.472229 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.472237 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [374,31](374) acting [374,31,20](374) maybe_went_rw) 2014-08-13 17:38:03.472239 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.472257 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472258 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472250 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.472249 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [374,31](374) acting [374,31](374)) 2014-08-13 17:38:03.472271 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472262 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.472272 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472262 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) 2014-08-13 17:38:03.472273 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5460480 osd_map(194478..194485 src has 185139..194485) v3 2014-08-13 17:38:03.472279 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194478,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.472273 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.472282 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.472284 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472286 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472275 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,374,31](4) acting [374,31,20](374)) 2014-08-13 17:38:03.472283 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.472287 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [374,31](374) acting [374,31,20](374) maybe_went_rw) 2014-08-13 17:38:03.472297 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472298 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472300 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce3c0 pg_notify(22.619(31) epoch 194485) v5 2014-08-13 17:38:03.472294 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472298 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [374,31](374) acting [374,31](374)) 2014-08-13 17:38:03.472308 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,20,31,337,363,374 down blocked_by {} 2014-08-13 17:38:03.472325 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.314 2014-08-13 17:38:03.472327 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce3c0 2014-08-13 17:38:03.472317 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,381]/[356,364] -- 356/356 2014-08-13 17:38:03.472319 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.472333 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472335 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2869 2014-08-13 17:38:03.472336 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.472348 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472350 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472346 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.20 2014-08-13 17:38:03.472346 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.472358 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472359 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.31 2014-08-13 17:38:03.472368 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472369 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472371 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cbb80 pg_notify(24.b6(37) epoch 194485) v5 2014-08-13 17:38:03.472369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.337 2014-08-13 17:38:03.472376 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,4]/[356,364] -- 356/356 2014-08-13 17:38:03.472378 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 17:38:03.472390 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472393 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.315 2014-08-13 17:38:03.472395 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cbb80 2014-08-13 17:38:03.472390 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.374 2014-08-13 17:38:03.472393 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.472401 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.472404 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.472416 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472418 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472411 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.472414 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364] 2014-08-13 17:38:03.472424 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.472433 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472435 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472436 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e900 pg_notify(24.3d epoch 194485) v5 2014-08-13 17:38:03.472434 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,381]/[356,364] r=-1 lpr=194483 pi=193151-194482/29 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194485 2014-08-13 17:38:03.472456 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.25 2014-08-13 17:38:03.472458 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e900 2014-08-13 17:38:03.472452 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [356,364,381](356) acting [356,364](356)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [356,364,381](356) acting [356,364](356) maybe_went_rw) : primary up 192969-194483 includes interval 2014-08-13 17:38:03.472472 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.472466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] noting past interval(194483-194484 up [356,364,381](356) acting [356,364](356) maybe_went_rw) 2014-08-13 17:38:03.472477 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472479 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472477 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] up [356,364,381] -> [356,364,4], acting [356,364] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.472489 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472491 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472492 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8fc0 pg_notify(22.f3(23) epoch 194485) v5 2014-08-13 17:38:03.472492 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.472488 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.472497 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c71b80 don't know epoch, doing nothing 2014-08-13 17:38:03.472499 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.20 on 1 PGs 2014-08-13 17:38:03.472498 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.472514 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.311 2014-08-13 17:38:03.472515 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8fc0 2014-08-13 17:38:03.472508 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.472518 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:03.472520 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.31 on 1 PGs 2014-08-13 17:38:03.472518 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.472533 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:03.472528 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.472535 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.337 on 1 PGs 2014-08-13 17:38:03.472535 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472537 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472537 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.472548 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4a680 don't know epoch, doing nothing 2014-08-13 17:38:03.472550 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472550 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.363 on 1 PGs 2014-08-13 17:38:03.472552 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472553 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ce040 pg_notify(22.619(31) epoch 194485) v5 2014-08-13 17:38:03.472547 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.472562 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:03.472564 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.374 on 1 PGs 2014-08-13 17:38:03.472557 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.472568 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.472577 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.48 2014-08-13 17:38:03.472579 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ce040 2014-08-13 17:38:03.472578 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.472587 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.472597 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472607 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_activate_map 2014-08-13 17:38:03.472635 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472623 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 17:38:03.472638 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472643 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472648 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.472662 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472663 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472655 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] update_heartbeat_peers unchanged 2014-08-13 17:38:03.472664 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb2c0 pg_notify(22.22c(25) epoch 194485) v5 2014-08-13 17:38:03.472659 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [396,368] newacting [396,363] 2014-08-13 17:38:03.472667 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] take_waiters 2014-08-13 17:38:03.472669 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.472677 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Reset 0.006764 7 0.006580 2014-08-13 17:38:03.472688 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.368 2014-08-13 17:38:03.472680 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.278629 3 0.000281 2014-08-13 17:38:03.472690 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb2c0 2014-08-13 17:38:03.472690 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started 2014-08-13 17:38:03.472693 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.278831 0 0.000000 2014-08-13 17:38:03.472701 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Start 2014-08-13 17:38:03.472705 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.472711 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: transitioning to Stray 2014-08-13 17:38:03.472720 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472722 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472715 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194477 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.472723 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Start 0.000022 0 0.000000 2014-08-13 17:38:03.472726 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.472740 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472742 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472735 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started/Stray 2014-08-13 17:38:03.472743 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143b80 pg_notify(22.f3(23) epoch 194485) v5 2014-08-13 17:38:03.472736 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.472747 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [396,368] newacting [396,363] 2014-08-13 17:38:03.472749 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.472757 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.472759 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.472768 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.472770 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143b80 2014-08-13 17:38:03.472768 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368,4]/[396,363] r=-1 lpr=194479 pi=194462-194475/5 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.472794 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472795 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472787 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [396,368,4](396) acting [396,363](396)): not rw, up_thru 194476 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194476-194478 up [396,368,4](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194476 includes interval 2014-08-13 17:38:03.472798 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194476/194476/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194476-194478 up [396,368,4](396) acting [396,363](396) maybe_went_rw) 2014-08-13 17:38:03.472811 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472812 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472813 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51440c0 pg_notify(22.1d2(27) epoch 194485) v5 2014-08-13 17:38:03.472815 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.472808 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368,4] -> [396,368], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 17:38:03.472821 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.472836 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.58 2014-08-13 17:38:03.472830 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.472838 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51440c0 2014-08-13 17:38:03.472839 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.472851 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.472858 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472845 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_advance_map [59,351]/[351,55] -- 59/351 2014-08-13 17:38:03.472860 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472865 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472860 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.472875 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472876 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472869 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Reset advmap 2014-08-13 17:38:03.472877 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16c80 pg_notify(24.b6 epoch 194485) v5 2014-08-13 17:38:03.472870 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.472879 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.472880 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.472892 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.472890 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [59,351] newacting [351,55] 2014-08-13 17:38:03.472902 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.28 2014-08-13 17:38:03.472904 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16c80 2014-08-13 17:38:03.472903 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.472902 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.472914 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [4,59,351]/[4,351,55] r=0 lpr=194478 pi=193678-194476/29 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194479 2014-08-13 17:38:03.472915 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.472924 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472925 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472925 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.472935 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472936 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472938 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb640 pg_notify(22.619 epoch 194485) v5 2014-08-13 17:38:03.472936 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.472936 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,59,351](4) acting [4,351,55](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.472948 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194476/194477/194477) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] noting past interval(194477-194478 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) 2014-08-13 17:38:03.472961 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.62 2014-08-13 17:38:03.472962 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb640 2014-08-13 17:38:03.472959 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] up [4,59,351] -> [59,351], acting [4,351,55] -> [351,55], acting_primary 4 -> 351, up_primary 4 -> 59, role 0 -> -1 2014-08-13 17:38:03.472962 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 17:38:03.472971 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] clear_primary_state 2014-08-13 17:38:03.472978 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.472982 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472984 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472981 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] agent_stop 2014-08-13 17:38:03.472981 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.472995 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.472996 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.472997 7fe9a63f4700 20 osd.4 194485 _dispatch 0x42601c0 pg_notify(24.b6(37) epoch 194485) v5 2014-08-13 17:38:03.472992 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] on_change 2014-08-13 17:38:03.472992 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473003 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473004 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473019 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.17 2014-08-13 17:38:03.473012 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] requeue_ops 2014-08-13 17:38:03.473021 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x42601c0 2014-08-13 17:38:03.473023 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] cancel_copy_ops 2014-08-13 17:38:03.473025 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 17:38:03.473039 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473033 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] cancel_flush_ops 2014-08-13 17:38:03.473041 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473043 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473042 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] on_change_cleanup 2014-08-13 17:38:03.473042 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.473058 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473060 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473053 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] on_change 2014-08-13 17:38:03.473061 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8c40 pg_notify(24.b6(1) epoch 194485) v5 2014-08-13 17:38:03.473058 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473064 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] exit NotTrimming 2014-08-13 17:38:03.473069 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473074 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped m=62] enter NotTrimming 2014-08-13 17:38:03.473084 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.327 2014-08-13 17:38:03.473086 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8c40 2014-08-13 17:38:03.473084 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_stats 2014-08-13 17:38:03.473093 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.473093 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 17:38:03.473108 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473109 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473110 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473103 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_role_change 2014-08-13 17:38:03.473110 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.473112 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.473129 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473120 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473130 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473121 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 17:38:03.473133 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb100 pg_notify(24.b6 epoch 194485) v5 2014-08-13 17:38:03.473133 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [396,368,284] newacting [396,363] 2014-08-13 17:38:03.473134 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 17:38:03.473143 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.473145 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473158 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.330 2014-08-13 17:38:03.473159 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb100 2014-08-13 17:38:03.473153 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368]/[396,363] r=-1 lpr=194479 pi=194462-194478/6 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.473165 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [59,351]/[351,55,376] -- 59/351 2014-08-13 17:38:03.473178 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473170 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [396,368](396) acting [396,363](396)): not rw, up_thru 194479 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194479-194481 up [396,368](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194479 includes interval 2014-08-13 17:38:03.473179 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473180 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473195 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473180 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194479/194479/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194479-194481 up [396,368](396) acting [396,363](396) maybe_went_rw) 2014-08-13 17:38:03.473197 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473184 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.473198 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c9f80 pg_notify(24.9b epoch 194485) v5 2014-08-13 17:38:03.473200 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.473198 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368] -> [396,368,284], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 17:38:03.473211 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [59,351] newacting [351,55,376] 2014-08-13 17:38:03.473212 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.473228 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.378 2014-08-13 17:38:03.473230 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c9f80 2014-08-13 17:38:03.473223 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.473224 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473233 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55] r=-1 lpr=194479 pi=193678-194478/30 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194480 2014-08-13 17:38:03.473234 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.473244 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.473254 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473256 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473252 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [59,351](59) acting [351,55](351)): not rw, up_thru 194477 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [59,351](59) acting [351,55](351)) : primary up 192967-194477 does not include interval 2014-08-13 17:38:03.473254 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.473271 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473262 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194479/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] noting past interval(194479-194479 up [59,351](59) acting [351,55](351)) 2014-08-13 17:38:03.473273 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473264 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.473274 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51439c0 pg_notify(22.99 epoch 194485) v5 2014-08-13 17:38:03.473275 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.473274 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] up [59,351] -> [59,351], acting [351,55] -> [351,55,376], acting_primary 351 -> 351, up_primary 59 -> 59, role -1 -> -1 2014-08-13 17:38:03.473286 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.473301 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.31 2014-08-13 17:38:03.473286 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 17:38:03.473303 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51439c0 2014-08-13 17:38:03.473298 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.473305 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473308 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.473323 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473315 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.473324 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473317 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.473326 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_copy_ops 2014-08-13 17:38:03.473328 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473335 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_flush_ops 2014-08-13 17:38:03.473344 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change_cleanup 2014-08-13 17:38:03.473353 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 17:38:03.473352 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 17:38:03.473369 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473363 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit NotTrimming 2014-08-13 17:38:03.473372 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter NotTrimming 2014-08-13 17:38:03.473372 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.473382 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 17:38:03.473383 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473394 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473396 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473397 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143640 pg_notify(22.99 epoch 194485) v5 2014-08-13 17:38:03.473392 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 17:38:03.473393 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473403 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473418 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.374 2014-08-13 17:38:03.473420 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143640 2014-08-13 17:38:03.473418 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 17:38:03.473432 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473424 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [59,351]/[351,55,376] -- 59/351 2014-08-13 17:38:03.473439 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473441 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473441 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473435 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.473444 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.473445 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473454 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.473455 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473464 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473476 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 17:38:03.473488 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473497 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473487 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [55,351,376]/[351,55,376] -- 55/351 2014-08-13 17:38:03.473498 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473491 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.473500 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513de80 pg_notify(22.99(27) epoch 194485) v5 2014-08-13 17:38:03.473501 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473501 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 17:38:03.473504 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.473511 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [396,368,4] newacting [396,363] 2014-08-13 17:38:03.473523 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.20 2014-08-13 17:38:03.473514 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.473526 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513de80 2014-08-13 17:38:03.473522 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.473527 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [55,351,376] newacting [351,55,376] 2014-08-13 17:38:03.473532 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,284]/[396,363] r=-1 lpr=194482 pi=194462-194481/7 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.473538 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.473548 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473550 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473548 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [59,351]/[351,55,376] r=-1 lpr=194480 pi=193678-194479/31 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194482 2014-08-13 17:38:03.473551 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [396,368,284](396) acting [396,363](396)): not rw, up_thru 194482 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194482-194484 up [396,368,284](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194482 includes interval 2014-08-13 17:38:03.473561 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194482/194482/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194482-194484 up [396,368,284](396) acting [396,363](396) maybe_went_rw) 2014-08-13 17:38:03.473566 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [59,351](59) acting [351,55,376](351)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [59,351](59) acting [351,55,376](351) maybe_went_rw) : primary up 192967-194480 includes interval 2014-08-13 17:38:03.473572 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368,284] -> [396,368,4], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 17:38:03.473576 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194479/194480/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] noting past interval(194480-194481 up [59,351](59) acting [351,55,376](351) maybe_went_rw) 2014-08-13 17:38:03.473584 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.473587 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] up [59,351] -> [55,351,376], acting [351,55,376] -> [351,55,376], acting_primary 351 -> 351, up_primary 59 -> 55, role -1 -> -1 2014-08-13 17:38:03.473599 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473602 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473593 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473603 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265240 pg_notify(22.99(1) epoch 194485) v5 2014-08-13 17:38:03.473599 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 17:38:03.473604 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.473611 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473614 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.473627 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.363 2014-08-13 17:38:03.473629 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265240 2014-08-13 17:38:03.473621 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.473625 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.473635 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.473632 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_copy_ops 2014-08-13 17:38:03.473650 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473651 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473645 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.473645 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_flush_ops 2014-08-13 17:38:03.473662 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473664 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473665 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4265b00 pg_notify(22.99(27) epoch 194485) v5 2014-08-13 17:38:03.473657 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change_cleanup 2014-08-13 17:38:03.473657 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.473668 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 17:38:03.473669 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.473688 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.337 2014-08-13 17:38:03.473680 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.473680 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit NotTrimming 2014-08-13 17:38:03.473690 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4265b00 2014-08-13 17:38:03.473691 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.473692 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter NotTrimming 2014-08-13 17:38:03.473702 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473702 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 17:38:03.473712 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.473714 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.473713 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 17:38:03.473713 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 17:38:03.473724 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473736 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.473746 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 17:38:03.473746 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [55,351,376]/[55,351,376] -- 55/55 2014-08-13 17:38:03.473761 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473756 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.001051 7 0.000884 2014-08-13 17:38:03.473764 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.473768 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started 2014-08-13 17:38:03.473774 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.473777 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start 2014-08-13 17:38:03.473787 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.473786 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [55,351,376] newacting [55,351,376] 2014-08-13 17:38:03.473797 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.473798 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.473806 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376]/[351,55,376] r=-1 lpr=194482 pi=193678-194481/32 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194483 2014-08-13 17:38:03.473810 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray 2014-08-13 17:38:03.473823 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.473824 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) : primary up 192967-194480 does not include interval 2014-08-13 17:38:03.473835 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.473835 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194482/194479) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] noting past interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 17:38:03.473847 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] up [55,351,376] -> [55,351,376], acting [351,55,376] -> [55,351,376], acting_primary 351 -> 55, up_primary 55 -> 55, role -1 -> -1 2014-08-13 17:38:03.473857 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 17:38:03.473866 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.473874 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.473874 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.473883 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_copy_ops 2014-08-13 17:38:03.473891 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_flush_ops 2014-08-13 17:38:03.473898 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change_cleanup 2014-08-13 17:38:03.473907 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 17:38:03.473903 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 17:38:03.473918 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.473916 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] exit NotTrimming 2014-08-13 17:38:03.473923 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.473925 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] enter NotTrimming 2014-08-13 17:38:03.473932 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.473934 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] [351,55,376] -> [55,351,376], acting primary 351 -> 55 2014-08-13 17:38:03.473941 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.473950 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_recovery 2014-08-13 17:38:03.473953 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.281955 2 0.000430 2014-08-13 17:38:03.473959 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] clear_recovery_state 2014-08-13 17:38:03.473965 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.473969 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.473974 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.281987 0 0.000000 2014-08-13 17:38:03.473986 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.282010 0 0.000000 2014-08-13 17:38:03.473992 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] handle_advance_map [55,351,376]/[55,351,376] -- 55/55 2014-08-13 17:38:03.473996 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started 0.282308 0 0.000000 2014-08-13 17:38:03.474006 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474006 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.474009 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.474014 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194477 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.474018 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.474023 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194479 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.474028 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474034 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194479 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.474043 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194479 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [343,286] newacting [343,286] 2014-08-13 17:38:03.474046 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] handle_advance_map [4,59,351]/[4,59,351] -- 4/4 2014-08-13 17:38:03.474051 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194479 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.474059 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474060 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [4,343,286] r=0 lpr=194479 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.474062 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: Reset advmap 2014-08-13 17:38:03.474071 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.474077 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,343,286](4) acting [4,343,286](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.474079 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] acting_up_affected newup [4,59,351] newacting [4,59,351] 2014-08-13 17:38:03.474087 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194476/194476/194476) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] noting past interval(194476-194478 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 17:38:03.474089 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.474098 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [55,351,376] r=-1 lpr=194483 pi=193678-194482/33 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] set_last_peering_reset 194485 2014-08-13 17:38:03.474096 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] up [4,343,286] -> [343,286], acting [4,343,286] -> [343,286], acting_primary 4 -> 343, up_primary 4 -> 343, role 0 -> -1 2014-08-13 17:38:03.474107 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.474115 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.474115 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [55,351,376](55) acting [55,351,376](55)): not rw, up_thru 194483 up_from 192973 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) : primary up 192973-194483 includes interval 2014-08-13 17:38:03.474124 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.474127 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194482/194483/194483) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] noting past interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 17:38:03.474132 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.474138 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] up [55,351,376] -> [4,59,351], acting [55,351,376] -> [4,59,351], acting_primary 55 -> 4, up_primary 55 -> 4, role -1 -> 0 2014-08-13 17:38:03.474145 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.474150 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] clear_primary_state 2014-08-13 17:38:03.474155 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.474160 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] agent_stop 2014-08-13 17:38:03.474163 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.474170 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 17:38:03.474171 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.474180 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] publish_stats_to_osd 194485:25633 2014-08-13 17:38:03.474180 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.474195 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.474190 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.474204 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.474206 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] cancel_copy_ops 2014-08-13 17:38:03.474212 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.474215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] cancel_flush_ops 2014-08-13 17:38:03.474221 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474224 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.474230 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.474233 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.474238 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474242 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 17:38:03.474246 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.474251 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change_cleanup 2014-08-13 17:38:03.474254 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.474261 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 17:38:03.474264 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474271 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] exit NotTrimming 2014-08-13 17:38:03.474280 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] enter NotTrimming 2014-08-13 17:38:03.474286 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 17:38:03.474288 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] clear_stats 2014-08-13 17:38:03.474297 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474297 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] requeue_ops 2014-08-13 17:38:03.474300 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474306 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] on_role_change 2014-08-13 17:38:03.474308 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474315 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] requeue_ops 2014-08-13 17:38:03.474317 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474324 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] cancel_recovery 2014-08-13 17:38:03.474332 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] clear_recovery_state 2014-08-13 17:38:03.474337 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 17:38:03.474342 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474348 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474352 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_activate_map 2014-08-13 17:38:03.474353 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474362 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474363 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] update_heartbeat_peers -> 4,59,351 2014-08-13 17:38:03.474373 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.474370 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474375 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] take_waiters 2014-08-13 17:38:03.474384 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Reset 0.521167 9 0.001343 2014-08-13 17:38:03.474391 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 17:38:03.474402 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474395 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started 2014-08-13 17:38:03.474404 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474404 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Start 2014-08-13 17:38:03.474413 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474414 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: transitioning to Primary 2014-08-13 17:38:03.474421 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] acting_up_affected newup [10,343,286] newacting [10,343,286] 2014-08-13 17:38:03.474425 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.474430 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.474436 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary 2014-08-13 17:38:03.474440 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [343,286] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'426 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.474447 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering 2014-08-13 17:38:03.474455 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [343,286](343) acting [343,286](343)): not rw, up_thru 194479 up_from 192942 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) : primary up 192942-194479 includes interval 2014-08-13 17:38:03.474457 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.474464 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194479/194479/194479) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 17:38:03.474466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.474474 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] up [343,286] -> [10,343,286], acting [343,286] -> [10,343,286], acting_primary 343 -> 10, up_primary 343 -> 10, role -1 -> -1 2014-08-13 17:38:03.474477 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 17:38:03.474485 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.474489 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 17:38:03.474493 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.474501 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474500 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194480-194481 up [59,351](59) acting [351,55,376](351) maybe_went_rw) 2014-08-13 17:38:03.474510 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.474512 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194479-194479 up [59,351](59) acting [351,55](351)) 2014-08-13 17:38:03.474518 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.474523 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194477-194478 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) 2014-08-13 17:38:03.474527 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.474536 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.474534 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194476-194476 up [4,59,351](4) acting [351,55,376](351)) 2014-08-13 17:38:03.474545 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.474545 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194474-194475 up [59,351](59) acting [351,55,376](351) maybe_went_rw) 2014-08-13 17:38:03.474554 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.474556 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194473-194473 up [59,351](59) acting [351,55](351)) 2014-08-13 17:38:03.474567 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 17:38:03.474563 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] [343,286] -> [10,343,286], acting primary 343 -> 10 2014-08-13 17:38:03.474578 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.474579 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.474587 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.474589 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] publish_stats_to_osd 194485:25634 2014-08-13 17:38:03.474596 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474599 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.55 2014-08-13 17:38:03.474612 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.59 2014-08-13 17:38:03.474618 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 17:38:03.474630 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474622 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.351 2014-08-13 17:38:03.474632 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474632 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.376 2014-08-13 17:38:03.474641 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474643 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.474650 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474657 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.474671 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 17:38:03.474681 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474683 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474691 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474699 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474719 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.474717 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 17:38:03.474727 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474729 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.474737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.474748 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,343,286] newacting [4,343,286] 2014-08-13 17:38:03.474757 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.474765 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [10,343,286] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'426 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.474780 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:03.474786 7fe99f5e9700 7 osd.4 194485 do_notify osd 356 on 1 PGs 2014-08-13 17:38:03.474781 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [10,343,286](10) acting [10,343,286](10)): not rw, up_thru 194482 up_from 192975 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) : primary up 192975-194482 includes interval 2014-08-13 17:38:03.474792 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194482/194482/194482) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 17:38:03.474805 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing 2014-08-13 17:38:03.474807 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.55 on 1 PGs 2014-08-13 17:38:03.474802 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [10,343,286] -> [4,343,286], acting [10,343,286] -> [4,343,286], acting_primary 10 -> 4, up_primary 10 -> 4, role -1 -> 0 2014-08-13 17:38:03.474813 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.474820 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x59cf640 don't know epoch, doing nothing 2014-08-13 17:38:03.474824 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.59 sending saperate messages on 1 PGs 2014-08-13 17:38:03.474822 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.474833 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:03.474835 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.351 on 1 PGs 2014-08-13 17:38:03.474831 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.474847 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.474841 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2646 2014-08-13 17:38:03.474849 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.376 on 1 PGs 2014-08-13 17:38:03.474850 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474860 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.474867 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.474875 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474883 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474890 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.474898 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.474892 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_advance_map [55,61]/[50,65] -- 55/50 2014-08-13 17:38:03.474912 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.474907 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.474916 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.474916 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Reset advmap 2014-08-13 17:38:03.474926 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.474927 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.474934 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.474937 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [55,61] newacting [50,65] 2014-08-13 17:38:03.474943 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.474947 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.474952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.474957 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [4,55,61]/[4,50,65] r=0 lpr=194478 pi=193747-194476/27 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194479 2014-08-13 17:38:03.474960 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.474968 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.474977 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.474977 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,55,61](4) acting [4,50,65](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194465 generate_past_intervals interval(194477-194478 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.474986 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.474987 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194476/194477/194477) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] noting past interval(194477-194478 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) 2014-08-13 17:38:03.474996 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.474998 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] up [4,55,61] -> [55,61], acting [4,50,65] -> [50,65], acting_primary 4 -> 50, up_primary 4 -> 55, role 0 -> -1 2014-08-13 17:38:03.475007 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 17:38:03.475010 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] clear_primary_state 2014-08-13 17:38:03.475016 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.475020 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] agent_stop 2014-08-13 17:38:03.475025 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001019 7 0.000924 2014-08-13 17:38:03.475030 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] on_change 2014-08-13 17:38:03.475037 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.475039 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.475046 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.475048 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] requeue_ops 2014-08-13 17:38:03.475055 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.475059 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] cancel_copy_ops 2014-08-13 17:38:03.475065 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.475068 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] cancel_flush_ops 2014-08-13 17:38:03.475077 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.475078 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] on_change_cleanup 2014-08-13 17:38:03.475086 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.475087 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] on_change 2014-08-13 17:38:03.475095 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.475097 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] exit NotTrimming 2014-08-13 17:38:03.475104 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.475108 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped m=51] enter NotTrimming 2014-08-13 17:38:03.475115 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 17:38:03.475117 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_stats 2014-08-13 17:38:03.475126 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.475126 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 17:38:03.475136 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_role_change 2014-08-13 17:38:03.475138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 17:38:03.475144 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.475149 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,10,286,343 down blocked_by {} 2014-08-13 17:38:03.475153 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.475159 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.475163 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.475170 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2647 2014-08-13 17:38:03.475173 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475180 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.10 2014-08-13 17:38:03.475196 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.286 2014-08-13 17:38:03.475198 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [55,61]/[50,65] -- 55/50 2014-08-13 17:38:03.475207 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 17:38:03.475213 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475217 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.475194 7fe97a802700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.475226 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.475235 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475217 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.475251 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.475253 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [55,61]/[50,65] -- 55/50 2014-08-13 17:38:03.475266 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475269 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.475278 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.475291 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.475287 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475308 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [50,65,356]/[50,65] -- 50/50 2014-08-13 17:38:03.475321 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475323 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.475323 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 17:38:03.475339 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475333 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.475342 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [50,65,356] newacting [50,65] 2014-08-13 17:38:03.475352 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.475343 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.475360 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [55,61]/[50,65] r=-1 lpr=194479 pi=193747-194478/28 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194482 2014-08-13 17:38:03.475368 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,308] newacting [51,308] 2014-08-13 17:38:03.475378 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.475377 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [55,61](55) acting [50,65](50)): not rw, up_thru 194479 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [55,61](55) acting [50,65](50) maybe_went_rw) : primary up 192979-194479 includes interval 2014-08-13 17:38:03.475388 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.282561 3 0.000206 2014-08-13 17:38:03.475387 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194479/194479/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] noting past interval(194479-194481 up [55,61](55) acting [50,65](50) maybe_went_rw) 2014-08-13 17:38:03.475400 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.282672 0 0.000000 2014-08-13 17:38:03.475399 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] up [55,61] -> [50,65,356], acting [50,65] -> [50,65], acting_primary 50 -> 50, up_primary 55 -> 50, role -1 -> -1 2014-08-13 17:38:03.475410 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.475410 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.475420 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194477 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.475421 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.475430 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194479 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.475430 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.475439 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194479 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.475447 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194479 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,308] newacting [51,308] 2014-08-13 17:38:03.475456 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194479 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.475441 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.475464 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,4,308] r=1 lpr=194479 pi=192975-194475/22 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.475468 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.475479 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.475484 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [51,4,308](51) acting [51,4,308](51)): not rw, up_thru 194476 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [51,4,308](51) acting [51,4,308](51) maybe_went_rw) : primary up 192971-194476 includes interval 2014-08-13 17:38:03.475488 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 17:38:03.475493 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194476/194476/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [51,4,308](51) acting [51,4,308](51) maybe_went_rw) 2014-08-13 17:38:03.475499 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.475504 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,4,308] -> [51,308], acting [51,4,308] -> [51,308], acting_primary 51 -> 51, up_primary 51 -> 51, role 1 -> -1 2014-08-13 17:38:03.475509 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.475515 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.475520 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.475525 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.475529 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.475534 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.475539 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475544 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.475552 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.475560 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.475564 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 17:38:03.475568 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.475578 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475578 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.475586 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.475585 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.475594 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.475596 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.475602 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.475605 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [50,65,356] newacting [50,65,356] 2014-08-13 17:38:03.475611 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.475616 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.475619 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.475626 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356]/[50,65] r=-1 lpr=194482 pi=193747-194481/29 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194483 2014-08-13 17:38:03.475629 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475643 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [50,65,356](50) acting [50,65](50)): not rw, up_thru 194479 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194482-194482 up [50,65,356](50) acting [50,65](50)) : primary up 192979-194479 does not include interval 2014-08-13 17:38:03.475650 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 17:38:03.475652 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194482/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] noting past interval(194482-194482 up [50,65,356](50) acting [50,65](50)) 2014-08-13 17:38:03.475661 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475663 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.475662 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] up [50,65,356] -> [50,65,356], acting [50,65] -> [50,65,356], acting_primary 50 -> 50, up_primary 50 -> 50, role -1 -> -1 2014-08-13 17:38:03.475672 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.475673 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.475681 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.475682 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.475695 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.475704 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.475704 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 17:38:03.475715 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.475717 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480474 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480479 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8700 pg_notify(24.1f7(1) epoch 194485) v5 2014-08-13 17:38:03.475713 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.480510 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.475718 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.480532 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.480542 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.480550 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.55 2014-08-13 17:38:03.480552 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8700 2014-08-13 17:38:03.480553 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.480543 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.480562 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 17:38:03.480566 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480571 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:03.480583 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480585 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480582 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480602 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480604 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480595 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 17:38:03.480605 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c17000 pg_notify(24.1f7(37) epoch 194485) v5 2014-08-13 17:38:03.480607 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480611 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.480611 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 17:38:03.480627 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.351 2014-08-13 17:38:03.480621 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.480629 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c17000 2014-08-13 17:38:03.480630 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480629 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,19,308] newacting [51,19,308] 2014-08-13 17:38:03.480634 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.480639 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.480652 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480645 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.480654 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480649 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,308] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.480656 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480668 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480669 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480671 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4d9f980 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.480668 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [51,308](51) acting [51,308](51)): not rw, up_thru 194479 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [51,308](51) acting [51,308](51) maybe_went_rw) : primary up 192971-194479 includes interval 2014-08-13 17:38:03.480675 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.480681 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.480683 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480684 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480674 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [4,55,61]/[4,55,61] -- 4/4 2014-08-13 17:38:03.480680 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194479/194479/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [51,308](51) acting [51,308](51) maybe_went_rw) 2014-08-13 17:38:03.480688 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480700 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480690 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,308] -> [51,19,308], acting [51,308] -> [51,19,308], acting_primary 51 -> 51, up_primary 51 -> 51, role -1 -> -1 2014-08-13 17:38:03.480691 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:03.480702 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480704 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51ccec0 pg_notify(24.1f7(33) epoch 194485) v5 2014-08-13 17:38:03.480702 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.480704 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.480711 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.480714 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [4,55,61] newacting [4,55,61] 2014-08-13 17:38:03.480727 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.480721 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.480729 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51ccec0 2014-08-13 17:38:03.480725 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.480731 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.480735 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [50,65,356] r=-1 lpr=194483 pi=193747-194482/30 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194485 2014-08-13 17:38:03.480740 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.480751 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480753 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480750 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.480758 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.480756 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [50,65,356](50) acting [50,65,356](50)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194465 generate_past_intervals interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) : primary up 192979-194483 includes interval 2014-08-13 17:38:03.480771 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480773 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480774 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8380 pg_notify(24.1f7 epoch 194485) v5 2014-08-13 17:38:03.480768 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.480768 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194482/194483/194479) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] noting past interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 17:38:03.480778 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.480780 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] up [50,65,356] -> [4,55,61], acting [50,65,356] -> [4,55,61], acting_primary 50 -> 4, up_primary 50 -> 4, role -1 -> 0 2014-08-13 17:38:03.480788 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.480796 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.59 2014-08-13 17:38:03.480797 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8380 2014-08-13 17:38:03.480793 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] clear_primary_state 2014-08-13 17:38:03.480797 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.480804 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] agent_stop 2014-08-13 17:38:03.480807 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480819 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.480821 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.480814 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.480825 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd 194485:22522 2014-08-13 17:38:03.480832 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 17:38:03.480834 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.480844 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480844 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:03.480846 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.480853 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:03.480855 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.480862 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.480864 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480871 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.480879 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.480884 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 17:38:03.480895 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480888 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:03.480897 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.480897 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:03.480908 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.480910 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:03.480917 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480921 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:03.480930 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] clear_stats 2014-08-13 17:38:03.480935 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 17:38:03.480939 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] requeue_ops 2014-08-13 17:38:03.480946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.480949 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] on_role_change 2014-08-13 17:38:03.480950 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.480958 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] requeue_ops 2014-08-13 17:38:03.480959 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.480967 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] cancel_recovery 2014-08-13 17:38:03.480968 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,4,308] newacting [51,4,308] 2014-08-13 17:38:03.480975 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] clear_recovery_state 2014-08-13 17:38:03.480977 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.480986 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,19,308] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.480985 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.480996 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_activate_map 2014-08-13 17:38:03.481003 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [51,19,308](51) acting [51,19,308](51)): not rw, up_thru 194482 up_from 192971 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [51,19,308](51) acting [51,19,308](51) maybe_went_rw) : primary up 192971-194482 includes interval 2014-08-13 17:38:03.481007 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] update_heartbeat_peers -> 4,55,61 2014-08-13 17:38:03.481017 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.481012 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194482/194482/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [51,19,308](51) acting [51,19,308](51) maybe_went_rw) 2014-08-13 17:38:03.481019 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] take_waiters 2014-08-13 17:38:03.481022 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,19,308] -> [51,4,308], acting [51,19,308] -> [51,4,308], acting_primary 51 -> 51, up_primary 51 -> 51, role -1 -> 1 2014-08-13 17:38:03.481029 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Reset 0.612065 9 0.005925 2014-08-13 17:38:03.481033 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481041 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.481041 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started 2014-08-13 17:38:03.481050 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481052 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Start 2014-08-13 17:38:03.481059 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.481061 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: transitioning to Primary 2014-08-13 17:38:03.481067 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.481072 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.481079 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.481084 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary 2014-08-13 17:38:03.481088 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481094 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering 2014-08-13 17:38:03.481097 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.481106 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.481106 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.481115 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.481117 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.481123 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481132 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.481128 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 17:38:03.481140 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.481141 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194482-194482 up [50,65,356](50) acting [50,65](50)) 2014-08-13 17:38:03.481149 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481152 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194479-194481 up [55,61](55) acting [50,65](50) maybe_went_rw) 2014-08-13 17:38:03.481159 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.481163 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194477-194478 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) 2014-08-13 17:38:03.481174 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194476-194476 up [4,55,61](4) acting [50,65,356](50)) 2014-08-13 17:38:03.481205 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.481184 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194474-194475 up [55,61](55) acting [50,65,356](50) maybe_went_rw) 2014-08-13 17:38:03.481215 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.481219 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194473-194473 up [55,61](55) acting [50,65](50)) 2014-08-13 17:38:03.481225 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.005815 7 0.005660 2014-08-13 17:38:03.481232 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 17:38:03.481237 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.481243 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.481247 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.481255 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.481256 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] publish_stats_to_osd 194485:22523 2014-08-13 17:38:03.481267 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.481267 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.50 2014-08-13 17:38:03.481278 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.481278 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.55 2014-08-13 17:38:03.481289 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.61 2014-08-13 17:38:03.481291 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.481299 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.65 2014-08-13 17:38:03.481301 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.481309 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.356 2014-08-13 17:38:03.481322 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.481332 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.481352 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.481390 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.481385 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 17:38:03.481400 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481404 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.481412 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.481420 7fe99ebe8700 1 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.481420 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 17:38:03.481436 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481431 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.288206 2 0.000264 2014-08-13 17:38:03.481440 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.481442 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.481449 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20] newacting [337,20] 2014-08-13 17:38:03.481450 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.288238 0 0.000000 2014-08-13 17:38:03.481458 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.481464 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.288272 0 0.000000 2014-08-13 17:38:03.481468 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.293964 3 0.000132 2014-08-13 17:38:03.481475 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started 0.288330 0 0.000000 2014-08-13 17:38:03.481479 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.294020 0 0.000000 2014-08-13 17:38:03.481485 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.481494 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194477 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.481490 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.481505 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194479 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.481505 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.481515 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481517 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194479 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.481524 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481527 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194479 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340,347] newacting [340,347] 2014-08-13 17:38:03.481533 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20] newacting [337,20] 2014-08-13 17:38:03.481537 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194479 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.481542 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.481546 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [4,340,347] r=0 lpr=194479 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.481552 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.481567 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,340,347](4) acting [4,340,347](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.481570 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [337,20,4](337) acting [337,20,4](337)): not rw, up_thru 194476 up_from 192943 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [337,20,4](337) acting [337,20,4](337) maybe_went_rw) : primary up 192943-194476 includes interval 2014-08-13 17:38:03.481579 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194476/194476/194476) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] noting past interval(194476-194478 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 17:38:03.481580 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [337,20,4](337) acting [337,20,4](337) maybe_went_rw) 2014-08-13 17:38:03.481589 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] up [4,340,347] -> [340,347], acting [4,340,347] -> [340,347], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 17:38:03.481590 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20,4] -> [337,20], acting [337,20,4] -> [337,20], acting_primary 337 -> 337, up_primary 337 -> 337, role 2 -> -1 2014-08-13 17:38:03.481600 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.481601 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481610 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.481610 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.481619 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.481619 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481629 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.481631 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.481637 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.481639 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.481647 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.481647 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.481655 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.481657 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481663 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.481666 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.481672 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.481676 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.481682 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.481684 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.481691 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.481692 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481700 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.481700 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.481709 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.481710 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481718 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.481718 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481726 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481734 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.481739 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 17:38:03.481741 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.481750 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481751 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481753 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481761 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481770 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481774 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 17:38:03.481785 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481788 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481787 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 17:38:03.481798 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481797 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481800 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481806 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481809 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481818 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481826 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 17:38:03.481836 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481836 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 17:38:03.481839 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481847 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481847 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481850 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481856 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.481858 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481866 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20,375] newacting [337,20,375] 2014-08-13 17:38:03.481875 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.481878 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 17:38:03.481889 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.481883 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.481891 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.481900 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.481899 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [337,20](337) acting [337,20](337)): not rw, up_thru 194479 up_from 192943 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [337,20](337) acting [337,20](337) maybe_went_rw) : primary up 192943-194479 includes interval 2014-08-13 17:38:03.481909 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] acting_up_affected newup [7,340,347] newacting [7,340,347] 2014-08-13 17:38:03.481909 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [337,20](337) acting [337,20](337) maybe_went_rw) 2014-08-13 17:38:03.481918 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.481919 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20] -> [337,20,375], acting [337,20] -> [337,20,375], acting_primary 337 -> 337, up_primary 337 -> 337, role -1 -> -1 2014-08-13 17:38:03.481927 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [340,347] r=-1 lpr=194479 pi=194476-194478/1 crt=191648'362 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.481930 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481939 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.481943 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [340,347](340) acting [340,347](340)): not rw, up_thru 194479 up_from 192944 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) : primary up 192944-194479 includes interval 2014-08-13 17:38:03.481948 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481952 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194479/194479/194479) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 17:38:03.481957 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.481962 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] up [340,347] -> [7,340,347], acting [340,347] -> [7,340,347], acting_primary 340 -> 7, up_primary 340 -> 7, role -1 -> -1 2014-08-13 17:38:03.481966 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.481974 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481975 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.481982 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.481984 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.481990 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.481992 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.481999 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.482002 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.482007 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.482011 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.482016 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.482020 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.482025 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482030 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482035 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.482044 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.482050 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 17:38:03.482061 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482052 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] [340,347] -> [7,340,347], acting primary 340 -> 7 2014-08-13 17:38:03.482063 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482066 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.482073 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482075 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.482082 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482088 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482104 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 17:38:03.482115 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482117 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482118 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 17:38:03.482130 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482125 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482133 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482134 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482141 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482150 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482151 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 17:38:03.482162 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482164 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482172 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482172 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 17:38:03.482181 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20,4] newacting [337,20,4] 2014-08-13 17:38:03.482183 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482191 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482189 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.482200 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482204 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,375] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.482209 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482222 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [337,20,375](337) acting [337,20,375](337)): not rw, up_thru 194482 up_from 192943 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [337,20,375](337) acting [337,20,375](337) maybe_went_rw) : primary up 192943-194482 includes interval 2014-08-13 17:38:03.482230 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 17:38:03.482230 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [337,20,375](337) acting [337,20,375](337) maybe_went_rw) 2014-08-13 17:38:03.482242 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482245 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482243 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20,375] -> [337,20,4], acting [337,20,375] -> [337,20,4], acting_primary 337 -> 337, up_primary 337 -> 337, role -1 -> 2 2014-08-13 17:38:03.482254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482255 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482262 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340,347] newacting [4,340,347] 2014-08-13 17:38:03.482263 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.482272 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482271 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.482282 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.482282 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [7,340,347] r=-1 lpr=194482 pi=194476-194481/2 crt=191648'362 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.482292 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.482300 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.482301 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [7,340,347](7) acting [7,340,347](7)): not rw, up_thru 194482 up_from 192980 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) : primary up 192980-194482 includes interval 2014-08-13 17:38:03.482308 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482310 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194482/194482/194482) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 17:38:03.482317 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.482321 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [7,340,347] -> [4,340,347], acting [7,340,347] -> [4,340,347], acting_primary 7 -> 4, up_primary 7 -> 4, role -1 -> 0 2014-08-13 17:38:03.482326 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.482332 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.482335 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.482342 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.482343 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482352 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.482351 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482360 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.482362 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2993 2014-08-13 17:38:03.482369 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482372 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482379 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.482381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.482390 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.482397 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482405 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482412 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482418 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.482420 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.482428 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.482429 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482439 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.482438 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.000948 7 0.000800 2014-08-13 17:38:03.482448 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.482449 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.482456 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.482458 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.482465 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.482466 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.482474 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.482480 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 2014-08-13 17:38:03.482483 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.482491 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.482491 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.482499 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.482504 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.482509 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482513 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.482519 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.482531 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 17:38:03.482540 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.482549 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001063 7 0.000944 2014-08-13 17:38:03.482562 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.482559 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.482568 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.482576 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.482585 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.482588 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,21]/[371,21] -- 371/371 2014-08-13 17:38:03.482603 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482596 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.482606 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.482606 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482616 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.482616 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.482625 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,21] newacting [371,21] 2014-08-13 17:38:03.482626 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482634 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.482636 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 17:38:03.482643 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,4,21] r=1 lpr=194478 pi=192977-194476/26 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.482649 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 17:38:03.482660 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [371,4,21](371) acting [371,4,21](371)): not rw, up_thru 194477 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [371,4,21](371) acting [371,4,21](371) maybe_went_rw) : primary up 192970-194477 includes interval 2014-08-13 17:38:03.482660 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 17:38:03.482670 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194476/194477/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194477-194478 up [371,4,21](371) acting [371,4,21](371) maybe_went_rw) 2014-08-13 17:38:03.482671 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,340,347 down blocked_by {} 2014-08-13 17:38:03.482680 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,4,21] -> [371,21], acting [371,4,21] -> [371,21], acting_primary 371 -> 371, up_primary 371 -> 371, role 1 -> -1 2014-08-13 17:38:03.482681 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.482691 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482693 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2994 2014-08-13 17:38:03.482699 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.482704 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.7 2014-08-13 17:38:03.482709 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482716 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 17:38:03.482719 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.482726 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.347 2014-08-13 17:38:03.482727 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.482737 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.482737 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.482747 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482747 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.482756 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.482765 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.482776 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.482785 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.482784 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.482791 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.482798 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.482807 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482812 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 17:38:03.482827 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482826 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,21]/[371,21] -- 371/371 2014-08-13 17:38:03.482837 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482830 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.482839 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482840 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24] newacting [345,24] 2014-08-13 17:38:03.482848 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.482852 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.482857 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482863 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.289098 3 0.000339 2014-08-13 17:38:03.482874 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.289162 0 0.000000 2014-08-13 17:38:03.482875 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,21]/[371,21] -- 371/371 2014-08-13 17:38:03.482885 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482884 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.482888 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482893 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194477 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.482897 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.482903 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194479 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482906 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.482912 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194479 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.482921 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194479 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24] newacting [345,24] 2014-08-13 17:38:03.482926 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 17:38:03.482929 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194479 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.482937 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.482938 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24,4] r=2 lpr=194479 pi=192984-194475/21 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.482939 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.482948 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.482956 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [345,24,4](345) acting [345,24,4](345)): not rw, up_thru 194476 up_from 192981 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [345,24,4](345) acting [345,24,4](345) maybe_went_rw) : primary up 192981-194476 includes interval 2014-08-13 17:38:03.482957 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,24,7] newacting [371,24,7] 2014-08-13 17:38:03.482967 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.482966 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194476/194476/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [345,24,4](345) acting [345,24,4](345) maybe_went_rw) 2014-08-13 17:38:03.482975 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,21] r=-1 lpr=194479 pi=192977-194478/27 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.482977 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24,4] -> [345,24], acting [345,24,4] -> [345,24], acting_primary 345 -> 345, up_primary 345 -> 345, role 2 -> -1 2014-08-13 17:38:03.482988 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.482991 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [371,21](371) acting [371,21](371)): not rw, up_thru 194479 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [371,21](371) acting [371,21](371) maybe_went_rw) : primary up 192970-194479 includes interval 2014-08-13 17:38:03.482996 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483001 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194479/194479/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [371,21](371) acting [371,21](371) maybe_went_rw) 2014-08-13 17:38:03.483005 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483011 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,21] -> [371,24,7], acting [371,21] -> [371,24,7], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> -1 2014-08-13 17:38:03.483015 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.483022 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483023 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.483031 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483031 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.483039 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483040 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483048 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.483049 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.483056 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.483058 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.483065 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.483070 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.483074 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483078 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483083 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.483087 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.483092 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.483095 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.483101 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.483105 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483109 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.483118 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483124 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 17:38:03.483135 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483137 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483138 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 17:38:03.483150 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483145 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483152 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483155 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483161 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.483170 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483172 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 17:38:03.483182 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483184 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483189 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 17:38:03.483199 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483206 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483208 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483209 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483218 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.483226 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483227 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 17:38:03.483238 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483240 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483243 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 17:38:03.483254 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483249 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483257 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483257 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24,60] newacting [345,24,60] 2014-08-13 17:38:03.483266 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.483267 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.483276 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24] r=-1 lpr=194479 pi=192984-194478/22 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.483275 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,4,21] newacting [371,4,21] 2014-08-13 17:38:03.483285 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.483293 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,24,7] r=-1 lpr=194482 pi=192977-194481/28 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.483292 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [345,24](345) acting [345,24](345)): not rw, up_thru 194479 up_from 192981 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [345,24](345) acting [345,24](345) maybe_went_rw) : primary up 192981-194479 includes interval 2014-08-13 17:38:03.483302 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194479/194479/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [345,24](345) acting [345,24](345) maybe_went_rw) 2014-08-13 17:38:03.483309 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [371,24,7](371) acting [371,24,7](371)): not rw, up_thru 194482 up_from 192970 last_epoch_clean 194466 generate_past_intervals interval(194482-194484 up [371,24,7](371) acting [371,24,7](371) maybe_went_rw) : primary up 192970-194482 includes interval 2014-08-13 17:38:03.483311 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24] -> [345,24,60], acting [345,24] -> [345,24,60], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> -1 2014-08-13 17:38:03.483322 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483321 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194482/194482/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [371,24,7](371) acting [371,24,7](371) maybe_went_rw) 2014-08-13 17:38:03.483331 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483332 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,24,7] -> [371,4,21], acting [371,24,7] -> [371,4,21], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> 1 2014-08-13 17:38:03.483339 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483343 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483348 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.483351 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483356 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.483360 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483365 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.483369 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.483374 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483378 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.483383 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.483387 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.483392 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.483395 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483401 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.483405 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.483410 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.483414 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.483420 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483423 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.483431 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483438 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.483440 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 17:38:03.483451 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483446 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.483453 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483455 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483462 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483465 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.483471 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483490 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 17:38:03.483500 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483502 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483499 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.483511 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483512 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.483520 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483521 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.768743 9 0.000744 2014-08-13 17:38:03.483532 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.483536 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 17:38:03.483541 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.483548 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483549 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.483550 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.483559 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.483560 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.483568 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24,4] newacting [345,24,4] 2014-08-13 17:38:03.483571 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.483577 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.483583 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.483586 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,60] r=-1 lpr=194482 pi=192984-194481/23 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.483593 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.483602 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [345,24,60](345) acting [345,24,60](345)): not rw, up_thru 194482 up_from 192981 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [345,24,60](345) acting [345,24,60](345) maybe_went_rw) : primary up 192981-194482 includes interval 2014-08-13 17:38:03.483611 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194482/194482/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [345,24,60](345) acting [345,24,60](345) maybe_went_rw) 2014-08-13 17:38:03.483620 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24,60] -> [345,24,4], acting [345,24,60] -> [345,24,4], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> 2 2014-08-13 17:38:03.483631 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483644 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.483638 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483646 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483654 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.483662 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.483669 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.483677 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.483672 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,334]/[356,32] -- 356/356 2014-08-13 17:38:03.483690 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483686 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.483694 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.483694 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.483703 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.483704 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.483711 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.483715 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,334] newacting [356,32] 2014-08-13 17:38:03.483719 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.483725 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.483728 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.483735 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,4,334]/[356,4,32] r=1 lpr=194478 pi=193150-194476/33 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194479 2014-08-13 17:38:03.483737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483746 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.483753 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [356,4,334](356) acting [356,4,32](356)): not rw, up_thru 194477 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [356,4,334](356) acting [356,4,32](356) maybe_went_rw) : primary up 192969-194477 includes interval 2014-08-13 17:38:03.483763 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194476/194477/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194477-194478 up [356,4,334](356) acting [356,4,32](356) maybe_went_rw) 2014-08-13 17:38:03.483773 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,4,334] -> [356,334], acting [356,4,32] -> [356,32], acting_primary 356 -> 356, up_primary 356 -> 356, role 1 -> -1 2014-08-13 17:38:03.483783 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.483783 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.483794 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.483794 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.483804 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.483804 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.000920 7 0.000784 2014-08-13 17:38:03.483816 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.483815 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.483825 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.483825 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.483834 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.483835 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.483844 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.483844 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.483855 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.483855 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.483865 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.483868 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.483874 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 17:38:03.483878 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.483884 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.483893 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.483902 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.483912 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.483926 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.483933 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,334]/[356,32,322] -- 356/356 2014-08-13 17:38:03.483947 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483950 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.483959 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.483957 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 17:38:03.483968 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,334] newacting [356,32,322] 2014-08-13 17:38:03.483976 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.483977 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.483980 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Started advmap 2014-08-13 17:38:03.483986 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32] r=-1 lpr=194479 pi=193150-194478/34 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194480 2014-08-13 17:38:03.483990 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374] newacting [6,374] 2014-08-13 17:38:03.484000 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.484004 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [356,334](356) acting [356,32](356)): not rw, up_thru 194477 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [356,334](356) acting [356,32](356)) : primary up 192969-194477 does not include interval 2014-08-13 17:38:03.484010 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 0.289440 3 0.000218 2014-08-13 17:38:03.484014 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194479/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194479-194479 up [356,334](356) acting [356,32](356)) 2014-08-13 17:38:03.484021 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started 0.289525 0 0.000000 2014-08-13 17:38:03.484025 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,334] -> [356,334], acting [356,32] -> [356,32,322], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.484032 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset 2014-08-13 17:38:03.484036 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.484041 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194479 2014-08-13 17:38:03.484046 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484051 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484057 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.484062 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484067 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.484072 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374] newacting [6,374] 2014-08-13 17:38:03.484076 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.484081 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484086 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.484090 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194479 2014-08-13 17:38:03.484095 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.484106 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.484108 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [6,374,4](6) acting [6,374,4](6)): not rw, up_thru 194476 up_from 192967 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [6,374,4](6) acting [6,374,4](6) maybe_went_rw) : primary up 192967-194476 includes interval 2014-08-13 17:38:03.484115 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.484117 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194476/194476/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194476-194478 up [6,374,4](6) acting [6,374,4](6) maybe_went_rw) 2014-08-13 17:38:03.484124 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.484127 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374,4] -> [6,374], acting [6,374,4] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role 2 -> -1 2014-08-13 17:38:03.484133 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.484138 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:03.484144 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484148 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484158 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.484167 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 17:38:03.484165 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,334]/[356,32,322] -- 356/356 2014-08-13 17:38:03.484180 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484176 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 17:38:03.484182 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.484199 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.484199 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 17:38:03.484212 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:03.484211 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484223 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 17:38:03.484232 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 17:38:03.484236 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,322,32]/[356,32,322] -- 356/356 2014-08-13 17:38:03.484240 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_role_change 2014-08-13 17:38:03.484250 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484249 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.484253 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.484257 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 17:38:03.484263 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.484267 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 17:38:03.484273 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,322,32] newacting [356,32,322] 2014-08-13 17:38:03.484277 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484283 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484293 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,334]/[356,32,322] r=-1 lpr=194480 pi=193150-194479/35 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194482 2014-08-13 17:38:03.484298 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 17:38:03.484313 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484310 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [356,334](356) acting [356,32,322](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [356,334](356) acting [356,32,322](356) maybe_went_rw) : primary up 192969-194480 includes interval 2014-08-13 17:38:03.484316 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484320 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194479/194480/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194480-194481 up [356,334](356) acting [356,32,322](356) maybe_went_rw) 2014-08-13 17:38:03.484326 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484331 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,334] -> [356,322,32], acting [356,32,322] -> [356,32,322], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.484336 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484343 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.484352 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484353 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 17:38:03.484366 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484362 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.484369 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484371 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.484378 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484382 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.484388 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484391 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.484401 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.484408 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,336]/[6,374,336] -- 6/6 2014-08-13 17:38:03.484410 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.484420 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484420 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.484423 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484430 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.484433 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484439 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.484442 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374,336] newacting [6,374,336] 2014-08-13 17:38:03.484450 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484452 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484461 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194482 2014-08-13 17:38:03.484474 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,322,32]/[356,322,32] -- 356/356 2014-08-13 17:38:03.484476 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [6,374](6) acting [6,374](6)): not rw, up_thru 194479 up_from 192967 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [6,374](6) acting [6,374](6) maybe_went_rw) : primary up 192967-194479 includes interval 2014-08-13 17:38:03.484488 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484486 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194479/194479/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194479-194481 up [6,374](6) acting [6,374](6) maybe_went_rw) 2014-08-13 17:38:03.484491 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.484496 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374] -> [6,374,336], acting [6,374] -> [6,374,336], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 17:38:03.484502 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.484508 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:03.484512 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,322,32] newacting [356,322,32] 2014-08-13 17:38:03.484517 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484523 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484526 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.484532 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32]/[356,32,322] r=-1 lpr=194482 pi=193150-194481/36 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194483 2014-08-13 17:38:03.484536 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 17:38:03.484545 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 17:38:03.484550 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [356,322,32](356) acting [356,32,322](356)): not rw, up_thru 194480 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [356,322,32](356) acting [356,32,322](356)) : primary up 192969-194480 does not include interval 2014-08-13 17:38:03.484553 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 17:38:03.484562 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:03.484561 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194482/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] noting past interval(194482-194482 up [356,322,32](356) acting [356,32,322](356)) 2014-08-13 17:38:03.484571 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 17:38:03.484571 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] up [356,322,32] -> [356,322,32], acting [356,32,322] -> [356,322,32], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 17:38:03.484581 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 17:38:03.484583 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.484590 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 17:38:03.484593 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484599 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 17:38:03.484602 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.484609 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484611 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.484620 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.484631 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.484630 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,336]/[6,374] -- 6/6 2014-08-13 17:38:03.484644 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484640 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.484646 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484649 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.484656 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484659 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.484665 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374,336] newacting [6,374] 2014-08-13 17:38:03.484668 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.484675 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484676 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.484684 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194483 2014-08-13 17:38:03.484686 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484701 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [6,374,336](6) acting [6,374,336](6)): not rw, up_thru 194479 up_from 192967 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [6,374,336](6) acting [6,374,336](6)) : primary up 192967-194479 does not include interval 2014-08-13 17:38:03.484710 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194482/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] noting past interval(194482-194482 up [6,374,336](6) acting [6,374,336](6)) 2014-08-13 17:38:03.484710 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,322,32]/[356,322,32] -- 356/356 2014-08-13 17:38:03.484725 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484721 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] up [6,374,336] -> [6,374,336], acting [6,374,336] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 17:38:03.484728 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.484733 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 17:38:03.484737 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.484743 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484747 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484752 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.484761 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_copy_ops 2014-08-13 17:38:03.484767 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,334] -- 356/356 2014-08-13 17:38:03.484770 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_flush_ops 2014-08-13 17:38:03.484780 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484779 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change_cleanup 2014-08-13 17:38:03.484783 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.484788 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 17:38:03.484792 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.484798 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit NotTrimming 2014-08-13 17:38:03.484802 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,334] 2014-08-13 17:38:03.484808 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter NotTrimming 2014-08-13 17:38:03.484811 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484818 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_recovery 2014-08-13 17:38:03.484820 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,322,32] r=-1 lpr=194483 pi=193150-194482/37 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194485 2014-08-13 17:38:03.484827 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] clear_recovery_state 2014-08-13 17:38:03.484837 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484837 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [356,322,32](356) acting [356,322,32](356)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [356,322,32](356) acting [356,322,32](356) maybe_went_rw) : primary up 192969-194483 includes interval 2014-08-13 17:38:03.484849 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194482/194483/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] noting past interval(194483-194484 up [356,322,32](356) acting [356,322,32](356) maybe_went_rw) 2014-08-13 17:38:03.484859 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] up [356,322,32] -> [356,4,334], acting [356,322,32] -> [356,4,334], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> 1 2014-08-13 17:38:03.484860 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,336]/[6,374] -- 6/6 2014-08-13 17:38:03.484875 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484870 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.484877 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484879 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.484888 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.484887 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484897 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.484897 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484907 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.484915 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.484917 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,4]/[6,374] -- 6/6 2014-08-13 17:38:03.484923 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.484930 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.484933 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.484933 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:03.484943 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.484943 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.484952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_role_change 2014-08-13 17:38:03.484953 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374] 2014-08-13 17:38:03.484961 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.484963 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.484969 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.484973 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,336]/[6,374] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194485 2014-08-13 17:38:03.484978 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.484988 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.484991 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [6,374,336](6) acting [6,374](6)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [6,374,336](6) acting [6,374](6) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.484998 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:03.485001 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194482/194483/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] noting past interval(194483-194484 up [6,374,336](6) acting [6,374](6) maybe_went_rw) 2014-08-13 17:38:03.485011 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] up [6,374,336] -> [6,374,4], acting [6,374] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 17:38:03.485022 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 17:38:03.485030 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.485039 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.485040 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 17:38:03.485050 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:03.485048 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_copy_ops 2014-08-13 17:38:03.485060 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_flush_ops 2014-08-13 17:38:03.485061 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.368096 9 0.001163 2014-08-13 17:38:03.485070 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change_cleanup 2014-08-13 17:38:03.485073 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 17:38:03.485079 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 17:38:03.485083 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 17:38:03.485091 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 17:38:03.485091 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit NotTrimming 2014-08-13 17:38:03.485101 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter NotTrimming 2014-08-13 17:38:03.485102 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.485112 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_recovery 2014-08-13 17:38:03.485113 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 17:38:03.485120 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] clear_recovery_state 2014-08-13 17:38:03.485125 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.485131 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485135 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.485141 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_activate_map 2014-08-13 17:38:03.485197 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.485191 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] update_heartbeat_peers unchanged 2014-08-13 17:38:03.485203 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] take_waiters 2014-08-13 17:38:03.485212 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Reset 0.001180 7 0.001004 2014-08-13 17:38:03.485223 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started 2014-08-13 17:38:03.485225 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 17:38:03.485233 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Start 2014-08-13 17:38:03.485241 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485242 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: transitioning to Stray 2014-08-13 17:38:03.485244 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.485253 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.485265 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started/Stray 2014-08-13 17:38:03.485254 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343] newacting [397,343] 2014-08-13 17:38:03.485276 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.485284 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.485286 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.485295 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.297348 3 0.000123 2014-08-13 17:38:03.485307 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.297400 0 0.000000 2014-08-13 17:38:03.485317 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.485325 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194477 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.485334 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194479 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.485342 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194479 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.485350 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194479 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343] newacting [397,343] 2014-08-13 17:38:03.485358 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.485358 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194479 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.485366 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343,4] r=2 lpr=194479 pi=192965-194475/21 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.485382 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [397,343,4](397) acting [397,343,4](397)): not rw, up_thru 194476 up_from 192958 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [397,343,4](397) acting [397,343,4](397) maybe_went_rw) : primary up 192958-194476 includes interval 2014-08-13 17:38:03.485392 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194476/194476/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [397,343,4](397) acting [397,343,4](397) maybe_went_rw) 2014-08-13 17:38:03.485387 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,303]/[376,303] -- 376/376 2014-08-13 17:38:03.485404 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485402 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343,4] -> [397,343], acting [397,343,4] -> [397,343], acting_primary 397 -> 397, up_primary 397 -> 397, role 2 -> -1 2014-08-13 17:38:03.485407 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.485412 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.485417 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.485422 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.485426 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,303] newacting [376,303] 2014-08-13 17:38:03.485430 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.485436 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.485440 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.485445 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,4,303] r=1 lpr=194478 pi=192975-194476/26 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.485448 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.485457 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.485462 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [376,4,303](376) acting [376,4,303](376)): not rw, up_thru 194477 up_from 192974 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [376,4,303](376) acting [376,4,303](376) maybe_went_rw) : primary up 192974-194477 includes interval 2014-08-13 17:38:03.485465 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.485473 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194476/194477/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194477-194478 up [376,4,303](376) acting [376,4,303](376) maybe_went_rw) 2014-08-13 17:38:03.485475 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.485484 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.485483 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,4,303] -> [376,303], acting [376,4,303] -> [376,303], acting_primary 376 -> 376, up_primary 376 -> 376, role 1 -> -1 2014-08-13 17:38:03.485493 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.485494 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.485501 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.485503 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.485509 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.485512 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.485517 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.485522 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.485529 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485530 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.485539 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.485547 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.485548 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 17:38:03.485560 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485556 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.485562 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.485565 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.485571 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.485574 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.485580 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485582 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.485590 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.485597 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.485596 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 17:38:03.485607 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485607 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485610 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.485618 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.485627 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485626 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,303]/[376,303] -- 376/376 2014-08-13 17:38:03.485637 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485639 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.485645 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 17:38:03.485648 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.485656 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.485657 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.485658 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.489858 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.485673 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,303]/[376,303] -- 376/376 2014-08-13 17:38:03.489879 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343,380] newacting [397,343,380] 2014-08-13 17:38:03.489889 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.489891 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.489898 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343] r=-1 lpr=194479 pi=192965-194478/22 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.489900 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.489915 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.489916 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [397,343](397) acting [397,343](397)): not rw, up_thru 194479 up_from 192958 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [397,343](397) acting [397,343](397) maybe_went_rw) : primary up 192958-194479 includes interval 2014-08-13 17:38:03.489926 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.489926 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194479/194479/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [397,343](397) acting [397,343](397) maybe_went_rw) 2014-08-13 17:38:03.489937 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343] -> [397,343,380], acting [397,343] -> [397,343,380], acting_primary 397 -> 397, up_primary 397 -> 397, role -1 -> -1 2014-08-13 17:38:03.489952 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.489954 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 17:38:03.489966 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.489961 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.489969 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.489968 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.489979 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.489981 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.489988 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.489990 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,313,319] newacting [376,313,319] 2014-08-13 17:38:03.489996 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.489999 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.490005 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490009 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,303] r=-1 lpr=194479 pi=192975-194478/27 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.490016 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.490025 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.490028 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [376,303](376) acting [376,303](376)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [376,303](376) acting [376,303](376) maybe_went_rw) : primary up 192974-194479 includes interval 2014-08-13 17:38:03.490034 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490038 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194479/194479/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [376,303](376) acting [376,303](376) maybe_went_rw) 2014-08-13 17:38:03.490043 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490048 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,303] -> [376,313,319], acting [376,303] -> [376,313,319], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> -1 2014-08-13 17:38:03.490053 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490060 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490068 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.490076 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490078 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 17:38:03.490090 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490086 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.490094 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.490094 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490103 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.490104 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.490113 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490114 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490123 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.490132 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.490133 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 17:38:03.490145 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490140 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490147 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490149 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490156 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.490159 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490165 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490192 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 17:38:03.490205 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490199 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 17:38:03.490212 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490207 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490216 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.490223 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.490226 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490232 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343,4] newacting [397,343,4] 2014-08-13 17:38:03.490241 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.490247 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 17:38:03.490249 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,380] r=-1 lpr=194482 pi=192965-194481/23 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.490258 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490260 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490267 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [397,343,380](397) acting [397,343,380](397)): not rw, up_thru 194482 up_from 192958 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [397,343,380](397) acting [397,343,380](397) maybe_went_rw) : primary up 192958-194482 includes interval 2014-08-13 17:38:03.490268 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.490276 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194482/194482/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [397,343,380](397) acting [397,343,380](397) maybe_went_rw) 2014-08-13 17:38:03.490278 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490287 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343,380] -> [397,343,4], acting [397,343,380] -> [397,343,4], acting_primary 397 -> 397, up_primary 397 -> 397, role -1 -> 2 2014-08-13 17:38:03.490297 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490296 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 17:38:03.490307 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490305 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.490309 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490314 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490318 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.490323 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.490327 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,4,303] newacting [376,4,303] 2014-08-13 17:38:03.490332 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.490337 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.490341 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.490346 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,313,319] r=-1 lpr=194482 pi=192975-194481/28 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.490350 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490359 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.490364 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [376,313,319](376) acting [376,313,319](376)): not rw, up_thru 194482 up_from 192974 last_epoch_clean 194466 generate_past_intervals interval(194482-194484 up [376,313,319](376) acting [376,313,319](376) maybe_went_rw) : primary up 192974-194482 includes interval 2014-08-13 17:38:03.490368 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.490373 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194482/194482/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [376,313,319](376) acting [376,313,319](376) maybe_went_rw) 2014-08-13 17:38:03.490377 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.490386 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490383 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,313,319] -> [376,4,303], acting [376,313,319] -> [376,4,303], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> 1 2014-08-13 17:38:03.490394 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490402 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490403 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.490412 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490412 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490422 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.490423 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.490430 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.490438 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.490445 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490453 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.490458 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.490462 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.490469 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.490471 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.490479 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490478 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.005161 7 0.005017 2014-08-13 17:38:03.490487 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490491 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.490495 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490501 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.490505 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490510 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.490515 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.490521 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.490532 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.490544 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.490553 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.490556 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.490568 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.490577 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.841544 9 0.000789 2014-08-13 17:38:03.490589 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.490597 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.490603 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.490607 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.490617 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.490627 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.490640 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.490638 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 17:38:03.490653 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490649 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.490656 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.490666 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335] newacting [30,335] 2014-08-13 17:38:03.490674 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.490683 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.302492 3 0.000120 2014-08-13 17:38:03.490701 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.490694 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.302539 0 0.000000 2014-08-13 17:38:03.490704 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.490712 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194477 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.490721 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490729 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.490737 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335] newacting [30,335] 2014-08-13 17:38:03.490731 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [338,376]/[338,376] -- 338/338 2014-08-13 17:38:03.490749 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490746 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.490753 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.490754 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335,4] r=2 lpr=194479 pi=192976-194475/21 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.490762 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.490771 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [338,376] newacting [338,376] 2014-08-13 17:38:03.490772 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [30,335,4](30) acting [30,335,4](30)): not rw, up_thru 194476 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [30,335,4](30) acting [30,335,4](30) maybe_went_rw) : primary up 192976-194476 includes interval 2014-08-13 17:38:03.490780 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.490781 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194476/194476/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [30,335,4](30) acting [30,335,4](30) maybe_went_rw) 2014-08-13 17:38:03.490789 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [4,338,376] r=0 lpr=194478 pi=193306-194476/45 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.490792 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335,4] -> [30,335], acting [30,335,4] -> [30,335], acting_primary 30 -> 30, up_primary 30 -> 30, role 2 -> -1 2014-08-13 17:38:03.490803 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490809 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,338,376](4) acting [4,338,376](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194389 generate_past_intervals interval(194477-194478 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.490811 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.490820 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490818 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194476/194477/194477) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] noting past interval(194477-194478 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) 2014-08-13 17:38:03.490829 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.490828 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] up [4,338,376] -> [338,376], acting [4,338,376] -> [338,376], acting_primary 4 -> 338, up_primary 4 -> 338, role 0 -> -1 2014-08-13 17:38:03.490837 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.490839 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.490849 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.490846 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.490857 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.490857 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.490866 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.490868 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.490874 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.490877 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.490884 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.490886 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.490892 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.490894 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490901 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.490902 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490910 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.490910 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490920 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.490921 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490929 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.490937 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.490945 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490943 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 17:38:03.490954 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.490953 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.490957 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.490961 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.490966 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.490970 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.490976 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490978 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.490987 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.490995 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 17:38:03.491005 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491007 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491005 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,376]/[338,376] -- 338/338 2014-08-13 17:38:03.491016 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491015 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.491019 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491024 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491031 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491039 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491045 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 17:38:03.491058 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491055 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,376]/[338,376] -- 338/338 2014-08-13 17:38:03.491066 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491060 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491068 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491069 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.491077 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491077 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335,25] newacting [30,335,25] 2014-08-13 17:38:03.491086 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491087 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.491096 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335] r=-1 lpr=194479 pi=192976-194478/22 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.491108 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 17:38:03.491111 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [30,335](30) acting [30,335](30)): not rw, up_thru 194479 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [30,335](30) acting [30,335](30) maybe_went_rw) : primary up 192976-194479 includes interval 2014-08-13 17:38:03.491118 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491121 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491120 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194479/194479/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [30,335](30) acting [30,335](30) maybe_went_rw) 2014-08-13 17:38:03.491130 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491131 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335] -> [30,335,25], acting [30,335] -> [30,335,25], acting_primary 30 -> 30, up_primary 30 -> 30, role -1 -> -1 2014-08-13 17:38:03.491138 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] acting_up_affected newup [338,373,396] newacting [338,373,396] 2014-08-13 17:38:03.491142 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491147 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.491151 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.491156 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,376] r=-1 lpr=194479 pi=193306-194478/46 crt=191727'441 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.491160 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491169 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.491172 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [338,376](338) acting [338,376](338)): not rw, up_thru 194479 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) : primary up 194414-194479 includes interval 2014-08-13 17:38:03.491177 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.491182 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194479/194479/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) 2014-08-13 17:38:03.491186 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.491199 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491198 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] up [338,376] -> [338,373,396], acting [338,376] -> [338,373,396], acting_primary 338 -> 338, up_primary 338 -> 338, role -1 -> -1 2014-08-13 17:38:03.491209 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491209 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.491218 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.491219 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.491227 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491228 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.491236 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.491236 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.491244 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.491246 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491253 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.491261 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491269 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.491269 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 17:38:03.491281 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491279 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.491283 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491287 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.491292 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.491296 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.491302 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491306 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491325 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 17:38:03.491336 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491338 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491338 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 17:38:03.491351 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491347 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.491353 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491355 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491362 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491371 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491375 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 17:38:03.491385 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491387 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491391 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 17:38:03.491402 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491396 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.491404 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491404 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335,4] newacting [30,335,4] 2014-08-13 17:38:03.491414 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491415 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.491423 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491423 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,25] r=-1 lpr=194482 pi=192976-194481/23 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.491443 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [30,335,25](30) acting [30,335,25](30)): not rw, up_thru 194482 up_from 192976 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [30,335,25](30) acting [30,335,25](30) maybe_went_rw) : primary up 192976-194482 includes interval 2014-08-13 17:38:03.491443 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 17:38:03.491454 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491453 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194482/194482/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [30,335,25](30) acting [30,335,25](30) maybe_went_rw) 2014-08-13 17:38:03.491456 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.491465 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491463 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335,25] -> [30,335,4], acting [30,335,25] -> [30,335,4], acting_primary 30 -> 30, up_primary 30 -> 30, role -1 -> 2 2014-08-13 17:38:03.491474 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491473 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,338,376] newacting [4,338,376] 2014-08-13 17:38:03.491484 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.491485 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.491493 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491494 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [338,373,396] r=-1 lpr=194482 pi=193306-194481/47 crt=191727'441 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.491502 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.491510 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.491512 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [338,373,396](338) acting [338,373,396](338)): not rw, up_thru 194482 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) : primary up 194414-194482 includes interval 2014-08-13 17:38:03.491519 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.491521 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194482/194482/194479) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 17:38:03.491527 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491532 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [338,373,396] -> [4,338,376], acting [338,373,396] -> [4,338,376], acting_primary 338 -> 4, up_primary 338 -> 4, role -1 -> 0 2014-08-13 17:38:03.491536 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.491543 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.491546 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.491553 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.491554 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.491562 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491562 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491571 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.491573 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2591 2014-08-13 17:38:03.491579 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.491583 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491589 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491593 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.491599 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.491603 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.491611 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491618 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491626 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.491634 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.491634 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.491643 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.491644 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.491652 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.491653 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.000949 7 0.000789 2014-08-13 17:38:03.491661 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.491666 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.491670 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.491675 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.491679 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.491684 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.491688 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.491695 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.491696 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.491705 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.491706 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.491713 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.491718 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.491723 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.491727 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.491733 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.491744 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,338,376 2014-08-13 17:38:03.491753 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.491755 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.491763 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.799503 9 0.000857 2014-08-13 17:38:03.491774 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.491783 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.491796 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.491791 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.491801 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.491811 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.491820 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.491829 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.491825 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 17:38:03.491841 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.491838 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:03.491845 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.491848 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 17:38:03.491854 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.491862 7fe99f5e9700 1 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.491861 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) 2014-08-13 17:38:03.491872 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.303208 2 0.000160 2014-08-13 17:38:03.491872 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) 2014-08-13 17:38:03.491884 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.491884 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,338,376](4) acting [338,376,293](338)) 2014-08-13 17:38:03.491894 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.303238 0 0.000000 2014-08-13 17:38:03.491895 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [338,376](338) acting [338,376,293](338) maybe_went_rw) 2014-08-13 17:38:03.491905 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.303259 0 0.000000 2014-08-13 17:38:03.491906 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [338,376](338) acting [338,376](338)) 2014-08-13 17:38:03.491916 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.303308 0 0.000000 2014-08-13 17:38:03.491916 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,293,338,373,376,396 down blocked_by {} 2014-08-13 17:38:03.491927 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.491927 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.491935 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194477 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.491939 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2592 2014-08-13 17:38:03.491944 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194479 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.491952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.293 2014-08-13 17:38:03.491955 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194479 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194477 >= end epoch 194476, nothing to do 2014-08-13 17:38:03.491963 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.338 2014-08-13 17:38:03.491964 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194479 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [50,289] newacting [50,289] 2014-08-13 17:38:03.491973 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194479 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.491973 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 17:38:03.491982 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [4,50,289] r=0 lpr=194479 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.491983 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 17:38:03.491993 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.396 2014-08-13 17:38:03.491999 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,50,289](4) acting [4,50,289](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.492005 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.492008 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194476/194476/194476) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] noting past interval(194476-194478 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 17:38:03.492015 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.492018 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] up [4,50,289] -> [50,289], acting [4,50,289] -> [50,289], acting_primary 4 -> 50, up_primary 4 -> 50, role 0 -> -1 2014-08-13 17:38:03.492029 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.492037 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.492045 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.492053 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.492060 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.492069 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.492076 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.492084 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.492091 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.492102 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.492100 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.492108 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.492116 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.492124 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492131 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.492131 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,314]/[374,12] -- 374/374 2014-08-13 17:38:03.492142 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492152 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492149 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.492157 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.492156 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Started advmap 2014-08-13 17:38:03.492166 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492167 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,314] newacting [374,12] 2014-08-13 17:38:03.492177 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.492194 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started/Stray 0.296810 3 0.000206 2014-08-13 17:38:03.492188 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 17:38:03.492206 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492206 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started 0.296893 0 0.000000 2014-08-13 17:38:03.492209 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492217 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492218 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset 2014-08-13 17:38:03.492227 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492228 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194477 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194479 2014-08-13 17:38:03.492239 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194479 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.492248 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194479 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.492247 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 17:38:03.492258 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492257 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194479 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,314] newacting [374,12] 2014-08-13 17:38:03.492260 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492267 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194479 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.492269 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492276 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,4,314]/[374,4,12] r=1 lpr=194479 pi=192688-194475/42 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194479 2014-08-13 17:38:03.492278 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492294 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [374,4,314](374) acting [374,4,12](374)): not rw, up_thru 194476 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194476-194478 up [374,4,314](374) acting [374,4,12](374) maybe_went_rw) : primary up 192967-194476 includes interval 2014-08-13 17:38:03.492299 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 17:38:03.492310 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492304 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194476/194476/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194476-194478 up [374,4,314](374) acting [374,4,12](374) maybe_went_rw) 2014-08-13 17:38:03.492312 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492315 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,4,314] -> [374,314], acting [374,4,12] -> [374,12], acting_primary 374 -> 374, up_primary 374 -> 374, role 1 -> -1 2014-08-13 17:38:03.492321 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492328 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.492330 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [32,50,289] newacting [32,50,289] 2014-08-13 17:38:03.492337 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.492339 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.492346 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.492348 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [50,289] r=-1 lpr=194479 pi=194476-194478/1 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.492357 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:03.492364 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [50,289](50) acting [50,289](50)): not rw, up_thru 194479 up_from 192979 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) : primary up 192979-194479 includes interval 2014-08-13 17:38:03.492366 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:03.492373 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194479/194479/194479) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 17:38:03.492375 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:03.492383 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] up [50,289] -> [32,50,289], acting [50,289] -> [32,50,289], acting_primary 50 -> 32, up_primary 50 -> 32, role -1 -> -1 2014-08-13 17:38:03.492385 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.492394 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.492395 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:03.492402 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.492405 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:03.492411 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492415 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_role_change 2014-08-13 17:38:03.492420 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.492424 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.492429 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.492433 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 17:38:03.492440 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.492442 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:03.492449 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.492452 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492458 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.492467 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.492473 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,314]/[374,12,294] -- 374/374 2014-08-13 17:38:03.492475 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] [50,289] -> [32,50,289], acting primary 50 -> 32 2014-08-13 17:38:03.492488 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492489 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.492497 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.492491 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.492578 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.492587 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,314] newacting [374,12,294] 2014-08-13 17:38:03.492596 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.492605 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12] r=-1 lpr=194479 pi=192688-194478/43 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194480 2014-08-13 17:38:03.492505 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492622 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [374,314](374) acting [374,12](374)): not rw, up_thru 194476 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [374,314](374) acting [374,12](374)) : primary up 192967-194476 does not include interval 2014-08-13 17:38:03.492631 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194479/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194479-194479 up [374,314](374) acting [374,12](374)) 2014-08-13 17:38:03.492639 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 17:38:03.492642 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,314] -> [374,314], acting [374,12] -> [374,12,294], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 17:38:03.492653 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492653 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.492656 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492663 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.492665 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492672 7fe99ebe8700 15 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.492674 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492682 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:03.492691 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:03.492695 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 17:38:03.492707 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492700 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:03.492709 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492710 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.492718 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492720 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:03.492727 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492730 7fe99ebe8700 20 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:03.492740 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 17:38:03.492746 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 17:38:03.492748 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:03.492757 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492758 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492759 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.492768 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.492776 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,50,289] newacting [4,50,289] 2014-08-13 17:38:03.492779 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,314]/[374,12,294] -- 374/374 2014-08-13 17:38:03.492785 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.492795 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492793 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [32,50,289] r=-1 lpr=194482 pi=194476-194481/2 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.492798 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.492808 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.492811 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [32,50,289](32) acting [32,50,289](32)): not rw, up_thru 194482 up_from 192973 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) : primary up 192973-194482 includes interval 2014-08-13 17:38:03.492818 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.492821 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194482/194482/194482) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 17:38:03.492831 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [32,50,289] -> [4,50,289], acting [32,50,289] -> [4,50,289], acting_primary 32 -> 4, up_primary 32 -> 4, role -1 -> 0 2014-08-13 17:38:03.492842 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.492839 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,294,12]/[374,12,294] -- 374/374 2014-08-13 17:38:03.492855 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.492851 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.492857 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.492861 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.492868 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.492871 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2700 2014-08-13 17:38:03.492877 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,294,12] newacting [374,12,294] 2014-08-13 17:38:03.492881 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492887 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.492891 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.492897 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,314]/[374,12,294] r=-1 lpr=194480 pi=192688-194479/44 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194482 2014-08-13 17:38:03.492900 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.492909 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492917 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492915 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [374,314](374) acting [374,12,294](374)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [374,314](374) acting [374,12,294](374) maybe_went_rw) : primary up 192967-194480 includes interval 2014-08-13 17:38:03.492925 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.492925 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194479/194480/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194480-194481 up [374,314](374) acting [374,12,294](374) maybe_went_rw) 2014-08-13 17:38:03.492934 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.492937 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,314] -> [374,294,12], acting [374,12,294] -> [374,12,294], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 17:38:03.492943 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.492949 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.492953 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.492959 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.492963 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.492968 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.492972 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.492979 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:03.492981 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.492988 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:03.492990 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.492998 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.492997 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:03.493007 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.493008 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:03.493015 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.493018 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:03.493024 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493028 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:03.493035 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.493038 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 17:38:03.493046 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,50,289 unchanged 2014-08-13 17:38:03.493047 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:03.493055 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.493057 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493064 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001137 7 0.001028 2014-08-13 17:38:03.493075 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.493079 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,294,12]/[374,294,12] -- 374/374 2014-08-13 17:38:03.493084 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.493094 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493092 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.493097 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.493102 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.493107 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493114 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.493116 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,294,12] newacting [374,294,12] 2014-08-13 17:38:03.493123 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.493126 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.493133 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.493136 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12]/[374,12,294] r=-1 lpr=194482 pi=192688-194481/45 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194483 2014-08-13 17:38:03.493142 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.493153 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 17:38:03.493154 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [374,294,12](374) acting [374,12,294](374)): not rw, up_thru 194480 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [374,294,12](374) acting [374,12,294](374)) : primary up 192967-194480 does not include interval 2014-08-13 17:38:03.493163 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194482/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] noting past interval(194482-194482 up [374,294,12](374) acting [374,12,294](374)) 2014-08-13 17:38:03.493164 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 17:38:03.493175 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 17:38:03.493174 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] up [374,294,12] -> [374,294,12], acting [374,12,294] -> [374,294,12], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 17:38:03.493190 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 17:38:03.493186 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,32,50,289 down blocked_by {} 2014-08-13 17:38:03.493199 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.493201 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.493207 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.493213 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2701 2014-08-13 17:38:03.493217 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:03.493224 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.32 2014-08-13 17:38:03.493226 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:03.493235 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.50 2014-08-13 17:38:03.493238 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:03.493245 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 17:38:03.493249 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 17:38:03.493255 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.493259 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:03.493266 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.493269 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:03.493278 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_recovery 2014-08-13 17:38:03.493287 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:03.493296 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493302 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.493317 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,294,12]/[374,294,12] -- 374/374 2014-08-13 17:38:03.493330 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493333 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.493333 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 17:38:03.493343 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493352 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493373 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493369 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,314] -- 374/374 2014-08-13 17:38:03.493382 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493377 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: Reset advmap 2014-08-13 17:38:03.493384 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:03.493388 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493398 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] acting_up_affected newup [344,296] newacting [296,320] 2014-08-13 17:38:03.493403 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,314] 2014-08-13 17:38:03.493408 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.493413 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.493419 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [4,344,296]/[4,296,320] r=0 lpr=194478 pi=193679-194476/32 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] set_last_peering_reset 194479 2014-08-13 17:38:03.493422 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,294,12] r=-1 lpr=194483 pi=192688-194482/46 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194485 2014-08-13 17:38:03.493439 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [374,294,12](374) acting [374,294,12](374)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [374,294,12](374) acting [374,294,12](374) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.493440 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,344,296](4) acting [4,296,320](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,344,296](4) acting [4,296,320](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.493448 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194482/194483/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] noting past interval(194483-194484 up [374,294,12](374) acting [374,294,12](374) maybe_went_rw) 2014-08-13 17:38:03.493450 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194476/194477/194477) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] noting past interval(194477-194478 up [4,344,296](4) acting [4,296,320](4) maybe_went_rw) 2014-08-13 17:38:03.493459 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] up [374,294,12] -> [374,4,314], acting [374,294,12] -> [374,4,314], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> 1 2014-08-13 17:38:03.493461 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] up [4,344,296] -> [344,296], acting [4,296,320] -> [296,320], acting_primary 4 -> 296, up_primary 4 -> 344, role 0 -> -1 2014-08-13 17:38:03.493470 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 17:38:03.493472 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] clear_primary_state 2014-08-13 17:38:03.493479 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.493483 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] agent_stop 2014-08-13 17:38:03.493488 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.493493 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] on_change 2014-08-13 17:38:03.493498 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:03.493503 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.493507 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:03.493513 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] requeue_ops 2014-08-13 17:38:03.493516 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:03.493524 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] cancel_copy_ops 2014-08-13 17:38:03.493525 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 17:38:03.493533 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] cancel_flush_ops 2014-08-13 17:38:03.493535 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:03.493542 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] on_change_cleanup 2014-08-13 17:38:03.493544 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:03.493553 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_role_change 2014-08-13 17:38:03.493552 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] on_change 2014-08-13 17:38:03.493562 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.493563 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] exit NotTrimming 2014-08-13 17:38:03.493570 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_recovery 2014-08-13 17:38:03.493573 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped m=47] enter NotTrimming 2014-08-13 17:38:03.493579 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:03.493583 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_stats 2014-08-13 17:38:03.493589 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493593 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.493599 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map 2014-08-13 17:38:03.493602 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_role_change 2014-08-13 17:38:03.493611 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.493619 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 17:38:03.493627 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 17:38:03.493637 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493652 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] update_heartbeat_peers unchanged 2014-08-13 17:38:03.493656 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 17:38:03.493663 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] take_waiters 2014-08-13 17:38:03.493672 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493673 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Reset 0.001456 7 0.001273 2014-08-13 17:38:03.493675 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.493684 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started 2014-08-13 17:38:03.493685 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493695 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Start 2014-08-13 17:38:03.493696 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493704 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: transitioning to Stray 2014-08-13 17:38:03.493714 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.493714 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 17:38:03.493728 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493726 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started/Stray 2014-08-13 17:38:03.493730 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.493737 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.493740 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493747 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.493750 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.493774 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[296,320] -- 320/296 2014-08-13 17:38:03.493788 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493790 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.493799 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.493813 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.493807 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [320,307,55] newacting [296,320] 2014-08-13 17:38:03.493817 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.493825 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [344,296]/[296,320] r=-1 lpr=194479 pi=193679-194478/33 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194482 2014-08-13 17:38:03.493843 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [344,296](344) acting [296,320](296)): not rw, up_thru 194479 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [344,296](344) acting [296,320](296) maybe_went_rw) : primary up 192967-194479 includes interval 2014-08-13 17:38:03.493853 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194479/194479/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194479-194481 up [344,296](344) acting [296,320](296) maybe_went_rw) 2014-08-13 17:38:03.493863 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [344,296] -> [320,307,55], acting [296,320] -> [296,320], acting_primary 296 -> 296, up_primary 344 -> 320, role -1 -> -1 2014-08-13 17:38:03.493864 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 17:38:03.493881 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.493874 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.493884 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.493884 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.493893 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.493894 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,20] newacting [328,20] 2014-08-13 17:38:03.493903 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.493903 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 17:38:03.493913 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 17:38:03.493914 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.298096 3 0.000184 2014-08-13 17:38:03.493923 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 17:38:03.493925 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.298191 0 0.000000 2014-08-13 17:38:03.493932 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.493936 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.493942 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 17:38:03.493944 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194477 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.493952 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 17:38:03.493953 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194479 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.493961 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 17:38:03.493962 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194479 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.493972 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194479 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,20] newacting [328,20] 2014-08-13 17:38:03.493982 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194479 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.493989 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,4,20] r=1 lpr=194479 pi=192975-194475/22 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.493971 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 17:38:03.493999 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494006 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [328,4,20](328) acting [328,4,20](328)): not rw, up_thru 194476 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [328,4,20](328) acting [328,4,20](328) maybe_went_rw) : primary up 192949-194476 includes interval 2014-08-13 17:38:03.494015 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194476/194476/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [328,4,20](328) acting [328,4,20](328) maybe_went_rw) 2014-08-13 17:38:03.494024 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,4,20] -> [328,20], acting [328,4,20] -> [328,20], acting_primary 328 -> 328, up_primary 328 -> 328, role 1 -> -1 2014-08-13 17:38:03.494029 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[320,307,296] -- 320/320 2014-08-13 17:38:03.494035 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494044 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494043 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.494048 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.494051 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.494058 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.494060 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.494069 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.494067 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [320,307,55] newacting [320,307,296] 2014-08-13 17:38:03.494077 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.494077 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.494086 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494087 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[296,320] r=-1 lpr=194482 pi=193679-194481/34 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194483 2014-08-13 17:38:03.494096 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.494104 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.494105 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [320,307,55](320) acting [296,320](296)): not rw, up_thru 194479 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [320,307,55](320) acting [296,320](296)) : primary up 192967-194479 does not include interval 2014-08-13 17:38:03.494112 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.494115 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194482/194479) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194482-194482 up [320,307,55](320) acting [296,320](296)) 2014-08-13 17:38:03.494121 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.494129 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.494126 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [320,307,55] -> [320,307,55], acting [296,320] -> [320,307,296], acting_primary 296 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 17:38:03.494137 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.494139 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.494146 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494148 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.494157 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.494167 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 17:38:03.494166 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 17:38:03.494178 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494177 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 17:38:03.494186 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 17:38:03.494181 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494200 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.494203 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494210 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 17:38:03.494213 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494220 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 17:38:03.494232 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 17:38:03.494243 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494229 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] [296,320] -> [320,307,296], acting primary 296 -> 320 2014-08-13 17:38:03.494245 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 17:38:03.494245 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494255 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494254 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 17:38:03.494264 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494265 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494287 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 17:38:03.494297 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494299 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494303 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[320,307,296] -- 320/320 2014-08-13 17:38:03.494308 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494320 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494317 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,10,20] newacting [328,10,20] 2014-08-13 17:38:03.494323 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.494326 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.494334 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,20] r=-1 lpr=194479 pi=192975-194478/23 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.494333 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.494345 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494351 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [328,20](328) acting [328,20](328)): not rw, up_thru 194479 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [328,20](328) acting [328,20](328) maybe_went_rw) : primary up 192949-194479 includes interval 2014-08-13 17:38:03.494361 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194479/194479/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [328,20](328) acting [328,20](328) maybe_went_rw) 2014-08-13 17:38:03.494364 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [4,344,296]/[320,307,296] -- 4/320 2014-08-13 17:38:03.494378 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494371 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,20] -> [328,10,20], acting [328,20] -> [328,10,20], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> -1 2014-08-13 17:38:03.494381 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:03.494384 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494391 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.494392 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.494400 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.494400 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [320,307,296] 2014-08-13 17:38:03.494410 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.494411 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.494418 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.494421 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [320,307,55]/[320,307,296] r=-1 lpr=194483 pi=193679-194482/35 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194485 2014-08-13 17:38:03.494427 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.494436 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494440 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [320,307,55](320) acting [320,307,296](320)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) : primary up 192979-194483 includes interval 2014-08-13 17:38:03.494445 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.494450 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194482/194483/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194483-194484 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) 2014-08-13 17:38:03.494454 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.494463 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.494471 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.494480 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494462 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [320,307,55] -> [4,344,296], acting [320,307,296] -> [320,307,296], acting_primary 320 -> 320, up_primary 320 -> 4, role -1 -> -1 2014-08-13 17:38:03.494490 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.494499 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.494500 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 17:38:03.494511 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494508 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.494514 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494519 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 17:38:03.494523 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494528 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 17:38:03.494532 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494539 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 17:38:03.494548 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:03.494551 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 17:38:03.494562 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494558 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 17:38:03.494564 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494569 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 17:38:03.494578 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 17:38:03.494587 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 17:38:03.494573 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494597 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494601 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494607 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_activate_map 2014-08-13 17:38:03.494620 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 17:38:03.494630 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.494632 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.494640 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.494646 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] update_heartbeat_peers unchanged 2014-08-13 17:38:03.494648 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,4,20] newacting [328,4,20] 2014-08-13 17:38:03.494657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.494656 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] take_waiters 2014-08-13 17:38:03.494666 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,10,20] r=-1 lpr=194482 pi=192975-194481/24 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.494667 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Reset 0.612100 9 0.001069 2014-08-13 17:38:03.494679 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started 2014-08-13 17:38:03.494682 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [328,10,20](328) acting [328,10,20](328)): not rw, up_thru 194482 up_from 192949 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [328,10,20](328) acting [328,10,20](328) maybe_went_rw) : primary up 192949-194482 includes interval 2014-08-13 17:38:03.494689 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Start 2014-08-13 17:38:03.494691 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194482/194482/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [328,10,20](328) acting [328,10,20](328) maybe_went_rw) 2014-08-13 17:38:03.494699 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: transitioning to Stray 2014-08-13 17:38:03.494701 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,10,20] -> [328,4,20], acting [328,10,20] -> [328,4,20], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> 1 2014-08-13 17:38:03.494712 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494710 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Start 0.000021 0 0.000000 2014-08-13 17:38:03.494720 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.494722 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started/Stray 2014-08-13 17:38:03.494728 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.494735 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.494737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.494745 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.494745 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.494754 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.494762 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.494770 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.494779 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.494786 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.494794 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.494801 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.494801 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.494808 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.494817 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.494827 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.494865 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.494875 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.494879 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing 2014-08-13 17:38:03.494884 7fe99f5e9700 7 osd.4 194485 do_notify osd 30 on 1 PGs 2014-08-13 17:38:03.494884 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.000947 7 0.000795 2014-08-13 17:38:03.494895 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.494903 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:03.494905 7fe99f5e9700 7 osd.4 194485 do_notify osd 320 on 1 PGs 2014-08-13 17:38:03.494903 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.494911 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.494917 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:03.494921 7fe99f5e9700 7 osd.4 194485 do_notify osd 337 on 1 PGs 2014-08-13 17:38:03.494922 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.494932 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:03.494934 7fe99f5e9700 7 osd.4 194485 do_notify osd 356 on 1 PGs 2014-08-13 17:38:03.494933 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.494947 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:03.494948 7fe99f5e9700 7 osd.4 194485 do_notify osd 371 on 1 PGs 2014-08-13 17:38:03.494945 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.494954 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.494962 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e48840 don't know epoch, doing nothing 2014-08-13 17:38:03.494964 7fe99f5e9700 7 osd.4 194485 do_notify osd 397 on 1 PGs 2014-08-13 17:38:03.494977 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d33180 don't know epoch, doing nothing 2014-08-13 17:38:03.494979 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.32 on 1 PGs 2014-08-13 17:38:03.494989 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing 2014-08-13 17:38:03.494991 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.50 on 2 PGs 2014-08-13 17:38:03.495000 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.495003 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing 2014-08-13 17:38:03.495004 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.55 on 1 PGs 2014-08-13 17:38:03.495015 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:03.495017 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.61 on 1 PGs 2014-08-13 17:38:03.495147 7fe99ebe8700 10 osd.4 pg_epoch: 194478 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 17:38:03.495169 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495171 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing 2014-08-13 17:38:03.495180 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.65 on 1 PGs 2014-08-13 17:38:03.495173 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 17:38:03.495182 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319] newacting [45,319] 2014-08-13 17:38:03.495197 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.495207 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:03.495210 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.289 on 1 PGs 2014-08-13 17:38:03.495208 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.299062 3 0.000161 2014-08-13 17:38:03.495222 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:03.495217 7fe97a600700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.495224 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.356 on 1 PGs 2014-08-13 17:38:03.495219 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started 0.299125 0 0.000000 2014-08-13 17:38:03.495230 7fe99ebe8700 5 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 17:38:03.495239 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194477 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194479 2014-08-13 17:38:03.495248 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194479 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.495256 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194479 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.495264 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194479 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319] newacting [45,319] 2014-08-13 17:38:03.495273 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194479 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.495281 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319,4] r=2 lpr=194479 pi=193186-194475/26 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194479 2014-08-13 17:38:03.495299 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [45,319,4](45) acting [45,319,4](45)): not rw, up_thru 194476 up_from 192974 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [45,319,4](45) acting [45,319,4](45) maybe_went_rw) : primary up 192974-194476 includes interval 2014-08-13 17:38:03.495308 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194476/194476/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194476-194478 up [45,319,4](45) acting [45,319,4](45) maybe_went_rw) 2014-08-13 17:38:03.495318 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319,4] -> [45,319], acting [45,319,4] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role 2 -> -1 2014-08-13 17:38:03.495328 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.495323 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 17:38:03.495337 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.495348 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495345 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.495352 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.495355 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.495363 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.495363 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.495373 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.495373 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,52] newacting [15,61] 2014-08-13 17:38:03.495382 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.495384 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.495392 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.495393 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,4,52]/[15,4,61] r=1 lpr=194478 pi=193264-194476/30 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194479 2014-08-13 17:38:03.495402 7fe99ebe8700 20 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.495410 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 17:38:03.495414 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [15,4,52](15) acting [15,4,61](15)): not rw, up_thru 194477 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [15,4,52](15) acting [15,4,61](15) maybe_went_rw) : primary up 192969-194477 includes interval 2014-08-13 17:38:03.495418 7fe99ebe8700 15 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.495427 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.495425 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194476/194477/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194477-194478 up [15,4,52](15) acting [15,4,61](15) maybe_went_rw) 2014-08-13 17:38:03.495435 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.495436 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,4,52] -> [15,52], acting [15,4,61] -> [15,61], acting_primary 15 -> 15, up_primary 15 -> 15, role 1 -> -1 2014-08-13 17:38:03.495446 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.495449 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.495459 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.495467 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.495470 7fe99ebe8700 10 osd.4 pg_epoch: 194479 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 17:38:03.495484 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495478 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.495489 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.495490 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.495499 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.495500 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.495508 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.495509 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.495519 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.495529 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.495531 7fe99ebe8700 10 osd.4 pg_epoch: 194480 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 17:38:03.495544 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495538 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 17:38:03.495546 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.495547 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.495555 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.495557 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.495565 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.495566 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.495577 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.495585 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.495588 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.495590 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102140 pg_notify(22.6bd(2) epoch 194485) v5 2014-08-13 17:38:03.495585 7fe99ebe8700 10 osd.4 pg_epoch: 194481 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,26]/[45,319,26] -- 45/45 2014-08-13 17:38:03.495600 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495598 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 17:38:03.501190 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.32 2014-08-13 17:38:03.501192 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102140 2014-08-13 17:38:03.501197 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.495603 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.501206 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501226 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501228 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501222 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501214 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.501234 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501246 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501237 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319,26] newacting [45,319,26] 2014-08-13 17:38:03.501247 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501249 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e3c0 pg_notify(24.136 epoch 194485) v5 2014-08-13 17:38:03.501247 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501257 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319] r=-1 lpr=194479 pi=193186-194478/27 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194482 2014-08-13 17:38:03.501258 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 17:38:03.501272 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.501274 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e3c0 2014-08-13 17:38:03.501277 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501278 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [45,319](45) acting [45,319](45)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [45,319](45) acting [45,319](45) maybe_went_rw) : primary up 192974-194479 includes interval 2014-08-13 17:38:03.501280 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501294 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501296 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501288 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194479/194479/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194479-194481 up [45,319](45) acting [45,319](45) maybe_went_rw) 2014-08-13 17:38:03.501291 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501299 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319] -> [45,319,26], acting [45,319] -> [45,319,26], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 17:38:03.501301 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501310 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501312 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501313 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260700 pg_notify(22.6bd(25) epoch 194485) v5 2014-08-13 17:38:03.501311 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.501321 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.501325 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,61,379]/[15,61] -- 15/15 2014-08-13 17:38:03.501330 7fe99ebe8700 15 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.501339 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.289 2014-08-13 17:38:03.501340 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260700 2014-08-13 17:38:03.501340 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501341 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.501343 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501350 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.501354 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501363 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501365 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501360 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.501364 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [15,61,379] newacting [15,61] 2014-08-13 17:38:03.501370 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.501384 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501375 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501386 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501387 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4264d00 pg_notify(24.136 epoch 194485) v5 2014-08-13 17:38:03.501381 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.501386 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,52]/[15,61] r=-1 lpr=194479 pi=193264-194478/31 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194482 2014-08-13 17:38:03.501392 7fe99ebe8700 20 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.501401 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.501410 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.55 2014-08-13 17:38:03.501411 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4264d00 2014-08-13 17:38:03.501410 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.501407 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [15,52](15) acting [15,61](15)): not rw, up_thru 194479 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194479-194481 up [15,52](15) acting [15,61](15) maybe_went_rw) : primary up 192969-194479 includes interval 2014-08-13 17:38:03.501421 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501430 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501432 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501422 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194479/194479/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194479-194481 up [15,52](15) acting [15,61](15) maybe_went_rw) 2014-08-13 17:38:03.501442 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501443 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501434 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,52] -> [15,61,379], acting [15,61] -> [15,61], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> -1 2014-08-13 17:38:03.501444 7fe9a63f4700 20 osd.4 194485 _dispatch 0x50f8000 pg_notify(22.6bd(2) epoch 194485) v5 2014-08-13 17:38:03.501445 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.501449 7fe99ebe8700 10 osd.4 pg_epoch: 194482 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,26]/[45,319] -- 45/45 2014-08-13 17:38:03.501455 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.501466 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.50 2014-08-13 17:38:03.501468 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x50f8000 2014-08-13 17:38:03.501467 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501466 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.501471 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.501477 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.501487 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501489 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501481 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.501488 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.501492 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319,26] newacting [45,319] 2014-08-13 17:38:03.501500 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501502 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501503 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e580 pg_notify(24.136(30) epoch 194485) v5 2014-08-13 17:38:03.501499 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.501502 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501509 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:03.501512 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26] r=-1 lpr=194482 pi=193186-194481/28 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194483 2014-08-13 17:38:03.501526 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.65 2014-08-13 17:38:03.501528 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e580 2014-08-13 17:38:03.501521 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.501531 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.501530 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [45,319,26](45) acting [45,319,26](45)): not rw, up_thru 194479 up_from 192974 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [45,319,26](45) acting [45,319,26](45)) : primary up 192974-194479 does not include interval 2014-08-13 17:38:03.501541 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.501541 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194482/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] noting past interval(194482-194482 up [45,319,26](45) acting [45,319,26](45)) 2014-08-13 17:38:03.501556 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501558 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501551 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.501553 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] up [45,319,26] -> [45,319,26], acting [45,319,26] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 17:38:03.501563 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501565 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.501573 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501574 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501576 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143480 pg_notify(24.136(30) epoch 194485) v5 2014-08-13 17:38:03.501574 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.501583 7fe99ebe8700 15 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.501587 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,61,379]/[15,61,379] -- 15/15 2014-08-13 17:38:03.501598 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.356 2014-08-13 17:38:03.501600 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143480 2014-08-13 17:38:03.501602 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501593 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.501605 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501609 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.501616 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501626 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501628 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501619 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.501626 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [15,61,379] newacting [15,61,379] 2014-08-13 17:38:03.501630 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.501643 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501645 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501637 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501646 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102f40 pg_notify(24.136(1) epoch 194485) v5 2014-08-13 17:38:03.501640 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.501647 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379]/[15,61] r=-1 lpr=194482 pi=193264-194481/32 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194483 2014-08-13 17:38:03.501651 7fe99ebe8700 20 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.501668 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.50 2014-08-13 17:38:03.501661 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.501670 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102f40 2014-08-13 17:38:03.501667 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [15,61,379](15) acting [15,61](15)): not rw, up_thru 194479 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [15,61,379](15) acting [15,61](15)) : primary up 192969-194479 does not include interval 2014-08-13 17:38:03.501671 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.501678 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194482/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] noting past interval(194482-194482 up [15,61,379](15) acting [15,61](15)) 2014-08-13 17:38:03.501683 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501692 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.501694 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.501689 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] up [15,61,379] -> [15,61,379], acting [15,61] -> [15,61,379], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> -1 2014-08-13 17:38:03.501701 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.501707 7fe99ebe8700 10 osd.4 pg_epoch: 194483 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,26]/[45,319] -- 45/45 2014-08-13 17:38:03.501710 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.501722 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501718 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.501725 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.501727 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.501735 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.501736 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.501745 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.501745 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501754 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.501763 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.501765 7fe99ebe8700 10 osd.4 pg_epoch: 194484 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,4]/[45,319] -- 45/45 2014-08-13 17:38:03.501772 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.501778 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501781 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.501784 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.501790 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.501794 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.501800 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501804 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319] 2014-08-13 17:38:03.501814 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501822 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,26]/[45,319] r=-1 lpr=194483 pi=193186-194482/29 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194485 2014-08-13 17:38:03.501823 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,61,379]/[15,61,379] -- 15/15 2014-08-13 17:38:03.501837 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501840 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [45,319,26](45) acting [45,319](45)): not rw, up_thru 194483 up_from 192974 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [45,319,26](45) acting [45,319](45) maybe_went_rw) : primary up 192974-194483 includes interval 2014-08-13 17:38:03.501840 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501850 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501849 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194482/194483/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] noting past interval(194483-194484 up [45,319,26](45) acting [45,319](45) maybe_went_rw) 2014-08-13 17:38:03.501860 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501861 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] up [45,319,26] -> [45,319,4], acting [45,319] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 17:38:03.501872 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.501878 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,52] -- 15/15 2014-08-13 17:38:03.501882 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.501891 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.501891 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.501894 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:03.501901 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.501904 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.501910 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.501914 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,52] 2014-08-13 17:38:03.501920 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.501924 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.501929 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.501933 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,61,379] r=-1 lpr=194483 pi=193264-194482/33 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194485 2014-08-13 17:38:03.501940 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.501950 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.501950 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [15,61,379](15) acting [15,61,379](15)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [15,61,379](15) acting [15,61,379](15) maybe_went_rw) : primary up 192969-194483 includes interval 2014-08-13 17:38:03.501959 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.501960 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194482/194483/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] noting past interval(194483-194484 up [15,61,379](15) acting [15,61,379](15) maybe_went_rw) 2014-08-13 17:38:03.501969 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.501971 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] up [15,61,379] -> [15,4,52], acting [15,61,379] -> [15,4,52], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> 1 2014-08-13 17:38:03.501979 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.501983 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.501990 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_activate_map 2014-08-13 17:38:03.501992 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.502000 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.502013 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:03.502020 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:03.502028 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:03.502029 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:03.502037 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 17:38:03.502040 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] take_waiters 2014-08-13 17:38:03.502047 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:03.502050 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.006820 7 0.006629 2014-08-13 17:38:03.502056 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:03.502063 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started 2014-08-13 17:38:03.502066 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_role_change 2014-08-13 17:38:03.502074 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.502074 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Start 2014-08-13 17:38:03.502085 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 17:38:03.502086 7fe99ebe8700 1 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:03.502093 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:03.502099 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000026 0 0.000000 2014-08-13 17:38:03.502104 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.502111 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:03.502114 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:03.502124 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.502134 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.502159 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 17:38:03.502169 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:03.502179 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.514097 9 0.001021 2014-08-13 17:38:03.502190 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 17:38:03.502203 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502205 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 17:38:03.502213 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 17:38:03.502224 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.502224 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.502235 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 17:38:03.502240 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.502250 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502247 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.502255 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 58 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.502257 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.502271 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.502306 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502316 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.502311 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 58 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.502328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194474/194466 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.58 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.502347 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338]/[31,338] -- 31/31 2014-08-13 17:38:03.502364 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.502368 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.502377 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.502377 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.58 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.502386 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] acting_up_affected newup [31,338] newacting [31,338] 2014-08-13 17:38:03.502391 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,61,349 -> 4,58,61,349 2014-08-13 17:38:03.502395 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.502402 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.502404 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338,4] r=2 lpr=194478 pi=193125-194476/38 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194479 2014-08-13 17:38:03.502406 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.502420 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 17:38:03.502423 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [31,338,4](31) acting [31,338,4](31)): not rw, up_thru 194477 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194477-194478 up [31,338,4](31) acting [31,338,4](31) maybe_went_rw) : primary up 192978-194477 includes interval 2014-08-13 17:38:03.502432 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194476/194477/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] noting past interval(194477-194478 up [31,338,4](31) acting [31,338,4](31) maybe_went_rw) 2014-08-13 17:38:03.502433 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 17:38:03.502443 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] up [31,338,4] -> [31,338], acting [31,338,4] -> [31,338], acting_primary 31 -> 31, up_primary 31 -> 31, role 2 -> -1 2014-08-13 17:38:03.502443 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,330,331,349 down blocked_by {} 2014-08-13 17:38:03.502455 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:03.502456 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.502465 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.502468 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: have osd.58 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.502473 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.502481 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.61 2014-08-13 17:38:03.502483 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_copy_ops 2014-08-13 17:38:03.502490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.330 2014-08-13 17:38:03.502492 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_flush_ops 2014-08-13 17:38:03.502499 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.331 2014-08-13 17:38:03.502501 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change_cleanup 2014-08-13 17:38:03.502507 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.349 2014-08-13 17:38:03.502510 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:03.502520 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit NotTrimming 2014-08-13 17:38:03.502535 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502529 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter NotTrimming 2014-08-13 17:38:03.502537 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_role_change 2014-08-13 17:38:03.502545 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.502553 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_recovery 2014-08-13 17:38:03.502551 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194477/194474 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 304 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.502561 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] clear_recovery_state 2014-08-13 17:38:03.502566 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] sending info 2014-08-13 17:38:03.502572 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.502590 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338]/[31,338] -- 31/31 2014-08-13 17:38:03.502603 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.502608 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502606 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.502615 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.502613 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194477/194477 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 9 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.502625 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.502626 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.502642 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338]/[31,338] -- 31/31 2014-08-13 17:38:03.502654 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.502657 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502656 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.502662 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 340 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.502669 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.502675 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.502679 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.502692 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 17:38:03.502699 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,374] -- 31/31 2014-08-13 17:38:03.502701 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.502712 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.502711 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 17:38:03.502715 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.502724 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.502723 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 17:38:03.502733 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] acting_up_affected newup [31,338,374] newacting [31,338,374] 2014-08-13 17:38:03.502734 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,340,347 down blocked_by {} 2014-08-13 17:38:03.502742 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.502744 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.502751 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338] r=-1 lpr=194479 pi=193125-194478/39 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194482 2014-08-13 17:38:03.502755 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.7 2014-08-13 17:38:03.502764 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: have osd.340 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.502768 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [31,338](31) acting [31,338](31)): not rw, up_thru 194479 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194479-194481 up [31,338](31) acting [31,338](31) maybe_went_rw) : primary up 192978-194479 includes interval 2014-08-13 17:38:03.502775 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.347 2014-08-13 17:38:03.502777 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194479/194479/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] noting past interval(194479-194481 up [31,338](31) acting [31,338](31) maybe_went_rw) 2014-08-13 17:38:03.502787 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] up [31,338] -> [31,338,374], acting [31,338] -> [31,338,374], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 17:38:03.502801 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502798 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:03.502807 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.502815 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.502816 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 28 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.502823 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_copy_ops 2014-08-13 17:38:03.502832 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_flush_ops 2014-08-13 17:38:03.502831 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.502841 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change_cleanup 2014-08-13 17:38:03.502847 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,28,304 unchanged 2014-08-13 17:38:03.502849 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:03.502857 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.502859 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit NotTrimming 2014-08-13 17:38:03.502868 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter NotTrimming 2014-08-13 17:38:03.502867 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 17:38:03.502877 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_recovery 2014-08-13 17:38:03.502879 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 17:38:03.502886 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] clear_recovery_state 2014-08-13 17:38:03.502889 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,28,304 down blocked_by {} 2014-08-13 17:38:03.502895 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.502900 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.502910 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.7 2014-08-13 17:38:03.502916 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 17:38:03.502918 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: have osd.28 info 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.502930 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.502929 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.304 2014-08-13 17:38:03.502933 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.502942 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.502957 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.502951 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] acting_up_affected newup [31,338,374] newacting [31,338,16] 2014-08-13 17:38:03.502960 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.502968 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374] r=-1 lpr=194482 pi=193125-194481/40 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194483 2014-08-13 17:38:03.502972 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 328 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.502986 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.502985 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [31,338,374](31) acting [31,338,374](31)): not rw, up_thru 194479 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194482-194482 up [31,338,374](31) acting [31,338,374](31)) : primary up 192978-194479 does not include interval 2014-08-13 17:38:03.502995 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194482/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] noting past interval(194482-194482 up [31,338,374](31) acting [31,338,374](31)) 2014-08-13 17:38:03.503005 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] up [31,338,374] -> [31,338,374], acting [31,338,374] -> [31,338,16], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 17:38:03.503016 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 17:38:03.503028 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.503025 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.503033 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.503033 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 301 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.503044 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_copy_ops 2014-08-13 17:38:03.503048 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194475/194466 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.503053 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_flush_ops 2014-08-13 17:38:03.503062 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change_cleanup 2014-08-13 17:38:03.503072 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 17:38:03.503069 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] osd.301 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.503083 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit NotTrimming 2014-08-13 17:38:03.503083 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,286,375 -> 4,286,301,375 2014-08-13 17:38:03.503094 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.503093 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter NotTrimming 2014-08-13 17:38:03.503096 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.503103 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_recovery 2014-08-13 17:38:03.503107 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 17:38:03.503112 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] clear_recovery_state 2014-08-13 17:38:03.503120 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 17:38:03.503123 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.503131 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 17:38:03.503141 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.503144 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 17:38:03.503158 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.503151 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.10 2014-08-13 17:38:03.503161 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.286 2014-08-13 17:38:03.503161 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.503171 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.503170 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: have osd.301 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.503181 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.503182 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.375 2014-08-13 17:38:03.503207 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,16] -- 31/31 2014-08-13 17:38:03.503222 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.503225 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:03.503231 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.503234 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:03.503249 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.503243 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,16] 2014-08-13 17:38:03.503254 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e3d800 don't know epoch, doing nothing 2014-08-13 17:38:03.503256 7fe99ebe8700 7 osd.4 194485 do_notify osd 6 on 1 PGs 2014-08-13 17:38:03.503253 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.503262 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,374]/[31,338,16] r=-1 lpr=194483 pi=193125-194482/41 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194485 2014-08-13 17:38:03.503272 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e2aa0 don't know epoch, doing nothing 2014-08-13 17:38:03.503275 7fe99ebe8700 7 osd.4 194485 do_notify osd 9 on 1 PGs 2014-08-13 17:38:03.503287 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:03.503280 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [31,338,374](31) acting [31,338,16](31)): not rw, up_thru 194483 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194483-194484 up [31,338,374](31) acting [31,338,16](31) maybe_went_rw) : primary up 192978-194483 includes interval 2014-08-13 17:38:03.503289 7fe99ebe8700 7 osd.4 194485 do_notify osd 45 on 1 PGs 2014-08-13 17:38:03.503289 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194482/194483/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] noting past interval(194483-194484 up [31,338,374](31) acting [31,338,16](31) maybe_went_rw) 2014-08-13 17:38:03.503301 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing 2014-08-13 17:38:03.503303 7fe99ebe8700 7 osd.4 194485 do_notify osd 51 on 1 PGs 2014-08-13 17:38:03.503300 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] up [31,338,374] -> [31,338,4], acting [31,338,16] -> [31,338,16], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 17:38:03.503315 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:03.503317 7fe99ebe8700 7 osd.4 194485 do_notify osd 58 on 1 PGs 2014-08-13 17:38:03.503312 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 17:38:03.503321 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.503330 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:03.503332 7fe99ebe8700 7 osd.4 194485 do_notify osd 304 on 1 PGs 2014-08-13 17:38:03.503330 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.503344 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:03.503346 7fe99ebe8700 7 osd.4 194485 do_notify osd 328 on 2 PGs 2014-08-13 17:38:03.503340 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_copy_ops 2014-08-13 17:38:03.503349 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_flush_ops 2014-08-13 17:38:03.503357 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c70f20 already has epoch 194485 2014-08-13 17:38:03.503359 7fe99ebe8700 7 osd.4 194485 do_notify osd 345 on 1 PGs 2014-08-13 17:38:03.503358 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change_cleanup 2014-08-13 17:38:03.503367 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 17:38:03.503376 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit NotTrimming 2014-08-13 17:38:03.503385 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter NotTrimming 2014-08-13 17:38:03.503394 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_recovery 2014-08-13 17:38:03.503402 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] clear_recovery_state 2014-08-13 17:38:03.503412 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.503422 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_activate_map 2014-08-13 17:38:03.503468 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:03.503473 7fe99ebe8700 7 osd.4 194485 do_notify osd 374 on 1 PGs 2014-08-13 17:38:03.503466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] update_heartbeat_peers unchanged 2014-08-13 17:38:03.503476 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] take_waiters 2014-08-13 17:38:03.503487 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.503489 7fe99ebe8700 7 osd.4 194485 do_notify osd 376 on 1 PGs 2014-08-13 17:38:03.503486 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Reset 0.578868 9 0.000931 2014-08-13 17:38:03.503499 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecb700 don't know epoch, doing nothing 2014-08-13 17:38:03.503501 7fe99ebe8700 7 osd.4 194485 do_notify osd 396 on 1 PGs 2014-08-13 17:38:03.503498 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started 2014-08-13 17:38:03.503513 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5250580 don't know epoch, doing nothing 2014-08-13 17:38:03.503514 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.7 on 1 PGs 2014-08-13 17:38:03.503508 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Start 2014-08-13 17:38:03.503517 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: transitioning to Stray 2014-08-13 17:38:03.503527 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503530 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.10 on 1 PGs 2014-08-13 17:38:03.503528 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.503543 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503545 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.286 on 1 PGs 2014-08-13 17:38:03.503539 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started/Stray 2014-08-13 17:38:03.503551 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.503561 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.503619 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.503722 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:03.503728 7fe99f5e9700 7 osd.4 194485 do_notify osd 15 on 1 PGs 2014-08-13 17:38:03.503725 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec89a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503731 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.293 on 1 PGs 2014-08-13 17:38:03.503749 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:03.503751 7fe99f5e9700 7 osd.4 194485 do_notify osd 31 on 1 PGs 2014-08-13 17:38:03.503785 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 17:38:03.503803 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.503808 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 17:38:03.503817 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336] newacting [285,336] 2014-08-13 17:38:03.503825 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.503836 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d789a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503841 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.338 on 1 PGs 2014-08-13 17:38:03.503835 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 0.314682 3 0.000143 2014-08-13 17:38:03.503847 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started 0.314733 0 0.000000 2014-08-13 17:38:03.503856 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503858 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.340 on 1 PGs 2014-08-13 17:38:03.503858 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 17:38:03.503868 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 don't know epoch, doing nothing 2014-08-13 17:38:03.503870 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.343 on 1 PGs 2014-08-13 17:38:03.503867 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194477 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194479 2014-08-13 17:38:03.503876 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.503885 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.503893 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336] newacting [285,336] 2014-08-13 17:38:03.503901 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.503909 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336,4] r=2 lpr=194479 pi=193125-194475/26 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194479 2014-08-13 17:38:03.503928 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [285,336,4](285) acting [285,336,4](285)): not rw, up_thru 194476 up_from 192975 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [285,336,4](285) acting [285,336,4](285) maybe_went_rw) : primary up 192975-194476 includes interval 2014-08-13 17:38:03.503936 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing 2014-08-13 17:38:03.503941 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.347 on 1 PGs 2014-08-13 17:38:03.503937 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194476/194476/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194476-194478 up [285,336,4](285) acting [285,336,4](285) maybe_went_rw) 2014-08-13 17:38:03.503947 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336,4] -> [285,336], acting [285,336,4] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role 2 -> -1 2014-08-13 17:38:03.503956 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:03.503958 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.373 on 1 PGs 2014-08-13 17:38:03.503958 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.503971 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.503973 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.376 on 1 PGs 2014-08-13 17:38:03.503967 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.503976 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.503983 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecb700 don't know epoch, doing nothing 2014-08-13 17:38:03.503985 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.396 on 1 PGs 2014-08-13 17:38:03.503985 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.503993 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.504001 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.504012 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.504021 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.504024 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194477/194477 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 54 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.504030 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.504039 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 17:38:03.504040 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.504047 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.504055 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.504063 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.504076 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504072 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504082 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 291 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.504094 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 17:38:03.504096 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194474/194466 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504108 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504112 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504112 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.291 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504118 7fe97a3fe700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.504123 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504129 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,367,378 -> 4,291,367,378 2014-08-13 17:38:03.504132 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504140 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.504142 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.504152 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 17:38:03.504166 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504168 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504152 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 17:38:03.504177 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504191 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504193 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504195 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260000 pg_notify(22.3d5(2) epoch 194485) v5 2014-08-13 17:38:03.504186 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504197 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [365,350,291](365) acting [291,350](291)) 2014-08-13 17:38:03.504209 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 17:38:03.504211 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,365]/[285,336,365] -- 285/285 2014-08-13 17:38:03.504226 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504228 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.10 2014-08-13 17:38:03.504230 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260000 2014-08-13 17:38:03.504223 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.504229 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504238 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504236 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: have osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504249 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.350 2014-08-13 17:38:03.504248 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336,365] newacting [285,336,365] 2014-08-13 17:38:03.504263 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504265 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504259 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.365 2014-08-13 17:38:03.504260 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.504268 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.367 2014-08-13 17:38:03.504270 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336] r=-1 lpr=194479 pi=193125-194478/27 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194482 2014-08-13 17:38:03.504280 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504281 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504283 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16c80 pg_notify(22.1b8(25) epoch 194485) v5 2014-08-13 17:38:03.504277 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.378 2014-08-13 17:38:03.504287 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [285,336](285) acting [285,336](285)): not rw, up_thru 194479 up_from 192975 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [285,336](285) acting [285,336](285) maybe_went_rw) : primary up 192975-194479 includes interval 2014-08-13 17:38:03.504305 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.7 2014-08-13 17:38:03.504297 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194479/194479/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194479-194481 up [285,336](285) acting [285,336](285) maybe_went_rw) 2014-08-13 17:38:03.504308 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504307 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16c80 2014-08-13 17:38:03.504308 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336] -> [285,336,365], acting [285,336] -> [285,336,365], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 17:38:03.504319 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.504329 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504331 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504324 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 345 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.504328 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.504337 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.504338 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.504347 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.504355 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.504363 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.504373 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504371 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:03.504380 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.504381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 20 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.504389 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.504398 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.504396 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194474/194466 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504407 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.504412 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.20 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504417 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504427 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,25,362 -> 4,20,25,362 2014-08-13 17:38:03.504436 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.504438 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.504437 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,365]/[285,336] -- 285/285 2014-08-13 17:38:03.504451 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504449 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 17:38:03.504454 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504461 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 17:38:03.504463 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504472 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 17:38:03.504473 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336,365] newacting [285,336] 2014-08-13 17:38:03.504487 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.504487 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.504502 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365] r=-1 lpr=194482 pi=193125-194481/28 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194483 2014-08-13 17:38:03.504502 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: have osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504515 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.25 2014-08-13 17:38:03.504519 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [285,336,365](285) acting [285,336,365](285)): not rw, up_thru 194479 up_from 192975 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [285,336,365](285) acting [285,336,365](285)) : primary up 192975-194479 does not include interval 2014-08-13 17:38:03.504525 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.303 2014-08-13 17:38:03.504530 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194482/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] noting past interval(194482-194482 up [285,336,365](285) acting [285,336,365](285)) 2014-08-13 17:38:03.504540 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] up [285,336,365] -> [285,336,365], acting [285,336,365] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 17:38:03.504551 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.504534 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.362 2014-08-13 17:38:03.504564 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504569 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504561 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.504571 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5102d80 pg_notify(22.3d5(25) epoch 194485) v5 2014-08-13 17:38:03.504568 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.373 2014-08-13 17:38:03.504572 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.504582 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.504595 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.286 2014-08-13 17:38:03.504597 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5102d80 2014-08-13 17:38:03.504592 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.504603 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.504613 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504612 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.504621 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504623 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504622 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.504631 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.504629 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 20 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.504640 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.504642 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194475/194465 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.20 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504649 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.504660 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504663 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.20 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504676 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,31,374 -> 4,20,31,374 2014-08-13 17:38:03.504685 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.504681 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,365]/[285,336] -- 285/285 2014-08-13 17:38:03.504687 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.504696 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504697 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 17:38:03.504699 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504709 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504709 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) 2014-08-13 17:38:03.504720 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504721 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,20,31,337,363,374 down blocked_by {} 2014-08-13 17:38:03.504732 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.504739 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,4]/[285,336] -- 285/285 2014-08-13 17:38:03.504742 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: have osd.20 info 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.504752 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.504758 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504760 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504753 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.31 2014-08-13 17:38:03.504761 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4260380 pg_notify(22.3d5(2) epoch 194485) v5 2014-08-13 17:38:03.504755 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:03.504762 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.337 2014-08-13 17:38:03.504765 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.504772 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.363 2014-08-13 17:38:03.504785 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.343 2014-08-13 17:38:03.504776 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336] 2014-08-13 17:38:03.504786 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4260380 2014-08-13 17:38:03.504782 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.374 2014-08-13 17:38:03.504787 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.504798 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,365]/[285,336] r=-1 lpr=194483 pi=193125-194482/29 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194485 2014-08-13 17:38:03.504809 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504809 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504811 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504822 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504815 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [285,336,365](285) acting [285,336](285)): not rw, up_thru 194483 up_from 192975 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [285,336,365](285) acting [285,336](285) maybe_went_rw) : primary up 192975-194483 includes interval 2014-08-13 17:38:03.504823 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504825 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cb800 pg_notify(22.1b8(2) epoch 194485) v5 2014-08-13 17:38:03.504824 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194482/194483/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] noting past interval(194483-194484 up [285,336,365](285) acting [285,336](285) maybe_went_rw) 2014-08-13 17:38:03.504823 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194477/194477 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 321 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.504839 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.504837 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] up [285,336,365] -> [285,336,4], acting [285,336] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 17:38:03.504849 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.340 2014-08-13 17:38:03.504851 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cb800 2014-08-13 17:38:03.504850 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.504859 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.504872 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504874 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504875 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504868 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.504885 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504887 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504879 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:03.504888 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cde80 pg_notify(22.3d9(49) epoch 194485) v5 2014-08-13 17:38:03.504880 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194477/194466 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 374 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.504888 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:03.504893 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] sending info 2014-08-13 17:38:03.504899 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:03.504912 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.293 2014-08-13 17:38:03.504913 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cde80 2014-08-13 17:38:03.504909 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 17:38:03.504920 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:03.504934 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504935 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504929 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:03.504939 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 17:38:03.504951 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.504947 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:03.504963 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.504964 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.504957 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.504966 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51401c0 pg_notify(22.3d9(42) epoch 194485) v5 2014-08-13 17:38:03.504956 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194477/194477 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 294 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.504968 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_activate_map 2014-08-13 17:38:03.504970 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.504987 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.373 2014-08-13 17:38:03.504989 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51401c0 2014-08-13 17:38:03.505008 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505008 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505011 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505008 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:03.505014 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194477/194466 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 364 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.505019 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] take_waiters 2014-08-13 17:38:03.505032 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505034 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505026 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] sending info 2014-08-13 17:38:03.505036 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51440c0 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.505030 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.001173 7 0.000993 2014-08-13 17:38:03.505043 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started 2014-08-13 17:38:03.505052 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Start 2014-08-13 17:38:03.505066 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505061 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:03.505071 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 45 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.505082 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.396 2014-08-13 17:38:03.505072 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.505083 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51440c0 2014-08-13 17:38:03.505086 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:03.505085 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.45 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505106 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505107 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505099 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.505100 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.45 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505110 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.505113 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,313,318 -> 4,45,313,318 2014-08-13 17:38:03.505123 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.505127 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505128 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505129 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cd400 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.505125 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.505135 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 17:38:03.505151 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.338 2014-08-13 17:38:03.505153 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cd400 2014-08-13 17:38:03.505147 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) 2014-08-13 17:38:03.505158 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,303,313,318,342 down blocked_by {} 2014-08-13 17:38:03.505173 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505174 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505169 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.505181 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.505180 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: have osd.45 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505197 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 17:38:03.505206 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.313 2014-08-13 17:38:03.505214 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.318 2014-08-13 17:38:03.505212 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 17:38:03.505222 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 17:38:03.505229 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.505234 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.505244 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359] newacting [323,359] 2014-08-13 17:38:03.505252 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.505267 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505261 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.315824 3 0.000136 2014-08-13 17:38:03.505272 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.315877 0 0.000000 2014-08-13 17:38:03.505284 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.505284 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 55 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.505293 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194477 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.505299 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194474/194466 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.55 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505303 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.505312 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.505315 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.55 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505320 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359] newacting [323,359] 2014-08-13 17:38:03.505329 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.505328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,339 -> 4,45,55,339 2014-08-13 17:38:03.505338 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.505337 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359,4] r=2 lpr=194479 pi=192978-194475/21 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.505340 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.505350 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 17:38:03.505355 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [323,359,4](323) acting [323,359,4](323)): not rw, up_thru 194476 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [323,359,4](323) acting [323,359,4](323) maybe_went_rw) : primary up 192978-194476 includes interval 2014-08-13 17:38:03.505361 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) 2014-08-13 17:38:03.505365 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194476/194476/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [323,359,4](323) acting [323,359,4](323) maybe_went_rw) 2014-08-13 17:38:03.505372 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,24,45,55,339,342 down blocked_by {} 2014-08-13 17:38:03.505375 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359,4] -> [323,359], acting [323,359,4] -> [323,359], acting_primary 323 -> 323, up_primary 323 -> 323, role 2 -> -1 2014-08-13 17:38:03.505383 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.505386 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.505395 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.505394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.24 2014-08-13 17:38:03.505404 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.505404 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.45 2014-08-13 17:38:03.505413 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.505413 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: have osd.55 info 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505422 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.505425 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.339 2014-08-13 17:38:03.505431 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.505434 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 17:38:03.505440 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.505450 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.505461 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505459 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.505467 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.505474 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.505484 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505486 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505475 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194477/194474 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 17 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.505487 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc0c0 pg_notify(22.1b8(25) epoch 194485) v5 2014-08-13 17:38:03.505481 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.505490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] sending info 2014-08-13 17:38:03.505492 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.505503 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.505513 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.347 2014-08-13 17:38:03.505515 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc0c0 2014-08-13 17:38:03.505523 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 17:38:03.505534 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.505535 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505536 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505537 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505536 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.505541 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 337 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.505546 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.505555 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.505563 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505554 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.337 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505566 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505567 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51439c0 pg_notify(22.3d9(45) epoch 194485) v5 2014-08-13 17:38:03.505570 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.337 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505574 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 17:38:03.505586 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.505591 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.505582 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,374 -> 4,20,31,337,374 2014-08-13 17:38:03.505592 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51439c0 2014-08-13 17:38:03.505593 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.505596 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.505589 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.505598 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.505600 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 44 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.505615 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.505607 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.505618 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.505613 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194475/194466 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.44 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505630 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 17:38:03.505641 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.505632 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.44 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.505643 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.505645 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,62,65 -> 4,44,62,65 2014-08-13 17:38:03.505655 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.505652 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.505656 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.505661 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359,364] newacting [323,359,364] 2014-08-13 17:38:03.505669 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.505667 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 17:38:03.511390 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 17:38:03.511403 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,48,62,65,314 down blocked_by {} 2014-08-13 17:38:03.505678 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359] r=-1 lpr=194479 pi=192978-194478/22 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.511413 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.511424 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: have osd.44 info 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.511434 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.48 2014-08-13 17:38:03.511443 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.62 2014-08-13 17:38:03.511440 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [323,359](323) acting [323,359](323)): not rw, up_thru 194479 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [323,359](323) acting [323,359](323) maybe_went_rw) : primary up 192978-194479 includes interval 2014-08-13 17:38:03.511451 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.65 2014-08-13 17:38:03.511456 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194479/194479/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [323,359](323) acting [323,359](323) maybe_went_rw) 2014-08-13 17:38:03.511460 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.314 2014-08-13 17:38:03.511467 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359] -> [323,359,364], acting [323,359] -> [323,359,364], acting_primary 323 -> 323, up_primary 323 -> 323, role -1 -> -1 2014-08-13 17:38:03.511478 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.511489 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.511487 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.511495 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.511504 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.511503 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 343 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.511513 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.511518 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194477/194477 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.511522 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.511531 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.511534 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 17:38:03.511540 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.511545 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.511550 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.511555 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 17:38:03.511564 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.511567 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 17:38:03.511572 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.511577 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,10,286,343 down blocked_by {} 2014-08-13 17:38:03.511586 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.511587 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.511597 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.10 2014-08-13 17:38:03.511606 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.286 2014-08-13 17:38:03.511612 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 17:38:03.511614 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: have osd.343 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.511625 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.511628 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.511641 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.511638 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.511647 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.511657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 15 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.511666 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 17:38:03.511677 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.511673 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194475/194466 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.15 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.511679 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.511688 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.511688 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.15 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.511697 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.511701 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,21,310 -> 4,15,21,310 2014-08-13 17:38:03.511711 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.511714 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 17:38:03.511716 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.511724 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.511727 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.511725 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 17:38:03.511735 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.511737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 17:38:03.511744 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359,4] newacting [323,359,4] 2014-08-13 17:38:03.511748 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,15,21,303,310,342 down blocked_by {} 2014-08-13 17:38:03.511753 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.511758 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.511763 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,364] r=-1 lpr=194482 pi=192978-194481/23 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.511770 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: have osd.15 info 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.511780 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [323,359,364](323) acting [323,359,364](323)): not rw, up_thru 194482 up_from 192978 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [323,359,364](323) acting [323,359,364](323) maybe_went_rw) : primary up 192978-194482 includes interval 2014-08-13 17:38:03.511781 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.21 2014-08-13 17:38:03.511790 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 17:38:03.511789 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194482/194482/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [323,359,364](323) acting [323,359,364](323) maybe_went_rw) 2014-08-13 17:38:03.511799 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.310 2014-08-13 17:38:03.511800 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359,364] -> [323,359,4], acting [323,359,364] -> [323,359,4], acting_primary 323 -> 323, up_primary 323 -> 323, role -1 -> 2 2014-08-13 17:38:03.511808 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 17:38:03.511811 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.511820 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.511828 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.511835 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.511836 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.511844 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.511852 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.511849 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 338 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.511860 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.511863 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.511869 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.511878 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.511879 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376 unchanged 2014-08-13 17:38:03.511886 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.511889 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.511894 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.511899 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 17:38:03.511903 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.511911 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.511911 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) 2014-08-13 17:38:03.511921 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.511922 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,338,373,376,396 down blocked_by {} 2014-08-13 17:38:03.511931 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.511933 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.511943 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: dropping osd.293 from info_requested, no longer in probe set 2014-08-13 17:38:03.511952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: have osd.338 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.511962 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.373 2014-08-13 17:38:03.511969 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.511971 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.376 2014-08-13 17:38:03.511979 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.511979 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.396 2014-08-13 17:38:03.511989 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.006705 7 0.006550 2014-08-13 17:38:03.512004 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512001 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.512009 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.512017 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.512017 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 358 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.512028 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 17:38:03.512030 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.512039 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.512051 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.512060 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.512076 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512081 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 19 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.512095 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] sending info 2014-08-13 17:38:03.512108 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.512113 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512117 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 362 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.512129 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.362 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512143 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.362 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512155 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343 -> 4,58,343,362 2014-08-13 17:38:03.512164 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.512166 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.512160 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [44,362]/[44,362] -- 44/44 2014-08-13 17:38:03.512176 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.512180 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.512176 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 17:38:03.512193 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) 2014-08-13 17:38:03.512189 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.512203 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,343,349,362 down blocked_by {} 2014-08-13 17:38:03.512206 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [44,362] newacting [44,362] 2014-08-13 17:38:03.512214 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.512215 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.512224 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [4,44,362] r=0 lpr=194478 pi=193306-194476/27 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.512224 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.58 2014-08-13 17:38:03.512235 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.61 2014-08-13 17:38:03.512244 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.343 2014-08-13 17:38:03.512245 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,44,362](4) acting [4,44,362](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.512253 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.349 2014-08-13 17:38:03.512255 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194476/194477/194477) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] noting past interval(194477-194478 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) 2014-08-13 17:38:03.512261 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: have osd.362 info 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512265 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] up [4,44,362] -> [44,362], acting [4,44,362] -> [44,362], acting_primary 4 -> 44, up_primary 4 -> 44, role 0 -> -1 2014-08-13 17:38:03.512276 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.512285 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.512291 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512293 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.512305 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.512301 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.512310 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:03.512312 7fe99ebe8700 7 osd.4 194485 do_notify osd 17 on 1 PGs 2014-08-13 17:38:03.512309 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.512318 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.512329 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d7b020 don't know epoch, doing nothing 2014-08-13 17:38:03.512331 7fe99ebe8700 7 osd.4 194485 do_notify osd 19 on 1 PGs 2014-08-13 17:38:03.512325 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.512333 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.512342 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a4f380 don't know epoch, doing nothing 2014-08-13 17:38:03.512344 7fe99ebe8700 7 osd.4 194485 do_notify osd 54 on 1 PGs 2014-08-13 17:38:03.512341 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.512356 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cfb2e0 don't know epoch, doing nothing 2014-08-13 17:38:03.512350 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.512358 7fe99ebe8700 7 osd.4 194485 do_notify osd 294 on 1 PGs 2014-08-13 17:38:03.512359 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.512370 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c73f40 don't know epoch, doing nothing 2014-08-13 17:38:03.512372 7fe99ebe8700 7 osd.4 194485 do_notify osd 321 on 1 PGs 2014-08-13 17:38:03.512367 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.512382 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c70f20 already has epoch 194485 2014-08-13 17:38:03.512375 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.512384 7fe99ebe8700 7 osd.4 194485 do_notify osd 345 on 1 PGs 2014-08-13 17:38:03.512384 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.512392 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.512399 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.512406 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.512415 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.512434 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [44,362]/[44,362,61] -- 44/44 2014-08-13 17:38:03.512445 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.512447 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.512455 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.512463 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] acting_up_affected newup [44,362] newacting [44,362,61] 2014-08-13 17:38:03.512471 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.512478 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d7c8e0 don't know epoch, doing nothing 2014-08-13 17:38:03.512483 7fe99ebe8700 7 osd.4 194485 do_notify osd 358 on 1 PGs 2014-08-13 17:38:03.512479 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362] r=-1 lpr=194479 pi=193306-194478/28 crt=191727'457 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.512497 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing 2014-08-13 17:38:03.512499 7fe99ebe8700 7 osd.4 194485 do_notify osd 364 on 1 PGs 2014-08-13 17:38:03.512495 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [44,362](44) acting [44,362](44)): not rw, up_thru 194476 up_from 192926 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [44,362](44) acting [44,362](44)) : primary up 192926-194476 does not include interval 2014-08-13 17:38:03.512511 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:03.512512 7fe99ebe8700 7 osd.4 194485 do_notify osd 374 on 1 PGs 2014-08-13 17:38:03.512505 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194479/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [44,362](44) acting [44,362](44)) 2014-08-13 17:38:03.512515 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] up [44,362] -> [44,362], acting [44,362] -> [44,362,61], acting_primary 44 -> 44, up_primary 44 -> 44, role -1 -> -1 2014-08-13 17:38:03.512526 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.512534 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.512542 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.512551 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.512547 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 286 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.512560 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.512562 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.512569 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.512577 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.512586 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 17:38:03.512587 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.512597 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512596 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.512602 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194477/194477 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 51 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.512605 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.512613 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.512614 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.512624 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.512641 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [44,362]/[44,362,61] -- 44/44 2014-08-13 17:38:03.512652 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512652 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.512655 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.512657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 347 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.512664 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.512670 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.512674 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.512685 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 17:38:03.512695 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512698 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [61,369,339]/[44,362,61] -- 61/44 2014-08-13 17:38:03.512700 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 24 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.512709 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.512711 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.512711 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.24 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512720 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.512725 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.24 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512729 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,369,339] newacting [44,362,61] 2014-08-13 17:38:03.512737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,55,339 -> 4,24,45,55,339 2014-08-13 17:38:03.512739 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.512747 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.512751 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512748 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [44,362]/[44,362,61] r=-1 lpr=194480 pi=193306-194479/29 crt=191727'457 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.512755 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 363 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.512765 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [44,362](44) acting [44,362,61](44)): not rw, up_thru 194480 up_from 192926 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [44,362](44) acting [44,362,61](44) maybe_went_rw) : primary up 192926-194480 includes interval 2014-08-13 17:38:03.512766 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.363 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512774 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194479/194480/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [44,362](44) acting [44,362,61](44) maybe_went_rw) 2014-08-13 17:38:03.512779 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.363 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.512787 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] up [44,362] -> [61,369,339], acting [44,362,61] -> [44,362,61], acting_primary 44 -> 44, up_primary 44 -> 61, role -1 -> -1 2014-08-13 17:38:03.512791 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,374 -> 4,20,31,337,363,374 2014-08-13 17:38:03.512802 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.512804 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512799 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.512808 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.512808 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194477/194466 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 346 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.512817 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.512821 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] sending info 2014-08-13 17:38:03.512827 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.512836 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.512844 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.512857 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512852 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.512861 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.512862 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 334 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.512871 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.512874 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.512880 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.512889 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.512899 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.512898 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.512903 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.512919 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194474/194465 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.512921 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [61,369,339]/[61,369,339] -- 61/61 2014-08-13 17:38:03.512932 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.512934 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,324 unchanged 2014-08-13 17:38:03.512935 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.512944 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.512945 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.512954 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,369,339] newacting [61,369,339] 2014-08-13 17:38:03.512954 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 17:38:03.512965 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.512966 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 17:38:03.512974 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339]/[44,362,61] r=-1 lpr=194482 pi=193306-194481/30 crt=191727'457 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.512977 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,302,324,368 down blocked_by {} 2014-08-13 17:38:03.512987 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.512991 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)): not rw, up_thru 194480 up_from 192926 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) : primary up 192926-194480 does not include interval 2014-08-13 17:38:03.512998 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.301 2014-08-13 17:38:03.513000 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194482/194479) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 17:38:03.513007 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.302 2014-08-13 17:38:03.513010 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] up [61,369,339] -> [61,369,339], acting [44,362,61] -> [61,369,339], acting_primary 44 -> 61, up_primary 61 -> 61, role -1 -> -1 2014-08-13 17:38:03.513015 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: have osd.324 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.513021 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.513027 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.368 2014-08-13 17:38:03.513030 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.513038 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.513054 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513047 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.513056 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.513064 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.513071 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.513071 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 6 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.513080 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.513085 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] sending info 2014-08-13 17:38:03.513090 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.513098 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] [44,362,61] -> [61,369,339], acting primary 44 -> 61 2014-08-13 17:38:03.513111 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.513119 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.513133 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513127 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.513138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.513149 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.303 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513150 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [61,369,339]/[61,369,339] -- 61/61 2014-08-13 17:38:03.513163 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.513164 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513166 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.513182 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.513177 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,310 -> 4,15,21,303,310 2014-08-13 17:38:03.513192 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513196 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513191 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.513200 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485)) 2014-08-13 17:38:03.513212 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) 2014-08-13 17:38:03.513214 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 17:38:03.513225 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.513228 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.513236 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.513244 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,44,362] newacting [4,44,362] 2014-08-13 17:38:03.513252 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.513260 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [61,369,339] r=-1 lpr=194483 pi=193306-194482/31 crt=191727'457 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.513259 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376 unchanged 2014-08-13 17:38:03.513271 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513276 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 367 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.513277 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [61,369,339](61) acting [61,369,339](61)): not rw, up_thru 194483 up_from 192938 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) : primary up 192938-194483 includes interval 2014-08-13 17:38:03.513287 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.513287 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194482/194483/194483) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 17:38:03.513298 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [61,369,339] -> [4,44,362], acting [61,369,339] -> [4,44,362], acting_primary 61 -> 4, up_primary 61 -> 4, role -1 -> 0 2014-08-13 17:38:03.513308 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.513316 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.513323 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513325 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.513328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 331 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.513335 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2680 2014-08-13 17:38:03.513341 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.331 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.513346 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.513355 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.513356 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.331 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.513364 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.513368 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,349 -> 4,58,61,331,349 2014-08-13 17:38:03.513378 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513372 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.513381 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513381 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.513385 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 373 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.513389 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.513398 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.513397 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513407 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.513416 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.513413 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.373 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513425 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.513426 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,362 -> 4,20,25,362,373 2014-08-13 17:38:03.513436 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513440 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513434 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.513442 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.513444 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194478/194465 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 320 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.513450 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.513456 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.513458 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.513467 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.513474 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.513489 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513483 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.513492 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.513494 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.513503 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,44,362 2014-08-13 17:38:03.513512 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513506 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.342 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513514 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.513520 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513524 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.830973 9 0.001212 2014-08-13 17:38:03.513532 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339 -> 4,24,45,55,339,342 2014-08-13 17:38:03.513535 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.513543 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513546 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513544 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.513552 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.513550 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 0 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.513563 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.513563 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194475/194466 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.0 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513574 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.513578 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.0 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513584 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.513590 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,56,353 -> 0,4,56,353 2014-08-13 17:38:03.513600 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513594 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.513602 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.513604 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.513612 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 17:38:03.513614 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 17:38:03.513624 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 17:38:03.513627 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 17:38:03.513635 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,4,56,63,353,369 down blocked_by {} 2014-08-13 17:38:03.513637 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [44,362](44) acting [44,362,61](44) maybe_went_rw) 2014-08-13 17:38:03.513645 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.513648 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [44,362](44) acting [44,362](44)) 2014-08-13 17:38:03.513656 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: have osd.0 info 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.513658 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) 2014-08-13 17:38:03.513667 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.56 2014-08-13 17:38:03.513669 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,44,362](4) acting [44,362,61](44)) 2014-08-13 17:38:03.513676 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.63 2014-08-13 17:38:03.513679 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [44,362](44) acting [44,362,61](44) maybe_went_rw) 2014-08-13 17:38:03.513685 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.353 2014-08-13 17:38:03.513690 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [44,362](44) acting [44,362](44)) 2014-08-13 17:38:03.513694 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 17:38:03.513721 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513701 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,61,339,362,369 down blocked_by {} 2014-08-13 17:38:03.513733 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.513737 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 293 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) 2014-08-13 17:38:03.513745 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2681 2014-08-13 17:38:03.513750 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.293 22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) 2014-08-13 17:38:03.513756 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 17:38:03.513767 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 17:38:03.513765 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.293 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) 2014-08-13 17:38:03.513777 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 17:38:03.513787 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 17:38:03.513795 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 17:38:03.513777 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376 -> 4,293,338,376 2014-08-13 17:38:03.513804 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.513809 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513804 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.513814 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.513814 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194478/194466 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 330 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.513827 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] sending info 2014-08-13 17:38:03.513872 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.513873 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.513879 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 312 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.513892 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194474/194465 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.513906 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,312 unchanged 2014-08-13 17:38:03.513903 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_advance_map [287,309]/[287,309] -- 287/287 2014-08-13 17:38:03.513919 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.513916 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.513923 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:03.513926 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 17:38:03.513933 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 17:38:03.513938 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 17:38:03.513942 7fe99f5e9700 1 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 17:38:03.513949 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,311,312,369 down blocked_by {} 2014-08-13 17:38:03.513953 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.322962 2 0.000309 2014-08-13 17:38:03.513959 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.513965 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:03.513970 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.285 2014-08-13 17:38:03.513974 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.322994 0 0.000000 2014-08-13 17:38:03.513979 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.311 2014-08-13 17:38:03.513987 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: have osd.312 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.513989 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.323021 0 0.000000 2014-08-13 17:38:03.513998 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 17:38:03.514000 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started 0.323108 0 0.000000 2014-08-13 17:38:03.514011 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 17:38:03.514024 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514019 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194477 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.514029 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194479 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.514038 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.514042 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e3d800 don't know epoch, doing nothing 2014-08-13 17:38:03.514044 7fe99ebe8700 7 osd.4 194485 do_notify osd 6 on 1 PGs 2014-08-13 17:38:03.514037 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194479 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.514046 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194479 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [287,309] newacting [287,309] 2014-08-13 17:38:03.514058 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing 2014-08-13 17:38:03.514059 7fe99ebe8700 7 osd.4 194485 do_notify osd 51 on 1 PGs 2014-08-13 17:38:03.514055 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194479 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.514063 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [4,287,309] r=0 lpr=194479 pi=193306-194475/42 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.514072 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:03.514073 7fe99ebe8700 7 osd.4 194485 do_notify osd 320 on 1 PGs 2014-08-13 17:38:03.514085 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:03.514086 7fe99ebe8700 7 osd.4 194485 do_notify osd 330 on 1 PGs 2014-08-13 17:38:03.514081 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [4,287,309](4) acting [4,287,309](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194304 generate_past_intervals interval(194476-194478 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.514097 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e49ce0 don't know epoch, doing nothing 2014-08-13 17:38:03.514090 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194476/194476/194476) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] noting past interval(194476-194478 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 17:38:03.514098 7fe99ebe8700 7 osd.4 194485 do_notify osd 334 on 1 PGs 2014-08-13 17:38:03.514099 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] up [4,287,309] -> [287,309], acting [4,287,309] -> [287,309], acting_primary 4 -> 287, up_primary 4 -> 287, role 0 -> -1 2014-08-13 17:38:03.514110 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4a100 don't know epoch, doing nothing 2014-08-13 17:38:03.514111 7fe99ebe8700 7 osd.4 194485 do_notify osd 346 on 1 PGs 2014-08-13 17:38:03.514110 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.514119 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.514126 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:03.514128 7fe99ebe8700 7 osd.4 194485 do_notify osd 367 on 1 PGs 2014-08-13 17:38:03.514127 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.514136 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.514143 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.514152 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.514159 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.514158 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 11 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.514168 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.514176 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.514174 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194475/194466 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.11 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514186 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.514200 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.514202 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.11 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514208 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.514215 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,47 -> 4,5,11,47 2014-08-13 17:38:03.514217 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.514225 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514225 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.514227 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.514233 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.514241 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.514240 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 17:38:03.514249 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.514252 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) 2014-08-13 17:38:03.514259 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.514263 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,5,11,47,54,368 down blocked_by {} 2014-08-13 17:38:03.514274 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.514279 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [287,309]/[287,309,303] -- 287/287 2014-08-13 17:38:03.514291 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.514284 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.5 2014-08-13 17:38:03.514293 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.514294 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: have osd.11 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514303 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.514306 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.47 2014-08-13 17:38:03.514312 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [287,309] newacting [287,309,303] 2014-08-13 17:38:03.514315 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.54 2014-08-13 17:38:03.514321 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.514323 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.368 2014-08-13 17:38:03.514330 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309] r=-1 lpr=194479 pi=193306-194478/43 crt=191727'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.514346 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [287,309](287) acting [287,309](287)): not rw, up_thru 194473 up_from 192975 last_epoch_clean 194304 generate_past_intervals interval(194479-194479 up [287,309](287) acting [287,309](287)) : primary up 192975-194473 does not include interval 2014-08-13 17:38:03.514357 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514355 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194479/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [287,309](287) acting [287,309](287)) 2014-08-13 17:38:03.514365 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] up [287,309] -> [287,309], acting [287,309] -> [287,309,303], acting_primary 287 -> 287, up_primary 287 -> 287, role -1 -> -1 2014-08-13 17:38:03.514375 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.514372 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 365 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.514384 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.514386 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514393 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.514402 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.514401 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.365 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514411 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.514415 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,367,378 -> 4,291,365,367,378 2014-08-13 17:38:03.514425 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514420 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.514430 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514428 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.514437 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.514447 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.514455 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.514463 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.514472 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.514490 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] handle_advance_map [287,309]/[287,309,303] -- 287/287 2014-08-13 17:38:03.514501 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.514503 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.514512 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.514435 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 396 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.514520 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.514525 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.396 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.514543 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] handle_advance_map [303,306,381]/[287,309,303] -- 303/287 2014-08-13 17:38:03.514542 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.396 has stray content: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.514554 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.514557 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.514556 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376 -> 4,293,338,376,396 2014-08-13 17:38:03.514566 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514569 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514566 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.514573 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 314 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.514575 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,306,381] newacting [287,309,303] 2014-08-13 17:38:03.514584 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.514584 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.314 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514595 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [287,309]/[287,309,303] r=-1 lpr=194480 pi=193306-194479/44 crt=191727'463 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.514599 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.314 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514611 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,62,65 -> 4,44,62,65,314 2014-08-13 17:38:03.514612 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [287,309](287) acting [287,309,303](287)): not rw, up_thru 194480 up_from 192975 last_epoch_clean 194304 generate_past_intervals interval(194480-194481 up [287,309](287) acting [287,309,303](287) maybe_went_rw) : primary up 192975-194480 includes interval 2014-08-13 17:38:03.514622 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514624 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514622 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194479/194480/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [287,309](287) acting [287,309,303](287) maybe_went_rw) 2014-08-13 17:38:03.514628 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 315 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.514632 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] up [287,309] -> [303,306,381], acting [287,309,303] -> [287,309,303], acting_primary 287 -> 287, up_primary 287 -> 303, role -1 -> -1 2014-08-13 17:38:03.514644 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.514641 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194475/194466 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514652 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.514661 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.514670 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.514671 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.315 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514679 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.514684 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,28,330 -> 4,28,315,330 2014-08-13 17:38:03.514695 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514688 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.514697 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.514696 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.514707 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.514708 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 17:38:03.514717 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.514721 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 17:38:03.514726 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.514732 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 17:38:03.514735 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.514743 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.514745 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.514754 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.17 2014-08-13 17:38:03.514763 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.28 2014-08-13 17:38:03.514767 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 17:38:03.514779 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.514772 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: have osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514781 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.514784 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.327 2014-08-13 17:38:03.514791 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.514793 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.330 2014-08-13 17:38:03.514800 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,306,381] newacting [303,306,381] 2014-08-13 17:38:03.514809 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.514820 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514818 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381]/[287,309,303] r=-1 lpr=194482 pi=193306-194481/45 crt=191727'463 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.514833 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)): not rw, up_thru 194480 up_from 192975 last_epoch_clean 194304 generate_past_intervals interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) : primary up 192975-194480 does not include interval 2014-08-13 17:38:03.514834 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 311 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.514842 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194482/194479) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 17:38:03.514848 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.311 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514853 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] up [303,306,381] -> [303,306,381], acting [287,309,303] -> [303,306,381], acting_primary 287 -> 303, up_primary 303 -> 303, role -1 -> -1 2014-08-13 17:38:03.514863 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.514863 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] osd.311 has stray content: 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514872 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.514876 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,312 -> 4,285,311,312 2014-08-13 17:38:03.514886 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514881 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.514890 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514890 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.514894 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 48 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.514899 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.514907 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.514907 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.48 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514916 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.514921 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.48 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.514925 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.514934 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.514934 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,62,65,314 -> 4,44,48,62,65,314 2014-08-13 17:38:03.514945 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.514947 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514943 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] [287,309,303] -> [303,306,381], acting primary 287 -> 303 2014-08-13 17:38:03.514951 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 302 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.514957 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.514962 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.302 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514966 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.514976 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.514976 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] osd.302 has stray content: 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.514988 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,324 -> 4,301,302,324 2014-08-13 17:38:03.514997 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515000 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.514998 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 17:38:03.515009 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.515004 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 370 notify: (query_epoch:194485, epoch_sent:194485, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.515017 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.370 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.515011 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.515030 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.370 has stray content: 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.515038 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.515041 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340 -> 4,340,370 2014-08-13 17:38:03.515053 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515048 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.515056 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.515065 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 17:38:03.515067 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.515076 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.515078 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.515077 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.515087 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.515087 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 17:38:03.515097 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,287,309] newacting [4,287,309] 2014-08-13 17:38:03.515099 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.515111 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.324 2014-08-13 17:38:03.515106 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.515132 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [303,306,381] r=-1 lpr=194483 pi=193306-194482/46 crt=191727'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.515120 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.340 2014-08-13 17:38:03.515148 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [303,306,381](303) acting [303,306,381](303)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194304 generate_past_intervals interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) : primary up 192967-194483 includes interval 2014-08-13 17:38:03.515147 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: have osd.370 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.515158 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194482/194483/194483) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 17:38:03.515176 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515168 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,306,381] -> [4,287,309], acting [303,306,381] -> [4,287,309], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 17:38:03.515179 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.515187 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.515193 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 375 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515200 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.515210 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2679 2014-08-13 17:38:03.515208 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515220 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515225 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,286,301,375 unchanged 2014-08-13 17:38:03.515237 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515230 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.515238 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.515241 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 371 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515246 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515254 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194474/194466 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.371 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515263 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515268 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.371 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515272 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.515281 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.515281 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,356,357 -> 4,356,357,371 2014-08-13 17:38:03.515292 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515291 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.515294 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.515300 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.515304 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 17:38:03.515309 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.515316 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 17:38:03.515319 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.515330 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.515328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,352,356,357,371,380 down blocked_by {} 2014-08-13 17:38:03.515339 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.515341 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.515347 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.515352 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.352 2014-08-13 17:38:03.515356 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.515362 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.356 2014-08-13 17:38:03.515366 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.515372 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.357 2014-08-13 17:38:03.515377 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.515381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: have osd.371 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515387 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,287,309 unchanged 2014-08-13 17:38:03.515392 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.380 2014-08-13 17:38:03.515397 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.515406 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001395 7 0.001306 2014-08-13 17:38:03.515417 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515417 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.515425 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.515434 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.515431 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 368 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.515444 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.515444 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.368 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.515455 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.515459 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] osd.368 has stray content: 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.515465 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.515471 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,302,324 -> 4,301,302,324,368 2014-08-13 17:38:03.515474 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.515482 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515486 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515485 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:03.515490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 380 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515496 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 17:38:03.515501 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.380 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515508 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 17:38:03.515515 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.380 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515518 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [287,309](287) acting [287,309,303](287) maybe_went_rw) 2014-08-13 17:38:03.515527 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,356,357,371 -> 4,356,357,371,380 2014-08-13 17:38:03.515537 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515529 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [287,309](287) acting [287,309](287)) 2014-08-13 17:38:03.515539 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515540 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194478 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 17:38:03.515543 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194478/194465 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 351 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.515551 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194475 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 17:38:03.515554 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.515561 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194470-194472 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 17:38:03.515571 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 17:38:03.515580 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.515590 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2680 2014-08-13 17:38:03.515601 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515599 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.287 2014-08-13 17:38:03.515606 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 7 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.515609 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 17:38:03.515619 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.306 2014-08-13 17:38:03.515618 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.7 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.515628 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.309 2014-08-13 17:38:03.515633 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] osd.7 has stray content: 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.515638 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.381 2014-08-13 17:38:03.515645 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,28,304 -> 4,7,28,304 2014-08-13 17:38:03.515655 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515648 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.515658 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515658 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.515662 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 63 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515673 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.63 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515686 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.63 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515697 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,353 -> 0,4,56,63,353 2014-08-13 17:38:03.515706 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515708 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515726 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.515712 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 349 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515738 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.349 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515751 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.349 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515753 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [289,341]/[289,341] -- 289/289 2014-08-13 17:38:03.515768 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.515763 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343,362 -> 4,58,343,349,362 2014-08-13 17:38:03.515773 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515775 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515771 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 17:38:03.515780 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.515779 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 350 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515789 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [289,341] newacting [289,341] 2014-08-13 17:38:03.515790 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515798 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.515806 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [4,289,341] r=0 lpr=194478 pi=192978-194476/25 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194479 2014-08-13 17:38:03.515804 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.350 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515817 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,365,367,378 -> 4,291,350,365,367,378 2014-08-13 17:38:03.515828 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515830 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515825 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,289,341](4) acting [4,289,341](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194466 generate_past_intervals interval(194477-194478 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.515834 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 352 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515834 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194476/194477/194477) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] noting past interval(194477-194478 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) 2014-08-13 17:38:03.515847 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] up [4,289,341] -> [289,341], acting [4,289,341] -> [289,341], acting_primary 4 -> 289, up_primary 4 -> 289, role 0 -> -1 2014-08-13 17:38:03.515847 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.352 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515858 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] clear_primary_state 2014-08-13 17:38:03.515861 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.352 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515867 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] agent_stop 2014-08-13 17:38:03.515873 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,356,357,371,380 -> 4,352,356,357,371,380 2014-08-13 17:38:03.515876 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.515884 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515887 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.515885 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.515893 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.515891 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 366 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.515901 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 17:38:03.515904 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194475/194466 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.366 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515911 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 17:38:03.515919 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] on_change_cleanup 2014-08-13 17:38:03.515918 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.366 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.515927 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] on_change 2014-08-13 17:38:03.515930 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,373 -> 4,344,366,373 2014-08-13 17:38:03.515939 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.515937 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] exit NotTrimming 2014-08-13 17:38:03.515941 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.515945 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive] enter NotTrimming 2014-08-13 17:38:03.515954 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 17:38:03.515951 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 17:38:03.515962 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515964 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 17:38:03.515970 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.515974 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,320,344,363,366,373 down blocked_by {} 2014-08-13 17:38:03.515978 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.515986 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.515985 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.515994 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.515996 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.320 2014-08-13 17:38:03.516003 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516005 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.344 2014-08-13 17:38:03.516013 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.363 2014-08-13 17:38:03.516021 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: have osd.366 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.516022 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [289,341]/[289,341,376] -- 289/289 2014-08-13 17:38:03.516034 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516032 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.373 2014-08-13 17:38:03.516036 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516045 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516054 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] acting_up_affected newup [289,341] newacting [289,341,376] 2014-08-13 17:38:03.516062 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.516070 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341] r=-1 lpr=194479 pi=192978-194478/26 crt=191727'473 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194480 2014-08-13 17:38:03.516078 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516084 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194479 up [289,341](289) acting [289,341](289)): not rw, up_thru 194476 up_from 192968 last_epoch_clean 194466 generate_past_intervals interval(194479-194479 up [289,341](289) acting [289,341](289)) : primary up 192968-194476 does not include interval 2014-08-13 17:38:03.516093 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.516098 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:03.516099 7fe99ebe8700 7 osd.4 194485 do_notify osd 351 on 1 PGs 2014-08-13 17:38:03.516094 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194479/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] noting past interval(194479-194479 up [289,341](289) acting [289,341](289)) 2014-08-13 17:38:03.516104 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] up [289,341] -> [289,341], acting [289,341] -> [289,341,376], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.516114 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.516122 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.516131 7fe99f5e9700 15 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.516139 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.516138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 382 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:03.516148 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.516151 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.382 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.516157 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.516165 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.516167 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.382 has stray content: 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.516174 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.516184 7fe99f5e9700 20 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.516180 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376,396 -> 4,293,338,376,382,396 2014-08-13 17:38:03.516202 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.516207 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516200 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.516209 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.516212 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 10 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.516219 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516224 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.10 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516238 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,341]/[289,341,376] -- 289/289 2014-08-13 17:38:03.516238 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] osd.10 has stray content: 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516250 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516250 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 -> 4,10,286,343 2014-08-13 17:38:03.516260 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.516253 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516262 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516265 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.041170 4 0.000357 2014-08-13 17:38:03.516284 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.516272 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516300 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516311 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.10 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516312 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,327,376]/[289,341,376] -- 289/289 2014-08-13 17:38:03.516324 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516322 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516327 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516333 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516336 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516348 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] acting_up_affected newup [289,327,376] newacting [289,341,376] 2014-08-13 17:38:03.516358 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.516365 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) osd.343 (up) accepted 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) osd.286 (up) accepted 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516367 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,341]/[289,341,376] r=-1 lpr=194480 pi=192978-194479/27 crt=191727'473 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.516377 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,286,343 2014-08-13 17:38:03.516384 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194480-194481 up [289,341](289) acting [289,341,376](289)): not rw, up_thru 194480 up_from 192968 last_epoch_clean 194466 generate_past_intervals interval(194480-194481 up [289,341](289) acting [289,341,376](289) maybe_went_rw) : primary up 192968-194480 includes interval 2014-08-13 17:38:03.516387 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,343,286] (== acting) backfill_targets 2014-08-13 17:38:03.516393 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194479/194480/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] noting past interval(194480-194481 up [289,341](289) acting [289,341,376](289) maybe_went_rw) 2014-08-13 17:38:03.516404 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] up [289,341] -> [289,327,376], acting [289,341,376] -> [289,341,376], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.516414 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.516401 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.516423 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.516432 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.516433 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000149 0 0.000000 2014-08-13 17:38:03.516441 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.516448 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.516449 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.516460 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:03.516459 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: osd.286 has no missing, identical log 2014-08-13 17:38:03.516469 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:03.516472 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: osd.343 has no missing, identical log 2014-08-13 17:38:03.516479 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:03.516481 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.516488 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:03.516493 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000045 0 0.000000 2014-08-13 17:38:03.516497 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 17:38:03.516505 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.516506 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 17:38:03.516515 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516516 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516530 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 10 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.516538 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,327,376]/[289,327,376] -- 289/289 2014-08-13 17:38:03.516550 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516544 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.516553 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516562 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516560 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] osd.10 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.516571 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] acting_up_affected newup [289,327,376] newacting [289,327,376] 2014-08-13 17:38:03.516574 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,286,301,375 -> 4,10,286,301,375 2014-08-13 17:38:03.516584 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.516587 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516581 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.516590 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376]/[289,341,376] r=-1 lpr=194482 pi=192978-194481/28 crt=191727'473 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194483 2014-08-13 17:38:03.516591 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 58 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.516602 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.516606 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)): not rw, up_thru 194480 up_from 192968 last_epoch_clean 194466 generate_past_intervals interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) : primary up 192968-194480 does not include interval 2014-08-13 17:38:03.516615 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194482/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] noting past interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) 2014-08-13 17:38:03.516616 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343,349,362 unchanged 2014-08-13 17:38:03.516627 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516626 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] up [289,327,376] -> [289,327,376], acting [289,341,376] -> [289,327,376], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 17:38:03.516631 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 304 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.516637 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.516642 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516646 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.516655 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.516657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,7,28,304 unchanged 2014-08-13 17:38:03.516663 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.516669 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.156641 4 0.000544 2014-08-13 17:38:03.516672 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.516681 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.516682 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.516689 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.516697 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.516695 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516706 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.516706 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.7 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516716 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.516719 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516724 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.516729 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516734 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516756 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [289,327,376]/[289,327,376] -- 289/289 2014-08-13 17:38:03.516767 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516755 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) osd.28 (up) accepted 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) osd.304 (up) accepted 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.516770 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516771 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,28,304 2014-08-13 17:38:03.516779 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516781 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,28,304] (== acting) backfill_targets 2014-08-13 17:38:03.516788 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.516792 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.516804 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000121 0 0.000000 2014-08-13 17:38:03.516805 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 17:38:03.516816 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.516816 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.516818 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.516825 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: osd.28 has no missing, identical log 2014-08-13 17:38:03.516827 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.516835 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: osd.304 has no missing, identical log 2014-08-13 17:38:03.516836 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,289,341] newacting [4,289,341] 2014-08-13 17:38:03.516845 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.516846 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.516854 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [289,327,376] r=-1 lpr=194483 pi=192978-194482/29 crt=191727'473 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.516857 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000042 0 0.000000 2014-08-13 17:38:03.516868 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.516878 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516871 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [289,327,376](289) acting [289,327,376](289)): not rw, up_thru 194483 up_from 192968 last_epoch_clean 194466 generate_past_intervals interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) : primary up 192968-194483 includes interval 2014-08-13 17:38:03.516880 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194482/194483/194479) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 17:38:03.516889 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [289,327,376] -> [4,289,341], acting [289,327,376] -> [4,289,341], acting_primary 289 -> 4, up_primary 289 -> 4, role -1 -> 0 2014-08-13 17:38:03.516892 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 28 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.516900 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 17:38:03.516905 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.28 2014-08-13 17:38:03.516909 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 17:38:03.516919 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.516915 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.28 info 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.516934 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.516928 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194485:2685 2014-08-13 17:38:03.516938 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.516938 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194477/194477 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 328 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.516948 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.516949 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.516956 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.516964 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.519769 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.519783 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.516982 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.519791 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.519799 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.519808 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.519817 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.519808 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 301 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.519825 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 17:38:03.519834 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.519832 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got dup osd.301 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.519847 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.519842 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 17:38:03.519850 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:03.519852 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 54 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.519859 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 17:38:03.519868 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 17:38:03.519865 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.54 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.519877 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.519881 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.54 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.519888 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 17:38:03.519895 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47 -> 4,5,11,47,54 2014-08-13 17:38:03.519906 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.519898 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,289,341 2014-08-13 17:38:03.519908 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.519909 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.519910 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 17:38:03.519914 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194477/194465 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 345 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.519919 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.821743 9 0.003997 2014-08-13 17:38:03.519926 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.519930 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 17:38:03.519939 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 17:38:03.519947 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 17:38:03.519959 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.519956 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 17:38:03.519966 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 17:38:03.519964 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.519976 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 17:38:03.519976 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.61 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.519985 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.519991 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.61 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.519995 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:03.520004 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343,349,362 -> 4,58,61,343,349,362 2014-08-13 17:38:03.520005 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 17:38:03.520015 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520020 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520017 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) 2014-08-13 17:38:03.520024 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194477/194474 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 45 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.520027 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [289,341](289) acting [289,341,376](289) maybe_went_rw) 2014-08-13 17:38:03.520036 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] sending info 2014-08-13 17:38:03.520042 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [289,341](289) acting [289,341](289)) 2014-08-13 17:38:03.520052 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194477-194478 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) 2014-08-13 17:38:03.520062 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194476-194476 up [4,289,341](4) acting [289,341,376](289)) 2014-08-13 17:38:03.520070 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520071 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194474-194475 up [289,341](289) acting [289,341,376](289) maybe_went_rw) 2014-08-13 17:38:03.520075 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194477/194474 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 289 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.520081 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194473-194473 up [289,341](289) acting [289,341](289)) 2014-08-13 17:38:03.520088 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] sending info 2014-08-13 17:38:03.520091 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,289,327,341,376 down blocked_by {} 2014-08-13 17:38:03.520101 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.520112 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194485:2686 2014-08-13 17:38:03.520121 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 17:38:03.520137 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520134 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.327 2014-08-13 17:38:03.520144 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.341 2014-08-13 17:38:03.520141 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 17 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520154 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 17:38:03.520154 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520165 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.520170 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.17 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520175 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.520184 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,28,315,330 -> 4,17,28,315,330 2014-08-13 17:38:03.520202 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520205 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520209 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 362 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520219 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.362 info 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.520231 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520234 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.520234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 286 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.520245 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.520260 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 17:38:03.520260 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 17:38:03.520276 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520273 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetInfo 0.163591 6 0.010176 2014-08-13 17:38:03.520279 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:03.520286 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.520289 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345] newacting [292,345] 2014-08-13 17:38:03.520297 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.520299 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.4 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520307 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.327308 3 0.000221 2014-08-13 17:38:03.520311 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520318 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started 0.327404 0 0.000000 2014-08-13 17:38:03.520322 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.520328 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:03.520333 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520337 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194477 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.520346 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194479 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520344 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520355 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194479 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520363 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194479 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345] newacting [292,345] 2014-08-13 17:38:03.520372 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194479 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.520376 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting newest update on osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) shard 286 (up) backfill 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.375 (up) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) shard 10 (stray) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520380 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345,4] r=2 lpr=194479 pi=192975-194475/21 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194479 2014-08-13 17:38:03.520389 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] choose_acting want [4,375,10] != acting [4,286,375], requesting pg_temp change 2014-08-13 17:38:03.520398 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [292,345,4](292) acting [292,345,4](292)): not rw, up_thru 194476 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194476-194478 up [292,345,4](292) acting [292,345,4](292) maybe_went_rw) : primary up 192972-194476 includes interval 2014-08-13 17:38:03.520405 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetLog 0.000120 0 0.000000 2014-08-13 17:38:03.520407 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194476/194476/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194476-194478 up [292,345,4](292) acting [292,345,4](292) maybe_went_rw) 2014-08-13 17:38:03.520417 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: Leaving Peering 2014-08-13 17:38:03.520417 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345,4] -> [292,345], acting [292,345,4] -> [292,345], acting_primary 292 -> 292, up_primary 292 -> 292, role 2 -> -1 2014-08-13 17:38:03.520428 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.520427 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering 0.163766 0 0.000000 2014-08-13 17:38:03.520437 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.520439 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.163802 0 0.000000 2014-08-13 17:38:03.520445 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.520451 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary 2014-08-13 17:38:03.520454 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.520462 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.520462 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.520473 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520471 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.520479 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.520478 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 363 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520488 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.520492 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.363 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520497 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.520505 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.520506 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.363 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520513 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.520521 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.520519 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,366,373 -> 4,344,363,366,373 2014-08-13 17:38:03.520529 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520532 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520529 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.520536 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.520538 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520546 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.324 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.520557 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 17:38:03.520568 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520559 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.324 has stray content: 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.520571 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520571 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,370 -> 4,324,340,370 2014-08-13 17:38:03.520581 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520583 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520580 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520588 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520586 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520601 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520605 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 17:38:03.520615 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520618 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520615 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.303 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520627 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520629 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,362,373 -> 4,20,25,303,362,373 2014-08-13 17:38:03.520640 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520643 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520636 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520647 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 327 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520655 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 17:38:03.520665 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520667 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520671 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.327 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520676 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520684 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345,349] newacting [292,345,349] 2014-08-13 17:38:03.520684 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,17,28,315,330 -> 4,17,28,315,327,330 2014-08-13 17:38:03.520695 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520698 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520694 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.520701 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.520705 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:03.520707 7fe99ebe8700 7 osd.4 194485 do_notify osd 45 on 1 PGs 2014-08-13 17:38:03.520702 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345] r=-1 lpr=194479 pi=192975-194478/22 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194482 2014-08-13 17:38:03.520717 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [292,345](292) acting [292,345](292)): not rw, up_thru 194479 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194479-194481 up [292,345](292) acting [292,345](292) maybe_went_rw) : primary up 192972-194479 includes interval 2014-08-13 17:38:03.520726 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:03.520728 7fe99ebe8700 7 osd.4 194485 do_notify osd 289 on 1 PGs 2014-08-13 17:38:03.520726 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194479/194479/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194479-194481 up [292,345](292) acting [292,345](292) maybe_went_rw) 2014-08-13 17:38:03.520740 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:03.520742 7fe99ebe8700 7 osd.4 194485 do_notify osd 328 on 1 PGs 2014-08-13 17:38:03.520736 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345] -> [292,345,349], acting [292,345] -> [292,345,349], acting_primary 292 -> 292, up_primary 292 -> 292, role -1 -> -1 2014-08-13 17:38:03.520752 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4c70f20 already has epoch 194485 2014-08-13 17:38:03.520746 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.520754 7fe99ebe8700 7 osd.4 194485 do_notify osd 345 on 1 PGs 2014-08-13 17:38:03.520754 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.520765 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.520763 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.520771 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.520779 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.520787 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.520794 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.520803 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.520804 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194477/194477 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 373 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.520811 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.520817 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.520820 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.520828 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.520836 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520850 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520855 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 17:38:03.520856 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.520867 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520869 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520868 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.342 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520878 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520883 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.520887 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520896 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,313,318 -> 4,45,313,318,342 2014-08-13 17:38:03.520906 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520908 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520906 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 17:38:03.520917 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520912 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 330 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.520919 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520924 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.330 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.520928 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520937 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.520937 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.330 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.520949 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,331,349 -> 4,58,61,330,331,349 2014-08-13 17:38:03.520960 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.520962 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520954 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 17:38:03.520965 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.520967 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:03.520966 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194477/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 396 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.520976 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194477 2014-08-13 17:38:03.520979 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] sending info 2014-08-13 17:38:03.520984 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345,4] newacting [292,345,4] 2014-08-13 17:38:03.520996 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.520993 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.521001 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,349] r=-1 lpr=194482 pi=192975-194481/23 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194485 2014-08-13 17:38:03.521000 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 370 notify: (query_epoch:194485, epoch_sent:194485, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.521012 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194477/194477 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.370 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521017 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194484 up [292,345,349](292) acting [292,345,349](292)): not rw, up_thru 194482 up_from 192972 last_epoch_clean 194477 generate_past_intervals interval(194482-194484 up [292,345,349](292) acting [292,345,349](292) maybe_went_rw) : primary up 192972-194482 includes interval 2014-08-13 17:38:03.521026 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194482/194482/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194482-194484 up [292,345,349](292) acting [292,345,349](292) maybe_went_rw) 2014-08-13 17:38:03.521027 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.370 has stray content: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521037 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345,349] -> [292,345,4], acting [292,345,349] -> [292,345,4], acting_primary 292 -> 292, up_primary 292 -> 292, role -1 -> 2 2014-08-13 17:38:03.521039 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340 -> 4,340,370 2014-08-13 17:38:03.521049 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.521048 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.521051 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.521057 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.521062 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:03.521065 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.521074 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 17:38:03.521074 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:03.521083 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 17:38:03.521085 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 17:38:03.521092 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 17:38:03.521096 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.521100 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 17:38:03.521107 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.324 2014-08-13 17:38:03.521109 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 17:38:03.521116 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.340 2014-08-13 17:38:03.521118 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 17:38:03.521127 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 17:38:03.521125 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: have osd.370 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521135 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.521142 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 17:38:03.521150 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 17:38:03.521159 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521159 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.521168 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:03.521175 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 375 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521195 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 17:38:03.521205 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521207 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 17:38:03.521209 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 368 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.521218 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:03.521221 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.368 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521227 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.000899 7 0.000756 2014-08-13 17:38:03.521235 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.368 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521241 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 17:38:03.521250 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 17:38:03.521248 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,54 -> 4,5,11,47,54,368 2014-08-13 17:38:03.521259 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.521263 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521258 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 17:38:03.521267 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 380 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521268 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 17:38:03.521279 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 17:38:03.521279 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.380 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521291 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521292 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.521295 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.521302 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.521305 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.369 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521319 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.521331 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,311,312 -> 4,285,311,312,369 2014-08-13 17:38:03.521340 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.521342 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521348 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.521346 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 7 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.521356 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.7 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521369 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] osd.7 has stray content: 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521373 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 17:38:03.521381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 -> 4,7,340,347 2014-08-13 17:38:03.521390 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.521392 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.521394 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.038777 4 0.000345 2014-08-13 17:38:03.521395 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Started advmap 2014-08-13 17:38:03.521406 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.521405 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45] newacting [343,45] 2014-08-13 17:38:03.521416 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:03.521425 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 0.327984 3 0.000184 2014-08-13 17:38:03.521436 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started 0.328051 0 0.000000 2014-08-13 17:38:03.521435 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521447 7fe99f5e9700 5 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset 2014-08-13 17:38:03.521449 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.7 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521456 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194477 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194479 2014-08-13 17:38:03.521460 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521466 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194479 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.521471 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521475 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194479 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.521484 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194479 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45] newacting [343,45] 2014-08-13 17:38:03.521493 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194479 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.521501 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45,4] r=2 lpr=194479 pi=193262-194475/26 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194479 2014-08-13 17:38:03.521519 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194476-194478 up [343,45,4](343) acting [343,45,4](343)): not rw, up_thru 194476 up_from 192942 last_epoch_clean 194474 generate_past_intervals interval(194476-194478 up [343,45,4](343) acting [343,45,4](343) maybe_went_rw) : primary up 192942-194476 includes interval 2014-08-13 17:38:03.521497 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) osd.340 (up) accepted 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) osd.347 (up) accepted 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.521549 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194476/194476/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194476-194478 up [343,45,4](343) acting [343,45,4](343) maybe_went_rw) 2014-08-13 17:38:03.521550 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340,347 2014-08-13 17:38:03.521560 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340,347] (== acting) backfill_targets 2014-08-13 17:38:03.521560 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45,4] -> [343,45], acting [343,45,4] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role 2 -> -1 2014-08-13 17:38:03.521572 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:03.521574 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.521581 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.521586 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000181 0 0.000000 2014-08-13 17:38:03.521590 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.521599 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.521600 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 17:38:03.521609 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 17:38:03.521609 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: osd.340 has no missing, identical log 2014-08-13 17:38:03.521617 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 17:38:03.521620 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: osd.347 has no missing, identical log 2014-08-13 17:38:03.521626 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:03.521629 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.521637 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 17:38:03.521640 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000041 0 0.000000 2014-08-13 17:38:03.521647 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 17:38:03.521651 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.521663 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521656 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_role_change 2014-08-13 17:38:03.521664 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.521672 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 17:38:03.521680 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 17:38:03.521678 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 63 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521690 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.521691 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.63 info 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521703 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521708 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 349 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521709 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 17:38:03.521724 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.521719 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.349 info 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521731 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521727 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.521734 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 352 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521736 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.521745 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.521745 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.352 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521756 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521760 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 366 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521763 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 17:38:03.521777 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.521775 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.366 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521786 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521779 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.521788 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.521790 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 357 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.521797 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.521801 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.521816 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357,371,380 unchanged 2014-08-13 17:38:03.521827 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521818 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,303]/[343,45,303] -- 343/343 2014-08-13 17:38:03.521831 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.521831 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 10 notify: (query_epoch:194485, epoch_sent:194485, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521834 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.521843 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.521842 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 17:38:03.521853 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521852 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45,303] newacting [343,45,303] 2014-08-13 17:38:03.521857 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 304 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.521861 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.521869 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.304 2014-08-13 17:38:03.521870 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45] r=-1 lpr=194479 pi=193262-194478/27 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194482 2014-08-13 17:38:03.521878 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.304 info 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521886 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [343,45](343) acting [343,45](343)): not rw, up_thru 194479 up_from 192942 last_epoch_clean 194474 generate_past_intervals interval(194479-194481 up [343,45](343) acting [343,45](343) maybe_went_rw) : primary up 192942-194479 includes interval 2014-08-13 17:38:03.521896 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521896 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194479/194479/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194479-194481 up [343,45](343) acting [343,45](343) maybe_went_rw) 2014-08-13 17:38:03.521900 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.521906 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45] -> [343,45,303], acting [343,45] -> [343,45,303], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 17:38:03.521911 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.61 info 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.521923 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.521920 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:03.521927 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 343 notify: (query_epoch:194485, epoch_sent:194485, info:22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.521929 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.521938 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.521937 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.521948 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 17:38:03.521952 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,343,349,362 unchanged 2014-08-13 17:38:03.521956 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 17:38:03.521963 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.165273 9 0.006921 2014-08-13 17:38:03.521965 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 17:38:03.521974 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:03.521975 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.521983 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 17:38:03.521991 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 17:38:03.522000 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 17:38:03.522008 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 17:38:03.522005 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522018 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.522018 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.522028 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522038 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.522040 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,303]/[343,45] -- 343/343 2014-08-13 17:38:03.522054 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.522048 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.349 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522057 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.522059 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.362 22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522066 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.522075 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45,303] newacting [343,45] 2014-08-13 17:38:03.522083 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) osd.58 (up) accepted 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.343 (up) accepted 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.522085 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.522093 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,58,343 2014-08-13 17:38:03.522094 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303] r=-1 lpr=194482 pi=193262-194481/28 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194483 2014-08-13 17:38:03.522103 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,58,343] (== acting) backfill_targets 2014-08-13 17:38:03.522112 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [343,45,303](343) acting [343,45,303](343)): not rw, up_thru 194479 up_from 192942 last_epoch_clean 194474 generate_past_intervals interval(194482-194482 up [343,45,303](343) acting [343,45,303](343)) : primary up 192942-194479 does not include interval 2014-08-13 17:38:03.522113 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.522122 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194482/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] noting past interval(194482-194482 up [343,45,303](343) acting [343,45,303](343)) 2014-08-13 17:38:03.522125 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000150 0 0.000000 2014-08-13 17:38:03.522133 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] up [343,45,303] -> [343,45,303], acting [343,45,303] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 17:38:03.522137 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.522145 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 17:38:03.522146 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.522155 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.522157 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 17:38:03.522164 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.522168 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.522178 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522174 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_copy_ops 2014-08-13 17:38:03.522183 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_flush_ops 2014-08-13 17:38:03.522197 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 363 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522191 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change_cleanup 2014-08-13 17:38:03.522210 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.363 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.522223 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522214 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 17:38:03.522228 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.522232 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:03.522233 7fe99ebe8700 7 osd.4 194485 do_notify osd 373 on 1 PGs 2014-08-13 17:38:03.522227 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit NotTrimming 2014-08-13 17:38:03.522237 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter NotTrimming 2014-08-13 17:38:03.522248 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ecb700 don't know epoch, doing nothing 2014-08-13 17:38:03.522250 7fe99ebe8700 7 osd.4 194485 do_notify osd 396 on 1 PGs 2014-08-13 17:38:03.522246 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_recovery 2014-08-13 17:38:03.522261 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.522255 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] clear_recovery_state 2014-08-13 17:38:03.522267 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.522290 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,303]/[343,45] -- 343/343 2014-08-13 17:38:03.522304 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.522298 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.522307 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.522310 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.324 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.522317 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.522326 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.522325 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.324 has stray content: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.522337 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,370 -> 4,324,340,370 2014-08-13 17:38:03.522346 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522349 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522344 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,4]/[343,45] -- 343/343 2014-08-13 17:38:03.522357 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.522352 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522360 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:03.522363 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.303 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522370 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194474 2014-08-13 17:38:03.522377 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522379 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45] 2014-08-13 17:38:03.522389 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.522389 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,313,318,342 -> 4,45,303,313,318,342 2014-08-13 17:38:03.522400 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522407 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522399 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,303]/[343,45] r=-1 lpr=194483 pi=193262-194482/29 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194485 2014-08-13 17:38:03.522411 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 397 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:03.522416 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [343,45,303](343) acting [343,45](343)): not rw, up_thru 194483 up_from 192942 last_epoch_clean 194474 generate_past_intervals interval(194483-194484 up [343,45,303](343) acting [343,45](343) maybe_went_rw) : primary up 192942-194483 includes interval 2014-08-13 17:38:03.522423 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.397 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.522426 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194482/194483/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] noting past interval(194483-194484 up [343,45,303](343) acting [343,45](343) maybe_went_rw) 2014-08-13 17:38:03.522437 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.397 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.522437 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] up [343,45,303] -> [343,45,4], acting [343,45] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 17:38:03.522449 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 17:38:03.522449 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376,382,396 -> 4,293,338,376,382,396,397 2014-08-13 17:38:03.522460 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522462 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522459 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.522466 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 373 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.522468 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.522478 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_copy_ops 2014-08-13 17:38:03.522476 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.373 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522487 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_flush_ops 2014-08-13 17:38:03.522490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.373 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522496 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change_cleanup 2014-08-13 17:38:03.522506 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 17:38:03.522503 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376,382,396,397 -> 4,293,338,373,376,382,396,397 2014-08-13 17:38:03.522516 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522516 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit NotTrimming 2014-08-13 17:38:03.522520 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.030691 8 0.000646 2014-08-13 17:38:03.522526 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter NotTrimming 2014-08-13 17:38:03.522532 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.522538 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_recovery 2014-08-13 17:38:03.522547 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] clear_recovery_state 2014-08-13 17:38:03.522546 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522558 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.522557 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.293 22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485) 2014-08-13 17:38:03.522569 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_activate_map 2014-08-13 17:38:03.522568 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522599 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.373 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522610 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) 2014-08-13 17:38:03.522611 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] update_heartbeat_peers unchanged 2014-08-13 17:38:03.522622 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] take_waiters 2014-08-13 17:38:03.522620 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.382 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.522632 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Reset 0.001185 7 0.001013 2014-08-13 17:38:03.522633 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.396 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:03.522644 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started 2014-08-13 17:38:03.522644 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.397 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485) 2014-08-13 17:38:03.522655 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Start 2014-08-13 17:38:03.522664 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: transitioning to Stray 2014-08-13 17:38:03.522670 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) osd.338 (up) accepted 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) osd.376 (up) accepted 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) 2014-08-13 17:38:03.522675 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.522682 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,338,376 2014-08-13 17:38:03.522686 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started/Stray 2014-08-13 17:38:03.522692 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,338,376] (== acting) backfill_targets 2014-08-13 17:38:03.522699 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194477 while current is 194485 2014-08-13 17:38:03.522706 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.522709 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.522723 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000191 0 0.000000 2014-08-13 17:38:03.522734 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.522743 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: osd.338 has no missing, identical log 2014-08-13 17:38:03.522753 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: requesting log+missing since 194480'0 from osd.376 2014-08-13 17:38:03.522762 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.522767 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522793 7fe99f5e9700 10 osd.4 pg_epoch: 194478 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 17:38:03.522813 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.522809 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 320 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522816 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Reset advmap 2014-08-13 17:38:03.522827 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.522825 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.320 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522836 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [348,331] newacting [332,337] 2014-08-13 17:38:03.522840 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.320 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522846 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.522853 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,363,366,373 -> 4,320,344,363,366,373 2014-08-13 17:38:03.522856 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [4,348,331]/[4,332,337] r=0 lpr=194478 pi=193644-194476/27 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194479 2014-08-13 17:38:03.522864 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522868 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522873 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522876 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194477-194478 up [4,348,331](4) acting [4,332,337](4)): not rw, up_thru 194477 up_from 194476 last_epoch_clean 194465 generate_past_intervals interval(194477-194478 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) : primary up 194476-194477 includes interval 2014-08-13 17:38:03.522884 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.342 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522886 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194476/194477/194477) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] noting past interval(194477-194478 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) 2014-08-13 17:38:03.522897 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] up [4,348,331] -> [348,331], acting [4,332,337] -> [332,337], acting_primary 4 -> 332, up_primary 4 -> 348, role 0 -> -1 2014-08-13 17:38:03.522897 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.522908 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] clear_primary_state 2014-08-13 17:38:03.522909 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310 -> 4,15,21,303,310,342 2014-08-13 17:38:03.522920 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.522923 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522918 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] agent_stop 2014-08-13 17:38:03.522928 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] on_change 2014-08-13 17:38:03.522927 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 0 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522937 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.522938 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.0 info 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.522950 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522946 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:03.522956 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] cancel_copy_ops 2014-08-13 17:38:03.522957 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 371 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.522965 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] cancel_flush_ops 2014-08-13 17:38:03.522969 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.371 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.522981 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.522974 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] on_change_cleanup 2014-08-13 17:38:03.522984 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] on_change 2014-08-13 17:38:03.522985 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.522994 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] exit NotTrimming 2014-08-13 17:38:03.522997 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.523007 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523003 7fe99f5e9700 20 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped m=48] enter NotTrimming 2014-08-13 17:38:03.523012 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_stats 2014-08-13 17:38:03.523014 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.523021 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523024 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.369 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523030 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_role_change 2014-08-13 17:38:03.523040 7fe99f5e9700 15 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523038 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523049 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.523050 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353 -> 0,4,56,63,353,369 2014-08-13 17:38:03.523061 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.523063 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523057 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.523066 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 7 notify: (query_epoch:194485, epoch_sent:194485, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.523067 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523078 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.7 2014-08-13 17:38:03.523086 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.7 info 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.523104 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523087 7fe99f5e9700 10 osd.4 pg_epoch: 194479 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 17:38:03.523108 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 353 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.523121 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523119 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523124 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523133 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353,369 unchanged 2014-08-13 17:38:03.523135 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523144 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523145 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523148 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 356 notify: (query_epoch:194485, epoch_sent:194485, info:22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.523159 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523163 7fe99f5e9700 10 osd.4 pg_epoch: 194480 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 17:38:03.523176 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523172 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357,371,380 unchanged 2014-08-13 17:38:03.523178 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523189 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.158775 9 0.000823 2014-08-13 17:38:03.523188 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523201 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.523203 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523213 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523224 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.352 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523225 7fe99f5e9700 10 osd.4 pg_epoch: 194481 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [332,337,379]/[332,337] -- 332/332 2014-08-13 17:38:03.523238 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523241 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523244 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523250 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.371 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523260 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [332,337,379] newacting [332,337] 2014-08-13 17:38:03.523264 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.380 22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523270 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.523279 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [348,331]/[332,337] r=-1 lpr=194479 pi=193644-194478/28 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194482 2014-08-13 17:38:03.523289 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) osd.356 (up) accepted 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.357 (up) accepted 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523296 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194481 up [348,331](348) acting [332,337](332)): not rw, up_thru 194479 up_from 192968 last_epoch_clean 194465 generate_past_intervals interval(194479-194481 up [348,331](348) acting [332,337](332) maybe_went_rw) : primary up 192968-194479 includes interval 2014-08-13 17:38:03.523299 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,356,357 2014-08-13 17:38:03.523306 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194479/194479/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] noting past interval(194479-194481 up [348,331](348) acting [332,337](332) maybe_went_rw) 2014-08-13 17:38:03.523308 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,356,357] (== acting) backfill_targets 2014-08-13 17:38:03.523316 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] up [348,331] -> [332,337,379], acting [332,337] -> [332,337], acting_primary 332 -> 332, up_primary 348 -> 332, role -1 -> -1 2014-08-13 17:38:03.523319 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.523328 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.523332 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000131 0 0.000000 2014-08-13 17:38:03.523338 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.523344 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.523348 7fe99f5e9700 15 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523354 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.523358 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.523365 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 17:38:03.523368 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.523376 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.523377 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.523386 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523386 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:03.523396 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.523405 7fe99f5e9700 20 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.523414 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.523401 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 344 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.523423 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.523430 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523432 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523446 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,320,344,363,366,373 unchanged 2014-08-13 17:38:03.523456 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523455 7fe99f5e9700 10 osd.4 pg_epoch: 194482 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [332,337,379]/[332,337,379] -- 332/332 2014-08-13 17:38:03.523461 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 56 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.523469 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523472 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523471 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523482 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523486 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353,369 unchanged 2014-08-13 17:38:03.523491 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [332,337,379] newacting [332,337,379] 2014-08-13 17:38:03.523498 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.123415 8 0.000819 2014-08-13 17:38:03.523501 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.523510 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.523511 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379]/[332,337] r=-1 lpr=194482 pi=193644-194481/29 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194483 2014-08-13 17:38:03.523522 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523527 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194482-194482 up [332,337,379](332) acting [332,337](332)): not rw, up_thru 194479 up_from 192968 last_epoch_clean 194465 generate_past_intervals interval(194482-194482 up [332,337,379](332) acting [332,337](332)) : primary up 192968-194479 does not include interval 2014-08-13 17:38:03.523534 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523541 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194482/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] noting past interval(194482-194482 up [332,337,379](332) acting [332,337](332)) 2014-08-13 17:38:03.523545 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523552 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] up [332,337,379] -> [332,337,379], acting [332,337] -> [332,337,379], acting_primary 332 -> 332, up_primary 332 -> 332, role -1 -> -1 2014-08-13 17:38:03.523555 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.63 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523563 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.523565 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523572 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:03.523575 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523581 7fe99f5e9700 15 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523586 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.4 because it is current primary 2014-08-13 17:38:03.523591 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.523600 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.523608 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.523608 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) osd.353 (up) accepted 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.56 (up) accepted 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523618 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.523619 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,56,353 2014-08-13 17:38:03.523627 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.523628 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,353,56] (== acting) backfill_targets 2014-08-13 17:38:03.523636 7fe99f5e9700 20 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.523639 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.523645 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 17:38:03.523651 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000141 0 0.000000 2014-08-13 17:38:03.523654 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:03.523663 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.523664 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523672 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.523685 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 17:38:03.523686 7fe99f5e9700 10 osd.4 pg_epoch: 194483 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [332,337,379]/[332,337,379] -- 332/332 2014-08-13 17:38:03.523700 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523695 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.523707 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523703 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523712 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523720 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.523737 7fe99f5e9700 10 osd.4 pg_epoch: 194484 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [4,348,331]/[4,348,331] -- 4/4 2014-08-13 17:38:03.523750 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:03.523742 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194478/194466 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 376 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.523753 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:03.523756 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.523762 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.523771 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [4,348,331] newacting [4,348,331] 2014-08-13 17:38:03.523779 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:03.523787 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.523787 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [332,337,379] r=-1 lpr=194483 pi=193644-194482/30 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194485 2014-08-13 17:38:03.523792 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 373 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.523804 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523805 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194483-194484 up [332,337,379](332) acting [332,337,379](332)): not rw, up_thru 194483 up_from 192968 last_epoch_clean 194465 generate_past_intervals interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) : primary up 192968-194483 includes interval 2014-08-13 17:38:03.523818 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194482/194483/194479) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] noting past interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 17:38:03.523821 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,320,344,363,366,373 unchanged 2014-08-13 17:38:03.523829 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] up [332,337,379] -> [4,348,331], acting [332,337,379] -> [4,348,331], acting_primary 332 -> 4, up_primary 332 -> 4, role -1 -> 0 2014-08-13 17:38:03.523833 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.155600 8 0.014705 2014-08-13 17:38:03.523840 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] clear_primary_state 2014-08-13 17:38:03.523844 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.523850 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] agent_stop 2014-08-13 17:38:03.523857 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523860 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.523869 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.320 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523871 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd 194485:22146 2014-08-13 17:38:03.523880 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523881 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523890 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.363 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523891 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:03.523900 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.366 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.523904 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:03.523911 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523913 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523922 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523931 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.523934 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) osd.373 (up) accepted 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.344 (up) accepted 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.523940 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:03.523945 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,344,373 2014-08-13 17:38:03.523949 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 17:38:03.523954 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,373,344] (== acting) backfill_targets 2014-08-13 17:38:03.523959 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:03.523966 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.523970 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:03.523979 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] clear_stats 2014-08-13 17:38:03.523978 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000133 0 0.000000 2014-08-13 17:38:03.523988 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] requeue_ops 2014-08-13 17:38:03.523989 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.523997 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] on_role_change 2014-08-13 17:38:03.523998 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.524005 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] requeue_ops 2014-08-13 17:38:03.524008 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000019 0 0.000000 2014-08-13 17:38:03.524014 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] cancel_recovery 2014-08-13 17:38:03.524020 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.524030 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524023 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] clear_recovery_state 2014-08-13 17:38:03.524033 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:03.524043 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_activate_map 2014-08-13 17:38:03.524043 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 320 notify: (query_epoch:194485, epoch_sent:194485, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.524054 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] update_heartbeat_peers -> 4,331,348 2014-08-13 17:38:03.524056 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.320 2014-08-13 17:38:03.524064 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.524068 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] take_waiters 2014-08-13 17:38:03.524067 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.320 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.524077 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Reset 0.625516 9 0.001081 2014-08-13 17:38:03.524086 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524089 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started 2014-08-13 17:38:03.524092 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.524099 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Start 2014-08-13 17:38:03.524103 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.369 2014-08-13 17:38:03.524108 7fe99f5e9700 1 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: transitioning to Primary 2014-08-13 17:38:03.524119 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Start 0.000020 0 0.000000 2014-08-13 17:38:03.524113 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.524131 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524131 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary 2014-08-13 17:38:03.524135 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 45 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.524141 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering 2014-08-13 17:38:03.524146 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.45 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.524159 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524151 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:03.524163 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.524167 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.524168 7fe99ebe8700 7 osd.4 194485 do_notify osd 376 on 1 PGs 2014-08-13 17:38:03.524161 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] _calc_past_interval_range: already have past intervals back to 194465 2014-08-13 17:38:03.524183 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.524190 7fe99ebe8700 7 osd.4 194485 do_queries querying osd.376 on 1 PGs 2014-08-13 17:38:03.524172 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 17:38:03.524205 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194482-194482 up [332,337,379](332) acting [332,337](332)) 2014-08-13 17:38:03.524216 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194479-194481 up [348,331](348) acting [332,337](332) maybe_went_rw) 2014-08-13 17:38:03.524226 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194477-194478 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) 2014-08-13 17:38:03.524236 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194476-194476 up [4,348,331](4) acting [332,337,379](332)) 2014-08-13 17:38:03.524246 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194474-194475 up [348,331](348) acting [332,337,379](332) maybe_went_rw) 2014-08-13 17:38:03.524256 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194473-194473 up [348,331](348) acting [332,337](332)) 2014-08-13 17:38:03.524266 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 17:38:03.524276 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.524286 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] publish_stats_to_osd 194485:22147 2014-08-13 17:38:03.524296 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.331 2014-08-13 17:38:03.524305 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.524306 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.332 2014-08-13 17:38:03.524316 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.337 2014-08-13 17:38:03.524325 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.348 2014-08-13 17:38:03.524335 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.379 2014-08-13 17:38:03.524345 7fe99f5e9700 20 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_activate_map: Not dirtying info: last_persisted is 194478 while current is 194485 2014-08-13 17:38:03.524354 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 NullEvt 2014-08-13 17:38:03.524356 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 313 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.524376 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.524394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,303,313,318,342 unchanged 2014-08-13 17:38:03.524406 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524409 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.524412 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.524422 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.524435 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524470 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:03.524476 7fe99f5e9700 7 osd.4 194485 do_notify osd 285 on 1 PGs 2014-08-13 17:38:03.524493 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e3c8e0 don't know epoch, doing nothing 2014-08-13 17:38:03.524495 7fe99f5e9700 7 osd.4 194485 do_notify osd 292 on 1 PGs 2014-08-13 17:38:03.524555 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.524560 7fe99f5e9700 7 osd.4 194485 do_notify osd 323 on 1 PGs 2014-08-13 17:38:03.524540 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.524564 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.524576 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 don't know epoch, doing nothing 2014-08-13 17:38:03.524577 7fe99f5e9700 7 osd.4 194485 do_notify osd 343 on 1 PGs 2014-08-13 17:38:03.524581 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524621 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d348e0 don't know epoch, doing nothing 2014-08-13 17:38:03.524629 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.44 on 1 PGs 2014-08-13 17:38:03.524619 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 318 notify: (query_epoch:194485, epoch_sent:194485, info:22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.524668 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.524682 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:03.524685 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.61 on 1 PGs 2014-08-13 17:38:03.524689 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,303,313,318,342 unchanged 2014-08-13 17:38:03.524697 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4df3700 don't know epoch, doing nothing 2014-08-13 17:38:03.524701 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.287 on 1 PGs 2014-08-13 17:38:03.524703 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.094259 9 0.000806 2014-08-13 17:38:03.524713 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:03.524715 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.289 on 1 PGs 2014-08-13 17:38:03.524716 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.524727 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.524729 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.303 on 1 PGs 2014-08-13 17:38:03.524730 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.524741 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x53806e0 don't know epoch, doing nothing 2014-08-13 17:38:03.524743 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.306 on 1 PGs 2014-08-13 17:38:03.524741 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.45 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.524755 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing 2014-08-13 17:38:03.524757 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.309 on 1 PGs 2014-08-13 17:38:03.524752 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.524762 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.524771 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.524779 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e82520 don't know epoch, doing nothing 2014-08-13 17:38:03.524781 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.327 on 1 PGs 2014-08-13 17:38:03.524780 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.524798 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing 2014-08-13 17:38:03.524802 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.331 on 1 PGs 2014-08-13 17:38:03.524806 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) osd.313 (up) accepted 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.318 (up) accepted 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.524816 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,313,318 2014-08-13 17:38:03.524824 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,313,318] (== acting) backfill_targets 2014-08-13 17:38:03.524837 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.524849 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000133 0 0.000000 2014-08-13 17:38:03.524861 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.524870 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.524880 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 17:38:03.524891 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.524901 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.524944 7fe97a1fc700 10 osd.4 194485 OSD::ms_get_authorizer type=osd 2014-08-13 17:38:03.524982 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing 2014-08-13 17:38:03.524987 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.332 on 1 PGs 2014-08-13 17:38:03.525004 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:03.525006 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.337 on 1 PGs 2014-08-13 17:38:03.524995 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.525016 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ec8160 don't know epoch, doing nothing 2014-08-13 17:38:03.525018 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.339 on 1 PGs 2014-08-13 17:38:03.525018 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525034 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525039 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 54 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.525050 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.54 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525062 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525070 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4df32e0 don't know epoch, doing nothing 2014-08-13 17:38:03.525066 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 15 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.525075 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.341 on 1 PGs 2014-08-13 17:38:03.525077 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.15 info 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525088 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525107 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing 2014-08-13 17:38:03.525105 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 11 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.525114 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.348 sending saperate messages on 1 PGs 2014-08-13 17:38:03.525116 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.11 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525128 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525128 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:03.525129 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.362 on 1 PGs 2014-08-13 17:38:03.525132 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 310 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.525142 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x5250840 don't know epoch, doing nothing 2014-08-13 17:38:03.525144 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.369 on 1 PGs 2014-08-13 17:38:03.525143 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525162 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e4d280 already has epoch 194485 2014-08-13 17:38:03.525163 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.376 on 1 PGs 2014-08-13 17:38:03.525163 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310,342 unchanged 2014-08-13 17:38:03.525174 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e82c00 don't know epoch, doing nothing 2014-08-13 17:38:03.525175 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525176 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.379 on 1 PGs 2014-08-13 17:38:03.525193 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecb9c0 don't know epoch, doing nothing 2014-08-13 17:38:03.525179 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 21 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.525195 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.381 on 1 PGs 2014-08-13 17:38:03.525196 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525211 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310,342 unchanged 2014-08-13 17:38:03.525229 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.525223 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.081144 8 0.000570 2014-08-13 17:38:03.525238 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.525250 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.525261 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.15 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.525271 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525280 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.525289 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525298 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.525323 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) osd.310 (up) accepted 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.21 (up) accepted 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525333 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,21,310 2014-08-13 17:38:03.525342 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,310,21] (== acting) backfill_targets 2014-08-13 17:38:03.525353 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.525364 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000127 0 0.000000 2014-08-13 17:38:03.525376 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.525385 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.525395 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 17:38:03.525405 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.525415 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525430 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.525442 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.342 2014-08-13 17:38:03.525452 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525468 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525472 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 368 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.525476 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.525483 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.368 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.525496 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525495 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.525498 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.525499 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e040 pg_notify(22.3af epoch 194485) v5 2014-08-13 17:38:03.525500 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 47 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.525512 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525526 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.44 2014-08-13 17:38:03.525528 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e040 2014-08-13 17:38:03.525528 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,54,368 unchanged 2014-08-13 17:38:03.525539 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.525543 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 5 notify: (query_epoch:194485, epoch_sent:194485, info:22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.525556 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.525558 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.525552 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.525573 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.525566 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,54,368 unchanged 2014-08-13 17:38:03.525575 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.525576 7fe9a63f4700 20 osd.4 194485 _dispatch 0x27a7800 pg_log(22.3d9 epoch 194485 log log((194215'511,194215'511], crt=194215'511) query_epoch 194485) v4 2014-08-13 17:38:03.525577 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.073746 9 0.000569 2014-08-13 17:38:03.525575 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194478/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 31 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.529799 7fe9a63f4700 7 osd.4 194485 handle_pg_log pg_log(22.3d9 epoch 194485 log log((194215'511,194215'511], crt=194215'511) query_epoch 194485) v4 from osd.376 2014-08-13 17:38:03.529804 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x27a7800 2014-08-13 17:38:03.525589 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.525596 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] sending info 2014-08-13 17:38:03.529828 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529830 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529821 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.529843 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529844 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529845 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8a80 pg_notify(24.1bb epoch 194485) v5 2014-08-13 17:38:03.529844 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.529855 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.11 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.529866 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.331 2014-08-13 17:38:03.529868 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8a80 2014-08-13 17:38:03.529866 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.529866 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.529876 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.54 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.529888 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529890 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529887 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.368 22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.529879 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 356 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.529901 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529902 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529903 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5105b00 pg_notify(24.1bb(34) epoch 194485) v5 2014-08-13 17:38:03.529900 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194475/194465 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.529913 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) osd.5 (up) accepted 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.47 (up) accepted 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.529931 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.379 2014-08-13 17:38:03.529919 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.356 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.529932 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5105b00 2014-08-13 17:38:03.529925 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,5,47 2014-08-13 17:38:03.529935 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,5,47] (== acting) backfill_targets 2014-08-13 17:38:03.529955 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529934 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,55,61 -> 4,55,61,356 2014-08-13 17:38:03.529957 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529948 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.529958 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.529960 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.529963 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.004373 0 0.000000 2014-08-13 17:38:03.529975 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.529976 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.529977 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51c8540 pg_notify(22.48e(29) epoch 194485) v5 2014-08-13 17:38:03.529972 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 17:38:03.529976 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.529986 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.529986 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194482-194482 up [50,65,356](50) acting [50,65](50)) 2014-08-13 17:38:03.530000 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.327 2014-08-13 17:38:03.530002 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51c8540 2014-08-13 17:38:03.529998 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000023 0 0.000000 2014-08-13 17:38:03.529998 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 17:38:03.530012 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.530012 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.530023 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530023 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530025 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530025 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.50 2014-08-13 17:38:03.530038 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530040 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530041 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc280 pg_notify(22.3af(31) epoch 194485) v5 2014-08-13 17:38:03.530035 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.55 2014-08-13 17:38:03.530044 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.61 2014-08-13 17:38:03.530043 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 44 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.530063 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.339 2014-08-13 17:38:03.530056 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.65 2014-08-13 17:38:03.530065 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc280 2014-08-13 17:38:03.530061 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.44 info 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.530075 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530066 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: have osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530085 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530087 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530079 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.530091 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.530105 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530107 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530108 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5105e80 pg_notify(22.481(46) epoch 194485) v5 2014-08-13 17:38:03.530116 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530108 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,330,331,349 unchanged 2014-08-13 17:38:03.530122 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530131 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.381 2014-08-13 17:38:03.530132 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5105e80 2014-08-13 17:38:03.530127 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 45 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.530138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.530135 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194477/194477 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 30 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530153 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530156 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530151 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339,342 unchanged 2014-08-13 17:38:03.530162 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530158 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.530173 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530175 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530166 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 65 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.530176 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c16e40 pg_notify(22.481(39) epoch 194485) v5 2014-08-13 17:38:03.530177 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.530196 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.309 2014-08-13 17:38:03.530198 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c16e40 2014-08-13 17:38:03.530206 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530198 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,48,62,65,314 unchanged 2014-08-13 17:38:03.530209 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530212 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194477/194474 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 285 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530215 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.530226 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530228 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530226 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] sending info 2014-08-13 17:38:03.530228 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.530239 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530244 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530245 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530246 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143d40 pg_notify(22.3af(31) epoch 194485) v5 2014-08-13 17:38:03.530243 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 302 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.530254 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.302 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.530266 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530268 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.369 2014-08-13 17:38:03.530269 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530269 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143d40 2014-08-13 17:38:03.530273 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.530279 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.530271 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.530285 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194475/194466 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.61 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530301 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.61 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530314 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,362 -> 4,44,61,362 2014-08-13 17:38:03.530323 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.530325 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.530320 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194477/194474 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 356 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530335 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 17:38:03.530336 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] sending info 2014-08-13 17:38:03.530348 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 17:38:03.530358 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,61,339,362,369 down blocked_by {} 2014-08-13 17:38:03.530368 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.530384 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530378 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.44 2014-08-13 17:38:03.530390 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 55 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.530392 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: have osd.61 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530404 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.339 2014-08-13 17:38:03.530404 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194475/194466 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530413 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.362 2014-08-13 17:38:03.530422 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 17:38:03.530421 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] osd.55 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530434 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,59,351 -> 4,55,59,351 2014-08-13 17:38:03.530444 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.530449 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530446 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.530457 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 17:38:03.530469 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 17:38:03.530481 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530483 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530468 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 289 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.530480 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 17:38:03.530491 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.530502 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530503 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530504 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5105240 pg_notify(22.481(44) epoch 194485) v5 2014-08-13 17:38:03.530487 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.530503 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: have osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530514 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,50,289 unchanged 2014-08-13 17:38:03.530516 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.59 2014-08-13 17:38:03.530528 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.306 2014-08-13 17:38:03.530530 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5105240 2014-08-13 17:38:03.530528 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.530529 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.351 2014-08-13 17:38:03.530539 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.376 2014-08-13 17:38:03.530539 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 17:38:03.530552 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530554 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530553 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 17:38:03.530568 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530564 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,32,50,289 down blocked_by {} 2014-08-13 17:38:03.530572 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530572 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530575 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5481d40 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.530574 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.530581 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.530585 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.530586 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530587 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530584 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.32 2014-08-13 17:38:03.530594 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.50 2014-08-13 17:38:03.530602 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530590 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194477/194477 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 337 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530604 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530605 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e200 pg_notify(22.3af(1) epoch 194485) v5 2014-08-13 17:38:03.530606 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.530603 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: have osd.289 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.530627 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.61 2014-08-13 17:38:03.530628 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e200 2014-08-13 17:38:03.530645 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530647 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530649 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530651 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530660 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530661 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530663 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5105080 pg_notify(22.48e(1) epoch 194485) v5 2014-08-13 17:38:03.530656 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194477/194474 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 343 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530660 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 50 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.530669 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] sending info 2014-08-13 17:38:03.530685 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.289 2014-08-13 17:38:03.530687 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5105080 2014-08-13 17:38:03.530676 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530706 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530706 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530707 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530701 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.50 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530711 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194478/194466 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 15 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530723 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530724 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530715 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,55,61,356 -> 4,50,55,61,356 2014-08-13 17:38:03.530725 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5143800 pg_notify(24.1bb(34) epoch 194485) v5 2014-08-13 17:38:03.530727 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.530730 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530723 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] sending info 2014-08-13 17:38:03.530735 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 32 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.530747 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.337 2014-08-13 17:38:03.530748 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5143800 2014-08-13 17:38:03.530756 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530746 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.32 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.530768 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530770 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530760 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194475/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 320 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530761 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.32 has stray content: 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.530774 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] sending info 2014-08-13 17:38:03.530774 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,50,289 -> 4,32,50,289 2014-08-13 17:38:03.530785 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.530787 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530787 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530789 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530790 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5105780 pg_notify(22.3af epoch 194485) v5 2014-08-13 17:38:03.530791 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194477/194477 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 323 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530802 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.530811 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.362 2014-08-13 17:38:03.530812 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5105780 2014-08-13 17:38:03.530813 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530818 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194478/194466 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 371 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530837 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530831 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.530838 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530839 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.530849 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.530851 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.530852 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5103b80 pg_notify(22.481(46) epoch 194485) v5 2014-08-13 17:38:03.530844 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 287 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:03.530856 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194474/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.530869 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530873 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.287 2014-08-13 17:38:03.530875 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5103b80 2014-08-13 17:38:03.530871 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309 unchanged 2014-08-13 17:38:03.530874 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 351 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.530882 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.530893 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 17:38:03.530892 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.530908 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 17:38:03.530909 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 17:38:03.530920 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530918 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194480-194481 up [287,309](287) acting [287,309,303](287) maybe_went_rw) 2014-08-13 17:38:03.530924 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194478/194466 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 356 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.530929 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194479 up [287,309](287) acting [287,309](287)) 2014-08-13 17:38:03.530936 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] sending info 2014-08-13 17:38:03.530939 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 17:38:03.530952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.530962 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.530974 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 17:38:03.530982 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.306 2014-08-13 17:38:03.530991 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.530990 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.309 2014-08-13 17:38:03.530998 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.381 2014-08-13 17:38:03.530996 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531012 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531023 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531029 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531040 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194477/194477 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 292 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.531041 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 2014-08-13 17:38:03.531052 7fe99ebe8700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531055 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531054 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531056 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531053 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.531058 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 370 notify: (query_epoch:194485, epoch_sent:194485, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.531071 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531072 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531074 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5480fc0 osd_map(194479..194485 src has 185139..194485) v3 2014-08-13 17:38:03.531078 7fe9a63f4700 3 osd.4 194485 handle_osd_map epochs [194479,194485], i have 194485, src has [185139,194485] 2014-08-13 17:38:03.531070 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.370 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.531081 7fe9a63f4700 10 osd.4 194485 no new maps here, dropping 2014-08-13 17:38:03.531082 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531082 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531083 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531085 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 370 notify: (query_epoch:194485, epoch_sent:194485, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.531095 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531098 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531099 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531100 7fe9a63f4700 20 osd.4 194485 _dispatch 0x513e900 pg_notify(24.1bb(1) epoch 194485) v5 2014-08-13 17:38:03.531095 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.370 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.531109 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531105 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 332 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531113 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 349 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.531125 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.332 2014-08-13 17:38:03.531127 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x513e900 2014-08-13 17:38:03.531121 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194475/194465 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531126 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531141 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,330,331,349 unchanged 2014-08-13 17:38:03.531138 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.332 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531153 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.088688 6 0.000560 2014-08-13 17:38:03.531157 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,348 -> 4,331,332,348 2014-08-13 17:38:03.531168 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531166 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.531170 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.531182 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 17:38:03.531179 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531194 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194482-194482 up [332,337,379](332) acting [332,337](332)) 2014-08-13 17:38:03.531196 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.58 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531205 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 17:38:03.531207 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531216 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.531217 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.330 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531228 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.331 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531232 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.331 2014-08-13 17:38:03.531238 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531243 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: have osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531255 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.337 2014-08-13 17:38:03.531262 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) osd.349 (up) accepted 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.61 (up) accepted 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531263 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.348 2014-08-13 17:38:03.531273 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.379 2014-08-13 17:38:03.531274 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,61,349 2014-08-13 17:38:03.531283 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,349,61] (== acting) backfill_targets 2014-08-13 17:38:03.531301 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531295 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.531307 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000141 0 0.000000 2014-08-13 17:38:03.531318 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.531319 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194477/194477 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MQuery from 397 query_epoch 194485 query: query(info 0'0) 2014-08-13 17:38:03.531332 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.531337 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531333 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 17:38:03.531339 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 17:38:03.531349 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531351 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531352 7fe9a63f4700 20 osd.4 194485 _dispatch 0x51cc440 pg_notify(22.481(39) epoch 194485) v5 2014-08-13 17:38:03.531351 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.531361 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531375 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.303 2014-08-13 17:38:03.531375 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531377 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x51cc440 2014-08-13 17:38:03.531380 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 350 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531382 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 379 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531397 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531399 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.531409 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531397 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531420 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531421 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531413 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 340 notify: (query_epoch:194485, epoch_sent:194485, info:1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.531422 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5104440 pg_notify(22.48e(29) epoch 194485) v5 2014-08-13 17:38:03.531424 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531422 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531444 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.376 2014-08-13 17:38:03.531446 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5104440 2014-08-13 17:38:03.531438 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,348 -> 4,331,332,348,379 2014-08-13 17:38:03.531440 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,324,340,370 unchanged 2014-08-13 17:38:03.531450 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531453 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531452 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.075953 5 0.007026 2014-08-13 17:38:03.531457 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 327 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531464 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.531473 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194475/194466 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.327 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531476 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531487 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531488 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] osd.327 has stray content: 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531498 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531501 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,341 -> 4,289,327,341 2014-08-13 17:38:03.531510 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531507 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.370 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531512 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.531523 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 17:38:03.531529 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) osd.340 (up) accepted 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531533 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) 2014-08-13 17:38:03.531539 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340 2014-08-13 17:38:03.531544 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,289,327,341,376 down blocked_by {} 2014-08-13 17:38:03.531548 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340] (== acting) backfill_targets 2014-08-13 17:38:03.531555 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.531558 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.531565 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.289 2014-08-13 17:38:03.531569 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000105 0 0.000000 2014-08-13 17:38:03.531574 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: have osd.327 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531581 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.531585 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.341 2014-08-13 17:38:03.531590 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.531594 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.376 2014-08-13 17:38:03.531601 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 17:38:03.531611 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.531620 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531620 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531639 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 340 notify: (query_epoch:194485, epoch_sent:194485, info:2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.531650 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531664 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,324,340,370 unchanged 2014-08-13 17:38:03.531675 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531677 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531675 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.086336 5 0.000396 2014-08-13 17:38:03.531661 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 65 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531693 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531686 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.531694 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531698 7fe9a63f4700 20 osd.4 194485 _dispatch 0x5106040 pg_notify(24.1bb epoch 194485) v5 2014-08-13 17:38:03.531687 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531699 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531720 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.348 2014-08-13 17:38:03.531711 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531721 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x5106040 2014-08-13 17:38:03.531704 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.65 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531722 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531740 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531742 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531732 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.370 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531733 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,356 -> 4,50,55,61,65,356 2014-08-13 17:38:03.531746 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531750 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531752 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531753 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531755 7fe9a63f4700 20 osd.4 194485 _dispatch 0x4c163c0 pg_notify(22.48e epoch 194485) v5 2014-08-13 17:38:03.531753 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) osd.340 (up) accepted 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531755 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 339 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.531764 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340 2014-08-13 17:38:03.531776 7fe9a63f4700 7 osd.4 194485 handle_pg_notify from osd.341 2014-08-13 17:38:03.531778 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x4c163c0 2014-08-13 17:38:03.531767 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.339 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531774 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340] (== acting) backfill_targets 2014-08-13 17:38:03.531784 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.339 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.531786 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.531799 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.531801 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.531797 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,362 -> 4,44,61,339,362 2014-08-13 17:38:03.531807 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531800 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000114 0 0.000000 2014-08-13 17:38:03.531809 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531812 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.531813 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 381 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.531820 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.531824 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.381 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.531830 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000019 0 0.000000 2014-08-13 17:38:03.531841 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.531838 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.381 has stray content: 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.531851 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.531850 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309 -> 4,287,309,381 2014-08-13 17:38:03.531860 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.531861 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:03.531863 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 301 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.531871 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 17:38:03.531876 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531883 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 17:38:03.531891 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,302,324,368 unchanged 2014-08-13 17:38:03.531893 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 17:38:03.531902 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.164540 6 0.000952 2014-08-13 17:38:03.531912 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194477 < same_since 194485, must notify monitor 2014-08-13 17:38:03.531913 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.531923 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.531925 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531934 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 17:38:03.531937 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.531943 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.306 2014-08-13 17:38:03.531946 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.302 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531952 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.309 2014-08-13 17:38:03.531957 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531961 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.381 info 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.531968 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.368 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.531990 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.531992 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) osd.301 (up) accepted 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.324 (up) accepted 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.532002 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,301,324 2014-08-13 17:38:03.532005 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 309 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:03.532011 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,301,324] (== acting) backfill_targets 2014-08-13 17:38:03.532018 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.532022 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.532033 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000120 0 0.000000 2014-08-13 17:38:03.532044 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.532035 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309,381 unchanged 2014-08-13 17:38:03.532055 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532053 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: osd.324 has no missing, identical log 2014-08-13 17:38:03.532059 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532063 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.532071 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.369 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532073 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 2014-08-13 17:38:03.532085 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.532085 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532098 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532098 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362 -> 4,44,61,339,362,369 2014-08-13 17:38:03.532111 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.532113 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532117 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.532120 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing 2014-08-13 17:38:03.532111 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 291 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532122 7fe99f5e9700 7 osd.4 194485 do_notify osd 30 on 1 PGs 2014-08-13 17:38:03.532124 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532136 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532142 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 285 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.532141 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:03.532156 7fe99f5e9700 7 osd.4 194485 do_notify osd 31 on 1 PGs 2014-08-13 17:38:03.532152 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532169 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:03.532171 7fe99f5e9700 7 osd.4 194485 do_notify osd 285 on 1 PGs 2014-08-13 17:38:03.532167 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,311,312,369 unchanged 2014-08-13 17:38:03.532181 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e3c8e0 don't know epoch, doing nothing 2014-08-13 17:38:03.532183 7fe99f5e9700 7 osd.4 194485 do_notify osd 292 on 1 PGs 2014-08-13 17:38:03.532178 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.116739 5 0.010346 2014-08-13 17:38:03.532194 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing 2014-08-13 17:38:03.532195 7fe99f5e9700 7 osd.4 194485 do_notify osd 323 on 1 PGs 2014-08-13 17:38:03.532194 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.532206 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4e48840 don't know epoch, doing nothing 2014-08-13 17:38:03.532208 7fe99f5e9700 7 osd.4 194485 do_notify osd 397 on 1 PGs 2014-08-13 17:38:03.532206 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.532220 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.532217 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532228 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.311 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.532238 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.532248 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:03.532256 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 306 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.532269 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.306 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.532272 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) osd.312 (up) accepted 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) osd.285 (up) accepted 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532288 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,285,312 2014-08-13 17:38:03.532289 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.306 has stray content: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.532297 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,312,285] (== acting) backfill_targets 2014-08-13 17:38:03.532308 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.532319 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000125 0 0.000000 2014-08-13 17:38:03.532307 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309,381 -> 4,287,306,309,381 2014-08-13 17:38:03.532335 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.532330 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.532339 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532339 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: osd.312 has no missing, identical log 2014-08-13 17:38:03.532349 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.532345 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 50 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.532359 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000029 0 0.000000 2014-08-13 17:38:03.532363 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532370 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.532380 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532388 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,32,50,289 unchanged 2014-08-13 17:38:03.532394 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 20 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532408 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532401 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.039268 4 0.000385 2014-08-13 17:38:03.532423 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532427 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.532431 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:03.532433 7fe99ebe8700 7 osd.4 194485 do_notify osd 15 on 1 PGs 2014-08-13 17:38:03.532427 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.532450 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:03.532452 7fe99ebe8700 7 osd.4 194485 do_notify osd 320 on 1 PGs 2014-08-13 17:38:03.532451 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532464 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:03.532466 7fe99ebe8700 7 osd.4 194485 do_notify osd 337 on 1 PGs 2014-08-13 17:38:03.532476 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 don't know epoch, doing nothing 2014-08-13 17:38:03.532478 7fe99ebe8700 7 osd.4 194485 do_notify osd 343 on 1 PGs 2014-08-13 17:38:03.532463 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.32 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532488 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:03.532490 7fe99ebe8700 7 osd.4 194485 do_notify osd 356 on 2 PGs 2014-08-13 17:38:03.532502 7fe99ebe8700 20 osd.4 194485 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:03.532503 7fe99ebe8700 7 osd.4 194485 do_notify osd 371 on 1 PGs 2014-08-13 17:38:03.532486 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532505 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532515 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 17:38:03.532533 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) osd.50 (up) accepted 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) osd.289 (up) accepted 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:03.532547 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 362 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.532560 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532557 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,50,289 2014-08-13 17:38:03.532572 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,50,289] (== acting) backfill_targets 2014-08-13 17:38:03.532576 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:03.532588 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532592 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532584 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.532603 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.324 2014-08-13 17:38:03.532612 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532629 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532608 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000181 0 0.000000 2014-08-13 17:38:03.532632 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532639 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.532643 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.324 2014-08-13 17:38:03.532649 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: osd.50 has no missing, identical log 2014-08-13 17:38:03.532651 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532660 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: osd.289 has no missing, identical log 2014-08-13 17:38:03.532670 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532669 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.532673 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 324 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532681 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000041 0 0.000000 2014-08-13 17:38:03.532685 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.324 2014-08-13 17:38:03.532694 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532702 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.532711 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532714 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532714 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532725 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got dup osd.303 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532737 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532734 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532741 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 367 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.532748 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.376 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532753 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532767 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:03.532778 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532764 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] osd.376 has stray content: 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532782 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 331 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532787 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341 -> 4,289,327,341,376 2014-08-13 17:38:03.532798 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.532801 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532793 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.331 2014-08-13 17:38:03.532806 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 365 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485)) 2014-08-13 17:38:03.532803 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.331 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532817 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.365 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485) 2014-08-13 17:38:03.532828 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532830 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.365 has stray content: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485) 2014-08-13 17:38:03.532832 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 373 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532843 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,306,309,381 -> 4,287,306,309,365,381 2014-08-13 17:38:03.532854 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.532843 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got dup osd.373 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532857 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532858 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532860 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 330 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532862 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 289 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532871 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.330 2014-08-13 17:38:03.532873 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532880 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.330 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532888 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341,376 unchanged 2014-08-13 17:38:03.532897 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532899 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532901 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 312 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.532903 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 337 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532911 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.312 2014-08-13 17:38:03.532914 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532920 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.312 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532939 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532943 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 365 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.532953 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.532930 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.337 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.532965 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.532966 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,348,379 -> 4,331,332,337,348,379 2014-08-13 17:38:03.532969 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 25 notify: (query_epoch:194485, epoch_sent:194485, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.532978 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.532980 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.532979 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.532984 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.532994 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:03.532997 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.303 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533009 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetInfo 0.102290 9 0.000842 2014-08-13 17:38:03.533010 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533021 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533022 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,306,309,365,381 -> 4,287,303,306,309,365,381 2014-08-13 17:38:03.533032 7fe99f5e9700 20 osd.4 194485 need_heartbeat_peer_update 2014-08-13 17:38:03.533036 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.017562 7 0.000711 2014-08-13 17:38:03.533035 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533049 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533049 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533061 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533070 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533077 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533081 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533089 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.533091 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533103 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533113 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.306 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533124 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.533127 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] choose_acting want [4,20,303] != acting [4,25,362], requesting pg_temp change 2014-08-13 17:38:03.533135 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.365 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485) 2014-08-13 17:38:03.533156 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetLog 0.000135 0 0.000000 2014-08-13 17:38:03.533156 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.381 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) 2014-08-13 17:38:03.533169 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Leaving Peering 2014-08-13 17:38:03.533179 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering 0.102479 0 0.000000 2014-08-13 17:38:03.533185 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) osd.287 (up) accepted 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) osd.309 (up) accepted 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:03.533196 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,287,309 2014-08-13 17:38:03.533197 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.102514 0 0.000000 2014-08-13 17:38:03.533206 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,287,309] (== acting) backfill_targets 2014-08-13 17:38:03.533209 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary 2014-08-13 17:38:03.533219 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.533229 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533218 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.533235 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000186 0 0.000000 2014-08-13 17:38:03.533234 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 311 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.533248 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.533248 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.311 2014-08-13 17:38:03.533259 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: osd.287 has no missing, identical log 2014-08-13 17:38:03.533258 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.311 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533276 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533270 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: requesting log+missing since 194481'0 from osd.309 2014-08-13 17:38:03.533282 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533280 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 368 notify: (query_epoch:194485, epoch_sent:194485, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.533290 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.368 2014-08-13 17:38:03.533299 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 55 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533298 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.368 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533317 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533313 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.55 info 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533326 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533321 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.533332 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.369 2014-08-13 17:38:03.533341 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 24 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533341 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533358 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533354 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.24 info 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533366 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533361 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 58 notify: (query_epoch:194485, epoch_sent:194485, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:03.533371 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.58 2014-08-13 17:38:03.533370 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 339 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.533382 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533381 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.58 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533399 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533398 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339,342 unchanged 2014-08-13 17:38:03.533403 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 378 notify: (query_epoch:194485, epoch_sent:194485, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.533410 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.081356 8 0.000554 2014-08-13 17:38:03.533414 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533422 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533429 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:03.533434 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533441 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.133079 9 0.000877 2014-08-13 17:38:03.533445 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.24 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533454 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533456 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533466 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.55 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533467 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533477 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533479 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533487 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533503 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533511 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) osd.339 (up) accepted 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.45 (up) accepted 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533514 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533522 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,45,339 2014-08-13 17:38:03.533524 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533531 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,339,45] (== acting) backfill_targets 2014-08-13 17:38:03.533542 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.533554 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533554 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000132 0 0.000000 2014-08-13 17:38:03.533565 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] choose_acting want [4,291,350] != acting [4,378,367], requesting pg_temp change 2014-08-13 17:38:03.533566 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.533576 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.533580 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000126 0 0.000000 2014-08-13 17:38:03.533586 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 17:38:03.533583 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.533592 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Leaving Peering 2014-08-13 17:38:03.533599 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.533610 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533603 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.133259 0 0.000000 2014-08-13 17:38:03.533615 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.133289 0 0.000000 2014-08-13 17:38:03.533625 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 17:38:03.533624 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 342 notify: (query_epoch:194485, epoch_sent:194485, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533634 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.533665 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533667 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.342 2014-08-13 17:38:03.533670 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 55 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533682 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533678 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533698 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533700 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533702 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 314 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533704 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 351 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533714 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.314 info 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533715 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533726 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533727 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533732 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 315 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533733 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533744 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533744 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533757 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.533758 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533760 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.533763 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.30b=[4,375,10]} 2014-08-13 17:38:03.533762 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 48 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.533772 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.48 info 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.533784 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533788 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 28 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.533798 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 59 notify: (query_epoch:194485, epoch_sent:194485, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.533799 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533811 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533817 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 17:38:03.533829 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.533827 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged 2014-08-13 17:38:03.533832 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 62 notify: (query_epoch:194485, epoch_sent:194485, info:22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.533839 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetInfo 0.059382 8 0.000541 2014-08-13 17:38:03.533852 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533845 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533864 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533876 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,48,62,65,314 unchanged 2014-08-13 17:38:03.533876 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533887 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533897 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533889 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.148663 9 0.006719 2014-08-13 17:38:03.533908 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533920 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.533935 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533946 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] choose_acting want [4,351,55] != acting [4,59,351], requesting pg_temp change 2014-08-13 17:38:03.533933 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533957 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.44 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533959 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetLog 0.000107 0 0.000000 2014-08-13 17:38:03.533968 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.48 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.533971 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: Leaving Peering 2014-08-13 17:38:03.533978 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533980 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering 0.059533 0 0.000000 2014-08-13 17:38:03.533988 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.533992 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.059556 0 0.000000 2014-08-13 17:38:03.533998 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.314 22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534003 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary 2014-08-13 17:38:03.534013 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.534023 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.534023 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) osd.65 (up) accepted 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.62 (up) accepted 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534028 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.534034 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,62,65 2014-08-13 17:38:03.534040 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534044 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,65,62] (== acting) backfill_targets 2014-08-13 17:38:03.534056 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.534055 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 17:38:03.534067 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.534068 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000148 0 0.000000 2014-08-13 17:38:03.534071 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 55 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.534053 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] flushed 2014-08-13 17:38:03.534080 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.534082 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534090 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.534096 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 17:38:03.534101 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 17:38:03.534108 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetInfo 0.053002 6 0.000646 2014-08-13 17:38:03.534112 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.534123 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.534120 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.534133 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534144 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534139 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 17 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.534155 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534155 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.534166 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534158 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.17 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.534181 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.534176 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534186 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 327 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.534192 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534198 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.534211 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.534215 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 330 notify: (query_epoch:194485, epoch_sent:194485, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.534226 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.534241 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 17:38:03.534223 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534253 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.078969 9 0.000661 2014-08-13 17:38:03.539591 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.534252 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] choose_acting want [4,50,65] != acting [4,55,61], requesting pg_temp change 2014-08-13 17:38:03.539619 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.539632 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.534782 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.539643 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.539634 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetLog 0.005513 0 0.000000 2014-08-13 17:38:03.539654 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.539657 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: Leaving Peering 2014-08-13 17:38:03.539668 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering 0.058574 0 0.000000 2014-08-13 17:38:03.539666 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.539674 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.539679 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.539682 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.058598 0 0.000000 2014-08-13 17:38:03.539695 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary 2014-08-13 17:38:03.539704 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.539715 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539714 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.539722 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 65 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.539726 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] choose_acting want [4,17,315] != acting [4,330,28], requesting pg_temp change 2014-08-13 17:38:03.539734 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 17:38:03.539746 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539743 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000152 0 0.000000 2014-08-13 17:38:03.539750 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 356 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.539755 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: Leaving Peering 2014-08-13 17:38:03.539762 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 17:38:03.539772 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539766 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.084492 0 0.000000 2014-08-13 17:38:03.539775 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 50 notify: (query_epoch:194485, epoch_sent:194485, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.539779 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.084517 0 0.000000 2014-08-13 17:38:03.539786 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 17:38:03.539800 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539791 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 17:38:03.539802 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.539813 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.539805 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 10 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.539819 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.10 2014-08-13 17:38:03.539820 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 31 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.539834 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.539829 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.10 info 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.539849 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539851 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,363,374 unchanged 2014-08-13 17:38:03.539853 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 7 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.539863 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.539867 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.539864 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.7 2014-08-13 17:38:03.539871 7fe99f5e9700 20 osd.4 194485 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing 2014-08-13 17:38:03.539873 7fe99f5e9700 7 osd.4 194485 do_queries querying osd.309 on 1 PGs 2014-08-13 17:38:03.539873 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.7 info 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.539890 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539896 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.539899 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 286 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.539909 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.286 2014-08-13 17:38:03.539918 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.286 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.539934 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539937 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 343 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.539947 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.343 2014-08-13 17:38:03.539957 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 374 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.539955 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.343 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.539956 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] flushed 2014-08-13 17:38:03.539973 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.539972 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.539978 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 340 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.539980 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.539989 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.340 2014-08-13 17:38:03.539988 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,363,374 unchanged 2014-08-13 17:38:03.539997 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540002 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.067812 6 0.000599 2014-08-13 17:38:03.540000 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.340 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540017 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540016 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540021 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 293 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) 2014-08-13 17:38:03.540017 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.540032 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.293 2014-08-13 17:38:03.540032 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] flushed 2014-08-13 17:38:03.540039 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.540050 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540052 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.20 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.540042 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.293 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540066 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540063 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540067 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] flushed 2014-08-13 17:38:03.540071 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 373 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.540073 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.337 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.540083 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.373 2014-08-13 17:38:03.540085 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.363 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.540086 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] flushed 2014-08-13 17:38:03.540097 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540093 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.373 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540111 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540113 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540114 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 396 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.540125 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.396 2014-08-13 17:38:03.540122 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) osd.374 (up) accepted 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.31 (up) accepted 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540127 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] flushed 2014-08-13 17:38:03.540136 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.396 info 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540153 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540138 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,31,374 2014-08-13 17:38:03.540149 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] flushed 2014-08-13 17:38:03.540155 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,374,31] (== acting) backfill_targets 2014-08-13 17:38:03.540157 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 338 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:03.540167 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.540168 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.338 2014-08-13 17:38:03.540168 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540183 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000166 0 0.000000 2014-08-13 17:38:03.540186 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540180 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.338 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540196 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.540207 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540206 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.540209 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] flushed 2014-08-13 17:38:03.540211 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 347 notify: (query_epoch:194485, epoch_sent:194485, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.540218 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000022 0 0.000000 2014-08-13 17:38:03.540225 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.347 2014-08-13 17:38:03.540231 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.540231 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] flushed 2014-08-13 17:38:03.540243 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540235 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.347 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540253 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540252 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540257 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485)) 2014-08-13 17:38:03.540261 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 20 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540268 7fe99ebe8700 7 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.376 2014-08-13 17:38:03.540273 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540275 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.20 2014-08-13 17:38:03.540277 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.376 info 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540295 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540287 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.20 info 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540305 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540299 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 44 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.540309 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 363 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540310 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540313 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540320 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.363 2014-08-13 17:38:03.540328 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362,369 unchanged 2014-08-13 17:38:03.540330 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540343 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540345 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 17:38:03.540347 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MLogRec from 376 2014-08-13 17:38:03.540329 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.363 info 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540360 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] flushed 2014-08-13 17:38:03.540358 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] proc_replica_log for osd.376: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) log((194215'511,194215'511], crt=194215'511) missing(28) 2014-08-13 17:38:03.540373 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540376 7fe99ebe8700 10 proc_replica_log for osd.376: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) log((194215'511,194215'511], crt=194215'511) missing(28) 2014-08-13 17:38:03.540375 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540381 7fe99ebe8700 20 before missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 need 191727'373 have 0'0 2014-08-13 17:38:03.540385 7fe99ebe8700 20 before missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 need 191648'48 have 0'0 2014-08-13 17:38:03.540387 7fe99ebe8700 20 before missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 need 191727'415 have 0'0 2014-08-13 17:38:03.540377 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 337 notify: (query_epoch:194485, epoch_sent:194485, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540389 7fe99ebe8700 20 before missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 need 191648'126 have 0'0 2014-08-13 17:38:03.540391 7fe99ebe8700 20 before missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 need 191727'441 have 0'0 2014-08-13 17:38:03.540397 7fe99ebe8700 20 before missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 need 191727'370 have 0'0 2014-08-13 17:38:03.540399 7fe99ebe8700 20 before missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 need 191727'404 have 0'0 2014-08-13 17:38:03.540390 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.337 2014-08-13 17:38:03.540390 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540401 7fe99ebe8700 20 before missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 need 191648'258 have 0'0 2014-08-13 17:38:03.540403 7fe99ebe8700 20 before missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 need 191727'422 have 0'0 2014-08-13 17:38:03.540405 7fe99ebe8700 20 before missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 need 191648'8 have 0'0 2014-08-13 17:38:03.540407 7fe99ebe8700 20 before missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 need 191727'397 have 0'0 2014-08-13 17:38:03.540409 7fe99ebe8700 20 before missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 need 191648'337 have 0'0 2014-08-13 17:38:03.540411 7fe99ebe8700 20 before missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 need 191727'392 have 0'0 2014-08-13 17:38:03.540412 7fe99ebe8700 20 before missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 need 191648'153 have 0'0 2014-08-13 17:38:03.540406 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540414 7fe99ebe8700 20 before missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 need 191648'320 have 0'0 2014-08-13 17:38:03.540416 7fe99ebe8700 20 before missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 need 191727'384 have 0'0 2014-08-13 17:38:03.540418 7fe99ebe8700 20 before missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 need 191727'371 have 0'0 2014-08-13 17:38:03.540401 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.337 info 22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540420 7fe99ebe8700 20 before missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 need 191727'425 have 0'0 2014-08-13 17:38:03.540422 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540422 7fe99ebe8700 20 before missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 need 191727'412 have 0'0 2014-08-13 17:38:03.540424 7fe99ebe8700 20 before missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 need 191727'375 have 0'0 2014-08-13 17:38:03.540426 7fe99ebe8700 20 before missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 need 191648'189 have 0'0 2014-08-13 17:38:03.540428 7fe99ebe8700 20 before missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 need 191727'424 have 0'0 2014-08-13 17:38:03.540430 7fe99ebe8700 20 before missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 need 191648'304 have 0'0 2014-08-13 17:38:03.540432 7fe99ebe8700 20 before missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 need 191727'411 have 0'0 2014-08-13 17:38:03.540434 7fe99ebe8700 20 before missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 need 191648'58 have 0'0 2014-08-13 17:38:03.540424 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] flushed 2014-08-13 17:38:03.540426 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 32 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.540436 7fe99ebe8700 20 before missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 need 191727'405 have 0'0 2014-08-13 17:38:03.540438 7fe99ebe8700 20 before missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 need 191648'198 have 0'0 2014-08-13 17:38:03.540440 7fe99ebe8700 20 before missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 need 191727'390 have 0'0 2014-08-13 17:38:03.540439 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.32 2014-08-13 17:38:03.540443 7fe99ebe8700 20 merge_log cut point (usually last shared) is 194215'511 (194215'496) delete 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:64502 2014-08-12 17:19:28.143258 2014-08-13 17:38:03.540443 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540448 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.32 info 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540467 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540457 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] flushed 2014-08-13 17:38:03.540457 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] peer osd.376 now 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) missing(28) 2014-08-13 17:38:03.540470 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 289 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.540471 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 need 191727'373 have 0'0 2014-08-13 17:38:03.540473 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] flushed 2014-08-13 17:38:03.540482 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.289 2014-08-13 17:38:03.540482 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 need 191648'48 have 0'0 2014-08-13 17:38:03.540489 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 17:38:03.540494 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 need 191727'415 have 0'0 2014-08-13 17:38:03.540494 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.289 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540513 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540505 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 need 191648'126 have 0'0 2014-08-13 17:38:03.540515 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 need 191727'441 have 0'0 2014-08-13 17:38:03.540515 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] flushed 2014-08-13 17:38:03.540516 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 50 notify: (query_epoch:194485, epoch_sent:194485, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:03.540526 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 need 191727'370 have 0'0 2014-08-13 17:38:03.540529 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.50 2014-08-13 17:38:03.540535 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] flushed 2014-08-13 17:38:03.540537 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 need 191727'404 have 0'0 2014-08-13 17:38:03.540539 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.50 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540547 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 need 191648'258 have 0'0 2014-08-13 17:38:03.540556 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540556 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 need 191727'422 have 0'0 2014-08-13 17:38:03.540560 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 331 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.540566 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 need 191648'8 have 0'0 2014-08-13 17:38:03.540572 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540575 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 need 191727'397 have 0'0 2014-08-13 17:38:03.540585 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 need 191648'337 have 0'0 2014-08-13 17:38:03.540589 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:03.540602 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540594 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 need 191727'392 have 0'0 2014-08-13 17:38:03.540603 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 need 191648'153 have 0'0 2014-08-13 17:38:03.540606 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 379 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540613 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 need 191648'320 have 0'0 2014-08-13 17:38:03.540618 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540623 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 need 191727'384 have 0'0 2014-08-13 17:38:03.540632 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540632 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 need 191727'371 have 0'0 2014-08-13 17:38:03.540636 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 327 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540642 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 need 191727'425 have 0'0 2014-08-13 17:38:03.540647 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.327 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540652 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 need 191727'412 have 0'0 2014-08-13 17:38:03.540660 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540661 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 need 191727'375 have 0'0 2014-08-13 17:38:03.540665 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 381 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.540670 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 need 191648'189 have 0'0 2014-08-13 17:38:03.540679 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 need 191727'424 have 0'0 2014-08-13 17:38:03.540680 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.381 2014-08-13 17:38:03.540688 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 need 191648'304 have 0'0 2014-08-13 17:38:03.540689 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.381 info 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540698 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 need 191727'411 have 0'0 2014-08-13 17:38:03.540707 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540707 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 need 191648'58 have 0'0 2014-08-13 17:38:03.540712 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 309 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:03.540717 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 need 191727'405 have 0'0 2014-08-13 17:38:03.540722 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.309 2014-08-13 17:38:03.540726 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 need 191648'198 have 0'0 2014-08-13 17:38:03.540736 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 need 191727'390 have 0'0 2014-08-13 17:38:03.540737 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.540732 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.309 info 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540750 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540747 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.540754 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 306 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.540761 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.018027 6 0.000445 2014-08-13 17:38:03.540765 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.306 2014-08-13 17:38:03.540773 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.540789 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540774 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.306 info 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540792 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540797 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 289 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540797 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 339 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540807 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.289 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540809 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.339 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540820 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540825 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540828 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.540826 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 337 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540834 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.540838 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540851 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540855 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 287 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:03.540867 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.287 2014-08-13 17:38:03.540876 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540891 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540888 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 369 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540895 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 332 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540903 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540915 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540907 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540919 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540919 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 61 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540923 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 303 notify: (query_epoch:194485, epoch_sent:194485, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:03.540929 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.61 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540934 7fe99f5e9700 7 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.303 2014-08-13 17:38:03.540943 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.540947 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 362 notify: (query_epoch:194485, epoch_sent:194485, info:22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.540943 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540967 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.540958 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.540971 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 376 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:03.540974 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362,369 unchanged 2014-08-13 17:38:03.540987 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.027393 9 0.000608 2014-08-13 17:38:03.540982 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.376 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485), identical to ours 2014-08-13 17:38:03.540998 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.541007 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541012 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 348 notify: (query_epoch:194485, epoch_sent:194485, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.541011 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541023 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541023 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541034 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541040 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:03.541044 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.339 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541053 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetInfo 0.016902 9 0.000647 2014-08-13 17:38:03.541055 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541064 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541066 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.541080 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541090 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) osd.44 (up) accepted 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.362 (up) accepted 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541100 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,44,362 2014-08-13 17:38:03.541092 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541110 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,44,362] (== acting) backfill_targets 2014-08-13 17:38:03.541118 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541121 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.541129 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541135 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000137 0 0.000000 2014-08-13 17:38:03.541148 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.541142 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541158 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.541158 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541170 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000022 0 0.000000 2014-08-13 17:38:03.541181 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.541196 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541192 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541204 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] choose_acting want [4,332,337] != acting [4,348,331], requesting pg_temp change 2014-08-13 17:38:03.541213 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541218 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetLog 0.000153 0 0.000000 2014-08-13 17:38:03.541227 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541237 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541230 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: Leaving Peering 2014-08-13 17:38:03.541240 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering 0.017099 0 0.000000 2014-08-13 17:38:03.541242 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541253 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 17:38:03.541252 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.017121 0 0.000000 2014-08-13 17:38:03.541262 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541264 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary 2014-08-13 17:38:03.541267 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541277 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.541279 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.541274 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering/WaitActingChange 2014-08-13 17:38:03.541281 7fe9a63f4700 20 osd.4 194485 _dispatch 0x27a3600 pg_log(22.481 epoch 194485 log log((194215'519,194215'519], crt=194215'519) query_epoch 194485) v4 2014-08-13 17:38:03.541278 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541286 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541287 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541293 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541304 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541293 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.541313 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541315 7fe9a63f4700 7 osd.4 194485 handle_pg_log pg_log(22.481 epoch 194485 log log((194215'519,194215'519], crt=194215'519) query_epoch 194485) v4 from osd.309 2014-08-13 17:38:03.541319 7fe9a63f4700 15 osd.4 194485 require_same_or_newer_map 194485 (i am 194485) 0x27a3600 2014-08-13 17:38:03.541314 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.541317 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541327 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541337 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541345 7fe9a63f4700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:03.541347 7fe9a63f4700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:03.541341 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541352 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 17:38:03.541361 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541365 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541362 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MNotifyRec from 341 notify: (query_epoch:194485, epoch_sent:194485, info:22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:03.541374 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541383 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541375 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541386 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541391 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341,376 unchanged 2014-08-13 17:38:03.541396 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541405 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541403 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.021418 8 0.000530 2014-08-13 17:38:03.541409 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541417 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:03.541419 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541428 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541431 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541433 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541446 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 17:38:03.541446 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541457 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541459 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.327 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541464 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541471 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541476 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541486 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541482 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.376 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:03.541490 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541500 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 17:38:03.541510 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541508 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) osd.289 (up) accepted 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) osd.341 (up) accepted 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:03.541513 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541519 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,289,341 2014-08-13 17:38:03.541525 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 17:38:03.541535 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541529 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,289,341] (== acting) backfill_targets 2014-08-13 17:38:03.541539 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541541 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 17:38:03.541549 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541558 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541554 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000138 0 0.000000 2014-08-13 17:38:03.541562 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541567 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:03.541573 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 17:38:03.541583 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541577 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: osd.289 has no missing, identical log 2014-08-13 17:38:03.541587 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541591 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.541598 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.541608 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541602 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000036 0 0.000000 2014-08-13 17:38:03.541612 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541614 7fe99f5e9700 5 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.541624 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541622 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541631 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541635 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.541639 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.541640 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541653 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541664 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541668 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541679 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 17:38:03.541689 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541685 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541693 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541698 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541708 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.541706 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:03.541716 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541713 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 MLogRec from 309 2014-08-13 17:38:03.541724 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] proc_replica_log for osd.309: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) log((194215'519,194215'519], crt=194215'519) missing(32) 2014-08-13 17:38:03.541736 7fe99ebe8700 10 proc_replica_log for osd.309: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) log((194215'519,194215'519], crt=194215'519) missing(32) 2014-08-13 17:38:03.541741 7fe99ebe8700 20 before missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 need 191727'405 have 0'0 2014-08-13 17:38:03.541744 7fe99ebe8700 20 before missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 need 191648'214 have 0'0 2014-08-13 17:38:03.541745 7fe99ebe8700 20 before missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 need 191727'410 have 0'0 2014-08-13 17:38:03.541747 7fe99ebe8700 20 before missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 need 191648'366 have 0'0 2014-08-13 17:38:03.541749 7fe99ebe8700 20 before missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 need 191648'245 have 0'0 2014-08-13 17:38:03.541750 7fe99ebe8700 20 before missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 need 191727'415 have 0'0 2014-08-13 17:38:03.541752 7fe99ebe8700 20 before missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 need 191727'445 have 0'0 2014-08-13 17:38:03.541753 7fe99ebe8700 20 before missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 need 191648'179 have 0'0 2014-08-13 17:38:03.541755 7fe99ebe8700 20 before missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 need 191648'128 have 0'0 2014-08-13 17:38:03.541757 7fe99ebe8700 20 before missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 need 191727'447 have 0'0 2014-08-13 17:38:03.541759 7fe99ebe8700 20 before missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 need 191648'355 have 0'0 2014-08-13 17:38:03.541745 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541760 7fe99ebe8700 20 before missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 need 191648'39 have 0'0 2014-08-13 17:38:03.541764 7fe99ebe8700 20 before missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 need 191727'462 have 0'0 2014-08-13 17:38:03.541765 7fe99ebe8700 20 before missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 need 191727'429 have 0'0 2014-08-13 17:38:03.541767 7fe99ebe8700 20 before missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 need 191727'428 have 0'0 2014-08-13 17:38:03.541769 7fe99ebe8700 20 before missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 need 191727'394 have 0'0 2014-08-13 17:38:03.541764 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541771 7fe99ebe8700 20 before missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 need 191727'398 have 0'0 2014-08-13 17:38:03.541773 7fe99ebe8700 20 before missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 need 191648'160 have 0'0 2014-08-13 17:38:03.541774 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541774 7fe99ebe8700 20 before missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 need 191727'403 have 0'0 2014-08-13 17:38:03.541776 7fe99ebe8700 20 before missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 need 191727'455 have 0'0 2014-08-13 17:38:03.541777 7fe99ebe8700 20 before missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 need 191727'401 have 0'0 2014-08-13 17:38:03.541779 7fe99ebe8700 20 before missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 need 191727'425 have 0'0 2014-08-13 17:38:03.541781 7fe99ebe8700 20 before missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 need 191648'385 have 0'0 2014-08-13 17:38:03.541782 7fe99ebe8700 20 before missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 need 191648'32 have 0'0 2014-08-13 17:38:03.541784 7fe99ebe8700 20 before missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 need 191727'448 have 0'0 2014-08-13 17:38:03.541786 7fe99ebe8700 20 before missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 need 191727'457 have 0'0 2014-08-13 17:38:03.541779 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541788 7fe99ebe8700 20 before missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 need 191727'411 have 0'0 2014-08-13 17:38:03.541789 7fe99ebe8700 20 before missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 need 191727'433 have 0'0 2014-08-13 17:38:03.541791 7fe99ebe8700 20 before missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 need 191727'437 have 0'0 2014-08-13 17:38:03.541793 7fe99ebe8700 20 before missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 need 191727'439 have 0'0 2014-08-13 17:38:03.541795 7fe99ebe8700 20 before missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 need 191727'430 have 0'0 2014-08-13 17:38:03.541789 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541796 7fe99ebe8700 20 before missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 need 191648'168 have 0'0 2014-08-13 17:38:03.541799 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541798 7fe99ebe8700 20 merge_log cut point (usually last shared) is 194215'519 (194215'503) delete 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:64338 2014-08-12 17:19:28.088153 2014-08-13 17:38:03.541803 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541813 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541809 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] peer osd.309 now 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) missing(32) 2014-08-13 17:38:03.541822 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541822 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 need 191727'405 have 0'0 2014-08-13 17:38:03.541827 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541832 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 need 191648'214 have 0'0 2014-08-13 17:38:03.541837 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541847 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541842 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 need 191727'410 have 0'0 2014-08-13 17:38:03.541851 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 need 191648'366 have 0'0 2014-08-13 17:38:03.541851 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541861 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 need 191648'245 have 0'0 2014-08-13 17:38:03.541862 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.541872 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541870 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 need 191727'415 have 0'0 2014-08-13 17:38:03.541875 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541880 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 need 191727'445 have 0'0 2014-08-13 17:38:03.541887 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 17:38:03.541889 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 need 191648'179 have 0'0 2014-08-13 17:38:03.541898 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541898 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 need 191648'128 have 0'0 2014-08-13 17:38:03.541902 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541907 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 need 191727'447 have 0'0 2014-08-13 17:38:03.541913 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541922 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541917 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 need 191648'355 have 0'0 2014-08-13 17:38:03.541925 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541926 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 need 191648'39 have 0'0 2014-08-13 17:38:03.541936 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541936 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 need 191727'462 have 0'0 2014-08-13 17:38:03.541945 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541946 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 need 191727'429 have 0'0 2014-08-13 17:38:03.541949 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541955 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 need 191727'428 have 0'0 2014-08-13 17:38:03.541960 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.541970 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541965 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 need 191727'394 have 0'0 2014-08-13 17:38:03.541973 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541975 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 need 191727'398 have 0'0 2014-08-13 17:38:03.541984 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 17:38:03.541994 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.541988 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 need 191648'160 have 0'0 2014-08-13 17:38:03.541998 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.541998 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 need 191727'403 have 0'0 2014-08-13 17:38:03.542009 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.542008 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 need 191727'455 have 0'0 2014-08-13 17:38:03.542018 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542017 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 need 191727'401 have 0'0 2014-08-13 17:38:03.542022 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542016 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542027 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 need 191727'425 have 0'0 2014-08-13 17:38:03.542034 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:03.542045 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542037 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 need 191648'385 have 0'0 2014-08-13 17:38:03.542042 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542047 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 need 191648'32 have 0'0 2014-08-13 17:38:03.542049 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542057 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 need 191727'448 have 0'0 2014-08-13 17:38:03.542063 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.542073 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542067 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 need 191727'457 have 0'0 2014-08-13 17:38:03.542077 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 need 191727'411 have 0'0 2014-08-13 17:38:03.542077 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542086 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 need 191727'433 have 0'0 2014-08-13 17:38:03.542088 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 17:38:03.542098 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542095 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 need 191727'437 have 0'0 2014-08-13 17:38:03.542104 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 need 191727'439 have 0'0 2014-08-13 17:38:03.542105 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542114 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 need 191727'430 have 0'0 2014-08-13 17:38:03.542117 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 17:38:03.542123 7fe99ebe8700 20 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 need 191648'168 have 0'0 2014-08-13 17:38:03.542137 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542133 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 17:38:03.542144 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.008895 6 0.000428 2014-08-13 17:38:03.542142 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542155 7fe99ebe8700 5 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:03.542167 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542159 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 17:38:03.542169 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542171 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542173 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542174 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542219 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542232 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542242 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542240 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542246 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542258 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542267 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542270 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542292 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542299 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542312 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542328 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542339 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542349 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542361 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542354 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542367 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542366 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542371 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542370 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542380 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 17:38:03.542398 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542403 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542411 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542417 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 17:38:03.542427 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542425 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542432 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542438 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542451 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542465 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542477 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542487 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542479 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542491 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542494 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] flushed 2014-08-13 17:38:03.542502 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542513 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542514 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542517 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542528 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542537 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542537 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.542534 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542542 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542553 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542554 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542564 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542566 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542567 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542570 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542572 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542584 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 17:38:03.542594 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.542598 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542602 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.542599 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542611 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542621 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542625 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.542636 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.542644 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.542648 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.542650 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543066 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543082 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543106 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543123 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543133 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543138 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543148 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543158 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543149 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543160 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543164 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543172 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543189 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543206 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543209 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543212 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543254 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543271 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543276 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543288 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543291 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543302 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543294 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543306 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543309 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543308 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543319 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543322 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543337 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543339 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543351 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543361 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543364 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543362 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543376 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543381 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543383 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543388 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543397 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543408 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543402 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543412 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543415 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543414 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543414 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543427 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543427 7fe9aad09700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 17:38:03.543439 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543443 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543446 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543454 7fe99ebe8700 15 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543464 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:03.543459 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543467 7fe99ebe8700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543469 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543470 7fe99ebe8700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.543473 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543484 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543493 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543497 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194485 epoch_requested: 194485 FlushedEvt 2014-08-13 17:38:03.543507 7fe99f5e9700 15 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 17:38:03.543516 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:03.543519 7fe99f5e9700 10 osd.4 194485 queue_want_up_thru want 194485 <= queued 194485, currently 194477 2014-08-13 17:38:03.543522 7fe99f5e9700 10 osd.4 194485 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:03.790665 7fe9a45f1700 10 osd.4 194485 note_peer_epoch osd.292 now has 194485 2014-08-13 17:38:03.790679 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4e3c8e0 already has epoch 194485 2014-08-13 17:38:03.790858 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4e3c8e0 already has epoch 194485 2014-08-13 17:38:03.915884 7fe9a45f1700 10 osd.4 194485 note_peer_epoch osd.321 now has 194485 2014-08-13 17:38:03.915894 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4c73f40 already has epoch 194485 2014-08-13 17:38:03.915967 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4c73f40 already has epoch 194485 2014-08-13 17:38:03.924626 7fe9a45f1700 10 osd.4 194485 note_peer_epoch osd.346 now has 194485 2014-08-13 17:38:03.924640 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4e4a100 already has epoch 194485 2014-08-13 17:38:03.924701 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4e4a100 already has epoch 194485 2014-08-13 17:38:04.022436 7fe9a45f1700 10 osd.4 194485 note_peer_epoch osd.343 now has 194485 2014-08-13 17:38:04.022450 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 already has epoch 194485 2014-08-13 17:38:04.022550 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4d31e40 already has epoch 194485 2014-08-13 17:38:04.186255 7fe9a31ef700 10 osd.4 194485 note_peer_epoch osd.6 now has 194485 2014-08-13 17:38:04.186264 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4e3d800 already has epoch 194485 2014-08-13 17:38:04.186311 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4e3d800 already has epoch 194485 2014-08-13 17:38:04.198451 7fe9a31ef700 10 osd.4 194485 note_peer_epoch osd.294 now has 194485 2014-08-13 17:38:04.198464 7fe9a31ef700 20 osd.4 194485 _share_map_outgoing 0x4cfb2e0 already has epoch 194485 2014-08-13 17:38:04.198700 7fe9a45f1700 20 osd.4 194485 _share_map_outgoing 0x4cfb2e0 already has epoch 194485 2014-08-13 17:38:04.269362 7fe9a77f6700 10 osd.4 194485 do_waiters -- start 2014-08-13 17:38:04.269367 7fe9a77f6700 10 osd.4 194485 do_waiters -- finish 2014-08-13 17:38:04.269369 7fe9a77f6700 20 osd.4 194485 _dispatch 0x5410fc0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.269454 7fe9a77f6700 3 osd.4 194485 handle_osd_map epochs [194486,194486], i have 194485, src has [185139,194486] 2014-08-13 17:38:04.269462 7fe9a77f6700 10 osd.4 194485 handle_osd_map got inc map for epoch 194486 2014-08-13 17:38:04.277603 7fe9a77f6700 10 osd.4 194485 advance to epoch 194486 (<= newest 194486) 2014-08-13 17:38:04.277705 7fe9a77f6700 7 osd.4 194486 advance_map epoch 194486 74 pgs 2014-08-13 17:38:04.277740 7fe9a77f6700 10 osd.4 194486 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194486 [185139,194486] lci=[194485,194486]) 2014-08-13 17:38:04.277824 7fe9a77f6700 0 osd.4 194486 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:38:04.277837 7fe9a77f6700 7 osd.4 194486 consume_map version 194486 2014-08-13 17:38:04.277903 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 17:38:04.277949 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278390 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 2014-08-13 17:38:04.278441 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.278427 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278451 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.278458 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278475 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Started advmap 2014-08-13 17:38:04.278485 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278464 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 17:38:04.278495 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [4,286,375] newacting [4,375,10] 2014-08-13 17:38:04.278512 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.278515 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.278520 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.278534 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary/Peering/WaitActingChange 0.758072 3 0.000251 2014-08-13 17:38:04.278541 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.278557 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.758106 0 0.000000 2014-08-13 17:38:04.278560 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.278577 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started 0.922021 0 0.000000 2014-08-13 17:38:04.278596 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset 2014-08-13 17:38:04.278611 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194485 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.278628 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194486 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Reset advmap 2014-08-13 17:38:04.278640 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.278644 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194486 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.278506 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278658 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.278662 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194486 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [4,286,375] newacting [4,375,10] 2014-08-13 17:38:04.278678 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278679 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.278682 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194486 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.278706 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.278701 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375] r=0 lpr=194486 pi=193640-194484/35 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.278704 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278718 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:04.278723 7fe99f5e9700 7 osd.4 194486 do_notify osd 351 on 1 PGs 2014-08-13 17:38:04.278725 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 17:38:04.278744 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,286,375](4) acting [4,286,375](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,286,375](4) acting [4,286,375](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.278753 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278756 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 17:38:04.278766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] noting past interval(194485-194485 up [4,286,375](4) acting [4,286,375](4)) 2014-08-13 17:38:04.278774 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278788 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] up [4,286,375] -> [4,286,375], acting [4,286,375] -> [4,375,10], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.278797 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278811 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] clear_primary_state 2014-08-13 17:38:04.278842 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 17:38:04.278846 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] agent_stop 2014-08-13 17:38:04.278868 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.278851 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] null 2014-08-13 17:38:04.278869 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change 2014-08-13 17:38:04.278876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.278878 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.278891 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194486:30294 2014-08-13 17:38:04.278897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.278904 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278912 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:04.278917 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.278932 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 17:38:04.278934 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_copy_ops 2014-08-13 17:38:04.278938 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.278953 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_flush_ops 2014-08-13 17:38:04.278956 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278960 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.278971 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:04.278980 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.278981 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.278991 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:04.279003 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2680 2014-08-13 17:38:04.279004 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:04.279012 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:04.279021 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.279030 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] null 2014-08-13 17:38:04.279032 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change_cleanup 2014-08-13 17:38:04.279042 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.756873 3 0.000342 2014-08-13 17:38:04.279052 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change 2014-08-13 17:38:04.279056 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279067 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.279074 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit NotTrimming 2014-08-13 17:38:04.279079 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279087 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.922417 0 0.000000 2014-08-13 17:38:04.279096 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter NotTrimming 2014-08-13 17:38:04.279103 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279116 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] [4,286,375] -> [4,375,10], replicas changed 2014-08-13 17:38:04.279122 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.279126 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279143 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.279136 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_recovery 2014-08-13 17:38:04.279161 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279164 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] clear_recovery_state 2014-08-13 17:38:04.279182 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279181 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.279186 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.279203 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.279205 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map 2014-08-13 17:38:04.279219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'479 -> 194215'479 2014-08-13 17:38:04.279228 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:04.279233 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] update_heartbeat_peers 4,10,286,301,375 -> 4,10,286,375 2014-08-13 17:38:04.279247 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.279267 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.279253 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 17:38:04.279275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] take_waiters 2014-08-13 17:38:04.279278 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279294 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Reset 0.000698 1 0.000759 2014-08-13 17:38:04.279300 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279317 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started 2014-08-13 17:38:04.279320 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 17:38:04.279335 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Start 2014-08-13 17:38:04.279355 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: transitioning to Primary 2014-08-13 17:38:04.279374 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Start 0.000039 0 0.000000 2014-08-13 17:38:04.279342 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279396 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary 2014-08-13 17:38:04.279407 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279416 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Peering 2014-08-13 17:38:04.279436 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.279430 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] null 2014-08-13 17:38:04.279452 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.279475 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279476 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194485-194485 up [4,286,375](4) acting [4,286,375](4)) 2014-08-13 17:38:04.279484 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 sending log((0'0,194215'479], crt=191727'415) 2014-08-13 17:38:04.279501 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279516 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:04.279499 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 17:38:04.279523 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 17:38:04.279538 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] null 2014-08-13 17:38:04.279543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 17:38:04.279554 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 22.362( DNE v 194215'479 lc 0'0 (0'0,194215'479] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 17:38:04.279563 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.279565 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.279581 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.279588 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279588 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194486:30295 2014-08-13 17:38:04.279610 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.10 2014-08-13 17:38:04.279611 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 17:38:04.279632 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.286 2014-08-13 17:38:04.279636 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] null 2014-08-13 17:38:04.279651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.301 2014-08-13 17:38:04.279660 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.279670 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.375 2014-08-13 17:38:04.279691 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.279708 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.279792 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 sending log((0'0,194215'479], crt=191727'415) 2014-08-13 17:38:04.279818 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.279818 7fe99f5e9700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4d31e40 192.168.1.104:6802/61960 2014-08-13 17:38:04.279851 7fe99f5e9700 10 osd.4 194486 note_peer_epoch osd.343 has 194486 2014-08-13 17:38:04.279883 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.362( DNE v 194215'479 lc 0'0 (0'0,194215'479] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 17:38:04.279963 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.58 has 23 missing 2014-08-13 17:38:04.279979 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.343 has 23 missing 2014-08-13 17:38:04.279997 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 is on osd.4 2014-08-13 17:38:04.280016 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 is on osd.4 2014-08-13 17:38:04.280032 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 is on osd.4 2014-08-13 17:38:04.279681 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] null 2014-08-13 17:38:04.280048 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 is on osd.4 2014-08-13 17:38:04.280065 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 is on osd.4 2014-08-13 17:38:04.280061 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 17:38:04.280081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 is on osd.4 2014-08-13 17:38:04.280088 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 17:38:04.280098 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.280111 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.280116 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 is on osd.4 2014-08-13 17:38:04.280134 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 is on osd.4 2014-08-13 17:38:04.280147 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru now 194486 (was 194485), currently 194485 2014-08-13 17:38:04.280167 7fe99ebe8700 10 osd.4 194486 send_alive up_thru currently 194485 want 194486 2014-08-13 17:38:04.280132 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.280170 7fe99ebe8700 10 osd.4 194486 send_alive want 194486 2014-08-13 17:38:04.280151 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 is on osd.4 2014-08-13 17:38:04.280181 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 is on osd.4 2014-08-13 17:38:04.280179 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:04.280203 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:04.280207 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.10 on 1 PGs 2014-08-13 17:38:04.280198 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 is on osd.4 2014-08-13 17:38:04.280210 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.280218 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.280244 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:04.280248 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.286 on 1 PGs 2014-08-13 17:38:04.280236 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.280253 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 is on osd.4 2014-08-13 17:38:04.280271 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing 2014-08-13 17:38:04.280274 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.301 on 1 PGs 2014-08-13 17:38:04.280269 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.280295 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing 2014-08-13 17:38:04.280298 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.375 on 1 PGs 2014-08-13 17:38:04.280286 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 is on osd.4 2014-08-13 17:38:04.280306 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 is on osd.4 2014-08-13 17:38:04.280317 7fe99ebe8700 10 osd.4 194486 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 17:38:04.280322 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 is on osd.4 2014-08-13 17:38:04.280339 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 is on osd.4 2014-08-13 17:38:04.280355 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 is on osd.4 2014-08-13 17:38:04.280371 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 is on osd.4 2014-08-13 17:38:04.280386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 is on osd.4 2014-08-13 17:38:04.280405 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 17:38:04.280432 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.280402 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 17:38:04.280440 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.280449 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate: adding 58 as a source 2014-08-13 17:38:04.280459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.280474 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 also missing on osd.58 2014-08-13 17:38:04.280478 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.280493 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 also missing on osd.58 2014-08-13 17:38:04.280509 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 also missing on osd.58 2014-08-13 17:38:04.280526 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 also missing on osd.58 2014-08-13 17:38:04.280545 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 also missing on osd.58 2014-08-13 17:38:04.280232 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.280558 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.280562 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 also missing on osd.58 2014-08-13 17:38:04.280572 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.280578 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.280582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 also missing on osd.58 2014-08-13 17:38:04.280595 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.280615 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.280602 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 also missing on osd.58 2014-08-13 17:38:04.280599 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] null 2014-08-13 17:38:04.280621 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 also missing on osd.58 2014-08-13 17:38:04.280624 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:04.280639 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 also missing on osd.58 2014-08-13 17:38:04.280641 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 2014-08-13 17:38:04.280657 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 also missing on osd.58 2014-08-13 17:38:04.280673 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.280668 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] null 2014-08-13 17:38:04.280675 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 also missing on osd.58 2014-08-13 17:38:04.280679 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.280693 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 also missing on osd.58 2014-08-13 17:38:04.280694 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] null 2014-08-13 17:38:04.280700 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Started advmap 2014-08-13 17:38:04.280713 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 also missing on osd.58 2014-08-13 17:38:04.280718 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [4,25,362] newacting [4,20,303] 2014-08-13 17:38:04.280733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 also missing on osd.58 2014-08-13 17:38:04.280737 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.280752 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 also missing on osd.58 2014-08-13 17:38:04.280757 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary/Peering/WaitActingChange 0.747537 1 0.000250 2014-08-13 17:38:04.280769 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 also missing on osd.58 2014-08-13 17:38:04.280778 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.747569 0 0.000000 2014-08-13 17:38:04.280787 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 also missing on osd.58 2014-08-13 17:38:04.280799 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started 0.850191 0 0.000000 2014-08-13 17:38:04.280805 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 also missing on osd.58 2014-08-13 17:38:04.280822 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset 2014-08-13 17:38:04.280823 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 also missing on osd.58 2014-08-13 17:38:04.280840 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194485 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194486 2014-08-13 17:38:04.280841 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 also missing on osd.58 2014-08-13 17:38:04.280857 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194486 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Reset advmap 2014-08-13 17:38:04.280859 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 also missing on osd.58 2014-08-13 17:38:04.280873 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194486 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.280877 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 also missing on osd.58 2014-08-13 17:38:04.280890 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194486 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [4,25,362] newacting [4,20,303] 2014-08-13 17:38:04.280907 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194486 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.280892 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 17:38:04.280923 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362] r=0 lpr=194486 pi=193635-194484/32 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194486 2014-08-13 17:38:04.280937 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate: adding 343 as a source 2014-08-13 17:38:04.280954 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 also missing on osd.343 2014-08-13 17:38:04.280959 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,25,362](4) acting [4,25,362](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,25,362](4) acting [4,25,362](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.280972 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 also missing on osd.343 2014-08-13 17:38:04.280984 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194485/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] noting past interval(194485-194485 up [4,25,362](4) acting [4,25,362](4)) 2014-08-13 17:38:04.280989 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 also missing on osd.343 2014-08-13 17:38:04.281007 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 also missing on osd.343 2014-08-13 17:38:04.281005 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] up [4,25,362] -> [4,25,362], acting [4,25,362] -> [4,20,303], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.281024 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 also missing on osd.343 2014-08-13 17:38:04.281027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] clear_primary_state 2014-08-13 17:38:04.281042 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 also missing on osd.343 2014-08-13 17:38:04.281049 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] agent_stop 2014-08-13 17:38:04.280728 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281060 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 also missing on osd.343 2014-08-13 17:38:04.281070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change 2014-08-13 17:38:04.281081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 also missing on osd.343 2014-08-13 17:38:04.281091 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194486:29703 2014-08-13 17:38:04.281099 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 also missing on osd.343 2014-08-13 17:38:04.281110 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:04.281118 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 also missing on osd.343 2014-08-13 17:38:04.281130 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_copy_ops 2014-08-13 17:38:04.281135 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 also missing on osd.343 2014-08-13 17:38:04.281147 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_flush_ops 2014-08-13 17:38:04.281153 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 also missing on osd.343 2014-08-13 17:38:04.281175 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:04.281178 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 also missing on osd.343 2014-08-13 17:38:04.281176 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281194 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:04.281197 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 also missing on osd.343 2014-08-13 17:38:04.281211 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:04.281217 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 also missing on osd.343 2014-08-13 17:38:04.281216 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281230 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change_cleanup 2014-08-13 17:38:04.281237 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 also missing on osd.343 2014-08-13 17:38:04.281249 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change 2014-08-13 17:38:04.281250 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281259 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 also missing on osd.343 2014-08-13 17:38:04.281271 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit NotTrimming 2014-08-13 17:38:04.281277 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281278 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 also missing on osd.343 2014-08-13 17:38:04.281291 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter NotTrimming 2014-08-13 17:38:04.281302 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 also missing on osd.343 2014-08-13 17:38:04.281302 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] null 2014-08-13 17:38:04.281312 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] [4,25,362] -> [4,20,303], replicas changed 2014-08-13 17:38:04.281322 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 also missing on osd.343 2014-08-13 17:38:04.281329 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 17:38:04.281334 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_recovery 2014-08-13 17:38:04.281342 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 also missing on osd.343 2014-08-13 17:38:04.281353 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] clear_recovery_state 2014-08-13 17:38:04.281363 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 also missing on osd.343 2014-08-13 17:38:04.281361 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] null 2014-08-13 17:38:04.281374 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.281384 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 also missing on osd.343 2014-08-13 17:38:04.281388 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281400 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map 2014-08-13 17:38:04.281412 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281425 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] update_heartbeat_peers 4,20,25,303,362,373 -> 4,20,25,303,362 2014-08-13 17:38:04.281401 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 17:38:04.281448 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.281451 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] take_waiters 2014-08-13 17:38:04.281454 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.281435 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281471 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.281470 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Reset 0.000648 1 0.000720 2014-08-13 17:38:04.281479 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281492 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 58,61,343,349,362 2014-08-13 17:38:04.281495 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started 2014-08-13 17:38:04.281502 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] null 2014-08-13 17:38:04.281513 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.281515 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Start 2014-08-13 17:38:04.281529 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281535 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.281536 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: transitioning to Primary 2014-08-13 17:38:04.281553 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 17:38:04.281556 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.281559 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Start 0.000044 0 0.000000 2014-08-13 17:38:04.281574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.281574 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 17:38:04.281583 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary 2014-08-13 17:38:04.281601 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 17:38:04.281603 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Peering 2014-08-13 17:38:04.281621 7fe9a77f6700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] null 2014-08-13 17:38:04.281623 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.281645 7fe9a77f6700 10 osd.4 194486 maybe_update_heartbeat_peers updating 2014-08-13 17:38:04.281649 7fe9a77f6700 20 osd.4 194486 24.30b heartbeat_peers 4,10,286,375 2014-08-13 17:38:04.281641 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.281660 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194485-194485 up [4,25,362](4) acting [4,25,362](4)) 2014-08-13 17:38:04.281685 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.281679 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 17:38:04.281700 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 17:38:04.281718 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 17:38:04.281742 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.281763 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194486:29704 2014-08-13 17:38:04.281780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.20 2014-08-13 17:38:04.281799 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.25 2014-08-13 17:38:04.281816 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.303 2014-08-13 17:38:04.281832 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.362 2014-08-13 17:38:04.281849 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.373 2014-08-13 17:38:04.281867 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.281888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.281906 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.10 192.168.1.115:6816/60573 192.168.100.115:6834/60573 2014-08-13 17:38:04.281991 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.282090 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 17:38:04.282123 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.282130 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.282154 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.282182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.282166 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 17:38:04.282246 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.282247 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.282255 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.282265 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.282281 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.282281 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.282301 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.282303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.282332 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 17:38:04.282352 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.282357 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.282373 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.282388 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.282412 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.282434 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.282458 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.282471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.282476 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.282491 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.282492 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.282508 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.282546 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 17:38:04.282571 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.282578 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.282573 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.282598 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.282630 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.282633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.282658 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.282675 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.282779 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.282802 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:3019 2014-08-13 17:38:04.282818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.282835 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.751484 6 0.000426 2014-08-13 17:38:04.282903 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.282931 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.840478 0 0.000000 2014-08-13 17:38:04.282927 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 17:38:04.282963 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.282963 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.282971 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.282981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194477 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.282991 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.283009 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.283006 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.283027 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.283028 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.283047 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.283050 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'510 -> 194215'510 2014-08-13 17:38:04.283065 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.283074 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.283084 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2829 2014-08-13 17:38:04.283101 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.283117 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.751032 5 0.000361 2014-08-13 17:38:04.283137 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.283153 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.915814 0 0.000000 2014-08-13 17:38:04.283192 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.283208 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194477 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.283229 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.283231 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.375 192.168.1.114:6831/52879 192.168.100.114:6832/52879 2014-08-13 17:38:04.283245 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.283258 7fe9a77f6700 20 osd.4 194486 22.220 heartbeat_peers 2014-08-13 17:38:04.283263 7fe9a77f6700 20 osd.4 194486 22.362 heartbeat_peers 4,58,61,343,349,362 2014-08-13 17:38:04.283259 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'514 -> 194215'514 2014-08-13 17:38:04.283370 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.58 192.168.1.111:6835/14316 192.168.100.111:6829/14316 2014-08-13 17:38:04.283388 7fe9a77f6700 20 osd.4 194486 22.28f heartbeat_peers 2014-08-13 17:38:04.283392 7fe9a77f6700 20 osd.4 194486 22.14a heartbeat_peers 2014-08-13 17:38:04.283394 7fe9a77f6700 20 osd.4 194486 22.22c heartbeat_peers 4,301,302,324,368 2014-08-13 17:38:04.283485 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.302 192.168.1.113:6811/15731 192.168.100.113:6834/15731 2014-08-13 17:38:04.283283 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.283572 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.368 192.168.1.102:6827/15861 192.168.100.102:6811/15861 2014-08-13 17:38:04.283587 7fe9a77f6700 20 osd.4 194486 22.619 heartbeat_peers 4,44,48,62,65,314 2014-08-13 17:38:04.283666 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.44 192.168.1.111:6823/10782 192.168.100.111:6802/10782 2014-08-13 17:38:04.283757 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.48 192.168.1.112:6839/8300 192.168.100.112:6842/8300 2014-08-13 17:38:04.283853 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.62 192.168.1.112:6825/5505 192.168.100.112:6810/5505 2014-08-13 17:38:04.283855 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 sending log((0'0,194215'514], crt=191727'446) 2014-08-13 17:38:04.283999 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.65 192.168.1.111:6843/17345 192.168.100.111:6842/17345 2014-08-13 17:38:04.284013 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing 2014-08-13 17:38:04.284058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 22.22c( DNE v 194215'514 lc 0'0 (0'0,194215'514] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(38) 2014-08-13 17:38:04.284095 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:04.284114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 is up to date, queueing in pending_activators 2014-08-13 17:38:04.284159 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) uptodate 2014-08-13 17:38:04.284224 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.301 has 38 missing 2014-08-13 17:38:04.284229 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.314 192.168.1.101:6803/22303 192.168.100.101:6803/22303 2014-08-13 17:38:04.284251 7fe9a77f6700 20 osd.4 194486 22.30f heartbeat_peers 2014-08-13 17:38:04.284242 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.284255 7fe9a77f6700 20 osd.4 194486 24.9b heartbeat_peers 4,291,350,365,367,378 2014-08-13 17:38:04.284261 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 is on osd.4 2014-08-13 17:38:04.284277 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 is on osd.4 2014-08-13 17:38:04.284297 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.284313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.284333 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 is on osd.4 2014-08-13 17:38:04.284344 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.365 192.168.1.102:6825/15623 192.168.100.102:6806/15623 2014-08-13 17:38:04.284348 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 is on osd.4 2014-08-13 17:38:04.284361 7fe9a77f6700 20 osd.4 194486 22.581 heartbeat_peers 2014-08-13 17:38:04.284365 7fe9a77f6700 20 osd.4 194486 22.f3 heartbeat_peers 4,285,311,312,369 2014-08-13 17:38:04.284364 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 is on osd.4 2014-08-13 17:38:04.284381 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.284401 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 is on osd.4 2014-08-13 17:38:04.284416 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 is on osd.4 2014-08-13 17:38:04.284436 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 is on osd.4 2014-08-13 17:38:04.284445 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.311 192.168.1.101:6817/24270 192.168.100.101:6831/24270 2014-08-13 17:38:04.284455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 is on osd.4 2014-08-13 17:38:04.284471 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 is on osd.4 2014-08-13 17:38:04.284489 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 is on osd.4 2014-08-13 17:38:04.284506 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 is on osd.4 2014-08-13 17:38:04.284521 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 is on osd.4 2014-08-13 17:38:04.284541 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 is on osd.4 2014-08-13 17:38:04.284545 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.369 192.168.1.108:6825/4037 192.168.100.108:6814/4037 2014-08-13 17:38:04.284565 7fe9a77f6700 20 osd.4 194486 22.116 heartbeat_peers 2014-08-13 17:38:04.284568 7fe9a77f6700 20 osd.4 194486 24.3d heartbeat_peers 4,20,25,303,362 2014-08-13 17:38:04.284557 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 is on osd.4 2014-08-13 17:38:04.284574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 is on osd.4 2014-08-13 17:38:04.284589 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 is on osd.4 2014-08-13 17:38:04.284604 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 is on osd.4 2014-08-13 17:38:04.284620 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 is on osd.4 2014-08-13 17:38:04.284635 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 is on osd.4 2014-08-13 17:38:04.284651 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 is on osd.4 2014-08-13 17:38:04.284666 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.284681 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 is on osd.4 2014-08-13 17:38:04.284696 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 is on osd.4 2014-08-13 17:38:04.284711 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 is on osd.4 2014-08-13 17:38:04.284726 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 is on osd.4 2014-08-13 17:38:04.284741 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 is on osd.4 2014-08-13 17:38:04.284759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 is on osd.4 2014-08-13 17:38:04.284779 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 is on osd.4 2014-08-13 17:38:04.284794 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 is on osd.4 2014-08-13 17:38:04.284809 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.284827 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 is on osd.4 2014-08-13 17:38:04.284843 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 is on osd.4 2014-08-13 17:38:04.284858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 is on osd.4 2014-08-13 17:38:04.284892 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.373 192.168.1.108:6841/7435 192.168.100.108:6850/7435 2014-08-13 17:38:04.284912 7fe9a77f6700 20 osd.4 194486 22.4aa heartbeat_peers 2014-08-13 17:38:04.284916 7fe9a77f6700 20 osd.4 194486 22.1d2 heartbeat_peers 4,58,61,330,331,349 2014-08-13 17:38:04.284878 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 17:38:04.284957 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate: adding 301 as a source 2014-08-13 17:38:04.284978 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 also missing on osd.301 2014-08-13 17:38:04.284999 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 also missing on osd.301 2014-08-13 17:38:04.285000 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.330 192.168.1.106:6803/4876 192.168.100.106:6807/4876 2014-08-13 17:38:04.285017 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 also missing on osd.301 2014-08-13 17:38:04.285034 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 also missing on osd.301 2014-08-13 17:38:04.285051 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 also missing on osd.301 2014-08-13 17:38:04.285067 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 also missing on osd.301 2014-08-13 17:38:04.285083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 also missing on osd.301 2014-08-13 17:38:04.285099 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 also missing on osd.301 2014-08-13 17:38:04.285115 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 also missing on osd.301 2014-08-13 17:38:04.285132 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 also missing on osd.301 2014-08-13 17:38:04.285148 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 also missing on osd.301 2014-08-13 17:38:04.285172 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 also missing on osd.301 2014-08-13 17:38:04.285189 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 also missing on osd.301 2014-08-13 17:38:04.285205 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 also missing on osd.301 2014-08-13 17:38:04.285222 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 also missing on osd.301 2014-08-13 17:38:04.285238 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 also missing on osd.301 2014-08-13 17:38:04.285254 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 also missing on osd.301 2014-08-13 17:38:04.285271 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 also missing on osd.301 2014-08-13 17:38:04.285287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 also missing on osd.301 2014-08-13 17:38:04.285304 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 also missing on osd.301 2014-08-13 17:38:04.285320 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 also missing on osd.301 2014-08-13 17:38:04.285337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 also missing on osd.301 2014-08-13 17:38:04.285353 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 also missing on osd.301 2014-08-13 17:38:04.285369 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 also missing on osd.301 2014-08-13 17:38:04.285386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 also missing on osd.301 2014-08-13 17:38:04.285402 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 also missing on osd.301 2014-08-13 17:38:04.285418 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 also missing on osd.301 2014-08-13 17:38:04.285435 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 also missing on osd.301 2014-08-13 17:38:04.285451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 also missing on osd.301 2014-08-13 17:38:04.285467 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 also missing on osd.301 2014-08-13 17:38:04.285483 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 also missing on osd.301 2014-08-13 17:38:04.285500 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 also missing on osd.301 2014-08-13 17:38:04.285516 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 also missing on osd.301 2014-08-13 17:38:04.285532 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 also missing on osd.301 2014-08-13 17:38:04.285548 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 also missing on osd.301 2014-08-13 17:38:04.285564 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 also missing on osd.301 2014-08-13 17:38:04.285580 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 also missing on osd.301 2014-08-13 17:38:04.285597 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 also missing on osd.301 2014-08-13 17:38:04.285611 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 17:38:04.285676 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate: adding 324 as a source 2014-08-13 17:38:04.285697 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 is on osd.324 2014-08-13 17:38:04.285705 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.331 192.168.1.104:6817/837 192.168.100.104:6843/837 2014-08-13 17:38:04.285715 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 is on osd.324 2014-08-13 17:38:04.285735 7fe9a77f6700 20 osd.4 194486 24.1de heartbeat_peers 2014-08-13 17:38:04.285741 7fe9a77f6700 20 osd.4 194486 2.2a heartbeat_peers 4,324,340,370 2014-08-13 17:38:04.285737 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 is on osd.324 2014-08-13 17:38:04.285756 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 is on osd.324 2014-08-13 17:38:04.285776 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 is on osd.324 2014-08-13 17:38:04.285794 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 is on osd.324 2014-08-13 17:38:04.285731 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 sending log((0'0,194215'510], crt=191648'372) 2014-08-13 17:38:04.285811 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 is on osd.324 2014-08-13 17:38:04.285828 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 is on osd.324 2014-08-13 17:38:04.285846 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 is on osd.324 2014-08-13 17:38:04.285869 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 is on osd.324 2014-08-13 17:38:04.285886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 is on osd.324 2014-08-13 17:38:04.285902 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 is on osd.324 2014-08-13 17:38:04.285919 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:04.285912 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 2014-08-13 17:38:04.285919 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 is on osd.324 2014-08-13 17:38:04.285936 7fe9a77f6700 20 osd.4 194486 22.113 heartbeat_peers 4,24,45,55,339,342 2014-08-13 17:38:04.285944 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 is on osd.324 2014-08-13 17:38:04.285961 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 is on osd.324 2014-08-13 17:38:04.285978 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 is on osd.324 2014-08-13 17:38:04.285979 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 22.1d2( DNE v 194215'510 lc 0'0 (0'0,194215'510] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(28) 2014-08-13 17:38:04.285996 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 is on osd.324 2014-08-13 17:38:04.286008 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.286017 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 is on osd.324 2014-08-13 17:38:04.286037 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 is on osd.324 2014-08-13 17:38:04.286053 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 is on osd.324 2014-08-13 17:38:04.286074 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 is on osd.324 2014-08-13 17:38:04.286084 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.24 192.168.1.107:6801/52669 192.168.100.107:6803/52669 2014-08-13 17:38:04.286091 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 is on osd.324 2014-08-13 17:38:04.286108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 is on osd.324 2014-08-13 17:38:04.286124 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 is on osd.324 2014-08-13 17:38:04.286145 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 is on osd.324 2014-08-13 17:38:04.286168 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 is on osd.324 2014-08-13 17:38:04.286185 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 is on osd.324 2014-08-13 17:38:04.286206 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 is on osd.324 2014-08-13 17:38:04.286224 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 is on osd.324 2014-08-13 17:38:04.286242 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.55 192.168.1.112:6835/6966 192.168.100.112:6836/6966 2014-08-13 17:38:04.286245 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 is on osd.324 2014-08-13 17:38:04.286250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 sending log((0'0,194215'510], crt=191648'372) 2014-08-13 17:38:04.286263 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 is on osd.324 2014-08-13 17:38:04.286281 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 is on osd.324 2014-08-13 17:38:04.286297 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 is on osd.324 2014-08-13 17:38:04.286314 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 is on osd.324 2014-08-13 17:38:04.286330 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 is on osd.324 2014-08-13 17:38:04.286347 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 is on osd.324 2014-08-13 17:38:04.286363 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 is on osd.324 2014-08-13 17:38:04.286379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 is on osd.324 2014-08-13 17:38:04.288582 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5380c60 don't know epoch, doing nothing 2014-08-13 17:38:04.288576 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.342 192.168.1.110:6803/41438 192.168.100.110:6810/41438 2014-08-13 17:38:04.288603 7fe9a77f6700 20 osd.4 194486 24.39a heartbeat_peers 2014-08-13 17:38:04.288610 7fe9a77f6700 20 osd.4 194486 24.33a heartbeat_peers 2014-08-13 17:38:04.288615 7fe9a77f6700 20 osd.4 194486 1.2b heartbeat_peers 4,324,340,370 2014-08-13 17:38:04.288621 7fe9a77f6700 20 osd.4 194486 24.34 heartbeat_peers 2014-08-13 17:38:04.288623 7fe9a77f6700 20 osd.4 194486 22.99 heartbeat_peers 4,20,31,337,363,374 2014-08-13 17:38:04.288646 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 22.1d2( DNE v 194215'510 lc 0'0 (0'0,194215'510] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(28) 2014-08-13 17:38:04.288752 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.61 has 28 missing 2014-08-13 17:38:04.288775 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.349 has 28 missing 2014-08-13 17:38:04.288790 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.288808 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 is on osd.4 2014-08-13 17:38:04.288825 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 is on osd.4 2014-08-13 17:38:04.288841 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 is on osd.4 2014-08-13 17:38:04.288857 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 is on osd.4 2014-08-13 17:38:04.288873 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 is on osd.4 2014-08-13 17:38:04.288888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 is on osd.4 2014-08-13 17:38:04.288904 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 is on osd.4 2014-08-13 17:38:04.288920 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 is on osd.4 2014-08-13 17:38:04.288935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 is on osd.4 2014-08-13 17:38:04.288950 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 is on osd.4 2014-08-13 17:38:04.288966 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 is on osd.4 2014-08-13 17:38:04.288981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 is on osd.4 2014-08-13 17:38:04.288996 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 is on osd.4 2014-08-13 17:38:04.289014 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.289029 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 is on osd.4 2014-08-13 17:38:04.289045 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 is on osd.4 2014-08-13 17:38:04.289060 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.289075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 is on osd.4 2014-08-13 17:38:04.289090 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 is on osd.4 2014-08-13 17:38:04.289106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 is on osd.4 2014-08-13 17:38:04.289121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 is on osd.4 2014-08-13 17:38:04.289136 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 is on osd.4 2014-08-13 17:38:04.289151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 is on osd.4 2014-08-13 17:38:04.289166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 is on osd.4 2014-08-13 17:38:04.289182 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 is on osd.4 2014-08-13 17:38:04.289197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 is on osd.4 2014-08-13 17:38:04.289223 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.286396 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 17:38:04.290752 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.31 192.168.1.103:6832/41157 192.168.100.103:6850/41157 2014-08-13 17:38:04.290768 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.290796 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 17:38:04.290815 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 301,302,324,368 2014-08-13 17:38:04.290832 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.290849 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.290874 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.290890 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.290985 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.289240 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 17:38:04.291323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate: adding 61 as a source 2014-08-13 17:38:04.291355 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 also missing on osd.61 2014-08-13 17:38:04.291375 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 also missing on osd.61 2014-08-13 17:38:04.291392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 also missing on osd.61 2014-08-13 17:38:04.291409 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 also missing on osd.61 2014-08-13 17:38:04.291425 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 also missing on osd.61 2014-08-13 17:38:04.291441 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 also missing on osd.61 2014-08-13 17:38:04.291458 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 also missing on osd.61 2014-08-13 17:38:04.291474 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 also missing on osd.61 2014-08-13 17:38:04.291490 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 also missing on osd.61 2014-08-13 17:38:04.291506 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 also missing on osd.61 2014-08-13 17:38:04.291522 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 also missing on osd.61 2014-08-13 17:38:04.291539 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 also missing on osd.61 2014-08-13 17:38:04.291555 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 also missing on osd.61 2014-08-13 17:38:04.291571 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 also missing on osd.61 2014-08-13 17:38:04.291587 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 also missing on osd.61 2014-08-13 17:38:04.291603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 also missing on osd.61 2014-08-13 17:38:04.291619 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 also missing on osd.61 2014-08-13 17:38:04.291635 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 also missing on osd.61 2014-08-13 17:38:04.291652 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 also missing on osd.61 2014-08-13 17:38:04.291668 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 also missing on osd.61 2014-08-13 17:38:04.291684 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 also missing on osd.61 2014-08-13 17:38:04.291700 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 also missing on osd.61 2014-08-13 17:38:04.291716 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 also missing on osd.61 2014-08-13 17:38:04.291732 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 also missing on osd.61 2014-08-13 17:38:04.291748 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 also missing on osd.61 2014-08-13 17:38:04.291764 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 also missing on osd.61 2014-08-13 17:38:04.291780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 also missing on osd.61 2014-08-13 17:38:04.291796 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 also missing on osd.61 2014-08-13 17:38:04.291810 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 17:38:04.291860 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate: adding 349 as a source 2014-08-13 17:38:04.291877 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 also missing on osd.349 2014-08-13 17:38:04.291893 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 also missing on osd.349 2014-08-13 17:38:04.291910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 also missing on osd.349 2014-08-13 17:38:04.291926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 also missing on osd.349 2014-08-13 17:38:04.291942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 also missing on osd.349 2014-08-13 17:38:04.291958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 also missing on osd.349 2014-08-13 17:38:04.291974 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 also missing on osd.349 2014-08-13 17:38:04.291990 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 also missing on osd.349 2014-08-13 17:38:04.292006 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 also missing on osd.349 2014-08-13 17:38:04.292022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 also missing on osd.349 2014-08-13 17:38:04.292038 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 also missing on osd.349 2014-08-13 17:38:04.292054 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 also missing on osd.349 2014-08-13 17:38:04.292070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 also missing on osd.349 2014-08-13 17:38:04.292086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 also missing on osd.349 2014-08-13 17:38:04.292102 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 also missing on osd.349 2014-08-13 17:38:04.292118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 also missing on osd.349 2014-08-13 17:38:04.292134 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 also missing on osd.349 2014-08-13 17:38:04.292150 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 also missing on osd.349 2014-08-13 17:38:04.292166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 also missing on osd.349 2014-08-13 17:38:04.292192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 also missing on osd.349 2014-08-13 17:38:04.292208 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 also missing on osd.349 2014-08-13 17:38:04.292225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 also missing on osd.349 2014-08-13 17:38:04.292241 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 also missing on osd.349 2014-08-13 17:38:04.292257 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 also missing on osd.349 2014-08-13 17:38:04.292273 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 also missing on osd.349 2014-08-13 17:38:04.292289 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 also missing on osd.349 2014-08-13 17:38:04.292305 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 also missing on osd.349 2014-08-13 17:38:04.292321 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 also missing on osd.349 2014-08-13 17:38:04.292335 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 17:38:04.292384 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.292397 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 17:38:04.292415 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 58,61,330,331,349 2014-08-13 17:38:04.292431 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.292448 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.292465 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.292481 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.292578 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.294954 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.337 192.168.1.104:6805/62197 192.168.100.104:6816/62197 2014-08-13 17:38:04.295024 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 17:38:04.295070 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.295079 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.295099 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.295116 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.295135 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.295164 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.295140 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 2014-08-13 17:38:04.295184 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.295197 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.295203 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2732 2014-08-13 17:38:04.295205 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Started advmap 2014-08-13 17:38:04.295219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.295225 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,301] 2014-08-13 17:38:04.295238 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.761126 3 0.000367 2014-08-13 17:38:04.295242 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.295260 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.295262 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started/Stray 0.851185 4 0.000172 2014-08-13 17:38:04.295277 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.910065 0 0.000000 2014-08-13 17:38:04.295284 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started 0.851250 0 0.000000 2014-08-13 17:38:04.295301 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.295305 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset 2014-08-13 17:38:04.295318 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.295321 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194485 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194486 2014-08-13 17:38:04.295339 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194486 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 17:38:04.295340 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.295356 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194486 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.295358 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.295373 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'511 -> 194215'511 2014-08-13 17:38:04.295373 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194486 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,301] 2014-08-13 17:38:04.295392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194486 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.295400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.295409 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360] r=1 lpr=194486 pi=193125-194484/33 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194486 2014-08-13 17:38:04.295444 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [364,4,360](364) acting [364,4,360](364)): not rw, up_thru 194483 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [364,4,360](364) acting [364,4,360](364)) : primary up 192966-194483 does not include interval 2014-08-13 17:38:04.295461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194485/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194485-194485 up [364,4,360](364) acting [364,4,360](364)) 2014-08-13 17:38:04.295479 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,4,360] -> [364,4,360], acting [364,4,360] -> [364,4,301], acting_primary 364 -> 364, up_primary 364 -> 364, role 1 -> 1 2014-08-13 17:38:04.295500 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:04.295518 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.295505 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.295534 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:04.295545 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.295558 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 17:38:04.295575 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 17:38:04.295596 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 17:38:04.295612 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 17:38:04.295631 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 17:38:04.295648 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 17:38:04.295663 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 17:38:04.295678 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 17:38:04.295696 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.295715 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map 2014-08-13 17:38:04.295798 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] update_heartbeat_peers unchanged 2014-08-13 17:38:04.295818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters 2014-08-13 17:38:04.295835 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Reset 0.000530 1 0.000511 2014-08-13 17:38:04.295854 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started 2014-08-13 17:38:04.295871 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Start 2014-08-13 17:38:04.295886 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: transitioning to Stray 2014-08-13 17:38:04.295910 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Start 0.000039 0 0.000000 2014-08-13 17:38:04.295930 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/Stray 2014-08-13 17:38:04.295952 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.295968 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.296351 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.296387 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.296919 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 sending log((0'0,194215'511], crt=191727'439) 2014-08-13 17:38:04.297035 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.363 192.168.1.102:6843/20017 192.168.100.102:6847/20017 2014-08-13 17:38:04.297038 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a4dd80 don't know epoch, doing nothing 2014-08-13 17:38:04.297178 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.297163 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 22.619( DNE v 194215'511 lc 0'0 (0'0,194215'511] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 17:38:04.297239 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:04.297270 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.297267 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.297278 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.297297 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.297313 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.297329 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.297347 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.297363 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.297380 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:176849 2014-08-13 17:38:04.297394 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.297411 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.765569 4 0.000315 2014-08-13 17:38:04.297429 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.297444 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.852113 0 0.000000 2014-08-13 17:38:04.297470 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.297485 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.297505 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.297521 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.297534 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'930 -> 194215'930 2014-08-13 17:38:04.297616 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.297672 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] flushed 2014-08-13 17:38:04.297767 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.374 192.168.1.102:6829/16129 192.168.100.102:6817/16129 2014-08-13 17:38:04.297792 7fe9a77f6700 20 osd.4 194486 22.84 heartbeat_peers 4,7,28,304 2014-08-13 17:38:04.297800 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 sending log((0'0,194215'511], crt=191727'439) 2014-08-13 17:38:04.298085 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 2014-08-13 17:38:04.298107 7fe9a77f6700 20 osd.4 194486 22.4df heartbeat_peers 4,352,356,357,371,380 2014-08-13 17:38:04.298319 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.352 192.168.1.108:6827/4273 192.168.100.108:6823/4273 2014-08-13 17:38:04.300389 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing 2014-08-13 17:38:04.300384 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.371 192.168.1.102:6833/16753 192.168.100.102:6832/16753 2014-08-13 17:38:04.300440 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 22.619( DNE v 194215'511 lc 0'0 (0'0,194215'511] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 17:38:04.300545 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.62 has 29 missing 2014-08-13 17:38:04.300566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.65 has 29 missing 2014-08-13 17:38:04.300582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 is on osd.4 2014-08-13 17:38:04.300600 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 is on osd.4 2014-08-13 17:38:04.300617 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 is on osd.4 2014-08-13 17:38:04.300634 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 is on osd.4 2014-08-13 17:38:04.300649 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.300665 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 is on osd.4 2014-08-13 17:38:04.300681 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 is on osd.4 2014-08-13 17:38:04.300697 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 is on osd.4 2014-08-13 17:38:04.300713 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 is on osd.4 2014-08-13 17:38:04.300729 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 is on osd.4 2014-08-13 17:38:04.300744 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.300759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 is on osd.4 2014-08-13 17:38:04.300775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 is on osd.4 2014-08-13 17:38:04.300794 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 is on osd.4 2014-08-13 17:38:04.300809 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 is on osd.4 2014-08-13 17:38:04.300825 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 is on osd.4 2014-08-13 17:38:04.300841 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 is on osd.4 2014-08-13 17:38:04.300856 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 is on osd.4 2014-08-13 17:38:04.300871 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.300886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 is on osd.4 2014-08-13 17:38:04.300902 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 is on osd.4 2014-08-13 17:38:04.300917 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 is on osd.4 2014-08-13 17:38:04.300932 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 is on osd.4 2014-08-13 17:38:04.300947 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 is on osd.4 2014-08-13 17:38:04.300961 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 is on osd.4 2014-08-13 17:38:04.300977 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 is on osd.4 2014-08-13 17:38:04.300995 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.301014 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 is on osd.4 2014-08-13 17:38:04.301029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 is on osd.4 2014-08-13 17:38:04.301044 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 17:38:04.301381 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate: adding 62 as a source 2014-08-13 17:38:04.301416 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 also missing on osd.62 2014-08-13 17:38:04.301440 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 also missing on osd.62 2014-08-13 17:38:04.301458 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 also missing on osd.62 2014-08-13 17:38:04.301475 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 also missing on osd.62 2014-08-13 17:38:04.301492 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 also missing on osd.62 2014-08-13 17:38:04.301509 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 also missing on osd.62 2014-08-13 17:38:04.301526 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 also missing on osd.62 2014-08-13 17:38:04.301543 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 also missing on osd.62 2014-08-13 17:38:04.301561 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 also missing on osd.62 2014-08-13 17:38:04.301578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 also missing on osd.62 2014-08-13 17:38:04.301594 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 also missing on osd.62 2014-08-13 17:38:04.301610 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 also missing on osd.62 2014-08-13 17:38:04.301628 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 also missing on osd.62 2014-08-13 17:38:04.301644 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 also missing on osd.62 2014-08-13 17:38:04.301661 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 also missing on osd.62 2014-08-13 17:38:04.301678 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 also missing on osd.62 2014-08-13 17:38:04.301694 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 also missing on osd.62 2014-08-13 17:38:04.301710 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 also missing on osd.62 2014-08-13 17:38:04.301726 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 also missing on osd.62 2014-08-13 17:38:04.301742 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 also missing on osd.62 2014-08-13 17:38:04.301759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 also missing on osd.62 2014-08-13 17:38:04.301775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 also missing on osd.62 2014-08-13 17:38:04.301791 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 also missing on osd.62 2014-08-13 17:38:04.301807 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 also missing on osd.62 2014-08-13 17:38:04.301823 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 also missing on osd.62 2014-08-13 17:38:04.301840 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 also missing on osd.62 2014-08-13 17:38:04.301856 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 also missing on osd.62 2014-08-13 17:38:04.301872 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 also missing on osd.62 2014-08-13 17:38:04.301889 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 also missing on osd.62 2014-08-13 17:38:04.301903 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 17:38:04.302107 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate: adding 65 as a source 2014-08-13 17:38:04.302179 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 also missing on osd.65 2014-08-13 17:38:04.302215 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 also missing on osd.65 2014-08-13 17:38:04.302252 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 also missing on osd.65 2014-08-13 17:38:04.302283 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 also missing on osd.65 2014-08-13 17:38:04.302301 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 also missing on osd.65 2014-08-13 17:38:04.302318 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 also missing on osd.65 2014-08-13 17:38:04.302335 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 also missing on osd.65 2014-08-13 17:38:04.302356 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 also missing on osd.65 2014-08-13 17:38:04.302372 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 also missing on osd.65 2014-08-13 17:38:04.302417 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 also missing on osd.65 2014-08-13 17:38:04.302445 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.380 192.168.1.114:6823/51844 192.168.100.114:6803/51844 2014-08-13 17:38:04.302450 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 also missing on osd.65 2014-08-13 17:38:04.302472 7fe9a77f6700 20 osd.4 194486 22.39d heartbeat_peers 4,320,344,363,366,373 2014-08-13 17:38:04.302471 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 also missing on osd.65 2014-08-13 17:38:04.302489 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 also missing on osd.65 2014-08-13 17:38:04.302540 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 also missing on osd.65 2014-08-13 17:38:04.302578 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.320 192.168.1.106:6821/10050 192.168.100.106:6849/10050 2014-08-13 17:38:04.302580 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 also missing on osd.65 2014-08-13 17:38:04.302758 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.344 192.168.1.106:6813/6757 192.168.100.106:6837/6757 2014-08-13 17:38:04.302686 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 also missing on osd.65 2014-08-13 17:38:04.302866 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 also missing on osd.65 2014-08-13 17:38:04.302924 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.366 192.168.1.108:6829/4534 192.168.100.108:6832/4534 2014-08-13 17:38:04.303014 7fe9a77f6700 20 osd.4 194486 22.2b9 heartbeat_peers 2014-08-13 17:38:04.303023 7fe9a77f6700 20 osd.4 194486 24.2c heartbeat_peers 2014-08-13 17:38:04.303026 7fe9a77f6700 20 osd.4 194486 24.3f7 heartbeat_peers 2014-08-13 17:38:04.303028 7fe9a77f6700 20 osd.4 194486 22.775 heartbeat_peers 0,4,56,63,353,369 2014-08-13 17:38:04.303002 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 also missing on osd.65 2014-08-13 17:38:04.303125 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.0 192.168.1.115:6811/59615 192.168.100.115:6825/59615 2014-08-13 17:38:04.303050 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 also missing on osd.65 2014-08-13 17:38:04.303236 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.56 192.168.1.111:6827/11446 192.168.100.111:6811/11446 2014-08-13 17:38:04.303197 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 also missing on osd.65 2014-08-13 17:38:04.303258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 also missing on osd.65 2014-08-13 17:38:04.303329 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 also missing on osd.65 2014-08-13 17:38:04.303370 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.63 192.168.1.111:6831/12680 192.168.100.111:6820/12680 2014-08-13 17:38:04.303354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 also missing on osd.65 2014-08-13 17:38:04.303411 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 also missing on osd.65 2014-08-13 17:38:04.303458 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 also missing on osd.65 2014-08-13 17:38:04.303473 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.353 192.168.1.108:6837/6227 192.168.100.108:6844/6227 2014-08-13 17:38:04.303494 7fe9a77f6700 20 osd.4 194486 22.630 heartbeat_peers 2014-08-13 17:38:04.303498 7fe9a77f6700 20 osd.4 194486 24.e2 heartbeat_peers 2014-08-13 17:38:04.303501 7fe9a77f6700 20 osd.4 194486 22.69f heartbeat_peers 2014-08-13 17:38:04.303503 7fe9a77f6700 20 osd.4 194486 22.3e4 heartbeat_peers 4,45,303,313,318,342 2014-08-13 17:38:04.303486 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 also missing on osd.65 2014-08-13 17:38:04.303578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 also missing on osd.65 2014-08-13 17:38:04.303742 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.313 192.168.1.109:6821/35445 192.168.100.109:6850/35445 2014-08-13 17:38:04.303669 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 also missing on osd.65 2014-08-13 17:38:04.303798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 also missing on osd.65 2014-08-13 17:38:04.304071 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 17:38:04.304253 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.304328 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.304376 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 44,48,62,65,314 2014-08-13 17:38:04.304397 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.304430 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.304450 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.304468 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.304445 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 sending log((0'0,194215'930], crt=0'0) 2014-08-13 17:38:04.304623 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.304618 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.318 192.168.1.110:6815/43857 192.168.100.110:6840/43857 2014-08-13 17:38:04.304637 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.304644 7fe9a77f6700 20 osd.4 194486 24.2a1 heartbeat_peers 2014-08-13 17:38:04.304649 7fe9a77f6700 20 osd.4 194486 22.639 heartbeat_peers 2014-08-13 17:38:04.304652 7fe9a77f6700 20 osd.4 194486 22.707 heartbeat_peers 4,15,21,303,310,342 2014-08-13 17:38:04.304670 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 2.2a( DNE v 194215'930 (0'0,194215'930] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) uptodate 2014-08-13 17:38:04.304699 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.304722 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.304744 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.304762 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.304778 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.304841 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.304895 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 17:38:04.304943 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.304954 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.304975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.304993 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.305054 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.305075 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.305091 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.305120 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.305316 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.305360 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.305679 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.15 192.168.1.107:6814/53408 192.168.100.107:6819/53408 2014-08-13 17:38:04.308116 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 2014-08-13 17:38:04.308134 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 17:38:04.308170 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.308184 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.308180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.308193 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.21 192.168.1.107:6822/54786 192.168.100.107:6838/54786 2014-08-13 17:38:04.308195 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.308199 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.308218 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Started advmap 2014-08-13 17:38:04.308226 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.308236 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,378,367] newacting [4,291,350] 2014-08-13 17:38:04.308250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.308258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.308272 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.308278 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.774644 1 0.000250 2014-08-13 17:38:04.308293 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.308307 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.310 192.168.1.109:6813/32413 192.168.100.109:6833/32413 2014-08-13 17:38:04.308305 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.774680 0 0.000000 2014-08-13 17:38:04.308314 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2932 2014-08-13 17:38:04.308327 7fe9a77f6700 20 osd.4 194486 24.5e heartbeat_peers 2014-08-13 17:38:04.308333 7fe9a77f6700 20 osd.4 194486 22.1f1 heartbeat_peers 4,5,11,47,54,368 2014-08-13 17:38:04.308328 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.908082 0 0.000000 2014-08-13 17:38:04.308332 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.308350 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 17:38:04.308351 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.774752 4 0.000362 2014-08-13 17:38:04.308372 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194485 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194486 2014-08-13 17:38:04.308378 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.308391 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194486 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 17:38:04.308395 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.856350 0 0.000000 2014-08-13 17:38:04.308408 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194486 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.308420 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.308424 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194486 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,378,367] newacting [4,291,350] 2014-08-13 17:38:04.308441 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194477 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.308446 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194486 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.308449 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.11 192.168.1.103:6814/37929 192.168.100.103:6821/37929 2014-08-13 17:38:04.308463 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367] r=0 lpr=194486 pi=193637-194484/30 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194486 2014-08-13 17:38:04.308465 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.308482 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.308501 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'528 -> 194215'528 2014-08-13 17:38:04.308508 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,378,367](4) acting [4,378,367](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,378,367](4) acting [4,378,367](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.308527 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.308531 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194485/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] noting past interval(194485-194485 up [4,378,367](4) acting [4,378,367](4)) 2014-08-13 17:38:04.308542 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.47 192.168.1.111:6829/12102 192.168.100.111:6815/12102 2014-08-13 17:38:04.308552 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] up [4,378,367] -> [4,378,367], acting [4,378,367] -> [4,291,350], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.308573 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] clear_primary_state 2014-08-13 17:38:04.308598 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] agent_stop 2014-08-13 17:38:04.308619 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 17:38:04.308638 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194486:25286 2014-08-13 17:38:04.308660 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.308668 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.54 192.168.1.111:6837/14923 192.168.100.111:6833/14923 2014-08-13 17:38:04.308689 7fe9a77f6700 20 osd.4 194486 22.2d1 heartbeat_peers 2014-08-13 17:38:04.308679 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 17:38:04.308693 7fe9a77f6700 20 osd.4 194486 24.b6 heartbeat_peers 4,17,28,315,327,330 2014-08-13 17:38:04.308695 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 17:38:04.308711 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.308726 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.308745 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.308761 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 17:38:04.308771 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 sending log((0'0,194215'528], crt=191648'383) 2014-08-13 17:38:04.308778 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 17:38:04.308803 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 17:38:04.308820 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 17:38:04.308847 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:04.308837 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] [4,378,367] -> [4,291,350], replicas changed 2014-08-13 17:38:04.308836 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.17 192.168.1.103:6812/37658 192.168.100.103:6818/37658 2014-08-13 17:38:04.308857 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_recovery 2014-08-13 17:38:04.308873 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] clear_recovery_state 2014-08-13 17:38:04.308893 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.308892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 22.113( DNE v 194215'528 lc 0'0 (0'0,194215'528] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(27) 2014-08-13 17:38:04.308915 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map 2014-08-13 17:38:04.308919 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.308938 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] update_heartbeat_peers 4,291,350,365,367,378 -> 4,291,350,367,378 2014-08-13 17:38:04.309252 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.315 192.168.1.113:6801/14161 192.168.100.113:6801/14161 2014-08-13 17:38:04.309263 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.309297 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.309425 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 sending log((0'0,194215'528], crt=191648'383) 2014-08-13 17:38:04.310407 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.327 192.168.1.106:6805/5112 192.168.100.106:6813/5112 2014-08-13 17:38:04.310442 7fe9a77f6700 20 osd.4 194486 24.3bf heartbeat_peers 2014-08-13 17:38:04.310437 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ec8160 don't know epoch, doing nothing 2014-08-13 17:38:04.310447 7fe9a77f6700 20 osd.4 194486 22.1fb heartbeat_peers 2014-08-13 17:38:04.310450 7fe9a77f6700 20 osd.4 194486 24.350 heartbeat_peers 2014-08-13 17:38:04.310453 7fe9a77f6700 20 osd.4 194486 24.1f7 heartbeat_peers 4,55,59,351,376 2014-08-13 17:38:04.310483 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 22.113( DNE v 194215'528 lc 0'0 (0'0,194215'528] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(27) 2014-08-13 17:38:04.310584 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.45 has 27 missing 2014-08-13 17:38:04.310603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.339 has 27 missing 2014-08-13 17:38:04.310622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 is on osd.4 2014-08-13 17:38:04.310640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 is on osd.4 2014-08-13 17:38:04.310646 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.59 192.168.1.112:6823/5265 192.168.100.112:6804/5265 2014-08-13 17:38:04.310657 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 is on osd.4 2014-08-13 17:38:04.310673 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 is on osd.4 2014-08-13 17:38:04.310691 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 is on osd.4 2014-08-13 17:38:04.310712 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 is on osd.4 2014-08-13 17:38:04.310727 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 is on osd.4 2014-08-13 17:38:04.310747 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 is on osd.4 2014-08-13 17:38:04.310762 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 is on osd.4 2014-08-13 17:38:04.310782 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 is on osd.4 2014-08-13 17:38:04.310797 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 is on osd.4 2014-08-13 17:38:04.310812 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 is on osd.4 2014-08-13 17:38:04.310828 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 is on osd.4 2014-08-13 17:38:04.310843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 is on osd.4 2014-08-13 17:38:04.310862 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.310878 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 is on osd.4 2014-08-13 17:38:04.310893 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 is on osd.4 2014-08-13 17:38:04.310909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 is on osd.4 2014-08-13 17:38:04.310924 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 is on osd.4 2014-08-13 17:38:04.310944 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 is on osd.4 2014-08-13 17:38:04.310959 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.310974 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 is on osd.4 2014-08-13 17:38:04.310989 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 is on osd.4 2014-08-13 17:38:04.311006 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 is on osd.4 2014-08-13 17:38:04.311022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 is on osd.4 2014-08-13 17:38:04.311037 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 is on osd.4 2014-08-13 17:38:04.311053 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 is on osd.4 2014-08-13 17:38:04.311068 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 17:38:04.311118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate: adding 45 as a source 2014-08-13 17:38:04.311136 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 also missing on osd.45 2014-08-13 17:38:04.311153 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 also missing on osd.45 2014-08-13 17:38:04.311173 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 also missing on osd.45 2014-08-13 17:38:04.311190 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 also missing on osd.45 2014-08-13 17:38:04.311217 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 also missing on osd.45 2014-08-13 17:38:04.311234 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 also missing on osd.45 2014-08-13 17:38:04.311236 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.351 192.168.1.102:6831/16444 192.168.100.102:6826/16444 2014-08-13 17:38:04.311251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 also missing on osd.45 2014-08-13 17:38:04.311267 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 also missing on osd.45 2014-08-13 17:38:04.311284 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 also missing on osd.45 2014-08-13 17:38:04.311305 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 also missing on osd.45 2014-08-13 17:38:04.311322 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 also missing on osd.45 2014-08-13 17:38:04.311339 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 also missing on osd.45 2014-08-13 17:38:04.311355 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 also missing on osd.45 2014-08-13 17:38:04.311371 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 also missing on osd.45 2014-08-13 17:38:04.311387 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 also missing on osd.45 2014-08-13 17:38:04.311403 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 also missing on osd.45 2014-08-13 17:38:04.311428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 also missing on osd.45 2014-08-13 17:38:04.311447 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 also missing on osd.45 2014-08-13 17:38:04.311463 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 also missing on osd.45 2014-08-13 17:38:04.311479 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 also missing on osd.45 2014-08-13 17:38:04.311498 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 also missing on osd.45 2014-08-13 17:38:04.311520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 also missing on osd.45 2014-08-13 17:38:04.311536 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 also missing on osd.45 2014-08-13 17:38:04.311553 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 also missing on osd.45 2014-08-13 17:38:04.311569 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 also missing on osd.45 2014-08-13 17:38:04.311585 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 also missing on osd.45 2014-08-13 17:38:04.311601 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 also missing on osd.45 2014-08-13 17:38:04.311616 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 17:38:04.311672 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate: adding 339 as a source 2014-08-13 17:38:04.311690 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 also missing on osd.339 2014-08-13 17:38:04.311706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 also missing on osd.339 2014-08-13 17:38:04.311722 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 also missing on osd.339 2014-08-13 17:38:04.311738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 also missing on osd.339 2014-08-13 17:38:04.311758 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 also missing on osd.339 2014-08-13 17:38:04.311779 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 also missing on osd.339 2014-08-13 17:38:04.311796 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 also missing on osd.339 2014-08-13 17:38:04.311812 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 also missing on osd.339 2014-08-13 17:38:04.311808 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.376 192.168.1.108:6839/6893 192.168.100.108:6847/6893 2014-08-13 17:38:04.311831 7fe9a77f6700 20 osd.4 194486 24.136 heartbeat_peers 4,50,55,61,65,356 2014-08-13 17:38:04.311828 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 also missing on osd.339 2014-08-13 17:38:04.311840 7fe9a77f6700 20 osd.4 194486 22.4fe heartbeat_peers 2014-08-13 17:38:04.311847 7fe9a77f6700 20 osd.4 194486 22.653 heartbeat_peers 2014-08-13 17:38:04.311849 7fe9a77f6700 20 osd.4 194486 24.3b7 heartbeat_peers 2014-08-13 17:38:04.311853 7fe9a77f6700 20 osd.4 194486 22.3b9 heartbeat_peers 2014-08-13 17:38:04.311855 7fe9a77f6700 20 osd.4 194486 22.791 heartbeat_peers 2014-08-13 17:38:04.311858 7fe9a77f6700 20 osd.4 194486 22.6bd heartbeat_peers 4,32,50,289 2014-08-13 17:38:04.311848 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 also missing on osd.339 2014-08-13 17:38:04.311867 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 also missing on osd.339 2014-08-13 17:38:04.311883 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 also missing on osd.339 2014-08-13 17:38:04.311904 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 also missing on osd.339 2014-08-13 17:38:04.311920 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 also missing on osd.339 2014-08-13 17:38:04.311936 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 also missing on osd.339 2014-08-13 17:38:04.311953 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 also missing on osd.339 2014-08-13 17:38:04.311973 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 also missing on osd.339 2014-08-13 17:38:04.311982 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 2014-08-13 17:38:04.312004 7fe9a77f6700 20 osd.4 194486 24.181 heartbeat_peers 2014-08-13 17:38:04.312009 7fe9a77f6700 20 osd.4 194486 24.20e heartbeat_peers 2014-08-13 17:38:04.311997 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 also missing on osd.339 2014-08-13 17:38:04.312012 7fe9a77f6700 20 osd.4 194486 24.1e4 heartbeat_peers 2014-08-13 17:38:04.312014 7fe9a77f6700 20 osd.4 194486 24.1e6 heartbeat_peers 2014-08-13 17:38:04.312018 7fe9a77f6700 20 osd.4 194486 22.12a heartbeat_peers 2014-08-13 17:38:04.312020 7fe9a77f6700 20 osd.4 194486 22.3af heartbeat_peers 4,44,61,339,362,369 2014-08-13 17:38:04.312014 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 also missing on osd.339 2014-08-13 17:38:04.312028 7fe9a77f6700 20 osd.4 194486 22.481 heartbeat_peers 4,287,303,306,309,365,381 2014-08-13 17:38:04.312032 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 also missing on osd.339 2014-08-13 17:38:04.312049 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 also missing on osd.339 2014-08-13 17:38:04.312070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 also missing on osd.339 2014-08-13 17:38:04.312086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 also missing on osd.339 2014-08-13 17:38:04.312102 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 also missing on osd.339 2014-08-13 17:38:04.312118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 also missing on osd.339 2014-08-13 17:38:04.312134 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 also missing on osd.339 2014-08-13 17:38:04.312149 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 also missing on osd.339 2014-08-13 17:38:04.312191 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.306 192.168.1.101:6807/6812 192.168.100.101:6805/6812 2014-08-13 17:38:04.312168 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 17:38:04.312221 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.312238 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.312258 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 24,45,55,339,342 2014-08-13 17:38:04.312276 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.312293 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.381 192.168.1.102:6823/15390 192.168.100.102:6802/15390 2014-08-13 17:38:04.312295 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.312311 7fe9a77f6700 20 osd.4 194486 22.48e heartbeat_peers 4,289,327,341,376 2014-08-13 17:38:04.312312 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.312328 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.312432 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.312451 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.341 192.168.1.106:6819/9142 192.168.100.106:6846/9142 2014-08-13 17:38:04.312474 7fe9a77f6700 20 osd.4 194486 22.3ad heartbeat_peers 2014-08-13 17:38:04.312479 7fe9a77f6700 20 osd.4 194486 24.206 heartbeat_peers 2014-08-13 17:38:04.312482 7fe9a77f6700 20 osd.4 194486 24.1bb heartbeat_peers 4,331,332,337,348,379 2014-08-13 17:38:04.312555 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:04.312599 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.312609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:04.312631 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.312631 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.332 192.168.1.110:6807/42024 192.168.100.110:6825/42024 2014-08-13 17:38:04.312651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:04.312770 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:04.312791 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194486 2014-08-13 17:38:04.312807 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.312825 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.313023 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.313057 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.313614 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.348 192.168.1.110:6801/41204 192.168.100.110:6804/41204 2014-08-13 17:38:04.313672 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 2014-08-13 17:38:04.313718 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.313726 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Started advmap 2014-08-13 17:38:04.313743 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323,4] 2014-08-13 17:38:04.313760 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.313779 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started/Stray 0.859572 4 0.000216 2014-08-13 17:38:04.313807 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started 0.859651 0 0.000000 2014-08-13 17:38:04.313835 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Reset 2014-08-13 17:38:04.313852 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194485 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194486 2014-08-13 17:38:04.313870 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194486 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 17:38:04.313890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194486 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.313906 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194486 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323,4] 2014-08-13 17:38:04.313922 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194486 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.313938 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4]/[304,323] r=-1 lpr=194486 pi=193250-194484/30 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194486 2014-08-13 17:38:04.313976 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [304,323,4](304) acting [304,323](304)): not rw, up_thru 194483 up_from 192965 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [304,323,4](304) acting [304,323](304)) : primary up 192965-194483 does not include interval 2014-08-13 17:38:04.313993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194485/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194485-194485 up [304,323,4](304) acting [304,323](304)) 2014-08-13 17:38:04.314015 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323,4] -> [304,323,4], acting [304,323] -> [304,323,4], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> 2 2014-08-13 17:38:04.314034 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:04.314050 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.314069 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:04.314086 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.379 192.168.1.108:6831/4861 192.168.100.108:6835/4861 2014-08-13 17:38:04.314089 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 17:38:04.314106 7fe9a77f6700 20 osd.4 194486 24.299 heartbeat_peers 2014-08-13 17:38:04.314111 7fe9a77f6700 20 osd.4 194486 22.3d5 heartbeat_peers 4,10,286,343 2014-08-13 17:38:04.314105 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 17:38:04.314119 7fe9a77f6700 20 osd.4 194486 22.537 heartbeat_peers 2014-08-13 17:38:04.314122 7fe9a77f6700 20 osd.4 194486 22.1b8 heartbeat_peers 4,7,340,347 2014-08-13 17:38:04.314129 7fe9a77f6700 20 osd.4 194486 22.29a heartbeat_peers 2014-08-13 17:38:04.314132 7fe9a77f6700 20 osd.4 194486 24.cd heartbeat_peers 2014-08-13 17:38:04.314134 7fe9a77f6700 20 osd.4 194486 22.226 heartbeat_peers 2014-08-13 17:38:04.314121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 17:38:04.314137 7fe9a77f6700 20 osd.4 194486 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 2014-08-13 17:38:04.314138 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 17:38:04.314156 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 17:38:04.314172 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 17:38:04.314187 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_role_change 2014-08-13 17:38:04.314214 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:04.314234 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 17:38:04.314248 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 17:38:04.314265 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.314294 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map 2014-08-13 17:38:04.314298 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.293 192.168.1.109:6817/33855 192.168.100.109:6844/33855 2014-08-13 17:38:04.314369 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] update_heartbeat_peers unchanged 2014-08-13 17:38:04.314391 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters 2014-08-13 17:38:04.314408 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.338 192.168.1.106:6815/25893 192.168.100.106:6802/25893 2014-08-13 17:38:04.314409 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.000574 1 0.000562 2014-08-13 17:38:04.314429 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started 2014-08-13 17:38:04.314444 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start 2014-08-13 17:38:04.314463 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray 2014-08-13 17:38:04.314482 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.314505 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray 2014-08-13 17:38:04.314506 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.382 192.168.1.108:6801/30527 192.168.100.108:6801/30527 2014-08-13 17:38:04.314526 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.314546 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.314593 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.396 192.168.1.105:6815/41806 192.168.100.105:6817/41806 2014-08-13 17:38:04.314645 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.314720 7fe9a77f6700 10 osd.4 194486 _add_heartbeat_peer: new peer osd.397 192.168.1.105:6809/41085 192.168.100.105:6802/41085 2014-08-13 17:38:04.314739 7fe9a77f6700 20 osd.4 194486 24.2ec heartbeat_peers 2014-08-13 17:38:04.314743 7fe9a77f6700 20 osd.4 194486 22.1a0 heartbeat_peers 2014-08-13 17:38:04.314747 7fe9a77f6700 20 osd.4 194486 24.369 heartbeat_peers 2014-08-13 17:38:04.314751 7fe9a77f6700 10 osd.4 194486 adding neighbor peer osd.3 2014-08-13 17:38:04.314754 7fe9a77f6700 10 osd.4 194486 adding neighbor peer osd.5 2014-08-13 17:38:04.314763 7fe9a77f6700 10 osd.4 194486 maybe_update_heartbeat_peers 91 peers, extras 3,5 2014-08-13 17:38:04.314774 7fe9a77f6700 7 osd.4 194486 activate_map version 194486 2014-08-13 17:38:04.314786 7fe9a77f6700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314789 7fe9a77f6700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314814 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314825 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314827 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54386c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.314837 7fe9986dd700 20 osd.4 194486 agent_entry empty queue 2014-08-13 17:38:04.314841 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.314809 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:04.314843 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.20 now has 194486 2014-08-13 17:38:04.314847 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.314850 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314851 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314851 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.314854 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4c71b80 already has epoch 194486 2014-08-13 17:38:04.314864 7fe9a77f6700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314870 7fe9a77f6700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314872 7fe9a77f6700 20 osd.4 194486 _dispatch 0x54860c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.314861 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.314886 7fe9a77f6700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.314895 7fe9a77f6700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.314898 7fe9a77f6700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.314900 7fe9a77f6700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314903 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.314921 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.314939 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.314952 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4c71b80 already has epoch 194486 2014-08-13 17:38:04.314961 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.314984 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.314992 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.314982 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:177953 2014-08-13 17:38:04.314996 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ccb40 pg_notify(22.6bd(25) epoch 194486) v5 2014-08-13 17:38:04.315001 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.315019 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.783407 4 0.000314 2014-08-13 17:38:04.315036 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.315051 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.289 2014-08-13 17:38:04.315055 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ccb40 2014-08-13 17:38:04.315042 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.315060 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.859570 0 0.000000 2014-08-13 17:38:04.315050 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] take_waiters 2014-08-13 17:38:04.315087 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.315083 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.006732 1 0.000722 2014-08-13 17:38:04.315112 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315115 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315105 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194477 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.315110 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started 2014-08-13 17:38:04.315128 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.315129 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Start 2014-08-13 17:38:04.315145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.315157 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315160 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315162 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9de80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315147 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary 2014-08-13 17:38:04.315160 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'1672 -> 194215'1672 2014-08-13 17:38:04.315177 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315182 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315184 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315185 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315198 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000069 0 0.000000 2014-08-13 17:38:04.315220 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315223 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315226 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db7740 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315231 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315235 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315220 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary 2014-08-13 17:38:04.315237 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315239 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315241 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering 2014-08-13 17:38:04.315259 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315262 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315263 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9af40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315268 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315272 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315274 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315259 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.315276 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315278 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.315295 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315297 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315299 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db6300 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315304 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315308 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315310 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315312 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315300 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194485-194485 up [4,378,367](4) acting [4,378,367](4)) 2014-08-13 17:38:04.315308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.315337 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315339 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315342 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9aac0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315326 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 17:38:04.315348 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315352 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315354 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315356 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315351 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194482-194482 up [365,350,291](365) acting [291,350](291)) 2014-08-13 17:38:04.315375 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315377 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315379 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d94380 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315384 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315388 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315371 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 17:38:04.315390 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315392 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315392 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.315410 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315412 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315414 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db0900 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315419 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315423 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315425 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315427 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315415 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194486:25287 2014-08-13 17:38:04.315445 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315447 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315449 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5410240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315435 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.291 2014-08-13 17:38:04.315454 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315458 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315461 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315462 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315457 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.350 2014-08-13 17:38:04.315482 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315484 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315485 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9f80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315476 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.365 2014-08-13 17:38:04.315490 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315495 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315497 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315499 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315496 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.367 2014-08-13 17:38:04.315527 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.378 2014-08-13 17:38:04.315545 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315547 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315551 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d8240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315559 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315564 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315566 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315568 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315547 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.315593 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315595 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315598 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5460900 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.315604 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315608 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315610 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315612 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315631 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315633 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315634 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db4140 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315639 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315644 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315645 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315647 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315664 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315666 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315668 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d94c80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315673 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315676 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315678 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315680 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315695 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.315697 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315700 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315701 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9d7c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315706 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315714 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315725 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315727 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315746 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315748 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315749 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db7080 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315755 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315759 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315761 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315763 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315751 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 17:38:04.315779 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315781 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315782 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.315783 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db69c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315791 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315796 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315798 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315799 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315790 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.315817 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315819 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315824 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5460d80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315809 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.315831 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315835 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315837 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315839 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315833 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.315858 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315860 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315861 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5498480 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315866 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315870 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315872 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315873 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315890 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315892 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315893 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5410d80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315898 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315903 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315905 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315906 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315900 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.315930 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315918 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.315932 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315934 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db18c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315945 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315949 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315952 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315953 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315934 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.315959 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.315973 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315975 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.315976 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d97740 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.315981 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.315985 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.315987 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315989 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316007 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.315987 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 17:38:04.316009 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316011 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d94800 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316011 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.316016 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316020 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316023 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316026 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316018 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.316043 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316045 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316047 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9d340 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316038 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.316052 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316056 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316058 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316060 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316056 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.316079 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316083 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d97080 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316074 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.316088 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316093 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316095 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316097 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316093 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.316114 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316116 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316118 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db4ec0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316123 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316111 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.316127 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316129 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316130 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316129 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2859 2014-08-13 17:38:04.316148 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316151 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316152 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d93a80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316158 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316162 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316164 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316166 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316146 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.316183 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316185 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316186 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5480480 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316192 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316196 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316179 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.783809 6 0.000396 2014-08-13 17:38:04.316197 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316199 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316201 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.316217 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.900796 0 0.000000 2014-08-13 17:38:04.316239 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316241 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316243 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d93600 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316248 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316252 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316254 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316256 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316249 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.316274 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316276 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316277 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d960c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316266 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.316284 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316288 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316289 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316291 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316290 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.316314 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316316 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316318 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d96540 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.316324 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316329 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316331 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316332 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316350 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316352 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316353 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5438240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316358 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316363 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316364 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316323 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'506 -> 194215'506 2014-08-13 17:38:04.316383 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316385 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316387 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db3180 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316392 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316395 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316398 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316400 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316383 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.316416 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316418 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316356 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] flushed 2014-08-13 17:38:04.316420 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db2640 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316425 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316429 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316431 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316432 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316449 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316451 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316452 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54da400 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316457 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316461 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316463 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316464 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316481 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316483 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316485 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d99b00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316490 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316494 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316495 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316497 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316513 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316515 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316516 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54da880 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316521 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316525 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316527 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316528 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316545 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316546 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316548 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d91f80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316553 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316557 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316558 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316560 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316577 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316579 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316581 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5450240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316586 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316589 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316591 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316593 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316609 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316611 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316613 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d8d80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316617 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316621 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316623 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316624 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316640 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316642 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316644 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d92d00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316649 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316653 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316654 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316656 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316671 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316673 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316675 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d92400 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316662 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 sending log((0'0,194215'506], crt=191648'343) 2014-08-13 17:38:04.316687 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:04.316680 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316692 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316694 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316696 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316712 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316714 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316716 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5428240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.316985 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.316990 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.316992 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.316994 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.316718 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 22.f3( DNE v 194215'506 lc 0'0 (0'0,194215'506] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 17:38:04.317013 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317016 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317017 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d95100 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317023 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317007 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) 2014-08-13 17:38:04.317026 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317029 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317031 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317027 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 is up to date, queueing in pending_activators 2014-08-13 17:38:04.317065 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317067 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317068 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9200 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317073 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317077 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317079 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317097 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317099 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317101 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9b00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317087 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) uptodate 2014-08-13 17:38:04.317106 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317110 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317112 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317113 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317130 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317132 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317134 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dbcc0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317139 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317143 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317145 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317131 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.285 has 29 missing 2014-08-13 17:38:04.317147 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317166 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317168 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317169 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54db840 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317148 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 is on osd.4 2014-08-13 17:38:04.317174 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317179 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317180 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317182 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317176 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 is on osd.4 2014-08-13 17:38:04.317200 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317195 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 is on osd.4 2014-08-13 17:38:04.317210 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317212 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dad00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317218 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317222 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317224 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317226 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317212 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 is on osd.4 2014-08-13 17:38:04.317244 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317231 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 is on osd.4 2014-08-13 17:38:04.317246 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317248 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dca40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317254 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317258 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317260 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317262 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317249 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 is on osd.4 2014-08-13 17:38:04.317279 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317281 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317283 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dc5c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317274 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 is on osd.4 2014-08-13 17:38:04.317288 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317292 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317294 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317296 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317291 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.317314 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317316 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317318 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54ddc40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317309 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 is on osd.4 2014-08-13 17:38:04.317323 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317327 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317330 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317331 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317326 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.317348 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317350 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317352 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54de780 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317343 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 is on osd.4 2014-08-13 17:38:04.317357 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317362 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317364 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317360 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.317383 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317386 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317387 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54de0c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317378 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 is on osd.4 2014-08-13 17:38:04.317393 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317397 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317399 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317401 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 is on osd.4 2014-08-13 17:38:04.317411 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.317425 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317428 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317430 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54df080 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317435 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317439 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 is on osd.4 2014-08-13 17:38:04.317441 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317443 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317444 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 is on osd.4 2014-08-13 17:38:04.317459 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317472 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317475 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5481680 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317480 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317484 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317471 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.317486 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317491 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317489 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 is on osd.4 2014-08-13 17:38:04.317508 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317511 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317513 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54df980 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317506 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 is on osd.4 2014-08-13 17:38:04.317518 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317522 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317524 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317527 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317522 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 is on osd.4 2014-08-13 17:38:04.317543 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317545 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317547 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54df500 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317540 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.317552 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317556 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317559 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317561 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317556 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 is on osd.4 2014-08-13 17:38:04.317579 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317581 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317582 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5484a40 osd_map(194479..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.317587 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194479,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317591 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317593 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317596 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317591 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 is on osd.4 2014-08-13 17:38:04.317615 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317617 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317619 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d8ee40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317609 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 is on osd.4 2014-08-13 17:38:04.317625 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317629 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317631 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317633 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317628 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 is on osd.4 2014-08-13 17:38:04.317650 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317653 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317654 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db3a80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317646 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 is on osd.4 2014-08-13 17:38:04.317659 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317664 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317666 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317668 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317662 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 is on osd.4 2014-08-13 17:38:04.317685 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317687 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317689 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9b600 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317694 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317698 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317702 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317718 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317720 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317722 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5410900 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317727 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317731 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317732 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317734 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317752 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317680 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 17:38:04.317756 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9a640 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317762 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317767 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317770 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317772 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate: adding 285 as a source 2014-08-13 17:38:04.317792 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317794 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317796 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5458480 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317783 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 also missing on osd.285 2014-08-13 17:38:04.317801 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317805 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317807 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317809 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317802 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 also missing on osd.285 2014-08-13 17:38:04.317826 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317829 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d98480 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317820 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 also missing on osd.285 2014-08-13 17:38:04.317835 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317839 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317842 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317844 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317838 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 also missing on osd.285 2014-08-13 17:38:04.317864 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317866 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317868 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5408240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317857 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 also missing on osd.285 2014-08-13 17:38:04.317874 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317878 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317880 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317882 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 also missing on osd.285 2014-08-13 17:38:04.317902 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317905 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317894 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 also missing on osd.285 2014-08-13 17:38:04.317907 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9bf00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317915 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317920 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317922 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317924 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317914 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 also missing on osd.285 2014-08-13 17:38:04.317941 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317943 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317945 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d906c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317933 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 also missing on osd.285 2014-08-13 17:38:04.317950 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317955 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.317957 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317951 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 also missing on osd.285 2014-08-13 17:38:04.317958 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317969 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 also missing on osd.285 2014-08-13 17:38:04.317984 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317986 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.317988 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d95c40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.317994 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.317998 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318000 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.317986 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 also missing on osd.285 2014-08-13 17:38:04.318002 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318007 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 also missing on osd.285 2014-08-13 17:38:04.318022 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318024 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318026 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d99200 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318032 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318036 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318038 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318024 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 also missing on osd.285 2014-08-13 17:38:04.318040 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318043 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 also missing on osd.285 2014-08-13 17:38:04.318059 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318061 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318063 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db0480 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318060 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 also missing on osd.285 2014-08-13 17:38:04.318069 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318082 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318084 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318085 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318078 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 also missing on osd.285 2014-08-13 17:38:04.318103 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318105 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318107 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9f740 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318096 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 also missing on osd.285 2014-08-13 17:38:04.318112 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318117 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318118 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318120 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 also missing on osd.285 2014-08-13 17:38:04.318137 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318140 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318141 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db1f80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318131 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 also missing on osd.285 2014-08-13 17:38:04.318147 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318151 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318153 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318155 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318149 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 also missing on osd.285 2014-08-13 17:38:04.318172 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318175 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318176 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5461200 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318182 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318185 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318187 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318189 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318176 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 also missing on osd.285 2014-08-13 17:38:04.318195 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 also missing on osd.285 2014-08-13 17:38:04.318211 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 also missing on osd.285 2014-08-13 17:38:04.318225 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318228 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318230 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db5580 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318236 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318240 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318241 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318243 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318229 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 also missing on osd.285 2014-08-13 17:38:04.318261 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318248 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 also missing on osd.285 2014-08-13 17:38:04.318263 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318265 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db2d00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318270 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318274 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318276 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318278 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318265 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 also missing on osd.285 2014-08-13 17:38:04.318295 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318297 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318299 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d98d80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318284 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 also missing on osd.285 2014-08-13 17:38:04.318304 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318308 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318310 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318312 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318309 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 also missing on osd.285 2014-08-13 17:38:04.318331 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318333 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318335 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d93f00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.318341 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.318345 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.318346 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318348 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318366 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318368 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318370 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e040 pg_notify(22.48e(1) epoch 194486) v5 2014-08-13 17:38:04.318325 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 17:38:04.318381 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate: adding 312 as a source 2014-08-13 17:38:04.318399 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 is on osd.312 2014-08-13 17:38:04.318415 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.289 2014-08-13 17:38:04.318418 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e040 2014-08-13 17:38:04.318418 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 is on osd.312 2014-08-13 17:38:04.318436 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 is on osd.312 2014-08-13 17:38:04.318465 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318454 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 is on osd.312 2014-08-13 17:38:04.318468 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318471 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 is on osd.312 2014-08-13 17:38:04.318492 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318494 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318495 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4263d40 pg_notify(22.113(29) epoch 194486) v5 2014-08-13 17:38:04.318488 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 is on osd.312 2014-08-13 17:38:04.318506 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 is on osd.312 2014-08-13 17:38:04.318533 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.55 2014-08-13 17:38:04.318535 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4263d40 2014-08-13 17:38:04.318525 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 is on osd.312 2014-08-13 17:38:04.318543 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 is on osd.312 2014-08-13 17:38:04.318561 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 is on osd.312 2014-08-13 17:38:04.318577 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318579 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 is on osd.312 2014-08-13 17:38:04.318596 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 is on osd.312 2014-08-13 17:38:04.318616 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318618 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318620 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102bc0 pg_notify(24.136(2),22.6bd(2) epoch 194486) v5 2014-08-13 17:38:04.318613 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 is on osd.312 2014-08-13 17:38:04.318630 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 is on osd.312 2014-08-13 17:38:04.318648 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 is on osd.312 2014-08-13 17:38:04.318661 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.50 2014-08-13 17:38:04.318668 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102bc0 2014-08-13 17:38:04.318667 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 is on osd.312 2014-08-13 17:38:04.318686 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 is on osd.312 2014-08-13 17:38:04.318704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 is on osd.312 2014-08-13 17:38:04.318718 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318721 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318721 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 is on osd.312 2014-08-13 17:38:04.318742 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318744 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318746 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c9f80 pg_notify(24.b6(38) epoch 194486) v5 2014-08-13 17:38:04.318739 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 is on osd.312 2014-08-13 17:38:04.318757 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 is on osd.312 2014-08-13 17:38:04.318782 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.17 2014-08-13 17:38:04.318785 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c9f80 2014-08-13 17:38:04.318775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 is on osd.312 2014-08-13 17:38:04.318792 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 is on osd.312 2014-08-13 17:38:04.318820 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318824 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318810 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 is on osd.312 2014-08-13 17:38:04.318829 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 is on osd.312 2014-08-13 17:38:04.318845 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 is on osd.312 2014-08-13 17:38:04.318860 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318862 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318865 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16c80 pg_notify(22.99(27) epoch 194486) v5 2014-08-13 17:38:04.318861 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 is on osd.312 2014-08-13 17:38:04.318880 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 is on osd.312 2014-08-13 17:38:04.318907 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.337 2014-08-13 17:38:04.318910 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16c80 2014-08-13 17:38:04.318899 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 is on osd.312 2014-08-13 17:38:04.318945 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318948 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318918 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 17:38:04.318976 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.318979 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.318981 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102840 pg_notify(22.6bd(2) epoch 194486) v5 2014-08-13 17:38:04.318974 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.318991 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 17:38:04.319017 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.32 2014-08-13 17:38:04.319020 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102840 2014-08-13 17:38:04.319009 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 285,311,312,369 2014-08-13 17:38:04.319026 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.319055 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319044 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.319058 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319062 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.319077 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319082 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826c80 pg_notify(22.619(1) epoch 194486) v5 2014-08-13 17:38:04.319078 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.319117 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.44 2014-08-13 17:38:04.319120 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826c80 2014-08-13 17:38:04.319152 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319154 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319172 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319174 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319173 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.319176 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc280 pg_notify(22.3d5(2) epoch 194486) v5 2014-08-13 17:38:04.319204 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.319213 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e2aa0 don't know epoch, doing nothing 2014-08-13 17:38:04.319217 7fe99f5e9700 7 osd.4 194486 do_notify osd 9 on 1 PGs 2014-08-13 17:38:04.319224 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.343 2014-08-13 17:38:04.319226 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc280 2014-08-13 17:38:04.319248 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a4f380 don't know epoch, doing nothing 2014-08-13 17:38:04.319251 7fe99f5e9700 7 osd.4 194486 do_notify osd 54 on 1 PGs 2014-08-13 17:38:04.319261 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319264 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319273 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:04.319276 7fe99f5e9700 7 osd.4 194486 do_notify osd 58 on 1 PGs 2014-08-13 17:38:04.319283 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319285 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319287 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103d40 pg_notify(22.707(33) epoch 194486) v5 2014-08-13 17:38:04.319321 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.15 2014-08-13 17:38:04.319324 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103d40 2014-08-13 17:38:04.319356 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319359 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319382 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319384 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319386 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c81c0 pg_notify(22.3d9(45) epoch 194486) v5 2014-08-13 17:38:04.319427 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.338 2014-08-13 17:38:04.319430 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c81c0 2014-08-13 17:38:04.319462 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319464 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319459 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:04.319469 7fe99f5e9700 7 osd.4 194486 do_notify osd 320 on 1 PGs 2014-08-13 17:38:04.319496 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing 2014-08-13 17:38:04.319499 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.291 on 1 PGs 2014-08-13 17:38:04.319500 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319506 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319508 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106580 pg_info(1 pgs e194486:22.2d1(2)) v4 2014-08-13 17:38:04.319550 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2d1(2)) v4 from osd.358 2014-08-13 17:38:04.319556 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106580 2014-08-13 17:38:04.319618 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319621 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319621 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing 2014-08-13 17:38:04.319629 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.350 on 1 PGs 2014-08-13 17:38:04.319645 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319647 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319649 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106200 pg_info(1 pgs e194486:22.12a(2)) v4 2014-08-13 17:38:04.319653 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ecac00 don't know epoch, doing nothing 2014-08-13 17:38:04.319656 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.365 on 1 PGs 2014-08-13 17:38:04.319676 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:04.319679 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.367 on 1 PGs 2014-08-13 17:38:04.319742 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.12a(2)) v4 from osd.323 2014-08-13 17:38:04.319746 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106200 2014-08-13 17:38:04.319754 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.319786 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing 2014-08-13 17:38:04.319794 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.378 on 1 PGs 2014-08-13 17:38:04.319798 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319802 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319791 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.319816 7fe99f5e9700 20 osd.4 194486 Sending info 22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) to shard 312 2014-08-13 17:38:04.319827 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319827 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9600 don't know epoch, doing nothing 2014-08-13 17:38:04.319830 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319832 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105b00 pg_info(1 pgs e194486:24.3bf(8)) v4 2014-08-13 17:38:04.319868 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.3bf(8)) v4 from osd.19 2014-08-13 17:38:04.319865 7fe99f5e9700 20 osd.4 194486 Sending info 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) to shard 324 2014-08-13 17:38:04.319873 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105b00 2014-08-13 17:38:04.319875 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:04.319917 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.319921 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.319948 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 17:38:04.320049 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 2014-08-13 17:38:04.320043 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 sending log((0'0,194215'1672], crt=0'0) 2014-08-13 17:38:04.320085 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.320089 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.320093 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Started advmap 2014-08-13 17:38:04.320110 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,288] 2014-08-13 17:38:04.320126 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.320123 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 1.2b( DNE v 194215'1672 (0'0,194215'1672] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) uptodate 2014-08-13 17:38:04.320145 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started/Stray 0.894517 4 0.000264 2014-08-13 17:38:04.320162 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.320176 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started 0.894623 0 0.000000 2014-08-13 17:38:04.320185 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.320197 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320212 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320196 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset 2014-08-13 17:38:04.320214 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106c80 pg_info(1 pgs e194486:22.537(2)) v4 2014-08-13 17:38:04.320206 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.320215 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194485 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194486 2014-08-13 17:38:04.320225 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.320235 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194486 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 17:38:04.320244 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.320259 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.537(2)) v4 from osd.51 2014-08-13 17:38:04.320264 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106c80 2014-08-13 17:38:04.320253 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194486 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.320275 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194486 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,288] 2014-08-13 17:38:04.320292 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194486 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.320310 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.320307 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304] r=1 lpr=194486 pi=193125-194484/32 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194486 2014-08-13 17:38:04.320323 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320326 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320350 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320353 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320355 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106900 pg_info(1 pgs e194486:22.3ad(2)) v4 2014-08-13 17:38:04.320342 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [346,4,304](346) acting [346,4,304](346)): not rw, up_thru 194483 up_from 192955 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [346,4,304](346) acting [346,4,304](346)) : primary up 192955-194483 does not include interval 2014-08-13 17:38:04.320361 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194485/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194485-194485 up [346,4,304](346) acting [346,4,304](346)) 2014-08-13 17:38:04.320396 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3ad(2)) v4 from osd.292 2014-08-13 17:38:04.320380 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,4,304] -> [346,4,304], acting [346,4,304] -> [346,4,288], acting_primary 346 -> 346, up_primary 346 -> 346, role 1 -> 1 2014-08-13 17:38:04.320400 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106900 2014-08-13 17:38:04.320402 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:04.320420 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.320446 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320449 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320436 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 17:38:04.320455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 17:38:04.320470 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320473 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320475 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c9500 pg_notify(22.1b8(25) epoch 194486) v5 2014-08-13 17:38:04.320472 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 17:38:04.320489 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 17:38:04.320513 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.347 2014-08-13 17:38:04.320516 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c9500 2014-08-13 17:38:04.320507 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 17:38:04.320527 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 17:38:04.320545 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 17:38:04.320563 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320566 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320561 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 17:38:04.320576 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 17:38:04.320594 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.320616 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320619 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320621 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c9880 pg_notify(22.113(29) epoch 194486) v5 2014-08-13 17:38:04.320613 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map 2014-08-13 17:38:04.320657 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.24 2014-08-13 17:38:04.320660 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c9880 2014-08-13 17:38:04.320695 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] update_heartbeat_peers unchanged 2014-08-13 17:38:04.320714 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters 2014-08-13 17:38:04.320732 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Reset 0.000535 1 0.000520 2014-08-13 17:38:04.320749 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320752 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320751 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started 2014-08-13 17:38:04.320769 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Start 2014-08-13 17:38:04.320786 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320788 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320790 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513db00 pg_notify(22.99(1) epoch 194486) v5 2014-08-13 17:38:04.320785 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: transitioning to Stray 2014-08-13 17:38:04.320808 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Start 0.000039 0 0.000000 2014-08-13 17:38:04.320832 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.363 2014-08-13 17:38:04.320835 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513db00 2014-08-13 17:38:04.320829 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/Stray 2014-08-13 17:38:04.320852 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.320870 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320873 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320870 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.320900 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.320902 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.320904 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5973de0 pg_query(24.e2 epoch 194486) v3 2014-08-13 17:38:04.320938 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.346 epoch 194486 2014-08-13 17:38:04.320941 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5973de0 2014-08-13 17:38:04.320965 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.325022 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[4] -- 26/4 2014-08-13 17:38:04.325062 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.325070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.325052 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(info 0'0) from replica 346 2014-08-13 17:38:04.325087 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] acting_up_affected newup [26,30,340] newacting [4] 2014-08-13 17:38:04.325103 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.325120 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.862026 3 0.000136 2014-08-13 17:38:04.325093 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 17:38:04.325143 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.325147 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.325140 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started 0.862085 0 0.000000 2014-08-13 17:38:04.325159 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 17:38:04.325184 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.325187 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.325188 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821500 pg_info(1 pgs e194486:22.69f(2)) v4 2014-08-13 17:38:04.325173 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194479 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 17:38:04.325204 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194486 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 17:38:04.325219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194486 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:04.325243 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.69f(2)) v4 from osd.334 2014-08-13 17:38:04.325234 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194486 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] acting_up_affected newup [26,30,340] newacting [4] 2014-08-13 17:38:04.325248 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821500 2014-08-13 17:38:04.325250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194486 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.325265 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340] r=-1 lpr=194486 pi=193125-194478/51 crt=193305'14233 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 17:38:04.325300 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194479-194485 up [26,30,340](26) acting [26,30,340](26)): not rw, up_thru 194479 up_from 192966 last_epoch_clean 194258 generate_past_intervals interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) : primary up 192966-194479 includes interval 2014-08-13 17:38:04.325142 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.325317 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194479/194479) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] noting past interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.325326 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.325336 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] up [26,30,340] -> [26,30,340], acting [26,30,340] -> [4], acting_primary 26 -> 4, up_primary 26 -> 26, role -1 -> 0 2014-08-13 17:38:04.325355 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.325356 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] clear_primary_state 2014-08-13 17:38:04.325376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.325377 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] agent_stop 2014-08-13 17:38:04.325395 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:04.325414 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] publish_stats_to_osd 194486:23841 2014-08-13 17:38:04.325431 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:04.325414 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.325449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 17:38:04.325450 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.325458 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.325467 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 17:38:04.325479 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.325483 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:04.325496 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.325498 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:04.325515 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.325514 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 17:38:04.325529 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 17:38:04.325544 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change 2014-08-13 17:38:04.325561 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 17:38:04.325556 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 17:38:04.325587 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.325578 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 17:38:04.325571 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] flushed 2014-08-13 17:38:04.325595 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] clear_stats 2014-08-13 17:38:04.325595 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.325614 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] requeue_ops 2014-08-13 17:38:04.325629 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] on_role_change 2014-08-13 17:38:04.325643 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] requeue_ops 2014-08-13 17:38:04.325657 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] cancel_recovery 2014-08-13 17:38:04.325617 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.325672 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] clear_recovery_state 2014-08-13 17:38:04.325689 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.325677 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.325709 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] handle_activate_map 2014-08-13 17:38:04.325710 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.325728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] update_heartbeat_peers -> 4,26,30,340 2014-08-13 17:38:04.325746 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.325735 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.325750 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] take_waiters 2014-08-13 17:38:04.325755 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.325767 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Reset 0.000607 1 0.000639 2014-08-13 17:38:04.325774 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2704 2014-08-13 17:38:04.325787 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started 2014-08-13 17:38:04.325804 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Start 2014-08-13 17:38:04.325792 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.325819 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: transitioning to Primary 2014-08-13 17:38:04.325832 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.800942 3 0.000390 2014-08-13 17:38:04.325845 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Start 0.000041 0 0.000000 2014-08-13 17:38:04.325865 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.325871 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary 2014-08-13 17:38:04.325893 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Peering 2014-08-13 17:38:04.325883 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.895456 0 0.000000 2014-08-13 17:38:04.325914 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.325929 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.325938 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 17:38:04.325954 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.325961 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.325981 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.325986 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.325977 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.325982 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194477-194478 up [26,30,340](26) acting [4](4)) 2014-08-13 17:38:04.326001 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194473-194476 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.326019 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326021 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326000 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.326024 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821a40 pg_notify(22.22c(27) epoch 194486) v5 2014-08-13 17:38:04.326021 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194471-194472 up [26,30,340](26) acting [4](4)) 2014-08-13 17:38:04.326027 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 17:38:04.326041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194462-194470 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.326070 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.324 2014-08-13 17:38:04.326074 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821a40 2014-08-13 17:38:04.326059 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior final: probe 4,26,30,340 down blocked_by {} 2014-08-13 17:38:04.326055 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.326080 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.326100 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194486:23842 2014-08-13 17:38:04.326130 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.26 2014-08-13 17:38:04.326133 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326137 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.30 2014-08-13 17:38:04.326154 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.340 2014-08-13 17:38:04.326180 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326182 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326184 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299bfc0 pg_query(24.cd epoch 194486) v3 2014-08-13 17:38:04.326171 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.326197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.326245 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.6 epoch 194486 2014-08-13 17:38:04.326249 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299bfc0 2014-08-13 17:38:04.326261 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_query query(info 0'0) from replica 6 2014-08-13 17:38:04.326317 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.326318 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326321 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326342 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326344 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326346 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ccd00 pg_notify(22.707(1) epoch 194486) v5 2014-08-13 17:38:04.326381 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.303 2014-08-13 17:38:04.326384 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ccd00 2014-08-13 17:38:04.326379 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 17:38:04.326410 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.326417 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.326444 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326447 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.326452 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.326471 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326473 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326475 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ca4c0 pg_notify(22.3d9(45) epoch 194486) v5 2014-08-13 17:38:04.326471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.326511 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.376 2014-08-13 17:38:04.326514 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ca4c0 2014-08-13 17:38:04.326490 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.326540 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.326570 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326573 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326560 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2870 2014-08-13 17:38:04.326577 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.326594 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.786362 6 0.000467 2014-08-13 17:38:04.326613 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326616 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326623 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513de80 pg_info(1 pgs e194486:22.639(2)) v4 2014-08-13 17:38:04.326613 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.326630 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.854457 0 0.000000 2014-08-13 17:38:04.326664 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.639(2)) v4 from osd.367 2014-08-13 17:38:04.326653 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.326669 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513de80 2014-08-13 17:38:04.326670 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194478 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.326692 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.326707 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326710 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326708 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.326733 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326736 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326738 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cdcc0 pg_notify(22.3d9(16) epoch 194486) v5 2014-08-13 17:38:04.326723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'504 -> 194215'504 2014-08-13 17:38:04.326754 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.326774 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.397 2014-08-13 17:38:04.326776 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cdcc0 2014-08-13 17:38:04.326822 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326825 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326848 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326850 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326852 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823640 pg_notify(22.1d2(1) epoch 194486) v5 2014-08-13 17:38:04.326886 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.331 2014-08-13 17:38:04.326889 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823640 2014-08-13 17:38:04.326933 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326936 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326954 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.326956 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.326958 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105940 pg_notify(24.1bb(35) epoch 194486) v5 2014-08-13 17:38:04.326992 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.379 2014-08-13 17:38:04.326995 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105940 2014-08-13 17:38:04.326981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 sending log((0'0,194215'504], crt=191648'365) 2014-08-13 17:38:04.327006 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:04.327039 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327042 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327082 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5143f00 pg_notify(22.48e(29) epoch 194486) v5 2014-08-13 17:38:04.327116 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.327 2014-08-13 17:38:04.327119 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5143f00 2014-08-13 17:38:04.327152 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327154 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327191 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327194 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327195 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823b80 pg_notify(24.136(31) epoch 194486) v5 2014-08-13 17:38:04.327238 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.65 2014-08-13 17:38:04.327241 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823b80 2014-08-13 17:38:04.327282 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327284 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327310 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327312 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327314 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104980 pg_notify(24.3d(32) epoch 194486) v5 2014-08-13 17:38:04.327349 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.373 2014-08-13 17:38:04.327351 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104980 2014-08-13 17:38:04.327401 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327404 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327345 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 sending log((0'0,194215'518], crt=191727'442) 2014-08-13 17:38:04.327437 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327439 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327441 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5438000 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.327451 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.327456 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.327458 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327478 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327480 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327481 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826040 pg_notify(22.3af(31) epoch 194486) v5 2014-08-13 17:38:04.327523 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.339 2014-08-13 17:38:04.327526 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826040 2014-08-13 17:38:04.327508 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 22.99( DNE v 194215'504 lc 0'0 (0'0,194215'504] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 17:38:04.327514 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfd6a0 don't know epoch, doing nothing 2014-08-13 17:38:04.327542 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.327578 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327581 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327629 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327631 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327633 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825b00 pg_notify(22.113(1) epoch 194486) v5 2014-08-13 17:38:04.327611 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 22.3e4( DNE v 194215'518 lc 0'0 (0'0,194215'518] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 17:38:04.327671 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.342 2014-08-13 17:38:04.327674 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825b00 2014-08-13 17:38:04.327659 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.327710 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327712 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327741 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327743 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327745 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c17000 pg_notify(22.3d9(49) epoch 194486) v5 2014-08-13 17:38:04.327780 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.293 2014-08-13 17:38:04.327764 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 sending log((0'0,194215'504], crt=191648'365) 2014-08-13 17:38:04.327783 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c17000 2014-08-13 17:38:04.327789 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:04.327823 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327826 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327854 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327856 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327858 7fe9a63f4700 20 osd.4 194486 _dispatch 0x59703c0 pg_query(24.3f7 epoch 194486) v3 2014-08-13 17:38:04.327891 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.330 epoch 194486 2014-08-13 17:38:04.327894 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x59703c0 2014-08-13 17:38:04.327905 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_query query(info 0'0) from replica 330 2014-08-13 17:38:04.327955 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327957 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327983 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.327985 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.327987 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ce200 pg_notify(22.f3(1) epoch 194486) v5 2014-08-13 17:38:04.328021 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.312 2014-08-13 17:38:04.328024 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ce200 2014-08-13 17:38:04.328013 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 sending log((0'0,194215'518], crt=191727'442) 2014-08-13 17:38:04.328060 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328062 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328090 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328093 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328094 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d080 pg_notify(22.1f1(1) epoch 194486) v5 2014-08-13 17:38:04.328128 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.11 2014-08-13 17:38:04.328131 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d080 2014-08-13 17:38:04.328169 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328171 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328189 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328191 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328193 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58271c0 pg_notify(24.9b(2) epoch 194486) v5 2014-08-13 17:38:04.328240 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.365 2014-08-13 17:38:04.328243 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58271c0 2014-08-13 17:38:04.328250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 22.99( DNE v 194215'504 lc 0'0 (0'0,194215'504] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 17:38:04.328290 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328293 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328313 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328315 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328317 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d98900 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.328324 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.328328 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.328330 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328332 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328357 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328360 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328362 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105cc0 pg_notify(22.481(46) epoch 194486) v5 2014-08-13 17:38:04.328358 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.31 has 29 missing 2014-08-13 17:38:04.328375 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.374 has 29 missing 2014-08-13 17:38:04.328401 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.381 2014-08-13 17:38:04.328404 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105cc0 2014-08-13 17:38:04.328391 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.328410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.328428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 is on osd.4 2014-08-13 17:38:04.328445 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328448 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328442 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x57cfe80 don't know epoch, doing nothing 2014-08-13 17:38:04.328444 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 is on osd.4 2014-08-13 17:38:04.328462 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 is on osd.4 2014-08-13 17:38:04.328482 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328485 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328487 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51047c0 pg_notify(22.3d9(45) epoch 194486) v5 2014-08-13 17:38:04.328478 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 is on osd.4 2014-08-13 17:38:04.328495 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 is on osd.4 2014-08-13 17:38:04.328489 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 22.3e4( DNE v 194215'518 lc 0'0 (0'0,194215'518] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 17:38:04.328513 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 is on osd.4 2014-08-13 17:38:04.328532 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.396 2014-08-13 17:38:04.328534 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51047c0 2014-08-13 17:38:04.328530 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 is on osd.4 2014-08-13 17:38:04.328546 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 is on osd.4 2014-08-13 17:38:04.328563 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 is on osd.4 2014-08-13 17:38:04.328585 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328588 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328579 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 is on osd.4 2014-08-13 17:38:04.328595 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 is on osd.4 2014-08-13 17:38:04.328617 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328619 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328622 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cd940 pg_notify(22.619(31) epoch 194486) v5 2014-08-13 17:38:04.328610 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 is on osd.4 2014-08-13 17:38:04.328615 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.313 has 32 missing 2014-08-13 17:38:04.328628 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 is on osd.4 2014-08-13 17:38:04.328635 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.318 has 32 missing 2014-08-13 17:38:04.328646 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 is on osd.4 2014-08-13 17:38:04.328663 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.314 2014-08-13 17:38:04.328666 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cd940 2014-08-13 17:38:04.328652 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 is on osd.4 2014-08-13 17:38:04.328668 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 is on osd.4 2014-08-13 17:38:04.328673 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.328686 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.328691 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 is on osd.4 2014-08-13 17:38:04.328714 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328718 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328704 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 is on osd.4 2014-08-13 17:38:04.328709 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 is on osd.4 2014-08-13 17:38:04.328721 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 is on osd.4 2014-08-13 17:38:04.328726 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.328749 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328751 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 is on osd.4 2014-08-13 17:38:04.328753 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc7c0 pg_notify(24.b6(38) epoch 194486) v5 2014-08-13 17:38:04.328743 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.328756 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.328762 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 is on osd.4 2014-08-13 17:38:04.328775 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.328794 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.315 2014-08-13 17:38:04.328781 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 is on osd.4 2014-08-13 17:38:04.328797 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc7c0 2014-08-13 17:38:04.328793 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.328799 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 is on osd.4 2014-08-13 17:38:04.328818 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.328815 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 is on osd.4 2014-08-13 17:38:04.328845 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328835 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 is on osd.4 2014-08-13 17:38:04.328849 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328840 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 is on osd.4 2014-08-13 17:38:04.328852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 is on osd.4 2014-08-13 17:38:04.328857 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 is on osd.4 2014-08-13 17:38:04.328869 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 is on osd.4 2014-08-13 17:38:04.328884 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328888 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328874 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 is on osd.4 2014-08-13 17:38:04.328890 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ce740 pg_notify(22.f3(23) epoch 194486) v5 2014-08-13 17:38:04.328887 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 is on osd.4 2014-08-13 17:38:04.328892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 is on osd.4 2014-08-13 17:38:04.328907 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.328931 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.311 2014-08-13 17:38:04.328934 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ce740 2014-08-13 17:38:04.328925 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 is on osd.4 2014-08-13 17:38:04.328943 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.328910 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 17:38:04.328960 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 is on osd.4 2014-08-13 17:38:04.328981 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328984 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.328969 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate: adding 31 as a source 2014-08-13 17:38:04.328977 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 is on osd.4 2014-08-13 17:38:04.328990 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 also missing on osd.31 2014-08-13 17:38:04.329008 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.328994 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 is on osd.4 2014-08-13 17:38:04.329011 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329013 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cbf00 pg_notify(22.22c(1) epoch 194486) v5 2014-08-13 17:38:04.329009 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 also missing on osd.31 2014-08-13 17:38:04.329015 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 is on osd.4 2014-08-13 17:38:04.329027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 also missing on osd.31 2014-08-13 17:38:04.329033 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 is on osd.4 2014-08-13 17:38:04.329053 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.302 2014-08-13 17:38:04.329056 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cbf00 2014-08-13 17:38:04.329047 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 also missing on osd.31 2014-08-13 17:38:04.329051 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 is on osd.4 2014-08-13 17:38:04.329066 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 also missing on osd.31 2014-08-13 17:38:04.329069 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.329084 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 also missing on osd.31 2014-08-13 17:38:04.329087 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 is on osd.4 2014-08-13 17:38:04.329105 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329109 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329102 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 also missing on osd.31 2014-08-13 17:38:04.329104 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 is on osd.4 2014-08-13 17:38:04.329128 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329120 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 also missing on osd.31 2014-08-13 17:38:04.329122 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 is on osd.4 2014-08-13 17:38:04.329139 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 also missing on osd.31 2014-08-13 17:38:04.329139 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 is on osd.4 2014-08-13 17:38:04.329161 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 is on osd.4 2014-08-13 17:38:04.329161 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 also missing on osd.31 2014-08-13 17:38:04.329189 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 also missing on osd.31 2014-08-13 17:38:04.329206 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 also missing on osd.31 2014-08-13 17:38:04.329179 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 is on osd.4 2014-08-13 17:38:04.329223 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 also missing on osd.31 2014-08-13 17:38:04.329229 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 is on osd.4 2014-08-13 17:38:04.329241 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 also missing on osd.31 2014-08-13 17:38:04.329248 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 is on osd.4 2014-08-13 17:38:04.329259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 also missing on osd.31 2014-08-13 17:38:04.329277 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 also missing on osd.31 2014-08-13 17:38:04.329294 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 also missing on osd.31 2014-08-13 17:38:04.329311 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 also missing on osd.31 2014-08-13 17:38:04.329265 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 17:38:04.329130 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329327 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 also missing on osd.31 2014-08-13 17:38:04.329332 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate: adding 313 as a source 2014-08-13 17:38:04.329343 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c17e00 pg_notify(22.481(39) epoch 194486) v5 2014-08-13 17:38:04.329345 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 also missing on osd.31 2014-08-13 17:38:04.329353 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 also missing on osd.313 2014-08-13 17:38:04.329363 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 also missing on osd.31 2014-08-13 17:38:04.329373 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 also missing on osd.313 2014-08-13 17:38:04.329395 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.309 2014-08-13 17:38:04.329383 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 also missing on osd.31 2014-08-13 17:38:04.329399 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c17e00 2014-08-13 17:38:04.329394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 also missing on osd.313 2014-08-13 17:38:04.329404 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 also missing on osd.31 2014-08-13 17:38:04.329414 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 also missing on osd.313 2014-08-13 17:38:04.329423 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 also missing on osd.31 2014-08-13 17:38:04.329434 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 also missing on osd.313 2014-08-13 17:38:04.329442 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 also missing on osd.31 2014-08-13 17:38:04.329462 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329466 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329452 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 also missing on osd.313 2014-08-13 17:38:04.329461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 also missing on osd.31 2014-08-13 17:38:04.329472 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 also missing on osd.313 2014-08-13 17:38:04.329480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 also missing on osd.31 2014-08-13 17:38:04.329500 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329503 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329490 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 also missing on osd.313 2014-08-13 17:38:04.329505 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103640 pg_notify(24.30b(39) epoch 194486) v5 2014-08-13 17:38:04.329510 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 also missing on osd.313 2014-08-13 17:38:04.329503 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 also missing on osd.31 2014-08-13 17:38:04.329544 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.375 2014-08-13 17:38:04.329527 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 also missing on osd.313 2014-08-13 17:38:04.329547 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103640 2014-08-13 17:38:04.329549 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 also missing on osd.31 2014-08-13 17:38:04.329551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 also missing on osd.313 2014-08-13 17:38:04.329570 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 also missing on osd.313 2014-08-13 17:38:04.329589 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329593 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329589 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 also missing on osd.313 2014-08-13 17:38:04.329607 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 also missing on osd.313 2014-08-13 17:38:04.329568 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 17:38:04.329633 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329636 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329623 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate: adding 374 as a source 2014-08-13 17:38:04.329624 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 also missing on osd.313 2014-08-13 17:38:04.329638 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8a80 pg_notify(22.4df(27) epoch 194486) v5 2014-08-13 17:38:04.329644 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 also missing on osd.313 2014-08-13 17:38:04.329644 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 also missing on osd.374 2014-08-13 17:38:04.329662 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 also missing on osd.313 2014-08-13 17:38:04.329677 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.371 2014-08-13 17:38:04.329680 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8a80 2014-08-13 17:38:04.329668 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 also missing on osd.374 2014-08-13 17:38:04.329681 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 also missing on osd.313 2014-08-13 17:38:04.329687 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 also missing on osd.374 2014-08-13 17:38:04.329700 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 also missing on osd.313 2014-08-13 17:38:04.329705 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 also missing on osd.374 2014-08-13 17:38:04.329718 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 also missing on osd.313 2014-08-13 17:38:04.329731 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329734 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 also missing on osd.374 2014-08-13 17:38:04.329735 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 also missing on osd.313 2014-08-13 17:38:04.329742 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 also missing on osd.374 2014-08-13 17:38:04.329760 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329763 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329754 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 also missing on osd.313 2014-08-13 17:38:04.329768 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cdb00 pg_notify(22.1f1(29) epoch 194486) v5 2014-08-13 17:38:04.329760 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 also missing on osd.374 2014-08-13 17:38:04.329772 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 also missing on osd.313 2014-08-13 17:38:04.329780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 also missing on osd.374 2014-08-13 17:38:04.329792 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 also missing on osd.313 2014-08-13 17:38:04.329808 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.368 2014-08-13 17:38:04.329811 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cdb00 2014-08-13 17:38:04.329800 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 also missing on osd.374 2014-08-13 17:38:04.329812 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 also missing on osd.313 2014-08-13 17:38:04.329818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 also missing on osd.374 2014-08-13 17:38:04.329831 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 also missing on osd.313 2014-08-13 17:38:04.329848 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329852 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329837 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 also missing on osd.374 2014-08-13 17:38:04.329849 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 also missing on osd.313 2014-08-13 17:38:04.329856 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 also missing on osd.374 2014-08-13 17:38:04.329877 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329880 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329867 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 also missing on osd.313 2014-08-13 17:38:04.329882 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8380 pg_info(1 pgs e194486:22.220(1)) v4 2014-08-13 17:38:04.329874 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 also missing on osd.374 2014-08-13 17:38:04.329892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 also missing on osd.374 2014-08-13 17:38:04.329919 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.220(1)) v4 from osd.351 2014-08-13 17:38:04.329910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 also missing on osd.374 2014-08-13 17:38:04.329924 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8380 2014-08-13 17:38:04.329927 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 also missing on osd.374 2014-08-13 17:38:04.329945 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 also missing on osd.374 2014-08-13 17:38:04.329962 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 also missing on osd.374 2014-08-13 17:38:04.329974 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.329977 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.329978 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 also missing on osd.374 2014-08-13 17:38:04.329996 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330004 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330006 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824ec0 pg_notify(22.3af(31) epoch 194486) v5 2014-08-13 17:38:04.329994 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 also missing on osd.374 2014-08-13 17:38:04.330015 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 also missing on osd.374 2014-08-13 17:38:04.330032 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 also missing on osd.374 2014-08-13 17:38:04.330047 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.369 2014-08-13 17:38:04.330050 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824ec0 2014-08-13 17:38:04.330050 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 also missing on osd.374 2014-08-13 17:38:04.330067 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 also missing on osd.374 2014-08-13 17:38:04.330085 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330088 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 also missing on osd.374 2014-08-13 17:38:04.329886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 also missing on osd.313 2014-08-13 17:38:04.330103 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 also missing on osd.374 2014-08-13 17:38:04.330107 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 also missing on osd.313 2014-08-13 17:38:04.330128 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330130 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330133 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5141500 pg_notify(22.775(31) epoch 194486) v5 2014-08-13 17:38:04.330121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 also missing on osd.374 2014-08-13 17:38:04.330126 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 also missing on osd.313 2014-08-13 17:38:04.330139 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 also missing on osd.374 2014-08-13 17:38:04.330144 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 also missing on osd.313 2014-08-13 17:38:04.330157 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 also missing on osd.374 2014-08-13 17:38:04.330171 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.63 2014-08-13 17:38:04.330174 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5141500 2014-08-13 17:38:04.330161 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 17:38:04.330227 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330231 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330174 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 17:38:04.330228 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate: adding 318 as a source 2014-08-13 17:38:04.330235 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.330261 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330248 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 also missing on osd.318 2014-08-13 17:38:04.330263 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.330266 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58247c0 pg_notify(22.362(31) epoch 194486) v5 2014-08-13 17:38:04.330266 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 also missing on osd.318 2014-08-13 17:38:04.330270 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 20,31,337,363,374 2014-08-13 17:38:04.330285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 also missing on osd.318 2014-08-13 17:38:04.330289 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.330308 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.349 2014-08-13 17:38:04.330311 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58247c0 2014-08-13 17:38:04.330304 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 also missing on osd.318 2014-08-13 17:38:04.330308 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.330322 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 also missing on osd.318 2014-08-13 17:38:04.330328 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.330340 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 also missing on osd.318 2014-08-13 17:38:04.330346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.330363 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330358 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 also missing on osd.318 2014-08-13 17:38:04.330376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 also missing on osd.318 2014-08-13 17:38:04.330401 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330403 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330405 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc600 pg_notify(24.9b(34) epoch 194486) v5 2014-08-13 17:38:04.330394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 also missing on osd.318 2014-08-13 17:38:04.330413 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 also missing on osd.318 2014-08-13 17:38:04.330439 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.330445 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.350 2014-08-13 17:38:04.330448 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc600 2014-08-13 17:38:04.330481 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330484 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330495 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 17:38:04.330521 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.330529 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330532 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330533 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104b40 pg_notify(22.481(44) epoch 194486) v5 2014-08-13 17:38:04.330528 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.330570 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.306 2014-08-13 17:38:04.330546 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.330572 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104b40 2014-08-13 17:38:04.330575 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.330431 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 also missing on osd.318 2014-08-13 17:38:04.330595 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.330613 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330616 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330615 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.330614 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 also missing on osd.318 2014-08-13 17:38:04.330636 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.330636 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 also missing on osd.318 2014-08-13 17:38:04.330653 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330656 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330658 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d5c0 pg_notify(22.4df(1) epoch 194486) v5 2014-08-13 17:38:04.330657 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2901 2014-08-13 17:38:04.330659 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 also missing on osd.318 2014-08-13 17:38:04.330677 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.330680 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 also missing on osd.318 2014-08-13 17:38:04.330701 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.352 2014-08-13 17:38:04.330704 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d5c0 2014-08-13 17:38:04.330695 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.813826 6 0.000419 2014-08-13 17:38:04.330700 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 also missing on osd.318 2014-08-13 17:38:04.330716 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.330719 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 also missing on osd.318 2014-08-13 17:38:04.330746 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330749 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330735 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.970723 0 0.000000 2014-08-13 17:38:04.330738 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 also missing on osd.318 2014-08-13 17:38:04.330756 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 also missing on osd.318 2014-08-13 17:38:04.330759 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.330777 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330780 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330781 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140380 pg_notify(22.39d(1) epoch 194486) v5 2014-08-13 17:38:04.330774 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 also missing on osd.318 2014-08-13 17:38:04.330776 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.330792 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 also missing on osd.318 2014-08-13 17:38:04.330801 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.330821 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.366 2014-08-13 17:38:04.330825 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140380 2014-08-13 17:38:04.330811 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 also missing on osd.318 2014-08-13 17:38:04.330820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.330831 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 also missing on osd.318 2014-08-13 17:38:04.330838 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'497 -> 194215'497 2014-08-13 17:38:04.330851 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 also missing on osd.318 2014-08-13 17:38:04.330867 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330870 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330864 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.330869 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 also missing on osd.318 2014-08-13 17:38:04.330884 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 is up to date, queueing in pending_activators 2014-08-13 17:38:04.330888 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 also missing on osd.318 2014-08-13 17:38:04.330902 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330904 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330906 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827e00 pg_notify(22.3d9(49) epoch 194486) v5 2014-08-13 17:38:04.330905 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 also missing on osd.318 2014-08-13 17:38:04.330912 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.330924 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 also missing on osd.318 2014-08-13 17:38:04.330945 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.382 2014-08-13 17:38:04.330948 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827e00 2014-08-13 17:38:04.330934 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.330943 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 also missing on osd.318 2014-08-13 17:38:04.330956 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 is up to date, queueing in pending_activators 2014-08-13 17:38:04.330962 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 also missing on osd.318 2014-08-13 17:38:04.330990 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.330993 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.330977 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.330980 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 also missing on osd.318 2014-08-13 17:38:04.330979 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.330999 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 also missing on osd.318 2014-08-13 17:38:04.330998 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.331019 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.331021 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.331039 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331044 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331046 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51071c0 pg_notify(24.34(39) epoch 194486) v5 2014-08-13 17:38:04.331038 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.331057 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.331084 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.26 2014-08-13 17:38:04.331074 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.331087 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51071c0 2014-08-13 17:38:04.331017 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 17:38:04.331097 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.331114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.331133 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331137 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331144 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.331134 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 45,303,313,318,342 2014-08-13 17:38:04.331152 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.331171 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331174 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331176 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db4a40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.331169 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.331183 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.331209 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.331213 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331215 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331200 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.331198 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 17:38:04.331227 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.331220 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.331234 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.331260 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331262 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331252 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.331265 7fe9a63f4700 20 osd.4 194486 _dispatch 0x42601c0 pg_notify(22.1d2(27) epoch 194486) v5 2014-08-13 17:38:04.331269 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.331290 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.331310 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.58 2014-08-13 17:38:04.331313 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x42601c0 2014-08-13 17:38:04.331318 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.331308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.331328 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.331346 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2657 2014-08-13 17:38:04.331365 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331368 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331362 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.331378 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.808002 3 0.000314 2014-08-13 17:38:04.331374 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 2014-08-13 17:38:04.331397 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.331409 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.331412 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.967015 0 0.000000 2014-08-13 17:38:04.331417 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Started advmap 2014-08-13 17:38:04.331443 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331446 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331449 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4264980 pg_notify(22.84(25) epoch 194486) v5 2014-08-13 17:38:04.331435 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,4,395] 2014-08-13 17:38:04.331448 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.331454 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.331467 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194477 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.331490 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.304 2014-08-13 17:38:04.331475 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started/Stray 0.890932 4 0.000274 2014-08-13 17:38:04.331494 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4264980 2014-08-13 17:38:04.331489 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.331499 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started 0.891031 0 0.000000 2014-08-13 17:38:04.331507 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.331520 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Reset 2014-08-13 17:38:04.331523 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'497 -> 194215'497 2014-08-13 17:38:04.331547 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331521 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.331550 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331539 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194485 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194486 2014-08-13 17:38:04.331548 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.331556 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.331559 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194486 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 17:38:04.331578 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331580 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331583 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820a80 pg_info(1 pgs e194486:22.28f(2)) v4 2014-08-13 17:38:04.331579 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194486 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.331596 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194486 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,4,395] 2014-08-13 17:38:04.331621 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.28f(2)) v4 from osd.9 2014-08-13 17:38:04.331625 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820a80 2014-08-13 17:38:04.331613 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194486 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.331630 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395]/[17,395] r=-1 lpr=194486 pi=193188-194484/36 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194486 2014-08-13 17:38:04.331675 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331678 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331665 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [17,4,395](17) acting [17,395](17)): not rw, up_thru 194483 up_from 192970 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [17,4,395](17) acting [17,395](17)) : primary up 192970-194483 does not include interval 2014-08-13 17:38:04.331682 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194485/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194485-194485 up [17,4,395](17) acting [17,395](17)) 2014-08-13 17:38:04.331700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.331702 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331704 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827a80 pg_notify(22.1b8(2) epoch 194486) v5 2014-08-13 17:38:04.331699 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,4,395] -> [17,4,395], acting [17,395] -> [17,4,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> 1 2014-08-13 17:38:04.331720 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:04.331741 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.340 2014-08-13 17:38:04.331744 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827a80 2014-08-13 17:38:04.331737 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.331756 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:04.331783 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335318 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 17:38:04.335362 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335364 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.331762 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.335366 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5971860 pg_query(24.1e4 epoch 194486) v3 2014-08-13 17:38:04.331814 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 sending log((0'0,194215'497], crt=191727'433) 2014-08-13 17:38:04.335366 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 17:38:04.335403 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.31 epoch 194486 2014-08-13 17:38:04.335407 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5971860 2014-08-13 17:38:04.335381 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.335404 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.335405 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 17:38:04.335425 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 17:38:04.335420 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_query query(info 0'0) from replica 31 2014-08-13 17:38:04.335448 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 17:38:04.335472 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335475 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335466 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 17:38:04.335453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 22.4df( DNE v 194215'497 lc 0'0 (0'0,194215'497] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 17:38:04.335502 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335485 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.335483 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_role_change 2014-08-13 17:38:04.335507 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335510 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821f80 pg_notify(22.84(2) epoch 194486) v5 2014-08-13 17:38:04.335507 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:04.335523 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 17:38:04.335548 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.28 2014-08-13 17:38:04.335538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 17:38:04.335551 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821f80 2014-08-13 17:38:04.335557 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.335579 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:04.335592 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335596 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335614 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335616 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335618 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8540 pg_notify(24.136(31) epoch 194486) v5 2014-08-13 17:38:04.335653 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.356 2014-08-13 17:38:04.335655 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8540 2014-08-13 17:38:04.335656 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 17:38:04.335674 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:04.335703 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335706 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335691 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.004172 1 0.004161 2014-08-13 17:38:04.335712 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started 2014-08-13 17:38:04.335728 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start 2014-08-13 17:38:04.335742 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 sending log((0'0,194215'497], crt=191727'433) 2014-08-13 17:38:04.335745 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335748 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104ec0 pg_info(1 pgs e194486:22.1a0(2)) v4 2014-08-13 17:38:04.335749 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5431600 don't know epoch, doing nothing 2014-08-13 17:38:04.335744 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray 2014-08-13 17:38:04.335765 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.335787 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1a0(2)) v4 from osd.328 2014-08-13 17:38:04.335791 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104ec0 2014-08-13 17:38:04.335777 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 22.4df( DNE v 194215'497 lc 0'0 (0'0,194215'497] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 17:38:04.335786 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray 2014-08-13 17:38:04.335814 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.335830 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335833 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335831 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.335853 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335856 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335858 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822840 pg_notify(24.30b(2) epoch 194486) v5 2014-08-13 17:38:04.335893 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.301 2014-08-13 17:38:04.335896 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822840 2014-08-13 17:38:04.335892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.356 has 35 missing 2014-08-13 17:38:04.335910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.357 has 35 missing 2014-08-13 17:38:04.335935 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.335925 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 is on osd.4 2014-08-13 17:38:04.335946 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335949 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 is on osd.4 2014-08-13 17:38:04.335960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 is on osd.4 2014-08-13 17:38:04.335975 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.335977 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.335980 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8c40 pg_info(1 pgs e194486:22.30f(2)) v4 2014-08-13 17:38:04.335977 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 is on osd.4 2014-08-13 17:38:04.335995 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 is on osd.4 2014-08-13 17:38:04.335991 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 17:38:04.336020 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.336020 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.30f(2)) v4 from osd.54 2014-08-13 17:38:04.336012 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 is on osd.4 2014-08-13 17:38:04.336027 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8c40 2014-08-13 17:38:04.336027 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.336030 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 is on osd.4 2014-08-13 17:38:04.336047 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.336065 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336068 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336048 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.336066 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.336089 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 is on osd.4 2014-08-13 17:38:04.336092 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336094 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d8f740 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.336101 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.336106 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.336108 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336095 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 is on osd.4 2014-08-13 17:38:04.336110 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336112 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 is on osd.4 2014-08-13 17:38:04.336125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.336140 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336128 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 is on osd.4 2014-08-13 17:38:04.336143 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336145 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58201c0 pg_notify(24.34(52) epoch 194486) v5 2014-08-13 17:38:04.336142 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.336145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 is on osd.4 2014-08-13 17:38:04.336159 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.336177 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.336185 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.30 2014-08-13 17:38:04.336187 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58201c0 2014-08-13 17:38:04.336163 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 is on osd.4 2014-08-13 17:38:04.336193 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 is on osd.4 2014-08-13 17:38:04.336210 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 is on osd.4 2014-08-13 17:38:04.336226 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 is on osd.4 2014-08-13 17:38:04.336243 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336224 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 17:38:04.336246 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336244 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 is on osd.4 2014-08-13 17:38:04.336253 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.336261 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 is on osd.4 2014-08-13 17:38:04.336267 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.336278 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.336296 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336299 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.336302 7fe9a63f4700 20 osd.4 194486 _dispatch 0x2998960 pg_query(24.1e6 epoch 194486) v3 2014-08-13 17:38:04.336296 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 is on osd.4 2014-08-13 17:38:04.336305 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.336315 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 is on osd.4 2014-08-13 17:38:04.336340 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.285 epoch 194486 2014-08-13 17:38:04.336325 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.336343 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x2998960 2014-08-13 17:38:04.336335 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 is on osd.4 2014-08-13 17:38:04.336346 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.336354 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.336346 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] flushed 2014-08-13 17:38:04.336359 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_query query(info 0'0) from replica 285 2014-08-13 17:38:04.336368 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.336373 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 is on osd.4 2014-08-13 17:38:04.336389 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2568 2014-08-13 17:38:04.336391 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 is on osd.4 2014-08-13 17:38:04.336411 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336414 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336406 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.336409 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 is on osd.4 2014-08-13 17:38:04.336434 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336436 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336439 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104d00 pg_info(1 pgs e194486:22.29a(2)) v4 2014-08-13 17:38:04.336424 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.811019 4 0.000336 2014-08-13 17:38:04.336426 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 is on osd.4 2014-08-13 17:38:04.336446 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.336446 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.336478 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.29a(2)) v4 from osd.345 2014-08-13 17:38:04.336482 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104d00 2014-08-13 17:38:04.336469 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.892399 0 0.000000 2014-08-13 17:38:04.336467 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 is on osd.4 2014-08-13 17:38:04.336493 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.336494 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 is on osd.4 2014-08-13 17:38:04.336511 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194478 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.336513 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.336532 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336536 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336530 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 is on osd.4 2014-08-13 17:38:04.336532 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.336557 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336561 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336547 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 is on osd.4 2014-08-13 17:38:04.336550 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.336563 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5138000 pg_notify(24.3d(2),22.99(27) epoch 194486) v5 2014-08-13 17:38:04.336565 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 is on osd.4 2014-08-13 17:38:04.336566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'483 -> 194215'483 2014-08-13 17:38:04.336609 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.20 2014-08-13 17:38:04.336593 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.336612 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5138000 2014-08-13 17:38:04.336585 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 17:38:04.336674 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336677 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336671 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate: adding 356 as a source 2014-08-13 17:38:04.336691 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 also missing on osd.356 2014-08-13 17:38:04.336706 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336709 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336711 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102680 pg_info(1 pgs e194486:22.4aa(2)) v4 2014-08-13 17:38:04.336709 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 also missing on osd.356 2014-08-13 17:38:04.336728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 also missing on osd.356 2014-08-13 17:38:04.336748 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4aa(2)) v4 from osd.321 2014-08-13 17:38:04.336753 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102680 2014-08-13 17:38:04.336746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 also missing on osd.356 2014-08-13 17:38:04.336763 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 also missing on osd.356 2014-08-13 17:38:04.336780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 also missing on osd.356 2014-08-13 17:38:04.336801 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336804 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336797 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 also missing on osd.356 2014-08-13 17:38:04.336824 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336827 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336809 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 sending log((0'0,194215'483], crt=191727'417) 2014-08-13 17:38:04.336814 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 also missing on osd.356 2014-08-13 17:38:04.336829 7fe9a63f4700 20 osd.4 194486 _dispatch 0x59712c0 pg_query(24.2ec epoch 194486) v3 2014-08-13 17:38:04.336835 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d335a0 don't know epoch, doing nothing 2014-08-13 17:38:04.336835 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 also missing on osd.356 2014-08-13 17:38:04.336864 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.374 epoch 194486 2014-08-13 17:38:04.336867 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x59712c0 2014-08-13 17:38:04.336854 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 also missing on osd.356 2014-08-13 17:38:04.336872 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 also missing on osd.356 2014-08-13 17:38:04.336871 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 22.707( DNE v 194215'483 lc 0'0 (0'0,194215'483] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(19) 2014-08-13 17:38:04.336879 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_query query(info 0'0) from replica 374 2014-08-13 17:38:04.336892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 also missing on osd.356 2014-08-13 17:38:04.336896 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.336911 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 also missing on osd.356 2014-08-13 17:38:04.336936 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336939 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336928 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 also missing on osd.356 2014-08-13 17:38:04.336945 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 also missing on osd.356 2014-08-13 17:38:04.336959 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.336962 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.336964 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824280 pg_notify(22.3af(1) epoch 194486) v5 2014-08-13 17:38:04.336962 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 also missing on osd.356 2014-08-13 17:38:04.336980 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 also missing on osd.356 2014-08-13 17:38:04.337007 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.61 2014-08-13 17:38:04.337010 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824280 2014-08-13 17:38:04.336997 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 also missing on osd.356 2014-08-13 17:38:04.337015 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 also missing on osd.356 2014-08-13 17:38:04.337045 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337033 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 also missing on osd.356 2014-08-13 17:38:04.337048 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337050 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 also missing on osd.356 2014-08-13 17:38:04.337077 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337067 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 also missing on osd.356 2014-08-13 17:38:04.337080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337085 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105240 pg_info(1 pgs e194486:22.630(2)) v4 2014-08-13 17:38:04.337085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 also missing on osd.356 2014-08-13 17:38:04.337103 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 also missing on osd.356 2014-08-13 17:38:04.337123 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.630(2)) v4 from osd.294 2014-08-13 17:38:04.337128 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105240 2014-08-13 17:38:04.337120 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 also missing on osd.356 2014-08-13 17:38:04.337118 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 sending log((0'0,194215'483], crt=191727'417) 2014-08-13 17:38:04.337146 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ecf900 don't know epoch, doing nothing 2014-08-13 17:38:04.337142 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 also missing on osd.356 2014-08-13 17:38:04.337159 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 also missing on osd.356 2014-08-13 17:38:04.337183 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 also missing on osd.356 2014-08-13 17:38:04.337171 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 22.707( DNE v 194215'483 lc 0'0 (0'0,194215'483] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(19) 2014-08-13 17:38:04.337202 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 also missing on osd.356 2014-08-13 17:38:04.337213 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337217 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337218 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 also missing on osd.356 2014-08-13 17:38:04.337246 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337235 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 also missing on osd.356 2014-08-13 17:38:04.337249 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337251 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5973fc0 pg_query(24.1de epoch 194486) v3 2014-08-13 17:38:04.337251 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.21 has 19 missing 2014-08-13 17:38:04.337252 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 also missing on osd.356 2014-08-13 17:38:04.337274 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.310 has 19 missing 2014-08-13 17:38:04.337276 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 also missing on osd.356 2014-08-13 17:38:04.337292 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.364 epoch 194486 2014-08-13 17:38:04.337296 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5973fc0 2014-08-13 17:38:04.337290 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.337295 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 also missing on osd.356 2014-08-13 17:38:04.337310 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 is on osd.4 2014-08-13 17:38:04.337307 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_query query(info 0'0) from replica 364 2014-08-13 17:38:04.337314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 also missing on osd.356 2014-08-13 17:38:04.337328 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 is on osd.4 2014-08-13 17:38:04.337361 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337350 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 is on osd.4 2014-08-13 17:38:04.337364 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337367 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 is on osd.4 2014-08-13 17:38:04.337384 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337387 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337389 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ccec0 pg_notify(22.3e4(31) epoch 194486) v5 2014-08-13 17:38:04.337331 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 17:38:04.337383 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 is on osd.4 2014-08-13 17:38:04.337397 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate: adding 357 as a source 2014-08-13 17:38:04.337400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 is on osd.4 2014-08-13 17:38:04.337429 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.45 2014-08-13 17:38:04.337416 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 also missing on osd.357 2014-08-13 17:38:04.337432 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ccec0 2014-08-13 17:38:04.337418 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 is on osd.4 2014-08-13 17:38:04.337436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 also missing on osd.357 2014-08-13 17:38:04.337437 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 is on osd.4 2014-08-13 17:38:04.337456 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 also missing on osd.357 2014-08-13 17:38:04.337455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 is on osd.4 2014-08-13 17:38:04.337486 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337489 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337477 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 is on osd.4 2014-08-13 17:38:04.337477 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 also missing on osd.357 2014-08-13 17:38:04.337496 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 is on osd.4 2014-08-13 17:38:04.337497 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 also missing on osd.357 2014-08-13 17:38:04.337523 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337526 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337529 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299ad00 pg_query(24.5e epoch 194486) v3 2014-08-13 17:38:04.337515 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.337516 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 also missing on osd.357 2014-08-13 17:38:04.337534 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 is on osd.4 2014-08-13 17:38:04.337536 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 also missing on osd.357 2014-08-13 17:38:04.337552 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 is on osd.4 2014-08-13 17:38:04.337567 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.289 epoch 194486 2014-08-13 17:38:04.337555 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 also missing on osd.357 2014-08-13 17:38:04.337570 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299ad00 2014-08-13 17:38:04.337570 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 is on osd.4 2014-08-13 17:38:04.337574 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 also missing on osd.357 2014-08-13 17:38:04.337588 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 is on osd.4 2014-08-13 17:38:04.337583 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_query query(info 0'0) from replica 289 2014-08-13 17:38:04.337593 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 also missing on osd.357 2014-08-13 17:38:04.337605 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 is on osd.4 2014-08-13 17:38:04.337612 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 also missing on osd.357 2014-08-13 17:38:04.337624 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 is on osd.4 2014-08-13 17:38:04.337640 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337629 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 also missing on osd.357 2014-08-13 17:38:04.337644 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337649 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 also missing on osd.357 2014-08-13 17:38:04.337667 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337669 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337672 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826740 pg_notify(24.1f7(2) epoch 194486) v5 2014-08-13 17:38:04.337640 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 17:38:04.337668 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 also missing on osd.357 2014-08-13 17:38:04.337683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate: adding 21 as a source 2014-08-13 17:38:04.337688 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 also missing on osd.357 2014-08-13 17:38:04.337712 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.55 2014-08-13 17:38:04.337715 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826740 2014-08-13 17:38:04.337702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 also missing on osd.21 2014-08-13 17:38:04.337707 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 also missing on osd.357 2014-08-13 17:38:04.337721 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 also missing on osd.21 2014-08-13 17:38:04.337751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337739 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 also missing on osd.21 2014-08-13 17:38:04.337754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337757 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 also missing on osd.21 2014-08-13 17:38:04.337725 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 also missing on osd.357 2014-08-13 17:38:04.337784 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337786 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337774 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 also missing on osd.21 2014-08-13 17:38:04.337788 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299d820 pg_query(24.2a1 epoch 194486) v3 2014-08-13 17:38:04.337778 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 also missing on osd.357 2014-08-13 17:38:04.337793 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 also missing on osd.21 2014-08-13 17:38:04.337798 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 also missing on osd.357 2014-08-13 17:38:04.337811 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 also missing on osd.21 2014-08-13 17:38:04.337826 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.17 epoch 194486 2014-08-13 17:38:04.337829 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299d820 2014-08-13 17:38:04.337817 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 also missing on osd.357 2014-08-13 17:38:04.337830 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 also missing on osd.21 2014-08-13 17:38:04.337836 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 also missing on osd.357 2014-08-13 17:38:04.337842 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(info 0'0) from replica 17 2014-08-13 17:38:04.337848 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 also missing on osd.21 2014-08-13 17:38:04.337855 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 also missing on osd.357 2014-08-13 17:38:04.337867 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 also missing on osd.21 2014-08-13 17:38:04.337874 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 also missing on osd.357 2014-08-13 17:38:04.337888 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337891 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 also missing on osd.357 2014-08-13 17:38:04.337911 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.337913 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.337915 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820700 pg_info(1 pgs e194486:22.4fe(2)) v4 2014-08-13 17:38:04.337910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 also missing on osd.357 2014-08-13 17:38:04.337885 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 also missing on osd.21 2014-08-13 17:38:04.337928 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 also missing on osd.357 2014-08-13 17:38:04.337937 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 also missing on osd.21 2014-08-13 17:38:04.337955 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe(2)) v4 from osd.337 2014-08-13 17:38:04.337960 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820700 2014-08-13 17:38:04.337947 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 also missing on osd.357 2014-08-13 17:38:04.337958 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 also missing on osd.21 2014-08-13 17:38:04.337968 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 also missing on osd.357 2014-08-13 17:38:04.337978 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 also missing on osd.21 2014-08-13 17:38:04.337987 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 also missing on osd.357 2014-08-13 17:38:04.337998 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 also missing on osd.21 2014-08-13 17:38:04.338012 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338016 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338005 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 also missing on osd.357 2014-08-13 17:38:04.338016 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 also missing on osd.21 2014-08-13 17:38:04.338023 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 also missing on osd.357 2014-08-13 17:38:04.338038 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338041 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338043 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5972760 pg_query(24.206 epoch 194486) v3 2014-08-13 17:38:04.338035 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 also missing on osd.21 2014-08-13 17:38:04.338042 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 also missing on osd.357 2014-08-13 17:38:04.338053 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 also missing on osd.21 2014-08-13 17:38:04.338063 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 also missing on osd.357 2014-08-13 17:38:04.338072 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 also missing on osd.21 2014-08-13 17:38:04.338088 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.343 epoch 194486 2014-08-13 17:38:04.338091 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5972760 2014-08-13 17:38:04.338083 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 also missing on osd.357 2014-08-13 17:38:04.338102 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 also missing on osd.357 2014-08-13 17:38:04.338103 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_query query(info 0'0) from replica 343 2014-08-13 17:38:04.338089 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 17:38:04.338130 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate: adding 310 as a source 2014-08-13 17:38:04.338144 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338146 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338148 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 also missing on osd.310 2014-08-13 17:38:04.338176 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338178 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338165 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 also missing on osd.310 2014-08-13 17:38:04.338180 7fe9a63f4700 20 osd.4 194486 _dispatch 0x2999fe0 pg_query(24.20e epoch 194486) v3 2014-08-13 17:38:04.338119 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 17:38:04.338183 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 also missing on osd.310 2014-08-13 17:38:04.338191 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.338209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.338209 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 also missing on osd.310 2014-08-13 17:38:04.338230 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.15 epoch 194486 2014-08-13 17:38:04.338235 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x2999fe0 2014-08-13 17:38:04.338229 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 also missing on osd.310 2014-08-13 17:38:04.338230 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 352,356,357,371,380 2014-08-13 17:38:04.338248 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 also missing on osd.310 2014-08-13 17:38:04.338250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.338268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 also missing on osd.310 2014-08-13 17:38:04.338273 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.338249 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 15 2014-08-13 17:38:04.338285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 also missing on osd.310 2014-08-13 17:38:04.338292 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.338302 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 also missing on osd.310 2014-08-13 17:38:04.338310 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.338327 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338330 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338321 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 also missing on osd.310 2014-08-13 17:38:04.338350 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338353 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338340 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 also missing on osd.310 2014-08-13 17:38:04.338355 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db5c40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.338362 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.338367 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.338369 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338371 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338358 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 also missing on osd.310 2014-08-13 17:38:04.338376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 also missing on osd.310 2014-08-13 17:38:04.338400 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.338402 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.338404 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e740 pg_notify(22.39d(29) epoch 194486) v5 2014-08-13 17:38:04.338393 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 also missing on osd.310 2014-08-13 17:38:04.338411 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.338410 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 also missing on osd.310 2014-08-13 17:38:04.338428 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 also missing on osd.310 2014-08-13 17:38:04.338442 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.363 2014-08-13 17:38:04.338445 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e740 2014-08-13 17:38:04.338446 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 also missing on osd.310 2014-08-13 17:38:04.338463 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 also missing on osd.310 2014-08-13 17:38:04.338466 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 17:38:04.338480 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 also missing on osd.310 2014-08-13 17:38:04.338494 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.338502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.338496 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 17:38:04.338521 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.338535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.338537 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.338551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.338568 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 15,21,303,310,342 2014-08-13 17:38:04.338584 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.338557 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.338601 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.338604 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.338618 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.338623 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.338634 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.338643 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2612 2014-08-13 17:38:04.338659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.338676 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.814656 4 0.000368 2014-08-13 17:38:04.338696 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.338711 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.970497 0 0.000000 2014-08-13 17:38:04.338731 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.338734 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.338749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194478 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.338770 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.338786 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.338780 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 2014-08-13 17:38:04.338802 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'480 -> 194215'480 2014-08-13 17:38:04.338814 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.338821 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Started advmap 2014-08-13 17:38:04.338824 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.338838 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,4,285] 2014-08-13 17:38:04.338855 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.338873 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started/Stray 0.893204 4 0.000192 2014-08-13 17:38:04.338893 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started 0.893267 0 0.000000 2014-08-13 17:38:04.338911 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Reset 2014-08-13 17:38:04.338927 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194485 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194486 2014-08-13 17:38:04.338944 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194486 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 17:38:04.338960 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194486 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.338975 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194486 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,4,285] 2014-08-13 17:38:04.338991 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194486 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.339006 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285]/[289,285] r=-1 lpr=194486 pi=193125-194484/43 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194486 2014-08-13 17:38:04.339039 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [289,4,285](289) acting [289,285](289)): not rw, up_thru 194483 up_from 192968 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [289,4,285](289) acting [289,285](289)) : primary up 192968-194483 does not include interval 2014-08-13 17:38:04.339037 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 sending log((0'0,194215'480], crt=191727'416) 2014-08-13 17:38:04.339062 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing 2014-08-13 17:38:04.339057 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194485/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194485-194485 up [289,4,285](289) acting [289,285](289)) 2014-08-13 17:38:04.339074 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,4,285] -> [289,4,285], acting [289,285] -> [289,4,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> 1 2014-08-13 17:38:04.339093 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:04.339093 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 22.39d( DNE v 194215'480 lc 0'0 (0'0,194215'480] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 17:38:04.339110 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.339117 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.339125 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:04.339144 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 17:38:04.339159 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 17:38:04.339122 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.339174 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 17:38:04.339180 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.339201 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 17:38:04.339221 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 17:38:04.339237 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 17:38:04.339251 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_role_change 2014-08-13 17:38:04.339265 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:04.339279 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 17:38:04.339293 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 17:38:04.339309 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.339315 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 sending log((0'0,194215'480], crt=191727'416) 2014-08-13 17:38:04.339339 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:04.339328 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map 2014-08-13 17:38:04.339365 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 22.39d( DNE v 194215'480 lc 0'0 (0'0,194215'480] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 17:38:04.339425 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] update_heartbeat_peers unchanged 2014-08-13 17:38:04.339443 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters 2014-08-13 17:38:04.339459 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.000547 1 0.000508 2014-08-13 17:38:04.339470 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.344 has 23 missing 2014-08-13 17:38:04.339477 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started 2014-08-13 17:38:04.339488 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.373 has 23 missing 2014-08-13 17:38:04.339504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 is on osd.4 2014-08-13 17:38:04.339484 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.339522 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 is on osd.4 2014-08-13 17:38:04.339526 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.339494 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start 2014-08-13 17:38:04.339540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 is on osd.4 2014-08-13 17:38:04.339547 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray 2014-08-13 17:38:04.339559 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 is on osd.4 2014-08-13 17:38:04.339567 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000072 0 0.000000 2014-08-13 17:38:04.339576 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 is on osd.4 2014-08-13 17:38:04.339587 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray 2014-08-13 17:38:04.339595 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 is on osd.4 2014-08-13 17:38:04.339609 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.339612 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 is on osd.4 2014-08-13 17:38:04.339626 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.339629 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 is on osd.4 2014-08-13 17:38:04.339646 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 is on osd.4 2014-08-13 17:38:04.339663 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 is on osd.4 2014-08-13 17:38:04.339681 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 is on osd.4 2014-08-13 17:38:04.339696 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.339711 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 is on osd.4 2014-08-13 17:38:04.339730 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.339726 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 is on osd.4 2014-08-13 17:38:04.339742 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.339757 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 is on osd.4 2014-08-13 17:38:04.339772 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 is on osd.4 2014-08-13 17:38:04.339777 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 17:38:04.339788 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 is on osd.4 2014-08-13 17:38:04.339803 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.339807 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 is on osd.4 2014-08-13 17:38:04.339810 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.339824 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 is on osd.4 2014-08-13 17:38:04.339829 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.339846 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.339840 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 is on osd.4 2014-08-13 17:38:04.339864 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.339864 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.339882 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 is on osd.4 2014-08-13 17:38:04.339883 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.339905 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.339923 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:3005 2014-08-13 17:38:04.339902 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 17:38:04.339938 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.339948 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate: adding 344 as a source 2014-08-13 17:38:04.339956 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.809944 3 0.004552 2014-08-13 17:38:04.339966 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 also missing on osd.344 2014-08-13 17:38:04.339975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.339985 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 also missing on osd.344 2014-08-13 17:38:04.339993 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.888172 0 0.000000 2014-08-13 17:38:04.340004 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 also missing on osd.344 2014-08-13 17:38:04.340015 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.340022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 also missing on osd.344 2014-08-13 17:38:04.340031 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194478 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.340040 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 also missing on osd.344 2014-08-13 17:38:04.340053 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.340069 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.340083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'493 -> 194215'493 2014-08-13 17:38:04.340059 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 also missing on osd.344 2014-08-13 17:38:04.340107 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.340112 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 also missing on osd.344 2014-08-13 17:38:04.340130 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 also missing on osd.344 2014-08-13 17:38:04.340147 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 also missing on osd.344 2014-08-13 17:38:04.340141 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] flushed 2014-08-13 17:38:04.340164 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 also missing on osd.344 2014-08-13 17:38:04.340189 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 also missing on osd.344 2014-08-13 17:38:04.340206 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 also missing on osd.344 2014-08-13 17:38:04.340223 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 also missing on osd.344 2014-08-13 17:38:04.340239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 also missing on osd.344 2014-08-13 17:38:04.340255 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 also missing on osd.344 2014-08-13 17:38:04.340271 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 also missing on osd.344 2014-08-13 17:38:04.340288 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 also missing on osd.344 2014-08-13 17:38:04.340304 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 also missing on osd.344 2014-08-13 17:38:04.340320 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 also missing on osd.344 2014-08-13 17:38:04.340336 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 also missing on osd.344 2014-08-13 17:38:04.340352 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 also missing on osd.344 2014-08-13 17:38:04.340368 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 also missing on osd.344 2014-08-13 17:38:04.340384 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 also missing on osd.344 2014-08-13 17:38:04.340383 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 sending log((0'0,194215'493], crt=191648'346) 2014-08-13 17:38:04.340410 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d36b40 don't know epoch, doing nothing 2014-08-13 17:38:04.340400 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 17:38:04.340443 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate: adding 373 as a source 2014-08-13 17:38:04.340442 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 22.1f1( DNE v 194215'493 lc 0'0 (0'0,194215'493] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 17:38:04.340461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 also missing on osd.373 2014-08-13 17:38:04.340465 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.340479 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 also missing on osd.373 2014-08-13 17:38:04.340496 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 also missing on osd.373 2014-08-13 17:38:04.340512 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 also missing on osd.373 2014-08-13 17:38:04.340528 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 also missing on osd.373 2014-08-13 17:38:04.340544 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 also missing on osd.373 2014-08-13 17:38:04.340561 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 also missing on osd.373 2014-08-13 17:38:04.340577 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 also missing on osd.373 2014-08-13 17:38:04.340593 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 also missing on osd.373 2014-08-13 17:38:04.340609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 also missing on osd.373 2014-08-13 17:38:04.340625 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 also missing on osd.373 2014-08-13 17:38:04.340641 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 also missing on osd.373 2014-08-13 17:38:04.340657 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 also missing on osd.373 2014-08-13 17:38:04.340673 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 also missing on osd.373 2014-08-13 17:38:04.340690 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 also missing on osd.373 2014-08-13 17:38:04.340706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 also missing on osd.373 2014-08-13 17:38:04.340721 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 also missing on osd.373 2014-08-13 17:38:04.340738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 also missing on osd.373 2014-08-13 17:38:04.340742 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 sending log((0'0,194215'493], crt=191648'346) 2014-08-13 17:38:04.340755 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 also missing on osd.373 2014-08-13 17:38:04.340768 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4829b80 don't know epoch, doing nothing 2014-08-13 17:38:04.340772 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 also missing on osd.373 2014-08-13 17:38:04.340789 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 also missing on osd.373 2014-08-13 17:38:04.340795 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 22.1f1( DNE v 194215'493 lc 0'0 (0'0,194215'493] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 17:38:04.340805 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 also missing on osd.373 2014-08-13 17:38:04.340823 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 also missing on osd.373 2014-08-13 17:38:04.340838 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 17:38:04.340879 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.340893 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.340905 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.5 has 35 missing 2014-08-13 17:38:04.340910 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 320,344,363,366,373 2014-08-13 17:38:04.340921 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.47 has 35 missing 2014-08-13 17:38:04.340928 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.340937 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 is on osd.4 2014-08-13 17:38:04.340946 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.340956 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 is on osd.4 2014-08-13 17:38:04.340964 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.340974 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 is on osd.4 2014-08-13 17:38:04.340981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.340991 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 is on osd.4 2014-08-13 17:38:04.341009 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 is on osd.4 2014-08-13 17:38:04.341025 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.341040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 is on osd.4 2014-08-13 17:38:04.341056 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.341076 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.341071 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 is on osd.4 2014-08-13 17:38:04.341086 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 is on osd.4 2014-08-13 17:38:04.341102 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 is on osd.4 2014-08-13 17:38:04.341117 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 is on osd.4 2014-08-13 17:38:04.341132 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 is on osd.4 2014-08-13 17:38:04.341131 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 17:38:04.341159 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.341149 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 is on osd.4 2014-08-13 17:38:04.341165 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 is on osd.4 2014-08-13 17:38:04.341166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.341182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 is on osd.4 2014-08-13 17:38:04.341192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.341207 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 is on osd.4 2014-08-13 17:38:04.341211 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.341228 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341233 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341224 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 is on osd.4 2014-08-13 17:38:04.341272 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341274 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341276 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821880 pg_notify(2.2a(1) epoch 194486) v5 2014-08-13 17:38:04.341244 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 is on osd.4 2014-08-13 17:38:04.341275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.341288 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 is on osd.4 2014-08-13 17:38:04.341293 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.341316 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.324 2014-08-13 17:38:04.341320 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821880 2014-08-13 17:38:04.341307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 is on osd.4 2014-08-13 17:38:04.341315 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.341336 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.341325 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 is on osd.4 2014-08-13 17:38:04.341344 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 is on osd.4 2014-08-13 17:38:04.341363 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341365 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341360 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.341386 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341389 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341377 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.341370 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:04.341391 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8000 pg_notify(24.3d(32) epoch 194486) v5 2014-08-13 17:38:04.341394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 is on osd.4 2014-08-13 17:38:04.341405 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.341411 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 is on osd.4 2014-08-13 17:38:04.341412 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:04.341434 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.303 2014-08-13 17:38:04.341437 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8000 2014-08-13 17:38:04.341428 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 is on osd.4 2014-08-13 17:38:04.341433 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.341445 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 is on osd.4 2014-08-13 17:38:04.341453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:04.341478 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341464 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 is on osd.4 2014-08-13 17:38:04.341481 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341484 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 is on osd.4 2014-08-13 17:38:04.341506 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341508 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341510 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cd780 pg_notify(22.48e(29) epoch 194486) v5 2014-08-13 17:38:04.341501 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.341517 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.341549 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.376 2014-08-13 17:38:04.341535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 is on osd.4 2014-08-13 17:38:04.341551 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cd780 2014-08-13 17:38:04.341540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:04.341553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 is on osd.4 2014-08-13 17:38:04.341561 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194486 2014-08-13 17:38:04.341580 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.341595 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341598 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341602 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.341630 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341632 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341634 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f02840 pg_notify(24.9b epoch 194486) v5 2014-08-13 17:38:04.341572 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 17:38:04.341627 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 2014-08-13 17:38:04.341639 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate: adding 5 as a source 2014-08-13 17:38:04.341656 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.341659 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 also missing on osd.5 2014-08-13 17:38:04.341675 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.367 2014-08-13 17:38:04.341662 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Started advmap 2014-08-13 17:38:04.341678 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f02840 2014-08-13 17:38:04.341632 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.341679 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 also missing on osd.5 2014-08-13 17:38:04.341680 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,57] 2014-08-13 17:38:04.341715 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341719 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341689 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.341702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 also missing on osd.5 2014-08-13 17:38:04.341702 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.341738 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341723 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 also missing on osd.5 2014-08-13 17:38:04.341740 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341725 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started/Stray 0.944604 4 0.000280 2014-08-13 17:38:04.341743 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102300 pg_info(1 pgs e194486:22.3b9(2)) v4 2014-08-13 17:38:04.341743 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 also missing on osd.5 2014-08-13 17:38:04.341748 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started 0.944701 0 0.000000 2014-08-13 17:38:04.341762 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 also missing on osd.5 2014-08-13 17:38:04.341769 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset 2014-08-13 17:38:04.341783 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3b9(2)) v4 from osd.397 2014-08-13 17:38:04.341788 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102300 2014-08-13 17:38:04.341780 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 also missing on osd.5 2014-08-13 17:38:04.341787 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194485 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194486 2014-08-13 17:38:04.341799 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 also missing on osd.5 2014-08-13 17:38:04.341806 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194486 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 17:38:04.341825 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341818 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 also missing on osd.5 2014-08-13 17:38:04.341823 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194486 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.341848 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.341837 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 also missing on osd.5 2014-08-13 17:38:04.341841 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194486 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,57] 2014-08-13 17:38:04.341856 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 also missing on osd.5 2014-08-13 17:38:04.341858 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194486 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.341873 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 also missing on osd.5 2014-08-13 17:38:04.341874 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288] r=1 lpr=194486 pi=193227-194484/38 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194486 2014-08-13 17:38:04.341851 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.341893 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5143b80 pg_notify(24.b6(2) epoch 194486) v5 2014-08-13 17:38:04.341892 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 also missing on osd.5 2014-08-13 17:38:04.341911 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 also missing on osd.5 2014-08-13 17:38:04.341920 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [330,4,288](330) acting [330,4,288](330)): not rw, up_thru 194483 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [330,4,288](330) acting [330,4,288](330)) : primary up 192965-194483 does not include interval 2014-08-13 17:38:04.341941 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.327 2014-08-13 17:38:04.341944 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5143b80 2014-08-13 17:38:04.341932 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 also missing on osd.5 2014-08-13 17:38:04.341958 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.396 now has 194486 2014-08-13 17:38:04.341943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194485/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194485-194485 up [330,4,288](330) acting [330,4,288](330)) 2014-08-13 17:38:04.341954 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 also missing on osd.5 2014-08-13 17:38:04.348635 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 2014-08-13 17:38:04.348638 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,4,288] -> [330,4,288], acting [330,4,288] -> [330,4,57], acting_primary 330 -> 330, up_primary 330 -> 330, role 1 -> 1 2014-08-13 17:38:04.348653 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 also missing on osd.5 2014-08-13 17:38:04.348678 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:04.348694 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 also missing on osd.5 2014-08-13 17:38:04.348697 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.348714 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 2014-08-13 17:38:04.348714 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 also missing on osd.5 2014-08-13 17:38:04.348722 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.348715 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:04.348736 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.348735 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 also missing on osd.5 2014-08-13 17:38:04.348740 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 17:38:04.348753 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 also missing on osd.5 2014-08-13 17:38:04.348758 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 17:38:04.348773 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.348777 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.348779 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cd5c0 pg_notify(22.3d9(42) epoch 194486) v5 2014-08-13 17:38:04.348774 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 also missing on osd.5 2014-08-13 17:38:04.348776 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 17:38:04.348793 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 also missing on osd.5 2014-08-13 17:38:04.348795 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 17:38:04.348811 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 also missing on osd.5 2014-08-13 17:38:04.348826 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.373 2014-08-13 17:38:04.348830 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cd5c0 2014-08-13 17:38:04.348816 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 17:38:04.348829 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 also missing on osd.5 2014-08-13 17:38:04.348848 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 also missing on osd.5 2014-08-13 17:38:04.348875 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.348879 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.348865 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 also missing on osd.5 2014-08-13 17:38:04.348836 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 17:38:04.348883 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 also missing on osd.5 2014-08-13 17:38:04.348891 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 17:38:04.348913 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.348901 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 also missing on osd.5 2014-08-13 17:38:04.348915 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.348917 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16e40 pg_info(1 pgs e194486:22.581(1)) v4 2014-08-13 17:38:04.348918 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 also missing on osd.5 2014-08-13 17:38:04.348909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 17:38:04.348936 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 also missing on osd.5 2014-08-13 17:38:04.348958 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.581(1)) v4 from osd.320 2014-08-13 17:38:04.348963 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16e40 2014-08-13 17:38:04.348948 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.348956 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 also missing on osd.5 2014-08-13 17:38:04.348974 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 also missing on osd.5 2014-08-13 17:38:04.348983 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map 2014-08-13 17:38:04.349001 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349004 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.348992 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 also missing on osd.5 2014-08-13 17:38:04.349010 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 also missing on osd.5 2014-08-13 17:38:04.349025 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349028 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349030 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825e80 pg_info(1 pgs e194486:22.113) v4 2014-08-13 17:38:04.349066 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.113) v4 from osd.339 2014-08-13 17:38:04.349070 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825e80 2014-08-13 17:38:04.349076 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] update_heartbeat_peers unchanged 2014-08-13 17:38:04.349024 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 17:38:04.349108 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349111 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349097 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate: adding 47 as a source 2014-08-13 17:38:04.349096 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters 2014-08-13 17:38:04.349133 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349119 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 also missing on osd.47 2014-08-13 17:38:04.349135 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349118 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Reset 0.007350 1 0.007319 2014-08-13 17:38:04.349138 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825940 pg_notify(22.3e4(31),22.707(33) epoch 194486) v5 2014-08-13 17:38:04.349139 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 also missing on osd.47 2014-08-13 17:38:04.349156 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 also missing on osd.47 2014-08-13 17:38:04.349144 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started 2014-08-13 17:38:04.349172 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 also missing on osd.47 2014-08-13 17:38:04.349209 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.342 2014-08-13 17:38:04.349214 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825940 2014-08-13 17:38:04.349200 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 also missing on osd.47 2014-08-13 17:38:04.349198 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Start 2014-08-13 17:38:04.349221 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 also missing on osd.47 2014-08-13 17:38:04.349222 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: transitioning to Stray 2014-08-13 17:38:04.349240 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 also missing on osd.47 2014-08-13 17:38:04.349245 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Start 0.000047 0 0.000000 2014-08-13 17:38:04.349269 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349273 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 also missing on osd.47 2014-08-13 17:38:04.349269 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/Stray 2014-08-13 17:38:04.349278 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 also missing on osd.47 2014-08-13 17:38:04.349295 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 also missing on osd.47 2014-08-13 17:38:04.349294 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.349313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 also missing on osd.47 2014-08-13 17:38:04.349328 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.349331 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349334 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102140 pg_notify(22.1d2(27) epoch 194486) v5 2014-08-13 17:38:04.349331 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 also missing on osd.47 2014-08-13 17:38:04.349350 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 also missing on osd.47 2014-08-13 17:38:04.349379 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.330 2014-08-13 17:38:04.349381 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102140 2014-08-13 17:38:04.349369 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 also missing on osd.47 2014-08-13 17:38:04.349388 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 also missing on osd.47 2014-08-13 17:38:04.349417 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349405 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 also missing on osd.47 2014-08-13 17:38:04.349422 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349433 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.349425 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 also missing on osd.47 2014-08-13 17:38:04.349448 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349450 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349452 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107380 pg_info(1 pgs e194486:22.f3) v4 2014-08-13 17:38:04.349443 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 also missing on osd.47 2014-08-13 17:38:04.349460 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 also missing on osd.47 2014-08-13 17:38:04.349488 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.f3) v4 from osd.312 2014-08-13 17:38:04.349478 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 also missing on osd.47 2014-08-13 17:38:04.349492 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107380 2014-08-13 17:38:04.349495 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 also missing on osd.47 2014-08-13 17:38:04.349513 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 also missing on osd.47 2014-08-13 17:38:04.349528 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349531 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349531 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 also missing on osd.47 2014-08-13 17:38:04.349551 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349553 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349555 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827000 pg_notify(22.481(39) epoch 194486) v5 2014-08-13 17:38:04.349548 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 also missing on osd.47 2014-08-13 17:38:04.349566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 also missing on osd.47 2014-08-13 17:38:04.349521 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 17:38:04.349592 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.365 2014-08-13 17:38:04.349595 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827000 2014-08-13 17:38:04.349583 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 also missing on osd.47 2014-08-13 17:38:04.349593 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.349609 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 also missing on osd.47 2014-08-13 17:38:04.349614 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.349632 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349638 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349628 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 also missing on osd.47 2014-08-13 17:38:04.349639 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.349647 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 also missing on osd.47 2014-08-13 17:38:04.349668 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349671 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.349673 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f02680 pg_notify(24.9b epoch 194486) v5 2014-08-13 17:38:04.349665 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 also missing on osd.47 2014-08-13 17:38:04.349680 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.349683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 also missing on osd.47 2014-08-13 17:38:04.349701 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.349702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 also missing on osd.47 2014-08-13 17:38:04.349719 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.378 2014-08-13 17:38:04.349721 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f02680 2014-08-13 17:38:04.349722 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 also missing on osd.47 2014-08-13 17:38:04.349721 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.349742 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 also missing on osd.47 2014-08-13 17:38:04.349757 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349760 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349760 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 also missing on osd.47 2014-08-13 17:38:04.349744 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2720 2014-08-13 17:38:04.349781 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349783 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349785 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d780 pg_info(1 pgs e194486:24.299(8)) v4 2014-08-13 17:38:04.349779 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.349802 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.826107 4 0.000362 2014-08-13 17:38:04.349826 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.299(8)) v4 from osd.396 2014-08-13 17:38:04.349831 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d780 2014-08-13 17:38:04.349824 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.349776 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 17:38:04.349842 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.949776 0 0.000000 2014-08-13 17:38:04.349843 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.349869 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349872 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349860 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 17:38:04.349870 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.349897 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349899 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349902 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825400 pg_notify(24.30b(39) epoch 194486) v5 2014-08-13 17:38:04.349888 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 5,11,47,54,368 2014-08-13 17:38:04.349889 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194478 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.349908 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.349914 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.349942 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.375 2014-08-13 17:38:04.349945 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825400 2014-08-13 17:38:04.349933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.349927 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.349984 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.349969 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.349986 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.349960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 17:38:04.349988 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.350017 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350019 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350021 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102a00 pg_info(1 pgs e194486:22.653(1)) v4 2014-08-13 17:38:04.350011 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.349992 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] flushed 2014-08-13 17:38:04.350060 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.653(1)) v4 from osd.371 2014-08-13 17:38:04.350064 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102a00 2014-08-13 17:38:04.350091 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.350100 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350103 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350124 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350126 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350128 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ca680 pg_notify(22.22c(25) epoch 194486) v5 2014-08-13 17:38:04.350163 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.368 2014-08-13 17:38:04.350165 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ca680 2014-08-13 17:38:04.350150 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 17:38:04.350181 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.350196 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.350209 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350213 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350215 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.350237 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350239 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350241 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107000 pg_notify(24.1f7(38) epoch 194486) v5 2014-08-13 17:38:04.350232 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.350281 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.351 2014-08-13 17:38:04.350283 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107000 2014-08-13 17:38:04.350294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.350320 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350322 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350311 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.350327 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.350343 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.350361 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.350364 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.350366 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824d00 pg_notify(22.775(1) epoch 194486) v5 2014-08-13 17:38:04.350369 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 2014-08-13 17:38:04.350401 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.350408 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.369 2014-08-13 17:38:04.350375 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 sending log((0'0,194215'513], crt=191727'453) 2014-08-13 17:38:04.350410 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824d00 2014-08-13 17:38:04.350416 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c77640 don't know epoch, doing nothing 2014-08-13 17:38:04.350409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.350429 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Started advmap 2014-08-13 17:38:04.350446 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,330,28] newacting [4,17,315] 2014-08-13 17:38:04.350462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.350453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 22.775( DNE v 194215'513 lc 0'0 (0'0,194215'513] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 17:38:04.350482 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.810679 1 0.005587 2014-08-13 17:38:04.350494 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.350502 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.810711 0 0.000000 2014-08-13 17:38:04.350522 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.895300 0 0.000000 2014-08-13 17:38:04.350540 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 17:38:04.350554 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194485 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194486 2014-08-13 17:38:04.350571 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194486 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 17:38:04.350586 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194486 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.350602 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194486 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,330,28] newacting [4,17,315] 2014-08-13 17:38:04.350583 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.350618 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194486 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.350628 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.350635 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28] r=0 lpr=194486 pi=193748-194484/34 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194486 2014-08-13 17:38:04.350673 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,330,28](4) acting [4,330,28](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,330,28](4) acting [4,330,28](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.350691 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194485/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] noting past interval(194485-194485 up [4,330,28](4) acting [4,330,28](4)) 2014-08-13 17:38:04.350710 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] up [4,330,28] -> [4,330,28], acting [4,330,28] -> [4,17,315], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.350730 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] clear_primary_state 2014-08-13 17:38:04.350762 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] agent_stop 2014-08-13 17:38:04.350781 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 17:38:04.350800 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194486:22347 2014-08-13 17:38:04.350773 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 sending log((0'0,194215'513], crt=191727'453) 2014-08-13 17:38:04.350829 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x57570c0 don't know epoch, doing nothing 2014-08-13 17:38:04.350817 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.350840 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 17:38:04.350856 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 17:38:04.350873 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.350862 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 22.775( DNE v 194215'513 lc 0'0 (0'0,194215'513] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 17:38:04.350888 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.350904 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:04.350919 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 17:38:04.350935 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 17:38:04.350953 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 17:38:04.350970 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 17:38:04.350986 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] [4,330,28] -> [4,17,315], replicas changed 2014-08-13 17:38:04.351003 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_recovery 2014-08-13 17:38:04.351018 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] clear_recovery_state 2014-08-13 17:38:04.351036 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.351055 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map 2014-08-13 17:38:04.351051 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.56 has 32 missing 2014-08-13 17:38:04.351079 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] update_heartbeat_peers 4,17,28,315,327,330 -> 4,17,28,315,330 2014-08-13 17:38:04.351101 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.351085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.353 has 32 missing 2014-08-13 17:38:04.351107 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 is on osd.4 2014-08-13 17:38:04.351109 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] take_waiters 2014-08-13 17:38:04.351127 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 is on osd.4 2014-08-13 17:38:04.351128 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.000589 1 0.000647 2014-08-13 17:38:04.351149 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started 2014-08-13 17:38:04.351166 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Start 2014-08-13 17:38:04.351181 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary 2014-08-13 17:38:04.351145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 is on osd.4 2014-08-13 17:38:04.351211 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000045 0 0.000000 2014-08-13 17:38:04.351232 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary 2014-08-13 17:38:04.351218 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 is on osd.4 2014-08-13 17:38:04.351250 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering 2014-08-13 17:38:04.351263 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.351268 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.351287 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 is on osd.4 2014-08-13 17:38:04.351287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.351306 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 is on osd.4 2014-08-13 17:38:04.351313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194485-194485 up [4,330,28](4) acting [4,330,28](4)) 2014-08-13 17:38:04.351334 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 17:38:04.351323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 is on osd.4 2014-08-13 17:38:04.351355 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 17:38:04.351364 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 is on osd.4 2014-08-13 17:38:04.351374 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 17:38:04.351382 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 is on osd.4 2014-08-13 17:38:04.351394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.351404 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 is on osd.4 2014-08-13 17:38:04.351416 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194486:22348 2014-08-13 17:38:04.351435 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.17 2014-08-13 17:38:04.351455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.28 2014-08-13 17:38:04.351421 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 is on osd.4 2014-08-13 17:38:04.351474 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.315 2014-08-13 17:38:04.351476 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 is on osd.4 2014-08-13 17:38:04.351492 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.327 2014-08-13 17:38:04.351494 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 is on osd.4 2014-08-13 17:38:04.351510 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.330 2014-08-13 17:38:04.351517 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 is on osd.4 2014-08-13 17:38:04.351530 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.351534 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 is on osd.4 2014-08-13 17:38:04.351548 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.351557 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 is on osd.4 2014-08-13 17:38:04.351581 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 is on osd.4 2014-08-13 17:38:04.351602 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 is on osd.4 2014-08-13 17:38:04.351622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 is on osd.4 2014-08-13 17:38:04.351638 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 is on osd.4 2014-08-13 17:38:04.351651 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.351653 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 is on osd.4 2014-08-13 17:38:04.351674 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 is on osd.4 2014-08-13 17:38:04.351689 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 is on osd.4 2014-08-13 17:38:04.351708 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 17:38:04.351705 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 is on osd.4 2014-08-13 17:38:04.351741 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.351748 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: Started advmap 2014-08-13 17:38:04.351737 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.351767 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.351786 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 17:38:04.351794 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 is on osd.4 2014-08-13 17:38:04.351816 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 is on osd.4 2014-08-13 17:38:04.351819 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 17:38:04.351836 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.351834 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 is on osd.4 2014-08-13 17:38:04.351853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.351872 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.351857 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 is on osd.4 2014-08-13 17:38:04.351884 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 is on osd.4 2014-08-13 17:38:04.351895 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 17:38:04.351918 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.351901 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 is on osd.4 2014-08-13 17:38:04.351923 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.351940 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.351957 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.351924 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 17:38:04.352018 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.352017 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate: adding 56 as a source 2014-08-13 17:38:04.352037 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.352041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 also missing on osd.56 2014-08-13 17:38:04.352053 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.352069 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.352073 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 also missing on osd.56 2014-08-13 17:38:04.352092 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 also missing on osd.56 2014-08-13 17:38:04.352093 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 2014-08-13 17:38:04.352122 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.352111 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 also missing on osd.56 2014-08-13 17:38:04.352128 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Started advmap 2014-08-13 17:38:04.352146 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364,4] 2014-08-13 17:38:04.352162 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.352134 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 also missing on osd.56 2014-08-13 17:38:04.352180 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started/Stray 0.879444 4 0.000193 2014-08-13 17:38:04.352235 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started 0.879545 0 0.000000 2014-08-13 17:38:04.352214 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 also missing on osd.56 2014-08-13 17:38:04.352255 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Reset 2014-08-13 17:38:04.352263 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 also missing on osd.56 2014-08-13 17:38:04.352272 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194485 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194486 2014-08-13 17:38:04.352282 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 also missing on osd.56 2014-08-13 17:38:04.352291 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194486 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 17:38:04.352306 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 also missing on osd.56 2014-08-13 17:38:04.352307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194486 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.352325 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194486 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364,4] 2014-08-13 17:38:04.352326 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 also missing on osd.56 2014-08-13 17:38:04.352342 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194486 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.352358 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4]/[356,364] r=-1 lpr=194486 pi=193151-194484/30 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194486 2014-08-13 17:38:04.352359 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 also missing on osd.56 2014-08-13 17:38:04.352378 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 also missing on osd.56 2014-08-13 17:38:04.352393 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [356,364,4](356) acting [356,364](356)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [356,364,4](356) acting [356,364](356)) : primary up 192969-194483 does not include interval 2014-08-13 17:38:04.352409 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 also missing on osd.56 2014-08-13 17:38:04.352410 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194485-194485 up [356,364,4](356) acting [356,364](356)) 2014-08-13 17:38:04.352430 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 also missing on osd.56 2014-08-13 17:38:04.352429 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364,4] -> [356,364,4], acting [356,364] -> [356,364,4], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> 2 2014-08-13 17:38:04.352451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:04.352449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 also missing on osd.56 2014-08-13 17:38:04.352467 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.352472 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 also missing on osd.56 2014-08-13 17:38:04.352482 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:04.352491 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 also missing on osd.56 2014-08-13 17:38:04.352500 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 17:38:04.352516 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 17:38:04.352515 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 also missing on osd.56 2014-08-13 17:38:04.352532 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 17:38:04.352548 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 17:38:04.352543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 also missing on osd.56 2014-08-13 17:38:04.352566 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 17:38:04.352568 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 also missing on osd.56 2014-08-13 17:38:04.352583 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 17:38:04.352586 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 also missing on osd.56 2014-08-13 17:38:04.352599 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_role_change 2014-08-13 17:38:04.352604 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 also missing on osd.56 2014-08-13 17:38:04.352613 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:04.352635 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 17:38:04.352632 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 also missing on osd.56 2014-08-13 17:38:04.352650 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 17:38:04.352655 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 also missing on osd.56 2014-08-13 17:38:04.352667 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.352673 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 also missing on osd.56 2014-08-13 17:38:04.352686 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map 2014-08-13 17:38:04.352702 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 also missing on osd.56 2014-08-13 17:38:04.352719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 also missing on osd.56 2014-08-13 17:38:04.352736 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 also missing on osd.56 2014-08-13 17:38:04.352752 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 also missing on osd.56 2014-08-13 17:38:04.352761 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] update_heartbeat_peers unchanged 2014-08-13 17:38:04.352769 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 also missing on osd.56 2014-08-13 17:38:04.352780 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters 2014-08-13 17:38:04.352796 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.000541 1 0.000558 2014-08-13 17:38:04.352796 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 also missing on osd.56 2014-08-13 17:38:04.352817 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started 2014-08-13 17:38:04.352817 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 also missing on osd.56 2014-08-13 17:38:04.352835 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start 2014-08-13 17:38:04.352851 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray 2014-08-13 17:38:04.352869 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000035 0 0.000000 2014-08-13 17:38:04.352888 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray 2014-08-13 17:38:04.352843 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 17:38:04.352907 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate: adding 353 as a source 2014-08-13 17:38:04.352908 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.352926 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.352937 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 also missing on osd.353 2014-08-13 17:38:04.352962 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 also missing on osd.353 2014-08-13 17:38:04.352980 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 also missing on osd.353 2014-08-13 17:38:04.353018 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.353007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 also missing on osd.353 2014-08-13 17:38:04.353026 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 also missing on osd.353 2014-08-13 17:38:04.353042 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 also missing on osd.353 2014-08-13 17:38:04.353063 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 also missing on osd.353 2014-08-13 17:38:04.353063 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 2014-08-13 17:38:04.353081 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 also missing on osd.353 2014-08-13 17:38:04.353094 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.353098 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 also missing on osd.353 2014-08-13 17:38:04.353101 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.353122 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Started advmap 2014-08-13 17:38:04.353121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 also missing on osd.353 2014-08-13 17:38:04.353139 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [4,59,351] newacting [4,351,55] 2014-08-13 17:38:04.353155 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.353172 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary/Peering/WaitActingChange 0.819158 1 0.000213 2014-08-13 17:38:04.353200 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.819196 0 0.000000 2014-08-13 17:38:04.353219 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started 0.878824 0 0.000000 2014-08-13 17:38:04.353237 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset 2014-08-13 17:38:04.353208 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 also missing on osd.353 2014-08-13 17:38:04.353251 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194485 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194486 2014-08-13 17:38:04.353267 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 also missing on osd.353 2014-08-13 17:38:04.353268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194486 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Reset advmap 2014-08-13 17:38:04.353285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194486 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.353286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 also missing on osd.353 2014-08-13 17:38:04.353301 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194486 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [4,59,351] newacting [4,351,55] 2014-08-13 17:38:04.353319 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194486 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.353311 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 also missing on osd.353 2014-08-13 17:38:04.353334 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351] r=0 lpr=194486 pi=193678-194484/34 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194486 2014-08-13 17:38:04.353341 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 also missing on osd.353 2014-08-13 17:38:04.353360 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 also missing on osd.353 2014-08-13 17:38:04.353369 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,59,351](4) acting [4,59,351](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,59,351](4) acting [4,59,351](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.353383 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 also missing on osd.353 2014-08-13 17:38:04.353388 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194485/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] noting past interval(194485-194485 up [4,59,351](4) acting [4,59,351](4)) 2014-08-13 17:38:04.353408 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] up [4,59,351] -> [4,59,351], acting [4,59,351] -> [4,351,55], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.353435 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] clear_primary_state 2014-08-13 17:38:04.353402 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 also missing on osd.353 2014-08-13 17:38:04.353455 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] agent_stop 2014-08-13 17:38:04.353459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 also missing on osd.353 2014-08-13 17:38:04.353473 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change 2014-08-13 17:38:04.353483 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 also missing on osd.353 2014-08-13 17:38:04.353494 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] publish_stats_to_osd 194486:25635 2014-08-13 17:38:04.353512 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 17:38:04.353502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 also missing on osd.353 2014-08-13 17:38:04.353530 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_copy_ops 2014-08-13 17:38:04.353536 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 also missing on osd.353 2014-08-13 17:38:04.353547 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_flush_ops 2014-08-13 17:38:04.353528 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] flushed 2014-08-13 17:38:04.353556 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 also missing on osd.353 2014-08-13 17:38:04.353565 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 17:38:04.353580 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 also missing on osd.353 2014-08-13 17:38:04.353582 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 17:38:04.353599 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 17:38:04.353598 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 also missing on osd.353 2014-08-13 17:38:04.353614 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change_cleanup 2014-08-13 17:38:04.353631 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 also missing on osd.353 2014-08-13 17:38:04.353631 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change 2014-08-13 17:38:04.353686 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 also missing on osd.353 2014-08-13 17:38:04.353694 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit NotTrimming 2014-08-13 17:38:04.353716 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter NotTrimming 2014-08-13 17:38:04.353733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] [4,59,351] -> [4,351,55], replicas changed 2014-08-13 17:38:04.353709 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 also missing on osd.353 2014-08-13 17:38:04.353751 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_recovery 2014-08-13 17:38:04.353761 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 also missing on osd.353 2014-08-13 17:38:04.353767 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] clear_recovery_state 2014-08-13 17:38:04.353781 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 also missing on osd.353 2014-08-13 17:38:04.353786 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.353804 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 also missing on osd.353 2014-08-13 17:38:04.353806 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map 2014-08-13 17:38:04.353821 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 also missing on osd.353 2014-08-13 17:38:04.353827 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] update_heartbeat_peers 4,55,59,351,376 -> 4,55,59,351 2014-08-13 17:38:04.353848 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.353851 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] take_waiters 2014-08-13 17:38:04.353869 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Reset 0.000633 1 0.000706 2014-08-13 17:38:04.353889 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started 2014-08-13 17:38:04.353838 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 17:38:04.353906 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Start 2014-08-13 17:38:04.353923 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: transitioning to Primary 2014-08-13 17:38:04.353912 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.353942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.353943 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Start 0.000036 0 0.000000 2014-08-13 17:38:04.353965 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary 2014-08-13 17:38:04.353963 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 0,56,63,353,369 2014-08-13 17:38:04.353982 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Peering 2014-08-13 17:38:04.353986 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.354000 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.354007 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.354017 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.354037 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194485-194485 up [4,59,351](4) acting [4,59,351](4)) 2014-08-13 17:38:04.354041 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.354057 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 17:38:04.354060 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.354077 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 17:38:04.354096 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 17:38:04.354114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.354133 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194486:25636 2014-08-13 17:38:04.354151 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.55 2014-08-13 17:38:04.354169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.59 2014-08-13 17:38:04.354194 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.351 2014-08-13 17:38:04.354214 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.376 2014-08-13 17:38:04.354236 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.354232 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.354250 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.354312 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 17:38:04.354345 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.354348 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.354377 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354381 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354391 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.354412 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354414 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354416 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824600 pg_info(1 pgs e194486:22.1d2) v4 2014-08-13 17:38:04.354413 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 2014-08-13 17:38:04.354413 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.354450 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.354457 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1d2) v4 from osd.349 2014-08-13 17:38:04.354462 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824600 2014-08-13 17:38:04.354454 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.354457 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.354478 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Started advmap 2014-08-13 17:38:04.354495 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [4,55,61] newacting [4,50,65] 2014-08-13 17:38:04.354508 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354511 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354511 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.354531 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354533 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354535 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f02d80 pg_notify(24.9b(34) epoch 194486) v5 2014-08-13 17:38:04.354528 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary/Peering/WaitActingChange 0.814824 4 0.005664 2014-08-13 17:38:04.354548 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.814854 0 0.000000 2014-08-13 17:38:04.354549 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.354575 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.350 2014-08-13 17:38:04.354578 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f02d80 2014-08-13 17:38:04.354568 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started 0.873527 0 0.000000 2014-08-13 17:38:04.354572 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.354587 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset 2014-08-13 17:38:04.354590 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.354609 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.354604 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194485 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194486 2014-08-13 17:38:04.354618 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.354618 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354626 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354625 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c71b80 already has epoch 194486 2014-08-13 17:38:04.354622 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194486 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Reset advmap 2014-08-13 17:38:04.354629 7fe99ebe8700 7 osd.4 194486 do_notify osd 20 on 1 PGs 2014-08-13 17:38:04.354638 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194486 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.354654 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194486 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [4,55,61] newacting [4,50,65] 2014-08-13 17:38:04.354667 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4cfb2e0 192.168.1.101:6812/23402 2014-08-13 17:38:04.354678 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354680 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354669 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194486 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.354683 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db0d80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.354692 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.354697 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.354700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354685 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61] r=0 lpr=194486 pi=193747-194484/31 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194486 2014-08-13 17:38:04.354702 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354708 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.294 has 194486 2014-08-13 17:38:04.354712 7fe99ebe8700 7 osd.4 194486 do_notify osd 294 on 1 PGs 2014-08-13 17:38:04.354722 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354727 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354729 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e900 pg_notify(22.3d5(2) epoch 194486) v5 2014-08-13 17:38:04.354721 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,55,61](4) acting [4,55,61](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,55,61](4) acting [4,55,61](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.354740 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194485/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] noting past interval(194485-194485 up [4,55,61](4) acting [4,55,61](4)) 2014-08-13 17:38:04.354766 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.10 2014-08-13 17:38:04.354769 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e900 2014-08-13 17:38:04.354759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] up [4,55,61] -> [4,55,61], acting [4,55,61] -> [4,50,65], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.354783 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] clear_primary_state 2014-08-13 17:38:04.354806 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] agent_stop 2014-08-13 17:38:04.354832 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354835 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354824 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change 2014-08-13 17:38:04.354855 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354858 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354843 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] publish_stats_to_osd 194486:22524 2014-08-13 17:38:04.354859 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16900 pg_info(1 pgs e194486:22.14a(2)) v4 2014-08-13 17:38:04.354861 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 17:38:04.354880 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_copy_ops 2014-08-13 17:38:04.354898 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.14a(2)) v4 from osd.58 2014-08-13 17:38:04.354903 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16900 2014-08-13 17:38:04.354897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_flush_ops 2014-08-13 17:38:04.354913 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 17:38:04.354939 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354941 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354928 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 17:38:04.354944 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 17:38:04.354964 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.354966 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.354968 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299ecc0 pg_query(24.33a epoch 194486) v3 2014-08-13 17:38:04.354960 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change_cleanup 2014-08-13 17:38:04.354977 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change 2014-08-13 17:38:04.354996 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit NotTrimming 2014-08-13 17:38:04.355011 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.304 epoch 194486 2014-08-13 17:38:04.355014 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299ecc0 2014-08-13 17:38:04.355013 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter NotTrimming 2014-08-13 17:38:04.355031 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] [4,55,61] -> [4,50,65], replicas changed 2014-08-13 17:38:04.355026 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(info 0'0) from replica 304 2014-08-13 17:38:04.355048 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_recovery 2014-08-13 17:38:04.355071 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355074 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355064 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] clear_recovery_state 2014-08-13 17:38:04.355095 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355097 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.355099 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58278c0 pg_notify(24.34(52) epoch 194486) v5 2014-08-13 17:38:04.355103 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map 2014-08-13 17:38:04.355137 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.340 2014-08-13 17:38:04.355140 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58278c0 2014-08-13 17:38:04.355123 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] update_heartbeat_peers 4,50,55,61,65,356 -> 4,50,55,61,65 2014-08-13 17:38:04.355144 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.355147 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] take_waiters 2014-08-13 17:38:04.355165 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Reset 0.000578 1 0.000646 2014-08-13 17:38:04.355180 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355183 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355192 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started 2014-08-13 17:38:04.355210 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Start 2014-08-13 17:38:04.355229 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355232 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355234 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ca840 pg_info(1 pgs e194486:22.99) v4 2014-08-13 17:38:04.355226 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: transitioning to Primary 2014-08-13 17:38:04.355245 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Start 0.000035 0 0.000000 2014-08-13 17:38:04.355270 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.99) v4 from osd.31 2014-08-13 17:38:04.355275 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ca840 2014-08-13 17:38:04.355266 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary 2014-08-13 17:38:04.355283 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Peering 2014-08-13 17:38:04.355296 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.355306 7fe99ebe8700 7 osd.4 194486 do_notify osd 304 on 1 PGs 2014-08-13 17:38:04.355310 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355313 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355301 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.355290 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.355333 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355320 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.355335 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355337 7fe9a63f4700 20 osd.4 194486 _dispatch 0x59714a0 pg_query(24.350,24.3b7 epoch 194486) v3 2014-08-13 17:38:04.355341 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4c73f40 192.168.1.110:6812/43236 2014-08-13 17:38:04.355336 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.355339 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194485-194485 up [4,55,61](4) acting [4,55,61](4)) 2014-08-13 17:38:04.355361 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 17:38:04.355380 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.356 epoch 194486 2014-08-13 17:38:04.355384 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x59714a0 2014-08-13 17:38:04.355390 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.321 has 194486 2014-08-13 17:38:04.355394 7fe99ebe8700 7 osd.4 194486 do_notify osd 321 on 1 PGs 2014-08-13 17:38:04.355385 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194482-194482 up [50,65,356](50) acting [50,65](50)) 2014-08-13 17:38:04.355395 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(info 0'0) from replica 356 2014-08-13 17:38:04.355405 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 17:38:04.355425 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.355424 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 356 2014-08-13 17:38:04.355448 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194486:22525 2014-08-13 17:38:04.355465 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355468 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355466 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.50 2014-08-13 17:38:04.355487 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355480 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:04.355491 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355493 7fe99ebe8700 7 osd.4 194486 do_notify osd 324 on 1 PGs 2014-08-13 17:38:04.355494 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d92f40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.355486 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.55 2014-08-13 17:38:04.355502 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.355506 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.355509 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355511 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355505 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.61 2014-08-13 17:38:04.355526 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:04.355532 7fe99ebe8700 7 osd.4 194486 do_notify osd 328 on 1 PGs 2014-08-13 17:38:04.355533 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355536 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355538 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822300 pg_info(1 pgs e194486:22.2b9(2)) v4 2014-08-13 17:38:04.355524 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.65 2014-08-13 17:38:04.355553 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:04.355556 7fe99ebe8700 7 osd.4 194486 do_notify osd 330 on 1 PGs 2014-08-13 17:38:04.355550 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.356 2014-08-13 17:38:04.355577 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2b9(2)) v4 from osd.328 2014-08-13 17:38:04.355575 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4c70f20 192.168.1.110:6820/46337 2014-08-13 17:38:04.355581 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822300 2014-08-13 17:38:04.355570 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.355589 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.355604 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.345 has 194486 2014-08-13 17:38:04.355608 7fe99ebe8700 7 osd.4 194486 do_notify osd 345 on 1 PGs 2014-08-13 17:38:04.355617 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355621 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355650 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355652 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355654 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822680 pg_notify(24.30b(2) epoch 194486) v5 2014-08-13 17:38:04.355672 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing 2014-08-13 17:38:04.355677 7fe99ebe8700 7 osd.4 194486 do_notify osd 364 on 1 PGs 2014-08-13 17:38:04.355690 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.301 2014-08-13 17:38:04.355693 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822680 2014-08-13 17:38:04.355698 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c71b80 already has epoch 194486 2014-08-13 17:38:04.355703 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.355703 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.20 on 1 PGs 2014-08-13 17:38:04.355731 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355733 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355752 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355756 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140000 pg_notify(22.1f1(29) epoch 194486) v5 2014-08-13 17:38:04.355790 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.54 2014-08-13 17:38:04.355793 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140000 2014-08-13 17:38:04.355825 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355853 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355855 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355857 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ca140 pg_notify(24.9b(34) epoch 194486) v5 2014-08-13 17:38:04.355858 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing 2014-08-13 17:38:04.355867 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.25 on 1 PGs 2014-08-13 17:38:04.355844 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 17:38:04.355875 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.355894 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.291 2014-08-13 17:38:04.355882 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.355897 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x592c360 don't know epoch, doing nothing 2014-08-13 17:38:04.355898 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ca140 2014-08-13 17:38:04.355902 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.26 on 1 PGs 2014-08-13 17:38:04.355901 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.355924 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing 2014-08-13 17:38:04.355928 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.30 on 1 PGs 2014-08-13 17:38:04.355922 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.355937 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355940 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355946 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:04.355949 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.303 on 1 PGs 2014-08-13 17:38:04.355969 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.355972 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.340 on 1 PGs 2014-08-13 17:38:04.355980 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.355982 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.355986 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820000 pg_info(1 pgs e194486:22.791(2)) v4 2014-08-13 17:38:04.355994 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:04.355998 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.362 on 1 PGs 2014-08-13 17:38:04.355994 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.356020 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:04.356023 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.373 on 1 PGs 2014-08-13 17:38:04.356012 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.356025 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.791(2)) v4 from osd.30 2014-08-13 17:38:04.356030 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820000 2014-08-13 17:38:04.356029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.356043 7fe99ebe8700 20 osd.4 194486 Sending info 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 28 2014-08-13 17:38:04.363735 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.363739 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:04.363770 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.363781 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.363784 7fe99ebe8700 20 osd.4 194486 Sending info 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 304 2014-08-13 17:38:04.363796 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.363772 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 17:38:04.363804 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.363815 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.363818 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.363820 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc980 pg_info(1 pgs e194486:22.f3) v4 2014-08-13 17:38:04.363810 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.363828 7fe99ebe8700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.363829 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.363847 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.363865 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.f3) v4 from osd.285 2014-08-13 17:38:04.363871 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc980 2014-08-13 17:38:04.363909 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.363932 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.363935 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.363925 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.363942 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.363926 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 17:38:04.363959 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.363962 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.363964 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.363967 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.363969 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ce040 pg_info(1 pgs e194486:22.116(1)) v4 2014-08-13 17:38:04.363970 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.363990 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.363992 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,32] -- 356/356 2014-08-13 17:38:04.364015 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.116(1)) v4 from osd.345 2014-08-13 17:38:04.364019 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ce040 2014-08-13 17:38:04.364009 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.364024 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.364029 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.364031 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:04.364058 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364061 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364049 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.364049 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,32] 2014-08-13 17:38:04.364081 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364069 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.364083 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364071 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.364086 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_notify(24.3d(2) epoch 194486) v5 2014-08-13 17:38:04.364088 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.878974 4 0.000184 2014-08-13 17:38:04.364091 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2702 2014-08-13 17:38:04.364110 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.364110 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started 0.879037 0 0.000000 2014-08-13 17:38:04.364127 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.20 2014-08-13 17:38:04.364130 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.364132 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 17:38:04.364131 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.831429 6 0.000527 2014-08-13 17:38:04.364150 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194485 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.364172 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364175 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364154 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.364168 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194486 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:04.364186 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.871063 0 0.000000 2014-08-13 17:38:04.364194 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194486 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.364224 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364227 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364212 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.364211 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194486 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,32] 2014-08-13 17:38:04.364229 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51ce3c0 pg_info(1 pgs e194486:22.99) v4 2014-08-13 17:38:04.364231 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.364231 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194486 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.364254 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334] r=1 lpr=194486 pi=193150-194484/38 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.364269 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.99) v4 from osd.374 2014-08-13 17:38:04.364273 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51ce3c0 2014-08-13 17:38:04.364260 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.364280 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.364289 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [356,4,334](356) acting [356,4,334](356)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [356,4,334](356) acting [356,4,334](356)) : primary up 192969-194483 does not include interval 2014-08-13 17:38:04.364298 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'504 -> 194215'504 2014-08-13 17:38:04.364316 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364319 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364311 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194485/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194485-194485 up [356,4,334](356) acting [356,4,334](356)) 2014-08-13 17:38:04.364340 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364322 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.364342 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364345 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58240c0 pg_notify(22.362(1) epoch 194486) v5 2014-08-13 17:38:04.364331 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,4,334] -> [356,4,334], acting [356,4,334] -> [356,4,32], acting_primary 356 -> 356, up_primary 356 -> 356, role 1 -> 1 2014-08-13 17:38:04.364344 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 is up to date, queueing in pending_activators 2014-08-13 17:38:04.364354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:04.364373 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.364393 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.61 2014-08-13 17:38:04.364396 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58240c0 2014-08-13 17:38:04.364389 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:04.364385 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.364409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:04.364407 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.364436 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364438 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364426 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:04.364428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 is up to date, queueing in pending_activators 2014-08-13 17:38:04.364458 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364445 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:04.364464 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299b660 pg_query(24.1de epoch 194486) v3 2014-08-13 17:38:04.364453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.364465 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:04.364477 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.364487 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:04.364503 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.364 epoch 194486 2014-08-13 17:38:04.364507 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299b660 2014-08-13 17:38:04.364496 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.364507 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:04.364516 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.364519 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_query query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192966) from replica 364 2014-08-13 17:38:04.364527 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:04.364537 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.364553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:04.364555 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.364569 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364572 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.364591 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364594 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364595 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299bc00 pg_query(24.369 epoch 194486) v3 2014-08-13 17:38:04.364595 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 17:38:04.364624 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.364631 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.45 epoch 194486 2014-08-13 17:38:04.364634 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299bc00 2014-08-13 17:38:04.364644 7fe9a63f4700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_query query(info 0'0) from replica 45 2014-08-13 17:38:04.364682 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364685 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364674 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [4,344,296]/[4,296,320] -- 4/4 2014-08-13 17:38:04.364704 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364707 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364709 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db33c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.364709 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.364703 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 17:38:04.364719 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.364725 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.364728 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364730 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364717 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Started advmap 2014-08-13 17:38:04.364724 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 17:38:04.364750 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.364752 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.364754 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826580 pg_notify(24.181(12) epoch 194486) v5 2014-08-13 17:38:04.364738 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [4,296,320] 2014-08-13 17:38:04.364744 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000611 1 0.000564 2014-08-13 17:38:04.364759 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.364767 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started 2014-08-13 17:38:04.364780 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started/Stray 0.870058 4 0.000169 2014-08-13 17:38:04.364785 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Start 2014-08-13 17:38:04.364807 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.55 2014-08-13 17:38:04.364810 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826580 2014-08-13 17:38:04.364802 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray 2014-08-13 17:38:04.364803 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started 0.870124 0 0.000000 2014-08-13 17:38:04.364824 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000039 0 0.000000 2014-08-13 17:38:04.364826 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Reset 2014-08-13 17:38:04.364846 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray 2014-08-13 17:38:04.364851 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194485 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194486 2014-08-13 17:38:04.364868 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.364870 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194486 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 17:38:04.364886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.364888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194486 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:04.364905 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194486 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [4,296,320] 2014-08-13 17:38:04.364924 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194486 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.364941 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[320,307,296] r=-1 lpr=194486 pi=193679-194484/36 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194486 2014-08-13 17:38:04.364991 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.364977 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,344,296](4) acting [320,307,296](320)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194466 generate_past_intervals interval(194485-194485 up [4,344,296](4) acting [320,307,296](320)) : primary up 192979-194483 does not include interval 2014-08-13 17:38:04.364995 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194485/194483) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] noting past interval(194485-194485 up [4,344,296](4) acting [320,307,296](320)) 2014-08-13 17:38:04.365013 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] up [4,344,296] -> [4,344,296], acting [320,307,296] -> [4,296,320], acting_primary 320 -> 4, up_primary 4 -> 4, role -1 -> 0 2014-08-13 17:38:04.365035 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] clear_primary_state 2014-08-13 17:38:04.365015 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.365044 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 17:38:04.365055 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] agent_stop 2014-08-13 17:38:04.365071 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.365062 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.365074 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:04.365077 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.365096 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.365096 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd 194486:28882 2014-08-13 17:38:04.365115 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.365117 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:04.365136 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 17:38:04.365152 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 17:38:04.365169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.365178 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:04.365195 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.365195 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:04.365215 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.365233 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.365218 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 17:38:04.365236 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 17:38:04.365252 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 17:38:04.365261 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 17:38:04.365283 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.365271 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 17:38:04.365289 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.365290 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 17:38:04.365305 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.365308 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] clear_stats 2014-08-13 17:38:04.365323 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.365327 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops 2014-08-13 17:38:04.365343 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] on_role_change 2014-08-13 17:38:04.365358 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops 2014-08-13 17:38:04.365373 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] cancel_recovery 2014-08-13 17:38:04.365386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.365388 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] clear_recovery_state 2014-08-13 17:38:04.365402 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.365417 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.365407 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.365435 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.365443 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.365449 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:04.365438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_activate_map 2014-08-13 17:38:04.365453 7fe99f5e9700 7 osd.4 194486 do_notify osd 17 on 1 PGs 2014-08-13 17:38:04.365459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] update_heartbeat_peers -> 4,296,320,344 2014-08-13 17:38:04.365477 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.365487 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 don't know epoch, doing nothing 2014-08-13 17:38:04.365491 7fe99f5e9700 7 osd.4 194486 do_notify osd 19 on 1 PGs 2014-08-13 17:38:04.365480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] take_waiters 2014-08-13 17:38:04.365452 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] flushed 2014-08-13 17:38:04.365512 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing 2014-08-13 17:38:04.365516 7fe99f5e9700 7 osd.4 194486 do_notify osd 30 on 1 PGs 2014-08-13 17:38:04.365499 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Reset 0.000673 1 0.000720 2014-08-13 17:38:04.365535 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:04.365522 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started 2014-08-13 17:38:04.365538 7fe99f5e9700 7 osd.4 194486 do_notify osd 289 on 1 PGs 2014-08-13 17:38:04.365539 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Start 2014-08-13 17:38:04.365557 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 don't know epoch, doing nothing 2014-08-13 17:38:04.365560 7fe99f5e9700 7 osd.4 194486 do_notify osd 334 on 1 PGs 2014-08-13 17:38:04.365556 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: transitioning to Primary 2014-08-13 17:38:04.365582 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:04.365584 7fe99f5e9700 7 osd.4 194486 do_notify osd 337 on 1 PGs 2014-08-13 17:38:04.365577 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.365598 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary 2014-08-13 17:38:04.365615 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Peering 2014-08-13 17:38:04.365632 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.365648 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 17:38:04.365667 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194485-194485 up [4,344,296](4) acting [320,307,296](320)) 2014-08-13 17:38:04.365686 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194483-194484 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) 2014-08-13 17:38:04.365706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194482-194482 up [320,307,55](320) acting [296,320](296)) 2014-08-13 17:38:04.365724 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior final: probe 4,296,307,320,344 down blocked_by {} 2014-08-13 17:38:04.365742 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.365750 7fe99f5e9700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e4a100 192.168.1.104:6820/2308 2014-08-13 17:38:04.365763 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] publish_stats_to_osd 194486:28883 2014-08-13 17:38:04.365790 7fe99f5e9700 10 osd.4 194486 note_peer_epoch osd.346 has 194486 2014-08-13 17:38:04.365794 7fe99f5e9700 7 osd.4 194486 do_notify osd 346 on 1 PGs 2014-08-13 17:38:04.365781 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.296 2014-08-13 17:38:04.365801 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.307 2014-08-13 17:38:04.365818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.320 2014-08-13 17:38:04.365835 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.344 2014-08-13 17:38:04.365853 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.365870 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.365877 7fe99f5e9700 7 osd.4 194486 do_notify osd 356 on 2 PGs 2014-08-13 17:38:04.365870 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.365903 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 don't know epoch, doing nothing 2014-08-13 17:38:04.365906 7fe99f5e9700 7 osd.4 194486 do_notify osd 358 on 1 PGs 2014-08-13 17:38:04.365925 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:04.365928 7fe99f5e9700 7 osd.4 194486 do_notify osd 367 on 1 PGs 2014-08-13 17:38:04.365950 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:04.365953 7fe99f5e9700 7 osd.4 194486 do_notify osd 371 on 1 PGs 2014-08-13 17:38:04.365971 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:04.365974 7fe99f5e9700 7 osd.4 194486 do_notify osd 373 on 1 PGs 2014-08-13 17:38:04.366049 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e48840 don't know epoch, doing nothing 2014-08-13 17:38:04.366053 7fe99f5e9700 7 osd.4 194486 do_notify osd 397 on 1 PGs 2014-08-13 17:38:04.366075 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:04.366078 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.17 on 1 PGs 2014-08-13 17:38:04.366202 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:04.366211 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.28 on 1 PGs 2014-08-13 17:38:04.366226 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.366262 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing 2014-08-13 17:38:04.366267 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.50 on 1 PGs 2014-08-13 17:38:04.366288 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing 2014-08-13 17:38:04.366291 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.55 on 2 PGs 2014-08-13 17:38:04.366312 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x59cf640 don't know epoch, doing nothing 2014-08-13 17:38:04.366315 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.59 on 1 PGs 2014-08-13 17:38:04.366303 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,61] -- 15/15 2014-08-13 17:38:04.366337 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.366334 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:04.366344 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.61 on 1 PGs 2014-08-13 17:38:04.366344 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:04.366362 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,61] 2014-08-13 17:38:04.366378 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.366395 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.864159 4 0.000167 2014-08-13 17:38:04.366413 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started 0.864223 0 0.000000 2014-08-13 17:38:04.366432 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 17:38:04.366450 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing 2014-08-13 17:38:04.366458 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.65 on 1 PGs 2014-08-13 17:38:04.366458 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366447 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194485 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.366463 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366465 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194486 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 17:38:04.366484 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ecce60 don't know epoch, doing nothing 2014-08-13 17:38:04.366488 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.315 on 1 PGs 2014-08-13 17:38:04.366480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194486 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.366503 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366506 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366509 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820540 pg_notify(24.1bb(35) epoch 194486) v5 2014-08-13 17:38:04.366496 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194486 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,61] 2014-08-13 17:38:04.366513 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194486 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.366529 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52] r=1 lpr=194486 pi=193264-194484/34 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194486 2014-08-13 17:38:04.366552 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.337 2014-08-13 17:38:04.366555 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820540 2014-08-13 17:38:04.366562 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [15,4,52](15) acting [15,4,52](15)): not rw, up_thru 194483 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [15,4,52](15) acting [15,4,52](15)) : primary up 192969-194483 does not include interval 2014-08-13 17:38:04.366595 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366580 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194485/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194485-194485 up [15,4,52](15) acting [15,4,52](15)) 2014-08-13 17:38:04.366597 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366603 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e82520 don't know epoch, doing nothing 2014-08-13 17:38:04.366611 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.327 on 1 PGs 2014-08-13 17:38:04.366599 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,4,52] -> [15,4,52], acting [15,4,52] -> [15,4,61], acting_primary 15 -> 15, up_primary 15 -> 15, role 1 -> 1 2014-08-13 17:38:04.366630 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366632 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366620 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:04.366634 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:04.366635 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16200 pg_info(1 pgs e194486:22.362) v4 2014-08-13 17:38:04.366638 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.330 on 1 PGs 2014-08-13 17:38:04.366637 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.366658 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:04.366661 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.351 on 1 PGs 2014-08-13 17:38:04.366654 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:04.366678 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.362) v4 from osd.343 2014-08-13 17:38:04.366681 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.366682 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16200 2014-08-13 17:38:04.366684 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.356 on 1 PGs 2014-08-13 17:38:04.366674 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 17:38:04.366703 7fe99f5e9700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e4d280 192.168.1.108:6838/6893 2014-08-13 17:38:04.366721 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366726 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366733 7fe99f5e9700 10 osd.4 194486 note_peer_epoch osd.376 has 194486 2014-08-13 17:38:04.366737 7fe99f5e9700 7 osd.4 194486 do_queries querying osd.376 on 1 PGs 2014-08-13 17:38:04.366749 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366751 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366753 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5460240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.366761 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.366766 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.366768 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366770 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366787 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.366789 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.366791 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826c80 pg_notify(24.30b epoch 194486) v5 2014-08-13 17:38:04.366691 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 17:38:04.366807 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.366805 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 17:38:04.366828 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.286 2014-08-13 17:38:04.366831 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826c80 2014-08-13 17:38:04.366826 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 17:38:04.366846 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 17:38:04.366864 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 17:38:04.366880 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 17:38:04.366895 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 17:38:04.366904 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 2014-08-13 17:38:04.366914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.366938 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.366935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 17:38:04.366946 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Started advmap 2014-08-13 17:38:04.366963 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,12] 2014-08-13 17:38:04.366979 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.366996 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started/Stray 0.873270 4 0.000197 2014-08-13 17:38:04.367010 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 17:38:04.367016 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started 0.873331 0 0.000000 2014-08-13 17:38:04.367040 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367043 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367029 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 17:38:04.367038 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset 2014-08-13 17:38:04.367049 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000616 1 0.000590 2014-08-13 17:38:04.367067 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367070 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367056 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194485 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194486 2014-08-13 17:38:04.367072 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58216c0 pg_notify(1.2b(1) epoch 194486) v5 2014-08-13 17:38:04.367072 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started 2014-08-13 17:38:04.367074 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194486 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 17:38:04.367092 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194486 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.367092 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Start 2014-08-13 17:38:04.367114 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.324 2014-08-13 17:38:04.367117 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58216c0 2014-08-13 17:38:04.367110 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194486 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,12] 2014-08-13 17:38:04.367111 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray 2014-08-13 17:38:04.367127 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194486 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.367133 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000041 0 0.000000 2014-08-13 17:38:04.367145 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314] r=1 lpr=194486 pi=192688-194484/47 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194486 2014-08-13 17:38:04.367160 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367163 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367156 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray 2014-08-13 17:38:04.367190 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367192 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367194 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823100 pg_notify(22.3e4(1) epoch 194486) v5 2014-08-13 17:38:04.367189 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.367180 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [374,4,314](374) acting [374,4,314](374)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [374,4,314](374) acting [374,4,314](374)) : primary up 192967-194483 does not include interval 2014-08-13 17:38:04.367209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.367211 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194485/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194485-194485 up [374,4,314](374) acting [374,4,314](374)) 2014-08-13 17:38:04.367246 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.303 2014-08-13 17:38:04.367249 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823100 2014-08-13 17:38:04.367234 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,4,314] -> [374,4,314], acting [374,4,314] -> [374,4,12], acting_primary 374 -> 374, up_primary 374 -> 374, role 1 -> 1 2014-08-13 17:38:04.367257 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:04.367274 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.367298 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367301 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367290 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:04.367313 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.367321 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367309 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 17:38:04.367324 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367326 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821180 pg_info(1 pgs e194486:22.226(1)) v4 2014-08-13 17:38:04.367325 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 17:38:04.367342 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 17:38:04.367363 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.226(1)) v4 from osd.376 2014-08-13 17:38:04.367368 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821180 2014-08-13 17:38:04.367359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 17:38:04.367367 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,4] -- 31/31 2014-08-13 17:38:04.367379 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 17:38:04.367401 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.367406 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367409 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367397 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 17:38:04.367414 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 17:38:04.367430 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 17:38:04.367448 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367451 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367453 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cb9c0 pg_info(1 pgs e194486:22.1fb(2)) v4 2014-08-13 17:38:04.367418 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Started advmap 2014-08-13 17:38:04.367447 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.367460 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,4] 2014-08-13 17:38:04.367472 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map 2014-08-13 17:38:04.367481 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.367500 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started/Stray 0.863960 4 0.004403 2014-08-13 17:38:04.367515 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1fb(2)) v4 from osd.373 2014-08-13 17:38:04.367520 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cb9c0 2014-08-13 17:38:04.367521 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started 0.864023 0 0.000000 2014-08-13 17:38:04.367541 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Reset 2014-08-13 17:38:04.367556 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367559 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367558 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194485 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194486 2014-08-13 17:38:04.367580 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367582 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367584 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102f40 pg_notify(22.39d(29) epoch 194486) v5 2014-08-13 17:38:04.367576 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194486 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 17:38:04.367593 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194486 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 17:38:04.367598 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] update_heartbeat_peers unchanged 2014-08-13 17:38:04.367621 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.320 2014-08-13 17:38:04.367625 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102f40 2014-08-13 17:38:04.367610 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194486 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,4] 2014-08-13 17:38:04.367622 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters 2014-08-13 17:38:04.367631 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194486 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.367643 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Reset 0.000606 1 0.000527 2014-08-13 17:38:04.367649 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4]/[31,338,16] r=-1 lpr=194486 pi=193125-194484/42 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194486 2014-08-13 17:38:04.367663 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367666 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367667 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started 2014-08-13 17:38:04.367685 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Start 2014-08-13 17:38:04.367701 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367684 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [31,338,4](31) acting [31,338,16](31)): not rw, up_thru 194483 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194485-194485 up [31,338,4](31) acting [31,338,16](31)) : primary up 192978-194483 does not include interval 2014-08-13 17:38:04.367704 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367707 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299fda0 pg_query(24.3f7 epoch 194486) v3 2014-08-13 17:38:04.367703 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: transitioning to Stray 2014-08-13 17:38:04.367706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194485/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] noting past interval(194485-194485 up [31,338,4](31) acting [31,338,16](31)) 2014-08-13 17:38:04.367725 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Start 0.000041 0 0.000000 2014-08-13 17:38:04.367745 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.330 epoch 194486 2014-08-13 17:38:04.367728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] up [31,338,4] -> [31,338,4], acting [31,338,16] -> [31,338,4], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> 2 2014-08-13 17:38:04.367749 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299fda0 2014-08-13 17:38:04.367749 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/Stray 2014-08-13 17:38:04.367751 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:04.367772 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.367776 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.367762 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_query query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193227) from replica 330 2014-08-13 17:38:04.367788 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:04.367797 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.367820 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367808 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_copy_ops 2014-08-13 17:38:04.367824 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367826 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_flush_ops 2014-08-13 17:38:04.367844 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367847 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367848 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826e40 pg_notify(24.9b(2) epoch 194486) v5 2014-08-13 17:38:04.367842 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change_cleanup 2014-08-13 17:38:04.367859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 17:38:04.367888 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.365 2014-08-13 17:38:04.367890 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826e40 2014-08-13 17:38:04.367879 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit NotTrimming 2014-08-13 17:38:04.367856 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] flushed 2014-08-13 17:38:04.367897 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter NotTrimming 2014-08-13 17:38:04.367915 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.367914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_role_change 2014-08-13 17:38:04.367927 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367930 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367929 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:04.367944 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_recovery 2014-08-13 17:38:04.367959 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367962 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367964 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d99f80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.367958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] clear_recovery_state 2014-08-13 17:38:04.367972 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.367977 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.367979 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.367982 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.367964 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 17:38:04.367977 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.367993 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.368004 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368006 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368008 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db2880 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.367999 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.368013 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.368018 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.368020 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368022 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368020 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.368041 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368043 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368045 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107e00 pg_notify(24.1f7(38) epoch 194486) v5 2014-08-13 17:38:04.368038 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.368080 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.351 2014-08-13 17:38:04.368082 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107e00 2014-08-13 17:38:04.368100 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.368116 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.368131 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.368148 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.368153 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368156 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368173 7fe99f5e9700 10 osd.4 pg_epoch: 194485 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 2014-08-13 17:38:04.368198 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368209 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368212 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824b40 pg_notify(22.f3(23) epoch 194486) v5 2014-08-13 17:38:04.368214 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.368220 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Started advmap 2014-08-13 17:38:04.367999 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map 2014-08-13 17:38:04.368238 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319,4] 2014-08-13 17:38:04.368257 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.369 2014-08-13 17:38:04.368260 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824b40 2014-08-13 17:38:04.368255 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.368274 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.866163 4 0.000170 2014-08-13 17:38:04.368296 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368300 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368295 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started 0.866232 0 0.000000 2014-08-13 17:38:04.368316 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Reset 2014-08-13 17:38:04.368333 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368335 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368337 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54de9c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.368344 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.368332 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194485 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194486 2014-08-13 17:38:04.368349 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.368352 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368354 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368340 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] update_heartbeat_peers unchanged 2014-08-13 17:38:04.368352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194486 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:04.368372 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368375 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368376 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d93180 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.368365 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] take_waiters 2014-08-13 17:38:04.368382 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.368370 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194486 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.368387 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.368390 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368392 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368385 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.000843 1 0.000591 2014-08-13 17:38:04.368389 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194486 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319,4] 2014-08-13 17:38:04.368411 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368414 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368416 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827e00 pg_info(1 pgs e194486:22.3e4) v4 2014-08-13 17:38:04.368390 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] flushed 2014-08-13 17:38:04.368408 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started 2014-08-13 17:38:04.368408 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194486 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.368437 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start 2014-08-13 17:38:04.368457 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3e4) v4 from osd.318 2014-08-13 17:38:04.368461 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827e00 2014-08-13 17:38:04.368450 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4]/[45,319] r=-1 lpr=194486 pi=193186-194484/30 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194486 2014-08-13 17:38:04.368454 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray 2014-08-13 17:38:04.368474 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.368498 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368501 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368486 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [45,319,4](45) acting [45,319](45)): not rw, up_thru 194483 up_from 192974 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [45,319,4](45) acting [45,319](45)) : primary up 192974-194483 does not include interval 2014-08-13 17:38:04.368496 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray 2014-08-13 17:38:04.368521 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368524 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368509 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194485/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194485-194485 up [45,319,4](45) acting [45,319](45)) 2014-08-13 17:38:04.368526 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c17e00 pg_notify(24.34(39) epoch 194486) v5 2014-08-13 17:38:04.368518 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.368529 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319,4] -> [45,319,4], acting [45,319] -> [45,319,4], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> 2 2014-08-13 17:38:04.368541 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.368565 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.26 2014-08-13 17:38:04.368551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:04.368568 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c17e00 2014-08-13 17:38:04.368571 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.368587 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.368613 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368616 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368605 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:04.368620 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:04.368642 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368644 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368646 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4263d40 pg_notify(24.30b(35) epoch 194486) v5 2014-08-13 17:38:04.368635 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:04.368649 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.368651 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:04.368668 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:04.368684 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.10 2014-08-13 17:38:04.368687 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4263d40 2014-08-13 17:38:04.368685 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:04.368703 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 17:38:04.368696 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 2014-08-13 17:38:04.368724 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368727 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368718 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.368730 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.368733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:04.368737 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Started advmap 2014-08-13 17:38:04.368759 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368761 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368748 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:04.368763 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5970780 pg_query(24.33a epoch 194486) v3 2014-08-13 17:38:04.368755 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336,4] 2014-08-13 17:38:04.368767 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.368775 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.368803 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.304 epoch 194486 2014-08-13 17:38:04.368788 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:04.368806 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5970780 2014-08-13 17:38:04.368795 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.863709 4 0.000174 2014-08-13 17:38:04.368817 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started 0.863775 0 0.000000 2014-08-13 17:38:04.368818 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193250) from replica 304 2014-08-13 17:38:04.368837 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Reset 2014-08-13 17:38:04.368861 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368864 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368854 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194486 2014-08-13 17:38:04.368883 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368885 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368872 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 17:38:04.368887 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104d00 pg_notify(24.34(52) epoch 194486) v5 2014-08-13 17:38:04.368878 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:04.368889 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.368899 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:04.368908 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336,4] 2014-08-13 17:38:04.368927 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.340 2014-08-13 17:38:04.368931 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104d00 2014-08-13 17:38:04.368917 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000600 1 0.000568 2014-08-13 17:38:04.368927 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.368939 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 17:38:04.368947 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4]/[285,336] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194486 2014-08-13 17:38:04.368957 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 17:38:04.368972 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368974 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.368974 7fe99f5e9700 1 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:04.368982 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [285,336,4](285) acting [285,336](285)): not rw, up_thru 194483 up_from 192975 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [285,336,4](285) acting [285,336](285)) : primary up 192975-194483 does not include interval 2014-08-13 17:38:04.369008 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.368994 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.369011 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369013 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823b80 pg_info(1 pgs e194486:22.84) v4 2014-08-13 17:38:04.369001 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194485/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194485-194485 up [285,336,4](285) acting [285,336](285)) 2014-08-13 17:38:04.369014 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:04.369020 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336,4] -> [285,336,4], acting [285,336] -> [285,336,4], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> 2 2014-08-13 17:38:04.369036 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.369052 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.84) v4 from osd.28 2014-08-13 17:38:04.369041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:04.369057 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823b80 2014-08-13 17:38:04.369039 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] flushed 2014-08-13 17:38:04.369056 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.369061 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.369079 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.369095 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369098 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369097 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 17:38:04.369122 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369124 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369113 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 17:38:04.369126 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299f9e0 pg_query(24.350,24.3b7 epoch 194486) v3 2014-08-13 17:38:04.369148 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369128 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 17:38:04.369167 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.356 epoch 194486 2014-08-13 17:38:04.369170 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299f9e0 2014-08-13 17:38:04.369155 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 17:38:04.369184 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 17:38:04.369177 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369181 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193151) from replica 356 2014-08-13 17:38:04.369202 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 17:38:04.369215 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369235 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369221 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194422'18845 ec=192688 les/c 194484/194484 194485/194486/192969) from replica 356 2014-08-13 17:38:04.369219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 17:38:04.369243 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.369243 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369267 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369271 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369261 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 17:38:04.369267 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369286 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369277 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 17:38:04.369290 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369299 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369302 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140380 pg_notify(24.181(33) epoch 194486) v5 2014-08-13 17:38:04.369293 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369296 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.369315 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369332 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369321 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:04.369347 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.344 2014-08-13 17:38:04.369350 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140380 2014-08-13 17:38:04.369339 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369360 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 17:38:04.369377 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369386 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369389 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369384 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369413 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369415 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369405 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369417 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58224c0 pg_info(1 pgs e194486:22.22c) v4 2014-08-13 17:38:04.369422 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369408 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 17:38:04.369429 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:04.369429 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369459 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.22c) v4 from osd.301 2014-08-13 17:38:04.369463 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58224c0 2014-08-13 17:38:04.369448 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000610 1 0.000584 2014-08-13 17:38:04.369453 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369469 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369470 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 17:38:04.369476 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369487 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 17:38:04.369501 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369504 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369503 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 17:38:04.369505 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.369523 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369523 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.369527 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.369529 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c163c0 pg_notify(24.9b(34) epoch 194486) v5 2014-08-13 17:38:04.369523 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000036 0 0.000000 2014-08-13 17:38:04.369530 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.369545 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 17:38:04.378536 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.291 2014-08-13 17:38:04.378539 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c163c0 2014-08-13 17:38:04.369550 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 17:38:04.378542 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.378578 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378581 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378581 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.378590 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.369559 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 17:38:04.378626 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378629 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378631 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104b40 pg_notify(24.34(52) epoch 194486) v5 2014-08-13 17:38:04.378611 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.378648 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 17:38:04.378668 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.30 2014-08-13 17:38:04.378671 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104b40 2014-08-13 17:38:04.378673 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378694 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.378681 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 50 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.378707 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378710 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378709 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.378741 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378743 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378745 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823f00 pg_notify(24.136 epoch 194486) v5 2014-08-13 17:38:04.378748 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65 unchanged 2014-08-13 17:38:04.378745 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 17:38:04.378773 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378775 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.378782 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.61 2014-08-13 17:38:04.378786 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823f00 2014-08-13 17:38:04.378782 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.378782 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 17 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.378804 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.378821 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378824 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378808 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.378822 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.378842 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378844 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378846 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cb480 pg_info(1 pgs e194486:22.113) v4 2014-08-13 17:38:04.378835 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 unchanged 2014-08-13 17:38:04.378856 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378864 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 337 notify: (query_epoch:194486, epoch_sent:194486, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.378881 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.113) v4 from osd.45 2014-08-13 17:38:04.378885 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cb480 2014-08-13 17:38:04.378887 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 17:38:04.378905 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.378922 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378909 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.378924 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378912 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 44 notify: (query_epoch:194486, epoch_sent:194486, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.378926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.378944 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.378945 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.378946 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.378933 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 44, already have info from that osd, ignoring 2014-08-13 17:38:04.378949 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5971a40 pg_query(24.5e epoch 194486) v3 2014-08-13 17:38:04.378951 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378959 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 15 notify: (query_epoch:194486, epoch_sent:194486, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.378985 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.289 epoch 194486 2014-08-13 17:38:04.378988 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5971a40 2014-08-13 17:38:04.378970 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 17:38:04.378981 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 15, already have info from that osd, ignoring 2014-08-13 17:38:04.378998 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.378997 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.379004 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.379000 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192968) from replica 289 2014-08-13 17:38:04.379005 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) 2014-08-13 17:38:04.379023 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.379044 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379047 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379032 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: got info from osd.358 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) 2014-08-13 17:38:04.379042 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.379068 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379071 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379058 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.925471 7 0.000279 2014-08-13 17:38:04.379061 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.379073 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d080 pg_info(1 pgs e194486:22.3e4) v4 2014-08-13 17:38:04.379083 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.379082 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.379116 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3e4) v4 from osd.313 2014-08-13 17:38:04.379104 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.379121 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d080 2014-08-13 17:38:04.379113 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.379124 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2682 2014-08-13 17:38:04.379136 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.379158 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379161 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.379155 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'526 -> 194215'526 2014-08-13 17:38:04.379178 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379181 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379183 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58263c0 pg_notify(24.136 epoch 194486) v5 2014-08-13 17:38:04.379173 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.837993 3 0.000327 2014-08-13 17:38:04.379181 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.379199 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.379216 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.865632 0 0.000000 2014-08-13 17:38:04.379232 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.55 2014-08-13 17:38:04.379235 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58263c0 2014-08-13 17:38:04.379248 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.379276 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379264 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194478 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.379279 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379283 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.379298 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379285 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.379300 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379302 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51024c0 pg_notify(24.136(2) epoch 194486) v5 2014-08-13 17:38:04.379301 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.379317 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'539 -> 194215'539 2014-08-13 17:38:04.379314 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.379342 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.50 2014-08-13 17:38:04.379346 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51024c0 2014-08-13 17:38:04.379342 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.379342 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.379366 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.379340 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 17:38:04.379383 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379386 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379374 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194486 n=44 ec=192688 les/c 194483/194463 194485/194485/194468) 2014-08-13 17:38:04.379404 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379406 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379408 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299eae0 pg_query(24.2a1 epoch 194486) v3 2014-08-13 17:38:04.379396 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.19 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194486 n=44 ec=192688 les/c 194483/194463 194485/194485/194468) 2014-08-13 17:38:04.379421 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.915908 7 0.000328 2014-08-13 17:38:04.379443 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.17 epoch 194486 2014-08-13 17:38:04.379446 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299eae0 2014-08-13 17:38:04.379442 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.379457 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193188) from replica 17 2014-08-13 17:38:04.379473 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.379497 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379500 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379492 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.379518 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379520 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379522 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51055c0 pg_notify(24.3d epoch 194486) v5 2014-08-13 17:38:04.379511 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'13811 -> 194422'13811 2014-08-13 17:38:04.379564 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.362 2014-08-13 17:38:04.379567 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51055c0 2014-08-13 17:38:04.379580 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] state: Activate Finished 2014-08-13 17:38:04.379598 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379601 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379619 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379622 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379624 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513db00 pg_notify(22.3d5(25) epoch 194486) v5 2014-08-13 17:38:04.379620 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 sending log((0'0,194215'539], crt=191727'457) 2014-08-13 17:38:04.379644 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.379648 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d348e0 don't know epoch, doing nothing 2014-08-13 17:38:04.379661 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.286 2014-08-13 17:38:04.379663 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513db00 2014-08-13 17:38:04.379697 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379699 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379688 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 22.3af( DNE v 194215'539 lc 0'0 (0'0,194215'539] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(24) 2014-08-13 17:38:04.379718 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.379712 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.379728 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:04.379731 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379732 7fe99f5e9700 7 osd.4 194486 do_notify osd 45 on 1 PGs 2014-08-13 17:38:04.379733 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379739 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5971e00 pg_query(24.e2 epoch 194486) v3 2014-08-13 17:38:04.379706 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.379773 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.346 epoch 194486 2014-08-13 17:38:04.379775 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5971e00 2014-08-13 17:38:04.379786 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194422'15014 ec=192688 les/c 194484/194484 194485/194486/192955) from replica 346 2014-08-13 17:38:04.379827 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379830 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379848 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379850 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379851 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f02300 pg_info(1 pgs e194486:22.22c) v4 2014-08-13 17:38:04.379858 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 don't know epoch, doing nothing 2014-08-13 17:38:04.379891 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.22c) v4 from osd.324 2014-08-13 17:38:04.379895 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f02300 2014-08-13 17:38:04.379931 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379913 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] flushed 2014-08-13 17:38:04.379936 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379955 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.379957 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.379959 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822f40 pg_notify(22.481(39) epoch 194486) v5 2014-08-13 17:38:04.379967 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 sending log((0'0,194215'539], crt=191727'457) 2014-08-13 17:38:04.379991 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:04.379996 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.303 2014-08-13 17:38:04.379999 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822f40 2014-08-13 17:38:04.380038 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380021 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 22.3af( DNE v 194215'539 lc 0'0 (0'0,194215'539] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(24) 2014-08-13 17:38:04.380041 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380082 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820fc0 pg_notify(24.1f7(34) epoch 194486) v5 2014-08-13 17:38:04.380119 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.376 2014-08-13 17:38:04.380106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.44 has 24 missing 2014-08-13 17:38:04.380122 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820fc0 2014-08-13 17:38:04.380124 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.362 has 24 missing 2014-08-13 17:38:04.380139 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 is on osd.4 2014-08-13 17:38:04.380158 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380162 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380159 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.380209 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380196 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 is on osd.4 2014-08-13 17:38:04.380212 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380214 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_notify(24.b6(2) epoch 194486) v5 2014-08-13 17:38:04.380214 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 is on osd.4 2014-08-13 17:38:04.380232 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 is on osd.4 2014-08-13 17:38:04.380260 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.327 2014-08-13 17:38:04.380249 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.380263 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.380266 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 is on osd.4 2014-08-13 17:38:04.380283 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 is on osd.4 2014-08-13 17:38:04.380299 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380301 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380300 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 is on osd.4 2014-08-13 17:38:04.380321 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380324 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380326 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58239c0 pg_notify(24.136(31) epoch 194486) v5 2014-08-13 17:38:04.380320 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 is on osd.4 2014-08-13 17:38:04.380338 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 is on osd.4 2014-08-13 17:38:04.380362 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.65 2014-08-13 17:38:04.380365 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58239c0 2014-08-13 17:38:04.380360 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.380377 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 is on osd.4 2014-08-13 17:38:04.380400 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380403 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380394 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 is on osd.4 2014-08-13 17:38:04.380411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 is on osd.4 2014-08-13 17:38:04.380425 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380428 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380430 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5138000 pg_info(1 pgs e194486:22.39d) v4 2014-08-13 17:38:04.380428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 is on osd.4 2014-08-13 17:38:04.380445 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.380466 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.39d) v4 from osd.373 2014-08-13 17:38:04.380470 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5138000 2014-08-13 17:38:04.380462 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 is on osd.4 2014-08-13 17:38:04.380477 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:04.380489 7fe99f5e9700 7 osd.4 194486 do_notify osd 328 on 1 PGs 2014-08-13 17:38:04.380479 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 is on osd.4 2014-08-13 17:38:04.380504 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380507 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380497 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.380518 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:04.380521 7fe99f5e9700 7 osd.4 194486 do_notify osd 374 on 1 PGs 2014-08-13 17:38:04.380527 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380514 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 is on osd.4 2014-08-13 17:38:04.380529 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380532 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51039c0 pg_notify(24.181(12) epoch 194486) v5 2014-08-13 17:38:04.380531 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 is on osd.4 2014-08-13 17:38:04.380547 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.380548 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 is on osd.4 2014-08-13 17:38:04.380569 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.320 2014-08-13 17:38:04.380576 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51039c0 2014-08-13 17:38:04.380566 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 is on osd.4 2014-08-13 17:38:04.380584 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 17:38:04.380623 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 24 notify: (query_epoch:194486, epoch_sent:194486, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.380631 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate: adding 44 as a source 2014-08-13 17:38:04.380652 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 24, already have info from that osd, ignoring 2014-08-13 17:38:04.380654 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 also missing on osd.44 2014-08-13 17:38:04.380675 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.380673 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 also missing on osd.44 2014-08-13 17:38:04.380683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 363 notify: (query_epoch:194486, epoch_sent:194486, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.380691 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 also missing on osd.44 2014-08-13 17:38:04.380705 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 17:38:04.380722 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.380710 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 also missing on osd.44 2014-08-13 17:38:04.380728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 also missing on osd.44 2014-08-13 17:38:04.380729 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 346 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.380747 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 also missing on osd.44 2014-08-13 17:38:04.380752 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info 2014-08-13 17:38:04.380764 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 also missing on osd.44 2014-08-13 17:38:04.380778 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380782 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380782 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 also missing on osd.44 2014-08-13 17:38:04.380806 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380808 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380810 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58271c0 pg_notify(24.b6 epoch 194486) v5 2014-08-13 17:38:04.380799 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 also missing on osd.44 2014-08-13 17:38:04.380817 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 also missing on osd.44 2014-08-13 17:38:04.380843 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.380848 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.330 2014-08-13 17:38:04.380834 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 also missing on osd.44 2014-08-13 17:38:04.380851 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58271c0 2014-08-13 17:38:04.380852 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 also missing on osd.44 2014-08-13 17:38:04.380852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.380873 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 also missing on osd.44 2014-08-13 17:38:04.380892 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380879 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.380861 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] _activate_committed 194486 telling primary 2014-08-13 17:38:04.380895 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380899 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.380892 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 also missing on osd.44 2014-08-13 17:38:04.380911 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 don't know epoch, doing nothing 2014-08-13 17:38:04.380905 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.380922 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.380910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 also missing on osd.44 2014-08-13 17:38:04.380925 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.380930 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827000 pg_notify(24.b6(38) epoch 194486) v5 2014-08-13 17:38:04.380925 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 17:38:04.380931 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 also missing on osd.44 2014-08-13 17:38:04.380947 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.380950 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 also missing on osd.44 2014-08-13 17:38:04.380946 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed 2014-08-13 17:38:04.380955 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) 2014-08-13 17:38:04.380975 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.315 2014-08-13 17:38:04.380977 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827000 2014-08-13 17:38:04.380970 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 also missing on osd.44 2014-08-13 17:38:04.380977 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: got info from osd.334 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) 2014-08-13 17:38:04.380992 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 also missing on osd.44 2014-08-13 17:38:04.381001 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.953292 7 0.000352 2014-08-13 17:38:04.381019 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381022 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381011 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 also missing on osd.44 2014-08-13 17:38:04.381023 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.381031 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 also missing on osd.44 2014-08-13 17:38:04.381052 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381055 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381057 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102300 pg_notify(24.3d epoch 194486) v5 2014-08-13 17:38:04.381045 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.381048 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 also missing on osd.44 2014-08-13 17:38:04.381064 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.381067 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 also missing on osd.44 2014-08-13 17:38:04.381095 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.25 2014-08-13 17:38:04.381082 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'528 -> 194215'528 2014-08-13 17:38:04.381098 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102300 2014-08-13 17:38:04.381086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 also missing on osd.44 2014-08-13 17:38:04.381108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.381132 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381136 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381105 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 17:38:04.381153 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381156 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381159 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104980 pg_info(1 pgs e194486:22.707) v4 2014-08-13 17:38:04.381153 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate: adding 362 as a source 2014-08-13 17:38:04.381180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 also missing on osd.362 2014-08-13 17:38:04.381194 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.707) v4 from osd.310 2014-08-13 17:38:04.381198 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104980 2014-08-13 17:38:04.381198 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 also missing on osd.362 2014-08-13 17:38:04.381215 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 also missing on osd.362 2014-08-13 17:38:04.381233 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 also missing on osd.362 2014-08-13 17:38:04.381246 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381244 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381249 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381250 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 also missing on osd.362 2014-08-13 17:38:04.381268 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381271 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381273 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827a80 pg_info(1 pgs e194486:22.4df) v4 2014-08-13 17:38:04.381267 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 also missing on osd.362 2014-08-13 17:38:04.381277 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 324 notify: (query_epoch:194486, epoch_sent:194486, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.381286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 also missing on osd.362 2014-08-13 17:38:04.381311 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4df) v4 from osd.357 2014-08-13 17:38:04.381315 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827a80 2014-08-13 17:38:04.381304 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 17:38:04.381328 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381307 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 also missing on osd.362 2014-08-13 17:38:04.381334 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 also missing on osd.362 2014-08-13 17:38:04.381351 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381335 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.381354 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381353 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 also missing on osd.362 2014-08-13 17:38:04.381357 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:04.381373 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381374 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381376 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381378 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102680 pg_info(1 pgs e194486:22.775) v4 2014-08-13 17:38:04.381372 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 also missing on osd.362 2014-08-13 17:38:04.381382 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) 2014-08-13 17:38:04.381393 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 also missing on osd.362 2014-08-13 17:38:04.381424 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.775) v4 from osd.353 2014-08-13 17:38:04.381407 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.367 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) 2014-08-13 17:38:04.381411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 also missing on osd.362 2014-08-13 17:38:04.381428 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102680 2014-08-13 17:38:04.381433 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 also missing on osd.362 2014-08-13 17:38:04.381432 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.938906 7 0.000260 2014-08-13 17:38:04.381467 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381454 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 also missing on osd.362 2014-08-13 17:38:04.381470 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381456 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.381474 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 also missing on osd.362 2014-08-13 17:38:04.381492 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381481 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.381496 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381499 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9c140 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.381492 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 also missing on osd.362 2014-08-13 17:38:04.381507 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.381512 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.381500 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.381515 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381518 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381512 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 also missing on osd.362 2014-08-13 17:38:04.381518 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'492 -> 194215'492 2014-08-13 17:38:04.381539 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381541 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381543 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5143800 pg_notify(24.30b(35) epoch 194486) v5 2014-08-13 17:38:04.381533 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 also missing on osd.362 2014-08-13 17:38:04.381542 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.381553 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 also missing on osd.362 2014-08-13 17:38:04.381583 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.10 2014-08-13 17:38:04.381586 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5143800 2014-08-13 17:38:04.381572 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 also missing on osd.362 2014-08-13 17:38:04.381591 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 also missing on osd.362 2014-08-13 17:38:04.381622 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381625 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 also missing on osd.362 2014-08-13 17:38:04.381630 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 also missing on osd.362 2014-08-13 17:38:04.381617 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.381660 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381663 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381666 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823640 pg_info(1 pgs e194486:22.84) v4 2014-08-13 17:38:04.381666 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 don't know epoch, doing nothing 2014-08-13 17:38:04.381656 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 331 notify: (query_epoch:194486, epoch_sent:194486, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.381680 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 331, already have info from that osd, ignoring 2014-08-13 17:38:04.381651 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 17:38:04.381703 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381709 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.84) v4 from osd.304 2014-08-13 17:38:04.381713 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823640 2014-08-13 17:38:04.381700 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.381700 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] flushed 2014-08-13 17:38:04.381712 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 65 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.381721 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.381751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381742 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 44,61,339,362,369 2014-08-13 17:38:04.381739 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.381773 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381761 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.381775 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381778 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821dc0 pg_notify(24.b6 epoch 194486) v5 2014-08-13 17:38:04.381774 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65 unchanged 2014-08-13 17:38:04.381778 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.381797 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381799 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.381818 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.28 2014-08-13 17:38:04.381821 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821dc0 2014-08-13 17:38:04.381804 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 373 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.381817 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.381828 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.381853 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381856 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381873 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381876 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381856 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] osd.373 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.381877 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00c40 pg_info(1 pgs e194486:22.4df) v4 2014-08-13 17:38:04.381880 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362 -> 4,20,25,303,362,373 2014-08-13 17:38:04.381899 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.381906 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381913 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4df) v4 from osd.356 2014-08-13 17:38:04.381917 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00c40 2014-08-13 17:38:04.381918 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.381915 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 342 notify: (query_epoch:194486, epoch_sent:194486, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.381936 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 17:38:04.381952 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.381951 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381957 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381975 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.381978 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.381961 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 330 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.381980 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e740 pg_info(1 pgs e194486:22.39d) v4 2014-08-13 17:38:04.381967 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 17:38:04.381997 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.381983 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info 2014-08-13 17:38:04.382018 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.39d) v4 from osd.344 2014-08-13 17:38:04.382004 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.382022 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e740 2014-08-13 17:38:04.382023 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.382027 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.382041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.382057 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382059 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382060 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:04.382081 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382084 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382061 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.382087 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382091 7fe9a63f4700 20 osd.4 194486 _dispatch 0x2999e00 pg_query(24.369 epoch 194486) v3 2014-08-13 17:38:04.382092 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] flushed 2014-08-13 17:38:04.382093 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.382099 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 312 notify: (query_epoch:194486, epoch_sent:194486, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.382131 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.45 epoch 194486 2014-08-13 17:38:04.382115 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.382134 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x2999e00 2014-08-13 17:38:04.382125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 312, already have info from that osd, ignoring 2014-08-13 17:38:04.382146 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382138 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2681 2014-08-13 17:38:04.382147 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193186) from replica 45 2014-08-13 17:38:04.382154 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 11 notify: (query_epoch:194486, epoch_sent:194486, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.382157 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.382190 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382194 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 11, already have info from that osd, ignoring 2014-08-13 17:38:04.382184 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.840029 3 0.000542 2014-08-13 17:38:04.382204 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382206 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.382221 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382224 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382226 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826200 pg_notify(24.1f7(2) epoch 194486) v5 2014-08-13 17:38:04.382211 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 365 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.382223 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.866759 0 0.000000 2014-08-13 17:38:04.382239 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.382266 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.55 2014-08-13 17:38:04.382269 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826200 2014-08-13 17:38:04.382255 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.382275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194477 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.382266 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.382268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] osd.365 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.382311 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382296 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.382314 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382299 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.382297 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,367,378 -> 4,291,350,365,367,378 2014-08-13 17:38:04.382323 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.382328 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382321 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.382335 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382339 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382342 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f01880 pg_notify(24.b6(38) epoch 194486) v5 2014-08-13 17:38:04.382337 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'519 -> 194215'519 2014-08-13 17:38:04.382335 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 314 notify: (query_epoch:194486, epoch_sent:194486, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.382359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 314, already have info from that osd, ignoring 2014-08-13 17:38:04.382379 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382363 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:04.382385 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.17 2014-08-13 17:38:04.382388 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f01880 2014-08-13 17:38:04.382384 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 is up to date, queueing in pending_activators 2014-08-13 17:38:04.382389 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 315 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.382410 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.382437 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 unchanged 2014-08-13 17:38:04.382457 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382470 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 311 notify: (query_epoch:194486, epoch_sent:194486, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.382496 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382491 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 311, already have info from that osd, ignoring 2014-08-13 17:38:04.382486 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) uptodate 2014-08-13 17:38:04.382506 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382509 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382515 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.382521 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:04.382525 7fe99f5e9700 7 osd.4 194486 do_notify osd 330 on 1 PGs 2014-08-13 17:38:04.382511 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) 2014-08-13 17:38:04.382531 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 is up to date, queueing in pending_activators 2014-08-13 17:38:04.382549 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e4a100 already has epoch 194486 2014-08-13 17:38:04.382559 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382559 7fe99f5e9700 7 osd.4 194486 do_notify osd 346 on 1 PGs 2014-08-13 17:38:04.382562 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382564 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824600 pg_info(1 pgs e194486:22.2d1) v4 2014-08-13 17:38:04.382582 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.382600 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2d1) v4 from osd.358 2014-08-13 17:38:04.382604 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824600 2014-08-13 17:38:04.382613 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485) missing missing(32) 2014-08-13 17:38:04.382653 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 302 notify: (query_epoch:194486, epoch_sent:194486, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.382662 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.309 has 32 missing 2014-08-13 17:38:04.382676 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 302, already have info from that osd, ignoring 2014-08-13 17:38:04.382679 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 is on osd.4 2014-08-13 17:38:04.382697 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382706 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382710 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382698 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 is on osd.4 2014-08-13 17:38:04.382705 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 375 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.382716 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 is on osd.4 2014-08-13 17:38:04.382739 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382742 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382744 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105400 pg_notify(22.362(31) epoch 194486) v5 2014-08-13 17:38:04.382733 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 is on osd.4 2014-08-13 17:38:04.382729 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.382751 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 is on osd.4 2014-08-13 17:38:04.382760 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,375 unchanged 2014-08-13 17:38:04.382783 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382769 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.382784 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.362 2014-08-13 17:38:04.382788 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105400 2014-08-13 17:38:04.382788 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 is on osd.4 2014-08-13 17:38:04.382791 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 371 notify: (query_epoch:194486, epoch_sent:194486, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.382806 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 is on osd.4 2014-08-13 17:38:04.382826 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382830 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382814 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 371, already have info from that osd, ignoring 2014-08-13 17:38:04.382834 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382824 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 is on osd.4 2014-08-13 17:38:04.382840 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 368 notify: (query_epoch:194486, epoch_sent:194486, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.382841 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 is on osd.4 2014-08-13 17:38:04.382866 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382868 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382870 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102840 pg_info(1 pgs e194486:22.69f) v4 2014-08-13 17:38:04.382860 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 17:38:04.382860 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 is on osd.4 2014-08-13 17:38:04.382882 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.382883 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 is on osd.4 2014-08-13 17:38:04.382889 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) 2014-08-13 17:38:04.382911 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.69f) v4 from osd.334 2014-08-13 17:38:04.382916 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102840 2014-08-13 17:38:04.382901 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 is on osd.4 2014-08-13 17:38:04.382912 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: got info from osd.351 22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) 2014-08-13 17:38:04.382921 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 is on osd.4 2014-08-13 17:38:04.382935 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.036240 7 0.000402 2014-08-13 17:38:04.382939 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 is on osd.4 2014-08-13 17:38:04.382956 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382959 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.382957 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.382956 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 is on osd.4 2014-08-13 17:38:04.382976 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.382980 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.382997 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.382999 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383001 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822d80 pg_notify(24.3d(32) epoch 194486) v5 2014-08-13 17:38:04.382993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 is on osd.4 2014-08-13 17:38:04.382999 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194478 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.383011 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 is on osd.4 2014-08-13 17:38:04.383018 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'521 -> 194215'521 2014-08-13 17:38:04.383042 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.303 2014-08-13 17:38:04.383030 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 is on osd.4 2014-08-13 17:38:04.383045 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822d80 2014-08-13 17:38:04.383043 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.383048 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 is on osd.4 2014-08-13 17:38:04.383066 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 is on osd.4 2014-08-13 17:38:04.383088 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383091 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 is on osd.4 2014-08-13 17:38:04.383103 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 is on osd.4 2014-08-13 17:38:04.383120 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383123 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383125 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820e00 pg_notify(24.1f7(34) epoch 194486) v5 2014-08-13 17:38:04.383118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 is on osd.4 2014-08-13 17:38:04.383135 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383135 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 is on osd.4 2014-08-13 17:38:04.383163 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.376 2014-08-13 17:38:04.383166 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820e00 2014-08-13 17:38:04.383153 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.383162 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 63 notify: (query_epoch:194486, epoch_sent:194486, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.383179 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 is on osd.4 2014-08-13 17:38:04.383196 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 63, already have info from that osd, ignoring 2014-08-13 17:38:04.383197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.383212 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383216 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383217 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383217 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 is on osd.4 2014-08-13 17:38:04.383241 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383225 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 349 notify: (query_epoch:194486, epoch_sent:194486, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.383243 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383246 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c163c0 pg_info(1 pgs e194486:22.639) v4 2014-08-13 17:38:04.383235 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 is on osd.4 2014-08-13 17:38:04.383246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 349, already have info from that osd, ignoring 2014-08-13 17:38:04.383268 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383254 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 is on osd.4 2014-08-13 17:38:04.383290 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.639) v4 from osd.367 2014-08-13 17:38:04.383294 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c163c0 2014-08-13 17:38:04.383275 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 350 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.383303 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.383331 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383274 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 17:38:04.383334 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383335 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate: adding 287 as a source 2014-08-13 17:38:04.383334 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:04.383354 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383357 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383358 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383360 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d5c0 pg_notify(24.3d(32) epoch 194486) v5 2014-08-13 17:38:04.383352 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 is on osd.287 2014-08-13 17:38:04.383365 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 352 notify: (query_epoch:194486, epoch_sent:194486, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.383372 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 is on osd.287 2014-08-13 17:38:04.383389 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 352, already have info from that osd, ignoring 2014-08-13 17:38:04.383404 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.373 2014-08-13 17:38:04.383391 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 is on osd.287 2014-08-13 17:38:04.383407 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383406 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d5c0 2014-08-13 17:38:04.383410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 is on osd.287 2014-08-13 17:38:04.383414 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 366 notify: (query_epoch:194486, epoch_sent:194486, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.383429 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 is on osd.287 2014-08-13 17:38:04.383443 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.383446 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.383434 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 366, already have info from that osd, ignoring 2014-08-13 17:38:04.383453 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.383447 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 is on osd.287 2014-08-13 17:38:04.392588 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392594 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392596 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824d00 pg_notify(24.1f7 epoch 194486) v5 2014-08-13 17:38:04.392581 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.392585 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 is on osd.287 2014-08-13 17:38:04.383591 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.392642 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.59 2014-08-13 17:38:04.392627 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.392645 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824d00 2014-08-13 17:38:04.392627 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 is on osd.287 2014-08-13 17:38:04.392651 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.392646 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x5250b00 don't know epoch, doing nothing 2014-08-13 17:38:04.392651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 is on osd.287 2014-08-13 17:38:04.392682 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392671 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 is on osd.287 2014-08-13 17:38:04.392685 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392691 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 is on osd.287 2014-08-13 17:38:04.392711 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392713 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392716 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140540 pg_notify(24.136(31) epoch 194486) v5 2014-08-13 17:38:04.392699 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] flushed 2014-08-13 17:38:04.392710 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 is on osd.287 2014-08-13 17:38:04.392730 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 is on osd.287 2014-08-13 17:38:04.392754 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.356 2014-08-13 17:38:04.392757 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140540 2014-08-13 17:38:04.392748 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 is on osd.287 2014-08-13 17:38:04.392767 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 is on osd.287 2014-08-13 17:38:04.392755 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 26 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.392785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 is on osd.287 2014-08-13 17:38:04.392801 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392805 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392802 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 is on osd.287 2014-08-13 17:38:04.392800 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194472/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.392822 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 is on osd.287 2014-08-13 17:38:04.392848 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392850 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392852 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299d280 pg_query(24.2ec epoch 194486) v3 2014-08-13 17:38:04.392839 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 is on osd.287 2014-08-13 17:38:04.392839 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 17:38:04.392860 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 is on osd.287 2014-08-13 17:38:04.392861 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 17:38:04.392890 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.374 epoch 194486 2014-08-13 17:38:04.392893 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299d280 2014-08-13 17:38:04.392881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 is on osd.287 2014-08-13 17:38:04.392883 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.392899 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 is on osd.287 2014-08-13 17:38:04.392906 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194477-194478 up [26,30,340](26) acting [4](4)) 2014-08-13 17:38:04.392905 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_query query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/193232) from replica 374 2014-08-13 17:38:04.392919 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 is on osd.287 2014-08-13 17:38:04.392927 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194473-194476 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.392953 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392940 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 is on osd.287 2014-08-13 17:38:04.392956 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392947 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior final: probe 4,26,30,340 down blocked_by {} 2014-08-13 17:38:04.392959 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 is on osd.287 2014-08-13 17:38:04.392976 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.392979 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392982 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d90b40 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.392968 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.392979 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 is on osd.287 2014-08-13 17:38:04.392993 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.392999 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.393001 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393003 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.392999 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 is on osd.287 2014-08-13 17:38:04.392994 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: have osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.393021 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393024 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393026 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00a80 pg_info(1 pgs e194486:22.2d1) v4 2014-08-13 17:38:04.393019 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 is on osd.287 2014-08-13 17:38:04.393019 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: already requested info from osd.30 2014-08-13 17:38:04.393040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: already requested info from osd.340 2014-08-13 17:38:04.393040 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 is on osd.287 2014-08-13 17:38:04.393071 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2d1) v4 from osd.358 2014-08-13 17:38:04.393075 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00a80 2014-08-13 17:38:04.393064 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 is on osd.287 2014-08-13 17:38:04.393084 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 is on osd.287 2014-08-13 17:38:04.393105 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393110 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393113 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393102 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 is on osd.287 2014-08-13 17:38:04.393137 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393139 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393141 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823100 pg_info(1 pgs e194486:22.3af) v4 2014-08-13 17:38:04.393134 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 58 notify: (query_epoch:194486, epoch_sent:194486, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.393160 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 58, already have info from that osd, ignoring 2014-08-13 17:38:04.393179 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393177 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3af) v4 from osd.362 2014-08-13 17:38:04.393122 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 17:38:04.393185 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823100 2014-08-13 17:38:04.393188 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate: adding 309 as a source 2014-08-13 17:38:04.393203 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 304 notify: (query_epoch:194486, epoch_sent:194486, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.393209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 also missing on osd.309 2014-08-13 17:38:04.393229 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393233 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393225 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 304, already have info from that osd, ignoring 2014-08-13 17:38:04.393229 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 also missing on osd.309 2014-08-13 17:38:04.393247 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393251 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393253 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393255 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f01180 pg_info(1 pgs e194486:22.69f) v4 2014-08-13 17:38:04.393248 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 also missing on osd.309 2014-08-13 17:38:04.393256 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.393268 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 also missing on osd.309 2014-08-13 17:38:04.393279 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.393294 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.69f) v4 from osd.334 2014-08-13 17:38:04.393298 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f01180 2014-08-13 17:38:04.393299 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393288 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 also missing on osd.309 2014-08-13 17:38:04.393308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 also missing on osd.309 2014-08-13 17:38:04.393307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) 2014-08-13 17:38:04.393332 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393335 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393327 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 also missing on osd.309 2014-08-13 17:38:04.393329 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.9 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) 2014-08-13 17:38:04.393353 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393355 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393358 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f008c0 pg_info(1 pgs e194486:22.639) v4 2014-08-13 17:38:04.393346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 also missing on osd.309 2014-08-13 17:38:04.393354 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.033032 7 0.000440 2014-08-13 17:38:04.393365 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 also missing on osd.309 2014-08-13 17:38:04.393378 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.393396 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.639) v4 from osd.367 2014-08-13 17:38:04.393385 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 also missing on osd.309 2014-08-13 17:38:04.393400 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f008c0 2014-08-13 17:38:04.393401 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.393405 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 also missing on osd.309 2014-08-13 17:38:04.393421 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194477 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.393437 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393439 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393439 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'495 -> 194215'495 2014-08-13 17:38:04.393459 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393461 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393463 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db7980 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.393424 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 also missing on osd.309 2014-08-13 17:38:04.393471 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.393475 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.393462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.393479 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393483 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393474 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 also missing on osd.309 2014-08-13 17:38:04.393500 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393502 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393504 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51071c0 pg_info(1 pgs e194486:2.2a) v4 2014-08-13 17:38:04.393495 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 also missing on osd.309 2014-08-13 17:38:04.393513 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 also missing on osd.309 2014-08-13 17:38:04.393541 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:2.2a) v4 from osd.340 2014-08-13 17:38:04.393545 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51071c0 2014-08-13 17:38:04.393532 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 also missing on osd.309 2014-08-13 17:38:04.393553 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393551 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 also missing on osd.309 2014-08-13 17:38:04.393569 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 also missing on osd.309 2014-08-13 17:38:04.393584 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393587 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393587 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 also missing on osd.309 2014-08-13 17:38:04.393588 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.393608 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393611 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393613 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.775) v4 2014-08-13 17:38:04.393606 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 also missing on osd.309 2014-08-13 17:38:04.393614 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.393637 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393625 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 also missing on osd.309 2014-08-13 17:38:04.393657 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.775) v4 from osd.56 2014-08-13 17:38:04.393645 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 also missing on osd.309 2014-08-13 17:38:04.393662 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.393644 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 28 notify: (query_epoch:194486, epoch_sent:194486, info:22.84( v 194215'497 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.393668 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 also missing on osd.309 2014-08-13 17:38:04.393669 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 28, already have info from that osd, ignoring 2014-08-13 17:38:04.393689 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393687 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 also missing on osd.309 2014-08-13 17:38:04.393703 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393706 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393695 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 356 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.393706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 also missing on osd.309 2014-08-13 17:38:04.393725 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393728 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393730 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513db00 pg_info(1 pgs e194486:22.1f1) v4 2014-08-13 17:38:04.393725 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 also missing on osd.309 2014-08-13 17:38:04.393719 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.393746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 also missing on osd.309 2014-08-13 17:38:04.393767 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1f1) v4 from osd.47 2014-08-13 17:38:04.393752 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] osd.356 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.393772 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513db00 2014-08-13 17:38:04.393765 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 also missing on osd.309 2014-08-13 17:38:04.393785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 also missing on osd.309 2014-08-13 17:38:04.393783 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65 -> 4,50,55,61,65,356 2014-08-13 17:38:04.393806 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.393811 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.393816 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.393804 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 also missing on osd.309 2014-08-13 17:38:04.393819 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.393818 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) 2014-08-13 17:38:04.393823 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 also missing on osd.309 2014-08-13 17:38:04.393839 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: got info from osd.328 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) 2014-08-13 17:38:04.393841 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 also missing on osd.309 2014-08-13 17:38:04.393861 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.898928 7 0.000253 2014-08-13 17:38:04.393881 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.393900 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.393866 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 17:38:04.393917 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194477 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.393925 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.393934 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'512 -> 194215'512 2014-08-13 17:38:04.393942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 17:38:04.393959 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.393970 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 287,303,306,309,365,381 2014-08-13 17:38:04.393989 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.394010 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.394036 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394027 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.394043 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.394061 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.394069 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.394120 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.394144 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 301 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.394167 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x49e2aa0 don't know epoch, doing nothing 2014-08-13 17:38:04.394170 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.394171 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.394209 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] osd.301 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.394221 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] flushed 2014-08-13 17:38:04.394233 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,375 -> 4,10,286,301,375 2014-08-13 17:38:04.394252 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.394257 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394265 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) 2014-08-13 17:38:04.394285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: got info from osd.54 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) 2014-08-13 17:38:04.394306 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.997045 7 0.000369 2014-08-13 17:38:04.394324 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.394344 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.394360 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194477 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.394376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'473 -> 194215'473 2014-08-13 17:38:04.394396 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.394392 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 17:38:04.394430 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.394407 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.394438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.394459 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:04.394458 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.394481 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394476 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.394494 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.394504 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] flushed 2014-08-13 17:38:04.394513 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.394534 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 30 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.394536 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.394564 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2687 2014-08-13 17:38:04.394560 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.394581 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.394592 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 17:38:04.394613 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394600 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.852986 3 0.000335 2014-08-13 17:38:04.394618 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.394621 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.394638 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 17:38:04.394638 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.874663 0 0.000000 2014-08-13 17:38:04.394657 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394663 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.394665 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 20 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.394680 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194478 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.394687 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.394703 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.394715 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:04.394719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.394735 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394734 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'543 -> 194215'543 2014-08-13 17:38:04.394743 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 20 notify: (query_epoch:194486, epoch_sent:194486, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.394760 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) 2014-08-13 17:38:04.394764 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 17:38:04.394787 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.394780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 is up to date, queueing in pending_activators 2014-08-13 17:38:04.394794 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) 2014-08-13 17:38:04.394814 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: got info from osd.321 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) 2014-08-13 17:38:04.394835 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.954697 7 0.000407 2014-08-13 17:38:04.394854 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.394850 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) uptodate 2014-08-13 17:38:04.394873 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.394871 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.394891 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.394906 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'473 -> 194215'473 2014-08-13 17:38:04.394928 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.394912 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.394947 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.395008 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.395036 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 374 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.395059 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info 2014-08-13 17:38:04.395119 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 sending log((0'0,194215'543], crt=191727'473) 2014-08-13 17:38:04.395119 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.395145 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df32e0 don't know epoch, doing nothing 2014-08-13 17:38:04.395154 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395157 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395159 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140380 pg_info(1 pgs e194486:22.28f) v4 2014-08-13 17:38:04.395168 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.395198 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.28f) v4 from osd.9 2014-08-13 17:38:04.395211 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140380 2014-08-13 17:38:04.395194 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 22.48e( DNE v 194215'543 lc 0'0 (0'0,194215'543] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(40) 2014-08-13 17:38:04.395253 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395256 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395249 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.341 has 40 missing 2014-08-13 17:38:04.395266 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 is on osd.4 2014-08-13 17:38:04.395283 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 is on osd.4 2014-08-13 17:38:04.395299 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 is on osd.4 2014-08-13 17:38:04.395315 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 is on osd.4 2014-08-13 17:38:04.395331 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 is on osd.4 2014-08-13 17:38:04.395346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 is on osd.4 2014-08-13 17:38:04.395366 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 is on osd.4 2014-08-13 17:38:04.395381 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 is on osd.4 2014-08-13 17:38:04.395397 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 is on osd.4 2014-08-13 17:38:04.395412 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 is on osd.4 2014-08-13 17:38:04.395428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 is on osd.4 2014-08-13 17:38:04.395443 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 is on osd.4 2014-08-13 17:38:04.395459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 is on osd.4 2014-08-13 17:38:04.395474 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 is on osd.4 2014-08-13 17:38:04.395489 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 is on osd.4 2014-08-13 17:38:04.395504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 is on osd.4 2014-08-13 17:38:04.395520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 is on osd.4 2014-08-13 17:38:04.395536 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 is on osd.4 2014-08-13 17:38:04.395551 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 is on osd.4 2014-08-13 17:38:04.395567 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 is on osd.4 2014-08-13 17:38:04.395581 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395585 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395587 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107a80 pg_info(1 pgs e194486:22.28f) v4 2014-08-13 17:38:04.395583 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 is on osd.4 2014-08-13 17:38:04.395600 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 is on osd.4 2014-08-13 17:38:04.395624 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.28f) v4 from osd.9 2014-08-13 17:38:04.395628 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107a80 2014-08-13 17:38:04.395617 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 is on osd.4 2014-08-13 17:38:04.395634 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 is on osd.4 2014-08-13 17:38:04.395650 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 is on osd.4 2014-08-13 17:38:04.395664 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395667 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395673 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a4f380 don't know epoch, doing nothing 2014-08-13 17:38:04.395666 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 is on osd.4 2014-08-13 17:38:04.395686 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395689 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395690 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823b80 pg_info(1 pgs e194486:22.1a0) v4 2014-08-13 17:38:04.395684 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 is on osd.4 2014-08-13 17:38:04.395701 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 is on osd.4 2014-08-13 17:38:04.395726 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1a0) v4 from osd.328 2014-08-13 17:38:04.395730 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823b80 2014-08-13 17:38:04.395718 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 is on osd.4 2014-08-13 17:38:04.395718 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] flushed 2014-08-13 17:38:04.395738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 is on osd.4 2014-08-13 17:38:04.395755 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 is on osd.4 2014-08-13 17:38:04.395770 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395774 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.395757 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.395785 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4c73f40 already has epoch 194486 2014-08-13 17:38:04.395773 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 is on osd.4 2014-08-13 17:38:04.395767 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) 2014-08-13 17:38:04.395797 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 is on osd.4 2014-08-13 17:38:04.395808 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] flushed 2014-08-13 17:38:04.395810 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: got info from osd.294 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) 2014-08-13 17:38:04.395820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 is on osd.4 2014-08-13 17:38:04.395837 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.982556 7 0.000457 2014-08-13 17:38:04.395843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 is on osd.4 2014-08-13 17:38:04.395859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 is on osd.4 2014-08-13 17:38:04.395859 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.395876 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 is on osd.4 2014-08-13 17:38:04.395889 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.395893 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 is on osd.4 2014-08-13 17:38:04.395909 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.395909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 is on osd.4 2014-08-13 17:38:04.395928 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 is on osd.4 2014-08-13 17:38:04.395928 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'517 -> 194215'517 2014-08-13 17:38:04.395953 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.396009 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.395946 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 17:38:04.396012 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396014 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827e00 pg_info(1 pgs e194486:22.1a0) v4 2014-08-13 17:38:04.396013 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate: adding 289 as a source 2014-08-13 17:38:04.396042 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396030 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 is on osd.289 2014-08-13 17:38:04.396052 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1a0) v4 from osd.328 2014-08-13 17:38:04.396056 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827e00 2014-08-13 17:38:04.396048 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 is on osd.289 2014-08-13 17:38:04.396067 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 is on osd.289 2014-08-13 17:38:04.396091 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.396095 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396075 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 364 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.396087 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 is on osd.289 2014-08-13 17:38:04.396103 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info 2014-08-13 17:38:04.396105 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 is on osd.289 2014-08-13 17:38:04.396124 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 is on osd.289 2014-08-13 17:38:04.396140 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 is on osd.289 2014-08-13 17:38:04.396156 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 is on osd.289 2014-08-13 17:38:04.396188 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 is on osd.289 2014-08-13 17:38:04.396205 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 is on osd.289 2014-08-13 17:38:04.396222 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396222 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 is on osd.289 2014-08-13 17:38:04.396233 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 45 notify: (query_epoch:194486, epoch_sent:194486, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.396239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 is on osd.289 2014-08-13 17:38:04.396257 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 is on osd.289 2014-08-13 17:38:04.396258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 45, already have info from that osd, ignoring 2014-08-13 17:38:04.396277 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 is on osd.289 2014-08-13 17:38:04.396285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 289 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.396294 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 is on osd.289 2014-08-13 17:38:04.396306 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info 2014-08-13 17:38:04.396313 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 is on osd.289 2014-08-13 17:38:04.396330 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 is on osd.289 2014-08-13 17:38:04.396349 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 is on osd.289 2014-08-13 17:38:04.396365 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 is on osd.289 2014-08-13 17:38:04.396378 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396382 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 is on osd.289 2014-08-13 17:38:04.396386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.396399 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 is on osd.289 2014-08-13 17:38:04.396412 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.396418 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 is on osd.289 2014-08-13 17:38:04.396441 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 is on osd.289 2014-08-13 17:38:04.396432 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.396447 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 17:38:04.396470 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396460 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 is on osd.289 2014-08-13 17:38:04.396473 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4cfb2e0 already has epoch 194486 2014-08-13 17:38:04.396479 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 is on osd.289 2014-08-13 17:38:04.396496 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 is on osd.289 2014-08-13 17:38:04.396509 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] flushed 2014-08-13 17:38:04.396512 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 is on osd.289 2014-08-13 17:38:04.396529 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 is on osd.289 2014-08-13 17:38:04.396546 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 is on osd.289 2014-08-13 17:38:04.396563 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 is on osd.289 2014-08-13 17:38:04.396576 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.396580 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396583 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824b40 pg_info(1 pgs e194486:22.30f) v4 2014-08-13 17:38:04.396580 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 is on osd.289 2014-08-13 17:38:04.396598 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 is on osd.289 2014-08-13 17:38:04.396622 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.30f) v4 from osd.54 2014-08-13 17:38:04.396627 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824b40 2014-08-13 17:38:04.396615 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 is on osd.289 2014-08-13 17:38:04.396634 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 is on osd.289 2014-08-13 17:38:04.396651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 is on osd.289 2014-08-13 17:38:04.396667 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.396670 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396647 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 17 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.396670 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 is on osd.289 2014-08-13 17:38:04.396680 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info 2014-08-13 17:38:04.396689 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 is on osd.289 2014-08-13 17:38:04.396707 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 is on osd.289 2014-08-13 17:38:04.396723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 is on osd.289 2014-08-13 17:38:04.396742 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396740 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 is on osd.289 2014-08-13 17:38:04.396751 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) 2014-08-13 17:38:04.396771 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: got info from osd.337 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) 2014-08-13 17:38:04.396793 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.914302 7 0.000288 2014-08-13 17:38:04.396757 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 17:38:04.396813 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.396822 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate: adding 341 as a source 2014-08-13 17:38:04.396834 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.396841 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 also missing on osd.341 2014-08-13 17:38:04.396853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194477 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.396859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 also missing on osd.341 2014-08-13 17:38:04.396877 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.396880 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396883 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826e40 pg_info(1 pgs e194486:22.4aa) v4 2014-08-13 17:38:04.396870 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'495 -> 194215'495 2014-08-13 17:38:04.396877 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 also missing on osd.341 2014-08-13 17:38:04.396894 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.396896 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 also missing on osd.341 2014-08-13 17:38:04.396924 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4aa) v4 from osd.321 2014-08-13 17:38:04.396928 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826e40 2014-08-13 17:38:04.396916 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 also missing on osd.341 2014-08-13 17:38:04.396934 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 also missing on osd.341 2014-08-13 17:38:04.396952 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 also missing on osd.341 2014-08-13 17:38:04.396966 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.396969 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.396977 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.396969 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 also missing on osd.341 2014-08-13 17:38:04.396987 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 also missing on osd.341 2014-08-13 17:38:04.397003 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 also missing on osd.341 2014-08-13 17:38:04.397006 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.397020 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 also missing on osd.341 2014-08-13 17:38:04.397032 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.397052 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397038 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 also missing on osd.341 2014-08-13 17:38:04.397056 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 also missing on osd.341 2014-08-13 17:38:04.397058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.397073 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 also missing on osd.341 2014-08-13 17:38:04.397078 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.397095 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397091 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 also missing on osd.341 2014-08-13 17:38:04.397101 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.397110 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 also missing on osd.341 2014-08-13 17:38:04.397121 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 17:38:04.397138 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397133 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 also missing on osd.341 2014-08-13 17:38:04.397145 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 363 notify: (query_epoch:194486, epoch_sent:194486, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.397151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 also missing on osd.341 2014-08-13 17:38:04.397164 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 17:38:04.397181 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 also missing on osd.341 2014-08-13 17:38:04.397196 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397198 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 also missing on osd.341 2014-08-13 17:38:04.397205 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 324 notify: (query_epoch:194486, epoch_sent:194486, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.397216 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 also missing on osd.341 2014-08-13 17:38:04.397226 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 17:38:04.397244 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397247 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397251 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397249 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.397253 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260700 pg_info(1 pgs e194486:22.4aa) v4 2014-08-13 17:38:04.397256 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:04.397260 7fe99f5e9700 7 osd.4 194486 do_notify osd 17 on 1 PGs 2014-08-13 17:38:04.397291 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 don't know epoch, doing nothing 2014-08-13 17:38:04.397291 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4aa) v4 from osd.321 2014-08-13 17:38:04.397295 7fe99f5e9700 7 osd.4 194486 do_notify osd 289 on 1 PGs 2014-08-13 17:38:04.397296 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260700 2014-08-13 17:38:04.397236 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 also missing on osd.341 2014-08-13 17:38:04.397303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 also missing on osd.341 2014-08-13 17:38:04.397329 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing 2014-08-13 17:38:04.397333 7fe99f5e9700 7 osd.4 194486 do_notify osd 364 on 1 PGs 2014-08-13 17:38:04.397322 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 also missing on osd.341 2014-08-13 17:38:04.397335 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397351 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397353 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:04.397356 7fe99f5e9700 7 osd.4 194486 do_notify osd 374 on 1 PGs 2014-08-13 17:38:04.397342 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 also missing on osd.341 2014-08-13 17:38:04.397361 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 also missing on osd.341 2014-08-13 17:38:04.397376 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.397382 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397385 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397387 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51078c0 pg_info(1 pgs e194486:22.30f) v4 2014-08-13 17:38:04.397378 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 also missing on osd.341 2014-08-13 17:38:04.397397 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 also missing on osd.341 2014-08-13 17:38:04.397416 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 also missing on osd.341 2014-08-13 17:38:04.397433 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 also missing on osd.341 2014-08-13 17:38:04.397449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 also missing on osd.341 2014-08-13 17:38:04.397447 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.397473 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.30f) v4 from osd.54 2014-08-13 17:38:04.397478 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51078c0 2014-08-13 17:38:04.397468 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 also missing on osd.341 2014-08-13 17:38:04.397475 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.397488 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 also missing on osd.341 2014-08-13 17:38:04.397515 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397518 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397506 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 also missing on osd.341 2014-08-13 17:38:04.397510 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:04.397533 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397525 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 also missing on osd.341 2014-08-13 17:38:04.397543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 also missing on osd.341 2014-08-13 17:38:04.397541 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 367 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.397561 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 also missing on osd.341 2014-08-13 17:38:04.397562 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.397579 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 also missing on osd.341 2014-08-13 17:38:04.397567 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.397597 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 also missing on osd.341 2014-08-13 17:38:04.397591 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:04.397616 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 don't know epoch, doing nothing 2014-08-13 17:38:04.397621 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397614 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 also missing on osd.341 2014-08-13 17:38:04.397659 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] flushed 2014-08-13 17:38:04.397685 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397687 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397689 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821180 pg_info(1 pgs e194486:22.630) v4 2014-08-13 17:38:04.397630 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 17:38:04.397674 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.397697 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.397713 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.397715 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.397734 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.630) v4 from osd.294 2014-08-13 17:38:04.397737 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397738 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821180 2014-08-13 17:38:04.397732 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 289,327,341,376 2014-08-13 17:38:04.397745 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) 2014-08-13 17:38:04.397749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.397777 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.397781 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.397767 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.397766 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: got info from osd.397 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) 2014-08-13 17:38:04.397787 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.397792 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.907260 7 0.000271 2014-08-13 17:38:04.397803 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.397814 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.397836 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.397854 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.397871 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 17:38:04.397899 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.397894 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.397948 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 17:38:04.397973 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.397976 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.397980 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.397997 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.398014 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.398011 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 327 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.398040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.398066 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.398071 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] osd.327 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.398083 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.398099 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.398098 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 -> 4,17,28,315,327,330 2014-08-13 17:38:04.398118 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.398120 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.398129 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398137 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) 2014-08-13 17:38:04.398143 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 2014-08-13 17:38:04.398158 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: got info from osd.320 22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) 2014-08-13 17:38:04.398184 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.398190 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.985251 7 0.000380 2014-08-13 17:38:04.398191 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Started advmap 2014-08-13 17:38:04.398212 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.398210 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45,4] 2014-08-13 17:38:04.398228 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.398233 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.398261 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.398248 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started/Stray 0.875561 4 0.000170 2014-08-13 17:38:04.398265 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.398252 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194478 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.398268 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260000 pg_info(1 pgs e194486:22.630) v4 2014-08-13 17:38:04.398272 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'527 -> 194215'527 2014-08-13 17:38:04.398270 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started 0.875626 0 0.000000 2014-08-13 17:38:04.398296 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.398302 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Reset 2014-08-13 17:38:04.398322 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.630) v4 from osd.294 2014-08-13 17:38:04.398328 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260000 2014-08-13 17:38:04.398322 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194485 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194486 2014-08-13 17:38:04.398299 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194484 same_interval_since 194485 2014-08-13 17:38:04.398343 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194486 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 17:38:04.398346 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.398366 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194486 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.398380 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.398383 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.398394 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398382 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194486 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45,4] 2014-08-13 17:38:04.398399 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194486 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.398415 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4]/[343,45] r=-1 lpr=194486 pi=193262-194484/30 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194486 2014-08-13 17:38:04.398423 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 339 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.398448 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.339 activated and committed 2014-08-13 17:38:04.398450 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [343,45,4](343) acting [343,45](343)): not rw, up_thru 194483 up_from 192942 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [343,45,4](343) acting [343,45](343)) : primary up 192942-194483 does not include interval 2014-08-13 17:38:04.398467 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398468 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194485/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194485-194485 up [343,45,4](343) acting [343,45](343)) 2014-08-13 17:38:04.398475 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 342 notify: (query_epoch:194486, epoch_sent:194486, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.398487 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45,4] -> [343,45,4], acting [343,45] -> [343,45,4], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> 2 2014-08-13 17:38:04.398497 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 17:38:04.398482 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.398516 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398510 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:04.398526 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e48840 don't know epoch, doing nothing 2014-08-13 17:38:04.398526 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.398522 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 342 notify: (query_epoch:194486, epoch_sent:194486, info:22.707( v 194215'483 (0'0,194215'483] local-les=194484 n=19 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.398542 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:04.398546 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 17:38:04.398564 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398560 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 17:38:04.398559 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] flushed 2014-08-13 17:38:04.398571 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 330 notify: (query_epoch:194486, epoch_sent:194486, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.398579 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 17:38:04.398593 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 330, already have info from that osd, ignoring 2014-08-13 17:38:04.398609 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398595 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 17:38:04.398611 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 17:38:04.398616 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 312 info: 22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.398629 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 17:38:04.398635 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.312 activated and committed 2014-08-13 17:38:04.398653 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398646 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 17:38:04.398663 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_role_change 2014-08-13 17:38:04.398660 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 378 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.398679 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:04.398683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.398694 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 17:38:04.398709 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 17:38:04.398715 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:04.398739 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398726 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.398723 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.398748 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map 2014-08-13 17:38:04.398747 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 375 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.398766 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:04.398770 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.375 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.398793 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398792 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.398796 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.398799 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513db00 pg_info(1 pgs e194486:22.4fe) v4 2014-08-13 17:38:04.398800 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] flushed 2014-08-13 17:38:04.398804 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.398830 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 17:38:04.398851 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.398850 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe) v4 from osd.337 2014-08-13 17:38:04.398855 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513db00 2014-08-13 17:38:04.398839 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] update_heartbeat_peers unchanged 2014-08-13 17:38:04.398860 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters 2014-08-13 17:38:04.398858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) 2014-08-13 17:38:04.398880 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.000578 1 0.000556 2014-08-13 17:38:04.398880 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: got info from osd.371 22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) 2014-08-13 17:38:04.398905 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.398908 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.398902 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started 2014-08-13 17:38:04.398905 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.915333 7 0.000259 2014-08-13 17:38:04.398919 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start 2014-08-13 17:38:04.398925 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.398935 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray 2014-08-13 17:38:04.398946 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.398956 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000037 0 0.000000 2014-08-13 17:38:04.398966 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.398977 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray 2014-08-13 17:38:04.398983 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'493 -> 194215'493 2014-08-13 17:38:04.398999 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.399004 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.399016 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.399078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399083 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00fc0 pg_info(1 pgs e194486:22.4fe) v4 2014-08-13 17:38:04.399089 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399106 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.399124 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe) v4 from osd.337 2014-08-13 17:38:04.399128 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00fc0 2014-08-13 17:38:04.399115 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 368 notify: (query_epoch:194486, epoch_sent:194486, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.399139 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 17:38:04.399161 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399169 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399173 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399147 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 2014-08-13 17:38:04.399190 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.399169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 351 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.399197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: verifying no want_acting [] targets didn't go down 2014-08-13 17:38:04.399204 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.399225 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399228 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399216 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Started advmap 2014-08-13 17:38:04.399230 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d5c0 pg_info(1 pgs e194486:22.3b9) v4 2014-08-13 17:38:04.399234 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] 2014-08-13 17:38:04.399236 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 17:38:04.399261 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.399270 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3b9) v4 from osd.397 2014-08-13 17:38:04.399276 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d5c0 2014-08-13 17:38:04.399270 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.399270 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary/Peering/WaitActingChange 0.857996 1 0.000262 2014-08-13 17:38:04.399294 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.399294 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.858030 0 0.000000 2014-08-13 17:38:04.399314 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399315 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399318 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399317 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started 0.875228 0 0.000000 2014-08-13 17:38:04.399322 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.399341 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 17:38:04.399343 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset 2014-08-13 17:38:04.399358 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399359 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194486 2014-08-13 17:38:04.399365 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.399377 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Reset advmap 2014-08-13 17:38:04.399386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:04.399403 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399394 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.399409 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.399416 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 17:38:04.399410 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] 2014-08-13 17:38:04.399426 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399429 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399431 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00380 pg_info(1 pgs e194486:22.3b9) v4 2014-08-13 17:38:04.399428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.399445 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194486 2014-08-13 17:38:04.399479 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3b9) v4 from osd.397 2014-08-13 17:38:04.399484 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00380 2014-08-13 17:38:04.399482 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) : primary up 194485-194477 does not include interval 2014-08-13 17:38:04.399496 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.399501 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] noting past interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) 2014-08-13 17:38:04.399493 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 349 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.399525 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:04.399525 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.349 activated and committed 2014-08-13 17:38:04.399522 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] up [4,348,331] -> [4,348,331], acting [4,348,331] -> [4,332,337], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 17:38:04.399545 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399544 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_primary_state 2014-08-13 17:38:04.399554 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] flushed 2014-08-13 17:38:04.399556 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 350 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.399574 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] agent_stop 2014-08-13 17:38:04.399580 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.399605 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.399593 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change 2014-08-13 17:38:04.399626 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399629 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) 2014-08-13 17:38:04.399615 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194486:22148 2014-08-13 17:38:04.399634 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:04.399633 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: got info from osd.58 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) 2014-08-13 17:38:04.399655 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_copy_ops 2014-08-13 17:38:04.399658 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.035759 7 0.001420 2014-08-13 17:38:04.399671 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_flush_ops 2014-08-13 17:38:04.399678 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.399687 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:04.399714 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.399701 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.399717 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399703 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:04.399719 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e740 pg_info(1 pgs e194486:22.581) v4 2014-08-13 17:38:04.399720 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.406341 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.581) v4 from osd.320 2014-08-13 17:38:04.406348 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e740 2014-08-13 17:38:04.406352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 17:38:04.406396 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406399 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.399720 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:04.406394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.399725 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] flushed 2014-08-13 17:38:04.406431 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406433 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406436 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107540 MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 2014-08-13 17:38:04.406414 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change_cleanup 2014-08-13 17:38:04.406456 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change 2014-08-13 17:38:04.406485 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.289 now has 194486 2014-08-13 17:38:04.406489 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.406486 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107540 2014-08-13 17:38:04.406478 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit NotTrimming 2014-08-13 17:38:04.406503 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.406497 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter NotTrimming 2014-08-13 17:38:04.406516 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406519 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406515 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] [4,348,331] -> [4,332,337], replicas changed 2014-08-13 17:38:04.406539 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406541 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406543 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.653) v4 2014-08-13 17:38:04.406553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 304 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.406581 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.653) v4 from osd.371 2014-08-13 17:38:04.406585 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.406584 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info 2014-08-13 17:38:04.406621 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406624 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406641 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406643 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406645 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107700 pg_info(1 pgs e194486:22.362) v4 2014-08-13 17:38:04.406534 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_recovery 2014-08-13 17:38:04.406659 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.406668 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.406688 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.362) v4 from osd.58 2014-08-13 17:38:04.406692 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107700 2014-08-13 17:38:04.406679 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.406654 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_recovery_state 2014-08-13 17:38:04.406708 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.406704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.406730 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406732 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406734 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map 2014-08-13 17:38:04.406751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406739 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 17:38:04.406756 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9a1c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.406766 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.406771 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.406774 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406776 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406760 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last maybe_went_rw interval was interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 17:38:04.406800 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406803 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406787 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.080873 4 0.000767 2014-08-13 17:38:04.406804 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823d40 pg_info(1 pgs e194486:22.1d2) v4 2014-08-13 17:38:04.406811 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.406839 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1d2) v4 from osd.61 2014-08-13 17:38:04.406843 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823d40 2014-08-13 17:38:04.406834 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.4 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.406855 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.406756 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] update_heartbeat_peers 4,331,332,337,348,379 -> 4,331,332,337,348 2014-08-13 17:38:04.406883 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406884 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.406885 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.406888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] take_waiters 2014-08-13 17:38:04.406905 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406907 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406909 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db5e80 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.406897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.406916 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.406921 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.406923 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406926 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406909 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Reset 0.007567 1 0.007536 2014-08-13 17:38:04.406943 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.406945 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406947 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103d40 pg_info(1 pgs e194486:24.3bf) v4 2014-08-13 17:38:04.406935 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started 2014-08-13 17:38:04.406954 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Start 2014-08-13 17:38:04.407019 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.3bf) v4 from osd.19 2014-08-13 17:38:04.407023 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103d40 2014-08-13 17:38:04.407040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting newest update on osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) up[0] needs backfill, osd.4 selected as primary instead calc_acting primary is osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) shard 26 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) shard 30 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) shard 340 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:04.407065 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407068 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407065 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] actingbackfill is 4,26,30,340 2014-08-13 17:38:04.407088 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407090 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407092 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5481b00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.407083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] choose_acting want [4] (== acting) backfill_targets 26,30,340 2014-08-13 17:38:04.407099 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.407104 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.407106 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407107 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.406971 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: transitioning to Primary 2014-08-13 17:38:04.407108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: leaving GetLog 2014-08-13 17:38:04.407128 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407130 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407132 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00700 MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 2014-08-13 17:38:04.407122 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Start 0.000168 0 0.000000 2014-08-13 17:38:04.407132 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000321 0 0.000000 2014-08-13 17:38:04.407151 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary 2014-08-13 17:38:04.407156 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.407170 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Peering 2014-08-13 17:38:04.407175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.26 will fully backfill; can infer empty missing set 2014-08-13 17:38:04.407188 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetInfo 2014-08-13 17:38:04.407219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.30 will fully backfill; can infer empty missing set 2014-08-13 17:38:04.407219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:04.407237 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.340 will fully backfill; can infer empty missing set 2014-08-13 17:38:04.407241 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) 2014-08-13 17:38:04.407255 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: still need up_thru update before going active 2014-08-13 17:38:04.407263 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 17:38:04.407276 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000119 0 0.000000 2014-08-13 17:38:04.407292 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194482-194482 up [332,337,379](332) acting [332,337](332)) 2014-08-13 17:38:04.407298 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.407314 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00700 2014-08-13 17:38:04.407319 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 17:38:04.407343 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407346 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407335 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] up_thru 194485 < same_since 194486, must notify monitor 2014-08-13 17:38:04.407367 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407370 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 31 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.407373 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140540 pg_info(1 pgs e194486:22.619) v4 2014-08-13 17:38:04.407359 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194486:22149 2014-08-13 17:38:04.407379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.31 activated and committed 2014-08-13 17:38:04.407379 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.331 2014-08-13 17:38:04.407404 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407410 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.619) v4 from osd.62 2014-08-13 17:38:04.407415 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140540 2014-08-13 17:38:04.407406 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.332 2014-08-13 17:38:04.407412 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.407427 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.337 2014-08-13 17:38:04.407435 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.407455 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407456 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.348 2014-08-13 17:38:04.407478 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.407480 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407482 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823640 pg_info(1 pgs e194486:22.707) v4 2014-08-13 17:38:04.407469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.379 2014-08-13 17:38:04.407484 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info 2014-08-13 17:38:04.407490 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.407518 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.707) v4 from osd.21 2014-08-13 17:38:04.407522 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823640 2014-08-13 17:38:04.407509 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.407555 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407558 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407574 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407574 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407577 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407579 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826200 pg_info(1 pgs e194486:22.48e) v4 2014-08-13 17:38:04.407584 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.407615 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.407616 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.48e) v4 from osd.341 2014-08-13 17:38:04.407620 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826200 2014-08-13 17:38:04.407606 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info 2014-08-13 17:38:04.407678 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407681 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407695 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407675 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 17:38:04.407705 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407707 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c17e00 pg_info(1 pgs e194486:22.220) v4 2014-08-13 17:38:04.407714 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.407704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) 2014-08-13 17:38:04.407723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: Started advmap 2014-08-13 17:38:04.407727 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: got info from osd.328 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) 2014-08-13 17:38:04.407745 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.220) v4 from osd.351 2014-08-13 17:38:04.407750 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c17e00 2014-08-13 17:38:04.407744 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.407751 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.023098 7 0.000370 2014-08-13 17:38:04.407766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 17:38:04.407773 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.407791 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.407752 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.407794 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.407805 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:04.407795 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.407804 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 17:38:04.407815 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.407824 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.407834 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'506 -> 194215'506 2014-08-13 17:38:04.407843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.407842 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] flushed 2014-08-13 17:38:04.407866 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.407858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.407883 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] flushed 2014-08-13 17:38:04.407895 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 17:38:04.407920 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.407926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.407947 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.407943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.407959 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.407975 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.407975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 301 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.407993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.408001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.301 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.408011 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.408026 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.408028 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2648 2014-08-13 17:38:04.408034 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 54 notify: (query_epoch:194486, epoch_sent:194486, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.408044 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.408068 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.408055 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 54, already have info from that osd, ignoring 2014-08-13 17:38:04.408071 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.408077 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.408063 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.891559 6 0.000448 2014-08-13 17:38:04.408077 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106e40 MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 2014-08-13 17:38:04.408087 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.408085 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 291 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.408104 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.933018 0 0.000000 2014-08-13 17:38:04.408108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408129 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106e40 2014-08-13 17:38:04.408128 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.408156 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.408159 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.408146 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.408142 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 17:38:04.408169 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.408186 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.408168 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.092910 7 0.006903 2014-08-13 17:38:04.408209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'466 -> 194215'466 2014-08-13 17:38:04.408208 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.408232 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.408235 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 is up to date, queueing in pending_activators 2014-08-13 17:38:04.408258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.408279 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408295 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.408302 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 is up to date, queueing in pending_activators 2014-08-13 17:38:04.408323 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.408336 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.408323 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.408345 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.408369 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing 2014-08-13 17:38:04.408359 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.408376 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.408392 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.408401 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] flushed 2014-08-13 17:38:04.408410 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.408407 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.408429 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.408431 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,291,350,367,378 2014-08-13 17:38:04.408451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,291,350] (== acting) backfill_targets 367,378 2014-08-13 17:38:04.408487 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.408473 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog 2014-08-13 17:38:04.408497 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000288 0 0.000000 2014-08-13 17:38:04.408519 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.408537 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.291 has no missing, identical log 2014-08-13 17:38:04.408535 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 17:38:04.408562 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.408558 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.350 has no missing, identical log 2014-08-13 17:38:04.408567 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.408582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active 2014-08-13 17:38:04.408586 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.408603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.408603 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000084 0 0.000000 2014-08-13 17:38:04.408626 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.408645 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.408680 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.408675 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) 2014-08-13 17:38:04.408697 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.408702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.30 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) 2014-08-13 17:38:04.408713 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.408731 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.408725 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.917019 7 0.000269 2014-08-13 17:38:04.408744 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.408762 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.408761 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 17:38:04.408785 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.408781 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.408791 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.408798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 17:38:04.408809 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.408822 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.408826 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.408843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.408859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.408878 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.408883 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.408885 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c163c0 pg_info(1 pgs e194486:22.14a) v4 2014-08-13 17:38:04.408876 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.408903 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.408871 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed 2014-08-13 17:38:04.408895 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2995 2014-08-13 17:38:04.408915 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.408933 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.14a) v4 from osd.58 2014-08-13 17:38:04.408938 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c163c0 2014-08-13 17:38:04.408932 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.887281 6 0.000470 2014-08-13 17:38:04.408939 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 285 info: 22.f3( v 194215'506 lc 0'0 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.408958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.408963 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.285 activated and committed 2014-08-13 17:38:04.408986 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.408990 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.408975 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.926369 0 0.000000 2014-08-13 17:38:04.408982 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.408990 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.408999 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.409005 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.409016 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.409016 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.409033 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:04.409044 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.409060 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.409070 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9600 don't know epoch, doing nothing 2014-08-13 17:38:04.409075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'479 -> 194215'479 2014-08-13 17:38:04.409097 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.409099 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2860 2014-08-13 17:38:04.409126 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.409116 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 is up to date, queueing in pending_activators 2014-08-13 17:38:04.409144 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.409162 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) 2014-08-13 17:38:04.409180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 is up to date, queueing in pending_activators 2014-08-13 17:38:04.409196 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) 2014-08-13 17:38:04.409209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate 2014-08-13 17:38:04.409221 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) 2014-08-13 17:38:04.409232 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 17:38:04.409245 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.981682 7 0.000370 2014-08-13 17:38:04.409249 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.409265 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.409269 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.409281 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.409289 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.409297 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.409307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.409305 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.409326 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'496 -> 194215'496 2014-08-13 17:38:04.409348 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing 2014-08-13 17:38:04.409355 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.409349 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.409360 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.409364 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.409366 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824d00 pg_info(1 pgs e194486:22.2b9) v4 2014-08-13 17:38:04.409381 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] flushed 2014-08-13 17:38:04.409412 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2b9) v4 from osd.328 2014-08-13 17:38:04.409416 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824d00 2014-08-13 17:38:04.409434 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.409457 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.409460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.409460 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 20 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.409486 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.409488 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.409490 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827e00 pg_info(1 pgs e194486:22.14a) v4 2014-08-13 17:38:04.409485 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.409509 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.409527 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.14a) v4 from osd.58 2014-08-13 17:38:04.409531 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827e00 2014-08-13 17:38:04.409531 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 374 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.409553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.374 activated and committed 2014-08-13 17:38:04.409566 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.409569 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.409569 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.409564 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 17:38:04.409585 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.409598 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.409607 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:04.409605 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.409622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.409637 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:04.409639 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.409659 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2871 2014-08-13 17:38:04.409683 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.409685 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.409701 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.409716 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.409733 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.409783 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.409830 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 17:38:04.409876 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.409875 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.409858 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 2014-08-13 17:38:04.409886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 364 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192966) 2014-08-13 17:38:04.409908 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 2014-08-13 17:38:04.409912 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.409911 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] log request from 364 2014-08-13 17:38:04.409922 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Started advmap 2014-08-13 17:38:04.409937 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] flushed 2014-08-13 17:38:04.409942 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] 2014-08-13 17:38:04.409960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, transitioning to Reset 2014-08-13 17:38:04.409967 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info+missing+log since 194477'0 2014-08-13 17:38:04.409980 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started/Stray 0.924716 4 0.000185 2014-08-13 17:38:04.409989 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending log((194422'17074,194422'17074], crt=194422'17074) missing(48) 2014-08-13 17:38:04.410011 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing 2014-08-13 17:38:04.410004 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started 0.924781 0 0.000000 2014-08-13 17:38:04.410024 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Reset 2014-08-13 17:38:04.410043 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410040 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194486 2014-08-13 17:38:04.410043 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.410058 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 17:38:04.410068 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.410076 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 17:38:04.410093 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] 2014-08-13 17:38:04.410108 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 17:38:04.410124 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194486 2014-08-13 17:38:04.410158 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [6,374,4](6) acting [6,374](6)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [6,374,4](6) acting [6,374](6)) : primary up 192967-194483 does not include interval 2014-08-13 17:38:04.410175 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194485-194485 up [6,374,4](6) acting [6,374](6)) 2014-08-13 17:38:04.410192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374,4] -> [6,374,4], acting [6,374] -> [6,374,4], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> 2 2014-08-13 17:38:04.410213 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.410230 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.410234 7fe99f5e9700 7 osd.4 194486 do_notify osd 304 on 1 PGs 2014-08-13 17:38:04.410223 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:04.410240 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 17:38:04.410254 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:04.410271 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 17:38:04.410277 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.410289 7fe99f5e9700 7 osd.4 194486 do_notify osd 356 on 2 PGs 2014-08-13 17:38:04.410292 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410296 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410298 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824ec0 pg_info(1 pgs e194486:22.2b9) v4 2014-08-13 17:38:04.410286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 17:38:04.410308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 17:38:04.410324 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 17:38:04.410342 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2b9) v4 from osd.328 2014-08-13 17:38:04.410347 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824ec0 2014-08-13 17:38:04.410359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 45 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.410342 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 17:38:04.410388 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410392 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410387 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info 2014-08-13 17:38:04.410389 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 17:38:04.410412 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410414 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410416 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820e00 pg_info(1 pgs e194486:22.791) v4 2014-08-13 17:38:04.410407 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_role_change 2014-08-13 17:38:04.410423 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:04.410438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 17:38:04.410452 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410452 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.791) v4 from osd.30 2014-08-13 17:38:04.410457 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820e00 2014-08-13 17:38:04.410453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 17:38:04.410471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.410467 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.410492 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410496 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410493 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map 2014-08-13 17:38:04.410492 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:04.410516 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410523 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 343 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.410554 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] update_heartbeat_peers unchanged 2014-08-13 17:38:04.410571 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters 2014-08-13 17:38:04.410544 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed 2014-08-13 17:38:04.410587 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.000563 1 0.000571 2014-08-13 17:38:04.410603 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410608 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started 2014-08-13 17:38:04.410612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 286 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.410624 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start 2014-08-13 17:38:04.410640 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray 2014-08-13 17:38:04.410637 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.410660 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000035 0 0.000000 2014-08-13 17:38:04.410671 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 17:38:04.410714 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410703 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray 2014-08-13 17:38:04.410739 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410723 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 324 notify: (query_epoch:194486, epoch_sent:194486, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.410726 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.410743 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410746 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f01500 pg_info(1 pgs e194486:22.791) v4 2014-08-13 17:38:04.410745 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.410793 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.791) v4 from osd.30 2014-08-13 17:38:04.410798 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f01500 2014-08-13 17:38:04.410786 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 17:38:04.410809 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.410837 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410840 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410840 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.410816 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.410862 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410864 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410866 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822d80 pg_info(1 pgs e194486:22.116) v4 2014-08-13 17:38:04.410900 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.116) v4 from osd.345 2014-08-13 17:38:04.410903 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822d80 2014-08-13 17:38:04.410902 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 17:38:04.410929 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.410935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 17:38:04.410947 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410954 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410955 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.410974 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410976 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.410978 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54db3c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.410972 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 17:38:04.410987 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.410992 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.410995 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.410996 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.411038 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 17:38:04.411054 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.411069 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.411093 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.411117 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 17:38:04.411140 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:04.411158 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.411146 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 17:38:04.411162 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.411167 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513dcc0 pg_notify(22.4df(27) epoch 194486) v5 2014-08-13 17:38:04.411166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false 2014-08-13 17:38:04.411153 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:04.411195 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.411184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 17:38:04.411212 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:04.411230 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.380 2014-08-13 17:38:04.411234 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513dcc0 2014-08-13 17:38:04.411219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) 2014-08-13 17:38:04.411231 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 17:38:04.411244 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.373 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) 2014-08-13 17:38:04.411252 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 17:38:04.411283 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.411271 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2593 2014-08-13 17:38:04.411270 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.945729 7 0.000263 2014-08-13 17:38:04.411287 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.411289 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 17:38:04.411306 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.870532 3 0.000524 2014-08-13 17:38:04.411310 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.411297 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] flushed 2014-08-13 17:38:04.411327 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 17:38:04.411334 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.411344 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.919524 0 0.000000 2014-08-13 17:38:04.411356 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.411367 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 17:38:04.411378 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'522 -> 194215'522 2014-08-13 17:38:04.411384 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 17:38:04.411406 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:04.411415 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.411423 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 17:38:04.411438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'511 -> 194215'511 2014-08-13 17:38:04.411461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) 2014-08-13 17:38:04.411480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 is up to date, queueing in pending_activators 2014-08-13 17:38:04.411520 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.411566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.411592 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 320, already have info from that osd, ignoring 2014-08-13 17:38:04.411610 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.411599 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) uptodate 2014-08-13 17:38:04.411621 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) 2014-08-13 17:38:04.411640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 is up to date, queueing in pending_activators 2014-08-13 17:38:04.411624 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 330 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193227) 2014-08-13 17:38:04.411683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] log request from 330 2014-08-13 17:38:04.411703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) missing missing(28) 2014-08-13 17:38:04.411733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info+missing+log since 194478'0 2014-08-13 17:38:04.411748 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.376 has 28 missing 2014-08-13 17:38:04.411759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending log((194422'15537,194422'15537], crt=194422'15537) missing(64) 2014-08-13 17:38:04.411766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.4 2014-08-13 17:38:04.411782 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing 2014-08-13 17:38:04.411785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.4 2014-08-13 17:38:04.411812 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.411803 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.4 2014-08-13 17:38:04.411820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.4 2014-08-13 17:38:04.411835 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.4 2014-08-13 17:38:04.411850 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.4 2014-08-13 17:38:04.411866 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.4 2014-08-13 17:38:04.411881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.4 2014-08-13 17:38:04.411897 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.4 2014-08-13 17:38:04.411889 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.411914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.4 2014-08-13 17:38:04.411929 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.4 2014-08-13 17:38:04.411945 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.4 2014-08-13 17:38:04.411960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.4 2014-08-13 17:38:04.411975 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.4 2014-08-13 17:38:04.411990 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.4 2014-08-13 17:38:04.411997 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:04.411993 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 365 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.412007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.4 2014-08-13 17:38:04.412024 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.4 2014-08-13 17:38:04.412023 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.365 2014-08-13 17:38:04.412041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.4 2014-08-13 17:38:04.412039 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] flushed 2014-08-13 17:38:04.412058 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.4 2014-08-13 17:38:04.412077 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.4 2014-08-13 17:38:04.412094 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.4 2014-08-13 17:38:04.412058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.412109 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.4 2014-08-13 17:38:04.412122 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412129 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.4 2014-08-13 17:38:04.412144 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.4 2014-08-13 17:38:04.412160 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.4 2014-08-13 17:38:04.412161 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 351 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.412176 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.4 2014-08-13 17:38:04.412192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.4 2014-08-13 17:38:04.412219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.4 2014-08-13 17:38:04.412193 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.412241 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412250 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.412235 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 17:38:04.412284 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:04.412286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 338 as a source 2014-08-13 17:38:04.412302 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.338 2014-08-13 17:38:04.412314 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.412320 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.338 2014-08-13 17:38:04.412339 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.338 2014-08-13 17:38:04.412338 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 17:38:04.412355 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.338 2014-08-13 17:38:04.412371 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412373 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.338 2014-08-13 17:38:04.412378 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 318 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.412390 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.338 2014-08-13 17:38:04.412400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.318 activated and committed 2014-08-13 17:38:04.412421 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.338 2014-08-13 17:38:04.412427 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.338 2014-08-13 17:38:04.412427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 26 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.412445 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.338 2014-08-13 17:38:04.412450 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.26 2014-08-13 17:38:04.412463 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.338 2014-08-13 17:38:04.412481 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.338 2014-08-13 17:38:04.412469 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours 2014-08-13 17:38:04.412506 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.412499 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.338 2014-08-13 17:38:04.412516 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.338 2014-08-13 17:38:04.412514 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.412533 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.338 2014-08-13 17:38:04.412535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412551 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.338 2014-08-13 17:38:04.412568 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.338 2014-08-13 17:38:04.412568 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 17:38:04.412587 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.338 2014-08-13 17:38:04.412593 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetInfo 0.133158 7 0.000735 2014-08-13 17:38:04.412604 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.338 2014-08-13 17:38:04.412618 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.412622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.338 2014-08-13 17:38:04.412640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.338 2014-08-13 17:38:04.412644 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.4 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412657 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.338 2014-08-13 17:38:04.412667 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412676 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.338 2014-08-13 17:38:04.412689 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.412694 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.338 2014-08-13 17:38:04.412712 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.338 2014-08-13 17:38:04.412708 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412730 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.338 2014-08-13 17:38:04.412730 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.338 2014-08-13 17:38:04.412767 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.338 2014-08-13 17:38:04.412784 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.338 2014-08-13 17:38:04.412788 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting newest update on osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) shard 286 (up) backfill 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.375 (up) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) shard 10 (stray) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.412812 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] actingbackfill is 4,10,286,375 2014-08-13 17:38:04.412802 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 17:38:04.412853 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 376 as a source 2014-08-13 17:38:04.412851 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] choose_acting want [4,375,10] (== acting) backfill_targets 286 2014-08-13 17:38:04.412871 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 also missing on osd.376 2014-08-13 17:38:04.412879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: leaving GetLog 2014-08-13 17:38:04.412890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 also missing on osd.376 2014-08-13 17:38:04.412908 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 also missing on osd.376 2014-08-13 17:38:04.412907 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetLog 0.000288 0 0.000000 2014-08-13 17:38:04.412926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 also missing on osd.376 2014-08-13 17:38:04.412932 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.412943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 also missing on osd.376 2014-08-13 17:38:04.412962 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 also missing on osd.376 2014-08-13 17:38:04.412959 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.10 has no missing, identical log 2014-08-13 17:38:04.412980 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 also missing on osd.376 2014-08-13 17:38:04.412983 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.375 has no missing, identical log 2014-08-13 17:38:04.412999 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.413008 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.413011 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824600 pg_info(1 pgs e194486:22.1fb) v4 2014-08-13 17:38:04.412998 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 also missing on osd.376 2014-08-13 17:38:04.413002 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: still need up_thru update before going active 2014-08-13 17:38:04.413022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 also missing on osd.376 2014-08-13 17:38:04.413025 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetMissing 0.000092 0 0.000000 2014-08-13 17:38:04.413040 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 also missing on osd.376 2014-08-13 17:38:04.413057 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1fb) v4 from osd.373 2014-08-13 17:38:04.413062 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824600 2014-08-13 17:38:04.413048 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.413071 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.413059 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 also missing on osd.376 2014-08-13 17:38:04.413078 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 also missing on osd.376 2014-08-13 17:38:04.413106 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.413109 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.413095 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 also missing on osd.376 2014-08-13 17:38:04.413101 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 304 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193250) 2014-08-13 17:38:04.413114 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 also missing on osd.376 2014-08-13 17:38:04.413125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log request from 304 2014-08-13 17:38:04.413133 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 also missing on osd.376 2014-08-13 17:38:04.413151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 also missing on osd.376 2014-08-13 17:38:04.413167 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 also missing on osd.376 2014-08-13 17:38:04.413184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 also missing on osd.376 2014-08-13 17:38:04.413189 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info+missing+log since 194477'0 2014-08-13 17:38:04.413206 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 also missing on osd.376 2014-08-13 17:38:04.413209 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending log((194422'16034,194422'16034], crt=194422'16034) missing(61) 2014-08-13 17:38:04.413230 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.413225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 also missing on osd.376 2014-08-13 17:38:04.413242 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 also missing on osd.376 2014-08-13 17:38:04.413259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 also missing on osd.376 2014-08-13 17:38:04.413275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 also missing on osd.376 2014-08-13 17:38:04.413292 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 also missing on osd.376 2014-08-13 17:38:04.413308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 also missing on osd.376 2014-08-13 17:38:04.413325 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 also missing on osd.376 2014-08-13 17:38:04.413339 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.413341 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 also missing on osd.376 2014-08-13 17:38:04.413359 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 also missing on osd.376 2014-08-13 17:38:04.413373 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 17:38:04.413422 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 17:38:04.413438 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.413441 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.413443 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00540 pg_info(1 pgs e194486:22.1fb) v4 2014-08-13 17:38:04.413436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 17:38:04.413468 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 293,338,373,376,382,396,397 2014-08-13 17:38:04.413487 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.413498 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1fb) v4 from osd.373 2014-08-13 17:38:04.413506 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00540 2014-08-13 17:38:04.413477 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] flushed 2014-08-13 17:38:04.413506 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 17:38:04.413524 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 2014-08-13 17:38:04.413540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt 2014-08-13 17:38:04.413571 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.413574 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.413651 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.413619 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.413680 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.413657 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.340 2014-08-13 17:38:04.413690 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e3d800 192.168.1.107:6809/53141 2014-08-13 17:38:04.413689 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.340 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours 2014-08-13 17:38:04.413736 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.413849 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.6 has 194486 2014-08-13 17:38:04.413858 7fe99ebe8700 7 osd.4 194486 do_notify osd 6 on 1 PGs 2014-08-13 17:38:04.414180 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 2014-08-13 17:38:04.414193 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 28 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.414219 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 17:38:04.414238 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.28 activated and committed 2014-08-13 17:38:04.414243 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:04.414259 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.414258 7fe99ebe8700 7 osd.4 194486 do_notify osd 15 on 1 PGs 2014-08-13 17:38:04.414267 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193151) 2014-08-13 17:38:04.414295 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:04.414299 7fe99ebe8700 7 osd.4 194486 do_notify osd 31 on 1 PGs 2014-08-13 17:38:04.414287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log request from 356 2014-08-13 17:38:04.414321 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing 2014-08-13 17:38:04.414324 7fe99ebe8700 7 osd.4 194486 do_notify osd 51 on 1 PGs 2014-08-13 17:38:04.414347 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:04.414350 7fe99ebe8700 7 osd.4 194486 do_notify osd 285 on 1 PGs 2014-08-13 17:38:04.414351 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info+missing+log since 194477'0 2014-08-13 17:38:04.414372 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending log((194422'17433,194422'17433], crt=194422'17433) missing(51) 2014-08-13 17:38:04.414414 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.414422 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e3c8e0 192.168.1.113:6812/16056 2014-08-13 17:38:04.414460 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.414662 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.414672 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.292 has 194486 2014-08-13 17:38:04.414687 7fe99ebe8700 7 osd.4 194486 do_notify osd 292 on 1 PGs 2014-08-13 17:38:04.414767 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.414771 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.414773 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dc140 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.414796 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.414802 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.414804 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.414806 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.414824 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:04.414828 7fe99f5e9700 7 osd.4 194486 do_notify osd 45 on 1 PGs 2014-08-13 17:38:04.414908 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing 2014-08-13 17:38:04.414919 7fe99ebe8700 7 osd.4 194486 do_notify osd 323 on 1 PGs 2014-08-13 17:38:04.414940 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.414944 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.414945 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265b00 pg_notify(22.1b8(25) epoch 194486) v5 2014-08-13 17:38:04.415052 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.7 2014-08-13 17:38:04.415056 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265b00 2014-08-13 17:38:04.415122 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194422'18845 ec=192688 les/c 194484/194484 194485/194486/192969) 2014-08-13 17:38:04.415181 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.415186 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.415192 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.415202 7fe99ebe8700 7 osd.4 194486 do_notify osd 343 on 1 PGs 2014-08-13 17:38:04.415225 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.415228 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.415229 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 2014-08-13 17:38:04.415232 7fe99ebe8700 7 osd.4 194486 do_notify osd 345 on 1 PGs 2014-08-13 17:38:04.415229 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299eea0 pg_query(24.cd epoch 194486) v3 2014-08-13 17:38:04.415252 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:04.415255 7fe99ebe8700 7 osd.4 194486 do_notify osd 376 on 1 PGs 2014-08-13 17:38:04.415276 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 2014-08-13 17:38:04.415279 7fe99ebe8700 7 osd.4 194486 do_notify osd 396 on 1 PGs 2014-08-13 17:38:04.415291 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.6 epoch 194486 2014-08-13 17:38:04.422272 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299eea0 2014-08-13 17:38:04.415300 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d791e0 don't know epoch, doing nothing 2014-08-13 17:38:04.422301 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194422'15992 ec=192688 les/c 194484/194480 194485/194486/192967) from replica 6 2014-08-13 17:38:04.415157 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 2014-08-13 17:38:04.422325 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.296 on 1 PGs 2014-08-13 17:38:04.422350 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422354 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422372 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5a432e0 don't know epoch, doing nothing 2014-08-13 17:38:04.422376 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.307 on 1 PGs 2014-08-13 17:38:04.422380 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422385 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265240 pg_notify(22.84(2) epoch 194486) v5 2014-08-13 17:38:04.422361 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194422'18845 2014-08-13 17:38:04.422390 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.337 now has 194486 2014-08-13 17:38:04.422397 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing 2014-08-13 17:38:04.422400 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.320 on 1 PGs 2014-08-13 17:38:04.422400 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 2014-08-13 17:38:04.422393 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) 2014-08-13 17:38:04.422426 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.7 2014-08-13 17:38:04.422429 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265240 2014-08-13 17:38:04.422440 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.422463 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 2014-08-13 17:38:04.422471 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422474 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422477 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.422493 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422495 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422497 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299f260 pg_query(24.1e4 epoch 194486) v3 2014-08-13 17:38:04.422530 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.31 epoch 194486 2014-08-13 17:38:04.422533 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299f260 2014-08-13 17:38:04.422596 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(log 194478'0 ec=192688 les/c 194484/194372 194485/194486/192978) from replica 31 2014-08-13 17:38:04.422447 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing 2014-08-13 17:38:04.422649 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422652 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422642 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.331 on 1 PGs 2014-08-13 17:38:04.422638 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 301 info: 22.22c( v 194215'514 lc 0'0 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.422672 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422674 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422677 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5974b00 pg_query(24.1e6 epoch 194486) v3 2014-08-13 17:38:04.422666 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.301 activated and committed 2014-08-13 17:38:04.422689 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.422694 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing 2014-08-13 17:38:04.422698 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.332 on 1 PGs 2014-08-13 17:38:04.422698 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 291 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.422720 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.285 epoch 194486 2014-08-13 17:38:04.422723 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5974b00 2014-08-13 17:38:04.422723 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 2014-08-13 17:38:04.422727 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.337 on 1 PGs 2014-08-13 17:38:04.422724 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.291 2014-08-13 17:38:04.422746 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing 2014-08-13 17:38:04.422750 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.344 on 1 PGs 2014-08-13 17:38:04.422736 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192975) from replica 285 2014-08-13 17:38:04.422769 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing 2014-08-13 17:38:04.422772 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.348 on 1 PGs 2014-08-13 17:38:04.422746 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.422786 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.422781 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422798 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422819 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422804 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.422821 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422823 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299dfa0 pg_query(24.206 epoch 194486) v3 2014-08-13 17:38:04.422826 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.422843 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.422850 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 30 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:04.422881 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.343 epoch 194486 2014-08-13 17:38:04.422884 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299dfa0 2014-08-13 17:38:04.422872 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.30 2014-08-13 17:38:04.422889 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.30 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours 2014-08-13 17:38:04.422897 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193262) from replica 343 2014-08-13 17:38:04.422924 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.422925 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e82c00 don't know epoch, doing nothing 2014-08-13 17:38:04.422934 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.379 on 1 PGs 2014-08-13 17:38:04.422934 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.422961 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422964 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422961 7fe99ebe8700 20 osd.4 194486 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 50 2014-08-13 17:38:04.422973 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing 2014-08-13 17:38:04.422957 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.422990 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.422993 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.422995 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db0900 osd_map(194479..194486 src has 185139..194486) v3 2014-08-13 17:38:04.422997 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 286 2014-08-13 17:38:04.423004 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194479,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.423006 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:04.422992 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 17:38:04.423012 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.423015 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423015 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.423017 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423039 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423034 7fe99ebe8700 20 osd.4 194486 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) to shard 287 2014-08-13 17:38:04.423024 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 45 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.423041 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423044 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df3700 don't know epoch, doing nothing 2014-08-13 17:38:04.423044 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5974740 pg_query(24.20e epoch 194486) v3 2014-08-13 17:38:04.423045 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.45 activated and committed 2014-08-13 17:38:04.423062 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.423080 7fe99ebe8700 20 osd.4 194486 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 289 2014-08-13 17:38:04.423078 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.423087 7fe99ebe8700 20 osd.4 194486 Sending info 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) to shard 289 2014-08-13 17:38:04.423096 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.423102 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:04.423106 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.15 epoch 194486 2014-08-13 17:38:04.423109 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5974740 2014-08-13 17:38:04.423144 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec8160 don't know epoch, doing nothing 2014-08-13 17:38:04.423156 7fe99ebe8700 20 osd.4 194486 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) to shard 309 2014-08-13 17:38:04.423192 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193264) from replica 15 2014-08-13 17:38:04.423252 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423255 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423270 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing 2014-08-13 17:38:04.423264 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2933 2014-08-13 17:38:04.423302 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.423347 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423350 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423352 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db1440 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.423372 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.423378 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.423380 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423384 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) to shard 338 2014-08-13 17:38:04.423374 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 289 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192968) 2014-08-13 17:38:04.423403 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d789a0 don't know epoch, doing nothing 2014-08-13 17:38:04.423406 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log request from 289 2014-08-13 17:38:04.423449 7fe99ebe8700 20 osd.4 194486 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 340 2014-08-13 17:38:04.423458 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.423462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info+missing+log since 194477'0 2014-08-13 17:38:04.423482 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 343 2014-08-13 17:38:04.423491 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.423480 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending log((194422'17034,194422'17034], crt=194422'17034) missing(63) 2014-08-13 17:38:04.423499 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.423624 7fe99ebe8700 20 osd.4 194486 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 347 2014-08-13 17:38:04.423645 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.423741 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.423747 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.423749 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16ac0 pg_notify(24.181(33) epoch 194486) v5 2014-08-13 17:38:04.423815 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing 2014-08-13 17:38:04.423800 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 313 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.423842 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.296 2014-08-13 17:38:04.423833 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.313 activated and committed 2014-08-13 17:38:04.423847 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16ac0 2014-08-13 17:38:04.423850 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) to shard 376 2014-08-13 17:38:04.423860 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:04.423852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.423868 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.423892 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfd6a0 don't know epoch, doing nothing 2014-08-13 17:38:04.423947 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x57cfe80 don't know epoch, doing nothing 2014-08-13 17:38:04.424051 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 289 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.424074 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2705 2014-08-13 17:38:04.424086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring 2014-08-13 17:38:04.424107 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424109 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.424122 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 289 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.424129 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.424143 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring 2014-08-13 17:38:04.424161 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424156 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.424168 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 50 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.424194 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 50, already have info from that osd, ignoring 2014-08-13 17:38:04.424212 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424190 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 17:38:04.424223 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424218 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 32 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.424233 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424227 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.068925 6 0.000727 2014-08-13 17:38:04.424236 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 32, already have info from that osd, ignoring 2014-08-13 17:38:04.424253 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424264 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424267 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424270 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8700 pg_notify(24.1bb(2) epoch 194486) v5 2014-08-13 17:38:04.424259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 343 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.424258 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.424280 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 343, already have info from that osd, ignoring 2014-08-13 17:38:04.424297 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424288 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.424304 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 338 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:04.424326 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.332 2014-08-13 17:38:04.424331 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8700 2014-08-13 17:38:04.424313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.424324 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 338, already have info from that osd, ignoring 2014-08-13 17:38:04.424343 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424336 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.424350 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) 2014-08-13 17:38:04.424356 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.424379 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424371 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.323 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) 2014-08-13 17:38:04.424376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.424395 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.912355 7 0.000272 2014-08-13 17:38:04.424397 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.424417 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.424444 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424447 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424449 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823480 pg_notify(24.1bb epoch 194486) v5 2014-08-13 17:38:04.424438 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.424458 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.424457 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.424477 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'491 -> 194215'491 2014-08-13 17:38:04.424496 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.331 2014-08-13 17:38:04.424481 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] actingbackfill is 4,50,55,61,65 2014-08-13 17:38:04.424499 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823480 2014-08-13 17:38:04.424502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.424502 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] choose_acting want [4,50,65] (== acting) backfill_targets 55,61 2014-08-13 17:38:04.424528 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: leaving GetLog 2014-08-13 17:38:04.424553 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000295 0 0.000000 2014-08-13 17:38:04.424569 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424583 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424591 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424577 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.424596 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.50 has no missing, identical log 2014-08-13 17:38:04.424616 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.65 has no missing, identical log 2014-08-13 17:38:04.424629 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424643 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) 2014-08-13 17:38:04.424634 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: still need up_thru update before going active 2014-08-13 17:38:04.424645 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc440 pg_notify(24.1bb(35) epoch 194486) v5 2014-08-13 17:38:04.424648 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.51 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) 2014-08-13 17:38:04.424656 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000 2014-08-13 17:38:04.424671 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.943393 7 0.000290 2014-08-13 17:38:04.424694 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.379 2014-08-13 17:38:04.424680 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.424698 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc440 2014-08-13 17:38:04.424701 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.424696 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.424726 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.424729 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 50 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.424744 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.424755 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.50 2014-08-13 17:38:04.424762 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'501 -> 194215'501 2014-08-13 17:38:04.424785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.424775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.50 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.424811 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.424822 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424820 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 17 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193188) 2014-08-13 17:38:04.424839 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log request from 17 2014-08-13 17:38:04.424881 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.424879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info+missing+log since 194477'0 2014-08-13 17:38:04.424906 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.424909 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.424896 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending log((194422'14353,194422'14353], crt=194422'14353) missing(49) 2014-08-13 17:38:04.424911 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5100000 pg_notify(24.181(12) epoch 194486) v5 2014-08-13 17:38:04.424917 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:04.424910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) 2014-08-13 17:38:04.424932 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: got info from osd.292 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) 2014-08-13 17:38:04.424954 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.903675 7 0.000259 2014-08-13 17:38:04.424973 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.424991 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.425007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.425022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 17:38:04.425044 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.425119 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425098 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.425126 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.425159 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing 2014-08-13 17:38:04.425145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 347 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.425145 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 362 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.425175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.425170 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 347, already have info from that osd, ignoring 2014-08-13 17:38:04.425211 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425206 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] flushed 2014-08-13 17:38:04.425216 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:04.425239 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.425220 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.425244 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] flushed 2014-08-13 17:38:04.425246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 346 query_epoch 194486 query: query(log 194422'15014 ec=192688 les/c 194484/194484 194485/194486/192955) 2014-08-13 17:38:04.425265 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info 2014-08-13 17:38:04.425269 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 2014-08-13 17:38:04.425301 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.320 2014-08-13 17:38:04.425305 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5100000 2014-08-13 17:38:04.425316 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194422'15014 2014-08-13 17:38:04.425336 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425334 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) 2014-08-13 17:38:04.425356 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e4a100 already has epoch 194486 2014-08-13 17:38:04.425362 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425345 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485)) 2014-08-13 17:38:04.425365 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425369 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 17:38:04.425385 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425387 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.425391 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425394 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425397 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5482d00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.425392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 397 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:04.425410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 397, already have info from that osd, ignoring 2014-08-13 17:38:04.425422 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.425429 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425429 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.425432 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425434 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425473 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425476 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425478 7fe9a63f4700 20 osd.4 194486 _dispatch 0x2999860 pg_query(24.3b7 epoch 194486) v3 2014-08-13 17:38:04.425468 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.425536 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.356 epoch 194486 2014-08-13 17:38:04.425540 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x2999860 2014-08-13 17:38:04.425553 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/192969) from replica 356 2014-08-13 17:38:04.425565 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 324 info: 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.425576 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 379 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.425606 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425593 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.324 activated and committed 2014-08-13 17:38:04.425610 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425611 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.425607 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.425627 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.425651 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing 2014-08-13 17:38:04.425639 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.425663 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425667 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425669 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104600 pg_notify(24.181(33) epoch 194486) v5 2014-08-13 17:38:04.425664 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348 -> 4,331,332,337,348,379 2014-08-13 17:38:04.425685 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.425693 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425689 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing 2014-08-13 17:38:04.425711 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.344 2014-08-13 17:38:04.425714 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104600 2014-08-13 17:38:04.425736 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:04.425741 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] flushed 2014-08-13 17:38:04.425767 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2830 2014-08-13 17:38:04.425777 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.425795 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.425794 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 327 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.425833 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 327, already have info from that osd, ignoring 2014-08-13 17:38:04.425853 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.425901 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e3c8e0 already has epoch 194486 2014-08-13 17:38:04.425898 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.425924 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.425942 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.425937 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] flushed 2014-08-13 17:38:04.425950 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.425967 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.425976 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.425972 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.426009 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426012 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426014 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265b00 pg_info(1 pgs e194486:22.48e) v4 2014-08-13 17:38:04.426001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.426025 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 2014-08-13 17:38:04.426044 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.426052 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426060 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 327 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.426086 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.48e) v4 from osd.289 2014-08-13 17:38:04.426070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 339 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.426099 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265b00 2014-08-13 17:38:04.426083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.426107 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 339, already have info from that osd, ignoring 2014-08-13 17:38:04.426128 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 65 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.426142 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426137 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 293 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) 2014-08-13 17:38:04.426137 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.65 2014-08-13 17:38:04.426165 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426157 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 293, already have info from that osd, ignoring 2014-08-13 17:38:04.426186 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426189 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426192 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426193 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822f40 pg_info(1 pgs e194486:22.6bd) v4 2014-08-13 17:38:04.426157 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.65 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.426206 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426193 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 381 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:04.426214 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 381, already have info from that osd, ignoring 2014-08-13 17:38:04.426232 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426218 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.426239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 396 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:04.426240 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.373 activated and committed 2014-08-13 17:38:04.426260 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426261 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.6bd) v4 from osd.50 2014-08-13 17:38:04.426266 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822f40 2014-08-13 17:38:04.426267 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.426258 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 396, already have info from that osd, ignoring 2014-08-13 17:38:04.426275 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426281 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 309 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:04.426299 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 309, already have info from that osd, ignoring 2014-08-13 17:38:04.426314 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426320 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.426326 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 2014-08-13 17:38:04.426313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 31 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.426330 7fe99ebe8700 7 osd.4 194486 do_notify osd 6 on 1 PGs 2014-08-13 17:38:04.426344 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426335 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info 2014-08-13 17:38:04.426350 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426373 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426375 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426377 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820380 pg_notify(24.1bb(35) epoch 194486) v5 2014-08-13 17:38:04.426412 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.426441 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426449 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.337 2014-08-13 17:38:04.426435 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:04.426452 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820380 2014-08-13 17:38:04.426453 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 285 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.426461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 306 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:04.426473 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info 2014-08-13 17:38:04.426480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 306, already have info from that osd, ignoring 2014-08-13 17:38:04.426497 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 382 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:04.426528 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426531 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 382, already have info from that osd, ignoring 2014-08-13 17:38:04.426536 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426545 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426542 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.426554 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) 2014-08-13 17:38:04.426560 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 340, already have info from that osd, ignoring 2014-08-13 17:38:04.426576 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) 2014-08-13 17:38:04.426597 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426601 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426590 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.426604 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106200 pg_info(1 pgs e194486:22.3d5) v4 2014-08-13 17:38:04.426597 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.942741 7 0.000242 2014-08-13 17:38:04.426609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 17:38:04.426628 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426619 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.426633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.426641 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.426661 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d5) v4 from osd.343 2014-08-13 17:38:04.426666 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106200 2014-08-13 17:38:04.426652 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 17:38:04.426671 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426661 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.426680 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'488 -> 194215'488 2014-08-13 17:38:04.426677 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 373 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:04.426702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.426704 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 17:38:04.426721 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426727 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 365 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485)) 2014-08-13 17:38:04.426746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 17:38:04.426761 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426774 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426778 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426767 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.426790 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426786 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 10, already have info from that osd, ignoring 2014-08-13 17:38:04.426802 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426804 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426807 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426809 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107380 pg_notify(24.181(33) epoch 194486) v5 2014-08-13 17:38:04.426809 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.426820 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 343 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.426832 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.426849 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.296 2014-08-13 17:38:04.426850 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426852 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107380 2014-08-13 17:38:04.426845 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info 2014-08-13 17:38:04.426858 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 337 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.426881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.426919 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.426914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:04.426936 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426945 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426929 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 15 query_epoch 194486 query: query(info 0'0) 2014-08-13 17:38:04.426949 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426944 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 286 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.426951 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info 2014-08-13 17:38:04.426964 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 286, already have info from that osd, ignoring 2014-08-13 17:38:04.426981 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.426985 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.426988 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.426990 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824b40 pg_info(1 pgs e194486:22.3d9) v4 2014-08-13 17:38:04.426987 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:04.427005 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:04.427021 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427029 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.427027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.427059 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d9) v4 from osd.338 2014-08-13 17:38:04.427044 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194483/194463 194485/194485/193205) 2014-08-13 17:38:04.427048 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle_pg_notify from osd.10 2014-08-13 17:38:04.427064 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824b40 2014-08-13 17:38:04.427068 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.396 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194483/194463 194485/194485/193205) 2014-08-13 17:38:04.427071 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.10 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.427109 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427097 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.953287 7 0.000214 2014-08-13 17:38:04.427114 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427118 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427119 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.427118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.427142 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.427159 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427144 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.427168 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427164 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.427171 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 304 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.427201 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820700 pg_info(1 pgs e194486:22.12a) v4 2014-08-13 17:38:04.427182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'15411 -> 194422'15411 2014-08-13 17:38:04.427197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.304 activated and committed 2014-08-13 17:38:04.427215 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.427231 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.427216 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.427253 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:04.427266 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.12a) v4 from osd.323 2014-08-13 17:38:04.427272 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820700 2014-08-13 17:38:04.427269 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] state: Activate Finished 2014-08-13 17:38:04.427291 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.427316 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427319 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427339 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427341 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427354 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.427342 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2902 2014-08-13 17:38:04.427343 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58263c0 pg_info(1 pgs e194486:22.3d5) v4 2014-08-13 17:38:04.427374 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427403 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.427426 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d5) v4 from osd.286 2014-08-13 17:38:04.427431 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58263c0 2014-08-13 17:38:04.427430 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.427463 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.55 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.427487 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 -> 4,55,296,320,344 2014-08-13 17:38:04.427506 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.427511 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.427540 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427543 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427563 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427576 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427578 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821880 pg_info(1 pgs e194486:22.537) v4 2014-08-13 17:38:04.427618 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.537) v4 from osd.51 2014-08-13 17:38:04.427622 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821880 2014-08-13 17:38:04.427673 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427649 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 344 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.427676 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427696 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427681 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.344 activated and committed 2014-08-13 17:38:04.427701 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427703 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107700 pg_info(1 pgs e194486:22.3ad) v4 2014-08-13 17:38:04.427703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.427719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.427744 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing 2014-08-13 17:38:04.427746 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3ad) v4 from osd.292 2014-08-13 17:38:04.427762 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107700 2014-08-13 17:38:04.427775 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing 2014-08-13 17:38:04.427810 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427798 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2613 2014-08-13 17:38:04.427813 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427824 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427847 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427850 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427851 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106580 pg_info(1 pgs e194486:22.1b8) v4 2014-08-13 17:38:04.427891 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1b8) v4 from osd.347 2014-08-13 17:38:04.427895 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106580 2014-08-13 17:38:04.427888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.427914 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.427932 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.427936 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427943 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427939 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 45 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193186) 2014-08-13 17:38:04.427958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log request from 45 2014-08-13 17:38:04.427982 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.427985 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.427986 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cbf00 pg_notify(24.1bb(2) epoch 194486) v5 2014-08-13 17:38:04.428003 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194477'0 2014-08-13 17:38:04.428030 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.332 2014-08-13 17:38:04.428033 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cbf00 2014-08-13 17:38:04.428021 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending log((194422'13576,194422'13576], crt=194422'13576) missing(50) 2014-08-13 17:38:04.428041 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:04.428070 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428087 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428090 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428110 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428112 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428114 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5971860 pg_query(24.e2 epoch 194486) v3 2014-08-13 17:38:04.428225 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.346 epoch 194486 2014-08-13 17:38:04.428228 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5971860 2014-08-13 17:38:04.428235 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.428250 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 2014-08-13 17:38:04.428240 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192955) from replica 346 2014-08-13 17:38:04.428262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.428262 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) 2014-08-13 17:38:04.428293 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428292 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428296 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428301 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.428294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.376 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) 2014-08-13 17:38:04.428294 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] flushed 2014-08-13 17:38:04.428326 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428328 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428330 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51039c0 pg_info(1 pgs e194486:22.3d9) v4 2014-08-13 17:38:04.428324 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.937697 7 0.000266 2014-08-13 17:38:04.428345 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 17:38:04.428369 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d9) v4 from osd.376 2014-08-13 17:38:04.428353 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.428375 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51039c0 2014-08-13 17:38:04.428367 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:04.428364 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] _activate_committed 194486 telling primary 2014-08-13 17:38:04.428380 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.428401 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428401 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 2014-08-13 17:38:04.428389 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 17:38:04.428416 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428419 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'485 -> 194215'485 2014-08-13 17:38:04.428438 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428440 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428442 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826e40 pg_info(1 pgs e194486:22.481) v4 2014-08-13 17:38:04.428432 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] state: Activate Finished 2014-08-13 17:38:04.428434 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed 2014-08-13 17:38:04.428478 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.481) v4 from osd.309 2014-08-13 17:38:04.428482 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826e40 2014-08-13 17:38:04.428524 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428545 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428548 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428568 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192966) 2014-08-13 17:38:04.428551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.428570 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428584 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cdb00 pg_notify(24.1bb epoch 194486) v5 2014-08-13 17:38:04.428582 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 17:38:04.428605 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428620 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.428631 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 17:38:04.428647 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428654 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 344 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.428650 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 362 notify: (query_epoch:194486, epoch_sent:194486, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.428686 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.348 2014-08-13 17:38:04.428689 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cdb00 2014-08-13 17:38:04.428678 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 362, already have info from that osd, ignoring 2014-08-13 17:38:04.428698 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428676 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.428706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192946) 2014-08-13 17:38:04.428711 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged 2014-08-13 17:38:04.428733 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428754 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428756 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428740 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.428764 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 17:38:04.428780 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428781 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.428787 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.428805 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.428824 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.428828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.428808 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.428829 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428830 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105b00 pg_info(1 pgs e194486:22.1b8) v4 2014-08-13 17:38:04.428837 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged 2014-08-13 17:38:04.428858 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428863 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192976) 2014-08-13 17:38:04.428865 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 330 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.428887 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.428926 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.428914 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 17:38:04.428934 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428941 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.428951 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 59 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.428963 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops 2014-08-13 17:38:04.428982 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.428976 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.428988 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 315 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.429008 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged 2014-08-13 17:38:04.429008 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.429032 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.429034 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.075034 8 0.000711 2014-08-13 17:38:04.429038 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 25 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.429059 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.429058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.429085 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 17:38:04.429086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429078 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] _activate_committed 194486 telling primary 2014-08-13 17:38:04.429123 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:04.429110 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.147487 7 0.000722 2014-08-13 17:38:04.429110 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429133 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.429140 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.428906 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1b8) v4 from osd.340 2014-08-13 17:38:04.429152 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429159 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] flushed 2014-08-13 17:38:04.429166 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429185 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105b00 2014-08-13 17:38:04.429184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429191 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429224 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.429247 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.429250 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.429243 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429272 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.429275 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.429277 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.6bd) v4 2014-08-13 17:38:04.429264 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.429267 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429283 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429291 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] actingbackfill is 4,55,59,351 2014-08-13 17:38:04.429311 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] choose_acting want [4,351,55] (== acting) backfill_targets 59 2014-08-13 17:38:04.429345 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.6bd) v4 from osd.289 2014-08-13 17:38:04.429350 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.429334 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: leaving GetLog 2014-08-13 17:38:04.429344 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.429362 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000304 0 0.000000 2014-08-13 17:38:04.429366 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] actingbackfill is 4,20,25,303,362 2014-08-13 17:38:04.429395 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.429398 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.429386 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.429386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] choose_acting want [4,20,303] (== acting) backfill_targets 25,362 2014-08-13 17:38:04.429417 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.429420 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.429423 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105780 pg_info(1 pgs e194486:22.12a) v4 2014-08-13 17:38:04.429411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.55 has no missing, identical log 2014-08-13 17:38:04.429416 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: leaving GetLog 2014-08-13 17:38:04.429434 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.351 has no missing, identical log 2014-08-13 17:38:04.429441 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000301 0 0.000000 2014-08-13 17:38:04.429461 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.12a) v4 from osd.323 2014-08-13 17:38:04.429454 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: still need up_thru update before going active 2014-08-13 17:38:04.429470 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105780 2014-08-13 17:38:04.429469 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.437931 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.437935 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.429477 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000091 0 0.000000 2014-08-13 17:38:04.437921 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.20 has no missing, identical log 2014-08-13 17:38:04.437961 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.437964 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.437967 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102300 pg_info(1 pgs e194486:22.537) v4 2014-08-13 17:38:04.437964 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.303 has no missing, identical log 2014-08-13 17:38:04.437985 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: still need up_thru update before going active 2014-08-13 17:38:04.437971 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.438016 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.438020 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.537) v4 from osd.51 2014-08-13 17:38:04.438025 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102300 2014-08-13 17:38:04.438010 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.008540 0 0.000000 2014-08-13 17:38:04.438035 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.438061 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438061 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.438063 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438056 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.438082 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438084 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438087 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103640 pg_info(1 pgs e194486:22.3ad) v4 2014-08-13 17:38:04.438084 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.438105 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.438096 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 310 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.438126 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3ad) v4 from osd.292 2014-08-13 17:38:04.438131 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103640 2014-08-13 17:38:04.438115 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 356 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.438232 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.310 activated and committed 2014-08-13 17:38:04.438235 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438255 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438258 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438259 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16ac0 pg_info(1 pgs e194486:22.226) v4 2014-08-13 17:38:04.438255 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.438140 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.356 2014-08-13 17:38:04.438295 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.226) v4 from osd.376 2014-08-13 17:38:04.438299 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16ac0 2014-08-13 17:38:04.438275 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 357 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.438311 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.357 activated and committed 2014-08-13 17:38:04.438332 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.438339 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 353 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.438298 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.438362 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438364 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.438366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.353 activated and committed 2014-08-13 17:38:04.438376 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.438382 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.438385 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438388 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438390 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823800 pg_info(1 pgs e194486:22.619) v4 2014-08-13 17:38:04.438390 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:04.438395 7fe99f5e9700 7 osd.4 194486 do_notify osd 15 on 1 PGs 2014-08-13 17:38:04.438381 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 374 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/193232) 2014-08-13 17:38:04.438406 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] log request from 374 2014-08-13 17:38:04.438426 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:04.438430 7fe99f5e9700 7 osd.4 194486 do_notify osd 31 on 1 PGs 2014-08-13 17:38:04.438403 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] flushed 2014-08-13 17:38:04.438441 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.619) v4 from osd.65 2014-08-13 17:38:04.438450 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:04.438446 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823800 2014-08-13 17:38:04.438454 7fe99f5e9700 7 osd.4 194486 do_notify osd 285 on 1 PGs 2014-08-13 17:38:04.438452 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info+missing+log since 194477'0 2014-08-13 17:38:04.438475 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.438456 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] flushed 2014-08-13 17:38:04.438481 7fe99f5e9700 7 osd.4 194486 do_notify osd 343 on 1 PGs 2014-08-13 17:38:04.438473 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending log((194422'16449,194422'16449], crt=194422'16449) missing(52) 2014-08-13 17:38:04.438500 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438503 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438521 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438523 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438525 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51078c0 pg_info(1 pgs e194486:24.299) v4 2014-08-13 17:38:04.438564 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 28 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.438589 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.299) v4 from osd.396 2014-08-13 17:38:04.438595 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51078c0 2014-08-13 17:38:04.438592 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.438625 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 17:38:04.438642 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438645 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.087382 8 0.000742 2014-08-13 17:38:04.438679 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438682 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438676 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:04.438684 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9680 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.438693 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.438698 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.438674 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.438700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438702 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438719 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.438710 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.438726 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438734 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438737 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58271c0 pg_info(1 pgs e194486:22.29a) v4 2014-08-13 17:38:04.438731 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) 2014-08-13 17:38:04.438733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.438756 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.438779 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.29a) v4 from osd.345 2014-08-13 17:38:04.438783 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58271c0 2014-08-13 17:38:04.438794 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.438776 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.438798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.438820 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438823 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438818 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.438842 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438844 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438846 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9da00 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.438852 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.438856 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.438858 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438860 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438878 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438880 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438882 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58239c0 MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 2014-08-13 17:38:04.438876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.438903 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,17,28,315,330 2014-08-13 17:38:04.438919 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58239c0 2014-08-13 17:38:04.438923 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,17,315] (== acting) backfill_targets 28,330 2014-08-13 17:38:04.438947 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438949 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438967 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.438952 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog 2014-08-13 17:38:04.438970 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.438972 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820fc0 MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 2014-08-13 17:38:04.438977 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000303 0 0.000000 2014-08-13 17:38:04.439012 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820fc0 2014-08-13 17:38:04.439001 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.439021 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.17 has no missing, identical log 2014-08-13 17:38:04.439007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 362 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.439040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.315 has no missing, identical log 2014-08-13 17:38:04.439050 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.362 activated and committed 2014-08-13 17:38:04.439059 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active 2014-08-13 17:38:04.439075 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439080 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 2014-08-13 17:38:04.439085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) 2014-08-13 17:38:04.439102 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439107 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439102 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.439123 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.439131 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439133 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439135 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9440 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.439143 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439142 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.439165 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.439168 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439170 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439192 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439194 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439196 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825b00 pg_info(1 pgs e194486:22.29a) v4 2014-08-13 17:38:04.439176 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 356 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.439195 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) 2014-08-13 17:38:04.439213 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.356 activated and committed 2014-08-13 17:38:04.439232 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.439248 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439249 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.29a) v4 from osd.345 2014-08-13 17:38:04.439254 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825b00 2014-08-13 17:38:04.439246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.439271 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.439274 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.439306 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x5431600 don't know epoch, doing nothing 2014-08-13 17:38:04.439299 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 17:38:04.439316 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.439329 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439333 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439331 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.439357 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439384 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821180 pg_info(1 pgs e194486:22.1f1) v4 2014-08-13 17:38:04.439400 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed 2014-08-13 17:38:04.439428 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1f1) v4 from osd.5 2014-08-13 17:38:04.439433 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821180 2014-08-13 17:38:04.439470 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.439466 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2658 2014-08-13 17:38:04.439502 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.439524 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439528 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439552 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439555 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439557 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105cc0 pg_notify(1.2b(2) epoch 194486) v5 2014-08-13 17:38:04.439542 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:176850 2014-08-13 17:38:04.439575 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 17 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.439584 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.17 2014-08-13 17:38:04.439618 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.370 2014-08-13 17:38:04.439621 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105cc0 2014-08-13 17:38:04.439603 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.17 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.439641 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.439661 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439664 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439683 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439685 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439686 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106e40 pg_notify(2.2a(2) epoch 194486) v5 2014-08-13 17:38:04.439760 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.370 2014-08-13 17:38:04.439764 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106e40 2014-08-13 17:38:04.439800 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.439803 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.439787 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 47 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.439797 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.439818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.47 activated and committed 2014-08-13 17:38:04.439822 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle_pg_notify from osd.303 2014-08-13 17:38:04.439839 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439848 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.439841 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.303 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.439876 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.439870 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.439888 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439885 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 373 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.439897 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.439905 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle_pg_notify from osd.373 2014-08-13 17:38:04.439917 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.439935 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439924 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.373 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.439942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.439962 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.439976 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.439978 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.439990 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 56 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.439994 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194480 194485/194485/192972) 2014-08-13 17:38:04.440011 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.56 activated and committed 2014-08-13 17:38:04.440029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.440047 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440043 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.440066 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c77640 don't know epoch, doing nothing 2014-08-13 17:38:04.440071 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.440096 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x57570c0 don't know epoch, doing nothing 2014-08-13 17:38:04.440113 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) 2014-08-13 17:38:04.440128 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2721 2014-08-13 17:38:04.440154 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440189 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440325 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.440343 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440318 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194480 194485/194485/192949) 2014-08-13 17:38:04.440352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440373 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.440392 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440402 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440410 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440418 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) 2014-08-13 17:38:04.440430 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.440448 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194480 194485/194485/192936) 2014-08-13 17:38:04.440469 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440491 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440511 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440514 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.440532 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440545 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192971) 2014-08-13 17:38:04.440560 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 17:38:04.440577 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440599 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440585 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) 2014-08-13 17:38:04.440630 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440616 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) 2014-08-13 17:38:04.440661 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440718 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.440728 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) 2014-08-13 17:38:04.440746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.58 activated and committed 2014-08-13 17:38:04.440765 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.440783 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.440804 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing 2014-08-13 17:38:04.440802 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.440824 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.440842 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440841 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.440867 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2681 2014-08-13 17:38:04.440874 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192968) 2014-08-13 17:38:04.440894 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.440924 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.440958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 61 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.440982 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.61 activated and committed 2014-08-13 17:38:04.440981 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) 2014-08-13 17:38:04.440999 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.441014 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.441033 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.441038 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:04.441072 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5380c60 don't know epoch, doing nothing 2014-08-13 17:38:04.441099 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:3020 2014-08-13 17:38:04.441123 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.441219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194486 n=44 ec=192688 les/c 194486/194463 194485/194485/194468) 2014-08-13 17:38:04.441282 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.441302 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery 2014-08-13 17:38:04.441328 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.042382 3 0.000163 2014-08-13 17:38:04.441353 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 17:38:04.441391 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.441383 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.441400 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 62 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.441423 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.62 activated and committed 2014-08-13 17:38:04.441425 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.441442 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.441446 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.441449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 21 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.441455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.441469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.21 activated and committed 2014-08-13 17:38:04.441480 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.441499 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.441488 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.441504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.441508 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.441526 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d335a0 don't know epoch, doing nothing 2014-08-13 17:38:04.441529 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.441548 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.441590 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf900 don't know epoch, doing nothing 2014-08-13 17:38:04.441575 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/192943) 2014-08-13 17:38:04.441620 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2569 2014-08-13 17:38:04.441638 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.441646 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.441659 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.441707 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) 2014-08-13 17:38:04.442013 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442045 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192958) 2014-08-13 17:38:04.442036 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) 2014-08-13 17:38:04.442076 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.442080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.442081 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826ac0 pg_info(1 pgs e194486:22.3af) v4 2014-08-13 17:38:04.442084 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442086 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) 2014-08-13 17:38:04.442107 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3af) v4 from osd.44 2014-08-13 17:38:04.442099 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442110 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826ac0 2014-08-13 17:38:04.442113 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442114 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442125 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442123 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) 2014-08-13 17:38:04.442138 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.442140 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.442129 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442151 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442145 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] requeue_ops 2014-08-13 17:38:04.442156 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442175 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery 2014-08-13 17:38:04.442175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery 2014-08-13 17:38:04.442210 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.043977 3 0.000177 2014-08-13 17:38:04.442239 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 17:38:04.442250 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442255 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 341 info: 22.48e( v 194215'543 lc 0'0 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.442266 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.341 activated and committed 2014-08-13 17:38:04.442278 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442208 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.059228 3 0.000177 2014-08-13 17:38:04.442282 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.442319 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.442337 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] needs_recovery osd.285 has 29 missing 2014-08-13 17:38:04.442295 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 17:38:04.442371 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.442433 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442441 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442455 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442466 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442483 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops 2014-08-13 17:38:04.442493 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442497 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2861 2014-08-13 17:38:04.442499 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192935) 2014-08-13 17:38:04.442508 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.442520 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.442529 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442529 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442550 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442560 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442548 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442566 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.442566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.442577 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442581 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.442578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.442586 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442610 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.31 has 29 missing 2014-08-13 17:38:04.442625 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.374 has 29 missing 2014-08-13 17:38:04.442639 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.442636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192949) 2014-08-13 17:38:04.442691 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442702 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) 2014-08-13 17:38:04.442729 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) 2014-08-13 17:38:04.442764 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442759 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2872 2014-08-13 17:38:04.442768 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.442773 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442777 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.442786 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442786 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.442796 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442790 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442802 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.442801 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 7 notify: (query_epoch:194486, epoch_sent:194486, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.442812 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442813 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring 2014-08-13 17:38:04.442823 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442816 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.442843 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 17:38:04.442842 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 31 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194372 194485/194486/192978) 2014-08-13 17:38:04.442860 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442863 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log request from 31 2014-08-13 17:38:04.442867 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194480 194485/194485/192976) 2014-08-13 17:38:04.442917 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.442910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info+missing+log since 194478'0 2014-08-13 17:38:04.442920 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending log((194422'18567,194422'18567], crt=194422'18567) missing(46) 2014-08-13 17:38:04.442931 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing 2014-08-13 17:38:04.442928 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) 2014-08-13 17:38:04.442951 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.442957 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443063 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 285 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192975) 2014-08-13 17:38:04.443079 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log request from 285 2014-08-13 17:38:04.443073 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) 2014-08-13 17:38:04.443113 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443111 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194477'0 2014-08-13 17:38:04.443121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending log((194422'17642,194422'17642], crt=194422'17642) missing(50) 2014-08-13 17:38:04.443132 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:04.443124 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 380 notify: (query_epoch:194486, epoch_sent:194486, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.443137 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 380, already have info from that osd, ignoring 2014-08-13 17:38:04.443164 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443165 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.443173 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.443200 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 17:38:04.443223 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 343 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193262) 2014-08-13 17:38:04.443293 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.443300 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log request from 343 2014-08-13 17:38:04.443306 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.443317 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443321 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/192978) 2014-08-13 17:38:04.443323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info+missing+log since 194477'0 2014-08-13 17:38:04.443333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending log((194422'14476,194422'14476], crt=194422'14476) missing(41) 2014-08-13 17:38:04.443344 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.443348 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443359 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.443359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.443373 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] requeue_ops 2014-08-13 17:38:04.443384 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 15 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193264) 2014-08-13 17:38:04.443441 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) 2014-08-13 17:38:04.443450 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] log request from 15 2014-08-13 17:38:04.443470 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443480 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.443480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194478'0 2014-08-13 17:38:04.443490 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending log((194422'11806,194422'11806], crt=194422'11806) missing(56) 2014-08-13 17:38:04.443501 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing 2014-08-13 17:38:04.443501 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 7 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) 2014-08-13 17:38:04.443515 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.443514 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring 2014-08-13 17:38:04.443524 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.443612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(log 194422'15992 ec=192688 les/c 194484/194480 194485/194486/192967) 2014-08-13 17:38:04.443624 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.443626 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 2014-08-13 17:38:04.443633 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.443642 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.45 has 27 missing 2014-08-13 17:38:04.443651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.339 has 27 missing 2014-08-13 17:38:04.443659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.443660 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194422'15992 2014-08-13 17:38:04.443671 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) 2014-08-13 17:38:04.443681 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 2014-08-13 17:38:04.443701 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443756 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2934 2014-08-13 17:38:04.443766 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.443788 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.443804 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.443819 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.443823 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 296 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.443844 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.443862 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.443847 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.443869 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.443882 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged 2014-08-13 17:38:04.443888 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.443907 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.443904 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.443915 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.443919 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] needs_recovery osd.301 has 38 missing 2014-08-13 17:38:04.443935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.443935 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.443952 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.443966 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.313 has 32 missing 2014-08-13 17:38:04.443981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.318 has 32 missing 2014-08-13 17:38:04.443995 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.444160 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2831 2014-08-13 17:38:04.444177 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.444178 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2706 2014-08-13 17:38:04.444207 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.444217 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444210 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.444220 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.444229 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444222 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.444233 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.444243 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444234 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 332 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.444247 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 289 info: 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.444246 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.444259 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed 2014-08-13 17:38:04.444264 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:04.444268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.444276 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444277 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.444282 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 331 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.444291 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.444294 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.444311 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df32e0 don't know epoch, doing nothing 2014-08-13 17:38:04.444309 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:04.444320 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444328 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2688 2014-08-13 17:38:04.444343 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 379 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.444346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.444359 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.444501 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 50 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.444517 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.444519 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.50 activated and committed 2014-08-13 17:38:04.444530 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444527 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444539 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.444541 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 343 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.444552 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] requeue_ops 2014-08-13 17:38:04.444553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed 2014-08-13 17:38:04.444565 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444565 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444574 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.444577 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 286 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.444588 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.286 activated and committed 2014-08-13 17:38:04.444586 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.320 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours 2014-08-13 17:38:04.444599 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444597 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.444603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/192969) 2014-08-13 17:38:04.444606 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.444619 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:04.444615 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 2014-08-13 17:38:04.444638 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.444636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194478'0 2014-08-13 17:38:04.444647 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) 2014-08-13 17:38:04.444659 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing 2014-08-13 17:38:04.444653 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2649 2014-08-13 17:38:04.444668 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444694 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194480 194485/194485/192971) 2014-08-13 17:38:04.444707 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444728 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444782 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 344 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.444796 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192972) 2014-08-13 17:38:04.444803 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.344 info 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours 2014-08-13 17:38:04.444819 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444823 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.444826 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444840 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.444847 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 337 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.444853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.444862 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.444870 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.344 has 23 missing 2014-08-13 17:38:04.444861 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.444883 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.373 has 23 missing 2014-08-13 17:38:04.444888 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.444888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 296 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.444900 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.296 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours 2014-08-13 17:38:04.444913 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444916 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 338 info: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) 2014-08-13 17:38:04.444929 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.338 activated and committed 2014-08-13 17:38:04.444938 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194480 194485/194485/192978) 2014-08-13 17:38:04.444976 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.444970 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2614 2014-08-13 17:38:04.444979 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.444988 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.444997 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.444993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.445001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 347 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.445007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.445014 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.347 activated and committed 2014-08-13 17:38:04.445024 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445017 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.445028 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.445029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 346 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192955) 2014-08-13 17:38:04.445037 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.445041 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 2014-08-13 17:38:04.445046 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.445066 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2903 2014-08-13 17:38:04.445065 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194477'0 2014-08-13 17:38:04.445076 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.445081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) 2014-08-13 17:38:04.445085 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.445093 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e4a100 already has epoch 194486 2014-08-13 17:38:04.445095 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.445100 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 332 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:04.445111 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445112 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours 2014-08-13 17:38:04.445124 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.445225 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.445240 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.445250 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445243 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) 2014-08-13 17:38:04.445254 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.445262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.376 activated and committed 2014-08-13 17:38:04.445267 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops 2014-08-13 17:38:04.445278 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445273 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.445282 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.445282 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 309 info: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) 2014-08-13 17:38:04.445295 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d789a0 don't know epoch, doing nothing 2014-08-13 17:38:04.445294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.309 activated and committed 2014-08-13 17:38:04.445304 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445303 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.445306 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.445307 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299d280 pg_query(24.cd epoch 194486) v3 2014-08-13 17:38:04.445319 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:04.445331 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.6 epoch 194486 2014-08-13 17:38:04.445333 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299d280 2014-08-13 17:38:04.445334 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2594 2014-08-13 17:38:04.445337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.445350 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.445341 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192967) from replica 6 2014-08-13 17:38:04.445351 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops 2014-08-13 17:38:04.445361 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445370 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.445372 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.445375 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 348 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 17:38:04.445377 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.445390 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 17:38:04.445390 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.445400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.445427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.445428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 17:38:04.445450 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.445453 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.038265 9 0.000991 2014-08-13 17:38:04.445479 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing 2014-08-13 17:38:04.445477 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.445503 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2996 2014-08-13 17:38:04.445502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.445535 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445530 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.445541 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.445552 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.445554 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 289 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) 2014-08-13 17:38:04.445563 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:04.445568 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed 2014-08-13 17:38:04.445574 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.445578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.445602 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.445623 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing 2014-08-13 17:38:04.445640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:04.445653 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.445659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] actingbackfill is 4,331,332,337,348 2014-08-13 17:38:04.445674 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2703 2014-08-13 17:38:04.445689 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] choose_acting want [4,332,337] (== acting) backfill_targets 331,348 2014-08-13 17:38:04.445701 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: leaving GetLog 2014-08-13 17:38:04.445716 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000239 0 0.000000 2014-08-13 17:38:04.445729 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.445739 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.332 has no missing, identical log 2014-08-13 17:38:04.445746 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) 2014-08-13 17:38:04.445764 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.337 has no missing, identical log 2014-08-13 17:38:04.445783 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: still need up_thru update before going active 2014-08-13 17:38:04.445812 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445804 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000075 0 0.000000 2014-08-13 17:38:04.445825 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.445845 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.445836 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) 2014-08-13 17:38:04.445884 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445870 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) 2014-08-13 17:38:04.445902 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) 2014-08-13 17:38:04.445923 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.445948 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445948 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.445975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194486/194463 194485/194485/193205) 2014-08-13 17:38:04.445978 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] requeue_ops 2014-08-13 17:38:04.445990 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.446004 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.445996 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 65 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.446009 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.65 activated and committed 2014-08-13 17:38:04.446013 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192981) 2014-08-13 17:38:04.446018 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.446027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.446040 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446040 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a4dd80 don't know epoch, doing nothing 2014-08-13 17:38:04.446062 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing 2014-08-13 17:38:04.446078 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2733 2014-08-13 17:38:04.446116 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery 2014-08-13 17:38:04.446150 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.017783 3 0.000167 2014-08-13 17:38:04.446170 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 17:38:04.446202 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446222 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestBackfillPrio: priority 1 2014-08-13 17:38:04.446226 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] flushed 2014-08-13 17:38:04.446240 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] exit Started/ReplicaActive/RepNotRecovering 0.019096 3 0.000234 2014-08-13 17:38:04.446253 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] enter Started/ReplicaActive/RepWaitBackfillReserved 2014-08-13 17:38:04.446264 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) 2014-08-13 17:38:04.446298 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446308 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.446320 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.446329 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.446337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.61 has 28 missing 2014-08-13 17:38:04.446346 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.349 has 28 missing 2014-08-13 17:38:04.446354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.446507 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:3021 2014-08-13 17:38:04.446516 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.446524 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.446533 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RemoteRecoveryReserved 2014-08-13 17:38:04.446553 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing 2014-08-13 17:38:04.446570 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.005217 1 0.000070 2014-08-13 17:38:04.446582 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepRecovering 2014-08-13 17:38:04.446593 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446603 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.446615 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.446628 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.446622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.446637 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.446646 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.21 has 19 missing 2014-08-13 17:38:04.446644 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] requeue_ops 2014-08-13 17:38:04.446657 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.446655 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.310 has 19 missing 2014-08-13 17:38:04.446663 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.446662 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.446676 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops 2014-08-13 17:38:04.446707 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.446714 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 5 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.446727 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.5 activated and committed 2014-08-13 17:38:04.446734 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2570 2014-08-13 17:38:04.446737 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.446743 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.446747 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.446753 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.446763 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.446762 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d36b40 don't know epoch, doing nothing 2014-08-13 17:38:04.446785 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4829b80 don't know epoch, doing nothing 2014-08-13 17:38:04.446802 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:3006 2014-08-13 17:38:04.446823 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.446859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.446872 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.446880 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.446891 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.356 has 35 missing 2014-08-13 17:38:04.446900 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.357 has 35 missing 2014-08-13 17:38:04.446909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.446900 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 44 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) 2014-08-13 17:38:04.446922 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.44 activated and committed 2014-08-13 17:38:04.446932 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.446941 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.446953 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d348e0 don't know epoch, doing nothing 2014-08-13 17:38:04.446972 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:04.446986 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2683 2014-08-13 17:38:04.447001 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447030 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2659 2014-08-13 17:38:04.447039 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447042 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447048 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447058 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447056 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.128013 9 0.003443 2014-08-13 17:38:04.447062 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.447069 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447073 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.447087 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447082 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.447090 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.447094 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447098 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.447107 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.447107 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116799 9 0.004248 2014-08-13 17:38:04.447119 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447121 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:176851 2014-08-13 17:38:04.447129 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447130 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447133 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447138 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447148 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447144 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.134850 9 0.004567 2014-08-13 17:38:04.447155 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447165 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447152 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 370 notify: (query_epoch:194486, epoch_sent:194486, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.447169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447172 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring 2014-08-13 17:38:04.447183 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447179 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.156330 9 0.008452 2014-08-13 17:38:04.447186 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 370 notify: (query_epoch:194486, epoch_sent:194486, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:04.447190 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] state: Active: got notify from 370, already have info from that osd, ignoring 2014-08-13 17:38:04.447206 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447207 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447211 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447213 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.447221 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116053 9 0.006157 2014-08-13 17:38:04.447225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.447232 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447242 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447235 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.447243 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.56 has 32 missing 2014-08-13 17:38:04.447246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.447252 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.353 has 32 missing 2014-08-13 17:38:04.447258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.447262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.447266 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.447276 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] needs_recovery osd.341 has 40 missing 2014-08-13 17:38:04.447284 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.447370 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2722 2014-08-13 17:38:04.447379 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447387 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447396 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447400 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.447409 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.447417 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.447419 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2689 2014-08-13 17:38:04.447425 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.58 has 23 missing 2014-08-13 17:38:04.447429 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447435 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.343 has 23 missing 2014-08-13 17:38:04.447438 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447447 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447444 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.447451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.447462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.447470 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.447477 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.447485 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.447493 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.447505 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2650 2014-08-13 17:38:04.447514 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447522 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447525 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2682 2014-08-13 17:38:04.447534 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447534 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.447538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447543 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.447557 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.447549 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.106604 9 0.002738 2014-08-13 17:38:04.447561 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.447563 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447574 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447588 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.447600 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116562 9 0.000788 2014-08-13 17:38:04.447600 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447613 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.155166 9 0.010200 2014-08-13 17:38:04.447615 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.447624 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447635 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447629 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.447640 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.447643 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000029 0 0.000000 2014-08-13 17:38:04.447650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.109049 9 0.002770 2014-08-13 17:38:04.447657 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.447661 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.447672 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.447666 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.448890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.448890 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.448914 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.448916 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 17:38:04.448937 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.448942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194479-194484/2 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 17:38:04.448946 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.44 has 24 missing 2014-08-13 17:38:04.448954 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194482-194484/1 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 17:38:04.448956 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.362 has 24 missing 2014-08-13 17:38:04.448965 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.448966 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.448979 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:04.448997 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing 2014-08-13 17:38:04.449011 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2904 2014-08-13 17:38:04.449047 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449050 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2684 2014-08-13 17:38:04.449059 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.449067 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.449065 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.449076 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.449079 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.449081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 LocalRecoveryReserved 2014-08-13 17:38:04.449088 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.449093 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.002024 1 0.000032 2014-08-13 17:38:04.449097 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] needs_recovery osd.376 has 28 missing 2014-08-13 17:38:04.449106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.449106 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved 2014-08-13 17:38:04.449140 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.449182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.449199 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.051432 9 0.003634 2014-08-13 17:38:04.449194 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.449207 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2595 2014-08-13 17:38:04.449224 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.449246 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.449231 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 2014-08-13 17:38:04.449237 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.449250 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 2014-08-13 17:38:04.449254 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.449253 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.449284 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449284 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x5250580 don't know epoch, doing nothing 2014-08-13 17:38:04.449284 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.040892 9 0.000639 2014-08-13 17:38:04.449295 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.449305 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.449315 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000020 0 0.000000 2014-08-13 17:38:04.449313 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,28,304 -> 4,28,304 2014-08-13 17:38:04.449327 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.449316 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192967) 2014-08-13 17:38:04.449327 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.449331 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 2014-08-13 17:38:04.449331 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2905 2014-08-13 17:38:04.449337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.449346 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.449357 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 17:38:04.449369 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194479-194484/2 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 17:38:04.449362 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194477'0 2014-08-13 17:38:04.449379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194482-194484/1 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 17:38:04.449390 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.449388 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) 2014-08-13 17:38:04.449400 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 2014-08-13 17:38:04.449403 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:04.449420 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449512 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 2014-08-13 17:38:04.449510 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.449526 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.449535 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.449535 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2651 2014-08-13 17:38:04.449545 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.449554 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.449562 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.449575 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.449576 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2997 2014-08-13 17:38:04.449585 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.449593 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.449602 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449606 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.449616 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.449624 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.449620 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.449633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.449637 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.067859 9 0.003059 2014-08-13 17:38:04.449642 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.449650 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.449650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.449665 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.449664 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2704 2014-08-13 17:38:04.449672 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.449680 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.449688 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449692 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.449705 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] requeue_ops 2014-08-13 17:38:04.449716 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449715 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.449721 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.449728 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] purge_strays 10 2014-08-13 17:38:04.449732 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.449738 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.10 2014-08-13 17:38:04.449740 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.449750 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:04.449749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.62 has 29 missing 2014-08-13 17:38:04.449758 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.65 has 29 missing 2014-08-13 17:38:04.449766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.449769 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,10,286,343 -> 4,286,343 2014-08-13 17:38:04.449781 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.449784 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2652 2014-08-13 17:38:04.449869 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2734 2014-08-13 17:38:04.449879 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.449887 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.449895 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.449899 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.449909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.449917 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.449924 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.5 has 35 missing 2014-08-13 17:38:04.449933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.47 has 35 missing 2014-08-13 17:38:04.449941 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.450100 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:3007 2014-08-13 17:38:04.450116 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.450130 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.450144 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.450177 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.111904 9 0.007559 2014-08-13 17:38:04.450204 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.450220 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.450235 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.145491 7 0.007806 2014-08-13 17:38:04.450245 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.450254 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.450267 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000022 0 0.000000 2014-08-13 17:38:04.450277 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.450285 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.450293 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.450303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:04.450314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194479-194484/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:04.450323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194482-194484/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:04.450333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.450344 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.450360 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:176852 2014-08-13 17:38:04.450397 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450465 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.450496 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.096489 8 0.004720 2014-08-13 17:38:04.450527 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.450538 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.450553 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.169019 9 0.003056 2014-08-13 17:38:04.450563 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.450572 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450576 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.450566 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.450602 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 2014-08-13 17:38:04.450617 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 2014-08-13 17:38:04.450636 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:04.450622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.450643 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.037137 13 0.002731 2014-08-13 17:38:04.450661 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.450667 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 2014-08-13 17:38:04.450684 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.450688 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4ecee00 don't know epoch, doing nothing 2014-08-13 17:38:04.450720 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 2014-08-13 17:38:04.450730 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.450733 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:176853 2014-08-13 17:38:04.450728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.450749 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.041484 9 0.000729 2014-08-13 17:38:04.450761 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.450772 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.450782 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000021 0 0.000000 2014-08-13 17:38:04.450793 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.450820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.450834 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.450854 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 17:38:04.450874 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194479-194484/2 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 17:38:04.450891 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194482-194484/1 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 17:38:04.450909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.450930 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.450975 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing 2014-08-13 17:38:04.450999 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2998 2014-08-13 17:38:04.451060 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.451075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.451088 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.086571 9 0.000770 2014-08-13 17:38:04.451099 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.451109 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.451119 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000020 0 0.000000 2014-08-13 17:38:04.451143 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.451158 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.451171 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.451207 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 17:38:04.451219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194479-194484/2 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 17:38:04.451229 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194482-194484/1 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 17:38:04.451239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.451235 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.451251 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing 2014-08-13 17:38:04.451250 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 2014-08-13 17:38:04.451274 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 2014-08-13 17:38:04.451292 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 2014-08-13 17:38:04.451333 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2705 2014-08-13 17:38:04.451364 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.451422 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x5250580 don't know epoch, doing nothing 2014-08-13 17:38:04.451419 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.451444 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.147013 8 0.010107 2014-08-13 17:38:04.451471 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.451483 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.451472 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,340,347 -> 4,340,347 2014-08-13 17:38:04.451485 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.451487 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.451489 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2999 2014-08-13 17:38:04.451498 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.101572 9 0.010520 2014-08-13 17:38:04.451508 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.451533 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.451730 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.451759 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] purge_strays 32 2014-08-13 17:38:04.451767 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.32 2014-08-13 17:38:04.451777 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4d33180 don't know epoch, doing nothing 2014-08-13 17:38:04.451794 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,32,50,289 -> 4,50,289 2014-08-13 17:38:04.451824 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.451830 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2706 2014-08-13 17:38:04.466713 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.466716 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.466717 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5143640 pg_info(1 pgs e194486:1.2b) v4 2014-08-13 17:38:04.466742 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:1.2b) v4 from osd.340 2014-08-13 17:38:04.466745 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5143640 2014-08-13 17:38:04.466777 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.466779 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.466805 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) 2014-08-13 17:38:04.466853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 17:38:04.466871 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.466885 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.466908 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.466943 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:177954 2014-08-13 17:38:04.466973 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.467088 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.467127 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.467143 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.467159 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.467184 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 17:38:04.467198 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 17:38:04.467229 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:177955 2014-08-13 17:38:04.467246 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.467261 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.467277 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.467289 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered 2014-08-13 17:38:04.467307 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.147102 7 0.005614 2014-08-13 17:38:04.467328 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 17:38:04.467345 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 17:38:04.467362 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000035 0 0.000000 2014-08-13 17:38:04.467381 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 17:38:04.467396 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 17:38:04.467409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 17:38:04.467427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 17:38:04.467447 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194479-194484/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 17:38:04.467463 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194482-194484/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 17:38:04.467480 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 17:38:04.467501 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:04.467532 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:177956 2014-08-13 17:38:04.467598 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.467797 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 17:38:04.467818 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 2014-08-13 17:38:04.467843 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 2014-08-13 17:38:04.467864 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:04.467901 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 2014-08-13 17:38:04.467919 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4ecee00 don't know epoch, doing nothing 2014-08-13 17:38:04.467952 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 2014-08-13 17:38:04.467962 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.467966 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:177957 2014-08-13 17:38:04.481029 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.58 now has 194486 2014-08-13 17:38:04.481036 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e1080 already has epoch 194486 2014-08-13 17:38:04.481066 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e1080 already has epoch 194486 2014-08-13 17:38:04.489064 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489068 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489069 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dad00 osd_map(194479..194486 src has 185139..194486) v3 2014-08-13 17:38:04.489076 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194479,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.489079 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.489080 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489161 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489164 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489165 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f03800 pg_notify(24.181(40) epoch 194486) v5 2014-08-13 17:38:04.489202 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.307 2014-08-13 17:38:04.489204 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f03800 2014-08-13 17:38:04.489238 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489240 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 307 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.489266 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489268 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489269 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f03b80 pg_notify(24.181(40) epoch 194486) v5 2014-08-13 17:38:04.489265 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489290 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.307 2014-08-13 17:38:04.489292 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f03b80 2014-08-13 17:38:04.489286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.307 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489301 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 -> 4,55,296,307,320,344 2014-08-13 17:38:04.489313 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update 2014-08-13 17:38:04.489318 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.123685 9 0.000691 2014-08-13 17:38:04.489331 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog 2014-08-13 17:38:04.489346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.4 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489358 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489370 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489381 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489403 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting newest update on osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) shard 344 (up) backfill 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) osd.296 (up) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) shard 320 (stray) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:04.489448 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] actingbackfill is 4,296,320,344 2014-08-13 17:38:04.489459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] choose_acting want [4,296,320] (== acting) backfill_targets 344 2014-08-13 17:38:04.489471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: leaving GetLog 2014-08-13 17:38:04.489485 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000154 0 0.000000 2014-08-13 17:38:04.489498 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing 2014-08-13 17:38:04.489508 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.296 has no missing, identical log 2014-08-13 17:38:04.489519 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.320 has no missing, identical log 2014-08-13 17:38:04.489529 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.344 will fully backfill; can infer empty missing set 2014-08-13 17:38:04.489539 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: still need up_thru update before going active 2014-08-13 17:38:04.489550 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000052 0 0.000000 2014-08-13 17:38:04.489562 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru 2014-08-13 17:38:04.489574 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.489597 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.489660 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489662 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489682 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489684 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489685 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d92880 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.489690 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.489693 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.489695 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489696 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489677 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 307 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) 2014-08-13 17:38:04.489703 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.307 2014-08-13 17:38:04.489711 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489724 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489726 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cd240 pg_notify(22.619(31) epoch 194486) v5 2014-08-13 17:38:04.489721 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.307 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours 2014-08-13 17:38:04.489752 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.489753 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.48 2014-08-13 17:38:04.489755 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cd240 2014-08-13 17:38:04.489761 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.489762 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] flushed 2014-08-13 17:38:04.489784 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.489786 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.489808 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 48 notify: (query_epoch:194486, epoch_sent:194486, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.489832 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring 2014-08-13 17:38:04.489843 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.489849 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt 2014-08-13 17:38:04.489864 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] requeue_ops 2014-08-13 17:38:04.489875 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.489879 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 2014-08-13 17:38:04.493717 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.493723 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.493725 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d957c0 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.493735 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.493740 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.493743 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.493744 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.493790 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.493792 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.493794 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823f00 pg_notify(22.481(46) epoch 194486) v5 2014-08-13 17:38:04.493840 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.287 2014-08-13 17:38:04.493842 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823f00 2014-08-13 17:38:04.493872 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.493874 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.493873 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 287 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) 2014-08-13 17:38:04.493897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 287, already have info from that osd, ignoring 2014-08-13 17:38:04.493909 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:04.495416 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.495421 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.495424 7fe9a63f4700 20 osd.4 194486 _dispatch 0x42655c0 pg_info(1 pgs e194486:22.481) v4 2014-08-13 17:38:04.495482 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.481) v4 from osd.287 2014-08-13 17:38:04.495487 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x42655c0 2014-08-13 17:38:04.495547 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.495549 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.495572 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 287 info: 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) 2014-08-13 17:38:04.495596 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.287 activated and committed 2014-08-13 17:38:04.495607 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 17:38:04.495616 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 17:38:04.495630 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df3700 don't know epoch, doing nothing 2014-08-13 17:38:04.495652 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing 2014-08-13 17:38:04.495667 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2682 2014-08-13 17:38:04.495685 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.495854 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated 2014-08-13 17:38:04.495901 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 17:38:04.495918 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 17:38:04.495933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] needs_recovery osd.309 has 32 missing 2014-08-13 17:38:04.495949 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:04.496114 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2683 2014-08-13 17:38:04.496135 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 17:38:04.496150 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 17:38:04.496166 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.496180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery 2014-08-13 17:38:04.496213 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.102204 12 0.012402 2014-08-13 17:38:04.496226 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 17:38:04.496238 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.639509 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.639514 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.639516 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d2d800 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.639605 7fe9a63f4700 10 osd.4 194486 handle_replica_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194486 2014-08-13 17:38:04.649942 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4d2d800 2014-08-13 17:38:04.649951 7fe9a63f4700 20 osd.4 194486 _share_map_incoming osd.337 192.168.1.104:6804/62197 194486 2014-08-13 17:38:04.649969 7fe9a63f4700 15 osd.4 194486 enqueue_op 0x27cb660 prio 127 cost 0 latency 0.010697 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.650007 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.650010 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.650081 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27cb660 prio 127 cost 0 latency 0.010809 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] 2014-08-13 17:38:04.650147 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_message: 0x27cb660 2014-08-13 17:38:04.650193 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.650215 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] sub_op_scrub_reserve 2014-08-13 17:38:04.650245 7fe99f5e9700 20 osd.4 194486 inc_scrubs_pending 0 -> 1 (max 1, active 0) 2014-08-13 17:38:04.650285 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27cb660 finish 2014-08-13 17:38:04.651283 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.304 now has 194486 2014-08-13 17:38:04.651332 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 2014-08-13 17:38:04.651425 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 2014-08-13 17:38:04.657395 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.30 now has 194486 2014-08-13 17:38:04.657408 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4a44780 already has epoch 194486 2014-08-13 17:38:04.657486 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4a44780 already has epoch 194486 2014-08-13 17:38:04.664327 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.664332 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.664334 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5490240 osd_map(194486..194486 src has 185139..194486) v3 2014-08-13 17:38:04.664345 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] 2014-08-13 17:38:04.664351 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping 2014-08-13 17:38:04.664354 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.664355 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.664441 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.664444 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.664446 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5142d80 pg_notify(22.775(31) epoch 194486) v5 2014-08-13 17:38:04.664490 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.0 2014-08-13 17:38:04.664494 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5142d80 2014-08-13 17:38:04.664555 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.664558 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.664558 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 0 notify: (query_epoch:194486, epoch_sent:194486, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:04.664603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-13 17:38:04.664624 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.669361 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.669366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.669369 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db60c0 replica scrub(pg: 22.4fe,from:0'0,to:194215'492,epoch:194486,start:0//0//-1,end:77e26cfe//0//-1,chunky:1,deep:0,version:5) v5 2014-08-13 17:38:04.669383 7fe9a63f4700 10 osd.4 194486 queueing MOSDRepScrub replica scrub(pg: 22.4fe,from:0'0,to:194215'492,epoch:194486,start:0//0//-1,end:77e26cfe//0//-1,chunky:1,deep:0,version:5) v5 2014-08-13 17:38:04.669395 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.669398 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.669425 7fe99d7e6700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] replica_scrub 2014-08-13 17:38:04.669481 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map 2014-08-13 17:38:04.669498 7fe99d7e6700 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] scrub_map_chunk [0//0//-1,77e26cfe//0//-1) 2014-08-13 17:38:04.670389 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] _scan_list scanning 25 objects 2014-08-13 17:38:04.676259 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.351 now has 194486 2014-08-13 17:38:04.676269 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x5250b00 already has epoch 194486 2014-08-13 17:38:04.676317 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x5250b00 already has epoch 194486 2014-08-13 17:38:04.696545 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.364 now has 194486 2014-08-13 17:38:04.696557 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 already has epoch 194486 2014-08-13 17:38:04.696605 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 already has epoch 194486 2014-08-13 17:38:04.766103 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map_chunk done. 2014-08-13 17:38:04.794997 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.795002 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.795004 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d90b40 replica scrub(pg: 22.4fe,from:0'0,to:194215'495,epoch:194486,start:77e26cfe//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 2014-08-13 17:38:04.795013 7fe9a63f4700 10 osd.4 194486 queueing MOSDRepScrub replica scrub(pg: 22.4fe,from:0'0,to:194215'495,epoch:194486,start:77e26cfe//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 2014-08-13 17:38:04.795024 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.795026 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.795080 7fe99d7e6700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] replica_scrub 2014-08-13 17:38:04.795120 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map 2014-08-13 17:38:04.795135 7fe99d7e6700 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] scrub_map_chunk [77e26cfe//0//-1,MAX) 2014-08-13 17:38:04.795864 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] _scan_list scanning 10 objects 2014-08-13 17:38:04.837718 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map_chunk done. 2014-08-13 17:38:04.841572 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.841578 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.841580 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5cd1600 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.841648 7fe9a63f4700 10 osd.4 194486 handle_replica_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194486 2014-08-13 17:38:04.841656 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5cd1600 2014-08-13 17:38:04.841661 7fe9a63f4700 20 osd.4 194486 _share_map_incoming osd.337 192.168.1.104:6804/62197 194486 2014-08-13 17:38:04.841671 7fe9a63f4700 15 osd.4 194486 enqueue_op 0x27ce450 prio 127 cost 0 latency 0.000379 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.841697 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.841700 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.841720 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.841722 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.841724 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.4fe) v4 2014-08-13 17:38:04.841759 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe) v4 from osd.337 2014-08-13 17:38:04.841764 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 2014-08-13 17:38:04.841767 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27ce450 prio 127 cost 0 latency 0.000474 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] 2014-08-13 17:38:04.841838 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_message: 0x27ce450 2014-08-13 17:38:04.841859 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 17:38:04.841879 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] sub_op_scrub_unreserve 2014-08-13 17:38:04.841907 7fe99f5e9700 20 osd.4 194486 dec_scrubs_pending 1 -> 0 (max 1, active 0) 2014-08-13 17:38:04.841911 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27ce450 finish 2014-08-13 17:38:04.842015 7fe9a63f4700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:04.842019 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:04.842035 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) 2014-08-13 17:38:04.842119 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:04.932999 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.334 now has 194486 2014-08-13 17:38:04.933010 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 already has epoch 194486 2014-08-13 17:38:04.933138 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 already has epoch 194486 2014-08-13 17:38:04.954403 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.51 now has 194486 2014-08-13 17:38:04.954414 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 already has epoch 194486 2014-08-13 17:38:04.954473 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 already has epoch 194486 2014-08-13 17:38:04.967175 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.31 now has 194486 2014-08-13 17:38:04.967189 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e5960 already has epoch 194486 2014-08-13 17:38:04.967604 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e5960 already has epoch 194486 2014-08-13 17:38:04.992034 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.397 now has 194486 2014-08-13 17:38:04.992044 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e48840 already has epoch 194486 2014-08-13 17:38:04.992090 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e48840 already has epoch 194486 2014-08-13 17:38:04.995983 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:04.996041 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 2014-08-13 17:38:05.018013 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.358 now has 194486 2014-08-13 17:38:05.018023 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 already has epoch 194486 2014-08-13 17:38:05.018124 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 already has epoch 194486 2014-08-13 17:38:05.148168 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.330 now has 194486 2014-08-13 17:38:05.148182 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 already has epoch 194486 2014-08-13 17:38:05.148285 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 already has epoch 194486 2014-08-13 17:38:05.153225 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 2014-08-13 17:38:05.153411 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 2014-08-13 17:38:05.155142 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.19 now has 194486 2014-08-13 17:38:05.155155 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 already has epoch 194486 2014-08-13 17:38:05.155191 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 already has epoch 194486 2014-08-13 17:38:05.313975 7fe9a77f6700 10 osd.4 194486 do_waiters -- start 2014-08-13 17:38:05.313980 7fe9a77f6700 10 osd.4 194486 do_waiters -- finish 2014-08-13 17:38:05.313982 7fe9a77f6700 20 osd.4 194486 _dispatch 0x5485e80 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.314025 7fe9a77f6700 3 osd.4 194486 handle_osd_map epochs [194487,194487], i have 194486, src has [185139,194487] 2014-08-13 17:38:05.314033 7fe9a77f6700 10 osd.4 194486 handle_osd_map got inc map for epoch 194487 2014-08-13 17:38:05.318488 7fe9a77f6700 10 osd.4 194486 advance to epoch 194487 (<= newest 194487) 2014-08-13 17:38:05.318545 7fe9a77f6700 7 osd.4 194487 advance_map epoch 194487 74 pgs 2014-08-13 17:38:05.318565 7fe9a77f6700 10 osd.4 194487 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194487 [185139,194487] lci=[194485,194487]) 2014-08-13 17:38:05.318617 7fe9a77f6700 0 osd.4 194487 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 17:38:05.318624 7fe9a77f6700 7 osd.4 194487 consume_map version 194487 2014-08-13 17:38:05.318667 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] null 2014-08-13 17:38:05.318701 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] null 2014-08-13 17:38:05.318718 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318731 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] null 2014-08-13 17:38:05.318745 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] null 2014-08-13 17:38:05.318759 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318771 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318768 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 2014-08-13 17:38:05.318786 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] null 2014-08-13 17:38:05.318775 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 17:38:05.318802 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.318802 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.318799 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] null 2014-08-13 17:38:05.318810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.318810 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Peering advmap 2014-08-13 17:38:05.318815 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] null 2014-08-13 17:38:05.318825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.318825 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.318827 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318839 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.318840 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] null 2014-08-13 17:38:05.318839 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Started advmap 2014-08-13 17:38:05.318851 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.318854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.318854 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] null 2014-08-13 17:38:05.318862 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.318868 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map 2014-08-13 17:38:05.318870 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] null 2014-08-13 17:38:05.318875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.318888 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.318883 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318882 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle ActMap primary 2014-08-13 17:38:05.318896 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] null 2014-08-13 17:38:05.318897 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194487:30296 2014-08-13 17:38:05.318908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] take_waiters 2014-08-13 17:38:05.318921 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 0.905872 4 0.000671 2014-08-13 17:38:05.318924 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.318932 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 17:38:05.318936 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Leaving Peering 2014-08-13 17:38:05.318938 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.318946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.318947 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 1.039531 0 0.000000 2014-08-13 17:38:05.318953 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.318952 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:05.318965 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.318965 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active 2014-08-13 17:38:05.318966 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 17:38:05.318976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.318977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: In Active, about to call activate 2014-08-13 17:38:05.318981 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.318986 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.318994 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.318997 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.318993 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] null 2014-08-13 17:38:05.319007 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.319006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - snap_trimq [] 2014-08-13 17:38:05.319019 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.319011 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - not complete, missing(56) 2014-08-13 17:38:05.319025 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.319033 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 17:38:05.319037 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319046 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.319049 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.319061 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] null 2014-08-13 17:38:05.319062 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.319073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,324) went down 2014-08-13 17:38:05.319076 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 17:38:05.319084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.319090 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] null 2014-08-13 17:38:05.319094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.319103 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319114 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] null 2014-08-13 17:38:05.319125 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] null 2014-08-13 17:38:05.319139 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] null 2014-08-13 17:38:05.319150 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319161 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 17:38:05.319172 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] null 2014-08-13 17:38:05.319183 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319199 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 17:38:05.319211 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319222 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] null 2014-08-13 17:38:05.319233 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] null 2014-08-13 17:38:05.319248 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null 2014-08-13 17:38:05.319261 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] null 2014-08-13 17:38:05.319272 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 17:38:05.319284 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] null 2014-08-13 17:38:05.319297 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] null 2014-08-13 17:38:05.319309 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] null 2014-08-13 17:38:05.319320 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] null 2014-08-13 17:38:05.319331 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] null 2014-08-13 17:38:05.319338 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.319347 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] null 2014-08-13 17:38:05.319348 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.319359 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] null 2014-08-13 17:38:05.319370 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.319381 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] null 2014-08-13 17:38:05.319393 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] null 2014-08-13 17:38:05.319405 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 17:38:05.319416 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:05.319428 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] null 2014-08-13 17:38:05.319439 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319450 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319461 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319465 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2832 2014-08-13 17:38:05.319475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.319476 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] null 2014-08-13 17:38:05.319484 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.319489 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 17:38:05.319494 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.319505 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.319501 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] null 2014-08-13 17:38:05.319514 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null 2014-08-13 17:38:05.319520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 17:38:05.319533 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.319527 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.319536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.319539 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] null 2014-08-13 17:38:05.319547 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.319551 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 17:38:05.319558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.319564 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] null 2014-08-13 17:38:05.319569 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.319576 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 17:38:05.319579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.319588 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] null 2014-08-13 17:38:05.319601 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 17:38:05.319612 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] null 2014-08-13 17:38:05.319624 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] null 2014-08-13 17:38:05.319635 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 17:38:05.319647 7fe9a77f6700 10 osd.4 194487 maybe_update_heartbeat_peers updating 2014-08-13 17:38:05.319649 7fe9a77f6700 20 osd.4 194487 24.30b heartbeat_peers 4,10,286,301,375 2014-08-13 17:38:05.319654 7fe9a77f6700 20 osd.4 194487 22.220 heartbeat_peers 2014-08-13 17:38:05.319657 7fe9a77f6700 20 osd.4 194487 22.362 heartbeat_peers 4,58,61,343,349,362 2014-08-13 17:38:05.319663 7fe9a77f6700 20 osd.4 194487 22.28f heartbeat_peers 2014-08-13 17:38:05.319664 7fe9a77f6700 20 osd.4 194487 22.14a heartbeat_peers 2014-08-13 17:38:05.319665 7fe9a77f6700 20 osd.4 194487 22.22c heartbeat_peers 4,301,302,324,368 2014-08-13 17:38:05.319669 7fe9a77f6700 20 osd.4 194487 22.619 heartbeat_peers 4,44,48,62,65,314 2014-08-13 17:38:05.319673 7fe9a77f6700 20 osd.4 194487 22.30f heartbeat_peers 2014-08-13 17:38:05.319674 7fe9a77f6700 20 osd.4 194487 24.9b heartbeat_peers 4,291,350,365,367,378 2014-08-13 17:38:05.319679 7fe9a77f6700 20 osd.4 194487 22.581 heartbeat_peers 2014-08-13 17:38:05.319680 7fe9a77f6700 20 osd.4 194487 22.f3 heartbeat_peers 4,285,311,312,369 2014-08-13 17:38:05.319683 7fe9a77f6700 20 osd.4 194487 22.116 heartbeat_peers 2014-08-13 17:38:05.319684 7fe9a77f6700 20 osd.4 194487 24.3d heartbeat_peers 4,20,25,303,362,373 2014-08-13 17:38:05.319689 7fe9a77f6700 20 osd.4 194487 22.4aa heartbeat_peers 2014-08-13 17:38:05.319690 7fe9a77f6700 20 osd.4 194487 22.1d2 heartbeat_peers 4,58,61,330,331,349 2014-08-13 17:38:05.319693 7fe9a77f6700 20 osd.4 194487 24.1de heartbeat_peers 2014-08-13 17:38:05.319695 7fe9a77f6700 20 osd.4 194487 2.2a heartbeat_peers 4,340 2014-08-13 17:38:05.319697 7fe9a77f6700 20 osd.4 194487 22.113 heartbeat_peers 4,24,45,55,339,342 2014-08-13 17:38:05.319701 7fe9a77f6700 20 osd.4 194487 24.39a heartbeat_peers 2014-08-13 17:38:05.319702 7fe9a77f6700 20 osd.4 194487 24.33a heartbeat_peers 2014-08-13 17:38:05.319703 7fe9a77f6700 20 osd.4 194487 1.2b heartbeat_peers 4,340 2014-08-13 17:38:05.319705 7fe9a77f6700 20 osd.4 194487 24.34 heartbeat_peers 4,26,30,340 2014-08-13 17:38:05.319755 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.319757 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.26 192.168.1.115:6803/58522 192.168.100.115:6804/58522 2014-08-13 17:38:05.319763 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.319789 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.30 192.168.1.107:6824/55491 192.168.100.107:6841/55491 2014-08-13 17:38:05.319797 7fe9a77f6700 20 osd.4 194487 22.99 heartbeat_peers 4,20,31,337,363,374 2014-08-13 17:38:05.319801 7fe9a77f6700 20 osd.4 194487 22.84 heartbeat_peers 4,28,304 2014-08-13 17:38:05.319803 7fe9a77f6700 20 osd.4 194487 22.4df heartbeat_peers 4,352,356,357,371,380 2014-08-13 17:38:05.319807 7fe9a77f6700 20 osd.4 194487 22.39d heartbeat_peers 4,320,344,363,366,373 2014-08-13 17:38:05.319810 7fe9a77f6700 20 osd.4 194487 22.2b9 heartbeat_peers 2014-08-13 17:38:05.319812 7fe9a77f6700 20 osd.4 194487 24.2c heartbeat_peers 2014-08-13 17:38:05.319813 7fe9a77f6700 20 osd.4 194487 24.3f7 heartbeat_peers 2014-08-13 17:38:05.319816 7fe9a77f6700 20 osd.4 194487 22.775 heartbeat_peers 0,4,56,63,353,369 2014-08-13 17:38:05.319819 7fe9a77f6700 20 osd.4 194487 22.630 heartbeat_peers 2014-08-13 17:38:05.319821 7fe9a77f6700 20 osd.4 194487 24.e2 heartbeat_peers 2014-08-13 17:38:05.319822 7fe9a77f6700 20 osd.4 194487 22.69f heartbeat_peers 2014-08-13 17:38:05.319823 7fe9a77f6700 20 osd.4 194487 22.3e4 heartbeat_peers 4,45,303,313,318,342 2014-08-13 17:38:05.319826 7fe9a77f6700 20 osd.4 194487 24.2a1 heartbeat_peers 2014-08-13 17:38:05.319828 7fe9a77f6700 20 osd.4 194487 22.639 heartbeat_peers 2014-08-13 17:38:05.319829 7fe9a77f6700 20 osd.4 194487 22.707 heartbeat_peers 4,15,21,303,310,342 2014-08-13 17:38:05.319832 7fe9a77f6700 20 osd.4 194487 24.5e heartbeat_peers 2014-08-13 17:38:05.319833 7fe9a77f6700 20 osd.4 194487 22.1f1 heartbeat_peers 4,5,11,47,54,368 2014-08-13 17:38:05.319836 7fe9a77f6700 20 osd.4 194487 22.2d1 heartbeat_peers 2014-08-13 17:38:05.319838 7fe9a77f6700 20 osd.4 194487 24.b6 heartbeat_peers 4,17,28,315,327,330 2014-08-13 17:38:05.319841 7fe9a77f6700 20 osd.4 194487 24.3bf heartbeat_peers 2014-08-13 17:38:05.319843 7fe9a77f6700 20 osd.4 194487 22.1fb heartbeat_peers 2014-08-13 17:38:05.319844 7fe9a77f6700 20 osd.4 194487 24.350 heartbeat_peers 2014-08-13 17:38:05.319845 7fe9a77f6700 20 osd.4 194487 24.1f7 heartbeat_peers 4,55,59,351,376 2014-08-13 17:38:05.319849 7fe9a77f6700 20 osd.4 194487 24.136 heartbeat_peers 4,50,55,61,65,356 2014-08-13 17:38:05.319853 7fe9a77f6700 20 osd.4 194487 22.4fe heartbeat_peers 2014-08-13 17:38:05.319854 7fe9a77f6700 20 osd.4 194487 22.653 heartbeat_peers 2014-08-13 17:38:05.319856 7fe9a77f6700 20 osd.4 194487 24.3b7 heartbeat_peers 2014-08-13 17:38:05.319857 7fe9a77f6700 20 osd.4 194487 22.3b9 heartbeat_peers 2014-08-13 17:38:05.319858 7fe9a77f6700 20 osd.4 194487 22.791 heartbeat_peers 2014-08-13 17:38:05.319852 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2735 2014-08-13 17:38:05.319860 7fe9a77f6700 20 osd.4 194487 22.6bd heartbeat_peers 4,50,289 2014-08-13 17:38:05.319862 7fe9a77f6700 20 osd.4 194487 24.181 heartbeat_peers 4,55,296,307,320,344 2014-08-13 17:38:05.319861 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.319870 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.319893 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.319904 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.319942 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.296 192.168.1.109:6807/31286 192.168.100.109:6815/31286 2014-08-13 17:38:05.320002 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.307 192.168.1.109:6805/31016 192.168.100.109:6811/31016 2014-08-13 17:38:05.320013 7fe9a77f6700 20 osd.4 194487 24.20e heartbeat_peers 2014-08-13 17:38:05.320014 7fe9a77f6700 20 osd.4 194487 24.1e4 heartbeat_peers 2014-08-13 17:38:05.320016 7fe9a77f6700 20 osd.4 194487 24.1e6 heartbeat_peers 2014-08-13 17:38:05.320017 7fe9a77f6700 20 osd.4 194487 22.12a heartbeat_peers 2014-08-13 17:38:05.320020 7fe9a77f6700 20 osd.4 194487 22.3af heartbeat_peers 4,44,61,339,362,369 2014-08-13 17:38:05.320023 7fe9a77f6700 20 osd.4 194487 22.481 heartbeat_peers 4,287,303,306,309,365,381 2014-08-13 17:38:05.320027 7fe9a77f6700 20 osd.4 194487 22.48e heartbeat_peers 4,289,327,341,376 2014-08-13 17:38:05.320029 7fe9a77f6700 20 osd.4 194487 22.3ad heartbeat_peers 2014-08-13 17:38:05.320031 7fe9a77f6700 20 osd.4 194487 24.206 heartbeat_peers 2014-08-13 17:38:05.320032 7fe9a77f6700 20 osd.4 194487 24.1bb heartbeat_peers 4,331,332,337,348,379 2014-08-13 17:38:05.320036 7fe9a77f6700 20 osd.4 194487 24.299 heartbeat_peers 2014-08-13 17:38:05.320037 7fe9a77f6700 20 osd.4 194487 22.3d5 heartbeat_peers 4,286,343 2014-08-13 17:38:05.320039 7fe9a77f6700 20 osd.4 194487 22.537 heartbeat_peers 2014-08-13 17:38:05.320041 7fe9a77f6700 20 osd.4 194487 22.1b8 heartbeat_peers 4,340,347 2014-08-13 17:38:05.320043 7fe9a77f6700 20 osd.4 194487 22.29a heartbeat_peers 2014-08-13 17:38:05.320044 7fe9a77f6700 20 osd.4 194487 24.cd heartbeat_peers 2014-08-13 17:38:05.320045 7fe9a77f6700 20 osd.4 194487 22.226 heartbeat_peers 2014-08-13 17:38:05.320047 7fe9a77f6700 20 osd.4 194487 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 2014-08-13 17:38:05.320051 7fe9a77f6700 20 osd.4 194487 24.2ec heartbeat_peers 2014-08-13 17:38:05.320052 7fe9a77f6700 20 osd.4 194487 22.1a0 heartbeat_peers 2014-08-13 17:38:05.320053 7fe9a77f6700 20 osd.4 194487 24.369 heartbeat_peers 2014-08-13 17:38:05.320055 7fe9a77f6700 10 osd.4 194487 adding neighbor peer osd.3 2014-08-13 17:38:05.320056 7fe9a77f6700 10 osd.4 194487 adding neighbor peer osd.5 2014-08-13 17:38:05.320063 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 2014-08-13 17:38:05.320055 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 17:38:05.320080 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.320085 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.320096 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.320122 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.320139 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.320148 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.320159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.320171 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.320193 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 2014-08-13 17:38:05.320215 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.320221 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap 2014-08-13 17:38:05.320233 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.320243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap 2014-08-13 17:38:05.320255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.320266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map 2014-08-13 17:38:05.320277 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary 2014-08-13 17:38:05.320288 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194487:25288 2014-08-13 17:38:05.320298 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters 2014-08-13 17:38:05.320312 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.911686 5 0.000759 2014-08-13 17:38:05.320325 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering 2014-08-13 17:38:05.320335 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.005095 0 0.000000 2014-08-13 17:38:05.320350 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active 2014-08-13 17:38:05.320357 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - starting recovery 2014-08-13 17:38:05.320380 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] 2014-08-13 17:38:05.320382 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 2014-08-13 17:38:05.320416 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.320430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 is up to date, queueing in pending_activators 2014-08-13 17:38:05.320362 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate 2014-08-13 17:38:05.320450 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.320462 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] 2014-08-13 17:38:05.320471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) 2014-08-13 17:38:05.320497 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.320510 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.320540 7fe99f5e9700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 2014-08-13 17:38:05.320723 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 2014-08-13 17:38:05.320891 7fe9a77f6700 10 osd.4 194487 maybe_update_heartbeat_peers 92 peers, extras 3,5,7,32,370 2014-08-13 17:38:05.320897 7fe9a77f6700 7 osd.4 194487 activate_map version 194487 2014-08-13 17:38:05.320913 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 sending log((192718'15573,194422'18660], crt=194422'18660) 2014-08-13 17:38:05.320933 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:05.320944 7fe9986dd700 20 osd.4 194487 agent_entry empty queue 2014-08-13 17:38:05.320951 7fe9a77f6700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.320954 7fe9a77f6700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.320955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.320964 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.320971 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.320973 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db5340 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.320970 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.320982 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.320985 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.320987 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.320988 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.320982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 is up to date, queueing in pending_activators 2014-08-13 17:38:05.321008 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321009 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321010 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e740 pg_notify(22.3af(1) epoch 194487) v5 2014-08-13 17:38:05.321029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.321052 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.61 2014-08-13 17:38:05.321054 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e740 2014-08-13 17:38:05.321085 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321087 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321098 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321100 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321101 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9a400 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.321104 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.321107 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.321108 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321109 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321125 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321126 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321127 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f03640 pg_notify(22.362(1) epoch 194487) v5 2014-08-13 17:38:05.321070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery primary has 56 missing 2014-08-13 17:38:05.321148 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.61 2014-08-13 17:38:05.321150 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f03640 2014-08-13 17:38:05.321172 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321173 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery 2014-08-13 17:38:05.321184 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321185 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321186 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cd5c0 pg_info(1 pgs e194487:24.1e4(20)) v4 2014-08-13 17:38:05.321181 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] 2014-08-13 17:38:05.321142 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.4 2014-08-13 17:38:05.321216 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e4(20)) v4 from osd.31 2014-08-13 17:38:05.321220 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cd5c0 2014-08-13 17:38:05.321210 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.321225 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 is up to date, queueing in pending_activators 2014-08-13 17:38:05.321246 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321247 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321216 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.4 2014-08-13 17:38:05.321283 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.321296 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.321308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 is up to date, queueing in pending_activators 2014-08-13 17:38:05.321285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.4 2014-08-13 17:38:05.321347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.321360 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.321377 7fe99ebe8700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 2014-08-13 17:38:05.321352 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.4 2014-08-13 17:38:05.321420 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.4 2014-08-13 17:38:05.321486 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.4 2014-08-13 17:38:05.321564 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 sending log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 17:38:05.321579 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:05.321599 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.321552 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.4 2014-08-13 17:38:05.321613 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.321629 7fe99ebe8700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 2014-08-13 17:38:05.321618 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.4 2014-08-13 17:38:05.321715 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321717 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321718 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5410000 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.321722 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.321725 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.321726 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321727 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321684 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.4 2014-08-13 17:38:05.321799 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 sending log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 17:38:05.321750 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.4 2014-08-13 17:38:05.321815 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing 2014-08-13 17:38:05.321831 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.321859 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321861 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321863 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f040c0 pg_notify(22.707(33) epoch 194487) v5 2014-08-13 17:38:05.321816 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.4 2014-08-13 17:38:05.321889 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.15 2014-08-13 17:38:05.321891 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f040c0 2014-08-13 17:38:05.321919 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.321920 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.321883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.4 2014-08-13 17:38:05.321894 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing 2014-08-13 17:38:05.321950 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.4 2014-08-13 17:38:05.321969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.4 2014-08-13 17:38:05.322016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.4 2014-08-13 17:38:05.322102 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322104 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322106 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db18c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.322040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.4 2014-08-13 17:38:05.322111 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.322114 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.322116 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322117 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322082 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.4 2014-08-13 17:38:05.322110 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.4 2014-08-13 17:38:05.322148 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.4 2014-08-13 17:38:05.322181 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.4 2014-08-13 17:38:05.322219 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.4 2014-08-13 17:38:05.322257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.4 2014-08-13 17:38:05.322338 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322340 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322341 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58240c0 pg_notify(24.136(31) epoch 194487) v5 2014-08-13 17:38:05.322285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.4 2014-08-13 17:38:05.322365 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.356 2014-08-13 17:38:05.322367 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58240c0 2014-08-13 17:38:05.322326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.4 2014-08-13 17:38:05.322397 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322399 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322352 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.4 2014-08-13 17:38:05.322418 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322419 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322421 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04440 pg_info(1 pgs e194487:24.350(4)) v4 2014-08-13 17:38:05.322442 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.350(4)) v4 from osd.356 2014-08-13 17:38:05.322445 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04440 2014-08-13 17:38:05.322396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.4 2014-08-13 17:38:05.322469 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322471 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.4 2014-08-13 17:38:05.322466 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.4 2014-08-13 17:38:05.322485 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.4 2014-08-13 17:38:05.322536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.4 2014-08-13 17:38:05.322551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.4 2014-08-13 17:38:05.322605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.4 2014-08-13 17:38:05.322617 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.4 2014-08-13 17:38:05.322737 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322739 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322740 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d93cc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.322674 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.4 2014-08-13 17:38:05.322745 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.322683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.4 2014-08-13 17:38:05.322748 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.322749 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322751 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.4 2014-08-13 17:38:05.322744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.4 2014-08-13 17:38:05.322820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.4 2014-08-13 17:38:05.322821 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.4 2014-08-13 17:38:05.322887 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.4 2014-08-13 17:38:05.322892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.4 2014-08-13 17:38:05.322980 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.322982 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.322983 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822680 pg_notify(22.3e4(31) epoch 194487) v5 2014-08-13 17:38:05.323006 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.45 2014-08-13 17:38:05.323007 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822680 2014-08-13 17:38:05.322955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.4 2014-08-13 17:38:05.322962 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.4 2014-08-13 17:38:05.323031 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323034 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.4 2014-08-13 17:38:05.323033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.4 2014-08-13 17:38:05.323092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.4 2014-08-13 17:38:05.323102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.4 2014-08-13 17:38:05.323204 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323206 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323208 7fe9a63f4700 20 osd.4 194487 _dispatch 0x272af40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.323213 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.323215 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.323217 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323218 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323161 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.4 2014-08-13 17:38:05.323247 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323248 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323249 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3840 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.323253 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.323255 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.323256 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323257 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323171 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.4 2014-08-13 17:38:05.323231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.4 2014-08-13 17:38:05.323304 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323305 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323310 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04ec0 pg_info(1 pgs e194487:24.33a(4)) v4 2014-08-13 17:38:05.323263 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.4 2014-08-13 17:38:05.323335 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.33a(4)) v4 from osd.304 2014-08-13 17:38:05.323338 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04ec0 2014-08-13 17:38:05.323362 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.4 2014-08-13 17:38:05.323374 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323376 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323377 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51024c0 pg_info(1 pgs e194487:24.cd(4)) v4 2014-08-13 17:38:05.323397 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.cd(4)) v4 from osd.6 2014-08-13 17:38:05.323399 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51024c0 2014-08-13 17:38:05.323336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.4 2014-08-13 17:38:05.323419 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323421 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323368 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.4 2014-08-13 17:38:05.323406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.4 2014-08-13 17:38:05.323434 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.4 2014-08-13 17:38:05.323530 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323532 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323533 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9ad00 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.323537 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.323540 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.323541 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323542 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323477 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.4 2014-08-13 17:38:05.323553 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323555 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323556 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5438480 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.323559 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.323561 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.323562 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323563 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323500 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.4 2014-08-13 17:38:05.323546 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.4 2014-08-13 17:38:05.323566 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.4 2014-08-13 17:38:05.323670 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323673 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323674 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02680 pg_info(1 pgs e194487:24.5e(4)) v4 2014-08-13 17:38:05.323616 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.4 2014-08-13 17:38:05.323632 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.4 2014-08-13 17:38:05.323701 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.5e(4)) v4 from osd.289 2014-08-13 17:38:05.323704 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02680 2014-08-13 17:38:05.323731 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323733 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323745 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323747 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323748 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f05080 pg_info(1 pgs e194487:24.2a1(4)) v4 2014-08-13 17:38:05.323685 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.4 2014-08-13 17:38:05.323698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.4 2014-08-13 17:38:05.323769 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2a1(4)) v4 from osd.17 2014-08-13 17:38:05.323772 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f05080 2014-08-13 17:38:05.323795 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323796 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323755 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.4 2014-08-13 17:38:05.323765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.4 2014-08-13 17:38:05.323825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.4 2014-08-13 17:38:05.323831 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.4 2014-08-13 17:38:05.323895 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323898 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323899 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5480240 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.323905 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.323908 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.323909 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.323910 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.323898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.4 2014-08-13 17:38:05.323895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.4 2014-08-13 17:38:05.323965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.4 2014-08-13 17:38:05.323965 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.4 2014-08-13 17:38:05.324086 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324089 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324091 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820000 pg_info(1 pgs e194487:24.369(4)) v4 2014-08-13 17:38:05.324032 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.4 2014-08-13 17:38:05.324035 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.4 2014-08-13 17:38:05.324133 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.369(4)) v4 from osd.45 2014-08-13 17:38:05.324136 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820000 2014-08-13 17:38:05.324099 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.4 2014-08-13 17:38:05.324105 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.4 2014-08-13 17:38:05.324207 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324210 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.4 2014-08-13 17:38:05.324237 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324238 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324240 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cc600 pg_notify(24.1bb(35) epoch 194487) v5 2014-08-13 17:38:05.324176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.4 2014-08-13 17:38:05.324263 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.379 2014-08-13 17:38:05.324265 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cc600 2014-08-13 17:38:05.324297 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324298 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324236 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.4 2014-08-13 17:38:05.324252 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.4 2014-08-13 17:38:05.324323 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324325 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324326 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d98fc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.324331 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.324334 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.324336 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324337 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324353 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324354 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324355 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9f80 pg_info(1 pgs e194487:24.1e6(4)) v4 2014-08-13 17:38:05.324303 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.4 2014-08-13 17:38:05.324377 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e6(4)) v4 from osd.285 2014-08-13 17:38:05.324380 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9f80 2014-08-13 17:38:05.324323 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.4 2014-08-13 17:38:05.324402 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324403 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324414 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324415 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324416 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51047c0 pg_notify(24.b6(38) epoch 194487) v5 2014-08-13 17:38:05.324369 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.4 2014-08-13 17:38:05.324452 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.17 2014-08-13 17:38:05.324453 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51047c0 2014-08-13 17:38:05.324393 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.4 2014-08-13 17:38:05.324475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324476 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324436 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.4 2014-08-13 17:38:05.324463 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.4 2014-08-13 17:38:05.324502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.4 2014-08-13 17:38:05.324532 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.4 2014-08-13 17:38:05.324568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.4 2014-08-13 17:38:05.324637 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324639 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324651 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d89d40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.324656 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.324659 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.324660 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.324661 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.324602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.4 2014-08-13 17:38:05.324634 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.4 2014-08-13 17:38:05.324672 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.4 2014-08-13 17:38:05.324700 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.4 2014-08-13 17:38:05.324741 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.4 2014-08-13 17:38:05.324766 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.4 2014-08-13 17:38:05.324810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.4 2014-08-13 17:38:05.324832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.4 2014-08-13 17:38:05.324880 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.4 2014-08-13 17:38:05.324896 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 17:38:05.325009 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325012 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325013 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d998c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.324952 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.4 2014-08-13 17:38:05.325020 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.325023 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.325025 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325026 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] activate: adding 10 as a source 2014-08-13 17:38:05.325022 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.4 2014-08-13 17:38:05.325073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.10 2014-08-13 17:38:05.325092 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.4 2014-08-13 17:38:05.325140 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=55] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.10 2014-08-13 17:38:05.325161 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.4 2014-08-13 17:38:05.325235 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325237 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325238 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104440 pg_notify(22.1d2(27) epoch 194487) v5 2014-08-13 17:38:05.325261 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.330 2014-08-13 17:38:05.325263 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104440 2014-08-13 17:38:05.325214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=54] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.10 2014-08-13 17:38:05.325288 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325290 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.4 2014-08-13 17:38:05.325287 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=53] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.10 2014-08-13 17:38:05.325308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.4 2014-08-13 17:38:05.325359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=52] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.10 2014-08-13 17:38:05.325377 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.4 2014-08-13 17:38:05.325503 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325435 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=51] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.10 2014-08-13 17:38:05.325509 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.4 2014-08-13 17:38:05.325513 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9c00 pg_info(1 pgs e194487:24.3f7(2)) v4 2014-08-13 17:38:05.325539 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3f7(2)) v4 from osd.330 2014-08-13 17:38:05.325542 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9c00 2014-08-13 17:38:05.325570 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325572 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.4 2014-08-13 17:38:05.325585 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325587 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325588 7fe9a63f4700 20 osd.4 194487 _dispatch 0x272b3c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.325512 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=50] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.10 2014-08-13 17:38:05.325593 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.325597 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.325598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325609 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325610 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325611 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c8700 pg_info(1 pgs e194487:24.206(4)) v4 2014-08-13 17:38:05.325632 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.206(4)) v4 from osd.343 2014-08-13 17:38:05.325634 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c8700 2014-08-13 17:38:05.325586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.4 2014-08-13 17:38:05.325663 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325665 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325594 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=49] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.10 2014-08-13 17:38:05.325656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.4 2014-08-13 17:38:05.325732 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325734 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325735 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103b80 pg_info(1 pgs e194487:24.e2(2)) v4 2014-08-13 17:38:05.325674 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=48] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.10 2014-08-13 17:38:05.325757 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.e2(2)) v4 from osd.346 2014-08-13 17:38:05.325760 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103b80 2014-08-13 17:38:05.325782 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325784 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.4 2014-08-13 17:38:05.325756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=47] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.10 2014-08-13 17:38:05.325796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.4 2014-08-13 17:38:05.325840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=46] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.10 2014-08-13 17:38:05.325865 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.4 2014-08-13 17:38:05.325937 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325940 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325941 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04280 pg_info(1 pgs e194487:24.3b7(2)) v4 2014-08-13 17:38:05.325966 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3b7(2)) v4 from osd.356 2014-08-13 17:38:05.325969 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04280 2014-08-13 17:38:05.325995 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.325996 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.325934 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.4 2014-08-13 17:38:05.325926 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=45] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.10 2014-08-13 17:38:05.326004 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.4 2014-08-13 17:38:05.326014 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=44] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.10 2014-08-13 17:38:05.326073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.4 2014-08-13 17:38:05.326103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=43] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.10 2014-08-13 17:38:05.326141 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 17:38:05.326199 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=42] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.10 2014-08-13 17:38:05.326282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 291 as a source 2014-08-13 17:38:05.326295 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=41] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.10 2014-08-13 17:38:05.326355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.291 2014-08-13 17:38:05.326392 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=40] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.10 2014-08-13 17:38:05.326427 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.291 2014-08-13 17:38:05.326502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.291 2014-08-13 17:38:05.326490 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=39] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.10 2014-08-13 17:38:05.326579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.291 2014-08-13 17:38:05.326588 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=38] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.10 2014-08-13 17:38:05.326657 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.291 2014-08-13 17:38:05.326687 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=37] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.10 2014-08-13 17:38:05.326800 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.326803 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.326804 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d98b40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.326810 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.326813 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.326814 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.326815 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.326737 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.291 2014-08-13 17:38:05.326790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=36] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.10 2014-08-13 17:38:05.326817 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.291 2014-08-13 17:38:05.326954 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.326957 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.326958 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_info(1 pgs e194487:24.1de(2)) v4 2014-08-13 17:38:05.326903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.291 2014-08-13 17:38:05.326984 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1de(2)) v4 from osd.364 2014-08-13 17:38:05.326987 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 2014-08-13 17:38:05.326892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=35] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.10 2014-08-13 17:38:05.327016 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.327017 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.326987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.291 2014-08-13 17:38:05.326996 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=34] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.10 2014-08-13 17:38:05.327073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.291 2014-08-13 17:38:05.327103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=33] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.10 2014-08-13 17:38:05.327159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.291 2014-08-13 17:38:05.327214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=32] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.10 2014-08-13 17:38:05.327255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.291 2014-08-13 17:38:05.327326 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=31] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.10 2014-08-13 17:38:05.327347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.291 2014-08-13 17:38:05.327441 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.291 2014-08-13 17:38:05.327440 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=30] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.10 2014-08-13 17:38:05.327538 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.291 2014-08-13 17:38:05.327555 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=29] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.10 2014-08-13 17:38:05.327635 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.291 2014-08-13 17:38:05.327748 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.327750 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.327752 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5460fc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.327757 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.327761 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.327762 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.327763 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.327672 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=28] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.10 2014-08-13 17:38:05.327735 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.291 2014-08-13 17:38:05.327855 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.327857 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.327859 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820540 pg_info(1 pgs e194487:24.2ec(2)) v4 2014-08-13 17:38:05.327885 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2ec(2)) v4 from osd.374 2014-08-13 17:38:05.327888 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820540 2014-08-13 17:38:05.327789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=27] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.10 2014-08-13 17:38:05.327916 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.327918 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.327834 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.291 2014-08-13 17:38:05.327906 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=26] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.10 2014-08-13 17:38:05.327937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.291 2014-08-13 17:38:05.328040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.291 2014-08-13 17:38:05.328028 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=25] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.10 2014-08-13 17:38:05.328146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.291 2014-08-13 17:38:05.328150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=24] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.10 2014-08-13 17:38:05.328258 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.291 2014-08-13 17:38:05.328277 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=23] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.10 2014-08-13 17:38:05.328368 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.291 2014-08-13 17:38:05.328402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=22] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.10 2014-08-13 17:38:05.328478 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.291 2014-08-13 17:38:05.328529 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=21] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.10 2014-08-13 17:38:05.328589 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.291 2014-08-13 17:38:05.328753 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.328757 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.328758 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f03f00 pg_info(1 pgs e194487:24.20e(2)) v4 2014-08-13 17:38:05.328658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=20] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.10 2014-08-13 17:38:05.328790 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.20e(2)) v4 from osd.15 2014-08-13 17:38:05.328793 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f03f00 2014-08-13 17:38:05.328704 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.291 2014-08-13 17:38:05.328825 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.328827 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.328790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=19] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.10 2014-08-13 17:38:05.328820 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.291 2014-08-13 17:38:05.328937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.291 2014-08-13 17:38:05.328925 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=18] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.10 2014-08-13 17:38:05.329057 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.291 2014-08-13 17:38:05.329060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=17] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.10 2014-08-13 17:38:05.329179 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.291 2014-08-13 17:38:05.329201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=16] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.10 2014-08-13 17:38:05.329305 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.291 2014-08-13 17:38:05.329342 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=15] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.10 2014-08-13 17:38:05.329431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.291 2014-08-13 17:38:05.329483 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=14] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.10 2014-08-13 17:38:05.329557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.291 2014-08-13 17:38:05.329626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=13] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.10 2014-08-13 17:38:05.329686 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.291 2014-08-13 17:38:05.329770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=12] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.10 2014-08-13 17:38:05.329837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.291 2014-08-13 17:38:05.329916 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=11] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.10 2014-08-13 17:38:05.329970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.291 2014-08-13 17:38:05.330067 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=10] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.10 2014-08-13 17:38:05.330104 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.291 2014-08-13 17:38:05.330219 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=9] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.10 2014-08-13 17:38:05.330242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.291 2014-08-13 17:38:05.330473 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.330475 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.330476 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54da1c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.330481 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.330484 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.330486 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.330487 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.330379 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.291 2014-08-13 17:38:05.330371 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=8] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.10 2014-08-13 17:38:05.330621 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.330622 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.330623 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f055c0 pg_notify(22.3d9(49) epoch 194487) v5 2014-08-13 17:38:05.330648 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.293 2014-08-13 17:38:05.330650 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f055c0 2014-08-13 17:38:05.330517 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.291 2014-08-13 17:38:05.330523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=7] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.10 2014-08-13 17:38:05.330677 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.330679 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.330658 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.291 2014-08-13 17:38:05.330677 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=6] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.10 2014-08-13 17:38:05.330837 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=5] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.10 2014-08-13 17:38:05.330997 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=4] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.10 2014-08-13 17:38:05.331158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=3] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.10 2014-08-13 17:38:05.330802 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.291 2014-08-13 17:38:05.331328 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.291 2014-08-13 17:38:05.331324 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=2] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.10 2014-08-13 17:38:05.331480 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.291 2014-08-13 17:38:05.331488 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=1] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.10 2014-08-13 17:38:05.331628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.291 2014-08-13 17:38:05.331654 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 17:38:05.331778 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.291 2014-08-13 17:38:05.331860 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 286 as a source 2014-08-13 17:38:05.331929 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.291 2014-08-13 17:38:05.332028 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.291 2014-08-13 17:38:05.332198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332241 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.291 2014-08-13 17:38:05.332365 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332400 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.291 2014-08-13 17:38:05.332532 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.291 2014-08-13 17:38:05.332725 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.332729 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.332731 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d921c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.332736 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.332739 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.332741 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.332742 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.332840 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.332842 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.332843 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f05b00 pg_notify(22.619(31) epoch 194487) v5 2014-08-13 17:38:05.332698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332867 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.314 2014-08-13 17:38:05.332869 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f05b00 2014-08-13 17:38:05.332715 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.291 2014-08-13 17:38:05.332896 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.332898 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.332863 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.332876 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.291 2014-08-13 17:38:05.333130 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333132 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333134 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484800 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.333138 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.333141 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.333142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333038 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.291 2014-08-13 17:38:05.333264 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333266 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333267 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103800 pg_notify(22.39d(29) epoch 194487) v5 2014-08-13 17:38:05.333289 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.320 2014-08-13 17:38:05.333291 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103800 2014-08-13 17:38:05.333316 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333318 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333334 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9d100 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.333339 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.333342 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.333344 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333345 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333197 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333533 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_notify(24.1f7(34) epoch 194487) v5 2014-08-13 17:38:05.333382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333561 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.376 2014-08-13 17:38:05.333563 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 2014-08-13 17:38:05.333611 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.333613 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.333205 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.291 2014-08-13 17:38:05.333548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333643 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.291 2014-08-13 17:38:05.333715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333816 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.291 2014-08-13 17:38:05.333882 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.333987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.291 2014-08-13 17:38:05.334047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334305 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.334307 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.334308 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f063c0 pg_notify(24.181(12) epoch 194487) v5 2014-08-13 17:38:05.334331 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.320 2014-08-13 17:38:05.334333 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f063c0 2014-08-13 17:38:05.334159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.291 2014-08-13 17:38:05.334368 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.334370 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.334215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334390 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.334391 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.334392 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f06c80 pg_notify(22.48e(29) epoch 194487) v5 2014-08-13 17:38:05.334414 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.376 2014-08-13 17:38:05.334416 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f06c80 2014-08-13 17:38:05.334437 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.334438 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.334379 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334337 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 17:38:05.334545 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 350 as a source 2014-08-13 17:38:05.334712 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334737 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.350 2014-08-13 17:38:05.334878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.334912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.350 2014-08-13 17:38:05.335045 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335088 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.350 2014-08-13 17:38:05.335214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335269 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.350 2014-08-13 17:38:05.335519 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.335522 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.335524 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9bcc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.335529 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.335533 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.335534 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.335535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.335381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.335604 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.335605 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce200 pg_notify(24.30b(2) epoch 194487) v5 2014-08-13 17:38:05.335445 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.350 2014-08-13 17:38:05.335630 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.301 2014-08-13 17:38:05.335632 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce200 2014-08-13 17:38:05.335546 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.350 2014-08-13 17:38:05.335710 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.350 2014-08-13 17:38:05.335876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.335972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.350 2014-08-13 17:38:05.336042 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.336148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.350 2014-08-13 17:38:05.336211 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.336331 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.350 2014-08-13 17:38:05.336377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.336508 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.350 2014-08-13 17:38:05.336543 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.336684 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.350 2014-08-13 17:38:05.336708 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.336860 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.350 2014-08-13 17:38:05.336872 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337042 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.350 2014-08-13 17:38:05.337214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.350 2014-08-13 17:38:05.337381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.350 2014-08-13 17:38:05.337547 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337567 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.350 2014-08-13 17:38:05.337713 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.350 2014-08-13 17:38:05.337880 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.337920 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.350 2014-08-13 17:38:05.338046 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338095 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.350 2014-08-13 17:38:05.338215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.350 2014-08-13 17:38:05.338382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.350 2014-08-13 17:38:05.338548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.350 2014-08-13 17:38:05.338714 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.350 2014-08-13 17:38:05.338880 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.338980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.350 2014-08-13 17:38:05.339047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.339156 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.350 2014-08-13 17:38:05.339217 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.339335 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.350 2014-08-13 17:38:05.339383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.339512 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.350 2014-08-13 17:38:05.339549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.339687 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.350 2014-08-13 17:38:05.339715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.339863 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.350 2014-08-13 17:38:05.339883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340039 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.350 2014-08-13 17:38:05.340048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340220 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.350 2014-08-13 17:38:05.340234 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.350 2014-08-13 17:38:05.340406 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340572 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340573 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.350 2014-08-13 17:38:05.340739 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.340748 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.350 2014-08-13 17:38:05.340922 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.350 2014-08-13 17:38:05.340906 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.341096 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.350 2014-08-13 17:38:05.341131 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.341275 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.350 2014-08-13 17:38:05.341404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.341452 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.350 2014-08-13 17:38:05.341574 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 17:38:05.341627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.350 2014-08-13 17:38:05.341780 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 375 as a source 2014-08-13 17:38:05.341802 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.350 2014-08-13 17:38:05.341948 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.375 2014-08-13 17:38:05.341976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.350 2014-08-13 17:38:05.342117 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.375 2014-08-13 17:38:05.342150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.350 2014-08-13 17:38:05.342287 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.375 2014-08-13 17:38:05.342327 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.350 2014-08-13 17:38:05.342453 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.375 2014-08-13 17:38:05.342502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.350 2014-08-13 17:38:05.342619 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.375 2014-08-13 17:38:05.342676 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.350 2014-08-13 17:38:05.342784 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.375 2014-08-13 17:38:05.342850 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.350 2014-08-13 17:38:05.342949 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.375 2014-08-13 17:38:05.343024 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.350 2014-08-13 17:38:05.343114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.375 2014-08-13 17:38:05.343201 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.350 2014-08-13 17:38:05.343283 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.375 2014-08-13 17:38:05.343376 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.350 2014-08-13 17:38:05.343451 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.375 2014-08-13 17:38:05.343552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.350 2014-08-13 17:38:05.343617 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.375 2014-08-13 17:38:05.343729 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.350 2014-08-13 17:38:05.343794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.375 2014-08-13 17:38:05.343903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.350 2014-08-13 17:38:05.343959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.375 2014-08-13 17:38:05.344077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.350 2014-08-13 17:38:05.344124 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.375 2014-08-13 17:38:05.344254 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.350 2014-08-13 17:38:05.344293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.375 2014-08-13 17:38:05.344430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.350 2014-08-13 17:38:05.344458 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.375 2014-08-13 17:38:05.344606 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.350 2014-08-13 17:38:05.344624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.375 2014-08-13 17:38:05.344789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.375 2014-08-13 17:38:05.344782 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.350 2014-08-13 17:38:05.344956 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.375 2014-08-13 17:38:05.344958 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.350 2014-08-13 17:38:05.345123 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.375 2014-08-13 17:38:05.345137 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 17:38:05.345293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.375 2014-08-13 17:38:05.345359 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 367 as a source 2014-08-13 17:38:05.345460 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.375 2014-08-13 17:38:05.345535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.345627 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.375 2014-08-13 17:38:05.345711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.345794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.375 2014-08-13 17:38:05.345887 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.345961 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.375 2014-08-13 17:38:05.346063 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.375 2014-08-13 17:38:05.346242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.375 2014-08-13 17:38:05.346418 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.375 2014-08-13 17:38:05.346593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.375 2014-08-13 17:38:05.346768 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.375 2014-08-13 17:38:05.346945 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.346965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.375 2014-08-13 17:38:05.347120 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.347131 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.375 2014-08-13 17:38:05.347300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.347320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.375 2014-08-13 17:38:05.347476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.347492 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.375 2014-08-13 17:38:05.347658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.375 2014-08-13 17:38:05.347651 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.347825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.347826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.375 2014-08-13 17:38:05.348001 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348111 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.375 2014-08-13 17:38:05.348176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.375 2014-08-13 17:38:05.348354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348456 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.375 2014-08-13 17:38:05.348529 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.375 2014-08-13 17:38:05.348702 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.375 2014-08-13 17:38:05.348892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.348990 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.375 2014-08-13 17:38:05.349067 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.349158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.375 2014-08-13 17:38:05.349330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.375 2014-08-13 17:38:05.349247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.349499 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.375 2014-08-13 17:38:05.349563 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.349666 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.375 2014-08-13 17:38:05.349743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.349839 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.375 2014-08-13 17:38:05.349918 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.375 2014-08-13 17:38:05.350093 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350174 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.375 2014-08-13 17:38:05.350273 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350345 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.375 2014-08-13 17:38:05.350448 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350512 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.375 2014-08-13 17:38:05.350624 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.375 2014-08-13 17:38:05.350799 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.350853 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.375 2014-08-13 17:38:05.350973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351020 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.375 2014-08-13 17:38:05.351148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351189 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.375 2014-08-13 17:38:05.351326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351356 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.375 2014-08-13 17:38:05.351502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351523 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 17:38:05.351677 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound 2014-08-13 17:38:05.351852 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.351896 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.352027 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.352064 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound: built 10,286,301,375 2014-08-13 17:38:05.352204 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.352235 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Activate Finished 2014-08-13 17:38:05.352380 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.352404 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.352555 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.352572 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.352737 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.352730 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.352963 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.353010 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 17:38:05.353028 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353031 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353030 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.353035 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.353049 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353051 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353052 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dda00 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.353058 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353066 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353067 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.353069 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.352903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.353070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.353080 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353082 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353083 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df2c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353086 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353079 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.353089 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353091 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353092 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353090 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.353101 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.353104 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353106 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353107 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484ec0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353110 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353112 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353113 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353114 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353124 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353125 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353126 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3600 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353117 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 17:38:05.353129 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353132 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353132 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.353133 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353135 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.353145 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353146 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353147 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9f500 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353150 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353152 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353153 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353155 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353146 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.353164 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353166 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.353166 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9dc40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353170 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353172 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353173 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353174 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353168 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.353184 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353185 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353186 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92f40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353189 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353182 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.353191 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353197 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353198 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353208 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353209 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353210 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db1200 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353213 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353215 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353216 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353217 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353226 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353227 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353228 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db6780 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353231 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353233 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353234 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353235 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353244 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353245 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353246 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484140 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353249 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353252 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353252 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353253 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.353268 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353270 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353271 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d933c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353273 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353276 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353277 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353278 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353287 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353288 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353289 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d95e80 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353292 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353294 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353295 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353296 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353305 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353306 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353307 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db2f40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353310 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353312 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353313 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353314 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353324 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353325 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353326 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92640 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353329 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353331 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353330 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.353339 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.353350 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353351 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353352 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54d8fc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353355 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353358 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353359 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353360 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353369 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353371 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353371 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db600 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353374 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353377 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353378 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353379 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353388 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353389 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353390 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc800 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353393 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353395 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353396 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353397 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353407 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353408 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353409 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc380 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.353411 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.353414 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.353415 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353416 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353413 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2683 2014-08-13 17:38:05.353427 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353428 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353429 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f071c0 pg_notify(22.775(31) epoch 194487) v5 2014-08-13 17:38:05.353423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.353267 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.353433 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.353443 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.353454 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.63 2014-08-13 17:38:05.353455 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.353456 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f071c0 2014-08-13 17:38:05.353462 7fe99f5e9700 20 osd.4 194487 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 10 2014-08-13 17:38:05.353470 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:05.353483 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353484 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353513 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5827c40 pg_notify(22.3d9(49) epoch 194487) v5 2014-08-13 17:38:05.353512 7fe99f5e9700 20 osd.4 194487 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 375 2014-08-13 17:38:05.353519 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing 2014-08-13 17:38:05.353340 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.353534 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.382 2014-08-13 17:38:05.353536 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5827c40 2014-08-13 17:38:05.353584 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353586 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 2014-08-13 17:38:05.353604 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.353442 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.353608 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Started advmap 2014-08-13 17:38:05.353619 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353621 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353622 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c17000 pg_notify(24.34(39) epoch 194487) v5 2014-08-13 17:38:05.353620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.353631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map 2014-08-13 17:38:05.353644 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.26 2014-08-13 17:38:05.353645 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c17000 2014-08-13 17:38:05.353667 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353668 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353692 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353694 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters 2014-08-13 17:38:05.353695 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f039c0 pg_notify(24.181(40) epoch 194487) v5 2014-08-13 17:38:05.353534 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] flushed 2014-08-13 17:38:05.353697 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.353707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.353719 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.353719 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.307 2014-08-13 17:38:05.353721 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f039c0 2014-08-13 17:38:05.353740 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353742 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353732 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:05.353746 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.353756 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.353757 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353760 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6011180 pg_notify(24.3d(2) epoch 194487) v5 2014-08-13 17:38:05.353761 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.353770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.353782 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.20 2014-08-13 17:38:05.353783 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6011180 2014-08-13 17:38:05.353781 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.353789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.353804 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353799 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.353806 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353618 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.353808 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:176854 2014-08-13 17:38:05.353817 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.353819 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.353820 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010700 pg_notify(22.113(29) epoch 194487) v5 2014-08-13 17:38:05.353817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.353826 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.353840 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 2014-08-13 17:38:05.353835 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.353842 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010700 2014-08-13 17:38:05.353845 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.353858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 17:38:05.353871 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.353874 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.353884 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.353894 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.353903 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.353912 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.353813 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354077 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.354085 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.353990 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354168 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2935 2014-08-13 17:38:05.354183 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.354192 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.354201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.354211 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.354233 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354235 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354233 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 17:38:05.354249 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354250 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354251 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.354252 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4265b00 pg_notify(22.99(27) epoch 194487) v5 2014-08-13 17:38:05.354255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:05.354273 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.337 2014-08-13 17:38:05.354265 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.354274 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4265b00 2014-08-13 17:38:05.354276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:05.354295 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354296 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354309 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354310 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354311 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_notify(24.1bb(2) epoch 194487) v5 2014-08-13 17:38:05.354331 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.332 2014-08-13 17:38:05.354332 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 2014-08-13 17:38:05.354341 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:05.354164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354355 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354356 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354352 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 2014-08-13 17:38:05.354368 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354369 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354362 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.354370 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010c40 pg_notify(22.113(29) epoch 194487) v5 2014-08-13 17:38:05.354373 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.354390 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.24 2014-08-13 17:38:05.354392 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010c40 2014-08-13 17:38:05.354388 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 2014-08-13 17:38:05.354405 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.354416 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Started advmap 2014-08-13 17:38:05.354417 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.354428 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map 2014-08-13 17:38:05.354443 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354444 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354445 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d945c0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.354449 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.354451 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.354453 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354454 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354471 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters 2014-08-13 17:38:05.354480 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354482 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354483 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07540 pg_notify(22.3d9(16) epoch 194487) v5 2014-08-13 17:38:05.354481 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.354491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.354501 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.354506 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.397 2014-08-13 17:38:05.354508 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07540 2014-08-13 17:38:05.354513 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 17:38:05.354525 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.354355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354529 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.354545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354546 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354539 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.354547 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5140000 pg_notify(24.b6(2) epoch 194487) v5 2014-08-13 17:38:05.354548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.354568 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.327 2014-08-13 17:38:05.354562 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.354570 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5140000 2014-08-13 17:38:05.354571 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.354580 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.354591 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:177958 2014-08-13 17:38:05.354593 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354599 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.354606 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354607 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.354617 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354619 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354620 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824d00 pg_notify(24.136(31) epoch 194487) v5 2014-08-13 17:38:05.354615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.354626 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.354640 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.65 2014-08-13 17:38:05.354641 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824d00 2014-08-13 17:38:05.354642 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_advance_map [26,30,340]/[4] -- 26/4 2014-08-13 17:38:05.354659 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.354662 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354663 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354678 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354680 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354681 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822300 pg_notify(22.3af(31) epoch 194487) v5 2014-08-13 17:38:05.354664 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Peering advmap 2014-08-13 17:38:05.354689 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.354701 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.339 2014-08-13 17:38:05.354703 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822300 2014-08-13 17:38:05.354531 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354700 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Started advmap 2014-08-13 17:38:05.354711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.354724 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354725 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354722 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map 2014-08-13 17:38:05.354733 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle ActMap primary 2014-08-13 17:38:05.354746 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354748 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354749 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54d8b40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.354743 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194487:23843 2014-08-13 17:38:05.354753 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.354756 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.354757 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354758 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354753 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] take_waiters 2014-08-13 17:38:05.354768 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354769 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354770 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6011c00 pg_notify(22.619(31) epoch 194487) v5 2014-08-13 17:38:05.354765 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 0.947467 6 0.000876 2014-08-13 17:38:05.354779 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Leaving Peering 2014-08-13 17:38:05.354791 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.48 2014-08-13 17:38:05.354793 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6011c00 2014-08-13 17:38:05.354789 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 1.028897 0 0.000000 2014-08-13 17:38:05.354812 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354814 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354807 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active 2014-08-13 17:38:05.354817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: In Active, about to call activate 2014-08-13 17:38:05.354832 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354833 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354834 7fe9a63f4700 20 osd.4 194487 _dispatch 0x60116c0 pg_notify(22.22c(1) epoch 194487) v5 2014-08-13 17:38:05.354830 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.354842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - snap_trimq [] 2014-08-13 17:38:05.354858 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.302 2014-08-13 17:38:05.354859 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x60116c0 2014-08-13 17:38:05.354852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - no missing, moving last_complete 194215'14514 -> 194215'14514 2014-08-13 17:38:05.354879 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354705 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.354880 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354891 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354892 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354893 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9500 pg_notify(22.4df(27) epoch 194487) v5 2014-08-13 17:38:05.354914 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.371 2014-08-13 17:38:05.354915 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9500 2014-08-13 17:38:05.354920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:05.354935 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354937 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 is up to date, queueing in pending_activators 2014-08-13 17:38:05.354950 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354951 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.354952 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07a80 pg_notify(24.1f7(38) epoch 194487) v5 2014-08-13 17:38:05.354972 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.351 2014-08-13 17:38:05.354973 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07a80 2014-08-13 17:38:05.354992 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.354994 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate 2014-08-13 17:38:05.355020 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355021 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355022 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824980 pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 2014-08-13 17:38:05.355021 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:05.355034 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 is up to date, queueing in pending_activators 2014-08-13 17:38:05.355046 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.369 2014-08-13 17:38:05.355048 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824980 2014-08-13 17:38:05.354881 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355076 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355078 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355083 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate 2014-08-13 17:38:05.355100 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355101 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355102 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54de540 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.355106 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.355096 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) 2014-08-13 17:38:05.355109 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.355110 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355111 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 is up to date, queueing in pending_activators 2014-08-13 17:38:05.355121 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355123 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355123 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010fc0 pg_notify(22.99(27) epoch 194487) v5 2014-08-13 17:38:05.355143 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.20 2014-08-13 17:38:05.355145 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010fc0 2014-08-13 17:38:05.355151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate 2014-08-13 17:38:05.355169 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355171 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] needs_recovery is recovered 2014-08-13 17:38:05.355187 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355188 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355176 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Activate Finished 2014-08-13 17:38:05.355189 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010540 pg_notify(24.1f7(2) epoch 194487) v5 2014-08-13 17:38:05.355193 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.355204 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.355216 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 2014-08-13 17:38:05.355217 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010540 2014-08-13 17:38:05.355214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.355055 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355244 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355246 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355257 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355258 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355259 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264d00 pg_notify(24.1bb(35) epoch 194487) v5 2014-08-13 17:38:05.355282 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.355290 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.337 2014-08-13 17:38:05.355291 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264d00 2014-08-13 17:38:05.355310 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355312 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355312 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 17:38:05.355328 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.355331 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355334 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822bc0 pg_notify(22.707(1) epoch 194487) v5 2014-08-13 17:38:05.355331 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.355343 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.355355 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 2014-08-13 17:38:05.355356 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822bc0 2014-08-13 17:38:05.355354 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.355364 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.355376 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355377 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355374 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.355388 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355389 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355390 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e900 pg_notify(22.48e(29) epoch 194487) v5 2014-08-13 17:38:05.355410 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.327 2014-08-13 17:38:05.355411 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e900 2014-08-13 17:38:05.355234 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355430 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355432 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355449 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355450 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355451 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4260380 pg_notify(22.481(46) epoch 194487) v5 2014-08-13 17:38:05.355470 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.381 2014-08-13 17:38:05.355472 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4260380 2014-08-13 17:38:05.355490 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355492 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355509 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355510 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355511 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07e00 pg_notify(22.775(1) epoch 194487) v5 2014-08-13 17:38:05.355530 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.369 2014-08-13 17:38:05.355532 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07e00 2014-08-13 17:38:05.355553 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355555 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355548 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.355566 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355559 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.355567 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355568 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513d5c0 pg_notify(22.362(31) epoch 194487) v5 2014-08-13 17:38:05.355587 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.349 2014-08-13 17:38:05.355589 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513d5c0 2014-08-13 17:38:05.355420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355608 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355609 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355621 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355623 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355624 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d90fc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.355627 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.355630 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.355631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355632 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355641 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355643 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355644 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010380 pg_notify(24.181(12) epoch 194487) v5 2014-08-13 17:38:05.355648 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2873 2014-08-13 17:38:05.355658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.355665 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 2014-08-13 17:38:05.355667 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010380 2014-08-13 17:38:05.355667 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.355676 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.355688 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.355689 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355690 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355702 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355703 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355679 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] _activate_committed 194487 peer_activated now 4 last_epoch_started 194478 same_interval_since 194486 2014-08-13 17:38:05.355704 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822a00 pg_notify(22.3e4(1) epoch 194487) v5 2014-08-13 17:38:05.355704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 17:38:05.355706 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] flushed 2014-08-13 17:38:05.355717 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.355727 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 2014-08-13 17:38:05.355728 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822a00 2014-08-13 17:38:05.355721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.355731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.355740 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.355748 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355749 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355750 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.355759 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355762 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_notify(24.3d(32) epoch 194487) v5 2014-08-13 17:38:05.355758 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.355595 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355768 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.355784 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 2014-08-13 17:38:05.355778 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:2906 2014-08-13 17:38:05.355785 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 2014-08-13 17:38:05.355787 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.355795 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.355806 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355808 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355804 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.355814 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.355821 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355822 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355823 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d93840 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.355827 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.355829 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.355830 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355831 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355826 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 17:38:05.355839 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.355841 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355843 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355844 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107e00 pg_notify(24.181(33) epoch 194487) v5 2014-08-13 17:38:05.355843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.355853 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.355865 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.344 2014-08-13 17:38:05.355867 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107e00 2014-08-13 17:38:05.355864 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.355883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.355896 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355897 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.355913 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355914 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355915 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58232c0 pg_notify(22.1d2(1) epoch 194487) v5 2014-08-13 17:38:05.355935 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.331 2014-08-13 17:38:05.355936 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58232c0 2014-08-13 17:38:05.355769 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.355956 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355958 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355967 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.355969 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.355970 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103b80 pg_notify(22.481(39) epoch 194487) v5 2014-08-13 17:38:05.355989 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 2014-08-13 17:38:05.355990 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103b80 2014-08-13 17:38:05.356009 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.356010 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.356105 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.355944 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.356113 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.356238 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2660 2014-08-13 17:38:05.356250 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.356259 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.356267 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.356279 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.356294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 17:38:05.356307 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.356310 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.356320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.356330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.356340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.356118 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 17:38:05.356349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.356491 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.356498 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.356354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 378 as a source 2014-08-13 17:38:05.356570 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2615 2014-08-13 17:38:05.356579 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.356587 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.356596 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.356606 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.356619 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 17:38:05.356632 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.356636 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.356645 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.356655 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.356663 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.356671 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.356680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.356689 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.356530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.356706 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 17:38:05.356723 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.356726 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 17:38:05.356736 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.356745 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 17:38:05.356780 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 17:38:05.356790 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 2014-08-13 17:38:05.356799 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.356809 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.356822 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 2014-08-13 17:38:05.356838 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.356841 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Started advmap 2014-08-13 17:38:05.356851 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.356861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map 2014-08-13 17:38:05.356704 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.356888 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.356891 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.356892 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d91d40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.356898 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.356901 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.356902 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.356903 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.356905 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters 2014-08-13 17:38:05.356915 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.356924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.356934 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.356949 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.356952 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.356954 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5823f00 pg_notify(22.1f1(1) epoch 194487) v5 2014-08-13 17:38:05.356948 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 17:38:05.356961 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.356965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.356980 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.11 2014-08-13 17:38:05.356974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.356982 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5823f00 2014-08-13 17:38:05.356985 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.356996 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.357011 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.357005 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.357013 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.356877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.357130 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.357132 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.357133 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9e300 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.357138 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.357141 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.357142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.357143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.357203 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.357204 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.357205 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_info(1 pgs e194487:24.9b) v4 2014-08-13 17:38:05.357203 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.357212 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.357052 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.357236 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.378 2014-08-13 17:38:05.357240 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 2014-08-13 17:38:05.357309 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2723 2014-08-13 17:38:05.357318 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.357327 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.357335 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.357345 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.357359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 17:38:05.357371 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.357375 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.357384 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.357393 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.357229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.357402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.357411 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.357420 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.357429 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.357442 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 2014-08-13 17:38:05.357458 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.357462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Started advmap 2014-08-13 17:38:05.357472 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.357481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map 2014-08-13 17:38:05.357527 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters 2014-08-13 17:38:05.357538 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.357547 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.357557 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.357403 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.357571 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 17:38:05.357584 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.357587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.357597 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.357606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.357615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.357623 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.357631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.357640 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.357653 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 17:38:05.357677 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.357680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.357690 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.357699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.357709 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.357717 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.357577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.357906 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.357914 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.357750 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358010 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2707 2014-08-13 17:38:05.358019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.358027 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.358035 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.358045 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.358058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 2014-08-13 17:38:05.358074 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.358078 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 17:38:05.358087 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.357924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 17:38:05.358145 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 17:38:05.358155 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.358163 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.358173 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.358183 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:05.358186 7fe99f5e9700 7 osd.4 194487 do_notify osd 17 on 1 PGs 2014-08-13 17:38:05.358205 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4c71b80 192.168.1.103:6815/38269 2014-08-13 17:38:05.358227 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.20 has 194487 2014-08-13 17:38:05.358229 7fe99f5e9700 7 osd.4 194487 do_notify osd 20 on 1 PGs 2014-08-13 17:38:05.358097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358315 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x49e39c0 192.168.1.101:6808/22782 2014-08-13 17:38:05.358344 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.304 has 194487 2014-08-13 17:38:05.358347 7fe99f5e9700 7 osd.4 194487 do_notify osd 304 on 1 PGs 2014-08-13 17:38:05.358356 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing 2014-08-13 17:38:05.358358 7fe99f5e9700 7 osd.4 194487 do_notify osd 324 on 1 PGs 2014-08-13 17:38:05.358371 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4ec9b80 192.168.1.106:6802/4876 2014-08-13 17:38:05.358383 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.330 has 194487 2014-08-13 17:38:05.358385 7fe99f5e9700 7 osd.4 194487 do_notify osd 330 on 1 PGs 2014-08-13 17:38:05.358275 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358449 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4e4a100 192.168.1.104:6820/2308 2014-08-13 17:38:05.358471 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.346 has 194487 2014-08-13 17:38:05.358473 7fe99f5e9700 7 osd.4 194487 do_notify osd 346 on 1 PGs 2014-08-13 17:38:05.358521 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4cfc620 192.168.1.114:6826/52317 2014-08-13 17:38:05.358537 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.364 has 194487 2014-08-13 17:38:05.358539 7fe99f5e9700 7 osd.4 194487 do_notify osd 364 on 1 PGs 2014-08-13 17:38:05.358578 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 26 2014-08-13 17:38:05.358587 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x592c360 don't know epoch, doing nothing 2014-08-13 17:38:05.358615 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 30 2014-08-13 17:38:05.358449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358621 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4a44780 192.168.1.107:6823/55491 2014-08-13 17:38:05.358636 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.30 has 194487 2014-08-13 17:38:05.358674 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 340 2014-08-13 17:38:05.358681 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:05.358752 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 17:38:05.358772 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.358778 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.358622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.358789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.358800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.358809 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.358818 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.358826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.358837 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.358852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 17:38:05.358865 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.358868 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.358878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.358889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.358898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.358907 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.358796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359030 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.359039 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.359102 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2571 2014-08-13 17:38:05.359111 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.359119 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.359128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.359138 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.358969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359152 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 2014-08-13 17:38:05.359171 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.359175 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Started advmap 2014-08-13 17:38:05.359192 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.359202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map 2014-08-13 17:38:05.359242 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters 2014-08-13 17:38:05.359252 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.359261 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.359271 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.359285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 17:38:05.359298 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.359301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.359142 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.359321 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.359331 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.359340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.359318 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359552 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.359560 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.359492 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359666 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:3008 2014-08-13 17:38:05.359676 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.359684 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.359693 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.359702 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.359716 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 17:38:05.359729 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.359732 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.359742 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.359751 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.359760 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.359768 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.359776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.359785 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.359798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 2014-08-13 17:38:05.359816 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.359820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap 2014-08-13 17:38:05.359665 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.359832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.359843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap 2014-08-13 17:38:05.359854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.359865 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map 2014-08-13 17:38:05.359876 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary 2014-08-13 17:38:05.359886 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194487:22349 2014-08-13 17:38:05.359896 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters 2014-08-13 17:38:05.359907 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.920806 4 0.000648 2014-08-13 17:38:05.359920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering 2014-08-13 17:38:05.359930 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.008680 0 0.000000 2014-08-13 17:38:05.359945 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active 2014-08-13 17:38:05.359955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate 2014-08-13 17:38:05.359967 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.359977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] 2014-08-13 17:38:05.359986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) 2014-08-13 17:38:05.359839 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360012 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360188 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360361 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery 2014-08-13 17:38:05.360627 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] 2014-08-13 17:38:05.360648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.360661 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 is up to date, queueing in pending_activators 2014-08-13 17:38:05.360534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360704 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.360718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.360737 7fe99f5e9700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 2014-08-13 17:38:05.360707 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.360897 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 sending log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 17:38:05.360913 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing 2014-08-13 17:38:05.360932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.360947 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.360959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 is up to date, queueing in pending_activators 2014-08-13 17:38:05.361011 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.361023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.361039 7fe99f5e9700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 2014-08-13 17:38:05.360881 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361216 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 sending log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 17:38:05.361233 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4ec9b80 already has epoch 194487 2014-08-13 17:38:05.361255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.361300 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing 2014-08-13 17:38:05.361230 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361374 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.4 2014-08-13 17:38:05.361445 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.4 2014-08-13 17:38:05.361408 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361515 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.4 2014-08-13 17:38:05.361584 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.4 2014-08-13 17:38:05.361654 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.4 2014-08-13 17:38:05.361582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.4 2014-08-13 17:38:05.361793 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.4 2014-08-13 17:38:05.361756 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.361862 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.4 2014-08-13 17:38:05.361931 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.4 2014-08-13 17:38:05.362001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.4 2014-08-13 17:38:05.361929 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.4 2014-08-13 17:38:05.362140 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.4 2014-08-13 17:38:05.362102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.4 2014-08-13 17:38:05.362285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.4 2014-08-13 17:38:05.362355 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.4 2014-08-13 17:38:05.362278 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362424 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.4 2014-08-13 17:38:05.362513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.4 2014-08-13 17:38:05.362452 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.4 2014-08-13 17:38:05.362661 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.4 2014-08-13 17:38:05.362732 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.4 2014-08-13 17:38:05.362625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362802 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.4 2014-08-13 17:38:05.362874 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.4 2014-08-13 17:38:05.362803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.362943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.4 2014-08-13 17:38:05.363016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.4 2014-08-13 17:38:05.362977 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.363087 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.4 2014-08-13 17:38:05.363157 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.4 2014-08-13 17:38:05.363231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.4 2014-08-13 17:38:05.363152 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.363301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.4 2014-08-13 17:38:05.363372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.4 2014-08-13 17:38:05.363330 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.363443 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.4 2014-08-13 17:38:05.363513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.4 2014-08-13 17:38:05.363583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.4 2014-08-13 17:38:05.363505 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.363654 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.4 2014-08-13 17:38:05.363725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.4 2014-08-13 17:38:05.363796 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.4 2014-08-13 17:38:05.363700 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.363867 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.4 2014-08-13 17:38:05.363938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.4 2014-08-13 17:38:05.363875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364008 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.4 2014-08-13 17:38:05.364079 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.4 2014-08-13 17:38:05.364150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.4 2014-08-13 17:38:05.364050 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364224 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.4 2014-08-13 17:38:05.364296 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.4 2014-08-13 17:38:05.364226 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364367 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.4 2014-08-13 17:38:05.364438 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.4 2014-08-13 17:38:05.364402 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364509 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.4 2014-08-13 17:38:05.364578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.4 2014-08-13 17:38:05.364649 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.4 2014-08-13 17:38:05.364577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364720 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.4 2014-08-13 17:38:05.364791 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.4 2014-08-13 17:38:05.364752 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.364862 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.4 2014-08-13 17:38:05.364932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.4 2014-08-13 17:38:05.365003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.4 2014-08-13 17:38:05.364927 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.4 2014-08-13 17:38:05.365144 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.4 2014-08-13 17:38:05.365102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365218 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.4 2014-08-13 17:38:05.365288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.4 2014-08-13 17:38:05.365359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.4 2014-08-13 17:38:05.365279 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.4 2014-08-13 17:38:05.365501 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.4 2014-08-13 17:38:05.365454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365571 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 17:38:05.365698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 17 as a source 2014-08-13 17:38:05.365629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.17 2014-08-13 17:38:05.365842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.17 2014-08-13 17:38:05.365804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.365918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.17 2014-08-13 17:38:05.365993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.17 2014-08-13 17:38:05.366071 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.17 2014-08-13 17:38:05.365979 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.366151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.17 2014-08-13 17:38:05.366237 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.17 2014-08-13 17:38:05.366152 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.366322 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.17 2014-08-13 17:38:05.366328 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.366427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.17 2014-08-13 17:38:05.366513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.17 2014-08-13 17:38:05.366504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.366601 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.17 2014-08-13 17:38:05.366691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.17 2014-08-13 17:38:05.366679 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.366783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.17 2014-08-13 17:38:05.366877 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.17 2014-08-13 17:38:05.366972 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.17 2014-08-13 17:38:05.366854 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 17:38:05.367070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.17 2014-08-13 17:38:05.367077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound 2014-08-13 17:38:05.367169 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.17 2014-08-13 17:38:05.367274 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.17 2014-08-13 17:38:05.367255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.367377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.17 2014-08-13 17:38:05.367481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.17 2014-08-13 17:38:05.367432 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 291,350,365,367,378 2014-08-13 17:38:05.367587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.17 2014-08-13 17:38:05.367608 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished 2014-08-13 17:38:05.367695 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.17 2014-08-13 17:38:05.367807 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.17 2014-08-13 17:38:05.367786 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.367918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.17 2014-08-13 17:38:05.367963 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.368031 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.17 2014-08-13 17:38:05.368145 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.17 2014-08-13 17:38:05.368139 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.368370 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.368265 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.17 2014-08-13 17:38:05.368411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 17:38:05.368430 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.368435 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.368447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.368454 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368457 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368457 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.368467 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.368475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368477 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368479 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d96300 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368476 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.368485 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368488 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368490 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368491 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368485 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.368496 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.368382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.17 2014-08-13 17:38:05.368502 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368503 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368504 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db2400 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368508 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368510 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368511 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 17:38:05.368522 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368526 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368526 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.368527 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92ac0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368531 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368533 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368534 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.368550 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368551 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368553 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dba80 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368556 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368558 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368559 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368560 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368541 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.368569 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368571 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368572 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df740 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368575 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368578 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368568 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,312) went down 2014-08-13 17:38:05.368579 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368580 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368581 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.368589 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368590 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368592 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d8f500 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368595 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368597 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368590 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.368614 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368615 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368616 7fe9a63f4700 20 osd.4 194487 _dispatch 0x549a880 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.368500 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.17 2014-08-13 17:38:05.368619 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.368621 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.368622 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368623 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368633 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368635 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368635 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_notify(24.3d(32) epoch 194487) v5 2014-08-13 17:38:05.368658 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.373 2014-08-13 17:38:05.368660 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 2014-08-13 17:38:05.368689 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368690 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368715 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368716 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368717 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825780 pg_notify(22.113(1) epoch 194487) v5 2014-08-13 17:38:05.368745 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.342 2014-08-13 17:38:05.368747 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825780 2014-08-13 17:38:05.368624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.17 2014-08-13 17:38:05.368768 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368769 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368780 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368781 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368782 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e3c0 pg_notify(22.3d9(45) epoch 194487) v5 2014-08-13 17:38:05.368776 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.368786 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.368803 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.396 2014-08-13 17:38:05.368804 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e3c0 2014-08-13 17:38:05.368824 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368825 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368843 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368844 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368845 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513dcc0 pg_notify(24.30b(39) epoch 194487) v5 2014-08-13 17:38:05.368865 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.375 2014-08-13 17:38:05.368866 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513dcc0 2014-08-13 17:38:05.368751 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.17 2014-08-13 17:38:05.368879 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2862 2014-08-13 17:38:05.368890 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368891 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368889 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.368898 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.368914 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368907 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.368915 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368916 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6012a00 pg_notify(22.39d(1) epoch 194487) v5 2014-08-13 17:38:05.368918 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.368937 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.366 2014-08-13 17:38:05.368938 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6012a00 2014-08-13 17:38:05.368933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 17:38:05.368946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.368958 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.368959 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.368973 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.368975 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.368976 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5821a40 pg_notify(24.9b(34) epoch 194487) v5 2014-08-13 17:38:05.368970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.368980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.368989 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.368998 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.291 2014-08-13 17:38:05.368876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.17 2014-08-13 17:38:05.369000 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5821a40 2014-08-13 17:38:05.368998 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.369009 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.369021 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369022 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369023 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 2014-08-13 17:38:05.369035 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369036 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369037 7fe9a63f4700 20 osd.4 194487 _dispatch 0x60124c0 pg_info(1 pgs e194487:24.9b) v4 2014-08-13 17:38:05.369044 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.369048 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Peering advmap 2014-08-13 17:38:05.369058 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.367 2014-08-13 17:38:05.369061 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x60124c0 2014-08-13 17:38:05.369061 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.369072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Started advmap 2014-08-13 17:38:05.369083 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369085 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.369096 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369097 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369099 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264980 pg_notify(22.3d9(42) epoch 194487) v5 2014-08-13 17:38:05.369094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map 2014-08-13 17:38:05.369106 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle ActMap primary 2014-08-13 17:38:05.369120 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.373 2014-08-13 17:38:05.369121 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264980 2014-08-13 17:38:05.369118 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194487:29705 2014-08-13 17:38:05.369001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.17 2014-08-13 17:38:05.369129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] take_waiters 2014-08-13 17:38:05.369142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369141 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 0.931106 5 0.009167 2014-08-13 17:38:05.369158 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369159 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369161 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58255c0 pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 2014-08-13 17:38:05.369154 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Leaving Peering 2014-08-13 17:38:05.369164 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 1.087561 0 0.000000 2014-08-13 17:38:05.369184 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.342 2014-08-13 17:38:05.369185 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58255c0 2014-08-13 17:38:05.369197 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active 2014-08-13 17:38:05.369208 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: In Active, about to call activate 2014-08-13 17:38:05.369226 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369227 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369222 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.369233 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - snap_trimq [] 2014-08-13 17:38:05.369242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - not complete, missing(57) 2014-08-13 17:38:05.369258 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369259 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369260 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825240 pg_info(1 pgs e194487:24.30b) v4 2014-08-13 17:38:05.369130 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.17 2014-08-13 17:38:05.369280 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.375 2014-08-13 17:38:05.369282 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825240 2014-08-13 17:38:05.369094 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.369324 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369326 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369337 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369338 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369339 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db840 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.369343 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.369346 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.369347 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369348 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369357 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369358 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369359 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5499d40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.369362 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.369364 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.369365 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369366 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369376 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369377 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369378 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6012140 pg_notify(24.34(52) epoch 194487) v5 2014-08-13 17:38:05.369263 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.17 2014-08-13 17:38:05.369398 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.340 2014-08-13 17:38:05.369400 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6012140 2014-08-13 17:38:05.369430 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369432 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369454 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369455 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369461 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102140 pg_notify(24.34(52) epoch 194487) v5 2014-08-13 17:38:05.369302 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] flushed 2014-08-13 17:38:05.369484 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.30 2014-08-13 17:38:05.369487 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102140 2014-08-13 17:38:05.369512 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.369514 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.369394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.17 2014-08-13 17:38:05.369527 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.17 2014-08-13 17:38:05.369660 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.17 2014-08-13 17:38:05.369795 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.17 2014-08-13 17:38:05.369933 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.17 2014-08-13 17:38:05.370072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.17 2014-08-13 17:38:05.370215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.17 2014-08-13 17:38:05.370358 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.17 2014-08-13 17:38:05.370495 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - starting recovery 2014-08-13 17:38:05.370511 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] 2014-08-13 17:38:05.370542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.370556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 is up to date, queueing in pending_activators 2014-08-13 17:38:05.370593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.370605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.370622 7fe99ebe8700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 2014-08-13 17:38:05.370502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.17 2014-08-13 17:38:05.370648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.17 2014-08-13 17:38:05.370914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 sending log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 17:38:05.370931 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing 2014-08-13 17:38:05.370795 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.17 2014-08-13 17:38:05.370951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.370966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.370978 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 is up to date, queueing in pending_activators 2014-08-13 17:38:05.371018 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.371033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.371049 7fe99ebe8700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 2014-08-13 17:38:05.370944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.17 2014-08-13 17:38:05.371097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.17 2014-08-13 17:38:05.371255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.17 2014-08-13 17:38:05.371412 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 sending log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 17:38:05.371432 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:05.371449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.371412 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.17 2014-08-13 17:38:05.371501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery primary has 57 missing 2014-08-13 17:38:05.371573 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.4 2014-08-13 17:38:05.371644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.4 2014-08-13 17:38:05.371571 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.17 2014-08-13 17:38:05.371713 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.4 2014-08-13 17:38:05.371783 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.4 2014-08-13 17:38:05.371731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.17 2014-08-13 17:38:05.371909 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.371912 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.371913 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ca4c0 pg_info(1 pgs e194487:24.34) v4 2014-08-13 17:38:05.371852 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.4 2014-08-13 17:38:05.371938 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.340 2014-08-13 17:38:05.371941 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ca4c0 2014-08-13 17:38:05.371920 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.4 2014-08-13 17:38:05.371997 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.371999 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.371988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.4 2014-08-13 17:38:05.371893 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.17 2014-08-13 17:38:05.372059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.4 2014-08-13 17:38:05.372129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.4 2014-08-13 17:38:05.372061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.17 2014-08-13 17:38:05.372202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.4 2014-08-13 17:38:05.372272 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.4 2014-08-13 17:38:05.372341 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.4 2014-08-13 17:38:05.372250 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.17 2014-08-13 17:38:05.372410 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.4 2014-08-13 17:38:05.372479 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.4 2014-08-13 17:38:05.372418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.17 2014-08-13 17:38:05.372548 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.4 2014-08-13 17:38:05.372617 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.4 2014-08-13 17:38:05.372686 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.4 2014-08-13 17:38:05.372587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.17 2014-08-13 17:38:05.372756 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.4 2014-08-13 17:38:05.372826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.4 2014-08-13 17:38:05.372756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.17 2014-08-13 17:38:05.372895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.4 2014-08-13 17:38:05.372964 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.4 2014-08-13 17:38:05.372929 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.17 2014-08-13 17:38:05.373033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.4 2014-08-13 17:38:05.373118 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.4 2014-08-13 17:38:05.373190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.4 2014-08-13 17:38:05.373260 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.4 2014-08-13 17:38:05.373104 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 17:38:05.373329 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.4 2014-08-13 17:38:05.373396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.4 2014-08-13 17:38:05.373465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.4 2014-08-13 17:38:05.373391 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 28 as a source 2014-08-13 17:38:05.373535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.4 2014-08-13 17:38:05.373604 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.4 2014-08-13 17:38:05.373673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.4 2014-08-13 17:38:05.373569 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.373742 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.4 2014-08-13 17:38:05.373811 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.4 2014-08-13 17:38:05.373767 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.373879 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.4 2014-08-13 17:38:05.373947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.4 2014-08-13 17:38:05.374016 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.4 2014-08-13 17:38:05.373945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.374084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.4 2014-08-13 17:38:05.374153 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.4 2014-08-13 17:38:05.374239 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.4 2014-08-13 17:38:05.374121 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.374309 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.4 2014-08-13 17:38:05.374378 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.4 2014-08-13 17:38:05.374316 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.374447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.4 2014-08-13 17:38:05.374516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.4 2014-08-13 17:38:05.374585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.4 2014-08-13 17:38:05.374494 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.374674 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.374677 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.374678 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dfbc0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.374683 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.374686 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.374687 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.374689 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.374655 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.4 2014-08-13 17:38:05.374744 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.374747 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.374748 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_notify(22.4df(1) epoch 194487) v5 2014-08-13 17:38:05.374777 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.352 2014-08-13 17:38:05.374779 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 2014-08-13 17:38:05.374723 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.4 2014-08-13 17:38:05.374807 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.374809 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.374668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.374790 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.4 2014-08-13 17:38:05.374863 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.4 2014-08-13 17:38:05.374930 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.4 2014-08-13 17:38:05.374842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375000 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.4 2014-08-13 17:38:05.375069 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.4 2014-08-13 17:38:05.375016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375137 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.4 2014-08-13 17:38:05.375215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.4 2014-08-13 17:38:05.375285 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.4 2014-08-13 17:38:05.375206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.4 2014-08-13 17:38:05.375423 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.4 2014-08-13 17:38:05.375383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375492 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.4 2014-08-13 17:38:05.375558 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 17:38:05.375558 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375678 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] activate: adding 20 as a source 2014-08-13 17:38:05.375745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.20 2014-08-13 17:38:05.375815 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=56] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.20 2014-08-13 17:38:05.375735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.375888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=55] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.20 2014-08-13 17:38:05.375981 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.375987 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.375988 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dee40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.375994 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.375997 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.375998 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.375999 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.375963 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=54] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.20 2014-08-13 17:38:05.375910 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376037 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=53] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.20 2014-08-13 17:38:05.376134 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.376136 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.376138 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264d00 pg_notify(22.481(44) epoch 194487) v5 2014-08-13 17:38:05.376173 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.306 2014-08-13 17:38:05.376175 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264d00 2014-08-13 17:38:05.376199 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.376200 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.376115 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=52] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.20 2014-08-13 17:38:05.376085 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376206 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=51] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.20 2014-08-13 17:38:05.376287 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=50] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.20 2014-08-13 17:38:05.376273 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376370 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=49] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.20 2014-08-13 17:38:05.376453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=48] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.20 2014-08-13 17:38:05.376449 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376539 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=47] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.20 2014-08-13 17:38:05.376645 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=46] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.20 2014-08-13 17:38:05.376625 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=45] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.20 2014-08-13 17:38:05.376828 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=44] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.20 2014-08-13 17:38:05.376800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.376921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=43] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.20 2014-08-13 17:38:05.377017 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=42] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.20 2014-08-13 17:38:05.376974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=41] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.20 2014-08-13 17:38:05.377219 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=40] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.20 2014-08-13 17:38:05.377150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377321 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=39] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.20 2014-08-13 17:38:05.377339 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377423 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=38] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.20 2014-08-13 17:38:05.377526 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=37] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.20 2014-08-13 17:38:05.377515 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377631 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=36] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.20 2014-08-13 17:38:05.377740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=35] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.20 2014-08-13 17:38:05.377691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377849 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=34] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.20 2014-08-13 17:38:05.377865 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.377960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=33] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.20 2014-08-13 17:38:05.378072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=32] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.20 2014-08-13 17:38:05.378040 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.378190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=31] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.20 2014-08-13 17:38:05.378218 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.378307 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=30] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.20 2014-08-13 17:38:05.378425 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=29] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.20 2014-08-13 17:38:05.378394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.378544 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=28] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.20 2014-08-13 17:38:05.378569 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.378664 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=27] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.20 2014-08-13 17:38:05.378787 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=26] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.20 2014-08-13 17:38:05.378745 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.378912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=25] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.20 2014-08-13 17:38:05.378918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379038 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=24] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.20 2014-08-13 17:38:05.379099 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379165 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=23] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.20 2014-08-13 17:38:05.379297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=22] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.20 2014-08-13 17:38:05.379278 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379429 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=21] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.20 2014-08-13 17:38:05.379453 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379562 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=20] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.20 2014-08-13 17:38:05.379628 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=19] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.20 2014-08-13 17:38:05.379834 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=18] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.20 2014-08-13 17:38:05.379803 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.379973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=17] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.20 2014-08-13 17:38:05.379977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380113 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=16] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.20 2014-08-13 17:38:05.380152 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=15] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.20 2014-08-13 17:38:05.380330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380401 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=14] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.20 2014-08-13 17:38:05.380505 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380546 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=13] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.20 2014-08-13 17:38:05.380692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=12] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.20 2014-08-13 17:38:05.380680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380840 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=11] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.20 2014-08-13 17:38:05.380883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.380991 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=10] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.20 2014-08-13 17:38:05.381060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.381143 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=9] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.20 2014-08-13 17:38:05.381239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.381300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=8] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.20 2014-08-13 17:38:05.381414 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.381456 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=7] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.20 2014-08-13 17:38:05.381589 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.381614 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=6] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.20 2014-08-13 17:38:05.381772 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=5] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.20 2014-08-13 17:38:05.381764 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.381933 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=4] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.20 2014-08-13 17:38:05.381938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.382097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=3] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.20 2014-08-13 17:38:05.382112 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.382266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=2] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.20 2014-08-13 17:38:05.382567 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.382569 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.382290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.382571 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db4800 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.382580 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.382583 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.382584 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.382585 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.382597 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.382599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.382600 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513d400 pg_notify(24.30b(35) epoch 194487) v5 2014-08-13 17:38:05.382624 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.10 2014-08-13 17:38:05.382626 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513d400 2014-08-13 17:38:05.382458 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=1] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.20 2014-08-13 17:38:05.382654 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.382656 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.382599 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.382648 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 17:38:05.382781 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.382862 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 25 as a source 2014-08-13 17:38:05.382956 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383030 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383129 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383307 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383538 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383656 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.383878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384047 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384219 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384185 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 17:38:05.384389 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384407 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 315 as a source 2014-08-13 17:38:05.384558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.315 2014-08-13 17:38:05.384727 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384757 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.315 2014-08-13 17:38:05.384895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.384934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.315 2014-08-13 17:38:05.385064 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.315 2014-08-13 17:38:05.385236 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.315 2014-08-13 17:38:05.385406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385466 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.315 2014-08-13 17:38:05.385575 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385642 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.315 2014-08-13 17:38:05.385744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.315 2014-08-13 17:38:05.385914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.385993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.315 2014-08-13 17:38:05.386083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.386169 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.315 2014-08-13 17:38:05.386255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.386485 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.386490 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.386493 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3600 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.386499 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.386502 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.386503 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.386505 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.386518 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.386519 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.386348 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.315 2014-08-13 17:38:05.386520 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5826c80 pg_notify(22.362(31) epoch 194487) v5 2014-08-13 17:38:05.386545 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.362 2014-08-13 17:38:05.386548 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5826c80 2014-08-13 17:38:05.386575 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.386576 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.386425 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.386523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.315 2014-08-13 17:38:05.386593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.386699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.315 2014-08-13 17:38:05.386762 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.386875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.315 2014-08-13 17:38:05.386931 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.315 2014-08-13 17:38:05.387100 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387230 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.315 2014-08-13 17:38:05.387272 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387407 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.315 2014-08-13 17:38:05.387441 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.315 2014-08-13 17:38:05.387610 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387758 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.315 2014-08-13 17:38:05.387779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.387934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.315 2014-08-13 17:38:05.387947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.315 2014-08-13 17:38:05.388288 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.315 2014-08-13 17:38:05.388458 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388466 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.315 2014-08-13 17:38:05.388627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388640 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.315 2014-08-13 17:38:05.388797 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388814 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.315 2014-08-13 17:38:05.388966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.388988 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.315 2014-08-13 17:38:05.389134 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.389163 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.315 2014-08-13 17:38:05.389306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.389343 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.315 2014-08-13 17:38:05.389475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.389521 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.315 2014-08-13 17:38:05.389644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.389697 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.315 2014-08-13 17:38:05.389813 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.389873 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.315 2014-08-13 17:38:05.389982 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.390049 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.315 2014-08-13 17:38:05.390151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.390228 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.315 2014-08-13 17:38:05.390323 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.390404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.315 2014-08-13 17:38:05.390493 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.390581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.315 2014-08-13 17:38:05.390662 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.390757 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.315 2014-08-13 17:38:05.390948 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.390954 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.390956 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d8ec00 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.390963 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.390966 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.390967 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.390968 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.390833 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391047 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.391051 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.391053 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102680 pg_notify(22.1d2(27) epoch 194487) v5 2014-08-13 17:38:05.391081 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.58 2014-08-13 17:38:05.391083 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102680 2014-08-13 17:38:05.390932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.315 2014-08-13 17:38:05.391113 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.391115 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.391001 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391107 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.315 2014-08-13 17:38:05.391170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391286 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.315 2014-08-13 17:38:05.391342 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.315 2014-08-13 17:38:05.391510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391637 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.315 2014-08-13 17:38:05.391678 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.391811 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.315 2014-08-13 17:38:05.391845 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.392026 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.392029 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.392030 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d90480 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.392035 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.392038 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.392039 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.392040 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.392098 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.392099 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.392100 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4a0af40 pg_notify(22.1f1(29) epoch 194487) v5 2014-08-13 17:38:05.392123 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.54 2014-08-13 17:38:05.392125 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4a0af40 2014-08-13 17:38:05.392157 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.392171 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.391986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.315 2014-08-13 17:38:05.392014 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.392183 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.392188 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.315 2014-08-13 17:38:05.392356 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.392371 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.315 2014-08-13 17:38:05.392525 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.392548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.315 2014-08-13 17:38:05.392723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.315 2014-08-13 17:38:05.392694 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 17:38:05.392903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 303 as a source 2014-08-13 17:38:05.392899 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.315 2014-08-13 17:38:05.393073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.303 2014-08-13 17:38:05.393074 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.315 2014-08-13 17:38:05.393249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.303 2014-08-13 17:38:05.393254 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.315 2014-08-13 17:38:05.393420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.303 2014-08-13 17:38:05.393429 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.315 2014-08-13 17:38:05.393740 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.393743 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.393744 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d94ec0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.393749 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.393752 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.393753 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.393754 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.393591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.303 2014-08-13 17:38:05.393603 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.315 2014-08-13 17:38:05.393863 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.393868 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.393870 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f039c0 pg_notify(22.f3(23) epoch 194487) v5 2014-08-13 17:38:05.393895 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.311 2014-08-13 17:38:05.393897 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f039c0 2014-08-13 17:38:05.393759 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.303 2014-08-13 17:38:05.393925 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.393927 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.393777 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.315 2014-08-13 17:38:05.393928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.303 2014-08-13 17:38:05.393951 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.315 2014-08-13 17:38:05.394097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.303 2014-08-13 17:38:05.394127 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.315 2014-08-13 17:38:05.394270 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.303 2014-08-13 17:38:05.394462 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.394464 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.394465 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5450000 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.394471 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.394474 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.394475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.394476 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.394304 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.315 2014-08-13 17:38:05.394539 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.394540 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.394541 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_notify(24.9b(2) epoch 194487) v5 2014-08-13 17:38:05.394565 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.365 2014-08-13 17:38:05.394566 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 2014-08-13 17:38:05.394593 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.394594 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.394440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.303 2014-08-13 17:38:05.394478 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.315 2014-08-13 17:38:05.394609 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.303 2014-08-13 17:38:05.394653 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.315 2014-08-13 17:38:05.394778 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.303 2014-08-13 17:38:05.394826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.315 2014-08-13 17:38:05.394948 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.303 2014-08-13 17:38:05.395004 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 17:38:05.395118 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.303 2014-08-13 17:38:05.395361 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395365 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54daac0 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.395370 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.395372 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.395373 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395374 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395225 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 330 as a source 2014-08-13 17:38:05.395290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.303 2014-08-13 17:38:05.395506 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395508 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395509 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58216c0 pg_notify(22.22c(25) epoch 194487) v5 2014-08-13 17:38:05.395531 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.368 2014-08-13 17:38:05.395533 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58216c0 2014-08-13 17:38:05.395556 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395557 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.395459 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.303 2014-08-13 17:38:05.395747 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.395749 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395750 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5458240 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.395755 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.395757 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.395758 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395759 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395771 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.395772 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.395773 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5823d40 pg_info(1 pgs e194487:24.b6) v4 2014-08-13 17:38:05.395628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.303 2014-08-13 17:38:05.395796 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.b6) v4 from osd.28 2014-08-13 17:38:05.395799 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5823d40 2014-08-13 17:38:05.395749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.395797 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.303 2014-08-13 17:38:05.395924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.395966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.303 2014-08-13 17:38:05.396100 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.396134 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.303 2014-08-13 17:38:05.396293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.396319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.303 2014-08-13 17:38:05.396469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.396489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.303 2014-08-13 17:38:05.396644 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.396668 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.303 2014-08-13 17:38:05.396818 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.396837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.303 2014-08-13 17:38:05.396993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.397005 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.303 2014-08-13 17:38:05.397168 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.397174 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.303 2014-08-13 17:38:05.397352 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.303 2014-08-13 17:38:05.397349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.397524 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.303 2014-08-13 17:38:05.397523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.397695 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.303 2014-08-13 17:38:05.397697 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.397865 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.303 2014-08-13 17:38:05.397871 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.303 2014-08-13 17:38:05.398045 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398209 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.303 2014-08-13 17:38:05.398222 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398379 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.303 2014-08-13 17:38:05.398398 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398550 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.303 2014-08-13 17:38:05.398573 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398720 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.303 2014-08-13 17:38:05.398748 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.398890 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.303 2014-08-13 17:38:05.398923 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399062 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.303 2014-08-13 17:38:05.399098 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.303 2014-08-13 17:38:05.399276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.303 2014-08-13 17:38:05.399451 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399576 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.303 2014-08-13 17:38:05.399626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399746 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.303 2014-08-13 17:38:05.399800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.399917 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.303 2014-08-13 17:38:05.399975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400087 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.303 2014-08-13 17:38:05.400150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400260 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.303 2014-08-13 17:38:05.400327 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.303 2014-08-13 17:38:05.400502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400600 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.303 2014-08-13 17:38:05.400677 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400771 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.303 2014-08-13 17:38:05.400852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.400941 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.303 2014-08-13 17:38:05.401027 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401111 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.303 2014-08-13 17:38:05.401205 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401284 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.303 2014-08-13 17:38:05.401380 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.303 2014-08-13 17:38:05.401555 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.303 2014-08-13 17:38:05.401730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401795 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.303 2014-08-13 17:38:05.401904 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.401968 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.303 2014-08-13 17:38:05.402079 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.402138 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.303 2014-08-13 17:38:05.402257 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.402311 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.303 2014-08-13 17:38:05.402433 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.402481 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.303 2014-08-13 17:38:05.402608 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.402651 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.303 2014-08-13 17:38:05.402783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.402821 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 17:38:05.402958 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403030 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 362 as a source 2014-08-13 17:38:05.403132 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403203 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403309 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403373 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403484 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403659 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403834 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.403880 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404009 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404049 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404186 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404221 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404361 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404536 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404560 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404728 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404886 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.404896 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405064 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405413 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405575 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405586 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405760 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 17:38:05.405913 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.405977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound 2014-08-13 17:38:05.406082 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.406152 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.406254 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.406332 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 17,28,315,327,330 2014-08-13 17:38:05.406424 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.406509 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished 2014-08-13 17:38:05.406593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.406686 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.406762 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.406863 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.406931 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407043 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.407100 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407280 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.407332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407335 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407331 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 17:38:05.407349 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407350 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407351 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07e00 pg_notify(22.481(39) epoch 194487) v5 2014-08-13 17:38:05.407353 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.407359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap 2014-08-13 17:38:05.407373 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.365 2014-08-13 17:38:05.407375 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07e00 2014-08-13 17:38:05.407372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.407385 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map 2014-08-13 17:38:05.407398 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407400 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407396 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters 2014-08-13 17:38:05.407406 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.407415 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407416 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407418 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_notify(22.1f1(29) epoch 194487) v5 2014-08-13 17:38:05.407416 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.407428 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.407438 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.368 2014-08-13 17:38:05.407439 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 2014-08-13 17:38:05.407274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407444 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 17:38:05.407459 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407459 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.407461 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.407474 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407475 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407476 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dde80 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.407473 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.407481 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.407484 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.407485 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407486 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407484 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.407497 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407498 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407499 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16580 pg_info(1 pgs e194487:24.34) v4 2014-08-13 17:38:05.407493 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.407502 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.407511 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.407521 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.407521 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.30 2014-08-13 17:38:05.407524 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16580 2014-08-13 17:38:05.407545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 2014-08-13 17:38:05.407553 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.407558 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407559 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407560 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db7500 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.407563 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.407557 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Started advmap 2014-08-13 17:38:05.407566 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.407568 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407569 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.407579 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407580 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407581 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824d00 pg_info(1 pgs e194487:24.b6) v4 2014-08-13 17:38:05.407578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map 2014-08-13 17:38:05.407608 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.b6) v4 from osd.330 2014-08-13 17:38:05.407443 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407610 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824d00 2014-08-13 17:38:05.407631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407633 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407627 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters 2014-08-13 17:38:05.407643 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407645 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407637 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.407646 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02bc0 pg_notify(24.9b(34) epoch 194487) v5 2014-08-13 17:38:05.407647 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.407657 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.407666 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.350 2014-08-13 17:38:05.407668 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02bc0 2014-08-13 17:38:05.407672 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 2014-08-13 17:38:05.407690 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407690 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.407691 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407695 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Peering advmap 2014-08-13 17:38:05.407712 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407713 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407714 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02d80 pg_notify(24.136(2) epoch 194487) v5 2014-08-13 17:38:05.407707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.407718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Started advmap 2014-08-13 17:38:05.407735 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.50 2014-08-13 17:38:05.407736 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02d80 2014-08-13 17:38:05.407730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.407742 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map 2014-08-13 17:38:05.407757 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.407759 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.407753 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: handle ActMap primary 2014-08-13 17:38:05.407766 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194487:25637 2014-08-13 17:38:05.407611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] take_waiters 2014-08-13 17:38:05.407788 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 0.969818 3 0.009102 2014-08-13 17:38:05.407801 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Leaving Peering 2014-08-13 17:38:05.407811 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 1.053829 0 0.000000 2014-08-13 17:38:05.407833 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active 2014-08-13 17:38:05.407843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: In Active, about to call activate 2014-08-13 17:38:05.407856 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.407867 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - snap_trimq [] 2014-08-13 17:38:05.407875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - not complete, missing(62) 2014-08-13 17:38:05.407779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.407947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.408115 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.408139 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.408287 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.408350 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] flushed 2014-08-13 17:38:05.408455 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.408723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - starting recovery 2014-08-13 17:38:05.408742 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] 2014-08-13 17:38:05.408768 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.408624 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.408782 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 is up to date, queueing in pending_activators 2014-08-13 17:38:05.408854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.408870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.408888 7fe99f5e9700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 2014-08-13 17:38:05.408792 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409056 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 sending log((192718'12872,194422'15456], crt=194422'15456) 2014-08-13 17:38:05.409075 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x59cf640 don't know epoch, doing nothing 2014-08-13 17:38:05.409093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.409110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.408961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409124 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 is up to date, queueing in pending_activators 2014-08-13 17:38:05.409153 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.409202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery primary has 62 missing 2014-08-13 17:38:05.409129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409280 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.4 2014-08-13 17:38:05.409356 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.4 2014-08-13 17:38:05.409301 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.4 2014-08-13 17:38:05.409504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.4 2014-08-13 17:38:05.409471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.4 2014-08-13 17:38:05.409650 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.4 2014-08-13 17:38:05.409724 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.4 2014-08-13 17:38:05.409640 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.4 2014-08-13 17:38:05.409871 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.4 2014-08-13 17:38:05.409808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.409945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.4 2014-08-13 17:38:05.410018 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.4 2014-08-13 17:38:05.409977 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.4 2014-08-13 17:38:05.410164 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.4 2014-08-13 17:38:05.410241 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.4 2014-08-13 17:38:05.410146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410314 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.4 2014-08-13 17:38:05.410389 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.4 2014-08-13 17:38:05.410317 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.4 2014-08-13 17:38:05.410536 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.4 2014-08-13 17:38:05.410487 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410609 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.4 2014-08-13 17:38:05.410683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.4 2014-08-13 17:38:05.410656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.4 2014-08-13 17:38:05.410828 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.4 2014-08-13 17:38:05.410902 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.4 2014-08-13 17:38:05.410826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.410975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.4 2014-08-13 17:38:05.411048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.4 2014-08-13 17:38:05.410995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.411122 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.4 2014-08-13 17:38:05.411198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.4 2014-08-13 17:38:05.411164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.411272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.4 2014-08-13 17:38:05.411344 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.4 2014-08-13 17:38:05.411418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.4 2014-08-13 17:38:05.411336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.411491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.4 2014-08-13 17:38:05.411564 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.4 2014-08-13 17:38:05.411504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.411638 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.4 2014-08-13 17:38:05.411711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.4 2014-08-13 17:38:05.411673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.411785 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.4 2014-08-13 17:38:05.411857 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.4 2014-08-13 17:38:05.411930 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.4 2014-08-13 17:38:05.411842 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.4 2014-08-13 17:38:05.412078 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.4 2014-08-13 17:38:05.412010 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.4 2014-08-13 17:38:05.412228 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.4 2014-08-13 17:38:05.412179 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.4 2014-08-13 17:38:05.412375 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.4 2014-08-13 17:38:05.412351 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412448 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.4 2014-08-13 17:38:05.412528 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.4 2014-08-13 17:38:05.412606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.4 2014-08-13 17:38:05.412523 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.4 2014-08-13 17:38:05.412754 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.4 2014-08-13 17:38:05.412691 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.412827 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.4 2014-08-13 17:38:05.412900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.4 2014-08-13 17:38:05.412974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.4 2014-08-13 17:38:05.412860 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 17:38:05.413048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.4 2014-08-13 17:38:05.413121 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.4 2014-08-13 17:38:05.413072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound 2014-08-13 17:38:05.413198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.4 2014-08-13 17:38:05.413272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.4 2014-08-13 17:38:05.413243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.413346 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.4 2014-08-13 17:38:05.413418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.4 2014-08-13 17:38:05.413492 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.4 2014-08-13 17:38:05.413414 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound: built 20,25,303,362,373 2014-08-13 17:38:05.413565 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.4 2014-08-13 17:38:05.413639 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.4 2014-08-13 17:38:05.413585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Activate Finished 2014-08-13 17:38:05.413712 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.4 2014-08-13 17:38:05.413786 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.4 2014-08-13 17:38:05.413756 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.413858 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 17:38:05.413977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] activate: adding 55 as a source 2014-08-13 17:38:05.413927 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.414050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.55 2014-08-13 17:38:05.414125 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=61] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.55 2014-08-13 17:38:05.414097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.414206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=60] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.55 2014-08-13 17:38:05.414339 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.414286 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=59] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.55 2014-08-13 17:38:05.414376 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 17:38:05.414394 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.414400 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.414411 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.414421 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.414431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.414439 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.414367 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=58] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.55 2014-08-13 17:38:05.414448 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.414458 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.414472 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 17:38:05.414485 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.414489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.414499 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.414510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.414520 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.414450 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=57] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.55 2014-08-13 17:38:05.414528 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.414533 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=56] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.55 2014-08-13 17:38:05.414620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=55] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.55 2014-08-13 17:38:05.414704 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.414712 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.414708 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=54] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.55 2014-08-13 17:38:05.414800 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:3022 2014-08-13 17:38:05.414810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.414819 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.414827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.414838 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.414843 7fe99ebe8700 20 osd.4 194487 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 20 2014-08-13 17:38:05.414851 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c71b80 already has epoch 194487 2014-08-13 17:38:05.414798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=53] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.55 2014-08-13 17:38:05.414884 7fe99ebe8700 20 osd.4 194487 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 291 2014-08-13 17:38:05.414897 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing 2014-08-13 17:38:05.414925 7fe99ebe8700 20 osd.4 194487 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 303 2014-08-13 17:38:05.414738 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.414931 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:05.414954 7fe99ebe8700 20 osd.4 194487 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 350 2014-08-13 17:38:05.414962 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing 2014-08-13 17:38:05.414889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=52] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.55 2014-08-13 17:38:05.415033 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 2014-08-13 17:38:05.415055 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.415059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 17:38:05.414982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=51] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.55 2014-08-13 17:38:05.415070 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.415081 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:05.414933 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] flushed 2014-08-13 17:38:05.415116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:05.415126 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.415135 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.415145 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.415076 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=50] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.55 2014-08-13 17:38:05.415160 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 17:38:05.415173 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.415177 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.415191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.415206 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.415215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.415224 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.415232 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.415242 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.415255 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 17:38:05.415267 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.415172 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=49] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.55 2014-08-13 17:38:05.415271 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.415282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.415292 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 17:38:05.415302 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.415310 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.415272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=48] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.55 2014-08-13 17:38:05.415458 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.415372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=47] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.55 2014-08-13 17:38:05.415468 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.415544 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2685 2014-08-13 17:38:05.415555 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.415564 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.415474 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=46] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.55 2014-08-13 17:38:05.415572 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.415583 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.415596 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 17:38:05.415609 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.415612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.415622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.415632 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,287) went down 2014-08-13 17:38:05.415641 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.415650 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.415577 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=45] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.55 2014-08-13 17:38:05.415772 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.415776 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.415779 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5428000 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.415681 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=44] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.55 2014-08-13 17:38:05.415786 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.415789 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.415791 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.415792 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.415842 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.415850 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.415786 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=43] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.55 2014-08-13 17:38:05.415910 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.415913 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.415914 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825940 pg_notify(24.b6(38) epoch 194487) v5 2014-08-13 17:38:05.415940 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.315 2014-08-13 17:38:05.415942 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825940 2014-08-13 17:38:05.415947 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2684 2014-08-13 17:38:05.415956 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.415971 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.415965 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.415973 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.415974 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.415985 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.415894 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=42] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.55 2014-08-13 17:38:05.415998 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 17:38:05.416010 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.416014 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.416023 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.416033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,289) went down 2014-08-13 17:38:05.416042 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.416051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.416002 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=41] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.55 2014-08-13 17:38:05.416113 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=40] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.55 2014-08-13 17:38:05.416293 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.416303 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.416230 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=39] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.55 2014-08-13 17:38:05.416423 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2690 2014-08-13 17:38:05.416434 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.416443 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.416451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.416346 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=38] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.55 2014-08-13 17:38:05.416461 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.416475 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 17:38:05.416488 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.416491 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.416500 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.416510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.416519 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.416527 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.416536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.416545 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.416559 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 2014-08-13 17:38:05.416575 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.416462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=37] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.55 2014-08-13 17:38:05.416578 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Started advmap 2014-08-13 17:38:05.416588 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.416620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map 2014-08-13 17:38:05.416656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters 2014-08-13 17:38:05.416665 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.416674 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.416684 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.416580 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=36] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.55 2014-08-13 17:38:05.416699 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 2014-08-13 17:38:05.416718 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.416722 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Peering advmap 2014-08-13 17:38:05.416733 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.416743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Started advmap 2014-08-13 17:38:05.416754 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.416764 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map 2014-08-13 17:38:05.416775 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: handle ActMap primary 2014-08-13 17:38:05.416786 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194487:22150 2014-08-13 17:38:05.416796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] take_waiters 2014-08-13 17:38:05.416806 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 0.970981 3 0.000511 2014-08-13 17:38:05.416699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=35] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.55 2014-08-13 17:38:05.416819 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Leaving Peering 2014-08-13 17:38:05.416830 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 1.009659 0 0.000000 2014-08-13 17:38:05.416845 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active 2014-08-13 17:38:05.416855 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: In Active, about to call activate 2014-08-13 17:38:05.416868 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.416878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - snap_trimq [] 2014-08-13 17:38:05.416888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - not complete, missing(48) 2014-08-13 17:38:05.416821 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=34] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.55 2014-08-13 17:38:05.416944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=33] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.55 2014-08-13 17:38:05.417070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=32] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.55 2014-08-13 17:38:05.417200 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=31] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.55 2014-08-13 17:38:05.417456 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.417460 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.417334 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=30] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.55 2014-08-13 17:38:05.417463 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01340 pg_info(1 pgs e194487:24.9b) v4 2014-08-13 17:38:05.417493 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.350 2014-08-13 17:38:05.417496 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01340 2014-08-13 17:38:05.417532 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.417536 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.417464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=29] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.55 2014-08-13 17:38:05.417709 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.417711 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.417712 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01dc0 pg_info(1 pgs e194487:24.3d) v4 2014-08-13 17:38:05.417597 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=28] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.55 2014-08-13 17:38:05.417734 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.20 2014-08-13 17:38:05.417737 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01dc0 2014-08-13 17:38:05.417760 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.417761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.417824 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - starting recovery 2014-08-13 17:38:05.417840 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] 2014-08-13 17:38:05.417733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=27] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.55 2014-08-13 17:38:05.417866 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.417884 7fe99ebe8700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 2014-08-13 17:38:05.417869 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=26] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.55 2014-08-13 17:38:05.418008 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.418010 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.418012 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01f80 pg_info(1 pgs e194487:24.9b) v4 2014-08-13 17:38:05.418036 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.291 2014-08-13 17:38:05.418039 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01f80 2014-08-13 17:38:05.418065 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.418067 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.418121 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 sending log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 17:38:05.418137 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing 2014-08-13 17:38:05.418006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=25] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.55 2014-08-13 17:38:05.418156 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.418171 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.418183 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 is up to date, queueing in pending_activators 2014-08-13 17:38:05.418243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.418256 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.418268 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 is up to date, queueing in pending_activators 2014-08-13 17:38:05.418144 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=24] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.55 2014-08-13 17:38:05.418299 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.418312 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.418327 7fe99ebe8700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 2014-08-13 17:38:05.418289 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=23] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.55 2014-08-13 17:38:05.418431 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=22] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.55 2014-08-13 17:38:05.418594 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 sending log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 17:38:05.418610 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing 2014-08-13 17:38:05.418627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.418574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=21] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.55 2014-08-13 17:38:05.418664 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery primary has 48 missing 2014-08-13 17:38:05.418726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.4 2014-08-13 17:38:05.418847 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.418850 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.418851 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02140 pg_info(1 pgs e194487:24.3d) v4 2014-08-13 17:38:05.418804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.4 2014-08-13 17:38:05.418719 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=20] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.55 2014-08-13 17:38:05.418877 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.303 2014-08-13 17:38:05.418880 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02140 2014-08-13 17:38:05.418907 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.418908 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.418864 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.4 2014-08-13 17:38:05.418924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.4 2014-08-13 17:38:05.418866 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=19] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.55 2014-08-13 17:38:05.418982 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.4 2014-08-13 17:38:05.419040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.4 2014-08-13 17:38:05.419099 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.4 2014-08-13 17:38:05.419015 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=18] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.55 2014-08-13 17:38:05.419157 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.4 2014-08-13 17:38:05.419220 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.4 2014-08-13 17:38:05.419165 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=17] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.55 2014-08-13 17:38:05.419278 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.4 2014-08-13 17:38:05.419336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.4 2014-08-13 17:38:05.419395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.4 2014-08-13 17:38:05.419455 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.419457 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.419458 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc800 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.419464 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.419467 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.419468 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.419470 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.419320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=16] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.55 2014-08-13 17:38:05.419453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.4 2014-08-13 17:38:05.419511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.419512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.419514 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104440 pg_info(1 pgs e194487:24.3d) v4 2014-08-13 17:38:05.419536 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.25 2014-08-13 17:38:05.419539 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104440 2014-08-13 17:38:05.419511 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.4 2014-08-13 17:38:05.419568 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.419570 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.419570 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.4 2014-08-13 17:38:05.419474 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=15] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.55 2014-08-13 17:38:05.419628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.4 2014-08-13 17:38:05.419687 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.4 2014-08-13 17:38:05.419630 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=14] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.55 2014-08-13 17:38:05.419745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.4 2014-08-13 17:38:05.419803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.4 2014-08-13 17:38:05.419861 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.4 2014-08-13 17:38:05.419788 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=13] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.55 2014-08-13 17:38:05.419922 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.4 2014-08-13 17:38:05.419983 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.4 2014-08-13 17:38:05.420041 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.4 2014-08-13 17:38:05.419947 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=12] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.55 2014-08-13 17:38:05.420101 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.4 2014-08-13 17:38:05.420161 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.4 2014-08-13 17:38:05.420108 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=11] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.55 2014-08-13 17:38:05.420223 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.4 2014-08-13 17:38:05.420282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.4 2014-08-13 17:38:05.420342 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.4 2014-08-13 17:38:05.420276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=10] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.55 2014-08-13 17:38:05.420402 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.4 2014-08-13 17:38:05.420460 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.4 2014-08-13 17:38:05.420519 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.4 2014-08-13 17:38:05.420444 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=9] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.55 2014-08-13 17:38:05.420579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.4 2014-08-13 17:38:05.420638 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.4 2014-08-13 17:38:05.420698 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.4 2014-08-13 17:38:05.420612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=8] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.55 2014-08-13 17:38:05.420758 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.4 2014-08-13 17:38:05.420818 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.4 2014-08-13 17:38:05.420878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.4 2014-08-13 17:38:05.420783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=7] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.55 2014-08-13 17:38:05.420938 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.4 2014-08-13 17:38:05.420998 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.4 2014-08-13 17:38:05.421057 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.4 2014-08-13 17:38:05.420975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=6] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.55 2014-08-13 17:38:05.421117 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.4 2014-08-13 17:38:05.421177 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.4 2014-08-13 17:38:05.421239 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.4 2014-08-13 17:38:05.421151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=5] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.55 2014-08-13 17:38:05.421297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.4 2014-08-13 17:38:05.421355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.4 2014-08-13 17:38:05.421415 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.4 2014-08-13 17:38:05.421329 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=4] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.55 2014-08-13 17:38:05.421475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.4 2014-08-13 17:38:05.421534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.4 2014-08-13 17:38:05.421507 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=3] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.55 2014-08-13 17:38:05.421594 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 17:38:05.421698 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 331 as a source 2014-08-13 17:38:05.421793 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.421686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=2] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.55 2014-08-13 17:38:05.421857 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.421915 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.421973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.422089 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.421900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=1] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.55 2014-08-13 17:38:05.422360 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 17:38:05.422266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.422791 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 59 as a source 2014-08-13 17:38:05.422843 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423052 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.422981 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423119 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423320 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423595 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423717 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423815 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423916 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.423976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424661 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424845 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.424980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425037 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425654 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425769 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425940 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.425997 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426072 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426074 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426076 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5138000 pg_info(1 pgs e194487:24.3d) v4 2014-08-13 17:38:05.426103 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.362 2014-08-13 17:38:05.426107 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5138000 2014-08-13 17:38:05.426054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426141 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426112 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426232 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426333 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426350 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426351 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5490000 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.426360 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.426363 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.426364 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426365 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426348 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426434 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426436 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426437 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f00c40 pg_notify(22.775(31) epoch 194487) v5 2014-08-13 17:38:05.426405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426480 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.0 2014-08-13 17:38:05.426487 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f00c40 2014-08-13 17:38:05.426465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426519 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.426535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.426523 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426581 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 17:38:05.426677 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 332 as a source 2014-08-13 17:38:05.426736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.332 2014-08-13 17:38:05.426796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=47] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.332 2014-08-13 17:38:05.423626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.426859 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=46] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.332 2014-08-13 17:38:05.426924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=45] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.332 2014-08-13 17:38:05.426988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=44] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.332 2014-08-13 17:38:05.426892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.427056 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=43] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.332 2014-08-13 17:38:05.427124 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=42] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.332 2014-08-13 17:38:05.427198 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=41] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.332 2014-08-13 17:38:05.427094 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.427274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=40] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.332 2014-08-13 17:38:05.427350 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=39] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.332 2014-08-13 17:38:05.427285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.427426 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=38] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.332 2014-08-13 17:38:05.427506 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=37] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.332 2014-08-13 17:38:05.427472 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.427586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=36] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.332 2014-08-13 17:38:05.427667 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=35] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.332 2014-08-13 17:38:05.427751 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=34] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.332 2014-08-13 17:38:05.427657 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.427837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=33] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.332 2014-08-13 17:38:05.427926 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=32] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.332 2014-08-13 17:38:05.427840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428015 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=31] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.332 2014-08-13 17:38:05.428108 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=30] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.332 2014-08-13 17:38:05.428023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428205 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=29] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.332 2014-08-13 17:38:05.428208 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=28] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.332 2014-08-13 17:38:05.428395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=27] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.332 2014-08-13 17:38:05.428394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428493 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=26] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.332 2014-08-13 17:38:05.428591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=25] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.332 2014-08-13 17:38:05.428579 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=24] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.332 2014-08-13 17:38:05.428796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=23] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.332 2014-08-13 17:38:05.428762 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.428901 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=22] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.332 2014-08-13 17:38:05.429008 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=21] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.332 2014-08-13 17:38:05.428945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.429116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=20] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.332 2014-08-13 17:38:05.429127 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.429229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=19] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.332 2014-08-13 17:38:05.429339 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=18] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.332 2014-08-13 17:38:05.429313 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.429453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=17] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.332 2014-08-13 17:38:05.429497 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.429568 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=16] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.332 2014-08-13 17:38:05.429689 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=15] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.332 2014-08-13 17:38:05.429686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.429808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=14] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.332 2014-08-13 17:38:05.429928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=13] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.332 2014-08-13 17:38:05.429870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=12] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.332 2014-08-13 17:38:05.430053 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=11] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.332 2014-08-13 17:38:05.430239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430304 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=10] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.332 2014-08-13 17:38:05.430431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=9] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.332 2014-08-13 17:38:05.430423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=8] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.332 2014-08-13 17:38:05.430606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=7] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.332 2014-08-13 17:38:05.430825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=6] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.332 2014-08-13 17:38:05.430789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.430960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=5] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.332 2014-08-13 17:38:05.430971 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431113 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=4] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.332 2014-08-13 17:38:05.431154 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=3] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.332 2014-08-13 17:38:05.431340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=2] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.332 2014-08-13 17:38:05.431535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=1] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.332 2014-08-13 17:38:05.431524 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431772 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.431775 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.431776 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df740 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.431782 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.431785 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.431786 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.431787 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.431810 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.431811 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.431812 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb480 pg_info(1 pgs e194487:24.30b) v4 2014-08-13 17:38:05.431836 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.286 2014-08-13 17:38:05.431839 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb480 2014-08-13 17:38:05.431678 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 17:38:05.431868 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.431870 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.431707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431856 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 337 as a source 2014-08-13 17:38:05.431889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.431999 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.337 2014-08-13 17:38:05.432073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.432145 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.337 2014-08-13 17:38:05.432293 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.337 2014-08-13 17:38:05.432259 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.432437 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.337 2014-08-13 17:38:05.432441 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.432582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.337 2014-08-13 17:38:05.432624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.432727 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.337 2014-08-13 17:38:05.432808 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.432871 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.337 2014-08-13 17:38:05.433015 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.337 2014-08-13 17:38:05.432991 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.433252 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.433254 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.433256 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb800 pg_info(1 pgs e194487:24.30b) v4 2014-08-13 17:38:05.433286 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.10 2014-08-13 17:38:05.433289 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb800 2014-08-13 17:38:05.433160 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.337 2014-08-13 17:38:05.433320 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.433321 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.433173 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.433306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.337 2014-08-13 17:38:05.433357 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.433451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.337 2014-08-13 17:38:05.433541 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.433596 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.337 2014-08-13 17:38:05.433739 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.337 2014-08-13 17:38:05.433725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.433883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.337 2014-08-13 17:38:05.433908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434027 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.337 2014-08-13 17:38:05.434092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434172 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.337 2014-08-13 17:38:05.434278 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.337 2014-08-13 17:38:05.434463 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.337 2014-08-13 17:38:05.434462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434607 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.337 2014-08-13 17:38:05.434645 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434752 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.337 2014-08-13 17:38:05.434829 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.434897 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.337 2014-08-13 17:38:05.435041 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.337 2014-08-13 17:38:05.435012 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.435190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.337 2014-08-13 17:38:05.435197 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.435336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.337 2014-08-13 17:38:05.435381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.435480 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.337 2014-08-13 17:38:05.435565 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.435625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.337 2014-08-13 17:38:05.435768 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.337 2014-08-13 17:38:05.435748 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.435911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.337 2014-08-13 17:38:05.435930 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.337 2014-08-13 17:38:05.436114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.337 2014-08-13 17:38:05.436301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.337 2014-08-13 17:38:05.436490 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.337 2014-08-13 17:38:05.436485 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436634 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.337 2014-08-13 17:38:05.436669 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.337 2014-08-13 17:38:05.436852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.436923 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.337 2014-08-13 17:38:05.437068 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.337 2014-08-13 17:38:05.437036 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.437215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.337 2014-08-13 17:38:05.437220 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.437359 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.337 2014-08-13 17:38:05.437404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.437504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.337 2014-08-13 17:38:05.437648 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.337 2014-08-13 17:38:05.437588 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 17:38:05.437792 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.337 2014-08-13 17:38:05.437818 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 351 as a source 2014-08-13 17:38:05.437937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.337 2014-08-13 17:38:05.438003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.351 2014-08-13 17:38:05.438082 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.337 2014-08-13 17:38:05.438192 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.351 2014-08-13 17:38:05.438247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.337 2014-08-13 17:38:05.438391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.337 2014-08-13 17:38:05.438376 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.351 2014-08-13 17:38:05.438536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.337 2014-08-13 17:38:05.438560 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.351 2014-08-13 17:38:05.438681 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.337 2014-08-13 17:38:05.438744 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.351 2014-08-13 17:38:05.438825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.337 2014-08-13 17:38:05.438929 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.351 2014-08-13 17:38:05.438969 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 17:38:05.439150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 348 as a source 2014-08-13 17:38:05.439113 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.351 2014-08-13 17:38:05.439297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.439299 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.351 2014-08-13 17:38:05.439442 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.439483 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.351 2014-08-13 17:38:05.439593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.439668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.351 2014-08-13 17:38:05.439740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.439883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.439870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.351 2014-08-13 17:38:05.440026 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440055 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.351 2014-08-13 17:38:05.440170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440242 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.351 2014-08-13 17:38:05.440317 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440461 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.351 2014-08-13 17:38:05.440604 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440610 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.351 2014-08-13 17:38:05.440748 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.351 2014-08-13 17:38:05.440892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.440978 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.351 2014-08-13 17:38:05.441036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.441178 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.441162 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.351 2014-08-13 17:38:05.441324 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.441349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.351 2014-08-13 17:38:05.441468 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.441534 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.351 2014-08-13 17:38:05.441612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442021 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.441718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.351 2014-08-13 17:38:05.442168 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442176 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.351 2014-08-13 17:38:05.442315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442365 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.351 2014-08-13 17:38:05.442459 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.351 2014-08-13 17:38:05.442602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.351 2014-08-13 17:38:05.442888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.442917 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.351 2014-08-13 17:38:05.443032 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.351 2014-08-13 17:38:05.443175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443322 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.351 2014-08-13 17:38:05.443465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443471 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.351 2014-08-13 17:38:05.443610 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443656 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.351 2014-08-13 17:38:05.443754 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.443840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.351 2014-08-13 17:38:05.443897 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444042 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444024 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.351 2014-08-13 17:38:05.444188 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444211 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.351 2014-08-13 17:38:05.444332 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444396 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.351 2014-08-13 17:38:05.444476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444580 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.351 2014-08-13 17:38:05.444620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444766 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.351 2014-08-13 17:38:05.444911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.444951 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.351 2014-08-13 17:38:05.445055 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445135 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.351 2014-08-13 17:38:05.445202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445322 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.351 2014-08-13 17:38:05.445419 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445507 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.351 2014-08-13 17:38:05.445588 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.351 2014-08-13 17:38:05.445732 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.445875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.351 2014-08-13 17:38:05.446020 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.446061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.351 2014-08-13 17:38:05.446163 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.446248 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.351 2014-08-13 17:38:05.446311 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.446576 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.446579 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.446580 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51055c0 pg_info(1 pgs e194487:24.34) v4 2014-08-13 17:38:05.446456 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.446606 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.26 2014-08-13 17:38:05.446609 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51055c0 2014-08-13 17:38:05.446432 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.351 2014-08-13 17:38:05.446639 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.446640 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.446598 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 17:38:05.446615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.351 2014-08-13 17:38:05.446777 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound 2014-08-13 17:38:05.446798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.351 2014-08-13 17:38:05.446921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.446982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.351 2014-08-13 17:38:05.447067 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound: built 331,332,337,348,379 2014-08-13 17:38:05.447166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.351 2014-08-13 17:38:05.447216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Activate Finished 2014-08-13 17:38:05.447361 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.447353 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.351 2014-08-13 17:38:05.447507 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.447538 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.351 2014-08-13 17:38:05.447652 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.447852 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.447722 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.351 2014-08-13 17:38:05.447935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 17:38:05.447907 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.351 2014-08-13 17:38:05.448110 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448119 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap 2014-08-13 17:38:05.448136 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map 2014-08-13 17:38:05.448159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters 2014-08-13 17:38:05.448215 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448227 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448238 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.351 2014-08-13 17:38:05.448259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 17:38:05.448290 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448295 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.448305 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.448315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448325 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.448334 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.448342 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.448381 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:2653 2014-08-13 17:38:05.448390 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.448398 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448407 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448417 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448434 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 17:38:05.448447 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.448279 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.351 2014-08-13 17:38:05.448298 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.448461 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448472 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.448481 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.448489 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448498 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448507 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448525 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 17:38:05.448536 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448539 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.448548 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.448557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448566 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.448574 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.448581 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.448590 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:3000 2014-08-13 17:38:05.448598 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.448606 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448614 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448623 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448485 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] flushed 2014-08-13 17:38:05.448462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.351 2014-08-13 17:38:05.448636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 17:38:05.448649 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448652 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.448662 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448671 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.448680 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.448688 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448705 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 2014-08-13 17:38:05.448736 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Started advmap 2014-08-13 17:38:05.448749 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448759 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map 2014-08-13 17:38:05.448809 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters 2014-08-13 17:38:05.448819 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448646 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.351 2014-08-13 17:38:05.448827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448838 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448852 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 17:38:05.448864 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448868 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.448877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.448886 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.448895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.448903 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.448911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.448921 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.448933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 17:38:05.448946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.448949 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 17:38:05.448959 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 17:38:05.448969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,338) went down 2014-08-13 17:38:05.448979 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 17:38:05.448987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 17:38:05.448829 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.351 2014-08-13 17:38:05.449163 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 17:38:05.449171 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 17:38:05.449012 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.351 2014-08-13 17:38:05.449263 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2596 2014-08-13 17:38:05.449273 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 17:38:05.449282 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.449291 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.449311 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.449326 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 2014-08-13 17:38:05.449343 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.449347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Started advmap 2014-08-13 17:38:05.449357 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.449367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map 2014-08-13 17:38:05.449198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.351 2014-08-13 17:38:05.449446 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters 2014-08-13 17:38:05.449456 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.449466 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.449476 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.449491 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 17:38:05.449503 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.449507 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.449516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.449525 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.449534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.449542 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.449381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.351 2014-08-13 17:38:05.449559 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.449570 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.449584 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 2014-08-13 17:38:05.449600 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.449603 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 17:38:05.449613 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.449622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 17:38:05.449663 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 17:38:05.449673 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.449681 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.449691 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.449696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 61 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.449710 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 17:38:05.449720 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.449726 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4e3d800 192.168.1.107:6809/53141 2014-08-13 17:38:05.449564 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.351 2014-08-13 17:38:05.449749 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.6 has 194487 2014-08-13 17:38:05.449751 7fe99ebe8700 7 osd.4 194487 do_notify osd 6 on 1 PGs 2014-08-13 17:38:05.449838 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:05.449844 7fe99ebe8700 7 osd.4 194487 do_notify osd 45 on 1 PGs 2014-08-13 17:38:05.449878 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:05.449880 7fe99ebe8700 7 osd.4 194487 do_notify osd 285 on 1 PGs 2014-08-13 17:38:05.449893 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4d31e40 192.168.1.104:6802/61960 2014-08-13 17:38:05.449945 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.343 has 194487 2014-08-13 17:38:05.449947 7fe99ebe8700 7 osd.4 194487 do_notify osd 343 on 1 PGs 2014-08-13 17:38:05.449748 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 17:38:05.450004 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:05.450011 7fe99ebe8700 7 osd.4 194487 do_notify osd 374 on 1 PGs 2014-08-13 17:38:05.450029 7fe99ebe8700 20 osd.4 194487 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 332 2014-08-13 17:38:05.450037 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing 2014-08-13 17:38:05.450068 7fe99ebe8700 20 osd.4 194487 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 337 2014-08-13 17:38:05.450075 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4d3c780 192.168.1.104:6804/62197 2014-08-13 17:38:05.450093 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.337 has 194487 2014-08-13 17:38:05.450000 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound 2014-08-13 17:38:05.450213 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 61 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.450238 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 17:38:05.450250 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.450257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 45 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.450270 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 45, already have info from that osd, ignoring 2014-08-13 17:38:05.450279 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.450284 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) 2014-08-13 17:38:05.450297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: got info from osd.304 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) 2014-08-13 17:38:05.450312 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 1.135807 8 0.009620 2014-08-13 17:38:05.450327 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive 2014-08-13 17:38:05.450355 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.450186 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.450367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.450417 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] activate - not complete, missing(61) 2014-08-13 17:38:05.450370 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound: built 55,59,351,376 2014-08-13 17:38:05.450552 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Activate Finished 2014-08-13 17:38:05.450736 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.450919 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.451101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.451313 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] state: Activate Finished 2014-08-13 17:38:05.451344 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.451379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 2014-08-13 17:38:05.451399 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.451405 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Peering advmap 2014-08-13 17:38:05.451415 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.451417 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.451427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Started advmap 2014-08-13 17:38:05.451433 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) 2014-08-13 17:38:05.451440 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.451449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: got info from osd.6 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) 2014-08-13 17:38:05.451452 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map 2014-08-13 17:38:05.451463 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 1.040760 9 0.000549 2014-08-13 17:38:05.451464 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle ActMap primary 2014-08-13 17:38:05.451475 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive 2014-08-13 17:38:05.451476 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194487:22526 2014-08-13 17:38:05.451487 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.451487 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] take_waiters 2014-08-13 17:38:05.451498 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.451500 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 1.026821 6 0.000961 2014-08-13 17:38:05.451508 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] activate - not complete, missing(53) 2014-08-13 17:38:05.451514 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Leaving Peering 2014-08-13 17:38:05.451525 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 1.096242 0 0.000000 2014-08-13 17:38:05.451540 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active 2014-08-13 17:38:05.451550 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: In Active, about to call activate 2014-08-13 17:38:05.451563 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.451574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - snap_trimq [] 2014-08-13 17:38:05.451583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - not complete, missing(51) 2014-08-13 17:38:05.451656 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.451860 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] flushed 2014-08-13 17:38:05.452059 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] _activate_committed 194487 telling primary 2014-08-13 17:38:05.452076 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x49e39c0 already has epoch 194487 2014-08-13 17:38:05.452096 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] flushed 2014-08-13 17:38:05.452379 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - starting recovery 2014-08-13 17:38:05.452396 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] 2014-08-13 17:38:05.452420 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.452423 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.452425 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825780 pg_info(1 pgs e194487:24.1bb) v4 2014-08-13 17:38:05.452423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.452437 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 is up to date, queueing in pending_activators 2014-08-13 17:38:05.452451 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.332 2014-08-13 17:38:05.452454 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825780 2014-08-13 17:38:05.452483 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.452485 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.452478 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.452491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.452512 7fe99f5e9700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 2014-08-13 17:38:05.452735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 sending log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 17:38:05.452754 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing 2014-08-13 17:38:05.452774 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.452788 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 17:38:05.452803 7fe99f5e9700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 2014-08-13 17:38:05.453032 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 sending log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 17:38:05.453051 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing 2014-08-13 17:38:05.453067 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.453081 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) 2014-08-13 17:38:05.453093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 is up to date, queueing in pending_activators 2014-08-13 17:38:05.453128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate 2014-08-13 17:38:05.453166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery primary has 51 missing 2014-08-13 17:38:05.453226 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] state: Activate Finished 2014-08-13 17:38:05.453237 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.4 2014-08-13 17:38:05.453304 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.453326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) 2014-08-13 17:38:05.453344 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: got info from osd.17 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) 2014-08-13 17:38:05.453303 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.4 2014-08-13 17:38:05.453358 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 1.117572 8 0.000660 2014-08-13 17:38:05.453370 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive 2014-08-13 17:38:05.453382 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.453392 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.453401 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] activate - not complete, missing(49) 2014-08-13 17:38:05.453366 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.4 2014-08-13 17:38:05.453428 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.4 2014-08-13 17:38:05.453490 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.4 2014-08-13 17:38:05.453551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.4 2014-08-13 17:38:05.453608 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] _activate_committed 194487 telling primary 2014-08-13 17:38:05.453631 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4e3d800 already has epoch 194487 2014-08-13 17:38:05.453649 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] flushed 2014-08-13 17:38:05.453612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.4 2014-08-13 17:38:05.453674 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.4 2014-08-13 17:38:05.453735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.4 2014-08-13 17:38:05.453798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.4 2014-08-13 17:38:05.453861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.4 2014-08-13 17:38:05.453924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.4 2014-08-13 17:38:05.453987 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.4 2014-08-13 17:38:05.454050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.4 2014-08-13 17:38:05.454114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.4 2014-08-13 17:38:05.454191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] state: Activate Finished 2014-08-13 17:38:05.454180 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.4 2014-08-13 17:38:05.454273 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.454290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) 2014-08-13 17:38:05.454244 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.4 2014-08-13 17:38:05.454306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.45 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) 2014-08-13 17:38:05.454319 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.085305 8 0.000489 2014-08-13 17:38:05.454332 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive 2014-08-13 17:38:05.454343 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.454354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.454361 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454365 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c163c0 pg_info(1 pgs e194487:24.33a) v4 2014-08-13 17:38:05.454307 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.4 2014-08-13 17:38:05.454364 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] activate - not complete, missing(50) 2014-08-13 17:38:05.454390 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.33a) v4 from osd.304 2014-08-13 17:38:05.454393 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c163c0 2014-08-13 17:38:05.454418 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454420 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454370 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.4 2014-08-13 17:38:05.454432 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.4 2014-08-13 17:38:05.454561 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454564 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454565 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_info(1 pgs e194487:24.1bb) v4 2014-08-13 17:38:05.454517 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.4 2014-08-13 17:38:05.454590 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.337 2014-08-13 17:38:05.454593 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 2014-08-13 17:38:05.454619 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454620 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.4 2014-08-13 17:38:05.454642 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.4 2014-08-13 17:38:05.454710 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454713 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454714 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822840 pg_info(1 pgs e194487:24.cd) v4 2014-08-13 17:38:05.454742 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.cd) v4 from osd.6 2014-08-13 17:38:05.454745 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822840 2014-08-13 17:38:05.454704 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.4 2014-08-13 17:38:05.454773 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.454775 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.454765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.4 2014-08-13 17:38:05.454827 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.4 2014-08-13 17:38:05.454890 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.4 2014-08-13 17:38:05.454954 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.4 2014-08-13 17:38:05.454997 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] _activate_committed 194487 telling primary 2014-08-13 17:38:05.455026 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing 2014-08-13 17:38:05.455050 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] flushed 2014-08-13 17:38:05.455017 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.4 2014-08-13 17:38:05.455080 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.4 2014-08-13 17:38:05.455180 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] state: Activate Finished 2014-08-13 17:38:05.455143 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.4 2014-08-13 17:38:05.455245 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.455208 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.4 2014-08-13 17:38:05.455275 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.4 2014-08-13 17:38:05.455339 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.4 2014-08-13 17:38:05.455266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 379 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.455402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.4 2014-08-13 17:38:05.455465 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.4 2014-08-13 17:38:05.455529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.455531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.455532 7fe9a63f4700 20 osd.4 194487 _dispatch 0x549d340 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.455541 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.455544 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.455546 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.455547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.455432 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 379, already have info from that osd, ignoring 2014-08-13 17:38:05.455580 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.455528 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.4 2014-08-13 17:38:05.455582 7fe9aa308700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] _activate_committed 194487 telling primary 2014-08-13 17:38:05.455602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.455604 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.455605 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4a0af40 pg_info(1 pgs e194487:24.1bb) v4 2014-08-13 17:38:05.455611 7fe9aa308700 20 osd.4 194487 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing 2014-08-13 17:38:05.455605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) 2014-08-13 17:38:05.455630 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.348 2014-08-13 17:38:05.455633 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4a0af40 2014-08-13 17:38:05.455629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.285 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) 2014-08-13 17:38:05.455632 7fe9aa308700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] flushed 2014-08-13 17:38:05.455590 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.4 2014-08-13 17:38:05.455646 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.086102 8 0.009439 2014-08-13 17:38:05.455659 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.455661 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.455660 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive 2014-08-13 17:38:05.455674 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.455685 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.455695 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] activate - not complete, missing(50) 2014-08-13 17:38:05.455652 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.4 2014-08-13 17:38:05.455715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.4 2014-08-13 17:38:05.455776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.4 2014-08-13 17:38:05.455837 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.4 2014-08-13 17:38:05.455898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.4 2014-08-13 17:38:05.455970 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.455973 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.455974 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4265240 pg_info(1 pgs e194487:24.2a1) v4 2014-08-13 17:38:05.456004 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2a1) v4 from osd.17 2014-08-13 17:38:05.456007 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4265240 2014-08-13 17:38:05.456010 7fe9a45f1700 10 osd.4 194487 note_peer_epoch osd.51 has 194487 2014-08-13 17:38:05.456017 7fe9a45f1700 20 osd.4 194487 _share_map_outgoing 0x4e39e40 already has epoch 194487 2014-08-13 17:38:05.456034 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.456036 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.456047 7fe9a31ef700 20 osd.4 194487 _share_map_outgoing 0x4e39e40 already has epoch 194487 2014-08-13 17:38:05.455959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.4 2014-08-13 17:38:05.456073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.4 2014-08-13 17:38:05.456136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.4 2014-08-13 17:38:05.456202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.4 2014-08-13 17:38:05.456263 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.4 2014-08-13 17:38:05.456324 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.4 2014-08-13 17:38:05.456385 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.4 2014-08-13 17:38:05.456449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] state: Activate Finished 2014-08-13 17:38:05.456446 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.4 2014-08-13 17:38:05.456515 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.456536 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.456539 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.456540 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01dc0 pg_info(1 pgs e194487:24.1bb) v4 2014-08-13 17:38:05.456534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 330 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.456552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 330, already have info from that osd, ignoring 2014-08-13 17:38:05.456563 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.456564 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.331 2014-08-13 17:38:05.456567 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01dc0 2014-08-13 17:38:05.456570 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) 2014-08-13 17:38:05.456594 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.456596 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.456584 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: got info from osd.330 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) 2014-08-13 17:38:05.456507 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 17:38:05.456599 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 1.107330 8 0.000609 2014-08-13 17:38:05.456611 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive 2014-08-13 17:38:05.456623 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.456634 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.456644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] activate - not complete, missing(64) 2014-08-13 17:38:05.456607 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] activate: adding 50 as a source 2014-08-13 17:38:05.456668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.50 2014-08-13 17:38:05.456780 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.456782 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.456783 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02f40 pg_info(1 pgs e194487:24.369) v4 2014-08-13 17:38:05.456771 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] _activate_committed 194487 telling primary 2014-08-13 17:38:05.456731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=50] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.50 2014-08-13 17:38:05.456801 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing 2014-08-13 17:38:05.456812 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.369) v4 from osd.45 2014-08-13 17:38:05.456816 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02f40 2014-08-13 17:38:05.456823 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] flushed 2014-08-13 17:38:05.456845 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.456847 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.456796 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=49] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.50 2014-08-13 17:38:05.456864 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=48] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.50 2014-08-13 17:38:05.456931 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=47] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.50 2014-08-13 17:38:05.457001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=46] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.50 2014-08-13 17:38:05.457072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=45] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.50 2014-08-13 17:38:05.457149 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=44] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.50 2014-08-13 17:38:05.457233 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=43] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.50 2014-08-13 17:38:05.457311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=42] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.50 2014-08-13 17:38:05.457397 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] state: Activate Finished 2014-08-13 17:38:05.457468 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.457391 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=41] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.50 2014-08-13 17:38:05.457497 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) 2014-08-13 17:38:05.457512 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: got info from osd.343 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) 2014-08-13 17:38:05.457525 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 1.058548 8 0.000465 2014-08-13 17:38:05.457536 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive 2014-08-13 17:38:05.457547 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.457480 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=40] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.50 2014-08-13 17:38:05.457557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.457567 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] activate - not complete, missing(41) 2014-08-13 17:38:05.457612 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.457614 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.457616 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb480 pg_info(1 pgs e194487:24.1e6) v4 2014-08-13 17:38:05.457640 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e6) v4 from osd.285 2014-08-13 17:38:05.457643 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb480 2014-08-13 17:38:05.457563 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=39] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.50 2014-08-13 17:38:05.457668 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.457669 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.457648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=38] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.50 2014-08-13 17:38:05.457732 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] _activate_committed 194487 telling primary 2014-08-13 17:38:05.457759 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4ec9b80 already has epoch 194487 2014-08-13 17:38:05.457778 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] flushed 2014-08-13 17:38:05.457735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=37] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.50 2014-08-13 17:38:05.457825 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=36] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.50 2014-08-13 17:38:05.457916 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=35] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.50 2014-08-13 17:38:05.458051 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458054 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458056 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5460b40 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.458061 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.458064 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.458065 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458066 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458008 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=34] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.50 2014-08-13 17:38:05.458099 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458101 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458102 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102140 pg_notify(22.99(1) epoch 194487) v5 2014-08-13 17:38:05.458125 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.363 2014-08-13 17:38:05.458128 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102140 2014-08-13 17:38:05.458152 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458153 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=33] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.50 2014-08-13 17:38:05.458200 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=32] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.50 2014-08-13 17:38:05.458362 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] state: Activate Finished 2014-08-13 17:38:05.458297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=31] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.50 2014-08-13 17:38:05.458421 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.458438 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) 2014-08-13 17:38:05.458454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: got info from osd.346 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) 2014-08-13 17:38:05.458467 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 1.137638 9 0.000593 2014-08-13 17:38:05.458479 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive 2014-08-13 17:38:05.458397 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=30] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.50 2014-08-13 17:38:05.458490 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.458501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.458511 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] activate - not complete, missing(66) 2014-08-13 17:38:05.458499 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=29] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.50 2014-08-13 17:38:05.458604 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458611 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458613 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_info(1 pgs e194487:24.3f7) v4 2014-08-13 17:38:05.458636 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3f7) v4 from osd.330 2014-08-13 17:38:05.458639 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 2014-08-13 17:38:05.458662 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.458664 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.458651 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] _activate_committed 194487 telling primary 2014-08-13 17:38:05.458679 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4d31e40 already has epoch 194487 2014-08-13 17:38:05.458700 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] flushed 2014-08-13 17:38:05.458609 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=28] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.50 2014-08-13 17:38:05.458713 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=27] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.50 2014-08-13 17:38:05.458820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=26] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.50 2014-08-13 17:38:05.458928 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=25] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.50 2014-08-13 17:38:05.459037 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=24] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.50 2014-08-13 17:38:05.459149 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=23] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.50 2014-08-13 17:38:05.459291 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] state: Activate Finished 2014-08-13 17:38:05.459344 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.459347 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.459348 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104600 pg_notify(22.39d(29) epoch 194487) v5 2014-08-13 17:38:05.459351 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.459374 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.363 2014-08-13 17:38:05.459264 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=22] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.50 2014-08-13 17:38:05.459376 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104600 2014-08-13 17:38:05.459367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) 2014-08-13 17:38:05.459382 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: got info from osd.364 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) 2014-08-13 17:38:05.459405 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.459397 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 1.163467 8 0.000572 2014-08-13 17:38:05.459407 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.459409 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive 2014-08-13 17:38:05.459420 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.459430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.459440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] activate - not complete, missing(48) 2014-08-13 17:38:05.459472 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.459474 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.459475 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ca4c0 pg_info(1 pgs e194487:24.206) v4 2014-08-13 17:38:05.459378 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=21] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.50 2014-08-13 17:38:05.459497 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.206) v4 from osd.343 2014-08-13 17:38:05.459500 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ca4c0 2014-08-13 17:38:05.459523 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.459525 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.459585 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] _activate_committed 194487 telling primary 2014-08-13 17:38:05.459604 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4e4a100 already has epoch 194487 2014-08-13 17:38:05.459494 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=20] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.50 2014-08-13 17:38:05.459622 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] flushed 2014-08-13 17:38:05.459613 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=19] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.50 2014-08-13 17:38:05.459733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=18] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.50 2014-08-13 17:38:05.459854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=17] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.50 2014-08-13 17:38:05.459978 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=16] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.50 2014-08-13 17:38:05.460162 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] state: Activate Finished 2014-08-13 17:38:05.460227 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.460252 7fe9a45f1700 10 osd.4 194487 note_peer_epoch osd.320 now has 194487 2014-08-13 17:38:05.460260 7fe9a45f1700 20 osd.4 194487 _share_map_outgoing 0x4ec8000 already has epoch 194487 2014-08-13 17:38:05.460103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=15] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.50 2014-08-13 17:38:05.460296 7fe9a31ef700 20 osd.4 194487 _share_map_outgoing 0x4ec8000 already has epoch 194487 2014-08-13 17:38:05.460315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) 2014-08-13 17:38:05.460341 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.460343 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.460344 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5105400 MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 2014-08-13 17:38:05.460333 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: got info from osd.374 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) 2014-08-13 17:38:05.460349 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 1.092600 8 0.000844 2014-08-13 17:38:05.460369 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5105400 2014-08-13 17:38:05.460362 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive 2014-08-13 17:38:05.460375 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 17:38:05.460385 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.460386 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.460386 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: In ReplicaActive, about to call activate 2014-08-13 17:38:05.460396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] activate - not complete, missing(52) 2014-08-13 17:38:05.460279 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=14] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.50 2014-08-13 17:38:05.460499 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.460501 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.460503 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51071c0 pg_info(1 pgs e194487:24.e2) v4 2014-08-13 17:38:05.460524 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.e2) v4 from osd.346 2014-08-13 17:38:05.460526 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51071c0 2014-08-13 17:38:05.460408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=13] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.50 2014-08-13 17:38:05.460549 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.460551 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.460576 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] _activate_committed 194487 telling primary 2014-08-13 17:38:05.460597 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4cfc620 already has epoch 194487 2014-08-13 17:38:05.460613 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] flushed 2014-08-13 17:38:05.460537 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=12] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.50 2014-08-13 17:38:05.460670 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=11] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.50 2014-08-13 17:38:05.460804 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=10] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.50 2014-08-13 17:38:05.460940 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=9] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.50 2014-08-13 17:38:05.461151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] state: Activate Finished 2014-08-13 17:38:05.461077 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=8] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.50 2014-08-13 17:38:05.461228 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 293 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) 2014-08-13 17:38:05.461264 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 293, already have info from that osd, ignoring 2014-08-13 17:38:05.461275 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461281 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 314 notify: (query_epoch:194487, epoch_sent:194487, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.461294 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 314, already have info from that osd, ignoring 2014-08-13 17:38:05.461303 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 320 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.461319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 320, already have info from that osd, ignoring 2014-08-13 17:38:05.461329 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461334 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 376 notify: (query_epoch:194487, epoch_sent:194487, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.461346 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 17:38:05.461355 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461221 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=7] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.50 2014-08-13 17:38:05.461361 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=6] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.50 2014-08-13 17:38:05.461495 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] _activate_committed 194487 telling primary 2014-08-13 17:38:05.461519 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing 2014-08-13 17:38:05.461535 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] flushed 2014-08-13 17:38:05.461360 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 301 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.461602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.461606 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.461607 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_info(1 pgs e194487:24.1de) v4 2014-08-13 17:38:05.461629 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1de) v4 from osd.364 2014-08-13 17:38:05.461632 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 2014-08-13 17:38:05.461504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=5] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.50 2014-08-13 17:38:05.461655 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.461657 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.461552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 301, already have info from that osd, ignoring 2014-08-13 17:38:05.461725 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461732 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 63 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.461746 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 63, already have info from that osd, ignoring 2014-08-13 17:38:05.461756 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461647 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=4] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.50 2014-08-13 17:38:05.461786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 382 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:05.461798 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 382, already have info from that osd, ignoring 2014-08-13 17:38:05.461807 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461811 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 26 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:05.461825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 26, already have info from that osd, ignoring 2014-08-13 17:38:05.461836 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.461792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=3] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.50 2014-08-13 17:38:05.461842 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.461943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=2] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.50 2014-08-13 17:38:05.462015 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 17:38:05.462185 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=1] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.50 2014-08-13 17:38:05.462191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 20 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.462248 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 17:38:05.462545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.462547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.462381 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 17:38:05.462548 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02140 pg_info(1 pgs e194487:24.2ec) v4 2014-08-13 17:38:05.462550 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 55 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462573 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2ec) v4 from osd.374 2014-08-13 17:38:05.462575 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02140 2014-08-13 17:38:05.462571 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 17:38:05.462581 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462437 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 55 as a source 2014-08-13 17:38:05.462585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 337 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.462600 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.462598 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 17:38:05.462607 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462589 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.462612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 332 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.462743 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.462765 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 332, already have info from that osd, ignoring 2014-08-13 17:38:05.462908 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 24 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 24, already have info from that osd, ignoring 2014-08-13 17:38:05.462934 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 397 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) 2014-08-13 17:38:05.462947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 397, already have info from that osd, ignoring 2014-08-13 17:38:05.462956 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 339 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 339, already have info from that osd, ignoring 2014-08-13 17:38:05.462981 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.462985 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 48 notify: (query_epoch:194487, epoch_sent:194487, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring 2014-08-13 17:38:05.463004 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463008 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 302 notify: (query_epoch:194487, epoch_sent:194487, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.463019 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 302, already have info from that osd, ignoring 2014-08-13 17:38:05.463029 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 371 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.462895 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463045 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 371, already have info from that osd, ignoring 2014-08-13 17:38:05.463055 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463069 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:05.463081 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.463097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:05.463106 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463110 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 20 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463125 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 17:38:05.463134 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463138 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 337 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.463201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463289 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 17:38:05.463434 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 327 notify: (query_epoch:194487, epoch_sent:194487, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 327, already have info from that osd, ignoring 2014-08-13 17:38:05.463460 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463464 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 381 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:05.463476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 381, already have info from that osd, ignoring 2014-08-13 17:38:05.463484 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463353 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463500 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 17:38:05.463509 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 349 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 349, already have info from that osd, ignoring 2014-08-13 17:38:05.463552 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.463567 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] requeue_ops 2014-08-13 17:38:05.463577 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.463593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:05.463607 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.463662 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463785 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:05.463955 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463813 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.463961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 331 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.463972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 331, already have info from that osd, ignoring 2014-08-13 17:38:05.463981 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463985 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:05.463995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 17:38:05.464003 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.463964 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464007 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 378 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.464116 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464208 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.378 activated and committed 2014-08-13 17:38:05.464387 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.464271 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464394 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 373 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.464423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464566 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 17:38:05.464734 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.464739 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 342 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.464750 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 17:38:05.464759 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.464763 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 396 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:05.464773 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 396, already have info from that osd, ignoring 2014-08-13 17:38:05.464782 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.464726 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 375 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.464878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.464955 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 375, already have info from that osd, ignoring 2014-08-13 17:38:05.465122 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 366 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.465137 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 366, already have info from that osd, ignoring 2014-08-13 17:38:05.465146 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465030 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 291 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.465185 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465336 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465333 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 291, already have info from that osd, ignoring 2014-08-13 17:38:05.465509 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465488 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465514 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 367 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.465640 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465691 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.367 activated and committed 2014-08-13 17:38:05.465867 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 373 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) 2014-08-13 17:38:05.465889 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 17:38:05.465898 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465902 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 342 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.465912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 17:38:05.465921 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.465792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.465924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 375 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.465943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.375 activated and committed 2014-08-13 17:38:05.466264 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466268 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 340 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:05.466280 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 340, already have info from that osd, ignoring 2014-08-13 17:38:05.466290 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466249 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466294 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.466401 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466473 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 17:38:05.466650 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 30 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) 2014-08-13 17:38:05.466668 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 30, already have info from that osd, ignoring 2014-08-13 17:38:05.466677 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466554 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466699 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 340 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) 2014-08-13 17:38:05.466711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.340 activated and committed 2014-08-13 17:38:05.466722 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 352 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.466736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 352, already have info from that osd, ignoring 2014-08-13 17:38:05.466745 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466749 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 306 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) 2014-08-13 17:38:05.466761 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 306, already have info from that osd, ignoring 2014-08-13 17:38:05.466771 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.466706 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466774 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 10 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.466859 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.466944 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 10, already have info from that osd, ignoring 2014-08-13 17:38:05.467111 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 362 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.467127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 362, already have info from that osd, ignoring 2014-08-13 17:38:05.467137 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467141 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 58 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.467151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 58, already have info from that osd, ignoring 2014-08-13 17:38:05.467160 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467011 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 311 notify: (query_epoch:194487, epoch_sent:194487, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.467175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 311, already have info from that osd, ignoring 2014-08-13 17:38:05.467185 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467161 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467189 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 365 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.467317 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 17:38:05.467548 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467554 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 368 notify: (query_epoch:194487, epoch_sent:194487, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) 2014-08-13 17:38:05.467565 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 17:38:05.467574 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 365 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485)) 2014-08-13 17:38:05.467587 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 17:38:05.467596 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467600 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 30 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) 2014-08-13 17:38:05.467469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.30 activated and committed 2014-08-13 17:38:05.467622 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467626 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 350 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) 2014-08-13 17:38:05.467772 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 350, already have info from that osd, ignoring 2014-08-13 17:38:05.467979 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.467924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.467984 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.468076 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468156 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 17:38:05.468330 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.468231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 350 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.468383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468535 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468513 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.350 activated and committed 2014-08-13 17:38:05.468696 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.468690 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468701 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 20 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.468842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.468874 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.20 activated and committed 2014-08-13 17:38:05.469044 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.468994 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469049 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 291 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.469146 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.291 activated and committed 2014-08-13 17:38:05.469302 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] all_activated_and_committed 2014-08-13 17:38:05.469454 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] share_pg_info 2014-08-13 17:38:05.469606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469759 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing 2014-08-13 17:38:05.469775 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.469778 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.469779 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db7080 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.469781 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing 2014-08-13 17:38:05.469785 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.469789 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.469790 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.469791 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.469795 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing 2014-08-13 17:38:05.469810 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing 2014-08-13 17:38:05.469880 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.469887 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.469889 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16e40 pg_notify(24.181(33) epoch 194487) v5 2014-08-13 17:38:05.469756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.469918 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.296 2014-08-13 17:38:05.469920 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16e40 2014-08-13 17:38:05.469948 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.469949 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.469824 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194487:25289 2014-08-13 17:38:05.470004 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.469908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.470010 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 303 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.470058 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.470182 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.303 activated and committed 2014-08-13 17:38:05.470357 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.470212 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.470363 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 25 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.470363 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 17:38:05.470614 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.470619 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.470620 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db180 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.470626 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.470630 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.470631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.470632 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.470549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 61 as a source 2014-08-13 17:38:05.470534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.25 activated and committed 2014-08-13 17:38:05.470702 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.470759 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.470761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.470762 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16ac0 pg_notify(22.4df(27) epoch 194487) v5 2014-08-13 17:38:05.470785 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.380 2014-08-13 17:38:05.470787 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16ac0 2014-08-13 17:38:05.470813 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.470815 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.470702 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.470708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 362 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.470854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.470877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.362 activated and committed 2014-08-13 17:38:05.471006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471045 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] all_activated_and_committed 2014-08-13 17:38:05.471158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] share_pg_info 2014-08-13 17:38:05.471392 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c71b80 already has epoch 194487 2014-08-13 17:38:05.471411 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing 2014-08-13 17:38:05.471425 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing 2014-08-13 17:38:05.471442 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing 2014-08-13 17:38:05.471312 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471465 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471456 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194487:29706 2014-08-13 17:38:05.471630 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.471636 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 0 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.471647 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-13 17:38:05.471656 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.471616 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471684 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 286 info: 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.471768 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.471856 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.286 activated and committed 2014-08-13 17:38:05.472027 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.471920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472032 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 10 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.472072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472203 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.10 activated and committed 2014-08-13 17:38:05.472227 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] all_activated_and_committed 2014-08-13 17:38:05.472530 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] share_pg_info 2014-08-13 17:38:05.472705 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing 2014-08-13 17:38:05.472723 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing 2014-08-13 17:38:05.472741 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing 2014-08-13 17:38:05.472682 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472755 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194487:30297 2014-08-13 17:38:05.472934 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.472940 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 26 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) 2014-08-13 17:38:05.472951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.26 activated and committed 2014-08-13 17:38:05.472961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] all_activated_and_committed 2014-08-13 17:38:05.472970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] share_pg_info 2014-08-13 17:38:05.472982 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x592c360 don't know epoch, doing nothing 2014-08-13 17:38:05.472835 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.472998 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a44780 already has epoch 194487 2014-08-13 17:38:05.473020 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing 2014-08-13 17:38:05.473102 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] publish_stats_to_osd 194487:23844 2014-08-13 17:38:05.473128 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.472986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473135 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.473136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473290 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 17:38:05.473437 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.473290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473443 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.473457 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] requeue_ops 2014-08-13 17:38:05.473467 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.473442 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 332 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.473594 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473617 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.332 activated and committed 2014-08-13 17:38:05.473760 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.473770 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.473784 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] requeue_ops 2014-08-13 17:38:05.473797 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.473801 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) 2014-08-13 17:38:05.473833 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.473746 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473848 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 337 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.473900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.473996 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.337 activated and committed 2014-08-13 17:38:05.474140 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) 2014-08-13 17:38:05.474174 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474186 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.474052 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474208 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] requeue_ops 2014-08-13 17:38:05.474219 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474224 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.474236 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] requeue_ops 2014-08-13 17:38:05.474245 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 348 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.474358 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474432 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.348 activated and committed 2014-08-13 17:38:05.474580 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) 2014-08-13 17:38:05.474617 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.474510 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474630 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 331 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) 2014-08-13 17:38:05.474662 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474785 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.331 activated and committed 2014-08-13 17:38:05.474815 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.474928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] all_activated_and_committed 2014-08-13 17:38:05.474966 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475071 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] share_pg_info 2014-08-13 17:38:05.475223 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing 2014-08-13 17:38:05.475244 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing 2014-08-13 17:38:05.475258 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d3c780 already has epoch 194487 2014-08-13 17:38:05.475117 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475272 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing 2014-08-13 17:38:05.475270 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475284 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194487:22151 2014-08-13 17:38:05.475432 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) 2014-08-13 17:38:05.475477 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475491 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475506 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] requeue_ops 2014-08-13 17:38:05.475517 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475521 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) 2014-08-13 17:38:05.475547 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475559 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475574 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] requeue_ops 2014-08-13 17:38:05.475586 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 363 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.475602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 17:38:05.475612 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475632 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) 2014-08-13 17:38:05.475660 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475685 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] requeue_ops 2014-08-13 17:38:05.475695 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475699 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 363 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.475710 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 17:38:05.475719 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475723 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) 2014-08-13 17:38:05.475749 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475758 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475770 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] requeue_ops 2014-08-13 17:38:05.475781 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475784 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 RequestRecovery 2014-08-13 17:38:05.475796 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] exit Started/ReplicaActive/RepNotRecovering 0.019173 3 0.000844 2014-08-13 17:38:05.475809 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 17:38:05.475823 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) 2014-08-13 17:38:05.475853 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475862 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475875 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] requeue_ops 2014-08-13 17:38:05.475887 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475891 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 17:38:05.475901 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] requeue_ops 2014-08-13 17:38:05.475911 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475915 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) 2014-08-13 17:38:05.475943 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475952 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) 2014-08-13 17:38:05.475979 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.475876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.475988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated 2014-08-13 17:38:05.476029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476182 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476173 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] check_local 2014-08-13 17:38:05.476335 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476354 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] requeue_ops 2014-08-13 17:38:05.476487 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] needs_recovery primary has 59 missing 2014-08-13 17:38:05.476638 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:05.476790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.476942 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477053 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] publish_stats_to_osd 194487:25290 2014-08-13 17:38:05.477093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477248 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477232 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 2014-08-13 17:38:05.477450 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.477453 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.477454 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9b840 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.477460 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.477463 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.477464 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.477465 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.477477 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.477478 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.477479 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825400 pg_info(1 pgs e194487:24.1f7) v4 2014-08-13 17:38:05.477505 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1f7) v4 from osd.59 2014-08-13 17:38:05.477508 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825400 2014-08-13 17:38:05.477535 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.477537 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.477400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477407 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 2014-08-13 17:38:05.477551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477581 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 2014-08-13 17:38:05.477703 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477760 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 17:38:05.477855 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.477969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create target_size 1000 fpp 0.05 2014-08-13 17:38:05.478006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.478157 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.478311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 17:38:05.478361 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_apply_log 194422'17302 .. 194422'17302 2014-08-13 17:38:05.478463 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 17:38:05.478553 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_setup allocated new state, position 7d546c9b//0//24 2014-08-13 17:38:05.478649 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 65 as a source 2014-08-13 17:38:05.478735 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode: num_objects: 61 num_bytes: 251658355 num_objects_dirty: 60 num_objects_omap: 0 num_dirty: 60 num_user_objects: 60 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 17:38:05.478801 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.65 2014-08-13 17:38:05.478913 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode dirty 0.061475 full 0.061475 2014-08-13 17:38:05.479094 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.478955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.65 2014-08-13 17:38:05.479101 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 380 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) 2014-08-13 17:38:05.479114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 380, already have info from that osd, ignoring 2014-08-13 17:38:05.479124 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.479106 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.65 2014-08-13 17:38:05.479127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated 2014-08-13 17:38:05.479262 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.65 2014-08-13 17:38:05.479306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] check_local 2014-08-13 17:38:05.479415 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.65 2014-08-13 17:38:05.479476 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] requeue_ops 2014-08-13 17:38:05.479568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.65 2014-08-13 17:38:05.479644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] needs_recovery primary has 57 missing 2014-08-13 17:38:05.479721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.65 2014-08-13 17:38:05.479814 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:05.479873 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.65 2014-08-13 17:38:05.480026 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.65 2014-08-13 17:38:05.480144 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] publish_stats_to_osd 194487:29707 2014-08-13 17:38:05.480181 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.65 2014-08-13 17:38:05.480333 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.65 2014-08-13 17:38:05.480317 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 2014-08-13 17:38:05.480486 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.65 2014-08-13 17:38:05.480485 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 2014-08-13 17:38:05.480639 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.65 2014-08-13 17:38:05.480652 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 2014-08-13 17:38:05.480792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.65 2014-08-13 17:38:05.480824 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 17:38:05.480944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.65 2014-08-13 17:38:05.480997 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create target_size 1000 fpp 0.05 2014-08-13 17:38:05.481097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.65 2014-08-13 17:38:05.481252 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.65 2014-08-13 17:38:05.481346 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_apply_log 194422'18731 .. 194422'18731 2014-08-13 17:38:05.481405 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.65 2014-08-13 17:38:05.481527 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_setup allocated new state, position ec6b743d//0//24 2014-08-13 17:38:05.481558 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.65 2014-08-13 17:38:05.481709 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.65 2014-08-13 17:38:05.481697 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode: num_objects: 59 num_bytes: 243269718 num_objects_dirty: 58 num_objects_omap: 0 num_dirty: 58 num_user_objects: 58 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 17:38:05.481861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.65 2014-08-13 17:38:05.481866 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode dirty 0.059426 full 0.059426 2014-08-13 17:38:05.482040 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.482019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.65 2014-08-13 17:38:05.482047 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated 2014-08-13 17:38:05.482171 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.65 2014-08-13 17:38:05.482222 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] check_local 2014-08-13 17:38:05.482327 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.65 2014-08-13 17:38:05.482389 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] requeue_ops 2014-08-13 17:38:05.482480 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.65 2014-08-13 17:38:05.482554 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] needs_recovery primary has 56 missing 2014-08-13 17:38:05.482633 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.65 2014-08-13 17:38:05.482719 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] activate not all replicas are up-to-date, queueing recovery 2014-08-13 17:38:05.482785 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.65 2014-08-13 17:38:05.482938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.65 2014-08-13 17:38:05.483043 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] publish_stats_to_osd 194487:30298 2014-08-13 17:38:05.483090 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.65 2014-08-13 17:38:05.483212 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] on activate: bft=286 from 0//0//-1 2014-08-13 17:38:05.483247 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.65 2014-08-13 17:38:05.483378 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] target shard 286 from 0//0//-1 2014-08-13 17:38:05.483400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.65 2014-08-13 17:38:05.483551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.65 2014-08-13 17:38:05.483547 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 17:38:05.483703 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.65 2014-08-13 17:38:05.483715 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_create target_size 1000 fpp 0.05 2014-08-13 17:38:05.483856 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.65 2014-08-13 17:38:05.484009 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.65 2014-08-13 17:38:05.484056 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_apply_log 194422'18660 .. 194422'18660 2014-08-13 17:38:05.484162 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.65 2014-08-13 17:38:05.484237 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_setup allocated new state, position 8ec5770b//0//24 2014-08-13 17:38:05.484317 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.65 2014-08-13 17:38:05.484404 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_choose_mode: num_objects: 56 num_bytes: 230686805 num_objects_dirty: 55 num_objects_omap: 0 num_dirty: 55 num_user_objects: 55 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 17:38:05.484469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.65 2014-08-13 17:38:05.484571 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_choose_mode dirty 0.056352 full 0.056352 2014-08-13 17:38:05.484741 7fe99ebe8700 10 log is not dirty 2014-08-13 17:38:05.484747 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated 2014-08-13 17:38:05.484760 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] check_local 2014-08-13 17:38:05.484622 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.65 2014-08-13 17:38:05.484769 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] requeue_ops 2014-08-13 17:38:05.484779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_recovery is recovered 2014-08-13 17:38:05.484789 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.26 has last_backfill 0//0//-1 2014-08-13 17:38:05.484798 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.30 has last_backfill 0//0//-1 2014-08-13 17:38:05.484808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.340 has last_backfill 0//0//-1 2014-08-13 17:38:05.484817 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] activate queueing backfill 2014-08-13 17:38:05.484831 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] publish_stats_to_osd 194487:23845 2014-08-13 17:38:05.484841 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 2014-08-13 17:38:05.484851 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 2014-08-13 17:38:05.484860 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 2014-08-13 17:38:05.484869 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 2014-08-13 17:38:05.484881 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 17:38:05.484893 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create target_size 1000 fpp 0.05 2014-08-13 17:38:05.484773 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.65 2014-08-13 17:38:05.484925 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.65 2014-08-13 17:38:05.485075 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_apply_log 194215'14514 .. 194215'14514 2014-08-13 17:38:05.485087 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_setup allocated new state, position 3be28434//0//24 2014-08-13 17:38:05.485102 7fe99ebe8700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 17:38:05.485105 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_choose_mode: num_objects: -1 num_bytes: -9223372036854775808 num_objects_dirty: -2 num_objects_omap: 0 num_dirty: 0 num_user_objects: 18446744073709551615 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 17:38:05.485077 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.65 2014-08-13 17:38:05.485236 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.65 2014-08-13 17:38:05.485389 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.65 2014-08-13 17:38:05.485542 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.65 2014-08-13 17:38:05.485694 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.65 2014-08-13 17:38:05.485847 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.65 2014-08-13 17:38:05.485999 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.65 2014-08-13 17:38:05.486151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.65 2014-08-13 17:38:05.486306 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.65 2014-08-13 17:38:05.486459 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.65 2014-08-13 17:38:05.486612 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 17:38:05.486799 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound 2014-08-13 17:38:05.486950 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 17:38:05.487104 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound: built 50,55,61,65,356 2014-08-13 17:38:05.487260 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Activate Finished 2014-08-13 17:38:05.487414 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating 2014-08-13 17:38:05.487569 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.487721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.487924 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488048 7fe99ebe8700 -1 *** Caught signal (Floating point exception) ** in thread 7fe99ebe8700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fe9b9e35710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fe9b9e2d9d1] 16: (clone()+0x6d) [0x7fe9b8dbdb6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -10000> 2014-08-13 17:38:04.398889 7fe9a63f4700 5 -- op tracker -- , seq: 473, time: 2014-08-13 17:38:04.398889, event: done, request: pg_info(1 pgs e194486:22.4fe) v4 -9999> 2014-08-13 17:38:04.398880 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.000578 1 0.000556 -9998> 2014-08-13 17:38:04.398880 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: got info from osd.371 22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) -9997> 2014-08-13 17:38:04.398905 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9996> 2014-08-13 17:38:04.398908 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9995> 2014-08-13 17:38:04.398902 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started -9994> 2014-08-13 17:38:04.398905 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.915333 7 0.000259 -9993> 2014-08-13 17:38:04.398919 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start -9992> 2014-08-13 17:38:04.398925 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9991> 2014-08-13 17:38:04.398935 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray -9990> 2014-08-13 17:38:04.398946 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9989> 2014-08-13 17:38:04.398956 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000037 0 0.000000 -9988> 2014-08-13 17:38:04.398966 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194478 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9987> 2014-08-13 17:38:04.398977 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray -9986> 2014-08-13 17:38:04.398983 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'493 -> 194215'493 -9985> 2014-08-13 17:38:04.398999 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9984> 2014-08-13 17:38:04.399004 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] state: Activate Finished -9983> 2014-08-13 17:38:04.399016 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9982> 2014-08-13 17:38:04.399061 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 12 ==== pg_info(1 pgs e194486:22.4fe) v4 ==== 714+0+0 (3560702952 0 0) 0x5f00fc0 con 0x4d3c780 -9981> 2014-08-13 17:38:04.399078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9980> 2014-08-13 17:38:04.399081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9979> 2014-08-13 17:38:04.399083 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00fc0 pg_info(1 pgs e194486:22.4fe) v4 -9978> 2014-08-13 17:38:04.399089 7fe99f5e9700 10 log is not dirty -9977> 2014-08-13 17:38:04.399090 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.398942, event: header_read, request: pg_info(1 pgs e194486:22.4fe) v4 -9976> 2014-08-13 17:38:04.399098 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.398943, event: throttled, request: pg_info(1 pgs e194486:22.4fe) v4 -9975> 2014-08-13 17:38:04.399106 7fe99ebe8700 10 log is not dirty -9974> 2014-08-13 17:38:04.399104 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.399036, event: all_read, request: pg_info(1 pgs e194486:22.4fe) v4 -9973> 2014-08-13 17:38:04.399110 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.399077, event: dispatched, request: pg_info(1 pgs e194486:22.4fe) v4 -9972> 2014-08-13 17:38:04.399117 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.399117, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.4fe) v4 -9971> 2014-08-13 17:38:04.399124 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe) v4 from osd.337 -9970> 2014-08-13 17:38:04.399128 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00fc0 -9969> 2014-08-13 17:38:04.399115 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 368 notify: (query_epoch:194486, epoch_sent:194486, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) -9968> 2014-08-13 17:38:04.399134 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.399133, event: started, request: pg_info(1 pgs e194486:22.4fe) v4 -9967> 2014-08-13 17:38:04.399139 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring -9966> 2014-08-13 17:38:04.399156 7fe9a63f4700 5 -- op tracker -- , seq: 474, time: 2014-08-13 17:38:04.399156, event: done, request: pg_info(1 pgs e194486:22.4fe) v4 -9965> 2014-08-13 17:38:04.399161 7fe99f5e9700 10 log is not dirty -9964> 2014-08-13 17:38:04.399169 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9963> 2014-08-13 17:38:04.399173 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9962> 2014-08-13 17:38:04.399147 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 -9961> 2014-08-13 17:38:04.399190 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9960> 2014-08-13 17:38:04.399169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 351 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) -9959> 2014-08-13 17:38:04.399197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: verifying no want_acting [] targets didn't go down -9958> 2014-08-13 17:38:04.399209 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.397 192.168.1.105:6808/41085 8 ==== pg_info(1 pgs e194486:22.3b9) v4 ==== 714+0+0 (3822333238 0 0) 0x513d5c0 con 0x4e48840 -9957> 2014-08-13 17:38:04.399204 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -9956> 2014-08-13 17:38:04.399225 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9955> 2014-08-13 17:38:04.399228 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9954> 2014-08-13 17:38:04.399216 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Started advmap -9953> 2014-08-13 17:38:04.399230 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513d5c0 pg_info(1 pgs e194486:22.3b9) v4 -9952> 2014-08-13 17:38:04.399236 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399100, event: header_read, request: pg_info(1 pgs e194486:22.3b9) v4 -9951> 2014-08-13 17:38:04.399244 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399102, event: throttled, request: pg_info(1 pgs e194486:22.3b9) v4 -9950> 2014-08-13 17:38:04.399234 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] -9949> 2014-08-13 17:38:04.399250 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399164, event: all_read, request: pg_info(1 pgs e194486:22.3b9) v4 -9948> 2014-08-13 17:38:04.399236 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 unchanged -9947> 2014-08-13 17:38:04.399257 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399225, event: dispatched, request: pg_info(1 pgs e194486:22.3b9) v4 -9946> 2014-08-13 17:38:04.399261 7fe99f5e9700 10 log is not dirty -9945> 2014-08-13 17:38:04.399251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, transitioning to Reset -9944> 2014-08-13 17:38:04.399264 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399264, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3b9) v4 -9943> 2014-08-13 17:38:04.399270 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3b9) v4 from osd.397 -9942> 2014-08-13 17:38:04.399276 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513d5c0 -9941> 2014-08-13 17:38:04.399281 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399280, event: started, request: pg_info(1 pgs e194486:22.3b9) v4 -9940> 2014-08-13 17:38:04.399270 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -9939> 2014-08-13 17:38:04.399270 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary/Peering/WaitActingChange 0.857996 1 0.000262 -9938> 2014-08-13 17:38:04.399303 7fe9a63f4700 5 -- op tracker -- , seq: 475, time: 2014-08-13 17:38:04.399303, event: done, request: pg_info(1 pgs e194486:22.3b9) v4 -9937> 2014-08-13 17:38:04.399294 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9936> 2014-08-13 17:38:04.399294 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.858030 0 0.000000 -9935> 2014-08-13 17:38:04.399314 7fe99f5e9700 10 log is not dirty -9934> 2014-08-13 17:38:04.399315 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9933> 2014-08-13 17:38:04.399318 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9932> 2014-08-13 17:38:04.399317 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started 0.875228 0 0.000000 -9931> 2014-08-13 17:38:04.399322 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -9930> 2014-08-13 17:38:04.399341 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops -9929> 2014-08-13 17:38:04.399343 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset -9928> 2014-08-13 17:38:04.399358 7fe99f5e9700 10 log is not dirty -9927> 2014-08-13 17:38:04.399359 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194485 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194486 -9926> 2014-08-13 17:38:04.399365 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -9925> 2014-08-13 17:38:04.399377 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Reset advmap -9924> 2014-08-13 17:38:04.399386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -9923> 2014-08-13 17:38:04.399403 7fe99f5e9700 10 log is not dirty -9922> 2014-08-13 17:38:04.399394 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] _calc_past_interval_range: already have past intervals back to 194484 -9921> 2014-08-13 17:38:04.399409 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -9920> 2014-08-13 17:38:04.399416 7fe99f5e9700 10 osd.4 194486 send_pg_temp {24.1bb=[4,332,337]} -9919> 2014-08-13 17:38:04.399409 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.397 192.168.1.105:6808/41085 9 ==== pg_info(1 pgs e194486:22.3b9) v4 ==== 714+0+0 (1982015908 0 0) 0x5f00380 con 0x4e48840 -9918> 2014-08-13 17:38:04.399410 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] -9917> 2014-08-13 17:38:04.399426 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9916> 2014-08-13 17:38:04.399429 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9915> 2014-08-13 17:38:04.399422 7fe99f5e9700 10 monclient: _send_mon_message to mon.storage0112-ib at 192.168.100.112:6789/0 -9914> 2014-08-13 17:38:04.399431 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00380 pg_info(1 pgs e194486:22.3b9) v4 -9913> 2014-08-13 17:38:04.399436 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399301, event: header_read, request: pg_info(1 pgs e194486:22.3b9) v4 -9912> 2014-08-13 17:38:04.399428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, calling start_peering_interval again -9911> 2014-08-13 17:38:04.399433 7fe99f5e9700 1 -- 192.168.100.115:6805/34805 --> 192.168.100.112:6789/0 -- osd_pgtemp(e194486 {24.1bb=[4,332,337]} v194486) v1 -- ?+0 0x59723a0 con 0x49dfd20 -9910> 2014-08-13 17:38:04.399444 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399303, event: throttled, request: pg_info(1 pgs e194486:22.3b9) v4 -9909> 2014-08-13 17:38:04.399450 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399387, event: all_read, request: pg_info(1 pgs e194486:22.3b9) v4 -9908> 2014-08-13 17:38:04.399445 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331] r=0 lpr=194486 pi=193644-194484/31 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194486 -9907> 2014-08-13 17:38:04.399455 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399425, event: dispatched, request: pg_info(1 pgs e194486:22.3b9) v4 -9906> 2014-08-13 17:38:04.399463 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399463, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3b9) v4 -9905> 2014-08-13 17:38:04.399479 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3b9) v4 from osd.397 -9904> 2014-08-13 17:38:04.399484 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00380 -9903> 2014-08-13 17:38:04.399482 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)): not rw, up_thru 194477 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) : primary up 194485-194477 does not include interval -9902> 2014-08-13 17:38:04.399496 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] _activate_committed 194486 telling primary -9901> 2014-08-13 17:38:04.399501 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194485/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] noting past interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) -9900> 2014-08-13 17:38:04.399493 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 349 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) -9899> 2014-08-13 17:38:04.399525 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing -9898> 2014-08-13 17:38:04.399529 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.371 192.168.1.102:6832/16753 -- pg_info(1 pgs e194486:22.653) v4 -- ?+0 0x4260380 -9897> 2014-08-13 17:38:04.399525 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.349 activated and committed -9896> 2014-08-13 17:38:04.399522 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] up [4,348,331] -> [4,348,331], acting [4,348,331] -> [4,332,337], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 -9895> 2014-08-13 17:38:04.399545 7fe99f5e9700 10 log is not dirty -9894> 2014-08-13 17:38:04.399544 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_primary_state -9893> 2014-08-13 17:38:04.399554 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] flushed -9892> 2014-08-13 17:38:04.399556 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 350 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -9891> 2014-08-13 17:38:04.399574 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] agent_stop -9890> 2014-08-13 17:38:04.399589 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399589, event: started, request: pg_info(1 pgs e194486:22.3b9) v4 -9889> 2014-08-13 17:38:04.399580 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -9888> 2014-08-13 17:38:04.399605 7fe99f5e9700 10 log is not dirty -9887> 2014-08-13 17:38:04.399593 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change -9886> 2014-08-13 17:38:04.399613 7fe9a63f4700 5 -- op tracker -- , seq: 476, time: 2014-08-13 17:38:04.399613, event: done, request: pg_info(1 pgs e194486:22.3b9) v4 -9885> 2014-08-13 17:38:04.399626 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9884> 2014-08-13 17:38:04.399629 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9883> 2014-08-13 17:38:04.399612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) -9882> 2014-08-13 17:38:04.399615 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194486:22148 -9881> 2014-08-13 17:38:04.399634 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -9880> 2014-08-13 17:38:04.399633 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: got info from osd.58 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) -9879> 2014-08-13 17:38:04.399655 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_copy_ops -9878> 2014-08-13 17:38:04.399658 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.035759 7 0.001420 -9877> 2014-08-13 17:38:04.399671 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_flush_ops -9876> 2014-08-13 17:38:04.399678 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9875> 2014-08-13 17:38:04.399687 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -9874> 2014-08-13 17:38:04.399696 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 11 ==== pg_info(1 pgs e194486:22.581) v4 ==== 714+0+0 (3835855521 0 0) 0x513e740 con 0x4ec8000 -9873> 2014-08-13 17:38:04.399714 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9872> 2014-08-13 17:38:04.399701 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9871> 2014-08-13 17:38:04.399717 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9870> 2014-08-13 17:38:04.399703 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -9869> 2014-08-13 17:38:04.399719 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513e740 pg_info(1 pgs e194486:22.581) v4 -9868> 2014-08-13 17:38:04.399726 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.399572, event: header_read, request: pg_info(1 pgs e194486:22.581) v4 -9867> 2014-08-13 17:38:04.406298 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.399574, event: throttled, request: pg_info(1 pgs e194486:22.581) v4 -9866> 2014-08-13 17:38:04.406319 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.399657, event: all_read, request: pg_info(1 pgs e194486:22.581) v4 -9865> 2014-08-13 17:38:04.406326 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.399713, event: dispatched, request: pg_info(1 pgs e194486:22.581) v4 -9864> 2014-08-13 17:38:04.399720 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9863> 2014-08-13 17:38:04.406334 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.406334, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.581) v4 -9862> 2014-08-13 17:38:04.406341 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.581) v4 from osd.320 -9861> 2014-08-13 17:38:04.406348 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513e740 -9860> 2014-08-13 17:38:04.406354 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.406354, event: started, request: pg_info(1 pgs e194486:22.581) v4 -9859> 2014-08-13 17:38:04.406352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 -9858> 2014-08-13 17:38:04.406380 7fe9a63f4700 5 -- op tracker -- , seq: 477, time: 2014-08-13 17:38:04.406380, event: done, request: pg_info(1 pgs e194486:22.581) v4 -9857> 2014-08-13 17:38:04.406396 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9856> 2014-08-13 17:38:04.406399 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9855> 2014-08-13 17:38:04.399720 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -9854> 2014-08-13 17:38:04.406394 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] state: Activate Finished -9853> 2014-08-13 17:38:04.399725 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] flushed -9852> 2014-08-13 17:38:04.406410 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 12 ==== MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 ==== 26+0+0 (1521606587 0 0) 0x5107540 con 0x4ec8000 -9851> 2014-08-13 17:38:04.406431 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9850> 2014-08-13 17:38:04.403528 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.403664) v2 ==== 47+0+0 (3448402760 0 0) 0x5140380 con 0x40569e0 -9849> 2014-08-13 17:38:04.406433 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9848> 2014-08-13 17:38:04.406436 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107540 MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9847> 2014-08-13 17:38:04.406442 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.400529, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9846> 2014-08-13 17:38:04.406414 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change_cleanup -9845> 2014-08-13 17:38:04.406449 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.400531, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9844> 2014-08-13 17:38:04.406458 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.400608, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9843> 2014-08-13 17:38:04.403586 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.403664) v2 ==== 47+0+0 (3448402760 0 0) 0x5827000 con 0x40d8b00 -9842> 2014-08-13 17:38:04.406464 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.406430, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9841> 2014-08-13 17:38:04.406456 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change -9840> 2014-08-13 17:38:04.406449 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.403664) v2 -- ?+0 0x5143f00 con 0x40569e0 -9839> 2014-08-13 17:38:04.406471 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.406471, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9838> 2014-08-13 17:38:04.406485 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.289 now has 194486 -9837> 2014-08-13 17:38:04.406489 7fe99f5e9700 10 log is not dirty -9836> 2014-08-13 17:38:04.406486 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107540 -9835> 2014-08-13 17:38:04.406478 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit NotTrimming -9834> 2014-08-13 17:38:04.406503 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -9833> 2014-08-13 17:38:04.406504 7fe9a63f4700 5 -- op tracker -- , seq: 478, time: 2014-08-13 17:38:04.406504, event: done, request: MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194486 v2 -9832> 2014-08-13 17:38:04.406497 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter NotTrimming -9831> 2014-08-13 17:38:04.406516 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9830> 2014-08-13 17:38:04.406519 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9829> 2014-08-13 17:38:04.406515 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] [4,348,331] -> [4,332,337], replicas changed -9828> 2014-08-13 17:38:04.406525 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.371 192.168.1.102:6832/16753 8 ==== pg_info(1 pgs e194486:22.653) v4 ==== 714+0+0 (4189662095 0 0) 0x4260380 con 0x41e5540 -9827> 2014-08-13 17:38:04.406539 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9826> 2014-08-13 17:38:04.406541 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9825> 2014-08-13 17:38:04.406543 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.653) v4 -9824> 2014-08-13 17:38:04.406548 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.404183, event: header_read, request: pg_info(1 pgs e194486:22.653) v4 -9823> 2014-08-13 17:38:04.406557 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.404186, event: throttled, request: pg_info(1 pgs e194486:22.653) v4 -9822> 2014-08-13 17:38:04.406563 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.404265, event: all_read, request: pg_info(1 pgs e194486:22.653) v4 -9821> 2014-08-13 17:38:04.406569 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.406538, event: dispatched, request: pg_info(1 pgs e194486:22.653) v4 -9820> 2014-08-13 17:38:04.406553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 304 query_epoch 194486 query: query(info 0'0) -9819> 2014-08-13 17:38:04.406575 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.406575, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.653) v4 -9818> 2014-08-13 17:38:04.406581 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.653) v4 from osd.371 -9817> 2014-08-13 17:38:04.406585 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 -9816> 2014-08-13 17:38:04.406590 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.406590, event: started, request: pg_info(1 pgs e194486:22.653) v4 -9815> 2014-08-13 17:38:04.406584 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info -9814> 2014-08-13 17:38:04.406610 7fe9a63f4700 5 -- op tracker -- , seq: 479, time: 2014-08-13 17:38:04.406610, event: done, request: pg_info(1 pgs e194486:22.653) v4 -9813> 2014-08-13 17:38:04.406621 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9812> 2014-08-13 17:38:04.406624 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9811> 2014-08-13 17:38:04.406612 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.403664) v2 -- ?+0 0x5141500 con 0x40d8b00 -9810> 2014-08-13 17:38:04.406629 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.58 192.168.1.111:6834/14316 9 ==== pg_info(1 pgs e194486:22.362) v4 ==== 714+0+0 (1473357834 0 0) 0x5107700 con 0x49e1080 -9809> 2014-08-13 17:38:04.406641 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9808> 2014-08-13 17:38:04.406643 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9807> 2014-08-13 17:38:04.406645 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107700 pg_info(1 pgs e194486:22.362) v4 -9806> 2014-08-13 17:38:04.406534 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_recovery -9805> 2014-08-13 17:38:04.406649 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.403946, event: header_read, request: pg_info(1 pgs e194486:22.362) v4 -9804> 2014-08-13 17:38:04.406655 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.403948, event: throttled, request: pg_info(1 pgs e194486:22.362) v4 -9803> 2014-08-13 17:38:04.406659 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -9802> 2014-08-13 17:38:04.406661 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.404060, event: all_read, request: pg_info(1 pgs e194486:22.362) v4 -9801> 2014-08-13 17:38:04.406668 7fe99f5e9700 10 log is not dirty -9800> 2014-08-13 17:38:04.406667 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.406640, event: dispatched, request: pg_info(1 pgs e194486:22.362) v4 -9799> 2014-08-13 17:38:04.406682 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.406681, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.362) v4 -9798> 2014-08-13 17:38:04.406688 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.362) v4 from osd.58 -9797> 2014-08-13 17:38:04.406692 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107700 -9796> 2014-08-13 17:38:04.406679 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -9795> 2014-08-13 17:38:04.406696 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.406696, event: started, request: pg_info(1 pgs e194486:22.362) v4 -9794> 2014-08-13 17:38:04.406654 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_recovery_state -9793> 2014-08-13 17:38:04.406718 7fe9a63f4700 5 -- op tracker -- , seq: 480, time: 2014-08-13 17:38:04.406718, event: done, request: pg_info(1 pgs e194486:22.362) v4 -9792> 2014-08-13 17:38:04.406708 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] check_recovery_sources no source osds () went down -9791> 2014-08-13 17:38:04.406704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -9790> 2014-08-13 17:38:04.406730 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9789> 2014-08-13 17:38:04.406732 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9788> 2014-08-13 17:38:04.406737 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.62 192.168.1.112:6824/5505 3 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4d9a1c0 con 0x4a4dd80 -9787> 2014-08-13 17:38:04.406734 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map -9786> 2014-08-13 17:38:04.406751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9785> 2014-08-13 17:38:04.406754 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9784> 2014-08-13 17:38:04.406739 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged -9783> 2014-08-13 17:38:04.406756 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9a1c0 osd_map(194486..194486 src has 185139..194486) v3 -9782> 2014-08-13 17:38:04.406766 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -9781> 2014-08-13 17:38:04.406771 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -9780> 2014-08-13 17:38:04.406774 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9779> 2014-08-13 17:38:04.406776 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9778> 2014-08-13 17:38:04.406760 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last maybe_went_rw interval was interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) -9777> 2014-08-13 17:38:04.406787 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.61 192.168.1.111:6840/16566 12 ==== pg_info(1 pgs e194486:22.1d2) v4 ==== 714+0+0 (3756212915 0 0) 0x5823d40 con 0x4cf86e0 -9776> 2014-08-13 17:38:04.406800 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9775> 2014-08-13 17:38:04.406803 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9774> 2014-08-13 17:38:04.406787 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.080873 4 0.000767 -9773> 2014-08-13 17:38:04.406804 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823d40 pg_info(1 pgs e194486:22.1d2) v4 -9772> 2014-08-13 17:38:04.406809 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.401606, event: header_read, request: pg_info(1 pgs e194486:22.1d2) v4 -9771> 2014-08-13 17:38:04.406816 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.401608, event: throttled, request: pg_info(1 pgs e194486:22.1d2) v4 -9770> 2014-08-13 17:38:04.406821 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.401689, event: all_read, request: pg_info(1 pgs e194486:22.1d2) v4 -9769> 2014-08-13 17:38:04.406811 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog -9768> 2014-08-13 17:38:04.406827 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.406800, event: dispatched, request: pg_info(1 pgs e194486:22.1d2) v4 -9767> 2014-08-13 17:38:04.406833 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.406833, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1d2) v4 -9766> 2014-08-13 17:38:04.406839 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1d2) v4 from osd.61 -9765> 2014-08-13 17:38:04.406843 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823d40 -9764> 2014-08-13 17:38:04.406847 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.406847, event: started, request: pg_info(1 pgs e194486:22.1d2) v4 -9763> 2014-08-13 17:38:04.406834 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.4 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) -9762> 2014-08-13 17:38:04.406855 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -9761> 2014-08-13 17:38:04.406872 7fe9a63f4700 5 -- op tracker -- , seq: 481, time: 2014-08-13 17:38:04.406872, event: done, request: pg_info(1 pgs e194486:22.1d2) v4 -9760> 2014-08-13 17:38:04.406756 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] update_heartbeat_peers 4,331,332,337,348,379 -> 4,331,332,337,348 -9759> 2014-08-13 17:38:04.406883 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9758> 2014-08-13 17:38:04.406884 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update -9757> 2014-08-13 17:38:04.406885 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9756> 2014-08-13 17:38:04.406876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -9755> 2014-08-13 17:38:04.406891 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.21 192.168.1.107:6821/54786 3 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4db5e80 con 0x4d335a0 -9754> 2014-08-13 17:38:04.406888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] take_waiters -9753> 2014-08-13 17:38:04.406905 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9752> 2014-08-13 17:38:04.406907 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9751> 2014-08-13 17:38:04.406909 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db5e80 osd_map(194486..194486 src has 185139..194486) v3 -9750> 2014-08-13 17:38:04.406897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -9749> 2014-08-13 17:38:04.406916 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -9748> 2014-08-13 17:38:04.406921 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -9747> 2014-08-13 17:38:04.406923 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9746> 2014-08-13 17:38:04.406926 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9745> 2014-08-13 17:38:04.406909 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Reset 0.007567 1 0.007536 -9744> 2014-08-13 17:38:04.406931 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.19 192.168.1.115:6818/61186 5 ==== pg_info(1 pgs e194486:24.3bf) v4 ==== 744+0+0 (2555437217 0 0) 0x5103d40 con 0x4d7b020 -9743> 2014-08-13 17:38:04.406943 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9742> 2014-08-13 17:38:04.406945 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9741> 2014-08-13 17:38:04.406947 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103d40 pg_info(1 pgs e194486:24.3bf) v4 -9740> 2014-08-13 17:38:04.406935 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started -9739> 2014-08-13 17:38:04.406952 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.405992, event: header_read, request: pg_info(1 pgs e194486:24.3bf) v4 -9738> 2014-08-13 17:38:04.406954 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Start -9737> 2014-08-13 17:38:04.406959 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.405994, event: throttled, request: pg_info(1 pgs e194486:24.3bf) v4 -9736> 2014-08-13 17:38:04.406997 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.406068, event: all_read, request: pg_info(1 pgs e194486:24.3bf) v4 -9735> 2014-08-13 17:38:04.407006 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.406942, event: dispatched, request: pg_info(1 pgs e194486:24.3bf) v4 -9734> 2014-08-13 17:38:04.407013 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.407013, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:24.3bf) v4 -9733> 2014-08-13 17:38:04.407019 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.3bf) v4 from osd.19 -9732> 2014-08-13 17:38:04.407023 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103d40 -9731> 2014-08-13 17:38:04.407028 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.407027, event: started, request: pg_info(1 pgs e194486:24.3bf) v4 -9730> 2014-08-13 17:38:04.407052 7fe9a63f4700 5 -- op tracker -- , seq: 482, time: 2014-08-13 17:38:04.407051, event: done, request: pg_info(1 pgs e194486:24.3bf) v4 -9729> 2014-08-13 17:38:04.407040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting newest update on osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) up[0] needs backfill, osd.4 selected as primary instead calc_acting primary is osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) shard 26 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) shard 30 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) shard 340 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -9728> 2014-08-13 17:38:04.407065 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9727> 2014-08-13 17:38:04.407068 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9726> 2014-08-13 17:38:04.407065 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] actingbackfill is 4,26,30,340 -9725> 2014-08-13 17:38:04.407073 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.341 192.168.1.106:6818/9142 3 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x5481b00 con 0x4df32e0 -9724> 2014-08-13 17:38:04.407088 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9723> 2014-08-13 17:38:04.407090 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9722> 2014-08-13 17:38:04.407092 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5481b00 osd_map(194486..194486 src has 185139..194486) v3 -9721> 2014-08-13 17:38:04.407083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] choose_acting want [4] (== acting) backfill_targets 26,30,340 -9720> 2014-08-13 17:38:04.407099 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -9719> 2014-08-13 17:38:04.407104 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -9718> 2014-08-13 17:38:04.407106 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9717> 2014-08-13 17:38:04.407107 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9716> 2014-08-13 17:38:04.406971 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: transitioning to Primary -9715> 2014-08-13 17:38:04.407108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: leaving GetLog -9714> 2014-08-13 17:38:04.407115 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.371 192.168.1.102:6832/16753 9 ==== MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 ==== 26+0+0 (3722662347 0 0) 0x5f00700 con 0x41e5540 -9713> 2014-08-13 17:38:04.407128 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9712> 2014-08-13 17:38:04.407130 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9711> 2014-08-13 17:38:04.407132 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00700 MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9710> 2014-08-13 17:38:04.407122 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Start 0.000168 0 0.000000 -9709> 2014-08-13 17:38:04.407138 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.404955, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9708> 2014-08-13 17:38:04.407132 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000321 0 0.000000 -9707> 2014-08-13 17:38:04.407148 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.404958, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9706> 2014-08-13 17:38:04.407151 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary -9705> 2014-08-13 17:38:04.407156 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing -9704> 2014-08-13 17:38:04.407170 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Peering -9703> 2014-08-13 17:38:04.407175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.26 will fully backfill; can infer empty missing set -9702> 2014-08-13 17:38:04.407188 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetInfo -9701> 2014-08-13 17:38:04.407219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.30 will fully backfill; can infer empty missing set -9700> 2014-08-13 17:38:04.407219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] _calc_past_interval_range: already have past intervals back to 194484 -9699> 2014-08-13 17:38:04.407237 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.340 will fully backfill; can infer empty missing set -9698> 2014-08-13 17:38:04.407241 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) -9697> 2014-08-13 17:38:04.407255 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: still need up_thru update before going active -9696> 2014-08-13 17:38:04.407263 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) -9695> 2014-08-13 17:38:04.407154 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.405003, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9694> 2014-08-13 17:38:04.407276 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000119 0 0.000000 -9693> 2014-08-13 17:38:04.407293 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.407127, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9692> 2014-08-13 17:38:04.407303 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.407303, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9691> 2014-08-13 17:38:04.407292 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194482-194482 up [332,337,379](332) acting [332,337](332)) -9690> 2014-08-13 17:38:04.407298 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru -9689> 2014-08-13 17:38:04.407314 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00700 -9688> 2014-08-13 17:38:04.407319 7fe99f5e9700 10 log is not dirty -9687> 2014-08-13 17:38:04.407314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} -9686> 2014-08-13 17:38:04.407331 7fe9a63f4700 5 -- op tracker -- , seq: 483, time: 2014-08-13 17:38:04.407330, event: done, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194486 v2 -9685> 2014-08-13 17:38:04.407343 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9684> 2014-08-13 17:38:04.407346 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9683> 2014-08-13 17:38:04.407335 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] up_thru 194485 < same_since 194486, must notify monitor -9682> 2014-08-13 17:38:04.407352 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.62 192.168.1.112:6824/5505 4 ==== pg_info(1 pgs e194486:22.619) v4 ==== 714+0+0 (565811158 0 0) 0x5140540 con 0x4a4dd80 -9681> 2014-08-13 17:38:04.407367 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9680> 2014-08-13 17:38:04.407370 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9679> 2014-08-13 17:38:04.407352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 31 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) -9678> 2014-08-13 17:38:04.407373 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5140540 pg_info(1 pgs e194486:22.619) v4 -9677> 2014-08-13 17:38:04.407359 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194486:22149 -9676> 2014-08-13 17:38:04.407378 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.401831, event: header_read, request: pg_info(1 pgs e194486:22.619) v4 -9675> 2014-08-13 17:38:04.407385 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.401832, event: throttled, request: pg_info(1 pgs e194486:22.619) v4 -9674> 2014-08-13 17:38:04.407391 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.401909, event: all_read, request: pg_info(1 pgs e194486:22.619) v4 -9673> 2014-08-13 17:38:04.407379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.31 activated and committed -9672> 2014-08-13 17:38:04.407379 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.331 -9671> 2014-08-13 17:38:04.407397 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.407367, event: dispatched, request: pg_info(1 pgs e194486:22.619) v4 -9670> 2014-08-13 17:38:04.407404 7fe99f5e9700 10 log is not dirty -9669> 2014-08-13 17:38:04.407404 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.407403, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.619) v4 -9668> 2014-08-13 17:38:04.407410 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.619) v4 from osd.62 -9667> 2014-08-13 17:38:04.407415 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5140540 -9666> 2014-08-13 17:38:04.407406 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.332 -9665> 2014-08-13 17:38:04.407419 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.407419, event: started, request: pg_info(1 pgs e194486:22.619) v4 -9664> 2014-08-13 17:38:04.407412 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -9663> 2014-08-13 17:38:04.407427 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.337 -9662> 2014-08-13 17:38:04.407444 7fe9a63f4700 5 -- op tracker -- , seq: 484, time: 2014-08-13 17:38:04.407443, event: done, request: pg_info(1 pgs e194486:22.619) v4 -9661> 2014-08-13 17:38:04.407435 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9660> 2014-08-13 17:38:04.407455 7fe99f5e9700 10 log is not dirty -9659> 2014-08-13 17:38:04.407456 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9658> 2014-08-13 17:38:04.407460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9657> 2014-08-13 17:38:04.407449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.348 -9656> 2014-08-13 17:38:04.407465 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.21 192.168.1.107:6821/54786 4 ==== pg_info(1 pgs e194486:22.707) v4 ==== 714+0+0 (3432097173 0 0) 0x5823640 con 0x4d335a0 -9655> 2014-08-13 17:38:04.407478 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9654> 2014-08-13 17:38:04.407462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(info 0'0) -9653> 2014-08-13 17:38:04.407480 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9652> 2014-08-13 17:38:04.407482 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823640 pg_info(1 pgs e194486:22.707) v4 -9651> 2014-08-13 17:38:04.407469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.379 -9650> 2014-08-13 17:38:04.407487 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.405617, event: header_read, request: pg_info(1 pgs e194486:22.707) v4 -9649> 2014-08-13 17:38:04.407494 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.405618, event: throttled, request: pg_info(1 pgs e194486:22.707) v4 -9648> 2014-08-13 17:38:04.407484 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info -9647> 2014-08-13 17:38:04.407500 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.405719, event: all_read, request: pg_info(1 pgs e194486:22.707) v4 -9646> 2014-08-13 17:38:04.407490 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9645> 2014-08-13 17:38:04.407506 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.407477, event: dispatched, request: pg_info(1 pgs e194486:22.707) v4 -9644> 2014-08-13 17:38:04.407513 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.407513, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.707) v4 -9643> 2014-08-13 17:38:04.407518 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.707) v4 from osd.21 -9642> 2014-08-13 17:38:04.407522 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823640 -9641> 2014-08-13 17:38:04.407509 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9640> 2014-08-13 17:38:04.407527 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.407527, event: started, request: pg_info(1 pgs e194486:22.707) v4 -9639> 2014-08-13 17:38:04.407546 7fe9a63f4700 5 -- op tracker -- , seq: 485, time: 2014-08-13 17:38:04.407545, event: done, request: pg_info(1 pgs e194486:22.707) v4 -9638> 2014-08-13 17:38:04.407555 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9637> 2014-08-13 17:38:04.407558 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9636> 2014-08-13 17:38:04.407562 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.341 192.168.1.106:6818/9142 4 ==== pg_info(1 pgs e194486:22.48e) v4 ==== 714+0+0 (1321494276 0 0) 0x5826200 con 0x4df32e0 -9635> 2014-08-13 17:38:04.407574 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9634> 2014-08-13 17:38:04.407574 7fe99f5e9700 10 log is not dirty -9633> 2014-08-13 17:38:04.407577 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9632> 2014-08-13 17:38:04.407579 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826200 pg_info(1 pgs e194486:22.48e) v4 -9631> 2014-08-13 17:38:04.407583 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.406312, event: header_read, request: pg_info(1 pgs e194486:22.48e) v4 -9630> 2014-08-13 17:38:04.407589 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.406313, event: throttled, request: pg_info(1 pgs e194486:22.48e) v4 -9629> 2014-08-13 17:38:04.407597 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.406368, event: all_read, request: pg_info(1 pgs e194486:22.48e) v4 -9628> 2014-08-13 17:38:04.407584 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(info 0'0) -9627> 2014-08-13 17:38:04.407603 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.407574, event: dispatched, request: pg_info(1 pgs e194486:22.48e) v4 -9626> 2014-08-13 17:38:04.407610 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.407610, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.48e) v4 -9625> 2014-08-13 17:38:04.407615 7fe99ebe8700 10 log is not dirty -9624> 2014-08-13 17:38:04.407616 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.48e) v4 from osd.341 -9623> 2014-08-13 17:38:04.407620 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826200 -9622> 2014-08-13 17:38:04.407606 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info -9621> 2014-08-13 17:38:04.407624 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.407624, event: started, request: pg_info(1 pgs e194486:22.48e) v4 -9620> 2014-08-13 17:38:04.407659 7fe9a63f4700 5 -- op tracker -- , seq: 486, time: 2014-08-13 17:38:04.407659, event: done, request: pg_info(1 pgs e194486:22.48e) v4 -9619> 2014-08-13 17:38:04.407678 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9618> 2014-08-13 17:38:04.407681 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9617> 2014-08-13 17:38:04.407695 7fe99f5e9700 10 log is not dirty -9616> 2014-08-13 17:38:04.407687 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.351 192.168.1.102:6830/16444 9 ==== pg_info(1 pgs e194486:22.220) v4 ==== 714+0+0 (524218899 0 0) 0x4c17e00 con 0x5250b00 -9615> 2014-08-13 17:38:04.407700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9614> 2014-08-13 17:38:04.407675 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 -9613> 2014-08-13 17:38:04.407705 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9612> 2014-08-13 17:38:04.407707 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c17e00 pg_info(1 pgs e194486:22.220) v4 -9611> 2014-08-13 17:38:04.407712 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407287, event: header_read, request: pg_info(1 pgs e194486:22.220) v4 -9610> 2014-08-13 17:38:04.407714 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9609> 2014-08-13 17:38:04.407719 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407291, event: throttled, request: pg_info(1 pgs e194486:22.220) v4 -9608> 2014-08-13 17:38:04.407704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) -9607> 2014-08-13 17:38:04.407725 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407380, event: all_read, request: pg_info(1 pgs e194486:22.220) v4 -9606> 2014-08-13 17:38:04.407731 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407700, event: dispatched, request: pg_info(1 pgs e194486:22.220) v4 -9605> 2014-08-13 17:38:04.407723 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: Started advmap -9604> 2014-08-13 17:38:04.407738 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407738, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.220) v4 -9603> 2014-08-13 17:38:04.407727 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: got info from osd.328 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) -9602> 2014-08-13 17:38:04.407745 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.220) v4 from osd.351 -9601> 2014-08-13 17:38:04.407750 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c17e00 -9600> 2014-08-13 17:38:04.407754 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407754, event: started, request: pg_info(1 pgs e194486:22.220) v4 -9599> 2014-08-13 17:38:04.407744 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down -9598> 2014-08-13 17:38:04.407751 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.023098 7 0.000370 -9597> 2014-08-13 17:38:04.407766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map -9596> 2014-08-13 17:38:04.407781 7fe9a63f4700 5 -- op tracker -- , seq: 487, time: 2014-08-13 17:38:04.407780, event: done, request: pg_info(1 pgs e194486:22.220) v4 -9595> 2014-08-13 17:38:04.407773 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9594> 2014-08-13 17:38:04.407791 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9593> 2014-08-13 17:38:04.407752 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] _activate_committed 194486 telling primary -9592> 2014-08-13 17:38:04.407794 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9591> 2014-08-13 17:38:04.407805 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing -9590> 2014-08-13 17:38:04.407795 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9589> 2014-08-13 17:38:04.407804 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters -9588> 2014-08-13 17:38:04.407810 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194486:22.14a) v4 -- ?+0 0x4c163c0 -9587> 2014-08-13 17:38:04.407815 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194477 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9586> 2014-08-13 17:38:04.407824 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9585> 2014-08-13 17:38:04.407834 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'506 -> 194215'506 -9584> 2014-08-13 17:38:04.407843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9583> 2014-08-13 17:38:04.407842 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] flushed -9582> 2014-08-13 17:38:04.407866 7fe99ebe8700 10 log is not dirty -9581> 2014-08-13 17:38:04.407858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] state: Activate Finished -9580> 2014-08-13 17:38:04.407883 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] flushed -9579> 2014-08-13 17:38:04.407895 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 -9578> 2014-08-13 17:38:04.407920 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9577> 2014-08-13 17:38:04.407926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering advmap -9576> 2014-08-13 17:38:04.407947 7fe99f5e9700 10 log is not dirty -9575> 2014-08-13 17:38:04.407943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false -9574> 2014-08-13 17:38:04.407959 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Started advmap -9573> 2014-08-13 17:38:04.407975 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down -9572> 2014-08-13 17:38:04.407975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 301 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -9571> 2014-08-13 17:38:04.407993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map -9570> 2014-08-13 17:38:04.408001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.301 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -9569> 2014-08-13 17:38:04.408011 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary -9568> 2014-08-13 17:38:04.408026 7fe99f5e9700 10 log is not dirty -9567> 2014-08-13 17:38:04.408028 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2648 -9566> 2014-08-13 17:38:04.408034 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 54 notify: (query_epoch:194486, epoch_sent:194486, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194483 n=35 ec=183096 les/c 194483/194483 194485/194485/194485)) -9565> 2014-08-13 17:38:04.408044 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] take_waiters -9564> 2014-08-13 17:38:04.408049 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.351 192.168.1.102:6830/16444 10 ==== MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 ==== 26+0+0 (1952673400 0 0) 0x5106e40 con 0x5250b00 -9563> 2014-08-13 17:38:04.408068 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9562> 2014-08-13 17:38:04.408055 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 54, already have info from that osd, ignoring -9561> 2014-08-13 17:38:04.408071 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9560> 2014-08-13 17:38:04.408077 7fe99f5e9700 10 log is not dirty -9559> 2014-08-13 17:38:04.408063 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.891559 6 0.000448 -9558> 2014-08-13 17:38:04.408077 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106e40 MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9557> 2014-08-13 17:38:04.408087 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.407945, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9556> 2014-08-13 17:38:04.408095 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.407948, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9555> 2014-08-13 17:38:04.408087 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering -9554> 2014-08-13 17:38:04.408085 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 291 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -9553> 2014-08-13 17:38:04.408101 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.408017, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9552> 2014-08-13 17:38:04.408115 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.408067, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9551> 2014-08-13 17:38:04.408104 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.933018 0 0.000000 -9550> 2014-08-13 17:38:04.408122 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.408122, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9549> 2014-08-13 17:38:04.408108 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9548> 2014-08-13 17:38:04.408129 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106e40 -9547> 2014-08-13 17:38:04.408128 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -9546> 2014-08-13 17:38:04.408143 7fe9a63f4700 5 -- op tracker -- , seq: 488, time: 2014-08-13 17:38:04.408142, event: done, request: MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194486 v2 -9545> 2014-08-13 17:38:04.408156 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9544> 2014-08-13 17:38:04.408159 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9543> 2014-08-13 17:38:04.408146 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194477 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate -9542> 2014-08-13 17:38:04.408142 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged -9541> 2014-08-13 17:38:04.408169 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] -9540> 2014-08-13 17:38:04.408186 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] -9539> 2014-08-13 17:38:04.408168 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.092910 7 0.006903 -9538> 2014-08-13 17:38:04.408209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'466 -> 194215'466 -9537> 2014-08-13 17:38:04.408208 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -9536> 2014-08-13 17:38:04.408232 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) -9535> 2014-08-13 17:38:04.408235 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9534> 2014-08-13 17:38:04.408251 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 is up to date, queueing in pending_activators -9533> 2014-08-13 17:38:04.408258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9532> 2014-08-13 17:38:04.408275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate -9531> 2014-08-13 17:38:04.408279 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9530> 2014-08-13 17:38:04.408295 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) -9529> 2014-08-13 17:38:04.408302 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9528> 2014-08-13 17:38:04.408314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 is up to date, queueing in pending_activators -9527> 2014-08-13 17:38:04.408323 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9526> 2014-08-13 17:38:04.408336 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate -9525> 2014-08-13 17:38:04.408323 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] _activate_committed 194486 telling primary -9524> 2014-08-13 17:38:04.408345 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9523> 2014-08-13 17:38:04.408369 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a491e0 don't know epoch, doing nothing -9522> 2014-08-13 17:38:04.408359 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered -9521> 2014-08-13 17:38:04.408374 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.328 192.168.1.104:6812/64303 -- pg_info(1 pgs e194486:22.2b9) v4 -- ?+0 0x5824d00 -9520> 2014-08-13 17:38:04.408376 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished -9519> 2014-08-13 17:38:04.408392 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -9518> 2014-08-13 17:38:04.408401 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] flushed -9517> 2014-08-13 17:38:04.408410 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9516> 2014-08-13 17:38:04.408407 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -9515> 2014-08-13 17:38:04.408429 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9514> 2014-08-13 17:38:04.408431 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,291,350,367,378 -9513> 2014-08-13 17:38:04.408451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,291,350] (== acting) backfill_targets 367,378 -9512> 2014-08-13 17:38:04.408487 7fe99ebe8700 10 log is not dirty -9511> 2014-08-13 17:38:04.408473 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog -9510> 2014-08-13 17:38:04.408497 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000288 0 0.000000 -9509> 2014-08-13 17:38:04.408519 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -9508> 2014-08-13 17:38:04.408537 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.291 has no missing, identical log -9507> 2014-08-13 17:38:04.408535 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 -9506> 2014-08-13 17:38:04.408562 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9505> 2014-08-13 17:38:04.408558 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.350 has no missing, identical log -9504> 2014-08-13 17:38:04.408567 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap -9503> 2014-08-13 17:38:04.408582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active -9502> 2014-08-13 17:38:04.408586 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -9501> 2014-08-13 17:38:04.408603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map -9500> 2014-08-13 17:38:04.408603 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000084 0 0.000000 -9499> 2014-08-13 17:38:04.408626 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -9498> 2014-08-13 17:38:04.408645 7fe99f5e9700 10 log is not dirty -9497> 2014-08-13 17:38:04.408680 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters -9496> 2014-08-13 17:38:04.408675 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) -9495> 2014-08-13 17:38:04.408697 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9494> 2014-08-13 17:38:04.408702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.30 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) -9493> 2014-08-13 17:38:04.408713 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9492> 2014-08-13 17:38:04.408731 7fe99ebe8700 10 log is not dirty -9491> 2014-08-13 17:38:04.408725 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.917019 7 0.000269 -9490> 2014-08-13 17:38:04.408744 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9489> 2014-08-13 17:38:04.408762 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9488> 2014-08-13 17:38:04.408761 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 -9487> 2014-08-13 17:38:04.408785 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9486> 2014-08-13 17:38:04.408781 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194477 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9485> 2014-08-13 17:38:04.408791 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering advmap -9484> 2014-08-13 17:38:04.408798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 -9483> 2014-08-13 17:38:04.408809 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false -9482> 2014-08-13 17:38:04.408822 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] state: Activate Finished -9481> 2014-08-13 17:38:04.408826 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Started advmap -9480> 2014-08-13 17:38:04.408843 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down -9479> 2014-08-13 17:38:04.408859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map -9478> 2014-08-13 17:38:04.408846 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.58 192.168.1.111:6834/14316 10 ==== pg_info(1 pgs e194486:22.14a) v4 ==== 714+0+0 (963460149 0 0) 0x4c163c0 con 0x49e1080 -9477> 2014-08-13 17:38:04.408878 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9476> 2014-08-13 17:38:04.408883 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9475> 2014-08-13 17:38:04.408885 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c163c0 pg_info(1 pgs e194486:22.14a) v4 -9474> 2014-08-13 17:38:04.408876 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary -9473> 2014-08-13 17:38:04.408892 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408720, event: header_read, request: pg_info(1 pgs e194486:22.14a) v4 -9472> 2014-08-13 17:38:04.408903 7fe99f5e9700 10 log is not dirty -9471> 2014-08-13 17:38:04.408871 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed -9470> 2014-08-13 17:38:04.408904 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408721, event: throttled, request: pg_info(1 pgs e194486:22.14a) v4 -9469> 2014-08-13 17:38:04.408895 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2995 -9468> 2014-08-13 17:38:04.408911 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408797, event: all_read, request: pg_info(1 pgs e194486:22.14a) v4 -9467> 2014-08-13 17:38:04.408919 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408877, event: dispatched, request: pg_info(1 pgs e194486:22.14a) v4 -9466> 2014-08-13 17:38:04.408915 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] take_waiters -9465> 2014-08-13 17:38:04.408926 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408926, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.14a) v4 -9464> 2014-08-13 17:38:04.408933 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.14a) v4 from osd.58 -9463> 2014-08-13 17:38:04.408938 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c163c0 -9462> 2014-08-13 17:38:04.408944 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408944, event: started, request: pg_info(1 pgs e194486:22.14a) v4 -9461> 2014-08-13 17:38:04.408932 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.887281 6 0.000470 -9460> 2014-08-13 17:38:04.408939 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 285 info: 22.f3( v 194215'506 lc 0'0 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) -9459> 2014-08-13 17:38:04.408958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering -9458> 2014-08-13 17:38:04.408971 7fe9a63f4700 5 -- op tracker -- , seq: 489, time: 2014-08-13 17:38:04.408971, event: done, request: pg_info(1 pgs e194486:22.14a) v4 -9457> 2014-08-13 17:38:04.408963 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.285 activated and committed -9456> 2014-08-13 17:38:04.408986 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9455> 2014-08-13 17:38:04.408990 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9454> 2014-08-13 17:38:04.408975 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.926369 0 0.000000 -9453> 2014-08-13 17:38:04.408982 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9452> 2014-08-13 17:38:04.408990 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 -9451> 2014-08-13 17:38:04.408999 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -9450> 2014-08-13 17:38:04.409005 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9449> 2014-08-13 17:38:04.409016 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] flushed -9448> 2014-08-13 17:38:04.409016 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194477 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate -9447> 2014-08-13 17:38:04.409033 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -9446> 2014-08-13 17:38:04.409038 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194486:22.f3) v4 -- ?+0 0x4c16200 -9445> 2014-08-13 17:38:04.409044 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] -9444> 2014-08-13 17:38:04.409060 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] -9443> 2014-08-13 17:38:04.409070 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9600 don't know epoch, doing nothing -9442> 2014-08-13 17:38:04.409077 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.312 192.168.1.113:6816/17521 -- pg_info(1 pgs e194486:22.f3) v4 -- ?+0 0x4c16900 -9441> 2014-08-13 17:38:04.409075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'479 -> 194215'479 -9440> 2014-08-13 17:38:04.409097 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) -9439> 2014-08-13 17:38:04.409099 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2860 -9438> 2014-08-13 17:38:04.409126 7fe99f5e9700 10 log is not dirty -9437> 2014-08-13 17:38:04.409116 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 is up to date, queueing in pending_activators -9436> 2014-08-13 17:38:04.409144 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate -9435> 2014-08-13 17:38:04.409162 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) -9434> 2014-08-13 17:38:04.409180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 is up to date, queueing in pending_activators -9433> 2014-08-13 17:38:04.409196 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) -9432> 2014-08-13 17:38:04.409209 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) uptodate -9431> 2014-08-13 17:38:04.409221 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) -9430> 2014-08-13 17:38:04.409232 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered -9429> 2014-08-13 17:38:04.409245 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.981682 7 0.000370 -9428> 2014-08-13 17:38:04.409249 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished -9427> 2014-08-13 17:38:04.409265 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -9426> 2014-08-13 17:38:04.409269 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9425> 2014-08-13 17:38:04.409281 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9424> 2014-08-13 17:38:04.409289 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9423> 2014-08-13 17:38:04.409297 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9422> 2014-08-13 17:38:04.409307 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194477 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9421> 2014-08-13 17:38:04.409305 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] _activate_committed 194486 telling primary -9420> 2014-08-13 17:38:04.409326 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'496 -> 194215'496 -9419> 2014-08-13 17:38:04.409348 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4a44780 don't know epoch, doing nothing -9418> 2014-08-13 17:38:04.409355 7fe99ebe8700 10 log is not dirty -9417> 2014-08-13 17:38:04.409337 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.328 192.168.1.104:6812/64303 9 ==== pg_info(1 pgs e194486:22.2b9) v4 ==== 714+0+0 (2010776045 0 0) 0x5824d00 con 0x4a491e0 -9416> 2014-08-13 17:38:04.409349 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] state: Activate Finished -9415> 2014-08-13 17:38:04.409360 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9414> 2014-08-13 17:38:04.409364 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9413> 2014-08-13 17:38:04.409353 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194486:22.791) v4 -- ?+0 0x5820e00 -9412> 2014-08-13 17:38:04.409366 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824d00 pg_info(1 pgs e194486:22.2b9) v4 -9411> 2014-08-13 17:38:04.409373 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409174, event: header_read, request: pg_info(1 pgs e194486:22.2b9) v4 -9410> 2014-08-13 17:38:04.409382 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409176, event: throttled, request: pg_info(1 pgs e194486:22.2b9) v4 -9409> 2014-08-13 17:38:04.409389 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409276, event: all_read, request: pg_info(1 pgs e194486:22.2b9) v4 -9408> 2014-08-13 17:38:04.409394 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409359, event: dispatched, request: pg_info(1 pgs e194486:22.2b9) v4 -9407> 2014-08-13 17:38:04.409381 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] flushed -9406> 2014-08-13 17:38:04.409405 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409405, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.2b9) v4 -9405> 2014-08-13 17:38:04.409412 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2b9) v4 from osd.328 -9404> 2014-08-13 17:38:04.409416 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824d00 -9403> 2014-08-13 17:38:04.409421 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409421, event: started, request: pg_info(1 pgs e194486:22.2b9) v4 -9402> 2014-08-13 17:38:04.409434 7fe99f5e9700 10 log is not dirty -9401> 2014-08-13 17:38:04.409444 7fe9a63f4700 5 -- op tracker -- , seq: 490, time: 2014-08-13 17:38:04.409444, event: done, request: pg_info(1 pgs e194486:22.2b9) v4 -9400> 2014-08-13 17:38:04.409457 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9399> 2014-08-13 17:38:04.409460 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9398> 2014-08-13 17:38:04.409460 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 20 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -9397> 2014-08-13 17:38:04.409472 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.58 192.168.1.111:6834/14316 11 ==== pg_info(1 pgs e194486:22.14a) v4 ==== 714+0+0 (1301956195 0 0) 0x5827e00 con 0x49e1080 -9396> 2014-08-13 17:38:04.409486 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9395> 2014-08-13 17:38:04.409488 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9394> 2014-08-13 17:38:04.409490 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5827e00 pg_info(1 pgs e194486:22.14a) v4 -9393> 2014-08-13 17:38:04.409494 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409329, event: header_read, request: pg_info(1 pgs e194486:22.14a) v4 -9392> 2014-08-13 17:38:04.409485 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -9391> 2014-08-13 17:38:04.409503 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409330, event: throttled, request: pg_info(1 pgs e194486:22.14a) v4 -9390> 2014-08-13 17:38:04.409509 7fe99f5e9700 10 log is not dirty -9389> 2014-08-13 17:38:04.409509 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409431, event: all_read, request: pg_info(1 pgs e194486:22.14a) v4 -9388> 2014-08-13 17:38:04.409515 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409485, event: dispatched, request: pg_info(1 pgs e194486:22.14a) v4 -9387> 2014-08-13 17:38:04.409521 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409521, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.14a) v4 -9386> 2014-08-13 17:38:04.409527 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.14a) v4 from osd.58 -9385> 2014-08-13 17:38:04.409531 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5827e00 -9384> 2014-08-13 17:38:04.409536 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409535, event: started, request: pg_info(1 pgs e194486:22.14a) v4 -9383> 2014-08-13 17:38:04.409531 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 374 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) -9382> 2014-08-13 17:38:04.409555 7fe9a63f4700 5 -- op tracker -- , seq: 491, time: 2014-08-13 17:38:04.409555, event: done, request: pg_info(1 pgs e194486:22.14a) v4 -9381> 2014-08-13 17:38:04.409553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.374 activated and committed -9380> 2014-08-13 17:38:04.409566 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9379> 2014-08-13 17:38:04.409569 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9378> 2014-08-13 17:38:04.409569 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9377> 2014-08-13 17:38:04.409564 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 -9376> 2014-08-13 17:38:04.409585 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9375> 2014-08-13 17:38:04.409598 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9374> 2014-08-13 17:38:04.409607 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing -9373> 2014-08-13 17:38:04.409605 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap -9372> 2014-08-13 17:38:04.409610 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194486:22.99) v4 -- ?+0 0x4c16e40 -9371> 2014-08-13 17:38:04.409622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -9370> 2014-08-13 17:38:04.409637 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing -9369> 2014-08-13 17:38:04.409641 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194486:22.99) v4 -- ?+0 0x5820540 -9368> 2014-08-13 17:38:04.409639 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map -9367> 2014-08-13 17:38:04.409659 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2871 -9366> 2014-08-13 17:38:04.409683 7fe99f5e9700 10 log is not dirty -9365> 2014-08-13 17:38:04.409685 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters -9364> 2014-08-13 17:38:04.409701 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9363> 2014-08-13 17:38:04.409716 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9362> 2014-08-13 17:38:04.409733 7fe99ebe8700 10 log is not dirty -9361> 2014-08-13 17:38:04.409783 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -9360> 2014-08-13 17:38:04.409830 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring -9359> 2014-08-13 17:38:04.409876 7fe99f5e9700 10 log is not dirty -9358> 2014-08-13 17:38:04.409875 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] _activate_committed 194486 telling primary -9357> 2014-08-13 17:38:04.409858 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 -9356> 2014-08-13 17:38:04.409886 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 364 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192966) -9355> 2014-08-13 17:38:04.409908 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 -9354> 2014-08-13 17:38:04.409912 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9353> 2014-08-13 17:38:04.409912 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194486:22.116) v4 -- ?+0 0x5822d80 -9352> 2014-08-13 17:38:04.409911 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] log request from 364 -9351> 2014-08-13 17:38:04.409922 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Started advmap -9350> 2014-08-13 17:38:04.409937 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] flushed -9349> 2014-08-13 17:38:04.409942 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] -9348> 2014-08-13 17:38:04.409960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, transitioning to Reset -9347> 2014-08-13 17:38:04.409967 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info+missing+log since 194477'0 -9346> 2014-08-13 17:38:04.409980 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started/Stray 0.924716 4 0.000185 -9345> 2014-08-13 17:38:04.409989 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending log((194422'17074,194422'17074], crt=194422'17074) missing(48) -9344> 2014-08-13 17:38:04.410011 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 don't know epoch, doing nothing -9343> 2014-08-13 17:38:04.410004 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started 0.924781 0 0.000000 -9342> 2014-08-13 17:38:04.410014 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.114:6826/52317 -- pg_log(24.1de epoch 194486 log log((194422'17074,194422'17074], crt=194422'17074) query_epoch 194486) v4 -- ?+0 0x5fb1800 con 0x4cfc620 -9341> 2014-08-13 17:38:04.410024 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Reset -9340> 2014-08-13 17:38:04.410043 7fe99f5e9700 10 log is not dirty -9339> 2014-08-13 17:38:04.410040 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194485 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194486 -9338> 2014-08-13 17:38:04.410043 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 -9337> 2014-08-13 17:38:04.410058 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap -9336> 2014-08-13 17:38:04.410068 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] flushed -9335> 2014-08-13 17:38:04.410076 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 -9334> 2014-08-13 17:38:04.410093 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] -9333> 2014-08-13 17:38:04.410108 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, calling start_peering_interval again -9332> 2014-08-13 17:38:04.410124 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4]/[6,374] r=-1 lpr=194486 pi=193125-194484/30 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194486 -9331> 2014-08-13 17:38:04.410158 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194485 up [6,374,4](6) acting [6,374](6)): not rw, up_thru 194483 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194485-194485 up [6,374,4](6) acting [6,374](6)) : primary up 192967-194483 does not include interval -9330> 2014-08-13 17:38:04.410175 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194485/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194485-194485 up [6,374,4](6) acting [6,374](6)) -9329> 2014-08-13 17:38:04.410192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374,4] -> [6,374,4], acting [6,374] -> [6,374,4], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> 2 -9328> 2014-08-13 17:38:04.410213 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -9327> 2014-08-13 17:38:04.410230 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing -9326> 2014-08-13 17:38:04.410234 7fe99f5e9700 7 osd.4 194486 do_notify osd 304 on 1 PGs -9325> 2014-08-13 17:38:04.410223 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change -9324> 2014-08-13 17:38:04.410240 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary -9323> 2014-08-13 17:38:04.410238 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(31) epoch 194486) v5 -- ?+0 0x5826580 con 0x49e39c0 -9322> 2014-08-13 17:38:04.410254 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops -9321> 2014-08-13 17:38:04.410271 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops -9320> 2014-08-13 17:38:04.410277 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing -9319> 2014-08-13 17:38:04.410267 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.328 192.168.1.104:6812/64303 10 ==== pg_info(1 pgs e194486:22.2b9) v4 ==== 714+0+0 (3601806094 0 0) 0x5824ec0 con 0x4a491e0 -9318> 2014-08-13 17:38:04.410289 7fe99f5e9700 7 osd.4 194486 do_notify osd 356 on 2 PGs -9317> 2014-08-13 17:38:04.410292 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9316> 2014-08-13 17:38:04.410296 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9315> 2014-08-13 17:38:04.410298 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824ec0 pg_info(1 pgs e194486:22.2b9) v4 -9314> 2014-08-13 17:38:04.410292 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(31),24.3b7(39) epoch 194486) v5 -- ?+0 0x58240c0 con 0x4a486e0 -9313> 2014-08-13 17:38:04.410286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops -9312> 2014-08-13 17:38:04.410307 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410154, event: header_read, request: pg_info(1 pgs e194486:22.2b9) v4 -9311> 2014-08-13 17:38:04.410316 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410156, event: throttled, request: pg_info(1 pgs e194486:22.2b9) v4 -9310> 2014-08-13 17:38:04.410308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup -9309> 2014-08-13 17:38:04.410322 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410243, event: all_read, request: pg_info(1 pgs e194486:22.2b9) v4 -9308> 2014-08-13 17:38:04.410328 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410291, event: dispatched, request: pg_info(1 pgs e194486:22.2b9) v4 -9307> 2014-08-13 17:38:04.410324 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change -9306> 2014-08-13 17:38:04.410335 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410335, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.2b9) v4 -9305> 2014-08-13 17:38:04.410342 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.2b9) v4 from osd.328 -9304> 2014-08-13 17:38:04.410347 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824ec0 -9303> 2014-08-13 17:38:04.410352 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410351, event: started, request: pg_info(1 pgs e194486:22.2b9) v4 -9302> 2014-08-13 17:38:04.410374 7fe9a63f4700 5 -- op tracker -- , seq: 492, time: 2014-08-13 17:38:04.410374, event: done, request: pg_info(1 pgs e194486:22.2b9) v4 -9301> 2014-08-13 17:38:04.410359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 45 query_epoch 194486 query: query(info 0'0) -9300> 2014-08-13 17:38:04.410342 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming -9299> 2014-08-13 17:38:04.410388 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9298> 2014-08-13 17:38:04.410392 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9297> 2014-08-13 17:38:04.410387 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info -9296> 2014-08-13 17:38:04.410389 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming -9295> 2014-08-13 17:38:04.410399 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.30 192.168.1.107:6823/55491 7 ==== pg_info(1 pgs e194486:22.791) v4 ==== 714+0+0 (2540518494 0 0) 0x5820e00 con 0x4a44780 -9294> 2014-08-13 17:38:04.410412 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9293> 2014-08-13 17:38:04.410414 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9292> 2014-08-13 17:38:04.410416 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820e00 pg_info(1 pgs e194486:22.791) v4 -9291> 2014-08-13 17:38:04.410407 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_role_change -9290> 2014-08-13 17:38:04.410421 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410237, event: header_read, request: pg_info(1 pgs e194486:22.791) v4 -9289> 2014-08-13 17:38:04.410428 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410241, event: throttled, request: pg_info(1 pgs e194486:22.791) v4 -9288> 2014-08-13 17:38:04.410423 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops -9287> 2014-08-13 17:38:04.410434 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410335, event: all_read, request: pg_info(1 pgs e194486:22.791) v4 -9286> 2014-08-13 17:38:04.410440 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410412, event: dispatched, request: pg_info(1 pgs e194486:22.791) v4 -9285> 2014-08-13 17:38:04.410447 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410446, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.791) v4 -9284> 2014-08-13 17:38:04.410438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery -9283> 2014-08-13 17:38:04.410452 7fe99f5e9700 10 log is not dirty -9282> 2014-08-13 17:38:04.410452 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.791) v4 from osd.30 -9281> 2014-08-13 17:38:04.410457 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820e00 -9280> 2014-08-13 17:38:04.410462 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410462, event: started, request: pg_info(1 pgs e194486:22.791) v4 -9279> 2014-08-13 17:38:04.410453 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state -9278> 2014-08-13 17:38:04.410481 7fe9a63f4700 5 -- op tracker -- , seq: 493, time: 2014-08-13 17:38:04.410481, event: done, request: pg_info(1 pgs e194486:22.791) v4 -9277> 2014-08-13 17:38:04.410471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down -9276> 2014-08-13 17:38:04.410467 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -9275> 2014-08-13 17:38:04.410492 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9274> 2014-08-13 17:38:04.410496 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9273> 2014-08-13 17:38:04.410493 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map -9272> 2014-08-13 17:38:04.410492 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops -9271> 2014-08-13 17:38:04.410516 7fe99f5e9700 10 log is not dirty -9270> 2014-08-13 17:38:04.410523 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 343 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) -9269> 2014-08-13 17:38:04.410554 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] update_heartbeat_peers unchanged -9268> 2014-08-13 17:38:04.410571 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters -9267> 2014-08-13 17:38:04.410544 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed -9266> 2014-08-13 17:38:04.410587 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.000563 1 0.000571 -9265> 2014-08-13 17:38:04.410603 7fe99f5e9700 10 log is not dirty -9264> 2014-08-13 17:38:04.410608 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started -9263> 2014-08-13 17:38:04.410612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 286 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -9262> 2014-08-13 17:38:04.410624 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start -9261> 2014-08-13 17:38:04.410640 7fe99ebe8700 1 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray -9260> 2014-08-13 17:38:04.410637 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9259> 2014-08-13 17:38:04.410660 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000035 0 0.000000 -9258> 2014-08-13 17:38:04.410671 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged -9257> 2014-08-13 17:38:04.410714 7fe99f5e9700 10 log is not dirty -9256> 2014-08-13 17:38:04.410703 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray -9255> 2014-08-13 17:38:04.410717 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.30 192.168.1.107:6823/55491 8 ==== pg_info(1 pgs e194486:22.791) v4 ==== 714+0+0 (1238204298 0 0) 0x5f01500 con 0x4a44780 -9254> 2014-08-13 17:38:04.410739 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9253> 2014-08-13 17:38:04.410723 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 324 notify: (query_epoch:194486, epoch_sent:194486, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) -9252> 2014-08-13 17:38:04.410726 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9251> 2014-08-13 17:38:04.410743 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9250> 2014-08-13 17:38:04.410746 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f01500 pg_info(1 pgs e194486:22.791) v4 -9249> 2014-08-13 17:38:04.410745 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9248> 2014-08-13 17:38:04.410752 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410590, event: header_read, request: pg_info(1 pgs e194486:22.791) v4 -9247> 2014-08-13 17:38:04.410768 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410594, event: throttled, request: pg_info(1 pgs e194486:22.791) v4 -9246> 2014-08-13 17:38:04.410775 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410667, event: all_read, request: pg_info(1 pgs e194486:22.791) v4 -9245> 2014-08-13 17:38:04.410780 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410739, event: dispatched, request: pg_info(1 pgs e194486:22.791) v4 -9244> 2014-08-13 17:38:04.410787 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410786, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.791) v4 -9243> 2014-08-13 17:38:04.410793 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.791) v4 from osd.30 -9242> 2014-08-13 17:38:04.410798 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f01500 -9241> 2014-08-13 17:38:04.410786 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring -9240> 2014-08-13 17:38:04.410803 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410802, event: started, request: pg_info(1 pgs e194486:22.791) v4 -9239> 2014-08-13 17:38:04.410809 7fe99f5e9700 10 log is not dirty -9238> 2014-08-13 17:38:04.410825 7fe9a63f4700 5 -- op tracker -- , seq: 494, time: 2014-08-13 17:38:04.410825, event: done, request: pg_info(1 pgs e194486:22.791) v4 -9237> 2014-08-13 17:38:04.410837 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9236> 2014-08-13 17:38:04.410840 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9235> 2014-08-13 17:38:04.410840 7fe99ebe8700 10 log is not dirty -9234> 2014-08-13 17:38:04.410849 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.345 192.168.1.110:6820/46337 7 ==== pg_info(1 pgs e194486:22.116) v4 ==== 714+0+0 (2252627472 0 0) 0x5822d80 con 0x4c70f20 -9233> 2014-08-13 17:38:04.410816 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -9232> 2014-08-13 17:38:04.410862 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9231> 2014-08-13 17:38:04.410864 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9230> 2014-08-13 17:38:04.410866 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822d80 pg_info(1 pgs e194486:22.116) v4 -9229> 2014-08-13 17:38:04.410870 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410592, event: header_read, request: pg_info(1 pgs e194486:22.116) v4 -9228> 2014-08-13 17:38:04.410877 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410595, event: throttled, request: pg_info(1 pgs e194486:22.116) v4 -9227> 2014-08-13 17:38:04.410882 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410685, event: all_read, request: pg_info(1 pgs e194486:22.116) v4 -9226> 2014-08-13 17:38:04.410888 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410861, event: dispatched, request: pg_info(1 pgs e194486:22.116) v4 -9225> 2014-08-13 17:38:04.410894 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410894, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.116) v4 -9224> 2014-08-13 17:38:04.410900 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.116) v4 from osd.345 -9223> 2014-08-13 17:38:04.410903 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822d80 -9222> 2014-08-13 17:38:04.410908 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410908, event: started, request: pg_info(1 pgs e194486:22.116) v4 -9221> 2014-08-13 17:38:04.410902 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 -9220> 2014-08-13 17:38:04.410929 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9219> 2014-08-13 17:38:04.410927 7fe9a63f4700 5 -- op tracker -- , seq: 495, time: 2014-08-13 17:38:04.410927, event: done, request: pg_info(1 pgs e194486:22.116) v4 -9218> 2014-08-13 17:38:04.410935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap -9217> 2014-08-13 17:38:04.410947 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9216> 2014-08-13 17:38:04.410954 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9215> 2014-08-13 17:38:04.410955 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -9214> 2014-08-13 17:38:04.410960 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.380 192.168.1.114:6822/51844 4 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x54db3c0 con 0x5250dc0 -9213> 2014-08-13 17:38:04.410974 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9212> 2014-08-13 17:38:04.410976 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9211> 2014-08-13 17:38:04.410978 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54db3c0 osd_map(194486..194486 src has 185139..194486) v3 -9210> 2014-08-13 17:38:04.410972 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map -9209> 2014-08-13 17:38:04.410987 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -9208> 2014-08-13 17:38:04.410992 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -9207> 2014-08-13 17:38:04.410995 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9206> 2014-08-13 17:38:04.410996 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9205> 2014-08-13 17:38:04.411038 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters -9204> 2014-08-13 17:38:04.411054 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -9203> 2014-08-13 17:38:04.411069 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -9202> 2014-08-13 17:38:04.411093 7fe99ebe8700 10 log is not dirty -9201> 2014-08-13 17:38:04.411117 7fe99ebe8700 10 osd.4 pg_epoch: 194485 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 -9200> 2014-08-13 17:38:04.411140 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -9199> 2014-08-13 17:38:04.411129 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.380 192.168.1.114:6822/51844 5 ==== pg_notify(22.4df(27) epoch 194486) v5 ==== 2227+0+0 (2899609086 0 0) 0x513dcc0 con 0x5250dc0 -9198> 2014-08-13 17:38:04.411158 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9197> 2014-08-13 17:38:04.411146 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering advmap -9196> 2014-08-13 17:38:04.411162 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9195> 2014-08-13 17:38:04.411167 7fe9a63f4700 20 osd.4 194486 _dispatch 0x513dcc0 pg_notify(22.4df(27) epoch 194486) v5 -9194> 2014-08-13 17:38:04.411166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194485, need_up_thru now false -9193> 2014-08-13 17:38:04.411153 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring -9192> 2014-08-13 17:38:04.411174 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.410903, event: header_read, request: pg_notify(22.4df(27) epoch 194486) v5 -9191> 2014-08-13 17:38:04.411195 7fe99f5e9700 10 log is not dirty -9190> 2014-08-13 17:38:04.411184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Started advmap -9189> 2014-08-13 17:38:04.411188 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.410907, event: throttled, request: pg_notify(22.4df(27) epoch 194486) v5 -9188> 2014-08-13 17:38:04.411208 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.411092, event: all_read, request: pg_notify(22.4df(27) epoch 194486) v5 -9187> 2014-08-13 17:38:04.411215 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.411157, event: dispatched, request: pg_notify(22.4df(27) epoch 194486) v5 -9186> 2014-08-13 17:38:04.411212 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down -9185> 2014-08-13 17:38:04.411223 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.411222, event: waiting_for_osdmap, request: pg_notify(22.4df(27) epoch 194486) v5 -9184> 2014-08-13 17:38:04.411230 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.380 -9183> 2014-08-13 17:38:04.411234 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x513dcc0 -9182> 2014-08-13 17:38:04.411219 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) -9181> 2014-08-13 17:38:04.411240 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.411239, event: started, request: pg_notify(22.4df(27) epoch 194486) v5 -9180> 2014-08-13 17:38:04.411231 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map -9179> 2014-08-13 17:38:04.411244 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.373 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) -9178> 2014-08-13 17:38:04.411252 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary -9177> 2014-08-13 17:38:04.411266 7fe9a63f4700 5 -- op tracker -- , seq: 496, time: 2014-08-13 17:38:04.411266, event: done, request: pg_notify(22.4df(27) epoch 194486) v5 -9176> 2014-08-13 17:38:04.411283 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9175> 2014-08-13 17:38:04.411271 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194486:2593 -9174> 2014-08-13 17:38:04.411270 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.945729 7 0.000263 -9173> 2014-08-13 17:38:04.411287 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9172> 2014-08-13 17:38:04.411289 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] take_waiters -9171> 2014-08-13 17:38:04.411306 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.870532 3 0.000524 -9170> 2014-08-13 17:38:04.411310 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9169> 2014-08-13 17:38:04.411297 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] flushed -9168> 2014-08-13 17:38:04.411327 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering -9167> 2014-08-13 17:38:04.411334 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9166> 2014-08-13 17:38:04.411344 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.919524 0 0.000000 -9165> 2014-08-13 17:38:04.411356 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194477 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9164> 2014-08-13 17:38:04.411367 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -9163> 2014-08-13 17:38:04.411378 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'522 -> 194215'522 -9162> 2014-08-13 17:38:04.411384 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194478 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate -9161> 2014-08-13 17:38:04.411406 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] -9160> 2014-08-13 17:38:04.411415 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] state: Activate Finished -9159> 2014-08-13 17:38:04.411423 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] -9158> 2014-08-13 17:38:04.411438 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'511 -> 194215'511 -9157> 2014-08-13 17:38:04.411461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) -9156> 2014-08-13 17:38:04.411480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 is up to date, queueing in pending_activators -9155> 2014-08-13 17:38:04.411520 7fe99f5e9700 10 log is not dirty -9154> 2014-08-13 17:38:04.411566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -9153> 2014-08-13 17:38:04.411592 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 320, already have info from that osd, ignoring -9152> 2014-08-13 17:38:04.411610 7fe99f5e9700 10 log is not dirty -9151> 2014-08-13 17:38:04.411599 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) uptodate -9150> 2014-08-13 17:38:04.411621 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) -9149> 2014-08-13 17:38:04.411640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 is up to date, queueing in pending_activators -9148> 2014-08-13 17:38:04.411624 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 330 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193227) -9147> 2014-08-13 17:38:04.411683 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] log request from 330 -9146> 2014-08-13 17:38:04.411703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485) missing missing(28) -9145> 2014-08-13 17:38:04.411733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info+missing+log since 194478'0 -9144> 2014-08-13 17:38:04.411748 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.376 has 28 missing -9143> 2014-08-13 17:38:04.411759 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending log((194422'15537,194422'15537], crt=194422'15537) missing(64) -9142> 2014-08-13 17:38:04.411766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.4 -9141> 2014-08-13 17:38:04.411782 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 don't know epoch, doing nothing -9140> 2014-08-13 17:38:04.411785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.4 -9139> 2014-08-13 17:38:04.411786 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6802/4876 -- pg_log(24.3f7 epoch 194486 log log((194422'15537,194422'15537], crt=194422'15537) query_epoch 194486) v4 -- ?+0 0x5fb1200 con 0x4ec9b80 -9138> 2014-08-13 17:38:04.411812 7fe99f5e9700 10 log is not dirty -9137> 2014-08-13 17:38:04.411803 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.4 -9136> 2014-08-13 17:38:04.411820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.4 -9135> 2014-08-13 17:38:04.411835 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.4 -9134> 2014-08-13 17:38:04.411850 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.4 -9133> 2014-08-13 17:38:04.411866 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.4 -9132> 2014-08-13 17:38:04.411881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.4 -9131> 2014-08-13 17:38:04.411897 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.4 -9130> 2014-08-13 17:38:04.411889 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] _activate_committed 194486 telling primary -9129> 2014-08-13 17:38:04.411914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.4 -9128> 2014-08-13 17:38:04.411929 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.4 -9127> 2014-08-13 17:38:04.411945 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.4 -9126> 2014-08-13 17:38:04.411960 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.4 -9125> 2014-08-13 17:38:04.411975 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.4 -9124> 2014-08-13 17:38:04.411990 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.4 -9123> 2014-08-13 17:38:04.411997 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing -9122> 2014-08-13 17:38:04.411993 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 365 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -9121> 2014-08-13 17:38:04.412007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.4 -9120> 2014-08-13 17:38:04.412007 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194486:22.1fb) v4 -- ?+0 0x5824600 -9119> 2014-08-13 17:38:04.412024 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.4 -9118> 2014-08-13 17:38:04.412023 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.365 -9117> 2014-08-13 17:38:04.412041 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.4 -9116> 2014-08-13 17:38:04.412039 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] flushed -9115> 2014-08-13 17:38:04.412058 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.4 -9114> 2014-08-13 17:38:04.412077 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.4 -9113> 2014-08-13 17:38:04.412094 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.4 -9112> 2014-08-13 17:38:04.412058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -9111> 2014-08-13 17:38:04.412109 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.4 -9110> 2014-08-13 17:38:04.412122 7fe99f5e9700 10 log is not dirty -9109> 2014-08-13 17:38:04.412129 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.4 -9108> 2014-08-13 17:38:04.412144 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.4 -9107> 2014-08-13 17:38:04.412160 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.4 -9106> 2014-08-13 17:38:04.412161 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 351 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) -9105> 2014-08-13 17:38:04.412176 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.4 -9104> 2014-08-13 17:38:04.412192 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.4 -9103> 2014-08-13 17:38:04.412219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.4 -9102> 2014-08-13 17:38:04.412193 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -9101> 2014-08-13 17:38:04.412241 7fe99f5e9700 10 log is not dirty -9100> 2014-08-13 17:38:04.412250 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) -9099> 2014-08-13 17:38:04.412235 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} -9098> 2014-08-13 17:38:04.412284 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -9097> 2014-08-13 17:38:04.412286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 338 as a source -9096> 2014-08-13 17:38:04.412302 7fe99f5e9700 10 log is not dirty -9095> 2014-08-13 17:38:04.412303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.338 -9094> 2014-08-13 17:38:04.412314 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -9093> 2014-08-13 17:38:04.412320 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.338 -9092> 2014-08-13 17:38:04.412339 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.338 -9091> 2014-08-13 17:38:04.412338 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops -9090> 2014-08-13 17:38:04.412355 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.338 -9089> 2014-08-13 17:38:04.412371 7fe99f5e9700 10 log is not dirty -9088> 2014-08-13 17:38:04.412373 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.338 -9087> 2014-08-13 17:38:04.412378 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 318 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) -9086> 2014-08-13 17:38:04.412390 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.338 -9085> 2014-08-13 17:38:04.412400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.318 activated and committed -9084> 2014-08-13 17:38:04.412421 7fe99f5e9700 10 log is not dirty -9083> 2014-08-13 17:38:04.412410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.338 -9082> 2014-08-13 17:38:04.412427 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.338 -9081> 2014-08-13 17:38:04.412427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 26 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -9080> 2014-08-13 17:38:04.412445 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.338 -9079> 2014-08-13 17:38:04.412450 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.26 -9078> 2014-08-13 17:38:04.412463 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.338 -9077> 2014-08-13 17:38:04.412481 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.338 -9076> 2014-08-13 17:38:04.412469 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours -9075> 2014-08-13 17:38:04.412506 7fe99f5e9700 10 log is not dirty -9074> 2014-08-13 17:38:04.412499 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.338 -9073> 2014-08-13 17:38:04.412516 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.338 -9072> 2014-08-13 17:38:04.412514 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -9071> 2014-08-13 17:38:04.412533 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.338 -9070> 2014-08-13 17:38:04.412535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9069> 2014-08-13 17:38:04.412551 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.338 -9068> 2014-08-13 17:38:04.412568 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.338 -9067> 2014-08-13 17:38:04.412568 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged -9066> 2014-08-13 17:38:04.412587 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.338 -9065> 2014-08-13 17:38:04.412593 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetInfo 0.133158 7 0.000735 -9064> 2014-08-13 17:38:04.412604 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.338 -9063> 2014-08-13 17:38:04.412618 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetLog -9062> 2014-08-13 17:38:04.412622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.338 -9061> 2014-08-13 17:38:04.412640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.338 -9060> 2014-08-13 17:38:04.412644 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.4 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9059> 2014-08-13 17:38:04.412657 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.338 -9058> 2014-08-13 17:38:04.412667 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9057> 2014-08-13 17:38:04.412676 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.338 -9056> 2014-08-13 17:38:04.412689 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9055> 2014-08-13 17:38:04.412694 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.338 -9054> 2014-08-13 17:38:04.412712 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.338 -9053> 2014-08-13 17:38:04.412708 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9052> 2014-08-13 17:38:04.412730 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.338 -9051> 2014-08-13 17:38:04.412730 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9050> 2014-08-13 17:38:04.412749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.338 -9049> 2014-08-13 17:38:04.412767 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.338 -9048> 2014-08-13 17:38:04.412784 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.338 -9047> 2014-08-13 17:38:04.412788 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting newest update on osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) shard 286 (up) backfill 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.375 (up) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) shard 10 (stray) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -9046> 2014-08-13 17:38:04.412812 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] actingbackfill is 4,10,286,375 -9045> 2014-08-13 17:38:04.412802 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} -9044> 2014-08-13 17:38:04.412853 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 376 as a source -9043> 2014-08-13 17:38:04.412851 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] choose_acting want [4,375,10] (== acting) backfill_targets 286 -9042> 2014-08-13 17:38:04.412871 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 also missing on osd.376 -9041> 2014-08-13 17:38:04.412879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: leaving GetLog -9040> 2014-08-13 17:38:04.412890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 also missing on osd.376 -9039> 2014-08-13 17:38:04.412908 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 also missing on osd.376 -9038> 2014-08-13 17:38:04.412907 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetLog 0.000288 0 0.000000 -9037> 2014-08-13 17:38:04.412926 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 also missing on osd.376 -9036> 2014-08-13 17:38:04.412932 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetMissing -9035> 2014-08-13 17:38:04.412943 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 also missing on osd.376 -9034> 2014-08-13 17:38:04.412962 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 also missing on osd.376 -9033> 2014-08-13 17:38:04.412959 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.10 has no missing, identical log -9032> 2014-08-13 17:38:04.412980 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 also missing on osd.376 -9031> 2014-08-13 17:38:04.412979 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.373 192.168.1.108:6840/7435 14 ==== pg_info(1 pgs e194486:22.1fb) v4 ==== 714+0+0 (1330360748 0 0) 0x5824600 con 0x4e85c20 -9030> 2014-08-13 17:38:04.412983 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.375 has no missing, identical log -9029> 2014-08-13 17:38:04.412999 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9028> 2014-08-13 17:38:04.413008 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9027> 2014-08-13 17:38:04.413011 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824600 pg_info(1 pgs e194486:22.1fb) v4 -9026> 2014-08-13 17:38:04.412998 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 also missing on osd.376 -9025> 2014-08-13 17:38:04.413002 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: still need up_thru update before going active -9024> 2014-08-13 17:38:04.413020 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.412865, event: header_read, request: pg_info(1 pgs e194486:22.1fb) v4 -9023> 2014-08-13 17:38:04.413030 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.412867, event: throttled, request: pg_info(1 pgs e194486:22.1fb) v4 -9022> 2014-08-13 17:38:04.413022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 also missing on osd.376 -9021> 2014-08-13 17:38:04.413036 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.412953, event: all_read, request: pg_info(1 pgs e194486:22.1fb) v4 -9020> 2014-08-13 17:38:04.413025 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetMissing 0.000092 0 0.000000 -9019> 2014-08-13 17:38:04.413042 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.412998, event: dispatched, request: pg_info(1 pgs e194486:22.1fb) v4 -9018> 2014-08-13 17:38:04.413050 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.413050, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1fb) v4 -9017> 2014-08-13 17:38:04.413040 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 also missing on osd.376 -9016> 2014-08-13 17:38:04.413057 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1fb) v4 from osd.373 -9015> 2014-08-13 17:38:04.413062 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824600 -9014> 2014-08-13 17:38:04.413048 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/WaitUpThru -9013> 2014-08-13 17:38:04.413071 7fe99f5e9700 10 log is not dirty -9012> 2014-08-13 17:38:04.413068 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.413067, event: started, request: pg_info(1 pgs e194486:22.1fb) v4 -9011> 2014-08-13 17:38:04.413059 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 also missing on osd.376 -9010> 2014-08-13 17:38:04.413078 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 also missing on osd.376 -9009> 2014-08-13 17:38:04.413091 7fe9a63f4700 5 -- op tracker -- , seq: 497, time: 2014-08-13 17:38:04.413091, event: done, request: pg_info(1 pgs e194486:22.1fb) v4 -9008> 2014-08-13 17:38:04.413106 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -9007> 2014-08-13 17:38:04.413109 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -9006> 2014-08-13 17:38:04.413095 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 also missing on osd.376 -9005> 2014-08-13 17:38:04.413101 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 304 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193250) -9004> 2014-08-13 17:38:04.413114 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 also missing on osd.376 -9003> 2014-08-13 17:38:04.413125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log request from 304 -9002> 2014-08-13 17:38:04.413133 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 also missing on osd.376 -9001> 2014-08-13 17:38:04.413151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 also missing on osd.376 -9000> 2014-08-13 17:38:04.413167 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 also missing on osd.376 -8999> 2014-08-13 17:38:04.413184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 also missing on osd.376 -8998> 2014-08-13 17:38:04.413189 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info+missing+log since 194477'0 -8997> 2014-08-13 17:38:04.413206 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 also missing on osd.376 -8996> 2014-08-13 17:38:04.413209 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending log((194422'16034,194422'16034], crt=194422'16034) missing(61) -8995> 2014-08-13 17:38:04.413230 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing -8994> 2014-08-13 17:38:04.413225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 also missing on osd.376 -8993> 2014-08-13 17:38:04.413242 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 also missing on osd.376 -8992> 2014-08-13 17:38:04.413259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 also missing on osd.376 -8991> 2014-08-13 17:38:04.413275 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 also missing on osd.376 -8990> 2014-08-13 17:38:04.413292 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 also missing on osd.376 -8989> 2014-08-13 17:38:04.413308 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 also missing on osd.376 -8988> 2014-08-13 17:38:04.413233 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6808/22782 -- pg_log(24.33a epoch 194486 log log((194422'16034,194422'16034], crt=194422'16034) query_epoch 194486) v4 -- ?+0 0x5fb0c00 con 0x49e39c0 -8987> 2014-08-13 17:38:04.413325 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 also missing on osd.376 -8986> 2014-08-13 17:38:04.413339 7fe99f5e9700 10 log is not dirty -8985> 2014-08-13 17:38:04.413341 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 also missing on osd.376 -8984> 2014-08-13 17:38:04.413359 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 also missing on osd.376 -8983> 2014-08-13 17:38:04.413373 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} -8982> 2014-08-13 17:38:04.413422 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound -8981> 2014-08-13 17:38:04.413417 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.373 192.168.1.108:6840/7435 15 ==== pg_info(1 pgs e194486:22.1fb) v4 ==== 714+0+0 (19717929 0 0) 0x5f00540 con 0x4e85c20 -8980> 2014-08-13 17:38:04.413438 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8979> 2014-08-13 17:38:04.413441 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8978> 2014-08-13 17:38:04.413443 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f00540 pg_info(1 pgs e194486:22.1fb) v4 -8977> 2014-08-13 17:38:04.413436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 -8976> 2014-08-13 17:38:04.413450 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413274, event: header_read, request: pg_info(1 pgs e194486:22.1fb) v4 -8975> 2014-08-13 17:38:04.413462 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413278, event: throttled, request: pg_info(1 pgs e194486:22.1fb) v4 -8974> 2014-08-13 17:38:04.413469 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413369, event: all_read, request: pg_info(1 pgs e194486:22.1fb) v4 -8973> 2014-08-13 17:38:04.413468 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 293,338,373,376,382,396,397 -8972> 2014-08-13 17:38:04.413482 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413437, event: dispatched, request: pg_info(1 pgs e194486:22.1fb) v4 -8971> 2014-08-13 17:38:04.413490 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413490, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1fb) v4 -8970> 2014-08-13 17:38:04.413487 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished -8969> 2014-08-13 17:38:04.413498 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1fb) v4 from osd.373 -8968> 2014-08-13 17:38:04.413506 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f00540 -8967> 2014-08-13 17:38:04.413477 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] flushed -8966> 2014-08-13 17:38:04.413512 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413512, event: started, request: pg_info(1 pgs e194486:22.1fb) v4 -8965> 2014-08-13 17:38:04.413506 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating -8964> 2014-08-13 17:38:04.413524 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194485 while current is 194486 -8963> 2014-08-13 17:38:04.413540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 NullEvt -8962> 2014-08-13 17:38:04.413540 7fe9a63f4700 5 -- op tracker -- , seq: 498, time: 2014-08-13 17:38:04.413540, event: done, request: pg_info(1 pgs e194486:22.1fb) v4 -8961> 2014-08-13 17:38:04.413571 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8960> 2014-08-13 17:38:04.413574 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8959> 2014-08-13 17:38:04.413651 7fe99ebe8700 10 log is not dirty -8958> 2014-08-13 17:38:04.413619 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -8957> 2014-08-13 17:38:04.413680 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -8956> 2014-08-13 17:38:04.413657 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.340 -8955> 2014-08-13 17:38:04.413690 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e3d800 192.168.1.107:6809/53141 -8954> 2014-08-13 17:38:04.413708 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- osd_map(194486..194486 src has 185139..194486) v3 -- ?+0 0x272af40 con 0x4e3d800 -8953> 2014-08-13 17:38:04.413689 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.340 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours -8952> 2014-08-13 17:38:04.413736 7fe99f5e9700 10 log is not dirty -8951> 2014-08-13 17:38:04.413849 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.6 has 194486 -8950> 2014-08-13 17:38:04.413858 7fe99ebe8700 7 osd.4 194486 do_notify osd 6 on 1 PGs -8949> 2014-08-13 17:38:04.413862 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(31) epoch 194486) v5 -- ?+0 0x5822840 con 0x4e3d800 -8948> 2014-08-13 17:38:04.414180 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194486 peer_activated now 4 last_epoch_started 194483 same_interval_since 194485 -8947> 2014-08-13 17:38:04.414193 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 28 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) -8946> 2014-08-13 17:38:04.414219 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] flushed -8945> 2014-08-13 17:38:04.414238 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.28 activated and committed -8944> 2014-08-13 17:38:04.414243 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing -8943> 2014-08-13 17:38:04.414259 7fe99f5e9700 10 log is not dirty -8942> 2014-08-13 17:38:04.414258 7fe99ebe8700 7 osd.4 194486 do_notify osd 15 on 1 PGs -8941> 2014-08-13 17:38:04.414263 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(35) epoch 194486) v5 -- ?+0 0x5821f80 con 0x4d78dc0 -8940> 2014-08-13 17:38:04.414267 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193151) -8939> 2014-08-13 17:38:04.414295 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing -8938> 2014-08-13 17:38:04.414299 7fe99ebe8700 7 osd.4 194486 do_notify osd 31 on 1 PGs -8937> 2014-08-13 17:38:04.414287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log request from 356 -8936> 2014-08-13 17:38:04.414304 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(43) epoch 194486) v5 -- ?+0 0x5104980 con 0x49e5960 -8935> 2014-08-13 17:38:04.414321 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing -8934> 2014-08-13 17:38:04.414324 7fe99ebe8700 7 osd.4 194486 do_notify osd 51 on 1 PGs -8933> 2014-08-13 17:38:04.414327 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.112:6832/6655 -- pg_notify(22.537(25) epoch 194486) v5 -- ?+0 0x5102300 con 0x4e39e40 -8932> 2014-08-13 17:38:04.414347 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -8931> 2014-08-13 17:38:04.414350 7fe99ebe8700 7 osd.4 194486 do_notify osd 285 on 1 PGs -8930> 2014-08-13 17:38:04.414351 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info+missing+log since 194477'0 -8929> 2014-08-13 17:38:04.414352 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(31) epoch 194486) v5 -- ?+0 0x5107380 con 0x4a45280 -8928> 2014-08-13 17:38:04.414372 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending log((194422'17433,194422'17433], crt=194422'17433) missing(51) -8927> 2014-08-13 17:38:04.414414 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing -8926> 2014-08-13 17:38:04.414422 7fe99ebe8700 10 osd.4 194486 send_incremental_map 194485 -> 194486 to 0x4e3c8e0 192.168.1.113:6812/16056 -8925> 2014-08-13 17:38:04.414420 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6832/5235 -- pg_log(24.350 epoch 194486 log log((194422'17433,194422'17433], crt=194422'17433) query_epoch 194486) v4 -- ?+0 0x5fb0600 con 0x4a486e0 -8924> 2014-08-13 17:38:04.414438 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.113:6812/16056 -- osd_map(194486..194486 src has 185139..194486) v3 -- ?+0 0x54df080 con 0x4e3c8e0 -8923> 2014-08-13 17:38:04.414460 7fe99f5e9700 10 log is not dirty -8922> 2014-08-13 17:38:04.414662 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -8921> 2014-08-13 17:38:04.414672 7fe99ebe8700 10 osd.4 194486 note_peer_epoch osd.292 has 194486 -8920> 2014-08-13 17:38:04.414687 7fe99ebe8700 7 osd.4 194486 do_notify osd 292 on 1 PGs -8919> 2014-08-13 17:38:04.414690 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.113:6812/16056 -- pg_notify(22.3ad(24) epoch 194486) v5 -- ?+0 0x5103640 con 0x4e3c8e0 -8918> 2014-08-13 17:38:04.414723 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.7 192.168.1.115:6838/62713 6 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x54dc140 con 0x5250580 -8917> 2014-08-13 17:38:04.414767 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8916> 2014-08-13 17:38:04.414771 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8915> 2014-08-13 17:38:04.414773 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dc140 osd_map(194486..194486 src has 185139..194486) v3 -8914> 2014-08-13 17:38:04.414796 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -8913> 2014-08-13 17:38:04.414802 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -8912> 2014-08-13 17:38:04.414804 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8911> 2014-08-13 17:38:04.414806 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8910> 2014-08-13 17:38:04.414824 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing -8909> 2014-08-13 17:38:04.414828 7fe99f5e9700 7 osd.4 194486 do_notify osd 45 on 1 PGs -8908> 2014-08-13 17:38:04.414849 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(31) epoch 194486) v5 -- ?+0 0x5820000 con 0x4d39080 -8907> 2014-08-13 17:38:04.414908 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing -8906> 2014-08-13 17:38:04.414919 7fe99ebe8700 7 osd.4 194486 do_notify osd 323 on 1 PGs -8905> 2014-08-13 17:38:04.414923 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6816/8118 -- pg_notify(22.12a(24) epoch 194486) v5 -- ?+0 0x5105780 con 0x4d7d6a0 -8904> 2014-08-13 17:38:04.414921 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.7 192.168.1.115:6838/62713 7 ==== pg_notify(22.1b8(25) epoch 194486) v5 ==== 2117+0+0 (1636954735 0 0) 0x4265b00 con 0x5250580 -8903> 2014-08-13 17:38:04.414940 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8902> 2014-08-13 17:38:04.414944 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8901> 2014-08-13 17:38:04.414945 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265b00 pg_notify(22.1b8(25) epoch 194486) v5 -8900> 2014-08-13 17:38:04.414952 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.414741, event: header_read, request: pg_notify(22.1b8(25) epoch 194486) v5 -8899> 2014-08-13 17:38:04.414986 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.414745, event: throttled, request: pg_notify(22.1b8(25) epoch 194486) v5 -8898> 2014-08-13 17:38:04.414993 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.414865, event: all_read, request: pg_notify(22.1b8(25) epoch 194486) v5 -8897> 2014-08-13 17:38:04.414999 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.414939, event: dispatched, request: pg_notify(22.1b8(25) epoch 194486) v5 -8896> 2014-08-13 17:38:04.415044 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.415043, event: waiting_for_osdmap, request: pg_notify(22.1b8(25) epoch 194486) v5 -8895> 2014-08-13 17:38:04.415052 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.7 -8894> 2014-08-13 17:38:04.415056 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265b00 -8893> 2014-08-13 17:38:04.415061 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.415061, event: started, request: pg_notify(22.1b8(25) epoch 194486) v5 -8892> 2014-08-13 17:38:04.415131 7fe9a63f4700 5 -- op tracker -- , seq: 499, time: 2014-08-13 17:38:04.415130, event: done, request: pg_notify(22.1b8(25) epoch 194486) v5 -8891> 2014-08-13 17:38:04.415122 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194422'18845 ec=192688 les/c 194484/194484 194485/194486/192969) -8890> 2014-08-13 17:38:04.415181 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8889> 2014-08-13 17:38:04.415186 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8888> 2014-08-13 17:38:04.415192 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -8887> 2014-08-13 17:38:04.415202 7fe99ebe8700 7 osd.4 194486 do_notify osd 343 on 1 PGs -8886> 2014-08-13 17:38:04.415205 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(31) epoch 194486) v5 -- ?+0 0x5106040 con 0x4d31e40 -8885> 2014-08-13 17:38:04.415198 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.6 192.168.1.107:6809/53141 5 ==== pg_query(24.cd epoch 194486) v3 ==== 144+0+0 (668563047 0 0) 0x299eea0 con 0x4e3d800 -8884> 2014-08-13 17:38:04.415225 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8883> 2014-08-13 17:38:04.415228 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8882> 2014-08-13 17:38:04.415229 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 -8881> 2014-08-13 17:38:04.415232 7fe99ebe8700 7 osd.4 194486 do_notify osd 345 on 1 PGs -8880> 2014-08-13 17:38:04.415229 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299eea0 pg_query(24.cd epoch 194486) v3 -8879> 2014-08-13 17:38:04.415235 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.110:6820/46337 -- pg_notify(22.29a(24) epoch 194486) v5 -- ?+0 0x58271c0 con 0x4c70f20 -8878> 2014-08-13 17:38:04.415240 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.415077, event: header_read, request: pg_query(24.cd epoch 194486) v3 -8877> 2014-08-13 17:38:04.415252 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -8876> 2014-08-13 17:38:04.415255 7fe99ebe8700 7 osd.4 194486 do_notify osd 376 on 1 PGs -8875> 2014-08-13 17:38:04.415258 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6838/6893 -- pg_notify(22.226(29) epoch 194486) v5 -- ?+0 0x58239c0 con 0x4e4d280 -8874> 2014-08-13 17:38:04.415254 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.415078, event: throttled, request: pg_query(24.cd epoch 194486) v3 -8873> 2014-08-13 17:38:04.415271 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.415147, event: all_read, request: pg_query(24.cd epoch 194486) v3 -8872> 2014-08-13 17:38:04.415276 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 -8871> 2014-08-13 17:38:04.415279 7fe99ebe8700 7 osd.4 194486 do_notify osd 396 on 1 PGs -8870> 2014-08-13 17:38:04.415277 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.415224, event: dispatched, request: pg_query(24.cd epoch 194486) v3 -8869> 2014-08-13 17:38:04.415284 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.415284, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194486) v3 -8868> 2014-08-13 17:38:04.415282 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.105:6814/41806 -- pg_notify(24.299(8) epoch 194486) v5 -- ?+0 0x5820fc0 con 0x4ecb700 -8867> 2014-08-13 17:38:04.415291 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.6 epoch 194486 -8866> 2014-08-13 17:38:04.422272 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299eea0 -8865> 2014-08-13 17:38:04.422287 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.422287, event: started, request: pg_query(24.cd epoch 194486) v3 -8864> 2014-08-13 17:38:04.415300 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d791e0 don't know epoch, doing nothing -8863> 2014-08-13 17:38:04.422301 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194422'15992 ec=192688 les/c 194484/194480 194485/194486/192967) from replica 6 -8862> 2014-08-13 17:38:04.415157 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 -8861> 2014-08-13 17:38:04.418376 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.337 192.168.1.104:0/62197 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.418761) v2 ==== 47+0+0 (293382657 0 0) 0x4c16200 con 0x51a89a0 -8860> 2014-08-13 17:38:04.422325 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.296 on 1 PGs -8859> 2014-08-13 17:38:04.422336 7fe9a63f4700 5 -- op tracker -- , seq: 500, time: 2014-08-13 17:38:04.422336, event: done, request: pg_query(24.cd epoch 194486) v3 -8858> 2014-08-13 17:38:04.418418 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.337 192.168.1.104:0/62197 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.418761) v2 ==== 47+0+0 (293382657 0 0) 0x4c16040 con 0x3fa06e0 -8857> 2014-08-13 17:38:04.422350 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8856> 2014-08-13 17:38:04.422354 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8855> 2014-08-13 17:38:04.422340 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.109:6806/31286 -- pg_query(24.181 epoch 194486) v3 -- ?+0 0x299d820 con 0x4d791e0 -8854> 2014-08-13 17:38:04.422372 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5a432e0 don't know epoch, doing nothing -8853> 2014-08-13 17:38:04.422376 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.307 on 1 PGs -8852> 2014-08-13 17:38:04.422361 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.7 192.168.1.115:6838/62713 8 ==== pg_notify(22.84(2) epoch 194486) v5 ==== 828+0+0 (3439664117 0 0) 0x4265240 con 0x5250580 -8851> 2014-08-13 17:38:04.422351 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.418761) v2 -- ?+0 0x5827000 con 0x51a89a0 -8850> 2014-08-13 17:38:04.422380 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8849> 2014-08-13 17:38:04.422382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8848> 2014-08-13 17:38:04.422379 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.109:6804/31016 -- pg_query(24.181 epoch 194486) v3 -- ?+0 0x299ad00 con 0x5a432e0 -8847> 2014-08-13 17:38:04.422385 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265240 pg_notify(22.84(2) epoch 194486) v5 -8846> 2014-08-13 17:38:04.422361 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194422'18845 -8845> 2014-08-13 17:38:04.422390 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.337 now has 194486 -8844> 2014-08-13 17:38:04.422391 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.415618, event: header_read, request: pg_notify(22.84(2) epoch 194486) v5 -8843> 2014-08-13 17:38:04.422397 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ec8000 don't know epoch, doing nothing -8842> 2014-08-13 17:38:04.422400 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.320 on 1 PGs -8841> 2014-08-13 17:38:04.422400 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 -8840> 2014-08-13 17:38:04.422399 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.415622, event: throttled, request: pg_notify(22.84(2) epoch 194486) v5 -8839> 2014-08-13 17:38:04.422405 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.415694, event: all_read, request: pg_notify(22.84(2) epoch 194486) v5 -8838> 2014-08-13 17:38:04.422404 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6820/10050 -- pg_query(24.181 epoch 194486) v3 -- ?+0 0x2998960 con 0x4ec8000 -8837> 2014-08-13 17:38:04.422393 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) -8836> 2014-08-13 17:38:04.422412 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.422379, event: dispatched, request: pg_notify(22.84(2) epoch 194486) v5 -8835> 2014-08-13 17:38:04.422419 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.422419, event: waiting_for_osdmap, request: pg_notify(22.84(2) epoch 194486) v5 -8834> 2014-08-13 17:38:04.422426 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.7 -8833> 2014-08-13 17:38:04.422429 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265240 -8832> 2014-08-13 17:38:04.422433 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.422433, event: started, request: pg_notify(22.84(2) epoch 194486) v5 -8831> 2014-08-13 17:38:04.422440 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing -8830> 2014-08-13 17:38:04.422418 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.104:0/62197 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.418761) v2 -- ?+0 0x5140380 con 0x3fa06e0 -8829> 2014-08-13 17:38:04.422456 7fe9a63f4700 5 -- op tracker -- , seq: 501, time: 2014-08-13 17:38:04.422456, event: done, request: pg_notify(22.84(2) epoch 194486) v5 -8828> 2014-08-13 17:38:04.422446 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194486 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194486) v4 -- ?+0 0x5fb0000 con 0x4a486e0 -8827> 2014-08-13 17:38:04.422463 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 -8826> 2014-08-13 17:38:04.422471 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8825> 2014-08-13 17:38:04.422474 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8824> 2014-08-13 17:38:04.422477 7fe99f5e9700 10 log is not dirty -8823> 2014-08-13 17:38:04.422480 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.31 192.168.1.103:6831/41157 7 ==== pg_query(24.1e4 epoch 194486) v3 ==== 144+0+0 (2549247500 0 0) 0x299f260 con 0x49e5960 -8822> 2014-08-13 17:38:04.422493 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8821> 2014-08-13 17:38:04.422495 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8820> 2014-08-13 17:38:04.422497 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299f260 pg_query(24.1e4 epoch 194486) v3 -8819> 2014-08-13 17:38:04.422501 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.415714, event: header_read, request: pg_query(24.1e4 epoch 194486) v3 -8818> 2014-08-13 17:38:04.422508 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.415716, event: throttled, request: pg_query(24.1e4 epoch 194486) v3 -8817> 2014-08-13 17:38:04.422513 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.415843, event: all_read, request: pg_query(24.1e4 epoch 194486) v3 -8816> 2014-08-13 17:38:04.422518 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.422493, event: dispatched, request: pg_query(24.1e4 epoch 194486) v3 -8815> 2014-08-13 17:38:04.422524 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.422524, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194486) v3 -8814> 2014-08-13 17:38:04.422530 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.31 epoch 194486 -8813> 2014-08-13 17:38:04.422533 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299f260 -8812> 2014-08-13 17:38:04.422537 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.422537, event: started, request: pg_query(24.1e4 epoch 194486) v3 -8811> 2014-08-13 17:38:04.422596 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(log 194478'0 ec=192688 les/c 194484/194372 194485/194486/192978) from replica 31 -8810> 2014-08-13 17:38:04.422447 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing -8809> 2014-08-13 17:38:04.422636 7fe9a63f4700 5 -- op tracker -- , seq: 502, time: 2014-08-13 17:38:04.422636, event: done, request: pg_query(24.1e4 epoch 194486) v3 -8808> 2014-08-13 17:38:04.422649 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8807> 2014-08-13 17:38:04.422652 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8806> 2014-08-13 17:38:04.422642 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.331 on 1 PGs -8805> 2014-08-13 17:38:04.422638 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 301 info: 22.22c( v 194215'514 lc 0'0 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) -8804> 2014-08-13 17:38:04.422658 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.285 192.168.1.101:6822/26295 7 ==== pg_query(24.1e6 epoch 194486) v3 ==== 144+0+0 (1848521998 0 0) 0x5974b00 con 0x4a45280 -8803> 2014-08-13 17:38:04.422672 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8802> 2014-08-13 17:38:04.422657 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6816/837 -- pg_query(24.1bb epoch 194486) v3 -- ?+0 0x299bfc0 con 0x4e83180 -8801> 2014-08-13 17:38:04.422674 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8800> 2014-08-13 17:38:04.422677 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5974b00 pg_query(24.1e6 epoch 194486) v3 -8799> 2014-08-13 17:38:04.422666 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.301 activated and committed -8798> 2014-08-13 17:38:04.422682 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.415830, event: header_read, request: pg_query(24.1e6 epoch 194486) v3 -8797> 2014-08-13 17:38:04.422689 7fe99f5e9700 10 log is not dirty -8796> 2014-08-13 17:38:04.422688 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.415833, event: throttled, request: pg_query(24.1e6 epoch 194486) v3 -8795> 2014-08-13 17:38:04.422694 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing -8794> 2014-08-13 17:38:04.422694 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.415911, event: all_read, request: pg_query(24.1e6 epoch 194486) v3 -8793> 2014-08-13 17:38:04.422698 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.332 on 1 PGs -8792> 2014-08-13 17:38:04.422700 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.422672, event: dispatched, request: pg_query(24.1e6 epoch 194486) v3 -8791> 2014-08-13 17:38:04.422705 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.110:6806/42024 -- pg_query(24.1bb epoch 194486) v3 -- ?+0 0x5975fa0 con 0x4e3b020 -8790> 2014-08-13 17:38:04.422708 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.422707, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194486) v3 -8789> 2014-08-13 17:38:04.422698 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 291 notify: (query_epoch:194486, epoch_sent:194486, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -8788> 2014-08-13 17:38:04.422720 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.285 epoch 194486 -8787> 2014-08-13 17:38:04.422723 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5974b00 -8786> 2014-08-13 17:38:04.422723 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3c780 already has epoch 194486 -8785> 2014-08-13 17:38:04.422727 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.337 on 1 PGs -8784> 2014-08-13 17:38:04.422728 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.422728, event: started, request: pg_query(24.1e6 epoch 194486) v3 -8783> 2014-08-13 17:38:04.422730 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- pg_query(24.1bb epoch 194486) v3 -- ?+0 0x5975dc0 con 0x4d3c780 -8782> 2014-08-13 17:38:04.422724 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.291 -8781> 2014-08-13 17:38:04.422746 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing -8780> 2014-08-13 17:38:04.422750 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.344 on 1 PGs -8779> 2014-08-13 17:38:04.422736 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192975) from replica 285 -8778> 2014-08-13 17:38:04.422753 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6812/6757 -- pg_query(24.181 epoch 194486) v3 -- ?+0 0x5975be0 con 0x4a482c0 -8777> 2014-08-13 17:38:04.422769 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing -8776> 2014-08-13 17:38:04.422772 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.348 on 1 PGs -8775> 2014-08-13 17:38:04.422770 7fe9a63f4700 5 -- op tracker -- , seq: 503, time: 2014-08-13 17:38:04.422770, event: done, request: pg_query(24.1e6 epoch 194486) v3 -8774> 2014-08-13 17:38:04.422774 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.110:6800/41204 -- pg_query(24.1bb epoch 194486) v3 -- ?+0 0x5975a00 con 0x5b96720 -8773> 2014-08-13 17:38:04.422746 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8772> 2014-08-13 17:38:04.422786 7fe99f5e9700 10 log is not dirty -8771> 2014-08-13 17:38:04.422781 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8770> 2014-08-13 17:38:04.422798 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8769> 2014-08-13 17:38:04.422804 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.343 192.168.1.104:6802/61960 10 ==== pg_query(24.206 epoch 194486) v3 ==== 144+0+0 (367545406 0 0) 0x299dfa0 con 0x4d31e40 -8768> 2014-08-13 17:38:04.422819 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8767> 2014-08-13 17:38:04.422804 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8766> 2014-08-13 17:38:04.422821 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8765> 2014-08-13 17:38:04.422823 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299dfa0 pg_query(24.206 epoch 194486) v3 -8764> 2014-08-13 17:38:04.422833 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.416516, event: header_read, request: pg_query(24.206 epoch 194486) v3 -8763> 2014-08-13 17:38:04.422826 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops -8762> 2014-08-13 17:38:04.422843 7fe99f5e9700 10 log is not dirty -8761> 2014-08-13 17:38:04.422840 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.416518, event: throttled, request: pg_query(24.206 epoch 194486) v3 -8760> 2014-08-13 17:38:04.422846 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.416578, event: all_read, request: pg_query(24.206 epoch 194486) v3 -8759> 2014-08-13 17:38:04.422852 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.422818, event: dispatched, request: pg_query(24.206 epoch 194486) v3 -8758> 2014-08-13 17:38:04.422850 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 30 notify: (query_epoch:194486, epoch_sent:194486, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -8757> 2014-08-13 17:38:04.422859 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.422858, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194486) v3 -8756> 2014-08-13 17:38:04.422881 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.343 epoch 194486 -8755> 2014-08-13 17:38:04.422884 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299dfa0 -8754> 2014-08-13 17:38:04.422872 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.30 -8753> 2014-08-13 17:38:04.422889 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.422889, event: started, request: pg_query(24.206 epoch 194486) v3 -8752> 2014-08-13 17:38:04.422889 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.30 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486), identical to ours -8751> 2014-08-13 17:38:04.422897 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193262) from replica 343 -8750> 2014-08-13 17:38:04.422924 7fe99f5e9700 10 log is not dirty -8749> 2014-08-13 17:38:04.422925 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e82c00 don't know epoch, doing nothing -8748> 2014-08-13 17:38:04.422934 7fe99ebe8700 7 osd.4 194486 do_queries querying osd.379 on 1 PGs -8747> 2014-08-13 17:38:04.422937 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6830/4861 -- pg_query(24.1bb epoch 194486) v3 -- ?+0 0x5975820 con 0x4e82c00 -8746> 2014-08-13 17:38:04.422943 7fe9a63f4700 5 -- op tracker -- , seq: 504, time: 2014-08-13 17:38:04.422943, event: done, request: pg_query(24.206 epoch 194486) v3 -8745> 2014-08-13 17:38:04.422934 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8744> 2014-08-13 17:38:04.422961 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8743> 2014-08-13 17:38:04.422964 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8742> 2014-08-13 17:38:04.422961 7fe99ebe8700 20 osd.4 194486 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 50 -8741> 2014-08-13 17:38:04.422973 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing -8740> 2014-08-13 17:38:04.422957 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8739> 2014-08-13 17:38:04.422969 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.296 192.168.1.109:6806/31286 1 ==== osd_map(194479..194486 src has 185139..194486) v3 ==== 5897+0+0 (2398167145 0 0) 0x4db0900 con 0x4d791e0 -8738> 2014-08-13 17:38:04.422980 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.112:6842/9607 -- pg_info(1 pgs e194486:22.6bd(3)) v4 -- ?+0 0x5822f40 con 0x4d3dac0 -8737> 2014-08-13 17:38:04.422990 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8736> 2014-08-13 17:38:04.422993 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8735> 2014-08-13 17:38:04.422995 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db0900 osd_map(194479..194486 src has 185139..194486) v3 -8734> 2014-08-13 17:38:04.422997 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 286 -8733> 2014-08-13 17:38:04.423004 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194479,194486], i have 194486, src has [185139,194486] -8732> 2014-08-13 17:38:04.423006 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing -8731> 2014-08-13 17:38:04.422992 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged -8730> 2014-08-13 17:38:04.423012 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -8729> 2014-08-13 17:38:04.423015 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8728> 2014-08-13 17:38:04.423015 7fe99f5e9700 10 log is not dirty -8727> 2014-08-13 17:38:04.423017 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8726> 2014-08-13 17:38:04.423015 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.113:6820/19173 -- pg_info(1 pgs e194486:22.3d5(3)) v4 -- ?+0 0x58263c0 con 0x4df61a0 -8725> 2014-08-13 17:38:04.423026 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.15 192.168.1.107:6812/53408 8 ==== pg_query(24.20e epoch 194486) v3 ==== 144+0+0 (2019670061 0 0) 0x5974740 con 0x4d78dc0 -8724> 2014-08-13 17:38:04.423039 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8723> 2014-08-13 17:38:04.423034 7fe99ebe8700 20 osd.4 194486 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) to shard 287 -8722> 2014-08-13 17:38:04.423024 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 45 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) -8721> 2014-08-13 17:38:04.423041 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8720> 2014-08-13 17:38:04.423044 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df3700 don't know epoch, doing nothing -8719> 2014-08-13 17:38:04.423044 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5974740 pg_query(24.20e epoch 194486) v3 -8718> 2014-08-13 17:38:04.423045 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.45 activated and committed -8717> 2014-08-13 17:38:04.423057 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.415856, event: header_read, request: pg_query(24.20e epoch 194486) v3 -8716> 2014-08-13 17:38:04.423065 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.415859, event: throttled, request: pg_query(24.20e epoch 194486) v3 -8715> 2014-08-13 17:38:04.423064 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.109:6814/33435 -- pg_info(1 pgs e194486:22.481(47)) v4 -- ?+0 0x5823f00 con 0x4df3700 -8714> 2014-08-13 17:38:04.423062 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8713> 2014-08-13 17:38:04.423080 7fe99ebe8700 20 osd.4 194486 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 289 -8712> 2014-08-13 17:38:04.423071 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.415921, event: all_read, request: pg_query(24.20e epoch 194486) v3 -8711> 2014-08-13 17:38:04.423078 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8710> 2014-08-13 17:38:04.423087 7fe99ebe8700 20 osd.4 194486 Sending info 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) to shard 289 -8709> 2014-08-13 17:38:04.423091 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.423039, event: dispatched, request: pg_query(24.20e epoch 194486) v3 -8708> 2014-08-13 17:38:04.423096 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -8707> 2014-08-13 17:38:04.423099 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.423099, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194486) v3 -8706> 2014-08-13 17:38:04.423102 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing -8705> 2014-08-13 17:38:04.423106 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.15 epoch 194486 -8704> 2014-08-13 17:38:04.423109 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5974740 -8703> 2014-08-13 17:38:04.423106 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194486:22.113) v4 -- ?+0 0x5822680 -8702> 2014-08-13 17:38:04.423112 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.113:6808/15284 -- pg_info(2 pgs e194486:22.6bd(3),22.48e(30)) v4 -- ?+0 0x58224c0 con 0x4cfdac0 -8701> 2014-08-13 17:38:04.423119 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.423119, event: started, request: pg_query(24.20e epoch 194486) v3 -8700> 2014-08-13 17:38:04.423144 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4ec8160 don't know epoch, doing nothing -8699> 2014-08-13 17:38:04.423156 7fe99ebe8700 20 osd.4 194486 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) to shard 309 -8698> 2014-08-13 17:38:04.423192 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193264) from replica 15 -8697> 2014-08-13 17:38:04.423215 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.339 192.168.1.110:6818/45268 -- pg_info(1 pgs e194486:22.113) v4 -- ?+0 0x5822300 -8696> 2014-08-13 17:38:04.423239 7fe9a63f4700 5 -- op tracker -- , seq: 505, time: 2014-08-13 17:38:04.423238, event: done, request: pg_query(24.20e epoch 194486) v3 -8695> 2014-08-13 17:38:04.423252 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8694> 2014-08-13 17:38:04.423255 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8693> 2014-08-13 17:38:04.423270 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing -8692> 2014-08-13 17:38:04.423264 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2933 -8691> 2014-08-13 17:38:04.423302 7fe99f5e9700 10 log is not dirty -8690> 2014-08-13 17:38:04.423319 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6814/23703 -- pg_info(1 pgs e194486:22.481(47)) v4 -- ?+0 0x5826e40 con 0x4ecf0c0 -8689> 2014-08-13 17:38:04.423328 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 4 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4db1440 con 0x4e3b020 -8688> 2014-08-13 17:38:04.423347 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8687> 2014-08-13 17:38:04.423350 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8686> 2014-08-13 17:38:04.423352 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db1440 osd_map(194486..194486 src has 185139..194486) v3 -8685> 2014-08-13 17:38:04.423372 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -8684> 2014-08-13 17:38:04.423378 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -8683> 2014-08-13 17:38:04.423380 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8682> 2014-08-13 17:38:04.423382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8681> 2014-08-13 17:38:04.423384 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) to shard 338 -8680> 2014-08-13 17:38:04.423374 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 289 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192968) -8679> 2014-08-13 17:38:04.423403 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d789a0 don't know epoch, doing nothing -8678> 2014-08-13 17:38:04.423406 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log request from 289 -8677> 2014-08-13 17:38:04.423426 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6814/25893 -- pg_info(1 pgs e194486:22.3d9(48)) v4 -- ?+0 0x5824b40 con 0x4d789a0 -8676> 2014-08-13 17:38:04.423449 7fe99ebe8700 20 osd.4 194486 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 340 -8675> 2014-08-13 17:38:04.423458 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -8674> 2014-08-13 17:38:04.423465 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:22.1b8(3)) v4 -- ?+0 0x5105b00 con 0x49e40a0 -8673> 2014-08-13 17:38:04.423462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info+missing+log since 194477'0 -8672> 2014-08-13 17:38:04.423482 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 343 -8671> 2014-08-13 17:38:04.423491 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -8670> 2014-08-13 17:38:04.423480 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending log((194422'17034,194422'17034], crt=194422'17034) missing(63) -8669> 2014-08-13 17:38:04.423499 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -8668> 2014-08-13 17:38:04.423496 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- pg_info(1 pgs e194486:22.3d5(3)) v4 -- ?+0 0x5106200 con 0x4d31e40 -8667> 2014-08-13 17:38:04.423503 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.113:6808/15284 -- pg_log(24.5e epoch 194486 log log((194422'17034,194422'17034], crt=194422'17034) query_epoch 194486) v4 -- ?+0 0x5fb6000 con 0x4cfdac0 -8666> 2014-08-13 17:38:04.423624 7fe99ebe8700 20 osd.4 194486 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) to shard 347 -8665> 2014-08-13 17:38:04.423645 7fe99f5e9700 10 log is not dirty -8664> 2014-08-13 17:38:04.423397 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.296 192.168.1.109:6806/31286 2 ==== pg_notify(24.181(33) epoch 194486) v5 ==== 2599+0+0 (4035973577 0 0) 0x4c16ac0 con 0x4d791e0 -8663> 2014-08-13 17:38:04.423741 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8662> 2014-08-13 17:38:04.423747 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8661> 2014-08-13 17:38:04.423749 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16ac0 pg_notify(24.181(33) epoch 194486) v5 -8660> 2014-08-13 17:38:04.423756 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.419957, event: header_read, request: pg_notify(24.181(33) epoch 194486) v5 -8659> 2014-08-13 17:38:04.423777 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.419960, event: throttled, request: pg_notify(24.181(33) epoch 194486) v5 -8658> 2014-08-13 17:38:04.423785 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.420197, event: all_read, request: pg_notify(24.181(33) epoch 194486) v5 -8657> 2014-08-13 17:38:04.423791 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.423740, event: dispatched, request: pg_notify(24.181(33) epoch 194486) v5 -8656> 2014-08-13 17:38:04.423815 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing -8655> 2014-08-13 17:38:04.423800 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 313 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) -8654> 2014-08-13 17:38:04.423805 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.423805, event: waiting_for_osdmap, request: pg_notify(24.181(33) epoch 194486) v5 -8653> 2014-08-13 17:38:04.423828 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6800/4640 -- pg_info(1 pgs e194486:22.1b8(3)) v4 -- ?+0 0x5106580 con 0x4e389a0 -8652> 2014-08-13 17:38:04.423842 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.296 -8651> 2014-08-13 17:38:04.423833 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.313 activated and committed -8650> 2014-08-13 17:38:04.423847 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16ac0 -8649> 2014-08-13 17:38:04.423850 7fe99ebe8700 20 osd.4 194486 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) to shard 376 -8648> 2014-08-13 17:38:04.423860 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -8647> 2014-08-13 17:38:04.423856 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.423855, event: started, request: pg_notify(24.181(33) epoch 194486) v5 -8646> 2014-08-13 17:38:04.423852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8645> 2014-08-13 17:38:04.423868 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8644> 2014-08-13 17:38:04.423892 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfd6a0 don't know epoch, doing nothing -8643> 2014-08-13 17:38:04.423893 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6838/6893 -- pg_info(1 pgs e194486:22.3d9(48)) v4 -- ?+0 0x51039c0 con 0x4e4d280 -8642> 2014-08-13 17:38:04.423892 7fe9a63f4700 5 -- op tracker -- , seq: 506, time: 2014-08-13 17:38:04.423892, event: done, request: pg_notify(24.181(33) epoch 194486) v5 -8641> 2014-08-13 17:38:04.423897 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.313 192.168.1.109:6820/35445 -- pg_info(1 pgs e194486:22.3e4) v4 -- ?+0 0x58278c0 -8640> 2014-08-13 17:38:04.423947 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x57cfe80 don't know epoch, doing nothing -8639> 2014-08-13 17:38:04.423951 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.318 192.168.1.110:6814/43857 -- pg_info(1 pgs e194486:22.3e4) v4 -- ?+0 0x51cd940 -8638> 2014-08-13 17:38:04.424051 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 289 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) -8637> 2014-08-13 17:38:04.424074 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2705 -8636> 2014-08-13 17:38:04.424086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring -8635> 2014-08-13 17:38:04.424107 7fe99ebe8700 10 log is not dirty -8634> 2014-08-13 17:38:04.424109 7fe99f5e9700 10 log is not dirty -8633> 2014-08-13 17:38:04.424122 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 289 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) -8632> 2014-08-13 17:38:04.424129 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8631> 2014-08-13 17:38:04.424143 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring -8630> 2014-08-13 17:38:04.424161 7fe99ebe8700 10 log is not dirty -8629> 2014-08-13 17:38:04.424156 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8628> 2014-08-13 17:38:04.424168 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 50 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) -8627> 2014-08-13 17:38:04.424194 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 50, already have info from that osd, ignoring -8626> 2014-08-13 17:38:04.424212 7fe99ebe8700 10 log is not dirty -8625> 2014-08-13 17:38:04.424190 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged -8624> 2014-08-13 17:38:04.424223 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8623> 2014-08-13 17:38:04.424218 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 32 notify: (query_epoch:194486, epoch_sent:194486, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194483 n=35 ec=183096 les/c 194483/194480 194485/194485/194485)) -8622> 2014-08-13 17:38:04.424233 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8621> 2014-08-13 17:38:04.424227 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.068925 6 0.000727 -8620> 2014-08-13 17:38:04.424236 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 32, already have info from that osd, ignoring -8619> 2014-08-13 17:38:04.424253 7fe99ebe8700 10 log is not dirty -8618> 2014-08-13 17:38:04.424246 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 5 ==== pg_notify(24.1bb(2) epoch 194486) v5 ==== 870+0+0 (3450949925 0 0) 0x51c8700 con 0x4e3b020 -8617> 2014-08-13 17:38:04.424264 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8616> 2014-08-13 17:38:04.424267 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8615> 2014-08-13 17:38:04.424270 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51c8700 pg_notify(24.1bb(2) epoch 194486) v5 -8614> 2014-08-13 17:38:04.424259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 343 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) -8613> 2014-08-13 17:38:04.424258 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog -8612> 2014-08-13 17:38:04.424276 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.420138, event: header_read, request: pg_notify(24.1bb(2) epoch 194486) v5 -8611> 2014-08-13 17:38:04.424280 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 343, already have info from that osd, ignoring -8610> 2014-08-13 17:38:04.424297 7fe99ebe8700 10 log is not dirty -8609> 2014-08-13 17:38:04.424291 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.420139, event: throttled, request: pg_notify(24.1bb(2) epoch 194486) v5 -8608> 2014-08-13 17:38:04.424304 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.420223, event: all_read, request: pg_notify(24.1bb(2) epoch 194486) v5 -8607> 2014-08-13 17:38:04.424288 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -8606> 2014-08-13 17:38:04.424311 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.424264, event: dispatched, request: pg_notify(24.1bb(2) epoch 194486) v5 -8605> 2014-08-13 17:38:04.424304 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 338 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) -8604> 2014-08-13 17:38:04.424319 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.424319, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194486) v5 -8603> 2014-08-13 17:38:04.424326 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.332 -8602> 2014-08-13 17:38:04.424331 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51c8700 -8601> 2014-08-13 17:38:04.424313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -8600> 2014-08-13 17:38:04.424324 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 338, already have info from that osd, ignoring -8599> 2014-08-13 17:38:04.424343 7fe99ebe8700 10 log is not dirty -8598> 2014-08-13 17:38:04.424341 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.424341, event: started, request: pg_notify(24.1bb(2) epoch 194486) v5 -8597> 2014-08-13 17:38:04.424336 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8596> 2014-08-13 17:38:04.424350 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) -8595> 2014-08-13 17:38:04.424366 7fe9a63f4700 5 -- op tracker -- , seq: 507, time: 2014-08-13 17:38:04.424366, event: done, request: pg_notify(24.1bb(2) epoch 194486) v5 -8594> 2014-08-13 17:38:04.424356 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8593> 2014-08-13 17:38:04.424379 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8592> 2014-08-13 17:38:04.424382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8591> 2014-08-13 17:38:04.424371 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.323 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) -8590> 2014-08-13 17:38:04.424376 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -8589> 2014-08-13 17:38:04.424395 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.912355 7 0.000272 -8588> 2014-08-13 17:38:04.424397 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -8587> 2014-08-13 17:38:04.424417 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -8586> 2014-08-13 17:38:04.424423 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.331 192.168.1.104:6816/837 7 ==== pg_notify(24.1bb epoch 194486) v5 ==== 694+0+0 (3284661620 0 0) 0x5823480 con 0x4e83180 -8585> 2014-08-13 17:38:04.424444 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8584> 2014-08-13 17:38:04.424447 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8583> 2014-08-13 17:38:04.424449 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823480 pg_notify(24.1bb epoch 194486) v5 -8582> 2014-08-13 17:38:04.424438 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8581> 2014-08-13 17:38:04.424454 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.423319, event: header_read, request: pg_notify(24.1bb epoch 194486) v5 -8580> 2014-08-13 17:38:04.424463 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.423321, event: throttled, request: pg_notify(24.1bb epoch 194486) v5 -8579> 2014-08-13 17:38:04.424458 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194477 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -8578> 2014-08-13 17:38:04.424457 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -8577> 2014-08-13 17:38:04.424468 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.423391, event: all_read, request: pg_notify(24.1bb epoch 194486) v5 -8576> 2014-08-13 17:38:04.424482 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.424443, event: dispatched, request: pg_notify(24.1bb epoch 194486) v5 -8575> 2014-08-13 17:38:04.424477 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'491 -> 194215'491 -8574> 2014-08-13 17:38:04.424490 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.424490, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194486) v5 -8573> 2014-08-13 17:38:04.424496 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.331 -8572> 2014-08-13 17:38:04.424481 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] actingbackfill is 4,50,55,61,65 -8571> 2014-08-13 17:38:04.424499 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823480 -8570> 2014-08-13 17:38:04.424504 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.424504, event: started, request: pg_notify(24.1bb epoch 194486) v5 -8569> 2014-08-13 17:38:04.424502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] state: Activate Finished -8568> 2014-08-13 17:38:04.424502 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] choose_acting want [4,50,65] (== acting) backfill_targets 55,61 -8567> 2014-08-13 17:38:04.424528 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: leaving GetLog -8566> 2014-08-13 17:38:04.424546 7fe9a63f4700 5 -- op tracker -- , seq: 508, time: 2014-08-13 17:38:04.424546, event: done, request: pg_notify(24.1bb epoch 194486) v5 -8565> 2014-08-13 17:38:04.424553 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000295 0 0.000000 -8564> 2014-08-13 17:38:04.424569 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8563> 2014-08-13 17:38:04.424583 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8562> 2014-08-13 17:38:04.424591 7fe99ebe8700 10 log is not dirty -8561> 2014-08-13 17:38:04.424577 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing -8560> 2014-08-13 17:38:04.424596 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.50 has no missing, identical log -8559> 2014-08-13 17:38:04.424613 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.379 192.168.1.108:6830/4861 6 ==== pg_notify(24.1bb(35) epoch 194486) v5 ==== 2721+0+0 (1302713664 0 0) 0x51cc440 con 0x4e82c00 -8558> 2014-08-13 17:38:04.424616 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.65 has no missing, identical log -8557> 2014-08-13 17:38:04.424629 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8556> 2014-08-13 17:38:04.424643 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8555> 2014-08-13 17:38:04.424622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) -8554> 2014-08-13 17:38:04.424634 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: still need up_thru update before going active -8553> 2014-08-13 17:38:04.424645 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cc440 pg_notify(24.1bb(35) epoch 194486) v5 -8552> 2014-08-13 17:38:04.424655 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424045, event: header_read, request: pg_notify(24.1bb(35) epoch 194486) v5 -8551> 2014-08-13 17:38:04.424648 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.51 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) -8550> 2014-08-13 17:38:04.424664 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424047, event: throttled, request: pg_notify(24.1bb(35) epoch 194486) v5 -8549> 2014-08-13 17:38:04.424656 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000 -8548> 2014-08-13 17:38:04.424670 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424181, event: all_read, request: pg_notify(24.1bb(35) epoch 194486) v5 -8547> 2014-08-13 17:38:04.424680 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424629, event: dispatched, request: pg_notify(24.1bb(35) epoch 194486) v5 -8546> 2014-08-13 17:38:04.424671 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.943393 7 0.000290 -8545> 2014-08-13 17:38:04.424688 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424688, event: waiting_for_osdmap, request: pg_notify(24.1bb(35) epoch 194486) v5 -8544> 2014-08-13 17:38:04.424694 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.379 -8543> 2014-08-13 17:38:04.424680 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru -8542> 2014-08-13 17:38:04.424698 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cc440 -8541> 2014-08-13 17:38:04.424701 7fe99f5e9700 10 log is not dirty -8540> 2014-08-13 17:38:04.424702 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424702, event: started, request: pg_notify(24.1bb(35) epoch 194486) v5 -8539> 2014-08-13 17:38:04.424696 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -8538> 2014-08-13 17:38:04.424726 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8537> 2014-08-13 17:38:04.424740 7fe9a63f4700 5 -- op tracker -- , seq: 509, time: 2014-08-13 17:38:04.424740, event: done, request: pg_notify(24.1bb(35) epoch 194486) v5 -8536> 2014-08-13 17:38:04.424729 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 50 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) -8535> 2014-08-13 17:38:04.424744 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194477 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -8534> 2014-08-13 17:38:04.424755 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.50 -8533> 2014-08-13 17:38:04.424762 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'501 -> 194215'501 -8532> 2014-08-13 17:38:04.424785 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] state: Activate Finished -8531> 2014-08-13 17:38:04.424775 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.50 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8530> 2014-08-13 17:38:04.424811 7fe99f5e9700 10 log is not dirty -8529> 2014-08-13 17:38:04.424822 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8528> 2014-08-13 17:38:04.424828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8527> 2014-08-13 17:38:04.424820 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 17 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193188) -8526> 2014-08-13 17:38:04.424839 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log request from 17 -8525> 2014-08-13 17:38:04.424881 7fe99ebe8700 10 log is not dirty -8524> 2014-08-13 17:38:04.424879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info+missing+log since 194477'0 -8523> 2014-08-13 17:38:04.424888 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 13 ==== pg_notify(24.181(12) epoch 194486) v5 ==== 1432+0+0 (1456742432 0 0) 0x5100000 con 0x4ec8000 -8522> 2014-08-13 17:38:04.424906 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8521> 2014-08-13 17:38:04.424909 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8520> 2014-08-13 17:38:04.424896 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending log((194422'14353,194422'14353], crt=194422'14353) missing(49) -8519> 2014-08-13 17:38:04.424911 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5100000 pg_notify(24.181(12) epoch 194486) v5 -8518> 2014-08-13 17:38:04.424917 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing -8517> 2014-08-13 17:38:04.424910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) -8516> 2014-08-13 17:38:04.424920 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6811/37658 -- pg_log(24.2a1 epoch 194486 log log((194422'14353,194422'14353], crt=194422'14353) query_epoch 194486) v4 -- ?+0 0x5fb5a00 con 0x4d3f900 -8515> 2014-08-13 17:38:04.424917 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.423162, event: header_read, request: pg_notify(24.181(12) epoch 194486) v5 -8514> 2014-08-13 17:38:04.424932 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: got info from osd.292 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) -8513> 2014-08-13 17:38:04.424942 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.423163, event: throttled, request: pg_notify(24.181(12) epoch 194486) v5 -8512> 2014-08-13 17:38:04.424954 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.423264, event: all_read, request: pg_notify(24.181(12) epoch 194486) v5 -8511> 2014-08-13 17:38:04.424954 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.903675 7 0.000259 -8510> 2014-08-13 17:38:04.424973 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -8509> 2014-08-13 17:38:04.424991 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8508> 2014-08-13 17:38:04.425007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194477 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -8507> 2014-08-13 17:38:04.425022 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 -8506> 2014-08-13 17:38:04.425044 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] state: Activate Finished -8505> 2014-08-13 17:38:04.425119 7fe99ebe8700 10 log is not dirty -8504> 2014-08-13 17:38:04.425098 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] _activate_committed 194486 telling primary -8503> 2014-08-13 17:38:04.425126 7fe99f5e9700 10 log is not dirty -8502> 2014-08-13 17:38:04.425159 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4d7d6a0 don't know epoch, doing nothing -8501> 2014-08-13 17:38:04.425145 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 347 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) -8500> 2014-08-13 17:38:04.425145 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 362 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8499> 2014-08-13 17:38:04.425165 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.323 192.168.1.106:6816/8118 -- pg_info(1 pgs e194486:22.12a) v4 -- ?+0 0x5820700 -8498> 2014-08-13 17:38:04.425175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8497> 2014-08-13 17:38:04.425170 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 347, already have info from that osd, ignoring -8496> 2014-08-13 17:38:04.425211 7fe99ebe8700 10 log is not dirty -8495> 2014-08-13 17:38:04.425206 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] flushed -8494> 2014-08-13 17:38:04.425216 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged -8493> 2014-08-13 17:38:04.425239 7fe99f5e9700 10 log is not dirty -8492> 2014-08-13 17:38:04.424961 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.424905, event: dispatched, request: pg_notify(24.181(12) epoch 194486) v5 -8491> 2014-08-13 17:38:04.425220 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(info 0'0) -8490> 2014-08-13 17:38:04.425244 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] flushed -8489> 2014-08-13 17:38:04.425246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 346 query_epoch 194486 query: query(log 194422'15014 ec=192688 les/c 194484/194484 194485/194486/192955) -8488> 2014-08-13 17:38:04.425265 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info -8487> 2014-08-13 17:38:04.425269 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 -8486> 2014-08-13 17:38:04.425285 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.425285, event: waiting_for_osdmap, request: pg_notify(24.181(12) epoch 194486) v5 -8485> 2014-08-13 17:38:04.425301 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.320 -8484> 2014-08-13 17:38:04.425305 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5100000 -8483> 2014-08-13 17:38:04.425310 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.425309, event: started, request: pg_notify(24.181(12) epoch 194486) v5 -8482> 2014-08-13 17:38:04.425316 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194422'15014 -8481> 2014-08-13 17:38:04.425336 7fe99ebe8700 10 log is not dirty -8480> 2014-08-13 17:38:04.425333 7fe9a63f4700 5 -- op tracker -- , seq: 510, time: 2014-08-13 17:38:04.425333, event: done, request: pg_notify(24.181(12) epoch 194486) v5 -8479> 2014-08-13 17:38:04.425334 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) -8478> 2014-08-13 17:38:04.425356 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e4a100 already has epoch 194486 -8477> 2014-08-13 17:38:04.425362 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8476> 2014-08-13 17:38:04.425345 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194480 n=28 ec=183096 les/c 194480/194389 194485/194485/194485)) -8475> 2014-08-13 17:38:04.425365 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8474> 2014-08-13 17:38:04.425360 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194486 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194486) v4 -- ?+0 0x5fb5400 con 0x4e4a100 -8473> 2014-08-13 17:38:04.425369 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring -8472> 2014-08-13 17:38:04.425385 7fe99ebe8700 10 log is not dirty -8471> 2014-08-13 17:38:04.425387 7fe99f5e9700 10 log is not dirty -8470> 2014-08-13 17:38:04.425374 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.348 192.168.1.110:6800/41204 3 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x5482d00 con 0x5b96720 -8469> 2014-08-13 17:38:04.425391 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8468> 2014-08-13 17:38:04.425394 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8467> 2014-08-13 17:38:04.425397 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5482d00 osd_map(194486..194486 src has 185139..194486) v3 -8466> 2014-08-13 17:38:04.425392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 397 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) -8465> 2014-08-13 17:38:04.425410 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 397, already have info from that osd, ignoring -8464> 2014-08-13 17:38:04.425422 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -8463> 2014-08-13 17:38:04.425429 7fe99ebe8700 10 log is not dirty -8462> 2014-08-13 17:38:04.425429 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -8461> 2014-08-13 17:38:04.425432 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8460> 2014-08-13 17:38:04.425434 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8459> 2014-08-13 17:38:04.425459 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.356 192.168.1.108:6832/5235 13 ==== pg_query(24.3b7 epoch 194486) v3 ==== 144+0+0 (3945455533 0 0) 0x2999860 con 0x4a486e0 -8458> 2014-08-13 17:38:04.425473 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8457> 2014-08-13 17:38:04.425476 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8456> 2014-08-13 17:38:04.425478 7fe9a63f4700 20 osd.4 194486 _dispatch 0x2999860 pg_query(24.3b7 epoch 194486) v3 -8455> 2014-08-13 17:38:04.425483 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.424199, event: header_read, request: pg_query(24.3b7 epoch 194486) v3 -8454> 2014-08-13 17:38:04.425490 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.424212, event: throttled, request: pg_query(24.3b7 epoch 194486) v3 -8453> 2014-08-13 17:38:04.425468 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] _activate_committed 194486 telling primary -8452> 2014-08-13 17:38:04.425496 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.424284, event: all_read, request: pg_query(24.3b7 epoch 194486) v3 -8451> 2014-08-13 17:38:04.425502 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.425472, event: dispatched, request: pg_query(24.3b7 epoch 194486) v3 -8450> 2014-08-13 17:38:04.425508 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.425508, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194486) v3 -8449> 2014-08-13 17:38:04.425536 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.356 epoch 194486 -8448> 2014-08-13 17:38:04.425540 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x2999860 -8447> 2014-08-13 17:38:04.425545 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.425544, event: started, request: pg_query(24.3b7 epoch 194486) v3 -8446> 2014-08-13 17:38:04.425553 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/192969) from replica 356 -8445> 2014-08-13 17:38:04.425565 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 324 info: 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) -8444> 2014-08-13 17:38:04.425589 7fe9a63f4700 5 -- op tracker -- , seq: 511, time: 2014-08-13 17:38:04.425588, event: done, request: pg_query(24.3b7 epoch 194486) v3 -8443> 2014-08-13 17:38:04.425576 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 379 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -8442> 2014-08-13 17:38:04.425606 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8441> 2014-08-13 17:38:04.425593 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.324 activated and committed -8440> 2014-08-13 17:38:04.425610 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8439> 2014-08-13 17:38:04.425611 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8438> 2014-08-13 17:38:04.425607 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -8437> 2014-08-13 17:38:04.425627 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8436> 2014-08-13 17:38:04.425651 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a4e1a0 don't know epoch, doing nothing -8435> 2014-08-13 17:38:04.425639 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -8434> 2014-08-13 17:38:04.425616 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.344 192.168.1.106:6812/6757 6 ==== pg_notify(24.181(33) epoch 194486) v5 ==== 2599+0+0 (3144239402 0 0) 0x5104600 con 0x4a482c0 -8433> 2014-08-13 17:38:04.425663 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8432> 2014-08-13 17:38:04.425655 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.301 192.168.1.113:6814/16727 -- pg_info(1 pgs e194486:22.22c) v4 -- ?+0 0x51ce200 -8431> 2014-08-13 17:38:04.425667 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8430> 2014-08-13 17:38:04.425669 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5104600 pg_notify(24.181(33) epoch 194486) v5 -8429> 2014-08-13 17:38:04.425675 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.423631, event: header_read, request: pg_notify(24.181(33) epoch 194486) v5 -8428> 2014-08-13 17:38:04.425664 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348 -> 4,331,332,337,348,379 -8427> 2014-08-13 17:38:04.425682 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.423633, event: throttled, request: pg_notify(24.181(33) epoch 194486) v5 -8426> 2014-08-13 17:38:04.425685 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update -8425> 2014-08-13 17:38:04.425693 7fe99ebe8700 10 log is not dirty -8424> 2014-08-13 17:38:04.425688 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.423755, event: all_read, request: pg_notify(24.181(33) epoch 194486) v5 -8423> 2014-08-13 17:38:04.425689 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 don't know epoch, doing nothing -8422> 2014-08-13 17:38:04.425698 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.425662, event: dispatched, request: pg_notify(24.181(33) epoch 194486) v5 -8421> 2014-08-13 17:38:04.425705 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.425705, event: waiting_for_osdmap, request: pg_notify(24.181(33) epoch 194486) v5 -8420> 2014-08-13 17:38:04.425711 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.344 -8419> 2014-08-13 17:38:04.425714 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5104600 -8418> 2014-08-13 17:38:04.425700 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.51 192.168.1.112:6832/6655 -- pg_info(1 pgs e194486:22.537) v4 -- ?+0 0x5821880 -8417> 2014-08-13 17:38:04.425718 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.425718, event: started, request: pg_notify(24.181(33) epoch 194486) v5 -8416> 2014-08-13 17:38:04.425736 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing -8415> 2014-08-13 17:38:04.425742 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.324 192.168.1.104:6814/65209 -- pg_info(1 pgs e194486:22.22c) v4 -- ?+0 0x51cdcc0 -8414> 2014-08-13 17:38:04.425741 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] flushed -8413> 2014-08-13 17:38:04.425767 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2830 -8412> 2014-08-13 17:38:04.425777 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] _activate_committed 194486 telling primary -8411> 2014-08-13 17:38:04.425795 7fe99f5e9700 10 log is not dirty -8410> 2014-08-13 17:38:04.425794 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 327 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) -8409> 2014-08-13 17:38:04.425833 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 327, already have info from that osd, ignoring -8408> 2014-08-13 17:38:04.425853 7fe99ebe8700 10 log is not dirty -8407> 2014-08-13 17:38:04.425901 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e3c8e0 already has epoch 194486 -8406> 2014-08-13 17:38:04.425898 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8405> 2014-08-13 17:38:04.425908 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.292 192.168.1.113:6812/16056 -- pg_info(1 pgs e194486:22.3ad) v4 -- ?+0 0x5107700 -8404> 2014-08-13 17:38:04.425924 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] requeue_ops -8403> 2014-08-13 17:38:04.425942 7fe99f5e9700 10 log is not dirty -8402> 2014-08-13 17:38:04.425937 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] flushed -8401> 2014-08-13 17:38:04.425915 7fe9a63f4700 5 -- op tracker -- , seq: 512, time: 2014-08-13 17:38:04.425914, event: done, request: pg_notify(24.181(33) epoch 194486) v5 -8400> 2014-08-13 17:38:04.425950 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) -8399> 2014-08-13 17:38:04.425967 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8398> 2014-08-13 17:38:04.425976 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8397> 2014-08-13 17:38:04.425972 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -8396> 2014-08-13 17:38:04.425987 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.289 192.168.1.113:6808/15284 12 ==== pg_info(1 pgs e194486:22.48e) v4 ==== 714+0+0 (2563903464 0 0) 0x4265b00 con 0x4cfdac0 -8395> 2014-08-13 17:38:04.426009 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8394> 2014-08-13 17:38:04.426012 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8393> 2014-08-13 17:38:04.426014 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4265b00 pg_info(1 pgs e194486:22.48e) v4 -8392> 2014-08-13 17:38:04.426001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -8391> 2014-08-13 17:38:04.426025 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 -8390> 2014-08-13 17:38:04.426044 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update -8389> 2014-08-13 17:38:04.426019 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.424619, event: header_read, request: pg_info(1 pgs e194486:22.48e) v4 -8388> 2014-08-13 17:38:04.426052 7fe99f5e9700 10 log is not dirty -8387> 2014-08-13 17:38:04.426052 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.424621, event: throttled, request: pg_info(1 pgs e194486:22.48e) v4 -8386> 2014-08-13 17:38:04.426060 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.424689, event: all_read, request: pg_info(1 pgs e194486:22.48e) v4 -8385> 2014-08-13 17:38:04.426066 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.426009, event: dispatched, request: pg_info(1 pgs e194486:22.48e) v4 -8384> 2014-08-13 17:38:04.426060 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 327 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) -8383> 2014-08-13 17:38:04.426073 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.426073, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.48e) v4 -8382> 2014-08-13 17:38:04.426086 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.48e) v4 from osd.289 -8381> 2014-08-13 17:38:04.426070 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 339 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -8380> 2014-08-13 17:38:04.426099 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4265b00 -8379> 2014-08-13 17:38:04.426083 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8378> 2014-08-13 17:38:04.426107 7fe99f5e9700 10 log is not dirty -8377> 2014-08-13 17:38:04.426105 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.426104, event: started, request: pg_info(1 pgs e194486:22.48e) v4 -8376> 2014-08-13 17:38:04.426106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 339, already have info from that osd, ignoring -8375> 2014-08-13 17:38:04.426128 7fe99ebe8700 10 log is not dirty -8374> 2014-08-13 17:38:04.426114 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 65 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) -8373> 2014-08-13 17:38:04.426130 7fe9a63f4700 5 -- op tracker -- , seq: 513, time: 2014-08-13 17:38:04.426130, event: done, request: pg_info(1 pgs e194486:22.48e) v4 -8372> 2014-08-13 17:38:04.426142 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8371> 2014-08-13 17:38:04.426137 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 293 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) -8370> 2014-08-13 17:38:04.426137 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.65 -8369> 2014-08-13 17:38:04.426165 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8368> 2014-08-13 17:38:04.426157 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 293, already have info from that osd, ignoring -8367> 2014-08-13 17:38:04.426186 7fe99ebe8700 10 log is not dirty -8366> 2014-08-13 17:38:04.426173 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.50 192.168.1.112:6842/9607 9 ==== pg_info(1 pgs e194486:22.6bd) v4 ==== 714+0+0 (2328150435 0 0) 0x5822f40 con 0x4d3dac0 -8365> 2014-08-13 17:38:04.426189 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8364> 2014-08-13 17:38:04.426192 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8363> 2014-08-13 17:38:04.426193 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5822f40 pg_info(1 pgs e194486:22.6bd) v4 -8362> 2014-08-13 17:38:04.426157 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.65 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8361> 2014-08-13 17:38:04.426199 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.425015, event: header_read, request: pg_info(1 pgs e194486:22.6bd) v4 -8360> 2014-08-13 17:38:04.426206 7fe99f5e9700 10 log is not dirty -8359> 2014-08-13 17:38:04.426193 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 381 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -8358> 2014-08-13 17:38:04.426207 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.425017, event: throttled, request: pg_info(1 pgs e194486:22.6bd) v4 -8357> 2014-08-13 17:38:04.426213 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.425101, event: all_read, request: pg_info(1 pgs e194486:22.6bd) v4 -8356> 2014-08-13 17:38:04.426219 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.426189, event: dispatched, request: pg_info(1 pgs e194486:22.6bd) v4 -8355> 2014-08-13 17:38:04.426214 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 381, already have info from that osd, ignoring -8354> 2014-08-13 17:38:04.426232 7fe99ebe8700 10 log is not dirty -8353> 2014-08-13 17:38:04.426218 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) -8352> 2014-08-13 17:38:04.426239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 396 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) -8351> 2014-08-13 17:38:04.426240 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.373 activated and committed -8350> 2014-08-13 17:38:04.426251 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.426250, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.6bd) v4 -8349> 2014-08-13 17:38:04.426260 7fe99f5e9700 10 log is not dirty -8348> 2014-08-13 17:38:04.426261 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.6bd) v4 from osd.50 -8347> 2014-08-13 17:38:04.426266 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5822f40 -8346> 2014-08-13 17:38:04.426267 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -8345> 2014-08-13 17:38:04.426258 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 396, already have info from that osd, ignoring -8344> 2014-08-13 17:38:04.426275 7fe99ebe8700 10 log is not dirty -8343> 2014-08-13 17:38:04.426271 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.426271, event: started, request: pg_info(1 pgs e194486:22.6bd) v4 -8342> 2014-08-13 17:38:04.426281 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 309 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) -8341> 2014-08-13 17:38:04.426299 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 309, already have info from that osd, ignoring -8340> 2014-08-13 17:38:04.426314 7fe99ebe8700 10 log is not dirty -8339> 2014-08-13 17:38:04.426320 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -8338> 2014-08-13 17:38:04.426326 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 -8337> 2014-08-13 17:38:04.426313 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 31 query_epoch 194486 query: query(info 0'0) -8336> 2014-08-13 17:38:04.426330 7fe99ebe8700 7 osd.4 194486 do_notify osd 6 on 1 PGs -8335> 2014-08-13 17:38:04.426325 7fe9a63f4700 5 -- op tracker -- , seq: 514, time: 2014-08-13 17:38:04.426325, event: done, request: pg_info(1 pgs e194486:22.6bd) v4 -8334> 2014-08-13 17:38:04.426344 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8333> 2014-08-13 17:38:04.426334 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(31) epoch 194486) v5 -- ?+0 0x51024c0 con 0x4e3d800 -8332> 2014-08-13 17:38:04.426335 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info -8331> 2014-08-13 17:38:04.426350 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8330> 2014-08-13 17:38:04.426358 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 13 ==== pg_notify(24.1bb(35) epoch 194486) v5 ==== 2721+0+0 (1302713664 0 0) 0x5820380 con 0x4d3c780 -8329> 2014-08-13 17:38:04.426373 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8328> 2014-08-13 17:38:04.426375 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8327> 2014-08-13 17:38:04.426377 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820380 pg_notify(24.1bb(35) epoch 194486) v5 -8326> 2014-08-13 17:38:04.426387 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.423553, event: header_read, request: pg_notify(24.1bb(35) epoch 194486) v5 -8325> 2014-08-13 17:38:04.426394 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.423555, event: throttled, request: pg_notify(24.1bb(35) epoch 194486) v5 -8324> 2014-08-13 17:38:04.426421 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.423674, event: all_read, request: pg_notify(24.1bb(35) epoch 194486) v5 -8323> 2014-08-13 17:38:04.426412 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 369 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -8322> 2014-08-13 17:38:04.426429 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.426372, event: dispatched, request: pg_notify(24.1bb(35) epoch 194486) v5 -8321> 2014-08-13 17:38:04.426441 7fe99f5e9700 10 log is not dirty -8320> 2014-08-13 17:38:04.426441 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.426441, event: waiting_for_osdmap, request: pg_notify(24.1bb(35) epoch 194486) v5 -8319> 2014-08-13 17:38:04.426449 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.337 -8318> 2014-08-13 17:38:04.426435 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -8317> 2014-08-13 17:38:04.426452 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820380 -8316> 2014-08-13 17:38:04.426453 7fe99ebe8700 10 log is not dirty -8315> 2014-08-13 17:38:04.426451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 285 query_epoch 194486 query: query(info 0'0) -8314> 2014-08-13 17:38:04.426461 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 306 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -8313> 2014-08-13 17:38:04.426471 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.426471, event: started, request: pg_notify(24.1bb(35) epoch 194486) v5 -8312> 2014-08-13 17:38:04.426473 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info -8311> 2014-08-13 17:38:04.426480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 306, already have info from that osd, ignoring -8310> 2014-08-13 17:38:04.426497 7fe99ebe8700 10 log is not dirty -8309> 2014-08-13 17:38:04.426503 7fe9a63f4700 5 -- op tracker -- , seq: 515, time: 2014-08-13 17:38:04.426503, event: done, request: pg_notify(24.1bb(35) epoch 194486) v5 -8308> 2014-08-13 17:38:04.426502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 382 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) -8307> 2014-08-13 17:38:04.426528 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8306> 2014-08-13 17:38:04.426531 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8305> 2014-08-13 17:38:04.426520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 382, already have info from that osd, ignoring -8304> 2014-08-13 17:38:04.426536 7fe99ebe8700 10 log is not dirty -8303> 2014-08-13 17:38:04.426545 7fe99f5e9700 10 log is not dirty -8302> 2014-08-13 17:38:04.426542 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 340 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) -8301> 2014-08-13 17:38:04.426554 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) -8300> 2014-08-13 17:38:04.426560 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 340, already have info from that osd, ignoring -8299> 2014-08-13 17:38:04.426576 7fe99ebe8700 10 log is not dirty -8298> 2014-08-13 17:38:04.426574 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) -8297> 2014-08-13 17:38:04.426549 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.343 192.168.1.104:6802/61960 11 ==== pg_info(1 pgs e194486:22.3d5) v4 ==== 714+0+0 (1371172176 0 0) 0x5106200 con 0x4d31e40 -8296> 2014-08-13 17:38:04.426597 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8295> 2014-08-13 17:38:04.426601 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8294> 2014-08-13 17:38:04.426590 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 61 notify: (query_epoch:194486, epoch_sent:194486, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -8293> 2014-08-13 17:38:04.426604 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106200 pg_info(1 pgs e194486:22.3d5) v4 -8292> 2014-08-13 17:38:04.426597 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.942741 7 0.000242 -8291> 2014-08-13 17:38:04.426610 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.425063, event: header_read, request: pg_info(1 pgs e194486:22.3d5) v4 -8290> 2014-08-13 17:38:04.426618 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.425065, event: throttled, request: pg_info(1 pgs e194486:22.3d5) v4 -8289> 2014-08-13 17:38:04.426609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring -8288> 2014-08-13 17:38:04.426628 7fe99ebe8700 10 log is not dirty -8287> 2014-08-13 17:38:04.426619 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -8286> 2014-08-13 17:38:04.426625 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.425191, event: all_read, request: pg_info(1 pgs e194486:22.3d5) v4 -8285> 2014-08-13 17:38:04.426633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) -8284> 2014-08-13 17:38:04.426646 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.426596, event: dispatched, request: pg_info(1 pgs e194486:22.3d5) v4 -8283> 2014-08-13 17:38:04.426641 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8282> 2014-08-13 17:38:04.426654 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.426653, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3d5) v4 -8281> 2014-08-13 17:38:04.426661 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d5) v4 from osd.343 -8280> 2014-08-13 17:38:04.426666 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106200 -8279> 2014-08-13 17:38:04.426652 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring -8278> 2014-08-13 17:38:04.426671 7fe99ebe8700 10 log is not dirty -8277> 2014-08-13 17:38:04.426661 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194477 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -8276> 2014-08-13 17:38:04.426680 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'488 -> 194215'488 -8275> 2014-08-13 17:38:04.426685 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.426685, event: started, request: pg_info(1 pgs e194486:22.3d5) v4 -8274> 2014-08-13 17:38:04.426677 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 373 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) -8273> 2014-08-13 17:38:04.426702 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] state: Activate Finished -8272> 2014-08-13 17:38:04.426704 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 373, already have info from that osd, ignoring -8271> 2014-08-13 17:38:04.426721 7fe99ebe8700 10 log is not dirty -8270> 2014-08-13 17:38:04.426727 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 365 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485)) -8269> 2014-08-13 17:38:04.426746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 365, already have info from that osd, ignoring -8268> 2014-08-13 17:38:04.426761 7fe99ebe8700 10 log is not dirty -8267> 2014-08-13 17:38:04.426730 7fe9a63f4700 5 -- op tracker -- , seq: 516, time: 2014-08-13 17:38:04.426730, event: done, request: pg_info(1 pgs e194486:22.3d5) v4 -8266> 2014-08-13 17:38:04.426774 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8265> 2014-08-13 17:38:04.426778 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8264> 2014-08-13 17:38:04.426767 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) -8263> 2014-08-13 17:38:04.426790 7fe99f5e9700 10 log is not dirty -8262> 2014-08-13 17:38:04.426786 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 10, already have info from that osd, ignoring -8261> 2014-08-13 17:38:04.426786 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.296 192.168.1.109:6806/31286 3 ==== pg_notify(24.181(33) epoch 194486) v5 ==== 2599+0+0 (4035973577 0 0) 0x5107380 con 0x4d791e0 -8260> 2014-08-13 17:38:04.426802 7fe99ebe8700 10 log is not dirty -8259> 2014-08-13 17:38:04.426804 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8258> 2014-08-13 17:38:04.426807 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8257> 2014-08-13 17:38:04.426809 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107380 pg_notify(24.181(33) epoch 194486) v5 -8256> 2014-08-13 17:38:04.426814 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.423443, event: header_read, request: pg_notify(24.181(33) epoch 194486) v5 -8255> 2014-08-13 17:38:04.426809 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8254> 2014-08-13 17:38:04.426822 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.423446, event: throttled, request: pg_notify(24.181(33) epoch 194486) v5 -8253> 2014-08-13 17:38:04.426829 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.423675, event: all_read, request: pg_notify(24.181(33) epoch 194486) v5 -8252> 2014-08-13 17:38:04.426835 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.426804, event: dispatched, request: pg_notify(24.181(33) epoch 194486) v5 -8251> 2014-08-13 17:38:04.426820 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 343 query_epoch 194486 query: query(info 0'0) -8250> 2014-08-13 17:38:04.426832 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8249> 2014-08-13 17:38:04.426842 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.426842, event: waiting_for_osdmap, request: pg_notify(24.181(33) epoch 194486) v5 -8248> 2014-08-13 17:38:04.426849 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.296 -8247> 2014-08-13 17:38:04.426850 7fe99ebe8700 10 log is not dirty -8246> 2014-08-13 17:38:04.426852 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107380 -8245> 2014-08-13 17:38:04.426845 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info -8244> 2014-08-13 17:38:04.426858 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 337 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -8243> 2014-08-13 17:38:04.426857 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.426857, event: started, request: pg_notify(24.181(33) epoch 194486) v5 -8242> 2014-08-13 17:38:04.426881 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -8241> 2014-08-13 17:38:04.426919 7fe99f5e9700 10 log is not dirty -8240> 2014-08-13 17:38:04.426914 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -8239> 2014-08-13 17:38:04.426927 7fe9a63f4700 5 -- op tracker -- , seq: 517, time: 2014-08-13 17:38:04.426927, event: done, request: pg_notify(24.181(33) epoch 194486) v5 -8238> 2014-08-13 17:38:04.426936 7fe99ebe8700 10 log is not dirty -8237> 2014-08-13 17:38:04.426945 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8236> 2014-08-13 17:38:04.426929 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 15 query_epoch 194486 query: query(info 0'0) -8235> 2014-08-13 17:38:04.426949 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8234> 2014-08-13 17:38:04.426944 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 286 notify: (query_epoch:194486, epoch_sent:194486, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194483 n=25 ec=183096 les/c 194483/194480 194485/194485/194485)) -8233> 2014-08-13 17:38:04.426951 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info -8232> 2014-08-13 17:38:04.426964 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 286, already have info from that osd, ignoring -8231> 2014-08-13 17:38:04.426981 7fe99ebe8700 10 log is not dirty -8230> 2014-08-13 17:38:04.426969 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.338 192.168.1.106:6814/25893 6 ==== pg_info(1 pgs e194486:22.3d9) v4 ==== 714+0+0 (700742062 0 0) 0x5824b40 con 0x4d789a0 -8229> 2014-08-13 17:38:04.426985 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8228> 2014-08-13 17:38:04.426988 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8227> 2014-08-13 17:38:04.426990 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5824b40 pg_info(1 pgs e194486:22.3d9) v4 -8226> 2014-08-13 17:38:04.426995 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.425567, event: header_read, request: pg_info(1 pgs e194486:22.3d9) v4 -8225> 2014-08-13 17:38:04.426987 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -8224> 2014-08-13 17:38:04.427003 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.425570, event: throttled, request: pg_info(1 pgs e194486:22.3d9) v4 -8223> 2014-08-13 17:38:04.427008 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.425640, event: all_read, request: pg_info(1 pgs e194486:22.3d9) v4 -8222> 2014-08-13 17:38:04.427005 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring -8221> 2014-08-13 17:38:04.427021 7fe99ebe8700 10 log is not dirty -8220> 2014-08-13 17:38:04.427029 7fe99f5e9700 10 log is not dirty -8219> 2014-08-13 17:38:04.427014 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.426984, event: dispatched, request: pg_info(1 pgs e194486:22.3d9) v4 -8218> 2014-08-13 17:38:04.427027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 10 notify: (query_epoch:194486, epoch_sent:194486, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -8217> 2014-08-13 17:38:04.427045 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.427044, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3d9) v4 -8216> 2014-08-13 17:38:04.427059 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d9) v4 from osd.338 -8215> 2014-08-13 17:38:04.427044 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194483/194463 194485/194485/193205) -8214> 2014-08-13 17:38:04.427048 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle_pg_notify from osd.10 -8213> 2014-08-13 17:38:04.427064 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5824b40 -8212> 2014-08-13 17:38:04.427074 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.427074, event: started, request: pg_info(1 pgs e194486:22.3d9) v4 -8211> 2014-08-13 17:38:04.427068 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.396 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194483/194463 194485/194485/193205) -8210> 2014-08-13 17:38:04.427071 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.10 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8209> 2014-08-13 17:38:04.427097 7fe9a63f4700 5 -- op tracker -- , seq: 518, time: 2014-08-13 17:38:04.427097, event: done, request: pg_info(1 pgs e194486:22.3d9) v4 -8208> 2014-08-13 17:38:04.427109 7fe99ebe8700 10 log is not dirty -8207> 2014-08-13 17:38:04.427097 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.953287 7 0.000214 -8206> 2014-08-13 17:38:04.427114 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8205> 2014-08-13 17:38:04.427118 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8204> 2014-08-13 17:38:04.427119 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive -8203> 2014-08-13 17:38:04.427118 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8202> 2014-08-13 17:38:04.427142 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] requeue_ops -8201> 2014-08-13 17:38:04.427159 7fe99ebe8700 10 log is not dirty -8200> 2014-08-13 17:38:04.427144 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8199> 2014-08-13 17:38:04.427139 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.323 192.168.1.106:6816/8118 5 ==== pg_info(1 pgs e194486:22.12a) v4 ==== 714+0+0 (4200565579 0 0) 0x5820700 con 0x4d7d6a0 -8198> 2014-08-13 17:38:04.427168 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8197> 2014-08-13 17:38:04.427164 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate -8196> 2014-08-13 17:38:04.427171 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8195> 2014-08-13 17:38:04.427166 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 304 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) -8194> 2014-08-13 17:38:04.427201 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820700 pg_info(1 pgs e194486:22.12a) v4 -8193> 2014-08-13 17:38:04.427182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'15411 -> 194422'15411 -8192> 2014-08-13 17:38:04.427197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.304 activated and committed -8191> 2014-08-13 17:38:04.427208 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.425971, event: header_read, request: pg_info(1 pgs e194486:22.12a) v4 -8190> 2014-08-13 17:38:04.427217 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.425974, event: throttled, request: pg_info(1 pgs e194486:22.12a) v4 -8189> 2014-08-13 17:38:04.427215 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8188> 2014-08-13 17:38:04.427229 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.426037, event: all_read, request: pg_info(1 pgs e194486:22.12a) v4 -8187> 2014-08-13 17:38:04.427238 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.427167, event: dispatched, request: pg_info(1 pgs e194486:22.12a) v4 -8186> 2014-08-13 17:38:04.427231 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8185> 2014-08-13 17:38:04.427216 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] _activate_committed 194486 telling primary -8184> 2014-08-13 17:38:04.427245 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.427245, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.12a) v4 -8183> 2014-08-13 17:38:04.427253 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing -8182> 2014-08-13 17:38:04.427257 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194486:22.84) v4 -- ?+0 0x5104b40 -8181> 2014-08-13 17:38:04.427266 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.12a) v4 from osd.323 -8180> 2014-08-13 17:38:04.427272 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820700 -8179> 2014-08-13 17:38:04.427269 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] state: Activate Finished -8178> 2014-08-13 17:38:04.427277 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.427277, event: started, request: pg_info(1 pgs e194486:22.12a) v4 -8177> 2014-08-13 17:38:04.427291 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing -8176> 2014-08-13 17:38:04.427296 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194486:22.84) v4 -- ?+0 0x5104d00 -8175> 2014-08-13 17:38:04.427304 7fe9a63f4700 5 -- op tracker -- , seq: 519, time: 2014-08-13 17:38:04.427304, event: done, request: pg_info(1 pgs e194486:22.12a) v4 -8174> 2014-08-13 17:38:04.427316 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8173> 2014-08-13 17:38:04.427319 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8172> 2014-08-13 17:38:04.427325 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.286 192.168.1.113:6820/19173 8 ==== pg_info(1 pgs e194486:22.3d5) v4 ==== 714+0+0 (320590084 0 0) 0x58263c0 con 0x4df61a0 -8171> 2014-08-13 17:38:04.427339 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8170> 2014-08-13 17:38:04.427341 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8169> 2014-08-13 17:38:04.427354 7fe99f5e9700 10 log is not dirty -8168> 2014-08-13 17:38:04.427342 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2902 -8167> 2014-08-13 17:38:04.427343 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58263c0 pg_info(1 pgs e194486:22.3d5) v4 -8166> 2014-08-13 17:38:04.427374 7fe99ebe8700 10 log is not dirty -8165> 2014-08-13 17:38:04.427369 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.424820, event: header_read, request: pg_info(1 pgs e194486:22.3d5) v4 -8164> 2014-08-13 17:38:04.427377 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.424822, event: throttled, request: pg_info(1 pgs e194486:22.3d5) v4 -8163> 2014-08-13 17:38:04.427382 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.424911, event: all_read, request: pg_info(1 pgs e194486:22.3d5) v4 -8162> 2014-08-13 17:38:04.427388 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.427339, event: dispatched, request: pg_info(1 pgs e194486:22.3d5) v4 -8161> 2014-08-13 17:38:04.427419 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.427418, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3d5) v4 -8160> 2014-08-13 17:38:04.427403 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) -8159> 2014-08-13 17:38:04.427426 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d5) v4 from osd.286 -8158> 2014-08-13 17:38:04.427431 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58263c0 -8157> 2014-08-13 17:38:04.427436 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.427435, event: started, request: pg_info(1 pgs e194486:22.3d5) v4 -8156> 2014-08-13 17:38:04.427430 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) -8155> 2014-08-13 17:38:04.427463 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.55 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) -8154> 2014-08-13 17:38:04.427487 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 -> 4,55,296,320,344 -8153> 2014-08-13 17:38:04.427506 7fe99f5e9700 20 osd.4 194486 need_heartbeat_peer_update -8152> 2014-08-13 17:38:04.427511 7fe99f5e9700 10 log is not dirty -8151> 2014-08-13 17:38:04.427526 7fe9a63f4700 5 -- op tracker -- , seq: 520, time: 2014-08-13 17:38:04.427526, event: done, request: pg_info(1 pgs e194486:22.3d5) v4 -8150> 2014-08-13 17:38:04.427540 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8149> 2014-08-13 17:38:04.427543 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8148> 2014-08-13 17:38:04.427549 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.51 192.168.1.112:6832/6655 5 ==== pg_info(1 pgs e194486:22.537) v4 ==== 714+0+0 (114231816 0 0) 0x5821880 con 0x4e39e40 -8147> 2014-08-13 17:38:04.427563 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8146> 2014-08-13 17:38:04.427576 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8145> 2014-08-13 17:38:04.427578 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821880 pg_info(1 pgs e194486:22.537) v4 -8144> 2014-08-13 17:38:04.427584 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.426458, event: header_read, request: pg_info(1 pgs e194486:22.537) v4 -8143> 2014-08-13 17:38:04.427590 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.426460, event: throttled, request: pg_info(1 pgs e194486:22.537) v4 -8142> 2014-08-13 17:38:04.427596 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.426530, event: all_read, request: pg_info(1 pgs e194486:22.537) v4 -8141> 2014-08-13 17:38:04.427601 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.427563, event: dispatched, request: pg_info(1 pgs e194486:22.537) v4 -8140> 2014-08-13 17:38:04.427612 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.427612, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.537) v4 -8139> 2014-08-13 17:38:04.427618 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.537) v4 from osd.51 -8138> 2014-08-13 17:38:04.427622 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821880 -8137> 2014-08-13 17:38:04.427627 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.427626, event: started, request: pg_info(1 pgs e194486:22.537) v4 -8136> 2014-08-13 17:38:04.427646 7fe9a63f4700 5 -- op tracker -- , seq: 521, time: 2014-08-13 17:38:04.427645, event: done, request: pg_info(1 pgs e194486:22.537) v4 -8135> 2014-08-13 17:38:04.427673 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8134> 2014-08-13 17:38:04.427649 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 344 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) -8133> 2014-08-13 17:38:04.427676 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8132> 2014-08-13 17:38:04.427682 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.292 192.168.1.113:6812/16056 5 ==== pg_info(1 pgs e194486:22.3ad) v4 ==== 714+0+0 (3036535460 0 0) 0x5107700 con 0x4e3c8e0 -8131> 2014-08-13 17:38:04.427696 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8130> 2014-08-13 17:38:04.427681 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.344 activated and committed -8129> 2014-08-13 17:38:04.427701 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8128> 2014-08-13 17:38:04.427703 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5107700 pg_info(1 pgs e194486:22.3ad) v4 -8127> 2014-08-13 17:38:04.427709 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.426620, event: header_read, request: pg_info(1 pgs e194486:22.3ad) v4 -8126> 2014-08-13 17:38:04.427703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8125> 2014-08-13 17:38:04.427715 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.426623, event: throttled, request: pg_info(1 pgs e194486:22.3ad) v4 -8124> 2014-08-13 17:38:04.427721 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.426688, event: all_read, request: pg_info(1 pgs e194486:22.3ad) v4 -8123> 2014-08-13 17:38:04.427719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8122> 2014-08-13 17:38:04.427733 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.427695, event: dispatched, request: pg_info(1 pgs e194486:22.3ad) v4 -8121> 2014-08-13 17:38:04.427740 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.427740, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3ad) v4 -8120> 2014-08-13 17:38:04.427744 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a482c0 don't know epoch, doing nothing -8119> 2014-08-13 17:38:04.427746 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3ad) v4 from osd.292 -8118> 2014-08-13 17:38:04.427748 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194486:22.39d) v4 -- ?+0 0x5107e00 -8117> 2014-08-13 17:38:04.427762 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5107700 -8116> 2014-08-13 17:38:04.427767 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.427767, event: started, request: pg_info(1 pgs e194486:22.3ad) v4 -8115> 2014-08-13 17:38:04.427775 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e85c20 don't know epoch, doing nothing -8114> 2014-08-13 17:38:04.427779 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194486:22.39d) v4 -- ?+0 0x5102f40 -8113> 2014-08-13 17:38:04.427792 7fe9a63f4700 5 -- op tracker -- , seq: 522, time: 2014-08-13 17:38:04.427791, event: done, request: pg_info(1 pgs e194486:22.3ad) v4 -8112> 2014-08-13 17:38:04.427810 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8111> 2014-08-13 17:38:04.427798 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2613 -8110> 2014-08-13 17:38:04.427813 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8109> 2014-08-13 17:38:04.427824 7fe99ebe8700 10 log is not dirty -8108> 2014-08-13 17:38:04.427833 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.347 192.168.1.106:6800/4640 6 ==== pg_info(1 pgs e194486:22.1b8) v4 ==== 714+0+0 (2904060855 0 0) 0x5106580 con 0x4e389a0 -8107> 2014-08-13 17:38:04.427847 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8106> 2014-08-13 17:38:04.427850 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8105> 2014-08-13 17:38:04.427851 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106580 pg_info(1 pgs e194486:22.1b8) v4 -8104> 2014-08-13 17:38:04.427856 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.425800, event: header_read, request: pg_info(1 pgs e194486:22.1b8) v4 -8103> 2014-08-13 17:38:04.427863 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.425801, event: throttled, request: pg_info(1 pgs e194486:22.1b8) v4 -8102> 2014-08-13 17:38:04.427873 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.425867, event: all_read, request: pg_info(1 pgs e194486:22.1b8) v4 -8101> 2014-08-13 17:38:04.427879 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.427847, event: dispatched, request: pg_info(1 pgs e194486:22.1b8) v4 -8100> 2014-08-13 17:38:04.427885 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.427885, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1b8) v4 -8099> 2014-08-13 17:38:04.427891 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1b8) v4 from osd.347 -8098> 2014-08-13 17:38:04.427895 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106580 -8097> 2014-08-13 17:38:04.427900 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.427899, event: started, request: pg_info(1 pgs e194486:22.1b8) v4 -8096> 2014-08-13 17:38:04.427888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8095> 2014-08-13 17:38:04.427914 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops -8094> 2014-08-13 17:38:04.427919 7fe9a63f4700 5 -- op tracker -- , seq: 523, time: 2014-08-13 17:38:04.427919, event: done, request: pg_info(1 pgs e194486:22.1b8) v4 -8093> 2014-08-13 17:38:04.427932 7fe99ebe8700 10 log is not dirty -8092> 2014-08-13 17:38:04.427936 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8091> 2014-08-13 17:38:04.427943 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8090> 2014-08-13 17:38:04.427939 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 45 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193186) -8089> 2014-08-13 17:38:04.427958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log request from 45 -8088> 2014-08-13 17:38:04.427949 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 6 ==== pg_notify(24.1bb(2) epoch 194486) v5 ==== 870+0+0 (3450949925 0 0) 0x51cbf00 con 0x4e3b020 -8087> 2014-08-13 17:38:04.427982 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8086> 2014-08-13 17:38:04.427985 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8085> 2014-08-13 17:38:04.427986 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cbf00 pg_notify(24.1bb(2) epoch 194486) v5 -8084> 2014-08-13 17:38:04.427991 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.423977, event: header_read, request: pg_notify(24.1bb(2) epoch 194486) v5 -8083> 2014-08-13 17:38:04.427998 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.423978, event: throttled, request: pg_notify(24.1bb(2) epoch 194486) v5 -8082> 2014-08-13 17:38:04.428004 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.424059, event: all_read, request: pg_notify(24.1bb(2) epoch 194486) v5 -8081> 2014-08-13 17:38:04.428003 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194477'0 -8080> 2014-08-13 17:38:04.428014 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.427981, event: dispatched, request: pg_notify(24.1bb(2) epoch 194486) v5 -8079> 2014-08-13 17:38:04.428024 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.428024, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194486) v5 -8078> 2014-08-13 17:38:04.428030 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.332 -8077> 2014-08-13 17:38:04.428033 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cbf00 -8076> 2014-08-13 17:38:04.428021 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending log((194422'13576,194422'13576], crt=194422'13576) missing(50) -8075> 2014-08-13 17:38:04.428041 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing -8074> 2014-08-13 17:38:04.428038 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.428038, event: started, request: pg_notify(24.1bb(2) epoch 194486) v5 -8073> 2014-08-13 17:38:04.428044 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.112:6828/6040 -- pg_log(24.369 epoch 194486 log log((194422'13576,194422'13576], crt=194422'13576) query_epoch 194486) v4 -- ?+0 0x5fb2400 con 0x4d39080 -8072> 2014-08-13 17:38:04.428059 7fe9a63f4700 5 -- op tracker -- , seq: 524, time: 2014-08-13 17:38:04.428059, event: done, request: pg_notify(24.1bb(2) epoch 194486) v5 -8071> 2014-08-13 17:38:04.428070 7fe99ebe8700 10 log is not dirty -8070> 2014-08-13 17:38:04.428087 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8069> 2014-08-13 17:38:04.428090 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8068> 2014-08-13 17:38:04.428097 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.346 192.168.1.104:6820/2308 6 ==== pg_query(24.e2 epoch 194486) v3 ==== 144+0+0 (952500435 0 0) 0x5971860 con 0x4e4a100 -8067> 2014-08-13 17:38:04.428110 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8066> 2014-08-13 17:38:04.428112 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8065> 2014-08-13 17:38:04.428114 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5971860 pg_query(24.e2 epoch 194486) v3 -8064> 2014-08-13 17:38:04.428118 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.426723, event: header_read, request: pg_query(24.e2 epoch 194486) v3 -8063> 2014-08-13 17:38:04.428130 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.426725, event: throttled, request: pg_query(24.e2 epoch 194486) v3 -8062> 2014-08-13 17:38:04.428136 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.426796, event: all_read, request: pg_query(24.e2 epoch 194486) v3 -8061> 2014-08-13 17:38:04.428209 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.428110, event: dispatched, request: pg_query(24.e2 epoch 194486) v3 -8060> 2014-08-13 17:38:04.428218 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.428218, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194486) v3 -8059> 2014-08-13 17:38:04.428225 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.346 epoch 194486 -8058> 2014-08-13 17:38:04.428228 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5971860 -8057> 2014-08-13 17:38:04.428232 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.428232, event: started, request: pg_query(24.e2 epoch 194486) v3 -8056> 2014-08-13 17:38:04.428235 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 55 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) -8055> 2014-08-13 17:38:04.428250 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4c70f20 already has epoch 194486 -8054> 2014-08-13 17:38:04.428240 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192955) from replica 346 -8053> 2014-08-13 17:38:04.428261 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194486:22.29a) v4 -- ?+0 0x5107540 -8052> 2014-08-13 17:38:04.428280 7fe9a63f4700 5 -- op tracker -- , seq: 525, time: 2014-08-13 17:38:04.428279, event: done, request: pg_query(24.e2 epoch 194486) v3 -8051> 2014-08-13 17:38:04.428262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -8050> 2014-08-13 17:38:04.428262 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) -8049> 2014-08-13 17:38:04.428293 7fe99ebe8700 10 log is not dirty -8048> 2014-08-13 17:38:04.428292 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8047> 2014-08-13 17:38:04.428296 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8046> 2014-08-13 17:38:04.428301 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -8045> 2014-08-13 17:38:04.428294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.376 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) -8044> 2014-08-13 17:38:04.428294 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] flushed -8043> 2014-08-13 17:38:04.428304 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 16 ==== pg_info(1 pgs e194486:22.3d9) v4 ==== 714+0+0 (3486186188 0 0) 0x51039c0 con 0x4e4d280 -8042> 2014-08-13 17:38:04.428326 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8041> 2014-08-13 17:38:04.428328 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8040> 2014-08-13 17:38:04.428330 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51039c0 pg_info(1 pgs e194486:22.3d9) v4 -8039> 2014-08-13 17:38:04.428324 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.937697 7 0.000266 -8038> 2014-08-13 17:38:04.428336 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.425808, event: header_read, request: pg_info(1 pgs e194486:22.3d9) v4 -8037> 2014-08-13 17:38:04.428343 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.425810, event: throttled, request: pg_info(1 pgs e194486:22.3d9) v4 -8036> 2014-08-13 17:38:04.428349 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.425901, event: all_read, request: pg_info(1 pgs e194486:22.3d9) v4 -8035> 2014-08-13 17:38:04.428355 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.428325, event: dispatched, request: pg_info(1 pgs e194486:22.3d9) v4 -8034> 2014-08-13 17:38:04.428345 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -8033> 2014-08-13 17:38:04.428363 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.428362, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3d9) v4 -8032> 2014-08-13 17:38:04.428369 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3d9) v4 from osd.376 -8031> 2014-08-13 17:38:04.428353 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -8030> 2014-08-13 17:38:04.428375 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51039c0 -8029> 2014-08-13 17:38:04.428367 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -8028> 2014-08-13 17:38:04.428380 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.428380, event: started, request: pg_info(1 pgs e194486:22.3d9) v4 -8027> 2014-08-13 17:38:04.428364 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 crt=0'0 lcod 0'0 remapped] _activate_committed 194486 telling primary -8026> 2014-08-13 17:38:04.428380 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8025> 2014-08-13 17:38:04.428401 7fe99ebe8700 10 log is not dirty -8024> 2014-08-13 17:38:04.428401 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4ecb700 already has epoch 194486 -8023> 2014-08-13 17:38:04.428389 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194478 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -8022> 2014-08-13 17:38:04.428405 7fe9a63f4700 5 -- op tracker -- , seq: 526, time: 2014-08-13 17:38:04.428404, event: done, request: pg_info(1 pgs e194486:22.3d9) v4 -8021> 2014-08-13 17:38:04.428406 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.396 192.168.1.105:6814/41806 -- pg_info(1 pgs e194486:24.299) v4 -- ?+0 0x51078c0 -8020> 2014-08-13 17:38:04.428416 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8019> 2014-08-13 17:38:04.428419 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8018> 2014-08-13 17:38:04.428409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'485 -> 194215'485 -8017> 2014-08-13 17:38:04.428425 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.309 192.168.1.101:6814/23703 7 ==== pg_info(1 pgs e194486:22.481) v4 ==== 714+0+0 (2522979252 0 0) 0x5826e40 con 0x4ecf0c0 -8016> 2014-08-13 17:38:04.428438 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8015> 2014-08-13 17:38:04.428440 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -8014> 2014-08-13 17:38:04.428442 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826e40 pg_info(1 pgs e194486:22.481) v4 -8013> 2014-08-13 17:38:04.428432 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] state: Activate Finished -8012> 2014-08-13 17:38:04.428447 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.427237, event: header_read, request: pg_info(1 pgs e194486:22.481) v4 -8011> 2014-08-13 17:38:04.428434 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed -8010> 2014-08-13 17:38:04.428454 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.427239, event: throttled, request: pg_info(1 pgs e194486:22.481) v4 -8009> 2014-08-13 17:38:04.428461 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.427305, event: all_read, request: pg_info(1 pgs e194486:22.481) v4 -8008> 2014-08-13 17:38:04.428466 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.428437, event: dispatched, request: pg_info(1 pgs e194486:22.481) v4 -8007> 2014-08-13 17:38:04.428473 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.428472, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.481) v4 -8006> 2014-08-13 17:38:04.428478 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.481) v4 from osd.309 -8005> 2014-08-13 17:38:04.428482 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826e40 -8004> 2014-08-13 17:38:04.428486 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.428486, event: started, request: pg_info(1 pgs e194486:22.481) v4 -8003> 2014-08-13 17:38:04.428524 7fe99f5e9700 10 log is not dirty -8002> 2014-08-13 17:38:04.428530 7fe9a63f4700 5 -- op tracker -- , seq: 527, time: 2014-08-13 17:38:04.428530, event: done, request: pg_info(1 pgs e194486:22.481) v4 -8001> 2014-08-13 17:38:04.428545 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -8000> 2014-08-13 17:38:04.428548 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7999> 2014-08-13 17:38:04.428554 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.348 192.168.1.110:6800/41204 4 ==== pg_notify(24.1bb epoch 194486) v5 ==== 694+0+0 (3284661620 0 0) 0x51cdb00 con 0x5b96720 -7998> 2014-08-13 17:38:04.428568 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7997> 2014-08-13 17:38:04.428543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192966) -7996> 2014-08-13 17:38:04.428551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7995> 2014-08-13 17:38:04.428570 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7994> 2014-08-13 17:38:04.428584 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cdb00 pg_notify(24.1bb epoch 194486) v5 -7993> 2014-08-13 17:38:04.428582 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops -7992> 2014-08-13 17:38:04.428605 7fe99f5e9700 10 log is not dirty -7991> 2014-08-13 17:38:04.428620 7fe99ebe8700 10 log is not dirty -7990> 2014-08-13 17:38:04.428612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7989> 2014-08-13 17:38:04.428631 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops -7988> 2014-08-13 17:38:04.428647 7fe99f5e9700 10 log is not dirty -7987> 2014-08-13 17:38:04.428590 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.424007, event: header_read, request: pg_notify(24.1bb epoch 194486) v5 -7986> 2014-08-13 17:38:04.428653 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.424024, event: throttled, request: pg_notify(24.1bb epoch 194486) v5 -7985> 2014-08-13 17:38:04.428661 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.424177, event: all_read, request: pg_notify(24.1bb epoch 194486) v5 -7984> 2014-08-13 17:38:04.428667 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.428567, event: dispatched, request: pg_notify(24.1bb epoch 194486) v5 -7983> 2014-08-13 17:38:04.428654 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 344 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) -7982> 2014-08-13 17:38:04.428650 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 362 notify: (query_epoch:194486, epoch_sent:194486, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -7981> 2014-08-13 17:38:04.428674 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.428674, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194486) v5 -7980> 2014-08-13 17:38:04.428686 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.348 -7979> 2014-08-13 17:38:04.428689 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cdb00 -7978> 2014-08-13 17:38:04.428678 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 362, already have info from that osd, ignoring -7977> 2014-08-13 17:38:04.428698 7fe99ebe8700 10 log is not dirty -7976> 2014-08-13 17:38:04.428676 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) -7975> 2014-08-13 17:38:04.428695 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.428695, event: started, request: pg_notify(24.1bb epoch 194486) v5 -7974> 2014-08-13 17:38:04.428716 7fe9a63f4700 5 -- op tracker -- , seq: 528, time: 2014-08-13 17:38:04.428716, event: done, request: pg_notify(24.1bb epoch 194486) v5 -7973> 2014-08-13 17:38:04.428706 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192946) -7972> 2014-08-13 17:38:04.428711 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged -7971> 2014-08-13 17:38:04.428733 7fe99f5e9700 10 log is not dirty -7970> 2014-08-13 17:38:04.428751 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7969> 2014-08-13 17:38:04.428754 7fe99ebe8700 10 log is not dirty -7968> 2014-08-13 17:38:04.428756 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7967> 2014-08-13 17:38:04.428740 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7966> 2014-08-13 17:38:04.428764 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops -7965> 2014-08-13 17:38:04.428780 7fe99f5e9700 10 log is not dirty -7964> 2014-08-13 17:38:04.428781 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 376 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485)) -7963> 2014-08-13 17:38:04.428787 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -7962> 2014-08-13 17:38:04.428765 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.340 192.168.1.104:6806/62463 10 ==== pg_info(1 pgs e194486:22.1b8) v4 ==== 714+0+0 (453562881 0 0) 0x5105b00 con 0x49e40a0 -7961> 2014-08-13 17:38:04.428805 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7960> 2014-08-13 17:38:04.428824 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7959> 2014-08-13 17:38:04.428828 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7958> 2014-08-13 17:38:04.428808 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -7957> 2014-08-13 17:38:04.428829 7fe99ebe8700 10 log is not dirty -7956> 2014-08-13 17:38:04.428830 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105b00 pg_info(1 pgs e194486:22.1b8) v4 -7955> 2014-08-13 17:38:04.428836 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.426061, event: header_read, request: pg_info(1 pgs e194486:22.1b8) v4 -7954> 2014-08-13 17:38:04.428844 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.426062, event: throttled, request: pg_info(1 pgs e194486:22.1b8) v4 -7953> 2014-08-13 17:38:04.428849 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.426140, event: all_read, request: pg_info(1 pgs e194486:22.1b8) v4 -7952> 2014-08-13 17:38:04.428837 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged -7951> 2014-08-13 17:38:04.428858 7fe99f5e9700 10 log is not dirty -7950> 2014-08-13 17:38:04.428856 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.428823, event: dispatched, request: pg_info(1 pgs e194486:22.1b8) v4 -7949> 2014-08-13 17:38:04.428863 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192976) -7948> 2014-08-13 17:38:04.428865 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 330 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7947> 2014-08-13 17:38:04.428877 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.428877, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1b8) v4 -7946> 2014-08-13 17:38:04.428887 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7945> 2014-08-13 17:38:04.428926 7fe99ebe8700 10 log is not dirty -7944> 2014-08-13 17:38:04.428914 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged -7943> 2014-08-13 17:38:04.428934 7fe99f5e9700 10 log is not dirty -7942> 2014-08-13 17:38:04.428941 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7941> 2014-08-13 17:38:04.428951 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 59 notify: (query_epoch:194486, epoch_sent:194486, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7940> 2014-08-13 17:38:04.428963 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops -7939> 2014-08-13 17:38:04.428982 7fe99f5e9700 10 log is not dirty -7938> 2014-08-13 17:38:04.428976 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7937> 2014-08-13 17:38:04.428988 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 315 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) -7936> 2014-08-13 17:38:04.429008 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged -7935> 2014-08-13 17:38:04.429008 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7934> 2014-08-13 17:38:04.429032 7fe99f5e9700 10 log is not dirty -7933> 2014-08-13 17:38:04.429034 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.075034 8 0.000711 -7932> 2014-08-13 17:38:04.429038 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 25 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7931> 2014-08-13 17:38:04.429059 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog -7930> 2014-08-13 17:38:04.429058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7929> 2014-08-13 17:38:04.429085 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged -7928> 2014-08-13 17:38:04.429086 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -7927> 2014-08-13 17:38:04.429078 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] _activate_committed 194486 telling primary -7926> 2014-08-13 17:38:04.429123 7fe9aad09700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -7925> 2014-08-13 17:38:04.429110 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.147487 7 0.000722 -7924> 2014-08-13 17:38:04.429110 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -7923> 2014-08-13 17:38:04.429129 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194486:22.226) v4 -- ?+0 0x4c16ac0 -7922> 2014-08-13 17:38:04.429133 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7921> 2014-08-13 17:38:04.429140 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog -7920> 2014-08-13 17:38:04.428906 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1b8) v4 from osd.340 -7919> 2014-08-13 17:38:04.429152 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -7918> 2014-08-13 17:38:04.429159 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] flushed -7917> 2014-08-13 17:38:04.429166 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -7916> 2014-08-13 17:38:04.429185 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105b00 -7915> 2014-08-13 17:38:04.429196 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.429196, event: started, request: pg_info(1 pgs e194486:22.1b8) v4 -7914> 2014-08-13 17:38:04.429184 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -7913> 2014-08-13 17:38:04.429191 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -7912> 2014-08-13 17:38:04.429224 7fe9a63f4700 5 -- op tracker -- , seq: 529, time: 2014-08-13 17:38:04.429224, event: done, request: pg_info(1 pgs e194486:22.1b8) v4 -7911> 2014-08-13 17:38:04.429224 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7910> 2014-08-13 17:38:04.429247 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7909> 2014-08-13 17:38:04.429250 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7908> 2014-08-13 17:38:04.429243 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -7907> 2014-08-13 17:38:04.429257 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.289 192.168.1.113:6808/15284 13 ==== pg_info(1 pgs e194486:22.6bd) v4 ==== 714+0+0 (55406756 0 0) 0x4260380 con 0x4cfdac0 -7906> 2014-08-13 17:38:04.429272 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7905> 2014-08-13 17:38:04.429275 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7904> 2014-08-13 17:38:04.429277 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.6bd) v4 -7903> 2014-08-13 17:38:04.429264 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7902> 2014-08-13 17:38:04.429267 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -7901> 2014-08-13 17:38:04.429282 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.424959, event: header_read, request: pg_info(1 pgs e194486:22.6bd) v4 -7900> 2014-08-13 17:38:04.429289 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.424960, event: throttled, request: pg_info(1 pgs e194486:22.6bd) v4 -7899> 2014-08-13 17:38:04.429283 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -7898> 2014-08-13 17:38:04.429291 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] actingbackfill is 4,55,59,351 -7897> 2014-08-13 17:38:04.429311 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] choose_acting want [4,351,55] (== acting) backfill_targets 59 -7896> 2014-08-13 17:38:04.429296 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.425038, event: all_read, request: pg_info(1 pgs e194486:22.6bd) v4 -7895> 2014-08-13 17:38:04.429330 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.429272, event: dispatched, request: pg_info(1 pgs e194486:22.6bd) v4 -7894> 2014-08-13 17:38:04.429338 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.429338, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.6bd) v4 -7893> 2014-08-13 17:38:04.429345 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.6bd) v4 from osd.289 -7892> 2014-08-13 17:38:04.429350 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 -7891> 2014-08-13 17:38:04.429334 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: leaving GetLog -7890> 2014-08-13 17:38:04.429344 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -7889> 2014-08-13 17:38:04.429355 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.429355, event: started, request: pg_info(1 pgs e194486:22.6bd) v4 -7888> 2014-08-13 17:38:04.429362 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000304 0 0.000000 -7887> 2014-08-13 17:38:04.429366 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] actingbackfill is 4,20,25,303,362 -7886> 2014-08-13 17:38:04.429383 7fe9a63f4700 5 -- op tracker -- , seq: 530, time: 2014-08-13 17:38:04.429383, event: done, request: pg_info(1 pgs e194486:22.6bd) v4 -7885> 2014-08-13 17:38:04.429395 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7884> 2014-08-13 17:38:04.429398 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7883> 2014-08-13 17:38:04.429386 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing -7882> 2014-08-13 17:38:04.429386 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] choose_acting want [4,20,303] (== acting) backfill_targets 25,362 -7881> 2014-08-13 17:38:04.429404 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.323 192.168.1.106:6816/8118 6 ==== pg_info(1 pgs e194486:22.12a) v4 ==== 714+0+0 (1683651789 0 0) 0x5105780 con 0x4d7d6a0 -7880> 2014-08-13 17:38:04.429417 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7879> 2014-08-13 17:38:04.429420 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7878> 2014-08-13 17:38:04.429423 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105780 pg_info(1 pgs e194486:22.12a) v4 -7877> 2014-08-13 17:38:04.429411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.55 has no missing, identical log -7876> 2014-08-13 17:38:04.429428 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.426258, event: header_read, request: pg_info(1 pgs e194486:22.12a) v4 -7875> 2014-08-13 17:38:04.429416 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: leaving GetLog -7874> 2014-08-13 17:38:04.429435 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.426261, event: throttled, request: pg_info(1 pgs e194486:22.12a) v4 -7873> 2014-08-13 17:38:04.429441 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.426333, event: all_read, request: pg_info(1 pgs e194486:22.12a) v4 -7872> 2014-08-13 17:38:04.429447 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.429417, event: dispatched, request: pg_info(1 pgs e194486:22.12a) v4 -7871> 2014-08-13 17:38:04.429434 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.351 has no missing, identical log -7870> 2014-08-13 17:38:04.429454 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.429454, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.12a) v4 -7869> 2014-08-13 17:38:04.429441 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000301 0 0.000000 -7868> 2014-08-13 17:38:04.429461 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.12a) v4 from osd.323 -7867> 2014-08-13 17:38:04.429454 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: still need up_thru update before going active -7866> 2014-08-13 17:38:04.429470 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105780 -7865> 2014-08-13 17:38:04.437879 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.437878, event: started, request: pg_info(1 pgs e194486:22.12a) v4 -7864> 2014-08-13 17:38:04.429469 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing -7863> 2014-08-13 17:38:04.437917 7fe9a63f4700 5 -- op tracker -- , seq: 531, time: 2014-08-13 17:38:04.437916, event: done, request: pg_info(1 pgs e194486:22.12a) v4 -7862> 2014-08-13 17:38:04.437931 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7861> 2014-08-13 17:38:04.437935 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7860> 2014-08-13 17:38:04.429477 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000091 0 0.000000 -7859> 2014-08-13 17:38:04.437943 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.51 192.168.1.112:6832/6655 6 ==== pg_info(1 pgs e194486:22.537) v4 ==== 714+0+0 (1589590788 0 0) 0x5102300 con 0x4e39e40 -7858> 2014-08-13 17:38:04.437921 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.20 has no missing, identical log -7857> 2014-08-13 17:38:04.437961 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7856> 2014-08-13 17:38:04.437964 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7855> 2014-08-13 17:38:04.437967 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5102300 pg_info(1 pgs e194486:22.537) v4 -7854> 2014-08-13 17:38:04.437964 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.303 has no missing, identical log -7853> 2014-08-13 17:38:04.437985 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.426759, event: header_read, request: pg_info(1 pgs e194486:22.537) v4 -7852> 2014-08-13 17:38:04.437993 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.426761, event: throttled, request: pg_info(1 pgs e194486:22.537) v4 -7851> 2014-08-13 17:38:04.437985 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: still need up_thru update before going active -7850> 2014-08-13 17:38:04.437999 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.426825, event: all_read, request: pg_info(1 pgs e194486:22.537) v4 -7849> 2014-08-13 17:38:04.437971 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru -7848> 2014-08-13 17:38:04.438006 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.437961, event: dispatched, request: pg_info(1 pgs e194486:22.537) v4 -7847> 2014-08-13 17:38:04.438016 7fe99ebe8700 10 log is not dirty -7846> 2014-08-13 17:38:04.438014 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.438013, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.537) v4 -7845> 2014-08-13 17:38:04.438020 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.537) v4 from osd.51 -7844> 2014-08-13 17:38:04.438025 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5102300 -7843> 2014-08-13 17:38:04.438010 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.008540 0 0.000000 -7842> 2014-08-13 17:38:04.438030 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.438030, event: started, request: pg_info(1 pgs e194486:22.537) v4 -7841> 2014-08-13 17:38:04.438035 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru -7840> 2014-08-13 17:38:04.438050 7fe9a63f4700 5 -- op tracker -- , seq: 532, time: 2014-08-13 17:38:04.438050, event: done, request: pg_info(1 pgs e194486:22.537) v4 -7839> 2014-08-13 17:38:04.438061 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7838> 2014-08-13 17:38:04.438061 7fe99f5e9700 10 log is not dirty -7837> 2014-08-13 17:38:04.438063 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7836> 2014-08-13 17:38:04.438056 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7835> 2014-08-13 17:38:04.438069 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.292 192.168.1.113:6812/16056 6 ==== pg_info(1 pgs e194486:22.3ad) v4 ==== 714+0+0 (1259716739 0 0) 0x5103640 con 0x4e3c8e0 -7834> 2014-08-13 17:38:04.438082 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7833> 2014-08-13 17:38:04.438084 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7832> 2014-08-13 17:38:04.438087 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5103640 pg_info(1 pgs e194486:22.3ad) v4 -7831> 2014-08-13 17:38:04.438092 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.427046, event: header_read, request: pg_info(1 pgs e194486:22.3ad) v4 -7830> 2014-08-13 17:38:04.438084 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops -7829> 2014-08-13 17:38:04.438099 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.427048, event: throttled, request: pg_info(1 pgs e194486:22.3ad) v4 -7828> 2014-08-13 17:38:04.438105 7fe99ebe8700 10 log is not dirty -7827> 2014-08-13 17:38:04.438106 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.427110, event: all_read, request: pg_info(1 pgs e194486:22.3ad) v4 -7826> 2014-08-13 17:38:04.438112 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.438082, event: dispatched, request: pg_info(1 pgs e194486:22.3ad) v4 -7825> 2014-08-13 17:38:04.438096 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 310 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) -7824> 2014-08-13 17:38:04.438120 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.438119, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3ad) v4 -7823> 2014-08-13 17:38:04.438126 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3ad) v4 from osd.292 -7822> 2014-08-13 17:38:04.438131 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5103640 -7821> 2014-08-13 17:38:04.438115 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 356 notify: (query_epoch:194486, epoch_sent:194486, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485)) -7820> 2014-08-13 17:38:04.438135 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.438135, event: started, request: pg_info(1 pgs e194486:22.3ad) v4 -7819> 2014-08-13 17:38:04.438219 7fe9a63f4700 5 -- op tracker -- , seq: 533, time: 2014-08-13 17:38:04.438219, event: done, request: pg_info(1 pgs e194486:22.3ad) v4 -7818> 2014-08-13 17:38:04.438232 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7817> 2014-08-13 17:38:04.438125 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.310 activated and committed -7816> 2014-08-13 17:38:04.438235 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7815> 2014-08-13 17:38:04.438242 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 17 ==== pg_info(1 pgs e194486:22.226) v4 ==== 714+0+0 (1676353387 0 0) 0x4c16ac0 con 0x4e4d280 -7814> 2014-08-13 17:38:04.438255 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7813> 2014-08-13 17:38:04.438258 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7812> 2014-08-13 17:38:04.438259 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4c16ac0 pg_info(1 pgs e194486:22.226) v4 -7811> 2014-08-13 17:38:04.438255 7fe99f5e9700 10 log is not dirty -7810> 2014-08-13 17:38:04.438264 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.429874, event: header_read, request: pg_info(1 pgs e194486:22.226) v4 -7809> 2014-08-13 17:38:04.438271 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.429875, event: throttled, request: pg_info(1 pgs e194486:22.226) v4 -7808> 2014-08-13 17:38:04.438277 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.429949, event: all_read, request: pg_info(1 pgs e194486:22.226) v4 -7807> 2014-08-13 17:38:04.438282 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.438255, event: dispatched, request: pg_info(1 pgs e194486:22.226) v4 -7806> 2014-08-13 17:38:04.438140 7fe99ebe8700 7 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.356 -7805> 2014-08-13 17:38:04.438289 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.438289, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.226) v4 -7804> 2014-08-13 17:38:04.438295 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.226) v4 from osd.376 -7803> 2014-08-13 17:38:04.438299 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4c16ac0 -7802> 2014-08-13 17:38:04.438275 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 357 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) -7801> 2014-08-13 17:38:04.438304 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.438304, event: started, request: pg_info(1 pgs e194486:22.226) v4 -7800> 2014-08-13 17:38:04.438311 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.357 activated and committed -7799> 2014-08-13 17:38:04.438332 7fe99f5e9700 10 log is not dirty -7798> 2014-08-13 17:38:04.438325 7fe9a63f4700 5 -- op tracker -- , seq: 534, time: 2014-08-13 17:38:04.438325, event: done, request: pg_info(1 pgs e194486:22.226) v4 -7797> 2014-08-13 17:38:04.438339 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 353 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) -7796> 2014-08-13 17:38:04.438298 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7795> 2014-08-13 17:38:04.438362 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7794> 2014-08-13 17:38:04.438364 7fe99ebe8700 10 log is not dirty -7793> 2014-08-13 17:38:04.438366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7792> 2014-08-13 17:38:04.438359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.353 activated and committed -7791> 2014-08-13 17:38:04.438376 7fe99f5e9700 10 log is not dirty -7790> 2014-08-13 17:38:04.438371 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.65 192.168.1.111:6842/17345 8 ==== pg_info(1 pgs e194486:22.619) v4 ==== 714+0+0 (565811158 0 0) 0x5823800 con 0x4e86040 -7789> 2014-08-13 17:38:04.438382 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7788> 2014-08-13 17:38:04.438385 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7787> 2014-08-13 17:38:04.438388 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7786> 2014-08-13 17:38:04.438390 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823800 pg_info(1 pgs e194486:22.619) v4 -7785> 2014-08-13 17:38:04.438390 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing -7784> 2014-08-13 17:38:04.438395 7fe99f5e9700 7 osd.4 194486 do_notify osd 15 on 1 PGs -7783> 2014-08-13 17:38:04.438381 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 374 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/193232) -7782> 2014-08-13 17:38:04.438396 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.434295, event: header_read, request: pg_info(1 pgs e194486:22.619) v4 -7781> 2014-08-13 17:38:04.438398 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(35) epoch 194486) v5 -- ?+0 0x51ccd00 con 0x4d78dc0 -7780> 2014-08-13 17:38:04.438414 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.434299, event: throttled, request: pg_info(1 pgs e194486:22.619) v4 -7779> 2014-08-13 17:38:04.438406 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] log request from 374 -7778> 2014-08-13 17:38:04.438421 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.434394, event: all_read, request: pg_info(1 pgs e194486:22.619) v4 -7777> 2014-08-13 17:38:04.438426 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing -7776> 2014-08-13 17:38:04.438430 7fe99f5e9700 7 osd.4 194486 do_notify osd 31 on 1 PGs -7775> 2014-08-13 17:38:04.438427 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.438385, event: dispatched, request: pg_info(1 pgs e194486:22.619) v4 -7774> 2014-08-13 17:38:04.438435 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.438435, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.619) v4 -7773> 2014-08-13 17:38:04.438433 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(43) epoch 194486) v5 -- ?+0 0x51cd5c0 con 0x49e5960 -7772> 2014-08-13 17:38:04.438403 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] flushed -7771> 2014-08-13 17:38:04.438441 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.619) v4 from osd.65 -7770> 2014-08-13 17:38:04.438450 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -7769> 2014-08-13 17:38:04.438446 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823800 -7768> 2014-08-13 17:38:04.438454 7fe99f5e9700 7 osd.4 194486 do_notify osd 285 on 1 PGs -7767> 2014-08-13 17:38:04.438457 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.438457, event: started, request: pg_info(1 pgs e194486:22.619) v4 -7766> 2014-08-13 17:38:04.438457 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(31) epoch 194486) v5 -- ?+0 0x51cd780 con 0x4a45280 -7765> 2014-08-13 17:38:04.438452 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info+missing+log since 194477'0 -7764> 2014-08-13 17:38:04.438475 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -7763> 2014-08-13 17:38:04.438456 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] flushed -7762> 2014-08-13 17:38:04.438481 7fe99f5e9700 7 osd.4 194486 do_notify osd 343 on 1 PGs -7761> 2014-08-13 17:38:04.438484 7fe9a63f4700 5 -- op tracker -- , seq: 535, time: 2014-08-13 17:38:04.438483, event: done, request: pg_info(1 pgs e194486:22.619) v4 -7760> 2014-08-13 17:38:04.438484 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(31) epoch 194486) v5 -- ?+0 0x51c8000 con 0x4d31e40 -7759> 2014-08-13 17:38:04.438473 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending log((194422'16449,194422'16449], crt=194422'16449) missing(52) -7758> 2014-08-13 17:38:04.438500 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7757> 2014-08-13 17:38:04.438503 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7756> 2014-08-13 17:38:04.438508 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.396 192.168.1.105:6814/41806 8 ==== pg_info(1 pgs e194486:24.299) v4 ==== 744+0+0 (3342772929 0 0) 0x51078c0 con 0x4ecb700 -7755> 2014-08-13 17:38:04.438521 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7754> 2014-08-13 17:38:04.438523 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7753> 2014-08-13 17:38:04.438525 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51078c0 pg_info(1 pgs e194486:24.299) v4 -7752> 2014-08-13 17:38:04.438548 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.429127, event: header_read, request: pg_info(1 pgs e194486:24.299) v4 -7751> 2014-08-13 17:38:04.438555 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.429128, event: throttled, request: pg_info(1 pgs e194486:24.299) v4 -7750> 2014-08-13 17:38:04.438561 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.429203, event: all_read, request: pg_info(1 pgs e194486:24.299) v4 -7749> 2014-08-13 17:38:04.438567 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.438520, event: dispatched, request: pg_info(1 pgs e194486:24.299) v4 -7748> 2014-08-13 17:38:04.438574 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.438574, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:24.299) v4 -7747> 2014-08-13 17:38:04.438564 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 28 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7746> 2014-08-13 17:38:04.438589 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:24.299) v4 from osd.396 -7745> 2014-08-13 17:38:04.438595 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51078c0 -7744> 2014-08-13 17:38:04.438592 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7743> 2014-08-13 17:38:04.438600 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.438599, event: started, request: pg_info(1 pgs e194486:24.299) v4 -7742> 2014-08-13 17:38:04.438631 7fe9a63f4700 5 -- op tracker -- , seq: 536, time: 2014-08-13 17:38:04.438630, event: done, request: pg_info(1 pgs e194486:24.299) v4 -7741> 2014-08-13 17:38:04.438625 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged -7740> 2014-08-13 17:38:04.438642 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7739> 2014-08-13 17:38:04.438645 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7738> 2014-08-13 17:38:04.438650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.087382 8 0.000742 -7737> 2014-08-13 17:38:04.438651 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.370 192.168.1.114:6842/56512 5 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x54d9680 con 0x4ecee00 -7736> 2014-08-13 17:38:04.438679 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7735> 2014-08-13 17:38:04.438682 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7734> 2014-08-13 17:38:04.438676 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing -7733> 2014-08-13 17:38:04.438684 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9680 osd_map(194486..194486 src has 185139..194486) v3 -7732> 2014-08-13 17:38:04.438693 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -7731> 2014-08-13 17:38:04.438698 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -7730> 2014-08-13 17:38:04.438674 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -7729> 2014-08-13 17:38:04.438700 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7728> 2014-08-13 17:38:04.438702 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7727> 2014-08-13 17:38:04.438686 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.102:6828/16129 -- pg_log(24.2ec epoch 194486 log log((194422'16449,194422'16449], crt=194422'16449) query_epoch 194486) v4 -- ?+0 0x4872400 con 0x4c75120 -7726> 2014-08-13 17:38:04.438719 7fe99ebe8700 10 log is not dirty -7725> 2014-08-13 17:38:04.438714 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.345 192.168.1.110:6820/46337 8 ==== pg_info(1 pgs e194486:22.29a) v4 ==== 714+0+0 (215095816 0 0) 0x58271c0 con 0x4c70f20 -7724> 2014-08-13 17:38:04.438710 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -7723> 2014-08-13 17:38:04.438726 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7722> 2014-08-13 17:38:04.438734 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7721> 2014-08-13 17:38:04.438737 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58271c0 pg_info(1 pgs e194486:22.29a) v4 -7720> 2014-08-13 17:38:04.438742 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.428987, event: header_read, request: pg_info(1 pgs e194486:22.29a) v4 -7719> 2014-08-13 17:38:04.438731 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) -7718> 2014-08-13 17:38:04.438733 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -7717> 2014-08-13 17:38:04.438752 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.428990, event: throttled, request: pg_info(1 pgs e194486:22.29a) v4 -7716> 2014-08-13 17:38:04.438759 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.429126, event: all_read, request: pg_info(1 pgs e194486:22.29a) v4 -7715> 2014-08-13 17:38:04.438765 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.438726, event: dispatched, request: pg_info(1 pgs e194486:22.29a) v4 -7714> 2014-08-13 17:38:04.438756 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7713> 2014-08-13 17:38:04.438773 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.438772, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.29a) v4 -7712> 2014-08-13 17:38:04.438779 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.29a) v4 from osd.345 -7711> 2014-08-13 17:38:04.438783 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58271c0 -7710> 2014-08-13 17:38:04.438788 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.438788, event: started, request: pg_info(1 pgs e194486:22.29a) v4 -7709> 2014-08-13 17:38:04.438794 7fe99ebe8700 10 log is not dirty -7708> 2014-08-13 17:38:04.438776 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -7707> 2014-08-13 17:38:04.438809 7fe9a63f4700 5 -- op tracker -- , seq: 537, time: 2014-08-13 17:38:04.438809, event: done, request: pg_info(1 pgs e194486:22.29a) v4 -7706> 2014-08-13 17:38:04.438798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -7705> 2014-08-13 17:38:04.438820 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7704> 2014-08-13 17:38:04.438823 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7703> 2014-08-13 17:38:04.438818 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7702> 2014-08-13 17:38:04.438828 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.5 192.168.1.103:6827/40192 3 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4d9da00 con 0x4d36b40 -7701> 2014-08-13 17:38:04.438842 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7700> 2014-08-13 17:38:04.438844 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7699> 2014-08-13 17:38:04.438846 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d9da00 osd_map(194486..194486 src has 185139..194486) v3 -7698> 2014-08-13 17:38:04.438852 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -7697> 2014-08-13 17:38:04.438856 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -7696> 2014-08-13 17:38:04.438858 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7695> 2014-08-13 17:38:04.438860 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7694> 2014-08-13 17:38:04.438865 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 18 ==== MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 ==== 26+0+0 (2983579979 0 0) 0x58239c0 con 0x4e4d280 -7693> 2014-08-13 17:38:04.438878 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7692> 2014-08-13 17:38:04.438880 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7691> 2014-08-13 17:38:04.438882 7fe9a63f4700 20 osd.4 194486 _dispatch 0x58239c0 MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7690> 2014-08-13 17:38:04.438887 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.430863, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7689> 2014-08-13 17:38:04.438876 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -7688> 2014-08-13 17:38:04.438894 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.430865, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7687> 2014-08-13 17:38:04.438900 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.430919, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7686> 2014-08-13 17:38:04.438905 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.438877, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7685> 2014-08-13 17:38:04.438912 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.438912, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7684> 2014-08-13 17:38:04.438903 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,17,28,315,330 -7683> 2014-08-13 17:38:04.438919 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x58239c0 -7682> 2014-08-13 17:38:04.438923 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,17,315] (== acting) backfill_targets 28,330 -7681> 2014-08-13 17:38:04.438934 7fe9a63f4700 5 -- op tracker -- , seq: 538, time: 2014-08-13 17:38:04.438934, event: done, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194486 v2 -7680> 2014-08-13 17:38:04.438947 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7679> 2014-08-13 17:38:04.438949 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7678> 2014-08-13 17:38:04.438954 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.396 192.168.1.105:6814/41806 9 ==== MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 ==== 30+0+0 (2404269850 0 0) 0x5820fc0 con 0x4ecb700 -7677> 2014-08-13 17:38:04.438967 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7676> 2014-08-13 17:38:04.438952 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog -7675> 2014-08-13 17:38:04.438970 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7674> 2014-08-13 17:38:04.438972 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5820fc0 MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7673> 2014-08-13 17:38:04.438977 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.429781, event: header_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7672> 2014-08-13 17:38:04.438986 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.429782, event: throttled, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7671> 2014-08-13 17:38:04.438977 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000303 0 0.000000 -7670> 2014-08-13 17:38:04.438992 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.429882, event: all_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7669> 2014-08-13 17:38:04.438998 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.438967, event: dispatched, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7668> 2014-08-13 17:38:04.439005 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.439005, event: waiting_for_osdmap, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7667> 2014-08-13 17:38:04.439012 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5820fc0 -7666> 2014-08-13 17:38:04.439001 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -7665> 2014-08-13 17:38:04.439021 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.17 has no missing, identical log -7664> 2014-08-13 17:38:04.439007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 362 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) -7663> 2014-08-13 17:38:04.439040 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.315 has no missing, identical log -7662> 2014-08-13 17:38:04.439050 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.362 activated and committed -7661> 2014-08-13 17:38:04.439059 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active -7660> 2014-08-13 17:38:04.439075 7fe99ebe8700 10 log is not dirty -7659> 2014-08-13 17:38:04.439060 7fe9a63f4700 5 -- op tracker -- , seq: 539, time: 2014-08-13 17:38:04.439060, event: done, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194486, prio: 1 v3 -7658> 2014-08-13 17:38:04.439080 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 -7657> 2014-08-13 17:38:04.439085 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) -7656> 2014-08-13 17:38:04.439102 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7655> 2014-08-13 17:38:04.439107 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7654> 2014-08-13 17:38:04.439102 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -7653> 2014-08-13 17:38:04.439123 7fe99f5e9700 10 log is not dirty -7652> 2014-08-13 17:38:04.439113 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.370 192.168.1.114:6842/56512 6 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x54d9440 con 0x4ecee00 -7651> 2014-08-13 17:38:04.439131 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7650> 2014-08-13 17:38:04.439133 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7649> 2014-08-13 17:38:04.439135 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54d9440 osd_map(194486..194486 src has 185139..194486) v3 -7648> 2014-08-13 17:38:04.439143 7fe99ebe8700 10 log is not dirty -7647> 2014-08-13 17:38:04.439142 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -7646> 2014-08-13 17:38:04.439165 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -7645> 2014-08-13 17:38:04.439168 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7644> 2014-08-13 17:38:04.439170 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7643> 2014-08-13 17:38:04.439179 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.345 192.168.1.110:6820/46337 9 ==== pg_info(1 pgs e194486:22.29a) v4 ==== 714+0+0 (2516494604 0 0) 0x5825b00 con 0x4c70f20 -7642> 2014-08-13 17:38:04.439192 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7641> 2014-08-13 17:38:04.439194 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7640> 2014-08-13 17:38:04.439196 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5825b00 pg_info(1 pgs e194486:22.29a) v4 -7639> 2014-08-13 17:38:04.439202 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.429511, event: header_read, request: pg_info(1 pgs e194486:22.29a) v4 -7638> 2014-08-13 17:38:04.439176 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 356 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) -7637> 2014-08-13 17:38:04.439210 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.429513, event: throttled, request: pg_info(1 pgs e194486:22.29a) v4 -7636> 2014-08-13 17:38:04.439195 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) -7635> 2014-08-13 17:38:04.439216 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.429609, event: all_read, request: pg_info(1 pgs e194486:22.29a) v4 -7634> 2014-08-13 17:38:04.439213 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.356 activated and committed -7633> 2014-08-13 17:38:04.439222 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.439191, event: dispatched, request: pg_info(1 pgs e194486:22.29a) v4 -7632> 2014-08-13 17:38:04.439232 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7631> 2014-08-13 17:38:04.439242 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.439242, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.29a) v4 -7630> 2014-08-13 17:38:04.439248 7fe99ebe8700 10 log is not dirty -7629> 2014-08-13 17:38:04.439249 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.29a) v4 from osd.345 -7628> 2014-08-13 17:38:04.439254 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5825b00 -7627> 2014-08-13 17:38:04.439246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7626> 2014-08-13 17:38:04.439271 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing -7625> 2014-08-13 17:38:04.439274 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194486:22.4df) v4 -- ?+0 0x51ccec0 -7624> 2014-08-13 17:38:04.439281 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.439280, event: started, request: pg_info(1 pgs e194486:22.29a) v4 -7623> 2014-08-13 17:38:04.439274 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) -7622> 2014-08-13 17:38:04.439306 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x5431600 don't know epoch, doing nothing -7621> 2014-08-13 17:38:04.439299 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -7620> 2014-08-13 17:38:04.439311 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.357 192.168.1.102:6834/17224 -- pg_info(1 pgs e194486:22.4df) v4 -- ?+0 0x51c8c40 -7619> 2014-08-13 17:38:04.439314 7fe9a63f4700 5 -- op tracker -- , seq: 540, time: 2014-08-13 17:38:04.439314, event: done, request: pg_info(1 pgs e194486:22.29a) v4 -7618> 2014-08-13 17:38:04.439316 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7617> 2014-08-13 17:38:04.439329 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7616> 2014-08-13 17:38:04.439333 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7615> 2014-08-13 17:38:04.439331 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7614> 2014-08-13 17:38:04.439343 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.5 192.168.1.103:6827/40192 4 ==== pg_info(1 pgs e194486:22.1f1) v4 ==== 714+0+0 (3129220304 0 0) 0x5821180 con 0x4d36b40 -7613> 2014-08-13 17:38:04.439357 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7612> 2014-08-13 17:38:04.439382 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7611> 2014-08-13 17:38:04.439384 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5821180 pg_info(1 pgs e194486:22.1f1) v4 -7610> 2014-08-13 17:38:04.439395 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.430467, event: header_read, request: pg_info(1 pgs e194486:22.1f1) v4 -7609> 2014-08-13 17:38:04.439403 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.430468, event: throttled, request: pg_info(1 pgs e194486:22.1f1) v4 -7608> 2014-08-13 17:38:04.439409 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.430539, event: all_read, request: pg_info(1 pgs e194486:22.1f1) v4 -7607> 2014-08-13 17:38:04.439415 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.439357, event: dispatched, request: pg_info(1 pgs e194486:22.1f1) v4 -7606> 2014-08-13 17:38:04.439400 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed -7605> 2014-08-13 17:38:04.439422 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.439421, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.1f1) v4 -7604> 2014-08-13 17:38:04.439428 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.1f1) v4 from osd.5 -7603> 2014-08-13 17:38:04.439433 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5821180 -7602> 2014-08-13 17:38:04.439437 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.439437, event: started, request: pg_info(1 pgs e194486:22.1f1) v4 -7601> 2014-08-13 17:38:04.439470 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -7600> 2014-08-13 17:38:04.439466 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2658 -7599> 2014-08-13 17:38:04.439476 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:2.2a) v4 -- ?+0 0x5102840 -7598> 2014-08-13 17:38:04.439502 7fe99f5e9700 10 log is not dirty -7597> 2014-08-13 17:38:04.439457 7fe9a63f4700 5 -- op tracker -- , seq: 541, time: 2014-08-13 17:38:04.439457, event: done, request: pg_info(1 pgs e194486:22.1f1) v4 -7596> 2014-08-13 17:38:04.439524 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7595> 2014-08-13 17:38:04.439528 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7594> 2014-08-13 17:38:04.439534 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.370 192.168.1.114:6842/56512 7 ==== pg_notify(1.2b(2) epoch 194486) v5 ==== 804+0+0 (3960060293 0 0) 0x5105cc0 con 0x4ecee00 -7593> 2014-08-13 17:38:04.439552 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7592> 2014-08-13 17:38:04.439555 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7591> 2014-08-13 17:38:04.439557 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5105cc0 pg_notify(1.2b(2) epoch 194486) v5 -7590> 2014-08-13 17:38:04.439542 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:176850 -7589> 2014-08-13 17:38:04.439564 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.428925, event: header_read, request: pg_notify(1.2b(2) epoch 194486) v5 -7588> 2014-08-13 17:38:04.439575 7fe99ebe8700 10 log is not dirty -7587> 2014-08-13 17:38:04.439551 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 17 notify: (query_epoch:194486, epoch_sent:194486, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485)) -7586> 2014-08-13 17:38:04.439572 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.428926, event: throttled, request: pg_notify(1.2b(2) epoch 194486) v5 -7585> 2014-08-13 17:38:04.439583 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.429019, event: all_read, request: pg_notify(1.2b(2) epoch 194486) v5 -7584> 2014-08-13 17:38:04.439590 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.439551, event: dispatched, request: pg_notify(1.2b(2) epoch 194486) v5 -7583> 2014-08-13 17:38:04.439584 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.17 -7582> 2014-08-13 17:38:04.439610 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.439609, event: waiting_for_osdmap, request: pg_notify(1.2b(2) epoch 194486) v5 -7581> 2014-08-13 17:38:04.439618 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.370 -7580> 2014-08-13 17:38:04.439621 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5105cc0 -7579> 2014-08-13 17:38:04.439626 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.439626, event: started, request: pg_notify(1.2b(2) epoch 194486) v5 -7578> 2014-08-13 17:38:04.439603 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.17 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7577> 2014-08-13 17:38:04.439641 7fe99f5e9700 10 log is not dirty -7576> 2014-08-13 17:38:04.439650 7fe9a63f4700 5 -- op tracker -- , seq: 542, time: 2014-08-13 17:38:04.439650, event: done, request: pg_notify(1.2b(2) epoch 194486) v5 -7575> 2014-08-13 17:38:04.439661 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7574> 2014-08-13 17:38:04.439664 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7573> 2014-08-13 17:38:04.439670 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.370 192.168.1.114:6842/56512 8 ==== pg_notify(2.2a(2) epoch 194486) v5 ==== 804+0+0 (351394628 0 0) 0x5106e40 con 0x4ecee00 -7572> 2014-08-13 17:38:04.439683 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7571> 2014-08-13 17:38:04.439685 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7570> 2014-08-13 17:38:04.439686 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5106e40 pg_notify(2.2a(2) epoch 194486) v5 -7569> 2014-08-13 17:38:04.439692 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.429035, event: header_read, request: pg_notify(2.2a(2) epoch 194486) v5 -7568> 2014-08-13 17:38:04.439716 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.429037, event: throttled, request: pg_notify(2.2a(2) epoch 194486) v5 -7567> 2014-08-13 17:38:04.439732 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.429242, event: all_read, request: pg_notify(2.2a(2) epoch 194486) v5 -7566> 2014-08-13 17:38:04.439738 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.439682, event: dispatched, request: pg_notify(2.2a(2) epoch 194486) v5 -7565> 2014-08-13 17:38:04.439744 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.439744, event: waiting_for_osdmap, request: pg_notify(2.2a(2) epoch 194486) v5 -7564> 2014-08-13 17:38:04.439760 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.370 -7563> 2014-08-13 17:38:04.439764 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5106e40 -7562> 2014-08-13 17:38:04.439768 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.439768, event: started, request: pg_notify(2.2a(2) epoch 194486) v5 -7561> 2014-08-13 17:38:04.439789 7fe9a63f4700 5 -- op tracker -- , seq: 543, time: 2014-08-13 17:38:04.439789, event: done, request: pg_notify(2.2a(2) epoch 194486) v5 -7560> 2014-08-13 17:38:04.439800 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7559> 2014-08-13 17:38:04.439803 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7558> 2014-08-13 17:38:04.439787 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 47 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) -7557> 2014-08-13 17:38:04.439797 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 303 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -7556> 2014-08-13 17:38:04.439818 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.47 activated and committed -7555> 2014-08-13 17:38:04.439822 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle_pg_notify from osd.303 -7554> 2014-08-13 17:38:04.439839 7fe99ebe8700 10 log is not dirty -7553> 2014-08-13 17:38:04.439848 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7552> 2014-08-13 17:38:04.439841 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.303 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7551> 2014-08-13 17:38:04.439876 7fe99f5e9700 10 log is not dirty -7550> 2014-08-13 17:38:04.439870 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops -7549> 2014-08-13 17:38:04.439888 7fe99ebe8700 10 log is not dirty -7548> 2014-08-13 17:38:04.439885 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 373 notify: (query_epoch:194486, epoch_sent:194486, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -7547> 2014-08-13 17:38:04.439897 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7546> 2014-08-13 17:38:04.439905 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle_pg_notify from osd.373 -7545> 2014-08-13 17:38:04.439917 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops -7544> 2014-08-13 17:38:04.439935 7fe99ebe8700 10 log is not dirty -7543> 2014-08-13 17:38:04.439924 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.373 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7542> 2014-08-13 17:38:04.439942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7541> 2014-08-13 17:38:04.439962 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops -7540> 2014-08-13 17:38:04.439976 7fe99f5e9700 10 log is not dirty -7539> 2014-08-13 17:38:04.439978 7fe99ebe8700 10 log is not dirty -7538> 2014-08-13 17:38:04.439990 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 56 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) -7537> 2014-08-13 17:38:04.439994 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194483/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194480 194485/194485/192972) -7536> 2014-08-13 17:38:04.440011 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.56 activated and committed -7535> 2014-08-13 17:38:04.440029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7534> 2014-08-13 17:38:04.440047 7fe99ebe8700 10 log is not dirty -7533> 2014-08-13 17:38:04.440043 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7532> 2014-08-13 17:38:04.440066 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4c77640 don't know epoch, doing nothing -7531> 2014-08-13 17:38:04.440071 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7530> 2014-08-13 17:38:04.440070 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.56 192.168.1.111:6826/11446 -- pg_info(1 pgs e194486:22.775) v4 -- ?+0 0x51c8540 -7529> 2014-08-13 17:38:04.440096 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x57570c0 don't know epoch, doing nothing -7528> 2014-08-13 17:38:04.440099 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.353 192.168.1.108:6836/6227 -- pg_info(1 pgs e194486:22.775) v4 -- ?+0 0x51ce3c0 -7527> 2014-08-13 17:38:04.440113 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194480 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) -7526> 2014-08-13 17:38:04.440128 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2721 -7525> 2014-08-13 17:38:04.440154 7fe99f5e9700 10 log is not dirty -7524> 2014-08-13 17:38:04.440189 7fe99ebe8700 10 log is not dirty -7523> 2014-08-13 17:38:04.440287 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7522> 2014-08-13 17:38:04.440325 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] requeue_ops -7521> 2014-08-13 17:38:04.440343 7fe99f5e9700 10 log is not dirty -7520> 2014-08-13 17:38:04.440318 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194483/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194480 194485/194485/192949) -7519> 2014-08-13 17:38:04.440352 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7518> 2014-08-13 17:38:04.440373 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] requeue_ops -7517> 2014-08-13 17:38:04.440392 7fe99ebe8700 10 log is not dirty -7516> 2014-08-13 17:38:04.440402 7fe99f5e9700 10 log is not dirty -7515> 2014-08-13 17:38:04.440410 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7514> 2014-08-13 17:38:04.440418 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194480 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) -7513> 2014-08-13 17:38:04.440430 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] requeue_ops -7512> 2014-08-13 17:38:04.440448 7fe99f5e9700 10 log is not dirty -7511> 2014-08-13 17:38:04.440455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194483/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194480 194485/194485/192936) -7510> 2014-08-13 17:38:04.440469 7fe99ebe8700 10 log is not dirty -7509> 2014-08-13 17:38:04.440491 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7508> 2014-08-13 17:38:04.440511 7fe99f5e9700 10 log is not dirty -7507> 2014-08-13 17:38:04.440514 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops -7506> 2014-08-13 17:38:04.440532 7fe99ebe8700 10 log is not dirty -7505> 2014-08-13 17:38:04.440540 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7504> 2014-08-13 17:38:04.440545 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192971) -7503> 2014-08-13 17:38:04.440560 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops -7502> 2014-08-13 17:38:04.440577 7fe99ebe8700 10 log is not dirty -7501> 2014-08-13 17:38:04.440599 7fe99f5e9700 10 log is not dirty -7500> 2014-08-13 17:38:04.440585 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194483/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) -7499> 2014-08-13 17:38:04.440630 7fe99ebe8700 10 log is not dirty -7498> 2014-08-13 17:38:04.440616 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) -7497> 2014-08-13 17:38:04.440661 7fe99f5e9700 10 log is not dirty -7496> 2014-08-13 17:38:04.440718 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) -7495> 2014-08-13 17:38:04.440728 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194480 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) -7494> 2014-08-13 17:38:04.440746 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.58 activated and committed -7493> 2014-08-13 17:38:04.440765 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7492> 2014-08-13 17:38:04.440783 7fe99f5e9700 10 log is not dirty -7491> 2014-08-13 17:38:04.440780 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7490> 2014-08-13 17:38:04.440804 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e1080 don't know epoch, doing nothing -7489> 2014-08-13 17:38:04.440802 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7488> 2014-08-13 17:38:04.440808 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194486:22.362) v4 -- ?+0 0x5102680 -7487> 2014-08-13 17:38:04.440824 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops -7486> 2014-08-13 17:38:04.440842 7fe99f5e9700 10 log is not dirty -7485> 2014-08-13 17:38:04.440841 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -7484> 2014-08-13 17:38:04.440846 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194486:22.362) v4 -- ?+0 0x5104980 -7483> 2014-08-13 17:38:04.440867 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2681 -7482> 2014-08-13 17:38:04.440874 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192968) -7481> 2014-08-13 17:38:04.440894 7fe99ebe8700 10 log is not dirty -7480> 2014-08-13 17:38:04.440924 7fe99f5e9700 10 log is not dirty -7479> 2014-08-13 17:38:04.440958 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 61 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) -7478> 2014-08-13 17:38:04.440982 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.61 activated and committed -7477> 2014-08-13 17:38:04.440981 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) -7476> 2014-08-13 17:38:04.440999 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7475> 2014-08-13 17:38:04.441014 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7474> 2014-08-13 17:38:04.441033 7fe99f5e9700 10 log is not dirty -7473> 2014-08-13 17:38:04.441038 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing -7472> 2014-08-13 17:38:04.441042 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194486:22.1d2) v4 -- ?+0 0x513e740 -7471> 2014-08-13 17:38:04.441072 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x5380c60 don't know epoch, doing nothing -7470> 2014-08-13 17:38:04.441075 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194486:22.1d2) v4 -- ?+0 0x513d5c0 -7469> 2014-08-13 17:38:04.441099 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:3020 -7468> 2014-08-13 17:38:04.441123 7fe99ebe8700 10 log is not dirty -7467> 2014-08-13 17:38:04.441219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194486 n=44 ec=192688 les/c 194486/194463 194485/194485/194468) -7466> 2014-08-13 17:38:04.441282 7fe99ebe8700 10 log is not dirty -7465> 2014-08-13 17:38:04.441302 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery -7464> 2014-08-13 17:38:04.441328 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.042382 3 0.000163 -7463> 2014-08-13 17:38:04.441353 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7462> 2014-08-13 17:38:04.441391 7fe99ebe8700 10 log is not dirty -7461> 2014-08-13 17:38:04.441383 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7460> 2014-08-13 17:38:04.441400 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 62 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) -7459> 2014-08-13 17:38:04.441423 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.62 activated and committed -7458> 2014-08-13 17:38:04.441425 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops -7457> 2014-08-13 17:38:04.441442 7fe99ebe8700 10 log is not dirty -7456> 2014-08-13 17:38:04.441446 7fe99f5e9700 10 log is not dirty -7455> 2014-08-13 17:38:04.441449 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 21 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) -7454> 2014-08-13 17:38:04.441455 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7453> 2014-08-13 17:38:04.441469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.21 activated and committed -7452> 2014-08-13 17:38:04.441480 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops -7451> 2014-08-13 17:38:04.441499 7fe99f5e9700 10 log is not dirty -7450> 2014-08-13 17:38:04.441488 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7449> 2014-08-13 17:38:04.441504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7448> 2014-08-13 17:38:04.441508 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7447> 2014-08-13 17:38:04.441526 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d335a0 don't know epoch, doing nothing -7446> 2014-08-13 17:38:04.441529 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] requeue_ops -7445> 2014-08-13 17:38:04.441530 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.21 192.168.1.107:6821/54786 -- pg_info(1 pgs e194486:22.707) v4 -- ?+0 0x513db00 -7444> 2014-08-13 17:38:04.441548 7fe99f5e9700 10 log is not dirty -7443> 2014-08-13 17:38:04.441590 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf900 don't know epoch, doing nothing -7442> 2014-08-13 17:38:04.441575 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/192943) -7441> 2014-08-13 17:38:04.441597 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.310 192.168.1.109:6812/32413 -- pg_info(1 pgs e194486:22.707) v4 -- ?+0 0x58224c0 -7440> 2014-08-13 17:38:04.441620 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2569 -7439> 2014-08-13 17:38:04.441638 7fe99f5e9700 10 log is not dirty -7438> 2014-08-13 17:38:04.441646 7fe99ebe8700 10 log is not dirty -7437> 2014-08-13 17:38:04.441659 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7436> 2014-08-13 17:38:04.441707 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) -7435> 2014-08-13 17:38:04.442013 7fe99f5e9700 10 log is not dirty -7434> 2014-08-13 17:38:04.442045 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192958) -7433> 2014-08-13 17:38:04.442036 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194484/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) -7432> 2014-08-13 17:38:04.442064 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.44 192.168.1.111:6822/10782 7 ==== pg_info(1 pgs e194486:22.3af) v4 ==== 714+0+0 (1992968203 0 0) 0x5826ac0 con 0x4d348e0 -7431> 2014-08-13 17:38:04.442076 7fe99f5e9700 10 log is not dirty -7430> 2014-08-13 17:38:04.442078 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7429> 2014-08-13 17:38:04.442080 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7428> 2014-08-13 17:38:04.442081 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5826ac0 pg_info(1 pgs e194486:22.3af) v4 -7427> 2014-08-13 17:38:04.442084 7fe99ebe8700 10 log is not dirty -7426> 2014-08-13 17:38:04.442085 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.441993, event: header_read, request: pg_info(1 pgs e194486:22.3af) v4 -7425> 2014-08-13 17:38:04.442091 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.441994, event: throttled, request: pg_info(1 pgs e194486:22.3af) v4 -7424> 2014-08-13 17:38:04.442086 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) -7423> 2014-08-13 17:38:04.442095 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.442042, event: all_read, request: pg_info(1 pgs e194486:22.3af) v4 -7422> 2014-08-13 17:38:04.442099 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.442077, event: dispatched, request: pg_info(1 pgs e194486:22.3af) v4 -7421> 2014-08-13 17:38:04.442103 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.442103, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.3af) v4 -7420> 2014-08-13 17:38:04.442107 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.3af) v4 from osd.44 -7419> 2014-08-13 17:38:04.442099 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7418> 2014-08-13 17:38:04.442110 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5826ac0 -7417> 2014-08-13 17:38:04.442113 7fe99f5e9700 10 log is not dirty -7416> 2014-08-13 17:38:04.442113 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.442113, event: started, request: pg_info(1 pgs e194486:22.3af) v4 -7415> 2014-08-13 17:38:04.442114 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops -7414> 2014-08-13 17:38:04.442125 7fe99ebe8700 10 log is not dirty -7413> 2014-08-13 17:38:04.442129 7fe9a63f4700 5 -- op tracker -- , seq: 544, time: 2014-08-13 17:38:04.442129, event: done, request: pg_info(1 pgs e194486:22.3af) v4 -7412> 2014-08-13 17:38:04.442123 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194483/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) -7411> 2014-08-13 17:38:04.442138 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7410> 2014-08-13 17:38:04.442140 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7409> 2014-08-13 17:38:04.442129 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7408> 2014-08-13 17:38:04.442151 7fe99f5e9700 10 log is not dirty -7407> 2014-08-13 17:38:04.442145 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] requeue_ops -7406> 2014-08-13 17:38:04.442156 7fe99ebe8700 10 log is not dirty -7405> 2014-08-13 17:38:04.442175 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery -7404> 2014-08-13 17:38:04.442175 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery -7403> 2014-08-13 17:38:04.442210 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.043977 3 0.000177 -7402> 2014-08-13 17:38:04.442239 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7401> 2014-08-13 17:38:04.442250 7fe99f5e9700 10 log is not dirty -7400> 2014-08-13 17:38:04.442255 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 341 info: 22.48e( v 194215'543 lc 0'0 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) -7399> 2014-08-13 17:38:04.442266 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.341 activated and committed -7398> 2014-08-13 17:38:04.442278 7fe99f5e9700 10 log is not dirty -7397> 2014-08-13 17:38:04.442208 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.059228 3 0.000177 -7396> 2014-08-13 17:38:04.442282 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7395> 2014-08-13 17:38:04.442319 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] check_local -7394> 2014-08-13 17:38:04.442337 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] requeue_ops -7393> 2014-08-13 17:38:04.442354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] needs_recovery osd.285 has 29 missing -7392> 2014-08-13 17:38:04.442295 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7391> 2014-08-13 17:38:04.442371 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7390> 2014-08-13 17:38:04.442433 7fe99ebe8700 10 log is not dirty -7389> 2014-08-13 17:38:04.442441 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7388> 2014-08-13 17:38:04.442455 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] requeue_ops -7387> 2014-08-13 17:38:04.442466 7fe99ebe8700 10 log is not dirty -7386> 2014-08-13 17:38:04.442469 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7385> 2014-08-13 17:38:04.442483 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops -7384> 2014-08-13 17:38:04.442493 7fe99ebe8700 10 log is not dirty -7383> 2014-08-13 17:38:04.442497 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2861 -7382> 2014-08-13 17:38:04.442499 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192935) -7381> 2014-08-13 17:38:04.442508 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] hit_set_clear -7380> 2014-08-13 17:38:04.442520 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] agent_stop -7379> 2014-08-13 17:38:04.442529 7fe99ebe8700 10 log is not dirty -7378> 2014-08-13 17:38:04.442529 7fe99f5e9700 10 log is not dirty -7377> 2014-08-13 17:38:04.442538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7376> 2014-08-13 17:38:04.442550 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops -7375> 2014-08-13 17:38:04.442560 7fe99f5e9700 10 log is not dirty -7374> 2014-08-13 17:38:04.442548 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7373> 2014-08-13 17:38:04.442566 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops -7372> 2014-08-13 17:38:04.442566 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7371> 2014-08-13 17:38:04.442577 7fe99ebe8700 10 log is not dirty -7370> 2014-08-13 17:38:04.442581 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7369> 2014-08-13 17:38:04.442578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] check_local -7368> 2014-08-13 17:38:04.442586 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] requeue_ops -7367> 2014-08-13 17:38:04.442610 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.31 has 29 missing -7366> 2014-08-13 17:38:04.442625 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.374 has 29 missing -7365> 2014-08-13 17:38:04.442639 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7364> 2014-08-13 17:38:04.442636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194483/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192949) -7363> 2014-08-13 17:38:04.442691 7fe99ebe8700 10 log is not dirty -7362> 2014-08-13 17:38:04.442702 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) -7361> 2014-08-13 17:38:04.442729 7fe99ebe8700 10 log is not dirty -7360> 2014-08-13 17:38:04.442738 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194480 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) -7359> 2014-08-13 17:38:04.442764 7fe99ebe8700 10 log is not dirty -7358> 2014-08-13 17:38:04.442759 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2872 -7357> 2014-08-13 17:38:04.442768 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] hit_set_clear -7356> 2014-08-13 17:38:04.442773 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7355> 2014-08-13 17:38:04.442777 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] agent_stop -7354> 2014-08-13 17:38:04.442786 7fe99f5e9700 10 log is not dirty -7353> 2014-08-13 17:38:04.442786 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops -7352> 2014-08-13 17:38:04.442796 7fe99ebe8700 10 log is not dirty -7351> 2014-08-13 17:38:04.442790 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7350> 2014-08-13 17:38:04.442802 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] requeue_ops -7349> 2014-08-13 17:38:04.442801 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 7 notify: (query_epoch:194486, epoch_sent:194486, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194483 n=29 ec=183096 les/c 194483/194480 194485/194485/194485)) -7348> 2014-08-13 17:38:04.442812 7fe99f5e9700 10 log is not dirty -7347> 2014-08-13 17:38:04.442813 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring -7346> 2014-08-13 17:38:04.442823 7fe99ebe8700 10 log is not dirty -7345> 2014-08-13 17:38:04.442816 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7344> 2014-08-13 17:38:04.442843 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops -7343> 2014-08-13 17:38:04.442842 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 31 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194372 194485/194486/192978) -7342> 2014-08-13 17:38:04.442860 7fe99f5e9700 10 log is not dirty -7341> 2014-08-13 17:38:04.442863 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log request from 31 -7340> 2014-08-13 17:38:04.442867 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194483/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194480 194485/194485/192976) -7339> 2014-08-13 17:38:04.442917 7fe99f5e9700 10 log is not dirty -7338> 2014-08-13 17:38:04.442910 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info+missing+log since 194478'0 -7337> 2014-08-13 17:38:04.442920 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending log((194422'18567,194422'18567], crt=194422'18567) missing(46) -7336> 2014-08-13 17:38:04.442931 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e5960 don't know epoch, doing nothing -7335> 2014-08-13 17:38:04.442928 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194480 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) -7334> 2014-08-13 17:38:04.442933 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6831/41157 -- pg_log(24.1e4 epoch 194486 log log((194422'18567,194422'18567], crt=194422'18567) query_epoch 194486) v4 -- ?+0 0x4873c00 con 0x49e5960 -7333> 2014-08-13 17:38:04.442951 7fe99ebe8700 10 log is not dirty -7332> 2014-08-13 17:38:04.442957 7fe99f5e9700 10 log is not dirty -7331> 2014-08-13 17:38:04.443063 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 285 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192975) -7330> 2014-08-13 17:38:04.443079 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log request from 285 -7329> 2014-08-13 17:38:04.443073 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) -7328> 2014-08-13 17:38:04.443113 7fe99f5e9700 10 log is not dirty -7327> 2014-08-13 17:38:04.443111 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194477'0 -7326> 2014-08-13 17:38:04.443121 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending log((194422'17642,194422'17642], crt=194422'17642) missing(50) -7325> 2014-08-13 17:38:04.443132 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -7324> 2014-08-13 17:38:04.443124 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 380 notify: (query_epoch:194486, epoch_sent:194486, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) -7323> 2014-08-13 17:38:04.443134 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6822/26295 -- pg_log(24.1e6 epoch 194486 log log((194422'17642,194422'17642], crt=194422'17642) query_epoch 194486) v4 -- ?+0 0x5fb7800 con 0x4a45280 -7322> 2014-08-13 17:38:04.443137 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 380, already have info from that osd, ignoring -7321> 2014-08-13 17:38:04.443164 7fe99f5e9700 10 log is not dirty -7320> 2014-08-13 17:38:04.443165 7fe99ebe8700 10 log is not dirty -7319> 2014-08-13 17:38:04.443173 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7318> 2014-08-13 17:38:04.443200 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops -7317> 2014-08-13 17:38:04.443223 7fe99f5e9700 10 log is not dirty -7316> 2014-08-13 17:38:04.443286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 343 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/193262) -7315> 2014-08-13 17:38:04.443293 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7314> 2014-08-13 17:38:04.443300 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log request from 343 -7313> 2014-08-13 17:38:04.443306 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops -7312> 2014-08-13 17:38:04.443317 7fe99f5e9700 10 log is not dirty -7311> 2014-08-13 17:38:04.443321 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194483/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/192978) -7310> 2014-08-13 17:38:04.443323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info+missing+log since 194477'0 -7309> 2014-08-13 17:38:04.443333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending log((194422'14476,194422'14476], crt=194422'14476) missing(41) -7308> 2014-08-13 17:38:04.443344 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -7307> 2014-08-13 17:38:04.443348 7fe99f5e9700 10 log is not dirty -7306> 2014-08-13 17:38:04.443346 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- pg_log(24.206 epoch 194486 log log((194422'14476,194422'14476], crt=194422'14476) query_epoch 194486) v4 -- ?+0 0x5fb7200 con 0x4d31e40 -7305> 2014-08-13 17:38:04.443359 7fe99ebe8700 10 log is not dirty -7304> 2014-08-13 17:38:04.443359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7303> 2014-08-13 17:38:04.443373 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] requeue_ops -7302> 2014-08-13 17:38:04.443384 7fe99f5e9700 10 log is not dirty -7301> 2014-08-13 17:38:04.443436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 15 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/193264) -7300> 2014-08-13 17:38:04.443441 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) -7299> 2014-08-13 17:38:04.443450 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] log request from 15 -7298> 2014-08-13 17:38:04.443470 7fe99f5e9700 10 log is not dirty -7297> 2014-08-13 17:38:04.443480 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7296> 2014-08-13 17:38:04.443480 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194478'0 -7295> 2014-08-13 17:38:04.443490 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending log((194422'11806,194422'11806], crt=194422'11806) missing(56) -7294> 2014-08-13 17:38:04.443501 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d78dc0 don't know epoch, doing nothing -7293> 2014-08-13 17:38:04.443503 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6812/53408 -- pg_log(24.20e epoch 194486 log log((194422'11806,194422'11806], crt=194422'11806) query_epoch 194486) v4 -- ?+0 0x5fb6c00 con 0x4d78dc0 -7292> 2014-08-13 17:38:04.443501 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 7 notify: (query_epoch:194486, epoch_sent:194486, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194483 n=34 ec=183096 les/c 194483/194480 194485/194485/194485)) -7291> 2014-08-13 17:38:04.443515 7fe99ebe8700 10 log is not dirty -7290> 2014-08-13 17:38:04.443514 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring -7289> 2014-08-13 17:38:04.443524 7fe99f5e9700 10 log is not dirty -7288> 2014-08-13 17:38:04.443609 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7287> 2014-08-13 17:38:04.443612 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(log 194422'15992 ec=192688 les/c 194484/194480 194485/194486/192967) -7286> 2014-08-13 17:38:04.443624 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] check_local -7285> 2014-08-13 17:38:04.443626 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 -7284> 2014-08-13 17:38:04.443633 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] requeue_ops -7283> 2014-08-13 17:38:04.443642 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.45 has 27 missing -7282> 2014-08-13 17:38:04.443651 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.339 has 27 missing -7281> 2014-08-13 17:38:04.443659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7280> 2014-08-13 17:38:04.443660 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194422'15992 -7279> 2014-08-13 17:38:04.443671 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) -7278> 2014-08-13 17:38:04.443681 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 -7277> 2014-08-13 17:38:04.443684 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194486 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194486) v4 -- ?+0 0x5fb4e00 con 0x4e3d800 -7276> 2014-08-13 17:38:04.443701 7fe99f5e9700 10 log is not dirty -7275> 2014-08-13 17:38:04.443756 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2934 -7274> 2014-08-13 17:38:04.443766 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] hit_set_clear -7273> 2014-08-13 17:38:04.443788 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] agent_stop -7272> 2014-08-13 17:38:04.443804 7fe99ebe8700 10 log is not dirty -7271> 2014-08-13 17:38:04.443819 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7270> 2014-08-13 17:38:04.443823 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 296 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -7269> 2014-08-13 17:38:04.443844 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops -7268> 2014-08-13 17:38:04.443862 7fe99f5e9700 10 log is not dirty -7267> 2014-08-13 17:38:04.443847 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -7266> 2014-08-13 17:38:04.443869 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7265> 2014-08-13 17:38:04.443882 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged -7264> 2014-08-13 17:38:04.443888 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] check_local -7263> 2014-08-13 17:38:04.443907 7fe99ebe8700 10 log is not dirty -7262> 2014-08-13 17:38:04.443904 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] requeue_ops -7261> 2014-08-13 17:38:04.443915 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7260> 2014-08-13 17:38:04.443919 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] needs_recovery osd.301 has 38 missing -7259> 2014-08-13 17:38:04.443935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local -7258> 2014-08-13 17:38:04.443935 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7257> 2014-08-13 17:38:04.443952 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops -7256> 2014-08-13 17:38:04.443966 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.313 has 32 missing -7255> 2014-08-13 17:38:04.443981 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.318 has 32 missing -7254> 2014-08-13 17:38:04.443995 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7253> 2014-08-13 17:38:04.444160 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2831 -7252> 2014-08-13 17:38:04.444177 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] hit_set_clear -7251> 2014-08-13 17:38:04.444178 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2706 -7250> 2014-08-13 17:38:04.444207 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] agent_stop -7249> 2014-08-13 17:38:04.444217 7fe99f5e9700 10 log is not dirty -7248> 2014-08-13 17:38:04.444210 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear -7247> 2014-08-13 17:38:04.444220 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop -7246> 2014-08-13 17:38:04.444229 7fe99ebe8700 10 log is not dirty -7245> 2014-08-13 17:38:04.444222 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7244> 2014-08-13 17:38:04.444233 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] requeue_ops -7243> 2014-08-13 17:38:04.444243 7fe99f5e9700 10 log is not dirty -7242> 2014-08-13 17:38:04.444234 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 332 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -7241> 2014-08-13 17:38:04.444247 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 289 info: 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) -7240> 2014-08-13 17:38:04.444246 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7239> 2014-08-13 17:38:04.444259 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed -7238> 2014-08-13 17:38:04.444264 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -7237> 2014-08-13 17:38:04.444268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7236> 2014-08-13 17:38:04.444276 7fe99ebe8700 10 log is not dirty -7235> 2014-08-13 17:38:04.444277 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7234> 2014-08-13 17:38:04.444282 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 331 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7233> 2014-08-13 17:38:04.444291 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -7232> 2014-08-13 17:38:04.444295 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194486:22.48e) v4 -- ?+0 0x51ca840 -7231> 2014-08-13 17:38:04.444294 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7230> 2014-08-13 17:38:04.444311 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df32e0 don't know epoch, doing nothing -7229> 2014-08-13 17:38:04.444313 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.341 192.168.1.106:6818/9142 -- pg_info(1 pgs e194486:22.48e) v4 -- ?+0 0x51ca680 -7228> 2014-08-13 17:38:04.444309 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -7227> 2014-08-13 17:38:04.444320 7fe99ebe8700 10 log is not dirty -7226> 2014-08-13 17:38:04.444328 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2688 -7225> 2014-08-13 17:38:04.444343 7fe99f5e9700 10 log is not dirty -7224> 2014-08-13 17:38:04.444333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 379 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -7223> 2014-08-13 17:38:04.444346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7222> 2014-08-13 17:38:04.444359 7fe99ebe8700 10 log is not dirty -7221> 2014-08-13 17:38:04.444504 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7220> 2014-08-13 17:38:04.444501 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 50 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) -7219> 2014-08-13 17:38:04.444517 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops -7218> 2014-08-13 17:38:04.444519 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.50 activated and committed -7217> 2014-08-13 17:38:04.444530 7fe99f5e9700 10 log is not dirty -7216> 2014-08-13 17:38:04.444527 7fe99ebe8700 10 log is not dirty -7215> 2014-08-13 17:38:04.444539 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7214> 2014-08-13 17:38:04.444541 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 343 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) -7213> 2014-08-13 17:38:04.444552 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] requeue_ops -7212> 2014-08-13 17:38:04.444553 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed -7211> 2014-08-13 17:38:04.444565 7fe99ebe8700 10 log is not dirty -7210> 2014-08-13 17:38:04.444565 7fe99f5e9700 10 log is not dirty -7209> 2014-08-13 17:38:04.444574 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 320 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -7208> 2014-08-13 17:38:04.444577 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 286 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) -7207> 2014-08-13 17:38:04.444588 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.286 activated and committed -7206> 2014-08-13 17:38:04.444586 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.320 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours -7205> 2014-08-13 17:38:04.444599 7fe99ebe8700 10 log is not dirty -7204> 2014-08-13 17:38:04.444597 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194483/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7203> 2014-08-13 17:38:04.444603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 356 query_epoch 194486 query: query(log 194478'0 ec=192688 les/c 194484/194484 194485/194486/192969) -7202> 2014-08-13 17:38:04.444606 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7201> 2014-08-13 17:38:04.444619 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing -7200> 2014-08-13 17:38:04.444615 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 -7199> 2014-08-13 17:38:04.444621 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194486:22.3d5) v4 -- ?+0 0x51cb480 -7198> 2014-08-13 17:38:04.444638 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -7197> 2014-08-13 17:38:04.444636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194478'0 -7196> 2014-08-13 17:38:04.444640 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194486:22.3d5) v4 -- ?+0 0x51cb9c0 -7195> 2014-08-13 17:38:04.444647 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) -7194> 2014-08-13 17:38:04.444659 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a486e0 don't know epoch, doing nothing -7193> 2014-08-13 17:38:04.444653 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2649 -7192> 2014-08-13 17:38:04.444668 7fe99f5e9700 10 log is not dirty -7191> 2014-08-13 17:38:04.444661 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194486 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194486) v4 -- ?+0 0x5fb6600 con 0x4a486e0 -7190> 2014-08-13 17:38:04.444694 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194483/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194480 194485/194485/192971) -7189> 2014-08-13 17:38:04.444707 7fe99ebe8700 10 log is not dirty -7188> 2014-08-13 17:38:04.444728 7fe99f5e9700 10 log is not dirty -7187> 2014-08-13 17:38:04.444782 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 344 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486)) -7186> 2014-08-13 17:38:04.444796 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194483/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192972) -7185> 2014-08-13 17:38:04.444803 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.344 info 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours -7184> 2014-08-13 17:38:04.444819 7fe99ebe8700 10 log is not dirty -7183> 2014-08-13 17:38:04.444823 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -7182> 2014-08-13 17:38:04.444826 7fe99f5e9700 10 log is not dirty -7181> 2014-08-13 17:38:04.444840 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7180> 2014-08-13 17:38:04.444847 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 337 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -7179> 2014-08-13 17:38:04.444853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] check_local -7178> 2014-08-13 17:38:04.444862 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] requeue_ops -7177> 2014-08-13 17:38:04.444870 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.344 has 23 missing -7176> 2014-08-13 17:38:04.444861 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7175> 2014-08-13 17:38:04.444883 7fe99ebe8700 10 log is not dirty -7174> 2014-08-13 17:38:04.444879 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.373 has 23 missing -7173> 2014-08-13 17:38:04.444888 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7172> 2014-08-13 17:38:04.444888 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 296 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -7171> 2014-08-13 17:38:04.444900 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.296 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours -7170> 2014-08-13 17:38:04.444913 7fe99ebe8700 10 log is not dirty -7169> 2014-08-13 17:38:04.444916 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 338 info: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) -7168> 2014-08-13 17:38:04.444929 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.338 activated and committed -7167> 2014-08-13 17:38:04.444938 7fe99ebe8700 10 log is not dirty -7166> 2014-08-13 17:38:04.444942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194483/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194480 194485/194485/192978) -7165> 2014-08-13 17:38:04.444976 7fe99ebe8700 10 log is not dirty -7164> 2014-08-13 17:38:04.444970 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2614 -7163> 2014-08-13 17:38:04.444979 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] hit_set_clear -7162> 2014-08-13 17:38:04.444988 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] agent_stop -7161> 2014-08-13 17:38:04.444997 7fe99f5e9700 10 log is not dirty -7160> 2014-08-13 17:38:04.444993 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7159> 2014-08-13 17:38:04.445001 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 347 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) -7158> 2014-08-13 17:38:04.445007 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] check_local -7157> 2014-08-13 17:38:04.445014 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.347 activated and committed -7156> 2014-08-13 17:38:04.445024 7fe99f5e9700 10 log is not dirty -7155> 2014-08-13 17:38:04.445017 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] requeue_ops -7154> 2014-08-13 17:38:04.445028 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7153> 2014-08-13 17:38:04.445029 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 346 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194484 194485/194486/192955) -7152> 2014-08-13 17:38:04.445037 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7151> 2014-08-13 17:38:04.445041 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 -7150> 2014-08-13 17:38:04.445046 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7149> 2014-08-13 17:38:04.445066 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2903 -7148> 2014-08-13 17:38:04.445065 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194477'0 -7147> 2014-08-13 17:38:04.445076 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] hit_set_clear -7146> 2014-08-13 17:38:04.445081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) -7145> 2014-08-13 17:38:04.445085 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] agent_stop -7144> 2014-08-13 17:38:04.445093 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e4a100 already has epoch 194486 -7143> 2014-08-13 17:38:04.445095 7fe99ebe8700 10 log is not dirty -7142> 2014-08-13 17:38:04.445095 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194486 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194486) v4 -- ?+0 0x5fb4800 con 0x4e4a100 -7141> 2014-08-13 17:38:04.445100 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 332 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -7140> 2014-08-13 17:38:04.445111 7fe99f5e9700 10 log is not dirty -7139> 2014-08-13 17:38:04.445112 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485), identical to ours -7138> 2014-08-13 17:38:04.445124 7fe99ebe8700 10 log is not dirty -7137> 2014-08-13 17:38:04.445225 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7136> 2014-08-13 17:38:04.445240 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops -7135> 2014-08-13 17:38:04.445250 7fe99f5e9700 10 log is not dirty -7134> 2014-08-13 17:38:04.445243 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) -7133> 2014-08-13 17:38:04.445254 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7132> 2014-08-13 17:38:04.445262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.376 activated and committed -7131> 2014-08-13 17:38:04.445267 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops -7130> 2014-08-13 17:38:04.445278 7fe99f5e9700 10 log is not dirty -7129> 2014-08-13 17:38:04.445273 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7128> 2014-08-13 17:38:04.445282 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7127> 2014-08-13 17:38:04.445282 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 309 info: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) -7126> 2014-08-13 17:38:04.445295 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d789a0 don't know epoch, doing nothing -7125> 2014-08-13 17:38:04.445287 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.6 192.168.1.107:6809/53141 6 ==== pg_query(24.cd epoch 194486) v3 ==== 144+0+0 (1195190698 0 0) 0x299d280 con 0x4e3d800 -7124> 2014-08-13 17:38:04.445294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.309 activated and committed -7123> 2014-08-13 17:38:04.445304 7fe99f5e9700 10 log is not dirty -7122> 2014-08-13 17:38:04.445303 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7121> 2014-08-13 17:38:04.445298 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.338 192.168.1.106:6814/25893 -- pg_info(1 pgs e194486:22.3d9) v4 -- ?+0 0x5822d80 -7120> 2014-08-13 17:38:04.445306 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7119> 2014-08-13 17:38:04.445307 7fe9a63f4700 20 osd.4 194486 _dispatch 0x299d280 pg_query(24.cd epoch 194486) v3 -7118> 2014-08-13 17:38:04.445311 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445170, event: header_read, request: pg_query(24.cd epoch 194486) v3 -7117> 2014-08-13 17:38:04.445316 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445171, event: throttled, request: pg_query(24.cd epoch 194486) v3 -7116> 2014-08-13 17:38:04.445319 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -7115> 2014-08-13 17:38:04.445320 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445217, event: all_read, request: pg_query(24.cd epoch 194486) v3 -7114> 2014-08-13 17:38:04.445323 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445303, event: dispatched, request: pg_query(24.cd epoch 194486) v3 -7113> 2014-08-13 17:38:04.445322 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194486:22.3d9) v4 -- ?+0 0x5820e00 -7112> 2014-08-13 17:38:04.445327 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445327, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194486) v3 -7111> 2014-08-13 17:38:04.445331 7fe9a63f4700 7 osd.4 194486 handle_pg_query from osd.6 epoch 194486 -7110> 2014-08-13 17:38:04.445333 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x299d280 -7109> 2014-08-13 17:38:04.445336 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445336, event: started, request: pg_query(24.cd epoch 194486) v3 -7108> 2014-08-13 17:38:04.445334 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2594 -7107> 2014-08-13 17:38:04.445337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7106> 2014-08-13 17:38:04.445350 7fe99ebe8700 10 log is not dirty -7105> 2014-08-13 17:38:04.445341 7fe9a63f4700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192967) from replica 6 -7104> 2014-08-13 17:38:04.445351 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops -7103> 2014-08-13 17:38:04.445361 7fe99f5e9700 10 log is not dirty -7102> 2014-08-13 17:38:04.445363 7fe9a63f4700 5 -- op tracker -- , seq: 545, time: 2014-08-13 17:38:04.445362, event: done, request: pg_query(24.cd epoch 194486) v3 -7101> 2014-08-13 17:38:04.445370 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -7100> 2014-08-13 17:38:04.445372 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -7099> 2014-08-13 17:38:04.445375 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 348 notify: (query_epoch:194486, epoch_sent:194486, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7098> 2014-08-13 17:38:04.445377 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) -7097> 2014-08-13 17:38:04.445390 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -7096> 2014-08-13 17:38:04.445390 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7095> 2014-08-13 17:38:04.445400 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194483/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7094> 2014-08-13 17:38:04.445427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7093> 2014-08-13 17:38:04.445428 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -7092> 2014-08-13 17:38:04.445450 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -7091> 2014-08-13 17:38:04.445454 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:22.1b8) v4 -- ?+0 0x5103f00 -7090> 2014-08-13 17:38:04.445453 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.038265 9 0.000991 -7089> 2014-08-13 17:38:04.445479 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing -7088> 2014-08-13 17:38:04.445483 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194486:22.1b8) v4 -- ?+0 0x51040c0 -7087> 2014-08-13 17:38:04.445477 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog -7086> 2014-08-13 17:38:04.445503 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2996 -7085> 2014-08-13 17:38:04.445502 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7084> 2014-08-13 17:38:04.445535 7fe99f5e9700 10 log is not dirty -7083> 2014-08-13 17:38:04.445530 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7082> 2014-08-13 17:38:04.445541 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7081> 2014-08-13 17:38:04.445552 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7080> 2014-08-13 17:38:04.445554 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 289 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) -7079> 2014-08-13 17:38:04.445563 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7078> 2014-08-13 17:38:04.445568 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed -7077> 2014-08-13 17:38:04.445574 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7076> 2014-08-13 17:38:04.445578 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194483/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7075> 2014-08-13 17:38:04.445602 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7074> 2014-08-13 17:38:04.445623 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing -7073> 2014-08-13 17:38:04.445626 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194486:22.6bd) v4 -- ?+0 0x5104280 -7072> 2014-08-13 17:38:04.445640 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -7071> 2014-08-13 17:38:04.445653 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -7070> 2014-08-13 17:38:04.445657 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194486:22.6bd) v4 -- ?+0 0x51c9180 -7069> 2014-08-13 17:38:04.445659 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] actingbackfill is 4,331,332,337,348 -7068> 2014-08-13 17:38:04.445674 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2703 -7067> 2014-08-13 17:38:04.445689 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] choose_acting want [4,332,337] (== acting) backfill_targets 331,348 -7066> 2014-08-13 17:38:04.445701 7fe99f5e9700 10 log is not dirty -7065> 2014-08-13 17:38:04.445703 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: leaving GetLog -7064> 2014-08-13 17:38:04.445716 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000239 0 0.000000 -7063> 2014-08-13 17:38:04.445729 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing -7062> 2014-08-13 17:38:04.445739 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.332 has no missing, identical log -7061> 2014-08-13 17:38:04.445746 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194480 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) -7060> 2014-08-13 17:38:04.445764 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.337 has no missing, identical log -7059> 2014-08-13 17:38:04.445783 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: still need up_thru update before going active -7058> 2014-08-13 17:38:04.445812 7fe99f5e9700 10 log is not dirty -7057> 2014-08-13 17:38:04.445804 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000075 0 0.000000 -7056> 2014-08-13 17:38:04.445825 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru -7055> 2014-08-13 17:38:04.445845 7fe99ebe8700 10 log is not dirty -7054> 2014-08-13 17:38:04.445836 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194480 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) -7053> 2014-08-13 17:38:04.445884 7fe99f5e9700 10 log is not dirty -7052> 2014-08-13 17:38:04.445870 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194480 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) -7051> 2014-08-13 17:38:04.445902 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194483/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) -7050> 2014-08-13 17:38:04.445923 7fe99ebe8700 10 log is not dirty -7049> 2014-08-13 17:38:04.445948 7fe99f5e9700 10 log is not dirty -7048> 2014-08-13 17:38:04.445948 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7047> 2014-08-13 17:38:04.445975 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194483/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194486 n=45 ec=192688 les/c 194486/194463 194485/194485/193205) -7046> 2014-08-13 17:38:04.445978 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] requeue_ops -7045> 2014-08-13 17:38:04.445990 7fe99ebe8700 10 log is not dirty -7044> 2014-08-13 17:38:04.446004 7fe99f5e9700 10 log is not dirty -7043> 2014-08-13 17:38:04.445996 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 65 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) -7042> 2014-08-13 17:38:04.446009 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.65 activated and committed -7041> 2014-08-13 17:38:04.446013 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194483/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192981) -7040> 2014-08-13 17:38:04.446018 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7039> 2014-08-13 17:38:04.446027 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7038> 2014-08-13 17:38:04.446040 7fe99f5e9700 10 log is not dirty -7037> 2014-08-13 17:38:04.446040 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4a4dd80 don't know epoch, doing nothing -7036> 2014-08-13 17:38:04.446043 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.62 192.168.1.112:6824/5505 -- pg_info(1 pgs e194486:22.619) v4 -- ?+0 0x5827e00 -7035> 2014-08-13 17:38:04.446062 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e86040 don't know epoch, doing nothing -7034> 2014-08-13 17:38:04.446064 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.65 192.168.1.111:6842/17345 -- pg_info(1 pgs e194486:22.619) v4 -- ?+0 0x5824d00 -7033> 2014-08-13 17:38:04.446078 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2733 -7032> 2014-08-13 17:38:04.446116 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestRecovery -7031> 2014-08-13 17:38:04.446150 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.017783 3 0.000167 -7030> 2014-08-13 17:38:04.446170 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7029> 2014-08-13 17:38:04.446202 7fe99f5e9700 10 log is not dirty -7028> 2014-08-13 17:38:04.446222 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RequestBackfillPrio: priority 1 -7027> 2014-08-13 17:38:04.446226 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] flushed -7026> 2014-08-13 17:38:04.446240 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] exit Started/ReplicaActive/RepNotRecovering 0.019096 3 0.000234 -7025> 2014-08-13 17:38:04.446253 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] enter Started/ReplicaActive/RepWaitBackfillReserved -7024> 2014-08-13 17:38:04.446264 7fe99f5e9700 10 log is not dirty -7023> 2014-08-13 17:38:04.446268 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194480 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) -7022> 2014-08-13 17:38:04.446298 7fe99f5e9700 10 log is not dirty -7021> 2014-08-13 17:38:04.446308 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7020> 2014-08-13 17:38:04.446320 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] check_local -7019> 2014-08-13 17:38:04.446329 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] requeue_ops -7018> 2014-08-13 17:38:04.446337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.61 has 28 missing -7017> 2014-08-13 17:38:04.446346 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.349 has 28 missing -7016> 2014-08-13 17:38:04.446354 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7015> 2014-08-13 17:38:04.446507 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:3021 -7014> 2014-08-13 17:38:04.446516 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] hit_set_clear -7013> 2014-08-13 17:38:04.446524 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] agent_stop -7012> 2014-08-13 17:38:04.446533 7fe99f5e9700 10 log is not dirty -7011> 2014-08-13 17:38:04.446538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 RemoteRecoveryReserved -7010> 2014-08-13 17:38:04.446553 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x41e5540 don't know epoch, doing nothing -7009> 2014-08-13 17:38:04.446555 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.371 192.168.1.102:6832/16753 -- MRecoveryReserve GRANT pgid: 22.653, query_epoch: 194486 v2 -- ?+0 0x51c9500 -7008> 2014-08-13 17:38:04.446570 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.005217 1 0.000070 -7007> 2014-08-13 17:38:04.446582 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepRecovering -7006> 2014-08-13 17:38:04.446593 7fe99f5e9700 10 log is not dirty -7005> 2014-08-13 17:38:04.446603 7fe99ebe8700 10 log is not dirty -7004> 2014-08-13 17:38:04.446615 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -7003> 2014-08-13 17:38:04.446628 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] check_local -7002> 2014-08-13 17:38:04.446622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -7001> 2014-08-13 17:38:04.446637 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] requeue_ops -7000> 2014-08-13 17:38:04.446646 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.21 has 19 missing -6999> 2014-08-13 17:38:04.446644 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] requeue_ops -6998> 2014-08-13 17:38:04.446657 7fe99ebe8700 10 log is not dirty -6997> 2014-08-13 17:38:04.446655 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.310 has 19 missing -6996> 2014-08-13 17:38:04.446663 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6995> 2014-08-13 17:38:04.446662 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -6994> 2014-08-13 17:38:04.446676 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops -6993> 2014-08-13 17:38:04.446707 7fe99ebe8700 10 log is not dirty -6992> 2014-08-13 17:38:04.446714 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 5 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) -6991> 2014-08-13 17:38:04.446727 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.5 activated and committed -6990> 2014-08-13 17:38:04.446734 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2570 -6989> 2014-08-13 17:38:04.446737 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -6988> 2014-08-13 17:38:04.446743 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] hit_set_clear -6987> 2014-08-13 17:38:04.446747 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] share_pg_info -6986> 2014-08-13 17:38:04.446753 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] agent_stop -6985> 2014-08-13 17:38:04.446763 7fe99f5e9700 10 log is not dirty -6984> 2014-08-13 17:38:04.446762 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d36b40 don't know epoch, doing nothing -6983> 2014-08-13 17:38:04.446764 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.5 192.168.1.103:6827/40192 -- pg_info(1 pgs e194486:22.1f1) v4 -- ?+0 0x5826200 -6982> 2014-08-13 17:38:04.446785 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4829b80 don't know epoch, doing nothing -6981> 2014-08-13 17:38:04.446788 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.47 192.168.1.111:6828/12102 -- pg_info(1 pgs e194486:22.1f1) v4 -- ?+0 0x5823640 -6980> 2014-08-13 17:38:04.446802 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:3006 -6979> 2014-08-13 17:38:04.446823 7fe99ebe8700 10 log is not dirty -6978> 2014-08-13 17:38:04.446859 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6977> 2014-08-13 17:38:04.446872 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] check_local -6976> 2014-08-13 17:38:04.446880 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] requeue_ops -6975> 2014-08-13 17:38:04.446891 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.356 has 35 missing -6974> 2014-08-13 17:38:04.446900 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.357 has 35 missing -6973> 2014-08-13 17:38:04.446909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6972> 2014-08-13 17:38:04.446900 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 44 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) -6971> 2014-08-13 17:38:04.446922 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.44 activated and committed -6970> 2014-08-13 17:38:04.446932 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -6969> 2014-08-13 17:38:04.446941 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] share_pg_info -6968> 2014-08-13 17:38:04.446953 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d348e0 don't know epoch, doing nothing -6967> 2014-08-13 17:38:04.446956 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.44 192.168.1.111:6822/10782 -- pg_info(1 pgs e194486:22.3af) v4 -- ?+0 0x51c81c0 -6966> 2014-08-13 17:38:04.446972 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing -6965> 2014-08-13 17:38:04.446974 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194486:22.3af) v4 -- ?+0 0x51cc280 -6964> 2014-08-13 17:38:04.446986 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2683 -6963> 2014-08-13 17:38:04.447001 7fe99f5e9700 10 log is not dirty -6962> 2014-08-13 17:38:04.447030 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2659 -6961> 2014-08-13 17:38:04.447039 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] hit_set_clear -6960> 2014-08-13 17:38:04.447042 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6959> 2014-08-13 17:38:04.447048 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] agent_stop -6958> 2014-08-13 17:38:04.447058 7fe99ebe8700 10 log is not dirty -6957> 2014-08-13 17:38:04.447056 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.128013 9 0.003443 -6956> 2014-08-13 17:38:04.447062 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6955> 2014-08-13 17:38:04.447069 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6954> 2014-08-13 17:38:04.447073 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local -6953> 2014-08-13 17:38:04.447087 7fe99f5e9700 10 log is not dirty -6952> 2014-08-13 17:38:04.447082 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops -6951> 2014-08-13 17:38:04.447090 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6950> 2014-08-13 17:38:04.447094 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6949> 2014-08-13 17:38:04.447098 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -6948> 2014-08-13 17:38:04.447107 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -6947> 2014-08-13 17:38:04.447107 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116799 9 0.004248 -6946> 2014-08-13 17:38:04.447119 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6945> 2014-08-13 17:38:04.447121 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:176851 -6944> 2014-08-13 17:38:04.447129 7fe99f5e9700 10 log is not dirty -6943> 2014-08-13 17:38:04.447130 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear -6942> 2014-08-13 17:38:04.447133 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6941> 2014-08-13 17:38:04.447138 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop -6940> 2014-08-13 17:38:04.447148 7fe99ebe8700 10 log is not dirty -6939> 2014-08-13 17:38:04.447144 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.134850 9 0.004567 -6938> 2014-08-13 17:38:04.447155 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6937> 2014-08-13 17:38:04.447165 7fe99f5e9700 10 log is not dirty -6936> 2014-08-13 17:38:04.447152 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 370 notify: (query_epoch:194486, epoch_sent:194486, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) -6935> 2014-08-13 17:38:04.447169 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6934> 2014-08-13 17:38:04.447172 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring -6933> 2014-08-13 17:38:04.447183 7fe99ebe8700 10 log is not dirty -6932> 2014-08-13 17:38:04.447179 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.156330 9 0.008452 -6931> 2014-08-13 17:38:04.447186 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 370 notify: (query_epoch:194486, epoch_sent:194486, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194483 n=0 ec=1 les/c 194483/194483 194485/194485/194485)) -6930> 2014-08-13 17:38:04.447190 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6929> 2014-08-13 17:38:04.447197 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] state: Active: got notify from 370, already have info from that osd, ignoring -6928> 2014-08-13 17:38:04.447206 7fe99ebe8700 10 log is not dirty -6927> 2014-08-13 17:38:04.447207 7fe99f5e9700 10 log is not dirty -6926> 2014-08-13 17:38:04.447211 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6925> 2014-08-13 17:38:04.447213 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6924> 2014-08-13 17:38:04.447221 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116053 9 0.006157 -6923> 2014-08-13 17:38:04.447225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] check_local -6922> 2014-08-13 17:38:04.447232 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6921> 2014-08-13 17:38:04.447242 7fe99f5e9700 10 log is not dirty -6920> 2014-08-13 17:38:04.447235 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] requeue_ops -6919> 2014-08-13 17:38:04.447243 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.56 has 32 missing -6918> 2014-08-13 17:38:04.447246 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6917> 2014-08-13 17:38:04.447252 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.353 has 32 missing -6916> 2014-08-13 17:38:04.447258 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] check_local -6915> 2014-08-13 17:38:04.447262 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6914> 2014-08-13 17:38:04.447266 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] requeue_ops -6913> 2014-08-13 17:38:04.447276 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] needs_recovery osd.341 has 40 missing -6912> 2014-08-13 17:38:04.447284 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6911> 2014-08-13 17:38:04.447370 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2722 -6910> 2014-08-13 17:38:04.447379 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] hit_set_clear -6909> 2014-08-13 17:38:04.447387 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] agent_stop -6908> 2014-08-13 17:38:04.447396 7fe99ebe8700 10 log is not dirty -6907> 2014-08-13 17:38:04.447400 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6906> 2014-08-13 17:38:04.447409 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] check_local -6905> 2014-08-13 17:38:04.447417 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] requeue_ops -6904> 2014-08-13 17:38:04.447419 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2689 -6903> 2014-08-13 17:38:04.447425 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.58 has 23 missing -6902> 2014-08-13 17:38:04.447429 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] hit_set_clear -6901> 2014-08-13 17:38:04.447435 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.343 has 23 missing -6900> 2014-08-13 17:38:04.447438 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] agent_stop -6899> 2014-08-13 17:38:04.447447 7fe99f5e9700 10 log is not dirty -6898> 2014-08-13 17:38:04.447444 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6897> 2014-08-13 17:38:04.447451 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6896> 2014-08-13 17:38:04.447462 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] check_local -6895> 2014-08-13 17:38:04.447470 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] requeue_ops -6894> 2014-08-13 17:38:04.447477 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6893> 2014-08-13 17:38:04.447485 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -6892> 2014-08-13 17:38:04.447493 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -6891> 2014-08-13 17:38:04.447505 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2650 -6890> 2014-08-13 17:38:04.447514 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] hit_set_clear -6889> 2014-08-13 17:38:04.447522 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] agent_stop -6888> 2014-08-13 17:38:04.447525 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2682 -6887> 2014-08-13 17:38:04.447534 7fe99f5e9700 10 log is not dirty -6886> 2014-08-13 17:38:04.447534 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] hit_set_clear -6885> 2014-08-13 17:38:04.447538 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6884> 2014-08-13 17:38:04.447543 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] agent_stop -6883> 2014-08-13 17:38:04.447557 7fe99ebe8700 10 log is not dirty -6882> 2014-08-13 17:38:04.447549 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.106604 9 0.002738 -6881> 2014-08-13 17:38:04.447561 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -6880> 2014-08-13 17:38:04.447563 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6879> 2014-08-13 17:38:04.447574 7fe99f5e9700 10 log is not dirty -6878> 2014-08-13 17:38:04.447588 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6877> 2014-08-13 17:38:04.447600 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.116562 9 0.000788 -6876> 2014-08-13 17:38:04.447600 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6875> 2014-08-13 17:38:04.447613 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.155166 9 0.010200 -6874> 2014-08-13 17:38:04.447615 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6873> 2014-08-13 17:38:04.447624 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6872> 2014-08-13 17:38:04.447635 7fe99f5e9700 10 log is not dirty -6871> 2014-08-13 17:38:04.447629 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6870> 2014-08-13 17:38:04.447640 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6869> 2014-08-13 17:38:04.447643 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000029 0 0.000000 -6868> 2014-08-13 17:38:04.447650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.109049 9 0.002770 -6867> 2014-08-13 17:38:04.447657 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6866> 2014-08-13 17:38:04.447661 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6865> 2014-08-13 17:38:04.447672 7fe99f5e9700 10 log is not dirty -6864> 2014-08-13 17:38:04.447666 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] finish_recovery -6863> 2014-08-13 17:38:04.448890 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194480 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6862> 2014-08-13 17:38:04.448890 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6861> 2014-08-13 17:38:04.448914 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] check_local -6860> 2014-08-13 17:38:04.448916 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194476-194484/3 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) -6859> 2014-08-13 17:38:04.448937 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] requeue_ops -6858> 2014-08-13 17:38:04.448942 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194479-194484/2 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [28,304](28) acting [28,304](28) maybe_went_rw) -6857> 2014-08-13 17:38:04.448946 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.44 has 24 missing -6856> 2014-08-13 17:38:04.448954 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 pi=194482-194484/1 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) -6855> 2014-08-13 17:38:04.448956 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.362 has 24 missing -6854> 2014-08-13 17:38:04.448965 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6853> 2014-08-13 17:38:04.448966 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6852> 2014-08-13 17:38:04.448979 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing -6851> 2014-08-13 17:38:04.448981 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194486:22.84) v4 -- ?+0 0x5823d40 -6850> 2014-08-13 17:38:04.448997 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 don't know epoch, doing nothing -6849> 2014-08-13 17:38:04.448999 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194486:22.84) v4 -- ?+0 0x5102a00 -6848> 2014-08-13 17:38:04.449011 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2904 -6847> 2014-08-13 17:38:04.449047 7fe99ebe8700 10 log is not dirty -6846> 2014-08-13 17:38:04.449050 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2684 -6845> 2014-08-13 17:38:04.449059 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] hit_set_clear -6844> 2014-08-13 17:38:04.449067 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] agent_stop -6843> 2014-08-13 17:38:04.449065 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6842> 2014-08-13 17:38:04.449076 7fe99f5e9700 10 log is not dirty -6841> 2014-08-13 17:38:04.449079 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] check_local -6840> 2014-08-13 17:38:04.449081 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 LocalRecoveryReserved -6839> 2014-08-13 17:38:04.449088 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] requeue_ops -6838> 2014-08-13 17:38:04.449093 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.002024 1 0.000032 -6837> 2014-08-13 17:38:04.449097 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] needs_recovery osd.376 has 28 missing -6836> 2014-08-13 17:38:04.449106 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6835> 2014-08-13 17:38:04.449106 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved -6834> 2014-08-13 17:38:04.449120 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6822/26295 -- MRecoveryReserve REQUEST pgid: 22.f3, query_epoch: 194486 v2 -- ?+0 0x51c9f80 con 0x4a45280 -6833> 2014-08-13 17:38:04.449140 7fe99f5e9700 10 log is not dirty -6832> 2014-08-13 17:38:04.449182 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6831> 2014-08-13 17:38:04.449199 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.051432 9 0.003634 -6830> 2014-08-13 17:38:04.449194 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6829> 2014-08-13 17:38:04.449207 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2595 -6828> 2014-08-13 17:38:04.449224 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6827> 2014-08-13 17:38:04.449246 7fe99f5e9700 10 log is not dirty -6826> 2014-08-13 17:38:04.449231 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 -6825> 2014-08-13 17:38:04.449237 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] hit_set_clear -6824> 2014-08-13 17:38:04.449250 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 -6823> 2014-08-13 17:38:04.449254 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] agent_stop -6822> 2014-08-13 17:38:04.449253 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6821> 2014-08-13 17:38:04.449284 7fe99ebe8700 10 log is not dirty -6820> 2014-08-13 17:38:04.449284 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x5250580 don't know epoch, doing nothing -6819> 2014-08-13 17:38:04.449284 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.040892 9 0.000639 -6818> 2014-08-13 17:38:04.449287 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194486; pg22.84; ) v2 -- ?+0 0x51cc0c0 -6817> 2014-08-13 17:38:04.449295 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6816> 2014-08-13 17:38:04.449305 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6815> 2014-08-13 17:38:04.449315 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000020 0 0.000000 -6814> 2014-08-13 17:38:04.449313 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,28,304 -> 4,28,304 -6813> 2014-08-13 17:38:04.449327 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6812> 2014-08-13 17:38:04.449316 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MQuery from 6 query_epoch 194486 query: query(log 194477'0 ec=192688 les/c 194484/194480 194485/194486/192967) -6811> 2014-08-13 17:38:04.449327 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6810> 2014-08-13 17:38:04.449331 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 -6809> 2014-08-13 17:38:04.449331 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2905 -6808> 2014-08-13 17:38:04.449337 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] finish_recovery -6807> 2014-08-13 17:38:04.449346 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194480 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6806> 2014-08-13 17:38:04.449357 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194476-194484/3 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) -6805> 2014-08-13 17:38:04.449369 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194479-194484/2 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [343,286](343) acting [343,286](343) maybe_went_rw) -6804> 2014-08-13 17:38:04.449362 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194477'0 -6803> 2014-08-13 17:38:04.449379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 pi=194482-194484/1 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) -6802> 2014-08-13 17:38:04.449390 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6801> 2014-08-13 17:38:04.449388 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) -6800> 2014-08-13 17:38:04.449400 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e3d800 already has epoch 194486 -6799> 2014-08-13 17:38:04.449403 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing -6798> 2014-08-13 17:38:04.449402 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194486 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194486) v4 -- ?+0 0x5fb0000 con 0x4e3d800 -6797> 2014-08-13 17:38:04.449405 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194486:22.3d5) v4 -- ?+0 0x51ccb40 -6796> 2014-08-13 17:38:04.449420 7fe99ebe8700 10 log is not dirty -6795> 2014-08-13 17:38:04.449512 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x4d31e40 already has epoch 194486 -6794> 2014-08-13 17:38:04.449510 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6793> 2014-08-13 17:38:04.449518 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194486:22.3d5) v4 -- ?+0 0x51c8700 -6792> 2014-08-13 17:38:04.449526 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] check_local -6791> 2014-08-13 17:38:04.449535 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] requeue_ops -6790> 2014-08-13 17:38:04.449535 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2651 -6789> 2014-08-13 17:38:04.449545 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6788> 2014-08-13 17:38:04.449554 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -6787> 2014-08-13 17:38:04.449562 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -6786> 2014-08-13 17:38:04.449575 7fe99f5e9700 10 log is not dirty -6785> 2014-08-13 17:38:04.449576 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2997 -6784> 2014-08-13 17:38:04.449585 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] hit_set_clear -6783> 2014-08-13 17:38:04.449593 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] agent_stop -6782> 2014-08-13 17:38:04.449602 7fe99ebe8700 10 log is not dirty -6781> 2014-08-13 17:38:04.449606 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6780> 2014-08-13 17:38:04.449616 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local -6779> 2014-08-13 17:38:04.449624 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops -6778> 2014-08-13 17:38:04.449620 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6777> 2014-08-13 17:38:04.449633 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6776> 2014-08-13 17:38:04.449637 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.067859 9 0.003059 -6775> 2014-08-13 17:38:04.449642 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -6774> 2014-08-13 17:38:04.449650 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -6773> 2014-08-13 17:38:04.449650 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6772> 2014-08-13 17:38:04.449665 7fe99f5e9700 10 log is not dirty -6771> 2014-08-13 17:38:04.449664 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2704 -6770> 2014-08-13 17:38:04.449672 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear -6769> 2014-08-13 17:38:04.449680 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop -6768> 2014-08-13 17:38:04.449688 7fe99ebe8700 10 log is not dirty -6767> 2014-08-13 17:38:04.449692 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -6766> 2014-08-13 17:38:04.449705 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] requeue_ops -6765> 2014-08-13 17:38:04.449716 7fe99ebe8700 10 log is not dirty -6764> 2014-08-13 17:38:04.449715 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6763> 2014-08-13 17:38:04.449721 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6762> 2014-08-13 17:38:04.449728 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] purge_strays 10 -6761> 2014-08-13 17:38:04.449732 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] check_local -6760> 2014-08-13 17:38:04.449738 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.10 -6759> 2014-08-13 17:38:04.449740 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] requeue_ops -6758> 2014-08-13 17:38:04.449750 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing -6757> 2014-08-13 17:38:04.449749 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.62 has 29 missing -6756> 2014-08-13 17:38:04.449752 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.10 192.168.1.115:6815/60573 -- osd pg remove(epoch 194486; pg22.3d5; ) v2 -- ?+0 0x51cb800 -6755> 2014-08-13 17:38:04.449758 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.65 has 29 missing -6754> 2014-08-13 17:38:04.449766 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6753> 2014-08-13 17:38:04.449769 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,10,286,343 -> 4,286,343 -6752> 2014-08-13 17:38:04.449781 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6751> 2014-08-13 17:38:04.449784 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2652 -6750> 2014-08-13 17:38:04.449869 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2734 -6749> 2014-08-13 17:38:04.449879 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] hit_set_clear -6748> 2014-08-13 17:38:04.449887 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] agent_stop -6747> 2014-08-13 17:38:04.449895 7fe99ebe8700 10 log is not dirty -6746> 2014-08-13 17:38:04.449899 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6745> 2014-08-13 17:38:04.449909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] check_local -6744> 2014-08-13 17:38:04.449917 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] requeue_ops -6743> 2014-08-13 17:38:04.449924 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.5 has 35 missing -6742> 2014-08-13 17:38:04.449933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.47 has 35 missing -6741> 2014-08-13 17:38:04.449941 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6740> 2014-08-13 17:38:04.450100 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:3007 -6739> 2014-08-13 17:38:04.450116 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] hit_set_clear -6738> 2014-08-13 17:38:04.450130 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] agent_stop -6737> 2014-08-13 17:38:04.450144 7fe99ebe8700 10 log is not dirty -6736> 2014-08-13 17:38:04.450151 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6735> 2014-08-13 17:38:04.450177 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.111904 9 0.007559 -6734> 2014-08-13 17:38:04.450204 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6733> 2014-08-13 17:38:04.450220 7fe99ebe8700 10 log is not dirty -6732> 2014-08-13 17:38:04.450225 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6731> 2014-08-13 17:38:04.450235 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.145491 7 0.007806 -6730> 2014-08-13 17:38:04.450245 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6729> 2014-08-13 17:38:04.450254 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6728> 2014-08-13 17:38:04.450267 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000022 0 0.000000 -6727> 2014-08-13 17:38:04.450277 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6726> 2014-08-13 17:38:04.450285 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery -6725> 2014-08-13 17:38:04.450293 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6724> 2014-08-13 17:38:04.450303 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) -6723> 2014-08-13 17:38:04.450314 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194479-194484/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) -6722> 2014-08-13 17:38:04.450323 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194482-194484/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) -6721> 2014-08-13 17:38:04.450333 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6720> 2014-08-13 17:38:04.450344 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -6719> 2014-08-13 17:38:04.450346 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:2.2a) v4 -- ?+0 0x5102bc0 -6718> 2014-08-13 17:38:04.450360 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:176852 -6717> 2014-08-13 17:38:04.450397 7fe99ebe8700 10 log is not dirty -6716> 2014-08-13 17:38:04.450465 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6715> 2014-08-13 17:38:04.450496 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.096489 8 0.004720 -6714> 2014-08-13 17:38:04.450527 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6713> 2014-08-13 17:38:04.450538 7fe99ebe8700 10 log is not dirty -6712> 2014-08-13 17:38:04.450543 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6711> 2014-08-13 17:38:04.450553 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.169019 9 0.003056 -6710> 2014-08-13 17:38:04.450563 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6709> 2014-08-13 17:38:04.450572 7fe99ebe8700 10 log is not dirty -6708> 2014-08-13 17:38:04.450576 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -6707> 2014-08-13 17:38:04.450566 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6706> 2014-08-13 17:38:04.450602 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 -6705> 2014-08-13 17:38:04.450617 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 -6704> 2014-08-13 17:38:04.450636 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing -6703> 2014-08-13 17:38:04.450622 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6702> 2014-08-13 17:38:04.450640 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194486; pg2.2a; ) v2 -- ?+0 0x51cde80 -6701> 2014-08-13 17:38:04.450643 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.037137 13 0.002731 -6700> 2014-08-13 17:38:04.450661 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6699> 2014-08-13 17:38:04.450667 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 -6698> 2014-08-13 17:38:04.450684 7fe99ebe8700 10 log is not dirty -6697> 2014-08-13 17:38:04.450688 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4ecee00 don't know epoch, doing nothing -6696> 2014-08-13 17:38:04.450692 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194486; pg2.2a; ) v2 -- ?+0 0x5f00700 -6695> 2014-08-13 17:38:04.450720 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 -6694> 2014-08-13 17:38:04.450730 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6693> 2014-08-13 17:38:04.450733 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:176853 -6692> 2014-08-13 17:38:04.450728 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6691> 2014-08-13 17:38:04.450749 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.041484 9 0.000729 -6690> 2014-08-13 17:38:04.450761 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6689> 2014-08-13 17:38:04.450772 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6688> 2014-08-13 17:38:04.450782 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000021 0 0.000000 -6687> 2014-08-13 17:38:04.450793 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6686> 2014-08-13 17:38:04.450820 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] finish_recovery -6685> 2014-08-13 17:38:04.450834 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194480 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6684> 2014-08-13 17:38:04.450854 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194476-194484/3 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) -6683> 2014-08-13 17:38:04.450874 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194479-194484/2 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340,347](340) acting [340,347](340) maybe_went_rw) -6682> 2014-08-13 17:38:04.450891 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 pi=194482-194484/1 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) -6681> 2014-08-13 17:38:04.450909 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6680> 2014-08-13 17:38:04.450930 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -6679> 2014-08-13 17:38:04.450934 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:22.1b8) v4 -- ?+0 0x5143640 -6678> 2014-08-13 17:38:04.450975 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4e389a0 don't know epoch, doing nothing -6677> 2014-08-13 17:38:04.450979 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194486:22.1b8) v4 -- ?+0 0x5f02840 -6676> 2014-08-13 17:38:04.450999 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2998 -6675> 2014-08-13 17:38:04.451060 7fe99ebe8700 10 log is not dirty -6674> 2014-08-13 17:38:04.451075 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6673> 2014-08-13 17:38:04.451088 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.086571 9 0.000770 -6672> 2014-08-13 17:38:04.451099 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6671> 2014-08-13 17:38:04.451109 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6670> 2014-08-13 17:38:04.451119 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000020 0 0.000000 -6669> 2014-08-13 17:38:04.451143 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6668> 2014-08-13 17:38:04.451158 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] finish_recovery -6667> 2014-08-13 17:38:04.451171 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194480 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6666> 2014-08-13 17:38:04.451207 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194476-194484/3 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) -6665> 2014-08-13 17:38:04.451219 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194479-194484/2 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [50,289](50) acting [50,289](50) maybe_went_rw) -6664> 2014-08-13 17:38:04.451229 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 pi=194482-194484/1 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) -6663> 2014-08-13 17:38:04.451239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6662> 2014-08-13 17:38:04.451235 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6661> 2014-08-13 17:38:04.451251 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4d3dac0 don't know epoch, doing nothing -6660> 2014-08-13 17:38:04.451250 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 -6659> 2014-08-13 17:38:04.451266 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194486:22.6bd) v4 -- ?+0 0x5f02d80 -6658> 2014-08-13 17:38:04.451274 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 -6657> 2014-08-13 17:38:04.451292 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4cfdac0 already has epoch 194486 -6656> 2014-08-13 17:38:04.451296 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194486:22.6bd) v4 -- ?+0 0x5f02680 -6655> 2014-08-13 17:38:04.451333 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2705 -6654> 2014-08-13 17:38:04.451364 7fe99ebe8700 10 log is not dirty -6653> 2014-08-13 17:38:04.451422 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x5250580 don't know epoch, doing nothing -6652> 2014-08-13 17:38:04.451419 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6651> 2014-08-13 17:38:04.451434 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194486; pg22.1b8; ) v2 -- ?+0 0x5f00380 -6650> 2014-08-13 17:38:04.451444 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.147013 8 0.010107 -6649> 2014-08-13 17:38:04.451471 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6648> 2014-08-13 17:38:04.451483 7fe99ebe8700 10 log is not dirty -6647> 2014-08-13 17:38:04.451472 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,340,347 -> 4,340,347 -6646> 2014-08-13 17:38:04.451485 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6645> 2014-08-13 17:38:04.451487 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6644> 2014-08-13 17:38:04.451489 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2999 -6643> 2014-08-13 17:38:04.451498 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.101572 9 0.010520 -6642> 2014-08-13 17:38:04.451508 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6641> 2014-08-13 17:38:04.451533 7fe99ebe8700 10 log is not dirty -6640> 2014-08-13 17:38:04.451730 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6639> 2014-08-13 17:38:04.451759 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] purge_strays 32 -6638> 2014-08-13 17:38:04.451767 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.32 -6637> 2014-08-13 17:38:04.451777 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4d33180 don't know epoch, doing nothing -6636> 2014-08-13 17:38:04.451779 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.32 192.168.1.115:6812/59915 -- osd pg remove(epoch 194486; pg22.6bd; ) v2 -- ?+0 0x5f00fc0 -6635> 2014-08-13 17:38:04.451794 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,32,50,289 -> 4,50,289 -6634> 2014-08-13 17:38:04.451824 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6633> 2014-08-13 17:38:04.451830 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:2706 -6632> 2014-08-13 17:38:04.466696 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.340 192.168.1.104:6806/62463 11 ==== pg_info(1 pgs e194486:1.2b) v4 ==== 706+0+0 (1678109909 0 0) 0x5143640 con 0x49e40a0 -6631> 2014-08-13 17:38:04.466713 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6630> 2014-08-13 17:38:04.466716 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6629> 2014-08-13 17:38:04.466717 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5143640 pg_info(1 pgs e194486:1.2b) v4 -6628> 2014-08-13 17:38:04.466721 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466614, event: header_read, request: pg_info(1 pgs e194486:1.2b) v4 -6627> 2014-08-13 17:38:04.466728 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466616, event: throttled, request: pg_info(1 pgs e194486:1.2b) v4 -6626> 2014-08-13 17:38:04.466731 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466668, event: all_read, request: pg_info(1 pgs e194486:1.2b) v4 -6625> 2014-08-13 17:38:04.466734 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466712, event: dispatched, request: pg_info(1 pgs e194486:1.2b) v4 -6624> 2014-08-13 17:38:04.466739 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466738, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:1.2b) v4 -6623> 2014-08-13 17:38:04.466742 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:1.2b) v4 from osd.340 -6622> 2014-08-13 17:38:04.466745 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5143640 -6621> 2014-08-13 17:38:04.466748 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466748, event: started, request: pg_info(1 pgs e194486:1.2b) v4 -6620> 2014-08-13 17:38:04.466766 7fe9a63f4700 5 -- op tracker -- , seq: 546, time: 2014-08-13 17:38:04.466766, event: done, request: pg_info(1 pgs e194486:1.2b) v4 -6619> 2014-08-13 17:38:04.466777 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6618> 2014-08-13 17:38:04.466779 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6617> 2014-08-13 17:38:04.466805 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 340 info: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) -6616> 2014-08-13 17:38:04.466853 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -6615> 2014-08-13 17:38:04.466871 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194483/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -6614> 2014-08-13 17:38:04.466885 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info -6613> 2014-08-13 17:38:04.466908 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -6612> 2014-08-13 17:38:04.466912 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:1.2b) v4 -- ?+0 0x51ce040 -6611> 2014-08-13 17:38:04.466943 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:177954 -6610> 2014-08-13 17:38:04.466973 7fe99f5e9700 10 log is not dirty -6609> 2014-08-13 17:38:04.467088 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6608> 2014-08-13 17:38:04.467127 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local -6607> 2014-08-13 17:38:04.467143 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops -6606> 2014-08-13 17:38:04.467159 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6605> 2014-08-13 17:38:04.467184 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -6604> 2014-08-13 17:38:04.467198 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -6603> 2014-08-13 17:38:04.467229 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:177955 -6602> 2014-08-13 17:38:04.467246 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear -6601> 2014-08-13 17:38:04.467261 7fe99f5e9700 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop -6600> 2014-08-13 17:38:04.467277 7fe99f5e9700 10 log is not dirty -6599> 2014-08-13 17:38:04.467289 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasRecovered -6598> 2014-08-13 17:38:04.467307 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.147102 7 0.005614 -6597> 2014-08-13 17:38:04.467328 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -6596> 2014-08-13 17:38:04.467345 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -6595> 2014-08-13 17:38:04.467362 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000035 0 0.000000 -6594> 2014-08-13 17:38:04.467381 7fe99f5e9700 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -6593> 2014-08-13 17:38:04.467396 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery -6592> 2014-08-13 17:38:04.467409 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194483 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state -6591> 2014-08-13 17:38:04.467427 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194476-194484/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194476-194478 up [4,340](4) acting [4,340](4) maybe_went_rw) -6590> 2014-08-13 17:38:04.467447 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194479-194484/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194479-194481 up [340](340) acting [340](340) maybe_went_rw) -6589> 2014-08-13 17:38:04.467463 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 pi=194482-194484/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194482-194484 up [324,370](324) acting [324,370](324) maybe_went_rw) -6588> 2014-08-13 17:38:04.467480 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -6587> 2014-08-13 17:38:04.467501 7fe99f5e9700 20 osd.4 194486 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -6586> 2014-08-13 17:38:04.467506 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194486:1.2b) v4 -- ?+0 0x51ca4c0 -6585> 2014-08-13 17:38:04.467532 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:177956 -6584> 2014-08-13 17:38:04.467598 7fe99f5e9700 10 log is not dirty -6583> 2014-08-13 17:38:04.467797 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -6582> 2014-08-13 17:38:04.467818 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 -6581> 2014-08-13 17:38:04.467843 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 -6580> 2014-08-13 17:38:04.467864 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing -6579> 2014-08-13 17:38:04.467868 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194486; pg1.2b; ) v2 -- ?+0 0x5f032c0 -6578> 2014-08-13 17:38:04.467901 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 -6577> 2014-08-13 17:38:04.467919 7fe9aa308700 20 osd.4 194486 _share_map_outgoing 0x4ecee00 don't know epoch, doing nothing -6576> 2014-08-13 17:38:04.467923 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194486; pg1.2b; ) v2 -- ?+0 0x5f03100 -6575> 2014-08-13 17:38:04.467952 7fe9aa308700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 -6574> 2014-08-13 17:38:04.467962 7fe9aa308700 20 osd.4 194486 need_heartbeat_peer_update -6573> 2014-08-13 17:38:04.467966 7fe9aa308700 15 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194486:177957 -6572> 2014-08-13 17:38:04.480977 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.480591) v2 ==== 47+0+0 (2632851826 0 0) 0x5f03640 con 0x4ec9fa0 -6571> 2014-08-13 17:38:04.480999 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.480591) v2 ==== 47+0+0 (2632851826 0 0) 0x5f03480 con 0x4eca680 -6570> 2014-08-13 17:38:04.481016 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.480591) v2 -- ?+0 0x4c16040 con 0x4ec9fa0 -6569> 2014-08-13 17:38:04.481029 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.58 now has 194486 -6568> 2014-08-13 17:38:04.481036 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e1080 already has epoch 194486 -6567> 2014-08-13 17:38:04.481046 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.480591) v2 -- ?+0 0x4c16200 con 0x4eca680 -6566> 2014-08-13 17:38:04.481066 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e1080 already has epoch 194486 -6565> 2014-08-13 17:38:04.489039 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.307 192.168.1.109:6804/31016 1 ==== osd_map(194479..194486 src has 185139..194486) v3 ==== 5897+0+0 (2398167145 0 0) 0x54dad00 con 0x5a432e0 -6564> 2014-08-13 17:38:04.489064 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6563> 2014-08-13 17:38:04.489068 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6562> 2014-08-13 17:38:04.489069 7fe9a63f4700 20 osd.4 194486 _dispatch 0x54dad00 osd_map(194479..194486 src has 185139..194486) v3 -6561> 2014-08-13 17:38:04.489076 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194479,194486], i have 194486, src has [185139,194486] -6560> 2014-08-13 17:38:04.489079 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -6559> 2014-08-13 17:38:04.489080 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6558> 2014-08-13 17:38:04.489081 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6557> 2014-08-13 17:38:04.489142 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.307 192.168.1.109:6804/31016 2 ==== pg_notify(24.181(40) epoch 194486) v5 ==== 3000+0+0 (3563394220 0 0) 0x5f03800 con 0x5a432e0 -6556> 2014-08-13 17:38:04.489161 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6555> 2014-08-13 17:38:04.489164 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6554> 2014-08-13 17:38:04.489165 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f03800 pg_notify(24.181(40) epoch 194486) v5 -6553> 2014-08-13 17:38:04.489176 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.488969, event: header_read, request: pg_notify(24.181(40) epoch 194486) v5 -6552> 2014-08-13 17:38:04.489187 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.488970, event: throttled, request: pg_notify(24.181(40) epoch 194486) v5 -6551> 2014-08-13 17:38:04.489191 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.489092, event: all_read, request: pg_notify(24.181(40) epoch 194486) v5 -6550> 2014-08-13 17:38:04.489194 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.489160, event: dispatched, request: pg_notify(24.181(40) epoch 194486) v5 -6549> 2014-08-13 17:38:04.489198 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.489198, event: waiting_for_osdmap, request: pg_notify(24.181(40) epoch 194486) v5 -6548> 2014-08-13 17:38:04.489202 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.307 -6547> 2014-08-13 17:38:04.489204 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f03800 -6546> 2014-08-13 17:38:04.489207 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.489207, event: started, request: pg_notify(24.181(40) epoch 194486) v5 -6545> 2014-08-13 17:38:04.489229 7fe9a63f4700 5 -- op tracker -- , seq: 547, time: 2014-08-13 17:38:04.489228, event: done, request: pg_notify(24.181(40) epoch 194486) v5 -6544> 2014-08-13 17:38:04.489238 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6543> 2014-08-13 17:38:04.489240 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6542> 2014-08-13 17:38:04.489239 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 307 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -6541> 2014-08-13 17:38:04.489257 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.307 192.168.1.109:6804/31016 3 ==== pg_notify(24.181(40) epoch 194486) v5 ==== 3000+0+0 (3563394220 0 0) 0x5f03b80 con 0x5a432e0 -6540> 2014-08-13 17:38:04.489266 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6539> 2014-08-13 17:38:04.489268 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6538> 2014-08-13 17:38:04.489269 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5f03b80 pg_notify(24.181(40) epoch 194486) v5 -6537> 2014-08-13 17:38:04.489272 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489108, event: header_read, request: pg_notify(24.181(40) epoch 194486) v5 -6536> 2014-08-13 17:38:04.489265 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6535> 2014-08-13 17:38:04.489276 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489109, event: throttled, request: pg_notify(24.181(40) epoch 194486) v5 -6534> 2014-08-13 17:38:04.489280 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489197, event: all_read, request: pg_notify(24.181(40) epoch 194486) v5 -6533> 2014-08-13 17:38:04.489283 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489266, event: dispatched, request: pg_notify(24.181(40) epoch 194486) v5 -6532> 2014-08-13 17:38:04.489287 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489287, event: waiting_for_osdmap, request: pg_notify(24.181(40) epoch 194486) v5 -6531> 2014-08-13 17:38:04.489290 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.307 -6530> 2014-08-13 17:38:04.489292 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5f03b80 -6529> 2014-08-13 17:38:04.489294 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489294, event: started, request: pg_notify(24.181(40) epoch 194486) v5 -6528> 2014-08-13 17:38:04.489286 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.307 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6527> 2014-08-13 17:38:04.489301 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 -> 4,55,296,307,320,344 -6526> 2014-08-13 17:38:04.489313 7fe99ebe8700 20 osd.4 194486 need_heartbeat_peer_update -6525> 2014-08-13 17:38:04.489318 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.123685 9 0.000691 -6524> 2014-08-13 17:38:04.489331 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog -6523> 2014-08-13 17:38:04.489346 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.4 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6522> 2014-08-13 17:38:04.489358 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) -6521> 2014-08-13 17:38:04.489370 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6520> 2014-08-13 17:38:04.489381 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6519> 2014-08-13 17:38:04.489392 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6518> 2014-08-13 17:38:04.489403 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) -6517> 2014-08-13 17:38:04.489436 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting newest update on osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) shard 344 (up) backfill 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) osd.296 (up) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) shard 320 (stray) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) -6516> 2014-08-13 17:38:04.489448 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] actingbackfill is 4,296,320,344 -6515> 2014-08-13 17:38:04.489459 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] choose_acting want [4,296,320] (== acting) backfill_targets 344 -6514> 2014-08-13 17:38:04.489471 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: leaving GetLog -6513> 2014-08-13 17:38:04.489485 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000154 0 0.000000 -6512> 2014-08-13 17:38:04.489498 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing -6511> 2014-08-13 17:38:04.489508 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.296 has no missing, identical log -6510> 2014-08-13 17:38:04.489519 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.320 has no missing, identical log -6509> 2014-08-13 17:38:04.489529 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.344 will fully backfill; can infer empty missing set -6508> 2014-08-13 17:38:04.489539 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: still need up_thru update before going active -6507> 2014-08-13 17:38:04.489550 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000052 0 0.000000 -6506> 2014-08-13 17:38:04.489562 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru -6505> 2014-08-13 17:38:04.489574 7fe99ebe8700 10 log is not dirty -6504> 2014-08-13 17:38:04.489597 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -6503> 2014-08-13 17:38:04.489641 7fe9a63f4700 5 -- op tracker -- , seq: 548, time: 2014-08-13 17:38:04.489641, event: done, request: pg_notify(24.181(40) epoch 194486) v5 -6502> 2014-08-13 17:38:04.489660 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6501> 2014-08-13 17:38:04.489662 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6500> 2014-08-13 17:38:04.489671 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.48 192.168.1.112:6838/8300 4 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4d92880 con 0x4ecf640 -6499> 2014-08-13 17:38:04.489682 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6498> 2014-08-13 17:38:04.489684 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6497> 2014-08-13 17:38:04.489685 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d92880 osd_map(194486..194486 src has 185139..194486) v3 -6496> 2014-08-13 17:38:04.489690 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -6495> 2014-08-13 17:38:04.489693 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -6494> 2014-08-13 17:38:04.489695 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6493> 2014-08-13 17:38:04.489696 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6492> 2014-08-13 17:38:04.489677 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 307 notify: (query_epoch:194486, epoch_sent:194486, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486)) -6491> 2014-08-13 17:38:04.489703 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.48 192.168.1.112:6838/8300 5 ==== pg_notify(22.619(31) epoch 194486) v5 ==== 2455+0+0 (4217602102 0 0) 0x51cd240 con 0x4ecf640 -6490> 2014-08-13 17:38:04.489703 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.307 -6489> 2014-08-13 17:38:04.489711 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6488> 2014-08-13 17:38:04.489724 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6487> 2014-08-13 17:38:04.489726 7fe9a63f4700 20 osd.4 194486 _dispatch 0x51cd240 pg_notify(22.619(31) epoch 194486) v5 -6486> 2014-08-13 17:38:04.489729 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489521, event: header_read, request: pg_notify(22.619(31) epoch 194486) v5 -6485> 2014-08-13 17:38:04.489737 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489523, event: throttled, request: pg_notify(22.619(31) epoch 194486) v5 -6484> 2014-08-13 17:38:04.489740 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489620, event: all_read, request: pg_notify(22.619(31) epoch 194486) v5 -6483> 2014-08-13 17:38:04.489744 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489710, event: dispatched, request: pg_notify(22.619(31) epoch 194486) v5 -6482> 2014-08-13 17:38:04.489721 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.307 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486), identical to ours -6481> 2014-08-13 17:38:04.489748 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489748, event: waiting_for_osdmap, request: pg_notify(22.619(31) epoch 194486) v5 -6480> 2014-08-13 17:38:04.489752 7fe99f5e9700 10 log is not dirty -6479> 2014-08-13 17:38:04.489753 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.48 -6478> 2014-08-13 17:38:04.489755 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x51cd240 -6477> 2014-08-13 17:38:04.489758 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489758, event: started, request: pg_notify(22.619(31) epoch 194486) v5 -6476> 2014-08-13 17:38:04.489761 7fe99f5e9700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -6475> 2014-08-13 17:38:04.489777 7fe9a63f4700 5 -- op tracker -- , seq: 549, time: 2014-08-13 17:38:04.489776, event: done, request: pg_notify(22.619(31) epoch 194486) v5 -6474> 2014-08-13 17:38:04.489762 7fe9aad09700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] flushed -6473> 2014-08-13 17:38:04.489784 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6472> 2014-08-13 17:38:04.489786 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6471> 2014-08-13 17:38:04.489808 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 48 notify: (query_epoch:194486, epoch_sent:194486, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -6470> 2014-08-13 17:38:04.489832 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring -6469> 2014-08-13 17:38:04.489843 7fe99ebe8700 10 log is not dirty -6468> 2014-08-13 17:38:04.489849 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 FlushedEvt -6467> 2014-08-13 17:38:04.489864 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] requeue_ops -6466> 2014-08-13 17:38:04.489875 7fe99ebe8700 10 log is not dirty -6465> 2014-08-13 17:38:04.489879 7fe99ebe8700 10 osd.4 194486 queue_want_up_thru want 194486 <= queued 194486, currently 194485 -6464> 2014-08-13 17:38:04.493676 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.287 192.168.1.109:6814/33435 4 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x4d957c0 con 0x4df3700 -6463> 2014-08-13 17:38:04.493717 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6462> 2014-08-13 17:38:04.493723 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6461> 2014-08-13 17:38:04.493725 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d957c0 osd_map(194486..194486 src has 185139..194486) v3 -6460> 2014-08-13 17:38:04.493735 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -6459> 2014-08-13 17:38:04.493740 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -6458> 2014-08-13 17:38:04.493743 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6457> 2014-08-13 17:38:04.493744 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6456> 2014-08-13 17:38:04.493774 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.287 192.168.1.109:6814/33435 5 ==== pg_notify(22.481(46) epoch 194486) v5 ==== 3296+0+0 (4074139918 0 0) 0x5823f00 con 0x4df3700 -6455> 2014-08-13 17:38:04.493790 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6454> 2014-08-13 17:38:04.493792 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6453> 2014-08-13 17:38:04.493794 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5823f00 pg_notify(22.481(46) epoch 194486) v5 -6452> 2014-08-13 17:38:04.493800 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493617, event: header_read, request: pg_notify(22.481(46) epoch 194486) v5 -6451> 2014-08-13 17:38:04.493811 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493618, event: throttled, request: pg_notify(22.481(46) epoch 194486) v5 -6450> 2014-08-13 17:38:04.493829 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493753, event: all_read, request: pg_notify(22.481(46) epoch 194486) v5 -6449> 2014-08-13 17:38:04.493832 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493789, event: dispatched, request: pg_notify(22.481(46) epoch 194486) v5 -6448> 2014-08-13 17:38:04.493836 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493836, event: waiting_for_osdmap, request: pg_notify(22.481(46) epoch 194486) v5 -6447> 2014-08-13 17:38:04.493840 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.287 -6446> 2014-08-13 17:38:04.493842 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5823f00 -6445> 2014-08-13 17:38:04.493845 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493845, event: started, request: pg_notify(22.481(46) epoch 194486) v5 -6444> 2014-08-13 17:38:04.493863 7fe9a63f4700 5 -- op tracker -- , seq: 550, time: 2014-08-13 17:38:04.493863, event: done, request: pg_notify(22.481(46) epoch 194486) v5 -6443> 2014-08-13 17:38:04.493872 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6442> 2014-08-13 17:38:04.493874 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6441> 2014-08-13 17:38:04.493873 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 287 notify: (query_epoch:194486, epoch_sent:194486, info:22.481( v 194215'519 (0'0,194215'519] local-les=194481 n=32 ec=183096 les/c 194481/194304 194485/194485/194485)) -6440> 2014-08-13 17:38:04.493897 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 287, already have info from that osd, ignoring -6439> 2014-08-13 17:38:04.493909 7fe99f5e9700 10 log is not dirty -6438> 2014-08-13 17:38:04.495376 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.287 192.168.1.109:6814/33435 6 ==== pg_info(1 pgs e194486:22.481) v4 ==== 714+0+0 (1566877661 0 0) 0x42655c0 con 0x4df3700 -6437> 2014-08-13 17:38:04.495416 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6436> 2014-08-13 17:38:04.495421 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6435> 2014-08-13 17:38:04.495424 7fe9a63f4700 20 osd.4 194486 _dispatch 0x42655c0 pg_info(1 pgs e194486:22.481) v4 -6434> 2014-08-13 17:38:04.495431 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495227, event: header_read, request: pg_info(1 pgs e194486:22.481) v4 -6433> 2014-08-13 17:38:04.495442 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495229, event: throttled, request: pg_info(1 pgs e194486:22.481) v4 -6432> 2014-08-13 17:38:04.495448 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495310, event: all_read, request: pg_info(1 pgs e194486:22.481) v4 -6431> 2014-08-13 17:38:04.495454 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495414, event: dispatched, request: pg_info(1 pgs e194486:22.481) v4 -6430> 2014-08-13 17:38:04.495461 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495461, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.481) v4 -6429> 2014-08-13 17:38:04.495482 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.481) v4 from osd.287 -6428> 2014-08-13 17:38:04.495487 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x42655c0 -6427> 2014-08-13 17:38:04.495493 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495493, event: started, request: pg_info(1 pgs e194486:22.481) v4 -6426> 2014-08-13 17:38:04.495523 7fe9a63f4700 5 -- op tracker -- , seq: 551, time: 2014-08-13 17:38:04.495523, event: done, request: pg_info(1 pgs e194486:22.481) v4 -6425> 2014-08-13 17:38:04.495547 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6424> 2014-08-13 17:38:04.495549 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6423> 2014-08-13 17:38:04.495572 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 287 info: 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) -6422> 2014-08-13 17:38:04.495596 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.287 activated and committed -6421> 2014-08-13 17:38:04.495607 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -6420> 2014-08-13 17:38:04.495616 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] share_pg_info -6419> 2014-08-13 17:38:04.495630 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4df3700 don't know epoch, doing nothing -6418> 2014-08-13 17:38:04.495633 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.287 192.168.1.109:6814/33435 -- pg_info(1 pgs e194486:22.481) v4 -- ?+0 0x5f02300 -6417> 2014-08-13 17:38:04.495652 7fe99ebe8700 20 osd.4 194486 _share_map_outgoing 0x4ecf0c0 don't know epoch, doing nothing -6416> 2014-08-13 17:38:04.495654 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.309 192.168.1.101:6814/23703 -- pg_info(1 pgs e194486:22.481) v4 -- ?+0 0x5f00000 -6415> 2014-08-13 17:38:04.495667 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194486:2682 -6414> 2014-08-13 17:38:04.495685 7fe99ebe8700 10 log is not dirty -6413> 2014-08-13 17:38:04.495854 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 AllReplicasActivated -6412> 2014-08-13 17:38:04.495901 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] check_local -6411> 2014-08-13 17:38:04.495918 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] requeue_ops -6410> 2014-08-13 17:38:04.495933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] needs_recovery osd.309 has 32 missing -6409> 2014-08-13 17:38:04.495949 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -6408> 2014-08-13 17:38:04.496114 7fe99ebe8700 15 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194486:2683 -6407> 2014-08-13 17:38:04.496135 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] hit_set_clear -6406> 2014-08-13 17:38:04.496150 7fe99ebe8700 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] agent_stop -6405> 2014-08-13 17:38:04.496166 7fe99ebe8700 10 log is not dirty -6404> 2014-08-13 17:38:04.496180 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 DoRecovery -6403> 2014-08-13 17:38:04.496213 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.102204 12 0.012402 -6402> 2014-08-13 17:38:04.496226 7fe99ebe8700 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -6401> 2014-08-13 17:38:04.496238 7fe99ebe8700 10 log is not dirty -6400> 2014-08-13 17:38:04.639462 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 14 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (789807499 0 0) 0x4d2d800 con 0x4d3c780 -6399> 2014-08-13 17:38:04.639509 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6398> 2014-08-13 17:38:04.639514 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6397> 2014-08-13 17:38:04.639516 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d2d800 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6396> 2014-08-13 17:38:04.639527 7fe9a63f4700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.639271, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6395> 2014-08-13 17:38:04.639542 7fe9a63f4700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.639275, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6394> 2014-08-13 17:38:04.639559 7fe9a63f4700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.639415, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6393> 2014-08-13 17:38:04.639581 7fe9a63f4700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.639507, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6392> 2014-08-13 17:38:04.639593 7fe9a63f4700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.639593, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6391> 2014-08-13 17:38:04.639605 7fe9a63f4700 10 osd.4 194486 handle_replica_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194486 -6390> 2014-08-13 17:38:04.649942 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4d2d800 -6389> 2014-08-13 17:38:04.649951 7fe9a63f4700 20 osd.4 194486 _share_map_incoming osd.337 192.168.1.104:6804/62197 194486 -6388> 2014-08-13 17:38:04.649969 7fe9a63f4700 15 osd.4 194486 enqueue_op 0x27cb660 prio 127 cost 0 latency 0.010697 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6387> 2014-08-13 17:38:04.650007 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6386> 2014-08-13 17:38:04.650010 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6385> 2014-08-13 17:38:04.650081 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27cb660 prio 127 cost 0 latency 0.010809 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] -6384> 2014-08-13 17:38:04.650128 7fe99f5e9700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.650127, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6383> 2014-08-13 17:38:04.650147 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_message: 0x27cb660 -6382> 2014-08-13 17:38:04.650193 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6381> 2014-08-13 17:38:04.650215 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] sub_op_scrub_reserve -6380> 2014-08-13 17:38:04.650231 7fe99f5e9700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.650231, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6379> 2014-08-13 17:38:04.650245 7fe99f5e9700 20 osd.4 194486 inc_scrubs_pending 0 -> 1 (max 1, active 0) -6378> 2014-08-13 17:38:04.650256 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- osd_sub_op_reply(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x47a8780 con 0x4d3c780 -6377> 2014-08-13 17:38:04.650285 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27cb660 finish -6376> 2014-08-13 17:38:04.650294 7fe99f5e9700 5 -- op tracker -- , seq: 552, time: 2014-08-13 17:38:04.650294, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6375> 2014-08-13 17:38:04.651145 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.304 192.168.100.101:0/22782 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.650931) v2 ==== 47+0+0 (2406972092 0 0) 0x51c8000 con 0x4ecd540 -6374> 2014-08-13 17:38:04.651146 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.304 192.168.100.101:0/22782 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.650931) v2 ==== 47+0+0 (2406972092 0 0) 0x513cec0 con 0x4ecdc20 -6373> 2014-08-13 17:38:04.651206 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.650931) v2 -- ?+0 0x5f03480 con 0x4ecd540 -6372> 2014-08-13 17:38:04.651283 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.304 now has 194486 -6371> 2014-08-13 17:38:04.651332 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 -6370> 2014-08-13 17:38:04.651375 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.650931) v2 -- ?+0 0x5f03640 con 0x4ecdc20 -6369> 2014-08-13 17:38:04.651425 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 -6368> 2014-08-13 17:38:04.657321 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.657214) v2 ==== 47+0+0 (3020041420 0 0) 0x51cb9c0 con 0x53874e0 -6367> 2014-08-13 17:38:04.657368 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.657214) v2 -- ?+0 0x513cec0 con 0x53874e0 -6366> 2014-08-13 17:38:04.657395 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.30 now has 194486 -6365> 2014-08-13 17:38:04.657408 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4a44780 already has epoch 194486 -6364> 2014-08-13 17:38:04.657412 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.30 192.168.100.107:0/55491 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.657214) v2 ==== 47+0+0 (3020041420 0 0) 0x51ccd00 con 0x53189a0 -6363> 2014-08-13 17:38:04.657456 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.107:0/55491 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.657214) v2 -- ?+0 0x51c8000 con 0x53189a0 -6362> 2014-08-13 17:38:04.657486 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4a44780 already has epoch 194486 -6361> 2014-08-13 17:38:04.664289 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.0 192.168.1.115:6810/59615 4 ==== osd_map(194486..194486 src has 185139..194486) v3 ==== 1194+0+0 (3547150570 0 0) 0x5490240 con 0x4ec8c60 -6360> 2014-08-13 17:38:04.664327 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6359> 2014-08-13 17:38:04.664332 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6358> 2014-08-13 17:38:04.664334 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5490240 osd_map(194486..194486 src has 185139..194486) v3 -6357> 2014-08-13 17:38:04.664345 7fe9a63f4700 3 osd.4 194486 handle_osd_map epochs [194486,194486], i have 194486, src has [185139,194486] -6356> 2014-08-13 17:38:04.664351 7fe9a63f4700 10 osd.4 194486 no new maps here, dropping -6355> 2014-08-13 17:38:04.664354 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6354> 2014-08-13 17:38:04.664355 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6353> 2014-08-13 17:38:04.664416 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.0 192.168.1.115:6810/59615 5 ==== pg_notify(22.775(31) epoch 194486) v5 ==== 2455+0+0 (3189431269 0 0) 0x5142d80 con 0x4ec8c60 -6352> 2014-08-13 17:38:04.664441 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6351> 2014-08-13 17:38:04.664444 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6350> 2014-08-13 17:38:04.664446 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5142d80 pg_notify(22.775(31) epoch 194486) v5 -6349> 2014-08-13 17:38:04.664453 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664261, event: header_read, request: pg_notify(22.775(31) epoch 194486) v5 -6348> 2014-08-13 17:38:04.664464 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664263, event: throttled, request: pg_notify(22.775(31) epoch 194486) v5 -6347> 2014-08-13 17:38:04.664470 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664381, event: all_read, request: pg_notify(22.775(31) epoch 194486) v5 -6346> 2014-08-13 17:38:04.664476 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664439, event: dispatched, request: pg_notify(22.775(31) epoch 194486) v5 -6345> 2014-08-13 17:38:04.664484 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664483, event: waiting_for_osdmap, request: pg_notify(22.775(31) epoch 194486) v5 -6344> 2014-08-13 17:38:04.664490 7fe9a63f4700 7 osd.4 194486 handle_pg_notify from osd.0 -6343> 2014-08-13 17:38:04.664494 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5142d80 -6342> 2014-08-13 17:38:04.664500 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664499, event: started, request: pg_notify(22.775(31) epoch 194486) v5 -6341> 2014-08-13 17:38:04.664539 7fe9a63f4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 17:38:04.664539, event: done, request: pg_notify(22.775(31) epoch 194486) v5 -6340> 2014-08-13 17:38:04.664555 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6339> 2014-08-13 17:38:04.664558 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6338> 2014-08-13 17:38:04.664558 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MNotifyRec from 0 notify: (query_epoch:194486, epoch_sent:194486, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -6337> 2014-08-13 17:38:04.664603 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring -6336> 2014-08-13 17:38:04.664624 7fe99ebe8700 10 log is not dirty -6335> 2014-08-13 17:38:04.669320 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 15 ==== replica scrub(pg: 22.4fe,from:0'0,to:194215'492,epoch:194486,start:0//0//-1,end:77e26cfe//0//-1,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2617556594 0 0) 0x4db60c0 con 0x4d3c780 -6334> 2014-08-13 17:38:04.669361 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6333> 2014-08-13 17:38:04.669366 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6332> 2014-08-13 17:38:04.669369 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4db60c0 replica scrub(pg: 22.4fe,from:0'0,to:194215'492,epoch:194486,start:0//0//-1,end:77e26cfe//0//-1,chunky:1,deep:0,version:5) v5 -6331> 2014-08-13 17:38:04.669383 7fe9a63f4700 10 osd.4 194486 queueing MOSDRepScrub replica scrub(pg: 22.4fe,from:0'0,to:194215'492,epoch:194486,start:0//0//-1,end:77e26cfe//0//-1,chunky:1,deep:0,version:5) v5 -6330> 2014-08-13 17:38:04.669395 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6329> 2014-08-13 17:38:04.669398 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6328> 2014-08-13 17:38:04.669425 7fe99d7e6700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] replica_scrub -6327> 2014-08-13 17:38:04.669481 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map -6326> 2014-08-13 17:38:04.669498 7fe99d7e6700 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] scrub_map_chunk [0//0//-1,77e26cfe//0//-1) -6325> 2014-08-13 17:38:04.670389 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] _scan_list scanning 25 objects -6324> 2014-08-13 17:38:04.676181 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.351 192.168.100.102:0/16444 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.676151) v2 ==== 47+0+0 (1763044977 0 0) 0x51cc440 con 0x40d8840 -6323> 2014-08-13 17:38:04.676204 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.351 192.168.100.102:0/16444 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.676151) v2 ==== 47+0+0 (1763044977 0 0) 0x51cbf00 con 0x4056ca0 -6322> 2014-08-13 17:38:04.676236 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.676151) v2 -- ?+0 0x51ccd00 con 0x40d8840 -6321> 2014-08-13 17:38:04.676259 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.351 now has 194486 -6320> 2014-08-13 17:38:04.676269 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x5250b00 already has epoch 194486 -6319> 2014-08-13 17:38:04.676288 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.102:0/16444 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.676151) v2 -- ?+0 0x51cb9c0 con 0x4056ca0 -6318> 2014-08-13 17:38:04.676317 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x5250b00 already has epoch 194486 -6317> 2014-08-13 17:38:04.696474 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.364 192.168.100.114:0/52317 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.696321) v2 ==== 47+0+0 (3764246626 0 0) 0x51ce040 con 0x582dd80 -6316> 2014-08-13 17:38:04.696519 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.696321) v2 -- ?+0 0x51cbf00 con 0x582dd80 -6315> 2014-08-13 17:38:04.696545 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.364 now has 194486 -6314> 2014-08-13 17:38:04.696557 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 already has epoch 194486 -6313> 2014-08-13 17:38:04.696535 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.364 192.168.100.114:0/52317 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.696321) v2 ==== 47+0+0 (3764246626 0 0) 0x51cdb00 con 0x582b860 -6312> 2014-08-13 17:38:04.696577 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.114:0/52317 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.696321) v2 -- ?+0 0x51cc440 con 0x582b860 -6311> 2014-08-13 17:38:04.696605 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4cfc620 already has epoch 194486 -6310> 2014-08-13 17:38:04.766103 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map_chunk done. -6309> 2014-08-13 17:38:04.766241 7fe99d7e6700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11537 0x5cd0000 con 0x4d3c780 -6308> 2014-08-13 17:38:04.794952 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 16 ==== replica scrub(pg: 22.4fe,from:0'0,to:194215'495,epoch:194486,start:77e26cfe//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 ==== 126+0+0 (2080110922 0 0) 0x4d90b40 con 0x4d3c780 -6307> 2014-08-13 17:38:04.794997 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6306> 2014-08-13 17:38:04.795002 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6305> 2014-08-13 17:38:04.795004 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4d90b40 replica scrub(pg: 22.4fe,from:0'0,to:194215'495,epoch:194486,start:77e26cfe//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 -6304> 2014-08-13 17:38:04.795013 7fe9a63f4700 10 osd.4 194486 queueing MOSDRepScrub replica scrub(pg: 22.4fe,from:0'0,to:194215'495,epoch:194486,start:77e26cfe//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 -6303> 2014-08-13 17:38:04.795024 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6302> 2014-08-13 17:38:04.795026 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6301> 2014-08-13 17:38:04.795080 7fe99d7e6700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] replica_scrub -6300> 2014-08-13 17:38:04.795120 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map -6299> 2014-08-13 17:38:04.795135 7fe99d7e6700 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] scrub_map_chunk [77e26cfe//0//-1,MAX) -6298> 2014-08-13 17:38:04.795864 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] _scan_list scanning 10 objects -6297> 2014-08-13 17:38:04.837718 7fe99d7e6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] build_scrub_map_chunk done. -6296> 2014-08-13 17:38:04.837801 7fe99d7e6700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+4646 0x5cd1600 con 0x4d3c780 -6295> 2014-08-13 17:38:04.841522 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 17 ==== osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3192017454 0 0) 0x5cd1600 con 0x4d3c780 -6294> 2014-08-13 17:38:04.841572 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6293> 2014-08-13 17:38:04.841578 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6292> 2014-08-13 17:38:04.841580 7fe9a63f4700 20 osd.4 194486 _dispatch 0x5cd1600 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6291> 2014-08-13 17:38:04.841591 7fe9a63f4700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841292, event: header_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6290> 2014-08-13 17:38:04.841607 7fe9a63f4700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841294, event: throttled, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6289> 2014-08-13 17:38:04.841617 7fe9a63f4700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841438, event: all_read, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6288> 2014-08-13 17:38:04.841627 7fe9a63f4700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841570, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6287> 2014-08-13 17:38:04.841638 7fe9a63f4700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841637, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6286> 2014-08-13 17:38:04.841648 7fe9a63f4700 10 osd.4 194486 handle_replica_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194486 -6285> 2014-08-13 17:38:04.841656 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x5cd1600 -6284> 2014-08-13 17:38:04.841661 7fe9a63f4700 20 osd.4 194486 _share_map_incoming osd.337 192.168.1.104:6804/62197 194486 -6283> 2014-08-13 17:38:04.841671 7fe9a63f4700 15 osd.4 194486 enqueue_op 0x27ce450 prio 127 cost 0 latency 0.000379 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6282> 2014-08-13 17:38:04.841697 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6281> 2014-08-13 17:38:04.841700 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6280> 2014-08-13 17:38:04.841707 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 18 ==== pg_info(1 pgs e194486:22.4fe) v4 ==== 714+0+0 (3897543659 0 0) 0x4260380 con 0x4d3c780 -6279> 2014-08-13 17:38:04.841720 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6278> 2014-08-13 17:38:04.841722 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6277> 2014-08-13 17:38:04.841724 7fe9a63f4700 20 osd.4 194486 _dispatch 0x4260380 pg_info(1 pgs e194486:22.4fe) v4 -6276> 2014-08-13 17:38:04.841729 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841462, event: header_read, request: pg_info(1 pgs e194486:22.4fe) v4 -6275> 2014-08-13 17:38:04.841736 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841464, event: throttled, request: pg_info(1 pgs e194486:22.4fe) v4 -6274> 2014-08-13 17:38:04.841742 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841535, event: all_read, request: pg_info(1 pgs e194486:22.4fe) v4 -6273> 2014-08-13 17:38:04.841747 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841720, event: dispatched, request: pg_info(1 pgs e194486:22.4fe) v4 -6272> 2014-08-13 17:38:04.841753 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841753, event: waiting_for_osdmap, request: pg_info(1 pgs e194486:22.4fe) v4 -6271> 2014-08-13 17:38:04.841759 7fe9a63f4700 7 osd.4 194486 handle_pg_info pg_info(1 pgs e194486:22.4fe) v4 from osd.337 -6270> 2014-08-13 17:38:04.841764 7fe9a63f4700 15 osd.4 194486 require_same_or_newer_map 194486 (i am 194486) 0x4260380 -6269> 2014-08-13 17:38:04.841768 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841768, event: started, request: pg_info(1 pgs e194486:22.4fe) v4 -6268> 2014-08-13 17:38:04.841767 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27ce450 prio 127 cost 0 latency 0.000474 osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] -6267> 2014-08-13 17:38:04.841813 7fe99f5e9700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841812, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6266> 2014-08-13 17:38:04.841838 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_message: 0x27ce450 -6265> 2014-08-13 17:38:04.841859 7fe99f5e9700 15 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6264> 2014-08-13 17:38:04.841879 7fe99f5e9700 7 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] sub_op_scrub_unreserve -6263> 2014-08-13 17:38:04.841894 7fe99f5e9700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841894, event: started, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6262> 2014-08-13 17:38:04.841907 7fe99f5e9700 20 osd.4 194486 dec_scrubs_pending 1 -> 0 (max 1, active 0) -6261> 2014-08-13 17:38:04.841911 7fe99f5e9700 10 osd.4 194486 dequeue_op 0x27ce450 finish -6260> 2014-08-13 17:38:04.841921 7fe99f5e9700 5 -- op tracker -- , seq: 554, time: 2014-08-13 17:38:04.841921, event: done, request: osd_sub_op(unknown.0.0:0 22.4fe 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -6259> 2014-08-13 17:38:04.841983 7fe9a63f4700 5 -- op tracker -- , seq: 555, time: 2014-08-13 17:38:04.841982, event: done, request: pg_info(1 pgs e194486:22.4fe) v4 -6258> 2014-08-13 17:38:04.842015 7fe9a63f4700 10 osd.4 194486 do_waiters -- start -6257> 2014-08-13 17:38:04.842019 7fe9a63f4700 10 osd.4 194486 do_waiters -- finish -6256> 2014-08-13 17:38:04.842035 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194486 epoch_requested: 194486 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) -6255> 2014-08-13 17:38:04.842119 7fe99ebe8700 10 log is not dirty -6254> 2014-08-13 17:38:04.932915 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.334 192.168.1.104:0/62751 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.933029) v2 ==== 47+0+0 (116869171 0 0) 0x4c17540 con 0x581e9e0 -6253> 2014-08-13 17:38:04.932975 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.933029) v2 -- ?+0 0x51cdb00 con 0x581e9e0 -6252> 2014-08-13 17:38:04.932999 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.334 now has 194486 -6251> 2014-08-13 17:38:04.933010 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 already has epoch 194486 -6250> 2014-08-13 17:38:04.932916 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.334 192.168.1.104:0/62751 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.933029) v2 ==== 47+0+0 (116869171 0 0) 0x4c16ac0 con 0x581dac0 -6249> 2014-08-13 17:38:04.933099 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.104:0/62751 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.933029) v2 -- ?+0 0x51ce040 con 0x581dac0 -6248> 2014-08-13 17:38:04.933138 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e49ce0 already has epoch 194486 -6247> 2014-08-13 17:38:04.954318 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.954134) v2 ==== 47+0+0 (2114303985 0 0) 0x5106040 con 0x582bde0 -6246> 2014-08-13 17:38:04.954370 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.954134) v2 -- ?+0 0x4c17540 con 0x582bde0 -6245> 2014-08-13 17:38:04.954403 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.51 now has 194486 -6244> 2014-08-13 17:38:04.954414 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 already has epoch 194486 -6243> 2014-08-13 17:38:04.954404 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.51 192.168.1.112:0/6655 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.954134) v2 ==== 47+0+0 (2114303985 0 0) 0x5104980 con 0x581d6a0 -6242> 2014-08-13 17:38:04.954447 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.954134) v2 -- ?+0 0x4c16ac0 con 0x581d6a0 -6241> 2014-08-13 17:38:04.954473 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e39e40 already has epoch 194486 -6240> 2014-08-13 17:38:04.967081 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.31 192.168.100.103:0/41157 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.966810) v2 ==== 47+0+0 (1064332372 0 0) 0x5104d00 con 0x4ecdee0 -6239> 2014-08-13 17:38:04.967126 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.966810) v2 -- ?+0 0x5104980 con 0x4ecdee0 -6238> 2014-08-13 17:38:04.967108 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.31 192.168.100.103:0/41157 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.966810) v2 ==== 47+0+0 (1064332372 0 0) 0x5102a00 con 0x4ecd800 -6237> 2014-08-13 17:38:04.967175 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.31 now has 194486 -6236> 2014-08-13 17:38:04.967189 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e5960 already has epoch 194486 -6235> 2014-08-13 17:38:04.967569 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.103:0/41157 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.966810) v2 -- ?+0 0x5106040 con 0x4ecd800 -6234> 2014-08-13 17:38:04.967604 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e5960 already has epoch 194486 -6233> 2014-08-13 17:38:04.979942 7fe977f9a700 2 -- 192.168.1.115:6817/34805 >> 192.168.100.115:0/58522 pipe(0x47ab980 sd=377 :6817 s=2 pgs=32571 cs=1 l=1 c=0x581d800).reader couldn't read tag, (0) Success -6232> 2014-08-13 17:38:04.980012 7fe977f9a700 2 -- 192.168.1.115:6817/34805 >> 192.168.100.115:0/58522 pipe(0x47ab980 sd=377 :6817 s=2 pgs=32571 cs=1 l=1 c=0x581d800).fault (0) Success -6231> 2014-08-13 17:38:04.980019 7fe977e99700 2 -- 192.168.100.115:6806/34805 >> 192.168.100.115:0/58522 pipe(0x47aa080 sd=378 :6806 s=2 pgs=32572 cs=1 l=1 c=0x582c0a0).reader couldn't read tag, (0) Success -6230> 2014-08-13 17:38:04.980075 7fe977e99700 2 -- 192.168.100.115:6806/34805 >> 192.168.100.115:0/58522 pipe(0x47aa080 sd=378 :6806 s=2 pgs=32572 cs=1 l=1 c=0x582c0a0).fault (0) Success -6229> 2014-08-13 17:38:04.991966 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.397 192.168.1.105:0/41085 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.991467) v2 ==== 47+0+0 (2564321966 0 0) 0x5104600 con 0x582dac0 -6228> 2014-08-13 17:38:04.992013 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.991467) v2 -- ?+0 0x5104d00 con 0x582dac0 -6227> 2014-08-13 17:38:04.992034 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.397 now has 194486 -6226> 2014-08-13 17:38:04.992004 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.397 192.168.1.105:0/41085 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.991467) v2 ==== 47+0+0 (2564321966 0 0) 0x5100000 con 0x57c0000 -6225> 2014-08-13 17:38:04.992044 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e48840 already has epoch 194486 -6224> 2014-08-13 17:38:04.992063 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.105:0/41085 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.991467) v2 -- ?+0 0x5102a00 con 0x57c0000 -6223> 2014-08-13 17:38:04.992090 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e48840 already has epoch 194486 -6222> 2014-08-13 17:38:04.995902 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.376 192.168.1.108:0/6893 2 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.995345) v2 ==== 47+0+0 (3067035315 0 0) 0x5107380 con 0x4ecfd20 -6221> 2014-08-13 17:38:04.995930 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.376 192.168.1.108:0/6893 2 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:04.995345) v2 ==== 47+0+0 (3067035315 0 0) 0x5106580 con 0x4eceb40 -6220> 2014-08-13 17:38:04.995954 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.995345) v2 -- ?+0 0x5100000 con 0x4ecfd20 -6219> 2014-08-13 17:38:04.995983 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -6218> 2014-08-13 17:38:04.996004 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.108:0/6893 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:04.995345) v2 -- ?+0 0x5104600 con 0x4eceb40 -6217> 2014-08-13 17:38:04.996041 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4e4d280 already has epoch 194486 -6216> 2014-08-13 17:38:05.017932 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.017887) v2 ==== 47+0+0 (409945444 0 0) 0x51ce580 con 0x52a06e0 -6215> 2014-08-13 17:38:05.017980 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.017887) v2 -- ?+0 0x5106580 con 0x52a06e0 -6214> 2014-08-13 17:38:05.018013 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.358 now has 194486 -6213> 2014-08-13 17:38:05.018023 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 already has epoch 194486 -6212> 2014-08-13 17:38:05.018066 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.017887) v2 ==== 47+0+0 (409945444 0 0) 0x51cbd40 con 0x51a86e0 -6211> 2014-08-13 17:38:05.018099 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.017887) v2 -- ?+0 0x5107380 con 0x51a86e0 -6210> 2014-08-13 17:38:05.018124 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d7c8e0 already has epoch 194486 -6209> 2014-08-13 17:38:05.148088 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.330 192.168.1.106:0/4876 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.147693) v2 ==== 47+0+0 (2457737832 0 0) 0x5100000 con 0x4ecc0a0 -6208> 2014-08-13 17:38:05.148104 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.330 192.168.1.106:0/4876 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.147693) v2 ==== 47+0+0 (2457737832 0 0) 0x5104600 con 0x4ecc620 -6207> 2014-08-13 17:38:05.148138 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.147693) v2 -- ?+0 0x51ce580 con 0x4ecc0a0 -6206> 2014-08-13 17:38:05.148168 7fe9a31ef700 10 osd.4 194486 note_peer_epoch osd.330 now has 194486 -6205> 2014-08-13 17:38:05.148182 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 already has epoch 194486 -6204> 2014-08-13 17:38:05.148239 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.106:0/4876 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.147693) v2 -- ?+0 0x51cbd40 con 0x4ecc620 -6203> 2014-08-13 17:38:05.148285 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4ec9b80 already has epoch 194486 -6202> 2014-08-13 17:38:05.153135 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.304 192.168.100.101:0/22782 2 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.152900) v2 ==== 47+0+0 (4234907905 0 0) 0x5105b00 con 0x4ecdc20 -6201> 2014-08-13 17:38:05.153163 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.304 192.168.100.101:0/22782 2 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.152900) v2 ==== 47+0+0 (4234907905 0 0) 0x5102300 con 0x4ecd540 -6200> 2014-08-13 17:38:05.153186 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.152900) v2 -- ?+0 0x5100000 con 0x4ecdc20 -6199> 2014-08-13 17:38:05.153225 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 -6198> 2014-08-13 17:38:05.153362 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.100.101:0/22782 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.152900) v2 -- ?+0 0x5104600 con 0x4ecd540 -6197> 2014-08-13 17:38:05.153411 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x49e39c0 already has epoch 194486 -6196> 2014-08-13 17:38:05.155078 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.19 192.168.1.115:0/61186 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.154904) v2 ==== 47+0+0 (313736602 0 0) 0x5104600 con 0x57c0b00 -6195> 2014-08-13 17:38:05.155119 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.115:0/61186 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.154904) v2 -- ?+0 0x5102300 con 0x57c0b00 -6194> 2014-08-13 17:38:05.155142 7fe9a45f1700 10 osd.4 194486 note_peer_epoch osd.19 now has 194486 -6193> 2014-08-13 17:38:05.155121 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.19 192.168.1.115:0/61186 1 ==== osd_ping(ping e194486 stamp 2014-08-13 17:38:05.154904) v2 ==== 47+0+0 (313736602 0 0) 0x5100000 con 0x53869e0 -6192> 2014-08-13 17:38:05.155155 7fe9a45f1700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 already has epoch 194486 -6191> 2014-08-13 17:38:05.155166 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.115:0/61186 -- osd_ping(ping_reply e194486 stamp 2014-08-13 17:38:05.154904) v2 -- ?+0 0x5105b00 con 0x53869e0 -6190> 2014-08-13 17:38:05.155191 7fe9a31ef700 20 osd.4 194486 _share_map_outgoing 0x4d7b020 already has epoch 194486 -6189> 2014-08-13 17:38:05.313923 7fe9a77f6700 1 -- 192.168.100.115:6805/34805 <== mon.1 192.168.100.112:6789/0 18 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5485e80 con 0x49dfd20 -6188> 2014-08-13 17:38:05.313975 7fe9a77f6700 10 osd.4 194486 do_waiters -- start -6187> 2014-08-13 17:38:05.313980 7fe9a77f6700 10 osd.4 194486 do_waiters -- finish -6186> 2014-08-13 17:38:05.313982 7fe9a77f6700 20 osd.4 194486 _dispatch 0x5485e80 osd_map(194487..194487 src has 185139..194487) v3 -6185> 2014-08-13 17:38:05.314025 7fe9a77f6700 3 osd.4 194486 handle_osd_map epochs [194487,194487], i have 194486, src has [185139,194487] -6184> 2014-08-13 17:38:05.314033 7fe9a77f6700 10 osd.4 194486 handle_osd_map got inc map for epoch 194487 -6183> 2014-08-13 17:38:05.318488 7fe9a77f6700 10 osd.4 194486 advance to epoch 194487 (<= newest 194487) -6182> 2014-08-13 17:38:05.318545 7fe9a77f6700 7 osd.4 194487 advance_map epoch 194487 74 pgs -6181> 2014-08-13 17:38:05.318565 7fe9a77f6700 10 osd.4 194487 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194487 [185139,194487] lci=[194485,194487]) -6180> 2014-08-13 17:38:05.318617 7fe9a77f6700 0 osd.4 194487 crush map has features 2578087936000, adjusting msgr requires for mons -6179> 2014-08-13 17:38:05.318624 7fe9a77f6700 7 osd.4 194487 consume_map version 194487 -6178> 2014-08-13 17:38:05.318667 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] null -6177> 2014-08-13 17:38:05.318701 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] null -6176> 2014-08-13 17:38:05.318718 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6175> 2014-08-13 17:38:05.318731 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] null -6174> 2014-08-13 17:38:05.318745 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] null -6173> 2014-08-13 17:38:05.318759 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6172> 2014-08-13 17:38:05.318771 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6171> 2014-08-13 17:38:05.318768 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 -6170> 2014-08-13 17:38:05.318786 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] null -6169> 2014-08-13 17:38:05.318775 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 -6168> 2014-08-13 17:38:05.318802 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6167> 2014-08-13 17:38:05.318802 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6166> 2014-08-13 17:38:05.318799 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] null -6165> 2014-08-13 17:38:05.318810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap -6164> 2014-08-13 17:38:05.318810 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Peering advmap -6163> 2014-08-13 17:38:05.318815 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] null -6162> 2014-08-13 17:38:05.318825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down -6161> 2014-08-13 17:38:05.318825 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] adjust_need_up_thru now 194486, need_up_thru now false -6160> 2014-08-13 17:38:05.318827 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6159> 2014-08-13 17:38:05.318839 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map -6158> 2014-08-13 17:38:05.318840 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] null -6157> 2014-08-13 17:38:05.318839 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Started advmap -6156> 2014-08-13 17:38:05.318851 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters -6155> 2014-08-13 17:38:05.318854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] check_recovery_sources no source osds () went down -6154> 2014-08-13 17:38:05.318848 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.61 192.168.1.111:6840/16566 13 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db5340 con 0x4cf86e0 -6153> 2014-08-13 17:38:05.318854 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] null -6152> 2014-08-13 17:38:05.318862 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -6151> 2014-08-13 17:38:05.318868 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map -6150> 2014-08-13 17:38:05.318870 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] null -6149> 2014-08-13 17:38:05.318875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194485 pi=192972-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -6148> 2014-08-13 17:38:05.318888 7fe99ebe8700 10 log is not dirty -6147> 2014-08-13 17:38:05.318883 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6146> 2014-08-13 17:38:05.318882 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle ActMap primary -6145> 2014-08-13 17:38:05.318896 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] null -6144> 2014-08-13 17:38:05.318897 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194487:30296 -6143> 2014-08-13 17:38:05.318908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] take_waiters -6142> 2014-08-13 17:38:05.318921 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 0.905872 4 0.000671 -6141> 2014-08-13 17:38:05.318924 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null -6140> 2014-08-13 17:38:05.318932 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 -6139> 2014-08-13 17:38:05.318936 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Leaving Peering -6138> 2014-08-13 17:38:05.318938 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6137> 2014-08-13 17:38:05.318946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6136> 2014-08-13 17:38:05.318947 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 1.039531 0 0.000000 -6135> 2014-08-13 17:38:05.318953 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap -6134> 2014-08-13 17:38:05.318952 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null -6133> 2014-08-13 17:38:05.318965 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down -6132> 2014-08-13 17:38:05.318965 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active -6131> 2014-08-13 17:38:05.318966 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null -6130> 2014-08-13 17:38:05.318976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map -6129> 2014-08-13 17:38:05.318977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: In Active, about to call activate -6128> 2014-08-13 17:38:05.318981 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null -6127> 2014-08-13 17:38:05.318986 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters -6126> 2014-08-13 17:38:05.318994 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - purged_snaps [] cached_removed_snaps [] -6125> 2014-08-13 17:38:05.318997 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -6124> 2014-08-13 17:38:05.318993 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] null -6123> 2014-08-13 17:38:05.319007 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -6122> 2014-08-13 17:38:05.319006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - snap_trimq [] -6121> 2014-08-13 17:38:05.319019 7fe99ebe8700 10 log is not dirty -6120> 2014-08-13 17:38:05.319011 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6119> 2014-08-13 17:38:05.319019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - not complete, missing(56) -6118> 2014-08-13 17:38:05.319025 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] null -6117> 2014-08-13 17:38:05.319033 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 -6116> 2014-08-13 17:38:05.319037 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6115> 2014-08-13 17:38:05.319046 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6114> 2014-08-13 17:38:05.319049 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6113> 2014-08-13 17:38:05.319051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -6112> 2014-08-13 17:38:05.319061 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] null -6111> 2014-08-13 17:38:05.319062 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -6110> 2014-08-13 17:38:05.319073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,324) went down -6109> 2014-08-13 17:38:05.319076 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null -6108> 2014-08-13 17:38:05.319084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -6107> 2014-08-13 17:38:05.319090 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] null -6106> 2014-08-13 17:38:05.319094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -6105> 2014-08-13 17:38:05.319103 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6104> 2014-08-13 17:38:05.319114 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] null -6103> 2014-08-13 17:38:05.319125 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] null -6102> 2014-08-13 17:38:05.319139 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] null -6101> 2014-08-13 17:38:05.319150 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6100> 2014-08-13 17:38:05.319161 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null -6099> 2014-08-13 17:38:05.319172 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] null -6098> 2014-08-13 17:38:05.319183 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6097> 2014-08-13 17:38:05.319199 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null -6096> 2014-08-13 17:38:05.319211 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6095> 2014-08-13 17:38:05.319222 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] null -6094> 2014-08-13 17:38:05.319233 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] null -6093> 2014-08-13 17:38:05.319248 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null -6092> 2014-08-13 17:38:05.319261 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] null -6091> 2014-08-13 17:38:05.319272 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null -6090> 2014-08-13 17:38:05.319284 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] null -6089> 2014-08-13 17:38:05.319297 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] null -6088> 2014-08-13 17:38:05.319309 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] null -6087> 2014-08-13 17:38:05.319320 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] null -6086> 2014-08-13 17:38:05.319331 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] null -6085> 2014-08-13 17:38:05.319338 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] -6084> 2014-08-13 17:38:05.319347 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] null -6083> 2014-08-13 17:38:05.319348 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -6082> 2014-08-13 17:38:05.319359 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] null -6081> 2014-08-13 17:38:05.319370 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] null -6080> 2014-08-13 17:38:05.319381 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] null -6079> 2014-08-13 17:38:05.319393 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194478 n=59 ec=192688 les/c 194484/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194486 pi=193264-194485/35 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] null -6078> 2014-08-13 17:38:05.319405 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194478 n=50 ec=192688 les/c 194484/194372 194485/194486/192978) [31,338,4] r=2 lpr=194486 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null -6077> 2014-08-13 17:38:05.319416 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null -6076> 2014-08-13 17:38:05.319428 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] null -6075> 2014-08-13 17:38:05.319439 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6074> 2014-08-13 17:38:05.319450 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6073> 2014-08-13 17:38:05.319461 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6072> 2014-08-13 17:38:05.319465 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2832 -6071> 2014-08-13 17:38:05.319475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -6070> 2014-08-13 17:38:05.319476 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] null -6069> 2014-08-13 17:38:05.319484 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -6068> 2014-08-13 17:38:05.319489 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null -6067> 2014-08-13 17:38:05.319494 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -6066> 2014-08-13 17:38:05.319505 7fe99ebe8700 10 log is not dirty -6065> 2014-08-13 17:38:05.319501 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] null -6064> 2014-08-13 17:38:05.319514 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null -6063> 2014-08-13 17:38:05.319520 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 -6062> 2014-08-13 17:38:05.319533 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6061> 2014-08-13 17:38:05.319527 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] null -6060> 2014-08-13 17:38:05.319536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -6059> 2014-08-13 17:38:05.319539 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] null -6058> 2014-08-13 17:38:05.319547 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -6057> 2014-08-13 17:38:05.319551 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] null -6056> 2014-08-13 17:38:05.319558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -6055> 2014-08-13 17:38:05.319564 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] null -6054> 2014-08-13 17:38:05.319569 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -6053> 2014-08-13 17:38:05.319576 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null -6052> 2014-08-13 17:38:05.319579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -6051> 2014-08-13 17:38:05.319588 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] null -6050> 2014-08-13 17:38:05.319601 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6049> 2014-08-13 17:38:05.319612 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] null -6048> 2014-08-13 17:38:05.319624 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] null -6047> 2014-08-13 17:38:05.319635 7fe9a77f6700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null -6046> 2014-08-13 17:38:05.319647 7fe9a77f6700 10 osd.4 194487 maybe_update_heartbeat_peers updating -6045> 2014-08-13 17:38:05.319649 7fe9a77f6700 20 osd.4 194487 24.30b heartbeat_peers 4,10,286,301,375 -6044> 2014-08-13 17:38:05.319654 7fe9a77f6700 20 osd.4 194487 22.220 heartbeat_peers -6043> 2014-08-13 17:38:05.319657 7fe9a77f6700 20 osd.4 194487 22.362 heartbeat_peers 4,58,61,343,349,362 -6042> 2014-08-13 17:38:05.319663 7fe9a77f6700 20 osd.4 194487 22.28f heartbeat_peers -6041> 2014-08-13 17:38:05.319664 7fe9a77f6700 20 osd.4 194487 22.14a heartbeat_peers -6040> 2014-08-13 17:38:05.319665 7fe9a77f6700 20 osd.4 194487 22.22c heartbeat_peers 4,301,302,324,368 -6039> 2014-08-13 17:38:05.319669 7fe9a77f6700 20 osd.4 194487 22.619 heartbeat_peers 4,44,48,62,65,314 -6038> 2014-08-13 17:38:05.319673 7fe9a77f6700 20 osd.4 194487 22.30f heartbeat_peers -6037> 2014-08-13 17:38:05.319674 7fe9a77f6700 20 osd.4 194487 24.9b heartbeat_peers 4,291,350,365,367,378 -6036> 2014-08-13 17:38:05.319679 7fe9a77f6700 20 osd.4 194487 22.581 heartbeat_peers -6035> 2014-08-13 17:38:05.319680 7fe9a77f6700 20 osd.4 194487 22.f3 heartbeat_peers 4,285,311,312,369 -6034> 2014-08-13 17:38:05.319683 7fe9a77f6700 20 osd.4 194487 22.116 heartbeat_peers -6033> 2014-08-13 17:38:05.319684 7fe9a77f6700 20 osd.4 194487 24.3d heartbeat_peers 4,20,25,303,362,373 -6032> 2014-08-13 17:38:05.319689 7fe9a77f6700 20 osd.4 194487 22.4aa heartbeat_peers -6031> 2014-08-13 17:38:05.319690 7fe9a77f6700 20 osd.4 194487 22.1d2 heartbeat_peers 4,58,61,330,331,349 -6030> 2014-08-13 17:38:05.319693 7fe9a77f6700 20 osd.4 194487 24.1de heartbeat_peers -6029> 2014-08-13 17:38:05.319695 7fe9a77f6700 20 osd.4 194487 2.2a heartbeat_peers 4,340 -6028> 2014-08-13 17:38:05.319697 7fe9a77f6700 20 osd.4 194487 22.113 heartbeat_peers 4,24,45,55,339,342 -6027> 2014-08-13 17:38:05.319701 7fe9a77f6700 20 osd.4 194487 24.39a heartbeat_peers -6026> 2014-08-13 17:38:05.319702 7fe9a77f6700 20 osd.4 194487 24.33a heartbeat_peers -6025> 2014-08-13 17:38:05.319703 7fe9a77f6700 20 osd.4 194487 1.2b heartbeat_peers 4,340 -6024> 2014-08-13 17:38:05.319705 7fe9a77f6700 20 osd.4 194487 24.34 heartbeat_peers 4,26,30,340 -6023> 2014-08-13 17:38:05.319755 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] -6022> 2014-08-13 17:38:05.319757 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.26 192.168.1.115:6803/58522 192.168.100.115:6804/58522 -6021> 2014-08-13 17:38:05.319763 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -6020> 2014-08-13 17:38:05.319789 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.30 192.168.1.107:6824/55491 192.168.100.107:6841/55491 -6019> 2014-08-13 17:38:05.319797 7fe9a77f6700 20 osd.4 194487 22.99 heartbeat_peers 4,20,31,337,363,374 -6018> 2014-08-13 17:38:05.319801 7fe9a77f6700 20 osd.4 194487 22.84 heartbeat_peers 4,28,304 -6017> 2014-08-13 17:38:05.319803 7fe9a77f6700 20 osd.4 194487 22.4df heartbeat_peers 4,352,356,357,371,380 -6016> 2014-08-13 17:38:05.319807 7fe9a77f6700 20 osd.4 194487 22.39d heartbeat_peers 4,320,344,363,366,373 -6015> 2014-08-13 17:38:05.319810 7fe9a77f6700 20 osd.4 194487 22.2b9 heartbeat_peers -6014> 2014-08-13 17:38:05.319812 7fe9a77f6700 20 osd.4 194487 24.2c heartbeat_peers -6013> 2014-08-13 17:38:05.319813 7fe9a77f6700 20 osd.4 194487 24.3f7 heartbeat_peers -6012> 2014-08-13 17:38:05.319816 7fe9a77f6700 20 osd.4 194487 22.775 heartbeat_peers 0,4,56,63,353,369 -6011> 2014-08-13 17:38:05.319819 7fe9a77f6700 20 osd.4 194487 22.630 heartbeat_peers -6010> 2014-08-13 17:38:05.319821 7fe9a77f6700 20 osd.4 194487 24.e2 heartbeat_peers -6009> 2014-08-13 17:38:05.319822 7fe9a77f6700 20 osd.4 194487 22.69f heartbeat_peers -6008> 2014-08-13 17:38:05.319823 7fe9a77f6700 20 osd.4 194487 22.3e4 heartbeat_peers 4,45,303,313,318,342 -6007> 2014-08-13 17:38:05.319826 7fe9a77f6700 20 osd.4 194487 24.2a1 heartbeat_peers -6006> 2014-08-13 17:38:05.319828 7fe9a77f6700 20 osd.4 194487 22.639 heartbeat_peers -6005> 2014-08-13 17:38:05.319829 7fe9a77f6700 20 osd.4 194487 22.707 heartbeat_peers 4,15,21,303,310,342 -6004> 2014-08-13 17:38:05.319832 7fe9a77f6700 20 osd.4 194487 24.5e heartbeat_peers -6003> 2014-08-13 17:38:05.319833 7fe9a77f6700 20 osd.4 194487 22.1f1 heartbeat_peers 4,5,11,47,54,368 -6002> 2014-08-13 17:38:05.319836 7fe9a77f6700 20 osd.4 194487 22.2d1 heartbeat_peers -6001> 2014-08-13 17:38:05.319838 7fe9a77f6700 20 osd.4 194487 24.b6 heartbeat_peers 4,17,28,315,327,330 -6000> 2014-08-13 17:38:05.319841 7fe9a77f6700 20 osd.4 194487 24.3bf heartbeat_peers -5999> 2014-08-13 17:38:05.319843 7fe9a77f6700 20 osd.4 194487 22.1fb heartbeat_peers -5998> 2014-08-13 17:38:05.319844 7fe9a77f6700 20 osd.4 194487 24.350 heartbeat_peers -5997> 2014-08-13 17:38:05.319845 7fe9a77f6700 20 osd.4 194487 24.1f7 heartbeat_peers 4,55,59,351,376 -5996> 2014-08-13 17:38:05.319849 7fe9a77f6700 20 osd.4 194487 24.136 heartbeat_peers 4,50,55,61,65,356 -5995> 2014-08-13 17:38:05.319853 7fe9a77f6700 20 osd.4 194487 22.4fe heartbeat_peers -5994> 2014-08-13 17:38:05.319854 7fe9a77f6700 20 osd.4 194487 22.653 heartbeat_peers -5993> 2014-08-13 17:38:05.319856 7fe9a77f6700 20 osd.4 194487 24.3b7 heartbeat_peers -5992> 2014-08-13 17:38:05.319857 7fe9a77f6700 20 osd.4 194487 22.3b9 heartbeat_peers -5991> 2014-08-13 17:38:05.319858 7fe9a77f6700 20 osd.4 194487 22.791 heartbeat_peers -5990> 2014-08-13 17:38:05.319852 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2735 -5989> 2014-08-13 17:38:05.319860 7fe9a77f6700 20 osd.4 194487 22.6bd heartbeat_peers 4,50,289 -5988> 2014-08-13 17:38:05.319862 7fe9a77f6700 20 osd.4 194487 24.181 heartbeat_peers 4,55,296,307,320,344 -5987> 2014-08-13 17:38:05.319861 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -5986> 2014-08-13 17:38:05.319870 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -5985> 2014-08-13 17:38:05.319893 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -5984> 2014-08-13 17:38:05.319904 7fe99ebe8700 10 log is not dirty -5983> 2014-08-13 17:38:05.319942 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.296 192.168.1.109:6807/31286 192.168.100.109:6815/31286 -5982> 2014-08-13 17:38:05.320002 7fe9a77f6700 10 osd.4 194487 _add_heartbeat_peer: new peer osd.307 192.168.1.109:6805/31016 192.168.100.109:6811/31016 -5981> 2014-08-13 17:38:05.320013 7fe9a77f6700 20 osd.4 194487 24.20e heartbeat_peers -5980> 2014-08-13 17:38:05.320014 7fe9a77f6700 20 osd.4 194487 24.1e4 heartbeat_peers -5979> 2014-08-13 17:38:05.320016 7fe9a77f6700 20 osd.4 194487 24.1e6 heartbeat_peers -5978> 2014-08-13 17:38:05.320017 7fe9a77f6700 20 osd.4 194487 22.12a heartbeat_peers -5977> 2014-08-13 17:38:05.320020 7fe9a77f6700 20 osd.4 194487 22.3af heartbeat_peers 4,44,61,339,362,369 -5976> 2014-08-13 17:38:05.320023 7fe9a77f6700 20 osd.4 194487 22.481 heartbeat_peers 4,287,303,306,309,365,381 -5975> 2014-08-13 17:38:05.320027 7fe9a77f6700 20 osd.4 194487 22.48e heartbeat_peers 4,289,327,341,376 -5974> 2014-08-13 17:38:05.320029 7fe9a77f6700 20 osd.4 194487 22.3ad heartbeat_peers -5973> 2014-08-13 17:38:05.320031 7fe9a77f6700 20 osd.4 194487 24.206 heartbeat_peers -5972> 2014-08-13 17:38:05.320032 7fe9a77f6700 20 osd.4 194487 24.1bb heartbeat_peers 4,331,332,337,348,379 -5971> 2014-08-13 17:38:05.320036 7fe9a77f6700 20 osd.4 194487 24.299 heartbeat_peers -5970> 2014-08-13 17:38:05.320037 7fe9a77f6700 20 osd.4 194487 22.3d5 heartbeat_peers 4,286,343 -5969> 2014-08-13 17:38:05.320039 7fe9a77f6700 20 osd.4 194487 22.537 heartbeat_peers -5968> 2014-08-13 17:38:05.320041 7fe9a77f6700 20 osd.4 194487 22.1b8 heartbeat_peers 4,340,347 -5967> 2014-08-13 17:38:05.320043 7fe9a77f6700 20 osd.4 194487 22.29a heartbeat_peers -5966> 2014-08-13 17:38:05.320044 7fe9a77f6700 20 osd.4 194487 24.cd heartbeat_peers -5965> 2014-08-13 17:38:05.320045 7fe9a77f6700 20 osd.4 194487 22.226 heartbeat_peers -5964> 2014-08-13 17:38:05.320047 7fe9a77f6700 20 osd.4 194487 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 -5963> 2014-08-13 17:38:05.320051 7fe9a77f6700 20 osd.4 194487 24.2ec heartbeat_peers -5962> 2014-08-13 17:38:05.320052 7fe9a77f6700 20 osd.4 194487 22.1a0 heartbeat_peers -5961> 2014-08-13 17:38:05.320053 7fe9a77f6700 20 osd.4 194487 24.369 heartbeat_peers -5960> 2014-08-13 17:38:05.320055 7fe9a77f6700 10 osd.4 194487 adding neighbor peer osd.3 -5959> 2014-08-13 17:38:05.320056 7fe9a77f6700 10 osd.4 194487 adding neighbor peer osd.5 -5958> 2014-08-13 17:38:05.320063 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 -5957> 2014-08-13 17:38:05.320073 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x4828f20 -- 0x4fa5f00 -5956> 2014-08-13 17:38:05.320055 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 -5955> 2014-08-13 17:38:05.320080 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -5954> 2014-08-13 17:38:05.320085 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] state: Started advmap -5953> 2014-08-13 17:38:05.320105 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x48291e0 -- 0x4fa5c80 -5952> 2014-08-13 17:38:05.320096 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] check_recovery_sources no source osds () went down -5951> 2014-08-13 17:38:05.320122 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map -5950> 2014-08-13 17:38:05.320139 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] take_waiters -5949> 2014-08-13 17:38:05.320148 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -5948> 2014-08-13 17:38:05.320159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194485 pi=192980-194484/24 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -5947> 2014-08-13 17:38:05.320171 7fe99ebe8700 10 log is not dirty -5946> 2014-08-13 17:38:05.320193 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 -5945> 2014-08-13 17:38:05.320215 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -5944> 2014-08-13 17:38:05.320221 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap -5943> 2014-08-13 17:38:05.320233 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194486, need_up_thru now false -5942> 2014-08-13 17:38:05.320243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap -5941> 2014-08-13 17:38:05.320255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down -5940> 2014-08-13 17:38:05.320266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map -5939> 2014-08-13 17:38:05.320277 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary -5938> 2014-08-13 17:38:05.320288 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194487:25288 -5937> 2014-08-13 17:38:05.320298 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters -5936> 2014-08-13 17:38:05.320312 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.911686 5 0.000759 -5935> 2014-08-13 17:38:05.320325 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering -5934> 2014-08-13 17:38:05.320335 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.005095 0 0.000000 -5933> 2014-08-13 17:38:05.320350 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -5932> 2014-08-13 17:38:05.320357 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - starting recovery -5931> 2014-08-13 17:38:05.320380 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] -5930> 2014-08-13 17:38:05.320382 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 -5929> 2014-08-13 17:38:05.320394 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x5de9080 -- 0x5d77d00 -5928> 2014-08-13 17:38:05.320373 7fe96f30e700 2 -- 192.168.100.115:0/34805 >> 192.168.100.115:6846/62713 pipe(0x4fa5c80 sd=448 :52794 s=4 pgs=141 cs=1 l=1 c=0x48291e0).reader couldn't read tag, (0) Success -5927> 2014-08-13 17:38:05.320416 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -5926> 2014-08-13 17:38:05.320427 7fe96f30e700 2 -- 192.168.100.115:0/34805 >> 192.168.100.115:6846/62713 pipe(0x4fa5c80 sd=448 :52794 s=4 pgs=141 cs=1 l=1 c=0x48291e0).fault (0) Success -5925> 2014-08-13 17:38:05.320430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 is up to date, queueing in pending_activators -5924> 2014-08-13 17:38:05.320362 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194477 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate -5923> 2014-08-13 17:38:05.320450 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] -5922> 2014-08-13 17:38:05.320462 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] -5921> 2014-08-13 17:38:05.320471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) -5920> 2014-08-13 17:38:05.320497 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5919> 2014-08-13 17:38:05.320510 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -5918> 2014-08-13 17:38:05.320540 7fe99f5e9700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 -5917> 2014-08-13 17:38:05.320551 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x5de8dc0 -- 0x5d77a80 -5916> 2014-08-13 17:38:05.320723 7fe9a77f6700 20 osd.4 194487 removing heartbeat peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 -5915> 2014-08-13 17:38:05.320735 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x5760dc0 -- 0x4fa2d00 -5914> 2014-08-13 17:38:05.320766 7fe97213c700 2 -- 192.168.100.115:0/34805 >> 192.168.1.114:6843/56512 pipe(0x4fa2d00 sd=425 :40541 s=4 pgs=100 cs=1 l=1 c=0x5760dc0).reader couldn't read tag, (0) Success -5913> 2014-08-13 17:38:05.320810 7fe97213c700 2 -- 192.168.100.115:0/34805 >> 192.168.1.114:6843/56512 pipe(0x4fa2d00 sd=425 :40541 s=4 pgs=100 cs=1 l=1 c=0x5760dc0).fault (0) Success -5912> 2014-08-13 17:38:05.320760 7fe9a77f6700 1 -- 192.168.100.115:0/34805 mark_down 0x5760b00 -- 0x4fa5500 -5911> 2014-08-13 17:38:05.320891 7fe9a77f6700 10 osd.4 194487 maybe_update_heartbeat_peers 92 peers, extras 3,5,7,32,370 -5910> 2014-08-13 17:38:05.320897 7fe9a77f6700 7 osd.4 194487 activate_map version 194487 -5909> 2014-08-13 17:38:05.320913 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 sending log((192718'15573,194422'18660], crt=194422'18660) -5908> 2014-08-13 17:38:05.320933 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing -5907> 2014-08-13 17:38:05.320936 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.286 192.168.1.113:6820/19173 -- pg_log(24.30b epoch 194487 log log((192718'15573,194422'18660], crt=194422'18660) query_epoch 194487) v4 -- ?+0 0x5fb4200 -5906> 2014-08-13 17:38:05.320944 7fe9986dd700 20 osd.4 194487 agent_entry empty queue -5905> 2014-08-13 17:38:05.320951 7fe9a77f6700 10 osd.4 194487 do_waiters -- start -5904> 2014-08-13 17:38:05.320954 7fe9a77f6700 10 osd.4 194487 do_waiters -- finish -5903> 2014-08-13 17:38:05.320921 7fe971e39700 2 -- 192.168.100.115:0/34805 >> 192.168.100.114:6850/56512 pipe(0x4fa5500 sd=433 :33476 s=4 pgs=100 cs=1 l=1 c=0x5760b00).reader couldn't read tag, (0) Success -5902> 2014-08-13 17:38:05.320955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5901> 2014-08-13 17:38:05.320964 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5900> 2014-08-13 17:38:05.320971 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5899> 2014-08-13 17:38:05.320965 7fe971e39700 2 -- 192.168.100.115:0/34805 >> 192.168.100.114:6850/56512 pipe(0x4fa5500 sd=433 :33476 s=4 pgs=100 cs=1 l=1 c=0x5760b00).fault (0) Success -5898> 2014-08-13 17:38:05.320973 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db5340 osd_map(194487..194487 src has 185139..194487) v3 -5897> 2014-08-13 17:38:05.320970 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) -5896> 2014-08-13 17:38:05.320982 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5895> 2014-08-13 17:38:05.320985 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5894> 2014-08-13 17:38:05.320987 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5893> 2014-08-13 17:38:05.320988 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5892> 2014-08-13 17:38:05.320982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 is up to date, queueing in pending_activators -5891> 2014-08-13 17:38:05.320995 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.61 192.168.1.111:6840/16566 14 ==== pg_notify(22.3af(1) epoch 194487) v5 ==== 777+0+0 (2255814963 0 0) 0x513e740 con 0x4cf86e0 -5890> 2014-08-13 17:38:05.321008 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5889> 2014-08-13 17:38:05.321009 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5888> 2014-08-13 17:38:05.321010 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e740 pg_notify(22.3af(1) epoch 194487) v5 -5887> 2014-08-13 17:38:05.320980 7fe968aa6700 2 -- 192.168.100.115:0/34805 >> 192.168.100.115:6831/59915 pipe(0x5d77a80 sd=498 :45816 s=4 pgs=124 cs=1 l=1 c=0x5de8dc0).reader couldn't read tag, (0) Success -5886> 2014-08-13 17:38:05.321015 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.318806, event: header_read, request: pg_notify(22.3af(1) epoch 194487) v5 -5885> 2014-08-13 17:38:05.321035 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.318806, event: throttled, request: pg_notify(22.3af(1) epoch 194487) v5 -5884> 2014-08-13 17:38:05.321019 7fe968aa6700 2 -- 192.168.100.115:0/34805 >> 192.168.100.115:6831/59915 pipe(0x5d77a80 sd=498 :45816 s=4 pgs=124 cs=1 l=1 c=0x5de8dc0).fault (0) Success -5883> 2014-08-13 17:38:05.321029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5882> 2014-08-13 17:38:05.321039 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.318851, event: all_read, request: pg_notify(22.3af(1) epoch 194487) v5 -5881> 2014-08-13 17:38:05.321043 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.321007, event: dispatched, request: pg_notify(22.3af(1) epoch 194487) v5 -5880> 2014-08-13 17:38:05.321047 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.321047, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194487) v5 -5879> 2014-08-13 17:38:05.321052 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.61 -5878> 2014-08-13 17:38:05.321054 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e740 -5877> 2014-08-13 17:38:05.321058 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.321058, event: started, request: pg_notify(22.3af(1) epoch 194487) v5 -5876> 2014-08-13 17:38:05.321075 7fe9a63f4700 5 -- op tracker -- , seq: 556, time: 2014-08-13 17:38:05.321075, event: done, request: pg_notify(22.3af(1) epoch 194487) v5 -5875> 2014-08-13 17:38:05.321085 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5874> 2014-08-13 17:38:05.321087 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5873> 2014-08-13 17:38:05.321090 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.31 192.168.1.103:6831/41157 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9a400 con 0x49e5960 -5872> 2014-08-13 17:38:05.321098 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5871> 2014-08-13 17:38:05.321100 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5870> 2014-08-13 17:38:05.321101 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9a400 osd_map(194487..194487 src has 185139..194487) v3 -5869> 2014-08-13 17:38:05.321104 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5868> 2014-08-13 17:38:05.321107 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5867> 2014-08-13 17:38:05.321108 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5866> 2014-08-13 17:38:05.321109 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5865> 2014-08-13 17:38:05.321117 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.61 192.168.1.111:6840/16566 15 ==== pg_notify(22.362(1) epoch 194487) v5 ==== 777+0+0 (1895365411 0 0) 0x5f03640 con 0x4cf86e0 -5864> 2014-08-13 17:38:05.321125 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5863> 2014-08-13 17:38:05.321126 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5862> 2014-08-13 17:38:05.321127 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f03640 pg_notify(22.362(1) epoch 194487) v5 -5861> 2014-08-13 17:38:05.321129 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.319478, event: header_read, request: pg_notify(22.362(1) epoch 194487) v5 -5860> 2014-08-13 17:38:05.321133 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.319480, event: throttled, request: pg_notify(22.362(1) epoch 194487) v5 -5859> 2014-08-13 17:38:05.321070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery primary has 56 missing -5858> 2014-08-13 17:38:05.321137 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.319536, event: all_read, request: pg_notify(22.362(1) epoch 194487) v5 -5857> 2014-08-13 17:38:05.321141 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.321124, event: dispatched, request: pg_notify(22.362(1) epoch 194487) v5 -5856> 2014-08-13 17:38:05.321144 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.321144, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194487) v5 -5855> 2014-08-13 17:38:05.321148 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.61 -5854> 2014-08-13 17:38:05.321150 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f03640 -5853> 2014-08-13 17:38:05.321152 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.321152, event: started, request: pg_notify(22.362(1) epoch 194487) v5 -5852> 2014-08-13 17:38:05.321165 7fe9a63f4700 5 -- op tracker -- , seq: 557, time: 2014-08-13 17:38:05.321165, event: done, request: pg_notify(22.362(1) epoch 194487) v5 -5851> 2014-08-13 17:38:05.321172 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5850> 2014-08-13 17:38:05.321173 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5849> 2014-08-13 17:38:05.321159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery -5848> 2014-08-13 17:38:05.321176 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.31 192.168.1.103:6831/41157 9 ==== pg_info(1 pgs e194487:24.1e4(20)) v4 ==== 1872+0+0 (3445789632 0 0) 0x51cd5c0 con 0x49e5960 -5847> 2014-08-13 17:38:05.321184 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5846> 2014-08-13 17:38:05.321185 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5845> 2014-08-13 17:38:05.321186 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cd5c0 pg_info(1 pgs e194487:24.1e4(20)) v4 -5844> 2014-08-13 17:38:05.321189 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.320912, event: header_read, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5843> 2014-08-13 17:38:05.321181 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] -5842> 2014-08-13 17:38:05.321193 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.320923, event: throttled, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5841> 2014-08-13 17:38:05.321197 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.321006, event: all_read, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5840> 2014-08-13 17:38:05.321208 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.321184, event: dispatched, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5839> 2014-08-13 17:38:05.321142 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.4 -5838> 2014-08-13 17:38:05.321212 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.321212, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5837> 2014-08-13 17:38:05.321216 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e4(20)) v4 from osd.31 -5836> 2014-08-13 17:38:05.321220 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cd5c0 -5835> 2014-08-13 17:38:05.321210 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -5834> 2014-08-13 17:38:05.321222 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.321222, event: started, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5833> 2014-08-13 17:38:05.321225 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 is up to date, queueing in pending_activators -5832> 2014-08-13 17:38:05.321239 7fe9a63f4700 5 -- op tracker -- , seq: 558, time: 2014-08-13 17:38:05.321239, event: done, request: pg_info(1 pgs e194487:24.1e4(20)) v4 -5831> 2014-08-13 17:38:05.321246 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5830> 2014-08-13 17:38:05.321247 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5829> 2014-08-13 17:38:05.321216 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.4 -5828> 2014-08-13 17:38:05.321283 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5827> 2014-08-13 17:38:05.321296 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) -5826> 2014-08-13 17:38:05.321308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 is up to date, queueing in pending_activators -5825> 2014-08-13 17:38:05.321285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.4 -5824> 2014-08-13 17:38:05.321347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5823> 2014-08-13 17:38:05.321360 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -5822> 2014-08-13 17:38:05.321377 7fe99ebe8700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 -5821> 2014-08-13 17:38:05.321339 7fe968da9700 2 -- 192.168.100.115:0/34805 >> 192.168.1.115:6814/59915 pipe(0x5d77d00 sd=497 :37038 s=4 pgs=124 cs=1 l=1 c=0x5de9080).reader couldn't read tag, (0) Success -5820> 2014-08-13 17:38:05.321386 7fe968da9700 2 -- 192.168.100.115:0/34805 >> 192.168.1.115:6814/59915 pipe(0x5d77d00 sd=497 :37038 s=4 pgs=124 cs=1 l=1 c=0x5de9080).fault (0) Success -5819> 2014-08-13 17:38:05.321352 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.4 -5818> 2014-08-13 17:38:05.321429 7fe96ef0a700 2 -- 192.168.100.115:0/34805 >> 192.168.1.115:6839/62713 pipe(0x4fa5f00 sd=447 :50783 s=4 pgs=141 cs=1 l=1 c=0x4828f20).reader couldn't read tag, (0) Success -5817> 2014-08-13 17:38:05.321468 7fe96ef0a700 2 -- 192.168.100.115:0/34805 >> 192.168.1.115:6839/62713 pipe(0x4fa5f00 sd=447 :50783 s=4 pgs=141 cs=1 l=1 c=0x4828f20).fault (0) Success -5816> 2014-08-13 17:38:05.321420 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.4 -5815> 2014-08-13 17:38:05.321486 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.4 -5814> 2014-08-13 17:38:05.321564 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 sending log((192834'15069,194422'17302], crt=194422'17302) -5813> 2014-08-13 17:38:05.321579 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing -5812> 2014-08-13 17:38:05.321582 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.367 192.168.1.114:6840/55513 -- pg_log(24.9b epoch 194487 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194487) v4 -- ?+0 0x5fb0600 -5811> 2014-08-13 17:38:05.321599 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5810> 2014-08-13 17:38:05.321552 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.4 -5809> 2014-08-13 17:38:05.321613 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -5808> 2014-08-13 17:38:05.321629 7fe99ebe8700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 -5807> 2014-08-13 17:38:05.321618 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.4 -5806> 2014-08-13 17:38:05.321702 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.15 192.168.1.107:6812/53408 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5410000 con 0x4d78dc0 -5805> 2014-08-13 17:38:05.321715 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5804> 2014-08-13 17:38:05.321717 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5803> 2014-08-13 17:38:05.321718 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5410000 osd_map(194487..194487 src has 185139..194487) v3 -5802> 2014-08-13 17:38:05.321722 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5801> 2014-08-13 17:38:05.321725 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5800> 2014-08-13 17:38:05.321726 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5799> 2014-08-13 17:38:05.321727 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5798> 2014-08-13 17:38:05.321684 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.4 -5797> 2014-08-13 17:38:05.321799 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 sending log((192834'15069,194422'17302], crt=194422'17302) -5796> 2014-08-13 17:38:05.321750 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.4 -5795> 2014-08-13 17:38:05.321815 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing -5794> 2014-08-13 17:38:05.321817 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.378 192.168.1.102:6840/19195 -- pg_log(24.9b epoch 194487 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194487) v4 -- ?+0 0x4872a00 -5793> 2014-08-13 17:38:05.321831 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -5792> 2014-08-13 17:38:05.321842 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.15 192.168.1.107:6812/53408 10 ==== pg_notify(22.707(33) epoch 194487) v5 ==== 2573+0+0 (1731732501 0 0) 0x5f040c0 con 0x4d78dc0 -5791> 2014-08-13 17:38:05.321859 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5790> 2014-08-13 17:38:05.321861 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5789> 2014-08-13 17:38:05.321863 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f040c0 pg_notify(22.707(33) epoch 194487) v5 -5788> 2014-08-13 17:38:05.321867 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321751, event: header_read, request: pg_notify(22.707(33) epoch 194487) v5 -5787> 2014-08-13 17:38:05.321873 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321752, event: throttled, request: pg_notify(22.707(33) epoch 194487) v5 -5786> 2014-08-13 17:38:05.321877 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321826, event: all_read, request: pg_notify(22.707(33) epoch 194487) v5 -5785> 2014-08-13 17:38:05.321816 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.4 -5784> 2014-08-13 17:38:05.321881 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321858, event: dispatched, request: pg_notify(22.707(33) epoch 194487) v5 -5783> 2014-08-13 17:38:05.321885 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321885, event: waiting_for_osdmap, request: pg_notify(22.707(33) epoch 194487) v5 -5782> 2014-08-13 17:38:05.321889 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.15 -5781> 2014-08-13 17:38:05.321891 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f040c0 -5780> 2014-08-13 17:38:05.321894 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321894, event: started, request: pg_notify(22.707(33) epoch 194487) v5 -5779> 2014-08-13 17:38:05.321910 7fe9a63f4700 5 -- op tracker -- , seq: 559, time: 2014-08-13 17:38:05.321910, event: done, request: pg_notify(22.707(33) epoch 194487) v5 -5778> 2014-08-13 17:38:05.321919 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5777> 2014-08-13 17:38:05.321920 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5776> 2014-08-13 17:38:05.321883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.4 -5775> 2014-08-13 17:38:05.321894 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing -5774> 2014-08-13 17:38:05.321950 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.4 -5773> 2014-08-13 17:38:05.321969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.4 -5772> 2014-08-13 17:38:05.322016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.4 -5771> 2014-08-13 17:38:05.322089 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.356 192.168.1.108:6832/5235 14 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db18c0 con 0x4a486e0 -5770> 2014-08-13 17:38:05.322102 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5769> 2014-08-13 17:38:05.322104 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5768> 2014-08-13 17:38:05.322106 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db18c0 osd_map(194487..194487 src has 185139..194487) v3 -5767> 2014-08-13 17:38:05.322040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.4 -5766> 2014-08-13 17:38:05.322111 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5765> 2014-08-13 17:38:05.322114 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5764> 2014-08-13 17:38:05.322116 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5763> 2014-08-13 17:38:05.322117 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5762> 2014-08-13 17:38:05.322082 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.4 -5761> 2014-08-13 17:38:05.322110 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.4 -5760> 2014-08-13 17:38:05.322148 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.4 -5759> 2014-08-13 17:38:05.322181 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.4 -5758> 2014-08-13 17:38:05.322219 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.4 -5757> 2014-08-13 17:38:05.322257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.4 -5756> 2014-08-13 17:38:05.322324 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.356 192.168.1.108:6832/5235 15 ==== pg_notify(24.136(31) epoch 194487) v5 ==== 2485+0+0 (497636557 0 0) 0x58240c0 con 0x4a486e0 -5755> 2014-08-13 17:38:05.322338 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5754> 2014-08-13 17:38:05.322340 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5753> 2014-08-13 17:38:05.322341 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58240c0 pg_notify(24.136(31) epoch 194487) v5 -5752> 2014-08-13 17:38:05.322345 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322150, event: header_read, request: pg_notify(24.136(31) epoch 194487) v5 -5751> 2014-08-13 17:38:05.322285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.4 -5750> 2014-08-13 17:38:05.322350 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322150, event: throttled, request: pg_notify(24.136(31) epoch 194487) v5 -5749> 2014-08-13 17:38:05.322353 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322238, event: all_read, request: pg_notify(24.136(31) epoch 194487) v5 -5748> 2014-08-13 17:38:05.322357 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322337, event: dispatched, request: pg_notify(24.136(31) epoch 194487) v5 -5747> 2014-08-13 17:38:05.322361 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322361, event: waiting_for_osdmap, request: pg_notify(24.136(31) epoch 194487) v5 -5746> 2014-08-13 17:38:05.322365 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.356 -5745> 2014-08-13 17:38:05.322367 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58240c0 -5744> 2014-08-13 17:38:05.322370 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322369, event: started, request: pg_notify(24.136(31) epoch 194487) v5 -5743> 2014-08-13 17:38:05.322389 7fe9a63f4700 5 -- op tracker -- , seq: 560, time: 2014-08-13 17:38:05.322389, event: done, request: pg_notify(24.136(31) epoch 194487) v5 -5742> 2014-08-13 17:38:05.322326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.4 -5741> 2014-08-13 17:38:05.322397 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5740> 2014-08-13 17:38:05.322399 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5739> 2014-08-13 17:38:05.322352 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.4 -5738> 2014-08-13 17:38:05.322409 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.356 192.168.1.108:6832/5235 16 ==== pg_info(1 pgs e194487:24.350(4)) v4 ==== 968+0+0 (1119531868 0 0) 0x5f04440 con 0x4a486e0 -5737> 2014-08-13 17:38:05.322418 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5736> 2014-08-13 17:38:05.322419 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5735> 2014-08-13 17:38:05.322421 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04440 pg_info(1 pgs e194487:24.350(4)) v4 -5734> 2014-08-13 17:38:05.322424 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322248, event: header_read, request: pg_info(1 pgs e194487:24.350(4)) v4 -5733> 2014-08-13 17:38:05.322428 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322249, event: throttled, request: pg_info(1 pgs e194487:24.350(4)) v4 -5732> 2014-08-13 17:38:05.322431 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322288, event: all_read, request: pg_info(1 pgs e194487:24.350(4)) v4 -5731> 2014-08-13 17:38:05.322435 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322418, event: dispatched, request: pg_info(1 pgs e194487:24.350(4)) v4 -5730> 2014-08-13 17:38:05.322439 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322438, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.350(4)) v4 -5729> 2014-08-13 17:38:05.322442 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.350(4)) v4 from osd.356 -5728> 2014-08-13 17:38:05.322445 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04440 -5727> 2014-08-13 17:38:05.322447 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322447, event: started, request: pg_info(1 pgs e194487:24.350(4)) v4 -5726> 2014-08-13 17:38:05.322396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.4 -5725> 2014-08-13 17:38:05.322463 7fe9a63f4700 5 -- op tracker -- , seq: 561, time: 2014-08-13 17:38:05.322463, event: done, request: pg_info(1 pgs e194487:24.350(4)) v4 -5724> 2014-08-13 17:38:05.322469 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5723> 2014-08-13 17:38:05.322471 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5722> 2014-08-13 17:38:05.322418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.4 -5721> 2014-08-13 17:38:05.322466 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.4 -5720> 2014-08-13 17:38:05.322485 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.4 -5719> 2014-08-13 17:38:05.322536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.4 -5718> 2014-08-13 17:38:05.322551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.4 -5717> 2014-08-13 17:38:05.322605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.4 -5716> 2014-08-13 17:38:05.322617 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.4 -5715> 2014-08-13 17:38:05.322720 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.45 192.168.1.112:6828/6040 11 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d93cc0 con 0x4d39080 -5714> 2014-08-13 17:38:05.322737 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5713> 2014-08-13 17:38:05.322739 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5712> 2014-08-13 17:38:05.322740 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d93cc0 osd_map(194487..194487 src has 185139..194487) v3 -5711> 2014-08-13 17:38:05.322674 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.4 -5710> 2014-08-13 17:38:05.322745 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5709> 2014-08-13 17:38:05.322683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.4 -5708> 2014-08-13 17:38:05.322748 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5707> 2014-08-13 17:38:05.322749 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5706> 2014-08-13 17:38:05.322751 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5705> 2014-08-13 17:38:05.322749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.4 -5704> 2014-08-13 17:38:05.322744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.4 -5703> 2014-08-13 17:38:05.322820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.4 -5702> 2014-08-13 17:38:05.322821 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.4 -5701> 2014-08-13 17:38:05.322887 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.4 -5700> 2014-08-13 17:38:05.322892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.4 -5699> 2014-08-13 17:38:05.322966 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.45 192.168.1.112:6828/6040 12 ==== pg_notify(22.3e4(31) epoch 194487) v5 ==== 2455+0+0 (1756086913 0 0) 0x5822680 con 0x4d39080 -5698> 2014-08-13 17:38:05.322980 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5697> 2014-08-13 17:38:05.322982 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5696> 2014-08-13 17:38:05.322983 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822680 pg_notify(22.3e4(31) epoch 194487) v5 -5695> 2014-08-13 17:38:05.322986 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.322877, event: header_read, request: pg_notify(22.3e4(31) epoch 194487) v5 -5694> 2014-08-13 17:38:05.322991 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.322878, event: throttled, request: pg_notify(22.3e4(31) epoch 194487) v5 -5693> 2014-08-13 17:38:05.322995 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.322947, event: all_read, request: pg_notify(22.3e4(31) epoch 194487) v5 -5692> 2014-08-13 17:38:05.322998 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.322979, event: dispatched, request: pg_notify(22.3e4(31) epoch 194487) v5 -5691> 2014-08-13 17:38:05.323002 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.323002, event: waiting_for_osdmap, request: pg_notify(22.3e4(31) epoch 194487) v5 -5690> 2014-08-13 17:38:05.323006 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.45 -5689> 2014-08-13 17:38:05.323007 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822680 -5688> 2014-08-13 17:38:05.323010 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.323010, event: started, request: pg_notify(22.3e4(31) epoch 194487) v5 -5687> 2014-08-13 17:38:05.322955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.4 -5686> 2014-08-13 17:38:05.323024 7fe9a63f4700 5 -- op tracker -- , seq: 562, time: 2014-08-13 17:38:05.323024, event: done, request: pg_notify(22.3e4(31) epoch 194487) v5 -5685> 2014-08-13 17:38:05.322962 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.4 -5684> 2014-08-13 17:38:05.323031 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5683> 2014-08-13 17:38:05.323034 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5682> 2014-08-13 17:38:05.323023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.4 -5681> 2014-08-13 17:38:05.323033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.4 -5680> 2014-08-13 17:38:05.323092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.4 -5679> 2014-08-13 17:38:05.323102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.4 -5678> 2014-08-13 17:38:05.323188 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.6 192.168.1.107:6809/53141 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x272af40 con 0x4e3d800 -5677> 2014-08-13 17:38:05.323204 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5676> 2014-08-13 17:38:05.323206 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5675> 2014-08-13 17:38:05.323208 7fe9a63f4700 20 osd.4 194487 _dispatch 0x272af40 osd_map(194487..194487 src has 185139..194487) v3 -5674> 2014-08-13 17:38:05.323213 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5673> 2014-08-13 17:38:05.323215 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5672> 2014-08-13 17:38:05.323217 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5671> 2014-08-13 17:38:05.323218 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5670> 2014-08-13 17:38:05.323161 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.4 -5669> 2014-08-13 17:38:05.323238 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.304 192.168.1.101:6808/22782 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db3840 con 0x49e39c0 -5668> 2014-08-13 17:38:05.323247 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5667> 2014-08-13 17:38:05.323248 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5666> 2014-08-13 17:38:05.323249 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3840 osd_map(194487..194487 src has 185139..194487) v3 -5665> 2014-08-13 17:38:05.323253 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5664> 2014-08-13 17:38:05.323255 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5663> 2014-08-13 17:38:05.323256 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5662> 2014-08-13 17:38:05.323257 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5661> 2014-08-13 17:38:05.323171 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.4 -5660> 2014-08-13 17:38:05.323231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.4 -5659> 2014-08-13 17:38:05.323294 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.304 192.168.1.101:6808/22782 11 ==== pg_info(1 pgs e194487:24.33a(4)) v4 ==== 968+0+0 (1159237853 0 0) 0x5f04ec0 con 0x49e39c0 -5658> 2014-08-13 17:38:05.323304 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5657> 2014-08-13 17:38:05.323305 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5656> 2014-08-13 17:38:05.323310 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04ec0 pg_info(1 pgs e194487:24.33a(4)) v4 -5655> 2014-08-13 17:38:05.323313 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323229, event: header_read, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5654> 2014-08-13 17:38:05.323318 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323230, event: throttled, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5653> 2014-08-13 17:38:05.323322 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323282, event: all_read, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5652> 2014-08-13 17:38:05.323326 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323303, event: dispatched, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5651> 2014-08-13 17:38:05.323329 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323329, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5650> 2014-08-13 17:38:05.323263 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.4 -5649> 2014-08-13 17:38:05.323335 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.33a(4)) v4 from osd.304 -5648> 2014-08-13 17:38:05.323338 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04ec0 -5647> 2014-08-13 17:38:05.323340 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323340, event: started, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5646> 2014-08-13 17:38:05.323354 7fe9a63f4700 5 -- op tracker -- , seq: 563, time: 2014-08-13 17:38:05.323354, event: done, request: pg_info(1 pgs e194487:24.33a(4)) v4 -5645> 2014-08-13 17:38:05.323362 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5644> 2014-08-13 17:38:05.323363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5643> 2014-08-13 17:38:05.323297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.4 -5642> 2014-08-13 17:38:05.323367 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.6 192.168.1.107:6809/53141 8 ==== pg_info(1 pgs e194487:24.cd(4)) v4 ==== 968+0+0 (44152863 0 0) 0x51024c0 con 0x4e3d800 -5641> 2014-08-13 17:38:05.323374 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5640> 2014-08-13 17:38:05.323376 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5639> 2014-08-13 17:38:05.323377 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51024c0 pg_info(1 pgs e194487:24.cd(4)) v4 -5638> 2014-08-13 17:38:05.323379 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323209, event: header_read, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5637> 2014-08-13 17:38:05.323383 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323210, event: throttled, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5636> 2014-08-13 17:38:05.323387 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323291, event: all_read, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5635> 2014-08-13 17:38:05.323390 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323374, event: dispatched, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5634> 2014-08-13 17:38:05.323394 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323394, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5633> 2014-08-13 17:38:05.323397 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.cd(4)) v4 from osd.6 -5632> 2014-08-13 17:38:05.323399 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51024c0 -5631> 2014-08-13 17:38:05.323402 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323401, event: started, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5630> 2014-08-13 17:38:05.323336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.4 -5629> 2014-08-13 17:38:05.323413 7fe9a63f4700 5 -- op tracker -- , seq: 564, time: 2014-08-13 17:38:05.323413, event: done, request: pg_info(1 pgs e194487:24.cd(4)) v4 -5628> 2014-08-13 17:38:05.323419 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5627> 2014-08-13 17:38:05.323421 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5626> 2014-08-13 17:38:05.323368 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.4 -5625> 2014-08-13 17:38:05.323406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.4 -5624> 2014-08-13 17:38:05.323434 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.4 -5623> 2014-08-13 17:38:05.323517 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.17 192.168.1.103:6811/37658 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9ad00 con 0x4d3f900 -5622> 2014-08-13 17:38:05.323530 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5621> 2014-08-13 17:38:05.323532 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5620> 2014-08-13 17:38:05.323533 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9ad00 osd_map(194487..194487 src has 185139..194487) v3 -5619> 2014-08-13 17:38:05.323537 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5618> 2014-08-13 17:38:05.323540 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5617> 2014-08-13 17:38:05.323541 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5616> 2014-08-13 17:38:05.323542 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5615> 2014-08-13 17:38:05.323477 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.4 -5614> 2014-08-13 17:38:05.323546 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.289 192.168.1.113:6808/15284 14 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5438480 con 0x4cfdac0 -5613> 2014-08-13 17:38:05.323553 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5612> 2014-08-13 17:38:05.323555 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5611> 2014-08-13 17:38:05.323556 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5438480 osd_map(194487..194487 src has 185139..194487) v3 -5610> 2014-08-13 17:38:05.323559 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5609> 2014-08-13 17:38:05.323561 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5608> 2014-08-13 17:38:05.323562 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5607> 2014-08-13 17:38:05.323563 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5606> 2014-08-13 17:38:05.323500 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.4 -5605> 2014-08-13 17:38:05.323546 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.4 -5604> 2014-08-13 17:38:05.323566 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.4 -5603> 2014-08-13 17:38:05.323653 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.289 192.168.1.113:6808/15284 15 ==== pg_info(1 pgs e194487:24.5e(4)) v4 ==== 968+0+0 (3373643007 0 0) 0x5f02680 con 0x4cfdac0 -5602> 2014-08-13 17:38:05.323670 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5601> 2014-08-13 17:38:05.323673 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5600> 2014-08-13 17:38:05.323674 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02680 pg_info(1 pgs e194487:24.5e(4)) v4 -5599> 2014-08-13 17:38:05.323616 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.4 -5598> 2014-08-13 17:38:05.323678 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323527, event: header_read, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5597> 2014-08-13 17:38:05.323685 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323528, event: throttled, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5596> 2014-08-13 17:38:05.323689 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323566, event: all_read, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5595> 2014-08-13 17:38:05.323693 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323669, event: dispatched, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5594> 2014-08-13 17:38:05.323632 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.4 -5593> 2014-08-13 17:38:05.323697 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323697, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5592> 2014-08-13 17:38:05.323701 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.5e(4)) v4 from osd.289 -5591> 2014-08-13 17:38:05.323704 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02680 -5590> 2014-08-13 17:38:05.323708 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323707, event: started, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5589> 2014-08-13 17:38:05.323723 7fe9a63f4700 5 -- op tracker -- , seq: 565, time: 2014-08-13 17:38:05.323722, event: done, request: pg_info(1 pgs e194487:24.5e(4)) v4 -5588> 2014-08-13 17:38:05.323731 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5587> 2014-08-13 17:38:05.323733 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5586> 2014-08-13 17:38:05.323737 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.17 192.168.1.103:6811/37658 11 ==== pg_info(1 pgs e194487:24.2a1(4)) v4 ==== 968+0+0 (1663665772 0 0) 0x5f05080 con 0x4d3f900 -5585> 2014-08-13 17:38:05.323745 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5584> 2014-08-13 17:38:05.323747 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5583> 2014-08-13 17:38:05.323748 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f05080 pg_info(1 pgs e194487:24.2a1(4)) v4 -5582> 2014-08-13 17:38:05.323685 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.4 -5581> 2014-08-13 17:38:05.323751 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323571, event: header_read, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5580> 2014-08-13 17:38:05.323755 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323573, event: throttled, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5579> 2014-08-13 17:38:05.323758 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323619, event: all_read, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5578> 2014-08-13 17:38:05.323698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.4 -5577> 2014-08-13 17:38:05.323762 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323745, event: dispatched, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5576> 2014-08-13 17:38:05.323766 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323766, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5575> 2014-08-13 17:38:05.323769 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2a1(4)) v4 from osd.17 -5574> 2014-08-13 17:38:05.323772 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f05080 -5573> 2014-08-13 17:38:05.323774 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323774, event: started, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5572> 2014-08-13 17:38:05.323787 7fe9a63f4700 5 -- op tracker -- , seq: 566, time: 2014-08-13 17:38:05.323787, event: done, request: pg_info(1 pgs e194487:24.2a1(4)) v4 -5571> 2014-08-13 17:38:05.323795 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5570> 2014-08-13 17:38:05.323796 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5569> 2014-08-13 17:38:05.323755 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.4 -5568> 2014-08-13 17:38:05.323765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.4 -5567> 2014-08-13 17:38:05.323825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.4 -5566> 2014-08-13 17:38:05.323882 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.379 192.168.1.108:6830/4861 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5480240 con 0x4e82c00 -5565> 2014-08-13 17:38:05.323831 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.4 -5564> 2014-08-13 17:38:05.323895 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5563> 2014-08-13 17:38:05.323898 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5562> 2014-08-13 17:38:05.323899 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5480240 osd_map(194487..194487 src has 185139..194487) v3 -5561> 2014-08-13 17:38:05.323905 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5560> 2014-08-13 17:38:05.323908 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5559> 2014-08-13 17:38:05.323909 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5558> 2014-08-13 17:38:05.323910 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5557> 2014-08-13 17:38:05.323898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.4 -5556> 2014-08-13 17:38:05.323895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.4 -5555> 2014-08-13 17:38:05.323965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.4 -5554> 2014-08-13 17:38:05.323965 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.4 -5553> 2014-08-13 17:38:05.324067 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.45 192.168.1.112:6828/6040 13 ==== pg_info(1 pgs e194487:24.369(4)) v4 ==== 968+0+0 (3631982190 0 0) 0x5820000 con 0x4d39080 -5552> 2014-08-13 17:38:05.324086 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5551> 2014-08-13 17:38:05.324089 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5550> 2014-08-13 17:38:05.324091 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820000 pg_info(1 pgs e194487:24.369(4)) v4 -5549> 2014-08-13 17:38:05.324032 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.4 -5548> 2014-08-13 17:38:05.324095 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.323984, event: header_read, request: pg_info(1 pgs e194487:24.369(4)) v4 -5547> 2014-08-13 17:38:05.324035 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.4 -5546> 2014-08-13 17:38:05.324101 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.323986, event: throttled, request: pg_info(1 pgs e194487:24.369(4)) v4 -5545> 2014-08-13 17:38:05.324105 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.324045, event: all_read, request: pg_info(1 pgs e194487:24.369(4)) v4 -5544> 2014-08-13 17:38:05.324109 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.324085, event: dispatched, request: pg_info(1 pgs e194487:24.369(4)) v4 -5543> 2014-08-13 17:38:05.324128 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.324127, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.369(4)) v4 -5542> 2014-08-13 17:38:05.324133 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.369(4)) v4 from osd.45 -5541> 2014-08-13 17:38:05.324136 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820000 -5540> 2014-08-13 17:38:05.324139 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.324139, event: started, request: pg_info(1 pgs e194487:24.369(4)) v4 -5539> 2014-08-13 17:38:05.324099 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.4 -5538> 2014-08-13 17:38:05.324105 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.4 -5537> 2014-08-13 17:38:05.324175 7fe9a63f4700 5 -- op tracker -- , seq: 567, time: 2014-08-13 17:38:05.324175, event: done, request: pg_info(1 pgs e194487:24.369(4)) v4 -5536> 2014-08-13 17:38:05.324207 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5535> 2014-08-13 17:38:05.324210 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5534> 2014-08-13 17:38:05.324166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.4 -5533> 2014-08-13 17:38:05.324224 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.379 192.168.1.108:6830/4861 8 ==== pg_notify(24.1bb(35) epoch 194487) v5 ==== 2721+0+0 (1604338514 0 0) 0x51cc600 con 0x4e82c00 -5532> 2014-08-13 17:38:05.324237 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5531> 2014-08-13 17:38:05.324238 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5530> 2014-08-13 17:38:05.324240 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cc600 pg_notify(24.1bb(35) epoch 194487) v5 -5529> 2014-08-13 17:38:05.324243 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.323998, event: header_read, request: pg_notify(24.1bb(35) epoch 194487) v5 -5528> 2014-08-13 17:38:05.324176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.4 -5527> 2014-08-13 17:38:05.324248 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.323999, event: throttled, request: pg_notify(24.1bb(35) epoch 194487) v5 -5526> 2014-08-13 17:38:05.324252 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.324074, event: all_read, request: pg_notify(24.1bb(35) epoch 194487) v5 -5525> 2014-08-13 17:38:05.324255 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.324236, event: dispatched, request: pg_notify(24.1bb(35) epoch 194487) v5 -5524> 2014-08-13 17:38:05.324259 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.324259, event: waiting_for_osdmap, request: pg_notify(24.1bb(35) epoch 194487) v5 -5523> 2014-08-13 17:38:05.324263 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.379 -5522> 2014-08-13 17:38:05.324265 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cc600 -5521> 2014-08-13 17:38:05.324268 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.324268, event: started, request: pg_notify(24.1bb(35) epoch 194487) v5 -5520> 2014-08-13 17:38:05.324289 7fe9a63f4700 5 -- op tracker -- , seq: 568, time: 2014-08-13 17:38:05.324289, event: done, request: pg_notify(24.1bb(35) epoch 194487) v5 -5519> 2014-08-13 17:38:05.324297 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5518> 2014-08-13 17:38:05.324298 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5517> 2014-08-13 17:38:05.324236 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.4 -5516> 2014-08-13 17:38:05.324252 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.4 -5515> 2014-08-13 17:38:05.324314 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.285 192.168.1.101:6822/26295 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d98fc0 con 0x4a45280 -5514> 2014-08-13 17:38:05.324323 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5513> 2014-08-13 17:38:05.324325 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5512> 2014-08-13 17:38:05.324326 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d98fc0 osd_map(194487..194487 src has 185139..194487) v3 -5511> 2014-08-13 17:38:05.324331 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5510> 2014-08-13 17:38:05.324334 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5509> 2014-08-13 17:38:05.324336 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5508> 2014-08-13 17:38:05.324337 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5507> 2014-08-13 17:38:05.324345 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.285 192.168.1.101:6822/26295 9 ==== pg_info(1 pgs e194487:24.1e6(4)) v4 ==== 968+0+0 (1166602159 0 0) 0x51c9f80 con 0x4a45280 -5506> 2014-08-13 17:38:05.324353 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5505> 2014-08-13 17:38:05.324354 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5504> 2014-08-13 17:38:05.324355 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9f80 pg_info(1 pgs e194487:24.1e6(4)) v4 -5503> 2014-08-13 17:38:05.324358 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324216, event: header_read, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5502> 2014-08-13 17:38:05.324363 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324218, event: throttled, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5501> 2014-08-13 17:38:05.324303 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.4 -5500> 2014-08-13 17:38:05.324366 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324268, event: all_read, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5499> 2014-08-13 17:38:05.324370 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324353, event: dispatched, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5498> 2014-08-13 17:38:05.324374 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324373, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5497> 2014-08-13 17:38:05.324377 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e6(4)) v4 from osd.285 -5496> 2014-08-13 17:38:05.324380 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9f80 -5495> 2014-08-13 17:38:05.324382 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324382, event: started, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5494> 2014-08-13 17:38:05.324323 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.4 -5493> 2014-08-13 17:38:05.324395 7fe9a63f4700 5 -- op tracker -- , seq: 569, time: 2014-08-13 17:38:05.324395, event: done, request: pg_info(1 pgs e194487:24.1e6(4)) v4 -5492> 2014-08-13 17:38:05.324402 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5491> 2014-08-13 17:38:05.324403 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5490> 2014-08-13 17:38:05.324406 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.17 192.168.1.103:6811/37658 12 ==== pg_notify(24.b6(38) epoch 194487) v5 ==== 2898+0+0 (3208193875 0 0) 0x51047c0 con 0x4d3f900 -5489> 2014-08-13 17:38:05.324414 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5488> 2014-08-13 17:38:05.324415 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5487> 2014-08-13 17:38:05.324416 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51047c0 pg_notify(24.b6(38) epoch 194487) v5 -5486> 2014-08-13 17:38:05.324369 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.4 -5485> 2014-08-13 17:38:05.324419 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324214, event: header_read, request: pg_notify(24.b6(38) epoch 194487) v5 -5484> 2014-08-13 17:38:05.324436 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324216, event: throttled, request: pg_notify(24.b6(38) epoch 194487) v5 -5483> 2014-08-13 17:38:05.324440 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324299, event: all_read, request: pg_notify(24.b6(38) epoch 194487) v5 -5482> 2014-08-13 17:38:05.324443 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324414, event: dispatched, request: pg_notify(24.b6(38) epoch 194487) v5 -5481> 2014-08-13 17:38:05.324448 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324448, event: waiting_for_osdmap, request: pg_notify(24.b6(38) epoch 194487) v5 -5480> 2014-08-13 17:38:05.324452 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.17 -5479> 2014-08-13 17:38:05.324453 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51047c0 -5478> 2014-08-13 17:38:05.324456 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324456, event: started, request: pg_notify(24.b6(38) epoch 194487) v5 -5477> 2014-08-13 17:38:05.324393 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.4 -5476> 2014-08-13 17:38:05.324468 7fe9a63f4700 5 -- op tracker -- , seq: 570, time: 2014-08-13 17:38:05.324468, event: done, request: pg_notify(24.b6(38) epoch 194487) v5 -5475> 2014-08-13 17:38:05.324475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5474> 2014-08-13 17:38:05.324476 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5473> 2014-08-13 17:38:05.324436 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.4 -5472> 2014-08-13 17:38:05.324463 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.4 -5471> 2014-08-13 17:38:05.324502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.4 -5470> 2014-08-13 17:38:05.324532 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.4 -5469> 2014-08-13 17:38:05.324568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.4 -5468> 2014-08-13 17:38:05.324621 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.346 192.168.1.104:6820/2308 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d89d40 con 0x4e4a100 -5467> 2014-08-13 17:38:05.324637 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5466> 2014-08-13 17:38:05.324639 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5465> 2014-08-13 17:38:05.324651 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d89d40 osd_map(194487..194487 src has 185139..194487) v3 -5464> 2014-08-13 17:38:05.324656 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5463> 2014-08-13 17:38:05.324659 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5462> 2014-08-13 17:38:05.324660 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5461> 2014-08-13 17:38:05.324661 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5460> 2014-08-13 17:38:05.324602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.4 -5459> 2014-08-13 17:38:05.324634 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.4 -5458> 2014-08-13 17:38:05.324672 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.4 -5457> 2014-08-13 17:38:05.324700 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.4 -5456> 2014-08-13 17:38:05.324741 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.4 -5455> 2014-08-13 17:38:05.324766 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.4 -5454> 2014-08-13 17:38:05.324810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.4 -5453> 2014-08-13 17:38:05.324832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.4 -5452> 2014-08-13 17:38:05.324880 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.4 -5451> 2014-08-13 17:38:05.324896 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5450> 2014-08-13 17:38:05.324993 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 11 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d998c0 con 0x4ec9b80 -5449> 2014-08-13 17:38:05.325009 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5448> 2014-08-13 17:38:05.325012 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5447> 2014-08-13 17:38:05.325013 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d998c0 osd_map(194487..194487 src has 185139..194487) v3 -5446> 2014-08-13 17:38:05.324952 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.4 -5445> 2014-08-13 17:38:05.325020 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5444> 2014-08-13 17:38:05.325023 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5443> 2014-08-13 17:38:05.325025 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5442> 2014-08-13 17:38:05.325026 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5441> 2014-08-13 17:38:05.325006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] activate: adding 10 as a source -5440> 2014-08-13 17:38:05.325022 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.4 -5439> 2014-08-13 17:38:05.325073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.10 -5438> 2014-08-13 17:38:05.325092 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.4 -5437> 2014-08-13 17:38:05.325140 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=55] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.10 -5436> 2014-08-13 17:38:05.325220 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 12 ==== pg_notify(22.1d2(27) epoch 194487) v5 ==== 2231+0+0 (3320793000 0 0) 0x5104440 con 0x4ec9b80 -5435> 2014-08-13 17:38:05.325161 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.4 -5434> 2014-08-13 17:38:05.325235 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5433> 2014-08-13 17:38:05.325237 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5432> 2014-08-13 17:38:05.325238 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104440 pg_notify(22.1d2(27) epoch 194487) v5 -5431> 2014-08-13 17:38:05.325242 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325057, event: header_read, request: pg_notify(22.1d2(27) epoch 194487) v5 -5430> 2014-08-13 17:38:05.325247 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325060, event: throttled, request: pg_notify(22.1d2(27) epoch 194487) v5 -5429> 2014-08-13 17:38:05.325251 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325137, event: all_read, request: pg_notify(22.1d2(27) epoch 194487) v5 -5428> 2014-08-13 17:38:05.325254 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325234, event: dispatched, request: pg_notify(22.1d2(27) epoch 194487) v5 -5427> 2014-08-13 17:38:05.325258 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325258, event: waiting_for_osdmap, request: pg_notify(22.1d2(27) epoch 194487) v5 -5426> 2014-08-13 17:38:05.325261 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.330 -5425> 2014-08-13 17:38:05.325263 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104440 -5424> 2014-08-13 17:38:05.325266 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325266, event: started, request: pg_notify(22.1d2(27) epoch 194487) v5 -5423> 2014-08-13 17:38:05.325281 7fe9a63f4700 5 -- op tracker -- , seq: 571, time: 2014-08-13 17:38:05.325281, event: done, request: pg_notify(22.1d2(27) epoch 194487) v5 -5422> 2014-08-13 17:38:05.325214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=54] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.10 -5421> 2014-08-13 17:38:05.325288 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5420> 2014-08-13 17:38:05.325290 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5419> 2014-08-13 17:38:05.325235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.4 -5418> 2014-08-13 17:38:05.325287 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=53] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.10 -5417> 2014-08-13 17:38:05.325308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.4 -5416> 2014-08-13 17:38:05.325359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=52] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.10 -5415> 2014-08-13 17:38:05.325377 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.4 -5414> 2014-08-13 17:38:05.325484 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 13 ==== pg_info(1 pgs e194487:24.3f7(2)) v4 ==== 866+0+0 (2806122021 0 0) 0x51c9c00 con 0x4ec9b80 -5413> 2014-08-13 17:38:05.325503 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5412> 2014-08-13 17:38:05.325435 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=51] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.10 -5411> 2014-08-13 17:38:05.325509 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5410> 2014-08-13 17:38:05.325447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.4 -5409> 2014-08-13 17:38:05.325513 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9c00 pg_info(1 pgs e194487:24.3f7(2)) v4 -5408> 2014-08-13 17:38:05.325518 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325383, event: header_read, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5407> 2014-08-13 17:38:05.325524 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325385, event: throttled, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5406> 2014-08-13 17:38:05.325528 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325433, event: all_read, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5405> 2014-08-13 17:38:05.325531 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325502, event: dispatched, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5404> 2014-08-13 17:38:05.325536 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325535, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5403> 2014-08-13 17:38:05.325539 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3f7(2)) v4 from osd.330 -5402> 2014-08-13 17:38:05.325542 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9c00 -5401> 2014-08-13 17:38:05.325546 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325545, event: started, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5400> 2014-08-13 17:38:05.325561 7fe9a63f4700 5 -- op tracker -- , seq: 572, time: 2014-08-13 17:38:05.325561, event: done, request: pg_info(1 pgs e194487:24.3f7(2)) v4 -5399> 2014-08-13 17:38:05.325570 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5398> 2014-08-13 17:38:05.325572 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5397> 2014-08-13 17:38:05.325516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.4 -5396> 2014-08-13 17:38:05.325577 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.343 192.168.1.104:6802/61960 12 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x272b3c0 con 0x4d31e40 -5395> 2014-08-13 17:38:05.325585 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5394> 2014-08-13 17:38:05.325587 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5393> 2014-08-13 17:38:05.325588 7fe9a63f4700 20 osd.4 194487 _dispatch 0x272b3c0 osd_map(194487..194487 src has 185139..194487) v3 -5392> 2014-08-13 17:38:05.325512 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=50] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.10 -5391> 2014-08-13 17:38:05.325593 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5390> 2014-08-13 17:38:05.325597 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5389> 2014-08-13 17:38:05.325598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5388> 2014-08-13 17:38:05.325599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5387> 2014-08-13 17:38:05.325602 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.343 192.168.1.104:6802/61960 13 ==== pg_info(1 pgs e194487:24.206(4)) v4 ==== 968+0+0 (2101806737 0 0) 0x51c8700 con 0x4d31e40 -5386> 2014-08-13 17:38:05.325609 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5385> 2014-08-13 17:38:05.325610 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5384> 2014-08-13 17:38:05.325611 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c8700 pg_info(1 pgs e194487:24.206(4)) v4 -5383> 2014-08-13 17:38:05.325614 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325496, event: header_read, request: pg_info(1 pgs e194487:24.206(4)) v4 -5382> 2014-08-13 17:38:05.325618 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325496, event: throttled, request: pg_info(1 pgs e194487:24.206(4)) v4 -5381> 2014-08-13 17:38:05.325621 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325545, event: all_read, request: pg_info(1 pgs e194487:24.206(4)) v4 -5380> 2014-08-13 17:38:05.325625 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325609, event: dispatched, request: pg_info(1 pgs e194487:24.206(4)) v4 -5379> 2014-08-13 17:38:05.325628 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325628, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.206(4)) v4 -5378> 2014-08-13 17:38:05.325632 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.206(4)) v4 from osd.343 -5377> 2014-08-13 17:38:05.325634 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c8700 -5376> 2014-08-13 17:38:05.325637 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325637, event: started, request: pg_info(1 pgs e194487:24.206(4)) v4 -5375> 2014-08-13 17:38:05.325586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.4 -5374> 2014-08-13 17:38:05.325656 7fe9a63f4700 5 -- op tracker -- , seq: 573, time: 2014-08-13 17:38:05.325656, event: done, request: pg_info(1 pgs e194487:24.206(4)) v4 -5373> 2014-08-13 17:38:05.325663 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5372> 2014-08-13 17:38:05.325665 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5371> 2014-08-13 17:38:05.325594 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=49] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.10 -5370> 2014-08-13 17:38:05.325656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.4 -5369> 2014-08-13 17:38:05.325718 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.346 192.168.1.104:6820/2308 8 ==== pg_info(1 pgs e194487:24.e2(2)) v4 ==== 866+0+0 (2687364431 0 0) 0x5103b80 con 0x4e4a100 -5368> 2014-08-13 17:38:05.325732 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5367> 2014-08-13 17:38:05.325734 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5366> 2014-08-13 17:38:05.325735 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103b80 pg_info(1 pgs e194487:24.e2(2)) v4 -5365> 2014-08-13 17:38:05.325739 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325636, event: header_read, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5364> 2014-08-13 17:38:05.325743 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325637, event: throttled, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5363> 2014-08-13 17:38:05.325746 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325687, event: all_read, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5362> 2014-08-13 17:38:05.325750 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325731, event: dispatched, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5361> 2014-08-13 17:38:05.325674 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=48] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.10 -5360> 2014-08-13 17:38:05.325753 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325753, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5359> 2014-08-13 17:38:05.325757 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.e2(2)) v4 from osd.346 -5358> 2014-08-13 17:38:05.325760 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103b80 -5357> 2014-08-13 17:38:05.325762 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325762, event: started, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5356> 2014-08-13 17:38:05.325775 7fe9a63f4700 5 -- op tracker -- , seq: 574, time: 2014-08-13 17:38:05.325775, event: done, request: pg_info(1 pgs e194487:24.e2(2)) v4 -5355> 2014-08-13 17:38:05.325782 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5354> 2014-08-13 17:38:05.325784 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5353> 2014-08-13 17:38:05.325726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.4 -5352> 2014-08-13 17:38:05.325756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=47] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.10 -5351> 2014-08-13 17:38:05.325796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.4 -5350> 2014-08-13 17:38:05.325840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=46] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.10 -5349> 2014-08-13 17:38:05.325865 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.4 -5348> 2014-08-13 17:38:05.325920 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.356 192.168.1.108:6832/5235 17 ==== pg_info(1 pgs e194487:24.3b7(2)) v4 ==== 866+0+0 (121504060 0 0) 0x5f04280 con 0x4a486e0 -5347> 2014-08-13 17:38:05.325937 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5346> 2014-08-13 17:38:05.325940 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5345> 2014-08-13 17:38:05.325941 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f04280 pg_info(1 pgs e194487:24.3b7(2)) v4 -5344> 2014-08-13 17:38:05.325945 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325846, event: header_read, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5343> 2014-08-13 17:38:05.325951 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325847, event: throttled, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5342> 2014-08-13 17:38:05.325955 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325893, event: all_read, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5341> 2014-08-13 17:38:05.325959 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325936, event: dispatched, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5340> 2014-08-13 17:38:05.325963 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325963, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5339> 2014-08-13 17:38:05.325966 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3b7(2)) v4 from osd.356 -5338> 2014-08-13 17:38:05.325969 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f04280 -5337> 2014-08-13 17:38:05.325972 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325972, event: started, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5336> 2014-08-13 17:38:05.325987 7fe9a63f4700 5 -- op tracker -- , seq: 575, time: 2014-08-13 17:38:05.325987, event: done, request: pg_info(1 pgs e194487:24.3b7(2)) v4 -5335> 2014-08-13 17:38:05.325995 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5334> 2014-08-13 17:38:05.325996 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5333> 2014-08-13 17:38:05.325934 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.4 -5332> 2014-08-13 17:38:05.325926 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=45] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.10 -5331> 2014-08-13 17:38:05.326004 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.4 -5330> 2014-08-13 17:38:05.326014 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=44] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.10 -5329> 2014-08-13 17:38:05.326073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.4 -5328> 2014-08-13 17:38:05.326103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=43] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.10 -5327> 2014-08-13 17:38:05.326141 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5326> 2014-08-13 17:38:05.326199 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=42] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.10 -5325> 2014-08-13 17:38:05.326282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 291 as a source -5324> 2014-08-13 17:38:05.326295 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=41] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.10 -5323> 2014-08-13 17:38:05.326355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.291 -5322> 2014-08-13 17:38:05.326392 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=40] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.10 -5321> 2014-08-13 17:38:05.326427 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.291 -5320> 2014-08-13 17:38:05.326502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.291 -5319> 2014-08-13 17:38:05.326490 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=39] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.10 -5318> 2014-08-13 17:38:05.326579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.291 -5317> 2014-08-13 17:38:05.326588 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=38] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.10 -5316> 2014-08-13 17:38:05.326657 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.291 -5315> 2014-08-13 17:38:05.326687 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=37] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.10 -5314> 2014-08-13 17:38:05.326781 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.364 192.168.1.114:6826/52317 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d98b40 con 0x4cfc620 -5313> 2014-08-13 17:38:05.326800 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5312> 2014-08-13 17:38:05.326803 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5311> 2014-08-13 17:38:05.326804 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d98b40 osd_map(194487..194487 src has 185139..194487) v3 -5310> 2014-08-13 17:38:05.326810 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5309> 2014-08-13 17:38:05.326813 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5308> 2014-08-13 17:38:05.326814 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5307> 2014-08-13 17:38:05.326815 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5306> 2014-08-13 17:38:05.326737 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.291 -5305> 2014-08-13 17:38:05.326790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=36] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.10 -5304> 2014-08-13 17:38:05.326817 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.291 -5303> 2014-08-13 17:38:05.326935 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.364 192.168.1.114:6826/52317 7 ==== pg_info(1 pgs e194487:24.1de(2)) v4 ==== 866+0+0 (4170479942 0 0) 0x5107000 con 0x4cfc620 -5302> 2014-08-13 17:38:05.326954 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5301> 2014-08-13 17:38:05.326957 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5300> 2014-08-13 17:38:05.326958 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_info(1 pgs e194487:24.1de(2)) v4 -5299> 2014-08-13 17:38:05.326962 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326830, event: header_read, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5298> 2014-08-13 17:38:05.326968 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326832, event: throttled, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5297> 2014-08-13 17:38:05.326972 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326883, event: all_read, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5296> 2014-08-13 17:38:05.326976 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326953, event: dispatched, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5295> 2014-08-13 17:38:05.326980 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326980, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5294> 2014-08-13 17:38:05.326903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.291 -5293> 2014-08-13 17:38:05.326984 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1de(2)) v4 from osd.364 -5292> 2014-08-13 17:38:05.326987 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 -5291> 2014-08-13 17:38:05.326991 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.326991, event: started, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5290> 2014-08-13 17:38:05.326892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=35] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.10 -5289> 2014-08-13 17:38:05.327007 7fe9a63f4700 5 -- op tracker -- , seq: 576, time: 2014-08-13 17:38:05.327007, event: done, request: pg_info(1 pgs e194487:24.1de(2)) v4 -5288> 2014-08-13 17:38:05.327016 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5287> 2014-08-13 17:38:05.327017 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5286> 2014-08-13 17:38:05.326987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.291 -5285> 2014-08-13 17:38:05.326996 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=34] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.10 -5284> 2014-08-13 17:38:05.327073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.291 -5283> 2014-08-13 17:38:05.327103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=33] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.10 -5282> 2014-08-13 17:38:05.327159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.291 -5281> 2014-08-13 17:38:05.327214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=32] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.10 -5280> 2014-08-13 17:38:05.327255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.291 -5279> 2014-08-13 17:38:05.327326 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=31] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.10 -5278> 2014-08-13 17:38:05.327347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.291 -5277> 2014-08-13 17:38:05.327441 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.291 -5276> 2014-08-13 17:38:05.327440 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=30] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.10 -5275> 2014-08-13 17:38:05.327538 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.291 -5274> 2014-08-13 17:38:05.327555 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=29] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.10 -5273> 2014-08-13 17:38:05.327635 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.291 -5272> 2014-08-13 17:38:05.327728 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.374 192.168.1.102:6828/16129 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5460fc0 con 0x4c75120 -5271> 2014-08-13 17:38:05.327748 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5270> 2014-08-13 17:38:05.327750 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5269> 2014-08-13 17:38:05.327752 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5460fc0 osd_map(194487..194487 src has 185139..194487) v3 -5268> 2014-08-13 17:38:05.327757 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5267> 2014-08-13 17:38:05.327761 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5266> 2014-08-13 17:38:05.327762 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5265> 2014-08-13 17:38:05.327763 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5264> 2014-08-13 17:38:05.327672 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=28] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.10 -5263> 2014-08-13 17:38:05.327735 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.291 -5262> 2014-08-13 17:38:05.327834 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.374 192.168.1.102:6828/16129 9 ==== pg_info(1 pgs e194487:24.2ec(2)) v4 ==== 866+0+0 (3866829036 0 0) 0x5820540 con 0x4c75120 -5261> 2014-08-13 17:38:05.327855 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5260> 2014-08-13 17:38:05.327857 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5259> 2014-08-13 17:38:05.327859 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820540 pg_info(1 pgs e194487:24.2ec(2)) v4 -5258> 2014-08-13 17:38:05.327863 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327763, event: header_read, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5257> 2014-08-13 17:38:05.327870 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327764, event: throttled, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5256> 2014-08-13 17:38:05.327873 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327813, event: all_read, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5255> 2014-08-13 17:38:05.327877 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327854, event: dispatched, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5254> 2014-08-13 17:38:05.327881 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327881, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5253> 2014-08-13 17:38:05.327885 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2ec(2)) v4 from osd.374 -5252> 2014-08-13 17:38:05.327888 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820540 -5251> 2014-08-13 17:38:05.327891 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327891, event: started, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5250> 2014-08-13 17:38:05.327789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=27] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.10 -5249> 2014-08-13 17:38:05.327908 7fe9a63f4700 5 -- op tracker -- , seq: 577, time: 2014-08-13 17:38:05.327907, event: done, request: pg_info(1 pgs e194487:24.2ec(2)) v4 -5248> 2014-08-13 17:38:05.327916 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5247> 2014-08-13 17:38:05.327918 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5246> 2014-08-13 17:38:05.327834 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.291 -5245> 2014-08-13 17:38:05.327906 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=26] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.10 -5244> 2014-08-13 17:38:05.327937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.291 -5243> 2014-08-13 17:38:05.328040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.291 -5242> 2014-08-13 17:38:05.328028 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=25] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.10 -5241> 2014-08-13 17:38:05.328146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.291 -5240> 2014-08-13 17:38:05.328150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=24] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.10 -5239> 2014-08-13 17:38:05.328258 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.291 -5238> 2014-08-13 17:38:05.328277 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=23] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.10 -5237> 2014-08-13 17:38:05.328368 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.291 -5236> 2014-08-13 17:38:05.328402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=22] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.10 -5235> 2014-08-13 17:38:05.328478 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.291 -5234> 2014-08-13 17:38:05.328529 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=21] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.10 -5233> 2014-08-13 17:38:05.328589 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.291 -5232> 2014-08-13 17:38:05.328735 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.15 192.168.1.107:6812/53408 11 ==== pg_info(1 pgs e194487:24.20e(2)) v4 ==== 866+0+0 (1279492643 0 0) 0x5f03f00 con 0x4d78dc0 -5231> 2014-08-13 17:38:05.328753 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5230> 2014-08-13 17:38:05.328757 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5229> 2014-08-13 17:38:05.328758 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f03f00 pg_info(1 pgs e194487:24.20e(2)) v4 -5228> 2014-08-13 17:38:05.328763 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328668, event: header_read, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5227> 2014-08-13 17:38:05.328768 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328670, event: throttled, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5226> 2014-08-13 17:38:05.328772 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328716, event: all_read, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5225> 2014-08-13 17:38:05.328775 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328752, event: dispatched, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5224> 2014-08-13 17:38:05.328658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=20] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.10 -5223> 2014-08-13 17:38:05.328779 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328779, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5222> 2014-08-13 17:38:05.328790 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.20e(2)) v4 from osd.15 -5221> 2014-08-13 17:38:05.328793 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f03f00 -5220> 2014-08-13 17:38:05.328801 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328800, event: started, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5219> 2014-08-13 17:38:05.328704 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.291 -5218> 2014-08-13 17:38:05.328817 7fe9a63f4700 5 -- op tracker -- , seq: 578, time: 2014-08-13 17:38:05.328817, event: done, request: pg_info(1 pgs e194487:24.20e(2)) v4 -5217> 2014-08-13 17:38:05.328825 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5216> 2014-08-13 17:38:05.328827 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5215> 2014-08-13 17:38:05.328790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=19] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.10 -5214> 2014-08-13 17:38:05.328820 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.291 -5213> 2014-08-13 17:38:05.328937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.291 -5212> 2014-08-13 17:38:05.328925 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=18] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.10 -5211> 2014-08-13 17:38:05.329057 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.291 -5210> 2014-08-13 17:38:05.329060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=17] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.10 -5209> 2014-08-13 17:38:05.329179 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.291 -5208> 2014-08-13 17:38:05.329201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=16] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.10 -5207> 2014-08-13 17:38:05.329305 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.291 -5206> 2014-08-13 17:38:05.329342 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=15] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.10 -5205> 2014-08-13 17:38:05.329431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.291 -5204> 2014-08-13 17:38:05.329483 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=14] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.10 -5203> 2014-08-13 17:38:05.329557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.291 -5202> 2014-08-13 17:38:05.329626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=13] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.10 -5201> 2014-08-13 17:38:05.329686 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.291 -5200> 2014-08-13 17:38:05.329770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=12] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.10 -5199> 2014-08-13 17:38:05.329837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.291 -5198> 2014-08-13 17:38:05.329916 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=11] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.10 -5197> 2014-08-13 17:38:05.329970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.291 -5196> 2014-08-13 17:38:05.330067 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=10] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.10 -5195> 2014-08-13 17:38:05.330104 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.291 -5194> 2014-08-13 17:38:05.330219 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=9] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.10 -5193> 2014-08-13 17:38:05.330242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.291 -5192> 2014-08-13 17:38:05.330457 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.293 192.168.1.109:6816/33855 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54da1c0 con 0x4ec89a0 -5191> 2014-08-13 17:38:05.330473 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5190> 2014-08-13 17:38:05.330475 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5189> 2014-08-13 17:38:05.330476 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54da1c0 osd_map(194487..194487 src has 185139..194487) v3 -5188> 2014-08-13 17:38:05.330481 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5187> 2014-08-13 17:38:05.330484 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5186> 2014-08-13 17:38:05.330486 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5185> 2014-08-13 17:38:05.330487 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5184> 2014-08-13 17:38:05.330379 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.291 -5183> 2014-08-13 17:38:05.330371 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=8] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.10 -5182> 2014-08-13 17:38:05.330608 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.293 192.168.1.109:6816/33855 7 ==== pg_notify(22.3d9(49) epoch 194487) v5 ==== 3377+0+0 (570058746 0 0) 0x5f055c0 con 0x4ec89a0 -5181> 2014-08-13 17:38:05.330621 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5180> 2014-08-13 17:38:05.330622 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5179> 2014-08-13 17:38:05.330623 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f055c0 pg_notify(22.3d9(49) epoch 194487) v5 -5178> 2014-08-13 17:38:05.330627 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330507, event: header_read, request: pg_notify(22.3d9(49) epoch 194487) v5 -5177> 2014-08-13 17:38:05.330632 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330509, event: throttled, request: pg_notify(22.3d9(49) epoch 194487) v5 -5176> 2014-08-13 17:38:05.330637 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330595, event: all_read, request: pg_notify(22.3d9(49) epoch 194487) v5 -5175> 2014-08-13 17:38:05.330641 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330620, event: dispatched, request: pg_notify(22.3d9(49) epoch 194487) v5 -5174> 2014-08-13 17:38:05.330645 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330645, event: waiting_for_osdmap, request: pg_notify(22.3d9(49) epoch 194487) v5 -5173> 2014-08-13 17:38:05.330648 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.293 -5172> 2014-08-13 17:38:05.330650 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f055c0 -5171> 2014-08-13 17:38:05.330653 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330653, event: started, request: pg_notify(22.3d9(49) epoch 194487) v5 -5170> 2014-08-13 17:38:05.330517 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.291 -5169> 2014-08-13 17:38:05.330670 7fe9a63f4700 5 -- op tracker -- , seq: 579, time: 2014-08-13 17:38:05.330670, event: done, request: pg_notify(22.3d9(49) epoch 194487) v5 -5168> 2014-08-13 17:38:05.330523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=7] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.10 -5167> 2014-08-13 17:38:05.330677 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5166> 2014-08-13 17:38:05.330679 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5165> 2014-08-13 17:38:05.330658 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.291 -5164> 2014-08-13 17:38:05.330677 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=6] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.10 -5163> 2014-08-13 17:38:05.330837 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=5] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.10 -5162> 2014-08-13 17:38:05.330997 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=4] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.10 -5161> 2014-08-13 17:38:05.331158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=3] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.10 -5160> 2014-08-13 17:38:05.330802 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.291 -5159> 2014-08-13 17:38:05.331328 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.291 -5158> 2014-08-13 17:38:05.331324 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=2] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.10 -5157> 2014-08-13 17:38:05.331480 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.291 -5156> 2014-08-13 17:38:05.331488 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=1] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.10 -5155> 2014-08-13 17:38:05.331628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.291 -5154> 2014-08-13 17:38:05.331654 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5153> 2014-08-13 17:38:05.331778 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.291 -5152> 2014-08-13 17:38:05.331860 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 286 as a source -5151> 2014-08-13 17:38:05.331929 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.291 -5150> 2014-08-13 17:38:05.332028 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.286 (past last_backfill 0//0//-1) -5149> 2014-08-13 17:38:05.332083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.291 -5148> 2014-08-13 17:38:05.332198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.286 (past last_backfill 0//0//-1) -5147> 2014-08-13 17:38:05.332241 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.291 -5146> 2014-08-13 17:38:05.332365 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.286 (past last_backfill 0//0//-1) -5145> 2014-08-13 17:38:05.332400 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.291 -5144> 2014-08-13 17:38:05.332532 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.286 (past last_backfill 0//0//-1) -5143> 2014-08-13 17:38:05.332556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.291 -5142> 2014-08-13 17:38:05.332708 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.314 192.168.1.101:6802/22303 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d921c0 con 0x4ecd120 -5141> 2014-08-13 17:38:05.332725 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5140> 2014-08-13 17:38:05.332729 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5139> 2014-08-13 17:38:05.332731 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d921c0 osd_map(194487..194487 src has 185139..194487) v3 -5138> 2014-08-13 17:38:05.332736 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5137> 2014-08-13 17:38:05.332739 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5136> 2014-08-13 17:38:05.332741 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5135> 2014-08-13 17:38:05.332742 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5134> 2014-08-13 17:38:05.332831 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.314 192.168.1.101:6802/22303 7 ==== pg_notify(22.619(31) epoch 194487) v5 ==== 2455+0+0 (2275569936 0 0) 0x5f05b00 con 0x4ecd120 -5133> 2014-08-13 17:38:05.332840 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5132> 2014-08-13 17:38:05.332842 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5131> 2014-08-13 17:38:05.332843 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f05b00 pg_notify(22.619(31) epoch 194487) v5 -5130> 2014-08-13 17:38:05.332846 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332742, event: header_read, request: pg_notify(22.619(31) epoch 194487) v5 -5129> 2014-08-13 17:38:05.332852 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332744, event: throttled, request: pg_notify(22.619(31) epoch 194487) v5 -5128> 2014-08-13 17:38:05.332855 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332817, event: all_read, request: pg_notify(22.619(31) epoch 194487) v5 -5127> 2014-08-13 17:38:05.332859 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332840, event: dispatched, request: pg_notify(22.619(31) epoch 194487) v5 -5126> 2014-08-13 17:38:05.332698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.286 (past last_backfill 0//0//-1) -5125> 2014-08-13 17:38:05.332863 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332863, event: waiting_for_osdmap, request: pg_notify(22.619(31) epoch 194487) v5 -5124> 2014-08-13 17:38:05.332867 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.314 -5123> 2014-08-13 17:38:05.332869 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f05b00 -5122> 2014-08-13 17:38:05.332715 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.291 -5121> 2014-08-13 17:38:05.332872 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332872, event: started, request: pg_notify(22.619(31) epoch 194487) v5 -5120> 2014-08-13 17:38:05.332888 7fe9a63f4700 5 -- op tracker -- , seq: 580, time: 2014-08-13 17:38:05.332888, event: done, request: pg_notify(22.619(31) epoch 194487) v5 -5119> 2014-08-13 17:38:05.332896 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5118> 2014-08-13 17:38:05.332898 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5117> 2014-08-13 17:38:05.332863 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.286 (past last_backfill 0//0//-1) -5116> 2014-08-13 17:38:05.332876 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.291 -5115> 2014-08-13 17:38:05.333115 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 14 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5484800 con 0x4ec8000 -5114> 2014-08-13 17:38:05.333130 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5113> 2014-08-13 17:38:05.333132 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5112> 2014-08-13 17:38:05.333134 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484800 osd_map(194487..194487 src has 185139..194487) v3 -5111> 2014-08-13 17:38:05.333138 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5110> 2014-08-13 17:38:05.333141 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5109> 2014-08-13 17:38:05.333142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5108> 2014-08-13 17:38:05.333143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5107> 2014-08-13 17:38:05.333029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.286 (past last_backfill 0//0//-1) -5106> 2014-08-13 17:38:05.333038 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.291 -5105> 2014-08-13 17:38:05.333252 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 15 ==== pg_notify(22.39d(29) epoch 194487) v5 ==== 2341+0+0 (3134800639 0 0) 0x5103800 con 0x4ec8000 -5104> 2014-08-13 17:38:05.333264 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5103> 2014-08-13 17:38:05.333266 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5102> 2014-08-13 17:38:05.333267 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103800 pg_notify(22.39d(29) epoch 194487) v5 -5101> 2014-08-13 17:38:05.333271 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333139, event: header_read, request: pg_notify(22.39d(29) epoch 194487) v5 -5100> 2014-08-13 17:38:05.333275 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333140, event: throttled, request: pg_notify(22.39d(29) epoch 194487) v5 -5099> 2014-08-13 17:38:05.333279 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333237, event: all_read, request: pg_notify(22.39d(29) epoch 194487) v5 -5098> 2014-08-13 17:38:05.333282 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333263, event: dispatched, request: pg_notify(22.39d(29) epoch 194487) v5 -5097> 2014-08-13 17:38:05.333286 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333286, event: waiting_for_osdmap, request: pg_notify(22.39d(29) epoch 194487) v5 -5096> 2014-08-13 17:38:05.333289 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.320 -5095> 2014-08-13 17:38:05.333291 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103800 -5094> 2014-08-13 17:38:05.333294 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333294, event: started, request: pg_notify(22.39d(29) epoch 194487) v5 -5093> 2014-08-13 17:38:05.333309 7fe9a63f4700 5 -- op tracker -- , seq: 581, time: 2014-08-13 17:38:05.333309, event: done, request: pg_notify(22.39d(29) epoch 194487) v5 -5092> 2014-08-13 17:38:05.333316 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5091> 2014-08-13 17:38:05.333318 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5090> 2014-08-13 17:38:05.333324 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 19 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9d100 con 0x4e4d280 -5089> 2014-08-13 17:38:05.333332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5088> 2014-08-13 17:38:05.333333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5087> 2014-08-13 17:38:05.333334 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9d100 osd_map(194487..194487 src has 185139..194487) v3 -5086> 2014-08-13 17:38:05.333339 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5085> 2014-08-13 17:38:05.333342 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5084> 2014-08-13 17:38:05.333344 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5083> 2014-08-13 17:38:05.333345 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5082> 2014-08-13 17:38:05.333197 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.286 (past last_backfill 0//0//-1) -5081> 2014-08-13 17:38:05.333509 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 20 ==== pg_notify(24.1f7(34) epoch 194487) v5 ==== 2662+0+0 (2086247411 0 0) 0x5820e00 con 0x4e4d280 -5080> 2014-08-13 17:38:05.333529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5079> 2014-08-13 17:38:05.333531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5078> 2014-08-13 17:38:05.333533 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_notify(24.1f7(34) epoch 194487) v5 -5077> 2014-08-13 17:38:05.333537 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333419, event: header_read, request: pg_notify(24.1f7(34) epoch 194487) v5 -5076> 2014-08-13 17:38:05.333543 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333421, event: throttled, request: pg_notify(24.1f7(34) epoch 194487) v5 -5075> 2014-08-13 17:38:05.333382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.286 (past last_backfill 0//0//-1) -5074> 2014-08-13 17:38:05.333546 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333492, event: all_read, request: pg_notify(24.1f7(34) epoch 194487) v5 -5073> 2014-08-13 17:38:05.333550 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333528, event: dispatched, request: pg_notify(24.1f7(34) epoch 194487) v5 -5072> 2014-08-13 17:38:05.333555 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333554, event: waiting_for_osdmap, request: pg_notify(24.1f7(34) epoch 194487) v5 -5071> 2014-08-13 17:38:05.333561 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.376 -5070> 2014-08-13 17:38:05.333563 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 -5069> 2014-08-13 17:38:05.333586 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333586, event: started, request: pg_notify(24.1f7(34) epoch 194487) v5 -5068> 2014-08-13 17:38:05.333603 7fe9a63f4700 5 -- op tracker -- , seq: 582, time: 2014-08-13 17:38:05.333603, event: done, request: pg_notify(24.1f7(34) epoch 194487) v5 -5067> 2014-08-13 17:38:05.333611 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5066> 2014-08-13 17:38:05.333613 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5065> 2014-08-13 17:38:05.333205 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.291 -5064> 2014-08-13 17:38:05.333548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.286 (past last_backfill 0//0//-1) -5063> 2014-08-13 17:38:05.333643 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.291 -5062> 2014-08-13 17:38:05.333715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.286 (past last_backfill 0//0//-1) -5061> 2014-08-13 17:38:05.333816 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.291 -5060> 2014-08-13 17:38:05.333882 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.286 (past last_backfill 0//0//-1) -5059> 2014-08-13 17:38:05.333987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.291 -5058> 2014-08-13 17:38:05.334047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.286 (past last_backfill 0//0//-1) -5057> 2014-08-13 17:38:05.334287 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.320 192.168.1.106:6820/10050 16 ==== pg_notify(24.181(12) epoch 194487) v5 ==== 1432+0+0 (2252251829 0 0) 0x5f063c0 con 0x4ec8000 -5056> 2014-08-13 17:38:05.334305 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5055> 2014-08-13 17:38:05.334307 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5054> 2014-08-13 17:38:05.334308 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f063c0 pg_notify(24.181(12) epoch 194487) v5 -5053> 2014-08-13 17:38:05.334312 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.333820, event: header_read, request: pg_notify(24.181(12) epoch 194487) v5 -5052> 2014-08-13 17:38:05.334317 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.333821, event: throttled, request: pg_notify(24.181(12) epoch 194487) v5 -5051> 2014-08-13 17:38:05.334320 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.333876, event: all_read, request: pg_notify(24.181(12) epoch 194487) v5 -5050> 2014-08-13 17:38:05.334323 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.334304, event: dispatched, request: pg_notify(24.181(12) epoch 194487) v5 -5049> 2014-08-13 17:38:05.334327 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.334327, event: waiting_for_osdmap, request: pg_notify(24.181(12) epoch 194487) v5 -5048> 2014-08-13 17:38:05.334331 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.320 -5047> 2014-08-13 17:38:05.334333 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f063c0 -5046> 2014-08-13 17:38:05.334159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.291 -5045> 2014-08-13 17:38:05.334336 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.334335, event: started, request: pg_notify(24.181(12) epoch 194487) v5 -5044> 2014-08-13 17:38:05.334360 7fe9a63f4700 5 -- op tracker -- , seq: 583, time: 2014-08-13 17:38:05.334360, event: done, request: pg_notify(24.181(12) epoch 194487) v5 -5043> 2014-08-13 17:38:05.334368 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5042> 2014-08-13 17:38:05.334370 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5041> 2014-08-13 17:38:05.334215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.286 (past last_backfill 0//0//-1) -5040> 2014-08-13 17:38:05.334381 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.376 192.168.1.108:6838/6893 21 ==== pg_notify(22.48e(29) epoch 194487) v5 ==== 2341+0+0 (2024069630 0 0) 0x5f06c80 con 0x4e4d280 -5039> 2014-08-13 17:38:05.334390 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5038> 2014-08-13 17:38:05.334391 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5037> 2014-08-13 17:38:05.334392 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f06c80 pg_notify(22.48e(29) epoch 194487) v5 -5036> 2014-08-13 17:38:05.334395 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.333854, event: header_read, request: pg_notify(22.48e(29) epoch 194487) v5 -5035> 2014-08-13 17:38:05.334400 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.333856, event: throttled, request: pg_notify(22.48e(29) epoch 194487) v5 -5034> 2014-08-13 17:38:05.334404 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.333937, event: all_read, request: pg_notify(22.48e(29) epoch 194487) v5 -5033> 2014-08-13 17:38:05.334407 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.334390, event: dispatched, request: pg_notify(22.48e(29) epoch 194487) v5 -5032> 2014-08-13 17:38:05.334411 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.334411, event: waiting_for_osdmap, request: pg_notify(22.48e(29) epoch 194487) v5 -5031> 2014-08-13 17:38:05.334414 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.376 -5030> 2014-08-13 17:38:05.334416 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f06c80 -5029> 2014-08-13 17:38:05.334418 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.334418, event: started, request: pg_notify(22.48e(29) epoch 194487) v5 -5028> 2014-08-13 17:38:05.334430 7fe9a63f4700 5 -- op tracker -- , seq: 584, time: 2014-08-13 17:38:05.334430, event: done, request: pg_notify(22.48e(29) epoch 194487) v5 -5027> 2014-08-13 17:38:05.334437 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5026> 2014-08-13 17:38:05.334438 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5025> 2014-08-13 17:38:05.334379 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.286 (past last_backfill 0//0//-1) -5024> 2014-08-13 17:38:05.334337 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5023> 2014-08-13 17:38:05.334545 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.286 (past last_backfill 0//0//-1) -5022> 2014-08-13 17:38:05.334561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 350 as a source -5021> 2014-08-13 17:38:05.334712 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.286 (past last_backfill 0//0//-1) -5020> 2014-08-13 17:38:05.334737 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.350 -5019> 2014-08-13 17:38:05.334878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.286 (past last_backfill 0//0//-1) -5018> 2014-08-13 17:38:05.334912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.350 -5017> 2014-08-13 17:38:05.335045 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.286 (past last_backfill 0//0//-1) -5016> 2014-08-13 17:38:05.335088 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.350 -5015> 2014-08-13 17:38:05.335214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.286 (past last_backfill 0//0//-1) -5014> 2014-08-13 17:38:05.335269 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.350 -5013> 2014-08-13 17:38:05.335498 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.301 192.168.1.113:6814/16727 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9bcc0 con 0x4a4e1a0 -5012> 2014-08-13 17:38:05.335519 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5011> 2014-08-13 17:38:05.335522 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5010> 2014-08-13 17:38:05.335524 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9bcc0 osd_map(194487..194487 src has 185139..194487) v3 -5009> 2014-08-13 17:38:05.335529 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -5008> 2014-08-13 17:38:05.335533 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -5007> 2014-08-13 17:38:05.335534 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5006> 2014-08-13 17:38:05.335535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5005> 2014-08-13 17:38:05.335381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.286 (past last_backfill 0//0//-1) -5004> 2014-08-13 17:38:05.335585 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.301 192.168.1.113:6814/16727 10 ==== pg_notify(24.30b(2) epoch 194487) v5 ==== 870+0+0 (3245418543 0 0) 0x51ce200 con 0x4a4e1a0 -5003> 2014-08-13 17:38:05.335602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -5002> 2014-08-13 17:38:05.335604 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -5001> 2014-08-13 17:38:05.335605 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce200 pg_notify(24.30b(2) epoch 194487) v5 -5000> 2014-08-13 17:38:05.335609 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335495, event: header_read, request: pg_notify(24.30b(2) epoch 194487) v5 -4999> 2014-08-13 17:38:05.335614 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335498, event: throttled, request: pg_notify(24.30b(2) epoch 194487) v5 -4998> 2014-08-13 17:38:05.335445 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.350 -4997> 2014-08-13 17:38:05.335618 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335548, event: all_read, request: pg_notify(24.30b(2) epoch 194487) v5 -4996> 2014-08-13 17:38:05.335622 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335601, event: dispatched, request: pg_notify(24.30b(2) epoch 194487) v5 -4995> 2014-08-13 17:38:05.335626 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335626, event: waiting_for_osdmap, request: pg_notify(24.30b(2) epoch 194487) v5 -4994> 2014-08-13 17:38:05.335630 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.301 -4993> 2014-08-13 17:38:05.335632 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce200 -4992> 2014-08-13 17:38:05.335636 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.335635, event: started, request: pg_notify(24.30b(2) epoch 194487) v5 -4991> 2014-08-13 17:38:05.335546 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.286 (past last_backfill 0//0//-1) -4990> 2014-08-13 17:38:05.335620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.350 -4989> 2014-08-13 17:38:05.335710 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.286 (past last_backfill 0//0//-1) -4988> 2014-08-13 17:38:05.335796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.350 -4987> 2014-08-13 17:38:05.335876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.286 (past last_backfill 0//0//-1) -4986> 2014-08-13 17:38:05.335972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.350 -4985> 2014-08-13 17:38:05.336042 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.286 (past last_backfill 0//0//-1) -4984> 2014-08-13 17:38:05.336148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.350 -4983> 2014-08-13 17:38:05.336211 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.286 (past last_backfill 0//0//-1) -4982> 2014-08-13 17:38:05.336331 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.350 -4981> 2014-08-13 17:38:05.336377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.286 (past last_backfill 0//0//-1) -4980> 2014-08-13 17:38:05.336508 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.350 -4979> 2014-08-13 17:38:05.336543 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.286 (past last_backfill 0//0//-1) -4978> 2014-08-13 17:38:05.336684 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.350 -4977> 2014-08-13 17:38:05.336708 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.286 (past last_backfill 0//0//-1) -4976> 2014-08-13 17:38:05.336860 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.350 -4975> 2014-08-13 17:38:05.336872 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.286 (past last_backfill 0//0//-1) -4974> 2014-08-13 17:38:05.337042 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.286 (past last_backfill 0//0//-1) -4973> 2014-08-13 17:38:05.337036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.350 -4972> 2014-08-13 17:38:05.337214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.286 (past last_backfill 0//0//-1) -4971> 2014-08-13 17:38:05.337216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.350 -4970> 2014-08-13 17:38:05.337381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.286 (past last_backfill 0//0//-1) -4969> 2014-08-13 17:38:05.337391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.350 -4968> 2014-08-13 17:38:05.337547 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.286 (past last_backfill 0//0//-1) -4967> 2014-08-13 17:38:05.337567 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.350 -4966> 2014-08-13 17:38:05.337713 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.286 (past last_backfill 0//0//-1) -4965> 2014-08-13 17:38:05.337743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.350 -4964> 2014-08-13 17:38:05.337880 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.286 (past last_backfill 0//0//-1) -4963> 2014-08-13 17:38:05.337920 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.350 -4962> 2014-08-13 17:38:05.338046 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.286 (past last_backfill 0//0//-1) -4961> 2014-08-13 17:38:05.338095 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.350 -4960> 2014-08-13 17:38:05.338215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.286 (past last_backfill 0//0//-1) -4959> 2014-08-13 17:38:05.338274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.350 -4958> 2014-08-13 17:38:05.338382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.286 (past last_backfill 0//0//-1) -4957> 2014-08-13 17:38:05.338453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.350 -4956> 2014-08-13 17:38:05.338548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.286 (past last_backfill 0//0//-1) -4955> 2014-08-13 17:38:05.338629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.350 -4954> 2014-08-13 17:38:05.338714 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.286 (past last_backfill 0//0//-1) -4953> 2014-08-13 17:38:05.338804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.350 -4952> 2014-08-13 17:38:05.338880 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.286 (past last_backfill 0//0//-1) -4951> 2014-08-13 17:38:05.338980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.350 -4950> 2014-08-13 17:38:05.339047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.286 (past last_backfill 0//0//-1) -4949> 2014-08-13 17:38:05.339156 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.350 -4948> 2014-08-13 17:38:05.339217 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.286 (past last_backfill 0//0//-1) -4947> 2014-08-13 17:38:05.339335 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.350 -4946> 2014-08-13 17:38:05.339383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.286 (past last_backfill 0//0//-1) -4945> 2014-08-13 17:38:05.339512 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.350 -4944> 2014-08-13 17:38:05.339549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.286 (past last_backfill 0//0//-1) -4943> 2014-08-13 17:38:05.339687 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.350 -4942> 2014-08-13 17:38:05.339715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.286 (past last_backfill 0//0//-1) -4941> 2014-08-13 17:38:05.339863 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.350 -4940> 2014-08-13 17:38:05.339883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.286 (past last_backfill 0//0//-1) -4939> 2014-08-13 17:38:05.340039 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.350 -4938> 2014-08-13 17:38:05.340048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.286 (past last_backfill 0//0//-1) -4937> 2014-08-13 17:38:05.340220 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.350 -4936> 2014-08-13 17:38:05.340234 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.286 (past last_backfill 0//0//-1) -4935> 2014-08-13 17:38:05.340396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.350 -4934> 2014-08-13 17:38:05.340406 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.286 (past last_backfill 0//0//-1) -4933> 2014-08-13 17:38:05.340572 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.286 (past last_backfill 0//0//-1) -4932> 2014-08-13 17:38:05.340573 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.350 -4931> 2014-08-13 17:38:05.340739 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.286 (past last_backfill 0//0//-1) -4930> 2014-08-13 17:38:05.340748 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.350 -4929> 2014-08-13 17:38:05.340922 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.350 -4928> 2014-08-13 17:38:05.340906 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.286 (past last_backfill 0//0//-1) -4927> 2014-08-13 17:38:05.341096 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.350 -4926> 2014-08-13 17:38:05.341131 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.286 (past last_backfill 0//0//-1) -4925> 2014-08-13 17:38:05.341275 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.350 -4924> 2014-08-13 17:38:05.341404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.286 (past last_backfill 0//0//-1) -4923> 2014-08-13 17:38:05.341452 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.350 -4922> 2014-08-13 17:38:05.341574 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -4921> 2014-08-13 17:38:05.341627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.350 -4920> 2014-08-13 17:38:05.341780 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 375 as a source -4919> 2014-08-13 17:38:05.341802 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.350 -4918> 2014-08-13 17:38:05.341948 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.375 -4917> 2014-08-13 17:38:05.341976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.350 -4916> 2014-08-13 17:38:05.342117 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.375 -4915> 2014-08-13 17:38:05.342150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.350 -4914> 2014-08-13 17:38:05.342287 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.375 -4913> 2014-08-13 17:38:05.342327 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.350 -4912> 2014-08-13 17:38:05.342453 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.375 -4911> 2014-08-13 17:38:05.342502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.350 -4910> 2014-08-13 17:38:05.342619 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.375 -4909> 2014-08-13 17:38:05.342676 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.350 -4908> 2014-08-13 17:38:05.342784 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.375 -4907> 2014-08-13 17:38:05.342850 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.350 -4906> 2014-08-13 17:38:05.342949 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.375 -4905> 2014-08-13 17:38:05.343024 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.350 -4904> 2014-08-13 17:38:05.343114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.375 -4903> 2014-08-13 17:38:05.343201 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.350 -4902> 2014-08-13 17:38:05.343283 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.375 -4901> 2014-08-13 17:38:05.343376 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.350 -4900> 2014-08-13 17:38:05.343451 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.375 -4899> 2014-08-13 17:38:05.343552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.350 -4898> 2014-08-13 17:38:05.343617 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.375 -4897> 2014-08-13 17:38:05.343729 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.350 -4896> 2014-08-13 17:38:05.343794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.375 -4895> 2014-08-13 17:38:05.343903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.350 -4894> 2014-08-13 17:38:05.343959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.375 -4893> 2014-08-13 17:38:05.344077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.350 -4892> 2014-08-13 17:38:05.344124 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.375 -4891> 2014-08-13 17:38:05.344254 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.350 -4890> 2014-08-13 17:38:05.344293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.375 -4889> 2014-08-13 17:38:05.344430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.350 -4888> 2014-08-13 17:38:05.344458 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.375 -4887> 2014-08-13 17:38:05.344606 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.350 -4886> 2014-08-13 17:38:05.344624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.375 -4885> 2014-08-13 17:38:05.344789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.375 -4884> 2014-08-13 17:38:05.344782 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.350 -4883> 2014-08-13 17:38:05.344956 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.375 -4882> 2014-08-13 17:38:05.344958 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.350 -4881> 2014-08-13 17:38:05.345123 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.375 -4880> 2014-08-13 17:38:05.345137 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -4879> 2014-08-13 17:38:05.345293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.375 -4878> 2014-08-13 17:38:05.345359 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 367 as a source -4877> 2014-08-13 17:38:05.345460 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.375 -4876> 2014-08-13 17:38:05.345535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.367 (past last_backfill 0//0//-1) -4875> 2014-08-13 17:38:05.345627 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.375 -4874> 2014-08-13 17:38:05.345711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.367 (past last_backfill 0//0//-1) -4873> 2014-08-13 17:38:05.345794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.375 -4872> 2014-08-13 17:38:05.345887 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.367 (past last_backfill 0//0//-1) -4871> 2014-08-13 17:38:05.345961 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.375 -4870> 2014-08-13 17:38:05.346063 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.367 (past last_backfill 0//0//-1) -4869> 2014-08-13 17:38:05.346128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.375 -4868> 2014-08-13 17:38:05.346242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.367 (past last_backfill 0//0//-1) -4867> 2014-08-13 17:38:05.346297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.375 -4866> 2014-08-13 17:38:05.346418 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.367 (past last_backfill 0//0//-1) -4865> 2014-08-13 17:38:05.346464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.375 -4864> 2014-08-13 17:38:05.346593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.367 (past last_backfill 0//0//-1) -4863> 2014-08-13 17:38:05.346631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.375 -4862> 2014-08-13 17:38:05.346768 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.367 (past last_backfill 0//0//-1) -4861> 2014-08-13 17:38:05.346798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.375 -4860> 2014-08-13 17:38:05.346945 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.367 (past last_backfill 0//0//-1) -4859> 2014-08-13 17:38:05.346965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.375 -4858> 2014-08-13 17:38:05.347120 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.367 (past last_backfill 0//0//-1) -4857> 2014-08-13 17:38:05.347131 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.375 -4856> 2014-08-13 17:38:05.347300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.367 (past last_backfill 0//0//-1) -4855> 2014-08-13 17:38:05.347320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.375 -4854> 2014-08-13 17:38:05.347476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.367 (past last_backfill 0//0//-1) -4853> 2014-08-13 17:38:05.347492 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.375 -4852> 2014-08-13 17:38:05.347658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.375 -4851> 2014-08-13 17:38:05.347651 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.367 (past last_backfill 0//0//-1) -4850> 2014-08-13 17:38:05.347825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.367 (past last_backfill 0//0//-1) -4849> 2014-08-13 17:38:05.347826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.375 -4848> 2014-08-13 17:38:05.348001 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.367 (past last_backfill 0//0//-1) -4847> 2014-08-13 17:38:05.348111 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.375 -4846> 2014-08-13 17:38:05.348176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.367 (past last_backfill 0//0//-1) -4845> 2014-08-13 17:38:05.348288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.375 -4844> 2014-08-13 17:38:05.348354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.367 (past last_backfill 0//0//-1) -4843> 2014-08-13 17:38:05.348456 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.375 -4842> 2014-08-13 17:38:05.348529 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.367 (past last_backfill 0//0//-1) -4841> 2014-08-13 17:38:05.348624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.375 -4840> 2014-08-13 17:38:05.348702 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.367 (past last_backfill 0//0//-1) -4839> 2014-08-13 17:38:05.348790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.375 -4838> 2014-08-13 17:38:05.348892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.367 (past last_backfill 0//0//-1) -4837> 2014-08-13 17:38:05.348990 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.375 -4836> 2014-08-13 17:38:05.349067 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.367 (past last_backfill 0//0//-1) -4835> 2014-08-13 17:38:05.349158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.375 -4834> 2014-08-13 17:38:05.349330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.375 -4833> 2014-08-13 17:38:05.349247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.367 (past last_backfill 0//0//-1) -4832> 2014-08-13 17:38:05.349499 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.375 -4831> 2014-08-13 17:38:05.349563 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.367 (past last_backfill 0//0//-1) -4830> 2014-08-13 17:38:05.349666 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.375 -4829> 2014-08-13 17:38:05.349743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.367 (past last_backfill 0//0//-1) -4828> 2014-08-13 17:38:05.349839 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.375 -4827> 2014-08-13 17:38:05.349918 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.367 (past last_backfill 0//0//-1) -4826> 2014-08-13 17:38:05.350007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.375 -4825> 2014-08-13 17:38:05.350093 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.367 (past last_backfill 0//0//-1) -4824> 2014-08-13 17:38:05.350174 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.375 -4823> 2014-08-13 17:38:05.350273 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.367 (past last_backfill 0//0//-1) -4822> 2014-08-13 17:38:05.350345 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.375 -4821> 2014-08-13 17:38:05.350448 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.367 (past last_backfill 0//0//-1) -4820> 2014-08-13 17:38:05.350512 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.375 -4819> 2014-08-13 17:38:05.350624 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.367 (past last_backfill 0//0//-1) -4818> 2014-08-13 17:38:05.350683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.375 -4817> 2014-08-13 17:38:05.350799 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.367 (past last_backfill 0//0//-1) -4816> 2014-08-13 17:38:05.350853 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.375 -4815> 2014-08-13 17:38:05.350973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.367 (past last_backfill 0//0//-1) -4814> 2014-08-13 17:38:05.351020 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.375 -4813> 2014-08-13 17:38:05.351148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.367 (past last_backfill 0//0//-1) -4812> 2014-08-13 17:38:05.351189 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.375 -4811> 2014-08-13 17:38:05.351326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.367 (past last_backfill 0//0//-1) -4810> 2014-08-13 17:38:05.351356 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.375 -4809> 2014-08-13 17:38:05.351502 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.367 (past last_backfill 0//0//-1) -4808> 2014-08-13 17:38:05.351523 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -4807> 2014-08-13 17:38:05.351677 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.367 (past last_backfill 0//0//-1) -4806> 2014-08-13 17:38:05.351730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound -4805> 2014-08-13 17:38:05.351852 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.367 (past last_backfill 0//0//-1) -4804> 2014-08-13 17:38:05.351896 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _calc_past_interval_range: already have past intervals back to 194484 -4803> 2014-08-13 17:38:05.352027 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.367 (past last_backfill 0//0//-1) -4802> 2014-08-13 17:38:05.352064 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound: built 10,286,301,375 -4801> 2014-08-13 17:38:05.352204 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.367 (past last_backfill 0//0//-1) -4800> 2014-08-13 17:38:05.352235 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Activate Finished -4799> 2014-08-13 17:38:05.352380 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.367 (past last_backfill 0//0//-1) -4798> 2014-08-13 17:38:05.352404 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating -4797> 2014-08-13 17:38:05.352555 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.367 (past last_backfill 0//0//-1) -4796> 2014-08-13 17:38:05.352572 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4795> 2014-08-13 17:38:05.352737 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4794> 2014-08-13 17:38:05.352730 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.367 (past last_backfill 0//0//-1) -4793> 2014-08-13 17:38:05.352963 7fe99f5e9700 10 log is not dirty -4792> 2014-08-13 17:38:05.353007 7fe9a63f4700 5 -- op tracker -- , seq: 585, time: 2014-08-13 17:38:05.353007, event: done, request: pg_notify(24.30b(2) epoch 194487) v5 -4791> 2014-08-13 17:38:05.353010 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 -4790> 2014-08-13 17:38:05.353028 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4789> 2014-08-13 17:38:05.353031 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4788> 2014-08-13 17:38:05.353030 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4787> 2014-08-13 17:38:05.353035 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap -4786> 2014-08-13 17:38:05.353037 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.63 192.168.1.111:6830/12680 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dda00 con 0x52502c0 -4785> 2014-08-13 17:38:05.353049 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4784> 2014-08-13 17:38:05.353051 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4783> 2014-08-13 17:38:05.353052 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dda00 osd_map(194487..194487 src has 185139..194487) v3 -4782> 2014-08-13 17:38:05.353048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down -4781> 2014-08-13 17:38:05.353058 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4780> 2014-08-13 17:38:05.353066 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4779> 2014-08-13 17:38:05.353067 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4778> 2014-08-13 17:38:05.353060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map -4777> 2014-08-13 17:38:05.353069 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4776> 2014-08-13 17:38:05.352903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.367 (past last_backfill 0//0//-1) -4775> 2014-08-13 17:38:05.353070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters -4774> 2014-08-13 17:38:05.353072 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.382 192.168.1.108:6800/30527 5 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54df2c0 con 0x53853e0 -4773> 2014-08-13 17:38:05.353080 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4772> 2014-08-13 17:38:05.353082 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4771> 2014-08-13 17:38:05.353083 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df2c0 osd_map(194487..194487 src has 185139..194487) v3 -4770> 2014-08-13 17:38:05.353086 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4769> 2014-08-13 17:38:05.353079 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4768> 2014-08-13 17:38:05.353089 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4767> 2014-08-13 17:38:05.353091 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4766> 2014-08-13 17:38:05.353092 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4765> 2014-08-13 17:38:05.353090 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194485 pi=192970-194484/31 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4764> 2014-08-13 17:38:05.353101 7fe99f5e9700 10 log is not dirty -4763> 2014-08-13 17:38:05.353095 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.26 192.168.1.115:6802/58522 4 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5484ec0 con 0x592c360 -4762> 2014-08-13 17:38:05.353104 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4761> 2014-08-13 17:38:05.353106 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4760> 2014-08-13 17:38:05.353107 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484ec0 osd_map(194487..194487 src has 185139..194487) v3 -4759> 2014-08-13 17:38:05.353110 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4758> 2014-08-13 17:38:05.353112 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4757> 2014-08-13 17:38:05.353113 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4756> 2014-08-13 17:38:05.353114 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4755> 2014-08-13 17:38:05.353117 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.307 192.168.1.109:6804/31016 4 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db3600 con 0x5a432e0 -4754> 2014-08-13 17:38:05.353124 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4753> 2014-08-13 17:38:05.353125 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4752> 2014-08-13 17:38:05.353126 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3600 osd_map(194487..194487 src has 185139..194487) v3 -4751> 2014-08-13 17:38:05.353117 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 -4750> 2014-08-13 17:38:05.353129 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4749> 2014-08-13 17:38:05.353132 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4748> 2014-08-13 17:38:05.353132 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4747> 2014-08-13 17:38:05.353133 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4746> 2014-08-13 17:38:05.353135 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4745> 2014-08-13 17:38:05.353138 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.20 192.168.1.103:6815/38269 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9f500 con 0x4c71b80 -4744> 2014-08-13 17:38:05.353136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4743> 2014-08-13 17:38:05.353145 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4742> 2014-08-13 17:38:05.353146 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4741> 2014-08-13 17:38:05.353147 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9f500 osd_map(194487..194487 src has 185139..194487) v3 -4740> 2014-08-13 17:38:05.353150 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4739> 2014-08-13 17:38:05.353152 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4738> 2014-08-13 17:38:05.353153 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4737> 2014-08-13 17:38:05.353155 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4736> 2014-08-13 17:38:05.353146 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4735> 2014-08-13 17:38:05.353157 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.55 192.168.1.112:6834/6966 13 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9dc40 con 0x4d3dee0 -4734> 2014-08-13 17:38:05.353164 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4733> 2014-08-13 17:38:05.353166 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4732> 2014-08-13 17:38:05.353158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4731> 2014-08-13 17:38:05.353166 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9dc40 osd_map(194487..194487 src has 185139..194487) v3 -4730> 2014-08-13 17:38:05.353170 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4729> 2014-08-13 17:38:05.353172 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4728> 2014-08-13 17:38:05.353173 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4727> 2014-08-13 17:38:05.353174 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4726> 2014-08-13 17:38:05.353168 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4725> 2014-08-13 17:38:05.353177 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 19 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d92f40 con 0x4d3c780 -4724> 2014-08-13 17:38:05.353184 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4723> 2014-08-13 17:38:05.353185 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4722> 2014-08-13 17:38:05.353186 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92f40 osd_map(194487..194487 src has 185139..194487) v3 -4721> 2014-08-13 17:38:05.353189 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4720> 2014-08-13 17:38:05.353182 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4719> 2014-08-13 17:38:05.353191 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4718> 2014-08-13 17:38:05.353197 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4717> 2014-08-13 17:38:05.353198 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4716> 2014-08-13 17:38:05.353201 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db1200 con 0x4e3b020 -4715> 2014-08-13 17:38:05.353208 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4714> 2014-08-13 17:38:05.353209 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4713> 2014-08-13 17:38:05.353210 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db1200 osd_map(194487..194487 src has 185139..194487) v3 -4712> 2014-08-13 17:38:05.353213 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4711> 2014-08-13 17:38:05.353215 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4710> 2014-08-13 17:38:05.353216 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4709> 2014-08-13 17:38:05.353217 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4708> 2014-08-13 17:38:05.353219 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.24 192.168.1.107:6800/52669 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db6780 con 0x4e3ef60 -4707> 2014-08-13 17:38:05.353226 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4706> 2014-08-13 17:38:05.353227 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4705> 2014-08-13 17:38:05.353228 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db6780 osd_map(194487..194487 src has 185139..194487) v3 -4704> 2014-08-13 17:38:05.353231 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4703> 2014-08-13 17:38:05.353233 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4702> 2014-08-13 17:38:05.353234 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4701> 2014-08-13 17:38:05.353235 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4700> 2014-08-13 17:38:05.353238 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.397 192.168.1.105:6808/41085 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5484140 con 0x4e48840 -4699> 2014-08-13 17:38:05.353244 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4698> 2014-08-13 17:38:05.353245 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4697> 2014-08-13 17:38:05.353246 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5484140 osd_map(194487..194487 src has 185139..194487) v3 -4696> 2014-08-13 17:38:05.353249 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4695> 2014-08-13 17:38:05.353252 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4694> 2014-08-13 17:38:05.353252 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4693> 2014-08-13 17:38:05.353253 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4692> 2014-08-13 17:38:05.353077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.367 (past last_backfill 0//0//-1) -4691> 2014-08-13 17:38:05.353261 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.327 192.168.1.106:6804/5112 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d933c0 con 0x4e82520 -4690> 2014-08-13 17:38:05.353268 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4689> 2014-08-13 17:38:05.353270 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4688> 2014-08-13 17:38:05.353271 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d933c0 osd_map(194487..194487 src has 185139..194487) v3 -4687> 2014-08-13 17:38:05.353273 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4686> 2014-08-13 17:38:05.353276 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4685> 2014-08-13 17:38:05.353277 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4684> 2014-08-13 17:38:05.353278 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4683> 2014-08-13 17:38:05.353280 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.65 192.168.1.111:6842/17345 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d95e80 con 0x4e86040 -4682> 2014-08-13 17:38:05.353287 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4681> 2014-08-13 17:38:05.353288 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4680> 2014-08-13 17:38:05.353289 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d95e80 osd_map(194487..194487 src has 185139..194487) v3 -4679> 2014-08-13 17:38:05.353292 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4678> 2014-08-13 17:38:05.353294 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4677> 2014-08-13 17:38:05.353295 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4676> 2014-08-13 17:38:05.353296 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4675> 2014-08-13 17:38:05.353299 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.339 192.168.1.110:6818/45268 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db2f40 con 0x4ec8160 -4674> 2014-08-13 17:38:05.353305 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4673> 2014-08-13 17:38:05.353306 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4672> 2014-08-13 17:38:05.353307 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db2f40 osd_map(194487..194487 src has 185139..194487) v3 -4671> 2014-08-13 17:38:05.353310 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4670> 2014-08-13 17:38:05.353312 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4669> 2014-08-13 17:38:05.353313 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4668> 2014-08-13 17:38:05.353314 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4667> 2014-08-13 17:38:05.353317 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.48 192.168.1.112:6838/8300 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d92640 con 0x4ecf640 -4666> 2014-08-13 17:38:05.353324 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4665> 2014-08-13 17:38:05.353325 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4664> 2014-08-13 17:38:05.353326 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92640 osd_map(194487..194487 src has 185139..194487) v3 -4663> 2014-08-13 17:38:05.353329 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4662> 2014-08-13 17:38:05.353331 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4661> 2014-08-13 17:38:05.353332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4660> 2014-08-13 17:38:05.353333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4659> 2014-08-13 17:38:05.353330 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] -4658> 2014-08-13 17:38:05.353339 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4657> 2014-08-13 17:38:05.353342 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.302 192.168.1.113:6810/15731 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54d8fc0 con 0x4ecf380 -4656> 2014-08-13 17:38:05.353350 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4655> 2014-08-13 17:38:05.353351 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4654> 2014-08-13 17:38:05.353352 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54d8fc0 osd_map(194487..194487 src has 185139..194487) v3 -4653> 2014-08-13 17:38:05.353355 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4652> 2014-08-13 17:38:05.353358 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4651> 2014-08-13 17:38:05.353359 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4650> 2014-08-13 17:38:05.353360 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4649> 2014-08-13 17:38:05.353362 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.371 192.168.1.102:6832/16753 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54db600 con 0x41e5540 -4648> 2014-08-13 17:38:05.353369 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4647> 2014-08-13 17:38:05.353371 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4646> 2014-08-13 17:38:05.353371 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db600 osd_map(194487..194487 src has 185139..194487) v3 -4645> 2014-08-13 17:38:05.353374 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4644> 2014-08-13 17:38:05.353377 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4643> 2014-08-13 17:38:05.353378 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4642> 2014-08-13 17:38:05.353379 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4641> 2014-08-13 17:38:05.353381 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.351 192.168.1.102:6830/16444 11 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dc800 con 0x5250b00 -4640> 2014-08-13 17:38:05.353388 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4639> 2014-08-13 17:38:05.353389 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4638> 2014-08-13 17:38:05.353390 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc800 osd_map(194487..194487 src has 185139..194487) v3 -4637> 2014-08-13 17:38:05.353393 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4636> 2014-08-13 17:38:05.353395 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4635> 2014-08-13 17:38:05.353396 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4634> 2014-08-13 17:38:05.353397 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4633> 2014-08-13 17:38:05.353400 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.369 192.168.1.108:6824/4037 12 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dc380 con 0x5250840 -4632> 2014-08-13 17:38:05.353407 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4631> 2014-08-13 17:38:05.353408 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4630> 2014-08-13 17:38:05.353409 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc380 osd_map(194487..194487 src has 185139..194487) v3 -4629> 2014-08-13 17:38:05.353411 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4628> 2014-08-13 17:38:05.353414 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4627> 2014-08-13 17:38:05.353415 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4626> 2014-08-13 17:38:05.353416 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4625> 2014-08-13 17:38:05.353413 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2683 -4624> 2014-08-13 17:38:05.353419 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.63 192.168.1.111:6830/12680 7 ==== pg_notify(22.775(31) epoch 194487) v5 ==== 2455+0+0 (3269143235 0 0) 0x5f071c0 con 0x52502c0 -4623> 2014-08-13 17:38:05.353427 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4622> 2014-08-13 17:38:05.353428 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4621> 2014-08-13 17:38:05.353429 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f071c0 pg_notify(22.775(31) epoch 194487) v5 -4620> 2014-08-13 17:38:05.353423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4619> 2014-08-13 17:38:05.353433 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.337132, event: header_read, request: pg_notify(22.775(31) epoch 194487) v5 -4618> 2014-08-13 17:38:05.353438 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.337134, event: throttled, request: pg_notify(22.775(31) epoch 194487) v5 -4617> 2014-08-13 17:38:05.353267 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.367 (past last_backfill 0//0//-1) -4616> 2014-08-13 17:38:05.353433 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4615> 2014-08-13 17:38:05.353442 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.337211, event: all_read, request: pg_notify(22.775(31) epoch 194487) v5 -4614> 2014-08-13 17:38:05.353446 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.353427, event: dispatched, request: pg_notify(22.775(31) epoch 194487) v5 -4613> 2014-08-13 17:38:05.353443 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4612> 2014-08-13 17:38:05.353450 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.353450, event: waiting_for_osdmap, request: pg_notify(22.775(31) epoch 194487) v5 -4611> 2014-08-13 17:38:05.353454 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.63 -4610> 2014-08-13 17:38:05.353455 7fe99f5e9700 10 log is not dirty -4609> 2014-08-13 17:38:05.353456 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f071c0 -4608> 2014-08-13 17:38:05.353460 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.353460, event: started, request: pg_notify(22.775(31) epoch 194487) v5 -4607> 2014-08-13 17:38:05.353462 7fe99f5e9700 20 osd.4 194487 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 10 -4606> 2014-08-13 17:38:05.353470 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing -4605> 2014-08-13 17:38:05.353475 7fe9a63f4700 5 -- op tracker -- , seq: 586, time: 2014-08-13 17:38:05.353475, event: done, request: pg_notify(22.775(31) epoch 194487) v5 -4604> 2014-08-13 17:38:05.353483 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4603> 2014-08-13 17:38:05.353484 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4602> 2014-08-13 17:38:05.353496 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.115:6815/60573 -- pg_info(1 pgs e194487:24.30b(36)) v4 -- ?+0 0x513d400 con 0x49e09a0 -4601> 2014-08-13 17:38:05.353498 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.382 192.168.1.108:6800/30527 6 ==== pg_notify(22.3d9(49) epoch 194487) v5 ==== 3377+0+0 (2982170719 0 0) 0x5827c40 con 0x53853e0 -4600> 2014-08-13 17:38:05.353511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4599> 2014-08-13 17:38:05.353512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4598> 2014-08-13 17:38:05.353513 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5827c40 pg_notify(22.3d9(49) epoch 194487) v5 -4597> 2014-08-13 17:38:05.353512 7fe99f5e9700 20 osd.4 194487 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 375 -4596> 2014-08-13 17:38:05.353516 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.341269, event: header_read, request: pg_notify(22.3d9(49) epoch 194487) v5 -4595> 2014-08-13 17:38:05.353519 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing -4594> 2014-08-13 17:38:05.353520 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.341270, event: throttled, request: pg_notify(22.3d9(49) epoch 194487) v5 -4593> 2014-08-13 17:38:05.353524 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.341359, event: all_read, request: pg_notify(22.3d9(49) epoch 194487) v5 -4592> 2014-08-13 17:38:05.353527 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.353511, event: dispatched, request: pg_notify(22.3d9(49) epoch 194487) v5 -4591> 2014-08-13 17:38:05.353340 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -4590> 2014-08-13 17:38:05.353531 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.353531, event: waiting_for_osdmap, request: pg_notify(22.3d9(49) epoch 194487) v5 -4589> 2014-08-13 17:38:05.353534 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.382 -4588> 2014-08-13 17:38:05.353532 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.114:6830/52879 -- pg_info(1 pgs e194487:24.30b(36)) v4 -- ?+0 0x513dcc0 con 0x4313f40 -4587> 2014-08-13 17:38:05.353536 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5827c40 -4586> 2014-08-13 17:38:05.353540 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.353540, event: started, request: pg_notify(22.3d9(49) epoch 194487) v5 -4585> 2014-08-13 17:38:05.353576 7fe9a63f4700 5 -- op tracker -- , seq: 587, time: 2014-08-13 17:38:05.353576, event: done, request: pg_notify(22.3d9(49) epoch 194487) v5 -4584> 2014-08-13 17:38:05.353584 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4583> 2014-08-13 17:38:05.353586 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4582> 2014-08-13 17:38:05.353582 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 -4581> 2014-08-13 17:38:05.353604 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4580> 2014-08-13 17:38:05.353442 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.367 (past last_backfill 0//0//-1) -4579> 2014-08-13 17:38:05.353608 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Started advmap -4578> 2014-08-13 17:38:05.353610 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.26 192.168.1.115:6802/58522 5 ==== pg_notify(24.34(39) epoch 194487) v5 ==== 2733+0+0 (2533927597 0 0) 0x4c17000 con 0x592c360 -4577> 2014-08-13 17:38:05.353619 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4576> 2014-08-13 17:38:05.353621 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4575> 2014-08-13 17:38:05.353622 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c17000 pg_notify(24.34(39) epoch 194487) v5 -4574> 2014-08-13 17:38:05.353625 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.349501, event: header_read, request: pg_notify(24.34(39) epoch 194487) v5 -4573> 2014-08-13 17:38:05.353620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down -4572> 2014-08-13 17:38:05.353629 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.349501, event: throttled, request: pg_notify(24.34(39) epoch 194487) v5 -4571> 2014-08-13 17:38:05.353633 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.349571, event: all_read, request: pg_notify(24.34(39) epoch 194487) v5 -4570> 2014-08-13 17:38:05.353636 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.353619, event: dispatched, request: pg_notify(24.34(39) epoch 194487) v5 -4569> 2014-08-13 17:38:05.353631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map -4568> 2014-08-13 17:38:05.353640 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.353640, event: waiting_for_osdmap, request: pg_notify(24.34(39) epoch 194487) v5 -4567> 2014-08-13 17:38:05.353644 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.26 -4566> 2014-08-13 17:38:05.353645 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c17000 -4565> 2014-08-13 17:38:05.353648 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.353648, event: started, request: pg_notify(24.34(39) epoch 194487) v5 -4564> 2014-08-13 17:38:05.353661 7fe9a63f4700 5 -- op tracker -- , seq: 588, time: 2014-08-13 17:38:05.353661, event: done, request: pg_notify(24.34(39) epoch 194487) v5 -4563> 2014-08-13 17:38:05.353667 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4562> 2014-08-13 17:38:05.353668 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4561> 2014-08-13 17:38:05.353684 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.307 192.168.1.109:6804/31016 5 ==== pg_notify(24.181(40) epoch 194487) v5 ==== 3000+0+0 (4101414576 0 0) 0x5f039c0 con 0x5a432e0 -4560> 2014-08-13 17:38:05.353692 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4559> 2014-08-13 17:38:05.353694 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4558> 2014-08-13 17:38:05.353686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters -4557> 2014-08-13 17:38:05.353695 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f039c0 pg_notify(24.181(40) epoch 194487) v5 -4556> 2014-08-13 17:38:05.353698 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.348902, event: header_read, request: pg_notify(24.181(40) epoch 194487) v5 -4555> 2014-08-13 17:38:05.353534 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] flushed -4554> 2014-08-13 17:38:05.353697 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4553> 2014-08-13 17:38:05.353704 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.348903, event: throttled, request: pg_notify(24.181(40) epoch 194487) v5 -4552> 2014-08-13 17:38:05.353708 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.349020, event: all_read, request: pg_notify(24.181(40) epoch 194487) v5 -4551> 2014-08-13 17:38:05.353712 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.353692, event: dispatched, request: pg_notify(24.181(40) epoch 194487) v5 -4550> 2014-08-13 17:38:05.353707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4549> 2014-08-13 17:38:05.353715 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.353715, event: waiting_for_osdmap, request: pg_notify(24.181(40) epoch 194487) v5 -4548> 2014-08-13 17:38:05.353719 7fe99f5e9700 10 log is not dirty -4547> 2014-08-13 17:38:05.353719 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.307 -4546> 2014-08-13 17:38:05.353721 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f039c0 -4545> 2014-08-13 17:38:05.353723 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.353723, event: started, request: pg_notify(24.181(40) epoch 194487) v5 -4544> 2014-08-13 17:38:05.353734 7fe9a63f4700 5 -- op tracker -- , seq: 589, time: 2014-08-13 17:38:05.353734, event: done, request: pg_notify(24.181(40) epoch 194487) v5 -4543> 2014-08-13 17:38:05.353740 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4542> 2014-08-13 17:38:05.353742 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4541> 2014-08-13 17:38:05.353732 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 -4540> 2014-08-13 17:38:05.353746 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4539> 2014-08-13 17:38:05.353748 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.20 192.168.1.103:6815/38269 9 ==== pg_notify(24.3d(2) epoch 194487) v5 ==== 870+0+0 (3015517029 0 0) 0x6011180 con 0x4c71b80 -4538> 2014-08-13 17:38:05.353756 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4537> 2014-08-13 17:38:05.353749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -4536> 2014-08-13 17:38:05.353757 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4535> 2014-08-13 17:38:05.353760 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6011180 pg_notify(24.3d(2) epoch 194487) v5 -4534> 2014-08-13 17:38:05.353763 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.349339, event: header_read, request: pg_notify(24.3d(2) epoch 194487) v5 -4533> 2014-08-13 17:38:05.353761 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -4532> 2014-08-13 17:38:05.353767 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.349340, event: throttled, request: pg_notify(24.3d(2) epoch 194487) v5 -4531> 2014-08-13 17:38:05.353771 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.349383, event: all_read, request: pg_notify(24.3d(2) epoch 194487) v5 -4530> 2014-08-13 17:38:05.353774 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.353756, event: dispatched, request: pg_notify(24.3d(2) epoch 194487) v5 -4529> 2014-08-13 17:38:05.353770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -4528> 2014-08-13 17:38:05.353778 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.353778, event: waiting_for_osdmap, request: pg_notify(24.3d(2) epoch 194487) v5 -4527> 2014-08-13 17:38:05.353782 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.20 -4526> 2014-08-13 17:38:05.353783 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6011180 -4525> 2014-08-13 17:38:05.353781 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -4524> 2014-08-13 17:38:05.353786 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.353786, event: started, request: pg_notify(24.3d(2) epoch 194487) v5 -4523> 2014-08-13 17:38:05.353789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -4522> 2014-08-13 17:38:05.353798 7fe9a63f4700 5 -- op tracker -- , seq: 590, time: 2014-08-13 17:38:05.353798, event: done, request: pg_notify(24.3d(2) epoch 194487) v5 -4521> 2014-08-13 17:38:05.353804 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4520> 2014-08-13 17:38:05.353799 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -4519> 2014-08-13 17:38:05.353806 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4518> 2014-08-13 17:38:05.353618 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.367 (past last_backfill 0//0//-1) -4517> 2014-08-13 17:38:05.353809 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.55 192.168.1.112:6834/6966 14 ==== pg_notify(22.113(29) epoch 194487) v5 ==== 2345+0+0 (4234572472 0 0) 0x6010700 con 0x4d3dee0 -4516> 2014-08-13 17:38:05.353808 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:176854 -4515> 2014-08-13 17:38:05.353817 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4514> 2014-08-13 17:38:05.353819 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4513> 2014-08-13 17:38:05.353820 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010700 pg_notify(22.113(29) epoch 194487) v5 -4512> 2014-08-13 17:38:05.353817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters -4511> 2014-08-13 17:38:05.353822 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.348383, event: header_read, request: pg_notify(22.113(29) epoch 194487) v5 -4510> 2014-08-13 17:38:05.353827 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.348384, event: throttled, request: pg_notify(22.113(29) epoch 194487) v5 -4509> 2014-08-13 17:38:05.353830 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.348486, event: all_read, request: pg_notify(22.113(29) epoch 194487) v5 -4508> 2014-08-13 17:38:05.353826 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4507> 2014-08-13 17:38:05.353833 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.353817, event: dispatched, request: pg_notify(22.113(29) epoch 194487) v5 -4506> 2014-08-13 17:38:05.353837 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.353837, event: waiting_for_osdmap, request: pg_notify(22.113(29) epoch 194487) v5 -4505> 2014-08-13 17:38:05.353840 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 -4504> 2014-08-13 17:38:05.353835 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4503> 2014-08-13 17:38:05.353842 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010700 -4502> 2014-08-13 17:38:05.353845 7fe99f5e9700 10 log is not dirty -4501> 2014-08-13 17:38:05.353845 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.353845, event: started, request: pg_notify(22.113(29) epoch 194487) v5 -4500> 2014-08-13 17:38:05.353858 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 -4499> 2014-08-13 17:38:05.353871 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4498> 2014-08-13 17:38:05.353874 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4497> 2014-08-13 17:38:05.353884 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4496> 2014-08-13 17:38:05.353894 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4495> 2014-08-13 17:38:05.353903 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4494> 2014-08-13 17:38:05.353912 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4493> 2014-08-13 17:38:05.353813 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.367 (past last_backfill 0//0//-1) -4492> 2014-08-13 17:38:05.354077 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] -4491> 2014-08-13 17:38:05.354085 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4490> 2014-08-13 17:38:05.353990 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.367 (past last_backfill 0//0//-1) -4489> 2014-08-13 17:38:05.354168 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2935 -4488> 2014-08-13 17:38:05.354183 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4487> 2014-08-13 17:38:05.354192 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4486> 2014-08-13 17:38:05.354201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4485> 2014-08-13 17:38:05.354211 7fe99f5e9700 10 log is not dirty -4484> 2014-08-13 17:38:05.354225 7fe9a63f4700 5 -- op tracker -- , seq: 591, time: 2014-08-13 17:38:05.354225, event: done, request: pg_notify(22.113(29) epoch 194487) v5 -4483> 2014-08-13 17:38:05.354233 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4482> 2014-08-13 17:38:05.354235 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4481> 2014-08-13 17:38:05.354233 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 -4480> 2014-08-13 17:38:05.354241 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 20 ==== pg_notify(22.99(27) epoch 194487) v5 ==== 2227+0+0 (4277613584 0 0) 0x4265b00 con 0x4d3c780 -4479> 2014-08-13 17:38:05.354249 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4478> 2014-08-13 17:38:05.354250 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4477> 2014-08-13 17:38:05.354251 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4476> 2014-08-13 17:38:05.354252 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4265b00 pg_notify(22.99(27) epoch 194487) v5 -4475> 2014-08-13 17:38:05.354254 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.348281, event: header_read, request: pg_notify(22.99(27) epoch 194487) v5 -4474> 2014-08-13 17:38:05.354258 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.348282, event: throttled, request: pg_notify(22.99(27) epoch 194487) v5 -4473> 2014-08-13 17:38:05.354255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap -4472> 2014-08-13 17:38:05.354262 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.348359, event: all_read, request: pg_notify(22.99(27) epoch 194487) v5 -4471> 2014-08-13 17:38:05.354265 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.354249, event: dispatched, request: pg_notify(22.99(27) epoch 194487) v5 -4470> 2014-08-13 17:38:05.354269 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.354269, event: waiting_for_osdmap, request: pg_notify(22.99(27) epoch 194487) v5 -4469> 2014-08-13 17:38:05.354273 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.337 -4468> 2014-08-13 17:38:05.354265 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down -4467> 2014-08-13 17:38:05.354274 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4265b00 -4466> 2014-08-13 17:38:05.354277 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.354277, event: started, request: pg_notify(22.99(27) epoch 194487) v5 -4465> 2014-08-13 17:38:05.354276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map -4464> 2014-08-13 17:38:05.354289 7fe9a63f4700 5 -- op tracker -- , seq: 592, time: 2014-08-13 17:38:05.354289, event: done, request: pg_notify(22.99(27) epoch 194487) v5 -4463> 2014-08-13 17:38:05.354295 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4462> 2014-08-13 17:38:05.354296 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4461> 2014-08-13 17:38:05.354302 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 8 ==== pg_notify(24.1bb(2) epoch 194487) v5 ==== 870+0+0 (680382204 0 0) 0x51ce740 con 0x4e3b020 -4460> 2014-08-13 17:38:05.354309 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4459> 2014-08-13 17:38:05.354310 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4458> 2014-08-13 17:38:05.354311 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_notify(24.1bb(2) epoch 194487) v5 -4457> 2014-08-13 17:38:05.354314 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.349703, event: header_read, request: pg_notify(24.1bb(2) epoch 194487) v5 -4456> 2014-08-13 17:38:05.354317 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.349703, event: throttled, request: pg_notify(24.1bb(2) epoch 194487) v5 -4455> 2014-08-13 17:38:05.354321 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.349868, event: all_read, request: pg_notify(24.1bb(2) epoch 194487) v5 -4454> 2014-08-13 17:38:05.354324 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.354309, event: dispatched, request: pg_notify(24.1bb(2) epoch 194487) v5 -4453> 2014-08-13 17:38:05.354328 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.354328, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194487) v5 -4452> 2014-08-13 17:38:05.354331 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.332 -4451> 2014-08-13 17:38:05.354332 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 -4450> 2014-08-13 17:38:05.354335 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.354335, event: started, request: pg_notify(24.1bb(2) epoch 194487) v5 -4449> 2014-08-13 17:38:05.354341 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters -4448> 2014-08-13 17:38:05.354348 7fe9a63f4700 5 -- op tracker -- , seq: 593, time: 2014-08-13 17:38:05.354348, event: done, request: pg_notify(24.1bb(2) epoch 194487) v5 -4447> 2014-08-13 17:38:05.354164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.367 (past last_backfill 0//0//-1) -4446> 2014-08-13 17:38:05.354355 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4445> 2014-08-13 17:38:05.354356 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4444> 2014-08-13 17:38:05.354352 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 -4443> 2014-08-13 17:38:05.354360 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.24 192.168.1.107:6800/52669 7 ==== pg_notify(22.113(29) epoch 194487) v5 ==== 2345+0+0 (4234572472 0 0) 0x6010c40 con 0x4e3ef60 -4442> 2014-08-13 17:38:05.354368 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4441> 2014-08-13 17:38:05.354369 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4440> 2014-08-13 17:38:05.354362 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4439> 2014-08-13 17:38:05.354370 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010c40 pg_notify(22.113(29) epoch 194487) v5 -4438> 2014-08-13 17:38:05.354373 7fe99f5e9700 10 log is not dirty -4437> 2014-08-13 17:38:05.354373 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.348789, event: header_read, request: pg_notify(22.113(29) epoch 194487) v5 -4436> 2014-08-13 17:38:05.354377 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.348790, event: throttled, request: pg_notify(22.113(29) epoch 194487) v5 -4435> 2014-08-13 17:38:05.354380 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.348893, event: all_read, request: pg_notify(22.113(29) epoch 194487) v5 -4434> 2014-08-13 17:38:05.354383 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.354368, event: dispatched, request: pg_notify(22.113(29) epoch 194487) v5 -4433> 2014-08-13 17:38:05.354387 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.354386, event: waiting_for_osdmap, request: pg_notify(22.113(29) epoch 194487) v5 -4432> 2014-08-13 17:38:05.354390 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.24 -4431> 2014-08-13 17:38:05.354392 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010c40 -4430> 2014-08-13 17:38:05.354394 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.354394, event: started, request: pg_notify(22.113(29) epoch 194487) v5 -4429> 2014-08-13 17:38:05.354388 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 -4428> 2014-08-13 17:38:05.354405 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4427> 2014-08-13 17:38:05.354407 7fe9a63f4700 5 -- op tracker -- , seq: 594, time: 2014-08-13 17:38:05.354407, event: done, request: pg_notify(22.113(29) epoch 194487) v5 -4426> 2014-08-13 17:38:05.354416 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4425> 2014-08-13 17:38:05.354408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Started advmap -4424> 2014-08-13 17:38:05.354417 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4423> 2014-08-13 17:38:05.354418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down -4422> 2014-08-13 17:38:05.354428 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map -4421> 2014-08-13 17:38:05.354435 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 16 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d945c0 con 0x4e4d6a0 -4420> 2014-08-13 17:38:05.354443 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4419> 2014-08-13 17:38:05.354444 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4418> 2014-08-13 17:38:05.354445 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d945c0 osd_map(194487..194487 src has 185139..194487) v3 -4417> 2014-08-13 17:38:05.354449 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4416> 2014-08-13 17:38:05.354451 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4415> 2014-08-13 17:38:05.354453 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4414> 2014-08-13 17:38:05.354454 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4413> 2014-08-13 17:38:05.354473 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.397 192.168.1.105:6808/41085 11 ==== pg_notify(22.3d9(16) epoch 194487) v5 ==== 1622+0+0 (30220174 0 0) 0x5f07540 con 0x4e48840 -4412> 2014-08-13 17:38:05.354471 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters -4411> 2014-08-13 17:38:05.354480 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4410> 2014-08-13 17:38:05.354482 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4409> 2014-08-13 17:38:05.354483 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07540 pg_notify(22.3d9(16) epoch 194487) v5 -4408> 2014-08-13 17:38:05.354486 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.338069, event: header_read, request: pg_notify(22.3d9(16) epoch 194487) v5 -4407> 2014-08-13 17:38:05.354481 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4406> 2014-08-13 17:38:05.354490 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.338070, event: throttled, request: pg_notify(22.3d9(16) epoch 194487) v5 -4405> 2014-08-13 17:38:05.354493 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.338121, event: all_read, request: pg_notify(22.3d9(16) epoch 194487) v5 -4404> 2014-08-13 17:38:05.354491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4403> 2014-08-13 17:38:05.354497 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.354480, event: dispatched, request: pg_notify(22.3d9(16) epoch 194487) v5 -4402> 2014-08-13 17:38:05.354501 7fe99f5e9700 10 log is not dirty -4401> 2014-08-13 17:38:05.354501 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.354501, event: waiting_for_osdmap, request: pg_notify(22.3d9(16) epoch 194487) v5 -4400> 2014-08-13 17:38:05.354506 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.397 -4399> 2014-08-13 17:38:05.354508 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07540 -4398> 2014-08-13 17:38:05.354510 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.354510, event: started, request: pg_notify(22.3d9(16) epoch 194487) v5 -4397> 2014-08-13 17:38:05.354513 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 -4396> 2014-08-13 17:38:05.354522 7fe9a63f4700 5 -- op tracker -- , seq: 595, time: 2014-08-13 17:38:05.354522, event: done, request: pg_notify(22.3d9(16) epoch 194487) v5 -4395> 2014-08-13 17:38:05.354525 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4394> 2014-08-13 17:38:05.354355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.367 (past last_backfill 0//0//-1) -4393> 2014-08-13 17:38:05.354529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4392> 2014-08-13 17:38:05.354531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4391> 2014-08-13 17:38:05.354529 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -4390> 2014-08-13 17:38:05.354537 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.327 192.168.1.106:6804/5112 11 ==== pg_notify(24.b6(2) epoch 194487) v5 ==== 870+0+0 (3306849507 0 0) 0x5140000 con 0x4e82520 -4389> 2014-08-13 17:38:05.354545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4388> 2014-08-13 17:38:05.354546 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4387> 2014-08-13 17:38:05.354539 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -4386> 2014-08-13 17:38:05.354547 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5140000 pg_notify(24.b6(2) epoch 194487) v5 -4385> 2014-08-13 17:38:05.354550 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.346814, event: header_read, request: pg_notify(24.b6(2) epoch 194487) v5 -4384> 2014-08-13 17:38:05.354554 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.346816, event: throttled, request: pg_notify(24.b6(2) epoch 194487) v5 -4383> 2014-08-13 17:38:05.354558 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.346878, event: all_read, request: pg_notify(24.b6(2) epoch 194487) v5 -4382> 2014-08-13 17:38:05.354548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -4381> 2014-08-13 17:38:05.354561 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.354545, event: dispatched, request: pg_notify(24.b6(2) epoch 194487) v5 -4380> 2014-08-13 17:38:05.354565 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.354565, event: waiting_for_osdmap, request: pg_notify(24.b6(2) epoch 194487) v5 -4379> 2014-08-13 17:38:05.354568 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.327 -4378> 2014-08-13 17:38:05.354562 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -4377> 2014-08-13 17:38:05.354570 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5140000 -4376> 2014-08-13 17:38:05.354572 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.354572, event: started, request: pg_notify(24.b6(2) epoch 194487) v5 -4375> 2014-08-13 17:38:05.354571 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -4374> 2014-08-13 17:38:05.354580 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -4373> 2014-08-13 17:38:05.354584 7fe9a63f4700 5 -- op tracker -- , seq: 596, time: 2014-08-13 17:38:05.354584, event: done, request: pg_notify(24.b6(2) epoch 194487) v5 -4372> 2014-08-13 17:38:05.354591 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:177958 -4371> 2014-08-13 17:38:05.354593 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4370> 2014-08-13 17:38:05.354599 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters -4369> 2014-08-13 17:38:05.354606 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4368> 2014-08-13 17:38:05.354607 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4367> 2014-08-13 17:38:05.354610 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.65 192.168.1.111:6842/17345 10 ==== pg_notify(24.136(31) epoch 194487) v5 ==== 2485+0+0 (497636557 0 0) 0x5824d00 con 0x4e86040 -4366> 2014-08-13 17:38:05.354617 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4365> 2014-08-13 17:38:05.354619 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4364> 2014-08-13 17:38:05.354620 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824d00 pg_notify(24.136(31) epoch 194487) v5 -4363> 2014-08-13 17:38:05.354615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194485 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4362> 2014-08-13 17:38:05.354622 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.344022, event: header_read, request: pg_notify(24.136(31) epoch 194487) v5 -4361> 2014-08-13 17:38:05.354626 7fe99f5e9700 10 log is not dirty -4360> 2014-08-13 17:38:05.354627 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.344022, event: throttled, request: pg_notify(24.136(31) epoch 194487) v5 -4359> 2014-08-13 17:38:05.354630 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.344088, event: all_read, request: pg_notify(24.136(31) epoch 194487) v5 -4358> 2014-08-13 17:38:05.354633 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.354617, event: dispatched, request: pg_notify(24.136(31) epoch 194487) v5 -4357> 2014-08-13 17:38:05.354637 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.354637, event: waiting_for_osdmap, request: pg_notify(24.136(31) epoch 194487) v5 -4356> 2014-08-13 17:38:05.354640 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.65 -4355> 2014-08-13 17:38:05.354641 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824d00 -4354> 2014-08-13 17:38:05.354644 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.354644, event: started, request: pg_notify(24.136(31) epoch 194487) v5 -4353> 2014-08-13 17:38:05.354642 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_advance_map [26,30,340]/[4] -- 26/4 -4352> 2014-08-13 17:38:05.354656 7fe9a63f4700 5 -- op tracker -- , seq: 597, time: 2014-08-13 17:38:05.354656, event: done, request: pg_notify(24.136(31) epoch 194487) v5 -4351> 2014-08-13 17:38:05.354659 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4350> 2014-08-13 17:38:05.354662 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4349> 2014-08-13 17:38:05.354663 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4348> 2014-08-13 17:38:05.354671 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.339 192.168.1.110:6818/45268 9 ==== pg_notify(22.3af(31) epoch 194487) v5 ==== 2455+0+0 (2107490442 0 0) 0x5822300 con 0x4ec8160 -4347> 2014-08-13 17:38:05.354678 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4346> 2014-08-13 17:38:05.354680 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4345> 2014-08-13 17:38:05.354681 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822300 pg_notify(22.3af(31) epoch 194487) v5 -4344> 2014-08-13 17:38:05.354683 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.345113, event: header_read, request: pg_notify(22.3af(31) epoch 194487) v5 -4343> 2014-08-13 17:38:05.354664 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Peering advmap -4342> 2014-08-13 17:38:05.354687 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.345114, event: throttled, request: pg_notify(22.3af(31) epoch 194487) v5 -4341> 2014-08-13 17:38:05.354691 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.345196, event: all_read, request: pg_notify(22.3af(31) epoch 194487) v5 -4340> 2014-08-13 17:38:05.354694 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.354678, event: dispatched, request: pg_notify(22.3af(31) epoch 194487) v5 -4339> 2014-08-13 17:38:05.354689 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] adjust_need_up_thru now 194486, need_up_thru now false -4338> 2014-08-13 17:38:05.354698 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.354698, event: waiting_for_osdmap, request: pg_notify(22.3af(31) epoch 194487) v5 -4337> 2014-08-13 17:38:05.354701 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.339 -4336> 2014-08-13 17:38:05.354703 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822300 -4335> 2014-08-13 17:38:05.354531 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.367 (past last_backfill 0//0//-1) -4334> 2014-08-13 17:38:05.354705 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.354705, event: started, request: pg_notify(22.3af(31) epoch 194487) v5 -4333> 2014-08-13 17:38:05.354700 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Started advmap -4332> 2014-08-13 17:38:05.354711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] check_recovery_sources no source osds () went down -4331> 2014-08-13 17:38:05.354718 7fe9a63f4700 5 -- op tracker -- , seq: 598, time: 2014-08-13 17:38:05.354717, event: done, request: pg_notify(22.3af(31) epoch 194487) v5 -4330> 2014-08-13 17:38:05.354724 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4329> 2014-08-13 17:38:05.354725 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4328> 2014-08-13 17:38:05.354722 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map -4327> 2014-08-13 17:38:05.354733 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle ActMap primary -4326> 2014-08-13 17:38:05.354738 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.381 192.168.1.102:6822/15390 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54d8b40 con 0x4ecb9c0 -4325> 2014-08-13 17:38:05.354746 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4324> 2014-08-13 17:38:05.354748 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4323> 2014-08-13 17:38:05.354749 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54d8b40 osd_map(194487..194487 src has 185139..194487) v3 -4322> 2014-08-13 17:38:05.354743 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194487:23843 -4321> 2014-08-13 17:38:05.354753 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4320> 2014-08-13 17:38:05.354756 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4319> 2014-08-13 17:38:05.354757 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4318> 2014-08-13 17:38:05.354758 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4317> 2014-08-13 17:38:05.354753 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] take_waiters -4316> 2014-08-13 17:38:05.354761 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.48 192.168.1.112:6838/8300 7 ==== pg_notify(22.619(31) epoch 194487) v5 ==== 2455+0+0 (2275569936 0 0) 0x6011c00 con 0x4ecf640 -4315> 2014-08-13 17:38:05.354768 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4314> 2014-08-13 17:38:05.354769 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4313> 2014-08-13 17:38:05.354770 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6011c00 pg_notify(22.619(31) epoch 194487) v5 -4312> 2014-08-13 17:38:05.354765 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 0.947467 6 0.000876 -4311> 2014-08-13 17:38:05.354773 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.353351, event: header_read, request: pg_notify(22.619(31) epoch 194487) v5 -4310> 2014-08-13 17:38:05.354777 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.353353, event: throttled, request: pg_notify(22.619(31) epoch 194487) v5 -4309> 2014-08-13 17:38:05.354781 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.353446, event: all_read, request: pg_notify(22.619(31) epoch 194487) v5 -4308> 2014-08-13 17:38:05.354784 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.354768, event: dispatched, request: pg_notify(22.619(31) epoch 194487) v5 -4307> 2014-08-13 17:38:05.354779 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Leaving Peering -4306> 2014-08-13 17:38:05.354788 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.354787, event: waiting_for_osdmap, request: pg_notify(22.619(31) epoch 194487) v5 -4305> 2014-08-13 17:38:05.354791 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.48 -4304> 2014-08-13 17:38:05.354793 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6011c00 -4303> 2014-08-13 17:38:05.354795 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.354795, event: started, request: pg_notify(22.619(31) epoch 194487) v5 -4302> 2014-08-13 17:38:05.354789 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 1.028897 0 0.000000 -4301> 2014-08-13 17:38:05.354806 7fe9a63f4700 5 -- op tracker -- , seq: 599, time: 2014-08-13 17:38:05.354806, event: done, request: pg_notify(22.619(31) epoch 194487) v5 -4300> 2014-08-13 17:38:05.354812 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4299> 2014-08-13 17:38:05.354814 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4298> 2014-08-13 17:38:05.354807 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active -4297> 2014-08-13 17:38:05.354817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194478 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: In Active, about to call activate -4296> 2014-08-13 17:38:05.354824 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.302 192.168.1.113:6810/15731 7 ==== pg_notify(22.22c(1) epoch 194487) v5 ==== 777+0+0 (3221878805 0 0) 0x60116c0 con 0x4ecf380 -4295> 2014-08-13 17:38:05.354832 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4294> 2014-08-13 17:38:05.354833 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4293> 2014-08-13 17:38:05.354834 7fe9a63f4700 20 osd.4 194487 _dispatch 0x60116c0 pg_notify(22.22c(1) epoch 194487) v5 -4292> 2014-08-13 17:38:05.354830 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - purged_snaps [] cached_removed_snaps [] -4291> 2014-08-13 17:38:05.354837 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.349739, event: header_read, request: pg_notify(22.22c(1) epoch 194487) v5 -4290> 2014-08-13 17:38:05.354843 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.349741, event: throttled, request: pg_notify(22.22c(1) epoch 194487) v5 -4289> 2014-08-13 17:38:05.354847 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.349781, event: all_read, request: pg_notify(22.22c(1) epoch 194487) v5 -4288> 2014-08-13 17:38:05.354842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - snap_trimq [] -4287> 2014-08-13 17:38:05.354850 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.354831, event: dispatched, request: pg_notify(22.22c(1) epoch 194487) v5 -4286> 2014-08-13 17:38:05.354854 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.354854, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194487) v5 -4285> 2014-08-13 17:38:05.354858 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.302 -4284> 2014-08-13 17:38:05.354859 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x60116c0 -4283> 2014-08-13 17:38:05.354852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - no missing, moving last_complete 194215'14514 -> 194215'14514 -4282> 2014-08-13 17:38:05.354862 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.354862, event: started, request: pg_notify(22.22c(1) epoch 194487) v5 -4281> 2014-08-13 17:38:05.354873 7fe9a63f4700 5 -- op tracker -- , seq: 600, time: 2014-08-13 17:38:05.354873, event: done, request: pg_notify(22.22c(1) epoch 194487) v5 -4280> 2014-08-13 17:38:05.354879 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4279> 2014-08-13 17:38:05.354705 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.367 (past last_backfill 0//0//-1) -4278> 2014-08-13 17:38:05.354880 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4277> 2014-08-13 17:38:05.354883 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.371 192.168.1.102:6832/16753 11 ==== pg_notify(22.4df(27) epoch 194487) v5 ==== 2227+0+0 (4133003232 0 0) 0x51c9500 con 0x41e5540 -4276> 2014-08-13 17:38:05.354891 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4275> 2014-08-13 17:38:05.354892 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4274> 2014-08-13 17:38:05.354893 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c9500 pg_notify(22.4df(27) epoch 194487) v5 -4273> 2014-08-13 17:38:05.354895 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.352050, event: header_read, request: pg_notify(22.4df(27) epoch 194487) v5 -4272> 2014-08-13 17:38:05.354899 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.352052, event: throttled, request: pg_notify(22.4df(27) epoch 194487) v5 -4271> 2014-08-13 17:38:05.354902 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.352129, event: all_read, request: pg_notify(22.4df(27) epoch 194487) v5 -4270> 2014-08-13 17:38:05.354905 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.354891, event: dispatched, request: pg_notify(22.4df(27) epoch 194487) v5 -4269> 2014-08-13 17:38:05.354909 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.354909, event: waiting_for_osdmap, request: pg_notify(22.4df(27) epoch 194487) v5 -4268> 2014-08-13 17:38:05.354914 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.371 -4267> 2014-08-13 17:38:05.354915 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c9500 -4266> 2014-08-13 17:38:05.354918 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.354917, event: started, request: pg_notify(22.4df(27) epoch 194487) v5 -4265> 2014-08-13 17:38:05.354929 7fe9a63f4700 5 -- op tracker -- , seq: 601, time: 2014-08-13 17:38:05.354929, event: done, request: pg_notify(22.4df(27) epoch 194487) v5 -4264> 2014-08-13 17:38:05.354920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -4263> 2014-08-13 17:38:05.354935 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4262> 2014-08-13 17:38:05.354937 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4261> 2014-08-13 17:38:05.354934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 is up to date, queueing in pending_activators -4260> 2014-08-13 17:38:05.354942 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.351 192.168.1.102:6830/16444 12 ==== pg_notify(24.1f7(38) epoch 194487) v5 ==== 2898+0+0 (3764987790 0 0) 0x5f07a80 con 0x5250b00 -4259> 2014-08-13 17:38:05.354950 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4258> 2014-08-13 17:38:05.354951 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4257> 2014-08-13 17:38:05.354952 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07a80 pg_notify(24.1f7(38) epoch 194487) v5 -4256> 2014-08-13 17:38:05.354955 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.338619, event: header_read, request: pg_notify(24.1f7(38) epoch 194487) v5 -4255> 2014-08-13 17:38:05.354958 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.338620, event: throttled, request: pg_notify(24.1f7(38) epoch 194487) v5 -4254> 2014-08-13 17:38:05.354962 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.338700, event: all_read, request: pg_notify(24.1f7(38) epoch 194487) v5 -4253> 2014-08-13 17:38:05.354965 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.354950, event: dispatched, request: pg_notify(24.1f7(38) epoch 194487) v5 -4252> 2014-08-13 17:38:05.354968 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.354968, event: waiting_for_osdmap, request: pg_notify(24.1f7(38) epoch 194487) v5 -4251> 2014-08-13 17:38:05.354972 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.351 -4250> 2014-08-13 17:38:05.354973 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07a80 -4249> 2014-08-13 17:38:05.354975 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.354975, event: started, request: pg_notify(24.1f7(38) epoch 194487) v5 -4248> 2014-08-13 17:38:05.354987 7fe9a63f4700 5 -- op tracker -- , seq: 602, time: 2014-08-13 17:38:05.354986, event: done, request: pg_notify(24.1f7(38) epoch 194487) v5 -4247> 2014-08-13 17:38:05.354992 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4246> 2014-08-13 17:38:05.354994 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4245> 2014-08-13 17:38:05.355009 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.369 192.168.1.108:6824/4037 13 ==== pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 ==== 4442+0+0 (1441779146 0 0) 0x5824980 con 0x5250840 -4244> 2014-08-13 17:38:05.355007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate -4243> 2014-08-13 17:38:05.355020 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4242> 2014-08-13 17:38:05.355021 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4241> 2014-08-13 17:38:05.355022 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824980 pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4240> 2014-08-13 17:38:05.355026 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.347155, event: header_read, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4239> 2014-08-13 17:38:05.355021 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -4238> 2014-08-13 17:38:05.355030 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.347156, event: throttled, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4237> 2014-08-13 17:38:05.355034 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.347251, event: all_read, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4236> 2014-08-13 17:38:05.355038 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.355020, event: dispatched, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4235> 2014-08-13 17:38:05.355034 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 is up to date, queueing in pending_activators -4234> 2014-08-13 17:38:05.355043 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.355042, event: waiting_for_osdmap, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4233> 2014-08-13 17:38:05.355046 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.369 -4232> 2014-08-13 17:38:05.355048 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824980 -4231> 2014-08-13 17:38:05.355050 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.355050, event: started, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4230> 2014-08-13 17:38:05.354881 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.367 (past last_backfill 0//0//-1) -4229> 2014-08-13 17:38:05.355069 7fe9a63f4700 5 -- op tracker -- , seq: 603, time: 2014-08-13 17:38:05.355069, event: done, request: pg_notify(22.3af(31),22.f3(23) epoch 194487) v5 -4228> 2014-08-13 17:38:05.355076 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4227> 2014-08-13 17:38:05.355078 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4226> 2014-08-13 17:38:05.355083 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate -4225> 2014-08-13 17:38:05.355092 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.349 192.168.1.104:6800/61726 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54de540 con 0x5380c60 -4224> 2014-08-13 17:38:05.355100 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4223> 2014-08-13 17:38:05.355101 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4222> 2014-08-13 17:38:05.355102 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54de540 osd_map(194487..194487 src has 185139..194487) v3 -4221> 2014-08-13 17:38:05.355106 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4220> 2014-08-13 17:38:05.355096 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) -4219> 2014-08-13 17:38:05.355109 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4218> 2014-08-13 17:38:05.355110 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4217> 2014-08-13 17:38:05.355111 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4216> 2014-08-13 17:38:05.355110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 is up to date, queueing in pending_activators -4215> 2014-08-13 17:38:05.355114 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.20 192.168.1.103:6815/38269 10 ==== pg_notify(22.99(27) epoch 194487) v5 ==== 2227+0+0 (4277613584 0 0) 0x6010fc0 con 0x4c71b80 -4214> 2014-08-13 17:38:05.355121 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4213> 2014-08-13 17:38:05.355123 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4212> 2014-08-13 17:38:05.355123 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010fc0 pg_notify(22.99(27) epoch 194487) v5 -4211> 2014-08-13 17:38:05.355126 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.351341, event: header_read, request: pg_notify(22.99(27) epoch 194487) v5 -4210> 2014-08-13 17:38:05.355130 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.351343, event: throttled, request: pg_notify(22.99(27) epoch 194487) v5 -4209> 2014-08-13 17:38:05.355133 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.351428, event: all_read, request: pg_notify(22.99(27) epoch 194487) v5 -4208> 2014-08-13 17:38:05.355137 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.355121, event: dispatched, request: pg_notify(22.99(27) epoch 194487) v5 -4207> 2014-08-13 17:38:05.355140 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.355140, event: waiting_for_osdmap, request: pg_notify(22.99(27) epoch 194487) v5 -4206> 2014-08-13 17:38:05.355143 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.20 -4205> 2014-08-13 17:38:05.355145 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010fc0 -4204> 2014-08-13 17:38:05.355147 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.355147, event: started, request: pg_notify(22.99(27) epoch 194487) v5 -4203> 2014-08-13 17:38:05.355151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486) uptodate -4202> 2014-08-13 17:38:05.355163 7fe9a63f4700 5 -- op tracker -- , seq: 604, time: 2014-08-13 17:38:05.355163, event: done, request: pg_notify(22.99(27) epoch 194487) v5 -4201> 2014-08-13 17:38:05.355169 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4200> 2014-08-13 17:38:05.355171 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4199> 2014-08-13 17:38:05.355166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] needs_recovery is recovered -4198> 2014-08-13 17:38:05.355179 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.55 192.168.1.112:6834/6966 15 ==== pg_notify(24.1f7(2) epoch 194487) v5 ==== 870+0+0 (1056347338 0 0) 0x6010540 con 0x4d3dee0 -4197> 2014-08-13 17:38:05.355187 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4196> 2014-08-13 17:38:05.355188 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4195> 2014-08-13 17:38:05.355176 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Activate Finished -4194> 2014-08-13 17:38:05.355189 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010540 pg_notify(24.1f7(2) epoch 194487) v5 -4193> 2014-08-13 17:38:05.355197 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.349198, event: header_read, request: pg_notify(24.1f7(2) epoch 194487) v5 -4192> 2014-08-13 17:38:05.355193 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating -4191> 2014-08-13 17:38:05.355201 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.349199, event: throttled, request: pg_notify(24.1f7(2) epoch 194487) v5 -4190> 2014-08-13 17:38:05.355205 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.349243, event: all_read, request: pg_notify(24.1f7(2) epoch 194487) v5 -4189> 2014-08-13 17:38:05.355208 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.355186, event: dispatched, request: pg_notify(24.1f7(2) epoch 194487) v5 -4188> 2014-08-13 17:38:05.355204 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4187> 2014-08-13 17:38:05.355212 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.355212, event: waiting_for_osdmap, request: pg_notify(24.1f7(2) epoch 194487) v5 -4186> 2014-08-13 17:38:05.355216 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 -4185> 2014-08-13 17:38:05.355217 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010540 -4184> 2014-08-13 17:38:05.355220 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.355220, event: started, request: pg_notify(24.1f7(2) epoch 194487) v5 -4183> 2014-08-13 17:38:05.355214 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4182> 2014-08-13 17:38:05.355055 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.367 (past last_backfill 0//0//-1) -4181> 2014-08-13 17:38:05.355231 7fe9a63f4700 5 -- op tracker -- , seq: 605, time: 2014-08-13 17:38:05.355231, event: done, request: pg_notify(24.1f7(2) epoch 194487) v5 -4180> 2014-08-13 17:38:05.355244 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4179> 2014-08-13 17:38:05.355246 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4178> 2014-08-13 17:38:05.355249 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 21 ==== pg_notify(24.1bb(35) epoch 194487) v5 ==== 2721+0+0 (1604338514 0 0) 0x4264d00 con 0x4d3c780 -4177> 2014-08-13 17:38:05.355257 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4176> 2014-08-13 17:38:05.355258 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4175> 2014-08-13 17:38:05.355259 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264d00 pg_notify(24.1bb(35) epoch 194487) v5 -4174> 2014-08-13 17:38:05.355261 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.349710, event: header_read, request: pg_notify(24.1bb(35) epoch 194487) v5 -4173> 2014-08-13 17:38:05.355276 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.349711, event: throttled, request: pg_notify(24.1bb(35) epoch 194487) v5 -4172> 2014-08-13 17:38:05.355280 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.349812, event: all_read, request: pg_notify(24.1bb(35) epoch 194487) v5 -4171> 2014-08-13 17:38:05.355282 7fe99f5e9700 10 log is not dirty -4170> 2014-08-13 17:38:05.355283 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.355256, event: dispatched, request: pg_notify(24.1bb(35) epoch 194487) v5 -4169> 2014-08-13 17:38:05.355287 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.355286, event: waiting_for_osdmap, request: pg_notify(24.1bb(35) epoch 194487) v5 -4168> 2014-08-13 17:38:05.355290 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.337 -4167> 2014-08-13 17:38:05.355291 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264d00 -4166> 2014-08-13 17:38:05.355294 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.355293, event: started, request: pg_notify(24.1bb(35) epoch 194487) v5 -4165> 2014-08-13 17:38:05.355305 7fe9a63f4700 5 -- op tracker -- , seq: 606, time: 2014-08-13 17:38:05.355304, event: done, request: pg_notify(24.1bb(35) epoch 194487) v5 -4164> 2014-08-13 17:38:05.355310 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4163> 2014-08-13 17:38:05.355312 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4162> 2014-08-13 17:38:05.355312 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 -4161> 2014-08-13 17:38:05.355328 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4160> 2014-08-13 17:38:05.355323 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 17 ==== pg_notify(22.707(1) epoch 194487) v5 ==== 777+0+0 (2918448130 0 0) 0x5822bc0 con 0x4e4d6a0 -4159> 2014-08-13 17:38:05.355331 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4158> 2014-08-13 17:38:05.355333 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4157> 2014-08-13 17:38:05.355334 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822bc0 pg_notify(22.707(1) epoch 194487) v5 -4156> 2014-08-13 17:38:05.355337 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.353343, event: header_read, request: pg_notify(22.707(1) epoch 194487) v5 -4155> 2014-08-13 17:38:05.355331 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4154> 2014-08-13 17:38:05.355341 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.353344, event: throttled, request: pg_notify(22.707(1) epoch 194487) v5 -4153> 2014-08-13 17:38:05.355344 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.353380, event: all_read, request: pg_notify(22.707(1) epoch 194487) v5 -4152> 2014-08-13 17:38:05.355347 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.355331, event: dispatched, request: pg_notify(22.707(1) epoch 194487) v5 -4151> 2014-08-13 17:38:05.355343 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4150> 2014-08-13 17:38:05.355351 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.355351, event: waiting_for_osdmap, request: pg_notify(22.707(1) epoch 194487) v5 -4149> 2014-08-13 17:38:05.355355 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 -4148> 2014-08-13 17:38:05.355356 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822bc0 -4147> 2014-08-13 17:38:05.355359 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.355359, event: started, request: pg_notify(22.707(1) epoch 194487) v5 -4146> 2014-08-13 17:38:05.355354 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4145> 2014-08-13 17:38:05.355364 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4144> 2014-08-13 17:38:05.355370 7fe9a63f4700 5 -- op tracker -- , seq: 607, time: 2014-08-13 17:38:05.355370, event: done, request: pg_notify(22.707(1) epoch 194487) v5 -4143> 2014-08-13 17:38:05.355376 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4142> 2014-08-13 17:38:05.355377 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4141> 2014-08-13 17:38:05.355374 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4140> 2014-08-13 17:38:05.355381 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.327 192.168.1.106:6804/5112 12 ==== pg_notify(22.48e(29) epoch 194487) v5 ==== 2341+0+0 (2024069630 0 0) 0x513e900 con 0x4e82520 -4139> 2014-08-13 17:38:05.355388 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4138> 2014-08-13 17:38:05.355389 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4137> 2014-08-13 17:38:05.355390 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e900 pg_notify(22.48e(29) epoch 194487) v5 -4136> 2014-08-13 17:38:05.355393 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.348785, event: header_read, request: pg_notify(22.48e(29) epoch 194487) v5 -4135> 2014-08-13 17:38:05.355396 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.348785, event: throttled, request: pg_notify(22.48e(29) epoch 194487) v5 -4134> 2014-08-13 17:38:05.355400 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.348888, event: all_read, request: pg_notify(22.48e(29) epoch 194487) v5 -4133> 2014-08-13 17:38:05.355403 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.355388, event: dispatched, request: pg_notify(22.48e(29) epoch 194487) v5 -4132> 2014-08-13 17:38:05.355407 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.355406, event: waiting_for_osdmap, request: pg_notify(22.48e(29) epoch 194487) v5 -4131> 2014-08-13 17:38:05.355410 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.327 -4130> 2014-08-13 17:38:05.355411 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e900 -4129> 2014-08-13 17:38:05.355413 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.355413, event: started, request: pg_notify(22.48e(29) epoch 194487) v5 -4128> 2014-08-13 17:38:05.355234 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.367 (past last_backfill 0//0//-1) -4127> 2014-08-13 17:38:05.355424 7fe9a63f4700 5 -- op tracker -- , seq: 608, time: 2014-08-13 17:38:05.355424, event: done, request: pg_notify(22.48e(29) epoch 194487) v5 -4126> 2014-08-13 17:38:05.355430 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4125> 2014-08-13 17:38:05.355432 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4124> 2014-08-13 17:38:05.355441 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.381 192.168.1.102:6822/15390 7 ==== pg_notify(22.481(46) epoch 194487) v5 ==== 3296+0+0 (2265937782 0 0) 0x4260380 con 0x4ecb9c0 -4123> 2014-08-13 17:38:05.355449 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4122> 2014-08-13 17:38:05.355450 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4121> 2014-08-13 17:38:05.355451 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4260380 pg_notify(22.481(46) epoch 194487) v5 -4120> 2014-08-13 17:38:05.355453 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.354338, event: header_read, request: pg_notify(22.481(46) epoch 194487) v5 -4119> 2014-08-13 17:38:05.355457 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.354339, event: throttled, request: pg_notify(22.481(46) epoch 194487) v5 -4118> 2014-08-13 17:38:05.355460 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.354441, event: all_read, request: pg_notify(22.481(46) epoch 194487) v5 -4117> 2014-08-13 17:38:05.355463 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.355448, event: dispatched, request: pg_notify(22.481(46) epoch 194487) v5 -4116> 2014-08-13 17:38:05.355467 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.355467, event: waiting_for_osdmap, request: pg_notify(22.481(46) epoch 194487) v5 -4115> 2014-08-13 17:38:05.355470 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.381 -4114> 2014-08-13 17:38:05.355472 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4260380 -4113> 2014-08-13 17:38:05.355474 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.355474, event: started, request: pg_notify(22.481(46) epoch 194487) v5 -4112> 2014-08-13 17:38:05.355485 7fe9a63f4700 5 -- op tracker -- , seq: 609, time: 2014-08-13 17:38:05.355484, event: done, request: pg_notify(22.481(46) epoch 194487) v5 -4111> 2014-08-13 17:38:05.355490 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4110> 2014-08-13 17:38:05.355492 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4109> 2014-08-13 17:38:05.355501 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.369 192.168.1.108:6824/4037 14 ==== pg_notify(22.775(1) epoch 194487) v5 ==== 777+0+0 (2128480054 0 0) 0x5f07e00 con 0x5250840 -4108> 2014-08-13 17:38:05.355509 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4107> 2014-08-13 17:38:05.355510 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4106> 2014-08-13 17:38:05.355511 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07e00 pg_notify(22.775(1) epoch 194487) v5 -4105> 2014-08-13 17:38:05.355513 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.347959, event: header_read, request: pg_notify(22.775(1) epoch 194487) v5 -4104> 2014-08-13 17:38:05.355517 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.347961, event: throttled, request: pg_notify(22.775(1) epoch 194487) v5 -4103> 2014-08-13 17:38:05.355520 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.348008, event: all_read, request: pg_notify(22.775(1) epoch 194487) v5 -4102> 2014-08-13 17:38:05.355523 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.355508, event: dispatched, request: pg_notify(22.775(1) epoch 194487) v5 -4101> 2014-08-13 17:38:05.355527 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.355527, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194487) v5 -4100> 2014-08-13 17:38:05.355530 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.369 -4099> 2014-08-13 17:38:05.355532 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07e00 -4098> 2014-08-13 17:38:05.355534 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.355534, event: started, request: pg_notify(22.775(1) epoch 194487) v5 -4097> 2014-08-13 17:38:05.355547 7fe9a63f4700 5 -- op tracker -- , seq: 610, time: 2014-08-13 17:38:05.355546, event: done, request: pg_notify(22.775(1) epoch 194487) v5 -4096> 2014-08-13 17:38:05.355553 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4095> 2014-08-13 17:38:05.355555 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4094> 2014-08-13 17:38:05.355548 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] -4093> 2014-08-13 17:38:05.355558 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.349 192.168.1.104:6800/61726 9 ==== pg_notify(22.362(31) epoch 194487) v5 ==== 2455+0+0 (3265461919 0 0) 0x513d5c0 con 0x5380c60 -4092> 2014-08-13 17:38:05.355566 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4091> 2014-08-13 17:38:05.355559 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4090> 2014-08-13 17:38:05.355567 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4089> 2014-08-13 17:38:05.355568 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513d5c0 pg_notify(22.362(31) epoch 194487) v5 -4088> 2014-08-13 17:38:05.355571 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.354390, event: header_read, request: pg_notify(22.362(31) epoch 194487) v5 -4087> 2014-08-13 17:38:05.355574 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.354392, event: throttled, request: pg_notify(22.362(31) epoch 194487) v5 -4086> 2014-08-13 17:38:05.355578 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.354469, event: all_read, request: pg_notify(22.362(31) epoch 194487) v5 -4085> 2014-08-13 17:38:05.355581 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.355566, event: dispatched, request: pg_notify(22.362(31) epoch 194487) v5 -4084> 2014-08-13 17:38:05.355584 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.355584, event: waiting_for_osdmap, request: pg_notify(22.362(31) epoch 194487) v5 -4083> 2014-08-13 17:38:05.355587 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.349 -4082> 2014-08-13 17:38:05.355589 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513d5c0 -4081> 2014-08-13 17:38:05.355591 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.355591, event: started, request: pg_notify(22.362(31) epoch 194487) v5 -4080> 2014-08-13 17:38:05.355420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.367 (past last_backfill 0//0//-1) -4079> 2014-08-13 17:38:05.355602 7fe9a63f4700 5 -- op tracker -- , seq: 611, time: 2014-08-13 17:38:05.355602, event: done, request: pg_notify(22.362(31) epoch 194487) v5 -4078> 2014-08-13 17:38:05.355608 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4077> 2014-08-13 17:38:05.355609 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4076> 2014-08-13 17:38:05.355614 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.344 192.168.1.106:6812/6757 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d90fc0 con 0x4a482c0 -4075> 2014-08-13 17:38:05.355621 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4074> 2014-08-13 17:38:05.355623 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4073> 2014-08-13 17:38:05.355624 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d90fc0 osd_map(194487..194487 src has 185139..194487) v3 -4072> 2014-08-13 17:38:05.355627 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -4071> 2014-08-13 17:38:05.355630 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -4070> 2014-08-13 17:38:05.355631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4069> 2014-08-13 17:38:05.355632 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4068> 2014-08-13 17:38:05.355635 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.55 192.168.1.112:6834/6966 16 ==== pg_notify(24.181(12) epoch 194487) v5 ==== 1432+0+0 (612873932 0 0) 0x6010380 con 0x4d3dee0 -4067> 2014-08-13 17:38:05.355641 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4066> 2014-08-13 17:38:05.355643 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4065> 2014-08-13 17:38:05.355644 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010380 pg_notify(24.181(12) epoch 194487) v5 -4064> 2014-08-13 17:38:05.355646 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.349322, event: header_read, request: pg_notify(24.181(12) epoch 194487) v5 -4063> 2014-08-13 17:38:05.355650 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.349323, event: throttled, request: pg_notify(24.181(12) epoch 194487) v5 -4062> 2014-08-13 17:38:05.355648 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2873 -4061> 2014-08-13 17:38:05.355653 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.349389, event: all_read, request: pg_notify(24.181(12) epoch 194487) v5 -4060> 2014-08-13 17:38:05.355658 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.355641, event: dispatched, request: pg_notify(24.181(12) epoch 194487) v5 -4059> 2014-08-13 17:38:05.355662 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.355662, event: waiting_for_osdmap, request: pg_notify(24.181(12) epoch 194487) v5 -4058> 2014-08-13 17:38:05.355658 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4057> 2014-08-13 17:38:05.355665 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.55 -4056> 2014-08-13 17:38:05.355667 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010380 -4055> 2014-08-13 17:38:05.355670 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.355669, event: started, request: pg_notify(24.181(12) epoch 194487) v5 -4054> 2014-08-13 17:38:05.355667 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4053> 2014-08-13 17:38:05.355682 7fe9a63f4700 5 -- op tracker -- , seq: 612, time: 2014-08-13 17:38:05.355682, event: done, request: pg_notify(24.181(12) epoch 194487) v5 -4052> 2014-08-13 17:38:05.355676 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4051> 2014-08-13 17:38:05.355688 7fe99f5e9700 10 log is not dirty -4050> 2014-08-13 17:38:05.355689 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4049> 2014-08-13 17:38:05.355690 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4048> 2014-08-13 17:38:05.355694 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 18 ==== pg_notify(22.3e4(1) epoch 194487) v5 ==== 777+0+0 (383177298 0 0) 0x5822a00 con 0x4e4d6a0 -4047> 2014-08-13 17:38:05.355702 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4046> 2014-08-13 17:38:05.355703 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4045> 2014-08-13 17:38:05.355679 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] _activate_committed 194487 peer_activated now 4 last_epoch_started 194478 same_interval_since 194486 -4044> 2014-08-13 17:38:05.355704 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822a00 pg_notify(22.3e4(1) epoch 194487) v5 -4043> 2014-08-13 17:38:05.355707 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.354227, event: header_read, request: pg_notify(22.3e4(1) epoch 194487) v5 -4042> 2014-08-13 17:38:05.355711 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.354228, event: throttled, request: pg_notify(22.3e4(1) epoch 194487) v5 -4041> 2014-08-13 17:38:05.355704 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 -4040> 2014-08-13 17:38:05.355706 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] flushed -4039> 2014-08-13 17:38:05.355715 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.354273, event: all_read, request: pg_notify(22.3e4(1) epoch 194487) v5 -4038> 2014-08-13 17:38:05.355717 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4037> 2014-08-13 17:38:05.355719 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.355701, event: dispatched, request: pg_notify(22.3e4(1) epoch 194487) v5 -4036> 2014-08-13 17:38:05.355723 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.355723, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194487) v5 -4035> 2014-08-13 17:38:05.355727 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 -4034> 2014-08-13 17:38:05.355728 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822a00 -4033> 2014-08-13 17:38:05.355721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -4032> 2014-08-13 17:38:05.355730 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.355730, event: started, request: pg_notify(22.3e4(1) epoch 194487) v5 -4031> 2014-08-13 17:38:05.355731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -4030> 2014-08-13 17:38:05.355742 7fe9a63f4700 5 -- op tracker -- , seq: 613, time: 2014-08-13 17:38:05.355742, event: done, request: pg_notify(22.3e4(1) epoch 194487) v5 -4029> 2014-08-13 17:38:05.355740 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -4028> 2014-08-13 17:38:05.355748 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4027> 2014-08-13 17:38:05.355749 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4026> 2014-08-13 17:38:05.355750 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -4025> 2014-08-13 17:38:05.355752 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 19 ==== pg_notify(24.3d(32) epoch 194487) v5 ==== 2548+0+0 (3638720033 0 0) 0x5107000 con 0x4e4d6a0 -4024> 2014-08-13 17:38:05.355759 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4023> 2014-08-13 17:38:05.355761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4022> 2014-08-13 17:38:05.355762 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_notify(24.3d(32) epoch 194487) v5 -4021> 2014-08-13 17:38:05.355758 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -4020> 2014-08-13 17:38:05.355764 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.354706, event: header_read, request: pg_notify(24.3d(32) epoch 194487) v5 -4019> 2014-08-13 17:38:05.355595 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.367 (past last_backfill 0//0//-1) -4018> 2014-08-13 17:38:05.355768 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.354707, event: throttled, request: pg_notify(24.3d(32) epoch 194487) v5 -4017> 2014-08-13 17:38:05.355773 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.354783, event: all_read, request: pg_notify(24.3d(32) epoch 194487) v5 -4016> 2014-08-13 17:38:05.355768 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -4015> 2014-08-13 17:38:05.355776 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.355759, event: dispatched, request: pg_notify(24.3d(32) epoch 194487) v5 -4014> 2014-08-13 17:38:05.355780 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.355780, event: waiting_for_osdmap, request: pg_notify(24.3d(32) epoch 194487) v5 -4013> 2014-08-13 17:38:05.355784 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 -4012> 2014-08-13 17:38:05.355778 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:2906 -4011> 2014-08-13 17:38:05.355785 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 -4010> 2014-08-13 17:38:05.355789 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.355789, event: started, request: pg_notify(24.3d(32) epoch 194487) v5 -4009> 2014-08-13 17:38:05.355787 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] take_waiters -4008> 2014-08-13 17:38:05.355795 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -4007> 2014-08-13 17:38:05.355800 7fe9a63f4700 5 -- op tracker -- , seq: 614, time: 2014-08-13 17:38:05.355800, event: done, request: pg_notify(24.3d(32) epoch 194487) v5 -4006> 2014-08-13 17:38:05.355806 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4005> 2014-08-13 17:38:05.355808 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -4004> 2014-08-13 17:38:05.355804 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194485 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -4003> 2014-08-13 17:38:05.355814 7fe99f5e9700 10 log is not dirty -4002> 2014-08-13 17:38:05.355813 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.331 192.168.1.104:6816/837 8 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d93840 con 0x4e83180 -4001> 2014-08-13 17:38:05.355821 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -4000> 2014-08-13 17:38:05.355822 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3999> 2014-08-13 17:38:05.355823 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d93840 osd_map(194487..194487 src has 185139..194487) v3 -3998> 2014-08-13 17:38:05.355827 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3997> 2014-08-13 17:38:05.355829 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3996> 2014-08-13 17:38:05.355830 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3995> 2014-08-13 17:38:05.355831 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3994> 2014-08-13 17:38:05.355826 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 -3993> 2014-08-13 17:38:05.355839 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3992> 2014-08-13 17:38:05.355834 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.344 192.168.1.106:6812/6757 8 ==== pg_notify(24.181(33) epoch 194487) v5 ==== 2599+0+0 (1393183035 0 0) 0x5107e00 con 0x4a482c0 -3991> 2014-08-13 17:38:05.355841 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3990> 2014-08-13 17:38:05.355843 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3989> 2014-08-13 17:38:05.355844 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107e00 pg_notify(24.181(33) epoch 194487) v5 -3988> 2014-08-13 17:38:05.355847 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355682, event: header_read, request: pg_notify(24.181(33) epoch 194487) v5 -3987> 2014-08-13 17:38:05.355843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3986> 2014-08-13 17:38:05.355851 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355683, event: throttled, request: pg_notify(24.181(33) epoch 194487) v5 -3985> 2014-08-13 17:38:05.355854 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355777, event: all_read, request: pg_notify(24.181(33) epoch 194487) v5 -3984> 2014-08-13 17:38:05.355858 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355841, event: dispatched, request: pg_notify(24.181(33) epoch 194487) v5 -3983> 2014-08-13 17:38:05.355853 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3982> 2014-08-13 17:38:05.355862 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355862, event: waiting_for_osdmap, request: pg_notify(24.181(33) epoch 194487) v5 -3981> 2014-08-13 17:38:05.355865 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.344 -3980> 2014-08-13 17:38:05.355867 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107e00 -3979> 2014-08-13 17:38:05.355869 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355869, event: started, request: pg_notify(24.181(33) epoch 194487) v5 -3978> 2014-08-13 17:38:05.355864 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3977> 2014-08-13 17:38:05.355883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3976> 2014-08-13 17:38:05.355890 7fe9a63f4700 5 -- op tracker -- , seq: 615, time: 2014-08-13 17:38:05.355890, event: done, request: pg_notify(24.181(33) epoch 194487) v5 -3975> 2014-08-13 17:38:05.355896 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3974> 2014-08-13 17:38:05.355897 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3973> 2014-08-13 17:38:05.355892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3972> 2014-08-13 17:38:05.355906 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.331 192.168.1.104:6816/837 9 ==== pg_notify(22.1d2(1) epoch 194487) v5 ==== 777+0+0 (2372295521 0 0) 0x58232c0 con 0x4e83180 -3971> 2014-08-13 17:38:05.355913 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3970> 2014-08-13 17:38:05.355914 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3969> 2014-08-13 17:38:05.355915 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58232c0 pg_notify(22.1d2(1) epoch 194487) v5 -3968> 2014-08-13 17:38:05.355918 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355804, event: header_read, request: pg_notify(22.1d2(1) epoch 194487) v5 -3967> 2014-08-13 17:38:05.355922 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355806, event: throttled, request: pg_notify(22.1d2(1) epoch 194487) v5 -3966> 2014-08-13 17:38:05.355925 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355852, event: all_read, request: pg_notify(22.1d2(1) epoch 194487) v5 -3965> 2014-08-13 17:38:05.355928 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355913, event: dispatched, request: pg_notify(22.1d2(1) epoch 194487) v5 -3964> 2014-08-13 17:38:05.355932 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355932, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194487) v5 -3963> 2014-08-13 17:38:05.355935 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.331 -3962> 2014-08-13 17:38:05.355936 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58232c0 -3961> 2014-08-13 17:38:05.355939 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355939, event: started, request: pg_notify(22.1d2(1) epoch 194487) v5 -3960> 2014-08-13 17:38:05.355769 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.367 (past last_backfill 0//0//-1) -3959> 2014-08-13 17:38:05.355950 7fe9a63f4700 5 -- op tracker -- , seq: 616, time: 2014-08-13 17:38:05.355950, event: done, request: pg_notify(22.1d2(1) epoch 194487) v5 -3958> 2014-08-13 17:38:05.355956 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3957> 2014-08-13 17:38:05.355958 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3956> 2014-08-13 17:38:05.355960 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 20 ==== pg_notify(22.481(39) epoch 194487) v5 ==== 2911+0+0 (183002284 0 0) 0x5103b80 con 0x4e4d6a0 -3955> 2014-08-13 17:38:05.355967 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3954> 2014-08-13 17:38:05.355969 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3953> 2014-08-13 17:38:05.355970 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5103b80 pg_notify(22.481(39) epoch 194487) v5 -3952> 2014-08-13 17:38:05.355972 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355803, event: header_read, request: pg_notify(22.481(39) epoch 194487) v5 -3951> 2014-08-13 17:38:05.355976 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355804, event: throttled, request: pg_notify(22.481(39) epoch 194487) v5 -3950> 2014-08-13 17:38:05.355979 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355878, event: all_read, request: pg_notify(22.481(39) epoch 194487) v5 -3949> 2014-08-13 17:38:05.355982 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355967, event: dispatched, request: pg_notify(22.481(39) epoch 194487) v5 -3948> 2014-08-13 17:38:05.355986 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355986, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194487) v5 -3947> 2014-08-13 17:38:05.355989 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.303 -3946> 2014-08-13 17:38:05.355990 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5103b80 -3945> 2014-08-13 17:38:05.355992 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.355992, event: started, request: pg_notify(22.481(39) epoch 194487) v5 -3944> 2014-08-13 17:38:05.356003 7fe9a63f4700 5 -- op tracker -- , seq: 617, time: 2014-08-13 17:38:05.356003, event: done, request: pg_notify(22.481(39) epoch 194487) v5 -3943> 2014-08-13 17:38:05.356009 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3942> 2014-08-13 17:38:05.356010 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3941> 2014-08-13 17:38:05.356105 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] -3940> 2014-08-13 17:38:05.355944 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.367 (past last_backfill 0//0//-1) -3939> 2014-08-13 17:38:05.356113 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3938> 2014-08-13 17:38:05.356238 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2660 -3937> 2014-08-13 17:38:05.356250 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3936> 2014-08-13 17:38:05.356259 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3935> 2014-08-13 17:38:05.356267 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3934> 2014-08-13 17:38:05.356279 7fe99f5e9700 10 log is not dirty -3933> 2014-08-13 17:38:05.356294 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 -3932> 2014-08-13 17:38:05.356307 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3931> 2014-08-13 17:38:05.356310 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3930> 2014-08-13 17:38:05.356320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3929> 2014-08-13 17:38:05.356330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3928> 2014-08-13 17:38:05.356340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3927> 2014-08-13 17:38:05.356118 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -3926> 2014-08-13 17:38:05.356349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3925> 2014-08-13 17:38:05.356491 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] -3924> 2014-08-13 17:38:05.356498 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3923> 2014-08-13 17:38:05.356354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 378 as a source -3922> 2014-08-13 17:38:05.356570 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2615 -3921> 2014-08-13 17:38:05.356579 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3920> 2014-08-13 17:38:05.356587 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3919> 2014-08-13 17:38:05.356596 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3918> 2014-08-13 17:38:05.356606 7fe99f5e9700 10 log is not dirty -3917> 2014-08-13 17:38:05.356619 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 -3916> 2014-08-13 17:38:05.356632 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3915> 2014-08-13 17:38:05.356636 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] state: Started advmap -3914> 2014-08-13 17:38:05.356645 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] check_recovery_sources no source osds () went down -3913> 2014-08-13 17:38:05.356655 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map -3912> 2014-08-13 17:38:05.356663 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] take_waiters -3911> 2014-08-13 17:38:05.356671 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3910> 2014-08-13 17:38:05.356680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3909> 2014-08-13 17:38:05.356689 7fe99f5e9700 10 log is not dirty -3908> 2014-08-13 17:38:05.356530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.378 (past last_backfill 0//0//-1) -3907> 2014-08-13 17:38:05.356706 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 -3906> 2014-08-13 17:38:05.356723 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3905> 2014-08-13 17:38:05.356726 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap -3904> 2014-08-13 17:38:05.356736 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down -3903> 2014-08-13 17:38:05.356745 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map -3902> 2014-08-13 17:38:05.356780 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters -3901> 2014-08-13 17:38:05.356790 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 -3900> 2014-08-13 17:38:05.356799 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3899> 2014-08-13 17:38:05.356809 7fe99f5e9700 10 log is not dirty -3898> 2014-08-13 17:38:05.356822 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 -3897> 2014-08-13 17:38:05.356838 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3896> 2014-08-13 17:38:05.356841 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Started advmap -3895> 2014-08-13 17:38:05.356851 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down -3894> 2014-08-13 17:38:05.356861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map -3893> 2014-08-13 17:38:05.356704 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.378 (past last_backfill 0//0//-1) -3892> 2014-08-13 17:38:05.356871 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.11 192.168.1.103:6813/37929 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d91d40 con 0x4ecaec0 -3891> 2014-08-13 17:38:05.356888 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3890> 2014-08-13 17:38:05.356891 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3889> 2014-08-13 17:38:05.356892 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d91d40 osd_map(194487..194487 src has 185139..194487) v3 -3888> 2014-08-13 17:38:05.356898 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3887> 2014-08-13 17:38:05.356901 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3886> 2014-08-13 17:38:05.356902 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3885> 2014-08-13 17:38:05.356903 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3884> 2014-08-13 17:38:05.356905 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters -3883> 2014-08-13 17:38:05.356915 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3882> 2014-08-13 17:38:05.356924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3881> 2014-08-13 17:38:05.356934 7fe99f5e9700 10 log is not dirty -3880> 2014-08-13 17:38:05.356931 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.11 192.168.1.103:6813/37929 7 ==== pg_notify(22.1f1(1) epoch 194487) v5 ==== 777+0+0 (4173836126 0 0) 0x5823f00 con 0x4ecaec0 -3879> 2014-08-13 17:38:05.356949 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3878> 2014-08-13 17:38:05.356952 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3877> 2014-08-13 17:38:05.356954 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5823f00 pg_notify(22.1f1(1) epoch 194487) v5 -3876> 2014-08-13 17:38:05.356948 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 -3875> 2014-08-13 17:38:05.356961 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3874> 2014-08-13 17:38:05.356958 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356871, event: header_read, request: pg_notify(22.1f1(1) epoch 194487) v5 -3873> 2014-08-13 17:38:05.356964 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356872, event: throttled, request: pg_notify(22.1f1(1) epoch 194487) v5 -3872> 2014-08-13 17:38:05.356968 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356912, event: all_read, request: pg_notify(22.1f1(1) epoch 194487) v5 -3871> 2014-08-13 17:38:05.356965 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3870> 2014-08-13 17:38:05.356972 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356949, event: dispatched, request: pg_notify(22.1f1(1) epoch 194487) v5 -3869> 2014-08-13 17:38:05.356976 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356976, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194487) v5 -3868> 2014-08-13 17:38:05.356980 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.11 -3867> 2014-08-13 17:38:05.356974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3866> 2014-08-13 17:38:05.356982 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5823f00 -3865> 2014-08-13 17:38:05.356985 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.356985, event: started, request: pg_notify(22.1f1(1) epoch 194487) v5 -3864> 2014-08-13 17:38:05.356985 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3863> 2014-08-13 17:38:05.356996 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3862> 2014-08-13 17:38:05.357002 7fe9a63f4700 5 -- op tracker -- , seq: 618, time: 2014-08-13 17:38:05.357002, event: done, request: pg_notify(22.1f1(1) epoch 194487) v5 -3861> 2014-08-13 17:38:05.357011 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3860> 2014-08-13 17:38:05.357005 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3859> 2014-08-13 17:38:05.357013 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3858> 2014-08-13 17:38:05.356877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.378 (past last_backfill 0//0//-1) -3857> 2014-08-13 17:38:05.357115 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.378 192.168.1.102:6840/19195 5 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9e300 con 0x4eca940 -3856> 2014-08-13 17:38:05.357130 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3855> 2014-08-13 17:38:05.357132 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3854> 2014-08-13 17:38:05.357133 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9e300 osd_map(194487..194487 src has 185139..194487) v3 -3853> 2014-08-13 17:38:05.357138 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3852> 2014-08-13 17:38:05.357141 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3851> 2014-08-13 17:38:05.357142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3850> 2014-08-13 17:38:05.357143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3849> 2014-08-13 17:38:05.357194 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.378 192.168.1.102:6840/19195 6 ==== pg_info(1 pgs e194487:24.9b) v4 ==== 724+0+0 (2053231628 0 0) 0x5820e00 con 0x4eca940 -3848> 2014-08-13 17:38:05.357203 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3847> 2014-08-13 17:38:05.357204 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3846> 2014-08-13 17:38:05.357205 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_info(1 pgs e194487:24.9b) v4 -3845> 2014-08-13 17:38:05.357203 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] -3844> 2014-08-13 17:38:05.357209 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357115, event: header_read, request: pg_info(1 pgs e194487:24.9b) v4 -3843> 2014-08-13 17:38:05.357215 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357116, event: throttled, request: pg_info(1 pgs e194487:24.9b) v4 -3842> 2014-08-13 17:38:05.357212 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3841> 2014-08-13 17:38:05.357218 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357177, event: all_read, request: pg_info(1 pgs e194487:24.9b) v4 -3840> 2014-08-13 17:38:05.357052 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.378 (past last_backfill 0//0//-1) -3839> 2014-08-13 17:38:05.357228 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357202, event: dispatched, request: pg_info(1 pgs e194487:24.9b) v4 -3838> 2014-08-13 17:38:05.357233 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357232, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.9b) v4 -3837> 2014-08-13 17:38:05.357236 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.378 -3836> 2014-08-13 17:38:05.357240 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 -3835> 2014-08-13 17:38:05.357243 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.357243, event: started, request: pg_info(1 pgs e194487:24.9b) v4 -3834> 2014-08-13 17:38:05.357309 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2723 -3833> 2014-08-13 17:38:05.357318 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3832> 2014-08-13 17:38:05.357327 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3831> 2014-08-13 17:38:05.357335 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3830> 2014-08-13 17:38:05.357345 7fe99f5e9700 10 log is not dirty -3829> 2014-08-13 17:38:05.357359 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 -3828> 2014-08-13 17:38:05.357371 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3827> 2014-08-13 17:38:05.357375 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] state: Started advmap -3826> 2014-08-13 17:38:05.357384 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] check_recovery_sources no source osds () went down -3825> 2014-08-13 17:38:05.357393 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map -3824> 2014-08-13 17:38:05.357229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.378 (past last_backfill 0//0//-1) -3823> 2014-08-13 17:38:05.357402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] take_waiters -3822> 2014-08-13 17:38:05.357411 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3821> 2014-08-13 17:38:05.357420 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194485 pi=192977-194484/24 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3820> 2014-08-13 17:38:05.357429 7fe99f5e9700 10 log is not dirty -3819> 2014-08-13 17:38:05.357442 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 -3818> 2014-08-13 17:38:05.357458 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3817> 2014-08-13 17:38:05.357462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Started advmap -3816> 2014-08-13 17:38:05.357472 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down -3815> 2014-08-13 17:38:05.357481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map -3814> 2014-08-13 17:38:05.357527 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters -3813> 2014-08-13 17:38:05.357538 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3812> 2014-08-13 17:38:05.357547 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3811> 2014-08-13 17:38:05.357557 7fe99f5e9700 10 log is not dirty -3810> 2014-08-13 17:38:05.357403 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.378 (past last_backfill 0//0//-1) -3809> 2014-08-13 17:38:05.357571 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 -3808> 2014-08-13 17:38:05.357584 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3807> 2014-08-13 17:38:05.357587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] state: Started advmap -3806> 2014-08-13 17:38:05.357597 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] check_recovery_sources no source osds () went down -3805> 2014-08-13 17:38:05.357606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map -3804> 2014-08-13 17:38:05.357615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] take_waiters -3803> 2014-08-13 17:38:05.357623 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3802> 2014-08-13 17:38:05.357631 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194485 pi=192974-194484/24 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3801> 2014-08-13 17:38:05.357640 7fe99f5e9700 10 log is not dirty -3800> 2014-08-13 17:38:05.357653 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 -3799> 2014-08-13 17:38:05.357677 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3798> 2014-08-13 17:38:05.357680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3797> 2014-08-13 17:38:05.357690 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3796> 2014-08-13 17:38:05.357699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3795> 2014-08-13 17:38:05.357709 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3794> 2014-08-13 17:38:05.357717 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3793> 2014-08-13 17:38:05.357577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.378 (past last_backfill 0//0//-1) -3792> 2014-08-13 17:38:05.357906 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] -3791> 2014-08-13 17:38:05.357914 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3790> 2014-08-13 17:38:05.357750 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.378 (past last_backfill 0//0//-1) -3789> 2014-08-13 17:38:05.358010 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2707 -3788> 2014-08-13 17:38:05.358019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3787> 2014-08-13 17:38:05.358027 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3786> 2014-08-13 17:38:05.358035 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3785> 2014-08-13 17:38:05.358045 7fe99f5e9700 10 log is not dirty -3784> 2014-08-13 17:38:05.358058 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 -3783> 2014-08-13 17:38:05.358074 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3782> 2014-08-13 17:38:05.358078 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Started advmap -3781> 2014-08-13 17:38:05.358087 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down -3780> 2014-08-13 17:38:05.357924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.378 (past last_backfill 0//0//-1) -3779> 2014-08-13 17:38:05.358097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map -3778> 2014-08-13 17:38:05.358145 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters -3777> 2014-08-13 17:38:05.358155 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3776> 2014-08-13 17:38:05.358163 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3775> 2014-08-13 17:38:05.358173 7fe99f5e9700 10 log is not dirty -3774> 2014-08-13 17:38:05.358183 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing -3773> 2014-08-13 17:38:05.358186 7fe99f5e9700 7 osd.4 194487 do_notify osd 17 on 1 PGs -3772> 2014-08-13 17:38:05.358188 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(37) epoch 194487) v5 -- ?+0 0x5823480 con 0x4d3f900 -3771> 2014-08-13 17:38:05.358205 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4c71b80 192.168.1.103:6815/38269 -3770> 2014-08-13 17:38:05.358216 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6815/38269 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x4d898c0 con 0x4c71b80 -3769> 2014-08-13 17:38:05.358227 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.20 has 194487 -3768> 2014-08-13 17:38:05.358229 7fe99f5e9700 7 osd.4 194487 do_notify osd 20 on 1 PGs -3767> 2014-08-13 17:38:05.358230 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194487) v5 -- ?+0 0x58201c0 con 0x4c71b80 -3766> 2014-08-13 17:38:05.358097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.378 (past last_backfill 0//0//-1) -3765> 2014-08-13 17:38:05.358315 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x49e39c0 192.168.1.101:6808/22782 -3764> 2014-08-13 17:38:05.358332 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6808/22782 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54da880 con 0x49e39c0 -3763> 2014-08-13 17:38:05.358344 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.304 has 194487 -3762> 2014-08-13 17:38:05.358347 7fe99f5e9700 7 osd.4 194487 do_notify osd 304 on 1 PGs -3761> 2014-08-13 17:38:05.358348 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(31) epoch 194487) v5 -- ?+0 0x5824280 con 0x49e39c0 -3760> 2014-08-13 17:38:05.358356 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4e49760 don't know epoch, doing nothing -3759> 2014-08-13 17:38:05.358358 7fe99f5e9700 7 osd.4 194487 do_notify osd 324 on 1 PGs -3758> 2014-08-13 17:38:05.358359 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194487) v5 -- ?+0 0x5826740 con 0x4e49760 -3757> 2014-08-13 17:38:05.358371 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4ec9b80 192.168.1.106:6802/4876 -3756> 2014-08-13 17:38:05.358376 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6802/4876 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54da400 con 0x4ec9b80 -3755> 2014-08-13 17:38:05.358383 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.330 has 194487 -3754> 2014-08-13 17:38:05.358385 7fe99f5e9700 7 osd.4 194487 do_notify osd 330 on 1 PGs -3753> 2014-08-13 17:38:05.358386 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(39) epoch 194487) v5 -- ?+0 0x5105400 con 0x4ec9b80 -3752> 2014-08-13 17:38:05.358275 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.378 (past last_backfill 0//0//-1) -3751> 2014-08-13 17:38:05.358449 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4e4a100 192.168.1.104:6820/2308 -3750> 2014-08-13 17:38:05.358460 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6820/2308 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54df500 con 0x4e4a100 -3749> 2014-08-13 17:38:05.358471 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.346 has 194487 -3748> 2014-08-13 17:38:05.358473 7fe99f5e9700 7 osd.4 194487 do_notify osd 346 on 1 PGs -3747> 2014-08-13 17:38:05.358474 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(33) epoch 194487) v5 -- ?+0 0x51071c0 con 0x4e4a100 -3746> 2014-08-13 17:38:05.358521 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4cfc620 192.168.1.114:6826/52317 -3745> 2014-08-13 17:38:05.358528 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.114:6826/52317 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54df980 con 0x4cfc620 -3744> 2014-08-13 17:38:05.358537 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.364 has 194487 -3743> 2014-08-13 17:38:05.358539 7fe99f5e9700 7 osd.4 194487 do_notify osd 364 on 1 PGs -3742> 2014-08-13 17:38:05.358541 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(34) epoch 194487) v5 -- ?+0 0x5107a80 con 0x4cfc620 -3741> 2014-08-13 17:38:05.358578 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 26 -3740> 2014-08-13 17:38:05.358587 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x592c360 don't know epoch, doing nothing -3739> 2014-08-13 17:38:05.358604 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.115:6802/58522 -- pg_info(1 pgs e194487:24.34(52)) v4 -- ?+0 0x51055c0 con 0x592c360 -3738> 2014-08-13 17:38:05.358615 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 30 -3737> 2014-08-13 17:38:05.358449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.378 (past last_backfill 0//0//-1) -3736> 2014-08-13 17:38:05.358621 7fe99f5e9700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4a44780 192.168.1.107:6823/55491 -3735> 2014-08-13 17:38:05.358627 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6823/55491 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x5499d40 con 0x4a44780 -3734> 2014-08-13 17:38:05.358636 7fe99f5e9700 10 osd.4 194487 note_peer_epoch osd.30 has 194487 -3733> 2014-08-13 17:38:05.358648 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6823/55491 -- pg_info(1 pgs e194487:24.34(52)) v4 -- ?+0 0x5102140 con 0x4a44780 -3732> 2014-08-13 17:38:05.358674 7fe99f5e9700 20 osd.4 194487 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) to shard 340 -3731> 2014-08-13 17:38:05.358681 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -3730> 2014-08-13 17:38:05.358694 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194487:24.34(52)) v4 -- ?+0 0x6012140 con 0x49e40a0 -3729> 2014-08-13 17:38:05.358752 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 -3728> 2014-08-13 17:38:05.358772 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3727> 2014-08-13 17:38:05.358778 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap -3726> 2014-08-13 17:38:05.358622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.378 (past last_backfill 0//0//-1) -3725> 2014-08-13 17:38:05.358789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down -3724> 2014-08-13 17:38:05.358800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map -3723> 2014-08-13 17:38:05.358809 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters -3722> 2014-08-13 17:38:05.358818 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3721> 2014-08-13 17:38:05.358826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3720> 2014-08-13 17:38:05.358837 7fe99f5e9700 10 log is not dirty -3719> 2014-08-13 17:38:05.358852 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 -3718> 2014-08-13 17:38:05.358865 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3717> 2014-08-13 17:38:05.358868 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3716> 2014-08-13 17:38:05.358878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3715> 2014-08-13 17:38:05.358889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3714> 2014-08-13 17:38:05.358898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3713> 2014-08-13 17:38:05.358907 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3712> 2014-08-13 17:38:05.358796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.378 (past last_backfill 0//0//-1) -3711> 2014-08-13 17:38:05.359030 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] -3710> 2014-08-13 17:38:05.359039 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3709> 2014-08-13 17:38:05.359102 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2571 -3708> 2014-08-13 17:38:05.359111 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3707> 2014-08-13 17:38:05.359119 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3706> 2014-08-13 17:38:05.359128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194486/194484 194485/194485/194485) [4,310,21] r=0 lpr=194485 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3705> 2014-08-13 17:38:05.359138 7fe99f5e9700 10 log is not dirty -3704> 2014-08-13 17:38:05.358969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.378 (past last_backfill 0//0//-1) -3703> 2014-08-13 17:38:05.359152 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 -3702> 2014-08-13 17:38:05.359171 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3701> 2014-08-13 17:38:05.359175 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Started advmap -3700> 2014-08-13 17:38:05.359192 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down -3699> 2014-08-13 17:38:05.359202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map -3698> 2014-08-13 17:38:05.359242 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters -3697> 2014-08-13 17:38:05.359252 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3696> 2014-08-13 17:38:05.359261 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194477 n=63 ec=192688 les/c 194484/194480 194485/194486/192968) [289,4,285] r=1 lpr=194486 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3695> 2014-08-13 17:38:05.359271 7fe99f5e9700 10 log is not dirty -3694> 2014-08-13 17:38:05.359285 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 -3693> 2014-08-13 17:38:05.359298 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3692> 2014-08-13 17:38:05.359301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3691> 2014-08-13 17:38:05.359142 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.378 (past last_backfill 0//0//-1) -3690> 2014-08-13 17:38:05.359311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3689> 2014-08-13 17:38:05.359321 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3688> 2014-08-13 17:38:05.359331 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3687> 2014-08-13 17:38:05.359340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3686> 2014-08-13 17:38:05.359318 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.378 (past last_backfill 0//0//-1) -3685> 2014-08-13 17:38:05.359552 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] -3684> 2014-08-13 17:38:05.359560 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3683> 2014-08-13 17:38:05.359492 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.378 (past last_backfill 0//0//-1) -3682> 2014-08-13 17:38:05.359666 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:3008 -3681> 2014-08-13 17:38:05.359676 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3680> 2014-08-13 17:38:05.359684 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3679> 2014-08-13 17:38:05.359693 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194486/194483 194485/194485/194485) [4,5,47] r=0 lpr=194485 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3678> 2014-08-13 17:38:05.359702 7fe99f5e9700 10 log is not dirty -3677> 2014-08-13 17:38:05.359716 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 -3676> 2014-08-13 17:38:05.359729 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3675> 2014-08-13 17:38:05.359732 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] state: Started advmap -3674> 2014-08-13 17:38:05.359742 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] check_recovery_sources no source osds () went down -3673> 2014-08-13 17:38:05.359751 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map -3672> 2014-08-13 17:38:05.359760 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] take_waiters -3671> 2014-08-13 17:38:05.359768 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3670> 2014-08-13 17:38:05.359776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194485 pi=192969-194484/24 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3669> 2014-08-13 17:38:05.359785 7fe99f5e9700 10 log is not dirty -3668> 2014-08-13 17:38:05.359798 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 -3667> 2014-08-13 17:38:05.359816 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3666> 2014-08-13 17:38:05.359820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap -3665> 2014-08-13 17:38:05.359665 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.378 (past last_backfill 0//0//-1) -3664> 2014-08-13 17:38:05.359832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194486, need_up_thru now false -3663> 2014-08-13 17:38:05.359843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap -3662> 2014-08-13 17:38:05.359854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down -3661> 2014-08-13 17:38:05.359865 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map -3660> 2014-08-13 17:38:05.359876 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary -3659> 2014-08-13 17:38:05.359886 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194487:22349 -3658> 2014-08-13 17:38:05.359896 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters -3657> 2014-08-13 17:38:05.359907 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 0.920806 4 0.000648 -3656> 2014-08-13 17:38:05.359920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering -3655> 2014-08-13 17:38:05.359930 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.008680 0 0.000000 -3654> 2014-08-13 17:38:05.359945 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -3653> 2014-08-13 17:38:05.359955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194478 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate -3652> 2014-08-13 17:38:05.359967 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] -3651> 2014-08-13 17:38:05.359977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] -3650> 2014-08-13 17:38:05.359986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) -3649> 2014-08-13 17:38:05.359839 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.378 (past last_backfill 0//0//-1) -3648> 2014-08-13 17:38:05.360012 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.378 (past last_backfill 0//0//-1) -3647> 2014-08-13 17:38:05.360188 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.378 (past last_backfill 0//0//-1) -3646> 2014-08-13 17:38:05.360361 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.378 (past last_backfill 0//0//-1) -3645> 2014-08-13 17:38:05.360612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery -3644> 2014-08-13 17:38:05.360627 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] -3643> 2014-08-13 17:38:05.360648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -3642> 2014-08-13 17:38:05.360661 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 is up to date, queueing in pending_activators -3641> 2014-08-13 17:38:05.360534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.378 (past last_backfill 0//0//-1) -3640> 2014-08-13 17:38:05.360704 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3639> 2014-08-13 17:38:05.360718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -3638> 2014-08-13 17:38:05.360737 7fe99f5e9700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 -3637> 2014-08-13 17:38:05.360707 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.378 (past last_backfill 0//0//-1) -3636> 2014-08-13 17:38:05.360897 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 sending log((192834'13289,194422'15454], crt=194422'15454) -3635> 2014-08-13 17:38:05.360913 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x49e6300 don't know epoch, doing nothing -3634> 2014-08-13 17:38:05.360916 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.28 192.168.1.103:6805/37391 -- pg_log(24.b6 epoch 194487 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194487) v4 -- ?+0 0x5fb3c00 -3633> 2014-08-13 17:38:05.360932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3632> 2014-08-13 17:38:05.360947 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) -3631> 2014-08-13 17:38:05.360959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 is up to date, queueing in pending_activators -3630> 2014-08-13 17:38:05.361011 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194484 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3629> 2014-08-13 17:38:05.361023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -3628> 2014-08-13 17:38:05.361039 7fe99f5e9700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 -3627> 2014-08-13 17:38:05.360881 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.378 (past last_backfill 0//0//-1) -3626> 2014-08-13 17:38:05.361054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.378 (past last_backfill 0//0//-1) -3625> 2014-08-13 17:38:05.361216 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 sending log((192834'13289,194422'15454], crt=194422'15454) -3624> 2014-08-13 17:38:05.361233 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4ec9b80 already has epoch 194487 -3623> 2014-08-13 17:38:05.361240 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.330 192.168.1.106:6802/4876 -- pg_log(24.b6 epoch 194487 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194487) v4 -- ?+0 0x5fb4800 -3622> 2014-08-13 17:38:05.361255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3621> 2014-08-13 17:38:05.361300 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing -3620> 2014-08-13 17:38:05.361230 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.378 (past last_backfill 0//0//-1) -3619> 2014-08-13 17:38:05.361374 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.4 -3618> 2014-08-13 17:38:05.361445 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.4 -3617> 2014-08-13 17:38:05.361408 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.378 (past last_backfill 0//0//-1) -3616> 2014-08-13 17:38:05.361515 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.4 -3615> 2014-08-13 17:38:05.361584 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.4 -3614> 2014-08-13 17:38:05.361654 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.4 -3613> 2014-08-13 17:38:05.361582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.378 (past last_backfill 0//0//-1) -3612> 2014-08-13 17:38:05.361723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.4 -3611> 2014-08-13 17:38:05.361793 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.4 -3610> 2014-08-13 17:38:05.361756 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.378 (past last_backfill 0//0//-1) -3609> 2014-08-13 17:38:05.361862 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.4 -3608> 2014-08-13 17:38:05.361931 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.4 -3607> 2014-08-13 17:38:05.362001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.4 -3606> 2014-08-13 17:38:05.361929 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.378 (past last_backfill 0//0//-1) -3605> 2014-08-13 17:38:05.362070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.4 -3604> 2014-08-13 17:38:05.362140 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.4 -3603> 2014-08-13 17:38:05.362102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.378 (past last_backfill 0//0//-1) -3602> 2014-08-13 17:38:05.362215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.4 -3601> 2014-08-13 17:38:05.362285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.4 -3600> 2014-08-13 17:38:05.362355 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.4 -3599> 2014-08-13 17:38:05.362278 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.378 (past last_backfill 0//0//-1) -3598> 2014-08-13 17:38:05.362424 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.4 -3597> 2014-08-13 17:38:05.362513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.4 -3596> 2014-08-13 17:38:05.362452 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.378 (past last_backfill 0//0//-1) -3595> 2014-08-13 17:38:05.362583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.4 -3594> 2014-08-13 17:38:05.362661 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.4 -3593> 2014-08-13 17:38:05.362732 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.4 -3592> 2014-08-13 17:38:05.362625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.378 (past last_backfill 0//0//-1) -3591> 2014-08-13 17:38:05.362802 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.4 -3590> 2014-08-13 17:38:05.362874 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.4 -3589> 2014-08-13 17:38:05.362803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.378 (past last_backfill 0//0//-1) -3588> 2014-08-13 17:38:05.362943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.4 -3587> 2014-08-13 17:38:05.363016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.4 -3586> 2014-08-13 17:38:05.362977 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.378 (past last_backfill 0//0//-1) -3585> 2014-08-13 17:38:05.363087 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.4 -3584> 2014-08-13 17:38:05.363157 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.4 -3583> 2014-08-13 17:38:05.363231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.4 -3582> 2014-08-13 17:38:05.363152 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.378 (past last_backfill 0//0//-1) -3581> 2014-08-13 17:38:05.363301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.4 -3580> 2014-08-13 17:38:05.363372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.4 -3579> 2014-08-13 17:38:05.363330 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.378 (past last_backfill 0//0//-1) -3578> 2014-08-13 17:38:05.363443 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.4 -3577> 2014-08-13 17:38:05.363513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.4 -3576> 2014-08-13 17:38:05.363583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.4 -3575> 2014-08-13 17:38:05.363505 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.378 (past last_backfill 0//0//-1) -3574> 2014-08-13 17:38:05.363654 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.4 -3573> 2014-08-13 17:38:05.363725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.4 -3572> 2014-08-13 17:38:05.363796 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.4 -3571> 2014-08-13 17:38:05.363700 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.378 (past last_backfill 0//0//-1) -3570> 2014-08-13 17:38:05.363867 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.4 -3569> 2014-08-13 17:38:05.363938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.4 -3568> 2014-08-13 17:38:05.363875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.378 (past last_backfill 0//0//-1) -3567> 2014-08-13 17:38:05.364008 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.4 -3566> 2014-08-13 17:38:05.364079 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.4 -3565> 2014-08-13 17:38:05.364150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.4 -3564> 2014-08-13 17:38:05.364050 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.378 (past last_backfill 0//0//-1) -3563> 2014-08-13 17:38:05.364224 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.4 -3562> 2014-08-13 17:38:05.364296 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.4 -3561> 2014-08-13 17:38:05.364226 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.378 (past last_backfill 0//0//-1) -3560> 2014-08-13 17:38:05.364367 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.4 -3559> 2014-08-13 17:38:05.364438 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.4 -3558> 2014-08-13 17:38:05.364402 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.378 (past last_backfill 0//0//-1) -3557> 2014-08-13 17:38:05.364509 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.4 -3556> 2014-08-13 17:38:05.364578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.4 -3555> 2014-08-13 17:38:05.364649 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.4 -3554> 2014-08-13 17:38:05.364577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.378 (past last_backfill 0//0//-1) -3553> 2014-08-13 17:38:05.364720 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.4 -3552> 2014-08-13 17:38:05.364791 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.4 -3551> 2014-08-13 17:38:05.364752 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.378 (past last_backfill 0//0//-1) -3550> 2014-08-13 17:38:05.364862 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.4 -3549> 2014-08-13 17:38:05.364932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.4 -3548> 2014-08-13 17:38:05.365003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.4 -3547> 2014-08-13 17:38:05.364927 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.378 (past last_backfill 0//0//-1) -3546> 2014-08-13 17:38:05.365073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.4 -3545> 2014-08-13 17:38:05.365144 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.4 -3544> 2014-08-13 17:38:05.365102 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.378 (past last_backfill 0//0//-1) -3543> 2014-08-13 17:38:05.365218 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.4 -3542> 2014-08-13 17:38:05.365288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.4 -3541> 2014-08-13 17:38:05.365359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.4 -3540> 2014-08-13 17:38:05.365279 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.378 (past last_backfill 0//0//-1) -3539> 2014-08-13 17:38:05.365430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.4 -3538> 2014-08-13 17:38:05.365501 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.4 -3537> 2014-08-13 17:38:05.365454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.378 (past last_backfill 0//0//-1) -3536> 2014-08-13 17:38:05.365571 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -3535> 2014-08-13 17:38:05.365698 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 17 as a source -3534> 2014-08-13 17:38:05.365629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.378 (past last_backfill 0//0//-1) -3533> 2014-08-13 17:38:05.365770 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.17 -3532> 2014-08-13 17:38:05.365842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.17 -3531> 2014-08-13 17:38:05.365804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.378 (past last_backfill 0//0//-1) -3530> 2014-08-13 17:38:05.365918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.17 -3529> 2014-08-13 17:38:05.365993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.17 -3528> 2014-08-13 17:38:05.366071 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.17 -3527> 2014-08-13 17:38:05.365979 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.378 (past last_backfill 0//0//-1) -3526> 2014-08-13 17:38:05.366151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.17 -3525> 2014-08-13 17:38:05.366237 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.17 -3524> 2014-08-13 17:38:05.366152 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.378 (past last_backfill 0//0//-1) -3523> 2014-08-13 17:38:05.366322 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.17 -3522> 2014-08-13 17:38:05.366328 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.378 (past last_backfill 0//0//-1) -3521> 2014-08-13 17:38:05.366427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.17 -3520> 2014-08-13 17:38:05.366513 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.17 -3519> 2014-08-13 17:38:05.366504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.378 (past last_backfill 0//0//-1) -3518> 2014-08-13 17:38:05.366601 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.17 -3517> 2014-08-13 17:38:05.366691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.17 -3516> 2014-08-13 17:38:05.366679 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.378 (past last_backfill 0//0//-1) -3515> 2014-08-13 17:38:05.366783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.17 -3514> 2014-08-13 17:38:05.366877 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.17 -3513> 2014-08-13 17:38:05.366972 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.17 -3512> 2014-08-13 17:38:05.366854 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -3511> 2014-08-13 17:38:05.367070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.17 -3510> 2014-08-13 17:38:05.367077 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound -3509> 2014-08-13 17:38:05.367169 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.17 -3508> 2014-08-13 17:38:05.367274 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.17 -3507> 2014-08-13 17:38:05.367255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194484 -3506> 2014-08-13 17:38:05.367377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.17 -3505> 2014-08-13 17:38:05.367481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.17 -3504> 2014-08-13 17:38:05.367432 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 291,350,365,367,378 -3503> 2014-08-13 17:38:05.367587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.17 -3502> 2014-08-13 17:38:05.367608 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished -3501> 2014-08-13 17:38:05.367695 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.17 -3500> 2014-08-13 17:38:05.367807 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.17 -3499> 2014-08-13 17:38:05.367786 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -3498> 2014-08-13 17:38:05.367918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.17 -3497> 2014-08-13 17:38:05.367963 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3496> 2014-08-13 17:38:05.368031 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.17 -3495> 2014-08-13 17:38:05.368145 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.17 -3494> 2014-08-13 17:38:05.368139 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3493> 2014-08-13 17:38:05.368370 7fe99ebe8700 10 log is not dirty -3492> 2014-08-13 17:38:05.368265 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.17 -3491> 2014-08-13 17:38:05.368411 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 -3490> 2014-08-13 17:38:05.368430 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3489> 2014-08-13 17:38:05.368435 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] state: Started advmap -3488> 2014-08-13 17:38:05.368435 7fe9a63f4700 5 -- op tracker -- , seq: 619, time: 2014-08-13 17:38:05.368434, event: done, request: pg_info(1 pgs e194487:24.9b) v4 -3487> 2014-08-13 17:38:05.368447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] check_recovery_sources no source osds () went down -3486> 2014-08-13 17:38:05.368454 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3485> 2014-08-13 17:38:05.368457 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3484> 2014-08-13 17:38:05.368457 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map -3483> 2014-08-13 17:38:05.368463 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.373 192.168.1.108:6840/7435 16 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d96300 con 0x4e85c20 -3482> 2014-08-13 17:38:05.368467 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] take_waiters -3481> 2014-08-13 17:38:05.368475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3480> 2014-08-13 17:38:05.368477 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3479> 2014-08-13 17:38:05.368479 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d96300 osd_map(194487..194487 src has 185139..194487) v3 -3478> 2014-08-13 17:38:05.368476 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3477> 2014-08-13 17:38:05.368485 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3476> 2014-08-13 17:38:05.368488 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3475> 2014-08-13 17:38:05.368490 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3474> 2014-08-13 17:38:05.368491 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3473> 2014-08-13 17:38:05.368485 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194485 pi=192979-194484/29 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3472> 2014-08-13 17:38:05.368496 7fe99ebe8700 10 log is not dirty -3471> 2014-08-13 17:38:05.368382 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.17 -3470> 2014-08-13 17:38:05.368495 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.342 192.168.1.110:6802/41438 11 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db2400 con 0x4ec8580 -3469> 2014-08-13 17:38:05.368502 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3468> 2014-08-13 17:38:05.368503 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3467> 2014-08-13 17:38:05.368504 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db2400 osd_map(194487..194487 src has 185139..194487) v3 -3466> 2014-08-13 17:38:05.368508 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3465> 2014-08-13 17:38:05.368510 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3464> 2014-08-13 17:38:05.368511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3463> 2014-08-13 17:38:05.368512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3462> 2014-08-13 17:38:05.368515 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.396 192.168.1.105:6814/41806 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d92ac0 con 0x4ecb700 -3461> 2014-08-13 17:38:05.368511 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 -3460> 2014-08-13 17:38:05.368522 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3459> 2014-08-13 17:38:05.368526 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3458> 2014-08-13 17:38:05.368526 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3457> 2014-08-13 17:38:05.368527 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d92ac0 osd_map(194487..194487 src has 185139..194487) v3 -3456> 2014-08-13 17:38:05.368531 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3455> 2014-08-13 17:38:05.368533 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3454> 2014-08-13 17:38:05.368534 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3453> 2014-08-13 17:38:05.368535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3452> 2014-08-13 17:38:05.368530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3451> 2014-08-13 17:38:05.368543 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.375 192.168.1.114:6830/52879 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dba80 con 0x4313f40 -3450> 2014-08-13 17:38:05.368550 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3449> 2014-08-13 17:38:05.368551 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3448> 2014-08-13 17:38:05.368553 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dba80 osd_map(194487..194487 src has 185139..194487) v3 -3447> 2014-08-13 17:38:05.368556 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3446> 2014-08-13 17:38:05.368558 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3445> 2014-08-13 17:38:05.368559 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3444> 2014-08-13 17:38:05.368560 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3443> 2014-08-13 17:38:05.368541 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3442> 2014-08-13 17:38:05.368562 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.366 192.168.1.108:6828/4534 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54df740 con 0x53856a0 -3441> 2014-08-13 17:38:05.368569 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3440> 2014-08-13 17:38:05.368571 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3439> 2014-08-13 17:38:05.368572 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df740 osd_map(194487..194487 src has 185139..194487) v3 -3438> 2014-08-13 17:38:05.368575 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3437> 2014-08-13 17:38:05.368578 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3436> 2014-08-13 17:38:05.368568 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,312) went down -3435> 2014-08-13 17:38:05.368579 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3434> 2014-08-13 17:38:05.368580 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3433> 2014-08-13 17:38:05.368582 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.291 192.168.1.101:6820/25633 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d8f500 con 0x4a423c0 -3432> 2014-08-13 17:38:05.368581 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3431> 2014-08-13 17:38:05.368589 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3430> 2014-08-13 17:38:05.368590 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3429> 2014-08-13 17:38:05.368592 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d8f500 osd_map(194487..194487 src has 185139..194487) v3 -3428> 2014-08-13 17:38:05.368595 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3427> 2014-08-13 17:38:05.368597 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3426> 2014-08-13 17:38:05.368598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3425> 2014-08-13 17:38:05.368599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3424> 2014-08-13 17:38:05.368590 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3423> 2014-08-13 17:38:05.368607 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.367 192.168.1.114:6840/55513 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x549a880 con 0x4e48c60 -3422> 2014-08-13 17:38:05.368614 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3421> 2014-08-13 17:38:05.368615 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3420> 2014-08-13 17:38:05.368616 7fe9a63f4700 20 osd.4 194487 _dispatch 0x549a880 osd_map(194487..194487 src has 185139..194487) v3 -3419> 2014-08-13 17:38:05.368500 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.17 -3418> 2014-08-13 17:38:05.368619 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3417> 2014-08-13 17:38:05.368621 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3416> 2014-08-13 17:38:05.368622 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3415> 2014-08-13 17:38:05.368623 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3414> 2014-08-13 17:38:05.368626 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.373 192.168.1.108:6840/7435 17 ==== pg_notify(24.3d(32) epoch 194487) v5 ==== 2548+0+0 (3638720033 0 0) 0x5102f40 con 0x4e85c20 -3413> 2014-08-13 17:38:05.368633 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3412> 2014-08-13 17:38:05.368635 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3411> 2014-08-13 17:38:05.368635 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_notify(24.3d(32) epoch 194487) v5 -3410> 2014-08-13 17:38:05.368639 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.358447, event: header_read, request: pg_notify(24.3d(32) epoch 194487) v5 -3409> 2014-08-13 17:38:05.368644 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.358448, event: throttled, request: pg_notify(24.3d(32) epoch 194487) v5 -3408> 2014-08-13 17:38:05.368647 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.358536, event: all_read, request: pg_notify(24.3d(32) epoch 194487) v5 -3407> 2014-08-13 17:38:05.368651 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.368633, event: dispatched, request: pg_notify(24.3d(32) epoch 194487) v5 -3406> 2014-08-13 17:38:05.368655 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.368654, event: waiting_for_osdmap, request: pg_notify(24.3d(32) epoch 194487) v5 -3405> 2014-08-13 17:38:05.368658 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.373 -3404> 2014-08-13 17:38:05.368660 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 -3403> 2014-08-13 17:38:05.368663 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.368663, event: started, request: pg_notify(24.3d(32) epoch 194487) v5 -3402> 2014-08-13 17:38:05.368681 7fe9a63f4700 5 -- op tracker -- , seq: 620, time: 2014-08-13 17:38:05.368681, event: done, request: pg_notify(24.3d(32) epoch 194487) v5 -3401> 2014-08-13 17:38:05.368689 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3400> 2014-08-13 17:38:05.368690 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3399> 2014-08-13 17:38:05.368706 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.342 192.168.1.110:6802/41438 12 ==== pg_notify(22.113(1) epoch 194487) v5 ==== 777+0+0 (611239838 0 0) 0x5825780 con 0x4ec8580 -3398> 2014-08-13 17:38:05.368715 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3397> 2014-08-13 17:38:05.368716 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3396> 2014-08-13 17:38:05.368717 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825780 pg_notify(22.113(1) epoch 194487) v5 -3395> 2014-08-13 17:38:05.368720 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.364627, event: header_read, request: pg_notify(22.113(1) epoch 194487) v5 -3394> 2014-08-13 17:38:05.368724 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.364628, event: throttled, request: pg_notify(22.113(1) epoch 194487) v5 -3393> 2014-08-13 17:38:05.368727 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.364667, event: all_read, request: pg_notify(22.113(1) epoch 194487) v5 -3392> 2014-08-13 17:38:05.368730 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.368714, event: dispatched, request: pg_notify(22.113(1) epoch 194487) v5 -3391> 2014-08-13 17:38:05.368734 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.368734, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194487) v5 -3390> 2014-08-13 17:38:05.368745 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.342 -3389> 2014-08-13 17:38:05.368747 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825780 -3388> 2014-08-13 17:38:05.368624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.17 -3387> 2014-08-13 17:38:05.368749 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.368749, event: started, request: pg_notify(22.113(1) epoch 194487) v5 -3386> 2014-08-13 17:38:05.368762 7fe9a63f4700 5 -- op tracker -- , seq: 621, time: 2014-08-13 17:38:05.368762, event: done, request: pg_notify(22.113(1) epoch 194487) v5 -3385> 2014-08-13 17:38:05.368768 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3384> 2014-08-13 17:38:05.368769 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3383> 2014-08-13 17:38:05.368773 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.396 192.168.1.105:6814/41806 11 ==== pg_notify(22.3d9(45) epoch 194487) v5 ==== 3165+0+0 (2965190535 0 0) 0x513e3c0 con 0x4ecb700 -3382> 2014-08-13 17:38:05.368780 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3381> 2014-08-13 17:38:05.368781 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3380> 2014-08-13 17:38:05.368782 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513e3c0 pg_notify(22.3d9(45) epoch 194487) v5 -3379> 2014-08-13 17:38:05.368776 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] -3378> 2014-08-13 17:38:05.368785 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.362506, event: header_read, request: pg_notify(22.3d9(45) epoch 194487) v5 -3377> 2014-08-13 17:38:05.368789 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.362506, event: throttled, request: pg_notify(22.3d9(45) epoch 194487) v5 -3376> 2014-08-13 17:38:05.368786 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3375> 2014-08-13 17:38:05.368792 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.362581, event: all_read, request: pg_notify(22.3d9(45) epoch 194487) v5 -3374> 2014-08-13 17:38:05.368796 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.368780, event: dispatched, request: pg_notify(22.3d9(45) epoch 194487) v5 -3373> 2014-08-13 17:38:05.368800 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.368800, event: waiting_for_osdmap, request: pg_notify(22.3d9(45) epoch 194487) v5 -3372> 2014-08-13 17:38:05.368803 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.396 -3371> 2014-08-13 17:38:05.368804 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513e3c0 -3370> 2014-08-13 17:38:05.368806 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.368806, event: started, request: pg_notify(22.3d9(45) epoch 194487) v5 -3369> 2014-08-13 17:38:05.368818 7fe9a63f4700 5 -- op tracker -- , seq: 622, time: 2014-08-13 17:38:05.368818, event: done, request: pg_notify(22.3d9(45) epoch 194487) v5 -3368> 2014-08-13 17:38:05.368824 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3367> 2014-08-13 17:38:05.368825 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3366> 2014-08-13 17:38:05.368835 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.375 192.168.1.114:6830/52879 8 ==== pg_notify(24.30b(39) epoch 194487) v5 ==== 2957+0+0 (2090817232 0 0) 0x513dcc0 con 0x4313f40 -3365> 2014-08-13 17:38:05.368843 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3364> 2014-08-13 17:38:05.368844 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3363> 2014-08-13 17:38:05.368845 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513dcc0 pg_notify(24.30b(39) epoch 194487) v5 -3362> 2014-08-13 17:38:05.368848 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.362766, event: header_read, request: pg_notify(24.30b(39) epoch 194487) v5 -3361> 2014-08-13 17:38:05.368851 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.362767, event: throttled, request: pg_notify(24.30b(39) epoch 194487) v5 -3360> 2014-08-13 17:38:05.368855 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.362839, event: all_read, request: pg_notify(24.30b(39) epoch 194487) v5 -3359> 2014-08-13 17:38:05.368858 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.368842, event: dispatched, request: pg_notify(24.30b(39) epoch 194487) v5 -3358> 2014-08-13 17:38:05.368861 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.368861, event: waiting_for_osdmap, request: pg_notify(24.30b(39) epoch 194487) v5 -3357> 2014-08-13 17:38:05.368865 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.375 -3356> 2014-08-13 17:38:05.368866 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513dcc0 -3355> 2014-08-13 17:38:05.368869 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.368869, event: started, request: pg_notify(24.30b(39) epoch 194487) v5 -3354> 2014-08-13 17:38:05.368751 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.17 -3353> 2014-08-13 17:38:05.368881 7fe9a63f4700 5 -- op tracker -- , seq: 623, time: 2014-08-13 17:38:05.368881, event: done, request: pg_notify(24.30b(39) epoch 194487) v5 -3352> 2014-08-13 17:38:05.368879 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2862 -3351> 2014-08-13 17:38:05.368890 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3350> 2014-08-13 17:38:05.368891 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3349> 2014-08-13 17:38:05.368889 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3348> 2014-08-13 17:38:05.368898 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3347> 2014-08-13 17:38:05.368906 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.366 192.168.1.108:6828/4534 7 ==== pg_notify(22.39d(1) epoch 194487) v5 ==== 777+0+0 (4091920854 0 0) 0x6012a00 con 0x53856a0 -3346> 2014-08-13 17:38:05.368914 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3345> 2014-08-13 17:38:05.368907 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3344> 2014-08-13 17:38:05.368915 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3343> 2014-08-13 17:38:05.368916 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6012a00 pg_notify(22.39d(1) epoch 194487) v5 -3342> 2014-08-13 17:38:05.368918 7fe99ebe8700 10 log is not dirty -3341> 2014-08-13 17:38:05.368919 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.361660, event: header_read, request: pg_notify(22.39d(1) epoch 194487) v5 -3340> 2014-08-13 17:38:05.368923 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.361661, event: throttled, request: pg_notify(22.39d(1) epoch 194487) v5 -3339> 2014-08-13 17:38:05.368926 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.361699, event: all_read, request: pg_notify(22.39d(1) epoch 194487) v5 -3338> 2014-08-13 17:38:05.368929 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.368914, event: dispatched, request: pg_notify(22.39d(1) epoch 194487) v5 -3337> 2014-08-13 17:38:05.368933 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.368933, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194487) v5 -3336> 2014-08-13 17:38:05.368937 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.366 -3335> 2014-08-13 17:38:05.368938 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6012a00 -3334> 2014-08-13 17:38:05.368940 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.368940, event: started, request: pg_notify(22.39d(1) epoch 194487) v5 -3333> 2014-08-13 17:38:05.368933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 -3332> 2014-08-13 17:38:05.368946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3331> 2014-08-13 17:38:05.368952 7fe9a63f4700 5 -- op tracker -- , seq: 624, time: 2014-08-13 17:38:05.368952, event: done, request: pg_notify(22.39d(1) epoch 194487) v5 -3330> 2014-08-13 17:38:05.368958 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3329> 2014-08-13 17:38:05.368951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] state: Started advmap -3328> 2014-08-13 17:38:05.368959 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3327> 2014-08-13 17:38:05.368960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] check_recovery_sources no source osds () went down -3326> 2014-08-13 17:38:05.368966 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.291 192.168.1.101:6820/25633 8 ==== pg_notify(24.9b(34) epoch 194487) v5 ==== 2666+0+0 (3903675377 0 0) 0x5821a40 con 0x4a423c0 -3325> 2014-08-13 17:38:05.368973 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3324> 2014-08-13 17:38:05.368975 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3323> 2014-08-13 17:38:05.368976 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5821a40 pg_notify(24.9b(34) epoch 194487) v5 -3322> 2014-08-13 17:38:05.368970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map -3321> 2014-08-13 17:38:05.368979 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.361136, event: header_read, request: pg_notify(24.9b(34) epoch 194487) v5 -3320> 2014-08-13 17:38:05.368983 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.361137, event: throttled, request: pg_notify(24.9b(34) epoch 194487) v5 -3319> 2014-08-13 17:38:05.368980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] take_waiters -3318> 2014-08-13 17:38:05.368986 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.361213, event: all_read, request: pg_notify(24.9b(34) epoch 194487) v5 -3317> 2014-08-13 17:38:05.368991 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.368973, event: dispatched, request: pg_notify(24.9b(34) epoch 194487) v5 -3316> 2014-08-13 17:38:05.368989 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -3315> 2014-08-13 17:38:05.368995 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.368995, event: waiting_for_osdmap, request: pg_notify(24.9b(34) epoch 194487) v5 -3314> 2014-08-13 17:38:05.368998 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.291 -3313> 2014-08-13 17:38:05.368876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.17 -3312> 2014-08-13 17:38:05.369000 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5821a40 -3311> 2014-08-13 17:38:05.369003 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.369003, event: started, request: pg_notify(24.9b(34) epoch 194487) v5 -3310> 2014-08-13 17:38:05.368998 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194485 pi=192981-194484/25 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -3309> 2014-08-13 17:38:05.369009 7fe99ebe8700 10 log is not dirty -3308> 2014-08-13 17:38:05.369015 7fe9a63f4700 5 -- op tracker -- , seq: 625, time: 2014-08-13 17:38:05.369015, event: done, request: pg_notify(24.9b(34) epoch 194487) v5 -3307> 2014-08-13 17:38:05.369021 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3306> 2014-08-13 17:38:05.369022 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3305> 2014-08-13 17:38:05.369028 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.367 192.168.1.114:6840/55513 10 ==== pg_info(1 pgs e194487:24.9b) v4 ==== 724+0+0 (2053231628 0 0) 0x60124c0 con 0x4e48c60 -3304> 2014-08-13 17:38:05.369023 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 -3303> 2014-08-13 17:38:05.369035 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3302> 2014-08-13 17:38:05.369036 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3301> 2014-08-13 17:38:05.369037 7fe9a63f4700 20 osd.4 194487 _dispatch 0x60124c0 pg_info(1 pgs e194487:24.9b) v4 -3300> 2014-08-13 17:38:05.369040 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.361489, event: header_read, request: pg_info(1 pgs e194487:24.9b) v4 -3299> 2014-08-13 17:38:05.369044 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3298> 2014-08-13 17:38:05.369044 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.361490, event: throttled, request: pg_info(1 pgs e194487:24.9b) v4 -3297> 2014-08-13 17:38:05.369047 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.361532, event: all_read, request: pg_info(1 pgs e194487:24.9b) v4 -3296> 2014-08-13 17:38:05.369051 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.369035, event: dispatched, request: pg_info(1 pgs e194487:24.9b) v4 -3295> 2014-08-13 17:38:05.369054 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.369054, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.9b) v4 -3294> 2014-08-13 17:38:05.369048 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Peering advmap -3293> 2014-08-13 17:38:05.369058 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.367 -3292> 2014-08-13 17:38:05.369061 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x60124c0 -3291> 2014-08-13 17:38:05.369063 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.369063, event: started, request: pg_info(1 pgs e194487:24.9b) v4 -3290> 2014-08-13 17:38:05.369061 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] adjust_need_up_thru now 194486, need_up_thru now false -3289> 2014-08-13 17:38:05.369076 7fe9a63f4700 5 -- op tracker -- , seq: 626, time: 2014-08-13 17:38:05.369076, event: done, request: pg_info(1 pgs e194487:24.9b) v4 -3288> 2014-08-13 17:38:05.369072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Started advmap -3287> 2014-08-13 17:38:05.369083 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3286> 2014-08-13 17:38:05.369085 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3285> 2014-08-13 17:38:05.369083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] check_recovery_sources no source osds () went down -3284> 2014-08-13 17:38:05.369088 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.373 192.168.1.108:6840/7435 18 ==== pg_notify(22.3d9(42) epoch 194487) v5 ==== 3020+0+0 (420402540 0 0) 0x4264980 con 0x4e85c20 -3283> 2014-08-13 17:38:05.369096 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3282> 2014-08-13 17:38:05.369097 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3281> 2014-08-13 17:38:05.369099 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264980 pg_notify(22.3d9(42) epoch 194487) v5 -3280> 2014-08-13 17:38:05.369094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map -3279> 2014-08-13 17:38:05.369102 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.359785, event: header_read, request: pg_notify(22.3d9(42) epoch 194487) v5 -3278> 2014-08-13 17:38:05.369105 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.359786, event: throttled, request: pg_notify(22.3d9(42) epoch 194487) v5 -3277> 2014-08-13 17:38:05.369109 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.359868, event: all_read, request: pg_notify(22.3d9(42) epoch 194487) v5 -3276> 2014-08-13 17:38:05.369112 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.369096, event: dispatched, request: pg_notify(22.3d9(42) epoch 194487) v5 -3275> 2014-08-13 17:38:05.369106 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle ActMap primary -3274> 2014-08-13 17:38:05.369116 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.369116, event: waiting_for_osdmap, request: pg_notify(22.3d9(42) epoch 194487) v5 -3273> 2014-08-13 17:38:05.369120 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.373 -3272> 2014-08-13 17:38:05.369121 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264980 -3271> 2014-08-13 17:38:05.369124 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.369124, event: started, request: pg_notify(22.3d9(42) epoch 194487) v5 -3270> 2014-08-13 17:38:05.369118 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194487:29705 -3269> 2014-08-13 17:38:05.369001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.17 -3268> 2014-08-13 17:38:05.369129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] take_waiters -3267> 2014-08-13 17:38:05.369136 7fe9a63f4700 5 -- op tracker -- , seq: 627, time: 2014-08-13 17:38:05.369135, event: done, request: pg_notify(22.3d9(42) epoch 194487) v5 -3266> 2014-08-13 17:38:05.369142 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3265> 2014-08-13 17:38:05.369143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3264> 2014-08-13 17:38:05.369141 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 0.931106 5 0.009167 -3263> 2014-08-13 17:38:05.369150 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.342 192.168.1.110:6802/41438 13 ==== pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 ==== 5016+0+0 (2403502653 0 0) 0x58255c0 con 0x4ec8580 -3262> 2014-08-13 17:38:05.369158 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3261> 2014-08-13 17:38:05.369159 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3260> 2014-08-13 17:38:05.369161 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58255c0 pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3259> 2014-08-13 17:38:05.369154 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Leaving Peering -3258> 2014-08-13 17:38:05.369164 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.366712, event: header_read, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3257> 2014-08-13 17:38:05.369168 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.366713, event: throttled, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3256> 2014-08-13 17:38:05.369164 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 1.087561 0 0.000000 -3255> 2014-08-13 17:38:05.369172 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.366819, event: all_read, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3254> 2014-08-13 17:38:05.369176 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.369158, event: dispatched, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3253> 2014-08-13 17:38:05.369180 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.369180, event: waiting_for_osdmap, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3252> 2014-08-13 17:38:05.369184 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.342 -3251> 2014-08-13 17:38:05.369185 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58255c0 -3250> 2014-08-13 17:38:05.369187 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.369187, event: started, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3249> 2014-08-13 17:38:05.369197 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active -3248> 2014-08-13 17:38:05.369208 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194477 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: In Active, about to call activate -3247> 2014-08-13 17:38:05.369218 7fe9a63f4700 5 -- op tracker -- , seq: 628, time: 2014-08-13 17:38:05.369218, event: done, request: pg_notify(22.3e4(31),22.707(33) epoch 194487) v5 -3246> 2014-08-13 17:38:05.369226 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3245> 2014-08-13 17:38:05.369227 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3244> 2014-08-13 17:38:05.369222 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - purged_snaps [] cached_removed_snaps [] -3243> 2014-08-13 17:38:05.369233 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - snap_trimq [] -3242> 2014-08-13 17:38:05.369242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - not complete, missing(57) -3241> 2014-08-13 17:38:05.369249 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.375 192.168.1.114:6830/52879 9 ==== pg_info(1 pgs e194487:24.30b) v4 ==== 748+0+0 (2182383727 0 0) 0x5825240 con 0x4313f40 -3240> 2014-08-13 17:38:05.369258 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3239> 2014-08-13 17:38:05.369259 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3238> 2014-08-13 17:38:05.369260 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825240 pg_info(1 pgs e194487:24.30b) v4 -3237> 2014-08-13 17:38:05.369130 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.17 -3236> 2014-08-13 17:38:05.369263 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.365616, event: header_read, request: pg_info(1 pgs e194487:24.30b) v4 -3235> 2014-08-13 17:38:05.369267 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.365617, event: throttled, request: pg_info(1 pgs e194487:24.30b) v4 -3234> 2014-08-13 17:38:05.369270 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.365656, event: all_read, request: pg_info(1 pgs e194487:24.30b) v4 -3233> 2014-08-13 17:38:05.369273 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.369257, event: dispatched, request: pg_info(1 pgs e194487:24.30b) v4 -3232> 2014-08-13 17:38:05.369277 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.369277, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.30b) v4 -3231> 2014-08-13 17:38:05.369280 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.375 -3230> 2014-08-13 17:38:05.369282 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825240 -3229> 2014-08-13 17:38:05.369285 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.369285, event: started, request: pg_info(1 pgs e194487:24.30b) v4 -3228> 2014-08-13 17:38:05.369094 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -3227> 2014-08-13 17:38:05.369309 7fe9a63f4700 5 -- op tracker -- , seq: 629, time: 2014-08-13 17:38:05.369308, event: done, request: pg_info(1 pgs e194487:24.30b) v4 -3226> 2014-08-13 17:38:05.369324 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3225> 2014-08-13 17:38:05.369326 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3224> 2014-08-13 17:38:05.369329 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.340 192.168.1.104:6806/62463 12 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54db840 con 0x49e40a0 -3223> 2014-08-13 17:38:05.369337 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3222> 2014-08-13 17:38:05.369338 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3221> 2014-08-13 17:38:05.369339 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db840 osd_map(194487..194487 src has 185139..194487) v3 -3220> 2014-08-13 17:38:05.369343 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3219> 2014-08-13 17:38:05.369346 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3218> 2014-08-13 17:38:05.369347 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3217> 2014-08-13 17:38:05.369348 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3216> 2014-08-13 17:38:05.369350 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.30 192.168.1.107:6823/55491 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5499d40 con 0x4a44780 -3215> 2014-08-13 17:38:05.369357 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3214> 2014-08-13 17:38:05.369358 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3213> 2014-08-13 17:38:05.369359 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5499d40 osd_map(194487..194487 src has 185139..194487) v3 -3212> 2014-08-13 17:38:05.369362 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3211> 2014-08-13 17:38:05.369364 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3210> 2014-08-13 17:38:05.369365 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3209> 2014-08-13 17:38:05.369366 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3208> 2014-08-13 17:38:05.369369 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.340 192.168.1.104:6806/62463 13 ==== pg_notify(24.34(52) epoch 194487) v5 ==== 3416+0+0 (2801406691 0 0) 0x6012140 con 0x49e40a0 -3207> 2014-08-13 17:38:05.369376 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3206> 2014-08-13 17:38:05.369377 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3205> 2014-08-13 17:38:05.369378 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6012140 pg_notify(24.34(52) epoch 194487) v5 -3204> 2014-08-13 17:38:05.369381 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369051, event: header_read, request: pg_notify(24.34(52) epoch 194487) v5 -3203> 2014-08-13 17:38:05.369384 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369052, event: throttled, request: pg_notify(24.34(52) epoch 194487) v5 -3202> 2014-08-13 17:38:05.369388 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369135, event: all_read, request: pg_notify(24.34(52) epoch 194487) v5 -3201> 2014-08-13 17:38:05.369263 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.17 -3200> 2014-08-13 17:38:05.369391 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369376, event: dispatched, request: pg_notify(24.34(52) epoch 194487) v5 -3199> 2014-08-13 17:38:05.369395 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369395, event: waiting_for_osdmap, request: pg_notify(24.34(52) epoch 194487) v5 -3198> 2014-08-13 17:38:05.369398 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.340 -3197> 2014-08-13 17:38:05.369400 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6012140 -3196> 2014-08-13 17:38:05.369402 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369402, event: started, request: pg_notify(24.34(52) epoch 194487) v5 -3195> 2014-08-13 17:38:05.369419 7fe9a63f4700 5 -- op tracker -- , seq: 630, time: 2014-08-13 17:38:05.369418, event: done, request: pg_notify(24.34(52) epoch 194487) v5 -3194> 2014-08-13 17:38:05.369430 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3193> 2014-08-13 17:38:05.369432 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3192> 2014-08-13 17:38:05.369444 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.30 192.168.1.107:6823/55491 10 ==== pg_notify(24.34(52) epoch 194487) v5 ==== 3416+0+0 (484976335 0 0) 0x5102140 con 0x4a44780 -3191> 2014-08-13 17:38:05.369454 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3190> 2014-08-13 17:38:05.369455 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3189> 2014-08-13 17:38:05.369461 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102140 pg_notify(24.34(52) epoch 194487) v5 -3188> 2014-08-13 17:38:05.369464 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369208, event: header_read, request: pg_notify(24.34(52) epoch 194487) v5 -3187> 2014-08-13 17:38:05.369468 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369210, event: throttled, request: pg_notify(24.34(52) epoch 194487) v5 -3186> 2014-08-13 17:38:05.369472 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369294, event: all_read, request: pg_notify(24.34(52) epoch 194487) v5 -3185> 2014-08-13 17:38:05.369476 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369454, event: dispatched, request: pg_notify(24.34(52) epoch 194487) v5 -3184> 2014-08-13 17:38:05.369302 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] flushed -3183> 2014-08-13 17:38:05.369480 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369480, event: waiting_for_osdmap, request: pg_notify(24.34(52) epoch 194487) v5 -3182> 2014-08-13 17:38:05.369484 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.30 -3181> 2014-08-13 17:38:05.369487 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102140 -3180> 2014-08-13 17:38:05.369493 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369493, event: started, request: pg_notify(24.34(52) epoch 194487) v5 -3179> 2014-08-13 17:38:05.369505 7fe9a63f4700 5 -- op tracker -- , seq: 631, time: 2014-08-13 17:38:05.369505, event: done, request: pg_notify(24.34(52) epoch 194487) v5 -3178> 2014-08-13 17:38:05.369512 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3177> 2014-08-13 17:38:05.369514 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3176> 2014-08-13 17:38:05.369394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.17 -3175> 2014-08-13 17:38:05.369527 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.17 -3174> 2014-08-13 17:38:05.369660 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.17 -3173> 2014-08-13 17:38:05.369795 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.17 -3172> 2014-08-13 17:38:05.369933 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.17 -3171> 2014-08-13 17:38:05.370072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.17 -3170> 2014-08-13 17:38:05.370215 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.17 -3169> 2014-08-13 17:38:05.370358 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.17 -3168> 2014-08-13 17:38:05.370495 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - starting recovery -3167> 2014-08-13 17:38:05.370511 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] -3166> 2014-08-13 17:38:05.370542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -3165> 2014-08-13 17:38:05.370556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 is up to date, queueing in pending_activators -3164> 2014-08-13 17:38:05.370593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3163> 2014-08-13 17:38:05.370605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -3162> 2014-08-13 17:38:05.370622 7fe99ebe8700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 -3161> 2014-08-13 17:38:05.370502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.17 -3160> 2014-08-13 17:38:05.370648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.17 -3159> 2014-08-13 17:38:05.370914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 sending log((192803'16165,194422'18731], crt=194422'18731) -3158> 2014-08-13 17:38:05.370931 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing -3157> 2014-08-13 17:38:05.370795 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.17 -3156> 2014-08-13 17:38:05.370933 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.25 192.168.1.103:6800/36920 -- pg_log(24.3d epoch 194487 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194487) v4 -- ?+0 0x4871e00 -3155> 2014-08-13 17:38:05.370951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3154> 2014-08-13 17:38:05.370966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) -3153> 2014-08-13 17:38:05.370978 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 is up to date, queueing in pending_activators -3152> 2014-08-13 17:38:05.371018 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3151> 2014-08-13 17:38:05.371033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -3150> 2014-08-13 17:38:05.371049 7fe99ebe8700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 -3149> 2014-08-13 17:38:05.370944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.17 -3148> 2014-08-13 17:38:05.371097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.17 -3147> 2014-08-13 17:38:05.371255 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.17 -3146> 2014-08-13 17:38:05.371412 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 sending log((192803'16165,194422'18731], crt=194422'18731) -3145> 2014-08-13 17:38:05.371432 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing -3144> 2014-08-13 17:38:05.371434 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.362 192.168.1.108:6842/8170 -- pg_log(24.3d epoch 194487 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194487) v4 -- ?+0 0x4870000 -3143> 2014-08-13 17:38:05.371449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -3142> 2014-08-13 17:38:05.371412 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.17 -3141> 2014-08-13 17:38:05.371501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery primary has 57 missing -3140> 2014-08-13 17:38:05.371573 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.4 -3139> 2014-08-13 17:38:05.371644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.4 -3138> 2014-08-13 17:38:05.371571 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.17 -3137> 2014-08-13 17:38:05.371713 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.4 -3136> 2014-08-13 17:38:05.371783 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.4 -3135> 2014-08-13 17:38:05.371731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.17 -3134> 2014-08-13 17:38:05.371895 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.340 192.168.1.104:6806/62463 14 ==== pg_info(1 pgs e194487:24.34) v4 ==== 740+0+0 (2196385509 0 0) 0x51ca4c0 con 0x49e40a0 -3133> 2014-08-13 17:38:05.371909 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3132> 2014-08-13 17:38:05.371912 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3131> 2014-08-13 17:38:05.371913 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ca4c0 pg_info(1 pgs e194487:24.34) v4 -3130> 2014-08-13 17:38:05.371852 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.4 -3129> 2014-08-13 17:38:05.371917 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371817, event: header_read, request: pg_info(1 pgs e194487:24.34) v4 -3128> 2014-08-13 17:38:05.371923 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371819, event: throttled, request: pg_info(1 pgs e194487:24.34) v4 -3127> 2014-08-13 17:38:05.371926 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371873, event: all_read, request: pg_info(1 pgs e194487:24.34) v4 -3126> 2014-08-13 17:38:05.371930 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371909, event: dispatched, request: pg_info(1 pgs e194487:24.34) v4 -3125> 2014-08-13 17:38:05.371934 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371934, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.34) v4 -3124> 2014-08-13 17:38:05.371938 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.340 -3123> 2014-08-13 17:38:05.371941 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ca4c0 -3122> 2014-08-13 17:38:05.371944 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371944, event: started, request: pg_info(1 pgs e194487:24.34) v4 -3121> 2014-08-13 17:38:05.371959 7fe9a63f4700 5 -- op tracker -- , seq: 632, time: 2014-08-13 17:38:05.371959, event: done, request: pg_info(1 pgs e194487:24.34) v4 -3120> 2014-08-13 17:38:05.371920 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.4 -3119> 2014-08-13 17:38:05.371997 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3118> 2014-08-13 17:38:05.371999 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3117> 2014-08-13 17:38:05.371988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.4 -3116> 2014-08-13 17:38:05.371893 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.17 -3115> 2014-08-13 17:38:05.372059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.4 -3114> 2014-08-13 17:38:05.372129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.4 -3113> 2014-08-13 17:38:05.372061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.17 -3112> 2014-08-13 17:38:05.372202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.4 -3111> 2014-08-13 17:38:05.372272 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.4 -3110> 2014-08-13 17:38:05.372341 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.4 -3109> 2014-08-13 17:38:05.372250 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.17 -3108> 2014-08-13 17:38:05.372410 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.4 -3107> 2014-08-13 17:38:05.372479 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.4 -3106> 2014-08-13 17:38:05.372418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.17 -3105> 2014-08-13 17:38:05.372548 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.4 -3104> 2014-08-13 17:38:05.372617 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.4 -3103> 2014-08-13 17:38:05.372686 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.4 -3102> 2014-08-13 17:38:05.372587 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.17 -3101> 2014-08-13 17:38:05.372756 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.4 -3100> 2014-08-13 17:38:05.372826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.4 -3099> 2014-08-13 17:38:05.372756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.17 -3098> 2014-08-13 17:38:05.372895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.4 -3097> 2014-08-13 17:38:05.372964 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.4 -3096> 2014-08-13 17:38:05.372929 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.17 -3095> 2014-08-13 17:38:05.373033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.4 -3094> 2014-08-13 17:38:05.373118 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.4 -3093> 2014-08-13 17:38:05.373190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.4 -3092> 2014-08-13 17:38:05.373260 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.4 -3091> 2014-08-13 17:38:05.373104 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -3090> 2014-08-13 17:38:05.373329 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.4 -3089> 2014-08-13 17:38:05.373396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.4 -3088> 2014-08-13 17:38:05.373465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.4 -3087> 2014-08-13 17:38:05.373391 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 28 as a source -3086> 2014-08-13 17:38:05.373535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.4 -3085> 2014-08-13 17:38:05.373604 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.4 -3084> 2014-08-13 17:38:05.373673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.4 -3083> 2014-08-13 17:38:05.373569 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.28 (past last_backfill 0//0//-1) -3082> 2014-08-13 17:38:05.373742 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.4 -3081> 2014-08-13 17:38:05.373811 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.4 -3080> 2014-08-13 17:38:05.373767 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.28 (past last_backfill 0//0//-1) -3079> 2014-08-13 17:38:05.373879 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.4 -3078> 2014-08-13 17:38:05.373947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.4 -3077> 2014-08-13 17:38:05.374016 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.4 -3076> 2014-08-13 17:38:05.373945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.28 (past last_backfill 0//0//-1) -3075> 2014-08-13 17:38:05.374084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.4 -3074> 2014-08-13 17:38:05.374153 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.4 -3073> 2014-08-13 17:38:05.374239 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.4 -3072> 2014-08-13 17:38:05.374121 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.28 (past last_backfill 0//0//-1) -3071> 2014-08-13 17:38:05.374309 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.4 -3070> 2014-08-13 17:38:05.374378 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.4 -3069> 2014-08-13 17:38:05.374316 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.28 (past last_backfill 0//0//-1) -3068> 2014-08-13 17:38:05.374447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.4 -3067> 2014-08-13 17:38:05.374516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.4 -3066> 2014-08-13 17:38:05.374585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.4 -3065> 2014-08-13 17:38:05.374494 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.28 (past last_backfill 0//0//-1) -3064> 2014-08-13 17:38:05.374659 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.352 192.168.1.108:6826/4273 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dfbc0 con 0x5385960 -3063> 2014-08-13 17:38:05.374674 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3062> 2014-08-13 17:38:05.374677 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3061> 2014-08-13 17:38:05.374678 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dfbc0 osd_map(194487..194487 src has 185139..194487) v3 -3060> 2014-08-13 17:38:05.374683 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3059> 2014-08-13 17:38:05.374686 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3058> 2014-08-13 17:38:05.374687 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3057> 2014-08-13 17:38:05.374689 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3056> 2014-08-13 17:38:05.374655 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.4 -3055> 2014-08-13 17:38:05.374726 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.352 192.168.1.108:6826/4273 7 ==== pg_notify(22.4df(1) epoch 194487) v5 ==== 777+0+0 (4005984518 0 0) 0x51ce740 con 0x5385960 -3054> 2014-08-13 17:38:05.374744 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3053> 2014-08-13 17:38:05.374747 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3052> 2014-08-13 17:38:05.374748 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_notify(22.4df(1) epoch 194487) v5 -3051> 2014-08-13 17:38:05.374753 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374662, event: header_read, request: pg_notify(22.4df(1) epoch 194487) v5 -3050> 2014-08-13 17:38:05.374758 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374663, event: throttled, request: pg_notify(22.4df(1) epoch 194487) v5 -3049> 2014-08-13 17:38:05.374764 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374710, event: all_read, request: pg_notify(22.4df(1) epoch 194487) v5 -3048> 2014-08-13 17:38:05.374768 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374743, event: dispatched, request: pg_notify(22.4df(1) epoch 194487) v5 -3047> 2014-08-13 17:38:05.374772 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374772, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194487) v5 -3046> 2014-08-13 17:38:05.374777 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.352 -3045> 2014-08-13 17:38:05.374779 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 -3044> 2014-08-13 17:38:05.374782 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374782, event: started, request: pg_notify(22.4df(1) epoch 194487) v5 -3043> 2014-08-13 17:38:05.374723 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.4 -3042> 2014-08-13 17:38:05.374798 7fe9a63f4700 5 -- op tracker -- , seq: 633, time: 2014-08-13 17:38:05.374798, event: done, request: pg_notify(22.4df(1) epoch 194487) v5 -3041> 2014-08-13 17:38:05.374807 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3040> 2014-08-13 17:38:05.374809 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3039> 2014-08-13 17:38:05.374668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.28 (past last_backfill 0//0//-1) -3038> 2014-08-13 17:38:05.374790 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.4 -3037> 2014-08-13 17:38:05.374863 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.4 -3036> 2014-08-13 17:38:05.374930 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.4 -3035> 2014-08-13 17:38:05.374842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.28 (past last_backfill 0//0//-1) -3034> 2014-08-13 17:38:05.375000 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.4 -3033> 2014-08-13 17:38:05.375069 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.4 -3032> 2014-08-13 17:38:05.375016 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.28 (past last_backfill 0//0//-1) -3031> 2014-08-13 17:38:05.375137 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.4 -3030> 2014-08-13 17:38:05.375215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.4 -3029> 2014-08-13 17:38:05.375285 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.4 -3028> 2014-08-13 17:38:05.375206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.28 (past last_backfill 0//0//-1) -3027> 2014-08-13 17:38:05.375354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.4 -3026> 2014-08-13 17:38:05.375423 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.4 -3025> 2014-08-13 17:38:05.375383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.28 (past last_backfill 0//0//-1) -3024> 2014-08-13 17:38:05.375492 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.4 -3023> 2014-08-13 17:38:05.375558 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -3022> 2014-08-13 17:38:05.375558 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.28 (past last_backfill 0//0//-1) -3021> 2014-08-13 17:38:05.375678 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] activate: adding 20 as a source -3020> 2014-08-13 17:38:05.375745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.20 -3019> 2014-08-13 17:38:05.375815 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=56] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.20 -3018> 2014-08-13 17:38:05.375735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.28 (past last_backfill 0//0//-1) -3017> 2014-08-13 17:38:05.375888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=55] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.20 -3016> 2014-08-13 17:38:05.375965 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.306 192.168.1.101:6806/6812 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dee40 con 0x53806e0 -3015> 2014-08-13 17:38:05.375981 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3014> 2014-08-13 17:38:05.375987 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3013> 2014-08-13 17:38:05.375988 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dee40 osd_map(194487..194487 src has 185139..194487) v3 -3012> 2014-08-13 17:38:05.375994 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -3011> 2014-08-13 17:38:05.375997 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -3010> 2014-08-13 17:38:05.375998 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3009> 2014-08-13 17:38:05.375999 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3008> 2014-08-13 17:38:05.375963 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=54] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.20 -3007> 2014-08-13 17:38:05.375910 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.28 (past last_backfill 0//0//-1) -3006> 2014-08-13 17:38:05.376037 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=53] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.20 -3005> 2014-08-13 17:38:05.376122 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.306 192.168.1.101:6806/6812 7 ==== pg_notify(22.481(44) epoch 194487) v5 ==== 3194+0+0 (1204498158 0 0) 0x4264d00 con 0x53806e0 -3004> 2014-08-13 17:38:05.376134 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -3003> 2014-08-13 17:38:05.376136 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -3002> 2014-08-13 17:38:05.376138 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4264d00 pg_notify(22.481(44) epoch 194487) v5 -3001> 2014-08-13 17:38:05.376141 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376030, event: header_read, request: pg_notify(22.481(44) epoch 194487) v5 -3000> 2014-08-13 17:38:05.376146 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376031, event: throttled, request: pg_notify(22.481(44) epoch 194487) v5 -2999> 2014-08-13 17:38:05.376150 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376110, event: all_read, request: pg_notify(22.481(44) epoch 194487) v5 -2998> 2014-08-13 17:38:05.376164 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376133, event: dispatched, request: pg_notify(22.481(44) epoch 194487) v5 -2997> 2014-08-13 17:38:05.376169 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376169, event: waiting_for_osdmap, request: pg_notify(22.481(44) epoch 194487) v5 -2996> 2014-08-13 17:38:05.376173 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.306 -2995> 2014-08-13 17:38:05.376175 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4264d00 -2994> 2014-08-13 17:38:05.376178 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376178, event: started, request: pg_notify(22.481(44) epoch 194487) v5 -2993> 2014-08-13 17:38:05.376192 7fe9a63f4700 5 -- op tracker -- , seq: 634, time: 2014-08-13 17:38:05.376192, event: done, request: pg_notify(22.481(44) epoch 194487) v5 -2992> 2014-08-13 17:38:05.376199 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2991> 2014-08-13 17:38:05.376200 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2990> 2014-08-13 17:38:05.376115 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=52] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.20 -2989> 2014-08-13 17:38:05.376085 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.28 (past last_backfill 0//0//-1) -2988> 2014-08-13 17:38:05.376206 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=51] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.20 -2987> 2014-08-13 17:38:05.376287 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=50] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.20 -2986> 2014-08-13 17:38:05.376273 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.28 (past last_backfill 0//0//-1) -2985> 2014-08-13 17:38:05.376370 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=49] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.20 -2984> 2014-08-13 17:38:05.376453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=48] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.20 -2983> 2014-08-13 17:38:05.376449 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.28 (past last_backfill 0//0//-1) -2982> 2014-08-13 17:38:05.376539 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=47] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.20 -2981> 2014-08-13 17:38:05.376645 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=46] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.20 -2980> 2014-08-13 17:38:05.376625 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.28 (past last_backfill 0//0//-1) -2979> 2014-08-13 17:38:05.376736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=45] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.20 -2978> 2014-08-13 17:38:05.376828 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=44] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.20 -2977> 2014-08-13 17:38:05.376800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.28 (past last_backfill 0//0//-1) -2976> 2014-08-13 17:38:05.376921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=43] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.20 -2975> 2014-08-13 17:38:05.377017 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=42] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.20 -2974> 2014-08-13 17:38:05.376974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.28 (past last_backfill 0//0//-1) -2973> 2014-08-13 17:38:05.377114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=41] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.20 -2972> 2014-08-13 17:38:05.377219 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=40] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.20 -2971> 2014-08-13 17:38:05.377150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.28 (past last_backfill 0//0//-1) -2970> 2014-08-13 17:38:05.377321 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=39] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.20 -2969> 2014-08-13 17:38:05.377339 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.28 (past last_backfill 0//0//-1) -2968> 2014-08-13 17:38:05.377423 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=38] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.20 -2967> 2014-08-13 17:38:05.377526 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=37] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.20 -2966> 2014-08-13 17:38:05.377515 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.28 (past last_backfill 0//0//-1) -2965> 2014-08-13 17:38:05.377631 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=36] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.20 -2964> 2014-08-13 17:38:05.377740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=35] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.20 -2963> 2014-08-13 17:38:05.377691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.28 (past last_backfill 0//0//-1) -2962> 2014-08-13 17:38:05.377849 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=34] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.20 -2961> 2014-08-13 17:38:05.377865 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.28 (past last_backfill 0//0//-1) -2960> 2014-08-13 17:38:05.377960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=33] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.20 -2959> 2014-08-13 17:38:05.378072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=32] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.20 -2958> 2014-08-13 17:38:05.378040 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.28 (past last_backfill 0//0//-1) -2957> 2014-08-13 17:38:05.378190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=31] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.20 -2956> 2014-08-13 17:38:05.378218 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.28 (past last_backfill 0//0//-1) -2955> 2014-08-13 17:38:05.378307 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=30] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.20 -2954> 2014-08-13 17:38:05.378425 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=29] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.20 -2953> 2014-08-13 17:38:05.378394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.28 (past last_backfill 0//0//-1) -2952> 2014-08-13 17:38:05.378544 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=28] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.20 -2951> 2014-08-13 17:38:05.378569 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.28 (past last_backfill 0//0//-1) -2950> 2014-08-13 17:38:05.378664 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=27] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.20 -2949> 2014-08-13 17:38:05.378787 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=26] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.20 -2948> 2014-08-13 17:38:05.378745 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.28 (past last_backfill 0//0//-1) -2947> 2014-08-13 17:38:05.378912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=25] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.20 -2946> 2014-08-13 17:38:05.378918 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.28 (past last_backfill 0//0//-1) -2945> 2014-08-13 17:38:05.379038 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=24] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.20 -2944> 2014-08-13 17:38:05.379099 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.28 (past last_backfill 0//0//-1) -2943> 2014-08-13 17:38:05.379165 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=23] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.20 -2942> 2014-08-13 17:38:05.379297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=22] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.20 -2941> 2014-08-13 17:38:05.379278 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.28 (past last_backfill 0//0//-1) -2940> 2014-08-13 17:38:05.379429 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=21] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.20 -2939> 2014-08-13 17:38:05.379453 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.28 (past last_backfill 0//0//-1) -2938> 2014-08-13 17:38:05.379562 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=20] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.20 -2937> 2014-08-13 17:38:05.379628 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.28 (past last_backfill 0//0//-1) -2936> 2014-08-13 17:38:05.379696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=19] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.20 -2935> 2014-08-13 17:38:05.379834 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=18] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.20 -2934> 2014-08-13 17:38:05.379803 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.28 (past last_backfill 0//0//-1) -2933> 2014-08-13 17:38:05.379973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=17] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.20 -2932> 2014-08-13 17:38:05.379977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.28 (past last_backfill 0//0//-1) -2931> 2014-08-13 17:38:05.380113 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=16] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.20 -2930> 2014-08-13 17:38:05.380152 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.28 (past last_backfill 0//0//-1) -2929> 2014-08-13 17:38:05.380257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=15] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.20 -2928> 2014-08-13 17:38:05.380330 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.28 (past last_backfill 0//0//-1) -2927> 2014-08-13 17:38:05.380401 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=14] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.20 -2926> 2014-08-13 17:38:05.380505 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.28 (past last_backfill 0//0//-1) -2925> 2014-08-13 17:38:05.380546 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=13] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.20 -2924> 2014-08-13 17:38:05.380692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=12] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.20 -2923> 2014-08-13 17:38:05.380680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.28 (past last_backfill 0//0//-1) -2922> 2014-08-13 17:38:05.380840 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=11] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.20 -2921> 2014-08-13 17:38:05.380883 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.28 (past last_backfill 0//0//-1) -2920> 2014-08-13 17:38:05.380991 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=10] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.20 -2919> 2014-08-13 17:38:05.381060 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.28 (past last_backfill 0//0//-1) -2918> 2014-08-13 17:38:05.381143 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=9] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.20 -2917> 2014-08-13 17:38:05.381239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.28 (past last_backfill 0//0//-1) -2916> 2014-08-13 17:38:05.381300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=8] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.20 -2915> 2014-08-13 17:38:05.381414 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.28 (past last_backfill 0//0//-1) -2914> 2014-08-13 17:38:05.381456 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=7] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.20 -2913> 2014-08-13 17:38:05.381589 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.28 (past last_backfill 0//0//-1) -2912> 2014-08-13 17:38:05.381614 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=6] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.20 -2911> 2014-08-13 17:38:05.381772 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=5] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.20 -2910> 2014-08-13 17:38:05.381764 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.28 (past last_backfill 0//0//-1) -2909> 2014-08-13 17:38:05.381933 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=4] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.20 -2908> 2014-08-13 17:38:05.381938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.28 (past last_backfill 0//0//-1) -2907> 2014-08-13 17:38:05.382097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=3] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.20 -2906> 2014-08-13 17:38:05.382112 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.28 (past last_backfill 0//0//-1) -2905> 2014-08-13 17:38:05.382266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=2] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.20 -2904> 2014-08-13 17:38:05.382548 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.10 192.168.1.115:6815/60573 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db4800 con 0x49e09a0 -2903> 2014-08-13 17:38:05.382567 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2902> 2014-08-13 17:38:05.382569 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2901> 2014-08-13 17:38:05.382290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.28 (past last_backfill 0//0//-1) -2900> 2014-08-13 17:38:05.382571 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db4800 osd_map(194487..194487 src has 185139..194487) v3 -2899> 2014-08-13 17:38:05.382580 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2898> 2014-08-13 17:38:05.382583 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2897> 2014-08-13 17:38:05.382584 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2896> 2014-08-13 17:38:05.382585 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2895> 2014-08-13 17:38:05.382589 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.10 192.168.1.115:6815/60573 10 ==== pg_notify(24.30b(35) epoch 194487) v5 ==== 2721+0+0 (3679074660 0 0) 0x513d400 con 0x49e09a0 -2894> 2014-08-13 17:38:05.382597 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2893> 2014-08-13 17:38:05.382599 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2892> 2014-08-13 17:38:05.382600 7fe9a63f4700 20 osd.4 194487 _dispatch 0x513d400 pg_notify(24.30b(35) epoch 194487) v5 -2891> 2014-08-13 17:38:05.382603 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382448, event: header_read, request: pg_notify(24.30b(35) epoch 194487) v5 -2890> 2014-08-13 17:38:05.382610 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382449, event: throttled, request: pg_notify(24.30b(35) epoch 194487) v5 -2889> 2014-08-13 17:38:05.382613 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382515, event: all_read, request: pg_notify(24.30b(35) epoch 194487) v5 -2888> 2014-08-13 17:38:05.382617 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382597, event: dispatched, request: pg_notify(24.30b(35) epoch 194487) v5 -2887> 2014-08-13 17:38:05.382621 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382621, event: waiting_for_osdmap, request: pg_notify(24.30b(35) epoch 194487) v5 -2886> 2014-08-13 17:38:05.382624 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.10 -2885> 2014-08-13 17:38:05.382626 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x513d400 -2884> 2014-08-13 17:38:05.382630 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382630, event: started, request: pg_notify(24.30b(35) epoch 194487) v5 -2883> 2014-08-13 17:38:05.382458 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=1] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.20 -2882> 2014-08-13 17:38:05.382646 7fe9a63f4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 17:38:05.382646, event: done, request: pg_notify(24.30b(35) epoch 194487) v5 -2881> 2014-08-13 17:38:05.382654 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2880> 2014-08-13 17:38:05.382656 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2879> 2014-08-13 17:38:05.382599 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.28 (past last_backfill 0//0//-1) -2878> 2014-08-13 17:38:05.382648 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -2877> 2014-08-13 17:38:05.382781 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.28 (past last_backfill 0//0//-1) -2876> 2014-08-13 17:38:05.382862 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 25 as a source -2875> 2014-08-13 17:38:05.382956 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.28 (past last_backfill 0//0//-1) -2874> 2014-08-13 17:38:05.383030 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.25 (past last_backfill 0//0//-1) -2873> 2014-08-13 17:38:05.383129 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.28 (past last_backfill 0//0//-1) -2872> 2014-08-13 17:38:05.383202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.25 (past last_backfill 0//0//-1) -2871> 2014-08-13 17:38:05.383307 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.28 (past last_backfill 0//0//-1) -2870> 2014-08-13 17:38:05.383371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.25 (past last_backfill 0//0//-1) -2869> 2014-08-13 17:38:05.383481 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.28 (past last_backfill 0//0//-1) -2868> 2014-08-13 17:38:05.383538 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.25 (past last_backfill 0//0//-1) -2867> 2014-08-13 17:38:05.383656 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.28 (past last_backfill 0//0//-1) -2866> 2014-08-13 17:38:05.383708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.25 (past last_backfill 0//0//-1) -2865> 2014-08-13 17:38:05.383832 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.28 (past last_backfill 0//0//-1) -2864> 2014-08-13 17:38:05.383878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.25 (past last_backfill 0//0//-1) -2863> 2014-08-13 17:38:05.384007 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.28 (past last_backfill 0//0//-1) -2862> 2014-08-13 17:38:05.384047 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.25 (past last_backfill 0//0//-1) -2861> 2014-08-13 17:38:05.384219 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.25 (past last_backfill 0//0//-1) -2860> 2014-08-13 17:38:05.384185 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -2859> 2014-08-13 17:38:05.384389 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.25 (past last_backfill 0//0//-1) -2858> 2014-08-13 17:38:05.384407 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 315 as a source -2857> 2014-08-13 17:38:05.384558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.25 (past last_backfill 0//0//-1) -2856> 2014-08-13 17:38:05.384581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.315 -2855> 2014-08-13 17:38:05.384727 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.25 (past last_backfill 0//0//-1) -2854> 2014-08-13 17:38:05.384757 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.315 -2853> 2014-08-13 17:38:05.384895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.25 (past last_backfill 0//0//-1) -2852> 2014-08-13 17:38:05.384934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.315 -2851> 2014-08-13 17:38:05.385064 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.25 (past last_backfill 0//0//-1) -2850> 2014-08-13 17:38:05.385110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.315 -2849> 2014-08-13 17:38:05.385236 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.25 (past last_backfill 0//0//-1) -2848> 2014-08-13 17:38:05.385290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.315 -2847> 2014-08-13 17:38:05.385406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.25 (past last_backfill 0//0//-1) -2846> 2014-08-13 17:38:05.385466 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.315 -2845> 2014-08-13 17:38:05.385575 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.25 (past last_backfill 0//0//-1) -2844> 2014-08-13 17:38:05.385642 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.315 -2843> 2014-08-13 17:38:05.385744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.25 (past last_backfill 0//0//-1) -2842> 2014-08-13 17:38:05.385817 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.315 -2841> 2014-08-13 17:38:05.385914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.25 (past last_backfill 0//0//-1) -2840> 2014-08-13 17:38:05.385993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.315 -2839> 2014-08-13 17:38:05.386083 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.25 (past last_backfill 0//0//-1) -2838> 2014-08-13 17:38:05.386169 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.315 -2837> 2014-08-13 17:38:05.386255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.25 (past last_backfill 0//0//-1) -2836> 2014-08-13 17:38:05.386462 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.362 192.168.1.108:6842/8170 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db3600 con 0x4d7d280 -2835> 2014-08-13 17:38:05.386485 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2834> 2014-08-13 17:38:05.386490 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2833> 2014-08-13 17:38:05.386493 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db3600 osd_map(194487..194487 src has 185139..194487) v3 -2832> 2014-08-13 17:38:05.386499 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2831> 2014-08-13 17:38:05.386502 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2830> 2014-08-13 17:38:05.386503 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2829> 2014-08-13 17:38:05.386505 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2828> 2014-08-13 17:38:05.386510 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.362 192.168.1.108:6842/8170 11 ==== pg_notify(22.362(31) epoch 194487) v5 ==== 2455+0+0 (3265461919 0 0) 0x5826c80 con 0x4d7d280 -2827> 2014-08-13 17:38:05.386518 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2826> 2014-08-13 17:38:05.386519 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2825> 2014-08-13 17:38:05.386348 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.315 -2824> 2014-08-13 17:38:05.386520 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5826c80 pg_notify(22.362(31) epoch 194487) v5 -2823> 2014-08-13 17:38:05.386524 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386421, event: header_read, request: pg_notify(22.362(31) epoch 194487) v5 -2822> 2014-08-13 17:38:05.386530 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386422, event: throttled, request: pg_notify(22.362(31) epoch 194487) v5 -2821> 2014-08-13 17:38:05.386534 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386498, event: all_read, request: pg_notify(22.362(31) epoch 194487) v5 -2820> 2014-08-13 17:38:05.386538 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386518, event: dispatched, request: pg_notify(22.362(31) epoch 194487) v5 -2819> 2014-08-13 17:38:05.386542 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386542, event: waiting_for_osdmap, request: pg_notify(22.362(31) epoch 194487) v5 -2818> 2014-08-13 17:38:05.386545 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.362 -2817> 2014-08-13 17:38:05.386548 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5826c80 -2816> 2014-08-13 17:38:05.386551 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386551, event: started, request: pg_notify(22.362(31) epoch 194487) v5 -2815> 2014-08-13 17:38:05.386566 7fe9a63f4700 5 -- op tracker -- , seq: 636, time: 2014-08-13 17:38:05.386566, event: done, request: pg_notify(22.362(31) epoch 194487) v5 -2814> 2014-08-13 17:38:05.386575 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2813> 2014-08-13 17:38:05.386576 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2812> 2014-08-13 17:38:05.386425 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.25 (past last_backfill 0//0//-1) -2811> 2014-08-13 17:38:05.386523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.315 -2810> 2014-08-13 17:38:05.386593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.25 (past last_backfill 0//0//-1) -2809> 2014-08-13 17:38:05.386699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.315 -2808> 2014-08-13 17:38:05.386762 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.25 (past last_backfill 0//0//-1) -2807> 2014-08-13 17:38:05.386875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.315 -2806> 2014-08-13 17:38:05.386931 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.25 (past last_backfill 0//0//-1) -2805> 2014-08-13 17:38:05.387050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.315 -2804> 2014-08-13 17:38:05.387100 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.25 (past last_backfill 0//0//-1) -2803> 2014-08-13 17:38:05.387230 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.315 -2802> 2014-08-13 17:38:05.387272 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.25 (past last_backfill 0//0//-1) -2801> 2014-08-13 17:38:05.387407 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.315 -2800> 2014-08-13 17:38:05.387441 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.25 (past last_backfill 0//0//-1) -2799> 2014-08-13 17:38:05.387583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.315 -2798> 2014-08-13 17:38:05.387610 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.25 (past last_backfill 0//0//-1) -2797> 2014-08-13 17:38:05.387758 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.315 -2796> 2014-08-13 17:38:05.387779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.25 (past last_backfill 0//0//-1) -2795> 2014-08-13 17:38:05.387934 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.315 -2794> 2014-08-13 17:38:05.387947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.25 (past last_backfill 0//0//-1) -2793> 2014-08-13 17:38:05.388114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.25 (past last_backfill 0//0//-1) -2792> 2014-08-13 17:38:05.388110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.315 -2791> 2014-08-13 17:38:05.388288 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.25 (past last_backfill 0//0//-1) -2790> 2014-08-13 17:38:05.388290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.315 -2789> 2014-08-13 17:38:05.388458 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.25 (past last_backfill 0//0//-1) -2788> 2014-08-13 17:38:05.388466 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.315 -2787> 2014-08-13 17:38:05.388627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.25 (past last_backfill 0//0//-1) -2786> 2014-08-13 17:38:05.388640 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.315 -2785> 2014-08-13 17:38:05.388797 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.25 (past last_backfill 0//0//-1) -2784> 2014-08-13 17:38:05.388814 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.315 -2783> 2014-08-13 17:38:05.388966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.25 (past last_backfill 0//0//-1) -2782> 2014-08-13 17:38:05.388988 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.315 -2781> 2014-08-13 17:38:05.389134 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.25 (past last_backfill 0//0//-1) -2780> 2014-08-13 17:38:05.389163 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.315 -2779> 2014-08-13 17:38:05.389306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.25 (past last_backfill 0//0//-1) -2778> 2014-08-13 17:38:05.389343 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.315 -2777> 2014-08-13 17:38:05.389475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.25 (past last_backfill 0//0//-1) -2776> 2014-08-13 17:38:05.389521 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.315 -2775> 2014-08-13 17:38:05.389644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.25 (past last_backfill 0//0//-1) -2774> 2014-08-13 17:38:05.389697 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.315 -2773> 2014-08-13 17:38:05.389813 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.25 (past last_backfill 0//0//-1) -2772> 2014-08-13 17:38:05.389873 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.315 -2771> 2014-08-13 17:38:05.389982 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.25 (past last_backfill 0//0//-1) -2770> 2014-08-13 17:38:05.390049 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.315 -2769> 2014-08-13 17:38:05.390151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.25 (past last_backfill 0//0//-1) -2768> 2014-08-13 17:38:05.390228 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.315 -2767> 2014-08-13 17:38:05.390323 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.25 (past last_backfill 0//0//-1) -2766> 2014-08-13 17:38:05.390404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.315 -2765> 2014-08-13 17:38:05.390493 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.25 (past last_backfill 0//0//-1) -2764> 2014-08-13 17:38:05.390581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.315 -2763> 2014-08-13 17:38:05.390662 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.25 (past last_backfill 0//0//-1) -2762> 2014-08-13 17:38:05.390757 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.315 -2761> 2014-08-13 17:38:05.390928 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.58 192.168.1.111:6834/14316 12 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d8ec00 con 0x49e1080 -2760> 2014-08-13 17:38:05.390948 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2759> 2014-08-13 17:38:05.390954 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2758> 2014-08-13 17:38:05.390956 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d8ec00 osd_map(194487..194487 src has 185139..194487) v3 -2757> 2014-08-13 17:38:05.390963 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2756> 2014-08-13 17:38:05.390966 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2755> 2014-08-13 17:38:05.390967 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2754> 2014-08-13 17:38:05.390968 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2753> 2014-08-13 17:38:05.390833 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.25 (past last_backfill 0//0//-1) -2752> 2014-08-13 17:38:05.391035 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.58 192.168.1.111:6834/14316 13 ==== pg_notify(22.1d2(27) epoch 194487) v5 ==== 2231+0+0 (3320793000 0 0) 0x5102680 con 0x49e1080 -2751> 2014-08-13 17:38:05.391047 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2750> 2014-08-13 17:38:05.391051 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2749> 2014-08-13 17:38:05.391053 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102680 pg_notify(22.1d2(27) epoch 194487) v5 -2748> 2014-08-13 17:38:05.391057 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.390945, event: header_read, request: pg_notify(22.1d2(27) epoch 194487) v5 -2747> 2014-08-13 17:38:05.391063 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.390946, event: throttled, request: pg_notify(22.1d2(27) epoch 194487) v5 -2746> 2014-08-13 17:38:05.391067 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.391019, event: all_read, request: pg_notify(22.1d2(27) epoch 194487) v5 -2745> 2014-08-13 17:38:05.391070 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.391046, event: dispatched, request: pg_notify(22.1d2(27) epoch 194487) v5 -2744> 2014-08-13 17:38:05.391074 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.391074, event: waiting_for_osdmap, request: pg_notify(22.1d2(27) epoch 194487) v5 -2743> 2014-08-13 17:38:05.391081 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.58 -2742> 2014-08-13 17:38:05.391083 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102680 -2741> 2014-08-13 17:38:05.391087 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.391086, event: started, request: pg_notify(22.1d2(27) epoch 194487) v5 -2740> 2014-08-13 17:38:05.390932 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.315 -2739> 2014-08-13 17:38:05.391104 7fe9a63f4700 5 -- op tracker -- , seq: 637, time: 2014-08-13 17:38:05.391103, event: done, request: pg_notify(22.1d2(27) epoch 194487) v5 -2738> 2014-08-13 17:38:05.391113 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2737> 2014-08-13 17:38:05.391115 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2736> 2014-08-13 17:38:05.391001 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.25 (past last_backfill 0//0//-1) -2735> 2014-08-13 17:38:05.391107 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.315 -2734> 2014-08-13 17:38:05.391170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.25 (past last_backfill 0//0//-1) -2733> 2014-08-13 17:38:05.391286 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.315 -2732> 2014-08-13 17:38:05.391342 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.25 (past last_backfill 0//0//-1) -2731> 2014-08-13 17:38:05.391462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.315 -2730> 2014-08-13 17:38:05.391510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.25 (past last_backfill 0//0//-1) -2729> 2014-08-13 17:38:05.391637 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.315 -2728> 2014-08-13 17:38:05.391678 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.25 (past last_backfill 0//0//-1) -2727> 2014-08-13 17:38:05.391811 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.315 -2726> 2014-08-13 17:38:05.391845 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.25 (past last_backfill 0//0//-1) -2725> 2014-08-13 17:38:05.392010 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.54 192.168.1.111:6836/14923 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d90480 con 0x4a4f380 -2724> 2014-08-13 17:38:05.392026 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2723> 2014-08-13 17:38:05.392029 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2722> 2014-08-13 17:38:05.392030 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d90480 osd_map(194487..194487 src has 185139..194487) v3 -2721> 2014-08-13 17:38:05.392035 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2720> 2014-08-13 17:38:05.392038 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2719> 2014-08-13 17:38:05.392039 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2718> 2014-08-13 17:38:05.392040 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2717> 2014-08-13 17:38:05.392088 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.54 192.168.1.111:6836/14923 11 ==== pg_notify(22.1f1(29) epoch 194487) v5 ==== 2341+0+0 (1738206692 0 0) 0x4a0af40 con 0x4a4f380 -2716> 2014-08-13 17:38:05.392098 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2715> 2014-08-13 17:38:05.392099 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2714> 2014-08-13 17:38:05.392100 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4a0af40 pg_notify(22.1f1(29) epoch 194487) v5 -2713> 2014-08-13 17:38:05.392104 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392000, event: header_read, request: pg_notify(22.1f1(29) epoch 194487) v5 -2712> 2014-08-13 17:38:05.392109 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392001, event: throttled, request: pg_notify(22.1f1(29) epoch 194487) v5 -2711> 2014-08-13 17:38:05.392112 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392070, event: all_read, request: pg_notify(22.1f1(29) epoch 194487) v5 -2710> 2014-08-13 17:38:05.392116 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392097, event: dispatched, request: pg_notify(22.1f1(29) epoch 194487) v5 -2709> 2014-08-13 17:38:05.392120 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392120, event: waiting_for_osdmap, request: pg_notify(22.1f1(29) epoch 194487) v5 -2708> 2014-08-13 17:38:05.392123 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.54 -2707> 2014-08-13 17:38:05.392125 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4a0af40 -2706> 2014-08-13 17:38:05.392134 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392134, event: started, request: pg_notify(22.1f1(29) epoch 194487) v5 -2705> 2014-08-13 17:38:05.392149 7fe9a63f4700 5 -- op tracker -- , seq: 638, time: 2014-08-13 17:38:05.392149, event: done, request: pg_notify(22.1f1(29) epoch 194487) v5 -2704> 2014-08-13 17:38:05.392157 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2703> 2014-08-13 17:38:05.392171 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2702> 2014-08-13 17:38:05.391986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.315 -2701> 2014-08-13 17:38:05.392014 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.25 (past last_backfill 0//0//-1) -2700> 2014-08-13 17:38:05.392183 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.25 (past last_backfill 0//0//-1) -2699> 2014-08-13 17:38:05.392188 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.315 -2698> 2014-08-13 17:38:05.392356 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.25 (past last_backfill 0//0//-1) -2697> 2014-08-13 17:38:05.392371 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.315 -2696> 2014-08-13 17:38:05.392525 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.25 (past last_backfill 0//0//-1) -2695> 2014-08-13 17:38:05.392548 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.315 -2694> 2014-08-13 17:38:05.392723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.315 -2693> 2014-08-13 17:38:05.392694 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -2692> 2014-08-13 17:38:05.392903 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 303 as a source -2691> 2014-08-13 17:38:05.392899 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.315 -2690> 2014-08-13 17:38:05.393073 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.303 -2689> 2014-08-13 17:38:05.393074 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.315 -2688> 2014-08-13 17:38:05.393249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.303 -2687> 2014-08-13 17:38:05.393254 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.315 -2686> 2014-08-13 17:38:05.393420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.303 -2685> 2014-08-13 17:38:05.393429 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.315 -2684> 2014-08-13 17:38:05.393723 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.311 192.168.1.101:6816/24270 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d94ec0 con 0x4ecc8e0 -2683> 2014-08-13 17:38:05.393740 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2682> 2014-08-13 17:38:05.393743 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2681> 2014-08-13 17:38:05.393744 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d94ec0 osd_map(194487..194487 src has 185139..194487) v3 -2680> 2014-08-13 17:38:05.393749 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2679> 2014-08-13 17:38:05.393752 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2678> 2014-08-13 17:38:05.393753 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2677> 2014-08-13 17:38:05.393754 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2676> 2014-08-13 17:38:05.393591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.303 -2675> 2014-08-13 17:38:05.393603 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.315 -2674> 2014-08-13 17:38:05.393842 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.311 192.168.1.101:6816/24270 7 ==== pg_notify(22.f3(23) epoch 194487) v5 ==== 1999+0+0 (309258720 0 0) 0x5f039c0 con 0x4ecc8e0 -2673> 2014-08-13 17:38:05.393863 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2672> 2014-08-13 17:38:05.393868 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2671> 2014-08-13 17:38:05.393870 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f039c0 pg_notify(22.f3(23) epoch 194487) v5 -2670> 2014-08-13 17:38:05.393874 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393742, event: header_read, request: pg_notify(22.f3(23) epoch 194487) v5 -2669> 2014-08-13 17:38:05.393880 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393743, event: throttled, request: pg_notify(22.f3(23) epoch 194487) v5 -2668> 2014-08-13 17:38:05.393883 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393809, event: all_read, request: pg_notify(22.f3(23) epoch 194487) v5 -2667> 2014-08-13 17:38:05.393887 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393862, event: dispatched, request: pg_notify(22.f3(23) epoch 194487) v5 -2666> 2014-08-13 17:38:05.393891 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393891, event: waiting_for_osdmap, request: pg_notify(22.f3(23) epoch 194487) v5 -2665> 2014-08-13 17:38:05.393895 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.311 -2664> 2014-08-13 17:38:05.393897 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f039c0 -2663> 2014-08-13 17:38:05.393900 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393900, event: started, request: pg_notify(22.f3(23) epoch 194487) v5 -2662> 2014-08-13 17:38:05.393916 7fe9a63f4700 5 -- op tracker -- , seq: 639, time: 2014-08-13 17:38:05.393916, event: done, request: pg_notify(22.f3(23) epoch 194487) v5 -2661> 2014-08-13 17:38:05.393759 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.303 -2660> 2014-08-13 17:38:05.393925 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2659> 2014-08-13 17:38:05.393927 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2658> 2014-08-13 17:38:05.393777 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.315 -2657> 2014-08-13 17:38:05.393928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.303 -2656> 2014-08-13 17:38:05.393951 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.315 -2655> 2014-08-13 17:38:05.394097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.303 -2654> 2014-08-13 17:38:05.394127 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.315 -2653> 2014-08-13 17:38:05.394270 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.303 -2652> 2014-08-13 17:38:05.394449 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.365 192.168.1.102:6824/15623 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5450000 con 0x4ecac00 -2651> 2014-08-13 17:38:05.394462 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2650> 2014-08-13 17:38:05.394464 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2649> 2014-08-13 17:38:05.394465 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5450000 osd_map(194487..194487 src has 185139..194487) v3 -2648> 2014-08-13 17:38:05.394471 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2647> 2014-08-13 17:38:05.394474 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2646> 2014-08-13 17:38:05.394475 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2645> 2014-08-13 17:38:05.394476 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2644> 2014-08-13 17:38:05.394304 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.315 -2643> 2014-08-13 17:38:05.394526 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.365 192.168.1.102:6824/15623 10 ==== pg_notify(24.9b(2) epoch 194487) v5 ==== 870+0+0 (3287693125 0 0) 0x5107000 con 0x4ecac00 -2642> 2014-08-13 17:38:05.394539 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2641> 2014-08-13 17:38:05.394540 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2640> 2014-08-13 17:38:05.394541 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5107000 pg_notify(24.9b(2) epoch 194487) v5 -2639> 2014-08-13 17:38:05.394545 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394435, event: header_read, request: pg_notify(24.9b(2) epoch 194487) v5 -2638> 2014-08-13 17:38:05.394550 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394436, event: throttled, request: pg_notify(24.9b(2) epoch 194487) v5 -2637> 2014-08-13 17:38:05.394553 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394498, event: all_read, request: pg_notify(24.9b(2) epoch 194487) v5 -2636> 2014-08-13 17:38:05.394556 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394538, event: dispatched, request: pg_notify(24.9b(2) epoch 194487) v5 -2635> 2014-08-13 17:38:05.394561 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394561, event: waiting_for_osdmap, request: pg_notify(24.9b(2) epoch 194487) v5 -2634> 2014-08-13 17:38:05.394565 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.365 -2633> 2014-08-13 17:38:05.394566 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5107000 -2632> 2014-08-13 17:38:05.394569 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394569, event: started, request: pg_notify(24.9b(2) epoch 194487) v5 -2631> 2014-08-13 17:38:05.394586 7fe9a63f4700 5 -- op tracker -- , seq: 640, time: 2014-08-13 17:38:05.394585, event: done, request: pg_notify(24.9b(2) epoch 194487) v5 -2630> 2014-08-13 17:38:05.394593 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2629> 2014-08-13 17:38:05.394594 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2628> 2014-08-13 17:38:05.394440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.303 -2627> 2014-08-13 17:38:05.394478 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.315 -2626> 2014-08-13 17:38:05.394609 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.303 -2625> 2014-08-13 17:38:05.394653 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.315 -2624> 2014-08-13 17:38:05.394778 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.303 -2623> 2014-08-13 17:38:05.394826 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.315 -2622> 2014-08-13 17:38:05.394948 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.303 -2621> 2014-08-13 17:38:05.395004 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -2620> 2014-08-13 17:38:05.395118 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.303 -2619> 2014-08-13 17:38:05.395349 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.368 192.168.1.102:6826/15861 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54daac0 con 0x5251080 -2618> 2014-08-13 17:38:05.395361 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2617> 2014-08-13 17:38:05.395363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2616> 2014-08-13 17:38:05.395365 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54daac0 osd_map(194487..194487 src has 185139..194487) v3 -2615> 2014-08-13 17:38:05.395370 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2614> 2014-08-13 17:38:05.395372 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2613> 2014-08-13 17:38:05.395373 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2612> 2014-08-13 17:38:05.395374 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2611> 2014-08-13 17:38:05.395225 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 330 as a source -2610> 2014-08-13 17:38:05.395290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.303 -2609> 2014-08-13 17:38:05.395492 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.368 192.168.1.102:6826/15861 10 ==== pg_notify(22.22c(25) epoch 194487) v5 ==== 2113+0+0 (2729433784 0 0) 0x58216c0 con 0x5251080 -2608> 2014-08-13 17:38:05.395506 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2607> 2014-08-13 17:38:05.395508 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2606> 2014-08-13 17:38:05.395509 7fe9a63f4700 20 osd.4 194487 _dispatch 0x58216c0 pg_notify(22.22c(25) epoch 194487) v5 -2605> 2014-08-13 17:38:05.395512 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395390, event: header_read, request: pg_notify(22.22c(25) epoch 194487) v5 -2604> 2014-08-13 17:38:05.395517 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395391, event: throttled, request: pg_notify(22.22c(25) epoch 194487) v5 -2603> 2014-08-13 17:38:05.395520 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395458, event: all_read, request: pg_notify(22.22c(25) epoch 194487) v5 -2602> 2014-08-13 17:38:05.395524 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395506, event: dispatched, request: pg_notify(22.22c(25) epoch 194487) v5 -2601> 2014-08-13 17:38:05.395528 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395528, event: waiting_for_osdmap, request: pg_notify(22.22c(25) epoch 194487) v5 -2600> 2014-08-13 17:38:05.395531 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.368 -2599> 2014-08-13 17:38:05.395533 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x58216c0 -2598> 2014-08-13 17:38:05.395535 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395535, event: started, request: pg_notify(22.22c(25) epoch 194487) v5 -2597> 2014-08-13 17:38:05.395549 7fe9a63f4700 5 -- op tracker -- , seq: 641, time: 2014-08-13 17:38:05.395549, event: done, request: pg_notify(22.22c(25) epoch 194487) v5 -2596> 2014-08-13 17:38:05.395556 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2595> 2014-08-13 17:38:05.395557 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2594> 2014-08-13 17:38:05.395400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.330 (past last_backfill 0//0//-1) -2593> 2014-08-13 17:38:05.395459 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.303 -2592> 2014-08-13 17:38:05.395732 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.28 192.168.1.103:6805/37391 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5458240 con 0x49e6300 -2591> 2014-08-13 17:38:05.395747 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2590> 2014-08-13 17:38:05.395574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.330 (past last_backfill 0//0//-1) -2589> 2014-08-13 17:38:05.395749 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2588> 2014-08-13 17:38:05.395750 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5458240 osd_map(194487..194487 src has 185139..194487) v3 -2587> 2014-08-13 17:38:05.395755 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2586> 2014-08-13 17:38:05.395757 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2585> 2014-08-13 17:38:05.395758 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2584> 2014-08-13 17:38:05.395759 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2583> 2014-08-13 17:38:05.395764 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.28 192.168.1.103:6805/37391 10 ==== pg_info(1 pgs e194487:24.b6) v4 ==== 724+0+0 (2704810676 0 0) 0x5823d40 con 0x49e6300 -2582> 2014-08-13 17:38:05.395771 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2581> 2014-08-13 17:38:05.395772 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2580> 2014-08-13 17:38:05.395773 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5823d40 pg_info(1 pgs e194487:24.b6) v4 -2579> 2014-08-13 17:38:05.395776 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395717, event: header_read, request: pg_info(1 pgs e194487:24.b6) v4 -2578> 2014-08-13 17:38:05.395780 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395718, event: throttled, request: pg_info(1 pgs e194487:24.b6) v4 -2577> 2014-08-13 17:38:05.395786 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395752, event: all_read, request: pg_info(1 pgs e194487:24.b6) v4 -2576> 2014-08-13 17:38:05.395789 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395770, event: dispatched, request: pg_info(1 pgs e194487:24.b6) v4 -2575> 2014-08-13 17:38:05.395628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.303 -2574> 2014-08-13 17:38:05.395793 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395793, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.b6) v4 -2573> 2014-08-13 17:38:05.395796 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.b6) v4 from osd.28 -2572> 2014-08-13 17:38:05.395799 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5823d40 -2571> 2014-08-13 17:38:05.395802 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.395802, event: started, request: pg_info(1 pgs e194487:24.b6) v4 -2570> 2014-08-13 17:38:05.395749 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.330 (past last_backfill 0//0//-1) -2569> 2014-08-13 17:38:05.395797 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.303 -2568> 2014-08-13 17:38:05.395924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.330 (past last_backfill 0//0//-1) -2567> 2014-08-13 17:38:05.395966 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.303 -2566> 2014-08-13 17:38:05.396100 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.330 (past last_backfill 0//0//-1) -2565> 2014-08-13 17:38:05.396134 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.303 -2564> 2014-08-13 17:38:05.396293 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.330 (past last_backfill 0//0//-1) -2563> 2014-08-13 17:38:05.396319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.303 -2562> 2014-08-13 17:38:05.396469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.330 (past last_backfill 0//0//-1) -2561> 2014-08-13 17:38:05.396489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.303 -2560> 2014-08-13 17:38:05.396644 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.330 (past last_backfill 0//0//-1) -2559> 2014-08-13 17:38:05.396668 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.303 -2558> 2014-08-13 17:38:05.396818 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.330 (past last_backfill 0//0//-1) -2557> 2014-08-13 17:38:05.396837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.303 -2556> 2014-08-13 17:38:05.396993 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.330 (past last_backfill 0//0//-1) -2555> 2014-08-13 17:38:05.397005 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.303 -2554> 2014-08-13 17:38:05.397168 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.330 (past last_backfill 0//0//-1) -2553> 2014-08-13 17:38:05.397174 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.303 -2552> 2014-08-13 17:38:05.397352 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.303 -2551> 2014-08-13 17:38:05.397349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.330 (past last_backfill 0//0//-1) -2550> 2014-08-13 17:38:05.397524 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.303 -2549> 2014-08-13 17:38:05.397523 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.330 (past last_backfill 0//0//-1) -2548> 2014-08-13 17:38:05.397695 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.303 -2547> 2014-08-13 17:38:05.397697 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.330 (past last_backfill 0//0//-1) -2546> 2014-08-13 17:38:05.397865 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.303 -2545> 2014-08-13 17:38:05.397871 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.330 (past last_backfill 0//0//-1) -2544> 2014-08-13 17:38:05.398036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.303 -2543> 2014-08-13 17:38:05.398045 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.330 (past last_backfill 0//0//-1) -2542> 2014-08-13 17:38:05.398209 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.303 -2541> 2014-08-13 17:38:05.398222 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.330 (past last_backfill 0//0//-1) -2540> 2014-08-13 17:38:05.398379 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.303 -2539> 2014-08-13 17:38:05.398398 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.330 (past last_backfill 0//0//-1) -2538> 2014-08-13 17:38:05.398550 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.303 -2537> 2014-08-13 17:38:05.398573 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.330 (past last_backfill 0//0//-1) -2536> 2014-08-13 17:38:05.398720 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.303 -2535> 2014-08-13 17:38:05.398748 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.330 (past last_backfill 0//0//-1) -2534> 2014-08-13 17:38:05.398890 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.303 -2533> 2014-08-13 17:38:05.398923 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.330 (past last_backfill 0//0//-1) -2532> 2014-08-13 17:38:05.399062 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.303 -2531> 2014-08-13 17:38:05.399098 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.330 (past last_backfill 0//0//-1) -2530> 2014-08-13 17:38:05.399235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.303 -2529> 2014-08-13 17:38:05.399276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.330 (past last_backfill 0//0//-1) -2528> 2014-08-13 17:38:05.399406 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.303 -2527> 2014-08-13 17:38:05.399451 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.330 (past last_backfill 0//0//-1) -2526> 2014-08-13 17:38:05.399576 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.303 -2525> 2014-08-13 17:38:05.399626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.330 (past last_backfill 0//0//-1) -2524> 2014-08-13 17:38:05.399746 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.303 -2523> 2014-08-13 17:38:05.399800 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.330 (past last_backfill 0//0//-1) -2522> 2014-08-13 17:38:05.399917 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.303 -2521> 2014-08-13 17:38:05.399975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.330 (past last_backfill 0//0//-1) -2520> 2014-08-13 17:38:05.400087 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.303 -2519> 2014-08-13 17:38:05.400150 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.330 (past last_backfill 0//0//-1) -2518> 2014-08-13 17:38:05.400260 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.303 -2517> 2014-08-13 17:38:05.400327 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.330 (past last_backfill 0//0//-1) -2516> 2014-08-13 17:38:05.400430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.303 -2515> 2014-08-13 17:38:05.400502 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.330 (past last_backfill 0//0//-1) -2514> 2014-08-13 17:38:05.400600 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.303 -2513> 2014-08-13 17:38:05.400677 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.330 (past last_backfill 0//0//-1) -2512> 2014-08-13 17:38:05.400771 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.303 -2511> 2014-08-13 17:38:05.400852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.330 (past last_backfill 0//0//-1) -2510> 2014-08-13 17:38:05.400941 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.303 -2509> 2014-08-13 17:38:05.401027 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.330 (past last_backfill 0//0//-1) -2508> 2014-08-13 17:38:05.401111 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.303 -2507> 2014-08-13 17:38:05.401205 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.330 (past last_backfill 0//0//-1) -2506> 2014-08-13 17:38:05.401284 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.303 -2505> 2014-08-13 17:38:05.401380 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.330 (past last_backfill 0//0//-1) -2504> 2014-08-13 17:38:05.401454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.303 -2503> 2014-08-13 17:38:05.401555 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.330 (past last_backfill 0//0//-1) -2502> 2014-08-13 17:38:05.401625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.303 -2501> 2014-08-13 17:38:05.401730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.330 (past last_backfill 0//0//-1) -2500> 2014-08-13 17:38:05.401795 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.303 -2499> 2014-08-13 17:38:05.401904 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.330 (past last_backfill 0//0//-1) -2498> 2014-08-13 17:38:05.401968 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.303 -2497> 2014-08-13 17:38:05.402079 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.330 (past last_backfill 0//0//-1) -2496> 2014-08-13 17:38:05.402138 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.303 -2495> 2014-08-13 17:38:05.402257 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.330 (past last_backfill 0//0//-1) -2494> 2014-08-13 17:38:05.402311 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.303 -2493> 2014-08-13 17:38:05.402433 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.330 (past last_backfill 0//0//-1) -2492> 2014-08-13 17:38:05.402481 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.303 -2491> 2014-08-13 17:38:05.402608 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.330 (past last_backfill 0//0//-1) -2490> 2014-08-13 17:38:05.402651 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.303 -2489> 2014-08-13 17:38:05.402783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.330 (past last_backfill 0//0//-1) -2488> 2014-08-13 17:38:05.402821 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -2487> 2014-08-13 17:38:05.402958 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.330 (past last_backfill 0//0//-1) -2486> 2014-08-13 17:38:05.403030 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 362 as a source -2485> 2014-08-13 17:38:05.403132 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.330 (past last_backfill 0//0//-1) -2484> 2014-08-13 17:38:05.403203 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.362 (past last_backfill 0//0//-1) -2483> 2014-08-13 17:38:05.403309 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.330 (past last_backfill 0//0//-1) -2482> 2014-08-13 17:38:05.403373 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.362 (past last_backfill 0//0//-1) -2481> 2014-08-13 17:38:05.403484 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.330 (past last_backfill 0//0//-1) -2480> 2014-08-13 17:38:05.403542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.362 (past last_backfill 0//0//-1) -2479> 2014-08-13 17:38:05.403659 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.330 (past last_backfill 0//0//-1) -2478> 2014-08-13 17:38:05.403711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.362 (past last_backfill 0//0//-1) -2477> 2014-08-13 17:38:05.403834 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.330 (past last_backfill 0//0//-1) -2476> 2014-08-13 17:38:05.403880 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.362 (past last_backfill 0//0//-1) -2475> 2014-08-13 17:38:05.404009 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.330 (past last_backfill 0//0//-1) -2474> 2014-08-13 17:38:05.404049 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.362 (past last_backfill 0//0//-1) -2473> 2014-08-13 17:38:05.404186 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.330 (past last_backfill 0//0//-1) -2472> 2014-08-13 17:38:05.404221 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.362 (past last_backfill 0//0//-1) -2471> 2014-08-13 17:38:05.404361 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.330 (past last_backfill 0//0//-1) -2470> 2014-08-13 17:38:05.404391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.362 (past last_backfill 0//0//-1) -2469> 2014-08-13 17:38:05.404536 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.330 (past last_backfill 0//0//-1) -2468> 2014-08-13 17:38:05.404560 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.362 (past last_backfill 0//0//-1) -2467> 2014-08-13 17:38:05.404711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.330 (past last_backfill 0//0//-1) -2466> 2014-08-13 17:38:05.404728 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.362 (past last_backfill 0//0//-1) -2465> 2014-08-13 17:38:05.404886 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.330 (past last_backfill 0//0//-1) -2464> 2014-08-13 17:38:05.404896 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.362 (past last_backfill 0//0//-1) -2463> 2014-08-13 17:38:05.405064 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.362 (past last_backfill 0//0//-1) -2462> 2014-08-13 17:38:05.405061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.330 (past last_backfill 0//0//-1) -2461> 2014-08-13 17:38:05.405235 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.362 (past last_backfill 0//0//-1) -2460> 2014-08-13 17:38:05.405239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.330 (past last_backfill 0//0//-1) -2459> 2014-08-13 17:38:05.405405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.362 (past last_backfill 0//0//-1) -2458> 2014-08-13 17:38:05.405413 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.330 (past last_backfill 0//0//-1) -2457> 2014-08-13 17:38:05.405575 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.362 (past last_backfill 0//0//-1) -2456> 2014-08-13 17:38:05.405586 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.330 (past last_backfill 0//0//-1) -2455> 2014-08-13 17:38:05.405744 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.362 (past last_backfill 0//0//-1) -2454> 2014-08-13 17:38:05.405760 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -2453> 2014-08-13 17:38:05.405913 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.362 (past last_backfill 0//0//-1) -2452> 2014-08-13 17:38:05.405977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound -2451> 2014-08-13 17:38:05.406082 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.362 (past last_backfill 0//0//-1) -2450> 2014-08-13 17:38:05.406152 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194484 -2449> 2014-08-13 17:38:05.406254 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.362 (past last_backfill 0//0//-1) -2448> 2014-08-13 17:38:05.406332 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 17,28,315,327,330 -2447> 2014-08-13 17:38:05.406424 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.362 (past last_backfill 0//0//-1) -2446> 2014-08-13 17:38:05.406509 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished -2445> 2014-08-13 17:38:05.406593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.362 (past last_backfill 0//0//-1) -2444> 2014-08-13 17:38:05.406686 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -2443> 2014-08-13 17:38:05.406762 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.362 (past last_backfill 0//0//-1) -2442> 2014-08-13 17:38:05.406863 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2441> 2014-08-13 17:38:05.406931 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.362 (past last_backfill 0//0//-1) -2440> 2014-08-13 17:38:05.407043 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2439> 2014-08-13 17:38:05.407100 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.362 (past last_backfill 0//0//-1) -2438> 2014-08-13 17:38:05.407280 7fe99f5e9700 10 log is not dirty -2437> 2014-08-13 17:38:05.407317 7fe9a63f4700 5 -- op tracker -- , seq: 642, time: 2014-08-13 17:38:05.407317, event: done, request: pg_info(1 pgs e194487:24.b6) v4 -2436> 2014-08-13 17:38:05.407332 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2435> 2014-08-13 17:38:05.407335 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2434> 2014-08-13 17:38:05.407331 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [4,47,378]/[19,65] -- 4/19 -2433> 2014-08-13 17:38:05.407339 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.365 192.168.1.102:6824/15623 11 ==== pg_notify(22.481(39) epoch 194487) v5 ==== 2911+0+0 (3431541840 0 0) 0x5f07e00 con 0x4ecac00 -2432> 2014-08-13 17:38:05.407349 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2431> 2014-08-13 17:38:05.407350 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2430> 2014-08-13 17:38:05.407351 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f07e00 pg_notify(22.481(39) epoch 194487) v5 -2429> 2014-08-13 17:38:05.407353 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2428> 2014-08-13 17:38:05.407355 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.395834, event: header_read, request: pg_notify(22.481(39) epoch 194487) v5 -2427> 2014-08-13 17:38:05.407359 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.395837, event: throttled, request: pg_notify(22.481(39) epoch 194487) v5 -2426> 2014-08-13 17:38:05.407362 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.395924, event: all_read, request: pg_notify(22.481(39) epoch 194487) v5 -2425> 2014-08-13 17:38:05.407366 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.407348, event: dispatched, request: pg_notify(22.481(39) epoch 194487) v5 -2424> 2014-08-13 17:38:05.407359 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap -2423> 2014-08-13 17:38:05.407370 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.407370, event: waiting_for_osdmap, request: pg_notify(22.481(39) epoch 194487) v5 -2422> 2014-08-13 17:38:05.407373 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.365 -2421> 2014-08-13 17:38:05.407375 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f07e00 -2420> 2014-08-13 17:38:05.407378 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.407378, event: started, request: pg_notify(22.481(39) epoch 194487) v5 -2419> 2014-08-13 17:38:05.407372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down -2418> 2014-08-13 17:38:05.407385 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map -2417> 2014-08-13 17:38:05.407392 7fe9a63f4700 5 -- op tracker -- , seq: 643, time: 2014-08-13 17:38:05.407392, event: done, request: pg_notify(22.481(39) epoch 194487) v5 -2416> 2014-08-13 17:38:05.407398 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2415> 2014-08-13 17:38:05.407400 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2414> 2014-08-13 17:38:05.407396 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters -2413> 2014-08-13 17:38:05.407407 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.368 192.168.1.102:6826/15861 11 ==== pg_notify(22.1f1(29) epoch 194487) v5 ==== 2341+0+0 (1738206692 0 0) 0x5102f40 con 0x5251080 -2412> 2014-08-13 17:38:05.407406 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2411> 2014-08-13 17:38:05.407415 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2410> 2014-08-13 17:38:05.407416 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2409> 2014-08-13 17:38:05.407418 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_notify(22.1f1(29) epoch 194487) v5 -2408> 2014-08-13 17:38:05.407421 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.395973, event: header_read, request: pg_notify(22.1f1(29) epoch 194487) v5 -2407> 2014-08-13 17:38:05.407416 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2406> 2014-08-13 17:38:05.407424 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.395973, event: throttled, request: pg_notify(22.1f1(29) epoch 194487) v5 -2405> 2014-08-13 17:38:05.407428 7fe99f5e9700 10 log is not dirty -2404> 2014-08-13 17:38:05.407428 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.396047, event: all_read, request: pg_notify(22.1f1(29) epoch 194487) v5 -2403> 2014-08-13 17:38:05.407431 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.407415, event: dispatched, request: pg_notify(22.1f1(29) epoch 194487) v5 -2402> 2014-08-13 17:38:05.407435 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.407435, event: waiting_for_osdmap, request: pg_notify(22.1f1(29) epoch 194487) v5 -2401> 2014-08-13 17:38:05.407438 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.368 -2400> 2014-08-13 17:38:05.407439 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 -2399> 2014-08-13 17:38:05.407274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.362 (past last_backfill 0//0//-1) -2398> 2014-08-13 17:38:05.407442 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.407442, event: started, request: pg_notify(22.1f1(29) epoch 194487) v5 -2397> 2014-08-13 17:38:05.407453 7fe9a63f4700 5 -- op tracker -- , seq: 644, time: 2014-08-13 17:38:05.407453, event: done, request: pg_notify(22.1f1(29) epoch 194487) v5 -2396> 2014-08-13 17:38:05.407444 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 -2395> 2014-08-13 17:38:05.407459 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2394> 2014-08-13 17:38:05.407459 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2393> 2014-08-13 17:38:05.407461 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2392> 2014-08-13 17:38:05.407464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap -2391> 2014-08-13 17:38:05.407466 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.350 192.168.1.114:6832/53185 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dde80 con 0x53809a0 -2390> 2014-08-13 17:38:05.407474 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2389> 2014-08-13 17:38:05.407475 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2388> 2014-08-13 17:38:05.407476 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dde80 osd_map(194487..194487 src has 185139..194487) v3 -2387> 2014-08-13 17:38:05.407473 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down -2386> 2014-08-13 17:38:05.407481 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2385> 2014-08-13 17:38:05.407484 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2384> 2014-08-13 17:38:05.407485 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2383> 2014-08-13 17:38:05.407486 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2382> 2014-08-13 17:38:05.407484 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map -2381> 2014-08-13 17:38:05.407490 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.30 192.168.1.107:6823/55491 11 ==== pg_info(1 pgs e194487:24.34) v4 ==== 740+0+0 (2862715399 0 0) 0x4c16580 con 0x4a44780 -2380> 2014-08-13 17:38:05.407497 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2379> 2014-08-13 17:38:05.407498 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2378> 2014-08-13 17:38:05.407499 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16580 pg_info(1 pgs e194487:24.34) v4 -2377> 2014-08-13 17:38:05.407493 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters -2376> 2014-08-13 17:38:05.407502 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.399222, event: header_read, request: pg_info(1 pgs e194487:24.34) v4 -2375> 2014-08-13 17:38:05.407506 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.399224, event: throttled, request: pg_info(1 pgs e194487:24.34) v4 -2374> 2014-08-13 17:38:05.407502 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2373> 2014-08-13 17:38:05.407510 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.399269, event: all_read, request: pg_info(1 pgs e194487:24.34) v4 -2372> 2014-08-13 17:38:05.407514 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.407497, event: dispatched, request: pg_info(1 pgs e194487:24.34) v4 -2371> 2014-08-13 17:38:05.407511 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2370> 2014-08-13 17:38:05.407518 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.407518, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.34) v4 -2369> 2014-08-13 17:38:05.407521 7fe99f5e9700 10 log is not dirty -2368> 2014-08-13 17:38:05.407521 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.30 -2367> 2014-08-13 17:38:05.407524 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16580 -2366> 2014-08-13 17:38:05.407526 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.407526, event: started, request: pg_info(1 pgs e194487:24.34) v4 -2365> 2014-08-13 17:38:05.407539 7fe9a63f4700 5 -- op tracker -- , seq: 645, time: 2014-08-13 17:38:05.407539, event: done, request: pg_info(1 pgs e194487:24.34) v4 -2364> 2014-08-13 17:38:05.407545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2363> 2014-08-13 17:38:05.407547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2362> 2014-08-13 17:38:05.407535 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 -2361> 2014-08-13 17:38:05.407553 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2360> 2014-08-13 17:38:05.407550 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.50 192.168.1.112:6842/9607 10 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db7500 con 0x4d3dac0 -2359> 2014-08-13 17:38:05.407558 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2358> 2014-08-13 17:38:05.407559 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2357> 2014-08-13 17:38:05.407560 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db7500 osd_map(194487..194487 src has 185139..194487) v3 -2356> 2014-08-13 17:38:05.407563 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2355> 2014-08-13 17:38:05.407557 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Started advmap -2354> 2014-08-13 17:38:05.407566 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2353> 2014-08-13 17:38:05.407568 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2352> 2014-08-13 17:38:05.407569 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2351> 2014-08-13 17:38:05.407568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down -2350> 2014-08-13 17:38:05.407572 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 14 ==== pg_info(1 pgs e194487:24.b6) v4 ==== 724+0+0 (2704810676 0 0) 0x5824d00 con 0x4ec9b80 -2349> 2014-08-13 17:38:05.407579 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2348> 2014-08-13 17:38:05.407580 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2347> 2014-08-13 17:38:05.407581 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5824d00 pg_info(1 pgs e194487:24.b6) v4 -2346> 2014-08-13 17:38:05.407578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map -2345> 2014-08-13 17:38:05.407584 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.395905, event: header_read, request: pg_info(1 pgs e194487:24.b6) v4 -2344> 2014-08-13 17:38:05.407595 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.395907, event: throttled, request: pg_info(1 pgs e194487:24.b6) v4 -2343> 2014-08-13 17:38:05.407598 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.395958, event: all_read, request: pg_info(1 pgs e194487:24.b6) v4 -2342> 2014-08-13 17:38:05.407601 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.407579, event: dispatched, request: pg_info(1 pgs e194487:24.b6) v4 -2341> 2014-08-13 17:38:05.407605 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.407605, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.b6) v4 -2340> 2014-08-13 17:38:05.407608 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.b6) v4 from osd.330 -2339> 2014-08-13 17:38:05.407443 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.362 (past last_backfill 0//0//-1) -2338> 2014-08-13 17:38:05.407610 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5824d00 -2337> 2014-08-13 17:38:05.407613 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.407613, event: started, request: pg_info(1 pgs e194487:24.b6) v4 -2336> 2014-08-13 17:38:05.407626 7fe9a63f4700 5 -- op tracker -- , seq: 646, time: 2014-08-13 17:38:05.407625, event: done, request: pg_info(1 pgs e194487:24.b6) v4 -2335> 2014-08-13 17:38:05.407631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2334> 2014-08-13 17:38:05.407633 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2333> 2014-08-13 17:38:05.407627 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters -2332> 2014-08-13 17:38:05.407636 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.350 192.168.1.114:6832/53185 8 ==== pg_notify(24.9b(34) epoch 194487) v5 ==== 2666+0+0 (1950631126 0 0) 0x5f02bc0 con 0x53809a0 -2331> 2014-08-13 17:38:05.407643 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2330> 2014-08-13 17:38:05.407645 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2329> 2014-08-13 17:38:05.407637 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2328> 2014-08-13 17:38:05.407646 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02bc0 pg_notify(24.9b(34) epoch 194487) v5 -2327> 2014-08-13 17:38:05.407649 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.396282, event: header_read, request: pg_notify(24.9b(34) epoch 194487) v5 -2326> 2014-08-13 17:38:05.407647 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/193151) [356,364,4] r=2 lpr=194486 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2325> 2014-08-13 17:38:05.407652 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.396283, event: throttled, request: pg_notify(24.9b(34) epoch 194487) v5 -2324> 2014-08-13 17:38:05.407657 7fe99f5e9700 10 log is not dirty -2323> 2014-08-13 17:38:05.407656 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.396358, event: all_read, request: pg_notify(24.9b(34) epoch 194487) v5 -2322> 2014-08-13 17:38:05.407659 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.407643, event: dispatched, request: pg_notify(24.9b(34) epoch 194487) v5 -2321> 2014-08-13 17:38:05.407663 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.407663, event: waiting_for_osdmap, request: pg_notify(24.9b(34) epoch 194487) v5 -2320> 2014-08-13 17:38:05.407666 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.350 -2319> 2014-08-13 17:38:05.407668 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02bc0 -2318> 2014-08-13 17:38:05.407670 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.407670, event: started, request: pg_notify(24.9b(34) epoch 194487) v5 -2317> 2014-08-13 17:38:05.407681 7fe9a63f4700 5 -- op tracker -- , seq: 647, time: 2014-08-13 17:38:05.407681, event: done, request: pg_notify(24.9b(34) epoch 194487) v5 -2316> 2014-08-13 17:38:05.407672 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 -2315> 2014-08-13 17:38:05.407690 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2314> 2014-08-13 17:38:05.407690 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2313> 2014-08-13 17:38:05.407691 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2312> 2014-08-13 17:38:05.407695 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Peering advmap -2311> 2014-08-13 17:38:05.407704 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.50 192.168.1.112:6842/9607 11 ==== pg_notify(24.136(2) epoch 194487) v5 ==== 870+0+0 (983766347 0 0) 0x5f02d80 con 0x4d3dac0 -2310> 2014-08-13 17:38:05.407712 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2309> 2014-08-13 17:38:05.407713 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2308> 2014-08-13 17:38:05.407714 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02d80 pg_notify(24.136(2) epoch 194487) v5 -2307> 2014-08-13 17:38:05.407707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] adjust_need_up_thru now 194486, need_up_thru now false -2306> 2014-08-13 17:38:05.407717 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.396096, event: header_read, request: pg_notify(24.136(2) epoch 194487) v5 -2305> 2014-08-13 17:38:05.407721 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.396097, event: throttled, request: pg_notify(24.136(2) epoch 194487) v5 -2304> 2014-08-13 17:38:05.407724 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.396137, event: all_read, request: pg_notify(24.136(2) epoch 194487) v5 -2303> 2014-08-13 17:38:05.407718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Started advmap -2302> 2014-08-13 17:38:05.407728 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.407711, event: dispatched, request: pg_notify(24.136(2) epoch 194487) v5 -2301> 2014-08-13 17:38:05.407732 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.407732, event: waiting_for_osdmap, request: pg_notify(24.136(2) epoch 194487) v5 -2300> 2014-08-13 17:38:05.407735 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.50 -2299> 2014-08-13 17:38:05.407736 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02d80 -2298> 2014-08-13 17:38:05.407730 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] check_recovery_sources no source osds () went down -2297> 2014-08-13 17:38:05.407739 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.407739, event: started, request: pg_notify(24.136(2) epoch 194487) v5 -2296> 2014-08-13 17:38:05.407742 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map -2295> 2014-08-13 17:38:05.407751 7fe9a63f4700 5 -- op tracker -- , seq: 648, time: 2014-08-13 17:38:05.407751, event: done, request: pg_notify(24.136(2) epoch 194487) v5 -2294> 2014-08-13 17:38:05.407757 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2293> 2014-08-13 17:38:05.407759 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2292> 2014-08-13 17:38:05.407753 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: handle ActMap primary -2291> 2014-08-13 17:38:05.407766 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194487:25637 -2290> 2014-08-13 17:38:05.407611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.362 (past last_backfill 0//0//-1) -2289> 2014-08-13 17:38:05.407776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] take_waiters -2288> 2014-08-13 17:38:05.407788 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 0.969818 3 0.009102 -2287> 2014-08-13 17:38:05.407801 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Leaving Peering -2286> 2014-08-13 17:38:05.407811 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 1.053829 0 0.000000 -2285> 2014-08-13 17:38:05.407833 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active -2284> 2014-08-13 17:38:05.407843 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194478 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: In Active, about to call activate -2283> 2014-08-13 17:38:05.407856 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - purged_snaps [] cached_removed_snaps [] -2282> 2014-08-13 17:38:05.407867 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - snap_trimq [] -2281> 2014-08-13 17:38:05.407875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - not complete, missing(62) -2280> 2014-08-13 17:38:05.407779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.362 (past last_backfill 0//0//-1) -2279> 2014-08-13 17:38:05.407947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.362 (past last_backfill 0//0//-1) -2278> 2014-08-13 17:38:05.408115 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.362 (past last_backfill 0//0//-1) -2277> 2014-08-13 17:38:05.408139 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -2276> 2014-08-13 17:38:05.408287 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.362 (past last_backfill 0//0//-1) -2275> 2014-08-13 17:38:05.408350 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194486 pi=193748-194485/35 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] flushed -2274> 2014-08-13 17:38:05.408455 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.362 (past last_backfill 0//0//-1) -2273> 2014-08-13 17:38:05.408723 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - starting recovery -2272> 2014-08-13 17:38:05.408742 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] -2271> 2014-08-13 17:38:05.408768 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -2270> 2014-08-13 17:38:05.408624 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.362 (past last_backfill 0//0//-1) -2269> 2014-08-13 17:38:05.408782 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 is up to date, queueing in pending_activators -2268> 2014-08-13 17:38:05.408854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -2267> 2014-08-13 17:38:05.408870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -2266> 2014-08-13 17:38:05.408888 7fe99f5e9700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 -2265> 2014-08-13 17:38:05.408792 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.362 (past last_backfill 0//0//-1) -2264> 2014-08-13 17:38:05.409056 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 sending log((192718'12872,194422'15456], crt=194422'15456) -2263> 2014-08-13 17:38:05.409075 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x59cf640 don't know epoch, doing nothing -2262> 2014-08-13 17:38:05.409078 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.59 192.168.1.112:6822/5265 -- pg_log(24.1f7 epoch 194487 log log((192718'12872,194422'15456], crt=194422'15456) query_epoch 194487) v4 -- ?+0 0x5fb5400 -2261> 2014-08-13 17:38:05.409093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -2260> 2014-08-13 17:38:05.409110 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) -2259> 2014-08-13 17:38:05.408961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.362 (past last_backfill 0//0//-1) -2258> 2014-08-13 17:38:05.409124 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 is up to date, queueing in pending_activators -2257> 2014-08-13 17:38:05.409153 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194484 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -2256> 2014-08-13 17:38:05.409202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery primary has 62 missing -2255> 2014-08-13 17:38:05.409129 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.362 (past last_backfill 0//0//-1) -2254> 2014-08-13 17:38:05.409280 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.4 -2253> 2014-08-13 17:38:05.409356 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.4 -2252> 2014-08-13 17:38:05.409301 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.362 (past last_backfill 0//0//-1) -2251> 2014-08-13 17:38:05.409430 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.4 -2250> 2014-08-13 17:38:05.409504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.4 -2249> 2014-08-13 17:38:05.409471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.362 (past last_backfill 0//0//-1) -2248> 2014-08-13 17:38:05.409578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.4 -2247> 2014-08-13 17:38:05.409650 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.4 -2246> 2014-08-13 17:38:05.409724 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.4 -2245> 2014-08-13 17:38:05.409640 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.362 (past last_backfill 0//0//-1) -2244> 2014-08-13 17:38:05.409798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.4 -2243> 2014-08-13 17:38:05.409871 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.4 -2242> 2014-08-13 17:38:05.409808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.362 (past last_backfill 0//0//-1) -2241> 2014-08-13 17:38:05.409945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.4 -2240> 2014-08-13 17:38:05.410018 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.4 -2239> 2014-08-13 17:38:05.409977 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.362 (past last_backfill 0//0//-1) -2238> 2014-08-13 17:38:05.410092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.4 -2237> 2014-08-13 17:38:05.410164 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.4 -2236> 2014-08-13 17:38:05.410241 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.4 -2235> 2014-08-13 17:38:05.410146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.362 (past last_backfill 0//0//-1) -2234> 2014-08-13 17:38:05.410314 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.4 -2233> 2014-08-13 17:38:05.410389 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.4 -2232> 2014-08-13 17:38:05.410317 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.362 (past last_backfill 0//0//-1) -2231> 2014-08-13 17:38:05.410462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.4 -2230> 2014-08-13 17:38:05.410536 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.4 -2229> 2014-08-13 17:38:05.410487 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.362 (past last_backfill 0//0//-1) -2228> 2014-08-13 17:38:05.410609 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.4 -2227> 2014-08-13 17:38:05.410683 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.4 -2226> 2014-08-13 17:38:05.410656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.362 (past last_backfill 0//0//-1) -2225> 2014-08-13 17:38:05.410756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.4 -2224> 2014-08-13 17:38:05.410828 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.4 -2223> 2014-08-13 17:38:05.410902 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.4 -2222> 2014-08-13 17:38:05.410826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.362 (past last_backfill 0//0//-1) -2221> 2014-08-13 17:38:05.410975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.4 -2220> 2014-08-13 17:38:05.411048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.4 -2219> 2014-08-13 17:38:05.410995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.362 (past last_backfill 0//0//-1) -2218> 2014-08-13 17:38:05.411122 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.4 -2217> 2014-08-13 17:38:05.411198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.4 -2216> 2014-08-13 17:38:05.411164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.362 (past last_backfill 0//0//-1) -2215> 2014-08-13 17:38:05.411272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.4 -2214> 2014-08-13 17:38:05.411344 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.4 -2213> 2014-08-13 17:38:05.411418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.4 -2212> 2014-08-13 17:38:05.411336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.362 (past last_backfill 0//0//-1) -2211> 2014-08-13 17:38:05.411491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.4 -2210> 2014-08-13 17:38:05.411564 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.4 -2209> 2014-08-13 17:38:05.411504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.362 (past last_backfill 0//0//-1) -2208> 2014-08-13 17:38:05.411638 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.4 -2207> 2014-08-13 17:38:05.411711 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.4 -2206> 2014-08-13 17:38:05.411673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.362 (past last_backfill 0//0//-1) -2205> 2014-08-13 17:38:05.411785 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.4 -2204> 2014-08-13 17:38:05.411857 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.4 -2203> 2014-08-13 17:38:05.411930 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.4 -2202> 2014-08-13 17:38:05.411842 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.362 (past last_backfill 0//0//-1) -2201> 2014-08-13 17:38:05.412003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.4 -2200> 2014-08-13 17:38:05.412078 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.4 -2199> 2014-08-13 17:38:05.412010 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.362 (past last_backfill 0//0//-1) -2198> 2014-08-13 17:38:05.412151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.4 -2197> 2014-08-13 17:38:05.412228 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.4 -2196> 2014-08-13 17:38:05.412179 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.362 (past last_backfill 0//0//-1) -2195> 2014-08-13 17:38:05.412301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.4 -2194> 2014-08-13 17:38:05.412375 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.4 -2193> 2014-08-13 17:38:05.412351 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.362 (past last_backfill 0//0//-1) -2192> 2014-08-13 17:38:05.412448 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.4 -2191> 2014-08-13 17:38:05.412528 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.4 -2190> 2014-08-13 17:38:05.412606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.4 -2189> 2014-08-13 17:38:05.412523 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.362 (past last_backfill 0//0//-1) -2188> 2014-08-13 17:38:05.412680 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.4 -2187> 2014-08-13 17:38:05.412754 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.4 -2186> 2014-08-13 17:38:05.412691 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.362 (past last_backfill 0//0//-1) -2185> 2014-08-13 17:38:05.412827 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.4 -2184> 2014-08-13 17:38:05.412900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.4 -2183> 2014-08-13 17:38:05.412974 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.4 -2182> 2014-08-13 17:38:05.412860 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -2181> 2014-08-13 17:38:05.413048 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.4 -2180> 2014-08-13 17:38:05.413121 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.4 -2179> 2014-08-13 17:38:05.413072 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound -2178> 2014-08-13 17:38:05.413198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.4 -2177> 2014-08-13 17:38:05.413272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.4 -2176> 2014-08-13 17:38:05.413243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _calc_past_interval_range: already have past intervals back to 194484 -2175> 2014-08-13 17:38:05.413346 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.4 -2174> 2014-08-13 17:38:05.413418 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.4 -2173> 2014-08-13 17:38:05.413492 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.4 -2172> 2014-08-13 17:38:05.413414 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound: built 20,25,303,362,373 -2171> 2014-08-13 17:38:05.413565 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.4 -2170> 2014-08-13 17:38:05.413639 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.4 -2169> 2014-08-13 17:38:05.413585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Activate Finished -2168> 2014-08-13 17:38:05.413712 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.4 -2167> 2014-08-13 17:38:05.413786 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.4 -2166> 2014-08-13 17:38:05.413756 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating -2165> 2014-08-13 17:38:05.413858 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -2164> 2014-08-13 17:38:05.413977 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] activate: adding 55 as a source -2163> 2014-08-13 17:38:05.413927 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2162> 2014-08-13 17:38:05.414050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.55 -2161> 2014-08-13 17:38:05.414125 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=61] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.55 -2160> 2014-08-13 17:38:05.414097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2159> 2014-08-13 17:38:05.414206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=60] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.55 -2158> 2014-08-13 17:38:05.414339 7fe99ebe8700 10 log is not dirty -2157> 2014-08-13 17:38:05.414286 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=59] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.55 -2156> 2014-08-13 17:38:05.414376 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 -2155> 2014-08-13 17:38:05.414394 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2154> 2014-08-13 17:38:05.414400 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] state: Started advmap -2153> 2014-08-13 17:38:05.414411 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] check_recovery_sources no source osds () went down -2152> 2014-08-13 17:38:05.414421 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map -2151> 2014-08-13 17:38:05.414431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] take_waiters -2150> 2014-08-13 17:38:05.414439 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2149> 2014-08-13 17:38:05.414367 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=58] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.55 -2148> 2014-08-13 17:38:05.414448 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194485 pi=192971-194484/24 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2147> 2014-08-13 17:38:05.414458 7fe99ebe8700 10 log is not dirty -2146> 2014-08-13 17:38:05.414472 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 -2145> 2014-08-13 17:38:05.414485 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2144> 2014-08-13 17:38:05.414489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2143> 2014-08-13 17:38:05.414499 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2142> 2014-08-13 17:38:05.414510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -2141> 2014-08-13 17:38:05.414520 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2140> 2014-08-13 17:38:05.414450 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=57] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.55 -2139> 2014-08-13 17:38:05.414528 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2138> 2014-08-13 17:38:05.414533 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=56] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.55 -2137> 2014-08-13 17:38:05.414620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=55] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.55 -2136> 2014-08-13 17:38:05.414704 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] -2135> 2014-08-13 17:38:05.414712 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2134> 2014-08-13 17:38:05.414708 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=54] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.55 -2133> 2014-08-13 17:38:05.414800 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:3022 -2132> 2014-08-13 17:38:05.414810 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2131> 2014-08-13 17:38:05.414819 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2130> 2014-08-13 17:38:05.414827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2129> 2014-08-13 17:38:05.414838 7fe99ebe8700 10 log is not dirty -2128> 2014-08-13 17:38:05.414843 7fe99ebe8700 20 osd.4 194487 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 20 -2127> 2014-08-13 17:38:05.414851 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c71b80 already has epoch 194487 -2126> 2014-08-13 17:38:05.414866 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.103:6815/38269 -- pg_info(1 pgs e194487:24.3d(33)) v4 -- ?+0 0x5f01dc0 con 0x4c71b80 -2125> 2014-08-13 17:38:05.414798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=53] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.55 -2124> 2014-08-13 17:38:05.414884 7fe99ebe8700 20 osd.4 194487 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 291 -2123> 2014-08-13 17:38:05.414897 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing -2122> 2014-08-13 17:38:05.414913 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6820/25633 -- pg_info(1 pgs e194487:24.9b(31)) v4 -- ?+0 0x5f01f80 con 0x4a423c0 -2121> 2014-08-13 17:38:05.414925 7fe99ebe8700 20 osd.4 194487 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 303 -2120> 2014-08-13 17:38:05.414738 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -2119> 2014-08-13 17:38:05.414931 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing -2118> 2014-08-13 17:38:05.414943 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.109:6802/30768 -- pg_info(1 pgs e194487:24.3d(33)) v4 -- ?+0 0x5f02140 con 0x4e4d6a0 -2117> 2014-08-13 17:38:05.414954 7fe99ebe8700 20 osd.4 194487 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 350 -2116> 2014-08-13 17:38:05.414962 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing -2115> 2014-08-13 17:38:05.414972 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.114:6832/53185 -- pg_info(1 pgs e194487:24.9b(31)) v4 -- ?+0 0x5f01340 con 0x53809a0 -2114> 2014-08-13 17:38:05.414889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=52] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.55 -2113> 2014-08-13 17:38:05.415033 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 -2112> 2014-08-13 17:38:05.415055 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2111> 2014-08-13 17:38:05.415059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Started advmap -2110> 2014-08-13 17:38:05.414982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=51] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.55 -2109> 2014-08-13 17:38:05.415070 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down -2108> 2014-08-13 17:38:05.415081 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map -2107> 2014-08-13 17:38:05.414933 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] flushed -2106> 2014-08-13 17:38:05.415116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters -2105> 2014-08-13 17:38:05.415126 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2104> 2014-08-13 17:38:05.415135 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2103> 2014-08-13 17:38:05.415145 7fe99ebe8700 10 log is not dirty -2102> 2014-08-13 17:38:05.415076 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=50] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.55 -2101> 2014-08-13 17:38:05.415160 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 -2100> 2014-08-13 17:38:05.415173 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2099> 2014-08-13 17:38:05.415177 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap -2098> 2014-08-13 17:38:05.415191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down -2097> 2014-08-13 17:38:05.415206 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map -2096> 2014-08-13 17:38:05.415215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters -2095> 2014-08-13 17:38:05.415224 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2094> 2014-08-13 17:38:05.415232 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194485 pi=192978-194484/24 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2093> 2014-08-13 17:38:05.415242 7fe99ebe8700 10 log is not dirty -2092> 2014-08-13 17:38:05.415255 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 -2091> 2014-08-13 17:38:05.415267 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2090> 2014-08-13 17:38:05.415172 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=49] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.55 -2089> 2014-08-13 17:38:05.415271 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2088> 2014-08-13 17:38:05.415282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2087> 2014-08-13 17:38:05.415292 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -2086> 2014-08-13 17:38:05.415302 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2085> 2014-08-13 17:38:05.415310 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2084> 2014-08-13 17:38:05.415272 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=48] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.55 -2083> 2014-08-13 17:38:05.415458 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] -2082> 2014-08-13 17:38:05.415372 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=47] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.55 -2081> 2014-08-13 17:38:05.415468 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2080> 2014-08-13 17:38:05.415544 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2685 -2079> 2014-08-13 17:38:05.415555 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2078> 2014-08-13 17:38:05.415564 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2077> 2014-08-13 17:38:05.415474 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=46] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.55 -2076> 2014-08-13 17:38:05.415572 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2075> 2014-08-13 17:38:05.415583 7fe99ebe8700 10 log is not dirty -2074> 2014-08-13 17:38:05.415596 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 -2073> 2014-08-13 17:38:05.415609 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2072> 2014-08-13 17:38:05.415612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2071> 2014-08-13 17:38:05.415622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2070> 2014-08-13 17:38:05.415632 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,287) went down -2069> 2014-08-13 17:38:05.415641 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2068> 2014-08-13 17:38:05.415650 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2067> 2014-08-13 17:38:05.415577 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=45] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.55 -2066> 2014-08-13 17:38:05.415750 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.315 192.168.1.113:6800/14161 7 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5428000 con 0x4ecce60 -2065> 2014-08-13 17:38:05.415772 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2064> 2014-08-13 17:38:05.415776 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2063> 2014-08-13 17:38:05.415779 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5428000 osd_map(194487..194487 src has 185139..194487) v3 -2062> 2014-08-13 17:38:05.415681 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=44] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.55 -2061> 2014-08-13 17:38:05.415786 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -2060> 2014-08-13 17:38:05.415789 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -2059> 2014-08-13 17:38:05.415791 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2058> 2014-08-13 17:38:05.415792 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2057> 2014-08-13 17:38:05.415842 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] -2056> 2014-08-13 17:38:05.415850 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2055> 2014-08-13 17:38:05.415786 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=43] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.55 -2054> 2014-08-13 17:38:05.415891 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.315 192.168.1.113:6800/14161 8 ==== pg_notify(24.b6(38) epoch 194487) v5 ==== 2898+0+0 (1601963620 0 0) 0x5825940 con 0x4ecce60 -2053> 2014-08-13 17:38:05.415910 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2052> 2014-08-13 17:38:05.415913 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2051> 2014-08-13 17:38:05.415914 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825940 pg_notify(24.b6(38) epoch 194487) v5 -2050> 2014-08-13 17:38:05.415918 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415765, event: header_read, request: pg_notify(24.b6(38) epoch 194487) v5 -2049> 2014-08-13 17:38:05.415924 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415766, event: throttled, request: pg_notify(24.b6(38) epoch 194487) v5 -2048> 2014-08-13 17:38:05.415928 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415840, event: all_read, request: pg_notify(24.b6(38) epoch 194487) v5 -2047> 2014-08-13 17:38:05.415932 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415909, event: dispatched, request: pg_notify(24.b6(38) epoch 194487) v5 -2046> 2014-08-13 17:38:05.415936 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415936, event: waiting_for_osdmap, request: pg_notify(24.b6(38) epoch 194487) v5 -2045> 2014-08-13 17:38:05.415940 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.315 -2044> 2014-08-13 17:38:05.415942 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825940 -2043> 2014-08-13 17:38:05.415945 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415945, event: started, request: pg_notify(24.b6(38) epoch 194487) v5 -2042> 2014-08-13 17:38:05.415947 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2684 -2041> 2014-08-13 17:38:05.415956 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2040> 2014-08-13 17:38:05.415961 7fe9a63f4700 5 -- op tracker -- , seq: 649, time: 2014-08-13 17:38:05.415961, event: done, request: pg_notify(24.b6(38) epoch 194487) v5 -2039> 2014-08-13 17:38:05.415971 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -2038> 2014-08-13 17:38:05.415965 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2037> 2014-08-13 17:38:05.415973 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -2036> 2014-08-13 17:38:05.415974 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2035> 2014-08-13 17:38:05.415985 7fe99ebe8700 10 log is not dirty -2034> 2014-08-13 17:38:05.415894 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=42] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.55 -2033> 2014-08-13 17:38:05.415998 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 -2032> 2014-08-13 17:38:05.416010 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2031> 2014-08-13 17:38:05.416014 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2030> 2014-08-13 17:38:05.416023 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2029> 2014-08-13 17:38:05.416033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,289) went down -2028> 2014-08-13 17:38:05.416042 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2027> 2014-08-13 17:38:05.416051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2026> 2014-08-13 17:38:05.416002 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=41] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.55 -2025> 2014-08-13 17:38:05.416113 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=40] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.55 -2024> 2014-08-13 17:38:05.416293 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] -2023> 2014-08-13 17:38:05.416303 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2022> 2014-08-13 17:38:05.416230 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=39] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.55 -2021> 2014-08-13 17:38:05.416423 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2690 -2020> 2014-08-13 17:38:05.416434 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2019> 2014-08-13 17:38:05.416443 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2018> 2014-08-13 17:38:05.416451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2017> 2014-08-13 17:38:05.416346 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=38] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.55 -2016> 2014-08-13 17:38:05.416461 7fe99ebe8700 10 log is not dirty -2015> 2014-08-13 17:38:05.416475 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 -2014> 2014-08-13 17:38:05.416488 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2013> 2014-08-13 17:38:05.416491 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] state: Started advmap -2012> 2014-08-13 17:38:05.416500 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] check_recovery_sources no source osds () went down -2011> 2014-08-13 17:38:05.416510 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map -2010> 2014-08-13 17:38:05.416519 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] take_waiters -2009> 2014-08-13 17:38:05.416527 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2008> 2014-08-13 17:38:05.416536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194485 pi=192975-194484/24 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -2007> 2014-08-13 17:38:05.416545 7fe99ebe8700 10 log is not dirty -2006> 2014-08-13 17:38:05.416559 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 -2005> 2014-08-13 17:38:05.416575 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2004> 2014-08-13 17:38:05.416462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=37] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.55 -2003> 2014-08-13 17:38:05.416578 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Started advmap -2002> 2014-08-13 17:38:05.416588 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down -2001> 2014-08-13 17:38:05.416620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map -2000> 2014-08-13 17:38:05.416656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters -1999> 2014-08-13 17:38:05.416665 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1998> 2014-08-13 17:38:05.416674 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1997> 2014-08-13 17:38:05.416684 7fe99ebe8700 10 log is not dirty -1996> 2014-08-13 17:38:05.416580 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=36] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.55 -1995> 2014-08-13 17:38:05.416699 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 -1994> 2014-08-13 17:38:05.416718 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1993> 2014-08-13 17:38:05.416722 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Peering advmap -1992> 2014-08-13 17:38:05.416733 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] adjust_need_up_thru now 194486, need_up_thru now false -1991> 2014-08-13 17:38:05.416743 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Started advmap -1990> 2014-08-13 17:38:05.416754 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] check_recovery_sources no source osds () went down -1989> 2014-08-13 17:38:05.416764 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map -1988> 2014-08-13 17:38:05.416775 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: handle ActMap primary -1987> 2014-08-13 17:38:05.416786 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194487:22150 -1986> 2014-08-13 17:38:05.416796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] take_waiters -1985> 2014-08-13 17:38:05.416806 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 0.970981 3 0.000511 -1984> 2014-08-13 17:38:05.416699 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=35] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.55 -1983> 2014-08-13 17:38:05.416819 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Leaving Peering -1982> 2014-08-13 17:38:05.416830 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 1.009659 0 0.000000 -1981> 2014-08-13 17:38:05.416845 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active -1980> 2014-08-13 17:38:05.416855 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194478 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: In Active, about to call activate -1979> 2014-08-13 17:38:05.416868 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - purged_snaps [] cached_removed_snaps [] -1978> 2014-08-13 17:38:05.416878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - snap_trimq [] -1977> 2014-08-13 17:38:05.416888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - not complete, missing(48) -1976> 2014-08-13 17:38:05.416821 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=34] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.55 -1975> 2014-08-13 17:38:05.416944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=33] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.55 -1974> 2014-08-13 17:38:05.417070 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=32] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.55 -1973> 2014-08-13 17:38:05.417200 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=31] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.55 -1972> 2014-08-13 17:38:05.417434 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.350 192.168.1.114:6832/53185 9 ==== pg_info(1 pgs e194487:24.9b) v4 ==== 748+0+0 (978563680 0 0) 0x5f01340 con 0x53809a0 -1971> 2014-08-13 17:38:05.417456 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1970> 2014-08-13 17:38:05.417460 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1969> 2014-08-13 17:38:05.417334 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=30] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.55 -1968> 2014-08-13 17:38:05.417463 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01340 pg_info(1 pgs e194487:24.9b) v4 -1967> 2014-08-13 17:38:05.417468 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417335, event: header_read, request: pg_info(1 pgs e194487:24.9b) v4 -1966> 2014-08-13 17:38:05.417474 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417336, event: throttled, request: pg_info(1 pgs e194487:24.9b) v4 -1965> 2014-08-13 17:38:05.417481 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417382, event: all_read, request: pg_info(1 pgs e194487:24.9b) v4 -1964> 2014-08-13 17:38:05.417484 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417455, event: dispatched, request: pg_info(1 pgs e194487:24.9b) v4 -1963> 2014-08-13 17:38:05.417489 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417489, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.9b) v4 -1962> 2014-08-13 17:38:05.417493 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.350 -1961> 2014-08-13 17:38:05.417496 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01340 -1960> 2014-08-13 17:38:05.417499 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417499, event: started, request: pg_info(1 pgs e194487:24.9b) v4 -1959> 2014-08-13 17:38:05.417514 7fe9a63f4700 5 -- op tracker -- , seq: 650, time: 2014-08-13 17:38:05.417514, event: done, request: pg_info(1 pgs e194487:24.9b) v4 -1958> 2014-08-13 17:38:05.417532 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1957> 2014-08-13 17:38:05.417536 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1956> 2014-08-13 17:38:05.417464 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=29] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.55 -1955> 2014-08-13 17:38:05.417697 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.20 192.168.1.103:6815/38269 11 ==== pg_info(1 pgs e194487:24.3d) v4 ==== 748+0+0 (510834547 0 0) 0x5f01dc0 con 0x4c71b80 -1954> 2014-08-13 17:38:05.417709 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1953> 2014-08-13 17:38:05.417711 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1952> 2014-08-13 17:38:05.417712 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01dc0 pg_info(1 pgs e194487:24.3d) v4 -1951> 2014-08-13 17:38:05.417716 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417511, event: header_read, request: pg_info(1 pgs e194487:24.3d) v4 -1950> 2014-08-13 17:38:05.417720 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417513, event: throttled, request: pg_info(1 pgs e194487:24.3d) v4 -1949> 2014-08-13 17:38:05.417723 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417557, event: all_read, request: pg_info(1 pgs e194487:24.3d) v4 -1948> 2014-08-13 17:38:05.417727 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417708, event: dispatched, request: pg_info(1 pgs e194487:24.3d) v4 -1947> 2014-08-13 17:38:05.417597 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=28] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.55 -1946> 2014-08-13 17:38:05.417730 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417730, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3d) v4 -1945> 2014-08-13 17:38:05.417734 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.20 -1944> 2014-08-13 17:38:05.417737 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01dc0 -1943> 2014-08-13 17:38:05.417740 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417739, event: started, request: pg_info(1 pgs e194487:24.3d) v4 -1942> 2014-08-13 17:38:05.417753 7fe9a63f4700 5 -- op tracker -- , seq: 651, time: 2014-08-13 17:38:05.417753, event: done, request: pg_info(1 pgs e194487:24.3d) v4 -1941> 2014-08-13 17:38:05.417760 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1940> 2014-08-13 17:38:05.417761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1939> 2014-08-13 17:38:05.417824 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - starting recovery -1938> 2014-08-13 17:38:05.417840 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] -1937> 2014-08-13 17:38:05.417733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=27] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.55 -1936> 2014-08-13 17:38:05.417866 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1935> 2014-08-13 17:38:05.417884 7fe99ebe8700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 -1934> 2014-08-13 17:38:05.417869 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=26] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.55 -1933> 2014-08-13 17:38:05.417990 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.291 192.168.1.101:6820/25633 9 ==== pg_info(1 pgs e194487:24.9b) v4 ==== 748+0+0 (1970246097 0 0) 0x5f01f80 con 0x4a423c0 -1932> 2014-08-13 17:38:05.418008 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1931> 2014-08-13 17:38:05.418010 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1930> 2014-08-13 17:38:05.418012 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01f80 pg_info(1 pgs e194487:24.9b) v4 -1929> 2014-08-13 17:38:05.418016 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.417894, event: header_read, request: pg_info(1 pgs e194487:24.9b) v4 -1928> 2014-08-13 17:38:05.418022 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.417895, event: throttled, request: pg_info(1 pgs e194487:24.9b) v4 -1927> 2014-08-13 17:38:05.418025 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.417956, event: all_read, request: pg_info(1 pgs e194487:24.9b) v4 -1926> 2014-08-13 17:38:05.418029 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.418007, event: dispatched, request: pg_info(1 pgs e194487:24.9b) v4 -1925> 2014-08-13 17:38:05.418033 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.418033, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.9b) v4 -1924> 2014-08-13 17:38:05.418036 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.9b) v4 from osd.291 -1923> 2014-08-13 17:38:05.418039 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01f80 -1922> 2014-08-13 17:38:05.418042 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.418042, event: started, request: pg_info(1 pgs e194487:24.9b) v4 -1921> 2014-08-13 17:38:05.418057 7fe9a63f4700 5 -- op tracker -- , seq: 652, time: 2014-08-13 17:38:05.418057, event: done, request: pg_info(1 pgs e194487:24.9b) v4 -1920> 2014-08-13 17:38:05.418065 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1919> 2014-08-13 17:38:05.418067 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1918> 2014-08-13 17:38:05.418121 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 sending log((192718'9695,194422'12517], crt=194422'12517) -1917> 2014-08-13 17:38:05.418137 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing -1916> 2014-08-13 17:38:05.418006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=25] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.55 -1915> 2014-08-13 17:38:05.418139 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.331 192.168.1.104:6816/837 -- pg_log(24.1bb epoch 194487 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194487) v4 -- ?+0 0x4872a00 -1914> 2014-08-13 17:38:05.418156 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1913> 2014-08-13 17:38:05.418171 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -1912> 2014-08-13 17:38:05.418183 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 is up to date, queueing in pending_activators -1911> 2014-08-13 17:38:05.418243 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1910> 2014-08-13 17:38:05.418256 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) -1909> 2014-08-13 17:38:05.418268 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 is up to date, queueing in pending_activators -1908> 2014-08-13 17:38:05.418144 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=24] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.55 -1907> 2014-08-13 17:38:05.418299 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1906> 2014-08-13 17:38:05.418312 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1905> 2014-08-13 17:38:05.418327 7fe99ebe8700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 -1904> 2014-08-13 17:38:05.418289 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=23] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.55 -1903> 2014-08-13 17:38:05.418431 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=22] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.55 -1902> 2014-08-13 17:38:05.418594 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 sending log((192718'9695,194422'12517], crt=194422'12517) -1901> 2014-08-13 17:38:05.418610 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing -1900> 2014-08-13 17:38:05.418612 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.348 192.168.1.110:6800/41204 -- pg_log(24.1bb epoch 194487 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194487) v4 -- ?+0 0x5fb2400 -1899> 2014-08-13 17:38:05.418627 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1898> 2014-08-13 17:38:05.418574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=21] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.55 -1897> 2014-08-13 17:38:05.418664 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery primary has 48 missing -1896> 2014-08-13 17:38:05.418726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.4 -1895> 2014-08-13 17:38:05.418829 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.303 192.168.1.109:6802/30768 21 ==== pg_info(1 pgs e194487:24.3d) v4 ==== 748+0+0 (1654874937 0 0) 0x5f02140 con 0x4e4d6a0 -1894> 2014-08-13 17:38:05.418847 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1893> 2014-08-13 17:38:05.418850 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1892> 2014-08-13 17:38:05.418851 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02140 pg_info(1 pgs e194487:24.3d) v4 -1891> 2014-08-13 17:38:05.418855 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418760, event: header_read, request: pg_info(1 pgs e194487:24.3d) v4 -1890> 2014-08-13 17:38:05.418804 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.4 -1889> 2014-08-13 17:38:05.418861 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418761, event: throttled, request: pg_info(1 pgs e194487:24.3d) v4 -1888> 2014-08-13 17:38:05.418719 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=20] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.55 -1887> 2014-08-13 17:38:05.418865 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418811, event: all_read, request: pg_info(1 pgs e194487:24.3d) v4 -1886> 2014-08-13 17:38:05.418869 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418846, event: dispatched, request: pg_info(1 pgs e194487:24.3d) v4 -1885> 2014-08-13 17:38:05.418873 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418873, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3d) v4 -1884> 2014-08-13 17:38:05.418877 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.303 -1883> 2014-08-13 17:38:05.418880 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02140 -1882> 2014-08-13 17:38:05.418883 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418883, event: started, request: pg_info(1 pgs e194487:24.3d) v4 -1881> 2014-08-13 17:38:05.418898 7fe9a63f4700 5 -- op tracker -- , seq: 653, time: 2014-08-13 17:38:05.418898, event: done, request: pg_info(1 pgs e194487:24.3d) v4 -1880> 2014-08-13 17:38:05.418907 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1879> 2014-08-13 17:38:05.418908 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1878> 2014-08-13 17:38:05.418864 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.4 -1877> 2014-08-13 17:38:05.418924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.4 -1876> 2014-08-13 17:38:05.418866 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=19] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.55 -1875> 2014-08-13 17:38:05.418982 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.4 -1874> 2014-08-13 17:38:05.419040 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.4 -1873> 2014-08-13 17:38:05.419099 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.4 -1872> 2014-08-13 17:38:05.419015 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=18] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.55 -1871> 2014-08-13 17:38:05.419157 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.4 -1870> 2014-08-13 17:38:05.419220 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.4 -1869> 2014-08-13 17:38:05.419165 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=17] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.55 -1868> 2014-08-13 17:38:05.419278 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.4 -1867> 2014-08-13 17:38:05.419336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.4 -1866> 2014-08-13 17:38:05.419395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.4 -1865> 2014-08-13 17:38:05.419440 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.25 192.168.1.103:6800/36920 5 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54dc800 con 0x4eccba0 -1864> 2014-08-13 17:38:05.419455 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1863> 2014-08-13 17:38:05.419457 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1862> 2014-08-13 17:38:05.419458 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54dc800 osd_map(194487..194487 src has 185139..194487) v3 -1861> 2014-08-13 17:38:05.419464 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -1860> 2014-08-13 17:38:05.419467 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -1859> 2014-08-13 17:38:05.419468 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1858> 2014-08-13 17:38:05.419470 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1857> 2014-08-13 17:38:05.419320 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=16] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.55 -1856> 2014-08-13 17:38:05.419500 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.25 192.168.1.103:6800/36920 6 ==== pg_info(1 pgs e194487:24.3d) v4 ==== 724+0+0 (2304187538 0 0) 0x5104440 con 0x4eccba0 -1855> 2014-08-13 17:38:05.419453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.4 -1854> 2014-08-13 17:38:05.419511 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1853> 2014-08-13 17:38:05.419512 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1852> 2014-08-13 17:38:05.419514 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104440 pg_info(1 pgs e194487:24.3d) v4 -1851> 2014-08-13 17:38:05.419517 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419440, event: header_read, request: pg_info(1 pgs e194487:24.3d) v4 -1850> 2014-08-13 17:38:05.419522 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419440, event: throttled, request: pg_info(1 pgs e194487:24.3d) v4 -1849> 2014-08-13 17:38:05.419525 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419486, event: all_read, request: pg_info(1 pgs e194487:24.3d) v4 -1848> 2014-08-13 17:38:05.419529 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419510, event: dispatched, request: pg_info(1 pgs e194487:24.3d) v4 -1847> 2014-08-13 17:38:05.419533 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419533, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3d) v4 -1846> 2014-08-13 17:38:05.419536 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.25 -1845> 2014-08-13 17:38:05.419539 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104440 -1844> 2014-08-13 17:38:05.419542 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419541, event: started, request: pg_info(1 pgs e194487:24.3d) v4 -1843> 2014-08-13 17:38:05.419558 7fe9a63f4700 5 -- op tracker -- , seq: 654, time: 2014-08-13 17:38:05.419557, event: done, request: pg_info(1 pgs e194487:24.3d) v4 -1842> 2014-08-13 17:38:05.419511 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.4 -1841> 2014-08-13 17:38:05.419568 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1840> 2014-08-13 17:38:05.419570 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1839> 2014-08-13 17:38:05.419570 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.4 -1838> 2014-08-13 17:38:05.419474 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=15] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.55 -1837> 2014-08-13 17:38:05.419628 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.4 -1836> 2014-08-13 17:38:05.419687 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.4 -1835> 2014-08-13 17:38:05.419630 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=14] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.55 -1834> 2014-08-13 17:38:05.419745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.4 -1833> 2014-08-13 17:38:05.419803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.4 -1832> 2014-08-13 17:38:05.419861 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.4 -1831> 2014-08-13 17:38:05.419788 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=13] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.55 -1830> 2014-08-13 17:38:05.419922 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.4 -1829> 2014-08-13 17:38:05.419983 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.4 -1828> 2014-08-13 17:38:05.420041 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.4 -1827> 2014-08-13 17:38:05.419947 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=12] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.55 -1826> 2014-08-13 17:38:05.420101 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.4 -1825> 2014-08-13 17:38:05.420161 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.4 -1824> 2014-08-13 17:38:05.420108 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=11] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.55 -1823> 2014-08-13 17:38:05.420223 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.4 -1822> 2014-08-13 17:38:05.420282 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.4 -1821> 2014-08-13 17:38:05.420342 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.4 -1820> 2014-08-13 17:38:05.420276 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=10] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.55 -1819> 2014-08-13 17:38:05.420402 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.4 -1818> 2014-08-13 17:38:05.420460 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.4 -1817> 2014-08-13 17:38:05.420519 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.4 -1816> 2014-08-13 17:38:05.420444 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=9] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.55 -1815> 2014-08-13 17:38:05.420579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.4 -1814> 2014-08-13 17:38:05.420638 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.4 -1813> 2014-08-13 17:38:05.420698 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.4 -1812> 2014-08-13 17:38:05.420612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=8] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.55 -1811> 2014-08-13 17:38:05.420758 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.4 -1810> 2014-08-13 17:38:05.420818 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.4 -1809> 2014-08-13 17:38:05.420878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.4 -1808> 2014-08-13 17:38:05.420783 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=7] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.55 -1807> 2014-08-13 17:38:05.420938 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.4 -1806> 2014-08-13 17:38:05.420998 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.4 -1805> 2014-08-13 17:38:05.421057 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.4 -1804> 2014-08-13 17:38:05.420975 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=6] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.55 -1803> 2014-08-13 17:38:05.421117 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.4 -1802> 2014-08-13 17:38:05.421177 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.4 -1801> 2014-08-13 17:38:05.421239 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.4 -1800> 2014-08-13 17:38:05.421151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=5] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.55 -1799> 2014-08-13 17:38:05.421297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.4 -1798> 2014-08-13 17:38:05.421355 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.4 -1797> 2014-08-13 17:38:05.421415 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.4 -1796> 2014-08-13 17:38:05.421329 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=4] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.55 -1795> 2014-08-13 17:38:05.421475 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.4 -1794> 2014-08-13 17:38:05.421534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.4 -1793> 2014-08-13 17:38:05.421507 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=3] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.55 -1792> 2014-08-13 17:38:05.421594 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1791> 2014-08-13 17:38:05.421698 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 331 as a source -1790> 2014-08-13 17:38:05.421793 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.331 (past last_backfill 0//0//-1) -1789> 2014-08-13 17:38:05.421686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=2] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.55 -1788> 2014-08-13 17:38:05.421857 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.331 (past last_backfill 0//0//-1) -1787> 2014-08-13 17:38:05.421915 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.331 (past last_backfill 0//0//-1) -1786> 2014-08-13 17:38:05.421973 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.331 (past last_backfill 0//0//-1) -1785> 2014-08-13 17:38:05.422089 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.331 (past last_backfill 0//0//-1) -1784> 2014-08-13 17:38:05.421900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=1] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.55 -1783> 2014-08-13 17:38:05.422360 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -1782> 2014-08-13 17:38:05.422266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.331 (past last_backfill 0//0//-1) -1781> 2014-08-13 17:38:05.422791 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 59 as a source -1780> 2014-08-13 17:38:05.422843 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.331 (past last_backfill 0//0//-1) -1779> 2014-08-13 17:38:05.423052 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.331 (past last_backfill 0//0//-1) -1778> 2014-08-13 17:38:05.422981 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.59 (past last_backfill 0//0//-1) -1777> 2014-08-13 17:38:05.423119 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.331 (past last_backfill 0//0//-1) -1776> 2014-08-13 17:38:05.423191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.331 (past last_backfill 0//0//-1) -1775> 2014-08-13 17:38:05.423249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.331 (past last_backfill 0//0//-1) -1774> 2014-08-13 17:38:05.423166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.59 (past last_backfill 0//0//-1) -1773> 2014-08-13 17:38:05.423320 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.331 (past last_backfill 0//0//-1) -1772> 2014-08-13 17:38:05.423420 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.331 (past last_backfill 0//0//-1) -1771> 2014-08-13 17:38:05.423408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.59 (past last_backfill 0//0//-1) -1770> 2014-08-13 17:38:05.423595 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.331 (past last_backfill 0//0//-1) -1769> 2014-08-13 17:38:05.423656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.331 (past last_backfill 0//0//-1) -1768> 2014-08-13 17:38:05.423717 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.331 (past last_backfill 0//0//-1) -1767> 2014-08-13 17:38:05.423815 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.331 (past last_backfill 0//0//-1) -1766> 2014-08-13 17:38:05.423916 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.331 (past last_backfill 0//0//-1) -1765> 2014-08-13 17:38:05.423976 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.331 (past last_backfill 0//0//-1) -1764> 2014-08-13 17:38:05.424176 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.331 (past last_backfill 0//0//-1) -1763> 2014-08-13 17:38:05.424242 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.331 (past last_backfill 0//0//-1) -1762> 2014-08-13 17:38:05.424300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.331 (past last_backfill 0//0//-1) -1761> 2014-08-13 17:38:05.424501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.331 (past last_backfill 0//0//-1) -1760> 2014-08-13 17:38:05.424561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.331 (past last_backfill 0//0//-1) -1759> 2014-08-13 17:38:05.424661 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.331 (past last_backfill 0//0//-1) -1758> 2014-08-13 17:38:05.424726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.331 (past last_backfill 0//0//-1) -1757> 2014-08-13 17:38:05.424786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.331 (past last_backfill 0//0//-1) -1756> 2014-08-13 17:38:05.424845 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.331 (past last_backfill 0//0//-1) -1755> 2014-08-13 17:38:05.424921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.331 (past last_backfill 0//0//-1) -1754> 2014-08-13 17:38:05.424980 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.331 (past last_backfill 0//0//-1) -1753> 2014-08-13 17:38:05.425037 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.331 (past last_backfill 0//0//-1) -1752> 2014-08-13 17:38:05.425591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.331 (past last_backfill 0//0//-1) -1751> 2014-08-13 17:38:05.425654 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.331 (past last_backfill 0//0//-1) -1750> 2014-08-13 17:38:05.425711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.331 (past last_backfill 0//0//-1) -1749> 2014-08-13 17:38:05.425769 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.331 (past last_backfill 0//0//-1) -1748> 2014-08-13 17:38:05.425826 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.331 (past last_backfill 0//0//-1) -1747> 2014-08-13 17:38:05.425883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.331 (past last_backfill 0//0//-1) -1746> 2014-08-13 17:38:05.425940 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.331 (past last_backfill 0//0//-1) -1745> 2014-08-13 17:38:05.425997 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.331 (past last_backfill 0//0//-1) -1744> 2014-08-13 17:38:05.426053 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.362 192.168.1.108:6842/8170 12 ==== pg_info(1 pgs e194487:24.3d) v4 ==== 724+0+0 (2304187538 0 0) 0x5138000 con 0x4d7d280 -1743> 2014-08-13 17:38:05.426072 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1742> 2014-08-13 17:38:05.426074 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1741> 2014-08-13 17:38:05.426076 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5138000 pg_info(1 pgs e194487:24.3d) v4 -1740> 2014-08-13 17:38:05.426082 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.425954, event: header_read, request: pg_info(1 pgs e194487:24.3d) v4 -1739> 2014-08-13 17:38:05.426088 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.425956, event: throttled, request: pg_info(1 pgs e194487:24.3d) v4 -1738> 2014-08-13 17:38:05.426092 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.426031, event: all_read, request: pg_info(1 pgs e194487:24.3d) v4 -1737> 2014-08-13 17:38:05.426095 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.426070, event: dispatched, request: pg_info(1 pgs e194487:24.3d) v4 -1736> 2014-08-13 17:38:05.426100 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.426100, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3d) v4 -1735> 2014-08-13 17:38:05.426103 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3d) v4 from osd.362 -1734> 2014-08-13 17:38:05.426107 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5138000 -1733> 2014-08-13 17:38:05.426054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.331 (past last_backfill 0//0//-1) -1732> 2014-08-13 17:38:05.426111 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.426111, event: started, request: pg_info(1 pgs e194487:24.3d) v4 -1731> 2014-08-13 17:38:05.426131 7fe9a63f4700 5 -- op tracker -- , seq: 655, time: 2014-08-13 17:38:05.426131, event: done, request: pg_info(1 pgs e194487:24.3d) v4 -1730> 2014-08-13 17:38:05.426141 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1729> 2014-08-13 17:38:05.426143 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1728> 2014-08-13 17:38:05.426112 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.331 (past last_backfill 0//0//-1) -1727> 2014-08-13 17:38:05.426170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.331 (past last_backfill 0//0//-1) -1726> 2014-08-13 17:38:05.426232 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.331 (past last_backfill 0//0//-1) -1725> 2014-08-13 17:38:05.426318 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.0 192.168.1.115:6810/59615 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5490000 con 0x4ec8c60 -1724> 2014-08-13 17:38:05.426290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.331 (past last_backfill 0//0//-1) -1723> 2014-08-13 17:38:05.426333 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1722> 2014-08-13 17:38:05.426350 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1721> 2014-08-13 17:38:05.426351 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5490000 osd_map(194487..194487 src has 185139..194487) v3 -1720> 2014-08-13 17:38:05.426360 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -1719> 2014-08-13 17:38:05.426363 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -1718> 2014-08-13 17:38:05.426364 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1717> 2014-08-13 17:38:05.426365 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1716> 2014-08-13 17:38:05.426348 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.331 (past last_backfill 0//0//-1) -1715> 2014-08-13 17:38:05.426422 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.0 192.168.1.115:6810/59615 7 ==== pg_notify(22.775(31) epoch 194487) v5 ==== 2455+0+0 (3269143235 0 0) 0x5f00c40 con 0x4ec8c60 -1714> 2014-08-13 17:38:05.426434 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1713> 2014-08-13 17:38:05.426436 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1712> 2014-08-13 17:38:05.426437 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f00c40 pg_notify(22.775(31) epoch 194487) v5 -1711> 2014-08-13 17:38:05.426441 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426332, event: header_read, request: pg_notify(22.775(31) epoch 194487) v5 -1710> 2014-08-13 17:38:05.426446 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426334, event: throttled, request: pg_notify(22.775(31) epoch 194487) v5 -1709> 2014-08-13 17:38:05.426405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.331 (past last_backfill 0//0//-1) -1708> 2014-08-13 17:38:05.426464 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426409, event: all_read, request: pg_notify(22.775(31) epoch 194487) v5 -1707> 2014-08-13 17:38:05.426470 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426434, event: dispatched, request: pg_notify(22.775(31) epoch 194487) v5 -1706> 2014-08-13 17:38:05.426475 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426475, event: waiting_for_osdmap, request: pg_notify(22.775(31) epoch 194487) v5 -1705> 2014-08-13 17:38:05.426480 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.0 -1704> 2014-08-13 17:38:05.426487 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f00c40 -1703> 2014-08-13 17:38:05.426491 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426491, event: started, request: pg_notify(22.775(31) epoch 194487) v5 -1702> 2014-08-13 17:38:05.426511 7fe9a63f4700 5 -- op tracker -- , seq: 656, time: 2014-08-13 17:38:05.426511, event: done, request: pg_notify(22.775(31) epoch 194487) v5 -1701> 2014-08-13 17:38:05.426465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.331 (past last_backfill 0//0//-1) -1700> 2014-08-13 17:38:05.426519 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1699> 2014-08-13 17:38:05.426535 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1698> 2014-08-13 17:38:05.426523 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.331 (past last_backfill 0//0//-1) -1697> 2014-08-13 17:38:05.426581 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1696> 2014-08-13 17:38:05.426677 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 332 as a source -1695> 2014-08-13 17:38:05.426736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.332 -1694> 2014-08-13 17:38:05.426796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=47] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.332 -1693> 2014-08-13 17:38:05.423626 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.59 (past last_backfill 0//0//-1) -1692> 2014-08-13 17:38:05.426859 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=46] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.332 -1691> 2014-08-13 17:38:05.426924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=45] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.332 -1690> 2014-08-13 17:38:05.426988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=44] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.332 -1689> 2014-08-13 17:38:05.426892 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.59 (past last_backfill 0//0//-1) -1688> 2014-08-13 17:38:05.427056 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=43] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.332 -1687> 2014-08-13 17:38:05.427124 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=42] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.332 -1686> 2014-08-13 17:38:05.427198 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=41] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.332 -1685> 2014-08-13 17:38:05.427094 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.59 (past last_backfill 0//0//-1) -1684> 2014-08-13 17:38:05.427274 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=40] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.332 -1683> 2014-08-13 17:38:05.427350 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=39] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.332 -1682> 2014-08-13 17:38:05.427285 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.59 (past last_backfill 0//0//-1) -1681> 2014-08-13 17:38:05.427426 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=38] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.332 -1680> 2014-08-13 17:38:05.427506 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=37] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.332 -1679> 2014-08-13 17:38:05.427472 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.59 (past last_backfill 0//0//-1) -1678> 2014-08-13 17:38:05.427586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=36] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.332 -1677> 2014-08-13 17:38:05.427667 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=35] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.332 -1676> 2014-08-13 17:38:05.427751 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=34] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.332 -1675> 2014-08-13 17:38:05.427657 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.59 (past last_backfill 0//0//-1) -1674> 2014-08-13 17:38:05.427837 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=33] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.332 -1673> 2014-08-13 17:38:05.427926 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=32] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.332 -1672> 2014-08-13 17:38:05.427840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.59 (past last_backfill 0//0//-1) -1671> 2014-08-13 17:38:05.428015 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=31] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.332 -1670> 2014-08-13 17:38:05.428108 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=30] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.332 -1669> 2014-08-13 17:38:05.428023 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.59 (past last_backfill 0//0//-1) -1668> 2014-08-13 17:38:05.428205 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=29] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.332 -1667> 2014-08-13 17:38:05.428208 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.59 (past last_backfill 0//0//-1) -1666> 2014-08-13 17:38:05.428300 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=28] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.332 -1665> 2014-08-13 17:38:05.428395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=27] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.332 -1664> 2014-08-13 17:38:05.428394 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.59 (past last_backfill 0//0//-1) -1663> 2014-08-13 17:38:05.428493 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=26] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.332 -1662> 2014-08-13 17:38:05.428591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=25] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.332 -1661> 2014-08-13 17:38:05.428579 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.59 (past last_backfill 0//0//-1) -1660> 2014-08-13 17:38:05.428692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=24] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.332 -1659> 2014-08-13 17:38:05.428796 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=23] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.332 -1658> 2014-08-13 17:38:05.428762 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.59 (past last_backfill 0//0//-1) -1657> 2014-08-13 17:38:05.428901 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=22] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.332 -1656> 2014-08-13 17:38:05.429008 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=21] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.332 -1655> 2014-08-13 17:38:05.428945 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.59 (past last_backfill 0//0//-1) -1654> 2014-08-13 17:38:05.429116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=20] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.332 -1653> 2014-08-13 17:38:05.429127 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.59 (past last_backfill 0//0//-1) -1652> 2014-08-13 17:38:05.429229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=19] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.332 -1651> 2014-08-13 17:38:05.429339 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=18] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.332 -1650> 2014-08-13 17:38:05.429313 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.59 (past last_backfill 0//0//-1) -1649> 2014-08-13 17:38:05.429453 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=17] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.332 -1648> 2014-08-13 17:38:05.429497 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.59 (past last_backfill 0//0//-1) -1647> 2014-08-13 17:38:05.429568 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=16] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.332 -1646> 2014-08-13 17:38:05.429689 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=15] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.332 -1645> 2014-08-13 17:38:05.429686 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.59 (past last_backfill 0//0//-1) -1644> 2014-08-13 17:38:05.429808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=14] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.332 -1643> 2014-08-13 17:38:05.429928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=13] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.332 -1642> 2014-08-13 17:38:05.429870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.59 (past last_backfill 0//0//-1) -1641> 2014-08-13 17:38:05.430051 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=12] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.332 -1640> 2014-08-13 17:38:05.430053 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.59 (past last_backfill 0//0//-1) -1639> 2014-08-13 17:38:05.430175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=11] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.332 -1638> 2014-08-13 17:38:05.430239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.59 (past last_backfill 0//0//-1) -1637> 2014-08-13 17:38:05.430304 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=10] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.332 -1636> 2014-08-13 17:38:05.430431 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=9] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.332 -1635> 2014-08-13 17:38:05.430423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.59 (past last_backfill 0//0//-1) -1634> 2014-08-13 17:38:05.430561 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=8] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.332 -1633> 2014-08-13 17:38:05.430606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.59 (past last_backfill 0//0//-1) -1632> 2014-08-13 17:38:05.430692 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=7] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.332 -1631> 2014-08-13 17:38:05.430825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=6] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.332 -1630> 2014-08-13 17:38:05.430789 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.59 (past last_backfill 0//0//-1) -1629> 2014-08-13 17:38:05.430960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=5] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.332 -1628> 2014-08-13 17:38:05.430971 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.59 (past last_backfill 0//0//-1) -1627> 2014-08-13 17:38:05.431113 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=4] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.332 -1626> 2014-08-13 17:38:05.431154 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.59 (past last_backfill 0//0//-1) -1625> 2014-08-13 17:38:05.431255 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=3] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.332 -1624> 2014-08-13 17:38:05.431340 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.59 (past last_backfill 0//0//-1) -1623> 2014-08-13 17:38:05.431395 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=2] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.332 -1622> 2014-08-13 17:38:05.431535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=1] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.332 -1621> 2014-08-13 17:38:05.431524 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.59 (past last_backfill 0//0//-1) -1620> 2014-08-13 17:38:05.431754 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.286 192.168.1.113:6820/19173 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54df740 con 0x4df61a0 -1619> 2014-08-13 17:38:05.431772 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1618> 2014-08-13 17:38:05.431775 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1617> 2014-08-13 17:38:05.431776 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54df740 osd_map(194487..194487 src has 185139..194487) v3 -1616> 2014-08-13 17:38:05.431782 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -1615> 2014-08-13 17:38:05.431785 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -1614> 2014-08-13 17:38:05.431786 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1613> 2014-08-13 17:38:05.431787 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1612> 2014-08-13 17:38:05.431801 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.286 192.168.1.113:6820/19173 10 ==== pg_info(1 pgs e194487:24.30b) v4 ==== 724+0+0 (3090028228 0 0) 0x51cb480 con 0x4df61a0 -1611> 2014-08-13 17:38:05.431810 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1610> 2014-08-13 17:38:05.431811 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1609> 2014-08-13 17:38:05.431812 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb480 pg_info(1 pgs e194487:24.30b) v4 -1608> 2014-08-13 17:38:05.431816 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431751, event: header_read, request: pg_info(1 pgs e194487:24.30b) v4 -1607> 2014-08-13 17:38:05.431821 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431752, event: throttled, request: pg_info(1 pgs e194487:24.30b) v4 -1606> 2014-08-13 17:38:05.431825 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431789, event: all_read, request: pg_info(1 pgs e194487:24.30b) v4 -1605> 2014-08-13 17:38:05.431828 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431810, event: dispatched, request: pg_info(1 pgs e194487:24.30b) v4 -1604> 2014-08-13 17:38:05.431832 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431832, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.30b) v4 -1603> 2014-08-13 17:38:05.431836 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.286 -1602> 2014-08-13 17:38:05.431839 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb480 -1601> 2014-08-13 17:38:05.431842 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431842, event: started, request: pg_info(1 pgs e194487:24.30b) v4 -1600> 2014-08-13 17:38:05.431678 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1599> 2014-08-13 17:38:05.431860 7fe9a63f4700 5 -- op tracker -- , seq: 657, time: 2014-08-13 17:38:05.431859, event: done, request: pg_info(1 pgs e194487:24.30b) v4 -1598> 2014-08-13 17:38:05.431868 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1597> 2014-08-13 17:38:05.431870 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1596> 2014-08-13 17:38:05.431707 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.59 (past last_backfill 0//0//-1) -1595> 2014-08-13 17:38:05.431856 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 337 as a source -1594> 2014-08-13 17:38:05.431889 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.59 (past last_backfill 0//0//-1) -1593> 2014-08-13 17:38:05.431999 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.337 -1592> 2014-08-13 17:38:05.432073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.59 (past last_backfill 0//0//-1) -1591> 2014-08-13 17:38:05.432145 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.337 -1590> 2014-08-13 17:38:05.432293 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.337 -1589> 2014-08-13 17:38:05.432259 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.59 (past last_backfill 0//0//-1) -1588> 2014-08-13 17:38:05.432437 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.337 -1587> 2014-08-13 17:38:05.432441 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.59 (past last_backfill 0//0//-1) -1586> 2014-08-13 17:38:05.432582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.337 -1585> 2014-08-13 17:38:05.432624 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.59 (past last_backfill 0//0//-1) -1584> 2014-08-13 17:38:05.432727 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.337 -1583> 2014-08-13 17:38:05.432808 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.59 (past last_backfill 0//0//-1) -1582> 2014-08-13 17:38:05.432871 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.337 -1581> 2014-08-13 17:38:05.433015 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.337 -1580> 2014-08-13 17:38:05.432991 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.59 (past last_backfill 0//0//-1) -1579> 2014-08-13 17:38:05.433233 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.10 192.168.1.115:6815/60573 11 ==== pg_info(1 pgs e194487:24.30b) v4 ==== 748+0+0 (287331700 0 0) 0x51cb800 con 0x49e09a0 -1578> 2014-08-13 17:38:05.433252 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1577> 2014-08-13 17:38:05.433254 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1576> 2014-08-13 17:38:05.433256 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb800 pg_info(1 pgs e194487:24.30b) v4 -1575> 2014-08-13 17:38:05.433260 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433163, event: header_read, request: pg_info(1 pgs e194487:24.30b) v4 -1574> 2014-08-13 17:38:05.433266 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433164, event: throttled, request: pg_info(1 pgs e194487:24.30b) v4 -1573> 2014-08-13 17:38:05.433269 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433213, event: all_read, request: pg_info(1 pgs e194487:24.30b) v4 -1572> 2014-08-13 17:38:05.433273 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433251, event: dispatched, request: pg_info(1 pgs e194487:24.30b) v4 -1571> 2014-08-13 17:38:05.433277 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433277, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.30b) v4 -1570> 2014-08-13 17:38:05.433286 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.30b) v4 from osd.10 -1569> 2014-08-13 17:38:05.433289 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb800 -1568> 2014-08-13 17:38:05.433293 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433292, event: started, request: pg_info(1 pgs e194487:24.30b) v4 -1567> 2014-08-13 17:38:05.433160 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.337 -1566> 2014-08-13 17:38:05.433311 7fe9a63f4700 5 -- op tracker -- , seq: 658, time: 2014-08-13 17:38:05.433310, event: done, request: pg_info(1 pgs e194487:24.30b) v4 -1565> 2014-08-13 17:38:05.433320 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1564> 2014-08-13 17:38:05.433321 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1563> 2014-08-13 17:38:05.433173 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.59 (past last_backfill 0//0//-1) -1562> 2014-08-13 17:38:05.433306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.337 -1561> 2014-08-13 17:38:05.433357 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.59 (past last_backfill 0//0//-1) -1560> 2014-08-13 17:38:05.433451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.337 -1559> 2014-08-13 17:38:05.433541 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.59 (past last_backfill 0//0//-1) -1558> 2014-08-13 17:38:05.433596 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.337 -1557> 2014-08-13 17:38:05.433739 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.337 -1556> 2014-08-13 17:38:05.433725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.59 (past last_backfill 0//0//-1) -1555> 2014-08-13 17:38:05.433883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.337 -1554> 2014-08-13 17:38:05.433908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.59 (past last_backfill 0//0//-1) -1553> 2014-08-13 17:38:05.434027 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.337 -1552> 2014-08-13 17:38:05.434092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.59 (past last_backfill 0//0//-1) -1551> 2014-08-13 17:38:05.434172 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.337 -1550> 2014-08-13 17:38:05.434278 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.59 (past last_backfill 0//0//-1) -1549> 2014-08-13 17:38:05.434319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.337 -1548> 2014-08-13 17:38:05.434463 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.337 -1547> 2014-08-13 17:38:05.434462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.59 (past last_backfill 0//0//-1) -1546> 2014-08-13 17:38:05.434607 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.337 -1545> 2014-08-13 17:38:05.434645 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.59 (past last_backfill 0//0//-1) -1544> 2014-08-13 17:38:05.434752 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.337 -1543> 2014-08-13 17:38:05.434829 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.59 (past last_backfill 0//0//-1) -1542> 2014-08-13 17:38:05.434897 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.337 -1541> 2014-08-13 17:38:05.435041 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.337 -1540> 2014-08-13 17:38:05.435012 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.59 (past last_backfill 0//0//-1) -1539> 2014-08-13 17:38:05.435190 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.337 -1538> 2014-08-13 17:38:05.435197 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.59 (past last_backfill 0//0//-1) -1537> 2014-08-13 17:38:05.435336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.337 -1536> 2014-08-13 17:38:05.435381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.59 (past last_backfill 0//0//-1) -1535> 2014-08-13 17:38:05.435480 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.337 -1534> 2014-08-13 17:38:05.435565 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.59 (past last_backfill 0//0//-1) -1533> 2014-08-13 17:38:05.435625 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.337 -1532> 2014-08-13 17:38:05.435768 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.337 -1531> 2014-08-13 17:38:05.435748 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.59 (past last_backfill 0//0//-1) -1530> 2014-08-13 17:38:05.435911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.337 -1529> 2014-08-13 17:38:05.435930 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.59 (past last_backfill 0//0//-1) -1528> 2014-08-13 17:38:05.436054 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.337 -1527> 2014-08-13 17:38:05.436114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.59 (past last_backfill 0//0//-1) -1526> 2014-08-13 17:38:05.436202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.337 -1525> 2014-08-13 17:38:05.436301 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.59 (past last_backfill 0//0//-1) -1524> 2014-08-13 17:38:05.436347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.337 -1523> 2014-08-13 17:38:05.436490 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.337 -1522> 2014-08-13 17:38:05.436485 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.59 (past last_backfill 0//0//-1) -1521> 2014-08-13 17:38:05.436634 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.337 -1520> 2014-08-13 17:38:05.436669 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.59 (past last_backfill 0//0//-1) -1519> 2014-08-13 17:38:05.436779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.337 -1518> 2014-08-13 17:38:05.436852 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.59 (past last_backfill 0//0//-1) -1517> 2014-08-13 17:38:05.436923 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.337 -1516> 2014-08-13 17:38:05.437068 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.337 -1515> 2014-08-13 17:38:05.437036 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.59 (past last_backfill 0//0//-1) -1514> 2014-08-13 17:38:05.437215 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.337 -1513> 2014-08-13 17:38:05.437220 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.59 (past last_backfill 0//0//-1) -1512> 2014-08-13 17:38:05.437359 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.337 -1511> 2014-08-13 17:38:05.437404 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.59 (past last_backfill 0//0//-1) -1510> 2014-08-13 17:38:05.437504 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.337 -1509> 2014-08-13 17:38:05.437648 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.337 -1508> 2014-08-13 17:38:05.437588 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -1507> 2014-08-13 17:38:05.437792 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.337 -1506> 2014-08-13 17:38:05.437818 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 351 as a source -1505> 2014-08-13 17:38:05.437937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.337 -1504> 2014-08-13 17:38:05.438003 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.351 -1503> 2014-08-13 17:38:05.438082 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.337 -1502> 2014-08-13 17:38:05.438192 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.351 -1501> 2014-08-13 17:38:05.438247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.337 -1500> 2014-08-13 17:38:05.438391 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.337 -1499> 2014-08-13 17:38:05.438376 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.351 -1498> 2014-08-13 17:38:05.438536 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.337 -1497> 2014-08-13 17:38:05.438560 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.351 -1496> 2014-08-13 17:38:05.438681 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.337 -1495> 2014-08-13 17:38:05.438744 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.351 -1494> 2014-08-13 17:38:05.438825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.337 -1493> 2014-08-13 17:38:05.438929 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.351 -1492> 2014-08-13 17:38:05.438969 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1491> 2014-08-13 17:38:05.439150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 348 as a source -1490> 2014-08-13 17:38:05.439113 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.351 -1489> 2014-08-13 17:38:05.439297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.348 (past last_backfill 0//0//-1) -1488> 2014-08-13 17:38:05.439299 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.351 -1487> 2014-08-13 17:38:05.439442 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.348 (past last_backfill 0//0//-1) -1486> 2014-08-13 17:38:05.439483 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.351 -1485> 2014-08-13 17:38:05.439593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.348 (past last_backfill 0//0//-1) -1484> 2014-08-13 17:38:05.439668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.351 -1483> 2014-08-13 17:38:05.439740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.348 (past last_backfill 0//0//-1) -1482> 2014-08-13 17:38:05.439883 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.348 (past last_backfill 0//0//-1) -1481> 2014-08-13 17:38:05.439870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.351 -1480> 2014-08-13 17:38:05.440026 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.348 (past last_backfill 0//0//-1) -1479> 2014-08-13 17:38:05.440055 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.351 -1478> 2014-08-13 17:38:05.440170 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.348 (past last_backfill 0//0//-1) -1477> 2014-08-13 17:38:05.440242 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.351 -1476> 2014-08-13 17:38:05.440317 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.348 (past last_backfill 0//0//-1) -1475> 2014-08-13 17:38:05.440461 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.348 (past last_backfill 0//0//-1) -1474> 2014-08-13 17:38:05.440427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.351 -1473> 2014-08-13 17:38:05.440604 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.348 (past last_backfill 0//0//-1) -1472> 2014-08-13 17:38:05.440610 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.351 -1471> 2014-08-13 17:38:05.440748 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.348 (past last_backfill 0//0//-1) -1470> 2014-08-13 17:38:05.440794 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.351 -1469> 2014-08-13 17:38:05.440892 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.348 (past last_backfill 0//0//-1) -1468> 2014-08-13 17:38:05.440978 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.351 -1467> 2014-08-13 17:38:05.441036 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.348 (past last_backfill 0//0//-1) -1466> 2014-08-13 17:38:05.441178 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.348 (past last_backfill 0//0//-1) -1465> 2014-08-13 17:38:05.441162 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.351 -1464> 2014-08-13 17:38:05.441324 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.348 (past last_backfill 0//0//-1) -1463> 2014-08-13 17:38:05.441349 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.351 -1462> 2014-08-13 17:38:05.441468 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.348 (past last_backfill 0//0//-1) -1461> 2014-08-13 17:38:05.441534 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.351 -1460> 2014-08-13 17:38:05.441612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.348 (past last_backfill 0//0//-1) -1459> 2014-08-13 17:38:05.442021 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.348 (past last_backfill 0//0//-1) -1458> 2014-08-13 17:38:05.441718 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.351 -1457> 2014-08-13 17:38:05.442168 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.348 (past last_backfill 0//0//-1) -1456> 2014-08-13 17:38:05.442176 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.351 -1455> 2014-08-13 17:38:05.442315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.348 (past last_backfill 0//0//-1) -1454> 2014-08-13 17:38:05.442365 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.351 -1453> 2014-08-13 17:38:05.442459 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.348 (past last_backfill 0//0//-1) -1452> 2014-08-13 17:38:05.442549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.351 -1451> 2014-08-13 17:38:05.442602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.348 (past last_backfill 0//0//-1) -1450> 2014-08-13 17:38:05.442745 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.348 (past last_backfill 0//0//-1) -1449> 2014-08-13 17:38:05.442733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.351 -1448> 2014-08-13 17:38:05.442888 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.348 (past last_backfill 0//0//-1) -1447> 2014-08-13 17:38:05.442917 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.351 -1446> 2014-08-13 17:38:05.443032 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.348 (past last_backfill 0//0//-1) -1445> 2014-08-13 17:38:05.443101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.351 -1444> 2014-08-13 17:38:05.443175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.348 (past last_backfill 0//0//-1) -1443> 2014-08-13 17:38:05.443322 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.348 (past last_backfill 0//0//-1) -1442> 2014-08-13 17:38:05.443288 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.351 -1441> 2014-08-13 17:38:05.443465 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.348 (past last_backfill 0//0//-1) -1440> 2014-08-13 17:38:05.443471 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.351 -1439> 2014-08-13 17:38:05.443610 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.348 (past last_backfill 0//0//-1) -1438> 2014-08-13 17:38:05.443656 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.351 -1437> 2014-08-13 17:38:05.443754 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.348 (past last_backfill 0//0//-1) -1436> 2014-08-13 17:38:05.443840 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.351 -1435> 2014-08-13 17:38:05.443897 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.348 (past last_backfill 0//0//-1) -1434> 2014-08-13 17:38:05.444042 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.348 (past last_backfill 0//0//-1) -1433> 2014-08-13 17:38:05.444024 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.351 -1432> 2014-08-13 17:38:05.444188 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.348 (past last_backfill 0//0//-1) -1431> 2014-08-13 17:38:05.444211 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.351 -1430> 2014-08-13 17:38:05.444332 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.348 (past last_backfill 0//0//-1) -1429> 2014-08-13 17:38:05.444396 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.351 -1428> 2014-08-13 17:38:05.444476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.348 (past last_backfill 0//0//-1) -1427> 2014-08-13 17:38:05.444580 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.351 -1426> 2014-08-13 17:38:05.444620 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.348 (past last_backfill 0//0//-1) -1425> 2014-08-13 17:38:05.444766 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.348 (past last_backfill 0//0//-1) -1424> 2014-08-13 17:38:05.444765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.351 -1423> 2014-08-13 17:38:05.444911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.348 (past last_backfill 0//0//-1) -1422> 2014-08-13 17:38:05.444951 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.351 -1421> 2014-08-13 17:38:05.445055 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.348 (past last_backfill 0//0//-1) -1420> 2014-08-13 17:38:05.445135 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.351 -1419> 2014-08-13 17:38:05.445202 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.348 (past last_backfill 0//0//-1) -1418> 2014-08-13 17:38:05.445322 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.351 -1417> 2014-08-13 17:38:05.445419 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.348 (past last_backfill 0//0//-1) -1416> 2014-08-13 17:38:05.445507 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.351 -1415> 2014-08-13 17:38:05.445588 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.348 (past last_backfill 0//0//-1) -1414> 2014-08-13 17:38:05.445691 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.351 -1413> 2014-08-13 17:38:05.445732 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.348 (past last_backfill 0//0//-1) -1412> 2014-08-13 17:38:05.445875 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.348 (past last_backfill 0//0//-1) -1411> 2014-08-13 17:38:05.445875 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.351 -1410> 2014-08-13 17:38:05.446020 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.348 (past last_backfill 0//0//-1) -1409> 2014-08-13 17:38:05.446061 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.351 -1408> 2014-08-13 17:38:05.446163 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.348 (past last_backfill 0//0//-1) -1407> 2014-08-13 17:38:05.446248 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.351 -1406> 2014-08-13 17:38:05.446311 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.348 (past last_backfill 0//0//-1) -1405> 2014-08-13 17:38:05.446555 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.26 192.168.1.115:6802/58522 6 ==== pg_info(1 pgs e194487:24.34) v4 ==== 740+0+0 (2285091063 0 0) 0x51055c0 con 0x592c360 -1404> 2014-08-13 17:38:05.446576 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1403> 2014-08-13 17:38:05.446579 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1402> 2014-08-13 17:38:05.446580 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51055c0 pg_info(1 pgs e194487:24.34) v4 -1401> 2014-08-13 17:38:05.446584 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446480, event: header_read, request: pg_info(1 pgs e194487:24.34) v4 -1400> 2014-08-13 17:38:05.446590 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446481, event: throttled, request: pg_info(1 pgs e194487:24.34) v4 -1399> 2014-08-13 17:38:05.446594 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446534, event: all_read, request: pg_info(1 pgs e194487:24.34) v4 -1398> 2014-08-13 17:38:05.446456 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.348 (past last_backfill 0//0//-1) -1397> 2014-08-13 17:38:05.446597 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446574, event: dispatched, request: pg_info(1 pgs e194487:24.34) v4 -1396> 2014-08-13 17:38:05.446602 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446602, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.34) v4 -1395> 2014-08-13 17:38:05.446606 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.34) v4 from osd.26 -1394> 2014-08-13 17:38:05.446609 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51055c0 -1393> 2014-08-13 17:38:05.446432 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.351 -1392> 2014-08-13 17:38:05.446612 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446612, event: started, request: pg_info(1 pgs e194487:24.34) v4 -1391> 2014-08-13 17:38:05.446629 7fe9a63f4700 5 -- op tracker -- , seq: 659, time: 2014-08-13 17:38:05.446629, event: done, request: pg_info(1 pgs e194487:24.34) v4 -1390> 2014-08-13 17:38:05.446639 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1389> 2014-08-13 17:38:05.446640 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1388> 2014-08-13 17:38:05.446598 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1387> 2014-08-13 17:38:05.446615 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.351 -1386> 2014-08-13 17:38:05.446777 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound -1385> 2014-08-13 17:38:05.446798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.351 -1384> 2014-08-13 17:38:05.446921 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _calc_past_interval_range: already have past intervals back to 194484 -1383> 2014-08-13 17:38:05.446982 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.351 -1382> 2014-08-13 17:38:05.447067 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound: built 331,332,337,348,379 -1381> 2014-08-13 17:38:05.447166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.351 -1380> 2014-08-13 17:38:05.447216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Activate Finished -1379> 2014-08-13 17:38:05.447361 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating -1378> 2014-08-13 17:38:05.447353 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.351 -1377> 2014-08-13 17:38:05.447507 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1376> 2014-08-13 17:38:05.447538 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.351 -1375> 2014-08-13 17:38:05.447652 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1374> 2014-08-13 17:38:05.447852 7fe99ebe8700 10 log is not dirty -1373> 2014-08-13 17:38:05.447722 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.351 -1372> 2014-08-13 17:38:05.447935 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [396,368,4]/[396,363] -- 396/396 -1371> 2014-08-13 17:38:05.447907 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.351 -1370> 2014-08-13 17:38:05.448110 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1369> 2014-08-13 17:38:05.448119 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap -1368> 2014-08-13 17:38:05.448136 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down -1367> 2014-08-13 17:38:05.448148 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map -1366> 2014-08-13 17:38:05.448159 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters -1365> 2014-08-13 17:38:05.448215 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1364> 2014-08-13 17:38:05.448227 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194485 pi=194462-194484/8 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1363> 2014-08-13 17:38:05.448238 7fe99ebe8700 10 log is not dirty -1362> 2014-08-13 17:38:05.448092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.351 -1361> 2014-08-13 17:38:05.448259 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 -1360> 2014-08-13 17:38:05.448290 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1359> 2014-08-13 17:38:05.448295 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -1358> 2014-08-13 17:38:05.448305 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -1357> 2014-08-13 17:38:05.448315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -1356> 2014-08-13 17:38:05.448325 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -1355> 2014-08-13 17:38:05.448334 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -1354> 2014-08-13 17:38:05.448342 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -1353> 2014-08-13 17:38:05.448381 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:2653 -1352> 2014-08-13 17:38:05.448390 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] take_waiters -1351> 2014-08-13 17:38:05.448398 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1350> 2014-08-13 17:38:05.448407 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194485 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1349> 2014-08-13 17:38:05.448417 7fe99ebe8700 10 log is not dirty -1348> 2014-08-13 17:38:05.448434 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 -1347> 2014-08-13 17:38:05.448447 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1346> 2014-08-13 17:38:05.448451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap -1345> 2014-08-13 17:38:05.448279 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.351 -1344> 2014-08-13 17:38:05.448298 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -1343> 2014-08-13 17:38:05.448461 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down -1342> 2014-08-13 17:38:05.448472 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map -1341> 2014-08-13 17:38:05.448481 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters -1340> 2014-08-13 17:38:05.448489 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1339> 2014-08-13 17:38:05.448498 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1338> 2014-08-13 17:38:05.448507 7fe99ebe8700 10 log is not dirty -1337> 2014-08-13 17:38:05.448525 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 -1336> 2014-08-13 17:38:05.448536 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1335> 2014-08-13 17:38:05.448539 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -1334> 2014-08-13 17:38:05.448548 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -1333> 2014-08-13 17:38:05.448557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -1332> 2014-08-13 17:38:05.448566 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -1331> 2014-08-13 17:38:05.448574 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -1330> 2014-08-13 17:38:05.448581 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -1329> 2014-08-13 17:38:05.448590 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:3000 -1328> 2014-08-13 17:38:05.448598 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] take_waiters -1327> 2014-08-13 17:38:05.448606 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1326> 2014-08-13 17:38:05.448614 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194485 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1325> 2014-08-13 17:38:05.448623 7fe99ebe8700 10 log is not dirty -1324> 2014-08-13 17:38:05.448485 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] flushed -1323> 2014-08-13 17:38:05.448462 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.351 -1322> 2014-08-13 17:38:05.448636 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 -1321> 2014-08-13 17:38:05.448649 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1320> 2014-08-13 17:38:05.448652 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap -1319> 2014-08-13 17:38:05.448662 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down -1318> 2014-08-13 17:38:05.448671 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map -1317> 2014-08-13 17:38:05.448680 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters -1316> 2014-08-13 17:38:05.448688 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1315> 2014-08-13 17:38:05.448696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194485 pi=192984-194484/24 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1314> 2014-08-13 17:38:05.448705 7fe99ebe8700 10 log is not dirty -1313> 2014-08-13 17:38:05.448719 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 -1312> 2014-08-13 17:38:05.448736 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1311> 2014-08-13 17:38:05.448740 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Started advmap -1310> 2014-08-13 17:38:05.448749 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down -1309> 2014-08-13 17:38:05.448759 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map -1308> 2014-08-13 17:38:05.448809 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters -1307> 2014-08-13 17:38:05.448819 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1306> 2014-08-13 17:38:05.448646 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.351 -1305> 2014-08-13 17:38:05.448827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1304> 2014-08-13 17:38:05.448838 7fe99ebe8700 10 log is not dirty -1303> 2014-08-13 17:38:05.448852 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 -1302> 2014-08-13 17:38:05.448864 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1301> 2014-08-13 17:38:05.448868 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap -1300> 2014-08-13 17:38:05.448877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down -1299> 2014-08-13 17:38:05.448886 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map -1298> 2014-08-13 17:38:05.448895 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters -1297> 2014-08-13 17:38:05.448903 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1296> 2014-08-13 17:38:05.448911 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194485 pi=192975-194484/29 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1295> 2014-08-13 17:38:05.448921 7fe99ebe8700 10 log is not dirty -1294> 2014-08-13 17:38:05.448933 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 -1293> 2014-08-13 17:38:05.448946 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1292> 2014-08-13 17:38:05.448949 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -1291> 2014-08-13 17:38:05.448959 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -1290> 2014-08-13 17:38:05.448969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,338) went down -1289> 2014-08-13 17:38:05.448979 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -1288> 2014-08-13 17:38:05.448987 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -1287> 2014-08-13 17:38:05.448829 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.351 -1286> 2014-08-13 17:38:05.449163 7fe99ebe8700 10 osd.4 194487 queue_for_recovery queued pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] -1285> 2014-08-13 17:38:05.449171 7fe99ebe8700 7 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -1284> 2014-08-13 17:38:05.449012 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.351 -1283> 2014-08-13 17:38:05.449263 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194487:2596 -1282> 2014-08-13 17:38:05.449273 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -1281> 2014-08-13 17:38:05.449282 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1280> 2014-08-13 17:38:05.449291 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1279> 2014-08-13 17:38:05.449311 7fe99ebe8700 10 log is not dirty -1278> 2014-08-13 17:38:05.449326 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 -1277> 2014-08-13 17:38:05.449343 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1276> 2014-08-13 17:38:05.449347 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Started advmap -1275> 2014-08-13 17:38:05.449357 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down -1274> 2014-08-13 17:38:05.449367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map -1273> 2014-08-13 17:38:05.449198 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.351 -1272> 2014-08-13 17:38:05.449446 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters -1271> 2014-08-13 17:38:05.449456 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1270> 2014-08-13 17:38:05.449466 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1269> 2014-08-13 17:38:05.449476 7fe99ebe8700 10 log is not dirty -1268> 2014-08-13 17:38:05.449491 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 -1267> 2014-08-13 17:38:05.449503 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1266> 2014-08-13 17:38:05.449507 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap -1265> 2014-08-13 17:38:05.449516 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down -1264> 2014-08-13 17:38:05.449525 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map -1263> 2014-08-13 17:38:05.449534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters -1262> 2014-08-13 17:38:05.449542 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1261> 2014-08-13 17:38:05.449381 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.351 -1260> 2014-08-13 17:38:05.449559 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194485 pi=192975-194484/25 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1259> 2014-08-13 17:38:05.449570 7fe99ebe8700 10 log is not dirty -1258> 2014-08-13 17:38:05.449584 7fe99ebe8700 10 osd.4 pg_epoch: 194486 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 -1257> 2014-08-13 17:38:05.449600 7fe99ebe8700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1256> 2014-08-13 17:38:05.449603 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Started advmap -1255> 2014-08-13 17:38:05.449613 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down -1254> 2014-08-13 17:38:05.449622 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map -1253> 2014-08-13 17:38:05.449663 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters -1252> 2014-08-13 17:38:05.449673 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1251> 2014-08-13 17:38:05.449681 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1250> 2014-08-13 17:38:05.449691 7fe99ebe8700 10 log is not dirty -1249> 2014-08-13 17:38:05.449696 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 61 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -1248> 2014-08-13 17:38:05.449710 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring -1247> 2014-08-13 17:38:05.449720 7fe99ebe8700 10 log is not dirty -1246> 2014-08-13 17:38:05.449726 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4e3d800 192.168.1.107:6809/53141 -1245> 2014-08-13 17:38:05.449737 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54dc5c0 con 0x4e3d800 -1244> 2014-08-13 17:38:05.449564 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.351 -1243> 2014-08-13 17:38:05.449749 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.6 has 194487 -1242> 2014-08-13 17:38:05.449751 7fe99ebe8700 7 osd.4 194487 do_notify osd 6 on 1 PGs -1241> 2014-08-13 17:38:05.449753 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(31) epoch 194487) v5 -- ?+0 0x5f01500 con 0x4e3d800 -1240> 2014-08-13 17:38:05.449838 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing -1239> 2014-08-13 17:38:05.449844 7fe99ebe8700 7 osd.4 194487 do_notify osd 45 on 1 PGs -1238> 2014-08-13 17:38:05.449850 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(31) epoch 194487) v5 -- ?+0 0x513dcc0 con 0x4d39080 -1237> 2014-08-13 17:38:05.449878 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -1236> 2014-08-13 17:38:05.449880 7fe99ebe8700 7 osd.4 194487 do_notify osd 285 on 1 PGs -1235> 2014-08-13 17:38:05.449881 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(31) epoch 194487) v5 -- ?+0 0x513e3c0 con 0x4a45280 -1234> 2014-08-13 17:38:05.449893 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4d31e40 192.168.1.104:6802/61960 -1233> 2014-08-13 17:38:05.449930 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x54dca40 con 0x4d31e40 -1232> 2014-08-13 17:38:05.449945 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.343 has 194487 -1231> 2014-08-13 17:38:05.449947 7fe99ebe8700 7 osd.4 194487 do_notify osd 343 on 1 PGs -1230> 2014-08-13 17:38:05.449949 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(31) epoch 194487) v5 -- ?+0 0x513d5c0 con 0x4d31e40 -1229> 2014-08-13 17:38:05.449748 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -1228> 2014-08-13 17:38:05.450004 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing -1227> 2014-08-13 17:38:05.450011 7fe99ebe8700 7 osd.4 194487 do_notify osd 374 on 1 PGs -1226> 2014-08-13 17:38:05.450014 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(48) epoch 194487) v5 -- ?+0 0x513e900 con 0x4c75120 -1225> 2014-08-13 17:38:05.450029 7fe99ebe8700 20 osd.4 194487 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 332 -1224> 2014-08-13 17:38:05.450037 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing -1223> 2014-08-13 17:38:05.450056 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.110:6806/42024 -- pg_info(1 pgs e194487:24.1bb(32)) v4 -- ?+0 0x5825780 con 0x4e3b020 -1222> 2014-08-13 17:38:05.450068 7fe99ebe8700 20 osd.4 194487 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) to shard 337 -1221> 2014-08-13 17:38:05.450075 7fe99ebe8700 10 osd.4 194487 send_incremental_map 194486 -> 194487 to 0x4d3c780 192.168.1.104:6804/62197 -1220> 2014-08-13 17:38:05.450083 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- osd_map(194487..194487 src has 185139..194487) v3 -- ?+0 0x549d100 con 0x4d3c780 -1219> 2014-08-13 17:38:05.450093 7fe99ebe8700 10 osd.4 194487 note_peer_epoch osd.337 has 194487 -1218> 2014-08-13 17:38:05.450104 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> 192.168.1.104:6804/62197 -- pg_info(1 pgs e194487:24.1bb(32)) v4 -- ?+0 0x5820e00 con 0x4d3c780 -1217> 2014-08-13 17:38:05.450000 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound -1216> 2014-08-13 17:38:05.450213 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 61 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -1215> 2014-08-13 17:38:05.450238 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring -1214> 2014-08-13 17:38:05.450250 7fe99ebe8700 10 log is not dirty -1213> 2014-08-13 17:38:05.450257 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 45 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -1212> 2014-08-13 17:38:05.450270 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 45, already have info from that osd, ignoring -1211> 2014-08-13 17:38:05.450279 7fe99ebe8700 10 log is not dirty -1210> 2014-08-13 17:38:05.450284 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) -1209> 2014-08-13 17:38:05.450297 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: got info from osd.304 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) -1208> 2014-08-13 17:38:05.450312 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 1.135807 8 0.009620 -1207> 2014-08-13 17:38:05.450327 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive -1206> 2014-08-13 17:38:05.450355 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering -1205> 2014-08-13 17:38:05.450186 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _calc_past_interval_range: already have past intervals back to 194484 -1204> 2014-08-13 17:38:05.450367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194477 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: In ReplicaActive, about to call activate -1203> 2014-08-13 17:38:05.450417 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] activate - not complete, missing(61) -1202> 2014-08-13 17:38:05.450370 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound: built 55,59,351,376 -1201> 2014-08-13 17:38:05.450552 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Activate Finished -1200> 2014-08-13 17:38:05.450736 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating -1199> 2014-08-13 17:38:05.450919 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -1198> 2014-08-13 17:38:05.451101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1197> 2014-08-13 17:38:05.451313 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] state: Activate Finished -1196> 2014-08-13 17:38:05.451344 7fe99f5e9700 10 log is not dirty -1195> 2014-08-13 17:38:05.451379 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 -1194> 2014-08-13 17:38:05.451399 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1193> 2014-08-13 17:38:05.451405 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Peering advmap -1192> 2014-08-13 17:38:05.451415 7fe99ebe8700 10 log is not dirty -1191> 2014-08-13 17:38:05.451417 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] adjust_need_up_thru now 194486, need_up_thru now false -1190> 2014-08-13 17:38:05.451427 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Started advmap -1189> 2014-08-13 17:38:05.451433 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) -1188> 2014-08-13 17:38:05.451440 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] check_recovery_sources no source osds () went down -1187> 2014-08-13 17:38:05.451449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: got info from osd.6 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) -1186> 2014-08-13 17:38:05.451452 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map -1185> 2014-08-13 17:38:05.451463 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 1.040760 9 0.000549 -1184> 2014-08-13 17:38:05.451464 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle ActMap primary -1183> 2014-08-13 17:38:05.451475 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive -1182> 2014-08-13 17:38:05.451476 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194487:22526 -1181> 2014-08-13 17:38:05.451487 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering -1180> 2014-08-13 17:38:05.451487 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] take_waiters -1179> 2014-08-13 17:38:05.451498 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194477 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: In ReplicaActive, about to call activate -1178> 2014-08-13 17:38:05.451500 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 1.026821 6 0.000961 -1177> 2014-08-13 17:38:05.451508 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] activate - not complete, missing(53) -1176> 2014-08-13 17:38:05.451514 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Leaving Peering -1175> 2014-08-13 17:38:05.451525 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 1.096242 0 0.000000 -1174> 2014-08-13 17:38:05.451540 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active -1173> 2014-08-13 17:38:05.451550 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194478 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: In Active, about to call activate -1172> 2014-08-13 17:38:05.451563 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - purged_snaps [] cached_removed_snaps [] -1171> 2014-08-13 17:38:05.451574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - snap_trimq [] -1170> 2014-08-13 17:38:05.451583 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - not complete, missing(51) -1169> 2014-08-13 17:38:05.451656 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 -1168> 2014-08-13 17:38:05.451860 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194486 pi=193678-194485/35 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] flushed -1167> 2014-08-13 17:38:05.452059 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] _activate_committed 194487 telling primary -1166> 2014-08-13 17:38:05.452076 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x49e39c0 already has epoch 194487 -1165> 2014-08-13 17:38:05.452079 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194487:24.33a) v4 -- ?+0 0x4c163c0 -1164> 2014-08-13 17:38:05.452096 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] flushed -1163> 2014-08-13 17:38:05.452379 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - starting recovery -1162> 2014-08-13 17:38:05.452396 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] -1161> 2014-08-13 17:38:05.452403 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.332 192.168.1.110:6806/42024 9 ==== pg_info(1 pgs e194487:24.1bb) v4 ==== 748+0+0 (1237337267 0 0) 0x5825780 con 0x4e3b020 -1160> 2014-08-13 17:38:05.452420 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1159> 2014-08-13 17:38:05.452423 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1158> 2014-08-13 17:38:05.452425 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825780 pg_info(1 pgs e194487:24.1bb) v4 -1157> 2014-08-13 17:38:05.452429 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452308, event: header_read, request: pg_info(1 pgs e194487:24.1bb) v4 -1156> 2014-08-13 17:38:05.452423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -1155> 2014-08-13 17:38:05.452435 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452309, event: throttled, request: pg_info(1 pgs e194487:24.1bb) v4 -1154> 2014-08-13 17:38:05.452439 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452381, event: all_read, request: pg_info(1 pgs e194487:24.1bb) v4 -1153> 2014-08-13 17:38:05.452442 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452419, event: dispatched, request: pg_info(1 pgs e194487:24.1bb) v4 -1152> 2014-08-13 17:38:05.452437 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 is up to date, queueing in pending_activators -1151> 2014-08-13 17:38:05.452447 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452447, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1bb) v4 -1150> 2014-08-13 17:38:05.452451 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.332 -1149> 2014-08-13 17:38:05.452454 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825780 -1148> 2014-08-13 17:38:05.452458 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452457, event: started, request: pg_info(1 pgs e194487:24.1bb) v4 -1147> 2014-08-13 17:38:05.452474 7fe9a63f4700 5 -- op tracker -- , seq: 660, time: 2014-08-13 17:38:05.452474, event: done, request: pg_info(1 pgs e194487:24.1bb) v4 -1146> 2014-08-13 17:38:05.452483 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1145> 2014-08-13 17:38:05.452485 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1144> 2014-08-13 17:38:05.452478 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1143> 2014-08-13 17:38:05.452491 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1142> 2014-08-13 17:38:05.452512 7fe99f5e9700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 -1141> 2014-08-13 17:38:05.452735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 sending log((192834'12379,194422'15470], crt=194422'15470) -1140> 2014-08-13 17:38:05.452754 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4d3dee0 don't know epoch, doing nothing -1139> 2014-08-13 17:38:05.452757 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.55 192.168.1.112:6834/6966 -- pg_log(24.136 epoch 194487 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194487) v4 -- ?+0 0x5fb7200 -1138> 2014-08-13 17:38:05.452774 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1137> 2014-08-13 17:38:05.452788 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1136> 2014-08-13 17:38:05.452803 7fe99f5e9700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 -1135> 2014-08-13 17:38:05.453032 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 sending log((192834'12379,194422'15470], crt=194422'15470) -1134> 2014-08-13 17:38:05.453051 7fe99f5e9700 20 osd.4 194487 _share_map_outgoing 0x4cf86e0 don't know epoch, doing nothing -1133> 2014-08-13 17:38:05.453053 7fe99f5e9700 1 -- 192.168.1.115:6813/34805 --> osd.61 192.168.1.111:6840/16566 -- pg_log(24.136 epoch 194487 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194487) v4 -- ?+0 0x5fb0000 -1132> 2014-08-13 17:38:05.453067 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1131> 2014-08-13 17:38:05.453081 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) -1130> 2014-08-13 17:38:05.453093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 is up to date, queueing in pending_activators -1129> 2014-08-13 17:38:05.453128 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194484 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) uptodate -1128> 2014-08-13 17:38:05.453166 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery primary has 51 missing -1127> 2014-08-13 17:38:05.453226 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] state: Activate Finished -1126> 2014-08-13 17:38:05.453237 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.4 -1125> 2014-08-13 17:38:05.453304 7fe99ebe8700 10 log is not dirty -1124> 2014-08-13 17:38:05.453326 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) -1123> 2014-08-13 17:38:05.453344 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: got info from osd.17 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) -1122> 2014-08-13 17:38:05.453303 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.4 -1121> 2014-08-13 17:38:05.453358 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 1.117572 8 0.000660 -1120> 2014-08-13 17:38:05.453370 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive -1119> 2014-08-13 17:38:05.453382 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering -1118> 2014-08-13 17:38:05.453392 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194477 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: In ReplicaActive, about to call activate -1117> 2014-08-13 17:38:05.453401 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] activate - not complete, missing(49) -1116> 2014-08-13 17:38:05.453366 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.4 -1115> 2014-08-13 17:38:05.453428 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.4 -1114> 2014-08-13 17:38:05.453490 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.4 -1113> 2014-08-13 17:38:05.453551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.4 -1112> 2014-08-13 17:38:05.453608 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] _activate_committed 194487 telling primary -1111> 2014-08-13 17:38:05.453631 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4e3d800 already has epoch 194487 -1110> 2014-08-13 17:38:05.453634 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.6 192.168.1.107:6809/53141 -- pg_info(1 pgs e194487:24.cd) v4 -- ?+0 0x4c17e00 -1109> 2014-08-13 17:38:05.453649 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] flushed -1108> 2014-08-13 17:38:05.453612 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.4 -1107> 2014-08-13 17:38:05.453674 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.4 -1106> 2014-08-13 17:38:05.453735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.4 -1105> 2014-08-13 17:38:05.453798 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.4 -1104> 2014-08-13 17:38:05.453861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.4 -1103> 2014-08-13 17:38:05.453924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.4 -1102> 2014-08-13 17:38:05.453987 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.4 -1101> 2014-08-13 17:38:05.454050 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.4 -1100> 2014-08-13 17:38:05.454114 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.4 -1099> 2014-08-13 17:38:05.454191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] state: Activate Finished -1098> 2014-08-13 17:38:05.454180 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.4 -1097> 2014-08-13 17:38:05.454273 7fe99ebe8700 10 log is not dirty -1096> 2014-08-13 17:38:05.454290 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) -1095> 2014-08-13 17:38:05.454244 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.4 -1094> 2014-08-13 17:38:05.454306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.45 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) -1093> 2014-08-13 17:38:05.454319 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.085305 8 0.000489 -1092> 2014-08-13 17:38:05.454332 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -1091> 2014-08-13 17:38:05.454343 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -1090> 2014-08-13 17:38:05.454345 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.304 192.168.1.101:6808/22782 12 ==== pg_info(1 pgs e194487:24.33a) v4 ==== 748+0+0 (1729647007 0 0) 0x4c163c0 con 0x49e39c0 -1089> 2014-08-13 17:38:05.454354 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194477 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate -1088> 2014-08-13 17:38:05.454361 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1087> 2014-08-13 17:38:05.454363 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1086> 2014-08-13 17:38:05.454365 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c163c0 pg_info(1 pgs e194487:24.33a) v4 -1085> 2014-08-13 17:38:05.454307 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.4 -1084> 2014-08-13 17:38:05.454369 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.453081, event: header_read, request: pg_info(1 pgs e194487:24.33a) v4 -1083> 2014-08-13 17:38:05.454364 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] activate - not complete, missing(50) -1082> 2014-08-13 17:38:05.454376 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.453082, event: throttled, request: pg_info(1 pgs e194487:24.33a) v4 -1081> 2014-08-13 17:38:05.454379 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.453144, event: all_read, request: pg_info(1 pgs e194487:24.33a) v4 -1080> 2014-08-13 17:38:05.454383 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.454360, event: dispatched, request: pg_info(1 pgs e194487:24.33a) v4 -1079> 2014-08-13 17:38:05.454387 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.454387, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.33a) v4 -1078> 2014-08-13 17:38:05.454390 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.33a) v4 from osd.304 -1077> 2014-08-13 17:38:05.454393 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c163c0 -1076> 2014-08-13 17:38:05.454396 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.454396, event: started, request: pg_info(1 pgs e194487:24.33a) v4 -1075> 2014-08-13 17:38:05.454411 7fe9a63f4700 5 -- op tracker -- , seq: 661, time: 2014-08-13 17:38:05.454411, event: done, request: pg_info(1 pgs e194487:24.33a) v4 -1074> 2014-08-13 17:38:05.454418 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1073> 2014-08-13 17:38:05.454420 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1072> 2014-08-13 17:38:05.454370 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.4 -1071> 2014-08-13 17:38:05.454432 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.4 -1070> 2014-08-13 17:38:05.454546 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.337 192.168.1.104:6804/62197 22 ==== pg_info(1 pgs e194487:24.1bb) v4 ==== 748+0+0 (570767963 0 0) 0x5820e00 con 0x4d3c780 -1069> 2014-08-13 17:38:05.454561 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1068> 2014-08-13 17:38:05.454564 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1067> 2014-08-13 17:38:05.454565 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5820e00 pg_info(1 pgs e194487:24.1bb) v4 -1066> 2014-08-13 17:38:05.454569 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454470, event: header_read, request: pg_info(1 pgs e194487:24.1bb) v4 -1065> 2014-08-13 17:38:05.454574 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454472, event: throttled, request: pg_info(1 pgs e194487:24.1bb) v4 -1064> 2014-08-13 17:38:05.454517 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.4 -1063> 2014-08-13 17:38:05.454578 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454528, event: all_read, request: pg_info(1 pgs e194487:24.1bb) v4 -1062> 2014-08-13 17:38:05.454582 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454560, event: dispatched, request: pg_info(1 pgs e194487:24.1bb) v4 -1061> 2014-08-13 17:38:05.454586 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454586, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1bb) v4 -1060> 2014-08-13 17:38:05.454590 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.337 -1059> 2014-08-13 17:38:05.454593 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5820e00 -1058> 2014-08-13 17:38:05.454596 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454596, event: started, request: pg_info(1 pgs e194487:24.1bb) v4 -1057> 2014-08-13 17:38:05.454611 7fe9a63f4700 5 -- op tracker -- , seq: 662, time: 2014-08-13 17:38:05.454610, event: done, request: pg_info(1 pgs e194487:24.1bb) v4 -1056> 2014-08-13 17:38:05.454619 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1055> 2014-08-13 17:38:05.454620 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1054> 2014-08-13 17:38:05.454581 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.4 -1053> 2014-08-13 17:38:05.454642 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.4 -1052> 2014-08-13 17:38:05.454692 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.6 192.168.1.107:6809/53141 9 ==== pg_info(1 pgs e194487:24.cd) v4 ==== 748+0+0 (2154805169 0 0) 0x5822840 con 0x4e3d800 -1051> 2014-08-13 17:38:05.454710 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1050> 2014-08-13 17:38:05.454713 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1049> 2014-08-13 17:38:05.454714 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5822840 pg_info(1 pgs e194487:24.cd) v4 -1048> 2014-08-13 17:38:05.454719 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454581, event: header_read, request: pg_info(1 pgs e194487:24.cd) v4 -1047> 2014-08-13 17:38:05.454727 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454582, event: throttled, request: pg_info(1 pgs e194487:24.cd) v4 -1046> 2014-08-13 17:38:05.454731 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454638, event: all_read, request: pg_info(1 pgs e194487:24.cd) v4 -1045> 2014-08-13 17:38:05.454734 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454709, event: dispatched, request: pg_info(1 pgs e194487:24.cd) v4 -1044> 2014-08-13 17:38:05.454739 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454738, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.cd) v4 -1043> 2014-08-13 17:38:05.454742 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.cd) v4 from osd.6 -1042> 2014-08-13 17:38:05.454745 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5822840 -1041> 2014-08-13 17:38:05.454748 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454748, event: started, request: pg_info(1 pgs e194487:24.cd) v4 -1040> 2014-08-13 17:38:05.454704 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.4 -1039> 2014-08-13 17:38:05.454764 7fe9a63f4700 5 -- op tracker -- , seq: 663, time: 2014-08-13 17:38:05.454764, event: done, request: pg_info(1 pgs e194487:24.cd) v4 -1038> 2014-08-13 17:38:05.454773 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1037> 2014-08-13 17:38:05.454775 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1036> 2014-08-13 17:38:05.454765 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.4 -1035> 2014-08-13 17:38:05.454827 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.4 -1034> 2014-08-13 17:38:05.454890 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.4 -1033> 2014-08-13 17:38:05.454954 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.4 -1032> 2014-08-13 17:38:05.454997 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] _activate_committed 194487 telling primary -1031> 2014-08-13 17:38:05.455026 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4d3f900 don't know epoch, doing nothing -1030> 2014-08-13 17:38:05.455029 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194487:24.2a1) v4 -- ?+0 0x4265240 -1029> 2014-08-13 17:38:05.455050 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] flushed -1028> 2014-08-13 17:38:05.455017 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.4 -1027> 2014-08-13 17:38:05.455080 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.4 -1026> 2014-08-13 17:38:05.455180 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] state: Activate Finished -1025> 2014-08-13 17:38:05.455143 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.4 -1024> 2014-08-13 17:38:05.455245 7fe99ebe8700 10 log is not dirty -1023> 2014-08-13 17:38:05.455208 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.4 -1022> 2014-08-13 17:38:05.455275 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.4 -1021> 2014-08-13 17:38:05.455339 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.4 -1020> 2014-08-13 17:38:05.455266 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 379 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -1019> 2014-08-13 17:38:05.455402 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.4 -1018> 2014-08-13 17:38:05.455465 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.4 -1017> 2014-08-13 17:38:05.455514 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.348 192.168.1.110:6800/41204 5 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x549d340 con 0x5b96720 -1016> 2014-08-13 17:38:05.455529 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1015> 2014-08-13 17:38:05.455531 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1014> 2014-08-13 17:38:05.455532 7fe9a63f4700 20 osd.4 194487 _dispatch 0x549d340 osd_map(194487..194487 src has 185139..194487) v3 -1013> 2014-08-13 17:38:05.455541 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -1012> 2014-08-13 17:38:05.455544 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -1011> 2014-08-13 17:38:05.455546 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1010> 2014-08-13 17:38:05.455547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1009> 2014-08-13 17:38:05.455432 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 379, already have info from that osd, ignoring -1008> 2014-08-13 17:38:05.455580 7fe99ebe8700 10 log is not dirty -1007> 2014-08-13 17:38:05.455528 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.4 -1006> 2014-08-13 17:38:05.455592 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.348 192.168.1.110:6800/41204 6 ==== pg_info(1 pgs e194487:24.1bb) v4 ==== 724+0+0 (3574936488 0 0) 0x4a0af40 con 0x5b96720 -1005> 2014-08-13 17:38:05.455582 7fe9aa308700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] _activate_committed 194487 telling primary -1004> 2014-08-13 17:38:05.455602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -1003> 2014-08-13 17:38:05.455604 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -1002> 2014-08-13 17:38:05.455605 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4a0af40 pg_info(1 pgs e194487:24.1bb) v4 -1001> 2014-08-13 17:38:05.455608 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455516, event: header_read, request: pg_info(1 pgs e194487:24.1bb) v4 -1000> 2014-08-13 17:38:05.455611 7fe9aa308700 20 osd.4 194487 _share_map_outgoing 0x4d39080 don't know epoch, doing nothing -999> 2014-08-13 17:38:05.455613 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455517, event: throttled, request: pg_info(1 pgs e194487:24.1bb) v4 -998> 2014-08-13 17:38:05.455617 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455578, event: all_read, request: pg_info(1 pgs e194487:24.1bb) v4 -997> 2014-08-13 17:38:05.455614 7fe9aa308700 1 -- 192.168.1.115:6813/34805 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194487:24.369) v4 -- ?+0 0x5f02f40 -996> 2014-08-13 17:38:05.455620 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455602, event: dispatched, request: pg_info(1 pgs e194487:24.1bb) v4 -995> 2014-08-13 17:38:05.455605 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) -994> 2014-08-13 17:38:05.455626 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455626, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1bb) v4 -993> 2014-08-13 17:38:05.455630 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.348 -992> 2014-08-13 17:38:05.455633 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4a0af40 -991> 2014-08-13 17:38:05.455636 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455636, event: started, request: pg_info(1 pgs e194487:24.1bb) v4 -990> 2014-08-13 17:38:05.455629 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.285 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) -989> 2014-08-13 17:38:05.455632 7fe9aa308700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] flushed -988> 2014-08-13 17:38:05.455590 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.4 -987> 2014-08-13 17:38:05.455651 7fe9a63f4700 5 -- op tracker -- , seq: 664, time: 2014-08-13 17:38:05.455651, event: done, request: pg_info(1 pgs e194487:24.1bb) v4 -986> 2014-08-13 17:38:05.455646 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.086102 8 0.009439 -985> 2014-08-13 17:38:05.455659 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -984> 2014-08-13 17:38:05.455661 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -983> 2014-08-13 17:38:05.455660 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -982> 2014-08-13 17:38:05.455674 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -981> 2014-08-13 17:38:05.455685 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194477 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate -980> 2014-08-13 17:38:05.455695 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] activate - not complete, missing(50) -979> 2014-08-13 17:38:05.455652 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.4 -978> 2014-08-13 17:38:05.455715 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.4 -977> 2014-08-13 17:38:05.455776 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.4 -976> 2014-08-13 17:38:05.455837 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.4 -975> 2014-08-13 17:38:05.455898 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.4 -974> 2014-08-13 17:38:05.455957 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.17 192.168.1.103:6811/37658 13 ==== pg_info(1 pgs e194487:24.2a1) v4 ==== 748+0+0 (1248463261 0 0) 0x4265240 con 0x4d3f900 -973> 2014-08-13 17:38:05.455970 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -972> 2014-08-13 17:38:05.455973 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -971> 2014-08-13 17:38:05.455974 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4265240 pg_info(1 pgs e194487:24.2a1) v4 -970> 2014-08-13 17:38:05.455977 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.455880, event: header_read, request: pg_info(1 pgs e194487:24.2a1) v4 -969> 2014-08-13 17:38:05.455982 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.455882, event: throttled, request: pg_info(1 pgs e194487:24.2a1) v4 -968> 2014-08-13 17:38:05.455983 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.51 192.168.1.112:0/6655 2 ==== osd_ping(ping e194487 stamp 2014-08-13 17:38:05.455872) v2 ==== 47+0+0 (2680370802 0 0) 0x5138000 con 0x582bde0 -967> 2014-08-13 17:38:05.455985 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.455939, event: all_read, request: pg_info(1 pgs e194487:24.2a1) v4 -966> 2014-08-13 17:38:05.455997 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.455970, event: dispatched, request: pg_info(1 pgs e194487:24.2a1) v4 -965> 2014-08-13 17:38:05.456001 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.456001, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.2a1) v4 -964> 2014-08-13 17:38:05.456004 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2a1) v4 from osd.17 -963> 2014-08-13 17:38:05.456000 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194487 stamp 2014-08-13 17:38:05.455872) v2 -- ?+0 0x5104600 con 0x582bde0 -962> 2014-08-13 17:38:05.456007 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4265240 -961> 2014-08-13 17:38:05.456010 7fe9a45f1700 10 osd.4 194487 note_peer_epoch osd.51 has 194487 -960> 2014-08-13 17:38:05.456010 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.456010, event: started, request: pg_info(1 pgs e194487:24.2a1) v4 -959> 2014-08-13 17:38:05.456017 7fe9a45f1700 20 osd.4 194487 _share_map_outgoing 0x4e39e40 already has epoch 194487 -958> 2014-08-13 17:38:05.456012 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.51 192.168.1.112:0/6655 2 ==== osd_ping(ping e194487 stamp 2014-08-13 17:38:05.455872) v2 ==== 47+0+0 (2680370802 0 0) 0x5f07e00 con 0x581d6a0 -957> 2014-08-13 17:38:05.456026 7fe9a63f4700 5 -- op tracker -- , seq: 665, time: 2014-08-13 17:38:05.456026, event: done, request: pg_info(1 pgs e194487:24.2a1) v4 -956> 2014-08-13 17:38:05.456034 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -955> 2014-08-13 17:38:05.456036 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -954> 2014-08-13 17:38:05.456033 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.112:0/6655 -- osd_ping(ping_reply e194487 stamp 2014-08-13 17:38:05.455872) v2 -- ?+0 0x5100000 con 0x581d6a0 -953> 2014-08-13 17:38:05.456047 7fe9a31ef700 20 osd.4 194487 _share_map_outgoing 0x4e39e40 already has epoch 194487 -952> 2014-08-13 17:38:05.455959 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.4 -951> 2014-08-13 17:38:05.456073 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.4 -950> 2014-08-13 17:38:05.456136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.4 -949> 2014-08-13 17:38:05.456202 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.4 -948> 2014-08-13 17:38:05.456263 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.4 -947> 2014-08-13 17:38:05.456324 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.4 -946> 2014-08-13 17:38:05.456385 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.4 -945> 2014-08-13 17:38:05.456449 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] state: Activate Finished -944> 2014-08-13 17:38:05.456446 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.4 -943> 2014-08-13 17:38:05.456515 7fe99ebe8700 10 log is not dirty -942> 2014-08-13 17:38:05.456522 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.331 192.168.1.104:6816/837 10 ==== pg_info(1 pgs e194487:24.1bb) v4 ==== 724+0+0 (3574936488 0 0) 0x5f01dc0 con 0x4e83180 -941> 2014-08-13 17:38:05.456536 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -940> 2014-08-13 17:38:05.456539 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -939> 2014-08-13 17:38:05.456540 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f01dc0 pg_info(1 pgs e194487:24.1bb) v4 -938> 2014-08-13 17:38:05.456543 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456442, event: header_read, request: pg_info(1 pgs e194487:24.1bb) v4 -937> 2014-08-13 17:38:05.456534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 330 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) -936> 2014-08-13 17:38:05.456549 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456443, event: throttled, request: pg_info(1 pgs e194487:24.1bb) v4 -935> 2014-08-13 17:38:05.456552 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456496, event: all_read, request: pg_info(1 pgs e194487:24.1bb) v4 -934> 2014-08-13 17:38:05.456556 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456536, event: dispatched, request: pg_info(1 pgs e194487:24.1bb) v4 -933> 2014-08-13 17:38:05.456552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 330, already have info from that osd, ignoring -932> 2014-08-13 17:38:05.456560 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456560, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1bb) v4 -931> 2014-08-13 17:38:05.456563 7fe99ebe8700 10 log is not dirty -930> 2014-08-13 17:38:05.456564 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1bb) v4 from osd.331 -929> 2014-08-13 17:38:05.456567 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f01dc0 -928> 2014-08-13 17:38:05.456569 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456569, event: started, request: pg_info(1 pgs e194487:24.1bb) v4 -927> 2014-08-13 17:38:05.456570 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) -926> 2014-08-13 17:38:05.456584 7fe9a63f4700 5 -- op tracker -- , seq: 666, time: 2014-08-13 17:38:05.456584, event: done, request: pg_info(1 pgs e194487:24.1bb) v4 -925> 2014-08-13 17:38:05.456594 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -924> 2014-08-13 17:38:05.456596 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -923> 2014-08-13 17:38:05.456584 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: got info from osd.330 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) -922> 2014-08-13 17:38:05.456507 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -921> 2014-08-13 17:38:05.456599 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 1.107330 8 0.000609 -920> 2014-08-13 17:38:05.456611 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive -919> 2014-08-13 17:38:05.456623 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering -918> 2014-08-13 17:38:05.456634 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194478 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: In ReplicaActive, about to call activate -917> 2014-08-13 17:38:05.456644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] activate - not complete, missing(64) -916> 2014-08-13 17:38:05.456607 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] activate: adding 50 as a source -915> 2014-08-13 17:38:05.456668 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.50 -914> 2014-08-13 17:38:05.456767 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.45 192.168.1.112:6828/6040 14 ==== pg_info(1 pgs e194487:24.369) v4 ==== 748+0+0 (1867150127 0 0) 0x5f02f40 con 0x4d39080 -913> 2014-08-13 17:38:05.456780 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -912> 2014-08-13 17:38:05.456782 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -911> 2014-08-13 17:38:05.456783 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02f40 pg_info(1 pgs e194487:24.369) v4 -910> 2014-08-13 17:38:05.456771 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] _activate_committed 194487 telling primary -909> 2014-08-13 17:38:05.456787 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456695, event: header_read, request: pg_info(1 pgs e194487:24.369) v4 -908> 2014-08-13 17:38:05.456731 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=50] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.50 -907> 2014-08-13 17:38:05.456793 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456695, event: throttled, request: pg_info(1 pgs e194487:24.369) v4 -906> 2014-08-13 17:38:05.456797 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456752, event: all_read, request: pg_info(1 pgs e194487:24.369) v4 -905> 2014-08-13 17:38:05.456801 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4a45280 don't know epoch, doing nothing -904> 2014-08-13 17:38:05.456800 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456779, event: dispatched, request: pg_info(1 pgs e194487:24.369) v4 -903> 2014-08-13 17:38:05.456807 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456807, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.369) v4 -902> 2014-08-13 17:38:05.456805 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194487:24.1e6) v4 -- ?+0 0x51cb480 -901> 2014-08-13 17:38:05.456812 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.369) v4 from osd.45 -900> 2014-08-13 17:38:05.456816 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02f40 -899> 2014-08-13 17:38:05.456819 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456819, event: started, request: pg_info(1 pgs e194487:24.369) v4 -898> 2014-08-13 17:38:05.456823 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] flushed -897> 2014-08-13 17:38:05.456836 7fe9a63f4700 5 -- op tracker -- , seq: 667, time: 2014-08-13 17:38:05.456836, event: done, request: pg_info(1 pgs e194487:24.369) v4 -896> 2014-08-13 17:38:05.456845 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -895> 2014-08-13 17:38:05.456847 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -894> 2014-08-13 17:38:05.456796 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=49] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.50 -893> 2014-08-13 17:38:05.456864 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=48] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.50 -892> 2014-08-13 17:38:05.456931 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=47] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.50 -891> 2014-08-13 17:38:05.457001 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=46] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.50 -890> 2014-08-13 17:38:05.457072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=45] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.50 -889> 2014-08-13 17:38:05.457149 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=44] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.50 -888> 2014-08-13 17:38:05.457233 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=43] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.50 -887> 2014-08-13 17:38:05.457311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=42] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.50 -886> 2014-08-13 17:38:05.457397 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] state: Activate Finished -885> 2014-08-13 17:38:05.457468 7fe99ebe8700 10 log is not dirty -884> 2014-08-13 17:38:05.457391 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=41] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.50 -883> 2014-08-13 17:38:05.457497 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) -882> 2014-08-13 17:38:05.457512 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: got info from osd.343 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) -881> 2014-08-13 17:38:05.457525 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 1.058548 8 0.000465 -880> 2014-08-13 17:38:05.457536 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive -879> 2014-08-13 17:38:05.457547 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering -878> 2014-08-13 17:38:05.457480 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=40] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.50 -877> 2014-08-13 17:38:05.457557 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194477 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: In ReplicaActive, about to call activate -876> 2014-08-13 17:38:05.457567 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] activate - not complete, missing(41) -875> 2014-08-13 17:38:05.457597 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.285 192.168.1.101:6822/26295 10 ==== pg_info(1 pgs e194487:24.1e6) v4 ==== 748+0+0 (681562148 0 0) 0x51cb480 con 0x4a45280 -874> 2014-08-13 17:38:05.457612 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -873> 2014-08-13 17:38:05.457614 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -872> 2014-08-13 17:38:05.457616 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51cb480 pg_info(1 pgs e194487:24.1e6) v4 -871> 2014-08-13 17:38:05.457619 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457493, event: header_read, request: pg_info(1 pgs e194487:24.1e6) v4 -870> 2014-08-13 17:38:05.457626 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457498, event: throttled, request: pg_info(1 pgs e194487:24.1e6) v4 -869> 2014-08-13 17:38:05.457629 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457583, event: all_read, request: pg_info(1 pgs e194487:24.1e6) v4 -868> 2014-08-13 17:38:05.457632 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457611, event: dispatched, request: pg_info(1 pgs e194487:24.1e6) v4 -867> 2014-08-13 17:38:05.457637 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457637, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1e6) v4 -866> 2014-08-13 17:38:05.457640 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1e6) v4 from osd.285 -865> 2014-08-13 17:38:05.457643 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51cb480 -864> 2014-08-13 17:38:05.457563 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=39] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.50 -863> 2014-08-13 17:38:05.457645 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457645, event: started, request: pg_info(1 pgs e194487:24.1e6) v4 -862> 2014-08-13 17:38:05.457660 7fe9a63f4700 5 -- op tracker -- , seq: 668, time: 2014-08-13 17:38:05.457660, event: done, request: pg_info(1 pgs e194487:24.1e6) v4 -861> 2014-08-13 17:38:05.457668 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -860> 2014-08-13 17:38:05.457669 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -859> 2014-08-13 17:38:05.457648 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=38] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.50 -858> 2014-08-13 17:38:05.457732 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 crt=194422'15537 lcod 0'0 remapped m=64] _activate_committed 194487 telling primary -857> 2014-08-13 17:38:05.457759 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4ec9b80 already has epoch 194487 -856> 2014-08-13 17:38:05.457762 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194487:24.3f7) v4 -- ?+0 0x51ce740 -855> 2014-08-13 17:38:05.457778 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] flushed -854> 2014-08-13 17:38:05.457735 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=37] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.50 -853> 2014-08-13 17:38:05.457825 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=36] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.50 -852> 2014-08-13 17:38:05.457916 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=35] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.50 -851> 2014-08-13 17:38:05.458034 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.363 192.168.1.102:6842/20017 9 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x5460b40 con 0x4e4a680 -850> 2014-08-13 17:38:05.458051 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -849> 2014-08-13 17:38:05.458054 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -848> 2014-08-13 17:38:05.458056 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5460b40 osd_map(194487..194487 src has 185139..194487) v3 -847> 2014-08-13 17:38:05.458061 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -846> 2014-08-13 17:38:05.458064 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -845> 2014-08-13 17:38:05.458065 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -844> 2014-08-13 17:38:05.458066 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -843> 2014-08-13 17:38:05.458089 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.363 192.168.1.102:6842/20017 10 ==== pg_notify(22.99(1) epoch 194487) v5 ==== 777+0+0 (1928677791 0 0) 0x5102140 con 0x4e4a680 -842> 2014-08-13 17:38:05.458008 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=34] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.50 -841> 2014-08-13 17:38:05.458099 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -840> 2014-08-13 17:38:05.458101 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -839> 2014-08-13 17:38:05.458102 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102140 pg_notify(22.99(1) epoch 194487) v5 -838> 2014-08-13 17:38:05.458106 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458036, event: header_read, request: pg_notify(22.99(1) epoch 194487) v5 -837> 2014-08-13 17:38:05.458111 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458037, event: throttled, request: pg_notify(22.99(1) epoch 194487) v5 -836> 2014-08-13 17:38:05.458114 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458078, event: all_read, request: pg_notify(22.99(1) epoch 194487) v5 -835> 2014-08-13 17:38:05.458118 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458099, event: dispatched, request: pg_notify(22.99(1) epoch 194487) v5 -834> 2014-08-13 17:38:05.458122 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458121, event: waiting_for_osdmap, request: pg_notify(22.99(1) epoch 194487) v5 -833> 2014-08-13 17:38:05.458125 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.363 -832> 2014-08-13 17:38:05.458128 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102140 -831> 2014-08-13 17:38:05.458130 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458130, event: started, request: pg_notify(22.99(1) epoch 194487) v5 -830> 2014-08-13 17:38:05.458145 7fe9a63f4700 5 -- op tracker -- , seq: 669, time: 2014-08-13 17:38:05.458145, event: done, request: pg_notify(22.99(1) epoch 194487) v5 -829> 2014-08-13 17:38:05.458152 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -828> 2014-08-13 17:38:05.458153 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -827> 2014-08-13 17:38:05.458101 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=33] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.50 -826> 2014-08-13 17:38:05.458200 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=32] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.50 -825> 2014-08-13 17:38:05.458362 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] state: Activate Finished -824> 2014-08-13 17:38:05.458297 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=31] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.50 -823> 2014-08-13 17:38:05.458421 7fe99ebe8700 10 log is not dirty -822> 2014-08-13 17:38:05.458438 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) -821> 2014-08-13 17:38:05.458454 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: got info from osd.346 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) -820> 2014-08-13 17:38:05.458467 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 1.137638 9 0.000593 -819> 2014-08-13 17:38:05.458479 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive -818> 2014-08-13 17:38:05.458397 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=30] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.50 -817> 2014-08-13 17:38:05.458490 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering -816> 2014-08-13 17:38:05.458501 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194477 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: In ReplicaActive, about to call activate -815> 2014-08-13 17:38:05.458511 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] activate - not complete, missing(66) -814> 2014-08-13 17:38:05.458585 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 15 ==== pg_info(1 pgs e194487:24.3f7) v4 ==== 748+0+0 (3933871075 0 0) 0x51ce740 con 0x4ec9b80 -813> 2014-08-13 17:38:05.458499 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=29] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.50 -812> 2014-08-13 17:38:05.458604 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -811> 2014-08-13 17:38:05.458611 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -810> 2014-08-13 17:38:05.458613 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ce740 pg_info(1 pgs e194487:24.3f7) v4 -809> 2014-08-13 17:38:05.458617 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458520, event: header_read, request: pg_info(1 pgs e194487:24.3f7) v4 -808> 2014-08-13 17:38:05.458622 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458521, event: throttled, request: pg_info(1 pgs e194487:24.3f7) v4 -807> 2014-08-13 17:38:05.458625 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458572, event: all_read, request: pg_info(1 pgs e194487:24.3f7) v4 -806> 2014-08-13 17:38:05.458629 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458603, event: dispatched, request: pg_info(1 pgs e194487:24.3f7) v4 -805> 2014-08-13 17:38:05.458632 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458632, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.3f7) v4 -804> 2014-08-13 17:38:05.458636 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.3f7) v4 from osd.330 -803> 2014-08-13 17:38:05.458639 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ce740 -802> 2014-08-13 17:38:05.458641 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458641, event: started, request: pg_info(1 pgs e194487:24.3f7) v4 -801> 2014-08-13 17:38:05.458655 7fe9a63f4700 5 -- op tracker -- , seq: 670, time: 2014-08-13 17:38:05.458654, event: done, request: pg_info(1 pgs e194487:24.3f7) v4 -800> 2014-08-13 17:38:05.458662 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -799> 2014-08-13 17:38:05.458664 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -798> 2014-08-13 17:38:05.458651 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] _activate_committed 194487 telling primary -797> 2014-08-13 17:38:05.458679 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4d31e40 already has epoch 194487 -796> 2014-08-13 17:38:05.458682 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194487:24.206) v4 -- ?+0 0x51ca4c0 -795> 2014-08-13 17:38:05.458700 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] flushed -794> 2014-08-13 17:38:05.458609 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=28] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.50 -793> 2014-08-13 17:38:05.458713 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=27] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.50 -792> 2014-08-13 17:38:05.458820 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=26] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.50 -791> 2014-08-13 17:38:05.458928 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=25] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.50 -790> 2014-08-13 17:38:05.459037 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=24] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.50 -789> 2014-08-13 17:38:05.459149 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=23] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.50 -788> 2014-08-13 17:38:05.459291 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] state: Activate Finished -787> 2014-08-13 17:38:05.459330 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.363 192.168.1.102:6842/20017 11 ==== pg_notify(22.39d(29) epoch 194487) v5 ==== 2341+0+0 (3134800639 0 0) 0x5104600 con 0x4e4a680 -786> 2014-08-13 17:38:05.459344 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -785> 2014-08-13 17:38:05.459347 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -784> 2014-08-13 17:38:05.459348 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5104600 pg_notify(22.39d(29) epoch 194487) v5 -783> 2014-08-13 17:38:05.459351 7fe99ebe8700 10 log is not dirty -782> 2014-08-13 17:38:05.459352 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459237, event: header_read, request: pg_notify(22.39d(29) epoch 194487) v5 -781> 2014-08-13 17:38:05.459358 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459238, event: throttled, request: pg_notify(22.39d(29) epoch 194487) v5 -780> 2014-08-13 17:38:05.459362 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459308, event: all_read, request: pg_notify(22.39d(29) epoch 194487) v5 -779> 2014-08-13 17:38:05.459366 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459344, event: dispatched, request: pg_notify(22.39d(29) epoch 194487) v5 -778> 2014-08-13 17:38:05.459371 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459370, event: waiting_for_osdmap, request: pg_notify(22.39d(29) epoch 194487) v5 -777> 2014-08-13 17:38:05.459374 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.363 -776> 2014-08-13 17:38:05.459264 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=22] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.50 -775> 2014-08-13 17:38:05.459376 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5104600 -774> 2014-08-13 17:38:05.459367 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) -773> 2014-08-13 17:38:05.459380 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459380, event: started, request: pg_notify(22.39d(29) epoch 194487) v5 -772> 2014-08-13 17:38:05.459382 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: got info from osd.364 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) -771> 2014-08-13 17:38:05.459396 7fe9a63f4700 5 -- op tracker -- , seq: 671, time: 2014-08-13 17:38:05.459396, event: done, request: pg_notify(22.39d(29) epoch 194487) v5 -770> 2014-08-13 17:38:05.459405 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -769> 2014-08-13 17:38:05.459397 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 1.163467 8 0.000572 -768> 2014-08-13 17:38:05.459407 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -767> 2014-08-13 17:38:05.459409 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive -766> 2014-08-13 17:38:05.459420 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering -765> 2014-08-13 17:38:05.459430 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194477 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: In ReplicaActive, about to call activate -764> 2014-08-13 17:38:05.459440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] activate - not complete, missing(48) -763> 2014-08-13 17:38:05.459462 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.343 192.168.1.104:6802/61960 14 ==== pg_info(1 pgs e194487:24.206) v4 ==== 748+0+0 (3312974797 0 0) 0x51ca4c0 con 0x4d31e40 -762> 2014-08-13 17:38:05.459472 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -761> 2014-08-13 17:38:05.459474 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -760> 2014-08-13 17:38:05.459475 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51ca4c0 pg_info(1 pgs e194487:24.206) v4 -759> 2014-08-13 17:38:05.459478 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459388, event: header_read, request: pg_info(1 pgs e194487:24.206) v4 -758> 2014-08-13 17:38:05.459482 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459388, event: throttled, request: pg_info(1 pgs e194487:24.206) v4 -757> 2014-08-13 17:38:05.459486 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459448, event: all_read, request: pg_info(1 pgs e194487:24.206) v4 -756> 2014-08-13 17:38:05.459489 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459472, event: dispatched, request: pg_info(1 pgs e194487:24.206) v4 -755> 2014-08-13 17:38:05.459378 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=21] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.50 -754> 2014-08-13 17:38:05.459493 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459493, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.206) v4 -753> 2014-08-13 17:38:05.459497 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.206) v4 from osd.343 -752> 2014-08-13 17:38:05.459500 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51ca4c0 -751> 2014-08-13 17:38:05.459502 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459502, event: started, request: pg_info(1 pgs e194487:24.206) v4 -750> 2014-08-13 17:38:05.459515 7fe9a63f4700 5 -- op tracker -- , seq: 672, time: 2014-08-13 17:38:05.459515, event: done, request: pg_info(1 pgs e194487:24.206) v4 -749> 2014-08-13 17:38:05.459523 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -748> 2014-08-13 17:38:05.459525 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -747> 2014-08-13 17:38:05.459585 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 crt=194422'15014 lcod 0'0 remapped m=66] _activate_committed 194487 telling primary -746> 2014-08-13 17:38:05.459604 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4e4a100 already has epoch 194487 -745> 2014-08-13 17:38:05.459494 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=20] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.50 -744> 2014-08-13 17:38:05.459607 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.346 192.168.1.104:6820/2308 -- pg_info(1 pgs e194487:24.e2) v4 -- ?+0 0x51c8a80 -743> 2014-08-13 17:38:05.459622 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] flushed -742> 2014-08-13 17:38:05.459613 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=19] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.50 -741> 2014-08-13 17:38:05.459733 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=18] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.50 -740> 2014-08-13 17:38:05.459854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=17] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.50 -739> 2014-08-13 17:38:05.459978 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=16] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.50 -738> 2014-08-13 17:38:05.460162 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] state: Activate Finished -737> 2014-08-13 17:38:05.460227 7fe99ebe8700 10 log is not dirty -736> 2014-08-13 17:38:05.460226 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194487 stamp 2014-08-13 17:38:05.459980) v2 ==== 47+0+0 (1922465991 0 0) 0x5102680 con 0x581eca0 -735> 2014-08-13 17:38:05.460240 7fe9a45f1700 1 -- 192.168.100.115:6806/34805 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194487 stamp 2014-08-13 17:38:05.459980) v2 -- ?+0 0x5138000 con 0x581eca0 -734> 2014-08-13 17:38:05.460252 7fe9a45f1700 10 osd.4 194487 note_peer_epoch osd.320 now has 194487 -733> 2014-08-13 17:38:05.460260 7fe9a45f1700 20 osd.4 194487 _share_map_outgoing 0x4ec8000 already has epoch 194487 -732> 2014-08-13 17:38:05.460262 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 <== osd.320 192.168.1.106:0/10050 1 ==== osd_ping(ping e194487 stamp 2014-08-13 17:38:05.459980) v2 ==== 47+0+0 (1922465991 0 0) 0x5107000 con 0x581bc80 -731> 2014-08-13 17:38:05.460103 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=15] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.50 -730> 2014-08-13 17:38:05.460280 7fe9a31ef700 1 -- 192.168.1.115:6817/34805 --> 192.168.1.106:0/10050 -- osd_ping(ping_reply e194487 stamp 2014-08-13 17:38:05.459980) v2 -- ?+0 0x5f07e00 con 0x581bc80 -729> 2014-08-13 17:38:05.460296 7fe9a31ef700 20 osd.4 194487 _share_map_outgoing 0x4ec8000 already has epoch 194487 -728> 2014-08-13 17:38:05.460315 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) -727> 2014-08-13 17:38:05.460326 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.330 192.168.1.106:6802/4876 16 ==== MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 ==== 26+0+0 (1401951176 0 0) 0x5105400 con 0x4ec9b80 -726> 2014-08-13 17:38:05.460341 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -725> 2014-08-13 17:38:05.460343 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -724> 2014-08-13 17:38:05.460344 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5105400 MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -723> 2014-08-13 17:38:05.460333 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: got info from osd.374 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) -722> 2014-08-13 17:38:05.460348 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460264, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -721> 2014-08-13 17:38:05.460353 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460265, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -720> 2014-08-13 17:38:05.460356 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460308, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -719> 2014-08-13 17:38:05.460349 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 1.092600 8 0.000844 -718> 2014-08-13 17:38:05.460360 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460340, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -717> 2014-08-13 17:38:05.460364 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460364, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -716> 2014-08-13 17:38:05.460369 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5105400 -715> 2014-08-13 17:38:05.460362 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive -714> 2014-08-13 17:38:05.460377 7fe9a63f4700 5 -- op tracker -- , seq: 673, time: 2014-08-13 17:38:05.460377, event: done, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194487 v2 -713> 2014-08-13 17:38:05.460375 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering -712> 2014-08-13 17:38:05.460385 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -711> 2014-08-13 17:38:05.460386 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -710> 2014-08-13 17:38:05.460386 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194477 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: In ReplicaActive, about to call activate -709> 2014-08-13 17:38:05.460396 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] activate - not complete, missing(52) -708> 2014-08-13 17:38:05.460279 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=14] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.50 -707> 2014-08-13 17:38:05.460489 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.346 192.168.1.104:6820/2308 9 ==== pg_info(1 pgs e194487:24.e2) v4 ==== 748+0+0 (1468171109 0 0) 0x51071c0 con 0x4e4a100 -706> 2014-08-13 17:38:05.460499 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -705> 2014-08-13 17:38:05.460501 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -704> 2014-08-13 17:38:05.460503 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51071c0 pg_info(1 pgs e194487:24.e2) v4 -703> 2014-08-13 17:38:05.460506 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460422, event: header_read, request: pg_info(1 pgs e194487:24.e2) v4 -702> 2014-08-13 17:38:05.460511 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460423, event: throttled, request: pg_info(1 pgs e194487:24.e2) v4 -701> 2014-08-13 17:38:05.460514 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460476, event: all_read, request: pg_info(1 pgs e194487:24.e2) v4 -700> 2014-08-13 17:38:05.460517 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460499, event: dispatched, request: pg_info(1 pgs e194487:24.e2) v4 -699> 2014-08-13 17:38:05.460520 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460520, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.e2) v4 -698> 2014-08-13 17:38:05.460524 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.e2) v4 from osd.346 -697> 2014-08-13 17:38:05.460526 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51071c0 -696> 2014-08-13 17:38:05.460529 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460529, event: started, request: pg_info(1 pgs e194487:24.e2) v4 -695> 2014-08-13 17:38:05.460408 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=13] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.50 -694> 2014-08-13 17:38:05.460542 7fe9a63f4700 5 -- op tracker -- , seq: 674, time: 2014-08-13 17:38:05.460542, event: done, request: pg_info(1 pgs e194487:24.e2) v4 -693> 2014-08-13 17:38:05.460549 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -692> 2014-08-13 17:38:05.460551 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -691> 2014-08-13 17:38:05.460576 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 crt=194422'17074 lcod 0'0 remapped m=48] _activate_committed 194487 telling primary -690> 2014-08-13 17:38:05.460597 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4cfc620 already has epoch 194487 -689> 2014-08-13 17:38:05.460599 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.364 192.168.1.114:6826/52317 -- pg_info(1 pgs e194487:24.1de) v4 -- ?+0 0x5f00c40 -688> 2014-08-13 17:38:05.460613 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] flushed -687> 2014-08-13 17:38:05.460537 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=12] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.50 -686> 2014-08-13 17:38:05.460670 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=11] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.50 -685> 2014-08-13 17:38:05.460804 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=10] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.50 -684> 2014-08-13 17:38:05.460940 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=9] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.50 -683> 2014-08-13 17:38:05.461151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] state: Activate Finished -682> 2014-08-13 17:38:05.461077 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=8] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.50 -681> 2014-08-13 17:38:05.461228 7fe99ebe8700 10 log is not dirty -680> 2014-08-13 17:38:05.461247 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 293 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194475 n=28 ec=183096 les/c 194475/194389 194485/194485/194485)) -679> 2014-08-13 17:38:05.461264 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 293, already have info from that osd, ignoring -678> 2014-08-13 17:38:05.461275 7fe99ebe8700 10 log is not dirty -677> 2014-08-13 17:38:05.461281 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 314 notify: (query_epoch:194487, epoch_sent:194487, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -676> 2014-08-13 17:38:05.461294 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 314, already have info from that osd, ignoring -675> 2014-08-13 17:38:05.461303 7fe99ebe8700 10 log is not dirty -674> 2014-08-13 17:38:05.461308 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 320 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -673> 2014-08-13 17:38:05.461319 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 320, already have info from that osd, ignoring -672> 2014-08-13 17:38:05.461329 7fe99ebe8700 10 log is not dirty -671> 2014-08-13 17:38:05.461334 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 376 notify: (query_epoch:194487, epoch_sent:194487, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) -670> 2014-08-13 17:38:05.461346 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 376, already have info from that osd, ignoring -669> 2014-08-13 17:38:05.461355 7fe99ebe8700 10 log is not dirty -668> 2014-08-13 17:38:05.461221 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=7] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.50 -667> 2014-08-13 17:38:05.461361 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=6] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.50 -666> 2014-08-13 17:38:05.461495 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 crt=194422'16449 lcod 0'0 remapped m=52] _activate_committed 194487 telling primary -665> 2014-08-13 17:38:05.461519 7fe9aad09700 20 osd.4 194487 _share_map_outgoing 0x4c75120 don't know epoch, doing nothing -664> 2014-08-13 17:38:05.461521 7fe9aad09700 1 -- 192.168.1.115:6813/34805 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194487:24.2ec) v4 -- ?+0 0x5f02140 -663> 2014-08-13 17:38:05.461535 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] flushed -662> 2014-08-13 17:38:05.461360 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 301 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -661> 2014-08-13 17:38:05.461588 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.364 192.168.1.114:6826/52317 8 ==== pg_info(1 pgs e194487:24.1de) v4 ==== 748+0+0 (165822563 0 0) 0x5102f40 con 0x4cfc620 -660> 2014-08-13 17:38:05.461602 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -659> 2014-08-13 17:38:05.461606 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -658> 2014-08-13 17:38:05.461607 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5102f40 pg_info(1 pgs e194487:24.1de) v4 -657> 2014-08-13 17:38:05.461611 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461479, event: header_read, request: pg_info(1 pgs e194487:24.1de) v4 -656> 2014-08-13 17:38:05.461616 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461480, event: throttled, request: pg_info(1 pgs e194487:24.1de) v4 -655> 2014-08-13 17:38:05.461619 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461551, event: all_read, request: pg_info(1 pgs e194487:24.1de) v4 -654> 2014-08-13 17:38:05.461622 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461601, event: dispatched, request: pg_info(1 pgs e194487:24.1de) v4 -653> 2014-08-13 17:38:05.461626 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461626, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1de) v4 -652> 2014-08-13 17:38:05.461629 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1de) v4 from osd.364 -651> 2014-08-13 17:38:05.461632 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5102f40 -650> 2014-08-13 17:38:05.461635 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461635, event: started, request: pg_info(1 pgs e194487:24.1de) v4 -649> 2014-08-13 17:38:05.461504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=5] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.50 -648> 2014-08-13 17:38:05.461648 7fe9a63f4700 5 -- op tracker -- , seq: 675, time: 2014-08-13 17:38:05.461648, event: done, request: pg_info(1 pgs e194487:24.1de) v4 -647> 2014-08-13 17:38:05.461655 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -646> 2014-08-13 17:38:05.461657 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -645> 2014-08-13 17:38:05.461552 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 301, already have info from that osd, ignoring -644> 2014-08-13 17:38:05.461725 7fe99ebe8700 10 log is not dirty -643> 2014-08-13 17:38:05.461732 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 63 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -642> 2014-08-13 17:38:05.461746 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 63, already have info from that osd, ignoring -641> 2014-08-13 17:38:05.461756 7fe99ebe8700 10 log is not dirty -640> 2014-08-13 17:38:05.461647 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=4] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.50 -639> 2014-08-13 17:38:05.461786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 382 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) -638> 2014-08-13 17:38:05.461798 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 382, already have info from that osd, ignoring -637> 2014-08-13 17:38:05.461807 7fe99ebe8700 10 log is not dirty -636> 2014-08-13 17:38:05.461811 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 26 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -635> 2014-08-13 17:38:05.461825 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 26, already have info from that osd, ignoring -634> 2014-08-13 17:38:05.461836 7fe99ebe8700 10 log is not dirty -633> 2014-08-13 17:38:05.461792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=3] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.50 -632> 2014-08-13 17:38:05.461842 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -631> 2014-08-13 17:38:05.461943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=2] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.50 -630> 2014-08-13 17:38:05.462015 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops -629> 2014-08-13 17:38:05.462185 7fe99ebe8700 10 log is not dirty -628> 2014-08-13 17:38:05.462092 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=1] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.50 -627> 2014-08-13 17:38:05.462191 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 20 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -626> 2014-08-13 17:38:05.462248 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -625> 2014-08-13 17:38:05.462524 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.374 192.168.1.102:6828/16129 10 ==== pg_info(1 pgs e194487:24.2ec) v4 ==== 748+0+0 (3532027794 0 0) 0x5f02140 con 0x4c75120 -624> 2014-08-13 17:38:05.462545 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -623> 2014-08-13 17:38:05.462547 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -622> 2014-08-13 17:38:05.462381 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 20, already have info from that osd, ignoring -621> 2014-08-13 17:38:05.462548 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5f02140 pg_info(1 pgs e194487:24.2ec) v4 -620> 2014-08-13 17:38:05.462550 7fe99ebe8700 10 log is not dirty -619> 2014-08-13 17:38:05.462552 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462449, event: header_read, request: pg_info(1 pgs e194487:24.2ec) v4 -618> 2014-08-13 17:38:05.462556 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462450, event: throttled, request: pg_info(1 pgs e194487:24.2ec) v4 -617> 2014-08-13 17:38:05.462559 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462498, event: all_read, request: pg_info(1 pgs e194487:24.2ec) v4 -616> 2014-08-13 17:38:05.462565 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462545, event: dispatched, request: pg_info(1 pgs e194487:24.2ec) v4 -615> 2014-08-13 17:38:05.462558 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 55 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) -614> 2014-08-13 17:38:05.462569 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462569, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.2ec) v4 -613> 2014-08-13 17:38:05.462573 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.2ec) v4 from osd.374 -612> 2014-08-13 17:38:05.462575 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5f02140 -611> 2014-08-13 17:38:05.462571 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 55, already have info from that osd, ignoring -610> 2014-08-13 17:38:05.462578 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462578, event: started, request: pg_info(1 pgs e194487:24.2ec) v4 -609> 2014-08-13 17:38:05.462581 7fe99ebe8700 10 log is not dirty -608> 2014-08-13 17:38:05.462437 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 55 as a source -607> 2014-08-13 17:38:05.462591 7fe9a63f4700 5 -- op tracker -- , seq: 676, time: 2014-08-13 17:38:05.462591, event: done, request: pg_info(1 pgs e194487:24.2ec) v4 -606> 2014-08-13 17:38:05.462585 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 337 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -605> 2014-08-13 17:38:05.462598 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -604> 2014-08-13 17:38:05.462600 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -603> 2014-08-13 17:38:05.462598 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 337, already have info from that osd, ignoring -602> 2014-08-13 17:38:05.462607 7fe99ebe8700 10 log is not dirty -601> 2014-08-13 17:38:05.462589 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.55 (past last_backfill 0//0//-1) -600> 2014-08-13 17:38:05.462612 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 332 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -599> 2014-08-13 17:38:05.462743 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.55 (past last_backfill 0//0//-1) -598> 2014-08-13 17:38:05.462765 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 332, already have info from that osd, ignoring -597> 2014-08-13 17:38:05.462908 7fe99ebe8700 10 log is not dirty -596> 2014-08-13 17:38:05.462914 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 24 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) -595> 2014-08-13 17:38:05.462924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 24, already have info from that osd, ignoring -594> 2014-08-13 17:38:05.462934 7fe99ebe8700 10 log is not dirty -593> 2014-08-13 17:38:05.462937 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 397 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194485/194485/194485)) -592> 2014-08-13 17:38:05.462947 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 397, already have info from that osd, ignoring -591> 2014-08-13 17:38:05.462956 7fe99ebe8700 10 log is not dirty -590> 2014-08-13 17:38:05.462960 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 339 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -589> 2014-08-13 17:38:05.462972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 339, already have info from that osd, ignoring -588> 2014-08-13 17:38:05.462981 7fe99ebe8700 10 log is not dirty -587> 2014-08-13 17:38:05.462985 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 48 notify: (query_epoch:194487, epoch_sent:194487, info:22.619( v 194215'511 (0'0,194215'511] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -586> 2014-08-13 17:38:05.462995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,65,62] r=0 lpr=194485 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring -585> 2014-08-13 17:38:05.463004 7fe99ebe8700 10 log is not dirty -584> 2014-08-13 17:38:05.463008 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 302 notify: (query_epoch:194487, epoch_sent:194487, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) -583> 2014-08-13 17:38:05.463019 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 302, already have info from that osd, ignoring -582> 2014-08-13 17:38:05.463029 7fe99ebe8700 10 log is not dirty -581> 2014-08-13 17:38:05.463033 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 371 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) -580> 2014-08-13 17:38:05.462895 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.55 (past last_backfill 0//0//-1) -579> 2014-08-13 17:38:05.463045 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 371, already have info from that osd, ignoring -578> 2014-08-13 17:38:05.463055 7fe99ebe8700 10 log is not dirty -577> 2014-08-13 17:38:05.463059 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194484 n=24 ec=183096 les/c 194484/194484 194485/194485/194485)) -576> 2014-08-13 17:38:05.463069 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194486/194484 194485/194485/194485) [4,44,362] r=0 lpr=194485 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -575> 2014-08-13 17:38:05.463081 7fe99ebe8700 10 log is not dirty -574> 2014-08-13 17:38:05.463084 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) -573> 2014-08-13 17:38:05.463097 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -572> 2014-08-13 17:38:05.463106 7fe99ebe8700 10 log is not dirty -571> 2014-08-13 17:38:05.463110 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 20 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -570> 2014-08-13 17:38:05.463125 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 20, already have info from that osd, ignoring -569> 2014-08-13 17:38:05.463134 7fe99ebe8700 10 log is not dirty -568> 2014-08-13 17:38:05.463047 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.55 (past last_backfill 0//0//-1) -567> 2014-08-13 17:38:05.463138 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 337 notify: (query_epoch:194487, epoch_sent:194487, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194484 n=50 ec=192688 les/c 194484/194484 194485/194486/194485)) -566> 2014-08-13 17:38:05.463201 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.55 (past last_backfill 0//0//-1) -565> 2014-08-13 17:38:05.463289 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 337, already have info from that osd, ignoring -564> 2014-08-13 17:38:05.463434 7fe99ebe8700 10 log is not dirty -563> 2014-08-13 17:38:05.463440 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 327 notify: (query_epoch:194487, epoch_sent:194487, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194484 n=40 ec=183096 les/c 194484/194484 194485/194485/194485)) -562> 2014-08-13 17:38:05.463451 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194486/194484 194485/194485/194485) [4,289,341] r=0 lpr=194485 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 327, already have info from that osd, ignoring -561> 2014-08-13 17:38:05.463460 7fe99ebe8700 10 log is not dirty -560> 2014-08-13 17:38:05.463464 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 381 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -559> 2014-08-13 17:38:05.463476 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 381, already have info from that osd, ignoring -558> 2014-08-13 17:38:05.463484 7fe99ebe8700 10 log is not dirty -557> 2014-08-13 17:38:05.463489 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 369 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -556> 2014-08-13 17:38:05.463353 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.55 (past last_backfill 0//0//-1) -555> 2014-08-13 17:38:05.463500 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -554> 2014-08-13 17:38:05.463509 7fe99ebe8700 10 log is not dirty -553> 2014-08-13 17:38:05.463530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 349 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -552> 2014-08-13 17:38:05.463542 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 349, already have info from that osd, ignoring -551> 2014-08-13 17:38:05.463552 7fe99ebe8700 10 log is not dirty -550> 2014-08-13 17:38:05.463556 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -549> 2014-08-13 17:38:05.463567 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] requeue_ops -548> 2014-08-13 17:38:05.463577 7fe99ebe8700 10 log is not dirty -547> 2014-08-13 17:38:05.463582 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -546> 2014-08-13 17:38:05.463593 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring -545> 2014-08-13 17:38:05.463607 7fe99ebe8700 10 log is not dirty -544> 2014-08-13 17:38:05.463504 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.55 (past last_backfill 0//0//-1) -543> 2014-08-13 17:38:05.463611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -542> 2014-08-13 17:38:05.463662 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.55 (past last_backfill 0//0//-1) -541> 2014-08-13 17:38:05.463785 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 303, already have info from that osd, ignoring -540> 2014-08-13 17:38:05.463955 7fe99ebe8700 10 log is not dirty -539> 2014-08-13 17:38:05.463813 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.55 (past last_backfill 0//0//-1) -538> 2014-08-13 17:38:05.463961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 331 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) -537> 2014-08-13 17:38:05.463972 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 331, already have info from that osd, ignoring -536> 2014-08-13 17:38:05.463981 7fe99ebe8700 10 log is not dirty -535> 2014-08-13 17:38:05.463985 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 303 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -534> 2014-08-13 17:38:05.463995 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring -533> 2014-08-13 17:38:05.464003 7fe99ebe8700 10 log is not dirty -532> 2014-08-13 17:38:05.463964 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.55 (past last_backfill 0//0//-1) -531> 2014-08-13 17:38:05.464007 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 378 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -530> 2014-08-13 17:38:05.464116 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.55 (past last_backfill 0//0//-1) -529> 2014-08-13 17:38:05.464208 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.378 activated and committed -528> 2014-08-13 17:38:05.464387 7fe99ebe8700 10 log is not dirty -527> 2014-08-13 17:38:05.464271 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.55 (past last_backfill 0//0//-1) -526> 2014-08-13 17:38:05.464394 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 373 notify: (query_epoch:194487, epoch_sent:194487, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194484 n=59 ec=192688 les/c 194484/194484 194485/194486/194485)) -525> 2014-08-13 17:38:05.464423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.55 (past last_backfill 0//0//-1) -524> 2014-08-13 17:38:05.464574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.55 (past last_backfill 0//0//-1) -523> 2014-08-13 17:38:05.464566 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 373, already have info from that osd, ignoring -522> 2014-08-13 17:38:05.464734 7fe99ebe8700 10 log is not dirty -521> 2014-08-13 17:38:05.464739 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 342 notify: (query_epoch:194487, epoch_sent:194487, info:22.113( v 194215'528 (0'0,194215'528] local-les=194484 n=27 ec=183096 les/c 194484/194484 194485/194485/194485)) -520> 2014-08-13 17:38:05.464750 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194486/194484 194485/194485/194485) [4,339,45] r=0 lpr=194485 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring -519> 2014-08-13 17:38:05.464759 7fe99ebe8700 10 log is not dirty -518> 2014-08-13 17:38:05.464763 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 396 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) -517> 2014-08-13 17:38:05.464773 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 396, already have info from that osd, ignoring -516> 2014-08-13 17:38:05.464782 7fe99ebe8700 10 log is not dirty -515> 2014-08-13 17:38:05.464726 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.55 (past last_backfill 0//0//-1) -514> 2014-08-13 17:38:05.464786 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 375 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -513> 2014-08-13 17:38:05.464878 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.55 (past last_backfill 0//0//-1) -512> 2014-08-13 17:38:05.464955 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 375, already have info from that osd, ignoring -511> 2014-08-13 17:38:05.465122 7fe99ebe8700 10 log is not dirty -510> 2014-08-13 17:38:05.465127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 366 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -509> 2014-08-13 17:38:05.465137 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 366, already have info from that osd, ignoring -508> 2014-08-13 17:38:05.465146 7fe99ebe8700 10 log is not dirty -507> 2014-08-13 17:38:05.465030 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.55 (past last_backfill 0//0//-1) -506> 2014-08-13 17:38:05.465150 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 291 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -505> 2014-08-13 17:38:05.465185 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.55 (past last_backfill 0//0//-1) -504> 2014-08-13 17:38:05.465336 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.55 (past last_backfill 0//0//-1) -503> 2014-08-13 17:38:05.465333 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 291, already have info from that osd, ignoring -502> 2014-08-13 17:38:05.465509 7fe99ebe8700 10 log is not dirty -501> 2014-08-13 17:38:05.465488 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.55 (past last_backfill 0//0//-1) -500> 2014-08-13 17:38:05.465514 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 367 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -499> 2014-08-13 17:38:05.465640 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.55 (past last_backfill 0//0//-1) -498> 2014-08-13 17:38:05.465691 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.367 activated and committed -497> 2014-08-13 17:38:05.465867 7fe99ebe8700 10 log is not dirty -496> 2014-08-13 17:38:05.465878 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 373 notify: (query_epoch:194487, epoch_sent:194487, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194483 n=28 ec=183096 les/c 194483/194389 194485/194485/194485)) -495> 2014-08-13 17:38:05.465889 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194486/194389 194485/194485/194485) [4,338,376] r=0 lpr=194485 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 373, already have info from that osd, ignoring -494> 2014-08-13 17:38:05.465898 7fe99ebe8700 10 log is not dirty -493> 2014-08-13 17:38:05.465902 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 342 notify: (query_epoch:194487, epoch_sent:194487, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -492> 2014-08-13 17:38:05.465912 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,313,318] r=0 lpr=194485 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring -491> 2014-08-13 17:38:05.465921 7fe99ebe8700 10 log is not dirty -490> 2014-08-13 17:38:05.465792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.55 (past last_backfill 0//0//-1) -489> 2014-08-13 17:38:05.465924 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 375 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) -488> 2014-08-13 17:38:05.465943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.55 (past last_backfill 0//0//-1) -487> 2014-08-13 17:38:05.466093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.55 (past last_backfill 0//0//-1) -486> 2014-08-13 17:38:05.466094 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.375 activated and committed -485> 2014-08-13 17:38:05.466264 7fe99ebe8700 10 log is not dirty -484> 2014-08-13 17:38:05.466268 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 340 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -483> 2014-08-13 17:38:05.466280 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 340, already have info from that osd, ignoring -482> 2014-08-13 17:38:05.466290 7fe99ebe8700 10 log is not dirty -481> 2014-08-13 17:38:05.466249 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.55 (past last_backfill 0//0//-1) -480> 2014-08-13 17:38:05.466294 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -479> 2014-08-13 17:38:05.466401 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.55 (past last_backfill 0//0//-1) -478> 2014-08-13 17:38:05.466473 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops -477> 2014-08-13 17:38:05.466650 7fe99ebe8700 10 log is not dirty -476> 2014-08-13 17:38:05.466656 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 30 notify: (query_epoch:194487, epoch_sent:194487, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194478/194258 194459/194486/194486)) -475> 2014-08-13 17:38:05.466668 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 30, already have info from that osd, ignoring -474> 2014-08-13 17:38:05.466677 7fe99ebe8700 10 log is not dirty -473> 2014-08-13 17:38:05.466554 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.55 (past last_backfill 0//0//-1) -472> 2014-08-13 17:38:05.466699 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 340 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) -471> 2014-08-13 17:38:05.466711 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.340 activated and committed -470> 2014-08-13 17:38:05.466722 7fe99ebe8700 10 log is not dirty -469> 2014-08-13 17:38:05.466726 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 352 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) -468> 2014-08-13 17:38:05.466736 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 352, already have info from that osd, ignoring -467> 2014-08-13 17:38:05.466745 7fe99ebe8700 10 log is not dirty -466> 2014-08-13 17:38:05.466749 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 306 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194484 n=32 ec=183096 les/c 194484/194304 194485/194485/194485)) -465> 2014-08-13 17:38:05.466761 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 306, already have info from that osd, ignoring -464> 2014-08-13 17:38:05.466771 7fe99ebe8700 10 log is not dirty -463> 2014-08-13 17:38:05.466706 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.55 (past last_backfill 0//0//-1) -462> 2014-08-13 17:38:05.466774 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 10 notify: (query_epoch:194487, epoch_sent:194487, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194484 n=56 ec=192688 les/c 194484/194484 194485/194486/194485)) -461> 2014-08-13 17:38:05.466859 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.55 (past last_backfill 0//0//-1) -460> 2014-08-13 17:38:05.466944 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 10, already have info from that osd, ignoring -459> 2014-08-13 17:38:05.467111 7fe99ebe8700 10 log is not dirty -458> 2014-08-13 17:38:05.467116 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 362 notify: (query_epoch:194487, epoch_sent:194487, info:22.362( v 194215'479 (0'0,194215'479] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -457> 2014-08-13 17:38:05.467127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,58,343] r=0 lpr=194485 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 362, already have info from that osd, ignoring -456> 2014-08-13 17:38:05.467137 7fe99ebe8700 10 log is not dirty -455> 2014-08-13 17:38:05.467141 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 58 notify: (query_epoch:194487, epoch_sent:194487, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194483 n=28 ec=183096 les/c 194483/194483 194485/194485/194485)) -454> 2014-08-13 17:38:05.467151 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194486/194483 194485/194485/194485) [4,349,61] r=0 lpr=194485 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 58, already have info from that osd, ignoring -453> 2014-08-13 17:38:05.467160 7fe99ebe8700 10 log is not dirty -452> 2014-08-13 17:38:05.467011 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.55 (past last_backfill 0//0//-1) -451> 2014-08-13 17:38:05.467164 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 311 notify: (query_epoch:194487, epoch_sent:194487, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194483 n=29 ec=183096 les/c 194483/194483 194485/194485/194485)) -450> 2014-08-13 17:38:05.467175 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/194485) [4,312,285] r=0 lpr=194485 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 311, already have info from that osd, ignoring -449> 2014-08-13 17:38:05.467185 7fe99ebe8700 10 log is not dirty -448> 2014-08-13 17:38:05.467161 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.55 (past last_backfill 0//0//-1) -447> 2014-08-13 17:38:05.467189 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 365 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -446> 2014-08-13 17:38:05.467317 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.55 (past last_backfill 0//0//-1) -445> 2014-08-13 17:38:05.467371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 365, already have info from that osd, ignoring -444> 2014-08-13 17:38:05.467548 7fe99ebe8700 10 log is not dirty -443> 2014-08-13 17:38:05.467554 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 368 notify: (query_epoch:194487, epoch_sent:194487, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194483 n=38 ec=183096 les/c 194483/194483 194485/194485/194485)) -442> 2014-08-13 17:38:05.467565 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194486/194483 194485/194485/194485) [4,301,324] r=0 lpr=194485 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring -441> 2014-08-13 17:38:05.467574 7fe99ebe8700 10 log is not dirty -440> 2014-08-13 17:38:05.467577 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 365 notify: (query_epoch:194487, epoch_sent:194487, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194485/194485/194485)) -439> 2014-08-13 17:38:05.467587 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194486/194304 194485/194485/194485) [4,287,309] r=0 lpr=194485 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 365, already have info from that osd, ignoring -438> 2014-08-13 17:38:05.467596 7fe99ebe8700 10 log is not dirty -437> 2014-08-13 17:38:05.467600 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 30 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) -436> 2014-08-13 17:38:05.467469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.55 (past last_backfill 0//0//-1) -435> 2014-08-13 17:38:05.467611 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.30 activated and committed -434> 2014-08-13 17:38:05.467622 7fe99ebe8700 10 log is not dirty -433> 2014-08-13 17:38:05.467620 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.55 (past last_backfill 0//0//-1) -432> 2014-08-13 17:38:05.467626 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 350 notify: (query_epoch:194487, epoch_sent:194487, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194484 n=61 ec=192688 les/c 194484/194484 194485/194486/194485)) -431> 2014-08-13 17:38:05.467772 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.55 (past last_backfill 0//0//-1) -430> 2014-08-13 17:38:05.467803 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 350, already have info from that osd, ignoring -429> 2014-08-13 17:38:05.467979 7fe99ebe8700 10 log is not dirty -428> 2014-08-13 17:38:05.467924 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.55 (past last_backfill 0//0//-1) -427> 2014-08-13 17:38:05.467984 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -426> 2014-08-13 17:38:05.468076 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.55 (past last_backfill 0//0//-1) -425> 2014-08-13 17:38:05.468156 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops -424> 2014-08-13 17:38:05.468330 7fe99ebe8700 10 log is not dirty -423> 2014-08-13 17:38:05.468231 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.55 (past last_backfill 0//0//-1) -422> 2014-08-13 17:38:05.468336 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 350 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) -421> 2014-08-13 17:38:05.468383 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.55 (past last_backfill 0//0//-1) -420> 2014-08-13 17:38:05.468535 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.55 (past last_backfill 0//0//-1) -419> 2014-08-13 17:38:05.468513 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.350 activated and committed -418> 2014-08-13 17:38:05.468696 7fe99ebe8700 10 log is not dirty -417> 2014-08-13 17:38:05.468690 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.55 (past last_backfill 0//0//-1) -416> 2014-08-13 17:38:05.468701 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 20 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) -415> 2014-08-13 17:38:05.468842 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.55 (past last_backfill 0//0//-1) -414> 2014-08-13 17:38:05.468874 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.20 activated and committed -413> 2014-08-13 17:38:05.469044 7fe99ebe8700 10 log is not dirty -412> 2014-08-13 17:38:05.468994 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.55 (past last_backfill 0//0//-1) -411> 2014-08-13 17:38:05.469049 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 291 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) -410> 2014-08-13 17:38:05.469146 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.55 (past last_backfill 0//0//-1) -409> 2014-08-13 17:38:05.469229 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.291 activated and committed -408> 2014-08-13 17:38:05.469302 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.55 (past last_backfill 0//0//-1) -407> 2014-08-13 17:38:05.469405 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] all_activated_and_committed -406> 2014-08-13 17:38:05.469454 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.55 (past last_backfill 0//0//-1) -405> 2014-08-13 17:38:05.469579 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] share_pg_info -404> 2014-08-13 17:38:05.469606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.55 (past last_backfill 0//0//-1) -403> 2014-08-13 17:38:05.469759 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a423c0 don't know epoch, doing nothing -402> 2014-08-13 17:38:05.469762 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.291 192.168.1.101:6820/25633 -- pg_info(1 pgs e194487:24.9b) v4 -- ?+0 0x5f01180 -401> 2014-08-13 17:38:05.469756 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.296 192.168.1.109:6806/31286 4 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4db7080 con 0x4d791e0 -400> 2014-08-13 17:38:05.469775 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -399> 2014-08-13 17:38:05.469778 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -398> 2014-08-13 17:38:05.469779 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4db7080 osd_map(194487..194487 src has 185139..194487) v3 -397> 2014-08-13 17:38:05.469781 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x53809a0 don't know epoch, doing nothing -396> 2014-08-13 17:38:05.469785 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -395> 2014-08-13 17:38:05.469783 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.350 192.168.1.114:6832/53185 -- pg_info(1 pgs e194487:24.9b) v4 -- ?+0 0x5f00a80 -394> 2014-08-13 17:38:05.469789 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -393> 2014-08-13 17:38:05.469790 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -392> 2014-08-13 17:38:05.469791 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -391> 2014-08-13 17:38:05.469795 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e48c60 don't know epoch, doing nothing -390> 2014-08-13 17:38:05.469797 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194487:24.9b) v4 -- ?+0 0x5f01880 -389> 2014-08-13 17:38:05.469810 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eca940 don't know epoch, doing nothing -388> 2014-08-13 17:38:05.469812 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.378 192.168.1.102:6840/19195 -- pg_info(1 pgs e194487:24.9b) v4 -- ?+0 0x51ccb40 -387> 2014-08-13 17:38:05.469863 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.296 192.168.1.109:6806/31286 5 ==== pg_notify(24.181(33) epoch 194487) v5 ==== 2599+0+0 (418583000 0 0) 0x4c16e40 con 0x4d791e0 -386> 2014-08-13 17:38:05.469880 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -385> 2014-08-13 17:38:05.469887 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -384> 2014-08-13 17:38:05.469889 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16e40 pg_notify(24.181(33) epoch 194487) v5 -383> 2014-08-13 17:38:05.469893 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469774, event: header_read, request: pg_notify(24.181(33) epoch 194487) v5 -382> 2014-08-13 17:38:05.469899 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469776, event: throttled, request: pg_notify(24.181(33) epoch 194487) v5 -381> 2014-08-13 17:38:05.469903 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469847, event: all_read, request: pg_notify(24.181(33) epoch 194487) v5 -380> 2014-08-13 17:38:05.469756 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.55 (past last_backfill 0//0//-1) -379> 2014-08-13 17:38:05.469906 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469879, event: dispatched, request: pg_notify(24.181(33) epoch 194487) v5 -378> 2014-08-13 17:38:05.469913 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469913, event: waiting_for_osdmap, request: pg_notify(24.181(33) epoch 194487) v5 -377> 2014-08-13 17:38:05.469918 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.296 -376> 2014-08-13 17:38:05.469920 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16e40 -375> 2014-08-13 17:38:05.469923 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469923, event: started, request: pg_notify(24.181(33) epoch 194487) v5 -374> 2014-08-13 17:38:05.469939 7fe9a63f4700 5 -- op tracker -- , seq: 677, time: 2014-08-13 17:38:05.469939, event: done, request: pg_notify(24.181(33) epoch 194487) v5 -373> 2014-08-13 17:38:05.469948 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -372> 2014-08-13 17:38:05.469949 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -371> 2014-08-13 17:38:05.469824 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194487:25289 -370> 2014-08-13 17:38:05.470004 7fe99ebe8700 10 log is not dirty -369> 2014-08-13 17:38:05.469908 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.55 (past last_backfill 0//0//-1) -368> 2014-08-13 17:38:05.470010 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 303 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) -367> 2014-08-13 17:38:05.470058 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.55 (past last_backfill 0//0//-1) -366> 2014-08-13 17:38:05.470182 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.303 activated and committed -365> 2014-08-13 17:38:05.470357 7fe99ebe8700 10 log is not dirty -364> 2014-08-13 17:38:05.470212 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.55 (past last_backfill 0//0//-1) -363> 2014-08-13 17:38:05.470363 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 25 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -362> 2014-08-13 17:38:05.470363 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -361> 2014-08-13 17:38:05.470591 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.380 192.168.1.114:6822/51844 6 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x54db180 con 0x5250dc0 -360> 2014-08-13 17:38:05.470614 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -359> 2014-08-13 17:38:05.470619 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -358> 2014-08-13 17:38:05.470620 7fe9a63f4700 20 osd.4 194487 _dispatch 0x54db180 osd_map(194487..194487 src has 185139..194487) v3 -357> 2014-08-13 17:38:05.470626 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -356> 2014-08-13 17:38:05.470630 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -355> 2014-08-13 17:38:05.470631 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -354> 2014-08-13 17:38:05.470632 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -353> 2014-08-13 17:38:05.470549 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 61 as a source -352> 2014-08-13 17:38:05.470534 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.25 activated and committed -351> 2014-08-13 17:38:05.470702 7fe99ebe8700 10 log is not dirty -350> 2014-08-13 17:38:05.470746 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.380 192.168.1.114:6822/51844 7 ==== pg_notify(22.4df(27) epoch 194487) v5 ==== 2227+0+0 (4133003232 0 0) 0x4c16ac0 con 0x5250dc0 -349> 2014-08-13 17:38:05.470759 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -348> 2014-08-13 17:38:05.470761 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -347> 2014-08-13 17:38:05.470762 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4c16ac0 pg_notify(22.4df(27) epoch 194487) v5 -346> 2014-08-13 17:38:05.470765 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470666, event: header_read, request: pg_notify(22.4df(27) epoch 194487) v5 -345> 2014-08-13 17:38:05.470771 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470667, event: throttled, request: pg_notify(22.4df(27) epoch 194487) v5 -344> 2014-08-13 17:38:05.470774 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470734, event: all_read, request: pg_notify(22.4df(27) epoch 194487) v5 -343> 2014-08-13 17:38:05.470778 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470759, event: dispatched, request: pg_notify(22.4df(27) epoch 194487) v5 -342> 2014-08-13 17:38:05.470782 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470782, event: waiting_for_osdmap, request: pg_notify(22.4df(27) epoch 194487) v5 -341> 2014-08-13 17:38:05.470785 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.380 -340> 2014-08-13 17:38:05.470787 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x4c16ac0 -339> 2014-08-13 17:38:05.470791 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470790, event: started, request: pg_notify(22.4df(27) epoch 194487) v5 -338> 2014-08-13 17:38:05.470805 7fe9a63f4700 5 -- op tracker -- , seq: 678, time: 2014-08-13 17:38:05.470805, event: done, request: pg_notify(22.4df(27) epoch 194487) v5 -337> 2014-08-13 17:38:05.470813 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -336> 2014-08-13 17:38:05.470815 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -335> 2014-08-13 17:38:05.470702 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.61 (past last_backfill 0//0//-1) -334> 2014-08-13 17:38:05.470708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 362 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -333> 2014-08-13 17:38:05.470854 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.61 (past last_backfill 0//0//-1) -332> 2014-08-13 17:38:05.470877 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.362 activated and committed -331> 2014-08-13 17:38:05.471006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.61 (past last_backfill 0//0//-1) -330> 2014-08-13 17:38:05.471045 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] all_activated_and_committed -329> 2014-08-13 17:38:05.471158 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.61 (past last_backfill 0//0//-1) -328> 2014-08-13 17:38:05.471216 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] share_pg_info -327> 2014-08-13 17:38:05.471392 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4c71b80 already has epoch 194487 -326> 2014-08-13 17:38:05.471395 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194487:24.3d) v4 -- ?+0 0x5822a00 -325> 2014-08-13 17:38:05.471411 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4eccba0 don't know epoch, doing nothing -324> 2014-08-13 17:38:05.471413 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.25 192.168.1.103:6800/36920 -- pg_info(1 pgs e194487:24.3d) v4 -- ?+0 0x5822bc0 -323> 2014-08-13 17:38:05.471425 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e4d6a0 don't know epoch, doing nothing -322> 2014-08-13 17:38:05.471429 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.303 192.168.1.109:6802/30768 -- pg_info(1 pgs e194487:24.3d) v4 -- ?+0 0x5824980 -321> 2014-08-13 17:38:05.471442 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d7d280 don't know epoch, doing nothing -320> 2014-08-13 17:38:05.471445 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194487:24.3d) v4 -- ?+0 0x5822300 -319> 2014-08-13 17:38:05.471312 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.61 (past last_backfill 0//0//-1) -318> 2014-08-13 17:38:05.471465 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.61 (past last_backfill 0//0//-1) -317> 2014-08-13 17:38:05.471456 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194487:29706 -316> 2014-08-13 17:38:05.471630 7fe99ebe8700 10 log is not dirty -315> 2014-08-13 17:38:05.471636 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 0 notify: (query_epoch:194487, epoch_sent:194487, info:22.775( v 194215'513 (0'0,194215'513] local-les=194484 n=32 ec=183096 les/c 194484/194484 194485/194485/194485)) -314> 2014-08-13 17:38:05.471647 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194486/194484 194485/194485/194485) [4,353,56] r=0 lpr=194485 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring -313> 2014-08-13 17:38:05.471656 7fe99ebe8700 10 log is not dirty -312> 2014-08-13 17:38:05.471616 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.61 (past last_backfill 0//0//-1) -311> 2014-08-13 17:38:05.471684 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 286 info: 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -310> 2014-08-13 17:38:05.471768 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.61 (past last_backfill 0//0//-1) -309> 2014-08-13 17:38:05.471856 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.286 activated and committed -308> 2014-08-13 17:38:05.472027 7fe99ebe8700 10 log is not dirty -307> 2014-08-13 17:38:05.471920 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.61 (past last_backfill 0//0//-1) -306> 2014-08-13 17:38:05.472032 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 10 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) -305> 2014-08-13 17:38:05.472072 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.61 (past last_backfill 0//0//-1) -304> 2014-08-13 17:38:05.472203 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.10 activated and committed -303> 2014-08-13 17:38:05.472227 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.61 (past last_backfill 0//0//-1) -302> 2014-08-13 17:38:05.472377 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.61 (past last_backfill 0//0//-1) -301> 2014-08-13 17:38:05.472371 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] all_activated_and_committed -300> 2014-08-13 17:38:05.472530 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.61 (past last_backfill 0//0//-1) -299> 2014-08-13 17:38:05.472535 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] share_pg_info -298> 2014-08-13 17:38:05.472705 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x49e09a0 don't know epoch, doing nothing -297> 2014-08-13 17:38:05.472707 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.10 192.168.1.115:6815/60573 -- pg_info(1 pgs e194487:24.30b) v4 -- ?+0 0x5822840 -296> 2014-08-13 17:38:05.472723 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4df61a0 don't know epoch, doing nothing -295> 2014-08-13 17:38:05.472725 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194487:24.30b) v4 -- ?+0 0x5820e00 -294> 2014-08-13 17:38:05.472741 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4313f40 don't know epoch, doing nothing -293> 2014-08-13 17:38:05.472745 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.375 192.168.1.114:6830/52879 -- pg_info(1 pgs e194487:24.30b) v4 -- ?+0 0x5825780 -292> 2014-08-13 17:38:05.472682 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.61 (past last_backfill 0//0//-1) -291> 2014-08-13 17:38:05.472755 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194487:30297 -290> 2014-08-13 17:38:05.472934 7fe99ebe8700 10 log is not dirty -289> 2014-08-13 17:38:05.472940 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 26 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194486/194486) -288> 2014-08-13 17:38:05.472951 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.26 activated and committed -287> 2014-08-13 17:38:05.472961 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] all_activated_and_committed -286> 2014-08-13 17:38:05.472970 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] share_pg_info -285> 2014-08-13 17:38:05.472982 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x592c360 don't know epoch, doing nothing -284> 2014-08-13 17:38:05.472835 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.61 (past last_backfill 0//0//-1) -283> 2014-08-13 17:38:05.472984 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.26 192.168.1.115:6802/58522 -- pg_info(1 pgs e194487:24.34) v4 -- ?+0 0x5825940 -282> 2014-08-13 17:38:05.472998 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4a44780 already has epoch 194487 -281> 2014-08-13 17:38:05.473000 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194487:24.34) v4 -- ?+0 0x5824d00 -280> 2014-08-13 17:38:05.473020 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x49e40a0 don't know epoch, doing nothing -279> 2014-08-13 17:38:05.473022 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194487:24.34) v4 -- ?+0 0x5823d40 -278> 2014-08-13 17:38:05.473102 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] publish_stats_to_osd 194487:23844 -277> 2014-08-13 17:38:05.473128 7fe99ebe8700 10 log is not dirty -276> 2014-08-13 17:38:05.472986 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.61 (past last_backfill 0//0//-1) -275> 2014-08-13 17:38:05.473135 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -274> 2014-08-13 17:38:05.473136 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.61 (past last_backfill 0//0//-1) -273> 2014-08-13 17:38:05.473290 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -272> 2014-08-13 17:38:05.473437 7fe99ebe8700 10 log is not dirty -271> 2014-08-13 17:38:05.473290 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.61 (past last_backfill 0//0//-1) -270> 2014-08-13 17:38:05.473443 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -269> 2014-08-13 17:38:05.473457 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] requeue_ops -268> 2014-08-13 17:38:05.473467 7fe99ebe8700 10 log is not dirty -267> 2014-08-13 17:38:05.473442 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.61 (past last_backfill 0//0//-1) -266> 2014-08-13 17:38:05.473471 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 332 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) -265> 2014-08-13 17:38:05.473594 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.61 (past last_backfill 0//0//-1) -264> 2014-08-13 17:38:05.473617 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.332 activated and committed -263> 2014-08-13 17:38:05.473760 7fe99ebe8700 10 log is not dirty -262> 2014-08-13 17:38:05.473770 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -261> 2014-08-13 17:38:05.473784 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] requeue_ops -260> 2014-08-13 17:38:05.473797 7fe99ebe8700 10 log is not dirty -259> 2014-08-13 17:38:05.473801 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194484/194480 194485/194486/193250) [304,323,4] r=2 lpr=194486 pi=193250-194485/31 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) -258> 2014-08-13 17:38:05.473833 7fe99ebe8700 10 log is not dirty -257> 2014-08-13 17:38:05.473746 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.61 (past last_backfill 0//0//-1) -256> 2014-08-13 17:38:05.473848 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 337 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) -255> 2014-08-13 17:38:05.473900 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.61 (past last_backfill 0//0//-1) -254> 2014-08-13 17:38:05.473996 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.337 activated and committed -253> 2014-08-13 17:38:05.474140 7fe99ebe8700 10 log is not dirty -252> 2014-08-13 17:38:05.474146 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194484/194480 194485/194486/192967) [6,374,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) -251> 2014-08-13 17:38:05.474174 7fe99ebe8700 10 log is not dirty -250> 2014-08-13 17:38:05.474186 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -249> 2014-08-13 17:38:05.474052 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.61 (past last_backfill 0//0//-1) -248> 2014-08-13 17:38:05.474208 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] requeue_ops -247> 2014-08-13 17:38:05.474219 7fe99ebe8700 10 log is not dirty -246> 2014-08-13 17:38:05.474224 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -245> 2014-08-13 17:38:05.474236 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] requeue_ops -244> 2014-08-13 17:38:05.474245 7fe99ebe8700 10 log is not dirty -243> 2014-08-13 17:38:05.474206 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.61 (past last_backfill 0//0//-1) -242> 2014-08-13 17:38:05.474249 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 348 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -241> 2014-08-13 17:38:05.474358 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.61 (past last_backfill 0//0//-1) -240> 2014-08-13 17:38:05.474432 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.348 activated and committed -239> 2014-08-13 17:38:05.474580 7fe99ebe8700 10 log is not dirty -238> 2014-08-13 17:38:05.474586 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194484/194480 194485/194486/193188) [17,4,395] r=1 lpr=194486 pi=193188-194485/37 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) -237> 2014-08-13 17:38:05.474617 7fe99ebe8700 10 log is not dirty -236> 2014-08-13 17:38:05.474510 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.61 (past last_backfill 0//0//-1) -235> 2014-08-13 17:38:05.474630 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 331 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194487/194484 194485/194486/194485) -234> 2014-08-13 17:38:05.474662 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.61 (past last_backfill 0//0//-1) -233> 2014-08-13 17:38:05.474785 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.331 activated and committed -232> 2014-08-13 17:38:05.474815 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.61 (past last_backfill 0//0//-1) -231> 2014-08-13 17:38:05.474928 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] all_activated_and_committed -230> 2014-08-13 17:38:05.474966 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.61 (past last_backfill 0//0//-1) -229> 2014-08-13 17:38:05.475071 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] share_pg_info -228> 2014-08-13 17:38:05.475223 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e83180 don't know epoch, doing nothing -227> 2014-08-13 17:38:05.475226 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.331 192.168.1.104:6816/837 -- pg_info(1 pgs e194487:24.1bb) v4 -- ?+0 0x58216c0 -226> 2014-08-13 17:38:05.475244 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4e3b020 don't know epoch, doing nothing -225> 2014-08-13 17:38:05.475246 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.332 192.168.1.110:6806/42024 -- pg_info(1 pgs e194487:24.1bb) v4 -- ?+0 0x5826c80 -224> 2014-08-13 17:38:05.475258 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x4d3c780 already has epoch 194487 -223> 2014-08-13 17:38:05.475260 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194487:24.1bb) v4 -- ?+0 0x5825240 -222> 2014-08-13 17:38:05.475117 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.61 (past last_backfill 0//0//-1) -221> 2014-08-13 17:38:05.475272 7fe99ebe8700 20 osd.4 194487 _share_map_outgoing 0x5b96720 don't know epoch, doing nothing -220> 2014-08-13 17:38:05.475274 7fe99ebe8700 1 -- 192.168.1.115:6813/34805 --> osd.348 192.168.1.110:6800/41204 -- pg_info(1 pgs e194487:24.1bb) v4 -- ?+0 0x58255c0 -219> 2014-08-13 17:38:05.475270 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.61 (past last_backfill 0//0//-1) -218> 2014-08-13 17:38:05.475284 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194487/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194486 pi=193644-194485/32 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194487:22151 -217> 2014-08-13 17:38:05.475432 7fe99ebe8700 10 log is not dirty -216> 2014-08-13 17:38:05.475447 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194484/194480 194485/194486/193186) [45,319,4] r=2 lpr=194486 pi=193186-194485/31 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) -215> 2014-08-13 17:38:05.475477 7fe99ebe8700 10 log is not dirty -214> 2014-08-13 17:38:05.475491 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -213> 2014-08-13 17:38:05.475506 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] requeue_ops -212> 2014-08-13 17:38:05.475517 7fe99ebe8700 10 log is not dirty -211> 2014-08-13 17:38:05.475521 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194484/194480 194485/194486/192975) [285,336,4] r=2 lpr=194486 pi=193125-194485/31 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) -210> 2014-08-13 17:38:05.475547 7fe99ebe8700 10 log is not dirty -209> 2014-08-13 17:38:05.475559 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -208> 2014-08-13 17:38:05.475423 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.61 (past last_backfill 0//0//-1) -207> 2014-08-13 17:38:05.475574 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] requeue_ops -206> 2014-08-13 17:38:05.475586 7fe99ebe8700 10 log is not dirty -205> 2014-08-13 17:38:05.475591 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 363 notify: (query_epoch:194487, epoch_sent:194487, info:22.99( v 194215'504 (0'0,194215'504] local-les=194484 n=29 ec=183096 les/c 194484/194484 194485/194485/194485)) -204> 2014-08-13 17:38:05.475602 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194486/194484 194485/194485/194485) [4,374,31] r=0 lpr=194485 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring -203> 2014-08-13 17:38:05.475612 7fe99ebe8700 10 log is not dirty -202> 2014-08-13 17:38:05.475632 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194484/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) -201> 2014-08-13 17:38:05.475660 7fe99ebe8700 10 log is not dirty -200> 2014-08-13 17:38:05.475673 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -199> 2014-08-13 17:38:05.475685 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] requeue_ops -198> 2014-08-13 17:38:05.475695 7fe99ebe8700 10 log is not dirty -197> 2014-08-13 17:38:05.475699 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 363 notify: (query_epoch:194487, epoch_sent:194487, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194484 n=23 ec=183096 les/c 194484/194484 194485/194485/194485)) -196> 2014-08-13 17:38:05.475710 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194486/194484 194485/194485/194485) [4,373,344] r=0 lpr=194485 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring -195> 2014-08-13 17:38:05.475719 7fe99ebe8700 10 log is not dirty -194> 2014-08-13 17:38:05.475574 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.61 (past last_backfill 0//0//-1) -193> 2014-08-13 17:38:05.475723 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194484/194480 194485/194486/193262) [343,45,4] r=2 lpr=194486 pi=193262-194485/31 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) -192> 2014-08-13 17:38:05.475749 7fe99ebe8700 10 log is not dirty -191> 2014-08-13 17:38:05.475758 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -190> 2014-08-13 17:38:05.475770 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] requeue_ops -189> 2014-08-13 17:38:05.475781 7fe99ebe8700 10 log is not dirty -188> 2014-08-13 17:38:05.475784 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 RequestRecovery -187> 2014-08-13 17:38:05.475796 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] exit Started/ReplicaActive/RepNotRecovering 0.019173 3 0.000844 -186> 2014-08-13 17:38:05.475809 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194486 pi=193227-194485/39 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] enter Started/ReplicaActive/RepWaitRecoveryReserved -185> 2014-08-13 17:38:05.475823 7fe99ebe8700 10 log is not dirty -184> 2014-08-13 17:38:05.475827 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194484/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194486 pi=193125-194485/33 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) -183> 2014-08-13 17:38:05.475853 7fe99ebe8700 10 log is not dirty -182> 2014-08-13 17:38:05.475862 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -181> 2014-08-13 17:38:05.475725 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.61 (past last_backfill 0//0//-1) -180> 2014-08-13 17:38:05.475875 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] requeue_ops -179> 2014-08-13 17:38:05.475887 7fe99ebe8700 10 log is not dirty -178> 2014-08-13 17:38:05.475891 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt -177> 2014-08-13 17:38:05.475901 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] requeue_ops -176> 2014-08-13 17:38:05.475911 7fe99ebe8700 10 log is not dirty -175> 2014-08-13 17:38:05.475915 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194484/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194486 pi=193125-194485/34 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) -174> 2014-08-13 17:38:05.475943 7fe99ebe8700 10 log is not dirty -173> 2014-08-13 17:38:05.475952 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194486 pi=192688-194485/48 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) -172> 2014-08-13 17:38:05.475979 7fe99ebe8700 10 log is not dirty -171> 2014-08-13 17:38:05.475876 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.61 (past last_backfill 0//0//-1) -170> 2014-08-13 17:38:05.475988 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated -169> 2014-08-13 17:38:05.476029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.61 (past last_backfill 0//0//-1) -168> 2014-08-13 17:38:05.476182 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.61 (past last_backfill 0//0//-1) -167> 2014-08-13 17:38:05.476173 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] check_local -166> 2014-08-13 17:38:05.476335 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.61 (past last_backfill 0//0//-1) -165> 2014-08-13 17:38:05.476354 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] requeue_ops -164> 2014-08-13 17:38:05.476487 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.61 (past last_backfill 0//0//-1) -163> 2014-08-13 17:38:05.476530 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] needs_recovery primary has 59 missing -162> 2014-08-13 17:38:05.476638 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.61 (past last_backfill 0//0//-1) -161> 2014-08-13 17:38:05.476708 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] activate not all replicas are up-to-date, queueing recovery -160> 2014-08-13 17:38:05.476790 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.61 (past last_backfill 0//0//-1) -159> 2014-08-13 17:38:05.476942 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.61 (past last_backfill 0//0//-1) -158> 2014-08-13 17:38:05.477053 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] publish_stats_to_osd 194487:25290 -157> 2014-08-13 17:38:05.477093 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.61 (past last_backfill 0//0//-1) -156> 2014-08-13 17:38:05.477248 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.61 (past last_backfill 0//0//-1) -155> 2014-08-13 17:38:05.477232 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 -154> 2014-08-13 17:38:05.477432 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.59 192.168.1.112:6822/5265 5 ==== osd_map(194487..194487 src has 185139..194487) v3 ==== 326+0+0 (879390508 0 0) 0x4d9b840 con 0x59cf640 -153> 2014-08-13 17:38:05.477450 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -152> 2014-08-13 17:38:05.477453 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -151> 2014-08-13 17:38:05.477454 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d9b840 osd_map(194487..194487 src has 185139..194487) v3 -150> 2014-08-13 17:38:05.477460 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] -149> 2014-08-13 17:38:05.477463 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping -148> 2014-08-13 17:38:05.477464 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -147> 2014-08-13 17:38:05.477465 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -146> 2014-08-13 17:38:05.477470 7fe9a63f4700 1 -- 192.168.1.115:6813/34805 <== osd.59 192.168.1.112:6822/5265 6 ==== pg_info(1 pgs e194487:24.1f7) v4 ==== 724+0+0 (3738759013 0 0) 0x5825400 con 0x59cf640 -145> 2014-08-13 17:38:05.477477 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -144> 2014-08-13 17:38:05.477478 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -143> 2014-08-13 17:38:05.477479 7fe9a63f4700 20 osd.4 194487 _dispatch 0x5825400 pg_info(1 pgs e194487:24.1f7) v4 -142> 2014-08-13 17:38:05.477482 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477413, event: header_read, request: pg_info(1 pgs e194487:24.1f7) v4 -141> 2014-08-13 17:38:05.477491 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477414, event: throttled, request: pg_info(1 pgs e194487:24.1f7) v4 -140> 2014-08-13 17:38:05.477494 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477448, event: all_read, request: pg_info(1 pgs e194487:24.1f7) v4 -139> 2014-08-13 17:38:05.477498 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477477, event: dispatched, request: pg_info(1 pgs e194487:24.1f7) v4 -138> 2014-08-13 17:38:05.477502 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477502, event: waiting_for_osdmap, request: pg_info(1 pgs e194487:24.1f7) v4 -137> 2014-08-13 17:38:05.477505 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.1f7) v4 from osd.59 -136> 2014-08-13 17:38:05.477508 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x5825400 -135> 2014-08-13 17:38:05.477512 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477511, event: started, request: pg_info(1 pgs e194487:24.1f7) v4 -134> 2014-08-13 17:38:05.477527 7fe9a63f4700 5 -- op tracker -- , seq: 679, time: 2014-08-13 17:38:05.477527, event: done, request: pg_info(1 pgs e194487:24.1f7) v4 -133> 2014-08-13 17:38:05.477535 7fe9a63f4700 10 osd.4 194487 do_waiters -- start -132> 2014-08-13 17:38:05.477537 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish -131> 2014-08-13 17:38:05.477400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.61 (past last_backfill 0//0//-1) -130> 2014-08-13 17:38:05.477407 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 -129> 2014-08-13 17:38:05.477551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.61 (past last_backfill 0//0//-1) -128> 2014-08-13 17:38:05.477581 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 -127> 2014-08-13 17:38:05.477703 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.61 (past last_backfill 0//0//-1) -126> 2014-08-13 17:38:05.477760 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -125> 2014-08-13 17:38:05.477855 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.61 (past last_backfill 0//0//-1) -124> 2014-08-13 17:38:05.477969 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create target_size 1000 fpp 0.05 -123> 2014-08-13 17:38:05.478006 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.61 (past last_backfill 0//0//-1) -122> 2014-08-13 17:38:05.478157 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.61 (past last_backfill 0//0//-1) -121> 2014-08-13 17:38:05.478311 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.61 (past last_backfill 0//0//-1) -120> 2014-08-13 17:38:05.478361 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_apply_log 194422'17302 .. 194422'17302 -119> 2014-08-13 17:38:05.478463 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -118> 2014-08-13 17:38:05.478553 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_setup allocated new state, position 7d546c9b//0//24 -117> 2014-08-13 17:38:05.478649 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 65 as a source -116> 2014-08-13 17:38:05.478735 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode: num_objects: 61 num_bytes: 251658355 num_objects_dirty: 60 num_objects_omap: 0 num_dirty: 60 num_user_objects: 60 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -115> 2014-08-13 17:38:05.478801 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.65 -114> 2014-08-13 17:38:05.478913 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194487/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194486 pi=193637-194485/31 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode dirty 0.061475 full 0.061475 -113> 2014-08-13 17:38:05.479094 7fe99ebe8700 10 log is not dirty -112> 2014-08-13 17:38:05.478955 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.65 -111> 2014-08-13 17:38:05.479101 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 MNotifyRec from 380 notify: (query_epoch:194487, epoch_sent:194487, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194484 n=35 ec=183096 les/c 194484/194484 194485/194485/194485)) -110> 2014-08-13 17:38:05.479114 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194486/194484 194485/194485/194485) [4,356,357] r=0 lpr=194485 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 380, already have info from that osd, ignoring -109> 2014-08-13 17:38:05.479124 7fe99ebe8700 10 log is not dirty -108> 2014-08-13 17:38:05.479106 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.65 -107> 2014-08-13 17:38:05.479127 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated -106> 2014-08-13 17:38:05.479262 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.65 -105> 2014-08-13 17:38:05.479306 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] check_local -104> 2014-08-13 17:38:05.479415 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.65 -103> 2014-08-13 17:38:05.479476 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] requeue_ops -102> 2014-08-13 17:38:05.479568 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.65 -101> 2014-08-13 17:38:05.479644 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] needs_recovery primary has 57 missing -100> 2014-08-13 17:38:05.479721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.65 -99> 2014-08-13 17:38:05.479814 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] activate not all replicas are up-to-date, queueing recovery -98> 2014-08-13 17:38:05.479873 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.65 -97> 2014-08-13 17:38:05.480026 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.65 -96> 2014-08-13 17:38:05.480144 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] publish_stats_to_osd 194487:29707 -95> 2014-08-13 17:38:05.480181 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.65 -94> 2014-08-13 17:38:05.480333 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.65 -93> 2014-08-13 17:38:05.480317 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 -92> 2014-08-13 17:38:05.480486 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.65 -91> 2014-08-13 17:38:05.480485 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 -90> 2014-08-13 17:38:05.480639 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.65 -89> 2014-08-13 17:38:05.480652 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 -88> 2014-08-13 17:38:05.480792 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.65 -87> 2014-08-13 17:38:05.480824 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -86> 2014-08-13 17:38:05.480944 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.65 -85> 2014-08-13 17:38:05.480997 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create target_size 1000 fpp 0.05 -84> 2014-08-13 17:38:05.481097 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.65 -83> 2014-08-13 17:38:05.481252 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.65 -82> 2014-08-13 17:38:05.481346 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_apply_log 194422'18731 .. 194422'18731 -81> 2014-08-13 17:38:05.481405 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.65 -80> 2014-08-13 17:38:05.481527 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_setup allocated new state, position ec6b743d//0//24 -79> 2014-08-13 17:38:05.481558 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.65 -78> 2014-08-13 17:38:05.481709 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.65 -77> 2014-08-13 17:38:05.481697 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode: num_objects: 59 num_bytes: 243269718 num_objects_dirty: 58 num_objects_omap: 0 num_dirty: 58 num_user_objects: 58 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -76> 2014-08-13 17:38:05.481861 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.65 -75> 2014-08-13 17:38:05.481866 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194486 pi=193635-194485/33 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode dirty 0.059426 full 0.059426 -74> 2014-08-13 17:38:05.482040 7fe99ebe8700 10 log is not dirty -73> 2014-08-13 17:38:05.482019 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.65 -72> 2014-08-13 17:38:05.482047 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated -71> 2014-08-13 17:38:05.482171 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.65 -70> 2014-08-13 17:38:05.482222 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] check_local -69> 2014-08-13 17:38:05.482327 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.65 -68> 2014-08-13 17:38:05.482389 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] requeue_ops -67> 2014-08-13 17:38:05.482480 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.65 -66> 2014-08-13 17:38:05.482554 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] needs_recovery primary has 56 missing -65> 2014-08-13 17:38:05.482633 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.65 -64> 2014-08-13 17:38:05.482719 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] activate not all replicas are up-to-date, queueing recovery -63> 2014-08-13 17:38:05.482785 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.65 -62> 2014-08-13 17:38:05.482938 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.65 -61> 2014-08-13 17:38:05.483043 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] publish_stats_to_osd 194487:30298 -60> 2014-08-13 17:38:05.483090 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.65 -59> 2014-08-13 17:38:05.483212 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] on activate: bft=286 from 0//0//-1 -58> 2014-08-13 17:38:05.483247 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.65 -57> 2014-08-13 17:38:05.483378 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] target shard 286 from 0//0//-1 -56> 2014-08-13 17:38:05.483400 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.65 -55> 2014-08-13 17:38:05.483551 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.65 -54> 2014-08-13 17:38:05.483547 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -53> 2014-08-13 17:38:05.483703 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.65 -52> 2014-08-13 17:38:05.483715 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_create target_size 1000 fpp 0.05 -51> 2014-08-13 17:38:05.483856 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.65 -50> 2014-08-13 17:38:05.484009 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.65 -49> 2014-08-13 17:38:05.484056 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] hit_set_apply_log 194422'18660 .. 194422'18660 -48> 2014-08-13 17:38:05.484162 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.65 -47> 2014-08-13 17:38:05.484237 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_setup allocated new state, position 8ec5770b//0//24 -46> 2014-08-13 17:38:05.484317 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.65 -45> 2014-08-13 17:38:05.484404 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_choose_mode: num_objects: 56 num_bytes: 230686805 num_objects_dirty: 55 num_objects_omap: 0 num_dirty: 55 num_user_objects: 55 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -44> 2014-08-13 17:38:05.484469 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.65 -43> 2014-08-13 17:38:05.484571 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194486 pi=193640-194485/36 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 active+remapped m=56] agent_choose_mode dirty 0.056352 full 0.056352 -42> 2014-08-13 17:38:05.484741 7fe99ebe8700 10 log is not dirty -41> 2014-08-13 17:38:05.484747 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 AllReplicasActivated -40> 2014-08-13 17:38:05.484760 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] check_local -39> 2014-08-13 17:38:05.484622 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.65 -38> 2014-08-13 17:38:05.484769 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] requeue_ops -37> 2014-08-13 17:38:05.484779 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_recovery is recovered -36> 2014-08-13 17:38:05.484789 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.26 has last_backfill 0//0//-1 -35> 2014-08-13 17:38:05.484798 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.30 has last_backfill 0//0//-1 -34> 2014-08-13 17:38:05.484808 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.340 has last_backfill 0//0//-1 -33> 2014-08-13 17:38:05.484817 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] activate queueing backfill -32> 2014-08-13 17:38:05.484831 7fe99ebe8700 15 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] publish_stats_to_osd 194487:23845 -31> 2014-08-13 17:38:05.484841 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 -30> 2014-08-13 17:38:05.484851 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 -29> 2014-08-13 17:38:05.484860 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 -28> 2014-08-13 17:38:05.484869 7fe99ebe8700 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 -27> 2014-08-13 17:38:05.484881 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -26> 2014-08-13 17:38:05.484893 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create target_size 1000 fpp 0.05 -25> 2014-08-13 17:38:05.484773 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.65 -24> 2014-08-13 17:38:05.484925 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.65 -23> 2014-08-13 17:38:05.485075 7fe99ebe8700 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_apply_log 194215'14514 .. 194215'14514 -22> 2014-08-13 17:38:05.485087 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_setup allocated new state, position 3be28434//0//24 -21> 2014-08-13 17:38:05.485102 7fe99ebe8700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate -20> 2014-08-13 17:38:05.485105 7fe99ebe8700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194486 pi=193125-194485/52 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_choose_mode: num_objects: -1 num_bytes: -9223372036854775808 num_objects_dirty: -2 num_objects_omap: 0 num_dirty: 0 num_user_objects: 18446744073709551615 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -19> 2014-08-13 17:38:05.485077 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.65 -18> 2014-08-13 17:38:05.485236 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.65 -17> 2014-08-13 17:38:05.485389 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.65 -16> 2014-08-13 17:38:05.485542 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.65 -15> 2014-08-13 17:38:05.485694 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.65 -14> 2014-08-13 17:38:05.485847 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.65 -13> 2014-08-13 17:38:05.485999 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.65 -12> 2014-08-13 17:38:05.486151 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.65 -11> 2014-08-13 17:38:05.486306 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.65 -10> 2014-08-13 17:38:05.486459 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.65 -9> 2014-08-13 17:38:05.486612 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -8> 2014-08-13 17:38:05.486799 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound -7> 2014-08-13 17:38:05.486950 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _calc_past_interval_range: already have past intervals back to 194484 -6> 2014-08-13 17:38:05.487104 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound: built 50,55,61,65,356 -5> 2014-08-13 17:38:05.487260 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Activate Finished -4> 2014-08-13 17:38:05.487414 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating -3> 2014-08-13 17:38:05.487569 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 -2> 2014-08-13 17:38:05.487721 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt -1> 2014-08-13 17:38:05.487924 7fe99f5e9700 10 log is not dirty 0> 2014-08-13 17:38:05.488048 7fe99ebe8700 -1 *** Caught signal (Floating point exception) ** in thread 7fe99ebe8700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7fe9b9e35710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7fe9b9e2d9d1] 16: (clone()+0x6d) [0x7fe9b8dbdb6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 20/20 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.4.log --- end dump of recent events --- 2014-08-13 17:38:05.488186 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 17:38:05.488214 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488226 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.488239 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488253 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488255 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488257 7fe9a63f4700 20 osd.4 194487 _dispatch 0x6010000 pg_info(1 pgs e194487:24.136) v4 2014-08-13 17:38:05.488251 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.488262 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.488271 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488285 7fe9a63f4700 7 osd.4 194487 handle_pg_info pg_info(1 pgs e194487:24.136) v4 from osd.55 2014-08-13 17:38:05.488288 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x6010000 2014-08-13 17:38:05.488280 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488294 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488319 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488310 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 17:38:05.488321 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488323 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488327 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.488336 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488345 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.488354 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.488362 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488370 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194485 pi=192977-194484/29 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488379 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488396 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,32] -- 356/356 2014-08-13 17:38:05.488414 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488414 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488416 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488417 7fe9a63f4700 20 osd.4 194487 _dispatch 0x4d94140 osd_map(194487..194487 src has 185139..194487) v3 2014-08-13 17:38:05.488423 7fe9a63f4700 3 osd.4 194487 handle_osd_map epochs [194487,194487], i have 194487, src has [185139,194487] 2014-08-13 17:38:05.488426 7fe9a63f4700 10 osd.4 194487 no new maps here, dropping 2014-08-13 17:38:05.488419 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Started advmap 2014-08-13 17:38:05.488428 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488429 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488429 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488441 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488442 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488443 7fe9a63f4700 20 osd.4 194487 _dispatch 0x51c81c0 pg_notify(22.619(1) epoch 194487) v5 2014-08-13 17:38:05.488440 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 17:38:05.488465 7fe9a63f4700 7 osd.4 194487 handle_pg_notify from osd.44 2014-08-13 17:38:05.488467 7fe9a63f4700 15 osd.4 194487 require_same_or_newer_map 194487 (i am 194487) 0x51c81c0 2014-08-13 17:38:05.488494 7fe9a63f4700 10 osd.4 194487 do_waiters -- start 2014-08-13 17:38:05.488496 7fe9a63f4700 10 osd.4 194487 do_waiters -- finish 2014-08-13 17:38:05.488517 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 17:38:05.488528 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488537 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194478 n=56 ec=192688 les/c 194484/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194486 pi=193150-194485/39 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488548 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488562 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 17:38:05.488575 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488578 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.488588 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488597 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.488606 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.488614 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488623 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194485 pi=192965-194484/24 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488632 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488482 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _activate_committed 194487 peer_activated now 4 last_epoch_started 194484 same_interval_since 194486 2014-08-13 17:38:05.488647 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 17:38:05.488660 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488663 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap 2014-08-13 17:38:05.488673 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488682 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map 2014-08-13 17:38:05.488690 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters 2014-08-13 17:38:05.488698 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488706 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194485 pi=192976-194484/24 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488716 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488731 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 17:38:05.488743 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488747 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 17:38:05.488757 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 17:38:05.488766 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488774 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 17:38:05.488782 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 17:38:05.488791 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 17:38:05.488800 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194487:2707 2014-08-13 17:38:05.488655 7fe9aad09700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194486 pi=193747-194485/32 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] flushed 2014-08-13 17:38:05.488808 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 17:38:05.488817 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 17:38:05.488824 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194485 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 17:38:05.488834 7fe99f5e9700 10 log is not dirty 2014-08-13 17:38:05.488847 7fe99f5e9700 10 osd.4 pg_epoch: 194486 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_advance_map [4,344,296]/[4,296,320] -- 4/4 2014-08-13 17:38:05.488865 7fe99f5e9700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 17:38:05.488870 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Peering advmap 2014-08-13 17:38:05.488881 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] adjust_need_up_thru now 194486, need_up_thru now false 2014-08-13 17:38:05.488891 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Started advmap 2014-08-13 17:38:05.488901 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] check_recovery_sources no source osds () went down 2014-08-13 17:38:05.488912 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_activate_map 2014-08-13 17:38:05.488922 7fe99f5e9700 7 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle ActMap primary 2014-08-13 17:38:05.488933 7fe99f5e9700 15 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] publish_stats_to_osd 194487:28884 2014-08-13 17:38:05.488942 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] take_waiters 2014-08-13 17:38:05.488954 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/WaitUpThru 0.999392 4 0.000416 2014-08-13 17:38:05.488967 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Leaving Peering 2014-08-13 17:38:05.488977 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering 1.123361 0 0.000000 2014-08-13 17:38:05.488992 7fe99f5e9700 5 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active 2014-08-13 17:38:05.489002 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194478 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: In Active, about to call activate 2014-08-13 17:38:05.489015 7fe99f5e9700 20 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 17:38:05.489025 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - snap_trimq [] 2014-08-13 17:38:05.489034 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - not complete, missing(47) 2014-08-13 17:38:05.489767 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - starting recovery 2014-08-13 17:38:05.489784 7fe99f5e9700 10 osd.4 194487 queue_for_recovery queued pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] 2014-08-13 17:38:05.489806 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:05.489819 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 is up to date, queueing in pending_activators 2014-08-13 17:38:05.489869 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) uptodate 2014-08-13 17:38:05.489882 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:05.489893 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 is up to date, queueing in pending_activators 2014-08-13 17:38:05.489931 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194484 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) uptodate 2014-08-13 17:38:05.489943 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) 2014-08-13 17:38:05.489958 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 is up to date, queueing in pending_activators 2014-08-13 17:38:05.489991 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194484/194466 194485/194486/194486) uptodate 2014-08-13 17:38:05.490029 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] needs_recovery primary has 47 missing 2014-08-13 17:38:05.490091 7fe99f5e9700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194486 pi=193679-194485/37 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 also missing on osd.4 2014-08-13 18:06:05.939966 7f5c973a57a0 0 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process ceph-osd, pid 37708 2014-08-13 18:06:05.947443 7f5c973a57a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 18:06:05.947477 7f5c973a57a0 1 filestore(/var/lib/ceph/osd/ceph-4) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-08-13 18:06:05.950098 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 18:06:05.950117 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 18:06:05.966838 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 18:06:05.966966 7f5c973a57a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 18:06:05.984529 7f5c973a57a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-13 18:06:05.989945 7f5c973a57a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 18:06:06.188103 7f5c973a57a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 20: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 18:06:06.200966 7f5c973a57a0 1 journal close /var/lib/ceph/osd/ceph-4/journal 2014-08-13 18:06:06.202752 7f5c973a57a0 2 osd.4 0 mounting /var/lib/ceph/osd/ceph-4 /var/lib/ceph/osd/ceph-4/journal 2014-08-13 18:06:06.202844 7f5c973a57a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount detected xfs (libxfs) 2014-08-13 18:06:06.205171 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-13 18:06:06.205190 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-13 18:06:06.217383 7f5c973a57a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-08-13 18:06:06.217464 7f5c973a57a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-4) detect_feature: extsize is disabled by conf 2014-08-13 18:06:06.231681 7f5c973a57a0 0 filestore(/var/lib/ceph/osd/ceph-4) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-13 18:06:06.237056 7f5c973a57a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 18:06:06.330716 7f5c973a57a0 1 journal _open /var/lib/ceph/osd/ceph-4/journal fd 21: 2146435072 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-08-13 18:06:06.331085 7f5c973a57a0 2 osd.4 0 boot 2014-08-13 18:06:06.331331 7f5c973a57a0 10 osd.4 0 read_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194487 [185139,194487] lci=[194485,194487]) 2014-08-13 18:06:06.331579 7f5c973a57a0 10 open_all_classes 2014-08-13 18:06:06.331652 7f5c973a57a0 10 open_all_classes found rbd 2014-08-13 18:06:06.331671 7f5c973a57a0 10 _get_class adding new class name rbd 0x20c8028 2014-08-13 18:06:06.331677 7f5c973a57a0 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so 2014-08-13 18:06:06.332513 7f5c973a57a0 10 register_class rbd status 3 2014-08-13 18:06:06.332529 7f5c973a57a0 10 register_cxx_method rbd.create flags 3 0x7f5c866178b0 2014-08-13 18:06:06.332538 7f5c973a57a0 10 register_cxx_method rbd.get_features flags 1 0x7f5c8661c830 2014-08-13 18:06:06.332544 7f5c973a57a0 10 register_cxx_method rbd.get_size flags 1 0x7f5c8661dd50 2014-08-13 18:06:06.332548 7f5c973a57a0 10 register_cxx_method rbd.set_size flags 3 0x7f5c8661cbd0 2014-08-13 18:06:06.332551 7f5c973a57a0 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f5c866188b0 2014-08-13 18:06:06.332555 7f5c973a57a0 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f5c8661c330 2014-08-13 18:06:06.332559 7f5c973a57a0 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f5c8661b070 2014-08-13 18:06:06.332564 7f5c973a57a0 10 register_cxx_method rbd.snapshot_add flags 3 0x7f5c8661d1a0 2014-08-13 18:06:06.332568 7f5c973a57a0 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f5c8661ae20 2014-08-13 18:06:06.332571 7f5c973a57a0 10 register_cxx_method rbd.get_all_features flags 1 0x7f5c866143f0 2014-08-13 18:06:06.332575 7f5c973a57a0 10 register_cxx_method rbd.copyup flags 3 0x7f5c86614370 2014-08-13 18:06:06.332578 7f5c973a57a0 10 register_cxx_method rbd.get_parent flags 1 0x7f5c8661e1c0 2014-08-13 18:06:06.332581 7f5c973a57a0 10 register_cxx_method rbd.set_parent flags 3 0x7f5c8661bd20 2014-08-13 18:06:06.332584 7f5c973a57a0 10 register_cxx_method rbd.remove_parent flags 3 0x7f5c8661bb10 2014-08-13 18:06:06.332588 7f5c973a57a0 10 register_cxx_method rbd.set_protection_status flags 3 0x7f5c8661b530 2014-08-13 18:06:06.332591 7f5c973a57a0 10 register_cxx_method rbd.get_protection_status flags 1 0x7f5c8661b2c0 2014-08-13 18:06:06.332594 7f5c973a57a0 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f5c8661c550 2014-08-13 18:06:06.332597 7f5c973a57a0 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f5c866163d0 2014-08-13 18:06:06.332600 7f5c973a57a0 10 register_cxx_method rbd.add_child flags 3 0x7f5c86618f60 2014-08-13 18:06:06.332603 7f5c973a57a0 10 register_cxx_method rbd.remove_child flags 3 0x7f5c8661a2e0 2014-08-13 18:06:06.332607 7f5c973a57a0 10 register_cxx_method rbd.get_children flags 1 0x7f5c866194c0 2014-08-13 18:06:06.332610 7f5c973a57a0 10 register_cxx_method rbd.get_id flags 1 0x7f5c86614960 2014-08-13 18:06:06.332634 7f5c973a57a0 10 register_cxx_method rbd.set_id flags 3 0x7f5c86618cd0 2014-08-13 18:06:06.332637 7f5c973a57a0 10 register_cxx_method rbd.dir_get_id flags 1 0x7f5c86615fd0 2014-08-13 18:06:06.332640 7f5c973a57a0 10 register_cxx_method rbd.dir_get_name flags 1 0x7f5c866167d0 2014-08-13 18:06:06.332643 7f5c973a57a0 10 register_cxx_method rbd.dir_list flags 1 0x7f5c86616d70 2014-08-13 18:06:06.332647 7f5c973a57a0 10 register_cxx_method rbd.dir_add_image flags 3 0x7f5c86618670 2014-08-13 18:06:06.332650 7f5c973a57a0 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f5c86615db0 2014-08-13 18:06:06.332660 7f5c973a57a0 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f5c86618390 2014-08-13 18:06:06.332663 7f5c973a57a0 10 register_cxx_method rbd.snap_list flags 1 0x7f5c8661b860 2014-08-13 18:06:06.332667 7f5c973a57a0 10 register_cxx_method rbd.snap_add flags 3 0x7f5c86619830 2014-08-13 18:06:06.332670 7f5c973a57a0 10 register_cxx_method rbd.snap_remove flags 3 0x7f5c86619d50 2014-08-13 18:06:06.332673 7f5c973a57a0 10 _load_class rbd success 2014-08-13 18:06:06.332677 7f5c973a57a0 10 open_all_classes found refcount 2014-08-13 18:06:06.332681 7f5c973a57a0 10 _get_class adding new class name refcount 0x20c82c8 2014-08-13 18:06:06.332685 7f5c973a57a0 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so 2014-08-13 18:06:06.337576 7f5c973a57a0 10 register_class refcount status 3 2014-08-13 18:06:06.337579 7f5c973a57a0 10 register_cxx_method refcount.get flags 3 0x7f5c860e4050 2014-08-13 18:06:06.337584 7f5c973a57a0 10 register_cxx_method refcount.put flags 3 0x7f5c860e4610 2014-08-13 18:06:06.337587 7f5c973a57a0 10 register_cxx_method refcount.set flags 3 0x7f5c860e2c60 2014-08-13 18:06:06.337590 7f5c973a57a0 10 register_cxx_method refcount.read flags 1 0x7f5c860e3aa0 2014-08-13 18:06:06.337594 7f5c973a57a0 10 _load_class refcount success 2014-08-13 18:06:06.337596 7f5c973a57a0 10 open_all_classes found replica_log 2014-08-13 18:06:06.337600 7f5c973a57a0 10 _get_class adding new class name replica_log 0x20c81e8 2014-08-13 18:06:06.337604 7f5c973a57a0 10 _load_class replica_log from /usr/lib64/rados-classes/libcls_replica_log.so 2014-08-13 18:06:06.337816 7f5c973a57a0 10 register_class replica_log status 3 2014-08-13 18:06:06.337819 7f5c973a57a0 10 register_cxx_method replica_log.set flags 3 0x7f5c85e39b30 2014-08-13 18:06:06.337824 7f5c973a57a0 10 register_cxx_method replica_log.get flags 1 0x7f5c85e38ac0 2014-08-13 18:06:06.337827 7f5c973a57a0 10 register_cxx_method replica_log.delete flags 3 0x7f5c85e395b0 2014-08-13 18:06:06.337830 7f5c973a57a0 10 _load_class replica_log success 2014-08-13 18:06:06.337833 7f5c973a57a0 10 open_all_classes found statelog 2014-08-13 18:06:06.337836 7f5c973a57a0 10 _get_class adding new class name statelog 0x20c8108 2014-08-13 18:06:06.337840 7f5c973a57a0 10 _load_class statelog from /usr/lib64/rados-classes/libcls_statelog.so 2014-08-13 18:06:06.338038 7f5c973a57a0 10 register_class statelog status 3 2014-08-13 18:06:06.338041 7f5c973a57a0 10 register_cxx_method statelog.add flags 3 0x7f5c85c32360 2014-08-13 18:06:06.338046 7f5c973a57a0 10 register_cxx_method statelog.list flags 1 0x7f5c85c310e0 2014-08-13 18:06:06.338049 7f5c973a57a0 10 register_cxx_method statelog.remove flags 3 0x7f5c85c33330 2014-08-13 18:06:06.338052 7f5c973a57a0 10 register_cxx_method statelog.check_state flags 1 0x7f5c85c32aa0 2014-08-13 18:06:06.338056 7f5c973a57a0 10 _load_class statelog success 2014-08-13 18:06:06.338058 7f5c973a57a0 10 open_all_classes found rgw 2014-08-13 18:06:06.338063 7f5c973a57a0 10 _get_class adding new class name rgw 0x20c8648 2014-08-13 18:06:06.338066 7f5c973a57a0 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so 2014-08-13 18:06:06.342252 7f5c973a57a0 10 register_class rgw status 3 2014-08-13 18:06:06.342255 7f5c973a57a0 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f5c858ebcc0 2014-08-13 18:06:06.342260 7f5c973a57a0 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f5c858eb740 2014-08-13 18:06:06.342264 7f5c973a57a0 10 register_cxx_method rgw.bucket_list flags 1 0x7f5c858f2480 2014-08-13 18:06:06.342267 7f5c973a57a0 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f5c858f2030 2014-08-13 18:06:06.342270 7f5c973a57a0 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f5c858f22d0 2014-08-13 18:06:06.342273 7f5c973a57a0 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f5c858f40f0 2014-08-13 18:06:06.342277 7f5c973a57a0 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f5c858f4af0 2014-08-13 18:06:06.342322 7f5c973a57a0 10 register_cxx_method rgw.bi_log_list flags 1 0x7f5c858f0280 2014-08-13 18:06:06.342331 7f5c973a57a0 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f5c858efdf0 2014-08-13 18:06:06.342335 7f5c973a57a0 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f5c858f35e0 2014-08-13 18:06:06.342338 7f5c973a57a0 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f5c858ef000 2014-08-13 18:06:06.342341 7f5c973a57a0 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f5c858ecf30 2014-08-13 18:06:06.342345 7f5c973a57a0 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f5c858ecbd0 2014-08-13 18:06:06.342348 7f5c973a57a0 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f5c858ee960 2014-08-13 18:06:06.342351 7f5c973a57a0 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f5c858ee660 2014-08-13 18:06:06.342354 7f5c973a57a0 10 register_cxx_method rgw.gc_list flags 1 0x7f5c858ed7d0 2014-08-13 18:06:06.342358 7f5c973a57a0 10 register_cxx_method rgw.gc_remove flags 3 0x7f5c858eec00 2014-08-13 18:06:06.342361 7f5c973a57a0 10 _load_class rgw success 2014-08-13 18:06:06.342364 7f5c973a57a0 10 open_all_classes found version 2014-08-13 18:06:06.342374 7f5c973a57a0 10 _get_class adding new class name version 0x20c8568 2014-08-13 18:06:06.342378 7f5c973a57a0 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so 2014-08-13 18:06:06.342691 7f5c973a57a0 10 register_class version status 3 2014-08-13 18:06:06.342694 7f5c973a57a0 10 register_cxx_method version.set flags 3 0x7f5c8563c3c0 2014-08-13 18:06:06.342699 7f5c973a57a0 10 register_cxx_method version.inc flags 3 0x7f5c8563d6b0 2014-08-13 18:06:06.342702 7f5c973a57a0 10 register_cxx_method version.inc_conds flags 3 0x7f5c8563d6b0 2014-08-13 18:06:06.342706 7f5c973a57a0 10 register_cxx_method version.read flags 1 0x7f5c8563cd20 2014-08-13 18:06:06.342769 7f5c973a57a0 10 register_cxx_method version.check_conds flags 1 0x7f5c8563d0a0 2014-08-13 18:06:06.342773 7f5c973a57a0 10 _load_class version success 2014-08-13 18:06:06.342775 7f5c973a57a0 10 open_all_classes found lock 2014-08-13 18:06:06.342794 7f5c973a57a0 10 _get_class adding new class name lock 0x20c8488 2014-08-13 18:06:06.342821 7f5c973a57a0 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so 2014-08-13 18:06:06.343119 7f5c973a57a0 10 register_class lock status 3 2014-08-13 18:06:06.343130 7f5c973a57a0 10 register_cxx_method lock.lock flags 3 0x7f5c854345f0 2014-08-13 18:06:06.343138 7f5c973a57a0 10 register_cxx_method lock.unlock flags 3 0x7f5c85435510 2014-08-13 18:06:06.343142 7f5c973a57a0 10 register_cxx_method lock.break_lock flags 3 0x7f5c85435950 2014-08-13 18:06:06.343145 7f5c973a57a0 10 register_cxx_method lock.get_info flags 1 0x7f5c85433ad0 2014-08-13 18:06:06.343158 7f5c973a57a0 10 register_cxx_method lock.list_locks flags 1 0x7f5c85432630 2014-08-13 18:06:06.343162 7f5c973a57a0 10 _load_class lock success 2014-08-13 18:06:06.343164 7f5c973a57a0 10 open_all_classes found kvs 2014-08-13 18:06:06.343168 7f5c973a57a0 10 _get_class adding new class name kvs 0x20c83a8 2014-08-13 18:06:06.343172 7f5c973a57a0 10 _load_class kvs from /usr/lib64/rados-classes/libcls_kvs.so 2014-08-13 18:06:06.343505 7f5c973a57a0 10 register_class kvs status 3 2014-08-13 18:06:06.343517 7f5c973a57a0 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7f5c85225b10 2014-08-13 18:06:06.343526 7f5c973a57a0 10 register_cxx_method kvs.get_next_idata flags 1 0x7f5c85224b70 2014-08-13 18:06:06.343529 7f5c973a57a0 10 register_cxx_method kvs.get_prev_idata flags 1 0x7f5c85226560 2014-08-13 18:06:06.343532 7f5c973a57a0 10 register_cxx_method kvs.read_many flags 1 0x7f5c85225100 2014-08-13 18:06:06.343535 7f5c973a57a0 10 register_cxx_method kvs.check_writable flags 3 0x7f5c85222ab0 2014-08-13 18:06:06.343538 7f5c973a57a0 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7f5c85222ec0 2014-08-13 18:06:06.343541 7f5c973a57a0 10 register_cxx_method kvs.omap_insert flags 2 0x7f5c85224390 2014-08-13 18:06:06.343544 7f5c973a57a0 10 register_cxx_method kvs.create_with_omap flags 2 0x7f5c852230e0 2014-08-13 18:06:06.343547 7f5c973a57a0 10 register_cxx_method kvs.omap_remove flags 2 0x7f5c85223c80 2014-08-13 18:06:06.343550 7f5c973a57a0 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7f5c852234a0 2014-08-13 18:06:06.343553 7f5c973a57a0 10 _load_class kvs success 2014-08-13 18:06:06.343556 7f5c973a57a0 10 open_all_classes found hello 2014-08-13 18:06:06.343560 7f5c973a57a0 10 _get_class adding new class name hello 0x20c8aa8 2014-08-13 18:06:06.343564 7f5c973a57a0 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so 2014-08-13 18:06:06.343781 7f5c973a57a0 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-13 18:06:06.343790 7f5c973a57a0 10 register_class hello status 3 2014-08-13 18:06:06.343793 7f5c973a57a0 10 register_cxx_method hello.say_hello flags 1 0x7f5c8501b1f0 2014-08-13 18:06:06.343798 7f5c973a57a0 10 register_cxx_method hello.record_hello flags 2 0x7f5c8501b3e0 2014-08-13 18:06:06.343801 7f5c973a57a0 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f5c8501b960 2014-08-13 18:06:06.343812 7f5c973a57a0 10 register_cxx_method hello.replay flags 1 0x7f5c8501b1c0 2014-08-13 18:06:06.343815 7f5c973a57a0 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7f5c8501bc40 2014-08-13 18:06:06.343819 7f5c973a57a0 10 register_cxx_method hello.bad_reader flags 2 0x7f5c8501b1b0 2014-08-13 18:06:06.343821 7f5c973a57a0 10 register_cxx_method hello.bad_writer flags 1 0x7f5c8501b1a0 2014-08-13 18:06:06.343825 7f5c973a57a0 10 _load_class hello success 2014-08-13 18:06:06.343827 7f5c973a57a0 10 open_all_classes found log 2014-08-13 18:06:06.343831 7f5c973a57a0 10 _get_class adding new class name log 0x20c89c8 2014-08-13 18:06:06.343834 7f5c973a57a0 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so 2014-08-13 18:06:06.344029 7f5c973a57a0 10 register_class log status 3 2014-08-13 18:06:06.344031 7f5c973a57a0 10 register_cxx_method log.add flags 3 0x7f5c84e14370 2014-08-13 18:06:06.344036 7f5c973a57a0 10 register_cxx_method log.list flags 1 0x7f5c84e151f0 2014-08-13 18:06:06.344039 7f5c973a57a0 10 register_cxx_method log.trim flags 3 0x7f5c84e13b40 2014-08-13 18:06:06.344042 7f5c973a57a0 10 register_cxx_method log.info flags 1 0x7f5c84e165d0 2014-08-13 18:06:06.344045 7f5c973a57a0 10 _load_class log success 2014-08-13 18:06:06.344047 7f5c973a57a0 10 open_all_classes found user 2014-08-13 18:06:06.344052 7f5c973a57a0 10 _get_class adding new class name user 0x20c88e8 2014-08-13 18:06:06.344058 7f5c973a57a0 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so 2014-08-13 18:06:06.344284 7f5c973a57a0 10 register_class user status 3 2014-08-13 18:06:06.344291 7f5c973a57a0 10 register_cxx_method user.set_buckets_info flags 3 0x7f5c84c0c220 2014-08-13 18:06:06.344298 7f5c973a57a0 10 register_cxx_method user.complete_stats_sync flags 3 0x7f5c84c0aff0 2014-08-13 18:06:06.344302 7f5c973a57a0 10 register_cxx_method user.remove_bucket flags 3 0x7f5c84c0d210 2014-08-13 18:06:06.344305 7f5c973a57a0 10 register_cxx_method user.list_buckets flags 1 0x7f5c84c0b3c0 2014-08-13 18:06:06.344349 7f5c973a57a0 10 register_cxx_method user.get_header flags 1 0x7f5c84c0ac50 2014-08-13 18:06:06.344354 7f5c973a57a0 10 _load_class user success 2014-08-13 18:06:06.344382 7f5c973a57a0 20 osd.4 0 get_map 194487 - loading and decoding 0x201ba00 2014-08-13 18:06:06.344733 7f5c973a57a0 10 osd.4 0 add_map_bl 194487 268447 bytes 2014-08-13 18:06:06.346315 7f5c973a57a0 0 osd.4 194487 crush map has features 2303210029056, adjusting msgr requires for clients 2014-08-13 18:06:06.346338 7f5c973a57a0 0 osd.4 194487 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 18:06:06.346348 7f5c973a57a0 0 osd.4 194487 crush map has features 2578087936000, adjusting msgr requires for osds 2014-08-13 18:06:06.346358 7f5c973a57a0 10 osd.4 194487 create_recoverystate_perf 2014-08-13 18:06:06.346378 7f5c973a57a0 0 osd.4 194487 load_pgs 2014-08-13 18:06:06.346930 7f5c973a57a0 10 osd.4 194487 load_pgs ignoring unrecognized meta 2014-08-13 18:06:06.347144 7f5c973a57a0 10 osd.4 194487 pgid 1.2b coll 1.2b_head 2014-08-13 18:06:06.347589 7f5c973a57a0 20 osd.4 0 get_map 194486 - loading and decoding 0x201f600 2014-08-13 18:06:06.347893 7f5c973a57a0 10 osd.4 0 add_map_bl 194486 268447 bytes 2014-08-13 18:06:06.350757 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 1.2b 2014-08-13 18:06:06.350783 7f5c973a57a0 10 osd.4 194487 _get_pool 1 2014-08-13 18:06:06.350905 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[1.2b(unlocked)] enter Initial 2014-08-13 18:06:06.350961 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[1.2b(unlocked)] enter NotTrimming 2014-08-13 18:06:06.351260 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.351610 7f5c973a57a0 20 read_log 192888'1 (0'0) modify 968c7c2b/benchmark_data_storage0111-ib.pocstor_11336_object26/head//1 by client.1312877.0:27 2014-08-11 16:25:04.098381 2014-08-13 18:06:06.351680 7f5c973a57a0 20 read_log 192888'2 (0'0) modify 458d82b/benchmark_data_storage0111-ib.pocstor_11336_object85/head//1 by client.1312877.0:86 2014-08-11 16:25:04.370979 2014-08-13 18:06:06.351710 7f5c973a57a0 20 read_log 192888'3 (0'0) modify 8d3a1cab/benchmark_data_storage0111-ib.pocstor_11336_object115/head//1 by client.1312877.0:116 2014-08-11 16:25:04.477993 2014-08-13 18:06:06.351748 7f5c973a57a0 20 read_log 192888'4 (0'0) modify 174a7a6b/benchmark_data_storage0111-ib.pocstor_11336_object120/head//1 by client.1312877.0:121 2014-08-11 16:25:04.495532 2014-08-13 18:06:06.351774 7f5c973a57a0 20 read_log 192888'5 (0'0) modify 709b852b/benchmark_data_storage0111-ib.pocstor_11336_object514/head//1 by client.1312877.0:515 2014-08-11 16:25:05.966566 2014-08-13 18:06:06.351796 7f5c973a57a0 20 read_log 192888'6 (0'0) modify f9c4ff6b/benchmark_data_storage0111-ib.pocstor_11336_object529/head//1 by client.1312877.0:530 2014-08-11 16:25:06.020839 2014-08-13 18:06:06.351818 7f5c973a57a0 20 read_log 192888'7 (0'0) modify 5c599e6b/benchmark_data_storage0111-ib.pocstor_11336_object584/head//1 by client.1312877.0:585 2014-08-11 16:25:06.289680 2014-08-13 18:06:06.351840 7f5c973a57a0 20 read_log 192888'8 (0'0) modify cc5ad2ab/benchmark_data_storage0111-ib.pocstor_11336_object589/head//1 by client.1312877.0:590 2014-08-11 16:25:06.316558 2014-08-13 18:06:06.351865 7f5c973a57a0 20 read_log 192888'9 (0'0) modify e2cda9ab/benchmark_data_storage0111-ib.pocstor_11336_object680/head//1 by client.1312877.0:681 2014-08-11 16:25:06.690737 2014-08-13 18:06:06.351894 7f5c973a57a0 20 read_log 192888'10 (0'0) modify 6a2d76b/benchmark_data_storage0111-ib.pocstor_11336_object727/head//1 by client.1312877.0:728 2014-08-11 16:25:06.857642 2014-08-13 18:06:06.351917 7f5c973a57a0 20 read_log 192888'11 (0'0) modify d4542b6b/benchmark_data_storage0111-ib.pocstor_11336_object785/head//1 by client.1312877.0:786 2014-08-11 16:25:07.094331 2014-08-13 18:06:06.351939 7f5c973a57a0 20 read_log 192888'12 (0'0) modify 13924b6b/benchmark_data_storage0111-ib.pocstor_11336_object893/head//1 by client.1312877.0:894 2014-08-11 16:25:07.534706 2014-08-13 18:06:06.351961 7f5c973a57a0 20 read_log 192888'13 (0'0) modify 2df8bfab/benchmark_data_storage0111-ib.pocstor_11336_object920/head//1 by client.1312877.0:921 2014-08-11 16:25:07.631660 2014-08-13 18:06:06.351990 7f5c973a57a0 20 read_log 192888'14 (0'0) modify 1142b5eb/benchmark_data_storage0111-ib.pocstor_11336_object942/head//1 by client.1312877.0:943 2014-08-11 16:25:07.711037 2014-08-13 18:06:06.352012 7f5c973a57a0 20 read_log 192888'15 (0'0) modify afd4156b/benchmark_data_storage0111-ib.pocstor_11336_object1007/head//1 by client.1312877.0:1008 2014-08-11 16:25:07.946317 2014-08-13 18:06:06.352035 7f5c973a57a0 20 read_log 192888'16 (0'0) modify 1bd77ceb/benchmark_data_storage0111-ib.pocstor_11336_object1053/head//1 by client.1312877.0:1054 2014-08-11 16:25:08.111621 2014-08-13 18:06:06.352056 7f5c973a57a0 20 read_log 192888'17 (0'0) modify 4c3c3eab/benchmark_data_storage0111-ib.pocstor_11336_object1107/head//1 by client.1312877.0:1108 2014-08-11 16:25:08.304277 2014-08-13 18:06:06.352079 7f5c973a57a0 20 read_log 192888'18 (0'0) modify 84e0c2b/benchmark_data_storage0111-ib.pocstor_11336_object1197/head//1 by client.1312877.0:1198 2014-08-11 16:25:08.623939 2014-08-13 18:06:06.352104 7f5c973a57a0 20 read_log 192888'19 (0'0) modify 5acc686b/benchmark_data_storage0111-ib.pocstor_11336_object1402/head//1 by client.1312877.0:1403 2014-08-11 16:25:09.360339 2014-08-13 18:06:06.352132 7f5c973a57a0 20 read_log 192888'20 (0'0) modify cd6caab/benchmark_data_storage0111-ib.pocstor_11336_object1404/head//1 by client.1312877.0:1405 2014-08-11 16:25:09.367539 2014-08-13 18:06:06.352165 7f5c973a57a0 20 read_log 192888'21 (0'0) modify f337eceb/benchmark_data_storage0111-ib.pocstor_11336_object1448/head//1 by client.1312877.0:1449 2014-08-11 16:25:09.524131 2014-08-13 18:06:06.352191 7f5c973a57a0 20 read_log 192888'22 (0'0) modify fdf8f3eb/benchmark_data_storage0111-ib.pocstor_11336_object1668/head//1 by client.1312877.0:1669 2014-08-11 16:25:10.333172 2014-08-13 18:06:06.352213 7f5c973a57a0 20 read_log 192888'23 (0'0) modify 784e352b/benchmark_data_storage0111-ib.pocstor_11336_object1682/head//1 by client.1312877.0:1683 2014-08-11 16:25:10.382879 2014-08-13 18:06:06.352247 7f5c973a57a0 20 read_log 192888'24 (0'0) modify dda332b/benchmark_data_storage0111-ib.pocstor_11336_object1930/head//1 by client.1312877.0:1931 2014-08-11 16:25:11.400791 2014-08-13 18:06:06.352272 7f5c973a57a0 20 read_log 192888'25 (0'0) modify 93f31b2b/benchmark_data_storage0111-ib.pocstor_11336_object2038/head//1 by client.1312877.0:2039 2014-08-11 16:25:11.783817 2014-08-13 18:06:06.352294 7f5c973a57a0 20 read_log 192888'26 (0'0) modify 357eddeb/benchmark_data_storage0111-ib.pocstor_11336_object2073/head//1 by client.1312877.0:2074 2014-08-11 16:25:11.908139 2014-08-13 18:06:06.352333 7f5c973a57a0 20 read_log 192888'27 (0'0) modify 27ad09eb/benchmark_data_storage0111-ib.pocstor_11336_object2157/head//1 by client.1312877.0:2158 2014-08-11 16:25:12.208789 2014-08-13 18:06:06.352368 7f5c973a57a0 20 read_log 192888'28 (0'0) modify 78fc9beb/benchmark_data_storage0111-ib.pocstor_11336_object2224/head//1 by client.1312877.0:2225 2014-08-11 16:25:12.447857 2014-08-13 18:06:06.352390 7f5c973a57a0 20 read_log 192888'29 (0'0) modify 4721d42b/benchmark_data_storage0111-ib.pocstor_11336_object2247/head//1 by client.1312877.0:2248 2014-08-11 16:25:12.530070 2014-08-13 18:06:06.352412 7f5c973a57a0 20 read_log 192888'30 (0'0) modify d935726b/benchmark_data_storage0111-ib.pocstor_11336_object2299/head//1 by client.1312877.0:2300 2014-08-11 16:25:12.715972 2014-08-13 18:06:06.352452 7f5c973a57a0 20 read_log 192888'31 (0'0) modify 28d2c6ab/benchmark_data_storage0111-ib.pocstor_11336_object2510/head//1 by client.1312877.0:2511 2014-08-11 16:25:13.464648 2014-08-13 18:06:06.352481 7f5c973a57a0 20 read_log 192888'32 (0'0) modify a9ff716b/benchmark_data_storage0111-ib.pocstor_11336_object2683/head//1 by client.1312877.0:2684 2014-08-11 16:25:14.225172 2014-08-13 18:06:06.352503 7f5c973a57a0 20 read_log 192888'33 (0'0) modify 6eac06ab/benchmark_data_storage0111-ib.pocstor_11336_object2861/head//1 by client.1312877.0:2862 2014-08-11 16:25:14.854269 2014-08-13 18:06:06.352525 7f5c973a57a0 20 read_log 192888'34 (0'0) modify e39450ab/benchmark_data_storage0111-ib.pocstor_11336_object2944/head//1 by client.1312877.0:2945 2014-08-11 16:25:15.149385 2014-08-13 18:06:06.352546 7f5c973a57a0 20 read_log 192888'35 (0'0) modify 372d6ab/benchmark_data_storage0111-ib.pocstor_11336_object2981/head//1 by client.1312877.0:2982 2014-08-11 16:25:15.278781 2014-08-13 18:06:06.352567 7f5c973a57a0 20 read_log 192888'36 (0'0) modify aaad6a2b/benchmark_data_storage0111-ib.pocstor_11336_object2989/head//1 by client.1312877.0:2990 2014-08-11 16:25:15.306677 2014-08-13 18:06:06.352588 7f5c973a57a0 20 read_log 192888'37 (0'0) modify dbd623ab/benchmark_data_storage0111-ib.pocstor_11336_object3227/head//1 by client.1312877.0:3228 2014-08-11 16:25:16.152623 2014-08-13 18:06:06.352609 7f5c973a57a0 20 read_log 192888'38 (0'0) modify f13350eb/benchmark_data_storage0111-ib.pocstor_11336_object3246/head//1 by client.1312877.0:3247 2014-08-11 16:25:16.219261 2014-08-13 18:06:06.352630 7f5c973a57a0 20 read_log 192888'39 (0'0) modify 752770eb/benchmark_data_storage0111-ib.pocstor_11336_object3306/head//1 by client.1312877.0:3307 2014-08-11 16:25:16.433463 2014-08-13 18:06:06.352651 7f5c973a57a0 20 read_log 192888'40 (0'0) modify dd84e9ab/benchmark_data_storage0111-ib.pocstor_11336_object3341/head//1 by client.1312877.0:3342 2014-08-11 16:25:16.556429 2014-08-13 18:06:06.352680 7f5c973a57a0 20 read_log 192888'41 (0'0) modify de0a9deb/benchmark_data_storage0111-ib.pocstor_11336_object3370/head//1 by client.1312877.0:3371 2014-08-11 16:25:16.659507 2014-08-13 18:06:06.352706 7f5c973a57a0 20 read_log 192888'42 (0'0) modify 7d7620ab/benchmark_data_storage0111-ib.pocstor_11336_object3373/head//1 by client.1312877.0:3374 2014-08-11 16:25:16.670278 2014-08-13 18:06:06.352728 7f5c973a57a0 20 read_log 192888'43 (0'0) modify 83a675ab/benchmark_data_storage0111-ib.pocstor_11336_object3566/head//1 by client.1312877.0:3567 2014-08-11 16:25:17.379436 2014-08-13 18:06:06.352776 7f5c973a57a0 20 read_log 192888'44 (0'0) modify 8ef2e3eb/benchmark_data_storage0111-ib.pocstor_11336_object3609/head//1 by client.1312877.0:3610 2014-08-11 16:25:17.542587 2014-08-13 18:06:06.352803 7f5c973a57a0 20 read_log 192888'45 (0'0) modify c40043eb/benchmark_data_storage0111-ib.pocstor_11336_object3630/head//1 by client.1312877.0:3631 2014-08-11 16:25:17.621035 2014-08-13 18:06:06.352825 7f5c973a57a0 20 read_log 192888'46 (0'0) modify 83bd3aeb/benchmark_data_storage0111-ib.pocstor_11336_object3634/head//1 by client.1312877.0:3635 2014-08-11 16:25:17.635998 2014-08-13 18:06:06.352846 7f5c973a57a0 20 read_log 192888'47 (0'0) modify a7339fab/benchmark_data_storage0111-ib.pocstor_11336_object3636/head//1 by client.1312877.0:3637 2014-08-11 16:25:17.643430 2014-08-13 18:06:06.352867 7f5c973a57a0 20 read_log 192888'48 (0'0) modify b51ef32b/benchmark_data_storage0111-ib.pocstor_11336_object3841/head//1 by client.1312877.0:3842 2014-08-11 16:25:18.437037 2014-08-13 18:06:06.352888 7f5c973a57a0 20 read_log 192888'49 (0'0) modify e3142eab/benchmark_data_storage0111-ib.pocstor_11336_object4049/head//1 by client.1312877.0:4050 2014-08-11 16:25:19.254584 2014-08-13 18:06:06.352910 7f5c973a57a0 20 read_log 192888'50 (0'0) modify 2e39aeb/benchmark_data_storage0111-ib.pocstor_11336_object4076/head//1 by client.1312877.0:4077 2014-08-11 16:25:19.360926 2014-08-13 18:06:06.352931 7f5c973a57a0 20 read_log 192888'51 (0'0) modify 52e94fab/benchmark_data_storage0111-ib.pocstor_11336_object4179/head//1 by client.1312877.0:4180 2014-08-11 16:25:19.979133 2014-08-13 18:06:06.352953 7f5c973a57a0 20 read_log 192888'52 (0'0) modify a4d05a6b/benchmark_data_storage0111-ib.pocstor_11336_object4217/head//1 by client.1312877.0:4218 2014-08-11 16:25:20.149909 2014-08-13 18:06:06.352975 7f5c973a57a0 20 read_log 192888'53 (0'0) modify 8e026b6b/benchmark_data_storage0111-ib.pocstor_11336_object4269/head//1 by client.1312877.0:4270 2014-08-11 16:25:20.388934 2014-08-13 18:06:06.353004 7f5c973a57a0 20 read_log 192888'54 (0'0) modify 45e77deb/benchmark_data_storage0111-ib.pocstor_11336_object4277/head//1 by client.1312877.0:4278 2014-08-11 16:25:20.425898 2014-08-13 18:06:06.353028 7f5c973a57a0 20 read_log 192888'55 (0'0) modify 2779eb/benchmark_data_storage0111-ib.pocstor_11336_object4350/head//1 by client.1312877.0:4351 2014-08-11 16:25:20.748320 2014-08-13 18:06:06.353050 7f5c973a57a0 20 read_log 192888'56 (0'0) modify 200acdeb/benchmark_data_storage0111-ib.pocstor_11336_object4425/head//1 by client.1312877.0:4426 2014-08-11 16:25:21.033727 2014-08-13 18:06:06.353075 7f5c973a57a0 20 read_log 192888'57 (0'0) modify 4967bfeb/benchmark_data_storage0111-ib.pocstor_11336_object4506/head//1 by client.1312877.0:4507 2014-08-11 16:25:21.338278 2014-08-13 18:06:06.353097 7f5c973a57a0 20 read_log 192888'58 (0'0) modify 13ab5f6b/benchmark_data_storage0111-ib.pocstor_11336_object4629/head//1 by client.1312877.0:4630 2014-08-11 16:25:21.813661 2014-08-13 18:06:06.353122 7f5c973a57a0 20 read_log 192888'59 (0'0) modify e9c088eb/benchmark_data_storage0111-ib.pocstor_11336_object4640/head//1 by client.1312877.0:4641 2014-08-11 16:25:21.854777 2014-08-13 18:06:06.353143 7f5c973a57a0 20 read_log 192888'60 (0'0) modify 84ba19eb/benchmark_data_storage0111-ib.pocstor_11336_object4722/head//1 by client.1312877.0:4723 2014-08-11 16:25:22.710537 2014-08-13 18:06:06.353179 7f5c973a57a0 20 read_log 192888'61 (0'0) modify 64f0a7ab/benchmark_data_storage0111-ib.pocstor_11336_object4767/head//1 by client.1312877.0:4768 2014-08-11 16:25:23.037677 2014-08-13 18:06:06.353201 7f5c973a57a0 20 read_log 192888'62 (0'0) modify 18830eeb/benchmark_data_storage0111-ib.pocstor_11336_object4790/head//1 by client.1312877.0:4791 2014-08-11 16:25:23.150607 2014-08-13 18:06:06.353222 7f5c973a57a0 20 read_log 192888'63 (0'0) modify 80cba86b/benchmark_data_storage0111-ib.pocstor_11336_object4818/head//1 by client.1312877.0:4819 2014-08-11 16:25:23.257310 2014-08-13 18:06:06.353259 7f5c973a57a0 20 read_log 192888'64 (0'0) modify 7b42aa6b/benchmark_data_storage0111-ib.pocstor_11336_object4861/head//1 by client.1312877.0:4862 2014-08-11 16:25:23.422893 2014-08-13 18:06:06.353285 7f5c973a57a0 20 read_log 192888'65 (0'0) modify cb176c6b/benchmark_data_storage0111-ib.pocstor_11336_object4943/head//1 by client.1312877.0:4944 2014-08-11 16:25:23.742523 2014-08-13 18:06:06.353307 7f5c973a57a0 20 read_log 192888'66 (0'0) modify 34a403eb/benchmark_data_storage0111-ib.pocstor_11336_object5103/head//1 by client.1312877.0:5104 2014-08-11 16:25:24.664707 2014-08-13 18:06:06.353328 7f5c973a57a0 20 read_log 192888'67 (0'0) modify 43cf3beb/benchmark_data_storage0111-ib.pocstor_11336_object5115/head//1 by client.1312877.0:5116 2014-08-11 16:25:24.711221 2014-08-13 18:06:06.353349 7f5c973a57a0 20 read_log 192888'68 (0'0) modify ef97dfab/benchmark_data_storage0111-ib.pocstor_11336_object5267/head//1 by client.1312877.0:5268 2014-08-11 16:25:25.292076 2014-08-13 18:06:06.353382 7f5c973a57a0 20 read_log 192888'69 (0'0) modify 2a7643eb/benchmark_data_storage0111-ib.pocstor_11336_object5291/head//1 by client.1312877.0:5292 2014-08-11 16:25:25.383847 2014-08-13 18:06:06.353404 7f5c973a57a0 20 read_log 192888'70 (0'0) modify c91d172b/benchmark_data_storage0111-ib.pocstor_11336_object5349/head//1 by client.1312877.0:5350 2014-08-11 16:25:25.605487 2014-08-13 18:06:06.353431 7f5c973a57a0 20 read_log 192888'71 (0'0) modify 86f5e06b/benchmark_data_storage0111-ib.pocstor_11336_object5415/head//1 by client.1312877.0:5416 2014-08-11 16:25:25.855272 2014-08-13 18:06:06.353457 7f5c973a57a0 20 read_log 192888'72 (0'0) modify 199e7beb/benchmark_data_storage0111-ib.pocstor_11336_object5466/head//1 by client.1312877.0:5467 2014-08-11 16:25:26.048325 2014-08-13 18:06:06.353485 7f5c973a57a0 20 read_log 192888'73 (0'0) modify 7c15dbab/benchmark_data_storage0111-ib.pocstor_11336_object5510/head//1 by client.1312877.0:5511 2014-08-11 16:25:26.215683 2014-08-13 18:06:06.353506 7f5c973a57a0 20 read_log 192888'74 (0'0) modify 5bad7f6b/benchmark_data_storage0111-ib.pocstor_11336_object5611/head//1 by client.1312877.0:5612 2014-08-11 16:25:26.598058 2014-08-13 18:06:06.353527 7f5c973a57a0 20 read_log 192888'75 (0'0) modify b65b286b/benchmark_data_storage0111-ib.pocstor_11336_object5618/head//1 by client.1312877.0:5619 2014-08-11 16:25:26.624552 2014-08-13 18:06:06.353548 7f5c973a57a0 20 read_log 192888'76 (0'0) modify 51e9e32b/benchmark_data_storage0111-ib.pocstor_11336_object5649/head//1 by client.1312877.0:5650 2014-08-11 16:25:26.741722 2014-08-13 18:06:06.353586 7f5c973a57a0 20 read_log 192888'77 (0'0) modify be6f1b2b/benchmark_data_storage0111-ib.pocstor_11336_object5663/head//1 by client.1312877.0:5664 2014-08-11 16:25:26.796655 2014-08-13 18:06:06.353613 7f5c973a57a0 20 read_log 192888'78 (0'0) modify ef81c46b/benchmark_data_storage0111-ib.pocstor_11336_object5705/head//1 by client.1312877.0:5706 2014-08-11 16:25:26.955338 2014-08-13 18:06:06.353635 7f5c973a57a0 20 read_log 192888'79 (0'0) modify 7460ea2b/benchmark_data_storage0111-ib.pocstor_11336_object5758/head//1 by client.1312877.0:5759 2014-08-11 16:25:27.156703 2014-08-13 18:06:06.353689 7f5c973a57a0 20 read_log 192888'80 (0'0) modify d8711a2b/benchmark_data_storage0111-ib.pocstor_11336_object5802/head//1 by client.1312877.0:5803 2014-08-11 16:25:27.323639 2014-08-13 18:06:06.353711 7f5c973a57a0 20 read_log 192888'81 (0'0) modify 7cea612b/benchmark_data_storage0111-ib.pocstor_11336_object5943/head//1 by client.1312877.0:5944 2014-08-11 16:25:27.878556 2014-08-13 18:06:06.353732 7f5c973a57a0 20 read_log 192888'82 (0'0) modify b862b52b/benchmark_data_storage0111-ib.pocstor_11336_object5996/head//1 by client.1312877.0:5997 2014-08-11 16:25:28.083412 2014-08-13 18:06:06.353753 7f5c973a57a0 20 read_log 192888'83 (0'0) modify aa0fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object6043/head//1 by client.1312877.0:6044 2014-08-11 16:25:28.262041 2014-08-13 18:06:06.353803 7f5c973a57a0 20 read_log 192888'84 (0'0) modify 1f2e13eb/benchmark_data_storage0111-ib.pocstor_11336_object6114/head//1 by client.1312877.0:6115 2014-08-11 16:25:28.535889 2014-08-13 18:06:06.353848 7f5c973a57a0 20 read_log 192888'85 (0'0) modify 26185aeb/benchmark_data_storage0111-ib.pocstor_11336_object6115/head//1 by client.1312877.0:6116 2014-08-11 16:25:28.539681 2014-08-13 18:06:06.353876 7f5c973a57a0 20 read_log 192888'86 (0'0) modify bc9f81ab/benchmark_data_storage0111-ib.pocstor_11336_object6256/head//1 by client.1312877.0:6257 2014-08-11 16:25:29.081741 2014-08-13 18:06:06.353907 7f5c973a57a0 20 read_log 192888'87 (0'0) modify 665d8cab/benchmark_data_storage0111-ib.pocstor_11336_object6345/head//1 by client.1312877.0:6346 2014-08-11 16:25:29.421793 2014-08-13 18:06:06.353933 7f5c973a57a0 20 read_log 192888'88 (0'0) modify 2797a2b/benchmark_data_storage0111-ib.pocstor_11336_object6356/head//1 by client.1312877.0:6357 2014-08-11 16:25:29.463723 2014-08-13 18:06:06.353956 7f5c973a57a0 20 read_log 192888'89 (0'0) modify dd86336b/benchmark_data_storage0111-ib.pocstor_11336_object6360/head//1 by client.1312877.0:6361 2014-08-11 16:25:29.479031 2014-08-13 18:06:06.353990 7f5c973a57a0 20 read_log 192888'90 (0'0) modify 81e68deb/benchmark_data_storage0111-ib.pocstor_11336_object6546/head//1 by client.1312877.0:6547 2014-08-11 16:25:30.256045 2014-08-13 18:06:06.354033 7f5c973a57a0 20 read_log 192888'91 (0'0) modify cacbb16b/benchmark_data_storage0111-ib.pocstor_11336_object6567/head//1 by client.1312877.0:6568 2014-08-11 16:25:30.336123 2014-08-13 18:06:06.354061 7f5c973a57a0 20 read_log 192888'92 (0'0) modify 982d3c2b/benchmark_data_storage0111-ib.pocstor_11336_object6610/head//1 by client.1312877.0:6611 2014-08-11 16:25:30.502431 2014-08-13 18:06:06.354083 7f5c973a57a0 20 read_log 192888'93 (0'0) modify 7c69c2b/benchmark_data_storage0111-ib.pocstor_11336_object6688/head//1 by client.1312877.0:6689 2014-08-11 16:25:30.801607 2014-08-13 18:06:06.354105 7f5c973a57a0 20 read_log 192888'94 (0'0) modify 4db680ab/benchmark_data_storage0111-ib.pocstor_11336_object6735/head//1 by client.1312877.0:6736 2014-08-11 16:25:30.982438 2014-08-13 18:06:06.354128 7f5c973a57a0 20 read_log 192888'95 (0'0) modify ee8926eb/benchmark_data_storage0111-ib.pocstor_11336_object6785/head//1 by client.1312877.0:6786 2014-08-11 16:25:31.174880 2014-08-13 18:06:06.354163 7f5c973a57a0 20 read_log 192888'96 (0'0) modify 183f4fab/benchmark_data_storage0111-ib.pocstor_11336_object6867/head//1 by client.1312877.0:6868 2014-08-11 16:25:31.489574 2014-08-13 18:06:06.354190 7f5c973a57a0 20 read_log 192888'97 (0'0) modify 77e164ab/benchmark_data_storage0111-ib.pocstor_11336_object6990/head//1 by client.1312877.0:6991 2014-08-11 16:25:31.957041 2014-08-13 18:06:06.354212 7f5c973a57a0 20 read_log 192888'98 (0'0) modify e34b5a2b/benchmark_data_storage0111-ib.pocstor_11336_object6999/head//1 by client.1312877.0:7000 2014-08-11 16:25:31.990958 2014-08-13 18:06:06.354234 7f5c973a57a0 20 read_log 192888'99 (0'0) modify aa9237eb/benchmark_data_storage0111-ib.pocstor_11336_object7008/head//1 by client.1312877.0:7009 2014-08-11 16:25:32.026460 2014-08-13 18:06:06.354255 7f5c973a57a0 20 read_log 192888'100 (0'0) modify ac4a732b/benchmark_data_storage0111-ib.pocstor_11336_object7094/head//1 by client.1312877.0:7095 2014-08-11 16:25:32.355711 2014-08-13 18:06:06.354277 7f5c973a57a0 20 read_log 192888'101 (0'0) modify c413e7ab/benchmark_data_storage0111-ib.pocstor_11336_object7152/head//1 by client.1312877.0:7153 2014-08-11 16:25:32.576689 2014-08-13 18:06:06.354298 7f5c973a57a0 20 read_log 192888'102 (0'0) modify ef342f6b/benchmark_data_storage0111-ib.pocstor_11336_object7178/head//1 by client.1312877.0:7179 2014-08-11 16:25:32.674886 2014-08-13 18:06:06.354319 7f5c973a57a0 20 read_log 192888'103 (0'0) modify 581f046b/benchmark_data_storage0111-ib.pocstor_11336_object7210/head//1 by client.1312877.0:7211 2014-08-11 16:25:32.798047 2014-08-13 18:06:06.354365 7f5c973a57a0 20 read_log 192888'104 (0'0) modify 6e01b76b/benchmark_data_storage0111-ib.pocstor_11336_object7236/head//1 by client.1312877.0:7237 2014-08-11 16:25:32.895808 2014-08-13 18:06:06.354394 7f5c973a57a0 20 read_log 192888'105 (0'0) modify d200a6eb/benchmark_data_storage0111-ib.pocstor_11336_object7281/head//1 by client.1312877.0:7282 2014-08-11 16:25:33.067087 2014-08-13 18:06:06.354416 7f5c973a57a0 20 read_log 192888'106 (0'0) modify 84fb60eb/benchmark_data_storage0111-ib.pocstor_11336_object7370/head//1 by client.1312877.0:7371 2014-08-11 16:25:33.403090 2014-08-13 18:06:06.354438 7f5c973a57a0 20 read_log 192888'107 (0'0) modify d6b5f52b/benchmark_data_storage0111-ib.pocstor_11336_object7447/head//1 by client.1312877.0:7448 2014-08-11 16:25:33.697163 2014-08-13 18:06:06.354462 7f5c973a57a0 20 read_log 192888'108 (0'0) modify 8552af2b/benchmark_data_storage0111-ib.pocstor_11336_object7477/head//1 by client.1312877.0:7478 2014-08-11 16:25:33.810783 2014-08-13 18:06:06.354483 7f5c973a57a0 20 read_log 192888'109 (0'0) modify d583a86b/benchmark_data_storage0111-ib.pocstor_11336_object7517/head//1 by client.1312877.0:7518 2014-08-11 16:25:33.963205 2014-08-13 18:06:06.354514 7f5c973a57a0 20 read_log 192888'110 (0'0) modify 5fb5a3eb/benchmark_data_storage0111-ib.pocstor_11336_object7554/head//1 by client.1312877.0:7555 2014-08-11 16:25:34.105265 2014-08-13 18:06:06.354539 7f5c973a57a0 20 read_log 192888'111 (0'0) modify fb03b0eb/benchmark_data_storage0111-ib.pocstor_11336_object7647/head//1 by client.1312877.0:7648 2014-08-11 16:25:34.459397 2014-08-13 18:06:06.354585 7f5c973a57a0 20 read_log 192888'112 (0'0) modify 6b05496b/benchmark_data_storage0111-ib.pocstor_11336_object7737/head//1 by client.1312877.0:7738 2014-08-11 16:25:34.798043 2014-08-13 18:06:06.354625 7f5c973a57a0 20 read_log 192888'113 (0'0) modify 8326d86b/benchmark_data_storage0111-ib.pocstor_11336_object7812/head//1 by client.1312877.0:7813 2014-08-11 16:25:35.091267 2014-08-13 18:06:06.354647 7f5c973a57a0 20 read_log 192888'114 (0'0) modify e8e52e2b/benchmark_data_storage0111-ib.pocstor_11336_object7900/head//1 by client.1312877.0:7901 2014-08-11 16:25:35.566020 2014-08-13 18:06:06.354668 7f5c973a57a0 20 read_log 192888'115 (0'0) modify e48691eb/benchmark_data_storage0111-ib.pocstor_11336_object7922/head//1 by client.1312877.0:7923 2014-08-11 16:25:35.648126 2014-08-13 18:06:06.354690 7f5c973a57a0 20 read_log 192888'116 (0'0) modify 10dd5aeb/benchmark_data_storage0111-ib.pocstor_11336_object7942/head//1 by client.1312877.0:7943 2014-08-11 16:25:35.722210 2014-08-13 18:06:06.354723 7f5c973a57a0 20 read_log 192888'117 (0'0) modify 1dd889ab/benchmark_data_storage0111-ib.pocstor_11336_object8027/head//1 by client.1312877.0:8028 2014-08-11 16:25:36.105574 2014-08-13 18:06:06.354750 7f5c973a57a0 20 read_log 192888'118 (0'0) modify 8d332b2b/benchmark_data_storage0111-ib.pocstor_11336_object8120/head//1 by client.1312877.0:8121 2014-08-11 16:25:36.593921 2014-08-13 18:06:06.354782 7f5c973a57a0 20 read_log 192888'119 (0'0) modify 1793a06b/benchmark_data_storage0111-ib.pocstor_11336_object8121/head//1 by client.1312877.0:8122 2014-08-11 16:25:36.597541 2014-08-13 18:06:06.354804 7f5c973a57a0 20 read_log 192888'120 (0'0) modify b4faa32b/benchmark_data_storage0111-ib.pocstor_11336_object8144/head//1 by client.1312877.0:8145 2014-08-11 16:25:36.689453 2014-08-13 18:06:06.354826 7f5c973a57a0 20 read_log 192888'121 (0'0) modify 4035182b/benchmark_data_storage0111-ib.pocstor_11336_object8213/head//1 by client.1312877.0:8214 2014-08-11 16:25:37.082509 2014-08-13 18:06:06.354875 7f5c973a57a0 20 read_log 192888'122 (0'0) modify 461c62b/benchmark_data_storage0111-ib.pocstor_11336_object8226/head//1 by client.1312877.0:8227 2014-08-11 16:25:37.144136 2014-08-13 18:06:06.354899 7f5c973a57a0 20 read_log 192888'123 (0'0) modify fef695ab/benchmark_data_storage0111-ib.pocstor_11336_object8282/head//1 by client.1312877.0:8283 2014-08-11 16:25:37.439393 2014-08-13 18:06:06.354938 7f5c973a57a0 20 read_log 192888'124 (0'0) modify 133eef2b/benchmark_data_storage0111-ib.pocstor_11336_object8316/head//1 by client.1312877.0:8317 2014-08-11 16:25:37.582436 2014-08-13 18:06:06.354969 7f5c973a57a0 20 read_log 192888'125 (0'0) modify 5905afab/benchmark_data_storage0111-ib.pocstor_11336_object8508/head//1 by client.1312877.0:8509 2014-08-11 16:25:38.443785 2014-08-13 18:06:06.354991 7f5c973a57a0 20 read_log 192888'126 (0'0) modify 1c3d296b/benchmark_data_storage0111-ib.pocstor_11336_object8550/head//1 by client.1312877.0:8551 2014-08-11 16:25:38.887026 2014-08-13 18:06:06.355031 7f5c973a57a0 20 read_log 192888'127 (0'0) modify 582141eb/benchmark_data_storage0111-ib.pocstor_11336_object8559/head//1 by client.1312877.0:8560 2014-08-11 16:25:38.927100 2014-08-13 18:06:06.355059 7f5c973a57a0 20 read_log 192888'128 (0'0) modify 39758f6b/benchmark_data_storage0111-ib.pocstor_11336_object8581/head//1 by client.1312877.0:8582 2014-08-11 16:25:39.007912 2014-08-13 18:06:06.355101 7f5c973a57a0 20 read_log 192888'129 (0'0) modify 5016fc6b/benchmark_data_storage0111-ib.pocstor_11336_object8602/head//1 by client.1312877.0:8603 2014-08-11 16:25:39.100287 2014-08-13 18:06:06.355135 7f5c973a57a0 20 read_log 192888'130 (0'0) modify 11560a2b/benchmark_data_storage0111-ib.pocstor_11336_object8708/head//1 by client.1312877.0:8709 2014-08-11 16:25:39.514490 2014-08-13 18:06:06.355174 7f5c973a57a0 20 read_log 192888'131 (0'0) modify 22b07aab/benchmark_data_storage0111-ib.pocstor_11336_object8772/head//1 by client.1312877.0:8773 2014-08-11 16:25:39.820079 2014-08-13 18:06:06.355201 7f5c973a57a0 20 read_log 192888'132 (0'0) modify 7e5d59ab/benchmark_data_storage0111-ib.pocstor_11336_object8779/head//1 by client.1312877.0:8780 2014-08-11 16:25:39.863261 2014-08-13 18:06:06.355222 7f5c973a57a0 20 read_log 192888'133 (0'0) modify ec37116b/benchmark_data_storage0111-ib.pocstor_11336_object8784/head//1 by client.1312877.0:8785 2014-08-11 16:25:39.888051 2014-08-13 18:06:06.355243 7f5c973a57a0 20 read_log 192888'134 (0'0) modify b0de23eb/benchmark_data_storage0111-ib.pocstor_11336_object8926/head//1 by client.1312877.0:8927 2014-08-11 16:25:40.838236 2014-08-13 18:06:06.355265 7f5c973a57a0 20 read_log 192888'135 (0'0) modify bb89eab/benchmark_data_storage0111-ib.pocstor_11336_object8969/head//1 by client.1312877.0:8970 2014-08-11 16:25:41.099505 2014-08-13 18:06:06.355300 7f5c973a57a0 20 read_log 192888'136 (0'0) modify 1573a02b/benchmark_data_storage0111-ib.pocstor_11336_object9147/head//1 by client.1312877.0:9148 2014-08-11 16:25:41.911075 2014-08-13 18:06:06.355324 7f5c973a57a0 20 read_log 192888'137 (0'0) modify 9b3d4deb/benchmark_data_storage0111-ib.pocstor_11336_object9217/head//1 by client.1312877.0:9218 2014-08-11 16:25:42.164072 2014-08-13 18:06:06.355346 7f5c973a57a0 20 read_log 192888'138 (0'0) modify 7c824dab/benchmark_data_storage0111-ib.pocstor_11336_object9339/head//1 by client.1312877.0:9340 2014-08-11 16:25:42.595095 2014-08-13 18:06:06.355367 7f5c973a57a0 20 read_log 192888'139 (0'0) modify 7ce138eb/benchmark_data_storage0111-ib.pocstor_11336_object9430/head//1 by client.1312877.0:9431 2014-08-11 16:25:43.020432 2014-08-13 18:06:06.355416 7f5c973a57a0 20 read_log 192888'140 (0'0) modify 5dd698ab/benchmark_data_storage0111-ib.pocstor_11336_object9467/head//1 by client.1312877.0:9468 2014-08-11 16:25:43.149613 2014-08-13 18:06:06.355445 7f5c973a57a0 20 read_log 192888'141 (0'0) modify 6fa2c96b/benchmark_data_storage0111-ib.pocstor_11336_object9547/head//1 by client.1312877.0:9548 2014-08-11 16:25:43.467545 2014-08-13 18:06:06.355467 7f5c973a57a0 20 read_log 192888'142 (0'0) modify af93b72b/benchmark_data_storage0111-ib.pocstor_11336_object9599/head//1 by client.1312877.0:9600 2014-08-11 16:25:43.665776 2014-08-13 18:06:06.355488 7f5c973a57a0 20 read_log 192888'143 (0'0) modify 9e00ec6b/benchmark_data_storage0111-ib.pocstor_11336_object9624/head//1 by client.1312877.0:9625 2014-08-11 16:25:43.884548 2014-08-13 18:06:06.355523 7f5c973a57a0 20 read_log 192888'144 (0'0) modify 30d7d7ab/benchmark_data_storage0111-ib.pocstor_11336_object9654/head//1 by client.1312877.0:9655 2014-08-11 16:25:44.037661 2014-08-13 18:06:06.355548 7f5c973a57a0 20 read_log 192888'145 (0'0) modify c30ef8eb/benchmark_data_storage0111-ib.pocstor_11336_object9675/head//1 by client.1312877.0:9676 2014-08-11 16:25:44.146318 2014-08-13 18:06:06.355570 7f5c973a57a0 20 read_log 192888'146 (0'0) modify e5c5686b/benchmark_data_storage0111-ib.pocstor_11336_object9798/head//1 by client.1312877.0:9799 2014-08-11 16:25:44.702352 2014-08-13 18:06:06.355620 7f5c973a57a0 20 read_log 192888'147 (0'0) modify f0c1866b/benchmark_data_storage0111-ib.pocstor_11336_object9809/head//1 by client.1312877.0:9810 2014-08-11 16:25:44.740451 2014-08-13 18:06:06.355647 7f5c973a57a0 20 read_log 192888'148 (0'0) modify 7dfac8eb/benchmark_data_storage0111-ib.pocstor_11336_object9862/head//1 by client.1312877.0:9863 2014-08-11 16:25:44.925703 2014-08-13 18:06:06.355669 7f5c973a57a0 20 read_log 192888'149 (0'0) modify 7385cbab/benchmark_data_storage0111-ib.pocstor_11336_object10044/head//1 by client.1312877.0:10045 2014-08-11 16:25:45.589416 2014-08-13 18:06:06.355692 7f5c973a57a0 20 read_log 192888'150 (0'0) modify 8cef43ab/benchmark_data_storage0111-ib.pocstor_11336_object10070/head//1 by client.1312877.0:10071 2014-08-11 16:25:45.682268 2014-08-13 18:06:06.355713 7f5c973a57a0 20 read_log 192888'151 (0'0) modify a97eac2b/benchmark_data_storage0111-ib.pocstor_11336_object10215/head//1 by client.1312877.0:10216 2014-08-11 16:25:46.234787 2014-08-13 18:06:06.355734 7f5c973a57a0 20 read_log 192888'152 (0'0) modify 9dccadab/benchmark_data_storage0111-ib.pocstor_11336_object10293/head//1 by client.1312877.0:10294 2014-08-11 16:25:46.668768 2014-08-13 18:06:06.355755 7f5c973a57a0 20 read_log 192888'153 (0'0) modify 403f876b/benchmark_data_storage0111-ib.pocstor_11336_object10422/head//1 by client.1312877.0:10423 2014-08-11 16:25:47.419241 2014-08-13 18:06:06.355776 7f5c973a57a0 20 read_log 192888'154 (0'0) modify d913feab/benchmark_data_storage0111-ib.pocstor_11336_object10499/head//1 by client.1312877.0:10500 2014-08-11 16:25:47.706624 2014-08-13 18:06:06.355797 7f5c973a57a0 20 read_log 192888'155 (0'0) modify 1067f22b/benchmark_data_storage0111-ib.pocstor_11336_object10519/head//1 by client.1312877.0:10520 2014-08-11 16:25:47.779852 2014-08-13 18:06:06.355818 7f5c973a57a0 20 read_log 192888'156 (0'0) modify 4ca0adab/benchmark_data_storage0111-ib.pocstor_11336_object10563/head//1 by client.1312877.0:10564 2014-08-11 16:25:47.936464 2014-08-13 18:06:06.355839 7f5c973a57a0 20 read_log 192888'157 (0'0) modify cd10626b/benchmark_data_storage0111-ib.pocstor_11336_object10629/head//1 by client.1312877.0:10630 2014-08-11 16:25:48.170968 2014-08-13 18:06:06.355860 7f5c973a57a0 20 read_log 192888'158 (0'0) modify 2d074d6b/benchmark_data_storage0111-ib.pocstor_11336_object10684/head//1 by client.1312877.0:10685 2014-08-11 16:25:48.362526 2014-08-13 18:06:06.355881 7f5c973a57a0 20 read_log 192888'159 (0'0) modify 7094446b/benchmark_data_storage0111-ib.pocstor_11336_object10863/head//1 by client.1312877.0:10864 2014-08-11 16:25:49.013359 2014-08-13 18:06:06.355940 7f5c973a57a0 20 read_log 192888'160 (0'0) modify 24fc1eab/benchmark_data_storage0111-ib.pocstor_11336_object10891/head//1 by client.1312877.0:10892 2014-08-11 16:25:49.135547 2014-08-13 18:06:06.355962 7f5c973a57a0 20 read_log 192888'161 (0'0) modify ba27f86b/benchmark_data_storage0111-ib.pocstor_11336_object10963/head//1 by client.1312877.0:10964 2014-08-11 16:25:49.451940 2014-08-13 18:06:06.355987 7f5c973a57a0 20 read_log 192888'162 (0'0) modify 979816b/benchmark_data_storage0111-ib.pocstor_11336_object10980/head//1 by client.1312877.0:10981 2014-08-11 16:25:49.537095 2014-08-13 18:06:06.356009 7f5c973a57a0 20 read_log 192888'163 (0'0) modify 7f2fe8eb/benchmark_data_storage0111-ib.pocstor_11336_object10990/head//1 by client.1312877.0:10991 2014-08-11 16:25:49.595577 2014-08-13 18:06:06.356057 7f5c973a57a0 20 read_log 192888'164 (0'0) modify 53e63eb/benchmark_data_storage0111-ib.pocstor_11336_object11017/head//1 by client.1312877.0:11018 2014-08-11 16:25:49.733507 2014-08-13 18:06:06.356084 7f5c973a57a0 20 read_log 192888'165 (0'0) modify b5c20a6b/benchmark_data_storage0111-ib.pocstor_11336_object11120/head//1 by client.1312877.0:11121 2014-08-11 16:25:50.148548 2014-08-13 18:06:06.356106 7f5c973a57a0 20 read_log 192888'166 (0'0) modify 2c9accab/benchmark_data_storage0111-ib.pocstor_11336_object11135/head//1 by client.1312877.0:11136 2014-08-11 16:25:50.227359 2014-08-13 18:06:06.356131 7f5c973a57a0 20 read_log 192888'167 (0'0) modify 8cd4cbab/benchmark_data_storage0111-ib.pocstor_11336_object11148/head//1 by client.1312877.0:11149 2014-08-11 16:25:50.297770 2014-08-13 18:06:06.356166 7f5c973a57a0 20 read_log 192888'168 (0'0) modify 5a894beb/benchmark_data_storage0111-ib.pocstor_11336_object11176/head//1 by client.1312877.0:11177 2014-08-11 16:25:50.461762 2014-08-13 18:06:06.356196 7f5c973a57a0 20 read_log 192888'169 (0'0) modify a326b22b/benchmark_data_storage0111-ib.pocstor_11336_object11234/head//1 by client.1312877.0:11235 2014-08-11 16:25:50.769210 2014-08-13 18:06:06.356219 7f5c973a57a0 20 read_log 192888'170 (0'0) modify f27ed76b/benchmark_data_storage0111-ib.pocstor_11336_object11321/head//1 by client.1312877.0:11322 2014-08-11 16:25:51.348582 2014-08-13 18:06:06.356240 7f5c973a57a0 20 read_log 192888'171 (0'0) modify 36effaeb/benchmark_data_storage0111-ib.pocstor_11336_object11355/head//1 by client.1312877.0:11356 2014-08-11 16:25:51.761371 2014-08-13 18:06:06.356265 7f5c973a57a0 20 read_log 192888'172 (0'0) modify c959e92b/benchmark_data_storage0111-ib.pocstor_11336_object11363/head//1 by client.1312877.0:11364 2014-08-11 16:25:51.874245 2014-08-13 18:06:06.356286 7f5c973a57a0 20 read_log 192888'173 (0'0) modify 8a569feb/benchmark_data_storage0111-ib.pocstor_11336_object11379/head//1 by client.1312877.0:11380 2014-08-11 16:25:52.056754 2014-08-13 18:06:06.356312 7f5c973a57a0 20 read_log 192888'174 (0'0) modify 265609ab/benchmark_data_storage0111-ib.pocstor_11336_object11434/head//1 by client.1312877.0:11435 2014-08-11 16:25:52.547532 2014-08-13 18:06:06.356333 7f5c973a57a0 20 read_log 192888'175 (0'0) modify 2f3985ab/benchmark_data_storage0111-ib.pocstor_11336_object11591/head//1 by client.1312877.0:11592 2014-08-11 16:25:53.158672 2014-08-13 18:06:06.356354 7f5c973a57a0 20 read_log 192888'176 (0'0) modify 2218ab6b/benchmark_data_storage0111-ib.pocstor_11336_object11594/head//1 by client.1312877.0:11595 2014-08-11 16:25:53.170196 2014-08-13 18:06:06.356381 7f5c973a57a0 20 read_log 192888'177 (0'0) modify 780536ab/benchmark_data_storage0111-ib.pocstor_11336_object11678/head//1 by client.1312877.0:11679 2014-08-11 16:25:53.520527 2014-08-13 18:06:06.356404 7f5c973a57a0 20 read_log 192888'178 (0'0) modify a4d98e2b/benchmark_data_storage0111-ib.pocstor_11336_object11711/head//1 by client.1312877.0:11712 2014-08-11 16:25:53.675553 2014-08-13 18:06:06.356431 7f5c973a57a0 20 read_log 192888'179 (0'0) modify 3ae3c7eb/benchmark_data_storage0111-ib.pocstor_11336_object11840/head//1 by client.1312877.0:11841 2014-08-11 16:25:54.146662 2014-08-13 18:06:06.356455 7f5c973a57a0 20 read_log 192888'180 (0'0) modify 28cb2aeb/benchmark_data_storage0111-ib.pocstor_11336_object11864/head//1 by client.1312877.0:11865 2014-08-11 16:25:54.242230 2014-08-13 18:06:06.356477 7f5c973a57a0 20 read_log 192888'181 (0'0) modify a7751f6b/benchmark_data_storage0111-ib.pocstor_11336_object11902/head//1 by client.1312877.0:11903 2014-08-11 16:25:54.454656 2014-08-13 18:06:06.356506 7f5c973a57a0 20 read_log 192888'182 (0'0) modify 4b1d62ab/benchmark_data_storage0111-ib.pocstor_11336_object11920/head//1 by client.1312877.0:11921 2014-08-11 16:25:54.552529 2014-08-13 18:06:06.356528 7f5c973a57a0 20 read_log 192888'183 (0'0) modify 9ef299ab/benchmark_data_storage0111-ib.pocstor_11336_object11921/head//1 by client.1312877.0:11922 2014-08-11 16:25:54.558418 2014-08-13 18:06:06.356583 7f5c973a57a0 20 read_log 192888'184 (0'0) modify 47510d6b/benchmark_data_storage0111-ib.pocstor_11336_object12100/head//1 by client.1312877.0:12101 2014-08-11 16:25:55.404498 2014-08-13 18:06:06.356623 7f5c973a57a0 20 read_log 192888'185 (0'0) modify 69a89f2b/benchmark_data_storage0111-ib.pocstor_11336_object12168/head//1 by client.1312877.0:12169 2014-08-11 16:25:55.783872 2014-08-13 18:06:06.356652 7f5c973a57a0 20 read_log 192888'186 (0'0) modify 6b100eeb/benchmark_data_storage0111-ib.pocstor_11336_object12257/head//1 by client.1312877.0:12258 2014-08-11 16:25:56.099515 2014-08-13 18:06:06.356685 7f5c973a57a0 20 read_log 192888'187 (0'0) modify cdb5142b/benchmark_data_storage0111-ib.pocstor_11336_object12280/head//1 by client.1312877.0:12281 2014-08-11 16:25:56.181331 2014-08-13 18:06:06.356712 7f5c973a57a0 20 read_log 192888'188 (0'0) modify 50af2eeb/benchmark_data_storage0111-ib.pocstor_11336_object12322/head//1 by client.1312877.0:12323 2014-08-11 16:25:56.330605 2014-08-13 18:06:06.356746 7f5c973a57a0 20 read_log 192888'189 (0'0) modify d5d1452b/benchmark_data_storage0111-ib.pocstor_11336_object12326/head//1 by client.1312877.0:12327 2014-08-11 16:25:56.345229 2014-08-13 18:06:06.356770 7f5c973a57a0 20 read_log 192888'190 (0'0) modify a8750ceb/benchmark_data_storage0111-ib.pocstor_11336_object12327/head//1 by client.1312877.0:12328 2014-08-11 16:25:56.348840 2014-08-13 18:06:06.356803 7f5c973a57a0 20 read_log 192888'191 (0'0) modify 6cd4e4eb/benchmark_data_storage0111-ib.pocstor_11336_object12360/head//1 by client.1312877.0:12361 2014-08-11 16:25:56.464192 2014-08-13 18:06:06.356833 7f5c973a57a0 20 read_log 192888'192 (0'0) modify e3f5d66b/benchmark_data_storage0111-ib.pocstor_11336_object12391/head//1 by client.1312877.0:12392 2014-08-11 16:25:56.576819 2014-08-13 18:06:06.356855 7f5c973a57a0 20 read_log 192888'193 (0'0) modify 75ceb6eb/benchmark_data_storage0111-ib.pocstor_11336_object12417/head//1 by client.1312877.0:12418 2014-08-11 16:25:56.669960 2014-08-13 18:06:06.356876 7f5c973a57a0 20 read_log 192888'194 (0'0) modify 8e72486b/benchmark_data_storage0111-ib.pocstor_11336_object12632/head//1 by client.1312877.0:12633 2014-08-11 16:25:57.439549 2014-08-13 18:06:06.356904 7f5c973a57a0 20 read_log 192888'195 (0'0) modify cce08cab/benchmark_data_storage0111-ib.pocstor_11336_object12674/head//1 by client.1312877.0:12675 2014-08-11 16:25:57.604735 2014-08-13 18:06:06.356946 7f5c973a57a0 20 read_log 192888'196 (0'0) modify 3b28fc6b/benchmark_data_storage0111-ib.pocstor_11336_object12837/head//1 by client.1312877.0:12838 2014-08-11 16:25:58.218206 2014-08-13 18:06:06.356968 7f5c973a57a0 20 read_log 192888'197 (0'0) modify d4d8526b/benchmark_data_storage0111-ib.pocstor_11336_object12951/head//1 by client.1312877.0:12952 2014-08-11 16:25:58.735928 2014-08-13 18:06:06.356989 7f5c973a57a0 20 read_log 192888'198 (0'0) modify 1454bcab/benchmark_data_storage0111-ib.pocstor_11336_object12984/head//1 by client.1312877.0:12985 2014-08-11 16:25:58.874452 2014-08-13 18:06:06.357013 7f5c973a57a0 20 read_log 192888'199 (0'0) modify 7ad66aab/benchmark_data_storage0111-ib.pocstor_11336_object13208/head//1 by client.1312877.0:13209 2014-08-11 16:25:59.940374 2014-08-13 18:06:06.357039 7f5c973a57a0 20 read_log 192888'200 (0'0) modify ef9a3c6b/benchmark_data_storage0111-ib.pocstor_11336_object13239/head//1 by client.1312877.0:13240 2014-08-11 16:26:00.105622 2014-08-13 18:06:06.357065 7f5c973a57a0 20 read_log 192888'201 (0'0) modify a87af4eb/benchmark_data_storage0111-ib.pocstor_11336_object13360/head//1 by client.1312877.0:13361 2014-08-11 16:26:00.687417 2014-08-13 18:06:06.357087 7f5c973a57a0 20 read_log 192888'202 (0'0) modify 2d5e4eab/benchmark_data_storage0111-ib.pocstor_11336_object13477/head//1 by client.1312877.0:13478 2014-08-11 16:26:01.317237 2014-08-13 18:06:06.357108 7f5c973a57a0 20 read_log 192888'203 (0'0) modify 6a50cd6b/benchmark_data_storage0111-ib.pocstor_11336_object13497/head//1 by client.1312877.0:13498 2014-08-11 16:26:01.393200 2014-08-13 18:06:06.357177 7f5c973a57a0 20 read_log 192888'204 (0'0) modify 34362b6b/benchmark_data_storage0111-ib.pocstor_11336_object13615/head//1 by client.1312877.0:13616 2014-08-11 16:26:02.038657 2014-08-13 18:06:06.357208 7f5c973a57a0 20 read_log 192888'205 (0'0) modify dfdebe2b/benchmark_data_storage0111-ib.pocstor_11336_object13622/head//1 by client.1312877.0:13623 2014-08-11 16:26:02.075108 2014-08-13 18:06:06.357243 7f5c973a57a0 20 read_log 192888'206 (0'0) modify 4c4339eb/benchmark_data_storage0111-ib.pocstor_11336_object13623/head//1 by client.1312877.0:13624 2014-08-11 16:26:02.080817 2014-08-13 18:06:06.357282 7f5c973a57a0 20 read_log 192888'207 (0'0) modify 898a53ab/benchmark_data_storage0111-ib.pocstor_11336_object14154/head//1 by client.1312877.0:14155 2014-08-11 16:26:04.271930 2014-08-13 18:06:06.357305 7f5c973a57a0 20 read_log 192888'208 (0'0) modify 48a3ff2b/benchmark_data_storage0111-ib.pocstor_11336_object14170/head//1 by client.1312877.0:14171 2014-08-11 16:26:04.328388 2014-08-13 18:06:06.357327 7f5c973a57a0 20 read_log 192888'209 (0'0) modify 6545c6ab/benchmark_data_storage0111-ib.pocstor_11336_object14196/head//1 by client.1312877.0:14197 2014-08-11 16:26:04.416323 2014-08-13 18:06:06.357348 7f5c973a57a0 20 read_log 192888'210 (0'0) modify da0040ab/benchmark_data_storage0111-ib.pocstor_11336_object14203/head//1 by client.1312877.0:14204 2014-08-11 16:26:04.439554 2014-08-13 18:06:06.357387 7f5c973a57a0 20 read_log 192888'211 (0'0) modify 3b3442ab/benchmark_data_storage0111-ib.pocstor_11336_object14209/head//1 by client.1312877.0:14210 2014-08-11 16:26:04.464633 2014-08-13 18:06:06.357410 7f5c973a57a0 20 read_log 192888'212 (0'0) modify 138d602b/benchmark_data_storage0111-ib.pocstor_11336_object14220/head//1 by client.1312877.0:14221 2014-08-11 16:26:04.501975 2014-08-13 18:06:06.357436 7f5c973a57a0 20 read_log 192888'213 (0'0) modify d6709e6b/benchmark_data_storage0111-ib.pocstor_11336_object14223/head//1 by client.1312877.0:14224 2014-08-11 16:26:04.512638 2014-08-13 18:06:06.357461 7f5c973a57a0 20 read_log 192888'214 (0'0) modify fe3dc06b/benchmark_data_storage0111-ib.pocstor_11336_object14364/head//1 by client.1312877.0:14365 2014-08-11 16:26:05.097526 2014-08-13 18:06:06.357483 7f5c973a57a0 20 read_log 192888'215 (0'0) modify 4a1483ab/benchmark_data_storage0111-ib.pocstor_11336_object14418/head//1 by client.1312877.0:14419 2014-08-11 16:26:05.296619 2014-08-13 18:06:06.357506 7f5c973a57a0 20 read_log 192888'216 (0'0) modify 89e9b56b/benchmark_data_storage0111-ib.pocstor_11336_object14424/head//1 by client.1312877.0:14425 2014-08-11 16:26:05.317343 2014-08-13 18:06:06.357529 7f5c973a57a0 20 read_log 192888'217 (0'0) modify d45ab2ab/benchmark_data_storage0111-ib.pocstor_11336_object14441/head//1 by client.1312877.0:14442 2014-08-11 16:26:05.377716 2014-08-13 18:06:06.357553 7f5c973a57a0 20 read_log 192888'218 (0'0) modify 35e2e1eb/benchmark_data_storage0111-ib.pocstor_11336_object14593/head//1 by client.1312877.0:14594 2014-08-11 16:26:05.968657 2014-08-13 18:06:06.357574 7f5c973a57a0 20 read_log 192888'219 (0'0) modify f9cba92b/benchmark_data_storage0111-ib.pocstor_11336_object14598/head//1 by client.1312877.0:14599 2014-08-11 16:26:05.990404 2014-08-13 18:06:06.357596 7f5c973a57a0 20 read_log 192888'220 (0'0) modify 5ddbe7eb/benchmark_data_storage0111-ib.pocstor_11336_object14613/head//1 by client.1312877.0:14614 2014-08-11 16:26:06.049295 2014-08-13 18:06:06.357618 7f5c973a57a0 20 read_log 192888'221 (0'0) modify 44537cab/benchmark_data_storage0111-ib.pocstor_11336_object14743/head//1 by client.1312877.0:14744 2014-08-11 16:26:06.545889 2014-08-13 18:06:06.357639 7f5c973a57a0 20 read_log 192888'222 (0'0) modify 3cd17fab/benchmark_data_storage0111-ib.pocstor_11336_object14794/head//1 by client.1312877.0:14795 2014-08-11 16:26:06.811632 2014-08-13 18:06:06.357661 7f5c973a57a0 20 read_log 192888'223 (0'0) modify fa8a42eb/benchmark_data_storage0111-ib.pocstor_11336_object14952/head//1 by client.1312877.0:14953 2014-08-11 16:26:07.385380 2014-08-13 18:06:06.357698 7f5c973a57a0 20 read_log 192888'224 (0'0) modify 96108eb/benchmark_data_storage0111-ib.pocstor_11336_object15084/head//1 by client.1312877.0:15085 2014-08-11 16:26:07.925003 2014-08-13 18:06:06.357725 7f5c973a57a0 20 read_log 192888'225 (0'0) modify 20b9f32b/benchmark_data_storage0111-ib.pocstor_11336_object15097/head//1 by client.1312877.0:15098 2014-08-11 16:26:07.975770 2014-08-13 18:06:06.357746 7f5c973a57a0 20 read_log 192888'226 (0'0) modify f6085eb/benchmark_data_storage0111-ib.pocstor_11336_object15386/head//1 by client.1312877.0:15387 2014-08-11 16:26:09.437707 2014-08-13 18:06:06.357768 7f5c973a57a0 20 read_log 192888'227 (0'0) modify f1deeceb/benchmark_data_storage0111-ib.pocstor_11336_object15406/head//1 by client.1312877.0:15407 2014-08-11 16:26:09.561403 2014-08-13 18:06:06.357789 7f5c973a57a0 20 read_log 192888'228 (0'0) modify 211e31ab/benchmark_data_storage0111-ib.pocstor_11336_object15650/head//1 by client.1312877.0:15651 2014-08-11 16:26:10.467129 2014-08-13 18:06:06.357810 7f5c973a57a0 20 read_log 192888'229 (0'0) modify 5ea9c7eb/benchmark_data_storage0111-ib.pocstor_11336_object15752/head//1 by client.1312877.0:15753 2014-08-11 16:26:10.830397 2014-08-13 18:06:06.357831 7f5c973a57a0 20 read_log 192888'230 (0'0) modify b8c42aab/benchmark_data_storage0111-ib.pocstor_11336_object15766/head//1 by client.1312877.0:15767 2014-08-11 16:26:10.877610 2014-08-13 18:06:06.357853 7f5c973a57a0 20 read_log 192888'231 (0'0) modify e8b2fbab/benchmark_data_storage0111-ib.pocstor_11336_object15797/head//1 by client.1312877.0:15798 2014-08-11 16:26:11.015944 2014-08-13 18:06:06.357874 7f5c973a57a0 20 read_log 192888'232 (0'0) modify 81b9caab/benchmark_data_storage0111-ib.pocstor_11336_object15801/head//1 by client.1312877.0:15802 2014-08-11 16:26:11.036260 2014-08-13 18:06:06.357895 7f5c973a57a0 20 read_log 192888'233 (0'0) modify 459c8bab/benchmark_data_storage0111-ib.pocstor_11336_object15913/head//1 by client.1312877.0:15914 2014-08-11 16:26:11.533291 2014-08-13 18:06:06.357917 7f5c973a57a0 20 read_log 192888'234 (0'0) modify 8fbd32b/benchmark_data_storage0111-ib.pocstor_11336_object15924/head//1 by client.1312877.0:15925 2014-08-11 16:26:11.585885 2014-08-13 18:06:06.357938 7f5c973a57a0 20 read_log 192888'235 (0'0) modify 709995eb/benchmark_data_storage0111-ib.pocstor_11336_object15934/head//1 by client.1312877.0:15935 2014-08-11 16:26:11.633813 2014-08-13 18:06:06.357959 7f5c973a57a0 20 read_log 192888'236 (0'0) modify 4f2410eb/benchmark_data_storage0111-ib.pocstor_11336_object15947/head//1 by client.1312877.0:15948 2014-08-11 16:26:11.682349 2014-08-13 18:06:06.358018 7f5c973a57a0 20 read_log 192888'237 (0'0) modify 4fadd6ab/benchmark_data_storage0111-ib.pocstor_11336_object16038/head//1 by client.1312877.0:16039 2014-08-11 16:26:12.059985 2014-08-13 18:06:06.358040 7f5c973a57a0 20 read_log 192888'238 (0'0) modify 5b14ab6b/benchmark_data_storage0111-ib.pocstor_11336_object16162/head//1 by client.1312877.0:16163 2014-08-11 16:26:12.982216 2014-08-13 18:06:06.358065 7f5c973a57a0 20 read_log 192888'239 (0'0) modify 6ab0302b/benchmark_data_storage0111-ib.pocstor_11336_object16219/head//1 by client.1312877.0:16220 2014-08-11 16:26:13.216689 2014-08-13 18:06:06.358117 7f5c973a57a0 20 read_log 192888'240 (0'0) modify b76318eb/benchmark_data_storage0111-ib.pocstor_11336_object16290/head//1 by client.1312877.0:16291 2014-08-11 16:26:13.583907 2014-08-13 18:06:06.358143 7f5c973a57a0 20 read_log 192888'241 (0'0) modify 89d8c5eb/benchmark_data_storage0111-ib.pocstor_11336_object16371/head//1 by client.1312877.0:16372 2014-08-11 16:26:13.878059 2014-08-13 18:06:06.358183 7f5c973a57a0 20 read_log 192888'242 (0'0) modify 88c4ec2b/benchmark_data_storage0111-ib.pocstor_11336_object16404/head//1 by client.1312877.0:16405 2014-08-11 16:26:13.992092 2014-08-13 18:06:06.358206 7f5c973a57a0 20 read_log 192888'243 (0'0) modify d2930fab/benchmark_data_storage0111-ib.pocstor_11336_object16420/head//1 by client.1312877.0:16421 2014-08-11 16:26:14.081966 2014-08-13 18:06:06.358269 7f5c973a57a0 20 read_log 192888'244 (0'0) modify 4bcd04eb/benchmark_data_storage0111-ib.pocstor_11336_object16464/head//1 by client.1312877.0:16465 2014-08-11 16:26:14.296561 2014-08-13 18:06:06.358296 7f5c973a57a0 20 read_log 192888'245 (0'0) modify f5411b6b/benchmark_data_storage0111-ib.pocstor_11336_object16539/head//1 by client.1312877.0:16540 2014-08-11 16:26:14.572867 2014-08-13 18:06:06.358318 7f5c973a57a0 20 read_log 192888'246 (0'0) modify 5b8992ab/benchmark_data_storage0111-ib.pocstor_11336_object16582/head//1 by client.1312877.0:16583 2014-08-11 16:26:14.800789 2014-08-13 18:06:06.358345 7f5c973a57a0 20 read_log 192888'247 (0'0) modify 968ceeeb/benchmark_data_storage0111-ib.pocstor_11336_object16597/head//1 by client.1312877.0:16598 2014-08-11 16:26:14.894820 2014-08-13 18:06:06.358367 7f5c973a57a0 20 read_log 192888'248 (0'0) modify 478cf8eb/benchmark_data_storage0111-ib.pocstor_11336_object16708/head//1 by client.1312877.0:16709 2014-08-11 16:26:15.468936 2014-08-13 18:06:06.358388 7f5c973a57a0 20 read_log 192888'249 (0'0) modify e3afee6b/benchmark_data_storage0111-ib.pocstor_11336_object16711/head//1 by client.1312877.0:16712 2014-08-11 16:26:15.484023 2014-08-13 18:06:06.358419 7f5c973a57a0 20 read_log 192888'250 (0'0) modify b42d546b/benchmark_data_storage0111-ib.pocstor_11336_object16737/head//1 by client.1312877.0:16738 2014-08-11 16:26:15.601881 2014-08-13 18:06:06.358441 7f5c973a57a0 20 read_log 192888'251 (0'0) modify 6b39d7ab/benchmark_data_storage0111-ib.pocstor_11336_object16757/head//1 by client.1312877.0:16758 2014-08-11 16:26:15.689243 2014-08-13 18:06:06.358484 7f5c973a57a0 20 read_log 192888'252 (0'0) modify 6c846b6b/benchmark_data_storage0111-ib.pocstor_11336_object16841/head//1 by client.1312877.0:16842 2014-08-11 16:26:16.021770 2014-08-13 18:06:06.358514 7f5c973a57a0 20 read_log 192888'253 (0'0) modify 15deb2eb/benchmark_data_storage0111-ib.pocstor_11336_object16904/head//1 by client.1312877.0:16905 2014-08-11 16:26:16.355920 2014-08-13 18:06:06.358541 7f5c973a57a0 20 read_log 192888'254 (0'0) modify 16e9aaeb/benchmark_data_storage0111-ib.pocstor_11336_object17042/head//1 by client.1312877.0:17043 2014-08-11 16:26:16.843505 2014-08-13 18:06:06.358575 7f5c973a57a0 20 read_log 192888'255 (0'0) modify d0004aeb/benchmark_data_storage0111-ib.pocstor_11336_object17045/head//1 by client.1312877.0:17046 2014-08-11 16:26:16.854420 2014-08-13 18:06:06.358614 7f5c973a57a0 20 read_log 192888'256 (0'0) modify 96728e6b/benchmark_data_storage0111-ib.pocstor_11336_object17143/head//1 by client.1312877.0:17144 2014-08-11 16:26:17.192096 2014-08-13 18:06:06.358643 7f5c973a57a0 20 read_log 192888'257 (0'0) modify f7e3ef2b/benchmark_data_storage0111-ib.pocstor_11336_object17271/head//1 by client.1312877.0:17272 2014-08-11 16:26:17.640454 2014-08-13 18:06:06.358666 7f5c973a57a0 20 read_log 192888'258 (0'0) modify cb867cab/benchmark_data_storage0111-ib.pocstor_11336_object17286/head//1 by client.1312877.0:17287 2014-08-11 16:26:17.692229 2014-08-13 18:06:06.358687 7f5c973a57a0 20 read_log 192888'259 (0'0) modify 286a36b/benchmark_data_storage0111-ib.pocstor_11336_object17353/head//1 by client.1312877.0:17354 2014-08-11 16:26:17.948725 2014-08-13 18:06:06.358717 7f5c973a57a0 20 read_log 192888'260 (0'0) modify 57509cab/benchmark_data_storage0111-ib.pocstor_11336_object17406/head//1 by client.1312877.0:17407 2014-08-11 16:26:18.185142 2014-08-13 18:06:06.358743 7f5c973a57a0 20 read_log 192888'261 (0'0) modify 1978a66b/benchmark_data_storage0111-ib.pocstor_11336_object17504/head//1 by client.1312877.0:17505 2014-08-11 16:26:18.535967 2014-08-13 18:06:06.358764 7f5c973a57a0 20 read_log 192888'262 (0'0) modify 1ff709eb/benchmark_data_storage0111-ib.pocstor_11336_object17536/head//1 by client.1312877.0:17537 2014-08-11 16:26:18.652135 2014-08-13 18:06:06.358786 7f5c973a57a0 20 read_log 192888'263 (0'0) modify 504892ab/benchmark_data_storage0111-ib.pocstor_11336_object17552/head//1 by client.1312877.0:17553 2014-08-11 16:26:18.706436 2014-08-13 18:06:06.358835 7f5c973a57a0 20 read_log 192888'264 (0'0) modify aa8340ab/benchmark_data_storage0111-ib.pocstor_11336_object17582/head//1 by client.1312877.0:17583 2014-08-11 16:26:18.812276 2014-08-13 18:06:06.358863 7f5c973a57a0 20 read_log 192888'265 (0'0) modify b666282b/benchmark_data_storage0111-ib.pocstor_11336_object17624/head//1 by client.1312877.0:17625 2014-08-11 16:26:18.956782 2014-08-13 18:06:06.358907 7f5c973a57a0 20 read_log 192888'266 (0'0) modify 792ade6b/benchmark_data_storage0111-ib.pocstor_11336_object17776/head//1 by client.1312877.0:17777 2014-08-11 16:26:19.572287 2014-08-13 18:06:06.358934 7f5c973a57a0 20 read_log 192888'267 (0'0) modify e4e4e3ab/benchmark_data_storage0111-ib.pocstor_11336_object17791/head//1 by client.1312877.0:17792 2014-08-11 16:26:19.628627 2014-08-13 18:06:06.358963 7f5c973a57a0 20 read_log 192888'268 (0'0) modify e22515eb/benchmark_data_storage0111-ib.pocstor_11336_object17812/head//1 by client.1312877.0:17813 2014-08-11 16:26:19.699954 2014-08-13 18:06:06.359006 7f5c973a57a0 20 read_log 192888'269 (0'0) modify 378d10eb/benchmark_data_storage0111-ib.pocstor_11336_object17841/head//1 by client.1312877.0:17842 2014-08-11 16:26:19.801003 2014-08-13 18:06:06.359031 7f5c973a57a0 20 read_log 192888'270 (0'0) modify a1c1b56b/benchmark_data_storage0111-ib.pocstor_11336_object17879/head//1 by client.1312877.0:17880 2014-08-11 16:26:19.943818 2014-08-13 18:06:06.359069 7f5c973a57a0 20 read_log 192888'271 (0'0) modify 2d29506b/benchmark_data_storage0111-ib.pocstor_11336_object18047/head//1 by client.1312877.0:18048 2014-08-11 16:26:20.739758 2014-08-13 18:06:06.359091 7f5c973a57a0 20 read_log 192888'272 (0'0) modify a954006b/benchmark_data_storage0111-ib.pocstor_11336_object18067/head//1 by client.1312877.0:18068 2014-08-11 16:26:20.923256 2014-08-13 18:06:06.359113 7f5c973a57a0 20 read_log 192888'273 (0'0) modify 3ff41aeb/benchmark_data_storage0111-ib.pocstor_11336_object18232/head//1 by client.1312877.0:18233 2014-08-11 16:26:21.998756 2014-08-13 18:06:06.359134 7f5c973a57a0 20 read_log 192888'274 (0'0) modify 1953402b/benchmark_data_storage0111-ib.pocstor_11336_object18323/head//1 by client.1312877.0:18324 2014-08-11 16:26:22.460038 2014-08-13 18:06:06.359165 7f5c973a57a0 20 read_log 192888'275 (0'0) modify c2fa4d2b/benchmark_data_storage0111-ib.pocstor_11336_object18387/head//1 by client.1312877.0:18388 2014-08-11 16:26:22.691623 2014-08-13 18:06:06.359187 7f5c973a57a0 20 read_log 192888'276 (0'0) modify 9efed92b/benchmark_data_storage0111-ib.pocstor_11336_object18517/head//1 by client.1312877.0:18518 2014-08-11 16:26:23.145076 2014-08-13 18:06:06.359209 7f5c973a57a0 20 read_log 192888'277 (0'0) modify 1846c0eb/benchmark_data_storage0111-ib.pocstor_11336_object18527/head//1 by client.1312877.0:18528 2014-08-11 16:26:23.181015 2014-08-13 18:06:06.359230 7f5c973a57a0 20 read_log 192888'278 (0'0) modify 275089ab/benchmark_data_storage0111-ib.pocstor_11336_object18528/head//1 by client.1312877.0:18529 2014-08-11 16:26:23.183648 2014-08-13 18:06:06.359251 7f5c973a57a0 20 read_log 192888'279 (0'0) modify bcd079eb/benchmark_data_storage0111-ib.pocstor_11336_object18542/head//1 by client.1312877.0:18543 2014-08-11 16:26:23.233281 2014-08-13 18:06:06.359273 7f5c973a57a0 20 read_log 192888'280 (0'0) modify 52868e6b/benchmark_data_storage0111-ib.pocstor_11336_object18592/head//1 by client.1312877.0:18593 2014-08-11 16:26:23.408111 2014-08-13 18:06:06.359294 7f5c973a57a0 20 read_log 192888'281 (0'0) modify 426531eb/benchmark_data_storage0111-ib.pocstor_11336_object18604/head//1 by client.1312877.0:18605 2014-08-11 16:26:23.449747 2014-08-13 18:06:06.359325 7f5c973a57a0 20 read_log 192888'282 (0'0) modify 2aad1c6b/benchmark_data_storage0111-ib.pocstor_11336_object18663/head//1 by client.1312877.0:18664 2014-08-11 16:26:23.661060 2014-08-13 18:06:06.359352 7f5c973a57a0 20 read_log 192888'283 (0'0) modify 5869a02b/benchmark_data_storage0111-ib.pocstor_11336_object18781/head//1 by client.1312877.0:18782 2014-08-11 16:26:24.108098 2014-08-13 18:06:06.359386 7f5c973a57a0 20 read_log 192888'284 (0'0) modify 4187e2eb/benchmark_data_storage0111-ib.pocstor_11336_object18834/head//1 by client.1312877.0:18835 2014-08-11 16:26:24.289663 2014-08-13 18:06:06.359431 7f5c973a57a0 20 read_log 192888'285 (0'0) modify 4f44796b/benchmark_data_storage0111-ib.pocstor_11336_object18875/head//1 by client.1312877.0:18876 2014-08-11 16:26:24.427360 2014-08-13 18:06:06.359466 7f5c973a57a0 20 read_log 192888'286 (0'0) modify fd4b836b/benchmark_data_storage0111-ib.pocstor_11336_object18888/head//1 by client.1312877.0:18889 2014-08-11 16:26:24.473249 2014-08-13 18:06:06.359506 7f5c973a57a0 20 read_log 192888'287 (0'0) modify 2b80b52b/benchmark_data_storage0111-ib.pocstor_11336_object18990/head//1 by client.1312877.0:18991 2014-08-11 16:26:24.828966 2014-08-13 18:06:06.359528 7f5c973a57a0 20 read_log 192888'288 (0'0) modify 1c3083ab/benchmark_data_storage0111-ib.pocstor_11336_object19003/head//1 by client.1312877.0:19004 2014-08-11 16:26:24.875878 2014-08-13 18:06:06.359550 7f5c973a57a0 20 read_log 192888'289 (0'0) modify c2464ceb/benchmark_data_storage0111-ib.pocstor_11336_object19103/head//1 by client.1312877.0:19104 2014-08-11 16:26:25.228126 2014-08-13 18:06:06.359571 7f5c973a57a0 20 read_log 192888'290 (0'0) modify 122a33ab/benchmark_data_storage0111-ib.pocstor_11336_object19227/head//1 by client.1312877.0:19228 2014-08-11 16:26:25.665095 2014-08-13 18:06:06.359593 7f5c973a57a0 20 read_log 192888'291 (0'0) modify e7ba396b/benchmark_data_storage0111-ib.pocstor_11336_object19274/head//1 by client.1312877.0:19275 2014-08-11 16:26:25.858676 2014-08-13 18:06:06.359614 7f5c973a57a0 20 read_log 192888'292 (0'0) modify 9bd5cdeb/benchmark_data_storage0111-ib.pocstor_11336_object19328/head//1 by client.1312877.0:19329 2014-08-11 16:26:26.119132 2014-08-13 18:06:06.359637 7f5c973a57a0 20 read_log 192888'293 (0'0) modify d0eba76b/benchmark_data_storage0111-ib.pocstor_11336_object19418/head//1 by client.1312877.0:19419 2014-08-11 16:26:26.505366 2014-08-13 18:06:06.359658 7f5c973a57a0 20 read_log 192888'294 (0'0) modify 1dc295eb/benchmark_data_storage0111-ib.pocstor_11336_object19457/head//1 by client.1312877.0:19458 2014-08-11 16:26:26.700667 2014-08-13 18:06:06.359680 7f5c973a57a0 20 read_log 192888'295 (0'0) modify 1dd11fab/benchmark_data_storage0111-ib.pocstor_11336_object19469/head//1 by client.1312877.0:19470 2014-08-11 16:26:26.743086 2014-08-13 18:06:06.359701 7f5c973a57a0 20 read_log 192888'296 (0'0) modify c64d172b/benchmark_data_storage0111-ib.pocstor_11336_object19516/head//1 by client.1312877.0:19517 2014-08-11 16:26:26.918807 2014-08-13 18:06:06.359723 7f5c973a57a0 20 read_log 192888'297 (0'0) modify 475791eb/benchmark_data_storage0111-ib.pocstor_11336_object19587/head//1 by client.1312877.0:19588 2014-08-11 16:26:27.171672 2014-08-13 18:06:06.359745 7f5c973a57a0 20 read_log 192888'298 (0'0) modify e2681e6b/benchmark_data_storage0111-ib.pocstor_11336_object19629/head//1 by client.1312877.0:19630 2014-08-11 16:26:27.322787 2014-08-13 18:06:06.359767 7f5c973a57a0 20 read_log 192888'299 (0'0) modify e272fb6b/benchmark_data_storage0111-ib.pocstor_11336_object19824/head//1 by client.1312877.0:19825 2014-08-11 16:26:28.356552 2014-08-13 18:06:06.359788 7f5c973a57a0 20 read_log 192888'300 (0'0) modify 24e902b/benchmark_data_storage0111-ib.pocstor_11336_object19842/head//1 by client.1312877.0:19843 2014-08-11 16:26:28.474235 2014-08-13 18:06:06.359809 7f5c973a57a0 20 read_log 192888'301 (0'0) modify d95a362b/benchmark_data_storage0111-ib.pocstor_11336_object19869/head//1 by client.1312877.0:19870 2014-08-11 16:26:28.733567 2014-08-13 18:06:06.359830 7f5c973a57a0 20 read_log 192888'302 (0'0) modify 46edc8eb/benchmark_data_storage0111-ib.pocstor_11336_object19881/head//1 by client.1312877.0:19882 2014-08-11 16:26:28.894063 2014-08-13 18:06:06.359852 7f5c973a57a0 20 read_log 192888'303 (0'0) modify 17820beb/benchmark_data_storage0111-ib.pocstor_11336_object19886/head//1 by client.1312877.0:19887 2014-08-11 16:26:29.384621 2014-08-13 18:06:06.359889 7f5c973a57a0 20 read_log 192888'304 (0'0) modify 86a8e56b/benchmark_data_storage0111-ib.pocstor_11336_object19954/head//1 by client.1312877.0:19955 2014-08-11 16:26:29.691965 2014-08-13 18:06:06.359916 7f5c973a57a0 20 read_log 192888'305 (0'0) modify 68ddf0ab/benchmark_data_storage0111-ib.pocstor_11336_object19987/head//1 by client.1312877.0:19988 2014-08-11 16:26:29.808390 2014-08-13 18:06:06.359938 7f5c973a57a0 20 read_log 192888'306 (0'0) modify 1fe4d72b/benchmark_data_storage0111-ib.pocstor_11336_object20033/head//1 by client.1312877.0:20034 2014-08-11 16:26:29.971244 2014-08-13 18:06:06.359959 7f5c973a57a0 20 read_log 192888'307 (0'0) modify a91445eb/benchmark_data_storage0111-ib.pocstor_11336_object20056/head//1 by client.1312877.0:20057 2014-08-11 16:26:30.052028 2014-08-13 18:06:06.359981 7f5c973a57a0 20 read_log 192888'308 (0'0) modify 7faf2f6b/benchmark_data_storage0111-ib.pocstor_11336_object20068/head//1 by client.1312877.0:20069 2014-08-11 16:26:30.096129 2014-08-13 18:06:06.360002 7f5c973a57a0 20 read_log 192888'309 (0'0) modify 8b39baab/benchmark_data_storage0111-ib.pocstor_11336_object20099/head//1 by client.1312877.0:20100 2014-08-11 16:26:30.204525 2014-08-13 18:06:06.360023 7f5c973a57a0 20 read_log 192888'310 (0'0) modify f67c52b/benchmark_data_storage0111-ib.pocstor_11336_object20190/head//1 by client.1312877.0:20191 2014-08-11 16:26:30.526991 2014-08-13 18:06:06.360044 7f5c973a57a0 20 read_log 192888'311 (0'0) modify 9b83842b/benchmark_data_storage0111-ib.pocstor_11336_object20253/head//1 by client.1312877.0:20254 2014-08-11 16:26:30.760511 2014-08-13 18:06:06.360065 7f5c973a57a0 20 read_log 192888'312 (0'0) modify 5f94746b/benchmark_data_storage0111-ib.pocstor_11336_object20281/head//1 by client.1312877.0:20282 2014-08-11 16:26:30.863300 2014-08-13 18:06:06.360086 7f5c973a57a0 20 read_log 192888'313 (0'0) modify f708caab/benchmark_data_storage0111-ib.pocstor_11336_object20467/head//1 by client.1312877.0:20468 2014-08-11 16:26:31.851549 2014-08-13 18:06:06.360108 7f5c973a57a0 20 read_log 192888'314 (0'0) modify 5b14d96b/benchmark_data_storage0111-ib.pocstor_11336_object20478/head//1 by client.1312877.0:20479 2014-08-11 16:26:31.980115 2014-08-13 18:06:06.360129 7f5c973a57a0 20 read_log 192888'315 (0'0) modify b820eb2b/benchmark_data_storage0111-ib.pocstor_11336_object20492/head//1 by client.1312877.0:20493 2014-08-11 16:26:32.232457 2014-08-13 18:06:06.360159 7f5c973a57a0 20 read_log 192888'316 (0'0) modify d119eab/benchmark_data_storage0111-ib.pocstor_11336_object20632/head//1 by client.1312877.0:20633 2014-08-11 16:26:32.902642 2014-08-13 18:06:06.360181 7f5c973a57a0 20 read_log 192888'317 (0'0) modify 11cba56b/benchmark_data_storage0111-ib.pocstor_11336_object20947/head//1 by client.1312877.0:20948 2014-08-11 16:26:34.361669 2014-08-13 18:06:06.360203 7f5c973a57a0 20 read_log 192888'318 (0'0) modify 60a1adeb/benchmark_data_storage0111-ib.pocstor_11336_object21180/head//1 by client.1312877.0:21181 2014-08-11 16:26:35.381536 2014-08-13 18:06:06.360228 7f5c973a57a0 20 read_log 192888'319 (0'0) modify 7125f2ab/benchmark_data_storage0111-ib.pocstor_11336_object21256/head//1 by client.1312877.0:21257 2014-08-11 16:26:35.733707 2014-08-13 18:06:06.360257 7f5c973a57a0 20 read_log 192888'320 (0'0) modify 7726eb2b/benchmark_data_storage0111-ib.pocstor_11336_object21372/head//1 by client.1312877.0:21373 2014-08-11 16:26:36.885139 2014-08-13 18:06:06.360280 7f5c973a57a0 20 read_log 192888'321 (0'0) modify 242250ab/benchmark_data_storage0111-ib.pocstor_11336_object21400/head//1 by client.1312877.0:21401 2014-08-11 16:26:36.989791 2014-08-13 18:06:06.360304 7f5c973a57a0 20 read_log 192888'322 (0'0) modify 54fc23ab/benchmark_data_storage0111-ib.pocstor_11336_object21424/head//1 by client.1312877.0:21425 2014-08-11 16:26:37.081022 2014-08-13 18:06:06.360335 7f5c973a57a0 20 read_log 192888'323 (0'0) modify e546a22b/benchmark_data_storage0111-ib.pocstor_11336_object21443/head//1 by client.1312877.0:21444 2014-08-11 16:26:37.175245 2014-08-13 18:06:06.360374 7f5c973a57a0 20 read_log 192888'324 (0'0) modify b69c4a2b/benchmark_data_storage0111-ib.pocstor_11336_object21466/head//1 by client.1312877.0:21467 2014-08-11 16:26:37.297286 2014-08-13 18:06:06.360400 7f5c973a57a0 20 read_log 192888'325 (0'0) modify 5923652b/benchmark_data_storage0111-ib.pocstor_11336_object21528/head//1 by client.1312877.0:21529 2014-08-11 16:26:37.596583 2014-08-13 18:06:06.360437 7f5c973a57a0 20 read_log 192888'326 (0'0) modify da17c26b/benchmark_data_storage0111-ib.pocstor_11336_object21763/head//1 by client.1312877.0:21764 2014-08-11 16:26:38.579171 2014-08-13 18:06:06.360470 7f5c973a57a0 20 read_log 192888'327 (0'0) modify 87148eb/benchmark_data_storage0111-ib.pocstor_11336_object21812/head//1 by client.1312877.0:21813 2014-08-11 16:26:38.780873 2014-08-13 18:06:06.360499 7f5c973a57a0 20 read_log 192888'328 (0'0) modify 9101e96b/benchmark_data_storage0111-ib.pocstor_11336_object21868/head//1 by client.1312877.0:21869 2014-08-11 16:26:38.994835 2014-08-13 18:06:06.360539 7f5c973a57a0 20 read_log 192888'329 (0'0) modify 7c4fe4ab/benchmark_data_storage0111-ib.pocstor_11336_object21885/head//1 by client.1312877.0:21886 2014-08-11 16:26:39.079840 2014-08-13 18:06:06.360562 7f5c973a57a0 20 read_log 192888'330 (0'0) modify ad24d8ab/benchmark_data_storage0111-ib.pocstor_11336_object21935/head//1 by client.1312877.0:21936 2014-08-11 16:26:39.313238 2014-08-13 18:06:06.360583 7f5c973a57a0 20 read_log 192888'331 (0'0) modify 787deceb/benchmark_data_storage0111-ib.pocstor_11336_object21956/head//1 by client.1312877.0:21957 2014-08-11 16:26:39.391347 2014-08-13 18:06:06.360617 7f5c973a57a0 20 read_log 192888'332 (0'0) modify f978002b/benchmark_data_storage0111-ib.pocstor_11336_object21997/head//1 by client.1312877.0:21998 2014-08-11 16:26:39.547116 2014-08-13 18:06:06.360685 7f5c973a57a0 20 read_log 192888'333 (0'0) modify 959dd26b/benchmark_data_storage0111-ib.pocstor_11336_object22000/head//1 by client.1312877.0:22001 2014-08-11 16:26:39.558471 2014-08-13 18:06:06.360741 7f5c973a57a0 20 read_log 192888'334 (0'0) modify 4e472ceb/benchmark_data_storage0111-ib.pocstor_11336_object22034/head//1 by client.1312877.0:22035 2014-08-11 16:26:39.687552 2014-08-13 18:06:06.360806 7f5c973a57a0 20 read_log 192888'335 (0'0) modify 6c1ac06b/benchmark_data_storage0111-ib.pocstor_11336_object22051/head//1 by client.1312877.0:22052 2014-08-11 16:26:39.751101 2014-08-13 18:06:06.360836 7f5c973a57a0 20 read_log 192888'336 (0'0) modify 373a052b/benchmark_data_storage0111-ib.pocstor_11336_object22091/head//1 by client.1312877.0:22092 2014-08-11 16:26:39.900417 2014-08-13 18:06:06.360862 7f5c973a57a0 20 read_log 192888'337 (0'0) modify 31cfc02b/benchmark_data_storage0111-ib.pocstor_11336_object22258/head//1 by client.1312877.0:22259 2014-08-11 16:26:40.598516 2014-08-13 18:06:06.360884 7f5c973a57a0 20 read_log 192888'338 (0'0) modify 6de7eaab/benchmark_data_storage0111-ib.pocstor_11336_object22272/head//1 by client.1312877.0:22273 2014-08-11 16:26:40.663523 2014-08-13 18:06:06.360905 7f5c973a57a0 20 read_log 192888'339 (0'0) modify 1781c26b/benchmark_data_storage0111-ib.pocstor_11336_object22296/head//1 by client.1312877.0:22297 2014-08-11 16:26:40.758480 2014-08-13 18:06:06.360929 7f5c973a57a0 20 read_log 192888'340 (0'0) modify f157deab/benchmark_data_storage0111-ib.pocstor_11336_object22346/head//1 by client.1312877.0:22347 2014-08-11 16:26:40.955046 2014-08-13 18:06:06.361000 7f5c973a57a0 20 read_log 192888'341 (0'0) modify f68908eb/benchmark_data_storage0111-ib.pocstor_11336_object22441/head//1 by client.1312877.0:22442 2014-08-11 16:26:41.304878 2014-08-13 18:06:06.361034 7f5c973a57a0 20 read_log 192888'342 (0'0) modify a20011ab/benchmark_data_storage0111-ib.pocstor_11336_object22467/head//1 by client.1312877.0:22468 2014-08-11 16:26:41.397941 2014-08-13 18:06:06.361100 7f5c973a57a0 20 read_log 192888'343 (0'0) modify 82490b2b/benchmark_data_storage0111-ib.pocstor_11336_object22469/head//1 by client.1312877.0:22470 2014-08-11 16:26:41.405354 2014-08-13 18:06:06.361178 7f5c973a57a0 20 read_log 192888'344 (0'0) modify 520cf2eb/benchmark_data_storage0111-ib.pocstor_11336_object22473/head//1 by client.1312877.0:22474 2014-08-11 16:26:41.418601 2014-08-13 18:06:06.361209 7f5c973a57a0 20 read_log 192888'345 (0'0) modify 42e912ab/benchmark_data_storage0111-ib.pocstor_11336_object22545/head//1 by client.1312877.0:22546 2014-08-11 16:26:41.803872 2014-08-13 18:06:06.361237 7f5c973a57a0 20 read_log 192888'346 (0'0) modify 96bbf22b/benchmark_data_storage0111-ib.pocstor_11336_object22579/head//1 by client.1312877.0:22580 2014-08-11 16:26:41.923292 2014-08-13 18:06:06.361264 7f5c973a57a0 20 read_log 192888'347 (0'0) modify a0f15c6b/benchmark_data_storage0111-ib.pocstor_11336_object22587/head//1 by client.1312877.0:22588 2014-08-11 16:26:41.956565 2014-08-13 18:06:06.361303 7f5c973a57a0 20 read_log 192888'348 (0'0) modify 698c546b/benchmark_data_storage0111-ib.pocstor_11336_object22636/head//1 by client.1312877.0:22637 2014-08-11 16:26:42.159265 2014-08-13 18:06:06.361328 7f5c973a57a0 20 read_log 192888'349 (0'0) modify a9a305eb/benchmark_data_storage0111-ib.pocstor_11336_object22716/head//1 by client.1312877.0:22717 2014-08-11 16:26:42.550061 2014-08-13 18:06:06.361356 7f5c973a57a0 20 read_log 192888'350 (0'0) modify 20cdd2b/benchmark_data_storage0111-ib.pocstor_11336_object22738/head//1 by client.1312877.0:22739 2014-08-11 16:26:42.642179 2014-08-13 18:06:06.361378 7f5c973a57a0 20 read_log 192888'351 (0'0) modify 13e43f2b/benchmark_data_storage0111-ib.pocstor_11336_object22869/head//1 by client.1312877.0:22870 2014-08-11 16:26:43.345894 2014-08-13 18:06:06.361415 7f5c973a57a0 20 read_log 192888'352 (0'0) modify efa1d42b/benchmark_data_storage0111-ib.pocstor_11336_object22914/head//1 by client.1312877.0:22915 2014-08-11 16:26:43.690619 2014-08-13 18:06:06.361449 7f5c973a57a0 20 read_log 192888'353 (0'0) modify 96aeefeb/benchmark_data_storage0111-ib.pocstor_11336_object22932/head//1 by client.1312877.0:22933 2014-08-11 16:26:43.849886 2014-08-13 18:06:06.361481 7f5c973a57a0 20 read_log 192888'354 (0'0) modify 1a986b6b/benchmark_data_storage0111-ib.pocstor_11336_object22937/head//1 by client.1312877.0:22938 2014-08-11 16:26:43.880915 2014-08-13 18:06:06.361513 7f5c973a57a0 20 read_log 192888'355 (0'0) modify a0706d2b/benchmark_data_storage0111-ib.pocstor_11336_object22960/head//1 by client.1312877.0:22961 2014-08-11 16:26:44.122557 2014-08-13 18:06:06.361555 7f5c973a57a0 20 read_log 192888'356 (0'0) modify 8e2579ab/benchmark_data_storage0111-ib.pocstor_11336_object22962/head//1 by client.1312877.0:22963 2014-08-11 16:26:44.141679 2014-08-13 18:06:06.361594 7f5c973a57a0 20 read_log 192888'357 (0'0) modify 7ebd15eb/benchmark_data_storage0111-ib.pocstor_11336_object23023/head//1 by client.1312877.0:23024 2014-08-11 16:26:44.487575 2014-08-13 18:06:06.361621 7f5c973a57a0 20 read_log 192888'358 (0'0) modify 53266aeb/benchmark_data_storage0111-ib.pocstor_11336_object23033/head//1 by client.1312877.0:23034 2014-08-11 16:26:44.525347 2014-08-13 18:06:06.361643 7f5c973a57a0 20 read_log 192888'359 (0'0) modify 45b50ab/benchmark_data_storage0111-ib.pocstor_11336_object23056/head//1 by client.1312877.0:23057 2014-08-11 16:26:44.603328 2014-08-13 18:06:06.361665 7f5c973a57a0 20 read_log 192888'360 (0'0) modify 686380ab/benchmark_data_storage0111-ib.pocstor_11336_object23170/head//1 by client.1312877.0:23171 2014-08-11 16:26:45.014761 2014-08-13 18:06:06.361686 7f5c973a57a0 20 read_log 192888'361 (0'0) modify a62bfc6b/benchmark_data_storage0111-ib.pocstor_11336_object23205/head//1 by client.1312877.0:23206 2014-08-11 16:26:45.167732 2014-08-13 18:06:06.361708 7f5c973a57a0 20 read_log 192888'362 (0'0) modify b6784d6b/benchmark_data_storage0111-ib.pocstor_11336_object23214/head//1 by client.1312877.0:23215 2014-08-11 16:26:45.202423 2014-08-13 18:06:06.361731 7f5c973a57a0 20 read_log 192888'363 (0'0) modify e4717fab/benchmark_data_storage0111-ib.pocstor_11336_object23363/head//1 by client.1312877.0:23364 2014-08-11 16:26:45.763882 2014-08-13 18:06:06.361800 7f5c973a57a0 20 read_log 192888'364 (0'0) modify 793e702b/benchmark_data_storage0111-ib.pocstor_11336_object23422/head//1 by client.1312877.0:23423 2014-08-11 16:26:45.976633 2014-08-13 18:06:06.361836 7f5c973a57a0 20 read_log 192888'365 (0'0) modify 90f044eb/benchmark_data_storage0111-ib.pocstor_11336_object23495/head//1 by client.1312877.0:23496 2014-08-11 16:26:46.326997 2014-08-13 18:06:06.361863 7f5c973a57a0 20 read_log 192888'366 (0'0) modify 30bf61ab/benchmark_data_storage0111-ib.pocstor_11336_object23554/head//1 by client.1312877.0:23555 2014-08-11 16:26:46.693405 2014-08-13 18:06:06.361886 7f5c973a57a0 20 read_log 192888'367 (0'0) modify c494546b/benchmark_data_storage0111-ib.pocstor_11336_object23580/head//1 by client.1312877.0:23581 2014-08-11 16:26:46.880615 2014-08-13 18:06:06.361908 7f5c973a57a0 20 read_log 192888'368 (0'0) modify 60e4f62b/benchmark_data_storage0111-ib.pocstor_11336_object23637/head//1 by client.1312877.0:23638 2014-08-11 16:26:47.341780 2014-08-13 18:06:06.361929 7f5c973a57a0 20 read_log 192888'369 (0'0) modify 4c4bf4ab/benchmark_data_storage0111-ib.pocstor_11336_object23654/head//1 by client.1312877.0:23655 2014-08-11 16:26:47.403860 2014-08-13 18:06:06.361951 7f5c973a57a0 20 read_log 192888'370 (0'0) modify a946212b/benchmark_data_storage0111-ib.pocstor_11336_object23753/head//1 by client.1312877.0:23754 2014-08-11 16:26:47.833072 2014-08-13 18:06:06.361972 7f5c973a57a0 20 read_log 192888'371 (0'0) modify a1fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object23786/head//1 by client.1312877.0:23787 2014-08-11 16:26:47.966950 2014-08-13 18:06:06.361993 7f5c973a57a0 20 read_log 192888'372 (0'0) modify b45acb2b/benchmark_data_storage0111-ib.pocstor_11336_object23815/head//1 by client.1312877.0:23816 2014-08-11 16:26:48.088858 2014-08-13 18:06:06.362014 7f5c973a57a0 20 read_log 192888'373 (0'0) modify f2e080eb/benchmark_data_storage0111-ib.pocstor_11336_object23867/head//1 by client.1312877.0:23868 2014-08-11 16:26:48.418532 2014-08-13 18:06:06.362036 7f5c973a57a0 20 read_log 192888'374 (0'0) modify a083866b/benchmark_data_storage0111-ib.pocstor_11336_object23921/head//1 by client.1312877.0:23922 2014-08-11 16:26:48.844292 2014-08-13 18:06:06.362057 7f5c973a57a0 20 read_log 192888'375 (0'0) modify 8bbc9eeb/benchmark_data_storage0111-ib.pocstor_11336_object23984/head//1 by client.1312877.0:23985 2014-08-11 16:26:49.142340 2014-08-13 18:06:06.362078 7f5c973a57a0 20 read_log 192888'376 (0'0) modify e7d972b/benchmark_data_storage0111-ib.pocstor_11336_object24002/head//1 by client.1312877.0:24003 2014-08-11 16:26:49.206748 2014-08-13 18:06:06.362099 7f5c973a57a0 20 read_log 192888'377 (0'0) modify a7b66c6b/benchmark_data_storage0111-ib.pocstor_11336_object24011/head//1 by client.1312877.0:24012 2014-08-11 16:26:49.238714 2014-08-13 18:06:06.362120 7f5c973a57a0 20 read_log 192888'378 (0'0) modify 8d09746b/benchmark_data_storage0111-ib.pocstor_11336_object24032/head//1 by client.1312877.0:24033 2014-08-11 16:26:49.328222 2014-08-13 18:06:06.362142 7f5c973a57a0 20 read_log 192888'379 (0'0) modify f96e4feb/benchmark_data_storage0111-ib.pocstor_11336_object24132/head//1 by client.1312877.0:24133 2014-08-11 16:26:49.772456 2014-08-13 18:06:06.362175 7f5c973a57a0 20 read_log 192888'380 (0'0) modify 12d73bab/benchmark_data_storage0111-ib.pocstor_11336_object24218/head//1 by client.1312877.0:24219 2014-08-11 16:26:50.086561 2014-08-13 18:06:06.362197 7f5c973a57a0 20 read_log 192888'381 (0'0) modify 9866d8eb/benchmark_data_storage0111-ib.pocstor_11336_object24269/head//1 by client.1312877.0:24270 2014-08-11 16:26:50.351521 2014-08-13 18:06:06.362218 7f5c973a57a0 20 read_log 192888'382 (0'0) modify de90c7eb/benchmark_data_storage0111-ib.pocstor_11336_object24341/head//1 by client.1312877.0:24342 2014-08-11 16:26:50.788055 2014-08-13 18:06:06.362242 7f5c973a57a0 20 read_log 192888'383 (0'0) modify 537521eb/benchmark_data_storage0111-ib.pocstor_11336_object24410/head//1 by client.1312877.0:24411 2014-08-11 16:26:51.184347 2014-08-13 18:06:06.362305 7f5c973a57a0 20 read_log 192888'384 (0'0) modify e6d1e6ab/benchmark_data_storage0111-ib.pocstor_11336_object24502/head//1 by client.1312877.0:24503 2014-08-11 16:26:51.807228 2014-08-13 18:06:06.362334 7f5c973a57a0 20 read_log 192888'385 (0'0) modify 570d676b/benchmark_data_storage0111-ib.pocstor_11336_object24513/head//1 by client.1312877.0:24514 2014-08-11 16:26:51.846662 2014-08-13 18:06:06.362361 7f5c973a57a0 20 read_log 192888'386 (0'0) modify 4d3698ab/benchmark_data_storage0111-ib.pocstor_11336_object24640/head//1 by client.1312877.0:24641 2014-08-11 16:26:52.400721 2014-08-13 18:06:06.362387 7f5c973a57a0 20 read_log 192888'387 (0'0) modify 6fd4ceb/benchmark_data_storage0111-ib.pocstor_11336_object24678/head//1 by client.1312877.0:24679 2014-08-11 16:26:52.584560 2014-08-13 18:06:06.362414 7f5c973a57a0 20 read_log 192888'388 (0'0) modify dda641eb/benchmark_data_storage0111-ib.pocstor_11336_object24696/head//1 by client.1312877.0:24697 2014-08-11 16:26:52.678130 2014-08-13 18:06:06.362440 7f5c973a57a0 20 read_log 192888'389 (0'0) modify 3edb27eb/benchmark_data_storage0111-ib.pocstor_11336_object24704/head//1 by client.1312877.0:24705 2014-08-11 16:26:52.720373 2014-08-13 18:06:06.362462 7f5c973a57a0 20 read_log 192888'390 (0'0) modify c80d832b/benchmark_data_storage0111-ib.pocstor_11336_object24729/head//1 by client.1312877.0:24730 2014-08-11 16:26:52.821854 2014-08-13 18:06:06.362506 7f5c973a57a0 20 read_log 192888'391 (0'0) modify 7fdffdeb/benchmark_data_storage0111-ib.pocstor_11336_object24731/head//1 by client.1312877.0:24732 2014-08-11 16:26:52.829238 2014-08-13 18:06:06.362542 7f5c973a57a0 20 read_log 192888'392 (0'0) modify fbb74cab/benchmark_data_storage0111-ib.pocstor_11336_object24889/head//1 by client.1312877.0:24890 2014-08-11 16:26:53.602759 2014-08-13 18:06:06.362569 7f5c973a57a0 20 read_log 192888'393 (0'0) modify abc784eb/benchmark_data_storage0111-ib.pocstor_11336_object24918/head//1 by client.1312877.0:24919 2014-08-11 16:26:53.755724 2014-08-13 18:06:06.362630 7f5c973a57a0 20 read_log 192888'394 (0'0) modify aa96b5eb/benchmark_data_storage0111-ib.pocstor_11336_object25001/head//1 by client.1312877.0:25002 2014-08-11 16:26:54.064487 2014-08-13 18:06:06.362652 7f5c973a57a0 20 read_log 192888'395 (0'0) modify f622b72b/benchmark_data_storage0111-ib.pocstor_11336_object25080/head//1 by client.1312877.0:25081 2014-08-11 16:26:54.352433 2014-08-13 18:06:06.362674 7f5c973a57a0 20 read_log 192888'396 (0'0) modify 42235aeb/benchmark_data_storage0111-ib.pocstor_11336_object25120/head//1 by client.1312877.0:25121 2014-08-11 16:26:54.493495 2014-08-13 18:06:06.362695 7f5c973a57a0 20 read_log 192888'397 (0'0) modify 965b2cab/benchmark_data_storage0111-ib.pocstor_11336_object25176/head//1 by client.1312877.0:25177 2014-08-11 16:26:54.691821 2014-08-13 18:06:06.362717 7f5c973a57a0 20 read_log 192888'398 (0'0) modify 802de92b/benchmark_data_storage0111-ib.pocstor_11336_object25189/head//1 by client.1312877.0:25190 2014-08-11 16:26:54.737328 2014-08-13 18:06:06.362738 7f5c973a57a0 20 read_log 192888'399 (0'0) modify 9409426b/benchmark_data_storage0111-ib.pocstor_11336_object25191/head//1 by client.1312877.0:25192 2014-08-11 16:26:54.744548 2014-08-13 18:06:06.362760 7f5c973a57a0 20 read_log 192888'400 (0'0) modify c26e8ceb/benchmark_data_storage0111-ib.pocstor_11336_object25320/head//1 by client.1312877.0:25321 2014-08-11 16:26:55.410502 2014-08-13 18:06:06.362782 7f5c973a57a0 20 read_log 192888'401 (0'0) modify 5bd198eb/benchmark_data_storage0111-ib.pocstor_11336_object25342/head//1 by client.1312877.0:25343 2014-08-11 16:26:55.488592 2014-08-13 18:06:06.362803 7f5c973a57a0 20 read_log 192888'402 (0'0) modify f2bae72b/benchmark_data_storage0111-ib.pocstor_11336_object25411/head//1 by client.1312877.0:25412 2014-08-11 16:26:55.732698 2014-08-13 18:06:06.362825 7f5c973a57a0 20 read_log 192888'403 (0'0) modify 22b339ab/benchmark_data_storage0111-ib.pocstor_11336_object25485/head//1 by client.1312877.0:25486 2014-08-11 16:26:55.993717 2014-08-13 18:06:06.362859 7f5c973a57a0 20 read_log 192888'404 (0'0) modify 4e638a6b/benchmark_data_storage0111-ib.pocstor_11336_object25526/head//1 by client.1312877.0:25527 2014-08-11 16:26:56.202836 2014-08-13 18:06:06.362882 7f5c973a57a0 20 read_log 192888'405 (0'0) modify 1129b96b/benchmark_data_storage0111-ib.pocstor_11336_object25572/head//1 by client.1312877.0:25573 2014-08-11 16:26:56.395695 2014-08-13 18:06:06.362904 7f5c973a57a0 20 read_log 192888'406 (0'0) modify 934612eb/benchmark_data_storage0111-ib.pocstor_11336_object25811/head//1 by client.1312877.0:25812 2014-08-11 16:26:57.761164 2014-08-13 18:06:06.362925 7f5c973a57a0 20 read_log 192888'407 (0'0) modify 8a0e956b/benchmark_data_storage0111-ib.pocstor_11336_object25812/head//1 by client.1312877.0:25813 2014-08-11 16:26:57.764724 2014-08-13 18:06:06.362946 7f5c973a57a0 20 read_log 192888'408 (0'0) modify 5275656b/benchmark_data_storage0111-ib.pocstor_11336_object25868/head//1 by client.1312877.0:25869 2014-08-11 16:26:57.962180 2014-08-13 18:06:06.362971 7f5c973a57a0 20 read_log 192888'409 (0'0) modify 75b001eb/benchmark_data_storage0111-ib.pocstor_11336_object26161/head//1 by client.1312877.0:26162 2014-08-11 16:26:59.376527 2014-08-13 18:06:06.362992 7f5c973a57a0 20 read_log 192888'410 (0'0) modify b69ac26b/benchmark_data_storage0111-ib.pocstor_11336_object26351/head//1 by client.1312877.0:26352 2014-08-11 16:27:00.037628 2014-08-13 18:06:06.363014 7f5c973a57a0 20 read_log 192888'411 (0'0) modify 784d40ab/benchmark_data_storage0111-ib.pocstor_11336_object26478/head//1 by client.1312877.0:26479 2014-08-11 16:27:00.596340 2014-08-13 18:06:06.363082 7f5c973a57a0 20 read_log 192888'412 (0'0) modify b4b81d2b/benchmark_data_storage0111-ib.pocstor_11336_object26538/head//1 by client.1312877.0:26539 2014-08-11 16:27:00.805272 2014-08-13 18:06:06.363158 7f5c973a57a0 20 read_log 192888'413 (0'0) modify 940ab0ab/benchmark_data_storage0111-ib.pocstor_11336_object26596/head//1 by client.1312877.0:26597 2014-08-11 16:27:01.010879 2014-08-13 18:06:06.363226 7f5c973a57a0 20 read_log 192888'414 (0'0) modify f7f71f6b/benchmark_data_storage0111-ib.pocstor_11336_object26678/head//1 by client.1312877.0:26679 2014-08-11 16:27:01.300260 2014-08-13 18:06:06.363428 7f5c973a57a0 20 read_log 192888'415 (0'0) modify 2b79936b/benchmark_data_storage0111-ib.pocstor_11336_object26822/head//1 by client.1312877.0:26823 2014-08-11 16:27:01.800044 2014-08-13 18:06:06.363497 7f5c973a57a0 20 read_log 192888'416 (0'0) modify 9d858b2b/benchmark_data_storage0111-ib.pocstor_11336_object26873/head//1 by client.1312877.0:26874 2014-08-11 16:27:01.977353 2014-08-13 18:06:06.363562 7f5c973a57a0 20 read_log 192888'417 (0'0) modify d46b0eab/benchmark_data_storage0111-ib.pocstor_11336_object26900/head//1 by client.1312877.0:26901 2014-08-11 16:27:02.070385 2014-08-13 18:06:06.363593 7f5c973a57a0 20 read_log 192888'418 (0'0) modify 3596656b/benchmark_data_storage0111-ib.pocstor_11336_object27149/head//1 by client.1312877.0:27150 2014-08-11 16:27:03.049271 2014-08-13 18:06:06.363628 7f5c973a57a0 20 read_log 192888'419 (0'0) modify 800c4c2b/benchmark_data_storage0111-ib.pocstor_11336_object27214/head//1 by client.1312877.0:27215 2014-08-11 16:27:03.570119 2014-08-13 18:06:06.363670 7f5c973a57a0 20 read_log 192888'420 (192888'1) delete 968c7c2b/benchmark_data_storage0111-ib.pocstor_11336_object26/head//1 by client.1312877.0:27310 2014-08-11 16:27:05.039769 2014-08-13 18:06:06.363728 7f5c973a57a0 20 read_log 192888'421 (192888'2) delete 458d82b/benchmark_data_storage0111-ib.pocstor_11336_object85/head//1 by client.1312877.0:27369 2014-08-11 16:27:05.065866 2014-08-13 18:06:06.363757 7f5c973a57a0 20 read_log 192888'422 (192888'3) delete 8d3a1cab/benchmark_data_storage0111-ib.pocstor_11336_object115/head//1 by client.1312877.0:27399 2014-08-11 16:27:05.115545 2014-08-13 18:06:06.363780 7f5c973a57a0 20 read_log 192888'423 (192888'4) delete 174a7a6b/benchmark_data_storage0111-ib.pocstor_11336_object120/head//1 by client.1312877.0:27404 2014-08-11 16:27:05.130440 2014-08-13 18:06:06.363853 7f5c973a57a0 20 read_log 192888'424 (192888'5) delete 709b852b/benchmark_data_storage0111-ib.pocstor_11336_object514/head//1 by client.1312877.0:27798 2014-08-11 16:27:05.352177 2014-08-13 18:06:06.363899 7f5c973a57a0 20 read_log 192888'425 (192888'6) delete f9c4ff6b/benchmark_data_storage0111-ib.pocstor_11336_object529/head//1 by client.1312877.0:27813 2014-08-11 16:27:05.368026 2014-08-13 18:06:06.363961 7f5c973a57a0 20 read_log 192888'426 (192888'7) delete 5c599e6b/benchmark_data_storage0111-ib.pocstor_11336_object584/head//1 by client.1312877.0:27868 2014-08-11 16:27:05.457356 2014-08-13 18:06:06.363989 7f5c973a57a0 20 read_log 192888'427 (192888'8) delete cc5ad2ab/benchmark_data_storage0111-ib.pocstor_11336_object589/head//1 by client.1312877.0:27873 2014-08-11 16:27:05.460574 2014-08-13 18:06:06.364017 7f5c973a57a0 20 read_log 192888'428 (192888'9) delete e2cda9ab/benchmark_data_storage0111-ib.pocstor_11336_object680/head//1 by client.1312877.0:27964 2014-08-11 16:27:05.529342 2014-08-13 18:06:06.364066 7f5c973a57a0 20 read_log 192888'429 (192888'10) delete 6a2d76b/benchmark_data_storage0111-ib.pocstor_11336_object727/head//1 by client.1312877.0:28011 2014-08-11 16:27:05.584039 2014-08-13 18:06:06.364089 7f5c973a57a0 20 read_log 192888'430 (192888'11) delete d4542b6b/benchmark_data_storage0111-ib.pocstor_11336_object785/head//1 by client.1312877.0:28069 2014-08-11 16:27:05.677339 2014-08-13 18:06:06.364113 7f5c973a57a0 20 read_log 192888'431 (192888'12) delete 13924b6b/benchmark_data_storage0111-ib.pocstor_11336_object893/head//1 by client.1312877.0:28177 2014-08-11 16:27:05.734545 2014-08-13 18:06:06.364163 7f5c973a57a0 20 read_log 192888'432 (192888'13) delete 2df8bfab/benchmark_data_storage0111-ib.pocstor_11336_object920/head//1 by client.1312877.0:28204 2014-08-11 16:27:05.750387 2014-08-13 18:06:06.364201 7f5c973a57a0 20 read_log 192888'433 (192888'14) delete 1142b5eb/benchmark_data_storage0111-ib.pocstor_11336_object942/head//1 by client.1312877.0:28226 2014-08-11 16:27:05.771329 2014-08-13 18:06:06.364244 7f5c973a57a0 20 read_log 192888'434 (192888'15) delete afd4156b/benchmark_data_storage0111-ib.pocstor_11336_object1007/head//1 by client.1312877.0:28291 2014-08-11 16:27:06.082181 2014-08-13 18:06:06.364273 7f5c973a57a0 20 read_log 192888'435 (192888'16) delete 1bd77ceb/benchmark_data_storage0111-ib.pocstor_11336_object1053/head//1 by client.1312877.0:28337 2014-08-11 16:27:06.103443 2014-08-13 18:06:06.364296 7f5c973a57a0 20 read_log 192888'436 (192888'17) delete 4c3c3eab/benchmark_data_storage0111-ib.pocstor_11336_object1107/head//1 by client.1312877.0:28391 2014-08-11 16:27:06.129095 2014-08-13 18:06:06.364338 7f5c973a57a0 20 read_log 192888'437 (192888'18) delete 84e0c2b/benchmark_data_storage0111-ib.pocstor_11336_object1197/head//1 by client.1312877.0:28481 2014-08-11 16:27:06.205483 2014-08-13 18:06:06.364363 7f5c973a57a0 20 read_log 192888'438 (192888'19) delete 5acc686b/benchmark_data_storage0111-ib.pocstor_11336_object1402/head//1 by client.1312877.0:28686 2014-08-11 16:27:06.372340 2014-08-13 18:06:06.364389 7f5c973a57a0 20 read_log 192888'439 (192888'20) delete cd6caab/benchmark_data_storage0111-ib.pocstor_11336_object1404/head//1 by client.1312877.0:28688 2014-08-11 16:27:06.374471 2014-08-13 18:06:06.364412 7f5c973a57a0 20 read_log 192888'440 (192888'21) delete f337eceb/benchmark_data_storage0111-ib.pocstor_11336_object1448/head//1 by client.1312877.0:28732 2014-08-11 16:27:06.390419 2014-08-13 18:06:06.364434 7f5c973a57a0 20 read_log 192888'441 (192888'22) delete fdf8f3eb/benchmark_data_storage0111-ib.pocstor_11336_object1668/head//1 by client.1312877.0:28952 2014-08-11 16:27:06.466848 2014-08-13 18:06:06.364455 7f5c973a57a0 20 read_log 192888'442 (192888'23) delete 784e352b/benchmark_data_storage0111-ib.pocstor_11336_object1682/head//1 by client.1312877.0:28966 2014-08-11 16:27:06.472175 2014-08-13 18:06:06.364477 7f5c973a57a0 20 read_log 192888'443 (192888'24) delete dda332b/benchmark_data_storage0111-ib.pocstor_11336_object1930/head//1 by client.1312877.0:29214 2014-08-11 16:27:06.539051 2014-08-13 18:06:06.364512 7f5c973a57a0 20 read_log 192888'444 (192888'25) delete 93f31b2b/benchmark_data_storage0111-ib.pocstor_11336_object2038/head//1 by client.1312877.0:29322 2014-08-11 16:27:06.564439 2014-08-13 18:06:06.364539 7f5c973a57a0 20 read_log 192888'445 (192888'26) delete 357eddeb/benchmark_data_storage0111-ib.pocstor_11336_object2073/head//1 by client.1312877.0:29357 2014-08-11 16:27:06.574050 2014-08-13 18:06:06.364561 7f5c973a57a0 20 read_log 192888'446 (192888'27) delete 27ad09eb/benchmark_data_storage0111-ib.pocstor_11336_object2157/head//1 by client.1312877.0:29441 2014-08-11 16:27:06.608517 2014-08-13 18:06:06.364585 7f5c973a57a0 20 read_log 192888'447 (192888'28) delete 78fc9beb/benchmark_data_storage0111-ib.pocstor_11336_object2224/head//1 by client.1312877.0:29508 2014-08-11 16:27:06.671425 2014-08-13 18:06:06.364607 7f5c973a57a0 20 read_log 192888'448 (192888'29) delete 4721d42b/benchmark_data_storage0111-ib.pocstor_11336_object2247/head//1 by client.1312877.0:29531 2014-08-11 16:27:06.679764 2014-08-13 18:06:06.364629 7f5c973a57a0 20 read_log 192888'449 (192888'30) delete d935726b/benchmark_data_storage0111-ib.pocstor_11336_object2299/head//1 by client.1312877.0:29583 2014-08-11 16:27:06.701980 2014-08-13 18:06:06.364650 7f5c973a57a0 20 read_log 192888'450 (192888'31) delete 28d2c6ab/benchmark_data_storage0111-ib.pocstor_11336_object2510/head//1 by client.1312877.0:29794 2014-08-11 16:27:06.966418 2014-08-13 18:06:06.364672 7f5c973a57a0 20 read_log 192888'451 (192888'32) delete a9ff716b/benchmark_data_storage0111-ib.pocstor_11336_object2683/head//1 by client.1312877.0:29967 2014-08-11 16:27:07.010376 2014-08-13 18:06:06.364694 7f5c973a57a0 20 read_log 192888'452 (192888'33) delete 6eac06ab/benchmark_data_storage0111-ib.pocstor_11336_object2861/head//1 by client.1312877.0:30145 2014-08-11 16:27:07.127585 2014-08-13 18:06:06.364715 7f5c973a57a0 20 read_log 192888'453 (192888'34) delete e39450ab/benchmark_data_storage0111-ib.pocstor_11336_object2944/head//1 by client.1312877.0:30228 2014-08-11 16:27:07.148317 2014-08-13 18:06:06.364737 7f5c973a57a0 20 read_log 192888'454 (192888'35) delete 372d6ab/benchmark_data_storage0111-ib.pocstor_11336_object2981/head//1 by client.1312877.0:30265 2014-08-11 16:27:07.157884 2014-08-13 18:06:06.364758 7f5c973a57a0 20 read_log 192888'455 (192888'36) delete aaad6a2b/benchmark_data_storage0111-ib.pocstor_11336_object2989/head//1 by client.1312877.0:30273 2014-08-11 16:27:07.159726 2014-08-13 18:06:06.364780 7f5c973a57a0 20 read_log 192888'456 (192888'37) delete dbd623ab/benchmark_data_storage0111-ib.pocstor_11336_object3227/head//1 by client.1312877.0:30511 2014-08-11 16:27:07.220869 2014-08-13 18:06:06.364801 7f5c973a57a0 20 read_log 192888'457 (192888'38) delete f13350eb/benchmark_data_storage0111-ib.pocstor_11336_object3246/head//1 by client.1312877.0:30530 2014-08-11 16:27:07.229216 2014-08-13 18:06:06.364832 7f5c973a57a0 20 read_log 192888'458 (192888'39) delete 752770eb/benchmark_data_storage0111-ib.pocstor_11336_object3306/head//1 by client.1312877.0:30590 2014-08-11 16:27:07.257440 2014-08-13 18:06:06.364855 7f5c973a57a0 20 read_log 192888'459 (192888'40) delete dd84e9ab/benchmark_data_storage0111-ib.pocstor_11336_object3341/head//1 by client.1312877.0:30625 2014-08-11 16:27:07.279826 2014-08-13 18:06:06.364877 7f5c973a57a0 20 read_log 192888'460 (192888'41) delete de0a9deb/benchmark_data_storage0111-ib.pocstor_11336_object3370/head//1 by client.1312877.0:30654 2014-08-11 16:27:07.330510 2014-08-13 18:06:06.364899 7f5c973a57a0 20 read_log 192888'461 (192888'42) delete 7d7620ab/benchmark_data_storage0111-ib.pocstor_11336_object3373/head//1 by client.1312877.0:30657 2014-08-11 16:27:07.337355 2014-08-13 18:06:06.364921 7f5c973a57a0 20 read_log 192888'462 (192888'43) delete 83a675ab/benchmark_data_storage0111-ib.pocstor_11336_object3566/head//1 by client.1312877.0:30850 2014-08-11 16:27:07.796474 2014-08-13 18:06:06.364942 7f5c973a57a0 20 read_log 192888'463 (192888'44) delete 8ef2e3eb/benchmark_data_storage0111-ib.pocstor_11336_object3609/head//1 by client.1312877.0:30893 2014-08-11 16:27:07.818724 2014-08-13 18:06:06.364976 7f5c973a57a0 20 read_log 192888'464 (192888'45) delete c40043eb/benchmark_data_storage0111-ib.pocstor_11336_object3630/head//1 by client.1312877.0:30914 2014-08-11 16:27:07.826470 2014-08-13 18:06:06.365003 7f5c973a57a0 20 read_log 192888'465 (192888'46) delete 83bd3aeb/benchmark_data_storage0111-ib.pocstor_11336_object3634/head//1 by client.1312877.0:30918 2014-08-11 16:27:07.827428 2014-08-13 18:06:06.365025 7f5c973a57a0 20 read_log 192888'466 (192888'47) delete a7339fab/benchmark_data_storage0111-ib.pocstor_11336_object3636/head//1 by client.1312877.0:30920 2014-08-11 16:27:07.828803 2014-08-13 18:06:06.365051 7f5c973a57a0 20 read_log 192888'467 (192888'48) delete b51ef32b/benchmark_data_storage0111-ib.pocstor_11336_object3841/head//1 by client.1312877.0:31125 2014-08-11 16:27:07.888104 2014-08-13 18:06:06.365077 7f5c973a57a0 20 read_log 192888'468 (192888'49) delete e3142eab/benchmark_data_storage0111-ib.pocstor_11336_object4049/head//1 by client.1312877.0:31333 2014-08-11 16:27:07.963841 2014-08-13 18:06:06.365102 7f5c973a57a0 20 read_log 192888'469 (192888'50) delete 2e39aeb/benchmark_data_storage0111-ib.pocstor_11336_object4076/head//1 by client.1312877.0:31360 2014-08-11 16:27:07.990807 2014-08-13 18:06:06.365136 7f5c973a57a0 20 read_log 192888'470 (192888'51) delete 52e94fab/benchmark_data_storage0111-ib.pocstor_11336_object4179/head//1 by client.1312877.0:31463 2014-08-11 16:27:08.082818 2014-08-13 18:06:06.365167 7f5c973a57a0 20 read_log 192888'471 (192888'52) delete a4d05a6b/benchmark_data_storage0111-ib.pocstor_11336_object4217/head//1 by client.1312877.0:31501 2014-08-11 16:27:08.213040 2014-08-13 18:06:06.365195 7f5c973a57a0 20 read_log 192888'472 (192888'53) delete 8e026b6b/benchmark_data_storage0111-ib.pocstor_11336_object4269/head//1 by client.1312877.0:31553 2014-08-11 16:27:08.230094 2014-08-13 18:06:06.365218 7f5c973a57a0 20 read_log 192888'473 (192888'54) delete 45e77deb/benchmark_data_storage0111-ib.pocstor_11336_object4277/head//1 by client.1312877.0:31561 2014-08-11 16:27:08.232812 2014-08-13 18:06:06.365239 7f5c973a57a0 20 read_log 192888'474 (192888'55) delete 2779eb/benchmark_data_storage0111-ib.pocstor_11336_object4350/head//1 by client.1312877.0:31634 2014-08-11 16:27:08.253031 2014-08-13 18:06:06.365261 7f5c973a57a0 20 read_log 192888'475 (192888'56) delete 200acdeb/benchmark_data_storage0111-ib.pocstor_11336_object4425/head//1 by client.1312877.0:31709 2014-08-11 16:27:08.276387 2014-08-13 18:06:06.365283 7f5c973a57a0 20 read_log 192888'476 (192888'57) delete 4967bfeb/benchmark_data_storage0111-ib.pocstor_11336_object4506/head//1 by client.1312877.0:31790 2014-08-11 16:27:08.296640 2014-08-13 18:06:06.365305 7f5c973a57a0 20 read_log 192888'477 (192888'58) delete 13ab5f6b/benchmark_data_storage0111-ib.pocstor_11336_object4629/head//1 by client.1312877.0:31913 2014-08-11 16:27:08.347008 2014-08-13 18:06:06.365334 7f5c973a57a0 20 read_log 192888'478 (192888'59) delete e9c088eb/benchmark_data_storage0111-ib.pocstor_11336_object4640/head//1 by client.1312877.0:31924 2014-08-11 16:27:08.352284 2014-08-13 18:06:06.365361 7f5c973a57a0 20 read_log 192888'479 (192888'60) delete 84ba19eb/benchmark_data_storage0111-ib.pocstor_11336_object4722/head//1 by client.1312877.0:32006 2014-08-11 16:27:08.386537 2014-08-13 18:06:06.365389 7f5c973a57a0 20 read_log 192888'480 (192888'61) delete 64f0a7ab/benchmark_data_storage0111-ib.pocstor_11336_object4767/head//1 by client.1312877.0:32051 2014-08-11 16:27:08.397934 2014-08-13 18:06:06.365414 7f5c973a57a0 20 read_log 192888'481 (192888'62) delete 18830eeb/benchmark_data_storage0111-ib.pocstor_11336_object4790/head//1 by client.1312877.0:32074 2014-08-11 16:27:08.403833 2014-08-13 18:06:06.365442 7f5c973a57a0 20 read_log 192888'482 (192888'63) delete 80cba86b/benchmark_data_storage0111-ib.pocstor_11336_object4818/head//1 by client.1312877.0:32102 2014-08-11 16:27:08.409580 2014-08-13 18:06:06.365469 7f5c973a57a0 20 read_log 192888'483 (192888'64) delete 7b42aa6b/benchmark_data_storage0111-ib.pocstor_11336_object4861/head//1 by client.1312877.0:32145 2014-08-11 16:27:08.419858 2014-08-13 18:06:06.365532 7f5c973a57a0 20 read_log 192888'484 (192888'65) delete cb176c6b/benchmark_data_storage0111-ib.pocstor_11336_object4943/head//1 by client.1312877.0:32227 2014-08-11 16:27:08.444235 2014-08-13 18:06:06.365577 7f5c973a57a0 20 read_log 192888'485 (192888'66) delete 34a403eb/benchmark_data_storage0111-ib.pocstor_11336_object5103/head//1 by client.1312877.0:32387 2014-08-11 16:27:08.525121 2014-08-13 18:06:06.365604 7f5c973a57a0 20 read_log 192888'486 (192888'67) delete 43cf3beb/benchmark_data_storage0111-ib.pocstor_11336_object5115/head//1 by client.1312877.0:32399 2014-08-11 16:27:08.529820 2014-08-13 18:06:06.365633 7f5c973a57a0 20 read_log 192888'487 (192888'68) delete ef97dfab/benchmark_data_storage0111-ib.pocstor_11336_object5267/head//1 by client.1312877.0:32551 2014-08-11 16:27:08.608980 2014-08-13 18:06:06.365656 7f5c973a57a0 20 read_log 192888'488 (192888'69) delete 2a7643eb/benchmark_data_storage0111-ib.pocstor_11336_object5291/head//1 by client.1312877.0:32575 2014-08-11 16:27:08.615061 2014-08-13 18:06:06.365688 7f5c973a57a0 20 read_log 192888'489 (192888'70) delete c91d172b/benchmark_data_storage0111-ib.pocstor_11336_object5349/head//1 by client.1312877.0:32633 2014-08-11 16:27:08.629343 2014-08-13 18:06:06.365728 7f5c973a57a0 20 read_log 192888'490 (192888'71) delete 86f5e06b/benchmark_data_storage0111-ib.pocstor_11336_object5415/head//1 by client.1312877.0:32699 2014-08-11 16:27:08.644360 2014-08-13 18:06:06.365751 7f5c973a57a0 20 read_log 192888'491 (192888'72) delete 199e7beb/benchmark_data_storage0111-ib.pocstor_11336_object5466/head//1 by client.1312877.0:32750 2014-08-11 16:27:08.656118 2014-08-13 18:06:06.365772 7f5c973a57a0 20 read_log 192888'492 (192888'73) delete 7c15dbab/benchmark_data_storage0111-ib.pocstor_11336_object5510/head//1 by client.1312877.0:32794 2014-08-11 16:27:08.666479 2014-08-13 18:06:06.365800 7f5c973a57a0 20 read_log 192888'493 (192888'74) delete 5bad7f6b/benchmark_data_storage0111-ib.pocstor_11336_object5611/head//1 by client.1312877.0:32895 2014-08-11 16:27:08.688927 2014-08-13 18:06:06.365826 7f5c973a57a0 20 read_log 192888'494 (192888'75) delete b65b286b/benchmark_data_storage0111-ib.pocstor_11336_object5618/head//1 by client.1312877.0:32902 2014-08-11 16:27:08.690725 2014-08-13 18:06:06.365848 7f5c973a57a0 20 read_log 192888'495 (192888'76) delete 51e9e32b/benchmark_data_storage0111-ib.pocstor_11336_object5649/head//1 by client.1312877.0:32933 2014-08-11 16:27:08.697349 2014-08-13 18:06:06.365870 7f5c973a57a0 20 read_log 192888'496 (192888'77) delete be6f1b2b/benchmark_data_storage0111-ib.pocstor_11336_object5663/head//1 by client.1312877.0:32947 2014-08-11 16:27:08.701244 2014-08-13 18:06:06.365900 7f5c973a57a0 20 read_log 192888'497 (192888'78) delete ef81c46b/benchmark_data_storage0111-ib.pocstor_11336_object5705/head//1 by client.1312877.0:32989 2014-08-11 16:27:08.711661 2014-08-13 18:06:06.365922 7f5c973a57a0 20 read_log 192888'498 (192888'79) delete 7460ea2b/benchmark_data_storage0111-ib.pocstor_11336_object5758/head//1 by client.1312877.0:33042 2014-08-11 16:27:08.724624 2014-08-13 18:06:06.365944 7f5c973a57a0 20 read_log 192888'499 (192888'80) delete d8711a2b/benchmark_data_storage0111-ib.pocstor_11336_object5802/head//1 by client.1312877.0:33086 2014-08-11 16:27:08.738423 2014-08-13 18:06:06.365966 7f5c973a57a0 20 read_log 192888'500 (192888'81) delete 7cea612b/benchmark_data_storage0111-ib.pocstor_11336_object5943/head//1 by client.1312877.0:33227 2014-08-11 16:27:08.791516 2014-08-13 18:06:06.365992 7f5c973a57a0 20 read_log 192888'501 (192888'82) delete b862b52b/benchmark_data_storage0111-ib.pocstor_11336_object5996/head//1 by client.1312877.0:33280 2014-08-11 16:27:08.821243 2014-08-13 18:06:06.366014 7f5c973a57a0 20 read_log 192888'502 (192888'83) delete aa0fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object6043/head//1 by client.1312877.0:33327 2014-08-11 16:27:08.832412 2014-08-13 18:06:06.366036 7f5c973a57a0 20 read_log 192888'503 (192888'84) delete 1f2e13eb/benchmark_data_storage0111-ib.pocstor_11336_object6114/head//1 by client.1312877.0:33398 2014-08-11 16:27:08.850424 2014-08-13 18:06:06.366069 7f5c973a57a0 20 read_log 192888'504 (192888'85) delete 26185aeb/benchmark_data_storage0111-ib.pocstor_11336_object6115/head//1 by client.1312877.0:33399 2014-08-11 16:27:08.850627 2014-08-13 18:06:06.366096 7f5c973a57a0 20 read_log 192888'505 (192888'86) delete bc9f81ab/benchmark_data_storage0111-ib.pocstor_11336_object6256/head//1 by client.1312877.0:33540 2014-08-11 16:27:08.888229 2014-08-13 18:06:06.366118 7f5c973a57a0 20 read_log 192888'506 (192888'87) delete 665d8cab/benchmark_data_storage0111-ib.pocstor_11336_object6345/head//1 by client.1312877.0:33629 2014-08-11 16:27:08.909014 2014-08-13 18:06:06.366140 7f5c973a57a0 20 read_log 192888'507 (192888'88) delete 2797a2b/benchmark_data_storage0111-ib.pocstor_11336_object6356/head//1 by client.1312877.0:33640 2014-08-11 16:27:08.912070 2014-08-13 18:06:06.366177 7f5c973a57a0 20 read_log 192888'508 (192888'89) delete dd86336b/benchmark_data_storage0111-ib.pocstor_11336_object6360/head//1 by client.1312877.0:33644 2014-08-11 16:27:08.912470 2014-08-13 18:06:06.366200 7f5c973a57a0 20 read_log 192888'509 (192888'90) delete 81e68deb/benchmark_data_storage0111-ib.pocstor_11336_object6546/head//1 by client.1312877.0:33830 2014-08-11 16:27:08.965431 2014-08-13 18:06:06.366221 7f5c973a57a0 20 read_log 192888'510 (192888'91) delete cacbb16b/benchmark_data_storage0111-ib.pocstor_11336_object6567/head//1 by client.1312877.0:33851 2014-08-11 16:27:08.970488 2014-08-13 18:06:06.366243 7f5c973a57a0 20 read_log 192888'511 (192888'92) delete 982d3c2b/benchmark_data_storage0111-ib.pocstor_11336_object6610/head//1 by client.1312877.0:33894 2014-08-11 16:27:08.980162 2014-08-13 18:06:06.366274 7f5c973a57a0 20 read_log 192888'512 (192888'93) delete 7c69c2b/benchmark_data_storage0111-ib.pocstor_11336_object6688/head//1 by client.1312877.0:33972 2014-08-11 16:27:08.998024 2014-08-13 18:06:06.366296 7f5c973a57a0 20 read_log 192888'513 (192888'94) delete 4db680ab/benchmark_data_storage0111-ib.pocstor_11336_object6735/head//1 by client.1312877.0:34019 2014-08-11 16:27:09.010135 2014-08-13 18:06:06.366318 7f5c973a57a0 20 read_log 192888'514 (192888'95) delete ee8926eb/benchmark_data_storage0111-ib.pocstor_11336_object6785/head//1 by client.1312877.0:34069 2014-08-11 16:27:09.022110 2014-08-13 18:06:06.366346 7f5c973a57a0 20 read_log 192888'515 (192888'96) delete 183f4fab/benchmark_data_storage0111-ib.pocstor_11336_object6867/head//1 by client.1312877.0:34151 2014-08-11 16:27:09.044992 2014-08-13 18:06:06.366367 7f5c973a57a0 20 read_log 192888'516 (192888'97) delete 77e164ab/benchmark_data_storage0111-ib.pocstor_11336_object6990/head//1 by client.1312877.0:34274 2014-08-11 16:27:09.077190 2014-08-13 18:06:06.366389 7f5c973a57a0 20 read_log 192888'517 (192888'98) delete e34b5a2b/benchmark_data_storage0111-ib.pocstor_11336_object6999/head//1 by client.1312877.0:34283 2014-08-11 16:27:09.078934 2014-08-13 18:06:06.366419 7f5c973a57a0 20 read_log 192888'518 (192888'99) delete aa9237eb/benchmark_data_storage0111-ib.pocstor_11336_object7008/head//1 by client.1312877.0:34292 2014-08-11 16:27:09.081592 2014-08-13 18:06:06.366509 7f5c973a57a0 20 read_log 192888'519 (192888'100) delete ac4a732b/benchmark_data_storage0111-ib.pocstor_11336_object7094/head//1 by client.1312877.0:34378 2014-08-11 16:27:09.105363 2014-08-13 18:06:06.366551 7f5c973a57a0 20 read_log 192888'520 (192888'101) delete c413e7ab/benchmark_data_storage0111-ib.pocstor_11336_object7152/head//1 by client.1312877.0:34436 2014-08-11 16:27:09.119085 2014-08-13 18:06:06.366580 7f5c973a57a0 20 read_log 192888'521 (192888'102) delete ef342f6b/benchmark_data_storage0111-ib.pocstor_11336_object7178/head//1 by client.1312877.0:34462 2014-08-11 16:27:09.124925 2014-08-13 18:06:06.366603 7f5c973a57a0 20 read_log 192888'522 (192888'103) delete 581f046b/benchmark_data_storage0111-ib.pocstor_11336_object7210/head//1 by client.1312877.0:34494 2014-08-11 16:27:09.135867 2014-08-13 18:06:06.366625 7f5c973a57a0 20 read_log 192888'523 (192888'104) delete 6e01b76b/benchmark_data_storage0111-ib.pocstor_11336_object7236/head//1 by client.1312877.0:34520 2014-08-11 16:27:09.141889 2014-08-13 18:06:06.366695 7f5c973a57a0 20 read_log 192888'524 (192888'105) delete d200a6eb/benchmark_data_storage0111-ib.pocstor_11336_object7281/head//1 by client.1312877.0:34565 2014-08-11 16:27:09.153200 2014-08-13 18:06:06.366740 7f5c973a57a0 20 read_log 192888'525 (192888'106) delete 84fb60eb/benchmark_data_storage0111-ib.pocstor_11336_object7370/head//1 by client.1312877.0:34654 2014-08-11 16:27:09.173412 2014-08-13 18:06:06.366763 7f5c973a57a0 20 read_log 192888'526 (192888'107) delete d6b5f52b/benchmark_data_storage0111-ib.pocstor_11336_object7447/head//1 by client.1312877.0:34731 2014-08-11 16:27:09.196895 2014-08-13 18:06:06.366785 7f5c973a57a0 20 read_log 192888'527 (192888'108) delete 8552af2b/benchmark_data_storage0111-ib.pocstor_11336_object7477/head//1 by client.1312877.0:34761 2014-08-11 16:27:09.203097 2014-08-13 18:06:06.366809 7f5c973a57a0 20 read_log 192888'528 (192888'109) delete d583a86b/benchmark_data_storage0111-ib.pocstor_11336_object7517/head//1 by client.1312877.0:34801 2014-08-11 16:27:09.214764 2014-08-13 18:06:06.366834 7f5c973a57a0 20 read_log 192888'529 (192888'110) delete 5fb5a3eb/benchmark_data_storage0111-ib.pocstor_11336_object7554/head//1 by client.1312877.0:34838 2014-08-11 16:27:09.225151 2014-08-13 18:06:06.366857 7f5c973a57a0 20 read_log 192888'530 (192888'111) delete fb03b0eb/benchmark_data_storage0111-ib.pocstor_11336_object7647/head//1 by client.1312877.0:34931 2014-08-11 16:27:09.247817 2014-08-13 18:06:06.366878 7f5c973a57a0 20 read_log 192888'531 (192888'112) delete 6b05496b/benchmark_data_storage0111-ib.pocstor_11336_object7737/head//1 by client.1312877.0:35021 2014-08-11 16:27:09.269725 2014-08-13 18:06:06.366900 7f5c973a57a0 20 read_log 192888'532 (192888'113) delete 8326d86b/benchmark_data_storage0111-ib.pocstor_11336_object7812/head//1 by client.1312877.0:35096 2014-08-11 16:27:09.290064 2014-08-13 18:06:06.366922 7f5c973a57a0 20 read_log 192888'533 (192888'114) delete e8e52e2b/benchmark_data_storage0111-ib.pocstor_11336_object7900/head//1 by client.1312877.0:35184 2014-08-11 16:27:09.316802 2014-08-13 18:06:06.366943 7f5c973a57a0 20 read_log 192888'534 (192888'115) delete e48691eb/benchmark_data_storage0111-ib.pocstor_11336_object7922/head//1 by client.1312877.0:35206 2014-08-11 16:27:09.322732 2014-08-13 18:06:06.366965 7f5c973a57a0 20 read_log 192888'535 (192888'116) delete 10dd5aeb/benchmark_data_storage0111-ib.pocstor_11336_object7942/head//1 by client.1312877.0:35226 2014-08-11 16:27:09.327566 2014-08-13 18:06:06.366986 7f5c973a57a0 20 read_log 192888'536 (192888'117) delete 1dd889ab/benchmark_data_storage0111-ib.pocstor_11336_object8027/head//1 by client.1312877.0:35311 2014-08-11 16:27:09.352224 2014-08-13 18:06:06.367008 7f5c973a57a0 20 read_log 192888'537 (192888'118) delete 8d332b2b/benchmark_data_storage0111-ib.pocstor_11336_object8120/head//1 by client.1312877.0:35404 2014-08-11 16:27:09.375595 2014-08-13 18:06:06.367030 7f5c973a57a0 20 read_log 192888'538 (192888'119) delete 1793a06b/benchmark_data_storage0111-ib.pocstor_11336_object8121/head//1 by client.1312877.0:35405 2014-08-11 16:27:09.376030 2014-08-13 18:06:06.367051 7f5c973a57a0 20 read_log 192888'539 (192888'120) delete b4faa32b/benchmark_data_storage0111-ib.pocstor_11336_object8144/head//1 by client.1312877.0:35428 2014-08-11 16:27:09.381408 2014-08-13 18:06:06.367073 7f5c973a57a0 20 read_log 192888'540 (192888'121) delete 4035182b/benchmark_data_storage0111-ib.pocstor_11336_object8213/head//1 by client.1312877.0:35497 2014-08-11 16:27:09.400123 2014-08-13 18:06:06.367094 7f5c973a57a0 20 read_log 192888'541 (192888'122) delete 461c62b/benchmark_data_storage0111-ib.pocstor_11336_object8226/head//1 by client.1312877.0:35510 2014-08-11 16:27:09.403041 2014-08-13 18:06:06.367116 7f5c973a57a0 20 read_log 192888'542 (192888'123) delete fef695ab/benchmark_data_storage0111-ib.pocstor_11336_object8282/head//1 by client.1312877.0:35566 2014-08-11 16:27:09.415444 2014-08-13 18:06:06.367137 7f5c973a57a0 20 read_log 192888'543 (192888'124) delete 133eef2b/benchmark_data_storage0111-ib.pocstor_11336_object8316/head//1 by client.1312877.0:35600 2014-08-11 16:27:09.423728 2014-08-13 18:06:06.367185 7f5c973a57a0 20 read_log 192888'544 (192888'125) delete 5905afab/benchmark_data_storage0111-ib.pocstor_11336_object8508/head//1 by client.1312877.0:35792 2014-08-11 16:27:09.472886 2014-08-13 18:06:06.367222 7f5c973a57a0 20 read_log 192888'545 (192888'126) delete 1c3d296b/benchmark_data_storage0111-ib.pocstor_11336_object8550/head//1 by client.1312877.0:35834 2014-08-11 16:27:09.483813 2014-08-13 18:06:06.367244 7f5c973a57a0 20 read_log 192888'546 (192888'127) delete 582141eb/benchmark_data_storage0111-ib.pocstor_11336_object8559/head//1 by client.1312877.0:35843 2014-08-11 16:27:09.485813 2014-08-13 18:06:06.367265 7f5c973a57a0 20 read_log 192888'547 (192888'128) delete 39758f6b/benchmark_data_storage0111-ib.pocstor_11336_object8581/head//1 by client.1312877.0:35865 2014-08-11 16:27:09.491999 2014-08-13 18:06:06.367287 7f5c973a57a0 20 read_log 192888'548 (192888'129) delete 5016fc6b/benchmark_data_storage0111-ib.pocstor_11336_object8602/head//1 by client.1312877.0:35886 2014-08-11 16:27:09.496789 2014-08-13 18:06:06.367342 7f5c973a57a0 20 read_log 192888'549 (192888'130) delete 11560a2b/benchmark_data_storage0111-ib.pocstor_11336_object8708/head//1 by client.1312877.0:35992 2014-08-11 16:27:09.522619 2014-08-13 18:06:06.367372 7f5c973a57a0 20 read_log 192888'550 (192888'131) delete 22b07aab/benchmark_data_storage0111-ib.pocstor_11336_object8772/head//1 by client.1312877.0:36056 2014-08-11 16:27:09.541103 2014-08-13 18:06:06.367437 7f5c973a57a0 20 read_log 192888'551 (192888'132) delete 7e5d59ab/benchmark_data_storage0111-ib.pocstor_11336_object8779/head//1 by client.1312877.0:36063 2014-08-11 16:27:09.542679 2014-08-13 18:06:06.367488 7f5c973a57a0 20 read_log 192888'552 (192888'133) delete ec37116b/benchmark_data_storage0111-ib.pocstor_11336_object8784/head//1 by client.1312877.0:36068 2014-08-11 16:27:09.543661 2014-08-13 18:06:06.367528 7f5c973a57a0 20 read_log 192888'553 (192888'134) delete b0de23eb/benchmark_data_storage0111-ib.pocstor_11336_object8926/head//1 by client.1312877.0:36210 2014-08-11 16:27:09.579567 2014-08-13 18:06:06.367562 7f5c973a57a0 20 read_log 192888'554 (192888'135) delete bb89eab/benchmark_data_storage0111-ib.pocstor_11336_object8969/head//1 by client.1312877.0:36253 2014-08-11 16:27:09.590497 2014-08-13 18:06:06.367592 7f5c973a57a0 20 read_log 192888'555 (192888'136) delete 1573a02b/benchmark_data_storage0111-ib.pocstor_11336_object9147/head//1 by client.1312877.0:36431 2014-08-11 16:27:09.636140 2014-08-13 18:06:06.367614 7f5c973a57a0 20 read_log 192888'556 (192888'137) delete 9b3d4deb/benchmark_data_storage0111-ib.pocstor_11336_object9217/head//1 by client.1312877.0:36501 2014-08-11 16:27:09.651926 2014-08-13 18:06:06.367636 7f5c973a57a0 20 read_log 192888'557 (192888'138) delete 7c824dab/benchmark_data_storage0111-ib.pocstor_11336_object9339/head//1 by client.1312877.0:36623 2014-08-11 16:27:09.681301 2014-08-13 18:06:06.367665 7f5c973a57a0 20 read_log 192888'558 (192888'139) delete 7ce138eb/benchmark_data_storage0111-ib.pocstor_11336_object9430/head//1 by client.1312877.0:36714 2014-08-11 16:27:09.704268 2014-08-13 18:06:06.367691 7f5c973a57a0 20 read_log 192888'559 (192888'140) delete 5dd698ab/benchmark_data_storage0111-ib.pocstor_11336_object9467/head//1 by client.1312877.0:36751 2014-08-11 16:27:09.713700 2014-08-13 18:06:06.367713 7f5c973a57a0 20 read_log 192888'560 (192888'141) delete 6fa2c96b/benchmark_data_storage0111-ib.pocstor_11336_object9547/head//1 by client.1312877.0:36831 2014-08-11 16:27:09.735681 2014-08-13 18:06:06.367734 7f5c973a57a0 20 read_log 192888'561 (192888'142) delete af93b72b/benchmark_data_storage0111-ib.pocstor_11336_object9599/head//1 by client.1312877.0:36883 2014-08-11 16:27:09.750025 2014-08-13 18:06:06.367756 7f5c973a57a0 20 read_log 192888'562 (192888'143) delete 9e00ec6b/benchmark_data_storage0111-ib.pocstor_11336_object9624/head//1 by client.1312877.0:36908 2014-08-11 16:27:09.756796 2014-08-13 18:06:06.367779 7f5c973a57a0 20 read_log 192888'563 (192888'144) delete 30d7d7ab/benchmark_data_storage0111-ib.pocstor_11336_object9654/head//1 by client.1312877.0:36938 2014-08-11 16:27:09.763927 2014-08-13 18:06:06.367814 7f5c973a57a0 20 read_log 192888'564 (192888'145) delete c30ef8eb/benchmark_data_storage0111-ib.pocstor_11336_object9675/head//1 by client.1312877.0:36959 2014-08-11 16:27:09.768622 2014-08-13 18:06:06.367840 7f5c973a57a0 20 read_log 192888'565 (192888'146) delete e5c5686b/benchmark_data_storage0111-ib.pocstor_11336_object9798/head//1 by client.1312877.0:37082 2014-08-11 16:27:09.802523 2014-08-13 18:06:06.367862 7f5c973a57a0 20 read_log 192888'566 (192888'147) delete f0c1866b/benchmark_data_storage0111-ib.pocstor_11336_object9809/head//1 by client.1312877.0:37093 2014-08-11 16:27:09.803864 2014-08-13 18:06:06.367914 7f5c973a57a0 20 read_log 192888'567 (192888'148) delete 7dfac8eb/benchmark_data_storage0111-ib.pocstor_11336_object9862/head//1 by client.1312877.0:37146 2014-08-11 16:27:09.817216 2014-08-13 18:06:06.367937 7f5c973a57a0 20 read_log 192888'568 (192888'149) delete 7385cbab/benchmark_data_storage0111-ib.pocstor_11336_object10044/head//1 by client.1312877.0:37328 2014-08-11 16:27:09.863174 2014-08-13 18:06:06.367959 7f5c973a57a0 20 read_log 192888'569 (192888'150) delete 8cef43ab/benchmark_data_storage0111-ib.pocstor_11336_object10070/head//1 by client.1312877.0:37354 2014-08-11 16:27:09.869667 2014-08-13 18:06:06.367981 7f5c973a57a0 20 read_log 192888'570 (192888'151) delete a97eac2b/benchmark_data_storage0111-ib.pocstor_11336_object10215/head//1 by client.1312877.0:37499 2014-08-11 16:27:09.907559 2014-08-13 18:06:06.368003 7f5c973a57a0 20 read_log 192888'571 (192888'152) delete 9dccadab/benchmark_data_storage0111-ib.pocstor_11336_object10293/head//1 by client.1312877.0:37577 2014-08-11 16:27:09.924944 2014-08-13 18:06:06.368024 7f5c973a57a0 20 read_log 192888'572 (192888'153) delete 403f876b/benchmark_data_storage0111-ib.pocstor_11336_object10422/head//1 by client.1312877.0:37706 2014-08-11 16:27:09.956368 2014-08-13 18:06:06.368046 7f5c973a57a0 20 read_log 192888'573 (192888'154) delete d913feab/benchmark_data_storage0111-ib.pocstor_11336_object10499/head//1 by client.1312877.0:37783 2014-08-11 16:27:09.973350 2014-08-13 18:06:06.368068 7f5c973a57a0 20 read_log 192888'574 (192888'155) delete 1067f22b/benchmark_data_storage0111-ib.pocstor_11336_object10519/head//1 by client.1312877.0:37803 2014-08-11 16:27:09.977347 2014-08-13 18:06:06.368090 7f5c973a57a0 20 read_log 192888'575 (192888'156) delete 4ca0adab/benchmark_data_storage0111-ib.pocstor_11336_object10563/head//1 by client.1312877.0:37847 2014-08-11 16:27:09.986805 2014-08-13 18:06:06.368112 7f5c973a57a0 20 read_log 192888'576 (192888'157) delete cd10626b/benchmark_data_storage0111-ib.pocstor_11336_object10629/head//1 by client.1312877.0:37913 2014-08-11 16:27:10.002592 2014-08-13 18:06:06.368133 7f5c973a57a0 20 read_log 192888'577 (192888'158) delete 2d074d6b/benchmark_data_storage0111-ib.pocstor_11336_object10684/head//1 by client.1312877.0:37968 2014-08-11 16:27:10.016038 2014-08-13 18:06:06.368165 7f5c973a57a0 20 read_log 192888'578 (192888'159) delete 7094446b/benchmark_data_storage0111-ib.pocstor_11336_object10863/head//1 by client.1312877.0:38147 2014-08-11 16:27:10.060241 2014-08-13 18:06:06.368188 7f5c973a57a0 20 read_log 192888'579 (192888'160) delete 24fc1eab/benchmark_data_storage0111-ib.pocstor_11336_object10891/head//1 by client.1312877.0:38175 2014-08-11 16:27:10.066778 2014-08-13 18:06:06.368212 7f5c973a57a0 20 read_log 192888'580 (192888'161) delete ba27f86b/benchmark_data_storage0111-ib.pocstor_11336_object10963/head//1 by client.1312877.0:38247 2014-08-11 16:27:10.083951 2014-08-13 18:06:06.368234 7f5c973a57a0 20 read_log 192888'581 (192888'162) delete 979816b/benchmark_data_storage0111-ib.pocstor_11336_object10980/head//1 by client.1312877.0:38264 2014-08-11 16:27:10.087771 2014-08-13 18:06:06.368260 7f5c973a57a0 20 read_log 192888'582 (192888'163) delete 7f2fe8eb/benchmark_data_storage0111-ib.pocstor_11336_object10990/head//1 by client.1312877.0:38274 2014-08-11 16:27:10.090581 2014-08-13 18:06:06.368287 7f5c973a57a0 20 read_log 192888'583 (192888'164) delete 53e63eb/benchmark_data_storage0111-ib.pocstor_11336_object11017/head//1 by client.1312877.0:38301 2014-08-11 16:27:10.098585 2014-08-13 18:06:06.368325 7f5c973a57a0 20 read_log 192888'584 (192888'165) delete b5c20a6b/benchmark_data_storage0111-ib.pocstor_11336_object11120/head//1 by client.1312877.0:38404 2014-08-11 16:27:10.123768 2014-08-13 18:06:06.368358 7f5c973a57a0 20 read_log 192888'585 (192888'166) delete 2c9accab/benchmark_data_storage0111-ib.pocstor_11336_object11135/head//1 by client.1312877.0:38419 2014-08-11 16:27:10.127628 2014-08-13 18:06:06.368386 7f5c973a57a0 20 read_log 192888'586 (192888'167) delete 8cd4cbab/benchmark_data_storage0111-ib.pocstor_11336_object11148/head//1 by client.1312877.0:38432 2014-08-11 16:27:10.130794 2014-08-13 18:06:06.368424 7f5c973a57a0 20 read_log 192888'587 (192888'168) delete 5a894beb/benchmark_data_storage0111-ib.pocstor_11336_object11176/head//1 by client.1312877.0:38460 2014-08-11 16:27:10.137972 2014-08-13 18:06:06.368468 7f5c973a57a0 20 read_log 192888'588 (192888'169) delete a326b22b/benchmark_data_storage0111-ib.pocstor_11336_object11234/head//1 by client.1312877.0:38518 2014-08-11 16:27:10.150589 2014-08-13 18:06:06.368505 7f5c973a57a0 20 read_log 192888'589 (192888'170) delete f27ed76b/benchmark_data_storage0111-ib.pocstor_11336_object11321/head//1 by client.1312877.0:38605 2014-08-11 16:27:10.171756 2014-08-13 18:06:06.368534 7f5c973a57a0 20 read_log 192888'590 (192888'171) delete 36effaeb/benchmark_data_storage0111-ib.pocstor_11336_object11355/head//1 by client.1312877.0:38639 2014-08-11 16:27:10.182191 2014-08-13 18:06:06.368556 7f5c973a57a0 20 read_log 192888'591 (192888'172) delete c959e92b/benchmark_data_storage0111-ib.pocstor_11336_object11363/head//1 by client.1312877.0:38647 2014-08-11 16:27:10.183439 2014-08-13 18:06:06.368580 7f5c973a57a0 20 read_log 192888'592 (192888'173) delete 8a569feb/benchmark_data_storage0111-ib.pocstor_11336_object11379/head//1 by client.1312877.0:38663 2014-08-11 16:27:10.189154 2014-08-13 18:06:06.368602 7f5c973a57a0 20 read_log 192888'593 (192888'174) delete 265609ab/benchmark_data_storage0111-ib.pocstor_11336_object11434/head//1 by client.1312877.0:38718 2014-08-11 16:27:10.201777 2014-08-13 18:06:06.368635 7f5c973a57a0 20 read_log 192888'594 (192888'175) delete 2f3985ab/benchmark_data_storage0111-ib.pocstor_11336_object11591/head//1 by client.1312877.0:38875 2014-08-11 16:27:10.243339 2014-08-13 18:06:06.368657 7f5c973a57a0 20 read_log 192888'595 (192888'176) delete 2218ab6b/benchmark_data_storage0111-ib.pocstor_11336_object11594/head//1 by client.1312877.0:38878 2014-08-11 16:27:10.244598 2014-08-13 18:06:06.368696 7f5c973a57a0 20 read_log 192888'596 (192888'177) delete 780536ab/benchmark_data_storage0111-ib.pocstor_11336_object11678/head//1 by client.1312877.0:38962 2014-08-11 16:27:10.268573 2014-08-13 18:06:06.368721 7f5c973a57a0 20 read_log 192888'597 (192888'178) delete a4d98e2b/benchmark_data_storage0111-ib.pocstor_11336_object11711/head//1 by client.1312877.0:38995 2014-08-11 16:27:10.275492 2014-08-13 18:06:06.368747 7f5c973a57a0 20 read_log 192888'598 (192888'179) delete 3ae3c7eb/benchmark_data_storage0111-ib.pocstor_11336_object11840/head//1 by client.1312877.0:39124 2014-08-11 16:27:10.308037 2014-08-13 18:06:06.368769 7f5c973a57a0 20 read_log 192888'599 (192888'180) delete 28cb2aeb/benchmark_data_storage0111-ib.pocstor_11336_object11864/head//1 by client.1312877.0:39148 2014-08-11 16:27:10.316166 2014-08-13 18:06:06.368790 7f5c973a57a0 20 read_log 192888'600 (192888'181) delete a7751f6b/benchmark_data_storage0111-ib.pocstor_11336_object11902/head//1 by client.1312877.0:39186 2014-08-11 16:27:10.324062 2014-08-13 18:06:06.368812 7f5c973a57a0 20 read_log 192888'601 (192888'182) delete 4b1d62ab/benchmark_data_storage0111-ib.pocstor_11336_object11920/head//1 by client.1312877.0:39204 2014-08-11 16:27:10.328075 2014-08-13 18:06:06.368834 7f5c973a57a0 20 read_log 192888'602 (192888'183) delete 9ef299ab/benchmark_data_storage0111-ib.pocstor_11336_object11921/head//1 by client.1312877.0:39205 2014-08-11 16:27:10.328180 2014-08-13 18:06:06.368855 7f5c973a57a0 20 read_log 192888'603 (192888'184) delete 47510d6b/benchmark_data_storage0111-ib.pocstor_11336_object12100/head//1 by client.1312877.0:39384 2014-08-11 16:27:10.373175 2014-08-13 18:06:06.368894 7f5c973a57a0 20 read_log 192888'604 (192888'185) delete 69a89f2b/benchmark_data_storage0111-ib.pocstor_11336_object12168/head//1 by client.1312877.0:39452 2014-08-11 16:27:10.390862 2014-08-13 18:06:06.368921 7f5c973a57a0 20 read_log 192888'605 (192888'186) delete 6b100eeb/benchmark_data_storage0111-ib.pocstor_11336_object12257/head//1 by client.1312877.0:39541 2014-08-11 16:27:10.412329 2014-08-13 18:06:06.368943 7f5c973a57a0 20 read_log 192888'606 (192888'187) delete cdb5142b/benchmark_data_storage0111-ib.pocstor_11336_object12280/head//1 by client.1312877.0:39564 2014-08-11 16:27:10.418654 2014-08-13 18:06:06.368965 7f5c973a57a0 20 read_log 192888'607 (192888'188) delete 50af2eeb/benchmark_data_storage0111-ib.pocstor_11336_object12322/head//1 by client.1312877.0:39606 2014-08-11 16:27:10.429707 2014-08-13 18:06:06.368988 7f5c973a57a0 20 read_log 192888'608 (192888'189) delete d5d1452b/benchmark_data_storage0111-ib.pocstor_11336_object12326/head//1 by client.1312877.0:39610 2014-08-11 16:27:10.430450 2014-08-13 18:06:06.369010 7f5c973a57a0 20 read_log 192888'609 (192888'190) delete a8750ceb/benchmark_data_storage0111-ib.pocstor_11336_object12327/head//1 by client.1312877.0:39611 2014-08-11 16:27:10.430839 2014-08-13 18:06:06.369040 7f5c973a57a0 20 read_log 192888'610 (192888'191) delete 6cd4e4eb/benchmark_data_storage0111-ib.pocstor_11336_object12360/head//1 by client.1312877.0:39644 2014-08-11 16:27:10.440934 2014-08-13 18:06:06.369063 7f5c973a57a0 20 read_log 192888'611 (192888'192) delete e3f5d66b/benchmark_data_storage0111-ib.pocstor_11336_object12391/head//1 by client.1312877.0:39675 2014-08-11 16:27:10.447294 2014-08-13 18:06:06.369114 7f5c973a57a0 20 read_log 192888'612 (192888'193) delete 75ceb6eb/benchmark_data_storage0111-ib.pocstor_11336_object12417/head//1 by client.1312877.0:39701 2014-08-11 16:27:10.453521 2014-08-13 18:06:06.369141 7f5c973a57a0 20 read_log 192888'613 (192888'194) delete 8e72486b/benchmark_data_storage0111-ib.pocstor_11336_object12632/head//1 by client.1312877.0:39916 2014-08-11 16:27:10.505622 2014-08-13 18:06:06.369173 7f5c973a57a0 20 read_log 192888'614 (192888'195) delete cce08cab/benchmark_data_storage0111-ib.pocstor_11336_object12674/head//1 by client.1312877.0:39958 2014-08-11 16:27:10.515088 2014-08-13 18:06:06.369195 7f5c973a57a0 20 read_log 192888'615 (192888'196) delete 3b28fc6b/benchmark_data_storage0111-ib.pocstor_11336_object12837/head//1 by client.1312877.0:40121 2014-08-11 16:27:10.558608 2014-08-13 18:06:06.369221 7f5c973a57a0 20 read_log 192888'616 (192888'197) delete d4d8526b/benchmark_data_storage0111-ib.pocstor_11336_object12951/head//1 by client.1312877.0:40235 2014-08-11 16:27:10.585349 2014-08-13 18:06:06.369242 7f5c973a57a0 20 read_log 192888'617 (192888'198) delete 1454bcab/benchmark_data_storage0111-ib.pocstor_11336_object12984/head//1 by client.1312877.0:40268 2014-08-11 16:27:10.593713 2014-08-13 18:06:06.369264 7f5c973a57a0 20 read_log 192888'618 (192888'199) delete 7ad66aab/benchmark_data_storage0111-ib.pocstor_11336_object13208/head//1 by client.1312877.0:40492 2014-08-11 16:27:10.652949 2014-08-13 18:06:06.369286 7f5c973a57a0 20 read_log 192888'619 (192888'200) delete ef9a3c6b/benchmark_data_storage0111-ib.pocstor_11336_object13239/head//1 by client.1312877.0:40523 2014-08-11 16:27:10.660620 2014-08-13 18:06:06.369312 7f5c973a57a0 20 read_log 192888'620 (192888'201) delete a87af4eb/benchmark_data_storage0111-ib.pocstor_11336_object13360/head//1 by client.1312877.0:40644 2014-08-11 16:27:10.695009 2014-08-13 18:06:06.369339 7f5c973a57a0 20 read_log 192888'621 (192888'202) delete 2d5e4eab/benchmark_data_storage0111-ib.pocstor_11336_object13477/head//1 by client.1312877.0:40761 2014-08-11 16:27:10.724492 2014-08-13 18:06:06.369366 7f5c973a57a0 20 read_log 192888'622 (192888'203) delete 6a50cd6b/benchmark_data_storage0111-ib.pocstor_11336_object13497/head//1 by client.1312877.0:40781 2014-08-11 16:27:10.729012 2014-08-13 18:06:06.369391 7f5c973a57a0 20 read_log 192888'623 (192888'204) delete 34362b6b/benchmark_data_storage0111-ib.pocstor_11336_object13615/head//1 by client.1312877.0:40899 2014-08-11 16:27:10.758717 2014-08-13 18:06:06.369452 7f5c973a57a0 20 read_log 192888'624 (192888'205) delete dfdebe2b/benchmark_data_storage0111-ib.pocstor_11336_object13622/head//1 by client.1312877.0:40906 2014-08-11 16:27:10.760123 2014-08-13 18:06:06.369494 7f5c973a57a0 20 read_log 192888'625 (192888'206) delete 4c4339eb/benchmark_data_storage0111-ib.pocstor_11336_object13623/head//1 by client.1312877.0:40907 2014-08-11 16:27:10.760230 2014-08-13 18:06:06.369521 7f5c973a57a0 20 read_log 192888'626 (192888'207) delete 898a53ab/benchmark_data_storage0111-ib.pocstor_11336_object14154/head//1 by client.1312877.0:41438 2014-08-11 16:27:10.906781 2014-08-13 18:06:06.369543 7f5c973a57a0 20 read_log 192888'627 (192888'208) delete 48a3ff2b/benchmark_data_storage0111-ib.pocstor_11336_object14170/head//1 by client.1312877.0:41454 2014-08-11 16:27:10.910138 2014-08-13 18:06:06.369564 7f5c973a57a0 20 read_log 192888'628 (192888'209) delete 6545c6ab/benchmark_data_storage0111-ib.pocstor_11336_object14196/head//1 by client.1312877.0:41480 2014-08-11 16:27:10.917163 2014-08-13 18:06:06.369586 7f5c973a57a0 20 read_log 192888'629 (192888'210) delete da0040ab/benchmark_data_storage0111-ib.pocstor_11336_object14203/head//1 by client.1312877.0:41487 2014-08-11 16:27:10.918588 2014-08-13 18:06:06.369610 7f5c973a57a0 20 read_log 192888'630 (192888'211) delete 3b3442ab/benchmark_data_storage0111-ib.pocstor_11336_object14209/head//1 by client.1312877.0:41493 2014-08-11 16:27:10.920307 2014-08-13 18:06:06.369654 7f5c973a57a0 20 read_log 192888'631 (192888'212) delete 138d602b/benchmark_data_storage0111-ib.pocstor_11336_object14220/head//1 by client.1312877.0:41504 2014-08-11 16:27:10.922793 2014-08-13 18:06:06.369677 7f5c973a57a0 20 read_log 192888'632 (192888'213) delete d6709e6b/benchmark_data_storage0111-ib.pocstor_11336_object14223/head//1 by client.1312877.0:41507 2014-08-11 16:27:10.923563 2014-08-13 18:06:06.369699 7f5c973a57a0 20 read_log 192888'633 (192888'214) delete fe3dc06b/benchmark_data_storage0111-ib.pocstor_11336_object14364/head//1 by client.1312877.0:41648 2014-08-11 16:27:10.960824 2014-08-13 18:06:06.369721 7f5c973a57a0 20 read_log 192888'634 (192888'215) delete 4a1483ab/benchmark_data_storage0111-ib.pocstor_11336_object14418/head//1 by client.1312877.0:41702 2014-08-11 16:27:10.974038 2014-08-13 18:06:06.369744 7f5c973a57a0 20 read_log 192888'635 (192888'216) delete 89e9b56b/benchmark_data_storage0111-ib.pocstor_11336_object14424/head//1 by client.1312877.0:41708 2014-08-11 16:27:10.976360 2014-08-13 18:06:06.369765 7f5c973a57a0 20 read_log 192888'636 (192888'217) delete d45ab2ab/benchmark_data_storage0111-ib.pocstor_11336_object14441/head//1 by client.1312877.0:41725 2014-08-11 16:27:10.981286 2014-08-13 18:06:06.369787 7f5c973a57a0 20 read_log 192888'637 (192888'218) delete 35e2e1eb/benchmark_data_storage0111-ib.pocstor_11336_object14593/head//1 by client.1312877.0:41877 2014-08-11 16:27:11.020569 2014-08-13 18:06:06.369811 7f5c973a57a0 20 read_log 192888'638 (192888'219) delete f9cba92b/benchmark_data_storage0111-ib.pocstor_11336_object14598/head//1 by client.1312877.0:41882 2014-08-11 16:27:11.021511 2014-08-13 18:06:06.369833 7f5c973a57a0 20 read_log 192888'639 (192888'220) delete 5ddbe7eb/benchmark_data_storage0111-ib.pocstor_11336_object14613/head//1 by client.1312877.0:41897 2014-08-11 16:27:11.025250 2014-08-13 18:06:06.369855 7f5c973a57a0 20 read_log 192888'640 (192888'221) delete 44537cab/benchmark_data_storage0111-ib.pocstor_11336_object14743/head//1 by client.1312877.0:42027 2014-08-11 16:27:11.059361 2014-08-13 18:06:06.369879 7f5c973a57a0 20 read_log 192888'641 (192888'222) delete 3cd17fab/benchmark_data_storage0111-ib.pocstor_11336_object14794/head//1 by client.1312877.0:42078 2014-08-11 16:27:11.073162 2014-08-13 18:06:06.369901 7f5c973a57a0 20 read_log 192888'642 (192888'223) delete fa8a42eb/benchmark_data_storage0111-ib.pocstor_11336_object14952/head//1 by client.1312877.0:42236 2014-08-11 16:27:11.114397 2014-08-13 18:06:06.369923 7f5c973a57a0 20 read_log 192888'643 (192888'224) delete 96108eb/benchmark_data_storage0111-ib.pocstor_11336_object15084/head//1 by client.1312877.0:42368 2014-08-11 16:27:11.147236 2014-08-13 18:06:06.369956 7f5c973a57a0 20 read_log 192888'644 (192888'225) delete 20b9f32b/benchmark_data_storage0111-ib.pocstor_11336_object15097/head//1 by client.1312877.0:42381 2014-08-11 16:27:11.150480 2014-08-13 18:06:06.369982 7f5c973a57a0 20 read_log 192888'645 (192888'226) delete f6085eb/benchmark_data_storage0111-ib.pocstor_11336_object15386/head//1 by client.1312877.0:42670 2014-08-11 16:27:11.219754 2014-08-13 18:06:06.370009 7f5c973a57a0 20 read_log 192888'646 (192888'227) delete f1deeceb/benchmark_data_storage0111-ib.pocstor_11336_object15406/head//1 by client.1312877.0:42690 2014-08-11 16:27:11.226261 2014-08-13 18:06:06.370035 7f5c973a57a0 20 read_log 192888'647 (192888'228) delete 211e31ab/benchmark_data_storage0111-ib.pocstor_11336_object15650/head//1 by client.1312877.0:42934 2014-08-11 16:27:11.287529 2014-08-13 18:06:06.370057 7f5c973a57a0 20 read_log 192888'648 (192888'229) delete 5ea9c7eb/benchmark_data_storage0111-ib.pocstor_11336_object15752/head//1 by client.1312877.0:43036 2014-08-11 16:27:11.311606 2014-08-13 18:06:06.370079 7f5c973a57a0 20 read_log 192888'649 (192888'230) delete b8c42aab/benchmark_data_storage0111-ib.pocstor_11336_object15766/head//1 by client.1312877.0:43050 2014-08-11 16:27:11.315986 2014-08-13 18:06:06.370100 7f5c973a57a0 20 read_log 192888'650 (192888'231) delete e8b2fbab/benchmark_data_storage0111-ib.pocstor_11336_object15797/head//1 by client.1312877.0:43081 2014-08-11 16:27:11.323149 2014-08-13 18:06:06.370124 7f5c973a57a0 20 read_log 192888'651 (192888'232) delete 81b9caab/benchmark_data_storage0111-ib.pocstor_11336_object15801/head//1 by client.1312877.0:43085 2014-08-11 16:27:11.323687 2014-08-13 18:06:06.370154 7f5c973a57a0 20 read_log 192888'652 (192888'233) delete 459c8bab/benchmark_data_storage0111-ib.pocstor_11336_object15913/head//1 by client.1312877.0:43197 2014-08-11 16:27:11.351091 2014-08-13 18:06:06.370179 7f5c973a57a0 20 read_log 192888'653 (192888'234) delete 8fbd32b/benchmark_data_storage0111-ib.pocstor_11336_object15924/head//1 by client.1312877.0:43208 2014-08-11 16:27:11.353056 2014-08-13 18:06:06.370206 7f5c973a57a0 20 read_log 192888'654 (192888'235) delete 709995eb/benchmark_data_storage0111-ib.pocstor_11336_object15934/head//1 by client.1312877.0:43218 2014-08-11 16:27:11.355462 2014-08-13 18:06:06.370228 7f5c973a57a0 20 read_log 192888'655 (192888'236) delete 4f2410eb/benchmark_data_storage0111-ib.pocstor_11336_object15947/head//1 by client.1312877.0:43231 2014-08-11 16:27:11.359137 2014-08-13 18:06:06.370250 7f5c973a57a0 20 read_log 192888'656 (192888'237) delete 4fadd6ab/benchmark_data_storage0111-ib.pocstor_11336_object16038/head//1 by client.1312877.0:43322 2014-08-11 16:27:11.380007 2014-08-13 18:06:06.370271 7f5c973a57a0 20 read_log 192888'657 (192888'238) delete 5b14ab6b/benchmark_data_storage0111-ib.pocstor_11336_object16162/head//1 by client.1312877.0:43446 2014-08-11 16:27:11.414025 2014-08-13 18:06:06.370292 7f5c973a57a0 20 read_log 192888'658 (192888'239) delete 6ab0302b/benchmark_data_storage0111-ib.pocstor_11336_object16219/head//1 by client.1312877.0:43503 2014-08-11 16:27:11.428505 2014-08-13 18:06:06.370314 7f5c973a57a0 20 read_log 192888'659 (192888'240) delete b76318eb/benchmark_data_storage0111-ib.pocstor_11336_object16290/head//1 by client.1312877.0:43574 2014-08-11 16:27:11.446474 2014-08-13 18:06:06.370335 7f5c973a57a0 20 read_log 192888'660 (192888'241) delete 89d8c5eb/benchmark_data_storage0111-ib.pocstor_11336_object16371/head//1 by client.1312877.0:43655 2014-08-11 16:27:11.468964 2014-08-13 18:06:06.370361 7f5c973a57a0 20 read_log 192888'661 (192888'242) delete 88c4ec2b/benchmark_data_storage0111-ib.pocstor_11336_object16404/head//1 by client.1312877.0:43688 2014-08-11 16:27:11.478367 2014-08-13 18:06:06.370387 7f5c973a57a0 20 read_log 192888'662 (192888'243) delete d2930fab/benchmark_data_storage0111-ib.pocstor_11336_object16420/head//1 by client.1312877.0:43704 2014-08-11 16:27:11.482998 2014-08-13 18:06:06.370414 7f5c973a57a0 20 read_log 192888'663 (192888'244) delete 4bcd04eb/benchmark_data_storage0111-ib.pocstor_11336_object16464/head//1 by client.1312877.0:43748 2014-08-11 16:27:11.493244 2014-08-13 18:06:06.370509 7f5c973a57a0 20 read_log 192888'664 (192888'245) delete f5411b6b/benchmark_data_storage0111-ib.pocstor_11336_object16539/head//1 by client.1312877.0:43823 2014-08-11 16:27:11.513397 2014-08-13 18:06:06.370540 7f5c973a57a0 20 read_log 192888'665 (192888'246) delete 5b8992ab/benchmark_data_storage0111-ib.pocstor_11336_object16582/head//1 by client.1312877.0:43866 2014-08-11 16:27:11.526696 2014-08-13 18:06:06.370579 7f5c973a57a0 20 read_log 192888'666 (192888'247) delete 968ceeeb/benchmark_data_storage0111-ib.pocstor_11336_object16597/head//1 by client.1312877.0:43881 2014-08-11 16:27:11.530468 2014-08-13 18:06:06.370602 7f5c973a57a0 20 read_log 192888'667 (192888'248) delete 478cf8eb/benchmark_data_storage0111-ib.pocstor_11336_object16708/head//1 by client.1312877.0:43992 2014-08-11 16:27:11.557651 2014-08-13 18:06:06.370624 7f5c973a57a0 20 read_log 192888'668 (192888'249) delete e3afee6b/benchmark_data_storage0111-ib.pocstor_11336_object16711/head//1 by client.1312877.0:43995 2014-08-11 16:27:11.559620 2014-08-13 18:06:06.370645 7f5c973a57a0 20 read_log 192888'669 (192888'250) delete b42d546b/benchmark_data_storage0111-ib.pocstor_11336_object16737/head//1 by client.1312877.0:44021 2014-08-11 16:27:11.565532 2014-08-13 18:06:06.370667 7f5c973a57a0 20 read_log 192888'670 (192888'251) delete 6b39d7ab/benchmark_data_storage0111-ib.pocstor_11336_object16757/head//1 by client.1312877.0:44041 2014-08-11 16:27:11.571703 2014-08-13 18:06:06.370710 7f5c973a57a0 20 read_log 192888'671 (192888'252) delete 6c846b6b/benchmark_data_storage0111-ib.pocstor_11336_object16841/head//1 by client.1312877.0:44125 2014-08-11 16:27:11.594960 2014-08-13 18:06:06.370733 7f5c973a57a0 20 read_log 192888'672 (192888'253) delete 15deb2eb/benchmark_data_storage0111-ib.pocstor_11336_object16904/head//1 by client.1312877.0:44188 2014-08-11 16:27:11.610823 2014-08-13 18:06:06.370755 7f5c973a57a0 20 read_log 192888'673 (192888'254) delete 16e9aaeb/benchmark_data_storage0111-ib.pocstor_11336_object17042/head//1 by client.1312877.0:44326 2014-08-11 16:27:11.645859 2014-08-13 18:06:06.370791 7f5c973a57a0 20 read_log 192888'674 (192888'255) delete d0004aeb/benchmark_data_storage0111-ib.pocstor_11336_object17045/head//1 by client.1312877.0:44329 2014-08-11 16:27:11.646718 2014-08-13 18:06:06.370818 7f5c973a57a0 20 read_log 192888'675 (192888'256) delete 96728e6b/benchmark_data_storage0111-ib.pocstor_11336_object17143/head//1 by client.1312877.0:44427 2014-08-11 16:27:11.671980 2014-08-13 18:06:06.370840 7f5c973a57a0 20 read_log 192888'676 (192888'257) delete f7e3ef2b/benchmark_data_storage0111-ib.pocstor_11336_object17271/head//1 by client.1312877.0:44555 2014-08-11 16:27:11.708031 2014-08-13 18:06:06.370863 7f5c973a57a0 20 read_log 192888'677 (192888'258) delete cb867cab/benchmark_data_storage0111-ib.pocstor_11336_object17286/head//1 by client.1312877.0:44570 2014-08-11 16:27:11.712021 2014-08-13 18:06:06.370885 7f5c973a57a0 20 read_log 192888'678 (192888'259) delete 286a36b/benchmark_data_storage0111-ib.pocstor_11336_object17353/head//1 by client.1312877.0:44637 2014-08-11 16:27:11.728949 2014-08-13 18:06:06.370907 7f5c973a57a0 20 read_log 192888'679 (192888'260) delete 57509cab/benchmark_data_storage0111-ib.pocstor_11336_object17406/head//1 by client.1312877.0:44690 2014-08-11 16:27:11.743606 2014-08-13 18:06:06.370928 7f5c973a57a0 20 read_log 192888'680 (192888'261) delete 1978a66b/benchmark_data_storage0111-ib.pocstor_11336_object17504/head//1 by client.1312877.0:44788 2014-08-11 16:27:11.770086 2014-08-13 18:06:06.370953 7f5c973a57a0 20 read_log 192888'681 (192888'262) delete 1ff709eb/benchmark_data_storage0111-ib.pocstor_11336_object17536/head//1 by client.1312877.0:44820 2014-08-11 16:27:11.777013 2014-08-13 18:06:06.370984 7f5c973a57a0 20 read_log 192888'682 (192888'263) delete 504892ab/benchmark_data_storage0111-ib.pocstor_11336_object17552/head//1 by client.1312877.0:44836 2014-08-11 16:27:11.781393 2014-08-13 18:06:06.371006 7f5c973a57a0 20 read_log 192888'683 (192888'264) delete aa8340ab/benchmark_data_storage0111-ib.pocstor_11336_object17582/head//1 by client.1312877.0:44866 2014-08-11 16:27:11.788763 2014-08-13 18:06:06.371045 7f5c973a57a0 20 read_log 192888'684 (192888'265) delete b666282b/benchmark_data_storage0111-ib.pocstor_11336_object17624/head//1 by client.1312877.0:44908 2014-08-11 16:27:11.798476 2014-08-13 18:06:06.371089 7f5c973a57a0 20 read_log 192888'685 (192888'266) delete 792ade6b/benchmark_data_storage0111-ib.pocstor_11336_object17776/head//1 by client.1312877.0:45060 2014-08-11 16:27:11.836781 2014-08-13 18:06:06.371118 7f5c973a57a0 20 read_log 192888'686 (192888'267) delete e4e4e3ab/benchmark_data_storage0111-ib.pocstor_11336_object17791/head//1 by client.1312877.0:45075 2014-08-11 16:27:11.840190 2014-08-13 18:06:06.371165 7f5c973a57a0 20 read_log 192888'687 (192888'268) delete e22515eb/benchmark_data_storage0111-ib.pocstor_11336_object17812/head//1 by client.1312877.0:45096 2014-08-11 16:27:11.845145 2014-08-13 18:06:06.371194 7f5c973a57a0 20 read_log 192888'688 (192888'269) delete 378d10eb/benchmark_data_storage0111-ib.pocstor_11336_object17841/head//1 by client.1312877.0:45125 2014-08-11 16:27:11.851959 2014-08-13 18:06:06.371230 7f5c973a57a0 20 read_log 192888'689 (192888'270) delete a1c1b56b/benchmark_data_storage0111-ib.pocstor_11336_object17879/head//1 by client.1312877.0:45163 2014-08-11 16:27:11.860414 2014-08-13 18:06:06.371262 7f5c973a57a0 20 read_log 192888'690 (192888'271) delete 2d29506b/benchmark_data_storage0111-ib.pocstor_11336_object18047/head//1 by client.1312877.0:45331 2014-08-11 16:27:11.901604 2014-08-13 18:06:06.371290 7f5c973a57a0 20 read_log 192888'691 (192888'272) delete a954006b/benchmark_data_storage0111-ib.pocstor_11336_object18067/head//1 by client.1312877.0:45351 2014-08-11 16:27:11.906077 2014-08-13 18:06:06.371316 7f5c973a57a0 20 read_log 192888'692 (192888'273) delete 3ff41aeb/benchmark_data_storage0111-ib.pocstor_11336_object18232/head//1 by client.1312877.0:45516 2014-08-11 16:27:11.947543 2014-08-13 18:06:06.371342 7f5c973a57a0 20 read_log 192888'693 (192888'274) delete 1953402b/benchmark_data_storage0111-ib.pocstor_11336_object18323/head//1 by client.1312877.0:45607 2014-08-11 16:27:11.971845 2014-08-13 18:06:06.371364 7f5c973a57a0 20 read_log 192888'694 (192888'275) delete c2fa4d2b/benchmark_data_storage0111-ib.pocstor_11336_object18387/head//1 by client.1312877.0:45671 2014-08-11 16:27:11.986546 2014-08-13 18:06:06.371407 7f5c973a57a0 20 read_log 192888'695 (192888'276) delete 9efed92b/benchmark_data_storage0111-ib.pocstor_11336_object18517/head//1 by client.1312877.0:45801 2014-08-11 16:27:12.016387 2014-08-13 18:06:06.371441 7f5c973a57a0 20 read_log 192888'696 (192888'277) delete 1846c0eb/benchmark_data_storage0111-ib.pocstor_11336_object18527/head//1 by client.1312877.0:45811 2014-08-11 16:27:12.019480 2014-08-13 18:06:06.371467 7f5c973a57a0 20 read_log 192888'697 (192888'278) delete 275089ab/benchmark_data_storage0111-ib.pocstor_11336_object18528/head//1 by client.1312877.0:45812 2014-08-11 16:27:12.019664 2014-08-13 18:06:06.371494 7f5c973a57a0 20 read_log 192888'698 (192888'279) delete bcd079eb/benchmark_data_storage0111-ib.pocstor_11336_object18542/head//1 by client.1312877.0:45826 2014-08-11 16:27:12.023626 2014-08-13 18:06:06.371536 7f5c973a57a0 20 read_log 192888'699 (192888'280) delete 52868e6b/benchmark_data_storage0111-ib.pocstor_11336_object18592/head//1 by client.1312877.0:45876 2014-08-11 16:27:12.036502 2014-08-13 18:06:06.371564 7f5c973a57a0 20 read_log 192888'700 (192888'281) delete 426531eb/benchmark_data_storage0111-ib.pocstor_11336_object18604/head//1 by client.1312877.0:45888 2014-08-11 16:27:12.039485 2014-08-13 18:06:06.371591 7f5c973a57a0 20 read_log 192888'701 (192888'282) delete 2aad1c6b/benchmark_data_storage0111-ib.pocstor_11336_object18663/head//1 by client.1312877.0:45947 2014-08-11 16:27:12.052571 2014-08-13 18:06:06.371635 7f5c973a57a0 20 read_log 192888'702 (192888'283) delete 5869a02b/benchmark_data_storage0111-ib.pocstor_11336_object18781/head//1 by client.1312877.0:46065 2014-08-11 16:27:12.081703 2014-08-13 18:06:06.371662 7f5c973a57a0 20 read_log 192888'703 (192888'284) delete 4187e2eb/benchmark_data_storage0111-ib.pocstor_11336_object18834/head//1 by client.1312877.0:46118 2014-08-11 16:27:12.093643 2014-08-13 18:06:06.371718 7f5c973a57a0 20 read_log 192888'704 (192888'285) delete 4f44796b/benchmark_data_storage0111-ib.pocstor_11336_object18875/head//1 by client.1312877.0:46159 2014-08-11 16:27:12.105002 2014-08-13 18:06:06.371786 7f5c973a57a0 20 read_log 192888'705 (192888'286) delete fd4b836b/benchmark_data_storage0111-ib.pocstor_11336_object18888/head//1 by client.1312877.0:46172 2014-08-11 16:27:12.108257 2014-08-13 18:06:06.371814 7f5c973a57a0 20 read_log 192888'706 (192888'287) delete 2b80b52b/benchmark_data_storage0111-ib.pocstor_11336_object18990/head//1 by client.1312877.0:46274 2014-08-11 16:27:12.133362 2014-08-13 18:06:06.371836 7f5c973a57a0 20 read_log 192888'707 (192888'288) delete 1c3083ab/benchmark_data_storage0111-ib.pocstor_11336_object19003/head//1 by client.1312877.0:46287 2014-08-11 16:27:12.136537 2014-08-13 18:06:06.371904 7f5c973a57a0 20 read_log 192888'708 (192888'289) delete c2464ceb/benchmark_data_storage0111-ib.pocstor_11336_object19103/head//1 by client.1312877.0:46387 2014-08-11 16:27:12.160144 2014-08-13 18:06:06.371926 7f5c973a57a0 20 read_log 192888'709 (192888'290) delete 122a33ab/benchmark_data_storage0111-ib.pocstor_11336_object19227/head//1 by client.1312877.0:46511 2014-08-11 16:27:12.190816 2014-08-13 18:06:06.371949 7f5c973a57a0 20 read_log 192888'710 (192888'291) delete e7ba396b/benchmark_data_storage0111-ib.pocstor_11336_object19274/head//1 by client.1312877.0:46558 2014-08-11 16:27:12.202176 2014-08-13 18:06:06.371970 7f5c973a57a0 20 read_log 192888'711 (192888'292) delete 9bd5cdeb/benchmark_data_storage0111-ib.pocstor_11336_object19328/head//1 by client.1312877.0:46612 2014-08-11 16:27:12.216483 2014-08-13 18:06:06.371992 7f5c973a57a0 20 read_log 192888'712 (192888'293) delete d0eba76b/benchmark_data_storage0111-ib.pocstor_11336_object19418/head//1 by client.1312877.0:46702 2014-08-11 16:27:12.239150 2014-08-13 18:06:06.372014 7f5c973a57a0 20 read_log 192888'713 (192888'294) delete 1dc295eb/benchmark_data_storage0111-ib.pocstor_11336_object19457/head//1 by client.1312877.0:46741 2014-08-11 16:27:12.248567 2014-08-13 18:06:06.372035 7f5c973a57a0 20 read_log 192888'714 (192888'295) delete 1dd11fab/benchmark_data_storage0111-ib.pocstor_11336_object19469/head//1 by client.1312877.0:46753 2014-08-11 16:27:12.252748 2014-08-13 18:06:06.372057 7f5c973a57a0 20 read_log 192888'715 (192888'296) delete c64d172b/benchmark_data_storage0111-ib.pocstor_11336_object19516/head//1 by client.1312877.0:46800 2014-08-11 16:27:12.267414 2014-08-13 18:06:06.372078 7f5c973a57a0 20 read_log 192888'716 (192888'297) delete 475791eb/benchmark_data_storage0111-ib.pocstor_11336_object19587/head//1 by client.1312877.0:46871 2014-08-11 16:27:12.283858 2014-08-13 18:06:06.372100 7f5c973a57a0 20 read_log 192888'717 (192888'298) delete e2681e6b/benchmark_data_storage0111-ib.pocstor_11336_object19629/head//1 by client.1312877.0:46913 2014-08-11 16:27:12.295019 2014-08-13 18:06:06.372122 7f5c973a57a0 20 read_log 192888'718 (192888'299) delete e272fb6b/benchmark_data_storage0111-ib.pocstor_11336_object19824/head//1 by client.1312877.0:47108 2014-08-11 16:27:12.348318 2014-08-13 18:06:06.372144 7f5c973a57a0 20 read_log 192888'719 (192888'300) delete 24e902b/benchmark_data_storage0111-ib.pocstor_11336_object19842/head//1 by client.1312877.0:47126 2014-08-11 16:27:12.352277 2014-08-13 18:06:06.372175 7f5c973a57a0 20 read_log 192888'720 (192888'301) delete d95a362b/benchmark_data_storage0111-ib.pocstor_11336_object19869/head//1 by client.1312877.0:47153 2014-08-11 16:27:12.359775 2014-08-13 18:06:06.372197 7f5c973a57a0 20 read_log 192888'721 (192888'302) delete 46edc8eb/benchmark_data_storage0111-ib.pocstor_11336_object19881/head//1 by client.1312877.0:47165 2014-08-11 16:27:12.362302 2014-08-13 18:06:06.372223 7f5c973a57a0 20 read_log 192888'722 (192888'303) delete 17820beb/benchmark_data_storage0111-ib.pocstor_11336_object19886/head//1 by client.1312877.0:47170 2014-08-11 16:27:12.363323 2014-08-13 18:06:06.372245 7f5c973a57a0 20 read_log 192888'723 (192888'304) delete 86a8e56b/benchmark_data_storage0111-ib.pocstor_11336_object19954/head//1 by client.1312877.0:47238 2014-08-11 16:27:12.381422 2014-08-13 18:06:06.372286 7f5c973a57a0 20 read_log 192888'724 (192888'305) delete 68ddf0ab/benchmark_data_storage0111-ib.pocstor_11336_object19987/head//1 by client.1312877.0:47271 2014-08-11 16:27:12.388873 2014-08-13 18:06:06.372319 7f5c973a57a0 20 read_log 192888'725 (192888'306) delete 1fe4d72b/benchmark_data_storage0111-ib.pocstor_11336_object20033/head//1 by client.1312877.0:47317 2014-08-11 16:27:12.399721 2014-08-13 18:06:06.372346 7f5c973a57a0 20 read_log 192888'726 (192888'307) delete a91445eb/benchmark_data_storage0111-ib.pocstor_11336_object20056/head//1 by client.1312877.0:47340 2014-08-11 16:27:12.405372 2014-08-13 18:06:06.372372 7f5c973a57a0 20 read_log 192888'727 (192888'308) delete 7faf2f6b/benchmark_data_storage0111-ib.pocstor_11336_object20068/head//1 by client.1312877.0:47352 2014-08-11 16:27:12.408134 2014-08-13 18:06:06.372395 7f5c973a57a0 20 read_log 192888'728 (192888'309) delete 8b39baab/benchmark_data_storage0111-ib.pocstor_11336_object20099/head//1 by client.1312877.0:47383 2014-08-11 16:27:12.417092 2014-08-13 18:06:06.372433 7f5c973a57a0 20 read_log 192888'729 (192888'310) delete f67c52b/benchmark_data_storage0111-ib.pocstor_11336_object20190/head//1 by client.1312877.0:47474 2014-08-11 16:27:12.439320 2014-08-13 18:06:06.372461 7f5c973a57a0 20 read_log 192888'730 (192888'311) delete 9b83842b/benchmark_data_storage0111-ib.pocstor_11336_object20253/head//1 by client.1312877.0:47537 2014-08-11 16:27:12.454398 2014-08-13 18:06:06.372483 7f5c973a57a0 20 read_log 192888'731 (192888'312) delete 5f94746b/benchmark_data_storage0111-ib.pocstor_11336_object20281/head//1 by client.1312877.0:47565 2014-08-11 16:27:12.460701 2014-08-13 18:06:06.372505 7f5c973a57a0 20 read_log 192888'732 (192888'313) delete f708caab/benchmark_data_storage0111-ib.pocstor_11336_object20467/head//1 by client.1312877.0:47751 2014-08-11 16:27:12.508290 2014-08-13 18:06:06.372526 7f5c973a57a0 20 read_log 192888'733 (192888'314) delete 5b14d96b/benchmark_data_storage0111-ib.pocstor_11336_object20478/head//1 by client.1312877.0:47762 2014-08-11 16:27:12.511251 2014-08-13 18:06:06.372547 7f5c973a57a0 20 read_log 192888'734 (192888'315) delete b820eb2b/benchmark_data_storage0111-ib.pocstor_11336_object20492/head//1 by client.1312877.0:47776 2014-08-11 16:27:12.514511 2014-08-13 18:06:06.372569 7f5c973a57a0 20 read_log 192888'735 (192888'316) delete d119eab/benchmark_data_storage0111-ib.pocstor_11336_object20632/head//1 by client.1312877.0:47916 2014-08-11 16:27:12.549861 2014-08-13 18:06:06.372590 7f5c973a57a0 20 read_log 192888'736 (192888'317) delete 11cba56b/benchmark_data_storage0111-ib.pocstor_11336_object20947/head//1 by client.1312877.0:48231 2014-08-11 16:27:12.635101 2014-08-13 18:06:06.372645 7f5c973a57a0 20 read_log 192888'737 (192888'318) delete 60a1adeb/benchmark_data_storage0111-ib.pocstor_11336_object21180/head//1 by client.1312877.0:48464 2014-08-11 16:27:12.698629 2014-08-13 18:06:06.372667 7f5c973a57a0 20 read_log 192888'738 (192888'319) delete 7125f2ab/benchmark_data_storage0111-ib.pocstor_11336_object21256/head//1 by client.1312877.0:48540 2014-08-11 16:27:12.719218 2014-08-13 18:06:06.372689 7f5c973a57a0 20 read_log 192888'739 (192888'320) delete 7726eb2b/benchmark_data_storage0111-ib.pocstor_11336_object21372/head//1 by client.1312877.0:48656 2014-08-11 16:27:12.749523 2014-08-13 18:06:06.372711 7f5c973a57a0 20 read_log 192888'740 (192888'321) delete 242250ab/benchmark_data_storage0111-ib.pocstor_11336_object21400/head//1 by client.1312877.0:48684 2014-08-11 16:27:12.755612 2014-08-13 18:06:06.372732 7f5c973a57a0 20 read_log 192888'741 (192888'322) delete 54fc23ab/benchmark_data_storage0111-ib.pocstor_11336_object21424/head//1 by client.1312877.0:48708 2014-08-11 16:27:12.763093 2014-08-13 18:06:06.372754 7f5c973a57a0 20 read_log 192888'742 (192888'323) delete e546a22b/benchmark_data_storage0111-ib.pocstor_11336_object21443/head//1 by client.1312877.0:48727 2014-08-11 16:27:12.767557 2014-08-13 18:06:06.372775 7f5c973a57a0 20 read_log 192888'743 (192888'324) delete b69c4a2b/benchmark_data_storage0111-ib.pocstor_11336_object21466/head//1 by client.1312877.0:48750 2014-08-11 16:27:12.773584 2014-08-13 18:06:06.372809 7f5c973a57a0 20 read_log 192888'744 (192888'325) delete 5923652b/benchmark_data_storage0111-ib.pocstor_11336_object21528/head//1 by client.1312877.0:48812 2014-08-11 16:27:12.787682 2014-08-13 18:06:06.372836 7f5c973a57a0 20 read_log 192888'745 (192888'326) delete da17c26b/benchmark_data_storage0111-ib.pocstor_11336_object21763/head//1 by client.1312877.0:49047 2014-08-11 16:27:12.844874 2014-08-13 18:06:06.372858 7f5c973a57a0 20 read_log 192888'746 (192888'327) delete 87148eb/benchmark_data_storage0111-ib.pocstor_11336_object21812/head//1 by client.1312877.0:49096 2014-08-11 16:27:12.856122 2014-08-13 18:06:06.372883 7f5c973a57a0 20 read_log 192888'747 (192888'328) delete 9101e96b/benchmark_data_storage0111-ib.pocstor_11336_object21868/head//1 by client.1312877.0:49152 2014-08-11 16:27:12.872203 2014-08-13 18:06:06.372905 7f5c973a57a0 20 read_log 192888'748 (192888'329) delete 7c4fe4ab/benchmark_data_storage0111-ib.pocstor_11336_object21885/head//1 by client.1312877.0:49169 2014-08-11 16:27:12.876884 2014-08-13 18:06:06.372926 7f5c973a57a0 20 read_log 192888'749 (192888'330) delete ad24d8ab/benchmark_data_storage0111-ib.pocstor_11336_object21935/head//1 by client.1312877.0:49219 2014-08-11 16:27:12.889064 2014-08-13 18:06:06.372948 7f5c973a57a0 20 read_log 192888'750 (192888'331) delete 787deceb/benchmark_data_storage0111-ib.pocstor_11336_object21956/head//1 by client.1312877.0:49240 2014-08-11 16:27:12.895379 2014-08-13 18:06:06.372969 7f5c973a57a0 20 read_log 192888'751 (192888'332) delete f978002b/benchmark_data_storage0111-ib.pocstor_11336_object21997/head//1 by client.1312877.0:49281 2014-08-11 16:27:12.905336 2014-08-13 18:06:06.372991 7f5c973a57a0 20 read_log 192888'752 (192888'333) delete 959dd26b/benchmark_data_storage0111-ib.pocstor_11336_object22000/head//1 by client.1312877.0:49284 2014-08-11 16:27:12.907143 2014-08-13 18:06:06.373012 7f5c973a57a0 20 read_log 192888'753 (192888'334) delete 4e472ceb/benchmark_data_storage0111-ib.pocstor_11336_object22034/head//1 by client.1312877.0:49318 2014-08-11 16:27:12.914790 2014-08-13 18:06:06.373037 7f5c973a57a0 20 read_log 192888'754 (192888'335) delete 6c1ac06b/benchmark_data_storage0111-ib.pocstor_11336_object22051/head//1 by client.1312877.0:49335 2014-08-11 16:27:12.918804 2014-08-13 18:06:06.373059 7f5c973a57a0 20 read_log 192888'755 (192888'336) delete 373a052b/benchmark_data_storage0111-ib.pocstor_11336_object22091/head//1 by client.1312877.0:49375 2014-08-11 16:27:12.928357 2014-08-13 18:06:06.373084 7f5c973a57a0 20 read_log 192888'756 (192888'337) delete 31cfc02b/benchmark_data_storage0111-ib.pocstor_11336_object22258/head//1 by client.1312877.0:49542 2014-08-11 16:27:12.967129 2014-08-13 18:06:06.373106 7f5c973a57a0 20 read_log 192888'757 (192888'338) delete 6de7eaab/benchmark_data_storage0111-ib.pocstor_11336_object22272/head//1 by client.1312877.0:49556 2014-08-11 16:27:12.971208 2014-08-13 18:06:06.373131 7f5c973a57a0 20 read_log 192888'758 (192888'339) delete 1781c26b/benchmark_data_storage0111-ib.pocstor_11336_object22296/head//1 by client.1312877.0:49580 2014-08-11 16:27:12.977309 2014-08-13 18:06:06.373162 7f5c973a57a0 20 read_log 192888'759 (192888'340) delete f157deab/benchmark_data_storage0111-ib.pocstor_11336_object22346/head//1 by client.1312877.0:49630 2014-08-11 16:27:12.988370 2014-08-13 18:06:06.373223 7f5c973a57a0 20 read_log 192888'760 (192888'341) delete f68908eb/benchmark_data_storage0111-ib.pocstor_11336_object22441/head//1 by client.1312877.0:49725 2014-08-11 16:27:13.011609 2014-08-13 18:06:06.373252 7f5c973a57a0 20 read_log 192888'761 (192888'342) delete a20011ab/benchmark_data_storage0111-ib.pocstor_11336_object22467/head//1 by client.1312877.0:49751 2014-08-11 16:27:13.017487 2014-08-13 18:06:06.373274 7f5c973a57a0 20 read_log 192888'762 (192888'343) delete 82490b2b/benchmark_data_storage0111-ib.pocstor_11336_object22469/head//1 by client.1312877.0:49753 2014-08-11 16:27:13.018165 2014-08-13 18:06:06.373296 7f5c973a57a0 20 read_log 192888'763 (192888'344) delete 520cf2eb/benchmark_data_storage0111-ib.pocstor_11336_object22473/head//1 by client.1312877.0:49757 2014-08-11 16:27:13.019544 2014-08-13 18:06:06.373333 7f5c973a57a0 20 read_log 192888'764 (192888'345) delete 42e912ab/benchmark_data_storage0111-ib.pocstor_11336_object22545/head//1 by client.1312877.0:49829 2014-08-11 16:27:13.037029 2014-08-13 18:06:06.373367 7f5c973a57a0 20 read_log 192888'765 (192888'346) delete 96bbf22b/benchmark_data_storage0111-ib.pocstor_11336_object22579/head//1 by client.1312877.0:49863 2014-08-11 16:27:13.046206 2014-08-13 18:06:06.373393 7f5c973a57a0 20 read_log 192888'766 (192888'347) delete a0f15c6b/benchmark_data_storage0111-ib.pocstor_11336_object22587/head//1 by client.1312877.0:49871 2014-08-11 16:27:13.047879 2014-08-13 18:06:06.373421 7f5c973a57a0 20 read_log 192888'767 (192888'348) delete 698c546b/benchmark_data_storage0111-ib.pocstor_11336_object22636/head//1 by client.1312877.0:49920 2014-08-11 16:27:13.060644 2014-08-13 18:06:06.373443 7f5c973a57a0 20 read_log 192888'768 (192888'349) delete a9a305eb/benchmark_data_storage0111-ib.pocstor_11336_object22716/head//1 by client.1312877.0:50000 2014-08-11 16:27:13.082160 2014-08-13 18:06:06.373467 7f5c973a57a0 20 read_log 192888'769 (192888'350) delete 20cdd2b/benchmark_data_storage0111-ib.pocstor_11336_object22738/head//1 by client.1312877.0:50022 2014-08-11 16:27:13.087488 2014-08-13 18:06:06.373510 7f5c973a57a0 20 read_log 192888'770 (192888'351) delete 13e43f2b/benchmark_data_storage0111-ib.pocstor_11336_object22869/head//1 by client.1312877.0:50153 2014-08-11 16:27:13.123032 2014-08-13 18:06:06.373532 7f5c973a57a0 20 read_log 192888'771 (192888'352) delete efa1d42b/benchmark_data_storage0111-ib.pocstor_11336_object22914/head//1 by client.1312877.0:50198 2014-08-11 16:27:13.134496 2014-08-13 18:06:06.373555 7f5c973a57a0 20 read_log 192888'772 (192888'353) delete 96aeefeb/benchmark_data_storage0111-ib.pocstor_11336_object22932/head//1 by client.1312877.0:50216 2014-08-11 16:27:13.139077 2014-08-13 18:06:06.373593 7f5c973a57a0 20 read_log 192888'773 (192888'354) delete 1a986b6b/benchmark_data_storage0111-ib.pocstor_11336_object22937/head//1 by client.1312877.0:50221 2014-08-11 16:27:13.140583 2014-08-13 18:06:06.373622 7f5c973a57a0 20 read_log 192888'774 (192888'355) delete a0706d2b/benchmark_data_storage0111-ib.pocstor_11336_object22960/head//1 by client.1312877.0:50244 2014-08-11 16:27:13.147693 2014-08-13 18:06:06.373644 7f5c973a57a0 20 read_log 192888'775 (192888'356) delete 8e2579ab/benchmark_data_storage0111-ib.pocstor_11336_object22962/head//1 by client.1312877.0:50246 2014-08-11 16:27:13.148019 2014-08-13 18:06:06.373682 7f5c973a57a0 20 read_log 192888'776 (192888'357) delete 7ebd15eb/benchmark_data_storage0111-ib.pocstor_11336_object23023/head//1 by client.1312877.0:50307 2014-08-11 16:27:13.161802 2014-08-13 18:06:06.373719 7f5c973a57a0 20 read_log 192888'777 (192888'358) delete 53266aeb/benchmark_data_storage0111-ib.pocstor_11336_object23033/head//1 by client.1312877.0:50317 2014-08-11 16:27:13.164869 2014-08-13 18:06:06.373748 7f5c973a57a0 20 read_log 192888'778 (192888'359) delete 45b50ab/benchmark_data_storage0111-ib.pocstor_11336_object23056/head//1 by client.1312877.0:50340 2014-08-11 16:27:13.169617 2014-08-13 18:06:06.373776 7f5c973a57a0 20 read_log 192888'779 (192888'360) delete 686380ab/benchmark_data_storage0111-ib.pocstor_11336_object23170/head//1 by client.1312877.0:50454 2014-08-11 16:27:13.198670 2014-08-13 18:06:06.373802 7f5c973a57a0 20 read_log 192888'780 (192888'361) delete a62bfc6b/benchmark_data_storage0111-ib.pocstor_11336_object23205/head//1 by client.1312877.0:50489 2014-08-11 16:27:13.206738 2014-08-13 18:06:06.373824 7f5c973a57a0 20 read_log 192888'781 (192888'362) delete b6784d6b/benchmark_data_storage0111-ib.pocstor_11336_object23214/head//1 by client.1312877.0:50498 2014-08-11 16:27:13.208506 2014-08-13 18:06:06.373846 7f5c973a57a0 20 read_log 192888'782 (192888'363) delete e4717fab/benchmark_data_storage0111-ib.pocstor_11336_object23363/head//1 by client.1312877.0:50647 2014-08-11 16:27:13.247860 2014-08-13 18:06:06.373867 7f5c973a57a0 20 read_log 192888'783 (192888'364) delete 793e702b/benchmark_data_storage0111-ib.pocstor_11336_object23422/head//1 by client.1312877.0:50706 2014-08-11 16:27:13.262483 2014-08-13 18:06:06.373905 7f5c973a57a0 20 read_log 192888'784 (192888'365) delete 90f044eb/benchmark_data_storage0111-ib.pocstor_11336_object23495/head//1 by client.1312877.0:50779 2014-08-11 16:27:13.278767 2014-08-13 18:06:06.373932 7f5c973a57a0 20 read_log 192888'785 (192888'366) delete 30bf61ab/benchmark_data_storage0111-ib.pocstor_11336_object23554/head//1 by client.1312877.0:50838 2014-08-11 16:27:13.293079 2014-08-13 18:06:06.373954 7f5c973a57a0 20 read_log 192888'786 (192888'367) delete c494546b/benchmark_data_storage0111-ib.pocstor_11336_object23580/head//1 by client.1312877.0:50864 2014-08-11 16:27:13.299444 2014-08-13 18:06:06.373976 7f5c973a57a0 20 read_log 192888'787 (192888'368) delete 60e4f62b/benchmark_data_storage0111-ib.pocstor_11336_object23637/head//1 by client.1312877.0:50921 2014-08-11 16:27:13.311435 2014-08-13 18:06:06.373998 7f5c973a57a0 20 read_log 192888'788 (192888'369) delete 4c4bf4ab/benchmark_data_storage0111-ib.pocstor_11336_object23654/head//1 by client.1312877.0:50938 2014-08-11 16:27:13.315264 2014-08-13 18:06:06.374019 7f5c973a57a0 20 read_log 192888'789 (192888'370) delete a946212b/benchmark_data_storage0111-ib.pocstor_11336_object23753/head//1 by client.1312877.0:51037 2014-08-11 16:27:13.337745 2014-08-13 18:06:06.374042 7f5c973a57a0 20 read_log 192888'790 (192888'371) delete a1fe9ab/benchmark_data_storage0111-ib.pocstor_11336_object23786/head//1 by client.1312877.0:51070 2014-08-11 16:27:13.346893 2014-08-13 18:06:06.374064 7f5c973a57a0 20 read_log 192888'791 (192888'372) delete b45acb2b/benchmark_data_storage0111-ib.pocstor_11336_object23815/head//1 by client.1312877.0:51099 2014-08-11 16:27:13.352368 2014-08-13 18:06:06.374085 7f5c973a57a0 20 read_log 192888'792 (192888'373) delete f2e080eb/benchmark_data_storage0111-ib.pocstor_11336_object23867/head//1 by client.1312877.0:51151 2014-08-11 16:27:13.368798 2014-08-13 18:06:06.374106 7f5c973a57a0 20 read_log 192888'793 (192888'374) delete a083866b/benchmark_data_storage0111-ib.pocstor_11336_object23921/head//1 by client.1312877.0:51205 2014-08-11 16:27:13.382086 2014-08-13 18:06:06.374130 7f5c973a57a0 20 read_log 192888'794 (192888'375) delete 8bbc9eeb/benchmark_data_storage0111-ib.pocstor_11336_object23984/head//1 by client.1312877.0:51268 2014-08-11 16:27:13.397936 2014-08-13 18:06:06.374161 7f5c973a57a0 20 read_log 192888'795 (192888'376) delete e7d972b/benchmark_data_storage0111-ib.pocstor_11336_object24002/head//1 by client.1312877.0:51286 2014-08-11 16:27:13.402011 2014-08-13 18:06:06.374201 7f5c973a57a0 20 read_log 192888'796 (192888'377) delete a7b66c6b/benchmark_data_storage0111-ib.pocstor_11336_object24011/head//1 by client.1312877.0:51295 2014-08-11 16:27:13.404012 2014-08-13 18:06:06.374246 7f5c973a57a0 20 read_log 192888'797 (192888'378) delete 8d09746b/benchmark_data_storage0111-ib.pocstor_11336_object24032/head//1 by client.1312877.0:51316 2014-08-11 16:27:13.411714 2014-08-13 18:06:06.374311 7f5c973a57a0 20 read_log 192888'798 (192888'379) delete f96e4feb/benchmark_data_storage0111-ib.pocstor_11336_object24132/head//1 by client.1312877.0:51416 2014-08-11 16:27:13.436529 2014-08-13 18:06:06.374346 7f5c973a57a0 20 read_log 192888'799 (192888'380) delete 12d73bab/benchmark_data_storage0111-ib.pocstor_11336_object24218/head//1 by client.1312877.0:51502 2014-08-11 16:27:13.461534 2014-08-13 18:06:06.374373 7f5c973a57a0 20 read_log 192888'800 (192888'381) delete 9866d8eb/benchmark_data_storage0111-ib.pocstor_11336_object24269/head//1 by client.1312877.0:51553 2014-08-11 16:27:13.475672 2014-08-13 18:06:06.374400 7f5c973a57a0 20 read_log 192888'801 (192888'382) delete de90c7eb/benchmark_data_storage0111-ib.pocstor_11336_object24341/head//1 by client.1312877.0:51625 2014-08-11 16:27:13.492981 2014-08-13 18:06:06.374426 7f5c973a57a0 20 read_log 192888'802 (192888'383) delete 537521eb/benchmark_data_storage0111-ib.pocstor_11336_object24410/head//1 by client.1312877.0:51694 2014-08-11 16:27:13.513164 2014-08-13 18:06:06.374448 7f5c973a57a0 20 read_log 192888'803 (192888'384) delete e6d1e6ab/benchmark_data_storage0111-ib.pocstor_11336_object24502/head//1 by client.1312877.0:51786 2014-08-11 16:27:13.536056 2014-08-13 18:06:06.374546 7f5c973a57a0 20 read_log 192888'804 (192888'385) delete 570d676b/benchmark_data_storage0111-ib.pocstor_11336_object24513/head//1 by client.1312877.0:51797 2014-08-11 16:27:13.538322 2014-08-13 18:06:06.374580 7f5c973a57a0 20 read_log 192888'805 (192888'386) delete 4d3698ab/benchmark_data_storage0111-ib.pocstor_11336_object24640/head//1 by client.1312877.0:51924 2014-08-11 16:27:13.572565 2014-08-13 18:06:06.374602 7f5c973a57a0 20 read_log 192888'806 (192888'387) delete 6fd4ceb/benchmark_data_storage0111-ib.pocstor_11336_object24678/head//1 by client.1312877.0:51962 2014-08-11 16:27:13.580577 2014-08-13 18:06:06.374624 7f5c973a57a0 20 read_log 192888'807 (192888'388) delete dda641eb/benchmark_data_storage0111-ib.pocstor_11336_object24696/head//1 by client.1312877.0:51980 2014-08-11 16:27:13.585828 2014-08-13 18:06:06.374645 7f5c973a57a0 20 read_log 192888'808 (192888'389) delete 3edb27eb/benchmark_data_storage0111-ib.pocstor_11336_object24704/head//1 by client.1312877.0:51988 2014-08-11 16:27:13.587950 2014-08-13 18:06:06.374667 7f5c973a57a0 20 read_log 192888'809 (192888'390) delete c80d832b/benchmark_data_storage0111-ib.pocstor_11336_object24729/head//1 by client.1312877.0:52013 2014-08-11 16:27:13.593963 2014-08-13 18:06:06.374689 7f5c973a57a0 20 read_log 192888'810 (192888'391) delete 7fdffdeb/benchmark_data_storage0111-ib.pocstor_11336_object24731/head//1 by client.1312877.0:52015 2014-08-11 16:27:13.594288 2014-08-13 18:06:06.374711 7f5c973a57a0 20 read_log 192888'811 (192888'392) delete fbb74cab/benchmark_data_storage0111-ib.pocstor_11336_object24889/head//1 by client.1312877.0:52173 2014-08-11 16:27:13.635479 2014-08-13 18:06:06.374732 7f5c973a57a0 20 read_log 192888'812 (192888'393) delete abc784eb/benchmark_data_storage0111-ib.pocstor_11336_object24918/head//1 by client.1312877.0:52202 2014-08-11 16:27:13.643201 2014-08-13 18:06:06.374754 7f5c973a57a0 20 read_log 192888'813 (192888'394) delete aa96b5eb/benchmark_data_storage0111-ib.pocstor_11336_object25001/head//1 by client.1312877.0:52285 2014-08-11 16:27:13.664317 2014-08-13 18:06:06.374775 7f5c973a57a0 20 read_log 192888'814 (192888'395) delete f622b72b/benchmark_data_storage0111-ib.pocstor_11336_object25080/head//1 by client.1312877.0:52364 2014-08-11 16:27:13.686984 2014-08-13 18:06:06.374797 7f5c973a57a0 20 read_log 192888'815 (192888'396) delete 42235aeb/benchmark_data_storage0111-ib.pocstor_11336_object25120/head//1 by client.1312877.0:52404 2014-08-11 16:27:13.698099 2014-08-13 18:06:06.374818 7f5c973a57a0 20 read_log 192888'816 (192888'397) delete 965b2cab/benchmark_data_storage0111-ib.pocstor_11336_object25176/head//1 by client.1312877.0:52460 2014-08-11 16:27:13.710829 2014-08-13 18:06:06.374840 7f5c973a57a0 20 read_log 192888'817 (192888'398) delete 802de92b/benchmark_data_storage0111-ib.pocstor_11336_object25189/head//1 by client.1312877.0:52473 2014-08-11 16:27:13.713793 2014-08-13 18:06:06.374861 7f5c973a57a0 20 read_log 192888'818 (192888'399) delete 9409426b/benchmark_data_storage0111-ib.pocstor_11336_object25191/head//1 by client.1312877.0:52475 2014-08-11 16:27:13.715060 2014-08-13 18:06:06.374883 7f5c973a57a0 20 read_log 192888'819 (192888'400) delete c26e8ceb/benchmark_data_storage0111-ib.pocstor_11336_object25320/head//1 by client.1312877.0:52604 2014-08-11 16:27:13.751180 2014-08-13 18:06:06.374904 7f5c973a57a0 20 read_log 192888'820 (192888'401) delete 5bd198eb/benchmark_data_storage0111-ib.pocstor_11336_object25342/head//1 by client.1312877.0:52626 2014-08-11 16:27:13.760748 2014-08-13 18:06:06.374926 7f5c973a57a0 20 read_log 192888'821 (192888'402) delete f2bae72b/benchmark_data_storage0111-ib.pocstor_11336_object25411/head//1 by client.1312877.0:52695 2014-08-11 16:27:13.777063 2014-08-13 18:06:06.374947 7f5c973a57a0 20 read_log 192888'822 (192888'403) delete 22b339ab/benchmark_data_storage0111-ib.pocstor_11336_object25485/head//1 by client.1312877.0:52769 2014-08-11 16:27:13.796341 2014-08-13 18:06:06.374969 7f5c973a57a0 20 read_log 192888'823 (192888'404) delete 4e638a6b/benchmark_data_storage0111-ib.pocstor_11336_object25526/head//1 by client.1312877.0:52810 2014-08-11 16:27:13.804427 2014-08-13 18:06:06.375005 7f5c973a57a0 20 read_log 192888'824 (192888'405) delete 1129b96b/benchmark_data_storage0111-ib.pocstor_11336_object25572/head//1 by client.1312877.0:52856 2014-08-11 16:27:13.815918 2014-08-13 18:06:06.375032 7f5c973a57a0 20 read_log 192888'825 (192888'406) delete 934612eb/benchmark_data_storage0111-ib.pocstor_11336_object25811/head//1 by client.1312877.0:53095 2014-08-11 16:27:13.880971 2014-08-13 18:06:06.375054 7f5c973a57a0 20 read_log 192888'826 (192888'407) delete 8a0e956b/benchmark_data_storage0111-ib.pocstor_11336_object25812/head//1 by client.1312877.0:53096 2014-08-11 16:27:13.881648 2014-08-13 18:06:06.375075 7f5c973a57a0 20 read_log 192888'827 (192888'408) delete 5275656b/benchmark_data_storage0111-ib.pocstor_11336_object25868/head//1 by client.1312877.0:53152 2014-08-11 16:27:13.895301 2014-08-13 18:06:06.375097 7f5c973a57a0 20 read_log 192888'828 (192888'409) delete 75b001eb/benchmark_data_storage0111-ib.pocstor_11336_object26161/head//1 by client.1312877.0:53445 2014-08-11 16:27:13.969101 2014-08-13 18:06:06.375118 7f5c973a57a0 20 read_log 192888'829 (192888'410) delete b69ac26b/benchmark_data_storage0111-ib.pocstor_11336_object26351/head//1 by client.1312877.0:53635 2014-08-11 16:27:14.016727 2014-08-13 18:06:06.375140 7f5c973a57a0 20 read_log 192888'830 (192888'411) delete 784d40ab/benchmark_data_storage0111-ib.pocstor_11336_object26478/head//1 by client.1312877.0:53762 2014-08-11 16:27:14.049132 2014-08-13 18:06:06.375171 7f5c973a57a0 20 read_log 192888'831 (192888'412) delete b4b81d2b/benchmark_data_storage0111-ib.pocstor_11336_object26538/head//1 by client.1312877.0:53822 2014-08-11 16:27:14.062870 2014-08-13 18:06:06.375193 7f5c973a57a0 20 read_log 192888'832 (192888'413) delete 940ab0ab/benchmark_data_storage0111-ib.pocstor_11336_object26596/head//1 by client.1312877.0:53880 2014-08-11 16:27:14.077266 2014-08-13 18:06:06.375219 7f5c973a57a0 20 read_log 192888'833 (192888'414) delete f7f71f6b/benchmark_data_storage0111-ib.pocstor_11336_object26678/head//1 by client.1312877.0:53962 2014-08-11 16:27:14.097705 2014-08-13 18:06:06.375241 7f5c973a57a0 20 read_log 192888'834 (192888'415) delete 2b79936b/benchmark_data_storage0111-ib.pocstor_11336_object26822/head//1 by client.1312877.0:54106 2014-08-11 16:27:14.132971 2014-08-13 18:06:06.375262 7f5c973a57a0 20 read_log 192888'835 (192888'416) delete 9d858b2b/benchmark_data_storage0111-ib.pocstor_11336_object26873/head//1 by client.1312877.0:54157 2014-08-11 16:27:14.144678 2014-08-13 18:06:06.375284 7f5c973a57a0 20 read_log 192888'836 (192888'417) delete d46b0eab/benchmark_data_storage0111-ib.pocstor_11336_object26900/head//1 by client.1312877.0:54184 2014-08-11 16:27:14.151013 2014-08-13 18:06:06.375310 7f5c973a57a0 20 read_log 192888'837 (192888'418) delete 3596656b/benchmark_data_storage0111-ib.pocstor_11336_object27149/head//1 by client.1312877.0:54433 2014-08-11 16:27:14.211365 2014-08-13 18:06:06.375335 7f5c973a57a0 20 read_log 192888'838 (192888'419) delete 800c4c2b/benchmark_data_storage0111-ib.pocstor_11336_object27214/head//1 by client.1312877.0:54498 2014-08-11 16:27:14.225984 2014-08-13 18:06:06.375357 7f5c973a57a0 20 read_log 194215'839 (0'0) modify 6fb923eb/benchmark_data_storage0111-ib.pocstor_19374_object0/head//1 by client.1048726.0:1 2014-08-12 17:10:00.590828 2014-08-13 18:06:06.375379 7f5c973a57a0 20 read_log 194215'840 (0'0) modify f1c09dab/benchmark_data_storage0111-ib.pocstor_19374_object53/head//1 by client.1048726.0:54 2014-08-12 17:10:00.857408 2014-08-13 18:06:06.375401 7f5c973a57a0 20 read_log 194215'841 (0'0) modify 322ce82b/benchmark_data_storage0111-ib.pocstor_19374_object55/head//1 by client.1048726.0:56 2014-08-12 17:10:00.865181 2014-08-13 18:06:06.375422 7f5c973a57a0 20 read_log 194215'842 (0'0) modify 42b2c8ab/benchmark_data_storage0111-ib.pocstor_19374_object77/head//1 by client.1048726.0:78 2014-08-12 17:10:00.949824 2014-08-13 18:06:06.375446 7f5c973a57a0 20 read_log 194215'843 (0'0) modify 3792f3eb/benchmark_data_storage0111-ib.pocstor_19374_object104/head//1 by client.1048726.0:105 2014-08-12 17:10:01.054574 2014-08-13 18:06:06.375507 7f5c973a57a0 20 read_log 194215'844 (0'0) modify c21e90ab/benchmark_data_storage0111-ib.pocstor_19374_object106/head//1 by client.1048726.0:107 2014-08-12 17:10:01.061936 2014-08-13 18:06:06.375554 7f5c973a57a0 20 read_log 194215'845 (0'0) modify f13767eb/benchmark_data_storage0111-ib.pocstor_19374_object112/head//1 by client.1048726.0:113 2014-08-12 17:10:01.084846 2014-08-13 18:06:06.375592 7f5c973a57a0 20 read_log 194215'846 (0'0) modify 90f4b5eb/benchmark_data_storage0111-ib.pocstor_19374_object117/head//1 by client.1048726.0:118 2014-08-12 17:10:01.103934 2014-08-13 18:06:06.375621 7f5c973a57a0 20 read_log 194215'847 (0'0) modify ec96382b/benchmark_data_storage0111-ib.pocstor_19374_object134/head//1 by client.1048726.0:135 2014-08-12 17:10:01.168850 2014-08-13 18:06:06.375643 7f5c973a57a0 20 read_log 194215'848 (0'0) modify 58c1d5eb/benchmark_data_storage0111-ib.pocstor_19374_object146/head//1 by client.1048726.0:147 2014-08-12 17:10:01.214706 2014-08-13 18:06:06.375664 7f5c973a57a0 20 read_log 194215'849 (0'0) modify fb95d6ab/benchmark_data_storage0111-ib.pocstor_19374_object204/head//1 by client.1048726.0:205 2014-08-12 17:10:01.451445 2014-08-13 18:06:06.375686 7f5c973a57a0 20 read_log 194215'850 (0'0) modify b112016b/benchmark_data_storage0111-ib.pocstor_19374_object273/head//1 by client.1048726.0:274 2014-08-12 17:10:01.789491 2014-08-13 18:06:06.375707 7f5c973a57a0 20 read_log 194215'851 (0'0) modify a02a832b/benchmark_data_storage0111-ib.pocstor_19374_object284/head//1 by client.1048726.0:285 2014-08-12 17:10:01.830122 2014-08-13 18:06:06.375730 7f5c973a57a0 20 read_log 194215'852 (0'0) modify a5225cab/benchmark_data_storage0111-ib.pocstor_19374_object539/head//1 by client.1048726.0:540 2014-08-12 17:10:02.825535 2014-08-13 18:06:06.375770 7f5c973a57a0 20 read_log 194215'853 (0'0) modify a60191ab/benchmark_data_storage0111-ib.pocstor_19374_object649/head//1 by client.1048726.0:650 2014-08-12 17:10:03.256870 2014-08-13 18:06:06.375792 7f5c973a57a0 20 read_log 194215'854 (0'0) modify 6fc2f02b/benchmark_data_storage0111-ib.pocstor_19374_object659/head//1 by client.1048726.0:660 2014-08-12 17:10:03.294233 2014-08-13 18:06:06.375814 7f5c973a57a0 20 read_log 194215'855 (0'0) modify 7f48662b/benchmark_data_storage0111-ib.pocstor_19374_object1055/head//1 by client.1048726.0:1056 2014-08-12 17:10:04.742733 2014-08-13 18:06:06.375836 7f5c973a57a0 20 read_log 194215'856 (0'0) modify 17650d2b/benchmark_data_storage0111-ib.pocstor_19374_object1157/head//1 by client.1048726.0:1158 2014-08-12 17:10:05.115254 2014-08-13 18:06:06.375857 7f5c973a57a0 20 read_log 194215'857 (0'0) modify 39bc27ab/benchmark_data_storage0111-ib.pocstor_19374_object1180/head//1 by client.1048726.0:1181 2014-08-12 17:10:05.209249 2014-08-13 18:06:06.375879 7f5c973a57a0 20 read_log 194215'858 (0'0) modify e51879ab/benchmark_data_storage0111-ib.pocstor_19374_object1244/head//1 by client.1048726.0:1245 2014-08-12 17:10:05.444267 2014-08-13 18:06:06.375900 7f5c973a57a0 20 read_log 194215'859 (0'0) modify fefa32eb/benchmark_data_storage0111-ib.pocstor_19374_object1265/head//1 by client.1048726.0:1266 2014-08-12 17:10:05.520797 2014-08-13 18:06:06.375922 7f5c973a57a0 20 read_log 194215'860 (0'0) modify ee0c2aab/benchmark_data_storage0111-ib.pocstor_19374_object1278/head//1 by client.1048726.0:1279 2014-08-12 17:10:05.568368 2014-08-13 18:06:06.375944 7f5c973a57a0 20 read_log 194215'861 (0'0) modify 3560e9ab/benchmark_data_storage0111-ib.pocstor_19374_object1318/head//1 by client.1048726.0:1319 2014-08-12 17:10:05.715945 2014-08-13 18:06:06.375965 7f5c973a57a0 20 read_log 194215'862 (0'0) modify 510c1a6b/benchmark_data_storage0111-ib.pocstor_19374_object1337/head//1 by client.1048726.0:1338 2014-08-12 17:10:05.784135 2014-08-13 18:06:06.375987 7f5c973a57a0 20 read_log 194215'863 (0'0) modify 2298276b/benchmark_data_storage0111-ib.pocstor_19374_object1339/head//1 by client.1048726.0:1340 2014-08-12 17:10:05.791557 2014-08-13 18:06:06.376020 7f5c973a57a0 20 read_log 194215'864 (0'0) modify 6340222b/benchmark_data_storage0111-ib.pocstor_19374_object1349/head//1 by client.1048726.0:1350 2014-08-12 17:10:05.827220 2014-08-13 18:06:06.376076 7f5c973a57a0 20 read_log 194215'865 (0'0) modify 96f019ab/benchmark_data_storage0111-ib.pocstor_19374_object1487/head//1 by client.1048726.0:1488 2014-08-12 17:10:06.346758 2014-08-13 18:06:06.376098 7f5c973a57a0 20 read_log 194215'866 (0'0) modify 309188eb/benchmark_data_storage0111-ib.pocstor_19374_object1517/head//1 by client.1048726.0:1518 2014-08-12 17:10:06.456804 2014-08-13 18:06:06.376120 7f5c973a57a0 20 read_log 194215'867 (0'0) modify 5e9b32eb/benchmark_data_storage0111-ib.pocstor_19374_object1568/head//1 by client.1048726.0:1569 2014-08-12 17:10:06.660441 2014-08-13 18:06:06.376141 7f5c973a57a0 20 read_log 194215'868 (0'0) modify bb2497ab/benchmark_data_storage0111-ib.pocstor_19374_object1632/head//1 by client.1048726.0:1633 2014-08-12 17:10:06.899760 2014-08-13 18:06:06.376172 7f5c973a57a0 20 read_log 194215'869 (0'0) modify cf5e382b/benchmark_data_storage0111-ib.pocstor_19374_object1744/head//1 by client.1048726.0:1745 2014-08-12 17:10:07.316858 2014-08-13 18:06:06.376194 7f5c973a57a0 20 read_log 194215'870 (0'0) modify 3c5c7dab/benchmark_data_storage0111-ib.pocstor_19374_object1756/head//1 by client.1048726.0:1757 2014-08-12 17:10:07.362150 2014-08-13 18:06:06.376249 7f5c973a57a0 20 read_log 194215'871 (0'0) modify 724fd5eb/benchmark_data_storage0111-ib.pocstor_19374_object1757/head//1 by client.1048726.0:1758 2014-08-12 17:10:07.365884 2014-08-13 18:06:06.376272 7f5c973a57a0 20 read_log 194215'872 (0'0) modify 912098ab/benchmark_data_storage0111-ib.pocstor_19374_object1776/head//1 by client.1048726.0:1777 2014-08-12 17:10:07.436954 2014-08-13 18:06:06.376293 7f5c973a57a0 20 read_log 194215'873 (0'0) modify 3939e16b/benchmark_data_storage0111-ib.pocstor_19374_object1894/head//1 by client.1048726.0:1895 2014-08-12 17:10:07.879247 2014-08-13 18:06:06.376323 7f5c973a57a0 20 read_log 194215'874 (0'0) modify 6a0792eb/benchmark_data_storage0111-ib.pocstor_19374_object1896/head//1 by client.1048726.0:1897 2014-08-12 17:10:07.886451 2014-08-13 18:06:06.376351 7f5c973a57a0 20 read_log 194215'875 (0'0) modify 6f099eeb/benchmark_data_storage0111-ib.pocstor_19374_object2059/head//1 by client.1048726.0:2060 2014-08-12 17:10:08.500386 2014-08-13 18:06:06.376378 7f5c973a57a0 20 read_log 194215'876 (0'0) modify eafb902b/benchmark_data_storage0111-ib.pocstor_19374_object2077/head//1 by client.1048726.0:2078 2014-08-12 17:10:08.565170 2014-08-13 18:06:06.376405 7f5c973a57a0 20 read_log 194215'877 (0'0) modify 946c36eb/benchmark_data_storage0111-ib.pocstor_19374_object2102/head//1 by client.1048726.0:2103 2014-08-12 17:10:08.657330 2014-08-13 18:06:06.376446 7f5c973a57a0 20 read_log 194215'878 (0'0) modify f47e62b/benchmark_data_storage0111-ib.pocstor_19374_object2180/head//1 by client.1048726.0:2181 2014-08-12 17:10:08.950520 2014-08-13 18:06:06.376485 7f5c973a57a0 20 read_log 194215'879 (0'0) modify 934f442b/benchmark_data_storage0111-ib.pocstor_19374_object2262/head//1 by client.1048726.0:2263 2014-08-12 17:10:09.262995 2014-08-13 18:06:06.376513 7f5c973a57a0 20 read_log 194215'880 (0'0) modify 7f57086b/benchmark_data_storage0111-ib.pocstor_19374_object2343/head//1 by client.1048726.0:2344 2014-08-12 17:10:09.563691 2014-08-13 18:06:06.376549 7f5c973a57a0 20 read_log 194215'881 (0'0) modify 13d2392b/benchmark_data_storage0111-ib.pocstor_19374_object2352/head//1 by client.1048726.0:2353 2014-08-12 17:10:09.596336 2014-08-13 18:06:06.376572 7f5c973a57a0 20 read_log 194215'882 (0'0) modify 3d536ceb/benchmark_data_storage0111-ib.pocstor_19374_object2361/head//1 by client.1048726.0:2362 2014-08-12 17:10:09.631118 2014-08-13 18:06:06.376594 7f5c973a57a0 20 read_log 194215'883 (0'0) modify c6e9f26b/benchmark_data_storage0111-ib.pocstor_19374_object2394/head//1 by client.1048726.0:2395 2014-08-12 17:10:09.751730 2014-08-13 18:06:06.376636 7f5c973a57a0 20 read_log 194215'884 (0'0) modify 85e6a52b/benchmark_data_storage0111-ib.pocstor_19374_object2418/head//1 by client.1048726.0:2419 2014-08-12 17:10:09.839253 2014-08-13 18:06:06.376663 7f5c973a57a0 20 read_log 194215'885 (0'0) modify c7cc53ab/benchmark_data_storage0111-ib.pocstor_19374_object2432/head//1 by client.1048726.0:2433 2014-08-12 17:10:09.891934 2014-08-13 18:06:06.376685 7f5c973a57a0 20 read_log 194215'886 (0'0) modify 3a7fe4ab/benchmark_data_storage0111-ib.pocstor_19374_object2594/head//1 by client.1048726.0:2595 2014-08-12 17:10:10.492847 2014-08-13 18:06:06.376711 7f5c973a57a0 20 read_log 194215'887 (0'0) modify 4305176b/benchmark_data_storage0111-ib.pocstor_19374_object2684/head//1 by client.1048726.0:2685 2014-08-12 17:10:10.840476 2014-08-13 18:06:06.376734 7f5c973a57a0 20 read_log 194215'888 (0'0) modify e7b7062b/benchmark_data_storage0111-ib.pocstor_19374_object2725/head//1 by client.1048726.0:2726 2014-08-12 17:10:10.991411 2014-08-13 18:06:06.376776 7f5c973a57a0 20 read_log 194215'889 (0'0) modify a1e105ab/benchmark_data_storage0111-ib.pocstor_19374_object2744/head//1 by client.1048726.0:2745 2014-08-12 17:10:11.061577 2014-08-13 18:06:06.376799 7f5c973a57a0 20 read_log 194215'890 (0'0) modify 9bee462b/benchmark_data_storage0111-ib.pocstor_19374_object2763/head//1 by client.1048726.0:2764 2014-08-12 17:10:11.131220 2014-08-13 18:06:06.376866 7f5c973a57a0 20 read_log 194215'891 (0'0) modify 2d10016b/benchmark_data_storage0111-ib.pocstor_19374_object2791/head//1 by client.1048726.0:2792 2014-08-12 17:10:11.234844 2014-08-13 18:06:06.376898 7f5c973a57a0 20 read_log 194215'892 (0'0) modify 7464b42b/benchmark_data_storage0111-ib.pocstor_19374_object2865/head//1 by client.1048726.0:2866 2014-08-12 17:10:11.527280 2014-08-13 18:06:06.376920 7f5c973a57a0 20 read_log 194215'893 (0'0) modify 6a28beb/benchmark_data_storage0111-ib.pocstor_19374_object2901/head//1 by client.1048726.0:2902 2014-08-12 17:10:11.661259 2014-08-13 18:06:06.376942 7f5c973a57a0 20 read_log 194215'894 (0'0) modify df15306b/benchmark_data_storage0111-ib.pocstor_19374_object2944/head//1 by client.1048726.0:2945 2014-08-12 17:10:11.835788 2014-08-13 18:06:06.376968 7f5c973a57a0 20 read_log 194215'895 (0'0) modify 505175ab/benchmark_data_storage0111-ib.pocstor_19374_object2950/head//1 by client.1048726.0:2951 2014-08-12 17:10:11.857698 2014-08-13 18:06:06.376989 7f5c973a57a0 20 read_log 194215'896 (0'0) modify 3fa3deeb/benchmark_data_storage0111-ib.pocstor_19374_object3045/head//1 by client.1048726.0:3046 2014-08-12 17:10:12.212259 2014-08-13 18:06:06.377015 7f5c973a57a0 20 read_log 194215'897 (0'0) modify f1bcb3ab/benchmark_data_storage0111-ib.pocstor_19374_object3073/head//1 by client.1048726.0:3074 2014-08-12 17:10:12.318267 2014-08-13 18:06:06.377036 7f5c973a57a0 20 read_log 194215'898 (0'0) modify 6eb5e6ab/benchmark_data_storage0111-ib.pocstor_19374_object3178/head//1 by client.1048726.0:3179 2014-08-12 17:10:12.723788 2014-08-13 18:06:06.377063 7f5c973a57a0 20 read_log 194215'899 (0'0) modify 71213f6b/benchmark_data_storage0111-ib.pocstor_19374_object3222/head//1 by client.1048726.0:3223 2014-08-12 17:10:12.883970 2014-08-13 18:06:06.377108 7f5c973a57a0 20 read_log 194215'900 (0'0) modify 2847d6ab/benchmark_data_storage0111-ib.pocstor_19374_object3286/head//1 by client.1048726.0:3287 2014-08-12 17:10:13.119342 2014-08-13 18:06:06.377131 7f5c973a57a0 20 read_log 194215'901 (0'0) modify b38c2f6b/benchmark_data_storage0111-ib.pocstor_19374_object3415/head//1 by client.1048726.0:3416 2014-08-12 17:10:13.609279 2014-08-13 18:06:06.377161 7f5c973a57a0 20 read_log 194215'902 (0'0) modify 518a656b/benchmark_data_storage0111-ib.pocstor_19374_object3454/head//1 by client.1048726.0:3455 2014-08-12 17:10:13.755288 2014-08-13 18:06:06.377184 7f5c973a57a0 20 read_log 194215'903 (0'0) modify cad818ab/benchmark_data_storage0111-ib.pocstor_19374_object3466/head//1 by client.1048726.0:3467 2014-08-12 17:10:13.799432 2014-08-13 18:06:06.377217 7f5c973a57a0 20 read_log 194215'904 (0'0) modify 52ed8aab/benchmark_data_storage0111-ib.pocstor_19374_object3511/head//1 by client.1048726.0:3512 2014-08-12 17:10:13.975505 2014-08-13 18:06:06.377259 7f5c973a57a0 20 read_log 194215'905 (0'0) modify 376b872b/benchmark_data_storage0111-ib.pocstor_19374_object3523/head//1 by client.1048726.0:3524 2014-08-12 17:10:14.018984 2014-08-13 18:06:06.377284 7f5c973a57a0 20 read_log 194215'906 (0'0) modify 69cdf92b/benchmark_data_storage0111-ib.pocstor_19374_object3584/head//1 by client.1048726.0:3585 2014-08-12 17:10:14.248686 2014-08-13 18:06:06.377315 7f5c973a57a0 20 read_log 194215'907 (0'0) modify 5dad4cab/benchmark_data_storage0111-ib.pocstor_19374_object3612/head//1 by client.1048726.0:3613 2014-08-12 17:10:14.351100 2014-08-13 18:06:06.377337 7f5c973a57a0 20 read_log 194215'908 (0'0) modify 8a29ddeb/benchmark_data_storage0111-ib.pocstor_19374_object3677/head//1 by client.1048726.0:3678 2014-08-12 17:10:14.592968 2014-08-13 18:06:06.377372 7f5c973a57a0 20 read_log 194215'909 (0'0) modify cb77c3ab/benchmark_data_storage0111-ib.pocstor_19374_object3717/head//1 by client.1048726.0:3718 2014-08-12 17:10:14.742560 2014-08-13 18:06:06.377401 7f5c973a57a0 20 read_log 194215'910 (0'0) modify 7096f52b/benchmark_data_storage0111-ib.pocstor_19374_object3810/head//1 by client.1048726.0:3811 2014-08-12 17:10:15.096844 2014-08-13 18:06:06.377423 7f5c973a57a0 20 read_log 194215'911 (0'0) modify ee1fffeb/benchmark_data_storage0111-ib.pocstor_19374_object3837/head//1 by client.1048726.0:3838 2014-08-12 17:10:15.195941 2014-08-13 18:06:06.377444 7f5c973a57a0 20 read_log 194215'912 (0'0) modify a32a26eb/benchmark_data_storage0111-ib.pocstor_19374_object3838/head//1 by client.1048726.0:3839 2014-08-12 17:10:15.199764 2014-08-13 18:06:06.377465 7f5c973a57a0 20 read_log 194215'913 (0'0) modify 6fb1deeb/benchmark_data_storage0111-ib.pocstor_19374_object4048/head//1 by client.1048726.0:4049 2014-08-12 17:10:15.978347 2014-08-13 18:06:06.377486 7f5c973a57a0 20 read_log 194215'914 (0'0) modify 6c58c7ab/benchmark_data_storage0111-ib.pocstor_19374_object4059/head//1 by client.1048726.0:4060 2014-08-12 17:10:16.027647 2014-08-13 18:06:06.377508 7f5c973a57a0 20 read_log 194215'915 (0'0) modify 660f996b/benchmark_data_storage0111-ib.pocstor_19374_object4105/head//1 by client.1048726.0:4106 2014-08-12 17:10:16.233117 2014-08-13 18:06:06.377529 7f5c973a57a0 20 read_log 194215'916 (0'0) modify 612f78ab/benchmark_data_storage0111-ib.pocstor_19374_object4125/head//1 by client.1048726.0:4126 2014-08-12 17:10:16.346790 2014-08-13 18:06:06.377550 7f5c973a57a0 20 read_log 194215'917 (0'0) modify bd30982b/benchmark_data_storage0111-ib.pocstor_19374_object4136/head//1 by client.1048726.0:4137 2014-08-12 17:10:16.387277 2014-08-13 18:06:06.377571 7f5c973a57a0 20 read_log 194215'918 (0'0) modify d1ec98ab/benchmark_data_storage0111-ib.pocstor_19374_object4183/head//1 by client.1048726.0:4184 2014-08-12 17:10:16.564420 2014-08-13 18:06:06.377592 7f5c973a57a0 20 read_log 194215'919 (0'0) modify cd9ad76b/benchmark_data_storage0111-ib.pocstor_19374_object4187/head//1 by client.1048726.0:4188 2014-08-12 17:10:16.579161 2014-08-13 18:06:06.377613 7f5c973a57a0 20 read_log 194215'920 (0'0) modify 6236346b/benchmark_data_storage0111-ib.pocstor_19374_object4191/head//1 by client.1048726.0:4192 2014-08-12 17:10:16.594033 2014-08-13 18:06:06.377635 7f5c973a57a0 20 read_log 194215'921 (0'0) modify c47eb16b/benchmark_data_storage0111-ib.pocstor_19374_object4242/head//1 by client.1048726.0:4243 2014-08-12 17:10:16.780918 2014-08-13 18:06:06.377656 7f5c973a57a0 20 read_log 194215'922 (0'0) modify 7f4a59eb/benchmark_data_storage0111-ib.pocstor_19374_object4360/head//1 by client.1048726.0:4361 2014-08-12 17:10:17.287713 2014-08-13 18:06:06.377677 7f5c973a57a0 20 read_log 194215'923 (0'0) modify 3ac7842b/benchmark_data_storage0111-ib.pocstor_19374_object4440/head//1 by client.1048726.0:4441 2014-08-12 17:10:17.867496 2014-08-13 18:06:06.377713 7f5c973a57a0 20 read_log 194215'924 (0'0) modify 3be00eeb/benchmark_data_storage0111-ib.pocstor_19374_object4498/head//1 by client.1048726.0:4499 2014-08-12 17:10:18.103629 2014-08-13 18:06:06.377740 7f5c973a57a0 20 read_log 194215'925 (0'0) modify 20baeeab/benchmark_data_storage0111-ib.pocstor_19374_object4674/head//1 by client.1048726.0:4675 2014-08-12 17:10:18.772248 2014-08-13 18:06:06.377762 7f5c973a57a0 20 read_log 194215'926 (0'0) modify ae2e9d2b/benchmark_data_storage0111-ib.pocstor_19374_object4826/head//1 by client.1048726.0:4827 2014-08-12 17:10:19.348107 2014-08-13 18:06:06.377783 7f5c973a57a0 20 read_log 194215'927 (0'0) modify e1d72b2b/benchmark_data_storage0111-ib.pocstor_19374_object4840/head//1 by client.1048726.0:4841 2014-08-12 17:10:19.416924 2014-08-13 18:06:06.377804 7f5c973a57a0 20 read_log 194215'928 (0'0) modify 14b2daeb/benchmark_data_storage0111-ib.pocstor_19374_object4851/head//1 by client.1048726.0:4852 2014-08-12 17:10:19.467367 2014-08-13 18:06:06.377825 7f5c973a57a0 20 read_log 194215'929 (0'0) modify f058a86b/benchmark_data_storage0111-ib.pocstor_19374_object4891/head//1 by client.1048726.0:4892 2014-08-12 17:10:19.634432 2014-08-13 18:06:06.377847 7f5c973a57a0 20 read_log 194215'930 (0'0) modify 1cc0022b/benchmark_data_storage0111-ib.pocstor_19374_object4938/head//1 by client.1048726.0:4939 2014-08-12 17:10:19.950373 2014-08-13 18:06:06.377868 7f5c973a57a0 20 read_log 194215'931 (0'0) modify add0cc2b/benchmark_data_storage0111-ib.pocstor_19374_object5085/head//1 by client.1048726.0:5086 2014-08-12 17:10:20.624221 2014-08-13 18:06:06.377889 7f5c973a57a0 20 read_log 194215'932 (0'0) modify 942a0e2b/benchmark_data_storage0111-ib.pocstor_19374_object5225/head//1 by client.1048726.0:5226 2014-08-12 17:10:21.444376 2014-08-13 18:06:06.377910 7f5c973a57a0 20 read_log 194215'933 (0'0) modify b41983eb/benchmark_data_storage0111-ib.pocstor_19374_object5345/head//1 by client.1048726.0:5346 2014-08-12 17:10:22.029188 2014-08-13 18:06:06.377931 7f5c973a57a0 20 read_log 194215'934 (0'0) modify 6f06a4eb/benchmark_data_storage0111-ib.pocstor_19374_object5356/head//1 by client.1048726.0:5357 2014-08-12 17:10:22.069586 2014-08-13 18:06:06.377953 7f5c973a57a0 20 read_log 194215'935 (0'0) modify a176b/benchmark_data_storage0111-ib.pocstor_19374_object5547/head//1 by client.1048726.0:5548 2014-08-12 17:10:22.811798 2014-08-13 18:06:06.377974 7f5c973a57a0 20 read_log 194215'936 (0'0) modify 2bbd78ab/benchmark_data_storage0111-ib.pocstor_19374_object5562/head//1 by client.1048726.0:5563 2014-08-12 17:10:22.870801 2014-08-13 18:06:06.377995 7f5c973a57a0 20 read_log 194215'937 (0'0) modify 54ab51eb/benchmark_data_storage0111-ib.pocstor_19374_object5571/head//1 by client.1048726.0:5572 2014-08-12 17:10:22.917778 2014-08-13 18:06:06.378016 7f5c973a57a0 20 read_log 194215'938 (0'0) modify 53bf81eb/benchmark_data_storage0111-ib.pocstor_19374_object5590/head//1 by client.1048726.0:5591 2014-08-12 17:10:22.989437 2014-08-13 18:06:06.378040 7f5c973a57a0 20 read_log 194215'939 (0'0) modify 93b6132b/benchmark_data_storage0111-ib.pocstor_19374_object5651/head//1 by client.1048726.0:5652 2014-08-12 17:10:23.226343 2014-08-13 18:06:06.378065 7f5c973a57a0 20 read_log 194215'940 (0'0) modify 2f4423eb/benchmark_data_storage0111-ib.pocstor_19374_object5849/head//1 by client.1048726.0:5850 2014-08-12 17:10:24.092365 2014-08-13 18:06:06.378086 7f5c973a57a0 20 read_log 194215'941 (0'0) modify 37100d2b/benchmark_data_storage0111-ib.pocstor_19374_object5900/head//1 by client.1048726.0:5901 2014-08-12 17:10:24.281509 2014-08-13 18:06:06.378111 7f5c973a57a0 20 read_log 194215'942 (0'0) modify 2dd007ab/benchmark_data_storage0111-ib.pocstor_19374_object5917/head//1 by client.1048726.0:5918 2014-08-12 17:10:24.344833 2014-08-13 18:06:06.378133 7f5c973a57a0 20 read_log 194215'943 (0'0) modify 86a4fcab/benchmark_data_storage0111-ib.pocstor_19374_object5945/head//1 by client.1048726.0:5946 2014-08-12 17:10:24.458547 2014-08-13 18:06:06.378212 7f5c973a57a0 20 read_log 194215'944 (0'0) modify 920492b/benchmark_data_storage0111-ib.pocstor_19374_object5965/head//1 by client.1048726.0:5966 2014-08-12 17:10:24.532532 2014-08-13 18:06:06.378241 7f5c973a57a0 20 read_log 194215'945 (0'0) modify 1055bb2b/benchmark_data_storage0111-ib.pocstor_19374_object5969/head//1 by client.1048726.0:5970 2014-08-12 17:10:24.547212 2014-08-13 18:06:06.378263 7f5c973a57a0 20 read_log 194215'946 (0'0) modify 58e49beb/benchmark_data_storage0111-ib.pocstor_19374_object6067/head//1 by client.1048726.0:6068 2014-08-12 17:10:25.363336 2014-08-13 18:06:06.378284 7f5c973a57a0 20 read_log 194215'947 (0'0) modify f187cf6b/benchmark_data_storage0111-ib.pocstor_19374_object6085/head//1 by client.1048726.0:6086 2014-08-12 17:10:25.533802 2014-08-13 18:06:06.378311 7f5c973a57a0 20 read_log 194215'948 (0'0) modify 700ddfab/benchmark_data_storage0111-ib.pocstor_19374_object6199/head//1 by client.1048726.0:6200 2014-08-12 17:10:26.633810 2014-08-13 18:06:06.378332 7f5c973a57a0 20 read_log 194215'949 (0'0) modify fd0e98ab/benchmark_data_storage0111-ib.pocstor_19374_object6210/head//1 by client.1048726.0:6211 2014-08-12 17:10:26.685672 2014-08-13 18:06:06.378360 7f5c973a57a0 20 read_log 194215'950 (0'0) modify b470446b/benchmark_data_storage0111-ib.pocstor_19374_object6353/head//1 by client.1048726.0:6354 2014-08-12 17:10:27.231921 2014-08-13 18:06:06.378382 7f5c973a57a0 20 read_log 194215'951 (0'0) modify e85327ab/benchmark_data_storage0111-ib.pocstor_19374_object6362/head//1 by client.1048726.0:6363 2014-08-12 17:10:27.264520 2014-08-13 18:06:06.378409 7f5c973a57a0 20 read_log 194215'952 (0'0) modify e40363ab/benchmark_data_storage0111-ib.pocstor_19374_object6440/head//1 by client.1048726.0:6441 2014-08-12 17:10:27.563351 2014-08-13 18:06:06.378450 7f5c973a57a0 20 read_log 194215'953 (0'0) modify 2dd5496b/benchmark_data_storage0111-ib.pocstor_19374_object6446/head//1 by client.1048726.0:6447 2014-08-12 17:10:27.596203 2014-08-13 18:06:06.378474 7f5c973a57a0 20 read_log 194215'954 (0'0) modify 75bccceb/benchmark_data_storage0111-ib.pocstor_19374_object6497/head//1 by client.1048726.0:6498 2014-08-12 17:10:27.784276 2014-08-13 18:06:06.378500 7f5c973a57a0 20 read_log 194215'955 (0'0) modify df89832b/benchmark_data_storage0111-ib.pocstor_19374_object6513/head//1 by client.1048726.0:6514 2014-08-12 17:10:27.854578 2014-08-13 18:06:06.378539 7f5c973a57a0 20 read_log 194215'956 (0'0) modify f02cbe6b/benchmark_data_storage0111-ib.pocstor_19374_object6524/head//1 by client.1048726.0:6525 2014-08-12 17:10:27.895191 2014-08-13 18:06:06.378573 7f5c973a57a0 20 read_log 194215'957 (0'0) modify a3faf6b/benchmark_data_storage0111-ib.pocstor_19374_object6613/head//1 by client.1048726.0:6614 2014-08-12 17:10:28.323216 2014-08-13 18:06:06.378640 7f5c973a57a0 20 read_log 194215'958 (0'0) modify ff3983eb/benchmark_data_storage0111-ib.pocstor_19374_object6657/head//1 by client.1048726.0:6658 2014-08-12 17:10:28.558897 2014-08-13 18:06:06.378684 7f5c973a57a0 20 read_log 194215'959 (0'0) modify 1f301fab/benchmark_data_storage0111-ib.pocstor_19374_object6816/head//1 by client.1048726.0:6817 2014-08-12 17:10:29.158546 2014-08-13 18:06:06.378712 7f5c973a57a0 20 read_log 194215'960 (0'0) modify f6de92eb/benchmark_data_storage0111-ib.pocstor_19374_object6841/head//1 by client.1048726.0:6842 2014-08-12 17:10:29.251845 2014-08-13 18:06:06.378734 7f5c973a57a0 20 read_log 194215'961 (0'0) modify b6c1042b/benchmark_data_storage0111-ib.pocstor_19374_object6885/head//1 by client.1048726.0:6886 2014-08-12 17:10:29.420715 2014-08-13 18:06:06.378756 7f5c973a57a0 20 read_log 194215'962 (0'0) modify af36f9eb/benchmark_data_storage0111-ib.pocstor_19374_object6904/head//1 by client.1048726.0:6905 2014-08-12 17:10:29.490280 2014-08-13 18:06:06.378777 7f5c973a57a0 20 read_log 194215'963 (0'0) modify a5777ab/benchmark_data_storage0111-ib.pocstor_19374_object7029/head//1 by client.1048726.0:7030 2014-08-12 17:10:29.950996 2014-08-13 18:06:06.378831 7f5c973a57a0 20 read_log 194215'964 (0'0) modify 6919b6b/benchmark_data_storage0111-ib.pocstor_19374_object7059/head//1 by client.1048726.0:7060 2014-08-12 17:10:30.060800 2014-08-13 18:06:06.378862 7f5c973a57a0 20 read_log 194215'965 (0'0) modify efbee52b/benchmark_data_storage0111-ib.pocstor_19374_object7069/head//1 by client.1048726.0:7070 2014-08-12 17:10:30.097601 2014-08-13 18:06:06.378884 7f5c973a57a0 20 read_log 194215'966 (0'0) modify 5f8c13eb/benchmark_data_storage0111-ib.pocstor_19374_object7102/head//1 by client.1048726.0:7103 2014-08-12 17:10:30.219407 2014-08-13 18:06:06.378905 7f5c973a57a0 20 read_log 194215'967 (0'0) modify 651b9fab/benchmark_data_storage0111-ib.pocstor_19374_object7248/head//1 by client.1048726.0:7249 2014-08-12 17:10:30.761181 2014-08-13 18:06:06.378927 7f5c973a57a0 20 read_log 194215'968 (0'0) modify 4698106b/benchmark_data_storage0111-ib.pocstor_19374_object7302/head//1 by client.1048726.0:7303 2014-08-12 17:10:30.968787 2014-08-13 18:06:06.378948 7f5c973a57a0 20 read_log 194215'969 (0'0) modify 95b2bd2b/benchmark_data_storage0111-ib.pocstor_19374_object7355/head//1 by client.1048726.0:7356 2014-08-12 17:10:31.166111 2014-08-13 18:06:06.378969 7f5c973a57a0 20 read_log 194215'970 (0'0) modify 3565086b/benchmark_data_storage0111-ib.pocstor_19374_object7394/head//1 by client.1048726.0:7395 2014-08-12 17:10:31.311296 2014-08-13 18:06:06.378990 7f5c973a57a0 20 read_log 194215'971 (0'0) modify 7723e2eb/benchmark_data_storage0111-ib.pocstor_19374_object7432/head//1 by client.1048726.0:7433 2014-08-12 17:10:31.451457 2014-08-13 18:06:06.379012 7f5c973a57a0 20 read_log 194215'972 (0'0) modify 3876f66b/benchmark_data_storage0111-ib.pocstor_19374_object7456/head//1 by client.1048726.0:7457 2014-08-12 17:10:31.553376 2014-08-13 18:06:06.379033 7f5c973a57a0 20 read_log 194215'973 (0'0) modify 7c404ceb/benchmark_data_storage0111-ib.pocstor_19374_object7471/head//1 by client.1048726.0:7472 2014-08-12 17:10:31.608965 2014-08-13 18:06:06.379054 7f5c973a57a0 20 read_log 194215'974 (0'0) modify dee414ab/benchmark_data_storage0111-ib.pocstor_19374_object7498/head//1 by client.1048726.0:7499 2014-08-12 17:10:31.708966 2014-08-13 18:06:06.379076 7f5c973a57a0 20 read_log 194215'975 (0'0) modify b0b328ab/benchmark_data_storage0111-ib.pocstor_19374_object7503/head//1 by client.1048726.0:7504 2014-08-12 17:10:31.727589 2014-08-13 18:06:06.379097 7f5c973a57a0 20 read_log 194215'976 (0'0) modify fbaf77ab/benchmark_data_storage0111-ib.pocstor_19374_object7653/head//1 by client.1048726.0:7654 2014-08-12 17:10:32.300647 2014-08-13 18:06:06.379119 7f5c973a57a0 20 read_log 194215'977 (0'0) modify f080c82b/benchmark_data_storage0111-ib.pocstor_19374_object7664/head//1 by client.1048726.0:7665 2014-08-12 17:10:32.353688 2014-08-13 18:06:06.379140 7f5c973a57a0 20 read_log 194215'978 (0'0) modify cf53d6eb/benchmark_data_storage0111-ib.pocstor_19374_object7819/head//1 by client.1048726.0:7820 2014-08-12 17:10:33.004816 2014-08-13 18:06:06.379170 7f5c973a57a0 20 read_log 194215'979 (0'0) modify 7e7fc76b/benchmark_data_storage0111-ib.pocstor_19374_object7822/head//1 by client.1048726.0:7823 2014-08-12 17:10:33.016213 2014-08-13 18:06:06.379192 7f5c973a57a0 20 read_log 194215'980 (0'0) modify fa90ac2b/benchmark_data_storage0111-ib.pocstor_19374_object7839/head//1 by client.1048726.0:7840 2014-08-12 17:10:33.090661 2014-08-13 18:06:06.379214 7f5c973a57a0 20 read_log 194215'981 (0'0) modify 5918ea2b/benchmark_data_storage0111-ib.pocstor_19374_object7899/head//1 by client.1048726.0:7900 2014-08-12 17:10:33.314113 2014-08-13 18:06:06.379235 7f5c973a57a0 20 read_log 194215'982 (0'0) modify 4b04ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8097/head//1 by client.1048726.0:8098 2014-08-12 17:10:34.059672 2014-08-13 18:06:06.379256 7f5c973a57a0 20 read_log 194215'983 (0'0) modify 3b37c4ab/benchmark_data_storage0111-ib.pocstor_19374_object8171/head//1 by client.1048726.0:8172 2014-08-12 17:10:34.335508 2014-08-13 18:06:06.379331 7f5c973a57a0 20 read_log 194215'984 (0'0) modify fabb5dab/benchmark_data_storage0111-ib.pocstor_19374_object8252/head//1 by client.1048726.0:8253 2014-08-12 17:10:34.635614 2014-08-13 18:06:06.379361 7f5c973a57a0 20 read_log 194215'985 (0'0) modify be4caceb/benchmark_data_storage0111-ib.pocstor_19374_object8295/head//1 by client.1048726.0:8296 2014-08-12 17:10:34.807030 2014-08-13 18:06:06.379388 7f5c973a57a0 20 read_log 194215'986 (0'0) modify 95c4ed2b/benchmark_data_storage0111-ib.pocstor_19374_object8373/head//1 by client.1048726.0:8374 2014-08-12 17:10:35.105503 2014-08-13 18:06:06.379410 7f5c973a57a0 20 read_log 194215'987 (0'0) modify bfddf56b/benchmark_data_storage0111-ib.pocstor_19374_object8440/head//1 by client.1048726.0:8441 2014-08-12 17:10:35.355647 2014-08-13 18:06:06.379437 7f5c973a57a0 20 read_log 194215'988 (0'0) modify 63bb63eb/benchmark_data_storage0111-ib.pocstor_19374_object8570/head//1 by client.1048726.0:8571 2014-08-12 17:10:35.944718 2014-08-13 18:06:06.379458 7f5c973a57a0 20 read_log 194215'989 (0'0) modify 819622eb/benchmark_data_storage0111-ib.pocstor_19374_object8723/head//1 by client.1048726.0:8724 2014-08-12 17:10:36.530742 2014-08-13 18:06:06.379484 7f5c973a57a0 20 read_log 194215'990 (0'0) modify 872863eb/benchmark_data_storage0111-ib.pocstor_19374_object8755/head//1 by client.1048726.0:8756 2014-08-12 17:10:36.649383 2014-08-13 18:06:06.379530 7f5c973a57a0 20 read_log 194215'991 (0'0) modify ba99d16b/benchmark_data_storage0111-ib.pocstor_19374_object8788/head//1 by client.1048726.0:8789 2014-08-12 17:10:36.773391 2014-08-13 18:06:06.379559 7f5c973a57a0 20 read_log 194215'992 (0'0) modify 89e660ab/benchmark_data_storage0111-ib.pocstor_19374_object8796/head//1 by client.1048726.0:8797 2014-08-12 17:10:36.802335 2014-08-13 18:06:06.379594 7f5c973a57a0 20 read_log 194215'993 (0'0) modify 3d2f22b/benchmark_data_storage0111-ib.pocstor_19374_object8882/head//1 by client.1048726.0:8883 2014-08-12 17:10:37.125816 2014-08-13 18:06:06.379628 7f5c973a57a0 20 read_log 194215'994 (0'0) modify 5366ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8986/head//1 by client.1048726.0:8987 2014-08-12 17:10:37.553142 2014-08-13 18:06:06.379655 7f5c973a57a0 20 read_log 194215'995 (0'0) modify e3a50aab/benchmark_data_storage0111-ib.pocstor_19374_object9008/head//1 by client.1048726.0:9009 2014-08-12 17:10:37.667724 2014-08-13 18:06:06.379677 7f5c973a57a0 20 read_log 194215'996 (0'0) modify 96aab4eb/benchmark_data_storage0111-ib.pocstor_19374_object9145/head//1 by client.1048726.0:9146 2014-08-12 17:10:38.188224 2014-08-13 18:06:06.379705 7f5c973a57a0 20 read_log 194215'997 (0'0) modify 14f1502b/benchmark_data_storage0111-ib.pocstor_19374_object9193/head//1 by client.1048726.0:9194 2014-08-12 17:10:38.366272 2014-08-13 18:06:06.379731 7f5c973a57a0 20 read_log 194215'998 (0'0) modify 6a91646b/benchmark_data_storage0111-ib.pocstor_19374_object9229/head//1 by client.1048726.0:9230 2014-08-12 17:10:38.505359 2014-08-13 18:06:06.379757 7f5c973a57a0 20 read_log 194215'999 (0'0) modify c93002eb/benchmark_data_storage0111-ib.pocstor_19374_object9287/head//1 by client.1048726.0:9288 2014-08-12 17:10:38.720418 2014-08-13 18:06:06.379779 7f5c973a57a0 20 read_log 194215'1000 (0'0) modify 2257b96b/benchmark_data_storage0111-ib.pocstor_19374_object9441/head//1 by client.1048726.0:9442 2014-08-12 17:10:39.330165 2014-08-13 18:06:06.379820 7f5c973a57a0 20 read_log 194215'1001 (0'0) modify 33077deb/benchmark_data_storage0111-ib.pocstor_19374_object9442/head//1 by client.1048726.0:9443 2014-08-12 17:10:39.333815 2014-08-13 18:06:06.379853 7f5c973a57a0 20 read_log 194215'1002 (0'0) modify e2ce132b/benchmark_data_storage0111-ib.pocstor_19374_object9679/head//1 by client.1048726.0:9680 2014-08-12 17:10:40.321794 2014-08-13 18:06:06.379879 7f5c973a57a0 20 read_log 194215'1003 (0'0) modify 9ce657ab/benchmark_data_storage0111-ib.pocstor_19374_object9686/head//1 by client.1048726.0:9687 2014-08-12 17:10:40.353095 2014-08-13 18:06:06.379918 7f5c973a57a0 20 read_log 194215'1004 (0'0) modify a535d1ab/benchmark_data_storage0111-ib.pocstor_19374_object9692/head//1 by client.1048726.0:9693 2014-08-12 17:10:40.382097 2014-08-13 18:06:06.379963 7f5c973a57a0 20 read_log 194215'1005 (0'0) modify 3ad652eb/benchmark_data_storage0111-ib.pocstor_19374_object9703/head//1 by client.1048726.0:9704 2014-08-12 17:10:40.443125 2014-08-13 18:06:06.379990 7f5c973a57a0 20 read_log 194215'1006 (0'0) modify 2daa772b/benchmark_data_storage0111-ib.pocstor_19374_object9929/head//1 by client.1048726.0:9930 2014-08-12 17:10:41.728624 2014-08-13 18:06:06.380012 7f5c973a57a0 20 read_log 194215'1007 (0'0) modify 1659f86b/benchmark_data_storage0111-ib.pocstor_19374_object9992/head//1 by client.1048726.0:9993 2014-08-12 17:10:42.087794 2014-08-13 18:06:06.380034 7f5c973a57a0 20 read_log 194215'1008 (0'0) modify 50cd18ab/benchmark_data_storage0111-ib.pocstor_19374_object10038/head//1 by client.1048726.0:10039 2014-08-12 17:10:42.327808 2014-08-13 18:06:06.380056 7f5c973a57a0 20 read_log 194215'1009 (0'0) modify 57ad5fab/benchmark_data_storage0111-ib.pocstor_19374_object10124/head//1 by client.1048726.0:10125 2014-08-12 17:10:42.748778 2014-08-13 18:06:06.380078 7f5c973a57a0 20 read_log 194215'1010 (0'0) modify 20113dab/benchmark_data_storage0111-ib.pocstor_19374_object10313/head//1 by client.1048726.0:10314 2014-08-12 17:10:43.554899 2014-08-13 18:06:06.380120 7f5c973a57a0 20 read_log 194215'1011 (0'0) modify fc13012b/benchmark_data_storage0111-ib.pocstor_19374_object10319/head//1 by client.1048726.0:10320 2014-08-12 17:10:43.577183 2014-08-13 18:06:06.380143 7f5c973a57a0 20 read_log 194215'1012 (0'0) modify 55f420ab/benchmark_data_storage0111-ib.pocstor_19374_object10414/head//1 by client.1048726.0:10415 2014-08-12 17:10:43.973605 2014-08-13 18:06:06.380177 7f5c973a57a0 20 read_log 194215'1013 (0'0) modify 44b26d2b/benchmark_data_storage0111-ib.pocstor_19374_object10437/head//1 by client.1048726.0:10438 2014-08-12 17:10:44.060397 2014-08-13 18:06:06.380202 7f5c973a57a0 20 read_log 194215'1014 (0'0) modify 6c33baab/benchmark_data_storage0111-ib.pocstor_19374_object10493/head//1 by client.1048726.0:10494 2014-08-12 17:10:44.265021 2014-08-13 18:06:06.380224 7f5c973a57a0 20 read_log 194215'1015 (0'0) modify 56652dab/benchmark_data_storage0111-ib.pocstor_19374_object10603/head//1 by client.1048726.0:10604 2014-08-12 17:10:44.841636 2014-08-13 18:06:06.380249 7f5c973a57a0 20 read_log 194215'1016 (0'0) modify 7aabe3eb/benchmark_data_storage0111-ib.pocstor_19374_object10674/head//1 by client.1048726.0:10675 2014-08-12 17:10:45.204485 2014-08-13 18:06:06.380271 7f5c973a57a0 20 read_log 194215'1017 (0'0) modify bdbfb66b/benchmark_data_storage0111-ib.pocstor_19374_object10688/head//1 by client.1048726.0:10689 2014-08-12 17:10:45.264289 2014-08-13 18:06:06.380293 7f5c973a57a0 20 read_log 194215'1018 (0'0) modify 9f6f4eeb/benchmark_data_storage0111-ib.pocstor_19374_object10736/head//1 by client.1048726.0:10737 2014-08-12 17:10:45.459936 2014-08-13 18:06:06.380315 7f5c973a57a0 20 read_log 194215'1019 (0'0) modify 55cd0b2b/benchmark_data_storage0111-ib.pocstor_19374_object10796/head//1 by client.1048726.0:10797 2014-08-12 17:10:45.734965 2014-08-13 18:06:06.380345 7f5c973a57a0 20 read_log 194215'1020 (0'0) modify 190a722b/benchmark_data_storage0111-ib.pocstor_19374_object10846/head//1 by client.1048726.0:10847 2014-08-12 17:10:45.932635 2014-08-13 18:06:06.380367 7f5c973a57a0 20 read_log 194215'1021 (0'0) modify 960912b/benchmark_data_storage0111-ib.pocstor_19374_object10955/head//1 by client.1048726.0:10956 2014-08-12 17:10:46.748083 2014-08-13 18:06:06.380432 7f5c973a57a0 20 read_log 194215'1022 (0'0) modify fcdecfeb/benchmark_data_storage0111-ib.pocstor_19374_object11112/head//1 by client.1048726.0:11113 2014-08-12 17:10:47.539122 2014-08-13 18:06:06.380477 7f5c973a57a0 20 read_log 194215'1023 (0'0) modify d5fcf32b/benchmark_data_storage0111-ib.pocstor_19374_object11303/head//1 by client.1048726.0:11304 2014-08-12 17:10:48.536652 2014-08-13 18:06:06.380570 7f5c973a57a0 20 read_log 194215'1024 (0'0) modify 75ac9eb/benchmark_data_storage0111-ib.pocstor_19374_object11344/head//1 by client.1048726.0:11345 2014-08-12 17:10:48.702631 2014-08-13 18:06:06.380601 7f5c973a57a0 20 read_log 194215'1025 (0'0) modify 3bda6ab/benchmark_data_storage0111-ib.pocstor_19374_object11393/head//1 by client.1048726.0:11394 2014-08-12 17:10:48.899216 2014-08-13 18:06:06.380623 7f5c973a57a0 20 read_log 194215'1026 (0'0) modify 940c086b/benchmark_data_storage0111-ib.pocstor_19374_object11474/head//1 by client.1048726.0:11475 2014-08-12 17:10:49.253261 2014-08-13 18:06:06.380647 7f5c973a57a0 20 read_log 194215'1027 (0'0) modify 316ace6b/benchmark_data_storage0111-ib.pocstor_19374_object11516/head//1 by client.1048726.0:11517 2014-08-12 17:10:49.420810 2014-08-13 18:06:06.380669 7f5c973a57a0 20 read_log 194215'1028 (0'0) modify dd29fdeb/benchmark_data_storage0111-ib.pocstor_19374_object11523/head//1 by client.1048726.0:11524 2014-08-12 17:10:49.447159 2014-08-13 18:06:06.380690 7f5c973a57a0 20 read_log 194215'1029 (0'0) modify f5bded2b/benchmark_data_storage0111-ib.pocstor_19374_object11543/head//1 by client.1048726.0:11544 2014-08-12 17:10:49.523414 2014-08-13 18:06:06.380712 7f5c973a57a0 20 read_log 194215'1030 (0'0) modify c4b3dc6b/benchmark_data_storage0111-ib.pocstor_19374_object11655/head//1 by client.1048726.0:11656 2014-08-12 17:10:50.068298 2014-08-13 18:06:06.380734 7f5c973a57a0 20 read_log 194215'1031 (0'0) modify 2095576b/benchmark_data_storage0111-ib.pocstor_19374_object11672/head//1 by client.1048726.0:11673 2014-08-12 17:10:50.139254 2014-08-13 18:06:06.380755 7f5c973a57a0 20 read_log 194215'1032 (0'0) modify a6115a2b/benchmark_data_storage0111-ib.pocstor_19374_object11714/head//1 by client.1048726.0:11715 2014-08-12 17:10:50.332239 2014-08-13 18:06:06.380778 7f5c973a57a0 20 read_log 194215'1033 (0'0) modify 8aafa9ab/benchmark_data_storage0111-ib.pocstor_19374_object11744/head//1 by client.1048726.0:11745 2014-08-12 17:10:50.525189 2014-08-13 18:06:06.380803 7f5c973a57a0 20 read_log 194215'1034 (0'0) modify d9131ceb/benchmark_data_storage0111-ib.pocstor_19374_object11815/head//1 by client.1048726.0:11816 2014-08-12 17:10:51.027982 2014-08-13 18:06:06.380824 7f5c973a57a0 20 read_log 194215'1035 (0'0) modify 7091a6b/benchmark_data_storage0111-ib.pocstor_19374_object12125/head//1 by client.1048726.0:12126 2014-08-12 17:10:52.717446 2014-08-13 18:06:06.380846 7f5c973a57a0 20 read_log 194215'1036 (0'0) modify 96552bab/benchmark_data_storage0111-ib.pocstor_19374_object12215/head//1 by client.1048726.0:12216 2014-08-12 17:10:53.169165 2014-08-13 18:06:06.380867 7f5c973a57a0 20 read_log 194215'1037 (0'0) modify 84c0f3eb/benchmark_data_storage0111-ib.pocstor_19374_object12617/head//1 by client.1048726.0:12618 2014-08-12 17:10:55.405083 2014-08-13 18:06:06.380888 7f5c973a57a0 20 read_log 194215'1038 (0'0) modify 6b222c6b/benchmark_data_storage0111-ib.pocstor_19374_object12626/head//1 by client.1048726.0:12627 2014-08-12 17:10:55.470454 2014-08-13 18:06:06.380910 7f5c973a57a0 20 read_log 194215'1039 (0'0) modify 304a35eb/benchmark_data_storage0111-ib.pocstor_19374_object12721/head//1 by client.1048726.0:12722 2014-08-12 17:10:56.018848 2014-08-13 18:06:06.380931 7f5c973a57a0 20 read_log 194215'1040 (0'0) modify ce855f2b/benchmark_data_storage0111-ib.pocstor_19374_object12758/head//1 by client.1048726.0:12759 2014-08-12 17:10:56.160994 2014-08-13 18:06:06.380953 7f5c973a57a0 20 read_log 194215'1041 (0'0) modify 91d427ab/benchmark_data_storage0111-ib.pocstor_19374_object12778/head//1 by client.1048726.0:12779 2014-08-12 17:10:56.233117 2014-08-13 18:06:06.380974 7f5c973a57a0 20 read_log 194215'1042 (0'0) modify fb4560ab/benchmark_data_storage0111-ib.pocstor_19374_object12863/head//1 by client.1048726.0:12864 2014-08-12 17:10:56.573943 2014-08-13 18:06:06.380995 7f5c973a57a0 20 read_log 194215'1043 (0'0) modify 883592b/benchmark_data_storage0111-ib.pocstor_19374_object12865/head//1 by client.1048726.0:12866 2014-08-12 17:10:56.584963 2014-08-13 18:06:06.381032 7f5c973a57a0 20 read_log 194215'1044 (0'0) modify 64527e2b/benchmark_data_storage0111-ib.pocstor_19374_object12908/head//1 by client.1048726.0:12909 2014-08-12 17:10:56.782793 2014-08-13 18:06:06.381058 7f5c973a57a0 20 read_log 194215'1045 (0'0) modify bb45c1eb/benchmark_data_storage0111-ib.pocstor_19374_object12937/head//1 by client.1048726.0:12938 2014-08-12 17:10:56.911124 2014-08-13 18:06:06.381081 7f5c973a57a0 20 read_log 194215'1046 (0'0) modify 4f81c16b/benchmark_data_storage0111-ib.pocstor_19374_object12968/head//1 by client.1048726.0:12969 2014-08-12 17:10:57.059413 2014-08-13 18:06:06.381103 7f5c973a57a0 20 read_log 194215'1047 (0'0) modify ca6ba7eb/benchmark_data_storage0111-ib.pocstor_19374_object12985/head//1 by client.1048726.0:12986 2014-08-12 17:10:57.153096 2014-08-13 18:06:06.381124 7f5c973a57a0 20 read_log 194215'1048 (0'0) modify d6caf0eb/benchmark_data_storage0111-ib.pocstor_19374_object13037/head//1 by client.1048726.0:13038 2014-08-12 17:10:57.434942 2014-08-13 18:06:06.381154 7f5c973a57a0 20 read_log 194215'1049 (0'0) modify 995517eb/benchmark_data_storage0111-ib.pocstor_19374_object13098/head//1 by client.1048726.0:13099 2014-08-12 17:10:57.754754 2014-08-13 18:06:06.381178 7f5c973a57a0 20 read_log 194215'1050 (0'0) modify 69b7392b/benchmark_data_storage0111-ib.pocstor_19374_object13278/head//1 by client.1048726.0:13279 2014-08-12 17:10:58.809494 2014-08-13 18:06:06.381200 7f5c973a57a0 20 read_log 194215'1051 (0'0) modify 5d3e5d6b/benchmark_data_storage0111-ib.pocstor_19374_object13368/head//1 by client.1048726.0:13369 2014-08-12 17:10:59.479526 2014-08-13 18:06:06.381221 7f5c973a57a0 20 read_log 194215'1052 (0'0) modify b7bdedeb/benchmark_data_storage0111-ib.pocstor_19374_object13400/head//1 by client.1048726.0:13401 2014-08-12 17:10:59.596906 2014-08-13 18:06:06.381243 7f5c973a57a0 20 read_log 194215'1053 (0'0) modify 6dc64eeb/benchmark_data_storage0111-ib.pocstor_19374_object13453/head//1 by client.1048726.0:13454 2014-08-12 17:10:59.795662 2014-08-13 18:06:06.381264 7f5c973a57a0 20 read_log 194215'1054 (0'0) modify 8c0f1d2b/benchmark_data_storage0111-ib.pocstor_19374_object13462/head//1 by client.1048726.0:13463 2014-08-12 17:10:59.829868 2014-08-13 18:06:06.381286 7f5c973a57a0 20 read_log 194215'1055 (0'0) modify 37a0adab/benchmark_data_storage0111-ib.pocstor_19374_object13587/head//1 by client.1048726.0:13588 2014-08-12 17:11:00.345687 2014-08-13 18:06:06.381317 7f5c973a57a0 20 read_log 194215'1056 (0'0) modify 5a6f74eb/benchmark_data_storage0111-ib.pocstor_19374_object13600/head//1 by client.1048726.0:13601 2014-08-12 17:11:00.392623 2014-08-13 18:06:06.381339 7f5c973a57a0 20 read_log 194215'1057 (0'0) modify d3458feb/benchmark_data_storage0111-ib.pocstor_19374_object13641/head//1 by client.1048726.0:13642 2014-08-12 17:11:00.542529 2014-08-13 18:06:06.381361 7f5c973a57a0 20 read_log 194215'1058 (0'0) modify b1e814eb/benchmark_data_storage0111-ib.pocstor_19374_object13681/head//1 by client.1048726.0:13682 2014-08-12 17:11:00.696193 2014-08-13 18:06:06.381392 7f5c973a57a0 20 read_log 194215'1059 (0'0) modify 97b4952b/benchmark_data_storage0111-ib.pocstor_19374_object13697/head//1 by client.1048726.0:13698 2014-08-12 17:11:00.753113 2014-08-13 18:06:06.381414 7f5c973a57a0 20 read_log 194215'1060 (0'0) modify b156fbab/benchmark_data_storage0111-ib.pocstor_19374_object13795/head//1 by client.1048726.0:13796 2014-08-12 17:11:01.117694 2014-08-13 18:06:06.381447 7f5c973a57a0 20 read_log 194215'1061 (0'0) modify b9abdaeb/benchmark_data_storage0111-ib.pocstor_19374_object13915/head//1 by client.1048726.0:13916 2014-08-12 17:11:01.568882 2014-08-13 18:06:06.381475 7f5c973a57a0 20 read_log 194215'1062 (0'0) modify 15bb4aeb/benchmark_data_storage0111-ib.pocstor_19374_object14018/head//1 by client.1048726.0:14019 2014-08-12 17:11:01.956741 2014-08-13 18:06:06.381521 7f5c973a57a0 20 read_log 194215'1063 (0'0) modify 86d77e6b/benchmark_data_storage0111-ib.pocstor_19374_object14045/head//1 by client.1048726.0:14046 2014-08-12 17:11:02.054905 2014-08-13 18:06:06.381573 7f5c973a57a0 20 read_log 194215'1064 (0'0) modify f762602b/benchmark_data_storage0111-ib.pocstor_19374_object14072/head//1 by client.1048726.0:14073 2014-08-12 17:11:02.155399 2014-08-13 18:06:06.381602 7f5c973a57a0 20 read_log 194215'1065 (0'0) modify 656a9d6b/benchmark_data_storage0111-ib.pocstor_19374_object14108/head//1 by client.1048726.0:14109 2014-08-12 17:11:02.288002 2014-08-13 18:06:06.381637 7f5c973a57a0 20 read_log 194215'1066 (0'0) modify 12017d6b/benchmark_data_storage0111-ib.pocstor_19374_object14130/head//1 by client.1048726.0:14131 2014-08-12 17:11:02.380293 2014-08-13 18:06:06.381660 7f5c973a57a0 20 read_log 194215'1067 (0'0) modify 6837d1eb/benchmark_data_storage0111-ib.pocstor_19374_object14218/head//1 by client.1048726.0:14219 2014-08-12 17:11:02.782849 2014-08-13 18:06:06.381682 7f5c973a57a0 20 read_log 194215'1068 (0'0) modify e0890e6b/benchmark_data_storage0111-ib.pocstor_19374_object14272/head//1 by client.1048726.0:14273 2014-08-12 17:11:02.980867 2014-08-13 18:06:06.381717 7f5c973a57a0 20 read_log 194215'1069 (0'0) modify 51a9f76b/benchmark_data_storage0111-ib.pocstor_19374_object14273/head//1 by client.1048726.0:14274 2014-08-12 17:11:02.984570 2014-08-13 18:06:06.381744 7f5c973a57a0 20 read_log 194215'1070 (0'0) modify ff105aeb/benchmark_data_storage0111-ib.pocstor_19374_object14288/head//1 by client.1048726.0:14289 2014-08-12 17:11:03.038863 2014-08-13 18:06:06.381768 7f5c973a57a0 20 read_log 194215'1071 (0'0) modify 7c054ceb/benchmark_data_storage0111-ib.pocstor_19374_object14442/head//1 by client.1048726.0:14443 2014-08-12 17:11:03.778734 2014-08-13 18:06:06.381790 7f5c973a57a0 20 read_log 194215'1072 (0'0) modify fc1d02eb/benchmark_data_storage0111-ib.pocstor_19374_object14443/head//1 by client.1048726.0:14444 2014-08-12 17:11:03.784749 2014-08-13 18:06:06.381813 7f5c973a57a0 20 read_log 194215'1073 (0'0) modify b7463aeb/benchmark_data_storage0111-ib.pocstor_19374_object14446/head//1 by client.1048726.0:14447 2014-08-12 17:11:03.802855 2014-08-13 18:06:06.381835 7f5c973a57a0 20 read_log 194215'1074 (0'0) modify 56be36eb/benchmark_data_storage0111-ib.pocstor_19374_object14581/head//1 by client.1048726.0:14582 2014-08-12 17:11:04.570010 2014-08-13 18:06:06.381856 7f5c973a57a0 20 read_log 194215'1075 (0'0) modify bc8678eb/benchmark_data_storage0111-ib.pocstor_19374_object14608/head//1 by client.1048726.0:14609 2014-08-12 17:11:04.710252 2014-08-13 18:06:06.381878 7f5c973a57a0 20 read_log 194215'1076 (0'0) modify 49787bab/benchmark_data_storage0111-ib.pocstor_19374_object14741/head//1 by client.1048726.0:14742 2014-08-12 17:11:05.631531 2014-08-13 18:06:06.381931 7f5c973a57a0 20 read_log 194215'1077 (0'0) modify af3d7a2b/benchmark_data_storage0111-ib.pocstor_19374_object14751/head//1 by client.1048726.0:14752 2014-08-12 17:11:05.713953 2014-08-13 18:06:06.381958 7f5c973a57a0 20 read_log 194215'1078 (0'0) modify 3cec11eb/benchmark_data_storage0111-ib.pocstor_19374_object14764/head//1 by client.1048726.0:14765 2014-08-12 17:11:05.774351 2014-08-13 18:06:06.381980 7f5c973a57a0 20 read_log 194215'1079 (0'0) modify 97eeeab/benchmark_data_storage0111-ib.pocstor_19374_object14796/head//1 by client.1048726.0:14797 2014-08-12 17:11:05.926313 2014-08-13 18:06:06.382001 7f5c973a57a0 20 read_log 194215'1080 (0'0) modify 708ad02b/benchmark_data_storage0111-ib.pocstor_19374_object14818/head//1 by client.1048726.0:14819 2014-08-12 17:11:06.055132 2014-08-13 18:06:06.382023 7f5c973a57a0 20 read_log 194215'1081 (0'0) modify ed21ddeb/benchmark_data_storage0111-ib.pocstor_19374_object14866/head//1 by client.1048726.0:14867 2014-08-12 17:11:06.282019 2014-08-13 18:06:06.382074 7f5c973a57a0 20 read_log 194215'1082 (0'0) modify 7847fa6b/benchmark_data_storage0111-ib.pocstor_19374_object15056/head//1 by client.1048726.0:15057 2014-08-12 17:11:07.237033 2014-08-13 18:06:06.382097 7f5c973a57a0 20 read_log 194215'1083 (0'0) modify 91326eeb/benchmark_data_storage0111-ib.pocstor_19374_object15087/head//1 by client.1048726.0:15088 2014-08-12 17:11:07.431239 2014-08-13 18:06:06.382169 7f5c973a57a0 20 read_log 194215'1084 (0'0) modify f966fa2b/benchmark_data_storage0111-ib.pocstor_19374_object15209/head//1 by client.1048726.0:15210 2014-08-12 17:11:07.900154 2014-08-13 18:06:06.382198 7f5c973a57a0 20 read_log 194215'1085 (0'0) modify 228e7c6b/benchmark_data_storage0111-ib.pocstor_19374_object15227/head//1 by client.1048726.0:15228 2014-08-12 17:11:07.967280 2014-08-13 18:06:06.382220 7f5c973a57a0 20 read_log 194215'1086 (0'0) modify 7acf24ab/benchmark_data_storage0111-ib.pocstor_19374_object15235/head//1 by client.1048726.0:15236 2014-08-12 17:11:07.996324 2014-08-13 18:06:06.382241 7f5c973a57a0 20 read_log 194215'1087 (0'0) modify c50ebeeb/benchmark_data_storage0111-ib.pocstor_19374_object15256/head//1 by client.1048726.0:15257 2014-08-12 17:11:08.074474 2014-08-13 18:06:06.382263 7f5c973a57a0 20 read_log 194215'1088 (0'0) modify ce6e186b/benchmark_data_storage0111-ib.pocstor_19374_object15291/head//1 by client.1048726.0:15292 2014-08-12 17:11:08.202782 2014-08-13 18:06:06.382285 7f5c973a57a0 20 read_log 194215'1089 (0'0) modify 4248daab/benchmark_data_storage0111-ib.pocstor_19374_object15301/head//1 by client.1048726.0:15302 2014-08-12 17:11:08.239529 2014-08-13 18:06:06.382307 7f5c973a57a0 20 read_log 194215'1090 (0'0) modify 7db6d4eb/benchmark_data_storage0111-ib.pocstor_19374_object15375/head//1 by client.1048726.0:15376 2014-08-12 17:11:08.579730 2014-08-13 18:06:06.382328 7f5c973a57a0 20 read_log 194215'1091 (0'0) modify d27cd12b/benchmark_data_storage0111-ib.pocstor_19374_object15494/head//1 by client.1048726.0:15495 2014-08-12 17:11:09.064518 2014-08-13 18:06:06.382355 7f5c973a57a0 20 read_log 194215'1092 (0'0) modify 29efcab/benchmark_data_storage0111-ib.pocstor_19374_object15598/head//1 by client.1048726.0:15599 2014-08-12 17:11:09.753328 2014-08-13 18:06:06.382377 7f5c973a57a0 20 read_log 194215'1093 (0'0) modify 725d332b/benchmark_data_storage0111-ib.pocstor_19374_object15691/head//1 by client.1048726.0:15692 2014-08-12 17:11:10.201985 2014-08-13 18:06:06.382404 7f5c973a57a0 20 read_log 194215'1094 (0'0) modify 427577eb/benchmark_data_storage0111-ib.pocstor_19374_object15706/head//1 by client.1048726.0:15707 2014-08-12 17:11:10.258348 2014-08-13 18:06:06.382429 7f5c973a57a0 20 read_log 194215'1095 (0'0) modify 3774496b/benchmark_data_storage0111-ib.pocstor_19374_object15775/head//1 by client.1048726.0:15776 2014-08-12 17:11:10.511897 2014-08-13 18:06:06.382456 7f5c973a57a0 20 read_log 194215'1096 (0'0) modify fb1ddcab/benchmark_data_storage0111-ib.pocstor_19374_object15797/head//1 by client.1048726.0:15798 2014-08-12 17:11:10.592274 2014-08-13 18:06:06.382502 7f5c973a57a0 20 read_log 194215'1097 (0'0) modify 72b1916b/benchmark_data_storage0111-ib.pocstor_19374_object15974/head//1 by client.1048726.0:15975 2014-08-12 17:11:11.421520 2014-08-13 18:06:06.382541 7f5c973a57a0 20 read_log 194215'1098 (0'0) modify 1afad6eb/benchmark_data_storage0111-ib.pocstor_19374_object16016/head//1 by client.1048726.0:16017 2014-08-12 17:11:11.659355 2014-08-13 18:06:06.382568 7f5c973a57a0 20 read_log 194215'1099 (0'0) modify 36c3d4ab/benchmark_data_storage0111-ib.pocstor_19374_object16089/head//1 by client.1048726.0:16090 2014-08-12 17:11:11.942172 2014-08-13 18:06:06.382591 7f5c973a57a0 20 read_log 194215'1100 (0'0) modify 41ee41eb/benchmark_data_storage0111-ib.pocstor_19374_object16099/head//1 by client.1048726.0:16100 2014-08-12 17:11:11.989962 2014-08-13 18:06:06.382612 7f5c973a57a0 20 read_log 194215'1101 (0'0) modify f52c0f6b/benchmark_data_storage0111-ib.pocstor_19374_object16114/head//1 by client.1048726.0:16115 2014-08-12 17:11:12.070909 2014-08-13 18:06:06.382634 7f5c973a57a0 20 read_log 194215'1102 (0'0) modify 343512ab/benchmark_data_storage0111-ib.pocstor_19374_object16151/head//1 by client.1048726.0:16152 2014-08-12 17:11:12.261179 2014-08-13 18:06:06.382655 7f5c973a57a0 20 read_log 194215'1103 (0'0) modify 5811406b/benchmark_data_storage0111-ib.pocstor_19374_object16504/head//1 by client.1048726.0:16505 2014-08-12 17:11:13.777636 2014-08-13 18:06:06.382710 7f5c973a57a0 20 read_log 194215'1104 (0'0) modify 9896e7ab/benchmark_data_storage0111-ib.pocstor_19374_object16534/head//1 by client.1048726.0:16535 2014-08-12 17:11:13.887725 2014-08-13 18:06:06.382740 7f5c973a57a0 20 read_log 194215'1105 (0'0) modify d7af0eab/benchmark_data_storage0111-ib.pocstor_19374_object16646/head//1 by client.1048726.0:16647 2014-08-12 17:11:14.311649 2014-08-13 18:06:06.382762 7f5c973a57a0 20 read_log 194215'1106 (0'0) modify 915c5a2b/benchmark_data_storage0111-ib.pocstor_19374_object16648/head//1 by client.1048726.0:16649 2014-08-12 17:11:14.318961 2014-08-13 18:06:06.382784 7f5c973a57a0 20 read_log 194215'1107 (0'0) modify a73d206b/benchmark_data_storage0111-ib.pocstor_19374_object16746/head//1 by client.1048726.0:16747 2014-08-12 17:11:14.681228 2014-08-13 18:06:06.382805 7f5c973a57a0 20 read_log 194215'1108 (0'0) modify c47c9beb/benchmark_data_storage0111-ib.pocstor_19374_object16760/head//1 by client.1048726.0:16761 2014-08-12 17:11:14.734844 2014-08-13 18:06:06.382827 7f5c973a57a0 20 read_log 194215'1109 (0'0) modify dea573ab/benchmark_data_storage0111-ib.pocstor_19374_object16841/head//1 by client.1048726.0:16842 2014-08-12 17:11:15.046110 2014-08-13 18:06:06.382848 7f5c973a57a0 20 read_log 194215'1110 (0'0) modify b9e0d1ab/benchmark_data_storage0111-ib.pocstor_19374_object16881/head//1 by client.1048726.0:16882 2014-08-12 17:11:15.193801 2014-08-13 18:06:06.382870 7f5c973a57a0 20 read_log 194215'1111 (0'0) modify 3755296b/benchmark_data_storage0111-ib.pocstor_19374_object16916/head//1 by client.1048726.0:16917 2014-08-12 17:11:15.352323 2014-08-13 18:06:06.382892 7f5c973a57a0 20 read_log 194215'1112 (0'0) modify c45cbd6b/benchmark_data_storage0111-ib.pocstor_19374_object16925/head//1 by client.1048726.0:16926 2014-08-12 17:11:15.409489 2014-08-13 18:06:06.382913 7f5c973a57a0 20 read_log 194215'1113 (0'0) modify d5f0a92b/benchmark_data_storage0111-ib.pocstor_19374_object16947/head//1 by client.1048726.0:16948 2014-08-12 17:11:15.552798 2014-08-13 18:06:06.382934 7f5c973a57a0 20 read_log 194215'1114 (0'0) modify 5fceb76b/benchmark_data_storage0111-ib.pocstor_19374_object17225/head//1 by client.1048726.0:17226 2014-08-12 17:11:16.642545 2014-08-13 18:06:06.382956 7f5c973a57a0 20 read_log 194215'1115 (0'0) modify bf0f31eb/benchmark_data_storage0111-ib.pocstor_19374_object17240/head//1 by client.1048726.0:17241 2014-08-12 17:11:16.727026 2014-08-13 18:06:06.382977 7f5c973a57a0 20 read_log 194215'1116 (0'0) modify ab105e2b/benchmark_data_storage0111-ib.pocstor_19374_object17300/head//1 by client.1048726.0:17301 2014-08-12 17:11:16.984455 2014-08-13 18:06:06.383000 7f5c973a57a0 20 read_log 194215'1117 (0'0) modify 8b06b7ab/benchmark_data_storage0111-ib.pocstor_19374_object17532/head//1 by client.1048726.0:17533 2014-08-12 17:11:18.127760 2014-08-13 18:06:06.383022 7f5c973a57a0 20 read_log 194215'1118 (0'0) modify ff580c6b/benchmark_data_storage0111-ib.pocstor_19374_object17566/head//1 by client.1048726.0:17567 2014-08-12 17:11:18.391187 2014-08-13 18:06:06.383044 7f5c973a57a0 20 read_log 194215'1119 (0'0) modify feff186b/benchmark_data_storage0111-ib.pocstor_19374_object17666/head//1 by client.1048726.0:17667 2014-08-12 17:11:18.814107 2014-08-13 18:06:06.383065 7f5c973a57a0 20 read_log 194215'1120 (0'0) modify 92eb096b/benchmark_data_storage0111-ib.pocstor_19374_object17678/head//1 by client.1048726.0:17679 2014-08-12 17:11:18.868790 2014-08-13 18:06:06.383086 7f5c973a57a0 20 read_log 194215'1121 (0'0) modify 4795ccab/benchmark_data_storage0111-ib.pocstor_19374_object17689/head//1 by client.1048726.0:17690 2014-08-12 17:11:18.910377 2014-08-13 18:06:06.383107 7f5c973a57a0 20 read_log 194215'1122 (0'0) modify 7610516b/benchmark_data_storage0111-ib.pocstor_19374_object17776/head//1 by client.1048726.0:17777 2014-08-12 17:11:19.346114 2014-08-13 18:06:06.383131 7f5c973a57a0 20 read_log 194215'1123 (0'0) modify 1941352b/benchmark_data_storage0111-ib.pocstor_19374_object17799/head//1 by client.1048726.0:17800 2014-08-12 17:11:19.430094 2014-08-13 18:06:06.383174 7f5c973a57a0 20 read_log 194215'1124 (0'0) modify a663f52b/benchmark_data_storage0111-ib.pocstor_19374_object17824/head//1 by client.1048726.0:17825 2014-08-12 17:11:19.520873 2014-08-13 18:06:06.383202 7f5c973a57a0 20 read_log 194215'1125 (0'0) modify 85edc6ab/benchmark_data_storage0111-ib.pocstor_19374_object17839/head//1 by client.1048726.0:17840 2014-08-12 17:11:19.582707 2014-08-13 18:06:06.383224 7f5c973a57a0 20 read_log 194215'1126 (0'0) modify 3a62892b/benchmark_data_storage0111-ib.pocstor_19374_object17894/head//1 by client.1048726.0:17895 2014-08-12 17:11:19.786243 2014-08-13 18:06:06.383245 7f5c973a57a0 20 read_log 194215'1127 (0'0) modify a9f9016b/benchmark_data_storage0111-ib.pocstor_19374_object17942/head//1 by client.1048726.0:17943 2014-08-12 17:11:19.960010 2014-08-13 18:06:06.383267 7f5c973a57a0 20 read_log 194215'1128 (0'0) modify 49c8032b/benchmark_data_storage0111-ib.pocstor_19374_object18015/head//1 by client.1048726.0:18016 2014-08-12 17:11:20.228997 2014-08-13 18:06:06.383293 7f5c973a57a0 20 read_log 194215'1129 (0'0) modify 63b44fab/benchmark_data_storage0111-ib.pocstor_19374_object18232/head//1 by client.1048726.0:18233 2014-08-12 17:11:21.110630 2014-08-13 18:06:06.383319 7f5c973a57a0 20 read_log 194215'1130 (0'0) modify c05a8beb/benchmark_data_storage0111-ib.pocstor_19374_object18250/head//1 by client.1048726.0:18251 2014-08-12 17:11:21.176143 2014-08-13 18:06:06.383346 7f5c973a57a0 20 read_log 194215'1131 (0'0) modify 622415eb/benchmark_data_storage0111-ib.pocstor_19374_object18344/head//1 by client.1048726.0:18345 2014-08-12 17:11:21.589700 2014-08-13 18:06:06.383373 7f5c973a57a0 20 read_log 194215'1132 (0'0) modify d404aeb/benchmark_data_storage0111-ib.pocstor_19374_object18373/head//1 by client.1048726.0:18374 2014-08-12 17:11:21.697244 2014-08-13 18:06:06.383395 7f5c973a57a0 20 read_log 194215'1133 (0'0) modify 19dc8e2b/benchmark_data_storage0111-ib.pocstor_19374_object18496/head//1 by client.1048726.0:18497 2014-08-12 17:11:22.168059 2014-08-13 18:06:06.383423 7f5c973a57a0 20 read_log 194215'1134 (0'0) modify 379abaab/benchmark_data_storage0111-ib.pocstor_19374_object18497/head//1 by client.1048726.0:18498 2014-08-12 17:11:22.174034 2014-08-13 18:06:06.383470 7f5c973a57a0 20 read_log 194215'1135 (0'0) modify 64e0802b/benchmark_data_storage0111-ib.pocstor_19374_object18498/head//1 by client.1048726.0:18499 2014-08-12 17:11:22.178663 2014-08-13 18:06:06.383510 7f5c973a57a0 20 read_log 194215'1136 (0'0) modify c6e8922b/benchmark_data_storage0111-ib.pocstor_19374_object18505/head//1 by client.1048726.0:18506 2014-08-12 17:11:22.210691 2014-08-13 18:06:06.383533 7f5c973a57a0 20 read_log 194215'1137 (0'0) modify 2855efeb/benchmark_data_storage0111-ib.pocstor_19374_object18551/head//1 by client.1048726.0:18552 2014-08-12 17:11:22.558483 2014-08-13 18:06:06.383559 7f5c973a57a0 20 read_log 194215'1138 (0'0) modify 646eadab/benchmark_data_storage0111-ib.pocstor_19374_object18577/head//1 by client.1048726.0:18578 2014-08-12 17:11:22.727357 2014-08-13 18:06:06.383581 7f5c973a57a0 20 read_log 194215'1139 (0'0) modify 6c5018eb/benchmark_data_storage0111-ib.pocstor_19374_object18780/head//1 by client.1048726.0:18781 2014-08-12 17:11:23.601122 2014-08-13 18:06:06.383603 7f5c973a57a0 20 read_log 194215'1140 (0'0) modify d3893eeb/benchmark_data_storage0111-ib.pocstor_19374_object18819/head//1 by client.1048726.0:18820 2014-08-12 17:11:23.828227 2014-08-13 18:06:06.383650 7f5c973a57a0 20 read_log 194215'1141 (0'0) modify f022432b/benchmark_data_storage0111-ib.pocstor_19374_object18842/head//1 by client.1048726.0:18843 2014-08-12 17:11:24.025785 2014-08-13 18:06:06.383677 7f5c973a57a0 20 read_log 194215'1142 (0'0) modify 2476c96b/benchmark_data_storage0111-ib.pocstor_19374_object18852/head//1 by client.1048726.0:18853 2014-08-12 17:11:24.087535 2014-08-13 18:06:06.383699 7f5c973a57a0 20 read_log 194215'1143 (0'0) modify e1cd2ceb/benchmark_data_storage0111-ib.pocstor_19374_object19065/head//1 by client.1048726.0:19066 2014-08-12 17:11:24.935380 2014-08-13 18:06:06.383732 7f5c973a57a0 20 read_log 194215'1144 (0'0) modify ae4f962b/benchmark_data_storage0111-ib.pocstor_19374_object19226/head//1 by client.1048726.0:19227 2014-08-12 17:11:25.634967 2014-08-13 18:06:06.383757 7f5c973a57a0 20 read_log 194215'1145 (0'0) modify b8cf256b/benchmark_data_storage0111-ib.pocstor_19374_object19231/head//1 by client.1048726.0:19232 2014-08-12 17:11:25.670638 2014-08-13 18:06:06.383779 7f5c973a57a0 20 read_log 194215'1146 (0'0) modify f0edd2b/benchmark_data_storage0111-ib.pocstor_19374_object19233/head//1 by client.1048726.0:19234 2014-08-12 17:11:25.683402 2014-08-13 18:06:06.383813 7f5c973a57a0 20 read_log 194215'1147 (0'0) modify 2f22b1ab/benchmark_data_storage0111-ib.pocstor_19374_object19274/head//1 by client.1048726.0:19275 2014-08-12 17:11:26.361721 2014-08-13 18:06:06.383839 7f5c973a57a0 20 read_log 194215'1148 (0'0) modify 115748eb/benchmark_data_storage0111-ib.pocstor_19374_object19293/head//1 by client.1048726.0:19294 2014-08-12 17:11:26.466906 2014-08-13 18:06:06.383865 7f5c973a57a0 20 read_log 194215'1149 (0'0) modify ce72ce6b/benchmark_data_storage0111-ib.pocstor_19374_object19399/head//1 by client.1048726.0:19400 2014-08-12 17:11:26.872335 2014-08-13 18:06:06.383887 7f5c973a57a0 20 read_log 194215'1150 (0'0) modify d56f90ab/benchmark_data_storage0111-ib.pocstor_19374_object19623/head//1 by client.1048726.0:19624 2014-08-12 17:11:27.697981 2014-08-13 18:06:06.383909 7f5c973a57a0 20 read_log 194215'1151 (0'0) modify ee91eceb/benchmark_data_storage0111-ib.pocstor_19374_object19625/head//1 by client.1048726.0:19626 2014-08-12 17:11:27.705429 2014-08-13 18:06:06.383930 7f5c973a57a0 20 read_log 194215'1152 (0'0) modify 563df2ab/benchmark_data_storage0111-ib.pocstor_19374_object19626/head//1 by client.1048726.0:19627 2014-08-12 17:11:27.709066 2014-08-13 18:06:06.383961 7f5c973a57a0 20 read_log 194215'1153 (0'0) modify f16196eb/benchmark_data_storage0111-ib.pocstor_19374_object19875/head//1 by client.1048726.0:19876 2014-08-12 17:11:28.659242 2014-08-13 18:06:06.383995 7f5c973a57a0 20 read_log 194215'1154 (0'0) modify 43e8486b/benchmark_data_storage0111-ib.pocstor_19374_object19917/head//1 by client.1048726.0:19918 2014-08-12 17:11:28.895145 2014-08-13 18:06:06.384021 7f5c973a57a0 20 read_log 194215'1155 (0'0) modify 3847442b/benchmark_data_storage0111-ib.pocstor_19374_object19991/head//1 by client.1048726.0:19992 2014-08-12 17:11:29.242744 2014-08-13 18:06:06.384047 7f5c973a57a0 20 read_log 194215'1156 (0'0) modify 2e8e12ab/benchmark_data_storage0111-ib.pocstor_19374_object20016/head//1 by client.1048726.0:20017 2014-08-12 17:11:29.336224 2014-08-13 18:06:06.384085 7f5c973a57a0 20 read_log 194215'1157 (0'0) modify 9ea9826b/benchmark_data_storage0111-ib.pocstor_19374_object20100/head//1 by client.1048726.0:20101 2014-08-12 17:11:29.651300 2014-08-13 18:06:06.384114 7f5c973a57a0 20 read_log 194215'1158 (0'0) modify de30edeb/benchmark_data_storage0111-ib.pocstor_19374_object20132/head//1 by client.1048726.0:20133 2014-08-12 17:11:29.768937 2014-08-13 18:06:06.384157 7f5c973a57a0 20 read_log 194215'1159 (0'0) modify 47ed002b/benchmark_data_storage0111-ib.pocstor_19374_object20189/head//1 by client.1048726.0:20190 2014-08-12 17:11:29.981605 2014-08-13 18:06:06.384187 7f5c973a57a0 20 read_log 194215'1160 (0'0) modify 5be3cf2b/benchmark_data_storage0111-ib.pocstor_19374_object20205/head//1 by client.1048726.0:20206 2014-08-12 17:11:30.040316 2014-08-13 18:06:06.384213 7f5c973a57a0 20 read_log 194215'1161 (0'0) modify fbdfe52b/benchmark_data_storage0111-ib.pocstor_19374_object20231/head//1 by client.1048726.0:20232 2014-08-12 17:11:30.138232 2014-08-13 18:06:06.384240 7f5c973a57a0 20 read_log 194215'1162 (0'0) modify 45de5dab/benchmark_data_storage0111-ib.pocstor_19374_object20287/head//1 by client.1048726.0:20288 2014-08-12 17:11:30.357060 2014-08-13 18:06:06.384265 7f5c973a57a0 20 read_log 194215'1163 (0'0) modify 94efc5eb/benchmark_data_storage0111-ib.pocstor_19374_object20346/head//1 by client.1048726.0:20347 2014-08-12 17:11:30.703703 2014-08-13 18:06:06.384302 7f5c973a57a0 20 read_log 194215'1164 (0'0) modify 1347d22b/benchmark_data_storage0111-ib.pocstor_19374_object20407/head//1 by client.1048726.0:20408 2014-08-12 17:11:30.987836 2014-08-13 18:06:06.384334 7f5c973a57a0 20 read_log 194215'1165 (0'0) modify e33e54ab/benchmark_data_storage0111-ib.pocstor_19374_object20410/head//1 by client.1048726.0:20411 2014-08-12 17:11:30.998699 2014-08-13 18:06:06.384377 7f5c973a57a0 20 read_log 194215'1166 (0'0) modify e858402b/benchmark_data_storage0111-ib.pocstor_19374_object20437/head//1 by client.1048726.0:20438 2014-08-12 17:11:31.099054 2014-08-13 18:06:06.384405 7f5c973a57a0 20 read_log 194215'1167 (0'0) modify 19a5632b/benchmark_data_storage0111-ib.pocstor_19374_object20464/head//1 by client.1048726.0:20465 2014-08-12 17:11:31.197916 2014-08-13 18:06:06.384433 7f5c973a57a0 20 read_log 194215'1168 (0'0) modify 19fb1ceb/benchmark_data_storage0111-ib.pocstor_19374_object20580/head//1 by client.1048726.0:20581 2014-08-12 17:11:31.636049 2014-08-13 18:06:06.384455 7f5c973a57a0 20 read_log 194215'1169 (0'0) modify 63afe4eb/benchmark_data_storage0111-ib.pocstor_19374_object20590/head//1 by client.1048726.0:20591 2014-08-12 17:11:31.674216 2014-08-13 18:06:06.384512 7f5c973a57a0 20 read_log 194215'1170 (0'0) modify 5e2d476b/benchmark_data_storage0111-ib.pocstor_19374_object20594/head//1 by client.1048726.0:20595 2014-08-12 17:11:31.690890 2014-08-13 18:06:06.384554 7f5c973a57a0 20 read_log 194215'1171 (0'0) modify 7ffdd76b/benchmark_data_storage0111-ib.pocstor_19374_object20637/head//1 by client.1048726.0:20638 2014-08-12 17:11:31.856883 2014-08-13 18:06:06.384582 7f5c973a57a0 20 read_log 194215'1172 (0'0) modify 595617eb/benchmark_data_storage0111-ib.pocstor_19374_object20646/head//1 by client.1048726.0:20647 2014-08-12 17:11:31.894409 2014-08-13 18:06:06.384604 7f5c973a57a0 20 read_log 194215'1173 (0'0) modify 2860b42b/benchmark_data_storage0111-ib.pocstor_19374_object20753/head//1 by client.1048726.0:20754 2014-08-12 17:11:32.554768 2014-08-13 18:06:06.384626 7f5c973a57a0 20 read_log 194215'1174 (0'0) modify 496c846b/benchmark_data_storage0111-ib.pocstor_19374_object20823/head//1 by client.1048726.0:20824 2014-08-12 17:11:32.826697 2014-08-13 18:06:06.384648 7f5c973a57a0 20 read_log 194215'1175 (0'0) modify 22036eeb/benchmark_data_storage0111-ib.pocstor_19374_object20913/head//1 by client.1048726.0:20914 2014-08-12 17:11:33.261477 2014-08-13 18:06:06.384670 7f5c973a57a0 20 read_log 194215'1176 (0'0) modify ab250fab/benchmark_data_storage0111-ib.pocstor_19374_object20914/head//1 by client.1048726.0:20915 2014-08-12 17:11:33.265285 2014-08-13 18:06:06.384691 7f5c973a57a0 20 read_log 194215'1177 (0'0) modify ae70372b/benchmark_data_storage0111-ib.pocstor_19374_object21028/head//1 by client.1048726.0:21029 2014-08-12 17:11:33.809639 2014-08-13 18:06:06.384713 7f5c973a57a0 20 read_log 194215'1178 (0'0) modify 1036baeb/benchmark_data_storage0111-ib.pocstor_19374_object21032/head//1 by client.1048726.0:21033 2014-08-12 17:11:33.842624 2014-08-13 18:06:06.384769 7f5c973a57a0 20 read_log 194215'1179 (0'0) modify 4a8e3aeb/benchmark_data_storage0111-ib.pocstor_19374_object21109/head//1 by client.1048726.0:21110 2014-08-12 17:11:34.314900 2014-08-13 18:06:06.384791 7f5c973a57a0 20 read_log 194215'1180 (0'0) modify 753fa2ab/benchmark_data_storage0111-ib.pocstor_19374_object21247/head//1 by client.1048726.0:21248 2014-08-12 17:11:35.099555 2014-08-13 18:06:06.384813 7f5c973a57a0 20 read_log 194215'1181 (0'0) modify f67584eb/benchmark_data_storage0111-ib.pocstor_19374_object21267/head//1 by client.1048726.0:21268 2014-08-12 17:11:35.186587 2014-08-13 18:06:06.384835 7f5c973a57a0 20 read_log 194215'1182 (0'0) modify fe828eab/benchmark_data_storage0111-ib.pocstor_19374_object21345/head//1 by client.1048726.0:21346 2014-08-12 17:11:36.038996 2014-08-13 18:06:06.384856 7f5c973a57a0 20 read_log 194215'1183 (0'0) modify 2ef9f66b/benchmark_data_storage0111-ib.pocstor_19374_object21352/head//1 by client.1048726.0:21353 2014-08-12 17:11:36.144223 2014-08-13 18:06:06.384893 7f5c973a57a0 20 read_log 194215'1184 (0'0) modify 3560716b/benchmark_data_storage0111-ib.pocstor_19374_object21409/head//1 by client.1048726.0:21410 2014-08-12 17:11:36.488438 2014-08-13 18:06:06.384920 7f5c973a57a0 20 read_log 194215'1185 (0'0) modify 7876b26b/benchmark_data_storage0111-ib.pocstor_19374_object21474/head//1 by client.1048726.0:21475 2014-08-12 17:11:36.881618 2014-08-13 18:06:06.384942 7f5c973a57a0 20 read_log 194215'1186 (0'0) modify a090132b/benchmark_data_storage0111-ib.pocstor_19374_object21525/head//1 by client.1048726.0:21526 2014-08-12 17:11:37.351706 2014-08-13 18:06:06.384964 7f5c973a57a0 20 read_log 194215'1187 (0'0) modify 2b02b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21551/head//1 by client.1048726.0:21552 2014-08-12 17:11:37.581206 2014-08-13 18:06:06.384988 7f5c973a57a0 20 read_log 194215'1188 (0'0) modify 2af0d52b/benchmark_data_storage0111-ib.pocstor_19374_object21651/head//1 by client.1048726.0:21652 2014-08-12 17:11:37.956609 2014-08-13 18:06:06.385010 7f5c973a57a0 20 read_log 194215'1189 (0'0) modify df94b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21660/head//1 by client.1048726.0:21661 2014-08-12 17:11:37.992064 2014-08-13 18:06:06.385033 7f5c973a57a0 20 read_log 194215'1190 (0'0) modify 157f89ab/benchmark_data_storage0111-ib.pocstor_19374_object21754/head//1 by client.1048726.0:21755 2014-08-12 17:11:38.345697 2014-08-13 18:06:06.385055 7f5c973a57a0 20 read_log 194215'1191 (0'0) modify 267742eb/benchmark_data_storage0111-ib.pocstor_19374_object21769/head//1 by client.1048726.0:21770 2014-08-12 17:11:38.400622 2014-08-13 18:06:06.385076 7f5c973a57a0 20 read_log 194215'1192 (0'0) modify 7086f06b/benchmark_data_storage0111-ib.pocstor_19374_object21770/head//1 by client.1048726.0:21771 2014-08-12 17:11:38.404254 2014-08-13 18:06:06.385097 7f5c973a57a0 20 read_log 194215'1193 (0'0) modify 5d6f6f6b/benchmark_data_storage0111-ib.pocstor_19374_object21877/head//1 by client.1048726.0:21878 2014-08-12 17:11:39.417066 2014-08-13 18:06:06.385119 7f5c973a57a0 20 read_log 194215'1194 (0'0) modify 6a5b3deb/benchmark_data_storage0111-ib.pocstor_19374_object21945/head//1 by client.1048726.0:21946 2014-08-12 17:11:39.685592 2014-08-13 18:06:06.385140 7f5c973a57a0 20 read_log 194215'1195 (0'0) modify 66a13f2b/benchmark_data_storage0111-ib.pocstor_19374_object22113/head//1 by client.1048726.0:22114 2014-08-12 17:11:40.338964 2014-08-13 18:06:06.385175 7f5c973a57a0 20 read_log 194215'1196 (0'0) modify 207ab2eb/benchmark_data_storage0111-ib.pocstor_19374_object22174/head//1 by client.1048726.0:22175 2014-08-12 17:11:40.569044 2014-08-13 18:06:06.385198 7f5c973a57a0 20 read_log 194215'1197 (0'0) modify 752cbeab/benchmark_data_storage0111-ib.pocstor_19374_object22208/head//1 by client.1048726.0:22209 2014-08-12 17:11:40.693348 2014-08-13 18:06:06.385220 7f5c973a57a0 20 read_log 194215'1198 (0'0) modify 5f7e08eb/benchmark_data_storage0111-ib.pocstor_19374_object22348/head//1 by client.1048726.0:22349 2014-08-12 17:11:41.214701 2014-08-13 18:06:06.385241 7f5c973a57a0 20 read_log 194215'1199 (0'0) modify 1c91296b/benchmark_data_storage0111-ib.pocstor_19374_object22399/head//1 by client.1048726.0:22400 2014-08-12 17:11:41.403636 2014-08-13 18:06:06.385263 7f5c973a57a0 20 read_log 194215'1200 (0'0) modify ed68a1eb/benchmark_data_storage0111-ib.pocstor_19374_object22466/head//1 by client.1048726.0:22467 2014-08-12 17:11:41.711256 2014-08-13 18:06:06.385284 7f5c973a57a0 20 read_log 194215'1201 (0'0) modify db847beb/benchmark_data_storage0111-ib.pocstor_19374_object22490/head//1 by client.1048726.0:22491 2014-08-12 17:11:41.840569 2014-08-13 18:06:06.385311 7f5c973a57a0 20 read_log 194215'1202 (0'0) modify 9b61f7eb/benchmark_data_storage0111-ib.pocstor_19374_object22596/head//1 by client.1048726.0:22597 2014-08-12 17:11:42.312656 2014-08-13 18:06:06.385337 7f5c973a57a0 20 read_log 194215'1203 (0'0) modify 8df4ebab/benchmark_data_storage0111-ib.pocstor_19374_object22647/head//1 by client.1048726.0:22648 2014-08-12 17:11:42.552403 2014-08-13 18:06:06.385374 7f5c973a57a0 20 read_log 194215'1204 (0'0) modify ade90cab/benchmark_data_storage0111-ib.pocstor_19374_object22661/head//1 by client.1048726.0:22662 2014-08-12 17:11:42.626530 2014-08-13 18:06:06.385407 7f5c973a57a0 20 read_log 194215'1205 (0'0) modify d2df432b/benchmark_data_storage0111-ib.pocstor_19374_object22688/head//1 by client.1048726.0:22689 2014-08-12 17:11:42.727273 2014-08-13 18:06:06.385432 7f5c973a57a0 20 read_log 194215'1206 (0'0) modify f0a1f42b/benchmark_data_storage0111-ib.pocstor_19374_object22740/head//1 by client.1048726.0:22741 2014-08-12 17:11:42.942474 2014-08-13 18:06:06.385473 7f5c973a57a0 20 read_log 194215'1207 (0'0) modify b467b16b/benchmark_data_storage0111-ib.pocstor_19374_object22893/head//1 by client.1048726.0:22894 2014-08-12 17:11:43.526333 2014-08-13 18:06:06.385501 7f5c973a57a0 20 read_log 194215'1208 (0'0) modify 2febd7ab/benchmark_data_storage0111-ib.pocstor_19374_object22894/head//1 by client.1048726.0:22895 2014-08-12 17:11:43.530043 2014-08-13 18:06:06.385540 7f5c973a57a0 20 read_log 194215'1209 (0'0) modify e0db75ab/benchmark_data_storage0111-ib.pocstor_19374_object22965/head//1 by client.1048726.0:22966 2014-08-12 17:11:43.911346 2014-08-13 18:06:06.385576 7f5c973a57a0 20 read_log 194215'1210 (0'0) modify 6344e4eb/benchmark_data_storage0111-ib.pocstor_19374_object22992/head//1 by client.1048726.0:22993 2014-08-12 17:11:44.143999 2014-08-13 18:06:06.385603 7f5c973a57a0 20 read_log 194215'1211 (0'0) modify 7ba9fd2b/benchmark_data_storage0111-ib.pocstor_19374_object23012/head//1 by client.1048726.0:23013 2014-08-12 17:11:44.237251 2014-08-13 18:06:06.385633 7f5c973a57a0 20 read_log 194215'1212 (0'0) modify 8666e06b/benchmark_data_storage0111-ib.pocstor_19374_object23095/head//1 by client.1048726.0:23096 2014-08-12 17:11:44.628537 2014-08-13 18:06:06.385660 7f5c973a57a0 20 read_log 194215'1213 (0'0) modify ed901d2b/benchmark_data_storage0111-ib.pocstor_19374_object23113/head//1 by client.1048726.0:23114 2014-08-12 17:11:44.719472 2014-08-13 18:06:06.385682 7f5c973a57a0 20 read_log 194215'1214 (0'0) modify 84443b6b/benchmark_data_storage0111-ib.pocstor_19374_object23132/head//1 by client.1048726.0:23133 2014-08-12 17:11:44.788734 2014-08-13 18:06:06.385708 7f5c973a57a0 20 read_log 194215'1215 (0'0) modify 8cff746b/benchmark_data_storage0111-ib.pocstor_19374_object23361/head//1 by client.1048726.0:23362 2014-08-12 17:11:46.106681 2014-08-13 18:06:06.385734 7f5c973a57a0 20 read_log 194215'1216 (0'0) modify 4edb3b2b/benchmark_data_storage0111-ib.pocstor_19374_object23379/head//1 by client.1048726.0:23380 2014-08-12 17:11:46.174136 2014-08-13 18:06:06.385781 7f5c973a57a0 20 read_log 194215'1217 (0'0) modify 1c53ca2b/benchmark_data_storage0111-ib.pocstor_19374_object23426/head//1 by client.1048726.0:23427 2014-08-12 17:11:46.350067 2014-08-13 18:06:06.385809 7f5c973a57a0 20 read_log 194215'1218 (0'0) modify fa5b282b/benchmark_data_storage0111-ib.pocstor_19374_object23429/head//1 by client.1048726.0:23430 2014-08-12 17:11:46.361085 2014-08-13 18:06:06.385845 7f5c973a57a0 20 read_log 194215'1219 (0'0) modify 5334536b/benchmark_data_storage0111-ib.pocstor_19374_object23433/head//1 by client.1048726.0:23434 2014-08-12 17:11:46.376233 2014-08-13 18:06:06.385873 7f5c973a57a0 20 read_log 194215'1220 (0'0) modify 6211b72b/benchmark_data_storage0111-ib.pocstor_19374_object23453/head//1 by client.1048726.0:23454 2014-08-12 17:11:46.452007 2014-08-13 18:06:06.385896 7f5c973a57a0 20 read_log 194215'1221 (0'0) modify 2514846b/benchmark_data_storage0111-ib.pocstor_19374_object23592/head//1 by client.1048726.0:23593 2014-08-12 17:11:47.010317 2014-08-13 18:06:06.385918 7f5c973a57a0 20 read_log 194215'1222 (0'0) modify aae84e2b/benchmark_data_storage0111-ib.pocstor_19374_object23706/head//1 by client.1048726.0:23707 2014-08-12 17:11:47.503011 2014-08-13 18:06:06.385939 7f5c973a57a0 20 read_log 194215'1223 (0'0) modify b9e840ab/benchmark_data_storage0111-ib.pocstor_19374_object23839/head//1 by client.1048726.0:23840 2014-08-12 17:11:48.002145 2014-08-13 18:06:06.386022 7f5c973a57a0 20 read_log 194215'1224 (0'0) modify 4cb71eb/benchmark_data_storage0111-ib.pocstor_19374_object23922/head//1 by client.1048726.0:23923 2014-08-12 17:11:48.315188 2014-08-13 18:06:06.386051 7f5c973a57a0 20 read_log 194215'1225 (0'0) modify e4b2e06b/benchmark_data_storage0111-ib.pocstor_19374_object23948/head//1 by client.1048726.0:23949 2014-08-12 17:11:48.412255 2014-08-13 18:06:06.386074 7f5c973a57a0 20 read_log 194215'1226 (0'0) modify d835436b/benchmark_data_storage0111-ib.pocstor_19374_object24094/head//1 by client.1048726.0:24095 2014-08-12 17:11:48.970521 2014-08-13 18:06:06.386095 7f5c973a57a0 20 read_log 194215'1227 (0'0) modify a7483a2b/benchmark_data_storage0111-ib.pocstor_19374_object24133/head//1 by client.1048726.0:24134 2014-08-12 17:11:49.165289 2014-08-13 18:06:06.386119 7f5c973a57a0 20 read_log 194215'1228 (0'0) modify 4afb40eb/benchmark_data_storage0111-ib.pocstor_19374_object24160/head//1 by client.1048726.0:24161 2014-08-12 17:11:49.424597 2014-08-13 18:06:06.386141 7f5c973a57a0 20 read_log 194215'1229 (0'0) modify 273f5cab/benchmark_data_storage0111-ib.pocstor_19374_object24162/head//1 by client.1048726.0:24163 2014-08-12 17:11:49.437223 2014-08-13 18:06:06.386177 7f5c973a57a0 20 read_log 194215'1230 (0'0) modify 438a14eb/benchmark_data_storage0111-ib.pocstor_19374_object24263/head//1 by client.1048726.0:24264 2014-08-12 17:11:49.898535 2014-08-13 18:06:06.386200 7f5c973a57a0 20 read_log 194215'1231 (0'0) modify 33c0b76b/benchmark_data_storage0111-ib.pocstor_19374_object24565/head//1 by client.1048726.0:24566 2014-08-12 17:11:51.610931 2014-08-13 18:06:06.386222 7f5c973a57a0 20 read_log 194215'1232 (0'0) modify c08eceab/benchmark_data_storage0111-ib.pocstor_19374_object24597/head//1 by client.1048726.0:24598 2014-08-12 17:11:51.732984 2014-08-13 18:06:06.386244 7f5c973a57a0 20 read_log 194215'1233 (0'0) modify befd7d6b/benchmark_data_storage0111-ib.pocstor_19374_object24728/head//1 by client.1048726.0:24729 2014-08-12 17:11:52.255888 2014-08-13 18:06:06.386266 7f5c973a57a0 20 read_log 194215'1234 (0'0) modify e425ca6b/benchmark_data_storage0111-ib.pocstor_19374_object24734/head//1 by client.1048726.0:24735 2014-08-12 17:11:52.278843 2014-08-13 18:06:06.386288 7f5c973a57a0 20 read_log 194215'1235 (0'0) modify 961e2f2b/benchmark_data_storage0111-ib.pocstor_19374_object24748/head//1 by client.1048726.0:24749 2014-08-12 17:11:52.345353 2014-08-13 18:06:06.386315 7f5c973a57a0 20 read_log 194215'1236 (0'0) modify 4a2485eb/benchmark_data_storage0111-ib.pocstor_19374_object24798/head//1 by client.1048726.0:24799 2014-08-12 17:11:52.679872 2014-08-13 18:06:06.386337 7f5c973a57a0 20 read_log 194215'1237 (0'0) modify 66eb74eb/benchmark_data_storage0111-ib.pocstor_19374_object24869/head//1 by client.1048726.0:24870 2014-08-12 17:11:53.059794 2014-08-13 18:06:06.386364 7f5c973a57a0 20 read_log 194215'1238 (0'0) modify cdd7d8eb/benchmark_data_storage0111-ib.pocstor_19374_object24973/head//1 by client.1048726.0:24974 2014-08-12 17:11:53.452075 2014-08-13 18:06:06.386386 7f5c973a57a0 20 read_log 194215'1239 (0'0) modify a8ff1b2b/benchmark_data_storage0111-ib.pocstor_19374_object25163/head//1 by client.1048726.0:25164 2014-08-12 17:11:54.516344 2014-08-13 18:06:06.386432 7f5c973a57a0 20 read_log 194215'1240 (0'0) modify 80a012eb/benchmark_data_storage0111-ib.pocstor_19374_object25209/head//1 by client.1048726.0:25210 2014-08-12 17:11:54.712278 2014-08-13 18:06:06.386463 7f5c973a57a0 20 read_log 194215'1241 (0'0) modify 6a06c22b/benchmark_data_storage0111-ib.pocstor_19374_object25224/head//1 by client.1048726.0:25225 2014-08-12 17:11:54.768778 2014-08-13 18:06:06.386486 7f5c973a57a0 20 read_log 194215'1242 (0'0) modify 1c47e46b/benchmark_data_storage0111-ib.pocstor_19374_object25488/head//1 by client.1048726.0:25489 2014-08-12 17:11:56.124305 2014-08-13 18:06:06.386526 7f5c973a57a0 20 read_log 194215'1243 (0'0) modify d92e436b/benchmark_data_storage0111-ib.pocstor_19374_object25630/head//1 by client.1048726.0:25631 2014-08-12 17:11:56.760259 2014-08-13 18:06:06.386565 7f5c973a57a0 20 read_log 194215'1244 (0'0) modify abae06ab/benchmark_data_storage0111-ib.pocstor_19374_object25701/head//1 by client.1048726.0:25702 2014-08-12 17:11:57.038710 2014-08-13 18:06:06.386592 7f5c973a57a0 20 read_log 194215'1245 (0'0) modify 63f6f26b/benchmark_data_storage0111-ib.pocstor_19374_object25733/head//1 by client.1048726.0:25734 2014-08-12 17:11:57.157771 2014-08-13 18:06:06.386614 7f5c973a57a0 20 read_log 194215'1246 (0'0) modify 65c6a56b/benchmark_data_storage0111-ib.pocstor_19374_object25806/head//1 by client.1048726.0:25807 2014-08-12 17:11:57.429477 2014-08-13 18:06:06.386663 7f5c973a57a0 20 read_log 194215'1247 (0'0) modify f2f70f6b/benchmark_data_storage0111-ib.pocstor_19374_object25849/head//1 by client.1048726.0:25850 2014-08-12 17:11:57.588962 2014-08-13 18:06:06.386693 7f5c973a57a0 20 read_log 194215'1248 (0'0) modify c4335c2b/benchmark_data_storage0111-ib.pocstor_19374_object25922/head//1 by client.1048726.0:25923 2014-08-12 17:11:57.923719 2014-08-13 18:06:06.386720 7f5c973a57a0 20 read_log 194215'1249 (0'0) modify 559f96eb/benchmark_data_storage0111-ib.pocstor_19374_object26064/head//1 by client.1048726.0:26065 2014-08-12 17:11:58.551746 2014-08-13 18:06:06.386747 7f5c973a57a0 20 read_log 194215'1250 (0'0) modify b6adc82b/benchmark_data_storage0111-ib.pocstor_19374_object26221/head//1 by client.1048726.0:26222 2014-08-12 17:11:59.139076 2014-08-13 18:06:06.386772 7f5c973a57a0 20 read_log 194215'1251 (0'0) modify 76fd52ab/benchmark_data_storage0111-ib.pocstor_19374_object26229/head//1 by client.1048726.0:26230 2014-08-12 17:11:59.168189 2014-08-13 18:06:06.386817 7f5c973a57a0 20 read_log 194215'1252 (0'0) modify 94dc9c2b/benchmark_data_storage0111-ib.pocstor_19374_object26322/head//1 by client.1048726.0:26323 2014-08-12 17:11:59.523186 2014-08-13 18:06:06.386848 7f5c973a57a0 20 read_log 194215'1253 (0'0) modify 7560a26b/benchmark_data_storage0111-ib.pocstor_19374_object26345/head//1 by client.1048726.0:26346 2014-08-12 17:11:59.608713 2014-08-13 18:06:06.386870 7f5c973a57a0 20 read_log 194215'1254 (0'0) modify 1f77502b/benchmark_data_storage0111-ib.pocstor_19374_object26385/head//1 by client.1048726.0:26386 2014-08-12 17:11:59.764776 2014-08-13 18:06:06.386901 7f5c973a57a0 20 read_log 194215'1255 (0'0) modify 6b90772b/benchmark_data_storage0111-ib.pocstor_19374_object26540/head//1 by client.1048726.0:26541 2014-08-12 17:12:00.347803 2014-08-13 18:06:06.386978 7f5c973a57a0 20 read_log 194215'1256 (194215'839) delete 6fb923eb/benchmark_data_storage0111-ib.pocstor_19374_object0/head//1 by client.1048726.0:26608 2014-08-12 17:12:01.646592 2014-08-13 18:06:06.387007 7f5c973a57a0 20 read_log 194215'1257 (194215'840) delete f1c09dab/benchmark_data_storage0111-ib.pocstor_19374_object53/head//1 by client.1048726.0:26661 2014-08-12 17:12:01.665773 2014-08-13 18:06:06.387034 7f5c973a57a0 20 read_log 194215'1258 (194215'841) delete 322ce82b/benchmark_data_storage0111-ib.pocstor_19374_object55/head//1 by client.1048726.0:26663 2014-08-12 17:12:01.665897 2014-08-13 18:06:06.387056 7f5c973a57a0 20 read_log 194215'1259 (194215'842) delete 42b2c8ab/benchmark_data_storage0111-ib.pocstor_19374_object77/head//1 by client.1048726.0:26685 2014-08-12 17:12:01.681085 2014-08-13 18:06:06.387096 7f5c973a57a0 20 read_log 194215'1260 (194215'843) delete 3792f3eb/benchmark_data_storage0111-ib.pocstor_19374_object104/head//1 by client.1048726.0:26712 2014-08-12 17:12:01.698569 2014-08-13 18:06:06.387124 7f5c973a57a0 20 read_log 194215'1261 (194215'844) delete c21e90ab/benchmark_data_storage0111-ib.pocstor_19374_object106/head//1 by client.1048726.0:26714 2014-08-12 17:12:01.699070 2014-08-13 18:06:06.387154 7f5c973a57a0 20 read_log 194215'1262 (194215'845) delete f13767eb/benchmark_data_storage0111-ib.pocstor_19374_object112/head//1 by client.1048726.0:26720 2014-08-12 17:12:01.703304 2014-08-13 18:06:06.387206 7f5c973a57a0 20 read_log 194215'1263 (194215'846) delete 90f4b5eb/benchmark_data_storage0111-ib.pocstor_19374_object117/head//1 by client.1048726.0:26725 2014-08-12 17:12:01.708569 2014-08-13 18:06:06.387246 7f5c973a57a0 20 read_log 194215'1264 (194215'847) delete ec96382b/benchmark_data_storage0111-ib.pocstor_19374_object134/head//1 by client.1048726.0:26742 2014-08-12 17:12:01.726473 2014-08-13 18:06:06.387272 7f5c973a57a0 20 read_log 194215'1265 (194215'848) delete 58c1d5eb/benchmark_data_storage0111-ib.pocstor_19374_object146/head//1 by client.1048726.0:26754 2014-08-12 17:12:01.733928 2014-08-13 18:06:06.387302 7f5c973a57a0 20 read_log 194215'1266 (194215'849) delete fb95d6ab/benchmark_data_storage0111-ib.pocstor_19374_object204/head//1 by client.1048726.0:26812 2014-08-12 17:12:01.805343 2014-08-13 18:06:06.387329 7f5c973a57a0 20 read_log 194215'1267 (194215'850) delete b112016b/benchmark_data_storage0111-ib.pocstor_19374_object273/head//1 by client.1048726.0:26881 2014-08-12 17:12:01.970029 2014-08-13 18:06:06.387352 7f5c973a57a0 20 read_log 194215'1268 (194215'851) delete a02a832b/benchmark_data_storage0111-ib.pocstor_19374_object284/head//1 by client.1048726.0:26892 2014-08-12 17:12:01.978514 2014-08-13 18:06:06.387374 7f5c973a57a0 20 read_log 194215'1269 (194215'852) delete a5225cab/benchmark_data_storage0111-ib.pocstor_19374_object539/head//1 by client.1048726.0:27147 2014-08-12 17:12:03.613247 2014-08-13 18:06:06.387415 7f5c973a57a0 20 read_log 194215'1270 (194215'853) delete a60191ab/benchmark_data_storage0111-ib.pocstor_19374_object649/head//1 by client.1048726.0:27257 2014-08-12 17:12:03.717744 2014-08-13 18:06:06.387438 7f5c973a57a0 20 read_log 194215'1271 (194215'854) delete 6fc2f02b/benchmark_data_storage0111-ib.pocstor_19374_object659/head//1 by client.1048726.0:27267 2014-08-12 17:12:03.725953 2014-08-13 18:06:06.387460 7f5c973a57a0 20 read_log 194215'1272 (194215'855) delete 7f48662b/benchmark_data_storage0111-ib.pocstor_19374_object1055/head//1 by client.1048726.0:27663 2014-08-12 17:12:04.650034 2014-08-13 18:06:06.387482 7f5c973a57a0 20 read_log 194215'1273 (194215'856) delete 17650d2b/benchmark_data_storage0111-ib.pocstor_19374_object1157/head//1 by client.1048726.0:27765 2014-08-12 17:12:04.690623 2014-08-13 18:06:06.387504 7f5c973a57a0 20 read_log 194215'1274 (194215'857) delete 39bc27ab/benchmark_data_storage0111-ib.pocstor_19374_object1180/head//1 by client.1048726.0:27788 2014-08-12 17:12:04.699424 2014-08-13 18:06:06.387526 7f5c973a57a0 20 read_log 194215'1275 (194215'858) delete e51879ab/benchmark_data_storage0111-ib.pocstor_19374_object1244/head//1 by client.1048726.0:27852 2014-08-12 17:12:04.726231 2014-08-13 18:06:06.387556 7f5c973a57a0 20 read_log 194215'1276 (194215'859) delete fefa32eb/benchmark_data_storage0111-ib.pocstor_19374_object1265/head//1 by client.1048726.0:27873 2014-08-12 17:12:04.734258 2014-08-13 18:06:06.387579 7f5c973a57a0 20 read_log 194215'1277 (194215'860) delete ee0c2aab/benchmark_data_storage0111-ib.pocstor_19374_object1278/head//1 by client.1048726.0:27886 2014-08-12 17:12:04.743340 2014-08-13 18:06:06.387605 7f5c973a57a0 20 read_log 194215'1278 (194215'861) delete 3560e9ab/benchmark_data_storage0111-ib.pocstor_19374_object1318/head//1 by client.1048726.0:27926 2014-08-12 17:12:04.760426 2014-08-13 18:06:06.387627 7f5c973a57a0 20 read_log 194215'1279 (194215'862) delete 510c1a6b/benchmark_data_storage0111-ib.pocstor_19374_object1337/head//1 by client.1048726.0:27945 2014-08-12 17:12:04.771262 2014-08-13 18:06:06.387648 7f5c973a57a0 20 read_log 194215'1280 (194215'863) delete 2298276b/benchmark_data_storage0111-ib.pocstor_19374_object1339/head//1 by client.1048726.0:27947 2014-08-12 17:12:04.772474 2014-08-13 18:06:06.387670 7f5c973a57a0 20 read_log 194215'1281 (194215'864) delete 6340222b/benchmark_data_storage0111-ib.pocstor_19374_object1349/head//1 by client.1048726.0:27957 2014-08-12 17:12:04.777947 2014-08-13 18:06:06.387694 7f5c973a57a0 20 read_log 194215'1282 (194215'865) delete 96f019ab/benchmark_data_storage0111-ib.pocstor_19374_object1487/head//1 by client.1048726.0:28095 2014-08-12 17:12:05.020409 2014-08-13 18:06:06.387716 7f5c973a57a0 20 read_log 194215'1283 (194215'866) delete 309188eb/benchmark_data_storage0111-ib.pocstor_19374_object1517/head//1 by client.1048726.0:28125 2014-08-12 17:12:05.050683 2014-08-13 18:06:06.387749 7f5c973a57a0 20 read_log 194215'1284 (194215'867) delete 5e9b32eb/benchmark_data_storage0111-ib.pocstor_19374_object1568/head//1 by client.1048726.0:28176 2014-08-12 17:12:05.080999 2014-08-13 18:06:06.387776 7f5c973a57a0 20 read_log 194215'1285 (194215'868) delete bb2497ab/benchmark_data_storage0111-ib.pocstor_19374_object1632/head//1 by client.1048726.0:28240 2014-08-12 17:12:05.101271 2014-08-13 18:06:06.387798 7f5c973a57a0 20 read_log 194215'1286 (194215'869) delete cf5e382b/benchmark_data_storage0111-ib.pocstor_19374_object1744/head//1 by client.1048726.0:28352 2014-08-12 17:12:05.136706 2014-08-13 18:06:06.387820 7f5c973a57a0 20 read_log 194215'1287 (194215'870) delete 3c5c7dab/benchmark_data_storage0111-ib.pocstor_19374_object1756/head//1 by client.1048726.0:28364 2014-08-12 17:12:05.139966 2014-08-13 18:06:06.387842 7f5c973a57a0 20 read_log 194215'1288 (194215'871) delete 724fd5eb/benchmark_data_storage0111-ib.pocstor_19374_object1757/head//1 by client.1048726.0:28365 2014-08-12 17:12:05.140017 2014-08-13 18:06:06.387864 7f5c973a57a0 20 read_log 194215'1289 (194215'872) delete 912098ab/benchmark_data_storage0111-ib.pocstor_19374_object1776/head//1 by client.1048726.0:28384 2014-08-12 17:12:05.147015 2014-08-13 18:06:06.387886 7f5c973a57a0 20 read_log 194215'1290 (194215'873) delete 3939e16b/benchmark_data_storage0111-ib.pocstor_19374_object1894/head//1 by client.1048726.0:28502 2014-08-12 17:12:05.188935 2014-08-13 18:06:06.387907 7f5c973a57a0 20 read_log 194215'1291 (194215'874) delete 6a0792eb/benchmark_data_storage0111-ib.pocstor_19374_object1896/head//1 by client.1048726.0:28504 2014-08-12 17:12:05.189089 2014-08-13 18:06:06.387929 7f5c973a57a0 20 read_log 194215'1292 (194215'875) delete 6f099eeb/benchmark_data_storage0111-ib.pocstor_19374_object2059/head//1 by client.1048726.0:28667 2014-08-12 17:12:05.242177 2014-08-13 18:06:06.387951 7f5c973a57a0 20 read_log 194215'1293 (194215'876) delete eafb902b/benchmark_data_storage0111-ib.pocstor_19374_object2077/head//1 by client.1048726.0:28685 2014-08-12 17:12:05.248508 2014-08-13 18:06:06.387972 7f5c973a57a0 20 read_log 194215'1294 (194215'877) delete 946c36eb/benchmark_data_storage0111-ib.pocstor_19374_object2102/head//1 by client.1048726.0:28710 2014-08-12 17:12:05.258439 2014-08-13 18:06:06.387995 7f5c973a57a0 20 read_log 194215'1295 (194215'878) delete f47e62b/benchmark_data_storage0111-ib.pocstor_19374_object2180/head//1 by client.1048726.0:28788 2014-08-12 17:12:05.287419 2014-08-13 18:06:06.388017 7f5c973a57a0 20 read_log 194215'1296 (194215'879) delete 934f442b/benchmark_data_storage0111-ib.pocstor_19374_object2262/head//1 by client.1048726.0:28870 2014-08-12 17:12:05.327409 2014-08-13 18:06:06.388039 7f5c973a57a0 20 read_log 194215'1297 (194215'880) delete 7f57086b/benchmark_data_storage0111-ib.pocstor_19374_object2343/head//1 by client.1048726.0:28951 2014-08-12 17:12:05.359161 2014-08-13 18:06:06.388063 7f5c973a57a0 20 read_log 194215'1298 (194215'881) delete 13d2392b/benchmark_data_storage0111-ib.pocstor_19374_object2352/head//1 by client.1048726.0:28960 2014-08-12 17:12:05.362707 2014-08-13 18:06:06.388088 7f5c973a57a0 20 read_log 194215'1299 (194215'882) delete 3d536ceb/benchmark_data_storage0111-ib.pocstor_19374_object2361/head//1 by client.1048726.0:28969 2014-08-12 17:12:05.366551 2014-08-13 18:06:06.388110 7f5c973a57a0 20 read_log 194215'1300 (194215'883) delete c6e9f26b/benchmark_data_storage0111-ib.pocstor_19374_object2394/head//1 by client.1048726.0:29002 2014-08-12 17:12:05.401353 2014-08-13 18:06:06.388135 7f5c973a57a0 20 read_log 194215'1301 (194215'884) delete 85e6a52b/benchmark_data_storage0111-ib.pocstor_19374_object2418/head//1 by client.1048726.0:29026 2014-08-12 17:12:05.476470 2014-08-13 18:06:06.388166 7f5c973a57a0 20 read_log 194215'1302 (194215'885) delete c7cc53ab/benchmark_data_storage0111-ib.pocstor_19374_object2432/head//1 by client.1048726.0:29040 2014-08-12 17:12:05.483029 2014-08-13 18:06:06.388192 7f5c973a57a0 20 read_log 194215'1303 (194215'886) delete 3a7fe4ab/benchmark_data_storage0111-ib.pocstor_19374_object2594/head//1 by client.1048726.0:29202 2014-08-12 17:12:05.607483 2014-08-13 18:06:06.388230 7f5c973a57a0 20 read_log 194215'1304 (194215'887) delete 4305176b/benchmark_data_storage0111-ib.pocstor_19374_object2684/head//1 by client.1048726.0:29292 2014-08-12 17:12:06.259135 2014-08-13 18:06:06.388256 7f5c973a57a0 20 read_log 194215'1305 (194215'888) delete e7b7062b/benchmark_data_storage0111-ib.pocstor_19374_object2725/head//1 by client.1048726.0:29333 2014-08-12 17:12:06.306363 2014-08-13 18:06:06.388278 7f5c973a57a0 20 read_log 194215'1306 (194215'889) delete a1e105ab/benchmark_data_storage0111-ib.pocstor_19374_object2744/head//1 by client.1048726.0:29352 2014-08-12 17:12:06.316619 2014-08-13 18:06:06.388300 7f5c973a57a0 20 read_log 194215'1307 (194215'890) delete 9bee462b/benchmark_data_storage0111-ib.pocstor_19374_object2763/head//1 by client.1048726.0:29371 2014-08-12 17:12:06.322373 2014-08-13 18:06:06.388322 7f5c973a57a0 20 read_log 194215'1308 (194215'891) delete 2d10016b/benchmark_data_storage0111-ib.pocstor_19374_object2791/head//1 by client.1048726.0:29399 2014-08-12 17:12:06.334504 2014-08-13 18:06:06.388351 7f5c973a57a0 20 read_log 194215'1309 (194215'892) delete 7464b42b/benchmark_data_storage0111-ib.pocstor_19374_object2865/head//1 by client.1048726.0:29473 2014-08-12 17:12:06.375063 2014-08-13 18:06:06.388377 7f5c973a57a0 20 read_log 194215'1310 (194215'893) delete 6a28beb/benchmark_data_storage0111-ib.pocstor_19374_object2901/head//1 by client.1048726.0:29509 2014-08-12 17:12:06.403708 2014-08-13 18:06:06.388408 7f5c973a57a0 20 read_log 194215'1311 (194215'894) delete df15306b/benchmark_data_storage0111-ib.pocstor_19374_object2944/head//1 by client.1048726.0:29552 2014-08-12 17:12:06.459753 2014-08-13 18:06:06.388430 7f5c973a57a0 20 read_log 194215'1312 (194215'895) delete 505175ab/benchmark_data_storage0111-ib.pocstor_19374_object2950/head//1 by client.1048726.0:29558 2014-08-12 17:12:06.463067 2014-08-13 18:06:06.388452 7f5c973a57a0 20 read_log 194215'1313 (194215'896) delete 3fa3deeb/benchmark_data_storage0111-ib.pocstor_19374_object3045/head//1 by client.1048726.0:29653 2014-08-12 17:12:06.496969 2014-08-13 18:06:06.388499 7f5c973a57a0 20 read_log 194215'1314 (194215'897) delete f1bcb3ab/benchmark_data_storage0111-ib.pocstor_19374_object3073/head//1 by client.1048726.0:29681 2014-08-12 17:12:06.508057 2014-08-13 18:06:06.388522 7f5c973a57a0 20 read_log 194215'1315 (194215'898) delete 6eb5e6ab/benchmark_data_storage0111-ib.pocstor_19374_object3178/head//1 by client.1048726.0:29786 2014-08-12 17:12:06.540511 2014-08-13 18:06:06.388569 7f5c973a57a0 20 read_log 194215'1316 (194215'899) delete 71213f6b/benchmark_data_storage0111-ib.pocstor_19374_object3222/head//1 by client.1048726.0:29830 2014-08-12 17:12:06.553801 2014-08-13 18:06:06.388614 7f5c973a57a0 20 read_log 194215'1317 (194215'900) delete 2847d6ab/benchmark_data_storage0111-ib.pocstor_19374_object3286/head//1 by client.1048726.0:29894 2014-08-12 17:12:06.576296 2014-08-13 18:06:06.388637 7f5c973a57a0 20 read_log 194215'1318 (194215'901) delete b38c2f6b/benchmark_data_storage0111-ib.pocstor_19374_object3415/head//1 by client.1048726.0:30023 2014-08-12 17:12:06.615270 2014-08-13 18:06:06.388665 7f5c973a57a0 20 read_log 194215'1319 (194215'902) delete 518a656b/benchmark_data_storage0111-ib.pocstor_19374_object3454/head//1 by client.1048726.0:30062 2014-08-12 17:12:06.627505 2014-08-13 18:06:06.388688 7f5c973a57a0 20 read_log 194215'1320 (194215'903) delete cad818ab/benchmark_data_storage0111-ib.pocstor_19374_object3466/head//1 by client.1048726.0:30074 2014-08-12 17:12:06.631630 2014-08-13 18:06:06.388734 7f5c973a57a0 20 read_log 194215'1321 (194215'904) delete 52ed8aab/benchmark_data_storage0111-ib.pocstor_19374_object3511/head//1 by client.1048726.0:30119 2014-08-12 17:12:06.644221 2014-08-13 18:06:06.388760 7f5c973a57a0 20 read_log 194215'1322 (194215'905) delete 376b872b/benchmark_data_storage0111-ib.pocstor_19374_object3523/head//1 by client.1048726.0:30131 2014-08-12 17:12:06.647796 2014-08-13 18:06:06.388783 7f5c973a57a0 20 read_log 194215'1323 (194215'906) delete 69cdf92b/benchmark_data_storage0111-ib.pocstor_19374_object3584/head//1 by client.1048726.0:30192 2014-08-12 17:12:06.672741 2014-08-13 18:06:06.388817 7f5c973a57a0 20 read_log 194215'1324 (194215'907) delete 5dad4cab/benchmark_data_storage0111-ib.pocstor_19374_object3612/head//1 by client.1048726.0:30220 2014-08-12 17:12:06.684853 2014-08-13 18:06:06.388839 7f5c973a57a0 20 read_log 194215'1325 (194215'908) delete 8a29ddeb/benchmark_data_storage0111-ib.pocstor_19374_object3677/head//1 by client.1048726.0:30285 2014-08-12 17:12:06.772314 2014-08-13 18:06:06.388862 7f5c973a57a0 20 read_log 194215'1326 (194215'909) delete cb77c3ab/benchmark_data_storage0111-ib.pocstor_19374_object3717/head//1 by client.1048726.0:30325 2014-08-12 17:12:06.796707 2014-08-13 18:06:06.388884 7f5c973a57a0 20 read_log 194215'1327 (194215'910) delete 7096f52b/benchmark_data_storage0111-ib.pocstor_19374_object3810/head//1 by client.1048726.0:30418 2014-08-12 17:12:06.841884 2014-08-13 18:06:06.388911 7f5c973a57a0 20 read_log 194215'1328 (194215'911) delete ee1fffeb/benchmark_data_storage0111-ib.pocstor_19374_object3837/head//1 by client.1048726.0:30445 2014-08-12 17:12:06.858370 2014-08-13 18:06:06.388933 7f5c973a57a0 20 read_log 194215'1329 (194215'912) delete a32a26eb/benchmark_data_storage0111-ib.pocstor_19374_object3838/head//1 by client.1048726.0:30446 2014-08-12 17:12:06.858972 2014-08-13 18:06:06.388955 7f5c973a57a0 20 read_log 194215'1330 (194215'913) delete 6fb1deeb/benchmark_data_storage0111-ib.pocstor_19374_object4048/head//1 by client.1048726.0:30656 2014-08-12 17:12:06.964556 2014-08-13 18:06:06.388977 7f5c973a57a0 20 read_log 194215'1331 (194215'914) delete 6c58c7ab/benchmark_data_storage0111-ib.pocstor_19374_object4059/head//1 by client.1048726.0:30667 2014-08-12 17:12:06.969486 2014-08-13 18:06:06.389005 7f5c973a57a0 20 read_log 194215'1332 (194215'915) delete 660f996b/benchmark_data_storage0111-ib.pocstor_19374_object4105/head//1 by client.1048726.0:30713 2014-08-12 17:12:06.999342 2014-08-13 18:06:06.389027 7f5c973a57a0 20 read_log 194215'1333 (194215'916) delete 612f78ab/benchmark_data_storage0111-ib.pocstor_19374_object4125/head//1 by client.1048726.0:30733 2014-08-12 17:12:07.010599 2014-08-13 18:06:06.389049 7f5c973a57a0 20 read_log 194215'1334 (194215'917) delete bd30982b/benchmark_data_storage0111-ib.pocstor_19374_object4136/head//1 by client.1048726.0:30744 2014-08-12 17:12:07.016489 2014-08-13 18:06:06.389073 7f5c973a57a0 20 read_log 194215'1335 (194215'918) delete d1ec98ab/benchmark_data_storage0111-ib.pocstor_19374_object4183/head//1 by client.1048726.0:30791 2014-08-12 17:12:07.107874 2014-08-13 18:06:06.389129 7f5c973a57a0 20 read_log 194215'1336 (194215'919) delete cd9ad76b/benchmark_data_storage0111-ib.pocstor_19374_object4187/head//1 by client.1048726.0:30795 2014-08-12 17:12:07.127328 2014-08-13 18:06:06.389160 7f5c973a57a0 20 read_log 194215'1337 (194215'920) delete 6236346b/benchmark_data_storage0111-ib.pocstor_19374_object4191/head//1 by client.1048726.0:30799 2014-08-12 17:12:07.136540 2014-08-13 18:06:06.389183 7f5c973a57a0 20 read_log 194215'1338 (194215'921) delete c47eb16b/benchmark_data_storage0111-ib.pocstor_19374_object4242/head//1 by client.1048726.0:30850 2014-08-12 17:12:07.156218 2014-08-13 18:06:06.389208 7f5c973a57a0 20 read_log 194215'1339 (194215'922) delete 7f4a59eb/benchmark_data_storage0111-ib.pocstor_19374_object4360/head//1 by client.1048726.0:30968 2014-08-12 17:12:07.196367 2014-08-13 18:06:06.389230 7f5c973a57a0 20 read_log 194215'1340 (194215'923) delete 3ac7842b/benchmark_data_storage0111-ib.pocstor_19374_object4440/head//1 by client.1048726.0:31048 2014-08-12 17:12:07.239846 2014-08-13 18:06:06.389251 7f5c973a57a0 20 read_log 194215'1341 (194215'924) delete 3be00eeb/benchmark_data_storage0111-ib.pocstor_19374_object4498/head//1 by client.1048726.0:31106 2014-08-12 17:12:07.285475 2014-08-13 18:06:06.389273 7f5c973a57a0 20 read_log 194215'1342 (194215'925) delete 20baeeab/benchmark_data_storage0111-ib.pocstor_19374_object4674/head//1 by client.1048726.0:31282 2014-08-12 17:12:07.362385 2014-08-13 18:06:06.389295 7f5c973a57a0 20 read_log 194215'1343 (194215'926) delete ae2e9d2b/benchmark_data_storage0111-ib.pocstor_19374_object4826/head//1 by client.1048726.0:31434 2014-08-12 17:12:07.425514 2014-08-13 18:06:06.389332 7f5c973a57a0 20 read_log 194215'1344 (194215'927) delete e1d72b2b/benchmark_data_storage0111-ib.pocstor_19374_object4840/head//1 by client.1048726.0:31448 2014-08-12 17:12:07.431609 2014-08-13 18:06:06.389359 7f5c973a57a0 20 read_log 194215'1345 (194215'928) delete 14b2daeb/benchmark_data_storage0111-ib.pocstor_19374_object4851/head//1 by client.1048726.0:31459 2014-08-12 17:12:07.433564 2014-08-13 18:06:06.389387 7f5c973a57a0 20 read_log 194215'1346 (194215'929) delete f058a86b/benchmark_data_storage0111-ib.pocstor_19374_object4891/head//1 by client.1048726.0:31499 2014-08-12 17:12:07.450015 2014-08-13 18:06:06.389413 7f5c973a57a0 20 read_log 194215'1347 (194215'930) delete 1cc0022b/benchmark_data_storage0111-ib.pocstor_19374_object4938/head//1 by client.1048726.0:31546 2014-08-12 17:12:07.470916 2014-08-13 18:06:06.389458 7f5c973a57a0 20 read_log 194215'1348 (194215'931) delete add0cc2b/benchmark_data_storage0111-ib.pocstor_19374_object5085/head//1 by client.1048726.0:31693 2014-08-12 17:12:07.520933 2014-08-13 18:06:06.389502 7f5c973a57a0 20 read_log 194215'1349 (194215'932) delete 942a0e2b/benchmark_data_storage0111-ib.pocstor_19374_object5225/head//1 by client.1048726.0:31833 2014-08-12 17:12:07.565376 2014-08-13 18:06:06.389530 7f5c973a57a0 20 read_log 194215'1350 (194215'933) delete b41983eb/benchmark_data_storage0111-ib.pocstor_19374_object5345/head//1 by client.1048726.0:31953 2014-08-12 17:12:07.603937 2014-08-13 18:06:06.389552 7f5c973a57a0 20 read_log 194215'1351 (194215'934) delete 6f06a4eb/benchmark_data_storage0111-ib.pocstor_19374_object5356/head//1 by client.1048726.0:31964 2014-08-12 17:12:07.608232 2014-08-13 18:06:06.389574 7f5c973a57a0 20 read_log 194215'1352 (194215'935) delete a176b/benchmark_data_storage0111-ib.pocstor_19374_object5547/head//1 by client.1048726.0:32155 2014-08-12 17:12:07.667029 2014-08-13 18:06:06.389596 7f5c973a57a0 20 read_log 194215'1353 (194215'936) delete 2bbd78ab/benchmark_data_storage0111-ib.pocstor_19374_object5562/head//1 by client.1048726.0:32170 2014-08-12 17:12:07.671544 2014-08-13 18:06:06.389617 7f5c973a57a0 20 read_log 194215'1354 (194215'937) delete 54ab51eb/benchmark_data_storage0111-ib.pocstor_19374_object5571/head//1 by client.1048726.0:32179 2014-08-12 17:12:07.676824 2014-08-13 18:06:06.389659 7f5c973a57a0 20 read_log 194215'1355 (194215'938) delete 53bf81eb/benchmark_data_storage0111-ib.pocstor_19374_object5590/head//1 by client.1048726.0:32198 2014-08-12 17:12:07.682597 2014-08-13 18:06:06.389733 7f5c973a57a0 20 read_log 194215'1356 (194215'939) delete 93b6132b/benchmark_data_storage0111-ib.pocstor_19374_object5651/head//1 by client.1048726.0:32259 2014-08-12 17:12:07.701780 2014-08-13 18:06:06.389767 7f5c973a57a0 20 read_log 194215'1357 (194215'940) delete 2f4423eb/benchmark_data_storage0111-ib.pocstor_19374_object5849/head//1 by client.1048726.0:32457 2014-08-12 17:12:07.764354 2014-08-13 18:06:06.389789 7f5c973a57a0 20 read_log 194215'1358 (194215'941) delete 37100d2b/benchmark_data_storage0111-ib.pocstor_19374_object5900/head//1 by client.1048726.0:32508 2014-08-12 17:12:07.779429 2014-08-13 18:06:06.389819 7f5c973a57a0 20 read_log 194215'1359 (194215'942) delete 2dd007ab/benchmark_data_storage0111-ib.pocstor_19374_object5917/head//1 by client.1048726.0:32525 2014-08-12 17:12:07.785188 2014-08-13 18:06:06.389842 7f5c973a57a0 20 read_log 194215'1360 (194215'943) delete 86a4fcab/benchmark_data_storage0111-ib.pocstor_19374_object5945/head//1 by client.1048726.0:32553 2014-08-12 17:12:07.792698 2014-08-13 18:06:06.389863 7f5c973a57a0 20 read_log 194215'1361 (194215'944) delete 920492b/benchmark_data_storage0111-ib.pocstor_19374_object5965/head//1 by client.1048726.0:32573 2014-08-12 17:12:07.797787 2014-08-13 18:06:06.389885 7f5c973a57a0 20 read_log 194215'1362 (194215'945) delete 1055bb2b/benchmark_data_storage0111-ib.pocstor_19374_object5969/head//1 by client.1048726.0:32577 2014-08-12 17:12:07.799086 2014-08-13 18:06:06.389907 7f5c973a57a0 20 read_log 194215'1363 (194215'946) delete 58e49beb/benchmark_data_storage0111-ib.pocstor_19374_object6067/head//1 by client.1048726.0:32675 2014-08-12 17:12:07.828662 2014-08-13 18:06:06.389974 7f5c973a57a0 20 read_log 194215'1364 (194215'947) delete f187cf6b/benchmark_data_storage0111-ib.pocstor_19374_object6085/head//1 by client.1048726.0:32693 2014-08-12 17:12:07.833464 2014-08-13 18:06:06.390003 7f5c973a57a0 20 read_log 194215'1365 (194215'948) delete 700ddfab/benchmark_data_storage0111-ib.pocstor_19374_object6199/head//1 by client.1048726.0:32807 2014-08-12 17:12:07.868800 2014-08-13 18:06:06.390025 7f5c973a57a0 20 read_log 194215'1366 (194215'949) delete fd0e98ab/benchmark_data_storage0111-ib.pocstor_19374_object6210/head//1 by client.1048726.0:32818 2014-08-12 17:12:07.871794 2014-08-13 18:06:06.390079 7f5c973a57a0 20 read_log 194215'1367 (194215'950) delete b470446b/benchmark_data_storage0111-ib.pocstor_19374_object6353/head//1 by client.1048726.0:32961 2014-08-12 17:12:07.917575 2014-08-13 18:06:06.390108 7f5c973a57a0 20 read_log 194215'1368 (194215'951) delete e85327ab/benchmark_data_storage0111-ib.pocstor_19374_object6362/head//1 by client.1048726.0:32970 2014-08-12 17:12:07.920429 2014-08-13 18:06:06.390130 7f5c973a57a0 20 read_log 194215'1369 (194215'952) delete e40363ab/benchmark_data_storage0111-ib.pocstor_19374_object6440/head//1 by client.1048726.0:33048 2014-08-12 17:12:07.946771 2014-08-13 18:06:06.390161 7f5c973a57a0 20 read_log 194215'1370 (194215'953) delete 2dd5496b/benchmark_data_storage0111-ib.pocstor_19374_object6446/head//1 by client.1048726.0:33054 2014-08-12 17:12:07.949162 2014-08-13 18:06:06.390219 7f5c973a57a0 20 read_log 194215'1371 (194215'954) delete 75bccceb/benchmark_data_storage0111-ib.pocstor_19374_object6497/head//1 by client.1048726.0:33105 2014-08-12 17:12:07.967320 2014-08-13 18:06:06.390291 7f5c973a57a0 20 read_log 194215'1372 (194215'955) delete df89832b/benchmark_data_storage0111-ib.pocstor_19374_object6513/head//1 by client.1048726.0:33121 2014-08-12 17:12:07.971893 2014-08-13 18:06:06.390355 7f5c973a57a0 20 read_log 194215'1373 (194215'956) delete f02cbe6b/benchmark_data_storage0111-ib.pocstor_19374_object6524/head//1 by client.1048726.0:33132 2014-08-12 17:12:07.975849 2014-08-13 18:06:06.390410 7f5c973a57a0 20 read_log 194215'1374 (194215'957) delete a3faf6b/benchmark_data_storage0111-ib.pocstor_19374_object6613/head//1 by client.1048726.0:33221 2014-08-12 17:12:08.003763 2014-08-13 18:06:06.390448 7f5c973a57a0 20 read_log 194215'1375 (194215'958) delete ff3983eb/benchmark_data_storage0111-ib.pocstor_19374_object6657/head//1 by client.1048726.0:33265 2014-08-12 17:12:08.016655 2014-08-13 18:06:06.390495 7f5c973a57a0 20 read_log 194215'1376 (194215'959) delete 1f301fab/benchmark_data_storage0111-ib.pocstor_19374_object6816/head//1 by client.1048726.0:33424 2014-08-12 17:12:08.064019 2014-08-13 18:06:06.390531 7f5c973a57a0 20 read_log 194215'1377 (194215'960) delete f6de92eb/benchmark_data_storage0111-ib.pocstor_19374_object6841/head//1 by client.1048726.0:33449 2014-08-12 17:12:08.073605 2014-08-13 18:06:06.390576 7f5c973a57a0 20 read_log 194215'1378 (194215'961) delete b6c1042b/benchmark_data_storage0111-ib.pocstor_19374_object6885/head//1 by client.1048726.0:33493 2014-08-12 17:12:08.084838 2014-08-13 18:06:06.390607 7f5c973a57a0 20 read_log 194215'1379 (194215'962) delete af36f9eb/benchmark_data_storage0111-ib.pocstor_19374_object6904/head//1 by client.1048726.0:33512 2014-08-12 17:12:08.090667 2014-08-13 18:06:06.390653 7f5c973a57a0 20 read_log 194215'1380 (194215'963) delete a5777ab/benchmark_data_storage0111-ib.pocstor_19374_object7029/head//1 by client.1048726.0:33637 2014-08-12 17:12:08.127986 2014-08-13 18:06:06.390676 7f5c973a57a0 20 read_log 194215'1381 (194215'964) delete 6919b6b/benchmark_data_storage0111-ib.pocstor_19374_object7059/head//1 by client.1048726.0:33667 2014-08-12 17:12:08.137666 2014-08-13 18:06:06.390716 7f5c973a57a0 20 read_log 194215'1382 (194215'965) delete efbee52b/benchmark_data_storage0111-ib.pocstor_19374_object7069/head//1 by client.1048726.0:33677 2014-08-12 17:12:08.140866 2014-08-13 18:06:06.390739 7f5c973a57a0 20 read_log 194215'1383 (194215'966) delete 5f8c13eb/benchmark_data_storage0111-ib.pocstor_19374_object7102/head//1 by client.1048726.0:33710 2014-08-12 17:12:08.151016 2014-08-13 18:06:06.390774 7f5c973a57a0 20 read_log 194215'1384 (194215'967) delete 651b9fab/benchmark_data_storage0111-ib.pocstor_19374_object7248/head//1 by client.1048726.0:33856 2014-08-12 17:12:08.212719 2014-08-13 18:06:06.390797 7f5c973a57a0 20 read_log 194215'1385 (194215'968) delete 4698106b/benchmark_data_storage0111-ib.pocstor_19374_object7302/head//1 by client.1048726.0:33910 2014-08-12 17:12:08.228384 2014-08-13 18:06:06.390818 7f5c973a57a0 20 read_log 194215'1386 (194215'969) delete 95b2bd2b/benchmark_data_storage0111-ib.pocstor_19374_object7355/head//1 by client.1048726.0:33963 2014-08-12 17:12:08.245593 2014-08-13 18:06:06.390840 7f5c973a57a0 20 read_log 194215'1387 (194215'970) delete 3565086b/benchmark_data_storage0111-ib.pocstor_19374_object7394/head//1 by client.1048726.0:34002 2014-08-12 17:12:08.256276 2014-08-13 18:06:06.390861 7f5c973a57a0 20 read_log 194215'1388 (194215'971) delete 7723e2eb/benchmark_data_storage0111-ib.pocstor_19374_object7432/head//1 by client.1048726.0:34040 2014-08-12 17:12:08.266736 2014-08-13 18:06:06.390883 7f5c973a57a0 20 read_log 194215'1389 (194215'972) delete 3876f66b/benchmark_data_storage0111-ib.pocstor_19374_object7456/head//1 by client.1048726.0:34064 2014-08-12 17:12:08.276472 2014-08-13 18:06:06.390905 7f5c973a57a0 20 read_log 194215'1390 (194215'973) delete 7c404ceb/benchmark_data_storage0111-ib.pocstor_19374_object7471/head//1 by client.1048726.0:34079 2014-08-12 17:12:08.281546 2014-08-13 18:06:06.390927 7f5c973a57a0 20 read_log 194215'1391 (194215'974) delete dee414ab/benchmark_data_storage0111-ib.pocstor_19374_object7498/head//1 by client.1048726.0:34106 2014-08-12 17:12:08.288998 2014-08-13 18:06:06.390948 7f5c973a57a0 20 read_log 194215'1392 (194215'975) delete b0b328ab/benchmark_data_storage0111-ib.pocstor_19374_object7503/head//1 by client.1048726.0:34111 2014-08-12 17:12:08.290744 2014-08-13 18:06:06.390970 7f5c973a57a0 20 read_log 194215'1393 (194215'976) delete fbaf77ab/benchmark_data_storage0111-ib.pocstor_19374_object7653/head//1 by client.1048726.0:34261 2014-08-12 17:12:08.337138 2014-08-13 18:06:06.390991 7f5c973a57a0 20 read_log 194215'1394 (194215'977) delete f080c82b/benchmark_data_storage0111-ib.pocstor_19374_object7664/head//1 by client.1048726.0:34272 2014-08-12 17:12:08.341162 2014-08-13 18:06:06.391013 7f5c973a57a0 20 read_log 194215'1395 (194215'978) delete cf53d6eb/benchmark_data_storage0111-ib.pocstor_19374_object7819/head//1 by client.1048726.0:34427 2014-08-12 17:12:08.389764 2014-08-13 18:06:06.391034 7f5c973a57a0 20 read_log 194215'1396 (194215'979) delete 7e7fc76b/benchmark_data_storage0111-ib.pocstor_19374_object7822/head//1 by client.1048726.0:34430 2014-08-12 17:12:08.390837 2014-08-13 18:06:06.391056 7f5c973a57a0 20 read_log 194215'1397 (194215'980) delete fa90ac2b/benchmark_data_storage0111-ib.pocstor_19374_object7839/head//1 by client.1048726.0:34447 2014-08-12 17:12:08.395610 2014-08-13 18:06:06.391077 7f5c973a57a0 20 read_log 194215'1398 (194215'981) delete 5918ea2b/benchmark_data_storage0111-ib.pocstor_19374_object7899/head//1 by client.1048726.0:34507 2014-08-12 17:12:08.415096 2014-08-13 18:06:06.391099 7f5c973a57a0 20 read_log 194215'1399 (194215'982) delete 4b04ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8097/head//1 by client.1048726.0:34705 2014-08-12 17:12:08.477220 2014-08-13 18:06:06.391120 7f5c973a57a0 20 read_log 194215'1400 (194215'983) delete 3b37c4ab/benchmark_data_storage0111-ib.pocstor_19374_object8171/head//1 by client.1048726.0:34779 2014-08-12 17:12:08.501921 2014-08-13 18:06:06.391142 7f5c973a57a0 20 read_log 194215'1401 (194215'984) delete fabb5dab/benchmark_data_storage0111-ib.pocstor_19374_object8252/head//1 by client.1048726.0:34860 2014-08-12 17:12:08.526699 2014-08-13 18:06:06.391179 7f5c973a57a0 20 read_log 194215'1402 (194215'985) delete be4caceb/benchmark_data_storage0111-ib.pocstor_19374_object8295/head//1 by client.1048726.0:34903 2014-08-12 17:12:08.539493 2014-08-13 18:06:06.391201 7f5c973a57a0 20 read_log 194215'1403 (194215'986) delete 95c4ed2b/benchmark_data_storage0111-ib.pocstor_19374_object8373/head//1 by client.1048726.0:34981 2014-08-12 17:12:08.566000 2014-08-13 18:06:06.391234 7f5c973a57a0 20 read_log 194215'1404 (194215'987) delete bfddf56b/benchmark_data_storage0111-ib.pocstor_19374_object8440/head//1 by client.1048726.0:35048 2014-08-12 17:12:08.585129 2014-08-13 18:06:06.391257 7f5c973a57a0 20 read_log 194215'1405 (194215'988) delete 63bb63eb/benchmark_data_storage0111-ib.pocstor_19374_object8570/head//1 by client.1048726.0:35178 2014-08-12 17:12:08.623071 2014-08-13 18:06:06.391278 7f5c973a57a0 20 read_log 194215'1406 (194215'989) delete 819622eb/benchmark_data_storage0111-ib.pocstor_19374_object8723/head//1 by client.1048726.0:35331 2014-08-12 17:12:08.677012 2014-08-13 18:06:06.391300 7f5c973a57a0 20 read_log 194215'1407 (194215'990) delete 872863eb/benchmark_data_storage0111-ib.pocstor_19374_object8755/head//1 by client.1048726.0:35363 2014-08-12 17:12:08.686847 2014-08-13 18:06:06.391322 7f5c973a57a0 20 read_log 194215'1408 (194215'991) delete ba99d16b/benchmark_data_storage0111-ib.pocstor_19374_object8788/head//1 by client.1048726.0:35396 2014-08-12 17:12:08.696027 2014-08-13 18:06:06.391348 7f5c973a57a0 20 read_log 194215'1409 (194215'992) delete 89e660ab/benchmark_data_storage0111-ib.pocstor_19374_object8796/head//1 by client.1048726.0:35404 2014-08-12 17:12:08.698316 2014-08-13 18:06:06.391371 7f5c973a57a0 20 read_log 194215'1410 (194215'993) delete 3d2f22b/benchmark_data_storage0111-ib.pocstor_19374_object8882/head//1 by client.1048726.0:35490 2014-08-12 17:12:08.723702 2014-08-13 18:06:06.391397 7f5c973a57a0 20 read_log 194215'1411 (194215'994) delete 5366ae2b/benchmark_data_storage0111-ib.pocstor_19374_object8986/head//1 by client.1048726.0:35594 2014-08-12 17:12:08.757162 2014-08-13 18:06:06.391420 7f5c973a57a0 20 read_log 194215'1412 (194215'995) delete e3a50aab/benchmark_data_storage0111-ib.pocstor_19374_object9008/head//1 by client.1048726.0:35616 2014-08-12 17:12:08.764539 2014-08-13 18:06:06.391441 7f5c973a57a0 20 read_log 194215'1413 (194215'996) delete 96aab4eb/benchmark_data_storage0111-ib.pocstor_19374_object9145/head//1 by client.1048726.0:35753 2014-08-12 17:12:08.805106 2014-08-13 18:06:06.391465 7f5c973a57a0 20 read_log 194215'1414 (194215'997) delete 14f1502b/benchmark_data_storage0111-ib.pocstor_19374_object9193/head//1 by client.1048726.0:35801 2014-08-12 17:12:08.820123 2014-08-13 18:06:06.391510 7f5c973a57a0 20 read_log 194215'1415 (194215'998) delete 6a91646b/benchmark_data_storage0111-ib.pocstor_19374_object9229/head//1 by client.1048726.0:35837 2014-08-12 17:12:08.832538 2014-08-13 18:06:06.391532 7f5c973a57a0 20 read_log 194215'1416 (194215'999) delete c93002eb/benchmark_data_storage0111-ib.pocstor_19374_object9287/head//1 by client.1048726.0:35895 2014-08-12 17:12:08.849201 2014-08-13 18:06:06.391566 7f5c973a57a0 20 read_log 194215'1417 (194215'1000) delete 2257b96b/benchmark_data_storage0111-ib.pocstor_19374_object9441/head//1 by client.1048726.0:36049 2014-08-12 17:12:08.899088 2014-08-13 18:06:06.391616 7f5c973a57a0 20 read_log 194215'1418 (194215'1001) delete 33077deb/benchmark_data_storage0111-ib.pocstor_19374_object9442/head//1 by client.1048726.0:36050 2014-08-12 17:12:08.899792 2014-08-13 18:06:06.391645 7f5c973a57a0 20 read_log 194215'1419 (194215'1002) delete e2ce132b/benchmark_data_storage0111-ib.pocstor_19374_object9679/head//1 by client.1048726.0:36287 2014-08-12 17:12:08.975858 2014-08-13 18:06:06.391667 7f5c973a57a0 20 read_log 194215'1420 (194215'1003) delete 9ce657ab/benchmark_data_storage0111-ib.pocstor_19374_object9686/head//1 by client.1048726.0:36294 2014-08-12 17:12:08.979043 2014-08-13 18:06:06.391689 7f5c973a57a0 20 read_log 194215'1421 (194215'1004) delete a535d1ab/benchmark_data_storage0111-ib.pocstor_19374_object9692/head//1 by client.1048726.0:36300 2014-08-12 17:12:08.979946 2014-08-13 18:06:06.391710 7f5c973a57a0 20 read_log 194215'1422 (194215'1005) delete 3ad652eb/benchmark_data_storage0111-ib.pocstor_19374_object9703/head//1 by client.1048726.0:36311 2014-08-12 17:12:08.982873 2014-08-13 18:06:06.391732 7f5c973a57a0 20 read_log 194215'1423 (194215'1006) delete 2daa772b/benchmark_data_storage0111-ib.pocstor_19374_object9929/head//1 by client.1048726.0:36537 2014-08-12 17:12:09.052370 2014-08-13 18:06:06.391785 7f5c973a57a0 20 read_log 194215'1424 (194215'1007) delete 1659f86b/benchmark_data_storage0111-ib.pocstor_19374_object9992/head//1 by client.1048726.0:36600 2014-08-12 17:12:09.073368 2014-08-13 18:06:06.391815 7f5c973a57a0 20 read_log 194215'1425 (194215'1008) delete 50cd18ab/benchmark_data_storage0111-ib.pocstor_19374_object10038/head//1 by client.1048726.0:36646 2014-08-12 17:12:09.094311 2014-08-13 18:06:06.391908 7f5c973a57a0 20 read_log 194215'1426 (194215'1009) delete 57ad5fab/benchmark_data_storage0111-ib.pocstor_19374_object10124/head//1 by client.1048726.0:36732 2014-08-12 17:12:09.119654 2014-08-13 18:06:06.391931 7f5c973a57a0 20 read_log 194215'1427 (194215'1010) delete 20113dab/benchmark_data_storage0111-ib.pocstor_19374_object10313/head//1 by client.1048726.0:36921 2014-08-12 17:12:09.183933 2014-08-13 18:06:06.391953 7f5c973a57a0 20 read_log 194215'1428 (194215'1011) delete fc13012b/benchmark_data_storage0111-ib.pocstor_19374_object10319/head//1 by client.1048726.0:36927 2014-08-12 17:12:09.184336 2014-08-13 18:06:06.391988 7f5c973a57a0 20 read_log 194215'1429 (194215'1012) delete 55f420ab/benchmark_data_storage0111-ib.pocstor_19374_object10414/head//1 by client.1048726.0:37022 2014-08-12 17:12:09.220648 2014-08-13 18:06:06.392019 7f5c973a57a0 20 read_log 194215'1430 (194215'1013) delete 44b26d2b/benchmark_data_storage0111-ib.pocstor_19374_object10437/head//1 by client.1048726.0:37045 2014-08-12 17:12:09.229211 2014-08-13 18:06:06.392042 7f5c973a57a0 20 read_log 194215'1431 (194215'1014) delete 6c33baab/benchmark_data_storage0111-ib.pocstor_19374_object10493/head//1 by client.1048726.0:37101 2014-08-12 17:12:09.245366 2014-08-13 18:06:06.392070 7f5c973a57a0 20 read_log 194215'1432 (194215'1015) delete 56652dab/benchmark_data_storage0111-ib.pocstor_19374_object10603/head//1 by client.1048726.0:37211 2014-08-12 17:12:09.284058 2014-08-13 18:06:06.392096 7f5c973a57a0 20 read_log 194215'1433 (194215'1016) delete 7aabe3eb/benchmark_data_storage0111-ib.pocstor_19374_object10674/head//1 by client.1048726.0:37282 2014-08-12 17:12:09.308402 2014-08-13 18:06:06.392119 7f5c973a57a0 20 read_log 194215'1434 (194215'1017) delete bdbfb66b/benchmark_data_storage0111-ib.pocstor_19374_object10688/head//1 by client.1048726.0:37296 2014-08-12 17:12:09.314929 2014-08-13 18:06:06.392140 7f5c973a57a0 20 read_log 194215'1435 (194215'1018) delete 9f6f4eeb/benchmark_data_storage0111-ib.pocstor_19374_object10736/head//1 by client.1048726.0:37344 2014-08-12 17:12:09.332252 2014-08-13 18:06:06.392174 7f5c973a57a0 20 read_log 194215'1436 (194215'1019) delete 55cd0b2b/benchmark_data_storage0111-ib.pocstor_19374_object10796/head//1 by client.1048726.0:37404 2014-08-12 17:12:09.353540 2014-08-13 18:06:06.392197 7f5c973a57a0 20 read_log 194215'1437 (194215'1020) delete 190a722b/benchmark_data_storage0111-ib.pocstor_19374_object10846/head//1 by client.1048726.0:37454 2014-08-12 17:12:09.368913 2014-08-13 18:06:06.392218 7f5c973a57a0 20 read_log 194215'1438 (194215'1021) delete 960912b/benchmark_data_storage0111-ib.pocstor_19374_object10955/head//1 by client.1048726.0:37563 2014-08-12 17:12:09.410246 2014-08-13 18:06:06.392240 7f5c973a57a0 20 read_log 194215'1439 (194215'1022) delete fcdecfeb/benchmark_data_storage0111-ib.pocstor_19374_object11112/head//1 by client.1048726.0:37720 2014-08-12 17:12:09.465773 2014-08-13 18:06:06.392262 7f5c973a57a0 20 read_log 194215'1440 (194215'1023) delete d5fcf32b/benchmark_data_storage0111-ib.pocstor_19374_object11303/head//1 by client.1048726.0:37911 2014-08-12 17:12:09.523005 2014-08-13 18:06:06.392286 7f5c973a57a0 20 read_log 194215'1441 (194215'1024) delete 75ac9eb/benchmark_data_storage0111-ib.pocstor_19374_object11344/head//1 by client.1048726.0:37952 2014-08-12 17:12:09.535472 2014-08-13 18:06:06.392308 7f5c973a57a0 20 read_log 194215'1442 (194215'1025) delete 3bda6ab/benchmark_data_storage0111-ib.pocstor_19374_object11393/head//1 by client.1048726.0:38001 2014-08-12 17:12:09.550396 2014-08-13 18:06:06.392334 7f5c973a57a0 20 read_log 194215'1443 (194215'1026) delete 940c086b/benchmark_data_storage0111-ib.pocstor_19374_object11474/head//1 by client.1048726.0:38082 2014-08-12 17:12:09.575617 2014-08-13 18:06:06.392369 7f5c973a57a0 20 read_log 194215'1444 (194215'1027) delete 316ace6b/benchmark_data_storage0111-ib.pocstor_19374_object11516/head//1 by client.1048726.0:38124 2014-08-12 17:12:09.587722 2014-08-13 18:06:06.392392 7f5c973a57a0 20 read_log 194215'1445 (194215'1028) delete dd29fdeb/benchmark_data_storage0111-ib.pocstor_19374_object11523/head//1 by client.1048726.0:38131 2014-08-12 17:12:09.590937 2014-08-13 18:06:06.392414 7f5c973a57a0 20 read_log 194215'1446 (194215'1029) delete f5bded2b/benchmark_data_storage0111-ib.pocstor_19374_object11543/head//1 by client.1048726.0:38151 2014-08-12 17:12:09.597176 2014-08-13 18:06:06.392458 7f5c973a57a0 20 read_log 194215'1447 (194215'1030) delete c4b3dc6b/benchmark_data_storage0111-ib.pocstor_19374_object11655/head//1 by client.1048726.0:38263 2014-08-12 17:12:09.637386 2014-08-13 18:06:06.392480 7f5c973a57a0 20 read_log 194215'1448 (194215'1031) delete 2095576b/benchmark_data_storage0111-ib.pocstor_19374_object11672/head//1 by client.1048726.0:38280 2014-08-12 17:12:09.645359 2014-08-13 18:06:06.392562 7f5c973a57a0 20 read_log 194215'1449 (194215'1032) delete a6115a2b/benchmark_data_storage0111-ib.pocstor_19374_object11714/head//1 by client.1048726.0:38322 2014-08-12 17:12:09.660996 2014-08-13 18:06:06.392590 7f5c973a57a0 20 read_log 194215'1450 (194215'1033) delete 8aafa9ab/benchmark_data_storage0111-ib.pocstor_19374_object11744/head//1 by client.1048726.0:38352 2014-08-12 17:12:09.670450 2014-08-13 18:06:06.392618 7f5c973a57a0 20 read_log 194215'1451 (194215'1034) delete d9131ceb/benchmark_data_storage0111-ib.pocstor_19374_object11815/head//1 by client.1048726.0:38423 2014-08-12 17:12:09.694564 2014-08-13 18:06:06.392659 7f5c973a57a0 20 read_log 194215'1452 (194215'1035) delete 7091a6b/benchmark_data_storage0111-ib.pocstor_19374_object12125/head//1 by client.1048726.0:38733 2014-08-12 17:12:09.804142 2014-08-13 18:06:06.392686 7f5c973a57a0 20 read_log 194215'1453 (194215'1036) delete 96552bab/benchmark_data_storage0111-ib.pocstor_19374_object12215/head//1 by client.1048726.0:38823 2014-08-12 17:12:09.839432 2014-08-13 18:06:06.392709 7f5c973a57a0 20 read_log 194215'1454 (194215'1037) delete 84c0f3eb/benchmark_data_storage0111-ib.pocstor_19374_object12617/head//1 by client.1048726.0:39225 2014-08-12 17:12:09.982090 2014-08-13 18:06:06.392731 7f5c973a57a0 20 read_log 194215'1455 (194215'1038) delete 6b222c6b/benchmark_data_storage0111-ib.pocstor_19374_object12626/head//1 by client.1048726.0:39234 2014-08-12 17:12:09.986105 2014-08-13 18:06:06.392753 7f5c973a57a0 20 read_log 194215'1456 (194215'1039) delete 304a35eb/benchmark_data_storage0111-ib.pocstor_19374_object12721/head//1 by client.1048726.0:39329 2014-08-12 17:12:10.013472 2014-08-13 18:06:06.392774 7f5c973a57a0 20 read_log 194215'1457 (194215'1040) delete ce855f2b/benchmark_data_storage0111-ib.pocstor_19374_object12758/head//1 by client.1048726.0:39366 2014-08-12 17:12:10.024379 2014-08-13 18:06:06.392796 7f5c973a57a0 20 read_log 194215'1458 (194215'1041) delete 91d427ab/benchmark_data_storage0111-ib.pocstor_19374_object12778/head//1 by client.1048726.0:39386 2014-08-12 17:12:10.029642 2014-08-13 18:06:06.392817 7f5c973a57a0 20 read_log 194215'1459 (194215'1042) delete fb4560ab/benchmark_data_storage0111-ib.pocstor_19374_object12863/head//1 by client.1048726.0:39471 2014-08-12 17:12:10.058395 2014-08-13 18:06:06.392842 7f5c973a57a0 20 read_log 194215'1460 (194215'1043) delete 883592b/benchmark_data_storage0111-ib.pocstor_19374_object12865/head//1 by client.1048726.0:39473 2014-08-12 17:12:10.060577 2014-08-13 18:06:06.392864 7f5c973a57a0 20 read_log 194215'1461 (194215'1044) delete 64527e2b/benchmark_data_storage0111-ib.pocstor_19374_object12908/head//1 by client.1048726.0:39516 2014-08-12 17:12:10.074158 2014-08-13 18:06:06.392886 7f5c973a57a0 20 read_log 194215'1462 (194215'1045) delete bb45c1eb/benchmark_data_storage0111-ib.pocstor_19374_object12937/head//1 by client.1048726.0:39545 2014-08-12 17:12:10.084668 2014-08-13 18:06:06.392908 7f5c973a57a0 20 read_log 194215'1463 (194215'1046) delete 4f81c16b/benchmark_data_storage0111-ib.pocstor_19374_object12968/head//1 by client.1048726.0:39576 2014-08-12 17:12:10.096728 2014-08-13 18:06:06.392941 7f5c973a57a0 20 read_log 194215'1464 (194215'1047) delete ca6ba7eb/benchmark_data_storage0111-ib.pocstor_19374_object12985/head//1 by client.1048726.0:39593 2014-08-12 17:12:10.103326 2014-08-13 18:06:06.392967 7f5c973a57a0 20 read_log 194215'1465 (194215'1048) delete d6caf0eb/benchmark_data_storage0111-ib.pocstor_19374_object13037/head//1 by client.1048726.0:39645 2014-08-12 17:12:10.118536 2014-08-13 18:06:06.392994 7f5c973a57a0 20 read_log 194215'1466 (194215'1049) delete 995517eb/benchmark_data_storage0111-ib.pocstor_19374_object13098/head//1 by client.1048726.0:39706 2014-08-12 17:12:10.137401 2014-08-13 18:06:06.393016 7f5c973a57a0 20 read_log 194215'1467 (194215'1050) delete 69b7392b/benchmark_data_storage0111-ib.pocstor_19374_object13278/head//1 by client.1048726.0:39886 2014-08-12 17:12:10.198794 2014-08-13 18:06:06.393038 7f5c973a57a0 20 read_log 194215'1468 (194215'1051) delete 5d3e5d6b/benchmark_data_storage0111-ib.pocstor_19374_object13368/head//1 by client.1048726.0:39976 2014-08-12 17:12:10.226623 2014-08-13 18:06:06.393059 7f5c973a57a0 20 read_log 194215'1469 (194215'1052) delete b7bdedeb/benchmark_data_storage0111-ib.pocstor_19374_object13400/head//1 by client.1048726.0:40008 2014-08-12 17:12:10.239865 2014-08-13 18:06:06.393085 7f5c973a57a0 20 read_log 194215'1470 (194215'1053) delete 6dc64eeb/benchmark_data_storage0111-ib.pocstor_19374_object13453/head//1 by client.1048726.0:40061 2014-08-12 17:12:10.260396 2014-08-13 18:06:06.393110 7f5c973a57a0 20 read_log 194215'1471 (194215'1054) delete 8c0f1d2b/benchmark_data_storage0111-ib.pocstor_19374_object13462/head//1 by client.1048726.0:40070 2014-08-12 17:12:10.263464 2014-08-13 18:06:06.393132 7f5c973a57a0 20 read_log 194215'1472 (194215'1055) delete 37a0adab/benchmark_data_storage0111-ib.pocstor_19374_object13587/head//1 by client.1048726.0:40195 2014-08-12 17:12:10.321025 2014-08-13 18:06:06.393163 7f5c973a57a0 20 read_log 194215'1473 (194215'1056) delete 5a6f74eb/benchmark_data_storage0111-ib.pocstor_19374_object13600/head//1 by client.1048726.0:40208 2014-08-12 17:12:10.328063 2014-08-13 18:06:06.393190 7f5c973a57a0 20 read_log 194215'1474 (194215'1057) delete d3458feb/benchmark_data_storage0111-ib.pocstor_19374_object13641/head//1 by client.1048726.0:40249 2014-08-12 17:12:10.347720 2014-08-13 18:06:06.393212 7f5c973a57a0 20 read_log 194215'1475 (194215'1058) delete b1e814eb/benchmark_data_storage0111-ib.pocstor_19374_object13681/head//1 by client.1048726.0:40289 2014-08-12 17:12:10.366181 2014-08-13 18:06:06.393237 7f5c973a57a0 20 read_log 194215'1476 (194215'1059) delete 97b4952b/benchmark_data_storage0111-ib.pocstor_19374_object13697/head//1 by client.1048726.0:40305 2014-08-12 17:12:10.373564 2014-08-13 18:06:06.393258 7f5c973a57a0 20 read_log 194215'1477 (194215'1060) delete b156fbab/benchmark_data_storage0111-ib.pocstor_19374_object13795/head//1 by client.1048726.0:40403 2014-08-12 17:12:10.413287 2014-08-13 18:06:06.393280 7f5c973a57a0 20 read_log 194215'1478 (194215'1061) delete b9abdaeb/benchmark_data_storage0111-ib.pocstor_19374_object13915/head//1 by client.1048726.0:40523 2014-08-12 17:12:10.462657 2014-08-13 18:06:06.393301 7f5c973a57a0 20 read_log 194215'1479 (194215'1062) delete 15bb4aeb/benchmark_data_storage0111-ib.pocstor_19374_object14018/head//1 by client.1048726.0:40626 2014-08-12 17:12:10.496295 2014-08-13 18:06:06.393323 7f5c973a57a0 20 read_log 194215'1480 (194215'1063) delete 86d77e6b/benchmark_data_storage0111-ib.pocstor_19374_object14045/head//1 by client.1048726.0:40653 2014-08-12 17:12:10.503974 2014-08-13 18:06:06.393344 7f5c973a57a0 20 read_log 194215'1481 (194215'1064) delete f762602b/benchmark_data_storage0111-ib.pocstor_19374_object14072/head//1 by client.1048726.0:40680 2014-08-12 17:12:10.512713 2014-08-13 18:06:06.393370 7f5c973a57a0 20 read_log 194215'1482 (194215'1065) delete 656a9d6b/benchmark_data_storage0111-ib.pocstor_19374_object14108/head//1 by client.1048726.0:40716 2014-08-12 17:12:10.527145 2014-08-13 18:06:06.393392 7f5c973a57a0 20 read_log 194215'1483 (194215'1066) delete 12017d6b/benchmark_data_storage0111-ib.pocstor_19374_object14130/head//1 by client.1048726.0:40738 2014-08-12 17:12:10.534227 2014-08-13 18:06:06.393433 7f5c973a57a0 20 read_log 194215'1484 (194215'1067) delete 6837d1eb/benchmark_data_storage0111-ib.pocstor_19374_object14218/head//1 by client.1048726.0:40826 2014-08-12 17:12:10.566668 2014-08-13 18:06:06.393466 7f5c973a57a0 20 read_log 194215'1485 (194215'1068) delete e0890e6b/benchmark_data_storage0111-ib.pocstor_19374_object14272/head//1 by client.1048726.0:40880 2014-08-12 17:12:10.584392 2014-08-13 18:06:06.393510 7f5c973a57a0 20 read_log 194215'1486 (194215'1069) delete 51a9f76b/benchmark_data_storage0111-ib.pocstor_19374_object14273/head//1 by client.1048726.0:40881 2014-08-12 17:12:10.584524 2014-08-13 18:06:06.393538 7f5c973a57a0 20 read_log 194215'1487 (194215'1070) delete ff105aeb/benchmark_data_storage0111-ib.pocstor_19374_object14288/head//1 by client.1048726.0:40896 2014-08-12 17:12:10.590143 2014-08-13 18:06:06.393560 7f5c973a57a0 20 read_log 194215'1488 (194215'1071) delete 7c054ceb/benchmark_data_storage0111-ib.pocstor_19374_object14442/head//1 by client.1048726.0:41050 2014-08-12 17:12:10.657515 2014-08-13 18:06:06.393581 7f5c973a57a0 20 read_log 194215'1489 (194215'1072) delete fc1d02eb/benchmark_data_storage0111-ib.pocstor_19374_object14443/head//1 by client.1048726.0:41051 2014-08-12 17:12:10.657688 2014-08-13 18:06:06.393621 7f5c973a57a0 20 read_log 194215'1490 (194215'1073) delete b7463aeb/benchmark_data_storage0111-ib.pocstor_19374_object14446/head//1 by client.1048726.0:41054 2014-08-12 17:12:10.660490 2014-08-13 18:06:06.393656 7f5c973a57a0 20 read_log 194215'1491 (194215'1074) delete 56be36eb/benchmark_data_storage0111-ib.pocstor_19374_object14581/head//1 by client.1048726.0:41189 2014-08-12 17:12:10.731834 2014-08-13 18:06:06.393718 7f5c973a57a0 20 read_log 194215'1492 (194215'1075) delete bc8678eb/benchmark_data_storage0111-ib.pocstor_19374_object14608/head//1 by client.1048726.0:41216 2014-08-12 17:12:10.744467 2014-08-13 18:06:06.393741 7f5c973a57a0 20 read_log 194215'1493 (194215'1076) delete 49787bab/benchmark_data_storage0111-ib.pocstor_19374_object14741/head//1 by client.1048726.0:41349 2014-08-12 17:12:10.808010 2014-08-13 18:06:06.393765 7f5c973a57a0 20 read_log 194215'1494 (194215'1077) delete af3d7a2b/benchmark_data_storage0111-ib.pocstor_19374_object14751/head//1 by client.1048726.0:41359 2014-08-12 17:12:10.811951 2014-08-13 18:06:06.393786 7f5c973a57a0 20 read_log 194215'1495 (194215'1078) delete 3cec11eb/benchmark_data_storage0111-ib.pocstor_19374_object14764/head//1 by client.1048726.0:41372 2014-08-12 17:12:10.816174 2014-08-13 18:06:06.393808 7f5c973a57a0 20 read_log 194215'1496 (194215'1079) delete 97eeeab/benchmark_data_storage0111-ib.pocstor_19374_object14796/head//1 by client.1048726.0:41404 2014-08-12 17:12:10.833680 2014-08-13 18:06:06.393830 7f5c973a57a0 20 read_log 194215'1497 (194215'1080) delete 708ad02b/benchmark_data_storage0111-ib.pocstor_19374_object14818/head//1 by client.1048726.0:41426 2014-08-12 17:12:10.848022 2014-08-13 18:06:06.393856 7f5c973a57a0 20 read_log 194215'1498 (194215'1081) delete ed21ddeb/benchmark_data_storage0111-ib.pocstor_19374_object14866/head//1 by client.1048726.0:41474 2014-08-12 17:12:10.877986 2014-08-13 18:06:06.393898 7f5c973a57a0 20 read_log 194215'1499 (194215'1082) delete 7847fa6b/benchmark_data_storage0111-ib.pocstor_19374_object15056/head//1 by client.1048726.0:41664 2014-08-12 17:12:10.968868 2014-08-13 18:06:06.393921 7f5c973a57a0 20 read_log 194215'1500 (194215'1083) delete 91326eeb/benchmark_data_storage0111-ib.pocstor_19374_object15087/head//1 by client.1048726.0:41695 2014-08-12 17:12:10.979866 2014-08-13 18:06:06.393943 7f5c973a57a0 20 read_log 194215'1501 (194215'1084) delete f966fa2b/benchmark_data_storage0111-ib.pocstor_19374_object15209/head//1 by client.1048726.0:41817 2014-08-12 17:12:11.025827 2014-08-13 18:06:06.393965 7f5c973a57a0 20 read_log 194215'1502 (194215'1085) delete 228e7c6b/benchmark_data_storage0111-ib.pocstor_19374_object15227/head//1 by client.1048726.0:41835 2014-08-12 17:12:11.032740 2014-08-13 18:06:06.393986 7f5c973a57a0 20 read_log 194215'1503 (194215'1086) delete 7acf24ab/benchmark_data_storage0111-ib.pocstor_19374_object15235/head//1 by client.1048726.0:41843 2014-08-12 17:12:11.036396 2014-08-13 18:06:06.394052 7f5c973a57a0 20 read_log 194215'1504 (194215'1087) delete c50ebeeb/benchmark_data_storage0111-ib.pocstor_19374_object15256/head//1 by client.1048726.0:41864 2014-08-12 17:12:11.047265 2014-08-13 18:06:06.394080 7f5c973a57a0 20 read_log 194215'1505 (194215'1088) delete ce6e186b/benchmark_data_storage0111-ib.pocstor_19374_object15291/head//1 by client.1048726.0:41899 2014-08-12 17:12:11.062901 2014-08-13 18:06:06.394102 7f5c973a57a0 20 read_log 194215'1506 (194215'1089) delete 4248daab/benchmark_data_storage0111-ib.pocstor_19374_object15301/head//1 by client.1048726.0:41909 2014-08-12 17:12:11.067919 2014-08-13 18:06:06.394125 7f5c973a57a0 20 read_log 194215'1507 (194215'1090) delete 7db6d4eb/benchmark_data_storage0111-ib.pocstor_19374_object15375/head//1 by client.1048726.0:41983 2014-08-12 17:12:11.105704 2014-08-13 18:06:06.394154 7f5c973a57a0 20 read_log 194215'1508 (194215'1091) delete d27cd12b/benchmark_data_storage0111-ib.pocstor_19374_object15494/head//1 by client.1048726.0:42102 2014-08-12 17:12:11.163798 2014-08-13 18:06:06.394187 7f5c973a57a0 20 read_log 194215'1509 (194215'1092) delete 29efcab/benchmark_data_storage0111-ib.pocstor_19374_object15598/head//1 by client.1048726.0:42206 2014-08-12 17:12:11.216115 2014-08-13 18:06:06.394209 7f5c973a57a0 20 read_log 194215'1510 (194215'1093) delete 725d332b/benchmark_data_storage0111-ib.pocstor_19374_object15691/head//1 by client.1048726.0:42299 2014-08-12 17:12:11.254170 2014-08-13 18:06:06.394232 7f5c973a57a0 20 read_log 194215'1511 (194215'1094) delete 427577eb/benchmark_data_storage0111-ib.pocstor_19374_object15706/head//1 by client.1048726.0:42314 2014-08-12 17:12:11.259763 2014-08-13 18:06:06.394254 7f5c973a57a0 20 read_log 194215'1512 (194215'1095) delete 3774496b/benchmark_data_storage0111-ib.pocstor_19374_object15775/head//1 by client.1048726.0:42383 2014-08-12 17:12:11.282713 2014-08-13 18:06:06.394279 7f5c973a57a0 20 read_log 194215'1513 (194215'1096) delete fb1ddcab/benchmark_data_storage0111-ib.pocstor_19374_object15797/head//1 by client.1048726.0:42405 2014-08-12 17:12:11.290841 2014-08-13 18:06:06.394301 7f5c973a57a0 20 read_log 194215'1514 (194215'1097) delete 72b1916b/benchmark_data_storage0111-ib.pocstor_19374_object15974/head//1 by client.1048726.0:42582 2014-08-12 17:12:11.365685 2014-08-13 18:06:06.394329 7f5c973a57a0 20 read_log 194215'1515 (194215'1098) delete 1afad6eb/benchmark_data_storage0111-ib.pocstor_19374_object16016/head//1 by client.1048726.0:42624 2014-08-12 17:12:11.380790 2014-08-13 18:06:06.394351 7f5c973a57a0 20 read_log 194215'1516 (194215'1099) delete 36c3d4ab/benchmark_data_storage0111-ib.pocstor_19374_object16089/head//1 by client.1048726.0:42697 2014-08-12 17:12:11.410101 2014-08-13 18:06:06.394377 7f5c973a57a0 20 read_log 194215'1517 (194215'1100) delete 41ee41eb/benchmark_data_storage0111-ib.pocstor_19374_object16099/head//1 by client.1048726.0:42707 2014-08-12 17:12:11.415323 2014-08-13 18:06:06.394400 7f5c973a57a0 20 read_log 194215'1518 (194215'1101) delete f52c0f6b/benchmark_data_storage0111-ib.pocstor_19374_object16114/head//1 by client.1048726.0:42722 2014-08-12 17:12:11.423609 2014-08-13 18:06:06.394426 7f5c973a57a0 20 read_log 194215'1519 (194215'1102) delete 343512ab/benchmark_data_storage0111-ib.pocstor_19374_object16151/head//1 by client.1048726.0:42759 2014-08-12 17:12:11.440682 2014-08-13 18:06:06.394452 7f5c973a57a0 20 read_log 194215'1520 (194215'1103) delete 5811406b/benchmark_data_storage0111-ib.pocstor_19374_object16504/head//1 by client.1048726.0:43112 2014-08-12 17:12:11.585163 2014-08-13 18:06:06.394479 7f5c973a57a0 20 read_log 194215'1521 (194215'1104) delete 9896e7ab/benchmark_data_storage0111-ib.pocstor_19374_object16534/head//1 by client.1048726.0:43142 2014-08-12 17:12:11.599824 2014-08-13 18:06:06.394505 7f5c973a57a0 20 read_log 194215'1522 (194215'1105) delete d7af0eab/benchmark_data_storage0111-ib.pocstor_19374_object16646/head//1 by client.1048726.0:43254 2014-08-12 17:12:11.656798 2014-08-13 18:06:06.394552 7f5c973a57a0 20 read_log 194215'1523 (194215'1106) delete 915c5a2b/benchmark_data_storage0111-ib.pocstor_19374_object16648/head//1 by client.1048726.0:43256 2014-08-12 17:12:11.658431 2014-08-13 18:06:06.394599 7f5c973a57a0 20 read_log 194215'1524 (194215'1107) delete a73d206b/benchmark_data_storage0111-ib.pocstor_19374_object16746/head//1 by client.1048726.0:43354 2014-08-12 17:12:11.708415 2014-08-13 18:06:06.394640 7f5c973a57a0 20 read_log 194215'1525 (194215'1108) delete c47c9beb/benchmark_data_storage0111-ib.pocstor_19374_object16760/head//1 by client.1048726.0:43368 2014-08-12 17:12:11.712816 2014-08-13 18:06:06.394663 7f5c973a57a0 20 read_log 194215'1526 (194215'1109) delete dea573ab/benchmark_data_storage0111-ib.pocstor_19374_object16841/head//1 by client.1048726.0:43449 2014-08-12 17:12:11.746116 2014-08-13 18:06:06.394685 7f5c973a57a0 20 read_log 194215'1527 (194215'1110) delete b9e0d1ab/benchmark_data_storage0111-ib.pocstor_19374_object16881/head//1 by client.1048726.0:43489 2014-08-12 17:12:11.762373 2014-08-13 18:06:06.394713 7f5c973a57a0 20 read_log 194215'1528 (194215'1111) delete 3755296b/benchmark_data_storage0111-ib.pocstor_19374_object16916/head//1 by client.1048726.0:43524 2014-08-12 17:12:11.777259 2014-08-13 18:06:06.394738 7f5c973a57a0 20 read_log 194215'1529 (194215'1112) delete c45cbd6b/benchmark_data_storage0111-ib.pocstor_19374_object16925/head//1 by client.1048726.0:43533 2014-08-12 17:12:11.781025 2014-08-13 18:06:06.394760 7f5c973a57a0 20 read_log 194215'1530 (194215'1113) delete d5f0a92b/benchmark_data_storage0111-ib.pocstor_19374_object16947/head//1 by client.1048726.0:43555 2014-08-12 17:12:11.791448 2014-08-13 18:06:06.394794 7f5c973a57a0 20 read_log 194215'1531 (194215'1114) delete 5fceb76b/benchmark_data_storage0111-ib.pocstor_19374_object17225/head//1 by client.1048726.0:43833 2014-08-12 17:12:11.939847 2014-08-13 18:06:06.394822 7f5c973a57a0 20 read_log 194215'1532 (194215'1115) delete bf0f31eb/benchmark_data_storage0111-ib.pocstor_19374_object17240/head//1 by client.1048726.0:43848 2014-08-12 17:12:11.948426 2014-08-13 18:06:06.394844 7f5c973a57a0 20 read_log 194215'1533 (194215'1116) delete ab105e2b/benchmark_data_storage0111-ib.pocstor_19374_object17300/head//1 by client.1048726.0:43908 2014-08-12 17:12:11.983474 2014-08-13 18:06:06.394866 7f5c973a57a0 20 read_log 194215'1534 (194215'1117) delete 8b06b7ab/benchmark_data_storage0111-ib.pocstor_19374_object17532/head//1 by client.1048726.0:44140 2014-08-12 17:12:12.123563 2014-08-13 18:06:06.394888 7f5c973a57a0 20 read_log 194215'1535 (194215'1118) delete ff580c6b/benchmark_data_storage0111-ib.pocstor_19374_object17566/head//1 by client.1048726.0:44174 2014-08-12 17:12:12.145512 2014-08-13 18:06:06.394909 7f5c973a57a0 20 read_log 194215'1536 (194215'1119) delete feff186b/benchmark_data_storage0111-ib.pocstor_19374_object17666/head//1 by client.1048726.0:44274 2014-08-12 17:12:12.221742 2014-08-13 18:06:06.394931 7f5c973a57a0 20 read_log 194215'1537 (194215'1120) delete 92eb096b/benchmark_data_storage0111-ib.pocstor_19374_object17678/head//1 by client.1048726.0:44286 2014-08-12 17:12:12.229387 2014-08-13 18:06:06.394952 7f5c973a57a0 20 read_log 194215'1538 (194215'1121) delete 4795ccab/benchmark_data_storage0111-ib.pocstor_19374_object17689/head//1 by client.1048726.0:44297 2014-08-12 17:12:12.236925 2014-08-13 18:06:06.394974 7f5c973a57a0 20 read_log 194215'1539 (194215'1122) delete 7610516b/benchmark_data_storage0111-ib.pocstor_19374_object17776/head//1 by client.1048726.0:44384 2014-08-12 17:12:12.299358 2014-08-13 18:06:06.394995 7f5c973a57a0 20 read_log 194215'1540 (194215'1123) delete 1941352b/benchmark_data_storage0111-ib.pocstor_19374_object17799/head//1 by client.1048726.0:44407 2014-08-12 17:12:12.308405 2014-08-13 18:06:06.395017 7f5c973a57a0 20 read_log 194215'1541 (194215'1124) delete a663f52b/benchmark_data_storage0111-ib.pocstor_19374_object17824/head//1 by client.1048726.0:44432 2014-08-12 17:12:12.319432 2014-08-13 18:06:06.395069 7f5c973a57a0 20 read_log 194215'1542 (194215'1125) delete 85edc6ab/benchmark_data_storage0111-ib.pocstor_19374_object17839/head//1 by client.1048726.0:44447 2014-08-12 17:12:12.325370 2014-08-13 18:06:06.395096 7f5c973a57a0 20 read_log 194215'1543 (194215'1126) delete 3a62892b/benchmark_data_storage0111-ib.pocstor_19374_object17894/head//1 by client.1048726.0:44502 2014-08-12 17:12:12.349169 2014-08-13 18:06:06.395133 7f5c973a57a0 20 read_log 194215'1544 (194215'1127) delete a9f9016b/benchmark_data_storage0111-ib.pocstor_19374_object17942/head//1 by client.1048726.0:44550 2014-08-12 17:12:12.366679 2014-08-13 18:06:06.395173 7f5c973a57a0 20 read_log 194215'1545 (194215'1128) delete 49c8032b/benchmark_data_storage0111-ib.pocstor_19374_object18015/head//1 by client.1048726.0:44623 2014-08-12 17:12:12.398194 2014-08-13 18:06:06.395196 7f5c973a57a0 20 read_log 194215'1546 (194215'1129) delete 63b44fab/benchmark_data_storage0111-ib.pocstor_19374_object18232/head//1 by client.1048726.0:44840 2014-08-12 17:12:12.494064 2014-08-13 18:06:06.395218 7f5c973a57a0 20 read_log 194215'1547 (194215'1130) delete c05a8beb/benchmark_data_storage0111-ib.pocstor_19374_object18250/head//1 by client.1048726.0:44858 2014-08-12 17:12:12.501151 2014-08-13 18:06:06.395240 7f5c973a57a0 20 read_log 194215'1548 (194215'1131) delete 622415eb/benchmark_data_storage0111-ib.pocstor_19374_object18344/head//1 by client.1048726.0:44952 2014-08-12 17:12:12.533542 2014-08-13 18:06:06.395261 7f5c973a57a0 20 read_log 194215'1549 (194215'1132) delete d404aeb/benchmark_data_storage0111-ib.pocstor_19374_object18373/head//1 by client.1048726.0:44981 2014-08-12 17:12:12.544879 2014-08-13 18:06:06.395283 7f5c973a57a0 20 read_log 194215'1550 (194215'1133) delete 19dc8e2b/benchmark_data_storage0111-ib.pocstor_19374_object18496/head//1 by client.1048726.0:45104 2014-08-12 17:12:12.592582 2014-08-13 18:06:06.395305 7f5c973a57a0 20 read_log 194215'1551 (194215'1134) delete 379abaab/benchmark_data_storage0111-ib.pocstor_19374_object18497/head//1 by client.1048726.0:45105 2014-08-12 17:12:12.592775 2014-08-13 18:06:06.395326 7f5c973a57a0 20 read_log 194215'1552 (194215'1135) delete 64e0802b/benchmark_data_storage0111-ib.pocstor_19374_object18498/head//1 by client.1048726.0:45106 2014-08-12 17:12:12.592891 2014-08-13 18:06:06.395350 7f5c973a57a0 20 read_log 194215'1553 (194215'1136) delete c6e8922b/benchmark_data_storage0111-ib.pocstor_19374_object18505/head//1 by client.1048726.0:45113 2014-08-12 17:12:12.597526 2014-08-13 18:06:06.395380 7f5c973a57a0 20 read_log 194215'1554 (194215'1137) delete 2855efeb/benchmark_data_storage0111-ib.pocstor_19374_object18551/head//1 by client.1048726.0:45159 2014-08-12 17:12:12.616209 2014-08-13 18:06:06.395402 7f5c973a57a0 20 read_log 194215'1555 (194215'1138) delete 646eadab/benchmark_data_storage0111-ib.pocstor_19374_object18577/head//1 by client.1048726.0:45185 2014-08-12 17:12:12.626993 2014-08-13 18:06:06.395431 7f5c973a57a0 20 read_log 194215'1556 (194215'1139) delete 6c5018eb/benchmark_data_storage0111-ib.pocstor_19374_object18780/head//1 by client.1048726.0:45388 2014-08-12 17:12:12.710014 2014-08-13 18:06:06.395458 7f5c973a57a0 20 read_log 194215'1557 (194215'1140) delete d3893eeb/benchmark_data_storage0111-ib.pocstor_19374_object18819/head//1 by client.1048726.0:45427 2014-08-12 17:12:12.724198 2014-08-13 18:06:06.395494 7f5c973a57a0 20 read_log 194215'1558 (194215'1141) delete f022432b/benchmark_data_storage0111-ib.pocstor_19374_object18842/head//1 by client.1048726.0:45450 2014-08-12 17:12:12.732856 2014-08-13 18:06:06.395539 7f5c973a57a0 20 read_log 194215'1559 (194215'1142) delete 2476c96b/benchmark_data_storage0111-ib.pocstor_19374_object18852/head//1 by client.1048726.0:45460 2014-08-12 17:12:12.735763 2014-08-13 18:06:06.395567 7f5c973a57a0 20 read_log 194215'1560 (194215'1143) delete e1cd2ceb/benchmark_data_storage0111-ib.pocstor_19374_object19065/head//1 by client.1048726.0:45673 2014-08-12 17:12:12.818077 2014-08-13 18:06:06.395592 7f5c973a57a0 20 read_log 194215'1561 (194215'1144) delete ae4f962b/benchmark_data_storage0111-ib.pocstor_19374_object19226/head//1 by client.1048726.0:45834 2014-08-12 17:12:12.879829 2014-08-13 18:06:06.395616 7f5c973a57a0 20 read_log 194215'1562 (194215'1145) delete b8cf256b/benchmark_data_storage0111-ib.pocstor_19374_object19231/head//1 by client.1048726.0:45839 2014-08-12 17:12:12.881421 2014-08-13 18:06:06.395656 7f5c973a57a0 20 read_log 194215'1563 (194215'1146) delete f0edd2b/benchmark_data_storage0111-ib.pocstor_19374_object19233/head//1 by client.1048726.0:45841 2014-08-12 17:12:12.882068 2014-08-13 18:06:06.395699 7f5c973a57a0 20 read_log 194215'1564 (194215'1147) delete 2f22b1ab/benchmark_data_storage0111-ib.pocstor_19374_object19274/head//1 by client.1048726.0:45882 2014-08-12 17:12:12.895240 2014-08-13 18:06:06.395726 7f5c973a57a0 20 read_log 194215'1565 (194215'1148) delete 115748eb/benchmark_data_storage0111-ib.pocstor_19374_object19293/head//1 by client.1048726.0:45901 2014-08-12 17:12:12.901849 2014-08-13 18:06:06.395749 7f5c973a57a0 20 read_log 194215'1566 (194215'1149) delete ce72ce6b/benchmark_data_storage0111-ib.pocstor_19374_object19399/head//1 by client.1048726.0:46007 2014-08-12 17:12:12.935554 2014-08-13 18:06:06.395788 7f5c973a57a0 20 read_log 194215'1567 (194215'1150) delete d56f90ab/benchmark_data_storage0111-ib.pocstor_19374_object19623/head//1 by client.1048726.0:46231 2014-08-12 17:12:13.008926 2014-08-13 18:06:06.395815 7f5c973a57a0 20 read_log 194215'1568 (194215'1151) delete ee91eceb/benchmark_data_storage0111-ib.pocstor_19374_object19625/head//1 by client.1048726.0:46233 2014-08-12 17:12:13.009384 2014-08-13 18:06:06.395838 7f5c973a57a0 20 read_log 194215'1569 (194215'1152) delete 563df2ab/benchmark_data_storage0111-ib.pocstor_19374_object19626/head//1 by client.1048726.0:46234 2014-08-12 17:12:13.009468 2014-08-13 18:06:06.395870 7f5c973a57a0 20 read_log 194215'1570 (194215'1153) delete f16196eb/benchmark_data_storage0111-ib.pocstor_19374_object19875/head//1 by client.1048726.0:46483 2014-08-12 17:12:13.091595 2014-08-13 18:06:06.395897 7f5c973a57a0 20 read_log 194215'1571 (194215'1154) delete 43e8486b/benchmark_data_storage0111-ib.pocstor_19374_object19917/head//1 by client.1048726.0:46525 2014-08-12 17:12:13.107435 2014-08-13 18:06:06.395919 7f5c973a57a0 20 read_log 194215'1572 (194215'1155) delete 3847442b/benchmark_data_storage0111-ib.pocstor_19374_object19991/head//1 by client.1048726.0:46599 2014-08-12 17:12:13.132025 2014-08-13 18:06:06.395941 7f5c973a57a0 20 read_log 194215'1573 (194215'1156) delete 2e8e12ab/benchmark_data_storage0111-ib.pocstor_19374_object20016/head//1 by client.1048726.0:46624 2014-08-12 17:12:13.143630 2014-08-13 18:06:06.395962 7f5c973a57a0 20 read_log 194215'1574 (194215'1157) delete 9ea9826b/benchmark_data_storage0111-ib.pocstor_19374_object20100/head//1 by client.1048726.0:46708 2014-08-12 17:12:13.169849 2014-08-13 18:06:06.395983 7f5c973a57a0 20 read_log 194215'1575 (194215'1158) delete de30edeb/benchmark_data_storage0111-ib.pocstor_19374_object20132/head//1 by client.1048726.0:46740 2014-08-12 17:12:13.179443 2014-08-13 18:06:06.396005 7f5c973a57a0 20 read_log 194215'1576 (194215'1159) delete 47ed002b/benchmark_data_storage0111-ib.pocstor_19374_object20189/head//1 by client.1048726.0:46797 2014-08-12 17:12:13.199224 2014-08-13 18:06:06.396027 7f5c973a57a0 20 read_log 194215'1577 (194215'1160) delete 5be3cf2b/benchmark_data_storage0111-ib.pocstor_19374_object20205/head//1 by client.1048726.0:46813 2014-08-12 17:12:13.204286 2014-08-13 18:06:06.396048 7f5c973a57a0 20 read_log 194215'1578 (194215'1161) delete fbdfe52b/benchmark_data_storage0111-ib.pocstor_19374_object20231/head//1 by client.1048726.0:46839 2014-08-12 17:12:13.213770 2014-08-13 18:06:06.396070 7f5c973a57a0 20 read_log 194215'1579 (194215'1162) delete 45de5dab/benchmark_data_storage0111-ib.pocstor_19374_object20287/head//1 by client.1048726.0:46895 2014-08-12 17:12:13.230430 2014-08-13 18:06:06.396091 7f5c973a57a0 20 read_log 194215'1580 (194215'1163) delete 94efc5eb/benchmark_data_storage0111-ib.pocstor_19374_object20346/head//1 by client.1048726.0:46954 2014-08-12 17:12:13.249723 2014-08-13 18:06:06.396113 7f5c973a57a0 20 read_log 194215'1581 (194215'1164) delete 1347d22b/benchmark_data_storage0111-ib.pocstor_19374_object20407/head//1 by client.1048726.0:47015 2014-08-12 17:12:13.271648 2014-08-13 18:06:06.396135 7f5c973a57a0 20 read_log 194215'1582 (194215'1165) delete e33e54ab/benchmark_data_storage0111-ib.pocstor_19374_object20410/head//1 by client.1048726.0:47018 2014-08-12 17:12:13.273639 2014-08-13 18:06:06.396166 7f5c973a57a0 20 read_log 194215'1583 (194215'1166) delete e858402b/benchmark_data_storage0111-ib.pocstor_19374_object20437/head//1 by client.1048726.0:47045 2014-08-12 17:12:13.282106 2014-08-13 18:06:06.396203 7f5c973a57a0 20 read_log 194215'1584 (194215'1167) delete 19a5632b/benchmark_data_storage0111-ib.pocstor_19374_object20464/head//1 by client.1048726.0:47072 2014-08-12 17:12:13.290914 2014-08-13 18:06:06.396230 7f5c973a57a0 20 read_log 194215'1585 (194215'1168) delete 19fb1ceb/benchmark_data_storage0111-ib.pocstor_19374_object20580/head//1 by client.1048726.0:47188 2014-08-12 17:12:13.330303 2014-08-13 18:06:06.396251 7f5c973a57a0 20 read_log 194215'1586 (194215'1169) delete 63afe4eb/benchmark_data_storage0111-ib.pocstor_19374_object20590/head//1 by client.1048726.0:47198 2014-08-12 17:12:13.333747 2014-08-13 18:06:06.396272 7f5c973a57a0 20 read_log 194215'1587 (194215'1170) delete 5e2d476b/benchmark_data_storage0111-ib.pocstor_19374_object20594/head//1 by client.1048726.0:47202 2014-08-12 17:12:13.337531 2014-08-13 18:06:06.396294 7f5c973a57a0 20 read_log 194215'1588 (194215'1171) delete 7ffdd76b/benchmark_data_storage0111-ib.pocstor_19374_object20637/head//1 by client.1048726.0:47245 2014-08-12 17:12:13.349506 2014-08-13 18:06:06.396315 7f5c973a57a0 20 read_log 194215'1589 (194215'1172) delete 595617eb/benchmark_data_storage0111-ib.pocstor_19374_object20646/head//1 by client.1048726.0:47254 2014-08-12 17:12:13.351589 2014-08-13 18:06:06.396336 7f5c973a57a0 20 read_log 194215'1590 (194215'1173) delete 2860b42b/benchmark_data_storage0111-ib.pocstor_19374_object20753/head//1 by client.1048726.0:47361 2014-08-12 17:12:13.386695 2014-08-13 18:06:06.396362 7f5c973a57a0 20 read_log 194215'1591 (194215'1174) delete 496c846b/benchmark_data_storage0111-ib.pocstor_19374_object20823/head//1 by client.1048726.0:47431 2014-08-12 17:12:13.409981 2014-08-13 18:06:06.396388 7f5c973a57a0 20 read_log 194215'1592 (194215'1175) delete 22036eeb/benchmark_data_storage0111-ib.pocstor_19374_object20913/head//1 by client.1048726.0:47521 2014-08-12 17:12:13.436982 2014-08-13 18:06:06.396414 7f5c973a57a0 20 read_log 194215'1593 (194215'1176) delete ab250fab/benchmark_data_storage0111-ib.pocstor_19374_object20914/head//1 by client.1048726.0:47522 2014-08-12 17:12:13.439482 2014-08-13 18:06:06.396436 7f5c973a57a0 20 read_log 194215'1594 (194215'1177) delete ae70372b/benchmark_data_storage0111-ib.pocstor_19374_object21028/head//1 by client.1048726.0:47636 2014-08-12 17:12:13.478139 2014-08-13 18:06:06.396463 7f5c973a57a0 20 read_log 194215'1595 (194215'1178) delete 1036baeb/benchmark_data_storage0111-ib.pocstor_19374_object21032/head//1 by client.1048726.0:47640 2014-08-12 17:12:13.478452 2014-08-13 18:06:06.396555 7f5c973a57a0 20 read_log 194215'1596 (194215'1179) delete 4a8e3aeb/benchmark_data_storage0111-ib.pocstor_19374_object21109/head//1 by client.1048726.0:47717 2014-08-12 17:12:13.505238 2014-08-13 18:06:06.396579 7f5c973a57a0 20 read_log 194215'1597 (194215'1180) delete 753fa2ab/benchmark_data_storage0111-ib.pocstor_19374_object21247/head//1 by client.1048726.0:47855 2014-08-12 17:12:13.551714 2014-08-13 18:06:06.396601 7f5c973a57a0 20 read_log 194215'1598 (194215'1181) delete f67584eb/benchmark_data_storage0111-ib.pocstor_19374_object21267/head//1 by client.1048726.0:47875 2014-08-12 17:12:13.558048 2014-08-13 18:06:06.396629 7f5c973a57a0 20 read_log 194215'1599 (194215'1182) delete fe828eab/benchmark_data_storage0111-ib.pocstor_19374_object21345/head//1 by client.1048726.0:47953 2014-08-12 17:12:13.579470 2014-08-13 18:06:06.396651 7f5c973a57a0 20 read_log 194215'1600 (194215'1183) delete 2ef9f66b/benchmark_data_storage0111-ib.pocstor_19374_object21352/head//1 by client.1048726.0:47960 2014-08-12 17:12:13.580847 2014-08-13 18:06:06.396673 7f5c973a57a0 20 read_log 194215'1601 (194215'1184) delete 3560716b/benchmark_data_storage0111-ib.pocstor_19374_object21409/head//1 by client.1048726.0:48017 2014-08-12 17:12:13.602273 2014-08-13 18:06:06.396718 7f5c973a57a0 20 read_log 194215'1602 (194215'1185) delete 7876b26b/benchmark_data_storage0111-ib.pocstor_19374_object21474/head//1 by client.1048726.0:48082 2014-08-12 17:12:13.625859 2014-08-13 18:06:06.396740 7f5c973a57a0 20 read_log 194215'1603 (194215'1186) delete a090132b/benchmark_data_storage0111-ib.pocstor_19374_object21525/head//1 by client.1048726.0:48133 2014-08-12 17:12:13.641818 2014-08-13 18:06:06.396780 7f5c973a57a0 20 read_log 194215'1604 (194215'1187) delete 2b02b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21551/head//1 by client.1048726.0:48159 2014-08-12 17:12:13.650570 2014-08-13 18:06:06.396808 7f5c973a57a0 20 read_log 194215'1605 (194215'1188) delete 2af0d52b/benchmark_data_storage0111-ib.pocstor_19374_object21651/head//1 by client.1048726.0:48259 2014-08-12 17:12:13.683146 2014-08-13 18:06:06.396830 7f5c973a57a0 20 read_log 194215'1606 (194215'1189) delete df94b4eb/benchmark_data_storage0111-ib.pocstor_19374_object21660/head//1 by client.1048726.0:48268 2014-08-12 17:12:13.685663 2014-08-13 18:06:06.396851 7f5c973a57a0 20 read_log 194215'1607 (194215'1190) delete 157f89ab/benchmark_data_storage0111-ib.pocstor_19374_object21754/head//1 by client.1048726.0:48362 2014-08-12 17:12:13.717148 2014-08-13 18:06:06.396873 7f5c973a57a0 20 read_log 194215'1608 (194215'1191) delete 267742eb/benchmark_data_storage0111-ib.pocstor_19374_object21769/head//1 by client.1048726.0:48377 2014-08-12 17:12:13.723023 2014-08-13 18:06:06.396894 7f5c973a57a0 20 read_log 194215'1609 (194215'1192) delete 7086f06b/benchmark_data_storage0111-ib.pocstor_19374_object21770/head//1 by client.1048726.0:48378 2014-08-12 17:12:13.723428 2014-08-13 18:06:06.396915 7f5c973a57a0 20 read_log 194215'1610 (194215'1193) delete 5d6f6f6b/benchmark_data_storage0111-ib.pocstor_19374_object21877/head//1 by client.1048726.0:48485 2014-08-12 17:12:13.756359 2014-08-13 18:06:06.396937 7f5c973a57a0 20 read_log 194215'1611 (194215'1194) delete 6a5b3deb/benchmark_data_storage0111-ib.pocstor_19374_object21945/head//1 by client.1048726.0:48553 2014-08-12 17:12:13.777382 2014-08-13 18:06:06.396961 7f5c973a57a0 20 read_log 194215'1612 (194215'1195) delete 66a13f2b/benchmark_data_storage0111-ib.pocstor_19374_object22113/head//1 by client.1048726.0:48721 2014-08-12 17:12:13.828658 2014-08-13 18:06:06.396982 7f5c973a57a0 20 read_log 194215'1613 (194215'1196) delete 207ab2eb/benchmark_data_storage0111-ib.pocstor_19374_object22174/head//1 by client.1048726.0:48782 2014-08-12 17:12:13.847221 2014-08-13 18:06:06.397004 7f5c973a57a0 20 read_log 194215'1614 (194215'1197) delete 752cbeab/benchmark_data_storage0111-ib.pocstor_19374_object22208/head//1 by client.1048726.0:48816 2014-08-12 17:12:13.859637 2014-08-13 18:06:06.397025 7f5c973a57a0 20 read_log 194215'1615 (194215'1198) delete 5f7e08eb/benchmark_data_storage0111-ib.pocstor_19374_object22348/head//1 by client.1048726.0:48956 2014-08-12 17:12:13.907818 2014-08-13 18:06:06.397046 7f5c973a57a0 20 read_log 194215'1616 (194215'1199) delete 1c91296b/benchmark_data_storage0111-ib.pocstor_19374_object22399/head//1 by client.1048726.0:49007 2014-08-12 17:12:13.925737 2014-08-13 18:06:06.397067 7f5c973a57a0 20 read_log 194215'1617 (194215'1200) delete ed68a1eb/benchmark_data_storage0111-ib.pocstor_19374_object22466/head//1 by client.1048726.0:49074 2014-08-12 17:12:13.947357 2014-08-13 18:06:06.397089 7f5c973a57a0 20 read_log 194215'1618 (194215'1201) delete db847beb/benchmark_data_storage0111-ib.pocstor_19374_object22490/head//1 by client.1048726.0:49098 2014-08-12 17:12:13.954034 2014-08-13 18:06:06.397110 7f5c973a57a0 20 read_log 194215'1619 (194215'1202) delete 9b61f7eb/benchmark_data_storage0111-ib.pocstor_19374_object22596/head//1 by client.1048726.0:49204 2014-08-12 17:12:13.992700 2014-08-13 18:06:06.397132 7f5c973a57a0 20 read_log 194215'1620 (194215'1203) delete 8df4ebab/benchmark_data_storage0111-ib.pocstor_19374_object22647/head//1 by client.1048726.0:49255 2014-08-12 17:12:14.009508 2014-08-13 18:06:06.397164 7f5c973a57a0 20 read_log 194215'1621 (194215'1204) delete ade90cab/benchmark_data_storage0111-ib.pocstor_19374_object22661/head//1 by client.1048726.0:49269 2014-08-12 17:12:14.013083 2014-08-13 18:06:06.397188 7f5c973a57a0 20 read_log 194215'1622 (194215'1205) delete d2df432b/benchmark_data_storage0111-ib.pocstor_19374_object22688/head//1 by client.1048726.0:49296 2014-08-12 17:12:14.022525 2014-08-13 18:06:06.397241 7f5c973a57a0 20 read_log 194215'1623 (194215'1206) delete f0a1f42b/benchmark_data_storage0111-ib.pocstor_19374_object22740/head//1 by client.1048726.0:49348 2014-08-12 17:12:14.041292 2014-08-13 18:06:06.397511 7f5c973a57a0 20 read_log 194215'1624 (194215'1207) delete b467b16b/benchmark_data_storage0111-ib.pocstor_19374_object22893/head//1 by client.1048726.0:49501 2014-08-12 17:12:14.097520 2014-08-13 18:06:06.397604 7f5c973a57a0 20 read_log 194215'1625 (194215'1208) delete 2febd7ab/benchmark_data_storage0111-ib.pocstor_19374_object22894/head//1 by client.1048726.0:49502 2014-08-12 17:12:14.097577 2014-08-13 18:06:06.397670 7f5c973a57a0 20 read_log 194215'1626 (194215'1209) delete e0db75ab/benchmark_data_storage0111-ib.pocstor_19374_object22965/head//1 by client.1048726.0:49573 2014-08-12 17:12:14.123655 2014-08-13 18:06:06.397721 7f5c973a57a0 20 read_log 194215'1627 (194215'1210) delete 6344e4eb/benchmark_data_storage0111-ib.pocstor_19374_object22992/head//1 by client.1048726.0:49600 2014-08-12 17:12:14.132342 2014-08-13 18:06:06.397764 7f5c973a57a0 20 read_log 194215'1628 (194215'1211) delete 7ba9fd2b/benchmark_data_storage0111-ib.pocstor_19374_object23012/head//1 by client.1048726.0:49620 2014-08-12 17:12:14.138767 2014-08-13 18:06:06.397793 7f5c973a57a0 20 read_log 194215'1629 (194215'1212) delete 8666e06b/benchmark_data_storage0111-ib.pocstor_19374_object23095/head//1 by client.1048726.0:49703 2014-08-12 17:12:14.171793 2014-08-13 18:06:06.397819 7f5c973a57a0 20 read_log 194215'1630 (194215'1213) delete ed901d2b/benchmark_data_storage0111-ib.pocstor_19374_object23113/head//1 by client.1048726.0:49721 2014-08-12 17:12:14.178096 2014-08-13 18:06:06.397841 7f5c973a57a0 20 read_log 194215'1631 (194215'1214) delete 84443b6b/benchmark_data_storage0111-ib.pocstor_19374_object23132/head//1 by client.1048726.0:49740 2014-08-12 17:12:14.183503 2014-08-13 18:06:06.397863 7f5c973a57a0 20 read_log 194215'1632 (194215'1215) delete 8cff746b/benchmark_data_storage0111-ib.pocstor_19374_object23361/head//1 by client.1048726.0:49969 2014-08-12 17:12:14.259734 2014-08-13 18:06:06.397885 7f5c973a57a0 20 read_log 194215'1633 (194215'1216) delete 4edb3b2b/benchmark_data_storage0111-ib.pocstor_19374_object23379/head//1 by client.1048726.0:49987 2014-08-12 17:12:14.264414 2014-08-13 18:06:06.397907 7f5c973a57a0 20 read_log 194215'1634 (194215'1217) delete 1c53ca2b/benchmark_data_storage0111-ib.pocstor_19374_object23426/head//1 by client.1048726.0:50034 2014-08-12 17:12:14.278685 2014-08-13 18:06:06.397974 7f5c973a57a0 20 read_log 194215'1635 (194215'1218) delete fa5b282b/benchmark_data_storage0111-ib.pocstor_19374_object23429/head//1 by client.1048726.0:50037 2014-08-12 17:12:14.279660 2014-08-13 18:06:06.398002 7f5c973a57a0 20 read_log 194215'1636 (194215'1219) delete 5334536b/benchmark_data_storage0111-ib.pocstor_19374_object23433/head//1 by client.1048726.0:50041 2014-08-12 17:12:14.281084 2014-08-13 18:06:06.398025 7f5c973a57a0 20 read_log 194215'1637 (194215'1220) delete 6211b72b/benchmark_data_storage0111-ib.pocstor_19374_object23453/head//1 by client.1048726.0:50061 2014-08-12 17:12:14.289776 2014-08-13 18:06:06.398046 7f5c973a57a0 20 read_log 194215'1638 (194215'1221) delete 2514846b/benchmark_data_storage0111-ib.pocstor_19374_object23592/head//1 by client.1048726.0:50200 2014-08-12 17:12:14.335200 2014-08-13 18:06:06.398068 7f5c973a57a0 20 read_log 194215'1639 (194215'1222) delete aae84e2b/benchmark_data_storage0111-ib.pocstor_19374_object23706/head//1 by client.1048726.0:50314 2014-08-12 17:12:14.376996 2014-08-13 18:06:06.398089 7f5c973a57a0 20 read_log 194215'1640 (194215'1223) delete b9e840ab/benchmark_data_storage0111-ib.pocstor_19374_object23839/head//1 by client.1048726.0:50447 2014-08-12 17:12:14.422302 2014-08-13 18:06:06.398111 7f5c973a57a0 20 read_log 194215'1641 (194215'1224) delete 4cb71eb/benchmark_data_storage0111-ib.pocstor_19374_object23922/head//1 by client.1048726.0:50530 2014-08-12 17:12:14.451835 2014-08-13 18:06:06.398179 7f5c973a57a0 20 read_log 194215'1642 (194215'1225) delete e4b2e06b/benchmark_data_storage0111-ib.pocstor_19374_object23948/head//1 by client.1048726.0:50556 2014-08-12 17:12:14.461716 2014-08-13 18:06:06.398237 7f5c973a57a0 20 read_log 194215'1643 (194215'1226) delete d835436b/benchmark_data_storage0111-ib.pocstor_19374_object24094/head//1 by client.1048726.0:50702 2014-08-12 17:12:14.522150 2014-08-13 18:06:06.398326 7f5c973a57a0 20 read_log 194215'1644 (194215'1227) delete a7483a2b/benchmark_data_storage0111-ib.pocstor_19374_object24133/head//1 by client.1048726.0:50741 2014-08-12 17:12:14.538926 2014-08-13 18:06:06.398361 7f5c973a57a0 20 read_log 194215'1645 (194215'1228) delete 4afb40eb/benchmark_data_storage0111-ib.pocstor_19374_object24160/head//1 by client.1048726.0:50768 2014-08-12 17:12:14.551542 2014-08-13 18:06:06.398406 7f5c973a57a0 20 read_log 194215'1646 (194215'1229) delete 273f5cab/benchmark_data_storage0111-ib.pocstor_19374_object24162/head//1 by client.1048726.0:50770 2014-08-12 17:12:14.554395 2014-08-13 18:06:06.398456 7f5c973a57a0 20 read_log 194215'1647 (194215'1230) delete 438a14eb/benchmark_data_storage0111-ib.pocstor_19374_object24263/head//1 by client.1048726.0:50871 2014-08-12 17:12:14.612903 2014-08-13 18:06:06.398503 7f5c973a57a0 20 read_log 194215'1648 (194215'1231) delete 33c0b76b/benchmark_data_storage0111-ib.pocstor_19374_object24565/head//1 by client.1048726.0:51173 2014-08-12 17:12:14.777834 2014-08-13 18:06:06.398556 7f5c973a57a0 20 read_log 194215'1649 (194215'1232) delete c08eceab/benchmark_data_storage0111-ib.pocstor_19374_object24597/head//1 by client.1048726.0:51205 2014-08-12 17:12:14.789974 2014-08-13 18:06:06.398604 7f5c973a57a0 20 read_log 194215'1650 (194215'1233) delete befd7d6b/benchmark_data_storage0111-ib.pocstor_19374_object24728/head//1 by client.1048726.0:51336 2014-08-12 17:12:14.838717 2014-08-13 18:06:06.398631 7f5c973a57a0 20 read_log 194215'1651 (194215'1234) delete e425ca6b/benchmark_data_storage0111-ib.pocstor_19374_object24734/head//1 by client.1048726.0:51342 2014-08-12 17:12:14.841588 2014-08-13 18:06:06.398653 7f5c973a57a0 20 read_log 194215'1652 (194215'1235) delete 961e2f2b/benchmark_data_storage0111-ib.pocstor_19374_object24748/head//1 by client.1048726.0:51356 2014-08-12 17:12:14.845298 2014-08-13 18:06:06.398676 7f5c973a57a0 20 read_log 194215'1653 (194215'1236) delete 4a2485eb/benchmark_data_storage0111-ib.pocstor_19374_object24798/head//1 by client.1048726.0:51406 2014-08-12 17:12:14.863505 2014-08-13 18:06:06.398697 7f5c973a57a0 20 read_log 194215'1654 (194215'1237) delete 66eb74eb/benchmark_data_storage0111-ib.pocstor_19374_object24869/head//1 by client.1048726.0:51477 2014-08-12 17:12:14.889857 2014-08-13 18:06:06.398719 7f5c973a57a0 20 read_log 194215'1655 (194215'1238) delete cdd7d8eb/benchmark_data_storage0111-ib.pocstor_19374_object24973/head//1 by client.1048726.0:51581 2014-08-12 17:12:14.930499 2014-08-13 18:06:06.398740 7f5c973a57a0 20 read_log 194215'1656 (194215'1239) delete a8ff1b2b/benchmark_data_storage0111-ib.pocstor_19374_object25163/head//1 by client.1048726.0:51771 2014-08-12 17:12:15.003035 2014-08-13 18:06:06.398762 7f5c973a57a0 20 read_log 194215'1657 (194215'1240) delete 80a012eb/benchmark_data_storage0111-ib.pocstor_19374_object25209/head//1 by client.1048726.0:51817 2014-08-12 17:12:15.018532 2014-08-13 18:06:06.398784 7f5c973a57a0 20 read_log 194215'1658 (194215'1241) delete 6a06c22b/benchmark_data_storage0111-ib.pocstor_19374_object25224/head//1 by client.1048726.0:51832 2014-08-12 17:12:15.024416 2014-08-13 18:06:06.398807 7f5c973a57a0 20 read_log 194215'1659 (194215'1242) delete 1c47e46b/benchmark_data_storage0111-ib.pocstor_19374_object25488/head//1 by client.1048726.0:52096 2014-08-12 17:12:15.114148 2014-08-13 18:06:06.398828 7f5c973a57a0 20 read_log 194215'1660 (194215'1243) delete d92e436b/benchmark_data_storage0111-ib.pocstor_19374_object25630/head//1 by client.1048726.0:52238 2014-08-12 17:12:15.160166 2014-08-13 18:06:06.398850 7f5c973a57a0 20 read_log 194215'1661 (194215'1244) delete abae06ab/benchmark_data_storage0111-ib.pocstor_19374_object25701/head//1 by client.1048726.0:52309 2014-08-12 17:12:15.185855 2014-08-13 18:06:06.398871 7f5c973a57a0 20 read_log 194215'1662 (194215'1245) delete 63f6f26b/benchmark_data_storage0111-ib.pocstor_19374_object25733/head//1 by client.1048726.0:52341 2014-08-12 17:12:15.195757 2014-08-13 18:06:06.398893 7f5c973a57a0 20 read_log 194215'1663 (194215'1246) delete 65c6a56b/benchmark_data_storage0111-ib.pocstor_19374_object25806/head//1 by client.1048726.0:52414 2014-08-12 17:12:15.220855 2014-08-13 18:06:06.398939 7f5c973a57a0 20 read_log 194215'1664 (194215'1247) delete f2f70f6b/benchmark_data_storage0111-ib.pocstor_19374_object25849/head//1 by client.1048726.0:52457 2014-08-12 17:12:15.237658 2014-08-13 18:06:06.398966 7f5c973a57a0 20 read_log 194215'1665 (194215'1248) delete c4335c2b/benchmark_data_storage0111-ib.pocstor_19374_object25922/head//1 by client.1048726.0:52530 2014-08-12 17:12:15.272334 2014-08-13 18:06:06.398988 7f5c973a57a0 20 read_log 194215'1666 (194215'1249) delete 559f96eb/benchmark_data_storage0111-ib.pocstor_19374_object26064/head//1 by client.1048726.0:52672 2014-08-12 17:12:15.328418 2014-08-13 18:06:06.399010 7f5c973a57a0 20 read_log 194215'1667 (194215'1250) delete b6adc82b/benchmark_data_storage0111-ib.pocstor_19374_object26221/head//1 by client.1048726.0:52829 2014-08-12 17:12:15.392797 2014-08-13 18:06:06.399032 7f5c973a57a0 20 read_log 194215'1668 (194215'1251) delete 76fd52ab/benchmark_data_storage0111-ib.pocstor_19374_object26229/head//1 by client.1048726.0:52837 2014-08-12 17:12:15.397904 2014-08-13 18:06:06.399053 7f5c973a57a0 20 read_log 194215'1669 (194215'1252) delete 94dc9c2b/benchmark_data_storage0111-ib.pocstor_19374_object26322/head//1 by client.1048726.0:52930 2014-08-12 17:12:15.436794 2014-08-13 18:06:06.399075 7f5c973a57a0 20 read_log 194215'1670 (194215'1253) delete 7560a26b/benchmark_data_storage0111-ib.pocstor_19374_object26345/head//1 by client.1048726.0:52953 2014-08-12 17:12:15.444980 2014-08-13 18:06:06.399096 7f5c973a57a0 20 read_log 194215'1671 (194215'1254) delete 1f77502b/benchmark_data_storage0111-ib.pocstor_19374_object26385/head//1 by client.1048726.0:52993 2014-08-12 17:12:15.462192 2014-08-13 18:06:06.399117 7f5c973a57a0 20 read_log 194215'1672 (194215'1255) delete 6b90772b/benchmark_data_storage0111-ib.pocstor_19374_object26540/head//1 by client.1048726.0:53148 2014-08-12 17:12:15.531821 2014-08-13 18:06:06.410603 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.410705 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.410755 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.059852 0 0.000000 2014-08-13 18:06:06.410782 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.410797 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.410813 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'1672], crt=0'0) 2014-08-13 18:06:06.410829 7f5c973a57a0 10 osd.4 194487 pgid 2.2a coll 2.2a_head 2014-08-13 18:06:06.411171 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 2.2a 2014-08-13 18:06:06.411188 7f5c973a57a0 10 osd.4 194487 _get_pool 2 2014-08-13 18:06:06.411252 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[2.2a(unlocked)] enter Initial 2014-08-13 18:06:06.411268 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[2.2a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.411510 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.411764 7f5c973a57a0 20 read_log 194215'1 (0'0) modify 68edfb2a/benchmark_data_storage0111-ib.pocstor_48865_object150/head//2 by client.2191635.0:151 2014-08-12 17:14:57.943540 2014-08-13 18:06:06.411803 7f5c973a57a0 20 read_log 194215'2 (0'0) modify 11847ea/benchmark_data_storage0111-ib.pocstor_48865_object183/head//2 by client.2191635.0:184 2014-08-12 17:14:58.069710 2014-08-13 18:06:06.411827 7f5c973a57a0 20 read_log 194215'3 (0'0) modify 282e8a2a/benchmark_data_storage0111-ib.pocstor_48865_object269/head//2 by client.2191635.0:270 2014-08-12 17:14:58.390170 2014-08-13 18:06:06.411849 7f5c973a57a0 20 read_log 194215'4 (0'0) modify 2975e0ea/benchmark_data_storage0111-ib.pocstor_48865_object333/head//2 by client.2191635.0:334 2014-08-12 17:14:58.630161 2014-08-13 18:06:06.411871 7f5c973a57a0 20 read_log 194215'5 (0'0) modify 65323c2a/benchmark_data_storage0111-ib.pocstor_48865_object358/head//2 by client.2191635.0:359 2014-08-12 17:14:58.725101 2014-08-13 18:06:06.411892 7f5c973a57a0 20 read_log 194215'6 (0'0) modify 80c081ea/benchmark_data_storage0111-ib.pocstor_48865_object538/head//2 by client.2191635.0:539 2014-08-12 17:14:59.397550 2014-08-13 18:06:06.411913 7f5c973a57a0 20 read_log 194215'7 (0'0) modify 277a9c6a/benchmark_data_storage0111-ib.pocstor_48865_object629/head//2 by client.2191635.0:630 2014-08-12 17:14:59.742225 2014-08-13 18:06:06.411934 7f5c973a57a0 20 read_log 194215'8 (0'0) modify ce42036a/benchmark_data_storage0111-ib.pocstor_48865_object668/head//2 by client.2191635.0:669 2014-08-12 17:14:59.892946 2014-08-13 18:06:06.411955 7f5c973a57a0 20 read_log 194215'9 (0'0) modify 5955272a/benchmark_data_storage0111-ib.pocstor_48865_object715/head//2 by client.2191635.0:716 2014-08-12 17:15:00.103594 2014-08-13 18:06:06.411977 7f5c973a57a0 20 read_log 194215'10 (0'0) modify 52f5892a/benchmark_data_storage0111-ib.pocstor_48865_object726/head//2 by client.2191635.0:727 2014-08-12 17:15:00.171781 2014-08-13 18:06:06.412001 7f5c973a57a0 20 read_log 194215'11 (0'0) modify d1e810aa/benchmark_data_storage0111-ib.pocstor_48865_object1003/head//2 by client.2191635.0:1004 2014-08-12 17:15:01.226924 2014-08-13 18:06:06.412023 7f5c973a57a0 20 read_log 194215'12 (0'0) modify 70ee6b6a/benchmark_data_storage0111-ib.pocstor_48865_object1014/head//2 by client.2191635.0:1015 2014-08-12 17:15:01.268015 2014-08-13 18:06:06.412044 7f5c973a57a0 20 read_log 194215'13 (0'0) modify 81dd632a/benchmark_data_storage0111-ib.pocstor_48865_object1055/head//2 by client.2191635.0:1056 2014-08-12 17:15:01.421283 2014-08-13 18:06:06.412065 7f5c973a57a0 20 read_log 194215'14 (0'0) modify a31ef9aa/benchmark_data_storage0111-ib.pocstor_48865_object1066/head//2 by client.2191635.0:1067 2014-08-12 17:15:01.462438 2014-08-13 18:06:06.412099 7f5c973a57a0 20 read_log 194215'15 (0'0) modify bb8082a/benchmark_data_storage0111-ib.pocstor_48865_object1073/head//2 by client.2191635.0:1074 2014-08-12 17:15:01.488055 2014-08-13 18:06:06.412122 7f5c973a57a0 20 read_log 194215'16 (0'0) modify 8c5d0caa/benchmark_data_storage0111-ib.pocstor_48865_object1105/head//2 by client.2191635.0:1106 2014-08-12 17:15:01.607193 2014-08-13 18:06:06.412143 7f5c973a57a0 20 read_log 194215'17 (0'0) modify e781e86a/benchmark_data_storage0111-ib.pocstor_48865_object1126/head//2 by client.2191635.0:1127 2014-08-12 17:15:01.687538 2014-08-13 18:06:06.412177 7f5c973a57a0 20 read_log 194215'18 (0'0) modify bb59aaea/benchmark_data_storage0111-ib.pocstor_48865_object1239/head//2 by client.2191635.0:1240 2014-08-12 17:15:02.164730 2014-08-13 18:06:06.412199 7f5c973a57a0 20 read_log 194215'19 (0'0) modify f812992a/benchmark_data_storage0111-ib.pocstor_48865_object1316/head//2 by client.2191635.0:1317 2014-08-12 17:15:02.454095 2014-08-13 18:06:06.412223 7f5c973a57a0 20 read_log 194215'20 (0'0) modify 8845246a/benchmark_data_storage0111-ib.pocstor_48865_object1340/head//2 by client.2191635.0:1341 2014-08-12 17:15:02.544008 2014-08-13 18:06:06.412244 7f5c973a57a0 20 read_log 194215'21 (0'0) modify ac40caa/benchmark_data_storage0111-ib.pocstor_48865_object1424/head//2 by client.2191635.0:1425 2014-08-12 17:15:02.868950 2014-08-13 18:06:06.412266 7f5c973a57a0 20 read_log 194215'22 (0'0) modify ff0610aa/benchmark_data_storage0111-ib.pocstor_48865_object1571/head//2 by client.2191635.0:1572 2014-08-12 17:15:03.430301 2014-08-13 18:06:06.412287 7f5c973a57a0 20 read_log 194215'23 (0'0) modify 95a794ea/benchmark_data_storage0111-ib.pocstor_48865_object1601/head//2 by client.2191635.0:1602 2014-08-12 17:15:03.546025 2014-08-13 18:06:06.412310 7f5c973a57a0 20 read_log 194215'24 (0'0) modify cae8062a/benchmark_data_storage0111-ib.pocstor_48865_object1620/head//2 by client.2191635.0:1621 2014-08-12 17:15:03.618251 2014-08-13 18:06:06.412338 7f5c973a57a0 20 read_log 194215'25 (0'0) modify 920ef56a/benchmark_data_storage0111-ib.pocstor_48865_object1759/head//2 by client.2191635.0:1760 2014-08-12 17:15:04.156730 2014-08-13 18:06:06.412360 7f5c973a57a0 20 read_log 194215'26 (0'0) modify a732e2a/benchmark_data_storage0111-ib.pocstor_48865_object1807/head//2 by client.2191635.0:1808 2014-08-12 17:15:04.343397 2014-08-13 18:06:06.412381 7f5c973a57a0 20 read_log 194215'27 (0'0) modify c4c702aa/benchmark_data_storage0111-ib.pocstor_48865_object1871/head//2 by client.2191635.0:1872 2014-08-12 17:15:04.591112 2014-08-13 18:06:06.412409 7f5c973a57a0 20 read_log 194215'28 (0'0) modify 279c346a/benchmark_data_storage0111-ib.pocstor_48865_object1875/head//2 by client.2191635.0:1876 2014-08-12 17:15:04.606512 2014-08-13 18:06:06.412431 7f5c973a57a0 20 read_log 194215'29 (0'0) modify 553263ea/benchmark_data_storage0111-ib.pocstor_48865_object2043/head//2 by client.2191635.0:2044 2014-08-12 17:15:05.590851 2014-08-13 18:06:06.412452 7f5c973a57a0 20 read_log 194215'30 (0'0) modify f16ac3ea/benchmark_data_storage0111-ib.pocstor_48865_object2102/head//2 by client.2191635.0:2103 2014-08-12 17:15:05.812638 2014-08-13 18:06:06.412479 7f5c973a57a0 20 read_log 194215'31 (0'0) modify 5fc8512a/benchmark_data_storage0111-ib.pocstor_48865_object2104/head//2 by client.2191635.0:2105 2014-08-12 17:15:05.820138 2014-08-13 18:06:06.412500 7f5c973a57a0 20 read_log 194215'32 (0'0) modify a896f2ea/benchmark_data_storage0111-ib.pocstor_48865_object2127/head//2 by client.2191635.0:2128 2014-08-12 17:15:05.905528 2014-08-13 18:06:06.412522 7f5c973a57a0 20 read_log 194215'33 (0'0) modify b516fdaa/benchmark_data_storage0111-ib.pocstor_48865_object2151/head//2 by client.2191635.0:2152 2014-08-12 17:15:05.994619 2014-08-13 18:06:06.412569 7f5c973a57a0 20 read_log 194215'34 (0'0) modify 8cda0e6a/benchmark_data_storage0111-ib.pocstor_48865_object2185/head//2 by client.2191635.0:2186 2014-08-12 17:15:06.120792 2014-08-13 18:06:06.412623 7f5c973a57a0 20 read_log 194215'35 (0'0) modify 130ec9aa/benchmark_data_storage0111-ib.pocstor_48865_object2195/head//2 by client.2191635.0:2196 2014-08-12 17:15:06.157124 2014-08-13 18:06:06.412657 7f5c973a57a0 20 read_log 194215'36 (0'0) modify d4544c2a/benchmark_data_storage0111-ib.pocstor_48865_object2225/head//2 by client.2191635.0:2226 2014-08-12 17:15:06.267584 2014-08-13 18:06:06.412696 7f5c973a57a0 20 read_log 194215'37 (0'0) modify 8855e5aa/benchmark_data_storage0111-ib.pocstor_48865_object2244/head//2 by client.2191635.0:2245 2014-08-12 17:15:06.338488 2014-08-13 18:06:06.412720 7f5c973a57a0 20 read_log 194215'38 (0'0) modify 875f70ea/benchmark_data_storage0111-ib.pocstor_48865_object2251/head//2 by client.2191635.0:2252 2014-08-12 17:15:06.364235 2014-08-13 18:06:06.412742 7f5c973a57a0 20 read_log 194215'39 (0'0) modify 365a75ea/benchmark_data_storage0111-ib.pocstor_48865_object2593/head//2 by client.2191635.0:2594 2014-08-12 17:15:07.629790 2014-08-13 18:06:06.412765 7f5c973a57a0 20 read_log 194215'40 (0'0) modify 1cc465aa/benchmark_data_storage0111-ib.pocstor_48865_object2601/head//2 by client.2191635.0:2602 2014-08-12 17:15:07.662124 2014-08-13 18:06:06.412786 7f5c973a57a0 20 read_log 194215'41 (0'0) modify b7b2a22a/benchmark_data_storage0111-ib.pocstor_48865_object2629/head//2 by client.2191635.0:2630 2014-08-12 17:15:07.765164 2014-08-13 18:06:06.412808 7f5c973a57a0 20 read_log 194215'42 (0'0) modify 109903aa/benchmark_data_storage0111-ib.pocstor_48865_object2691/head//2 by client.2191635.0:2692 2014-08-12 17:15:07.996861 2014-08-13 18:06:06.412829 7f5c973a57a0 20 read_log 194215'43 (0'0) modify db63af6a/benchmark_data_storage0111-ib.pocstor_48865_object2699/head//2 by client.2191635.0:2700 2014-08-12 17:15:08.027377 2014-08-13 18:06:06.412861 7f5c973a57a0 20 read_log 194215'44 (0'0) modify 3dfdc96a/benchmark_data_storage0111-ib.pocstor_48865_object2712/head//2 by client.2191635.0:2713 2014-08-12 17:15:08.074848 2014-08-13 18:06:06.412890 7f5c973a57a0 20 read_log 194215'45 (0'0) modify 839a1faa/benchmark_data_storage0111-ib.pocstor_48865_object2786/head//2 by client.2191635.0:2787 2014-08-12 17:15:08.347678 2014-08-13 18:06:06.412913 7f5c973a57a0 20 read_log 194215'46 (0'0) modify 7d0316aa/benchmark_data_storage0111-ib.pocstor_48865_object2888/head//2 by client.2191635.0:2889 2014-08-12 17:15:08.721395 2014-08-13 18:06:06.412934 7f5c973a57a0 20 read_log 194215'47 (0'0) modify 8fd59f2a/benchmark_data_storage0111-ib.pocstor_48865_object3016/head//2 by client.2191635.0:3017 2014-08-12 17:15:09.188152 2014-08-13 18:06:06.412956 7f5c973a57a0 20 read_log 194215'48 (0'0) modify 2f6cc42a/benchmark_data_storage0111-ib.pocstor_48865_object3021/head//2 by client.2191635.0:3022 2014-08-12 17:15:09.206474 2014-08-13 18:06:06.412977 7f5c973a57a0 20 read_log 194215'49 (0'0) modify 32c490ea/benchmark_data_storage0111-ib.pocstor_48865_object3081/head//2 by client.2191635.0:3082 2014-08-12 17:15:09.426514 2014-08-13 18:06:06.412998 7f5c973a57a0 20 read_log 194215'50 (0'0) modify ead47aea/benchmark_data_storage0111-ib.pocstor_48865_object3105/head//2 by client.2191635.0:3106 2014-08-12 17:15:09.514927 2014-08-13 18:06:06.413019 7f5c973a57a0 20 read_log 194215'51 (0'0) modify bddd492a/benchmark_data_storage0111-ib.pocstor_48865_object3141/head//2 by client.2191635.0:3142 2014-08-12 17:15:09.648681 2014-08-13 18:06:06.413040 7f5c973a57a0 20 read_log 194215'52 (0'0) modify f9981caa/benchmark_data_storage0111-ib.pocstor_48865_object3321/head//2 by client.2191635.0:3322 2014-08-12 17:15:10.397453 2014-08-13 18:06:06.413099 7f5c973a57a0 20 read_log 194215'53 (0'0) modify d01bb4ea/benchmark_data_storage0111-ib.pocstor_48865_object3371/head//2 by client.2191635.0:3372 2014-08-12 17:15:10.624977 2014-08-13 18:06:06.413121 7f5c973a57a0 20 read_log 194215'54 (0'0) modify f0690b6a/benchmark_data_storage0111-ib.pocstor_48865_object3420/head//2 by client.2191635.0:3421 2014-08-12 17:15:10.802546 2014-08-13 18:06:06.413188 7f5c973a57a0 20 read_log 194215'55 (0'0) modify ef7fe82a/benchmark_data_storage0111-ib.pocstor_48865_object3426/head//2 by client.2191635.0:3427 2014-08-12 17:15:10.825579 2014-08-13 18:06:06.413257 7f5c973a57a0 20 read_log 194215'56 (0'0) modify 4f5aa4ea/benchmark_data_storage0111-ib.pocstor_48865_object3456/head//2 by client.2191635.0:3457 2014-08-12 17:15:10.934509 2014-08-13 18:06:06.413283 7f5c973a57a0 20 read_log 194215'57 (0'0) modify e192b76a/benchmark_data_storage0111-ib.pocstor_48865_object3465/head//2 by client.2191635.0:3466 2014-08-12 17:15:10.967414 2014-08-13 18:06:06.413305 7f5c973a57a0 20 read_log 194215'58 (0'0) modify ed61ad6a/benchmark_data_storage0111-ib.pocstor_48865_object3599/head//2 by client.2191635.0:3600 2014-08-12 17:15:11.462967 2014-08-13 18:06:06.413331 7f5c973a57a0 20 read_log 194215'59 (0'0) modify 284476ea/benchmark_data_storage0111-ib.pocstor_48865_object3637/head//2 by client.2191635.0:3638 2014-08-12 17:15:11.604438 2014-08-13 18:06:06.413353 7f5c973a57a0 20 read_log 194215'60 (0'0) modify 69ffe8ea/benchmark_data_storage0111-ib.pocstor_48865_object3643/head//2 by client.2191635.0:3644 2014-08-12 17:15:11.626588 2014-08-13 18:06:06.413380 7f5c973a57a0 20 read_log 194215'61 (0'0) modify 4443936a/benchmark_data_storage0111-ib.pocstor_48865_object3673/head//2 by client.2191635.0:3674 2014-08-12 17:15:11.738317 2014-08-13 18:06:06.413401 7f5c973a57a0 20 read_log 194215'62 (0'0) modify b1fb3aaa/benchmark_data_storage0111-ib.pocstor_48865_object3684/head//2 by client.2191635.0:3685 2014-08-12 17:15:11.778152 2014-08-13 18:06:06.413428 7f5c973a57a0 20 read_log 194215'63 (0'0) modify abbf486a/benchmark_data_storage0111-ib.pocstor_48865_object3708/head//2 by client.2191635.0:3709 2014-08-12 17:15:11.868006 2014-08-13 18:06:06.413449 7f5c973a57a0 20 read_log 194215'64 (0'0) modify d2774aaa/benchmark_data_storage0111-ib.pocstor_48865_object3723/head//2 by client.2191635.0:3724 2014-08-12 17:15:11.922298 2014-08-13 18:06:06.413497 7f5c973a57a0 20 read_log 194215'65 (0'0) modify d4716aaa/benchmark_data_storage0111-ib.pocstor_48865_object3873/head//2 by client.2191635.0:3874 2014-08-12 17:15:12.588987 2014-08-13 18:06:06.413525 7f5c973a57a0 20 read_log 194215'66 (0'0) modify 2546412a/benchmark_data_storage0111-ib.pocstor_48865_object3982/head//2 by client.2191635.0:3983 2014-08-12 17:15:13.175586 2014-08-13 18:06:06.413549 7f5c973a57a0 20 read_log 194215'67 (0'0) modify fce3fb2a/benchmark_data_storage0111-ib.pocstor_48865_object4047/head//2 by client.2191635.0:4048 2014-08-12 17:15:13.522121 2014-08-13 18:06:06.413585 7f5c973a57a0 20 read_log 194215'68 (0'0) modify 880c6cea/benchmark_data_storage0111-ib.pocstor_48865_object4061/head//2 by client.2191635.0:4062 2014-08-12 17:15:13.593244 2014-08-13 18:06:06.413608 7f5c973a57a0 20 read_log 194215'69 (0'0) modify da367caa/benchmark_data_storage0111-ib.pocstor_48865_object4094/head//2 by client.2191635.0:4095 2014-08-12 17:15:13.797370 2014-08-13 18:06:06.413630 7f5c973a57a0 20 read_log 194215'70 (0'0) modify fbd37baa/benchmark_data_storage0111-ib.pocstor_48865_object4145/head//2 by client.2191635.0:4146 2014-08-12 17:15:14.564150 2014-08-13 18:06:06.413657 7f5c973a57a0 20 read_log 194215'71 (0'0) modify df68caea/benchmark_data_storage0111-ib.pocstor_48865_object4151/head//2 by client.2191635.0:4152 2014-08-12 17:15:14.594531 2014-08-13 18:06:06.413679 7f5c973a57a0 20 read_log 194215'72 (0'0) modify 7757da6a/benchmark_data_storage0111-ib.pocstor_48865_object4234/head//2 by client.2191635.0:4235 2014-08-12 17:15:14.898093 2014-08-13 18:06:06.413701 7f5c973a57a0 20 read_log 194215'73 (0'0) modify aef9282a/benchmark_data_storage0111-ib.pocstor_48865_object4266/head//2 by client.2191635.0:4267 2014-08-12 17:15:15.017677 2014-08-13 18:06:06.413738 7f5c973a57a0 20 read_log 194215'74 (0'0) modify ca7790ea/benchmark_data_storage0111-ib.pocstor_48865_object4302/head//2 by client.2191635.0:4303 2014-08-12 17:15:15.183823 2014-08-13 18:06:06.413774 7f5c973a57a0 20 read_log 194215'75 (0'0) modify 5c96bb6a/benchmark_data_storage0111-ib.pocstor_48865_object4312/head//2 by client.2191635.0:4313 2014-08-12 17:15:15.229432 2014-08-13 18:06:06.413801 7f5c973a57a0 20 read_log 194215'76 (0'0) modify 463bdeaa/benchmark_data_storage0111-ib.pocstor_48865_object4392/head//2 by client.2191635.0:4393 2014-08-12 17:15:15.546223 2014-08-13 18:06:06.413822 7f5c973a57a0 20 read_log 194215'77 (0'0) modify d37ec2ea/benchmark_data_storage0111-ib.pocstor_48865_object4404/head//2 by client.2191635.0:4405 2014-08-12 17:15:15.589482 2014-08-13 18:06:06.413844 7f5c973a57a0 20 read_log 194215'78 (0'0) modify 50c68dea/benchmark_data_storage0111-ib.pocstor_48865_object4406/head//2 by client.2191635.0:4407 2014-08-12 17:15:15.596585 2014-08-13 18:06:06.413865 7f5c973a57a0 20 read_log 194215'79 (0'0) modify 4e5fdaaa/benchmark_data_storage0111-ib.pocstor_48865_object4437/head//2 by client.2191635.0:4438 2014-08-12 17:15:15.711200 2014-08-13 18:06:06.413886 7f5c973a57a0 20 read_log 194215'80 (0'0) modify c56423aa/benchmark_data_storage0111-ib.pocstor_48865_object4456/head//2 by client.2191635.0:4457 2014-08-12 17:15:15.780426 2014-08-13 18:06:06.413907 7f5c973a57a0 20 read_log 194215'81 (0'0) modify 48228f2a/benchmark_data_storage0111-ib.pocstor_48865_object4543/head//2 by client.2191635.0:4544 2014-08-12 17:15:16.099710 2014-08-13 18:06:06.413929 7f5c973a57a0 20 read_log 194215'82 (0'0) modify f096ccea/benchmark_data_storage0111-ib.pocstor_48865_object4594/head//2 by client.2191635.0:4595 2014-08-12 17:15:16.287510 2014-08-13 18:06:06.413950 7f5c973a57a0 20 read_log 194215'83 (0'0) modify 775974ea/benchmark_data_storage0111-ib.pocstor_48865_object4605/head//2 by client.2191635.0:4606 2014-08-12 17:15:16.328242 2014-08-13 18:06:06.413971 7f5c973a57a0 20 read_log 194215'84 (0'0) modify dee4596a/benchmark_data_storage0111-ib.pocstor_48865_object4660/head//2 by client.2191635.0:4661 2014-08-12 17:15:16.531495 2014-08-13 18:06:06.413992 7f5c973a57a0 20 read_log 194215'85 (0'0) modify b3d8632a/benchmark_data_storage0111-ib.pocstor_48865_object4719/head//2 by client.2191635.0:4720 2014-08-12 17:15:16.750627 2014-08-13 18:06:06.414013 7f5c973a57a0 20 read_log 194215'86 (0'0) modify dfa2a76a/benchmark_data_storage0111-ib.pocstor_48865_object4725/head//2 by client.2191635.0:4726 2014-08-12 17:15:16.773159 2014-08-13 18:06:06.414034 7f5c973a57a0 20 read_log 194215'87 (0'0) modify ea96c2aa/benchmark_data_storage0111-ib.pocstor_48865_object4778/head//2 by client.2191635.0:4779 2014-08-12 17:15:16.969685 2014-08-13 18:06:06.414055 7f5c973a57a0 20 read_log 194215'88 (0'0) modify 37ca6bea/benchmark_data_storage0111-ib.pocstor_48865_object4949/head//2 by client.2191635.0:4950 2014-08-12 17:15:17.604129 2014-08-13 18:06:06.414076 7f5c973a57a0 20 read_log 194215'89 (0'0) modify 62eca2ea/benchmark_data_storage0111-ib.pocstor_48865_object4961/head//2 by client.2191635.0:4962 2014-08-12 17:15:17.647996 2014-08-13 18:06:06.414097 7f5c973a57a0 20 read_log 194215'90 (0'0) modify 489460ea/benchmark_data_storage0111-ib.pocstor_48865_object4986/head//2 by client.2191635.0:4987 2014-08-12 17:15:17.742016 2014-08-13 18:06:06.414118 7f5c973a57a0 20 read_log 194215'91 (0'0) modify fc746b2a/benchmark_data_storage0111-ib.pocstor_48865_object5031/head//2 by client.2191635.0:5032 2014-08-12 17:15:17.906698 2014-08-13 18:06:06.414139 7f5c973a57a0 20 read_log 194215'92 (0'0) modify dc8ffd6a/benchmark_data_storage0111-ib.pocstor_48865_object5063/head//2 by client.2191635.0:5064 2014-08-12 17:15:18.026716 2014-08-13 18:06:06.414180 7f5c973a57a0 20 read_log 194215'93 (0'0) modify 98f0566a/benchmark_data_storage0111-ib.pocstor_48865_object5274/head//2 by client.2191635.0:5275 2014-08-12 17:15:18.976790 2014-08-13 18:06:06.414202 7f5c973a57a0 20 read_log 194215'94 (0'0) modify a673bdea/benchmark_data_storage0111-ib.pocstor_48865_object5308/head//2 by client.2191635.0:5309 2014-08-12 17:15:19.156388 2014-08-13 18:06:06.414246 7f5c973a57a0 20 read_log 194215'95 (0'0) modify dd5e12aa/benchmark_data_storage0111-ib.pocstor_48865_object5321/head//2 by client.2191635.0:5322 2014-08-12 17:15:19.218769 2014-08-13 18:06:06.414274 7f5c973a57a0 20 read_log 194215'96 (0'0) modify 743f146a/benchmark_data_storage0111-ib.pocstor_48865_object5342/head//2 by client.2191635.0:5343 2014-08-12 17:15:19.297748 2014-08-13 18:06:06.414300 7f5c973a57a0 20 read_log 194215'97 (0'0) modify 4b19ae2a/benchmark_data_storage0111-ib.pocstor_48865_object5365/head//2 by client.2191635.0:5366 2014-08-12 17:15:19.382771 2014-08-13 18:06:06.414367 7f5c973a57a0 20 read_log 194215'98 (0'0) modify 4187d4aa/benchmark_data_storage0111-ib.pocstor_48865_object5418/head//2 by client.2191635.0:5419 2014-08-12 17:15:19.579006 2014-08-13 18:06:06.414395 7f5c973a57a0 20 read_log 194215'99 (0'0) modify 76e737aa/benchmark_data_storage0111-ib.pocstor_48865_object5442/head//2 by client.2191635.0:5443 2014-08-12 17:15:19.667852 2014-08-13 18:06:06.414417 7f5c973a57a0 20 read_log 194215'100 (0'0) modify 90d3712a/benchmark_data_storage0111-ib.pocstor_48865_object5446/head//2 by client.2191635.0:5447 2014-08-12 17:15:19.684038 2014-08-13 18:06:06.414439 7f5c973a57a0 20 read_log 194215'101 (0'0) modify 59cb2aea/benchmark_data_storage0111-ib.pocstor_48865_object5453/head//2 by client.2191635.0:5454 2014-08-12 17:15:19.710685 2014-08-13 18:06:06.414460 7f5c973a57a0 20 read_log 194215'102 (0'0) modify d0ed1c6a/benchmark_data_storage0111-ib.pocstor_48865_object5498/head//2 by client.2191635.0:5499 2014-08-12 17:15:19.929410 2014-08-13 18:06:06.414498 7f5c973a57a0 20 read_log 194215'103 (0'0) modify 48e796a/benchmark_data_storage0111-ib.pocstor_48865_object5511/head//2 by client.2191635.0:5512 2014-08-12 17:15:20.000046 2014-08-13 18:06:06.414537 7f5c973a57a0 20 read_log 194215'104 (0'0) modify b179e16a/benchmark_data_storage0111-ib.pocstor_48865_object5567/head//2 by client.2191635.0:5568 2014-08-12 17:15:20.225598 2014-08-13 18:06:06.414562 7f5c973a57a0 20 read_log 194215'105 (0'0) modify d38639aa/benchmark_data_storage0111-ib.pocstor_48865_object5574/head//2 by client.2191635.0:5575 2014-08-12 17:15:20.251008 2014-08-13 18:06:06.414584 7f5c973a57a0 20 read_log 194215'106 (0'0) modify 6cb251aa/benchmark_data_storage0111-ib.pocstor_48865_object5598/head//2 by client.2191635.0:5599 2014-08-12 17:15:20.337605 2014-08-13 18:06:06.414605 7f5c973a57a0 20 read_log 194215'107 (0'0) modify eaccd2a/benchmark_data_storage0111-ib.pocstor_48865_object5617/head//2 by client.2191635.0:5618 2014-08-12 17:15:20.407130 2014-08-13 18:06:06.414627 7f5c973a57a0 20 read_log 194215'108 (0'0) modify a3f082aa/benchmark_data_storage0111-ib.pocstor_48865_object5622/head//2 by client.2191635.0:5623 2014-08-12 17:15:20.425080 2014-08-13 18:06:06.414648 7f5c973a57a0 20 read_log 194215'109 (0'0) modify a3af342a/benchmark_data_storage0111-ib.pocstor_48865_object5745/head//2 by client.2191635.0:5746 2014-08-12 17:15:20.875893 2014-08-13 18:06:06.414669 7f5c973a57a0 20 read_log 194215'110 (0'0) modify e572e62a/benchmark_data_storage0111-ib.pocstor_48865_object5964/head//2 by client.2191635.0:5965 2014-08-12 17:15:21.668664 2014-08-13 18:06:06.414691 7f5c973a57a0 20 read_log 194215'111 (0'0) modify b874d76a/benchmark_data_storage0111-ib.pocstor_48865_object5981/head//2 by client.2191635.0:5982 2014-08-12 17:15:21.731061 2014-08-13 18:06:06.414712 7f5c973a57a0 20 read_log 194215'112 (0'0) modify b9b163ea/benchmark_data_storage0111-ib.pocstor_48865_object5987/head//2 by client.2191635.0:5988 2014-08-12 17:15:21.752577 2014-08-13 18:06:06.414734 7f5c973a57a0 20 read_log 194215'113 (0'0) modify 48d2f52a/benchmark_data_storage0111-ib.pocstor_48865_object6057/head//2 by client.2191635.0:6058 2014-08-12 17:15:22.003642 2014-08-13 18:06:06.414755 7f5c973a57a0 20 read_log 194215'114 (0'0) modify 857610ea/benchmark_data_storage0111-ib.pocstor_48865_object6104/head//2 by client.2191635.0:6105 2014-08-12 17:15:22.217161 2014-08-13 18:06:06.414796 7f5c973a57a0 20 read_log 194215'115 (0'0) modify 2628a2ea/benchmark_data_storage0111-ib.pocstor_48865_object6300/head//2 by client.2191635.0:6301 2014-08-12 17:15:22.958415 2014-08-13 18:06:06.414819 7f5c973a57a0 20 read_log 194215'116 (0'0) modify 6374406a/benchmark_data_storage0111-ib.pocstor_48865_object6344/head//2 by client.2191635.0:6345 2014-08-12 17:15:23.119532 2014-08-13 18:06:06.414840 7f5c973a57a0 20 read_log 194215'117 (0'0) modify 1a2b33ea/benchmark_data_storage0111-ib.pocstor_48865_object6404/head//2 by client.2191635.0:6405 2014-08-12 17:15:23.334451 2014-08-13 18:06:06.414861 7f5c973a57a0 20 read_log 194215'118 (0'0) modify 56ba632a/benchmark_data_storage0111-ib.pocstor_48865_object6415/head//2 by client.2191635.0:6416 2014-08-12 17:15:23.374708 2014-08-13 18:06:06.414884 7f5c973a57a0 20 read_log 194215'119 (0'0) modify 7073ea/benchmark_data_storage0111-ib.pocstor_48865_object6523/head//2 by client.2191635.0:6524 2014-08-12 17:15:23.765336 2014-08-13 18:06:06.414906 7f5c973a57a0 20 read_log 194215'120 (0'0) modify 4be729ea/benchmark_data_storage0111-ib.pocstor_48865_object6640/head//2 by client.2191635.0:6641 2014-08-12 17:15:24.190079 2014-08-13 18:06:06.414927 7f5c973a57a0 20 read_log 194215'121 (0'0) modify f386bea/benchmark_data_storage0111-ib.pocstor_48865_object6650/head//2 by client.2191635.0:6651 2014-08-12 17:15:24.228296 2014-08-13 18:06:06.414990 7f5c973a57a0 20 read_log 194215'122 (0'0) modify 64fac96a/benchmark_data_storage0111-ib.pocstor_48865_object6732/head//2 by client.2191635.0:6733 2014-08-12 17:15:24.521396 2014-08-13 18:06:06.415012 7f5c973a57a0 20 read_log 194215'123 (0'0) modify 68644faa/benchmark_data_storage0111-ib.pocstor_48865_object6763/head//2 by client.2191635.0:6764 2014-08-12 17:15:24.633924 2014-08-13 18:06:06.415033 7f5c973a57a0 20 read_log 194215'124 (0'0) modify e27b086a/benchmark_data_storage0111-ib.pocstor_48865_object6779/head//2 by client.2191635.0:6780 2014-08-12 17:15:24.690074 2014-08-13 18:06:06.415054 7f5c973a57a0 20 read_log 194215'125 (0'0) modify db1f17ea/benchmark_data_storage0111-ib.pocstor_48865_object6852/head//2 by client.2191635.0:6853 2014-08-12 17:15:25.125365 2014-08-13 18:06:06.415078 7f5c973a57a0 20 read_log 194215'126 (0'0) modify 45f4e1ea/benchmark_data_storage0111-ib.pocstor_48865_object6955/head//2 by client.2191635.0:6956 2014-08-12 17:15:25.530331 2014-08-13 18:06:06.415099 7f5c973a57a0 20 read_log 194215'127 (0'0) modify 223884ea/benchmark_data_storage0111-ib.pocstor_48865_object7007/head//2 by client.2191635.0:7008 2014-08-12 17:15:25.717952 2014-08-13 18:06:06.415120 7f5c973a57a0 20 read_log 194215'128 (0'0) modify 84d19fea/benchmark_data_storage0111-ib.pocstor_48865_object7058/head//2 by client.2191635.0:7059 2014-08-12 17:15:25.901598 2014-08-13 18:06:06.415141 7f5c973a57a0 20 read_log 194215'129 (0'0) modify a64a352a/benchmark_data_storage0111-ib.pocstor_48865_object7111/head//2 by client.2191635.0:7112 2014-08-12 17:15:26.091912 2014-08-13 18:06:06.415176 7f5c973a57a0 20 read_log 194215'130 (0'0) modify 9abeec2a/benchmark_data_storage0111-ib.pocstor_48865_object7201/head//2 by client.2191635.0:7202 2014-08-12 17:15:26.413796 2014-08-13 18:06:06.415197 7f5c973a57a0 20 read_log 194215'131 (0'0) modify 2a4d06aa/benchmark_data_storage0111-ib.pocstor_48865_object7255/head//2 by client.2191635.0:7256 2014-08-12 17:15:26.606790 2014-08-13 18:06:06.415221 7f5c973a57a0 20 read_log 194215'132 (0'0) modify 64ec0faa/benchmark_data_storage0111-ib.pocstor_48865_object7261/head//2 by client.2191635.0:7262 2014-08-12 17:15:26.627828 2014-08-13 18:06:06.415242 7f5c973a57a0 20 read_log 194215'133 (0'0) modify d703ceaa/benchmark_data_storage0111-ib.pocstor_48865_object7323/head//2 by client.2191635.0:7324 2014-08-12 17:15:26.851411 2014-08-13 18:06:06.415266 7f5c973a57a0 20 read_log 194215'134 (0'0) modify afc50fea/benchmark_data_storage0111-ib.pocstor_48865_object7361/head//2 by client.2191635.0:7362 2014-08-12 17:15:26.988591 2014-08-13 18:06:06.415333 7f5c973a57a0 20 read_log 194215'135 (0'0) modify 70d399aa/benchmark_data_storage0111-ib.pocstor_48865_object7567/head//2 by client.2191635.0:7568 2014-08-12 17:15:27.729882 2014-08-13 18:06:06.415357 7f5c973a57a0 20 read_log 194215'136 (0'0) modify 7e608b6a/benchmark_data_storage0111-ib.pocstor_48865_object7574/head//2 by client.2191635.0:7575 2014-08-12 17:15:27.755390 2014-08-13 18:06:06.415379 7f5c973a57a0 20 read_log 194215'137 (0'0) modify 55a67f2a/benchmark_data_storage0111-ib.pocstor_48865_object7751/head//2 by client.2191635.0:7752 2014-08-12 17:15:28.392786 2014-08-13 18:06:06.415400 7f5c973a57a0 20 read_log 194215'138 (0'0) modify c7ab856a/benchmark_data_storage0111-ib.pocstor_48865_object7992/head//2 by client.2191635.0:7993 2014-08-12 17:15:30.007218 2014-08-13 18:06:06.415438 7f5c973a57a0 20 read_log 194215'139 (0'0) modify 89882f2a/benchmark_data_storage0111-ib.pocstor_48865_object8149/head//2 by client.2191635.0:8150 2014-08-12 17:15:30.596244 2014-08-13 18:06:06.415461 7f5c973a57a0 20 read_log 194215'140 (0'0) modify 560df9ea/benchmark_data_storage0111-ib.pocstor_48865_object8228/head//2 by client.2191635.0:8229 2014-08-12 17:15:30.880045 2014-08-13 18:06:06.415504 7f5c973a57a0 20 read_log 194215'141 (0'0) modify 88c4606a/benchmark_data_storage0111-ib.pocstor_48865_object8266/head//2 by client.2191635.0:8267 2014-08-12 17:15:31.016827 2014-08-13 18:06:06.415533 7f5c973a57a0 20 read_log 194215'142 (0'0) modify addd1f6a/benchmark_data_storage0111-ib.pocstor_48865_object8285/head//2 by client.2191635.0:8286 2014-08-12 17:15:31.118403 2014-08-13 18:06:06.415568 7f5c973a57a0 20 read_log 194215'143 (0'0) modify 53f2bbaa/benchmark_data_storage0111-ib.pocstor_48865_object8316/head//2 by client.2191635.0:8317 2014-08-12 17:15:31.312440 2014-08-13 18:06:06.415598 7f5c973a57a0 20 read_log 194215'144 (0'0) modify e60a5caa/benchmark_data_storage0111-ib.pocstor_48865_object8605/head//2 by client.2191635.0:8606 2014-08-12 17:15:32.690445 2014-08-13 18:06:06.415625 7f5c973a57a0 20 read_log 194215'145 (0'0) modify e9f3b4aa/benchmark_data_storage0111-ib.pocstor_48865_object8698/head//2 by client.2191635.0:8699 2014-08-12 17:15:33.027523 2014-08-13 18:06:06.415677 7f5c973a57a0 20 read_log 194215'146 (0'0) modify 66d1cb6a/benchmark_data_storage0111-ib.pocstor_48865_object8716/head//2 by client.2191635.0:8717 2014-08-12 17:15:33.091962 2014-08-13 18:06:06.415721 7f5c973a57a0 20 read_log 194215'147 (0'0) modify fbb9d36a/benchmark_data_storage0111-ib.pocstor_48865_object8783/head//2 by client.2191635.0:8784 2014-08-12 17:15:33.336870 2014-08-13 18:06:06.415744 7f5c973a57a0 20 read_log 194215'148 (0'0) modify c746816a/benchmark_data_storage0111-ib.pocstor_48865_object8787/head//2 by client.2191635.0:8788 2014-08-12 17:15:33.351242 2014-08-13 18:06:06.415765 7f5c973a57a0 20 read_log 194215'149 (0'0) modify 3a4a1e6a/benchmark_data_storage0111-ib.pocstor_48865_object8947/head//2 by client.2191635.0:8948 2014-08-12 17:15:33.930406 2014-08-13 18:06:06.415786 7f5c973a57a0 20 read_log 194215'150 (0'0) modify bc379f6a/benchmark_data_storage0111-ib.pocstor_48865_object9016/head//2 by client.2191635.0:9017 2014-08-12 17:15:34.181016 2014-08-13 18:06:06.415820 7f5c973a57a0 20 read_log 194215'151 (0'0) modify cb6d2d6a/benchmark_data_storage0111-ib.pocstor_48865_object9125/head//2 by client.2191635.0:9126 2014-08-12 17:15:34.576941 2014-08-13 18:06:06.415843 7f5c973a57a0 20 read_log 194215'152 (0'0) modify 1ef336ea/benchmark_data_storage0111-ib.pocstor_48865_object9155/head//2 by client.2191635.0:9156 2014-08-12 17:15:34.687613 2014-08-13 18:06:06.415865 7f5c973a57a0 20 read_log 194215'153 (0'0) modify 7bc5afea/benchmark_data_storage0111-ib.pocstor_48865_object9205/head//2 by client.2191635.0:9206 2014-08-12 17:15:34.868034 2014-08-13 18:06:06.415887 7f5c973a57a0 20 read_log 194215'154 (0'0) modify 4d0f22ea/benchmark_data_storage0111-ib.pocstor_48865_object9210/head//2 by client.2191635.0:9211 2014-08-12 17:15:34.885872 2014-08-13 18:06:06.415920 7f5c973a57a0 20 read_log 194215'155 (0'0) modify 33b536ea/benchmark_data_storage0111-ib.pocstor_48865_object9217/head//2 by client.2191635.0:9218 2014-08-12 17:15:34.912468 2014-08-13 18:06:06.415942 7f5c973a57a0 20 read_log 194215'156 (0'0) modify c09e996a/benchmark_data_storage0111-ib.pocstor_48865_object9321/head//2 by client.2191635.0:9322 2014-08-12 17:15:35.289806 2014-08-13 18:06:06.415963 7f5c973a57a0 20 read_log 194215'157 (0'0) modify 884b2aa/benchmark_data_storage0111-ib.pocstor_48865_object9349/head//2 by client.2191635.0:9350 2014-08-12 17:15:35.428529 2014-08-13 18:06:06.415984 7f5c973a57a0 20 read_log 194215'158 (0'0) modify 7bea2ea/benchmark_data_storage0111-ib.pocstor_48865_object9369/head//2 by client.2191635.0:9370 2014-08-12 17:15:35.528811 2014-08-13 18:06:06.416005 7f5c973a57a0 20 read_log 194215'159 (0'0) modify d092c3ea/benchmark_data_storage0111-ib.pocstor_48865_object9464/head//2 by client.2191635.0:9465 2014-08-12 17:15:35.887006 2014-08-13 18:06:06.416026 7f5c973a57a0 20 read_log 194215'160 (0'0) modify f2d72e2a/benchmark_data_storage0111-ib.pocstor_48865_object9497/head//2 by client.2191635.0:9498 2014-08-12 17:15:36.006585 2014-08-13 18:06:06.416048 7f5c973a57a0 20 read_log 194215'161 (0'0) modify 7ed5766a/benchmark_data_storage0111-ib.pocstor_48865_object9512/head//2 by client.2191635.0:9513 2014-08-12 17:15:36.059959 2014-08-13 18:06:06.416069 7f5c973a57a0 20 read_log 194215'162 (0'0) modify 3108d22a/benchmark_data_storage0111-ib.pocstor_48865_object9561/head//2 by client.2191635.0:9562 2014-08-12 17:15:36.237341 2014-08-13 18:06:06.416090 7f5c973a57a0 20 read_log 194215'163 (0'0) modify 55a7182a/benchmark_data_storage0111-ib.pocstor_48865_object9618/head//2 by client.2191635.0:9619 2014-08-12 17:15:36.443842 2014-08-13 18:06:06.416111 7f5c973a57a0 20 read_log 194215'164 (0'0) modify dfc811ea/benchmark_data_storage0111-ib.pocstor_48865_object9687/head//2 by client.2191635.0:9688 2014-08-12 17:15:36.785253 2014-08-13 18:06:06.416132 7f5c973a57a0 20 read_log 194215'165 (0'0) modify 9b7a8baa/benchmark_data_storage0111-ib.pocstor_48865_object9869/head//2 by client.2191635.0:9870 2014-08-12 17:15:37.706117 2014-08-13 18:06:06.416162 7f5c973a57a0 20 read_log 194215'166 (0'0) modify 2b1b79ea/benchmark_data_storage0111-ib.pocstor_48865_object9880/head//2 by client.2191635.0:9881 2014-08-12 17:15:37.759971 2014-08-13 18:06:06.416190 7f5c973a57a0 20 read_log 194215'167 (0'0) modify 21a23aea/benchmark_data_storage0111-ib.pocstor_48865_object9933/head//2 by client.2191635.0:9934 2014-08-12 17:15:38.050817 2014-08-13 18:06:06.416212 7f5c973a57a0 20 read_log 194215'168 (0'0) modify b5c1d3aa/benchmark_data_storage0111-ib.pocstor_48865_object10008/head//2 by client.2191635.0:10009 2014-08-12 17:15:38.403669 2014-08-13 18:06:06.416233 7f5c973a57a0 20 read_log 194215'169 (0'0) modify 87325caa/benchmark_data_storage0111-ib.pocstor_48865_object10072/head//2 by client.2191635.0:10073 2014-08-12 17:15:38.767291 2014-08-13 18:06:06.416254 7f5c973a57a0 20 read_log 194215'170 (0'0) modify a5bef9aa/benchmark_data_storage0111-ib.pocstor_48865_object10123/head//2 by client.2191635.0:10124 2014-08-12 17:15:38.950737 2014-08-13 18:06:06.416276 7f5c973a57a0 20 read_log 194215'171 (0'0) modify 839ef8aa/benchmark_data_storage0111-ib.pocstor_48865_object10189/head//2 by client.2191635.0:10190 2014-08-12 17:15:39.206925 2014-08-13 18:06:06.416297 7f5c973a57a0 20 read_log 194215'172 (0'0) modify a72c32aa/benchmark_data_storage0111-ib.pocstor_48865_object10248/head//2 by client.2191635.0:10249 2014-08-12 17:15:39.434879 2014-08-13 18:06:06.416323 7f5c973a57a0 20 read_log 194215'173 (0'0) modify 3759752a/benchmark_data_storage0111-ib.pocstor_48865_object10253/head//2 by client.2191635.0:10254 2014-08-12 17:15:39.453378 2014-08-13 18:06:06.416345 7f5c973a57a0 20 read_log 194215'174 (0'0) modify 83b5eaa/benchmark_data_storage0111-ib.pocstor_48865_object10313/head//2 by client.2191635.0:10314 2014-08-12 17:15:39.671946 2014-08-13 18:06:06.416386 7f5c973a57a0 20 read_log 194215'175 (0'0) modify 6a9613aa/benchmark_data_storage0111-ib.pocstor_48865_object10396/head//2 by client.2191635.0:10397 2014-08-12 17:15:40.014965 2014-08-13 18:06:06.416415 7f5c973a57a0 20 read_log 194215'176 (0'0) modify b37aadaa/benchmark_data_storage0111-ib.pocstor_48865_object10453/head//2 by client.2191635.0:10454 2014-08-12 17:15:40.227327 2014-08-13 18:06:06.416462 7f5c973a57a0 20 read_log 194215'177 (0'0) modify 292338aa/benchmark_data_storage0111-ib.pocstor_48865_object10482/head//2 by client.2191635.0:10483 2014-08-12 17:15:40.333410 2014-08-13 18:06:06.416493 7f5c973a57a0 20 read_log 194215'178 (0'0) modify cb33af2a/benchmark_data_storage0111-ib.pocstor_48865_object10644/head//2 by client.2191635.0:10645 2014-08-12 17:15:40.970800 2014-08-13 18:06:06.416516 7f5c973a57a0 20 read_log 194215'179 (0'0) modify 1d50792a/benchmark_data_storage0111-ib.pocstor_48865_object10690/head//2 by client.2191635.0:10691 2014-08-12 17:15:41.224375 2014-08-13 18:06:06.416537 7f5c973a57a0 20 read_log 194215'180 (0'0) modify f9ca0fea/benchmark_data_storage0111-ib.pocstor_48865_object10730/head//2 by client.2191635.0:10731 2014-08-12 17:15:41.397617 2014-08-13 18:06:06.416558 7f5c973a57a0 20 read_log 194215'181 (0'0) modify a596172a/benchmark_data_storage0111-ib.pocstor_48865_object10814/head//2 by client.2191635.0:10815 2014-08-12 17:15:41.855059 2014-08-13 18:06:06.416580 7f5c973a57a0 20 read_log 194215'182 (0'0) modify 2e3f30aa/benchmark_data_storage0111-ib.pocstor_48865_object10886/head//2 by client.2191635.0:10887 2014-08-12 17:15:42.256517 2014-08-13 18:06:06.416601 7f5c973a57a0 20 read_log 194215'183 (0'0) modify 3359f32a/benchmark_data_storage0111-ib.pocstor_48865_object10910/head//2 by client.2191635.0:10911 2014-08-12 17:15:42.388701 2014-08-13 18:06:06.416622 7f5c973a57a0 20 read_log 194215'184 (0'0) modify d5ffa2a/benchmark_data_storage0111-ib.pocstor_48865_object10933/head//2 by client.2191635.0:10934 2014-08-12 17:15:42.494485 2014-08-13 18:06:06.416661 7f5c973a57a0 20 read_log 194215'185 (0'0) modify 5db5596a/benchmark_data_storage0111-ib.pocstor_48865_object10936/head//2 by client.2191635.0:10937 2014-08-12 17:15:42.505098 2014-08-13 18:06:06.416688 7f5c973a57a0 20 read_log 194215'186 (0'0) modify aa36ecaa/benchmark_data_storage0111-ib.pocstor_48865_object10979/head//2 by client.2191635.0:10980 2014-08-12 17:15:42.658928 2014-08-13 18:06:06.416712 7f5c973a57a0 20 read_log 194215'187 (0'0) modify 7039232a/benchmark_data_storage0111-ib.pocstor_48865_object11094/head//2 by client.2191635.0:11095 2014-08-12 17:15:43.109776 2014-08-13 18:06:06.416734 7f5c973a57a0 20 read_log 194215'188 (0'0) modify 5b3c0d2a/benchmark_data_storage0111-ib.pocstor_48865_object11111/head//2 by client.2191635.0:11112 2014-08-12 17:15:43.190326 2014-08-13 18:06:06.416755 7f5c973a57a0 20 read_log 194215'189 (0'0) modify 3f026e2a/benchmark_data_storage0111-ib.pocstor_48865_object11117/head//2 by client.2191635.0:11118 2014-08-12 17:15:43.227487 2014-08-13 18:06:06.416776 7f5c973a57a0 20 read_log 194215'190 (0'0) modify 6072102a/benchmark_data_storage0111-ib.pocstor_48865_object11176/head//2 by client.2191635.0:11177 2014-08-12 17:15:43.593006 2014-08-13 18:06:06.416800 7f5c973a57a0 20 read_log 194215'191 (0'0) modify b6ed77aa/benchmark_data_storage0111-ib.pocstor_48865_object11187/head//2 by client.2191635.0:11188 2014-08-12 17:15:43.632697 2014-08-13 18:06:06.416821 7f5c973a57a0 20 read_log 194215'192 (0'0) modify 205c17aa/benchmark_data_storage0111-ib.pocstor_48865_object11236/head//2 by client.2191635.0:11237 2014-08-12 17:15:43.817950 2014-08-13 18:06:06.416843 7f5c973a57a0 20 read_log 194215'193 (0'0) modify 8d426cea/benchmark_data_storage0111-ib.pocstor_48865_object11349/head//2 by client.2191635.0:11350 2014-08-12 17:15:44.242918 2014-08-13 18:06:06.416864 7f5c973a57a0 20 read_log 194215'194 (0'0) modify d297416a/benchmark_data_storage0111-ib.pocstor_48865_object11397/head//2 by client.2191635.0:11398 2014-08-12 17:15:44.413801 2014-08-13 18:06:06.416896 7f5c973a57a0 20 read_log 194215'195 (0'0) modify d6cefa2a/benchmark_data_storage0111-ib.pocstor_48865_object11498/head//2 by client.2191635.0:11499 2014-08-12 17:15:44.781522 2014-08-13 18:06:06.416918 7f5c973a57a0 20 read_log 194215'196 (0'0) modify 53e842ea/benchmark_data_storage0111-ib.pocstor_48865_object11552/head//2 by client.2191635.0:11553 2014-08-12 17:15:44.985019 2014-08-13 18:06:06.416942 7f5c973a57a0 20 read_log 194215'197 (0'0) modify 6030336a/benchmark_data_storage0111-ib.pocstor_48865_object11556/head//2 by client.2191635.0:11557 2014-08-12 17:15:45.024211 2014-08-13 18:06:06.416963 7f5c973a57a0 20 read_log 194215'198 (0'0) modify 89d46bea/benchmark_data_storage0111-ib.pocstor_48865_object11776/head//2 by client.2191635.0:11777 2014-08-12 17:15:46.004313 2014-08-13 18:06:06.416985 7f5c973a57a0 20 read_log 194215'199 (0'0) modify 35ffcaa/benchmark_data_storage0111-ib.pocstor_48865_object11861/head//2 by client.2191635.0:11862 2014-08-12 17:15:46.336605 2014-08-13 18:06:06.417006 7f5c973a57a0 20 read_log 194215'200 (0'0) modify 5b650a2a/benchmark_data_storage0111-ib.pocstor_48865_object11920/head//2 by client.2191635.0:11921 2014-08-12 17:15:46.549607 2014-08-13 18:06:06.417027 7f5c973a57a0 20 read_log 194215'201 (0'0) modify 4a1b016a/benchmark_data_storage0111-ib.pocstor_48865_object12014/head//2 by client.2191635.0:12015 2014-08-12 17:15:46.886851 2014-08-13 18:06:06.417048 7f5c973a57a0 20 read_log 194215'202 (0'0) modify 6c2c76aa/benchmark_data_storage0111-ib.pocstor_48865_object12017/head//2 by client.2191635.0:12018 2014-08-12 17:15:46.897357 2014-08-13 18:06:06.417069 7f5c973a57a0 20 read_log 194215'203 (0'0) modify b1b052a/benchmark_data_storage0111-ib.pocstor_48865_object12039/head//2 by client.2191635.0:12040 2014-08-12 17:15:46.975486 2014-08-13 18:06:06.417090 7f5c973a57a0 20 read_log 194215'204 (0'0) modify 18d92c2a/benchmark_data_storage0111-ib.pocstor_48865_object12137/head//2 by client.2191635.0:12138 2014-08-12 17:15:47.326641 2014-08-13 18:06:06.417111 7f5c973a57a0 20 read_log 194215'205 (0'0) modify e293eb6a/benchmark_data_storage0111-ib.pocstor_48865_object12200/head//2 by client.2191635.0:12201 2014-08-12 17:15:47.552061 2014-08-13 18:06:06.417132 7f5c973a57a0 20 read_log 194215'206 (0'0) modify aec82caa/benchmark_data_storage0111-ib.pocstor_48865_object12205/head//2 by client.2191635.0:12206 2014-08-12 17:15:47.569872 2014-08-13 18:06:06.417162 7f5c973a57a0 20 read_log 194215'207 (0'0) modify e4c381ea/benchmark_data_storage0111-ib.pocstor_48865_object12239/head//2 by client.2191635.0:12240 2014-08-12 17:15:47.691485 2014-08-13 18:06:06.417185 7f5c973a57a0 20 read_log 194215'208 (0'0) modify 5597feaa/benchmark_data_storage0111-ib.pocstor_48865_object12386/head//2 by client.2191635.0:12387 2014-08-12 17:15:48.391108 2014-08-13 18:06:06.417206 7f5c973a57a0 20 read_log 194215'209 (0'0) modify 43f2212a/benchmark_data_storage0111-ib.pocstor_48865_object12393/head//2 by client.2191635.0:12394 2014-08-12 17:15:48.416114 2014-08-13 18:06:06.417227 7f5c973a57a0 20 read_log 194215'210 (0'0) modify 2b5bb7ea/benchmark_data_storage0111-ib.pocstor_48865_object12418/head//2 by client.2191635.0:12419 2014-08-12 17:15:48.546188 2014-08-13 18:06:06.417248 7f5c973a57a0 20 read_log 194215'211 (0'0) modify b49f39aa/benchmark_data_storage0111-ib.pocstor_48865_object12547/head//2 by client.2191635.0:12548 2014-08-12 17:15:49.977299 2014-08-13 18:06:06.417269 7f5c973a57a0 20 read_log 194215'212 (0'0) modify affb68aa/benchmark_data_storage0111-ib.pocstor_48865_object12599/head//2 by client.2191635.0:12600 2014-08-12 17:15:50.219212 2014-08-13 18:06:06.417290 7f5c973a57a0 20 read_log 194215'213 (0'0) modify 2e68e72a/benchmark_data_storage0111-ib.pocstor_48865_object12623/head//2 by client.2191635.0:12624 2014-08-12 17:15:50.321994 2014-08-13 18:06:06.417318 7f5c973a57a0 20 read_log 194215'214 (0'0) modify b033de6a/benchmark_data_storage0111-ib.pocstor_48865_object12628/head//2 by client.2191635.0:12629 2014-08-12 17:15:50.342151 2014-08-13 18:06:06.417356 7f5c973a57a0 20 read_log 194215'215 (0'0) modify ae98b7ea/benchmark_data_storage0111-ib.pocstor_48865_object12659/head//2 by client.2191635.0:12660 2014-08-12 17:15:50.476108 2014-08-13 18:06:06.417384 7f5c973a57a0 20 read_log 194215'216 (0'0) modify 2aaf702a/benchmark_data_storage0111-ib.pocstor_48865_object12668/head//2 by client.2191635.0:12669 2014-08-12 17:15:50.515635 2014-08-13 18:06:06.417412 7f5c973a57a0 20 read_log 194215'217 (0'0) modify f57663aa/benchmark_data_storage0111-ib.pocstor_48865_object12750/head//2 by client.2191635.0:12751 2014-08-12 17:15:50.854116 2014-08-13 18:06:06.417454 7f5c973a57a0 20 read_log 194215'218 (0'0) modify f4212caa/benchmark_data_storage0111-ib.pocstor_48865_object12838/head//2 by client.2191635.0:12839 2014-08-12 17:15:51.171945 2014-08-13 18:06:06.417477 7f5c973a57a0 20 read_log 194215'219 (0'0) modify 7240e7aa/benchmark_data_storage0111-ib.pocstor_48865_object12999/head//2 by client.2191635.0:13000 2014-08-12 17:15:51.757431 2014-08-13 18:06:06.417516 7f5c973a57a0 20 read_log 194215'220 (0'0) modify 667d846a/benchmark_data_storage0111-ib.pocstor_48865_object13090/head//2 by client.2191635.0:13091 2014-08-12 17:15:52.102955 2014-08-13 18:06:06.417544 7f5c973a57a0 20 read_log 194215'221 (0'0) modify 4728a26a/benchmark_data_storage0111-ib.pocstor_48865_object13104/head//2 by client.2191635.0:13105 2014-08-12 17:15:52.177461 2014-08-13 18:06:06.417566 7f5c973a57a0 20 read_log 194215'222 (0'0) modify 6830202a/benchmark_data_storage0111-ib.pocstor_48865_object13121/head//2 by client.2191635.0:13122 2014-08-12 17:15:52.255868 2014-08-13 18:06:06.417587 7f5c973a57a0 20 read_log 194215'223 (0'0) modify e698f6ea/benchmark_data_storage0111-ib.pocstor_48865_object13189/head//2 by client.2191635.0:13190 2014-08-12 17:15:52.612221 2014-08-13 18:06:06.417609 7f5c973a57a0 20 read_log 194215'224 (0'0) modify 1d7d93ea/benchmark_data_storage0111-ib.pocstor_48865_object13262/head//2 by client.2191635.0:13263 2014-08-12 17:15:52.877192 2014-08-13 18:06:06.417642 7f5c973a57a0 20 read_log 194215'225 (0'0) modify 4a9e062a/benchmark_data_storage0111-ib.pocstor_48865_object13318/head//2 by client.2191635.0:13319 2014-08-12 17:15:53.079486 2014-08-13 18:06:06.417666 7f5c973a57a0 20 read_log 194215'226 (0'0) modify 71b2d72a/benchmark_data_storage0111-ib.pocstor_48865_object13371/head//2 by client.2191635.0:13372 2014-08-12 17:15:53.272076 2014-08-13 18:06:06.417688 7f5c973a57a0 20 read_log 194215'227 (0'0) modify 4abb7a2a/benchmark_data_storage0111-ib.pocstor_48865_object13374/head//2 by client.2191635.0:13375 2014-08-12 17:15:53.282562 2014-08-13 18:06:06.417722 7f5c973a57a0 20 read_log 194215'228 (0'0) modify 8e43f9aa/benchmark_data_storage0111-ib.pocstor_48865_object13392/head//2 by client.2191635.0:13393 2014-08-12 17:15:53.349656 2014-08-13 18:06:06.417747 7f5c973a57a0 20 read_log 194215'229 (0'0) modify 4f37bbea/benchmark_data_storage0111-ib.pocstor_48865_object13561/head//2 by client.2191635.0:13562 2014-08-12 17:15:53.962136 2014-08-13 18:06:06.417773 7f5c973a57a0 20 read_log 194215'230 (0'0) modify 19487bea/benchmark_data_storage0111-ib.pocstor_48865_object13635/head//2 by client.2191635.0:13636 2014-08-12 17:15:54.231884 2014-08-13 18:06:06.417794 7f5c973a57a0 20 read_log 194215'231 (0'0) modify dd220cea/benchmark_data_storage0111-ib.pocstor_48865_object13725/head//2 by client.2191635.0:13726 2014-08-12 17:15:54.554549 2014-08-13 18:06:06.417816 7f5c973a57a0 20 read_log 194215'232 (0'0) modify 9440cd2a/benchmark_data_storage0111-ib.pocstor_48865_object13735/head//2 by client.2191635.0:13736 2014-08-12 17:15:54.590517 2014-08-13 18:06:06.417840 7f5c973a57a0 20 read_log 194215'233 (0'0) modify d720b2a/benchmark_data_storage0111-ib.pocstor_48865_object13783/head//2 by client.2191635.0:13784 2014-08-12 17:15:54.763533 2014-08-13 18:06:06.417865 7f5c973a57a0 20 read_log 194215'234 (0'0) modify c9345c2a/benchmark_data_storage0111-ib.pocstor_48865_object13852/head//2 by client.2191635.0:13853 2014-08-12 17:15:55.053480 2014-08-13 18:06:06.417902 7f5c973a57a0 20 read_log 194215'235 (0'0) modify ea0b76a/benchmark_data_storage0111-ib.pocstor_48865_object13973/head//2 by client.2191635.0:13974 2014-08-12 17:15:55.567926 2014-08-13 18:06:06.417933 7f5c973a57a0 20 read_log 194215'236 (0'0) modify 2a0c802a/benchmark_data_storage0111-ib.pocstor_48865_object14145/head//2 by client.2191635.0:14146 2014-08-12 17:15:56.291566 2014-08-13 18:06:06.417956 7f5c973a57a0 20 read_log 194215'237 (0'0) modify 63208a6a/benchmark_data_storage0111-ib.pocstor_48865_object14286/head//2 by client.2191635.0:14287 2014-08-12 17:15:56.800077 2014-08-13 18:06:06.417977 7f5c973a57a0 20 read_log 194215'238 (0'0) modify 40f6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object14317/head//2 by client.2191635.0:14318 2014-08-12 17:15:56.912607 2014-08-13 18:06:06.418039 7f5c973a57a0 20 read_log 194215'239 (0'0) modify 41de21ea/benchmark_data_storage0111-ib.pocstor_48865_object14598/head//2 by client.2191635.0:14599 2014-08-12 17:15:58.065932 2014-08-13 18:06:06.418086 7f5c973a57a0 20 read_log 194215'240 (0'0) modify b8a8542a/benchmark_data_storage0111-ib.pocstor_48865_object14672/head//2 by client.2191635.0:14673 2014-08-12 17:15:58.333255 2014-08-13 18:06:06.418112 7f5c973a57a0 20 read_log 194215'241 (0'0) modify 4a6306aa/benchmark_data_storage0111-ib.pocstor_48865_object14713/head//2 by client.2191635.0:14714 2014-08-12 17:15:58.483677 2014-08-13 18:06:06.418138 7f5c973a57a0 20 read_log 194215'242 (0'0) modify 2991aaaa/benchmark_data_storage0111-ib.pocstor_48865_object14741/head//2 by client.2191635.0:14742 2014-08-12 17:15:58.584487 2014-08-13 18:06:06.418186 7f5c973a57a0 20 read_log 194215'243 (0'0) modify 565f662a/benchmark_data_storage0111-ib.pocstor_48865_object14767/head//2 by client.2191635.0:14768 2014-08-12 17:15:58.679345 2014-08-13 18:06:06.418209 7f5c973a57a0 20 read_log 194215'244 (0'0) modify 5192f8aa/benchmark_data_storage0111-ib.pocstor_48865_object14776/head//2 by client.2191635.0:14777 2014-08-12 17:15:58.712169 2014-08-13 18:06:06.418231 7f5c973a57a0 20 read_log 194215'245 (0'0) modify eeea566a/benchmark_data_storage0111-ib.pocstor_48865_object14778/head//2 by client.2191635.0:14779 2014-08-12 17:15:58.719188 2014-08-13 18:06:06.418264 7f5c973a57a0 20 read_log 194215'246 (0'0) modify 8558d5aa/benchmark_data_storage0111-ib.pocstor_48865_object14801/head//2 by client.2191635.0:14802 2014-08-12 17:15:58.818408 2014-08-13 18:06:06.418292 7f5c973a57a0 20 read_log 194215'247 (0'0) modify 9d0eaaea/benchmark_data_storage0111-ib.pocstor_48865_object14866/head//2 by client.2191635.0:14867 2014-08-12 17:15:59.065600 2014-08-13 18:06:06.418314 7f5c973a57a0 20 read_log 194215'248 (0'0) modify 305b2a6a/benchmark_data_storage0111-ib.pocstor_48865_object14950/head//2 by client.2191635.0:14951 2014-08-12 17:15:59.370240 2014-08-13 18:06:06.418341 7f5c973a57a0 20 read_log 194215'249 (0'0) modify 731b0e6a/benchmark_data_storage0111-ib.pocstor_48865_object15136/head//2 by client.2191635.0:15137 2014-08-12 17:16:00.043103 2014-08-13 18:06:06.418363 7f5c973a57a0 20 read_log 194215'250 (0'0) modify 65f3ae6a/benchmark_data_storage0111-ib.pocstor_48865_object15161/head//2 by client.2191635.0:15162 2014-08-12 17:16:00.132222 2014-08-13 18:06:06.418393 7f5c973a57a0 20 read_log 194215'251 (0'0) modify 77250b2a/benchmark_data_storage0111-ib.pocstor_48865_object15203/head//2 by client.2191635.0:15204 2014-08-12 17:16:00.285023 2014-08-13 18:06:06.418415 7f5c973a57a0 20 read_log 194215'252 (0'0) modify daba3e6a/benchmark_data_storage0111-ib.pocstor_48865_object15306/head//2 by client.2191635.0:15307 2014-08-12 17:16:00.665089 2014-08-13 18:06:06.418459 7f5c973a57a0 20 read_log 194215'253 (0'0) modify 8338bcea/benchmark_data_storage0111-ib.pocstor_48865_object15415/head//2 by client.2191635.0:15416 2014-08-12 17:16:01.090982 2014-08-13 18:06:06.418482 7f5c973a57a0 20 read_log 194215'254 (0'0) modify 79bf0b6a/benchmark_data_storage0111-ib.pocstor_48865_object15441/head//2 by client.2191635.0:15442 2014-08-12 17:16:01.217189 2014-08-13 18:06:06.418515 7f5c973a57a0 20 read_log 194215'255 (0'0) modify 22e21cea/benchmark_data_storage0111-ib.pocstor_48865_object15451/head//2 by client.2191635.0:15452 2014-08-12 17:16:01.262365 2014-08-13 18:06:06.418540 7f5c973a57a0 20 read_log 194215'256 (0'0) modify 722c47ea/benchmark_data_storage0111-ib.pocstor_48865_object15566/head//2 by client.2191635.0:15567 2014-08-12 17:16:01.962561 2014-08-13 18:06:06.418562 7f5c973a57a0 20 read_log 194215'257 (0'0) modify a865f92a/benchmark_data_storage0111-ib.pocstor_48865_object15671/head//2 by client.2191635.0:15672 2014-08-12 17:16:02.535524 2014-08-13 18:06:06.418583 7f5c973a57a0 20 read_log 194215'258 (0'0) modify eb9d4f6a/benchmark_data_storage0111-ib.pocstor_48865_object15686/head//2 by client.2191635.0:15687 2014-08-12 17:16:02.614819 2014-08-13 18:06:06.418631 7f5c973a57a0 20 read_log 194215'259 (0'0) modify bfafc2a/benchmark_data_storage0111-ib.pocstor_48865_object15765/head//2 by client.2191635.0:15766 2014-08-12 17:16:02.989209 2014-08-13 18:06:06.418663 7f5c973a57a0 20 read_log 194215'260 (0'0) modify c4918aea/benchmark_data_storage0111-ib.pocstor_48865_object15774/head//2 by client.2191635.0:15775 2014-08-12 17:16:03.026942 2014-08-13 18:06:06.418705 7f5c973a57a0 20 read_log 194215'261 (0'0) modify 577cf22a/benchmark_data_storage0111-ib.pocstor_48865_object15886/head//2 by client.2191635.0:15887 2014-08-12 17:16:03.434523 2014-08-13 18:06:06.418728 7f5c973a57a0 20 read_log 194215'262 (0'0) modify 47b95a6a/benchmark_data_storage0111-ib.pocstor_48865_object15917/head//2 by client.2191635.0:15918 2014-08-12 17:16:03.549837 2014-08-13 18:06:06.418749 7f5c973a57a0 20 read_log 194215'263 (0'0) modify d8a6b6ea/benchmark_data_storage0111-ib.pocstor_48865_object15948/head//2 by client.2191635.0:15949 2014-08-12 17:16:03.661576 2014-08-13 18:06:06.418771 7f5c973a57a0 20 read_log 194215'264 (0'0) modify b36d1d6a/benchmark_data_storage0111-ib.pocstor_48865_object15982/head//2 by client.2191635.0:15983 2014-08-12 17:16:03.787983 2014-08-13 18:06:06.418804 7f5c973a57a0 20 read_log 194215'265 (0'0) modify 45f476ea/benchmark_data_storage0111-ib.pocstor_48865_object16063/head//2 by client.2191635.0:16064 2014-08-12 17:16:04.080351 2014-08-13 18:06:06.418831 7f5c973a57a0 20 read_log 194215'266 (0'0) modify c3d0156a/benchmark_data_storage0111-ib.pocstor_48865_object16141/head//2 by client.2191635.0:16142 2014-08-12 17:16:04.366973 2014-08-13 18:06:06.418853 7f5c973a57a0 20 read_log 194215'267 (0'0) modify 81a3d32a/benchmark_data_storage0111-ib.pocstor_48865_object16150/head//2 by client.2191635.0:16151 2014-08-12 17:16:04.398979 2014-08-13 18:06:06.418875 7f5c973a57a0 20 read_log 194215'268 (0'0) modify 8c0d182a/benchmark_data_storage0111-ib.pocstor_48865_object16296/head//2 by client.2191635.0:16297 2014-08-12 17:16:04.926972 2014-08-13 18:06:06.418898 7f5c973a57a0 20 read_log 194215'269 (0'0) modify a521d6aa/benchmark_data_storage0111-ib.pocstor_48865_object16315/head//2 by client.2191635.0:16316 2014-08-12 17:16:04.997455 2014-08-13 18:06:06.418920 7f5c973a57a0 20 read_log 194215'270 (0'0) modify df2ea2ea/benchmark_data_storage0111-ib.pocstor_48865_object16352/head//2 by client.2191635.0:16353 2014-08-12 17:16:05.130873 2014-08-13 18:06:06.418941 7f5c973a57a0 20 read_log 194215'271 (0'0) modify 826376aa/benchmark_data_storage0111-ib.pocstor_48865_object16361/head//2 by client.2191635.0:16362 2014-08-12 17:16:05.162821 2014-08-13 18:06:06.418962 7f5c973a57a0 20 read_log 194215'272 (0'0) modify 6afdb52a/benchmark_data_storage0111-ib.pocstor_48865_object16427/head//2 by client.2191635.0:16428 2014-08-12 17:16:05.404967 2014-08-13 18:06:06.418983 7f5c973a57a0 20 read_log 194215'273 (0'0) modify bc7adeaa/benchmark_data_storage0111-ib.pocstor_48865_object16515/head//2 by client.2191635.0:16516 2014-08-12 17:16:05.724474 2014-08-13 18:06:06.419005 7f5c973a57a0 20 read_log 194215'274 (0'0) modify ccb6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object16564/head//2 by client.2191635.0:16565 2014-08-12 17:16:05.905227 2014-08-13 18:06:06.419071 7f5c973a57a0 20 read_log 194215'275 (0'0) modify 7f37d7aa/benchmark_data_storage0111-ib.pocstor_48865_object16565/head//2 by client.2191635.0:16566 2014-08-12 17:16:05.908721 2014-08-13 18:06:06.419099 7f5c973a57a0 20 read_log 194215'276 (0'0) modify 8cf4382a/benchmark_data_storage0111-ib.pocstor_48865_object16604/head//2 by client.2191635.0:16605 2014-08-12 17:16:06.046947 2014-08-13 18:06:06.419121 7f5c973a57a0 20 read_log 194215'277 (0'0) modify fbdbd5aa/benchmark_data_storage0111-ib.pocstor_48865_object16791/head//2 by client.2191635.0:16792 2014-08-12 17:16:06.727006 2014-08-13 18:06:06.419142 7f5c973a57a0 20 read_log 194215'278 (0'0) modify 46242aea/benchmark_data_storage0111-ib.pocstor_48865_object16862/head//2 by client.2191635.0:16863 2014-08-12 17:16:06.980908 2014-08-13 18:06:06.419213 7f5c973a57a0 20 read_log 194215'279 (0'0) modify 8e0b2d2a/benchmark_data_storage0111-ib.pocstor_48865_object16863/head//2 by client.2191635.0:16864 2014-08-12 17:16:06.984476 2014-08-13 18:06:06.419235 7f5c973a57a0 20 read_log 194215'280 (0'0) modify d8478eea/benchmark_data_storage0111-ib.pocstor_48865_object16870/head//2 by client.2191635.0:16871 2014-08-12 17:16:07.009231 2014-08-13 18:06:06.419261 7f5c973a57a0 20 read_log 194215'281 (0'0) modify 23a472a/benchmark_data_storage0111-ib.pocstor_48865_object16907/head//2 by client.2191635.0:16908 2014-08-12 17:16:07.145148 2014-08-13 18:06:06.419283 7f5c973a57a0 20 read_log 194215'282 (0'0) modify 861c596a/benchmark_data_storage0111-ib.pocstor_48865_object16946/head//2 by client.2191635.0:16947 2014-08-12 17:16:07.342214 2014-08-13 18:06:06.419304 7f5c973a57a0 20 read_log 194215'283 (0'0) modify 70e3cf2a/benchmark_data_storage0111-ib.pocstor_48865_object16999/head//2 by client.2191635.0:17000 2014-08-12 17:16:07.624824 2014-08-13 18:06:06.419326 7f5c973a57a0 20 read_log 194215'284 (0'0) modify 66eb7d2a/benchmark_data_storage0111-ib.pocstor_48865_object17022/head//2 by client.2191635.0:17023 2014-08-12 17:16:07.711065 2014-08-13 18:06:06.419352 7f5c973a57a0 20 read_log 194215'285 (0'0) modify a97b1aa/benchmark_data_storage0111-ib.pocstor_48865_object17041/head//2 by client.2191635.0:17042 2014-08-12 17:16:07.777491 2014-08-13 18:06:06.419378 7f5c973a57a0 20 read_log 194215'286 (0'0) modify 36b9c7ea/benchmark_data_storage0111-ib.pocstor_48865_object17067/head//2 by client.2191635.0:17068 2014-08-12 17:16:07.871923 2014-08-13 18:06:06.419404 7f5c973a57a0 20 read_log 194215'287 (0'0) modify d2246d2a/benchmark_data_storage0111-ib.pocstor_48865_object17159/head//2 by client.2191635.0:17160 2014-08-12 17:16:08.218770 2014-08-13 18:06:06.419428 7f5c973a57a0 20 read_log 194215'288 (0'0) modify 9e6f64ea/benchmark_data_storage0111-ib.pocstor_48865_object17166/head//2 by client.2191635.0:17167 2014-08-12 17:16:08.267875 2014-08-13 18:06:06.419458 7f5c973a57a0 20 read_log 194215'289 (0'0) modify 1ce7ab2a/benchmark_data_storage0111-ib.pocstor_48865_object17300/head//2 by client.2191635.0:17301 2014-08-12 17:16:09.093686 2014-08-13 18:06:06.419499 7f5c973a57a0 20 read_log 194215'290 (0'0) modify 93cde52a/benchmark_data_storage0111-ib.pocstor_48865_object17356/head//2 by client.2191635.0:17357 2014-08-12 17:16:09.435192 2014-08-13 18:06:06.419538 7f5c973a57a0 20 read_log 194215'291 (0'0) modify 6543b36a/benchmark_data_storage0111-ib.pocstor_48865_object17395/head//2 by client.2191635.0:17396 2014-08-12 17:16:09.657999 2014-08-13 18:06:06.419567 7f5c973a57a0 20 read_log 194215'292 (0'0) modify 907fe66a/benchmark_data_storage0111-ib.pocstor_48865_object17420/head//2 by client.2191635.0:17421 2014-08-12 17:16:09.764588 2014-08-13 18:06:06.419589 7f5c973a57a0 20 read_log 194215'293 (0'0) modify f04168aa/benchmark_data_storage0111-ib.pocstor_48865_object17493/head//2 by client.2191635.0:17494 2014-08-12 17:16:10.029089 2014-08-13 18:06:06.419611 7f5c973a57a0 20 read_log 194215'294 (0'0) modify 3dd28a2a/benchmark_data_storage0111-ib.pocstor_48865_object17533/head//2 by client.2191635.0:17534 2014-08-12 17:16:10.175906 2014-08-13 18:06:06.419648 7f5c973a57a0 20 read_log 194215'295 (0'0) modify 1e4657ea/benchmark_data_storage0111-ib.pocstor_48865_object17612/head//2 by client.2191635.0:17613 2014-08-12 17:16:10.608645 2014-08-13 18:06:06.419682 7f5c973a57a0 20 read_log 194215'296 (0'0) modify 6041b6ea/benchmark_data_storage0111-ib.pocstor_48865_object17635/head//2 by client.2191635.0:17636 2014-08-12 17:16:10.770191 2014-08-13 18:06:06.419724 7f5c973a57a0 20 read_log 194215'297 (0'0) modify 224e726a/benchmark_data_storage0111-ib.pocstor_48865_object17676/head//2 by client.2191635.0:17677 2014-08-12 17:16:11.018959 2014-08-13 18:06:06.419760 7f5c973a57a0 20 read_log 194215'298 (0'0) modify b888422a/benchmark_data_storage0111-ib.pocstor_48865_object17741/head//2 by client.2191635.0:17742 2014-08-12 17:16:11.332637 2014-08-13 18:06:06.419782 7f5c973a57a0 20 read_log 194215'299 (0'0) modify 6161082a/benchmark_data_storage0111-ib.pocstor_48865_object17851/head//2 by client.2191635.0:17852 2014-08-12 17:16:11.727213 2014-08-13 18:06:06.419813 7f5c973a57a0 20 read_log 194215'300 (0'0) modify f91dcc2a/benchmark_data_storage0111-ib.pocstor_48865_object18013/head//2 by client.2191635.0:18014 2014-08-12 17:16:12.405398 2014-08-13 18:06:06.419839 7f5c973a57a0 20 read_log 194215'301 (0'0) modify d714946a/benchmark_data_storage0111-ib.pocstor_48865_object18071/head//2 by client.2191635.0:18072 2014-08-12 17:16:12.711413 2014-08-13 18:06:06.419861 7f5c973a57a0 20 read_log 194215'302 (0'0) modify 76da69aa/benchmark_data_storage0111-ib.pocstor_48865_object18175/head//2 by client.2191635.0:18176 2014-08-12 17:16:13.088383 2014-08-13 18:06:06.419905 7f5c973a57a0 20 read_log 194215'303 (0'0) modify ca0bf7ea/benchmark_data_storage0111-ib.pocstor_48865_object18353/head//2 by client.2191635.0:18354 2014-08-12 17:16:13.731627 2014-08-13 18:06:06.419928 7f5c973a57a0 20 read_log 194215'304 (0'0) modify cb5fc6ea/benchmark_data_storage0111-ib.pocstor_48865_object18362/head//2 by client.2191635.0:18363 2014-08-12 17:16:13.763791 2014-08-13 18:06:06.419958 7f5c973a57a0 20 read_log 194215'305 (0'0) modify ba58eaaa/benchmark_data_storage0111-ib.pocstor_48865_object18438/head//2 by client.2191635.0:18439 2014-08-12 17:16:14.061156 2014-08-13 18:06:06.419980 7f5c973a57a0 20 read_log 194215'306 (0'0) modify 774fb76a/benchmark_data_storage0111-ib.pocstor_48865_object18602/head//2 by client.2191635.0:18603 2014-08-12 17:16:14.916162 2014-08-13 18:06:06.420015 7f5c973a57a0 20 read_log 194215'307 (0'0) modify 3f75e6a/benchmark_data_storage0111-ib.pocstor_48865_object18627/head//2 by client.2191635.0:18628 2014-08-12 17:16:15.017009 2014-08-13 18:06:06.420043 7f5c973a57a0 20 read_log 194215'308 (0'0) modify 76eaf2a/benchmark_data_storage0111-ib.pocstor_48865_object18644/head//2 by client.2191635.0:18645 2014-08-12 17:16:15.076769 2014-08-13 18:06:06.420067 7f5c973a57a0 20 read_log 194215'309 (0'0) modify 5bc40b6a/benchmark_data_storage0111-ib.pocstor_48865_object18652/head//2 by client.2191635.0:18653 2014-08-12 17:16:15.109627 2014-08-13 18:06:06.420088 7f5c973a57a0 20 read_log 194215'310 (0'0) modify 9c195bea/benchmark_data_storage0111-ib.pocstor_48865_object18750/head//2 by client.2191635.0:18751 2014-08-12 17:16:15.542111 2014-08-13 18:06:06.420110 7f5c973a57a0 20 read_log 194215'311 (0'0) modify bcd1ddea/benchmark_data_storage0111-ib.pocstor_48865_object18803/head//2 by client.2191635.0:18804 2014-08-12 17:16:15.953270 2014-08-13 18:06:06.420131 7f5c973a57a0 20 read_log 194215'312 (0'0) modify 57e7232a/benchmark_data_storage0111-ib.pocstor_48865_object18851/head//2 by client.2191635.0:18852 2014-08-12 17:16:17.205211 2014-08-13 18:06:06.420180 7f5c973a57a0 20 read_log 194215'313 (0'0) modify 3f8d12a/benchmark_data_storage0111-ib.pocstor_48865_object18898/head//2 by client.2191635.0:18899 2014-08-12 17:16:17.423178 2014-08-13 18:06:06.420210 7f5c973a57a0 20 read_log 194215'314 (0'0) modify eda4702a/benchmark_data_storage0111-ib.pocstor_48865_object18934/head//2 by client.2191635.0:18935 2014-08-12 17:16:17.557912 2014-08-13 18:06:06.420254 7f5c973a57a0 20 read_log 194215'315 (0'0) modify 8ad8c9ea/benchmark_data_storage0111-ib.pocstor_48865_object19074/head//2 by client.2191635.0:19075 2014-08-12 17:16:18.063779 2014-08-13 18:06:06.420281 7f5c973a57a0 20 read_log 194215'316 (0'0) modify 45feb6aa/benchmark_data_storage0111-ib.pocstor_48865_object19194/head//2 by client.2191635.0:19195 2014-08-12 17:16:18.851198 2014-08-13 18:06:06.420303 7f5c973a57a0 20 read_log 194215'317 (0'0) modify 67fe712a/benchmark_data_storage0111-ib.pocstor_48865_object19218/head//2 by client.2191635.0:19219 2014-08-12 17:16:18.937502 2014-08-13 18:06:06.420324 7f5c973a57a0 20 read_log 194215'318 (0'0) modify 87a9a2ea/benchmark_data_storage0111-ib.pocstor_48865_object19230/head//2 by client.2191635.0:19231 2014-08-12 17:16:18.979622 2014-08-13 18:06:06.420354 7f5c973a57a0 20 read_log 194215'319 (0'0) modify d4cfa56a/benchmark_data_storage0111-ib.pocstor_48865_object19259/head//2 by client.2191635.0:19260 2014-08-12 17:16:19.084466 2014-08-13 18:06:06.420381 7f5c973a57a0 20 read_log 194215'320 (0'0) modify 3d60586a/benchmark_data_storage0111-ib.pocstor_48865_object19315/head//2 by client.2191635.0:19316 2014-08-12 17:16:19.286260 2014-08-13 18:06:06.420403 7f5c973a57a0 20 read_log 194215'321 (0'0) modify a5cf01aa/benchmark_data_storage0111-ib.pocstor_48865_object19413/head//2 by client.2191635.0:19414 2014-08-12 17:16:19.736977 2014-08-13 18:06:06.420425 7f5c973a57a0 20 read_log 194215'322 (0'0) modify eac666ea/benchmark_data_storage0111-ib.pocstor_48865_object19418/head//2 by client.2191635.0:19419 2014-08-12 17:16:19.769140 2014-08-13 18:06:06.420466 7f5c973a57a0 20 read_log 194215'323 (0'0) modify 701512a/benchmark_data_storage0111-ib.pocstor_48865_object19425/head//2 by client.2191635.0:19426 2014-08-12 17:16:19.814572 2014-08-13 18:06:06.420489 7f5c973a57a0 20 read_log 194215'324 (0'0) modify 109f076a/benchmark_data_storage0111-ib.pocstor_48865_object19461/head//2 by client.2191635.0:19462 2014-08-12 17:16:20.028673 2014-08-13 18:06:06.420511 7f5c973a57a0 20 read_log 194215'325 (0'0) modify b65a122a/benchmark_data_storage0111-ib.pocstor_48865_object19517/head//2 by client.2191635.0:19518 2014-08-12 17:16:20.285607 2014-08-13 18:06:06.420532 7f5c973a57a0 20 read_log 194215'326 (0'0) modify bca3f02a/benchmark_data_storage0111-ib.pocstor_48865_object19526/head//2 by client.2191635.0:19527 2014-08-12 17:16:20.317334 2014-08-13 18:06:06.420553 7f5c973a57a0 20 read_log 194215'327 (0'0) modify cf2e0e2a/benchmark_data_storage0111-ib.pocstor_48865_object19566/head//2 by client.2191635.0:19567 2014-08-12 17:16:20.463822 2014-08-13 18:06:06.420574 7f5c973a57a0 20 read_log 194215'328 (0'0) modify 56fd8d6a/benchmark_data_storage0111-ib.pocstor_48865_object19582/head//2 by client.2191635.0:19583 2014-08-12 17:16:20.521305 2014-08-13 18:06:06.420599 7f5c973a57a0 20 read_log 194215'329 (0'0) modify bea29c2a/benchmark_data_storage0111-ib.pocstor_48865_object19690/head//2 by client.2191635.0:19691 2014-08-12 17:16:20.911949 2014-08-13 18:06:06.420620 7f5c973a57a0 20 read_log 194215'330 (0'0) modify 971f7a6a/benchmark_data_storage0111-ib.pocstor_48865_object19692/head//2 by client.2191635.0:19693 2014-08-12 17:16:20.918927 2014-08-13 18:06:06.420641 7f5c973a57a0 20 read_log 194215'331 (0'0) modify 9a63936a/benchmark_data_storage0111-ib.pocstor_48865_object19899/head//2 by client.2191635.0:19900 2014-08-12 17:16:21.762855 2014-08-13 18:06:06.420695 7f5c973a57a0 20 read_log 194215'332 (0'0) modify f7da2aea/benchmark_data_storage0111-ib.pocstor_48865_object19982/head//2 by client.2191635.0:19983 2014-08-12 17:16:22.225361 2014-08-13 18:06:06.420722 7f5c973a57a0 20 read_log 194215'333 (0'0) modify 5bb981aa/benchmark_data_storage0111-ib.pocstor_48865_object20036/head//2 by client.2191635.0:20037 2014-08-12 17:16:22.426859 2014-08-13 18:06:06.420744 7f5c973a57a0 20 read_log 194215'334 (0'0) modify cf11302a/benchmark_data_storage0111-ib.pocstor_48865_object20045/head//2 by client.2191635.0:20046 2014-08-12 17:16:22.458846 2014-08-13 18:06:06.420777 7f5c973a57a0 20 read_log 194215'335 (0'0) modify fe4a30aa/benchmark_data_storage0111-ib.pocstor_48865_object20057/head//2 by client.2191635.0:20058 2014-08-12 17:16:22.502342 2014-08-13 18:06:06.420799 7f5c973a57a0 20 read_log 194215'336 (0'0) modify 44997bea/benchmark_data_storage0111-ib.pocstor_48865_object20125/head//2 by client.2191635.0:20126 2014-08-12 17:16:22.789954 2014-08-13 18:06:06.420821 7f5c973a57a0 20 read_log 194215'337 (0'0) modify c8c401aa/benchmark_data_storage0111-ib.pocstor_48865_object20134/head//2 by client.2191635.0:20135 2014-08-12 17:16:22.845537 2014-08-13 18:06:06.420850 7f5c973a57a0 20 read_log 194215'338 (0'0) modify ae295caa/benchmark_data_storage0111-ib.pocstor_48865_object20201/head//2 by client.2191635.0:20202 2014-08-12 17:16:23.652760 2014-08-13 18:06:06.420872 7f5c973a57a0 20 read_log 194215'339 (0'0) modify f203046a/benchmark_data_storage0111-ib.pocstor_48865_object20204/head//2 by client.2191635.0:20205 2014-08-12 17:16:23.687471 2014-08-13 18:06:06.420893 7f5c973a57a0 20 read_log 194215'340 (0'0) modify de6e16aa/benchmark_data_storage0111-ib.pocstor_48865_object20326/head//2 by client.2191635.0:20327 2014-08-12 17:16:24.292247 2014-08-13 18:06:06.420914 7f5c973a57a0 20 read_log 194215'341 (0'0) modify e793e8aa/benchmark_data_storage0111-ib.pocstor_48865_object20420/head//2 by client.2191635.0:20421 2014-08-12 17:16:24.642523 2014-08-13 18:06:06.420938 7f5c973a57a0 20 read_log 194215'342 (0'0) modify 97c150ea/benchmark_data_storage0111-ib.pocstor_48865_object20422/head//2 by client.2191635.0:20423 2014-08-12 17:16:24.661076 2014-08-13 18:06:06.420959 7f5c973a57a0 20 read_log 194215'343 (0'0) modify c581916a/benchmark_data_storage0111-ib.pocstor_48865_object20680/head//2 by client.2191635.0:20681 2014-08-12 17:16:26.226605 2014-08-13 18:06:06.420983 7f5c973a57a0 20 read_log 194215'344 (0'0) modify 226308aa/benchmark_data_storage0111-ib.pocstor_48865_object20698/head//2 by client.2191635.0:20699 2014-08-12 17:16:26.290535 2014-08-13 18:06:06.421004 7f5c973a57a0 20 read_log 194215'345 (0'0) modify 217b9c2a/benchmark_data_storage0111-ib.pocstor_48865_object20701/head//2 by client.2191635.0:20702 2014-08-12 17:16:26.301012 2014-08-13 18:06:06.421030 7f5c973a57a0 20 read_log 194215'346 (0'0) modify d22aaeea/benchmark_data_storage0111-ib.pocstor_48865_object20724/head//2 by client.2191635.0:20725 2014-08-12 17:16:26.385467 2014-08-13 18:06:06.421051 7f5c973a57a0 20 read_log 194215'347 (0'0) modify 3056b9aa/benchmark_data_storage0111-ib.pocstor_48865_object20781/head//2 by client.2191635.0:20782 2014-08-12 17:16:26.588930 2014-08-13 18:06:06.421077 7f5c973a57a0 20 read_log 194215'348 (0'0) modify e128ee2a/benchmark_data_storage0111-ib.pocstor_48865_object20832/head//2 by client.2191635.0:20833 2014-08-12 17:16:26.780106 2014-08-13 18:06:06.421101 7f5c973a57a0 20 read_log 194215'349 (0'0) modify a037016a/benchmark_data_storage0111-ib.pocstor_48865_object21051/head//2 by client.2191635.0:21052 2014-08-12 17:16:27.760213 2014-08-13 18:06:06.421125 7f5c973a57a0 20 read_log 194215'350 (0'0) modify b10367ea/benchmark_data_storage0111-ib.pocstor_48865_object21068/head//2 by client.2191635.0:21069 2014-08-12 17:16:27.826846 2014-08-13 18:06:06.421155 7f5c973a57a0 20 read_log 194215'351 (0'0) modify d46985aa/benchmark_data_storage0111-ib.pocstor_48865_object21162/head//2 by client.2191635.0:21163 2014-08-12 17:16:28.737133 2014-08-13 18:06:06.421182 7f5c973a57a0 20 read_log 194215'352 (0'0) modify 971d17ea/benchmark_data_storage0111-ib.pocstor_48865_object21246/head//2 by client.2191635.0:21247 2014-08-12 17:16:29.113516 2014-08-13 18:06:06.421204 7f5c973a57a0 20 read_log 194215'353 (0'0) modify 41d7a72a/benchmark_data_storage0111-ib.pocstor_48865_object21255/head//2 by client.2191635.0:21256 2014-08-12 17:16:29.144956 2014-08-13 18:06:06.421225 7f5c973a57a0 20 read_log 194215'354 (0'0) modify 6adc306a/benchmark_data_storage0111-ib.pocstor_48865_object21329/head//2 by client.2191635.0:21330 2014-08-12 17:16:29.411652 2014-08-13 18:06:06.421261 7f5c973a57a0 20 read_log 194215'355 (0'0) modify 19802aaa/benchmark_data_storage0111-ib.pocstor_48865_object21368/head//2 by client.2191635.0:21369 2014-08-12 17:16:29.576418 2014-08-13 18:06:06.421287 7f5c973a57a0 20 read_log 194215'356 (0'0) modify 87842baa/benchmark_data_storage0111-ib.pocstor_48865_object21403/head//2 by client.2191635.0:21404 2014-08-12 17:16:29.801240 2014-08-13 18:06:06.421309 7f5c973a57a0 20 read_log 194215'357 (0'0) modify b59226a/benchmark_data_storage0111-ib.pocstor_48865_object21404/head//2 by client.2191635.0:21405 2014-08-12 17:16:29.812016 2014-08-13 18:06:06.421330 7f5c973a57a0 20 read_log 194215'358 (0'0) modify d68ecdaa/benchmark_data_storage0111-ib.pocstor_48865_object21520/head//2 by client.2191635.0:21521 2014-08-12 17:16:30.562160 2014-08-13 18:06:06.421352 7f5c973a57a0 20 read_log 194215'359 (0'0) modify 37caa06a/benchmark_data_storage0111-ib.pocstor_48865_object21568/head//2 by client.2191635.0:21569 2014-08-12 17:16:30.735149 2014-08-13 18:06:06.421383 7f5c973a57a0 20 read_log 194215'360 (0'0) modify 615151ea/benchmark_data_storage0111-ib.pocstor_48865_object21657/head//2 by client.2191635.0:21658 2014-08-12 17:16:31.062712 2014-08-13 18:06:06.421409 7f5c973a57a0 20 read_log 194215'361 (0'0) modify 7abdc1ea/benchmark_data_storage0111-ib.pocstor_48865_object21668/head//2 by client.2191635.0:21669 2014-08-12 17:16:31.102711 2014-08-13 18:06:06.421435 7f5c973a57a0 20 read_log 194215'362 (0'0) modify 97743f2a/benchmark_data_storage0111-ib.pocstor_48865_object21779/head//2 by client.2191635.0:21780 2014-08-12 17:16:31.541261 2014-08-13 18:06:06.421457 7f5c973a57a0 20 read_log 194215'363 (0'0) modify 605fde2a/benchmark_data_storage0111-ib.pocstor_48865_object21799/head//2 by client.2191635.0:21800 2014-08-12 17:16:31.622469 2014-08-13 18:06:06.421478 7f5c973a57a0 20 read_log 194215'364 (0'0) modify 7e36cb6a/benchmark_data_storage0111-ib.pocstor_48865_object21980/head//2 by client.2191635.0:21981 2014-08-12 17:16:32.463014 2014-08-13 18:06:06.421500 7f5c973a57a0 20 read_log 194215'365 (0'0) modify 31ca3faa/benchmark_data_storage0111-ib.pocstor_48865_object22026/head//2 by client.2191635.0:22027 2014-08-12 17:16:32.627850 2014-08-13 18:06:06.421521 7f5c973a57a0 20 read_log 194215'366 (0'0) modify b22ed1aa/benchmark_data_storage0111-ib.pocstor_48865_object22036/head//2 by client.2191635.0:22037 2014-08-12 17:16:32.663051 2014-08-13 18:06:06.421542 7f5c973a57a0 20 read_log 194215'367 (0'0) modify 9d60f46a/benchmark_data_storage0111-ib.pocstor_48865_object22243/head//2 by client.2191635.0:22244 2014-08-12 17:16:33.639658 2014-08-13 18:06:06.421584 7f5c973a57a0 20 read_log 194215'368 (0'0) modify d38544aa/benchmark_data_storage0111-ib.pocstor_48865_object22284/head//2 by client.2191635.0:22285 2014-08-12 17:16:33.815647 2014-08-13 18:06:06.421619 7f5c973a57a0 20 read_log 194215'369 (0'0) modify 4248b2ea/benchmark_data_storage0111-ib.pocstor_48865_object22314/head//2 by client.2191635.0:22315 2014-08-12 17:16:33.929669 2014-08-13 18:06:06.421646 7f5c973a57a0 20 read_log 194215'370 (0'0) modify 9d1470aa/benchmark_data_storage0111-ib.pocstor_48865_object22348/head//2 by client.2191635.0:22349 2014-08-12 17:16:34.050102 2014-08-13 18:06:06.421682 7f5c973a57a0 20 read_log 194215'371 (0'0) modify 8501b66a/benchmark_data_storage0111-ib.pocstor_48865_object22363/head//2 by client.2191635.0:22364 2014-08-12 17:16:34.107552 2014-08-13 18:06:06.421726 7f5c973a57a0 20 read_log 194215'372 (0'0) modify 4baa0dea/benchmark_data_storage0111-ib.pocstor_48865_object22414/head//2 by client.2191635.0:22415 2014-08-12 17:16:34.373672 2014-08-13 18:06:06.421754 7f5c973a57a0 20 read_log 194215'373 (0'0) modify c76fe86a/benchmark_data_storage0111-ib.pocstor_48865_object22497/head//2 by client.2191635.0:22498 2014-08-12 17:16:34.702994 2014-08-13 18:06:06.421777 7f5c973a57a0 20 read_log 194215'374 (0'0) modify 1378006a/benchmark_data_storage0111-ib.pocstor_48865_object22538/head//2 by client.2191635.0:22539 2014-08-12 17:16:34.853009 2014-08-13 18:06:06.421809 7f5c973a57a0 20 read_log 194215'375 (0'0) modify 51bf9b2a/benchmark_data_storage0111-ib.pocstor_48865_object22569/head//2 by client.2191635.0:22570 2014-08-12 17:16:34.962874 2014-08-13 18:06:06.421831 7f5c973a57a0 20 read_log 194215'376 (0'0) modify d91a75ea/benchmark_data_storage0111-ib.pocstor_48865_object22581/head//2 by client.2191635.0:22582 2014-08-12 17:16:35.005339 2014-08-13 18:06:06.421852 7f5c973a57a0 20 read_log 194215'377 (0'0) modify 877f52a/benchmark_data_storage0111-ib.pocstor_48865_object22590/head//2 by client.2191635.0:22591 2014-08-12 17:16:35.039981 2014-08-13 18:06:06.421877 7f5c973a57a0 20 read_log 194215'378 (0'0) modify 7772482a/benchmark_data_storage0111-ib.pocstor_48865_object22594/head//2 by client.2191635.0:22595 2014-08-12 17:16:35.055278 2014-08-13 18:06:06.421922 7f5c973a57a0 20 read_log 194215'379 (0'0) modify 4f9362a/benchmark_data_storage0111-ib.pocstor_48865_object22640/head//2 by client.2191635.0:22641 2014-08-12 17:16:35.250146 2014-08-13 18:06:06.421950 7f5c973a57a0 20 read_log 194215'380 (0'0) modify 5af1bdaa/benchmark_data_storage0111-ib.pocstor_48865_object22673/head//2 by client.2191635.0:22674 2014-08-12 17:16:35.399686 2014-08-13 18:06:06.421972 7f5c973a57a0 20 read_log 194215'381 (0'0) modify 8f9836aa/benchmark_data_storage0111-ib.pocstor_48865_object22690/head//2 by client.2191635.0:22691 2014-08-12 17:16:35.514386 2014-08-13 18:06:06.421998 7f5c973a57a0 20 read_log 194215'382 (0'0) modify 1e47b9ea/benchmark_data_storage0111-ib.pocstor_48865_object22731/head//2 by client.2191635.0:22732 2014-08-12 17:16:35.736001 2014-08-13 18:06:06.422025 7f5c973a57a0 20 read_log 194215'383 (0'0) modify dfa6716a/benchmark_data_storage0111-ib.pocstor_48865_object22765/head//2 by client.2191635.0:22766 2014-08-12 17:16:35.858332 2014-08-13 18:06:06.422047 7f5c973a57a0 20 read_log 194215'384 (0'0) modify 10a9a06a/benchmark_data_storage0111-ib.pocstor_48865_object22886/head//2 by client.2191635.0:22887 2014-08-12 17:16:36.298808 2014-08-13 18:06:06.422069 7f5c973a57a0 20 read_log 194215'385 (0'0) modify 8c0ab1aa/benchmark_data_storage0111-ib.pocstor_48865_object22899/head//2 by client.2191635.0:22900 2014-08-12 17:16:36.345587 2014-08-13 18:06:06.422090 7f5c973a57a0 20 read_log 194215'386 (0'0) modify 8bceefea/benchmark_data_storage0111-ib.pocstor_48865_object22913/head//2 by client.2191635.0:22914 2014-08-12 17:16:36.395120 2014-08-13 18:06:06.422112 7f5c973a57a0 20 read_log 194215'387 (0'0) modify 2fbf146a/benchmark_data_storage0111-ib.pocstor_48865_object22932/head//2 by client.2191635.0:22933 2014-08-12 17:16:36.462020 2014-08-13 18:06:06.422156 7f5c973a57a0 20 read_log 194215'388 (0'0) modify 7df62c2a/benchmark_data_storage0111-ib.pocstor_48865_object23119/head//2 by client.2191635.0:23120 2014-08-12 17:16:37.141437 2014-08-13 18:06:06.422186 7f5c973a57a0 20 read_log 194215'389 (0'0) modify b614c76a/benchmark_data_storage0111-ib.pocstor_48865_object23197/head//2 by client.2191635.0:23198 2014-08-12 17:16:37.475105 2014-08-13 18:06:06.422245 7f5c973a57a0 20 read_log 194215'390 (0'0) modify 62b055ea/benchmark_data_storage0111-ib.pocstor_48865_object23212/head//2 by client.2191635.0:23213 2014-08-12 17:16:37.542049 2014-08-13 18:06:06.422267 7f5c973a57a0 20 read_log 194215'391 (0'0) modify 972960ea/benchmark_data_storage0111-ib.pocstor_48865_object23269/head//2 by client.2191635.0:23270 2014-08-12 17:16:37.814742 2014-08-13 18:06:06.422291 7f5c973a57a0 20 read_log 194215'392 (0'0) modify 64c4eb6a/benchmark_data_storage0111-ib.pocstor_48865_object23325/head//2 by client.2191635.0:23326 2014-08-12 17:16:38.093389 2014-08-13 18:06:06.422313 7f5c973a57a0 20 read_log 194215'393 (0'0) modify c10f2b2a/benchmark_data_storage0111-ib.pocstor_48865_object23339/head//2 by client.2191635.0:23340 2014-08-12 17:16:38.194023 2014-08-13 18:06:06.422344 7f5c973a57a0 20 read_log 194215'394 (0'0) modify f00c10aa/benchmark_data_storage0111-ib.pocstor_48865_object23473/head//2 by client.2191635.0:23474 2014-08-12 17:16:39.514178 2014-08-13 18:06:06.422383 7f5c973a57a0 20 read_log 194215'395 (0'0) modify e2f975aa/benchmark_data_storage0111-ib.pocstor_48865_object23699/head//2 by client.2191635.0:23700 2014-08-12 17:16:40.566220 2014-08-13 18:06:06.422410 7f5c973a57a0 20 read_log 194215'396 (0'0) modify 68a661ea/benchmark_data_storage0111-ib.pocstor_48865_object23758/head//2 by client.2191635.0:23759 2014-08-12 17:16:40.871178 2014-08-13 18:06:06.422436 7f5c973a57a0 20 read_log 194215'397 (0'0) modify baa697aa/benchmark_data_storage0111-ib.pocstor_48865_object23818/head//2 by client.2191635.0:23819 2014-08-12 17:16:41.102698 2014-08-13 18:06:06.422471 7f5c973a57a0 20 read_log 194215'398 (0'0) modify 6b48b46a/benchmark_data_storage0111-ib.pocstor_48865_object23835/head//2 by client.2191635.0:23836 2014-08-12 17:16:41.162200 2014-08-13 18:06:06.422499 7f5c973a57a0 20 read_log 194215'399 (0'0) modify d8ebb3aa/benchmark_data_storage0111-ib.pocstor_48865_object23888/head//2 by client.2191635.0:23889 2014-08-12 17:16:41.380060 2014-08-13 18:06:06.422521 7f5c973a57a0 20 read_log 194215'400 (0'0) modify 34bd3aaa/benchmark_data_storage0111-ib.pocstor_48865_object23938/head//2 by client.2191635.0:23939 2014-08-12 17:16:41.659877 2014-08-13 18:06:06.422542 7f5c973a57a0 20 read_log 194215'401 (0'0) modify 7504a3aa/benchmark_data_storage0111-ib.pocstor_48865_object23964/head//2 by client.2191635.0:23965 2014-08-12 17:16:41.843009 2014-08-13 18:06:06.422587 7f5c973a57a0 20 read_log 194215'402 (0'0) modify e16bca6a/benchmark_data_storage0111-ib.pocstor_48865_object24022/head//2 by client.2191635.0:24023 2014-08-12 17:16:42.218380 2014-08-13 18:06:06.422615 7f5c973a57a0 20 read_log 194215'403 (0'0) modify 4b4b8cea/benchmark_data_storage0111-ib.pocstor_48865_object24030/head//2 by client.2191635.0:24031 2014-08-12 17:16:42.257717 2014-08-13 18:06:06.422641 7f5c973a57a0 20 read_log 194215'404 (0'0) modify e441ef6a/benchmark_data_storage0111-ib.pocstor_48865_object24080/head//2 by client.2191635.0:24081 2014-08-12 17:16:42.460966 2014-08-13 18:06:06.422671 7f5c973a57a0 20 read_log 194215'405 (0'0) modify 5fcc2aaa/benchmark_data_storage0111-ib.pocstor_48865_object24241/head//2 by client.2191635.0:24242 2014-08-12 17:16:43.044977 2014-08-13 18:06:06.422697 7f5c973a57a0 20 read_log 194215'406 (0'0) modify 3c8eaeaa/benchmark_data_storage0111-ib.pocstor_48865_object24315/head//2 by client.2191635.0:24316 2014-08-12 17:16:43.315794 2014-08-13 18:06:06.422737 7f5c973a57a0 20 read_log 194215'407 (0'0) modify fb00662a/benchmark_data_storage0111-ib.pocstor_48865_object24327/head//2 by client.2191635.0:24328 2014-08-12 17:16:43.358796 2014-08-13 18:06:06.422769 7f5c973a57a0 20 read_log 194215'408 (0'0) modify b50f7ea/benchmark_data_storage0111-ib.pocstor_48865_object24343/head//2 by client.2191635.0:24344 2014-08-12 17:16:43.415391 2014-08-13 18:06:06.422807 7f5c973a57a0 20 read_log 194215'409 (0'0) modify f5caafaa/benchmark_data_storage0111-ib.pocstor_48865_object24349/head//2 by client.2191635.0:24350 2014-08-12 17:16:43.436839 2014-08-13 18:06:06.422834 7f5c973a57a0 20 read_log 194215'410 (0'0) modify d65f39aa/benchmark_data_storage0111-ib.pocstor_48865_object24366/head//2 by client.2191635.0:24367 2014-08-12 17:16:43.497237 2014-08-13 18:06:06.422861 7f5c973a57a0 20 read_log 194215'411 (0'0) modify d785ae2a/benchmark_data_storage0111-ib.pocstor_48865_object24461/head//2 by client.2191635.0:24462 2014-08-12 17:16:43.926709 2014-08-13 18:06:06.422887 7f5c973a57a0 20 read_log 194215'412 (0'0) modify 94e6396a/benchmark_data_storage0111-ib.pocstor_48865_object24499/head//2 by client.2191635.0:24500 2014-08-12 17:16:44.064613 2014-08-13 18:06:06.422917 7f5c973a57a0 20 read_log 194215'413 (0'0) modify ec9780ea/benchmark_data_storage0111-ib.pocstor_48865_object24551/head//2 by client.2191635.0:24552 2014-08-12 17:16:44.252457 2014-08-13 18:06:06.422955 7f5c973a57a0 20 read_log 194215'414 (0'0) modify 76527aaa/benchmark_data_storage0111-ib.pocstor_48865_object24564/head//2 by client.2191635.0:24565 2014-08-12 17:16:44.298816 2014-08-13 18:06:06.423033 7f5c973a57a0 20 read_log 194215'415 (0'0) modify 1fe906ea/benchmark_data_storage0111-ib.pocstor_48865_object24630/head//2 by client.2191635.0:24631 2014-08-12 17:16:44.537451 2014-08-13 18:06:06.423064 7f5c973a57a0 20 read_log 194215'416 (0'0) modify d088b7aa/benchmark_data_storage0111-ib.pocstor_48865_object24638/head//2 by client.2191635.0:24639 2014-08-12 17:16:44.566378 2014-08-13 18:06:06.423087 7f5c973a57a0 20 read_log 194215'417 (0'0) modify c957872a/benchmark_data_storage0111-ib.pocstor_48865_object24706/head//2 by client.2191635.0:24707 2014-08-12 17:16:44.809434 2014-08-13 18:06:06.423108 7f5c973a57a0 20 read_log 194215'418 (0'0) modify 7e977daa/benchmark_data_storage0111-ib.pocstor_48865_object24941/head//2 by client.2191635.0:24942 2014-08-12 17:16:45.770427 2014-08-13 18:06:06.423130 7f5c973a57a0 20 read_log 194215'419 (0'0) modify c441e36a/benchmark_data_storage0111-ib.pocstor_48865_object25044/head//2 by client.2191635.0:25045 2014-08-12 17:16:46.214769 2014-08-13 18:06:06.423160 7f5c973a57a0 20 read_log 194215'420 (0'0) modify 3b05822a/benchmark_data_storage0111-ib.pocstor_48865_object25074/head//2 by client.2191635.0:25075 2014-08-12 17:16:46.364290 2014-08-13 18:06:06.423186 7f5c973a57a0 20 read_log 194215'421 (0'0) modify dac3a2ea/benchmark_data_storage0111-ib.pocstor_48865_object25144/head//2 by client.2191635.0:25145 2014-08-12 17:16:46.618312 2014-08-13 18:06:06.423208 7f5c973a57a0 20 read_log 194215'422 (0'0) modify ed943f2a/benchmark_data_storage0111-ib.pocstor_48865_object25159/head//2 by client.2191635.0:25160 2014-08-12 17:16:46.671518 2014-08-13 18:06:06.423229 7f5c973a57a0 20 read_log 194215'423 (0'0) modify 68276bea/benchmark_data_storage0111-ib.pocstor_48865_object25166/head//2 by client.2191635.0:25167 2014-08-12 17:16:46.696500 2014-08-13 18:06:06.423250 7f5c973a57a0 20 read_log 194215'424 (0'0) modify bd3e026a/benchmark_data_storage0111-ib.pocstor_48865_object25310/head//2 by client.2191635.0:25311 2014-08-12 17:16:47.211674 2014-08-13 18:06:06.423301 7f5c973a57a0 20 read_log 194215'425 (0'0) modify 48d2462a/benchmark_data_storage0111-ib.pocstor_48865_object25331/head//2 by client.2191635.0:25332 2014-08-12 17:16:47.288629 2014-08-13 18:06:06.423328 7f5c973a57a0 20 read_log 194215'426 (0'0) modify 9c95526a/benchmark_data_storage0111-ib.pocstor_48865_object25369/head//2 by client.2191635.0:25370 2014-08-12 17:16:47.423563 2014-08-13 18:06:06.423355 7f5c973a57a0 20 read_log 194215'427 (0'0) modify 9e2944aa/benchmark_data_storage0111-ib.pocstor_48865_object25391/head//2 by client.2191635.0:25392 2014-08-12 17:16:47.504594 2014-08-13 18:06:06.423376 7f5c973a57a0 20 read_log 194215'428 (0'0) modify aff1fc6a/benchmark_data_storage0111-ib.pocstor_48865_object25463/head//2 by client.2191635.0:25464 2014-08-12 17:16:47.766553 2014-08-13 18:06:06.423407 7f5c973a57a0 20 read_log 194215'429 (0'0) modify 9f6824aa/benchmark_data_storage0111-ib.pocstor_48865_object25483/head//2 by client.2191635.0:25484 2014-08-12 17:16:47.839544 2014-08-13 18:06:06.423433 7f5c973a57a0 20 read_log 194215'430 (0'0) modify 9cc1c82a/benchmark_data_storage0111-ib.pocstor_48865_object25512/head//2 by client.2191635.0:25513 2014-08-12 17:16:47.944868 2014-08-13 18:06:06.423468 7f5c973a57a0 20 read_log 194215'431 (0'0) modify f2fcef2a/benchmark_data_storage0111-ib.pocstor_48865_object25660/head//2 by client.2191635.0:25661 2014-08-12 17:16:49.637166 2014-08-13 18:06:06.423501 7f5c973a57a0 20 read_log 194215'432 (0'0) modify deeda82a/benchmark_data_storage0111-ib.pocstor_48865_object25706/head//2 by client.2191635.0:25707 2014-08-12 17:16:49.805121 2014-08-13 18:06:06.423535 7f5c973a57a0 20 read_log 194215'433 (0'0) modify b504f56a/benchmark_data_storage0111-ib.pocstor_48865_object25726/head//2 by client.2191635.0:25727 2014-08-12 17:16:49.875936 2014-08-13 18:06:06.423562 7f5c973a57a0 20 read_log 194215'434 (0'0) modify 652e19ea/benchmark_data_storage0111-ib.pocstor_48865_object25735/head//2 by client.2191635.0:25736 2014-08-12 17:16:49.907590 2014-08-13 18:06:06.423605 7f5c973a57a0 20 read_log 194215'435 (0'0) modify ff82672a/benchmark_data_storage0111-ib.pocstor_48865_object25879/head//2 by client.2191635.0:25880 2014-08-12 17:16:50.483030 2014-08-13 18:06:06.423668 7f5c973a57a0 20 read_log 194215'436 (0'0) modify 1deec7ea/benchmark_data_storage0111-ib.pocstor_48865_object25948/head//2 by client.2191635.0:25949 2014-08-12 17:16:50.750259 2014-08-13 18:06:06.423715 7f5c973a57a0 20 read_log 194215'437 (0'0) modify 6898cd6a/benchmark_data_storage0111-ib.pocstor_48865_object25978/head//2 by client.2191635.0:25979 2014-08-12 17:16:50.855998 2014-08-13 18:06:06.423738 7f5c973a57a0 20 read_log 194215'438 (0'0) modify 9c55f66a/benchmark_data_storage0111-ib.pocstor_48865_object26060/head//2 by client.2191635.0:26061 2014-08-12 17:16:51.261313 2014-08-13 18:06:06.423759 7f5c973a57a0 20 read_log 194215'439 (0'0) modify 99abde2a/benchmark_data_storage0111-ib.pocstor_48865_object26093/head//2 by client.2191635.0:26094 2014-08-12 17:16:51.409383 2014-08-13 18:06:06.423797 7f5c973a57a0 20 read_log 194215'440 (0'0) modify 1696b9ea/benchmark_data_storage0111-ib.pocstor_48865_object26147/head//2 by client.2191635.0:26148 2014-08-12 17:16:51.608739 2014-08-13 18:06:06.423820 7f5c973a57a0 20 read_log 194215'441 (0'0) modify 29311a2a/benchmark_data_storage0111-ib.pocstor_48865_object26180/head//2 by client.2191635.0:26181 2014-08-12 17:16:51.729545 2014-08-13 18:06:06.423841 7f5c973a57a0 20 read_log 194215'442 (0'0) modify f9505d2a/benchmark_data_storage0111-ib.pocstor_48865_object26181/head//2 by client.2191635.0:26182 2014-08-12 17:16:51.733103 2014-08-13 18:06:06.423871 7f5c973a57a0 20 read_log 194215'443 (0'0) modify 5991912a/benchmark_data_storage0111-ib.pocstor_48865_object26212/head//2 by client.2191635.0:26213 2014-08-12 17:16:51.844592 2014-08-13 18:06:06.423893 7f5c973a57a0 20 read_log 194215'444 (0'0) modify b036cdea/benchmark_data_storage0111-ib.pocstor_48865_object26253/head//2 by client.2191635.0:26254 2014-08-12 17:16:51.994567 2014-08-13 18:06:06.423915 7f5c973a57a0 20 read_log 194215'445 (0'0) modify 3571b8ea/benchmark_data_storage0111-ib.pocstor_48865_object26474/head//2 by client.2191635.0:26475 2014-08-12 17:16:52.798075 2014-08-13 18:06:06.423937 7f5c973a57a0 20 read_log 194215'446 (0'0) modify cdf9102a/benchmark_data_storage0111-ib.pocstor_48865_object26574/head//2 by client.2191635.0:26575 2014-08-12 17:16:53.164843 2014-08-13 18:06:06.423958 7f5c973a57a0 20 read_log 194215'447 (0'0) modify acb3246a/benchmark_data_storage0111-ib.pocstor_48865_object26583/head//2 by client.2191635.0:26584 2014-08-12 17:16:53.196926 2014-08-13 18:06:06.423980 7f5c973a57a0 20 read_log 194215'448 (0'0) modify c64e22ea/benchmark_data_storage0111-ib.pocstor_48865_object26626/head//2 by client.2191635.0:26627 2014-08-12 17:16:53.354760 2014-08-13 18:06:06.424001 7f5c973a57a0 20 read_log 194215'449 (0'0) modify d778c46a/benchmark_data_storage0111-ib.pocstor_48865_object26634/head//2 by client.2191635.0:26635 2014-08-12 17:16:53.383282 2014-08-13 18:06:06.424022 7f5c973a57a0 20 read_log 194215'450 (0'0) modify 6da2d2ea/benchmark_data_storage0111-ib.pocstor_48865_object26711/head//2 by client.2191635.0:26712 2014-08-12 17:16:53.661811 2014-08-13 18:06:06.424043 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 6655c36a/benchmark_data_storage0111-ib.pocstor_48865_object26778/head//2 by client.2191635.0:26779 2014-08-12 17:16:53.907259 2014-08-13 18:06:06.424070 7f5c973a57a0 20 read_log 194215'452 (0'0) modify d935beaa/benchmark_data_storage0111-ib.pocstor_48865_object26946/head//2 by client.2191635.0:26947 2014-08-12 17:16:54.585600 2014-08-13 18:06:06.424093 7f5c973a57a0 20 read_log 194215'453 (0'0) modify 143d096a/benchmark_data_storage0111-ib.pocstor_48865_object26976/head//2 by client.2191635.0:26977 2014-08-12 17:16:54.786591 2014-08-13 18:06:06.424114 7f5c973a57a0 20 read_log 194215'454 (0'0) modify a5899caa/benchmark_data_storage0111-ib.pocstor_48865_object27029/head//2 by client.2191635.0:27030 2014-08-12 17:16:55.110411 2014-08-13 18:06:06.424156 7f5c973a57a0 20 read_log 194215'455 (0'0) modify b23c082a/benchmark_data_storage0111-ib.pocstor_48865_object27049/head//2 by client.2191635.0:27050 2014-08-12 17:16:55.184521 2014-08-13 18:06:06.424184 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 128e18aa/benchmark_data_storage0111-ib.pocstor_48865_object27097/head//2 by client.2191635.0:27098 2014-08-12 17:16:55.352536 2014-08-13 18:06:06.424206 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 1fef4b2a/benchmark_data_storage0111-ib.pocstor_48865_object27156/head//2 by client.2191635.0:27157 2014-08-12 17:16:55.597599 2014-08-13 18:06:06.424228 7f5c973a57a0 20 read_log 194215'458 (0'0) modify 53f22aaa/benchmark_data_storage0111-ib.pocstor_48865_object27164/head//2 by client.2191635.0:27165 2014-08-12 17:16:55.625229 2014-08-13 18:06:06.424249 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 25a647ea/benchmark_data_storage0111-ib.pocstor_48865_object27294/head//2 by client.2191635.0:27295 2014-08-12 17:16:56.172477 2014-08-13 18:06:06.424270 7f5c973a57a0 20 read_log 194215'460 (0'0) modify edbf666a/benchmark_data_storage0111-ib.pocstor_48865_object27302/head//2 by client.2191635.0:27303 2014-08-12 17:16:56.201858 2014-08-13 18:06:06.424291 7f5c973a57a0 20 read_log 194215'461 (0'0) modify e0414e6a/benchmark_data_storage0111-ib.pocstor_48865_object27325/head//2 by client.2191635.0:27326 2014-08-12 17:16:56.279786 2014-08-13 18:06:06.424313 7f5c973a57a0 20 read_log 194215'462 (0'0) modify fa7d5f6a/benchmark_data_storage0111-ib.pocstor_48865_object27402/head//2 by client.2191635.0:27403 2014-08-12 17:16:56.549606 2014-08-13 18:06:06.424334 7f5c973a57a0 20 read_log 194215'463 (0'0) modify fee55a2a/benchmark_data_storage0111-ib.pocstor_48865_object27451/head//2 by client.2191635.0:27452 2014-08-12 17:16:56.752869 2014-08-13 18:06:06.424360 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 9b85faaa/benchmark_data_storage0111-ib.pocstor_48865_object27523/head//2 by client.2191635.0:27524 2014-08-12 17:16:57.044696 2014-08-13 18:06:06.424416 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 233bee2a/benchmark_data_storage0111-ib.pocstor_48865_object27577/head//2 by client.2191635.0:27578 2014-08-12 17:16:57.235154 2014-08-13 18:06:06.424446 7f5c973a57a0 20 read_log 194215'466 (194215'1) delete 68edfb2a/benchmark_data_storage0111-ib.pocstor_48865_object150/head//2 by client.2191635.0:27747 2014-08-12 17:16:58.740066 2014-08-13 18:06:06.424494 7f5c973a57a0 20 read_log 194215'467 (194215'2) delete 11847ea/benchmark_data_storage0111-ib.pocstor_48865_object183/head//2 by client.2191635.0:27780 2014-08-12 17:16:58.764497 2014-08-13 18:06:06.424534 7f5c973a57a0 20 read_log 194215'468 (194215'3) delete 282e8a2a/benchmark_data_storage0111-ib.pocstor_48865_object269/head//2 by client.2191635.0:27866 2014-08-12 17:16:58.888264 2014-08-13 18:06:06.424562 7f5c973a57a0 20 read_log 194215'469 (194215'4) delete 2975e0ea/benchmark_data_storage0111-ib.pocstor_48865_object333/head//2 by client.2191635.0:27930 2014-08-12 17:16:58.907677 2014-08-13 18:06:06.424585 7f5c973a57a0 20 read_log 194215'470 (194215'5) delete 65323c2a/benchmark_data_storage0111-ib.pocstor_48865_object358/head//2 by client.2191635.0:27955 2014-08-12 17:16:58.915552 2014-08-13 18:06:06.424607 7f5c973a57a0 20 read_log 194215'471 (194215'6) delete 80c081ea/benchmark_data_storage0111-ib.pocstor_48865_object538/head//2 by client.2191635.0:28135 2014-08-12 17:16:58.969893 2014-08-13 18:06:06.424633 7f5c973a57a0 20 read_log 194215'472 (194215'7) delete 277a9c6a/benchmark_data_storage0111-ib.pocstor_48865_object629/head//2 by client.2191635.0:28226 2014-08-12 17:16:58.996966 2014-08-13 18:06:06.424655 7f5c973a57a0 20 read_log 194215'473 (194215'8) delete ce42036a/benchmark_data_storage0111-ib.pocstor_48865_object668/head//2 by client.2191635.0:28265 2014-08-12 17:16:59.009266 2014-08-13 18:06:06.424686 7f5c973a57a0 20 read_log 194215'474 (194215'9) delete 5955272a/benchmark_data_storage0111-ib.pocstor_48865_object715/head//2 by client.2191635.0:28312 2014-08-12 17:16:59.023657 2014-08-13 18:06:06.424727 7f5c973a57a0 20 read_log 194215'475 (194215'10) delete 52f5892a/benchmark_data_storage0111-ib.pocstor_48865_object726/head//2 by client.2191635.0:28323 2014-08-12 17:16:59.028038 2014-08-13 18:06:06.424777 7f5c973a57a0 20 read_log 194215'476 (194215'11) delete d1e810aa/benchmark_data_storage0111-ib.pocstor_48865_object1003/head//2 by client.2191635.0:28600 2014-08-12 17:16:59.115602 2014-08-13 18:06:06.424805 7f5c973a57a0 20 read_log 194215'477 (194215'12) delete 70ee6b6a/benchmark_data_storage0111-ib.pocstor_48865_object1014/head//2 by client.2191635.0:28611 2014-08-12 17:16:59.116995 2014-08-13 18:06:06.424832 7f5c973a57a0 20 read_log 194215'478 (194215'13) delete 81dd632a/benchmark_data_storage0111-ib.pocstor_48865_object1055/head//2 by client.2191635.0:28652 2014-08-12 17:16:59.128804 2014-08-13 18:06:06.424858 7f5c973a57a0 20 read_log 194215'479 (194215'14) delete a31ef9aa/benchmark_data_storage0111-ib.pocstor_48865_object1066/head//2 by client.2191635.0:28663 2014-08-12 17:16:59.131648 2014-08-13 18:06:06.424880 7f5c973a57a0 20 read_log 194215'480 (194215'15) delete bb8082a/benchmark_data_storage0111-ib.pocstor_48865_object1073/head//2 by client.2191635.0:28670 2014-08-12 17:16:59.133969 2014-08-13 18:06:06.424919 7f5c973a57a0 20 read_log 194215'481 (194215'16) delete 8c5d0caa/benchmark_data_storage0111-ib.pocstor_48865_object1105/head//2 by client.2191635.0:28702 2014-08-12 17:16:59.142775 2014-08-13 18:06:06.424955 7f5c973a57a0 20 read_log 194215'482 (194215'17) delete e781e86a/benchmark_data_storage0111-ib.pocstor_48865_object1126/head//2 by client.2191635.0:28723 2014-08-12 17:16:59.149257 2014-08-13 18:06:06.424982 7f5c973a57a0 20 read_log 194215'483 (194215'18) delete bb59aaea/benchmark_data_storage0111-ib.pocstor_48865_object1239/head//2 by client.2191635.0:28836 2014-08-12 17:16:59.184349 2014-08-13 18:06:06.425005 7f5c973a57a0 20 read_log 194215'484 (194215'19) delete f812992a/benchmark_data_storage0111-ib.pocstor_48865_object1316/head//2 by client.2191635.0:28913 2014-08-12 17:16:59.211942 2014-08-13 18:06:06.425027 7f5c973a57a0 20 read_log 194215'485 (194215'20) delete 8845246a/benchmark_data_storage0111-ib.pocstor_48865_object1340/head//2 by client.2191635.0:28937 2014-08-12 17:16:59.215979 2014-08-13 18:06:06.425049 7f5c973a57a0 20 read_log 194215'486 (194215'21) delete ac40caa/benchmark_data_storage0111-ib.pocstor_48865_object1424/head//2 by client.2191635.0:29021 2014-08-12 17:16:59.238876 2014-08-13 18:06:06.425071 7f5c973a57a0 20 read_log 194215'487 (194215'22) delete ff0610aa/benchmark_data_storage0111-ib.pocstor_48865_object1571/head//2 by client.2191635.0:29168 2014-08-12 17:16:59.276674 2014-08-13 18:06:06.425092 7f5c973a57a0 20 read_log 194215'488 (194215'23) delete 95a794ea/benchmark_data_storage0111-ib.pocstor_48865_object1601/head//2 by client.2191635.0:29198 2014-08-12 17:16:59.287444 2014-08-13 18:06:06.425121 7f5c973a57a0 20 read_log 194215'489 (194215'24) delete cae8062a/benchmark_data_storage0111-ib.pocstor_48865_object1620/head//2 by client.2191635.0:29217 2014-08-12 17:16:59.292860 2014-08-13 18:06:06.425161 7f5c973a57a0 20 read_log 194215'490 (194215'25) delete 920ef56a/benchmark_data_storage0111-ib.pocstor_48865_object1759/head//2 by client.2191635.0:29356 2014-08-12 17:16:59.329537 2014-08-13 18:06:06.425195 7f5c973a57a0 20 read_log 194215'491 (194215'26) delete a732e2a/benchmark_data_storage0111-ib.pocstor_48865_object1807/head//2 by client.2191635.0:29404 2014-08-12 17:16:59.341555 2014-08-13 18:06:06.425235 7f5c973a57a0 20 read_log 194215'492 (194215'27) delete c4c702aa/benchmark_data_storage0111-ib.pocstor_48865_object1871/head//2 by client.2191635.0:29468 2014-08-12 17:16:59.358921 2014-08-13 18:06:06.425268 7f5c973a57a0 20 read_log 194215'493 (194215'28) delete 279c346a/benchmark_data_storage0111-ib.pocstor_48865_object1875/head//2 by client.2191635.0:29472 2014-08-12 17:16:59.359980 2014-08-13 18:06:06.425291 7f5c973a57a0 20 read_log 194215'494 (194215'29) delete 553263ea/benchmark_data_storage0111-ib.pocstor_48865_object2043/head//2 by client.2191635.0:29640 2014-08-12 17:16:59.407869 2014-08-13 18:06:06.425347 7f5c973a57a0 20 read_log 194215'495 (194215'30) delete f16ac3ea/benchmark_data_storage0111-ib.pocstor_48865_object2102/head//2 by client.2191635.0:29699 2014-08-12 17:16:59.424162 2014-08-13 18:06:06.425376 7f5c973a57a0 20 read_log 194215'496 (194215'31) delete 5fc8512a/benchmark_data_storage0111-ib.pocstor_48865_object2104/head//2 by client.2191635.0:29701 2014-08-12 17:16:59.424878 2014-08-13 18:06:06.425403 7f5c973a57a0 20 read_log 194215'497 (194215'32) delete a896f2ea/benchmark_data_storage0111-ib.pocstor_48865_object2127/head//2 by client.2191635.0:29724 2014-08-12 17:16:59.430160 2014-08-13 18:06:06.425428 7f5c973a57a0 20 read_log 194215'498 (194215'33) delete b516fdaa/benchmark_data_storage0111-ib.pocstor_48865_object2151/head//2 by client.2191635.0:29748 2014-08-12 17:16:59.436895 2014-08-13 18:06:06.425471 7f5c973a57a0 20 read_log 194215'499 (194215'34) delete 8cda0e6a/benchmark_data_storage0111-ib.pocstor_48865_object2185/head//2 by client.2191635.0:29782 2014-08-12 17:16:59.445768 2014-08-13 18:06:06.425503 7f5c973a57a0 20 read_log 194215'500 (194215'35) delete 130ec9aa/benchmark_data_storage0111-ib.pocstor_48865_object2195/head//2 by client.2191635.0:29792 2014-08-12 17:16:59.448530 2014-08-13 18:06:06.425526 7f5c973a57a0 20 read_log 194215'501 (194215'36) delete d4544c2a/benchmark_data_storage0111-ib.pocstor_48865_object2225/head//2 by client.2191635.0:29822 2014-08-12 17:16:59.456450 2014-08-13 18:06:06.425553 7f5c973a57a0 20 read_log 194215'502 (194215'37) delete 8855e5aa/benchmark_data_storage0111-ib.pocstor_48865_object2244/head//2 by client.2191635.0:29841 2014-08-12 17:16:59.461293 2014-08-13 18:06:06.425579 7f5c973a57a0 20 read_log 194215'503 (194215'38) delete 875f70ea/benchmark_data_storage0111-ib.pocstor_48865_object2251/head//2 by client.2191635.0:29848 2014-08-12 17:16:59.463802 2014-08-13 18:06:06.425624 7f5c973a57a0 20 read_log 194215'504 (194215'39) delete 365a75ea/benchmark_data_storage0111-ib.pocstor_48865_object2593/head//2 by client.2191635.0:30190 2014-08-12 17:16:59.567147 2014-08-13 18:06:06.425652 7f5c973a57a0 20 read_log 194215'505 (194215'40) delete 1cc465aa/benchmark_data_storage0111-ib.pocstor_48865_object2601/head//2 by client.2191635.0:30198 2014-08-12 17:16:59.569483 2014-08-13 18:06:06.425677 7f5c973a57a0 20 read_log 194215'506 (194215'41) delete b7b2a22a/benchmark_data_storage0111-ib.pocstor_48865_object2629/head//2 by client.2191635.0:30226 2014-08-12 17:16:59.580240 2014-08-13 18:06:06.425704 7f5c973a57a0 20 read_log 194215'507 (194215'42) delete 109903aa/benchmark_data_storage0111-ib.pocstor_48865_object2691/head//2 by client.2191635.0:30288 2014-08-12 17:16:59.601237 2014-08-13 18:06:06.425726 7f5c973a57a0 20 read_log 194215'508 (194215'43) delete db63af6a/benchmark_data_storage0111-ib.pocstor_48865_object2699/head//2 by client.2191635.0:30296 2014-08-12 17:16:59.602450 2014-08-13 18:06:06.425770 7f5c973a57a0 20 read_log 194215'509 (194215'44) delete 3dfdc96a/benchmark_data_storage0111-ib.pocstor_48865_object2712/head//2 by client.2191635.0:30309 2014-08-12 17:16:59.609500 2014-08-13 18:06:06.425797 7f5c973a57a0 20 read_log 194215'510 (194215'45) delete 839a1faa/benchmark_data_storage0111-ib.pocstor_48865_object2786/head//2 by client.2191635.0:30383 2014-08-12 17:16:59.631825 2014-08-13 18:06:06.425839 7f5c973a57a0 20 read_log 194215'511 (194215'46) delete 7d0316aa/benchmark_data_storage0111-ib.pocstor_48865_object2888/head//2 by client.2191635.0:30485 2014-08-12 17:16:59.661130 2014-08-13 18:06:06.425867 7f5c973a57a0 20 read_log 194215'512 (194215'47) delete 8fd59f2a/benchmark_data_storage0111-ib.pocstor_48865_object3016/head//2 by client.2191635.0:30613 2014-08-12 17:16:59.696494 2014-08-13 18:06:06.425894 7f5c973a57a0 20 read_log 194215'513 (194215'48) delete 2f6cc42a/benchmark_data_storage0111-ib.pocstor_48865_object3021/head//2 by client.2191635.0:30618 2014-08-12 17:16:59.698194 2014-08-13 18:06:06.425920 7f5c973a57a0 20 read_log 194215'514 (194215'49) delete 32c490ea/benchmark_data_storage0111-ib.pocstor_48865_object3081/head//2 by client.2191635.0:30678 2014-08-12 17:16:59.714601 2014-08-13 18:06:06.425958 7f5c973a57a0 20 read_log 194215'515 (194215'50) delete ead47aea/benchmark_data_storage0111-ib.pocstor_48865_object3105/head//2 by client.2191635.0:30702 2014-08-12 17:16:59.721304 2014-08-13 18:06:06.426008 7f5c973a57a0 20 read_log 194215'516 (194215'51) delete bddd492a/benchmark_data_storage0111-ib.pocstor_48865_object3141/head//2 by client.2191635.0:30738 2014-08-12 17:16:59.731462 2014-08-13 18:06:06.426037 7f5c973a57a0 20 read_log 194215'517 (194215'52) delete f9981caa/benchmark_data_storage0111-ib.pocstor_48865_object3321/head//2 by client.2191635.0:30918 2014-08-12 17:16:59.784791 2014-08-13 18:06:06.426103 7f5c973a57a0 20 read_log 194215'518 (194215'53) delete d01bb4ea/benchmark_data_storage0111-ib.pocstor_48865_object3371/head//2 by client.2191635.0:30968 2014-08-12 17:16:59.799870 2014-08-13 18:06:06.426157 7f5c973a57a0 20 read_log 194215'519 (194215'54) delete f0690b6a/benchmark_data_storage0111-ib.pocstor_48865_object3420/head//2 by client.2191635.0:31017 2014-08-12 17:16:59.813627 2014-08-13 18:06:06.426190 7f5c973a57a0 20 read_log 194215'520 (194215'55) delete ef7fe82a/benchmark_data_storage0111-ib.pocstor_48865_object3426/head//2 by client.2191635.0:31023 2014-08-12 17:16:59.815586 2014-08-13 18:06:06.426212 7f5c973a57a0 20 read_log 194215'521 (194215'56) delete 4f5aa4ea/benchmark_data_storage0111-ib.pocstor_48865_object3456/head//2 by client.2191635.0:31053 2014-08-12 17:16:59.824320 2014-08-13 18:06:06.426234 7f5c973a57a0 20 read_log 194215'522 (194215'57) delete e192b76a/benchmark_data_storage0111-ib.pocstor_48865_object3465/head//2 by client.2191635.0:31062 2014-08-12 17:16:59.828559 2014-08-13 18:06:06.426274 7f5c973a57a0 20 read_log 194215'523 (194215'58) delete ed61ad6a/benchmark_data_storage0111-ib.pocstor_48865_object3599/head//2 by client.2191635.0:31196 2014-08-12 17:16:59.873026 2014-08-13 18:06:06.426300 7f5c973a57a0 20 read_log 194215'524 (194215'59) delete 284476ea/benchmark_data_storage0111-ib.pocstor_48865_object3637/head//2 by client.2191635.0:31234 2014-08-12 17:16:59.886034 2014-08-13 18:06:06.426322 7f5c973a57a0 20 read_log 194215'525 (194215'60) delete 69ffe8ea/benchmark_data_storage0111-ib.pocstor_48865_object3643/head//2 by client.2191635.0:31240 2014-08-12 17:16:59.886460 2014-08-13 18:06:06.426349 7f5c973a57a0 20 read_log 194215'526 (194215'61) delete 4443936a/benchmark_data_storage0111-ib.pocstor_48865_object3673/head//2 by client.2191635.0:31270 2014-08-12 17:16:59.896764 2014-08-13 18:06:06.426371 7f5c973a57a0 20 read_log 194215'527 (194215'62) delete b1fb3aaa/benchmark_data_storage0111-ib.pocstor_48865_object3684/head//2 by client.2191635.0:31281 2014-08-12 17:16:59.899705 2014-08-13 18:06:06.426393 7f5c973a57a0 20 read_log 194215'528 (194215'63) delete abbf486a/benchmark_data_storage0111-ib.pocstor_48865_object3708/head//2 by client.2191635.0:31305 2014-08-12 17:16:59.907925 2014-08-13 18:06:06.426414 7f5c973a57a0 20 read_log 194215'529 (194215'64) delete d2774aaa/benchmark_data_storage0111-ib.pocstor_48865_object3723/head//2 by client.2191635.0:31320 2014-08-12 17:16:59.911646 2014-08-13 18:06:06.426436 7f5c973a57a0 20 read_log 194215'530 (194215'65) delete d4716aaa/benchmark_data_storage0111-ib.pocstor_48865_object3873/head//2 by client.2191635.0:31470 2014-08-12 17:16:59.957188 2014-08-13 18:06:06.426458 7f5c973a57a0 20 read_log 194215'531 (194215'66) delete 2546412a/benchmark_data_storage0111-ib.pocstor_48865_object3982/head//2 by client.2191635.0:31579 2014-08-12 17:16:59.989662 2014-08-13 18:06:06.426480 7f5c973a57a0 20 read_log 194215'532 (194215'67) delete fce3fb2a/benchmark_data_storage0111-ib.pocstor_48865_object4047/head//2 by client.2191635.0:31644 2014-08-12 17:17:00.010584 2014-08-13 18:06:06.426502 7f5c973a57a0 20 read_log 194215'533 (194215'68) delete 880c6cea/benchmark_data_storage0111-ib.pocstor_48865_object4061/head//2 by client.2191635.0:31658 2014-08-12 17:17:00.015500 2014-08-13 18:06:06.426524 7f5c973a57a0 20 read_log 194215'534 (194215'69) delete da367caa/benchmark_data_storage0111-ib.pocstor_48865_object4094/head//2 by client.2191635.0:31691 2014-08-12 17:17:00.026961 2014-08-13 18:06:06.426624 7f5c973a57a0 20 read_log 194215'535 (194215'70) delete fbd37baa/benchmark_data_storage0111-ib.pocstor_48865_object4145/head//2 by client.2191635.0:31742 2014-08-12 17:17:00.041234 2014-08-13 18:06:06.426655 7f5c973a57a0 20 read_log 194215'536 (194215'71) delete df68caea/benchmark_data_storage0111-ib.pocstor_48865_object4151/head//2 by client.2191635.0:31748 2014-08-12 17:17:00.042368 2014-08-13 18:06:06.426677 7f5c973a57a0 20 read_log 194215'537 (194215'72) delete 7757da6a/benchmark_data_storage0111-ib.pocstor_48865_object4234/head//2 by client.2191635.0:31831 2014-08-12 17:17:00.069978 2014-08-13 18:06:06.426699 7f5c973a57a0 20 read_log 194215'538 (194215'73) delete aef9282a/benchmark_data_storage0111-ib.pocstor_48865_object4266/head//2 by client.2191635.0:31863 2014-08-12 17:17:00.078478 2014-08-13 18:06:06.426721 7f5c973a57a0 20 read_log 194215'539 (194215'74) delete ca7790ea/benchmark_data_storage0111-ib.pocstor_48865_object4302/head//2 by client.2191635.0:31899 2014-08-12 17:17:00.089311 2014-08-13 18:06:06.426743 7f5c973a57a0 20 read_log 194215'540 (194215'75) delete 5c96bb6a/benchmark_data_storage0111-ib.pocstor_48865_object4312/head//2 by client.2191635.0:31909 2014-08-12 17:17:00.091635 2014-08-13 18:06:06.426767 7f5c973a57a0 20 read_log 194215'541 (194215'76) delete 463bdeaa/benchmark_data_storage0111-ib.pocstor_48865_object4392/head//2 by client.2191635.0:31989 2014-08-12 17:17:00.115263 2014-08-13 18:06:06.426849 7f5c973a57a0 20 read_log 194215'542 (194215'77) delete d37ec2ea/benchmark_data_storage0111-ib.pocstor_48865_object4404/head//2 by client.2191635.0:32001 2014-08-12 17:17:00.118611 2014-08-13 18:06:06.426884 7f5c973a57a0 20 read_log 194215'543 (194215'78) delete 50c68dea/benchmark_data_storage0111-ib.pocstor_48865_object4406/head//2 by client.2191635.0:32003 2014-08-12 17:17:00.119336 2014-08-13 18:06:06.426907 7f5c973a57a0 20 read_log 194215'544 (194215'79) delete 4e5fdaaa/benchmark_data_storage0111-ib.pocstor_48865_object4437/head//2 by client.2191635.0:32034 2014-08-12 17:17:00.127312 2014-08-13 18:06:06.426952 7f5c973a57a0 20 read_log 194215'545 (194215'80) delete c56423aa/benchmark_data_storage0111-ib.pocstor_48865_object4456/head//2 by client.2191635.0:32053 2014-08-12 17:17:00.135904 2014-08-13 18:06:06.426975 7f5c973a57a0 20 read_log 194215'546 (194215'81) delete 48228f2a/benchmark_data_storage0111-ib.pocstor_48865_object4543/head//2 by client.2191635.0:32140 2014-08-12 17:17:00.164226 2014-08-13 18:06:06.427042 7f5c973a57a0 20 read_log 194215'547 (194215'82) delete f096ccea/benchmark_data_storage0111-ib.pocstor_48865_object4594/head//2 by client.2191635.0:32191 2014-08-12 17:17:00.178487 2014-08-13 18:06:06.427065 7f5c973a57a0 20 read_log 194215'548 (194215'83) delete 775974ea/benchmark_data_storage0111-ib.pocstor_48865_object4605/head//2 by client.2191635.0:32202 2014-08-12 17:17:00.180353 2014-08-13 18:06:06.427087 7f5c973a57a0 20 read_log 194215'549 (194215'84) delete dee4596a/benchmark_data_storage0111-ib.pocstor_48865_object4660/head//2 by client.2191635.0:32257 2014-08-12 17:17:00.201680 2014-08-13 18:06:06.427116 7f5c973a57a0 20 read_log 194215'550 (194215'85) delete b3d8632a/benchmark_data_storage0111-ib.pocstor_48865_object4719/head//2 by client.2191635.0:32316 2014-08-12 17:17:00.216758 2014-08-13 18:06:06.427141 7f5c973a57a0 20 read_log 194215'551 (194215'86) delete dfa2a76a/benchmark_data_storage0111-ib.pocstor_48865_object4725/head//2 by client.2191635.0:32322 2014-08-12 17:17:00.217721 2014-08-13 18:06:06.427174 7f5c973a57a0 20 read_log 194215'552 (194215'87) delete ea96c2aa/benchmark_data_storage0111-ib.pocstor_48865_object4778/head//2 by client.2191635.0:32375 2014-08-12 17:17:00.235434 2014-08-13 18:06:06.427213 7f5c973a57a0 20 read_log 194215'553 (194215'88) delete 37ca6bea/benchmark_data_storage0111-ib.pocstor_48865_object4949/head//2 by client.2191635.0:32546 2014-08-12 17:17:00.281456 2014-08-13 18:06:06.427250 7f5c973a57a0 20 read_log 194215'554 (194215'89) delete 62eca2ea/benchmark_data_storage0111-ib.pocstor_48865_object4961/head//2 by client.2191635.0:32558 2014-08-12 17:17:00.284197 2014-08-13 18:06:06.427326 7f5c973a57a0 20 read_log 194215'555 (194215'90) delete 489460ea/benchmark_data_storage0111-ib.pocstor_48865_object4986/head//2 by client.2191635.0:32583 2014-08-12 17:17:00.290632 2014-08-13 18:06:06.427375 7f5c973a57a0 20 read_log 194215'556 (194215'91) delete fc746b2a/benchmark_data_storage0111-ib.pocstor_48865_object5031/head//2 by client.2191635.0:32628 2014-08-12 17:17:00.303341 2014-08-13 18:06:06.427407 7f5c973a57a0 20 read_log 194215'557 (194215'92) delete dc8ffd6a/benchmark_data_storage0111-ib.pocstor_48865_object5063/head//2 by client.2191635.0:32660 2014-08-12 17:17:00.312776 2014-08-13 18:06:06.427434 7f5c973a57a0 20 read_log 194215'558 (194215'93) delete 98f0566a/benchmark_data_storage0111-ib.pocstor_48865_object5274/head//2 by client.2191635.0:32871 2014-08-12 17:17:00.368232 2014-08-13 18:06:06.427473 7f5c973a57a0 20 read_log 194215'559 (194215'94) delete a673bdea/benchmark_data_storage0111-ib.pocstor_48865_object5308/head//2 by client.2191635.0:32905 2014-08-12 17:17:00.377189 2014-08-13 18:06:06.427521 7f5c973a57a0 20 read_log 194215'560 (194215'95) delete dd5e12aa/benchmark_data_storage0111-ib.pocstor_48865_object5321/head//2 by client.2191635.0:32918 2014-08-12 17:17:00.381361 2014-08-13 18:06:06.427552 7f5c973a57a0 20 read_log 194215'561 (194215'96) delete 743f146a/benchmark_data_storage0111-ib.pocstor_48865_object5342/head//2 by client.2191635.0:32939 2014-08-12 17:17:00.386094 2014-08-13 18:06:06.427574 7f5c973a57a0 20 read_log 194215'562 (194215'97) delete 4b19ae2a/benchmark_data_storage0111-ib.pocstor_48865_object5365/head//2 by client.2191635.0:32962 2014-08-12 17:17:00.393665 2014-08-13 18:06:06.427596 7f5c973a57a0 20 read_log 194215'563 (194215'98) delete 4187d4aa/benchmark_data_storage0111-ib.pocstor_48865_object5418/head//2 by client.2191635.0:33015 2014-08-12 17:17:00.407853 2014-08-13 18:06:06.427621 7f5c973a57a0 20 read_log 194215'564 (194215'99) delete 76e737aa/benchmark_data_storage0111-ib.pocstor_48865_object5442/head//2 by client.2191635.0:33039 2014-08-12 17:17:00.413907 2014-08-13 18:06:06.427643 7f5c973a57a0 20 read_log 194215'565 (194215'100) delete 90d3712a/benchmark_data_storage0111-ib.pocstor_48865_object5446/head//2 by client.2191635.0:33043 2014-08-12 17:17:00.415286 2014-08-13 18:06:06.427666 7f5c973a57a0 20 read_log 194215'566 (194215'101) delete 59cb2aea/benchmark_data_storage0111-ib.pocstor_48865_object5453/head//2 by client.2191635.0:33050 2014-08-12 17:17:00.418769 2014-08-13 18:06:06.427688 7f5c973a57a0 20 read_log 194215'567 (194215'102) delete d0ed1c6a/benchmark_data_storage0111-ib.pocstor_48865_object5498/head//2 by client.2191635.0:33095 2014-08-12 17:17:00.432742 2014-08-13 18:06:06.427709 7f5c973a57a0 20 read_log 194215'568 (194215'103) delete 48e796a/benchmark_data_storage0111-ib.pocstor_48865_object5511/head//2 by client.2191635.0:33108 2014-08-12 17:17:00.437389 2014-08-13 18:06:06.427731 7f5c973a57a0 20 read_log 194215'569 (194215'104) delete b179e16a/benchmark_data_storage0111-ib.pocstor_48865_object5567/head//2 by client.2191635.0:33164 2014-08-12 17:17:00.453265 2014-08-13 18:06:06.427752 7f5c973a57a0 20 read_log 194215'570 (194215'105) delete d38639aa/benchmark_data_storage0111-ib.pocstor_48865_object5574/head//2 by client.2191635.0:33171 2014-08-12 17:17:00.454545 2014-08-13 18:06:06.427774 7f5c973a57a0 20 read_log 194215'571 (194215'106) delete 6cb251aa/benchmark_data_storage0111-ib.pocstor_48865_object5598/head//2 by client.2191635.0:33195 2014-08-12 17:17:00.461774 2014-08-13 18:06:06.427795 7f5c973a57a0 20 read_log 194215'572 (194215'107) delete eaccd2a/benchmark_data_storage0111-ib.pocstor_48865_object5617/head//2 by client.2191635.0:33214 2014-08-12 17:17:00.469469 2014-08-13 18:06:06.427822 7f5c973a57a0 20 read_log 194215'573 (194215'108) delete a3f082aa/benchmark_data_storage0111-ib.pocstor_48865_object5622/head//2 by client.2191635.0:33219 2014-08-12 17:17:00.470477 2014-08-13 18:06:06.427844 7f5c973a57a0 20 read_log 194215'574 (194215'109) delete a3af342a/benchmark_data_storage0111-ib.pocstor_48865_object5745/head//2 by client.2191635.0:33342 2014-08-12 17:17:00.503283 2014-08-13 18:06:06.427878 7f5c973a57a0 20 read_log 194215'575 (194215'110) delete e572e62a/benchmark_data_storage0111-ib.pocstor_48865_object5964/head//2 by client.2191635.0:33561 2014-08-12 17:17:00.571300 2014-08-13 18:06:06.427904 7f5c973a57a0 20 read_log 194215'576 (194215'111) delete b874d76a/benchmark_data_storage0111-ib.pocstor_48865_object5981/head//2 by client.2191635.0:33578 2014-08-12 17:17:00.576497 2014-08-13 18:06:06.427926 7f5c973a57a0 20 read_log 194215'577 (194215'112) delete b9b163ea/benchmark_data_storage0111-ib.pocstor_48865_object5987/head//2 by client.2191635.0:33584 2014-08-12 17:17:00.578577 2014-08-13 18:06:06.427948 7f5c973a57a0 20 read_log 194215'578 (194215'113) delete 48d2f52a/benchmark_data_storage0111-ib.pocstor_48865_object6057/head//2 by client.2191635.0:33654 2014-08-12 17:17:00.598759 2014-08-13 18:06:06.427969 7f5c973a57a0 20 read_log 194215'579 (194215'114) delete 857610ea/benchmark_data_storage0111-ib.pocstor_48865_object6104/head//2 by client.2191635.0:33701 2014-08-12 17:17:00.611316 2014-08-13 18:06:06.427991 7f5c973a57a0 20 read_log 194215'580 (194215'115) delete 2628a2ea/benchmark_data_storage0111-ib.pocstor_48865_object6300/head//2 by client.2191635.0:33897 2014-08-12 17:17:00.670677 2014-08-13 18:06:06.428012 7f5c973a57a0 20 read_log 194215'581 (194215'116) delete 6374406a/benchmark_data_storage0111-ib.pocstor_48865_object6344/head//2 by client.2191635.0:33941 2014-08-12 17:17:00.684812 2014-08-13 18:06:06.428033 7f5c973a57a0 20 read_log 194215'582 (194215'117) delete 1a2b33ea/benchmark_data_storage0111-ib.pocstor_48865_object6404/head//2 by client.2191635.0:34001 2014-08-12 17:17:00.701931 2014-08-13 18:06:06.428055 7f5c973a57a0 20 read_log 194215'583 (194215'118) delete 56ba632a/benchmark_data_storage0111-ib.pocstor_48865_object6415/head//2 by client.2191635.0:34012 2014-08-12 17:17:00.706675 2014-08-13 18:06:06.428076 7f5c973a57a0 20 read_log 194215'584 (194215'119) delete 7073ea/benchmark_data_storage0111-ib.pocstor_48865_object6523/head//2 by client.2191635.0:34120 2014-08-12 17:17:00.741268 2014-08-13 18:06:06.428098 7f5c973a57a0 20 read_log 194215'585 (194215'120) delete 4be729ea/benchmark_data_storage0111-ib.pocstor_48865_object6640/head//2 by client.2191635.0:34237 2014-08-12 17:17:00.775265 2014-08-13 18:06:06.428119 7f5c973a57a0 20 read_log 194215'586 (194215'121) delete f386bea/benchmark_data_storage0111-ib.pocstor_48865_object6650/head//2 by client.2191635.0:34247 2014-08-12 17:17:00.778133 2014-08-13 18:06:06.428141 7f5c973a57a0 20 read_log 194215'587 (194215'122) delete 64fac96a/benchmark_data_storage0111-ib.pocstor_48865_object6732/head//2 by client.2191635.0:34329 2014-08-12 17:17:00.803044 2014-08-13 18:06:06.428177 7f5c973a57a0 20 read_log 194215'588 (194215'123) delete 68644faa/benchmark_data_storage0111-ib.pocstor_48865_object6763/head//2 by client.2191635.0:34360 2014-08-12 17:17:00.811578 2014-08-13 18:06:06.428199 7f5c973a57a0 20 read_log 194215'589 (194215'124) delete e27b086a/benchmark_data_storage0111-ib.pocstor_48865_object6779/head//2 by client.2191635.0:34376 2014-08-12 17:17:00.815761 2014-08-13 18:06:06.428224 7f5c973a57a0 20 read_log 194215'590 (194215'125) delete db1f17ea/benchmark_data_storage0111-ib.pocstor_48865_object6852/head//2 by client.2191635.0:34449 2014-08-12 17:17:00.837682 2014-08-13 18:06:06.428246 7f5c973a57a0 20 read_log 194215'591 (194215'126) delete 45f4e1ea/benchmark_data_storage0111-ib.pocstor_48865_object6955/head//2 by client.2191635.0:34552 2014-08-12 17:17:00.868563 2014-08-13 18:06:06.428271 7f5c973a57a0 20 read_log 194215'592 (194215'127) delete 223884ea/benchmark_data_storage0111-ib.pocstor_48865_object7007/head//2 by client.2191635.0:34604 2014-08-12 17:17:00.885514 2014-08-13 18:06:06.428338 7f5c973a57a0 20 read_log 194215'593 (194215'128) delete 84d19fea/benchmark_data_storage0111-ib.pocstor_48865_object7058/head//2 by client.2191635.0:34655 2014-08-12 17:17:00.899884 2014-08-13 18:06:06.428365 7f5c973a57a0 20 read_log 194215'594 (194215'129) delete a64a352a/benchmark_data_storage0111-ib.pocstor_48865_object7111/head//2 by client.2191635.0:34708 2014-08-12 17:17:00.915067 2014-08-13 18:06:06.428399 7f5c973a57a0 20 read_log 194215'595 (194215'130) delete 9abeec2a/benchmark_data_storage0111-ib.pocstor_48865_object7201/head//2 by client.2191635.0:34798 2014-08-12 17:17:00.944448 2014-08-13 18:06:06.428425 7f5c973a57a0 20 read_log 194215'596 (194215'131) delete 2a4d06aa/benchmark_data_storage0111-ib.pocstor_48865_object7255/head//2 by client.2191635.0:34852 2014-08-12 17:17:00.960979 2014-08-13 18:06:06.428447 7f5c973a57a0 20 read_log 194215'597 (194215'132) delete 64ec0faa/benchmark_data_storage0111-ib.pocstor_48865_object7261/head//2 by client.2191635.0:34858 2014-08-12 17:17:00.962247 2014-08-13 18:06:06.428489 7f5c973a57a0 20 read_log 194215'598 (194215'133) delete d703ceaa/benchmark_data_storage0111-ib.pocstor_48865_object7323/head//2 by client.2191635.0:34920 2014-08-12 17:17:00.979774 2014-08-13 18:06:06.428539 7f5c973a57a0 20 read_log 194215'599 (194215'134) delete afc50fea/benchmark_data_storage0111-ib.pocstor_48865_object7361/head//2 by client.2191635.0:34958 2014-08-12 17:17:00.991800 2014-08-13 18:06:06.428569 7f5c973a57a0 20 read_log 194215'600 (194215'135) delete 70d399aa/benchmark_data_storage0111-ib.pocstor_48865_object7567/head//2 by client.2191635.0:35164 2014-08-12 17:17:01.053507 2014-08-13 18:06:06.428592 7f5c973a57a0 20 read_log 194215'601 (194215'136) delete 7e608b6a/benchmark_data_storage0111-ib.pocstor_48865_object7574/head//2 by client.2191635.0:35171 2014-08-12 17:17:01.056899 2014-08-13 18:06:06.428614 7f5c973a57a0 20 read_log 194215'602 (194215'137) delete 55a67f2a/benchmark_data_storage0111-ib.pocstor_48865_object7751/head//2 by client.2191635.0:35348 2014-08-12 17:17:01.115762 2014-08-13 18:06:06.428656 7f5c973a57a0 20 read_log 194215'603 (194215'138) delete c7ab856a/benchmark_data_storage0111-ib.pocstor_48865_object7992/head//2 by client.2191635.0:35589 2014-08-12 17:17:01.198568 2014-08-13 18:06:06.428685 7f5c973a57a0 20 read_log 194215'604 (194215'139) delete 89882f2a/benchmark_data_storage0111-ib.pocstor_48865_object8149/head//2 by client.2191635.0:35746 2014-08-12 17:17:01.245238 2014-08-13 18:06:06.428711 7f5c973a57a0 20 read_log 194215'605 (194215'140) delete 560df9ea/benchmark_data_storage0111-ib.pocstor_48865_object8228/head//2 by client.2191635.0:35825 2014-08-12 17:17:01.265276 2014-08-13 18:06:06.428733 7f5c973a57a0 20 read_log 194215'606 (194215'141) delete 88c4606a/benchmark_data_storage0111-ib.pocstor_48865_object8266/head//2 by client.2191635.0:35863 2014-08-12 17:17:01.277415 2014-08-13 18:06:06.428755 7f5c973a57a0 20 read_log 194215'607 (194215'142) delete addd1f6a/benchmark_data_storage0111-ib.pocstor_48865_object8285/head//2 by client.2191635.0:35882 2014-08-12 17:17:01.282786 2014-08-13 18:06:06.428777 7f5c973a57a0 20 read_log 194215'608 (194215'143) delete 53f2bbaa/benchmark_data_storage0111-ib.pocstor_48865_object8316/head//2 by client.2191635.0:35913 2014-08-12 17:17:01.290901 2014-08-13 18:06:06.428798 7f5c973a57a0 20 read_log 194215'609 (194215'144) delete e60a5caa/benchmark_data_storage0111-ib.pocstor_48865_object8605/head//2 by client.2191635.0:36202 2014-08-12 17:17:01.380864 2014-08-13 18:06:06.428820 7f5c973a57a0 20 read_log 194215'610 (194215'145) delete e9f3b4aa/benchmark_data_storage0111-ib.pocstor_48865_object8698/head//2 by client.2191635.0:36295 2014-08-12 17:17:01.410528 2014-08-13 18:06:06.428872 7f5c973a57a0 20 read_log 194215'611 (194215'146) delete 66d1cb6a/benchmark_data_storage0111-ib.pocstor_48865_object8716/head//2 by client.2191635.0:36313 2014-08-12 17:17:01.415226 2014-08-13 18:06:06.428900 7f5c973a57a0 20 read_log 194215'612 (194215'147) delete fbb9d36a/benchmark_data_storage0111-ib.pocstor_48865_object8783/head//2 by client.2191635.0:36380 2014-08-12 17:17:01.438106 2014-08-13 18:06:06.428922 7f5c973a57a0 20 read_log 194215'613 (194215'148) delete c746816a/benchmark_data_storage0111-ib.pocstor_48865_object8787/head//2 by client.2191635.0:36384 2014-08-12 17:17:01.438574 2014-08-13 18:06:06.428944 7f5c973a57a0 20 read_log 194215'614 (194215'149) delete 3a4a1e6a/benchmark_data_storage0111-ib.pocstor_48865_object8947/head//2 by client.2191635.0:36544 2014-08-12 17:17:01.489975 2014-08-13 18:06:06.428982 7f5c973a57a0 20 read_log 194215'615 (194215'150) delete bc379f6a/benchmark_data_storage0111-ib.pocstor_48865_object9016/head//2 by client.2191635.0:36613 2014-08-12 17:17:01.513775 2014-08-13 18:06:06.429009 7f5c973a57a0 20 read_log 194215'616 (194215'151) delete cb6d2d6a/benchmark_data_storage0111-ib.pocstor_48865_object9125/head//2 by client.2191635.0:36722 2014-08-12 17:17:01.545025 2014-08-13 18:06:06.429032 7f5c973a57a0 20 read_log 194215'617 (194215'152) delete 1ef336ea/benchmark_data_storage0111-ib.pocstor_48865_object9155/head//2 by client.2191635.0:36752 2014-08-12 17:17:01.555386 2014-08-13 18:06:06.429056 7f5c973a57a0 20 read_log 194215'618 (194215'153) delete 7bc5afea/benchmark_data_storage0111-ib.pocstor_48865_object9205/head//2 by client.2191635.0:36802 2014-08-12 17:17:01.571963 2014-08-13 18:06:06.429078 7f5c973a57a0 20 read_log 194215'619 (194215'154) delete 4d0f22ea/benchmark_data_storage0111-ib.pocstor_48865_object9210/head//2 by client.2191635.0:36807 2014-08-12 17:17:01.573478 2014-08-13 18:06:06.429100 7f5c973a57a0 20 read_log 194215'620 (194215'155) delete 33b536ea/benchmark_data_storage0111-ib.pocstor_48865_object9217/head//2 by client.2191635.0:36814 2014-08-12 17:17:01.576447 2014-08-13 18:06:06.429121 7f5c973a57a0 20 read_log 194215'621 (194215'156) delete c09e996a/benchmark_data_storage0111-ib.pocstor_48865_object9321/head//2 by client.2191635.0:36918 2014-08-12 17:17:01.607609 2014-08-13 18:06:06.429143 7f5c973a57a0 20 read_log 194215'622 (194215'157) delete 884b2aa/benchmark_data_storage0111-ib.pocstor_48865_object9349/head//2 by client.2191635.0:36946 2014-08-12 17:17:01.614956 2014-08-13 18:06:06.429178 7f5c973a57a0 20 read_log 194215'623 (194215'158) delete 7bea2ea/benchmark_data_storage0111-ib.pocstor_48865_object9369/head//2 by client.2191635.0:36966 2014-08-12 17:17:01.623012 2014-08-13 18:06:06.429200 7f5c973a57a0 20 read_log 194215'624 (194215'159) delete d092c3ea/benchmark_data_storage0111-ib.pocstor_48865_object9464/head//2 by client.2191635.0:37061 2014-08-12 17:17:01.652005 2014-08-13 18:06:06.429222 7f5c973a57a0 20 read_log 194215'625 (194215'160) delete f2d72e2a/benchmark_data_storage0111-ib.pocstor_48865_object9497/head//2 by client.2191635.0:37094 2014-08-12 17:17:01.661834 2014-08-13 18:06:06.429243 7f5c973a57a0 20 read_log 194215'626 (194215'161) delete 7ed5766a/benchmark_data_storage0111-ib.pocstor_48865_object9512/head//2 by client.2191635.0:37109 2014-08-12 17:17:01.664897 2014-08-13 18:06:06.429265 7f5c973a57a0 20 read_log 194215'627 (194215'162) delete 3108d22a/benchmark_data_storage0111-ib.pocstor_48865_object9561/head//2 by client.2191635.0:37158 2014-08-12 17:17:01.679067 2014-08-13 18:06:06.429291 7f5c973a57a0 20 read_log 194215'628 (194215'163) delete 55a7182a/benchmark_data_storage0111-ib.pocstor_48865_object9618/head//2 by client.2191635.0:37215 2014-08-12 17:17:01.694910 2014-08-13 18:06:06.429317 7f5c973a57a0 20 read_log 194215'629 (194215'164) delete dfc811ea/benchmark_data_storage0111-ib.pocstor_48865_object9687/head//2 by client.2191635.0:37284 2014-08-12 17:17:01.716646 2014-08-13 18:06:06.429340 7f5c973a57a0 20 read_log 194215'630 (194215'165) delete 9b7a8baa/benchmark_data_storage0111-ib.pocstor_48865_object9869/head//2 by client.2191635.0:37466 2014-08-12 17:17:01.771122 2014-08-13 18:06:06.429361 7f5c973a57a0 20 read_log 194215'631 (194215'166) delete 2b1b79ea/benchmark_data_storage0111-ib.pocstor_48865_object9880/head//2 by client.2191635.0:37477 2014-08-12 17:17:01.774093 2014-08-13 18:06:06.429390 7f5c973a57a0 20 read_log 194215'632 (194215'167) delete 21a23aea/benchmark_data_storage0111-ib.pocstor_48865_object9933/head//2 by client.2191635.0:37530 2014-08-12 17:17:01.789386 2014-08-13 18:06:06.429432 7f5c973a57a0 20 read_log 194215'633 (194215'168) delete b5c1d3aa/benchmark_data_storage0111-ib.pocstor_48865_object10008/head//2 by client.2191635.0:37605 2014-08-12 17:17:01.809652 2014-08-13 18:06:06.429476 7f5c973a57a0 20 read_log 194215'634 (194215'169) delete 87325caa/benchmark_data_storage0111-ib.pocstor_48865_object10072/head//2 by client.2191635.0:37669 2014-08-12 17:17:01.826989 2014-08-13 18:06:06.429525 7f5c973a57a0 20 read_log 194215'635 (194215'170) delete a5bef9aa/benchmark_data_storage0111-ib.pocstor_48865_object10123/head//2 by client.2191635.0:37720 2014-08-12 17:17:01.840452 2014-08-13 18:06:06.429553 7f5c973a57a0 20 read_log 194215'636 (194215'171) delete 839ef8aa/benchmark_data_storage0111-ib.pocstor_48865_object10189/head//2 by client.2191635.0:37786 2014-08-12 17:17:01.858320 2014-08-13 18:06:06.429581 7f5c973a57a0 20 read_log 194215'637 (194215'172) delete a72c32aa/benchmark_data_storage0111-ib.pocstor_48865_object10248/head//2 by client.2191635.0:37845 2014-08-12 17:17:01.875657 2014-08-13 18:06:06.429609 7f5c973a57a0 20 read_log 194215'638 (194215'173) delete 3759752a/benchmark_data_storage0111-ib.pocstor_48865_object10253/head//2 by client.2191635.0:37850 2014-08-12 17:17:01.877538 2014-08-13 18:06:06.429639 7f5c973a57a0 20 read_log 194215'639 (194215'174) delete 83b5eaa/benchmark_data_storage0111-ib.pocstor_48865_object10313/head//2 by client.2191635.0:37910 2014-08-12 17:17:01.895381 2014-08-13 18:06:06.429666 7f5c973a57a0 20 read_log 194215'640 (194215'175) delete 6a9613aa/benchmark_data_storage0111-ib.pocstor_48865_object10396/head//2 by client.2191635.0:37993 2014-08-12 17:17:01.921356 2014-08-13 18:06:06.429703 7f5c973a57a0 20 read_log 194215'641 (194215'176) delete b37aadaa/benchmark_data_storage0111-ib.pocstor_48865_object10453/head//2 by client.2191635.0:38050 2014-08-12 17:17:01.937071 2014-08-13 18:06:06.429767 7f5c973a57a0 20 read_log 194215'642 (194215'177) delete 292338aa/benchmark_data_storage0111-ib.pocstor_48865_object10482/head//2 by client.2191635.0:38079 2014-08-12 17:17:01.944349 2014-08-13 18:06:06.429799 7f5c973a57a0 20 read_log 194215'643 (194215'178) delete cb33af2a/benchmark_data_storage0111-ib.pocstor_48865_object10644/head//2 by client.2191635.0:38241 2014-08-12 17:17:01.994035 2014-08-13 18:06:06.429834 7f5c973a57a0 20 read_log 194215'644 (194215'179) delete 1d50792a/benchmark_data_storage0111-ib.pocstor_48865_object10690/head//2 by client.2191635.0:38287 2014-08-12 17:17:02.008780 2014-08-13 18:06:06.429862 7f5c973a57a0 20 read_log 194215'645 (194215'180) delete f9ca0fea/benchmark_data_storage0111-ib.pocstor_48865_object10730/head//2 by client.2191635.0:38327 2014-08-12 17:17:02.019746 2014-08-13 18:06:06.429885 7f5c973a57a0 20 read_log 194215'646 (194215'181) delete a596172a/benchmark_data_storage0111-ib.pocstor_48865_object10814/head//2 by client.2191635.0:38411 2014-08-12 17:17:02.046679 2014-08-13 18:06:06.429907 7f5c973a57a0 20 read_log 194215'647 (194215'182) delete 2e3f30aa/benchmark_data_storage0111-ib.pocstor_48865_object10886/head//2 by client.2191635.0:38483 2014-08-12 17:17:02.071115 2014-08-13 18:06:06.429928 7f5c973a57a0 20 read_log 194215'648 (194215'183) delete 3359f32a/benchmark_data_storage0111-ib.pocstor_48865_object10910/head//2 by client.2191635.0:38507 2014-08-12 17:17:02.077350 2014-08-13 18:06:06.429950 7f5c973a57a0 20 read_log 194215'649 (194215'184) delete d5ffa2a/benchmark_data_storage0111-ib.pocstor_48865_object10933/head//2 by client.2191635.0:38530 2014-08-12 17:17:02.085659 2014-08-13 18:06:06.429973 7f5c973a57a0 20 read_log 194215'650 (194215'185) delete 5db5596a/benchmark_data_storage0111-ib.pocstor_48865_object10936/head//2 by client.2191635.0:38533 2014-08-12 17:17:02.086218 2014-08-13 18:06:06.429995 7f5c973a57a0 20 read_log 194215'651 (194215'186) delete aa36ecaa/benchmark_data_storage0111-ib.pocstor_48865_object10979/head//2 by client.2191635.0:38576 2014-08-12 17:17:02.098229 2014-08-13 18:06:06.430019 7f5c973a57a0 20 read_log 194215'652 (194215'187) delete 7039232a/benchmark_data_storage0111-ib.pocstor_48865_object11094/head//2 by client.2191635.0:38691 2014-08-12 17:17:02.131366 2014-08-13 18:06:06.430041 7f5c973a57a0 20 read_log 194215'653 (194215'188) delete 5b3c0d2a/benchmark_data_storage0111-ib.pocstor_48865_object11111/head//2 by client.2191635.0:38708 2014-08-12 17:17:02.136343 2014-08-13 18:06:06.430063 7f5c973a57a0 20 read_log 194215'654 (194215'189) delete 3f026e2a/benchmark_data_storage0111-ib.pocstor_48865_object11117/head//2 by client.2191635.0:38714 2014-08-12 17:17:02.137690 2014-08-13 18:06:06.430097 7f5c973a57a0 20 read_log 194215'655 (194215'190) delete 6072102a/benchmark_data_storage0111-ib.pocstor_48865_object11176/head//2 by client.2191635.0:38773 2014-08-12 17:17:02.156822 2014-08-13 18:06:06.430124 7f5c973a57a0 20 read_log 194215'656 (194215'191) delete b6ed77aa/benchmark_data_storage0111-ib.pocstor_48865_object11187/head//2 by client.2191635.0:38784 2014-08-12 17:17:02.160391 2014-08-13 18:06:06.430153 7f5c973a57a0 20 read_log 194215'657 (194215'192) delete 205c17aa/benchmark_data_storage0111-ib.pocstor_48865_object11236/head//2 by client.2191635.0:38833 2014-08-12 17:17:02.178340 2014-08-13 18:06:06.430186 7f5c973a57a0 20 read_log 194215'658 (194215'193) delete 8d426cea/benchmark_data_storage0111-ib.pocstor_48865_object11349/head//2 by client.2191635.0:38946 2014-08-12 17:17:02.211427 2014-08-13 18:06:06.430208 7f5c973a57a0 20 read_log 194215'659 (194215'194) delete d297416a/benchmark_data_storage0111-ib.pocstor_48865_object11397/head//2 by client.2191635.0:38994 2014-08-12 17:17:02.230013 2014-08-13 18:06:06.430230 7f5c973a57a0 20 read_log 194215'660 (194215'195) delete d6cefa2a/benchmark_data_storage0111-ib.pocstor_48865_object11498/head//2 by client.2191635.0:39095 2014-08-12 17:17:02.263973 2014-08-13 18:06:06.430252 7f5c973a57a0 20 read_log 194215'661 (194215'196) delete 53e842ea/benchmark_data_storage0111-ib.pocstor_48865_object11552/head//2 by client.2191635.0:39149 2014-08-12 17:17:02.280025 2014-08-13 18:06:06.430287 7f5c973a57a0 20 read_log 194215'662 (194215'197) delete 6030336a/benchmark_data_storage0111-ib.pocstor_48865_object11556/head//2 by client.2191635.0:39153 2014-08-12 17:17:02.282002 2014-08-13 18:06:06.430310 7f5c973a57a0 20 read_log 194215'663 (194215'198) delete 89d46bea/benchmark_data_storage0111-ib.pocstor_48865_object11776/head//2 by client.2191635.0:39373 2014-08-12 17:17:02.344881 2014-08-13 18:06:06.430342 7f5c973a57a0 20 read_log 194215'664 (194215'199) delete 35ffcaa/benchmark_data_storage0111-ib.pocstor_48865_object11861/head//2 by client.2191635.0:39458 2014-08-12 17:17:02.367192 2014-08-13 18:06:06.430369 7f5c973a57a0 20 read_log 194215'665 (194215'200) delete 5b650a2a/benchmark_data_storage0111-ib.pocstor_48865_object11920/head//2 by client.2191635.0:39517 2014-08-12 17:17:02.384450 2014-08-13 18:06:06.430408 7f5c973a57a0 20 read_log 194215'666 (194215'201) delete 4a1b016a/benchmark_data_storage0111-ib.pocstor_48865_object12014/head//2 by client.2191635.0:39611 2014-08-12 17:17:02.409950 2014-08-13 18:06:06.430452 7f5c973a57a0 20 read_log 194215'667 (194215'202) delete 6c2c76aa/benchmark_data_storage0111-ib.pocstor_48865_object12017/head//2 by client.2191635.0:39614 2014-08-12 17:17:02.411267 2014-08-13 18:06:06.430480 7f5c973a57a0 20 read_log 194215'668 (194215'203) delete b1b052a/benchmark_data_storage0111-ib.pocstor_48865_object12039/head//2 by client.2191635.0:39636 2014-08-12 17:17:02.417871 2014-08-13 18:06:06.430508 7f5c973a57a0 20 read_log 194215'669 (194215'204) delete 18d92c2a/benchmark_data_storage0111-ib.pocstor_48865_object12137/head//2 by client.2191635.0:39734 2014-08-12 17:17:02.447524 2014-08-13 18:06:06.430535 7f5c973a57a0 20 read_log 194215'670 (194215'205) delete e293eb6a/benchmark_data_storage0111-ib.pocstor_48865_object12200/head//2 by client.2191635.0:39797 2014-08-12 17:17:02.466436 2014-08-13 18:06:06.430561 7f5c973a57a0 20 read_log 194215'671 (194215'206) delete aec82caa/benchmark_data_storage0111-ib.pocstor_48865_object12205/head//2 by client.2191635.0:39802 2014-08-12 17:17:02.467324 2014-08-13 18:06:06.430584 7f5c973a57a0 20 read_log 194215'672 (194215'207) delete e4c381ea/benchmark_data_storage0111-ib.pocstor_48865_object12239/head//2 by client.2191635.0:39836 2014-08-12 17:17:02.480375 2014-08-13 18:06:06.430610 7f5c973a57a0 20 read_log 194215'673 (194215'208) delete 5597feaa/benchmark_data_storage0111-ib.pocstor_48865_object12386/head//2 by client.2191635.0:39983 2014-08-12 17:17:02.527260 2014-08-13 18:06:06.430632 7f5c973a57a0 20 read_log 194215'674 (194215'209) delete 43f2212a/benchmark_data_storage0111-ib.pocstor_48865_object12393/head//2 by client.2191635.0:39990 2014-08-12 17:17:02.528065 2014-08-13 18:06:06.430693 7f5c973a57a0 20 read_log 194215'675 (194215'210) delete 2b5bb7ea/benchmark_data_storage0111-ib.pocstor_48865_object12418/head//2 by client.2191635.0:40015 2014-08-12 17:17:02.538756 2014-08-13 18:06:06.430727 7f5c973a57a0 20 read_log 194215'676 (194215'211) delete b49f39aa/benchmark_data_storage0111-ib.pocstor_48865_object12547/head//2 by client.2191635.0:40144 2014-08-12 17:17:02.575749 2014-08-13 18:06:06.430750 7f5c973a57a0 20 read_log 194215'677 (194215'212) delete affb68aa/benchmark_data_storage0111-ib.pocstor_48865_object12599/head//2 by client.2191635.0:40196 2014-08-12 17:17:02.594413 2014-08-13 18:06:06.430785 7f5c973a57a0 20 read_log 194215'678 (194215'213) delete 2e68e72a/benchmark_data_storage0111-ib.pocstor_48865_object12623/head//2 by client.2191635.0:40220 2014-08-12 17:17:02.602057 2014-08-13 18:06:06.430813 7f5c973a57a0 20 read_log 194215'679 (194215'214) delete b033de6a/benchmark_data_storage0111-ib.pocstor_48865_object12628/head//2 by client.2191635.0:40225 2014-08-12 17:17:02.602276 2014-08-13 18:06:06.430836 7f5c973a57a0 20 read_log 194215'680 (194215'215) delete ae98b7ea/benchmark_data_storage0111-ib.pocstor_48865_object12659/head//2 by client.2191635.0:40256 2014-08-12 17:17:02.612988 2014-08-13 18:06:06.430858 7f5c973a57a0 20 read_log 194215'681 (194215'216) delete 2aaf702a/benchmark_data_storage0111-ib.pocstor_48865_object12668/head//2 by client.2191635.0:40265 2014-08-12 17:17:02.615687 2014-08-13 18:06:06.430879 7f5c973a57a0 20 read_log 194215'682 (194215'217) delete f57663aa/benchmark_data_storage0111-ib.pocstor_48865_object12750/head//2 by client.2191635.0:40347 2014-08-12 17:17:02.642122 2014-08-13 18:06:06.430901 7f5c973a57a0 20 read_log 194215'683 (194215'218) delete f4212caa/benchmark_data_storage0111-ib.pocstor_48865_object12838/head//2 by client.2191635.0:40435 2014-08-12 17:17:02.666168 2014-08-13 18:06:06.430922 7f5c973a57a0 20 read_log 194215'684 (194215'219) delete 7240e7aa/benchmark_data_storage0111-ib.pocstor_48865_object12999/head//2 by client.2191635.0:40596 2014-08-12 17:17:02.718920 2014-08-13 18:06:06.430944 7f5c973a57a0 20 read_log 194215'685 (194215'220) delete 667d846a/benchmark_data_storage0111-ib.pocstor_48865_object13090/head//2 by client.2191635.0:40687 2014-08-12 17:17:02.746435 2014-08-13 18:06:06.430965 7f5c973a57a0 20 read_log 194215'686 (194215'221) delete 4728a26a/benchmark_data_storage0111-ib.pocstor_48865_object13104/head//2 by client.2191635.0:40701 2014-08-12 17:17:02.750471 2014-08-13 18:06:06.430987 7f5c973a57a0 20 read_log 194215'687 (194215'222) delete 6830202a/benchmark_data_storage0111-ib.pocstor_48865_object13121/head//2 by client.2191635.0:40718 2014-08-12 17:17:02.755839 2014-08-13 18:06:06.431015 7f5c973a57a0 20 read_log 194215'688 (194215'223) delete e698f6ea/benchmark_data_storage0111-ib.pocstor_48865_object13189/head//2 by client.2191635.0:40786 2014-08-12 17:17:02.774956 2014-08-13 18:06:06.431038 7f5c973a57a0 20 read_log 194215'689 (194215'224) delete 1d7d93ea/benchmark_data_storage0111-ib.pocstor_48865_object13262/head//2 by client.2191635.0:40859 2014-08-12 17:17:02.798534 2014-08-13 18:06:06.431060 7f5c973a57a0 20 read_log 194215'690 (194215'225) delete 4a9e062a/benchmark_data_storage0111-ib.pocstor_48865_object13318/head//2 by client.2191635.0:40915 2014-08-12 17:17:02.815048 2014-08-13 18:06:06.431082 7f5c973a57a0 20 read_log 194215'691 (194215'226) delete 71b2d72a/benchmark_data_storage0111-ib.pocstor_48865_object13371/head//2 by client.2191635.0:40968 2014-08-12 17:17:02.830144 2014-08-13 18:06:06.431103 7f5c973a57a0 20 read_log 194215'692 (194215'227) delete 4abb7a2a/benchmark_data_storage0111-ib.pocstor_48865_object13374/head//2 by client.2191635.0:40971 2014-08-12 17:17:02.830829 2014-08-13 18:06:06.431125 7f5c973a57a0 20 read_log 194215'693 (194215'228) delete 8e43f9aa/benchmark_data_storage0111-ib.pocstor_48865_object13392/head//2 by client.2191635.0:40989 2014-08-12 17:17:02.835994 2014-08-13 18:06:06.431160 7f5c973a57a0 20 read_log 194215'694 (194215'229) delete 4f37bbea/benchmark_data_storage0111-ib.pocstor_48865_object13561/head//2 by client.2191635.0:41158 2014-08-12 17:17:02.886568 2014-08-13 18:06:06.431227 7f5c973a57a0 20 read_log 194215'695 (194215'230) delete 19487bea/benchmark_data_storage0111-ib.pocstor_48865_object13635/head//2 by client.2191635.0:41232 2014-08-12 17:17:02.907478 2014-08-13 18:06:06.431257 7f5c973a57a0 20 read_log 194215'696 (194215'231) delete dd220cea/benchmark_data_storage0111-ib.pocstor_48865_object13725/head//2 by client.2191635.0:41322 2014-08-12 17:17:02.933577 2014-08-13 18:06:06.431284 7f5c973a57a0 20 read_log 194215'697 (194215'232) delete 9440cd2a/benchmark_data_storage0111-ib.pocstor_48865_object13735/head//2 by client.2191635.0:41332 2014-08-12 17:17:02.936490 2014-08-13 18:06:06.431310 7f5c973a57a0 20 read_log 194215'698 (194215'233) delete d720b2a/benchmark_data_storage0111-ib.pocstor_48865_object13783/head//2 by client.2191635.0:41380 2014-08-12 17:17:02.951374 2014-08-13 18:06:06.431337 7f5c973a57a0 20 read_log 194215'699 (194215'234) delete c9345c2a/benchmark_data_storage0111-ib.pocstor_48865_object13852/head//2 by client.2191635.0:41449 2014-08-12 17:17:02.972106 2014-08-13 18:06:06.431363 7f5c973a57a0 20 read_log 194215'700 (194215'235) delete ea0b76a/benchmark_data_storage0111-ib.pocstor_48865_object13973/head//2 by client.2191635.0:41570 2014-08-12 17:17:03.012603 2014-08-13 18:06:06.431390 7f5c973a57a0 20 read_log 194215'701 (194215'236) delete 2a0c802a/benchmark_data_storage0111-ib.pocstor_48865_object14145/head//2 by client.2191635.0:41742 2014-08-12 17:17:03.062495 2014-08-13 18:06:06.431415 7f5c973a57a0 20 read_log 194215'702 (194215'237) delete 63208a6a/benchmark_data_storage0111-ib.pocstor_48865_object14286/head//2 by client.2191635.0:41883 2014-08-12 17:17:03.102690 2014-08-13 18:06:06.431461 7f5c973a57a0 20 read_log 194215'703 (194215'238) delete 40f6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object14317/head//2 by client.2191635.0:41914 2014-08-12 17:17:03.112314 2014-08-13 18:06:06.431522 7f5c973a57a0 20 read_log 194215'704 (194215'239) delete 41de21ea/benchmark_data_storage0111-ib.pocstor_48865_object14598/head//2 by client.2191635.0:42195 2014-08-12 17:17:03.199725 2014-08-13 18:06:06.431566 7f5c973a57a0 20 read_log 194215'705 (194215'240) delete b8a8542a/benchmark_data_storage0111-ib.pocstor_48865_object14672/head//2 by client.2191635.0:42269 2014-08-12 17:17:03.220543 2014-08-13 18:06:06.431598 7f5c973a57a0 20 read_log 194215'706 (194215'241) delete 4a6306aa/benchmark_data_storage0111-ib.pocstor_48865_object14713/head//2 by client.2191635.0:42310 2014-08-12 17:17:03.231459 2014-08-13 18:06:06.431624 7f5c973a57a0 20 read_log 194215'707 (194215'242) delete 2991aaaa/benchmark_data_storage0111-ib.pocstor_48865_object14741/head//2 by client.2191635.0:42338 2014-08-12 17:17:03.239683 2014-08-13 18:06:06.431646 7f5c973a57a0 20 read_log 194215'708 (194215'243) delete 565f662a/benchmark_data_storage0111-ib.pocstor_48865_object14767/head//2 by client.2191635.0:42364 2014-08-12 17:17:03.248980 2014-08-13 18:06:06.431668 7f5c973a57a0 20 read_log 194215'709 (194215'244) delete 5192f8aa/benchmark_data_storage0111-ib.pocstor_48865_object14776/head//2 by client.2191635.0:42373 2014-08-12 17:17:03.250696 2014-08-13 18:06:06.431692 7f5c973a57a0 20 read_log 194215'710 (194215'245) delete eeea566a/benchmark_data_storage0111-ib.pocstor_48865_object14778/head//2 by client.2191635.0:42375 2014-08-12 17:17:03.251470 2014-08-13 18:06:06.431713 7f5c973a57a0 20 read_log 194215'711 (194215'246) delete 8558d5aa/benchmark_data_storage0111-ib.pocstor_48865_object14801/head//2 by client.2191635.0:42398 2014-08-12 17:17:03.258463 2014-08-13 18:06:06.431735 7f5c973a57a0 20 read_log 194215'712 (194215'247) delete 9d0eaaea/benchmark_data_storage0111-ib.pocstor_48865_object14866/head//2 by client.2191635.0:42463 2014-08-12 17:17:03.278564 2014-08-13 18:06:06.431781 7f5c973a57a0 20 read_log 194215'713 (194215'248) delete 305b2a6a/benchmark_data_storage0111-ib.pocstor_48865_object14950/head//2 by client.2191635.0:42547 2014-08-12 17:17:03.304029 2014-08-13 18:06:06.431822 7f5c973a57a0 20 read_log 194215'714 (194215'249) delete 731b0e6a/benchmark_data_storage0111-ib.pocstor_48865_object15136/head//2 by client.2191635.0:42733 2014-08-12 17:17:03.358968 2014-08-13 18:06:06.431857 7f5c973a57a0 20 read_log 194215'715 (194215'250) delete 65f3ae6a/benchmark_data_storage0111-ib.pocstor_48865_object15161/head//2 by client.2191635.0:42758 2014-08-12 17:17:03.364942 2014-08-13 18:06:06.431887 7f5c973a57a0 20 read_log 194215'716 (194215'251) delete 77250b2a/benchmark_data_storage0111-ib.pocstor_48865_object15203/head//2 by client.2191635.0:42800 2014-08-12 17:17:03.376741 2014-08-13 18:06:06.431950 7f5c973a57a0 20 read_log 194215'717 (194215'252) delete daba3e6a/benchmark_data_storage0111-ib.pocstor_48865_object15306/head//2 by client.2191635.0:42903 2014-08-12 17:17:03.409742 2014-08-13 18:06:06.431977 7f5c973a57a0 20 read_log 194215'718 (194215'253) delete 8338bcea/benchmark_data_storage0111-ib.pocstor_48865_object15415/head//2 by client.2191635.0:43012 2014-08-12 17:17:03.442888 2014-08-13 18:06:06.432017 7f5c973a57a0 20 read_log 194215'719 (194215'254) delete 79bf0b6a/benchmark_data_storage0111-ib.pocstor_48865_object15441/head//2 by client.2191635.0:43038 2014-08-12 17:17:03.451008 2014-08-13 18:06:06.432046 7f5c973a57a0 20 read_log 194215'720 (194215'255) delete 22e21cea/benchmark_data_storage0111-ib.pocstor_48865_object15451/head//2 by client.2191635.0:43048 2014-08-12 17:17:03.454459 2014-08-13 18:06:06.432068 7f5c973a57a0 20 read_log 194215'721 (194215'256) delete 722c47ea/benchmark_data_storage0111-ib.pocstor_48865_object15566/head//2 by client.2191635.0:43163 2014-08-12 17:17:03.487870 2014-08-13 18:06:06.432106 7f5c973a57a0 20 read_log 194215'722 (194215'257) delete a865f92a/benchmark_data_storage0111-ib.pocstor_48865_object15671/head//2 by client.2191635.0:43268 2014-08-12 17:17:03.522171 2014-08-13 18:06:06.432129 7f5c973a57a0 20 read_log 194215'723 (194215'258) delete eb9d4f6a/benchmark_data_storage0111-ib.pocstor_48865_object15686/head//2 by client.2191635.0:43283 2014-08-12 17:17:03.526607 2014-08-13 18:06:06.432159 7f5c973a57a0 20 read_log 194215'724 (194215'259) delete bfafc2a/benchmark_data_storage0111-ib.pocstor_48865_object15765/head//2 by client.2191635.0:43362 2014-08-12 17:17:03.549829 2014-08-13 18:06:06.432182 7f5c973a57a0 20 read_log 194215'725 (194215'260) delete c4918aea/benchmark_data_storage0111-ib.pocstor_48865_object15774/head//2 by client.2191635.0:43371 2014-08-12 17:17:03.553490 2014-08-13 18:06:06.432203 7f5c973a57a0 20 read_log 194215'726 (194215'261) delete 577cf22a/benchmark_data_storage0111-ib.pocstor_48865_object15886/head//2 by client.2191635.0:43483 2014-08-12 17:17:03.586559 2014-08-13 18:06:06.432229 7f5c973a57a0 20 read_log 194215'727 (194215'262) delete 47b95a6a/benchmark_data_storage0111-ib.pocstor_48865_object15917/head//2 by client.2191635.0:43514 2014-08-12 17:17:03.595653 2014-08-13 18:06:06.432251 7f5c973a57a0 20 read_log 194215'728 (194215'263) delete d8a6b6ea/benchmark_data_storage0111-ib.pocstor_48865_object15948/head//2 by client.2191635.0:43545 2014-08-12 17:17:03.605682 2014-08-13 18:06:06.432273 7f5c973a57a0 20 read_log 194215'729 (194215'264) delete b36d1d6a/benchmark_data_storage0111-ib.pocstor_48865_object15982/head//2 by client.2191635.0:43579 2014-08-12 17:17:03.615985 2014-08-13 18:06:06.432294 7f5c973a57a0 20 read_log 194215'730 (194215'265) delete 45f476ea/benchmark_data_storage0111-ib.pocstor_48865_object16063/head//2 by client.2191635.0:43660 2014-08-12 17:17:03.638650 2014-08-13 18:06:06.432315 7f5c973a57a0 20 read_log 194215'731 (194215'266) delete c3d0156a/benchmark_data_storage0111-ib.pocstor_48865_object16141/head//2 by client.2191635.0:43738 2014-08-12 17:17:03.660512 2014-08-13 18:06:06.432337 7f5c973a57a0 20 read_log 194215'732 (194215'267) delete 81a3d32a/benchmark_data_storage0111-ib.pocstor_48865_object16150/head//2 by client.2191635.0:43747 2014-08-12 17:17:03.663635 2014-08-13 18:06:06.432363 7f5c973a57a0 20 read_log 194215'733 (194215'268) delete 8c0d182a/benchmark_data_storage0111-ib.pocstor_48865_object16296/head//2 by client.2191635.0:43893 2014-08-12 17:17:03.707017 2014-08-13 18:06:06.432385 7f5c973a57a0 20 read_log 194215'734 (194215'269) delete a521d6aa/benchmark_data_storage0111-ib.pocstor_48865_object16315/head//2 by client.2191635.0:43912 2014-08-12 17:17:03.712091 2014-08-13 18:06:06.432428 7f5c973a57a0 20 read_log 194215'735 (194215'270) delete df2ea2ea/benchmark_data_storage0111-ib.pocstor_48865_object16352/head//2 by client.2191635.0:43949 2014-08-12 17:17:03.724378 2014-08-13 18:06:06.432456 7f5c973a57a0 20 read_log 194215'736 (194215'271) delete 826376aa/benchmark_data_storage0111-ib.pocstor_48865_object16361/head//2 by client.2191635.0:43958 2014-08-12 17:17:03.725546 2014-08-13 18:06:06.432497 7f5c973a57a0 20 read_log 194215'737 (194215'272) delete 6afdb52a/benchmark_data_storage0111-ib.pocstor_48865_object16427/head//2 by client.2191635.0:44024 2014-08-12 17:17:03.744165 2014-08-13 18:06:06.432540 7f5c973a57a0 20 read_log 194215'738 (194215'273) delete bc7adeaa/benchmark_data_storage0111-ib.pocstor_48865_object16515/head//2 by client.2191635.0:44112 2014-08-12 17:17:03.770449 2014-08-13 18:06:06.432569 7f5c973a57a0 20 read_log 194215'739 (194215'274) delete ccb6aa2a/benchmark_data_storage0111-ib.pocstor_48865_object16564/head//2 by client.2191635.0:44161 2014-08-12 17:17:03.783113 2014-08-13 18:06:06.432591 7f5c973a57a0 20 read_log 194215'740 (194215'275) delete 7f37d7aa/benchmark_data_storage0111-ib.pocstor_48865_object16565/head//2 by client.2191635.0:44162 2014-08-12 17:17:03.783180 2014-08-13 18:06:06.432623 7f5c973a57a0 20 read_log 194215'741 (194215'276) delete 8cf4382a/benchmark_data_storage0111-ib.pocstor_48865_object16604/head//2 by client.2191635.0:44201 2014-08-12 17:17:03.794113 2014-08-13 18:06:06.432645 7f5c973a57a0 20 read_log 194215'742 (194215'277) delete fbdbd5aa/benchmark_data_storage0111-ib.pocstor_48865_object16791/head//2 by client.2191635.0:44388 2014-08-12 17:17:03.848527 2014-08-13 18:06:06.432672 7f5c973a57a0 20 read_log 194215'743 (194215'278) delete 46242aea/benchmark_data_storage0111-ib.pocstor_48865_object16862/head//2 by client.2191635.0:44459 2014-08-12 17:17:03.870692 2014-08-13 18:06:06.432699 7f5c973a57a0 20 read_log 194215'744 (194215'279) delete 8e0b2d2a/benchmark_data_storage0111-ib.pocstor_48865_object16863/head//2 by client.2191635.0:44460 2014-08-12 17:17:03.871189 2014-08-13 18:06:06.432729 7f5c973a57a0 20 read_log 194215'745 (194215'280) delete d8478eea/benchmark_data_storage0111-ib.pocstor_48865_object16870/head//2 by client.2191635.0:44467 2014-08-12 17:17:03.872842 2014-08-13 18:06:06.432770 7f5c973a57a0 20 read_log 194215'746 (194215'281) delete 23a472a/benchmark_data_storage0111-ib.pocstor_48865_object16907/head//2 by client.2191635.0:44504 2014-08-12 17:17:03.882741 2014-08-13 18:06:06.432798 7f5c973a57a0 20 read_log 194215'747 (194215'282) delete 861c596a/benchmark_data_storage0111-ib.pocstor_48865_object16946/head//2 by client.2191635.0:44543 2014-08-12 17:17:03.895417 2014-08-13 18:06:06.432825 7f5c973a57a0 20 read_log 194215'748 (194215'283) delete 70e3cf2a/benchmark_data_storage0111-ib.pocstor_48865_object16999/head//2 by client.2191635.0:44596 2014-08-12 17:17:03.910478 2014-08-13 18:06:06.432847 7f5c973a57a0 20 read_log 194215'749 (194215'284) delete 66eb7d2a/benchmark_data_storage0111-ib.pocstor_48865_object17022/head//2 by client.2191635.0:44619 2014-08-12 17:17:03.918986 2014-08-13 18:06:06.432930 7f5c973a57a0 20 read_log 194215'750 (194215'285) delete a97b1aa/benchmark_data_storage0111-ib.pocstor_48865_object17041/head//2 by client.2191635.0:44638 2014-08-12 17:17:03.923117 2014-08-13 18:06:06.432957 7f5c973a57a0 20 read_log 194215'751 (194215'286) delete 36b9c7ea/benchmark_data_storage0111-ib.pocstor_48865_object17067/head//2 by client.2191635.0:44664 2014-08-12 17:17:03.931740 2014-08-13 18:06:06.432982 7f5c973a57a0 20 read_log 194215'752 (194215'287) delete d2246d2a/benchmark_data_storage0111-ib.pocstor_48865_object17159/head//2 by client.2191635.0:44756 2014-08-12 17:17:03.959638 2014-08-13 18:06:06.433021 7f5c973a57a0 20 read_log 194215'753 (194215'288) delete 9e6f64ea/benchmark_data_storage0111-ib.pocstor_48865_object17166/head//2 by client.2191635.0:44763 2014-08-12 17:17:03.960583 2014-08-13 18:06:06.433044 7f5c973a57a0 20 read_log 194215'754 (194215'289) delete 1ce7ab2a/benchmark_data_storage0111-ib.pocstor_48865_object17300/head//2 by client.2191635.0:44897 2014-08-12 17:17:04.000255 2014-08-13 18:06:06.433079 7f5c973a57a0 20 read_log 194215'755 (194215'290) delete 93cde52a/benchmark_data_storage0111-ib.pocstor_48865_object17356/head//2 by client.2191635.0:44953 2014-08-12 17:17:04.017759 2014-08-13 18:06:06.433105 7f5c973a57a0 20 read_log 194215'756 (194215'291) delete 6543b36a/benchmark_data_storage0111-ib.pocstor_48865_object17395/head//2 by client.2191635.0:44992 2014-08-12 17:17:04.027964 2014-08-13 18:06:06.433128 7f5c973a57a0 20 read_log 194215'757 (194215'292) delete 907fe66a/benchmark_data_storage0111-ib.pocstor_48865_object17420/head//2 by client.2191635.0:45017 2014-08-12 17:17:04.034809 2014-08-13 18:06:06.433158 7f5c973a57a0 20 read_log 194215'758 (194215'293) delete f04168aa/benchmark_data_storage0111-ib.pocstor_48865_object17493/head//2 by client.2191635.0:45090 2014-08-12 17:17:04.058589 2014-08-13 18:06:06.433181 7f5c973a57a0 20 read_log 194215'759 (194215'294) delete 3dd28a2a/benchmark_data_storage0111-ib.pocstor_48865_object17533/head//2 by client.2191635.0:45130 2014-08-12 17:17:04.070768 2014-08-13 18:06:06.433203 7f5c973a57a0 20 read_log 194215'760 (194215'295) delete 1e4657ea/benchmark_data_storage0111-ib.pocstor_48865_object17612/head//2 by client.2191635.0:45209 2014-08-12 17:17:04.098224 2014-08-13 18:06:06.433224 7f5c973a57a0 20 read_log 194215'761 (194215'296) delete 6041b6ea/benchmark_data_storage0111-ib.pocstor_48865_object17635/head//2 by client.2191635.0:45232 2014-08-12 17:17:04.107833 2014-08-13 18:06:06.433246 7f5c973a57a0 20 read_log 194215'762 (194215'297) delete 224e726a/benchmark_data_storage0111-ib.pocstor_48865_object17676/head//2 by client.2191635.0:45273 2014-08-12 17:17:04.120504 2014-08-13 18:06:06.433267 7f5c973a57a0 20 read_log 194215'763 (194215'298) delete b888422a/benchmark_data_storage0111-ib.pocstor_48865_object17741/head//2 by client.2191635.0:45338 2014-08-12 17:17:04.141765 2014-08-13 18:06:06.433288 7f5c973a57a0 20 read_log 194215'764 (194215'299) delete 6161082a/benchmark_data_storage0111-ib.pocstor_48865_object17851/head//2 by client.2191635.0:45448 2014-08-12 17:17:04.171589 2014-08-13 18:06:06.433315 7f5c973a57a0 20 read_log 194215'765 (194215'300) delete f91dcc2a/benchmark_data_storage0111-ib.pocstor_48865_object18013/head//2 by client.2191635.0:45610 2014-08-12 17:17:04.224937 2014-08-13 18:06:06.433342 7f5c973a57a0 20 read_log 194215'766 (194215'301) delete d714946a/benchmark_data_storage0111-ib.pocstor_48865_object18071/head//2 by client.2191635.0:45668 2014-08-12 17:17:04.243752 2014-08-13 18:06:06.433368 7f5c973a57a0 20 read_log 194215'767 (194215'302) delete 76da69aa/benchmark_data_storage0111-ib.pocstor_48865_object18175/head//2 by client.2191635.0:45772 2014-08-12 17:17:04.276736 2014-08-13 18:06:06.433395 7f5c973a57a0 20 read_log 194215'768 (194215'303) delete ca0bf7ea/benchmark_data_storage0111-ib.pocstor_48865_object18353/head//2 by client.2191635.0:45950 2014-08-12 17:17:04.334746 2014-08-13 18:06:06.433417 7f5c973a57a0 20 read_log 194215'769 (194215'304) delete cb5fc6ea/benchmark_data_storage0111-ib.pocstor_48865_object18362/head//2 by client.2191635.0:45959 2014-08-12 17:17:04.338043 2014-08-13 18:06:06.433452 7f5c973a57a0 20 read_log 194215'770 (194215'305) delete ba58eaaa/benchmark_data_storage0111-ib.pocstor_48865_object18438/head//2 by client.2191635.0:46035 2014-08-12 17:17:04.363405 2014-08-13 18:06:06.433485 7f5c973a57a0 20 read_log 194215'771 (194215'306) delete 774fb76a/benchmark_data_storage0111-ib.pocstor_48865_object18602/head//2 by client.2191635.0:46199 2014-08-12 17:17:04.414922 2014-08-13 18:06:06.433508 7f5c973a57a0 20 read_log 194215'772 (194215'307) delete 3f75e6a/benchmark_data_storage0111-ib.pocstor_48865_object18627/head//2 by client.2191635.0:46224 2014-08-12 17:17:04.423816 2014-08-13 18:06:06.433548 7f5c973a57a0 20 read_log 194215'773 (194215'308) delete 76eaf2a/benchmark_data_storage0111-ib.pocstor_48865_object18644/head//2 by client.2191635.0:46241 2014-08-12 17:17:04.428065 2014-08-13 18:06:06.433582 7f5c973a57a0 20 read_log 194215'774 (194215'309) delete 5bc40b6a/benchmark_data_storage0111-ib.pocstor_48865_object18652/head//2 by client.2191635.0:46249 2014-08-12 17:17:04.430713 2014-08-13 18:06:06.433627 7f5c973a57a0 20 read_log 194215'775 (194215'310) delete 9c195bea/benchmark_data_storage0111-ib.pocstor_48865_object18750/head//2 by client.2191635.0:46347 2014-08-12 17:17:04.463048 2014-08-13 18:06:06.433655 7f5c973a57a0 20 read_log 194215'776 (194215'311) delete bcd1ddea/benchmark_data_storage0111-ib.pocstor_48865_object18803/head//2 by client.2191635.0:46400 2014-08-12 17:17:04.477059 2014-08-13 18:06:06.433702 7f5c973a57a0 20 read_log 194215'777 (194215'312) delete 57e7232a/benchmark_data_storage0111-ib.pocstor_48865_object18851/head//2 by client.2191635.0:46448 2014-08-12 17:17:04.491213 2014-08-13 18:06:06.433729 7f5c973a57a0 20 read_log 194215'778 (194215'313) delete 3f8d12a/benchmark_data_storage0111-ib.pocstor_48865_object18898/head//2 by client.2191635.0:46495 2014-08-12 17:17:04.504495 2014-08-13 18:06:06.433755 7f5c973a57a0 20 read_log 194215'779 (194215'314) delete eda4702a/benchmark_data_storage0111-ib.pocstor_48865_object18934/head//2 by client.2191635.0:46531 2014-08-12 17:17:04.514927 2014-08-13 18:06:06.433782 7f5c973a57a0 20 read_log 194215'780 (194215'315) delete 8ad8c9ea/benchmark_data_storage0111-ib.pocstor_48865_object19074/head//2 by client.2191635.0:46671 2014-08-12 17:17:04.551775 2014-08-13 18:06:06.433822 7f5c973a57a0 20 read_log 194215'781 (194215'316) delete 45feb6aa/benchmark_data_storage0111-ib.pocstor_48865_object19194/head//2 by client.2191635.0:46791 2014-08-12 17:17:04.585919 2014-08-13 18:06:06.433850 7f5c973a57a0 20 read_log 194215'782 (194215'317) delete 67fe712a/benchmark_data_storage0111-ib.pocstor_48865_object19218/head//2 by client.2191635.0:46815 2014-08-12 17:17:04.592239 2014-08-13 18:06:06.433877 7f5c973a57a0 20 read_log 194215'783 (194215'318) delete 87a9a2ea/benchmark_data_storage0111-ib.pocstor_48865_object19230/head//2 by client.2191635.0:46827 2014-08-12 17:17:04.596308 2014-08-13 18:06:06.433916 7f5c973a57a0 20 read_log 194215'784 (194215'319) delete d4cfa56a/benchmark_data_storage0111-ib.pocstor_48865_object19259/head//2 by client.2191635.0:46856 2014-08-12 17:17:04.604276 2014-08-13 18:06:06.433939 7f5c973a57a0 20 read_log 194215'785 (194215'320) delete 3d60586a/benchmark_data_storage0111-ib.pocstor_48865_object19315/head//2 by client.2191635.0:46912 2014-08-12 17:17:04.618845 2014-08-13 18:06:06.433961 7f5c973a57a0 20 read_log 194215'786 (194215'321) delete a5cf01aa/benchmark_data_storage0111-ib.pocstor_48865_object19413/head//2 by client.2191635.0:47010 2014-08-12 17:17:04.646090 2014-08-13 18:06:06.433993 7f5c973a57a0 20 read_log 194215'787 (194215'322) delete eac666ea/benchmark_data_storage0111-ib.pocstor_48865_object19418/head//2 by client.2191635.0:47015 2014-08-12 17:17:04.647628 2014-08-13 18:06:06.434022 7f5c973a57a0 20 read_log 194215'788 (194215'323) delete 701512a/benchmark_data_storage0111-ib.pocstor_48865_object19425/head//2 by client.2191635.0:47022 2014-08-12 17:17:04.649464 2014-08-13 18:06:06.434044 7f5c973a57a0 20 read_log 194215'789 (194215'324) delete 109f076a/benchmark_data_storage0111-ib.pocstor_48865_object19461/head//2 by client.2191635.0:47058 2014-08-12 17:17:04.660461 2014-08-13 18:06:06.434066 7f5c973a57a0 20 read_log 194215'790 (194215'325) delete b65a122a/benchmark_data_storage0111-ib.pocstor_48865_object19517/head//2 by client.2191635.0:47114 2014-08-12 17:17:04.678486 2014-08-13 18:06:06.434087 7f5c973a57a0 20 read_log 194215'791 (194215'326) delete bca3f02a/benchmark_data_storage0111-ib.pocstor_48865_object19526/head//2 by client.2191635.0:47123 2014-08-12 17:17:04.681640 2014-08-13 18:06:06.434109 7f5c973a57a0 20 read_log 194215'792 (194215'327) delete cf2e0e2a/benchmark_data_storage0111-ib.pocstor_48865_object19566/head//2 by client.2191635.0:47163 2014-08-12 17:17:04.693856 2014-08-13 18:06:06.434131 7f5c973a57a0 20 read_log 194215'793 (194215'328) delete 56fd8d6a/benchmark_data_storage0111-ib.pocstor_48865_object19582/head//2 by client.2191635.0:47179 2014-08-12 17:17:04.697927 2014-08-13 18:06:06.434161 7f5c973a57a0 20 read_log 194215'794 (194215'329) delete bea29c2a/benchmark_data_storage0111-ib.pocstor_48865_object19690/head//2 by client.2191635.0:47287 2014-08-12 17:17:04.730034 2014-08-13 18:06:06.434206 7f5c973a57a0 20 read_log 194215'795 (194215'330) delete 971f7a6a/benchmark_data_storage0111-ib.pocstor_48865_object19692/head//2 by client.2191635.0:47289 2014-08-12 17:17:04.730147 2014-08-13 18:06:06.434232 7f5c973a57a0 20 read_log 194215'796 (194215'331) delete 9a63936a/benchmark_data_storage0111-ib.pocstor_48865_object19899/head//2 by client.2191635.0:47496 2014-08-12 17:17:04.795100 2014-08-13 18:06:06.434285 7f5c973a57a0 20 read_log 194215'797 (194215'332) delete f7da2aea/benchmark_data_storage0111-ib.pocstor_48865_object19982/head//2 by client.2191635.0:47579 2014-08-12 17:17:04.823185 2014-08-13 18:06:06.434311 7f5c973a57a0 20 read_log 194215'798 (194215'333) delete 5bb981aa/benchmark_data_storage0111-ib.pocstor_48865_object20036/head//2 by client.2191635.0:47633 2014-08-12 17:17:04.838572 2014-08-13 18:06:06.434333 7f5c973a57a0 20 read_log 194215'799 (194215'334) delete cf11302a/benchmark_data_storage0111-ib.pocstor_48865_object20045/head//2 by client.2191635.0:47642 2014-08-12 17:17:04.841469 2014-08-13 18:06:06.434360 7f5c973a57a0 20 read_log 194215'800 (194215'335) delete fe4a30aa/benchmark_data_storage0111-ib.pocstor_48865_object20057/head//2 by client.2191635.0:47654 2014-08-12 17:17:04.844276 2014-08-13 18:06:06.434383 7f5c973a57a0 20 read_log 194215'801 (194215'336) delete 44997bea/benchmark_data_storage0111-ib.pocstor_48865_object20125/head//2 by client.2191635.0:47722 2014-08-12 17:17:04.860248 2014-08-13 18:06:06.434414 7f5c973a57a0 20 read_log 194215'802 (194215'337) delete c8c401aa/benchmark_data_storage0111-ib.pocstor_48865_object20134/head//2 by client.2191635.0:47731 2014-08-12 17:17:04.862174 2014-08-13 18:06:06.434440 7f5c973a57a0 20 read_log 194215'803 (194215'338) delete ae295caa/benchmark_data_storage0111-ib.pocstor_48865_object20201/head//2 by client.2191635.0:47798 2014-08-12 17:17:04.881751 2014-08-13 18:06:06.434467 7f5c973a57a0 20 read_log 194215'804 (194215'339) delete f203046a/benchmark_data_storage0111-ib.pocstor_48865_object20204/head//2 by client.2191635.0:47801 2014-08-12 17:17:04.882237 2014-08-13 18:06:06.434516 7f5c973a57a0 20 read_log 194215'805 (194215'340) delete de6e16aa/benchmark_data_storage0111-ib.pocstor_48865_object20326/head//2 by client.2191635.0:47923 2014-08-12 17:17:04.921246 2014-08-13 18:06:06.434555 7f5c973a57a0 20 read_log 194215'806 (194215'341) delete e793e8aa/benchmark_data_storage0111-ib.pocstor_48865_object20420/head//2 by client.2191635.0:48017 2014-08-12 17:17:04.948763 2014-08-13 18:06:06.434589 7f5c973a57a0 20 read_log 194215'807 (194215'342) delete 97c150ea/benchmark_data_storage0111-ib.pocstor_48865_object20422/head//2 by client.2191635.0:48019 2014-08-12 17:17:04.949670 2014-08-13 18:06:06.434613 7f5c973a57a0 20 read_log 194215'808 (194215'343) delete c581916a/benchmark_data_storage0111-ib.pocstor_48865_object20680/head//2 by client.2191635.0:48277 2014-08-12 17:17:05.023896 2014-08-13 18:06:06.434635 7f5c973a57a0 20 read_log 194215'809 (194215'344) delete 226308aa/benchmark_data_storage0111-ib.pocstor_48865_object20698/head//2 by client.2191635.0:48295 2014-08-12 17:17:05.028773 2014-08-13 18:06:06.434657 7f5c973a57a0 20 read_log 194215'810 (194215'345) delete 217b9c2a/benchmark_data_storage0111-ib.pocstor_48865_object20701/head//2 by client.2191635.0:48298 2014-08-12 17:17:05.029894 2014-08-13 18:06:06.434696 7f5c973a57a0 20 read_log 194215'811 (194215'346) delete d22aaeea/benchmark_data_storage0111-ib.pocstor_48865_object20724/head//2 by client.2191635.0:48321 2014-08-12 17:17:05.035948 2014-08-13 18:06:06.434719 7f5c973a57a0 20 read_log 194215'812 (194215'347) delete 3056b9aa/benchmark_data_storage0111-ib.pocstor_48865_object20781/head//2 by client.2191635.0:48378 2014-08-12 17:17:05.053283 2014-08-13 18:06:06.434741 7f5c973a57a0 20 read_log 194215'813 (194215'348) delete e128ee2a/benchmark_data_storage0111-ib.pocstor_48865_object20832/head//2 by client.2191635.0:48429 2014-08-12 17:17:05.068034 2014-08-13 18:06:06.434763 7f5c973a57a0 20 read_log 194215'814 (194215'349) delete a037016a/benchmark_data_storage0111-ib.pocstor_48865_object21051/head//2 by client.2191635.0:48648 2014-08-12 17:17:05.134132 2014-08-13 18:06:06.434796 7f5c973a57a0 20 read_log 194215'815 (194215'350) delete b10367ea/benchmark_data_storage0111-ib.pocstor_48865_object21068/head//2 by client.2191635.0:48665 2014-08-12 17:17:05.139039 2014-08-13 18:06:06.434822 7f5c973a57a0 20 read_log 194215'816 (194215'351) delete d46985aa/benchmark_data_storage0111-ib.pocstor_48865_object21162/head//2 by client.2191635.0:48759 2014-08-12 17:17:05.165129 2014-08-13 18:06:06.434844 7f5c973a57a0 20 read_log 194215'817 (194215'352) delete 971d17ea/benchmark_data_storage0111-ib.pocstor_48865_object21246/head//2 by client.2191635.0:48843 2014-08-12 17:17:05.190199 2014-08-13 18:06:06.434866 7f5c973a57a0 20 read_log 194215'818 (194215'353) delete 41d7a72a/benchmark_data_storage0111-ib.pocstor_48865_object21255/head//2 by client.2191635.0:48852 2014-08-12 17:17:05.192642 2014-08-13 18:06:06.434887 7f5c973a57a0 20 read_log 194215'819 (194215'354) delete 6adc306a/benchmark_data_storage0111-ib.pocstor_48865_object21329/head//2 by client.2191635.0:48926 2014-08-12 17:17:05.215068 2014-08-13 18:06:06.434909 7f5c973a57a0 20 read_log 194215'820 (194215'355) delete 19802aaa/benchmark_data_storage0111-ib.pocstor_48865_object21368/head//2 by client.2191635.0:48965 2014-08-12 17:17:05.226607 2014-08-13 18:06:06.434930 7f5c973a57a0 20 read_log 194215'821 (194215'356) delete 87842baa/benchmark_data_storage0111-ib.pocstor_48865_object21403/head//2 by client.2191635.0:49000 2014-08-12 17:17:05.237146 2014-08-13 18:06:06.434952 7f5c973a57a0 20 read_log 194215'822 (194215'357) delete b59226a/benchmark_data_storage0111-ib.pocstor_48865_object21404/head//2 by client.2191635.0:49001 2014-08-12 17:17:05.237302 2014-08-13 18:06:06.434973 7f5c973a57a0 20 read_log 194215'823 (194215'358) delete d68ecdaa/benchmark_data_storage0111-ib.pocstor_48865_object21520/head//2 by client.2191635.0:49117 2014-08-12 17:17:05.272097 2014-08-13 18:06:06.434995 7f5c973a57a0 20 read_log 194215'824 (194215'359) delete 37caa06a/benchmark_data_storage0111-ib.pocstor_48865_object21568/head//2 by client.2191635.0:49165 2014-08-12 17:17:05.286430 2014-08-13 18:06:06.435016 7f5c973a57a0 20 read_log 194215'825 (194215'360) delete 615151ea/benchmark_data_storage0111-ib.pocstor_48865_object21657/head//2 by client.2191635.0:49254 2014-08-12 17:17:05.310822 2014-08-13 18:06:06.435037 7f5c973a57a0 20 read_log 194215'826 (194215'361) delete 7abdc1ea/benchmark_data_storage0111-ib.pocstor_48865_object21668/head//2 by client.2191635.0:49265 2014-08-12 17:17:05.313722 2014-08-13 18:06:06.435059 7f5c973a57a0 20 read_log 194215'827 (194215'362) delete 97743f2a/benchmark_data_storage0111-ib.pocstor_48865_object21779/head//2 by client.2191635.0:49376 2014-08-12 17:17:05.347539 2014-08-13 18:06:06.435080 7f5c973a57a0 20 read_log 194215'828 (194215'363) delete 605fde2a/benchmark_data_storage0111-ib.pocstor_48865_object21799/head//2 by client.2191635.0:49396 2014-08-12 17:17:05.354213 2014-08-13 18:06:06.435101 7f5c973a57a0 20 read_log 194215'829 (194215'364) delete 7e36cb6a/benchmark_data_storage0111-ib.pocstor_48865_object21980/head//2 by client.2191635.0:49577 2014-08-12 17:17:05.406600 2014-08-13 18:06:06.435126 7f5c973a57a0 20 read_log 194215'830 (194215'365) delete 31ca3faa/benchmark_data_storage0111-ib.pocstor_48865_object22026/head//2 by client.2191635.0:49623 2014-08-12 17:17:05.421925 2014-08-13 18:06:06.435156 7f5c973a57a0 20 read_log 194215'831 (194215'366) delete b22ed1aa/benchmark_data_storage0111-ib.pocstor_48865_object22036/head//2 by client.2191635.0:49633 2014-08-12 17:17:05.424507 2014-08-13 18:06:06.435179 7f5c973a57a0 20 read_log 194215'832 (194215'367) delete 9d60f46a/benchmark_data_storage0111-ib.pocstor_48865_object22243/head//2 by client.2191635.0:49840 2014-08-12 17:17:05.481920 2014-08-13 18:06:06.435201 7f5c973a57a0 20 read_log 194215'833 (194215'368) delete d38544aa/benchmark_data_storage0111-ib.pocstor_48865_object22284/head//2 by client.2191635.0:49881 2014-08-12 17:17:05.495587 2014-08-13 18:06:06.435223 7f5c973a57a0 20 read_log 194215'834 (194215'369) delete 4248b2ea/benchmark_data_storage0111-ib.pocstor_48865_object22314/head//2 by client.2191635.0:49911 2014-08-12 17:17:05.504481 2014-08-13 18:06:06.435282 7f5c973a57a0 20 read_log 194215'835 (194215'370) delete 9d1470aa/benchmark_data_storage0111-ib.pocstor_48865_object22348/head//2 by client.2191635.0:49945 2014-08-12 17:17:05.514218 2014-08-13 18:06:06.435310 7f5c973a57a0 20 read_log 194215'836 (194215'371) delete 8501b66a/benchmark_data_storage0111-ib.pocstor_48865_object22363/head//2 by client.2191635.0:49960 2014-08-12 17:17:05.519001 2014-08-13 18:06:06.435332 7f5c973a57a0 20 read_log 194215'837 (194215'372) delete 4baa0dea/benchmark_data_storage0111-ib.pocstor_48865_object22414/head//2 by client.2191635.0:50011 2014-08-12 17:17:05.532797 2014-08-13 18:06:06.435359 7f5c973a57a0 20 read_log 194215'838 (194215'373) delete c76fe86a/benchmark_data_storage0111-ib.pocstor_48865_object22497/head//2 by client.2191635.0:50094 2014-08-12 17:17:05.556744 2014-08-13 18:06:06.435381 7f5c973a57a0 20 read_log 194215'839 (194215'374) delete 1378006a/benchmark_data_storage0111-ib.pocstor_48865_object22538/head//2 by client.2191635.0:50135 2014-08-12 17:17:05.567810 2014-08-13 18:06:06.435406 7f5c973a57a0 20 read_log 194215'840 (194215'375) delete 51bf9b2a/benchmark_data_storage0111-ib.pocstor_48865_object22569/head//2 by client.2191635.0:50166 2014-08-12 17:17:05.579086 2014-08-13 18:06:06.435433 7f5c973a57a0 20 read_log 194215'841 (194215'376) delete d91a75ea/benchmark_data_storage0111-ib.pocstor_48865_object22581/head//2 by client.2191635.0:50178 2014-08-12 17:17:05.583122 2014-08-13 18:06:06.435460 7f5c973a57a0 20 read_log 194215'842 (194215'377) delete 877f52a/benchmark_data_storage0111-ib.pocstor_48865_object22590/head//2 by client.2191635.0:50187 2014-08-12 17:17:05.584999 2014-08-13 18:06:06.435502 7f5c973a57a0 20 read_log 194215'843 (194215'378) delete 7772482a/benchmark_data_storage0111-ib.pocstor_48865_object22594/head//2 by client.2191635.0:50191 2014-08-12 17:17:05.586579 2014-08-13 18:06:06.435529 7f5c973a57a0 20 read_log 194215'844 (194215'379) delete 4f9362a/benchmark_data_storage0111-ib.pocstor_48865_object22640/head//2 by client.2191635.0:50237 2014-08-12 17:17:05.600634 2014-08-13 18:06:06.435552 7f5c973a57a0 20 read_log 194215'845 (194215'380) delete 5af1bdaa/benchmark_data_storage0111-ib.pocstor_48865_object22673/head//2 by client.2191635.0:50270 2014-08-12 17:17:05.610069 2014-08-13 18:06:06.435583 7f5c973a57a0 20 read_log 194215'846 (194215'381) delete 8f9836aa/benchmark_data_storage0111-ib.pocstor_48865_object22690/head//2 by client.2191635.0:50287 2014-08-12 17:17:05.615026 2014-08-13 18:06:06.435605 7f5c973a57a0 20 read_log 194215'847 (194215'382) delete 1e47b9ea/benchmark_data_storage0111-ib.pocstor_48865_object22731/head//2 by client.2191635.0:50328 2014-08-12 17:17:05.627444 2014-08-13 18:06:06.435631 7f5c973a57a0 20 read_log 194215'848 (194215'383) delete dfa6716a/benchmark_data_storage0111-ib.pocstor_48865_object22765/head//2 by client.2191635.0:50362 2014-08-12 17:17:05.638534 2014-08-13 18:06:06.435671 7f5c973a57a0 20 read_log 194215'849 (194215'384) delete 10a9a06a/benchmark_data_storage0111-ib.pocstor_48865_object22886/head//2 by client.2191635.0:50483 2014-08-12 17:17:05.677317 2014-08-13 18:06:06.435716 7f5c973a57a0 20 read_log 194215'850 (194215'385) delete 8c0ab1aa/benchmark_data_storage0111-ib.pocstor_48865_object22899/head//2 by client.2191635.0:50496 2014-08-12 17:17:05.683824 2014-08-13 18:06:06.435744 7f5c973a57a0 20 read_log 194215'851 (194215'386) delete 8bceefea/benchmark_data_storage0111-ib.pocstor_48865_object22913/head//2 by client.2191635.0:50510 2014-08-12 17:17:05.687141 2014-08-13 18:06:06.435767 7f5c973a57a0 20 read_log 194215'852 (194215'387) delete 2fbf146a/benchmark_data_storage0111-ib.pocstor_48865_object22932/head//2 by client.2191635.0:50529 2014-08-12 17:17:05.692264 2014-08-13 18:06:06.435789 7f5c973a57a0 20 read_log 194215'853 (194215'388) delete 7df62c2a/benchmark_data_storage0111-ib.pocstor_48865_object23119/head//2 by client.2191635.0:50716 2014-08-12 17:17:05.745373 2014-08-13 18:06:06.435810 7f5c973a57a0 20 read_log 194215'854 (194215'389) delete b614c76a/benchmark_data_storage0111-ib.pocstor_48865_object23197/head//2 by client.2191635.0:50794 2014-08-12 17:17:05.769603 2014-08-13 18:06:06.435852 7f5c973a57a0 20 read_log 194215'855 (194215'390) delete 62b055ea/benchmark_data_storage0111-ib.pocstor_48865_object23212/head//2 by client.2191635.0:50809 2014-08-12 17:17:05.772777 2014-08-13 18:06:06.435892 7f5c973a57a0 20 read_log 194215'856 (194215'391) delete 972960ea/benchmark_data_storage0111-ib.pocstor_48865_object23269/head//2 by client.2191635.0:50866 2014-08-12 17:17:05.788953 2014-08-13 18:06:06.435926 7f5c973a57a0 20 read_log 194215'857 (194215'392) delete 64c4eb6a/benchmark_data_storage0111-ib.pocstor_48865_object23325/head//2 by client.2191635.0:50922 2014-08-12 17:17:05.804096 2014-08-13 18:06:06.435952 7f5c973a57a0 20 read_log 194215'858 (194215'393) delete c10f2b2a/benchmark_data_storage0111-ib.pocstor_48865_object23339/head//2 by client.2191635.0:50936 2014-08-12 17:17:05.810037 2014-08-13 18:06:06.435974 7f5c973a57a0 20 read_log 194215'859 (194215'394) delete f00c10aa/benchmark_data_storage0111-ib.pocstor_48865_object23473/head//2 by client.2191635.0:51070 2014-08-12 17:17:05.849116 2014-08-13 18:06:06.436005 7f5c973a57a0 20 read_log 194215'860 (194215'395) delete e2f975aa/benchmark_data_storage0111-ib.pocstor_48865_object23699/head//2 by client.2191635.0:51296 2014-08-12 17:17:05.914597 2014-08-13 18:06:06.436047 7f5c973a57a0 20 read_log 194215'861 (194215'396) delete 68a661ea/benchmark_data_storage0111-ib.pocstor_48865_object23758/head//2 by client.2191635.0:51355 2014-08-12 17:17:05.931338 2014-08-13 18:06:06.436075 7f5c973a57a0 20 read_log 194215'862 (194215'397) delete baa697aa/benchmark_data_storage0111-ib.pocstor_48865_object23818/head//2 by client.2191635.0:51415 2014-08-12 17:17:05.950516 2014-08-13 18:06:06.436101 7f5c973a57a0 20 read_log 194215'863 (194215'398) delete 6b48b46a/benchmark_data_storage0111-ib.pocstor_48865_object23835/head//2 by client.2191635.0:51432 2014-08-12 17:17:05.954788 2014-08-13 18:06:06.436123 7f5c973a57a0 20 read_log 194215'864 (194215'399) delete d8ebb3aa/benchmark_data_storage0111-ib.pocstor_48865_object23888/head//2 by client.2191635.0:51485 2014-08-12 17:17:05.970616 2014-08-13 18:06:06.436156 7f5c973a57a0 20 read_log 194215'865 (194215'400) delete 34bd3aaa/benchmark_data_storage0111-ib.pocstor_48865_object23938/head//2 by client.2191635.0:51535 2014-08-12 17:17:05.984325 2014-08-13 18:06:06.436179 7f5c973a57a0 20 read_log 194215'866 (194215'401) delete 7504a3aa/benchmark_data_storage0111-ib.pocstor_48865_object23964/head//2 by client.2191635.0:51561 2014-08-12 17:17:05.992081 2014-08-13 18:06:06.436220 7f5c973a57a0 20 read_log 194215'867 (194215'402) delete e16bca6a/benchmark_data_storage0111-ib.pocstor_48865_object24022/head//2 by client.2191635.0:51619 2014-08-12 17:17:06.009384 2014-08-13 18:06:06.436242 7f5c973a57a0 20 read_log 194215'868 (194215'403) delete 4b4b8cea/benchmark_data_storage0111-ib.pocstor_48865_object24030/head//2 by client.2191635.0:51627 2014-08-12 17:17:06.012324 2014-08-13 18:06:06.436275 7f5c973a57a0 20 read_log 194215'869 (194215'404) delete e441ef6a/benchmark_data_storage0111-ib.pocstor_48865_object24080/head//2 by client.2191635.0:51677 2014-08-12 17:17:06.028919 2014-08-13 18:06:06.436301 7f5c973a57a0 20 read_log 194215'870 (194215'405) delete 5fcc2aaa/benchmark_data_storage0111-ib.pocstor_48865_object24241/head//2 by client.2191635.0:51838 2014-08-12 17:17:06.077254 2014-08-13 18:06:06.436331 7f5c973a57a0 20 read_log 194215'871 (194215'406) delete 3c8eaeaa/benchmark_data_storage0111-ib.pocstor_48865_object24315/head//2 by client.2191635.0:51912 2014-08-12 17:17:06.100277 2014-08-13 18:06:06.436353 7f5c973a57a0 20 read_log 194215'872 (194215'407) delete fb00662a/benchmark_data_storage0111-ib.pocstor_48865_object24327/head//2 by client.2191635.0:51924 2014-08-12 17:17:06.105571 2014-08-13 18:06:06.436383 7f5c973a57a0 20 read_log 194215'873 (194215'408) delete b50f7ea/benchmark_data_storage0111-ib.pocstor_48865_object24343/head//2 by client.2191635.0:51940 2014-08-12 17:17:06.109352 2014-08-13 18:06:06.436457 7f5c973a57a0 20 read_log 194215'874 (194215'409) delete f5caafaa/benchmark_data_storage0111-ib.pocstor_48865_object24349/head//2 by client.2191635.0:51946 2014-08-12 17:17:06.111288 2014-08-13 18:06:06.436498 7f5c973a57a0 20 read_log 194215'875 (194215'410) delete d65f39aa/benchmark_data_storage0111-ib.pocstor_48865_object24366/head//2 by client.2191635.0:51963 2014-08-12 17:17:06.116141 2014-08-13 18:06:06.436525 7f5c973a57a0 20 read_log 194215'876 (194215'411) delete d785ae2a/benchmark_data_storage0111-ib.pocstor_48865_object24461/head//2 by client.2191635.0:52058 2014-08-12 17:17:06.153743 2014-08-13 18:06:06.436547 7f5c973a57a0 20 read_log 194215'877 (194215'412) delete 94e6396a/benchmark_data_storage0111-ib.pocstor_48865_object24499/head//2 by client.2191635.0:52096 2014-08-12 17:17:06.164450 2014-08-13 18:06:06.436569 7f5c973a57a0 20 read_log 194215'878 (194215'413) delete ec9780ea/benchmark_data_storage0111-ib.pocstor_48865_object24551/head//2 by client.2191635.0:52148 2014-08-12 17:17:06.180755 2014-08-13 18:06:06.436591 7f5c973a57a0 20 read_log 194215'879 (194215'414) delete 76527aaa/benchmark_data_storage0111-ib.pocstor_48865_object24564/head//2 by client.2191635.0:52161 2014-08-12 17:17:06.184804 2014-08-13 18:06:06.436613 7f5c973a57a0 20 read_log 194215'880 (194215'415) delete 1fe906ea/benchmark_data_storage0111-ib.pocstor_48865_object24630/head//2 by client.2191635.0:52227 2014-08-12 17:17:06.207428 2014-08-13 18:06:06.436634 7f5c973a57a0 20 read_log 194215'881 (194215'416) delete d088b7aa/benchmark_data_storage0111-ib.pocstor_48865_object24638/head//2 by client.2191635.0:52235 2014-08-12 17:17:06.209632 2014-08-13 18:06:06.436656 7f5c973a57a0 20 read_log 194215'882 (194215'417) delete c957872a/benchmark_data_storage0111-ib.pocstor_48865_object24706/head//2 by client.2191635.0:52303 2014-08-12 17:17:06.228196 2014-08-13 18:06:06.436677 7f5c973a57a0 20 read_log 194215'883 (194215'418) delete 7e977daa/benchmark_data_storage0111-ib.pocstor_48865_object24941/head//2 by client.2191635.0:52538 2014-08-12 17:17:06.300089 2014-08-13 18:06:06.436699 7f5c973a57a0 20 read_log 194215'884 (194215'419) delete c441e36a/benchmark_data_storage0111-ib.pocstor_48865_object25044/head//2 by client.2191635.0:52641 2014-08-12 17:17:06.329291 2014-08-13 18:06:06.436720 7f5c973a57a0 20 read_log 194215'885 (194215'420) delete 3b05822a/benchmark_data_storage0111-ib.pocstor_48865_object25074/head//2 by client.2191635.0:52671 2014-08-12 17:17:06.336455 2014-08-13 18:06:06.436741 7f5c973a57a0 20 read_log 194215'886 (194215'421) delete dac3a2ea/benchmark_data_storage0111-ib.pocstor_48865_object25144/head//2 by client.2191635.0:52741 2014-08-12 17:17:06.357227 2014-08-13 18:06:06.436794 7f5c973a57a0 20 read_log 194215'887 (194215'422) delete ed943f2a/benchmark_data_storage0111-ib.pocstor_48865_object25159/head//2 by client.2191635.0:52756 2014-08-12 17:17:06.362533 2014-08-13 18:06:06.436817 7f5c973a57a0 20 read_log 194215'888 (194215'423) delete 68276bea/benchmark_data_storage0111-ib.pocstor_48865_object25166/head//2 by client.2191635.0:52763 2014-08-12 17:17:06.364580 2014-08-13 18:06:06.436839 7f5c973a57a0 20 read_log 194215'889 (194215'424) delete bd3e026a/benchmark_data_storage0111-ib.pocstor_48865_object25310/head//2 by client.2191635.0:52907 2014-08-12 17:17:06.409555 2014-08-13 18:06:06.436893 7f5c973a57a0 20 read_log 194215'890 (194215'425) delete 48d2462a/benchmark_data_storage0111-ib.pocstor_48865_object25331/head//2 by client.2191635.0:52928 2014-08-12 17:17:06.416164 2014-08-13 18:06:06.436919 7f5c973a57a0 20 read_log 194215'891 (194215'426) delete 9c95526a/benchmark_data_storage0111-ib.pocstor_48865_object25369/head//2 by client.2191635.0:52966 2014-08-12 17:17:06.428690 2014-08-13 18:06:06.436941 7f5c973a57a0 20 read_log 194215'892 (194215'427) delete 9e2944aa/benchmark_data_storage0111-ib.pocstor_48865_object25391/head//2 by client.2191635.0:52988 2014-08-12 17:17:06.436814 2014-08-13 18:06:06.436963 7f5c973a57a0 20 read_log 194215'893 (194215'428) delete aff1fc6a/benchmark_data_storage0111-ib.pocstor_48865_object25463/head//2 by client.2191635.0:53060 2014-08-12 17:17:06.457812 2014-08-13 18:06:06.436985 7f5c973a57a0 20 read_log 194215'894 (194215'429) delete 9f6824aa/benchmark_data_storage0111-ib.pocstor_48865_object25483/head//2 by client.2191635.0:53080 2014-08-12 17:17:06.464333 2014-08-13 18:06:06.437018 7f5c973a57a0 20 read_log 194215'895 (194215'430) delete 9cc1c82a/benchmark_data_storage0111-ib.pocstor_48865_object25512/head//2 by client.2191635.0:53109 2014-08-12 17:17:06.473807 2014-08-13 18:06:06.437044 7f5c973a57a0 20 read_log 194215'896 (194215'431) delete f2fcef2a/benchmark_data_storage0111-ib.pocstor_48865_object25660/head//2 by client.2191635.0:53257 2014-08-12 17:17:06.521469 2014-08-13 18:06:06.437066 7f5c973a57a0 20 read_log 194215'897 (194215'432) delete deeda82a/benchmark_data_storage0111-ib.pocstor_48865_object25706/head//2 by client.2191635.0:53303 2014-08-12 17:17:06.534550 2014-08-13 18:06:06.437087 7f5c973a57a0 20 read_log 194215'898 (194215'433) delete b504f56a/benchmark_data_storage0111-ib.pocstor_48865_object25726/head//2 by client.2191635.0:53323 2014-08-12 17:17:06.540764 2014-08-13 18:06:06.437109 7f5c973a57a0 20 read_log 194215'899 (194215'434) delete 652e19ea/benchmark_data_storage0111-ib.pocstor_48865_object25735/head//2 by client.2191635.0:53332 2014-08-12 17:17:06.543907 2014-08-13 18:06:06.437131 7f5c973a57a0 20 read_log 194215'900 (194215'435) delete ff82672a/benchmark_data_storage0111-ib.pocstor_48865_object25879/head//2 by client.2191635.0:53476 2014-08-12 17:17:06.584962 2014-08-13 18:06:06.437161 7f5c973a57a0 20 read_log 194215'901 (194215'436) delete 1deec7ea/benchmark_data_storage0111-ib.pocstor_48865_object25948/head//2 by client.2191635.0:53545 2014-08-12 17:17:06.611299 2014-08-13 18:06:06.437188 7f5c973a57a0 20 read_log 194215'902 (194215'437) delete 6898cd6a/benchmark_data_storage0111-ib.pocstor_48865_object25978/head//2 by client.2191635.0:53575 2014-08-12 17:17:06.620961 2014-08-13 18:06:06.437212 7f5c973a57a0 20 read_log 194215'903 (194215'438) delete 9c55f66a/benchmark_data_storage0111-ib.pocstor_48865_object26060/head//2 by client.2191635.0:53657 2014-08-12 17:17:06.647657 2014-08-13 18:06:06.437234 7f5c973a57a0 20 read_log 194215'904 (194215'439) delete 99abde2a/benchmark_data_storage0111-ib.pocstor_48865_object26093/head//2 by client.2191635.0:53690 2014-08-12 17:17:06.656776 2014-08-13 18:06:06.437255 7f5c973a57a0 20 read_log 194215'905 (194215'440) delete 1696b9ea/benchmark_data_storage0111-ib.pocstor_48865_object26147/head//2 by client.2191635.0:53744 2014-08-12 17:17:06.672322 2014-08-13 18:06:06.437278 7f5c973a57a0 20 read_log 194215'906 (194215'441) delete 29311a2a/benchmark_data_storage0111-ib.pocstor_48865_object26180/head//2 by client.2191635.0:53777 2014-08-12 17:17:06.682090 2014-08-13 18:06:06.437334 7f5c973a57a0 20 read_log 194215'907 (194215'442) delete f9505d2a/benchmark_data_storage0111-ib.pocstor_48865_object26181/head//2 by client.2191635.0:53778 2014-08-12 17:17:06.682204 2014-08-13 18:06:06.437361 7f5c973a57a0 20 read_log 194215'908 (194215'443) delete 5991912a/benchmark_data_storage0111-ib.pocstor_48865_object26212/head//2 by client.2191635.0:53809 2014-08-12 17:17:06.691957 2014-08-13 18:06:06.437384 7f5c973a57a0 20 read_log 194215'909 (194215'444) delete b036cdea/benchmark_data_storage0111-ib.pocstor_48865_object26253/head//2 by client.2191635.0:53850 2014-08-12 17:17:06.703696 2014-08-13 18:06:06.437413 7f5c973a57a0 20 read_log 194215'910 (194215'445) delete 3571b8ea/benchmark_data_storage0111-ib.pocstor_48865_object26474/head//2 by client.2191635.0:54071 2014-08-12 17:17:06.764192 2014-08-13 18:06:06.437439 7f5c973a57a0 20 read_log 194215'911 (194215'446) delete cdf9102a/benchmark_data_storage0111-ib.pocstor_48865_object26574/head//2 by client.2191635.0:54171 2014-08-12 17:17:06.792174 2014-08-13 18:06:06.437466 7f5c973a57a0 20 read_log 194215'912 (194215'447) delete acb3246a/benchmark_data_storage0111-ib.pocstor_48865_object26583/head//2 by client.2191635.0:54180 2014-08-12 17:17:06.794112 2014-08-13 18:06:06.437510 7f5c973a57a0 20 read_log 194215'913 (194215'448) delete c64e22ea/benchmark_data_storage0111-ib.pocstor_48865_object26626/head//2 by client.2191635.0:54223 2014-08-12 17:17:06.807865 2014-08-13 18:06:06.437538 7f5c973a57a0 20 read_log 194215'914 (194215'449) delete d778c46a/benchmark_data_storage0111-ib.pocstor_48865_object26634/head//2 by client.2191635.0:54231 2014-08-12 17:17:06.809462 2014-08-13 18:06:06.437581 7f5c973a57a0 20 read_log 194215'915 (194215'450) delete 6da2d2ea/benchmark_data_storage0111-ib.pocstor_48865_object26711/head//2 by client.2191635.0:54308 2014-08-12 17:17:06.831468 2014-08-13 18:06:06.437603 7f5c973a57a0 20 read_log 194215'916 (194215'451) delete 6655c36a/benchmark_data_storage0111-ib.pocstor_48865_object26778/head//2 by client.2191635.0:54375 2014-08-12 17:17:06.848835 2014-08-13 18:06:06.437625 7f5c973a57a0 20 read_log 194215'917 (194215'452) delete d935beaa/benchmark_data_storage0111-ib.pocstor_48865_object26946/head//2 by client.2191635.0:54543 2014-08-12 17:17:06.892700 2014-08-13 18:06:06.437659 7f5c973a57a0 20 read_log 194215'918 (194215'453) delete 143d096a/benchmark_data_storage0111-ib.pocstor_48865_object26976/head//2 by client.2191635.0:54573 2014-08-12 17:17:06.901272 2014-08-13 18:06:06.437694 7f5c973a57a0 20 read_log 194215'919 (194215'454) delete a5899caa/benchmark_data_storage0111-ib.pocstor_48865_object27029/head//2 by client.2191635.0:54626 2014-08-12 17:17:06.915040 2014-08-13 18:06:06.437721 7f5c973a57a0 20 read_log 194215'920 (194215'455) delete b23c082a/benchmark_data_storage0111-ib.pocstor_48865_object27049/head//2 by client.2191635.0:54646 2014-08-12 17:17:06.921441 2014-08-13 18:06:06.437743 7f5c973a57a0 20 read_log 194215'921 (194215'456) delete 128e18aa/benchmark_data_storage0111-ib.pocstor_48865_object27097/head//2 by client.2191635.0:54694 2014-08-12 17:17:06.935428 2014-08-13 18:06:06.437765 7f5c973a57a0 20 read_log 194215'922 (194215'457) delete 1fef4b2a/benchmark_data_storage0111-ib.pocstor_48865_object27156/head//2 by client.2191635.0:54753 2014-08-12 17:17:06.952099 2014-08-13 18:06:06.437786 7f5c973a57a0 20 read_log 194215'923 (194215'458) delete 53f22aaa/benchmark_data_storage0111-ib.pocstor_48865_object27164/head//2 by client.2191635.0:54761 2014-08-12 17:17:06.953868 2014-08-13 18:06:06.437808 7f5c973a57a0 20 read_log 194215'924 (194215'459) delete 25a647ea/benchmark_data_storage0111-ib.pocstor_48865_object27294/head//2 by client.2191635.0:54891 2014-08-12 17:17:06.989198 2014-08-13 18:06:06.437844 7f5c973a57a0 20 read_log 194215'925 (194215'460) delete edbf666a/benchmark_data_storage0111-ib.pocstor_48865_object27302/head//2 by client.2191635.0:54899 2014-08-12 17:17:06.992355 2014-08-13 18:06:06.437881 7f5c973a57a0 20 read_log 194215'926 (194215'461) delete e0414e6a/benchmark_data_storage0111-ib.pocstor_48865_object27325/head//2 by client.2191635.0:54922 2014-08-12 17:17:06.997593 2014-08-13 18:06:06.437903 7f5c973a57a0 20 read_log 194215'927 (194215'462) delete fa7d5f6a/benchmark_data_storage0111-ib.pocstor_48865_object27402/head//2 by client.2191635.0:54999 2014-08-12 17:17:07.019557 2014-08-13 18:06:06.437931 7f5c973a57a0 20 read_log 194215'928 (194215'463) delete fee55a2a/benchmark_data_storage0111-ib.pocstor_48865_object27451/head//2 by client.2191635.0:55048 2014-08-12 17:17:07.035724 2014-08-13 18:06:06.437956 7f5c973a57a0 20 read_log 194215'929 (194215'464) delete 9b85faaa/benchmark_data_storage0111-ib.pocstor_48865_object27523/head//2 by client.2191635.0:55120 2014-08-12 17:17:07.057376 2014-08-13 18:06:06.437983 7f5c973a57a0 20 read_log 194215'930 (194215'465) delete 233bee2a/benchmark_data_storage0111-ib.pocstor_48865_object27577/head//2 by client.2191635.0:55174 2014-08-12 17:17:07.076475 2014-08-13 18:06:06.439241 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.439298 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.439328 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.028075 0 0.000000 2014-08-13 18:06:06.439348 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.439362 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=0 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.439376 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'930], crt=0'0) 2014-08-13 18:06:06.439392 7f5c973a57a0 10 osd.4 194487 pgid 22.84 coll 22.84_head 2014-08-13 18:06:06.439687 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.84 2014-08-13 18:06:06.439704 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.439755 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.84(unlocked)] enter Initial 2014-08-13 18:06:06.439775 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.84(unlocked)] enter NotTrimming 2014-08-13 18:06:06.439978 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.440249 7f5c973a57a0 20 read_log 191648'20 (0'0) modify 205a9884/benchmark_data_storage0111-ib.pocstor_40745_object1580/head//22 by client.273158.0:1581 2014-07-30 14:08:54.591086 2014-08-13 18:06:06.440288 7f5c973a57a0 20 read_log 191648'56 (0'0) modify b5421084/benchmark_data_storage0111-ib.pocstor_40745_object3978/head//22 by client.273158.0:3979 2014-07-30 14:09:06.140534 2014-08-13 18:06:06.440321 7f5c973a57a0 20 read_log 191648'254 (0'0) modify 85632084/benchmark_data_storage0111-ib.pocstor_40745_object18447/head//22 by client.273158.0:18448 2014-07-30 14:10:20.511565 2014-08-13 18:06:06.440350 7f5c973a57a0 20 read_log 191648'268 (0'0) modify 2a0ab084/benchmark_data_storage0111-ib.pocstor_40745_object18918/head//22 by client.273158.0:18919 2014-07-30 14:10:23.231329 2014-08-13 18:06:06.440386 7f5c973a57a0 20 read_log 191648'285 (0'0) modify bf866884/benchmark_data_storage0111-ib.pocstor_40745_object19765/head//22 by client.273158.0:19766 2014-07-30 14:10:30.417399 2014-08-13 18:06:06.440440 7f5c973a57a0 20 read_log 191648'303 (0'0) modify c7aba884/benchmark_data_storage0111-ib.pocstor_40745_object20683/head//22 by client.273158.0:20684 2014-07-30 14:10:34.707216 2014-08-13 18:06:06.440463 7f5c973a57a0 20 read_log 191727'341 (0'0) modify aec4084/benchmark_data_storage0111-ib.pocstor_51285_object1142/head//22 by client.275175.0:1143 2014-07-30 14:18:37.978666 2014-08-13 18:06:06.440485 7f5c973a57a0 20 read_log 191727'343 (0'0) modify 57eff084/benchmark_data_storage0111-ib.pocstor_51285_object2198/head//22 by client.275175.0:2199 2014-07-30 14:18:41.542418 2014-08-13 18:06:06.440506 7f5c973a57a0 20 read_log 191727'348 (0'0) modify ad464084/benchmark_data_storage0111-ib.pocstor_51285_object6729/head//22 by client.275175.0:6730 2014-07-30 14:18:55.826527 2014-08-13 18:06:06.440527 7f5c973a57a0 20 read_log 191727'349 (0'0) modify 81d55084/benchmark_data_storage0111-ib.pocstor_51285_object6936/head//22 by client.275175.0:6937 2014-07-30 14:18:56.541008 2014-08-13 18:06:06.440549 7f5c973a57a0 20 read_log 191727'350 (0'0) modify 14822084/benchmark_data_storage0111-ib.pocstor_51285_object7084/head//22 by client.275175.0:7085 2014-07-30 14:18:56.956303 2014-08-13 18:06:06.440570 7f5c973a57a0 20 read_log 191727'351 (0'0) modify bfbb8084/benchmark_data_storage0111-ib.pocstor_51285_object7632/head//22 by client.275175.0:7633 2014-07-30 14:18:58.500817 2014-08-13 18:06:06.440606 7f5c973a57a0 20 read_log 191727'356 (0'0) modify 974d1084/benchmark_data_storage0111-ib.pocstor_51285_object10697/head//22 by client.275175.0:10698 2014-07-30 14:19:07.195713 2014-08-13 18:06:06.440635 7f5c973a57a0 20 read_log 191727'360 (0'0) modify f8db0084/benchmark_data_storage0111-ib.pocstor_51285_object12830/head//22 by client.275175.0:12831 2014-07-30 14:19:13.964975 2014-08-13 18:06:06.440657 7f5c973a57a0 20 read_log 191727'364 (0'0) modify 21650884/benchmark_data_storage0111-ib.pocstor_51285_object15290/head//22 by client.275175.0:15291 2014-07-30 14:19:21.640370 2014-08-13 18:06:06.440678 7f5c973a57a0 20 read_log 191727'365 (0'0) modify 2b9d4084/benchmark_data_storage0111-ib.pocstor_51285_object15665/head//22 by client.275175.0:15666 2014-07-30 14:19:22.876135 2014-08-13 18:06:06.440699 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 8211d084/benchmark_data_storage0111-ib.pocstor_51285_object18767/head//22 by client.275175.0:18768 2014-07-30 14:19:32.369003 2014-08-13 18:06:06.440721 7f5c973a57a0 20 read_log 191727'400 (0'0) modify 1729d084/benchmark_data_storage0111-ib.pocstor_51285_object27218/head//22 by client.275175.0:27219 2014-07-30 14:19:58.436718 2014-08-13 18:06:06.440742 7f5c973a57a0 20 read_log 191727'401 (0'0) modify fdfec084/benchmark_data_storage0111-ib.pocstor_51285_object28283/head//22 by client.275175.0:28284 2014-07-30 14:20:01.818767 2014-08-13 18:06:06.440763 7f5c973a57a0 20 read_log 191727'402 (0'0) modify db3e7084/benchmark_data_storage0111-ib.pocstor_51285_object29027/head//22 by client.275175.0:29028 2014-07-30 14:20:03.966527 2014-08-13 18:06:06.440784 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 46c9a084/benchmark_data_storage0111-ib.pocstor_51285_object29801/head//22 by client.275175.0:29802 2014-07-30 14:20:06.390757 2014-08-13 18:06:06.440815 7f5c973a57a0 20 read_log 191727'409 (0'0) modify 7778f884/benchmark_data_storage0111-ib.pocstor_51285_object32228/head//22 by client.275175.0:32229 2014-07-30 14:20:13.557324 2014-08-13 18:06:06.440845 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 5df73884/benchmark_data_storage0111-ib.pocstor_51285_object33582/head//22 by client.275175.0:33583 2014-07-30 14:20:17.799915 2014-08-13 18:06:06.440886 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 7c177884/benchmark_data_storage0111-ib.pocstor_51285_object33968/head//22 by client.275175.0:33969 2014-07-30 14:20:18.902465 2014-08-13 18:06:06.440913 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 1a64b084/benchmark_data_storage0111-ib.pocstor_51285_object34143/head//22 by client.275175.0:34144 2014-07-30 14:20:19.458283 2014-08-13 18:06:06.440955 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 8b01b884/benchmark_data_storage0111-ib.pocstor_51285_object34402/head//22 by client.275175.0:34403 2014-07-30 14:20:20.267697 2014-08-13 18:06:06.440995 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 1622d084/benchmark_data_storage0111-ib.pocstor_51285_object36130/head//22 by client.275175.0:36131 2014-07-30 14:20:25.665252 2014-08-13 18:06:06.441022 7f5c973a57a0 20 read_log 191727'425 (0'0) modify c7bb9884/benchmark_data_storage0111-ib.pocstor_51285_object37519/head//22 by client.275175.0:37520 2014-07-30 14:20:29.903390 2014-08-13 18:06:06.441044 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 2c757084/benchmark_data_storage0111-ib.pocstor_51285_object37881/head//22 by client.275175.0:37882 2014-07-30 14:20:30.906287 2014-08-13 18:06:06.441066 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 36c52884/benchmark_data_storage0111-ib.pocstor_35625_object1157/head//22 by client.940330.0:1158 2014-08-11 16:29:45.375958 2014-08-13 18:06:06.441087 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 81389084/benchmark_data_storage0111-ib.pocstor_35625_object3335/head//22 by client.940330.0:3336 2014-08-11 16:29:53.868950 2014-08-13 18:06:06.441109 7f5c973a57a0 20 read_log 192888'430 (0'0) modify b96f2884/benchmark_data_storage0111-ib.pocstor_35625_object6702/head//22 by client.940330.0:6703 2014-08-11 16:30:06.266578 2014-08-13 18:06:06.441140 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 40cef084/benchmark_data_storage0111-ib.pocstor_35625_object7298/head//22 by client.940330.0:7299 2014-08-11 16:30:08.371085 2014-08-13 18:06:06.441178 7f5c973a57a0 20 read_log 192888'432 (0'0) modify cf459884/benchmark_data_storage0111-ib.pocstor_35625_object8956/head//22 by client.940330.0:8957 2014-08-11 16:30:14.180582 2014-08-13 18:06:06.441219 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 1a881884/benchmark_data_storage0111-ib.pocstor_35625_object10500/head//22 by client.940330.0:10501 2014-08-11 16:30:19.802513 2014-08-13 18:06:06.441242 7f5c973a57a0 20 read_log 192888'434 (0'0) modify ad5e9084/benchmark_data_storage0111-ib.pocstor_35625_object10730/head//22 by client.940330.0:10731 2014-08-11 16:30:20.751785 2014-08-13 18:06:06.441263 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 33576084/benchmark_data_storage0111-ib.pocstor_35625_object11454/head//22 by client.940330.0:11455 2014-08-11 16:30:23.616894 2014-08-13 18:06:06.441284 7f5c973a57a0 20 read_log 192888'436 (0'0) modify b8ec0884/benchmark_data_storage0111-ib.pocstor_35625_object13040/head//22 by client.940330.0:13041 2014-08-11 16:30:29.995136 2014-08-13 18:06:06.441305 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 51b7b884/benchmark_data_storage0111-ib.pocstor_35625_object14783/head//22 by client.940330.0:14784 2014-08-11 16:30:36.684727 2014-08-13 18:06:06.441335 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 7dfd8884/benchmark_data_storage0111-ib.pocstor_35625_object15735/head//22 by client.940330.0:15736 2014-08-11 16:30:40.233008 2014-08-13 18:06:06.441357 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 408a9884/benchmark_data_storage0111-ib.pocstor_35625_object16251/head//22 by client.940330.0:16252 2014-08-11 16:30:42.169432 2014-08-13 18:06:06.441388 7f5c973a57a0 20 read_log 192888'440 (0'0) modify b6e47884/benchmark_data_storage0111-ib.pocstor_35625_object23093/head//22 by client.940330.0:23094 2014-08-11 16:31:08.756569 2014-08-13 18:06:06.441430 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 9f5b6084/benchmark_data_storage0111-ib.pocstor_35625_object24964/head//22 by client.940330.0:24965 2014-08-11 16:31:15.730467 2014-08-13 18:06:06.441475 7f5c973a57a0 20 read_log 192888'442 (0'0) modify d8b82084/benchmark_data_storage0111-ib.pocstor_35625_object26836/head//22 by client.940330.0:26837 2014-08-11 16:31:23.170115 2014-08-13 18:06:06.441502 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 49743084/benchmark_data_storage0111-ib.pocstor_35625_object26861/head//22 by client.940330.0:26862 2014-08-11 16:31:23.265734 2014-08-13 18:06:06.441542 7f5c973a57a0 20 read_log 192888'444 (192888'428) delete 36c52884/benchmark_data_storage0111-ib.pocstor_35625_object1157/head//22 by client.940330.0:32465 2014-08-11 16:31:41.787500 2014-08-13 18:06:06.441569 7f5c973a57a0 20 read_log 192888'445 (192888'429) delete 81389084/benchmark_data_storage0111-ib.pocstor_35625_object3335/head//22 by client.940330.0:34643 2014-08-11 16:31:42.344781 2014-08-13 18:06:06.441595 7f5c973a57a0 20 read_log 192888'446 (192888'430) delete b96f2884/benchmark_data_storage0111-ib.pocstor_35625_object6702/head//22 by client.940330.0:38010 2014-08-11 16:31:43.263000 2014-08-13 18:06:06.441638 7f5c973a57a0 20 read_log 192888'447 (192888'431) delete 40cef084/benchmark_data_storage0111-ib.pocstor_35625_object7298/head//22 by client.940330.0:38606 2014-08-11 16:31:43.414228 2014-08-13 18:06:06.441661 7f5c973a57a0 20 read_log 192888'448 (192888'432) delete cf459884/benchmark_data_storage0111-ib.pocstor_35625_object8956/head//22 by client.940330.0:40264 2014-08-11 16:31:43.830718 2014-08-13 18:06:06.441687 7f5c973a57a0 20 read_log 192888'449 (192888'433) delete 1a881884/benchmark_data_storage0111-ib.pocstor_35625_object10500/head//22 by client.940330.0:41808 2014-08-11 16:31:44.253052 2014-08-13 18:06:06.441713 7f5c973a57a0 20 read_log 192888'450 (192888'434) delete ad5e9084/benchmark_data_storage0111-ib.pocstor_35625_object10730/head//22 by client.940330.0:42038 2014-08-11 16:31:44.320351 2014-08-13 18:06:06.441782 7f5c973a57a0 20 read_log 192888'451 (192888'435) delete 33576084/benchmark_data_storage0111-ib.pocstor_35625_object11454/head//22 by client.940330.0:42762 2014-08-11 16:31:44.529030 2014-08-13 18:06:06.441828 7f5c973a57a0 20 read_log 192888'452 (192888'436) delete b8ec0884/benchmark_data_storage0111-ib.pocstor_35625_object13040/head//22 by client.940330.0:44348 2014-08-11 16:31:44.929482 2014-08-13 18:06:06.441855 7f5c973a57a0 20 read_log 192888'453 (192888'437) delete 51b7b884/benchmark_data_storage0111-ib.pocstor_35625_object14783/head//22 by client.940330.0:46091 2014-08-11 16:31:45.394940 2014-08-13 18:06:06.441880 7f5c973a57a0 20 read_log 192888'454 (192888'438) delete 7dfd8884/benchmark_data_storage0111-ib.pocstor_35625_object15735/head//22 by client.940330.0:47043 2014-08-11 16:31:45.654188 2014-08-13 18:06:06.441902 7f5c973a57a0 20 read_log 192888'455 (192888'439) delete 408a9884/benchmark_data_storage0111-ib.pocstor_35625_object16251/head//22 by client.940330.0:47559 2014-08-11 16:31:45.791300 2014-08-13 18:06:06.441934 7f5c973a57a0 20 read_log 192888'456 (192888'440) delete b6e47884/benchmark_data_storage0111-ib.pocstor_35625_object23093/head//22 by client.940330.0:54401 2014-08-11 16:31:47.789243 2014-08-13 18:06:06.441962 7f5c973a57a0 20 read_log 192888'457 (192888'441) delete 9f5b6084/benchmark_data_storage0111-ib.pocstor_35625_object24964/head//22 by client.940330.0:56272 2014-08-11 16:31:48.329240 2014-08-13 18:06:06.441984 7f5c973a57a0 20 read_log 192888'458 (192888'442) delete d8b82084/benchmark_data_storage0111-ib.pocstor_35625_object26836/head//22 by client.940330.0:58144 2014-08-11 16:31:48.882049 2014-08-13 18:06:06.442006 7f5c973a57a0 20 read_log 192888'459 (192888'443) delete 49743084/benchmark_data_storage0111-ib.pocstor_35625_object26861/head//22 by client.940330.0:58169 2014-08-11 16:31:48.890513 2014-08-13 18:06:06.442027 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 4fc3b884/benchmark_data_storage0111-ib.pocstor_64498_object792/head//22 by client.1049635.0:793 2014-08-12 17:17:19.683040 2014-08-13 18:06:06.442062 7f5c973a57a0 20 read_log 194215'461 (0'0) modify a4984884/benchmark_data_storage0111-ib.pocstor_64498_object5513/head//22 by client.1049635.0:5514 2014-08-12 17:17:34.654642 2014-08-13 18:06:06.442084 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 1943a084/benchmark_data_storage0111-ib.pocstor_64498_object9092/head//22 by client.1049635.0:9093 2014-08-12 17:17:45.285566 2014-08-13 18:06:06.442106 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 66030084/benchmark_data_storage0111-ib.pocstor_64498_object10169/head//22 by client.1049635.0:10170 2014-08-12 17:17:48.411223 2014-08-13 18:06:06.442127 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 10f56084/benchmark_data_storage0111-ib.pocstor_64498_object12292/head//22 by client.1049635.0:12293 2014-08-12 17:17:55.327966 2014-08-13 18:06:06.442157 7f5c973a57a0 20 read_log 194215'465 (0'0) modify cc164884/benchmark_data_storage0111-ib.pocstor_64498_object13405/head//22 by client.1049635.0:13406 2014-08-12 17:17:58.663314 2014-08-13 18:06:06.442185 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 756e084/benchmark_data_storage0111-ib.pocstor_64498_object14092/head//22 by client.1049635.0:14093 2014-08-12 17:18:00.797029 2014-08-13 18:06:06.442207 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 426ac884/benchmark_data_storage0111-ib.pocstor_64498_object15892/head//22 by client.1049635.0:15893 2014-08-12 17:18:07.151254 2014-08-13 18:06:06.442229 7f5c973a57a0 20 read_log 194215'468 (0'0) modify f5950884/benchmark_data_storage0111-ib.pocstor_64498_object17120/head//22 by client.1049635.0:17121 2014-08-12 17:18:11.822259 2014-08-13 18:06:06.442250 7f5c973a57a0 20 read_log 194215'469 (0'0) modify ed32f884/benchmark_data_storage0111-ib.pocstor_64498_object18769/head//22 by client.1049635.0:18770 2014-08-12 17:18:18.067064 2014-08-13 18:06:06.442272 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 42e7c084/benchmark_data_storage0111-ib.pocstor_64498_object18881/head//22 by client.1049635.0:18882 2014-08-12 17:18:18.481676 2014-08-13 18:06:06.442293 7f5c973a57a0 20 read_log 194215'471 (0'0) modify f2ae884/benchmark_data_storage0111-ib.pocstor_64498_object20404/head//22 by client.1049635.0:20405 2014-08-12 17:18:24.313106 2014-08-13 18:06:06.442314 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 4495d084/benchmark_data_storage0111-ib.pocstor_64498_object22141/head//22 by client.1049635.0:22142 2014-08-12 17:18:30.861306 2014-08-13 18:06:06.442343 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 32449884/benchmark_data_storage0111-ib.pocstor_64498_object24160/head//22 by client.1049635.0:24161 2014-08-12 17:18:38.448951 2014-08-13 18:06:06.442374 7f5c973a57a0 20 read_log 194215'474 (0'0) modify cce09084/benchmark_data_storage0111-ib.pocstor_64498_object26483/head//22 by client.1049635.0:26484 2014-08-12 17:18:47.227456 2014-08-13 18:06:06.442402 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 19ee6084/benchmark_data_storage0111-ib.pocstor_64498_object27649/head//22 by client.1049635.0:27650 2014-08-12 17:18:51.571620 2014-08-13 18:06:06.442428 7f5c973a57a0 20 read_log 194215'476 (0'0) modify c5a2e884/benchmark_data_storage0111-ib.pocstor_64498_object28642/head//22 by client.1049635.0:28643 2014-08-12 17:18:55.268252 2014-08-13 18:06:06.442455 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 9d68c884/benchmark_data_storage0111-ib.pocstor_64498_object29164/head//22 by client.1049635.0:29165 2014-08-12 17:18:57.208449 2014-08-13 18:06:06.442481 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 525c3884/benchmark_data_storage0111-ib.pocstor_64498_object33928/head//22 by client.1049635.0:33929 2014-08-12 17:19:15.908121 2014-08-13 18:06:06.442526 7f5c973a57a0 20 read_log 194215'479 (194215'460) delete 4fc3b884/benchmark_data_storage0111-ib.pocstor_64498_object792/head//22 by client.1049635.0:34995 2014-08-12 17:19:18.249519 2014-08-13 18:06:06.442565 7f5c973a57a0 20 read_log 194215'480 (194215'461) delete a4984884/benchmark_data_storage0111-ib.pocstor_64498_object5513/head//22 by client.1049635.0:39716 2014-08-12 17:19:19.775451 2014-08-13 18:06:06.442623 7f5c973a57a0 20 read_log 194215'481 (194215'462) delete 1943a084/benchmark_data_storage0111-ib.pocstor_64498_object9092/head//22 by client.1049635.0:43295 2014-08-12 17:19:20.951142 2014-08-13 18:06:06.442664 7f5c973a57a0 20 read_log 194215'482 (194215'463) delete 66030084/benchmark_data_storage0111-ib.pocstor_64498_object10169/head//22 by client.1049635.0:44372 2014-08-12 17:19:21.317144 2014-08-13 18:06:06.442687 7f5c973a57a0 20 read_log 194215'483 (194215'464) delete 10f56084/benchmark_data_storage0111-ib.pocstor_64498_object12292/head//22 by client.1049635.0:46495 2014-08-12 17:19:21.989034 2014-08-13 18:06:06.442711 7f5c973a57a0 20 read_log 194215'484 (194215'465) delete cc164884/benchmark_data_storage0111-ib.pocstor_64498_object13405/head//22 by client.1049635.0:47608 2014-08-12 17:19:22.362622 2014-08-13 18:06:06.442733 7f5c973a57a0 20 read_log 194215'485 (194215'466) delete 756e084/benchmark_data_storage0111-ib.pocstor_64498_object14092/head//22 by client.1049635.0:48295 2014-08-12 17:19:22.591906 2014-08-13 18:06:06.442754 7f5c973a57a0 20 read_log 194215'486 (194215'467) delete 426ac884/benchmark_data_storage0111-ib.pocstor_64498_object15892/head//22 by client.1049635.0:50095 2014-08-12 17:19:23.156735 2014-08-13 18:06:06.442794 7f5c973a57a0 20 read_log 194215'487 (194215'468) delete f5950884/benchmark_data_storage0111-ib.pocstor_64498_object17120/head//22 by client.1049635.0:51323 2014-08-12 17:19:23.544119 2014-08-13 18:06:06.442816 7f5c973a57a0 20 read_log 194215'488 (194215'469) delete ed32f884/benchmark_data_storage0111-ib.pocstor_64498_object18769/head//22 by client.1049635.0:52972 2014-08-12 17:19:24.086206 2014-08-13 18:06:06.442838 7f5c973a57a0 20 read_log 194215'489 (194215'470) delete 42e7c084/benchmark_data_storage0111-ib.pocstor_64498_object18881/head//22 by client.1049635.0:53084 2014-08-12 17:19:24.125119 2014-08-13 18:06:06.442860 7f5c973a57a0 20 read_log 194215'490 (194215'471) delete f2ae884/benchmark_data_storage0111-ib.pocstor_64498_object20404/head//22 by client.1049635.0:54607 2014-08-12 17:19:24.670421 2014-08-13 18:06:06.442881 7f5c973a57a0 20 read_log 194215'491 (194215'472) delete 4495d084/benchmark_data_storage0111-ib.pocstor_64498_object22141/head//22 by client.1049635.0:56344 2014-08-12 17:19:25.286401 2014-08-13 18:06:06.442902 7f5c973a57a0 20 read_log 194215'492 (194215'473) delete 32449884/benchmark_data_storage0111-ib.pocstor_64498_object24160/head//22 by client.1049635.0:58363 2014-08-12 17:19:26.017810 2014-08-13 18:06:06.442924 7f5c973a57a0 20 read_log 194215'493 (194215'474) delete cce09084/benchmark_data_storage0111-ib.pocstor_64498_object26483/head//22 by client.1049635.0:60686 2014-08-12 17:19:26.816526 2014-08-13 18:06:06.442945 7f5c973a57a0 20 read_log 194215'494 (194215'475) delete 19ee6084/benchmark_data_storage0111-ib.pocstor_64498_object27649/head//22 by client.1049635.0:61852 2014-08-12 17:19:27.229367 2014-08-13 18:06:06.442967 7f5c973a57a0 20 read_log 194215'495 (194215'476) delete c5a2e884/benchmark_data_storage0111-ib.pocstor_64498_object28642/head//22 by client.1049635.0:62845 2014-08-12 17:19:27.575359 2014-08-13 18:06:06.442988 7f5c973a57a0 20 read_log 194215'496 (194215'477) delete 9d68c884/benchmark_data_storage0111-ib.pocstor_64498_object29164/head//22 by client.1049635.0:63367 2014-08-12 17:19:27.766786 2014-08-13 18:06:06.443009 7f5c973a57a0 20 read_log 194215'497 (194215'478) delete 525c3884/benchmark_data_storage0111-ib.pocstor_64498_object33928/head//22 by client.1049635.0:68131 2014-08-12 17:19:29.452313 2014-08-13 18:06:06.443176 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.443216 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.443243 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003489 0 0.000000 2014-08-13 18:06:06.443262 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.443277 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=0 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.443291 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'497], crt=191648'337) 2014-08-13 18:06:06.443309 7f5c973a57a0 10 osd.4 194487 pgid 22.99 coll 22.99_head 2014-08-13 18:06:06.443613 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.99 2014-08-13 18:06:06.443631 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.443680 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.99(unlocked)] enter Initial 2014-08-13 18:06:06.443700 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.99(unlocked)] enter NotTrimming 2014-08-13 18:06:06.443953 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.444191 7f5c973a57a0 20 read_log 191648'2 (0'0) modify e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 by client.273158.0:185 2014-07-30 14:08:49.401106 2014-08-13 18:06:06.444237 7f5c973a57a0 20 read_log 191648'21 (0'0) modify ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 by client.273158.0:1462 2014-07-30 14:08:54.158576 2014-08-13 18:06:06.444269 7f5c973a57a0 20 read_log 191648'54 (0'0) modify 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 by client.273158.0:3013 2014-07-30 14:09:01.437743 2014-08-13 18:06:06.444292 7f5c973a57a0 20 read_log 191648'67 (0'0) modify a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 by client.273158.0:3715 2014-07-30 14:09:04.874600 2014-08-13 18:06:06.444322 7f5c973a57a0 20 read_log 191648'116 (0'0) modify c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 by client.273158.0:6616 2014-07-30 14:09:19.415313 2014-08-13 18:06:06.444365 7f5c973a57a0 20 read_log 191648'166 (0'0) modify b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 by client.273158.0:9878 2014-07-30 14:09:35.189406 2014-08-13 18:06:06.444392 7f5c973a57a0 20 read_log 191648'172 (0'0) modify 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 by client.273158.0:10200 2014-07-30 14:09:36.828464 2014-08-13 18:06:06.444415 7f5c973a57a0 20 read_log 191648'196 (0'0) modify 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 by client.273158.0:11571 2014-07-30 14:09:45.102648 2014-08-13 18:06:06.444436 7f5c973a57a0 20 read_log 191648'202 (0'0) modify d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 by client.273158.0:11893 2014-07-30 14:09:46.431387 2014-08-13 18:06:06.444458 7f5c973a57a0 20 read_log 191648'310 (0'0) modify 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 by client.273158.0:19422 2014-07-30 14:10:25.983506 2014-08-13 18:06:06.444519 7f5c973a57a0 20 read_log 191648'343 (0'0) modify b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 by client.273158.0:21536 2014-07-30 14:10:39.632431 2014-08-13 18:06:06.444584 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 by client.275175.0:754 2014-07-30 14:18:36.412444 2014-08-13 18:06:06.444612 7f5c973a57a0 20 read_log 191727'369 (0'0) modify 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 by client.275175.0:1570 2014-07-30 14:18:39.452929 2014-08-13 18:06:06.444634 7f5c973a57a0 20 read_log 191727'375 (0'0) modify ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 by client.275175.0:4555 2014-07-30 14:18:49.024949 2014-08-13 18:06:06.444655 7f5c973a57a0 20 read_log 191727'387 (0'0) modify c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 by client.275175.0:10914 2014-07-30 14:19:07.800919 2014-08-13 18:06:06.444676 7f5c973a57a0 20 read_log 191727'395 (0'0) modify a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 by client.275175.0:14436 2014-07-30 14:19:19.105976 2014-08-13 18:06:06.444698 7f5c973a57a0 20 read_log 191727'397 (0'0) modify e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 by client.275175.0:14939 2014-07-30 14:19:20.529496 2014-08-13 18:06:06.444719 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 by client.275175.0:15114 2014-07-30 14:19:21.090244 2014-08-13 18:06:06.444740 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 by client.275175.0:15494 2014-07-30 14:19:22.397025 2014-08-13 18:06:06.444762 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 by client.275175.0:19337 2014-07-30 14:19:34.319796 2014-08-13 18:06:06.444783 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 by client.275175.0:21149 2014-07-30 14:19:39.551859 2014-08-13 18:06:06.444804 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 by client.275175.0:21336 2014-07-30 14:19:40.119379 2014-08-13 18:06:06.444825 7f5c973a57a0 20 read_log 191727'420 (0'0) modify e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 by client.275175.0:23727 2014-07-30 14:19:48.063239 2014-08-13 18:06:06.444846 7f5c973a57a0 20 read_log 191727'423 (0'0) modify b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 by client.275175.0:25206 2014-07-30 14:19:52.433927 2014-08-13 18:06:06.444867 7f5c973a57a0 20 read_log 191727'428 (0'0) modify 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 by client.275175.0:29448 2014-07-30 14:20:05.231231 2014-08-13 18:06:06.444888 7f5c973a57a0 20 read_log 191727'432 (0'0) modify 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 by client.275175.0:30598 2014-07-30 14:20:08.846399 2014-08-13 18:06:06.444909 7f5c973a57a0 20 read_log 191727'434 (0'0) modify f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 by client.275175.0:32207 2014-07-30 14:20:13.496910 2014-08-13 18:06:06.444930 7f5c973a57a0 20 read_log 191727'440 (0'0) modify b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 by client.275175.0:38337 2014-07-30 14:20:32.203438 2014-08-13 18:06:06.444952 7f5c973a57a0 20 read_log 192674'441 (0'0) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8671 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.444974 7f5c973a57a0 20 read_log 192674'442 (192674'441) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8672 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.444996 7f5c973a57a0 20 read_log 192674'443 (192674'442) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8673 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445030 7f5c973a57a0 20 read_log 192674'444 (192674'443) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8674 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445056 7f5c973a57a0 20 read_log 192674'445 (192674'444) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8675 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445078 7f5c973a57a0 20 read_log 192674'446 (192674'445) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8676 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445103 7f5c973a57a0 20 read_log 192674'447 (192674'446) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8677 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445124 7f5c973a57a0 20 read_log 192674'448 (192674'447) modify 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 by client.279772.1:8678 2014-07-31 18:03:50.857000 2014-08-13 18:06:06.445180 7f5c973a57a0 20 read_log 192888'449 (0'0) modify fc63b099/benchmark_data_storage0111-ib.pocstor_35625_object1416/head//22 by client.940330.0:1417 2014-08-11 16:29:46.368792 2014-08-13 18:06:06.445212 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 4ee8c899/benchmark_data_storage0111-ib.pocstor_35625_object3692/head//22 by client.940330.0:3693 2014-08-11 16:29:55.207679 2014-08-13 18:06:06.445245 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 10403899/benchmark_data_storage0111-ib.pocstor_35625_object4202/head//22 by client.940330.0:4203 2014-08-11 16:29:57.065365 2014-08-13 18:06:06.445272 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 730f6099/benchmark_data_storage0111-ib.pocstor_35625_object6761/head//22 by client.940330.0:6762 2014-08-11 16:30:06.478093 2014-08-13 18:06:06.445294 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 27f82899/benchmark_data_storage0111-ib.pocstor_35625_object14667/head//22 by client.940330.0:14668 2014-08-11 16:30:36.255223 2014-08-13 18:06:06.445316 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 9568e099/benchmark_data_storage0111-ib.pocstor_35625_object20645/head//22 by client.940330.0:20646 2014-08-11 16:30:59.228846 2014-08-13 18:06:06.445345 7f5c973a57a0 20 read_log 192888'455 (0'0) modify abd87899/benchmark_data_storage0111-ib.pocstor_35625_object23696/head//22 by client.940330.0:23697 2014-08-11 16:31:11.037530 2014-08-13 18:06:06.445367 7f5c973a57a0 20 read_log 192888'456 (0'0) modify cbc83099/benchmark_data_storage0111-ib.pocstor_35625_object24625/head//22 by client.940330.0:24626 2014-08-11 16:31:14.437461 2014-08-13 18:06:06.445397 7f5c973a57a0 20 read_log 192888'457 (0'0) modify f3eac099/benchmark_data_storage0111-ib.pocstor_35625_object24808/head//22 by client.940330.0:24809 2014-08-11 16:31:15.125160 2014-08-13 18:06:06.445464 7f5c973a57a0 20 read_log 192888'458 (0'0) modify fb915099/benchmark_data_storage0111-ib.pocstor_35625_object26265/head//22 by client.940330.0:26266 2014-08-11 16:31:20.833187 2014-08-13 18:06:06.445498 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 97271899/benchmark_data_storage0111-ib.pocstor_35625_object30552/head//22 by client.940330.0:30553 2014-08-11 16:31:37.547016 2014-08-13 18:06:06.445533 7f5c973a57a0 20 read_log 192888'460 (192888'449) delete fc63b099/benchmark_data_storage0111-ib.pocstor_35625_object1416/head//22 by client.940330.0:32724 2014-08-11 16:31:41.847343 2014-08-13 18:06:06.445561 7f5c973a57a0 20 read_log 192888'461 (192888'450) delete 4ee8c899/benchmark_data_storage0111-ib.pocstor_35625_object3692/head//22 by client.940330.0:35000 2014-08-11 16:31:42.439818 2014-08-13 18:06:06.445584 7f5c973a57a0 20 read_log 192888'462 (192888'451) delete 10403899/benchmark_data_storage0111-ib.pocstor_35625_object4202/head//22 by client.940330.0:35510 2014-08-11 16:31:42.593220 2014-08-13 18:06:06.445606 7f5c973a57a0 20 read_log 192888'463 (192888'452) delete 730f6099/benchmark_data_storage0111-ib.pocstor_35625_object6761/head//22 by client.940330.0:38069 2014-08-11 16:31:43.278783 2014-08-13 18:06:06.445666 7f5c973a57a0 20 read_log 192888'464 (192888'453) delete 27f82899/benchmark_data_storage0111-ib.pocstor_35625_object14667/head//22 by client.940330.0:45975 2014-08-11 16:31:45.363455 2014-08-13 18:06:06.445695 7f5c973a57a0 20 read_log 192888'465 (192888'454) delete 9568e099/benchmark_data_storage0111-ib.pocstor_35625_object20645/head//22 by client.940330.0:51953 2014-08-11 16:31:47.064844 2014-08-13 18:06:06.445717 7f5c973a57a0 20 read_log 192888'466 (192888'455) delete abd87899/benchmark_data_storage0111-ib.pocstor_35625_object23696/head//22 by client.940330.0:55004 2014-08-11 16:31:47.950649 2014-08-13 18:06:06.445739 7f5c973a57a0 20 read_log 192888'467 (192888'456) delete cbc83099/benchmark_data_storage0111-ib.pocstor_35625_object24625/head//22 by client.940330.0:55933 2014-08-11 16:31:48.224016 2014-08-13 18:06:06.445760 7f5c973a57a0 20 read_log 192888'468 (192888'457) delete f3eac099/benchmark_data_storage0111-ib.pocstor_35625_object24808/head//22 by client.940330.0:56116 2014-08-11 16:31:48.275672 2014-08-13 18:06:06.445781 7f5c973a57a0 20 read_log 192888'469 (192888'458) delete fb915099/benchmark_data_storage0111-ib.pocstor_35625_object26265/head//22 by client.940330.0:57573 2014-08-11 16:31:48.723950 2014-08-13 18:06:06.445803 7f5c973a57a0 20 read_log 192888'470 (192888'459) delete 97271899/benchmark_data_storage0111-ib.pocstor_35625_object30552/head//22 by client.940330.0:61860 2014-08-11 16:31:49.915565 2014-08-13 18:06:06.445825 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 5a58c899/benchmark_data_storage0111-ib.pocstor_64498_object867/head//22 by client.1049635.0:868 2014-08-12 17:17:19.916802 2014-08-13 18:06:06.445846 7f5c973a57a0 20 read_log 194215'472 (0'0) modify f18bb899/benchmark_data_storage0111-ib.pocstor_64498_object4334/head//22 by client.1049635.0:4335 2014-08-12 17:17:31.042066 2014-08-13 18:06:06.445868 7f5c973a57a0 20 read_log 194215'473 (0'0) modify ee8c1899/benchmark_data_storage0111-ib.pocstor_64498_object4463/head//22 by client.1049635.0:4464 2014-08-12 17:17:31.399839 2014-08-13 18:06:06.445889 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 7bd8a099/benchmark_data_storage0111-ib.pocstor_64498_object12364/head//22 by client.1049635.0:12365 2014-08-12 17:17:55.523666 2014-08-13 18:06:06.445910 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 8c4a5099/benchmark_data_storage0111-ib.pocstor_64498_object12958/head//22 by client.1049635.0:12959 2014-08-12 17:17:57.236712 2014-08-13 18:06:06.445932 7f5c973a57a0 20 read_log 194215'476 (0'0) modify e4bb2899/benchmark_data_storage0111-ib.pocstor_64498_object13516/head//22 by client.1049635.0:13517 2014-08-12 17:17:59.033256 2014-08-13 18:06:06.445953 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 9c431899/benchmark_data_storage0111-ib.pocstor_64498_object13576/head//22 by client.1049635.0:13577 2014-08-12 17:17:59.278724 2014-08-13 18:06:06.445974 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 2319b899/benchmark_data_storage0111-ib.pocstor_64498_object20266/head//22 by client.1049635.0:20267 2014-08-12 17:18:23.787060 2014-08-13 18:06:06.445996 7f5c973a57a0 20 read_log 194215'479 (0'0) modify ed8e8099/benchmark_data_storage0111-ib.pocstor_64498_object23250/head//22 by client.1049635.0:23251 2014-08-12 17:18:35.061591 2014-08-13 18:06:06.446020 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 2e3d9899/benchmark_data_storage0111-ib.pocstor_64498_object24116/head//22 by client.1049635.0:24117 2014-08-12 17:18:38.284606 2014-08-13 18:06:06.446042 7f5c973a57a0 20 read_log 194215'481 (0'0) modify bcb54899/benchmark_data_storage0111-ib.pocstor_64498_object24337/head//22 by client.1049635.0:24338 2014-08-12 17:18:39.112025 2014-08-13 18:06:06.446066 7f5c973a57a0 20 read_log 194215'482 (0'0) modify f1a02099/benchmark_data_storage0111-ib.pocstor_64498_object24577/head//22 by client.1049635.0:24578 2014-08-12 17:18:40.007564 2014-08-13 18:06:06.446102 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 467bb899/benchmark_data_storage0111-ib.pocstor_64498_object25821/head//22 by client.1049635.0:25822 2014-08-12 17:18:44.759396 2014-08-13 18:06:06.446129 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 14320099/benchmark_data_storage0111-ib.pocstor_64498_object26372/head//22 by client.1049635.0:26373 2014-08-12 17:18:46.803700 2014-08-13 18:06:06.446160 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 56034899/benchmark_data_storage0111-ib.pocstor_64498_object27271/head//22 by client.1049635.0:27272 2014-08-12 17:18:50.156872 2014-08-13 18:06:06.446183 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 7d537099/benchmark_data_storage0111-ib.pocstor_64498_object29215/head//22 by client.1049635.0:29216 2014-08-12 17:18:57.392101 2014-08-13 18:06:06.446205 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 89178099/benchmark_data_storage0111-ib.pocstor_64498_object31705/head//22 by client.1049635.0:31706 2014-08-12 17:19:07.432383 2014-08-13 18:06:06.446226 7f5c973a57a0 20 read_log 194215'488 (194215'471) delete 5a58c899/benchmark_data_storage0111-ib.pocstor_64498_object867/head//22 by client.1049635.0:35070 2014-08-12 17:19:18.271290 2014-08-13 18:06:06.446248 7f5c973a57a0 20 read_log 194215'489 (194215'472) delete f18bb899/benchmark_data_storage0111-ib.pocstor_64498_object4334/head//22 by client.1049635.0:38537 2014-08-12 17:19:19.382495 2014-08-13 18:06:06.446270 7f5c973a57a0 20 read_log 194215'490 (194215'473) delete ee8c1899/benchmark_data_storage0111-ib.pocstor_64498_object4463/head//22 by client.1049635.0:38666 2014-08-12 17:19:19.426154 2014-08-13 18:06:06.446291 7f5c973a57a0 20 read_log 194215'491 (194215'474) delete 7bd8a099/benchmark_data_storage0111-ib.pocstor_64498_object12364/head//22 by client.1049635.0:46567 2014-08-12 17:19:22.011447 2014-08-13 18:06:06.446313 7f5c973a57a0 20 read_log 194215'492 (194215'475) delete 8c4a5099/benchmark_data_storage0111-ib.pocstor_64498_object12958/head//22 by client.1049635.0:47161 2014-08-12 17:19:22.199112 2014-08-13 18:06:06.446339 7f5c973a57a0 20 read_log 194215'493 (194215'476) delete e4bb2899/benchmark_data_storage0111-ib.pocstor_64498_object13516/head//22 by client.1049635.0:47719 2014-08-12 17:19:22.405676 2014-08-13 18:06:06.446361 7f5c973a57a0 20 read_log 194215'494 (194215'477) delete 9c431899/benchmark_data_storage0111-ib.pocstor_64498_object13576/head//22 by client.1049635.0:47779 2014-08-12 17:19:22.431245 2014-08-13 18:06:06.446392 7f5c973a57a0 20 read_log 194215'495 (194215'478) delete 2319b899/benchmark_data_storage0111-ib.pocstor_64498_object20266/head//22 by client.1049635.0:54469 2014-08-12 17:19:24.620091 2014-08-13 18:06:06.446419 7f5c973a57a0 20 read_log 194215'496 (194215'479) delete ed8e8099/benchmark_data_storage0111-ib.pocstor_64498_object23250/head//22 by client.1049635.0:57453 2014-08-12 17:19:25.708866 2014-08-13 18:06:06.446444 7f5c973a57a0 20 read_log 194215'497 (194215'480) delete 2e3d9899/benchmark_data_storage0111-ib.pocstor_64498_object24116/head//22 by client.1049635.0:58319 2014-08-12 17:19:26.002724 2014-08-13 18:06:06.446472 7f5c973a57a0 20 read_log 194215'498 (194215'481) delete bcb54899/benchmark_data_storage0111-ib.pocstor_64498_object24337/head//22 by client.1049635.0:58540 2014-08-12 17:19:26.078611 2014-08-13 18:06:06.446517 7f5c973a57a0 20 read_log 194215'499 (194215'482) delete f1a02099/benchmark_data_storage0111-ib.pocstor_64498_object24577/head//22 by client.1049635.0:58780 2014-08-12 17:19:26.165694 2014-08-13 18:06:06.446545 7f5c973a57a0 20 read_log 194215'500 (194215'483) delete 467bb899/benchmark_data_storage0111-ib.pocstor_64498_object25821/head//22 by client.1049635.0:60024 2014-08-12 17:19:26.597681 2014-08-13 18:06:06.446578 7f5c973a57a0 20 read_log 194215'501 (194215'484) delete 14320099/benchmark_data_storage0111-ib.pocstor_64498_object26372/head//22 by client.1049635.0:60575 2014-08-12 17:19:26.782295 2014-08-13 18:06:06.446605 7f5c973a57a0 20 read_log 194215'502 (194215'485) delete 56034899/benchmark_data_storage0111-ib.pocstor_64498_object27271/head//22 by client.1049635.0:61474 2014-08-12 17:19:27.097012 2014-08-13 18:06:06.446640 7f5c973a57a0 20 read_log 194215'503 (194215'486) delete 7d537099/benchmark_data_storage0111-ib.pocstor_64498_object29215/head//22 by client.1049635.0:63418 2014-08-12 17:19:27.781807 2014-08-13 18:06:06.446685 7f5c973a57a0 20 read_log 194215'504 (194215'487) delete 89178099/benchmark_data_storage0111-ib.pocstor_64498_object31705/head//22 by client.1049635.0:65908 2014-08-12 17:19:28.642175 2014-08-13 18:06:06.446870 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.446905 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=0 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.446933 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=0 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003254 0 0.000000 2014-08-13 18:06:06.446954 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=0 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.446970 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=0 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.446986 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'504], crt=191648'365) 2014-08-13 18:06:06.447003 7f5c973a57a0 10 osd.4 194487 pgid 22.f3 coll 22.f3_head 2014-08-13 18:06:06.447313 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.f3 2014-08-13 18:06:06.447331 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.447388 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.f3(unlocked)] enter Initial 2014-08-13 18:06:06.447433 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.f3(unlocked)] enter NotTrimming 2014-08-13 18:06:06.447738 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.447937 7f5c973a57a0 20 read_log 191648'159 (0'0) modify 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 by client.273158.0:10812 2014-07-30 14:09:40.255053 2014-08-13 18:06:06.447974 7f5c973a57a0 20 read_log 191648'163 (0'0) modify 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 by client.273158.0:11293 2014-07-30 14:09:43.480720 2014-08-13 18:06:06.447997 7f5c973a57a0 20 read_log 191648'219 (0'0) modify e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 by client.273158.0:14986 2014-07-30 14:10:02.628657 2014-08-13 18:06:06.448019 7f5c973a57a0 20 read_log 191648'326 (0'0) modify 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 by client.273158.0:21658 2014-07-30 14:10:41.075122 2014-08-13 18:06:06.448041 7f5c973a57a0 20 read_log 191648'332 (0'0) modify f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 by client.273158.0:22197 2014-07-30 14:10:44.963874 2014-08-13 18:06:06.448063 7f5c973a57a0 20 read_log 191727'344 (0'0) modify b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 by client.275175.0:772 2014-07-30 14:18:36.498159 2014-08-13 18:06:06.448084 7f5c973a57a0 20 read_log 191727'347 (0'0) modify 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 by client.275175.0:1539 2014-07-30 14:18:39.343225 2014-08-13 18:06:06.448122 7f5c973a57a0 20 read_log 191727'351 (0'0) modify dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 by client.275175.0:2722 2014-07-30 14:18:43.326956 2014-08-13 18:06:06.448161 7f5c973a57a0 20 read_log 191727'353 (0'0) modify 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 by client.275175.0:3410 2014-07-30 14:18:45.512736 2014-08-13 18:06:06.448188 7f5c973a57a0 20 read_log 191727'355 (0'0) modify bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 by client.275175.0:5007 2014-07-30 14:18:50.370849 2014-08-13 18:06:06.448210 7f5c973a57a0 20 read_log 191727'357 (0'0) modify 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 by client.275175.0:5814 2014-07-30 14:18:52.707995 2014-08-13 18:06:06.448232 7f5c973a57a0 20 read_log 191727'360 (0'0) modify d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 by client.275175.0:6318 2014-07-30 14:18:54.202712 2014-08-13 18:06:06.448253 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 by client.275175.0:14781 2014-07-30 14:19:20.075602 2014-08-13 18:06:06.448274 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 by client.275175.0:15186 2014-07-30 14:19:21.330810 2014-08-13 18:06:06.448295 7f5c973a57a0 20 read_log 191727'377 (0'0) modify c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 by client.275175.0:18666 2014-07-30 14:19:31.947870 2014-08-13 18:06:06.448317 7f5c973a57a0 20 read_log 191727'378 (0'0) modify aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 by client.275175.0:20665 2014-07-30 14:19:38.168090 2014-08-13 18:06:06.448338 7f5c973a57a0 20 read_log 191727'380 (0'0) modify a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 by client.275175.0:20875 2014-07-30 14:19:38.772419 2014-08-13 18:06:06.448367 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 by client.275175.0:22622 2014-07-30 14:19:44.766224 2014-08-13 18:06:06.448394 7f5c973a57a0 20 read_log 191727'388 (0'0) modify a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 by client.275175.0:23630 2014-07-30 14:19:47.790262 2014-08-13 18:06:06.448415 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 by client.275175.0:23748 2014-07-30 14:19:48.121416 2014-08-13 18:06:06.448437 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 by client.275175.0:24358 2014-07-30 14:19:49.829737 2014-08-13 18:06:06.448458 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 by client.275175.0:26651 2014-07-30 14:19:56.778705 2014-08-13 18:06:06.448507 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 by client.275175.0:27046 2014-07-30 14:19:57.938973 2014-08-13 18:06:06.448530 7f5c973a57a0 20 read_log 191727'397 (0'0) modify acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 by client.275175.0:27158 2014-07-30 14:19:58.259424 2014-08-13 18:06:06.448576 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 by client.275175.0:27195 2014-07-30 14:19:58.367206 2014-08-13 18:06:06.448599 7f5c973a57a0 20 read_log 191727'405 (0'0) modify f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 by client.275175.0:31675 2014-07-30 14:20:11.938008 2014-08-13 18:06:06.448621 7f5c973a57a0 20 read_log 191727'409 (0'0) modify 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 by client.275175.0:35270 2014-07-30 14:20:22.784675 2014-08-13 18:06:06.448675 7f5c973a57a0 20 read_log 191727'414 (0'0) modify aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 by client.275175.0:37144 2014-07-30 14:20:28.774611 2014-08-13 18:06:06.448705 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 by client.275175.0:37255 2014-07-30 14:20:29.080004 2014-08-13 18:06:06.448730 7f5c973a57a0 20 read_log 192888'417 (0'0) modify ede050f3/benchmark_data_storage0111-ib.pocstor_35625_object102/head//22 by client.940330.0:103 2014-08-11 16:29:40.940627 2014-08-13 18:06:06.448762 7f5c973a57a0 20 read_log 192888'418 (0'0) modify 93d410f3/benchmark_data_storage0111-ib.pocstor_35625_object1085/head//22 by client.940330.0:1086 2014-08-11 16:29:45.110786 2014-08-13 18:06:06.448789 7f5c973a57a0 20 read_log 192888'419 (0'0) modify b955e0f3/benchmark_data_storage0111-ib.pocstor_35625_object1305/head//22 by client.940330.0:1306 2014-08-11 16:29:45.926415 2014-08-13 18:06:06.448824 7f5c973a57a0 20 read_log 192888'420 (0'0) modify ebaaa0f3/benchmark_data_storage0111-ib.pocstor_35625_object1953/head//22 by client.940330.0:1954 2014-08-11 16:29:48.364829 2014-08-13 18:06:06.448856 7f5c973a57a0 20 read_log 192888'421 (0'0) modify 8387f0f3/benchmark_data_storage0111-ib.pocstor_35625_object2593/head//22 by client.940330.0:2594 2014-08-11 16:29:50.836801 2014-08-13 18:06:06.448879 7f5c973a57a0 20 read_log 192888'422 (0'0) modify 5df5f8f3/benchmark_data_storage0111-ib.pocstor_35625_object2757/head//22 by client.940330.0:2758 2014-08-11 16:29:51.464343 2014-08-13 18:06:06.448909 7f5c973a57a0 20 read_log 192888'423 (0'0) modify 69a528f3/benchmark_data_storage0111-ib.pocstor_35625_object4192/head//22 by client.940330.0:4193 2014-08-11 16:29:57.030578 2014-08-13 18:06:06.448946 7f5c973a57a0 20 read_log 192888'424 (0'0) modify 76a6c0f3/benchmark_data_storage0111-ib.pocstor_35625_object6937/head//22 by client.940330.0:6938 2014-08-11 16:30:07.096967 2014-08-13 18:06:06.448974 7f5c973a57a0 20 read_log 192888'425 (0'0) modify 438d80f3/benchmark_data_storage0111-ib.pocstor_35625_object8266/head//22 by client.940330.0:8267 2014-08-11 16:30:11.770462 2014-08-13 18:06:06.449004 7f5c973a57a0 20 read_log 192888'426 (0'0) modify 8f1f40f3/benchmark_data_storage0111-ib.pocstor_35625_object10408/head//22 by client.940330.0:10409 2014-08-11 16:30:19.334590 2014-08-13 18:06:06.449028 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 72d960f3/benchmark_data_storage0111-ib.pocstor_35625_object12528/head//22 by client.940330.0:12529 2014-08-11 16:30:27.998239 2014-08-13 18:06:06.449058 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 5b4b58f3/benchmark_data_storage0111-ib.pocstor_35625_object13002/head//22 by client.940330.0:13003 2014-08-11 16:30:29.799170 2014-08-13 18:06:06.449418 7f5c973a57a0 20 read_log 192888'429 (0'0) modify f07590f3/benchmark_data_storage0111-ib.pocstor_35625_object16269/head//22 by client.940330.0:16270 2014-08-11 16:30:42.235521 2014-08-13 18:06:06.449435 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 349030f3/benchmark_data_storage0111-ib.pocstor_35625_object16327/head//22 by client.940330.0:16328 2014-08-11 16:30:42.450013 2014-08-13 18:06:06.449470 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 9da710f3/benchmark_data_storage0111-ib.pocstor_35625_object16516/head//22 by client.940330.0:16517 2014-08-11 16:30:43.155094 2014-08-13 18:06:06.449485 7f5c973a57a0 20 read_log 192888'432 (0'0) modify df1ea8f3/benchmark_data_storage0111-ib.pocstor_35625_object16937/head//22 by client.940330.0:16938 2014-08-11 16:30:44.730897 2014-08-13 18:06:06.449498 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 9a9b90f3/benchmark_data_storage0111-ib.pocstor_35625_object17532/head//22 by client.940330.0:17533 2014-08-11 16:30:46.957078 2014-08-13 18:06:06.449527 7f5c973a57a0 20 read_log 192888'434 (0'0) modify e34040f3/benchmark_data_storage0111-ib.pocstor_35625_object19743/head//22 by client.940330.0:19744 2014-08-11 16:30:55.787083 2014-08-13 18:06:06.449544 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 800ba8f3/benchmark_data_storage0111-ib.pocstor_35625_object24244/head//22 by client.940330.0:24245 2014-08-11 16:31:13.039710 2014-08-13 18:06:06.449556 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 23a640f3/benchmark_data_storage0111-ib.pocstor_35625_object25406/head//22 by client.940330.0:25407 2014-08-11 16:31:17.475217 2014-08-13 18:06:06.449570 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 7c0760f3/benchmark_data_storage0111-ib.pocstor_35625_object25751/head//22 by client.940330.0:25752 2014-08-11 16:31:18.814698 2014-08-13 18:06:06.449582 7f5c973a57a0 20 read_log 192888'438 (0'0) modify df9d80f3/benchmark_data_storage0111-ib.pocstor_35625_object25874/head//22 by client.940330.0:25875 2014-08-11 16:31:19.297911 2014-08-13 18:06:06.449595 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 3d4dc0f3/benchmark_data_storage0111-ib.pocstor_35625_object29385/head//22 by client.940330.0:29386 2014-08-11 16:31:33.134814 2014-08-13 18:06:06.449613 7f5c973a57a0 20 read_log 192888'440 (0'0) modify a96ce0f3/benchmark_data_storage0111-ib.pocstor_35625_object30850/head//22 by client.940330.0:30851 2014-08-11 16:31:38.656249 2014-08-13 18:06:06.449626 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 4f5540f3/benchmark_data_storage0111-ib.pocstor_35625_object31154/head//22 by client.940330.0:31155 2014-08-11 16:31:39.800042 2014-08-13 18:06:06.449638 7f5c973a57a0 20 read_log 192888'442 (192888'417) delete ede050f3/benchmark_data_storage0111-ib.pocstor_35625_object102/head//22 by client.940330.0:31410 2014-08-11 16:31:41.480819 2014-08-13 18:06:06.449651 7f5c973a57a0 20 read_log 192888'443 (192888'418) delete 93d410f3/benchmark_data_storage0111-ib.pocstor_35625_object1085/head//22 by client.940330.0:32393 2014-08-11 16:31:41.771030 2014-08-13 18:06:06.449664 7f5c973a57a0 20 read_log 192888'444 (192888'419) delete b955e0f3/benchmark_data_storage0111-ib.pocstor_35625_object1305/head//22 by client.940330.0:32613 2014-08-11 16:31:41.821440 2014-08-13 18:06:06.449676 7f5c973a57a0 20 read_log 192888'445 (192888'420) delete ebaaa0f3/benchmark_data_storage0111-ib.pocstor_35625_object1953/head//22 by client.940330.0:33261 2014-08-11 16:31:41.980003 2014-08-13 18:06:06.449689 7f5c973a57a0 20 read_log 192888'446 (192888'421) delete 8387f0f3/benchmark_data_storage0111-ib.pocstor_35625_object2593/head//22 by client.940330.0:33901 2014-08-11 16:31:42.143719 2014-08-13 18:06:06.449701 7f5c973a57a0 20 read_log 192888'447 (192888'422) delete 5df5f8f3/benchmark_data_storage0111-ib.pocstor_35625_object2757/head//22 by client.940330.0:34065 2014-08-11 16:31:42.185179 2014-08-13 18:06:06.449713 7f5c973a57a0 20 read_log 192888'448 (192888'423) delete 69a528f3/benchmark_data_storage0111-ib.pocstor_35625_object4192/head//22 by client.940330.0:35500 2014-08-11 16:31:42.590687 2014-08-13 18:06:06.449725 7f5c973a57a0 20 read_log 192888'449 (192888'424) delete 76a6c0f3/benchmark_data_storage0111-ib.pocstor_35625_object6937/head//22 by client.940330.0:38245 2014-08-11 16:31:43.319425 2014-08-13 18:06:06.449738 7f5c973a57a0 20 read_log 192888'450 (192888'425) delete 438d80f3/benchmark_data_storage0111-ib.pocstor_35625_object8266/head//22 by client.940330.0:39574 2014-08-11 16:31:43.658641 2014-08-13 18:06:06.449772 7f5c973a57a0 20 read_log 192888'451 (192888'426) delete 8f1f40f3/benchmark_data_storage0111-ib.pocstor_35625_object10408/head//22 by client.940330.0:41716 2014-08-11 16:31:44.227474 2014-08-13 18:06:06.449785 7f5c973a57a0 20 read_log 192888'452 (192888'427) delete 72d960f3/benchmark_data_storage0111-ib.pocstor_35625_object12528/head//22 by client.940330.0:43836 2014-08-11 16:31:44.790119 2014-08-13 18:06:06.449798 7f5c973a57a0 20 read_log 192888'453 (192888'428) delete 5b4b58f3/benchmark_data_storage0111-ib.pocstor_35625_object13002/head//22 by client.940330.0:44310 2014-08-11 16:31:44.918773 2014-08-13 18:06:06.449817 7f5c973a57a0 20 read_log 192888'454 (192888'429) delete f07590f3/benchmark_data_storage0111-ib.pocstor_35625_object16269/head//22 by client.940330.0:47577 2014-08-11 16:31:45.796495 2014-08-13 18:06:06.449832 7f5c973a57a0 20 read_log 192888'455 (192888'430) delete 349030f3/benchmark_data_storage0111-ib.pocstor_35625_object16327/head//22 by client.940330.0:47635 2014-08-11 16:31:45.810418 2014-08-13 18:06:06.449845 7f5c973a57a0 20 read_log 192888'456 (192888'431) delete 9da710f3/benchmark_data_storage0111-ib.pocstor_35625_object16516/head//22 by client.940330.0:47824 2014-08-11 16:31:45.858683 2014-08-13 18:06:06.449858 7f5c973a57a0 20 read_log 192888'457 (192888'432) delete df1ea8f3/benchmark_data_storage0111-ib.pocstor_35625_object16937/head//22 by client.940330.0:48245 2014-08-11 16:31:45.970799 2014-08-13 18:06:06.449870 7f5c973a57a0 20 read_log 192888'458 (192888'433) delete 9a9b90f3/benchmark_data_storage0111-ib.pocstor_35625_object17532/head//22 by client.940330.0:48840 2014-08-11 16:31:46.137092 2014-08-13 18:06:06.449882 7f5c973a57a0 20 read_log 192888'459 (192888'434) delete e34040f3/benchmark_data_storage0111-ib.pocstor_35625_object19743/head//22 by client.940330.0:51051 2014-08-11 16:31:46.793475 2014-08-13 18:06:06.449894 7f5c973a57a0 20 read_log 192888'460 (192888'435) delete 800ba8f3/benchmark_data_storage0111-ib.pocstor_35625_object24244/head//22 by client.940330.0:55552 2014-08-11 16:31:48.112868 2014-08-13 18:06:06.449906 7f5c973a57a0 20 read_log 192888'461 (192888'436) delete 23a640f3/benchmark_data_storage0111-ib.pocstor_35625_object25406/head//22 by client.940330.0:56714 2014-08-11 16:31:48.469191 2014-08-13 18:06:06.449919 7f5c973a57a0 20 read_log 192888'462 (192888'437) delete 7c0760f3/benchmark_data_storage0111-ib.pocstor_35625_object25751/head//22 by client.940330.0:57059 2014-08-11 16:31:48.573741 2014-08-13 18:06:06.449931 7f5c973a57a0 20 read_log 192888'463 (192888'438) delete df9d80f3/benchmark_data_storage0111-ib.pocstor_35625_object25874/head//22 by client.940330.0:57182 2014-08-11 16:31:48.609202 2014-08-13 18:06:06.449945 7f5c973a57a0 20 read_log 192888'464 (192888'439) delete 3d4dc0f3/benchmark_data_storage0111-ib.pocstor_35625_object29385/head//22 by client.940330.0:60693 2014-08-11 16:31:49.579925 2014-08-13 18:06:06.449959 7f5c973a57a0 20 read_log 192888'465 (192888'440) delete a96ce0f3/benchmark_data_storage0111-ib.pocstor_35625_object30850/head//22 by client.940330.0:62158 2014-08-11 16:31:50.008413 2014-08-13 18:06:06.449992 7f5c973a57a0 20 read_log 192888'466 (192888'441) delete 4f5540f3/benchmark_data_storage0111-ib.pocstor_35625_object31154/head//22 by client.940330.0:62462 2014-08-11 16:31:50.106344 2014-08-13 18:06:06.450007 7f5c973a57a0 20 read_log 194215'467 (0'0) modify c933f8f3/benchmark_data_storage0111-ib.pocstor_64498_object4282/head//22 by client.1049635.0:4283 2014-08-12 17:17:30.900696 2014-08-13 18:06:06.450022 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 5bbf98f3/benchmark_data_storage0111-ib.pocstor_64498_object5387/head//22 by client.1049635.0:5388 2014-08-12 17:17:34.120280 2014-08-13 18:06:06.450034 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 2056f0f3/benchmark_data_storage0111-ib.pocstor_64498_object8530/head//22 by client.1049635.0:8531 2014-08-12 17:17:43.660009 2014-08-13 18:06:06.450048 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 7400f0f3/benchmark_data_storage0111-ib.pocstor_64498_object10337/head//22 by client.1049635.0:10338 2014-08-12 17:17:48.885257 2014-08-13 18:06:06.450061 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 2fbb90f3/benchmark_data_storage0111-ib.pocstor_64498_object12591/head//22 by client.1049635.0:12592 2014-08-12 17:17:56.146414 2014-08-13 18:06:06.450075 7f5c973a57a0 20 read_log 194215'472 (0'0) modify dcde18f3/benchmark_data_storage0111-ib.pocstor_64498_object12798/head//22 by client.1049635.0:12799 2014-08-12 17:17:56.738055 2014-08-13 18:06:06.450111 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 8f2420f3/benchmark_data_storage0111-ib.pocstor_64498_object15056/head//22 by client.1049635.0:15057 2014-08-12 17:18:04.152103 2014-08-13 18:06:06.450127 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 6f2f88f3/benchmark_data_storage0111-ib.pocstor_64498_object15434/head//22 by client.1049635.0:15435 2014-08-12 17:18:05.415548 2014-08-13 18:06:06.450141 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 4f6b70f3/benchmark_data_storage0111-ib.pocstor_64498_object16535/head//22 by client.1049635.0:16536 2014-08-12 17:18:09.595642 2014-08-13 18:06:06.450162 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 1b7b68f3/benchmark_data_storage0111-ib.pocstor_64498_object19505/head//22 by client.1049635.0:19506 2014-08-12 17:18:20.926361 2014-08-13 18:06:06.450175 7f5c973a57a0 20 read_log 194215'477 (0'0) modify d8a7b0f3/benchmark_data_storage0111-ib.pocstor_64498_object22023/head//22 by client.1049635.0:22024 2014-08-12 17:18:30.415875 2014-08-13 18:06:06.450187 7f5c973a57a0 20 read_log 194215'478 (0'0) modify ded0d0f3/benchmark_data_storage0111-ib.pocstor_64498_object23683/head//22 by client.1049635.0:23684 2014-08-12 17:18:36.678849 2014-08-13 18:06:06.450202 7f5c973a57a0 20 read_log 194215'479 (0'0) modify ecd880f3/benchmark_data_storage0111-ib.pocstor_64498_object24505/head//22 by client.1049635.0:24506 2014-08-12 17:18:39.734691 2014-08-13 18:06:06.450214 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 8594a8f3/benchmark_data_storage0111-ib.pocstor_64498_object24824/head//22 by client.1049635.0:24825 2014-08-12 17:18:40.951288 2014-08-13 18:06:06.450227 7f5c973a57a0 20 read_log 194215'481 (0'0) modify c3b8d0f3/benchmark_data_storage0111-ib.pocstor_64498_object24852/head//22 by client.1049635.0:24853 2014-08-12 17:18:41.103829 2014-08-13 18:06:06.450239 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 72dd18f3/benchmark_data_storage0111-ib.pocstor_64498_object31703/head//22 by client.1049635.0:31704 2014-08-12 17:19:07.424649 2014-08-13 18:06:06.450253 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 288d68f3/benchmark_data_storage0111-ib.pocstor_64498_object32568/head//22 by client.1049635.0:32569 2014-08-12 17:19:10.753373 2014-08-13 18:06:06.450266 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 5d2d68f3/benchmark_data_storage0111-ib.pocstor_64498_object33193/head//22 by client.1049635.0:33194 2014-08-12 17:19:13.120862 2014-08-13 18:06:06.450280 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 4cab58f3/benchmark_data_storage0111-ib.pocstor_64498_object33522/head//22 by client.1049635.0:33523 2014-08-12 17:19:14.368449 2014-08-13 18:06:06.450292 7f5c973a57a0 20 read_log 194215'486 (0'0) modify c3f9d0f3/benchmark_data_storage0111-ib.pocstor_64498_object33780/head//22 by client.1049635.0:33781 2014-08-12 17:19:15.348101 2014-08-13 18:06:06.450310 7f5c973a57a0 20 read_log 194215'487 (194215'467) delete c933f8f3/benchmark_data_storage0111-ib.pocstor_64498_object4282/head//22 by client.1049635.0:38485 2014-08-12 17:19:19.366885 2014-08-13 18:06:06.450323 7f5c973a57a0 20 read_log 194215'488 (194215'468) delete 5bbf98f3/benchmark_data_storage0111-ib.pocstor_64498_object5387/head//22 by client.1049635.0:39590 2014-08-12 17:19:19.734019 2014-08-13 18:06:06.450337 7f5c973a57a0 20 read_log 194215'489 (194215'469) delete 2056f0f3/benchmark_data_storage0111-ib.pocstor_64498_object8530/head//22 by client.1049635.0:42733 2014-08-12 17:19:20.770215 2014-08-13 18:06:06.450355 7f5c973a57a0 20 read_log 194215'490 (194215'470) delete 7400f0f3/benchmark_data_storage0111-ib.pocstor_64498_object10337/head//22 by client.1049635.0:44540 2014-08-12 17:19:21.371302 2014-08-13 18:06:06.450368 7f5c973a57a0 20 read_log 194215'491 (194215'471) delete 2fbb90f3/benchmark_data_storage0111-ib.pocstor_64498_object12591/head//22 by client.1049635.0:46794 2014-08-12 17:19:22.082017 2014-08-13 18:06:06.450380 7f5c973a57a0 20 read_log 194215'492 (194215'472) delete dcde18f3/benchmark_data_storage0111-ib.pocstor_64498_object12798/head//22 by client.1049635.0:47001 2014-08-12 17:19:22.147529 2014-08-13 18:06:06.450404 7f5c973a57a0 20 read_log 194215'493 (194215'473) delete 8f2420f3/benchmark_data_storage0111-ib.pocstor_64498_object15056/head//22 by client.1049635.0:49259 2014-08-12 17:19:22.891587 2014-08-13 18:06:06.450419 7f5c973a57a0 20 read_log 194215'494 (194215'474) delete 6f2f88f3/benchmark_data_storage0111-ib.pocstor_64498_object15434/head//22 by client.1049635.0:49637 2014-08-12 17:19:23.012225 2014-08-13 18:06:06.450445 7f5c973a57a0 20 read_log 194215'495 (194215'475) delete 4f6b70f3/benchmark_data_storage0111-ib.pocstor_64498_object16535/head//22 by client.1049635.0:50738 2014-08-12 17:19:23.362654 2014-08-13 18:06:06.450460 7f5c973a57a0 20 read_log 194215'496 (194215'476) delete 1b7b68f3/benchmark_data_storage0111-ib.pocstor_64498_object19505/head//22 by client.1049635.0:53708 2014-08-12 17:19:24.351261 2014-08-13 18:06:06.450473 7f5c973a57a0 20 read_log 194215'497 (194215'477) delete d8a7b0f3/benchmark_data_storage0111-ib.pocstor_64498_object22023/head//22 by client.1049635.0:56226 2014-08-12 17:19:25.245997 2014-08-13 18:06:06.450496 7f5c973a57a0 20 read_log 194215'498 (194215'478) delete ded0d0f3/benchmark_data_storage0111-ib.pocstor_64498_object23683/head//22 by client.1049635.0:57886 2014-08-12 17:19:25.860231 2014-08-13 18:06:06.450509 7f5c973a57a0 20 read_log 194215'499 (194215'479) delete ecd880f3/benchmark_data_storage0111-ib.pocstor_64498_object24505/head//22 by client.1049635.0:58708 2014-08-12 17:19:26.137638 2014-08-13 18:06:06.450522 7f5c973a57a0 20 read_log 194215'500 (194215'480) delete 8594a8f3/benchmark_data_storage0111-ib.pocstor_64498_object24824/head//22 by client.1049635.0:59027 2014-08-12 17:19:26.263642 2014-08-13 18:06:06.450541 7f5c973a57a0 20 read_log 194215'501 (194215'481) delete c3b8d0f3/benchmark_data_storage0111-ib.pocstor_64498_object24852/head//22 by client.1049635.0:59055 2014-08-12 17:19:26.274854 2014-08-13 18:06:06.450557 7f5c973a57a0 20 read_log 194215'502 (194215'482) delete 72dd18f3/benchmark_data_storage0111-ib.pocstor_64498_object31703/head//22 by client.1049635.0:65906 2014-08-12 17:19:28.641464 2014-08-13 18:06:06.450570 7f5c973a57a0 20 read_log 194215'503 (194215'483) delete 288d68f3/benchmark_data_storage0111-ib.pocstor_64498_object32568/head//22 by client.1049635.0:66771 2014-08-12 17:19:28.954974 2014-08-13 18:06:06.450582 7f5c973a57a0 20 read_log 194215'504 (194215'484) delete 5d2d68f3/benchmark_data_storage0111-ib.pocstor_64498_object33193/head//22 by client.1049635.0:67396 2014-08-12 17:19:29.182515 2014-08-13 18:06:06.450595 7f5c973a57a0 20 read_log 194215'505 (194215'485) delete 4cab58f3/benchmark_data_storage0111-ib.pocstor_64498_object33522/head//22 by client.1049635.0:67725 2014-08-12 17:19:29.299821 2014-08-13 18:06:06.450607 7f5c973a57a0 20 read_log 194215'506 (194215'486) delete c3f9d0f3/benchmark_data_storage0111-ib.pocstor_64498_object33780/head//22 by client.1049635.0:67983 2014-08-12 17:19:29.398243 2014-08-13 18:06:06.450710 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.450730 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=0 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.450746 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=0 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.003359 0 0.000000 2014-08-13 18:06:06.450758 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=0 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.450767 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=0 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.450776 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'506], crt=191648'343) 2014-08-13 18:06:06.450786 7f5c973a57a0 10 osd.4 194487 pgid 22.113 coll 22.113_head 2014-08-13 18:06:06.450935 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.113 2014-08-13 18:06:06.450945 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.450979 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.113(unlocked)] enter Initial 2014-08-13 18:06:06.450990 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.113(unlocked)] enter NotTrimming 2014-08-13 18:06:06.451177 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.451304 7f5c973a57a0 20 read_log 191648'77 (0'0) modify 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 by client.273158.0:4384 2014-07-30 14:09:08.285865 2014-08-13 18:06:06.451325 7f5c973a57a0 20 read_log 191648'103 (0'0) modify 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 by client.273158.0:5574 2014-07-30 14:09:13.732468 2014-08-13 18:06:06.451338 7f5c973a57a0 20 read_log 191648'105 (0'0) modify 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 by client.273158.0:5622 2014-07-30 14:09:13.925369 2014-08-13 18:06:06.451350 7f5c973a57a0 20 read_log 191648'140 (0'0) modify 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 by client.273158.0:7672 2014-07-30 14:09:24.203072 2014-08-13 18:06:06.451362 7f5c973a57a0 20 read_log 191648'168 (0'0) modify 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 by client.273158.0:9439 2014-07-30 14:09:33.051653 2014-08-13 18:06:06.451375 7f5c973a57a0 20 read_log 191648'177 (0'0) modify 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 by client.273158.0:10164 2014-07-30 14:09:36.668970 2014-08-13 18:06:06.451387 7f5c973a57a0 20 read_log 191648'198 (0'0) modify 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 by client.273158.0:11899 2014-07-30 14:09:46.454073 2014-08-13 18:06:06.451399 7f5c973a57a0 20 read_log 191648'330 (0'0) modify ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 by client.273158.0:19187 2014-07-30 14:10:24.334369 2014-08-13 18:06:06.451411 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 by client.275175.0:527 2014-07-30 14:18:35.577293 2014-08-13 18:06:06.451423 7f5c973a57a0 20 read_log 191727'400 (0'0) modify 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 by client.275175.0:9936 2014-07-30 14:19:04.999388 2014-08-13 18:06:06.451436 7f5c973a57a0 20 read_log 191727'404 (0'0) modify bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 by client.275175.0:12690 2014-07-30 14:19:13.575432 2014-08-13 18:06:06.451449 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 by client.275175.0:13683 2014-07-30 14:19:16.758273 2014-08-13 18:06:06.451474 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 by client.275175.0:15903 2014-07-30 14:19:23.542605 2014-08-13 18:06:06.451491 7f5c973a57a0 20 read_log 191727'417 (0'0) modify de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 by client.275175.0:19817 2014-07-30 14:19:35.653150 2014-08-13 18:06:06.451531 7f5c973a57a0 20 read_log 191727'418 (0'0) modify ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 by client.275175.0:21132 2014-07-30 14:19:39.502002 2014-08-13 18:06:06.451547 7f5c973a57a0 20 read_log 191727'422 (0'0) modify b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 by client.275175.0:22569 2014-07-30 14:19:44.534397 2014-08-13 18:06:06.451559 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 by client.275175.0:22734 2014-07-30 14:19:45.236432 2014-08-13 18:06:06.451571 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 by client.275175.0:23832 2014-07-30 14:19:48.357355 2014-08-13 18:06:06.451584 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 by client.275175.0:26391 2014-07-30 14:19:55.973261 2014-08-13 18:06:06.451596 7f5c973a57a0 20 read_log 191727'436 (0'0) modify d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 by client.275175.0:27708 2014-07-30 14:19:59.812197 2014-08-13 18:06:06.451608 7f5c973a57a0 20 read_log 191727'439 (0'0) modify 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 by client.275175.0:28887 2014-07-30 14:20:03.571262 2014-08-13 18:06:06.451620 7f5c973a57a0 20 read_log 191727'443 (0'0) modify c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 by client.275175.0:29730 2014-07-30 14:20:06.131708 2014-08-13 18:06:06.451632 7f5c973a57a0 20 read_log 191727'445 (0'0) modify 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 by client.275175.0:30639 2014-07-30 14:20:08.959585 2014-08-13 18:06:06.451645 7f5c973a57a0 20 read_log 191727'451 (0'0) modify 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 by client.275175.0:34068 2014-07-30 14:20:19.184755 2014-08-13 18:06:06.451657 7f5c973a57a0 20 read_log 191727'453 (0'0) modify 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 by client.275175.0:34534 2014-07-30 14:20:20.642953 2014-08-13 18:06:06.451669 7f5c973a57a0 20 read_log 191727'462 (0'0) modify d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 by client.275175.0:38324 2014-07-30 14:20:32.165867 2014-08-13 18:06:06.451681 7f5c973a57a0 20 read_log 192674'463 (0'0) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6975 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451694 7f5c973a57a0 20 read_log 192674'464 (192674'463) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6976 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451706 7f5c973a57a0 20 read_log 192674'465 (192674'464) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6977 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451718 7f5c973a57a0 20 read_log 192674'466 (192674'465) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6978 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451730 7f5c973a57a0 20 read_log 192674'467 (192674'466) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6979 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451742 7f5c973a57a0 20 read_log 192674'468 (192674'467) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6980 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451754 7f5c973a57a0 20 read_log 192674'469 (192674'468) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6981 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451767 7f5c973a57a0 20 read_log 192674'470 (192674'469) modify b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 by client.279772.1:6982 2014-07-31 18:03:49.048000 2014-08-13 18:06:06.451789 7f5c973a57a0 20 read_log 192888'471 (0'0) modify a29a7113/benchmark_data_storage0111-ib.pocstor_35625_object487/head//22 by client.940330.0:488 2014-08-11 16:29:42.654200 2014-08-13 18:06:06.451804 7f5c973a57a0 20 read_log 192888'472 (0'0) modify 3ee94913/benchmark_data_storage0111-ib.pocstor_35625_object1034/head//22 by client.940330.0:1035 2014-08-11 16:29:44.920692 2014-08-13 18:06:06.451817 7f5c973a57a0 20 read_log 192888'473 (0'0) modify 5ff60913/benchmark_data_storage0111-ib.pocstor_35625_object1433/head//22 by client.940330.0:1434 2014-08-11 16:29:46.441610 2014-08-13 18:06:06.451831 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 991e8113/benchmark_data_storage0111-ib.pocstor_35625_object4649/head//22 by client.940330.0:4650 2014-08-11 16:29:58.796035 2014-08-13 18:06:06.451845 7f5c973a57a0 20 read_log 192888'475 (0'0) modify 59df0113/benchmark_data_storage0111-ib.pocstor_35625_object6133/head//22 by client.940330.0:6134 2014-08-11 16:30:04.056385 2014-08-13 18:06:06.451857 7f5c973a57a0 20 read_log 192888'476 (0'0) modify 1cbc7113/benchmark_data_storage0111-ib.pocstor_35625_object6435/head//22 by client.940330.0:6436 2014-08-11 16:30:05.117149 2014-08-13 18:06:06.451872 7f5c973a57a0 20 read_log 192888'477 (0'0) modify fff7e913/benchmark_data_storage0111-ib.pocstor_35625_object9788/head//22 by client.940330.0:9789 2014-08-11 16:30:17.088891 2014-08-13 18:06:06.451884 7f5c973a57a0 20 read_log 192888'478 (0'0) modify adc70113/benchmark_data_storage0111-ib.pocstor_35625_object10411/head//22 by client.940330.0:10412 2014-08-11 16:30:19.344429 2014-08-13 18:06:06.451900 7f5c973a57a0 20 read_log 192888'479 (0'0) modify e34aa913/benchmark_data_storage0111-ib.pocstor_35625_object14746/head//22 by client.940330.0:14747 2014-08-11 16:30:36.547234 2014-08-13 18:06:06.451912 7f5c973a57a0 20 read_log 192888'480 (0'0) modify c6825113/benchmark_data_storage0111-ib.pocstor_35625_object15937/head//22 by client.940330.0:15938 2014-08-11 16:30:41.001055 2014-08-13 18:06:06.451925 7f5c973a57a0 20 read_log 192888'481 (0'0) modify ffa21113/benchmark_data_storage0111-ib.pocstor_35625_object21650/head//22 by client.940330.0:21651 2014-08-11 16:31:03.227759 2014-08-13 18:06:06.451937 7f5c973a57a0 20 read_log 192888'482 (0'0) modify ed496113/benchmark_data_storage0111-ib.pocstor_35625_object24733/head//22 by client.940330.0:24734 2014-08-11 16:31:14.838225 2014-08-13 18:06:06.451969 7f5c973a57a0 20 read_log 192888'483 (0'0) modify 1015d113/benchmark_data_storage0111-ib.pocstor_35625_object28967/head//22 by client.940330.0:28968 2014-08-11 16:31:31.461202 2014-08-13 18:06:06.451985 7f5c973a57a0 20 read_log 192888'484 (0'0) modify 4e926913/benchmark_data_storage0111-ib.pocstor_35625_object29849/head//22 by client.940330.0:29850 2014-08-11 16:31:34.888119 2014-08-13 18:06:06.451997 7f5c973a57a0 20 read_log 192888'485 (192888'471) delete a29a7113/benchmark_data_storage0111-ib.pocstor_35625_object487/head//22 by client.940330.0:31795 2014-08-11 16:31:41.619035 2014-08-13 18:06:06.452011 7f5c973a57a0 20 read_log 192888'486 (192888'472) delete 3ee94913/benchmark_data_storage0111-ib.pocstor_35625_object1034/head//22 by client.940330.0:32342 2014-08-11 16:31:41.757850 2014-08-13 18:06:06.452024 7f5c973a57a0 20 read_log 192888'487 (192888'473) delete 5ff60913/benchmark_data_storage0111-ib.pocstor_35625_object1433/head//22 by client.940330.0:32741 2014-08-11 16:31:41.851358 2014-08-13 18:06:06.452038 7f5c973a57a0 20 read_log 192888'488 (192888'474) delete 991e8113/benchmark_data_storage0111-ib.pocstor_35625_object4649/head//22 by client.940330.0:35957 2014-08-11 16:31:42.724099 2014-08-13 18:06:06.452051 7f5c973a57a0 20 read_log 192888'489 (192888'475) delete 59df0113/benchmark_data_storage0111-ib.pocstor_35625_object6133/head//22 by client.940330.0:37441 2014-08-11 16:31:43.112713 2014-08-13 18:06:06.452065 7f5c973a57a0 20 read_log 192888'490 (192888'476) delete 1cbc7113/benchmark_data_storage0111-ib.pocstor_35625_object6435/head//22 by client.940330.0:37743 2014-08-11 16:31:43.191394 2014-08-13 18:06:06.452085 7f5c973a57a0 20 read_log 192888'491 (192888'477) delete fff7e913/benchmark_data_storage0111-ib.pocstor_35625_object9788/head//22 by client.940330.0:41096 2014-08-11 16:31:44.055721 2014-08-13 18:06:06.452100 7f5c973a57a0 20 read_log 192888'492 (192888'478) delete adc70113/benchmark_data_storage0111-ib.pocstor_35625_object10411/head//22 by client.940330.0:41719 2014-08-11 16:31:44.227819 2014-08-13 18:06:06.452112 7f5c973a57a0 20 read_log 192888'493 (192888'479) delete e34aa913/benchmark_data_storage0111-ib.pocstor_35625_object14746/head//22 by client.940330.0:46054 2014-08-11 16:31:45.384329 2014-08-13 18:06:06.452127 7f5c973a57a0 20 read_log 192888'494 (192888'480) delete c6825113/benchmark_data_storage0111-ib.pocstor_35625_object15937/head//22 by client.940330.0:47245 2014-08-11 16:31:45.707721 2014-08-13 18:06:06.452140 7f5c973a57a0 20 read_log 192888'495 (192888'481) delete ffa21113/benchmark_data_storage0111-ib.pocstor_35625_object21650/head//22 by client.940330.0:52958 2014-08-11 16:31:47.336090 2014-08-13 18:06:06.452163 7f5c973a57a0 20 read_log 192888'496 (192888'482) delete ed496113/benchmark_data_storage0111-ib.pocstor_35625_object24733/head//22 by client.940330.0:56041 2014-08-11 16:31:48.254288 2014-08-13 18:06:06.452176 7f5c973a57a0 20 read_log 192888'497 (192888'483) delete 1015d113/benchmark_data_storage0111-ib.pocstor_35625_object28967/head//22 by client.940330.0:60275 2014-08-11 16:31:49.462419 2014-08-13 18:06:06.452189 7f5c973a57a0 20 read_log 192888'498 (192888'484) delete 4e926913/benchmark_data_storage0111-ib.pocstor_35625_object29849/head//22 by client.940330.0:61157 2014-08-11 16:31:49.713518 2014-08-13 18:06:06.452203 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 2dace913/benchmark_data_storage0111-ib.pocstor_64498_object547/head//22 by client.1049635.0:548 2014-08-12 17:17:18.885541 2014-08-13 18:06:06.452216 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 7ad1c113/benchmark_data_storage0111-ib.pocstor_64498_object5043/head//22 by client.1049635.0:5044 2014-08-12 17:17:33.048664 2014-08-13 18:06:06.452228 7f5c973a57a0 20 read_log 194215'501 (0'0) modify d650e113/benchmark_data_storage0111-ib.pocstor_64498_object6498/head//22 by client.1049635.0:6499 2014-08-12 17:17:37.568170 2014-08-13 18:06:06.452240 7f5c973a57a0 20 read_log 194215'502 (0'0) modify a88fe913/benchmark_data_storage0111-ib.pocstor_64498_object9300/head//22 by client.1049635.0:9301 2014-08-12 17:17:45.890116 2014-08-13 18:06:06.452252 7f5c973a57a0 20 read_log 194215'503 (0'0) modify 69ba6113/benchmark_data_storage0111-ib.pocstor_64498_object11501/head//22 by client.1049635.0:11502 2014-08-12 17:17:52.889304 2014-08-13 18:06:06.452267 7f5c973a57a0 20 read_log 194215'504 (0'0) modify c46bc113/benchmark_data_storage0111-ib.pocstor_64498_object12235/head//22 by client.1049635.0:12236 2014-08-12 17:17:55.173580 2014-08-13 18:06:06.452281 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 1cf87913/benchmark_data_storage0111-ib.pocstor_64498_object17919/head//22 by client.1049635.0:17920 2014-08-12 17:18:14.857735 2014-08-13 18:06:06.452299 7f5c973a57a0 20 read_log 194215'506 (0'0) modify 94522913/benchmark_data_storage0111-ib.pocstor_64498_object21505/head//22 by client.1049635.0:21506 2014-08-12 17:18:28.472680 2014-08-13 18:06:06.452312 7f5c973a57a0 20 read_log 194215'507 (0'0) modify fb4bc113/benchmark_data_storage0111-ib.pocstor_64498_object21970/head//22 by client.1049635.0:21971 2014-08-12 17:18:30.219047 2014-08-13 18:06:06.452324 7f5c973a57a0 20 read_log 194215'508 (0'0) modify 5e3e8913/benchmark_data_storage0111-ib.pocstor_64498_object23403/head//22 by client.1049635.0:23404 2014-08-12 17:18:35.627203 2014-08-13 18:06:06.452336 7f5c973a57a0 20 read_log 194215'509 (0'0) modify f49eb913/benchmark_data_storage0111-ib.pocstor_64498_object24983/head//22 by client.1049635.0:24984 2014-08-12 17:18:41.652724 2014-08-13 18:06:06.452356 7f5c973a57a0 20 read_log 194215'510 (0'0) modify ecba113/benchmark_data_storage0111-ib.pocstor_64498_object29063/head//22 by client.1049635.0:29064 2014-08-12 17:18:56.832696 2014-08-13 18:06:06.452371 7f5c973a57a0 20 read_log 194215'511 (0'0) modify 5d317113/benchmark_data_storage0111-ib.pocstor_64498_object29569/head//22 by client.1049635.0:29570 2014-08-12 17:18:58.831244 2014-08-13 18:06:06.452389 7f5c973a57a0 20 read_log 194215'512 (0'0) modify c20d6113/benchmark_data_storage0111-ib.pocstor_64498_object31859/head//22 by client.1049635.0:31860 2014-08-12 17:19:08.020374 2014-08-13 18:06:06.452404 7f5c973a57a0 20 read_log 194215'513 (0'0) modify a561d913/benchmark_data_storage0111-ib.pocstor_64498_object32728/head//22 by client.1049635.0:32729 2014-08-12 17:19:11.360882 2014-08-13 18:06:06.452431 7f5c973a57a0 20 read_log 194215'514 (194215'499) delete 2dace913/benchmark_data_storage0111-ib.pocstor_64498_object547/head//22 by client.1049635.0:34750 2014-08-12 17:19:18.171575 2014-08-13 18:06:06.452444 7f5c973a57a0 20 read_log 194215'515 (194215'500) delete 7ad1c113/benchmark_data_storage0111-ib.pocstor_64498_object5043/head//22 by client.1049635.0:39246 2014-08-12 17:19:19.618790 2014-08-13 18:06:06.452463 7f5c973a57a0 20 read_log 194215'516 (194215'501) delete d650e113/benchmark_data_storage0111-ib.pocstor_64498_object6498/head//22 by client.1049635.0:40701 2014-08-12 17:19:20.116449 2014-08-13 18:06:06.452479 7f5c973a57a0 20 read_log 194215'517 (194215'502) delete a88fe913/benchmark_data_storage0111-ib.pocstor_64498_object9300/head//22 by client.1049635.0:43503 2014-08-12 17:19:21.024506 2014-08-13 18:06:06.452492 7f5c973a57a0 20 read_log 194215'518 (194215'503) delete 69ba6113/benchmark_data_storage0111-ib.pocstor_64498_object11501/head//22 by client.1049635.0:45704 2014-08-12 17:19:21.737115 2014-08-13 18:06:06.452505 7f5c973a57a0 20 read_log 194215'519 (194215'504) delete c46bc113/benchmark_data_storage0111-ib.pocstor_64498_object12235/head//22 by client.1049635.0:46438 2014-08-12 17:19:21.969929 2014-08-13 18:06:06.452526 7f5c973a57a0 20 read_log 194215'520 (194215'505) delete 1cf87913/benchmark_data_storage0111-ib.pocstor_64498_object17919/head//22 by client.1049635.0:52122 2014-08-12 17:19:23.800541 2014-08-13 18:06:06.452539 7f5c973a57a0 20 read_log 194215'521 (194215'506) delete 94522913/benchmark_data_storage0111-ib.pocstor_64498_object21505/head//22 by client.1049635.0:55708 2014-08-12 17:19:25.057474 2014-08-13 18:06:06.452568 7f5c973a57a0 20 read_log 194215'522 (194215'507) delete fb4bc113/benchmark_data_storage0111-ib.pocstor_64498_object21970/head//22 by client.1049635.0:56173 2014-08-12 17:19:25.222178 2014-08-13 18:06:06.452584 7f5c973a57a0 20 read_log 194215'523 (194215'508) delete 5e3e8913/benchmark_data_storage0111-ib.pocstor_64498_object23403/head//22 by client.1049635.0:57606 2014-08-12 17:19:25.763617 2014-08-13 18:06:06.452597 7f5c973a57a0 20 read_log 194215'524 (194215'509) delete f49eb913/benchmark_data_storage0111-ib.pocstor_64498_object24983/head//22 by client.1049635.0:59186 2014-08-12 17:19:26.317695 2014-08-13 18:06:06.452617 7f5c973a57a0 20 read_log 194215'525 (194215'510) delete ecba113/benchmark_data_storage0111-ib.pocstor_64498_object29063/head//22 by client.1049635.0:63266 2014-08-12 17:19:27.727565 2014-08-13 18:06:06.452653 7f5c973a57a0 20 read_log 194215'526 (194215'511) delete 5d317113/benchmark_data_storage0111-ib.pocstor_64498_object29569/head//22 by client.1049635.0:63772 2014-08-12 17:19:27.907729 2014-08-13 18:06:06.452666 7f5c973a57a0 20 read_log 194215'527 (194215'512) delete c20d6113/benchmark_data_storage0111-ib.pocstor_64498_object31859/head//22 by client.1049635.0:66062 2014-08-12 17:19:28.692136 2014-08-13 18:06:06.452678 7f5c973a57a0 20 read_log 194215'528 (194215'513) delete a561d913/benchmark_data_storage0111-ib.pocstor_64498_object32728/head//22 by client.1049635.0:66931 2014-08-12 17:19:29.014971 2014-08-13 18:06:06.452796 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.452816 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=0 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.452831 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=0 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001853 0 0.000000 2014-08-13 18:06:06.452843 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=0 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.452852 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=0 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.452861 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'528], crt=191648'383) 2014-08-13 18:06:06.452871 7f5c973a57a0 10 osd.4 194487 pgid 22.116 coll 22.116_head 2014-08-13 18:06:06.453025 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.116 2014-08-13 18:06:06.453035 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.453061 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.116(unlocked)] enter Initial 2014-08-13 18:06:06.453074 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.116(unlocked)] enter NotTrimming 2014-08-13 18:06:06.453221 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.453387 7f5c973a57a0 20 read_log 191648'85 (0'0) modify 8fdc916/benchmark_data_storage0111-ib.pocstor_40745_object5633/head//22 by client.273158.0:5634 2014-07-30 14:09:13.974314 2014-08-13 18:06:06.453409 7f5c973a57a0 20 read_log 191648'99 (0'0) modify f1132916/benchmark_data_storage0111-ib.pocstor_40745_object6261/head//22 by client.273158.0:6262 2014-07-30 14:09:17.289800 2014-08-13 18:06:06.453422 7f5c973a57a0 20 read_log 191648'118 (0'0) modify 7efac116/benchmark_data_storage0111-ib.pocstor_40745_object6993/head//22 by client.273158.0:6994 2014-07-30 14:09:21.065974 2014-08-13 18:06:06.453435 7f5c973a57a0 20 read_log 191648'122 (0'0) modify 5e6c0916/benchmark_data_storage0111-ib.pocstor_40745_object7304/head//22 by client.273158.0:7305 2014-07-30 14:09:22.292581 2014-08-13 18:06:06.453447 7f5c973a57a0 20 read_log 191648'132 (0'0) modify f6c20116/benchmark_data_storage0111-ib.pocstor_40745_object8285/head//22 by client.273158.0:8286 2014-07-30 14:09:27.764161 2014-08-13 18:06:06.453459 7f5c973a57a0 20 read_log 191648'182 (0'0) modify c621f116/benchmark_data_storage0111-ib.pocstor_40745_object11977/head//22 by client.273158.0:11978 2014-07-30 14:09:46.927074 2014-08-13 18:06:06.453472 7f5c973a57a0 20 read_log 191648'189 (0'0) modify d25fb116/benchmark_data_storage0111-ib.pocstor_40745_object12496/head//22 by client.273158.0:12497 2014-07-30 14:09:49.991885 2014-08-13 18:06:06.453484 7f5c973a57a0 20 read_log 191648'206 (0'0) modify 6fd2e916/benchmark_data_storage0111-ib.pocstor_40745_object13441/head//22 by client.273158.0:13442 2014-07-30 14:09:54.468489 2014-08-13 18:06:06.453496 7f5c973a57a0 20 read_log 191648'214 (0'0) modify d7cd7916/benchmark_data_storage0111-ib.pocstor_40745_object13744/head//22 by client.273158.0:13745 2014-07-30 14:09:55.808656 2014-08-13 18:06:06.453518 7f5c973a57a0 20 read_log 191648'247 (0'0) modify 9a579916/benchmark_data_storage0111-ib.pocstor_40745_object15685/head//22 by client.273158.0:15686 2014-07-30 14:10:05.622621 2014-08-13 18:06:06.453534 7f5c973a57a0 20 read_log 191648'256 (0'0) modify 1d60a116/benchmark_data_storage0111-ib.pocstor_40745_object16275/head//22 by client.273158.0:16276 2014-07-30 14:10:08.961420 2014-08-13 18:06:06.453547 7f5c973a57a0 20 read_log 191648'337 (0'0) modify be72a116/benchmark_data_storage0111-ib.pocstor_40745_object21012/head//22 by client.273158.0:21013 2014-07-30 14:10:37.305279 2014-08-13 18:06:06.453577 7f5c973a57a0 20 read_log 191727'366 (0'0) modify e5e51116/benchmark_data_storage0111-ib.pocstor_51285_object2257/head//22 by client.275175.0:2258 2014-07-30 14:18:41.735075 2014-08-13 18:06:06.453592 7f5c973a57a0 20 read_log 191727'369 (0'0) modify ce6e4916/benchmark_data_storage0111-ib.pocstor_51285_object3050/head//22 by client.275175.0:3051 2014-07-30 14:18:44.384996 2014-08-13 18:06:06.453604 7f5c973a57a0 20 read_log 191727'374 (0'0) modify 1cb1b116/benchmark_data_storage0111-ib.pocstor_51285_object4253/head//22 by client.275175.0:4254 2014-07-30 14:18:48.126636 2014-08-13 18:06:06.453617 7f5c973a57a0 20 read_log 191727'383 (0'0) modify 795f7916/benchmark_data_storage0111-ib.pocstor_51285_object9359/head//22 by client.275175.0:9360 2014-07-30 14:19:03.389468 2014-08-13 18:06:06.453629 7f5c973a57a0 20 read_log 191727'385 (0'0) modify ec53c916/benchmark_data_storage0111-ib.pocstor_51285_object10011/head//22 by client.275175.0:10012 2014-07-30 14:19:05.214295 2014-08-13 18:06:06.453642 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 5b347116/benchmark_data_storage0111-ib.pocstor_51285_object15330/head//22 by client.275175.0:15331 2014-07-30 14:19:21.797340 2014-08-13 18:06:06.453654 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 800d4116/benchmark_data_storage0111-ib.pocstor_51285_object17063/head//22 by client.275175.0:17064 2014-07-30 14:19:27.338250 2014-08-13 18:06:06.453666 7f5c973a57a0 20 read_log 191727'396 (0'0) modify ec971116/benchmark_data_storage0111-ib.pocstor_51285_object17381/head//22 by client.275175.0:17382 2014-07-30 14:19:28.217820 2014-08-13 18:06:06.453678 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 587dc916/benchmark_data_storage0111-ib.pocstor_51285_object18232/head//22 by client.275175.0:18233 2014-07-30 14:19:30.669624 2014-08-13 18:06:06.453690 7f5c973a57a0 20 read_log 191727'408 (0'0) modify de7c3116/benchmark_data_storage0111-ib.pocstor_51285_object23390/head//22 by client.275175.0:23391 2014-07-30 14:19:47.128678 2014-08-13 18:06:06.453702 7f5c973a57a0 20 read_log 191727'417 (0'0) modify c0e70916/benchmark_data_storage0111-ib.pocstor_51285_object26712/head//22 by client.275175.0:26713 2014-07-30 14:19:56.952443 2014-08-13 18:06:06.453714 7f5c973a57a0 20 read_log 191727'419 (0'0) modify 1195b916/benchmark_data_storage0111-ib.pocstor_51285_object28254/head//22 by client.275175.0:28255 2014-07-30 14:20:01.739328 2014-08-13 18:06:06.453726 7f5c973a57a0 20 read_log 191727'428 (0'0) modify 49106916/benchmark_data_storage0111-ib.pocstor_51285_object33237/head//22 by client.275175.0:33238 2014-07-30 14:20:16.558157 2014-08-13 18:06:06.453738 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 3ba6d116/benchmark_data_storage0111-ib.pocstor_51285_object33603/head//22 by client.275175.0:33604 2014-07-30 14:20:17.861400 2014-08-13 18:06:06.453752 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 77a43116/benchmark_data_storage0111-ib.pocstor_51285_object34152/head//22 by client.275175.0:34153 2014-07-30 14:20:19.495304 2014-08-13 18:06:06.453764 7f5c973a57a0 20 read_log 191727'433 (0'0) modify 6ca4a116/benchmark_data_storage0111-ib.pocstor_51285_object34574/head//22 by client.275175.0:34575 2014-07-30 14:20:20.759030 2014-08-13 18:06:06.453783 7f5c973a57a0 20 read_log 191727'438 (0'0) modify 6db26116/benchmark_data_storage0111-ib.pocstor_51285_object38680/head//22 by client.275175.0:38681 2014-07-30 14:20:33.184814 2014-08-13 18:06:06.453796 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 624a4116/benchmark_data_storage0111-ib.pocstor_35625_object1207/head//22 by client.940330.0:1208 2014-08-11 16:29:45.563713 2014-08-13 18:06:06.453810 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 3dc95916/benchmark_data_storage0111-ib.pocstor_35625_object2937/head//22 by client.940330.0:2938 2014-08-11 16:29:52.350322 2014-08-13 18:06:06.453822 7f5c973a57a0 20 read_log 192888'441 (0'0) modify b75d8916/benchmark_data_storage0111-ib.pocstor_35625_object3841/head//22 by client.940330.0:3842 2014-08-11 16:29:55.750183 2014-08-13 18:06:06.453834 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 977ff916/benchmark_data_storage0111-ib.pocstor_35625_object5894/head//22 by client.940330.0:5895 2014-08-11 16:30:03.188692 2014-08-13 18:06:06.453850 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 70db5916/benchmark_data_storage0111-ib.pocstor_35625_object7685/head//22 by client.940330.0:7686 2014-08-11 16:30:09.726737 2014-08-13 18:06:06.453864 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 14e69116/benchmark_data_storage0111-ib.pocstor_35625_object16925/head//22 by client.940330.0:16926 2014-08-11 16:30:44.686558 2014-08-13 18:06:06.453877 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 17fe5916/benchmark_data_storage0111-ib.pocstor_35625_object20454/head//22 by client.940330.0:20455 2014-08-11 16:30:58.526998 2014-08-13 18:06:06.453892 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 40fbb116/benchmark_data_storage0111-ib.pocstor_35625_object23666/head//22 by client.940330.0:23667 2014-08-11 16:31:10.909061 2014-08-13 18:06:06.453906 7f5c973a57a0 20 read_log 192888'447 (0'0) modify a6c2c116/benchmark_data_storage0111-ib.pocstor_35625_object23743/head//22 by client.940330.0:23744 2014-08-11 16:31:11.207063 2014-08-13 18:06:06.453918 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 52f62916/benchmark_data_storage0111-ib.pocstor_35625_object24451/head//22 by client.940330.0:24452 2014-08-11 16:31:13.788043 2014-08-13 18:06:06.453932 7f5c973a57a0 20 read_log 192888'449 (0'0) modify a3cc5916/benchmark_data_storage0111-ib.pocstor_35625_object24649/head//22 by client.940330.0:24650 2014-08-11 16:31:14.524787 2014-08-13 18:06:06.453945 7f5c973a57a0 20 read_log 192888'450 (0'0) modify ad34116/benchmark_data_storage0111-ib.pocstor_35625_object26568/head//22 by client.940330.0:26569 2014-08-11 16:31:22.061749 2014-08-13 18:06:06.453957 7f5c973a57a0 20 read_log 192888'451 (0'0) modify a15ab916/benchmark_data_storage0111-ib.pocstor_35625_object31265/head//22 by client.940330.0:31266 2014-08-11 16:31:40.273133 2014-08-13 18:06:06.453972 7f5c973a57a0 20 read_log 192888'452 (192888'439) delete 624a4116/benchmark_data_storage0111-ib.pocstor_35625_object1207/head//22 by client.940330.0:32515 2014-08-11 16:31:41.798468 2014-08-13 18:06:06.453987 7f5c973a57a0 20 read_log 192888'453 (192888'440) delete 3dc95916/benchmark_data_storage0111-ib.pocstor_35625_object2937/head//22 by client.940330.0:34245 2014-08-11 16:31:42.235722 2014-08-13 18:06:06.454000 7f5c973a57a0 20 read_log 192888'454 (192888'441) delete b75d8916/benchmark_data_storage0111-ib.pocstor_35625_object3841/head//22 by client.940330.0:35149 2014-08-11 16:31:42.486955 2014-08-13 18:06:06.454014 7f5c973a57a0 20 read_log 192888'455 (192888'442) delete 977ff916/benchmark_data_storage0111-ib.pocstor_35625_object5894/head//22 by client.940330.0:37202 2014-08-11 16:31:43.047469 2014-08-13 18:06:06.454027 7f5c973a57a0 20 read_log 192888'456 (192888'443) delete 70db5916/benchmark_data_storage0111-ib.pocstor_35625_object7685/head//22 by client.940330.0:38993 2014-08-11 16:31:43.510337 2014-08-13 18:06:06.454048 7f5c973a57a0 20 read_log 192888'457 (192888'444) delete 14e69116/benchmark_data_storage0111-ib.pocstor_35625_object16925/head//22 by client.940330.0:48233 2014-08-11 16:31:45.969859 2014-08-13 18:06:06.454063 7f5c973a57a0 20 read_log 192888'458 (192888'445) delete 17fe5916/benchmark_data_storage0111-ib.pocstor_35625_object20454/head//22 by client.940330.0:51762 2014-08-11 16:31:47.009868 2014-08-13 18:06:06.454075 7f5c973a57a0 20 read_log 192888'459 (192888'446) delete 40fbb116/benchmark_data_storage0111-ib.pocstor_35625_object23666/head//22 by client.940330.0:54974 2014-08-11 16:31:47.941262 2014-08-13 18:06:06.454089 7f5c973a57a0 20 read_log 192888'460 (192888'447) delete a6c2c116/benchmark_data_storage0111-ib.pocstor_35625_object23743/head//22 by client.940330.0:55051 2014-08-11 16:31:47.962117 2014-08-13 18:06:06.454102 7f5c973a57a0 20 read_log 192888'461 (192888'448) delete 52f62916/benchmark_data_storage0111-ib.pocstor_35625_object24451/head//22 by client.940330.0:55759 2014-08-11 16:31:48.171608 2014-08-13 18:06:06.454116 7f5c973a57a0 20 read_log 192888'462 (192888'449) delete a3cc5916/benchmark_data_storage0111-ib.pocstor_35625_object24649/head//22 by client.940330.0:55957 2014-08-11 16:31:48.230535 2014-08-13 18:06:06.454131 7f5c973a57a0 20 read_log 192888'463 (192888'450) delete ad34116/benchmark_data_storage0111-ib.pocstor_35625_object26568/head//22 by client.940330.0:57876 2014-08-11 16:31:48.810270 2014-08-13 18:06:06.454151 7f5c973a57a0 20 read_log 192888'464 (192888'451) delete a15ab916/benchmark_data_storage0111-ib.pocstor_35625_object31265/head//22 by client.940330.0:62573 2014-08-11 16:31:50.142084 2014-08-13 18:06:06.454166 7f5c973a57a0 20 read_log 194215'465 (0'0) modify a4087116/benchmark_data_storage0111-ib.pocstor_64498_object1090/head//22 by client.1049635.0:1091 2014-08-12 17:17:20.734071 2014-08-13 18:06:06.454179 7f5c973a57a0 20 read_log 194215'466 (0'0) modify fac8a116/benchmark_data_storage0111-ib.pocstor_64498_object5156/head//22 by client.1049635.0:5157 2014-08-12 17:17:33.394579 2014-08-13 18:06:06.454191 7f5c973a57a0 20 read_log 194215'467 (0'0) modify a140b916/benchmark_data_storage0111-ib.pocstor_64498_object5365/head//22 by client.1049635.0:5366 2014-08-12 17:17:34.040797 2014-08-13 18:06:06.454206 7f5c973a57a0 20 read_log 194215'468 (0'0) modify f673e916/benchmark_data_storage0111-ib.pocstor_64498_object8586/head//22 by client.1049635.0:8587 2014-08-12 17:17:43.841742 2014-08-13 18:06:06.454218 7f5c973a57a0 20 read_log 194215'469 (0'0) modify ac992916/benchmark_data_storage0111-ib.pocstor_64498_object8839/head//22 by client.1049635.0:8840 2014-08-12 17:17:44.565023 2014-08-13 18:06:06.454232 7f5c973a57a0 20 read_log 194215'470 (0'0) modify dc13a116/benchmark_data_storage0111-ib.pocstor_64498_object12504/head//22 by client.1049635.0:12505 2014-08-12 17:17:55.906356 2014-08-13 18:06:06.454245 7f5c973a57a0 20 read_log 194215'471 (0'0) modify e3fe2916/benchmark_data_storage0111-ib.pocstor_64498_object17820/head//22 by client.1049635.0:17821 2014-08-12 17:18:14.480255 2014-08-13 18:06:06.454258 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 4e9f4916/benchmark_data_storage0111-ib.pocstor_64498_object18225/head//22 by client.1049635.0:18226 2014-08-12 17:18:16.011056 2014-08-13 18:06:06.454270 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 71470116/benchmark_data_storage0111-ib.pocstor_64498_object24781/head//22 by client.1049635.0:24782 2014-08-12 17:18:40.761724 2014-08-13 18:06:06.454285 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 19214916/benchmark_data_storage0111-ib.pocstor_64498_object26570/head//22 by client.1049635.0:26571 2014-08-12 17:18:47.545961 2014-08-13 18:06:06.454303 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 36749916/benchmark_data_storage0111-ib.pocstor_64498_object26633/head//22 by client.1049635.0:26634 2014-08-12 17:18:47.784194 2014-08-13 18:06:06.454325 7f5c973a57a0 20 read_log 194215'476 (0'0) modify b477a116/benchmark_data_storage0111-ib.pocstor_64498_object28846/head//22 by client.1049635.0:28847 2014-08-12 17:18:56.030367 2014-08-13 18:06:06.454344 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 9fb36916/benchmark_data_storage0111-ib.pocstor_64498_object29857/head//22 by client.1049635.0:29858 2014-08-12 17:18:59.912795 2014-08-13 18:06:06.454357 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 6c957116/benchmark_data_storage0111-ib.pocstor_64498_object30231/head//22 by client.1049635.0:30232 2014-08-12 17:19:01.472489 2014-08-13 18:06:06.454370 7f5c973a57a0 20 read_log 194215'479 (0'0) modify f48be116/benchmark_data_storage0111-ib.pocstor_64498_object32849/head//22 by client.1049635.0:32850 2014-08-12 17:19:11.819207 2014-08-13 18:06:06.454382 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 9153f116/benchmark_data_storage0111-ib.pocstor_64498_object34000/head//22 by client.1049635.0:34001 2014-08-12 17:19:16.183194 2014-08-13 18:06:06.454394 7f5c973a57a0 20 read_log 194215'481 (194215'465) delete a4087116/benchmark_data_storage0111-ib.pocstor_64498_object1090/head//22 by client.1049635.0:35293 2014-08-12 17:19:18.348349 2014-08-13 18:06:06.454407 7f5c973a57a0 20 read_log 194215'482 (194215'466) delete fac8a116/benchmark_data_storage0111-ib.pocstor_64498_object5156/head//22 by client.1049635.0:39359 2014-08-12 17:19:19.658271 2014-08-13 18:06:06.454437 7f5c973a57a0 20 read_log 194215'483 (194215'467) delete a140b916/benchmark_data_storage0111-ib.pocstor_64498_object5365/head//22 by client.1049635.0:39568 2014-08-12 17:19:19.727550 2014-08-13 18:06:06.454453 7f5c973a57a0 20 read_log 194215'484 (194215'468) delete f673e916/benchmark_data_storage0111-ib.pocstor_64498_object8586/head//22 by client.1049635.0:42789 2014-08-12 17:19:20.791626 2014-08-13 18:06:06.454466 7f5c973a57a0 20 read_log 194215'485 (194215'469) delete ac992916/benchmark_data_storage0111-ib.pocstor_64498_object8839/head//22 by client.1049635.0:43042 2014-08-12 17:19:20.873686 2014-08-13 18:06:06.454489 7f5c973a57a0 20 read_log 194215'486 (194215'470) delete dc13a116/benchmark_data_storage0111-ib.pocstor_64498_object12504/head//22 by client.1049635.0:46707 2014-08-12 17:19:22.054246 2014-08-13 18:06:06.454502 7f5c973a57a0 20 read_log 194215'487 (194215'471) delete e3fe2916/benchmark_data_storage0111-ib.pocstor_64498_object17820/head//22 by client.1049635.0:52023 2014-08-12 17:19:23.766792 2014-08-13 18:06:06.454527 7f5c973a57a0 20 read_log 194215'488 (194215'472) delete 4e9f4916/benchmark_data_storage0111-ib.pocstor_64498_object18225/head//22 by client.1049635.0:52428 2014-08-12 17:19:23.904939 2014-08-13 18:06:06.454543 7f5c973a57a0 20 read_log 194215'489 (194215'473) delete 71470116/benchmark_data_storage0111-ib.pocstor_64498_object24781/head//22 by client.1049635.0:58984 2014-08-12 17:19:26.246505 2014-08-13 18:06:06.454556 7f5c973a57a0 20 read_log 194215'490 (194215'474) delete 19214916/benchmark_data_storage0111-ib.pocstor_64498_object26570/head//22 by client.1049635.0:60773 2014-08-12 17:19:26.846081 2014-08-13 18:06:06.454571 7f5c973a57a0 20 read_log 194215'491 (194215'475) delete 36749916/benchmark_data_storage0111-ib.pocstor_64498_object26633/head//22 by client.1049635.0:60836 2014-08-12 17:19:26.868477 2014-08-13 18:06:06.454584 7f5c973a57a0 20 read_log 194215'492 (194215'476) delete b477a116/benchmark_data_storage0111-ib.pocstor_64498_object28846/head//22 by client.1049635.0:63049 2014-08-12 17:19:27.643104 2014-08-13 18:06:06.454596 7f5c973a57a0 20 read_log 194215'493 (194215'477) delete 9fb36916/benchmark_data_storage0111-ib.pocstor_64498_object29857/head//22 by client.1049635.0:64060 2014-08-12 17:19:28.002724 2014-08-13 18:06:06.454608 7f5c973a57a0 20 read_log 194215'494 (194215'478) delete 6c957116/benchmark_data_storage0111-ib.pocstor_64498_object30231/head//22 by client.1049635.0:64434 2014-08-12 17:19:28.118949 2014-08-13 18:06:06.454628 7f5c973a57a0 20 read_log 194215'495 (194215'479) delete f48be116/benchmark_data_storage0111-ib.pocstor_64498_object32849/head//22 by client.1049635.0:67052 2014-08-12 17:19:29.058729 2014-08-13 18:06:06.454652 7f5c973a57a0 20 read_log 194215'496 (194215'480) delete 9153f116/benchmark_data_storage0111-ib.pocstor_64498_object34000/head//22 by client.1049635.0:68203 2014-08-12 17:19:29.482676 2014-08-13 18:06:06.454741 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.454758 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=0 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.454782 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=0 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Initial 0.001721 0 0.000000 2014-08-13 18:06:06.454794 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=0 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.454803 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=0 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.454812 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] log((0'0,194215'496], crt=192888'464) 2014-08-13 18:06:06.454822 7f5c973a57a0 10 osd.4 194487 pgid 22.12a coll 22.12a_head 2014-08-13 18:06:06.454994 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.12a 2014-08-13 18:06:06.455003 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.455029 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.12a(unlocked)] enter Initial 2014-08-13 18:06:06.455044 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.12a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.455199 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.455319 7f5c973a57a0 20 read_log 191648'70 (0'0) modify 87c1e92a/benchmark_data_storage0111-ib.pocstor_40745_object4743/head//22 by client.273158.0:4744 2014-07-30 14:09:09.824350 2014-08-13 18:06:06.455340 7f5c973a57a0 20 read_log 191648'106 (0'0) modify 647592a/benchmark_data_storage0111-ib.pocstor_40745_object7036/head//22 by client.273158.0:7037 2014-07-30 14:09:21.235503 2014-08-13 18:06:06.455353 7f5c973a57a0 20 read_log 191648'142 (0'0) modify 2a19992a/benchmark_data_storage0111-ib.pocstor_40745_object9701/head//22 by client.273158.0:9702 2014-07-30 14:09:34.401634 2014-08-13 18:06:06.455365 7f5c973a57a0 20 read_log 191648'148 (0'0) modify 6969c92a/benchmark_data_storage0111-ib.pocstor_40745_object9826/head//22 by client.273158.0:9827 2014-07-30 14:09:34.901010 2014-08-13 18:06:06.455377 7f5c973a57a0 20 read_log 191648'220 (0'0) modify 581c92a/benchmark_data_storage0111-ib.pocstor_40745_object15560/head//22 by client.273158.0:15561 2014-07-30 14:10:05.073565 2014-08-13 18:06:06.455390 7f5c973a57a0 20 read_log 191648'251 (0'0) modify af6ef92a/benchmark_data_storage0111-ib.pocstor_40745_object17068/head//22 by client.273158.0:17069 2014-07-30 14:10:13.401297 2014-08-13 18:06:06.455402 7f5c973a57a0 20 read_log 191648'256 (0'0) modify b6e8092a/benchmark_data_storage0111-ib.pocstor_40745_object17182/head//22 by client.273158.0:17183 2014-07-30 14:10:13.997832 2014-08-13 18:06:06.455414 7f5c973a57a0 20 read_log 191648'266 (0'0) modify 8a7d912a/benchmark_data_storage0111-ib.pocstor_40745_object17584/head//22 by client.273158.0:17585 2014-07-30 14:10:16.023032 2014-08-13 18:06:06.455426 7f5c973a57a0 20 read_log 191648'324 (0'0) modify 7be2592a/benchmark_data_storage0111-ib.pocstor_40745_object22107/head//22 by client.273158.0:22108 2014-07-30 14:10:44.617145 2014-08-13 18:06:06.455439 7f5c973a57a0 20 read_log 191727'336 (0'0) modify d77d992a/benchmark_data_storage0111-ib.pocstor_51285_object251/head//22 by client.275175.0:252 2014-07-30 14:18:34.407369 2014-08-13 18:06:06.455451 7f5c973a57a0 20 read_log 191727'337 (0'0) modify f612012a/benchmark_data_storage0111-ib.pocstor_51285_object374/head//22 by client.275175.0:375 2014-07-30 14:18:34.938935 2014-08-13 18:06:06.455463 7f5c973a57a0 20 read_log 191727'339 (0'0) modify 7257392a/benchmark_data_storage0111-ib.pocstor_51285_object1244/head//22 by client.275175.0:1245 2014-07-30 14:18:38.332118 2014-08-13 18:06:06.455475 7f5c973a57a0 20 read_log 191727'340 (0'0) modify 2cd3812a/benchmark_data_storage0111-ib.pocstor_51285_object1631/head//22 by client.275175.0:1632 2014-07-30 14:18:39.666150 2014-08-13 18:06:06.455489 7f5c973a57a0 20 read_log 191727'341 (0'0) modify fc9ba92a/benchmark_data_storage0111-ib.pocstor_51285_object1852/head//22 by client.275175.0:1853 2014-07-30 14:18:40.417035 2014-08-13 18:06:06.455501 7f5c973a57a0 20 read_log 191727'344 (0'0) modify b088992a/benchmark_data_storage0111-ib.pocstor_51285_object3975/head//22 by client.275175.0:3976 2014-07-30 14:18:47.280165 2014-08-13 18:06:06.455534 7f5c973a57a0 20 read_log 191727'349 (0'0) modify 8733812a/benchmark_data_storage0111-ib.pocstor_51285_object7812/head//22 by client.275175.0:7813 2014-07-30 14:18:59.001810 2014-08-13 18:06:06.455549 7f5c973a57a0 20 read_log 191727'350 (0'0) modify aaf8612a/benchmark_data_storage0111-ib.pocstor_51285_object8521/head//22 by client.275175.0:8522 2014-07-30 14:19:00.987795 2014-08-13 18:06:06.455562 7f5c973a57a0 20 read_log 191727'357 (0'0) modify 7568a92a/benchmark_data_storage0111-ib.pocstor_51285_object10652/head//22 by client.275175.0:10653 2014-07-30 14:19:07.072632 2014-08-13 18:06:06.455575 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 825e692a/benchmark_data_storage0111-ib.pocstor_51285_object12969/head//22 by client.275175.0:12970 2014-07-30 14:19:14.531347 2014-08-13 18:06:06.455595 7f5c973a57a0 20 read_log 191727'362 (0'0) modify 297912a/benchmark_data_storage0111-ib.pocstor_51285_object15500/head//22 by client.275175.0:15501 2014-07-30 14:19:22.416357 2014-08-13 18:06:06.455610 7f5c973a57a0 20 read_log 191727'363 (0'0) modify b98d192a/benchmark_data_storage0111-ib.pocstor_51285_object16368/head//22 by client.275175.0:16369 2014-07-30 14:19:25.261820 2014-08-13 18:06:06.455622 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 4eaad12a/benchmark_data_storage0111-ib.pocstor_51285_object18134/head//22 by client.275175.0:18135 2014-07-30 14:19:30.333846 2014-08-13 18:06:06.455634 7f5c973a57a0 20 read_log 191727'369 (0'0) modify 885ff12a/benchmark_data_storage0111-ib.pocstor_51285_object18417/head//22 by client.275175.0:18418 2014-07-30 14:19:31.248060 2014-08-13 18:06:06.455646 7f5c973a57a0 20 read_log 191727'370 (0'0) modify bbe0292a/benchmark_data_storage0111-ib.pocstor_51285_object18753/head//22 by client.275175.0:18754 2014-07-30 14:19:32.290424 2014-08-13 18:06:06.455658 7f5c973a57a0 20 read_log 191727'374 (0'0) modify e3e3792a/benchmark_data_storage0111-ib.pocstor_51285_object19780/head//22 by client.275175.0:19781 2014-07-30 14:19:35.553511 2014-08-13 18:06:06.455670 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 1f27792a/benchmark_data_storage0111-ib.pocstor_51285_object19874/head//22 by client.275175.0:19875 2014-07-30 14:19:35.817071 2014-08-13 18:06:06.455682 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 3d0f312a/benchmark_data_storage0111-ib.pocstor_51285_object21500/head//22 by client.275175.0:21501 2014-07-30 14:19:40.666866 2014-08-13 18:06:06.455695 7f5c973a57a0 20 read_log 191727'385 (0'0) modify fe3c392a/benchmark_data_storage0111-ib.pocstor_51285_object25479/head//22 by client.275175.0:25480 2014-07-30 14:19:53.203645 2014-08-13 18:06:06.455707 7f5c973a57a0 20 read_log 191727'387 (0'0) modify c78b192a/benchmark_data_storage0111-ib.pocstor_51285_object25908/head//22 by client.275175.0:25909 2014-07-30 14:19:54.468897 2014-08-13 18:06:06.455718 7f5c973a57a0 20 read_log 191727'388 (0'0) modify d6fff92a/benchmark_data_storage0111-ib.pocstor_51285_object26164/head//22 by client.275175.0:26165 2014-07-30 14:19:55.335846 2014-08-13 18:06:06.455731 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 4e28012a/benchmark_data_storage0111-ib.pocstor_51285_object27916/head//22 by client.275175.0:27917 2014-07-30 14:20:00.768446 2014-08-13 18:06:06.455742 7f5c973a57a0 20 read_log 191727'393 (0'0) modify 881da92a/benchmark_data_storage0111-ib.pocstor_51285_object28150/head//22 by client.275175.0:28151 2014-07-30 14:20:01.450873 2014-08-13 18:06:06.455754 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 3899192a/benchmark_data_storage0111-ib.pocstor_51285_object30322/head//22 by client.275175.0:30323 2014-07-30 14:20:08.078527 2014-08-13 18:06:06.455766 7f5c973a57a0 20 read_log 191727'398 (0'0) modify c732312a/benchmark_data_storage0111-ib.pocstor_51285_object30774/head//22 by client.275175.0:30775 2014-07-30 14:20:09.341375 2014-08-13 18:06:06.455778 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 996a012a/benchmark_data_storage0111-ib.pocstor_51285_object31879/head//22 by client.275175.0:31880 2014-07-30 14:20:12.510839 2014-08-13 18:06:06.455790 7f5c973a57a0 20 read_log 191727'404 (0'0) modify 431b992a/benchmark_data_storage0111-ib.pocstor_51285_object32036/head//22 by client.275175.0:32037 2014-07-30 14:20:12.953760 2014-08-13 18:06:06.455802 7f5c973a57a0 20 read_log 191727'407 (0'0) modify f51f812a/benchmark_data_storage0111-ib.pocstor_51285_object32537/head//22 by client.275175.0:32538 2014-07-30 14:20:14.406656 2014-08-13 18:06:06.455817 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 898bc12a/benchmark_data_storage0111-ib.pocstor_51285_object35016/head//22 by client.275175.0:35017 2014-07-30 14:20:22.031759 2014-08-13 18:06:06.455832 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 3952d92a/benchmark_data_storage0111-ib.pocstor_51285_object37282/head//22 by client.275175.0:37283 2014-07-30 14:20:29.156295 2014-08-13 18:06:06.455854 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 4949f12a/benchmark_data_storage0111-ib.pocstor_51285_object38749/head//22 by client.275175.0:38750 2014-07-30 14:20:33.377345 2014-08-13 18:06:06.455868 7f5c973a57a0 20 read_log 192674'422 (0'0) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2463 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455881 7f5c973a57a0 20 read_log 192674'423 (192674'422) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2464 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455895 7f5c973a57a0 20 read_log 192674'424 (192674'423) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2465 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455907 7f5c973a57a0 20 read_log 192674'425 (192674'424) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2466 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455941 7f5c973a57a0 20 read_log 192674'426 (192674'425) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2467 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455972 7f5c973a57a0 20 read_log 192674'427 (192674'426) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2468 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.455989 7f5c973a57a0 20 read_log 192674'428 (192674'427) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2469 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.456001 7f5c973a57a0 20 read_log 192674'429 (192674'428) modify 1489612a/rb.0.43b76.238e1f29.0000000000d6/head//22 by client.279772.1:2470 2014-07-31 18:03:44.302000 2014-08-13 18:06:06.456014 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 2ae2f12a/benchmark_data_storage0111-ib.pocstor_35625_object255/head//22 by client.940330.0:256 2014-08-11 16:29:41.740666 2014-08-13 18:06:06.456026 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 680d612a/benchmark_data_storage0111-ib.pocstor_35625_object2015/head//22 by client.940330.0:2016 2014-08-11 16:29:48.613779 2014-08-13 18:06:06.456045 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 82b8892a/benchmark_data_storage0111-ib.pocstor_35625_object7489/head//22 by client.940330.0:7490 2014-08-11 16:30:09.039951 2014-08-13 18:06:06.456058 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 9ff3d92a/benchmark_data_storage0111-ib.pocstor_35625_object7813/head//22 by client.940330.0:7814 2014-08-11 16:30:10.179044 2014-08-13 18:06:06.456072 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 13fa912a/benchmark_data_storage0111-ib.pocstor_35625_object8599/head//22 by client.940330.0:8600 2014-08-11 16:30:12.931333 2014-08-13 18:06:06.456085 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 4120392a/benchmark_data_storage0111-ib.pocstor_35625_object13555/head//22 by client.940330.0:13556 2014-08-11 16:30:31.988268 2014-08-13 18:06:06.456099 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 3a90012a/benchmark_data_storage0111-ib.pocstor_35625_object15524/head//22 by client.940330.0:15525 2014-08-11 16:30:39.449368 2014-08-13 18:06:06.456112 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 575c912a/benchmark_data_storage0111-ib.pocstor_35625_object21817/head//22 by client.940330.0:21818 2014-08-11 16:31:03.889463 2014-08-13 18:06:06.456127 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 3bd3a92a/benchmark_data_storage0111-ib.pocstor_35625_object23944/head//22 by client.940330.0:23945 2014-08-11 16:31:11.947531 2014-08-13 18:06:06.456142 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 5b4a092a/benchmark_data_storage0111-ib.pocstor_35625_object24360/head//22 by client.940330.0:24361 2014-08-11 16:31:13.460155 2014-08-13 18:06:06.456177 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 8aa3912a/benchmark_data_storage0111-ib.pocstor_35625_object24503/head//22 by client.940330.0:24504 2014-08-11 16:31:13.985335 2014-08-13 18:06:06.456201 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 7044892a/benchmark_data_storage0111-ib.pocstor_35625_object24607/head//22 by client.940330.0:24608 2014-08-11 16:31:14.372515 2014-08-13 18:06:06.456221 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 11d0392a/benchmark_data_storage0111-ib.pocstor_35625_object25772/head//22 by client.940330.0:25773 2014-08-11 16:31:18.895464 2014-08-13 18:06:06.456236 7f5c973a57a0 20 read_log 192888'443 (0'0) modify b513c92a/benchmark_data_storage0111-ib.pocstor_35625_object25997/head//22 by client.940330.0:25998 2014-08-11 16:31:19.781326 2014-08-13 18:06:06.456248 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 1fd5612a/benchmark_data_storage0111-ib.pocstor_35625_object26880/head//22 by client.940330.0:26881 2014-08-11 16:31:23.340354 2014-08-13 18:06:06.456262 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 6407f92a/benchmark_data_storage0111-ib.pocstor_35625_object28894/head//22 by client.940330.0:28895 2014-08-11 16:31:31.184224 2014-08-13 18:06:06.456288 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 29b9392a/benchmark_data_storage0111-ib.pocstor_35625_object31078/head//22 by client.940330.0:31079 2014-08-11 16:31:39.518108 2014-08-13 18:06:06.456307 7f5c973a57a0 20 read_log 192888'447 (192888'430) delete 2ae2f12a/benchmark_data_storage0111-ib.pocstor_35625_object255/head//22 by client.940330.0:31563 2014-08-11 16:31:41.546524 2014-08-13 18:06:06.456320 7f5c973a57a0 20 read_log 192888'448 (192888'431) delete 680d612a/benchmark_data_storage0111-ib.pocstor_35625_object2015/head//22 by client.940330.0:33323 2014-08-11 16:31:41.995430 2014-08-13 18:06:06.456335 7f5c973a57a0 20 read_log 192888'449 (192888'432) delete 82b8892a/benchmark_data_storage0111-ib.pocstor_35625_object7489/head//22 by client.940330.0:38797 2014-08-11 16:31:43.461043 2014-08-13 18:06:06.456358 7f5c973a57a0 20 read_log 192888'450 (192888'433) delete 9ff3d92a/benchmark_data_storage0111-ib.pocstor_35625_object7813/head//22 by client.940330.0:39121 2014-08-11 16:31:43.543246 2014-08-13 18:06:06.456371 7f5c973a57a0 20 read_log 192888'451 (192888'434) delete 13fa912a/benchmark_data_storage0111-ib.pocstor_35625_object8599/head//22 by client.940330.0:39907 2014-08-11 16:31:43.739005 2014-08-13 18:06:06.456384 7f5c973a57a0 20 read_log 192888'452 (192888'435) delete 4120392a/benchmark_data_storage0111-ib.pocstor_35625_object13555/head//22 by client.940330.0:44863 2014-08-11 16:31:45.068963 2014-08-13 18:06:06.456412 7f5c973a57a0 20 read_log 192888'453 (192888'436) delete 3a90012a/benchmark_data_storage0111-ib.pocstor_35625_object15524/head//22 by client.940330.0:46832 2014-08-11 16:31:45.596697 2014-08-13 18:06:06.456458 7f5c973a57a0 20 read_log 192888'454 (192888'437) delete 575c912a/benchmark_data_storage0111-ib.pocstor_35625_object21817/head//22 by client.940330.0:53125 2014-08-11 16:31:47.389161 2014-08-13 18:06:06.456484 7f5c973a57a0 20 read_log 192888'455 (192888'438) delete 3bd3a92a/benchmark_data_storage0111-ib.pocstor_35625_object23944/head//22 by client.940330.0:55252 2014-08-11 16:31:48.021494 2014-08-13 18:06:06.456499 7f5c973a57a0 20 read_log 192888'456 (192888'439) delete 5b4a092a/benchmark_data_storage0111-ib.pocstor_35625_object24360/head//22 by client.940330.0:55668 2014-08-11 16:31:48.145378 2014-08-13 18:06:06.456536 7f5c973a57a0 20 read_log 192888'457 (192888'440) delete 8aa3912a/benchmark_data_storage0111-ib.pocstor_35625_object24503/head//22 by client.940330.0:55811 2014-08-11 16:31:48.190113 2014-08-13 18:06:06.456550 7f5c973a57a0 20 read_log 192888'458 (192888'441) delete 7044892a/benchmark_data_storage0111-ib.pocstor_35625_object24607/head//22 by client.940330.0:55915 2014-08-11 16:31:48.218663 2014-08-13 18:06:06.456566 7f5c973a57a0 20 read_log 192888'459 (192888'442) delete 11d0392a/benchmark_data_storage0111-ib.pocstor_35625_object25772/head//22 by client.940330.0:57080 2014-08-11 16:31:48.579845 2014-08-13 18:06:06.456579 7f5c973a57a0 20 read_log 192888'460 (192888'443) delete b513c92a/benchmark_data_storage0111-ib.pocstor_35625_object25997/head//22 by client.940330.0:57305 2014-08-11 16:31:48.647121 2014-08-13 18:06:06.456619 7f5c973a57a0 20 read_log 192888'461 (192888'444) delete 1fd5612a/benchmark_data_storage0111-ib.pocstor_35625_object26880/head//22 by client.940330.0:58188 2014-08-11 16:31:48.895189 2014-08-13 18:06:06.456646 7f5c973a57a0 20 read_log 192888'462 (192888'445) delete 6407f92a/benchmark_data_storage0111-ib.pocstor_35625_object28894/head//22 by client.940330.0:60202 2014-08-11 16:31:49.444197 2014-08-13 18:06:06.456659 7f5c973a57a0 20 read_log 192888'463 (192888'446) delete 29b9392a/benchmark_data_storage0111-ib.pocstor_35625_object31078/head//22 by client.940330.0:62386 2014-08-11 16:31:50.080511 2014-08-13 18:06:06.456671 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 2ee612a/benchmark_data_storage0111-ib.pocstor_64498_object379/head//22 by client.1049635.0:380 2014-08-12 17:17:18.353063 2014-08-13 18:06:06.456688 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 2c48c12a/benchmark_data_storage0111-ib.pocstor_64498_object1059/head//22 by client.1049635.0:1060 2014-08-12 17:17:20.610912 2014-08-13 18:06:06.456700 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 8af2112a/benchmark_data_storage0111-ib.pocstor_64498_object4083/head//22 by client.1049635.0:4084 2014-08-12 17:17:30.333738 2014-08-13 18:06:06.456712 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 147dc92a/benchmark_data_storage0111-ib.pocstor_64498_object10160/head//22 by client.1049635.0:10161 2014-08-12 17:17:48.385336 2014-08-13 18:06:06.456725 7f5c973a57a0 20 read_log 194215'468 (0'0) modify d568392a/benchmark_data_storage0111-ib.pocstor_64498_object10885/head//22 by client.1049635.0:10886 2014-08-12 17:17:50.767452 2014-08-13 18:06:06.456737 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 9e11f12a/benchmark_data_storage0111-ib.pocstor_64498_object14759/head//22 by client.1049635.0:14760 2014-08-12 17:18:03.193799 2014-08-13 18:06:06.456750 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 9e17192a/benchmark_data_storage0111-ib.pocstor_64498_object16689/head//22 by client.1049635.0:16690 2014-08-12 17:18:10.190169 2014-08-13 18:06:06.456762 7f5c973a57a0 20 read_log 194215'471 (0'0) modify ffa1c12a/benchmark_data_storage0111-ib.pocstor_64498_object22262/head//22 by client.1049635.0:22263 2014-08-12 17:18:31.317648 2014-08-13 18:06:06.456774 7f5c973a57a0 20 read_log 194215'472 (0'0) modify f6b0e92a/benchmark_data_storage0111-ib.pocstor_64498_object23093/head//22 by client.1049635.0:23094 2014-08-12 17:18:34.475902 2014-08-13 18:06:06.456786 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 7f64192a/benchmark_data_storage0111-ib.pocstor_64498_object24449/head//22 by client.1049635.0:24450 2014-08-12 17:18:39.524038 2014-08-13 18:06:06.456798 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 70cf912a/benchmark_data_storage0111-ib.pocstor_64498_object24585/head//22 by client.1049635.0:24586 2014-08-12 17:18:40.036578 2014-08-13 18:06:06.456810 7f5c973a57a0 20 read_log 194215'475 (0'0) modify e836812a/benchmark_data_storage0111-ib.pocstor_64498_object24977/head//22 by client.1049635.0:24978 2014-08-12 17:18:41.631543 2014-08-13 18:06:06.456822 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 172e592a/benchmark_data_storage0111-ib.pocstor_64498_object29066/head//22 by client.1049635.0:29067 2014-08-12 17:18:56.843588 2014-08-13 18:06:06.456835 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 6ec1012a/benchmark_data_storage0111-ib.pocstor_64498_object29425/head//22 by client.1049635.0:29426 2014-08-12 17:18:58.191693 2014-08-13 18:06:06.456847 7f5c973a57a0 20 read_log 194215'478 (194215'464) delete 2ee612a/benchmark_data_storage0111-ib.pocstor_64498_object379/head//22 by client.1049635.0:34582 2014-08-12 17:19:18.121056 2014-08-13 18:06:06.456859 7f5c973a57a0 20 read_log 194215'479 (194215'465) delete 2c48c12a/benchmark_data_storage0111-ib.pocstor_64498_object1059/head//22 by client.1049635.0:35262 2014-08-12 17:19:18.339151 2014-08-13 18:06:06.456879 7f5c973a57a0 20 read_log 194215'480 (194215'466) delete 8af2112a/benchmark_data_storage0111-ib.pocstor_64498_object4083/head//22 by client.1049635.0:38286 2014-08-12 17:19:19.301383 2014-08-13 18:06:06.456892 7f5c973a57a0 20 read_log 194215'481 (194215'467) delete 147dc92a/benchmark_data_storage0111-ib.pocstor_64498_object10160/head//22 by client.1049635.0:44363 2014-08-12 17:19:21.314842 2014-08-13 18:06:06.456905 7f5c973a57a0 20 read_log 194215'482 (194215'468) delete d568392a/benchmark_data_storage0111-ib.pocstor_64498_object10885/head//22 by client.1049635.0:45088 2014-08-12 17:19:21.543309 2014-08-13 18:06:06.456918 7f5c973a57a0 20 read_log 194215'483 (194215'469) delete 9e11f12a/benchmark_data_storage0111-ib.pocstor_64498_object14759/head//22 by client.1049635.0:48962 2014-08-12 17:19:22.802175 2014-08-13 18:06:06.456931 7f5c973a57a0 20 read_log 194215'484 (194215'470) delete 9e17192a/benchmark_data_storage0111-ib.pocstor_64498_object16689/head//22 by client.1049635.0:50892 2014-08-12 17:19:23.414098 2014-08-13 18:06:06.456943 7f5c973a57a0 20 read_log 194215'485 (194215'471) delete ffa1c12a/benchmark_data_storage0111-ib.pocstor_64498_object22262/head//22 by client.1049635.0:56465 2014-08-12 17:19:25.328865 2014-08-13 18:06:06.456955 7f5c973a57a0 20 read_log 194215'486 (194215'472) delete f6b0e92a/benchmark_data_storage0111-ib.pocstor_64498_object23093/head//22 by client.1049635.0:57296 2014-08-12 17:19:25.654589 2014-08-13 18:06:06.456967 7f5c973a57a0 20 read_log 194215'487 (194215'473) delete 7f64192a/benchmark_data_storage0111-ib.pocstor_64498_object24449/head//22 by client.1049635.0:58652 2014-08-12 17:19:26.115871 2014-08-13 18:06:06.456979 7f5c973a57a0 20 read_log 194215'488 (194215'474) delete 70cf912a/benchmark_data_storage0111-ib.pocstor_64498_object24585/head//22 by client.1049635.0:58788 2014-08-12 17:19:26.168053 2014-08-13 18:06:06.456991 7f5c973a57a0 20 read_log 194215'489 (194215'475) delete e836812a/benchmark_data_storage0111-ib.pocstor_64498_object24977/head//22 by client.1049635.0:59180 2014-08-12 17:19:26.314211 2014-08-13 18:06:06.457022 7f5c973a57a0 20 read_log 194215'490 (194215'476) delete 172e592a/benchmark_data_storage0111-ib.pocstor_64498_object29066/head//22 by client.1049635.0:63269 2014-08-12 17:19:27.727725 2014-08-13 18:06:06.457035 7f5c973a57a0 20 read_log 194215'491 (194215'477) delete 6ec1012a/benchmark_data_storage0111-ib.pocstor_64498_object29425/head//22 by client.1049635.0:63628 2014-08-12 17:19:27.859889 2014-08-13 18:06:06.457142 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.457166 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.457180 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.002151 0 0.000000 2014-08-13 18:06:06.457193 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.457201 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.457210 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] log((0'0,194215'491], crt=192888'463) 2014-08-13 18:06:06.457220 7f5c973a57a0 10 osd.4 194487 pgid 22.14a coll 22.14a_head 2014-08-13 18:06:06.457361 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.14a 2014-08-13 18:06:06.457370 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.457417 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.14a(unlocked)] enter Initial 2014-08-13 18:06:06.457426 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.14a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.457594 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.457709 7f5c973a57a0 20 read_log 191648'11 (0'0) modify 58d3594a/benchmark_data_storage0111-ib.pocstor_40745_object984/head//22 by client.273158.0:985 2014-07-30 14:08:52.424666 2014-08-13 18:06:06.457730 7f5c973a57a0 20 read_log 191648'36 (0'0) modify 441e514a/benchmark_data_storage0111-ib.pocstor_40745_object2331/head//22 by client.273158.0:2332 2014-07-30 14:08:57.801631 2014-08-13 18:06:06.457743 7f5c973a57a0 20 read_log 191648'52 (0'0) modify e483894a/benchmark_data_storage0111-ib.pocstor_40745_object3123/head//22 by client.273158.0:3124 2014-07-30 14:09:01.863211 2014-08-13 18:06:06.457755 7f5c973a57a0 20 read_log 191648'58 (0'0) modify 5752f94a/benchmark_data_storage0111-ib.pocstor_40745_object3516/head//22 by client.273158.0:3517 2014-07-30 14:09:03.797950 2014-08-13 18:06:06.457767 7f5c973a57a0 20 read_log 191648'126 (0'0) modify 8bfb394a/benchmark_data_storage0111-ib.pocstor_40745_object7135/head//22 by client.273158.0:7136 2014-07-30 14:09:21.622499 2014-08-13 18:06:06.457779 7f5c973a57a0 20 read_log 191648'127 (0'0) modify dba1094a/benchmark_data_storage0111-ib.pocstor_40745_object7202/head//22 by client.273158.0:7203 2014-07-30 14:09:21.886306 2014-08-13 18:06:06.457801 7f5c973a57a0 20 read_log 191648'142 (0'0) modify 4475314a/benchmark_data_storage0111-ib.pocstor_40745_object8182/head//22 by client.273158.0:8183 2014-07-30 14:09:27.071935 2014-08-13 18:06:06.457814 7f5c973a57a0 20 read_log 191648'227 (0'0) modify 1ed6894a/benchmark_data_storage0111-ib.pocstor_40745_object14094/head//22 by client.273158.0:14095 2014-07-30 14:09:57.444232 2014-08-13 18:06:06.457826 7f5c973a57a0 20 read_log 191648'230 (0'0) modify 1b50c14a/benchmark_data_storage0111-ib.pocstor_40745_object14494/head//22 by client.273158.0:14495 2014-07-30 14:10:00.064442 2014-08-13 18:06:06.457839 7f5c973a57a0 20 read_log 191648'232 (0'0) modify cfc9e14a/benchmark_data_storage0111-ib.pocstor_40745_object14681/head//22 by client.273158.0:14682 2014-07-30 14:10:00.915715 2014-08-13 18:06:06.457851 7f5c973a57a0 20 read_log 191648'268 (0'0) modify 7221e14a/benchmark_data_storage0111-ib.pocstor_40745_object16901/head//22 by client.273158.0:16902 2014-07-30 14:10:12.219844 2014-08-13 18:06:06.457863 7f5c973a57a0 20 read_log 191648'281 (0'0) modify 5ef0814a/benchmark_data_storage0111-ib.pocstor_40745_object17758/head//22 by client.273158.0:17759 2014-07-30 14:10:16.898139 2014-08-13 18:06:06.457876 7f5c973a57a0 20 read_log 191648'292 (0'0) modify 64c5414a/benchmark_data_storage0111-ib.pocstor_40745_object18496/head//22 by client.273158.0:18497 2014-07-30 14:10:20.854550 2014-08-13 18:06:06.457888 7f5c973a57a0 20 read_log 191648'301 (0'0) modify d5b3a14a/benchmark_data_storage0111-ib.pocstor_40745_object19469/head//22 by client.273158.0:19470 2014-07-30 14:10:26.248542 2014-08-13 18:06:06.457901 7f5c973a57a0 20 read_log 191727'361 (0'0) modify ed57514a/benchmark_data_storage0111-ib.pocstor_51285_object993/head//22 by client.275175.0:994 2014-07-30 14:18:37.425929 2014-08-13 18:06:06.457914 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 4488d94a/benchmark_data_storage0111-ib.pocstor_51285_object4714/head//22 by client.275175.0:4715 2014-07-30 14:18:49.499713 2014-08-13 18:06:06.457926 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 6011494a/benchmark_data_storage0111-ib.pocstor_51285_object5151/head//22 by client.275175.0:5152 2014-07-30 14:18:50.796850 2014-08-13 18:06:06.457938 7f5c973a57a0 20 read_log 191727'377 (0'0) modify 7a4fa94a/benchmark_data_storage0111-ib.pocstor_51285_object8499/head//22 by client.275175.0:8500 2014-07-30 14:19:00.924572 2014-08-13 18:06:06.457950 7f5c973a57a0 20 read_log 191727'380 (0'0) modify 4ed2b94a/benchmark_data_storage0111-ib.pocstor_51285_object9611/head//22 by client.275175.0:9612 2014-07-30 14:19:04.101174 2014-08-13 18:06:06.457962 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 9b4c114a/benchmark_data_storage0111-ib.pocstor_51285_object10789/head//22 by client.275175.0:10790 2014-07-30 14:19:07.453352 2014-08-13 18:06:06.457974 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 96ffb14a/benchmark_data_storage0111-ib.pocstor_51285_object13539/head//22 by client.275175.0:13540 2014-07-30 14:19:16.354459 2014-08-13 18:06:06.457986 7f5c973a57a0 20 read_log 191727'387 (0'0) modify a5c2114a/benchmark_data_storage0111-ib.pocstor_51285_object14513/head//22 by client.275175.0:14514 2014-07-30 14:19:19.328359 2014-08-13 18:06:06.457998 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 314bd94a/benchmark_data_storage0111-ib.pocstor_51285_object20377/head//22 by client.275175.0:20378 2014-07-30 14:19:37.347686 2014-08-13 18:06:06.458010 7f5c973a57a0 20 read_log 191727'397 (0'0) modify ecde14a/benchmark_data_storage0111-ib.pocstor_51285_object20680/head//22 by client.275175.0:20681 2014-07-30 14:19:38.214708 2014-08-13 18:06:06.458021 7f5c973a57a0 20 read_log 191727'399 (0'0) modify c1b5e94a/benchmark_data_storage0111-ib.pocstor_51285_object22884/head//22 by client.275175.0:22885 2014-07-30 14:19:45.695087 2014-08-13 18:06:06.458040 7f5c973a57a0 20 read_log 191727'410 (0'0) modify f668f14a/benchmark_data_storage0111-ib.pocstor_51285_object27436/head//22 by client.275175.0:27437 2014-07-30 14:19:59.050386 2014-08-13 18:06:06.458052 7f5c973a57a0 20 read_log 191727'412 (0'0) modify a60f594a/benchmark_data_storage0111-ib.pocstor_51285_object27958/head//22 by client.275175.0:27959 2014-07-30 14:20:00.913642 2014-08-13 18:06:06.458064 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 3bf314a/benchmark_data_storage0111-ib.pocstor_51285_object28306/head//22 by client.275175.0:28307 2014-07-30 14:20:01.883309 2014-08-13 18:06:06.458076 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 5bef594a/benchmark_data_storage0111-ib.pocstor_51285_object30136/head//22 by client.275175.0:30137 2014-07-30 14:20:07.403920 2014-08-13 18:06:06.458088 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 433d714a/benchmark_data_storage0111-ib.pocstor_51285_object32336/head//22 by client.275175.0:32337 2014-07-30 14:20:13.856982 2014-08-13 18:06:06.458100 7f5c973a57a0 20 read_log 191727'431 (0'0) modify a310894a/benchmark_data_storage0111-ib.pocstor_51285_object38685/head//22 by client.275175.0:38686 2014-07-30 14:20:33.197517 2014-08-13 18:06:06.458112 7f5c973a57a0 20 read_log 192674'432 (0'0) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2143 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458154 7f5c973a57a0 20 read_log 192674'433 (192674'432) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2144 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458167 7f5c973a57a0 20 read_log 192674'434 (192674'433) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2145 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458180 7f5c973a57a0 20 read_log 192674'435 (192674'434) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2146 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458192 7f5c973a57a0 20 read_log 192674'436 (192674'435) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2147 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458205 7f5c973a57a0 20 read_log 192674'437 (192674'436) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2148 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458217 7f5c973a57a0 20 read_log 192674'438 (192674'437) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2149 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458229 7f5c973a57a0 20 read_log 192674'439 (192674'438) modify 353ce94a/rb.0.43b76.238e1f29.0000000000ae/head//22 by client.279772.1:2150 2014-07-31 18:03:43.955000 2014-08-13 18:06:06.458241 7f5c973a57a0 20 read_log 192674'440 (0'0) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4151 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458262 7f5c973a57a0 20 read_log 192674'441 (192674'440) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4152 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458275 7f5c973a57a0 20 read_log 192674'442 (192674'441) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4153 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458287 7f5c973a57a0 20 read_log 192674'443 (192674'442) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4154 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458299 7f5c973a57a0 20 read_log 192674'444 (192674'443) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4155 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458311 7f5c973a57a0 20 read_log 192674'445 (192674'444) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4156 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458323 7f5c973a57a0 20 read_log 192674'446 (192674'445) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4157 2014-07-31 18:03:46.080000 2014-08-13 18:06:06.458358 7f5c973a57a0 20 read_log 192674'447 (192674'446) modify 3f9bf14a/rb.0.43b76.238e1f29.0000000001a9/head//22 by client.279772.1:4158 2014-07-31 18:03:46.113000 2014-08-13 18:06:06.458383 7f5c973a57a0 20 read_log 192674'448 (0'0) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6463 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458400 7f5c973a57a0 20 read_log 192674'449 (192674'448) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6464 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458412 7f5c973a57a0 20 read_log 192674'450 (192674'449) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6465 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458438 7f5c973a57a0 20 read_log 192674'451 (192674'450) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6466 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458455 7f5c973a57a0 20 read_log 192674'452 (192674'451) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6467 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458467 7f5c973a57a0 20 read_log 192674'453 (192674'452) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6468 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458490 7f5c973a57a0 20 read_log 192674'454 (192674'453) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6469 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458503 7f5c973a57a0 20 read_log 192674'455 (192674'454) modify e58814a/rb.0.43b76.238e1f29.0000000002ca/head//22 by client.279772.1:6470 2014-07-31 18:03:48.593000 2014-08-13 18:06:06.458515 7f5c973a57a0 20 read_log 192888'456 (0'0) modify f1ce114a/benchmark_data_storage0111-ib.pocstor_35625_object1859/head//22 by client.940330.0:1860 2014-08-11 16:29:48.025890 2014-08-13 18:06:06.458527 7f5c973a57a0 20 read_log 192888'457 (0'0) modify c410b94a/benchmark_data_storage0111-ib.pocstor_35625_object2761/head//22 by client.940330.0:2762 2014-08-11 16:29:51.480235 2014-08-13 18:06:06.458540 7f5c973a57a0 20 read_log 192888'458 (0'0) modify bd08994a/benchmark_data_storage0111-ib.pocstor_35625_object5903/head//22 by client.940330.0:5904 2014-08-11 16:30:03.219611 2014-08-13 18:06:06.458552 7f5c973a57a0 20 read_log 192888'459 (0'0) modify f186c94a/benchmark_data_storage0111-ib.pocstor_35625_object9344/head//22 by client.940330.0:9345 2014-08-11 16:30:15.535506 2014-08-13 18:06:06.458567 7f5c973a57a0 20 read_log 192888'460 (0'0) modify b1b4914a/benchmark_data_storage0111-ib.pocstor_35625_object9958/head//22 by client.940330.0:9959 2014-08-11 16:30:17.709794 2014-08-13 18:06:06.458579 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 98e014a/benchmark_data_storage0111-ib.pocstor_35625_object13525/head//22 by client.940330.0:13526 2014-08-11 16:30:31.880706 2014-08-13 18:06:06.458591 7f5c973a57a0 20 read_log 192888'462 (0'0) modify ff0bf14a/benchmark_data_storage0111-ib.pocstor_35625_object13835/head//22 by client.940330.0:13836 2014-08-11 16:30:33.028338 2014-08-13 18:06:06.458603 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 658ec14a/benchmark_data_storage0111-ib.pocstor_35625_object17710/head//22 by client.940330.0:17711 2014-08-11 16:30:47.626530 2014-08-13 18:06:06.458615 7f5c973a57a0 20 read_log 192888'464 (0'0) modify d832994a/benchmark_data_storage0111-ib.pocstor_35625_object20941/head//22 by client.940330.0:20942 2014-08-11 16:31:00.478687 2014-08-13 18:06:06.458640 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 383bb14a/benchmark_data_storage0111-ib.pocstor_35625_object22915/head//22 by client.940330.0:22916 2014-08-11 16:31:08.109239 2014-08-13 18:06:06.458653 7f5c973a57a0 20 read_log 192888'466 (0'0) modify c32af94a/benchmark_data_storage0111-ib.pocstor_35625_object27150/head//22 by client.940330.0:27151 2014-08-11 16:31:24.381537 2014-08-13 18:06:06.458676 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 9984b14a/benchmark_data_storage0111-ib.pocstor_35625_object29527/head//22 by client.940330.0:29528 2014-08-11 16:31:33.673173 2014-08-13 18:06:06.458691 7f5c973a57a0 20 read_log 192888'468 (0'0) modify 6888194a/benchmark_data_storage0111-ib.pocstor_35625_object29996/head//22 by client.940330.0:29997 2014-08-11 16:31:35.446557 2014-08-13 18:06:06.458704 7f5c973a57a0 20 read_log 192888'469 (192888'456) delete f1ce114a/benchmark_data_storage0111-ib.pocstor_35625_object1859/head//22 by client.940330.0:33167 2014-08-11 16:31:41.955710 2014-08-13 18:06:06.458716 7f5c973a57a0 20 read_log 192888'470 (192888'457) delete c410b94a/benchmark_data_storage0111-ib.pocstor_35625_object2761/head//22 by client.940330.0:34069 2014-08-11 16:31:42.186630 2014-08-13 18:06:06.458728 7f5c973a57a0 20 read_log 192888'471 (192888'458) delete bd08994a/benchmark_data_storage0111-ib.pocstor_35625_object5903/head//22 by client.940330.0:37211 2014-08-11 16:31:43.049509 2014-08-13 18:06:06.458762 7f5c973a57a0 20 read_log 192888'472 (192888'459) delete f186c94a/benchmark_data_storage0111-ib.pocstor_35625_object9344/head//22 by client.940330.0:40652 2014-08-11 16:31:43.932129 2014-08-13 18:06:06.458775 7f5c973a57a0 20 read_log 192888'473 (192888'460) delete b1b4914a/benchmark_data_storage0111-ib.pocstor_35625_object9958/head//22 by client.940330.0:41266 2014-08-11 16:31:44.102231 2014-08-13 18:06:06.458787 7f5c973a57a0 20 read_log 192888'474 (192888'461) delete 98e014a/benchmark_data_storage0111-ib.pocstor_35625_object13525/head//22 by client.940330.0:44833 2014-08-11 16:31:45.060642 2014-08-13 18:06:06.458799 7f5c973a57a0 20 read_log 192888'475 (192888'462) delete ff0bf14a/benchmark_data_storage0111-ib.pocstor_35625_object13835/head//22 by client.940330.0:45143 2014-08-11 16:31:45.140180 2014-08-13 18:06:06.458811 7f5c973a57a0 20 read_log 192888'476 (192888'463) delete 658ec14a/benchmark_data_storage0111-ib.pocstor_35625_object17710/head//22 by client.940330.0:49018 2014-08-11 16:31:46.194261 2014-08-13 18:06:06.458824 7f5c973a57a0 20 read_log 192888'477 (192888'464) delete d832994a/benchmark_data_storage0111-ib.pocstor_35625_object20941/head//22 by client.940330.0:52249 2014-08-11 16:31:47.146610 2014-08-13 18:06:06.458836 7f5c973a57a0 20 read_log 192888'478 (192888'465) delete 383bb14a/benchmark_data_storage0111-ib.pocstor_35625_object22915/head//22 by client.940330.0:54223 2014-08-11 16:31:47.734145 2014-08-13 18:06:06.458867 7f5c973a57a0 20 read_log 192888'479 (192888'466) delete c32af94a/benchmark_data_storage0111-ib.pocstor_35625_object27150/head//22 by client.940330.0:58458 2014-08-11 16:31:48.966898 2014-08-13 18:06:06.458880 7f5c973a57a0 20 read_log 192888'480 (192888'467) delete 9984b14a/benchmark_data_storage0111-ib.pocstor_35625_object29527/head//22 by client.940330.0:60835 2014-08-11 16:31:49.621632 2014-08-13 18:06:06.458892 7f5c973a57a0 20 read_log 192888'481 (192888'468) delete 6888194a/benchmark_data_storage0111-ib.pocstor_35625_object29996/head//22 by client.940330.0:61304 2014-08-11 16:31:49.754541 2014-08-13 18:06:06.458907 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 717dd14a/benchmark_data_storage0111-ib.pocstor_64498_object1244/head//22 by client.1049635.0:1245 2014-08-12 17:17:21.182110 2014-08-13 18:06:06.458921 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 874d314a/benchmark_data_storage0111-ib.pocstor_64498_object2274/head//22 by client.1049635.0:2275 2014-08-12 17:17:24.419676 2014-08-13 18:06:06.458935 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 2d5914a/benchmark_data_storage0111-ib.pocstor_64498_object6707/head//22 by client.1049635.0:6708 2014-08-12 17:17:38.192236 2014-08-13 18:06:06.458948 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 39c6194a/benchmark_data_storage0111-ib.pocstor_64498_object7150/head//22 by client.1049635.0:7151 2014-08-12 17:17:39.556441 2014-08-13 18:06:06.458969 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 8062814a/benchmark_data_storage0111-ib.pocstor_64498_object10380/head//22 by client.1049635.0:10381 2014-08-12 17:17:49.005388 2014-08-13 18:06:06.458984 7f5c973a57a0 20 read_log 194215'487 (0'0) modify c3f4a94a/benchmark_data_storage0111-ib.pocstor_64498_object10397/head//22 by client.1049635.0:10398 2014-08-12 17:17:49.059861 2014-08-13 18:06:06.458996 7f5c973a57a0 20 read_log 194215'488 (0'0) modify a279614a/benchmark_data_storage0111-ib.pocstor_64498_object10557/head//22 by client.1049635.0:10558 2014-08-12 17:17:49.738377 2014-08-13 18:06:06.459066 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 1477f14a/benchmark_data_storage0111-ib.pocstor_64498_object16559/head//22 by client.1049635.0:16560 2014-08-12 17:18:09.684658 2014-08-13 18:06:06.459082 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 7685e14a/benchmark_data_storage0111-ib.pocstor_64498_object20125/head//22 by client.1049635.0:20126 2014-08-12 17:18:23.251118 2014-08-13 18:06:06.459095 7f5c973a57a0 20 read_log 194215'491 (0'0) modify a54ef94a/benchmark_data_storage0111-ib.pocstor_64498_object20451/head//22 by client.1049635.0:20452 2014-08-12 17:18:24.487405 2014-08-13 18:06:06.459109 7f5c973a57a0 20 read_log 194215'492 (0'0) modify cd23894a/benchmark_data_storage0111-ib.pocstor_64498_object22255/head//22 by client.1049635.0:22256 2014-08-12 17:18:31.291835 2014-08-13 18:06:06.459123 7f5c973a57a0 20 read_log 194215'493 (0'0) modify bd71494a/benchmark_data_storage0111-ib.pocstor_64498_object22587/head//22 by client.1049635.0:22588 2014-08-12 17:18:32.598005 2014-08-13 18:06:06.459135 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 56e0814a/benchmark_data_storage0111-ib.pocstor_64498_object23154/head//22 by client.1049635.0:23155 2014-08-12 17:18:34.705690 2014-08-13 18:06:06.459155 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 6ba494a/benchmark_data_storage0111-ib.pocstor_64498_object26578/head//22 by client.1049635.0:26579 2014-08-12 17:18:47.575725 2014-08-13 18:06:06.459170 7f5c973a57a0 20 read_log 194215'496 (0'0) modify fe95014a/benchmark_data_storage0111-ib.pocstor_64498_object26843/head//22 by client.1049635.0:26844 2014-08-12 17:18:48.565141 2014-08-13 18:06:06.459183 7f5c973a57a0 20 read_log 194215'497 (0'0) modify b1a6514a/benchmark_data_storage0111-ib.pocstor_64498_object33383/head//22 by client.1049635.0:33384 2014-08-12 17:19:13.830703 2014-08-13 18:06:06.459195 7f5c973a57a0 20 read_log 194215'498 (194215'482) delete 717dd14a/benchmark_data_storage0111-ib.pocstor_64498_object1244/head//22 by client.1049635.0:35447 2014-08-12 17:19:18.399394 2014-08-13 18:06:06.459207 7f5c973a57a0 20 read_log 194215'499 (194215'483) delete 874d314a/benchmark_data_storage0111-ib.pocstor_64498_object2274/head//22 by client.1049635.0:36477 2014-08-12 17:19:18.725286 2014-08-13 18:06:06.459222 7f5c973a57a0 20 read_log 194215'500 (194215'484) delete 2d5914a/benchmark_data_storage0111-ib.pocstor_64498_object6707/head//22 by client.1049635.0:40910 2014-08-12 17:19:20.188979 2014-08-13 18:06:06.459234 7f5c973a57a0 20 read_log 194215'501 (194215'485) delete 39c6194a/benchmark_data_storage0111-ib.pocstor_64498_object7150/head//22 by client.1049635.0:41353 2014-08-12 17:19:20.334217 2014-08-13 18:06:06.459255 7f5c973a57a0 20 read_log 194215'502 (194215'486) delete 8062814a/benchmark_data_storage0111-ib.pocstor_64498_object10380/head//22 by client.1049635.0:44583 2014-08-12 17:19:21.383194 2014-08-13 18:06:06.459268 7f5c973a57a0 20 read_log 194215'503 (194215'487) delete c3f4a94a/benchmark_data_storage0111-ib.pocstor_64498_object10397/head//22 by client.1049635.0:44600 2014-08-12 17:19:21.388686 2014-08-13 18:06:06.459280 7f5c973a57a0 20 read_log 194215'504 (194215'488) delete a279614a/benchmark_data_storage0111-ib.pocstor_64498_object10557/head//22 by client.1049635.0:44760 2014-08-12 17:19:21.438346 2014-08-13 18:06:06.459292 7f5c973a57a0 20 read_log 194215'505 (194215'489) delete 1477f14a/benchmark_data_storage0111-ib.pocstor_64498_object16559/head//22 by client.1049635.0:50762 2014-08-12 17:19:23.369862 2014-08-13 18:06:06.459312 7f5c973a57a0 20 read_log 194215'506 (194215'490) delete 7685e14a/benchmark_data_storage0111-ib.pocstor_64498_object20125/head//22 by client.1049635.0:54328 2014-08-12 17:19:24.569386 2014-08-13 18:06:06.459327 7f5c973a57a0 20 read_log 194215'507 (194215'491) delete a54ef94a/benchmark_data_storage0111-ib.pocstor_64498_object20451/head//22 by client.1049635.0:54654 2014-08-12 17:19:24.684566 2014-08-13 18:06:06.459340 7f5c973a57a0 20 read_log 194215'508 (194215'492) delete cd23894a/benchmark_data_storage0111-ib.pocstor_64498_object22255/head//22 by client.1049635.0:56458 2014-08-12 17:19:25.327311 2014-08-13 18:06:06.459352 7f5c973a57a0 20 read_log 194215'509 (194215'493) delete bd71494a/benchmark_data_storage0111-ib.pocstor_64498_object22587/head//22 by client.1049635.0:56790 2014-08-12 17:19:25.453033 2014-08-13 18:06:06.459364 7f5c973a57a0 20 read_log 194215'510 (194215'494) delete 56e0814a/benchmark_data_storage0111-ib.pocstor_64498_object23154/head//22 by client.1049635.0:57357 2014-08-12 17:19:25.674075 2014-08-13 18:06:06.459388 7f5c973a57a0 20 read_log 194215'511 (194215'495) delete 6ba494a/benchmark_data_storage0111-ib.pocstor_64498_object26578/head//22 by client.1049635.0:60781 2014-08-12 17:19:26.849926 2014-08-13 18:06:06.459402 7f5c973a57a0 20 read_log 194215'512 (194215'496) delete fe95014a/benchmark_data_storage0111-ib.pocstor_64498_object26843/head//22 by client.1049635.0:61046 2014-08-12 17:19:26.940480 2014-08-13 18:06:06.459414 7f5c973a57a0 20 read_log 194215'513 (194215'497) delete b1a6514a/benchmark_data_storage0111-ib.pocstor_64498_object33383/head//22 by client.1049635.0:67586 2014-08-12 17:19:29.249756 2014-08-13 18:06:06.459558 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.459581 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.459597 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002181 0 0.000000 2014-08-13 18:06:06.459609 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.459617 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.459626 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] log((0'0,194215'513], crt=192888'481) 2014-08-13 18:06:06.459636 7f5c973a57a0 10 osd.4 194487 pgid 22.1a0 coll 22.1a0_head 2014-08-13 18:06:06.459803 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.1a0 2014-08-13 18:06:06.459812 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.459839 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1a0(unlocked)] enter Initial 2014-08-13 18:06:06.459853 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1a0(unlocked)] enter NotTrimming 2014-08-13 18:06:06.460001 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.460182 7f5c973a57a0 20 read_log 191648'1 (0'0) modify 600b39a0/benchmark_data_storage0111-ib.pocstor_40745_object110/head//22 by client.273158.0:111 2014-07-30 14:08:49.119811 2014-08-13 18:06:06.460208 7f5c973a57a0 20 read_log 191648'26 (0'0) modify 83a281a0/benchmark_data_storage0111-ib.pocstor_40745_object1682/head//22 by client.273158.0:1683 2014-07-30 14:08:54.961875 2014-08-13 18:06:06.460221 7f5c973a57a0 20 read_log 191648'28 (0'0) modify b9ba9a0/benchmark_data_storage0111-ib.pocstor_40745_object1712/head//22 by client.273158.0:1713 2014-07-30 14:08:55.072096 2014-08-13 18:06:06.460233 7f5c973a57a0 20 read_log 191648'86 (0'0) modify fa9371a0/benchmark_data_storage0111-ib.pocstor_40745_object5208/head//22 by client.273158.0:5209 2014-07-30 14:09:12.265080 2014-08-13 18:06:06.460248 7f5c973a57a0 20 read_log 191648'94 (0'0) modify 7e2321a0/benchmark_data_storage0111-ib.pocstor_40745_object5523/head//22 by client.273158.0:5524 2014-07-30 14:09:13.529328 2014-08-13 18:06:06.460260 7f5c973a57a0 20 read_log 191648'150 (0'0) modify d713f9a0/benchmark_data_storage0111-ib.pocstor_40745_object8715/head//22 by client.273158.0:8716 2014-07-30 14:09:30.052874 2014-08-13 18:06:06.460275 7f5c973a57a0 20 read_log 191648'275 (0'0) modify cfba71a0/benchmark_data_storage0111-ib.pocstor_40745_object16955/head//22 by client.273158.0:16956 2014-07-30 14:10:12.812878 2014-08-13 18:06:06.460287 7f5c973a57a0 20 read_log 191648'325 (0'0) modify 82ee1a0/benchmark_data_storage0111-ib.pocstor_40745_object19964/head//22 by client.273158.0:19965 2014-07-30 14:10:31.165516 2014-08-13 18:06:06.460300 7f5c973a57a0 20 read_log 191648'337 (0'0) modify 98ad99a0/benchmark_data_storage0111-ib.pocstor_40745_object20610/head//22 by client.273158.0:20611 2014-07-30 14:10:34.369909 2014-08-13 18:06:06.460312 7f5c973a57a0 20 read_log 191727'373 (0'0) modify b2cb59a0/benchmark_data_storage0111-ib.pocstor_51285_object401/head//22 by client.275175.0:402 2014-07-30 14:18:35.044411 2014-08-13 18:06:06.460324 7f5c973a57a0 20 read_log 191727'374 (0'0) modify f07e29a0/benchmark_data_storage0111-ib.pocstor_51285_object569/head//22 by client.275175.0:570 2014-07-30 14:18:35.720956 2014-08-13 18:06:06.460336 7f5c973a57a0 20 read_log 191727'379 (0'0) modify ebe239a0/benchmark_data_storage0111-ib.pocstor_51285_object2631/head//22 by client.275175.0:2632 2014-07-30 14:18:43.039442 2014-08-13 18:06:06.460348 7f5c973a57a0 20 read_log 191727'381 (0'0) modify 590059a0/benchmark_data_storage0111-ib.pocstor_51285_object3955/head//22 by client.275175.0:3956 2014-07-30 14:18:47.219045 2014-08-13 18:06:06.460362 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 5a8a61a0/benchmark_data_storage0111-ib.pocstor_51285_object4222/head//22 by client.275175.0:4223 2014-07-30 14:18:48.030043 2014-08-13 18:06:06.460374 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 133461a0/benchmark_data_storage0111-ib.pocstor_51285_object7769/head//22 by client.275175.0:7770 2014-07-30 14:18:58.883388 2014-08-13 18:06:06.460388 7f5c973a57a0 20 read_log 191727'389 (0'0) modify b4f0c1a0/benchmark_data_storage0111-ib.pocstor_51285_object8978/head//22 by client.275175.0:8979 2014-07-30 14:19:02.286745 2014-08-13 18:06:06.460408 7f5c973a57a0 20 read_log 191727'395 (0'0) modify f61981a0/benchmark_data_storage0111-ib.pocstor_51285_object11051/head//22 by client.275175.0:11052 2014-07-30 14:19:08.188204 2014-08-13 18:06:06.460423 7f5c973a57a0 20 read_log 191727'396 (0'0) modify be1ee9a0/benchmark_data_storage0111-ib.pocstor_51285_object12293/head//22 by client.275175.0:12294 2014-07-30 14:19:12.460330 2014-08-13 18:06:06.460435 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 9ff0e1a0/benchmark_data_storage0111-ib.pocstor_51285_object12356/head//22 by client.275175.0:12357 2014-07-30 14:19:12.633360 2014-08-13 18:06:06.460447 7f5c973a57a0 20 read_log 191727'405 (0'0) modify d15511a0/benchmark_data_storage0111-ib.pocstor_51285_object16469/head//22 by client.275175.0:16470 2014-07-30 14:19:25.579341 2014-08-13 18:06:06.460459 7f5c973a57a0 20 read_log 191727'410 (0'0) modify eb4a59a0/benchmark_data_storage0111-ib.pocstor_51285_object20107/head//22 by client.275175.0:20108 2014-07-30 14:19:36.565767 2014-08-13 18:06:06.460471 7f5c973a57a0 20 read_log 191727'416 (0'0) modify a29c11a0/benchmark_data_storage0111-ib.pocstor_51285_object22774/head//22 by client.275175.0:22775 2014-07-30 14:19:45.351281 2014-08-13 18:06:06.460483 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 440b39a0/benchmark_data_storage0111-ib.pocstor_51285_object23168/head//22 by client.275175.0:23169 2014-07-30 14:19:46.498028 2014-08-13 18:06:06.460495 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 47c059a0/benchmark_data_storage0111-ib.pocstor_51285_object24890/head//22 by client.275175.0:24891 2014-07-30 14:19:51.330283 2014-08-13 18:06:06.460507 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 426b9a0/benchmark_data_storage0111-ib.pocstor_51285_object30670/head//22 by client.275175.0:30671 2014-07-30 14:20:09.052033 2014-08-13 18:06:06.460520 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 87c9b1a0/benchmark_data_storage0111-ib.pocstor_51285_object30816/head//22 by client.275175.0:30817 2014-07-30 14:20:09.460213 2014-08-13 18:06:06.460532 7f5c973a57a0 20 read_log 191727'430 (0'0) modify b0e561a0/benchmark_data_storage0111-ib.pocstor_51285_object31260/head//22 by client.275175.0:31261 2014-07-30 14:20:10.724977 2014-08-13 18:06:06.460544 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 546a11a0/benchmark_data_storage0111-ib.pocstor_51285_object31654/head//22 by client.275175.0:31655 2014-07-30 14:20:11.883103 2014-08-13 18:06:06.460556 7f5c973a57a0 20 read_log 191727'441 (0'0) modify 338bd1a0/benchmark_data_storage0111-ib.pocstor_51285_object35804/head//22 by client.275175.0:35805 2014-07-30 14:20:24.662355 2014-08-13 18:06:06.460568 7f5c973a57a0 20 read_log 192674'447 (0'0) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6383 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460581 7f5c973a57a0 20 read_log 192674'448 (192674'447) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6384 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460593 7f5c973a57a0 20 read_log 192674'449 (192674'448) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6385 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460605 7f5c973a57a0 20 read_log 192674'450 (192674'449) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6386 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460617 7f5c973a57a0 20 read_log 192674'451 (192674'450) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6387 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460629 7f5c973a57a0 20 read_log 192674'452 (192674'451) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6388 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460641 7f5c973a57a0 20 read_log 192674'453 (192674'452) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6389 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460665 7f5c973a57a0 20 read_log 192674'454 (192674'453) modify 852409a0/rb.0.43b76.238e1f29.0000000002c0/head//22 by client.279772.1:6390 2014-07-31 18:03:48.454000 2014-08-13 18:06:06.460681 7f5c973a57a0 20 read_log 192888'455 (0'0) modify 5b49f1a0/benchmark_data_storage0111-ib.pocstor_35625_object376/head//22 by client.940330.0:377 2014-08-11 16:29:42.200388 2014-08-13 18:06:06.460694 7f5c973a57a0 20 read_log 192888'456 (0'0) modify ded991a0/benchmark_data_storage0111-ib.pocstor_35625_object1054/head//22 by client.940330.0:1055 2014-08-11 16:29:44.993610 2014-08-13 18:06:06.460708 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 6c4139a0/benchmark_data_storage0111-ib.pocstor_35625_object3140/head//22 by client.940330.0:3141 2014-08-11 16:29:53.077569 2014-08-13 18:06:06.460721 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 6bee09a0/benchmark_data_storage0111-ib.pocstor_35625_object3321/head//22 by client.940330.0:3322 2014-08-11 16:29:53.817583 2014-08-13 18:06:06.460736 7f5c973a57a0 20 read_log 192888'459 (0'0) modify a731a1a0/benchmark_data_storage0111-ib.pocstor_35625_object4118/head//22 by client.940330.0:4119 2014-08-11 16:29:56.760193 2014-08-13 18:06:06.460748 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 702041a0/benchmark_data_storage0111-ib.pocstor_35625_object6608/head//22 by client.940330.0:6609 2014-08-11 16:30:05.859199 2014-08-13 18:06:06.460761 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 440fb9a0/benchmark_data_storage0111-ib.pocstor_35625_object8841/head//22 by client.940330.0:8842 2014-08-11 16:30:13.777862 2014-08-13 18:06:06.460773 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 90a701a0/benchmark_data_storage0111-ib.pocstor_35625_object9970/head//22 by client.940330.0:9971 2014-08-11 16:30:17.769307 2014-08-13 18:06:06.460785 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 6653b9a0/benchmark_data_storage0111-ib.pocstor_35625_object21052/head//22 by client.940330.0:21053 2014-08-11 16:31:01.029940 2014-08-13 18:06:06.460797 7f5c973a57a0 20 read_log 192888'464 (0'0) modify ad1361a0/benchmark_data_storage0111-ib.pocstor_35625_object21274/head//22 by client.940330.0:21275 2014-08-11 16:31:01.852186 2014-08-13 18:06:06.460811 7f5c973a57a0 20 read_log 192888'465 (0'0) modify ae5089a0/benchmark_data_storage0111-ib.pocstor_35625_object22073/head//22 by client.940330.0:22074 2014-08-11 16:31:04.844227 2014-08-13 18:06:06.460823 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 1aad71a0/benchmark_data_storage0111-ib.pocstor_35625_object29233/head//22 by client.940330.0:29234 2014-08-11 16:31:32.559058 2014-08-13 18:06:06.460838 7f5c973a57a0 20 read_log 192888'467 (192888'455) delete 5b49f1a0/benchmark_data_storage0111-ib.pocstor_35625_object376/head//22 by client.940330.0:31684 2014-08-11 16:31:41.586859 2014-08-13 18:06:06.460852 7f5c973a57a0 20 read_log 192888'468 (192888'456) delete ded991a0/benchmark_data_storage0111-ib.pocstor_35625_object1054/head//22 by client.940330.0:32362 2014-08-11 16:31:41.762189 2014-08-13 18:06:06.460866 7f5c973a57a0 20 read_log 192888'469 (192888'457) delete 6c4139a0/benchmark_data_storage0111-ib.pocstor_35625_object3140/head//22 by client.940330.0:34448 2014-08-11 16:31:42.291153 2014-08-13 18:06:06.460879 7f5c973a57a0 20 read_log 192888'470 (192888'458) delete 6bee09a0/benchmark_data_storage0111-ib.pocstor_35625_object3321/head//22 by client.940330.0:34629 2014-08-11 16:31:42.341146 2014-08-13 18:06:06.460893 7f5c973a57a0 20 read_log 192888'471 (192888'459) delete a731a1a0/benchmark_data_storage0111-ib.pocstor_35625_object4118/head//22 by client.940330.0:35426 2014-08-11 16:31:42.568026 2014-08-13 18:06:06.460906 7f5c973a57a0 20 read_log 192888'472 (192888'460) delete 702041a0/benchmark_data_storage0111-ib.pocstor_35625_object6608/head//22 by client.940330.0:37916 2014-08-11 16:31:43.236526 2014-08-13 18:06:06.460921 7f5c973a57a0 20 read_log 192888'473 (192888'461) delete 440fb9a0/benchmark_data_storage0111-ib.pocstor_35625_object8841/head//22 by client.940330.0:40149 2014-08-11 16:31:43.804048 2014-08-13 18:06:06.460959 7f5c973a57a0 20 read_log 192888'474 (192888'462) delete 90a701a0/benchmark_data_storage0111-ib.pocstor_35625_object9970/head//22 by client.940330.0:41278 2014-08-11 16:31:44.104809 2014-08-13 18:06:06.460975 7f5c973a57a0 20 read_log 192888'475 (192888'463) delete 6653b9a0/benchmark_data_storage0111-ib.pocstor_35625_object21052/head//22 by client.940330.0:52360 2014-08-11 16:31:47.176053 2014-08-13 18:06:06.460987 7f5c973a57a0 20 read_log 192888'476 (192888'464) delete ad1361a0/benchmark_data_storage0111-ib.pocstor_35625_object21274/head//22 by client.940330.0:52582 2014-08-11 16:31:47.233382 2014-08-13 18:06:06.461002 7f5c973a57a0 20 read_log 192888'477 (192888'465) delete ae5089a0/benchmark_data_storage0111-ib.pocstor_35625_object22073/head//22 by client.940330.0:53381 2014-08-11 16:31:47.464951 2014-08-13 18:06:06.461014 7f5c973a57a0 20 read_log 192888'478 (192888'466) delete 1aad71a0/benchmark_data_storage0111-ib.pocstor_35625_object29233/head//22 by client.940330.0:60541 2014-08-11 16:31:49.533375 2014-08-13 18:06:06.461028 7f5c973a57a0 20 read_log 194215'479 (0'0) modify c87389a0/benchmark_data_storage0111-ib.pocstor_64498_object1497/head//22 by client.1049635.0:1498 2014-08-12 17:17:21.985253 2014-08-13 18:06:06.461045 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 448151a0/benchmark_data_storage0111-ib.pocstor_64498_object2771/head//22 by client.1049635.0:2772 2014-08-12 17:17:26.046075 2014-08-13 18:06:06.461060 7f5c973a57a0 20 read_log 194215'481 (0'0) modify c79731a0/benchmark_data_storage0111-ib.pocstor_64498_object3592/head//22 by client.1049635.0:3593 2014-08-12 17:17:28.779857 2014-08-13 18:06:06.461072 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 150129a0/benchmark_data_storage0111-ib.pocstor_64498_object8387/head//22 by client.1049635.0:8388 2014-08-12 17:17:43.232331 2014-08-13 18:06:06.461100 7f5c973a57a0 20 read_log 194215'483 (0'0) modify b16921a0/benchmark_data_storage0111-ib.pocstor_64498_object10172/head//22 by client.1049635.0:10173 2014-08-12 17:17:48.419376 2014-08-13 18:06:06.461115 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 9c1c49a0/benchmark_data_storage0111-ib.pocstor_64498_object11620/head//22 by client.1049635.0:11621 2014-08-12 17:17:53.272455 2014-08-13 18:06:06.461127 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 528881a0/benchmark_data_storage0111-ib.pocstor_64498_object15022/head//22 by client.1049635.0:15023 2014-08-12 17:18:04.042953 2014-08-13 18:06:06.461140 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 786519a0/benchmark_data_storage0111-ib.pocstor_64498_object15081/head//22 by client.1049635.0:15082 2014-08-12 17:18:04.233294 2014-08-13 18:06:06.461160 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 8d821a0/benchmark_data_storage0111-ib.pocstor_64498_object16553/head//22 by client.1049635.0:16554 2014-08-12 17:18:09.662764 2014-08-13 18:06:06.461172 7f5c973a57a0 20 read_log 194215'488 (0'0) modify b4c219a0/benchmark_data_storage0111-ib.pocstor_64498_object16949/head//22 by client.1049635.0:16950 2014-08-12 17:18:11.172551 2014-08-13 18:06:06.461185 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 98d7d1a0/benchmark_data_storage0111-ib.pocstor_64498_object17756/head//22 by client.1049635.0:17757 2014-08-12 17:18:14.232965 2014-08-13 18:06:06.461197 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 427319a0/benchmark_data_storage0111-ib.pocstor_64498_object23448/head//22 by client.1049635.0:23449 2014-08-12 17:18:35.794615 2014-08-13 18:06:06.461211 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 5028e9a0/benchmark_data_storage0111-ib.pocstor_64498_object24117/head//22 by client.1049635.0:24118 2014-08-12 17:18:38.288347 2014-08-13 18:06:06.461223 7f5c973a57a0 20 read_log 194215'492 (0'0) modify e52d1a0/benchmark_data_storage0111-ib.pocstor_64498_object25642/head//22 by client.1049635.0:25643 2014-08-12 17:18:44.097285 2014-08-13 18:06:06.461245 7f5c973a57a0 20 read_log 194215'493 (0'0) modify b1a129a0/benchmark_data_storage0111-ib.pocstor_64498_object28097/head//22 by client.1049635.0:28098 2014-08-12 17:18:53.235708 2014-08-13 18:06:06.461263 7f5c973a57a0 20 read_log 194215'494 (0'0) modify cfd061a0/benchmark_data_storage0111-ib.pocstor_64498_object29846/head//22 by client.1049635.0:29847 2014-08-12 17:18:59.872725 2014-08-13 18:06:06.461297 7f5c973a57a0 20 read_log 194215'495 (0'0) modify a3dfc1a0/benchmark_data_storage0111-ib.pocstor_64498_object32736/head//22 by client.1049635.0:32737 2014-08-12 17:19:11.390281 2014-08-13 18:06:06.461316 7f5c973a57a0 20 read_log 194215'496 (194215'479) delete c87389a0/benchmark_data_storage0111-ib.pocstor_64498_object1497/head//22 by client.1049635.0:35700 2014-08-12 17:19:18.484174 2014-08-13 18:06:06.461329 7f5c973a57a0 20 read_log 194215'497 (194215'480) delete 448151a0/benchmark_data_storage0111-ib.pocstor_64498_object2771/head//22 by client.1049635.0:36974 2014-08-12 17:19:18.890511 2014-08-13 18:06:06.461342 7f5c973a57a0 20 read_log 194215'498 (194215'481) delete c79731a0/benchmark_data_storage0111-ib.pocstor_64498_object3592/head//22 by client.1049635.0:37795 2014-08-12 17:19:19.147688 2014-08-13 18:06:06.461360 7f5c973a57a0 20 read_log 194215'499 (194215'482) delete 150129a0/benchmark_data_storage0111-ib.pocstor_64498_object8387/head//22 by client.1049635.0:42590 2014-08-12 17:19:20.725944 2014-08-13 18:06:06.461375 7f5c973a57a0 20 read_log 194215'500 (194215'483) delete b16921a0/benchmark_data_storage0111-ib.pocstor_64498_object10172/head//22 by client.1049635.0:44375 2014-08-12 17:19:21.318279 2014-08-13 18:06:06.461398 7f5c973a57a0 20 read_log 194215'501 (194215'484) delete 9c1c49a0/benchmark_data_storage0111-ib.pocstor_64498_object11620/head//22 by client.1049635.0:45823 2014-08-12 17:19:21.772568 2014-08-13 18:06:06.461418 7f5c973a57a0 20 read_log 194215'502 (194215'485) delete 528881a0/benchmark_data_storage0111-ib.pocstor_64498_object15022/head//22 by client.1049635.0:49225 2014-08-12 17:19:22.882110 2014-08-13 18:06:06.461431 7f5c973a57a0 20 read_log 194215'503 (194215'486) delete 786519a0/benchmark_data_storage0111-ib.pocstor_64498_object15081/head//22 by client.1049635.0:49284 2014-08-12 17:19:22.899606 2014-08-13 18:06:06.461454 7f5c973a57a0 20 read_log 194215'504 (194215'487) delete 8d821a0/benchmark_data_storage0111-ib.pocstor_64498_object16553/head//22 by client.1049635.0:50756 2014-08-12 17:19:23.368538 2014-08-13 18:06:06.461467 7f5c973a57a0 20 read_log 194215'505 (194215'488) delete b4c219a0/benchmark_data_storage0111-ib.pocstor_64498_object16949/head//22 by client.1049635.0:51152 2014-08-12 17:19:23.492091 2014-08-13 18:06:06.461480 7f5c973a57a0 20 read_log 194215'506 (194215'489) delete 98d7d1a0/benchmark_data_storage0111-ib.pocstor_64498_object17756/head//22 by client.1049635.0:51959 2014-08-12 17:19:23.746670 2014-08-13 18:06:06.461502 7f5c973a57a0 20 read_log 194215'507 (194215'490) delete 427319a0/benchmark_data_storage0111-ib.pocstor_64498_object23448/head//22 by client.1049635.0:57651 2014-08-12 17:19:25.778543 2014-08-13 18:06:06.461515 7f5c973a57a0 20 read_log 194215'508 (194215'491) delete 5028e9a0/benchmark_data_storage0111-ib.pocstor_64498_object24117/head//22 by client.1049635.0:58320 2014-08-12 17:19:26.003026 2014-08-13 18:06:06.461528 7f5c973a57a0 20 read_log 194215'509 (194215'492) delete e52d1a0/benchmark_data_storage0111-ib.pocstor_64498_object25642/head//22 by client.1049635.0:59845 2014-08-12 17:19:26.534634 2014-08-13 18:06:06.461553 7f5c973a57a0 20 read_log 194215'510 (194215'493) delete b1a129a0/benchmark_data_storage0111-ib.pocstor_64498_object28097/head//22 by client.1049635.0:62300 2014-08-12 17:19:27.382909 2014-08-13 18:06:06.461566 7f5c973a57a0 20 read_log 194215'511 (194215'494) delete cfd061a0/benchmark_data_storage0111-ib.pocstor_64498_object29846/head//22 by client.1049635.0:64049 2014-08-12 17:19:28.000324 2014-08-13 18:06:06.461579 7f5c973a57a0 20 read_log 194215'512 (194215'495) delete a3dfc1a0/benchmark_data_storage0111-ib.pocstor_64498_object32736/head//22 by client.1049635.0:66939 2014-08-12 17:19:29.016427 2014-08-13 18:06:06.461701 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.461721 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=0 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.461737 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=0 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.001899 0 0.000000 2014-08-13 18:06:06.461749 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=0 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.461758 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=0 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.461770 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] log((0'0,194215'512], crt=192888'478) 2014-08-13 18:06:06.461780 7f5c973a57a0 10 osd.4 194487 pgid 22.1b8 coll 22.1b8_head 2014-08-13 18:06:06.461955 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.1b8 2014-08-13 18:06:06.461966 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.461994 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1b8(unlocked)] enter Initial 2014-08-13 18:06:06.462006 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1b8(unlocked)] enter NotTrimming 2014-08-13 18:06:06.462118 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.462258 7f5c973a57a0 20 read_log 191648'2 (0'0) modify 4f4b9b8/benchmark_data_storage0111-ib.pocstor_40745_object152/head//22 by client.273158.0:153 2014-07-30 14:08:49.277162 2014-08-13 18:06:06.462279 7f5c973a57a0 20 read_log 191648'42 (0'0) modify 1d5b09b8/benchmark_data_storage0111-ib.pocstor_40745_object2665/head//22 by client.273158.0:2666 2014-07-30 14:08:59.710222 2014-08-13 18:06:06.462300 7f5c973a57a0 20 read_log 191648'55 (0'0) modify 23bf39b8/benchmark_data_storage0111-ib.pocstor_40745_object3547/head//22 by client.273158.0:3548 2014-07-30 14:09:03.912460 2014-08-13 18:06:06.462350 7f5c973a57a0 20 read_log 191648'80 (0'0) modify d1d411b8/benchmark_data_storage0111-ib.pocstor_40745_object5405/head//22 by client.273158.0:5406 2014-07-30 14:09:13.052785 2014-08-13 18:06:06.462363 7f5c973a57a0 20 read_log 191648'97 (0'0) modify a3e0b9b8/benchmark_data_storage0111-ib.pocstor_40745_object6665/head//22 by client.273158.0:6666 2014-07-30 14:09:19.617243 2014-08-13 18:06:06.462375 7f5c973a57a0 20 read_log 191648'109 (0'0) modify fbb4b1b8/benchmark_data_storage0111-ib.pocstor_40745_object7277/head//22 by client.273158.0:7278 2014-07-30 14:09:22.185254 2014-08-13 18:06:06.462387 7f5c973a57a0 20 read_log 191648'135 (0'0) modify b519e1b8/benchmark_data_storage0111-ib.pocstor_40745_object9043/head//22 by client.273158.0:9044 2014-07-30 14:09:31.349216 2014-08-13 18:06:06.462405 7f5c973a57a0 20 read_log 191648'138 (0'0) modify 926801b8/benchmark_data_storage0111-ib.pocstor_40745_object9158/head//22 by client.273158.0:9159 2014-07-30 14:09:31.795808 2014-08-13 18:06:06.462417 7f5c973a57a0 20 read_log 191648'152 (0'0) modify 467fc9b8/benchmark_data_storage0111-ib.pocstor_40745_object9922/head//22 by client.273158.0:9923 2014-07-30 14:09:35.412045 2014-08-13 18:06:06.462429 7f5c973a57a0 20 read_log 191648'232 (0'0) modify 5e601b8/benchmark_data_storage0111-ib.pocstor_40745_object14236/head//22 by client.273158.0:14237 2014-07-30 14:09:58.964895 2014-08-13 18:06:06.462441 7f5c973a57a0 20 read_log 191648'257 (0'0) modify 21c779b8/benchmark_data_storage0111-ib.pocstor_40745_object16169/head//22 by client.273158.0:16170 2014-07-30 14:10:08.403247 2014-08-13 18:06:06.462454 7f5c973a57a0 20 read_log 191648'259 (0'0) modify 4ef7b9b8/benchmark_data_storage0111-ib.pocstor_40745_object16189/head//22 by client.273158.0:16190 2014-07-30 14:10:08.476618 2014-08-13 18:06:06.462467 7f5c973a57a0 20 read_log 191648'281 (0'0) modify 1b9919b8/benchmark_data_storage0111-ib.pocstor_40745_object17481/head//22 by client.273158.0:17482 2014-07-30 14:10:15.640829 2014-08-13 18:06:06.462479 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 77b2f9b8/benchmark_data_storage0111-ib.pocstor_51285_object1125/head//22 by client.275175.0:1126 2014-07-30 14:18:37.920252 2014-08-13 18:06:06.462491 7f5c973a57a0 20 read_log 191727'372 (0'0) modify b63a79b8/benchmark_data_storage0111-ib.pocstor_51285_object4473/head//22 by client.275175.0:4474 2014-07-30 14:18:48.776633 2014-08-13 18:06:06.462503 7f5c973a57a0 20 read_log 191727'374 (0'0) modify caadc1b8/benchmark_data_storage0111-ib.pocstor_51285_object7163/head//22 by client.275175.0:7164 2014-07-30 14:18:57.183927 2014-08-13 18:06:06.462516 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 2b4699b8/benchmark_data_storage0111-ib.pocstor_51285_object7806/head//22 by client.275175.0:7807 2014-07-30 14:18:58.984867 2014-08-13 18:06:06.462528 7f5c973a57a0 20 read_log 191727'377 (0'0) modify 76ab11b8/benchmark_data_storage0111-ib.pocstor_51285_object9639/head//22 by client.275175.0:9640 2014-07-30 14:19:04.178076 2014-08-13 18:06:06.462540 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 557d01b8/benchmark_data_storage0111-ib.pocstor_51285_object10400/head//22 by client.275175.0:10401 2014-07-30 14:19:06.354520 2014-08-13 18:06:06.462552 7f5c973a57a0 20 read_log 191727'380 (0'0) modify 4075b1b8/benchmark_data_storage0111-ib.pocstor_51285_object10661/head//22 by client.275175.0:10662 2014-07-30 14:19:07.097451 2014-08-13 18:06:06.462564 7f5c973a57a0 20 read_log 191727'381 (0'0) modify c80c1b8/benchmark_data_storage0111-ib.pocstor_51285_object11066/head//22 by client.275175.0:11067 2014-07-30 14:19:08.229162 2014-08-13 18:06:06.462584 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 5b0781b8/benchmark_data_storage0111-ib.pocstor_51285_object11101/head//22 by client.275175.0:11102 2014-07-30 14:19:08.328105 2014-08-13 18:06:06.462599 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 70bf71b8/benchmark_data_storage0111-ib.pocstor_51285_object12994/head//22 by client.275175.0:12995 2014-07-30 14:19:14.644235 2014-08-13 18:06:06.462611 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 12c599b8/benchmark_data_storage0111-ib.pocstor_51285_object16328/head//22 by client.275175.0:16329 2014-07-30 14:19:25.074944 2014-08-13 18:06:06.462623 7f5c973a57a0 20 read_log 191727'390 (0'0) modify e330d1b8/benchmark_data_storage0111-ib.pocstor_51285_object17896/head//22 by client.275175.0:17897 2014-07-30 14:19:29.665509 2014-08-13 18:06:06.462635 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 676db1b8/benchmark_data_storage0111-ib.pocstor_51285_object19875/head//22 by client.275175.0:19876 2014-07-30 14:19:35.820034 2014-08-13 18:06:06.462647 7f5c973a57a0 20 read_log 191727'398 (0'0) modify da7959b8/benchmark_data_storage0111-ib.pocstor_51285_object24072/head//22 by client.275175.0:24073 2014-07-30 14:19:49.036353 2014-08-13 18:06:06.462659 7f5c973a57a0 20 read_log 191727'402 (0'0) modify f25ec9b8/benchmark_data_storage0111-ib.pocstor_51285_object25508/head//22 by client.275175.0:25509 2014-07-30 14:19:53.284666 2014-08-13 18:06:06.462671 7f5c973a57a0 20 read_log 191727'409 (0'0) modify 8bbe29b8/benchmark_data_storage0111-ib.pocstor_51285_object29216/head//22 by client.275175.0:29217 2014-07-30 14:20:04.584603 2014-08-13 18:06:06.462683 7f5c973a57a0 20 read_log 191727'411 (0'0) modify ff9289b8/benchmark_data_storage0111-ib.pocstor_51285_object29649/head//22 by client.275175.0:29650 2014-07-30 14:20:05.908631 2014-08-13 18:06:06.462695 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 6298a9b8/benchmark_data_storage0111-ib.pocstor_51285_object34696/head//22 by client.275175.0:34697 2014-07-30 14:20:21.108304 2014-08-13 18:06:06.462707 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 358889b8/benchmark_data_storage0111-ib.pocstor_51285_object36189/head//22 by client.275175.0:36190 2014-07-30 14:20:25.879477 2014-08-13 18:06:06.462719 7f5c973a57a0 20 read_log 191727'423 (0'0) modify 3b29e9b8/benchmark_data_storage0111-ib.pocstor_51285_object37895/head//22 by client.275175.0:37896 2014-07-30 14:20:30.945923 2014-08-13 18:06:06.462734 7f5c973a57a0 20 read_log 191727'425 (0'0) modify c28c69b8/benchmark_data_storage0111-ib.pocstor_51285_object38627/head//22 by client.275175.0:38628 2014-07-30 14:20:33.032499 2014-08-13 18:06:06.462746 7f5c973a57a0 20 read_log 192888'426 (0'0) modify b76ac1b8/benchmark_data_storage0111-ib.pocstor_35625_object1026/head//22 by client.940330.0:1027 2014-08-11 16:29:44.888953 2014-08-13 18:06:06.462760 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 809961b8/benchmark_data_storage0111-ib.pocstor_35625_object1642/head//22 by client.940330.0:1643 2014-08-11 16:29:47.239078 2014-08-13 18:06:06.462772 7f5c973a57a0 20 read_log 192888'428 (0'0) modify aba971b8/benchmark_data_storage0111-ib.pocstor_35625_object12080/head//22 by client.940330.0:12081 2014-08-11 16:30:26.268715 2014-08-13 18:06:06.462786 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 6f0bb1b8/benchmark_data_storage0111-ib.pocstor_35625_object13391/head//22 by client.940330.0:13392 2014-08-11 16:30:31.384561 2014-08-13 18:06:06.462801 7f5c973a57a0 20 read_log 192888'430 (0'0) modify ac4411b8/benchmark_data_storage0111-ib.pocstor_35625_object19519/head//22 by client.940330.0:19520 2014-08-11 16:30:54.825002 2014-08-13 18:06:06.462813 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 2b7e99b8/benchmark_data_storage0111-ib.pocstor_35625_object20876/head//22 by client.940330.0:20877 2014-08-11 16:31:00.231789 2014-08-13 18:06:06.462832 7f5c973a57a0 20 read_log 192888'432 (0'0) modify ad29b8/benchmark_data_storage0111-ib.pocstor_35625_object21963/head//22 by client.940330.0:21964 2014-08-11 16:31:04.430730 2014-08-13 18:06:06.462847 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 354e31b8/benchmark_data_storage0111-ib.pocstor_35625_object22084/head//22 by client.940330.0:22085 2014-08-11 16:31:04.888418 2014-08-13 18:06:06.462859 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 3f6c51b8/benchmark_data_storage0111-ib.pocstor_35625_object23682/head//22 by client.940330.0:23683 2014-08-11 16:31:10.979217 2014-08-13 18:06:06.462874 7f5c973a57a0 20 read_log 192888'435 (0'0) modify b3c819b8/benchmark_data_storage0111-ib.pocstor_35625_object25353/head//22 by client.940330.0:25354 2014-08-11 16:31:17.267199 2014-08-13 18:06:06.462887 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 737ab9b8/benchmark_data_storage0111-ib.pocstor_35625_object27066/head//22 by client.940330.0:27067 2014-08-11 16:31:24.059526 2014-08-13 18:06:06.462901 7f5c973a57a0 20 read_log 192888'437 (192888'426) delete b76ac1b8/benchmark_data_storage0111-ib.pocstor_35625_object1026/head//22 by client.940330.0:32334 2014-08-11 16:31:41.755100 2014-08-13 18:06:06.462914 7f5c973a57a0 20 read_log 192888'438 (192888'427) delete 809961b8/benchmark_data_storage0111-ib.pocstor_35625_object1642/head//22 by client.940330.0:32950 2014-08-11 16:31:41.900893 2014-08-13 18:06:06.462928 7f5c973a57a0 20 read_log 192888'439 (192888'428) delete aba971b8/benchmark_data_storage0111-ib.pocstor_35625_object12080/head//22 by client.940330.0:43388 2014-08-11 16:31:44.678392 2014-08-13 18:06:06.462942 7f5c973a57a0 20 read_log 192888'440 (192888'429) delete 6f0bb1b8/benchmark_data_storage0111-ib.pocstor_35625_object13391/head//22 by client.940330.0:44699 2014-08-11 16:31:45.028674 2014-08-13 18:06:06.462957 7f5c973a57a0 20 read_log 192888'441 (192888'430) delete ac4411b8/benchmark_data_storage0111-ib.pocstor_35625_object19519/head//22 by client.940330.0:50827 2014-08-11 16:31:46.725415 2014-08-13 18:06:06.462969 7f5c973a57a0 20 read_log 192888'442 (192888'431) delete 2b7e99b8/benchmark_data_storage0111-ib.pocstor_35625_object20876/head//22 by client.940330.0:52184 2014-08-11 16:31:47.127125 2014-08-13 18:06:06.462983 7f5c973a57a0 20 read_log 192888'443 (192888'432) delete ad29b8/benchmark_data_storage0111-ib.pocstor_35625_object21963/head//22 by client.940330.0:53271 2014-08-11 16:31:47.434894 2014-08-13 18:06:06.462996 7f5c973a57a0 20 read_log 192888'444 (192888'433) delete 354e31b8/benchmark_data_storage0111-ib.pocstor_35625_object22084/head//22 by client.940330.0:53392 2014-08-11 16:31:47.467808 2014-08-13 18:06:06.463010 7f5c973a57a0 20 read_log 192888'445 (192888'434) delete 3f6c51b8/benchmark_data_storage0111-ib.pocstor_35625_object23682/head//22 by client.940330.0:54990 2014-08-11 16:31:47.945650 2014-08-13 18:06:06.463023 7f5c973a57a0 20 read_log 192888'446 (192888'435) delete b3c819b8/benchmark_data_storage0111-ib.pocstor_35625_object25353/head//22 by client.940330.0:56661 2014-08-11 16:31:48.455759 2014-08-13 18:06:06.463037 7f5c973a57a0 20 read_log 192888'447 (192888'436) delete 737ab9b8/benchmark_data_storage0111-ib.pocstor_35625_object27066/head//22 by client.940330.0:58374 2014-08-11 16:31:48.944948 2014-08-13 18:06:06.463049 7f5c973a57a0 20 read_log 194215'448 (0'0) modify f30ba1b8/benchmark_data_storage0111-ib.pocstor_64498_object5394/head//22 by client.1049635.0:5395 2014-08-12 17:17:34.146143 2014-08-13 18:06:06.463063 7f5c973a57a0 20 read_log 194215'449 (0'0) modify 271989b8/benchmark_data_storage0111-ib.pocstor_64498_object6054/head//22 by client.1049635.0:6055 2014-08-12 17:17:36.235998 2014-08-13 18:06:06.463075 7f5c973a57a0 20 read_log 194215'450 (0'0) modify 4d3b79b8/benchmark_data_storage0111-ib.pocstor_64498_object9577/head//22 by client.1049635.0:9578 2014-08-12 17:17:46.723113 2014-08-13 18:06:06.463088 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 3a2449b8/benchmark_data_storage0111-ib.pocstor_64498_object9678/head//22 by client.1049635.0:9679 2014-08-12 17:17:47.008518 2014-08-13 18:06:06.463127 7f5c973a57a0 20 read_log 194215'452 (0'0) modify 1a5079b8/benchmark_data_storage0111-ib.pocstor_64498_object9933/head//22 by client.1049635.0:9934 2014-08-12 17:17:47.733902 2014-08-13 18:06:06.463143 7f5c973a57a0 20 read_log 194215'453 (0'0) modify 33d451b8/benchmark_data_storage0111-ib.pocstor_64498_object13096/head//22 by client.1049635.0:13097 2014-08-12 17:17:57.666365 2014-08-13 18:06:06.463167 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 6882e1b8/benchmark_data_storage0111-ib.pocstor_64498_object14280/head//22 by client.1049635.0:14281 2014-08-12 17:18:01.445569 2014-08-13 18:06:06.463180 7f5c973a57a0 20 read_log 194215'455 (0'0) modify 1b25c1b8/benchmark_data_storage0111-ib.pocstor_64498_object14315/head//22 by client.1049635.0:14316 2014-08-12 17:18:01.599549 2014-08-13 18:06:06.463193 7f5c973a57a0 20 read_log 194215'456 (0'0) modify c20819b8/benchmark_data_storage0111-ib.pocstor_64498_object16050/head//22 by client.1049635.0:16051 2014-08-12 17:18:07.738220 2014-08-13 18:06:06.463207 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 1957a9b8/benchmark_data_storage0111-ib.pocstor_64498_object17244/head//22 by client.1049635.0:17245 2014-08-12 17:18:12.297996 2014-08-13 18:06:06.463224 7f5c973a57a0 20 read_log 194215'458 (0'0) modify f0e649b8/benchmark_data_storage0111-ib.pocstor_64498_object19542/head//22 by client.1049635.0:19543 2014-08-12 17:18:21.065537 2014-08-13 18:06:06.463239 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 5ce6e1b8/benchmark_data_storage0111-ib.pocstor_64498_object20366/head//22 by client.1049635.0:20367 2014-08-12 17:18:24.169038 2014-08-13 18:06:06.463256 7f5c973a57a0 20 read_log 194215'460 (0'0) modify aad9b9b8/benchmark_data_storage0111-ib.pocstor_64498_object21736/head//22 by client.1049635.0:21737 2014-08-12 17:18:29.339374 2014-08-13 18:06:06.463269 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 4f2061b8/benchmark_data_storage0111-ib.pocstor_64498_object25898/head//22 by client.1049635.0:25899 2014-08-12 17:18:45.042049 2014-08-13 18:06:06.463281 7f5c973a57a0 20 read_log 194215'462 (0'0) modify e4f079b8/benchmark_data_storage0111-ib.pocstor_64498_object29203/head//22 by client.1049635.0:29204 2014-08-12 17:18:57.348701 2014-08-13 18:06:06.463297 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 9b2ac9b8/benchmark_data_storage0111-ib.pocstor_64498_object29565/head//22 by client.1049635.0:29566 2014-08-12 17:18:58.816754 2014-08-13 18:06:06.463311 7f5c973a57a0 20 read_log 194215'464 (194215'448) delete f30ba1b8/benchmark_data_storage0111-ib.pocstor_64498_object5394/head//22 by client.1049635.0:39597 2014-08-12 17:19:19.735444 2014-08-13 18:06:06.463324 7f5c973a57a0 20 read_log 194215'465 (194215'449) delete 271989b8/benchmark_data_storage0111-ib.pocstor_64498_object6054/head//22 by client.1049635.0:40257 2014-08-12 17:19:19.973959 2014-08-13 18:06:06.463336 7f5c973a57a0 20 read_log 194215'466 (194215'450) delete 4d3b79b8/benchmark_data_storage0111-ib.pocstor_64498_object9577/head//22 by client.1049635.0:43780 2014-08-12 17:19:21.113480 2014-08-13 18:06:06.463349 7f5c973a57a0 20 read_log 194215'467 (194215'451) delete 3a2449b8/benchmark_data_storage0111-ib.pocstor_64498_object9678/head//22 by client.1049635.0:43881 2014-08-12 17:19:21.148094 2014-08-13 18:06:06.463361 7f5c973a57a0 20 read_log 194215'468 (194215'452) delete 1a5079b8/benchmark_data_storage0111-ib.pocstor_64498_object9933/head//22 by client.1049635.0:44136 2014-08-12 17:19:21.238998 2014-08-13 18:06:06.463386 7f5c973a57a0 20 read_log 194215'469 (194215'453) delete 33d451b8/benchmark_data_storage0111-ib.pocstor_64498_object13096/head//22 by client.1049635.0:47299 2014-08-12 17:19:22.253610 2014-08-13 18:06:06.463399 7f5c973a57a0 20 read_log 194215'470 (194215'454) delete 6882e1b8/benchmark_data_storage0111-ib.pocstor_64498_object14280/head//22 by client.1049635.0:48483 2014-08-12 17:19:22.652556 2014-08-13 18:06:06.463411 7f5c973a57a0 20 read_log 194215'471 (194215'455) delete 1b25c1b8/benchmark_data_storage0111-ib.pocstor_64498_object14315/head//22 by client.1049635.0:48518 2014-08-12 17:19:22.664758 2014-08-13 18:06:06.463435 7f5c973a57a0 20 read_log 194215'472 (194215'456) delete c20819b8/benchmark_data_storage0111-ib.pocstor_64498_object16050/head//22 by client.1049635.0:50253 2014-08-12 17:19:23.205316 2014-08-13 18:06:06.463459 7f5c973a57a0 20 read_log 194215'473 (194215'457) delete 1957a9b8/benchmark_data_storage0111-ib.pocstor_64498_object17244/head//22 by client.1049635.0:51447 2014-08-12 17:19:23.582695 2014-08-13 18:06:06.463486 7f5c973a57a0 20 read_log 194215'474 (194215'458) delete f0e649b8/benchmark_data_storage0111-ib.pocstor_64498_object19542/head//22 by client.1049635.0:53745 2014-08-12 17:19:24.364664 2014-08-13 18:06:06.463499 7f5c973a57a0 20 read_log 194215'475 (194215'459) delete 5ce6e1b8/benchmark_data_storage0111-ib.pocstor_64498_object20366/head//22 by client.1049635.0:54569 2014-08-12 17:19:24.656346 2014-08-13 18:06:06.463511 7f5c973a57a0 20 read_log 194215'476 (194215'460) delete aad9b9b8/benchmark_data_storage0111-ib.pocstor_64498_object21736/head//22 by client.1049635.0:55939 2014-08-12 17:19:25.138994 2014-08-13 18:06:06.463528 7f5c973a57a0 20 read_log 194215'477 (194215'461) delete 4f2061b8/benchmark_data_storage0111-ib.pocstor_64498_object25898/head//22 by client.1049635.0:60101 2014-08-12 17:19:26.624223 2014-08-13 18:06:06.463540 7f5c973a57a0 20 read_log 194215'478 (194215'462) delete e4f079b8/benchmark_data_storage0111-ib.pocstor_64498_object29203/head//22 by client.1049635.0:63406 2014-08-12 17:19:27.778813 2014-08-13 18:06:06.463552 7f5c973a57a0 20 read_log 194215'479 (194215'463) delete 9b2ac9b8/benchmark_data_storage0111-ib.pocstor_64498_object29565/head//22 by client.1049635.0:63768 2014-08-12 17:19:27.907566 2014-08-13 18:06:06.463641 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.463671 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.463687 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001694 0 0.000000 2014-08-13 18:06:06.463698 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.463707 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=0 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.463715 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'479], crt=191648'362) 2014-08-13 18:06:06.463725 7f5c973a57a0 10 osd.4 194487 pgid 22.1d2 coll 22.1d2_head 2014-08-13 18:06:06.463897 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.1d2 2014-08-13 18:06:06.463907 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.463932 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1d2(unlocked)] enter Initial 2014-08-13 18:06:06.463943 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1d2(unlocked)] enter NotTrimming 2014-08-13 18:06:06.464093 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.464250 7f5c973a57a0 20 read_log 191648'55 (0'0) modify 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 by client.273158.0:3470 2014-07-30 14:09:03.578797 2014-08-13 18:06:06.464271 7f5c973a57a0 20 read_log 191648'82 (0'0) modify cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 by client.273158.0:4875 2014-07-30 14:09:10.421448 2014-08-13 18:06:06.464283 7f5c973a57a0 20 read_log 191648'119 (0'0) modify 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 by client.273158.0:7271 2014-07-30 14:09:22.157475 2014-08-13 18:06:06.464296 7f5c973a57a0 20 read_log 191648'140 (0'0) modify f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 by client.273158.0:8766 2014-07-30 14:09:30.247316 2014-08-13 18:06:06.464313 7f5c973a57a0 20 read_log 191648'176 (0'0) modify d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 by client.273158.0:11133 2014-07-30 14:09:42.819988 2014-08-13 18:06:06.464326 7f5c973a57a0 20 read_log 191648'183 (0'0) modify c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 by client.273158.0:11613 2014-07-30 14:09:45.258056 2014-08-13 18:06:06.464338 7f5c973a57a0 20 read_log 191648'208 (0'0) modify db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 by client.273158.0:12899 2014-07-30 14:09:51.928276 2014-08-13 18:06:06.464350 7f5c973a57a0 20 read_log 191648'231 (0'0) modify 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 by client.273158.0:14393 2014-07-30 14:09:59.636690 2014-08-13 18:06:06.464362 7f5c973a57a0 20 read_log 191648'246 (0'0) modify f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 by client.273158.0:15322 2014-07-30 14:10:03.958818 2014-08-13 18:06:06.464374 7f5c973a57a0 20 read_log 191648'265 (0'0) modify a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 by client.273158.0:16470 2014-07-30 14:10:10.039712 2014-08-13 18:06:06.464386 7f5c973a57a0 20 read_log 191648'294 (0'0) modify 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 by client.273158.0:18082 2014-07-30 14:10:18.452616 2014-08-13 18:06:06.464398 7f5c973a57a0 20 read_log 191648'297 (0'0) modify 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 by client.273158.0:18255 2014-07-30 14:10:19.335978 2014-08-13 18:06:06.464410 7f5c973a57a0 20 read_log 191648'332 (0'0) modify 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 by client.273158.0:20895 2014-07-30 14:10:36.449402 2014-08-13 18:06:06.464422 7f5c973a57a0 20 read_log 191648'345 (0'0) modify d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 by client.273158.0:21464 2014-07-30 14:10:39.332678 2014-08-13 18:06:06.464434 7f5c973a57a0 20 read_log 191727'377 (0'0) modify a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 by client.275175.0:1832 2014-07-30 14:18:40.346523 2014-08-13 18:06:06.464447 7f5c973a57a0 20 read_log 191727'378 (0'0) modify e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 by client.275175.0:1837 2014-07-30 14:18:40.364150 2014-08-13 18:06:06.464459 7f5c973a57a0 20 read_log 191727'382 (0'0) modify f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 by client.275175.0:5230 2014-07-30 14:18:51.028490 2014-08-13 18:06:06.464471 7f5c973a57a0 20 read_log 191727'383 (0'0) modify b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 by client.275175.0:5426 2014-07-30 14:18:51.599515 2014-08-13 18:06:06.464483 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 by client.275175.0:7723 2014-07-30 14:18:58.754197 2014-08-13 18:06:06.464503 7f5c973a57a0 20 read_log 191727'390 (0'0) modify 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 by client.275175.0:8994 2014-07-30 14:19:02.329439 2014-08-13 18:06:06.464518 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 by client.275175.0:9727 2014-07-30 14:19:04.417396 2014-08-13 18:06:06.464531 7f5c973a57a0 20 read_log 191727'401 (0'0) modify 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 by client.275175.0:15557 2014-07-30 14:19:22.575913 2014-08-13 18:06:06.464543 7f5c973a57a0 20 read_log 191727'415 (0'0) modify ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 by client.275175.0:24055 2014-07-30 14:19:48.986063 2014-08-13 18:06:06.464556 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 by client.275175.0:25855 2014-07-30 14:19:54.305346 2014-08-13 18:06:06.464568 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 by client.275175.0:26718 2014-07-30 14:19:56.966150 2014-08-13 18:06:06.464580 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 by client.275175.0:31402 2014-07-30 14:20:11.118127 2014-08-13 18:06:06.464592 7f5c973a57a0 20 read_log 191727'435 (0'0) modify 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 by client.275175.0:33452 2014-07-30 14:20:17.262020 2014-08-13 18:06:06.464604 7f5c973a57a0 20 read_log 191727'445 (0'0) modify 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 by client.275175.0:37148 2014-07-30 14:20:28.785511 2014-08-13 18:06:06.464617 7f5c973a57a0 20 read_log 192888'447 (0'0) modify fcbd01d2/benchmark_data_storage0111-ib.pocstor_35625_object497/head//22 by client.940330.0:498 2014-08-11 16:29:42.712906 2014-08-13 18:06:06.464629 7f5c973a57a0 20 read_log 192888'448 (0'0) modify e381d1d2/benchmark_data_storage0111-ib.pocstor_35625_object1349/head//22 by client.940330.0:1350 2014-08-11 16:29:46.094608 2014-08-13 18:06:06.464641 7f5c973a57a0 20 read_log 192888'449 (0'0) modify bf70f9d2/benchmark_data_storage0111-ib.pocstor_35625_object1562/head//22 by client.940330.0:1563 2014-08-11 16:29:46.935283 2014-08-13 18:06:06.464653 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 24da71d2/benchmark_data_storage0111-ib.pocstor_35625_object2329/head//22 by client.940330.0:2330 2014-08-11 16:29:49.744395 2014-08-13 18:06:06.464665 7f5c973a57a0 20 read_log 192888'451 (0'0) modify eeb259d2/benchmark_data_storage0111-ib.pocstor_35625_object4607/head//22 by client.940330.0:4608 2014-08-11 16:29:58.646409 2014-08-13 18:06:06.464677 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 91c09d2/benchmark_data_storage0111-ib.pocstor_35625_object7055/head//22 by client.940330.0:7056 2014-08-11 16:30:07.512600 2014-08-13 18:06:06.464690 7f5c973a57a0 20 read_log 192888'453 (0'0) modify d07a49d2/benchmark_data_storage0111-ib.pocstor_35625_object11324/head//22 by client.940330.0:11325 2014-08-11 16:30:23.111637 2014-08-13 18:06:06.464702 7f5c973a57a0 20 read_log 192888'454 (0'0) modify f8b119d2/benchmark_data_storage0111-ib.pocstor_35625_object12245/head//22 by client.940330.0:12246 2014-08-11 16:30:26.929677 2014-08-13 18:06:06.464715 7f5c973a57a0 20 read_log 192888'455 (0'0) modify f02ea9d2/benchmark_data_storage0111-ib.pocstor_35625_object12699/head//22 by client.940330.0:12700 2014-08-11 16:30:28.635629 2014-08-13 18:06:06.464730 7f5c973a57a0 20 read_log 192888'456 (0'0) modify b8ea29d2/benchmark_data_storage0111-ib.pocstor_35625_object13182/head//22 by client.940330.0:13183 2014-08-11 16:30:30.580365 2014-08-13 18:06:06.464750 7f5c973a57a0 20 read_log 192888'457 (0'0) modify cef5b1d2/benchmark_data_storage0111-ib.pocstor_35625_object14545/head//22 by client.940330.0:14546 2014-08-11 16:30:35.798230 2014-08-13 18:06:06.464764 7f5c973a57a0 20 read_log 192888'458 (0'0) modify aa71c1d2/benchmark_data_storage0111-ib.pocstor_35625_object14881/head//22 by client.940330.0:14882 2014-08-11 16:30:37.049651 2014-08-13 18:06:06.464777 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 9caf81d2/benchmark_data_storage0111-ib.pocstor_35625_object21727/head//22 by client.940330.0:21728 2014-08-11 16:31:03.511077 2014-08-13 18:06:06.464791 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 803b51d2/benchmark_data_storage0111-ib.pocstor_35625_object25016/head//22 by client.940330.0:25017 2014-08-11 16:31:15.945599 2014-08-13 18:06:06.464803 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 97c009d2/benchmark_data_storage0111-ib.pocstor_35625_object25699/head//22 by client.940330.0:25700 2014-08-11 16:31:18.609937 2014-08-13 18:06:06.464815 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 59c19d2/benchmark_data_storage0111-ib.pocstor_35625_object25956/head//22 by client.940330.0:25957 2014-08-11 16:31:19.621437 2014-08-13 18:06:06.464827 7f5c973a57a0 20 read_log 192888'463 (0'0) modify dc3401d2/benchmark_data_storage0111-ib.pocstor_35625_object26482/head//22 by client.940330.0:26483 2014-08-11 16:31:21.715035 2014-08-13 18:06:06.464841 7f5c973a57a0 20 read_log 192888'464 (0'0) modify c2faa1d2/benchmark_data_storage0111-ib.pocstor_35625_object27601/head//22 by client.940330.0:27602 2014-08-11 16:31:26.136550 2014-08-13 18:06:06.464853 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 3f0d59d2/benchmark_data_storage0111-ib.pocstor_35625_object30310/head//22 by client.940330.0:30311 2014-08-11 16:31:36.633627 2014-08-13 18:06:06.464867 7f5c973a57a0 20 read_log 192888'466 (192888'447) delete fcbd01d2/benchmark_data_storage0111-ib.pocstor_35625_object497/head//22 by client.940330.0:31805 2014-08-11 16:31:41.620514 2014-08-13 18:06:06.464880 7f5c973a57a0 20 read_log 192888'467 (192888'448) delete e381d1d2/benchmark_data_storage0111-ib.pocstor_35625_object1349/head//22 by client.940330.0:32657 2014-08-11 16:31:41.832125 2014-08-13 18:06:06.464894 7f5c973a57a0 20 read_log 192888'468 (192888'449) delete bf70f9d2/benchmark_data_storage0111-ib.pocstor_35625_object1562/head//22 by client.940330.0:32870 2014-08-11 16:31:41.882208 2014-08-13 18:06:06.464907 7f5c973a57a0 20 read_log 192888'469 (192888'450) delete 24da71d2/benchmark_data_storage0111-ib.pocstor_35625_object2329/head//22 by client.940330.0:33637 2014-08-11 16:31:42.076153 2014-08-13 18:06:06.464921 7f5c973a57a0 20 read_log 192888'470 (192888'451) delete eeb259d2/benchmark_data_storage0111-ib.pocstor_35625_object4607/head//22 by client.940330.0:35915 2014-08-11 16:31:42.710660 2014-08-13 18:06:06.464934 7f5c973a57a0 20 read_log 192888'471 (192888'452) delete 91c09d2/benchmark_data_storage0111-ib.pocstor_35625_object7055/head//22 by client.940330.0:38363 2014-08-11 16:31:43.349072 2014-08-13 18:06:06.464948 7f5c973a57a0 20 read_log 192888'472 (192888'453) delete d07a49d2/benchmark_data_storage0111-ib.pocstor_35625_object11324/head//22 by client.940330.0:42632 2014-08-11 16:31:44.493637 2014-08-13 18:06:06.464964 7f5c973a57a0 20 read_log 192888'473 (192888'454) delete f8b119d2/benchmark_data_storage0111-ib.pocstor_35625_object12245/head//22 by client.940330.0:43553 2014-08-11 16:31:44.718601 2014-08-13 18:06:06.464981 7f5c973a57a0 20 read_log 192888'474 (192888'455) delete f02ea9d2/benchmark_data_storage0111-ib.pocstor_35625_object12699/head//22 by client.940330.0:44007 2014-08-11 16:31:44.835154 2014-08-13 18:06:06.464995 7f5c973a57a0 20 read_log 192888'475 (192888'456) delete b8ea29d2/benchmark_data_storage0111-ib.pocstor_35625_object13182/head//22 by client.940330.0:44490 2014-08-11 16:31:44.969645 2014-08-13 18:06:06.465008 7f5c973a57a0 20 read_log 192888'476 (192888'457) delete cef5b1d2/benchmark_data_storage0111-ib.pocstor_35625_object14545/head//22 by client.940330.0:45853 2014-08-11 16:31:45.329587 2014-08-13 18:06:06.465029 7f5c973a57a0 20 read_log 192888'477 (192888'458) delete aa71c1d2/benchmark_data_storage0111-ib.pocstor_35625_object14881/head//22 by client.940330.0:46189 2014-08-11 16:31:45.423893 2014-08-13 18:06:06.465044 7f5c973a57a0 20 read_log 192888'478 (192888'459) delete 9caf81d2/benchmark_data_storage0111-ib.pocstor_35625_object21727/head//22 by client.940330.0:53035 2014-08-11 16:31:47.360711 2014-08-13 18:06:06.465083 7f5c973a57a0 20 read_log 192888'479 (192888'460) delete 803b51d2/benchmark_data_storage0111-ib.pocstor_35625_object25016/head//22 by client.940330.0:56324 2014-08-11 16:31:48.345550 2014-08-13 18:06:06.465096 7f5c973a57a0 20 read_log 192888'480 (192888'461) delete 97c009d2/benchmark_data_storage0111-ib.pocstor_35625_object25699/head//22 by client.940330.0:57007 2014-08-11 16:31:48.559287 2014-08-13 18:06:06.465109 7f5c973a57a0 20 read_log 192888'481 (192888'462) delete 59c19d2/benchmark_data_storage0111-ib.pocstor_35625_object25956/head//22 by client.940330.0:57264 2014-08-11 16:31:48.635211 2014-08-13 18:06:06.465123 7f5c973a57a0 20 read_log 192888'482 (192888'463) delete dc3401d2/benchmark_data_storage0111-ib.pocstor_35625_object26482/head//22 by client.940330.0:57790 2014-08-11 16:31:48.786918 2014-08-13 18:06:06.465158 7f5c973a57a0 20 read_log 192888'483 (192888'464) delete c2faa1d2/benchmark_data_storage0111-ib.pocstor_35625_object27601/head//22 by client.940330.0:58909 2014-08-11 16:31:49.094411 2014-08-13 18:06:06.465174 7f5c973a57a0 20 read_log 192888'484 (192888'465) delete 3f0d59d2/benchmark_data_storage0111-ib.pocstor_35625_object30310/head//22 by client.940330.0:61618 2014-08-11 16:31:49.847240 2014-08-13 18:06:06.465187 7f5c973a57a0 20 read_log 194215'485 (0'0) modify f95681d2/benchmark_data_storage0111-ib.pocstor_64498_object2601/head//22 by client.1049635.0:2602 2014-08-12 17:17:25.540184 2014-08-13 18:06:06.465199 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 9f8571d2/benchmark_data_storage0111-ib.pocstor_64498_object6629/head//22 by client.1049635.0:6630 2014-08-12 17:17:37.943883 2014-08-13 18:06:06.465211 7f5c973a57a0 20 read_log 194215'487 (0'0) modify cd39a1d2/benchmark_data_storage0111-ib.pocstor_64498_object9295/head//22 by client.1049635.0:9296 2014-08-12 17:17:45.876471 2014-08-13 18:06:06.465225 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 90e6f1d2/benchmark_data_storage0111-ib.pocstor_64498_object17081/head//22 by client.1049635.0:17082 2014-08-12 17:18:11.671638 2014-08-13 18:06:06.465238 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 49eb19d2/benchmark_data_storage0111-ib.pocstor_64498_object19560/head//22 by client.1049635.0:19561 2014-08-12 17:18:21.131607 2014-08-13 18:06:06.465255 7f5c973a57a0 20 read_log 194215'490 (0'0) modify da00a1d2/benchmark_data_storage0111-ib.pocstor_64498_object20336/head//22 by client.1049635.0:20337 2014-08-12 17:18:24.057806 2014-08-13 18:06:06.465270 7f5c973a57a0 20 read_log 194215'491 (0'0) modify abb989d2/benchmark_data_storage0111-ib.pocstor_64498_object20369/head//22 by client.1049635.0:20370 2014-08-12 17:18:24.179605 2014-08-13 18:06:06.465283 7f5c973a57a0 20 read_log 194215'492 (0'0) modify 4edaf1d2/benchmark_data_storage0111-ib.pocstor_64498_object23643/head//22 by client.1049635.0:23644 2014-08-12 17:18:36.527064 2014-08-13 18:06:06.465295 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 5be11d2/benchmark_data_storage0111-ib.pocstor_64498_object25489/head//22 by client.1049635.0:25490 2014-08-12 17:18:43.523704 2014-08-13 18:06:06.465310 7f5c973a57a0 20 read_log 194215'494 (0'0) modify f7139d2/benchmark_data_storage0111-ib.pocstor_64498_object29946/head//22 by client.1049635.0:29947 2014-08-12 17:19:00.245924 2014-08-13 18:06:06.465325 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 5d4e11d2/benchmark_data_storage0111-ib.pocstor_64498_object31153/head//22 by client.1049635.0:31154 2014-08-12 17:19:05.155529 2014-08-13 18:06:06.465339 7f5c973a57a0 20 read_log 194215'496 (0'0) modify df0179d2/benchmark_data_storage0111-ib.pocstor_64498_object33542/head//22 by client.1049635.0:33543 2014-08-12 17:19:14.442093 2014-08-13 18:06:06.465355 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 13441d2/benchmark_data_storage0111-ib.pocstor_64498_object33985/head//22 by client.1049635.0:33986 2014-08-12 17:19:16.123634 2014-08-13 18:06:06.465370 7f5c973a57a0 20 read_log 194215'498 (194215'485) delete f95681d2/benchmark_data_storage0111-ib.pocstor_64498_object2601/head//22 by client.1049635.0:36804 2014-08-12 17:19:18.831807 2014-08-13 18:06:06.465383 7f5c973a57a0 20 read_log 194215'499 (194215'486) delete 9f8571d2/benchmark_data_storage0111-ib.pocstor_64498_object6629/head//22 by client.1049635.0:40832 2014-08-12 17:19:20.158876 2014-08-13 18:06:06.465408 7f5c973a57a0 20 read_log 194215'500 (194215'487) delete cd39a1d2/benchmark_data_storage0111-ib.pocstor_64498_object9295/head//22 by client.1049635.0:43498 2014-08-12 17:19:21.023468 2014-08-13 18:06:06.465425 7f5c973a57a0 20 read_log 194215'501 (194215'488) delete 90e6f1d2/benchmark_data_storage0111-ib.pocstor_64498_object17081/head//22 by client.1049635.0:51284 2014-08-12 17:19:23.533305 2014-08-13 18:06:06.465438 7f5c973a57a0 20 read_log 194215'502 (194215'489) delete 49eb19d2/benchmark_data_storage0111-ib.pocstor_64498_object19560/head//22 by client.1049635.0:53763 2014-08-12 17:19:24.371102 2014-08-13 18:06:06.465458 7f5c973a57a0 20 read_log 194215'503 (194215'490) delete da00a1d2/benchmark_data_storage0111-ib.pocstor_64498_object20336/head//22 by client.1049635.0:54539 2014-08-12 17:19:24.643201 2014-08-13 18:06:06.465474 7f5c973a57a0 20 read_log 194215'504 (194215'491) delete abb989d2/benchmark_data_storage0111-ib.pocstor_64498_object20369/head//22 by client.1049635.0:54572 2014-08-12 17:19:24.657249 2014-08-13 18:06:06.465486 7f5c973a57a0 20 read_log 194215'505 (194215'492) delete 4edaf1d2/benchmark_data_storage0111-ib.pocstor_64498_object23643/head//22 by client.1049635.0:57846 2014-08-12 17:19:25.846737 2014-08-13 18:06:06.465506 7f5c973a57a0 20 read_log 194215'506 (194215'493) delete 5be11d2/benchmark_data_storage0111-ib.pocstor_64498_object25489/head//22 by client.1049635.0:59692 2014-08-12 17:19:26.480900 2014-08-13 18:06:06.465521 7f5c973a57a0 20 read_log 194215'507 (194215'494) delete f7139d2/benchmark_data_storage0111-ib.pocstor_64498_object29946/head//22 by client.1049635.0:64149 2014-08-12 17:19:28.029913 2014-08-13 18:06:06.465534 7f5c973a57a0 20 read_log 194215'508 (194215'495) delete 5d4e11d2/benchmark_data_storage0111-ib.pocstor_64498_object31153/head//22 by client.1049635.0:65356 2014-08-12 17:19:28.432108 2014-08-13 18:06:06.465553 7f5c973a57a0 20 read_log 194215'509 (194215'496) delete df0179d2/benchmark_data_storage0111-ib.pocstor_64498_object33542/head//22 by client.1049635.0:67745 2014-08-12 17:19:29.308447 2014-08-13 18:06:06.465568 7f5c973a57a0 20 read_log 194215'510 (194215'497) delete 13441d2/benchmark_data_storage0111-ib.pocstor_64498_object33985/head//22 by client.1049635.0:68188 2014-08-12 17:19:29.475150 2014-08-13 18:06:06.465664 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.465683 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=0 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.465698 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=0 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001766 0 0.000000 2014-08-13 18:06:06.465710 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=0 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.465719 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=0 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.465728 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'510], crt=191648'372) 2014-08-13 18:06:06.465738 7f5c973a57a0 10 osd.4 194487 pgid 22.1f1 coll 22.1f1_head 2014-08-13 18:06:06.465887 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.1f1 2014-08-13 18:06:06.465910 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.465949 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1f1(unlocked)] enter Initial 2014-08-13 18:06:06.465961 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1f1(unlocked)] enter NotTrimming 2014-08-13 18:06:06.466111 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.466278 7f5c973a57a0 20 read_log 191648'8 (0'0) modify 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 by client.273158.0:441 2014-07-30 14:08:50.431491 2014-08-13 18:06:06.466299 7f5c973a57a0 20 read_log 191648'10 (0'0) modify f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 by client.273158.0:593 2014-07-30 14:08:50.988411 2014-08-13 18:06:06.466322 7f5c973a57a0 20 read_log 191648'16 (0'0) modify c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 by client.273158.0:1144 2014-07-30 14:08:53.004008 2014-08-13 18:06:06.466339 7f5c973a57a0 20 read_log 191648'25 (0'0) modify 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 by client.273158.0:1583 2014-07-30 14:08:54.598458 2014-08-13 18:06:06.466352 7f5c973a57a0 20 read_log 191648'64 (0'0) modify 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 by client.273158.0:4386 2014-07-30 14:09:08.293062 2014-08-13 18:06:06.466364 7f5c973a57a0 20 read_log 191648'68 (0'0) modify a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 by client.273158.0:4725 2014-07-30 14:09:09.714701 2014-08-13 18:06:06.466376 7f5c973a57a0 20 read_log 191648'80 (0'0) modify 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 by client.273158.0:5479 2014-07-30 14:09:13.350131 2014-08-13 18:06:06.466389 7f5c973a57a0 20 read_log 191648'109 (0'0) modify a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 by client.273158.0:7623 2014-07-30 14:09:23.969602 2014-08-13 18:06:06.466401 7f5c973a57a0 20 read_log 191648'127 (0'0) modify d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 by client.273158.0:8471 2014-07-30 14:09:28.734832 2014-08-13 18:06:06.466413 7f5c973a57a0 20 read_log 191648'167 (0'0) modify bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 by client.273158.0:10846 2014-07-30 14:09:40.610335 2014-08-13 18:06:06.466425 7f5c973a57a0 20 read_log 191648'199 (0'0) modify d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 by client.273158.0:13343 2014-07-30 14:09:53.998432 2014-08-13 18:06:06.466438 7f5c973a57a0 20 read_log 191648'200 (0'0) modify 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 by client.273158.0:13425 2014-07-30 14:09:54.405244 2014-08-13 18:06:06.466471 7f5c973a57a0 20 read_log 191648'206 (0'0) modify c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 by client.273158.0:13680 2014-07-30 14:09:55.478228 2014-08-13 18:06:06.466483 7f5c973a57a0 20 read_log 191648'220 (0'0) modify 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 by client.273158.0:14805 2014-07-30 14:10:01.920444 2014-08-13 18:06:06.466496 7f5c973a57a0 20 read_log 191648'345 (0'0) modify 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 by client.273158.0:22731 2014-07-30 14:10:47.990653 2014-08-13 18:06:06.466508 7f5c973a57a0 20 read_log 191727'353 (0'0) modify 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 by client.275175.0:5300 2014-07-30 14:18:51.233571 2014-08-13 18:06:06.466520 7f5c973a57a0 20 read_log 191727'354 (0'0) modify 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 by client.275175.0:5871 2014-07-30 14:18:52.876107 2014-08-13 18:06:06.466543 7f5c973a57a0 20 read_log 191727'355 (0'0) modify ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 by client.275175.0:6315 2014-07-30 14:18:54.193940 2014-08-13 18:06:06.466559 7f5c973a57a0 20 read_log 191727'365 (0'0) modify 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 by client.275175.0:10384 2014-07-30 14:19:06.306709 2014-08-13 18:06:06.466571 7f5c973a57a0 20 read_log 191727'366 (0'0) modify d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 by client.275175.0:11576 2014-07-30 14:19:09.978218 2014-08-13 18:06:06.466583 7f5c973a57a0 20 read_log 191727'371 (0'0) modify a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 by client.275175.0:13816 2014-07-30 14:19:17.131022 2014-08-13 18:06:06.466595 7f5c973a57a0 20 read_log 191727'372 (0'0) modify e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 by client.275175.0:14725 2014-07-30 14:19:19.920346 2014-08-13 18:06:06.466607 7f5c973a57a0 20 read_log 191727'378 (0'0) modify e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 by client.275175.0:16928 2014-07-30 14:19:26.954126 2014-08-13 18:06:06.466620 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 by client.275175.0:18003 2014-07-30 14:19:29.964849 2014-08-13 18:06:06.466632 7f5c973a57a0 20 read_log 191727'388 (0'0) modify b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 by client.275175.0:19821 2014-07-30 14:19:35.664807 2014-08-13 18:06:06.466644 7f5c973a57a0 20 read_log 191727'389 (0'0) modify cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 by client.275175.0:20074 2014-07-30 14:19:36.461904 2014-08-13 18:06:06.466656 7f5c973a57a0 20 read_log 191727'395 (0'0) modify c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 by client.275175.0:24931 2014-07-30 14:19:51.440949 2014-08-13 18:06:06.466668 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 by client.275175.0:26005 2014-07-30 14:19:54.735969 2014-08-13 18:06:06.466680 7f5c973a57a0 20 read_log 191727'404 (0'0) modify f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 by client.275175.0:30474 2014-07-30 14:20:08.503529 2014-08-13 18:06:06.466692 7f5c973a57a0 20 read_log 191727'406 (0'0) modify e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 by client.275175.0:31180 2014-07-30 14:20:10.498636 2014-08-13 18:06:06.466704 7f5c973a57a0 20 read_log 191727'411 (0'0) modify ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 by client.275175.0:32722 2014-07-30 14:20:14.949205 2014-08-13 18:06:06.466716 7f5c973a57a0 20 read_log 191727'414 (0'0) modify d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 by client.275175.0:33477 2014-07-30 14:20:17.384036 2014-08-13 18:06:06.466728 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 by client.275175.0:33865 2014-07-30 14:20:18.602629 2014-08-13 18:06:06.466740 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 by client.275175.0:34627 2014-07-30 14:20:20.909357 2014-08-13 18:06:06.466752 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 by client.275175.0:35150 2014-07-30 14:20:22.408807 2014-08-13 18:06:06.466764 7f5c973a57a0 20 read_log 192888'422 (0'0) modify 62b871f1/benchmark_data_storage0111-ib.pocstor_35625_object2004/head//22 by client.940330.0:2005 2014-08-11 16:29:48.571298 2014-08-13 18:06:06.466787 7f5c973a57a0 20 read_log 192888'423 (0'0) modify 951251f1/benchmark_data_storage0111-ib.pocstor_35625_object3369/head//22 by client.940330.0:3370 2014-08-11 16:29:53.991930 2014-08-13 18:06:06.466802 7f5c973a57a0 20 read_log 192888'424 (0'0) modify b35761f1/benchmark_data_storage0111-ib.pocstor_35625_object5059/head//22 by client.940330.0:5060 2014-08-11 16:30:00.241162 2014-08-13 18:06:06.466815 7f5c973a57a0 20 read_log 192888'425 (0'0) modify 27eca1f1/benchmark_data_storage0111-ib.pocstor_35625_object5670/head//22 by client.940330.0:5671 2014-08-11 16:30:02.399752 2014-08-13 18:06:06.466827 7f5c973a57a0 20 read_log 192888'426 (0'0) modify 8b8029f1/benchmark_data_storage0111-ib.pocstor_35625_object6113/head//22 by client.940330.0:6114 2014-08-11 16:30:03.985223 2014-08-13 18:06:06.466839 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 5cdf71f1/benchmark_data_storage0111-ib.pocstor_35625_object6531/head//22 by client.940330.0:6532 2014-08-11 16:30:05.557542 2014-08-13 18:06:06.466854 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 689941f1/benchmark_data_storage0111-ib.pocstor_35625_object7206/head//22 by client.940330.0:7207 2014-08-11 16:30:08.047077 2014-08-13 18:06:06.466866 7f5c973a57a0 20 read_log 192888'429 (0'0) modify b5c5d1f1/benchmark_data_storage0111-ib.pocstor_35625_object7842/head//22 by client.940330.0:7843 2014-08-11 16:30:10.280738 2014-08-13 18:06:06.466880 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 57a7b9f1/benchmark_data_storage0111-ib.pocstor_35625_object12142/head//22 by client.940330.0:12143 2014-08-11 16:30:26.542445 2014-08-13 18:06:06.466892 7f5c973a57a0 20 read_log 192888'431 (0'0) modify aa43c1f1/benchmark_data_storage0111-ib.pocstor_35625_object16350/head//22 by client.940330.0:16351 2014-08-11 16:30:42.538811 2014-08-13 18:06:06.466907 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 50f111f1/benchmark_data_storage0111-ib.pocstor_35625_object16386/head//22 by client.940330.0:16387 2014-08-11 16:30:42.671121 2014-08-13 18:06:06.466919 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 7a2541f1/benchmark_data_storage0111-ib.pocstor_35625_object21915/head//22 by client.940330.0:21916 2014-08-11 16:31:04.254575 2014-08-13 18:06:06.466933 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 601a51f1/benchmark_data_storage0111-ib.pocstor_35625_object23749/head//22 by client.940330.0:23750 2014-08-11 16:31:11.228612 2014-08-13 18:06:06.466945 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 45bc29f1/benchmark_data_storage0111-ib.pocstor_35625_object25493/head//22 by client.940330.0:25494 2014-08-11 16:31:17.812384 2014-08-13 18:06:06.466959 7f5c973a57a0 20 read_log 192888'436 (0'0) modify dcbec1f1/benchmark_data_storage0111-ib.pocstor_35625_object27340/head//22 by client.940330.0:27341 2014-08-11 16:31:25.118134 2014-08-13 18:06:06.466972 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 292b59f1/benchmark_data_storage0111-ib.pocstor_35625_object29457/head//22 by client.940330.0:29458 2014-08-11 16:31:33.404355 2014-08-13 18:06:06.466985 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 1bb829f1/benchmark_data_storage0111-ib.pocstor_35625_object30141/head//22 by client.940330.0:30142 2014-08-11 16:31:35.988255 2014-08-13 18:06:06.466998 7f5c973a57a0 20 read_log 192888'439 (192888'422) delete 62b871f1/benchmark_data_storage0111-ib.pocstor_35625_object2004/head//22 by client.940330.0:33312 2014-08-11 16:31:41.992955 2014-08-13 18:06:06.467013 7f5c973a57a0 20 read_log 192888'440 (192888'423) delete 951251f1/benchmark_data_storage0111-ib.pocstor_35625_object3369/head//22 by client.940330.0:34677 2014-08-11 16:31:42.353131 2014-08-13 18:06:06.467028 7f5c973a57a0 20 read_log 192888'441 (192888'424) delete b35761f1/benchmark_data_storage0111-ib.pocstor_35625_object5059/head//22 by client.940330.0:36367 2014-08-11 16:31:42.832710 2014-08-13 18:06:06.467042 7f5c973a57a0 20 read_log 192888'442 (192888'425) delete 27eca1f1/benchmark_data_storage0111-ib.pocstor_35625_object5670/head//22 by client.940330.0:36978 2014-08-11 16:31:42.988614 2014-08-13 18:06:06.467064 7f5c973a57a0 20 read_log 192888'443 (192888'426) delete 8b8029f1/benchmark_data_storage0111-ib.pocstor_35625_object6113/head//22 by client.940330.0:37421 2014-08-11 16:31:43.107436 2014-08-13 18:06:06.467102 7f5c973a57a0 20 read_log 192888'444 (192888'427) delete 5cdf71f1/benchmark_data_storage0111-ib.pocstor_35625_object6531/head//22 by client.940330.0:37839 2014-08-11 16:31:43.215123 2014-08-13 18:06:06.467117 7f5c973a57a0 20 read_log 192888'445 (192888'428) delete 689941f1/benchmark_data_storage0111-ib.pocstor_35625_object7206/head//22 by client.940330.0:38514 2014-08-11 16:31:43.386297 2014-08-13 18:06:06.467129 7f5c973a57a0 20 read_log 192888'446 (192888'429) delete b5c5d1f1/benchmark_data_storage0111-ib.pocstor_35625_object7842/head//22 by client.940330.0:39150 2014-08-11 16:31:43.550652 2014-08-13 18:06:06.467142 7f5c973a57a0 20 read_log 192888'447 (192888'430) delete 57a7b9f1/benchmark_data_storage0111-ib.pocstor_35625_object12142/head//22 by client.940330.0:43450 2014-08-11 16:31:44.694172 2014-08-13 18:06:06.467167 7f5c973a57a0 20 read_log 192888'448 (192888'431) delete aa43c1f1/benchmark_data_storage0111-ib.pocstor_35625_object16350/head//22 by client.940330.0:47658 2014-08-11 16:31:45.816008 2014-08-13 18:06:06.467195 7f5c973a57a0 20 read_log 192888'449 (192888'432) delete 50f111f1/benchmark_data_storage0111-ib.pocstor_35625_object16386/head//22 by client.940330.0:47694 2014-08-11 16:31:45.825806 2014-08-13 18:06:06.467210 7f5c973a57a0 20 read_log 192888'450 (192888'433) delete 7a2541f1/benchmark_data_storage0111-ib.pocstor_35625_object21915/head//22 by client.940330.0:53223 2014-08-11 16:31:47.418220 2014-08-13 18:06:06.467223 7f5c973a57a0 20 read_log 192888'451 (192888'434) delete 601a51f1/benchmark_data_storage0111-ib.pocstor_35625_object23749/head//22 by client.940330.0:55057 2014-08-11 16:31:47.964273 2014-08-13 18:06:06.467237 7f5c973a57a0 20 read_log 192888'452 (192888'435) delete 45bc29f1/benchmark_data_storage0111-ib.pocstor_35625_object25493/head//22 by client.940330.0:56801 2014-08-11 16:31:48.497357 2014-08-13 18:06:06.467250 7f5c973a57a0 20 read_log 192888'453 (192888'436) delete dcbec1f1/benchmark_data_storage0111-ib.pocstor_35625_object27340/head//22 by client.940330.0:58648 2014-08-11 16:31:49.020609 2014-08-13 18:06:06.467268 7f5c973a57a0 20 read_log 192888'454 (192888'437) delete 292b59f1/benchmark_data_storage0111-ib.pocstor_35625_object29457/head//22 by client.940330.0:60765 2014-08-11 16:31:49.599119 2014-08-13 18:06:06.467280 7f5c973a57a0 20 read_log 192888'455 (192888'438) delete 1bb829f1/benchmark_data_storage0111-ib.pocstor_35625_object30141/head//22 by client.940330.0:61449 2014-08-11 16:31:49.799778 2014-08-13 18:06:06.467293 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 2e1be1f1/benchmark_data_storage0111-ib.pocstor_64498_object1155/head//22 by client.1049635.0:1156 2014-08-12 17:17:20.919622 2014-08-13 18:06:06.467308 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 6ec2f1f1/benchmark_data_storage0111-ib.pocstor_64498_object2536/head//22 by client.1049635.0:2537 2014-08-12 17:17:25.355718 2014-08-13 18:06:06.467321 7f5c973a57a0 20 read_log 194215'458 (0'0) modify fed131f1/benchmark_data_storage0111-ib.pocstor_64498_object3266/head//22 by client.1049635.0:3267 2014-08-12 17:17:27.722683 2014-08-13 18:06:06.467333 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 89b1d9f1/benchmark_data_storage0111-ib.pocstor_64498_object3515/head//22 by client.1049635.0:3516 2014-08-12 17:17:28.547918 2014-08-13 18:06:06.467347 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 606191f1/benchmark_data_storage0111-ib.pocstor_64498_object3528/head//22 by client.1049635.0:3529 2014-08-12 17:17:28.584991 2014-08-13 18:06:06.467359 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 971491f1/benchmark_data_storage0111-ib.pocstor_64498_object4989/head//22 by client.1049635.0:4990 2014-08-12 17:17:32.897095 2014-08-13 18:06:06.467406 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 8717b9f1/benchmark_data_storage0111-ib.pocstor_64498_object9514/head//22 by client.1049635.0:9515 2014-08-12 17:17:46.534206 2014-08-13 18:06:06.467423 7f5c973a57a0 20 read_log 194215'463 (0'0) modify e95e99f1/benchmark_data_storage0111-ib.pocstor_64498_object9713/head//22 by client.1049635.0:9714 2014-08-12 17:17:47.111547 2014-08-13 18:06:06.467448 7f5c973a57a0 20 read_log 194215'464 (0'0) modify a5b441f1/benchmark_data_storage0111-ib.pocstor_64498_object9854/head//22 by client.1049635.0:9855 2014-08-12 17:17:47.510961 2014-08-13 18:06:06.467461 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 60d2b1f1/benchmark_data_storage0111-ib.pocstor_64498_object9915/head//22 by client.1049635.0:9916 2014-08-12 17:17:47.681505 2014-08-13 18:06:06.467473 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 12f0b1f1/benchmark_data_storage0111-ib.pocstor_64498_object10755/head//22 by client.1049635.0:10756 2014-08-12 17:17:50.398895 2014-08-13 18:06:06.467499 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 6c7771f1/benchmark_data_storage0111-ib.pocstor_64498_object12514/head//22 by client.1049635.0:12515 2014-08-12 17:17:55.935626 2014-08-13 18:06:06.467512 7f5c973a57a0 20 read_log 194215'468 (0'0) modify fed49f1/benchmark_data_storage0111-ib.pocstor_64498_object12637/head//22 by client.1049635.0:12638 2014-08-12 17:17:56.270754 2014-08-13 18:06:06.467530 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 9f789f1/benchmark_data_storage0111-ib.pocstor_64498_object17683/head//22 by client.1049635.0:17684 2014-08-12 17:18:13.956777 2014-08-13 18:06:06.467546 7f5c973a57a0 20 read_log 194215'470 (0'0) modify e1aea1f1/benchmark_data_storage0111-ib.pocstor_64498_object21412/head//22 by client.1049635.0:21413 2014-08-12 17:18:28.125117 2014-08-13 18:06:06.467559 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 680e49f1/benchmark_data_storage0111-ib.pocstor_64498_object22252/head//22 by client.1049635.0:22253 2014-08-12 17:18:31.280620 2014-08-13 18:06:06.467572 7f5c973a57a0 20 read_log 194215'472 (0'0) modify d74ca1f1/benchmark_data_storage0111-ib.pocstor_64498_object25767/head//22 by client.1049635.0:25768 2014-08-12 17:18:44.559114 2014-08-13 18:06:06.467584 7f5c973a57a0 20 read_log 194215'473 (0'0) modify b1ba19f1/benchmark_data_storage0111-ib.pocstor_64498_object27424/head//22 by client.1049635.0:27425 2014-08-12 17:18:50.739556 2014-08-13 18:06:06.467597 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 146f59f1/benchmark_data_storage0111-ib.pocstor_64498_object27765/head//22 by client.1049635.0:27766 2014-08-12 17:18:52.004242 2014-08-13 18:06:06.467609 7f5c973a57a0 20 read_log 194215'475 (194215'456) delete 2e1be1f1/benchmark_data_storage0111-ib.pocstor_64498_object1155/head//22 by client.1049635.0:35358 2014-08-12 17:19:18.369557 2014-08-13 18:06:06.467624 7f5c973a57a0 20 read_log 194215'476 (194215'457) delete 6ec2f1f1/benchmark_data_storage0111-ib.pocstor_64498_object2536/head//22 by client.1049635.0:36739 2014-08-12 17:19:18.812742 2014-08-13 18:06:06.467637 7f5c973a57a0 20 read_log 194215'477 (194215'458) delete fed131f1/benchmark_data_storage0111-ib.pocstor_64498_object3266/head//22 by client.1049635.0:37469 2014-08-12 17:19:19.049289 2014-08-13 18:06:06.467660 7f5c973a57a0 20 read_log 194215'478 (194215'459) delete 89b1d9f1/benchmark_data_storage0111-ib.pocstor_64498_object3515/head//22 by client.1049635.0:37718 2014-08-12 17:19:19.126273 2014-08-13 18:06:06.467674 7f5c973a57a0 20 read_log 194215'479 (194215'460) delete 606191f1/benchmark_data_storage0111-ib.pocstor_64498_object3528/head//22 by client.1049635.0:37731 2014-08-12 17:19:19.130057 2014-08-13 18:06:06.467686 7f5c973a57a0 20 read_log 194215'480 (194215'461) delete 971491f1/benchmark_data_storage0111-ib.pocstor_64498_object4989/head//22 by client.1049635.0:39192 2014-08-12 17:19:19.597482 2014-08-13 18:06:06.467699 7f5c973a57a0 20 read_log 194215'481 (194215'462) delete 8717b9f1/benchmark_data_storage0111-ib.pocstor_64498_object9514/head//22 by client.1049635.0:43717 2014-08-12 17:19:21.094660 2014-08-13 18:06:06.467713 7f5c973a57a0 20 read_log 194215'482 (194215'463) delete e95e99f1/benchmark_data_storage0111-ib.pocstor_64498_object9713/head//22 by client.1049635.0:43916 2014-08-12 17:19:21.158584 2014-08-13 18:06:06.467728 7f5c973a57a0 20 read_log 194215'483 (194215'464) delete a5b441f1/benchmark_data_storage0111-ib.pocstor_64498_object9854/head//22 by client.1049635.0:44057 2014-08-12 17:19:21.211585 2014-08-13 18:06:06.467741 7f5c973a57a0 20 read_log 194215'484 (194215'465) delete 60d2b1f1/benchmark_data_storage0111-ib.pocstor_64498_object9915/head//22 by client.1049635.0:44118 2014-08-12 17:19:21.229546 2014-08-13 18:06:06.467755 7f5c973a57a0 20 read_log 194215'485 (194215'466) delete 12f0b1f1/benchmark_data_storage0111-ib.pocstor_64498_object10755/head//22 by client.1049635.0:44958 2014-08-12 17:19:21.502449 2014-08-13 18:06:06.467767 7f5c973a57a0 20 read_log 194215'486 (194215'467) delete 6c7771f1/benchmark_data_storage0111-ib.pocstor_64498_object12514/head//22 by client.1049635.0:46717 2014-08-12 17:19:22.058385 2014-08-13 18:06:06.467780 7f5c973a57a0 20 read_log 194215'487 (194215'468) delete fed49f1/benchmark_data_storage0111-ib.pocstor_64498_object12637/head//22 by client.1049635.0:46840 2014-08-12 17:19:22.097029 2014-08-13 18:06:06.467792 7f5c973a57a0 20 read_log 194215'488 (194215'469) delete 9f789f1/benchmark_data_storage0111-ib.pocstor_64498_object17683/head//22 by client.1049635.0:51886 2014-08-12 17:19:23.721819 2014-08-13 18:06:06.467804 7f5c973a57a0 20 read_log 194215'489 (194215'470) delete e1aea1f1/benchmark_data_storage0111-ib.pocstor_64498_object21412/head//22 by client.1049635.0:55615 2014-08-12 17:19:25.023712 2014-08-13 18:06:06.467817 7f5c973a57a0 20 read_log 194215'490 (194215'471) delete 680e49f1/benchmark_data_storage0111-ib.pocstor_64498_object22252/head//22 by client.1049635.0:56455 2014-08-12 17:19:25.326738 2014-08-13 18:06:06.467829 7f5c973a57a0 20 read_log 194215'491 (194215'472) delete d74ca1f1/benchmark_data_storage0111-ib.pocstor_64498_object25767/head//22 by client.1049635.0:59970 2014-08-12 17:19:26.580689 2014-08-13 18:06:06.467841 7f5c973a57a0 20 read_log 194215'492 (194215'473) delete b1ba19f1/benchmark_data_storage0111-ib.pocstor_64498_object27424/head//22 by client.1049635.0:61627 2014-08-12 17:19:27.152497 2014-08-13 18:06:06.467853 7f5c973a57a0 20 read_log 194215'493 (194215'474) delete 146f59f1/benchmark_data_storage0111-ib.pocstor_64498_object27765/head//22 by client.1049635.0:61968 2014-08-12 17:19:27.265758 2014-08-13 18:06:06.467948 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.467967 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=0 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.467982 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=0 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002034 0 0.000000 2014-08-13 18:06:06.467994 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=0 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.468003 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=0 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.468012 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'493], crt=191648'346) 2014-08-13 18:06:06.468022 7f5c973a57a0 10 osd.4 194487 pgid 22.1fb coll 22.1fb_head 2014-08-13 18:06:06.468214 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.1fb 2014-08-13 18:06:06.468224 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.468253 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1fb(unlocked)] enter Initial 2014-08-13 18:06:06.468265 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1fb(unlocked)] enter NotTrimming 2014-08-13 18:06:06.468414 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.468552 7f5c973a57a0 20 read_log 191648'30 (0'0) modify 136be9fb/benchmark_data_storage0111-ib.pocstor_40745_object1798/head//22 by client.273158.0:1799 2014-07-30 14:08:55.383993 2014-08-13 18:06:06.468573 7f5c973a57a0 20 read_log 191648'57 (0'0) modify 938109fb/benchmark_data_storage0111-ib.pocstor_40745_object3302/head//22 by client.273158.0:3303 2014-07-30 14:09:02.709759 2014-08-13 18:06:06.468586 7f5c973a57a0 20 read_log 191648'62 (0'0) modify c60759fb/benchmark_data_storage0111-ib.pocstor_40745_object3555/head//22 by client.273158.0:3556 2014-07-30 14:09:03.941073 2014-08-13 18:06:06.468599 7f5c973a57a0 20 read_log 191648'94 (0'0) modify 50ba91fb/benchmark_data_storage0111-ib.pocstor_40745_object5695/head//22 by client.273158.0:5696 2014-07-30 14:09:14.225153 2014-08-13 18:06:06.468611 7f5c973a57a0 20 read_log 191648'109 (0'0) modify c49c01fb/benchmark_data_storage0111-ib.pocstor_40745_object6507/head//22 by client.273158.0:6508 2014-07-30 14:09:18.748658 2014-08-13 18:06:06.468623 7f5c973a57a0 20 read_log 191648'112 (0'0) modify 92a179fb/benchmark_data_storage0111-ib.pocstor_40745_object6668/head//22 by client.273158.0:6669 2014-07-30 14:09:19.628958 2014-08-13 18:06:06.468636 7f5c973a57a0 20 read_log 191648'250 (0'0) modify 45f161fb/benchmark_data_storage0111-ib.pocstor_40745_object16692/head//22 by client.273158.0:16693 2014-07-30 14:10:11.289369 2014-08-13 18:06:06.468648 7f5c973a57a0 20 read_log 191648'304 (0'0) modify 94a299fb/benchmark_data_storage0111-ib.pocstor_40745_object19853/head//22 by client.273158.0:19854 2014-07-30 14:10:30.750352 2014-08-13 18:06:06.468660 7f5c973a57a0 20 read_log 191648'313 (0'0) modify 7aacc9fb/benchmark_data_storage0111-ib.pocstor_40745_object20227/head//22 by client.273158.0:20228 2014-07-30 14:10:32.369551 2014-08-13 18:06:06.468672 7f5c973a57a0 20 read_log 191648'322 (0'0) modify fd17e1fb/benchmark_data_storage0111-ib.pocstor_40745_object20836/head//22 by client.273158.0:20837 2014-07-30 14:10:36.215961 2014-08-13 18:06:06.468684 7f5c973a57a0 20 read_log 191727'351 (0'0) modify f41419fb/benchmark_data_storage0111-ib.pocstor_51285_object81/head//22 by client.275175.0:82 2014-07-30 14:18:33.786266 2014-08-13 18:06:06.468696 7f5c973a57a0 20 read_log 191727'353 (0'0) modify 1fea39fb/benchmark_data_storage0111-ib.pocstor_51285_object1851/head//22 by client.275175.0:1852 2014-07-30 14:18:40.413134 2014-08-13 18:06:06.468708 7f5c973a57a0 20 read_log 191727'354 (0'0) modify 11ca11fb/benchmark_data_storage0111-ib.pocstor_51285_object1954/head//22 by client.275175.0:1955 2014-07-30 14:18:40.749952 2014-08-13 18:06:06.468720 7f5c973a57a0 20 read_log 191727'358 (0'0) modify 9e9599fb/benchmark_data_storage0111-ib.pocstor_51285_object5856/head//22 by client.275175.0:5857 2014-07-30 14:18:52.835925 2014-08-13 18:06:06.468732 7f5c973a57a0 20 read_log 191727'368 (0'0) modify a94181fb/benchmark_data_storage0111-ib.pocstor_51285_object9622/head//22 by client.275175.0:9623 2014-07-30 14:19:04.131710 2014-08-13 18:06:06.468744 7f5c973a57a0 20 read_log 191727'371 (0'0) modify f7a0f1fb/benchmark_data_storage0111-ib.pocstor_51285_object10117/head//22 by client.275175.0:10118 2014-07-30 14:19:05.508789 2014-08-13 18:06:06.468756 7f5c973a57a0 20 read_log 191727'372 (0'0) modify 617e1fb/benchmark_data_storage0111-ib.pocstor_51285_object10998/head//22 by client.275175.0:10999 2014-07-30 14:19:08.039689 2014-08-13 18:06:06.468769 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 94ada1fb/benchmark_data_storage0111-ib.pocstor_51285_object12487/head//22 by client.275175.0:12488 2014-07-30 14:19:13.003252 2014-08-13 18:06:06.468783 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 6e061fb/benchmark_data_storage0111-ib.pocstor_51285_object20938/head//22 by client.275175.0:20939 2014-07-30 14:19:38.955297 2014-08-13 18:06:06.468803 7f5c973a57a0 20 read_log 191727'392 (0'0) modify c82459fb/benchmark_data_storage0111-ib.pocstor_51285_object21217/head//22 by client.275175.0:21218 2014-07-30 14:19:39.755515 2014-08-13 18:06:06.468818 7f5c973a57a0 20 read_log 191727'394 (0'0) modify f676f9fb/benchmark_data_storage0111-ib.pocstor_51285_object21392/head//22 by client.275175.0:21393 2014-07-30 14:19:40.280549 2014-08-13 18:06:06.468830 7f5c973a57a0 20 read_log 191727'396 (0'0) modify d13631fb/benchmark_data_storage0111-ib.pocstor_51285_object24265/head//22 by client.275175.0:24266 2014-07-30 14:19:49.573530 2014-08-13 18:06:06.468843 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 64d59fb/benchmark_data_storage0111-ib.pocstor_51285_object26346/head//22 by client.275175.0:26347 2014-07-30 14:19:55.846545 2014-08-13 18:06:06.468855 7f5c973a57a0 20 read_log 191727'408 (0'0) modify a3ba89fb/benchmark_data_storage0111-ib.pocstor_51285_object28449/head//22 by client.275175.0:28450 2014-07-30 14:20:02.279194 2014-08-13 18:06:06.468867 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 66e8b9fb/benchmark_data_storage0111-ib.pocstor_51285_object31368/head//22 by client.275175.0:31369 2014-07-30 14:20:11.022851 2014-08-13 18:06:06.468879 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 81bb51fb/benchmark_data_storage0111-ib.pocstor_51285_object32061/head//22 by client.275175.0:32062 2014-07-30 14:20:13.024473 2014-08-13 18:06:06.468891 7f5c973a57a0 20 read_log 191727'422 (0'0) modify cec859fb/benchmark_data_storage0111-ib.pocstor_51285_object36315/head//22 by client.275175.0:36316 2014-07-30 14:20:26.236339 2014-08-13 18:06:06.468904 7f5c973a57a0 20 read_log 191727'423 (0'0) modify 5f3b99fb/benchmark_data_storage0111-ib.pocstor_51285_object37312/head//22 by client.275175.0:37313 2014-07-30 14:20:29.242057 2014-08-13 18:06:06.468916 7f5c973a57a0 20 read_log 192666'425 (0'0) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1175 2014-07-30 16:28:47.187955 2014-08-13 18:06:06.468928 7f5c973a57a0 20 read_log 192666'426 (192666'425) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1177 2014-07-30 16:28:47.194955 2014-08-13 18:06:06.468940 7f5c973a57a0 20 read_log 192666'427 (192666'426) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1178 2014-07-30 16:28:47.198955 2014-08-13 18:06:06.468952 7f5c973a57a0 20 read_log 192666'428 (192666'427) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1179 2014-07-30 16:28:47.202954 2014-08-13 18:06:06.468964 7f5c973a57a0 20 read_log 192666'429 (192666'428) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1180 2014-07-30 16:28:47.206954 2014-08-13 18:06:06.468976 7f5c973a57a0 20 read_log 192666'430 (192666'429) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1181 2014-07-30 16:28:47.211954 2014-08-13 18:06:06.468988 7f5c973a57a0 20 read_log 192666'431 (192666'430) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1182 2014-07-30 16:28:47.215954 2014-08-13 18:06:06.469000 7f5c973a57a0 20 read_log 192666'432 (192666'431) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1183 2014-07-30 16:28:47.218954 2014-08-13 18:06:06.469012 7f5c973a57a0 20 read_log 192666'433 (192666'432) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1184 2014-07-30 16:28:47.222954 2014-08-13 18:06:06.469024 7f5c973a57a0 20 read_log 192666'434 (192666'433) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1185 2014-07-30 16:28:47.226954 2014-08-13 18:06:06.469036 7f5c973a57a0 20 read_log 192666'435 (192666'434) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1186 2014-07-30 16:28:47.230953 2014-08-13 18:06:06.469048 7f5c973a57a0 20 read_log 192666'436 (192666'435) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1187 2014-07-30 16:28:47.234953 2014-08-13 18:06:06.469068 7f5c973a57a0 20 read_log 192666'437 (192666'436) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1188 2014-07-30 16:28:47.238953 2014-08-13 18:06:06.469082 7f5c973a57a0 20 read_log 192666'438 (192666'437) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1189 2014-07-30 16:28:47.241953 2014-08-13 18:06:06.469094 7f5c973a57a0 20 read_log 192666'439 (192666'438) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1190 2014-07-30 16:28:47.244953 2014-08-13 18:06:06.469107 7f5c973a57a0 20 read_log 192666'440 (192666'439) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1191 2014-07-30 16:28:47.248953 2014-08-13 18:06:06.469119 7f5c973a57a0 20 read_log 192666'441 (192666'440) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.275486.1:1193 2014-07-30 16:28:47.291951 2014-08-13 18:06:06.469165 7f5c973a57a0 20 read_log 192666'442 (192666'441) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:799 2014-07-30 17:22:40.202000 2014-08-13 18:06:06.469182 7f5c973a57a0 20 read_log 192666'443 (192666'442) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:801 2014-07-30 17:22:40.211000 2014-08-13 18:06:06.469194 7f5c973a57a0 20 read_log 192666'444 (192666'443) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:802 2014-07-30 17:22:40.217000 2014-08-13 18:06:06.469208 7f5c973a57a0 20 read_log 192666'445 (192666'444) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:803 2014-07-30 17:22:40.223000 2014-08-13 18:06:06.469227 7f5c973a57a0 20 read_log 192666'446 (192666'445) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:804 2014-07-30 17:22:40.229000 2014-08-13 18:06:06.469242 7f5c973a57a0 20 read_log 192666'447 (192666'446) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:805 2014-07-30 17:22:40.235000 2014-08-13 18:06:06.469257 7f5c973a57a0 20 read_log 192666'448 (192666'447) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:806 2014-07-30 17:22:40.240000 2014-08-13 18:06:06.469270 7f5c973a57a0 20 read_log 192666'449 (192666'448) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:807 2014-07-30 17:22:40.245000 2014-08-13 18:06:06.469284 7f5c973a57a0 20 read_log 192666'450 (192666'449) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:808 2014-07-30 17:22:40.251000 2014-08-13 18:06:06.469297 7f5c973a57a0 20 read_log 192666'451 (192666'450) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:809 2014-07-30 17:22:40.256000 2014-08-13 18:06:06.469314 7f5c973a57a0 20 read_log 192666'452 (192666'451) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:810 2014-07-30 17:22:40.261000 2014-08-13 18:06:06.469327 7f5c973a57a0 20 read_log 192666'453 (192666'452) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:811 2014-07-30 17:22:40.266000 2014-08-13 18:06:06.469339 7f5c973a57a0 20 read_log 192666'454 (192666'453) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:812 2014-07-30 17:22:40.272000 2014-08-13 18:06:06.469351 7f5c973a57a0 20 read_log 192666'455 (192666'454) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:813 2014-07-30 17:22:40.277000 2014-08-13 18:06:06.469363 7f5c973a57a0 20 read_log 192666'456 (192666'455) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:814 2014-07-30 17:22:40.283000 2014-08-13 18:06:06.469375 7f5c973a57a0 20 read_log 192666'457 (192666'456) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:815 2014-07-30 17:22:40.288000 2014-08-13 18:06:06.469405 7f5c973a57a0 20 read_log 192666'458 (192666'457) modify 981a09fb/rb.0.43b76.238e1f29.000000003200/head//22 by client.276545.1:817 2014-07-30 17:22:40.294000 2014-08-13 18:06:06.469435 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 921031fb/benchmark_data_storage0111-ib.pocstor_35625_object579/head//22 by client.940330.0:580 2014-08-11 16:29:43.040595 2014-08-13 18:06:06.469451 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 18f419fb/benchmark_data_storage0111-ib.pocstor_35625_object4140/head//22 by client.940330.0:4141 2014-08-11 16:29:56.840380 2014-08-13 18:06:06.469463 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 2c26a9fb/benchmark_data_storage0111-ib.pocstor_35625_object6864/head//22 by client.940330.0:6865 2014-08-11 16:30:06.841979 2014-08-13 18:06:06.469476 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 4de21fb/benchmark_data_storage0111-ib.pocstor_35625_object9106/head//22 by client.940330.0:9107 2014-08-11 16:30:14.702129 2014-08-13 18:06:06.469489 7f5c973a57a0 20 read_log 192888'463 (0'0) modify df5061fb/benchmark_data_storage0111-ib.pocstor_35625_object9110/head//22 by client.940330.0:9111 2014-08-11 16:30:14.716489 2014-08-13 18:06:06.469511 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 126a79fb/benchmark_data_storage0111-ib.pocstor_35625_object12346/head//22 by client.940330.0:12347 2014-08-11 16:30:27.305793 2014-08-13 18:06:06.469528 7f5c973a57a0 20 read_log 192888'465 (0'0) modify e9d2d1fb/benchmark_data_storage0111-ib.pocstor_35625_object12374/head//22 by client.940330.0:12375 2014-08-11 16:30:27.412367 2014-08-13 18:06:06.469541 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 15ab9fb/benchmark_data_storage0111-ib.pocstor_35625_object14057/head//22 by client.940330.0:14058 2014-08-11 16:30:33.849231 2014-08-13 18:06:06.469553 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 201e01fb/benchmark_data_storage0111-ib.pocstor_35625_object17661/head//22 by client.940330.0:17662 2014-08-11 16:30:47.449097 2014-08-13 18:06:06.469565 7f5c973a57a0 20 read_log 192888'468 (0'0) modify ec9299fb/benchmark_data_storage0111-ib.pocstor_35625_object18051/head//22 by client.940330.0:18052 2014-08-11 16:30:48.927590 2014-08-13 18:06:06.469577 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 430bd1fb/benchmark_data_storage0111-ib.pocstor_35625_object21376/head//22 by client.940330.0:21377 2014-08-11 16:31:02.225114 2014-08-13 18:06:06.469589 7f5c973a57a0 20 read_log 192888'470 (0'0) modify fa2891fb/benchmark_data_storage0111-ib.pocstor_35625_object22572/head//22 by client.940330.0:22573 2014-08-11 16:31:06.798464 2014-08-13 18:06:06.469601 7f5c973a57a0 20 read_log 192888'471 (0'0) modify 4012c9fb/benchmark_data_storage0111-ib.pocstor_35625_object24261/head//22 by client.940330.0:24262 2014-08-11 16:31:13.100712 2014-08-13 18:06:06.469614 7f5c973a57a0 20 read_log 192888'472 (0'0) modify e49cc9fb/benchmark_data_storage0111-ib.pocstor_35625_object28716/head//22 by client.940330.0:28717 2014-08-11 16:31:30.504701 2014-08-13 18:06:06.469626 7f5c973a57a0 20 read_log 192888'473 (0'0) modify 2143b9fb/benchmark_data_storage0111-ib.pocstor_35625_object28772/head//22 by client.940330.0:28773 2014-08-11 16:31:30.715309 2014-08-13 18:06:06.469638 7f5c973a57a0 20 read_log 192888'474 (0'0) modify f65409fb/benchmark_data_storage0111-ib.pocstor_35625_object28848/head//22 by client.940330.0:28849 2014-08-11 16:31:31.003813 2014-08-13 18:06:06.469657 7f5c973a57a0 20 read_log 192888'475 (0'0) modify 11c731fb/benchmark_data_storage0111-ib.pocstor_35625_object31226/head//22 by client.940330.0:31227 2014-08-11 16:31:40.122664 2014-08-13 18:06:06.469673 7f5c973a57a0 20 read_log 192888'476 (192888'459) delete 921031fb/benchmark_data_storage0111-ib.pocstor_35625_object579/head//22 by client.940330.0:31887 2014-08-11 16:31:41.648094 2014-08-13 18:06:06.469686 7f5c973a57a0 20 read_log 192888'477 (192888'460) delete 18f419fb/benchmark_data_storage0111-ib.pocstor_35625_object4140/head//22 by client.940330.0:35448 2014-08-11 16:31:42.575077 2014-08-13 18:06:06.469699 7f5c973a57a0 20 read_log 192888'478 (192888'461) delete 2c26a9fb/benchmark_data_storage0111-ib.pocstor_35625_object6864/head//22 by client.940330.0:38172 2014-08-11 16:31:43.302762 2014-08-13 18:06:06.469722 7f5c973a57a0 20 read_log 192888'479 (192888'462) delete 4de21fb/benchmark_data_storage0111-ib.pocstor_35625_object9106/head//22 by client.940330.0:40414 2014-08-11 16:31:43.867858 2014-08-13 18:06:06.469738 7f5c973a57a0 20 read_log 192888'480 (192888'463) delete df5061fb/benchmark_data_storage0111-ib.pocstor_35625_object9110/head//22 by client.940330.0:40418 2014-08-11 16:31:43.868880 2014-08-13 18:06:06.469750 7f5c973a57a0 20 read_log 192888'481 (192888'464) delete 126a79fb/benchmark_data_storage0111-ib.pocstor_35625_object12346/head//22 by client.940330.0:43654 2014-08-11 16:31:44.744195 2014-08-13 18:06:06.469763 7f5c973a57a0 20 read_log 192888'482 (192888'465) delete e9d2d1fb/benchmark_data_storage0111-ib.pocstor_35625_object12374/head//22 by client.940330.0:43682 2014-08-11 16:31:44.750673 2014-08-13 18:06:06.469775 7f5c973a57a0 20 read_log 192888'483 (192888'466) delete 15ab9fb/benchmark_data_storage0111-ib.pocstor_35625_object14057/head//22 by client.940330.0:45365 2014-08-11 16:31:45.197280 2014-08-13 18:06:06.469787 7f5c973a57a0 20 read_log 192888'484 (192888'467) delete 201e01fb/benchmark_data_storage0111-ib.pocstor_35625_object17661/head//22 by client.940330.0:48969 2014-08-11 16:31:46.180881 2014-08-13 18:06:06.469810 7f5c973a57a0 20 read_log 192888'485 (192888'468) delete ec9299fb/benchmark_data_storage0111-ib.pocstor_35625_object18051/head//22 by client.940330.0:49359 2014-08-11 16:31:46.300935 2014-08-13 18:06:06.469825 7f5c973a57a0 20 read_log 192888'486 (192888'469) delete 430bd1fb/benchmark_data_storage0111-ib.pocstor_35625_object21376/head//22 by client.940330.0:52684 2014-08-11 16:31:47.261403 2014-08-13 18:06:06.469837 7f5c973a57a0 20 read_log 192888'487 (192888'470) delete fa2891fb/benchmark_data_storage0111-ib.pocstor_35625_object22572/head//22 by client.940330.0:53880 2014-08-11 16:31:47.625966 2014-08-13 18:06:06.469850 7f5c973a57a0 20 read_log 192888'488 (192888'471) delete 4012c9fb/benchmark_data_storage0111-ib.pocstor_35625_object24261/head//22 by client.940330.0:55569 2014-08-11 16:31:48.116812 2014-08-13 18:06:06.469862 7f5c973a57a0 20 read_log 192888'489 (192888'472) delete e49cc9fb/benchmark_data_storage0111-ib.pocstor_35625_object28716/head//22 by client.940330.0:60024 2014-08-11 16:31:49.395968 2014-08-13 18:06:06.469874 7f5c973a57a0 20 read_log 192888'490 (192888'473) delete 2143b9fb/benchmark_data_storage0111-ib.pocstor_35625_object28772/head//22 by client.940330.0:60080 2014-08-11 16:31:49.411129 2014-08-13 18:06:06.469886 7f5c973a57a0 20 read_log 192888'491 (192888'474) delete f65409fb/benchmark_data_storage0111-ib.pocstor_35625_object28848/head//22 by client.940330.0:60156 2014-08-11 16:31:49.432132 2014-08-13 18:06:06.469898 7f5c973a57a0 20 read_log 192888'492 (192888'475) delete 11c731fb/benchmark_data_storage0111-ib.pocstor_35625_object31226/head//22 by client.940330.0:62534 2014-08-11 16:31:50.128190 2014-08-13 18:06:06.469911 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 9903e1fb/benchmark_data_storage0111-ib.pocstor_64498_object324/head//22 by client.1049635.0:325 2014-08-12 17:17:18.136507 2014-08-13 18:06:06.469923 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 897231fb/benchmark_data_storage0111-ib.pocstor_64498_object1672/head//22 by client.1049635.0:1673 2014-08-12 17:17:22.498188 2014-08-13 18:06:06.469940 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 508bb1fb/benchmark_data_storage0111-ib.pocstor_64498_object11163/head//22 by client.1049635.0:11164 2014-08-12 17:17:51.726622 2014-08-13 18:06:06.469953 7f5c973a57a0 20 read_log 194215'496 (0'0) modify f42729fb/benchmark_data_storage0111-ib.pocstor_64498_object12326/head//22 by client.1049635.0:12327 2014-08-12 17:17:55.420344 2014-08-13 18:06:06.469965 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 2ce931fb/benchmark_data_storage0111-ib.pocstor_64498_object14190/head//22 by client.1049635.0:14191 2014-08-12 17:18:01.082516 2014-08-13 18:06:06.469985 7f5c973a57a0 20 read_log 194215'498 (0'0) modify b22b59fb/benchmark_data_storage0111-ib.pocstor_64498_object16283/head//22 by client.1049635.0:16284 2014-08-12 17:18:08.632257 2014-08-13 18:06:06.470000 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 713421fb/benchmark_data_storage0111-ib.pocstor_64498_object17669/head//22 by client.1049635.0:17670 2014-08-12 17:18:13.905150 2014-08-13 18:06:06.470017 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 3f5a51fb/benchmark_data_storage0111-ib.pocstor_64498_object20027/head//22 by client.1049635.0:20028 2014-08-12 17:18:22.883952 2014-08-13 18:06:06.470029 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 78cd61fb/benchmark_data_storage0111-ib.pocstor_64498_object20894/head//22 by client.1049635.0:20895 2014-08-12 17:18:26.131271 2014-08-13 18:06:06.470042 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 6eb279fb/benchmark_data_storage0111-ib.pocstor_64498_object25470/head//22 by client.1049635.0:25471 2014-08-12 17:18:43.454644 2014-08-13 18:06:06.470054 7f5c973a57a0 20 read_log 194215'503 (0'0) modify 33fe11fb/benchmark_data_storage0111-ib.pocstor_64498_object27034/head//22 by client.1049635.0:27035 2014-08-12 17:18:49.275326 2014-08-13 18:06:06.470080 7f5c973a57a0 20 read_log 194215'504 (0'0) modify 4e3d69fb/benchmark_data_storage0111-ib.pocstor_64498_object27074/head//22 by client.1049635.0:27075 2014-08-12 17:18:49.425237 2014-08-13 18:06:06.470093 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 7aa881fb/benchmark_data_storage0111-ib.pocstor_64498_object28819/head//22 by client.1049635.0:28820 2014-08-12 17:18:55.925099 2014-08-13 18:06:06.470120 7f5c973a57a0 20 read_log 194215'506 (0'0) modify 630141fb/benchmark_data_storage0111-ib.pocstor_64498_object30109/head//22 by client.1049635.0:30110 2014-08-12 17:19:00.901195 2014-08-13 18:06:06.470136 7f5c973a57a0 20 read_log 194215'507 (0'0) modify 92ba79fb/benchmark_data_storage0111-ib.pocstor_64498_object30676/head//22 by client.1049635.0:30677 2014-08-12 17:19:03.174201 2014-08-13 18:06:06.470162 7f5c973a57a0 20 read_log 194215'508 (194215'493) delete 9903e1fb/benchmark_data_storage0111-ib.pocstor_64498_object324/head//22 by client.1049635.0:34527 2014-08-12 17:19:18.098805 2014-08-13 18:06:06.470180 7f5c973a57a0 20 read_log 194215'509 (194215'494) delete 897231fb/benchmark_data_storage0111-ib.pocstor_64498_object1672/head//22 by client.1049635.0:35875 2014-08-12 17:19:18.541691 2014-08-13 18:06:06.470192 7f5c973a57a0 20 read_log 194215'510 (194215'495) delete 508bb1fb/benchmark_data_storage0111-ib.pocstor_64498_object11163/head//22 by client.1049635.0:45366 2014-08-12 17:19:21.630442 2014-08-13 18:06:06.470205 7f5c973a57a0 20 read_log 194215'511 (194215'496) delete f42729fb/benchmark_data_storage0111-ib.pocstor_64498_object12326/head//22 by client.1049635.0:46529 2014-08-12 17:19:22.000955 2014-08-13 18:06:06.470217 7f5c973a57a0 20 read_log 194215'512 (194215'497) delete 2ce931fb/benchmark_data_storage0111-ib.pocstor_64498_object14190/head//22 by client.1049635.0:48393 2014-08-12 17:19:22.619583 2014-08-13 18:06:06.470241 7f5c973a57a0 20 read_log 194215'513 (194215'498) delete b22b59fb/benchmark_data_storage0111-ib.pocstor_64498_object16283/head//22 by client.1049635.0:50486 2014-08-12 17:19:23.278073 2014-08-13 18:06:06.470262 7f5c973a57a0 20 read_log 194215'514 (194215'499) delete 713421fb/benchmark_data_storage0111-ib.pocstor_64498_object17669/head//22 by client.1049635.0:51872 2014-08-12 17:19:23.717968 2014-08-13 18:06:06.470278 7f5c973a57a0 20 read_log 194215'515 (194215'500) delete 3f5a51fb/benchmark_data_storage0111-ib.pocstor_64498_object20027/head//22 by client.1049635.0:54230 2014-08-12 17:19:24.523712 2014-08-13 18:06:06.470290 7f5c973a57a0 20 read_log 194215'516 (194215'501) delete 78cd61fb/benchmark_data_storage0111-ib.pocstor_64498_object20894/head//22 by client.1049635.0:55097 2014-08-12 17:19:24.847906 2014-08-13 18:06:06.470303 7f5c973a57a0 20 read_log 194215'517 (194215'502) delete 6eb279fb/benchmark_data_storage0111-ib.pocstor_64498_object25470/head//22 by client.1049635.0:59673 2014-08-12 17:19:26.474435 2014-08-13 18:06:06.470343 7f5c973a57a0 20 read_log 194215'518 (194215'503) delete 33fe11fb/benchmark_data_storage0111-ib.pocstor_64498_object27034/head//22 by client.1049635.0:61237 2014-08-12 17:19:27.012541 2014-08-13 18:06:06.470360 7f5c973a57a0 20 read_log 194215'519 (194215'504) delete 4e3d69fb/benchmark_data_storage0111-ib.pocstor_64498_object27074/head//22 by client.1049635.0:61277 2014-08-12 17:19:27.023532 2014-08-13 18:06:06.470373 7f5c973a57a0 20 read_log 194215'520 (194215'505) delete 7aa881fb/benchmark_data_storage0111-ib.pocstor_64498_object28819/head//22 by client.1049635.0:63022 2014-08-12 17:19:27.634200 2014-08-13 18:06:06.470385 7f5c973a57a0 20 read_log 194215'521 (194215'506) delete 630141fb/benchmark_data_storage0111-ib.pocstor_64498_object30109/head//22 by client.1049635.0:64312 2014-08-12 17:19:28.080511 2014-08-13 18:06:06.470398 7f5c973a57a0 20 read_log 194215'522 (194215'507) delete 92ba79fb/benchmark_data_storage0111-ib.pocstor_64498_object30676/head//22 by client.1049635.0:64879 2014-08-12 17:19:28.266623 2014-08-13 18:06:06.470515 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.470534 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.470550 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.002297 0 0.000000 2014-08-13 18:06:06.470562 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.470570 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=0 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.470580 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] log((0'0,194215'522], crt=192888'492) 2014-08-13 18:06:06.470591 7f5c973a57a0 10 osd.4 194487 pgid 22.220 coll 22.220_head 2014-08-13 18:06:06.470762 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.220 2014-08-13 18:06:06.470772 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.470797 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.220(unlocked)] enter Initial 2014-08-13 18:06:06.470809 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.220(unlocked)] enter NotTrimming 2014-08-13 18:06:06.470960 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.471100 7f5c973a57a0 20 read_log 191648'7 (0'0) modify d8526a20/benchmark_data_storage0111-ib.pocstor_40745_object555/head//22 by client.273158.0:556 2014-07-30 14:08:50.853061 2014-08-13 18:06:06.471122 7f5c973a57a0 20 read_log 191648'27 (0'0) modify a2194a20/benchmark_data_storage0111-ib.pocstor_40745_object1699/head//22 by client.273158.0:1700 2014-07-30 14:08:55.025610 2014-08-13 18:06:06.471135 7f5c973a57a0 20 read_log 191648'50 (0'0) modify 2d49ba20/benchmark_data_storage0111-ib.pocstor_40745_object2873/head//22 by client.273158.0:2874 2014-07-30 14:09:00.853219 2014-08-13 18:06:06.471157 7f5c973a57a0 20 read_log 191648'59 (0'0) modify c125d220/benchmark_data_storage0111-ib.pocstor_40745_object3402/head//22 by client.273158.0:3403 2014-07-30 14:09:03.246494 2014-08-13 18:06:06.471173 7f5c973a57a0 20 read_log 191648'80 (0'0) modify 83db220/benchmark_data_storage0111-ib.pocstor_40745_object4815/head//22 by client.273158.0:4816 2014-07-30 14:09:10.206601 2014-08-13 18:06:06.471185 7f5c973a57a0 20 read_log 191648'88 (0'0) modify 123d0220/benchmark_data_storage0111-ib.pocstor_40745_object5226/head//22 by client.273158.0:5227 2014-07-30 14:09:12.335079 2014-08-13 18:06:06.471198 7f5c973a57a0 20 read_log 191648'110 (0'0) modify fa035a20/benchmark_data_storage0111-ib.pocstor_40745_object6605/head//22 by client.273158.0:6606 2014-07-30 14:09:19.375385 2014-08-13 18:06:06.471210 7f5c973a57a0 20 read_log 191648'144 (0'0) modify 51a55220/benchmark_data_storage0111-ib.pocstor_40745_object8304/head//22 by client.273158.0:8305 2014-07-30 14:09:27.845135 2014-08-13 18:06:06.471222 7f5c973a57a0 20 read_log 191648'149 (0'0) modify 10b7b220/benchmark_data_storage0111-ib.pocstor_40745_object8710/head//22 by client.273158.0:8711 2014-07-30 14:09:30.032909 2014-08-13 18:06:06.471256 7f5c973a57a0 20 read_log 191648'213 (0'0) modify f2ad0220/benchmark_data_storage0111-ib.pocstor_40745_object12399/head//22 by client.273158.0:12400 2014-07-30 14:09:49.578673 2014-08-13 18:06:06.471271 7f5c973a57a0 20 read_log 191648'217 (0'0) modify aa7f9220/benchmark_data_storage0111-ib.pocstor_40745_object13009/head//22 by client.273158.0:13010 2014-07-30 14:09:52.399293 2014-08-13 18:06:06.471284 7f5c973a57a0 20 read_log 191648'241 (0'0) modify df2a9a20/benchmark_data_storage0111-ib.pocstor_40745_object14295/head//22 by client.273158.0:14296 2014-07-30 14:09:59.191602 2014-08-13 18:06:06.471304 7f5c973a57a0 20 read_log 191648'258 (0'0) modify a1011a20/benchmark_data_storage0111-ib.pocstor_40745_object16025/head//22 by client.273158.0:16026 2014-07-30 14:10:07.702993 2014-08-13 18:06:06.471343 7f5c973a57a0 20 read_log 191648'329 (0'0) modify 4235d220/benchmark_data_storage0111-ib.pocstor_40745_object20169/head//22 by client.273158.0:20170 2014-07-30 14:10:32.155820 2014-08-13 18:06:06.471359 7f5c973a57a0 20 read_log 191727'374 (0'0) modify fa3ada20/benchmark_data_storage0111-ib.pocstor_51285_object1425/head//22 by client.275175.0:1426 2014-07-30 14:18:38.954091 2014-08-13 18:06:06.471371 7f5c973a57a0 20 read_log 191727'375 (0'0) modify b1065a20/benchmark_data_storage0111-ib.pocstor_51285_object2148/head//22 by client.275175.0:2149 2014-07-30 14:18:41.381351 2014-08-13 18:06:06.471397 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 9f56aa20/benchmark_data_storage0111-ib.pocstor_51285_object4051/head//22 by client.275175.0:4052 2014-07-30 14:18:47.514397 2014-08-13 18:06:06.471423 7f5c973a57a0 20 read_log 191727'380 (0'0) modify 66b72220/benchmark_data_storage0111-ib.pocstor_51285_object4608/head//22 by client.275175.0:4609 2014-07-30 14:18:49.185942 2014-08-13 18:06:06.471440 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 53151220/benchmark_data_storage0111-ib.pocstor_51285_object5602/head//22 by client.275175.0:5603 2014-07-30 14:18:52.108010 2014-08-13 18:06:06.471453 7f5c973a57a0 20 read_log 191727'389 (0'0) modify c009ea20/benchmark_data_storage0111-ib.pocstor_51285_object9325/head//22 by client.275175.0:9326 2014-07-30 14:19:03.292991 2014-08-13 18:06:06.471472 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 6864a220/benchmark_data_storage0111-ib.pocstor_51285_object12381/head//22 by client.275175.0:12382 2014-07-30 14:19:12.703588 2014-08-13 18:06:06.471488 7f5c973a57a0 20 read_log 191727'407 (0'0) modify d357220/benchmark_data_storage0111-ib.pocstor_51285_object19103/head//22 by client.275175.0:19104 2014-07-30 14:19:33.624767 2014-08-13 18:06:06.471500 7f5c973a57a0 20 read_log 191727'408 (0'0) modify 9ce4da20/benchmark_data_storage0111-ib.pocstor_51285_object19841/head//22 by client.275175.0:19842 2014-07-30 14:19:35.726695 2014-08-13 18:06:06.471513 7f5c973a57a0 20 read_log 191727'409 (0'0) modify b8288a20/benchmark_data_storage0111-ib.pocstor_51285_object20289/head//22 by client.275175.0:20290 2014-07-30 14:19:37.093241 2014-08-13 18:06:06.471525 7f5c973a57a0 20 read_log 191727'410 (0'0) modify c5849220/benchmark_data_storage0111-ib.pocstor_51285_object20433/head//22 by client.275175.0:20434 2014-07-30 14:19:37.505847 2014-08-13 18:06:06.471548 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 9732e220/benchmark_data_storage0111-ib.pocstor_51285_object22699/head//22 by client.275175.0:22700 2014-07-30 14:19:45.090339 2014-08-13 18:06:06.471561 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 73fc1220/benchmark_data_storage0111-ib.pocstor_51285_object23070/head//22 by client.275175.0:23071 2014-07-30 14:19:46.223055 2014-08-13 18:06:06.471573 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 73e59a20/benchmark_data_storage0111-ib.pocstor_51285_object24191/head//22 by client.275175.0:24192 2014-07-30 14:19:49.365760 2014-08-13 18:06:06.471585 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 2c01ba20/benchmark_data_storage0111-ib.pocstor_51285_object24662/head//22 by client.275175.0:24663 2014-07-30 14:19:50.692751 2014-08-13 18:06:06.471598 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 2b3daa20/benchmark_data_storage0111-ib.pocstor_51285_object28342/head//22 by client.275175.0:28343 2014-07-30 14:20:01.981635 2014-08-13 18:06:06.471611 7f5c973a57a0 20 read_log 191727'432 (0'0) modify 4bec1a20/benchmark_data_storage0111-ib.pocstor_51285_object31025/head//22 by client.275175.0:31026 2014-07-30 14:20:10.054598 2014-08-13 18:06:06.471624 7f5c973a57a0 20 read_log 191727'434 (0'0) modify 1af60a20/benchmark_data_storage0111-ib.pocstor_51285_object31610/head//22 by client.275175.0:31611 2014-07-30 14:20:11.759890 2014-08-13 18:06:06.471636 7f5c973a57a0 20 read_log 191727'436 (0'0) modify 78132220/benchmark_data_storage0111-ib.pocstor_51285_object32172/head//22 by client.275175.0:32173 2014-07-30 14:20:13.386253 2014-08-13 18:06:06.471648 7f5c973a57a0 20 read_log 191727'437 (0'0) modify 1595f220/benchmark_data_storage0111-ib.pocstor_51285_object32760/head//22 by client.275175.0:32761 2014-07-30 14:20:15.096708 2014-08-13 18:06:06.471660 7f5c973a57a0 20 read_log 191727'442 (0'0) modify bb376a20/benchmark_data_storage0111-ib.pocstor_51285_object34868/head//22 by client.275175.0:34869 2014-07-30 14:20:21.610610 2014-08-13 18:06:06.471672 7f5c973a57a0 20 read_log 191727'449 (0'0) modify ea85ba20/benchmark_data_storage0111-ib.pocstor_51285_object38693/head//22 by client.275175.0:38694 2014-07-30 14:20:33.220940 2014-08-13 18:06:06.471693 7f5c973a57a0 20 read_log 192888'450 (0'0) modify e903ea20/benchmark_data_storage0111-ib.pocstor_35625_object941/head//22 by client.940330.0:942 2014-08-11 16:29:44.463380 2014-08-13 18:06:06.471708 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 65670220/benchmark_data_storage0111-ib.pocstor_35625_object4252/head//22 by client.940330.0:4253 2014-08-11 16:29:57.245207 2014-08-13 18:06:06.471721 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 3ce8ca20/benchmark_data_storage0111-ib.pocstor_35625_object5743/head//22 by client.940330.0:5744 2014-08-11 16:30:02.656987 2014-08-13 18:06:06.471733 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 2edd3220/benchmark_data_storage0111-ib.pocstor_35625_object8808/head//22 by client.940330.0:8809 2014-08-11 16:30:13.662150 2014-08-13 18:06:06.471745 7f5c973a57a0 20 read_log 192888'454 (0'0) modify d8068a20/benchmark_data_storage0111-ib.pocstor_35625_object9121/head//22 by client.940330.0:9122 2014-08-11 16:30:14.755318 2014-08-13 18:06:06.471757 7f5c973a57a0 20 read_log 192888'455 (0'0) modify ffe40220/benchmark_data_storage0111-ib.pocstor_35625_object9551/head//22 by client.940330.0:9552 2014-08-11 16:30:16.258341 2014-08-13 18:06:06.471769 7f5c973a57a0 20 read_log 192888'456 (0'0) modify 2f067220/benchmark_data_storage0111-ib.pocstor_35625_object10780/head//22 by client.940330.0:10781 2014-08-11 16:30:20.946971 2014-08-13 18:06:06.471782 7f5c973a57a0 20 read_log 192888'457 (0'0) modify edd4aa20/benchmark_data_storage0111-ib.pocstor_35625_object12386/head//22 by client.940330.0:12387 2014-08-11 16:30:27.456582 2014-08-13 18:06:06.471793 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 4dd7da20/benchmark_data_storage0111-ib.pocstor_35625_object12499/head//22 by client.940330.0:12500 2014-08-11 16:30:27.890650 2014-08-13 18:06:06.471805 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 274eba20/benchmark_data_storage0111-ib.pocstor_35625_object14106/head//22 by client.940330.0:14107 2014-08-11 16:30:34.138783 2014-08-13 18:06:06.471817 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 82d6da20/benchmark_data_storage0111-ib.pocstor_35625_object17347/head//22 by client.940330.0:17348 2014-08-11 16:30:46.264014 2014-08-13 18:06:06.471833 7f5c973a57a0 20 read_log 192888'461 (0'0) modify e3284a20/benchmark_data_storage0111-ib.pocstor_35625_object18589/head//22 by client.940330.0:18590 2014-08-11 16:30:51.093356 2014-08-13 18:06:06.471847 7f5c973a57a0 20 read_log 192888'462 (0'0) modify bd89e220/benchmark_data_storage0111-ib.pocstor_35625_object18980/head//22 by client.940330.0:18981 2014-08-11 16:30:52.595003 2014-08-13 18:06:06.471859 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 2f326a20/benchmark_data_storage0111-ib.pocstor_35625_object20637/head//22 by client.940330.0:20638 2014-08-11 16:30:59.200285 2014-08-13 18:06:06.471871 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 92dc1a20/benchmark_data_storage0111-ib.pocstor_35625_object24416/head//22 by client.940330.0:24417 2014-08-11 16:31:13.662052 2014-08-13 18:06:06.471883 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 85b54220/benchmark_data_storage0111-ib.pocstor_35625_object24660/head//22 by client.940330.0:24661 2014-08-11 16:31:14.564175 2014-08-13 18:06:06.471898 7f5c973a57a0 20 read_log 192888'466 (192888'450) delete e903ea20/benchmark_data_storage0111-ib.pocstor_35625_object941/head//22 by client.940330.0:32249 2014-08-11 16:31:41.735824 2014-08-13 18:06:06.471914 7f5c973a57a0 20 read_log 192888'467 (192888'451) delete 65670220/benchmark_data_storage0111-ib.pocstor_35625_object4252/head//22 by client.940330.0:35560 2014-08-11 16:31:42.607456 2014-08-13 18:06:06.471929 7f5c973a57a0 20 read_log 192888'468 (192888'452) delete 3ce8ca20/benchmark_data_storage0111-ib.pocstor_35625_object5743/head//22 by client.940330.0:37051 2014-08-11 16:31:43.008377 2014-08-13 18:06:06.471944 7f5c973a57a0 20 read_log 192888'469 (192888'453) delete 2edd3220/benchmark_data_storage0111-ib.pocstor_35625_object8808/head//22 by client.940330.0:40116 2014-08-11 16:31:43.793671 2014-08-13 18:06:06.471964 7f5c973a57a0 20 read_log 192888'470 (192888'454) delete d8068a20/benchmark_data_storage0111-ib.pocstor_35625_object9121/head//22 by client.940330.0:40429 2014-08-11 16:31:43.871541 2014-08-13 18:06:06.471979 7f5c973a57a0 20 read_log 192888'471 (192888'455) delete ffe40220/benchmark_data_storage0111-ib.pocstor_35625_object9551/head//22 by client.940330.0:40859 2014-08-11 16:31:43.990218 2014-08-13 18:06:06.471991 7f5c973a57a0 20 read_log 192888'472 (192888'456) delete 2f067220/benchmark_data_storage0111-ib.pocstor_35625_object10780/head//22 by client.940330.0:42088 2014-08-11 16:31:44.336405 2014-08-13 18:06:06.472005 7f5c973a57a0 20 read_log 192888'473 (192888'457) delete edd4aa20/benchmark_data_storage0111-ib.pocstor_35625_object12386/head//22 by client.940330.0:43694 2014-08-11 16:31:44.753447 2014-08-13 18:06:06.472019 7f5c973a57a0 20 read_log 192888'474 (192888'458) delete 4dd7da20/benchmark_data_storage0111-ib.pocstor_35625_object12499/head//22 by client.940330.0:43807 2014-08-11 16:31:44.782374 2014-08-13 18:06:06.472033 7f5c973a57a0 20 read_log 192888'475 (192888'459) delete 274eba20/benchmark_data_storage0111-ib.pocstor_35625_object14106/head//22 by client.940330.0:45414 2014-08-11 16:31:45.211039 2014-08-13 18:06:06.472046 7f5c973a57a0 20 read_log 192888'476 (192888'460) delete 82d6da20/benchmark_data_storage0111-ib.pocstor_35625_object17347/head//22 by client.940330.0:48655 2014-08-11 16:31:46.088339 2014-08-13 18:06:06.472060 7f5c973a57a0 20 read_log 192888'477 (192888'461) delete e3284a20/benchmark_data_storage0111-ib.pocstor_35625_object18589/head//22 by client.940330.0:49897 2014-08-11 16:31:46.470984 2014-08-13 18:06:06.472073 7f5c973a57a0 20 read_log 192888'478 (192888'462) delete bd89e220/benchmark_data_storage0111-ib.pocstor_35625_object18980/head//22 by client.940330.0:50288 2014-08-11 16:31:46.576234 2014-08-13 18:06:06.472087 7f5c973a57a0 20 read_log 192888'479 (192888'463) delete 2f326a20/benchmark_data_storage0111-ib.pocstor_35625_object20637/head//22 by client.940330.0:51945 2014-08-11 16:31:47.062947 2014-08-13 18:06:06.472099 7f5c973a57a0 20 read_log 192888'480 (192888'464) delete 92dc1a20/benchmark_data_storage0111-ib.pocstor_35625_object24416/head//22 by client.940330.0:55724 2014-08-11 16:31:48.162523 2014-08-13 18:06:06.472114 7f5c973a57a0 20 read_log 192888'481 (192888'465) delete 85b54220/benchmark_data_storage0111-ib.pocstor_35625_object24660/head//22 by client.940330.0:55968 2014-08-11 16:31:48.233028 2014-08-13 18:06:06.472126 7f5c973a57a0 20 read_log 194215'482 (0'0) modify daa2d220/benchmark_data_storage0111-ib.pocstor_64498_object2015/head//22 by client.1049635.0:2016 2014-08-12 17:17:23.601792 2014-08-13 18:06:06.472139 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 81737220/benchmark_data_storage0111-ib.pocstor_64498_object3735/head//22 by client.1049635.0:3736 2014-08-12 17:17:29.197328 2014-08-13 18:06:06.472161 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 70ce3220/benchmark_data_storage0111-ib.pocstor_64498_object4977/head//22 by client.1049635.0:4978 2014-08-12 17:17:32.864213 2014-08-13 18:06:06.472174 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 120e3a20/benchmark_data_storage0111-ib.pocstor_64498_object7336/head//22 by client.1049635.0:7337 2014-08-12 17:17:40.076761 2014-08-13 18:06:06.472186 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 3cff3a20/benchmark_data_storage0111-ib.pocstor_64498_object8659/head//22 by client.1049635.0:8660 2014-08-12 17:17:44.050581 2014-08-13 18:06:06.472201 7f5c973a57a0 20 read_log 194215'487 (0'0) modify de9e5a20/benchmark_data_storage0111-ib.pocstor_64498_object8728/head//22 by client.1049635.0:8729 2014-08-12 17:17:44.250155 2014-08-13 18:06:06.472213 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 9237a220/benchmark_data_storage0111-ib.pocstor_64498_object11397/head//22 by client.1049635.0:11398 2014-08-12 17:17:52.504411 2014-08-13 18:06:06.472237 7f5c973a57a0 20 read_log 194215'489 (0'0) modify d8282220/benchmark_data_storage0111-ib.pocstor_64498_object11630/head//22 by client.1049635.0:11631 2014-08-12 17:17:53.300539 2014-08-13 18:06:06.472256 7f5c973a57a0 20 read_log 194215'490 (0'0) modify f78eb220/benchmark_data_storage0111-ib.pocstor_64498_object11845/head//22 by client.1049635.0:11846 2014-08-12 17:17:53.949671 2014-08-13 18:06:06.472269 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 87457220/benchmark_data_storage0111-ib.pocstor_64498_object16664/head//22 by client.1049635.0:16665 2014-08-12 17:18:10.085813 2014-08-13 18:06:06.472281 7f5c973a57a0 20 read_log 194215'492 (0'0) modify 6d694220/benchmark_data_storage0111-ib.pocstor_64498_object18060/head//22 by client.1049635.0:18061 2014-08-12 17:18:15.392886 2014-08-13 18:06:06.472299 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 91fd6220/benchmark_data_storage0111-ib.pocstor_64498_object18135/head//22 by client.1049635.0:18136 2014-08-12 17:18:15.672932 2014-08-13 18:06:06.472314 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 882f2a20/benchmark_data_storage0111-ib.pocstor_64498_object19891/head//22 by client.1049635.0:19892 2014-08-12 17:18:22.368645 2014-08-13 18:06:06.472326 7f5c973a57a0 20 read_log 194215'495 (0'0) modify fa663a20/benchmark_data_storage0111-ib.pocstor_64498_object19967/head//22 by client.1049635.0:19968 2014-08-12 17:18:22.659504 2014-08-13 18:06:06.472342 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 615e7220/benchmark_data_storage0111-ib.pocstor_64498_object21627/head//22 by client.1049635.0:21628 2014-08-12 17:18:28.929614 2014-08-13 18:06:06.472357 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 484f4220/benchmark_data_storage0111-ib.pocstor_64498_object24344/head//22 by client.1049635.0:24345 2014-08-12 17:18:39.137481 2014-08-13 18:06:06.472378 7f5c973a57a0 20 read_log 194215'498 (0'0) modify ace15220/benchmark_data_storage0111-ib.pocstor_64498_object26081/head//22 by client.1049635.0:26082 2014-08-12 17:18:45.725292 2014-08-13 18:06:06.472398 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 15490220/benchmark_data_storage0111-ib.pocstor_64498_object28324/head//22 by client.1049635.0:28325 2014-08-12 17:18:54.084046 2014-08-13 18:06:06.472413 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 7c1a0220/benchmark_data_storage0111-ib.pocstor_64498_object30361/head//22 by client.1049635.0:30362 2014-08-12 17:19:01.981843 2014-08-13 18:06:06.472435 7f5c973a57a0 20 read_log 194215'501 (0'0) modify a7166220/benchmark_data_storage0111-ib.pocstor_64498_object33579/head//22 by client.1049635.0:33580 2014-08-12 17:19:14.586261 2014-08-13 18:06:06.472448 7f5c973a57a0 20 read_log 194215'502 (194215'482) delete daa2d220/benchmark_data_storage0111-ib.pocstor_64498_object2015/head//22 by client.1049635.0:36218 2014-08-12 17:19:18.652846 2014-08-13 18:06:06.472460 7f5c973a57a0 20 read_log 194215'503 (194215'483) delete 81737220/benchmark_data_storage0111-ib.pocstor_64498_object3735/head//22 by client.1049635.0:37938 2014-08-12 17:19:19.194478 2014-08-13 18:06:06.472482 7f5c973a57a0 20 read_log 194215'504 (194215'484) delete 70ce3220/benchmark_data_storage0111-ib.pocstor_64498_object4977/head//22 by client.1049635.0:39180 2014-08-12 17:19:19.595325 2014-08-13 18:06:06.472498 7f5c973a57a0 20 read_log 194215'505 (194215'485) delete 120e3a20/benchmark_data_storage0111-ib.pocstor_64498_object7336/head//22 by client.1049635.0:41539 2014-08-12 17:19:20.395976 2014-08-13 18:06:06.472511 7f5c973a57a0 20 read_log 194215'506 (194215'486) delete 3cff3a20/benchmark_data_storage0111-ib.pocstor_64498_object8659/head//22 by client.1049635.0:42862 2014-08-12 17:19:20.817571 2014-08-13 18:06:06.472523 7f5c973a57a0 20 read_log 194215'507 (194215'487) delete de9e5a20/benchmark_data_storage0111-ib.pocstor_64498_object8728/head//22 by client.1049635.0:42931 2014-08-12 17:19:20.840932 2014-08-13 18:06:06.472546 7f5c973a57a0 20 read_log 194215'508 (194215'488) delete 9237a220/benchmark_data_storage0111-ib.pocstor_64498_object11397/head//22 by client.1049635.0:45600 2014-08-12 17:19:21.703329 2014-08-13 18:06:06.472567 7f5c973a57a0 20 read_log 194215'509 (194215'489) delete d8282220/benchmark_data_storage0111-ib.pocstor_64498_object11630/head//22 by client.1049635.0:45833 2014-08-12 17:19:21.776101 2014-08-13 18:06:06.472589 7f5c973a57a0 20 read_log 194215'510 (194215'490) delete f78eb220/benchmark_data_storage0111-ib.pocstor_64498_object11845/head//22 by client.1049635.0:46048 2014-08-12 17:19:21.854274 2014-08-13 18:06:06.472601 7f5c973a57a0 20 read_log 194215'511 (194215'491) delete 87457220/benchmark_data_storage0111-ib.pocstor_64498_object16664/head//22 by client.1049635.0:50867 2014-08-12 17:19:23.401820 2014-08-13 18:06:06.472614 7f5c973a57a0 20 read_log 194215'512 (194215'492) delete 6d694220/benchmark_data_storage0111-ib.pocstor_64498_object18060/head//22 by client.1049635.0:52263 2014-08-12 17:19:23.851232 2014-08-13 18:06:06.472626 7f5c973a57a0 20 read_log 194215'513 (194215'493) delete 91fd6220/benchmark_data_storage0111-ib.pocstor_64498_object18135/head//22 by client.1049635.0:52338 2014-08-12 17:19:23.874911 2014-08-13 18:06:06.472638 7f5c973a57a0 20 read_log 194215'514 (194215'494) delete 882f2a20/benchmark_data_storage0111-ib.pocstor_64498_object19891/head//22 by client.1049635.0:54094 2014-08-12 17:19:24.475404 2014-08-13 18:06:06.472650 7f5c973a57a0 20 read_log 194215'515 (194215'495) delete fa663a20/benchmark_data_storage0111-ib.pocstor_64498_object19967/head//22 by client.1049635.0:54170 2014-08-12 17:19:24.501717 2014-08-13 18:06:06.472680 7f5c973a57a0 20 read_log 194215'516 (194215'496) delete 615e7220/benchmark_data_storage0111-ib.pocstor_64498_object21627/head//22 by client.1049635.0:55830 2014-08-12 17:19:25.098019 2014-08-13 18:06:06.472695 7f5c973a57a0 20 read_log 194215'517 (194215'497) delete 484f4220/benchmark_data_storage0111-ib.pocstor_64498_object24344/head//22 by client.1049635.0:58547 2014-08-12 17:19:26.081970 2014-08-13 18:06:06.472708 7f5c973a57a0 20 read_log 194215'518 (194215'498) delete ace15220/benchmark_data_storage0111-ib.pocstor_64498_object26081/head//22 by client.1049635.0:60284 2014-08-12 17:19:26.684738 2014-08-13 18:06:06.472720 7f5c973a57a0 20 read_log 194215'519 (194215'499) delete 15490220/benchmark_data_storage0111-ib.pocstor_64498_object28324/head//22 by client.1049635.0:62527 2014-08-12 17:19:27.462991 2014-08-13 18:06:06.472732 7f5c973a57a0 20 read_log 194215'520 (194215'500) delete 7c1a0220/benchmark_data_storage0111-ib.pocstor_64498_object30361/head//22 by client.1049635.0:64564 2014-08-12 17:19:28.162993 2014-08-13 18:06:06.472745 7f5c973a57a0 20 read_log 194215'521 (194215'501) delete a7166220/benchmark_data_storage0111-ib.pocstor_64498_object33579/head//22 by client.1049635.0:67782 2014-08-12 17:19:29.322810 2014-08-13 18:06:06.472837 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.472855 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=0 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.472870 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=0 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Initial 0.002073 0 0.000000 2014-08-13 18:06:06.472883 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=0 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.472891 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=0 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.472900 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] log((0'0,194215'521], crt=192888'481) 2014-08-13 18:06:06.472910 7f5c973a57a0 10 osd.4 194487 pgid 22.226 coll 22.226_head 2014-08-13 18:06:06.473065 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.226 2014-08-13 18:06:06.473074 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.473100 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.226(unlocked)] enter Initial 2014-08-13 18:06:06.473112 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.226(unlocked)] enter NotTrimming 2014-08-13 18:06:06.473261 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.473439 7f5c973a57a0 20 read_log 191648'79 (0'0) modify 5beeaa26/benchmark_data_storage0111-ib.pocstor_40745_object5094/head//22 by client.273158.0:5095 2014-07-30 14:09:11.434191 2014-08-13 18:06:06.473461 7f5c973a57a0 20 read_log 191648'87 (0'0) modify b564b226/benchmark_data_storage0111-ib.pocstor_40745_object5447/head//22 by client.273158.0:5448 2014-07-30 14:09:13.219988 2014-08-13 18:06:06.473474 7f5c973a57a0 20 read_log 191648'135 (0'0) modify 26c2aa26/benchmark_data_storage0111-ib.pocstor_40745_object8810/head//22 by client.273158.0:8811 2014-07-30 14:09:30.424843 2014-08-13 18:06:06.473486 7f5c973a57a0 20 read_log 191648'183 (0'0) modify e0ea9226/benchmark_data_storage0111-ib.pocstor_40745_object12276/head//22 by client.273158.0:12277 2014-07-30 14:09:48.976178 2014-08-13 18:06:06.473499 7f5c973a57a0 20 read_log 191648'186 (0'0) modify 533a226/benchmark_data_storage0111-ib.pocstor_40745_object12558/head//22 by client.273158.0:12559 2014-07-30 14:09:50.348134 2014-08-13 18:06:06.473511 7f5c973a57a0 20 read_log 191648'253 (0'0) modify f140a26/benchmark_data_storage0111-ib.pocstor_40745_object16781/head//22 by client.273158.0:16782 2014-07-30 14:10:11.620938 2014-08-13 18:06:06.473524 7f5c973a57a0 20 read_log 191648'254 (0'0) modify 70cc2226/benchmark_data_storage0111-ib.pocstor_40745_object16947/head//22 by client.273158.0:16948 2014-07-30 14:10:12.765066 2014-08-13 18:06:06.473536 7f5c973a57a0 20 read_log 191648'258 (0'0) modify f2a2fa26/benchmark_data_storage0111-ib.pocstor_40745_object17298/head//22 by client.273158.0:17299 2014-07-30 14:10:14.743060 2014-08-13 18:06:06.473548 7f5c973a57a0 20 read_log 191727'348 (0'0) modify 43d2ca26/benchmark_data_storage0111-ib.pocstor_51285_object980/head//22 by client.275175.0:981 2014-07-30 14:18:37.381352 2014-08-13 18:06:06.473565 7f5c973a57a0 20 read_log 191727'349 (0'0) modify e76be226/benchmark_data_storage0111-ib.pocstor_51285_object1838/head//22 by client.275175.0:1839 2014-07-30 14:18:40.370995 2014-08-13 18:06:06.473577 7f5c973a57a0 20 read_log 191727'353 (0'0) modify 9feffa26/benchmark_data_storage0111-ib.pocstor_51285_object3273/head//22 by client.275175.0:3274 2014-07-30 14:18:45.081706 2014-08-13 18:06:06.473598 7f5c973a57a0 20 read_log 191727'354 (0'0) modify 66e3ba26/benchmark_data_storage0111-ib.pocstor_51285_object3294/head//22 by client.275175.0:3295 2014-07-30 14:18:45.146923 2014-08-13 18:06:06.473613 7f5c973a57a0 20 read_log 191727'355 (0'0) modify 9f24aa26/benchmark_data_storage0111-ib.pocstor_51285_object4200/head//22 by client.275175.0:4201 2014-07-30 14:18:47.964507 2014-08-13 18:06:06.473626 7f5c973a57a0 20 read_log 191727'357 (0'0) modify f4072226/benchmark_data_storage0111-ib.pocstor_51285_object4791/head//22 by client.275175.0:4792 2014-07-30 14:18:49.732130 2014-08-13 18:06:06.473638 7f5c973a57a0 20 read_log 191727'359 (0'0) modify 7ef38226/benchmark_data_storage0111-ib.pocstor_51285_object5242/head//22 by client.275175.0:5243 2014-07-30 14:18:51.066689 2014-08-13 18:06:06.473650 7f5c973a57a0 20 read_log 191727'363 (0'0) modify 351f2226/benchmark_data_storage0111-ib.pocstor_51285_object6926/head//22 by client.275175.0:6927 2014-07-30 14:18:56.511740 2014-08-13 18:06:06.473662 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 1c452a26/benchmark_data_storage0111-ib.pocstor_51285_object11110/head//22 by client.275175.0:11111 2014-07-30 14:19:08.352260 2014-08-13 18:06:06.473700 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 44974a26/benchmark_data_storage0111-ib.pocstor_51285_object13020/head//22 by client.275175.0:13021 2014-07-30 14:19:14.753154 2014-08-13 18:06:06.473713 7f5c973a57a0 20 read_log 191727'381 (0'0) modify 6eddea26/benchmark_data_storage0111-ib.pocstor_51285_object18207/head//22 by client.275175.0:18208 2014-07-30 14:19:30.579104 2014-08-13 18:06:06.473725 7f5c973a57a0 20 read_log 191727'387 (0'0) modify 417f7a26/benchmark_data_storage0111-ib.pocstor_51285_object25116/head//22 by client.275175.0:25117 2014-07-30 14:19:52.081451 2014-08-13 18:06:06.473737 7f5c973a57a0 20 read_log 191727'390 (0'0) modify a156f226/benchmark_data_storage0111-ib.pocstor_51285_object25875/head//22 by client.275175.0:25876 2014-07-30 14:19:54.379013 2014-08-13 18:06:06.473749 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 316b4a26/benchmark_data_storage0111-ib.pocstor_51285_object26998/head//22 by client.275175.0:26999 2014-07-30 14:19:57.807183 2014-08-13 18:06:06.473761 7f5c973a57a0 20 read_log 191727'393 (0'0) modify c5382a26/benchmark_data_storage0111-ib.pocstor_51285_object27844/head//22 by client.275175.0:27845 2014-07-30 14:20:00.384169 2014-08-13 18:06:06.473773 7f5c973a57a0 20 read_log 191727'394 (0'0) modify d759ea26/benchmark_data_storage0111-ib.pocstor_51285_object27845/head//22 by client.275175.0:27846 2014-07-30 14:20:00.388669 2014-08-13 18:06:06.473785 7f5c973a57a0 20 read_log 191727'404 (0'0) modify 31e4a226/benchmark_data_storage0111-ib.pocstor_51285_object31642/head//22 by client.275175.0:31643 2014-07-30 14:20:11.850327 2014-08-13 18:06:06.473797 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 59b5226/benchmark_data_storage0111-ib.pocstor_51285_object32712/head//22 by client.275175.0:32713 2014-07-30 14:20:14.920491 2014-08-13 18:06:06.473809 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 296226/benchmark_data_storage0111-ib.pocstor_51285_object32919/head//22 by client.275175.0:32920 2014-07-30 14:20:15.571968 2014-08-13 18:06:06.473821 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 2d612a26/benchmark_data_storage0111-ib.pocstor_51285_object34599/head//22 by client.275175.0:34600 2014-07-30 14:20:20.830881 2014-08-13 18:06:06.473834 7f5c973a57a0 20 read_log 191727'414 (0'0) modify befcca26/benchmark_data_storage0111-ib.pocstor_51285_object35253/head//22 by client.275175.0:35254 2014-07-30 14:20:22.730597 2014-08-13 18:06:06.473846 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 874ffa26/benchmark_data_storage0111-ib.pocstor_51285_object35730/head//22 by client.275175.0:35731 2014-07-30 14:20:24.352034 2014-08-13 18:06:06.473858 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 39971a26/benchmark_data_storage0111-ib.pocstor_51285_object35961/head//22 by client.275175.0:35962 2014-07-30 14:20:25.143962 2014-08-13 18:06:06.473879 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 75ee6226/benchmark_data_storage0111-ib.pocstor_51285_object38501/head//22 by client.275175.0:38502 2014-07-30 14:20:32.671869 2014-08-13 18:06:06.473895 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 8edbd226/benchmark_data_storage0111-ib.pocstor_51285_object38593/head//22 by client.275175.0:38594 2014-07-30 14:20:32.936333 2014-08-13 18:06:06.473908 7f5c973a57a0 20 read_log 192888'426 (0'0) modify 28169226/benchmark_data_storage0111-ib.pocstor_35625_object930/head//22 by client.940330.0:931 2014-08-11 16:29:44.421330 2014-08-13 18:06:06.473921 7f5c973a57a0 20 read_log 192888'427 (0'0) modify c0a2e226/benchmark_data_storage0111-ib.pocstor_35625_object2409/head//22 by client.940330.0:2410 2014-08-11 16:29:50.032639 2014-08-13 18:06:06.473936 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 7a624226/benchmark_data_storage0111-ib.pocstor_35625_object3210/head//22 by client.940330.0:3211 2014-08-11 16:29:53.377910 2014-08-13 18:06:06.473950 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 5bbd5226/benchmark_data_storage0111-ib.pocstor_35625_object7386/head//22 by client.940330.0:7387 2014-08-11 16:30:08.679068 2014-08-13 18:06:06.473965 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 69cf6226/benchmark_data_storage0111-ib.pocstor_35625_object7675/head//22 by client.940330.0:7676 2014-08-11 16:30:09.690306 2014-08-13 18:06:06.473977 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 87567a26/benchmark_data_storage0111-ib.pocstor_35625_object8402/head//22 by client.940330.0:8403 2014-08-11 16:30:12.249023 2014-08-13 18:06:06.473991 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 6a30c226/benchmark_data_storage0111-ib.pocstor_35625_object8972/head//22 by client.940330.0:8973 2014-08-11 16:30:14.237050 2014-08-13 18:06:06.474004 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 723e9a26/benchmark_data_storage0111-ib.pocstor_35625_object9822/head//22 by client.940330.0:9823 2014-08-11 16:30:17.208055 2014-08-13 18:06:06.474018 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 523a1226/benchmark_data_storage0111-ib.pocstor_35625_object12469/head//22 by client.940330.0:12470 2014-08-11 16:30:27.778100 2014-08-13 18:06:06.474030 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 6a83a26/benchmark_data_storage0111-ib.pocstor_35625_object15689/head//22 by client.940330.0:15690 2014-08-11 16:30:40.052246 2014-08-13 18:06:06.474044 7f5c973a57a0 20 read_log 192888'436 (0'0) modify abe2d226/benchmark_data_storage0111-ib.pocstor_35625_object17639/head//22 by client.940330.0:17640 2014-08-11 16:30:47.363707 2014-08-13 18:06:06.474060 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 46e56226/benchmark_data_storage0111-ib.pocstor_35625_object21450/head//22 by client.940330.0:21451 2014-08-11 16:31:02.498031 2014-08-13 18:06:06.474075 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 71520226/benchmark_data_storage0111-ib.pocstor_35625_object22343/head//22 by client.940330.0:22344 2014-08-11 16:31:05.842558 2014-08-13 18:06:06.474087 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 67012a26/benchmark_data_storage0111-ib.pocstor_35625_object23662/head//22 by client.940330.0:23663 2014-08-11 16:31:10.889579 2014-08-13 18:06:06.474101 7f5c973a57a0 20 read_log 192888'440 (0'0) modify da86a226/benchmark_data_storage0111-ib.pocstor_35625_object23987/head//22 by client.940330.0:23988 2014-08-11 16:31:12.104568 2014-08-13 18:06:06.474113 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 9f30c226/benchmark_data_storage0111-ib.pocstor_35625_object26445/head//22 by client.940330.0:26446 2014-08-11 16:31:21.555578 2014-08-13 18:06:06.474127 7f5c973a57a0 20 read_log 192888'442 (0'0) modify b5366a26/benchmark_data_storage0111-ib.pocstor_35625_object26754/head//22 by client.940330.0:26755 2014-08-11 16:31:22.852783 2014-08-13 18:06:06.474174 7f5c973a57a0 20 read_log 192888'443 (0'0) modify a849e226/benchmark_data_storage0111-ib.pocstor_35625_object30156/head//22 by client.940330.0:30157 2014-08-11 16:31:36.047032 2014-08-13 18:06:06.474189 7f5c973a57a0 20 read_log 192888'444 (192888'426) delete 28169226/benchmark_data_storage0111-ib.pocstor_35625_object930/head//22 by client.940330.0:32238 2014-08-11 16:31:41.732762 2014-08-13 18:06:06.474202 7f5c973a57a0 20 read_log 192888'445 (192888'427) delete c0a2e226/benchmark_data_storage0111-ib.pocstor_35625_object2409/head//22 by client.940330.0:33717 2014-08-11 16:31:42.097585 2014-08-13 18:06:06.474217 7f5c973a57a0 20 read_log 192888'446 (192888'428) delete 7a624226/benchmark_data_storage0111-ib.pocstor_35625_object3210/head//22 by client.940330.0:34518 2014-08-11 16:31:42.310660 2014-08-13 18:06:06.474229 7f5c973a57a0 20 read_log 192888'447 (192888'429) delete 5bbd5226/benchmark_data_storage0111-ib.pocstor_35625_object7386/head//22 by client.940330.0:38694 2014-08-11 16:31:43.435033 2014-08-13 18:06:06.474247 7f5c973a57a0 20 read_log 192888'448 (192888'430) delete 69cf6226/benchmark_data_storage0111-ib.pocstor_35625_object7675/head//22 by client.940330.0:38983 2014-08-11 16:31:43.509036 2014-08-13 18:06:06.474260 7f5c973a57a0 20 read_log 192888'449 (192888'431) delete 87567a26/benchmark_data_storage0111-ib.pocstor_35625_object8402/head//22 by client.940330.0:39710 2014-08-11 16:31:43.690306 2014-08-13 18:06:06.474272 7f5c973a57a0 20 read_log 192888'450 (192888'432) delete 6a30c226/benchmark_data_storage0111-ib.pocstor_35625_object8972/head//22 by client.940330.0:40280 2014-08-11 16:31:43.834632 2014-08-13 18:06:06.474287 7f5c973a57a0 20 read_log 192888'451 (192888'433) delete 723e9a26/benchmark_data_storage0111-ib.pocstor_35625_object9822/head//22 by client.940330.0:41130 2014-08-11 16:31:44.064228 2014-08-13 18:06:06.474305 7f5c973a57a0 20 read_log 192888'452 (192888'434) delete 523a1226/benchmark_data_storage0111-ib.pocstor_35625_object12469/head//22 by client.940330.0:43777 2014-08-11 16:31:44.774837 2014-08-13 18:06:06.474318 7f5c973a57a0 20 read_log 192888'453 (192888'435) delete 6a83a26/benchmark_data_storage0111-ib.pocstor_35625_object15689/head//22 by client.940330.0:46997 2014-08-11 16:31:45.640074 2014-08-13 18:06:06.474330 7f5c973a57a0 20 read_log 192888'454 (192888'436) delete abe2d226/benchmark_data_storage0111-ib.pocstor_35625_object17639/head//22 by client.940330.0:48947 2014-08-11 16:31:46.173835 2014-08-13 18:06:06.474344 7f5c973a57a0 20 read_log 192888'455 (192888'437) delete 46e56226/benchmark_data_storage0111-ib.pocstor_35625_object21450/head//22 by client.940330.0:52758 2014-08-11 16:31:47.281494 2014-08-13 18:06:06.474357 7f5c973a57a0 20 read_log 192888'456 (192888'438) delete 71520226/benchmark_data_storage0111-ib.pocstor_35625_object22343/head//22 by client.940330.0:53651 2014-08-11 16:31:47.557965 2014-08-13 18:06:06.474379 7f5c973a57a0 20 read_log 192888'457 (192888'439) delete 67012a26/benchmark_data_storage0111-ib.pocstor_35625_object23662/head//22 by client.940330.0:54970 2014-08-11 16:31:47.940802 2014-08-13 18:06:06.474394 7f5c973a57a0 20 read_log 192888'458 (192888'440) delete da86a226/benchmark_data_storage0111-ib.pocstor_35625_object23987/head//22 by client.940330.0:55295 2014-08-11 16:31:48.034722 2014-08-13 18:06:06.474407 7f5c973a57a0 20 read_log 192888'459 (192888'441) delete 9f30c226/benchmark_data_storage0111-ib.pocstor_35625_object26445/head//22 by client.940330.0:57753 2014-08-11 16:31:48.776965 2014-08-13 18:06:06.474419 7f5c973a57a0 20 read_log 192888'460 (192888'442) delete b5366a26/benchmark_data_storage0111-ib.pocstor_35625_object26754/head//22 by client.940330.0:58062 2014-08-11 16:31:48.858663 2014-08-13 18:06:06.474442 7f5c973a57a0 20 read_log 192888'461 (192888'443) delete a849e226/benchmark_data_storage0111-ib.pocstor_35625_object30156/head//22 by client.940330.0:61464 2014-08-11 16:31:49.803069 2014-08-13 18:06:06.474455 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 9a5d0a26/benchmark_data_storage0111-ib.pocstor_64498_object12036/head//22 by client.1049635.0:12037 2014-08-12 17:17:54.507665 2014-08-13 18:06:06.474490 7f5c973a57a0 20 read_log 194215'463 (0'0) modify e05ca226/benchmark_data_storage0111-ib.pocstor_64498_object12225/head//22 by client.1049635.0:12226 2014-08-12 17:17:55.146239 2014-08-13 18:06:06.474507 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 9b348a26/benchmark_data_storage0111-ib.pocstor_64498_object16311/head//22 by client.1049635.0:16312 2014-08-12 17:18:08.737621 2014-08-13 18:06:06.474522 7f5c973a57a0 20 read_log 194215'465 (0'0) modify c255226/benchmark_data_storage0111-ib.pocstor_64498_object16715/head//22 by client.1049635.0:16716 2014-08-12 17:18:10.286659 2014-08-13 18:06:06.474534 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 5fa5ca26/benchmark_data_storage0111-ib.pocstor_64498_object18878/head//22 by client.1049635.0:18879 2014-08-12 17:18:18.470497 2014-08-13 18:06:06.474547 7f5c973a57a0 20 read_log 194215'467 (0'0) modify fa698226/benchmark_data_storage0111-ib.pocstor_64498_object21474/head//22 by client.1049635.0:21475 2014-08-12 17:18:28.360914 2014-08-13 18:06:06.474559 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 9000ea26/benchmark_data_storage0111-ib.pocstor_64498_object21572/head//22 by client.1049635.0:21573 2014-08-12 17:18:28.725271 2014-08-13 18:06:06.474580 7f5c973a57a0 20 read_log 194215'469 (0'0) modify d4445a26/benchmark_data_storage0111-ib.pocstor_64498_object26701/head//22 by client.1049635.0:26702 2014-08-12 17:18:48.035766 2014-08-13 18:06:06.474596 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 31ba5226/benchmark_data_storage0111-ib.pocstor_64498_object27404/head//22 by client.1049635.0:27405 2014-08-12 17:18:50.662091 2014-08-13 18:06:06.474608 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 79839226/benchmark_data_storage0111-ib.pocstor_64498_object31667/head//22 by client.1049635.0:31668 2014-08-12 17:19:07.282091 2014-08-13 18:06:06.474621 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 50bcd226/benchmark_data_storage0111-ib.pocstor_64498_object32530/head//22 by client.1049635.0:32531 2014-08-12 17:19:10.604750 2014-08-13 18:06:06.474633 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 2bcaea26/benchmark_data_storage0111-ib.pocstor_64498_object33220/head//22 by client.1049635.0:33221 2014-08-12 17:19:13.226539 2014-08-13 18:06:06.474649 7f5c973a57a0 20 read_log 194215'474 (194215'462) delete 9a5d0a26/benchmark_data_storage0111-ib.pocstor_64498_object12036/head//22 by client.1049635.0:46239 2014-08-12 17:19:21.910020 2014-08-13 18:06:06.474665 7f5c973a57a0 20 read_log 194215'475 (194215'463) delete e05ca226/benchmark_data_storage0111-ib.pocstor_64498_object12225/head//22 by client.1049635.0:46428 2014-08-12 17:19:21.967181 2014-08-13 18:06:06.474698 7f5c973a57a0 20 read_log 194215'476 (194215'464) delete 9b348a26/benchmark_data_storage0111-ib.pocstor_64498_object16311/head//22 by client.1049635.0:50514 2014-08-12 17:19:23.288652 2014-08-13 18:06:06.474728 7f5c973a57a0 20 read_log 194215'477 (194215'465) delete c255226/benchmark_data_storage0111-ib.pocstor_64498_object16715/head//22 by client.1049635.0:50918 2014-08-12 17:19:23.421406 2014-08-13 18:06:06.474742 7f5c973a57a0 20 read_log 194215'478 (194215'466) delete 5fa5ca26/benchmark_data_storage0111-ib.pocstor_64498_object18878/head//22 by client.1049635.0:53081 2014-08-12 17:19:24.123799 2014-08-13 18:06:06.474755 7f5c973a57a0 20 read_log 194215'479 (194215'467) delete fa698226/benchmark_data_storage0111-ib.pocstor_64498_object21474/head//22 by client.1049635.0:55677 2014-08-12 17:19:25.048645 2014-08-13 18:06:06.474768 7f5c973a57a0 20 read_log 194215'480 (194215'468) delete 9000ea26/benchmark_data_storage0111-ib.pocstor_64498_object21572/head//22 by client.1049635.0:55775 2014-08-12 17:19:25.079939 2014-08-13 18:06:06.474787 7f5c973a57a0 20 read_log 194215'481 (194215'469) delete d4445a26/benchmark_data_storage0111-ib.pocstor_64498_object26701/head//22 by client.1049635.0:60904 2014-08-12 17:19:26.890537 2014-08-13 18:06:06.474804 7f5c973a57a0 20 read_log 194215'482 (194215'470) delete 31ba5226/benchmark_data_storage0111-ib.pocstor_64498_object27404/head//22 by client.1049635.0:61607 2014-08-12 17:19:27.146544 2014-08-13 18:06:06.474834 7f5c973a57a0 20 read_log 194215'483 (194215'471) delete 79839226/benchmark_data_storage0111-ib.pocstor_64498_object31667/head//22 by client.1049635.0:65870 2014-08-12 17:19:28.627095 2014-08-13 18:06:06.474852 7f5c973a57a0 20 read_log 194215'484 (194215'472) delete 50bcd226/benchmark_data_storage0111-ib.pocstor_64498_object32530/head//22 by client.1049635.0:66733 2014-08-12 17:19:28.941097 2014-08-13 18:06:06.474865 7f5c973a57a0 20 read_log 194215'485 (194215'473) delete 2bcaea26/benchmark_data_storage0111-ib.pocstor_64498_object33220/head//22 by client.1049635.0:67423 2014-08-12 17:19:29.194183 2014-08-13 18:06:06.474966 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.474987 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=0 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.475003 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=0 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001903 0 0.000000 2014-08-13 18:06:06.475015 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=0 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.475023 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=0 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.475032 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] log((0'0,194215'485], crt=192888'461) 2014-08-13 18:06:06.475057 7f5c973a57a0 10 osd.4 194487 pgid 22.22c coll 22.22c_head 2014-08-13 18:06:06.475219 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.22c 2014-08-13 18:06:06.475229 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.475269 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.22c(unlocked)] enter Initial 2014-08-13 18:06:06.475281 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.22c(unlocked)] enter NotTrimming 2014-08-13 18:06:06.475430 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.475559 7f5c973a57a0 20 read_log 191648'4 (0'0) modify a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 by client.273158.0:85 2014-07-30 14:08:49.018175 2014-08-13 18:06:06.475581 7f5c973a57a0 20 read_log 191648'33 (0'0) modify 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 by client.273158.0:1707 2014-07-30 14:08:55.050437 2014-08-13 18:06:06.475594 7f5c973a57a0 20 read_log 191648'48 (0'0) modify cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 by client.273158.0:2886 2014-07-30 14:09:00.935167 2014-08-13 18:06:06.475606 7f5c973a57a0 20 read_log 191648'79 (0'0) modify 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 by client.273158.0:4481 2014-07-30 14:09:08.657039 2014-08-13 18:06:06.475619 7f5c973a57a0 20 read_log 191648'108 (0'0) modify a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 by client.273158.0:6329 2014-07-30 14:09:17.707193 2014-08-13 18:06:06.475631 7f5c973a57a0 20 read_log 191648'137 (0'0) modify 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 by client.273158.0:8601 2014-07-30 14:09:29.583263 2014-08-13 18:06:06.475643 7f5c973a57a0 20 read_log 191648'165 (0'0) modify 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 by client.273158.0:10170 2014-07-30 14:09:36.691027 2014-08-13 18:06:06.475656 7f5c973a57a0 20 read_log 191648'272 (0'0) modify 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 by client.273158.0:17240 2014-07-30 14:10:14.414754 2014-08-13 18:06:06.475668 7f5c973a57a0 20 read_log 191648'309 (0'0) modify 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 by client.273158.0:19165 2014-07-30 14:10:24.252127 2014-08-13 18:06:06.475680 7f5c973a57a0 20 read_log 191648'319 (0'0) modify f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 by client.273158.0:19544 2014-07-30 14:10:26.613669 2014-08-13 18:06:06.475692 7f5c973a57a0 20 read_log 191648'351 (0'0) modify e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 by client.273158.0:22218 2014-07-30 14:10:45.059339 2014-08-13 18:06:06.475704 7f5c973a57a0 20 read_log 191727'364 (0'0) modify 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 by client.275175.0:306 2014-07-30 14:18:34.607368 2014-08-13 18:06:06.475716 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 by client.275175.0:1488 2014-07-30 14:18:39.166845 2014-08-13 18:06:06.475728 7f5c973a57a0 20 read_log 191727'367 (0'0) modify 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 by client.275175.0:1567 2014-07-30 14:18:39.442377 2014-08-13 18:06:06.475748 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 by client.275175.0:2315 2014-07-30 14:18:41.922939 2014-08-13 18:06:06.475760 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 by client.275175.0:4220 2014-07-30 14:18:48.020490 2014-08-13 18:06:06.475777 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 by client.275175.0:4949 2014-07-30 14:18:50.201939 2014-08-13 18:06:06.475789 7f5c973a57a0 20 read_log 191727'381 (0'0) modify f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 by client.275175.0:6030 2014-07-30 14:18:53.332588 2014-08-13 18:06:06.475801 7f5c973a57a0 20 read_log 191727'382 (0'0) modify ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 by client.275175.0:6141 2014-07-30 14:18:53.697273 2014-08-13 18:06:06.475813 7f5c973a57a0 20 read_log 191727'386 (0'0) modify ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 by client.275175.0:8352 2014-07-30 14:19:00.504331 2014-08-13 18:06:06.475825 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 by client.275175.0:9585 2014-07-30 14:19:04.025852 2014-08-13 18:06:06.475838 7f5c973a57a0 20 read_log 191727'392 (0'0) modify f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 by client.275175.0:10497 2014-07-30 14:19:06.629527 2014-08-13 18:06:06.475850 7f5c973a57a0 20 read_log 191727'399 (0'0) modify fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 by client.275175.0:14386 2014-07-30 14:19:18.967524 2014-08-13 18:06:06.475862 7f5c973a57a0 20 read_log 191727'401 (0'0) modify e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 by client.275175.0:16209 2014-07-30 14:19:24.659592 2014-08-13 18:06:06.475874 7f5c973a57a0 20 read_log 191727'406 (0'0) modify fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 by client.275175.0:18817 2014-07-30 14:19:32.614990 2014-08-13 18:06:06.475886 7f5c973a57a0 20 read_log 191727'408 (0'0) modify df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 by client.275175.0:19175 2014-07-30 14:19:33.867449 2014-08-13 18:06:06.475898 7f5c973a57a0 20 read_log 191727'409 (0'0) modify ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 by client.275175.0:19483 2014-07-30 14:19:34.729754 2014-08-13 18:06:06.475910 7f5c973a57a0 20 read_log 191727'412 (0'0) modify a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 by client.275175.0:21428 2014-07-30 14:19:40.412254 2014-08-13 18:06:06.475924 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 by client.275175.0:23618 2014-07-30 14:19:47.757239 2014-08-13 18:06:06.475936 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 by client.275175.0:25895 2014-07-30 14:19:54.430752 2014-08-13 18:06:06.475948 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 by client.275175.0:26500 2014-07-30 14:19:56.358670 2014-08-13 18:06:06.475961 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 by client.275175.0:28003 2014-07-30 14:20:01.036910 2014-08-13 18:06:06.475973 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 by client.275175.0:31104 2014-07-30 14:20:10.277085 2014-08-13 18:06:06.475985 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 by client.275175.0:31578 2014-07-30 14:20:11.666639 2014-08-13 18:06:06.476004 7f5c973a57a0 20 read_log 191727'432 (0'0) modify a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 by client.275175.0:31976 2014-07-30 14:20:12.783822 2014-08-13 18:06:06.476019 7f5c973a57a0 20 read_log 191727'434 (0'0) modify d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 by client.275175.0:32956 2014-07-30 14:20:15.674789 2014-08-13 18:06:06.476031 7f5c973a57a0 20 read_log 191727'437 (0'0) modify 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 by client.275175.0:34706 2014-07-30 14:20:21.133299 2014-08-13 18:06:06.476046 7f5c973a57a0 20 read_log 191727'440 (0'0) modify 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 by client.275175.0:36163 2014-07-30 14:20:25.787147 2014-08-13 18:06:06.476060 7f5c973a57a0 20 read_log 192888'447 (0'0) modify f520ea2c/benchmark_data_storage0111-ib.pocstor_35625_object1093/head//22 by client.940330.0:1094 2014-08-11 16:29:45.140333 2014-08-13 18:06:06.476079 7f5c973a57a0 20 read_log 192888'448 (0'0) modify fd913a2c/benchmark_data_storage0111-ib.pocstor_35625_object6790/head//22 by client.940330.0:6791 2014-08-11 16:30:06.582473 2014-08-13 18:06:06.476093 7f5c973a57a0 20 read_log 192888'449 (0'0) modify c9c5b22c/benchmark_data_storage0111-ib.pocstor_35625_object7325/head//22 by client.940330.0:7326 2014-08-11 16:30:08.466079 2014-08-13 18:06:06.476106 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 92138a2c/benchmark_data_storage0111-ib.pocstor_35625_object15082/head//22 by client.940330.0:15083 2014-08-11 16:30:37.799336 2014-08-13 18:06:06.476120 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 2fb0fa2c/benchmark_data_storage0111-ib.pocstor_35625_object15095/head//22 by client.940330.0:15096 2014-08-11 16:30:37.846177 2014-08-13 18:06:06.476134 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 431f7a2c/benchmark_data_storage0111-ib.pocstor_35625_object18501/head//22 by client.940330.0:18502 2014-08-11 16:30:50.772548 2014-08-13 18:06:06.476156 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 1da4b22c/benchmark_data_storage0111-ib.pocstor_35625_object19175/head//22 by client.940330.0:19176 2014-08-11 16:30:53.317936 2014-08-13 18:06:06.476171 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 7ff722c/benchmark_data_storage0111-ib.pocstor_35625_object24393/head//22 by client.940330.0:24394 2014-08-11 16:31:13.578587 2014-08-13 18:06:06.476184 7f5c973a57a0 20 read_log 192888'455 (0'0) modify 8c32922c/benchmark_data_storage0111-ib.pocstor_35625_object27444/head//22 by client.940330.0:27445 2014-08-11 16:31:25.521409 2014-08-13 18:06:06.476196 7f5c973a57a0 20 read_log 192888'456 (0'0) modify f3f4da2c/benchmark_data_storage0111-ib.pocstor_35625_object27828/head//22 by client.940330.0:27829 2014-08-11 16:31:27.014716 2014-08-13 18:06:06.476209 7f5c973a57a0 20 read_log 192888'457 (192888'447) delete f520ea2c/benchmark_data_storage0111-ib.pocstor_35625_object1093/head//22 by client.940330.0:32401 2014-08-11 16:31:41.772162 2014-08-13 18:06:06.476222 7f5c973a57a0 20 read_log 192888'458 (192888'448) delete fd913a2c/benchmark_data_storage0111-ib.pocstor_35625_object6790/head//22 by client.940330.0:38098 2014-08-11 16:31:43.285386 2014-08-13 18:06:06.476234 7f5c973a57a0 20 read_log 192888'459 (192888'449) delete c9c5b22c/benchmark_data_storage0111-ib.pocstor_35625_object7325/head//22 by client.940330.0:38633 2014-08-11 16:31:43.419293 2014-08-13 18:06:06.476246 7f5c973a57a0 20 read_log 192888'460 (192888'450) delete 92138a2c/benchmark_data_storage0111-ib.pocstor_35625_object15082/head//22 by client.940330.0:46390 2014-08-11 16:31:45.478417 2014-08-13 18:06:06.476265 7f5c973a57a0 20 read_log 192888'461 (192888'451) delete 2fb0fa2c/benchmark_data_storage0111-ib.pocstor_35625_object15095/head//22 by client.940330.0:46403 2014-08-11 16:31:45.480921 2014-08-13 18:06:06.476290 7f5c973a57a0 20 read_log 192888'462 (192888'452) delete 431f7a2c/benchmark_data_storage0111-ib.pocstor_35625_object18501/head//22 by client.940330.0:49809 2014-08-11 16:31:46.447227 2014-08-13 18:06:06.476306 7f5c973a57a0 20 read_log 192888'463 (192888'453) delete 1da4b22c/benchmark_data_storage0111-ib.pocstor_35625_object19175/head//22 by client.940330.0:50483 2014-08-11 16:31:46.627118 2014-08-13 18:06:06.476318 7f5c973a57a0 20 read_log 192888'464 (192888'454) delete 7ff722c/benchmark_data_storage0111-ib.pocstor_35625_object24393/head//22 by client.940330.0:55701 2014-08-11 16:31:48.153730 2014-08-13 18:06:06.476331 7f5c973a57a0 20 read_log 192888'465 (192888'455) delete 8c32922c/benchmark_data_storage0111-ib.pocstor_35625_object27444/head//22 by client.940330.0:58752 2014-08-11 16:31:49.052237 2014-08-13 18:06:06.476343 7f5c973a57a0 20 read_log 192888'466 (192888'456) delete f3f4da2c/benchmark_data_storage0111-ib.pocstor_35625_object27828/head//22 by client.940330.0:59136 2014-08-11 16:31:49.159514 2014-08-13 18:06:06.476355 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 4999a22c/benchmark_data_storage0111-ib.pocstor_64498_object95/head//22 by client.1049635.0:96 2014-08-12 17:17:17.321755 2014-08-13 18:06:06.476368 7f5c973a57a0 20 read_log 194215'468 (0'0) modify ef8ca2c/benchmark_data_storage0111-ib.pocstor_64498_object522/head//22 by client.1049635.0:523 2014-08-12 17:17:18.801354 2014-08-13 18:06:06.476391 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 43617a2c/benchmark_data_storage0111-ib.pocstor_64498_object1409/head//22 by client.1049635.0:1410 2014-08-12 17:17:21.723544 2014-08-13 18:06:06.476407 7f5c973a57a0 20 read_log 194215'470 (0'0) modify cc3a322c/benchmark_data_storage0111-ib.pocstor_64498_object2728/head//22 by client.1049635.0:2729 2014-08-12 17:17:25.919402 2014-08-13 18:06:06.476435 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 9e98922c/benchmark_data_storage0111-ib.pocstor_64498_object3040/head//22 by client.1049635.0:3041 2014-08-12 17:17:26.956656 2014-08-13 18:06:06.476451 7f5c973a57a0 20 read_log 194215'472 (0'0) modify b7c4d22c/benchmark_data_storage0111-ib.pocstor_64498_object4936/head//22 by client.1049635.0:4937 2014-08-12 17:17:32.753191 2014-08-13 18:06:06.476463 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 10f1fa2c/benchmark_data_storage0111-ib.pocstor_64498_object6276/head//22 by client.1049635.0:6277 2014-08-12 17:17:36.922032 2014-08-13 18:06:06.476482 7f5c973a57a0 20 read_log 194215'474 (0'0) modify c5e61a2c/benchmark_data_storage0111-ib.pocstor_64498_object9119/head//22 by client.1049635.0:9120 2014-08-12 17:17:45.362286 2014-08-13 18:06:06.476501 7f5c973a57a0 20 read_log 194215'475 (0'0) modify cb8faa2c/benchmark_data_storage0111-ib.pocstor_64498_object13032/head//22 by client.1049635.0:13033 2014-08-12 17:17:57.488064 2014-08-13 18:06:06.476514 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 4e848a2c/benchmark_data_storage0111-ib.pocstor_64498_object16398/head//22 by client.1049635.0:16399 2014-08-12 17:18:09.077039 2014-08-13 18:06:06.476534 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 29e82a2c/benchmark_data_storage0111-ib.pocstor_64498_object17030/head//22 by client.1049635.0:17031 2014-08-12 17:18:11.475226 2014-08-13 18:06:06.476551 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 1571d22c/benchmark_data_storage0111-ib.pocstor_64498_object18729/head//22 by client.1049635.0:18730 2014-08-12 17:18:17.919097 2014-08-13 18:06:06.476564 7f5c973a57a0 20 read_log 194215'479 (0'0) modify b31e822c/benchmark_data_storage0111-ib.pocstor_64498_object19736/head//22 by client.1049635.0:19737 2014-08-12 17:18:21.791546 2014-08-13 18:06:06.476576 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 95c0922c/benchmark_data_storage0111-ib.pocstor_64498_object24070/head//22 by client.1049635.0:24071 2014-08-12 17:18:38.111832 2014-08-13 18:06:06.476597 7f5c973a57a0 20 read_log 194215'481 (0'0) modify aa10a2c/benchmark_data_storage0111-ib.pocstor_64498_object26853/head//22 by client.1049635.0:26854 2014-08-12 17:18:48.601610 2014-08-13 18:06:06.476612 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 1f99d22c/benchmark_data_storage0111-ib.pocstor_64498_object27334/head//22 by client.1049635.0:27335 2014-08-12 17:18:50.394231 2014-08-13 18:06:06.476625 7f5c973a57a0 20 read_log 194215'483 (0'0) modify e8a5d22c/benchmark_data_storage0111-ib.pocstor_64498_object27510/head//22 by client.1049635.0:27511 2014-08-12 17:18:51.058586 2014-08-13 18:06:06.476637 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 58d3c22c/benchmark_data_storage0111-ib.pocstor_64498_object28531/head//22 by client.1049635.0:28532 2014-08-12 17:18:54.859383 2014-08-13 18:06:06.476649 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 58916a2c/benchmark_data_storage0111-ib.pocstor_64498_object29137/head//22 by client.1049635.0:29138 2014-08-12 17:18:57.106440 2014-08-13 18:06:06.476661 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 395d922c/benchmark_data_storage0111-ib.pocstor_64498_object29811/head//22 by client.1049635.0:29812 2014-08-12 17:18:59.741211 2014-08-13 18:06:06.476674 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 5fbada2c/benchmark_data_storage0111-ib.pocstor_64498_object29889/head//22 by client.1049635.0:29890 2014-08-12 17:19:00.034149 2014-08-13 18:06:06.476686 7f5c973a57a0 20 read_log 194215'488 (0'0) modify e1b76a2c/benchmark_data_storage0111-ib.pocstor_64498_object30695/head//22 by client.1049635.0:30696 2014-08-12 17:19:03.269800 2014-08-13 18:06:06.476698 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 30bd7a2c/benchmark_data_storage0111-ib.pocstor_64498_object31147/head//22 by client.1049635.0:31148 2014-08-12 17:19:05.133313 2014-08-13 18:06:06.476727 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 8f37322c/benchmark_data_storage0111-ib.pocstor_64498_object31995/head//22 by client.1049635.0:31996 2014-08-12 17:19:08.554610 2014-08-13 18:06:06.476742 7f5c973a57a0 20 read_log 194215'491 (194215'467) delete 4999a22c/benchmark_data_storage0111-ib.pocstor_64498_object95/head//22 by client.1049635.0:34298 2014-08-12 17:19:18.012152 2014-08-13 18:06:06.476755 7f5c973a57a0 20 read_log 194215'492 (194215'468) delete ef8ca2c/benchmark_data_storage0111-ib.pocstor_64498_object522/head//22 by client.1049635.0:34725 2014-08-12 17:19:18.166130 2014-08-13 18:06:06.476768 7f5c973a57a0 20 read_log 194215'493 (194215'469) delete 43617a2c/benchmark_data_storage0111-ib.pocstor_64498_object1409/head//22 by client.1049635.0:35612 2014-08-12 17:19:18.454796 2014-08-13 18:06:06.476780 7f5c973a57a0 20 read_log 194215'494 (194215'470) delete cc3a322c/benchmark_data_storage0111-ib.pocstor_64498_object2728/head//22 by client.1049635.0:36931 2014-08-12 17:19:18.876406 2014-08-13 18:06:06.476793 7f5c973a57a0 20 read_log 194215'495 (194215'471) delete 9e98922c/benchmark_data_storage0111-ib.pocstor_64498_object3040/head//22 by client.1049635.0:37243 2014-08-12 17:19:18.980881 2014-08-13 18:06:06.476805 7f5c973a57a0 20 read_log 194215'496 (194215'472) delete b7c4d22c/benchmark_data_storage0111-ib.pocstor_64498_object4936/head//22 by client.1049635.0:39139 2014-08-12 17:19:19.582320 2014-08-13 18:06:06.476818 7f5c973a57a0 20 read_log 194215'497 (194215'473) delete 10f1fa2c/benchmark_data_storage0111-ib.pocstor_64498_object6276/head//22 by client.1049635.0:40479 2014-08-12 17:19:20.046112 2014-08-13 18:06:06.476830 7f5c973a57a0 20 read_log 194215'498 (194215'474) delete c5e61a2c/benchmark_data_storage0111-ib.pocstor_64498_object9119/head//22 by client.1049635.0:43322 2014-08-12 17:19:20.960325 2014-08-13 18:06:06.476842 7f5c973a57a0 20 read_log 194215'499 (194215'475) delete cb8faa2c/benchmark_data_storage0111-ib.pocstor_64498_object13032/head//22 by client.1049635.0:47235 2014-08-12 17:19:22.228760 2014-08-13 18:06:06.476854 7f5c973a57a0 20 read_log 194215'500 (194215'476) delete 4e848a2c/benchmark_data_storage0111-ib.pocstor_64498_object16398/head//22 by client.1049635.0:50601 2014-08-12 17:19:23.318639 2014-08-13 18:06:06.476874 7f5c973a57a0 20 read_log 194215'501 (194215'477) delete 29e82a2c/benchmark_data_storage0111-ib.pocstor_64498_object17030/head//22 by client.1049635.0:51233 2014-08-12 17:19:23.517296 2014-08-13 18:06:06.476889 7f5c973a57a0 20 read_log 194215'502 (194215'478) delete 1571d22c/benchmark_data_storage0111-ib.pocstor_64498_object18729/head//22 by client.1049635.0:52932 2014-08-12 17:19:24.073802 2014-08-13 18:06:06.476902 7f5c973a57a0 20 read_log 194215'503 (194215'479) delete b31e822c/benchmark_data_storage0111-ib.pocstor_64498_object19736/head//22 by client.1049635.0:53939 2014-08-12 17:19:24.426879 2014-08-13 18:06:06.476916 7f5c973a57a0 20 read_log 194215'504 (194215'480) delete 95c0922c/benchmark_data_storage0111-ib.pocstor_64498_object24070/head//22 by client.1049635.0:58273 2014-08-12 17:19:25.985682 2014-08-13 18:06:06.476930 7f5c973a57a0 20 read_log 194215'505 (194215'481) delete aa10a2c/benchmark_data_storage0111-ib.pocstor_64498_object26853/head//22 by client.1049635.0:61056 2014-08-12 17:19:26.946228 2014-08-13 18:06:06.476945 7f5c973a57a0 20 read_log 194215'506 (194215'482) delete 1f99d22c/benchmark_data_storage0111-ib.pocstor_64498_object27334/head//22 by client.1049635.0:61537 2014-08-12 17:19:27.119514 2014-08-13 18:06:06.476959 7f5c973a57a0 20 read_log 194215'507 (194215'483) delete e8a5d22c/benchmark_data_storage0111-ib.pocstor_64498_object27510/head//22 by client.1049635.0:61713 2014-08-12 17:19:27.183830 2014-08-13 18:06:06.476972 7f5c973a57a0 20 read_log 194215'508 (194215'484) delete 58d3c22c/benchmark_data_storage0111-ib.pocstor_64498_object28531/head//22 by client.1049635.0:62734 2014-08-12 17:19:27.534482 2014-08-13 18:06:06.476986 7f5c973a57a0 20 read_log 194215'509 (194215'485) delete 58916a2c/benchmark_data_storage0111-ib.pocstor_64498_object29137/head//22 by client.1049635.0:63340 2014-08-12 17:19:27.752110 2014-08-13 18:06:06.476998 7f5c973a57a0 20 read_log 194215'510 (194215'486) delete 395d922c/benchmark_data_storage0111-ib.pocstor_64498_object29811/head//22 by client.1049635.0:64014 2014-08-12 17:19:27.988983 2014-08-13 18:06:06.477033 7f5c973a57a0 20 read_log 194215'511 (194215'487) delete 5fbada2c/benchmark_data_storage0111-ib.pocstor_64498_object29889/head//22 by client.1049635.0:64092 2014-08-12 17:19:28.012836 2014-08-13 18:06:06.477045 7f5c973a57a0 20 read_log 194215'512 (194215'488) delete e1b76a2c/benchmark_data_storage0111-ib.pocstor_64498_object30695/head//22 by client.1049635.0:64898 2014-08-12 17:19:28.272136 2014-08-13 18:06:06.477059 7f5c973a57a0 20 read_log 194215'513 (194215'489) delete 30bd7a2c/benchmark_data_storage0111-ib.pocstor_64498_object31147/head//22 by client.1049635.0:65350 2014-08-12 17:19:28.431292 2014-08-13 18:06:06.477072 7f5c973a57a0 20 read_log 194215'514 (194215'490) delete 8f37322c/benchmark_data_storage0111-ib.pocstor_64498_object31995/head//22 by client.1049635.0:66198 2014-08-12 17:19:28.741494 2014-08-13 18:06:06.477167 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.477186 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=0 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.477202 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=0 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001933 0 0.000000 2014-08-13 18:06:06.477214 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=0 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.477222 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=0 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.477232 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'514], crt=191727'446) 2014-08-13 18:06:06.477242 7f5c973a57a0 10 osd.4 194487 pgid 22.28f coll 22.28f_head 2014-08-13 18:06:06.477414 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.28f 2014-08-13 18:06:06.477424 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.477460 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.28f(unlocked)] enter Initial 2014-08-13 18:06:06.477484 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.28f(unlocked)] enter NotTrimming 2014-08-13 18:06:06.477631 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.477737 7f5c973a57a0 20 read_log 191648'21 (0'0) modify fba01a8f/benchmark_data_storage0111-ib.pocstor_40745_object1856/head//22 by client.273158.0:1857 2014-07-30 14:08:55.607854 2014-08-13 18:06:06.477758 7f5c973a57a0 20 read_log 191648'69 (0'0) modify d62b3a8f/benchmark_data_storage0111-ib.pocstor_40745_object5137/head//22 by client.273158.0:5138 2014-07-30 14:09:11.692933 2014-08-13 18:06:06.477771 7f5c973a57a0 20 read_log 191648'82 (0'0) modify 4945728f/benchmark_data_storage0111-ib.pocstor_40745_object5835/head//22 by client.273158.0:5836 2014-07-30 14:09:15.068476 2014-08-13 18:06:06.477783 7f5c973a57a0 20 read_log 191648'102 (0'0) modify ffe228f/benchmark_data_storage0111-ib.pocstor_40745_object6850/head//22 by client.273158.0:6851 2014-07-30 14:09:20.504439 2014-08-13 18:06:06.477795 7f5c973a57a0 20 read_log 191648'131 (0'0) modify ae4e3a8f/benchmark_data_storage0111-ib.pocstor_40745_object8910/head//22 by client.273158.0:8911 2014-07-30 14:09:30.816666 2014-08-13 18:06:06.477807 7f5c973a57a0 20 read_log 191648'161 (0'0) modify 1719428f/benchmark_data_storage0111-ib.pocstor_40745_object10040/head//22 by client.273158.0:10041 2014-07-30 14:09:36.004066 2014-08-13 18:06:06.477820 7f5c973a57a0 20 read_log 191648'221 (0'0) modify f0c1528f/benchmark_data_storage0111-ib.pocstor_40745_object14371/head//22 by client.273158.0:14372 2014-07-30 14:09:59.558968 2014-08-13 18:06:06.477832 7f5c973a57a0 20 read_log 191648'251 (0'0) modify b2a9928f/benchmark_data_storage0111-ib.pocstor_40745_object16400/head//22 by client.273158.0:16401 2014-07-30 14:10:09.607719 2014-08-13 18:06:06.477844 7f5c973a57a0 20 read_log 191648'276 (0'0) modify 80fa3a8f/benchmark_data_storage0111-ib.pocstor_40745_object18322/head//22 by client.273158.0:18323 2014-07-30 14:10:19.764902 2014-08-13 18:06:06.477856 7f5c973a57a0 20 read_log 191648'294 (0'0) modify 1610628f/benchmark_data_storage0111-ib.pocstor_40745_object19372/head//22 by client.273158.0:19373 2014-07-30 14:10:25.733467 2014-08-13 18:06:06.477868 7f5c973a57a0 20 read_log 191648'303 (0'0) modify 9e0c828f/benchmark_data_storage0111-ib.pocstor_40745_object19886/head//22 by client.273158.0:19887 2014-07-30 14:10:30.869865 2014-08-13 18:06:06.477880 7f5c973a57a0 20 read_log 191648'309 (0'0) modify eb4b4a8f/benchmark_data_storage0111-ib.pocstor_40745_object20390/head//22 by client.273158.0:20391 2014-07-30 14:10:32.975475 2014-08-13 18:06:06.477892 7f5c973a57a0 20 read_log 191648'315 (0'0) modify c0ffc28f/benchmark_data_storage0111-ib.pocstor_40745_object20597/head//22 by client.273158.0:20598 2014-07-30 14:10:34.320948 2014-08-13 18:06:06.477905 7f5c973a57a0 20 read_log 191648'344 (0'0) modify 3d17228f/benchmark_data_storage0111-ib.pocstor_40745_object22319/head//22 by client.273158.0:22320 2014-07-30 14:10:46.009537 2014-08-13 18:06:06.477918 7f5c973a57a0 20 read_log 191727'353 (0'0) modify 711a828f/benchmark_data_storage0111-ib.pocstor_51285_object625/head//22 by client.275175.0:626 2014-07-30 14:18:35.904129 2014-08-13 18:06:06.477930 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 99411a8f/benchmark_data_storage0111-ib.pocstor_51285_object3094/head//22 by client.275175.0:3095 2014-07-30 14:18:44.522806 2014-08-13 18:06:06.477942 7f5c973a57a0 20 read_log 191727'362 (0'0) modify 814928f/benchmark_data_storage0111-ib.pocstor_51285_object3641/head//22 by client.275175.0:3642 2014-07-30 14:18:46.250602 2014-08-13 18:06:06.477964 7f5c973a57a0 20 read_log 191727'365 (0'0) modify 3c21328f/benchmark_data_storage0111-ib.pocstor_51285_object4804/head//22 by client.275175.0:4805 2014-07-30 14:18:49.770478 2014-08-13 18:06:06.477980 7f5c973a57a0 20 read_log 191727'370 (0'0) modify 7fb7828f/benchmark_data_storage0111-ib.pocstor_51285_object8195/head//22 by client.275175.0:8196 2014-07-30 14:19:00.066191 2014-08-13 18:06:06.477993 7f5c973a57a0 20 read_log 191727'376 (0'0) modify 4eea7a8f/benchmark_data_storage0111-ib.pocstor_51285_object12838/head//22 by client.275175.0:12839 2014-07-30 14:19:13.994371 2014-08-13 18:06:06.478005 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 2efd828f/benchmark_data_storage0111-ib.pocstor_51285_object14257/head//22 by client.275175.0:14258 2014-07-30 14:19:18.602821 2014-08-13 18:06:06.478017 7f5c973a57a0 20 read_log 191727'380 (0'0) modify b94d4a8f/benchmark_data_storage0111-ib.pocstor_51285_object14787/head//22 by client.275175.0:14788 2014-07-30 14:19:20.095525 2014-08-13 18:06:06.478029 7f5c973a57a0 20 read_log 191727'383 (0'0) modify 32dea8f/benchmark_data_storage0111-ib.pocstor_51285_object17769/head//22 by client.275175.0:17770 2014-07-30 14:19:29.308785 2014-08-13 18:06:06.478041 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 475db28f/benchmark_data_storage0111-ib.pocstor_51285_object18973/head//22 by client.275175.0:18974 2014-07-30 14:19:33.093278 2014-08-13 18:06:06.478053 7f5c973a57a0 20 read_log 191727'393 (0'0) modify 837f0a8f/benchmark_data_storage0111-ib.pocstor_51285_object25134/head//22 by client.275175.0:25135 2014-07-30 14:19:52.149190 2014-08-13 18:06:06.478065 7f5c973a57a0 20 read_log 191727'394 (0'0) modify bfcafa8f/benchmark_data_storage0111-ib.pocstor_51285_object25377/head//22 by client.275175.0:25378 2014-07-30 14:19:52.917477 2014-08-13 18:06:06.478077 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 1a3c5a8f/benchmark_data_storage0111-ib.pocstor_51285_object25503/head//22 by client.275175.0:25504 2014-07-30 14:19:53.270547 2014-08-13 18:06:06.478088 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 2e8ff28f/benchmark_data_storage0111-ib.pocstor_51285_object26529/head//22 by client.275175.0:26530 2014-07-30 14:19:56.444655 2014-08-13 18:06:06.478100 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 2224ca8f/benchmark_data_storage0111-ib.pocstor_51285_object27625/head//22 by client.275175.0:27626 2014-07-30 14:19:59.585256 2014-08-13 18:06:06.478113 7f5c973a57a0 20 read_log 191727'400 (0'0) modify 82876a8f/benchmark_data_storage0111-ib.pocstor_51285_object27747/head//22 by client.275175.0:27748 2014-07-30 14:19:59.977856 2014-08-13 18:06:06.478124 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 280128f/benchmark_data_storage0111-ib.pocstor_51285_object28968/head//22 by client.275175.0:28969 2014-07-30 14:20:03.799756 2014-08-13 18:06:06.478137 7f5c973a57a0 20 read_log 191727'406 (0'0) modify feb6ca8f/benchmark_data_storage0111-ib.pocstor_51285_object29765/head//22 by client.275175.0:29766 2014-07-30 14:20:06.258438 2014-08-13 18:06:06.478155 7f5c973a57a0 20 read_log 191727'409 (0'0) modify 78db928f/benchmark_data_storage0111-ib.pocstor_51285_object30073/head//22 by client.275175.0:30074 2014-07-30 14:20:07.154491 2014-08-13 18:06:06.478170 7f5c973a57a0 20 read_log 191727'410 (0'0) modify c1e0528f/benchmark_data_storage0111-ib.pocstor_51285_object30495/head//22 by client.275175.0:30496 2014-07-30 14:20:08.565892 2014-08-13 18:06:06.478182 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 4e63428f/benchmark_data_storage0111-ib.pocstor_51285_object32294/head//22 by client.275175.0:32295 2014-07-30 14:20:13.740559 2014-08-13 18:06:06.478194 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 8ec19a8f/benchmark_data_storage0111-ib.pocstor_51285_object33758/head//22 by client.275175.0:33759 2014-07-30 14:20:18.303809 2014-08-13 18:06:06.478208 7f5c973a57a0 20 read_log 191727'417 (0'0) modify d62dda8f/benchmark_data_storage0111-ib.pocstor_51285_object34064/head//22 by client.275175.0:34065 2014-07-30 14:20:19.175784 2014-08-13 18:06:06.478230 7f5c973a57a0 20 read_log 191727'419 (0'0) modify 5297ba8f/benchmark_data_storage0111-ib.pocstor_51285_object35422/head//22 by client.275175.0:35423 2014-07-30 14:20:23.224880 2014-08-13 18:06:06.478248 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 19f3928f/benchmark_data_storage0111-ib.pocstor_51285_object35711/head//22 by client.275175.0:35712 2014-07-30 14:20:24.184303 2014-08-13 18:06:06.478262 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 9922128f/benchmark_data_storage0111-ib.pocstor_51285_object37419/head//22 by client.275175.0:37420 2014-07-30 14:20:29.592777 2014-08-13 18:06:06.478275 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 3ef1628f/benchmark_data_storage0111-ib.pocstor_51285_object38330/head//22 by client.275175.0:38331 2014-07-30 14:20:32.185409 2014-08-13 18:06:06.478290 7f5c973a57a0 20 read_log 192674'430 (0'0) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4839 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478309 7f5c973a57a0 20 read_log 192674'431 (192674'430) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4840 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478321 7f5c973a57a0 20 read_log 192674'432 (192674'431) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4841 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478334 7f5c973a57a0 20 read_log 192674'433 (192674'432) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4842 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478346 7f5c973a57a0 20 read_log 192674'434 (192674'433) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4843 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478358 7f5c973a57a0 20 read_log 192674'435 (192674'434) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4844 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478370 7f5c973a57a0 20 read_log 192674'436 (192674'435) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4845 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478392 7f5c973a57a0 20 read_log 192674'437 (192674'436) modify 3ab7ca8f/rb.0.43b76.238e1f29.0000000001ff/head//22 by client.279772.1:4846 2014-07-31 18:03:46.802000 2014-08-13 18:06:06.478408 7f5c973a57a0 20 read_log 192888'438 (0'0) modify cfa9ea8f/benchmark_data_storage0111-ib.pocstor_35625_object3856/head//22 by client.940330.0:3857 2014-08-11 16:29:55.804830 2014-08-13 18:06:06.478421 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 175dd28f/benchmark_data_storage0111-ib.pocstor_35625_object3875/head//22 by client.940330.0:3876 2014-08-11 16:29:55.874346 2014-08-13 18:06:06.478441 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 76af428f/benchmark_data_storage0111-ib.pocstor_35625_object10037/head//22 by client.940330.0:10038 2014-08-11 16:30:18.036344 2014-08-13 18:06:06.478457 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 1e90c28f/benchmark_data_storage0111-ib.pocstor_35625_object11027/head//22 by client.940330.0:11028 2014-08-11 16:30:21.904001 2014-08-13 18:06:06.478470 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 3795328f/benchmark_data_storage0111-ib.pocstor_35625_object11154/head//22 by client.940330.0:11155 2014-08-11 16:30:22.405929 2014-08-13 18:06:06.478495 7f5c973a57a0 20 read_log 192888'443 (0'0) modify acd28f/benchmark_data_storage0111-ib.pocstor_35625_object14560/head//22 by client.940330.0:14561 2014-08-11 16:30:35.855325 2014-08-13 18:06:06.478510 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 1c4f8a8f/benchmark_data_storage0111-ib.pocstor_35625_object19867/head//22 by client.940330.0:19868 2014-08-11 16:30:56.328983 2014-08-13 18:06:06.478523 7f5c973a57a0 20 read_log 192888'445 (0'0) modify e927428f/benchmark_data_storage0111-ib.pocstor_35625_object21493/head//22 by client.940330.0:21494 2014-08-11 16:31:02.652402 2014-08-13 18:06:06.478551 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 9ea78a8f/benchmark_data_storage0111-ib.pocstor_35625_object25950/head//22 by client.940330.0:25951 2014-08-11 16:31:19.598189 2014-08-13 18:06:06.478566 7f5c973a57a0 20 read_log 192888'447 (0'0) modify 5e8fca8f/benchmark_data_storage0111-ib.pocstor_35625_object26627/head//22 by client.940330.0:26628 2014-08-11 16:31:22.352056 2014-08-13 18:06:06.478579 7f5c973a57a0 20 read_log 192888'448 (0'0) modify ef611a8f/benchmark_data_storage0111-ib.pocstor_35625_object28786/head//22 by client.940330.0:28787 2014-08-11 16:31:30.767306 2014-08-13 18:06:06.478591 7f5c973a57a0 20 read_log 192888'449 (0'0) modify affc4a8f/benchmark_data_storage0111-ib.pocstor_35625_object30183/head//22 by client.940330.0:30184 2014-08-11 16:31:36.145393 2014-08-13 18:06:06.478604 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 8bf19a8f/benchmark_data_storage0111-ib.pocstor_35625_object30875/head//22 by client.940330.0:30876 2014-08-11 16:31:38.747546 2014-08-13 18:06:06.478636 7f5c973a57a0 20 read_log 192888'451 (192888'438) delete cfa9ea8f/benchmark_data_storage0111-ib.pocstor_35625_object3856/head//22 by client.940330.0:35164 2014-08-11 16:31:42.489926 2014-08-13 18:06:06.478649 7f5c973a57a0 20 read_log 192888'452 (192888'439) delete 175dd28f/benchmark_data_storage0111-ib.pocstor_35625_object3875/head//22 by client.940330.0:35183 2014-08-11 16:31:42.495559 2014-08-13 18:06:06.478676 7f5c973a57a0 20 read_log 192888'453 (192888'440) delete 76af428f/benchmark_data_storage0111-ib.pocstor_35625_object10037/head//22 by client.940330.0:41345 2014-08-11 16:31:44.123624 2014-08-13 18:06:06.478725 7f5c973a57a0 20 read_log 192888'454 (192888'441) delete 1e90c28f/benchmark_data_storage0111-ib.pocstor_35625_object11027/head//22 by client.940330.0:42335 2014-08-11 16:31:44.406913 2014-08-13 18:06:06.478745 7f5c973a57a0 20 read_log 192888'455 (192888'442) delete 3795328f/benchmark_data_storage0111-ib.pocstor_35625_object11154/head//22 by client.940330.0:42462 2014-08-11 16:31:44.443937 2014-08-13 18:06:06.478760 7f5c973a57a0 20 read_log 192888'456 (192888'443) delete acd28f/benchmark_data_storage0111-ib.pocstor_35625_object14560/head//22 by client.940330.0:45868 2014-08-11 16:31:45.335000 2014-08-13 18:06:06.478773 7f5c973a57a0 20 read_log 192888'457 (192888'444) delete 1c4f8a8f/benchmark_data_storage0111-ib.pocstor_35625_object19867/head//22 by client.940330.0:51175 2014-08-11 16:31:46.830389 2014-08-13 18:06:06.478785 7f5c973a57a0 20 read_log 192888'458 (192888'445) delete e927428f/benchmark_data_storage0111-ib.pocstor_35625_object21493/head//22 by client.940330.0:52801 2014-08-11 16:31:47.293627 2014-08-13 18:06:06.478809 7f5c973a57a0 20 read_log 192888'459 (192888'446) delete 9ea78a8f/benchmark_data_storage0111-ib.pocstor_35625_object25950/head//22 by client.940330.0:57258 2014-08-11 16:31:48.633681 2014-08-13 18:06:06.478830 7f5c973a57a0 20 read_log 192888'460 (192888'447) delete 5e8fca8f/benchmark_data_storage0111-ib.pocstor_35625_object26627/head//22 by client.940330.0:57935 2014-08-11 16:31:48.824593 2014-08-13 18:06:06.478854 7f5c973a57a0 20 read_log 192888'461 (192888'448) delete ef611a8f/benchmark_data_storage0111-ib.pocstor_35625_object28786/head//22 by client.940330.0:60094 2014-08-11 16:31:49.414662 2014-08-13 18:06:06.478871 7f5c973a57a0 20 read_log 192888'462 (192888'449) delete affc4a8f/benchmark_data_storage0111-ib.pocstor_35625_object30183/head//22 by client.940330.0:61491 2014-08-11 16:31:49.810039 2014-08-13 18:06:06.478884 7f5c973a57a0 20 read_log 192888'463 (192888'450) delete 8bf19a8f/benchmark_data_storage0111-ib.pocstor_35625_object30875/head//22 by client.940330.0:62183 2014-08-11 16:31:50.015629 2014-08-13 18:06:06.478898 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 9da8428f/benchmark_data_storage0111-ib.pocstor_64498_object51/head//22 by client.1049635.0:52 2014-08-12 17:17:17.146640 2014-08-13 18:06:06.478940 7f5c973a57a0 20 read_log 194215'465 (0'0) modify ea24828f/benchmark_data_storage0111-ib.pocstor_64498_object190/head//22 by client.1049635.0:191 2014-08-12 17:17:17.625114 2014-08-13 18:06:06.478969 7f5c973a57a0 20 read_log 194215'466 (0'0) modify c8af3a8f/benchmark_data_storage0111-ib.pocstor_64498_object1053/head//22 by client.1049635.0:1054 2014-08-12 17:17:20.586727 2014-08-13 18:06:06.478986 7f5c973a57a0 20 read_log 194215'467 (0'0) modify ba25f28f/benchmark_data_storage0111-ib.pocstor_64498_object4858/head//22 by client.1049635.0:4859 2014-08-12 17:17:32.498633 2014-08-13 18:06:06.479002 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 2e05f28f/benchmark_data_storage0111-ib.pocstor_64498_object5728/head//22 by client.1049635.0:5729 2014-08-12 17:17:35.278269 2014-08-13 18:06:06.479014 7f5c973a57a0 20 read_log 194215'469 (0'0) modify a29c1a8f/benchmark_data_storage0111-ib.pocstor_64498_object8190/head//22 by client.1049635.0:8191 2014-08-12 17:17:42.665876 2014-08-13 18:06:06.479030 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 8186028f/benchmark_data_storage0111-ib.pocstor_64498_object8787/head//22 by client.1049635.0:8788 2014-08-12 17:17:44.419176 2014-08-13 18:06:06.479055 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 18b5a8f/benchmark_data_storage0111-ib.pocstor_64498_object10205/head//22 by client.1049635.0:10206 2014-08-12 17:17:48.511178 2014-08-13 18:06:06.479068 7f5c973a57a0 20 read_log 194215'472 (0'0) modify ea7fca8f/benchmark_data_storage0111-ib.pocstor_64498_object12352/head//22 by client.1049635.0:12353 2014-08-12 17:17:55.491521 2014-08-13 18:06:06.479087 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 377d8a8f/benchmark_data_storage0111-ib.pocstor_64498_object13263/head//22 by client.1049635.0:13264 2014-08-12 17:17:58.202154 2014-08-13 18:06:06.479101 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 88536a8f/benchmark_data_storage0111-ib.pocstor_64498_object14782/head//22 by client.1049635.0:14783 2014-08-12 17:18:03.266048 2014-08-13 18:06:06.479114 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 418a6a8f/benchmark_data_storage0111-ib.pocstor_64498_object22899/head//22 by client.1049635.0:22900 2014-08-12 17:18:33.759711 2014-08-13 18:06:06.479136 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 49bb6a8f/benchmark_data_storage0111-ib.pocstor_64498_object24885/head//22 by client.1049635.0:24886 2014-08-12 17:18:41.233275 2014-08-13 18:06:06.479167 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 3c022a8f/benchmark_data_storage0111-ib.pocstor_64498_object24891/head//22 by client.1049635.0:24892 2014-08-12 17:18:41.264968 2014-08-13 18:06:06.479180 7f5c973a57a0 20 read_log 194215'478 (0'0) modify fdc9428f/benchmark_data_storage0111-ib.pocstor_64498_object25881/head//22 by client.1049635.0:25882 2014-08-12 17:18:44.981241 2014-08-13 18:06:06.479193 7f5c973a57a0 20 read_log 194215'479 (0'0) modify e988e28f/benchmark_data_storage0111-ib.pocstor_64498_object33720/head//22 by client.1049635.0:33721 2014-08-12 17:19:15.118184 2014-08-13 18:06:06.479209 7f5c973a57a0 20 read_log 194215'480 (194215'464) delete 9da8428f/benchmark_data_storage0111-ib.pocstor_64498_object51/head//22 by client.1049635.0:34254 2014-08-12 17:19:17.996438 2014-08-13 18:06:06.479222 7f5c973a57a0 20 read_log 194215'481 (194215'465) delete ea24828f/benchmark_data_storage0111-ib.pocstor_64498_object190/head//22 by client.1049635.0:34393 2014-08-12 17:19:18.048768 2014-08-13 18:06:06.479237 7f5c973a57a0 20 read_log 194215'482 (194215'466) delete c8af3a8f/benchmark_data_storage0111-ib.pocstor_64498_object1053/head//22 by client.1049635.0:35256 2014-08-12 17:19:18.338035 2014-08-13 18:06:06.479250 7f5c973a57a0 20 read_log 194215'483 (194215'467) delete ba25f28f/benchmark_data_storage0111-ib.pocstor_64498_object4858/head//22 by client.1049635.0:39061 2014-08-12 17:19:19.558205 2014-08-13 18:06:06.479262 7f5c973a57a0 20 read_log 194215'484 (194215'468) delete 2e05f28f/benchmark_data_storage0111-ib.pocstor_64498_object5728/head//22 by client.1049635.0:39931 2014-08-12 17:19:19.864606 2014-08-13 18:06:06.479283 7f5c973a57a0 20 read_log 194215'485 (194215'469) delete a29c1a8f/benchmark_data_storage0111-ib.pocstor_64498_object8190/head//22 by client.1049635.0:42393 2014-08-12 17:19:20.664573 2014-08-13 18:06:06.479299 7f5c973a57a0 20 read_log 194215'486 (194215'470) delete 8186028f/benchmark_data_storage0111-ib.pocstor_64498_object8787/head//22 by client.1049635.0:42990 2014-08-12 17:19:20.857336 2014-08-13 18:06:06.479332 7f5c973a57a0 20 read_log 194215'487 (194215'471) delete 18b5a8f/benchmark_data_storage0111-ib.pocstor_64498_object10205/head//22 by client.1049635.0:44408 2014-08-12 17:19:21.329823 2014-08-13 18:06:06.479349 7f5c973a57a0 20 read_log 194215'488 (194215'472) delete ea7fca8f/benchmark_data_storage0111-ib.pocstor_64498_object12352/head//22 by client.1049635.0:46555 2014-08-12 17:19:22.008293 2014-08-13 18:06:06.479370 7f5c973a57a0 20 read_log 194215'489 (194215'473) delete 377d8a8f/benchmark_data_storage0111-ib.pocstor_64498_object13263/head//22 by client.1049635.0:47466 2014-08-12 17:19:22.310773 2014-08-13 18:06:06.479386 7f5c973a57a0 20 read_log 194215'490 (194215'474) delete 88536a8f/benchmark_data_storage0111-ib.pocstor_64498_object14782/head//22 by client.1049635.0:48985 2014-08-12 17:19:22.808377 2014-08-13 18:06:06.479404 7f5c973a57a0 20 read_log 194215'491 (194215'475) delete 418a6a8f/benchmark_data_storage0111-ib.pocstor_64498_object22899/head//22 by client.1049635.0:57102 2014-08-12 17:19:25.579191 2014-08-13 18:06:06.479427 7f5c973a57a0 20 read_log 194215'492 (194215'476) delete 49bb6a8f/benchmark_data_storage0111-ib.pocstor_64498_object24885/head//22 by client.1049635.0:59088 2014-08-12 17:19:26.284165 2014-08-13 18:06:06.479440 7f5c973a57a0 20 read_log 194215'493 (194215'477) delete 3c022a8f/benchmark_data_storage0111-ib.pocstor_64498_object24891/head//22 by client.1049635.0:59094 2014-08-12 17:19:26.285426 2014-08-13 18:06:06.479453 7f5c973a57a0 20 read_log 194215'494 (194215'478) delete fdc9428f/benchmark_data_storage0111-ib.pocstor_64498_object25881/head//22 by client.1049635.0:60084 2014-08-12 17:19:26.620124 2014-08-13 18:06:06.479475 7f5c973a57a0 20 read_log 194215'495 (194215'479) delete e988e28f/benchmark_data_storage0111-ib.pocstor_64498_object33720/head//22 by client.1049635.0:67923 2014-08-12 17:19:29.371767 2014-08-13 18:06:06.479575 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.479594 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=0 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.479609 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=0 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Initial 0.002149 0 0.000000 2014-08-13 18:06:06.479621 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=0 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.479630 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=0 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.479639 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] log((0'0,194215'495], crt=192888'463) 2014-08-13 18:06:06.479649 7f5c973a57a0 10 osd.4 194487 pgid 22.29a coll 22.29a_head 2014-08-13 18:06:06.479818 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.29a 2014-08-13 18:06:06.479828 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.479854 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.29a(unlocked)] enter Initial 2014-08-13 18:06:06.479865 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.29a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.479998 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.480109 7f5c973a57a0 20 read_log 191648'13 (0'0) modify 689fa29a/benchmark_data_storage0111-ib.pocstor_40745_object537/head//22 by client.273158.0:538 2014-07-30 14:08:50.784943 2014-08-13 18:06:06.480129 7f5c973a57a0 20 read_log 191648'41 (0'0) modify 90b5ca9a/benchmark_data_storage0111-ib.pocstor_40745_object2071/head//22 by client.273158.0:2072 2014-07-30 14:08:56.563366 2014-08-13 18:06:06.480142 7f5c973a57a0 20 read_log 191648'151 (0'0) modify 70ae429a/benchmark_data_storage0111-ib.pocstor_40745_object9938/head//22 by client.273158.0:9939 2014-07-30 14:09:35.518262 2014-08-13 18:06:06.480163 7f5c973a57a0 20 read_log 191648'152 (0'0) modify 40dd329a/benchmark_data_storage0111-ib.pocstor_40745_object9953/head//22 by client.273158.0:9954 2014-07-30 14:09:35.613585 2014-08-13 18:06:06.480176 7f5c973a57a0 20 read_log 191648'233 (0'0) modify 7023129a/benchmark_data_storage0111-ib.pocstor_40745_object14605/head//22 by client.273158.0:14606 2014-07-30 14:10:00.612468 2014-08-13 18:06:06.480189 7f5c973a57a0 20 read_log 191648'237 (0'0) modify 334a429a/benchmark_data_storage0111-ib.pocstor_40745_object14901/head//22 by client.273158.0:14902 2014-07-30 14:10:02.313537 2014-08-13 18:06:06.480201 7f5c973a57a0 20 read_log 191648'297 (0'0) modify 68297a9a/benchmark_data_storage0111-ib.pocstor_40745_object19664/head//22 by client.273158.0:19665 2014-07-30 14:10:27.682482 2014-08-13 18:06:06.480220 7f5c973a57a0 20 read_log 191727'362 (0'0) modify 88fe1a9a/benchmark_data_storage0111-ib.pocstor_51285_object3877/head//22 by client.275175.0:3878 2014-07-30 14:18:46.978801 2014-08-13 18:06:06.480240 7f5c973a57a0 20 read_log 191727'366 (0'0) modify ec98fa9a/benchmark_data_storage0111-ib.pocstor_51285_object6988/head//22 by client.275175.0:6989 2014-07-30 14:18:56.685492 2014-08-13 18:06:06.480255 7f5c973a57a0 20 read_log 191727'369 (0'0) modify ae04329a/benchmark_data_storage0111-ib.pocstor_51285_object8973/head//22 by client.275175.0:8974 2014-07-30 14:19:02.272865 2014-08-13 18:06:06.480267 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 8b24829a/benchmark_data_storage0111-ib.pocstor_51285_object10296/head//22 by client.275175.0:10297 2014-07-30 14:19:06.053286 2014-08-13 18:06:06.480283 7f5c973a57a0 20 read_log 191727'374 (0'0) modify 89c9a9a/benchmark_data_storage0111-ib.pocstor_51285_object10997/head//22 by client.275175.0:10998 2014-07-30 14:19:08.036743 2014-08-13 18:06:06.480298 7f5c973a57a0 20 read_log 191727'377 (0'0) modify cc2a0a9a/benchmark_data_storage0111-ib.pocstor_51285_object12431/head//22 by client.275175.0:12432 2014-07-30 14:19:12.847221 2014-08-13 18:06:06.480310 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 292e629a/benchmark_data_storage0111-ib.pocstor_51285_object13162/head//22 by client.275175.0:13163 2014-07-30 14:19:15.151537 2014-08-13 18:06:06.480323 7f5c973a57a0 20 read_log 191727'380 (0'0) modify c022e29a/benchmark_data_storage0111-ib.pocstor_51285_object15276/head//22 by client.275175.0:15277 2014-07-30 14:19:21.586933 2014-08-13 18:06:06.480335 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 9d99aa9a/benchmark_data_storage0111-ib.pocstor_51285_object16536/head//22 by client.275175.0:16537 2014-07-30 14:19:25.800780 2014-08-13 18:06:06.480361 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 5415fa9a/benchmark_data_storage0111-ib.pocstor_51285_object16853/head//22 by client.275175.0:16854 2014-07-30 14:19:26.743260 2014-08-13 18:06:06.480377 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 881129a/benchmark_data_storage0111-ib.pocstor_51285_object16957/head//22 by client.275175.0:16958 2014-07-30 14:19:27.036928 2014-08-13 18:06:06.480389 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 880a3a9a/benchmark_data_storage0111-ib.pocstor_51285_object17909/head//22 by client.275175.0:17910 2014-07-30 14:19:29.702951 2014-08-13 18:06:06.480416 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 9210f29a/benchmark_data_storage0111-ib.pocstor_51285_object19452/head//22 by client.275175.0:19453 2014-07-30 14:19:34.648687 2014-08-13 18:06:06.480429 7f5c973a57a0 20 read_log 191727'393 (0'0) modify e6ddaa9a/benchmark_data_storage0111-ib.pocstor_51285_object22035/head//22 by client.275175.0:22036 2014-07-30 14:19:42.563224 2014-08-13 18:06:06.480441 7f5c973a57a0 20 read_log 191727'396 (0'0) modify ec1ce29a/benchmark_data_storage0111-ib.pocstor_51285_object22784/head//22 by client.275175.0:22785 2014-07-30 14:19:45.380009 2014-08-13 18:06:06.480485 7f5c973a57a0 20 read_log 191727'399 (0'0) modify dc88029a/benchmark_data_storage0111-ib.pocstor_51285_object23752/head//22 by client.275175.0:23753 2014-07-30 14:19:48.135751 2014-08-13 18:06:06.480505 7f5c973a57a0 20 read_log 191727'402 (0'0) modify f36fba9a/benchmark_data_storage0111-ib.pocstor_51285_object25553/head//22 by client.275175.0:25554 2014-07-30 14:19:53.409010 2014-08-13 18:06:06.480517 7f5c973a57a0 20 read_log 191727'404 (0'0) modify c82e4a9a/benchmark_data_storage0111-ib.pocstor_51285_object26110/head//22 by client.275175.0:26111 2014-07-30 14:19:55.176914 2014-08-13 18:06:06.480530 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 28c9ca9a/benchmark_data_storage0111-ib.pocstor_51285_object27944/head//22 by client.275175.0:27945 2014-07-30 14:20:00.873965 2014-08-13 18:06:06.480545 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 747a029a/benchmark_data_storage0111-ib.pocstor_51285_object30016/head//22 by client.275175.0:30017 2014-07-30 14:20:06.996280 2014-08-13 18:06:06.480567 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 5aa5829a/benchmark_data_storage0111-ib.pocstor_51285_object31438/head//22 by client.275175.0:31439 2014-07-30 14:20:11.220391 2014-08-13 18:06:06.480581 7f5c973a57a0 20 read_log 191727'424 (0'0) modify d540329a/benchmark_data_storage0111-ib.pocstor_51285_object34231/head//22 by client.275175.0:34232 2014-07-30 14:20:19.714751 2014-08-13 18:06:06.480594 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 82820a9a/benchmark_data_storage0111-ib.pocstor_51285_object34369/head//22 by client.275175.0:34370 2014-07-30 14:20:20.154262 2014-08-13 18:06:06.480606 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 5812ea9a/benchmark_data_storage0111-ib.pocstor_51285_object36541/head//22 by client.275175.0:36542 2014-07-30 14:20:26.976725 2014-08-13 18:06:06.480627 7f5c973a57a0 20 read_log 192888'435 (0'0) modify a6d1629a/benchmark_data_storage0111-ib.pocstor_35625_object3338/head//22 by client.940330.0:3339 2014-08-11 16:29:53.880535 2014-08-13 18:06:06.480643 7f5c973a57a0 20 read_log 192888'436 (0'0) modify bf9f929a/benchmark_data_storage0111-ib.pocstor_35625_object5242/head//22 by client.940330.0:5243 2014-08-11 16:30:00.885204 2014-08-13 18:06:06.480656 7f5c973a57a0 20 read_log 192888'437 (0'0) modify ee73329a/benchmark_data_storage0111-ib.pocstor_35625_object7331/head//22 by client.940330.0:7332 2014-08-11 16:30:08.485592 2014-08-13 18:06:06.480668 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 5d63da9a/benchmark_data_storage0111-ib.pocstor_35625_object9065/head//22 by client.940330.0:9066 2014-08-11 16:30:14.560038 2014-08-13 18:06:06.480681 7f5c973a57a0 20 read_log 192888'439 (0'0) modify ff775a9a/benchmark_data_storage0111-ib.pocstor_35625_object12302/head//22 by client.940330.0:12303 2014-08-11 16:30:27.141225 2014-08-13 18:06:06.480693 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 18ffaa9a/benchmark_data_storage0111-ib.pocstor_35625_object13517/head//22 by client.940330.0:13518 2014-08-11 16:30:31.852520 2014-08-13 18:06:06.480705 7f5c973a57a0 20 read_log 192888'441 (0'0) modify b04f629a/benchmark_data_storage0111-ib.pocstor_35625_object14185/head//22 by client.940330.0:14186 2014-08-11 16:30:34.442736 2014-08-13 18:06:06.480717 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 15ac1a9a/benchmark_data_storage0111-ib.pocstor_35625_object18959/head//22 by client.940330.0:18960 2014-08-11 16:30:52.518495 2014-08-13 18:06:06.480729 7f5c973a57a0 20 read_log 192888'443 (0'0) modify d285829a/benchmark_data_storage0111-ib.pocstor_35625_object20871/head//22 by client.940330.0:20872 2014-08-11 16:31:00.213662 2014-08-13 18:06:06.480741 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 8f1e0a9a/benchmark_data_storage0111-ib.pocstor_35625_object20965/head//22 by client.940330.0:20966 2014-08-11 16:31:00.574395 2014-08-13 18:06:06.480754 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 4d7c8a9a/benchmark_data_storage0111-ib.pocstor_35625_object28021/head//22 by client.940330.0:28022 2014-08-11 16:31:27.810272 2014-08-13 18:06:06.480766 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 94293a9a/benchmark_data_storage0111-ib.pocstor_35625_object31158/head//22 by client.940330.0:31159 2014-08-11 16:31:39.815238 2014-08-13 18:06:06.480778 7f5c973a57a0 20 read_log 192888'447 (192888'435) delete a6d1629a/benchmark_data_storage0111-ib.pocstor_35625_object3338/head//22 by client.940330.0:34646 2014-08-11 16:31:42.345397 2014-08-13 18:06:06.480790 7f5c973a57a0 20 read_log 192888'448 (192888'436) delete bf9f929a/benchmark_data_storage0111-ib.pocstor_35625_object5242/head//22 by client.940330.0:36550 2014-08-11 16:31:42.878912 2014-08-13 18:06:06.480802 7f5c973a57a0 20 read_log 192888'449 (192888'437) delete ee73329a/benchmark_data_storage0111-ib.pocstor_35625_object7331/head//22 by client.940330.0:38639 2014-08-11 16:31:43.421986 2014-08-13 18:06:06.480824 7f5c973a57a0 20 read_log 192888'450 (192888'438) delete 5d63da9a/benchmark_data_storage0111-ib.pocstor_35625_object9065/head//22 by client.940330.0:40373 2014-08-11 16:31:43.858090 2014-08-13 18:06:06.480840 7f5c973a57a0 20 read_log 192888'451 (192888'439) delete ff775a9a/benchmark_data_storage0111-ib.pocstor_35625_object12302/head//22 by client.940330.0:43610 2014-08-11 16:31:44.733705 2014-08-13 18:06:06.480870 7f5c973a57a0 20 read_log 192888'452 (192888'440) delete 18ffaa9a/benchmark_data_storage0111-ib.pocstor_35625_object13517/head//22 by client.940330.0:44825 2014-08-11 16:31:45.057974 2014-08-13 18:06:06.480886 7f5c973a57a0 20 read_log 192888'453 (192888'441) delete b04f629a/benchmark_data_storage0111-ib.pocstor_35625_object14185/head//22 by client.940330.0:45493 2014-08-11 16:31:45.230848 2014-08-13 18:06:06.480898 7f5c973a57a0 20 read_log 192888'454 (192888'442) delete 15ac1a9a/benchmark_data_storage0111-ib.pocstor_35625_object18959/head//22 by client.940330.0:50267 2014-08-11 16:31:46.570405 2014-08-13 18:06:06.480910 7f5c973a57a0 20 read_log 192888'455 (192888'443) delete d285829a/benchmark_data_storage0111-ib.pocstor_35625_object20871/head//22 by client.940330.0:52179 2014-08-11 16:31:47.126257 2014-08-13 18:06:06.480923 7f5c973a57a0 20 read_log 192888'456 (192888'444) delete 8f1e0a9a/benchmark_data_storage0111-ib.pocstor_35625_object20965/head//22 by client.940330.0:52273 2014-08-11 16:31:47.152356 2014-08-13 18:06:06.480935 7f5c973a57a0 20 read_log 192888'457 (192888'445) delete 4d7c8a9a/benchmark_data_storage0111-ib.pocstor_35625_object28021/head//22 by client.940330.0:59329 2014-08-11 16:31:49.211905 2014-08-13 18:06:06.480947 7f5c973a57a0 20 read_log 192888'458 (192888'446) delete 94293a9a/benchmark_data_storage0111-ib.pocstor_35625_object31158/head//22 by client.940330.0:62466 2014-08-11 16:31:50.108233 2014-08-13 18:06:06.480959 7f5c973a57a0 20 read_log 194215'459 (0'0) modify a3e6429a/benchmark_data_storage0111-ib.pocstor_64498_object1440/head//22 by client.1049635.0:1441 2014-08-12 17:17:21.820545 2014-08-13 18:06:06.480972 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 2cd79a9a/benchmark_data_storage0111-ib.pocstor_64498_object5515/head//22 by client.1049635.0:5516 2014-08-12 17:17:34.660015 2014-08-13 18:06:06.480984 7f5c973a57a0 20 read_log 194215'461 (0'0) modify d9bff29a/benchmark_data_storage0111-ib.pocstor_64498_object7457/head//22 by client.1049635.0:7458 2014-08-12 17:17:40.416193 2014-08-13 18:06:06.480996 7f5c973a57a0 20 read_log 194215'462 (0'0) modify c8a7ea9a/benchmark_data_storage0111-ib.pocstor_64498_object8377/head//22 by client.1049635.0:8378 2014-08-12 17:17:43.202221 2014-08-13 18:06:06.481010 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 8b7c4a9a/benchmark_data_storage0111-ib.pocstor_64498_object8879/head//22 by client.1049635.0:8880 2014-08-12 17:17:44.677970 2014-08-13 18:06:06.481025 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 4c7eba9a/benchmark_data_storage0111-ib.pocstor_64498_object9444/head//22 by client.1049635.0:9445 2014-08-12 17:17:46.313717 2014-08-13 18:06:06.481037 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 31221a9a/benchmark_data_storage0111-ib.pocstor_64498_object9464/head//22 by client.1049635.0:9465 2014-08-12 17:17:46.371214 2014-08-13 18:06:06.481051 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 9f517a9a/benchmark_data_storage0111-ib.pocstor_64498_object10178/head//22 by client.1049635.0:10179 2014-08-12 17:17:48.435379 2014-08-13 18:06:06.481064 7f5c973a57a0 20 read_log 194215'467 (0'0) modify bd4b1a9a/benchmark_data_storage0111-ib.pocstor_64498_object14186/head//22 by client.1049635.0:14187 2014-08-12 17:18:01.069249 2014-08-13 18:06:06.481078 7f5c973a57a0 20 read_log 194215'468 (0'0) modify dda5129a/benchmark_data_storage0111-ib.pocstor_64498_object16118/head//22 by client.1049635.0:16119 2014-08-12 17:18:07.997347 2014-08-13 18:06:06.481098 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 258f529a/benchmark_data_storage0111-ib.pocstor_64498_object18818/head//22 by client.1049635.0:18819 2014-08-12 17:18:18.245369 2014-08-13 18:06:06.481112 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 9b491a9a/benchmark_data_storage0111-ib.pocstor_64498_object18967/head//22 by client.1049635.0:18968 2014-08-12 17:18:18.813307 2014-08-13 18:06:06.481125 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 740a3a9a/benchmark_data_storage0111-ib.pocstor_64498_object19482/head//22 by client.1049635.0:19483 2014-08-12 17:18:20.842384 2014-08-13 18:06:06.481139 7f5c973a57a0 20 read_log 194215'472 (0'0) modify d0db4a9a/benchmark_data_storage0111-ib.pocstor_64498_object25472/head//22 by client.1049635.0:25473 2014-08-12 17:18:43.462266 2014-08-13 18:06:06.481157 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 7cbd0a9a/benchmark_data_storage0111-ib.pocstor_64498_object32075/head//22 by client.1049635.0:32076 2014-08-12 17:19:08.855735 2014-08-13 18:06:06.481172 7f5c973a57a0 20 read_log 194215'474 (194215'459) delete a3e6429a/benchmark_data_storage0111-ib.pocstor_64498_object1440/head//22 by client.1049635.0:35643 2014-08-12 17:19:18.465088 2014-08-13 18:06:06.481185 7f5c973a57a0 20 read_log 194215'475 (194215'460) delete 2cd79a9a/benchmark_data_storage0111-ib.pocstor_64498_object5515/head//22 by client.1049635.0:39718 2014-08-12 17:19:19.776840 2014-08-13 18:06:06.481197 7f5c973a57a0 20 read_log 194215'476 (194215'461) delete d9bff29a/benchmark_data_storage0111-ib.pocstor_64498_object7457/head//22 by client.1049635.0:41660 2014-08-12 17:19:20.436838 2014-08-13 18:06:06.481210 7f5c973a57a0 20 read_log 194215'477 (194215'462) delete c8a7ea9a/benchmark_data_storage0111-ib.pocstor_64498_object8377/head//22 by client.1049635.0:42580 2014-08-12 17:19:20.720441 2014-08-13 18:06:06.481222 7f5c973a57a0 20 read_log 194215'478 (194215'463) delete 8b7c4a9a/benchmark_data_storage0111-ib.pocstor_64498_object8879/head//22 by client.1049635.0:43082 2014-08-12 17:19:20.885714 2014-08-13 18:06:06.481234 7f5c973a57a0 20 read_log 194215'479 (194215'464) delete 4c7eba9a/benchmark_data_storage0111-ib.pocstor_64498_object9444/head//22 by client.1049635.0:43647 2014-08-12 17:19:21.074107 2014-08-13 18:06:06.481249 7f5c973a57a0 20 read_log 194215'480 (194215'465) delete 31221a9a/benchmark_data_storage0111-ib.pocstor_64498_object9464/head//22 by client.1049635.0:43667 2014-08-12 17:19:21.079405 2014-08-13 18:06:06.481266 7f5c973a57a0 20 read_log 194215'481 (194215'466) delete 9f517a9a/benchmark_data_storage0111-ib.pocstor_64498_object10178/head//22 by client.1049635.0:44381 2014-08-12 17:19:21.321322 2014-08-13 18:06:06.481279 7f5c973a57a0 20 read_log 194215'482 (194215'467) delete bd4b1a9a/benchmark_data_storage0111-ib.pocstor_64498_object14186/head//22 by client.1049635.0:48389 2014-08-12 17:19:22.618870 2014-08-13 18:06:06.481291 7f5c973a57a0 20 read_log 194215'483 (194215'468) delete dda5129a/benchmark_data_storage0111-ib.pocstor_64498_object16118/head//22 by client.1049635.0:50321 2014-08-12 17:19:23.226018 2014-08-13 18:06:06.481307 7f5c973a57a0 20 read_log 194215'484 (194215'469) delete 258f529a/benchmark_data_storage0111-ib.pocstor_64498_object18818/head//22 by client.1049635.0:53021 2014-08-12 17:19:24.100637 2014-08-13 18:06:06.481322 7f5c973a57a0 20 read_log 194215'485 (194215'470) delete 9b491a9a/benchmark_data_storage0111-ib.pocstor_64498_object18967/head//22 by client.1049635.0:53170 2014-08-12 17:19:24.157210 2014-08-13 18:06:06.481334 7f5c973a57a0 20 read_log 194215'486 (194215'471) delete 740a3a9a/benchmark_data_storage0111-ib.pocstor_64498_object19482/head//22 by client.1049635.0:53685 2014-08-12 17:19:24.344152 2014-08-13 18:06:06.481347 7f5c973a57a0 20 read_log 194215'487 (194215'472) delete d0db4a9a/benchmark_data_storage0111-ib.pocstor_64498_object25472/head//22 by client.1049635.0:59675 2014-08-12 17:19:26.475418 2014-08-13 18:06:06.481362 7f5c973a57a0 20 read_log 194215'488 (194215'473) delete 7cbd0a9a/benchmark_data_storage0111-ib.pocstor_64498_object32075/head//22 by client.1049635.0:66278 2014-08-12 17:19:28.771812 2014-08-13 18:06:06.481598 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.481624 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=0 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.481641 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=0 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001787 0 0.000000 2014-08-13 18:06:06.481653 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=0 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.481661 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=0 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.481670 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] log((0'0,194215'488], crt=192888'458) 2014-08-13 18:06:06.481680 7f5c973a57a0 10 osd.4 194487 pgid 22.2b9 coll 22.2b9_head 2014-08-13 18:06:06.481810 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.2b9 2014-08-13 18:06:06.481819 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.481845 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2b9(unlocked)] enter Initial 2014-08-13 18:06:06.481854 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.2b9(unlocked)] enter NotTrimming 2014-08-13 18:06:06.482034 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.482172 7f5c973a57a0 20 read_log 191648'35 (0'0) modify d3f8f2b9/benchmark_data_storage0111-ib.pocstor_40745_object2578/head//22 by client.273158.0:2579 2014-07-30 14:08:59.235134 2014-08-13 18:06:06.482192 7f5c973a57a0 20 read_log 191648'50 (0'0) modify 7f207ab9/benchmark_data_storage0111-ib.pocstor_40745_object3459/head//22 by client.273158.0:3460 2014-07-30 14:09:03.526944 2014-08-13 18:06:06.482205 7f5c973a57a0 20 read_log 191648'57 (0'0) modify c0f0dab9/benchmark_data_storage0111-ib.pocstor_40745_object4202/head//22 by client.273158.0:4203 2014-07-30 14:09:07.625024 2014-08-13 18:06:06.482218 7f5c973a57a0 20 read_log 191648'122 (0'0) modify d72ab9/benchmark_data_storage0111-ib.pocstor_40745_object8496/head//22 by client.273158.0:8497 2014-07-30 14:09:28.890608 2014-08-13 18:06:06.482234 7f5c973a57a0 20 read_log 191648'131 (0'0) modify 8cb8ab9/benchmark_data_storage0111-ib.pocstor_40745_object9314/head//22 by client.273158.0:9315 2014-07-30 14:09:32.469869 2014-08-13 18:06:06.482249 7f5c973a57a0 20 read_log 191648'146 (0'0) modify ef02b2b9/benchmark_data_storage0111-ib.pocstor_40745_object10074/head//22 by client.273158.0:10075 2014-07-30 14:09:36.129504 2014-08-13 18:06:06.482261 7f5c973a57a0 20 read_log 191648'176 (0'0) modify 907fd2b9/benchmark_data_storage0111-ib.pocstor_40745_object11811/head//22 by client.273158.0:11812 2014-07-30 14:09:46.128749 2014-08-13 18:06:06.482289 7f5c973a57a0 20 read_log 191648'261 (0'0) modify 89e652b9/benchmark_data_storage0111-ib.pocstor_40745_object16679/head//22 by client.273158.0:16680 2014-07-30 14:10:11.242493 2014-08-13 18:06:06.482304 7f5c973a57a0 20 read_log 191648'267 (0'0) modify 146cb2b9/benchmark_data_storage0111-ib.pocstor_40745_object17206/head//22 by client.273158.0:17207 2014-07-30 14:10:14.122501 2014-08-13 18:06:06.482317 7f5c973a57a0 20 read_log 191648'272 (0'0) modify d0ea0ab9/benchmark_data_storage0111-ib.pocstor_40745_object17659/head//22 by client.273158.0:17660 2014-07-30 14:10:16.328821 2014-08-13 18:06:06.482342 7f5c973a57a0 20 read_log 191648'286 (0'0) modify 28e2fab9/benchmark_data_storage0111-ib.pocstor_40745_object18574/head//22 by client.273158.0:18575 2014-07-30 14:10:21.484947 2014-08-13 18:06:06.482360 7f5c973a57a0 20 read_log 191727'356 (0'0) modify 15b2eab9/benchmark_data_storage0111-ib.pocstor_51285_object1015/head//22 by client.275175.0:1016 2014-07-30 14:18:37.510632 2014-08-13 18:06:06.482372 7f5c973a57a0 20 read_log 191727'358 (0'0) modify ee6d2ab9/benchmark_data_storage0111-ib.pocstor_51285_object2208/head//22 by client.275175.0:2209 2014-07-30 14:18:41.573650 2014-08-13 18:06:06.482393 7f5c973a57a0 20 read_log 191727'360 (0'0) modify 113e8ab9/benchmark_data_storage0111-ib.pocstor_51285_object3238/head//22 by client.275175.0:3239 2014-07-30 14:18:44.972331 2014-08-13 18:06:06.482409 7f5c973a57a0 20 read_log 191727'365 (0'0) modify e8205ab9/benchmark_data_storage0111-ib.pocstor_51285_object4988/head//22 by client.275175.0:4989 2014-07-30 14:18:50.317372 2014-08-13 18:06:06.482422 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 6b5daab9/benchmark_data_storage0111-ib.pocstor_51285_object9391/head//22 by client.275175.0:9392 2014-07-30 14:19:03.480266 2014-08-13 18:06:06.482434 7f5c973a57a0 20 read_log 191727'378 (0'0) modify f85552b9/benchmark_data_storage0111-ib.pocstor_51285_object12132/head//22 by client.275175.0:12133 2014-07-30 14:19:12.009009 2014-08-13 18:06:06.482447 7f5c973a57a0 20 read_log 191727'380 (0'0) modify 80068ab9/benchmark_data_storage0111-ib.pocstor_51285_object12795/head//22 by client.275175.0:12796 2014-07-30 14:19:13.870536 2014-08-13 18:06:06.482459 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 26189ab9/benchmark_data_storage0111-ib.pocstor_51285_object14011/head//22 by client.275175.0:14012 2014-07-30 14:19:17.913542 2014-08-13 18:06:06.482472 7f5c973a57a0 20 read_log 191727'387 (0'0) modify 9fdee2b9/benchmark_data_storage0111-ib.pocstor_51285_object15409/head//22 by client.275175.0:15410 2014-07-30 14:19:22.120013 2014-08-13 18:06:06.482484 7f5c973a57a0 20 read_log 191727'388 (0'0) modify af35eab9/benchmark_data_storage0111-ib.pocstor_51285_object15464/head//22 by client.275175.0:15465 2014-07-30 14:19:22.313137 2014-08-13 18:06:06.482496 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 11a47ab9/benchmark_data_storage0111-ib.pocstor_51285_object18360/head//22 by client.275175.0:18361 2014-07-30 14:19:31.056235 2014-08-13 18:06:06.482508 7f5c973a57a0 20 read_log 191727'395 (0'0) modify b6478ab9/benchmark_data_storage0111-ib.pocstor_51285_object19555/head//22 by client.275175.0:19556 2014-07-30 14:19:34.931007 2014-08-13 18:06:06.482521 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 6c9ff2b9/benchmark_data_storage0111-ib.pocstor_51285_object20232/head//22 by client.275175.0:20233 2014-07-30 14:19:36.927640 2014-08-13 18:06:06.482533 7f5c973a57a0 20 read_log 191727'398 (0'0) modify ec344ab9/benchmark_data_storage0111-ib.pocstor_51285_object20497/head//22 by client.275175.0:20498 2014-07-30 14:19:37.686646 2014-08-13 18:06:06.482545 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 5de7eab9/benchmark_data_storage0111-ib.pocstor_51285_object25019/head//22 by client.275175.0:25020 2014-07-30 14:19:51.736299 2014-08-13 18:06:06.482565 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 183982b9/benchmark_data_storage0111-ib.pocstor_51285_object28811/head//22 by client.275175.0:28812 2014-07-30 14:20:03.335783 2014-08-13 18:06:06.482580 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 52bdc2b9/benchmark_data_storage0111-ib.pocstor_51285_object32990/head//22 by client.275175.0:32991 2014-07-30 14:20:15.770962 2014-08-13 18:06:06.482595 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 3fac2ab9/benchmark_data_storage0111-ib.pocstor_51285_object34206/head//22 by client.275175.0:34207 2014-07-30 14:20:19.645396 2014-08-13 18:06:06.482608 7f5c973a57a0 20 read_log 191727'426 (0'0) modify ea98fab9/benchmark_data_storage0111-ib.pocstor_51285_object35788/head//22 by client.275175.0:35789 2014-07-30 14:20:24.618718 2014-08-13 18:06:06.482620 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 7fc9dab9/benchmark_data_storage0111-ib.pocstor_51285_object35837/head//22 by client.275175.0:35838 2014-07-30 14:20:24.753367 2014-08-13 18:06:06.482632 7f5c973a57a0 20 read_log 191727'428 (0'0) modify 891bbab9/benchmark_data_storage0111-ib.pocstor_51285_object37239/head//22 by client.275175.0:37240 2014-07-30 14:20:29.038620 2014-08-13 18:06:06.482644 7f5c973a57a0 20 read_log 191727'430 (0'0) modify af75cab9/benchmark_data_storage0111-ib.pocstor_51285_object37400/head//22 by client.275175.0:37401 2014-07-30 14:20:29.520223 2014-08-13 18:06:06.482656 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 262f02b9/benchmark_data_storage0111-ib.pocstor_35625_object92/head//22 by client.940330.0:93 2014-08-11 16:29:40.880779 2014-08-13 18:06:06.482669 7f5c973a57a0 20 read_log 192888'434 (0'0) modify d0eb1ab9/benchmark_data_storage0111-ib.pocstor_35625_object439/head//22 by client.940330.0:440 2014-08-11 16:29:42.433265 2014-08-13 18:06:06.482681 7f5c973a57a0 20 read_log 192888'435 (0'0) modify f282cab9/benchmark_data_storage0111-ib.pocstor_35625_object1820/head//22 by client.940330.0:1821 2014-08-11 16:29:47.885147 2014-08-13 18:06:06.482693 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 7ecbcab9/benchmark_data_storage0111-ib.pocstor_35625_object7012/head//22 by client.940330.0:7013 2014-08-11 16:30:07.360659 2014-08-13 18:06:06.482705 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 8c6df2b9/benchmark_data_storage0111-ib.pocstor_35625_object9464/head//22 by client.940330.0:9465 2014-08-11 16:30:15.954132 2014-08-13 18:06:06.482717 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 6864dab9/benchmark_data_storage0111-ib.pocstor_35625_object11075/head//22 by client.940330.0:11076 2014-08-11 16:30:22.091043 2014-08-13 18:06:06.482729 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 6e8472b9/benchmark_data_storage0111-ib.pocstor_35625_object12694/head//22 by client.940330.0:12695 2014-08-11 16:30:28.618262 2014-08-13 18:06:06.482741 7f5c973a57a0 20 read_log 192888'440 (0'0) modify ef062b9/benchmark_data_storage0111-ib.pocstor_35625_object13219/head//22 by client.940330.0:13220 2014-08-11 16:30:30.739260 2014-08-13 18:06:06.482753 7f5c973a57a0 20 read_log 192888'441 (0'0) modify a366b2b9/benchmark_data_storage0111-ib.pocstor_35625_object16802/head//22 by client.940330.0:16803 2014-08-11 16:30:44.227856 2014-08-13 18:06:06.482765 7f5c973a57a0 20 read_log 192888'442 (0'0) modify da46d2b9/benchmark_data_storage0111-ib.pocstor_35625_object17401/head//22 by client.940330.0:17402 2014-08-11 16:30:46.462131 2014-08-13 18:06:06.482777 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 578d2ab9/benchmark_data_storage0111-ib.pocstor_35625_object21585/head//22 by client.940330.0:21586 2014-08-11 16:31:02.988341 2014-08-13 18:06:06.482792 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 610412b9/benchmark_data_storage0111-ib.pocstor_35625_object22270/head//22 by client.940330.0:22271 2014-08-11 16:31:05.576452 2014-08-13 18:06:06.482828 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 9303d2b9/benchmark_data_storage0111-ib.pocstor_35625_object24253/head//22 by client.940330.0:24254 2014-08-11 16:31:13.072121 2014-08-13 18:06:06.482844 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 5047b2b9/benchmark_data_storage0111-ib.pocstor_35625_object24409/head//22 by client.940330.0:24410 2014-08-11 16:31:13.635303 2014-08-13 18:06:06.482856 7f5c973a57a0 20 read_log 192888'447 (0'0) modify 34d39ab9/benchmark_data_storage0111-ib.pocstor_35625_object25823/head//22 by client.940330.0:25824 2014-08-11 16:31:19.098064 2014-08-13 18:06:06.482871 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 9fc76ab9/benchmark_data_storage0111-ib.pocstor_35625_object27778/head//22 by client.940330.0:27779 2014-08-11 16:31:26.821318 2014-08-13 18:06:06.482884 7f5c973a57a0 20 read_log 192888'449 (0'0) modify e9267ab9/benchmark_data_storage0111-ib.pocstor_35625_object27786/head//22 by client.940330.0:27787 2014-08-11 16:31:26.852202 2014-08-13 18:06:06.482898 7f5c973a57a0 20 read_log 192888'450 (0'0) modify e5fc3ab9/benchmark_data_storage0111-ib.pocstor_35625_object28447/head//22 by client.940330.0:28448 2014-08-11 16:31:29.452060 2014-08-13 18:06:06.482912 7f5c973a57a0 20 read_log 192888'451 (0'0) modify bfa62b9/benchmark_data_storage0111-ib.pocstor_35625_object29176/head//22 by client.940330.0:29177 2014-08-11 16:31:32.344930 2014-08-13 18:06:06.482924 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 4f51dab9/benchmark_data_storage0111-ib.pocstor_35625_object30528/head//22 by client.940330.0:30529 2014-08-11 16:31:37.456588 2014-08-13 18:06:06.482959 7f5c973a57a0 20 read_log 192888'453 (192888'433) delete 262f02b9/benchmark_data_storage0111-ib.pocstor_35625_object92/head//22 by client.940330.0:31400 2014-08-11 16:31:41.474492 2014-08-13 18:06:06.482975 7f5c973a57a0 20 read_log 192888'454 (192888'434) delete d0eb1ab9/benchmark_data_storage0111-ib.pocstor_35625_object439/head//22 by client.940330.0:31747 2014-08-11 16:31:41.605606 2014-08-13 18:06:06.482988 7f5c973a57a0 20 read_log 192888'455 (192888'435) delete f282cab9/benchmark_data_storage0111-ib.pocstor_35625_object1820/head//22 by client.940330.0:33128 2014-08-11 16:31:41.945367 2014-08-13 18:06:06.483002 7f5c973a57a0 20 read_log 192888'456 (192888'436) delete 7ecbcab9/benchmark_data_storage0111-ib.pocstor_35625_object7012/head//22 by client.940330.0:38320 2014-08-11 16:31:43.338398 2014-08-13 18:06:06.483018 7f5c973a57a0 20 read_log 192888'457 (192888'437) delete 8c6df2b9/benchmark_data_storage0111-ib.pocstor_35625_object9464/head//22 by client.940330.0:40772 2014-08-11 16:31:43.965609 2014-08-13 18:06:06.483031 7f5c973a57a0 20 read_log 192888'458 (192888'438) delete 6864dab9/benchmark_data_storage0111-ib.pocstor_35625_object11075/head//22 by client.940330.0:42383 2014-08-11 16:31:44.419791 2014-08-13 18:06:06.483043 7f5c973a57a0 20 read_log 192888'459 (192888'439) delete 6e8472b9/benchmark_data_storage0111-ib.pocstor_35625_object12694/head//22 by client.940330.0:44002 2014-08-11 16:31:44.832866 2014-08-13 18:06:06.483055 7f5c973a57a0 20 read_log 192888'460 (192888'440) delete ef062b9/benchmark_data_storage0111-ib.pocstor_35625_object13219/head//22 by client.940330.0:44527 2014-08-11 16:31:44.980969 2014-08-13 18:06:06.483070 7f5c973a57a0 20 read_log 192888'461 (192888'441) delete a366b2b9/benchmark_data_storage0111-ib.pocstor_35625_object16802/head//22 by client.940330.0:48110 2014-08-11 16:31:45.934941 2014-08-13 18:06:06.483082 7f5c973a57a0 20 read_log 192888'462 (192888'442) delete da46d2b9/benchmark_data_storage0111-ib.pocstor_35625_object17401/head//22 by client.940330.0:48709 2014-08-11 16:31:46.101895 2014-08-13 18:06:06.483095 7f5c973a57a0 20 read_log 192888'463 (192888'443) delete 578d2ab9/benchmark_data_storage0111-ib.pocstor_35625_object21585/head//22 by client.940330.0:52893 2014-08-11 16:31:47.317679 2014-08-13 18:06:06.483109 7f5c973a57a0 20 read_log 192888'464 (192888'444) delete 610412b9/benchmark_data_storage0111-ib.pocstor_35625_object22270/head//22 by client.940330.0:53578 2014-08-11 16:31:47.534444 2014-08-13 18:06:06.483131 7f5c973a57a0 20 read_log 192888'465 (192888'445) delete 9303d2b9/benchmark_data_storage0111-ib.pocstor_35625_object24253/head//22 by client.940330.0:55561 2014-08-11 16:31:48.115829 2014-08-13 18:06:06.483154 7f5c973a57a0 20 read_log 192888'466 (192888'446) delete 5047b2b9/benchmark_data_storage0111-ib.pocstor_35625_object24409/head//22 by client.940330.0:55717 2014-08-11 16:31:48.159218 2014-08-13 18:06:06.483170 7f5c973a57a0 20 read_log 192888'467 (192888'447) delete 34d39ab9/benchmark_data_storage0111-ib.pocstor_35625_object25823/head//22 by client.940330.0:57131 2014-08-11 16:31:48.594401 2014-08-13 18:06:06.483182 7f5c973a57a0 20 read_log 192888'468 (192888'448) delete 9fc76ab9/benchmark_data_storage0111-ib.pocstor_35625_object27778/head//22 by client.940330.0:59086 2014-08-11 16:31:49.144956 2014-08-13 18:06:06.483195 7f5c973a57a0 20 read_log 192888'469 (192888'449) delete e9267ab9/benchmark_data_storage0111-ib.pocstor_35625_object27786/head//22 by client.940330.0:59094 2014-08-11 16:31:49.147349 2014-08-13 18:06:06.483208 7f5c973a57a0 20 read_log 192888'470 (192888'450) delete e5fc3ab9/benchmark_data_storage0111-ib.pocstor_35625_object28447/head//22 by client.940330.0:59755 2014-08-11 16:31:49.323373 2014-08-13 18:06:06.483220 7f5c973a57a0 20 read_log 192888'471 (192888'451) delete bfa62b9/benchmark_data_storage0111-ib.pocstor_35625_object29176/head//22 by client.940330.0:60484 2014-08-11 16:31:49.517446 2014-08-13 18:06:06.483235 7f5c973a57a0 20 read_log 192888'472 (192888'452) delete 4f51dab9/benchmark_data_storage0111-ib.pocstor_35625_object30528/head//22 by client.940330.0:61836 2014-08-11 16:31:49.909042 2014-08-13 18:06:06.483247 7f5c973a57a0 20 read_log 194215'473 (0'0) modify a9d172b9/benchmark_data_storage0111-ib.pocstor_64498_object3760/head//22 by client.1049635.0:3761 2014-08-12 17:17:29.272366 2014-08-13 18:06:06.483260 7f5c973a57a0 20 read_log 194215'474 (0'0) modify e90592b9/benchmark_data_storage0111-ib.pocstor_64498_object3986/head//22 by client.1049635.0:3987 2014-08-12 17:17:30.044775 2014-08-13 18:06:06.483278 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 8ac7e2b9/benchmark_data_storage0111-ib.pocstor_64498_object4273/head//22 by client.1049635.0:4274 2014-08-12 17:17:30.876383 2014-08-13 18:06:06.483291 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 91e7ab9/benchmark_data_storage0111-ib.pocstor_64498_object4663/head//22 by client.1049635.0:4664 2014-08-12 17:17:31.947733 2014-08-13 18:06:06.483303 7f5c973a57a0 20 read_log 194215'477 (0'0) modify f97aeab9/benchmark_data_storage0111-ib.pocstor_64498_object12653/head//22 by client.1049635.0:12654 2014-08-12 17:17:56.313997 2014-08-13 18:06:06.483315 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 8f9d4ab9/benchmark_data_storage0111-ib.pocstor_64498_object13940/head//22 by client.1049635.0:13941 2014-08-12 17:18:00.341466 2014-08-13 18:06:06.483331 7f5c973a57a0 20 read_log 194215'479 (0'0) modify 5ba662b9/benchmark_data_storage0111-ib.pocstor_64498_object15216/head//22 by client.1049635.0:15217 2014-08-12 17:18:04.690502 2014-08-13 18:06:06.483344 7f5c973a57a0 20 read_log 194215'480 (0'0) modify cdfe3ab9/benchmark_data_storage0111-ib.pocstor_64498_object16216/head//22 by client.1049635.0:16217 2014-08-12 17:18:08.376666 2014-08-13 18:06:06.483356 7f5c973a57a0 20 read_log 194215'481 (0'0) modify 3e4a9ab9/benchmark_data_storage0111-ib.pocstor_64498_object18182/head//22 by client.1049635.0:18183 2014-08-12 17:18:15.851499 2014-08-13 18:06:06.483398 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 4bc3ab9/benchmark_data_storage0111-ib.pocstor_64498_object18556/head//22 by client.1049635.0:18557 2014-08-12 17:18:17.261424 2014-08-13 18:06:06.483415 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 47b7dab9/benchmark_data_storage0111-ib.pocstor_64498_object21260/head//22 by client.1049635.0:21261 2014-08-12 17:18:27.561650 2014-08-13 18:06:06.483439 7f5c973a57a0 20 read_log 194215'484 (0'0) modify a2ea42b9/benchmark_data_storage0111-ib.pocstor_64498_object22370/head//22 by client.1049635.0:22371 2014-08-12 17:18:31.721020 2014-08-13 18:06:06.483464 7f5c973a57a0 20 read_log 194215'485 (0'0) modify c476aab9/benchmark_data_storage0111-ib.pocstor_64498_object24913/head//22 by client.1049635.0:24914 2014-08-12 17:18:41.373895 2014-08-13 18:06:06.483477 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 315662b9/benchmark_data_storage0111-ib.pocstor_64498_object26498/head//22 by client.1049635.0:26499 2014-08-12 17:18:47.281435 2014-08-13 18:06:06.483496 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 29a7dab9/benchmark_data_storage0111-ib.pocstor_64498_object27780/head//22 by client.1049635.0:27781 2014-08-12 17:18:52.065774 2014-08-13 18:06:06.483513 7f5c973a57a0 20 read_log 194215'488 (0'0) modify cd2c82b9/benchmark_data_storage0111-ib.pocstor_64498_object28703/head//22 by client.1049635.0:28704 2014-08-12 17:18:55.495257 2014-08-13 18:06:06.483554 7f5c973a57a0 20 read_log 194215'489 (0'0) modify eb9c7ab9/benchmark_data_storage0111-ib.pocstor_64498_object29655/head//22 by client.1049635.0:29656 2014-08-12 17:18:59.157579 2014-08-13 18:06:06.483570 7f5c973a57a0 20 read_log 194215'490 (194215'473) delete a9d172b9/benchmark_data_storage0111-ib.pocstor_64498_object3760/head//22 by client.1049635.0:37963 2014-08-12 17:19:19.201888 2014-08-13 18:06:06.483583 7f5c973a57a0 20 read_log 194215'491 (194215'474) delete e90592b9/benchmark_data_storage0111-ib.pocstor_64498_object3986/head//22 by client.1049635.0:38189 2014-08-12 17:19:19.270674 2014-08-13 18:06:06.483595 7f5c973a57a0 20 read_log 194215'492 (194215'475) delete 8ac7e2b9/benchmark_data_storage0111-ib.pocstor_64498_object4273/head//22 by client.1049635.0:38476 2014-08-12 17:19:19.364508 2014-08-13 18:06:06.483616 7f5c973a57a0 20 read_log 194215'493 (194215'476) delete 91e7ab9/benchmark_data_storage0111-ib.pocstor_64498_object4663/head//22 by client.1049635.0:38866 2014-08-12 17:19:19.490071 2014-08-13 18:06:06.483633 7f5c973a57a0 20 read_log 194215'494 (194215'477) delete f97aeab9/benchmark_data_storage0111-ib.pocstor_64498_object12653/head//22 by client.1049635.0:46856 2014-08-12 17:19:22.101362 2014-08-13 18:06:06.483646 7f5c973a57a0 20 read_log 194215'495 (194215'478) delete 8f9d4ab9/benchmark_data_storage0111-ib.pocstor_64498_object13940/head//22 by client.1049635.0:48143 2014-08-12 17:19:22.545011 2014-08-13 18:06:06.483658 7f5c973a57a0 20 read_log 194215'496 (194215'479) delete 5ba662b9/benchmark_data_storage0111-ib.pocstor_64498_object15216/head//22 by client.1049635.0:49419 2014-08-12 17:19:22.942355 2014-08-13 18:06:06.483671 7f5c973a57a0 20 read_log 194215'497 (194215'480) delete cdfe3ab9/benchmark_data_storage0111-ib.pocstor_64498_object16216/head//22 by client.1049635.0:50419 2014-08-12 17:19:23.256361 2014-08-13 18:06:06.483683 7f5c973a57a0 20 read_log 194215'498 (194215'481) delete 3e4a9ab9/benchmark_data_storage0111-ib.pocstor_64498_object18182/head//22 by client.1049635.0:52385 2014-08-12 17:19:23.891171 2014-08-13 18:06:06.483696 7f5c973a57a0 20 read_log 194215'499 (194215'482) delete 4bc3ab9/benchmark_data_storage0111-ib.pocstor_64498_object18556/head//22 by client.1049635.0:52759 2014-08-12 17:19:24.016127 2014-08-13 18:06:06.483708 7f5c973a57a0 20 read_log 194215'500 (194215'483) delete 47b7dab9/benchmark_data_storage0111-ib.pocstor_64498_object21260/head//22 by client.1049635.0:55463 2014-08-12 17:19:24.975482 2014-08-13 18:06:06.483721 7f5c973a57a0 20 read_log 194215'501 (194215'484) delete a2ea42b9/benchmark_data_storage0111-ib.pocstor_64498_object22370/head//22 by client.1049635.0:56573 2014-08-12 17:19:25.371796 2014-08-13 18:06:06.483733 7f5c973a57a0 20 read_log 194215'502 (194215'485) delete c476aab9/benchmark_data_storage0111-ib.pocstor_64498_object24913/head//22 by client.1049635.0:59116 2014-08-12 17:19:26.292249 2014-08-13 18:06:06.483745 7f5c973a57a0 20 read_log 194215'503 (194215'486) delete 315662b9/benchmark_data_storage0111-ib.pocstor_64498_object26498/head//22 by client.1049635.0:60701 2014-08-12 17:19:26.821811 2014-08-13 18:06:06.483765 7f5c973a57a0 20 read_log 194215'504 (194215'487) delete 29a7dab9/benchmark_data_storage0111-ib.pocstor_64498_object27780/head//22 by client.1049635.0:61983 2014-08-12 17:19:27.270597 2014-08-13 18:06:06.483780 7f5c973a57a0 20 read_log 194215'505 (194215'488) delete cd2c82b9/benchmark_data_storage0111-ib.pocstor_64498_object28703/head//22 by client.1049635.0:62906 2014-08-12 17:19:27.598334 2014-08-13 18:06:06.483792 7f5c973a57a0 20 read_log 194215'506 (194215'489) delete eb9c7ab9/benchmark_data_storage0111-ib.pocstor_64498_object29655/head//22 by client.1049635.0:63858 2014-08-12 17:19:27.936472 2014-08-13 18:06:06.483914 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.483932 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.483948 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Initial 0.002104 0 0.000000 2014-08-13 18:06:06.483960 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.483969 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.483981 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] log((0'0,194215'506], crt=192888'472) 2014-08-13 18:06:06.483992 7f5c973a57a0 10 osd.4 194487 pgid 22.2d1 coll 22.2d1_head 2014-08-13 18:06:06.484154 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.2d1 2014-08-13 18:06:06.484164 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.484201 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2d1(unlocked)] enter Initial 2014-08-13 18:06:06.484213 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.2d1(unlocked)] enter NotTrimming 2014-08-13 18:06:06.484372 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.484511 7f5c973a57a0 20 read_log 191648'32 (0'0) modify ad8a5ad1/benchmark_data_storage0111-ib.pocstor_40745_object2205/head//22 by client.273158.0:2206 2014-07-30 14:08:57.183342 2014-08-13 18:06:06.484532 7f5c973a57a0 20 read_log 191648'41 (0'0) modify 6051ead1/benchmark_data_storage0111-ib.pocstor_40745_object2679/head//22 by client.273158.0:2680 2014-07-30 14:08:59.759974 2014-08-13 18:06:06.484545 7f5c973a57a0 20 read_log 191648'49 (0'0) modify 33ceb2d1/benchmark_data_storage0111-ib.pocstor_40745_object3158/head//22 by client.273158.0:3159 2014-07-30 14:09:02.039944 2014-08-13 18:06:06.484557 7f5c973a57a0 20 read_log 191648'66 (0'0) modify 2aa79ad1/benchmark_data_storage0111-ib.pocstor_40745_object4564/head//22 by client.273158.0:4565 2014-07-30 14:09:09.120361 2014-08-13 18:06:06.484569 7f5c973a57a0 20 read_log 191648'122 (0'0) modify 9bf872d1/benchmark_data_storage0111-ib.pocstor_40745_object7240/head//22 by client.273158.0:7241 2014-07-30 14:09:22.037233 2014-08-13 18:06:06.484582 7f5c973a57a0 20 read_log 191648'147 (0'0) modify c6ef32d1/benchmark_data_storage0111-ib.pocstor_40745_object8901/head//22 by client.273158.0:8902 2014-07-30 14:09:30.781815 2014-08-13 18:06:06.484594 7f5c973a57a0 20 read_log 191648'167 (0'0) modify b1ca12d1/benchmark_data_storage0111-ib.pocstor_40745_object10248/head//22 by client.273158.0:10249 2014-07-30 14:09:37.043171 2014-08-13 18:06:06.484606 7f5c973a57a0 20 read_log 191648'171 (0'0) modify 19d27ad1/benchmark_data_storage0111-ib.pocstor_40745_object10417/head//22 by client.273158.0:10418 2014-07-30 14:09:37.905583 2014-08-13 18:06:06.484618 7f5c973a57a0 20 read_log 191648'185 (0'0) modify ed430ad1/benchmark_data_storage0111-ib.pocstor_40745_object10953/head//22 by client.273158.0:10954 2014-07-30 14:09:41.416379 2014-08-13 18:06:06.484630 7f5c973a57a0 20 read_log 191648'198 (0'0) modify b751aad1/benchmark_data_storage0111-ib.pocstor_40745_object11545/head//22 by client.273158.0:11546 2014-07-30 14:09:45.003651 2014-08-13 18:06:06.484643 7f5c973a57a0 20 read_log 191648'208 (0'0) modify 3ff242d1/benchmark_data_storage0111-ib.pocstor_40745_object12109/head//22 by client.273158.0:12110 2014-07-30 14:09:48.208782 2014-08-13 18:06:06.484655 7f5c973a57a0 20 read_log 191648'225 (0'0) modify 25bfa2d1/benchmark_data_storage0111-ib.pocstor_40745_object13090/head//22 by client.273158.0:13091 2014-07-30 14:09:52.861537 2014-08-13 18:06:06.484667 7f5c973a57a0 20 read_log 191648'233 (0'0) modify 5c2c4ad1/benchmark_data_storage0111-ib.pocstor_40745_object13619/head//22 by client.273158.0:13620 2014-07-30 14:09:55.245600 2014-08-13 18:06:06.484679 7f5c973a57a0 20 read_log 191648'274 (0'0) modify 8a96b2d1/benchmark_data_storage0111-ib.pocstor_40745_object16184/head//22 by client.273158.0:16185 2014-07-30 14:10:08.458019 2014-08-13 18:06:06.484691 7f5c973a57a0 20 read_log 191648'299 (0'0) modify 767962d1/benchmark_data_storage0111-ib.pocstor_40745_object17600/head//22 by client.273158.0:17601 2014-07-30 14:10:16.082096 2014-08-13 18:06:06.484705 7f5c973a57a0 20 read_log 191648'339 (0'0) modify cb9e4ad1/benchmark_data_storage0111-ib.pocstor_40745_object20445/head//22 by client.273158.0:20446 2014-07-30 14:10:33.191753 2014-08-13 18:06:06.484717 7f5c973a57a0 20 read_log 191648'357 (0'0) modify a3b7c2d1/benchmark_data_storage0111-ib.pocstor_40745_object21782/head//22 by client.273158.0:21783 2014-07-30 14:10:41.582122 2014-08-13 18:06:06.484729 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 4d3de2d1/benchmark_data_storage0111-ib.pocstor_51285_object8734/head//22 by client.275175.0:8735 2014-07-30 14:19:01.591699 2014-08-13 18:06:06.484743 7f5c973a57a0 20 read_log 191727'388 (0'0) modify eeee2d1/benchmark_data_storage0111-ib.pocstor_51285_object11614/head//22 by client.275175.0:11615 2014-07-30 14:19:10.087029 2014-08-13 18:06:06.484767 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 8c379ad1/benchmark_data_storage0111-ib.pocstor_51285_object12033/head//22 by client.275175.0:12034 2014-07-30 14:19:11.730939 2014-08-13 18:06:06.484782 7f5c973a57a0 20 read_log 191727'394 (0'0) modify f5c1aad1/benchmark_data_storage0111-ib.pocstor_51285_object14555/head//22 by client.275175.0:14556 2014-07-30 14:19:19.446768 2014-08-13 18:06:06.484795 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 9b83ead1/benchmark_data_storage0111-ib.pocstor_51285_object14639/head//22 by client.275175.0:14640 2014-07-30 14:19:19.685388 2014-08-13 18:06:06.484807 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 965722d1/benchmark_data_storage0111-ib.pocstor_51285_object16439/head//22 by client.275175.0:16440 2014-07-30 14:19:25.497346 2014-08-13 18:06:06.484818 7f5c973a57a0 20 read_log 191727'410 (0'0) modify 7ea23ad1/benchmark_data_storage0111-ib.pocstor_51285_object23247/head//22 by client.275175.0:23248 2014-07-30 14:19:46.728335 2014-08-13 18:06:06.484832 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 8fb5e2d1/benchmark_data_storage0111-ib.pocstor_51285_object23868/head//22 by client.275175.0:23869 2014-07-30 14:19:48.464742 2014-08-13 18:06:06.484844 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 4644dad1/benchmark_data_storage0111-ib.pocstor_51285_object32821/head//22 by client.275175.0:32822 2014-07-30 14:20:15.294467 2014-08-13 18:06:06.484856 7f5c973a57a0 20 read_log 191727'427 (0'0) modify f1dc3ad1/benchmark_data_storage0111-ib.pocstor_51285_object33057/head//22 by client.275175.0:33058 2014-07-30 14:20:15.957920 2014-08-13 18:06:06.484869 7f5c973a57a0 20 read_log 191727'428 (0'0) modify fae99ad1/benchmark_data_storage0111-ib.pocstor_51285_object33486/head//22 by client.275175.0:33487 2014-07-30 14:20:17.431133 2014-08-13 18:06:06.484881 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 11d45ad1/benchmark_data_storage0111-ib.pocstor_51285_object33754/head//22 by client.275175.0:33755 2014-07-30 14:20:18.293502 2014-08-13 18:06:06.484893 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 41c42d1/benchmark_data_storage0111-ib.pocstor_51285_object33986/head//22 by client.275175.0:33987 2014-07-30 14:20:18.953694 2014-08-13 18:06:06.484905 7f5c973a57a0 20 read_log 191727'433 (0'0) modify aa6cd2d1/benchmark_data_storage0111-ib.pocstor_51285_object35827/head//22 by client.275175.0:35828 2014-07-30 14:20:24.726318 2014-08-13 18:06:06.484917 7f5c973a57a0 20 read_log 191727'435 (0'0) modify f9d62d1/benchmark_data_storage0111-ib.pocstor_51285_object36377/head//22 by client.275175.0:36378 2014-07-30 14:20:26.407419 2014-08-13 18:06:06.484929 7f5c973a57a0 20 read_log 191727'436 (0'0) modify a8372d1/benchmark_data_storage0111-ib.pocstor_51285_object37621/head//22 by client.275175.0:37622 2014-07-30 14:20:30.186969 2014-08-13 18:06:06.484941 7f5c973a57a0 20 read_log 191727'437 (0'0) modify d2b93ad1/benchmark_data_storage0111-ib.pocstor_51285_object38275/head//22 by client.275175.0:38276 2014-07-30 14:20:32.030022 2014-08-13 18:06:06.484954 7f5c973a57a0 20 read_log 192666'438 (0'0) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1273 2014-07-30 16:28:47.899928 2014-08-13 18:06:06.484966 7f5c973a57a0 20 read_log 192666'439 (192666'438) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1283 2014-07-30 16:28:48.102920 2014-08-13 18:06:06.485000 7f5c973a57a0 20 read_log 192666'440 (192666'439) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1284 2014-07-30 16:28:48.106920 2014-08-13 18:06:06.485016 7f5c973a57a0 20 read_log 192666'441 (192666'440) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1285 2014-07-30 16:28:48.149918 2014-08-13 18:06:06.485028 7f5c973a57a0 20 read_log 192666'442 (192666'441) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1286 2014-07-30 16:28:48.153918 2014-08-13 18:06:06.485067 7f5c973a57a0 20 read_log 192666'443 (192666'442) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1348 2014-07-30 16:28:48.910889 2014-08-13 18:06:06.485083 7f5c973a57a0 20 read_log 192666'444 (192666'443) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.275486.1:1349 2014-07-30 16:28:48.914889 2014-08-13 18:06:06.485096 7f5c973a57a0 20 read_log 192666'445 (192666'444) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:662 2014-07-30 17:22:39.389000 2014-08-13 18:06:06.485108 7f5c973a57a0 20 read_log 192666'446 (192666'445) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:897 2014-07-30 17:22:40.662000 2014-08-13 18:06:06.485120 7f5c973a57a0 20 read_log 192666'447 (192666'446) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:907 2014-07-30 17:22:40.700000 2014-08-13 18:06:06.485133 7f5c973a57a0 20 read_log 192666'448 (192666'447) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:908 2014-07-30 17:22:40.705000 2014-08-13 18:06:06.485153 7f5c973a57a0 20 read_log 192666'449 (192666'448) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:909 2014-07-30 17:22:40.708000 2014-08-13 18:06:06.485168 7f5c973a57a0 20 read_log 192666'450 (192666'449) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:910 2014-07-30 17:22:40.711000 2014-08-13 18:06:06.485180 7f5c973a57a0 20 read_log 192666'451 (192666'450) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:972 2014-07-30 17:22:41.059000 2014-08-13 18:06:06.485193 7f5c973a57a0 20 read_log 192666'452 (192666'451) modify 6fa29ad1/rb.0.43b76.238e1f29.0000000063f0/head//22 by client.276545.1:973 2014-07-30 17:22:41.063000 2014-08-13 18:06:06.485205 7f5c973a57a0 20 read_log 192674'453 (0'0) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1710 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485219 7f5c973a57a0 20 read_log 192674'454 (192674'453) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1711 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485231 7f5c973a57a0 20 read_log 192674'455 (192674'454) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1712 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485245 7f5c973a57a0 20 read_log 192674'456 (192674'455) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1713 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485258 7f5c973a57a0 20 read_log 192674'457 (192674'456) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1714 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485276 7f5c973a57a0 20 read_log 192674'458 (192674'457) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1715 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485288 7f5c973a57a0 20 read_log 192674'459 (192674'458) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1716 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485300 7f5c973a57a0 20 read_log 192674'460 (192674'459) modify 168d5ad1/rb.0.43b76.238e1f29.000000000078/head//22 by client.279772.1:1717 2014-07-31 18:03:43.479000 2014-08-13 18:06:06.485312 7f5c973a57a0 20 read_log 192674'461 (0'0) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4759 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485325 7f5c973a57a0 20 read_log 192674'462 (192674'461) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4760 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485337 7f5c973a57a0 20 read_log 192674'463 (192674'462) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4761 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485350 7f5c973a57a0 20 read_log 192674'464 (192674'463) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4762 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485371 7f5c973a57a0 20 read_log 192674'465 (192674'464) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4763 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485386 7f5c973a57a0 20 read_log 192674'466 (192674'465) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4764 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485409 7f5c973a57a0 20 read_log 192674'467 (192674'466) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4765 2014-07-31 18:03:46.699000 2014-08-13 18:06:06.485424 7f5c973a57a0 20 read_log 192674'468 (192674'467) modify aecf72d1/rb.0.43b76.238e1f29.0000000001f5/head//22 by client.279772.1:4766 2014-07-31 18:03:46.726000 2014-08-13 18:06:06.485436 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 5ee3e2d1/benchmark_data_storage0111-ib.pocstor_35625_object2742/head//22 by client.940330.0:2743 2014-08-11 16:29:51.404484 2014-08-13 18:06:06.485467 7f5c973a57a0 20 read_log 192888'470 (0'0) modify 7d6252d1/benchmark_data_storage0111-ib.pocstor_35625_object2854/head//22 by client.940330.0:2855 2014-08-11 16:29:51.988624 2014-08-13 18:06:06.485480 7f5c973a57a0 20 read_log 192888'471 (0'0) modify 15bb5ad1/benchmark_data_storage0111-ib.pocstor_35625_object3322/head//22 by client.940330.0:3323 2014-08-11 16:29:53.821111 2014-08-13 18:06:06.485495 7f5c973a57a0 20 read_log 192888'472 (0'0) modify 232732d1/benchmark_data_storage0111-ib.pocstor_35625_object10456/head//22 by client.940330.0:10457 2014-08-11 16:30:19.567983 2014-08-13 18:06:06.485509 7f5c973a57a0 20 read_log 192888'473 (0'0) modify 33b6b2d1/benchmark_data_storage0111-ib.pocstor_35625_object10838/head//22 by client.940330.0:10839 2014-08-11 16:30:21.172030 2014-08-13 18:06:06.485522 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 891c8ad1/benchmark_data_storage0111-ib.pocstor_35625_object10903/head//22 by client.940330.0:10904 2014-08-11 16:30:21.422951 2014-08-13 18:06:06.485534 7f5c973a57a0 20 read_log 192888'475 (0'0) modify 841902d1/benchmark_data_storage0111-ib.pocstor_35625_object11927/head//22 by client.940330.0:11928 2014-08-11 16:30:25.562521 2014-08-13 18:06:06.485546 7f5c973a57a0 20 read_log 192888'476 (0'0) modify c26f02d1/benchmark_data_storage0111-ib.pocstor_35625_object22095/head//22 by client.940330.0:22096 2014-08-11 16:31:04.928366 2014-08-13 18:06:06.485558 7f5c973a57a0 20 read_log 192888'477 (0'0) modify cbe69ad1/benchmark_data_storage0111-ib.pocstor_35625_object23792/head//22 by client.940330.0:23793 2014-08-11 16:31:11.390383 2014-08-13 18:06:06.485570 7f5c973a57a0 20 read_log 192888'478 (0'0) modify 45ea3ad1/benchmark_data_storage0111-ib.pocstor_35625_object25656/head//22 by client.940330.0:25657 2014-08-11 16:31:18.443058 2014-08-13 18:06:06.485582 7f5c973a57a0 20 read_log 192888'479 (0'0) modify d94e7ad1/benchmark_data_storage0111-ib.pocstor_35625_object27182/head//22 by client.940330.0:27183 2014-08-11 16:31:24.506100 2014-08-13 18:06:06.485605 7f5c973a57a0 20 read_log 192888'480 (0'0) modify 160f5ad1/benchmark_data_storage0111-ib.pocstor_35625_object27494/head//22 by client.940330.0:27495 2014-08-11 16:31:25.714233 2014-08-13 18:06:06.485621 7f5c973a57a0 20 read_log 192888'481 (0'0) modify 153ef2d1/benchmark_data_storage0111-ib.pocstor_35625_object27643/head//22 by client.940330.0:27644 2014-08-11 16:31:26.296654 2014-08-13 18:06:06.485634 7f5c973a57a0 20 read_log 192888'482 (0'0) modify 34074ad1/benchmark_data_storage0111-ib.pocstor_35625_object28159/head//22 by client.940330.0:28160 2014-08-11 16:31:28.352484 2014-08-13 18:06:06.485646 7f5c973a57a0 20 read_log 192888'483 (0'0) modify b0cf2ad1/benchmark_data_storage0111-ib.pocstor_35625_object30237/head//22 by client.940330.0:30238 2014-08-11 16:31:36.355808 2014-08-13 18:06:06.485658 7f5c973a57a0 20 read_log 192888'484 (0'0) modify e61dad1/benchmark_data_storage0111-ib.pocstor_35625_object30899/head//22 by client.940330.0:30900 2014-08-11 16:31:38.836073 2014-08-13 18:06:06.485679 7f5c973a57a0 20 read_log 192888'485 (192888'469) delete 5ee3e2d1/benchmark_data_storage0111-ib.pocstor_35625_object2742/head//22 by client.940330.0:34050 2014-08-11 16:31:42.181877 2014-08-13 18:06:06.485694 7f5c973a57a0 20 read_log 192888'486 (192888'470) delete 7d6252d1/benchmark_data_storage0111-ib.pocstor_35625_object2854/head//22 by client.940330.0:34162 2014-08-11 16:31:42.212112 2014-08-13 18:06:06.485706 7f5c973a57a0 20 read_log 192888'487 (192888'471) delete 15bb5ad1/benchmark_data_storage0111-ib.pocstor_35625_object3322/head//22 by client.940330.0:34630 2014-08-11 16:31:42.341265 2014-08-13 18:06:06.485718 7f5c973a57a0 20 read_log 192888'488 (192888'472) delete 232732d1/benchmark_data_storage0111-ib.pocstor_35625_object10456/head//22 by client.940330.0:41764 2014-08-11 16:31:44.240347 2014-08-13 18:06:06.485731 7f5c973a57a0 20 read_log 192888'489 (192888'473) delete 33b6b2d1/benchmark_data_storage0111-ib.pocstor_35625_object10838/head//22 by client.940330.0:42146 2014-08-11 16:31:44.352435 2014-08-13 18:06:06.485744 7f5c973a57a0 20 read_log 192888'490 (192888'474) delete 891c8ad1/benchmark_data_storage0111-ib.pocstor_35625_object10903/head//22 by client.940330.0:42211 2014-08-11 16:31:44.370802 2014-08-13 18:06:06.485757 7f5c973a57a0 20 read_log 192888'491 (192888'475) delete 841902d1/benchmark_data_storage0111-ib.pocstor_35625_object11927/head//22 by client.940330.0:43235 2014-08-11 16:31:44.638929 2014-08-13 18:06:06.485769 7f5c973a57a0 20 read_log 192888'492 (192888'476) delete c26f02d1/benchmark_data_storage0111-ib.pocstor_35625_object22095/head//22 by client.940330.0:53403 2014-08-11 16:31:47.472130 2014-08-13 18:06:06.485781 7f5c973a57a0 20 read_log 192888'493 (192888'477) delete cbe69ad1/benchmark_data_storage0111-ib.pocstor_35625_object23792/head//22 by client.940330.0:55100 2014-08-11 16:31:47.974299 2014-08-13 18:06:06.485793 7f5c973a57a0 20 read_log 192888'494 (192888'478) delete 45ea3ad1/benchmark_data_storage0111-ib.pocstor_35625_object25656/head//22 by client.940330.0:56964 2014-08-11 16:31:48.545771 2014-08-13 18:06:06.485805 7f5c973a57a0 20 read_log 192888'495 (192888'479) delete d94e7ad1/benchmark_data_storage0111-ib.pocstor_35625_object27182/head//22 by client.940330.0:58490 2014-08-11 16:31:48.978093 2014-08-13 18:06:06.485817 7f5c973a57a0 20 read_log 192888'496 (192888'480) delete 160f5ad1/benchmark_data_storage0111-ib.pocstor_35625_object27494/head//22 by client.940330.0:58802 2014-08-11 16:31:49.063934 2014-08-13 18:06:06.485830 7f5c973a57a0 20 read_log 192888'497 (192888'481) delete 153ef2d1/benchmark_data_storage0111-ib.pocstor_35625_object27643/head//22 by client.940330.0:58951 2014-08-11 16:31:49.105221 2014-08-13 18:06:06.485844 7f5c973a57a0 20 read_log 192888'498 (192888'482) delete 34074ad1/benchmark_data_storage0111-ib.pocstor_35625_object28159/head//22 by client.940330.0:59467 2014-08-11 16:31:49.248838 2014-08-13 18:06:06.485857 7f5c973a57a0 20 read_log 192888'499 (192888'483) delete b0cf2ad1/benchmark_data_storage0111-ib.pocstor_35625_object30237/head//22 by client.940330.0:61545 2014-08-11 16:31:49.826953 2014-08-13 18:06:06.485869 7f5c973a57a0 20 read_log 192888'500 (192888'484) delete e61dad1/benchmark_data_storage0111-ib.pocstor_35625_object30899/head//22 by client.940330.0:62207 2014-08-11 16:31:50.023502 2014-08-13 18:06:06.485882 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 99a1d2d1/benchmark_data_storage0111-ib.pocstor_64498_object3998/head//22 by client.1049635.0:3999 2014-08-12 17:17:30.080865 2014-08-13 18:06:06.485896 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 3493cad1/benchmark_data_storage0111-ib.pocstor_64498_object7740/head//22 by client.1049635.0:7741 2014-08-12 17:17:41.278668 2014-08-13 18:06:06.485909 7f5c973a57a0 20 read_log 194215'503 (0'0) modify 4268cad1/benchmark_data_storage0111-ib.pocstor_64498_object8489/head//22 by client.1049635.0:8490 2014-08-12 17:17:43.528281 2014-08-13 18:06:06.485923 7f5c973a57a0 20 read_log 194215'504 (0'0) modify 30f31ad1/benchmark_data_storage0111-ib.pocstor_64498_object10219/head//22 by client.1049635.0:10220 2014-08-12 17:17:48.549443 2014-08-13 18:06:06.485942 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 7ceea2d1/benchmark_data_storage0111-ib.pocstor_64498_object13546/head//22 by client.1049635.0:13547 2014-08-12 17:17:59.146307 2014-08-13 18:06:06.485957 7f5c973a57a0 20 read_log 194215'506 (0'0) modify 70405ad1/benchmark_data_storage0111-ib.pocstor_64498_object18763/head//22 by client.1049635.0:18764 2014-08-12 17:18:18.042437 2014-08-13 18:06:06.485970 7f5c973a57a0 20 read_log 194215'507 (0'0) modify e09ae2d1/benchmark_data_storage0111-ib.pocstor_64498_object20569/head//22 by client.1049635.0:20570 2014-08-12 17:18:24.926539 2014-08-13 18:06:06.485982 7f5c973a57a0 20 read_log 194215'508 (0'0) modify 411892d1/benchmark_data_storage0111-ib.pocstor_64498_object25202/head//22 by client.1049635.0:25203 2014-08-12 17:18:42.459424 2014-08-13 18:06:06.485994 7f5c973a57a0 20 read_log 194215'509 (0'0) modify 6e233ad1/benchmark_data_storage0111-ib.pocstor_64498_object25327/head//22 by client.1049635.0:25328 2014-08-12 17:18:42.926392 2014-08-13 18:06:06.486008 7f5c973a57a0 20 read_log 194215'510 (0'0) modify 874ddad1/benchmark_data_storage0111-ib.pocstor_64498_object25950/head//22 by client.1049635.0:25951 2014-08-12 17:18:45.240085 2014-08-13 18:06:06.486021 7f5c973a57a0 20 read_log 194215'511 (0'0) modify 4a9dcad1/benchmark_data_storage0111-ib.pocstor_64498_object26261/head//22 by client.1049635.0:26262 2014-08-12 17:18:46.393837 2014-08-13 18:06:06.486035 7f5c973a57a0 20 read_log 194215'512 (0'0) modify 504c7ad1/benchmark_data_storage0111-ib.pocstor_64498_object27514/head//22 by client.1049635.0:27515 2014-08-12 17:18:51.073620 2014-08-13 18:06:06.486047 7f5c973a57a0 20 read_log 194215'513 (0'0) modify 8934e2d1/benchmark_data_storage0111-ib.pocstor_64498_object31138/head//22 by client.1049635.0:31139 2014-08-12 17:19:05.099541 2014-08-13 18:06:06.486062 7f5c973a57a0 20 read_log 194215'514 (194215'501) delete 99a1d2d1/benchmark_data_storage0111-ib.pocstor_64498_object3998/head//22 by client.1049635.0:38201 2014-08-12 17:19:19.274120 2014-08-13 18:06:06.486077 7f5c973a57a0 20 read_log 194215'515 (194215'502) delete 3493cad1/benchmark_data_storage0111-ib.pocstor_64498_object7740/head//22 by client.1049635.0:41943 2014-08-12 17:19:20.523490 2014-08-13 18:06:06.486089 7f5c973a57a0 20 read_log 194215'516 (194215'503) delete 4268cad1/benchmark_data_storage0111-ib.pocstor_64498_object8489/head//22 by client.1049635.0:42692 2014-08-12 17:19:20.756972 2014-08-13 18:06:06.486103 7f5c973a57a0 20 read_log 194215'517 (194215'504) delete 30f31ad1/benchmark_data_storage0111-ib.pocstor_64498_object10219/head//22 by client.1049635.0:44422 2014-08-12 17:19:21.335209 2014-08-13 18:06:06.486116 7f5c973a57a0 20 read_log 194215'518 (194215'505) delete 7ceea2d1/benchmark_data_storage0111-ib.pocstor_64498_object13546/head//22 by client.1049635.0:47749 2014-08-12 17:19:22.416834 2014-08-13 18:06:06.486130 7f5c973a57a0 20 read_log 194215'519 (194215'506) delete 70405ad1/benchmark_data_storage0111-ib.pocstor_64498_object18763/head//22 by client.1049635.0:52966 2014-08-12 17:19:24.085344 2014-08-13 18:06:06.486143 7f5c973a57a0 20 read_log 194215'520 (194215'507) delete e09ae2d1/benchmark_data_storage0111-ib.pocstor_64498_object20569/head//22 by client.1049635.0:54772 2014-08-12 17:19:24.729533 2014-08-13 18:06:06.486180 7f5c973a57a0 20 read_log 194215'521 (194215'508) delete 411892d1/benchmark_data_storage0111-ib.pocstor_64498_object25202/head//22 by client.1049635.0:59405 2014-08-12 17:19:26.390213 2014-08-13 18:06:06.486197 7f5c973a57a0 20 read_log 194215'522 (194215'509) delete 6e233ad1/benchmark_data_storage0111-ib.pocstor_64498_object25327/head//22 by client.1049635.0:59530 2014-08-12 17:19:26.428187 2014-08-13 18:06:06.486209 7f5c973a57a0 20 read_log 194215'523 (194215'510) delete 874ddad1/benchmark_data_storage0111-ib.pocstor_64498_object25950/head//22 by client.1049635.0:60153 2014-08-12 17:19:26.639323 2014-08-13 18:06:06.486272 7f5c973a57a0 20 read_log 194215'524 (194215'511) delete 4a9dcad1/benchmark_data_storage0111-ib.pocstor_64498_object26261/head//22 by client.1049635.0:60464 2014-08-12 17:19:26.745244 2014-08-13 18:06:06.486289 7f5c973a57a0 20 read_log 194215'525 (194215'512) delete 504c7ad1/benchmark_data_storage0111-ib.pocstor_64498_object27514/head//22 by client.1049635.0:61717 2014-08-12 17:19:27.186207 2014-08-13 18:06:06.486302 7f5c973a57a0 20 read_log 194215'526 (194215'513) delete 8934e2d1/benchmark_data_storage0111-ib.pocstor_64498_object31138/head//22 by client.1049635.0:65341 2014-08-12 17:19:28.428102 2014-08-13 18:06:06.486458 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.486484 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=0 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.486506 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=0 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Initial 0.002305 0 0.000000 2014-08-13 18:06:06.486519 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=0 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.486531 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=0 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.486540 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] log((0'0,194215'526], crt=192888'500) 2014-08-13 18:06:06.486550 7f5c973a57a0 10 osd.4 194487 pgid 22.30f coll 22.30f_head 2014-08-13 18:06:06.486687 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.30f 2014-08-13 18:06:06.486696 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.486722 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.30f(unlocked)] enter Initial 2014-08-13 18:06:06.486733 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.30f(unlocked)] enter NotTrimming 2014-08-13 18:06:06.486863 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.486971 7f5c973a57a0 20 read_log 191648'14 (0'0) modify 5257fb0f/benchmark_data_storage0111-ib.pocstor_40745_object1025/head//22 by client.273158.0:1026 2014-07-30 14:08:52.574943 2014-08-13 18:06:06.487015 7f5c973a57a0 20 read_log 191648'207 (0'0) modify 4417b30f/benchmark_data_storage0111-ib.pocstor_40745_object13399/head//22 by client.273158.0:13400 2014-07-30 14:09:54.272143 2014-08-13 18:06:06.487032 7f5c973a57a0 20 read_log 191648'262 (0'0) modify d4209b0f/benchmark_data_storage0111-ib.pocstor_40745_object16906/head//22 by client.273158.0:16907 2014-07-30 14:10:12.278627 2014-08-13 18:06:06.487064 7f5c973a57a0 20 read_log 191648'285 (0'0) modify ebabb0f/benchmark_data_storage0111-ib.pocstor_40745_object18866/head//22 by client.273158.0:18867 2014-07-30 14:10:23.030276 2014-08-13 18:06:06.487080 7f5c973a57a0 20 read_log 191648'307 (0'0) modify 39bab0f/benchmark_data_storage0111-ib.pocstor_40745_object20214/head//22 by client.273158.0:20215 2014-07-30 14:10:32.320903 2014-08-13 18:06:06.487092 7f5c973a57a0 20 read_log 191648'335 (0'0) modify e2cd530f/benchmark_data_storage0111-ib.pocstor_40745_object21690/head//22 by client.273158.0:21691 2014-07-30 14:10:41.219192 2014-08-13 18:06:06.487104 7f5c973a57a0 20 read_log 191648'338 (0'0) modify 17b1db0f/benchmark_data_storage0111-ib.pocstor_40745_object22062/head//22 by client.273158.0:22063 2014-07-30 14:10:44.432712 2014-08-13 18:06:06.487117 7f5c973a57a0 20 read_log 191727'357 (0'0) modify 22167b0f/benchmark_data_storage0111-ib.pocstor_51285_object4699/head//22 by client.275175.0:4700 2014-07-30 14:18:49.453647 2014-08-13 18:06:06.487129 7f5c973a57a0 20 read_log 191727'359 (0'0) modify bdb78b0f/benchmark_data_storage0111-ib.pocstor_51285_object5308/head//22 by client.275175.0:5309 2014-07-30 14:18:51.261359 2014-08-13 18:06:06.487141 7f5c973a57a0 20 read_log 191727'362 (0'0) modify 1761cb0f/benchmark_data_storage0111-ib.pocstor_51285_object8972/head//22 by client.275175.0:8973 2014-07-30 14:19:02.269923 2014-08-13 18:06:06.487166 7f5c973a57a0 20 read_log 191727'367 (0'0) modify a8aea30f/benchmark_data_storage0111-ib.pocstor_51285_object11092/head//22 by client.275175.0:11093 2014-07-30 14:19:08.302941 2014-08-13 18:06:06.487179 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 3eedfb0f/benchmark_data_storage0111-ib.pocstor_51285_object11862/head//22 by client.275175.0:11863 2014-07-30 14:19:10.909377 2014-08-13 18:06:06.487191 7f5c973a57a0 20 read_log 191727'370 (0'0) modify 8ac5db0f/benchmark_data_storage0111-ib.pocstor_51285_object12222/head//22 by client.275175.0:12223 2014-07-30 14:19:12.261947 2014-08-13 18:06:06.487204 7f5c973a57a0 20 read_log 191727'374 (0'0) modify f534cb0f/benchmark_data_storage0111-ib.pocstor_51285_object13640/head//22 by client.275175.0:13641 2014-07-30 14:19:16.640561 2014-08-13 18:06:06.487216 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 2983130f/benchmark_data_storage0111-ib.pocstor_51285_object18003/head//22 by client.275175.0:18004 2014-07-30 14:19:29.967637 2014-08-13 18:06:06.487228 7f5c973a57a0 20 read_log 191727'380 (0'0) modify c59ab0f/benchmark_data_storage0111-ib.pocstor_51285_object18111/head//22 by client.275175.0:18112 2014-07-30 14:19:30.266706 2014-08-13 18:06:06.487243 7f5c973a57a0 20 read_log 191727'381 (0'0) modify b2726b0f/benchmark_data_storage0111-ib.pocstor_51285_object18395/head//22 by client.275175.0:18396 2014-07-30 14:19:31.178824 2014-08-13 18:06:06.487256 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 4c94630f/benchmark_data_storage0111-ib.pocstor_51285_object18541/head//22 by client.275175.0:18542 2014-07-30 14:19:31.602079 2014-08-13 18:06:06.487268 7f5c973a57a0 20 read_log 191727'389 (0'0) modify f7d22b0f/benchmark_data_storage0111-ib.pocstor_51285_object23433/head//22 by client.275175.0:23434 2014-07-30 14:19:47.249000 2014-08-13 18:06:06.487280 7f5c973a57a0 20 read_log 191727'394 (0'0) modify e8f9c30f/benchmark_data_storage0111-ib.pocstor_51285_object28246/head//22 by client.275175.0:28247 2014-07-30 14:20:01.717667 2014-08-13 18:06:06.487300 7f5c973a57a0 20 read_log 191727'395 (0'0) modify a59f830f/benchmark_data_storage0111-ib.pocstor_51285_object28870/head//22 by client.275175.0:28871 2014-07-30 14:20:03.527055 2014-08-13 18:06:06.487315 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 534c0b0f/benchmark_data_storage0111-ib.pocstor_51285_object28997/head//22 by client.275175.0:28998 2014-07-30 14:20:03.880283 2014-08-13 18:06:06.487327 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 1a415b0f/benchmark_data_storage0111-ib.pocstor_51285_object31144/head//22 by client.275175.0:31145 2014-07-30 14:20:10.396827 2014-08-13 18:06:06.487339 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 471e6b0f/benchmark_data_storage0111-ib.pocstor_51285_object33212/head//22 by client.275175.0:33213 2014-07-30 14:20:16.456685 2014-08-13 18:06:06.487351 7f5c973a57a0 20 read_log 191727'405 (0'0) modify 20ae0b0f/benchmark_data_storage0111-ib.pocstor_51285_object36282/head//22 by client.275175.0:36283 2014-07-30 14:20:26.143656 2014-08-13 18:06:06.487377 7f5c973a57a0 20 read_log 191727'406 (0'0) modify fe11330f/benchmark_data_storage0111-ib.pocstor_51285_object36427/head//22 by client.275175.0:36428 2014-07-30 14:20:26.555124 2014-08-13 18:06:06.487390 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 8426d30f/benchmark_data_storage0111-ib.pocstor_51285_object36896/head//22 by client.275175.0:36897 2014-07-30 14:20:28.086699 2014-08-13 18:06:06.487403 7f5c973a57a0 20 read_log 192888'412 (0'0) modify 6a64b30f/benchmark_data_storage0111-ib.pocstor_35625_object426/head//22 by client.940330.0:427 2014-08-11 16:29:42.386662 2014-08-13 18:06:06.487435 7f5c973a57a0 20 read_log 192888'413 (0'0) modify bd36830f/benchmark_data_storage0111-ib.pocstor_35625_object10675/head//22 by client.940330.0:10676 2014-08-11 16:30:20.536512 2014-08-13 18:06:06.487450 7f5c973a57a0 20 read_log 192888'414 (0'0) modify 43beb0f/benchmark_data_storage0111-ib.pocstor_35625_object11193/head//22 by client.940330.0:11194 2014-08-11 16:30:22.556156 2014-08-13 18:06:06.487470 7f5c973a57a0 20 read_log 192888'415 (0'0) modify d6919b0f/benchmark_data_storage0111-ib.pocstor_35625_object11219/head//22 by client.940330.0:11220 2014-08-11 16:30:22.659707 2014-08-13 18:06:06.487484 7f5c973a57a0 20 read_log 192888'416 (0'0) modify c9ad7b0f/benchmark_data_storage0111-ib.pocstor_35625_object18202/head//22 by client.940330.0:18203 2014-08-11 16:30:49.550388 2014-08-13 18:06:06.487497 7f5c973a57a0 20 read_log 192888'417 (0'0) modify 3598bb0f/benchmark_data_storage0111-ib.pocstor_35625_object19630/head//22 by client.940330.0:19631 2014-08-11 16:30:55.257991 2014-08-13 18:06:06.487509 7f5c973a57a0 20 read_log 192888'418 (0'0) modify 8f59430f/benchmark_data_storage0111-ib.pocstor_35625_object22716/head//22 by client.940330.0:22717 2014-08-11 16:31:07.352494 2014-08-13 18:06:06.487521 7f5c973a57a0 20 read_log 192888'419 (0'0) modify 203f9b0f/benchmark_data_storage0111-ib.pocstor_35625_object23891/head//22 by client.940330.0:23892 2014-08-11 16:31:11.750035 2014-08-13 18:06:06.487533 7f5c973a57a0 20 read_log 192888'420 (0'0) modify 11a26b0f/benchmark_data_storage0111-ib.pocstor_35625_object25834/head//22 by client.940330.0:25835 2014-08-11 16:31:19.140129 2014-08-13 18:06:06.487545 7f5c973a57a0 20 read_log 192888'421 (0'0) modify 3c5b5b0f/benchmark_data_storage0111-ib.pocstor_35625_object26759/head//22 by client.940330.0:26760 2014-08-11 16:31:22.871968 2014-08-13 18:06:06.487568 7f5c973a57a0 20 read_log 192888'422 (192888'412) delete 6a64b30f/benchmark_data_storage0111-ib.pocstor_35625_object426/head//22 by client.940330.0:31734 2014-08-11 16:31:41.601325 2014-08-13 18:06:06.487586 7f5c973a57a0 20 read_log 192888'423 (192888'413) delete bd36830f/benchmark_data_storage0111-ib.pocstor_35625_object10675/head//22 by client.940330.0:41983 2014-08-11 16:31:44.302893 2014-08-13 18:06:06.487615 7f5c973a57a0 20 read_log 192888'424 (192888'414) delete 43beb0f/benchmark_data_storage0111-ib.pocstor_35625_object11193/head//22 by client.940330.0:42501 2014-08-11 16:31:44.455158 2014-08-13 18:06:06.487631 7f5c973a57a0 20 read_log 192888'425 (192888'415) delete d6919b0f/benchmark_data_storage0111-ib.pocstor_35625_object11219/head//22 by client.940330.0:42527 2014-08-11 16:31:44.461937 2014-08-13 18:06:06.487644 7f5c973a57a0 20 read_log 192888'426 (192888'416) delete c9ad7b0f/benchmark_data_storage0111-ib.pocstor_35625_object18202/head//22 by client.940330.0:49510 2014-08-11 16:31:46.352948 2014-08-13 18:06:06.487656 7f5c973a57a0 20 read_log 192888'427 (192888'417) delete 3598bb0f/benchmark_data_storage0111-ib.pocstor_35625_object19630/head//22 by client.940330.0:50938 2014-08-11 16:31:46.759002 2014-08-13 18:06:06.487677 7f5c973a57a0 20 read_log 192888'428 (192888'418) delete 8f59430f/benchmark_data_storage0111-ib.pocstor_35625_object22716/head//22 by client.940330.0:54024 2014-08-11 16:31:47.669167 2014-08-13 18:06:06.487693 7f5c973a57a0 20 read_log 192888'429 (192888'419) delete 203f9b0f/benchmark_data_storage0111-ib.pocstor_35625_object23891/head//22 by client.940330.0:55199 2014-08-11 16:31:48.003009 2014-08-13 18:06:06.487706 7f5c973a57a0 20 read_log 192888'430 (192888'420) delete 11a26b0f/benchmark_data_storage0111-ib.pocstor_35625_object25834/head//22 by client.940330.0:57142 2014-08-11 16:31:48.597797 2014-08-13 18:06:06.487718 7f5c973a57a0 20 read_log 192888'431 (192888'421) delete 3c5b5b0f/benchmark_data_storage0111-ib.pocstor_35625_object26759/head//22 by client.940330.0:58067 2014-08-11 16:31:48.859550 2014-08-13 18:06:06.487731 7f5c973a57a0 20 read_log 194215'432 (0'0) modify 8dbbab0f/benchmark_data_storage0111-ib.pocstor_64498_object422/head//22 by client.1049635.0:423 2014-08-12 17:17:18.504715 2014-08-13 18:06:06.487743 7f5c973a57a0 20 read_log 194215'433 (0'0) modify 9bd41b0f/benchmark_data_storage0111-ib.pocstor_64498_object2589/head//22 by client.1049635.0:2590 2014-08-12 17:17:25.506150 2014-08-13 18:06:06.487755 7f5c973a57a0 20 read_log 194215'434 (0'0) modify 827e730f/benchmark_data_storage0111-ib.pocstor_64498_object2931/head//22 by client.1049635.0:2932 2014-08-12 17:17:26.603816 2014-08-13 18:06:06.487782 7f5c973a57a0 20 read_log 194215'435 (0'0) modify 8e27830f/benchmark_data_storage0111-ib.pocstor_64498_object2978/head//22 by client.1049635.0:2979 2014-08-12 17:17:26.777824 2014-08-13 18:06:06.487796 7f5c973a57a0 20 read_log 194215'436 (0'0) modify 546f6b0f/benchmark_data_storage0111-ib.pocstor_64498_object3949/head//22 by client.1049635.0:3950 2014-08-12 17:17:29.933062 2014-08-13 18:06:06.487808 7f5c973a57a0 20 read_log 194215'437 (0'0) modify 3f5d730f/benchmark_data_storage0111-ib.pocstor_64498_object6863/head//22 by client.1049635.0:6864 2014-08-12 17:17:38.693985 2014-08-13 18:06:06.487820 7f5c973a57a0 20 read_log 194215'438 (0'0) modify 6d99130f/benchmark_data_storage0111-ib.pocstor_64498_object6897/head//22 by client.1049635.0:6898 2014-08-12 17:17:38.820152 2014-08-13 18:06:06.487833 7f5c973a57a0 20 read_log 194215'439 (0'0) modify e5f43b0f/benchmark_data_storage0111-ib.pocstor_64498_object9706/head//22 by client.1049635.0:9707 2014-08-12 17:17:47.090526 2014-08-13 18:06:06.487845 7f5c973a57a0 20 read_log 194215'440 (0'0) modify 92df230f/benchmark_data_storage0111-ib.pocstor_64498_object10963/head//22 by client.1049635.0:10964 2014-08-12 17:17:50.987623 2014-08-13 18:06:06.487857 7f5c973a57a0 20 read_log 194215'441 (0'0) modify e0a1630f/benchmark_data_storage0111-ib.pocstor_64498_object12246/head//22 by client.1049635.0:12247 2014-08-12 17:17:55.203215 2014-08-13 18:06:06.487869 7f5c973a57a0 20 read_log 194215'442 (0'0) modify 561af30f/benchmark_data_storage0111-ib.pocstor_64498_object20013/head//22 by client.1049635.0:20014 2014-08-12 17:18:22.827657 2014-08-13 18:06:06.487891 7f5c973a57a0 20 read_log 194215'443 (0'0) modify 6012730f/benchmark_data_storage0111-ib.pocstor_64498_object21158/head//22 by client.1049635.0:21159 2014-08-12 17:18:27.184614 2014-08-13 18:06:06.487906 7f5c973a57a0 20 read_log 194215'444 (0'0) modify c09930f/benchmark_data_storage0111-ib.pocstor_64498_object22881/head//22 by client.1049635.0:22882 2014-08-12 17:18:33.694128 2014-08-13 18:06:06.487918 7f5c973a57a0 20 read_log 194215'445 (0'0) modify e433430f/benchmark_data_storage0111-ib.pocstor_64498_object26306/head//22 by client.1049635.0:26307 2014-08-12 17:18:46.561042 2014-08-13 18:06:06.487931 7f5c973a57a0 20 read_log 194215'446 (0'0) modify fbe3fb0f/benchmark_data_storage0111-ib.pocstor_64498_object26769/head//22 by client.1049635.0:26770 2014-08-12 17:18:48.291820 2014-08-13 18:06:06.487943 7f5c973a57a0 20 read_log 194215'447 (0'0) modify 79d5030f/benchmark_data_storage0111-ib.pocstor_64498_object27485/head//22 by client.1049635.0:27486 2014-08-12 17:18:50.967995 2014-08-13 18:06:06.487955 7f5c973a57a0 20 read_log 194215'448 (0'0) modify c54c4b0f/benchmark_data_storage0111-ib.pocstor_64498_object29235/head//22 by client.1049635.0:29236 2014-08-12 17:18:57.468862 2014-08-13 18:06:06.487967 7f5c973a57a0 20 read_log 194215'449 (0'0) modify b20d530f/benchmark_data_storage0111-ib.pocstor_64498_object30511/head//22 by client.1049635.0:30512 2014-08-12 17:19:02.541578 2014-08-13 18:06:06.487979 7f5c973a57a0 20 read_log 194215'450 (0'0) modify 38837b0f/benchmark_data_storage0111-ib.pocstor_64498_object30555/head//22 by client.1049635.0:30556 2014-08-12 17:19:02.708705 2014-08-13 18:06:06.487992 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 64ba1b0f/benchmark_data_storage0111-ib.pocstor_64498_object31114/head//22 by client.1049635.0:31115 2014-08-12 17:19:05.003841 2014-08-13 18:06:06.488004 7f5c973a57a0 20 read_log 194215'452 (0'0) modify 81b69b0f/benchmark_data_storage0111-ib.pocstor_64498_object32132/head//22 by client.1049635.0:32133 2014-08-12 17:19:09.076002 2014-08-13 18:06:06.488016 7f5c973a57a0 20 read_log 194215'453 (194215'432) delete 8dbbab0f/benchmark_data_storage0111-ib.pocstor_64498_object422/head//22 by client.1049635.0:34625 2014-08-12 17:19:18.135736 2014-08-13 18:06:06.488029 7f5c973a57a0 20 read_log 194215'454 (194215'433) delete 9bd41b0f/benchmark_data_storage0111-ib.pocstor_64498_object2589/head//22 by client.1049635.0:36792 2014-08-12 17:19:18.827465 2014-08-13 18:06:06.488041 7f5c973a57a0 20 read_log 194215'455 (194215'434) delete 827e730f/benchmark_data_storage0111-ib.pocstor_64498_object2931/head//22 by client.1049635.0:37134 2014-08-12 17:19:18.947392 2014-08-13 18:06:06.488053 7f5c973a57a0 20 read_log 194215'456 (194215'435) delete 8e27830f/benchmark_data_storage0111-ib.pocstor_64498_object2978/head//22 by client.1049635.0:37181 2014-08-12 17:19:18.962142 2014-08-13 18:06:06.488066 7f5c973a57a0 20 read_log 194215'457 (194215'436) delete 546f6b0f/benchmark_data_storage0111-ib.pocstor_64498_object3949/head//22 by client.1049635.0:38152 2014-08-12 17:19:19.259394 2014-08-13 18:06:06.488078 7f5c973a57a0 20 read_log 194215'458 (194215'437) delete 3f5d730f/benchmark_data_storage0111-ib.pocstor_64498_object6863/head//22 by client.1049635.0:41066 2014-08-12 17:19:20.240054 2014-08-13 18:06:06.488092 7f5c973a57a0 20 read_log 194215'459 (194215'438) delete 6d99130f/benchmark_data_storage0111-ib.pocstor_64498_object6897/head//22 by client.1049635.0:41100 2014-08-12 17:19:20.250815 2014-08-13 18:06:06.488105 7f5c973a57a0 20 read_log 194215'460 (194215'439) delete e5f43b0f/benchmark_data_storage0111-ib.pocstor_64498_object9706/head//22 by client.1049635.0:43909 2014-08-12 17:19:21.156697 2014-08-13 18:06:06.488119 7f5c973a57a0 20 read_log 194215'461 (194215'440) delete 92df230f/benchmark_data_storage0111-ib.pocstor_64498_object10963/head//22 by client.1049635.0:45166 2014-08-12 17:19:21.567271 2014-08-13 18:06:06.488133 7f5c973a57a0 20 read_log 194215'462 (194215'441) delete e0a1630f/benchmark_data_storage0111-ib.pocstor_64498_object12246/head//22 by client.1049635.0:46449 2014-08-12 17:19:21.973284 2014-08-13 18:06:06.488161 7f5c973a57a0 20 read_log 194215'463 (194215'442) delete 561af30f/benchmark_data_storage0111-ib.pocstor_64498_object20013/head//22 by client.1049635.0:54216 2014-08-12 17:19:24.518900 2014-08-13 18:06:06.488178 7f5c973a57a0 20 read_log 194215'464 (194215'443) delete 6012730f/benchmark_data_storage0111-ib.pocstor_64498_object21158/head//22 by client.1049635.0:55361 2014-08-12 17:19:24.941039 2014-08-13 18:06:06.488191 7f5c973a57a0 20 read_log 194215'465 (194215'444) delete c09930f/benchmark_data_storage0111-ib.pocstor_64498_object22881/head//22 by client.1049635.0:57084 2014-08-12 17:19:25.569026 2014-08-13 18:06:06.488203 7f5c973a57a0 20 read_log 194215'466 (194215'445) delete e433430f/benchmark_data_storage0111-ib.pocstor_64498_object26306/head//22 by client.1049635.0:60509 2014-08-12 17:19:26.761825 2014-08-13 18:06:06.488217 7f5c973a57a0 20 read_log 194215'467 (194215'446) delete fbe3fb0f/benchmark_data_storage0111-ib.pocstor_64498_object26769/head//22 by client.1049635.0:60972 2014-08-12 17:19:26.914422 2014-08-13 18:06:06.488232 7f5c973a57a0 20 read_log 194215'468 (194215'447) delete 79d5030f/benchmark_data_storage0111-ib.pocstor_64498_object27485/head//22 by client.1049635.0:61688 2014-08-12 17:19:27.175483 2014-08-13 18:06:06.488244 7f5c973a57a0 20 read_log 194215'469 (194215'448) delete c54c4b0f/benchmark_data_storage0111-ib.pocstor_64498_object29235/head//22 by client.1049635.0:63438 2014-08-12 17:19:27.787522 2014-08-13 18:06:06.488260 7f5c973a57a0 20 read_log 194215'470 (194215'449) delete b20d530f/benchmark_data_storage0111-ib.pocstor_64498_object30511/head//22 by client.1049635.0:64714 2014-08-12 17:19:28.212271 2014-08-13 18:06:06.488274 7f5c973a57a0 20 read_log 194215'471 (194215'450) delete 38837b0f/benchmark_data_storage0111-ib.pocstor_64498_object30555/head//22 by client.1049635.0:64758 2014-08-12 17:19:28.227520 2014-08-13 18:06:06.488287 7f5c973a57a0 20 read_log 194215'472 (194215'451) delete 64ba1b0f/benchmark_data_storage0111-ib.pocstor_64498_object31114/head//22 by client.1049635.0:65317 2014-08-12 17:19:28.418275 2014-08-13 18:06:06.488302 7f5c973a57a0 20 read_log 194215'473 (194215'452) delete 81b69b0f/benchmark_data_storage0111-ib.pocstor_64498_object32132/head//22 by client.1049635.0:66335 2014-08-12 17:19:28.794563 2014-08-13 18:06:06.488394 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.488421 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=0 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.488447 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=0 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Initial 0.001725 0 0.000000 2014-08-13 18:06:06.488460 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=0 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.488469 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=0 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.488478 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] log((0'0,194215'473], crt=192888'431) 2014-08-13 18:06:06.488489 7f5c973a57a0 10 osd.4 194487 pgid 22.362 coll 22.362_head 2014-08-13 18:06:06.488656 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.362 2014-08-13 18:06:06.488666 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.488693 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.362(unlocked)] enter Initial 2014-08-13 18:06:06.488702 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.362(unlocked)] enter NotTrimming 2014-08-13 18:06:06.488915 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.489049 7f5c973a57a0 20 read_log 191648'4 (0'0) modify 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 by client.273158.0:281 2014-07-30 14:08:49.759341 2014-08-13 18:06:06.489070 7f5c973a57a0 20 read_log 191648'80 (0'0) modify 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 by client.273158.0:5691 2014-07-30 14:09:14.204942 2014-08-13 18:06:06.489083 7f5c973a57a0 20 read_log 191648'82 (0'0) modify 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 by client.273158.0:5737 2014-07-30 14:09:14.390026 2014-08-13 18:06:06.489095 7f5c973a57a0 20 read_log 191648'135 (0'0) modify 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 by client.273158.0:9883 2014-07-30 14:09:35.219906 2014-08-13 18:06:06.489124 7f5c973a57a0 20 read_log 191648'162 (0'0) modify 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 by client.273158.0:11469 2014-07-30 14:09:44.540936 2014-08-13 18:06:06.489139 7f5c973a57a0 20 read_log 191648'198 (0'0) modify 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 by client.273158.0:13764 2014-07-30 14:09:55.896649 2014-08-13 18:06:06.489179 7f5c973a57a0 20 read_log 191648'224 (0'0) modify 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 by client.273158.0:16213 2014-07-30 14:10:08.572385 2014-08-13 18:06:06.489195 7f5c973a57a0 20 read_log 191648'242 (0'0) modify cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 by client.273158.0:17384 2014-07-30 14:10:15.207746 2014-08-13 18:06:06.489208 7f5c973a57a0 20 read_log 191648'273 (0'0) modify 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 by client.273158.0:19090 2014-07-30 14:10:23.975481 2014-08-13 18:06:06.489221 7f5c973a57a0 20 read_log 191648'290 (0'0) modify 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 by client.273158.0:20003 2014-07-30 14:10:31.307318 2014-08-13 18:06:06.489260 7f5c973a57a0 20 read_log 191727'347 (0'0) modify 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 by client.275175.0:6281 2014-07-30 14:18:54.097670 2014-08-13 18:06:06.489276 7f5c973a57a0 20 read_log 191727'358 (0'0) modify 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 by client.275175.0:10814 2014-07-30 14:19:07.520159 2014-08-13 18:06:06.489288 7f5c973a57a0 20 read_log 191727'360 (0'0) modify 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 by client.275175.0:11507 2014-07-30 14:19:09.737911 2014-08-13 18:06:06.489311 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 by client.275175.0:12519 2014-07-30 14:19:13.090565 2014-08-13 18:06:06.489345 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 by client.275175.0:15582 2014-07-30 14:19:22.645697 2014-08-13 18:06:06.489361 7f5c973a57a0 20 read_log 191727'370 (0'0) modify 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 by client.275175.0:17541 2014-07-30 14:19:28.659125 2014-08-13 18:06:06.489373 7f5c973a57a0 20 read_log 191727'390 (0'0) modify bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 by client.275175.0:27026 2014-07-30 14:19:57.884253 2014-08-13 18:06:06.489386 7f5c973a57a0 20 read_log 191727'394 (0'0) modify c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 by client.275175.0:29144 2014-07-30 14:20:04.349500 2014-08-13 18:06:06.489398 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 by client.275175.0:29212 2014-07-30 14:20:04.571099 2014-08-13 18:06:06.489410 7f5c973a57a0 20 read_log 191727'397 (0'0) modify b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 by client.275175.0:29912 2014-07-30 14:20:06.697728 2014-08-13 18:06:06.489422 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 by client.275175.0:29970 2014-07-30 14:20:06.859872 2014-08-13 18:06:06.489434 7f5c973a57a0 20 read_log 191727'411 (0'0) modify a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 by client.275175.0:37515 2014-07-30 14:20:29.889612 2014-08-13 18:06:06.489447 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 by client.275175.0:38715 2014-07-30 14:20:33.280192 2014-08-13 18:06:06.489459 7f5c973a57a0 20 read_log 192888'416 (0'0) modify bfeb4b62/benchmark_data_storage0111-ib.pocstor_35625_object214/head//22 by client.940330.0:215 2014-08-11 16:29:41.514064 2014-08-13 18:06:06.489471 7f5c973a57a0 20 read_log 192888'417 (0'0) modify 3a11cb62/benchmark_data_storage0111-ib.pocstor_35625_object256/head//22 by client.940330.0:257 2014-08-11 16:29:41.744578 2014-08-13 18:06:06.489483 7f5c973a57a0 20 read_log 192888'418 (0'0) modify 9a2d3b62/benchmark_data_storage0111-ib.pocstor_35625_object7122/head//22 by client.940330.0:7123 2014-08-11 16:30:07.750277 2014-08-13 18:06:06.489495 7f5c973a57a0 20 read_log 192888'419 (0'0) modify 96e2cb62/benchmark_data_storage0111-ib.pocstor_35625_object7908/head//22 by client.940330.0:7909 2014-08-11 16:30:10.514287 2014-08-13 18:06:06.489507 7f5c973a57a0 20 read_log 192888'420 (0'0) modify 12f7b362/benchmark_data_storage0111-ib.pocstor_35625_object8754/head//22 by client.940330.0:8755 2014-08-11 16:30:13.473690 2014-08-13 18:06:06.489519 7f5c973a57a0 20 read_log 192888'421 (0'0) modify e572b62/benchmark_data_storage0111-ib.pocstor_35625_object10737/head//22 by client.940330.0:10738 2014-08-11 16:30:20.779742 2014-08-13 18:06:06.489542 7f5c973a57a0 20 read_log 192888'422 (0'0) modify d7500b62/benchmark_data_storage0111-ib.pocstor_35625_object11247/head//22 by client.940330.0:11248 2014-08-11 16:30:22.767286 2014-08-13 18:06:06.489557 7f5c973a57a0 20 read_log 192888'423 (0'0) modify 9600b362/benchmark_data_storage0111-ib.pocstor_35625_object11628/head//22 by client.940330.0:11629 2014-08-11 16:30:24.275568 2014-08-13 18:06:06.489570 7f5c973a57a0 20 read_log 192888'424 (0'0) modify ffb48362/benchmark_data_storage0111-ib.pocstor_35625_object13210/head//22 by client.940330.0:13211 2014-08-11 16:30:30.701867 2014-08-13 18:06:06.489582 7f5c973a57a0 20 read_log 192888'425 (0'0) modify f5d1e362/benchmark_data_storage0111-ib.pocstor_35625_object15540/head//22 by client.940330.0:15541 2014-08-11 16:30:39.506202 2014-08-13 18:06:06.489595 7f5c973a57a0 20 read_log 192888'426 (0'0) modify dc951362/benchmark_data_storage0111-ib.pocstor_35625_object17520/head//22 by client.940330.0:17521 2014-08-11 16:30:46.902659 2014-08-13 18:06:06.489607 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 5779f362/benchmark_data_storage0111-ib.pocstor_35625_object22069/head//22 by client.940330.0:22070 2014-08-11 16:31:04.828566 2014-08-13 18:06:06.489619 7f5c973a57a0 20 read_log 192888'428 (0'0) modify e1296b62/benchmark_data_storage0111-ib.pocstor_35625_object24856/head//22 by client.940330.0:24857 2014-08-11 16:31:15.313885 2014-08-13 18:06:06.489631 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 7a051b62/benchmark_data_storage0111-ib.pocstor_35625_object29317/head//22 by client.940330.0:29318 2014-08-11 16:31:32.875685 2014-08-13 18:06:06.489643 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 6294362/benchmark_data_storage0111-ib.pocstor_35625_object29614/head//22 by client.940330.0:29615 2014-08-11 16:31:34.002687 2014-08-13 18:06:06.489655 7f5c973a57a0 20 read_log 192888'431 (0'0) modify c6045b62/benchmark_data_storage0111-ib.pocstor_35625_object30231/head//22 by client.940330.0:30232 2014-08-11 16:31:36.332403 2014-08-13 18:06:06.489667 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 91906362/benchmark_data_storage0111-ib.pocstor_35625_object30275/head//22 by client.940330.0:30276 2014-08-11 16:31:36.500706 2014-08-13 18:06:06.489679 7f5c973a57a0 20 read_log 192888'433 (192888'416) delete bfeb4b62/benchmark_data_storage0111-ib.pocstor_35625_object214/head//22 by client.940330.0:31522 2014-08-11 16:31:41.528597 2014-08-13 18:06:06.489713 7f5c973a57a0 20 read_log 192888'434 (192888'417) delete 3a11cb62/benchmark_data_storage0111-ib.pocstor_35625_object256/head//22 by client.940330.0:31564 2014-08-11 16:31:41.547229 2014-08-13 18:06:06.489726 7f5c973a57a0 20 read_log 192888'435 (192888'418) delete 9a2d3b62/benchmark_data_storage0111-ib.pocstor_35625_object7122/head//22 by client.940330.0:38430 2014-08-11 16:31:43.365382 2014-08-13 18:06:06.489741 7f5c973a57a0 20 read_log 192888'436 (192888'419) delete 96e2cb62/benchmark_data_storage0111-ib.pocstor_35625_object7908/head//22 by client.940330.0:39216 2014-08-11 16:31:43.566807 2014-08-13 18:06:06.489753 7f5c973a57a0 20 read_log 192888'437 (192888'420) delete 12f7b362/benchmark_data_storage0111-ib.pocstor_35625_object8754/head//22 by client.940330.0:40062 2014-08-11 16:31:43.778644 2014-08-13 18:06:06.489768 7f5c973a57a0 20 read_log 192888'438 (192888'421) delete e572b62/benchmark_data_storage0111-ib.pocstor_35625_object10737/head//22 by client.940330.0:42045 2014-08-11 16:31:44.323252 2014-08-13 18:06:06.489782 7f5c973a57a0 20 read_log 192888'439 (192888'422) delete d7500b62/benchmark_data_storage0111-ib.pocstor_35625_object11247/head//22 by client.940330.0:42555 2014-08-11 16:31:44.470958 2014-08-13 18:06:06.489797 7f5c973a57a0 20 read_log 192888'440 (192888'423) delete 9600b362/benchmark_data_storage0111-ib.pocstor_35625_object11628/head//22 by client.940330.0:42936 2014-08-11 16:31:44.568683 2014-08-13 18:06:06.489811 7f5c973a57a0 20 read_log 192888'441 (192888'424) delete ffb48362/benchmark_data_storage0111-ib.pocstor_35625_object13210/head//22 by client.940330.0:44518 2014-08-11 16:31:44.977500 2014-08-13 18:06:06.489832 7f5c973a57a0 20 read_log 192888'442 (192888'425) delete f5d1e362/benchmark_data_storage0111-ib.pocstor_35625_object15540/head//22 by client.940330.0:46848 2014-08-11 16:31:45.601562 2014-08-13 18:06:06.489847 7f5c973a57a0 20 read_log 192888'443 (192888'426) delete dc951362/benchmark_data_storage0111-ib.pocstor_35625_object17520/head//22 by client.940330.0:48828 2014-08-11 16:31:46.133742 2014-08-13 18:06:06.489860 7f5c973a57a0 20 read_log 192888'444 (192888'427) delete 5779f362/benchmark_data_storage0111-ib.pocstor_35625_object22069/head//22 by client.940330.0:53377 2014-08-11 16:31:47.463568 2014-08-13 18:06:06.489872 7f5c973a57a0 20 read_log 192888'445 (192888'428) delete e1296b62/benchmark_data_storage0111-ib.pocstor_35625_object24856/head//22 by client.940330.0:56164 2014-08-11 16:31:48.292418 2014-08-13 18:06:06.489885 7f5c973a57a0 20 read_log 192888'446 (192888'429) delete 7a051b62/benchmark_data_storage0111-ib.pocstor_35625_object29317/head//22 by client.940330.0:60625 2014-08-11 16:31:49.558048 2014-08-13 18:06:06.489899 7f5c973a57a0 20 read_log 192888'447 (192888'430) delete 6294362/benchmark_data_storage0111-ib.pocstor_35625_object29614/head//22 by client.940330.0:60922 2014-08-11 16:31:49.647528 2014-08-13 18:06:06.489912 7f5c973a57a0 20 read_log 192888'448 (192888'431) delete c6045b62/benchmark_data_storage0111-ib.pocstor_35625_object30231/head//22 by client.940330.0:61539 2014-08-11 16:31:49.823001 2014-08-13 18:06:06.489926 7f5c973a57a0 20 read_log 192888'449 (192888'432) delete 91906362/benchmark_data_storage0111-ib.pocstor_35625_object30275/head//22 by client.940330.0:61583 2014-08-11 16:31:49.837947 2014-08-13 18:06:06.489941 7f5c973a57a0 20 read_log 194215'450 (0'0) modify a6c07362/benchmark_data_storage0111-ib.pocstor_64498_object7055/head//22 by client.1049635.0:7056 2014-08-12 17:17:39.287541 2014-08-13 18:06:06.489958 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 2a611362/benchmark_data_storage0111-ib.pocstor_64498_object9383/head//22 by client.1049635.0:9384 2014-08-12 17:17:46.137210 2014-08-13 18:06:06.489972 7f5c973a57a0 20 read_log 194215'452 (0'0) modify 7f7d2362/benchmark_data_storage0111-ib.pocstor_64498_object10645/head//22 by client.1049635.0:10646 2014-08-12 17:17:50.063541 2014-08-13 18:06:06.489985 7f5c973a57a0 20 read_log 194215'453 (0'0) modify 9d377362/benchmark_data_storage0111-ib.pocstor_64498_object10960/head//22 by client.1049635.0:10961 2014-08-12 17:17:50.979302 2014-08-13 18:06:06.489999 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 446fb62/benchmark_data_storage0111-ib.pocstor_64498_object12968/head//22 by client.1049635.0:12969 2014-08-12 17:17:57.264085 2014-08-13 18:06:06.490011 7f5c973a57a0 20 read_log 194215'455 (0'0) modify 69232b62/benchmark_data_storage0111-ib.pocstor_64498_object13029/head//22 by client.1049635.0:13030 2014-08-12 17:17:57.479853 2014-08-13 18:06:06.490026 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 8dbd8362/benchmark_data_storage0111-ib.pocstor_64498_object13953/head//22 by client.1049635.0:13954 2014-08-12 17:18:00.383857 2014-08-13 18:06:06.490041 7f5c973a57a0 20 read_log 194215'457 (0'0) modify b6528b62/benchmark_data_storage0111-ib.pocstor_64498_object14620/head//22 by client.1049635.0:14621 2014-08-12 17:18:02.676161 2014-08-13 18:06:06.490055 7f5c973a57a0 20 read_log 194215'458 (0'0) modify 6d78c362/benchmark_data_storage0111-ib.pocstor_64498_object17345/head//22 by client.1049635.0:17346 2014-08-12 17:18:12.683975 2014-08-13 18:06:06.490069 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 8c33c362/benchmark_data_storage0111-ib.pocstor_64498_object17682/head//22 by client.1049635.0:17683 2014-08-12 17:18:13.953180 2014-08-13 18:06:06.490083 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 6b28e362/benchmark_data_storage0111-ib.pocstor_64498_object19659/head//22 by client.1049635.0:19660 2014-08-12 17:18:21.504723 2014-08-13 18:06:06.490103 7f5c973a57a0 20 read_log 194215'461 (0'0) modify c0644b62/benchmark_data_storage0111-ib.pocstor_64498_object25743/head//22 by client.1049635.0:25744 2014-08-12 17:18:44.469757 2014-08-13 18:06:06.490117 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 8023e362/benchmark_data_storage0111-ib.pocstor_64498_object26771/head//22 by client.1049635.0:26772 2014-08-12 17:18:48.299105 2014-08-13 18:06:06.490130 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 343362/benchmark_data_storage0111-ib.pocstor_64498_object29465/head//22 by client.1049635.0:29466 2014-08-12 17:18:58.436180 2014-08-13 18:06:06.490142 7f5c973a57a0 20 read_log 194215'464 (0'0) modify e0debb62/benchmark_data_storage0111-ib.pocstor_64498_object32182/head//22 by client.1049635.0:32183 2014-08-12 17:19:09.269924 2014-08-13 18:06:06.490162 7f5c973a57a0 20 read_log 194215'465 (194215'450) delete a6c07362/benchmark_data_storage0111-ib.pocstor_64498_object7055/head//22 by client.1049635.0:41258 2014-08-12 17:19:20.298677 2014-08-13 18:06:06.490175 7f5c973a57a0 20 read_log 194215'466 (194215'451) delete 2a611362/benchmark_data_storage0111-ib.pocstor_64498_object9383/head//22 by client.1049635.0:43586 2014-08-12 17:19:21.054960 2014-08-13 18:06:06.490187 7f5c973a57a0 20 read_log 194215'467 (194215'452) delete 7f7d2362/benchmark_data_storage0111-ib.pocstor_64498_object10645/head//22 by client.1049635.0:44848 2014-08-12 17:19:21.464491 2014-08-13 18:06:06.490201 7f5c973a57a0 20 read_log 194215'468 (194215'453) delete 9d377362/benchmark_data_storage0111-ib.pocstor_64498_object10960/head//22 by client.1049635.0:45163 2014-08-12 17:19:21.565895 2014-08-13 18:06:06.490216 7f5c973a57a0 20 read_log 194215'469 (194215'454) delete 446fb62/benchmark_data_storage0111-ib.pocstor_64498_object12968/head//22 by client.1049635.0:47171 2014-08-12 17:19:22.202596 2014-08-13 18:06:06.490229 7f5c973a57a0 20 read_log 194215'470 (194215'455) delete 69232b62/benchmark_data_storage0111-ib.pocstor_64498_object13029/head//22 by client.1049635.0:47232 2014-08-12 17:19:22.227935 2014-08-13 18:06:06.490243 7f5c973a57a0 20 read_log 194215'471 (194215'456) delete 8dbd8362/benchmark_data_storage0111-ib.pocstor_64498_object13953/head//22 by client.1049635.0:48156 2014-08-12 17:19:22.548643 2014-08-13 18:06:06.490255 7f5c973a57a0 20 read_log 194215'472 (194215'457) delete b6528b62/benchmark_data_storage0111-ib.pocstor_64498_object14620/head//22 by client.1049635.0:48823 2014-08-12 17:19:22.760422 2014-08-13 18:06:06.490272 7f5c973a57a0 20 read_log 194215'473 (194215'458) delete 6d78c362/benchmark_data_storage0111-ib.pocstor_64498_object17345/head//22 by client.1049635.0:51548 2014-08-12 17:19:23.616644 2014-08-13 18:06:06.490287 7f5c973a57a0 20 read_log 194215'474 (194215'459) delete 8c33c362/benchmark_data_storage0111-ib.pocstor_64498_object17682/head//22 by client.1049635.0:51885 2014-08-12 17:19:23.721755 2014-08-13 18:06:06.490300 7f5c973a57a0 20 read_log 194215'475 (194215'460) delete 6b28e362/benchmark_data_storage0111-ib.pocstor_64498_object19659/head//22 by client.1049635.0:53862 2014-08-12 17:19:24.403353 2014-08-13 18:06:06.490314 7f5c973a57a0 20 read_log 194215'476 (194215'461) delete c0644b62/benchmark_data_storage0111-ib.pocstor_64498_object25743/head//22 by client.1049635.0:59946 2014-08-12 17:19:26.572396 2014-08-13 18:06:06.490330 7f5c973a57a0 20 read_log 194215'477 (194215'462) delete 8023e362/benchmark_data_storage0111-ib.pocstor_64498_object26771/head//22 by client.1049635.0:60974 2014-08-12 17:19:26.914512 2014-08-13 18:06:06.490345 7f5c973a57a0 20 read_log 194215'478 (194215'463) delete 343362/benchmark_data_storage0111-ib.pocstor_64498_object29465/head//22 by client.1049635.0:63668 2014-08-12 17:19:27.874509 2014-08-13 18:06:06.490358 7f5c973a57a0 20 read_log 194215'479 (194215'464) delete e0debb62/benchmark_data_storage0111-ib.pocstor_64498_object32182/head//22 by client.1049635.0:66385 2014-08-12 17:19:28.811195 2014-08-13 18:06:06.490477 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.490497 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=0 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.490523 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=0 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001831 0 0.000000 2014-08-13 18:06:06.490535 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=0 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.490544 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=0 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.490560 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'479], crt=191727'415) 2014-08-13 18:06:06.490571 7f5c973a57a0 10 osd.4 194487 pgid 22.39d coll 22.39d_head 2014-08-13 18:06:06.490706 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.39d 2014-08-13 18:06:06.490716 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.490742 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.39d(unlocked)] enter Initial 2014-08-13 18:06:06.490753 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.39d(unlocked)] enter NotTrimming 2014-08-13 18:06:06.490892 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.491005 7f5c973a57a0 20 read_log 191648'26 (0'0) modify e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 by client.273158.0:1299 2014-07-30 14:08:53.563044 2014-08-13 18:06:06.491026 7f5c973a57a0 20 read_log 191648'122 (0'0) modify 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 by client.273158.0:7733 2014-07-30 14:09:24.642346 2014-08-13 18:06:06.491039 7f5c973a57a0 20 read_log 191648'139 (0'0) modify d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 by client.273158.0:8772 2014-07-30 14:09:30.270496 2014-08-13 18:06:06.491051 7f5c973a57a0 20 read_log 191648'196 (0'0) modify 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 by client.273158.0:12774 2014-07-30 14:09:51.305241 2014-08-13 18:06:06.491063 7f5c973a57a0 20 read_log 191648'231 (0'0) modify bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 by client.273158.0:15130 2014-07-30 14:10:03.184496 2014-08-13 18:06:06.491076 7f5c973a57a0 20 read_log 191648'238 (0'0) modify 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 by client.273158.0:15587 2014-07-30 14:10:05.248338 2014-08-13 18:06:06.491088 7f5c973a57a0 20 read_log 191727'348 (0'0) modify d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 by client.275175.0:460 2014-07-30 14:18:35.295944 2014-08-13 18:06:06.491128 7f5c973a57a0 20 read_log 191727'354 (0'0) modify 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 by client.275175.0:4105 2014-07-30 14:18:47.676496 2014-08-13 18:06:06.491144 7f5c973a57a0 20 read_log 191727'359 (0'0) modify 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 by client.275175.0:7849 2014-07-30 14:18:59.101798 2014-08-13 18:06:06.491166 7f5c973a57a0 20 read_log 191727'362 (0'0) modify e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 by client.275175.0:9837 2014-07-30 14:19:04.722733 2014-08-13 18:06:06.491178 7f5c973a57a0 20 read_log 191727'363 (0'0) modify 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 by client.275175.0:10098 2014-07-30 14:19:05.451862 2014-08-13 18:06:06.491191 7f5c973a57a0 20 read_log 191727'365 (0'0) modify e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 by client.275175.0:12885 2014-07-30 14:19:14.144831 2014-08-13 18:06:06.491203 7f5c973a57a0 20 read_log 191727'368 (0'0) modify e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 by client.275175.0:13708 2014-07-30 14:19:16.827656 2014-08-13 18:06:06.491215 7f5c973a57a0 20 read_log 191727'374 (0'0) modify ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 by client.275175.0:16549 2014-07-30 14:19:25.843951 2014-08-13 18:06:06.491227 7f5c973a57a0 20 read_log 191727'383 (0'0) modify 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 by client.275175.0:22486 2014-07-30 14:19:44.167235 2014-08-13 18:06:06.491242 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 by client.275175.0:22601 2014-07-30 14:19:44.678490 2014-08-13 18:06:06.491255 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 by client.275175.0:24145 2014-07-30 14:19:49.236554 2014-08-13 18:06:06.491270 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 by client.275175.0:24634 2014-07-30 14:19:50.611965 2014-08-13 18:06:06.491286 7f5c973a57a0 20 read_log 191727'390 (0'0) modify 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 by client.275175.0:25668 2014-07-30 14:19:53.722499 2014-08-13 18:06:06.491298 7f5c973a57a0 20 read_log 191727'393 (0'0) modify e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 by client.275175.0:26106 2014-07-30 14:19:55.164355 2014-08-13 18:06:06.491310 7f5c973a57a0 20 read_log 191727'398 (0'0) modify d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 by client.275175.0:28266 2014-07-30 14:20:01.770201 2014-08-13 18:06:06.491324 7f5c973a57a0 20 read_log 191727'403 (0'0) modify e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 by client.275175.0:31330 2014-07-30 14:20:10.912444 2014-08-13 18:06:06.491337 7f5c973a57a0 20 read_log 191727'411 (0'0) modify aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 by client.275175.0:35464 2014-07-30 14:20:23.339331 2014-08-13 18:06:06.491349 7f5c973a57a0 20 read_log 192888'417 (0'0) modify aff7939d/benchmark_data_storage0111-ib.pocstor_35625_object1287/head//22 by client.940330.0:1288 2014-08-11 16:29:45.861815 2014-08-13 18:06:06.491372 7f5c973a57a0 20 read_log 192888'418 (0'0) modify 3b158b9d/benchmark_data_storage0111-ib.pocstor_35625_object1448/head//22 by client.940330.0:1449 2014-08-11 16:29:46.505073 2014-08-13 18:06:06.491389 7f5c973a57a0 20 read_log 192888'419 (0'0) modify 96f7e39d/benchmark_data_storage0111-ib.pocstor_35625_object2238/head//22 by client.940330.0:2239 2014-08-11 16:29:49.420246 2014-08-13 18:06:06.491401 7f5c973a57a0 20 read_log 192888'420 (0'0) modify f730c39d/benchmark_data_storage0111-ib.pocstor_35625_object5880/head//22 by client.940330.0:5881 2014-08-11 16:30:03.137399 2014-08-13 18:06:06.491455 7f5c973a57a0 20 read_log 192888'421 (0'0) modify a5e58b9d/benchmark_data_storage0111-ib.pocstor_35625_object9479/head//22 by client.940330.0:9480 2014-08-11 16:30:16.008439 2014-08-13 18:06:06.491472 7f5c973a57a0 20 read_log 192888'422 (0'0) modify 47e94b9d/benchmark_data_storage0111-ib.pocstor_35625_object9885/head//22 by client.940330.0:9886 2014-08-11 16:30:17.428162 2014-08-13 18:06:06.491484 7f5c973a57a0 20 read_log 192888'423 (0'0) modify dba47b9d/benchmark_data_storage0111-ib.pocstor_35625_object10685/head//22 by client.940330.0:10686 2014-08-11 16:30:20.576240 2014-08-13 18:06:06.491497 7f5c973a57a0 20 read_log 192888'424 (0'0) modify 49060b9d/benchmark_data_storage0111-ib.pocstor_35625_object11965/head//22 by client.940330.0:11966 2014-08-11 16:30:25.779919 2014-08-13 18:06:06.491509 7f5c973a57a0 20 read_log 192888'425 (0'0) modify 357a439d/benchmark_data_storage0111-ib.pocstor_35625_object13078/head//22 by client.940330.0:13079 2014-08-11 16:30:30.197668 2014-08-13 18:06:06.491521 7f5c973a57a0 20 read_log 192888'426 (0'0) modify c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object15709/head//22 by client.940330.0:15710 2014-08-11 16:30:40.137252 2014-08-13 18:06:06.491533 7f5c973a57a0 20 read_log 192888'427 (0'0) modify a99c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object16423/head//22 by client.940330.0:16424 2014-08-11 16:30:42.811556 2014-08-13 18:06:06.491545 7f5c973a57a0 20 read_log 192888'428 (0'0) modify a7798b9d/benchmark_data_storage0111-ib.pocstor_35625_object17705/head//22 by client.940330.0:17706 2014-08-11 16:30:47.608735 2014-08-13 18:06:06.491557 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 96cf639d/benchmark_data_storage0111-ib.pocstor_35625_object21027/head//22 by client.940330.0:21028 2014-08-11 16:31:00.916326 2014-08-13 18:06:06.491569 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 9093db9d/benchmark_data_storage0111-ib.pocstor_35625_object22274/head//22 by client.940330.0:22275 2014-08-11 16:31:05.590831 2014-08-13 18:06:06.491581 7f5c973a57a0 20 read_log 192888'431 (0'0) modify a4c0bb9d/benchmark_data_storage0111-ib.pocstor_35625_object26157/head//22 by client.940330.0:26158 2014-08-11 16:31:20.407958 2014-08-13 18:06:06.491593 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 8d647b9d/benchmark_data_storage0111-ib.pocstor_35625_object29398/head//22 by client.940330.0:29399 2014-08-11 16:31:33.185068 2014-08-13 18:06:06.491605 7f5c973a57a0 20 read_log 192888'433 (0'0) modify b5d4fb9d/benchmark_data_storage0111-ib.pocstor_35625_object30589/head//22 by client.940330.0:30590 2014-08-11 16:31:37.682976 2014-08-13 18:06:06.491617 7f5c973a57a0 20 read_log 192888'434 (192888'417) delete aff7939d/benchmark_data_storage0111-ib.pocstor_35625_object1287/head//22 by client.940330.0:32595 2014-08-11 16:31:41.817054 2014-08-13 18:06:06.491629 7f5c973a57a0 20 read_log 192888'435 (192888'418) delete 3b158b9d/benchmark_data_storage0111-ib.pocstor_35625_object1448/head//22 by client.940330.0:32756 2014-08-11 16:31:41.855371 2014-08-13 18:06:06.491641 7f5c973a57a0 20 read_log 192888'436 (192888'419) delete 96f7e39d/benchmark_data_storage0111-ib.pocstor_35625_object2238/head//22 by client.940330.0:33546 2014-08-11 16:31:42.052572 2014-08-13 18:06:06.491654 7f5c973a57a0 20 read_log 192888'437 (192888'420) delete f730c39d/benchmark_data_storage0111-ib.pocstor_35625_object5880/head//22 by client.940330.0:37188 2014-08-11 16:31:43.043874 2014-08-13 18:06:06.491666 7f5c973a57a0 20 read_log 192888'438 (192888'421) delete a5e58b9d/benchmark_data_storage0111-ib.pocstor_35625_object9479/head//22 by client.940330.0:40787 2014-08-11 16:31:43.969923 2014-08-13 18:06:06.491678 7f5c973a57a0 20 read_log 192888'439 (192888'422) delete 47e94b9d/benchmark_data_storage0111-ib.pocstor_35625_object9885/head//22 by client.940330.0:41193 2014-08-11 16:31:44.082557 2014-08-13 18:06:06.491690 7f5c973a57a0 20 read_log 192888'440 (192888'423) delete dba47b9d/benchmark_data_storage0111-ib.pocstor_35625_object10685/head//22 by client.940330.0:41993 2014-08-11 16:31:44.305708 2014-08-13 18:06:06.491709 7f5c973a57a0 20 read_log 192888'441 (192888'424) delete 49060b9d/benchmark_data_storage0111-ib.pocstor_35625_object11965/head//22 by client.940330.0:43273 2014-08-11 16:31:44.648388 2014-08-13 18:06:06.491725 7f5c973a57a0 20 read_log 192888'442 (192888'425) delete 357a439d/benchmark_data_storage0111-ib.pocstor_35625_object13078/head//22 by client.940330.0:44386 2014-08-11 16:31:44.940655 2014-08-13 18:06:06.491737 7f5c973a57a0 20 read_log 192888'443 (192888'426) delete c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object15709/head//22 by client.940330.0:47017 2014-08-11 16:31:45.646283 2014-08-13 18:06:06.491749 7f5c973a57a0 20 read_log 192888'444 (192888'427) delete a99c4b9d/benchmark_data_storage0111-ib.pocstor_35625_object16423/head//22 by client.940330.0:47731 2014-08-11 16:31:45.834947 2014-08-13 18:06:06.491763 7f5c973a57a0 20 read_log 192888'445 (192888'428) delete a7798b9d/benchmark_data_storage0111-ib.pocstor_35625_object17705/head//22 by client.940330.0:49013 2014-08-11 16:31:46.193497 2014-08-13 18:06:06.491777 7f5c973a57a0 20 read_log 192888'446 (192888'429) delete 96cf639d/benchmark_data_storage0111-ib.pocstor_35625_object21027/head//22 by client.940330.0:52335 2014-08-11 16:31:47.168840 2014-08-13 18:06:06.491791 7f5c973a57a0 20 read_log 192888'447 (192888'430) delete 9093db9d/benchmark_data_storage0111-ib.pocstor_35625_object22274/head//22 by client.940330.0:53582 2014-08-11 16:31:47.534706 2014-08-13 18:06:06.491804 7f5c973a57a0 20 read_log 192888'448 (192888'431) delete a4c0bb9d/benchmark_data_storage0111-ib.pocstor_35625_object26157/head//22 by client.940330.0:57465 2014-08-11 16:31:48.690196 2014-08-13 18:06:06.491818 7f5c973a57a0 20 read_log 192888'449 (192888'432) delete 8d647b9d/benchmark_data_storage0111-ib.pocstor_35625_object29398/head//22 by client.940330.0:60706 2014-08-11 16:31:49.584116 2014-08-13 18:06:06.491832 7f5c973a57a0 20 read_log 192888'450 (192888'433) delete b5d4fb9d/benchmark_data_storage0111-ib.pocstor_35625_object30589/head//22 by client.940330.0:61897 2014-08-11 16:31:49.926372 2014-08-13 18:06:06.491845 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 9aa9a39d/benchmark_data_storage0111-ib.pocstor_64498_object1198/head//22 by client.1049635.0:1199 2014-08-12 17:17:21.039948 2014-08-13 18:06:06.491859 7f5c973a57a0 20 read_log 194215'452 (0'0) modify ee4f839d/benchmark_data_storage0111-ib.pocstor_64498_object7499/head//22 by client.1049635.0:7500 2014-08-12 17:17:40.535063 2014-08-13 18:06:06.491871 7f5c973a57a0 20 read_log 194215'453 (0'0) modify f1ff839d/benchmark_data_storage0111-ib.pocstor_64498_object8496/head//22 by client.1049635.0:8497 2014-08-12 17:17:43.549867 2014-08-13 18:06:06.491885 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 55c4439d/benchmark_data_storage0111-ib.pocstor_64498_object9283/head//22 by client.1049635.0:9284 2014-08-12 17:17:45.842712 2014-08-13 18:06:06.491898 7f5c973a57a0 20 read_log 194215'455 (0'0) modify 6e112b9d/benchmark_data_storage0111-ib.pocstor_64498_object11955/head//22 by client.1049635.0:11956 2014-08-12 17:17:54.258163 2014-08-13 18:06:06.491912 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 4c1b039d/benchmark_data_storage0111-ib.pocstor_64498_object13203/head//22 by client.1049635.0:13204 2014-08-12 17:17:57.996560 2014-08-13 18:06:06.491924 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 14bedb9d/benchmark_data_storage0111-ib.pocstor_64498_object13760/head//22 by client.1049635.0:13761 2014-08-12 17:17:59.822949 2014-08-13 18:06:06.491955 7f5c973a57a0 20 read_log 194215'458 (0'0) modify 929fd39d/benchmark_data_storage0111-ib.pocstor_64498_object14114/head//22 by client.1049635.0:14115 2014-08-12 17:18:00.860191 2014-08-13 18:06:06.491970 7f5c973a57a0 20 read_log 194215'459 (0'0) modify e1f7139d/benchmark_data_storage0111-ib.pocstor_64498_object14470/head//22 by client.1049635.0:14471 2014-08-12 17:18:02.151050 2014-08-13 18:06:06.491993 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 9530db9d/benchmark_data_storage0111-ib.pocstor_64498_object14696/head//22 by client.1049635.0:14697 2014-08-12 17:18:02.996079 2014-08-13 18:06:06.492009 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 77269b9d/benchmark_data_storage0111-ib.pocstor_64498_object15171/head//22 by client.1049635.0:15172 2014-08-12 17:18:04.534230 2014-08-13 18:06:06.492021 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 729a2b9d/benchmark_data_storage0111-ib.pocstor_64498_object24867/head//22 by client.1049635.0:24868 2014-08-12 17:18:41.158950 2014-08-13 18:06:06.492035 7f5c973a57a0 20 read_log 194215'463 (0'0) modify ed5ae39d/benchmark_data_storage0111-ib.pocstor_64498_object24923/head//22 by client.1049635.0:24924 2014-08-12 17:18:41.419768 2014-08-13 18:06:06.492048 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 185c5b9d/benchmark_data_storage0111-ib.pocstor_64498_object29834/head//22 by client.1049635.0:29835 2014-08-12 17:18:59.828932 2014-08-13 18:06:06.492062 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 713a339d/benchmark_data_storage0111-ib.pocstor_64498_object31111/head//22 by client.1049635.0:31112 2014-08-12 17:19:04.992614 2014-08-13 18:06:06.492074 7f5c973a57a0 20 read_log 194215'466 (194215'451) delete 9aa9a39d/benchmark_data_storage0111-ib.pocstor_64498_object1198/head//22 by client.1049635.0:35401 2014-08-12 17:19:18.383901 2014-08-13 18:06:06.492089 7f5c973a57a0 20 read_log 194215'467 (194215'452) delete ee4f839d/benchmark_data_storage0111-ib.pocstor_64498_object7499/head//22 by client.1049635.0:41702 2014-08-12 17:19:20.448516 2014-08-13 18:06:06.492101 7f5c973a57a0 20 read_log 194215'468 (194215'453) delete f1ff839d/benchmark_data_storage0111-ib.pocstor_64498_object8496/head//22 by client.1049635.0:42699 2014-08-12 17:19:20.758592 2014-08-13 18:06:06.492114 7f5c973a57a0 20 read_log 194215'469 (194215'454) delete 55c4439d/benchmark_data_storage0111-ib.pocstor_64498_object9283/head//22 by client.1049635.0:43486 2014-08-12 17:19:21.018972 2014-08-13 18:06:06.492126 7f5c973a57a0 20 read_log 194215'470 (194215'455) delete 6e112b9d/benchmark_data_storage0111-ib.pocstor_64498_object11955/head//22 by client.1049635.0:46158 2014-08-12 17:19:21.885671 2014-08-13 18:06:06.492138 7f5c973a57a0 20 read_log 194215'471 (194215'456) delete 4c1b039d/benchmark_data_storage0111-ib.pocstor_64498_object13203/head//22 by client.1049635.0:47406 2014-08-12 17:19:22.291853 2014-08-13 18:06:06.492161 7f5c973a57a0 20 read_log 194215'472 (194215'457) delete 14bedb9d/benchmark_data_storage0111-ib.pocstor_64498_object13760/head//22 by client.1049635.0:47963 2014-08-12 17:19:22.492087 2014-08-13 18:06:06.492176 7f5c973a57a0 20 read_log 194215'473 (194215'458) delete 929fd39d/benchmark_data_storage0111-ib.pocstor_64498_object14114/head//22 by client.1049635.0:48317 2014-08-12 17:19:22.597693 2014-08-13 18:06:06.492189 7f5c973a57a0 20 read_log 194215'474 (194215'459) delete e1f7139d/benchmark_data_storage0111-ib.pocstor_64498_object14470/head//22 by client.1049635.0:48673 2014-08-12 17:19:22.711244 2014-08-13 18:06:06.492201 7f5c973a57a0 20 read_log 194215'475 (194215'460) delete 9530db9d/benchmark_data_storage0111-ib.pocstor_64498_object14696/head//22 by client.1049635.0:48899 2014-08-12 17:19:22.783840 2014-08-13 18:06:06.492215 7f5c973a57a0 20 read_log 194215'476 (194215'461) delete 77269b9d/benchmark_data_storage0111-ib.pocstor_64498_object15171/head//22 by client.1049635.0:49374 2014-08-12 17:19:22.926265 2014-08-13 18:06:06.492230 7f5c973a57a0 20 read_log 194215'477 (194215'462) delete 729a2b9d/benchmark_data_storage0111-ib.pocstor_64498_object24867/head//22 by client.1049635.0:59070 2014-08-12 17:19:26.278811 2014-08-13 18:06:06.492242 7f5c973a57a0 20 read_log 194215'478 (194215'463) delete ed5ae39d/benchmark_data_storage0111-ib.pocstor_64498_object24923/head//22 by client.1049635.0:59126 2014-08-12 17:19:26.295113 2014-08-13 18:06:06.492256 7f5c973a57a0 20 read_log 194215'479 (194215'464) delete 185c5b9d/benchmark_data_storage0111-ib.pocstor_64498_object29834/head//22 by client.1049635.0:64037 2014-08-12 17:19:27.996815 2014-08-13 18:06:06.492276 7f5c973a57a0 20 read_log 194215'480 (194215'465) delete 713a339d/benchmark_data_storage0111-ib.pocstor_64498_object31111/head//22 by client.1049635.0:65314 2014-08-12 17:19:28.417085 2014-08-13 18:06:06.492364 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.492383 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=0 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.492399 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=0 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001657 0 0.000000 2014-08-13 18:06:06.492421 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=0 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.492433 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=0 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.492443 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'480], crt=191727'416) 2014-08-13 18:06:06.492453 7f5c973a57a0 10 osd.4 194487 pgid 22.3ad coll 22.3ad_head 2014-08-13 18:06:06.492643 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3ad 2014-08-13 18:06:06.492653 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.492686 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3ad(unlocked)] enter Initial 2014-08-13 18:06:06.492698 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3ad(unlocked)] enter NotTrimming 2014-08-13 18:06:06.492851 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.492996 7f5c973a57a0 20 read_log 191648'9 (0'0) modify f5b70bad/benchmark_data_storage0111-ib.pocstor_40745_object361/head//22 by client.273158.0:362 2014-07-30 14:08:50.141745 2014-08-13 18:06:06.493018 7f5c973a57a0 20 read_log 191648'20 (0'0) modify de67fbad/benchmark_data_storage0111-ib.pocstor_40745_object878/head//22 by client.273158.0:879 2014-07-30 14:08:52.037400 2014-08-13 18:06:06.493031 7f5c973a57a0 20 read_log 191648'55 (0'0) modify 631053ad/benchmark_data_storage0111-ib.pocstor_40745_object2799/head//22 by client.273158.0:2800 2014-07-30 14:09:00.349553 2014-08-13 18:06:06.493043 7f5c973a57a0 20 read_log 191648'72 (0'0) modify 5b437bad/benchmark_data_storage0111-ib.pocstor_40745_object3931/head//22 by client.273158.0:3932 2014-07-30 14:09:05.958864 2014-08-13 18:06:06.493056 7f5c973a57a0 20 read_log 191648'80 (0'0) modify 594613ad/benchmark_data_storage0111-ib.pocstor_40745_object4707/head//22 by client.273158.0:4708 2014-07-30 14:09:09.642401 2014-08-13 18:06:06.493068 7f5c973a57a0 20 read_log 191648'171 (0'0) modify f9557bad/benchmark_data_storage0111-ib.pocstor_40745_object11374/head//22 by client.273158.0:11375 2014-07-30 14:09:43.783183 2014-08-13 18:06:06.493087 7f5c973a57a0 20 read_log 191648'187 (0'0) modify a1240bad/benchmark_data_storage0111-ib.pocstor_40745_object12129/head//22 by client.273158.0:12130 2014-07-30 14:09:48.343980 2014-08-13 18:06:06.493102 7f5c973a57a0 20 read_log 191648'252 (0'0) modify 2c903bad/benchmark_data_storage0111-ib.pocstor_40745_object16338/head//22 by client.273158.0:16339 2014-07-30 14:10:09.295782 2014-08-13 18:06:06.493115 7f5c973a57a0 20 read_log 191648'294 (0'0) modify 3a24cbad/benchmark_data_storage0111-ib.pocstor_40745_object18210/head//22 by client.273158.0:18211 2014-07-30 14:10:19.109377 2014-08-13 18:06:06.493127 7f5c973a57a0 20 read_log 191648'301 (0'0) modify 2e3273ad/benchmark_data_storage0111-ib.pocstor_40745_object18704/head//22 by client.273158.0:18705 2014-07-30 14:10:22.342769 2014-08-13 18:06:06.493164 7f5c973a57a0 20 read_log 191648'313 (0'0) modify 5da043ad/benchmark_data_storage0111-ib.pocstor_40745_object19378/head//22 by client.273158.0:19379 2014-07-30 14:10:25.771154 2014-08-13 18:06:06.493180 7f5c973a57a0 20 read_log 191648'317 (0'0) modify 7658d3ad/benchmark_data_storage0111-ib.pocstor_40745_object19695/head//22 by client.273158.0:19696 2014-07-30 14:10:27.834449 2014-08-13 18:06:06.493193 7f5c973a57a0 20 read_log 191648'343 (0'0) modify 108f6bad/benchmark_data_storage0111-ib.pocstor_40745_object21182/head//22 by client.273158.0:21183 2014-07-30 14:10:38.020512 2014-08-13 18:06:06.493205 7f5c973a57a0 20 read_log 191648'347 (0'0) modify df803ad/benchmark_data_storage0111-ib.pocstor_40745_object21453/head//22 by client.273158.0:21454 2014-07-30 14:10:39.267993 2014-08-13 18:06:06.493218 7f5c973a57a0 20 read_log 191727'374 (0'0) modify 186f6bad/benchmark_data_storage0111-ib.pocstor_51285_object527/head//22 by client.275175.0:528 2014-07-30 14:18:35.580093 2014-08-13 18:06:06.493230 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 367a93ad/benchmark_data_storage0111-ib.pocstor_51285_object5248/head//22 by client.275175.0:5249 2014-07-30 14:18:51.084411 2014-08-13 18:06:06.493242 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 9d6ed3ad/benchmark_data_storage0111-ib.pocstor_51285_object6613/head//22 by client.275175.0:6614 2014-07-30 14:18:55.218980 2014-08-13 18:06:06.493254 7f5c973a57a0 20 read_log 191727'385 (0'0) modify bfe27bad/benchmark_data_storage0111-ib.pocstor_51285_object7152/head//22 by client.275175.0:7153 2014-07-30 14:18:57.152350 2014-08-13 18:06:06.493266 7f5c973a57a0 20 read_log 191727'387 (0'0) modify bd663ad/benchmark_data_storage0111-ib.pocstor_51285_object8531/head//22 by client.275175.0:8532 2014-07-30 14:19:01.016015 2014-08-13 18:06:06.493282 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 480e73ad/benchmark_data_storage0111-ib.pocstor_51285_object8759/head//22 by client.275175.0:8760 2014-07-30 14:19:01.662527 2014-08-13 18:06:06.493295 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 825783ad/benchmark_data_storage0111-ib.pocstor_51285_object12762/head//22 by client.275175.0:12763 2014-07-30 14:19:13.779839 2014-08-13 18:06:06.493307 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 5d358bad/benchmark_data_storage0111-ib.pocstor_51285_object13078/head//22 by client.275175.0:13079 2014-07-30 14:19:14.916472 2014-08-13 18:06:06.493319 7f5c973a57a0 20 read_log 191727'401 (0'0) modify 206cb3ad/benchmark_data_storage0111-ib.pocstor_51285_object14457/head//22 by client.275175.0:14458 2014-07-30 14:19:19.169473 2014-08-13 18:06:06.493331 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 6b955bad/benchmark_data_storage0111-ib.pocstor_51285_object15248/head//22 by client.275175.0:15249 2014-07-30 14:19:21.506584 2014-08-13 18:06:06.493343 7f5c973a57a0 20 read_log 191727'409 (0'0) modify a63efbad/benchmark_data_storage0111-ib.pocstor_51285_object20929/head//22 by client.275175.0:20930 2014-07-30 14:19:38.930325 2014-08-13 18:06:06.493383 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 89403ad/benchmark_data_storage0111-ib.pocstor_51285_object24803/head//22 by client.275175.0:24804 2014-07-30 14:19:51.086663 2014-08-13 18:06:06.493411 7f5c973a57a0 20 read_log 191727'419 (0'0) modify a9532bad/benchmark_data_storage0111-ib.pocstor_51285_object25781/head//22 by client.275175.0:25782 2014-07-30 14:19:54.035199 2014-08-13 18:06:06.493424 7f5c973a57a0 20 read_log 191727'421 (0'0) modify d2d3bad/benchmark_data_storage0111-ib.pocstor_51285_object25880/head//22 by client.275175.0:25881 2014-07-30 14:19:54.392441 2014-08-13 18:06:06.493444 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 70940bad/benchmark_data_storage0111-ib.pocstor_51285_object27497/head//22 by client.275175.0:27498 2014-07-30 14:19:59.226875 2014-08-13 18:06:06.493459 7f5c973a57a0 20 read_log 191727'424 (0'0) modify eeebf3ad/benchmark_data_storage0111-ib.pocstor_51285_object29626/head//22 by client.275175.0:29627 2014-07-30 14:20:05.840665 2014-08-13 18:06:06.493472 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 356933ad/benchmark_data_storage0111-ib.pocstor_51285_object31396/head//22 by client.275175.0:31397 2014-07-30 14:20:11.103763 2014-08-13 18:06:06.493484 7f5c973a57a0 20 read_log 191727'432 (0'0) modify 90f48bad/benchmark_data_storage0111-ib.pocstor_51285_object33661/head//22 by client.275175.0:33662 2014-07-30 14:20:18.026837 2014-08-13 18:06:06.493506 7f5c973a57a0 20 read_log 191727'435 (0'0) modify a3894bad/benchmark_data_storage0111-ib.pocstor_51285_object34965/head//22 by client.275175.0:34966 2014-07-30 14:20:21.890626 2014-08-13 18:06:06.493524 7f5c973a57a0 20 read_log 191727'436 (0'0) modify 621a33ad/benchmark_data_storage0111-ib.pocstor_51285_object35329/head//22 by client.275175.0:35330 2014-07-30 14:20:22.965074 2014-08-13 18:06:06.493537 7f5c973a57a0 20 read_log 191727'438 (0'0) modify 7bb8c3ad/benchmark_data_storage0111-ib.pocstor_51285_object35925/head//22 by client.275175.0:35926 2014-07-30 14:20:25.019068 2014-08-13 18:06:06.493551 7f5c973a57a0 20 read_log 191727'444 (0'0) modify 3e2d83ad/benchmark_data_storage0111-ib.pocstor_51285_object37914/head//22 by client.275175.0:37915 2014-07-30 14:20:31.000553 2014-08-13 18:06:06.493563 7f5c973a57a0 20 read_log 192674'448 (0'0) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6879 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493576 7f5c973a57a0 20 read_log 192674'449 (192674'448) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6880 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493588 7f5c973a57a0 20 read_log 192674'450 (192674'449) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6881 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493600 7f5c973a57a0 20 read_log 192674'451 (192674'450) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6882 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493613 7f5c973a57a0 20 read_log 192674'452 (192674'451) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6883 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493625 7f5c973a57a0 20 read_log 192674'453 (192674'452) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6884 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493637 7f5c973a57a0 20 read_log 192674'454 (192674'453) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6885 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493649 7f5c973a57a0 20 read_log 192674'455 (192674'454) modify 43f4b3ad/rb.0.43b76.238e1f29.0000000002fe/head//22 by client.279772.1:6886 2014-07-31 18:03:48.965000 2014-08-13 18:06:06.493662 7f5c973a57a0 20 read_log 192888'456 (0'0) modify 7070bbad/benchmark_data_storage0111-ib.pocstor_35625_object475/head//22 by client.940330.0:476 2014-08-11 16:29:42.587733 2014-08-13 18:06:06.493681 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 57448bad/benchmark_data_storage0111-ib.pocstor_35625_object4103/head//22 by client.940330.0:4104 2014-08-11 16:29:56.705615 2014-08-13 18:06:06.493696 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 4b262bad/benchmark_data_storage0111-ib.pocstor_35625_object4361/head//22 by client.940330.0:4362 2014-08-11 16:29:57.713970 2014-08-13 18:06:06.493709 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 78bc73ad/benchmark_data_storage0111-ib.pocstor_35625_object5581/head//22 by client.940330.0:5582 2014-08-11 16:30:02.085856 2014-08-13 18:06:06.493721 7f5c973a57a0 20 read_log 192888'460 (0'0) modify d5a703ad/benchmark_data_storage0111-ib.pocstor_35625_object17802/head//22 by client.940330.0:17803 2014-08-11 16:30:47.977241 2014-08-13 18:06:06.493733 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 9fe73bad/benchmark_data_storage0111-ib.pocstor_35625_object18683/head//22 by client.940330.0:18684 2014-08-11 16:30:51.501260 2014-08-13 18:06:06.493745 7f5c973a57a0 20 read_log 192888'462 (0'0) modify d6194bad/benchmark_data_storage0111-ib.pocstor_35625_object19308/head//22 by client.940330.0:19309 2014-08-11 16:30:53.810500 2014-08-13 18:06:06.493757 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 6fa56bad/benchmark_data_storage0111-ib.pocstor_35625_object21887/head//22 by client.940330.0:21888 2014-08-11 16:31:04.151087 2014-08-13 18:06:06.493769 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 4f4eebad/benchmark_data_storage0111-ib.pocstor_35625_object24509/head//22 by client.940330.0:24510 2014-08-11 16:31:14.007156 2014-08-13 18:06:06.493781 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 2bb3abad/benchmark_data_storage0111-ib.pocstor_35625_object25296/head//22 by client.940330.0:25297 2014-08-11 16:31:17.047030 2014-08-13 18:06:06.493793 7f5c973a57a0 20 read_log 192888'466 (0'0) modify ec9edbad/benchmark_data_storage0111-ib.pocstor_35625_object30472/head//22 by client.940330.0:30473 2014-08-11 16:31:37.250025 2014-08-13 18:06:06.493805 7f5c973a57a0 20 read_log 192888'467 (192888'456) delete 7070bbad/benchmark_data_storage0111-ib.pocstor_35625_object475/head//22 by client.940330.0:31783 2014-08-11 16:31:41.614663 2014-08-13 18:06:06.493818 7f5c973a57a0 20 read_log 192888'468 (192888'457) delete 57448bad/benchmark_data_storage0111-ib.pocstor_35625_object4103/head//22 by client.940330.0:35411 2014-08-11 16:31:42.564040 2014-08-13 18:06:06.493830 7f5c973a57a0 20 read_log 192888'469 (192888'458) delete 4b262bad/benchmark_data_storage0111-ib.pocstor_35625_object4361/head//22 by client.940330.0:35669 2014-08-11 16:31:42.638555 2014-08-13 18:06:06.493842 7f5c973a57a0 20 read_log 192888'470 (192888'459) delete 78bc73ad/benchmark_data_storage0111-ib.pocstor_35625_object5581/head//22 by client.940330.0:36889 2014-08-11 16:31:42.964034 2014-08-13 18:06:06.493854 7f5c973a57a0 20 read_log 192888'471 (192888'460) delete d5a703ad/benchmark_data_storage0111-ib.pocstor_35625_object17802/head//22 by client.940330.0:49110 2014-08-11 16:31:46.226350 2014-08-13 18:06:06.493866 7f5c973a57a0 20 read_log 192888'472 (192888'461) delete 9fe73bad/benchmark_data_storage0111-ib.pocstor_35625_object18683/head//22 by client.940330.0:49991 2014-08-11 16:31:46.496118 2014-08-13 18:06:06.493881 7f5c973a57a0 20 read_log 192888'473 (192888'462) delete d6194bad/benchmark_data_storage0111-ib.pocstor_35625_object19308/head//22 by client.940330.0:50616 2014-08-11 16:31:46.666018 2014-08-13 18:06:06.493894 7f5c973a57a0 20 read_log 192888'474 (192888'463) delete 6fa56bad/benchmark_data_storage0111-ib.pocstor_35625_object21887/head//22 by client.940330.0:53195 2014-08-11 16:31:47.410477 2014-08-13 18:06:06.493908 7f5c973a57a0 20 read_log 192888'475 (192888'464) delete 4f4eebad/benchmark_data_storage0111-ib.pocstor_35625_object24509/head//22 by client.940330.0:55817 2014-08-11 16:31:48.191393 2014-08-13 18:06:06.493921 7f5c973a57a0 20 read_log 192888'476 (192888'465) delete 2bb3abad/benchmark_data_storage0111-ib.pocstor_35625_object25296/head//22 by client.940330.0:56604 2014-08-11 16:31:48.436407 2014-08-13 18:06:06.493941 7f5c973a57a0 20 read_log 192888'477 (192888'466) delete ec9edbad/benchmark_data_storage0111-ib.pocstor_35625_object30472/head//22 by client.940330.0:61780 2014-08-11 16:31:49.892392 2014-08-13 18:06:06.493956 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 363cd3ad/benchmark_data_storage0111-ib.pocstor_64498_object1247/head//22 by client.1049635.0:1248 2014-08-12 17:17:21.190729 2014-08-13 18:06:06.493968 7f5c973a57a0 20 read_log 194215'479 (0'0) modify 2d1823ad/benchmark_data_storage0111-ib.pocstor_64498_object4302/head//22 by client.1049635.0:4303 2014-08-12 17:17:30.955348 2014-08-13 18:06:06.493983 7f5c973a57a0 20 read_log 194215'480 (0'0) modify a79e73ad/benchmark_data_storage0111-ib.pocstor_64498_object6240/head//22 by client.1049635.0:6241 2014-08-12 17:17:36.820480 2014-08-13 18:06:06.493995 7f5c973a57a0 20 read_log 194215'481 (0'0) modify e07a9bad/benchmark_data_storage0111-ib.pocstor_64498_object6324/head//22 by client.1049635.0:6325 2014-08-12 17:17:37.057291 2014-08-13 18:06:06.494009 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 2afacbad/benchmark_data_storage0111-ib.pocstor_64498_object6446/head//22 by client.1049635.0:6447 2014-08-12 17:17:37.400204 2014-08-13 18:06:06.494021 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 95b453ad/benchmark_data_storage0111-ib.pocstor_64498_object6913/head//22 by client.1049635.0:6914 2014-08-12 17:17:38.876489 2014-08-13 18:06:06.494036 7f5c973a57a0 20 read_log 194215'484 (0'0) modify c1c25bad/benchmark_data_storage0111-ib.pocstor_64498_object7198/head//22 by client.1049635.0:7199 2014-08-12 17:17:39.691508 2014-08-13 18:06:06.494052 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 507e53ad/benchmark_data_storage0111-ib.pocstor_64498_object9948/head//22 by client.1049635.0:9949 2014-08-12 17:17:47.775579 2014-08-13 18:06:06.494067 7f5c973a57a0 20 read_log 194215'486 (0'0) modify bccc03ad/benchmark_data_storage0111-ib.pocstor_64498_object11182/head//22 by client.1049635.0:11183 2014-08-12 17:17:51.778783 2014-08-13 18:06:06.494079 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 8ee75bad/benchmark_data_storage0111-ib.pocstor_64498_object16389/head//22 by client.1049635.0:16390 2014-08-12 17:18:09.043693 2014-08-13 18:06:06.494094 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 909993ad/benchmark_data_storage0111-ib.pocstor_64498_object17543/head//22 by client.1049635.0:17544 2014-08-12 17:18:13.434157 2014-08-13 18:06:06.494106 7f5c973a57a0 20 read_log 194215'489 (0'0) modify eacbfbad/benchmark_data_storage0111-ib.pocstor_64498_object22206/head//22 by client.1049635.0:22207 2014-08-12 17:18:31.102950 2014-08-13 18:06:06.494118 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 1adf3ad/benchmark_data_storage0111-ib.pocstor_64498_object26457/head//22 by client.1049635.0:26458 2014-08-12 17:18:47.120737 2014-08-13 18:06:06.494130 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 943dcbad/benchmark_data_storage0111-ib.pocstor_64498_object30277/head//22 by client.1049635.0:30278 2014-08-12 17:19:01.664544 2014-08-13 18:06:06.494144 7f5c973a57a0 20 read_log 194215'492 (0'0) modify dd6bb3ad/benchmark_data_storage0111-ib.pocstor_64498_object30900/head//22 by client.1049635.0:30901 2014-08-12 17:19:04.185559 2014-08-13 18:06:06.494165 7f5c973a57a0 20 read_log 194215'493 (0'0) modify c2b6ebad/benchmark_data_storage0111-ib.pocstor_64498_object31613/head//22 by client.1049635.0:31614 2014-08-12 17:19:07.068733 2014-08-13 18:06:06.494178 7f5c973a57a0 20 read_log 194215'494 (0'0) modify bbfc6bad/benchmark_data_storage0111-ib.pocstor_64498_object32995/head//22 by client.1049635.0:32996 2014-08-12 17:19:12.373899 2014-08-13 18:06:06.494190 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 1c93a3ad/benchmark_data_storage0111-ib.pocstor_64498_object33298/head//22 by client.1049635.0:33299 2014-08-12 17:19:13.515117 2014-08-13 18:06:06.494206 7f5c973a57a0 20 read_log 194215'496 (194215'478) delete 363cd3ad/benchmark_data_storage0111-ib.pocstor_64498_object1247/head//22 by client.1049635.0:35450 2014-08-12 17:19:18.399580 2014-08-13 18:06:06.494220 7f5c973a57a0 20 read_log 194215'497 (194215'479) delete 2d1823ad/benchmark_data_storage0111-ib.pocstor_64498_object4302/head//22 by client.1049635.0:38505 2014-08-12 17:19:19.373433 2014-08-13 18:06:06.494232 7f5c973a57a0 20 read_log 194215'498 (194215'480) delete a79e73ad/benchmark_data_storage0111-ib.pocstor_64498_object6240/head//22 by client.1049635.0:40443 2014-08-12 17:19:20.036063 2014-08-13 18:06:06.494247 7f5c973a57a0 20 read_log 194215'499 (194215'481) delete e07a9bad/benchmark_data_storage0111-ib.pocstor_64498_object6324/head//22 by client.1049635.0:40527 2014-08-12 17:19:20.061978 2014-08-13 18:06:06.494260 7f5c973a57a0 20 read_log 194215'500 (194215'482) delete 2afacbad/benchmark_data_storage0111-ib.pocstor_64498_object6446/head//22 by client.1049635.0:40649 2014-08-12 17:19:20.097510 2014-08-13 18:06:06.494274 7f5c973a57a0 20 read_log 194215'501 (194215'483) delete 95b453ad/benchmark_data_storage0111-ib.pocstor_64498_object6913/head//22 by client.1049635.0:41116 2014-08-12 17:19:20.255432 2014-08-13 18:06:06.494286 7f5c973a57a0 20 read_log 194215'502 (194215'484) delete c1c25bad/benchmark_data_storage0111-ib.pocstor_64498_object7198/head//22 by client.1049635.0:41401 2014-08-12 17:19:20.351115 2014-08-13 18:06:06.494303 7f5c973a57a0 20 read_log 194215'503 (194215'485) delete 507e53ad/benchmark_data_storage0111-ib.pocstor_64498_object9948/head//22 by client.1049635.0:44151 2014-08-12 17:19:21.244621 2014-08-13 18:06:06.494316 7f5c973a57a0 20 read_log 194215'504 (194215'486) delete bccc03ad/benchmark_data_storage0111-ib.pocstor_64498_object11182/head//22 by client.1049635.0:45385 2014-08-12 17:19:21.636639 2014-08-13 18:06:06.494330 7f5c973a57a0 20 read_log 194215'505 (194215'487) delete 8ee75bad/benchmark_data_storage0111-ib.pocstor_64498_object16389/head//22 by client.1049635.0:50592 2014-08-12 17:19:23.318138 2014-08-13 18:06:06.494343 7f5c973a57a0 20 read_log 194215'506 (194215'488) delete 909993ad/benchmark_data_storage0111-ib.pocstor_64498_object17543/head//22 by client.1049635.0:51746 2014-08-12 17:19:23.681103 2014-08-13 18:06:06.494357 7f5c973a57a0 20 read_log 194215'507 (194215'489) delete eacbfbad/benchmark_data_storage0111-ib.pocstor_64498_object22206/head//22 by client.1049635.0:56409 2014-08-12 17:19:25.310630 2014-08-13 18:06:06.494369 7f5c973a57a0 20 read_log 194215'508 (194215'490) delete 1adf3ad/benchmark_data_storage0111-ib.pocstor_64498_object26457/head//22 by client.1049635.0:60660 2014-08-12 17:19:26.808026 2014-08-13 18:06:06.494388 7f5c973a57a0 20 read_log 194215'509 (194215'491) delete 943dcbad/benchmark_data_storage0111-ib.pocstor_64498_object30277/head//22 by client.1049635.0:64480 2014-08-12 17:19:28.137180 2014-08-13 18:06:06.494400 7f5c973a57a0 20 read_log 194215'510 (194215'492) delete dd6bb3ad/benchmark_data_storage0111-ib.pocstor_64498_object30900/head//22 by client.1049635.0:65103 2014-08-12 17:19:28.349001 2014-08-13 18:06:06.494425 7f5c973a57a0 20 read_log 194215'511 (194215'493) delete c2b6ebad/benchmark_data_storage0111-ib.pocstor_64498_object31613/head//22 by client.1049635.0:65816 2014-08-12 17:19:28.604932 2014-08-13 18:06:06.494441 7f5c973a57a0 20 read_log 194215'512 (194215'494) delete bbfc6bad/benchmark_data_storage0111-ib.pocstor_64498_object32995/head//22 by client.1049635.0:67198 2014-08-12 17:19:29.112824 2014-08-13 18:06:06.494454 7f5c973a57a0 20 read_log 194215'513 (194215'495) delete 1c93a3ad/benchmark_data_storage0111-ib.pocstor_64498_object33298/head//22 by client.1049635.0:67501 2014-08-12 17:19:29.217217 2014-08-13 18:06:06.494557 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.494578 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=0 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.494594 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=0 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Initial 0.001908 0 0.000000 2014-08-13 18:06:06.494625 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=0 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.494634 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=0 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.494643 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] log((0'0,194215'513], crt=192888'477) 2014-08-13 18:06:06.494654 7f5c973a57a0 10 osd.4 194487 pgid 22.3af coll 22.3af_head 2014-08-13 18:06:06.494810 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3af 2014-08-13 18:06:06.494819 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.494845 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3af(unlocked)] enter Initial 2014-08-13 18:06:06.494862 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3af(unlocked)] enter NotTrimming 2014-08-13 18:06:06.495024 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.495166 7f5c973a57a0 20 read_log 191648'76 (0'0) modify 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 by client.273158.0:4306 2014-07-30 14:09:07.998582 2014-08-13 18:06:06.495187 7f5c973a57a0 20 read_log 191648'109 (0'0) modify 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 by client.273158.0:6338 2014-07-30 14:09:17.770657 2014-08-13 18:06:06.495204 7f5c973a57a0 20 read_log 191648'140 (0'0) modify 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 by client.273158.0:8285 2014-07-30 14:09:27.760006 2014-08-13 18:06:06.495219 7f5c973a57a0 20 read_log 191648'149 (0'0) modify 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 by client.273158.0:8900 2014-07-30 14:09:30.773627 2014-08-13 18:06:06.495231 7f5c973a57a0 20 read_log 191648'165 (0'0) modify f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 by client.273158.0:10091 2014-07-30 14:09:36.195486 2014-08-13 18:06:06.495244 7f5c973a57a0 20 read_log 191648'208 (0'0) modify 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 by client.273158.0:12745 2014-07-30 14:09:51.188747 2014-08-13 18:06:06.495256 7f5c973a57a0 20 read_log 191648'320 (0'0) modify b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 by client.273158.0:18769 2014-07-30 14:10:22.639242 2014-08-13 18:06:06.495271 7f5c973a57a0 20 read_log 191648'355 (0'0) modify 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 by client.273158.0:20631 2014-07-30 14:10:34.448792 2014-08-13 18:06:06.495283 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 by client.275175.0:1876 2014-07-30 14:18:40.498228 2014-08-13 18:06:06.495295 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 by client.275175.0:3580 2014-07-30 14:18:46.047221 2014-08-13 18:06:06.495308 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 by client.275175.0:3942 2014-07-30 14:18:47.177767 2014-08-13 18:06:06.495323 7f5c973a57a0 20 read_log 191727'397 (0'0) modify aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 by client.275175.0:5493 2014-07-30 14:18:51.793595 2014-08-13 18:06:06.495335 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 by client.275175.0:6917 2014-07-30 14:18:56.483335 2014-08-13 18:06:06.495347 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 by client.275175.0:8854 2014-07-30 14:19:01.927942 2014-08-13 18:06:06.495369 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 by client.275175.0:8969 2014-07-30 14:19:02.258534 2014-08-13 18:06:06.495385 7f5c973a57a0 20 read_log 191727'406 (0'0) modify c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 by client.275175.0:10506 2014-07-30 14:19:06.655258 2014-08-13 18:06:06.495397 7f5c973a57a0 20 read_log 191727'409 (0'0) modify ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 by client.275175.0:11865 2014-07-30 14:19:10.918722 2014-08-13 18:06:06.495409 7f5c973a57a0 20 read_log 191727'413 (0'0) modify f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 by client.275175.0:14382 2014-07-30 14:19:18.956215 2014-08-13 18:06:06.495421 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 by client.275175.0:16495 2014-07-30 14:19:25.649635 2014-08-13 18:06:06.495433 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 by client.275175.0:18154 2014-07-30 14:19:30.385252 2014-08-13 18:06:06.495445 7f5c973a57a0 20 read_log 191727'428 (0'0) modify 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 by client.275175.0:20701 2014-07-30 14:19:38.272502 2014-08-13 18:06:06.495458 7f5c973a57a0 20 read_log 191727'437 (0'0) modify b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 by client.275175.0:24201 2014-07-30 14:19:49.391611 2014-08-13 18:06:06.495469 7f5c973a57a0 20 read_log 191727'438 (0'0) modify 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 by client.275175.0:24689 2014-07-30 14:19:50.765134 2014-08-13 18:06:06.495482 7f5c973a57a0 20 read_log 192674'458 (0'0) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7007 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495494 7f5c973a57a0 20 read_log 192674'459 (192674'458) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7008 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495506 7f5c973a57a0 20 read_log 192674'460 (192674'459) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7009 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495519 7f5c973a57a0 20 read_log 192674'461 (192674'460) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7010 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495532 7f5c973a57a0 20 read_log 192674'462 (192674'461) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7011 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495544 7f5c973a57a0 20 read_log 192674'463 (192674'462) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7012 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495556 7f5c973a57a0 20 read_log 192674'464 (192674'463) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7013 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495568 7f5c973a57a0 20 read_log 192674'465 (192674'464) modify b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 by client.279772.1:7014 2014-07-31 18:03:49.071000 2014-08-13 18:06:06.495580 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 5bcdd3af/benchmark_data_storage0111-ib.pocstor_35625_object0/head//22 by client.940330.0:1 2014-08-11 16:29:40.419425 2014-08-13 18:06:06.495592 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 6512cbaf/benchmark_data_storage0111-ib.pocstor_35625_object485/head//22 by client.940330.0:486 2014-08-11 16:29:42.642917 2014-08-13 18:06:06.495604 7f5c973a57a0 20 read_log 192888'468 (0'0) modify 8b144baf/benchmark_data_storage0111-ib.pocstor_35625_object1526/head//22 by client.940330.0:1527 2014-08-11 16:29:46.799832 2014-08-13 18:06:06.495644 7f5c973a57a0 20 read_log 192888'469 (0'0) modify c27f6baf/benchmark_data_storage0111-ib.pocstor_35625_object2677/head//22 by client.940330.0:2678 2014-08-11 16:29:51.141644 2014-08-13 18:06:06.495661 7f5c973a57a0 20 read_log 192888'470 (0'0) modify c329a3af/benchmark_data_storage0111-ib.pocstor_35625_object4914/head//22 by client.940330.0:4915 2014-08-11 16:29:59.729074 2014-08-13 18:06:06.495673 7f5c973a57a0 20 read_log 192888'471 (0'0) modify 746763af/benchmark_data_storage0111-ib.pocstor_35625_object5143/head//22 by client.940330.0:5144 2014-08-11 16:30:00.537737 2014-08-13 18:06:06.495687 7f5c973a57a0 20 read_log 192888'472 (0'0) modify b3abc3af/benchmark_data_storage0111-ib.pocstor_35625_object5366/head//22 by client.940330.0:5367 2014-08-11 16:30:01.330123 2014-08-13 18:06:06.495700 7f5c973a57a0 20 read_log 192888'473 (0'0) modify ead2dbaf/benchmark_data_storage0111-ib.pocstor_35625_object13251/head//22 by client.940330.0:13252 2014-08-11 16:30:30.859372 2014-08-13 18:06:06.495715 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 84c513af/benchmark_data_storage0111-ib.pocstor_35625_object13740/head//22 by client.940330.0:13741 2014-08-11 16:30:32.659321 2014-08-13 18:06:06.495727 7f5c973a57a0 20 read_log 192888'475 (0'0) modify f01003af/benchmark_data_storage0111-ib.pocstor_35625_object13922/head//22 by client.940330.0:13923 2014-08-11 16:30:33.352296 2014-08-13 18:06:06.495741 7f5c973a57a0 20 read_log 192888'476 (0'0) modify 343b43af/benchmark_data_storage0111-ib.pocstor_35625_object14810/head//22 by client.940330.0:14811 2014-08-11 16:30:36.783848 2014-08-13 18:06:06.495758 7f5c973a57a0 20 read_log 192888'477 (0'0) modify 67d23baf/benchmark_data_storage0111-ib.pocstor_35625_object19907/head//22 by client.940330.0:19908 2014-08-11 16:30:56.471441 2014-08-13 18:06:06.495791 7f5c973a57a0 20 read_log 192888'478 (0'0) modify 5474b3af/benchmark_data_storage0111-ib.pocstor_35625_object20104/head//22 by client.940330.0:20105 2014-08-11 16:30:57.198105 2014-08-13 18:06:06.495804 7f5c973a57a0 20 read_log 192888'479 (0'0) modify c60f93af/benchmark_data_storage0111-ib.pocstor_35625_object22423/head//22 by client.940330.0:22424 2014-08-11 16:31:06.133146 2014-08-13 18:06:06.495818 7f5c973a57a0 20 read_log 192888'480 (0'0) modify b1891baf/benchmark_data_storage0111-ib.pocstor_35625_object24479/head//22 by client.940330.0:24480 2014-08-11 16:31:13.896141 2014-08-13 18:06:06.495830 7f5c973a57a0 20 read_log 192888'481 (0'0) modify bcfc53af/benchmark_data_storage0111-ib.pocstor_35625_object25442/head//22 by client.940330.0:25443 2014-08-11 16:31:17.614531 2014-08-13 18:06:06.495842 7f5c973a57a0 20 read_log 192888'482 (0'0) modify a46103af/benchmark_data_storage0111-ib.pocstor_35625_object25479/head//22 by client.940330.0:25480 2014-08-11 16:31:17.756817 2014-08-13 18:06:06.495854 7f5c973a57a0 20 read_log 192888'483 (0'0) modify 103f5baf/benchmark_data_storage0111-ib.pocstor_35625_object26257/head//22 by client.940330.0:26258 2014-08-11 16:31:20.802762 2014-08-13 18:06:06.495868 7f5c973a57a0 20 read_log 192888'484 (0'0) modify 86588baf/benchmark_data_storage0111-ib.pocstor_35625_object29331/head//22 by client.940330.0:29332 2014-08-11 16:31:32.927228 2014-08-13 18:06:06.495880 7f5c973a57a0 20 read_log 192888'485 (0'0) modify 5d0573af/benchmark_data_storage0111-ib.pocstor_35625_object31175/head//22 by client.940330.0:31176 2014-08-11 16:31:39.880561 2014-08-13 18:06:06.495894 7f5c973a57a0 20 read_log 192888'486 (192888'466) delete 5bcdd3af/benchmark_data_storage0111-ib.pocstor_35625_object0/head//22 by client.940330.0:31308 2014-08-11 16:31:41.446696 2014-08-13 18:06:06.495907 7f5c973a57a0 20 read_log 192888'487 (192888'467) delete 6512cbaf/benchmark_data_storage0111-ib.pocstor_35625_object485/head//22 by client.940330.0:31793 2014-08-11 16:31:41.618691 2014-08-13 18:06:06.495922 7f5c973a57a0 20 read_log 192888'488 (192888'468) delete 8b144baf/benchmark_data_storage0111-ib.pocstor_35625_object1526/head//22 by client.940330.0:32834 2014-08-11 16:31:41.873868 2014-08-13 18:06:06.495942 7f5c973a57a0 20 read_log 192888'489 (192888'469) delete c27f6baf/benchmark_data_storage0111-ib.pocstor_35625_object2677/head//22 by client.940330.0:33985 2014-08-11 16:31:42.165785 2014-08-13 18:06:06.495957 7f5c973a57a0 20 read_log 192888'490 (192888'470) delete c329a3af/benchmark_data_storage0111-ib.pocstor_35625_object4914/head//22 by client.940330.0:36222 2014-08-11 16:31:42.795676 2014-08-13 18:06:06.495969 7f5c973a57a0 20 read_log 192888'491 (192888'471) delete 746763af/benchmark_data_storage0111-ib.pocstor_35625_object5143/head//22 by client.940330.0:36451 2014-08-11 16:31:42.852981 2014-08-13 18:06:06.495983 7f5c973a57a0 20 read_log 192888'492 (192888'472) delete b3abc3af/benchmark_data_storage0111-ib.pocstor_35625_object5366/head//22 by client.940330.0:36674 2014-08-11 16:31:42.909023 2014-08-13 18:06:06.495996 7f5c973a57a0 20 read_log 192888'493 (192888'473) delete ead2dbaf/benchmark_data_storage0111-ib.pocstor_35625_object13251/head//22 by client.940330.0:44559 2014-08-11 16:31:44.990613 2014-08-13 18:06:06.496010 7f5c973a57a0 20 read_log 192888'494 (192888'474) delete 84c513af/benchmark_data_storage0111-ib.pocstor_35625_object13740/head//22 by client.940330.0:45048 2014-08-11 16:31:45.114960 2014-08-13 18:06:06.496023 7f5c973a57a0 20 read_log 192888'495 (192888'475) delete f01003af/benchmark_data_storage0111-ib.pocstor_35625_object13922/head//22 by client.940330.0:45230 2014-08-11 16:31:45.162688 2014-08-13 18:06:06.496036 7f5c973a57a0 20 read_log 192888'496 (192888'476) delete 343b43af/benchmark_data_storage0111-ib.pocstor_35625_object14810/head//22 by client.940330.0:46118 2014-08-11 16:31:45.402568 2014-08-13 18:06:06.496050 7f5c973a57a0 20 read_log 192888'497 (192888'477) delete 67d23baf/benchmark_data_storage0111-ib.pocstor_35625_object19907/head//22 by client.940330.0:51215 2014-08-11 16:31:46.840277 2014-08-13 18:06:06.496066 7f5c973a57a0 20 read_log 192888'498 (192888'478) delete 5474b3af/benchmark_data_storage0111-ib.pocstor_35625_object20104/head//22 by client.940330.0:51412 2014-08-11 16:31:46.911880 2014-08-13 18:06:06.496078 7f5c973a57a0 20 read_log 192888'499 (192888'479) delete c60f93af/benchmark_data_storage0111-ib.pocstor_35625_object22423/head//22 by client.940330.0:53731 2014-08-11 16:31:47.581184 2014-08-13 18:06:06.496093 7f5c973a57a0 20 read_log 192888'500 (192888'480) delete b1891baf/benchmark_data_storage0111-ib.pocstor_35625_object24479/head//22 by client.940330.0:55787 2014-08-11 16:31:48.183283 2014-08-13 18:06:06.496105 7f5c973a57a0 20 read_log 192888'501 (192888'481) delete bcfc53af/benchmark_data_storage0111-ib.pocstor_35625_object25442/head//22 by client.940330.0:56750 2014-08-11 16:31:48.481125 2014-08-13 18:06:06.496117 7f5c973a57a0 20 read_log 192888'502 (192888'482) delete a46103af/benchmark_data_storage0111-ib.pocstor_35625_object25479/head//22 by client.940330.0:56787 2014-08-11 16:31:48.493557 2014-08-13 18:06:06.496132 7f5c973a57a0 20 read_log 192888'503 (192888'483) delete 103f5baf/benchmark_data_storage0111-ib.pocstor_35625_object26257/head//22 by client.940330.0:57565 2014-08-11 16:31:48.720632 2014-08-13 18:06:06.496155 7f5c973a57a0 20 read_log 192888'504 (192888'484) delete 86588baf/benchmark_data_storage0111-ib.pocstor_35625_object29331/head//22 by client.940330.0:60639 2014-08-11 16:31:49.561774 2014-08-13 18:06:06.496171 7f5c973a57a0 20 read_log 192888'505 (192888'485) delete 5d0573af/benchmark_data_storage0111-ib.pocstor_35625_object31175/head//22 by client.940330.0:62483 2014-08-11 16:31:50.112879 2014-08-13 18:06:06.496184 7f5c973a57a0 20 read_log 194215'506 (0'0) modify 66b83af/benchmark_data_storage0111-ib.pocstor_64498_object131/head//22 by client.1049635.0:132 2014-08-12 17:17:17.447484 2014-08-13 18:06:06.496196 7f5c973a57a0 20 read_log 194215'507 (0'0) modify 903563af/benchmark_data_storage0111-ib.pocstor_64498_object262/head//22 by client.1049635.0:263 2014-08-12 17:17:17.922454 2014-08-13 18:06:06.496217 7f5c973a57a0 20 read_log 194215'508 (0'0) modify 2cdbaf/benchmark_data_storage0111-ib.pocstor_64498_object359/head//22 by client.1049635.0:360 2014-08-12 17:17:18.277079 2014-08-13 18:06:06.496232 7f5c973a57a0 20 read_log 194215'509 (0'0) modify fb8ab3af/benchmark_data_storage0111-ib.pocstor_64498_object425/head//22 by client.1049635.0:426 2014-08-12 17:17:18.513313 2014-08-13 18:06:06.496245 7f5c973a57a0 20 read_log 194215'510 (0'0) modify f542cbaf/benchmark_data_storage0111-ib.pocstor_64498_object763/head//22 by client.1049635.0:764 2014-08-12 17:17:19.583517 2014-08-13 18:06:06.496259 7f5c973a57a0 20 read_log 194215'511 (0'0) modify 2a7e03af/benchmark_data_storage0111-ib.pocstor_64498_object943/head//22 by client.1049635.0:944 2014-08-12 17:17:20.169978 2014-08-13 18:06:06.496276 7f5c973a57a0 20 read_log 194215'512 (0'0) modify dd537baf/benchmark_data_storage0111-ib.pocstor_64498_object9149/head//22 by client.1049635.0:9150 2014-08-12 17:17:45.449474 2014-08-13 18:06:06.496289 7f5c973a57a0 20 read_log 194215'513 (0'0) modify 1be2b3af/benchmark_data_storage0111-ib.pocstor_64498_object11766/head//22 by client.1049635.0:11767 2014-08-12 17:17:53.700790 2014-08-13 18:06:06.496303 7f5c973a57a0 20 read_log 194215'514 (0'0) modify 9e45bbaf/benchmark_data_storage0111-ib.pocstor_64498_object16982/head//22 by client.1049635.0:16983 2014-08-12 17:18:11.297403 2014-08-13 18:06:06.496319 7f5c973a57a0 20 read_log 194215'515 (0'0) modify 3f4eb3af/benchmark_data_storage0111-ib.pocstor_64498_object17847/head//22 by client.1049635.0:17848 2014-08-12 17:18:14.580045 2014-08-13 18:06:06.496333 7f5c973a57a0 20 read_log 194215'516 (0'0) modify 45e603af/benchmark_data_storage0111-ib.pocstor_64498_object19221/head//22 by client.1049635.0:19222 2014-08-12 17:18:19.868625 2014-08-13 18:06:06.496346 7f5c973a57a0 20 read_log 194215'517 (0'0) modify 842aabaf/benchmark_data_storage0111-ib.pocstor_64498_object20868/head//22 by client.1049635.0:20869 2014-08-12 17:18:26.035064 2014-08-13 18:06:06.496358 7f5c973a57a0 20 read_log 194215'518 (0'0) modify 5aa89baf/benchmark_data_storage0111-ib.pocstor_64498_object21961/head//22 by client.1049635.0:21962 2014-08-12 17:18:30.185481 2014-08-13 18:06:06.496375 7f5c973a57a0 20 read_log 194215'519 (0'0) modify cdd48baf/benchmark_data_storage0111-ib.pocstor_64498_object22353/head//22 by client.1049635.0:22354 2014-08-12 17:18:31.658260 2014-08-13 18:06:06.496388 7f5c973a57a0 20 read_log 194215'520 (0'0) modify 3a232baf/benchmark_data_storage0111-ib.pocstor_64498_object24288/head//22 by client.1049635.0:24289 2014-08-12 17:18:38.929540 2014-08-13 18:06:06.496412 7f5c973a57a0 20 read_log 194215'521 (0'0) modify 5b6a63af/benchmark_data_storage0111-ib.pocstor_64498_object30696/head//22 by client.1049635.0:30697 2014-08-12 17:19:03.274748 2014-08-13 18:06:06.496427 7f5c973a57a0 20 read_log 194215'522 (0'0) modify 82777baf/benchmark_data_storage0111-ib.pocstor_64498_object32715/head//22 by client.1049635.0:32716 2014-08-12 17:19:11.312579 2014-08-13 18:06:06.496451 7f5c973a57a0 20 read_log 194215'523 (194215'506) delete 66b83af/benchmark_data_storage0111-ib.pocstor_64498_object131/head//22 by client.1049635.0:34334 2014-08-12 17:19:18.029163 2014-08-13 18:06:06.496468 7f5c973a57a0 20 read_log 194215'524 (194215'507) delete 903563af/benchmark_data_storage0111-ib.pocstor_64498_object262/head//22 by client.1049635.0:34465 2014-08-12 17:19:18.079370 2014-08-13 18:06:06.496481 7f5c973a57a0 20 read_log 194215'525 (194215'508) delete 2cdbaf/benchmark_data_storage0111-ib.pocstor_64498_object359/head//22 by client.1049635.0:34562 2014-08-12 17:19:18.110409 2014-08-13 18:06:06.496564 7f5c973a57a0 20 read_log 194215'526 (194215'509) delete fb8ab3af/benchmark_data_storage0111-ib.pocstor_64498_object425/head//22 by client.1049635.0:34628 2014-08-12 17:19:18.136084 2014-08-13 18:06:06.496583 7f5c973a57a0 20 read_log 194215'527 (194215'510) delete f542cbaf/benchmark_data_storage0111-ib.pocstor_64498_object763/head//22 by client.1049635.0:34966 2014-08-12 17:19:18.240414 2014-08-13 18:06:06.496613 7f5c973a57a0 20 read_log 194215'528 (194215'511) delete 2a7e03af/benchmark_data_storage0111-ib.pocstor_64498_object943/head//22 by client.1049635.0:35146 2014-08-12 17:19:18.301070 2014-08-13 18:06:06.496626 7f5c973a57a0 20 read_log 194215'529 (194215'512) delete dd537baf/benchmark_data_storage0111-ib.pocstor_64498_object9149/head//22 by client.1049635.0:43352 2014-08-12 17:19:20.969513 2014-08-13 18:06:06.496639 7f5c973a57a0 20 read_log 194215'530 (194215'513) delete 1be2b3af/benchmark_data_storage0111-ib.pocstor_64498_object11766/head//22 by client.1049635.0:45969 2014-08-12 17:19:21.830180 2014-08-13 18:06:06.496652 7f5c973a57a0 20 read_log 194215'531 (194215'514) delete 9e45bbaf/benchmark_data_storage0111-ib.pocstor_64498_object16982/head//22 by client.1049635.0:51185 2014-08-12 17:19:23.503350 2014-08-13 18:06:06.496664 7f5c973a57a0 20 read_log 194215'532 (194215'515) delete 3f4eb3af/benchmark_data_storage0111-ib.pocstor_64498_object17847/head//22 by client.1049635.0:52050 2014-08-12 17:19:23.776275 2014-08-13 18:06:06.496676 7f5c973a57a0 20 read_log 194215'533 (194215'516) delete 45e603af/benchmark_data_storage0111-ib.pocstor_64498_object19221/head//22 by client.1049635.0:53424 2014-08-12 17:19:24.241367 2014-08-13 18:06:06.496688 7f5c973a57a0 20 read_log 194215'534 (194215'517) delete 842aabaf/benchmark_data_storage0111-ib.pocstor_64498_object20868/head//22 by client.1049635.0:55071 2014-08-12 17:19:24.839028 2014-08-13 18:06:06.496700 7f5c973a57a0 20 read_log 194215'535 (194215'518) delete 5aa89baf/benchmark_data_storage0111-ib.pocstor_64498_object21961/head//22 by client.1049635.0:56164 2014-08-12 17:19:25.218864 2014-08-13 18:06:06.496714 7f5c973a57a0 20 read_log 194215'536 (194215'519) delete cdd48baf/benchmark_data_storage0111-ib.pocstor_64498_object22353/head//22 by client.1049635.0:56556 2014-08-12 17:19:25.363899 2014-08-13 18:06:06.496726 7f5c973a57a0 20 read_log 194215'537 (194215'520) delete 3a232baf/benchmark_data_storage0111-ib.pocstor_64498_object24288/head//22 by client.1049635.0:58491 2014-08-12 17:19:26.061637 2014-08-13 18:06:06.496738 7f5c973a57a0 20 read_log 194215'538 (194215'521) delete 5b6a63af/benchmark_data_storage0111-ib.pocstor_64498_object30696/head//22 by client.1049635.0:64899 2014-08-12 17:19:28.272326 2014-08-13 18:06:06.496751 7f5c973a57a0 20 read_log 194215'539 (194215'522) delete 82777baf/benchmark_data_storage0111-ib.pocstor_64498_object32715/head//22 by client.1049635.0:66918 2014-08-12 17:19:29.010187 2014-08-13 18:06:06.496868 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.496887 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=0 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.496903 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=0 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002058 0 0.000000 2014-08-13 18:06:06.496916 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=0 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.496925 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=0 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.496934 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'539], crt=191727'457) 2014-08-13 18:06:06.496944 7f5c973a57a0 10 osd.4 194487 pgid 22.3b9 coll 22.3b9_head 2014-08-13 18:06:06.497103 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3b9 2014-08-13 18:06:06.497112 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.497138 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3b9(unlocked)] enter Initial 2014-08-13 18:06:06.497155 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3b9(unlocked)] enter NotTrimming 2014-08-13 18:06:06.497303 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.497453 7f5c973a57a0 20 read_log 191648'28 (0'0) modify 2a5b3bb9/benchmark_data_storage0111-ib.pocstor_40745_object1810/head//22 by client.273158.0:1811 2014-07-30 14:08:55.427627 2014-08-13 18:06:06.497474 7f5c973a57a0 20 read_log 191648'115 (0'0) modify 21a30bb9/benchmark_data_storage0111-ib.pocstor_40745_object8125/head//22 by client.273158.0:8126 2014-07-30 14:09:26.690956 2014-08-13 18:06:06.497488 7f5c973a57a0 20 read_log 191648'123 (0'0) modify 693b9/benchmark_data_storage0111-ib.pocstor_40745_object8507/head//22 by client.273158.0:8508 2014-07-30 14:09:28.932306 2014-08-13 18:06:06.497500 7f5c973a57a0 20 read_log 191648'188 (0'0) modify 203ef3b9/benchmark_data_storage0111-ib.pocstor_40745_object12560/head//22 by client.273158.0:12561 2014-07-30 14:09:50.361312 2014-08-13 18:06:06.497513 7f5c973a57a0 20 read_log 191648'214 (0'0) modify 567f43b9/benchmark_data_storage0111-ib.pocstor_40745_object14002/head//22 by client.273158.0:14003 2014-07-30 14:09:57.014126 2014-08-13 18:06:06.497525 7f5c973a57a0 20 read_log 191648'225 (0'0) modify e8d303b9/benchmark_data_storage0111-ib.pocstor_40745_object14493/head//22 by client.273158.0:14494 2014-07-30 14:10:00.057762 2014-08-13 18:06:06.497537 7f5c973a57a0 20 read_log 191648'235 (0'0) modify 2ca733b9/benchmark_data_storage0111-ib.pocstor_40745_object15068/head//22 by client.273158.0:15069 2014-07-30 14:10:02.937032 2014-08-13 18:06:06.497549 7f5c973a57a0 20 read_log 191648'236 (0'0) modify 1ce98bb9/benchmark_data_storage0111-ib.pocstor_40745_object15366/head//22 by client.273158.0:15367 2014-07-30 14:10:04.129794 2014-08-13 18:06:06.497562 7f5c973a57a0 20 read_log 191648'318 (0'0) modify e0efbb9/benchmark_data_storage0111-ib.pocstor_40745_object20766/head//22 by client.273158.0:20767 2014-07-30 14:10:35.381754 2014-08-13 18:06:06.497574 7f5c973a57a0 20 read_log 191648'352 (0'0) modify a9dcd3b9/benchmark_data_storage0111-ib.pocstor_40745_object22760/head//22 by client.273158.0:22761 2014-07-30 14:10:48.105926 2014-08-13 18:06:06.497586 7f5c973a57a0 20 read_log 191727'356 (0'0) modify e2d5f3b9/benchmark_data_storage0111-ib.pocstor_51285_object2765/head//22 by client.275175.0:2766 2014-07-30 14:18:43.475350 2014-08-13 18:06:06.497598 7f5c973a57a0 20 read_log 191727'362 (0'0) modify 320a23b9/benchmark_data_storage0111-ib.pocstor_51285_object5104/head//22 by client.275175.0:5105 2014-07-30 14:18:50.657353 2014-08-13 18:06:06.497610 7f5c973a57a0 20 read_log 191727'367 (0'0) modify f4667bb9/benchmark_data_storage0111-ib.pocstor_51285_object6287/head//22 by client.275175.0:6288 2014-07-30 14:18:54.117804 2014-08-13 18:06:06.497622 7f5c973a57a0 20 read_log 191727'369 (0'0) modify cd3d63b9/benchmark_data_storage0111-ib.pocstor_51285_object7539/head//22 by client.275175.0:7540 2014-07-30 14:18:58.242087 2014-08-13 18:06:06.497634 7f5c973a57a0 20 read_log 191727'370 (0'0) modify c4ca0bb9/benchmark_data_storage0111-ib.pocstor_51285_object7759/head//22 by client.275175.0:7760 2014-07-30 14:18:58.856387 2014-08-13 18:06:06.497646 7f5c973a57a0 20 read_log 191727'372 (0'0) modify f3844bb9/benchmark_data_storage0111-ib.pocstor_51285_object8114/head//22 by client.275175.0:8115 2014-07-30 14:18:59.841862 2014-08-13 18:06:06.497658 7f5c973a57a0 20 read_log 191727'376 (0'0) modify ba6093b9/benchmark_data_storage0111-ib.pocstor_51285_object10104/head//22 by client.275175.0:10105 2014-07-30 14:19:05.471588 2014-08-13 18:06:06.497670 7f5c973a57a0 20 read_log 191727'382 (0'0) modify e50d7bb9/benchmark_data_storage0111-ib.pocstor_51285_object14050/head//22 by client.275175.0:14051 2014-07-30 14:19:18.021904 2014-08-13 18:06:06.497691 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 751c93b9/benchmark_data_storage0111-ib.pocstor_51285_object18539/head//22 by client.275175.0:18540 2014-07-30 14:19:31.596072 2014-08-13 18:06:06.497706 7f5c973a57a0 20 read_log 191727'393 (0'0) modify 9d994bb9/benchmark_data_storage0111-ib.pocstor_51285_object19826/head//22 by client.275175.0:19827 2014-07-30 14:19:35.682687 2014-08-13 18:06:06.497718 7f5c973a57a0 20 read_log 191727'402 (0'0) modify e5b053b9/benchmark_data_storage0111-ib.pocstor_51285_object25986/head//22 by client.275175.0:25987 2014-07-30 14:19:54.685530 2014-08-13 18:06:06.497730 7f5c973a57a0 20 read_log 191727'403 (0'0) modify cfadbb9/benchmark_data_storage0111-ib.pocstor_51285_object26228/head//22 by client.275175.0:26229 2014-07-30 14:19:55.517587 2014-08-13 18:06:06.497742 7f5c973a57a0 20 read_log 191727'404 (0'0) modify ee764bb9/benchmark_data_storage0111-ib.pocstor_51285_object26342/head//22 by client.275175.0:26343 2014-07-30 14:19:55.835792 2014-08-13 18:06:06.497754 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 91a0ebb9/benchmark_data_storage0111-ib.pocstor_51285_object28143/head//22 by client.275175.0:28144 2014-07-30 14:20:01.430194 2014-08-13 18:06:06.497766 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 36d8ebb9/benchmark_data_storage0111-ib.pocstor_51285_object30225/head//22 by client.275175.0:30226 2014-07-30 14:20:07.791509 2014-08-13 18:06:06.497778 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 54abbbb9/benchmark_data_storage0111-ib.pocstor_51285_object30594/head//22 by client.275175.0:30595 2014-07-30 14:20:08.839064 2014-08-13 18:06:06.497790 7f5c973a57a0 20 read_log 191727'416 (0'0) modify c9b053b9/benchmark_data_storage0111-ib.pocstor_51285_object32643/head//22 by client.275175.0:32644 2014-07-30 14:20:14.704920 2014-08-13 18:06:06.497802 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 6eeb43b9/benchmark_data_storage0111-ib.pocstor_51285_object33445/head//22 by client.275175.0:33446 2014-07-30 14:20:17.234508 2014-08-13 18:06:06.497814 7f5c973a57a0 20 read_log 191727'419 (0'0) modify 67d0e3b9/benchmark_data_storage0111-ib.pocstor_51285_object33946/head//22 by client.275175.0:33947 2014-07-30 14:20:18.839143 2014-08-13 18:06:06.497826 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 803ccbb9/benchmark_data_storage0111-ib.pocstor_51285_object34472/head//22 by client.275175.0:34473 2014-07-30 14:20:20.470415 2014-08-13 18:06:06.497838 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 46957bb9/benchmark_data_storage0111-ib.pocstor_51285_object34917/head//22 by client.275175.0:34918 2014-07-30 14:20:21.751892 2014-08-13 18:06:06.497850 7f5c973a57a0 20 read_log 191727'432 (0'0) modify 680b5bb9/benchmark_data_storage0111-ib.pocstor_51285_object37458/head//22 by client.275175.0:37459 2014-07-30 14:20:29.732351 2014-08-13 18:06:06.497863 7f5c973a57a0 20 read_log 191727'433 (0'0) modify 969c53b9/benchmark_data_storage0111-ib.pocstor_51285_object37635/head//22 by client.275175.0:37636 2014-07-30 14:20:30.225573 2014-08-13 18:06:06.497875 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 738eabb9/benchmark_data_storage0111-ib.pocstor_35625_object2657/head//22 by client.940330.0:2658 2014-08-11 16:29:51.070731 2014-08-13 18:06:06.497887 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 9aea63b9/benchmark_data_storage0111-ib.pocstor_35625_object3811/head//22 by client.940330.0:3812 2014-08-11 16:29:55.641650 2014-08-13 18:06:06.497899 7f5c973a57a0 20 read_log 192888'439 (0'0) modify b1ea83b9/benchmark_data_storage0111-ib.pocstor_35625_object4015/head//22 by client.940330.0:4016 2014-08-11 16:29:56.386850 2014-08-13 18:06:06.497912 7f5c973a57a0 20 read_log 192888'440 (0'0) modify a8846bb9/benchmark_data_storage0111-ib.pocstor_35625_object4917/head//22 by client.940330.0:4918 2014-08-11 16:29:59.739930 2014-08-13 18:06:06.497934 7f5c973a57a0 20 read_log 192888'441 (0'0) modify be9243b9/benchmark_data_storage0111-ib.pocstor_35625_object5330/head//22 by client.940330.0:5331 2014-08-11 16:30:01.202480 2014-08-13 18:06:06.497949 7f5c973a57a0 20 read_log 192888'442 (0'0) modify ab2ee3b9/benchmark_data_storage0111-ib.pocstor_35625_object8334/head//22 by client.940330.0:8335 2014-08-11 16:30:12.010065 2014-08-13 18:06:06.497962 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 5c7df3b9/benchmark_data_storage0111-ib.pocstor_35625_object8868/head//22 by client.940330.0:8869 2014-08-11 16:30:13.872213 2014-08-13 18:06:06.497977 7f5c973a57a0 20 read_log 192888'444 (0'0) modify cbdec3b9/benchmark_data_storage0111-ib.pocstor_35625_object9820/head//22 by client.940330.0:9821 2014-08-11 16:30:17.201017 2014-08-13 18:06:06.497989 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 9851f3b9/benchmark_data_storage0111-ib.pocstor_35625_object10015/head//22 by client.940330.0:10016 2014-08-11 16:30:17.960196 2014-08-13 18:06:06.498003 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 76ca73b9/benchmark_data_storage0111-ib.pocstor_35625_object10720/head//22 by client.940330.0:10721 2014-08-11 16:30:20.713060 2014-08-13 18:06:06.498016 7f5c973a57a0 20 read_log 192888'447 (0'0) modify db73c3b9/benchmark_data_storage0111-ib.pocstor_35625_object11988/head//22 by client.940330.0:11989 2014-08-11 16:30:25.895466 2014-08-13 18:06:06.498049 7f5c973a57a0 20 read_log 192888'448 (0'0) modify cf8063b9/benchmark_data_storage0111-ib.pocstor_35625_object12943/head//22 by client.940330.0:12944 2014-08-11 16:30:29.551052 2014-08-13 18:06:06.498065 7f5c973a57a0 20 read_log 192888'449 (0'0) modify 254a23b9/benchmark_data_storage0111-ib.pocstor_35625_object18472/head//22 by client.940330.0:18473 2014-08-11 16:30:50.664448 2014-08-13 18:06:06.498078 7f5c973a57a0 20 read_log 192888'450 (0'0) modify b22e63b9/benchmark_data_storage0111-ib.pocstor_35625_object18627/head//22 by client.940330.0:18628 2014-08-11 16:30:51.233268 2014-08-13 18:06:06.498092 7f5c973a57a0 20 read_log 192888'451 (0'0) modify e132a3b9/benchmark_data_storage0111-ib.pocstor_35625_object19274/head//22 by client.940330.0:19275 2014-08-11 16:30:53.679987 2014-08-13 18:06:06.498105 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 620e8bb9/benchmark_data_storage0111-ib.pocstor_35625_object19312/head//22 by client.940330.0:19313 2014-08-11 16:30:53.824769 2014-08-13 18:06:06.498119 7f5c973a57a0 20 read_log 192888'453 (0'0) modify de36b3b9/benchmark_data_storage0111-ib.pocstor_35625_object20095/head//22 by client.940330.0:20096 2014-08-11 16:30:57.165253 2014-08-13 18:06:06.498131 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 8752b3b9/benchmark_data_storage0111-ib.pocstor_35625_object23292/head//22 by client.940330.0:23293 2014-08-11 16:31:09.484221 2014-08-13 18:06:06.498143 7f5c973a57a0 20 read_log 192888'455 (0'0) modify ec5a63b9/benchmark_data_storage0111-ib.pocstor_35625_object23813/head//22 by client.940330.0:23814 2014-08-11 16:31:11.466674 2014-08-13 18:06:06.498166 7f5c973a57a0 20 read_log 192888'456 (0'0) modify 39ac9bb9/benchmark_data_storage0111-ib.pocstor_35625_object24168/head//22 by client.940330.0:24169 2014-08-11 16:31:12.763904 2014-08-13 18:06:06.498184 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 452b53b9/benchmark_data_storage0111-ib.pocstor_35625_object31276/head//22 by client.940330.0:31277 2014-08-11 16:31:40.313662 2014-08-13 18:06:06.498196 7f5c973a57a0 20 read_log 192888'458 (192888'437) delete 738eabb9/benchmark_data_storage0111-ib.pocstor_35625_object2657/head//22 by client.940330.0:33965 2014-08-11 16:31:42.159731 2014-08-13 18:06:06.498211 7f5c973a57a0 20 read_log 192888'459 (192888'438) delete 9aea63b9/benchmark_data_storage0111-ib.pocstor_35625_object3811/head//22 by client.940330.0:35119 2014-08-11 16:31:42.476657 2014-08-13 18:06:06.498224 7f5c973a57a0 20 read_log 192888'460 (192888'439) delete b1ea83b9/benchmark_data_storage0111-ib.pocstor_35625_object4015/head//22 by client.940330.0:35323 2014-08-11 16:31:42.535797 2014-08-13 18:06:06.498265 7f5c973a57a0 20 read_log 192888'461 (192888'440) delete a8846bb9/benchmark_data_storage0111-ib.pocstor_35625_object4917/head//22 by client.940330.0:36225 2014-08-11 16:31:42.796438 2014-08-13 18:06:06.498281 7f5c973a57a0 20 read_log 192888'462 (192888'441) delete be9243b9/benchmark_data_storage0111-ib.pocstor_35625_object5330/head//22 by client.940330.0:36638 2014-08-11 16:31:42.899713 2014-08-13 18:06:06.498297 7f5c973a57a0 20 read_log 192888'463 (192888'442) delete ab2ee3b9/benchmark_data_storage0111-ib.pocstor_35625_object8334/head//22 by client.940330.0:39642 2014-08-11 16:31:43.674961 2014-08-13 18:06:06.498312 7f5c973a57a0 20 read_log 192888'464 (192888'443) delete 5c7df3b9/benchmark_data_storage0111-ib.pocstor_35625_object8868/head//22 by client.940330.0:40176 2014-08-11 16:31:43.809752 2014-08-13 18:06:06.498324 7f5c973a57a0 20 read_log 192888'465 (192888'444) delete cbdec3b9/benchmark_data_storage0111-ib.pocstor_35625_object9820/head//22 by client.940330.0:41128 2014-08-11 16:31:44.063966 2014-08-13 18:06:06.498337 7f5c973a57a0 20 read_log 192888'466 (192888'445) delete 9851f3b9/benchmark_data_storage0111-ib.pocstor_35625_object10015/head//22 by client.940330.0:41323 2014-08-11 16:31:44.118237 2014-08-13 18:06:06.498350 7f5c973a57a0 20 read_log 192888'467 (192888'446) delete 76ca73b9/benchmark_data_storage0111-ib.pocstor_35625_object10720/head//22 by client.940330.0:42028 2014-08-11 16:31:44.317599 2014-08-13 18:06:06.498362 7f5c973a57a0 20 read_log 192888'468 (192888'447) delete db73c3b9/benchmark_data_storage0111-ib.pocstor_35625_object11988/head//22 by client.940330.0:43296 2014-08-11 16:31:44.654943 2014-08-13 18:06:06.498389 7f5c973a57a0 20 read_log 192888'469 (192888'448) delete cf8063b9/benchmark_data_storage0111-ib.pocstor_35625_object12943/head//22 by client.940330.0:44251 2014-08-11 16:31:44.903657 2014-08-13 18:06:06.498402 7f5c973a57a0 20 read_log 192888'470 (192888'449) delete 254a23b9/benchmark_data_storage0111-ib.pocstor_35625_object18472/head//22 by client.940330.0:49780 2014-08-11 16:31:46.439613 2014-08-13 18:06:06.498414 7f5c973a57a0 20 read_log 192888'471 (192888'450) delete b22e63b9/benchmark_data_storage0111-ib.pocstor_35625_object18627/head//22 by client.940330.0:49935 2014-08-11 16:31:46.480119 2014-08-13 18:06:06.498427 7f5c973a57a0 20 read_log 192888'472 (192888'451) delete e132a3b9/benchmark_data_storage0111-ib.pocstor_35625_object19274/head//22 by client.940330.0:50582 2014-08-11 16:31:46.654929 2014-08-13 18:06:06.498449 7f5c973a57a0 20 read_log 192888'473 (192888'452) delete 620e8bb9/benchmark_data_storage0111-ib.pocstor_35625_object19312/head//22 by client.940330.0:50620 2014-08-11 16:31:46.666345 2014-08-13 18:06:06.498469 7f5c973a57a0 20 read_log 192888'474 (192888'453) delete de36b3b9/benchmark_data_storage0111-ib.pocstor_35625_object20095/head//22 by client.940330.0:51403 2014-08-11 16:31:46.908417 2014-08-13 18:06:06.498493 7f5c973a57a0 20 read_log 192888'475 (192888'454) delete 8752b3b9/benchmark_data_storage0111-ib.pocstor_35625_object23292/head//22 by client.940330.0:54600 2014-08-11 16:31:47.842640 2014-08-13 18:06:06.498509 7f5c973a57a0 20 read_log 192888'476 (192888'455) delete ec5a63b9/benchmark_data_storage0111-ib.pocstor_35625_object23813/head//22 by client.940330.0:55121 2014-08-11 16:31:47.980801 2014-08-13 18:06:06.498522 7f5c973a57a0 20 read_log 192888'477 (192888'456) delete 39ac9bb9/benchmark_data_storage0111-ib.pocstor_35625_object24168/head//22 by client.940330.0:55476 2014-08-11 16:31:48.091476 2014-08-13 18:06:06.498534 7f5c973a57a0 20 read_log 192888'478 (192888'457) delete 452b53b9/benchmark_data_storage0111-ib.pocstor_35625_object31276/head//22 by client.940330.0:62584 2014-08-11 16:31:50.144047 2014-08-13 18:06:06.498547 7f5c973a57a0 20 read_log 194215'479 (0'0) modify 6b5bebb9/benchmark_data_storage0111-ib.pocstor_64498_object2843/head//22 by client.1049635.0:2844 2014-08-12 17:17:26.266332 2014-08-13 18:06:06.498596 7f5c973a57a0 20 read_log 194215'480 (0'0) modify ddfd53b9/benchmark_data_storage0111-ib.pocstor_64498_object4265/head//22 by client.1049635.0:4266 2014-08-12 17:17:30.854922 2014-08-13 18:06:06.498613 7f5c973a57a0 20 read_log 194215'481 (0'0) modify b842abb9/benchmark_data_storage0111-ib.pocstor_64498_object5602/head//22 by client.1049635.0:5603 2014-08-12 17:17:34.907348 2014-08-13 18:06:06.498626 7f5c973a57a0 20 read_log 194215'482 (0'0) modify aa9f53b9/benchmark_data_storage0111-ib.pocstor_64498_object8634/head//22 by client.1049635.0:8635 2014-08-12 17:17:43.977204 2014-08-13 18:06:06.498646 7f5c973a57a0 20 read_log 194215'483 (0'0) modify fa5c43b9/benchmark_data_storage0111-ib.pocstor_64498_object9329/head//22 by client.1049635.0:9330 2014-08-12 17:17:45.975330 2014-08-13 18:06:06.498659 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 8eb7fbb9/benchmark_data_storage0111-ib.pocstor_64498_object10300/head//22 by client.1049635.0:10301 2014-08-12 17:17:48.781179 2014-08-13 18:06:06.498681 7f5c973a57a0 20 read_log 194215'485 (0'0) modify b0b14bb9/benchmark_data_storage0111-ib.pocstor_64498_object10614/head//22 by client.1049635.0:10615 2014-08-12 17:17:49.975104 2014-08-13 18:06:06.498697 7f5c973a57a0 20 read_log 194215'486 (0'0) modify d36a03b9/benchmark_data_storage0111-ib.pocstor_64498_object13616/head//22 by client.1049635.0:13617 2014-08-12 17:17:59.412824 2014-08-13 18:06:06.498730 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 58709bb9/benchmark_data_storage0111-ib.pocstor_64498_object16289/head//22 by client.1049635.0:16290 2014-08-12 17:18:08.654953 2014-08-13 18:06:06.498743 7f5c973a57a0 20 read_log 194215'488 (0'0) modify b64933b9/benchmark_data_storage0111-ib.pocstor_64498_object16505/head//22 by client.1049635.0:16506 2014-08-12 17:18:09.484027 2014-08-13 18:06:06.498756 7f5c973a57a0 20 read_log 194215'489 (0'0) modify ef739bb9/benchmark_data_storage0111-ib.pocstor_64498_object17678/head//22 by client.1049635.0:17679 2014-08-12 17:18:13.938898 2014-08-13 18:06:06.498768 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 80fdfbb9/benchmark_data_storage0111-ib.pocstor_64498_object22606/head//22 by client.1049635.0:22607 2014-08-12 17:18:32.668514 2014-08-13 18:06:06.498780 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 9dc7f3b9/benchmark_data_storage0111-ib.pocstor_64498_object25208/head//22 by client.1049635.0:25209 2014-08-12 17:18:42.485172 2014-08-13 18:06:06.498792 7f5c973a57a0 20 read_log 194215'492 (0'0) modify e53a93b9/benchmark_data_storage0111-ib.pocstor_64498_object27601/head//22 by client.1049635.0:27602 2014-08-12 17:18:51.393771 2014-08-13 18:06:06.498805 7f5c973a57a0 20 read_log 194215'493 (0'0) modify cd3df3b9/benchmark_data_storage0111-ib.pocstor_64498_object27690/head//22 by client.1049635.0:27691 2014-08-12 17:18:51.728081 2014-08-13 18:06:06.498817 7f5c973a57a0 20 read_log 194215'494 (0'0) modify e4e593b9/benchmark_data_storage0111-ib.pocstor_64498_object29725/head//22 by client.1049635.0:29726 2014-08-12 17:18:59.421720 2014-08-13 18:06:06.498829 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 3070a3b9/benchmark_data_storage0111-ib.pocstor_64498_object30382/head//22 by client.1049635.0:30383 2014-08-12 17:19:02.058064 2014-08-13 18:06:06.498841 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 65e683b9/benchmark_data_storage0111-ib.pocstor_64498_object31058/head//22 by client.1049635.0:31059 2014-08-12 17:19:04.786856 2014-08-13 18:06:06.498861 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 149c8bb9/benchmark_data_storage0111-ib.pocstor_64498_object33877/head//22 by client.1049635.0:33878 2014-08-12 17:19:15.720801 2014-08-13 18:06:06.498878 7f5c973a57a0 20 read_log 194215'498 (0'0) modify f3b8d3b9/benchmark_data_storage0111-ib.pocstor_64498_object33910/head//22 by client.1049635.0:33911 2014-08-12 17:19:15.841767 2014-08-13 18:06:06.498890 7f5c973a57a0 20 read_log 194215'499 (194215'479) delete 6b5bebb9/benchmark_data_storage0111-ib.pocstor_64498_object2843/head//22 by client.1049635.0:37046 2014-08-12 17:19:18.916625 2014-08-13 18:06:06.498913 7f5c973a57a0 20 read_log 194215'500 (194215'480) delete ddfd53b9/benchmark_data_storage0111-ib.pocstor_64498_object4265/head//22 by client.1049635.0:38468 2014-08-12 17:19:19.362711 2014-08-13 18:06:06.498928 7f5c973a57a0 20 read_log 194215'501 (194215'481) delete b842abb9/benchmark_data_storage0111-ib.pocstor_64498_object5602/head//22 by client.1049635.0:39805 2014-08-12 17:19:19.814380 2014-08-13 18:06:06.498941 7f5c973a57a0 20 read_log 194215'502 (194215'482) delete aa9f53b9/benchmark_data_storage0111-ib.pocstor_64498_object8634/head//22 by client.1049635.0:42837 2014-08-12 17:19:20.806279 2014-08-13 18:06:06.498953 7f5c973a57a0 20 read_log 194215'503 (194215'483) delete fa5c43b9/benchmark_data_storage0111-ib.pocstor_64498_object9329/head//22 by client.1049635.0:43532 2014-08-12 17:19:21.034170 2014-08-13 18:06:06.498966 7f5c973a57a0 20 read_log 194215'504 (194215'484) delete 8eb7fbb9/benchmark_data_storage0111-ib.pocstor_64498_object10300/head//22 by client.1049635.0:44503 2014-08-12 17:19:21.358604 2014-08-13 18:06:06.498979 7f5c973a57a0 20 read_log 194215'505 (194215'485) delete b0b14bb9/benchmark_data_storage0111-ib.pocstor_64498_object10614/head//22 by client.1049635.0:44817 2014-08-12 17:19:21.454425 2014-08-13 18:06:06.498991 7f5c973a57a0 20 read_log 194215'506 (194215'486) delete d36a03b9/benchmark_data_storage0111-ib.pocstor_64498_object13616/head//22 by client.1049635.0:47819 2014-08-12 17:19:22.444043 2014-08-13 18:06:06.499016 7f5c973a57a0 20 read_log 194215'507 (194215'487) delete 58709bb9/benchmark_data_storage0111-ib.pocstor_64498_object16289/head//22 by client.1049635.0:50492 2014-08-12 17:19:23.280667 2014-08-13 18:06:06.499029 7f5c973a57a0 20 read_log 194215'508 (194215'488) delete b64933b9/benchmark_data_storage0111-ib.pocstor_64498_object16505/head//22 by client.1049635.0:50708 2014-08-12 17:19:23.353655 2014-08-13 18:06:06.499041 7f5c973a57a0 20 read_log 194215'509 (194215'489) delete ef739bb9/benchmark_data_storage0111-ib.pocstor_64498_object17678/head//22 by client.1049635.0:51881 2014-08-12 17:19:23.721221 2014-08-13 18:06:06.499053 7f5c973a57a0 20 read_log 194215'510 (194215'490) delete 80fdfbb9/benchmark_data_storage0111-ib.pocstor_64498_object22606/head//22 by client.1049635.0:56809 2014-08-12 17:19:25.458512 2014-08-13 18:06:06.499065 7f5c973a57a0 20 read_log 194215'511 (194215'491) delete 9dc7f3b9/benchmark_data_storage0111-ib.pocstor_64498_object25208/head//22 by client.1049635.0:59411 2014-08-12 17:19:26.392107 2014-08-13 18:06:06.499077 7f5c973a57a0 20 read_log 194215'512 (194215'492) delete e53a93b9/benchmark_data_storage0111-ib.pocstor_64498_object27601/head//22 by client.1049635.0:61804 2014-08-12 17:19:27.213760 2014-08-13 18:06:06.499089 7f5c973a57a0 20 read_log 194215'513 (194215'493) delete cd3df3b9/benchmark_data_storage0111-ib.pocstor_64498_object27690/head//22 by client.1049635.0:61893 2014-08-12 17:19:27.241593 2014-08-13 18:06:06.499102 7f5c973a57a0 20 read_log 194215'514 (194215'494) delete e4e593b9/benchmark_data_storage0111-ib.pocstor_64498_object29725/head//22 by client.1049635.0:63928 2014-08-12 17:19:27.959511 2014-08-13 18:06:06.499114 7f5c973a57a0 20 read_log 194215'515 (194215'495) delete 3070a3b9/benchmark_data_storage0111-ib.pocstor_64498_object30382/head//22 by client.1049635.0:64585 2014-08-12 17:19:28.169382 2014-08-13 18:06:06.499126 7f5c973a57a0 20 read_log 194215'516 (194215'496) delete 65e683b9/benchmark_data_storage0111-ib.pocstor_64498_object31058/head//22 by client.1049635.0:65261 2014-08-12 17:19:28.400366 2014-08-13 18:06:06.499143 7f5c973a57a0 20 read_log 194215'517 (194215'497) delete 149c8bb9/benchmark_data_storage0111-ib.pocstor_64498_object33877/head//22 by client.1049635.0:68080 2014-08-12 17:19:29.433613 2014-08-13 18:06:06.499164 7f5c973a57a0 20 read_log 194215'518 (194215'498) delete f3b8d3b9/benchmark_data_storage0111-ib.pocstor_64498_object33910/head//22 by client.1049635.0:68113 2014-08-12 17:19:29.447013 2014-08-13 18:06:06.499307 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.499326 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=0 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.499342 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=0 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Initial 0.002204 0 0.000000 2014-08-13 18:06:06.499354 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=0 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.499362 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=0 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.499371 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] log((0'0,194215'518], crt=192888'478) 2014-08-13 18:06:06.499381 7f5c973a57a0 10 osd.4 194487 pgid 22.3d5 coll 22.3d5_head 2014-08-13 18:06:06.499532 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3d5 2014-08-13 18:06:06.499541 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.499567 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d5(unlocked)] enter Initial 2014-08-13 18:06:06.499579 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3d5(unlocked)] enter NotTrimming 2014-08-13 18:06:06.499690 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.499829 7f5c973a57a0 20 read_log 191648'3 (0'0) modify d6591bd5/benchmark_data_storage0111-ib.pocstor_40745_object136/head//22 by client.273158.0:137 2014-07-30 14:08:49.215673 2014-08-13 18:06:06.499849 7f5c973a57a0 20 read_log 191648'144 (0'0) modify deb973d5/benchmark_data_storage0111-ib.pocstor_40745_object8838/head//22 by client.273158.0:8839 2014-07-30 14:09:30.534650 2014-08-13 18:06:06.499862 7f5c973a57a0 20 read_log 191648'200 (0'0) modify 336bbbd5/benchmark_data_storage0111-ib.pocstor_40745_object12424/head//22 by client.273158.0:12425 2014-07-30 14:09:49.721435 2014-08-13 18:06:06.499874 7f5c973a57a0 20 read_log 191648'233 (0'0) modify a25f53d5/benchmark_data_storage0111-ib.pocstor_40745_object15186/head//22 by client.273158.0:15187 2014-07-30 14:10:03.428862 2014-08-13 18:06:06.499887 7f5c973a57a0 20 read_log 191648'269 (0'0) modify 6d90bbd5/benchmark_data_storage0111-ib.pocstor_40745_object17932/head//22 by client.273158.0:17933 2014-07-30 14:10:17.565080 2014-08-13 18:06:06.499899 7f5c973a57a0 20 read_log 191648'288 (0'0) modify 5b1cb3d5/benchmark_data_storage0111-ib.pocstor_40745_object19102/head//22 by client.273158.0:19103 2014-07-30 14:10:24.022907 2014-08-13 18:06:06.499911 7f5c973a57a0 20 read_log 191648'300 (0'0) modify 3cce4bd5/benchmark_data_storage0111-ib.pocstor_40745_object19916/head//22 by client.273158.0:19917 2014-07-30 14:10:30.988403 2014-08-13 18:06:06.499923 7f5c973a57a0 20 read_log 191648'347 (0'0) modify 7edc6bd5/benchmark_data_storage0111-ib.pocstor_40745_object22239/head//22 by client.273158.0:22240 2014-07-30 14:10:45.339247 2014-08-13 18:06:06.499937 7f5c973a57a0 20 read_log 191727'360 (0'0) modify adc4dbd5/benchmark_data_storage0111-ib.pocstor_51285_object1367/head//22 by client.275175.0:1368 2014-07-30 14:18:38.753795 2014-08-13 18:06:06.499978 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 16d2fbd5/benchmark_data_storage0111-ib.pocstor_51285_object2277/head//22 by client.275175.0:2278 2014-07-30 14:18:41.799399 2014-08-13 18:06:06.499994 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 73e533d5/benchmark_data_storage0111-ib.pocstor_51285_object3033/head//22 by client.275175.0:3034 2014-07-30 14:18:44.332016 2014-08-13 18:06:06.500006 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 95f50bd5/benchmark_data_storage0111-ib.pocstor_51285_object5286/head//22 by client.275175.0:5287 2014-07-30 14:18:51.196822 2014-08-13 18:06:06.500018 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 11de9bd5/benchmark_data_storage0111-ib.pocstor_51285_object7534/head//22 by client.275175.0:7535 2014-07-30 14:18:58.227507 2014-08-13 18:06:06.500030 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 43483bd5/benchmark_data_storage0111-ib.pocstor_51285_object8358/head//22 by client.275175.0:8359 2014-07-30 14:19:00.524356 2014-08-13 18:06:06.500043 7f5c973a57a0 20 read_log 191727'381 (0'0) modify 7abc73d5/benchmark_data_storage0111-ib.pocstor_51285_object10650/head//22 by client.275175.0:10651 2014-07-30 14:19:07.067114 2014-08-13 18:06:06.500055 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 43e9cbd5/benchmark_data_storage0111-ib.pocstor_51285_object11563/head//22 by client.275175.0:11564 2014-07-30 14:19:09.943863 2014-08-13 18:06:06.500067 7f5c973a57a0 20 read_log 191727'396 (0'0) modify ac0d8bd5/benchmark_data_storage0111-ib.pocstor_51285_object19412/head//22 by client.275175.0:19413 2014-07-30 14:19:34.536872 2014-08-13 18:06:06.500079 7f5c973a57a0 20 read_log 191727'404 (0'0) modify 7e0c53d5/benchmark_data_storage0111-ib.pocstor_51285_object25639/head//22 by client.275175.0:25640 2014-07-30 14:19:53.645674 2014-08-13 18:06:06.500091 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 6a8a7bd5/benchmark_data_storage0111-ib.pocstor_51285_object28386/head//22 by client.275175.0:28387 2014-07-30 14:20:02.102272 2014-08-13 18:06:06.500103 7f5c973a57a0 20 read_log 191727'410 (0'0) modify d24dfbd5/benchmark_data_storage0111-ib.pocstor_51285_object29132/head//22 by client.275175.0:29133 2014-07-30 14:20:04.308036 2014-08-13 18:06:06.500115 7f5c973a57a0 20 read_log 191727'412 (0'0) modify a92cbd5/benchmark_data_storage0111-ib.pocstor_51285_object29675/head//22 by client.275175.0:29676 2014-07-30 14:20:05.981278 2014-08-13 18:06:06.500127 7f5c973a57a0 20 read_log 191727'419 (0'0) modify bf621bd5/benchmark_data_storage0111-ib.pocstor_51285_object32066/head//22 by client.275175.0:32067 2014-07-30 14:20:13.038102 2014-08-13 18:06:06.500139 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 723913d5/benchmark_data_storage0111-ib.pocstor_51285_object35967/head//22 by client.275175.0:35968 2014-07-30 14:20:25.165793 2014-08-13 18:06:06.500162 7f5c973a57a0 20 read_log 191727'423 (0'0) modify 8cbf9bd5/benchmark_data_storage0111-ib.pocstor_51285_object36398/head//22 by client.275175.0:36399 2014-07-30 14:20:26.468067 2014-08-13 18:06:06.500175 7f5c973a57a0 20 read_log 191727'424 (0'0) modify b1d983d5/benchmark_data_storage0111-ib.pocstor_51285_object36492/head//22 by client.275175.0:36493 2014-07-30 14:20:26.797460 2014-08-13 18:06:06.500187 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 40865bd5/benchmark_data_storage0111-ib.pocstor_35625_object2794/head//22 by client.940330.0:2795 2014-08-11 16:29:51.639378 2014-08-13 18:06:06.500199 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 265433d5/benchmark_data_storage0111-ib.pocstor_35625_object9571/head//22 by client.940330.0:9572 2014-08-11 16:30:16.328912 2014-08-13 18:06:06.500211 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 293be3d5/benchmark_data_storage0111-ib.pocstor_35625_object11267/head//22 by client.940330.0:11268 2014-08-11 16:30:22.862199 2014-08-13 18:06:06.500224 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 580673d5/benchmark_data_storage0111-ib.pocstor_35625_object11489/head//22 by client.940330.0:11490 2014-08-11 16:30:23.750993 2014-08-13 18:06:06.500243 7f5c973a57a0 20 read_log 192888'431 (0'0) modify ab9a83d5/benchmark_data_storage0111-ib.pocstor_35625_object15695/head//22 by client.940330.0:15696 2014-08-11 16:30:40.073999 2014-08-13 18:06:06.500264 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 1f7d33d5/benchmark_data_storage0111-ib.pocstor_35625_object16402/head//22 by client.940330.0:16403 2014-08-11 16:30:42.733090 2014-08-13 18:06:06.500276 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 385ec3d5/benchmark_data_storage0111-ib.pocstor_35625_object16568/head//22 by client.940330.0:16569 2014-08-11 16:30:43.349925 2014-08-13 18:06:06.500288 7f5c973a57a0 20 read_log 192888'434 (0'0) modify d0f1fbd5/benchmark_data_storage0111-ib.pocstor_35625_object18645/head//22 by client.940330.0:18646 2014-08-11 16:30:51.305392 2014-08-13 18:06:06.500300 7f5c973a57a0 20 read_log 192888'435 (0'0) modify b0311bd5/benchmark_data_storage0111-ib.pocstor_35625_object25800/head//22 by client.940330.0:25801 2014-08-11 16:31:19.008561 2014-08-13 18:06:06.500312 7f5c973a57a0 20 read_log 192888'436 (0'0) modify c13143d5/benchmark_data_storage0111-ib.pocstor_35625_object27975/head//22 by client.940330.0:27976 2014-08-11 16:31:27.638150 2014-08-13 18:06:06.500324 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 8e8163d5/benchmark_data_storage0111-ib.pocstor_35625_object30712/head//22 by client.940330.0:30713 2014-08-11 16:31:38.141350 2014-08-13 18:06:06.500336 7f5c973a57a0 20 read_log 192888'438 (192888'427) delete 40865bd5/benchmark_data_storage0111-ib.pocstor_35625_object2794/head//22 by client.940330.0:34102 2014-08-11 16:31:42.194518 2014-08-13 18:06:06.500349 7f5c973a57a0 20 read_log 192888'439 (192888'428) delete 265433d5/benchmark_data_storage0111-ib.pocstor_35625_object9571/head//22 by client.940330.0:40879 2014-08-11 16:31:43.996919 2014-08-13 18:06:06.500362 7f5c973a57a0 20 read_log 192888'440 (192888'429) delete 293be3d5/benchmark_data_storage0111-ib.pocstor_35625_object11267/head//22 by client.940330.0:42575 2014-08-11 16:31:44.476987 2014-08-13 18:06:06.500375 7f5c973a57a0 20 read_log 192888'441 (192888'430) delete 580673d5/benchmark_data_storage0111-ib.pocstor_35625_object11489/head//22 by client.940330.0:42797 2014-08-11 16:31:44.537743 2014-08-13 18:06:06.500410 7f5c973a57a0 20 read_log 192888'442 (192888'431) delete ab9a83d5/benchmark_data_storage0111-ib.pocstor_35625_object15695/head//22 by client.940330.0:47003 2014-08-11 16:31:45.642692 2014-08-13 18:06:06.500426 7f5c973a57a0 20 read_log 192888'443 (192888'432) delete 1f7d33d5/benchmark_data_storage0111-ib.pocstor_35625_object16402/head//22 by client.940330.0:47710 2014-08-11 16:31:45.829864 2014-08-13 18:06:06.500452 7f5c973a57a0 20 read_log 192888'444 (192888'433) delete 385ec3d5/benchmark_data_storage0111-ib.pocstor_35625_object16568/head//22 by client.940330.0:47876 2014-08-11 16:31:45.873092 2014-08-13 18:06:06.500465 7f5c973a57a0 20 read_log 192888'445 (192888'434) delete d0f1fbd5/benchmark_data_storage0111-ib.pocstor_35625_object18645/head//22 by client.940330.0:49953 2014-08-11 16:31:46.485201 2014-08-13 18:06:06.500486 7f5c973a57a0 20 read_log 192888'446 (192888'435) delete b0311bd5/benchmark_data_storage0111-ib.pocstor_35625_object25800/head//22 by client.940330.0:57108 2014-08-11 16:31:48.588643 2014-08-13 18:06:06.500502 7f5c973a57a0 20 read_log 192888'447 (192888'436) delete c13143d5/benchmark_data_storage0111-ib.pocstor_35625_object27975/head//22 by client.940330.0:59283 2014-08-11 16:31:49.198758 2014-08-13 18:06:06.500515 7f5c973a57a0 20 read_log 192888'448 (192888'437) delete 8e8163d5/benchmark_data_storage0111-ib.pocstor_35625_object30712/head//22 by client.940330.0:62020 2014-08-11 16:31:49.964239 2014-08-13 18:06:06.500534 7f5c973a57a0 20 read_log 194215'449 (0'0) modify 8ae673d5/benchmark_data_storage0111-ib.pocstor_64498_object3640/head//22 by client.1049635.0:3641 2014-08-12 17:17:28.918286 2014-08-13 18:06:06.500560 7f5c973a57a0 20 read_log 194215'450 (0'0) modify df23abd5/benchmark_data_storage0111-ib.pocstor_64498_object12875/head//22 by client.1049635.0:12876 2014-08-12 17:17:57.007666 2014-08-13 18:06:06.500575 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 56eb3d5/benchmark_data_storage0111-ib.pocstor_64498_object15254/head//22 by client.1049635.0:15255 2014-08-12 17:18:04.817459 2014-08-13 18:06:06.500588 7f5c973a57a0 20 read_log 194215'452 (0'0) modify 6ba9dbd5/benchmark_data_storage0111-ib.pocstor_64498_object19389/head//22 by client.1049635.0:19390 2014-08-12 17:18:20.495989 2014-08-13 18:06:06.500617 7f5c973a57a0 20 read_log 194215'453 (0'0) modify e7db63d5/benchmark_data_storage0111-ib.pocstor_64498_object19764/head//22 by client.1049635.0:19765 2014-08-12 17:18:21.894548 2014-08-13 18:06:06.500632 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 572cfbd5/benchmark_data_storage0111-ib.pocstor_64498_object20280/head//22 by client.1049635.0:20281 2014-08-12 17:18:23.838974 2014-08-13 18:06:06.500645 7f5c973a57a0 20 read_log 194215'455 (0'0) modify e72ddbd5/benchmark_data_storage0111-ib.pocstor_64498_object22442/head//22 by client.1049635.0:22443 2014-08-12 17:18:32.005459 2014-08-13 18:06:06.500657 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 591063d5/benchmark_data_storage0111-ib.pocstor_64498_object30588/head//22 by client.1049635.0:30589 2014-08-12 17:19:02.831451 2014-08-13 18:06:06.500669 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 7d6343d5/benchmark_data_storage0111-ib.pocstor_64498_object32355/head//22 by client.1049635.0:32356 2014-08-12 17:19:09.938017 2014-08-13 18:06:06.500682 7f5c973a57a0 20 read_log 194215'458 (194215'449) delete 8ae673d5/benchmark_data_storage0111-ib.pocstor_64498_object3640/head//22 by client.1049635.0:37843 2014-08-12 17:19:19.162746 2014-08-13 18:06:06.500694 7f5c973a57a0 20 read_log 194215'459 (194215'450) delete df23abd5/benchmark_data_storage0111-ib.pocstor_64498_object12875/head//22 by client.1049635.0:47078 2014-08-12 17:19:22.174177 2014-08-13 18:06:06.500706 7f5c973a57a0 20 read_log 194215'460 (194215'451) delete 56eb3d5/benchmark_data_storage0111-ib.pocstor_64498_object15254/head//22 by client.1049635.0:49457 2014-08-12 17:19:22.954064 2014-08-13 18:06:06.500718 7f5c973a57a0 20 read_log 194215'461 (194215'452) delete 6ba9dbd5/benchmark_data_storage0111-ib.pocstor_64498_object19389/head//22 by client.1049635.0:53592 2014-08-12 17:19:24.313301 2014-08-13 18:06:06.500731 7f5c973a57a0 20 read_log 194215'462 (194215'453) delete e7db63d5/benchmark_data_storage0111-ib.pocstor_64498_object19764/head//22 by client.1049635.0:53967 2014-08-12 17:19:24.437265 2014-08-13 18:06:06.500743 7f5c973a57a0 20 read_log 194215'463 (194215'454) delete 572cfbd5/benchmark_data_storage0111-ib.pocstor_64498_object20280/head//22 by client.1049635.0:54483 2014-08-12 17:19:24.624134 2014-08-13 18:06:06.500755 7f5c973a57a0 20 read_log 194215'464 (194215'455) delete e72ddbd5/benchmark_data_storage0111-ib.pocstor_64498_object22442/head//22 by client.1049635.0:56645 2014-08-12 17:19:25.401091 2014-08-13 18:06:06.500767 7f5c973a57a0 20 read_log 194215'465 (194215'456) delete 591063d5/benchmark_data_storage0111-ib.pocstor_64498_object30588/head//22 by client.1049635.0:64791 2014-08-12 17:19:28.239253 2014-08-13 18:06:06.500779 7f5c973a57a0 20 read_log 194215'466 (194215'457) delete 7d6343d5/benchmark_data_storage0111-ib.pocstor_64498_object32355/head//22 by client.1049635.0:66558 2014-08-12 17:19:28.875006 2014-08-13 18:06:06.500837 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.500858 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.500872 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001305 0 0.000000 2014-08-13 18:06:06.500883 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.500891 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=0 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.500900 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'466], crt=191727'426) 2014-08-13 18:06:06.500911 7f5c973a57a0 10 osd.4 194487 pgid 22.3d9 coll 22.3d9_head 2014-08-13 18:06:06.501053 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3d9 2014-08-13 18:06:06.501062 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.501100 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d9(unlocked)] enter Initial 2014-08-13 18:06:06.501112 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3d9(unlocked)] enter NotTrimming 2014-08-13 18:06:06.501329 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.501502 7f5c973a57a0 20 read_log 191648'8 (0'0) modify 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 by client.273158.0:822 2014-07-30 14:08:51.828841 2014-08-13 18:06:06.501524 7f5c973a57a0 20 read_log 191648'48 (0'0) modify 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 by client.273158.0:2771 2014-07-30 14:09:00.243066 2014-08-13 18:06:06.501537 7f5c973a57a0 20 read_log 191648'58 (0'0) modify bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 by client.273158.0:3174 2014-07-30 14:09:02.147864 2014-08-13 18:06:06.501550 7f5c973a57a0 20 read_log 191648'126 (0'0) modify a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 by client.273158.0:7307 2014-07-30 14:09:22.300377 2014-08-13 18:06:06.501562 7f5c973a57a0 20 read_log 191648'153 (0'0) modify 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 by client.273158.0:8984 2014-07-30 14:09:31.110735 2014-08-13 18:06:06.501574 7f5c973a57a0 20 read_log 191648'189 (0'0) modify 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 by client.273158.0:11165 2014-07-30 14:09:43.006434 2014-08-13 18:06:06.501590 7f5c973a57a0 20 read_log 191648'198 (0'0) modify a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 by client.273158.0:11654 2014-07-30 14:09:45.413047 2014-08-13 18:06:06.501602 7f5c973a57a0 20 read_log 191648'258 (0'0) modify 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 by client.273158.0:15056 2014-07-30 14:10:02.887398 2014-08-13 18:06:06.501615 7f5c973a57a0 20 read_log 191648'304 (0'0) modify 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 by client.273158.0:19107 2014-07-30 14:10:24.037122 2014-08-13 18:06:06.501627 7f5c973a57a0 20 read_log 191648'320 (0'0) modify e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 by client.273158.0:19927 2014-07-30 14:10:31.025716 2014-08-13 18:06:06.501639 7f5c973a57a0 20 read_log 191648'337 (0'0) modify b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 by client.273158.0:21174 2014-07-30 14:10:37.984880 2014-08-13 18:06:06.501651 7f5c973a57a0 20 read_log 191727'370 (0'0) modify 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 by client.275175.0:2604 2014-07-30 14:18:42.948238 2014-08-13 18:06:06.501664 7f5c973a57a0 20 read_log 191727'371 (0'0) modify e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 by client.275175.0:4312 2014-07-30 14:18:48.297703 2014-08-13 18:06:06.501676 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 by client.275175.0:5364 2014-07-30 14:18:51.420098 2014-08-13 18:06:06.501697 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 by client.275175.0:7043 2014-07-30 14:18:56.837569 2014-08-13 18:06:06.501713 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 by client.275175.0:12054 2014-07-30 14:19:11.790011 2014-08-13 18:06:06.501727 7f5c973a57a0 20 read_log 191727'390 (0'0) modify 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 by client.275175.0:13697 2014-07-30 14:19:16.795566 2014-08-13 18:06:06.501739 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 by client.275175.0:14177 2014-07-30 14:19:18.375565 2014-08-13 18:06:06.501751 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 by client.275175.0:17324 2014-07-30 14:19:28.057673 2014-08-13 18:06:06.501763 7f5c973a57a0 20 read_log 191727'404 (0'0) modify a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 by client.275175.0:21166 2014-07-30 14:19:39.599975 2014-08-13 18:06:06.501775 7f5c973a57a0 20 read_log 191727'405 (0'0) modify 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 by client.275175.0:21243 2014-07-30 14:19:39.841571 2014-08-13 18:06:06.501787 7f5c973a57a0 20 read_log 191727'411 (0'0) modify a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 by client.275175.0:24210 2014-07-30 14:19:49.416172 2014-08-13 18:06:06.501799 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 by client.275175.0:24640 2014-07-30 14:19:50.627549 2014-08-13 18:06:06.501811 7f5c973a57a0 20 read_log 191727'415 (0'0) modify b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 by client.275175.0:25870 2014-07-30 14:19:54.362210 2014-08-13 18:06:06.501823 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 by client.275175.0:29894 2014-07-30 14:20:06.646699 2014-08-13 18:06:06.501835 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 by client.275175.0:30423 2014-07-30 14:20:08.361387 2014-08-13 18:06:06.501847 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 by client.275175.0:30533 2014-07-30 14:20:08.668276 2014-08-13 18:06:06.501859 7f5c973a57a0 20 read_log 191727'441 (0'0) modify d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 by client.275175.0:38262 2014-07-30 14:20:31.988938 2014-08-13 18:06:06.501871 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 836a9bd9/benchmark_data_storage0111-ib.pocstor_35625_object1500/head//22 by client.940330.0:1501 2014-08-11 16:29:46.702668 2014-08-13 18:06:06.501883 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 877e03d9/benchmark_data_storage0111-ib.pocstor_35625_object4171/head//22 by client.940330.0:4172 2014-08-11 16:29:56.953485 2014-08-13 18:06:06.501895 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 2adb73d9/benchmark_data_storage0111-ib.pocstor_35625_object4984/head//22 by client.940330.0:4985 2014-08-11 16:29:59.977474 2014-08-13 18:06:06.501907 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 490d0bd9/benchmark_data_storage0111-ib.pocstor_35625_object7078/head//22 by client.940330.0:7079 2014-08-11 16:30:07.594504 2014-08-13 18:06:06.501919 7f5c973a57a0 20 read_log 192888'446 (0'0) modify d99333d9/benchmark_data_storage0111-ib.pocstor_35625_object7200/head//22 by client.940330.0:7201 2014-08-11 16:30:08.025142 2014-08-13 18:06:06.501958 7f5c973a57a0 20 read_log 192888'447 (0'0) modify aa7a2bd9/benchmark_data_storage0111-ib.pocstor_35625_object7219/head//22 by client.940330.0:7220 2014-08-11 16:30:08.092820 2014-08-13 18:06:06.501974 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 1a7dcbd9/benchmark_data_storage0111-ib.pocstor_35625_object9000/head//22 by client.940330.0:9001 2014-08-11 16:30:14.334130 2014-08-13 18:06:06.501986 7f5c973a57a0 20 read_log 192888'449 (0'0) modify 5f4fe3d9/benchmark_data_storage0111-ib.pocstor_35625_object11838/head//22 by client.940330.0:11839 2014-08-11 16:30:25.136512 2014-08-13 18:06:06.502017 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 179893d9/benchmark_data_storage0111-ib.pocstor_35625_object14597/head//22 by client.940330.0:14598 2014-08-11 16:30:35.993758 2014-08-13 18:06:06.502029 7f5c973a57a0 20 read_log 192888'451 (0'0) modify b00db3d9/benchmark_data_storage0111-ib.pocstor_35625_object14757/head//22 by client.940330.0:14758 2014-08-11 16:30:36.586978 2014-08-13 18:06:06.502044 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 81912bd9/benchmark_data_storage0111-ib.pocstor_35625_object16479/head//22 by client.940330.0:16480 2014-08-11 16:30:43.019375 2014-08-13 18:06:06.502056 7f5c973a57a0 20 read_log 192888'453 (0'0) modify aa4553d9/benchmark_data_storage0111-ib.pocstor_35625_object16599/head//22 by client.940330.0:16600 2014-08-11 16:30:43.467916 2014-08-13 18:06:06.502070 7f5c973a57a0 20 read_log 192888'454 (0'0) modify d753e3d9/benchmark_data_storage0111-ib.pocstor_35625_object18936/head//22 by client.940330.0:18937 2014-08-11 16:30:52.434070 2014-08-13 18:06:06.502082 7f5c973a57a0 20 read_log 192888'455 (0'0) modify 8e1a3bd9/benchmark_data_storage0111-ib.pocstor_35625_object19639/head//22 by client.940330.0:19640 2014-08-11 16:30:55.290566 2014-08-13 18:06:06.502095 7f5c973a57a0 20 read_log 192888'456 (0'0) modify f93ebbd9/benchmark_data_storage0111-ib.pocstor_35625_object20302/head//22 by client.940330.0:20303 2014-08-11 16:30:57.963829 2014-08-13 18:06:06.502109 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 301b4bd9/benchmark_data_storage0111-ib.pocstor_35625_object20450/head//22 by client.940330.0:20451 2014-08-11 16:30:58.511994 2014-08-13 18:06:06.502122 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 532d13d9/benchmark_data_storage0111-ib.pocstor_35625_object20472/head//22 by client.940330.0:20473 2014-08-11 16:30:58.593007 2014-08-13 18:06:06.502135 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 10a87bd9/benchmark_data_storage0111-ib.pocstor_35625_object24843/head//22 by client.940330.0:24844 2014-08-11 16:31:15.260307 2014-08-13 18:06:06.502156 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 336c5bd9/benchmark_data_storage0111-ib.pocstor_35625_object27982/head//22 by client.940330.0:27983 2014-08-11 16:31:27.664109 2014-08-13 18:06:06.502172 7f5c973a57a0 20 read_log 192888'461 (0'0) modify e94bcbd9/benchmark_data_storage0111-ib.pocstor_35625_object28206/head//22 by client.940330.0:28207 2014-08-11 16:31:28.530840 2014-08-13 18:06:06.502184 7f5c973a57a0 20 read_log 192888'462 (192888'442) delete 836a9bd9/benchmark_data_storage0111-ib.pocstor_35625_object1500/head//22 by client.940330.0:32808 2014-08-11 16:31:41.867513 2014-08-13 18:06:06.502196 7f5c973a57a0 20 read_log 192888'463 (192888'443) delete 877e03d9/benchmark_data_storage0111-ib.pocstor_35625_object4171/head//22 by client.940330.0:35479 2014-08-11 16:31:42.584379 2014-08-13 18:06:06.502209 7f5c973a57a0 20 read_log 192888'464 (192888'444) delete 2adb73d9/benchmark_data_storage0111-ib.pocstor_35625_object4984/head//22 by client.940330.0:36292 2014-08-11 16:31:42.813027 2014-08-13 18:06:06.502221 7f5c973a57a0 20 read_log 192888'465 (192888'445) delete 490d0bd9/benchmark_data_storage0111-ib.pocstor_35625_object7078/head//22 by client.940330.0:38386 2014-08-11 16:31:43.353768 2014-08-13 18:06:06.502235 7f5c973a57a0 20 read_log 192888'466 (192888'446) delete d99333d9/benchmark_data_storage0111-ib.pocstor_35625_object7200/head//22 by client.940330.0:38508 2014-08-11 16:31:43.385351 2014-08-13 18:06:06.502256 7f5c973a57a0 20 read_log 192888'467 (192888'447) delete aa7a2bd9/benchmark_data_storage0111-ib.pocstor_35625_object7219/head//22 by client.940330.0:38527 2014-08-11 16:31:43.390890 2014-08-13 18:06:06.502274 7f5c973a57a0 20 read_log 192888'468 (192888'448) delete 1a7dcbd9/benchmark_data_storage0111-ib.pocstor_35625_object9000/head//22 by client.940330.0:40308 2014-08-11 16:31:43.841388 2014-08-13 18:06:06.502287 7f5c973a57a0 20 read_log 192888'469 (192888'449) delete 5f4fe3d9/benchmark_data_storage0111-ib.pocstor_35625_object11838/head//22 by client.940330.0:43146 2014-08-11 16:31:44.617396 2014-08-13 18:06:06.502299 7f5c973a57a0 20 read_log 192888'470 (192888'450) delete 179893d9/benchmark_data_storage0111-ib.pocstor_35625_object14597/head//22 by client.940330.0:45905 2014-08-11 16:31:45.345671 2014-08-13 18:06:06.502311 7f5c973a57a0 20 read_log 192888'471 (192888'451) delete b00db3d9/benchmark_data_storage0111-ib.pocstor_35625_object14757/head//22 by client.940330.0:46065 2014-08-11 16:31:45.387938 2014-08-13 18:06:06.502329 7f5c973a57a0 20 read_log 192888'472 (192888'452) delete 81912bd9/benchmark_data_storage0111-ib.pocstor_35625_object16479/head//22 by client.940330.0:47787 2014-08-11 16:31:45.848768 2014-08-13 18:06:06.502341 7f5c973a57a0 20 read_log 192888'473 (192888'453) delete aa4553d9/benchmark_data_storage0111-ib.pocstor_35625_object16599/head//22 by client.940330.0:47907 2014-08-11 16:31:45.880783 2014-08-13 18:06:06.502354 7f5c973a57a0 20 read_log 192888'474 (192888'454) delete d753e3d9/benchmark_data_storage0111-ib.pocstor_35625_object18936/head//22 by client.940330.0:50244 2014-08-11 16:31:46.564270 2014-08-13 18:06:06.502366 7f5c973a57a0 20 read_log 192888'475 (192888'455) delete 8e1a3bd9/benchmark_data_storage0111-ib.pocstor_35625_object19639/head//22 by client.940330.0:50947 2014-08-11 16:31:46.759804 2014-08-13 18:06:06.502378 7f5c973a57a0 20 read_log 192888'476 (192888'456) delete f93ebbd9/benchmark_data_storage0111-ib.pocstor_35625_object20302/head//22 by client.940330.0:51610 2014-08-11 16:31:46.968713 2014-08-13 18:06:06.502400 7f5c973a57a0 20 read_log 192888'477 (192888'457) delete 301b4bd9/benchmark_data_storage0111-ib.pocstor_35625_object20450/head//22 by client.940330.0:51758 2014-08-11 16:31:47.009068 2014-08-13 18:06:06.502416 7f5c973a57a0 20 read_log 192888'478 (192888'458) delete 532d13d9/benchmark_data_storage0111-ib.pocstor_35625_object20472/head//22 by client.940330.0:51780 2014-08-11 16:31:47.014473 2014-08-13 18:06:06.502429 7f5c973a57a0 20 read_log 192888'479 (192888'459) delete 10a87bd9/benchmark_data_storage0111-ib.pocstor_35625_object24843/head//22 by client.940330.0:56151 2014-08-11 16:31:48.288935 2014-08-13 18:06:06.502447 7f5c973a57a0 20 read_log 192888'480 (192888'460) delete 336c5bd9/benchmark_data_storage0111-ib.pocstor_35625_object27982/head//22 by client.940330.0:59290 2014-08-11 16:31:49.199621 2014-08-13 18:06:06.502463 7f5c973a57a0 20 read_log 192888'481 (192888'461) delete e94bcbd9/benchmark_data_storage0111-ib.pocstor_35625_object28206/head//22 by client.940330.0:59514 2014-08-11 16:31:49.261201 2014-08-13 18:06:06.502476 7f5c973a57a0 20 read_log 194215'482 (0'0) modify b091e3d9/benchmark_data_storage0111-ib.pocstor_64498_object1407/head//22 by client.1049635.0:1408 2014-08-12 17:17:21.717610 2014-08-13 18:06:06.502505 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 660853d9/benchmark_data_storage0111-ib.pocstor_64498_object3877/head//22 by client.1049635.0:3878 2014-08-12 17:17:29.676304 2014-08-13 18:06:06.502518 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 5136d3d9/benchmark_data_storage0111-ib.pocstor_64498_object4412/head//22 by client.1049635.0:4413 2014-08-12 17:17:31.257814 2014-08-13 18:06:06.502531 7f5c973a57a0 20 read_log 194215'485 (0'0) modify 100ee3d9/benchmark_data_storage0111-ib.pocstor_64498_object6374/head//22 by client.1049635.0:6375 2014-08-12 17:17:37.201333 2014-08-13 18:06:06.502560 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 74b4cbd9/benchmark_data_storage0111-ib.pocstor_64498_object9876/head//22 by client.1049635.0:9877 2014-08-12 17:17:47.572381 2014-08-13 18:06:06.502577 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 34bc63d9/benchmark_data_storage0111-ib.pocstor_64498_object11189/head//22 by client.1049635.0:11190 2014-08-12 17:17:51.798215 2014-08-13 18:06:06.502589 7f5c973a57a0 20 read_log 194215'488 (0'0) modify a7758bd9/benchmark_data_storage0111-ib.pocstor_64498_object15202/head//22 by client.1049635.0:15203 2014-08-12 17:18:04.645181 2014-08-13 18:06:06.502602 7f5c973a57a0 20 read_log 194215'489 (0'0) modify ebc083d9/benchmark_data_storage0111-ib.pocstor_64498_object15324/head//22 by client.1049635.0:15325 2014-08-12 17:18:05.053228 2014-08-13 18:06:06.502614 7f5c973a57a0 20 read_log 194215'490 (0'0) modify ecd3abd9/benchmark_data_storage0111-ib.pocstor_64498_object15674/head//22 by client.1049635.0:15675 2014-08-12 17:18:06.353886 2014-08-13 18:06:06.502627 7f5c973a57a0 20 read_log 194215'491 (0'0) modify fdafebd9/benchmark_data_storage0111-ib.pocstor_64498_object16076/head//22 by client.1049635.0:16077 2014-08-12 17:18:07.838477 2014-08-13 18:06:06.502660 7f5c973a57a0 20 read_log 194215'492 (0'0) modify cfb0cbd9/benchmark_data_storage0111-ib.pocstor_64498_object20689/head//22 by client.1049635.0:20690 2014-08-12 17:18:25.370515 2014-08-13 18:06:06.502675 7f5c973a57a0 20 read_log 194215'493 (0'0) modify f9f693d9/benchmark_data_storage0111-ib.pocstor_64498_object20727/head//22 by client.1049635.0:20728 2014-08-12 17:18:25.508224 2014-08-13 18:06:06.502688 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 44d9b3d9/benchmark_data_storage0111-ib.pocstor_64498_object23494/head//22 by client.1049635.0:23495 2014-08-12 17:18:35.969397 2014-08-13 18:06:06.502700 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 97346bd9/benchmark_data_storage0111-ib.pocstor_64498_object29753/head//22 by client.1049635.0:29754 2014-08-12 17:18:59.523995 2014-08-13 18:06:06.502712 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:30300 2014-08-12 17:19:01.749852 2014-08-13 18:06:06.502724 7f5c973a57a0 20 read_log 194215'497 (194215'482) delete b091e3d9/benchmark_data_storage0111-ib.pocstor_64498_object1407/head//22 by client.1049635.0:35610 2014-08-12 17:19:18.454089 2014-08-13 18:06:06.502737 7f5c973a57a0 20 read_log 194215'498 (194215'483) delete 660853d9/benchmark_data_storage0111-ib.pocstor_64498_object3877/head//22 by client.1049635.0:38080 2014-08-12 17:19:19.237653 2014-08-13 18:06:06.502749 7f5c973a57a0 20 read_log 194215'499 (194215'484) delete 5136d3d9/benchmark_data_storage0111-ib.pocstor_64498_object4412/head//22 by client.1049635.0:38615 2014-08-12 17:19:19.408602 2014-08-13 18:06:06.502761 7f5c973a57a0 20 read_log 194215'500 (194215'485) delete 100ee3d9/benchmark_data_storage0111-ib.pocstor_64498_object6374/head//22 by client.1049635.0:40577 2014-08-12 17:19:20.077232 2014-08-13 18:06:06.502773 7f5c973a57a0 20 read_log 194215'501 (194215'486) delete 74b4cbd9/benchmark_data_storage0111-ib.pocstor_64498_object9876/head//22 by client.1049635.0:44079 2014-08-12 17:19:21.216186 2014-08-13 18:06:06.502785 7f5c973a57a0 20 read_log 194215'502 (194215'487) delete 34bc63d9/benchmark_data_storage0111-ib.pocstor_64498_object11189/head//22 by client.1049635.0:45392 2014-08-12 17:19:21.637715 2014-08-13 18:06:06.502799 7f5c973a57a0 20 read_log 194215'503 (194215'488) delete a7758bd9/benchmark_data_storage0111-ib.pocstor_64498_object15202/head//22 by client.1049635.0:49405 2014-08-12 17:19:22.937262 2014-08-13 18:06:06.502812 7f5c973a57a0 20 read_log 194215'504 (194215'489) delete ebc083d9/benchmark_data_storage0111-ib.pocstor_64498_object15324/head//22 by client.1049635.0:49527 2014-08-12 17:19:22.974200 2014-08-13 18:06:06.502824 7f5c973a57a0 20 read_log 194215'505 (194215'490) delete ecd3abd9/benchmark_data_storage0111-ib.pocstor_64498_object15674/head//22 by client.1049635.0:49877 2014-08-12 17:19:23.086653 2014-08-13 18:06:06.502843 7f5c973a57a0 20 read_log 194215'506 (194215'491) delete fdafebd9/benchmark_data_storage0111-ib.pocstor_64498_object16076/head//22 by client.1049635.0:50279 2014-08-12 17:19:23.212421 2014-08-13 18:06:06.502858 7f5c973a57a0 20 read_log 194215'507 (194215'492) delete cfb0cbd9/benchmark_data_storage0111-ib.pocstor_64498_object20689/head//22 by client.1049635.0:54892 2014-08-12 17:19:24.770872 2014-08-13 18:06:06.502871 7f5c973a57a0 20 read_log 194215'508 (194215'493) delete f9f693d9/benchmark_data_storage0111-ib.pocstor_64498_object20727/head//22 by client.1049635.0:54930 2014-08-12 17:19:24.784831 2014-08-13 18:06:06.502883 7f5c973a57a0 20 read_log 194215'509 (194215'494) delete 44d9b3d9/benchmark_data_storage0111-ib.pocstor_64498_object23494/head//22 by client.1049635.0:57697 2014-08-12 17:19:25.796840 2014-08-13 18:06:06.502895 7f5c973a57a0 20 read_log 194215'510 (194215'495) delete 97346bd9/benchmark_data_storage0111-ib.pocstor_64498_object29753/head//22 by client.1049635.0:63956 2014-08-12 17:19:27.970049 2014-08-13 18:06:06.502908 7f5c973a57a0 20 read_log 194215'511 (194215'496) delete 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:64502 2014-08-12 17:19:28.143258 2014-08-13 18:06:06.502998 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.503016 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=0 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.503032 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=0 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001931 0 0.000000 2014-08-13 18:06:06.503043 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=0 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.503052 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=0 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.503061 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'511], crt=191727'441) 2014-08-13 18:06:06.503072 7f5c973a57a0 10 osd.4 194487 pgid 22.3e4 coll 22.3e4_head 2014-08-13 18:06:06.503226 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.3e4 2014-08-13 18:06:06.503236 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.503261 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3e4(unlocked)] enter Initial 2014-08-13 18:06:06.503276 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3e4(unlocked)] enter NotTrimming 2014-08-13 18:06:06.503424 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.503567 7f5c973a57a0 20 read_log 191648'13 (0'0) modify ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 by client.273158.0:660 2014-07-30 14:08:51.236138 2014-08-13 18:06:06.503588 7f5c973a57a0 20 read_log 191648'20 (0'0) modify c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 by client.273158.0:1374 2014-07-30 14:08:53.839019 2014-08-13 18:06:06.503602 7f5c973a57a0 20 read_log 191648'118 (0'0) modify 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 by client.273158.0:7408 2014-07-30 14:09:22.807776 2014-08-13 18:06:06.503614 7f5c973a57a0 20 read_log 191648'158 (0'0) modify e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 by client.273158.0:9202 2014-07-30 14:09:31.967465 2014-08-13 18:06:06.503626 7f5c973a57a0 20 read_log 191648'196 (0'0) modify 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 by client.273158.0:11828 2014-07-30 14:09:46.186522 2014-08-13 18:06:06.503638 7f5c973a57a0 20 read_log 191648'215 (0'0) modify 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 by client.273158.0:13103 2014-07-30 14:09:52.930745 2014-08-13 18:06:06.503651 7f5c973a57a0 20 read_log 191648'261 (0'0) modify f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 by client.273158.0:16500 2014-07-30 14:10:10.181943 2014-08-13 18:06:06.503663 7f5c973a57a0 20 read_log 191648'268 (0'0) modify 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 by client.273158.0:16897 2014-07-30 14:10:12.167252 2014-08-13 18:06:06.503690 7f5c973a57a0 20 read_log 191648'269 (0'0) modify e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 by client.273158.0:16910 2014-07-30 14:10:12.323702 2014-08-13 18:06:06.503704 7f5c973a57a0 20 read_log 191648'295 (0'0) modify 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 by client.273158.0:18817 2014-07-30 14:10:22.815945 2014-08-13 18:06:06.503716 7f5c973a57a0 20 read_log 191727'370 (0'0) modify e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 by client.275175.0:171 2014-07-30 14:18:34.112158 2014-08-13 18:06:06.503729 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 by client.275175.0:850 2014-07-30 14:18:36.863512 2014-08-13 18:06:06.503741 7f5c973a57a0 20 read_log 191727'375 (0'0) modify a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 by client.275175.0:2335 2014-07-30 14:18:41.990442 2014-08-13 18:06:06.503753 7f5c973a57a0 20 read_log 191727'379 (0'0) modify b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 by client.275175.0:3618 2014-07-30 14:18:46.163538 2014-08-13 18:06:06.503765 7f5c973a57a0 20 read_log 191727'380 (0'0) modify e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 by client.275175.0:3871 2014-07-30 14:18:46.957245 2014-08-13 18:06:06.503778 7f5c973a57a0 20 read_log 191727'387 (0'0) modify 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 by client.275175.0:7584 2014-07-30 14:18:58.364406 2014-08-13 18:06:06.503793 7f5c973a57a0 20 read_log 191727'390 (0'0) modify 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 by client.275175.0:8185 2014-07-30 14:19:00.035853 2014-08-13 18:06:06.503808 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 by client.275175.0:8512 2014-07-30 14:19:00.959329 2014-08-13 18:06:06.503829 7f5c973a57a0 20 read_log 191727'393 (0'0) modify 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 by client.275175.0:9117 2014-07-30 14:19:02.682215 2014-08-13 18:06:06.503844 7f5c973a57a0 20 read_log 191727'399 (0'0) modify c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 by client.275175.0:12240 2014-07-30 14:19:12.310416 2014-08-13 18:06:06.503856 7f5c973a57a0 20 read_log 191727'401 (0'0) modify fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 by client.275175.0:13471 2014-07-30 14:19:16.161477 2014-08-13 18:06:06.503873 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 by client.275175.0:13496 2014-07-30 14:19:16.233859 2014-08-13 18:06:06.503886 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 by client.275175.0:14901 2014-07-30 14:19:20.424930 2014-08-13 18:06:06.503906 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 by client.275175.0:18558 2014-07-30 14:19:31.647507 2014-08-13 18:06:06.503922 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 by client.275175.0:21790 2014-07-30 14:19:41.808153 2014-08-13 18:06:06.503935 7f5c973a57a0 20 read_log 191727'419 (0'0) modify e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 by client.275175.0:23147 2014-07-30 14:19:46.435420 2014-08-13 18:06:06.503956 7f5c973a57a0 20 read_log 191727'420 (0'0) modify fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 by client.275175.0:24195 2014-07-30 14:19:49.374369 2014-08-13 18:06:06.503971 7f5c973a57a0 20 read_log 191727'427 (0'0) modify b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 by client.275175.0:31110 2014-07-30 14:20:10.294807 2014-08-13 18:06:06.503984 7f5c973a57a0 20 read_log 191727'437 (0'0) modify 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 by client.275175.0:37692 2014-07-30 14:20:30.380147 2014-08-13 18:06:06.503996 7f5c973a57a0 20 read_log 191727'439 (0'0) modify 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 by client.275175.0:38662 2014-07-30 14:20:33.130508 2014-08-13 18:06:06.504008 7f5c973a57a0 20 read_log 191727'442 (0'0) modify a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 by client.275175.0:38712 2014-07-30 14:20:33.271825 2014-08-13 18:06:06.504021 7f5c973a57a0 20 read_log 192674'443 (0'0) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2663 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504034 7f5c973a57a0 20 read_log 192674'444 (192674'443) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2664 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504053 7f5c973a57a0 20 read_log 192674'445 (192674'444) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2665 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504076 7f5c973a57a0 20 read_log 192674'446 (192674'445) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2666 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504092 7f5c973a57a0 20 read_log 192674'447 (192674'446) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2667 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504104 7f5c973a57a0 20 read_log 192674'448 (192674'447) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2668 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504123 7f5c973a57a0 20 read_log 192674'449 (192674'448) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2669 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504179 7f5c973a57a0 20 read_log 192674'450 (192674'449) modify fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 by client.279772.1:2670 2014-07-31 18:03:44.524000 2014-08-13 18:06:06.504197 7f5c973a57a0 20 read_log 192888'451 (0'0) modify ab1eabe4/benchmark_data_storage0111-ib.pocstor_35625_object252/head//22 by client.940330.0:253 2014-08-11 16:29:41.724807 2014-08-13 18:06:06.504245 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 251db3e4/benchmark_data_storage0111-ib.pocstor_35625_object1039/head//22 by client.940330.0:1040 2014-08-11 16:29:44.939051 2014-08-13 18:06:06.504258 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 18092be4/benchmark_data_storage0111-ib.pocstor_35625_object2883/head//22 by client.940330.0:2884 2014-08-11 16:29:52.154787 2014-08-13 18:06:06.504274 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 294b1be4/benchmark_data_storage0111-ib.pocstor_35625_object3206/head//22 by client.940330.0:3207 2014-08-11 16:29:53.356262 2014-08-13 18:06:06.504287 7f5c973a57a0 20 read_log 192888'455 (0'0) modify a86353e4/benchmark_data_storage0111-ib.pocstor_35625_object4168/head//22 by client.940330.0:4169 2014-08-11 16:29:56.942276 2014-08-13 18:06:06.504299 7f5c973a57a0 20 read_log 192888'456 (0'0) modify df38d3e4/benchmark_data_storage0111-ib.pocstor_35625_object6828/head//22 by client.940330.0:6829 2014-08-11 16:30:06.715874 2014-08-13 18:06:06.504311 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 3a3d9be4/benchmark_data_storage0111-ib.pocstor_35625_object8496/head//22 by client.940330.0:8497 2014-08-11 16:30:12.573964 2014-08-13 18:06:06.504331 7f5c973a57a0 20 read_log 192888'458 (0'0) modify b860b3e4/benchmark_data_storage0111-ib.pocstor_35625_object10286/head//22 by client.940330.0:10287 2014-08-11 16:30:18.908661 2014-08-13 18:06:06.504347 7f5c973a57a0 20 read_log 192888'459 (0'0) modify c7e76be4/benchmark_data_storage0111-ib.pocstor_35625_object13242/head//22 by client.940330.0:13243 2014-08-11 16:30:30.826113 2014-08-13 18:06:06.504359 7f5c973a57a0 20 read_log 192888'460 (0'0) modify b1987be4/benchmark_data_storage0111-ib.pocstor_35625_object14257/head//22 by client.940330.0:14258 2014-08-11 16:30:34.721306 2014-08-13 18:06:06.504371 7f5c973a57a0 20 read_log 192888'461 (0'0) modify cf3f43e4/benchmark_data_storage0111-ib.pocstor_35625_object14945/head//22 by client.940330.0:14946 2014-08-11 16:30:37.287712 2014-08-13 18:06:06.504386 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 6e1813e4/benchmark_data_storage0111-ib.pocstor_35625_object15096/head//22 by client.940330.0:15097 2014-08-11 16:30:37.849964 2014-08-13 18:06:06.504398 7f5c973a57a0 20 read_log 192888'463 (0'0) modify f136abe4/benchmark_data_storage0111-ib.pocstor_35625_object17411/head//22 by client.940330.0:17412 2014-08-11 16:30:46.501450 2014-08-13 18:06:06.504410 7f5c973a57a0 20 read_log 192888'464 (0'0) modify a12053e4/benchmark_data_storage0111-ib.pocstor_35625_object17529/head//22 by client.940330.0:17530 2014-08-11 16:30:46.935071 2014-08-13 18:06:06.504422 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 56f9fbe4/benchmark_data_storage0111-ib.pocstor_35625_object19448/head//22 by client.940330.0:19449 2014-08-11 16:30:54.437749 2014-08-13 18:06:06.504435 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 548183e4/benchmark_data_storage0111-ib.pocstor_35625_object21478/head//22 by client.940330.0:21479 2014-08-11 16:31:02.598556 2014-08-13 18:06:06.504447 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 5bd8be4/benchmark_data_storage0111-ib.pocstor_35625_object22736/head//22 by client.940330.0:22737 2014-08-11 16:31:07.432192 2014-08-13 18:06:06.504459 7f5c973a57a0 20 read_log 192888'468 (0'0) modify ae923e4/benchmark_data_storage0111-ib.pocstor_35625_object23845/head//22 by client.940330.0:23846 2014-08-11 16:31:11.584553 2014-08-13 18:06:06.504471 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 72aa3e4/benchmark_data_storage0111-ib.pocstor_35625_object23952/head//22 by client.940330.0:23953 2014-08-11 16:31:11.975897 2014-08-13 18:06:06.504493 7f5c973a57a0 20 read_log 192888'470 (0'0) modify 9a123be4/benchmark_data_storage0111-ib.pocstor_35625_object29395/head//22 by client.940330.0:29396 2014-08-11 16:31:33.173709 2014-08-13 18:06:06.504508 7f5c973a57a0 20 read_log 192888'471 (0'0) modify 9b4fbe4/benchmark_data_storage0111-ib.pocstor_35625_object30551/head//22 by client.940330.0:30552 2014-08-11 16:31:37.543505 2014-08-13 18:06:06.504520 7f5c973a57a0 20 read_log 192888'472 (192888'451) delete ab1eabe4/benchmark_data_storage0111-ib.pocstor_35625_object252/head//22 by client.940330.0:31560 2014-08-11 16:31:41.545995 2014-08-13 18:06:06.504533 7f5c973a57a0 20 read_log 192888'473 (192888'452) delete 251db3e4/benchmark_data_storage0111-ib.pocstor_35625_object1039/head//22 by client.940330.0:32347 2014-08-11 16:31:41.758561 2014-08-13 18:06:06.504545 7f5c973a57a0 20 read_log 192888'474 (192888'453) delete 18092be4/benchmark_data_storage0111-ib.pocstor_35625_object2883/head//22 by client.940330.0:34191 2014-08-11 16:31:42.221088 2014-08-13 18:06:06.504557 7f5c973a57a0 20 read_log 192888'475 (192888'454) delete 294b1be4/benchmark_data_storage0111-ib.pocstor_35625_object3206/head//22 by client.940330.0:34514 2014-08-11 16:31:42.310168 2014-08-13 18:06:06.504570 7f5c973a57a0 20 read_log 192888'476 (192888'455) delete a86353e4/benchmark_data_storage0111-ib.pocstor_35625_object4168/head//22 by client.940330.0:35476 2014-08-11 16:31:42.584017 2014-08-13 18:06:06.504582 7f5c973a57a0 20 read_log 192888'477 (192888'456) delete df38d3e4/benchmark_data_storage0111-ib.pocstor_35625_object6828/head//22 by client.940330.0:38136 2014-08-11 16:31:43.294875 2014-08-13 18:06:06.504594 7f5c973a57a0 20 read_log 192888'478 (192888'457) delete 3a3d9be4/benchmark_data_storage0111-ib.pocstor_35625_object8496/head//22 by client.940330.0:39804 2014-08-11 16:31:43.715291 2014-08-13 18:06:06.504606 7f5c973a57a0 20 read_log 192888'479 (192888'458) delete b860b3e4/benchmark_data_storage0111-ib.pocstor_35625_object10286/head//22 by client.940330.0:41594 2014-08-11 16:31:44.195254 2014-08-13 18:06:06.504619 7f5c973a57a0 20 read_log 192888'480 (192888'459) delete c7e76be4/benchmark_data_storage0111-ib.pocstor_35625_object13242/head//22 by client.940330.0:44550 2014-08-11 16:31:44.987190 2014-08-13 18:06:06.504631 7f5c973a57a0 20 read_log 192888'481 (192888'460) delete b1987be4/benchmark_data_storage0111-ib.pocstor_35625_object14257/head//22 by client.940330.0:45565 2014-08-11 16:31:45.249495 2014-08-13 18:06:06.504662 7f5c973a57a0 20 read_log 192888'482 (192888'461) delete cf3f43e4/benchmark_data_storage0111-ib.pocstor_35625_object14945/head//22 by client.940330.0:46253 2014-08-11 16:31:45.441438 2014-08-13 18:06:06.504677 7f5c973a57a0 20 read_log 192888'483 (192888'462) delete 6e1813e4/benchmark_data_storage0111-ib.pocstor_35625_object15096/head//22 by client.940330.0:46404 2014-08-11 16:31:45.481628 2014-08-13 18:06:06.504690 7f5c973a57a0 20 read_log 192888'484 (192888'463) delete f136abe4/benchmark_data_storage0111-ib.pocstor_35625_object17411/head//22 by client.940330.0:48719 2014-08-11 16:31:46.105085 2014-08-13 18:06:06.504704 7f5c973a57a0 20 read_log 192888'485 (192888'464) delete a12053e4/benchmark_data_storage0111-ib.pocstor_35625_object17529/head//22 by client.940330.0:48837 2014-08-11 16:31:46.135320 2014-08-13 18:06:06.504718 7f5c973a57a0 20 read_log 192888'486 (192888'465) delete 56f9fbe4/benchmark_data_storage0111-ib.pocstor_35625_object19448/head//22 by client.940330.0:50756 2014-08-11 16:31:46.704365 2014-08-13 18:06:06.504733 7f5c973a57a0 20 read_log 192888'487 (192888'466) delete 548183e4/benchmark_data_storage0111-ib.pocstor_35625_object21478/head//22 by client.940330.0:52786 2014-08-11 16:31:47.289428 2014-08-13 18:06:06.504745 7f5c973a57a0 20 read_log 192888'488 (192888'467) delete 5bd8be4/benchmark_data_storage0111-ib.pocstor_35625_object22736/head//22 by client.940330.0:54044 2014-08-11 16:31:47.679853 2014-08-13 18:06:06.504760 7f5c973a57a0 20 read_log 192888'489 (192888'468) delete ae923e4/benchmark_data_storage0111-ib.pocstor_35625_object23845/head//22 by client.940330.0:55153 2014-08-11 16:31:47.988873 2014-08-13 18:06:06.504779 7f5c973a57a0 20 read_log 192888'490 (192888'469) delete 72aa3e4/benchmark_data_storage0111-ib.pocstor_35625_object23952/head//22 by client.940330.0:55260 2014-08-11 16:31:48.022299 2014-08-13 18:06:06.504794 7f5c973a57a0 20 read_log 192888'491 (192888'470) delete 9a123be4/benchmark_data_storage0111-ib.pocstor_35625_object29395/head//22 by client.940330.0:60703 2014-08-11 16:31:49.583299 2014-08-13 18:06:06.504806 7f5c973a57a0 20 read_log 192888'492 (192888'471) delete 9b4fbe4/benchmark_data_storage0111-ib.pocstor_35625_object30551/head//22 by client.940330.0:61859 2014-08-11 16:31:49.915067 2014-08-13 18:06:06.504821 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 6ea8be4/benchmark_data_storage0111-ib.pocstor_64498_object3003/head//22 by client.1049635.0:3004 2014-08-12 17:17:26.852274 2014-08-13 18:06:06.504833 7f5c973a57a0 20 read_log 194215'494 (0'0) modify d902e3e4/benchmark_data_storage0111-ib.pocstor_64498_object3395/head//22 by client.1049635.0:3396 2014-08-12 17:17:28.190801 2014-08-13 18:06:06.504868 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 62240be4/benchmark_data_storage0111-ib.pocstor_64498_object3504/head//22 by client.1049635.0:3505 2014-08-12 17:17:28.515281 2014-08-13 18:06:06.504880 7f5c973a57a0 20 read_log 194215'496 (0'0) modify b085f3e4/benchmark_data_storage0111-ib.pocstor_64498_object5535/head//22 by client.1049635.0:5536 2014-08-12 17:17:34.716840 2014-08-13 18:06:06.504894 7f5c973a57a0 20 read_log 194215'497 (0'0) modify b89993e4/benchmark_data_storage0111-ib.pocstor_64498_object11414/head//22 by client.1049635.0:11415 2014-08-12 17:17:52.564972 2014-08-13 18:06:06.504908 7f5c973a57a0 20 read_log 194215'498 (0'0) modify d53a4be4/benchmark_data_storage0111-ib.pocstor_64498_object14828/head//22 by client.1049635.0:14829 2014-08-12 17:18:03.413077 2014-08-13 18:06:06.504923 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 55eb0be4/benchmark_data_storage0111-ib.pocstor_64498_object20658/head//22 by client.1049635.0:20659 2014-08-12 17:18:25.253171 2014-08-13 18:06:06.504935 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 8cf67be4/benchmark_data_storage0111-ib.pocstor_64498_object21331/head//22 by client.1049635.0:21332 2014-08-12 17:18:27.827817 2014-08-13 18:06:06.504950 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 37bc5be4/benchmark_data_storage0111-ib.pocstor_64498_object22829/head//22 by client.1049635.0:22830 2014-08-12 17:18:33.499888 2014-08-13 18:06:06.504962 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 1d0f93e4/benchmark_data_storage0111-ib.pocstor_64498_object26258/head//22 by client.1049635.0:26259 2014-08-12 17:18:46.382980 2014-08-13 18:06:06.504976 7f5c973a57a0 20 read_log 194215'503 (0'0) modify ade7abe4/benchmark_data_storage0111-ib.pocstor_64498_object27199/head//22 by client.1049635.0:27200 2014-08-12 17:18:49.891666 2014-08-13 18:06:06.504989 7f5c973a57a0 20 read_log 194215'504 (0'0) modify 982fdbe4/benchmark_data_storage0111-ib.pocstor_64498_object33594/head//22 by client.1049635.0:33595 2014-08-12 17:19:14.641389 2014-08-13 18:06:06.505003 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 59e2a3e4/benchmark_data_storage0111-ib.pocstor_64498_object33849/head//22 by client.1049635.0:33850 2014-08-12 17:19:15.608005 2014-08-13 18:06:06.505015 7f5c973a57a0 20 read_log 194215'506 (194215'493) delete 6ea8be4/benchmark_data_storage0111-ib.pocstor_64498_object3003/head//22 by client.1049635.0:37206 2014-08-12 17:19:18.970763 2014-08-13 18:06:06.505030 7f5c973a57a0 20 read_log 194215'507 (194215'494) delete d902e3e4/benchmark_data_storage0111-ib.pocstor_64498_object3395/head//22 by client.1049635.0:37598 2014-08-12 17:19:19.087637 2014-08-13 18:06:06.505042 7f5c973a57a0 20 read_log 194215'508 (194215'495) delete 62240be4/benchmark_data_storage0111-ib.pocstor_64498_object3504/head//22 by client.1049635.0:37707 2014-08-12 17:19:19.120739 2014-08-13 18:06:06.505057 7f5c973a57a0 20 read_log 194215'509 (194215'496) delete b085f3e4/benchmark_data_storage0111-ib.pocstor_64498_object5535/head//22 by client.1049635.0:39738 2014-08-12 17:19:19.784670 2014-08-13 18:06:06.505072 7f5c973a57a0 20 read_log 194215'510 (194215'497) delete b89993e4/benchmark_data_storage0111-ib.pocstor_64498_object11414/head//22 by client.1049635.0:45617 2014-08-12 17:19:21.708168 2014-08-13 18:06:06.505084 7f5c973a57a0 20 read_log 194215'511 (194215'498) delete d53a4be4/benchmark_data_storage0111-ib.pocstor_64498_object14828/head//22 by client.1049635.0:49031 2014-08-12 17:19:22.823027 2014-08-13 18:06:06.505098 7f5c973a57a0 20 read_log 194215'512 (194215'499) delete 55eb0be4/benchmark_data_storage0111-ib.pocstor_64498_object20658/head//22 by client.1049635.0:54861 2014-08-12 17:19:24.760654 2014-08-13 18:06:06.505111 7f5c973a57a0 20 read_log 194215'513 (194215'500) delete 8cf67be4/benchmark_data_storage0111-ib.pocstor_64498_object21331/head//22 by client.1049635.0:55534 2014-08-12 17:19:24.998445 2014-08-13 18:06:06.505123 7f5c973a57a0 20 read_log 194215'514 (194215'501) delete 37bc5be4/benchmark_data_storage0111-ib.pocstor_64498_object22829/head//22 by client.1049635.0:57032 2014-08-12 17:19:25.547867 2014-08-13 18:06:06.505135 7f5c973a57a0 20 read_log 194215'515 (194215'502) delete 1d0f93e4/benchmark_data_storage0111-ib.pocstor_64498_object26258/head//22 by client.1049635.0:60461 2014-08-12 17:19:26.743747 2014-08-13 18:06:06.505155 7f5c973a57a0 20 read_log 194215'516 (194215'503) delete ade7abe4/benchmark_data_storage0111-ib.pocstor_64498_object27199/head//22 by client.1049635.0:61402 2014-08-12 17:19:27.073145 2014-08-13 18:06:06.505170 7f5c973a57a0 20 read_log 194215'517 (194215'504) delete 982fdbe4/benchmark_data_storage0111-ib.pocstor_64498_object33594/head//22 by client.1049635.0:67797 2014-08-12 17:19:29.328082 2014-08-13 18:06:06.505182 7f5c973a57a0 20 read_log 194215'518 (194215'505) delete 59e2a3e4/benchmark_data_storage0111-ib.pocstor_64498_object33849/head//22 by client.1049635.0:68052 2014-08-12 17:19:29.423476 2014-08-13 18:06:06.505289 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.505311 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=0 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.505327 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=0 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002066 0 0.000000 2014-08-13 18:06:06.505339 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=0 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.505348 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=0 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.505360 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'518], crt=191727'442) 2014-08-13 18:06:06.505370 7f5c973a57a0 10 osd.4 194487 pgid 22.481 coll 22.481_head 2014-08-13 18:06:06.505572 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.481 2014-08-13 18:06:06.505586 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.505612 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.481(unlocked)] enter Initial 2014-08-13 18:06:06.505624 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.481(unlocked)] enter NotTrimming 2014-08-13 18:06:06.505791 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.505922 7f5c973a57a0 20 read_log 191648'32 (0'0) modify f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 by client.273158.0:2001 2014-07-30 14:08:56.306848 2014-08-13 18:06:06.505943 7f5c973a57a0 20 read_log 191648'39 (0'0) modify 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 by client.273158.0:2550 2014-07-30 14:08:59.035976 2014-08-13 18:06:06.505956 7f5c973a57a0 20 read_log 191648'128 (0'0) modify c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 by client.273158.0:7329 2014-07-30 14:09:22.405062 2014-08-13 18:06:06.505969 7f5c973a57a0 20 read_log 191648'160 (0'0) modify e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 by client.273158.0:9612 2014-07-30 14:09:34.063892 2014-08-13 18:06:06.505981 7f5c973a57a0 20 read_log 191648'168 (0'0) modify dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 by client.273158.0:9898 2014-07-30 14:09:35.299912 2014-08-13 18:06:06.505993 7f5c973a57a0 20 read_log 191648'179 (0'0) modify a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 by client.273158.0:10617 2014-07-30 14:09:39.132313 2014-08-13 18:06:06.506012 7f5c973a57a0 20 read_log 191648'214 (0'0) modify 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 by client.273158.0:12472 2014-07-30 14:09:49.900374 2014-08-13 18:06:06.506025 7f5c973a57a0 20 read_log 191648'245 (0'0) modify 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 by client.273158.0:14322 2014-07-30 14:09:59.304984 2014-08-13 18:06:06.506037 7f5c973a57a0 20 read_log 191648'355 (0'0) modify a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 by client.273158.0:20931 2014-07-30 14:10:36.683387 2014-08-13 18:06:06.506049 7f5c973a57a0 20 read_log 191648'366 (0'0) modify fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 by client.273158.0:21582 2014-07-30 14:10:39.987529 2014-08-13 18:06:06.506061 7f5c973a57a0 20 read_log 191648'385 (0'0) modify 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 by client.273158.0:22847 2014-07-30 14:10:48.513560 2014-08-13 18:06:06.506073 7f5c973a57a0 20 read_log 191727'394 (0'0) modify bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 by client.275175.0:5071 2014-07-30 14:18:50.558639 2014-08-13 18:06:06.506085 7f5c973a57a0 20 read_log 191727'398 (0'0) modify f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 by client.275175.0:9016 2014-07-30 14:19:02.392588 2014-08-13 18:06:06.506097 7f5c973a57a0 20 read_log 191727'401 (0'0) modify 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 by client.275175.0:9586 2014-07-30 14:19:04.028818 2014-08-13 18:06:06.506109 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 by client.275175.0:10570 2014-07-30 14:19:06.837076 2014-08-13 18:06:06.506121 7f5c973a57a0 20 read_log 191727'405 (0'0) modify 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 by client.275175.0:11276 2014-07-30 14:19:08.876081 2014-08-13 18:06:06.506133 7f5c973a57a0 20 read_log 191727'410 (0'0) modify 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 by client.275175.0:13113 2014-07-30 14:19:15.010034 2014-08-13 18:06:06.506154 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 by client.275175.0:13180 2014-07-30 14:19:15.201136 2014-08-13 18:06:06.506170 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 by client.275175.0:14907 2014-07-30 14:19:20.441176 2014-08-13 18:06:06.506183 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 by client.275175.0:21338 2014-07-30 14:19:40.124246 2014-08-13 18:06:06.506195 7f5c973a57a0 20 read_log 191727'428 (0'0) modify b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 by client.275175.0:23755 2014-07-30 14:19:48.141566 2014-08-13 18:06:06.506207 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 by client.275175.0:24227 2014-07-30 14:19:49.462453 2014-08-13 18:06:06.506219 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 by client.275175.0:24242 2014-07-30 14:19:49.505140 2014-08-13 18:06:06.506230 7f5c973a57a0 20 read_log 191727'433 (0'0) modify f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 by client.275175.0:25072 2014-07-30 14:19:51.907957 2014-08-13 18:06:06.506242 7f5c973a57a0 20 read_log 191727'437 (0'0) modify e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 by client.275175.0:27294 2014-07-30 14:19:58.644997 2014-08-13 18:06:06.506263 7f5c973a57a0 20 read_log 191727'439 (0'0) modify f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 by client.275175.0:29700 2014-07-30 14:20:06.045307 2014-08-13 18:06:06.506278 7f5c973a57a0 20 read_log 191727'445 (0'0) modify 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 by client.275175.0:31689 2014-07-30 14:20:11.975694 2014-08-13 18:06:06.506298 7f5c973a57a0 20 read_log 191727'447 (0'0) modify 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 by client.275175.0:31920 2014-07-30 14:20:12.623608 2014-08-13 18:06:06.506310 7f5c973a57a0 20 read_log 191727'448 (0'0) modify f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 by client.275175.0:32224 2014-07-30 14:20:13.543611 2014-08-13 18:06:06.506324 7f5c973a57a0 20 read_log 191727'455 (0'0) modify 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 by client.275175.0:33789 2014-07-30 14:20:18.389634 2014-08-13 18:06:06.506337 7f5c973a57a0 20 read_log 191727'457 (0'0) modify d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 by client.275175.0:34793 2014-07-30 14:20:21.386137 2014-08-13 18:06:06.506349 7f5c973a57a0 20 read_log 191727'462 (0'0) modify 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 by client.275175.0:35835 2014-07-30 14:20:24.745200 2014-08-13 18:06:06.506361 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 616f5481/benchmark_data_storage0111-ib.pocstor_35625_object651/head//22 by client.940330.0:652 2014-08-11 16:29:43.355266 2014-08-13 18:06:06.506373 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 3154a481/benchmark_data_storage0111-ib.pocstor_35625_object760/head//22 by client.940330.0:761 2014-08-11 16:29:43.795479 2014-08-13 18:06:06.506385 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 9622ec81/benchmark_data_storage0111-ib.pocstor_35625_object1690/head//22 by client.940330.0:1691 2014-08-11 16:29:47.414585 2014-08-13 18:06:06.506397 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 48ce2481/benchmark_data_storage0111-ib.pocstor_35625_object2240/head//22 by client.940330.0:2241 2014-08-11 16:29:49.427494 2014-08-13 18:06:06.506409 7f5c973a57a0 20 read_log 192888'468 (0'0) modify 7b2bd481/benchmark_data_storage0111-ib.pocstor_35625_object9590/head//22 by client.940330.0:9591 2014-08-11 16:30:16.394039 2014-08-13 18:06:06.506432 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 1dba8c81/benchmark_data_storage0111-ib.pocstor_35625_object9759/head//22 by client.940330.0:9760 2014-08-11 16:30:16.983964 2014-08-13 18:06:06.506448 7f5c973a57a0 20 read_log 192888'470 (0'0) modify c2b20481/benchmark_data_storage0111-ib.pocstor_35625_object11967/head//22 by client.940330.0:11968 2014-08-11 16:30:25.790406 2014-08-13 18:06:06.506466 7f5c973a57a0 20 read_log 192888'471 (0'0) modify c56f481/benchmark_data_storage0111-ib.pocstor_35625_object17540/head//22 by client.940330.0:17541 2014-08-11 16:30:46.986613 2014-08-13 18:06:06.506479 7f5c973a57a0 20 read_log 192888'472 (0'0) modify a4f3dc81/benchmark_data_storage0111-ib.pocstor_35625_object17688/head//22 by client.940330.0:17689 2014-08-11 16:30:47.548075 2014-08-13 18:06:06.506491 7f5c973a57a0 20 read_log 192888'473 (0'0) modify 9b88bc81/benchmark_data_storage0111-ib.pocstor_35625_object17932/head//22 by client.940330.0:17933 2014-08-11 16:30:48.460851 2014-08-13 18:06:06.506503 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 415da481/benchmark_data_storage0111-ib.pocstor_35625_object21802/head//22 by client.940330.0:21803 2014-08-11 16:31:03.811189 2014-08-13 18:06:06.506515 7f5c973a57a0 20 read_log 192888'475 (0'0) modify 5a879481/benchmark_data_storage0111-ib.pocstor_35625_object24308/head//22 by client.940330.0:24309 2014-08-11 16:31:13.270508 2014-08-13 18:06:06.506539 7f5c973a57a0 20 read_log 192888'476 (192888'464) delete 616f5481/benchmark_data_storage0111-ib.pocstor_35625_object651/head//22 by client.940330.0:31959 2014-08-11 16:31:41.663758 2014-08-13 18:06:06.506555 7f5c973a57a0 20 read_log 192888'477 (192888'465) delete 3154a481/benchmark_data_storage0111-ib.pocstor_35625_object760/head//22 by client.940330.0:32068 2014-08-11 16:31:41.690227 2014-08-13 18:06:06.506568 7f5c973a57a0 20 read_log 192888'478 (192888'466) delete 9622ec81/benchmark_data_storage0111-ib.pocstor_35625_object1690/head//22 by client.940330.0:32998 2014-08-11 16:31:41.912839 2014-08-13 18:06:06.506580 7f5c973a57a0 20 read_log 192888'479 (192888'467) delete 48ce2481/benchmark_data_storage0111-ib.pocstor_35625_object2240/head//22 by client.940330.0:33548 2014-08-11 16:31:42.052680 2014-08-13 18:06:06.506592 7f5c973a57a0 20 read_log 192888'480 (192888'468) delete 7b2bd481/benchmark_data_storage0111-ib.pocstor_35625_object9590/head//22 by client.940330.0:40898 2014-08-11 16:31:44.002025 2014-08-13 18:06:06.506605 7f5c973a57a0 20 read_log 192888'481 (192888'469) delete 1dba8c81/benchmark_data_storage0111-ib.pocstor_35625_object9759/head//22 by client.940330.0:41067 2014-08-11 16:31:44.048525 2014-08-13 18:06:06.506617 7f5c973a57a0 20 read_log 192888'482 (192888'470) delete c2b20481/benchmark_data_storage0111-ib.pocstor_35625_object11967/head//22 by client.940330.0:43275 2014-08-11 16:31:44.650779 2014-08-13 18:06:06.506629 7f5c973a57a0 20 read_log 192888'483 (192888'471) delete c56f481/benchmark_data_storage0111-ib.pocstor_35625_object17540/head//22 by client.940330.0:48848 2014-08-11 16:31:46.139149 2014-08-13 18:06:06.506642 7f5c973a57a0 20 read_log 192888'484 (192888'472) delete a4f3dc81/benchmark_data_storage0111-ib.pocstor_35625_object17688/head//22 by client.940330.0:48996 2014-08-11 16:31:46.188777 2014-08-13 18:06:06.506674 7f5c973a57a0 20 read_log 192888'485 (192888'473) delete 9b88bc81/benchmark_data_storage0111-ib.pocstor_35625_object17932/head//22 by client.940330.0:49240 2014-08-11 16:31:46.263695 2014-08-13 18:06:06.506689 7f5c973a57a0 20 read_log 192888'486 (192888'474) delete 415da481/benchmark_data_storage0111-ib.pocstor_35625_object21802/head//22 by client.940330.0:53110 2014-08-11 16:31:47.384654 2014-08-13 18:06:06.506702 7f5c973a57a0 20 read_log 192888'487 (192888'475) delete 5a879481/benchmark_data_storage0111-ib.pocstor_35625_object24308/head//22 by client.940330.0:55616 2014-08-11 16:31:48.130594 2014-08-13 18:06:06.506715 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 73420c81/benchmark_data_storage0111-ib.pocstor_64498_object2591/head//22 by client.1049635.0:2592 2014-08-12 17:17:25.511792 2014-08-13 18:06:06.506727 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 7cfb3481/benchmark_data_storage0111-ib.pocstor_64498_object2640/head//22 by client.1049635.0:2641 2014-08-12 17:17:25.657717 2014-08-13 18:06:06.506739 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 227d6481/benchmark_data_storage0111-ib.pocstor_64498_object5057/head//22 by client.1049635.0:5058 2014-08-12 17:17:33.087833 2014-08-13 18:06:06.506753 7f5c973a57a0 20 read_log 194215'491 (0'0) modify f98e6481/benchmark_data_storage0111-ib.pocstor_64498_object8557/head//22 by client.1049635.0:8558 2014-08-12 17:17:43.748065 2014-08-13 18:06:06.506766 7f5c973a57a0 20 read_log 194215'492 (0'0) modify 6b624c81/benchmark_data_storage0111-ib.pocstor_64498_object10778/head//22 by client.1049635.0:10779 2014-08-12 17:17:50.466707 2014-08-13 18:06:06.506780 7f5c973a57a0 20 read_log 194215'493 (0'0) modify b9e75c81/benchmark_data_storage0111-ib.pocstor_64498_object12122/head//22 by client.1049635.0:12123 2014-08-12 17:17:54.858295 2014-08-13 18:06:06.506793 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 7ecb2c81/benchmark_data_storage0111-ib.pocstor_64498_object15034/head//22 by client.1049635.0:15035 2014-08-12 17:18:04.082533 2014-08-13 18:06:06.506831 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 79f93c81/benchmark_data_storage0111-ib.pocstor_64498_object19207/head//22 by client.1049635.0:19208 2014-08-12 17:18:19.817849 2014-08-13 18:06:06.506846 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 150b5c81/benchmark_data_storage0111-ib.pocstor_64498_object19519/head//22 by client.1049635.0:19520 2014-08-12 17:18:20.977017 2014-08-13 18:06:06.506859 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 7b752c81/benchmark_data_storage0111-ib.pocstor_64498_object19713/head//22 by client.1049635.0:19714 2014-08-12 17:18:21.703346 2014-08-13 18:06:06.506887 7f5c973a57a0 20 read_log 194215'498 (0'0) modify 110ea481/benchmark_data_storage0111-ib.pocstor_64498_object25890/head//22 by client.1049635.0:25891 2014-08-12 17:18:45.013390 2014-08-13 18:06:06.506903 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 91a68481/benchmark_data_storage0111-ib.pocstor_64498_object26478/head//22 by client.1049635.0:26479 2014-08-12 17:18:47.209202 2014-08-13 18:06:06.506916 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 7530cc81/benchmark_data_storage0111-ib.pocstor_64498_object28052/head//22 by client.1049635.0:28053 2014-08-12 17:18:53.066060 2014-08-13 18:06:06.506930 7f5c973a57a0 20 read_log 194215'501 (0'0) modify b6f13c81/benchmark_data_storage0111-ib.pocstor_64498_object28758/head//22 by client.1049635.0:28759 2014-08-12 17:18:55.695417 2014-08-13 18:06:06.506942 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 60f45481/benchmark_data_storage0111-ib.pocstor_64498_object28921/head//22 by client.1049635.0:28922 2014-08-12 17:18:56.308411 2014-08-13 18:06:06.506956 7f5c973a57a0 20 read_log 194215'503 (0'0) modify 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:30136 2014-08-12 17:19:01.018764 2014-08-13 18:06:06.506969 7f5c973a57a0 20 read_log 194215'504 (194215'488) delete 73420c81/benchmark_data_storage0111-ib.pocstor_64498_object2591/head//22 by client.1049635.0:36794 2014-08-12 17:19:18.827659 2014-08-13 18:06:06.506983 7f5c973a57a0 20 read_log 194215'505 (194215'489) delete 7cfb3481/benchmark_data_storage0111-ib.pocstor_64498_object2640/head//22 by client.1049635.0:36843 2014-08-12 17:19:18.843070 2014-08-13 18:06:06.506995 7f5c973a57a0 20 read_log 194215'506 (194215'490) delete 227d6481/benchmark_data_storage0111-ib.pocstor_64498_object5057/head//22 by client.1049635.0:39260 2014-08-12 17:19:19.624355 2014-08-13 18:06:06.507009 7f5c973a57a0 20 read_log 194215'507 (194215'491) delete f98e6481/benchmark_data_storage0111-ib.pocstor_64498_object8557/head//22 by client.1049635.0:42760 2014-08-12 17:19:20.780629 2014-08-13 18:06:06.507022 7f5c973a57a0 20 read_log 194215'508 (194215'492) delete 6b624c81/benchmark_data_storage0111-ib.pocstor_64498_object10778/head//22 by client.1049635.0:44981 2014-08-12 17:19:21.510483 2014-08-13 18:06:06.507036 7f5c973a57a0 20 read_log 194215'509 (194215'493) delete b9e75c81/benchmark_data_storage0111-ib.pocstor_64498_object12122/head//22 by client.1049635.0:46325 2014-08-12 17:19:21.935751 2014-08-13 18:06:06.507048 7f5c973a57a0 20 read_log 194215'510 (194215'494) delete 7ecb2c81/benchmark_data_storage0111-ib.pocstor_64498_object15034/head//22 by client.1049635.0:49237 2014-08-12 17:19:22.885215 2014-08-13 18:06:06.507062 7f5c973a57a0 20 read_log 194215'511 (194215'495) delete 79f93c81/benchmark_data_storage0111-ib.pocstor_64498_object19207/head//22 by client.1049635.0:53410 2014-08-12 17:19:24.235640 2014-08-13 18:06:06.507075 7f5c973a57a0 20 read_log 194215'512 (194215'496) delete 150b5c81/benchmark_data_storage0111-ib.pocstor_64498_object19519/head//22 by client.1049635.0:53722 2014-08-12 17:19:24.355477 2014-08-13 18:06:06.507088 7f5c973a57a0 20 read_log 194215'513 (194215'497) delete 7b752c81/benchmark_data_storage0111-ib.pocstor_64498_object19713/head//22 by client.1049635.0:53916 2014-08-12 17:19:24.419889 2014-08-13 18:06:06.507101 7f5c973a57a0 20 read_log 194215'514 (194215'498) delete 110ea481/benchmark_data_storage0111-ib.pocstor_64498_object25890/head//22 by client.1049635.0:60093 2014-08-12 17:19:26.621415 2014-08-13 18:06:06.507117 7f5c973a57a0 20 read_log 194215'515 (194215'499) delete 91a68481/benchmark_data_storage0111-ib.pocstor_64498_object26478/head//22 by client.1049635.0:60681 2014-08-12 17:19:26.814269 2014-08-13 18:06:06.507132 7f5c973a57a0 20 read_log 194215'516 (194215'500) delete 7530cc81/benchmark_data_storage0111-ib.pocstor_64498_object28052/head//22 by client.1049635.0:62255 2014-08-12 17:19:27.364377 2014-08-13 18:06:06.507144 7f5c973a57a0 20 read_log 194215'517 (194215'501) delete b6f13c81/benchmark_data_storage0111-ib.pocstor_64498_object28758/head//22 by client.1049635.0:62961 2014-08-12 17:19:27.615796 2014-08-13 18:06:06.507165 7f5c973a57a0 20 read_log 194215'518 (194215'502) delete 60f45481/benchmark_data_storage0111-ib.pocstor_64498_object28921/head//22 by client.1049635.0:63124 2014-08-12 17:19:27.675242 2014-08-13 18:06:06.507178 7f5c973a57a0 20 read_log 194215'519 (194215'503) delete 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:64338 2014-08-12 17:19:28.088153 2014-08-13 18:06:06.507278 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.507297 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=0 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.507313 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=0 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001700 0 0.000000 2014-08-13 18:06:06.507328 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=0 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.507337 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=0 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.507346 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'519], crt=191727'463) 2014-08-13 18:06:06.507356 7f5c973a57a0 10 osd.4 194487 pgid 22.48e coll 22.48e_head 2014-08-13 18:06:06.507528 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.48e 2014-08-13 18:06:06.507538 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.507567 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.48e(unlocked)] enter Initial 2014-08-13 18:06:06.507578 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.48e(unlocked)] enter NotTrimming 2014-08-13 18:06:06.507726 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.507825 7f5c973a57a0 20 read_log 191648'1 (0'0) modify c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 by client.273158.0:26 2014-07-30 14:08:48.646718 2014-08-13 18:06:06.507845 7f5c973a57a0 20 read_log 191648'8 (0'0) modify 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 by client.273158.0:594 2014-07-30 14:08:50.992311 2014-08-13 18:06:06.507858 7f5c973a57a0 20 read_log 191648'11 (0'0) modify ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 by client.273158.0:725 2014-07-30 14:08:51.474548 2014-08-13 18:06:06.507871 7f5c973a57a0 20 read_log 191648'22 (0'0) modify 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 by client.273158.0:1181 2014-07-30 14:08:53.136462 2014-08-13 18:06:06.507883 7f5c973a57a0 20 read_log 191648'51 (0'0) modify 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 by client.273158.0:2702 2014-07-30 14:08:59.851949 2014-08-13 18:06:06.507896 7f5c973a57a0 20 read_log 191648'67 (0'0) modify 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 by client.273158.0:3870 2014-07-30 14:09:05.668495 2014-08-13 18:06:06.507908 7f5c973a57a0 20 read_log 191648'109 (0'0) modify 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 by client.273158.0:6497 2014-07-30 14:09:18.669611 2014-08-13 18:06:06.507920 7f5c973a57a0 20 read_log 191648'110 (0'0) modify 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 by client.273158.0:6617 2014-07-30 14:09:19.419227 2014-08-13 18:06:06.507932 7f5c973a57a0 20 read_log 191648'125 (0'0) modify f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 by client.273158.0:7117 2014-07-30 14:09:21.548833 2014-08-13 18:06:06.507945 7f5c973a57a0 20 read_log 191648'159 (0'0) modify fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 by client.273158.0:8616 2014-07-30 14:09:29.656074 2014-08-13 18:06:06.507957 7f5c973a57a0 20 read_log 191648'246 (0'0) modify cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 by client.273158.0:14282 2014-07-30 14:09:59.139806 2014-08-13 18:06:06.507969 7f5c973a57a0 20 read_log 191648'265 (0'0) modify dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 by client.273158.0:15742 2014-07-30 14:10:05.831344 2014-08-13 18:06:06.507981 7f5c973a57a0 20 read_log 191648'273 (0'0) modify 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 by client.273158.0:16606 2014-07-30 14:10:10.763484 2014-08-13 18:06:06.507994 7f5c973a57a0 20 read_log 191648'309 (0'0) modify 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 by client.273158.0:18717 2014-07-30 14:10:22.416198 2014-08-13 18:06:06.508006 7f5c973a57a0 20 read_log 191648'371 (0'0) modify 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 by client.273158.0:22737 2014-07-30 14:10:48.013719 2014-08-13 18:06:06.508018 7f5c973a57a0 20 read_log 191727'380 (0'0) modify d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 by client.275175.0:2402 2014-07-30 14:18:42.259382 2014-08-13 18:06:06.508030 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 by client.275175.0:3270 2014-07-30 14:18:45.069555 2014-08-13 18:06:06.508043 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 by client.275175.0:3368 2014-07-30 14:18:45.378896 2014-08-13 18:06:06.508055 7f5c973a57a0 20 read_log 191727'398 (0'0) modify cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 by client.275175.0:7523 2014-07-30 14:18:58.193895 2014-08-13 18:06:06.508075 7f5c973a57a0 20 read_log 191727'399 (0'0) modify f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 by client.275175.0:7571 2014-07-30 14:18:58.328199 2014-08-13 18:06:06.508091 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 by client.275175.0:8445 2014-07-30 14:19:00.766469 2014-08-13 18:06:06.508129 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 by client.275175.0:10455 2014-07-30 14:19:06.510270 2014-08-13 18:06:06.508142 7f5c973a57a0 20 read_log 191727'408 (0'0) modify fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 by client.275175.0:10940 2014-07-30 14:19:07.874091 2014-08-13 18:06:06.508163 7f5c973a57a0 20 read_log 191727'410 (0'0) modify df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 by client.275175.0:11110 2014-07-30 14:19:08.349474 2014-08-13 18:06:06.508175 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 by client.275175.0:11412 2014-07-30 14:19:09.273830 2014-08-13 18:06:06.508188 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 by client.275175.0:14260 2014-07-30 14:19:18.608762 2014-08-13 18:06:06.508200 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 by client.275175.0:18340 2014-07-30 14:19:30.982002 2014-08-13 18:06:06.508212 7f5c973a57a0 20 read_log 191727'428 (0'0) modify 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 by client.275175.0:19101 2014-07-30 14:19:33.610221 2014-08-13 18:06:06.508224 7f5c973a57a0 20 read_log 191727'435 (0'0) modify bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 by client.275175.0:22750 2014-07-30 14:19:45.279965 2014-08-13 18:06:06.508236 7f5c973a57a0 20 read_log 191727'437 (0'0) modify efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 by client.275175.0:23762 2014-07-30 14:19:48.161179 2014-08-13 18:06:06.508248 7f5c973a57a0 20 read_log 191727'450 (0'0) modify 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 by client.275175.0:29672 2014-07-30 14:20:05.969957 2014-08-13 18:06:06.508264 7f5c973a57a0 20 read_log 191727'452 (0'0) modify c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 by client.275175.0:30865 2014-07-30 14:20:09.595983 2014-08-13 18:06:06.508276 7f5c973a57a0 20 read_log 191727'453 (0'0) modify e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 by client.275175.0:31893 2014-07-30 14:20:12.546838 2014-08-13 18:06:06.508288 7f5c973a57a0 20 read_log 191727'455 (0'0) modify 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 by client.275175.0:32199 2014-07-30 14:20:13.471727 2014-08-13 18:06:06.508300 7f5c973a57a0 20 read_log 191727'456 (0'0) modify 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 by client.275175.0:32241 2014-07-30 14:20:13.592020 2014-08-13 18:06:06.508312 7f5c973a57a0 20 read_log 191727'458 (0'0) modify 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 by client.275175.0:33955 2014-07-30 14:20:18.862563 2014-08-13 18:06:06.508324 7f5c973a57a0 20 read_log 191727'461 (0'0) modify 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 by client.275175.0:35063 2014-07-30 14:20:22.161828 2014-08-13 18:06:06.508336 7f5c973a57a0 20 read_log 191727'469 (0'0) modify d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 by client.275175.0:36638 2014-07-30 14:20:27.297370 2014-08-13 18:06:06.508348 7f5c973a57a0 20 read_log 191727'470 (0'0) modify 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 by client.275175.0:36838 2014-07-30 14:20:27.921555 2014-08-13 18:06:06.508382 7f5c973a57a0 20 read_log 191727'473 (0'0) modify e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 by client.275175.0:38457 2014-07-30 14:20:32.542729 2014-08-13 18:06:06.508400 7f5c973a57a0 20 read_log 192888'474 (0'0) modify cd2c6c8e/benchmark_data_storage0111-ib.pocstor_35625_object322/head//22 by client.940330.0:323 2014-08-11 16:29:41.998487 2014-08-13 18:06:06.508412 7f5c973a57a0 20 read_log 192888'475 (0'0) modify 669cac8e/benchmark_data_storage0111-ib.pocstor_35625_object538/head//22 by client.940330.0:539 2014-08-11 16:29:42.883101 2014-08-13 18:06:06.508443 7f5c973a57a0 20 read_log 192888'476 (0'0) modify 5bd1b48e/benchmark_data_storage0111-ib.pocstor_35625_object4062/head//22 by client.940330.0:4063 2014-08-11 16:29:56.558578 2014-08-13 18:06:06.508459 7f5c973a57a0 20 read_log 192888'477 (0'0) modify 3318b48e/benchmark_data_storage0111-ib.pocstor_35625_object5841/head//22 by client.940330.0:5842 2014-08-11 16:30:03.002894 2014-08-13 18:06:06.508481 7f5c973a57a0 20 read_log 192888'478 (0'0) modify 917aa48e/benchmark_data_storage0111-ib.pocstor_35625_object7990/head//22 by client.940330.0:7991 2014-08-11 16:30:10.803982 2014-08-13 18:06:06.508494 7f5c973a57a0 20 read_log 192888'479 (0'0) modify c160b48e/benchmark_data_storage0111-ib.pocstor_35625_object10453/head//22 by client.940330.0:10454 2014-08-11 16:30:19.552772 2014-08-13 18:06:06.508508 7f5c973a57a0 20 read_log 192888'480 (0'0) modify ea0e2c8e/benchmark_data_storage0111-ib.pocstor_35625_object10824/head//22 by client.940330.0:10825 2014-08-11 16:30:21.115997 2014-08-13 18:06:06.508521 7f5c973a57a0 20 read_log 192888'481 (0'0) modify 5a41b48e/benchmark_data_storage0111-ib.pocstor_35625_object15525/head//22 by client.940330.0:15526 2014-08-11 16:30:39.452952 2014-08-13 18:06:06.508533 7f5c973a57a0 20 read_log 192888'482 (0'0) modify a4ac8e/benchmark_data_storage0111-ib.pocstor_35625_object16743/head//22 by client.940330.0:16744 2014-08-11 16:30:44.009417 2014-08-13 18:06:06.508545 7f5c973a57a0 20 read_log 192888'483 (0'0) modify a707ec8e/benchmark_data_storage0111-ib.pocstor_35625_object16957/head//22 by client.940330.0:16958 2014-08-11 16:30:44.804103 2014-08-13 18:06:06.508557 7f5c973a57a0 20 read_log 192888'484 (0'0) modify fa9f248e/benchmark_data_storage0111-ib.pocstor_35625_object17327/head//22 by client.940330.0:17328 2014-08-11 16:30:46.189079 2014-08-13 18:06:06.508569 7f5c973a57a0 20 read_log 192888'485 (0'0) modify 7ed82c8e/benchmark_data_storage0111-ib.pocstor_35625_object21083/head//22 by client.940330.0:21084 2014-08-11 16:31:01.151956 2014-08-13 18:06:06.508582 7f5c973a57a0 20 read_log 192888'486 (0'0) modify 5d78fc8e/benchmark_data_storage0111-ib.pocstor_35625_object23545/head//22 by client.940330.0:23546 2014-08-11 16:31:10.422534 2014-08-13 18:06:06.508593 7f5c973a57a0 20 read_log 192888'487 (0'0) modify 42a7848e/benchmark_data_storage0111-ib.pocstor_35625_object26231/head//22 by client.940330.0:26232 2014-08-11 16:31:20.701197 2014-08-13 18:06:06.508606 7f5c973a57a0 20 read_log 192888'488 (0'0) modify 3bde748e/benchmark_data_storage0111-ib.pocstor_35625_object30416/head//22 by client.940330.0:30417 2014-08-11 16:31:37.036759 2014-08-13 18:06:06.508637 7f5c973a57a0 20 read_log 192888'489 (192888'474) delete cd2c6c8e/benchmark_data_storage0111-ib.pocstor_35625_object322/head//22 by client.940330.0:31630 2014-08-11 16:31:41.571063 2014-08-13 18:06:06.508660 7f5c973a57a0 20 read_log 192888'490 (192888'475) delete 669cac8e/benchmark_data_storage0111-ib.pocstor_35625_object538/head//22 by client.940330.0:31846 2014-08-11 16:31:41.632023 2014-08-13 18:06:06.508678 7f5c973a57a0 20 read_log 192888'491 (192888'476) delete 5bd1b48e/benchmark_data_storage0111-ib.pocstor_35625_object4062/head//22 by client.940330.0:35370 2014-08-11 16:31:42.552595 2014-08-13 18:06:06.508730 7f5c973a57a0 20 read_log 192888'492 (192888'477) delete 3318b48e/benchmark_data_storage0111-ib.pocstor_35625_object5841/head//22 by client.940330.0:37149 2014-08-11 16:31:43.034086 2014-08-13 18:06:06.508746 7f5c973a57a0 20 read_log 192888'493 (192888'478) delete 917aa48e/benchmark_data_storage0111-ib.pocstor_35625_object7990/head//22 by client.940330.0:39298 2014-08-11 16:31:43.587500 2014-08-13 18:06:06.508759 7f5c973a57a0 20 read_log 192888'494 (192888'479) delete c160b48e/benchmark_data_storage0111-ib.pocstor_35625_object10453/head//22 by client.940330.0:41761 2014-08-11 16:31:44.240058 2014-08-13 18:06:06.508771 7f5c973a57a0 20 read_log 192888'495 (192888'480) delete ea0e2c8e/benchmark_data_storage0111-ib.pocstor_35625_object10824/head//22 by client.940330.0:42132 2014-08-11 16:31:44.349897 2014-08-13 18:06:06.508783 7f5c973a57a0 20 read_log 192888'496 (192888'481) delete 5a41b48e/benchmark_data_storage0111-ib.pocstor_35625_object15525/head//22 by client.940330.0:46833 2014-08-11 16:31:45.597270 2014-08-13 18:06:06.508796 7f5c973a57a0 20 read_log 192888'497 (192888'482) delete a4ac8e/benchmark_data_storage0111-ib.pocstor_35625_object16743/head//22 by client.940330.0:48051 2014-08-11 16:31:45.918183 2014-08-13 18:06:06.508808 7f5c973a57a0 20 read_log 192888'498 (192888'483) delete a707ec8e/benchmark_data_storage0111-ib.pocstor_35625_object16957/head//22 by client.940330.0:48265 2014-08-11 16:31:45.977173 2014-08-13 18:06:06.508820 7f5c973a57a0 20 read_log 192888'499 (192888'484) delete fa9f248e/benchmark_data_storage0111-ib.pocstor_35625_object17327/head//22 by client.940330.0:48635 2014-08-11 16:31:46.081888 2014-08-13 18:06:06.508832 7f5c973a57a0 20 read_log 192888'500 (192888'485) delete 7ed82c8e/benchmark_data_storage0111-ib.pocstor_35625_object21083/head//22 by client.940330.0:52391 2014-08-11 16:31:47.183825 2014-08-13 18:06:06.508847 7f5c973a57a0 20 read_log 192888'501 (192888'486) delete 5d78fc8e/benchmark_data_storage0111-ib.pocstor_35625_object23545/head//22 by client.940330.0:54853 2014-08-11 16:31:47.908453 2014-08-13 18:06:06.508860 7f5c973a57a0 20 read_log 192888'502 (192888'487) delete 42a7848e/benchmark_data_storage0111-ib.pocstor_35625_object26231/head//22 by client.940330.0:57539 2014-08-11 16:31:48.713078 2014-08-13 18:06:06.508872 7f5c973a57a0 20 read_log 192888'503 (192888'488) delete 3bde748e/benchmark_data_storage0111-ib.pocstor_35625_object30416/head//22 by client.940330.0:61724 2014-08-11 16:31:49.876608 2014-08-13 18:06:06.508884 7f5c973a57a0 20 read_log 194215'504 (0'0) modify f98fdc8e/benchmark_data_storage0111-ib.pocstor_64498_object28/head//22 by client.1049635.0:29 2014-08-12 17:17:17.032421 2014-08-13 18:06:06.508896 7f5c973a57a0 20 read_log 194215'505 (0'0) modify ed082c8e/benchmark_data_storage0111-ib.pocstor_64498_object2105/head//22 by client.1049635.0:2106 2014-08-12 17:17:23.864246 2014-08-13 18:06:06.508909 7f5c973a57a0 20 read_log 194215'506 (0'0) modify fa38148e/benchmark_data_storage0111-ib.pocstor_64498_object2397/head//22 by client.1049635.0:2398 2014-08-12 17:17:24.923333 2014-08-13 18:06:06.508921 7f5c973a57a0 20 read_log 194215'507 (0'0) modify 46696c8e/benchmark_data_storage0111-ib.pocstor_64498_object5518/head//22 by client.1049635.0:5519 2014-08-12 17:17:34.668262 2014-08-13 18:06:06.508935 7f5c973a57a0 20 read_log 194215'508 (0'0) modify df5eb48e/benchmark_data_storage0111-ib.pocstor_64498_object5652/head//22 by client.1049635.0:5653 2014-08-12 17:17:35.045714 2014-08-13 18:06:06.508955 7f5c973a57a0 20 read_log 194215'509 (0'0) modify 8184a48e/benchmark_data_storage0111-ib.pocstor_64498_object7318/head//22 by client.1049635.0:7319 2014-08-12 17:17:40.025030 2014-08-13 18:06:06.508975 7f5c973a57a0 20 read_log 194215'510 (0'0) modify 80d8b48e/benchmark_data_storage0111-ib.pocstor_64498_object10767/head//22 by client.1049635.0:10768 2014-08-12 17:17:50.434516 2014-08-13 18:06:06.508999 7f5c973a57a0 20 read_log 194215'511 (0'0) modify b4523c8e/benchmark_data_storage0111-ib.pocstor_64498_object15072/head//22 by client.1049635.0:15073 2014-08-12 17:18:04.204502 2014-08-13 18:06:06.509025 7f5c973a57a0 20 read_log 194215'512 (0'0) modify 9291c48e/benchmark_data_storage0111-ib.pocstor_64498_object16532/head//22 by client.1049635.0:16533 2014-08-12 17:18:09.583986 2014-08-13 18:06:06.509041 7f5c973a57a0 20 read_log 194215'513 (0'0) modify 6fc1248e/benchmark_data_storage0111-ib.pocstor_64498_object17593/head//22 by client.1049635.0:17594 2014-08-12 17:18:13.622871 2014-08-13 18:06:06.509053 7f5c973a57a0 20 read_log 194215'514 (0'0) modify b180648e/benchmark_data_storage0111-ib.pocstor_64498_object21571/head//22 by client.1049635.0:21572 2014-08-12 17:18:28.721561 2014-08-13 18:06:06.509072 7f5c973a57a0 20 read_log 194215'515 (0'0) modify 717a5c8e/benchmark_data_storage0111-ib.pocstor_64498_object22512/head//22 by client.1049635.0:22513 2014-08-12 17:18:32.316627 2014-08-13 18:06:06.509091 7f5c973a57a0 20 read_log 194215'516 (0'0) modify d57b948e/benchmark_data_storage0111-ib.pocstor_64498_object22835/head//22 by client.1049635.0:22836 2014-08-12 17:18:33.521701 2014-08-13 18:06:06.509106 7f5c973a57a0 20 read_log 194215'517 (0'0) modify e04f548e/benchmark_data_storage0111-ib.pocstor_64498_object24458/head//22 by client.1049635.0:24459 2014-08-12 17:18:39.556547 2014-08-13 18:06:06.509118 7f5c973a57a0 20 read_log 194215'518 (0'0) modify d6678c8e/benchmark_data_storage0111-ib.pocstor_64498_object24663/head//22 by client.1049635.0:24664 2014-08-12 17:18:40.321956 2014-08-13 18:06:06.509130 7f5c973a57a0 20 read_log 194215'519 (0'0) modify 149bb48e/benchmark_data_storage0111-ib.pocstor_64498_object26333/head//22 by client.1049635.0:26334 2014-08-12 17:18:46.658119 2014-08-13 18:06:06.509143 7f5c973a57a0 20 read_log 194215'520 (0'0) modify cff48c8e/benchmark_data_storage0111-ib.pocstor_64498_object29414/head//22 by client.1049635.0:29415 2014-08-12 17:18:58.134134 2014-08-13 18:06:06.509161 7f5c973a57a0 20 read_log 194215'521 (0'0) modify 4c84148e/benchmark_data_storage0111-ib.pocstor_64498_object31608/head//22 by client.1049635.0:31609 2014-08-12 17:19:07.049100 2014-08-13 18:06:06.509173 7f5c973a57a0 20 read_log 194215'522 (0'0) modify 798a448e/benchmark_data_storage0111-ib.pocstor_64498_object33432/head//22 by client.1049635.0:33433 2014-08-12 17:19:14.019189 2014-08-13 18:06:06.509186 7f5c973a57a0 20 read_log 194215'523 (0'0) modify cd52148e/benchmark_data_storage0111-ib.pocstor_64498_object33494/head//22 by client.1049635.0:33495 2014-08-12 17:19:14.253182 2014-08-13 18:06:06.509198 7f5c973a57a0 20 read_log 194215'524 (194215'504) delete f98fdc8e/benchmark_data_storage0111-ib.pocstor_64498_object28/head//22 by client.1049635.0:34231 2014-08-12 17:19:17.986064 2014-08-13 18:06:06.509210 7f5c973a57a0 20 read_log 194215'525 (194215'505) delete ed082c8e/benchmark_data_storage0111-ib.pocstor_64498_object2105/head//22 by client.1049635.0:36308 2014-08-12 17:19:18.676401 2014-08-13 18:06:06.509225 7f5c973a57a0 20 read_log 194215'526 (194215'506) delete fa38148e/benchmark_data_storage0111-ib.pocstor_64498_object2397/head//22 by client.1049635.0:36600 2014-08-12 17:19:18.767261 2014-08-13 18:06:06.509240 7f5c973a57a0 20 read_log 194215'527 (194215'507) delete 46696c8e/benchmark_data_storage0111-ib.pocstor_64498_object5518/head//22 by client.1049635.0:39721 2014-08-12 17:19:19.778798 2014-08-13 18:06:06.509253 7f5c973a57a0 20 read_log 194215'528 (194215'508) delete df5eb48e/benchmark_data_storage0111-ib.pocstor_64498_object5652/head//22 by client.1049635.0:39855 2014-08-12 17:19:19.833243 2014-08-13 18:06:06.509265 7f5c973a57a0 20 read_log 194215'529 (194215'509) delete 8184a48e/benchmark_data_storage0111-ib.pocstor_64498_object7318/head//22 by client.1049635.0:41521 2014-08-12 17:19:20.389816 2014-08-13 18:06:06.509289 7f5c973a57a0 20 read_log 194215'530 (194215'510) delete 80d8b48e/benchmark_data_storage0111-ib.pocstor_64498_object10767/head//22 by client.1049635.0:44970 2014-08-12 17:19:21.507297 2014-08-13 18:06:06.509313 7f5c973a57a0 20 read_log 194215'531 (194215'511) delete b4523c8e/benchmark_data_storage0111-ib.pocstor_64498_object15072/head//22 by client.1049635.0:49275 2014-08-12 17:19:22.896798 2014-08-13 18:06:06.509334 7f5c973a57a0 20 read_log 194215'532 (194215'512) delete 9291c48e/benchmark_data_storage0111-ib.pocstor_64498_object16532/head//22 by client.1049635.0:50735 2014-08-12 17:19:23.361790 2014-08-13 18:06:06.509347 7f5c973a57a0 20 read_log 194215'533 (194215'513) delete 6fc1248e/benchmark_data_storage0111-ib.pocstor_64498_object17593/head//22 by client.1049635.0:51796 2014-08-12 17:19:23.695362 2014-08-13 18:06:06.509371 7f5c973a57a0 20 read_log 194215'534 (194215'514) delete b180648e/benchmark_data_storage0111-ib.pocstor_64498_object21571/head//22 by client.1049635.0:55774 2014-08-12 17:19:25.078168 2014-08-13 18:06:06.509387 7f5c973a57a0 20 read_log 194215'535 (194215'515) delete 717a5c8e/benchmark_data_storage0111-ib.pocstor_64498_object22512/head//22 by client.1049635.0:56715 2014-08-12 17:19:25.426543 2014-08-13 18:06:06.509406 7f5c973a57a0 20 read_log 194215'536 (194215'516) delete d57b948e/benchmark_data_storage0111-ib.pocstor_64498_object22835/head//22 by client.1049635.0:57038 2014-08-12 17:19:25.556395 2014-08-13 18:06:06.509422 7f5c973a57a0 20 read_log 194215'537 (194215'517) delete e04f548e/benchmark_data_storage0111-ib.pocstor_64498_object24458/head//22 by client.1049635.0:58661 2014-08-12 17:19:26.119173 2014-08-13 18:06:06.509435 7f5c973a57a0 20 read_log 194215'538 (194215'518) delete d6678c8e/benchmark_data_storage0111-ib.pocstor_64498_object24663/head//22 by client.1049635.0:58866 2014-08-12 17:19:26.197839 2014-08-13 18:06:06.509447 7f5c973a57a0 20 read_log 194215'539 (194215'519) delete 149bb48e/benchmark_data_storage0111-ib.pocstor_64498_object26333/head//22 by client.1049635.0:60536 2014-08-12 17:19:26.769915 2014-08-13 18:06:06.509468 7f5c973a57a0 20 read_log 194215'540 (194215'520) delete cff48c8e/benchmark_data_storage0111-ib.pocstor_64498_object29414/head//22 by client.1049635.0:63617 2014-08-12 17:19:27.856722 2014-08-13 18:06:06.509480 7f5c973a57a0 20 read_log 194215'541 (194215'521) delete 4c84148e/benchmark_data_storage0111-ib.pocstor_64498_object31608/head//22 by client.1049635.0:65811 2014-08-12 17:19:28.602741 2014-08-13 18:06:06.509493 7f5c973a57a0 20 read_log 194215'542 (194215'522) delete 798a448e/benchmark_data_storage0111-ib.pocstor_64498_object33432/head//22 by client.1049635.0:67635 2014-08-12 17:19:29.268938 2014-08-13 18:06:06.509505 7f5c973a57a0 20 read_log 194215'543 (194215'523) delete cd52148e/benchmark_data_storage0111-ib.pocstor_64498_object33494/head//22 by client.1049635.0:67697 2014-08-12 17:19:29.289388 2014-08-13 18:06:06.509616 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.509636 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=0 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.509651 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=0 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.002085 0 0.000000 2014-08-13 18:06:06.509663 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=0 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.509672 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=0 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.509681 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'543], crt=191727'473) 2014-08-13 18:06:06.509692 7f5c973a57a0 10 osd.4 194487 pgid 22.4aa coll 22.4aa_head 2014-08-13 18:06:06.509853 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.4aa 2014-08-13 18:06:06.509862 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.509898 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4aa(unlocked)] enter Initial 2014-08-13 18:06:06.509910 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4aa(unlocked)] enter NotTrimming 2014-08-13 18:06:06.510050 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.510167 7f5c973a57a0 20 read_log 191648'3 (0'0) modify b51acaa/benchmark_data_storage0111-ib.pocstor_40745_object45/head//22 by client.273158.0:46 2014-07-30 14:08:48.787550 2014-08-13 18:06:06.510188 7f5c973a57a0 20 read_log 191648'71 (0'0) modify 99ae54aa/benchmark_data_storage0111-ib.pocstor_40745_object4754/head//22 by client.273158.0:4755 2014-07-30 14:09:09.888332 2014-08-13 18:06:06.510200 7f5c973a57a0 20 read_log 191648'112 (0'0) modify af2f1caa/benchmark_data_storage0111-ib.pocstor_40745_object7536/head//22 by client.273158.0:7537 2014-07-30 14:09:23.628445 2014-08-13 18:06:06.510213 7f5c973a57a0 20 read_log 191648'159 (0'0) modify f33d4caa/benchmark_data_storage0111-ib.pocstor_40745_object10718/head//22 by client.273158.0:10719 2014-07-30 14:09:39.611444 2014-08-13 18:06:06.510225 7f5c973a57a0 20 read_log 191648'180 (0'0) modify 99ebd4aa/benchmark_data_storage0111-ib.pocstor_40745_object12728/head//22 by client.273158.0:12729 2014-07-30 14:09:51.130131 2014-08-13 18:06:06.510237 7f5c973a57a0 20 read_log 191648'218 (0'0) modify 457144aa/benchmark_data_storage0111-ib.pocstor_40745_object15319/head//22 by client.273158.0:15320 2014-07-30 14:10:03.951159 2014-08-13 18:06:06.510249 7f5c973a57a0 20 read_log 191648'328 (0'0) modify 7ce87caa/benchmark_data_storage0111-ib.pocstor_40745_object22191/head//22 by client.273158.0:22192 2014-07-30 14:10:44.944594 2014-08-13 18:06:06.510265 7f5c973a57a0 20 read_log 191727'336 (0'0) modify 891174aa/benchmark_data_storage0111-ib.pocstor_51285_object128/head//22 by client.275175.0:129 2014-07-30 14:18:33.960086 2014-08-13 18:06:06.510278 7f5c973a57a0 20 read_log 191727'338 (0'0) modify 93360caa/benchmark_data_storage0111-ib.pocstor_51285_object429/head//22 by client.275175.0:430 2014-07-30 14:18:35.151171 2014-08-13 18:06:06.510290 7f5c973a57a0 20 read_log 191727'339 (0'0) modify 1a7634aa/benchmark_data_storage0111-ib.pocstor_51285_object592/head//22 by client.275175.0:593 2014-07-30 14:18:35.798009 2014-08-13 18:06:06.510308 7f5c973a57a0 20 read_log 191727'342 (0'0) modify 99c09caa/benchmark_data_storage0111-ib.pocstor_51285_object1952/head//22 by client.275175.0:1953 2014-07-30 14:18:40.743701 2014-08-13 18:06:06.510323 7f5c973a57a0 20 read_log 191727'347 (0'0) modify 67c484aa/benchmark_data_storage0111-ib.pocstor_51285_object5443/head//22 by client.275175.0:5444 2014-07-30 14:18:51.655065 2014-08-13 18:06:06.510336 7f5c973a57a0 20 read_log 191727'349 (0'0) modify ed9f84aa/benchmark_data_storage0111-ib.pocstor_51285_object5952/head//22 by client.275175.0:5953 2014-07-30 14:18:53.108922 2014-08-13 18:06:06.510348 7f5c973a57a0 20 read_log 191727'352 (0'0) modify b7d22caa/benchmark_data_storage0111-ib.pocstor_51285_object6321/head//22 by client.275175.0:6322 2014-07-30 14:18:54.214929 2014-08-13 18:06:06.510366 7f5c973a57a0 20 read_log 191727'355 (0'0) modify 174a74aa/benchmark_data_storage0111-ib.pocstor_51285_object8009/head//22 by client.275175.0:8010 2014-07-30 14:18:59.549430 2014-08-13 18:06:06.510393 7f5c973a57a0 20 read_log 191727'357 (0'0) modify e823bcaa/benchmark_data_storage0111-ib.pocstor_51285_object8940/head//22 by client.275175.0:8941 2014-07-30 14:19:02.179241 2014-08-13 18:06:06.510412 7f5c973a57a0 20 read_log 191727'359 (0'0) modify 1d1b44aa/benchmark_data_storage0111-ib.pocstor_51285_object9631/head//22 by client.275175.0:9632 2014-07-30 14:19:04.156961 2014-08-13 18:06:06.510432 7f5c973a57a0 20 read_log 191727'363 (0'0) modify 9de244aa/benchmark_data_storage0111-ib.pocstor_51285_object13048/head//22 by client.275175.0:13049 2014-07-30 14:19:14.832051 2014-08-13 18:06:06.510447 7f5c973a57a0 20 read_log 191727'367 (0'0) modify e5da4caa/benchmark_data_storage0111-ib.pocstor_51285_object15028/head//22 by client.275175.0:15029 2014-07-30 14:19:20.782242 2014-08-13 18:06:06.510468 7f5c973a57a0 20 read_log 191727'371 (0'0) modify f5dddcaa/benchmark_data_storage0111-ib.pocstor_51285_object15992/head//22 by client.275175.0:15993 2014-07-30 14:19:23.813770 2014-08-13 18:06:06.510492 7f5c973a57a0 20 read_log 191727'380 (0'0) modify f632fcaa/benchmark_data_storage0111-ib.pocstor_51285_object20990/head//22 by client.275175.0:20991 2014-07-30 14:19:39.104325 2014-08-13 18:06:06.510505 7f5c973a57a0 20 read_log 191727'381 (0'0) modify 6d6534aa/benchmark_data_storage0111-ib.pocstor_51285_object20991/head//22 by client.275175.0:20992 2014-07-30 14:19:39.106129 2014-08-13 18:06:06.510517 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 43aa2caa/benchmark_data_storage0111-ib.pocstor_51285_object21283/head//22 by client.275175.0:21284 2014-07-30 14:19:39.970931 2014-08-13 18:06:06.510529 7f5c973a57a0 20 read_log 191727'383 (0'0) modify 6f764aa/benchmark_data_storage0111-ib.pocstor_51285_object22801/head//22 by client.275175.0:22802 2014-07-30 14:19:45.426390 2014-08-13 18:06:06.510541 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 214004aa/benchmark_data_storage0111-ib.pocstor_51285_object25497/head//22 by client.275175.0:25498 2014-07-30 14:19:53.253545 2014-08-13 18:06:06.510556 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 5a2104aa/benchmark_data_storage0111-ib.pocstor_51285_object29411/head//22 by client.275175.0:29412 2014-07-30 14:20:05.132669 2014-08-13 18:06:06.510571 7f5c973a57a0 20 read_log 191727'397 (0'0) modify ae4dcaa/benchmark_data_storage0111-ib.pocstor_51285_object31225/head//22 by client.275175.0:31226 2014-07-30 14:20:10.627748 2014-08-13 18:06:06.510583 7f5c973a57a0 20 read_log 191727'401 (0'0) modify ec101caa/benchmark_data_storage0111-ib.pocstor_51285_object33130/head//22 by client.275175.0:33131 2014-07-30 14:20:16.161536 2014-08-13 18:06:06.510595 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 7e4da4aa/benchmark_data_storage0111-ib.pocstor_51285_object33892/head//22 by client.275175.0:33893 2014-07-30 14:20:18.683513 2014-08-13 18:06:06.510607 7f5c973a57a0 20 read_log 191727'404 (0'0) modify f0dd54aa/benchmark_data_storage0111-ib.pocstor_51285_object34289/head//22 by client.275175.0:34290 2014-07-30 14:20:19.881177 2014-08-13 18:06:06.510619 7f5c973a57a0 20 read_log 191727'408 (0'0) modify 1ce2dcaa/benchmark_data_storage0111-ib.pocstor_51285_object36532/head//22 by client.275175.0:36533 2014-07-30 14:20:26.943308 2014-08-13 18:06:06.510631 7f5c973a57a0 20 read_log 191727'409 (0'0) modify c1364caa/benchmark_data_storage0111-ib.pocstor_51285_object37195/head//22 by client.275175.0:37196 2014-07-30 14:20:28.917566 2014-08-13 18:06:06.510644 7f5c973a57a0 20 read_log 191727'412 (0'0) modify a72e64aa/benchmark_data_storage0111-ib.pocstor_51285_object37846/head//22 by client.275175.0:37847 2014-07-30 14:20:30.811454 2014-08-13 18:06:06.510656 7f5c973a57a0 20 read_log 192674'416 (0'0) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5879 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510668 7f5c973a57a0 20 read_log 192674'417 (192674'416) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5880 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510699 7f5c973a57a0 20 read_log 192674'418 (192674'417) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5881 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510724 7f5c973a57a0 20 read_log 192674'419 (192674'418) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5882 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510740 7f5c973a57a0 20 read_log 192674'420 (192674'419) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5883 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510753 7f5c973a57a0 20 read_log 192674'421 (192674'420) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5884 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510773 7f5c973a57a0 20 read_log 192674'422 (192674'421) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5885 2014-07-31 18:03:47.835000 2014-08-13 18:06:06.510788 7f5c973a57a0 20 read_log 192674'423 (192674'422) modify 97df34aa/rb.0.43b76.238e1f29.000000000281/head//22 by client.279772.1:5886 2014-07-31 18:03:47.871000 2014-08-13 18:06:06.510800 7f5c973a57a0 20 read_log 192888'424 (0'0) modify 94d964aa/benchmark_data_storage0111-ib.pocstor_35625_object1907/head//22 by client.940330.0:1908 2014-08-11 16:29:48.198825 2014-08-13 18:06:06.510812 7f5c973a57a0 20 read_log 192888'425 (0'0) modify d2e02caa/benchmark_data_storage0111-ib.pocstor_35625_object2345/head//22 by client.940330.0:2346 2014-08-11 16:29:49.802854 2014-08-13 18:06:06.510824 7f5c973a57a0 20 read_log 192888'426 (0'0) modify 3f205caa/benchmark_data_storage0111-ib.pocstor_35625_object3104/head//22 by client.940330.0:3105 2014-08-11 16:29:52.947718 2014-08-13 18:06:06.510836 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 321694aa/benchmark_data_storage0111-ib.pocstor_35625_object3185/head//22 by client.940330.0:3186 2014-08-11 16:29:53.245566 2014-08-13 18:06:06.510848 7f5c973a57a0 20 read_log 192888'428 (0'0) modify f31f9caa/benchmark_data_storage0111-ib.pocstor_35625_object5271/head//22 by client.940330.0:5272 2014-08-11 16:30:00.987260 2014-08-13 18:06:06.510861 7f5c973a57a0 20 read_log 192888'429 (0'0) modify a28564aa/benchmark_data_storage0111-ib.pocstor_35625_object6086/head//22 by client.940330.0:6087 2014-08-11 16:30:03.889478 2014-08-13 18:06:06.510873 7f5c973a57a0 20 read_log 192888'430 (0'0) modify a4abdcaa/benchmark_data_storage0111-ib.pocstor_35625_object15804/head//22 by client.940330.0:15805 2014-08-11 16:30:40.498672 2014-08-13 18:06:06.510885 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 8d7974aa/benchmark_data_storage0111-ib.pocstor_35625_object17338/head//22 by client.940330.0:17339 2014-08-11 16:30:46.229075 2014-08-13 18:06:06.510897 7f5c973a57a0 20 read_log 192888'432 (0'0) modify dd609caa/benchmark_data_storage0111-ib.pocstor_35625_object18795/head//22 by client.940330.0:18796 2014-08-11 16:30:51.920245 2014-08-13 18:06:06.510908 7f5c973a57a0 20 read_log 192888'433 (0'0) modify f8b78caa/benchmark_data_storage0111-ib.pocstor_35625_object24492/head//22 by client.940330.0:24493 2014-08-11 16:31:13.945173 2014-08-13 18:06:06.510920 7f5c973a57a0 20 read_log 192888'434 (0'0) modify b5bf4caa/benchmark_data_storage0111-ib.pocstor_35625_object25590/head//22 by client.940330.0:25591 2014-08-11 16:31:18.187183 2014-08-13 18:06:06.510935 7f5c973a57a0 20 read_log 192888'435 (0'0) modify d91b5caa/benchmark_data_storage0111-ib.pocstor_35625_object27134/head//22 by client.940330.0:27135 2014-08-11 16:31:24.319755 2014-08-13 18:06:06.510947 7f5c973a57a0 20 read_log 192888'436 (0'0) modify ec22b4aa/benchmark_data_storage0111-ib.pocstor_35625_object28562/head//22 by client.940330.0:28563 2014-08-11 16:31:29.905926 2014-08-13 18:06:06.510961 7f5c973a57a0 20 read_log 192888'437 (0'0) modify bff364aa/benchmark_data_storage0111-ib.pocstor_35625_object30784/head//22 by client.940330.0:30785 2014-08-11 16:31:38.408271 2014-08-13 18:06:06.510973 7f5c973a57a0 20 read_log 192888'438 (192888'424) delete 94d964aa/benchmark_data_storage0111-ib.pocstor_35625_object1907/head//22 by client.940330.0:33215 2014-08-11 16:31:41.967945 2014-08-13 18:06:06.511039 7f5c973a57a0 20 read_log 192888'439 (192888'425) delete d2e02caa/benchmark_data_storage0111-ib.pocstor_35625_object2345/head//22 by client.940330.0:33653 2014-08-11 16:31:42.081424 2014-08-13 18:06:06.511054 7f5c973a57a0 20 read_log 192888'440 (192888'426) delete 3f205caa/benchmark_data_storage0111-ib.pocstor_35625_object3104/head//22 by client.940330.0:34412 2014-08-11 16:31:42.279530 2014-08-13 18:06:06.511067 7f5c973a57a0 20 read_log 192888'441 (192888'427) delete 321694aa/benchmark_data_storage0111-ib.pocstor_35625_object3185/head//22 by client.940330.0:34493 2014-08-11 16:31:42.304669 2014-08-13 18:06:06.511089 7f5c973a57a0 20 read_log 192888'442 (192888'428) delete f31f9caa/benchmark_data_storage0111-ib.pocstor_35625_object5271/head//22 by client.940330.0:36579 2014-08-11 16:31:42.885612 2014-08-13 18:06:06.511104 7f5c973a57a0 20 read_log 192888'443 (192888'429) delete a28564aa/benchmark_data_storage0111-ib.pocstor_35625_object6086/head//22 by client.940330.0:37394 2014-08-11 16:31:43.099669 2014-08-13 18:06:06.511116 7f5c973a57a0 20 read_log 192888'444 (192888'430) delete a4abdcaa/benchmark_data_storage0111-ib.pocstor_35625_object15804/head//22 by client.940330.0:47112 2014-08-11 16:31:45.673141 2014-08-13 18:06:06.511130 7f5c973a57a0 20 read_log 192888'445 (192888'431) delete 8d7974aa/benchmark_data_storage0111-ib.pocstor_35625_object17338/head//22 by client.940330.0:48646 2014-08-11 16:31:46.085531 2014-08-13 18:06:06.511151 7f5c973a57a0 20 read_log 192888'446 (192888'432) delete dd609caa/benchmark_data_storage0111-ib.pocstor_35625_object18795/head//22 by client.940330.0:50103 2014-08-11 16:31:46.525223 2014-08-13 18:06:06.511166 7f5c973a57a0 20 read_log 192888'447 (192888'433) delete f8b78caa/benchmark_data_storage0111-ib.pocstor_35625_object24492/head//22 by client.940330.0:55800 2014-08-11 16:31:48.184714 2014-08-13 18:06:06.511178 7f5c973a57a0 20 read_log 192888'448 (192888'434) delete b5bf4caa/benchmark_data_storage0111-ib.pocstor_35625_object25590/head//22 by client.940330.0:56898 2014-08-11 16:31:48.529000 2014-08-13 18:06:06.511191 7f5c973a57a0 20 read_log 192888'449 (192888'435) delete d91b5caa/benchmark_data_storage0111-ib.pocstor_35625_object27134/head//22 by client.940330.0:58442 2014-08-11 16:31:48.962032 2014-08-13 18:06:06.511203 7f5c973a57a0 20 read_log 192888'450 (192888'436) delete ec22b4aa/benchmark_data_storage0111-ib.pocstor_35625_object28562/head//22 by client.940330.0:59870 2014-08-11 16:31:49.353460 2014-08-13 18:06:06.511215 7f5c973a57a0 20 read_log 192888'451 (192888'437) delete bff364aa/benchmark_data_storage0111-ib.pocstor_35625_object30784/head//22 by client.940330.0:62092 2014-08-11 16:31:49.985182 2014-08-13 18:06:06.511228 7f5c973a57a0 20 read_log 194215'452 (0'0) modify d3493caa/benchmark_data_storage0111-ib.pocstor_64498_object1388/head//22 by client.1049635.0:1389 2014-08-12 17:17:21.657762 2014-08-13 18:06:06.511242 7f5c973a57a0 20 read_log 194215'453 (0'0) modify 2f813caa/benchmark_data_storage0111-ib.pocstor_64498_object1557/head//22 by client.1049635.0:1558 2014-08-12 17:17:22.170318 2014-08-13 18:06:06.511254 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 142d74aa/benchmark_data_storage0111-ib.pocstor_64498_object2336/head//22 by client.1049635.0:2337 2014-08-12 17:17:24.658250 2014-08-13 18:06:06.511291 7f5c973a57a0 20 read_log 194215'455 (0'0) modify 9e73a4aa/benchmark_data_storage0111-ib.pocstor_64498_object4572/head//22 by client.1049635.0:4573 2014-08-12 17:17:31.696988 2014-08-13 18:06:06.511306 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 3ffaecaa/benchmark_data_storage0111-ib.pocstor_64498_object5431/head//22 by client.1049635.0:5432 2014-08-12 17:17:34.312310 2014-08-13 18:06:06.511319 7f5c973a57a0 20 read_log 194215'457 (0'0) modify fab7fcaa/benchmark_data_storage0111-ib.pocstor_64498_object11361/head//22 by client.1049635.0:11362 2014-08-12 17:17:52.361725 2014-08-13 18:06:06.511334 7f5c973a57a0 20 read_log 194215'458 (0'0) modify ad684caa/benchmark_data_storage0111-ib.pocstor_64498_object13689/head//22 by client.1049635.0:13690 2014-08-12 17:17:59.617001 2014-08-13 18:06:06.511349 7f5c973a57a0 20 read_log 194215'459 (0'0) modify d7bd84aa/benchmark_data_storage0111-ib.pocstor_64498_object17483/head//22 by client.1049635.0:17484 2014-08-12 17:18:13.201423 2014-08-13 18:06:06.511361 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 96d0a4aa/benchmark_data_storage0111-ib.pocstor_64498_object19126/head//22 by client.1049635.0:19127 2014-08-12 17:18:19.494667 2014-08-13 18:06:06.511375 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 4dd09caa/benchmark_data_storage0111-ib.pocstor_64498_object23563/head//22 by client.1049635.0:23564 2014-08-12 17:18:36.228407 2014-08-13 18:06:06.511394 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 69484caa/benchmark_data_storage0111-ib.pocstor_64498_object23826/head//22 by client.1049635.0:23827 2014-08-12 17:18:37.207392 2014-08-13 18:06:06.511416 7f5c973a57a0 20 read_log 194215'463 (194215'452) delete d3493caa/benchmark_data_storage0111-ib.pocstor_64498_object1388/head//22 by client.1049635.0:35591 2014-08-12 17:19:18.448214 2014-08-13 18:06:06.511435 7f5c973a57a0 20 read_log 194215'464 (194215'453) delete 2f813caa/benchmark_data_storage0111-ib.pocstor_64498_object1557/head//22 by client.1049635.0:35760 2014-08-12 17:19:18.506422 2014-08-13 18:06:06.511450 7f5c973a57a0 20 read_log 194215'465 (194215'454) delete 142d74aa/benchmark_data_storage0111-ib.pocstor_64498_object2336/head//22 by client.1049635.0:36539 2014-08-12 17:19:18.749574 2014-08-13 18:06:06.511462 7f5c973a57a0 20 read_log 194215'466 (194215'455) delete 9e73a4aa/benchmark_data_storage0111-ib.pocstor_64498_object4572/head//22 by client.1049635.0:38775 2014-08-12 17:19:19.460370 2014-08-13 18:06:06.511485 7f5c973a57a0 20 read_log 194215'467 (194215'456) delete 3ffaecaa/benchmark_data_storage0111-ib.pocstor_64498_object5431/head//22 by client.1049635.0:39634 2014-08-12 17:19:19.748345 2014-08-13 18:06:06.511498 7f5c973a57a0 20 read_log 194215'468 (194215'457) delete fab7fcaa/benchmark_data_storage0111-ib.pocstor_64498_object11361/head//22 by client.1049635.0:45564 2014-08-12 17:19:21.691808 2014-08-13 18:06:06.511510 7f5c973a57a0 20 read_log 194215'469 (194215'458) delete ad684caa/benchmark_data_storage0111-ib.pocstor_64498_object13689/head//22 by client.1049635.0:47892 2014-08-12 17:19:22.468399 2014-08-13 18:06:06.511529 7f5c973a57a0 20 read_log 194215'470 (194215'459) delete d7bd84aa/benchmark_data_storage0111-ib.pocstor_64498_object17483/head//22 by client.1049635.0:51686 2014-08-12 17:19:23.661793 2014-08-13 18:06:06.511541 7f5c973a57a0 20 read_log 194215'471 (194215'460) delete 96d0a4aa/benchmark_data_storage0111-ib.pocstor_64498_object19126/head//22 by client.1049635.0:53329 2014-08-12 17:19:24.209204 2014-08-13 18:06:06.511554 7f5c973a57a0 20 read_log 194215'472 (194215'461) delete 4dd09caa/benchmark_data_storage0111-ib.pocstor_64498_object23563/head//22 by client.1049635.0:57766 2014-08-12 17:19:25.819288 2014-08-13 18:06:06.511573 7f5c973a57a0 20 read_log 194215'473 (194215'462) delete 69484caa/benchmark_data_storage0111-ib.pocstor_64498_object23826/head//22 by client.1049635.0:58029 2014-08-12 17:19:25.904558 2014-08-13 18:06:06.511668 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.511686 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=0 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.511702 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=0 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Initial 0.001804 0 0.000000 2014-08-13 18:06:06.511715 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=0 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.511724 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=0 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.511733 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] log((0'0,194215'473], crt=192888'451) 2014-08-13 18:06:06.511743 7f5c973a57a0 10 osd.4 194487 pgid 22.4df coll 22.4df_head 2014-08-13 18:06:06.511917 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.4df 2014-08-13 18:06:06.511927 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.511956 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4df(unlocked)] enter Initial 2014-08-13 18:06:06.511967 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4df(unlocked)] enter NotTrimming 2014-08-13 18:06:06.512107 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.512222 7f5c973a57a0 20 read_log 191648'7 (0'0) modify c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 by client.273158.0:417 2014-07-30 14:08:50.342842 2014-08-13 18:06:06.512243 7f5c973a57a0 20 read_log 191648'34 (0'0) modify cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 by client.273158.0:1923 2014-07-30 14:08:55.972827 2014-08-13 18:06:06.512259 7f5c973a57a0 20 read_log 191648'37 (0'0) modify 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 by client.273158.0:2195 2014-07-30 14:08:57.141519 2014-08-13 18:06:06.512274 7f5c973a57a0 20 read_log 191648'39 (0'0) modify 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 by client.273158.0:2228 2014-07-30 14:08:57.262622 2014-08-13 18:06:06.512287 7f5c973a57a0 20 read_log 191648'101 (0'0) modify 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 by client.273158.0:6390 2014-07-30 14:09:18.140107 2014-08-13 18:06:06.512299 7f5c973a57a0 20 read_log 191648'125 (0'0) modify d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 by client.273158.0:8184 2014-07-30 14:09:27.078589 2014-08-13 18:06:06.512315 7f5c973a57a0 20 read_log 191648'140 (0'0) modify 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 by client.273158.0:9292 2014-07-30 14:09:32.357356 2014-08-13 18:06:06.512327 7f5c973a57a0 20 read_log 191648'149 (0'0) modify 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 by client.273158.0:10005 2014-07-30 14:09:35.864018 2014-08-13 18:06:06.512339 7f5c973a57a0 20 read_log 191648'191 (0'0) modify 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 by client.273158.0:12482 2014-07-30 14:09:49.937056 2014-08-13 18:06:06.512352 7f5c973a57a0 20 read_log 191648'209 (0'0) modify 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 by client.273158.0:13383 2014-07-30 14:09:54.156483 2014-08-13 18:06:06.512379 7f5c973a57a0 20 read_log 191648'212 (0'0) modify dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 by client.273158.0:13431 2014-07-30 14:09:54.428087 2014-08-13 18:06:06.512395 7f5c973a57a0 20 read_log 191648'318 (0'0) modify a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 by client.273158.0:21241 2014-07-30 14:10:38.249873 2014-08-13 18:06:06.512411 7f5c973a57a0 20 read_log 191648'319 (0'0) modify 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 by client.273158.0:21297 2014-07-30 14:10:38.473365 2014-08-13 18:06:06.512434 7f5c973a57a0 20 read_log 191648'326 (0'0) modify 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 by client.273158.0:21669 2014-07-30 14:10:41.133320 2014-08-13 18:06:06.512449 7f5c973a57a0 20 read_log 191648'329 (0'0) modify 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 by client.273158.0:21822 2014-07-30 14:10:41.737108 2014-08-13 18:06:06.512470 7f5c973a57a0 20 read_log 191727'355 (0'0) modify 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 by client.275175.0:2976 2014-07-30 14:18:44.142832 2014-08-13 18:06:06.512491 7f5c973a57a0 20 read_log 191727'358 (0'0) modify e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 by client.275175.0:3589 2014-07-30 14:18:46.073779 2014-08-13 18:06:06.512504 7f5c973a57a0 20 read_log 191727'360 (0'0) modify 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 by client.275175.0:3807 2014-07-30 14:18:46.759267 2014-08-13 18:06:06.512516 7f5c973a57a0 20 read_log 191727'368 (0'0) modify 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 by client.275175.0:7295 2014-07-30 14:18:57.554106 2014-08-13 18:06:06.512531 7f5c973a57a0 20 read_log 191727'377 (0'0) modify fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 by client.275175.0:9108 2014-07-30 14:19:02.656325 2014-08-13 18:06:06.512546 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 by client.275175.0:9203 2014-07-30 14:19:02.929198 2014-08-13 18:06:06.512566 7f5c973a57a0 20 read_log 191727'379 (0'0) modify b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 by client.275175.0:10101 2014-07-30 14:19:05.461112 2014-08-13 18:06:06.512583 7f5c973a57a0 20 read_log 191727'381 (0'0) modify 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 by client.275175.0:11109 2014-07-30 14:19:08.346682 2014-08-13 18:06:06.512596 7f5c973a57a0 20 read_log 191727'382 (0'0) modify a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 by client.275175.0:13112 2014-07-30 14:19:15.008064 2014-08-13 18:06:06.512608 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 by client.275175.0:14090 2014-07-30 14:19:18.131063 2014-08-13 18:06:06.512621 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 by client.275175.0:17352 2014-07-30 14:19:28.134585 2014-08-13 18:06:06.512633 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 by client.275175.0:20351 2014-07-30 14:19:37.271027 2014-08-13 18:06:06.512645 7f5c973a57a0 20 read_log 191727'406 (0'0) modify a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 by client.275175.0:23326 2014-07-30 14:19:46.949147 2014-08-13 18:06:06.512665 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 by client.275175.0:26110 2014-07-30 14:19:55.174260 2014-08-13 18:06:06.512679 7f5c973a57a0 20 read_log 191727'413 (0'0) modify c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 by client.275175.0:26590 2014-07-30 14:19:56.609822 2014-08-13 18:06:06.512692 7f5c973a57a0 20 read_log 191727'414 (0'0) modify 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 by client.275175.0:27000 2014-07-30 14:19:57.810192 2014-08-13 18:06:06.512704 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 by client.275175.0:27589 2014-07-30 14:19:59.484122 2014-08-13 18:06:06.512724 7f5c973a57a0 20 read_log 191727'418 (0'0) modify d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 by client.275175.0:28171 2014-07-30 14:20:01.508142 2014-08-13 18:06:06.512739 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 by client.275175.0:35530 2014-07-30 14:20:23.524439 2014-08-13 18:06:06.512768 7f5c973a57a0 20 read_log 191727'431 (0'0) modify ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 by client.275175.0:37978 2014-07-30 14:20:31.179032 2014-08-13 18:06:06.512783 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 9efca4df/benchmark_data_storage0111-ib.pocstor_35625_object49/head//22 by client.940330.0:50 2014-08-11 16:29:40.666238 2014-08-13 18:06:06.512796 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 6fbf6cdf/benchmark_data_storage0111-ib.pocstor_35625_object572/head//22 by client.940330.0:573 2014-08-11 16:29:43.011003 2014-08-13 18:06:06.512808 7f5c973a57a0 20 read_log 192888'436 (0'0) modify a4dc74df/benchmark_data_storage0111-ib.pocstor_35625_object5561/head//22 by client.940330.0:5562 2014-08-11 16:30:02.016590 2014-08-13 18:06:06.512821 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 492c9cdf/benchmark_data_storage0111-ib.pocstor_35625_object6312/head//22 by client.940330.0:6313 2014-08-11 16:30:04.683541 2014-08-13 18:06:06.512833 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 9ceb7cdf/benchmark_data_storage0111-ib.pocstor_35625_object7989/head//22 by client.940330.0:7990 2014-08-11 16:30:10.800237 2014-08-13 18:06:06.512845 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 5ca1cdf/benchmark_data_storage0111-ib.pocstor_35625_object10740/head//22 by client.940330.0:10741 2014-08-11 16:30:20.791227 2014-08-13 18:06:06.512857 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 511044df/benchmark_data_storage0111-ib.pocstor_35625_object11089/head//22 by client.940330.0:11090 2014-08-11 16:30:22.145716 2014-08-13 18:06:06.512869 7f5c973a57a0 20 read_log 192888'441 (0'0) modify e58fb4df/benchmark_data_storage0111-ib.pocstor_35625_object15023/head//22 by client.940330.0:15024 2014-08-11 16:30:37.580948 2014-08-13 18:06:06.512881 7f5c973a57a0 20 read_log 192888'442 (0'0) modify e5936cdf/benchmark_data_storage0111-ib.pocstor_35625_object16272/head//22 by client.940330.0:16273 2014-08-11 16:30:42.246174 2014-08-13 18:06:06.512893 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 39a0b4df/benchmark_data_storage0111-ib.pocstor_35625_object17400/head//22 by client.940330.0:17401 2014-08-11 16:30:46.458370 2014-08-13 18:06:06.512905 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 90f5c4df/benchmark_data_storage0111-ib.pocstor_35625_object18527/head//22 by client.940330.0:18528 2014-08-11 16:30:50.868250 2014-08-13 18:06:06.512917 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 61343cdf/benchmark_data_storage0111-ib.pocstor_35625_object18817/head//22 by client.940330.0:18818 2014-08-11 16:30:51.999198 2014-08-13 18:06:06.512929 7f5c973a57a0 20 read_log 192888'446 (0'0) modify d30cd4df/benchmark_data_storage0111-ib.pocstor_35625_object21466/head//22 by client.940330.0:21467 2014-08-11 16:31:02.555551 2014-08-13 18:06:06.512941 7f5c973a57a0 20 read_log 192888'447 (0'0) modify ef929cdf/benchmark_data_storage0111-ib.pocstor_35625_object28670/head//22 by client.940330.0:28671 2014-08-11 16:31:30.329470 2014-08-13 18:06:06.512953 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 6f6e34df/benchmark_data_storage0111-ib.pocstor_35625_object29034/head//22 by client.940330.0:29035 2014-08-11 16:31:31.811822 2014-08-13 18:06:06.512965 7f5c973a57a0 20 read_log 192888'449 (0'0) modify 637db4df/benchmark_data_storage0111-ib.pocstor_35625_object30208/head//22 by client.940330.0:30209 2014-08-11 16:31:36.236705 2014-08-13 18:06:06.512985 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 791f7cdf/benchmark_data_storage0111-ib.pocstor_35625_object31220/head//22 by client.940330.0:31221 2014-08-11 16:31:40.095813 2014-08-13 18:06:06.513000 7f5c973a57a0 20 read_log 192888'451 (192888'434) delete 9efca4df/benchmark_data_storage0111-ib.pocstor_35625_object49/head//22 by client.940330.0:31357 2014-08-11 16:31:41.459217 2014-08-13 18:06:06.513017 7f5c973a57a0 20 read_log 192888'452 (192888'435) delete 6fbf6cdf/benchmark_data_storage0111-ib.pocstor_35625_object572/head//22 by client.940330.0:31880 2014-08-11 16:31:41.644861 2014-08-13 18:06:06.513032 7f5c973a57a0 20 read_log 192888'453 (192888'436) delete a4dc74df/benchmark_data_storage0111-ib.pocstor_35625_object5561/head//22 by client.940330.0:36869 2014-08-11 16:31:42.959155 2014-08-13 18:06:06.513045 7f5c973a57a0 20 read_log 192888'454 (192888'437) delete 492c9cdf/benchmark_data_storage0111-ib.pocstor_35625_object6312/head//22 by client.940330.0:37620 2014-08-11 16:31:43.160463 2014-08-13 18:06:06.513059 7f5c973a57a0 20 read_log 192888'455 (192888'438) delete 9ceb7cdf/benchmark_data_storage0111-ib.pocstor_35625_object7989/head//22 by client.940330.0:39297 2014-08-11 16:31:43.587256 2014-08-13 18:06:06.513072 7f5c973a57a0 20 read_log 192888'456 (192888'439) delete 5ca1cdf/benchmark_data_storage0111-ib.pocstor_35625_object10740/head//22 by client.940330.0:42048 2014-08-11 16:31:44.324738 2014-08-13 18:06:06.513086 7f5c973a57a0 20 read_log 192888'457 (192888'440) delete 511044df/benchmark_data_storage0111-ib.pocstor_35625_object11089/head//22 by client.940330.0:42397 2014-08-11 16:31:44.423895 2014-08-13 18:06:06.513100 7f5c973a57a0 20 read_log 192888'458 (192888'441) delete e58fb4df/benchmark_data_storage0111-ib.pocstor_35625_object15023/head//22 by client.940330.0:46331 2014-08-11 16:31:45.463375 2014-08-13 18:06:06.513116 7f5c973a57a0 20 read_log 192888'459 (192888'442) delete e5936cdf/benchmark_data_storage0111-ib.pocstor_35625_object16272/head//22 by client.940330.0:47580 2014-08-11 16:31:45.796683 2014-08-13 18:06:06.513131 7f5c973a57a0 20 read_log 192888'460 (192888'443) delete 39a0b4df/benchmark_data_storage0111-ib.pocstor_35625_object17400/head//22 by client.940330.0:48708 2014-08-11 16:31:46.101825 2014-08-13 18:06:06.513151 7f5c973a57a0 20 read_log 192888'461 (192888'444) delete 90f5c4df/benchmark_data_storage0111-ib.pocstor_35625_object18527/head//22 by client.940330.0:49835 2014-08-11 16:31:46.453477 2014-08-13 18:06:06.513166 7f5c973a57a0 20 read_log 192888'462 (192888'445) delete 61343cdf/benchmark_data_storage0111-ib.pocstor_35625_object18817/head//22 by client.940330.0:50125 2014-08-11 16:31:46.532344 2014-08-13 18:06:06.513179 7f5c973a57a0 20 read_log 192888'463 (192888'446) delete d30cd4df/benchmark_data_storage0111-ib.pocstor_35625_object21466/head//22 by client.940330.0:52774 2014-08-11 16:31:47.285847 2014-08-13 18:06:06.513191 7f5c973a57a0 20 read_log 192888'464 (192888'447) delete ef929cdf/benchmark_data_storage0111-ib.pocstor_35625_object28670/head//22 by client.940330.0:59978 2014-08-11 16:31:49.382148 2014-08-13 18:06:06.513204 7f5c973a57a0 20 read_log 192888'465 (192888'448) delete 6f6e34df/benchmark_data_storage0111-ib.pocstor_35625_object29034/head//22 by client.940330.0:60342 2014-08-11 16:31:49.481296 2014-08-13 18:06:06.513216 7f5c973a57a0 20 read_log 192888'466 (192888'449) delete 637db4df/benchmark_data_storage0111-ib.pocstor_35625_object30208/head//22 by client.940330.0:61516 2014-08-11 16:31:49.816975 2014-08-13 18:06:06.513231 7f5c973a57a0 20 read_log 192888'467 (192888'450) delete 791f7cdf/benchmark_data_storage0111-ib.pocstor_35625_object31220/head//22 by client.940330.0:62528 2014-08-11 16:31:50.127007 2014-08-13 18:06:06.513244 7f5c973a57a0 20 read_log 194215'468 (0'0) modify fd3654df/benchmark_data_storage0111-ib.pocstor_64498_object982/head//22 by client.1049635.0:983 2014-08-12 17:17:20.297293 2014-08-13 18:06:06.513288 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 55040cdf/benchmark_data_storage0111-ib.pocstor_64498_object1167/head//22 by client.1049635.0:1168 2014-08-12 17:17:20.953796 2014-08-13 18:06:06.513309 7f5c973a57a0 20 read_log 194215'470 (0'0) modify d01aa4df/benchmark_data_storage0111-ib.pocstor_64498_object3257/head//22 by client.1049635.0:3258 2014-08-12 17:17:27.694100 2014-08-13 18:06:06.513322 7f5c973a57a0 20 read_log 194215'471 (0'0) modify c581ecdf/benchmark_data_storage0111-ib.pocstor_64498_object5466/head//22 by client.1049635.0:5467 2014-08-12 17:17:34.475010 2014-08-13 18:06:06.513335 7f5c973a57a0 20 read_log 194215'472 (0'0) modify e54744df/benchmark_data_storage0111-ib.pocstor_64498_object6134/head//22 by client.1049635.0:6135 2014-08-12 17:17:36.491274 2014-08-13 18:06:06.513350 7f5c973a57a0 20 read_log 194215'473 (0'0) modify f7e44df/benchmark_data_storage0111-ib.pocstor_64498_object7395/head//22 by client.1049635.0:7396 2014-08-12 17:17:40.239530 2014-08-13 18:06:06.513365 7f5c973a57a0 20 read_log 194215'474 (0'0) modify f4af5cdf/benchmark_data_storage0111-ib.pocstor_64498_object13769/head//22 by client.1049635.0:13770 2014-08-12 17:17:59.848404 2014-08-13 18:06:06.513377 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 30962cdf/benchmark_data_storage0111-ib.pocstor_64498_object15259/head//22 by client.1049635.0:15260 2014-08-12 17:18:04.831917 2014-08-13 18:06:06.513392 7f5c973a57a0 20 read_log 194215'476 (0'0) modify fcb474df/benchmark_data_storage0111-ib.pocstor_64498_object15865/head//22 by client.1049635.0:15866 2014-08-12 17:18:07.047647 2014-08-13 18:06:06.513412 7f5c973a57a0 20 read_log 194215'477 (0'0) modify f66fdcdf/benchmark_data_storage0111-ib.pocstor_64498_object19484/head//22 by client.1049635.0:19485 2014-08-12 17:18:20.849600 2014-08-13 18:06:06.513434 7f5c973a57a0 20 read_log 194215'478 (0'0) modify eed494df/benchmark_data_storage0111-ib.pocstor_64498_object22229/head//22 by client.1049635.0:22230 2014-08-12 17:18:31.197050 2014-08-13 18:06:06.513452 7f5c973a57a0 20 read_log 194215'479 (0'0) modify c6719cdf/benchmark_data_storage0111-ib.pocstor_64498_object25872/head//22 by client.1049635.0:25873 2014-08-12 17:18:44.948569 2014-08-13 18:06:06.513467 7f5c973a57a0 20 read_log 194215'480 (0'0) modify a9cf34df/benchmark_data_storage0111-ib.pocstor_64498_object30446/head//22 by client.1049635.0:30447 2014-08-12 17:19:02.297618 2014-08-13 18:06:06.513479 7f5c973a57a0 20 read_log 194215'481 (0'0) modify ff0264df/benchmark_data_storage0111-ib.pocstor_64498_object31710/head//22 by client.1049635.0:31711 2014-08-12 17:19:07.451430 2014-08-13 18:06:06.513499 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 4eab34df/benchmark_data_storage0111-ib.pocstor_64498_object33007/head//22 by client.1049635.0:33008 2014-08-12 17:19:12.417594 2014-08-13 18:06:06.513515 7f5c973a57a0 20 read_log 194215'483 (194215'468) delete fd3654df/benchmark_data_storage0111-ib.pocstor_64498_object982/head//22 by client.1049635.0:35185 2014-08-12 17:19:18.314503 2014-08-13 18:06:06.513527 7f5c973a57a0 20 read_log 194215'484 (194215'469) delete 55040cdf/benchmark_data_storage0111-ib.pocstor_64498_object1167/head//22 by client.1049635.0:35370 2014-08-12 17:19:18.374305 2014-08-13 18:06:06.513550 7f5c973a57a0 20 read_log 194215'485 (194215'470) delete d01aa4df/benchmark_data_storage0111-ib.pocstor_64498_object3257/head//22 by client.1049635.0:37460 2014-08-12 17:19:19.046850 2014-08-13 18:06:06.513566 7f5c973a57a0 20 read_log 194215'486 (194215'471) delete c581ecdf/benchmark_data_storage0111-ib.pocstor_64498_object5466/head//22 by client.1049635.0:39669 2014-08-12 17:19:19.759466 2014-08-13 18:06:06.513579 7f5c973a57a0 20 read_log 194215'487 (194215'472) delete e54744df/benchmark_data_storage0111-ib.pocstor_64498_object6134/head//22 by client.1049635.0:40337 2014-08-12 17:19:20.001142 2014-08-13 18:06:06.513594 7f5c973a57a0 20 read_log 194215'488 (194215'473) delete f7e44df/benchmark_data_storage0111-ib.pocstor_64498_object7395/head//22 by client.1049635.0:41598 2014-08-12 17:19:20.415453 2014-08-13 18:06:06.513610 7f5c973a57a0 20 read_log 194215'489 (194215'474) delete f4af5cdf/benchmark_data_storage0111-ib.pocstor_64498_object13769/head//22 by client.1049635.0:47972 2014-08-12 17:19:22.494377 2014-08-13 18:06:06.513623 7f5c973a57a0 20 read_log 194215'490 (194215'475) delete 30962cdf/benchmark_data_storage0111-ib.pocstor_64498_object15259/head//22 by client.1049635.0:49462 2014-08-12 17:19:22.956087 2014-08-13 18:06:06.513642 7f5c973a57a0 20 read_log 194215'491 (194215'476) delete fcb474df/benchmark_data_storage0111-ib.pocstor_64498_object15865/head//22 by client.1049635.0:50068 2014-08-12 17:19:23.145147 2014-08-13 18:06:06.513655 7f5c973a57a0 20 read_log 194215'492 (194215'477) delete f66fdcdf/benchmark_data_storage0111-ib.pocstor_64498_object19484/head//22 by client.1049635.0:53687 2014-08-12 17:19:24.344397 2014-08-13 18:06:06.513667 7f5c973a57a0 20 read_log 194215'493 (194215'478) delete eed494df/benchmark_data_storage0111-ib.pocstor_64498_object22229/head//22 by client.1049635.0:56432 2014-08-12 17:19:25.317710 2014-08-13 18:06:06.513679 7f5c973a57a0 20 read_log 194215'494 (194215'479) delete c6719cdf/benchmark_data_storage0111-ib.pocstor_64498_object25872/head//22 by client.1049635.0:60075 2014-08-12 17:19:26.616397 2014-08-13 18:06:06.513692 7f5c973a57a0 20 read_log 194215'495 (194215'480) delete a9cf34df/benchmark_data_storage0111-ib.pocstor_64498_object30446/head//22 by client.1049635.0:64649 2014-08-12 17:19:28.189539 2014-08-13 18:06:06.513704 7f5c973a57a0 20 read_log 194215'496 (194215'481) delete ff0264df/benchmark_data_storage0111-ib.pocstor_64498_object31710/head//22 by client.1049635.0:65913 2014-08-12 17:19:28.644050 2014-08-13 18:06:06.513716 7f5c973a57a0 20 read_log 194215'497 (194215'482) delete 4eab34df/benchmark_data_storage0111-ib.pocstor_64498_object33007/head//22 by client.1049635.0:67210 2014-08-12 17:19:29.118303 2014-08-13 18:06:06.513811 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.513830 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=0 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.513855 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=0 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001900 0 0.000000 2014-08-13 18:06:06.513868 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=0 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.513877 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=0 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.513886 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'497], crt=191727'433) 2014-08-13 18:06:06.513896 7f5c973a57a0 10 osd.4 194487 pgid 22.4fe coll 22.4fe_head 2014-08-13 18:06:06.514049 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.4fe 2014-08-13 18:06:06.514059 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.514085 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4fe(unlocked)] enter Initial 2014-08-13 18:06:06.514101 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4fe(unlocked)] enter NotTrimming 2014-08-13 18:06:06.514259 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.514410 7f5c973a57a0 20 read_log 191648'4 (0'0) modify 58b384fe/benchmark_data_storage0111-ib.pocstor_40745_object471/head//22 by client.273158.0:472 2014-07-30 14:08:50.543500 2014-08-13 18:06:06.514432 7f5c973a57a0 20 read_log 191648'5 (0'0) modify f07e4fe/benchmark_data_storage0111-ib.pocstor_40745_object524/head//22 by client.273158.0:525 2014-07-30 14:08:50.737009 2014-08-13 18:06:06.514444 7f5c973a57a0 20 read_log 191648'6 (0'0) modify eb685cfe/benchmark_data_storage0111-ib.pocstor_40745_object531/head//22 by client.273158.0:532 2014-07-30 14:08:50.763301 2014-08-13 18:06:06.514457 7f5c973a57a0 20 read_log 191648'8 (0'0) modify 1b7684fe/benchmark_data_storage0111-ib.pocstor_40745_object823/head//22 by client.273158.0:824 2014-07-30 14:08:51.836025 2014-08-13 18:06:06.514469 7f5c973a57a0 20 read_log 191648'16 (0'0) modify 9872acfe/benchmark_data_storage0111-ib.pocstor_40745_object1221/head//22 by client.273158.0:1222 2014-07-30 14:08:53.283920 2014-08-13 18:06:06.514484 7f5c973a57a0 20 read_log 191648'51 (0'0) modify 2e57a4fe/benchmark_data_storage0111-ib.pocstor_40745_object3406/head//22 by client.273158.0:3407 2014-07-30 14:09:03.260646 2014-08-13 18:06:06.514496 7f5c973a57a0 20 read_log 191648'125 (0'0) modify 6afcdcfe/benchmark_data_storage0111-ib.pocstor_40745_object8785/head//22 by client.273158.0:8786 2014-07-30 14:09:30.325059 2014-08-13 18:06:06.514509 7f5c973a57a0 20 read_log 191648'150 (0'0) modify dad8f4fe/benchmark_data_storage0111-ib.pocstor_40745_object10683/head//22 by client.273158.0:10684 2014-07-30 14:09:39.428634 2014-08-13 18:06:06.514521 7f5c973a57a0 20 read_log 191648'214 (0'0) modify 78d17cfe/benchmark_data_storage0111-ib.pocstor_40745_object14902/head//22 by client.273158.0:14903 2014-07-30 14:10:02.317253 2014-08-13 18:06:06.514533 7f5c973a57a0 20 read_log 191648'224 (0'0) modify be8f44fe/benchmark_data_storage0111-ib.pocstor_40745_object15617/head//22 by client.273158.0:15618 2014-07-30 14:10:05.362651 2014-08-13 18:06:06.514546 7f5c973a57a0 20 read_log 191648'304 (0'0) modify 5a7bbcfe/benchmark_data_storage0111-ib.pocstor_40745_object19788/head//22 by client.273158.0:19789 2014-07-30 14:10:30.509366 2014-08-13 18:06:06.514558 7f5c973a57a0 20 read_log 191648'323 (0'0) modify cc265cfe/benchmark_data_storage0111-ib.pocstor_40745_object20820/head//22 by client.273158.0:20821 2014-07-30 14:10:36.155940 2014-08-13 18:06:06.514570 7f5c973a57a0 20 read_log 191648'353 (0'0) modify 294a84fe/benchmark_data_storage0111-ib.pocstor_40745_object22653/head//22 by client.273158.0:22654 2014-07-30 14:10:47.690078 2014-08-13 18:06:06.514582 7f5c973a57a0 20 read_log 191727'357 (0'0) modify c2c2f4fe/benchmark_data_storage0111-ib.pocstor_51285_object1659/head//22 by client.275175.0:1660 2014-07-30 14:18:39.768504 2014-08-13 18:06:06.514594 7f5c973a57a0 20 read_log 191727'359 (0'0) modify 3363f4fe/benchmark_data_storage0111-ib.pocstor_51285_object2105/head//22 by client.275175.0:2106 2014-07-30 14:18:41.239850 2014-08-13 18:06:06.514606 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 4b0054fe/benchmark_data_storage0111-ib.pocstor_51285_object2330/head//22 by client.275175.0:2331 2014-07-30 14:18:41.977090 2014-08-13 18:06:06.514618 7f5c973a57a0 20 read_log 191727'369 (0'0) modify 7f6154fe/benchmark_data_storage0111-ib.pocstor_51285_object3571/head//22 by client.275175.0:3572 2014-07-30 14:18:46.022522 2014-08-13 18:06:06.514630 7f5c973a57a0 20 read_log 191727'371 (0'0) modify b62524fe/benchmark_data_storage0111-ib.pocstor_51285_object4359/head//22 by client.275175.0:4360 2014-07-30 14:18:48.436460 2014-08-13 18:06:06.514642 7f5c973a57a0 20 read_log 191727'379 (0'0) modify 77e26cfe/benchmark_data_storage0111-ib.pocstor_51285_object7964/head//22 by client.275175.0:7965 2014-07-30 14:18:59.424286 2014-08-13 18:06:06.514662 7f5c973a57a0 20 read_log 191727'380 (0'0) modify 57a63cfe/benchmark_data_storage0111-ib.pocstor_51285_object8099/head//22 by client.275175.0:8100 2014-07-30 14:18:59.800414 2014-08-13 18:06:06.514674 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 84e90cfe/benchmark_data_storage0111-ib.pocstor_51285_object10730/head//22 by client.275175.0:10731 2014-07-30 14:19:07.287570 2014-08-13 18:06:06.514686 7f5c973a57a0 20 read_log 191727'390 (0'0) modify cd05e4fe/benchmark_data_storage0111-ib.pocstor_51285_object12138/head//22 by client.275175.0:12139 2014-07-30 14:19:12.026464 2014-08-13 18:06:06.514698 7f5c973a57a0 20 read_log 191727'396 (0'0) modify a1374fe/benchmark_data_storage0111-ib.pocstor_51285_object18591/head//22 by client.275175.0:18592 2014-07-30 14:19:31.743005 2014-08-13 18:06:06.514710 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 354eacfe/benchmark_data_storage0111-ib.pocstor_51285_object19372/head//22 by client.275175.0:19373 2014-07-30 14:19:34.421185 2014-08-13 18:06:06.514722 7f5c973a57a0 20 read_log 191727'399 (0'0) modify 206d04fe/benchmark_data_storage0111-ib.pocstor_51285_object19678/head//22 by client.275175.0:19679 2014-07-30 14:19:35.269935 2014-08-13 18:06:06.514754 7f5c973a57a0 20 read_log 191727'404 (0'0) modify ac2bfcfe/benchmark_data_storage0111-ib.pocstor_51285_object21140/head//22 by client.275175.0:21141 2014-07-30 14:19:39.528982 2014-08-13 18:06:06.514770 7f5c973a57a0 20 read_log 191727'405 (0'0) modify 63cd54fe/benchmark_data_storage0111-ib.pocstor_51285_object21538/head//22 by client.275175.0:21539 2014-07-30 14:19:40.804188 2014-08-13 18:06:06.514782 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 2e27ecfe/benchmark_data_storage0111-ib.pocstor_51285_object23873/head//22 by client.275175.0:23874 2014-07-30 14:19:48.479183 2014-08-13 18:06:06.514795 7f5c973a57a0 20 read_log 191727'408 (0'0) modify e9c5d4fe/benchmark_data_storage0111-ib.pocstor_51285_object25026/head//22 by client.275175.0:25027 2014-07-30 14:19:51.760474 2014-08-13 18:06:06.514807 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 287344fe/benchmark_data_storage0111-ib.pocstor_51285_object25412/head//22 by client.275175.0:25413 2014-07-30 14:19:53.016988 2014-08-13 18:06:06.514819 7f5c973a57a0 20 read_log 191727'414 (0'0) modify a21dc4fe/benchmark_data_storage0111-ib.pocstor_51285_object26863/head//22 by client.275175.0:26864 2014-07-30 14:19:57.426501 2014-08-13 18:06:06.514831 7f5c973a57a0 20 read_log 191727'415 (0'0) modify f42634fe/benchmark_data_storage0111-ib.pocstor_51285_object27351/head//22 by client.275175.0:27352 2014-07-30 14:19:58.810621 2014-08-13 18:06:06.514843 7f5c973a57a0 20 read_log 191727'416 (0'0) modify f39decfe/benchmark_data_storage0111-ib.pocstor_51285_object28444/head//22 by client.275175.0:28445 2014-07-30 14:20:02.264942 2014-08-13 18:06:06.514855 7f5c973a57a0 20 read_log 191727'425 (0'0) modify cc6a34fe/benchmark_data_storage0111-ib.pocstor_51285_object34347/head//22 by client.275175.0:34348 2014-07-30 14:20:20.075351 2014-08-13 18:06:06.514869 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 19dd7cfe/benchmark_data_storage0111-ib.pocstor_51285_object38492/head//22 by client.275175.0:38493 2014-07-30 14:20:32.645282 2014-08-13 18:06:06.514881 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 2788bcfe/benchmark_data_storage0111-ib.pocstor_35625_object897/head//22 by client.940330.0:898 2014-08-11 16:29:44.300577 2014-08-13 18:06:06.514896 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 5d77cfe/benchmark_data_storage0111-ib.pocstor_35625_object2367/head//22 by client.940330.0:2368 2014-08-11 16:29:49.879704 2014-08-13 18:06:06.514908 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 9f4e7cfe/benchmark_data_storage0111-ib.pocstor_35625_object2460/head//22 by client.940330.0:2461 2014-08-11 16:29:50.284171 2014-08-13 18:06:06.514920 7f5c973a57a0 20 read_log 192888'433 (0'0) modify a463d4fe/benchmark_data_storage0111-ib.pocstor_35625_object12266/head//22 by client.940330.0:12267 2014-08-11 16:30:27.008081 2014-08-13 18:06:06.514942 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 856b94fe/benchmark_data_storage0111-ib.pocstor_35625_object13173/head//22 by client.940330.0:13174 2014-08-11 16:30:30.547326 2014-08-13 18:06:06.514957 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 639c1cfe/benchmark_data_storage0111-ib.pocstor_35625_object15473/head//22 by client.940330.0:15474 2014-08-11 16:30:39.265453 2014-08-13 18:06:06.514969 7f5c973a57a0 20 read_log 192888'436 (0'0) modify ea2dccfe/benchmark_data_storage0111-ib.pocstor_35625_object15539/head//22 by client.940330.0:15540 2014-08-11 16:30:39.502579 2014-08-13 18:06:06.514983 7f5c973a57a0 20 read_log 192888'437 (0'0) modify d3f8ccfe/benchmark_data_storage0111-ib.pocstor_35625_object18332/head//22 by client.940330.0:18333 2014-08-11 16:30:50.045756 2014-08-13 18:06:06.514996 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 7f25b4fe/benchmark_data_storage0111-ib.pocstor_35625_object18376/head//22 by client.940330.0:18377 2014-08-11 16:30:50.212848 2014-08-13 18:06:06.515009 7f5c973a57a0 20 read_log 192888'439 (0'0) modify a96f0cfe/benchmark_data_storage0111-ib.pocstor_35625_object24128/head//22 by client.940330.0:24129 2014-08-11 16:31:12.614270 2014-08-13 18:06:06.515022 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 875b14fe/benchmark_data_storage0111-ib.pocstor_35625_object24796/head//22 by client.940330.0:24797 2014-08-11 16:31:15.079797 2014-08-13 18:06:06.515036 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 1c68c4fe/benchmark_data_storage0111-ib.pocstor_35625_object28381/head//22 by client.940330.0:28382 2014-08-11 16:31:29.190103 2014-08-13 18:06:06.515048 7f5c973a57a0 20 read_log 192888'442 (0'0) modify fac364fe/benchmark_data_storage0111-ib.pocstor_35625_object28439/head//22 by client.940330.0:28440 2014-08-11 16:31:29.421054 2014-08-13 18:06:06.515062 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 74f5e4fe/benchmark_data_storage0111-ib.pocstor_35625_object29630/head//22 by client.940330.0:29631 2014-08-11 16:31:34.062030 2014-08-13 18:06:06.515074 7f5c973a57a0 20 read_log 192888'444 (0'0) modify c1f7dcfe/benchmark_data_storage0111-ib.pocstor_35625_object30504/head//22 by client.940330.0:30505 2014-08-11 16:31:37.368485 2014-08-13 18:06:06.515088 7f5c973a57a0 20 read_log 192888'445 (192888'430) delete 2788bcfe/benchmark_data_storage0111-ib.pocstor_35625_object897/head//22 by client.940330.0:32205 2014-08-11 16:31:41.724590 2014-08-13 18:06:06.515102 7f5c973a57a0 20 read_log 192888'446 (192888'431) delete 5d77cfe/benchmark_data_storage0111-ib.pocstor_35625_object2367/head//22 by client.940330.0:33675 2014-08-11 16:31:42.085861 2014-08-13 18:06:06.515116 7f5c973a57a0 20 read_log 192888'447 (192888'432) delete 9f4e7cfe/benchmark_data_storage0111-ib.pocstor_35625_object2460/head//22 by client.940330.0:33768 2014-08-11 16:31:42.110540 2014-08-13 18:06:06.515130 7f5c973a57a0 20 read_log 192888'448 (192888'433) delete a463d4fe/benchmark_data_storage0111-ib.pocstor_35625_object12266/head//22 by client.940330.0:43574 2014-08-11 16:31:44.724608 2014-08-13 18:06:06.515142 7f5c973a57a0 20 read_log 192888'449 (192888'434) delete 856b94fe/benchmark_data_storage0111-ib.pocstor_35625_object13173/head//22 by client.940330.0:44481 2014-08-11 16:31:44.967098 2014-08-13 18:06:06.515163 7f5c973a57a0 20 read_log 192888'450 (192888'435) delete 639c1cfe/benchmark_data_storage0111-ib.pocstor_35625_object15473/head//22 by client.940330.0:46781 2014-08-11 16:31:45.584355 2014-08-13 18:06:06.515175 7f5c973a57a0 20 read_log 192888'451 (192888'436) delete ea2dccfe/benchmark_data_storage0111-ib.pocstor_35625_object15539/head//22 by client.940330.0:46847 2014-08-11 16:31:45.600863 2014-08-13 18:06:06.515188 7f5c973a57a0 20 read_log 192888'452 (192888'437) delete d3f8ccfe/benchmark_data_storage0111-ib.pocstor_35625_object18332/head//22 by client.940330.0:49640 2014-08-11 16:31:46.402516 2014-08-13 18:06:06.515200 7f5c973a57a0 20 read_log 192888'453 (192888'438) delete 7f25b4fe/benchmark_data_storage0111-ib.pocstor_35625_object18376/head//22 by client.940330.0:49684 2014-08-11 16:31:46.413892 2014-08-13 18:06:06.515237 7f5c973a57a0 20 read_log 192888'454 (192888'439) delete a96f0cfe/benchmark_data_storage0111-ib.pocstor_35625_object24128/head//22 by client.940330.0:55436 2014-08-11 16:31:48.080317 2014-08-13 18:06:06.515254 7f5c973a57a0 20 read_log 192888'455 (192888'440) delete 875b14fe/benchmark_data_storage0111-ib.pocstor_35625_object24796/head//22 by client.940330.0:56104 2014-08-11 16:31:48.273488 2014-08-13 18:06:06.515269 7f5c973a57a0 20 read_log 192888'456 (192888'441) delete 1c68c4fe/benchmark_data_storage0111-ib.pocstor_35625_object28381/head//22 by client.940330.0:59689 2014-08-11 16:31:49.305566 2014-08-13 18:06:06.515284 7f5c973a57a0 20 read_log 192888'457 (192888'442) delete fac364fe/benchmark_data_storage0111-ib.pocstor_35625_object28439/head//22 by client.940330.0:59747 2014-08-11 16:31:49.322330 2014-08-13 18:06:06.515296 7f5c973a57a0 20 read_log 192888'458 (192888'443) delete 74f5e4fe/benchmark_data_storage0111-ib.pocstor_35625_object29630/head//22 by client.940330.0:60938 2014-08-11 16:31:49.652241 2014-08-13 18:06:06.515309 7f5c973a57a0 20 read_log 192888'459 (192888'444) delete c1f7dcfe/benchmark_data_storage0111-ib.pocstor_35625_object30504/head//22 by client.940330.0:61812 2014-08-11 16:31:49.903415 2014-08-13 18:06:06.515326 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 41e73cfe/benchmark_data_storage0111-ib.pocstor_64498_object1675/head//22 by client.1049635.0:1676 2014-08-12 17:17:22.506625 2014-08-13 18:06:06.515339 7f5c973a57a0 20 read_log 194215'461 (0'0) modify d87c44fe/benchmark_data_storage0111-ib.pocstor_64498_object2418/head//22 by client.1049635.0:2419 2014-08-12 17:17:24.991169 2014-08-13 18:06:06.515352 7f5c973a57a0 20 read_log 194215'462 (0'0) modify f3738cfe/benchmark_data_storage0111-ib.pocstor_64498_object3262/head//22 by client.1049635.0:3263 2014-08-12 17:17:27.710487 2014-08-13 18:06:06.515368 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 87f1dcfe/benchmark_data_storage0111-ib.pocstor_64498_object4820/head//22 by client.1049635.0:4821 2014-08-12 17:17:32.378595 2014-08-13 18:06:06.515382 7f5c973a57a0 20 read_log 194215'464 (0'0) modify f77c3cfe/benchmark_data_storage0111-ib.pocstor_64498_object5164/head//22 by client.1049635.0:5165 2014-08-12 17:17:33.421968 2014-08-13 18:06:06.515408 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 20153cfe/benchmark_data_storage0111-ib.pocstor_64498_object6002/head//22 by client.1049635.0:6003 2014-08-12 17:17:36.083392 2014-08-13 18:06:06.515427 7f5c973a57a0 20 read_log 194215'466 (0'0) modify ec2644fe/benchmark_data_storage0111-ib.pocstor_64498_object11299/head//22 by client.1049635.0:11300 2014-08-12 17:17:52.111746 2014-08-13 18:06:06.515440 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 1acf5cfe/benchmark_data_storage0111-ib.pocstor_64498_object13628/head//22 by client.1049635.0:13629 2014-08-12 17:17:59.445232 2014-08-13 18:06:06.515462 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 903b14fe/benchmark_data_storage0111-ib.pocstor_64498_object15571/head//22 by client.1049635.0:15572 2014-08-12 17:18:05.996779 2014-08-13 18:06:06.515476 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 63ba7cfe/benchmark_data_storage0111-ib.pocstor_64498_object15746/head//22 by client.1049635.0:15747 2014-08-12 17:18:06.614892 2014-08-13 18:06:06.515488 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 5606ecfe/benchmark_data_storage0111-ib.pocstor_64498_object16345/head//22 by client.1049635.0:16346 2014-08-12 17:18:08.871425 2014-08-13 18:06:06.515510 7f5c973a57a0 20 read_log 194215'471 (0'0) modify dbfe6cfe/benchmark_data_storage0111-ib.pocstor_64498_object19601/head//22 by client.1049635.0:19602 2014-08-12 17:18:21.288159 2014-08-13 18:06:06.515523 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 7bdb1cfe/benchmark_data_storage0111-ib.pocstor_64498_object22759/head//22 by client.1049635.0:22760 2014-08-12 17:18:33.240760 2014-08-13 18:06:06.515550 7f5c973a57a0 20 read_log 194215'473 (0'0) modify d3aec4fe/benchmark_data_storage0111-ib.pocstor_64498_object25801/head//22 by client.1049635.0:25802 2014-08-12 17:18:44.682417 2014-08-13 18:06:06.515567 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 89bc84fe/benchmark_data_storage0111-ib.pocstor_64498_object26010/head//22 by client.1049635.0:26011 2014-08-12 17:18:45.462098 2014-08-13 18:06:06.515579 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 33f0ccfe/benchmark_data_storage0111-ib.pocstor_64498_object27370/head//22 by client.1049635.0:27371 2014-08-12 17:18:50.535886 2014-08-13 18:06:06.515613 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 8e498cfe/benchmark_data_storage0111-ib.pocstor_64498_object27382/head//22 by client.1049635.0:27383 2014-08-12 17:18:50.581043 2014-08-13 18:06:06.515630 7f5c973a57a0 20 read_log 194215'477 (0'0) modify d172ecfe/benchmark_data_storage0111-ib.pocstor_64498_object31545/head//22 by client.1049635.0:31546 2014-08-12 17:19:06.803234 2014-08-13 18:06:06.515642 7f5c973a57a0 20 read_log 194215'478 (194215'460) delete 41e73cfe/benchmark_data_storage0111-ib.pocstor_64498_object1675/head//22 by client.1049635.0:35878 2014-08-12 17:19:18.544663 2014-08-13 18:06:06.515655 7f5c973a57a0 20 read_log 194215'479 (194215'461) delete d87c44fe/benchmark_data_storage0111-ib.pocstor_64498_object2418/head//22 by client.1049635.0:36621 2014-08-12 17:19:18.773378 2014-08-13 18:06:06.515667 7f5c973a57a0 20 read_log 194215'480 (194215'462) delete f3738cfe/benchmark_data_storage0111-ib.pocstor_64498_object3262/head//22 by client.1049635.0:37465 2014-08-12 17:19:19.047973 2014-08-13 18:06:06.515691 7f5c973a57a0 20 read_log 194215'481 (194215'463) delete 87f1dcfe/benchmark_data_storage0111-ib.pocstor_64498_object4820/head//22 by client.1049635.0:39023 2014-08-12 17:19:19.542409 2014-08-13 18:06:06.515704 7f5c973a57a0 20 read_log 194215'482 (194215'464) delete f77c3cfe/benchmark_data_storage0111-ib.pocstor_64498_object5164/head//22 by client.1049635.0:39367 2014-08-12 17:19:19.660564 2014-08-13 18:06:06.515716 7f5c973a57a0 20 read_log 194215'483 (194215'465) delete 20153cfe/benchmark_data_storage0111-ib.pocstor_64498_object6002/head//22 by client.1049635.0:40205 2014-08-12 17:19:19.957174 2014-08-13 18:06:06.515729 7f5c973a57a0 20 read_log 194215'484 (194215'466) delete ec2644fe/benchmark_data_storage0111-ib.pocstor_64498_object11299/head//22 by client.1049635.0:45502 2014-08-12 17:19:21.672810 2014-08-13 18:06:06.515741 7f5c973a57a0 20 read_log 194215'485 (194215'467) delete 1acf5cfe/benchmark_data_storage0111-ib.pocstor_64498_object13628/head//22 by client.1049635.0:47831 2014-08-12 17:19:22.446745 2014-08-13 18:06:06.515754 7f5c973a57a0 20 read_log 194215'486 (194215'468) delete 903b14fe/benchmark_data_storage0111-ib.pocstor_64498_object15571/head//22 by client.1049635.0:49774 2014-08-12 17:19:23.055648 2014-08-13 18:06:06.515768 7f5c973a57a0 20 read_log 194215'487 (194215'469) delete 63ba7cfe/benchmark_data_storage0111-ib.pocstor_64498_object15746/head//22 by client.1049635.0:49949 2014-08-12 17:19:23.106847 2014-08-13 18:06:06.515780 7f5c973a57a0 20 read_log 194215'488 (194215'470) delete 5606ecfe/benchmark_data_storage0111-ib.pocstor_64498_object16345/head//22 by client.1049635.0:50548 2014-08-12 17:19:23.299188 2014-08-13 18:06:06.515793 7f5c973a57a0 20 read_log 194215'489 (194215'471) delete dbfe6cfe/benchmark_data_storage0111-ib.pocstor_64498_object19601/head//22 by client.1049635.0:53804 2014-08-12 17:19:24.386105 2014-08-13 18:06:06.515805 7f5c973a57a0 20 read_log 194215'490 (194215'472) delete 7bdb1cfe/benchmark_data_storage0111-ib.pocstor_64498_object22759/head//22 by client.1049635.0:56962 2014-08-12 17:19:25.523527 2014-08-13 18:06:06.515818 7f5c973a57a0 20 read_log 194215'491 (194215'473) delete d3aec4fe/benchmark_data_storage0111-ib.pocstor_64498_object25801/head//22 by client.1049635.0:60004 2014-08-12 17:19:26.591970 2014-08-13 18:06:06.515830 7f5c973a57a0 20 read_log 194215'492 (194215'474) delete 89bc84fe/benchmark_data_storage0111-ib.pocstor_64498_object26010/head//22 by client.1049635.0:60213 2014-08-12 17:19:26.658456 2014-08-13 18:06:06.515844 7f5c973a57a0 20 read_log 194215'493 (194215'475) delete 33f0ccfe/benchmark_data_storage0111-ib.pocstor_64498_object27370/head//22 by client.1049635.0:61573 2014-08-12 17:19:27.136365 2014-08-13 18:06:06.515857 7f5c973a57a0 20 read_log 194215'494 (194215'476) delete 8e498cfe/benchmark_data_storage0111-ib.pocstor_64498_object27382/head//22 by client.1049635.0:61585 2014-08-12 17:19:27.139273 2014-08-13 18:06:06.515869 7f5c973a57a0 20 read_log 194215'495 (194215'477) delete d172ecfe/benchmark_data_storage0111-ib.pocstor_64498_object31545/head//22 by client.1049635.0:65748 2014-08-12 17:19:28.573673 2014-08-13 18:06:06.515975 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.515993 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.516009 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001923 0 0.000000 2014-08-13 18:06:06.516021 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.516029 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.516038 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] log((0'0,194215'495], crt=192888'459) 2014-08-13 18:06:06.516048 7f5c973a57a0 10 osd.4 194487 pgid 22.537 coll 22.537_head 2014-08-13 18:06:06.516184 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.537 2014-08-13 18:06:06.516193 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.516219 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.537(unlocked)] enter Initial 2014-08-13 18:06:06.516230 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.537(unlocked)] enter NotTrimming 2014-08-13 18:06:06.516385 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.516525 7f5c973a57a0 20 read_log 191648'64 (0'0) modify 9f2d3537/benchmark_data_storage0111-ib.pocstor_40745_object4149/head//22 by client.273158.0:4150 2014-07-30 14:09:07.396439 2014-08-13 18:06:06.516546 7f5c973a57a0 20 read_log 191648'102 (0'0) modify e92e2537/benchmark_data_storage0111-ib.pocstor_40745_object6017/head//22 by client.273158.0:6018 2014-07-30 14:09:15.942553 2014-08-13 18:06:06.516559 7f5c973a57a0 20 read_log 191648'149 (0'0) modify 5cce3d37/benchmark_data_storage0111-ib.pocstor_40745_object9140/head//22 by client.273158.0:9141 2014-07-30 14:09:31.727335 2014-08-13 18:06:06.516572 7f5c973a57a0 20 read_log 191648'162 (0'0) modify 5abeb537/benchmark_data_storage0111-ib.pocstor_40745_object10175/head//22 by client.273158.0:10176 2014-07-30 14:09:36.713070 2014-08-13 18:06:06.516584 7f5c973a57a0 20 read_log 191648'217 (0'0) modify 163ced37/benchmark_data_storage0111-ib.pocstor_40745_object14668/head//22 by client.273158.0:14669 2014-07-30 14:10:00.860410 2014-08-13 18:06:06.516597 7f5c973a57a0 20 read_log 191648'268 (0'0) modify bcd06d37/benchmark_data_storage0111-ib.pocstor_40745_object17041/head//22 by client.273158.0:17042 2014-07-30 14:10:13.209255 2014-08-13 18:06:06.516617 7f5c973a57a0 20 read_log 191648'281 (0'0) modify e0988d37/benchmark_data_storage0111-ib.pocstor_40745_object17851/head//22 by client.273158.0:17852 2014-07-30 14:10:17.245423 2014-08-13 18:06:06.516632 7f5c973a57a0 20 read_log 191648'302 (0'0) modify a6dd7537/benchmark_data_storage0111-ib.pocstor_40745_object19182/head//22 by client.273158.0:19183 2014-07-30 14:10:24.319763 2014-08-13 18:06:06.516644 7f5c973a57a0 20 read_log 191648'329 (0'0) modify 321ecd37/benchmark_data_storage0111-ib.pocstor_40745_object21131/head//22 by client.273158.0:21132 2014-07-30 14:10:37.821865 2014-08-13 18:06:06.516656 7f5c973a57a0 20 read_log 191648'345 (0'0) modify 561e0d37/benchmark_data_storage0111-ib.pocstor_40745_object21999/head//22 by client.273158.0:22000 2014-07-30 14:10:44.192191 2014-08-13 18:06:06.516668 7f5c973a57a0 20 read_log 191648'353 (0'0) modify bbb3d37/benchmark_data_storage0111-ib.pocstor_40745_object22437/head//22 by client.273158.0:22438 2014-07-30 14:10:46.478246 2014-08-13 18:06:06.516680 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 9f800d37/benchmark_data_storage0111-ib.pocstor_51285_object2624/head//22 by client.275175.0:2625 2014-07-30 14:18:43.014894 2014-08-13 18:06:06.516693 7f5c973a57a0 20 read_log 191727'370 (0'0) modify e4a0c537/benchmark_data_storage0111-ib.pocstor_51285_object5657/head//22 by client.275175.0:5658 2014-07-30 14:18:52.265763 2014-08-13 18:06:06.516705 7f5c973a57a0 20 read_log 191727'371 (0'0) modify ab82537/benchmark_data_storage0111-ib.pocstor_51285_object6609/head//22 by client.275175.0:6610 2014-07-30 14:18:55.202703 2014-08-13 18:06:06.516735 7f5c973a57a0 20 read_log 191727'372 (0'0) modify 3c3d3537/benchmark_data_storage0111-ib.pocstor_51285_object6751/head//22 by client.275175.0:6752 2014-07-30 14:18:55.961904 2014-08-13 18:06:06.516750 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 533c7d37/benchmark_data_storage0111-ib.pocstor_51285_object7393/head//22 by client.275175.0:7394 2014-07-30 14:18:57.835060 2014-08-13 18:06:06.516764 7f5c973a57a0 20 read_log 191727'374 (0'0) modify 626e7d37/benchmark_data_storage0111-ib.pocstor_51285_object7462/head//22 by client.275175.0:7463 2014-07-30 14:18:58.026429 2014-08-13 18:06:06.516777 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 6d7d0537/benchmark_data_storage0111-ib.pocstor_51285_object7884/head//22 by client.275175.0:7885 2014-07-30 14:18:59.200890 2014-08-13 18:06:06.516789 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 8e980d37/benchmark_data_storage0111-ib.pocstor_51285_object11945/head//22 by client.275175.0:11946 2014-07-30 14:19:11.296820 2014-08-13 18:06:06.516801 7f5c973a57a0 20 read_log 191727'389 (0'0) modify b9cd1537/benchmark_data_storage0111-ib.pocstor_51285_object13273/head//22 by client.275175.0:13274 2014-07-30 14:19:15.465684 2014-08-13 18:06:06.516813 7f5c973a57a0 20 read_log 191727'393 (0'0) modify b73cc537/benchmark_data_storage0111-ib.pocstor_51285_object15116/head//22 by client.275175.0:15117 2014-07-30 14:19:21.100978 2014-08-13 18:06:06.516825 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 8b9bcd37/benchmark_data_storage0111-ib.pocstor_51285_object15958/head//22 by client.275175.0:15959 2014-07-30 14:19:23.703644 2014-08-13 18:06:06.516837 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 76fcfd37/benchmark_data_storage0111-ib.pocstor_51285_object16661/head//22 by client.275175.0:16662 2014-07-30 14:19:26.204367 2014-08-13 18:06:06.516849 7f5c973a57a0 20 read_log 191727'399 (0'0) modify bf420537/benchmark_data_storage0111-ib.pocstor_51285_object17034/head//22 by client.275175.0:17035 2014-07-30 14:19:27.255682 2014-08-13 18:06:06.516861 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 853f5d37/benchmark_data_storage0111-ib.pocstor_51285_object18814/head//22 by client.275175.0:18815 2014-07-30 14:19:32.605231 2014-08-13 18:06:06.516900 7f5c973a57a0 20 read_log 191727'414 (0'0) modify cc4fb537/benchmark_data_storage0111-ib.pocstor_51285_object28366/head//22 by client.275175.0:28367 2014-07-30 14:20:02.046246 2014-08-13 18:06:06.516916 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 2eae2537/benchmark_data_storage0111-ib.pocstor_51285_object35246/head//22 by client.275175.0:35247 2014-07-30 14:20:22.704942 2014-08-13 18:06:06.516929 7f5c973a57a0 20 read_log 191727'430 (0'0) modify e81b5537/benchmark_data_storage0111-ib.pocstor_51285_object36909/head//22 by client.275175.0:36910 2014-07-30 14:20:28.124031 2014-08-13 18:06:06.516941 7f5c973a57a0 20 read_log 192888'434 (0'0) modify fe549537/benchmark_data_storage0111-ib.pocstor_35625_object1111/head//22 by client.940330.0:1112 2014-08-11 16:29:45.205610 2014-08-13 18:06:06.516953 7f5c973a57a0 20 read_log 192888'435 (0'0) modify f0a82537/benchmark_data_storage0111-ib.pocstor_35625_object1868/head//22 by client.940330.0:1869 2014-08-11 16:29:48.057971 2014-08-13 18:06:06.516966 7f5c973a57a0 20 read_log 192888'436 (0'0) modify d6609d37/benchmark_data_storage0111-ib.pocstor_35625_object3626/head//22 by client.940330.0:3627 2014-08-11 16:29:54.913776 2014-08-13 18:06:06.516978 7f5c973a57a0 20 read_log 192888'437 (0'0) modify 7f693d37/benchmark_data_storage0111-ib.pocstor_35625_object4793/head//22 by client.940330.0:4794 2014-08-11 16:29:59.302241 2014-08-13 18:06:06.516990 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 48a16537/benchmark_data_storage0111-ib.pocstor_35625_object8829/head//22 by client.940330.0:8830 2014-08-11 16:30:13.735937 2014-08-13 18:06:06.517002 7f5c973a57a0 20 read_log 192888'439 (0'0) modify c38fc537/benchmark_data_storage0111-ib.pocstor_35625_object9379/head//22 by client.940330.0:9380 2014-08-11 16:30:15.657145 2014-08-13 18:06:06.517016 7f5c973a57a0 20 read_log 192888'440 (0'0) modify c01a3537/benchmark_data_storage0111-ib.pocstor_35625_object11996/head//22 by client.940330.0:11997 2014-08-11 16:30:25.927545 2014-08-13 18:06:06.517029 7f5c973a57a0 20 read_log 192888'441 (0'0) modify db259d37/benchmark_data_storage0111-ib.pocstor_35625_object14749/head//22 by client.940330.0:14750 2014-08-11 16:30:36.558485 2014-08-13 18:06:06.517043 7f5c973a57a0 20 read_log 192888'442 (0'0) modify d353b537/benchmark_data_storage0111-ib.pocstor_35625_object16821/head//22 by client.940330.0:16822 2014-08-11 16:30:44.298436 2014-08-13 18:06:06.517055 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 139a5537/benchmark_data_storage0111-ib.pocstor_35625_object20363/head//22 by client.940330.0:20364 2014-08-11 16:30:58.187839 2014-08-13 18:06:06.517069 7f5c973a57a0 20 read_log 192888'444 (0'0) modify defd8d37/benchmark_data_storage0111-ib.pocstor_35625_object24671/head//22 by client.940330.0:24672 2014-08-11 16:31:14.611331 2014-08-13 18:06:06.517081 7f5c973a57a0 20 read_log 192888'445 (0'0) modify e245d37/benchmark_data_storage0111-ib.pocstor_35625_object26935/head//22 by client.940330.0:26936 2014-08-11 16:31:23.553779 2014-08-13 18:06:06.517095 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 9b2c4537/benchmark_data_storage0111-ib.pocstor_35625_object29549/head//22 by client.940330.0:29550 2014-08-11 16:31:33.756040 2014-08-13 18:06:06.517107 7f5c973a57a0 20 read_log 192888'447 (0'0) modify b11edd37/benchmark_data_storage0111-ib.pocstor_35625_object30249/head//22 by client.940330.0:30250 2014-08-11 16:31:36.402100 2014-08-13 18:06:06.517121 7f5c973a57a0 20 read_log 192888'448 (192888'434) delete fe549537/benchmark_data_storage0111-ib.pocstor_35625_object1111/head//22 by client.940330.0:32419 2014-08-11 16:31:41.776212 2014-08-13 18:06:06.517163 7f5c973a57a0 20 read_log 192888'449 (192888'435) delete f0a82537/benchmark_data_storage0111-ib.pocstor_35625_object1868/head//22 by client.940330.0:33176 2014-08-11 16:31:41.957605 2014-08-13 18:06:06.517186 7f5c973a57a0 20 read_log 192888'450 (192888'436) delete d6609d37/benchmark_data_storage0111-ib.pocstor_35625_object3626/head//22 by client.940330.0:34934 2014-08-11 16:31:42.420971 2014-08-13 18:06:06.517200 7f5c973a57a0 20 read_log 192888'451 (192888'437) delete 7f693d37/benchmark_data_storage0111-ib.pocstor_35625_object4793/head//22 by client.940330.0:36101 2014-08-11 16:31:42.764209 2014-08-13 18:06:06.517213 7f5c973a57a0 20 read_log 192888'452 (192888'438) delete 48a16537/benchmark_data_storage0111-ib.pocstor_35625_object8829/head//22 by client.940330.0:40137 2014-08-11 16:31:43.799881 2014-08-13 18:06:06.517225 7f5c973a57a0 20 read_log 192888'453 (192888'439) delete c38fc537/benchmark_data_storage0111-ib.pocstor_35625_object9379/head//22 by client.940330.0:40687 2014-08-11 16:31:43.942556 2014-08-13 18:06:06.517238 7f5c973a57a0 20 read_log 192888'454 (192888'440) delete c01a3537/benchmark_data_storage0111-ib.pocstor_35625_object11996/head//22 by client.940330.0:43304 2014-08-11 16:31:44.656311 2014-08-13 18:06:06.517251 7f5c973a57a0 20 read_log 192888'455 (192888'441) delete db259d37/benchmark_data_storage0111-ib.pocstor_35625_object14749/head//22 by client.940330.0:46057 2014-08-11 16:31:45.386195 2014-08-13 18:06:06.517263 7f5c973a57a0 20 read_log 192888'456 (192888'442) delete d353b537/benchmark_data_storage0111-ib.pocstor_35625_object16821/head//22 by client.940330.0:48129 2014-08-11 16:31:45.939955 2014-08-13 18:06:06.517282 7f5c973a57a0 20 read_log 192888'457 (192888'443) delete 139a5537/benchmark_data_storage0111-ib.pocstor_35625_object20363/head//22 by client.940330.0:51671 2014-08-11 16:31:46.983849 2014-08-13 18:06:06.517297 7f5c973a57a0 20 read_log 192888'458 (192888'444) delete defd8d37/benchmark_data_storage0111-ib.pocstor_35625_object24671/head//22 by client.940330.0:55979 2014-08-11 16:31:48.235593 2014-08-13 18:06:06.517309 7f5c973a57a0 20 read_log 192888'459 (192888'445) delete e245d37/benchmark_data_storage0111-ib.pocstor_35625_object26935/head//22 by client.940330.0:58243 2014-08-11 16:31:48.911101 2014-08-13 18:06:06.517324 7f5c973a57a0 20 read_log 192888'460 (192888'446) delete 9b2c4537/benchmark_data_storage0111-ib.pocstor_35625_object29549/head//22 by client.940330.0:60857 2014-08-11 16:31:49.628931 2014-08-13 18:06:06.517340 7f5c973a57a0 20 read_log 192888'461 (192888'447) delete b11edd37/benchmark_data_storage0111-ib.pocstor_35625_object30249/head//22 by client.940330.0:61557 2014-08-11 16:31:49.828387 2014-08-13 18:06:06.517353 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 1caf6537/benchmark_data_storage0111-ib.pocstor_64498_object889/head//22 by client.1049635.0:890 2014-08-12 17:17:19.992609 2014-08-13 18:06:06.517366 7f5c973a57a0 20 read_log 194215'463 (0'0) modify ee9dbd37/benchmark_data_storage0111-ib.pocstor_64498_object1881/head//22 by client.1049635.0:1882 2014-08-12 17:17:23.176455 2014-08-13 18:06:06.517378 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 68637d37/benchmark_data_storage0111-ib.pocstor_64498_object4355/head//22 by client.1049635.0:4356 2014-08-12 17:17:31.099047 2014-08-13 18:06:06.517390 7f5c973a57a0 20 read_log 194215'465 (0'0) modify f4ec8537/benchmark_data_storage0111-ib.pocstor_64498_object9681/head//22 by client.1049635.0:9682 2014-08-12 17:17:47.016142 2014-08-13 18:06:06.517420 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 32adc537/benchmark_data_storage0111-ib.pocstor_64498_object10221/head//22 by client.1049635.0:10222 2014-08-12 17:17:48.555078 2014-08-13 18:06:06.517434 7f5c973a57a0 20 read_log 194215'467 (0'0) modify a86d5537/benchmark_data_storage0111-ib.pocstor_64498_object11839/head//22 by client.1049635.0:11840 2014-08-12 17:17:53.933524 2014-08-13 18:06:06.517456 7f5c973a57a0 20 read_log 194215'468 (0'0) modify b2a34537/benchmark_data_storage0111-ib.pocstor_64498_object14642/head//22 by client.1049635.0:14643 2014-08-12 17:18:02.766985 2014-08-13 18:06:06.517479 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 3afb8537/benchmark_data_storage0111-ib.pocstor_64498_object14690/head//22 by client.1049635.0:14691 2014-08-12 17:18:02.977905 2014-08-13 18:06:06.517511 7f5c973a57a0 20 read_log 194215'470 (0'0) modify f79d1d37/benchmark_data_storage0111-ib.pocstor_64498_object15153/head//22 by client.1049635.0:15154 2014-08-12 17:18:04.475618 2014-08-13 18:06:06.517524 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 4cb72537/benchmark_data_storage0111-ib.pocstor_64498_object15792/head//22 by client.1049635.0:15793 2014-08-12 17:18:06.783276 2014-08-13 18:06:06.517537 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 574a4537/benchmark_data_storage0111-ib.pocstor_64498_object18217/head//22 by client.1049635.0:18218 2014-08-12 17:18:15.981564 2014-08-13 18:06:06.517560 7f5c973a57a0 20 read_log 194215'473 (0'0) modify f84c6d37/benchmark_data_storage0111-ib.pocstor_64498_object18315/head//22 by client.1049635.0:18316 2014-08-12 17:18:16.355372 2014-08-13 18:06:06.517573 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 77140537/benchmark_data_storage0111-ib.pocstor_64498_object18943/head//22 by client.1049635.0:18944 2014-08-12 17:18:18.721127 2014-08-13 18:06:06.517586 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 30607d37/benchmark_data_storage0111-ib.pocstor_64498_object19997/head//22 by client.1049635.0:19998 2014-08-12 17:18:22.769519 2014-08-13 18:06:06.517598 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 43d19537/benchmark_data_storage0111-ib.pocstor_64498_object20224/head//22 by client.1049635.0:20225 2014-08-12 17:18:23.624964 2014-08-13 18:06:06.517611 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 941bd537/benchmark_data_storage0111-ib.pocstor_64498_object23926/head//22 by client.1049635.0:23927 2014-08-12 17:18:37.579130 2014-08-13 18:06:06.517623 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 74970537/benchmark_data_storage0111-ib.pocstor_64498_object24055/head//22 by client.1049635.0:24056 2014-08-12 17:18:38.057385 2014-08-13 18:06:06.517635 7f5c973a57a0 20 read_log 194215'479 (0'0) modify f848ad37/benchmark_data_storage0111-ib.pocstor_64498_object26220/head//22 by client.1049635.0:26221 2014-08-12 17:18:46.242193 2014-08-13 18:06:06.517650 7f5c973a57a0 20 read_log 194215'480 (0'0) modify ec0d3d37/benchmark_data_storage0111-ib.pocstor_64498_object26486/head//22 by client.1049635.0:26487 2014-08-12 17:18:47.238199 2014-08-13 18:06:06.517665 7f5c973a57a0 20 read_log 194215'481 (0'0) modify a97aad37/benchmark_data_storage0111-ib.pocstor_64498_object32816/head//22 by client.1049635.0:32817 2014-08-12 17:19:11.689309 2014-08-13 18:06:06.517678 7f5c973a57a0 20 read_log 194215'482 (194215'462) delete 1caf6537/benchmark_data_storage0111-ib.pocstor_64498_object889/head//22 by client.1049635.0:35092 2014-08-12 17:19:18.277797 2014-08-13 18:06:06.517691 7f5c973a57a0 20 read_log 194215'483 (194215'463) delete ee9dbd37/benchmark_data_storage0111-ib.pocstor_64498_object1881/head//22 by client.1049635.0:36084 2014-08-12 17:19:18.610953 2014-08-13 18:06:06.517703 7f5c973a57a0 20 read_log 194215'484 (194215'464) delete 68637d37/benchmark_data_storage0111-ib.pocstor_64498_object4355/head//22 by client.1049635.0:38558 2014-08-12 17:19:19.388681 2014-08-13 18:06:06.517716 7f5c973a57a0 20 read_log 194215'485 (194215'465) delete f4ec8537/benchmark_data_storage0111-ib.pocstor_64498_object9681/head//22 by client.1049635.0:43884 2014-08-12 17:19:21.148558 2014-08-13 18:06:06.517728 7f5c973a57a0 20 read_log 194215'486 (194215'466) delete 32adc537/benchmark_data_storage0111-ib.pocstor_64498_object10221/head//22 by client.1049635.0:44424 2014-08-12 17:19:21.335372 2014-08-13 18:06:06.517740 7f5c973a57a0 20 read_log 194215'487 (194215'467) delete a86d5537/benchmark_data_storage0111-ib.pocstor_64498_object11839/head//22 by client.1049635.0:46042 2014-08-12 17:19:21.850386 2014-08-13 18:06:06.517764 7f5c973a57a0 20 read_log 194215'488 (194215'468) delete b2a34537/benchmark_data_storage0111-ib.pocstor_64498_object14642/head//22 by client.1049635.0:48845 2014-08-12 17:19:22.767963 2014-08-13 18:06:06.517778 7f5c973a57a0 20 read_log 194215'489 (194215'469) delete 3afb8537/benchmark_data_storage0111-ib.pocstor_64498_object14690/head//22 by client.1049635.0:48893 2014-08-12 17:19:22.782057 2014-08-13 18:06:06.517791 7f5c973a57a0 20 read_log 194215'490 (194215'470) delete f79d1d37/benchmark_data_storage0111-ib.pocstor_64498_object15153/head//22 by client.1049635.0:49356 2014-08-12 17:19:22.921072 2014-08-13 18:06:06.517803 7f5c973a57a0 20 read_log 194215'491 (194215'471) delete 4cb72537/benchmark_data_storage0111-ib.pocstor_64498_object15792/head//22 by client.1049635.0:49995 2014-08-12 17:19:23.119120 2014-08-13 18:06:06.517815 7f5c973a57a0 20 read_log 194215'492 (194215'472) delete 574a4537/benchmark_data_storage0111-ib.pocstor_64498_object18217/head//22 by client.1049635.0:52420 2014-08-12 17:19:23.901882 2014-08-13 18:06:06.517827 7f5c973a57a0 20 read_log 194215'493 (194215'473) delete f84c6d37/benchmark_data_storage0111-ib.pocstor_64498_object18315/head//22 by client.1049635.0:52518 2014-08-12 17:19:23.936367 2014-08-13 18:06:06.517840 7f5c973a57a0 20 read_log 194215'494 (194215'474) delete 77140537/benchmark_data_storage0111-ib.pocstor_64498_object18943/head//22 by client.1049635.0:53146 2014-08-12 17:19:24.145069 2014-08-13 18:06:06.517852 7f5c973a57a0 20 read_log 194215'495 (194215'475) delete 30607d37/benchmark_data_storage0111-ib.pocstor_64498_object19997/head//22 by client.1049635.0:54200 2014-08-12 17:19:24.513925 2014-08-13 18:06:06.517864 7f5c973a57a0 20 read_log 194215'496 (194215'476) delete 43d19537/benchmark_data_storage0111-ib.pocstor_64498_object20224/head//22 by client.1049635.0:54427 2014-08-12 17:19:24.606307 2014-08-13 18:06:06.517876 7f5c973a57a0 20 read_log 194215'497 (194215'477) delete 941bd537/benchmark_data_storage0111-ib.pocstor_64498_object23926/head//22 by client.1049635.0:58129 2014-08-12 17:19:25.937475 2014-08-13 18:06:06.517889 7f5c973a57a0 20 read_log 194215'498 (194215'478) delete 74970537/benchmark_data_storage0111-ib.pocstor_64498_object24055/head//22 by client.1049635.0:58258 2014-08-12 17:19:25.980347 2014-08-13 18:06:06.517901 7f5c973a57a0 20 read_log 194215'499 (194215'479) delete f848ad37/benchmark_data_storage0111-ib.pocstor_64498_object26220/head//22 by client.1049635.0:60423 2014-08-12 17:19:26.732956 2014-08-13 18:06:06.517913 7f5c973a57a0 20 read_log 194215'500 (194215'480) delete ec0d3d37/benchmark_data_storage0111-ib.pocstor_64498_object26486/head//22 by client.1049635.0:60689 2014-08-12 17:19:26.817198 2014-08-13 18:06:06.517925 7f5c973a57a0 20 read_log 194215'501 (194215'481) delete a97aad37/benchmark_data_storage0111-ib.pocstor_64498_object32816/head//22 by client.1049635.0:67019 2014-08-12 17:19:29.045914 2014-08-13 18:06:06.518005 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.518023 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=0 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.518038 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=0 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Initial 0.001820 0 0.000000 2014-08-13 18:06:06.518050 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=0 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.518059 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=0 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.518068 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] log((0'0,194215'501], crt=192888'461) 2014-08-13 18:06:06.518078 7f5c973a57a0 10 osd.4 194487 pgid 22.581 coll 22.581_head 2014-08-13 18:06:06.518231 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.581 2014-08-13 18:06:06.518241 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.518283 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.581(unlocked)] enter Initial 2014-08-13 18:06:06.518295 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.581(unlocked)] enter NotTrimming 2014-08-13 18:06:06.518456 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.518584 7f5c973a57a0 20 read_log 191648'1 (0'0) modify d146b581/benchmark_data_storage0111-ib.pocstor_40745_object11/head//22 by client.273158.0:12 2014-07-30 14:08:48.538992 2014-08-13 18:06:06.518605 7f5c973a57a0 20 read_log 191648'16 (0'0) modify 272b0d81/benchmark_data_storage0111-ib.pocstor_40745_object1340/head//22 by client.273158.0:1341 2014-07-30 14:08:53.717594 2014-08-13 18:06:06.518618 7f5c973a57a0 20 read_log 191648'24 (0'0) modify 2c065581/benchmark_data_storage0111-ib.pocstor_40745_object1791/head//22 by client.273158.0:1792 2014-07-30 14:08:55.358268 2014-08-13 18:06:06.518630 7f5c973a57a0 20 read_log 191648'59 (0'0) modify 5068dd81/benchmark_data_storage0111-ib.pocstor_40745_object3427/head//22 by client.273158.0:3428 2014-07-30 14:09:03.353286 2014-08-13 18:06:06.518643 7f5c973a57a0 20 read_log 191648'65 (0'0) modify f5682581/benchmark_data_storage0111-ib.pocstor_40745_object3784/head//22 by client.273158.0:3785 2014-07-30 14:09:05.182665 2014-08-13 18:06:06.518655 7f5c973a57a0 20 read_log 191648'78 (0'0) modify 397aad81/benchmark_data_storage0111-ib.pocstor_40745_object4453/head//22 by client.273158.0:4454 2014-07-30 14:09:08.548982 2014-08-13 18:06:06.518667 7f5c973a57a0 20 read_log 191648'98 (0'0) modify e2aa9d81/benchmark_data_storage0111-ib.pocstor_40745_object5708/head//22 by client.273158.0:5709 2014-07-30 14:09:14.276817 2014-08-13 18:06:06.518679 7f5c973a57a0 20 read_log 191648'105 (0'0) modify 14eae581/benchmark_data_storage0111-ib.pocstor_40745_object6217/head//22 by client.273158.0:6218 2014-07-30 14:09:17.111124 2014-08-13 18:06:06.518723 7f5c973a57a0 20 read_log 191648'113 (0'0) modify 11464d81/benchmark_data_storage0111-ib.pocstor_40745_object6648/head//22 by client.273158.0:6649 2014-07-30 14:09:19.549437 2014-08-13 18:06:06.518740 7f5c973a57a0 20 read_log 191648'138 (0'0) modify 63fb9d81/benchmark_data_storage0111-ib.pocstor_40745_object8043/head//22 by client.273158.0:8044 2014-07-30 14:09:26.300856 2014-08-13 18:06:06.518752 7f5c973a57a0 20 read_log 191648'140 (0'0) modify 41112d81/benchmark_data_storage0111-ib.pocstor_40745_object8385/head//22 by client.273158.0:8386 2014-07-30 14:09:28.163359 2014-08-13 18:06:06.518765 7f5c973a57a0 20 read_log 191648'177 (0'0) modify 6741bd81/benchmark_data_storage0111-ib.pocstor_40745_object10398/head//22 by client.273158.0:10399 2014-07-30 14:09:37.827925 2014-08-13 18:06:06.518777 7f5c973a57a0 20 read_log 191648'191 (0'0) modify 9fe2581/benchmark_data_storage0111-ib.pocstor_40745_object11558/head//22 by client.273158.0:11559 2014-07-30 14:09:45.058157 2014-08-13 18:06:06.518790 7f5c973a57a0 20 read_log 191648'238 (0'0) modify f8824d81/benchmark_data_storage0111-ib.pocstor_40745_object14131/head//22 by client.273158.0:14132 2014-07-30 14:09:57.630527 2014-08-13 18:06:06.518804 7f5c973a57a0 20 read_log 191648'255 (0'0) modify a52c1d81/benchmark_data_storage0111-ib.pocstor_40745_object14762/head//22 by client.273158.0:14763 2014-07-30 14:10:01.462413 2014-08-13 18:06:06.518819 7f5c973a57a0 20 read_log 191648'272 (0'0) modify dfe2581/benchmark_data_storage0111-ib.pocstor_40745_object15659/head//22 by client.273158.0:15660 2014-07-30 14:10:05.527327 2014-08-13 18:06:06.518845 7f5c973a57a0 20 read_log 191648'360 (0'0) modify fe3c5d81/benchmark_data_storage0111-ib.pocstor_40745_object21320/head//22 by client.273158.0:21321 2014-07-30 14:10:38.582548 2014-08-13 18:06:06.518860 7f5c973a57a0 20 read_log 191648'377 (0'0) modify 9a720d81/benchmark_data_storage0111-ib.pocstor_40745_object22468/head//22 by client.273158.0:22469 2014-07-30 14:10:46.949357 2014-08-13 18:06:06.518873 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 5d0f2581/benchmark_data_storage0111-ib.pocstor_51285_object1297/head//22 by client.275175.0:1298 2014-07-30 14:18:38.510951 2014-08-13 18:06:06.518885 7f5c973a57a0 20 read_log 191727'390 (0'0) modify ed21ed81/benchmark_data_storage0111-ib.pocstor_51285_object1342/head//22 by client.275175.0:1343 2014-07-30 14:18:38.669717 2014-08-13 18:06:06.518897 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 1759ad81/benchmark_data_storage0111-ib.pocstor_51285_object2146/head//22 by client.275175.0:2147 2014-07-30 14:18:41.373638 2014-08-13 18:06:06.518909 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 75719d81/benchmark_data_storage0111-ib.pocstor_51285_object3172/head//22 by client.275175.0:3173 2014-07-30 14:18:44.769729 2014-08-13 18:06:06.518921 7f5c973a57a0 20 read_log 191727'398 (0'0) modify 5f4a7d81/benchmark_data_storage0111-ib.pocstor_51285_object5271/head//22 by client.275175.0:5272 2014-07-30 14:18:51.151681 2014-08-13 18:06:06.518942 7f5c973a57a0 20 read_log 191727'406 (0'0) modify 60441581/benchmark_data_storage0111-ib.pocstor_51285_object10141/head//22 by client.275175.0:10142 2014-07-30 14:19:05.575559 2014-08-13 18:06:06.518978 7f5c973a57a0 20 read_log 191727'411 (0'0) modify cbbdbd81/benchmark_data_storage0111-ib.pocstor_51285_object10895/head//22 by client.275175.0:10896 2014-07-30 14:19:07.750675 2014-08-13 18:06:06.519008 7f5c973a57a0 20 read_log 191727'413 (0'0) modify 30080d81/benchmark_data_storage0111-ib.pocstor_51285_object12202/head//22 by client.275175.0:12203 2014-07-30 14:19:12.207065 2014-08-13 18:06:06.519021 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 36ebcd81/benchmark_data_storage0111-ib.pocstor_51285_object13646/head//22 by client.275175.0:13647 2014-07-30 14:19:16.657749 2014-08-13 18:06:06.519042 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 55007581/benchmark_data_storage0111-ib.pocstor_51285_object19691/head//22 by client.275175.0:19692 2014-07-30 14:19:35.305345 2014-08-13 18:06:06.519058 7f5c973a57a0 20 read_log 191727'429 (0'0) modify a9112d81/benchmark_data_storage0111-ib.pocstor_51285_object20829/head//22 by client.275175.0:20830 2014-07-30 14:19:38.645198 2014-08-13 18:06:06.519071 7f5c973a57a0 20 read_log 191727'430 (0'0) modify dd9a2d81/benchmark_data_storage0111-ib.pocstor_51285_object21118/head//22 by client.275175.0:21119 2014-07-30 14:19:39.465456 2014-08-13 18:06:06.519083 7f5c973a57a0 20 read_log 191727'434 (0'0) modify beb91581/benchmark_data_storage0111-ib.pocstor_51285_object21905/head//22 by client.275175.0:21906 2014-07-30 14:19:42.155331 2014-08-13 18:06:06.519097 7f5c973a57a0 20 read_log 191727'445 (0'0) modify 6f609d81/benchmark_data_storage0111-ib.pocstor_51285_object25276/head//22 by client.275175.0:25277 2014-07-30 14:19:52.636574 2014-08-13 18:06:06.519110 7f5c973a57a0 20 read_log 191727'450 (0'0) modify d77ce581/benchmark_data_storage0111-ib.pocstor_51285_object26437/head//22 by client.275175.0:26438 2014-07-30 14:19:56.136719 2014-08-13 18:06:06.519122 7f5c973a57a0 20 read_log 191727'454 (0'0) modify b3379d81/benchmark_data_storage0111-ib.pocstor_51285_object28839/head//22 by client.275175.0:28840 2014-07-30 14:20:03.433788 2014-08-13 18:06:06.519135 7f5c973a57a0 20 read_log 191727'456 (0'0) modify 1c28c581/benchmark_data_storage0111-ib.pocstor_51285_object29540/head//22 by client.275175.0:29541 2014-07-30 14:20:05.554898 2014-08-13 18:06:06.519185 7f5c973a57a0 20 read_log 191727'458 (0'0) modify a129581/benchmark_data_storage0111-ib.pocstor_51285_object30379/head//22 by client.275175.0:30380 2014-07-30 14:20:08.240810 2014-08-13 18:06:06.519203 7f5c973a57a0 20 read_log 191727'460 (0'0) modify c552fd81/benchmark_data_storage0111-ib.pocstor_51285_object31719/head//22 by client.275175.0:31720 2014-07-30 14:20:12.062727 2014-08-13 18:06:06.519218 7f5c973a57a0 20 read_log 191727'461 (0'0) modify 6a495d81/benchmark_data_storage0111-ib.pocstor_51285_object32013/head//22 by client.275175.0:32014 2014-07-30 14:20:12.887831 2014-08-13 18:06:06.519233 7f5c973a57a0 20 read_log 191727'465 (0'0) modify 2d9d5581/benchmark_data_storage0111-ib.pocstor_51285_object34460/head//22 by client.275175.0:34461 2014-07-30 14:20:20.435826 2014-08-13 18:06:06.519245 7f5c973a57a0 20 read_log 191727'475 (0'0) modify 70e8f581/benchmark_data_storage0111-ib.pocstor_51285_object38678/head//22 by client.275175.0:38679 2014-07-30 14:20:33.179159 2014-08-13 18:06:06.519258 7f5c973a57a0 20 read_log 192888'476 (0'0) modify 28013d81/benchmark_data_storage0111-ib.pocstor_35625_object1558/head//22 by client.940330.0:1559 2014-08-11 16:29:46.921107 2014-08-13 18:06:06.519274 7f5c973a57a0 20 read_log 192888'477 (0'0) modify 8576c581/benchmark_data_storage0111-ib.pocstor_35625_object4090/head//22 by client.940330.0:4091 2014-08-11 16:29:56.660294 2014-08-13 18:06:06.519287 7f5c973a57a0 20 read_log 192888'478 (0'0) modify 821fed81/benchmark_data_storage0111-ib.pocstor_35625_object8710/head//22 by client.940330.0:8711 2014-08-11 16:30:13.321367 2014-08-13 18:06:06.519299 7f5c973a57a0 20 read_log 192888'479 (0'0) modify 458f1d81/benchmark_data_storage0111-ib.pocstor_35625_object12452/head//22 by client.940330.0:12453 2014-08-11 16:30:27.715445 2014-08-13 18:06:06.519311 7f5c973a57a0 20 read_log 192888'480 (0'0) modify bd1a0581/benchmark_data_storage0111-ib.pocstor_35625_object12722/head//22 by client.940330.0:12723 2014-08-11 16:30:28.720865 2014-08-13 18:06:06.519330 7f5c973a57a0 20 read_log 192888'481 (0'0) modify cfa2bd81/benchmark_data_storage0111-ib.pocstor_35625_object15617/head//22 by client.940330.0:15618 2014-08-11 16:30:39.789438 2014-08-13 18:06:06.519350 7f5c973a57a0 20 read_log 192888'482 (0'0) modify 781e1d81/benchmark_data_storage0111-ib.pocstor_35625_object19145/head//22 by client.940330.0:19146 2014-08-11 16:30:53.201350 2014-08-13 18:06:06.519369 7f5c973a57a0 20 read_log 192888'483 (0'0) modify 7f69cd81/benchmark_data_storage0111-ib.pocstor_35625_object25229/head//22 by client.940330.0:25230 2014-08-11 16:31:16.787923 2014-08-13 18:06:06.519384 7f5c973a57a0 20 read_log 192888'484 (0'0) modify 5dd64581/benchmark_data_storage0111-ib.pocstor_35625_object25720/head//22 by client.940330.0:25721 2014-08-11 16:31:18.691446 2014-08-13 18:06:06.519396 7f5c973a57a0 20 read_log 192888'485 (0'0) modify 75688d81/benchmark_data_storage0111-ib.pocstor_35625_object26050/head//22 by client.940330.0:26051 2014-08-11 16:31:19.994972 2014-08-13 18:06:06.519409 7f5c973a57a0 20 read_log 192888'486 (0'0) modify 4c108d81/benchmark_data_storage0111-ib.pocstor_35625_object26980/head//22 by client.940330.0:26981 2014-08-11 16:31:23.724757 2014-08-13 18:06:06.519421 7f5c973a57a0 20 read_log 192888'487 (192888'476) delete 28013d81/benchmark_data_storage0111-ib.pocstor_35625_object1558/head//22 by client.940330.0:32866 2014-08-11 16:31:41.881756 2014-08-13 18:06:06.519445 7f5c973a57a0 20 read_log 192888'488 (192888'477) delete 8576c581/benchmark_data_storage0111-ib.pocstor_35625_object4090/head//22 by client.940330.0:35398 2014-08-11 16:31:42.561525 2014-08-13 18:06:06.519464 7f5c973a57a0 20 read_log 192888'489 (192888'478) delete 821fed81/benchmark_data_storage0111-ib.pocstor_35625_object8710/head//22 by client.940330.0:40018 2014-08-11 16:31:43.766973 2014-08-13 18:06:06.519484 7f5c973a57a0 20 read_log 192888'490 (192888'479) delete 458f1d81/benchmark_data_storage0111-ib.pocstor_35625_object12452/head//22 by client.940330.0:43760 2014-08-11 16:31:44.770827 2014-08-13 18:06:06.519510 7f5c973a57a0 20 read_log 192888'491 (192888'480) delete bd1a0581/benchmark_data_storage0111-ib.pocstor_35625_object12722/head//22 by client.940330.0:44030 2014-08-11 16:31:44.841348 2014-08-13 18:06:06.519526 7f5c973a57a0 20 read_log 192888'492 (192888'481) delete cfa2bd81/benchmark_data_storage0111-ib.pocstor_35625_object15617/head//22 by client.940330.0:46925 2014-08-11 16:31:45.622949 2014-08-13 18:06:06.519539 7f5c973a57a0 20 read_log 192888'493 (192888'482) delete 781e1d81/benchmark_data_storage0111-ib.pocstor_35625_object19145/head//22 by client.940330.0:50453 2014-08-11 16:31:46.620029 2014-08-13 18:06:06.519555 7f5c973a57a0 20 read_log 192888'494 (192888'483) delete 7f69cd81/benchmark_data_storage0111-ib.pocstor_35625_object25229/head//22 by client.940330.0:56537 2014-08-11 16:31:48.414216 2014-08-13 18:06:06.519568 7f5c973a57a0 20 read_log 192888'495 (192888'484) delete 5dd64581/benchmark_data_storage0111-ib.pocstor_35625_object25720/head//22 by client.940330.0:57028 2014-08-11 16:31:48.565309 2014-08-13 18:06:06.519580 7f5c973a57a0 20 read_log 192888'496 (192888'485) delete 75688d81/benchmark_data_storage0111-ib.pocstor_35625_object26050/head//22 by client.940330.0:57358 2014-08-11 16:31:48.662122 2014-08-13 18:06:06.519592 7f5c973a57a0 20 read_log 192888'497 (192888'486) delete 4c108d81/benchmark_data_storage0111-ib.pocstor_35625_object26980/head//22 by client.940330.0:58288 2014-08-11 16:31:48.921349 2014-08-13 18:06:06.519610 7f5c973a57a0 20 read_log 194215'498 (0'0) modify 9d807d81/benchmark_data_storage0111-ib.pocstor_64498_object5781/head//22 by client.1049635.0:5782 2014-08-12 17:17:35.440085 2014-08-13 18:06:06.519623 7f5c973a57a0 20 read_log 194215'499 (0'0) modify c3b61581/benchmark_data_storage0111-ib.pocstor_64498_object7847/head//22 by client.1049635.0:7848 2014-08-12 17:17:41.678216 2014-08-13 18:06:06.519635 7f5c973a57a0 20 read_log 194215'500 (0'0) modify ad4a0581/benchmark_data_storage0111-ib.pocstor_64498_object7922/head//22 by client.1049635.0:7923 2014-08-12 17:17:41.898104 2014-08-13 18:06:06.519659 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 4074d81/benchmark_data_storage0111-ib.pocstor_64498_object8983/head//22 by client.1049635.0:8984 2014-08-12 17:17:44.974246 2014-08-13 18:06:06.519674 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 74106d81/benchmark_data_storage0111-ib.pocstor_64498_object11247/head//22 by client.1049635.0:11248 2014-08-12 17:17:51.962968 2014-08-13 18:06:06.519687 7f5c973a57a0 20 read_log 194215'503 (0'0) modify f725c581/benchmark_data_storage0111-ib.pocstor_64498_object12443/head//22 by client.1049635.0:12444 2014-08-12 17:17:55.739942 2014-08-13 18:06:06.519710 7f5c973a57a0 20 read_log 194215'504 (0'0) modify 2b3e2d81/benchmark_data_storage0111-ib.pocstor_64498_object14452/head//22 by client.1049635.0:14453 2014-08-12 17:18:02.095885 2014-08-13 18:06:06.519726 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 6396d581/benchmark_data_storage0111-ib.pocstor_64498_object16192/head//22 by client.1049635.0:16193 2014-08-12 17:18:08.282803 2014-08-13 18:06:06.519739 7f5c973a57a0 20 read_log 194215'506 (0'0) modify a2166d81/benchmark_data_storage0111-ib.pocstor_64498_object20000/head//22 by client.1049635.0:20001 2014-08-12 17:18:22.780614 2014-08-13 18:06:06.519756 7f5c973a57a0 20 read_log 194215'507 (0'0) modify f73f2d81/benchmark_data_storage0111-ib.pocstor_64498_object21974/head//22 by client.1049635.0:21975 2014-08-12 17:18:30.233423 2014-08-13 18:06:06.519771 7f5c973a57a0 20 read_log 194215'508 (0'0) modify 3ae62d81/benchmark_data_storage0111-ib.pocstor_64498_object23221/head//22 by client.1049635.0:23222 2014-08-12 17:18:34.953135 2014-08-13 18:06:06.519791 7f5c973a57a0 20 read_log 194215'509 (0'0) modify bb607d81/benchmark_data_storage0111-ib.pocstor_64498_object24641/head//22 by client.1049635.0:24642 2014-08-12 17:18:40.242869 2014-08-13 18:06:06.519806 7f5c973a57a0 20 read_log 194215'510 (0'0) modify c4f65d81/benchmark_data_storage0111-ib.pocstor_64498_object27772/head//22 by client.1049635.0:27773 2014-08-12 17:18:52.030026 2014-08-13 18:06:06.519830 7f5c973a57a0 20 read_log 194215'511 (0'0) modify 94586581/benchmark_data_storage0111-ib.pocstor_64498_object28398/head//22 by client.1049635.0:28399 2014-08-12 17:18:54.364140 2014-08-13 18:06:06.519843 7f5c973a57a0 20 read_log 194215'512 (0'0) modify c9722581/benchmark_data_storage0111-ib.pocstor_64498_object32664/head//22 by client.1049635.0:32665 2014-08-12 17:19:11.116031 2014-08-13 18:06:06.519855 7f5c973a57a0 20 read_log 194215'513 (194215'498) delete 9d807d81/benchmark_data_storage0111-ib.pocstor_64498_object5781/head//22 by client.1049635.0:39984 2014-08-12 17:19:19.881311 2014-08-13 18:06:06.519869 7f5c973a57a0 20 read_log 194215'514 (194215'499) delete c3b61581/benchmark_data_storage0111-ib.pocstor_64498_object7847/head//22 by client.1049635.0:42050 2014-08-12 17:19:20.557152 2014-08-13 18:06:06.519882 7f5c973a57a0 20 read_log 194215'515 (194215'500) delete ad4a0581/benchmark_data_storage0111-ib.pocstor_64498_object7922/head//22 by client.1049635.0:42125 2014-08-12 17:19:20.579298 2014-08-13 18:06:06.519894 7f5c973a57a0 20 read_log 194215'516 (194215'501) delete 4074d81/benchmark_data_storage0111-ib.pocstor_64498_object8983/head//22 by client.1049635.0:43186 2014-08-12 17:19:20.918859 2014-08-13 18:06:06.519907 7f5c973a57a0 20 read_log 194215'517 (194215'502) delete 74106d81/benchmark_data_storage0111-ib.pocstor_64498_object11247/head//22 by client.1049635.0:45450 2014-08-12 17:19:21.656410 2014-08-13 18:06:06.519919 7f5c973a57a0 20 read_log 194215'518 (194215'503) delete f725c581/benchmark_data_storage0111-ib.pocstor_64498_object12443/head//22 by client.1049635.0:46646 2014-08-12 17:19:22.035881 2014-08-13 18:06:06.519931 7f5c973a57a0 20 read_log 194215'519 (194215'504) delete 2b3e2d81/benchmark_data_storage0111-ib.pocstor_64498_object14452/head//22 by client.1049635.0:48655 2014-08-12 17:19:22.705833 2014-08-13 18:06:06.519944 7f5c973a57a0 20 read_log 194215'520 (194215'505) delete 6396d581/benchmark_data_storage0111-ib.pocstor_64498_object16192/head//22 by client.1049635.0:50395 2014-08-12 17:19:23.248446 2014-08-13 18:06:06.519956 7f5c973a57a0 20 read_log 194215'521 (194215'506) delete a2166d81/benchmark_data_storage0111-ib.pocstor_64498_object20000/head//22 by client.1049635.0:54203 2014-08-12 17:19:24.514357 2014-08-13 18:06:06.519968 7f5c973a57a0 20 read_log 194215'522 (194215'507) delete f73f2d81/benchmark_data_storage0111-ib.pocstor_64498_object21974/head//22 by client.1049635.0:56177 2014-08-12 17:19:25.223531 2014-08-13 18:06:06.519981 7f5c973a57a0 20 read_log 194215'523 (194215'508) delete 3ae62d81/benchmark_data_storage0111-ib.pocstor_64498_object23221/head//22 by client.1049635.0:57424 2014-08-12 17:19:25.698341 2014-08-13 18:06:06.519995 7f5c973a57a0 20 read_log 194215'524 (194215'509) delete bb607d81/benchmark_data_storage0111-ib.pocstor_64498_object24641/head//22 by client.1049635.0:58844 2014-08-12 17:19:26.189294 2014-08-13 18:06:06.520007 7f5c973a57a0 20 read_log 194215'525 (194215'510) delete c4f65d81/benchmark_data_storage0111-ib.pocstor_64498_object27772/head//22 by client.1049635.0:61975 2014-08-12 17:19:27.270115 2014-08-13 18:06:06.520019 7f5c973a57a0 20 read_log 194215'526 (194215'511) delete 94586581/benchmark_data_storage0111-ib.pocstor_64498_object28398/head//22 by client.1049635.0:62601 2014-08-12 17:19:27.489178 2014-08-13 18:06:06.520032 7f5c973a57a0 20 read_log 194215'527 (194215'512) delete c9722581/benchmark_data_storage0111-ib.pocstor_64498_object32664/head//22 by client.1049635.0:66867 2014-08-12 17:19:28.987628 2014-08-13 18:06:06.520135 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.520157 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=0 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.520173 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=0 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Initial 0.001890 0 0.000000 2014-08-13 18:06:06.520185 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=0 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.520194 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=0 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.520203 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] log((0'0,194215'527], crt=192888'497) 2014-08-13 18:06:06.520213 7f5c973a57a0 10 osd.4 194487 pgid 22.619 coll 22.619_head 2014-08-13 18:06:06.520356 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.619 2014-08-13 18:06:06.520366 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.520403 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.619(unlocked)] enter Initial 2014-08-13 18:06:06.520415 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.619(unlocked)] enter NotTrimming 2014-08-13 18:06:06.520584 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.520703 7f5c973a57a0 20 read_log 191648'29 (0'0) modify b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 by client.273158.0:1797 2014-07-30 14:08:55.376645 2014-08-13 18:06:06.520723 7f5c973a57a0 20 read_log 191648'35 (0'0) modify 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 by client.273158.0:2102 2014-07-30 14:08:56.673851 2014-08-13 18:06:06.520736 7f5c973a57a0 20 read_log 191648'155 (0'0) modify e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 by client.273158.0:9105 2014-07-30 14:09:31.580296 2014-08-13 18:06:06.520749 7f5c973a57a0 20 read_log 191648'157 (0'0) modify e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 by client.273158.0:9252 2014-07-30 14:09:32.163490 2014-08-13 18:06:06.520761 7f5c973a57a0 20 read_log 191648'200 (0'0) modify ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 by client.273158.0:11709 2014-07-30 14:09:45.618534 2014-08-13 18:06:06.520784 7f5c973a57a0 20 read_log 191648'207 (0'0) modify 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 by client.273158.0:12031 2014-07-30 14:09:47.326717 2014-08-13 18:06:06.520818 7f5c973a57a0 20 read_log 191648'215 (0'0) modify 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 by client.273158.0:12401 2014-07-30 14:09:49.584897 2014-08-13 18:06:06.520833 7f5c973a57a0 20 read_log 191648'216 (0'0) modify f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 by client.273158.0:12403 2014-07-30 14:09:49.597034 2014-08-13 18:06:06.520846 7f5c973a57a0 20 read_log 191648'222 (0'0) modify e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 by client.273158.0:12736 2014-07-30 14:09:51.156005 2014-08-13 18:06:06.520858 7f5c973a57a0 20 read_log 191648'295 (0'0) modify d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 by client.273158.0:18288 2014-07-30 14:10:19.550998 2014-08-13 18:06:06.520870 7f5c973a57a0 20 read_log 191648'298 (0'0) modify a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 by client.273158.0:18354 2014-07-30 14:10:19.908154 2014-08-13 18:06:06.520882 7f5c973a57a0 20 read_log 191648'361 (0'0) modify da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 by client.273158.0:22717 2014-07-30 14:10:47.933850 2014-08-13 18:06:06.520895 7f5c973a57a0 20 read_log 191727'366 (0'0) modify 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 by client.275175.0:618 2014-07-30 14:18:35.877657 2014-08-13 18:06:06.520907 7f5c973a57a0 20 read_log 191727'369 (0'0) modify 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 by client.275175.0:2413 2014-07-30 14:18:42.305242 2014-08-13 18:06:06.520919 7f5c973a57a0 20 read_log 191727'371 (0'0) modify fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 by client.275175.0:4043 2014-07-30 14:18:47.486778 2014-08-13 18:06:06.520931 7f5c973a57a0 20 read_log 191727'372 (0'0) modify 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 by client.275175.0:4443 2014-07-30 14:18:48.684121 2014-08-13 18:06:06.520943 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 by client.275175.0:5945 2014-07-30 14:18:53.087326 2014-08-13 18:06:06.520955 7f5c973a57a0 20 read_log 191727'374 (0'0) modify 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 by client.275175.0:6169 2014-07-30 14:18:53.780895 2014-08-13 18:06:06.520967 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 by client.275175.0:6426 2014-07-30 14:18:54.515886 2014-08-13 18:06:06.520980 7f5c973a57a0 20 read_log 191727'377 (0'0) modify 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 by client.275175.0:7435 2014-07-30 14:18:57.947408 2014-08-13 18:06:06.520992 7f5c973a57a0 20 read_log 191727'397 (0'0) modify 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 by client.275175.0:17498 2014-07-30 14:19:28.540203 2014-08-13 18:06:06.521004 7f5c973a57a0 20 read_log 191727'408 (0'0) modify 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 by client.275175.0:21276 2014-07-30 14:19:39.945185 2014-08-13 18:06:06.521016 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 by client.275175.0:26508 2014-07-30 14:19:56.383979 2014-08-13 18:06:06.521029 7f5c973a57a0 20 read_log 191727'420 (0'0) modify 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 by client.275175.0:28600 2014-07-30 14:20:02.711208 2014-08-13 18:06:06.521048 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 by client.275175.0:33054 2014-07-30 14:20:15.946725 2014-08-13 18:06:06.521061 7f5c973a57a0 20 read_log 191727'429 (0'0) modify dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 by client.275175.0:33392 2014-07-30 14:20:17.004800 2014-08-13 18:06:06.521073 7f5c973a57a0 20 read_log 191727'435 (0'0) modify 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 by client.275175.0:36750 2014-07-30 14:20:27.675461 2014-08-13 18:06:06.521085 7f5c973a57a0 20 read_log 191727'437 (0'0) modify d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 by client.275175.0:36944 2014-07-30 14:20:28.218693 2014-08-13 18:06:06.521097 7f5c973a57a0 20 read_log 191727'438 (0'0) modify 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 by client.275175.0:37179 2014-07-30 14:20:28.869719 2014-08-13 18:06:06.521114 7f5c973a57a0 20 read_log 192888'440 (0'0) modify d8799e19/benchmark_data_storage0111-ib.pocstor_35625_object2250/head//22 by client.940330.0:2251 2014-08-11 16:29:49.464596 2014-08-13 18:06:06.521127 7f5c973a57a0 20 read_log 192888'441 (0'0) modify a139de19/benchmark_data_storage0111-ib.pocstor_35625_object2379/head//22 by client.940330.0:2380 2014-08-11 16:29:49.922455 2014-08-13 18:06:06.521141 7f5c973a57a0 20 read_log 192888'442 (0'0) modify e01de19/benchmark_data_storage0111-ib.pocstor_35625_object2430/head//22 by client.940330.0:2431 2014-08-11 16:29:50.130489 2014-08-13 18:06:06.521162 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 3ded4619/benchmark_data_storage0111-ib.pocstor_35625_object3012/head//22 by client.940330.0:3013 2014-08-11 16:29:52.617984 2014-08-13 18:06:06.521174 7f5c973a57a0 20 read_log 192888'444 (0'0) modify c6684619/benchmark_data_storage0111-ib.pocstor_35625_object4761/head//22 by client.940330.0:4762 2014-08-11 16:29:59.190071 2014-08-13 18:06:06.521187 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 4a0eb619/benchmark_data_storage0111-ib.pocstor_35625_object6023/head//22 by client.940330.0:6024 2014-08-11 16:30:03.656338 2014-08-13 18:06:06.521201 7f5c973a57a0 20 read_log 192888'446 (0'0) modify f6306619/benchmark_data_storage0111-ib.pocstor_35625_object8063/head//22 by client.940330.0:8064 2014-08-11 16:30:11.061091 2014-08-13 18:06:06.521213 7f5c973a57a0 20 read_log 192888'447 (0'0) modify bee55619/benchmark_data_storage0111-ib.pocstor_35625_object10707/head//22 by client.940330.0:10708 2014-08-11 16:30:20.662088 2014-08-13 18:06:06.521225 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 29104619/benchmark_data_storage0111-ib.pocstor_35625_object18576/head//22 by client.940330.0:18577 2014-08-11 16:30:51.046149 2014-08-13 18:06:06.521237 7f5c973a57a0 20 read_log 192888'449 (0'0) modify ceda9e19/benchmark_data_storage0111-ib.pocstor_35625_object19159/head//22 by client.940330.0:19160 2014-08-11 16:30:53.251335 2014-08-13 18:06:06.521249 7f5c973a57a0 20 read_log 192888'450 (0'0) modify dcddfe19/benchmark_data_storage0111-ib.pocstor_35625_object24902/head//22 by client.940330.0:24903 2014-08-11 16:31:15.493569 2014-08-13 18:06:06.521261 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 3a4aa619/benchmark_data_storage0111-ib.pocstor_35625_object25750/head//22 by client.940330.0:25751 2014-08-11 16:31:18.810831 2014-08-13 18:06:06.521277 7f5c973a57a0 20 read_log 192888'452 (0'0) modify c6c53e19/benchmark_data_storage0111-ib.pocstor_35625_object25895/head//22 by client.940330.0:25896 2014-08-11 16:31:19.379818 2014-08-13 18:06:06.521291 7f5c973a57a0 20 read_log 192888'453 (0'0) modify fec94e19/benchmark_data_storage0111-ib.pocstor_35625_object29351/head//22 by client.940330.0:29352 2014-08-11 16:31:33.004646 2014-08-13 18:06:06.521315 7f5c973a57a0 20 read_log 192888'454 (192888'440) delete d8799e19/benchmark_data_storage0111-ib.pocstor_35625_object2250/head//22 by client.940330.0:33558 2014-08-11 16:31:42.056390 2014-08-13 18:06:06.521335 7f5c973a57a0 20 read_log 192888'455 (192888'441) delete a139de19/benchmark_data_storage0111-ib.pocstor_35625_object2379/head//22 by client.940330.0:33687 2014-08-11 16:31:42.090135 2014-08-13 18:06:06.521348 7f5c973a57a0 20 read_log 192888'456 (192888'442) delete e01de19/benchmark_data_storage0111-ib.pocstor_35625_object2430/head//22 by client.940330.0:33738 2014-08-11 16:31:42.103048 2014-08-13 18:06:06.521361 7f5c973a57a0 20 read_log 192888'457 (192888'443) delete 3ded4619/benchmark_data_storage0111-ib.pocstor_35625_object3012/head//22 by client.940330.0:34320 2014-08-11 16:31:42.253654 2014-08-13 18:06:06.521373 7f5c973a57a0 20 read_log 192888'458 (192888'444) delete c6684619/benchmark_data_storage0111-ib.pocstor_35625_object4761/head//22 by client.940330.0:36069 2014-08-11 16:31:42.756067 2014-08-13 18:06:06.521385 7f5c973a57a0 20 read_log 192888'459 (192888'445) delete 4a0eb619/benchmark_data_storage0111-ib.pocstor_35625_object6023/head//22 by client.940330.0:37331 2014-08-11 16:31:43.082103 2014-08-13 18:06:06.521397 7f5c973a57a0 20 read_log 192888'460 (192888'446) delete f6306619/benchmark_data_storage0111-ib.pocstor_35625_object8063/head//22 by client.940330.0:39371 2014-08-11 16:31:43.608541 2014-08-13 18:06:06.521424 7f5c973a57a0 20 read_log 192888'461 (192888'447) delete bee55619/benchmark_data_storage0111-ib.pocstor_35625_object10707/head//22 by client.940330.0:42015 2014-08-11 16:31:44.314017 2014-08-13 18:06:06.521437 7f5c973a57a0 20 read_log 192888'462 (192888'448) delete 29104619/benchmark_data_storage0111-ib.pocstor_35625_object18576/head//22 by client.940330.0:49884 2014-08-11 16:31:46.467810 2014-08-13 18:06:06.521455 7f5c973a57a0 20 read_log 192888'463 (192888'449) delete ceda9e19/benchmark_data_storage0111-ib.pocstor_35625_object19159/head//22 by client.940330.0:50467 2014-08-11 16:31:46.623093 2014-08-13 18:06:06.521475 7f5c973a57a0 20 read_log 192888'464 (192888'450) delete dcddfe19/benchmark_data_storage0111-ib.pocstor_35625_object24902/head//22 by client.940330.0:56210 2014-08-11 16:31:48.310835 2014-08-13 18:06:06.521490 7f5c973a57a0 20 read_log 192888'465 (192888'451) delete 3a4aa619/benchmark_data_storage0111-ib.pocstor_35625_object25750/head//22 by client.940330.0:57058 2014-08-11 16:31:48.573647 2014-08-13 18:06:06.521503 7f5c973a57a0 20 read_log 192888'466 (192888'452) delete c6c53e19/benchmark_data_storage0111-ib.pocstor_35625_object25895/head//22 by client.940330.0:57203 2014-08-11 16:31:48.615773 2014-08-13 18:06:06.521526 7f5c973a57a0 20 read_log 192888'467 (192888'453) delete fec94e19/benchmark_data_storage0111-ib.pocstor_35625_object29351/head//22 by client.940330.0:60659 2014-08-11 16:31:49.569061 2014-08-13 18:06:06.521539 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 90dbce19/benchmark_data_storage0111-ib.pocstor_64498_object2782/head//22 by client.1049635.0:2783 2014-08-12 17:17:26.077991 2014-08-13 18:06:06.521551 7f5c973a57a0 20 read_log 194215'469 (0'0) modify d0476e19/benchmark_data_storage0111-ib.pocstor_64498_object4601/head//22 by client.1049635.0:4602 2014-08-12 17:17:31.776557 2014-08-13 18:06:06.521564 7f5c973a57a0 20 read_log 194215'470 (0'0) modify 568b619/benchmark_data_storage0111-ib.pocstor_64498_object4720/head//22 by client.1049635.0:4721 2014-08-12 17:17:32.102126 2014-08-13 18:06:06.521576 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 41a81619/benchmark_data_storage0111-ib.pocstor_64498_object6022/head//22 by client.1049635.0:6023 2014-08-12 17:17:36.139569 2014-08-13 18:06:06.521588 7f5c973a57a0 20 read_log 194215'472 (0'0) modify f9b9ee19/benchmark_data_storage0111-ib.pocstor_64498_object10725/head//22 by client.1049635.0:10726 2014-08-12 17:17:50.306474 2014-08-13 18:06:06.521613 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 93b14e19/benchmark_data_storage0111-ib.pocstor_64498_object11619/head//22 by client.1049635.0:11620 2014-08-12 17:17:53.269704 2014-08-13 18:06:06.521628 7f5c973a57a0 20 read_log 194215'474 (0'0) modify 5e8eee19/benchmark_data_storage0111-ib.pocstor_64498_object12617/head//22 by client.1049635.0:12618 2014-08-12 17:17:56.215775 2014-08-13 18:06:06.521641 7f5c973a57a0 20 read_log 194215'475 (0'0) modify eb67c619/benchmark_data_storage0111-ib.pocstor_64498_object12901/head//22 by client.1049635.0:12902 2014-08-12 17:17:57.081811 2014-08-13 18:06:06.521653 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 58943e19/benchmark_data_storage0111-ib.pocstor_64498_object14044/head//22 by client.1049635.0:14045 2014-08-12 17:18:00.657127 2014-08-13 18:06:06.521674 7f5c973a57a0 20 read_log 194215'477 (0'0) modify 5c9ebe19/benchmark_data_storage0111-ib.pocstor_64498_object15633/head//22 by client.1049635.0:15634 2014-08-12 17:18:06.212147 2014-08-13 18:06:06.521690 7f5c973a57a0 20 read_log 194215'478 (0'0) modify 6b2cde19/benchmark_data_storage0111-ib.pocstor_64498_object15829/head//22 by client.1049635.0:15830 2014-08-12 17:18:06.915291 2014-08-13 18:06:06.521702 7f5c973a57a0 20 read_log 194215'479 (0'0) modify eb12e19/benchmark_data_storage0111-ib.pocstor_64498_object16157/head//22 by client.1049635.0:16158 2014-08-12 17:18:08.151499 2014-08-13 18:06:06.521714 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 7e639619/benchmark_data_storage0111-ib.pocstor_64498_object17752/head//22 by client.1049635.0:17753 2014-08-12 17:18:14.217650 2014-08-13 18:06:06.521727 7f5c973a57a0 20 read_log 194215'481 (0'0) modify e9bed619/benchmark_data_storage0111-ib.pocstor_64498_object21543/head//22 by client.1049635.0:21544 2014-08-12 17:18:28.615139 2014-08-13 18:06:06.521739 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 5f294e19/benchmark_data_storage0111-ib.pocstor_64498_object25229/head//22 by client.1049635.0:25230 2014-08-12 17:18:42.562190 2014-08-13 18:06:06.521751 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 25894619/benchmark_data_storage0111-ib.pocstor_64498_object26112/head//22 by client.1049635.0:26113 2014-08-12 17:18:45.842059 2014-08-13 18:06:06.521763 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 2c8d6619/benchmark_data_storage0111-ib.pocstor_64498_object29086/head//22 by client.1049635.0:29087 2014-08-12 17:18:56.914299 2014-08-13 18:06:06.521777 7f5c973a57a0 20 read_log 194215'485 (0'0) modify a48d1619/benchmark_data_storage0111-ib.pocstor_64498_object29131/head//22 by client.1049635.0:29132 2014-08-12 17:18:57.084320 2014-08-13 18:06:06.521790 7f5c973a57a0 20 read_log 194215'486 (0'0) modify b4bc4e19/benchmark_data_storage0111-ib.pocstor_64498_object29243/head//22 by client.1049635.0:29244 2014-08-12 17:18:57.498700 2014-08-13 18:06:06.521802 7f5c973a57a0 20 read_log 194215'487 (0'0) modify a5b04e19/benchmark_data_storage0111-ib.pocstor_64498_object29441/head//22 by client.1049635.0:29442 2014-08-12 17:18:58.287329 2014-08-13 18:06:06.521814 7f5c973a57a0 20 read_log 194215'488 (0'0) modify cef83e19/benchmark_data_storage0111-ib.pocstor_64498_object30080/head//22 by client.1049635.0:30081 2014-08-12 17:19:00.755689 2014-08-13 18:06:06.521826 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 4b5fee19/benchmark_data_storage0111-ib.pocstor_64498_object32902/head//22 by client.1049635.0:32903 2014-08-12 17:19:12.024870 2014-08-13 18:06:06.521839 7f5c973a57a0 20 read_log 194215'490 (194215'468) delete 90dbce19/benchmark_data_storage0111-ib.pocstor_64498_object2782/head//22 by client.1049635.0:36985 2014-08-12 17:19:18.893947 2014-08-13 18:06:06.521851 7f5c973a57a0 20 read_log 194215'491 (194215'469) delete d0476e19/benchmark_data_storage0111-ib.pocstor_64498_object4601/head//22 by client.1049635.0:38804 2014-08-12 17:19:19.469360 2014-08-13 18:06:06.521863 7f5c973a57a0 20 read_log 194215'492 (194215'470) delete 568b619/benchmark_data_storage0111-ib.pocstor_64498_object4720/head//22 by client.1049635.0:38923 2014-08-12 17:19:19.508658 2014-08-13 18:06:06.521883 7f5c973a57a0 20 read_log 194215'493 (194215'471) delete 41a81619/benchmark_data_storage0111-ib.pocstor_64498_object6022/head//22 by client.1049635.0:40225 2014-08-12 17:19:19.965279 2014-08-13 18:06:06.521898 7f5c973a57a0 20 read_log 194215'494 (194215'472) delete f9b9ee19/benchmark_data_storage0111-ib.pocstor_64498_object10725/head//22 by client.1049635.0:44928 2014-08-12 17:19:21.492121 2014-08-13 18:06:06.521911 7f5c973a57a0 20 read_log 194215'495 (194215'473) delete 93b14e19/benchmark_data_storage0111-ib.pocstor_64498_object11619/head//22 by client.1049635.0:45822 2014-08-12 17:19:21.772387 2014-08-13 18:06:06.521925 7f5c973a57a0 20 read_log 194215'496 (194215'474) delete 5e8eee19/benchmark_data_storage0111-ib.pocstor_64498_object12617/head//22 by client.1049635.0:46820 2014-08-12 17:19:22.089537 2014-08-13 18:06:06.521940 7f5c973a57a0 20 read_log 194215'497 (194215'475) delete eb67c619/benchmark_data_storage0111-ib.pocstor_64498_object12901/head//22 by client.1049635.0:47104 2014-08-12 17:19:22.182088 2014-08-13 18:06:06.521952 7f5c973a57a0 20 read_log 194215'498 (194215'476) delete 58943e19/benchmark_data_storage0111-ib.pocstor_64498_object14044/head//22 by client.1049635.0:48247 2014-08-12 17:19:22.577446 2014-08-13 18:06:06.521966 7f5c973a57a0 20 read_log 194215'499 (194215'477) delete 5c9ebe19/benchmark_data_storage0111-ib.pocstor_64498_object15633/head//22 by client.1049635.0:49836 2014-08-12 17:19:23.074197 2014-08-13 18:06:06.521979 7f5c973a57a0 20 read_log 194215'500 (194215'478) delete 6b2cde19/benchmark_data_storage0111-ib.pocstor_64498_object15829/head//22 by client.1049635.0:50032 2014-08-12 17:19:23.129989 2014-08-13 18:06:06.521993 7f5c973a57a0 20 read_log 194215'501 (194215'479) delete eb12e19/benchmark_data_storage0111-ib.pocstor_64498_object16157/head//22 by client.1049635.0:50360 2014-08-12 17:19:23.236921 2014-08-13 18:06:06.522006 7f5c973a57a0 20 read_log 194215'502 (194215'480) delete 7e639619/benchmark_data_storage0111-ib.pocstor_64498_object17752/head//22 by client.1049635.0:51955 2014-08-12 17:19:23.743676 2014-08-13 18:06:06.522022 7f5c973a57a0 20 read_log 194215'503 (194215'481) delete e9bed619/benchmark_data_storage0111-ib.pocstor_64498_object21543/head//22 by client.1049635.0:55746 2014-08-12 17:19:25.068569 2014-08-13 18:06:06.522036 7f5c973a57a0 20 read_log 194215'504 (194215'482) delete 5f294e19/benchmark_data_storage0111-ib.pocstor_64498_object25229/head//22 by client.1049635.0:59432 2014-08-12 17:19:26.398253 2014-08-13 18:06:06.522051 7f5c973a57a0 20 read_log 194215'505 (194215'483) delete 25894619/benchmark_data_storage0111-ib.pocstor_64498_object26112/head//22 by client.1049635.0:60315 2014-08-12 17:19:26.692702 2014-08-13 18:06:06.522068 7f5c973a57a0 20 read_log 194215'506 (194215'484) delete 2c8d6619/benchmark_data_storage0111-ib.pocstor_64498_object29086/head//22 by client.1049635.0:63289 2014-08-12 17:19:27.733861 2014-08-13 18:06:06.522081 7f5c973a57a0 20 read_log 194215'507 (194215'485) delete a48d1619/benchmark_data_storage0111-ib.pocstor_64498_object29131/head//22 by client.1049635.0:63334 2014-08-12 17:19:27.749972 2014-08-13 18:06:06.522093 7f5c973a57a0 20 read_log 194215'508 (194215'486) delete b4bc4e19/benchmark_data_storage0111-ib.pocstor_64498_object29243/head//22 by client.1049635.0:63446 2014-08-12 17:19:27.792695 2014-08-13 18:06:06.522108 7f5c973a57a0 20 read_log 194215'509 (194215'487) delete a5b04e19/benchmark_data_storage0111-ib.pocstor_64498_object29441/head//22 by client.1049635.0:63644 2014-08-12 17:19:27.864105 2014-08-13 18:06:06.522137 7f5c973a57a0 20 read_log 194215'510 (194215'488) delete cef83e19/benchmark_data_storage0111-ib.pocstor_64498_object30080/head//22 by client.1049635.0:64283 2014-08-12 17:19:28.072083 2014-08-13 18:06:06.522161 7f5c973a57a0 20 read_log 194215'511 (194215'489) delete 4b5fee19/benchmark_data_storage0111-ib.pocstor_64498_object32902/head//22 by client.1049635.0:67105 2014-08-12 17:19:29.083326 2014-08-13 18:06:06.522313 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.522333 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=0 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.522348 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=0 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001945 0 0.000000 2014-08-13 18:06:06.522361 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=0 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.522369 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=0 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.522379 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'511], crt=191727'439) 2014-08-13 18:06:06.522389 7f5c973a57a0 10 osd.4 194487 pgid 22.630 coll 22.630_head 2014-08-13 18:06:06.522579 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.630 2014-08-13 18:06:06.522590 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.522618 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.630(unlocked)] enter Initial 2014-08-13 18:06:06.522629 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.630(unlocked)] enter NotTrimming 2014-08-13 18:06:06.522774 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.522890 7f5c973a57a0 20 read_log 191648'66 (0'0) modify ccb6de30/benchmark_data_storage0111-ib.pocstor_40745_object4310/head//22 by client.273158.0:4311 2014-07-30 14:09:08.019610 2014-08-13 18:06:06.522910 7f5c973a57a0 20 read_log 191648'186 (0'0) modify edbf4e30/benchmark_data_storage0111-ib.pocstor_40745_object11670/head//22 by client.273158.0:11671 2014-07-30 14:09:45.475408 2014-08-13 18:06:06.522923 7f5c973a57a0 20 read_log 191648'188 (0'0) modify 4c565630/benchmark_data_storage0111-ib.pocstor_40745_object11781/head//22 by client.273158.0:11782 2014-07-30 14:09:45.941658 2014-08-13 18:06:06.522935 7f5c973a57a0 20 read_log 191648'196 (0'0) modify c197f630/benchmark_data_storage0111-ib.pocstor_40745_object12230/head//22 by client.273158.0:12231 2014-07-30 14:09:48.741592 2014-08-13 18:06:06.522948 7f5c973a57a0 20 read_log 191648'198 (0'0) modify 2fac6e30/benchmark_data_storage0111-ib.pocstor_40745_object12340/head//22 by client.273158.0:12341 2014-07-30 14:09:49.247038 2014-08-13 18:06:06.522960 7f5c973a57a0 20 read_log 191648'212 (0'0) modify 49753e30/benchmark_data_storage0111-ib.pocstor_40745_object13073/head//22 by client.273158.0:13074 2014-07-30 14:09:52.795488 2014-08-13 18:06:06.522972 7f5c973a57a0 20 read_log 191648'242 (0'0) modify 5f5dae30/benchmark_data_storage0111-ib.pocstor_40745_object14641/head//22 by client.273158.0:14642 2014-07-30 14:10:00.755469 2014-08-13 18:06:06.522984 7f5c973a57a0 20 read_log 191648'251 (0'0) modify 3560de30/benchmark_data_storage0111-ib.pocstor_40745_object15315/head//22 by client.273158.0:15316 2014-07-30 14:10:03.924044 2014-08-13 18:06:06.522996 7f5c973a57a0 20 read_log 191648'256 (0'0) modify 3e3d2630/benchmark_data_storage0111-ib.pocstor_40745_object15942/head//22 by client.273158.0:15943 2014-07-30 14:10:07.226671 2014-08-13 18:06:06.523008 7f5c973a57a0 20 read_log 191648'287 (0'0) modify 9065630/benchmark_data_storage0111-ib.pocstor_40745_object17781/head//22 by client.273158.0:17782 2014-07-30 14:10:16.980555 2014-08-13 18:06:06.523020 7f5c973a57a0 20 read_log 191648'316 (0'0) modify 2efb8630/benchmark_data_storage0111-ib.pocstor_40745_object19161/head//22 by client.273158.0:19162 2014-07-30 14:10:24.241436 2014-08-13 18:06:06.523032 7f5c973a57a0 20 read_log 191648'323 (0'0) modify 997c3630/benchmark_data_storage0111-ib.pocstor_40745_object19577/head//22 by client.273158.0:19578 2014-07-30 14:10:27.016986 2014-08-13 18:06:06.523044 7f5c973a57a0 20 read_log 191648'330 (0'0) modify bfb22630/benchmark_data_storage0111-ib.pocstor_40745_object20063/head//22 by client.273158.0:20064 2014-07-30 14:10:31.541128 2014-08-13 18:06:06.523056 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 90abe30/benchmark_data_storage0111-ib.pocstor_51285_object7394/head//22 by client.275175.0:7395 2014-07-30 14:18:57.836886 2014-08-13 18:06:06.523068 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 9e946e30/benchmark_data_storage0111-ib.pocstor_51285_object10525/head//22 by client.275175.0:10526 2014-07-30 14:19:06.711768 2014-08-13 18:06:06.523080 7f5c973a57a0 20 read_log 191727'395 (0'0) modify a4132e30/benchmark_data_storage0111-ib.pocstor_51285_object10791/head//22 by client.275175.0:10792 2014-07-30 14:19:07.459118 2014-08-13 18:06:06.523100 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 890d3630/benchmark_data_storage0111-ib.pocstor_51285_object10893/head//22 by client.275175.0:10894 2014-07-30 14:19:07.745140 2014-08-13 18:06:06.523113 7f5c973a57a0 20 read_log 191727'400 (0'0) modify ff3ece30/benchmark_data_storage0111-ib.pocstor_51285_object13102/head//22 by client.275175.0:13103 2014-07-30 14:19:14.982817 2014-08-13 18:06:06.523127 7f5c973a57a0 20 read_log 191727'401 (0'0) modify 80578e30/benchmark_data_storage0111-ib.pocstor_51285_object13945/head//22 by client.275175.0:13946 2014-07-30 14:19:17.604438 2014-08-13 18:06:06.523140 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 2699b630/benchmark_data_storage0111-ib.pocstor_51285_object15300/head//22 by client.275175.0:15301 2014-07-30 14:19:21.673280 2014-08-13 18:06:06.523161 7f5c973a57a0 20 read_log 191727'408 (0'0) modify 22037e30/benchmark_data_storage0111-ib.pocstor_51285_object18542/head//22 by client.275175.0:18543 2014-07-30 14:19:31.604818 2014-08-13 18:06:06.523176 7f5c973a57a0 20 read_log 191727'411 (0'0) modify 55029630/benchmark_data_storage0111-ib.pocstor_51285_object19583/head//22 by client.275175.0:19584 2014-07-30 14:19:35.009655 2014-08-13 18:06:06.523188 7f5c973a57a0 20 read_log 191727'417 (0'0) modify d804ce30/benchmark_data_storage0111-ib.pocstor_51285_object23812/head//22 by client.275175.0:23813 2014-07-30 14:19:48.303786 2014-08-13 18:06:06.523200 7f5c973a57a0 20 read_log 191727'423 (0'0) modify 1aeece30/benchmark_data_storage0111-ib.pocstor_51285_object26166/head//22 by client.275175.0:26167 2014-07-30 14:19:55.341759 2014-08-13 18:06:06.523212 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 5c244e30/benchmark_data_storage0111-ib.pocstor_51285_object27581/head//22 by client.275175.0:27582 2014-07-30 14:19:59.465720 2014-08-13 18:06:06.523224 7f5c973a57a0 20 read_log 191727'428 (0'0) modify f0465630/benchmark_data_storage0111-ib.pocstor_51285_object27756/head//22 by client.275175.0:27757 2014-07-30 14:20:00.030483 2014-08-13 18:06:06.523236 7f5c973a57a0 20 read_log 191727'431 (0'0) modify 7062f630/benchmark_data_storage0111-ib.pocstor_51285_object29294/head//22 by client.275175.0:29295 2014-07-30 14:20:04.802449 2014-08-13 18:06:06.523248 7f5c973a57a0 20 read_log 191727'432 (0'0) modify 5c3ed630/benchmark_data_storage0111-ib.pocstor_51285_object29583/head//22 by client.275175.0:29584 2014-07-30 14:20:05.711911 2014-08-13 18:06:06.523260 7f5c973a57a0 20 read_log 191727'442 (0'0) modify 2f19fe30/benchmark_data_storage0111-ib.pocstor_51285_object33514/head//22 by client.275175.0:33515 2014-07-30 14:20:17.549561 2014-08-13 18:06:06.523272 7f5c973a57a0 20 read_log 191727'446 (0'0) modify f02a7e30/benchmark_data_storage0111-ib.pocstor_51285_object35075/head//22 by client.275175.0:35076 2014-07-30 14:20:22.197940 2014-08-13 18:06:06.523284 7f5c973a57a0 20 read_log 191727'448 (0'0) modify defca630/benchmark_data_storage0111-ib.pocstor_51285_object35754/head//22 by client.275175.0:35755 2014-07-30 14:20:24.466230 2014-08-13 18:06:06.523297 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 1e072e30/benchmark_data_storage0111-ib.pocstor_35625_object253/head//22 by client.940330.0:254 2014-08-11 16:29:41.729995 2014-08-13 18:06:06.523333 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 86e1d630/benchmark_data_storage0111-ib.pocstor_35625_object1389/head//22 by client.940330.0:1390 2014-08-11 16:29:46.243928 2014-08-13 18:06:06.523346 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 14227630/benchmark_data_storage0111-ib.pocstor_35625_object6455/head//22 by client.940330.0:6456 2014-08-11 16:30:05.186232 2014-08-13 18:06:06.523358 7f5c973a57a0 20 read_log 192888'455 (0'0) modify 6dee7e30/benchmark_data_storage0111-ib.pocstor_35625_object7227/head//22 by client.940330.0:7228 2014-08-11 16:30:08.121350 2014-08-13 18:06:06.523378 7f5c973a57a0 20 read_log 192888'456 (0'0) modify 886a7e30/benchmark_data_storage0111-ib.pocstor_35625_object7279/head//22 by client.940330.0:7280 2014-08-11 16:30:08.305273 2014-08-13 18:06:06.523393 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 46c8e630/benchmark_data_storage0111-ib.pocstor_35625_object10117/head//22 by client.940330.0:10118 2014-08-11 16:30:18.318983 2014-08-13 18:06:06.523405 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 65cf7e30/benchmark_data_storage0111-ib.pocstor_35625_object12056/head//22 by client.940330.0:12057 2014-08-11 16:30:26.150625 2014-08-13 18:06:06.523418 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 506cce30/benchmark_data_storage0111-ib.pocstor_35625_object13218/head//22 by client.940330.0:13219 2014-08-11 16:30:30.735058 2014-08-13 18:06:06.523430 7f5c973a57a0 20 read_log 192888'460 (0'0) modify c2866630/benchmark_data_storage0111-ib.pocstor_35625_object14065/head//22 by client.940330.0:14066 2014-08-11 16:30:33.891881 2014-08-13 18:06:06.523442 7f5c973a57a0 20 read_log 192888'461 (0'0) modify ec85a630/benchmark_data_storage0111-ib.pocstor_35625_object16118/head//22 by client.940330.0:16119 2014-08-11 16:30:41.674790 2014-08-13 18:06:06.523454 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 9efb6630/benchmark_data_storage0111-ib.pocstor_35625_object17793/head//22 by client.940330.0:17794 2014-08-11 16:30:47.943539 2014-08-13 18:06:06.523466 7f5c973a57a0 20 read_log 192888'463 (0'0) modify ca56fe30/benchmark_data_storage0111-ib.pocstor_35625_object21019/head//22 by client.940330.0:21020 2014-08-11 16:31:00.879709 2014-08-13 18:06:06.523478 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 2ec3de30/benchmark_data_storage0111-ib.pocstor_35625_object21235/head//22 by client.940330.0:21236 2014-08-11 16:31:01.708108 2014-08-13 18:06:06.523490 7f5c973a57a0 20 read_log 192888'465 (0'0) modify 5652b630/benchmark_data_storage0111-ib.pocstor_35625_object23775/head//22 by client.940330.0:23776 2014-08-11 16:31:11.321911 2014-08-13 18:06:06.523502 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 17f87e30/benchmark_data_storage0111-ib.pocstor_35625_object24460/head//22 by client.940330.0:24461 2014-08-11 16:31:13.820467 2014-08-13 18:06:06.523514 7f5c973a57a0 20 read_log 192888'467 (0'0) modify e3936e30/benchmark_data_storage0111-ib.pocstor_35625_object26293/head//22 by client.940330.0:26294 2014-08-11 16:31:20.939065 2014-08-13 18:06:06.523526 7f5c973a57a0 20 read_log 192888'468 (192888'452) delete 1e072e30/benchmark_data_storage0111-ib.pocstor_35625_object253/head//22 by client.940330.0:31561 2014-08-11 16:31:41.546269 2014-08-13 18:06:06.523540 7f5c973a57a0 20 read_log 192888'469 (192888'453) delete 86e1d630/benchmark_data_storage0111-ib.pocstor_35625_object1389/head//22 by client.940330.0:32697 2014-08-11 16:31:41.841847 2014-08-13 18:06:06.523552 7f5c973a57a0 20 read_log 192888'470 (192888'454) delete 14227630/benchmark_data_storage0111-ib.pocstor_35625_object6455/head//22 by client.940330.0:37763 2014-08-11 16:31:43.196760 2014-08-13 18:06:06.523565 7f5c973a57a0 20 read_log 192888'471 (192888'455) delete 6dee7e30/benchmark_data_storage0111-ib.pocstor_35625_object7227/head//22 by client.940330.0:38535 2014-08-11 16:31:43.392899 2014-08-13 18:06:06.523577 7f5c973a57a0 20 read_log 192888'472 (192888'456) delete 886a7e30/benchmark_data_storage0111-ib.pocstor_35625_object7279/head//22 by client.940330.0:38587 2014-08-11 16:31:43.407930 2014-08-13 18:06:06.523590 7f5c973a57a0 20 read_log 192888'473 (192888'457) delete 46c8e630/benchmark_data_storage0111-ib.pocstor_35625_object10117/head//22 by client.940330.0:41425 2014-08-11 16:31:44.147606 2014-08-13 18:06:06.523602 7f5c973a57a0 20 read_log 192888'474 (192888'458) delete 65cf7e30/benchmark_data_storage0111-ib.pocstor_35625_object12056/head//22 by client.940330.0:43364 2014-08-11 16:31:44.671607 2014-08-13 18:06:06.523617 7f5c973a57a0 20 read_log 192888'475 (192888'459) delete 506cce30/benchmark_data_storage0111-ib.pocstor_35625_object13218/head//22 by client.940330.0:44526 2014-08-11 16:31:44.980844 2014-08-13 18:06:06.523636 7f5c973a57a0 20 read_log 192888'476 (192888'460) delete c2866630/benchmark_data_storage0111-ib.pocstor_35625_object14065/head//22 by client.940330.0:45373 2014-08-11 16:31:45.200174 2014-08-13 18:06:06.523651 7f5c973a57a0 20 read_log 192888'477 (192888'461) delete ec85a630/benchmark_data_storage0111-ib.pocstor_35625_object16118/head//22 by client.940330.0:47426 2014-08-11 16:31:45.756335 2014-08-13 18:06:06.523663 7f5c973a57a0 20 read_log 192888'478 (192888'462) delete 9efb6630/benchmark_data_storage0111-ib.pocstor_35625_object17793/head//22 by client.940330.0:49101 2014-08-11 16:31:46.220009 2014-08-13 18:06:06.523677 7f5c973a57a0 20 read_log 192888'479 (192888'463) delete ca56fe30/benchmark_data_storage0111-ib.pocstor_35625_object21019/head//22 by client.940330.0:52327 2014-08-11 16:31:47.167955 2014-08-13 18:06:06.523690 7f5c973a57a0 20 read_log 192888'480 (192888'464) delete 2ec3de30/benchmark_data_storage0111-ib.pocstor_35625_object21235/head//22 by client.940330.0:52543 2014-08-11 16:31:47.223453 2014-08-13 18:06:06.523704 7f5c973a57a0 20 read_log 192888'481 (192888'465) delete 5652b630/benchmark_data_storage0111-ib.pocstor_35625_object23775/head//22 by client.940330.0:55083 2014-08-11 16:31:47.970380 2014-08-13 18:06:06.523716 7f5c973a57a0 20 read_log 192888'482 (192888'466) delete 17f87e30/benchmark_data_storage0111-ib.pocstor_35625_object24460/head//22 by client.940330.0:55768 2014-08-11 16:31:48.175319 2014-08-13 18:06:06.523730 7f5c973a57a0 20 read_log 192888'483 (192888'467) delete e3936e30/benchmark_data_storage0111-ib.pocstor_35625_object26293/head//22 by client.940330.0:57601 2014-08-11 16:31:48.731381 2014-08-13 18:06:06.523743 7f5c973a57a0 20 read_log 194215'484 (0'0) modify e6629e30/benchmark_data_storage0111-ib.pocstor_64498_object778/head//22 by client.1049635.0:779 2014-08-12 17:17:19.634853 2014-08-13 18:06:06.523757 7f5c973a57a0 20 read_log 194215'485 (0'0) modify c9e6630/benchmark_data_storage0111-ib.pocstor_64498_object4406/head//22 by client.1049635.0:4407 2014-08-12 17:17:31.240668 2014-08-13 18:06:06.523769 7f5c973a57a0 20 read_log 194215'486 (0'0) modify 3f817630/benchmark_data_storage0111-ib.pocstor_64498_object4640/head//22 by client.1049635.0:4641 2014-08-12 17:17:31.884887 2014-08-13 18:06:06.523782 7f5c973a57a0 20 read_log 194215'487 (0'0) modify a8bdf630/benchmark_data_storage0111-ib.pocstor_64498_object4686/head//22 by client.1049635.0:4687 2014-08-12 17:17:32.009525 2014-08-13 18:06:06.523797 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 12e28630/benchmark_data_storage0111-ib.pocstor_64498_object4929/head//22 by client.1049635.0:4930 2014-08-12 17:17:32.733804 2014-08-13 18:06:06.523813 7f5c973a57a0 20 read_log 194215'489 (0'0) modify e80d5e30/benchmark_data_storage0111-ib.pocstor_64498_object6302/head//22 by client.1049635.0:6303 2014-08-12 17:17:36.996353 2014-08-13 18:06:06.523828 7f5c973a57a0 20 read_log 194215'490 (0'0) modify fed24630/benchmark_data_storage0111-ib.pocstor_64498_object7465/head//22 by client.1049635.0:7466 2014-08-12 17:17:40.438736 2014-08-13 18:06:06.523840 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 6aba4630/benchmark_data_storage0111-ib.pocstor_64498_object14935/head//22 by client.1049635.0:14936 2014-08-12 17:18:03.754370 2014-08-13 18:06:06.523854 7f5c973a57a0 20 read_log 194215'492 (0'0) modify 25bb1e30/benchmark_data_storage0111-ib.pocstor_64498_object19012/head//22 by client.1049635.0:19013 2014-08-12 17:18:18.984034 2014-08-13 18:06:06.523869 7f5c973a57a0 20 read_log 194215'493 (0'0) modify ef399630/benchmark_data_storage0111-ib.pocstor_64498_object19842/head//22 by client.1049635.0:19843 2014-08-12 17:18:22.189380 2014-08-13 18:06:06.523884 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 7e542630/benchmark_data_storage0111-ib.pocstor_64498_object23129/head//22 by client.1049635.0:23130 2014-08-12 17:18:34.613226 2014-08-13 18:06:06.523924 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 5c645630/benchmark_data_storage0111-ib.pocstor_64498_object24835/head//22 by client.1049635.0:24836 2014-08-12 17:18:41.012317 2014-08-13 18:06:06.523941 7f5c973a57a0 20 read_log 194215'496 (0'0) modify dac93e30/benchmark_data_storage0111-ib.pocstor_64498_object26247/head//22 by client.1049635.0:26248 2014-08-12 17:18:46.343797 2014-08-13 18:06:06.523953 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 18270630/benchmark_data_storage0111-ib.pocstor_64498_object26408/head//22 by client.1049635.0:26409 2014-08-12 17:18:46.942457 2014-08-13 18:06:06.523989 7f5c973a57a0 20 read_log 194215'498 (0'0) modify 58c28630/benchmark_data_storage0111-ib.pocstor_64498_object28675/head//22 by client.1049635.0:28676 2014-08-12 17:18:55.393165 2014-08-13 18:06:06.524002 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 6dc85630/benchmark_data_storage0111-ib.pocstor_64498_object31292/head//22 by client.1049635.0:31293 2014-08-12 17:19:05.697133 2014-08-13 18:06:06.524015 7f5c973a57a0 20 read_log 194215'500 (0'0) modify 9a8bc630/benchmark_data_storage0111-ib.pocstor_64498_object31461/head//22 by client.1049635.0:31462 2014-08-12 17:19:06.448808 2014-08-13 18:06:06.524027 7f5c973a57a0 20 read_log 194215'501 (194215'484) delete e6629e30/benchmark_data_storage0111-ib.pocstor_64498_object778/head//22 by client.1049635.0:34981 2014-08-12 17:19:18.245287 2014-08-13 18:06:06.524049 7f5c973a57a0 20 read_log 194215'502 (194215'485) delete c9e6630/benchmark_data_storage0111-ib.pocstor_64498_object4406/head//22 by client.1049635.0:38609 2014-08-12 17:19:19.408174 2014-08-13 18:06:06.524069 7f5c973a57a0 20 read_log 194215'503 (194215'486) delete 3f817630/benchmark_data_storage0111-ib.pocstor_64498_object4640/head//22 by client.1049635.0:38843 2014-08-12 17:19:19.482886 2014-08-13 18:06:06.524082 7f5c973a57a0 20 read_log 194215'504 (194215'487) delete a8bdf630/benchmark_data_storage0111-ib.pocstor_64498_object4686/head//22 by client.1049635.0:38889 2014-08-12 17:19:19.496888 2014-08-13 18:06:06.524106 7f5c973a57a0 20 read_log 194215'505 (194215'488) delete 12e28630/benchmark_data_storage0111-ib.pocstor_64498_object4929/head//22 by client.1049635.0:39132 2014-08-12 17:19:19.579881 2014-08-13 18:06:06.524118 7f5c973a57a0 20 read_log 194215'506 (194215'489) delete e80d5e30/benchmark_data_storage0111-ib.pocstor_64498_object6302/head//22 by client.1049635.0:40505 2014-08-12 17:19:20.055383 2014-08-13 18:06:06.524131 7f5c973a57a0 20 read_log 194215'507 (194215'490) delete fed24630/benchmark_data_storage0111-ib.pocstor_64498_object7465/head//22 by client.1049635.0:41668 2014-08-12 17:19:20.438798 2014-08-13 18:06:06.524160 7f5c973a57a0 20 read_log 194215'508 (194215'491) delete 6aba4630/benchmark_data_storage0111-ib.pocstor_64498_object14935/head//22 by client.1049635.0:49138 2014-08-12 17:19:22.855414 2014-08-13 18:06:06.524228 7f5c973a57a0 20 read_log 194215'509 (194215'492) delete 25bb1e30/benchmark_data_storage0111-ib.pocstor_64498_object19012/head//22 by client.1049635.0:53215 2014-08-12 17:19:24.170332 2014-08-13 18:06:06.524248 7f5c973a57a0 20 read_log 194215'510 (194215'493) delete ef399630/benchmark_data_storage0111-ib.pocstor_64498_object19842/head//22 by client.1049635.0:54045 2014-08-12 17:19:24.460480 2014-08-13 18:06:06.524261 7f5c973a57a0 20 read_log 194215'511 (194215'494) delete 7e542630/benchmark_data_storage0111-ib.pocstor_64498_object23129/head//22 by client.1049635.0:57332 2014-08-12 17:19:25.665517 2014-08-13 18:06:06.524273 7f5c973a57a0 20 read_log 194215'512 (194215'495) delete 5c645630/benchmark_data_storage0111-ib.pocstor_64498_object24835/head//22 by client.1049635.0:59038 2014-08-12 17:19:26.266897 2014-08-13 18:06:06.524286 7f5c973a57a0 20 read_log 194215'513 (194215'496) delete dac93e30/benchmark_data_storage0111-ib.pocstor_64498_object26247/head//22 by client.1049635.0:60450 2014-08-12 17:19:26.741059 2014-08-13 18:06:06.524311 7f5c973a57a0 20 read_log 194215'514 (194215'497) delete 18270630/benchmark_data_storage0111-ib.pocstor_64498_object26408/head//22 by client.1049635.0:60611 2014-08-12 17:19:26.792218 2014-08-13 18:06:06.524325 7f5c973a57a0 20 read_log 194215'515 (194215'498) delete 58c28630/benchmark_data_storage0111-ib.pocstor_64498_object28675/head//22 by client.1049635.0:62878 2014-08-12 17:19:27.588020 2014-08-13 18:06:06.524345 7f5c973a57a0 20 read_log 194215'516 (194215'499) delete 6dc85630/benchmark_data_storage0111-ib.pocstor_64498_object31292/head//22 by client.1049635.0:65495 2014-08-12 17:19:28.482205 2014-08-13 18:06:06.524358 7f5c973a57a0 20 read_log 194215'517 (194215'500) delete 9a8bc630/benchmark_data_storage0111-ib.pocstor_64498_object31461/head//22 by client.1049635.0:65664 2014-08-12 17:19:28.543841 2014-08-13 18:06:06.524438 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.524455 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=0 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.524470 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=0 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Initial 0.001852 0 0.000000 2014-08-13 18:06:06.524482 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=0 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.524490 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=0 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.524499 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] log((0'0,194215'517], crt=192888'483) 2014-08-13 18:06:06.524509 7f5c973a57a0 10 osd.4 194487 pgid 22.639 coll 22.639_head 2014-08-13 18:06:06.524651 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.639 2014-08-13 18:06:06.524661 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.524691 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.639(unlocked)] enter Initial 2014-08-13 18:06:06.524711 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.639(unlocked)] enter NotTrimming 2014-08-13 18:06:06.524869 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.525000 7f5c973a57a0 20 read_log 191648'1 (0'0) modify 481ec639/benchmark_data_storage0111-ib.pocstor_40745_object275/head//22 by client.273158.0:276 2014-07-30 14:08:49.740856 2014-08-13 18:06:06.525020 7f5c973a57a0 20 read_log 191648'37 (0'0) modify 2730639/benchmark_data_storage0111-ib.pocstor_40745_object2663/head//22 by client.273158.0:2664 2014-07-30 14:08:59.702920 2014-08-13 18:06:06.525033 7f5c973a57a0 20 read_log 191648'121 (0'0) modify 2dab0e39/benchmark_data_storage0111-ib.pocstor_40745_object8454/head//22 by client.273158.0:8455 2014-07-30 14:09:28.563838 2014-08-13 18:06:06.525045 7f5c973a57a0 20 read_log 191648'128 (0'0) modify 4cf57e39/benchmark_data_storage0111-ib.pocstor_40745_object9021/head//22 by client.273158.0:9022 2014-07-30 14:09:31.263988 2014-08-13 18:06:06.525058 7f5c973a57a0 20 read_log 191648'173 (0'0) modify 2f595639/benchmark_data_storage0111-ib.pocstor_40745_object11682/head//22 by client.273158.0:11683 2014-07-30 14:09:45.522309 2014-08-13 18:06:06.525070 7f5c973a57a0 20 read_log 191648'183 (0'0) modify 1ac56e39/benchmark_data_storage0111-ib.pocstor_40745_object12321/head//22 by client.273158.0:12322 2014-07-30 14:09:49.146674 2014-08-13 18:06:06.525082 7f5c973a57a0 20 read_log 191648'305 (0'0) modify 6e85f639/benchmark_data_storage0111-ib.pocstor_40745_object19685/head//22 by client.273158.0:19686 2014-07-30 14:10:27.772334 2014-08-13 18:06:06.525094 7f5c973a57a0 20 read_log 191648'347 (0'0) modify c0f8e39/benchmark_data_storage0111-ib.pocstor_40745_object22642/head//22 by client.273158.0:22643 2014-07-30 14:10:47.646378 2014-08-13 18:06:06.525108 7f5c973a57a0 20 read_log 191727'355 (0'0) modify c1b06639/benchmark_data_storage0111-ib.pocstor_51285_object1020/head//22 by client.275175.0:1021 2014-07-30 14:18:37.530957 2014-08-13 18:06:06.525120 7f5c973a57a0 20 read_log 191727'356 (0'0) modify ede4ce39/benchmark_data_storage0111-ib.pocstor_51285_object1230/head//22 by client.275175.0:1231 2014-07-30 14:18:38.285023 2014-08-13 18:06:06.525137 7f5c973a57a0 20 read_log 191727'359 (0'0) modify b4408639/benchmark_data_storage0111-ib.pocstor_51285_object4502/head//22 by client.275175.0:4503 2014-07-30 14:18:48.864614 2014-08-13 18:06:06.525155 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 4f8a6e39/benchmark_data_storage0111-ib.pocstor_51285_object6571/head//22 by client.275175.0:6572 2014-07-30 14:18:55.037271 2014-08-13 18:06:06.525170 7f5c973a57a0 20 read_log 191727'363 (0'0) modify 2a18de39/benchmark_data_storage0111-ib.pocstor_51285_object7315/head//22 by client.275175.0:7316 2014-07-30 14:18:57.617343 2014-08-13 18:06:06.525183 7f5c973a57a0 20 read_log 191727'369 (0'0) modify ada5c639/benchmark_data_storage0111-ib.pocstor_51285_object12282/head//22 by client.275175.0:12283 2014-07-30 14:19:12.430522 2014-08-13 18:06:06.525203 7f5c973a57a0 20 read_log 191727'375 (0'0) modify d58cfe39/benchmark_data_storage0111-ib.pocstor_51285_object13669/head//22 by client.275175.0:13670 2014-07-30 14:19:16.721800 2014-08-13 18:06:06.525218 7f5c973a57a0 20 read_log 191727'377 (0'0) modify f8884e39/benchmark_data_storage0111-ib.pocstor_51285_object15378/head//22 by client.275175.0:15379 2014-07-30 14:19:21.995669 2014-08-13 18:06:06.525230 7f5c973a57a0 20 read_log 191727'385 (0'0) modify d9b2e39/benchmark_data_storage0111-ib.pocstor_51285_object18936/head//22 by client.275175.0:18937 2014-07-30 14:19:32.991312 2014-08-13 18:06:06.525242 7f5c973a57a0 20 read_log 191727'387 (0'0) modify 8af2e639/benchmark_data_storage0111-ib.pocstor_51285_object19448/head//22 by client.275175.0:19449 2014-07-30 14:19:34.637565 2014-08-13 18:06:06.525255 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 7dc09e39/benchmark_data_storage0111-ib.pocstor_51285_object21016/head//22 by client.275175.0:21017 2014-07-30 14:19:39.177880 2014-08-13 18:06:06.525273 7f5c973a57a0 20 read_log 191727'392 (0'0) modify aafbbe39/benchmark_data_storage0111-ib.pocstor_51285_object22198/head//22 by client.275175.0:22199 2014-07-30 14:19:43.227643 2014-08-13 18:06:06.525285 7f5c973a57a0 20 read_log 191727'394 (0'0) modify e42f2e39/benchmark_data_storage0111-ib.pocstor_51285_object23735/head//22 by client.275175.0:23736 2014-07-30 14:19:48.087461 2014-08-13 18:06:06.525297 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 1c862639/benchmark_data_storage0111-ib.pocstor_51285_object24523/head//22 by client.275175.0:24524 2014-07-30 14:19:50.301945 2014-08-13 18:06:06.525312 7f5c973a57a0 20 read_log 191727'397 (0'0) modify b23b0e39/benchmark_data_storage0111-ib.pocstor_51285_object24643/head//22 by client.275175.0:24644 2014-07-30 14:19:50.638951 2014-08-13 18:06:06.525326 7f5c973a57a0 20 read_log 191727'400 (0'0) modify e6a2e639/benchmark_data_storage0111-ib.pocstor_51285_object26373/head//22 by client.275175.0:26374 2014-07-30 14:19:55.924454 2014-08-13 18:06:06.525339 7f5c973a57a0 20 read_log 191727'406 (0'0) modify c784d639/benchmark_data_storage0111-ib.pocstor_51285_object28589/head//22 by client.275175.0:28590 2014-07-30 14:20:02.683096 2014-08-13 18:06:06.525351 7f5c973a57a0 20 read_log 191727'407 (0'0) modify f6798639/benchmark_data_storage0111-ib.pocstor_51285_object28817/head//22 by client.275175.0:28818 2014-07-30 14:20:03.356140 2014-08-13 18:06:06.525363 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 72f1ae39/benchmark_data_storage0111-ib.pocstor_51285_object31279/head//22 by client.275175.0:31280 2014-07-30 14:20:10.775092 2014-08-13 18:06:06.525385 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 5d1bbe39/benchmark_data_storage0111-ib.pocstor_51285_object32701/head//22 by client.275175.0:32702 2014-07-30 14:20:14.875553 2014-08-13 18:06:06.525401 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 89ba5e39/benchmark_data_storage0111-ib.pocstor_51285_object33105/head//22 by client.275175.0:33106 2014-07-30 14:20:16.092023 2014-08-13 18:06:06.525413 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 37d2ae39/benchmark_data_storage0111-ib.pocstor_51285_object33517/head//22 by client.275175.0:33518 2014-07-30 14:20:17.562583 2014-08-13 18:06:06.525425 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 46606e39/benchmark_data_storage0111-ib.pocstor_51285_object35605/head//22 by client.275175.0:35606 2014-07-30 14:20:23.737240 2014-08-13 18:06:06.525437 7f5c973a57a0 20 read_log 191727'422 (0'0) modify e0ad9e39/benchmark_data_storage0111-ib.pocstor_51285_object35854/head//22 by client.275175.0:35855 2014-07-30 14:20:24.801181 2014-08-13 18:06:06.525449 7f5c973a57a0 20 read_log 191727'426 (0'0) modify 5da08639/benchmark_data_storage0111-ib.pocstor_51285_object38666/head//22 by client.275175.0:38667 2014-07-30 14:20:33.144862 2014-08-13 18:06:06.525462 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 55a70639/benchmark_data_storage0111-ib.pocstor_35625_object199/head//22 by client.940330.0:200 2014-08-11 16:29:41.430738 2014-08-13 18:06:06.525483 7f5c973a57a0 20 read_log 192888'428 (0'0) modify 1ffeb639/benchmark_data_storage0111-ib.pocstor_35625_object2658/head//22 by client.940330.0:2659 2014-08-11 16:29:51.074318 2014-08-13 18:06:06.525498 7f5c973a57a0 20 read_log 192888'429 (0'0) modify 7c3cc639/benchmark_data_storage0111-ib.pocstor_35625_object3860/head//22 by client.940330.0:3861 2014-08-11 16:29:55.819658 2014-08-13 18:06:06.525511 7f5c973a57a0 20 read_log 192888'430 (0'0) modify 5b7c2e39/benchmark_data_storage0111-ib.pocstor_35625_object4180/head//22 by client.940330.0:4181 2014-08-11 16:29:56.984870 2014-08-13 18:06:06.525523 7f5c973a57a0 20 read_log 192888'431 (0'0) modify 7c2fe39/benchmark_data_storage0111-ib.pocstor_35625_object4739/head//22 by client.940330.0:4740 2014-08-11 16:29:59.111739 2014-08-13 18:06:06.525535 7f5c973a57a0 20 read_log 192888'432 (0'0) modify 3b3a2e39/benchmark_data_storage0111-ib.pocstor_35625_object10805/head//22 by client.940330.0:10806 2014-08-11 16:30:21.042933 2014-08-13 18:06:06.525547 7f5c973a57a0 20 read_log 192888'433 (0'0) modify 42e43e39/benchmark_data_storage0111-ib.pocstor_35625_object14551/head//22 by client.940330.0:14552 2014-08-11 16:30:35.821354 2014-08-13 18:06:06.525559 7f5c973a57a0 20 read_log 192888'434 (0'0) modify 5a550639/benchmark_data_storage0111-ib.pocstor_35625_object16663/head//22 by client.940330.0:16664 2014-08-11 16:30:43.712293 2014-08-13 18:06:06.525571 7f5c973a57a0 20 read_log 192888'435 (0'0) modify 95ea8639/benchmark_data_storage0111-ib.pocstor_35625_object17898/head//22 by client.940330.0:17899 2014-08-11 16:30:48.332880 2014-08-13 18:06:06.525582 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 1e344e39/benchmark_data_storage0111-ib.pocstor_35625_object19146/head//22 by client.940330.0:19147 2014-08-11 16:30:53.204896 2014-08-13 18:06:06.525594 7f5c973a57a0 20 read_log 192888'437 (0'0) modify e2ccee39/benchmark_data_storage0111-ib.pocstor_35625_object19457/head//22 by client.940330.0:19458 2014-08-11 16:30:54.485156 2014-08-13 18:06:06.525606 7f5c973a57a0 20 read_log 192888'438 (0'0) modify fcc97639/benchmark_data_storage0111-ib.pocstor_35625_object21724/head//22 by client.940330.0:21725 2014-08-11 16:31:03.500175 2014-08-13 18:06:06.525618 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 78b00e39/benchmark_data_storage0111-ib.pocstor_35625_object26694/head//22 by client.940330.0:26695 2014-08-11 16:31:22.621379 2014-08-13 18:06:06.525630 7f5c973a57a0 20 read_log 192888'440 (0'0) modify 782e6639/benchmark_data_storage0111-ib.pocstor_35625_object26891/head//22 by client.940330.0:26892 2014-08-11 16:31:23.382190 2014-08-13 18:06:06.525642 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 82c62e39/benchmark_data_storage0111-ib.pocstor_35625_object28193/head//22 by client.940330.0:28194 2014-08-11 16:31:28.477998 2014-08-13 18:06:06.525657 7f5c973a57a0 20 read_log 192888'442 (0'0) modify e79c5639/benchmark_data_storage0111-ib.pocstor_35625_object29711/head//22 by client.940330.0:29712 2014-08-11 16:31:34.367474 2014-08-13 18:06:06.525669 7f5c973a57a0 20 read_log 192888'443 (192888'427) delete 55a70639/benchmark_data_storage0111-ib.pocstor_35625_object199/head//22 by client.940330.0:31507 2014-08-11 16:31:41.519756 2014-08-13 18:06:06.525681 7f5c973a57a0 20 read_log 192888'444 (192888'428) delete 1ffeb639/benchmark_data_storage0111-ib.pocstor_35625_object2658/head//22 by client.940330.0:33966 2014-08-11 16:31:42.160017 2014-08-13 18:06:06.525694 7f5c973a57a0 20 read_log 192888'445 (192888'429) delete 7c3cc639/benchmark_data_storage0111-ib.pocstor_35625_object3860/head//22 by client.940330.0:35168 2014-08-11 16:31:42.491133 2014-08-13 18:06:06.525706 7f5c973a57a0 20 read_log 192888'446 (192888'430) delete 5b7c2e39/benchmark_data_storage0111-ib.pocstor_35625_object4180/head//22 by client.940330.0:35488 2014-08-11 16:31:42.587388 2014-08-13 18:06:06.525721 7f5c973a57a0 20 read_log 192888'447 (192888'431) delete 7c2fe39/benchmark_data_storage0111-ib.pocstor_35625_object4739/head//22 by client.940330.0:36047 2014-08-11 16:31:42.750204 2014-08-13 18:06:06.525744 7f5c973a57a0 20 read_log 192888'448 (192888'432) delete 3b3a2e39/benchmark_data_storage0111-ib.pocstor_35625_object10805/head//22 by client.940330.0:42113 2014-08-11 16:31:44.343890 2014-08-13 18:06:06.525759 7f5c973a57a0 20 read_log 192888'449 (192888'433) delete 42e43e39/benchmark_data_storage0111-ib.pocstor_35625_object14551/head//22 by client.940330.0:45859 2014-08-11 16:31:45.332360 2014-08-13 18:06:06.525771 7f5c973a57a0 20 read_log 192888'450 (192888'434) delete 5a550639/benchmark_data_storage0111-ib.pocstor_35625_object16663/head//22 by client.940330.0:47971 2014-08-11 16:31:45.898185 2014-08-13 18:06:06.525786 7f5c973a57a0 20 read_log 192888'451 (192888'435) delete 95ea8639/benchmark_data_storage0111-ib.pocstor_35625_object17898/head//22 by client.940330.0:49206 2014-08-11 16:31:46.253072 2014-08-13 18:06:06.525798 7f5c973a57a0 20 read_log 192888'452 (192888'436) delete 1e344e39/benchmark_data_storage0111-ib.pocstor_35625_object19146/head//22 by client.940330.0:50454 2014-08-11 16:31:46.620527 2014-08-13 18:06:06.525810 7f5c973a57a0 20 read_log 192888'453 (192888'437) delete e2ccee39/benchmark_data_storage0111-ib.pocstor_35625_object19457/head//22 by client.940330.0:50765 2014-08-11 16:31:46.707819 2014-08-13 18:06:06.525822 7f5c973a57a0 20 read_log 192888'454 (192888'438) delete fcc97639/benchmark_data_storage0111-ib.pocstor_35625_object21724/head//22 by client.940330.0:53032 2014-08-11 16:31:47.358412 2014-08-13 18:06:06.525837 7f5c973a57a0 20 read_log 192888'455 (192888'439) delete 78b00e39/benchmark_data_storage0111-ib.pocstor_35625_object26694/head//22 by client.940330.0:58002 2014-08-11 16:31:48.843060 2014-08-13 18:06:06.525850 7f5c973a57a0 20 read_log 192888'456 (192888'440) delete 782e6639/benchmark_data_storage0111-ib.pocstor_35625_object26891/head//22 by client.940330.0:58199 2014-08-11 16:31:48.899753 2014-08-13 18:06:06.525863 7f5c973a57a0 20 read_log 192888'457 (192888'441) delete 82c62e39/benchmark_data_storage0111-ib.pocstor_35625_object28193/head//22 by client.940330.0:59501 2014-08-11 16:31:49.256998 2014-08-13 18:06:06.525878 7f5c973a57a0 20 read_log 192888'458 (192888'442) delete e79c5639/benchmark_data_storage0111-ib.pocstor_35625_object29711/head//22 by client.940330.0:61019 2014-08-11 16:31:49.676944 2014-08-13 18:06:06.525890 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 9e3f7639/benchmark_data_storage0111-ib.pocstor_64498_object1607/head//22 by client.1049635.0:1608 2014-08-12 17:17:22.313360 2014-08-13 18:06:06.525904 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 9188fe39/benchmark_data_storage0111-ib.pocstor_64498_object2060/head//22 by client.1049635.0:2061 2014-08-12 17:17:23.731623 2014-08-13 18:06:06.525916 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 9e90ce39/benchmark_data_storage0111-ib.pocstor_64498_object3326/head//22 by client.1049635.0:3327 2014-08-12 17:17:27.941857 2014-08-13 18:06:06.525930 7f5c973a57a0 20 read_log 194215'462 (0'0) modify f0463639/benchmark_data_storage0111-ib.pocstor_64498_object7865/head//22 by client.1049635.0:7866 2014-08-12 17:17:41.735718 2014-08-13 18:06:06.525943 7f5c973a57a0 20 read_log 194215'463 (0'0) modify 5a7f2e39/benchmark_data_storage0111-ib.pocstor_64498_object9034/head//22 by client.1049635.0:9035 2014-08-12 17:17:45.119487 2014-08-13 18:06:06.525957 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 9db63639/benchmark_data_storage0111-ib.pocstor_64498_object12088/head//22 by client.1049635.0:12089 2014-08-12 17:17:54.722646 2014-08-13 18:06:06.525970 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 5e5ede39/benchmark_data_storage0111-ib.pocstor_64498_object12243/head//22 by client.1049635.0:12244 2014-08-12 17:17:55.194568 2014-08-13 18:06:06.525984 7f5c973a57a0 20 read_log 194215'466 (0'0) modify 232ec639/benchmark_data_storage0111-ib.pocstor_64498_object12575/head//22 by client.1049635.0:12576 2014-08-12 17:17:56.103236 2014-08-13 18:06:06.526005 7f5c973a57a0 20 read_log 194215'467 (0'0) modify 91c27e39/benchmark_data_storage0111-ib.pocstor_64498_object13743/head//22 by client.1049635.0:13744 2014-08-12 17:17:59.772821 2014-08-13 18:06:06.526020 7f5c973a57a0 20 read_log 194215'468 (0'0) modify 69ce1e39/benchmark_data_storage0111-ib.pocstor_64498_object13794/head//22 by client.1049635.0:13795 2014-08-12 17:17:59.916989 2014-08-13 18:06:06.526032 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 13e1a639/benchmark_data_storage0111-ib.pocstor_64498_object14679/head//22 by client.1049635.0:14680 2014-08-12 17:18:02.928486 2014-08-13 18:06:06.526046 7f5c973a57a0 20 read_log 194215'470 (0'0) modify a4ea639/benchmark_data_storage0111-ib.pocstor_64498_object15315/head//22 by client.1049635.0:15316 2014-08-12 17:18:05.018218 2014-08-13 18:06:06.526060 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 75b9fe39/benchmark_data_storage0111-ib.pocstor_64498_object18236/head//22 by client.1049635.0:18237 2014-08-12 17:18:16.058331 2014-08-13 18:06:06.526072 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 8adce39/benchmark_data_storage0111-ib.pocstor_64498_object22869/head//22 by client.1049635.0:22870 2014-08-12 17:18:33.645837 2014-08-13 18:06:06.526084 7f5c973a57a0 20 read_log 194215'473 (0'0) modify a5a8c639/benchmark_data_storage0111-ib.pocstor_64498_object32892/head//22 by client.1049635.0:32893 2014-08-12 17:19:11.983903 2014-08-13 18:06:06.526100 7f5c973a57a0 20 read_log 194215'474 (0'0) modify e4bf7e39/benchmark_data_storage0111-ib.pocstor_64498_object33512/head//22 by client.1049635.0:33513 2014-08-12 17:19:14.321114 2014-08-13 18:06:06.526133 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 89c00639/benchmark_data_storage0111-ib.pocstor_64498_object34002/head//22 by client.1049635.0:34003 2014-08-12 17:19:16.190410 2014-08-13 18:06:06.526157 7f5c973a57a0 20 read_log 194215'476 (194215'459) delete 9e3f7639/benchmark_data_storage0111-ib.pocstor_64498_object1607/head//22 by client.1049635.0:35810 2014-08-12 17:19:18.520175 2014-08-13 18:06:06.526172 7f5c973a57a0 20 read_log 194215'477 (194215'460) delete 9188fe39/benchmark_data_storage0111-ib.pocstor_64498_object2060/head//22 by client.1049635.0:36263 2014-08-12 17:19:18.664074 2014-08-13 18:06:06.526185 7f5c973a57a0 20 read_log 194215'478 (194215'461) delete 9e90ce39/benchmark_data_storage0111-ib.pocstor_64498_object3326/head//22 by client.1049635.0:37529 2014-08-12 17:19:19.065813 2014-08-13 18:06:06.526198 7f5c973a57a0 20 read_log 194215'479 (194215'462) delete f0463639/benchmark_data_storage0111-ib.pocstor_64498_object7865/head//22 by client.1049635.0:42068 2014-08-12 17:19:20.561698 2014-08-13 18:06:06.526210 7f5c973a57a0 20 read_log 194215'480 (194215'463) delete 5a7f2e39/benchmark_data_storage0111-ib.pocstor_64498_object9034/head//22 by client.1049635.0:43237 2014-08-12 17:19:20.935454 2014-08-13 18:06:06.526223 7f5c973a57a0 20 read_log 194215'481 (194215'464) delete 9db63639/benchmark_data_storage0111-ib.pocstor_64498_object12088/head//22 by client.1049635.0:46291 2014-08-12 17:19:21.924279 2014-08-13 18:06:06.526236 7f5c973a57a0 20 read_log 194215'482 (194215'465) delete 5e5ede39/benchmark_data_storage0111-ib.pocstor_64498_object12243/head//22 by client.1049635.0:46446 2014-08-12 17:19:21.972416 2014-08-13 18:06:06.526252 7f5c973a57a0 20 read_log 194215'483 (194215'466) delete 232ec639/benchmark_data_storage0111-ib.pocstor_64498_object12575/head//22 by client.1049635.0:46778 2014-08-12 17:19:22.076791 2014-08-13 18:06:06.526269 7f5c973a57a0 20 read_log 194215'484 (194215'467) delete 91c27e39/benchmark_data_storage0111-ib.pocstor_64498_object13743/head//22 by client.1049635.0:47946 2014-08-12 17:19:22.488199 2014-08-13 18:06:06.526282 7f5c973a57a0 20 read_log 194215'485 (194215'468) delete 69ce1e39/benchmark_data_storage0111-ib.pocstor_64498_object13794/head//22 by client.1049635.0:47997 2014-08-12 17:19:22.501441 2014-08-13 18:06:06.526296 7f5c973a57a0 20 read_log 194215'486 (194215'469) delete 13e1a639/benchmark_data_storage0111-ib.pocstor_64498_object14679/head//22 by client.1049635.0:48882 2014-08-12 17:19:22.778021 2014-08-13 18:06:06.526333 7f5c973a57a0 20 read_log 194215'487 (194215'470) delete a4ea639/benchmark_data_storage0111-ib.pocstor_64498_object15315/head//22 by client.1049635.0:49518 2014-08-12 17:19:22.972058 2014-08-13 18:06:06.526353 7f5c973a57a0 20 read_log 194215'488 (194215'471) delete 75b9fe39/benchmark_data_storage0111-ib.pocstor_64498_object18236/head//22 by client.1049635.0:52439 2014-08-12 17:19:23.907650 2014-08-13 18:06:06.526368 7f5c973a57a0 20 read_log 194215'489 (194215'472) delete 8adce39/benchmark_data_storage0111-ib.pocstor_64498_object22869/head//22 by client.1049635.0:57072 2014-08-12 17:19:25.564703 2014-08-13 18:06:06.526387 7f5c973a57a0 20 read_log 194215'490 (194215'473) delete a5a8c639/benchmark_data_storage0111-ib.pocstor_64498_object32892/head//22 by client.1049635.0:67095 2014-08-12 17:19:29.079034 2014-08-13 18:06:06.526400 7f5c973a57a0 20 read_log 194215'491 (194215'474) delete e4bf7e39/benchmark_data_storage0111-ib.pocstor_64498_object33512/head//22 by client.1049635.0:67715 2014-08-12 17:19:29.295637 2014-08-13 18:06:06.526413 7f5c973a57a0 20 read_log 194215'492 (194215'475) delete 89c00639/benchmark_data_storage0111-ib.pocstor_64498_object34002/head//22 by client.1049635.0:68205 2014-08-12 17:19:29.482783 2014-08-13 18:06:06.526524 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.526549 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.526565 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Initial 0.001874 0 0.000000 2014-08-13 18:06:06.526579 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.526587 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.526597 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] log((0'0,194215'492], crt=192888'458) 2014-08-13 18:06:06.526606 7f5c973a57a0 10 osd.4 194487 pgid 22.653 coll 22.653_head 2014-08-13 18:06:06.526776 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.653 2014-08-13 18:06:06.526787 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.526826 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.653(unlocked)] enter Initial 2014-08-13 18:06:06.526838 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.653(unlocked)] enter NotTrimming 2014-08-13 18:06:06.526972 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.527118 7f5c973a57a0 20 read_log 191648'50 (0'0) modify df526e53/benchmark_data_storage0111-ib.pocstor_40745_object2992/head//22 by client.273158.0:2993 2014-07-30 14:09:01.366616 2014-08-13 18:06:06.527139 7f5c973a57a0 20 read_log 191648'99 (0'0) modify 5462a653/benchmark_data_storage0111-ib.pocstor_40745_object5509/head//22 by client.273158.0:5510 2014-07-30 14:09:13.472861 2014-08-13 18:06:06.527162 7f5c973a57a0 20 read_log 191648'141 (0'0) modify 5af0be53/benchmark_data_storage0111-ib.pocstor_40745_object7720/head//22 by client.273158.0:7721 2014-07-30 14:09:24.548679 2014-08-13 18:06:06.527176 7f5c973a57a0 20 read_log 191648'164 (0'0) modify 26873e53/benchmark_data_storage0111-ib.pocstor_40745_object9236/head//22 by client.273158.0:9237 2014-07-30 14:09:32.104715 2014-08-13 18:06:06.527188 7f5c973a57a0 20 read_log 191648'196 (0'0) modify 4c04ce53/benchmark_data_storage0111-ib.pocstor_40745_object11704/head//22 by client.273158.0:11705 2014-07-30 14:09:45.603765 2014-08-13 18:06:06.527200 7f5c973a57a0 20 read_log 191648'236 (0'0) modify 11479653/benchmark_data_storage0111-ib.pocstor_40745_object13796/head//22 by client.273158.0:13797 2014-07-30 14:09:56.026201 2014-08-13 18:06:06.527212 7f5c973a57a0 20 read_log 191648'243 (0'0) modify d3a72e53/benchmark_data_storage0111-ib.pocstor_40745_object14204/head//22 by client.273158.0:14205 2014-07-30 14:09:58.369131 2014-08-13 18:06:06.527224 7f5c973a57a0 20 read_log 191648'276 (0'0) modify 323c653/benchmark_data_storage0111-ib.pocstor_40745_object15872/head//22 by client.273158.0:15873 2014-07-30 14:10:06.843641 2014-08-13 18:06:06.527241 7f5c973a57a0 20 read_log 191648'279 (0'0) modify 5beba653/benchmark_data_storage0111-ib.pocstor_40745_object15948/head//22 by client.273158.0:15949 2014-07-30 14:10:07.260442 2014-08-13 18:06:06.527253 7f5c973a57a0 20 read_log 191648'350 (0'0) modify 3f228653/benchmark_data_storage0111-ib.pocstor_40745_object20444/head//22 by client.273158.0:20445 2014-07-30 14:10:33.185905 2014-08-13 18:06:06.527266 7f5c973a57a0 20 read_log 191648'367 (0'0) modify 13f4b653/benchmark_data_storage0111-ib.pocstor_40745_object21587/head//22 by client.273158.0:21588 2014-07-30 14:10:40.045428 2014-08-13 18:06:06.527287 7f5c973a57a0 20 read_log 191648'376 (0'0) modify 3788e653/benchmark_data_storage0111-ib.pocstor_40745_object22096/head//22 by client.273158.0:22097 2014-07-30 14:10:44.575946 2014-08-13 18:06:06.527303 7f5c973a57a0 20 read_log 191727'389 (0'0) modify 338d4e53/benchmark_data_storage0111-ib.pocstor_51285_object5636/head//22 by client.275175.0:5637 2014-07-30 14:18:52.207507 2014-08-13 18:06:06.527316 7f5c973a57a0 20 read_log 191727'390 (0'0) modify a1fe7e53/benchmark_data_storage0111-ib.pocstor_51285_object5883/head//22 by client.275175.0:5884 2014-07-30 14:18:52.913352 2014-08-13 18:06:06.527328 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 40cd6e53/benchmark_data_storage0111-ib.pocstor_51285_object10185/head//22 by client.275175.0:10186 2014-07-30 14:19:05.694861 2014-08-13 18:06:06.527351 7f5c973a57a0 20 read_log 191727'399 (0'0) modify d8d8a653/benchmark_data_storage0111-ib.pocstor_51285_object14490/head//22 by client.275175.0:14491 2014-07-30 14:19:19.263602 2014-08-13 18:06:06.527364 7f5c973a57a0 20 read_log 191727'400 (0'0) modify 77371e53/benchmark_data_storage0111-ib.pocstor_51285_object15005/head//22 by client.275175.0:15006 2014-07-30 14:19:20.716254 2014-08-13 18:06:06.527376 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 4f588653/benchmark_data_storage0111-ib.pocstor_51285_object17088/head//22 by client.275175.0:17089 2014-07-30 14:19:27.407011 2014-08-13 18:06:06.527398 7f5c973a57a0 20 read_log 191727'408 (0'0) modify 90392653/benchmark_data_storage0111-ib.pocstor_51285_object18667/head//22 by client.275175.0:18668 2014-07-30 14:19:31.953450 2014-08-13 18:06:06.527422 7f5c973a57a0 20 read_log 191727'424 (0'0) modify 21400e53/benchmark_data_storage0111-ib.pocstor_51285_object29524/head//22 by client.275175.0:29525 2014-07-30 14:20:05.494414 2014-08-13 18:06:06.527441 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 33bbde53/benchmark_data_storage0111-ib.pocstor_51285_object32140/head//22 by client.275175.0:32141 2014-07-30 14:20:13.270369 2014-08-13 18:06:06.527454 7f5c973a57a0 20 read_log 191727'431 (0'0) modify c1284653/benchmark_data_storage0111-ib.pocstor_51285_object34371/head//22 by client.275175.0:34372 2014-07-30 14:20:20.162069 2014-08-13 18:06:06.527479 7f5c973a57a0 20 read_log 191727'435 (0'0) modify 9181d653/benchmark_data_storage0111-ib.pocstor_51285_object38743/head//22 by client.275175.0:38744 2014-07-30 14:20:33.360591 2014-08-13 18:06:06.527495 7f5c973a57a0 20 read_log 192888'436 (0'0) modify 99f0ee53/benchmark_data_storage0111-ib.pocstor_35625_object3177/head//22 by client.940330.0:3178 2014-08-11 16:29:53.210028 2014-08-13 18:06:06.527508 7f5c973a57a0 20 read_log 192888'437 (0'0) modify a5049653/benchmark_data_storage0111-ib.pocstor_35625_object3827/head//22 by client.940330.0:3828 2014-08-11 16:29:55.700114 2014-08-13 18:06:06.527525 7f5c973a57a0 20 read_log 192888'438 (0'0) modify 2799d653/benchmark_data_storage0111-ib.pocstor_35625_object7714/head//22 by client.940330.0:7715 2014-08-11 16:30:09.827621 2014-08-13 18:06:06.527543 7f5c973a57a0 20 read_log 192888'439 (0'0) modify 63322e53/benchmark_data_storage0111-ib.pocstor_35625_object8106/head//22 by client.940330.0:8107 2014-08-11 16:30:11.212499 2014-08-13 18:06:06.527556 7f5c973a57a0 20 read_log 192888'440 (0'0) modify f3be0e53/benchmark_data_storage0111-ib.pocstor_35625_object11131/head//22 by client.940330.0:11132 2014-08-11 16:30:22.314295 2014-08-13 18:06:06.527568 7f5c973a57a0 20 read_log 192888'441 (0'0) modify b2889653/benchmark_data_storage0111-ib.pocstor_35625_object14766/head//22 by client.940330.0:14767 2014-08-11 16:30:36.621453 2014-08-13 18:06:06.527594 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 914f653/benchmark_data_storage0111-ib.pocstor_35625_object15593/head//22 by client.940330.0:15594 2014-08-11 16:30:39.702497 2014-08-13 18:06:06.527617 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 8bf33e53/benchmark_data_storage0111-ib.pocstor_35625_object16627/head//22 by client.940330.0:16628 2014-08-11 16:30:43.574868 2014-08-13 18:06:06.527636 7f5c973a57a0 20 read_log 192888'444 (0'0) modify 7fcff653/benchmark_data_storage0111-ib.pocstor_35625_object18979/head//22 by client.940330.0:18980 2014-08-11 16:30:52.591538 2014-08-13 18:06:06.527648 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 5db8fe53/benchmark_data_storage0111-ib.pocstor_35625_object19163/head//22 by client.940330.0:19164 2014-08-11 16:30:53.265434 2014-08-13 18:06:06.527661 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 2db68e53/benchmark_data_storage0111-ib.pocstor_35625_object30402/head//22 by client.940330.0:30403 2014-08-11 16:31:36.980670 2014-08-13 18:06:06.527673 7f5c973a57a0 20 read_log 192888'447 (0'0) modify 827f3e53/benchmark_data_storage0111-ib.pocstor_35625_object31033/head//22 by client.940330.0:31034 2014-08-11 16:31:39.350480 2014-08-13 18:06:06.527690 7f5c973a57a0 20 read_log 192888'448 (192888'436) delete 99f0ee53/benchmark_data_storage0111-ib.pocstor_35625_object3177/head//22 by client.940330.0:34485 2014-08-11 16:31:42.301018 2014-08-13 18:06:06.527711 7f5c973a57a0 20 read_log 192888'449 (192888'437) delete a5049653/benchmark_data_storage0111-ib.pocstor_35625_object3827/head//22 by client.940330.0:35135 2014-08-11 16:31:42.482825 2014-08-13 18:06:06.527731 7f5c973a57a0 20 read_log 192888'450 (192888'438) delete 2799d653/benchmark_data_storage0111-ib.pocstor_35625_object7714/head//22 by client.940330.0:39022 2014-08-11 16:31:43.518473 2014-08-13 18:06:06.527744 7f5c973a57a0 20 read_log 192888'451 (192888'439) delete 63322e53/benchmark_data_storage0111-ib.pocstor_35625_object8106/head//22 by client.940330.0:39414 2014-08-11 16:31:43.619474 2014-08-13 18:06:06.527758 7f5c973a57a0 20 read_log 192888'452 (192888'440) delete f3be0e53/benchmark_data_storage0111-ib.pocstor_35625_object11131/head//22 by client.940330.0:42439 2014-08-11 16:31:44.437385 2014-08-13 18:06:06.527770 7f5c973a57a0 20 read_log 192888'453 (192888'441) delete b2889653/benchmark_data_storage0111-ib.pocstor_35625_object14766/head//22 by client.940330.0:46074 2014-08-11 16:31:45.390471 2014-08-13 18:06:06.527788 7f5c973a57a0 20 read_log 192888'454 (192888'442) delete 914f653/benchmark_data_storage0111-ib.pocstor_35625_object15593/head//22 by client.940330.0:46901 2014-08-11 16:31:45.616019 2014-08-13 18:06:06.527803 7f5c973a57a0 20 read_log 192888'455 (192888'443) delete 8bf33e53/benchmark_data_storage0111-ib.pocstor_35625_object16627/head//22 by client.940330.0:47935 2014-08-11 16:31:45.888754 2014-08-13 18:06:06.527827 7f5c973a57a0 20 read_log 192888'456 (192888'444) delete 7fcff653/benchmark_data_storage0111-ib.pocstor_35625_object18979/head//22 by client.940330.0:50287 2014-08-11 16:31:46.576056 2014-08-13 18:06:06.527843 7f5c973a57a0 20 read_log 192888'457 (192888'445) delete 5db8fe53/benchmark_data_storage0111-ib.pocstor_35625_object19163/head//22 by client.940330.0:50471 2014-08-11 16:31:46.624824 2014-08-13 18:06:06.527856 7f5c973a57a0 20 read_log 192888'458 (192888'446) delete 2db68e53/benchmark_data_storage0111-ib.pocstor_35625_object30402/head//22 by client.940330.0:61710 2014-08-11 16:31:49.872201 2014-08-13 18:06:06.527880 7f5c973a57a0 20 read_log 192888'459 (192888'447) delete 827f3e53/benchmark_data_storage0111-ib.pocstor_35625_object31033/head//22 by client.940330.0:62341 2014-08-11 16:31:50.067541 2014-08-13 18:06:06.527896 7f5c973a57a0 20 read_log 194215'460 (0'0) modify 8c2bae53/benchmark_data_storage0111-ib.pocstor_64498_object726/head//22 by client.1049635.0:727 2014-08-12 17:17:19.463649 2014-08-13 18:06:06.527909 7f5c973a57a0 20 read_log 194215'461 (0'0) modify 825c2653/benchmark_data_storage0111-ib.pocstor_64498_object2005/head//22 by client.1049635.0:2006 2014-08-12 17:17:23.569963 2014-08-13 18:06:06.527932 7f5c973a57a0 20 read_log 194215'462 (0'0) modify 4d7d8653/benchmark_data_storage0111-ib.pocstor_64498_object2439/head//22 by client.1049635.0:2440 2014-08-12 17:17:25.064373 2014-08-13 18:06:06.527947 7f5c973a57a0 20 read_log 194215'463 (0'0) modify a254d653/benchmark_data_storage0111-ib.pocstor_64498_object2520/head//22 by client.1049635.0:2521 2014-08-12 17:17:25.308170 2014-08-13 18:06:06.527968 7f5c973a57a0 20 read_log 194215'464 (0'0) modify 380b8e53/benchmark_data_storage0111-ib.pocstor_64498_object2697/head//22 by client.1049635.0:2698 2014-08-12 17:17:25.827860 2014-08-13 18:06:06.527984 7f5c973a57a0 20 read_log 194215'465 (0'0) modify 16d6fe53/benchmark_data_storage0111-ib.pocstor_64498_object2776/head//22 by client.1049635.0:2777 2014-08-12 17:17:26.060757 2014-08-13 18:06:06.527997 7f5c973a57a0 20 read_log 194215'466 (0'0) modify fd5dbe53/benchmark_data_storage0111-ib.pocstor_64498_object3902/head//22 by client.1049635.0:3903 2014-08-12 17:17:29.781465 2014-08-13 18:06:06.528009 7f5c973a57a0 20 read_log 194215'467 (0'0) modify ca4a8653/benchmark_data_storage0111-ib.pocstor_64498_object9634/head//22 by client.1049635.0:9635 2014-08-12 17:17:46.883178 2014-08-13 18:06:06.528026 7f5c973a57a0 20 read_log 194215'468 (0'0) modify cb0fe53/benchmark_data_storage0111-ib.pocstor_64498_object10866/head//22 by client.1049635.0:10867 2014-08-12 17:17:50.713783 2014-08-13 18:06:06.528051 7f5c973a57a0 20 read_log 194215'469 (0'0) modify 10761653/benchmark_data_storage0111-ib.pocstor_64498_object18349/head//22 by client.1049635.0:18350 2014-08-12 17:18:16.479157 2014-08-13 18:06:06.528070 7f5c973a57a0 20 read_log 194215'470 (0'0) modify f928e53/benchmark_data_storage0111-ib.pocstor_64498_object22570/head//22 by client.1049635.0:22571 2014-08-12 17:18:32.532578 2014-08-13 18:06:06.528083 7f5c973a57a0 20 read_log 194215'471 (0'0) modify 6ccc6653/benchmark_data_storage0111-ib.pocstor_64498_object22728/head//22 by client.1049635.0:22729 2014-08-12 17:18:33.120476 2014-08-13 18:06:06.528098 7f5c973a57a0 20 read_log 194215'472 (0'0) modify 5153be53/benchmark_data_storage0111-ib.pocstor_64498_object24699/head//22 by client.1049635.0:24700 2014-08-12 17:18:40.458748 2014-08-13 18:06:06.528111 7f5c973a57a0 20 read_log 194215'473 (0'0) modify 1c3a1653/benchmark_data_storage0111-ib.pocstor_64498_object25680/head//22 by client.1049635.0:25681 2014-08-12 17:18:44.235916 2014-08-13 18:06:06.528133 7f5c973a57a0 20 read_log 194215'474 (0'0) modify b531e53/benchmark_data_storage0111-ib.pocstor_64498_object27038/head//22 by client.1049635.0:27039 2014-08-12 17:18:49.289440 2014-08-13 18:06:06.528154 7f5c973a57a0 20 read_log 194215'475 (0'0) modify 74122653/benchmark_data_storage0111-ib.pocstor_64498_object27078/head//22 by client.1049635.0:27079 2014-08-12 17:18:49.439743 2014-08-13 18:06:06.528172 7f5c973a57a0 20 read_log 194215'476 (0'0) modify 1a1dc653/benchmark_data_storage0111-ib.pocstor_64498_object27081/head//22 by client.1049635.0:27082 2014-08-12 17:18:49.450738 2014-08-13 18:06:06.528185 7f5c973a57a0 20 read_log 194215'477 (194215'460) delete 8c2bae53/benchmark_data_storage0111-ib.pocstor_64498_object726/head//22 by client.1049635.0:34929 2014-08-12 17:19:18.227827 2014-08-13 18:06:06.528198 7f5c973a57a0 20 read_log 194215'478 (194215'461) delete 825c2653/benchmark_data_storage0111-ib.pocstor_64498_object2005/head//22 by client.1049635.0:36208 2014-08-12 17:19:18.648082 2014-08-13 18:06:06.528210 7f5c973a57a0 20 read_log 194215'479 (194215'462) delete 4d7d8653/benchmark_data_storage0111-ib.pocstor_64498_object2439/head//22 by client.1049635.0:36642 2014-08-12 17:19:18.780168 2014-08-13 18:06:06.528240 7f5c973a57a0 20 read_log 194215'480 (194215'463) delete a254d653/benchmark_data_storage0111-ib.pocstor_64498_object2520/head//22 by client.1049635.0:36723 2014-08-12 17:19:18.807512 2014-08-13 18:06:06.528253 7f5c973a57a0 20 read_log 194215'481 (194215'464) delete 380b8e53/benchmark_data_storage0111-ib.pocstor_64498_object2697/head//22 by client.1049635.0:36900 2014-08-12 17:19:18.863824 2014-08-13 18:06:06.528270 7f5c973a57a0 20 read_log 194215'482 (194215'465) delete 16d6fe53/benchmark_data_storage0111-ib.pocstor_64498_object2776/head//22 by client.1049635.0:36979 2014-08-12 17:19:18.892572 2014-08-13 18:06:06.528283 7f5c973a57a0 20 read_log 194215'483 (194215'466) delete fd5dbe53/benchmark_data_storage0111-ib.pocstor_64498_object3902/head//22 by client.1049635.0:38105 2014-08-12 17:19:19.246990 2014-08-13 18:06:06.528296 7f5c973a57a0 20 read_log 194215'484 (194215'467) delete ca4a8653/benchmark_data_storage0111-ib.pocstor_64498_object9634/head//22 by client.1049635.0:43837 2014-08-12 17:19:21.132284 2014-08-13 18:06:06.528360 7f5c973a57a0 20 read_log 194215'485 (194215'468) delete cb0fe53/benchmark_data_storage0111-ib.pocstor_64498_object10866/head//22 by client.1049635.0:45069 2014-08-12 17:19:21.537160 2014-08-13 18:06:06.528379 7f5c973a57a0 20 read_log 194215'486 (194215'469) delete 10761653/benchmark_data_storage0111-ib.pocstor_64498_object18349/head//22 by client.1049635.0:52552 2014-08-12 17:19:23.947075 2014-08-13 18:06:06.528392 7f5c973a57a0 20 read_log 194215'487 (194215'470) delete f928e53/benchmark_data_storage0111-ib.pocstor_64498_object22570/head//22 by client.1049635.0:56773 2014-08-12 17:19:25.447544 2014-08-13 18:06:06.528405 7f5c973a57a0 20 read_log 194215'488 (194215'471) delete 6ccc6653/benchmark_data_storage0111-ib.pocstor_64498_object22728/head//22 by client.1049635.0:56931 2014-08-12 17:19:25.508161 2014-08-13 18:06:06.528417 7f5c973a57a0 20 read_log 194215'489 (194215'472) delete 5153be53/benchmark_data_storage0111-ib.pocstor_64498_object24699/head//22 by client.1049635.0:58902 2014-08-12 17:19:26.213327 2014-08-13 18:06:06.528429 7f5c973a57a0 20 read_log 194215'490 (194215'473) delete 1c3a1653/benchmark_data_storage0111-ib.pocstor_64498_object25680/head//22 by client.1049635.0:59883 2014-08-12 17:19:26.548351 2014-08-13 18:06:06.528442 7f5c973a57a0 20 read_log 194215'491 (194215'474) delete b531e53/benchmark_data_storage0111-ib.pocstor_64498_object27038/head//22 by client.1049635.0:61241 2014-08-12 17:19:27.013718 2014-08-13 18:06:06.528454 7f5c973a57a0 20 read_log 194215'492 (194215'475) delete 74122653/benchmark_data_storage0111-ib.pocstor_64498_object27078/head//22 by client.1049635.0:61281 2014-08-12 17:19:27.025291 2014-08-13 18:06:06.528466 7f5c973a57a0 20 read_log 194215'493 (194215'476) delete 1a1dc653/benchmark_data_storage0111-ib.pocstor_64498_object27081/head//22 by client.1049635.0:61284 2014-08-12 17:19:27.027157 2014-08-13 18:06:06.528543 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.528561 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=0 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.528578 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=0 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Initial 0.001753 0 0.000000 2014-08-13 18:06:06.528590 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=0 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.528599 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=0 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.528609 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] log((0'0,194215'493], crt=192888'459) 2014-08-13 18:06:06.528619 7f5c973a57a0 10 osd.4 194487 pgid 22.69f coll 22.69f_head 2014-08-13 18:06:06.528771 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.69f 2014-08-13 18:06:06.528780 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.528807 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.69f(unlocked)] enter Initial 2014-08-13 18:06:06.528818 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.69f(unlocked)] enter NotTrimming 2014-08-13 18:06:06.528965 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.529064 7f5c973a57a0 20 read_log 191648'26 (0'0) modify 59d5ce9f/benchmark_data_storage0111-ib.pocstor_40745_object1607/head//22 by client.273158.0:1608 2014-07-30 14:08:54.688978 2014-08-13 18:06:06.529084 7f5c973a57a0 20 read_log 191648'30 (0'0) modify 6a669f/benchmark_data_storage0111-ib.pocstor_40745_object1755/head//22 by client.273158.0:1756 2014-07-30 14:08:55.225036 2014-08-13 18:06:06.529097 7f5c973a57a0 20 read_log 191648'69 (0'0) modify fcd08e9f/benchmark_data_storage0111-ib.pocstor_40745_object4264/head//22 by client.273158.0:4265 2014-07-30 14:09:07.850442 2014-08-13 18:06:06.529109 7f5c973a57a0 20 read_log 191648'103 (0'0) modify ca2b0e9f/benchmark_data_storage0111-ib.pocstor_40745_object6471/head//22 by client.273158.0:6472 2014-07-30 14:09:18.567582 2014-08-13 18:06:06.529122 7f5c973a57a0 20 read_log 191648'184 (0'0) modify 2478969f/benchmark_data_storage0111-ib.pocstor_40745_object11905/head//22 by client.273158.0:11906 2014-07-30 14:09:46.482004 2014-08-13 18:06:06.529134 7f5c973a57a0 20 read_log 191648'192 (0'0) modify ff54369f/benchmark_data_storage0111-ib.pocstor_40745_object12517/head//22 by client.273158.0:12518 2014-07-30 14:09:50.070797 2014-08-13 18:06:06.529153 7f5c973a57a0 20 read_log 191648'223 (0'0) modify f0b1ee9f/benchmark_data_storage0111-ib.pocstor_40745_object14293/head//22 by client.273158.0:14294 2014-07-30 14:09:59.184372 2014-08-13 18:06:06.529170 7f5c973a57a0 20 read_log 191648'229 (0'0) modify 5b09de9f/benchmark_data_storage0111-ib.pocstor_40745_object14690/head//22 by client.273158.0:14691 2014-07-30 14:10:00.950754 2014-08-13 18:06:06.529193 7f5c973a57a0 20 read_log 191727'356 (0'0) modify 3c64a69f/benchmark_data_storage0111-ib.pocstor_51285_object2992/head//22 by client.275175.0:2993 2014-07-30 14:18:44.196632 2014-08-13 18:06:06.529208 7f5c973a57a0 20 read_log 191727'358 (0'0) modify 444a2e9f/benchmark_data_storage0111-ib.pocstor_51285_object3482/head//22 by client.275175.0:3483 2014-07-30 14:18:45.748545 2014-08-13 18:06:06.529221 7f5c973a57a0 20 read_log 191727'361 (0'0) modify 8282169f/benchmark_data_storage0111-ib.pocstor_51285_object5121/head//22 by client.275175.0:5122 2014-07-30 14:18:50.707225 2014-08-13 18:06:06.529235 7f5c973a57a0 20 read_log 191727'365 (0'0) modify a398f69f/benchmark_data_storage0111-ib.pocstor_51285_object6081/head//22 by client.275175.0:6082 2014-07-30 14:18:53.501523 2014-08-13 18:06:06.529249 7f5c973a57a0 20 read_log 191727'371 (0'0) modify a5b4e69f/benchmark_data_storage0111-ib.pocstor_51285_object7846/head//22 by client.275175.0:7847 2014-07-30 14:18:59.096908 2014-08-13 18:06:06.529261 7f5c973a57a0 20 read_log 191727'372 (0'0) modify 57937e9f/benchmark_data_storage0111-ib.pocstor_51285_object8504/head//22 by client.275175.0:8505 2014-07-30 14:19:00.938907 2014-08-13 18:06:06.529274 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 1db82e9f/benchmark_data_storage0111-ib.pocstor_51285_object9022/head//22 by client.275175.0:9023 2014-07-30 14:19:02.413649 2014-08-13 18:06:06.529286 7f5c973a57a0 20 read_log 191727'380 (0'0) modify ee27569f/benchmark_data_storage0111-ib.pocstor_51285_object11244/head//22 by client.275175.0:11245 2014-07-30 14:19:08.782711 2014-08-13 18:06:06.529298 7f5c973a57a0 20 read_log 191727'382 (0'0) modify 33bf469f/benchmark_data_storage0111-ib.pocstor_51285_object12465/head//22 by client.275175.0:12466 2014-07-30 14:19:12.942283 2014-08-13 18:06:06.529318 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 582869f/benchmark_data_storage0111-ib.pocstor_51285_object13347/head//22 by client.275175.0:13348 2014-07-30 14:19:15.727298 2014-08-13 18:06:06.529331 7f5c973a57a0 20 read_log 191727'385 (0'0) modify b964f69f/benchmark_data_storage0111-ib.pocstor_51285_object13791/head//22 by client.275175.0:13792 2014-07-30 14:19:17.064618 2014-08-13 18:06:06.529343 7f5c973a57a0 20 read_log 191727'392 (0'0) modify 6f08569f/benchmark_data_storage0111-ib.pocstor_51285_object16375/head//22 by client.275175.0:16376 2014-07-30 14:19:25.291650 2014-08-13 18:06:06.529359 7f5c973a57a0 20 read_log 191727'393 (0'0) modify 5a62669f/benchmark_data_storage0111-ib.pocstor_51285_object16670/head//22 by client.275175.0:16671 2014-07-30 14:19:26.229709 2014-08-13 18:06:06.529372 7f5c973a57a0 20 read_log 191727'396 (0'0) modify a8a5469f/benchmark_data_storage0111-ib.pocstor_51285_object17641/head//22 by client.275175.0:17642 2014-07-30 14:19:28.942289 2014-08-13 18:06:06.529384 7f5c973a57a0 20 read_log 191727'404 (0'0) modify 3300ee9f/benchmark_data_storage0111-ib.pocstor_51285_object20797/head//22 by client.275175.0:20798 2014-07-30 14:19:38.558042 2014-08-13 18:06:06.529396 7f5c973a57a0 20 read_log 191727'405 (0'0) modify 2ba9069f/benchmark_data_storage0111-ib.pocstor_51285_object20912/head//22 by client.275175.0:20913 2014-07-30 14:19:38.881399 2014-08-13 18:06:06.529411 7f5c973a57a0 20 read_log 191727'407 (0'0) modify 4146d69f/benchmark_data_storage0111-ib.pocstor_51285_object22447/head//22 by client.275175.0:22448 2014-07-30 14:19:44.047292 2014-08-13 18:06:06.529423 7f5c973a57a0 20 read_log 191727'409 (0'0) modify 3e13069f/benchmark_data_storage0111-ib.pocstor_51285_object25432/head//22 by client.275175.0:25433 2014-07-30 14:19:53.073705 2014-08-13 18:06:06.529452 7f5c973a57a0 20 read_log 191727'410 (0'0) modify 6481469f/benchmark_data_storage0111-ib.pocstor_51285_object26802/head//22 by client.275175.0:26803 2014-07-30 14:19:57.228522 2014-08-13 18:06:06.529468 7f5c973a57a0 20 read_log 191727'411 (0'0) modify df8ca69f/benchmark_data_storage0111-ib.pocstor_51285_object26958/head//22 by client.275175.0:26959 2014-07-30 14:19:57.694261 2014-08-13 18:06:06.529500 7f5c973a57a0 20 read_log 191727'412 (0'0) modify 13261e9f/benchmark_data_storage0111-ib.pocstor_51285_object27006/head//22 by client.275175.0:27007 2014-07-30 14:19:57.829539 2014-08-13 18:06:06.529516 7f5c973a57a0 20 read_log 191727'414 (0'0) modify ae27869f/benchmark_data_storage0111-ib.pocstor_51285_object27935/head//22 by client.275175.0:27936 2014-07-30 14:20:00.843512 2014-08-13 18:06:06.529536 7f5c973a57a0 20 read_log 191727'415 (0'0) modify 1c06669f/benchmark_data_storage0111-ib.pocstor_51285_object27987/head//22 by client.275175.0:27988 2014-07-30 14:20:00.995534 2014-08-13 18:06:06.529552 7f5c973a57a0 20 read_log 191727'418 (0'0) modify 935cf69f/benchmark_data_storage0111-ib.pocstor_51285_object30756/head//22 by client.275175.0:30757 2014-07-30 14:20:09.292499 2014-08-13 18:06:06.529565 7f5c973a57a0 20 read_log 191727'419 (0'0) modify 6d5efe9f/benchmark_data_storage0111-ib.pocstor_51285_object30784/head//22 by client.275175.0:30785 2014-07-30 14:20:09.370229 2014-08-13 18:06:06.529577 7f5c973a57a0 20 read_log 191727'433 (0'0) modify 25faae9f/benchmark_data_storage0111-ib.pocstor_51285_object36563/head//22 by client.275175.0:36564 2014-07-30 14:20:27.053999 2014-08-13 18:06:06.529590 7f5c973a57a0 20 read_log 191727'434 (0'0) modify b6c0ae9f/benchmark_data_storage0111-ib.pocstor_51285_object36680/head//22 by client.275175.0:36681 2014-07-30 14:20:27.483371 2014-08-13 18:06:06.529602 7f5c973a57a0 20 read_log 191727'437 (0'0) modify 2b32d69f/benchmark_data_storage0111-ib.pocstor_51285_object37213/head//22 by client.275175.0:37214 2014-07-30 14:20:28.966627 2014-08-13 18:06:06.529614 7f5c973a57a0 20 read_log 191727'439 (0'0) modify 26a4e69f/benchmark_data_storage0111-ib.pocstor_51285_object38269/head//22 by client.275175.0:38270 2014-07-30 14:20:32.014094 2014-08-13 18:06:06.529633 7f5c973a57a0 20 read_log 192888'441 (0'0) modify 54659e9f/benchmark_data_storage0111-ib.pocstor_35625_object11/head//22 by client.940330.0:12 2014-08-11 16:29:40.425568 2014-08-13 18:06:06.529646 7f5c973a57a0 20 read_log 192888'442 (0'0) modify 7837a69f/benchmark_data_storage0111-ib.pocstor_35625_object213/head//22 by client.940330.0:214 2014-08-11 16:29:41.508684 2014-08-13 18:06:06.529658 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 30a4569f/benchmark_data_storage0111-ib.pocstor_35625_object1188/head//22 by client.940330.0:1189 2014-08-11 16:29:45.491826 2014-08-13 18:06:06.529670 7f5c973a57a0 20 read_log 192888'444 (0'0) modify a97ece9f/benchmark_data_storage0111-ib.pocstor_35625_object1686/head//22 by client.940330.0:1687 2014-08-11 16:29:47.399999 2014-08-13 18:06:06.529683 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 62db4e9f/benchmark_data_storage0111-ib.pocstor_35625_object3016/head//22 by client.940330.0:3017 2014-08-11 16:29:52.631795 2014-08-13 18:06:06.529703 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 1b4e369f/benchmark_data_storage0111-ib.pocstor_35625_object3864/head//22 by client.940330.0:3865 2014-08-11 16:29:55.833961 2014-08-13 18:06:06.529719 7f5c973a57a0 20 read_log 192888'447 (0'0) modify 2abf869f/benchmark_data_storage0111-ib.pocstor_35625_object6049/head//22 by client.940330.0:6050 2014-08-11 16:30:03.757963 2014-08-13 18:06:06.529732 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 4871369f/benchmark_data_storage0111-ib.pocstor_35625_object7757/head//22 by client.940330.0:7758 2014-08-11 16:30:09.981196 2014-08-13 18:06:06.529744 7f5c973a57a0 20 read_log 192888'449 (0'0) modify fbc2769f/benchmark_data_storage0111-ib.pocstor_35625_object9075/head//22 by client.940330.0:9076 2014-08-11 16:30:14.595048 2014-08-13 18:06:06.529757 7f5c973a57a0 20 read_log 192888'450 (0'0) modify 8aae2e9f/benchmark_data_storage0111-ib.pocstor_35625_object11428/head//22 by client.940330.0:11429 2014-08-11 16:30:23.516874 2014-08-13 18:06:06.529786 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 9e78e9f/benchmark_data_storage0111-ib.pocstor_35625_object11505/head//22 by client.940330.0:11506 2014-08-11 16:30:23.812116 2014-08-13 18:06:06.529803 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 1d90b69f/benchmark_data_storage0111-ib.pocstor_35625_object13707/head//22 by client.940330.0:13708 2014-08-11 16:30:32.541308 2014-08-13 18:06:06.529815 7f5c973a57a0 20 read_log 192888'453 (0'0) modify ddca869f/benchmark_data_storage0111-ib.pocstor_35625_object14104/head//22 by client.940330.0:14105 2014-08-11 16:30:34.128146 2014-08-13 18:06:06.529827 7f5c973a57a0 20 read_log 192888'454 (0'0) modify b8d7ae9f/benchmark_data_storage0111-ib.pocstor_35625_object16641/head//22 by client.940330.0:16642 2014-08-11 16:30:43.628439 2014-08-13 18:06:06.529842 7f5c973a57a0 20 read_log 192888'455 (0'0) modify 68de7e9f/benchmark_data_storage0111-ib.pocstor_35625_object18613/head//22 by client.940330.0:18614 2014-08-11 16:30:51.182550 2014-08-13 18:06:06.529854 7f5c973a57a0 20 read_log 192888'456 (0'0) modify acbe2e9f/benchmark_data_storage0111-ib.pocstor_35625_object18837/head//22 by client.940330.0:18838 2014-08-11 16:30:52.071132 2014-08-13 18:06:06.529866 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 850c469f/benchmark_data_storage0111-ib.pocstor_35625_object19850/head//22 by client.940330.0:19851 2014-08-11 16:30:56.257301 2014-08-13 18:06:06.529881 7f5c973a57a0 20 read_log 192888'458 (0'0) modify c68e569f/benchmark_data_storage0111-ib.pocstor_35625_object23380/head//22 by client.940330.0:23381 2014-08-11 16:31:09.803097 2014-08-13 18:06:06.529893 7f5c973a57a0 20 read_log 192888'459 (0'0) modify 88bc469f/benchmark_data_storage0111-ib.pocstor_35625_object24089/head//22 by client.940330.0:24090 2014-08-11 16:31:12.472541 2014-08-13 18:06:06.529905 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 61d0c69f/benchmark_data_storage0111-ib.pocstor_35625_object26963/head//22 by client.940330.0:26964 2014-08-11 16:31:23.660290 2014-08-13 18:06:06.529918 7f5c973a57a0 20 read_log 192888'461 (0'0) modify 5eb66e9f/benchmark_data_storage0111-ib.pocstor_35625_object27334/head//22 by client.940330.0:27335 2014-08-11 16:31:25.095400 2014-08-13 18:06:06.529935 7f5c973a57a0 20 read_log 192888'462 (0'0) modify d405e69f/benchmark_data_storage0111-ib.pocstor_35625_object27468/head//22 by client.940330.0:27469 2014-08-11 16:31:25.613767 2014-08-13 18:06:06.529947 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 1ee2a69f/benchmark_data_storage0111-ib.pocstor_35625_object29830/head//22 by client.940330.0:29831 2014-08-11 16:31:34.816061 2014-08-13 18:06:06.529959 7f5c973a57a0 20 read_log 192888'464 (0'0) modify b59869f/benchmark_data_storage0111-ib.pocstor_35625_object31218/head//22 by client.940330.0:31219 2014-08-11 16:31:40.086476 2014-08-13 18:06:06.529974 7f5c973a57a0 20 read_log 192888'465 (192888'441) delete 54659e9f/benchmark_data_storage0111-ib.pocstor_35625_object11/head//22 by client.940330.0:31319 2014-08-11 16:31:41.447201 2014-08-13 18:06:06.529997 7f5c973a57a0 20 read_log 192888'466 (192888'442) delete 7837a69f/benchmark_data_storage0111-ib.pocstor_35625_object213/head//22 by client.940330.0:31521 2014-08-11 16:31:41.527264 2014-08-13 18:06:06.530014 7f5c973a57a0 20 read_log 192888'467 (192888'443) delete 30a4569f/benchmark_data_storage0111-ib.pocstor_35625_object1188/head//22 by client.940330.0:32496 2014-08-11 16:31:41.794200 2014-08-13 18:06:06.530031 7f5c973a57a0 20 read_log 192888'468 (192888'444) delete a97ece9f/benchmark_data_storage0111-ib.pocstor_35625_object1686/head//22 by client.940330.0:32994 2014-08-11 16:31:41.912087 2014-08-13 18:06:06.530054 7f5c973a57a0 20 read_log 192888'469 (192888'445) delete 62db4e9f/benchmark_data_storage0111-ib.pocstor_35625_object3016/head//22 by client.940330.0:34324 2014-08-11 16:31:42.254306 2014-08-13 18:06:06.530067 7f5c973a57a0 20 read_log 192888'470 (192888'446) delete 1b4e369f/benchmark_data_storage0111-ib.pocstor_35625_object3864/head//22 by client.940330.0:35172 2014-08-11 16:31:42.492670 2014-08-13 18:06:06.530087 7f5c973a57a0 20 read_log 192888'471 (192888'447) delete 2abf869f/benchmark_data_storage0111-ib.pocstor_35625_object6049/head//22 by client.940330.0:37357 2014-08-11 16:31:43.089554 2014-08-13 18:06:06.530102 7f5c973a57a0 20 read_log 192888'472 (192888'448) delete 4871369f/benchmark_data_storage0111-ib.pocstor_35625_object7757/head//22 by client.940330.0:39065 2014-08-11 16:31:43.529035 2014-08-13 18:06:06.530114 7f5c973a57a0 20 read_log 192888'473 (192888'449) delete fbc2769f/benchmark_data_storage0111-ib.pocstor_35625_object9075/head//22 by client.940330.0:40383 2014-08-11 16:31:43.859908 2014-08-13 18:06:06.530139 7f5c973a57a0 20 read_log 192888'474 (192888'450) delete 8aae2e9f/benchmark_data_storage0111-ib.pocstor_35625_object11428/head//22 by client.940330.0:42736 2014-08-11 16:31:44.521816 2014-08-13 18:06:06.530161 7f5c973a57a0 20 read_log 192888'475 (192888'451) delete 9e78e9f/benchmark_data_storage0111-ib.pocstor_35625_object11505/head//22 by client.940330.0:42813 2014-08-11 16:31:44.541226 2014-08-13 18:06:06.530191 7f5c973a57a0 20 read_log 192888'476 (192888'452) delete 1d90b69f/benchmark_data_storage0111-ib.pocstor_35625_object13707/head//22 by client.940330.0:45015 2014-08-11 16:31:45.106344 2014-08-13 18:06:06.530204 7f5c973a57a0 20 read_log 192888'477 (192888'453) delete ddca869f/benchmark_data_storage0111-ib.pocstor_35625_object14104/head//22 by client.940330.0:45412 2014-08-11 16:31:45.209866 2014-08-13 18:06:06.530219 7f5c973a57a0 20 read_log 192888'478 (192888'454) delete b8d7ae9f/benchmark_data_storage0111-ib.pocstor_35625_object16641/head//22 by client.940330.0:47949 2014-08-11 16:31:45.892229 2014-08-13 18:06:06.530234 7f5c973a57a0 20 read_log 192888'479 (192888'455) delete 68de7e9f/benchmark_data_storage0111-ib.pocstor_35625_object18613/head//22 by client.940330.0:49921 2014-08-11 16:31:46.476621 2014-08-13 18:06:06.530247 7f5c973a57a0 20 read_log 192888'480 (192888'456) delete acbe2e9f/benchmark_data_storage0111-ib.pocstor_35625_object18837/head//22 by client.940330.0:50145 2014-08-11 16:31:46.537627 2014-08-13 18:06:06.530267 7f5c973a57a0 20 read_log 192888'481 (192888'457) delete 850c469f/benchmark_data_storage0111-ib.pocstor_35625_object19850/head//22 by client.940330.0:51158 2014-08-11 16:31:46.825575 2014-08-13 18:06:06.530283 7f5c973a57a0 20 read_log 192888'482 (192888'458) delete c68e569f/benchmark_data_storage0111-ib.pocstor_35625_object23380/head//22 by client.940330.0:54688 2014-08-11 16:31:47.866428 2014-08-13 18:06:06.530296 7f5c973a57a0 20 read_log 192888'483 (192888'459) delete 88bc469f/benchmark_data_storage0111-ib.pocstor_35625_object24089/head//22 by client.940330.0:55397 2014-08-11 16:31:48.066324 2014-08-13 18:06:06.530312 7f5c973a57a0 20 read_log 192888'484 (192888'460) delete 61d0c69f/benchmark_data_storage0111-ib.pocstor_35625_object26963/head//22 by client.940330.0:58271 2014-08-11 16:31:48.916389 2014-08-13 18:06:06.530349 7f5c973a57a0 20 read_log 192888'485 (192888'461) delete 5eb66e9f/benchmark_data_storage0111-ib.pocstor_35625_object27334/head//22 by client.940330.0:58642 2014-08-11 16:31:49.018357 2014-08-13 18:06:06.530368 7f5c973a57a0 20 read_log 192888'486 (192888'462) delete d405e69f/benchmark_data_storage0111-ib.pocstor_35625_object27468/head//22 by client.940330.0:58776 2014-08-11 16:31:49.057124 2014-08-13 18:06:06.530381 7f5c973a57a0 20 read_log 192888'487 (192888'463) delete 1ee2a69f/benchmark_data_storage0111-ib.pocstor_35625_object29830/head//22 by client.940330.0:61138 2014-08-11 16:31:49.709055 2014-08-13 18:06:06.530393 7f5c973a57a0 20 read_log 192888'488 (192888'464) delete b59869f/benchmark_data_storage0111-ib.pocstor_35625_object31218/head//22 by client.940330.0:62526 2014-08-11 16:31:50.126683 2014-08-13 18:06:06.530405 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 3ecec69f/benchmark_data_storage0111-ib.pocstor_64498_object1404/head//22 by client.1049635.0:1405 2014-08-12 17:17:21.708199 2014-08-13 18:06:06.530428 7f5c973a57a0 20 read_log 194215'490 (0'0) modify 9ad4069f/benchmark_data_storage0111-ib.pocstor_64498_object3587/head//22 by client.1049635.0:3588 2014-08-12 17:17:28.766281 2014-08-13 18:06:06.530443 7f5c973a57a0 20 read_log 194215'491 (0'0) modify 24506e9f/benchmark_data_storage0111-ib.pocstor_64498_object4377/head//22 by client.1049635.0:4378 2014-08-12 17:17:31.161057 2014-08-13 18:06:06.530456 7f5c973a57a0 20 read_log 194215'492 (0'0) modify d1a7769f/benchmark_data_storage0111-ib.pocstor_64498_object5307/head//22 by client.1049635.0:5308 2014-08-12 17:17:33.857025 2014-08-13 18:06:06.530468 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 6fb34e9f/benchmark_data_storage0111-ib.pocstor_64498_object6549/head//22 by client.1049635.0:6550 2014-08-12 17:17:37.719641 2014-08-13 18:06:06.530480 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 22619e9f/benchmark_data_storage0111-ib.pocstor_64498_object9935/head//22 by client.1049635.0:9936 2014-08-12 17:17:47.739205 2014-08-13 18:06:06.530502 7f5c973a57a0 20 read_log 194215'495 (0'0) modify faf5a69f/benchmark_data_storage0111-ib.pocstor_64498_object9949/head//22 by client.1049635.0:9950 2014-08-12 17:17:47.778378 2014-08-13 18:06:06.530518 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 54de3e9f/benchmark_data_storage0111-ib.pocstor_64498_object10247/head//22 by client.1049635.0:10248 2014-08-12 17:17:48.628216 2014-08-13 18:06:06.530532 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 3eee969f/benchmark_data_storage0111-ib.pocstor_64498_object12153/head//22 by client.1049635.0:12154 2014-08-12 17:17:54.945107 2014-08-13 18:06:06.530544 7f5c973a57a0 20 read_log 194215'498 (0'0) modify 54d3069f/benchmark_data_storage0111-ib.pocstor_64498_object13146/head//22 by client.1049635.0:13147 2014-08-12 17:17:57.809358 2014-08-13 18:06:06.530557 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 67ec69f/benchmark_data_storage0111-ib.pocstor_64498_object13220/head//22 by client.1049635.0:13221 2014-08-12 17:17:58.059747 2014-08-13 18:06:06.530579 7f5c973a57a0 20 read_log 194215'500 (0'0) modify f8fbbe9f/benchmark_data_storage0111-ib.pocstor_64498_object14873/head//22 by client.1049635.0:14874 2014-08-12 17:18:03.556650 2014-08-13 18:06:06.530592 7f5c973a57a0 20 read_log 194215'501 (0'0) modify bd477e9f/benchmark_data_storage0111-ib.pocstor_64498_object15889/head//22 by client.1049635.0:15890 2014-08-12 17:18:07.139499 2014-08-13 18:06:06.530604 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 669ce69f/benchmark_data_storage0111-ib.pocstor_64498_object16308/head//22 by client.1049635.0:16309 2014-08-12 17:18:08.727523 2014-08-13 18:06:06.530617 7f5c973a57a0 20 read_log 194215'503 (0'0) modify 1de5e9f/benchmark_data_storage0111-ib.pocstor_64498_object20675/head//22 by client.1049635.0:20676 2014-08-12 17:18:25.320137 2014-08-13 18:06:06.530629 7f5c973a57a0 20 read_log 194215'504 (0'0) modify 23b8069f/benchmark_data_storage0111-ib.pocstor_64498_object23355/head//22 by client.1049635.0:23356 2014-08-12 17:18:35.451912 2014-08-13 18:06:06.530641 7f5c973a57a0 20 read_log 194215'505 (0'0) modify 4df4be9f/benchmark_data_storage0111-ib.pocstor_64498_object24264/head//22 by client.1049635.0:24265 2014-08-12 17:18:38.843329 2014-08-13 18:06:06.530654 7f5c973a57a0 20 read_log 194215'506 (0'0) modify c07e169f/benchmark_data_storage0111-ib.pocstor_64498_object24680/head//22 by client.1049635.0:24681 2014-08-12 17:18:40.386998 2014-08-13 18:06:06.530667 7f5c973a57a0 20 read_log 194215'507 (0'0) modify a896469f/benchmark_data_storage0111-ib.pocstor_64498_object24850/head//22 by client.1049635.0:24851 2014-08-12 17:18:41.096674 2014-08-13 18:06:06.530680 7f5c973a57a0 20 read_log 194215'508 (0'0) modify 1febae9f/benchmark_data_storage0111-ib.pocstor_64498_object27173/head//22 by client.1049635.0:27174 2014-08-12 17:18:49.792518 2014-08-13 18:06:06.530692 7f5c973a57a0 20 read_log 194215'509 (194215'489) delete 3ecec69f/benchmark_data_storage0111-ib.pocstor_64498_object1404/head//22 by client.1049635.0:35607 2014-08-12 17:19:18.453300 2014-08-13 18:06:06.530732 7f5c973a57a0 20 read_log 194215'510 (194215'490) delete 9ad4069f/benchmark_data_storage0111-ib.pocstor_64498_object3587/head//22 by client.1049635.0:37790 2014-08-12 17:19:19.144988 2014-08-13 18:06:06.530749 7f5c973a57a0 20 read_log 194215'511 (194215'491) delete 24506e9f/benchmark_data_storage0111-ib.pocstor_64498_object4377/head//22 by client.1049635.0:38580 2014-08-12 17:19:19.395858 2014-08-13 18:06:06.530761 7f5c973a57a0 20 read_log 194215'512 (194215'492) delete d1a7769f/benchmark_data_storage0111-ib.pocstor_64498_object5307/head//22 by client.1049635.0:39510 2014-08-12 17:19:19.711809 2014-08-13 18:06:06.530774 7f5c973a57a0 20 read_log 194215'513 (194215'493) delete 6fb34e9f/benchmark_data_storage0111-ib.pocstor_64498_object6549/head//22 by client.1049635.0:40752 2014-08-12 17:19:20.132582 2014-08-13 18:06:06.530786 7f5c973a57a0 20 read_log 194215'514 (194215'494) delete 22619e9f/benchmark_data_storage0111-ib.pocstor_64498_object9935/head//22 by client.1049635.0:44138 2014-08-12 17:19:21.239146 2014-08-13 18:06:06.530798 7f5c973a57a0 20 read_log 194215'515 (194215'495) delete faf5a69f/benchmark_data_storage0111-ib.pocstor_64498_object9949/head//22 by client.1049635.0:44152 2014-08-12 17:19:21.244711 2014-08-13 18:06:06.530811 7f5c973a57a0 20 read_log 194215'516 (194215'496) delete 54de3e9f/benchmark_data_storage0111-ib.pocstor_64498_object10247/head//22 by client.1049635.0:44450 2014-08-12 17:19:21.341615 2014-08-13 18:06:06.530823 7f5c973a57a0 20 read_log 194215'517 (194215'497) delete 3eee969f/benchmark_data_storage0111-ib.pocstor_64498_object12153/head//22 by client.1049635.0:46356 2014-08-12 17:19:21.945293 2014-08-13 18:06:06.530835 7f5c973a57a0 20 read_log 194215'518 (194215'498) delete 54d3069f/benchmark_data_storage0111-ib.pocstor_64498_object13146/head//22 by client.1049635.0:47349 2014-08-12 17:19:22.275502 2014-08-13 18:06:06.530849 7f5c973a57a0 20 read_log 194215'519 (194215'499) delete 67ec69f/benchmark_data_storage0111-ib.pocstor_64498_object13220/head//22 by client.1049635.0:47423 2014-08-12 17:19:22.297624 2014-08-13 18:06:06.530864 7f5c973a57a0 20 read_log 194215'520 (194215'500) delete f8fbbe9f/benchmark_data_storage0111-ib.pocstor_64498_object14873/head//22 by client.1049635.0:49076 2014-08-12 17:19:22.836593 2014-08-13 18:06:06.530877 7f5c973a57a0 20 read_log 194215'521 (194215'501) delete bd477e9f/benchmark_data_storage0111-ib.pocstor_64498_object15889/head//22 by client.1049635.0:50092 2014-08-12 17:19:23.156553 2014-08-13 18:06:06.530890 7f5c973a57a0 20 read_log 194215'522 (194215'502) delete 669ce69f/benchmark_data_storage0111-ib.pocstor_64498_object16308/head//22 by client.1049635.0:50511 2014-08-12 17:19:23.288147 2014-08-13 18:06:06.530902 7f5c973a57a0 20 read_log 194215'523 (194215'503) delete 1de5e9f/benchmark_data_storage0111-ib.pocstor_64498_object20675/head//22 by client.1049635.0:54878 2014-08-12 17:19:24.767212 2014-08-13 18:06:06.530915 7f5c973a57a0 20 read_log 194215'524 (194215'504) delete 23b8069f/benchmark_data_storage0111-ib.pocstor_64498_object23355/head//22 by client.1049635.0:57558 2014-08-12 17:19:25.747190 2014-08-13 18:06:06.530927 7f5c973a57a0 20 read_log 194215'525 (194215'505) delete 4df4be9f/benchmark_data_storage0111-ib.pocstor_64498_object24264/head//22 by client.1049635.0:58467 2014-08-12 17:19:26.051697 2014-08-13 18:06:06.530939 7f5c973a57a0 20 read_log 194215'526 (194215'506) delete c07e169f/benchmark_data_storage0111-ib.pocstor_64498_object24680/head//22 by client.1049635.0:58883 2014-08-12 17:19:26.202197 2014-08-13 18:06:06.530954 7f5c973a57a0 20 read_log 194215'527 (194215'507) delete a896469f/benchmark_data_storage0111-ib.pocstor_64498_object24850/head//22 by client.1049635.0:59053 2014-08-12 17:19:26.273128 2014-08-13 18:06:06.530966 7f5c973a57a0 20 read_log 194215'528 (194215'508) delete 1febae9f/benchmark_data_storage0111-ib.pocstor_64498_object27173/head//22 by client.1049635.0:61376 2014-08-12 17:19:27.064621 2014-08-13 18:06:06.531067 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.531085 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=0 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.531100 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=0 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Initial 0.002293 0 0.000000 2014-08-13 18:06:06.531114 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=0 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.531122 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=0 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.531131 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] log((0'0,194215'528], crt=192888'488) 2014-08-13 18:06:06.531141 7f5c973a57a0 10 osd.4 194487 pgid 22.6bd coll 22.6bd_head 2014-08-13 18:06:06.531269 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.6bd 2014-08-13 18:06:06.531278 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.531308 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.6bd(unlocked)] enter Initial 2014-08-13 18:06:06.531320 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.6bd(unlocked)] enter NotTrimming 2014-08-13 18:06:06.531459 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.531584 7f5c973a57a0 20 read_log 191648'69 (0'0) modify 6c0a7ebd/benchmark_data_storage0111-ib.pocstor_40745_object3741/head//22 by client.273158.0:3742 2014-07-30 14:09:04.972872 2014-08-13 18:06:06.531606 7f5c973a57a0 20 read_log 191648'89 (0'0) modify 587b1ebd/benchmark_data_storage0111-ib.pocstor_40745_object5009/head//22 by client.273158.0:5010 2014-07-30 14:09:11.065311 2014-08-13 18:06:06.531619 7f5c973a57a0 20 read_log 191648'93 (0'0) modify dc4a7ebd/benchmark_data_storage0111-ib.pocstor_40745_object5164/head//22 by client.273158.0:5165 2014-07-30 14:09:11.962809 2014-08-13 18:06:06.531632 7f5c973a57a0 20 read_log 191648'105 (0'0) modify 28253ebd/benchmark_data_storage0111-ib.pocstor_40745_object5962/head//22 by client.273158.0:5963 2014-07-30 14:09:15.655307 2014-08-13 18:06:06.531644 7f5c973a57a0 20 read_log 191648'114 (0'0) modify 937596bd/benchmark_data_storage0111-ib.pocstor_40745_object6376/head//22 by client.273158.0:6377 2014-07-30 14:09:18.064751 2014-08-13 18:06:06.531657 7f5c973a57a0 20 read_log 191648'119 (0'0) modify f1d2febd/benchmark_data_storage0111-ib.pocstor_40745_object6723/head//22 by client.273158.0:6724 2014-07-30 14:09:19.844380 2014-08-13 18:06:06.531680 7f5c973a57a0 20 read_log 191648'174 (0'0) modify 76f636bd/benchmark_data_storage0111-ib.pocstor_40745_object9923/head//22 by client.273158.0:9924 2014-07-30 14:09:35.421885 2014-08-13 18:06:06.531695 7f5c973a57a0 20 read_log 191648'226 (0'0) modify ae50eebd/benchmark_data_storage0111-ib.pocstor_40745_object13564/head//22 by client.273158.0:13565 2014-07-30 14:09:54.956422 2014-08-13 18:06:06.531709 7f5c973a57a0 20 read_log 191648'249 (0'0) modify 77816bd/benchmark_data_storage0111-ib.pocstor_40745_object15019/head//22 by client.273158.0:15020 2014-07-30 14:10:02.754545 2014-08-13 18:06:06.531721 7f5c973a57a0 20 read_log 191648'250 (0'0) modify ab7eebd/benchmark_data_storage0111-ib.pocstor_40745_object15035/head//22 by client.273158.0:15036 2014-07-30 14:10:02.813743 2014-08-13 18:06:06.531734 7f5c973a57a0 20 read_log 191648'255 (0'0) modify dbab9ebd/benchmark_data_storage0111-ib.pocstor_40745_object15269/head//22 by client.273158.0:15270 2014-07-30 14:10:03.747777 2014-08-13 18:06:06.531746 7f5c973a57a0 20 read_log 191648'261 (0'0) modify f292a6bd/benchmark_data_storage0111-ib.pocstor_40745_object15669/head//22 by client.273158.0:15670 2014-07-30 14:10:05.563476 2014-08-13 18:06:06.531758 7f5c973a57a0 20 read_log 191648'287 (0'0) modify b1c366bd/benchmark_data_storage0111-ib.pocstor_40745_object17364/head//22 by client.273158.0:17365 2014-07-30 14:10:15.035417 2014-08-13 18:06:06.531770 7f5c973a57a0 20 read_log 191648'295 (0'0) modify bda64ebd/benchmark_data_storage0111-ib.pocstor_40745_object17841/head//22 by client.273158.0:17842 2014-07-30 14:10:17.208194 2014-08-13 18:06:06.531782 7f5c973a57a0 20 read_log 191727'371 (0'0) modify 6f376ebd/benchmark_data_storage0111-ib.pocstor_51285_object1941/head//22 by client.275175.0:1942 2014-07-30 14:18:40.709689 2014-08-13 18:06:06.531794 7f5c973a57a0 20 read_log 191727'375 (0'0) modify c8078ebd/benchmark_data_storage0111-ib.pocstor_51285_object2881/head//22 by client.275175.0:2882 2014-07-30 14:18:43.846370 2014-08-13 18:06:06.531807 7f5c973a57a0 20 read_log 191727'376 (0'0) modify c21b76bd/benchmark_data_storage0111-ib.pocstor_51285_object3086/head//22 by client.275175.0:3087 2014-07-30 14:18:44.498279 2014-08-13 18:06:06.531820 7f5c973a57a0 20 read_log 191727'379 (0'0) modify ca9096bd/benchmark_data_storage0111-ib.pocstor_51285_object4142/head//22 by client.275175.0:4143 2014-07-30 14:18:47.793031 2014-08-13 18:06:06.531832 7f5c973a57a0 20 read_log 191727'385 (0'0) modify c4e5c6bd/benchmark_data_storage0111-ib.pocstor_51285_object6793/head//22 by client.275175.0:6794 2014-07-30 14:18:56.139313 2014-08-13 18:06:06.531844 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 238d56bd/benchmark_data_storage0111-ib.pocstor_51285_object9014/head//22 by client.275175.0:9015 2014-07-30 14:19:02.389751 2014-08-13 18:06:06.531856 7f5c973a57a0 20 read_log 191727'391 (0'0) modify 683b8ebd/benchmark_data_storage0111-ib.pocstor_51285_object9699/head//22 by client.275175.0:9700 2014-07-30 14:19:04.343122 2014-08-13 18:06:06.531869 7f5c973a57a0 20 read_log 191727'394 (0'0) modify 8f4eeebd/benchmark_data_storage0111-ib.pocstor_51285_object11551/head//22 by client.275175.0:11552 2014-07-30 14:19:09.910373 2014-08-13 18:06:06.531882 7f5c973a57a0 20 read_log 191727'395 (0'0) modify 95c68ebd/benchmark_data_storage0111-ib.pocstor_51285_object11606/head//22 by client.275175.0:11607 2014-07-30 14:19:10.064654 2014-08-13 18:06:06.531894 7f5c973a57a0 20 read_log 191727'399 (0'0) modify e079debd/benchmark_data_storage0111-ib.pocstor_51285_object15815/head//22 by client.275175.0:15816 2014-07-30 14:19:23.297614 2014-08-13 18:06:06.531906 7f5c973a57a0 20 read_log 191727'401 (0'0) modify 42e596bd/benchmark_data_storage0111-ib.pocstor_51285_object17487/head//22 by client.275175.0:17488 2014-07-30 14:19:28.512848 2014-08-13 18:06:06.531947 7f5c973a57a0 20 read_log 191727'403 (0'0) modify d484e6bd/benchmark_data_storage0111-ib.pocstor_51285_object18608/head//22 by client.275175.0:18609 2014-07-30 14:19:31.790986 2014-08-13 18:06:06.531962 7f5c973a57a0 20 read_log 191727'406 (0'0) modify d65536bd/benchmark_data_storage0111-ib.pocstor_51285_object21059/head//22 by client.275175.0:21060 2014-07-30 14:19:39.300085 2014-08-13 18:06:06.531974 7f5c973a57a0 20 read_log 191727'411 (0'0) modify b96bebd/benchmark_data_storage0111-ib.pocstor_51285_object22891/head//22 by client.275175.0:22892 2014-07-30 14:19:45.717804 2014-08-13 18:06:06.531986 7f5c973a57a0 20 read_log 191727'417 (0'0) modify 6eebd/benchmark_data_storage0111-ib.pocstor_51285_object27237/head//22 by client.275175.0:27238 2014-07-30 14:19:58.489338 2014-08-13 18:06:06.531999 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 2f09b6bd/benchmark_data_storage0111-ib.pocstor_51285_object28161/head//22 by client.275175.0:28162 2014-07-30 14:20:01.481766 2014-08-13 18:06:06.532011 7f5c973a57a0 20 read_log 191727'422 (0'0) modify 267cfebd/benchmark_data_storage0111-ib.pocstor_51285_object28855/head//22 by client.275175.0:28856 2014-07-30 14:20:03.484327 2014-08-13 18:06:06.532023 7f5c973a57a0 20 read_log 191727'425 (0'0) modify 31c3b6bd/benchmark_data_storage0111-ib.pocstor_51285_object30210/head//22 by client.275175.0:30211 2014-07-30 14:20:07.731180 2014-08-13 18:06:06.532035 7f5c973a57a0 20 read_log 191727'427 (0'0) modify 338abebd/benchmark_data_storage0111-ib.pocstor_51285_object31017/head//22 by client.275175.0:31018 2014-07-30 14:20:10.031751 2014-08-13 18:06:06.532047 7f5c973a57a0 20 read_log 191727'429 (0'0) modify 6c8fdebd/benchmark_data_storage0111-ib.pocstor_51285_object31418/head//22 by client.275175.0:31419 2014-07-30 14:20:11.165503 2014-08-13 18:06:06.532059 7f5c973a57a0 20 read_log 191727'430 (0'0) modify d87febd/benchmark_data_storage0111-ib.pocstor_51285_object32524/head//22 by client.275175.0:32525 2014-07-30 14:20:14.371434 2014-08-13 18:06:06.532072 7f5c973a57a0 20 read_log 192888'443 (0'0) modify 499fe6bd/benchmark_data_storage0111-ib.pocstor_35625_object1877/head//22 by client.940330.0:1878 2014-08-11 16:29:48.090530 2014-08-13 18:06:06.532087 7f5c973a57a0 20 read_log 192888'444 (0'0) modify ec534ebd/benchmark_data_storage0111-ib.pocstor_35625_object3073/head//22 by client.940330.0:3074 2014-08-11 16:29:52.837572 2014-08-13 18:06:06.532099 7f5c973a57a0 20 read_log 192888'445 (0'0) modify 59202ebd/benchmark_data_storage0111-ib.pocstor_35625_object4800/head//22 by client.940330.0:4801 2014-08-11 16:29:59.327603 2014-08-13 18:06:06.532112 7f5c973a57a0 20 read_log 192888'446 (0'0) modify 84445ebd/benchmark_data_storage0111-ib.pocstor_35625_object6105/head//22 by client.940330.0:6106 2014-08-11 16:30:03.955124 2014-08-13 18:06:06.532125 7f5c973a57a0 20 read_log 192888'447 (0'0) modify 2b64debd/benchmark_data_storage0111-ib.pocstor_35625_object7407/head//22 by client.940330.0:7408 2014-08-11 16:30:08.753977 2014-08-13 18:06:06.532139 7f5c973a57a0 20 read_log 192888'448 (0'0) modify 89a2a6bd/benchmark_data_storage0111-ib.pocstor_35625_object9352/head//22 by client.940330.0:9353 2014-08-11 16:30:15.562790 2014-08-13 18:06:06.532157 7f5c973a57a0 20 read_log 192888'449 (0'0) modify ac29a6bd/benchmark_data_storage0111-ib.pocstor_35625_object11055/head//22 by client.940330.0:11056 2014-08-11 16:30:22.013217 2014-08-13 18:06:06.532172 7f5c973a57a0 20 read_log 192888'450 (0'0) modify fdbb26bd/benchmark_data_storage0111-ib.pocstor_35625_object12712/head//22 by client.940330.0:12713 2014-08-11 16:30:28.684301 2014-08-13 18:06:06.532184 7f5c973a57a0 20 read_log 192888'451 (0'0) modify 8ecafebd/benchmark_data_storage0111-ib.pocstor_35625_object13683/head//22 by client.940330.0:13684 2014-08-11 16:30:32.452537 2014-08-13 18:06:06.532204 7f5c973a57a0 20 read_log 192888'452 (0'0) modify 5f603ebd/benchmark_data_storage0111-ib.pocstor_35625_object18152/head//22 by client.940330.0:18153 2014-08-11 16:30:49.357352 2014-08-13 18:06:06.532219 7f5c973a57a0 20 read_log 192888'453 (0'0) modify 2eaaebd/benchmark_data_storage0111-ib.pocstor_35625_object20659/head//22 by client.940330.0:20660 2014-08-11 16:30:59.279279 2014-08-13 18:06:06.532251 7f5c973a57a0 20 read_log 192888'454 (0'0) modify 7ad70ebd/benchmark_data_storage0111-ib.pocstor_35625_object23277/head//22 by client.940330.0:23278 2014-08-11 16:31:09.430039 2014-08-13 18:06:06.532266 7f5c973a57a0 20 read_log 192888'455 (0'0) modify e84c7ebd/benchmark_data_storage0111-ib.pocstor_35625_object23404/head//22 by client.940330.0:23405 2014-08-11 16:31:09.900039 2014-08-13 18:06:06.532278 7f5c973a57a0 20 read_log 192888'456 (0'0) modify 901d86bd/benchmark_data_storage0111-ib.pocstor_35625_object23801/head//22 by client.940330.0:23802 2014-08-11 16:31:11.422792 2014-08-13 18:06:06.532294 7f5c973a57a0 20 read_log 192888'457 (0'0) modify d762ebd/benchmark_data_storage0111-ib.pocstor_35625_object24691/head//22 by client.940330.0:24692 2014-08-11 16:31:14.684344 2014-08-13 18:06:06.532308 7f5c973a57a0 20 read_log 192888'458 (0'0) modify df3c4ebd/benchmark_data_storage0111-ib.pocstor_35625_object28332/head//22 by client.940330.0:28333 2014-08-11 16:31:29.007161 2014-08-13 18:06:06.532321 7f5c973a57a0 20 read_log 192888'459 (0'0) modify dbe02ebd/benchmark_data_storage0111-ib.pocstor_35625_object28470/head//22 by client.940330.0:28471 2014-08-11 16:31:29.549910 2014-08-13 18:06:06.532333 7f5c973a57a0 20 read_log 192888'460 (0'0) modify ee9126bd/benchmark_data_storage0111-ib.pocstor_35625_object30366/head//22 by client.940330.0:30367 2014-08-11 16:31:36.850174 2014-08-13 18:06:06.532345 7f5c973a57a0 20 read_log 192888'461 (192888'443) delete 499fe6bd/benchmark_data_storage0111-ib.pocstor_35625_object1877/head//22 by client.940330.0:33185 2014-08-11 16:31:41.960603 2014-08-13 18:06:06.532357 7f5c973a57a0 20 read_log 192888'462 (192888'444) delete ec534ebd/benchmark_data_storage0111-ib.pocstor_35625_object3073/head//22 by client.940330.0:34381 2014-08-11 16:31:42.270012 2014-08-13 18:06:06.532370 7f5c973a57a0 20 read_log 192888'463 (192888'445) delete 59202ebd/benchmark_data_storage0111-ib.pocstor_35625_object4800/head//22 by client.940330.0:36108 2014-08-11 16:31:42.766412 2014-08-13 18:06:06.532382 7f5c973a57a0 20 read_log 192888'464 (192888'446) delete 84445ebd/benchmark_data_storage0111-ib.pocstor_35625_object6105/head//22 by client.940330.0:37413 2014-08-11 16:31:43.105164 2014-08-13 18:06:06.532394 7f5c973a57a0 20 read_log 192888'465 (192888'447) delete 2b64debd/benchmark_data_storage0111-ib.pocstor_35625_object7407/head//22 by client.940330.0:38715 2014-08-11 16:31:43.440721 2014-08-13 18:06:06.532406 7f5c973a57a0 20 read_log 192888'466 (192888'448) delete 89a2a6bd/benchmark_data_storage0111-ib.pocstor_35625_object9352/head//22 by client.940330.0:40660 2014-08-11 16:31:43.933025 2014-08-13 18:06:06.532431 7f5c973a57a0 20 read_log 192888'467 (192888'449) delete ac29a6bd/benchmark_data_storage0111-ib.pocstor_35625_object11055/head//22 by client.940330.0:42363 2014-08-11 16:31:44.414944 2014-08-13 18:06:06.532444 7f5c973a57a0 20 read_log 192888'468 (192888'450) delete fdbb26bd/benchmark_data_storage0111-ib.pocstor_35625_object12712/head//22 by client.940330.0:44020 2014-08-11 16:31:44.838387 2014-08-13 18:06:06.532457 7f5c973a57a0 20 read_log 192888'469 (192888'451) delete 8ecafebd/benchmark_data_storage0111-ib.pocstor_35625_object13683/head//22 by client.940330.0:44991 2014-08-11 16:31:45.100298 2014-08-13 18:06:06.532501 7f5c973a57a0 20 read_log 192888'470 (192888'452) delete 5f603ebd/benchmark_data_storage0111-ib.pocstor_35625_object18152/head//22 by client.940330.0:49460 2014-08-11 16:31:46.338441 2014-08-13 18:06:06.532526 7f5c973a57a0 20 read_log 192888'471 (192888'453) delete 2eaaebd/benchmark_data_storage0111-ib.pocstor_35625_object20659/head//22 by client.940330.0:51967 2014-08-11 16:31:47.068080 2014-08-13 18:06:06.532552 7f5c973a57a0 20 read_log 192888'472 (192888'454) delete 7ad70ebd/benchmark_data_storage0111-ib.pocstor_35625_object23277/head//22 by client.940330.0:54585 2014-08-11 16:31:47.837831 2014-08-13 18:06:06.532579 7f5c973a57a0 20 read_log 192888'473 (192888'455) delete e84c7ebd/benchmark_data_storage0111-ib.pocstor_35625_object23404/head//22 by client.940330.0:54712 2014-08-11 16:31:47.871150 2014-08-13 18:06:06.532593 7f5c973a57a0 20 read_log 192888'474 (192888'456) delete 901d86bd/benchmark_data_storage0111-ib.pocstor_35625_object23801/head//22 by client.940330.0:55109 2014-08-11 16:31:47.976701 2014-08-13 18:06:06.532605 7f5c973a57a0 20 read_log 192888'475 (192888'457) delete d762ebd/benchmark_data_storage0111-ib.pocstor_35625_object24691/head//22 by client.940330.0:55999 2014-08-11 16:31:48.241379 2014-08-13 18:06:06.532618 7f5c973a57a0 20 read_log 192888'476 (192888'458) delete df3c4ebd/benchmark_data_storage0111-ib.pocstor_35625_object28332/head//22 by client.940330.0:59640 2014-08-11 16:31:49.293771 2014-08-13 18:06:06.532641 7f5c973a57a0 20 read_log 192888'477 (192888'459) delete dbe02ebd/benchmark_data_storage0111-ib.pocstor_35625_object28470/head//22 by client.940330.0:59778 2014-08-11 16:31:49.329750 2014-08-13 18:06:06.532657 7f5c973a57a0 20 read_log 192888'478 (192888'460) delete ee9126bd/benchmark_data_storage0111-ib.pocstor_35625_object30366/head//22 by client.940330.0:61674 2014-08-11 16:31:49.863707 2014-08-13 18:06:06.532671 7f5c973a57a0 20 read_log 194215'479 (0'0) modify d80e4ebd/benchmark_data_storage0111-ib.pocstor_64498_object7326/head//22 by client.1049635.0:7327 2014-08-12 17:17:40.049105 2014-08-13 18:06:06.532683 7f5c973a57a0 20 read_log 194215'480 (0'0) modify 3bceb6bd/benchmark_data_storage0111-ib.pocstor_64498_object12432/head//22 by client.1049635.0:12433 2014-08-12 17:17:55.711235 2014-08-13 18:06:06.532695 7f5c973a57a0 20 read_log 194215'481 (0'0) modify 176e3ebd/benchmark_data_storage0111-ib.pocstor_64498_object13452/head//22 by client.1049635.0:13453 2014-08-12 17:17:58.810035 2014-08-13 18:06:06.532708 7f5c973a57a0 20 read_log 194215'482 (0'0) modify 4e7d2ebd/benchmark_data_storage0111-ib.pocstor_64498_object13989/head//22 by client.1049635.0:13990 2014-08-12 17:18:00.496105 2014-08-13 18:06:06.532720 7f5c973a57a0 20 read_log 194215'483 (0'0) modify 7fc8b6bd/benchmark_data_storage0111-ib.pocstor_64498_object15927/head//22 by client.1049635.0:15928 2014-08-12 17:18:07.278169 2014-08-13 18:06:06.532732 7f5c973a57a0 20 read_log 194215'484 (0'0) modify 1ff32ebd/benchmark_data_storage0111-ib.pocstor_64498_object16682/head//22 by client.1049635.0:16683 2014-08-12 17:18:10.164760 2014-08-13 18:06:06.532744 7f5c973a57a0 20 read_log 194215'485 (0'0) modify b2a0eebd/benchmark_data_storage0111-ib.pocstor_64498_object20409/head//22 by client.1049635.0:20410 2014-08-12 17:18:24.330984 2014-08-13 18:06:06.532756 7f5c973a57a0 20 read_log 194215'486 (0'0) modify a611b6bd/benchmark_data_storage0111-ib.pocstor_64498_object21741/head//22 by client.1049635.0:21742 2014-08-12 17:18:29.357621 2014-08-13 18:06:06.532768 7f5c973a57a0 20 read_log 194215'487 (0'0) modify 64ff96bd/benchmark_data_storage0111-ib.pocstor_64498_object25750/head//22 by client.1049635.0:25751 2014-08-12 17:18:44.499040 2014-08-13 18:06:06.532781 7f5c973a57a0 20 read_log 194215'488 (0'0) modify 98271ebd/benchmark_data_storage0111-ib.pocstor_64498_object29249/head//22 by client.1049635.0:29250 2014-08-12 17:18:57.521174 2014-08-13 18:06:06.532795 7f5c973a57a0 20 read_log 194215'489 (0'0) modify 65a15ebd/benchmark_data_storage0111-ib.pocstor_64498_object31539/head//22 by client.1049635.0:31540 2014-08-12 17:19:06.778643 2014-08-13 18:06:06.532808 7f5c973a57a0 20 read_log 194215'490 (0'0) modify cd7296bd/benchmark_data_storage0111-ib.pocstor_64498_object32223/head//22 by client.1049635.0:32224 2014-08-12 17:19:09.425686 2014-08-13 18:06:06.532828 7f5c973a57a0 20 read_log 194215'491 (0'0) modify caa3cebd/benchmark_data_storage0111-ib.pocstor_64498_object32334/head//22 by client.1049635.0:32335 2014-08-12 17:19:09.856014 2014-08-13 18:06:06.532843 7f5c973a57a0 20 read_log 194215'492 (194215'479) delete d80e4ebd/benchmark_data_storage0111-ib.pocstor_64498_object7326/head//22 by client.1049635.0:41529 2014-08-12 17:19:20.394683 2014-08-13 18:06:06.532874 7f5c973a57a0 20 read_log 194215'493 (194215'480) delete 3bceb6bd/benchmark_data_storage0111-ib.pocstor_64498_object12432/head//22 by client.1049635.0:46635 2014-08-12 17:19:22.033587 2014-08-13 18:06:06.532891 7f5c973a57a0 20 read_log 194215'494 (194215'481) delete 176e3ebd/benchmark_data_storage0111-ib.pocstor_64498_object13452/head//22 by client.1049635.0:47655 2014-08-12 17:19:22.378598 2014-08-13 18:06:06.532903 7f5c973a57a0 20 read_log 194215'495 (194215'482) delete 4e7d2ebd/benchmark_data_storage0111-ib.pocstor_64498_object13989/head//22 by client.1049635.0:48192 2014-08-12 17:19:22.560768 2014-08-13 18:06:06.532916 7f5c973a57a0 20 read_log 194215'496 (194215'483) delete 7fc8b6bd/benchmark_data_storage0111-ib.pocstor_64498_object15927/head//22 by client.1049635.0:50130 2014-08-12 17:19:23.168613 2014-08-13 18:06:06.532928 7f5c973a57a0 20 read_log 194215'497 (194215'484) delete 1ff32ebd/benchmark_data_storage0111-ib.pocstor_64498_object16682/head//22 by client.1049635.0:50885 2014-08-12 17:19:23.410536 2014-08-13 18:06:06.532940 7f5c973a57a0 20 read_log 194215'498 (194215'485) delete b2a0eebd/benchmark_data_storage0111-ib.pocstor_64498_object20409/head//22 by client.1049635.0:54612 2014-08-12 17:19:24.670792 2014-08-13 18:06:06.532952 7f5c973a57a0 20 read_log 194215'499 (194215'486) delete a611b6bd/benchmark_data_storage0111-ib.pocstor_64498_object21741/head//22 by client.1049635.0:55944 2014-08-12 17:19:25.139696 2014-08-13 18:06:06.532966 7f5c973a57a0 20 read_log 194215'500 (194215'487) delete 64ff96bd/benchmark_data_storage0111-ib.pocstor_64498_object25750/head//22 by client.1049635.0:59953 2014-08-12 17:19:26.575293 2014-08-13 18:06:06.532978 7f5c973a57a0 20 read_log 194215'501 (194215'488) delete 98271ebd/benchmark_data_storage0111-ib.pocstor_64498_object29249/head//22 by client.1049635.0:63452 2014-08-12 17:19:27.801207 2014-08-13 18:06:06.532990 7f5c973a57a0 20 read_log 194215'502 (194215'489) delete 65a15ebd/benchmark_data_storage0111-ib.pocstor_64498_object31539/head//22 by client.1049635.0:65742 2014-08-12 17:19:28.572689 2014-08-13 18:06:06.533002 7f5c973a57a0 20 read_log 194215'503 (194215'490) delete cd7296bd/benchmark_data_storage0111-ib.pocstor_64498_object32223/head//22 by client.1049635.0:66426 2014-08-12 17:19:28.827080 2014-08-13 18:06:06.533014 7f5c973a57a0 20 read_log 194215'504 (194215'491) delete caa3cebd/benchmark_data_storage0111-ib.pocstor_64498_object32334/head//22 by client.1049635.0:66537 2014-08-12 17:19:28.867432 2014-08-13 18:06:06.533097 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.533118 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.533133 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001824 0 0.000000 2014-08-13 18:06:06.533144 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.533158 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=0 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.533167 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'504], crt=191727'442) 2014-08-13 18:06:06.533176 7f5c973a57a0 10 osd.4 194487 pgid 22.707 coll 22.707_head 2014-08-13 18:06:06.533341 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.707 2014-08-13 18:06:06.533351 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.533376 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.707(unlocked)] enter Initial 2014-08-13 18:06:06.533392 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.707(unlocked)] enter NotTrimming 2014-08-13 18:06:06.533572 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.533690 7f5c973a57a0 20 read_log 191648'85 (0'0) modify 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 by client.273158.0:5717 2014-07-30 14:09:14.310106 2014-08-13 18:06:06.533711 7f5c973a57a0 20 read_log 191648'92 (0'0) modify e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 by client.273158.0:6406 2014-07-30 14:09:18.250213 2014-08-13 18:06:06.533732 7f5c973a57a0 20 read_log 191648'116 (0'0) modify 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 by client.273158.0:8225 2014-07-30 14:09:27.485296 2014-08-13 18:06:06.533747 7f5c973a57a0 20 read_log 191648'130 (0'0) modify 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 by client.273158.0:8996 2014-07-30 14:09:31.157678 2014-08-13 18:06:06.533759 7f5c973a57a0 20 read_log 191648'282 (0'0) modify 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 by client.273158.0:18038 2014-07-30 14:10:18.225194 2014-08-13 18:06:06.533772 7f5c973a57a0 20 read_log 191648'285 (0'0) modify e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 by client.273158.0:18159 2014-07-30 14:10:18.774337 2014-08-13 18:06:06.533784 7f5c973a57a0 20 read_log 191648'312 (0'0) modify dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 by client.273158.0:20042 2014-07-30 14:10:31.453445 2014-08-13 18:06:06.533796 7f5c973a57a0 20 read_log 191648'320 (0'0) modify 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 by client.273158.0:20422 2014-07-30 14:10:33.090957 2014-08-13 18:06:06.533808 7f5c973a57a0 20 read_log 191727'351 (0'0) modify b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 by client.275175.0:262 2014-07-30 14:18:34.439021 2014-08-13 18:06:06.533821 7f5c973a57a0 20 read_log 191727'352 (0'0) modify 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 by client.275175.0:597 2014-07-30 14:18:35.811348 2014-08-13 18:06:06.533833 7f5c973a57a0 20 read_log 191727'373 (0'0) modify 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 by client.275175.0:10862 2014-07-30 14:19:07.658492 2014-08-13 18:06:06.533845 7f5c973a57a0 20 read_log 191727'374 (0'0) modify a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 by client.275175.0:11395 2014-07-30 14:19:09.208503 2014-08-13 18:06:06.533856 7f5c973a57a0 20 read_log 191727'379 (0'0) modify d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 by client.275175.0:14707 2014-07-30 14:19:19.872066 2014-08-13 18:06:06.533869 7f5c973a57a0 20 read_log 191727'385 (0'0) modify 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 by client.275175.0:18723 2014-07-30 14:19:32.163026 2014-08-13 18:06:06.533881 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 by client.275175.0:18847 2014-07-30 14:19:32.745583 2014-08-13 18:06:06.533893 7f5c973a57a0 20 read_log 191727'388 (0'0) modify 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 by client.275175.0:19494 2014-07-30 14:19:34.761816 2014-08-13 18:06:06.533905 7f5c973a57a0 20 read_log 191727'391 (0'0) modify e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 by client.275175.0:21352 2014-07-30 14:19:40.165345 2014-08-13 18:06:06.533917 7f5c973a57a0 20 read_log 191727'414 (0'0) modify f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 by client.275175.0:36300 2014-07-30 14:20:26.191623 2014-08-13 18:06:06.533929 7f5c973a57a0 20 read_log 191727'416 (0'0) modify effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 by client.275175.0:37312 2014-07-30 14:20:29.239298 2014-08-13 18:06:06.533941 7f5c973a57a0 20 read_log 192888'418 (0'0) modify ffdaa707/benchmark_data_storage0111-ib.pocstor_35625_object1811/head//22 by client.940330.0:1812 2014-08-11 16:29:47.854039 2014-08-13 18:06:06.533953 7f5c973a57a0 20 read_log 192888'419 (0'0) modify 192d8f07/benchmark_data_storage0111-ib.pocstor_35625_object9977/head//22 by client.940330.0:9978 2014-08-11 16:30:17.804275 2014-08-13 18:06:06.533973 7f5c973a57a0 20 read_log 192888'420 (0'0) modify a7600f07/benchmark_data_storage0111-ib.pocstor_35625_object11904/head//22 by client.940330.0:11905 2014-08-11 16:30:25.450136 2014-08-13 18:06:06.533985 7f5c973a57a0 20 read_log 192888'421 (0'0) modify 7132d707/benchmark_data_storage0111-ib.pocstor_35625_object14936/head//22 by client.940330.0:14937 2014-08-11 16:30:37.254766 2014-08-13 18:06:06.533999 7f5c973a57a0 20 read_log 192888'422 (0'0) modify 17fd8707/benchmark_data_storage0111-ib.pocstor_35625_object16064/head//22 by client.940330.0:16065 2014-08-11 16:30:41.474318 2014-08-13 18:06:06.534011 7f5c973a57a0 20 read_log 192888'423 (0'0) modify 7aadaf07/benchmark_data_storage0111-ib.pocstor_35625_object18029/head//22 by client.940330.0:18030 2014-08-11 16:30:48.833523 2014-08-13 18:06:06.534023 7f5c973a57a0 20 read_log 192888'424 (0'0) modify f68bdf07/benchmark_data_storage0111-ib.pocstor_35625_object19965/head//22 by client.940330.0:19966 2014-08-11 16:30:56.686064 2014-08-13 18:06:06.534035 7f5c973a57a0 20 read_log 192888'425 (0'0) modify f804a707/benchmark_data_storage0111-ib.pocstor_35625_object20351/head//22 by client.940330.0:20352 2014-08-11 16:30:58.144701 2014-08-13 18:06:06.534047 7f5c973a57a0 20 read_log 192888'426 (0'0) modify 2a46b707/benchmark_data_storage0111-ib.pocstor_35625_object23245/head//22 by client.940330.0:23246 2014-08-11 16:31:09.311945 2014-08-13 18:06:06.534058 7f5c973a57a0 20 read_log 192888'427 (0'0) modify 66df6707/benchmark_data_storage0111-ib.pocstor_35625_object23606/head//22 by client.940330.0:23607 2014-08-11 16:31:10.644784 2014-08-13 18:06:06.534070 7f5c973a57a0 20 read_log 192888'428 (192888'418) delete ffdaa707/benchmark_data_storage0111-ib.pocstor_35625_object1811/head//22 by client.940330.0:33119 2014-08-11 16:31:41.942777 2014-08-13 18:06:06.534083 7f5c973a57a0 20 read_log 192888'429 (192888'419) delete 192d8f07/benchmark_data_storage0111-ib.pocstor_35625_object9977/head//22 by client.940330.0:41285 2014-08-11 16:31:44.107939 2014-08-13 18:06:06.534095 7f5c973a57a0 20 read_log 192888'430 (192888'420) delete a7600f07/benchmark_data_storage0111-ib.pocstor_35625_object11904/head//22 by client.940330.0:43212 2014-08-11 16:31:44.634586 2014-08-13 18:06:06.534107 7f5c973a57a0 20 read_log 192888'431 (192888'421) delete 7132d707/benchmark_data_storage0111-ib.pocstor_35625_object14936/head//22 by client.940330.0:46244 2014-08-11 16:31:45.440139 2014-08-13 18:06:06.534119 7f5c973a57a0 20 read_log 192888'432 (192888'422) delete 17fd8707/benchmark_data_storage0111-ib.pocstor_35625_object16064/head//22 by client.940330.0:47372 2014-08-11 16:31:45.743082 2014-08-13 18:06:06.534134 7f5c973a57a0 20 read_log 192888'433 (192888'423) delete 7aadaf07/benchmark_data_storage0111-ib.pocstor_35625_object18029/head//22 by client.940330.0:49337 2014-08-11 16:31:46.295067 2014-08-13 18:06:06.534155 7f5c973a57a0 20 read_log 192888'434 (192888'424) delete f68bdf07/benchmark_data_storage0111-ib.pocstor_35625_object19965/head//22 by client.940330.0:51273 2014-08-11 16:31:46.859799 2014-08-13 18:06:06.534171 7f5c973a57a0 20 read_log 192888'435 (192888'425) delete f804a707/benchmark_data_storage0111-ib.pocstor_35625_object20351/head//22 by client.940330.0:51659 2014-08-11 16:31:46.980897 2014-08-13 18:06:06.534184 7f5c973a57a0 20 read_log 192888'436 (192888'426) delete 2a46b707/benchmark_data_storage0111-ib.pocstor_35625_object23245/head//22 by client.940330.0:54553 2014-08-11 16:31:47.829576 2014-08-13 18:06:06.534196 7f5c973a57a0 20 read_log 192888'437 (192888'427) delete 66df6707/benchmark_data_storage0111-ib.pocstor_35625_object23606/head//22 by client.940330.0:54914 2014-08-11 16:31:47.925474 2014-08-13 18:06:06.534209 7f5c973a57a0 20 read_log 194215'438 (0'0) modify b74b7f07/benchmark_data_storage0111-ib.pocstor_64498_object628/head//22 by client.1049635.0:629 2014-08-12 17:17:19.139874 2014-08-13 18:06:06.534230 7f5c973a57a0 20 read_log 194215'439 (0'0) modify a6b69707/benchmark_data_storage0111-ib.pocstor_64498_object5166/head//22 by client.1049635.0:5167 2014-08-12 17:17:33.429115 2014-08-13 18:06:06.534269 7f5c973a57a0 20 read_log 194215'440 (0'0) modify cd906f07/benchmark_data_storage0111-ib.pocstor_64498_object5298/head//22 by client.1049635.0:5299 2014-08-12 17:17:33.829471 2014-08-13 18:06:06.534285 7f5c973a57a0 20 read_log 194215'441 (0'0) modify 3c992707/benchmark_data_storage0111-ib.pocstor_64498_object6667/head//22 by client.1049635.0:6668 2014-08-12 17:17:38.051169 2014-08-13 18:06:06.534297 7f5c973a57a0 20 read_log 194215'442 (0'0) modify 78daf707/benchmark_data_storage0111-ib.pocstor_64498_object7060/head//22 by client.1049635.0:7061 2014-08-12 17:17:39.302128 2014-08-13 18:06:06.534309 7f5c973a57a0 20 read_log 194215'443 (0'0) modify f54fbf07/benchmark_data_storage0111-ib.pocstor_64498_object9880/head//22 by client.1049635.0:9881 2014-08-12 17:17:47.583868 2014-08-13 18:06:06.534324 7f5c973a57a0 20 read_log 194215'444 (0'0) modify ffcca707/benchmark_data_storage0111-ib.pocstor_64498_object11309/head//22 by client.1049635.0:11310 2014-08-12 17:17:52.146349 2014-08-13 18:06:06.534336 7f5c973a57a0 20 read_log 194215'445 (0'0) modify edf48f07/benchmark_data_storage0111-ib.pocstor_64498_object14376/head//22 by client.1049635.0:14377 2014-08-12 17:18:01.858342 2014-08-13 18:06:06.534349 7f5c973a57a0 20 read_log 194215'446 (0'0) modify a8ffa707/benchmark_data_storage0111-ib.pocstor_64498_object16042/head//22 by client.1049635.0:16043 2014-08-12 17:18:07.709330 2014-08-13 18:06:06.534361 7f5c973a57a0 20 read_log 194215'447 (0'0) modify cf65af07/benchmark_data_storage0111-ib.pocstor_64498_object16219/head//22 by client.1049635.0:16220 2014-08-12 17:18:08.388202 2014-08-13 18:06:06.534373 7f5c973a57a0 20 read_log 194215'448 (0'0) modify 9a61b707/benchmark_data_storage0111-ib.pocstor_64498_object16970/head//22 by client.1049635.0:16971 2014-08-12 17:18:11.249491 2014-08-13 18:06:06.534387 7f5c973a57a0 20 read_log 194215'449 (0'0) modify fa523707/benchmark_data_storage0111-ib.pocstor_64498_object18011/head//22 by client.1049635.0:18012 2014-08-12 17:18:15.205555 2014-08-13 18:06:06.534411 7f5c973a57a0 20 read_log 194215'450 (0'0) modify 13a8c707/benchmark_data_storage0111-ib.pocstor_64498_object18082/head//22 by client.1049635.0:18083 2014-08-12 17:18:15.475173 2014-08-13 18:06:06.534428 7f5c973a57a0 20 read_log 194215'451 (0'0) modify 54a42707/benchmark_data_storage0111-ib.pocstor_64498_object18214/head//22 by client.1049635.0:18215 2014-08-12 17:18:15.970648 2014-08-13 18:06:06.534440 7f5c973a57a0 20 read_log 194215'452 (0'0) modify e6c1e707/benchmark_data_storage0111-ib.pocstor_64498_object19980/head//22 by client.1049635.0:19981 2014-08-12 17:18:22.707134 2014-08-13 18:06:06.534469 7f5c973a57a0 20 read_log 194215'453 (0'0) modify 6cf86f07/benchmark_data_storage0111-ib.pocstor_64498_object20600/head//22 by client.1049635.0:20601 2014-08-12 17:18:25.039329 2014-08-13 18:06:06.534482 7f5c973a57a0 20 read_log 194215'454 (0'0) modify 2b385f07/benchmark_data_storage0111-ib.pocstor_64498_object24619/head//22 by client.1049635.0:24620 2014-08-12 17:18:40.163976 2014-08-13 18:06:06.534505 7f5c973a57a0 20 read_log 194215'455 (0'0) modify 63021f07/benchmark_data_storage0111-ib.pocstor_64498_object25371/head//22 by client.1049635.0:25372 2014-08-12 17:18:43.086215 2014-08-13 18:06:06.534520 7f5c973a57a0 20 read_log 194215'456 (0'0) modify 74249707/benchmark_data_storage0111-ib.pocstor_64498_object26493/head//22 by client.1049635.0:26494 2014-08-12 17:18:47.262989 2014-08-13 18:06:06.534535 7f5c973a57a0 20 read_log 194215'457 (0'0) modify 244f6707/benchmark_data_storage0111-ib.pocstor_64498_object27386/head//22 by client.1049635.0:27387 2014-08-12 17:18:50.595924 2014-08-13 18:06:06.534555 7f5c973a57a0 20 read_log 194215'458 (0'0) modify 384ad707/benchmark_data_storage0111-ib.pocstor_64498_object28792/head//22 by client.1049635.0:28793 2014-08-12 17:18:55.824861 2014-08-13 18:06:06.534579 7f5c973a57a0 20 read_log 194215'459 (0'0) modify 91cec707/benchmark_data_storage0111-ib.pocstor_64498_object28812/head//22 by client.1049635.0:28813 2014-08-12 17:18:55.898241 2014-08-13 18:06:06.534594 7f5c973a57a0 20 read_log 194215'460 (0'0) modify d05a7707/benchmark_data_storage0111-ib.pocstor_64498_object34198/head//22 by client.1049635.0:34199 2014-08-12 17:19:16.932840 2014-08-13 18:06:06.534606 7f5c973a57a0 20 read_log 194215'461 (194215'438) delete b74b7f07/benchmark_data_storage0111-ib.pocstor_64498_object628/head//22 by client.1049635.0:34831 2014-08-12 17:19:18.199222 2014-08-13 18:06:06.534619 7f5c973a57a0 20 read_log 194215'462 (194215'439) delete a6b69707/benchmark_data_storage0111-ib.pocstor_64498_object5166/head//22 by client.1049635.0:39369 2014-08-12 17:19:19.660729 2014-08-13 18:06:06.534631 7f5c973a57a0 20 read_log 194215'463 (194215'440) delete cd906f07/benchmark_data_storage0111-ib.pocstor_64498_object5298/head//22 by client.1049635.0:39501 2014-08-12 17:19:19.707448 2014-08-13 18:06:06.534643 7f5c973a57a0 20 read_log 194215'464 (194215'441) delete 3c992707/benchmark_data_storage0111-ib.pocstor_64498_object6667/head//22 by client.1049635.0:40870 2014-08-12 17:19:20.176318 2014-08-13 18:06:06.534656 7f5c973a57a0 20 read_log 194215'465 (194215'442) delete 78daf707/benchmark_data_storage0111-ib.pocstor_64498_object7060/head//22 by client.1049635.0:41263 2014-08-12 17:19:20.300553 2014-08-13 18:06:06.534668 7f5c973a57a0 20 read_log 194215'466 (194215'443) delete f54fbf07/benchmark_data_storage0111-ib.pocstor_64498_object9880/head//22 by client.1049635.0:44083 2014-08-12 17:19:21.219078 2014-08-13 18:06:06.534680 7f5c973a57a0 20 read_log 194215'467 (194215'444) delete ffcca707/benchmark_data_storage0111-ib.pocstor_64498_object11309/head//22 by client.1049635.0:45512 2014-08-12 17:19:21.675554 2014-08-13 18:06:06.534693 7f5c973a57a0 20 read_log 194215'468 (194215'445) delete edf48f07/benchmark_data_storage0111-ib.pocstor_64498_object14376/head//22 by client.1049635.0:48579 2014-08-12 17:19:22.682512 2014-08-13 18:06:06.534706 7f5c973a57a0 20 read_log 194215'469 (194215'446) delete a8ffa707/benchmark_data_storage0111-ib.pocstor_64498_object16042/head//22 by client.1049635.0:50245 2014-08-12 17:19:23.203423 2014-08-13 18:06:06.534718 7f5c973a57a0 20 read_log 194215'470 (194215'447) delete cf65af07/benchmark_data_storage0111-ib.pocstor_64498_object16219/head//22 by client.1049635.0:50422 2014-08-12 17:19:23.260054 2014-08-13 18:06:06.534730 7f5c973a57a0 20 read_log 194215'471 (194215'448) delete 9a61b707/benchmark_data_storage0111-ib.pocstor_64498_object16970/head//22 by client.1049635.0:51173 2014-08-12 17:19:23.499810 2014-08-13 18:06:06.534746 7f5c973a57a0 20 read_log 194215'472 (194215'449) delete fa523707/benchmark_data_storage0111-ib.pocstor_64498_object18011/head//22 by client.1049635.0:52214 2014-08-12 17:19:23.833534 2014-08-13 18:06:06.534759 7f5c973a57a0 20 read_log 194215'473 (194215'450) delete 13a8c707/benchmark_data_storage0111-ib.pocstor_64498_object18082/head//22 by client.1049635.0:52285 2014-08-12 17:19:23.859171 2014-08-13 18:06:06.534771 7f5c973a57a0 20 read_log 194215'474 (194215'451) delete 54a42707/benchmark_data_storage0111-ib.pocstor_64498_object18214/head//22 by client.1049635.0:52417 2014-08-12 17:19:23.901419 2014-08-13 18:06:06.534794 7f5c973a57a0 20 read_log 194215'475 (194215'452) delete e6c1e707/benchmark_data_storage0111-ib.pocstor_64498_object19980/head//22 by client.1049635.0:54183 2014-08-12 17:19:24.507027 2014-08-13 18:06:06.534809 7f5c973a57a0 20 read_log 194215'476 (194215'453) delete 6cf86f07/benchmark_data_storage0111-ib.pocstor_64498_object20600/head//22 by client.1049635.0:54803 2014-08-12 17:19:24.741678 2014-08-13 18:06:06.534822 7f5c973a57a0 20 read_log 194215'477 (194215'454) delete 2b385f07/benchmark_data_storage0111-ib.pocstor_64498_object24619/head//22 by client.1049635.0:58822 2014-08-12 17:19:26.180530 2014-08-13 18:06:06.534836 7f5c973a57a0 20 read_log 194215'478 (194215'455) delete 63021f07/benchmark_data_storage0111-ib.pocstor_64498_object25371/head//22 by client.1049635.0:59574 2014-08-12 17:19:26.440954 2014-08-13 18:06:06.534850 7f5c973a57a0 20 read_log 194215'479 (194215'456) delete 74249707/benchmark_data_storage0111-ib.pocstor_64498_object26493/head//22 by client.1049635.0:60696 2014-08-12 17:19:26.819296 2014-08-13 18:06:06.534863 7f5c973a57a0 20 read_log 194215'480 (194215'457) delete 244f6707/benchmark_data_storage0111-ib.pocstor_64498_object27386/head//22 by client.1049635.0:61589 2014-08-12 17:19:27.140798 2014-08-13 18:06:06.534875 7f5c973a57a0 20 read_log 194215'481 (194215'458) delete 384ad707/benchmark_data_storage0111-ib.pocstor_64498_object28792/head//22 by client.1049635.0:62995 2014-08-12 17:19:27.627696 2014-08-13 18:06:06.534889 7f5c973a57a0 20 read_log 194215'482 (194215'459) delete 91cec707/benchmark_data_storage0111-ib.pocstor_64498_object28812/head//22 by client.1049635.0:63015 2014-08-12 17:19:27.632442 2014-08-13 18:06:06.534902 7f5c973a57a0 20 read_log 194215'483 (194215'460) delete d05a7707/benchmark_data_storage0111-ib.pocstor_64498_object34198/head//22 by client.1049635.0:68401 2014-08-12 17:19:29.552584 2014-08-13 18:06:06.535009 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.535029 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=0 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.535052 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=0 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001676 0 0.000000 2014-08-13 18:06:06.535067 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=0 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.535076 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=0 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.535086 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'483], crt=191727'417) 2014-08-13 18:06:06.535096 7f5c973a57a0 10 osd.4 194487 pgid 22.775 coll 22.775_head 2014-08-13 18:06:06.535260 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.775 2014-08-13 18:06:06.535272 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.535308 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.775(unlocked)] enter Initial 2014-08-13 18:06:06.535320 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.775(unlocked)] enter NotTrimming 2014-08-13 18:06:06.535480 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.535623 7f5c973a57a0 20 read_log 191648'55 (0'0) modify f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 by client.273158.0:3019 2014-07-30 14:09:01.459405 2014-08-13 18:06:06.535644 7f5c973a57a0 20 read_log 191648'59 (0'0) modify d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 by client.273158.0:3104 2014-07-30 14:09:01.784898 2014-08-13 18:06:06.535657 7f5c973a57a0 20 read_log 191648'93 (0'0) modify c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 by client.273158.0:4778 2014-07-30 14:09:10.036012 2014-08-13 18:06:06.535669 7f5c973a57a0 20 read_log 191648'107 (0'0) modify e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 by client.273158.0:5727 2014-07-30 14:09:14.350650 2014-08-13 18:06:06.535681 7f5c973a57a0 20 read_log 191648'130 (0'0) modify 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 by client.273158.0:6835 2014-07-30 14:09:20.420522 2014-08-13 18:06:06.535693 7f5c973a57a0 20 read_log 191648'138 (0'0) modify f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 by client.273158.0:7388 2014-07-30 14:09:22.717046 2014-08-13 18:06:06.535705 7f5c973a57a0 20 read_log 191648'199 (0'0) modify f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 by client.273158.0:11404 2014-07-30 14:09:43.889130 2014-08-13 18:06:06.535718 7f5c973a57a0 20 read_log 191648'275 (0'0) modify c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 by client.273158.0:15851 2014-07-30 14:10:06.539259 2014-08-13 18:06:06.535730 7f5c973a57a0 20 read_log 191648'281 (0'0) modify 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 by client.273158.0:16442 2014-07-30 14:10:09.851192 2014-08-13 18:06:06.535742 7f5c973a57a0 20 read_log 191648'308 (0'0) modify 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 by client.273158.0:18988 2014-07-30 14:10:23.577343 2014-08-13 18:06:06.535754 7f5c973a57a0 20 read_log 191648'323 (0'0) modify 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 by client.273158.0:20241 2014-07-30 14:10:32.420087 2014-08-13 18:06:06.535766 7f5c973a57a0 20 read_log 191727'370 (0'0) modify 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 by client.275175.0:1991 2014-07-30 14:18:40.868696 2014-08-13 18:06:06.535778 7f5c973a57a0 20 read_log 191727'374 (0'0) modify ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 by client.275175.0:3077 2014-07-30 14:18:44.466494 2014-08-13 18:06:06.535790 7f5c973a57a0 20 read_log 191727'383 (0'0) modify cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 by client.275175.0:5894 2014-07-30 14:18:52.942030 2014-08-13 18:06:06.535802 7f5c973a57a0 20 read_log 191727'384 (0'0) modify 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 by client.275175.0:7758 2014-07-30 14:18:58.850466 2014-08-13 18:06:06.535814 7f5c973a57a0 20 read_log 191727'388 (0'0) modify fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 by client.275175.0:9745 2014-07-30 14:19:04.467815 2014-08-13 18:06:06.535825 7f5c973a57a0 20 read_log 191727'389 (0'0) modify ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 by client.275175.0:11770 2014-07-30 14:19:10.522279 2014-08-13 18:06:06.535837 7f5c973a57a0 20 read_log 191727'392 (0'0) modify a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 by client.275175.0:13336 2014-07-30 14:19:15.680276 2014-08-13 18:06:06.535849 7f5c973a57a0 20 read_log 191727'399 (0'0) modify ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 by client.275175.0:16060 2014-07-30 14:19:24.102573 2014-08-13 18:06:06.535870 7f5c973a57a0 20 read_log 191727'400 (0'0) modify 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 by client.275175.0:16311 2014-07-30 14:19:24.994509 2014-08-13 18:06:06.535885 7f5c973a57a0 20 read_log 191727'401 (0'0) modify d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 by client.275175.0:16477 2014-07-30 14:19:25.598642 2014-08-13 18:06:06.535897 7f5c973a57a0 20 read_log 191727'402 (0'0) modify 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 by client.275175.0:16647 2014-07-30 14:19:26.162746 2014-08-13 18:06:06.535909 7f5c973a57a0 20 read_log 191727'410 (0'0) modify 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 by client.275175.0:19897 2014-07-30 14:19:35.877318 2014-08-13 18:06:06.535921 7f5c973a57a0 20 read_log 191727'421 (0'0) modify 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 by client.275175.0:23920 2014-07-30 14:19:48.609469 2014-08-13 18:06:06.535933 7f5c973a57a0 20 read_log 191727'429 (0'0) modify fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 by client.275175.0:28023 2014-07-30 14:20:01.091986 2014-08-13 18:06:06.535945 7f5c973a57a0 20 read_log 191727'433 (0'0) modify 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 by client.275175.0:29710 2014-07-30 14:20:06.074603 2014-08-13 18:06:06.535957 7f5c973a57a0 20 read_log 191727'434 (0'0) modify 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 by client.275175.0:30317 2014-07-30 14:20:08.061941 2014-08-13 18:06:06.535971 7f5c973a57a0 20 read_log 191727'441 (0'0) modify 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 by client.275175.0:34100 2014-07-30 14:20:19.275316 2014-08-13 18:06:06.535983 7f5c973a57a0 20 read_log 191727'442 (0'0) modify 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 by client.275175.0:34242 2014-07-30 14:20:19.744007 2014-08-13 18:06:06.535995 7f5c973a57a0 20 read_log 191727'445 (0'0) modify 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 by client.275175.0:34778 2014-07-30 14:20:21.343072 2014-08-13 18:06:06.536008 7f5c973a57a0 20 read_log 191727'450 (0'0) modify 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 by client.275175.0:36744 2014-07-30 14:20:27.658549 2014-08-13 18:06:06.536021 7f5c973a57a0 20 read_log 192674'454 (0'0) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1150 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536033 7f5c973a57a0 20 read_log 192674'455 (192674'454) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1151 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536045 7f5c973a57a0 20 read_log 192674'456 (192674'455) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1152 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536060 7f5c973a57a0 20 read_log 192674'457 (192674'456) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1153 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536072 7f5c973a57a0 20 read_log 192674'458 (192674'457) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1154 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536087 7f5c973a57a0 20 read_log 192674'459 (192674'458) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1155 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536099 7f5c973a57a0 20 read_log 192674'460 (192674'459) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1156 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536112 7f5c973a57a0 20 read_log 192674'461 (192674'460) modify 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 by client.279772.1:1157 2014-07-31 18:03:42.896000 2014-08-13 18:06:06.536134 7f5c973a57a0 20 read_log 192888'462 (0'0) modify 701fa775/benchmark_data_storage0111-ib.pocstor_35625_object2166/head//22 by client.940330.0:2167 2014-08-11 16:29:49.159399 2014-08-13 18:06:06.536155 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 773a2f75/benchmark_data_storage0111-ib.pocstor_35625_object4806/head//22 by client.940330.0:4807 2014-08-11 16:29:59.347606 2014-08-13 18:06:06.536170 7f5c973a57a0 20 read_log 192888'464 (0'0) modify 99d48775/benchmark_data_storage0111-ib.pocstor_35625_object13110/head//22 by client.940330.0:13111 2014-08-11 16:30:30.315121 2014-08-13 18:06:06.536183 7f5c973a57a0 20 read_log 192888'465 (0'0) modify c8262775/benchmark_data_storage0111-ib.pocstor_35625_object14655/head//22 by client.940330.0:14656 2014-08-11 16:30:36.210678 2014-08-13 18:06:06.536195 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 89a37f75/benchmark_data_storage0111-ib.pocstor_35625_object15019/head//22 by client.940330.0:15020 2014-08-11 16:30:37.565249 2014-08-13 18:06:06.536207 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 5e9ca775/benchmark_data_storage0111-ib.pocstor_35625_object16815/head//22 by client.940330.0:16816 2014-08-11 16:30:44.275872 2014-08-13 18:06:06.536221 7f5c973a57a0 20 read_log 192888'468 (0'0) modify 787ff775/benchmark_data_storage0111-ib.pocstor_35625_object18765/head//22 by client.940330.0:18766 2014-08-11 16:30:51.806638 2014-08-13 18:06:06.536254 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 570bf775/benchmark_data_storage0111-ib.pocstor_35625_object18912/head//22 by client.940330.0:18913 2014-08-11 16:30:52.344670 2014-08-13 18:06:06.536273 7f5c973a57a0 20 read_log 192888'470 (0'0) modify 7f5e6f75/benchmark_data_storage0111-ib.pocstor_35625_object20067/head//22 by client.940330.0:20068 2014-08-11 16:30:57.063322 2014-08-13 18:06:06.536289 7f5c973a57a0 20 read_log 192888'471 (0'0) modify 2ffa9f75/benchmark_data_storage0111-ib.pocstor_35625_object20283/head//22 by client.940330.0:20284 2014-08-11 16:30:57.895618 2014-08-13 18:06:06.536301 7f5c973a57a0 20 read_log 192888'472 (0'0) modify 51c84775/benchmark_data_storage0111-ib.pocstor_35625_object20461/head//22 by client.940330.0:20462 2014-08-11 16:30:58.552190 2014-08-13 18:06:06.536313 7f5c973a57a0 20 read_log 192888'473 (0'0) modify c08a1775/benchmark_data_storage0111-ib.pocstor_35625_object22215/head//22 by client.940330.0:22216 2014-08-11 16:31:05.372008 2014-08-13 18:06:06.536325 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 1032bf75/benchmark_data_storage0111-ib.pocstor_35625_object24071/head//22 by client.940330.0:24072 2014-08-11 16:31:12.408178 2014-08-13 18:06:06.536337 7f5c973a57a0 20 read_log 192888'475 (0'0) modify d1c43f75/benchmark_data_storage0111-ib.pocstor_35625_object25401/head//22 by client.940330.0:25402 2014-08-11 16:31:17.456028 2014-08-13 18:06:06.536349 7f5c973a57a0 20 read_log 192888'476 (0'0) modify adcbe775/benchmark_data_storage0111-ib.pocstor_35625_object26205/head//22 by client.940330.0:26206 2014-08-11 16:31:20.600409 2014-08-13 18:06:06.536366 7f5c973a57a0 20 read_log 192888'477 (192888'462) delete 701fa775/benchmark_data_storage0111-ib.pocstor_35625_object2166/head//22 by client.940330.0:33474 2014-08-11 16:31:42.034247 2014-08-13 18:06:06.536381 7f5c973a57a0 20 read_log 192888'478 (192888'463) delete 773a2f75/benchmark_data_storage0111-ib.pocstor_35625_object4806/head//22 by client.940330.0:36114 2014-08-11 16:31:42.769490 2014-08-13 18:06:06.536393 7f5c973a57a0 20 read_log 192888'479 (192888'464) delete 99d48775/benchmark_data_storage0111-ib.pocstor_35625_object13110/head//22 by client.940330.0:44418 2014-08-11 16:31:44.949156 2014-08-13 18:06:06.536416 7f5c973a57a0 20 read_log 192888'480 (192888'465) delete c8262775/benchmark_data_storage0111-ib.pocstor_35625_object14655/head//22 by client.940330.0:45963 2014-08-11 16:31:45.361611 2014-08-13 18:06:06.536436 7f5c973a57a0 20 read_log 192888'481 (192888'466) delete 89a37f75/benchmark_data_storage0111-ib.pocstor_35625_object15019/head//22 by client.940330.0:46327 2014-08-11 16:31:45.462967 2014-08-13 18:06:06.536470 7f5c973a57a0 20 read_log 192888'482 (192888'467) delete 5e9ca775/benchmark_data_storage0111-ib.pocstor_35625_object16815/head//22 by client.940330.0:48123 2014-08-11 16:31:45.938397 2014-08-13 18:06:06.536487 7f5c973a57a0 20 read_log 192888'483 (192888'468) delete 787ff775/benchmark_data_storage0111-ib.pocstor_35625_object18765/head//22 by client.940330.0:50073 2014-08-11 16:31:46.516766 2014-08-13 18:06:06.536500 7f5c973a57a0 20 read_log 192888'484 (192888'469) delete 570bf775/benchmark_data_storage0111-ib.pocstor_35625_object18912/head//22 by client.940330.0:50220 2014-08-11 16:31:46.557309 2014-08-13 18:06:06.536526 7f5c973a57a0 20 read_log 192888'485 (192888'470) delete 7f5e6f75/benchmark_data_storage0111-ib.pocstor_35625_object20067/head//22 by client.940330.0:51375 2014-08-11 16:31:46.899816 2014-08-13 18:06:06.536539 7f5c973a57a0 20 read_log 192888'486 (192888'471) delete 2ffa9f75/benchmark_data_storage0111-ib.pocstor_35625_object20283/head//22 by client.940330.0:51591 2014-08-11 16:31:46.963907 2014-08-13 18:06:06.536553 7f5c973a57a0 20 read_log 192888'487 (192888'472) delete 51c84775/benchmark_data_storage0111-ib.pocstor_35625_object20461/head//22 by client.940330.0:51769 2014-08-11 16:31:47.011071 2014-08-13 18:06:06.536568 7f5c973a57a0 20 read_log 192888'488 (192888'473) delete c08a1775/benchmark_data_storage0111-ib.pocstor_35625_object22215/head//22 by client.940330.0:53523 2014-08-11 16:31:47.515081 2014-08-13 18:06:06.536581 7f5c973a57a0 20 read_log 192888'489 (192888'474) delete 1032bf75/benchmark_data_storage0111-ib.pocstor_35625_object24071/head//22 by client.940330.0:55379 2014-08-11 16:31:48.061982 2014-08-13 18:06:06.536593 7f5c973a57a0 20 read_log 192888'490 (192888'475) delete d1c43f75/benchmark_data_storage0111-ib.pocstor_35625_object25401/head//22 by client.940330.0:56709 2014-08-11 16:31:48.468905 2014-08-13 18:06:06.536606 7f5c973a57a0 20 read_log 192888'491 (192888'476) delete adcbe775/benchmark_data_storage0111-ib.pocstor_35625_object26205/head//22 by client.940330.0:57513 2014-08-11 16:31:48.706086 2014-08-13 18:06:06.536618 7f5c973a57a0 20 read_log 194215'492 (0'0) modify d8fe4f75/benchmark_data_storage0111-ib.pocstor_64498_object6040/head//22 by client.1049635.0:6041 2014-08-12 17:17:36.191556 2014-08-13 18:06:06.536630 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 50715775/benchmark_data_storage0111-ib.pocstor_64498_object9862/head//22 by client.1049635.0:9863 2014-08-12 17:17:47.533366 2014-08-13 18:06:06.536642 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 1edfcf75/benchmark_data_storage0111-ib.pocstor_64498_object13368/head//22 by client.1049635.0:13369 2014-08-12 17:17:58.551763 2014-08-13 18:06:06.536654 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 6f2d9775/benchmark_data_storage0111-ib.pocstor_64498_object13671/head//22 by client.1049635.0:13672 2014-08-12 17:17:59.565349 2014-08-13 18:06:06.536666 7f5c973a57a0 20 read_log 194215'496 (0'0) modify c57baf75/benchmark_data_storage0111-ib.pocstor_64498_object15930/head//22 by client.1049635.0:15931 2014-08-12 17:18:07.290047 2014-08-13 18:06:06.536678 7f5c973a57a0 20 read_log 194215'497 (0'0) modify e23e3775/benchmark_data_storage0111-ib.pocstor_64498_object17738/head//22 by client.1049635.0:17739 2014-08-12 17:18:14.163227 2014-08-13 18:06:06.536690 7f5c973a57a0 20 read_log 194215'498 (0'0) modify c52be775/benchmark_data_storage0111-ib.pocstor_64498_object20965/head//22 by client.1049635.0:20966 2014-08-12 17:18:26.393736 2014-08-13 18:06:06.536702 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 7aab3f75/benchmark_data_storage0111-ib.pocstor_64498_object22635/head//22 by client.1049635.0:22636 2014-08-12 17:18:32.773532 2014-08-13 18:06:06.536714 7f5c973a57a0 20 read_log 194215'500 (0'0) modify ca9adf75/benchmark_data_storage0111-ib.pocstor_64498_object24677/head//22 by client.1049635.0:24678 2014-08-12 17:18:40.376462 2014-08-13 18:06:06.536728 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 6c2ecf75/benchmark_data_storage0111-ib.pocstor_64498_object26415/head//22 by client.1049635.0:26416 2014-08-12 17:18:46.967325 2014-08-13 18:06:06.536743 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 9942ff75/benchmark_data_storage0111-ib.pocstor_64498_object29962/head//22 by client.1049635.0:29963 2014-08-12 17:19:00.304655 2014-08-13 18:06:06.536756 7f5c973a57a0 20 read_log 194215'503 (194215'492) delete d8fe4f75/benchmark_data_storage0111-ib.pocstor_64498_object6040/head//22 by client.1049635.0:40243 2014-08-12 17:19:19.969487 2014-08-13 18:06:06.536768 7f5c973a57a0 20 read_log 194215'504 (194215'493) delete 50715775/benchmark_data_storage0111-ib.pocstor_64498_object9862/head//22 by client.1049635.0:44065 2014-08-12 17:19:21.212105 2014-08-13 18:06:06.536780 7f5c973a57a0 20 read_log 194215'505 (194215'494) delete 1edfcf75/benchmark_data_storage0111-ib.pocstor_64498_object13368/head//22 by client.1049635.0:47571 2014-08-12 17:19:22.348911 2014-08-13 18:06:06.536792 7f5c973a57a0 20 read_log 194215'506 (194215'495) delete 6f2d9775/benchmark_data_storage0111-ib.pocstor_64498_object13671/head//22 by client.1049635.0:47874 2014-08-12 17:19:22.463990 2014-08-13 18:06:06.536804 7f5c973a57a0 20 read_log 194215'507 (194215'496) delete c57baf75/benchmark_data_storage0111-ib.pocstor_64498_object15930/head//22 by client.1049635.0:50133 2014-08-12 17:19:23.168753 2014-08-13 18:06:06.536816 7f5c973a57a0 20 read_log 194215'508 (194215'497) delete e23e3775/benchmark_data_storage0111-ib.pocstor_64498_object17738/head//22 by client.1049635.0:51941 2014-08-12 17:19:23.739302 2014-08-13 18:06:06.536828 7f5c973a57a0 20 read_log 194215'509 (194215'498) delete c52be775/benchmark_data_storage0111-ib.pocstor_64498_object20965/head//22 by client.1049635.0:55168 2014-08-12 17:19:24.869220 2014-08-13 18:06:06.536840 7f5c973a57a0 20 read_log 194215'510 (194215'499) delete 7aab3f75/benchmark_data_storage0111-ib.pocstor_64498_object22635/head//22 by client.1049635.0:56838 2014-08-12 17:19:25.472617 2014-08-13 18:06:06.536852 7f5c973a57a0 20 read_log 194215'511 (194215'500) delete ca9adf75/benchmark_data_storage0111-ib.pocstor_64498_object24677/head//22 by client.1049635.0:58880 2014-08-12 17:19:26.201755 2014-08-13 18:06:06.536864 7f5c973a57a0 20 read_log 194215'512 (194215'501) delete 6c2ecf75/benchmark_data_storage0111-ib.pocstor_64498_object26415/head//22 by client.1049635.0:60618 2014-08-12 17:19:26.795142 2014-08-13 18:06:06.536876 7f5c973a57a0 20 read_log 194215'513 (194215'502) delete 9942ff75/benchmark_data_storage0111-ib.pocstor_64498_object29962/head//22 by client.1049635.0:64165 2014-08-12 17:19:28.033744 2014-08-13 18:06:06.536953 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.536971 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=0 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.536986 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=0 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.001679 0 0.000000 2014-08-13 18:06:06.536998 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=0 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.537007 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=0 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194486 2014-08-13 18:06:06.537016 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] log((0'0,194215'513], crt=191727'453) 2014-08-13 18:06:06.537026 7f5c973a57a0 10 osd.4 194487 pgid 22.791 coll 22.791_head 2014-08-13 18:06:06.537160 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 22.791 2014-08-13 18:06:06.537169 7f5c973a57a0 10 osd.4 194487 _get_pool 22 2014-08-13 18:06:06.537195 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.791(unlocked)] enter Initial 2014-08-13 18:06:06.537206 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.791(unlocked)] enter NotTrimming 2014-08-13 18:06:06.537342 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.537484 7f5c973a57a0 20 read_log 191648'84 (0'0) modify db435f91/benchmark_data_storage0111-ib.pocstor_40745_object5693/head//22 by client.273158.0:5694 2014-07-30 14:09:14.216971 2014-08-13 18:06:06.537505 7f5c973a57a0 20 read_log 191648'115 (0'0) modify c5b0af91/benchmark_data_storage0111-ib.pocstor_40745_object6884/head//22 by client.273158.0:6885 2014-07-30 14:09:20.637862 2014-08-13 18:06:06.537518 7f5c973a57a0 20 read_log 191648'130 (0'0) modify c1783f91/benchmark_data_storage0111-ib.pocstor_40745_object7518/head//22 by client.273158.0:7519 2014-07-30 14:09:23.517134 2014-08-13 18:06:06.537531 7f5c973a57a0 20 read_log 191648'362 (0'0) modify d32a2f91/benchmark_data_storage0111-ib.pocstor_40745_object22333/head//22 by client.273158.0:22334 2014-07-30 14:10:46.067130 2014-08-13 18:06:06.537543 7f5c973a57a0 20 read_log 191727'373 (0'0) modify badb7f91/benchmark_data_storage0111-ib.pocstor_51285_object1837/head//22 by client.275175.0:1838 2014-07-30 14:18:40.367113 2014-08-13 18:06:06.537555 7f5c973a57a0 20 read_log 191727'375 (0'0) modify 110af791/benchmark_data_storage0111-ib.pocstor_51285_object3444/head//22 by client.275175.0:3445 2014-07-30 14:18:45.623214 2014-08-13 18:06:06.537567 7f5c973a57a0 20 read_log 191727'376 (0'0) modify 976a7791/benchmark_data_storage0111-ib.pocstor_51285_object4069/head//22 by client.275175.0:4070 2014-07-30 14:18:47.568962 2014-08-13 18:06:06.537579 7f5c973a57a0 20 read_log 191727'377 (0'0) modify 9a6bcf91/benchmark_data_storage0111-ib.pocstor_51285_object4125/head//22 by client.275175.0:4126 2014-07-30 14:18:47.741611 2014-08-13 18:06:06.537602 7f5c973a57a0 20 read_log 191727'378 (0'0) modify 45125791/benchmark_data_storage0111-ib.pocstor_51285_object4653/head//22 by client.275175.0:4654 2014-07-30 14:18:49.316676 2014-08-13 18:06:06.537618 7f5c973a57a0 20 read_log 191727'381 (0'0) modify f5483791/benchmark_data_storage0111-ib.pocstor_51285_object5709/head//22 by client.275175.0:5710 2014-07-30 14:18:52.414873 2014-08-13 18:06:06.537630 7f5c973a57a0 20 read_log 191727'384 (0'0) modify ab7df91/benchmark_data_storage0111-ib.pocstor_51285_object7282/head//22 by client.275175.0:7283 2014-07-30 14:18:57.518568 2014-08-13 18:06:06.537642 7f5c973a57a0 20 read_log 191727'386 (0'0) modify 4fc9f91/benchmark_data_storage0111-ib.pocstor_51285_object8150/head//22 by client.275175.0:8151 2014-07-30 14:18:59.942267 2014-08-13 18:06:06.537654 7f5c973a57a0 20 read_log 191727'396 (0'0) modify 1500e791/benchmark_data_storage0111-ib.pocstor_51285_object13316/head//22 by client.275175.0:13317 2014-07-30 14:19:15.586725 2014-08-13 18:06:06.537667 7f5c973a57a0 20 read_log 191727'398 (0'0) modify db9d3f91/benchmark_data_storage0111-ib.pocstor_51285_object14622/head//22 by client.275175.0:14623 2014-07-30 14:19:19.637423 2014-08-13 18:06:06.537680 7f5c973a57a0 20 read_log 191727'400 (0'0) modify b5261f91/benchmark_data_storage0111-ib.pocstor_51285_object16103/head//22 by client.275175.0:16104 2014-07-30 14:19:24.259928 2014-08-13 18:06:06.537692 7f5c973a57a0 20 read_log 191727'403 (0'0) modify 9d524791/benchmark_data_storage0111-ib.pocstor_51285_object16930/head//22 by client.275175.0:16931 2014-07-30 14:19:26.962855 2014-08-13 18:06:06.537704 7f5c973a57a0 20 read_log 191727'410 (0'0) modify bcf81791/benchmark_data_storage0111-ib.pocstor_51285_object19006/head//22 by client.275175.0:19007 2014-07-30 14:19:33.226863 2014-08-13 18:06:06.537716 7f5c973a57a0 20 read_log 191727'416 (0'0) modify 740d3f91/benchmark_data_storage0111-ib.pocstor_51285_object22673/head//22 by client.275175.0:22674 2014-07-30 14:19:44.993875 2014-08-13 18:06:06.537728 7f5c973a57a0 20 read_log 191727'421 (0'0) modify daaae791/benchmark_data_storage0111-ib.pocstor_51285_object25399/head//22 by client.275175.0:25400 2014-07-30 14:19:52.980564 2014-08-13 18:06:06.537740 7f5c973a57a0 20 read_log 191727'425 (0'0) modify ed5c0791/benchmark_data_storage0111-ib.pocstor_51285_object26739/head//22 by client.275175.0:26740 2014-07-30 14:19:57.027759 2014-08-13 18:06:06.537752 7f5c973a57a0 20 read_log 191727'430 (0'0) modify 8ae1cf91/benchmark_data_storage0111-ib.pocstor_51285_object29190/head//22 by client.275175.0:29191 2014-07-30 14:20:04.513232 2014-08-13 18:06:06.537764 7f5c973a57a0 20 read_log 191727'439 (0'0) modify dce47f91/benchmark_data_storage0111-ib.pocstor_51285_object34594/head//22 by client.275175.0:34595 2014-07-30 14:20:20.816333 2014-08-13 18:06:06.537776 7f5c973a57a0 20 read_log 191727'440 (0'0) modify 285adf91/benchmark_data_storage0111-ib.pocstor_51285_object35212/head//22 by client.275175.0:35213 2014-07-30 14:20:22.587266 2014-08-13 18:06:06.537788 7f5c973a57a0 20 read_log 191727'443 (0'0) modify 1b9ad791/benchmark_data_storage0111-ib.pocstor_51285_object35779/head//22 by client.275175.0:35780 2014-07-30 14:20:24.581943 2014-08-13 18:06:06.537800 7f5c973a57a0 20 read_log 191727'448 (0'0) modify 829a3f91/benchmark_data_storage0111-ib.pocstor_51285_object37867/head//22 by client.275175.0:37868 2014-07-30 14:20:30.868306 2014-08-13 18:06:06.537812 7f5c973a57a0 20 read_log 192674'449 (0'0) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1414 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537825 7f5c973a57a0 20 read_log 192674'450 (192674'449) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1415 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537837 7f5c973a57a0 20 read_log 192674'451 (192674'450) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1416 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537874 7f5c973a57a0 20 read_log 192674'452 (192674'451) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1417 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537890 7f5c973a57a0 20 read_log 192674'453 (192674'452) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1418 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537902 7f5c973a57a0 20 read_log 192674'454 (192674'453) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1419 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537920 7f5c973a57a0 20 read_log 192674'455 (192674'454) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1420 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537933 7f5c973a57a0 20 read_log 192674'456 (192674'455) modify c785b791/rb.0.43b76.238e1f29.000000000053/head//22 by client.279772.1:1421 2014-07-31 18:03:43.156000 2014-08-13 18:06:06.537945 7f5c973a57a0 20 read_log 192888'457 (0'0) modify 139ccf91/benchmark_data_storage0111-ib.pocstor_35625_object1200/head//22 by client.940330.0:1201 2014-08-11 16:29:45.538384 2014-08-13 18:06:06.537960 7f5c973a57a0 20 read_log 192888'458 (0'0) modify 70b79f91/benchmark_data_storage0111-ib.pocstor_35625_object7819/head//22 by client.940330.0:7820 2014-08-11 16:30:10.200604 2014-08-13 18:06:06.537972 7f5c973a57a0 20 read_log 192888'459 (0'0) modify a1c82f91/benchmark_data_storage0111-ib.pocstor_35625_object10305/head//22 by client.940330.0:10306 2014-08-11 16:30:18.974795 2014-08-13 18:06:06.537987 7f5c973a57a0 20 read_log 192888'460 (0'0) modify 82e97791/benchmark_data_storage0111-ib.pocstor_35625_object11477/head//22 by client.940330.0:11478 2014-08-11 16:30:23.704692 2014-08-13 18:06:06.537999 7f5c973a57a0 20 read_log 192888'461 (0'0) modify a06c6791/benchmark_data_storage0111-ib.pocstor_35625_object12714/head//22 by client.940330.0:12715 2014-08-11 16:30:28.691008 2014-08-13 18:06:06.538013 7f5c973a57a0 20 read_log 192888'462 (0'0) modify a2be6791/benchmark_data_storage0111-ib.pocstor_35625_object13953/head//22 by client.940330.0:13954 2014-08-11 16:30:33.466283 2014-08-13 18:06:06.538026 7f5c973a57a0 20 read_log 192888'463 (0'0) modify 99adf791/benchmark_data_storage0111-ib.pocstor_35625_object16306/head//22 by client.940330.0:16307 2014-08-11 16:30:42.373749 2014-08-13 18:06:06.538038 7f5c973a57a0 20 read_log 192888'464 (0'0) modify b0241791/benchmark_data_storage0111-ib.pocstor_35625_object19501/head//22 by client.940330.0:19502 2014-08-11 16:30:54.724858 2014-08-13 18:06:06.538050 7f5c973a57a0 20 read_log 192888'465 (0'0) modify efffbf91/benchmark_data_storage0111-ib.pocstor_35625_object20833/head//22 by client.940330.0:20834 2014-08-11 16:31:00.051004 2014-08-13 18:06:06.538062 7f5c973a57a0 20 read_log 192888'466 (0'0) modify 3f46791/benchmark_data_storage0111-ib.pocstor_35625_object21203/head//22 by client.940330.0:21204 2014-08-11 16:31:01.591759 2014-08-13 18:06:06.538076 7f5c973a57a0 20 read_log 192888'467 (0'0) modify 5b76f791/benchmark_data_storage0111-ib.pocstor_35625_object21469/head//22 by client.940330.0:21470 2014-08-11 16:31:02.566494 2014-08-13 18:06:06.538089 7f5c973a57a0 20 read_log 192888'468 (0'0) modify 4850c791/benchmark_data_storage0111-ib.pocstor_35625_object25594/head//22 by client.940330.0:25595 2014-08-11 16:31:18.202556 2014-08-13 18:06:06.538119 7f5c973a57a0 20 read_log 192888'469 (0'0) modify 22f57791/benchmark_data_storage0111-ib.pocstor_35625_object25976/head//22 by client.940330.0:25977 2014-08-11 16:31:19.698737 2014-08-13 18:06:06.538136 7f5c973a57a0 20 read_log 192888'470 (0'0) modify 57df1791/benchmark_data_storage0111-ib.pocstor_35625_object26789/head//22 by client.940330.0:26790 2014-08-11 16:31:22.989421 2014-08-13 18:06:06.538158 7f5c973a57a0 20 read_log 192888'471 (0'0) modify b6f4e791/benchmark_data_storage0111-ib.pocstor_35625_object26941/head//22 by client.940330.0:26942 2014-08-11 16:31:23.576652 2014-08-13 18:06:06.538181 7f5c973a57a0 20 read_log 192888'472 (0'0) modify 431a1f91/benchmark_data_storage0111-ib.pocstor_35625_object28219/head//22 by client.940330.0:28220 2014-08-11 16:31:28.580294 2014-08-13 18:06:06.538196 7f5c973a57a0 20 read_log 192888'473 (0'0) modify 10d69f91/benchmark_data_storage0111-ib.pocstor_35625_object28901/head//22 by client.940330.0:28902 2014-08-11 16:31:31.212324 2014-08-13 18:06:06.538208 7f5c973a57a0 20 read_log 192888'474 (0'0) modify 7f511791/benchmark_data_storage0111-ib.pocstor_35625_object30817/head//22 by client.940330.0:30818 2014-08-11 16:31:38.531910 2014-08-13 18:06:06.538221 7f5c973a57a0 20 read_log 192888'475 (192888'457) delete 139ccf91/benchmark_data_storage0111-ib.pocstor_35625_object1200/head//22 by client.940330.0:32508 2014-08-11 16:31:41.797572 2014-08-13 18:06:06.538236 7f5c973a57a0 20 read_log 192888'476 (192888'458) delete 70b79f91/benchmark_data_storage0111-ib.pocstor_35625_object7819/head//22 by client.940330.0:39127 2014-08-11 16:31:43.544852 2014-08-13 18:06:06.538249 7f5c973a57a0 20 read_log 192888'477 (192888'459) delete a1c82f91/benchmark_data_storage0111-ib.pocstor_35625_object10305/head//22 by client.940330.0:41613 2014-08-11 16:31:44.201391 2014-08-13 18:06:06.538266 7f5c973a57a0 20 read_log 192888'478 (192888'460) delete 82e97791/benchmark_data_storage0111-ib.pocstor_35625_object11477/head//22 by client.940330.0:42785 2014-08-11 16:31:44.535017 2014-08-13 18:06:06.538279 7f5c973a57a0 20 read_log 192888'479 (192888'461) delete a06c6791/benchmark_data_storage0111-ib.pocstor_35625_object12714/head//22 by client.940330.0:44022 2014-08-11 16:31:44.838569 2014-08-13 18:06:06.538291 7f5c973a57a0 20 read_log 192888'480 (192888'462) delete a2be6791/benchmark_data_storage0111-ib.pocstor_35625_object13953/head//22 by client.940330.0:45261 2014-08-11 16:31:45.170661 2014-08-13 18:06:06.538304 7f5c973a57a0 20 read_log 192888'481 (192888'463) delete 99adf791/benchmark_data_storage0111-ib.pocstor_35625_object16306/head//22 by client.940330.0:47614 2014-08-11 16:31:45.805416 2014-08-13 18:06:06.538316 7f5c973a57a0 20 read_log 192888'482 (192888'464) delete b0241791/benchmark_data_storage0111-ib.pocstor_35625_object19501/head//22 by client.940330.0:50809 2014-08-11 16:31:46.721006 2014-08-13 18:06:06.538328 7f5c973a57a0 20 read_log 192888'483 (192888'465) delete efffbf91/benchmark_data_storage0111-ib.pocstor_35625_object20833/head//22 by client.940330.0:52141 2014-08-11 16:31:47.115820 2014-08-13 18:06:06.538340 7f5c973a57a0 20 read_log 192888'484 (192888'466) delete 3f46791/benchmark_data_storage0111-ib.pocstor_35625_object21203/head//22 by client.940330.0:52511 2014-08-11 16:31:47.215150 2014-08-13 18:06:06.538352 7f5c973a57a0 20 read_log 192888'485 (192888'467) delete 5b76f791/benchmark_data_storage0111-ib.pocstor_35625_object21469/head//22 by client.940330.0:52777 2014-08-11 16:31:47.288636 2014-08-13 18:06:06.538364 7f5c973a57a0 20 read_log 192888'486 (192888'468) delete 4850c791/benchmark_data_storage0111-ib.pocstor_35625_object25594/head//22 by client.940330.0:56902 2014-08-11 16:31:48.529552 2014-08-13 18:06:06.538387 7f5c973a57a0 20 read_log 192888'487 (192888'469) delete 22f57791/benchmark_data_storage0111-ib.pocstor_35625_object25976/head//22 by client.940330.0:57284 2014-08-11 16:31:48.640579 2014-08-13 18:06:06.538403 7f5c973a57a0 20 read_log 192888'488 (192888'470) delete 57df1791/benchmark_data_storage0111-ib.pocstor_35625_object26789/head//22 by client.940330.0:58097 2014-08-11 16:31:48.870198 2014-08-13 18:06:06.538416 7f5c973a57a0 20 read_log 192888'489 (192888'471) delete b6f4e791/benchmark_data_storage0111-ib.pocstor_35625_object26941/head//22 by client.940330.0:58249 2014-08-11 16:31:48.912097 2014-08-13 18:06:06.538442 7f5c973a57a0 20 read_log 192888'490 (192888'472) delete 431a1f91/benchmark_data_storage0111-ib.pocstor_35625_object28219/head//22 by client.940330.0:59527 2014-08-11 16:31:49.264801 2014-08-13 18:06:06.538458 7f5c973a57a0 20 read_log 192888'491 (192888'473) delete 10d69f91/benchmark_data_storage0111-ib.pocstor_35625_object28901/head//22 by client.940330.0:60209 2014-08-11 16:31:49.446411 2014-08-13 18:06:06.538490 7f5c973a57a0 20 read_log 192888'492 (192888'474) delete 7f511791/benchmark_data_storage0111-ib.pocstor_35625_object30817/head//22 by client.940330.0:62125 2014-08-11 16:31:49.999423 2014-08-13 18:06:06.538507 7f5c973a57a0 20 read_log 194215'493 (0'0) modify 8a262791/benchmark_data_storage0111-ib.pocstor_64498_object3589/head//22 by client.1049635.0:3590 2014-08-12 17:17:28.771731 2014-08-13 18:06:06.538525 7f5c973a57a0 20 read_log 194215'494 (0'0) modify 9118a791/benchmark_data_storage0111-ib.pocstor_64498_object4103/head//22 by client.1049635.0:4104 2014-08-12 17:17:30.393766 2014-08-13 18:06:06.538540 7f5c973a57a0 20 read_log 194215'495 (0'0) modify 3ea80f91/benchmark_data_storage0111-ib.pocstor_64498_object8825/head//22 by client.1049635.0:8826 2014-08-12 17:17:44.525646 2014-08-13 18:06:06.538552 7f5c973a57a0 20 read_log 194215'496 (0'0) modify 6c896f91/benchmark_data_storage0111-ib.pocstor_64498_object9105/head//22 by client.1049635.0:9106 2014-08-12 17:17:45.323147 2014-08-13 18:06:06.538564 7f5c973a57a0 20 read_log 194215'497 (0'0) modify 2f23bf91/benchmark_data_storage0111-ib.pocstor_64498_object15443/head//22 by client.1049635.0:15444 2014-08-12 17:18:05.456863 2014-08-13 18:06:06.538577 7f5c973a57a0 20 read_log 194215'498 (0'0) modify 85e41f91/benchmark_data_storage0111-ib.pocstor_64498_object17017/head//22 by client.1049635.0:17018 2014-08-12 17:18:11.426990 2014-08-13 18:06:06.538601 7f5c973a57a0 20 read_log 194215'499 (0'0) modify 3c01df91/benchmark_data_storage0111-ib.pocstor_64498_object17633/head//22 by client.1049635.0:17634 2014-08-12 17:18:13.768857 2014-08-13 18:06:06.538614 7f5c973a57a0 20 read_log 194215'500 (0'0) modify fb1c6791/benchmark_data_storage0111-ib.pocstor_64498_object20467/head//22 by client.1049635.0:20468 2014-08-12 17:18:24.545801 2014-08-13 18:06:06.538626 7f5c973a57a0 20 read_log 194215'501 (0'0) modify 8a565791/benchmark_data_storage0111-ib.pocstor_64498_object20850/head//22 by client.1049635.0:20851 2014-08-12 17:18:25.964691 2014-08-13 18:06:06.538638 7f5c973a57a0 20 read_log 194215'502 (0'0) modify 7b045791/benchmark_data_storage0111-ib.pocstor_64498_object26311/head//22 by client.1049635.0:26312 2014-08-12 17:18:46.579387 2014-08-13 18:06:06.538651 7f5c973a57a0 20 read_log 194215'503 (0'0) modify e9d4c791/benchmark_data_storage0111-ib.pocstor_64498_object30275/head//22 by client.1049635.0:30276 2014-08-12 17:19:01.657233 2014-08-13 18:06:06.538663 7f5c973a57a0 20 read_log 194215'504 (0'0) modify a55c0f91/benchmark_data_storage0111-ib.pocstor_64498_object30969/head//22 by client.1049635.0:30970 2014-08-12 17:19:04.446525 2014-08-13 18:06:06.538676 7f5c973a57a0 20 read_log 194215'505 (0'0) modify e1d0ff91/benchmark_data_storage0111-ib.pocstor_64498_object31121/head//22 by client.1049635.0:31122 2014-08-12 17:19:05.030906 2014-08-13 18:06:06.538688 7f5c973a57a0 20 read_log 194215'506 (194215'493) delete 8a262791/benchmark_data_storage0111-ib.pocstor_64498_object3589/head//22 by client.1049635.0:37792 2014-08-12 17:19:19.145625 2014-08-13 18:06:06.538700 7f5c973a57a0 20 read_log 194215'507 (194215'494) delete 9118a791/benchmark_data_storage0111-ib.pocstor_64498_object4103/head//22 by client.1049635.0:38306 2014-08-12 17:19:19.307379 2014-08-13 18:06:06.538712 7f5c973a57a0 20 read_log 194215'508 (194215'495) delete 3ea80f91/benchmark_data_storage0111-ib.pocstor_64498_object8825/head//22 by client.1049635.0:43028 2014-08-12 17:19:20.869633 2014-08-13 18:06:06.538725 7f5c973a57a0 20 read_log 194215'509 (194215'496) delete 6c896f91/benchmark_data_storage0111-ib.pocstor_64498_object9105/head//22 by client.1049635.0:43308 2014-08-12 17:19:20.955008 2014-08-13 18:06:06.538737 7f5c973a57a0 20 read_log 194215'510 (194215'497) delete 2f23bf91/benchmark_data_storage0111-ib.pocstor_64498_object15443/head//22 by client.1049635.0:49646 2014-08-12 17:19:23.015177 2014-08-13 18:06:06.538751 7f5c973a57a0 20 read_log 194215'511 (194215'498) delete 85e41f91/benchmark_data_storage0111-ib.pocstor_64498_object17017/head//22 by client.1049635.0:51220 2014-08-12 17:19:23.513435 2014-08-13 18:06:06.538763 7f5c973a57a0 20 read_log 194215'512 (194215'499) delete 3c01df91/benchmark_data_storage0111-ib.pocstor_64498_object17633/head//22 by client.1049635.0:51836 2014-08-12 17:19:23.706481 2014-08-13 18:06:06.538776 7f5c973a57a0 20 read_log 194215'513 (194215'500) delete fb1c6791/benchmark_data_storage0111-ib.pocstor_64498_object20467/head//22 by client.1049635.0:54670 2014-08-12 17:19:24.692984 2014-08-13 18:06:06.538789 7f5c973a57a0 20 read_log 194215'514 (194215'501) delete 8a565791/benchmark_data_storage0111-ib.pocstor_64498_object20850/head//22 by client.1049635.0:55053 2014-08-12 17:19:24.834124 2014-08-13 18:06:06.538801 7f5c973a57a0 20 read_log 194215'515 (194215'502) delete 7b045791/benchmark_data_storage0111-ib.pocstor_64498_object26311/head//22 by client.1049635.0:60514 2014-08-12 17:19:26.762089 2014-08-13 18:06:06.538814 7f5c973a57a0 20 read_log 194215'516 (194215'503) delete e9d4c791/benchmark_data_storage0111-ib.pocstor_64498_object30275/head//22 by client.1049635.0:64478 2014-08-12 17:19:28.134152 2014-08-13 18:06:06.538826 7f5c973a57a0 20 read_log 194215'517 (194215'504) delete a55c0f91/benchmark_data_storage0111-ib.pocstor_64498_object30969/head//22 by client.1049635.0:65172 2014-08-12 17:19:28.371418 2014-08-13 18:06:06.538838 7f5c973a57a0 20 read_log 194215'518 (194215'505) delete e1d0ff91/benchmark_data_storage0111-ib.pocstor_64498_object31121/head//22 by client.1049635.0:65324 2014-08-12 17:19:28.423632 2014-08-13 18:06:06.538957 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.538980 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.538997 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Initial 0.001802 0 0.000000 2014-08-13 18:06:06.539010 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.539018 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=0 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.539043 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] log((0'0,194215'518], crt=192888'492) 2014-08-13 18:06:06.539053 7f5c973a57a0 10 osd.4 194487 pgid 24.2c coll 24.2c_head 2014-08-13 18:06:06.539292 7f5c973a57a0 20 osd.4 0 get_map 194470 - loading and decoding 0x2e01e00 2014-08-13 18:06:06.539513 7f5c973a57a0 10 osd.4 0 add_map_bl 194470 268258 bytes 2014-08-13 18:06:06.541200 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.2c 2014-08-13 18:06:06.541216 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.541249 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] enter Initial 2014-08-13 18:06:06.541261 7f5c973a57a0 20 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] enter NotTrimming 2014-08-13 18:06:06.541426 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.541561 7f5c973a57a0 20 read_log 192834'13423 (192834'13423) modify 2c/hit_set_24.2c_archive_2014-08-11 10:24:43.594071_2014-08-11 15:23:38.213353/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:38.213751 2014-08-13 18:06:06.541584 7f5c973a57a0 20 read_log 192834'13424 (192718'12963) delete 2c/hit_set_24.2c_archive_2014-08-06 14:29:46.086849_2014-08-06 15:00:27.044493/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:38.213751 2014-08-13 18:06:06.541598 7f5c973a57a0 20 read_log 192834'13437 (0'0) promote bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:14886 2014-08-11 15:23:40.131592 2014-08-13 18:06:06.541613 7f5c973a57a0 20 read_log 192834'13438 (192834'13437) modify bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by client.933448.0:1152 2014-08-11 15:23:40.124130 2014-08-13 18:06:06.541626 7f5c973a57a0 20 read_log 192834'13449 (0'0) promote b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by osd.20.0:14898 2014-08-11 15:23:44.965483 2014-08-13 18:06:06.541639 7f5c973a57a0 20 read_log 192834'13450 (192834'13449) modify b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.933448.0:2102 2014-08-11 15:23:44.958283 2014-08-13 18:06:06.541651 7f5c973a57a0 20 read_log 192834'13459 (0'0) promote b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by osd.20.0:14908 2014-08-11 15:23:49.940768 2014-08-13 18:06:06.541664 7f5c973a57a0 20 read_log 192834'13460 (192834'13459) modify b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.933448.0:3049 2014-08-11 15:23:49.933885 2014-08-13 18:06:06.541676 7f5c973a57a0 20 read_log 192834'13461 (0'0) promote 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by osd.20.0:14910 2014-08-11 15:23:52.013558 2014-08-13 18:06:06.541688 7f5c973a57a0 20 read_log 192834'13462 (192834'13461) modify 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.933448.0:3487 2014-08-11 15:23:52.004988 2014-08-13 18:06:06.541701 7f5c973a57a0 20 read_log 192834'13471 (0'0) promote 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:14920 2014-08-11 15:23:54.439774 2014-08-13 18:06:06.541713 7f5c973a57a0 20 read_log 192834'13472 (192834'13471) modify 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by client.933448.0:3907 2014-08-11 15:23:54.431556 2014-08-13 18:06:06.541725 7f5c973a57a0 20 read_log 192834'13475 (0'0) promote 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by osd.20.0:14924 2014-08-11 15:23:55.915896 2014-08-13 18:06:06.541737 7f5c973a57a0 20 read_log 192834'13476 (192834'13475) modify 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.933448.0:4213 2014-08-11 15:23:55.907164 2014-08-13 18:06:06.541750 7f5c973a57a0 20 read_log 192834'13481 (0'0) promote 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by osd.20.0:14930 2014-08-11 15:23:57.596052 2014-08-13 18:06:06.541762 7f5c973a57a0 20 read_log 192834'13482 (192834'13481) modify 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.933448.0:4559 2014-08-11 15:23:57.587823 2014-08-13 18:06:06.541782 7f5c973a57a0 20 read_log 192834'13495 (0'0) promote c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by osd.20.0:14944 2014-08-11 15:24:04.133269 2014-08-13 18:06:06.541795 7f5c973a57a0 20 read_log 192834'13496 (192834'13495) modify c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.933448.0:6105 2014-08-11 15:24:04.125430 2014-08-13 18:06:06.541808 7f5c973a57a0 20 read_log 192834'13529 (0'0) promote 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:14978 2014-08-11 15:24:12.004496 2014-08-13 18:06:06.541820 7f5c973a57a0 20 read_log 192834'13530 (192834'13529) modify 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by client.933448.0:7946 2014-08-11 15:24:11.996307 2014-08-13 18:06:06.541834 7f5c973a57a0 20 read_log 192834'13531 (0'0) promote c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by osd.20.0:14980 2014-08-11 15:24:12.819542 2014-08-13 18:06:06.541847 7f5c973a57a0 20 read_log 192834'13532 (192834'13531) modify c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.933448.0:8123 2014-08-11 15:24:12.810969 2014-08-13 18:06:06.541860 7f5c973a57a0 20 read_log 192834'13541 (0'0) promote 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by osd.20.0:14990 2014-08-11 15:24:17.843326 2014-08-13 18:06:06.541872 7f5c973a57a0 20 read_log 192834'13542 (192834'13541) modify 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.933448.0:9178 2014-08-11 15:24:17.835345 2014-08-13 18:06:06.541884 7f5c973a57a0 20 read_log 192834'13555 (0'0) promote 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by osd.20.0:15004 2014-08-11 15:24:22.438019 2014-08-13 18:06:06.541896 7f5c973a57a0 20 read_log 192834'13556 (192834'13555) modify 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.933448.0:10293 2014-08-11 15:24:22.428624 2014-08-13 18:06:06.541909 7f5c973a57a0 20 read_log 192834'13557 (0'0) promote 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15006 2014-08-11 15:24:22.536695 2014-08-13 18:06:06.541921 7f5c973a57a0 20 read_log 192834'13558 (192834'13557) modify 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by client.933448.0:10314 2014-08-11 15:24:22.527540 2014-08-13 18:06:06.541933 7f5c973a57a0 20 read_log 192834'13571 (0'0) promote 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by osd.20.0:15020 2014-08-11 15:24:27.431954 2014-08-13 18:06:06.541945 7f5c973a57a0 20 read_log 192834'13572 (192834'13571) modify 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.933448.0:11361 2014-08-11 15:24:27.422655 2014-08-13 18:06:06.541957 7f5c973a57a0 20 read_log 192834'13585 (0'0) promote cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by osd.20.0:15034 2014-08-11 15:24:29.656490 2014-08-13 18:06:06.541969 7f5c973a57a0 20 read_log 192834'13586 (192834'13585) modify cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.933448.0:11858 2014-08-11 15:24:29.649098 2014-08-13 18:06:06.541984 7f5c973a57a0 20 read_log 192834'13593 (0'0) promote 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:15042 2014-08-11 15:24:31.521354 2014-08-13 18:06:06.541997 7f5c973a57a0 20 read_log 192834'13594 (192834'13593) modify 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by client.933448.0:12214 2014-08-11 15:24:31.510472 2014-08-13 18:06:06.542009 7f5c973a57a0 20 read_log 192834'13607 (0'0) promote a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by osd.20.0:15056 2014-08-11 15:24:33.666961 2014-08-13 18:06:06.542022 7f5c973a57a0 20 read_log 192834'13608 (192834'13607) modify a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.933448.0:12737 2014-08-11 15:24:33.657343 2014-08-13 18:06:06.542045 7f5c973a57a0 20 read_log 192834'13625 (0'0) promote d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by osd.20.0:15074 2014-08-11 15:24:40.270540 2014-08-13 18:06:06.542061 7f5c973a57a0 20 read_log 192834'13626 (192834'13625) modify d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.933448.0:13872 2014-08-11 15:24:40.261490 2014-08-13 18:06:06.542073 7f5c973a57a0 20 read_log 192834'13629 (0'0) promote fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15078 2014-08-11 15:24:41.745306 2014-08-13 18:06:06.542088 7f5c973a57a0 20 read_log 192834'13630 (192834'13629) modify fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by client.933448.0:14154 2014-08-11 15:24:41.734143 2014-08-13 18:06:06.542104 7f5c973a57a0 20 read_log 192834'13639 (0'0) promote 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by osd.20.0:15088 2014-08-11 15:24:46.657282 2014-08-13 18:06:06.542119 7f5c973a57a0 20 read_log 192834'13640 (192834'13639) modify 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.933448.0:14669 2014-08-11 15:24:46.648684 2014-08-13 18:06:06.542132 7f5c973a57a0 20 read_log 192834'13651 (0'0) promote 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by osd.20.0:15100 2014-08-11 15:24:50.030814 2014-08-13 18:06:06.542155 7f5c973a57a0 20 read_log 192834'13652 (192834'13651) modify 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.933448.0:15539 2014-08-11 15:24:50.022585 2014-08-13 18:06:06.542171 7f5c973a57a0 20 read_log 192834'13665 (0'0) promote bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by osd.20.0:15114 2014-08-11 15:24:52.730496 2014-08-13 18:06:06.542183 7f5c973a57a0 20 read_log 192834'13666 (192834'13665) modify bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.933448.0:16065 2014-08-11 15:24:52.718086 2014-08-13 18:06:06.542196 7f5c973a57a0 20 read_log 192834'13667 (0'0) promote 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by osd.20.0:15116 2014-08-11 15:24:53.406304 2014-08-13 18:06:06.542209 7f5c973a57a0 20 read_log 192834'13668 (192834'13667) modify 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.933448.0:16174 2014-08-11 15:24:53.396511 2014-08-13 18:06:06.542221 7f5c973a57a0 20 read_log 192834'13669 (0'0) promote 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15118 2014-08-11 15:24:54.173432 2014-08-13 18:06:06.542233 7f5c973a57a0 20 read_log 192834'13670 (192834'13669) modify 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by client.933448.0:16339 2014-08-11 15:24:54.163611 2014-08-13 18:06:06.542248 7f5c973a57a0 20 read_log 192834'13673 (0'0) promote 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:15122 2014-08-11 15:24:54.336992 2014-08-13 18:06:06.542260 7f5c973a57a0 20 read_log 192834'13674 (192834'13673) modify 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by client.933448.0:16363 2014-08-11 15:24:54.327759 2014-08-13 18:06:06.542275 7f5c973a57a0 20 read_log 192834'13689 (0'0) promote 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by osd.20.0:15138 2014-08-11 15:24:58.665082 2014-08-13 18:06:06.542287 7f5c973a57a0 20 read_log 192834'13690 (192834'13689) modify 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.933448.0:17159 2014-08-11 15:24:58.657118 2014-08-13 18:06:06.542304 7f5c973a57a0 20 read_log 192834'13697 (0'0) promote e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15146 2014-08-11 15:24:59.517027 2014-08-13 18:06:06.542316 7f5c973a57a0 20 read_log 192834'13698 (192834'13697) modify e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by client.933448.0:17388 2014-08-11 15:24:59.508471 2014-08-13 18:06:06.542358 7f5c973a57a0 20 read_log 192834'13701 (0'0) promote 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by osd.20.0:15150 2014-08-11 15:25:00.989399 2014-08-13 18:06:06.542375 7f5c973a57a0 20 read_log 192834'13702 (192834'13701) modify 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.933448.0:17673 2014-08-11 15:25:00.978745 2014-08-13 18:06:06.542387 7f5c973a57a0 20 read_log 192834'13719 (0'0) promote 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by osd.20.0:15168 2014-08-11 15:25:11.461623 2014-08-13 18:06:06.542404 7f5c973a57a0 20 read_log 192834'13720 (192834'13719) modify 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.933448.0:19320 2014-08-11 15:25:11.452356 2014-08-13 18:06:06.542417 7f5c973a57a0 20 read_log 192834'13743 (0'0) promote 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by osd.20.0:15192 2014-08-11 15:25:35.166266 2014-08-13 18:06:06.542430 7f5c973a57a0 20 read_log 192834'13744 (192834'13743) modify 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.933448.0:22235 2014-08-11 15:25:35.156244 2014-08-13 18:06:06.542445 7f5c973a57a0 20 read_log 192834'13753 (192834'13438) modify bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by client.933448.0:23443 2014-08-11 15:25:39.356068 2014-08-13 18:06:06.542458 7f5c973a57a0 20 read_log 192834'13759 (192834'13450) modify b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.933448.0:24393 2014-08-11 15:25:40.750615 2014-08-13 18:06:06.542487 7f5c973a57a0 20 read_log 192834'13764 (192834'13460) modify b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.933448.0:25340 2014-08-11 15:25:42.015264 2014-08-13 18:06:06.542501 7f5c973a57a0 20 read_log 192834'13765 (192834'13462) modify 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.933448.0:25778 2014-08-11 15:25:42.434633 2014-08-13 18:06:06.542524 7f5c973a57a0 20 read_log 192834'13770 (192834'13472) modify 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by client.933448.0:26198 2014-08-11 15:25:43.103178 2014-08-13 18:06:06.542540 7f5c973a57a0 20 read_log 192834'13772 (192834'13476) modify 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.933448.0:26504 2014-08-11 15:25:43.468688 2014-08-13 18:06:06.542553 7f5c973a57a0 20 read_log 192834'13775 (192834'13482) modify 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.933448.0:26850 2014-08-11 15:25:43.815289 2014-08-13 18:06:06.542576 7f5c973a57a0 20 read_log 192834'13782 (192834'13496) modify c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.933448.0:28396 2014-08-11 15:25:46.131223 2014-08-13 18:06:06.542589 7f5c973a57a0 20 read_log 192834'13799 (192834'13530) modify 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by client.933448.0:30237 2014-08-11 15:25:48.623926 2014-08-13 18:06:06.542601 7f5c973a57a0 20 read_log 192834'13800 (192834'13532) modify c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.933448.0:30414 2014-08-11 15:25:48.821373 2014-08-13 18:06:06.542617 7f5c973a57a0 20 read_log 192834'13805 (192834'13542) modify 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.933448.0:31469 2014-08-11 15:25:50.098570 2014-08-13 18:06:06.542635 7f5c973a57a0 20 read_log 192834'13812 (192834'13556) modify 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.933448.0:32584 2014-08-11 15:25:52.046910 2014-08-13 18:06:06.542647 7f5c973a57a0 20 read_log 192834'13813 (192834'13558) modify 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by client.933448.0:32605 2014-08-11 15:25:52.074978 2014-08-13 18:06:06.542662 7f5c973a57a0 20 read_log 192834'13820 (192834'13572) modify 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.933448.0:33652 2014-08-11 15:25:53.915063 2014-08-13 18:06:06.542682 7f5c973a57a0 20 read_log 192834'13827 (192834'13586) modify cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.933448.0:34149 2014-08-11 15:25:54.641298 2014-08-13 18:06:06.542698 7f5c973a57a0 20 read_log 192834'13831 (192834'13594) modify 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by client.933448.0:34505 2014-08-11 15:25:55.091364 2014-08-13 18:06:06.542718 7f5c973a57a0 20 read_log 192834'13838 (192834'13608) modify a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.933448.0:35028 2014-08-11 15:25:55.720319 2014-08-13 18:06:06.542734 7f5c973a57a0 20 read_log 192834'13847 (192834'13626) modify d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.933448.0:36163 2014-08-11 15:25:57.313549 2014-08-13 18:06:06.542747 7f5c973a57a0 20 read_log 192834'13849 (192834'13630) modify fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by client.933448.0:36445 2014-08-11 15:25:57.735095 2014-08-13 18:06:06.542759 7f5c973a57a0 20 read_log 192834'13854 (192834'13640) modify 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.933448.0:36960 2014-08-11 15:25:58.575684 2014-08-13 18:06:06.542772 7f5c973a57a0 20 read_log 192834'13860 (192834'13652) modify 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.933448.0:37830 2014-08-11 15:26:00.125055 2014-08-13 18:06:06.542785 7f5c973a57a0 20 read_log 192834'13867 (192834'13666) modify bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.933448.0:38356 2014-08-11 15:26:00.808258 2014-08-13 18:06:06.542797 7f5c973a57a0 20 read_log 192834'13868 (192834'13668) modify 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.933448.0:38465 2014-08-11 15:26:00.895984 2014-08-13 18:06:06.542810 7f5c973a57a0 20 read_log 192834'13869 (192834'13670) modify 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by client.933448.0:38630 2014-08-11 15:26:01.075093 2014-08-13 18:06:06.542839 7f5c973a57a0 20 read_log 192834'13871 (192834'13674) modify 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by client.933448.0:38654 2014-08-11 15:26:01.107710 2014-08-13 18:06:06.542856 7f5c973a57a0 20 read_log 192834'13879 (192834'13690) modify 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.933448.0:39450 2014-08-11 15:26:01.886662 2014-08-13 18:06:06.542869 7f5c973a57a0 20 read_log 192834'13883 (192834'13698) modify e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by client.933448.0:39679 2014-08-11 15:26:02.026322 2014-08-13 18:06:06.542881 7f5c973a57a0 20 read_log 192834'13885 (192834'13702) modify 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.933448.0:39964 2014-08-11 15:26:02.189860 2014-08-13 18:06:06.542893 7f5c973a57a0 20 read_log 192834'13894 (192834'13720) modify 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.933448.0:41611 2014-08-11 15:26:02.997683 2014-08-13 18:06:06.542905 7f5c973a57a0 20 read_log 192834'13906 (192834'13744) modify 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.933448.0:44526 2014-08-11 15:26:03.975678 2014-08-13 18:06:06.542918 7f5c973a57a0 20 read_log 192834'13912 (0'0) promote f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15361 2014-08-11 15:26:15.957866 2014-08-13 18:06:06.542930 7f5c973a57a0 20 read_log 192834'13913 (192834'13912) modify f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by client.1883634.0:399 2014-08-11 15:26:15.950165 2014-08-13 18:06:06.542943 7f5c973a57a0 20 read_log 192834'13924 (0'0) promote 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by osd.20.0:15373 2014-08-11 15:26:17.336723 2014-08-13 18:06:06.542955 7f5c973a57a0 20 read_log 192834'13925 (192834'13924) modify 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.1883634.0:780 2014-08-11 15:26:17.325386 2014-08-13 18:06:06.542978 7f5c973a57a0 20 read_log 192834'13926 (0'0) promote 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15375 2014-08-11 15:26:17.587906 2014-08-13 18:06:06.542993 7f5c973a57a0 20 read_log 192834'13927 (192834'13926) modify 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by client.1883634.0:850 2014-08-11 15:26:17.577891 2014-08-13 18:06:06.543006 7f5c973a57a0 20 read_log 192834'13976 (0'0) promote 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by osd.20.0:15425 2014-08-11 15:26:30.182370 2014-08-13 18:06:06.543018 7f5c973a57a0 20 read_log 192834'13977 (192834'13976) modify 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.1883634.0:4335 2014-08-11 15:26:30.174160 2014-08-13 18:06:06.543033 7f5c973a57a0 20 read_log 192834'13978 (0'0) promote c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15427 2014-08-11 15:26:30.197686 2014-08-13 18:06:06.543045 7f5c973a57a0 20 read_log 192834'13979 (192834'13978) modify c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by client.1883634.0:4340 2014-08-11 15:26:30.190365 2014-08-13 18:06:06.543060 7f5c973a57a0 20 read_log 192834'14078 (0'0) promote 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15527 2014-08-11 15:26:53.540340 2014-08-13 18:06:06.543073 7f5c973a57a0 20 read_log 192834'14079 (192834'14078) modify 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by client.1883634.0:10057 2014-08-11 15:26:53.531842 2014-08-13 18:06:06.543086 7f5c973a57a0 20 read_log 192834'14080 (0'0) promote d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by osd.20.0:15529 2014-08-11 15:26:53.875358 2014-08-13 18:06:06.543100 7f5c973a57a0 20 read_log 192834'14081 (192834'14080) modify d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.1883634.0:10152 2014-08-11 15:26:53.868108 2014-08-13 18:06:06.543112 7f5c973a57a0 20 read_log 192834'14114 (0'0) promote 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by osd.20.0:15563 2014-08-11 15:27:01.006122 2014-08-13 18:06:06.543127 7f5c973a57a0 20 read_log 192834'14115 (192834'14114) modify 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.1883634.0:11696 2014-08-11 15:27:00.996432 2014-08-13 18:06:06.543139 7f5c973a57a0 20 read_log 192834'14148 (0'0) promote 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by osd.20.0:15597 2014-08-11 15:27:11.751591 2014-08-13 18:06:06.543158 7f5c973a57a0 20 read_log 192834'14149 (192834'14148) modify 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.1883634.0:14052 2014-08-11 15:27:11.742603 2014-08-13 18:06:06.543174 7f5c973a57a0 20 read_log 192834'14164 (0'0) promote 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by osd.20.0:15613 2014-08-11 15:27:15.740326 2014-08-13 18:06:06.543186 7f5c973a57a0 20 read_log 192834'14165 (192834'14164) modify 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.1883634.0:14862 2014-08-11 15:27:15.731500 2014-08-13 18:06:06.543199 7f5c973a57a0 20 read_log 192834'14172 (0'0) promote 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by osd.20.0:15621 2014-08-11 15:27:17.651940 2014-08-13 18:06:06.543211 7f5c973a57a0 20 read_log 192834'14173 (192834'14172) modify 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.1883634.0:15203 2014-08-11 15:27:17.642707 2014-08-13 18:06:06.543223 7f5c973a57a0 20 read_log 192834'14194 (0'0) promote 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by osd.20.0:15643 2014-08-11 15:27:27.821265 2014-08-13 18:06:06.543235 7f5c973a57a0 20 read_log 192834'14195 (192834'14194) modify 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.1883634.0:16494 2014-08-11 15:27:27.811592 2014-08-13 18:06:06.543255 7f5c973a57a0 20 read_log 192834'14196 (0'0) promote 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:15645 2014-08-11 15:27:28.115025 2014-08-13 18:06:06.543270 7f5c973a57a0 20 read_log 192834'14197 (192834'14196) modify 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by client.1883634.0:16530 2014-08-11 15:27:28.104925 2014-08-13 18:06:06.543283 7f5c973a57a0 20 read_log 192834'14222 (0'0) promote 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:15671 2014-08-11 15:27:49.212801 2014-08-13 18:06:06.543295 7f5c973a57a0 20 read_log 192834'14223 (192834'14222) modify 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by client.1883634.0:19067 2014-08-11 15:27:49.203240 2014-08-13 18:06:06.543310 7f5c973a57a0 20 read_log 192834'14226 (0'0) promote ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:15675 2014-08-11 15:27:51.244259 2014-08-13 18:06:06.543323 7f5c973a57a0 20 read_log 192834'14227 (192834'14226) modify ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by client.1883634.0:19320 2014-08-11 15:27:51.235439 2014-08-13 18:06:06.543335 7f5c973a57a0 20 read_log 192834'14238 (0'0) promote a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by osd.20.0:15687 2014-08-11 15:27:53.627656 2014-08-13 18:06:06.543353 7f5c973a57a0 20 read_log 192834'14239 (192834'14238) modify a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.1883634.0:19638 2014-08-11 15:27:53.616912 2014-08-13 18:06:06.543368 7f5c973a57a0 20 read_log 192834'14250 (0'0) promote a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:15699 2014-08-11 15:27:56.885319 2014-08-13 18:06:06.543380 7f5c973a57a0 20 read_log 192834'14251 (192834'14250) modify a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by client.1883634.0:20431 2014-08-11 15:27:56.875240 2014-08-13 18:06:06.543393 7f5c973a57a0 20 read_log 192834'14274 (0'0) promote 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by osd.20.0:15723 2014-08-11 15:28:04.979843 2014-08-13 18:06:06.543411 7f5c973a57a0 20 read_log 192834'14275 (192834'14274) modify 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.1883634.0:21691 2014-08-11 15:28:04.968934 2014-08-13 18:06:06.543424 7f5c973a57a0 20 read_log 192834'14292 (192834'13813) clean 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15741 2014-08-11 15:28:10.703623 2014-08-13 18:06:06.543436 7f5c973a57a0 20 read_log 192834'14293 (192834'13753) clean bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:15742 2014-08-11 15:28:10.707760 2014-08-13 18:06:06.543461 7f5c973a57a0 20 read_log 192834'14294 (192834'13979) clean c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15743 2014-08-11 15:28:10.770799 2014-08-13 18:06:06.543477 7f5c973a57a0 20 read_log 192834'14295 (192834'13849) clean fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15744 2014-08-11 15:28:10.780297 2014-08-13 18:06:06.543489 7f5c973a57a0 20 read_log 192834'14296 (192834'13883) clean e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15745 2014-08-11 15:28:10.801151 2014-08-13 18:06:06.543530 7f5c973a57a0 20 read_log 192834'14297 (192834'13799) clean 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:15746 2014-08-11 15:28:10.805351 2014-08-13 18:06:06.543554 7f5c973a57a0 20 read_log 192834'14298 (192834'13869) clean 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15747 2014-08-11 15:28:10.834660 2014-08-13 18:06:06.543569 7f5c973a57a0 20 read_log 192834'14299 (192834'13913) clean f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15748 2014-08-11 15:28:10.854708 2014-08-13 18:06:06.543590 7f5c973a57a0 20 read_log 192834'14300 (192834'13770) clean 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:15749 2014-08-11 15:28:11.064491 2014-08-13 18:06:06.543614 7f5c973a57a0 20 read_log 192834'14301 (192834'14079) clean 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15750 2014-08-11 15:28:11.064817 2014-08-13 18:06:06.543631 7f5c973a57a0 20 read_log 192834'14303 (192834'13927) clean 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15752 2014-08-11 15:28:11.179010 2014-08-13 18:06:06.543646 7f5c973a57a0 20 read_log 192834'14311 (192834'14292) delete 4d6d102c/benchmark_data_storage0111-ib.pocstor_26659_object10313/head//24 by osd.20.0:15760 2014-08-11 15:28:11.529122 2014-08-13 18:06:06.543659 7f5c973a57a0 20 read_log 192834'14312 (192834'14294) delete c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15761 2014-08-11 15:28:11.529584 2014-08-13 18:06:06.543673 7f5c973a57a0 20 read_log 192834'14313 (192834'14293) delete bac1402c/benchmark_data_storage0111-ib.pocstor_26659_object1151/head//24 by osd.20.0:15762 2014-08-11 15:28:11.533848 2014-08-13 18:06:06.543686 7f5c973a57a0 20 read_log 192834'14314 (192834'14299) delete f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15763 2014-08-11 15:28:11.534422 2014-08-13 18:06:06.543698 7f5c973a57a0 20 read_log 192834'14316 (192834'14295) delete fc85702c/benchmark_data_storage0111-ib.pocstor_26659_object14153/head//24 by osd.20.0:15765 2014-08-11 15:28:11.540842 2014-08-13 18:06:06.543710 7f5c973a57a0 20 read_log 192834'14317 (192834'14296) delete e9bba02c/benchmark_data_storage0111-ib.pocstor_26659_object17387/head//24 by osd.20.0:15766 2014-08-11 15:28:11.541287 2014-08-13 18:06:06.543723 7f5c973a57a0 20 read_log 192834'14318 (192834'14301) delete 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15767 2014-08-11 15:28:11.546450 2014-08-13 18:06:06.543736 7f5c973a57a0 20 read_log 192834'14319 (192834'14297) delete 9a8eb02c/benchmark_data_storage0111-ib.pocstor_26659_object7945/head//24 by osd.20.0:15768 2014-08-11 15:28:11.546881 2014-08-13 18:06:06.543756 7f5c973a57a0 20 read_log 192834'14321 (192834'14298) delete 2952f02c/benchmark_data_storage0111-ib.pocstor_26659_object16338/head//24 by osd.20.0:15770 2014-08-11 15:28:11.551686 2014-08-13 18:06:06.543772 7f5c973a57a0 20 read_log 192834'14322 (192834'14303) delete 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15771 2014-08-11 15:28:11.552063 2014-08-13 18:06:06.543784 7f5c973a57a0 20 read_log 192834'14323 (192834'14300) delete 9cbaf02c/benchmark_data_storage0111-ib.pocstor_26659_object3906/head//24 by osd.20.0:15772 2014-08-11 15:28:11.556394 2014-08-13 18:06:06.543797 7f5c973a57a0 20 read_log 192834'14335 (0'0) promote a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by osd.20.0:15784 2014-08-11 15:28:12.106564 2014-08-13 18:06:06.543809 7f5c973a57a0 20 read_log 192834'14336 (192834'14335) modify a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.1883634.0:23021 2014-08-11 15:28:12.097108 2014-08-13 18:06:06.543821 7f5c973a57a0 20 read_log 192834'14345 (0'0) promote f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:15794 2014-08-11 15:28:17.626691 2014-08-13 18:06:06.543834 7f5c973a57a0 20 read_log 192834'14346 (192834'14345) modify f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by client.1883634.0:23723 2014-08-11 15:28:17.601677 2014-08-13 18:06:06.543846 7f5c973a57a0 20 read_log 192834'14352 (192834'13925) modify 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.1883634.0:24104 2014-08-11 15:28:18.361842 2014-08-13 18:06:06.543858 7f5c973a57a0 20 read_log 192834'14353 (0'0) promote 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:15802 2014-08-11 15:28:18.462067 2014-08-13 18:06:06.543882 7f5c973a57a0 20 read_log 192834'14354 (192834'14353) modify 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by client.1883634.0:24174 2014-08-11 15:28:18.440741 2014-08-13 18:06:06.543895 7f5c973a57a0 20 read_log 192834'14379 (192834'13977) modify 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.1883634.0:27659 2014-08-11 15:28:24.752531 2014-08-13 18:06:06.543907 7f5c973a57a0 20 read_log 192834'14380 (0'0) promote c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:15829 2014-08-11 15:28:24.792537 2014-08-13 18:06:06.543921 7f5c973a57a0 20 read_log 192834'14381 (192834'14380) modify c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by client.1883634.0:27664 2014-08-11 15:28:24.770652 2014-08-13 18:06:06.543933 7f5c973a57a0 20 read_log 192834'14436 (0'0) promote 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:15885 2014-08-11 15:28:33.111085 2014-08-13 18:06:06.543945 7f5c973a57a0 20 read_log 192834'14437 (192834'14436) modify 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by client.1883634.0:33381 2014-08-11 15:28:33.085247 2014-08-13 18:06:06.543959 7f5c973a57a0 20 read_log 192834'14438 (192834'14081) modify d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.1883634.0:33476 2014-08-11 15:28:33.251623 2014-08-13 18:06:06.543972 7f5c973a57a0 20 read_log 192834'14453 (192834'14453) modify 2c/hit_set_24.2c_archive_2014-08-11 15:23:38.213813_2014-08-11 15:28:35.729117/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:35.729424 2014-08-13 18:06:06.543985 7f5c973a57a0 20 read_log 192834'14454 (192834'13423) delete 2c/hit_set_24.2c_archive_2014-08-11 10:24:43.594071_2014-08-11 15:23:38.213353/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:35.729424 2014-08-13 18:06:06.543997 7f5c973a57a0 20 read_log 192834'14457 (192834'14115) modify 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.1883634.0:35020 2014-08-11 15:28:35.856612 2014-08-13 18:06:06.544012 7f5c973a57a0 20 read_log 192834'14474 (192834'14149) modify 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.1883634.0:37376 2014-08-11 15:28:39.248162 2014-08-13 18:06:06.544024 7f5c973a57a0 20 read_log 192834'14482 (192834'14165) modify 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.1883634.0:38186 2014-08-11 15:28:40.335473 2014-08-13 18:06:06.544036 7f5c973a57a0 20 read_log 192834'14486 (192834'14173) modify 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.1883634.0:38527 2014-08-11 15:28:40.770378 2014-08-13 18:06:06.544050 7f5c973a57a0 20 read_log 192834'14497 (192834'14195) modify 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.1883634.0:39818 2014-08-11 15:28:42.609669 2014-08-13 18:06:06.544062 7f5c973a57a0 20 read_log 192834'14498 (192834'14197) modify 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by client.1883634.0:39854 2014-08-11 15:28:42.666827 2014-08-13 18:06:06.544074 7f5c973a57a0 20 read_log 192834'14511 (192834'14223) modify 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by client.1883634.0:42391 2014-08-11 15:28:46.228290 2014-08-13 18:06:06.544086 7f5c973a57a0 20 read_log 192834'14513 (192834'14227) modify ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by client.1883634.0:42644 2014-08-11 15:28:46.309661 2014-08-13 18:06:06.544098 7f5c973a57a0 20 read_log 192834'14519 (192834'14239) modify a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.1883634.0:42962 2014-08-11 15:28:46.415267 2014-08-13 18:06:06.544112 7f5c973a57a0 20 read_log 192834'14525 (192834'14251) modify a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by client.1883634.0:43755 2014-08-11 15:28:46.653316 2014-08-13 18:06:06.544132 7f5c973a57a0 20 read_log 192834'14537 (192834'14275) modify 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.1883634.0:45015 2014-08-11 15:28:47.122151 2014-08-13 18:06:06.544151 7f5c973a57a0 20 read_log 192834'14548 (192834'14336) modify a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.1883634.0:46345 2014-08-11 15:28:47.563588 2014-08-13 18:06:06.544166 7f5c973a57a0 20 read_log 192888'14552 (192888'14552) modify 2c/hit_set_24.2c_archive_2014-08-11 16:02:02.168428_2014-08-11 16:44:57.627389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.627917 2014-08-13 18:06:06.544179 7f5c973a57a0 20 read_log 192888'14553 (192834'14453) delete 2c/hit_set_24.2c_archive_2014-08-11 15:23:38.213813_2014-08-11 15:28:35.729117/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.627917 2014-08-13 18:06:06.544191 7f5c973a57a0 20 read_log 192888'14554 (0'0) promote 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:16314 2014-08-11 16:44:57.632293 2014-08-13 18:06:06.544205 7f5c973a57a0 20 read_log 192888'14555 (192888'14554) modify 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by client.1315259.0:613 2014-08-11 16:44:57.599462 2014-08-13 18:06:06.544220 7f5c973a57a0 20 read_log 192888'14558 (0'0) promote 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:16325 2014-08-11 16:45:02.951673 2014-08-13 18:06:06.544232 7f5c973a57a0 20 read_log 192888'14559 (192888'14558) modify 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by client.1315259.0:2021 2014-08-11 16:45:02.943705 2014-08-13 18:06:06.544247 7f5c973a57a0 20 read_log 192888'14560 (0'0) promote 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:16329 2014-08-11 16:45:04.059429 2014-08-13 18:06:06.544259 7f5c973a57a0 20 read_log 192888'14561 (192888'14560) modify 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by client.1315259.0:2321 2014-08-11 16:45:04.051921 2014-08-13 18:06:06.544273 7f5c973a57a0 20 read_log 192888'14566 (0'0) promote f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:16369 2014-08-11 16:45:17.809009 2014-08-13 18:06:06.544286 7f5c973a57a0 20 read_log 192888'14567 (192888'14566) modify f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by client.1315259.0:6007 2014-08-11 16:45:17.800013 2014-08-13 18:06:06.544301 7f5c973a57a0 20 read_log 192888'14568 (0'0) promote d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by osd.20.0:16373 2014-08-11 16:45:19.521818 2014-08-13 18:06:06.544313 7f5c973a57a0 20 read_log 192888'14569 (192888'14568) modify d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.1315259.0:6481 2014-08-11 16:45:19.513337 2014-08-13 18:06:06.544326 7f5c973a57a0 20 read_log 192888'14574 (0'0) promote 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by osd.20.0:16389 2014-08-11 16:45:23.849899 2014-08-13 18:06:06.544338 7f5c973a57a0 20 read_log 192888'14575 (192888'14574) modify 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.1315259.0:7680 2014-08-11 16:45:23.841994 2014-08-13 18:06:06.544354 7f5c973a57a0 20 read_log 192888'14578 (0'0) promote 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by osd.20.0:16397 2014-08-11 16:45:25.589587 2014-08-13 18:06:06.544367 7f5c973a57a0 20 read_log 192888'14579 (192888'14578) modify 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.1315259.0:8152 2014-08-11 16:45:25.581531 2014-08-13 18:06:06.544379 7f5c973a57a0 20 read_log 192888'14586 (0'0) promote 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by osd.20.0:16435 2014-08-11 16:45:38.627442 2014-08-13 18:06:06.544429 7f5c973a57a0 20 read_log 192888'14587 (192888'14586) modify 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.1315259.0:11508 2014-08-11 16:45:38.619645 2014-08-13 18:06:06.544449 7f5c973a57a0 20 read_log 192888'14588 (0'0) promote 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:16443 2014-08-11 16:45:44.323018 2014-08-13 18:06:06.544464 7f5c973a57a0 20 read_log 192888'14589 (192888'14588) modify 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by client.1315259.0:12875 2014-08-11 16:45:44.314861 2014-08-13 18:06:06.544477 7f5c973a57a0 20 read_log 192888'14594 (0'0) promote 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by osd.20.0:16451 2014-08-11 16:45:47.256906 2014-08-13 18:06:06.544502 7f5c973a57a0 20 read_log 192888'14595 (192888'14594) modify 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.1315259.0:13595 2014-08-11 16:45:47.247068 2014-08-13 18:06:06.544518 7f5c973a57a0 20 read_log 192888'14598 (0'0) promote ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by osd.20.0:16471 2014-08-11 16:45:51.435871 2014-08-13 18:06:06.544531 7f5c973a57a0 20 read_log 192888'14599 (192888'14598) modify ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.1315259.0:14704 2014-08-11 16:45:51.427885 2014-08-13 18:06:06.544551 7f5c973a57a0 20 read_log 192888'14600 (0'0) promote 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by osd.20.0:16473 2014-08-11 16:45:51.877645 2014-08-13 18:06:06.544567 7f5c973a57a0 20 read_log 192888'14601 (192888'14600) modify 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.1315259.0:14823 2014-08-11 16:45:51.870221 2014-08-13 18:06:06.544580 7f5c973a57a0 20 read_log 192888'14604 (0'0) promote 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by osd.20.0:16479 2014-08-11 16:45:54.192535 2014-08-13 18:06:06.544592 7f5c973a57a0 20 read_log 192888'14605 (192888'14604) modify 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.1315259.0:15412 2014-08-11 16:45:54.184254 2014-08-13 18:06:06.544604 7f5c973a57a0 20 read_log 192888'14606 (0'0) promote fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:16487 2014-08-11 16:45:56.770989 2014-08-13 18:06:06.544617 7f5c973a57a0 20 read_log 192888'14607 (192888'14606) modify fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by client.1315259.0:16090 2014-08-11 16:45:56.762083 2014-08-13 18:06:06.544629 7f5c973a57a0 20 read_log 192888'14610 (0'0) promote 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by osd.20.0:16507 2014-08-11 16:46:08.348284 2014-08-13 18:06:06.544641 7f5c973a57a0 20 read_log 192888'14611 (192888'14610) modify 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.1315259.0:19023 2014-08-11 16:46:08.339815 2014-08-13 18:06:06.544653 7f5c973a57a0 20 read_log 192888'14614 (0'0) promote d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by osd.20.0:16521 2014-08-11 16:46:12.588079 2014-08-13 18:06:06.544666 7f5c973a57a0 20 read_log 192888'14615 (192888'14614) modify d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.1315259.0:20127 2014-08-11 16:46:12.578599 2014-08-13 18:06:06.544678 7f5c973a57a0 20 read_log 192888'14620 (0'0) promote ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by osd.20.0:16547 2014-08-11 16:46:21.150570 2014-08-13 18:06:06.544690 7f5c973a57a0 20 read_log 192888'14621 (192888'14620) modify ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.1315259.0:22377 2014-08-11 16:46:21.142139 2014-08-13 18:06:06.544703 7f5c973a57a0 20 read_log 192888'14632 (0'0) promote ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by osd.20.0:16597 2014-08-11 16:46:38.635444 2014-08-13 18:06:06.544732 7f5c973a57a0 20 read_log 192888'14633 (192888'14632) modify ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.1315259.0:26690 2014-08-11 16:46:38.627527 2014-08-13 18:06:06.544749 7f5c973a57a0 20 read_log 192888'14634 (0'0) promote a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:16607 2014-08-11 16:46:39.834220 2014-08-13 18:06:06.544762 7f5c973a57a0 20 read_log 192888'14635 (192888'14634) modify a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by client.1315259.0:26990 2014-08-11 16:46:39.826626 2014-08-13 18:06:06.544774 7f5c973a57a0 20 read_log 192888'14636 (0'0) promote 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by osd.20.0:16609 2014-08-11 16:46:39.853956 2014-08-13 18:06:06.544788 7f5c973a57a0 20 read_log 192888'14637 (192888'14636) modify 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.1315259.0:26994 2014-08-11 16:46:39.847017 2014-08-13 18:06:06.544800 7f5c973a57a0 20 read_log 192888'14640 (0'0) promote a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:16621 2014-08-11 16:46:45.291346 2014-08-13 18:06:06.544812 7f5c973a57a0 20 read_log 192888'14641 (192888'14640) modify a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by client.1315259.0:28317 2014-08-11 16:46:45.283367 2014-08-13 18:06:06.544824 7f5c973a57a0 20 read_log 192888'14642 (0'0) promote 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by osd.20.0:16623 2014-08-11 16:46:45.318065 2014-08-13 18:06:06.544836 7f5c973a57a0 20 read_log 192888'14643 (192888'14642) modify 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.1315259.0:28324 2014-08-11 16:46:45.310496 2014-08-13 18:06:06.544848 7f5c973a57a0 20 read_log 192888'14646 (0'0) promote 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:16629 2014-08-11 16:46:48.709227 2014-08-13 18:06:06.544861 7f5c973a57a0 20 read_log 192888'14647 (192888'14646) modify 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by client.1315259.0:29175 2014-08-11 16:46:48.699853 2014-08-13 18:06:06.544873 7f5c973a57a0 20 read_log 192888'14648 (192888'14555) modify 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by client.1315259.0:31340 2014-08-11 16:46:56.066437 2014-08-13 18:06:06.544905 7f5c973a57a0 20 read_log 192888'14650 (192888'14559) modify 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by client.1315259.0:32748 2014-08-11 16:46:56.459492 2014-08-13 18:06:06.544920 7f5c973a57a0 20 read_log 192888'14651 (192888'14561) modify 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by client.1315259.0:33048 2014-08-11 16:46:56.541018 2014-08-13 18:06:06.544932 7f5c973a57a0 20 read_log 192888'14654 (192888'14567) modify f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by client.1315259.0:36734 2014-08-11 16:46:57.499069 2014-08-13 18:06:06.544951 7f5c973a57a0 20 read_log 192888'14655 (192888'14569) modify d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.1315259.0:37208 2014-08-11 16:46:57.622055 2014-08-13 18:06:06.544966 7f5c973a57a0 20 read_log 192888'14658 (192888'14575) modify 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.1315259.0:38407 2014-08-11 16:46:57.938860 2014-08-13 18:06:06.544978 7f5c973a57a0 20 read_log 192888'14660 (192888'14579) modify 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.1315259.0:38879 2014-08-11 16:46:58.068998 2014-08-13 18:06:06.544993 7f5c973a57a0 20 read_log 192888'14664 (192888'14587) modify 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.1315259.0:42235 2014-08-11 16:46:58.954832 2014-08-13 18:06:06.545005 7f5c973a57a0 20 read_log 192888'14665 (192888'14589) modify 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by client.1315259.0:43602 2014-08-11 16:46:59.336384 2014-08-13 18:06:06.545027 7f5c973a57a0 20 read_log 192888'14668 (192888'14595) modify 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.1315259.0:44322 2014-08-11 16:46:59.545884 2014-08-13 18:06:06.545042 7f5c973a57a0 20 read_log 192888'14670 (192888'14599) modify ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.1315259.0:45431 2014-08-11 16:46:59.868316 2014-08-13 18:06:06.545054 7f5c973a57a0 20 read_log 192888'14671 (192888'14601) modify 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.1315259.0:45550 2014-08-11 16:46:59.902936 2014-08-13 18:06:06.545069 7f5c973a57a0 20 read_log 192888'14673 (192888'14605) modify 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.1315259.0:46139 2014-08-11 16:47:00.070596 2014-08-13 18:06:06.545083 7f5c973a57a0 20 read_log 192888'14674 (192888'14607) modify fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by client.1315259.0:46817 2014-08-11 16:47:00.266746 2014-08-13 18:06:06.545097 7f5c973a57a0 20 read_log 192888'14676 (192888'14611) modify 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.1315259.0:49750 2014-08-11 16:47:01.162706 2014-08-13 18:06:06.545109 7f5c973a57a0 20 read_log 192888'14678 (192888'14615) modify d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.1315259.0:50854 2014-08-11 16:47:01.497844 2014-08-13 18:06:06.545124 7f5c973a57a0 20 read_log 192888'14681 (192888'14621) modify ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.1315259.0:53104 2014-08-11 16:47:02.156349 2014-08-13 18:06:06.545136 7f5c973a57a0 20 read_log 192888'14687 (192888'14633) modify ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.1315259.0:57417 2014-08-11 16:47:03.505942 2014-08-13 18:06:06.545156 7f5c973a57a0 20 read_log 192888'14688 (192888'14635) modify a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by client.1315259.0:57717 2014-08-11 16:47:03.592040 2014-08-13 18:06:06.545172 7f5c973a57a0 20 read_log 192888'14689 (192888'14637) modify 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.1315259.0:57721 2014-08-11 16:47:03.592927 2014-08-13 18:06:06.545185 7f5c973a57a0 20 read_log 192888'14691 (192888'14641) modify a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by client.1315259.0:59044 2014-08-11 16:47:03.981590 2014-08-13 18:06:06.545197 7f5c973a57a0 20 read_log 192888'14692 (192888'14643) modify 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.1315259.0:59051 2014-08-11 16:47:03.983701 2014-08-13 18:06:06.545230 7f5c973a57a0 20 read_log 192888'14694 (192888'14647) modify 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by client.1315259.0:59902 2014-08-11 16:47:04.278553 2014-08-13 18:06:06.545243 7f5c973a57a0 20 read_log 192888'14695 (192888'14695) modify 2c/hit_set_24.2c_archive_2014-08-11 16:44:57.627923_2014-08-11 17:05:15.447969/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.448742 2014-08-13 18:06:06.545256 7f5c973a57a0 20 read_log 192888'14696 (192888'14552) delete 2c/hit_set_24.2c_archive_2014-08-11 16:02:02.168428_2014-08-11 16:44:57.627389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.448742 2014-08-13 18:06:06.545272 7f5c973a57a0 20 read_log 192888'14701 (0'0) promote b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:16828 2014-08-11 17:05:17.495891 2014-08-13 18:06:06.545286 7f5c973a57a0 20 read_log 192888'14702 (192888'14701) modify b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by client.942868.0:607 2014-08-11 17:05:17.489278 2014-08-13 18:06:06.545299 7f5c973a57a0 20 read_log 192888'14703 (0'0) promote 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by osd.20.0:16833 2014-08-11 17:05:19.458162 2014-08-13 18:06:06.545318 7f5c973a57a0 20 read_log 192888'14704 (192888'14703) modify 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.942868.0:1087 2014-08-11 17:05:19.448933 2014-08-13 18:06:06.545336 7f5c973a57a0 20 read_log 192888'14707 (0'0) promote 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:16843 2014-08-11 17:05:21.300430 2014-08-13 18:06:06.545349 7f5c973a57a0 20 read_log 192888'14708 (192888'14707) modify 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by client.942868.0:1535 2014-08-11 17:05:21.289657 2014-08-13 18:06:06.545363 7f5c973a57a0 20 read_log 192888'14711 (0'0) promote 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by osd.20.0:16861 2014-08-11 17:05:29.541776 2014-08-13 18:06:06.545375 7f5c973a57a0 20 read_log 192888'14712 (192888'14711) modify 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.942868.0:3676 2014-08-11 17:05:29.533913 2014-08-13 18:06:06.545388 7f5c973a57a0 20 read_log 192888'14721 (0'0) promote badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by osd.20.0:16885 2014-08-11 17:05:36.723562 2014-08-13 18:06:06.545412 7f5c973a57a0 20 read_log 192888'14722 (192888'14721) modify badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.942868.0:5562 2014-08-11 17:05:36.715254 2014-08-13 18:06:06.545426 7f5c973a57a0 20 read_log 192888'14725 (0'0) promote c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:16897 2014-08-11 17:05:40.565429 2014-08-13 18:06:06.545452 7f5c973a57a0 20 read_log 192888'14726 (192888'14725) modify c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by client.942868.0:6623 2014-08-11 17:05:40.557477 2014-08-13 18:06:06.545468 7f5c973a57a0 20 read_log 192888'14729 (0'0) promote c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by osd.20.0:16905 2014-08-11 17:05:45.335005 2014-08-13 18:06:06.545480 7f5c973a57a0 20 read_log 192888'14730 (192888'14729) modify c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.942868.0:7862 2014-08-11 17:05:45.323751 2014-08-13 18:06:06.545504 7f5c973a57a0 20 read_log 192888'14733 (0'0) promote 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by osd.20.0:16909 2014-08-11 17:05:45.516811 2014-08-13 18:06:06.545520 7f5c973a57a0 20 read_log 192888'14734 (192888'14733) modify 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.942868.0:7913 2014-08-11 17:05:45.507752 2014-08-13 18:06:06.545532 7f5c973a57a0 20 read_log 192888'14739 (0'0) promote 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by osd.20.0:16925 2014-08-11 17:05:49.189002 2014-08-13 18:06:06.545553 7f5c973a57a0 20 read_log 192888'14740 (192888'14739) modify 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.942868.0:8908 2014-08-11 17:05:49.181303 2014-08-13 18:06:06.545566 7f5c973a57a0 20 read_log 192888'14747 (0'0) promote f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:16941 2014-08-11 17:05:57.488387 2014-08-13 18:06:06.545578 7f5c973a57a0 20 read_log 192888'14748 (192888'14747) modify f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by client.942868.0:11173 2014-08-11 17:05:57.480126 2014-08-13 18:06:06.545598 7f5c973a57a0 20 read_log 192888'14749 (0'0) promote 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:16951 2014-08-11 17:05:59.044874 2014-08-13 18:06:06.545613 7f5c973a57a0 20 read_log 192888'14750 (192888'14749) modify 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by client.942868.0:11604 2014-08-11 17:05:59.035058 2014-08-13 18:06:06.545626 7f5c973a57a0 20 read_log 192888'14755 (0'0) promote cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by osd.20.0:16967 2014-08-11 17:06:05.305636 2014-08-13 18:06:06.545648 7f5c973a57a0 20 read_log 192888'14756 (192888'14755) modify cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.942868.0:13308 2014-08-11 17:06:05.297336 2014-08-13 18:06:06.545663 7f5c973a57a0 20 read_log 192888'14757 (0'0) promote 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by osd.20.0:16969 2014-08-11 17:06:06.037330 2014-08-13 18:06:06.545675 7f5c973a57a0 20 read_log 192888'14758 (192888'14757) modify 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.942868.0:13467 2014-08-11 17:06:06.029406 2014-08-13 18:06:06.545688 7f5c973a57a0 20 read_log 192888'14761 (0'0) promote 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by osd.20.0:16985 2014-08-11 17:06:08.855077 2014-08-13 18:06:06.545700 7f5c973a57a0 20 read_log 192888'14762 (192888'14761) modify 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.942868.0:14153 2014-08-11 17:06:08.847597 2014-08-13 18:06:06.545714 7f5c973a57a0 20 read_log 192888'14763 (0'0) promote 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by osd.20.0:16989 2014-08-11 17:06:10.156218 2014-08-13 18:06:06.545738 7f5c973a57a0 20 read_log 192888'14764 (192888'14763) modify 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.942868.0:14477 2014-08-11 17:06:10.147480 2014-08-13 18:06:06.545751 7f5c973a57a0 20 read_log 192888'14765 (0'0) promote 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:16993 2014-08-11 17:06:11.791551 2014-08-13 18:06:06.545764 7f5c973a57a0 20 read_log 192888'14766 (192888'14765) modify 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by client.942868.0:14889 2014-08-11 17:06:11.781368 2014-08-13 18:06:06.545776 7f5c973a57a0 20 read_log 192888'14769 (0'0) promote 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by osd.20.0:17003 2014-08-11 17:06:13.618827 2014-08-13 18:06:06.545788 7f5c973a57a0 20 read_log 192888'14770 (192888'14769) modify 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.942868.0:15358 2014-08-11 17:06:13.610355 2014-08-13 18:06:06.545801 7f5c973a57a0 20 read_log 192888'14771 (0'0) promote 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by osd.20.0:17009 2014-08-11 17:06:15.239023 2014-08-13 18:06:06.545813 7f5c973a57a0 20 read_log 192888'14772 (192888'14771) modify 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.942868.0:15775 2014-08-11 17:06:15.231576 2014-08-13 18:06:06.545830 7f5c973a57a0 20 read_log 192888'14773 (0'0) promote 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:17013 2014-08-11 17:06:16.836399 2014-08-13 18:06:06.545843 7f5c973a57a0 20 read_log 192888'14774 (192888'14773) modify 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by client.942868.0:16189 2014-08-11 17:06:16.828252 2014-08-13 18:06:06.545855 7f5c973a57a0 20 read_log 192888'14775 (0'0) promote 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by osd.20.0:17035 2014-08-11 17:06:23.378175 2014-08-13 18:06:06.545868 7f5c973a57a0 20 read_log 192888'14776 (192888'14775) modify 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.942868.0:17879 2014-08-11 17:06:23.370234 2014-08-13 18:06:06.545880 7f5c973a57a0 20 read_log 192888'14787 (0'0) promote 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by osd.20.0:17061 2014-08-11 17:06:27.443121 2014-08-13 18:06:06.545892 7f5c973a57a0 20 read_log 192888'14788 (192888'14787) modify 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.942868.0:18960 2014-08-11 17:06:27.434935 2014-08-13 18:06:06.545904 7f5c973a57a0 20 read_log 192888'14789 (0'0) promote e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:17063 2014-08-11 17:06:28.229584 2014-08-13 18:06:06.545923 7f5c973a57a0 20 read_log 192888'14790 (192888'14789) modify e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by client.942868.0:19167 2014-08-11 17:06:28.220544 2014-08-13 18:06:06.545947 7f5c973a57a0 20 read_log 192888'14791 (0'0) promote ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:17067 2014-08-11 17:06:29.182067 2014-08-13 18:06:06.545960 7f5c973a57a0 20 read_log 192888'14792 (192888'14791) modify ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by client.942868.0:19421 2014-08-11 17:06:29.173724 2014-08-13 18:06:06.545984 7f5c973a57a0 20 read_log 192888'14793 (0'0) promote 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by osd.20.0:17073 2014-08-11 17:06:29.808701 2014-08-13 18:06:06.546000 7f5c973a57a0 20 read_log 192888'14794 (192888'14793) modify 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.942868.0:19589 2014-08-11 17:06:29.800747 2014-08-13 18:06:06.546013 7f5c973a57a0 20 read_log 192888'14795 (0'0) promote 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by osd.20.0:17089 2014-08-11 17:06:35.107506 2014-08-13 18:06:06.546025 7f5c973a57a0 20 read_log 192888'14796 (192888'14795) modify 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.942868.0:20975 2014-08-11 17:06:35.100073 2014-08-13 18:06:06.546045 7f5c973a57a0 20 read_log 192888'14799 (0'0) promote fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by osd.20.0:17095 2014-08-11 17:06:36.740182 2014-08-13 18:06:06.546061 7f5c973a57a0 20 read_log 192888'14800 (192888'14799) modify fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.942868.0:21399 2014-08-11 17:06:36.731758 2014-08-13 18:06:06.546074 7f5c973a57a0 20 read_log 192888'14801 (0'0) promote 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by osd.20.0:17099 2014-08-11 17:06:37.312404 2014-08-13 18:06:06.546086 7f5c973a57a0 20 read_log 192888'14802 (192888'14801) modify 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.942868.0:21554 2014-08-11 17:06:37.304605 2014-08-13 18:06:06.546098 7f5c973a57a0 20 read_log 192888'14805 (0'0) promote 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:17105 2014-08-11 17:06:39.721971 2014-08-13 18:06:06.546110 7f5c973a57a0 20 read_log 192888'14806 (192888'14805) modify 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by client.942868.0:22200 2014-08-11 17:06:39.713726 2014-08-13 18:06:06.546123 7f5c973a57a0 20 read_log 192888'14807 (0'0) promote 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by osd.20.0:17107 2014-08-11 17:06:40.081221 2014-08-13 18:06:06.546135 7f5c973a57a0 20 read_log 192888'14808 (192888'14807) modify 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.942868.0:22284 2014-08-11 17:06:40.072658 2014-08-13 18:06:06.546152 7f5c973a57a0 20 read_log 192888'14813 (0'0) promote f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by osd.20.0:17123 2014-08-11 17:06:46.132967 2014-08-13 18:06:06.546165 7f5c973a57a0 20 read_log 192888'14814 (192888'14813) modify f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.942868.0:23841 2014-08-11 17:06:46.122947 2014-08-13 18:06:06.546199 7f5c973a57a0 20 read_log 192888'14815 (0'0) promote 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by osd.20.0:17129 2014-08-11 17:06:47.845770 2014-08-13 18:06:06.546212 7f5c973a57a0 20 read_log 192888'14816 (192888'14815) modify 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.942868.0:24308 2014-08-11 17:06:47.837685 2014-08-13 18:06:06.546225 7f5c973a57a0 20 read_log 192888'14817 (0'0) promote a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by osd.20.0:17133 2014-08-11 17:06:48.223069 2014-08-13 18:06:06.546245 7f5c973a57a0 20 read_log 192888'14818 (192888'14817) modify a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.942868.0:24408 2014-08-11 17:06:48.215048 2014-08-13 18:06:06.546260 7f5c973a57a0 20 read_log 192888'14819 (0'0) promote 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:17137 2014-08-11 17:06:48.880798 2014-08-13 18:06:06.546272 7f5c973a57a0 20 read_log 192888'14820 (192888'14819) modify 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by client.942868.0:24585 2014-08-11 17:06:48.873097 2014-08-13 18:06:06.546285 7f5c973a57a0 20 read_log 192888'14827 (0'0) promote 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:17153 2014-08-11 17:06:54.669624 2014-08-13 18:06:06.546299 7f5c973a57a0 20 read_log 192888'14828 (192888'14827) modify 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by client.942868.0:26062 2014-08-11 17:06:54.660605 2014-08-13 18:06:06.546311 7f5c973a57a0 20 read_log 192888'14833 (0'0) promote 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by osd.20.0:17171 2014-08-11 17:06:57.906683 2014-08-13 18:06:06.546346 7f5c973a57a0 20 read_log 192888'14834 (192888'14833) modify 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.942868.0:26912 2014-08-11 17:06:57.896047 2014-08-13 18:06:06.546361 7f5c973a57a0 20 read_log 192888'14839 (0'0) promote 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:17189 2014-08-11 17:07:01.843597 2014-08-13 18:06:06.546380 7f5c973a57a0 20 read_log 192888'14840 (192888'14839) modify 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by client.942868.0:27958 2014-08-11 17:07:01.834611 2014-08-13 18:06:06.546393 7f5c973a57a0 20 read_log 192888'14847 (0'0) promote 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:17225 2014-08-11 17:07:12.171013 2014-08-13 18:06:06.546407 7f5c973a57a0 20 read_log 192888'14848 (192888'14847) modify 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by client.942868.0:30586 2014-08-11 17:07:12.162935 2014-08-13 18:06:06.546419 7f5c973a57a0 20 read_log 192888'14853 (192888'14702) modify b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by client.942868.0:31873 2014-08-11 17:07:16.082792 2014-08-13 18:06:06.546443 7f5c973a57a0 20 read_log 192888'14854 (192888'14704) modify 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.942868.0:32353 2014-08-11 17:07:16.230538 2014-08-13 18:06:06.546459 7f5c973a57a0 20 read_log 192888'14856 (192888'14708) modify 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by client.942868.0:32801 2014-08-11 17:07:16.348817 2014-08-13 18:06:06.546471 7f5c973a57a0 20 read_log 192888'14858 (192888'14712) modify 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.942868.0:34942 2014-08-11 17:07:16.939124 2014-08-13 18:06:06.546491 7f5c973a57a0 20 read_log 192888'14863 (192888'14722) modify badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.942868.0:36828 2014-08-11 17:07:17.444676 2014-08-13 18:06:06.546504 7f5c973a57a0 20 read_log 192888'14865 (192888'14726) modify c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by client.942868.0:37889 2014-08-11 17:07:17.735915 2014-08-13 18:06:06.546517 7f5c973a57a0 20 read_log 192888'14867 (192888'14730) modify c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.942868.0:39128 2014-08-11 17:07:18.090783 2014-08-13 18:06:06.546529 7f5c973a57a0 20 read_log 192888'14869 (192888'14734) modify 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.942868.0:39179 2014-08-11 17:07:18.108917 2014-08-13 18:06:06.546543 7f5c973a57a0 20 read_log 192888'14872 (192888'14740) modify 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.942868.0:40174 2014-08-11 17:07:18.383430 2014-08-13 18:06:06.546585 7f5c973a57a0 20 read_log 192888'14876 (192888'14748) modify f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by client.942868.0:42439 2014-08-11 17:07:18.997524 2014-08-13 18:06:06.546601 7f5c973a57a0 20 read_log 192888'14877 (192888'14750) modify 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by client.942868.0:42870 2014-08-11 17:07:19.114749 2014-08-13 18:06:06.546613 7f5c973a57a0 20 read_log 192888'14880 (192888'14756) modify cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.942868.0:44574 2014-08-11 17:07:19.603762 2014-08-13 18:06:06.546626 7f5c973a57a0 20 read_log 192888'14881 (192888'14758) modify 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.942868.0:44733 2014-08-11 17:07:19.649621 2014-08-13 18:06:06.546638 7f5c973a57a0 20 read_log 192888'14883 (192888'14762) modify 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.942868.0:45419 2014-08-11 17:07:19.833115 2014-08-13 18:06:06.546650 7f5c973a57a0 20 read_log 192888'14884 (192888'14764) modify 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.942868.0:45743 2014-08-11 17:07:19.921961 2014-08-13 18:06:06.546662 7f5c973a57a0 20 read_log 192888'14885 (192888'14766) modify 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by client.942868.0:46155 2014-08-11 17:07:20.035826 2014-08-13 18:06:06.546674 7f5c973a57a0 20 read_log 192888'14887 (192888'14770) modify 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.942868.0:46624 2014-08-11 17:07:20.174695 2014-08-13 18:06:06.546687 7f5c973a57a0 20 read_log 192888'14888 (192888'14772) modify 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.942868.0:47041 2014-08-11 17:07:20.291054 2014-08-13 18:06:06.546699 7f5c973a57a0 20 read_log 192888'14889 (192888'14774) modify 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by client.942868.0:47455 2014-08-11 17:07:20.412858 2014-08-13 18:06:06.546711 7f5c973a57a0 20 read_log 192888'14890 (192888'14776) modify 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.942868.0:49145 2014-08-11 17:07:20.922348 2014-08-13 18:06:06.546723 7f5c973a57a0 20 read_log 192888'14896 (192888'14788) modify 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.942868.0:50226 2014-08-11 17:07:21.252820 2014-08-13 18:06:06.546735 7f5c973a57a0 20 read_log 192888'14897 (192888'14790) modify e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by client.942868.0:50433 2014-08-11 17:07:21.317042 2014-08-13 18:06:06.546747 7f5c973a57a0 20 read_log 192888'14898 (192888'14792) modify ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by client.942868.0:50687 2014-08-11 17:07:21.399975 2014-08-13 18:06:06.546759 7f5c973a57a0 20 read_log 192888'14899 (192888'14794) modify 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.942868.0:50855 2014-08-11 17:07:21.453814 2014-08-13 18:06:06.546771 7f5c973a57a0 20 read_log 192888'14900 (192888'14796) modify 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.942868.0:52241 2014-08-11 17:07:21.880614 2014-08-13 18:06:06.546783 7f5c973a57a0 20 read_log 192888'14902 (192888'14800) modify fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.942868.0:52665 2014-08-11 17:07:22.013879 2014-08-13 18:06:06.546796 7f5c973a57a0 20 read_log 192888'14903 (192888'14802) modify 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.942868.0:52820 2014-08-11 17:07:22.061200 2014-08-13 18:06:06.546808 7f5c973a57a0 20 read_log 192888'14905 (192888'14806) modify 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by client.942868.0:53466 2014-08-11 17:07:22.271683 2014-08-13 18:06:06.546820 7f5c973a57a0 20 read_log 192888'14906 (192888'14808) modify 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.942868.0:53550 2014-08-11 17:07:22.301891 2014-08-13 18:06:06.546840 7f5c973a57a0 20 read_log 192888'14909 (192888'14814) modify f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.942868.0:55107 2014-08-11 17:07:22.745832 2014-08-13 18:06:06.546852 7f5c973a57a0 20 read_log 192888'14910 (192888'14816) modify 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.942868.0:55574 2014-08-11 17:07:22.887859 2014-08-13 18:06:06.546867 7f5c973a57a0 20 read_log 192888'14911 (192888'14818) modify a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.942868.0:55674 2014-08-11 17:07:22.918229 2014-08-13 18:06:06.546879 7f5c973a57a0 20 read_log 192888'14912 (192888'14820) modify 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by client.942868.0:55851 2014-08-11 17:07:22.978100 2014-08-13 18:06:06.546893 7f5c973a57a0 20 read_log 192888'14916 (192888'14828) modify 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by client.942868.0:57328 2014-08-11 17:07:23.424175 2014-08-13 18:06:06.546906 7f5c973a57a0 20 read_log 192888'14919 (192888'14834) modify 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.942868.0:58178 2014-08-11 17:07:23.664832 2014-08-13 18:06:06.546920 7f5c973a57a0 20 read_log 192888'14922 (192888'14840) modify 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by client.942868.0:59224 2014-08-11 17:07:23.983203 2014-08-13 18:06:06.546935 7f5c973a57a0 20 read_log 192888'14926 (192888'14848) modify 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by client.942868.0:61852 2014-08-11 17:07:24.772496 2014-08-13 18:06:06.546947 7f5c973a57a0 20 read_log 193305'14928 (193305'14928) modify 2c/hit_set_24.2c_archive_2014-08-11 18:37:04.911332_2014-08-11 18:42:33.028272/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.028549 2014-08-13 18:06:06.546962 7f5c973a57a0 20 read_log 193305'14929 (192888'14695) delete 2c/hit_set_24.2c_archive_2014-08-11 16:44:57.627923_2014-08-11 17:05:15.447969/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.028549 2014-08-13 18:06:06.546975 7f5c973a57a0 20 read_log 193305'14930 (0'0) promote 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by osd.20.0:9 2014-08-11 18:42:33.058887 2014-08-13 18:06:06.546989 7f5c973a57a0 20 read_log 193305'14931 (193305'14930) modify 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.951337.0:55 2014-08-11 18:42:33.023511 2014-08-13 18:06:06.547002 7f5c973a57a0 20 read_log 193305'14932 (0'0) promote fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by osd.20.0:19 2014-08-11 18:42:34.941479 2014-08-13 18:06:06.547014 7f5c973a57a0 20 read_log 193305'14933 (193305'14932) modify fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.951337.0:476 2014-08-11 18:42:34.935078 2014-08-13 18:06:06.547029 7f5c973a57a0 20 read_log 193305'14934 (0'0) promote 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by osd.20.0:21 2014-08-11 18:42:36.166828 2014-08-13 18:06:06.547041 7f5c973a57a0 20 read_log 193305'14935 (193305'14934) modify 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.951337.0:776 2014-08-11 18:42:36.159603 2014-08-13 18:06:06.547056 7f5c973a57a0 20 read_log 193305'14936 (0'0) promote d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by osd.20.0:50 2014-08-11 18:42:41.544733 2014-08-13 18:06:06.547068 7f5c973a57a0 20 read_log 193305'14937 (193305'14936) modify d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.951337.0:2216 2014-08-11 18:42:41.537162 2014-08-13 18:06:06.547082 7f5c973a57a0 20 read_log 193305'14938 (0'0) promote 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by osd.20.0:60 2014-08-11 18:42:44.272800 2014-08-13 18:06:06.547104 7f5c973a57a0 20 read_log 193305'14939 (193305'14938) modify 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.951337.0:2968 2014-08-11 18:42:44.265838 2014-08-13 18:06:06.547119 7f5c973a57a0 20 read_log 193305'14940 (0'0) promote 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:78 2014-08-11 18:42:46.283035 2014-08-13 18:06:06.547131 7f5c973a57a0 20 read_log 193305'14941 (193305'14940) modify 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by client.951337.0:3521 2014-08-11 18:42:46.275848 2014-08-13 18:06:06.547145 7f5c973a57a0 20 read_log 193305'14942 (0'0) promote dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:100 2014-08-11 18:42:51.146267 2014-08-13 18:06:06.547165 7f5c973a57a0 20 read_log 193305'14943 (193305'14942) modify dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by client.951337.0:4842 2014-08-11 18:42:51.139496 2014-08-13 18:06:06.547177 7f5c973a57a0 20 read_log 193305'14944 (0'0) promote 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by osd.20.0:158 2014-08-11 18:43:03.200041 2014-08-13 18:06:06.547190 7f5c973a57a0 20 read_log 193305'14945 (193305'14944) modify 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.951337.0:8144 2014-08-11 18:43:03.192415 2014-08-13 18:06:06.547204 7f5c973a57a0 20 read_log 193305'14946 (0'0) promote 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by osd.20.0:204 2014-08-11 18:43:11.152094 2014-08-13 18:06:06.547224 7f5c973a57a0 20 read_log 193305'14947 (193305'14946) modify 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.951337.0:10238 2014-08-11 18:43:11.144066 2014-08-13 18:06:06.547237 7f5c973a57a0 20 read_log 193305'14948 (0'0) promote 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:220 2014-08-11 18:43:13.834120 2014-08-13 18:06:06.547251 7f5c973a57a0 20 read_log 193305'14949 (193305'14948) modify 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by client.951337.0:10953 2014-08-11 18:43:13.826040 2014-08-13 18:06:06.547266 7f5c973a57a0 20 read_log 193305'14950 (0'0) promote 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:224 2014-08-11 18:43:15.328106 2014-08-13 18:06:06.547278 7f5c973a57a0 20 read_log 193305'14951 (193305'14950) modify 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by client.951337.0:11307 2014-08-11 18:43:15.319833 2014-08-13 18:06:06.547294 7f5c973a57a0 20 read_log 193305'14952 (0'0) promote 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:242 2014-08-11 18:43:19.060587 2014-08-13 18:06:06.547307 7f5c973a57a0 20 read_log 193305'14953 (193305'14952) modify 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by client.951337.0:12320 2014-08-11 18:43:19.053956 2014-08-13 18:06:06.547322 7f5c973a57a0 20 read_log 193305'14954 (0'0) promote 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:266 2014-08-11 18:43:22.725989 2014-08-13 18:06:06.547334 7f5c973a57a0 20 read_log 193305'14955 (193305'14954) modify 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by client.951337.0:13295 2014-08-11 18:43:22.718816 2014-08-13 18:06:06.547348 7f5c973a57a0 20 read_log 193305'14956 (0'0) promote b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:312 2014-08-11 18:43:30.401994 2014-08-13 18:06:06.547361 7f5c973a57a0 20 read_log 193305'14957 (193305'14956) modify b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by client.951337.0:15423 2014-08-11 18:43:30.394410 2014-08-13 18:06:06.547375 7f5c973a57a0 20 read_log 193305'14958 (0'0) promote 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by osd.20.0:318 2014-08-11 18:43:32.381306 2014-08-13 18:06:06.547401 7f5c973a57a0 20 read_log 193305'14959 (193305'14958) modify 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.951337.0:15995 2014-08-11 18:43:32.373416 2014-08-13 18:06:06.547426 7f5c973a57a0 20 read_log 193305'14960 (0'0) promote 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by osd.20.0:352 2014-08-11 18:43:40.697337 2014-08-13 18:06:06.547445 7f5c973a57a0 20 read_log 193305'14961 (193305'14960) modify 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.951337.0:18373 2014-08-11 18:43:40.689363 2014-08-13 18:06:06.547458 7f5c973a57a0 20 read_log 193305'14962 (0'0) promote b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:424 2014-08-11 18:43:51.504104 2014-08-13 18:06:06.547484 7f5c973a57a0 20 read_log 193305'14963 (193305'14962) modify b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by client.951337.0:21363 2014-08-11 18:43:51.496690 2014-08-13 18:06:06.547501 7f5c973a57a0 20 read_log 193305'14964 (0'0) promote bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by osd.20.0:428 2014-08-11 18:43:51.602208 2014-08-13 18:06:06.547514 7f5c973a57a0 20 read_log 193305'14965 (193305'14964) modify bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.951337.0:21392 2014-08-11 18:43:51.594314 2014-08-13 18:06:06.547536 7f5c973a57a0 20 read_log 193305'14966 (0'0) promote 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by osd.20.0:436 2014-08-11 18:43:54.051374 2014-08-13 18:06:06.547550 7f5c973a57a0 20 read_log 193305'14967 (193305'14966) modify 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.951337.0:22069 2014-08-11 18:43:54.042889 2014-08-13 18:06:06.547562 7f5c973a57a0 20 read_log 193305'14968 (0'0) promote e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by osd.20.0:466 2014-08-11 18:43:57.775627 2014-08-13 18:06:06.547581 7f5c973a57a0 20 read_log 193305'14969 (193305'14968) modify e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.951337.0:23096 2014-08-11 18:43:57.767140 2014-08-13 18:06:06.547597 7f5c973a57a0 20 read_log 193305'14970 (0'0) promote 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:506 2014-08-11 18:44:05.582627 2014-08-13 18:06:06.547609 7f5c973a57a0 20 read_log 193305'14971 (193305'14970) modify 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by client.951337.0:25259 2014-08-11 18:44:05.574492 2014-08-13 18:06:06.547631 7f5c973a57a0 20 read_log 193305'14972 (0'0) promote 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by osd.20.0:508 2014-08-11 18:44:05.631590 2014-08-13 18:06:06.547645 7f5c973a57a0 20 read_log 193305'14973 (193305'14972) modify 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.951337.0:25273 2014-08-11 18:44:05.624267 2014-08-13 18:06:06.547657 7f5c973a57a0 20 read_log 193305'14974 (0'0) promote 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:518 2014-08-11 18:44:08.160754 2014-08-13 18:06:06.547679 7f5c973a57a0 20 read_log 193305'14975 (193305'14974) modify 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by client.951337.0:25949 2014-08-11 18:44:08.153172 2014-08-13 18:06:06.547692 7f5c973a57a0 20 read_log 193305'14976 (0'0) promote aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:533 2014-08-11 18:44:09.145927 2014-08-13 18:06:06.547704 7f5c973a57a0 20 read_log 193305'14977 (193305'14976) modify aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by client.951337.0:26204 2014-08-11 18:44:09.137887 2014-08-13 18:06:06.547717 7f5c973a57a0 20 read_log 193305'14978 (0'0) promote a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by osd.20.0:535 2014-08-11 18:44:09.278703 2014-08-13 18:06:06.547768 7f5c973a57a0 20 read_log 193305'14979 (193305'14978) modify a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.951337.0:26241 2014-08-11 18:44:09.271564 2014-08-13 18:06:06.547786 7f5c973a57a0 20 read_log 193305'14980 (0'0) promote b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by osd.20.0:541 2014-08-11 18:44:10.630058 2014-08-13 18:06:06.547799 7f5c973a57a0 20 read_log 193305'14981 (193305'14980) modify b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.951337.0:26620 2014-08-11 18:44:10.621159 2014-08-13 18:06:06.547813 7f5c973a57a0 20 read_log 193305'14982 (0'0) promote 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by osd.20.0:545 2014-08-11 18:44:11.424946 2014-08-13 18:06:06.547834 7f5c973a57a0 20 read_log 193305'14983 (193305'14982) modify 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.951337.0:26844 2014-08-11 18:44:11.416571 2014-08-13 18:06:06.547850 7f5c973a57a0 20 read_log 193305'14984 (0'0) promote 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by osd.20.0:556 2014-08-11 18:44:12.959526 2014-08-13 18:06:06.547863 7f5c973a57a0 20 read_log 193305'14985 (193305'14984) modify 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.951337.0:27242 2014-08-11 18:44:12.950878 2014-08-13 18:06:06.547877 7f5c973a57a0 20 read_log 193305'14986 (0'0) promote 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by osd.20.0:566 2014-08-11 18:44:14.213485 2014-08-13 18:06:06.547889 7f5c973a57a0 20 read_log 193305'14987 (193305'14986) modify 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.951337.0:27594 2014-08-11 18:44:14.206000 2014-08-13 18:06:06.547902 7f5c973a57a0 20 read_log 193305'14988 (0'0) promote 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by osd.20.0:581 2014-08-11 18:44:14.966156 2014-08-13 18:06:06.547914 7f5c973a57a0 20 read_log 193305'14989 (193305'14988) modify 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.951337.0:27805 2014-08-11 18:44:14.954950 2014-08-13 18:06:06.547927 7f5c973a57a0 20 read_log 193305'14990 (0'0) promote 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:604 2014-08-11 18:44:19.044537 2014-08-13 18:06:06.547939 7f5c973a57a0 20 read_log 193305'14991 (193305'14990) modify 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by client.951337.0:28916 2014-08-11 18:44:19.035988 2014-08-13 18:06:06.547952 7f5c973a57a0 20 read_log 193305'14992 (0'0) promote ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:612 2014-08-11 18:44:20.688863 2014-08-13 18:06:06.547964 7f5c973a57a0 20 read_log 193305'14993 (193305'14992) modify ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by client.951337.0:29370 2014-08-11 18:44:20.681012 2014-08-13 18:06:06.547977 7f5c973a57a0 20 read_log 193305'14994 (0'0) promote 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by osd.20.0:616 2014-08-11 18:44:21.177696 2014-08-13 18:06:06.547990 7f5c973a57a0 20 read_log 193305'14995 (193305'14994) modify 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.951337.0:29503 2014-08-11 18:44:21.170083 2014-08-13 18:06:06.548002 7f5c973a57a0 20 read_log 193305'14996 (0'0) promote 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:626 2014-08-11 18:44:22.758610 2014-08-13 18:06:06.548015 7f5c973a57a0 20 read_log 193305'14997 (193305'14996) modify 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by client.951337.0:29935 2014-08-11 18:44:22.751340 2014-08-13 18:06:06.548027 7f5c973a57a0 20 read_log 193305'14998 (0'0) promote b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by osd.20.0:630 2014-08-11 18:44:23.176905 2014-08-13 18:06:06.548046 7f5c973a57a0 20 read_log 193305'14999 (193305'14998) modify b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.951337.0:30051 2014-08-11 18:44:23.170567 2014-08-13 18:06:06.548059 7f5c973a57a0 20 read_log 193305'15000 (0'0) promote 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by osd.20.0:638 2014-08-11 18:44:24.080425 2014-08-13 18:06:06.548071 7f5c973a57a0 20 read_log 193305'15001 (193305'15000) modify 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.951337.0:30267 2014-08-11 18:44:24.073436 2014-08-13 18:06:06.548084 7f5c973a57a0 20 read_log 193305'15002 (0'0) promote 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by osd.20.0:668 2014-08-11 18:44:28.266617 2014-08-13 18:06:06.548096 7f5c973a57a0 20 read_log 193305'15003 (193305'15002) modify 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.951337.0:31410 2014-08-11 18:44:28.257958 2014-08-13 18:06:06.548108 7f5c973a57a0 20 read_log 193305'15004 (0'0) promote a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by osd.20.0:674 2014-08-11 18:44:28.902414 2014-08-13 18:06:06.548120 7f5c973a57a0 20 read_log 193305'15005 (193305'15004) modify a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.951337.0:31601 2014-08-11 18:44:28.896396 2014-08-13 18:06:06.548132 7f5c973a57a0 20 read_log 193305'15006 (0'0) promote e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by osd.20.0:686 2014-08-11 18:44:31.176743 2014-08-13 18:06:06.548144 7f5c973a57a0 20 read_log 193305'15007 (193305'15006) modify e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.951337.0:32251 2014-08-11 18:44:31.168779 2014-08-13 18:06:06.548166 7f5c973a57a0 20 read_log 193305'15008 (0'0) promote c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by osd.20.0:688 2014-08-11 18:44:31.634297 2014-08-13 18:06:06.548181 7f5c973a57a0 20 read_log 193305'15009 (193305'15008) modify c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.951337.0:32369 2014-08-11 18:44:31.626227 2014-08-13 18:06:06.548194 7f5c973a57a0 20 read_log 193305'15010 (193305'14931) modify 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.951337.0:32766 2014-08-11 18:44:33.845773 2014-08-13 18:06:06.548206 7f5c973a57a0 20 read_log 193305'15011 (193305'14933) modify fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.951337.0:33187 2014-08-11 18:44:34.224507 2014-08-13 18:06:06.548219 7f5c973a57a0 20 read_log 193305'15012 (193305'14935) modify 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.951337.0:33487 2014-08-11 18:44:34.450137 2014-08-13 18:06:06.548231 7f5c973a57a0 20 read_log 193305'15013 (193305'14937) modify d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.951337.0:34927 2014-08-11 18:44:35.262336 2014-08-13 18:06:06.548243 7f5c973a57a0 20 read_log 193305'15014 (193305'14939) modify 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.951337.0:35679 2014-08-11 18:44:35.700702 2014-08-13 18:06:06.548259 7f5c973a57a0 20 read_log 193305'15015 (193305'14941) modify 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by client.951337.0:36232 2014-08-11 18:44:36.006679 2014-08-13 18:06:06.548271 7f5c973a57a0 20 read_log 193305'15016 (193305'14943) modify dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by client.951337.0:37553 2014-08-11 18:44:36.967667 2014-08-13 18:06:06.548284 7f5c973a57a0 20 read_log 193305'15017 (193305'14945) modify 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.951337.0:40855 2014-08-11 18:44:38.557334 2014-08-13 18:06:06.548299 7f5c973a57a0 20 read_log 193305'15018 (193305'14947) modify 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.951337.0:42949 2014-08-11 18:44:39.432583 2014-08-13 18:06:06.548322 7f5c973a57a0 20 read_log 193305'15019 (193305'14949) modify 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by client.951337.0:43664 2014-08-11 18:44:39.659310 2014-08-13 18:06:06.548338 7f5c973a57a0 20 read_log 193305'15020 (193305'14951) modify 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by client.951337.0:44018 2014-08-11 18:44:39.770485 2014-08-13 18:06:06.548354 7f5c973a57a0 20 read_log 193305'15021 (193305'14953) modify 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by client.951337.0:45031 2014-08-11 18:44:40.072384 2014-08-13 18:06:06.548369 7f5c973a57a0 20 read_log 193305'15022 (193305'14955) modify 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by client.951337.0:46006 2014-08-11 18:44:40.363967 2014-08-13 18:06:06.548388 7f5c973a57a0 20 read_log 193305'15023 (193305'14957) modify b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by client.951337.0:48134 2014-08-11 18:44:40.962087 2014-08-13 18:06:06.548406 7f5c973a57a0 20 read_log 193305'15024 (193305'14959) modify 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.951337.0:48706 2014-08-11 18:44:41.130597 2014-08-13 18:06:06.548419 7f5c973a57a0 20 read_log 193305'15025 (193305'14961) modify 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.951337.0:51084 2014-08-11 18:44:41.846686 2014-08-13 18:06:06.548442 7f5c973a57a0 20 read_log 193305'15026 (193305'14963) modify b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by client.951337.0:54074 2014-08-11 18:44:42.723905 2014-08-13 18:06:06.548455 7f5c973a57a0 20 read_log 193305'15027 (193305'14965) modify bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.951337.0:54103 2014-08-11 18:44:42.732046 2014-08-13 18:06:06.548467 7f5c973a57a0 20 read_log 193305'15028 (193305'14967) modify 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.951337.0:54780 2014-08-11 18:44:42.936662 2014-08-13 18:06:06.548490 7f5c973a57a0 20 read_log 193305'15029 (193305'14969) modify e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.951337.0:55807 2014-08-11 18:44:43.237489 2014-08-13 18:06:06.548503 7f5c973a57a0 20 read_log 193305'15030 (193305'14971) modify 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by client.951337.0:57970 2014-08-11 18:44:43.908532 2014-08-13 18:06:06.548515 7f5c973a57a0 20 read_log 193305'15031 (193305'14973) modify 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.951337.0:57984 2014-08-11 18:44:43.913513 2014-08-13 18:06:06.548528 7f5c973a57a0 20 read_log 193305'15032 (193305'14975) modify 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by client.951337.0:58660 2014-08-11 18:44:44.105953 2014-08-13 18:06:06.548551 7f5c973a57a0 20 read_log 193305'15033 (193305'14977) modify aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by client.951337.0:58915 2014-08-11 18:44:44.178908 2014-08-13 18:06:06.548564 7f5c973a57a0 20 read_log 193305'15034 (193305'14979) modify a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.951337.0:58952 2014-08-11 18:44:44.193036 2014-08-13 18:06:06.548582 7f5c973a57a0 20 read_log 193305'15035 (193305'14981) modify b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.951337.0:59331 2014-08-11 18:44:44.296093 2014-08-13 18:06:06.548597 7f5c973a57a0 20 read_log 193305'15036 (193305'14983) modify 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.951337.0:59555 2014-08-11 18:44:44.360932 2014-08-13 18:06:06.548610 7f5c973a57a0 20 read_log 193305'15037 (193305'14985) modify 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.951337.0:59953 2014-08-11 18:44:44.475825 2014-08-13 18:06:06.548625 7f5c973a57a0 20 read_log 193305'15038 (193305'14987) modify 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.951337.0:60305 2014-08-11 18:44:44.592501 2014-08-13 18:06:06.548678 7f5c973a57a0 20 read_log 193305'15039 (193305'14989) modify 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.951337.0:60516 2014-08-11 18:44:44.653153 2014-08-13 18:06:06.548696 7f5c973a57a0 20 read_log 193305'15040 (193305'14991) modify 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by client.951337.0:61627 2014-08-11 18:44:44.980858 2014-08-13 18:06:06.548708 7f5c973a57a0 20 read_log 193305'15041 (193305'14993) modify ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by client.951337.0:62081 2014-08-11 18:44:45.120566 2014-08-13 18:06:06.548723 7f5c973a57a0 20 read_log 193305'15042 (193305'14995) modify 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.951337.0:62214 2014-08-11 18:44:45.158887 2014-08-13 18:06:06.548745 7f5c973a57a0 20 read_log 193305'15043 (193305'14997) modify 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by client.951337.0:62646 2014-08-11 18:44:45.287766 2014-08-13 18:06:06.548761 7f5c973a57a0 20 read_log 193305'15044 (193305'14999) modify b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.951337.0:62762 2014-08-11 18:44:45.319749 2014-08-13 18:06:06.548774 7f5c973a57a0 20 read_log 193305'15045 (193305'15001) modify 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.951337.0:62978 2014-08-11 18:44:45.378017 2014-08-13 18:06:06.548787 7f5c973a57a0 20 read_log 193305'15046 (193305'15003) modify 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.951337.0:64121 2014-08-11 18:44:45.709650 2014-08-13 18:06:06.548799 7f5c973a57a0 20 read_log 193305'15047 (193305'15005) modify a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.951337.0:64312 2014-08-11 18:44:45.764384 2014-08-13 18:06:06.548811 7f5c973a57a0 20 read_log 193305'15048 (193305'15007) modify e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.951337.0:64962 2014-08-11 18:44:45.975401 2014-08-13 18:06:06.548824 7f5c973a57a0 20 read_log 193305'15049 (193305'15009) modify c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.951337.0:65080 2014-08-11 18:44:46.010291 2014-08-13 18:06:06.548836 7f5c973a57a0 20 read_log 194209'15050 (192888'14912) clean 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:1204 2014-08-12 14:05:46.720579 2014-08-13 18:06:06.548848 7f5c973a57a0 20 read_log 194209'15051 (193305'15019) clean 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:1205 2014-08-12 14:05:46.721139 2014-08-13 18:06:06.548861 7f5c973a57a0 20 read_log 194209'15052 (193305'15023) clean b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:1206 2014-08-12 14:05:46.721727 2014-08-13 18:06:06.548873 7f5c973a57a0 20 read_log 194209'15053 (192888'14856) clean 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:1207 2014-08-12 14:05:46.722180 2014-08-13 18:06:06.548885 7f5c973a57a0 20 read_log 194209'15054 (193305'15040) clean 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:1208 2014-08-12 14:05:46.793692 2014-08-13 18:06:06.548898 7f5c973a57a0 20 read_log 194209'15055 (193305'15041) clean ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:1209 2014-08-12 14:05:46.800826 2014-08-13 18:06:06.548910 7f5c973a57a0 20 read_log 194209'15056 (192834'14513) clean ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:1210 2014-08-12 14:05:46.801106 2014-08-13 18:06:06.548922 7f5c973a57a0 20 read_log 194209'15057 (192888'14905) clean 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:1211 2014-08-12 14:05:46.805219 2014-08-13 18:06:06.548943 7f5c973a57a0 20 read_log 194209'15058 (192888'14926) clean 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:1212 2014-08-12 14:05:46.808809 2014-08-13 18:06:06.548958 7f5c973a57a0 20 read_log 194209'15059 (192888'14922) clean 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:1213 2014-08-12 14:05:46.812479 2014-08-13 18:06:06.548971 7f5c973a57a0 20 read_log 194209'15060 (192888'14865) clean c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:1214 2014-08-12 14:05:46.816442 2014-08-13 18:06:06.549003 7f5c973a57a0 20 read_log 194209'15061 (193305'15033) clean aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:1215 2014-08-12 14:05:46.826229 2014-08-13 18:06:06.549030 7f5c973a57a0 20 read_log 194209'15062 (192834'14381) clean c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:1216 2014-08-12 14:05:46.826758 2014-08-13 18:06:06.549043 7f5c973a57a0 20 read_log 194209'15063 (193305'15043) clean 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:1217 2014-08-12 14:05:46.827203 2014-08-13 18:06:06.549055 7f5c973a57a0 20 read_log 194209'15064 (192888'14665) clean 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:1218 2014-08-12 14:05:46.827979 2014-08-13 18:06:06.549068 7f5c973a57a0 20 read_log 194209'15065 (192888'14648) clean 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:1219 2014-08-12 14:05:46.838912 2014-08-13 18:06:06.549080 7f5c973a57a0 20 read_log 194209'15066 (192888'14650) clean 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:1220 2014-08-12 14:05:46.839186 2014-08-13 18:06:06.549097 7f5c973a57a0 20 read_log 194209'15067 (192888'14853) clean b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:1221 2014-08-12 14:05:46.842597 2014-08-13 18:06:06.549109 7f5c973a57a0 20 read_log 194209'15068 (192834'14498) clean 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:1222 2014-08-12 14:05:46.846671 2014-08-13 18:06:06.549122 7f5c973a57a0 20 read_log 194209'15069 (192834'14346) clean f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:1223 2014-08-12 14:05:46.853816 2014-08-13 18:06:06.549161 7f5c973a57a0 20 read_log 194209'15070 (192834'14511) clean 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:1224 2014-08-12 14:05:46.854455 2014-08-13 18:06:06.549179 7f5c973a57a0 20 read_log 194209'15071 (192888'14674) clean fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:1225 2014-08-12 14:05:46.858203 2014-08-13 18:06:06.549223 7f5c973a57a0 20 read_log 194209'15072 (192888'14885) clean 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:1226 2014-08-12 14:05:46.861963 2014-08-13 18:06:06.549243 7f5c973a57a0 20 read_log 194209'15073 (192888'14898) clean ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:1227 2014-08-12 14:05:46.865993 2014-08-13 18:06:06.549266 7f5c973a57a0 20 read_log 194209'15074 (192888'14654) clean f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:1228 2014-08-12 14:05:46.869735 2014-08-13 18:06:06.549282 7f5c973a57a0 20 read_log 194209'15075 (193305'15026) clean b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:1229 2014-08-12 14:05:46.873378 2014-08-13 18:06:06.549295 7f5c973a57a0 20 read_log 194209'15076 (192888'14688) clean a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:1230 2014-08-12 14:05:46.876907 2014-08-13 18:06:06.549307 7f5c973a57a0 20 read_log 194209'15077 (192888'14897) clean e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:1231 2014-08-12 14:05:46.880566 2014-08-13 18:06:06.549341 7f5c973a57a0 20 read_log 194209'15078 (192888'14877) clean 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:1232 2014-08-12 14:05:46.884093 2014-08-13 18:06:06.549357 7f5c973a57a0 20 read_log 194209'15079 (192834'14437) clean 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:1233 2014-08-12 14:05:46.891178 2014-08-13 18:06:06.549370 7f5c973a57a0 20 read_log 194209'15080 (193305'15020) clean 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:1234 2014-08-12 14:05:46.891437 2014-08-13 18:06:06.549382 7f5c973a57a0 20 read_log 194209'15081 (193305'15016) clean dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:1235 2014-08-12 14:05:46.895172 2014-08-13 18:06:06.549398 7f5c973a57a0 20 read_log 194209'15082 (193305'15022) clean 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:1236 2014-08-12 14:05:46.899065 2014-08-13 18:06:06.549410 7f5c973a57a0 20 read_log 194209'15083 (192888'14651) clean 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:1237 2014-08-12 14:05:46.902823 2014-08-13 18:06:06.549423 7f5c973a57a0 20 read_log 194209'15084 (192888'14916) clean 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:1238 2014-08-12 14:05:46.906332 2014-08-13 18:06:06.549435 7f5c973a57a0 20 read_log 194209'15085 (192888'14876) clean f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:1239 2014-08-12 14:05:46.912937 2014-08-13 18:06:06.549447 7f5c973a57a0 20 read_log 194209'15086 (193305'15032) clean 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:1240 2014-08-12 14:05:46.913193 2014-08-13 18:06:06.549460 7f5c973a57a0 20 read_log 194209'15087 (192834'14354) clean 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:1241 2014-08-12 14:05:46.917355 2014-08-13 18:06:06.549472 7f5c973a57a0 20 read_log 194209'15088 (193305'15021) clean 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:1242 2014-08-12 14:05:46.920766 2014-08-13 18:06:06.549484 7f5c973a57a0 20 read_log 194209'15089 (192888'14691) clean a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:1243 2014-08-12 14:05:46.924335 2014-08-13 18:06:06.549496 7f5c973a57a0 20 read_log 194209'15090 (192834'14525) clean a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:1244 2014-08-12 14:05:46.928141 2014-08-13 18:06:06.549508 7f5c973a57a0 20 read_log 194209'15091 (193305'15015) clean 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:1245 2014-08-12 14:05:46.931829 2014-08-13 18:06:06.549520 7f5c973a57a0 20 read_log 194209'15092 (192888'14694) clean 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:1246 2014-08-12 14:05:46.935220 2014-08-13 18:06:06.549534 7f5c973a57a0 20 read_log 194209'15093 (192888'14890) clean 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by osd.20.0:1247 2014-08-12 14:05:46.939051 2014-08-13 18:06:06.549547 7f5c973a57a0 20 read_log 194209'15094 (192834'13831) clean 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:1248 2014-08-12 14:05:46.942719 2014-08-13 18:06:06.549560 7f5c973a57a0 20 read_log 194209'15095 (193305'15030) clean 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:1249 2014-08-12 14:05:46.949217 2014-08-13 18:06:06.549573 7f5c973a57a0 20 read_log 194209'15096 (192888'14896) clean 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by osd.20.0:1250 2014-08-12 14:05:46.949495 2014-08-13 18:06:06.549585 7f5c973a57a0 20 read_log 194209'15097 (192834'13871) clean 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:1251 2014-08-12 14:05:46.953026 2014-08-13 18:06:06.549605 7f5c973a57a0 20 read_log 194209'15098 (194209'15050) delete 6973dc2c/benchmark_data_storage0111-ib.pocstor_2228_object24584/head//24 by osd.20.0:1252 2014-08-12 14:05:46.992192 2014-08-13 18:06:06.549618 7f5c973a57a0 20 read_log 194209'15099 (194209'15051) delete 203fdc2c/benchmark_data_storage0111-ib.pocstor_62621_object10952/head//24 by osd.20.0:1253 2014-08-12 14:05:46.995516 2014-08-13 18:06:06.549631 7f5c973a57a0 20 read_log 194209'15100 (194209'15052) delete b6a6ec2c/benchmark_data_storage0111-ib.pocstor_62621_object15422/head//24 by osd.20.0:1254 2014-08-12 14:05:46.998899 2014-08-13 18:06:06.549643 7f5c973a57a0 20 read_log 194209'15101 (194209'15054) delete 8149fc2c/benchmark_data_storage0111-ib.pocstor_62621_object28915/head//24 by osd.20.0:1255 2014-08-12 14:05:47.002200 2014-08-13 18:06:06.549655 7f5c973a57a0 20 read_log 194209'15102 (194209'15055) delete ca07002c/benchmark_data_storage0111-ib.pocstor_62621_object29369/head//24 by osd.20.0:1256 2014-08-12 14:05:47.005881 2014-08-13 18:06:06.549668 7f5c973a57a0 20 read_log 194209'15103 (194209'15056) delete ddcc002c/benchmark_data_storage0111-ib.pocstor_41526_object19319/head//24 by osd.20.0:1257 2014-08-12 14:05:47.009432 2014-08-13 18:06:06.549680 7f5c973a57a0 20 read_log 194209'15104 (194209'15062) delete c2ba202c/benchmark_data_storage0111-ib.pocstor_41526_object4339/head//24 by osd.20.0:1258 2014-08-12 14:05:47.013594 2014-08-13 18:06:06.549693 7f5c973a57a0 20 read_log 194209'15105 (194209'15061) delete aee2302c/benchmark_data_storage0111-ib.pocstor_62621_object26203/head//24 by osd.20.0:1259 2014-08-12 14:05:47.016646 2014-08-13 18:06:06.549713 7f5c973a57a0 20 read_log 194209'15106 (194209'15063) delete 3ae7302c/benchmark_data_storage0111-ib.pocstor_62621_object29934/head//24 by osd.20.0:1260 2014-08-12 14:05:47.020476 2014-08-13 18:06:06.549726 7f5c973a57a0 20 read_log 194209'15107 (194209'15064) delete 583e302c/benchmark_data_storage0111-ib.pocstor_41097_object12874/head//24 by osd.20.0:1261 2014-08-12 14:05:47.024320 2014-08-13 18:06:06.549741 7f5c973a57a0 20 read_log 194209'15108 (194209'15065) delete 32f9402c/benchmark_data_storage0111-ib.pocstor_41097_object612/head//24 by osd.20.0:1262 2014-08-12 14:05:47.028325 2014-08-13 18:06:06.549753 7f5c973a57a0 20 read_log 194209'15109 (192888'14889) clean 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:1263 2014-08-12 14:05:47.028782 2014-08-13 18:06:06.549766 7f5c973a57a0 20 read_log 194209'15110 (194209'15066) delete 442e402c/benchmark_data_storage0111-ib.pocstor_41097_object2020/head//24 by osd.20.0:1264 2014-08-12 14:05:47.032468 2014-08-13 18:06:06.549778 7f5c973a57a0 20 read_log 194209'15111 (194209'15068) delete 3508502c/benchmark_data_storage0111-ib.pocstor_41526_object16529/head//24 by osd.20.0:1265 2014-08-12 14:05:47.035706 2014-08-13 18:06:06.549793 7f5c973a57a0 20 read_log 194209'15112 (194209'15069) delete f969602c/benchmark_data_storage0111-ib.pocstor_41526_object398/head//24 by osd.20.0:1266 2014-08-12 14:05:47.036107 2014-08-13 18:06:06.549806 7f5c973a57a0 20 read_log 194209'15113 (194209'15070) delete 70db602c/benchmark_data_storage0111-ib.pocstor_41526_object19066/head//24 by osd.20.0:1267 2014-08-12 14:05:47.039721 2014-08-13 18:06:06.549819 7f5c973a57a0 20 read_log 194209'15114 (194209'15071) delete fe2d602c/benchmark_data_storage0111-ib.pocstor_41097_object16089/head//24 by osd.20.0:1268 2014-08-12 14:05:47.040093 2014-08-13 18:06:06.549833 7f5c973a57a0 20 read_log 194209'15115 (194209'15075) delete b1f9702c/benchmark_data_storage0111-ib.pocstor_62621_object21362/head//24 by osd.20.0:1269 2014-08-12 14:05:47.043939 2014-08-13 18:06:06.549846 7f5c973a57a0 20 read_log 194209'15116 (194209'15076) delete a0ad702c/benchmark_data_storage0111-ib.pocstor_41097_object26989/head//24 by osd.20.0:1270 2014-08-12 14:05:47.044241 2014-08-13 18:06:06.549860 7f5c973a57a0 20 read_log 194209'15117 (194209'15079) delete 718fa02c/benchmark_data_storage0111-ib.pocstor_41526_object10056/head//24 by osd.20.0:1271 2014-08-12 14:05:47.047886 2014-08-13 18:06:06.549881 7f5c973a57a0 20 read_log 194209'15118 (194209'15080) delete 3746b02c/benchmark_data_storage0111-ib.pocstor_62621_object11306/head//24 by osd.20.0:1272 2014-08-12 14:05:47.048209 2014-08-13 18:06:06.549896 7f5c973a57a0 20 read_log 194209'15119 (194209'15081) delete dbc9b02c/benchmark_data_storage0111-ib.pocstor_62621_object4841/head//24 by osd.20.0:1273 2014-08-12 14:05:47.051660 2014-08-13 18:06:06.549908 7f5c973a57a0 20 read_log 194209'15120 (194209'15082) delete 8373c02c/benchmark_data_storage0111-ib.pocstor_62621_object13294/head//24 by osd.20.0:1274 2014-08-12 14:05:47.051986 2014-08-13 18:06:06.549923 7f5c973a57a0 20 read_log 194209'15121 (194209'15083) delete 61fad02c/benchmark_data_storage0111-ib.pocstor_41097_object2320/head//24 by osd.20.0:1275 2014-08-12 14:05:47.055494 2014-08-13 18:06:06.549936 7f5c973a57a0 20 read_log 194209'15122 (194209'15086) delete 665e02c/benchmark_data_storage0111-ib.pocstor_62621_object25948/head//24 by osd.20.0:1276 2014-08-12 14:05:47.055992 2014-08-13 18:06:06.549950 7f5c973a57a0 20 read_log 194209'15123 (194209'15087) delete 23a6f02c/benchmark_data_storage0111-ib.pocstor_41526_object849/head//24 by osd.20.0:1277 2014-08-12 14:05:47.059487 2014-08-13 18:06:06.549962 7f5c973a57a0 20 read_log 194209'15124 (194209'15088) delete 402af02c/benchmark_data_storage0111-ib.pocstor_62621_object12319/head//24 by osd.20.0:1278 2014-08-12 14:05:47.059815 2014-08-13 18:06:06.549977 7f5c973a57a0 20 read_log 194209'15125 (194209'15089) delete a3bdf02c/benchmark_data_storage0111-ib.pocstor_41097_object28316/head//24 by osd.20.0:1279 2014-08-12 14:05:47.063392 2014-08-13 18:06:06.549992 7f5c973a57a0 20 read_log 194209'15126 (194209'15090) delete a4e0142c/benchmark_data_storage0111-ib.pocstor_41526_object20430/head//24 by osd.20.0:1280 2014-08-12 14:05:47.063713 2014-08-13 18:06:06.550006 7f5c973a57a0 20 read_log 194209'15127 (194209'15091) delete 5f01142c/benchmark_data_storage0111-ib.pocstor_62621_object3520/head//24 by osd.20.0:1281 2014-08-12 14:05:47.067094 2014-08-13 18:06:06.550018 7f5c973a57a0 20 read_log 194209'15128 (194209'15092) delete 6593142c/benchmark_data_storage0111-ib.pocstor_41097_object29174/head//24 by osd.20.0:1282 2014-08-12 14:05:47.067436 2014-08-13 18:06:06.550031 7f5c973a57a0 20 read_log 194209'15129 (194209'15094) delete 173f142c/benchmark_data_storage0111-ib.pocstor_26659_object12213/head//24 by osd.20.0:1283 2014-08-12 14:05:47.068215 2014-08-13 18:06:06.550043 7f5c973a57a0 20 read_log 194209'15130 (194209'15095) delete 7b80242c/benchmark_data_storage0111-ib.pocstor_62621_object25258/head//24 by osd.20.0:1284 2014-08-12 14:05:47.072043 2014-08-13 18:06:06.550056 7f5c973a57a0 20 read_log 194209'15131 (194209'15097) delete 9062342c/benchmark_data_storage0111-ib.pocstor_26659_object16362/head//24 by osd.20.0:1285 2014-08-12 14:05:47.072595 2014-08-13 18:06:06.550071 7f5c973a57a0 20 read_log 194209'15132 (194209'15109) delete 9ae8342c/benchmark_data_storage0111-ib.pocstor_2228_object16188/head//24 by osd.20.0:1457 2014-08-12 14:07:07.667995 2014-08-13 18:06:06.550084 7f5c973a57a0 20 read_log 194209'15133 (194209'15053) delete 42f8ec2c/benchmark_data_storage0111-ib.pocstor_2228_object1534/head//24 by osd.20.0:1458 2014-08-12 14:07:07.699362 2014-08-13 18:06:06.550099 7f5c973a57a0 20 read_log 194209'15134 (194209'15057) delete 74e002c/benchmark_data_storage0111-ib.pocstor_2228_object22199/head//24 by osd.20.0:1459 2014-08-12 14:07:07.702478 2014-08-13 18:06:06.550113 7f5c973a57a0 20 read_log 194209'15135 (194209'15058) delete 8c54102c/benchmark_data_storage0111-ib.pocstor_2228_object30585/head//24 by osd.20.0:1460 2014-08-12 14:07:07.702804 2014-08-13 18:06:06.550127 7f5c973a57a0 20 read_log 194209'15136 (194209'15059) delete 231a102c/benchmark_data_storage0111-ib.pocstor_2228_object27957/head//24 by osd.20.0:1461 2014-08-12 14:07:07.703220 2014-08-13 18:06:06.550140 7f5c973a57a0 20 read_log 194209'15137 (194209'15060) delete c203202c/benchmark_data_storage0111-ib.pocstor_2228_object6622/head//24 by osd.20.0:1462 2014-08-12 14:07:07.703508 2014-08-13 18:06:06.550169 7f5c973a57a0 20 read_log 194209'15138 (194209'15067) delete b4e4502c/benchmark_data_storage0111-ib.pocstor_2228_object606/head//24 by osd.20.0:1463 2014-08-12 14:07:07.706541 2014-08-13 18:06:06.550184 7f5c973a57a0 20 read_log 194209'15139 (194209'15072) delete 914e602c/benchmark_data_storage0111-ib.pocstor_2228_object14888/head//24 by osd.20.0:1464 2014-08-12 14:07:07.706997 2014-08-13 18:06:06.550197 7f5c973a57a0 20 read_log 194209'15140 (194209'15073) delete ef602c/benchmark_data_storage0111-ib.pocstor_2228_object19420/head//24 by osd.20.0:1465 2014-08-12 14:07:07.707629 2014-08-13 18:06:06.550211 7f5c973a57a0 20 read_log 194209'15141 (194209'15074) delete f345702c/benchmark_data_storage0111-ib.pocstor_41097_object6006/head//24 by osd.20.0:1466 2014-08-12 14:07:07.707919 2014-08-13 18:06:06.550225 7f5c973a57a0 20 read_log 194209'15142 (194209'15077) delete e121902c/benchmark_data_storage0111-ib.pocstor_2228_object19166/head//24 by osd.20.0:1467 2014-08-12 14:07:07.711092 2014-08-13 18:06:06.550239 7f5c973a57a0 20 read_log 194209'15143 (194209'15078) delete 2ed9a02c/benchmark_data_storage0111-ib.pocstor_2228_object11603/head//24 by osd.20.0:1468 2014-08-12 14:07:07.711438 2014-08-13 18:06:06.550252 7f5c973a57a0 20 read_log 194209'15144 (194209'15084) delete 185ed02c/benchmark_data_storage0111-ib.pocstor_2228_object26061/head//24 by osd.20.0:1469 2014-08-12 14:07:07.712111 2014-08-13 18:06:06.550266 7f5c973a57a0 20 read_log 194209'15145 (194209'15085) delete f9f2e02c/benchmark_data_storage0111-ib.pocstor_2228_object11172/head//24 by osd.20.0:1470 2014-08-12 14:07:07.712397 2014-08-13 18:06:06.550279 7f5c973a57a0 20 read_log 194215'15146 (194209'15093) delete 4ccd142c/benchmark_data_storage0111-ib.pocstor_2228_object17878/head//24 by client.2174595.0:8655 0.000000 2014-08-13 18:06:06.550295 7f5c973a57a0 20 read_log 194215'15147 (194209'15096) delete 1d32342c/benchmark_data_storage0111-ib.pocstor_2228_object18959/head//24 by client.2174595.0:8657 0.000000 2014-08-13 18:06:06.550306 7f5c973a57a0 20 read_log 194215'15148 (192888'14671) clean 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by osd.20.0:1908 2014-08-12 15:36:44.627459 2014-08-13 18:06:06.550319 7f5c973a57a0 20 read_log 194215'15149 (194215'15148) delete 5157642c/benchmark_data_storage0111-ib.pocstor_41097_object14822/head//24 by client.2174595.0:8659 0.000000 2014-08-13 18:06:06.550333 7f5c973a57a0 20 read_log 194215'15150 (192888'14887) clean 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by osd.20.0:1910 2014-08-12 15:36:44.635361 2014-08-13 18:06:06.550347 7f5c973a57a0 20 read_log 194215'15151 (194215'15150) delete 6d08642c/benchmark_data_storage0111-ib.pocstor_2228_object15357/head//24 by client.2174595.0:8661 0.000000 2014-08-13 18:06:06.550358 7f5c973a57a0 20 read_log 194215'15152 (193305'15036) clean 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by osd.20.0:1912 2014-08-12 15:36:44.642858 2014-08-13 18:06:06.550372 7f5c973a57a0 20 read_log 194215'15153 (194215'15152) delete 2380842c/benchmark_data_storage0111-ib.pocstor_62621_object26843/head//24 by client.2174595.0:8663 0.000000 2014-08-13 18:06:06.550386 7f5c973a57a0 20 read_log 194215'15154 (192888'14906) clean 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by osd.20.0:1914 2014-08-12 15:36:44.650687 2014-08-13 18:06:06.550399 7f5c973a57a0 20 read_log 194215'15155 (194215'15154) delete 4eda842c/benchmark_data_storage0111-ib.pocstor_2228_object22283/head//24 by client.2174595.0:8665 0.000000 2014-08-13 18:06:06.550419 7f5c973a57a0 20 read_log 194215'15156 (193305'15049) clean c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by osd.20.0:1916 2014-08-12 15:36:44.657298 2014-08-13 18:06:06.550439 7f5c973a57a0 20 read_log 194215'15157 (194215'15156) delete c273b42c/benchmark_data_storage0111-ib.pocstor_62621_object32368/head//24 by client.2174595.0:8667 0.000000 2014-08-13 18:06:06.550458 7f5c973a57a0 20 read_log 194215'15158 (193305'15018) clean 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by osd.20.0:1918 2014-08-12 15:36:44.664504 2014-08-13 18:06:06.550479 7f5c973a57a0 20 read_log 194215'15159 (194215'15158) delete 168eb42c/benchmark_data_storage0111-ib.pocstor_62621_object10237/head//24 by client.2174595.0:8669 0.000000 2014-08-13 18:06:06.550491 7f5c973a57a0 20 read_log 194215'15160 (193305'15027) clean bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by osd.20.0:1920 2014-08-12 15:36:44.672105 2014-08-13 18:06:06.550503 7f5c973a57a0 20 read_log 194215'15161 (194215'15160) delete bc3ac42c/benchmark_data_storage0111-ib.pocstor_62621_object21391/head//24 by client.2174595.0:8671 0.000000 2014-08-13 18:06:06.550514 7f5c973a57a0 20 read_log 194215'15162 (192834'13894) clean 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by osd.20.0:1922 2014-08-12 15:36:44.678986 2014-08-13 18:06:06.550538 7f5c973a57a0 20 read_log 194215'15163 (194215'15162) delete 827cc42c/benchmark_data_storage0111-ib.pocstor_26659_object19319/head//24 by client.2174595.0:8673 0.000000 2014-08-13 18:06:06.550550 7f5c973a57a0 20 read_log 194215'15164 (192834'13847) clean d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by osd.20.0:1924 2014-08-12 15:36:44.686694 2014-08-13 18:06:06.550588 7f5c973a57a0 20 read_log 194215'15165 (194215'15164) delete d51fc42c/benchmark_data_storage0111-ib.pocstor_26659_object13871/head//24 by client.2174595.0:8675 0.000000 2014-08-13 18:06:06.550603 7f5c973a57a0 20 read_log 194215'15166 (192834'13812) clean 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by osd.20.0:1926 2014-08-12 15:36:44.694470 2014-08-13 18:06:06.550628 7f5c973a57a0 20 read_log 194215'15167 (194215'15166) delete 84d0d42c/benchmark_data_storage0111-ib.pocstor_26659_object10292/head//24 by client.2174595.0:8677 0.000000 2014-08-13 18:06:06.550643 7f5c973a57a0 20 read_log 194215'15168 (192834'14519) clean a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by osd.20.0:1928 2014-08-12 15:36:44.702010 2014-08-13 18:06:06.550655 7f5c973a57a0 20 read_log 194215'15169 (194215'15168) delete a494d42c/benchmark_data_storage0111-ib.pocstor_41526_object19637/head//24 by client.2174595.0:8679 0.000000 2014-08-13 18:06:06.550666 7f5c973a57a0 20 read_log 194215'15170 (192888'14673) clean 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by osd.20.0:1930 2014-08-12 15:36:44.710029 2014-08-13 18:06:06.550687 7f5c973a57a0 20 read_log 194215'15171 (194215'15170) delete 5ecdd42c/benchmark_data_storage0111-ib.pocstor_41097_object15411/head//24 by client.2174595.0:8681 0.000000 2014-08-13 18:06:06.550699 7f5c973a57a0 20 read_log 194215'15172 (192834'14548) clean a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by osd.20.0:1932 2014-08-12 15:36:44.717021 2014-08-13 18:06:06.550711 7f5c973a57a0 20 read_log 194215'15173 (194215'15172) delete a3d0e42c/benchmark_data_storage0111-ib.pocstor_41526_object23020/head//24 by client.2174595.0:8683 0.000000 2014-08-13 18:06:06.550722 7f5c973a57a0 20 read_log 194215'15174 (192888'14689) clean 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by osd.20.0:1934 2014-08-12 15:36:44.724679 2014-08-13 18:06:06.550735 7f5c973a57a0 20 read_log 194215'15175 (194215'15174) delete 49cfe42c/benchmark_data_storage0111-ib.pocstor_41097_object26993/head//24 by client.2174595.0:8685 0.000000 2014-08-13 18:06:06.550746 7f5c973a57a0 20 read_log 194215'15176 (192834'14486) clean 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by osd.20.0:1936 2014-08-12 15:36:44.731917 2014-08-13 18:06:06.550760 7f5c973a57a0 20 read_log 194215'15177 (194215'15176) delete 6225f42c/benchmark_data_storage0111-ib.pocstor_41526_object15202/head//24 by client.2174595.0:8687 0.000000 2014-08-13 18:06:06.550796 7f5c973a57a0 20 read_log 194215'15178 (193305'15035) clean b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by osd.20.0:1938 2014-08-12 15:36:44.739372 2014-08-13 18:06:06.550812 7f5c973a57a0 20 read_log 194215'15179 (194215'15178) delete b5c0082c/benchmark_data_storage0111-ib.pocstor_62621_object26619/head//24 by client.2174595.0:8689 0.000000 2014-08-13 18:06:06.550824 7f5c973a57a0 20 read_log 194215'15180 (192834'13827) clean cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by osd.20.0:1940 2014-08-12 15:36:44.747117 2014-08-13 18:06:06.550836 7f5c973a57a0 20 read_log 194215'15181 (194215'15180) delete cb95082c/benchmark_data_storage0111-ib.pocstor_26659_object11857/head//24 by client.2174595.0:8691 0.000000 2014-08-13 18:06:06.550847 7f5c973a57a0 20 read_log 194215'15182 (192888'14869) clean 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by osd.20.0:1942 2014-08-12 15:36:44.755253 2014-08-13 18:06:06.550859 7f5c973a57a0 20 read_log 194215'15183 (194215'15182) delete 930c082c/benchmark_data_storage0111-ib.pocstor_2228_object7912/head//24 by client.2174595.0:8693 0.000000 2014-08-13 18:06:06.550871 7f5c973a57a0 20 read_log 194215'15184 (192834'13775) clean 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by osd.20.0:1944 2014-08-12 15:36:44.763213 2014-08-13 18:06:06.550886 7f5c973a57a0 20 read_log 194215'15185 (194215'15184) delete 26d4182c/benchmark_data_storage0111-ib.pocstor_26659_object4558/head//24 by client.2174595.0:8695 0.000000 2014-08-13 18:06:06.550896 7f5c973a57a0 20 read_log 194215'15186 (192834'14482) clean 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by osd.20.0:1946 2014-08-12 15:36:44.770514 2014-08-13 18:06:06.550909 7f5c973a57a0 20 read_log 194215'15187 (194215'15186) delete 307c182c/benchmark_data_storage0111-ib.pocstor_41526_object14861/head//24 by client.2174595.0:8697 0.000000 2014-08-13 18:06:06.550919 7f5c973a57a0 20 read_log 194215'15188 (192834'14457) clean 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by osd.20.0:1948 2014-08-12 15:36:44.778547 2014-08-13 18:06:06.550932 7f5c973a57a0 20 read_log 194215'15189 (194215'15188) delete 9fcd182c/benchmark_data_storage0111-ib.pocstor_41526_object11695/head//24 by client.2174595.0:8699 0.000000 2014-08-13 18:06:06.550943 7f5c973a57a0 20 read_log 194215'15190 (192834'14537) clean 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by osd.20.0:1950 2014-08-12 15:36:44.785597 2014-08-13 18:06:06.550955 7f5c973a57a0 20 read_log 194215'15191 (194215'15190) delete 35c5282c/benchmark_data_storage0111-ib.pocstor_41526_object21690/head//24 by client.2174595.0:8701 0.000000 2014-08-13 18:06:06.550965 7f5c973a57a0 20 read_log 194215'15192 (192888'14687) clean ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by osd.20.0:1952 2014-08-12 15:36:44.794152 2014-08-13 18:06:06.550978 7f5c973a57a0 20 read_log 194215'15193 (194215'15192) delete ca28282c/benchmark_data_storage0111-ib.pocstor_41097_object26689/head//24 by client.2174595.0:8703 0.000000 2014-08-13 18:06:06.550988 7f5c973a57a0 20 read_log 194215'15194 (192834'13868) clean 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by osd.20.0:1954 2014-08-12 15:36:44.802563 2014-08-13 18:06:06.551001 7f5c973a57a0 20 read_log 194215'15195 (194215'15194) delete 81ea382c/benchmark_data_storage0111-ib.pocstor_26659_object16173/head//24 by client.2174595.0:8705 0.000000 2014-08-13 18:06:06.551011 7f5c973a57a0 20 read_log 194215'15196 (192888'14883) clean 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by osd.20.0:1956 2014-08-12 15:36:44.810450 2014-08-13 18:06:06.551024 7f5c973a57a0 20 read_log 194215'15197 (194215'15196) delete 67fc382c/benchmark_data_storage0111-ib.pocstor_2228_object14152/head//24 by client.2174595.0:8707 0.000000 2014-08-13 18:06:06.551044 7f5c973a57a0 20 read_log 194215'15198 (193305'15025) clean 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by osd.20.0:1958 2014-08-12 15:36:44.817737 2014-08-13 18:06:06.551057 7f5c973a57a0 20 read_log 194215'15199 (194215'15198) delete 1f50482c/benchmark_data_storage0111-ib.pocstor_62621_object18372/head//24 by client.2174595.0:8709 0.000000 2014-08-13 18:06:06.551068 7f5c973a57a0 20 read_log 194215'15200 (193305'15034) clean a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by osd.20.0:1960 2014-08-12 15:36:44.825157 2014-08-13 18:06:06.551080 7f5c973a57a0 20 read_log 194215'15201 (194215'15200) delete a1f3482c/benchmark_data_storage0111-ib.pocstor_62621_object26240/head//24 by client.2174595.0:8711 0.000000 2014-08-13 18:06:06.551094 7f5c973a57a0 20 read_log 194215'15202 (192834'13759) clean b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by osd.20.0:1962 2014-08-12 15:36:44.832647 2014-08-13 18:06:06.551106 7f5c973a57a0 20 read_log 194215'15203 (194215'15202) delete b144482c/benchmark_data_storage0111-ib.pocstor_26659_object2101/head//24 by client.2174595.0:8713 0.000000 2014-08-13 18:06:06.551120 7f5c973a57a0 20 read_log 194215'15204 (193305'15046) clean 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by osd.20.0:1964 2014-08-12 15:36:44.841069 2014-08-13 18:06:06.551132 7f5c973a57a0 20 read_log 194215'15205 (194215'15204) delete 3197482c/benchmark_data_storage0111-ib.pocstor_62621_object31409/head//24 by client.2174595.0:8715 0.000000 2014-08-13 18:06:06.551151 7f5c973a57a0 20 read_log 194215'15206 (192888'14888) clean 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by osd.20.0:1966 2014-08-12 15:36:44.848993 2014-08-13 18:06:06.551166 7f5c973a57a0 20 read_log 194215'15207 (194215'15206) delete 8ae7482c/benchmark_data_storage0111-ib.pocstor_2228_object15774/head//24 by client.2174595.0:8717 0.000000 2014-08-13 18:06:06.551178 7f5c973a57a0 20 read_log 194215'15208 (192834'13764) clean b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by osd.20.0:1968 2014-08-12 15:36:44.857737 2014-08-13 18:06:06.551191 7f5c973a57a0 20 read_log 194215'15209 (194215'15208) delete b11a482c/benchmark_data_storage0111-ib.pocstor_26659_object3048/head//24 by client.2174595.0:8719 0.000000 2014-08-13 18:06:06.551202 7f5c973a57a0 20 read_log 194215'15210 (193305'15048) clean e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by osd.20.0:1970 2014-08-12 15:36:44.866794 2014-08-13 18:06:06.551214 7f5c973a57a0 20 read_log 194215'15211 (194215'15210) delete e6ca582c/benchmark_data_storage0111-ib.pocstor_62621_object32250/head//24 by client.2174595.0:8721 0.000000 2014-08-13 18:06:06.551225 7f5c973a57a0 20 read_log 194215'15212 (192888'14660) clean 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by osd.20.0:1972 2014-08-12 15:36:44.875025 2014-08-13 18:06:06.551237 7f5c973a57a0 20 read_log 194215'15213 (194215'15212) delete 168b582c/benchmark_data_storage0111-ib.pocstor_41097_object8151/head//24 by client.2174595.0:8723 0.000000 2014-08-13 18:06:06.551247 7f5c973a57a0 20 read_log 194215'15214 (193305'15038) clean 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by osd.20.0:1974 2014-08-12 15:36:44.882935 2014-08-13 18:06:06.551259 7f5c973a57a0 20 read_log 194215'15215 (194215'15214) delete 8887682c/benchmark_data_storage0111-ib.pocstor_62621_object27593/head//24 by client.2174595.0:8725 0.000000 2014-08-13 18:06:06.551272 7f5c973a57a0 20 read_log 194215'15216 (193305'15013) clean d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by osd.20.0:1976 2014-08-12 15:36:44.890437 2014-08-13 18:06:06.551287 7f5c973a57a0 20 read_log 194215'15217 (194215'15216) delete d4e3882c/benchmark_data_storage0111-ib.pocstor_62621_object2215/head//24 by client.2174595.0:8727 0.000000 2014-08-13 18:06:06.551308 7f5c973a57a0 20 read_log 194215'15218 (192888'14880) clean cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by osd.20.0:1978 2014-08-12 15:36:44.898648 2014-08-13 18:06:06.551323 7f5c973a57a0 20 read_log 194215'15219 (194215'15218) delete cffe882c/benchmark_data_storage0111-ib.pocstor_2228_object13307/head//24 by client.2174595.0:8729 0.000000 2014-08-13 18:06:06.551334 7f5c973a57a0 20 read_log 194215'15220 (192888'14903) clean 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by osd.20.0:1980 2014-08-12 15:36:44.906141 2014-08-13 18:06:06.551346 7f5c973a57a0 20 read_log 194215'15221 (194215'15220) delete 8cd3982c/benchmark_data_storage0111-ib.pocstor_2228_object21553/head//24 by client.2174595.0:8731 0.000000 2014-08-13 18:06:06.551357 7f5c973a57a0 20 read_log 194215'15222 (193305'15010) clean 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by osd.20.0:1982 2014-08-12 15:36:44.914806 2014-08-13 18:06:06.551369 7f5c973a57a0 20 read_log 194215'15223 (194215'15222) delete 7bd4982c/benchmark_data_storage0111-ib.pocstor_62621_object54/head//24 by client.2174595.0:8733 0.000000 2014-08-13 18:06:06.551380 7f5c973a57a0 20 read_log 194215'15224 (192834'13906) clean 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by osd.20.0:1984 2014-08-12 15:36:44.923169 2014-08-13 18:06:06.551392 7f5c973a57a0 20 read_log 194215'15225 (194215'15224) delete 947b982c/benchmark_data_storage0111-ib.pocstor_26659_object22234/head//24 by client.2174595.0:8735 0.000000 2014-08-13 18:06:06.551403 7f5c973a57a0 20 read_log 194215'15226 (192888'14899) clean 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by osd.20.0:1986 2014-08-12 15:36:44.931864 2014-08-13 18:06:06.551424 7f5c973a57a0 20 read_log 194215'15227 (194215'15226) delete 961f982c/benchmark_data_storage0111-ib.pocstor_2228_object19588/head//24 by client.2174595.0:8737 0.000000 2014-08-13 18:06:06.551436 7f5c973a57a0 20 read_log 194215'15228 (192834'13879) clean 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by osd.20.0:1988 2014-08-12 15:36:44.940390 2014-08-13 18:06:06.551449 7f5c973a57a0 20 read_log 194215'15229 (194215'15228) delete 7707a82c/benchmark_data_storage0111-ib.pocstor_26659_object17158/head//24 by client.2174595.0:8739 0.000000 2014-08-13 18:06:06.551460 7f5c973a57a0 20 read_log 194215'15230 (192888'14658) clean 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by osd.20.0:1990 2014-08-12 15:36:44.948318 2014-08-13 18:06:06.551488 7f5c973a57a0 20 read_log 194215'15231 (194215'15230) delete 6a35b82c/benchmark_data_storage0111-ib.pocstor_41097_object7679/head//24 by client.2174595.0:8741 0.000000 2014-08-13 18:06:06.551500 7f5c973a57a0 20 read_log 194215'15232 (193305'15017) clean 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by osd.20.0:1992 2014-08-12 15:36:44.956302 2014-08-13 18:06:06.551519 7f5c973a57a0 20 read_log 194215'15233 (194215'15232) delete 595eb82c/benchmark_data_storage0111-ib.pocstor_62621_object8143/head//24 by client.2174595.0:8743 0.000000 2014-08-13 18:06:06.551536 7f5c973a57a0 20 read_log 194215'15234 (192834'14352) clean 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by osd.20.0:1994 2014-08-12 15:36:44.963909 2014-08-13 18:06:06.551552 7f5c973a57a0 20 read_log 194215'15235 (194215'15234) delete 4e73c82c/benchmark_data_storage0111-ib.pocstor_41526_object779/head//24 by client.2174595.0:8745 0.000000 2014-08-13 18:06:06.551574 7f5c973a57a0 20 read_log 194215'15236 (192834'13765) clean 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by osd.20.0:1996 2014-08-12 15:36:44.971859 2014-08-13 18:06:06.551593 7f5c973a57a0 20 read_log 194215'15237 (194215'15236) delete 9645c82c/benchmark_data_storage0111-ib.pocstor_26659_object3486/head//24 by client.2174595.0:8747 0.000000 2014-08-13 18:06:06.551612 7f5c973a57a0 20 read_log 194215'15238 (192834'14474) clean 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by osd.20.0:1998 2014-08-12 15:36:44.980030 2014-08-13 18:06:06.551627 7f5c973a57a0 20 read_log 194215'15239 (194215'15238) delete 8a51d82c/benchmark_data_storage0111-ib.pocstor_41526_object14051/head//24 by client.2174595.0:8749 0.000000 2014-08-13 18:06:06.551650 7f5c973a57a0 20 read_log 194215'15240 (192834'13860) clean 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by osd.20.0:2000 2014-08-12 15:36:44.988044 2014-08-13 18:06:06.551663 7f5c973a57a0 20 read_log 194215'15241 (194215'15240) delete 6b32d82c/benchmark_data_storage0111-ib.pocstor_26659_object15538/head//24 by client.2174595.0:8751 0.000000 2014-08-13 18:06:06.551674 7f5c973a57a0 20 read_log 194215'15242 (192834'14497) clean 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by osd.20.0:2002 2014-08-12 15:36:44.997030 2014-08-13 18:06:06.551693 7f5c973a57a0 20 read_log 194215'15243 (194215'15242) delete 8e53d82c/benchmark_data_storage0111-ib.pocstor_41526_object16493/head//24 by client.2174595.0:8753 0.000000 2014-08-13 18:06:06.551704 7f5c973a57a0 20 read_log 194215'15244 (193305'15042) clean 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by osd.20.0:2004 2014-08-12 15:36:45.005546 2014-08-13 18:06:06.551717 7f5c973a57a0 20 read_log 194215'15245 (194215'15244) delete 85a3d82c/benchmark_data_storage0111-ib.pocstor_62621_object29502/head//24 by client.2174595.0:8755 0.000000 2014-08-13 18:06:06.551730 7f5c973a57a0 20 read_log 194215'15246 (192888'14902) clean fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by osd.20.0:2006 2014-08-12 15:36:45.013739 2014-08-13 18:06:06.551745 7f5c973a57a0 20 read_log 194215'15247 (194215'15246) delete fcd4d82c/benchmark_data_storage0111-ib.pocstor_2228_object21398/head//24 by client.2174595.0:8757 0.000000 2014-08-13 18:06:06.551757 7f5c973a57a0 20 read_log 194215'15248 (192888'14854) clean 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by osd.20.0:2008 2014-08-12 15:36:45.021947 2014-08-13 18:06:06.551785 7f5c973a57a0 20 read_log 194215'15249 (194215'15248) delete 6b55d82c/benchmark_data_storage0111-ib.pocstor_2228_object1086/head//24 by client.2174595.0:8759 0.000000 2014-08-13 18:06:06.551796 7f5c973a57a0 20 read_log 194215'15250 (192834'13867) clean bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by osd.20.0:2010 2014-08-12 15:36:45.030869 2014-08-13 18:06:06.551809 7f5c973a57a0 20 read_log 194215'15251 (194215'15250) delete bd8cd82c/benchmark_data_storage0111-ib.pocstor_26659_object16064/head//24 by client.2174595.0:8761 0.000000 2014-08-13 18:06:06.551821 7f5c973a57a0 20 read_log 194215'15252 (193305'15014) clean 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by osd.20.0:2012 2014-08-12 15:36:45.038894 2014-08-13 18:06:06.551836 7f5c973a57a0 20 read_log 194215'15253 (194215'15252) delete 4d5fd82c/benchmark_data_storage0111-ib.pocstor_62621_object2967/head//24 by client.2174595.0:8763 0.000000 2014-08-13 18:06:06.551848 7f5c973a57a0 20 read_log 194215'15254 (193305'15028) clean 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by osd.20.0:2014 2014-08-12 15:36:45.047459 2014-08-13 18:06:06.551860 7f5c973a57a0 20 read_log 194215'15255 (194215'15254) delete 6678e82c/benchmark_data_storage0111-ib.pocstor_62621_object22068/head//24 by client.2174595.0:8765 0.000000 2014-08-13 18:06:06.551872 7f5c973a57a0 20 read_log 194215'15256 (193305'15045) clean 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by osd.20.0:2016 2014-08-12 15:36:45.055965 2014-08-13 18:06:06.551899 7f5c973a57a0 20 read_log 194215'15257 (194215'15256) delete 31e1f82c/benchmark_data_storage0111-ib.pocstor_62621_object30266/head//24 by client.2174595.0:8767 0.000000 2014-08-13 18:06:06.551942 7f5c973a57a0 20 read_log 194215'15258 (192888'14884) clean 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by osd.20.0:2018 2014-08-12 15:36:45.064452 2014-08-13 18:06:06.551959 7f5c973a57a0 20 read_log 194215'15259 (194215'15258) delete 5be2f82c/benchmark_data_storage0111-ib.pocstor_2228_object14476/head//24 by client.2174595.0:8769 0.000000 2014-08-13 18:06:06.551970 7f5c973a57a0 20 read_log 194215'15260 (192834'13782) clean c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by osd.20.0:2020 2014-08-12 15:36:45.072427 2014-08-13 18:06:06.551986 7f5c973a57a0 20 read_log 194215'15261 (194215'15260) delete c9a00c2c/benchmark_data_storage0111-ib.pocstor_26659_object6104/head//24 by client.2174595.0:8771 0.000000 2014-08-13 18:06:06.552006 7f5c973a57a0 20 read_log 194215'15262 (192834'13805) clean 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by osd.20.0:2022 2014-08-12 15:36:45.080612 2014-08-13 18:06:06.552021 7f5c973a57a0 20 read_log 194215'15263 (194215'15262) delete 88250c2c/benchmark_data_storage0111-ib.pocstor_26659_object9177/head//24 by client.2174595.0:8773 0.000000 2014-08-13 18:06:06.552033 7f5c973a57a0 20 read_log 194215'15264 (192834'14379) clean 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by osd.20.0:2024 2014-08-12 15:36:45.088643 2014-08-13 18:06:06.552045 7f5c973a57a0 20 read_log 194215'15265 (194215'15264) delete 2d060c2c/benchmark_data_storage0111-ib.pocstor_41526_object4334/head//24 by client.2174595.0:8775 0.000000 2014-08-13 18:06:06.552057 7f5c973a57a0 20 read_log 194215'15266 (193305'15039) clean 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by osd.20.0:2026 2014-08-12 15:36:45.097022 2014-08-13 18:06:06.552069 7f5c973a57a0 20 read_log 194215'15267 (194215'15266) delete 38ab0c2c/benchmark_data_storage0111-ib.pocstor_62621_object27804/head//24 by client.2174595.0:8777 0.000000 2014-08-13 18:06:06.552080 7f5c973a57a0 20 read_log 194215'15268 (192834'13800) clean c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by osd.20.0:2028 2014-08-12 15:36:45.104263 2014-08-13 18:06:06.552093 7f5c973a57a0 20 read_log 194215'15269 (194215'15268) delete c3c41c2c/benchmark_data_storage0111-ib.pocstor_26659_object8122/head//24 by client.2174595.0:8779 0.000000 2014-08-13 18:06:06.552104 7f5c973a57a0 20 read_log 194215'15270 (192888'14867) clean c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by osd.20.0:2030 2014-08-12 15:36:45.111517 2014-08-13 18:06:06.552117 7f5c973a57a0 20 read_log 194215'15271 (194215'15270) delete c9ca1c2c/benchmark_data_storage0111-ib.pocstor_2228_object7861/head//24 by client.2174595.0:8781 0.000000 2014-08-13 18:06:06.552127 7f5c973a57a0 20 read_log 194215'15272 (192888'14881) clean 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by osd.20.0:2032 2014-08-12 15:36:45.119682 2014-08-13 18:06:06.552140 7f5c973a57a0 20 read_log 194215'15273 (194215'15272) delete 59cd2c2c/benchmark_data_storage0111-ib.pocstor_2228_object13466/head//24 by client.2174595.0:8783 0.000000 2014-08-13 18:06:06.552169 7f5c973a57a0 20 read_log 194215'15274 (193305'15031) clean 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by osd.20.0:2034 2014-08-12 15:36:45.127445 2014-08-13 18:06:06.552186 7f5c973a57a0 20 read_log 194215'15275 (194215'15274) delete 757f2c2c/benchmark_data_storage0111-ib.pocstor_62621_object25272/head//24 by client.2174595.0:8785 0.000000 2014-08-13 18:06:06.552197 7f5c973a57a0 20 read_log 194215'15276 (193305'15037) clean 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by osd.20.0:2036 2014-08-12 15:36:45.134997 2014-08-13 18:06:06.552213 7f5c973a57a0 20 read_log 194215'15277 (194215'15276) delete 65083c2c/benchmark_data_storage0111-ib.pocstor_62621_object27241/head//24 by client.2174595.0:8787 0.000000 2014-08-13 18:06:06.552233 7f5c973a57a0 20 read_log 194215'15278 (193305'15012) clean 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by osd.20.0:2038 2014-08-12 15:36:45.142770 2014-08-13 18:06:06.552248 7f5c973a57a0 20 read_log 194215'15279 (194215'15278) delete 3b93c2c/benchmark_data_storage0111-ib.pocstor_62621_object775/head//24 by client.2174595.0:8789 0.000000 2014-08-13 18:06:06.552259 7f5c973a57a0 20 read_log 194215'15280 (192888'14872) clean 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by osd.20.0:2040 2014-08-12 15:36:45.149860 2014-08-13 18:06:06.552271 7f5c973a57a0 20 read_log 194215'15281 (194215'15280) delete 607b3c2c/benchmark_data_storage0111-ib.pocstor_2228_object8907/head//24 by client.2174595.0:8791 0.000000 2014-08-13 18:06:06.552282 7f5c973a57a0 20 read_log 194215'15282 (192888'14900) clean 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by osd.20.0:2042 2014-08-12 15:36:45.157208 2014-08-13 18:06:06.552295 7f5c973a57a0 20 read_log 194215'15283 (194215'15282) delete 498e3c2c/benchmark_data_storage0111-ib.pocstor_2228_object20974/head//24 by client.2174595.0:8793 0.000000 2014-08-13 18:06:06.552308 7f5c973a57a0 20 read_log 194215'15284 (192888'14692) clean 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by osd.20.0:2044 2014-08-12 15:36:45.163724 2014-08-13 18:06:06.552321 7f5c973a57a0 20 read_log 194215'15285 (194215'15284) delete 8d0a4c2c/benchmark_data_storage0111-ib.pocstor_41097_object28323/head//24 by client.2174595.0:8795 0.000000 2014-08-13 18:06:06.552332 7f5c973a57a0 20 read_log 194215'15286 (192888'14909) clean f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by osd.20.0:2046 2014-08-12 15:36:45.171822 2014-08-13 18:06:06.552358 7f5c973a57a0 20 read_log 194215'15287 (194215'15286) delete f3215c2c/benchmark_data_storage0111-ib.pocstor_2228_object23840/head//24 by client.2174595.0:8797 0.000000 2014-08-13 18:06:06.552370 7f5c973a57a0 20 read_log 194215'15288 (192888'14919) clean 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by osd.20.0:2048 2014-08-12 15:36:45.179108 2014-08-13 18:06:06.552385 7f5c973a57a0 20 read_log 194215'15289 (194215'15288) delete 53255c2c/benchmark_data_storage0111-ib.pocstor_2228_object26911/head//24 by client.2174595.0:8799 0.000000 2014-08-13 18:06:06.552396 7f5c973a57a0 20 read_log 194215'15290 (192834'13820) clean 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by osd.20.0:2050 2014-08-12 15:36:45.186986 2014-08-13 18:06:06.552411 7f5c973a57a0 20 read_log 194215'15291 (194215'15290) delete 50065c2c/benchmark_data_storage0111-ib.pocstor_26659_object11360/head//24 by client.2174595.0:8801 0.000000 2014-08-13 18:06:06.552422 7f5c973a57a0 20 read_log 194215'15292 (192888'14681) clean ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by osd.20.0:2052 2014-08-12 15:36:45.194424 2014-08-13 18:06:06.552443 7f5c973a57a0 20 read_log 194215'15293 (194215'15292) delete ea5a5c2c/benchmark_data_storage0111-ib.pocstor_41097_object22376/head//24 by client.2174595.0:8803 0.000000 2014-08-13 18:06:06.552461 7f5c973a57a0 20 read_log 194215'15294 (192888'14910) clean 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by osd.20.0:2054 2014-08-12 15:36:45.201742 2014-08-13 18:06:06.552486 7f5c973a57a0 20 read_log 194215'15295 (194215'15294) delete 52ac5c2c/benchmark_data_storage0111-ib.pocstor_2228_object24307/head//24 by client.2174595.0:8805 0.000000 2014-08-13 18:06:06.552501 7f5c973a57a0 20 read_log 194215'15296 (193305'15047) clean a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by osd.20.0:2056 2014-08-12 15:36:45.209963 2014-08-13 18:06:06.552513 7f5c973a57a0 20 read_log 194215'15297 (194215'15296) delete a1f5c2c/benchmark_data_storage0111-ib.pocstor_62621_object31600/head//24 by client.2174595.0:8807 0.000000 2014-08-13 18:06:06.552539 7f5c973a57a0 20 read_log 194215'15298 (192888'14655) clean d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by osd.20.0:2058 2014-08-12 15:36:45.217289 2014-08-13 18:06:06.552555 7f5c973a57a0 20 read_log 194215'15299 (194215'15298) delete d4d16c2c/benchmark_data_storage0111-ib.pocstor_41097_object6480/head//24 by client.2174595.0:8809 0.000000 2014-08-13 18:06:06.552566 7f5c973a57a0 20 read_log 194215'15300 (193305'15044) clean b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by osd.20.0:2060 2014-08-12 15:36:45.224361 2014-08-13 18:06:06.552586 7f5c973a57a0 20 read_log 194215'15301 (194215'15300) delete b7166c2c/benchmark_data_storage0111-ib.pocstor_62621_object30050/head//24 by client.2174595.0:8811 0.000000 2014-08-13 18:06:06.552600 7f5c973a57a0 20 read_log 194215'15302 (192888'14858) clean 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by osd.20.0:2062 2014-08-12 15:36:45.231834 2014-08-13 18:06:06.552617 7f5c973a57a0 20 read_log 194215'15303 (194215'15302) delete 29fb6c2c/benchmark_data_storage0111-ib.pocstor_2228_object3675/head//24 by client.2174595.0:8813 0.000000 2014-08-13 18:06:06.552635 7f5c973a57a0 20 read_log 194215'15304 (192888'14678) clean d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by osd.20.0:2064 2014-08-12 15:36:45.240026 2014-08-13 18:06:06.552651 7f5c973a57a0 20 read_log 194215'15305 (194215'15304) delete d687c2c/benchmark_data_storage0111-ib.pocstor_41097_object20126/head//24 by client.2174595.0:8815 0.000000 2014-08-13 18:06:06.552662 7f5c973a57a0 20 read_log 194215'15306 (192834'13854) clean 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by osd.20.0:2066 2014-08-12 15:36:45.247208 2014-08-13 18:06:06.552680 7f5c973a57a0 20 read_log 194215'15307 (194215'15306) delete 31797c2c/benchmark_data_storage0111-ib.pocstor_26659_object14668/head//24 by client.2174595.0:8817 0.000000 2014-08-13 18:06:06.552695 7f5c973a57a0 20 read_log 194215'15308 (193305'15024) clean 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by osd.20.0:2068 2014-08-12 15:36:45.254384 2014-08-13 18:06:06.552708 7f5c973a57a0 20 read_log 194215'15309 (194215'15308) delete 26fa7c2c/benchmark_data_storage0111-ib.pocstor_62621_object15994/head//24 by client.2174595.0:8819 0.000000 2014-08-13 18:06:06.552718 7f5c973a57a0 20 read_log 194215'15310 (192888'14676) clean 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by osd.20.0:2070 2014-08-12 15:36:45.262595 2014-08-13 18:06:06.552733 7f5c973a57a0 20 read_log 194215'15311 (194215'15310) delete 34ec7c2c/benchmark_data_storage0111-ib.pocstor_41097_object19022/head//24 by client.2174595.0:8821 0.000000 2014-08-13 18:06:06.552744 7f5c973a57a0 20 read_log 194215'15312 (192834'14438) clean d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by osd.20.0:2072 2014-08-12 15:36:45.270304 2014-08-13 18:06:06.552756 7f5c973a57a0 20 read_log 194215'15313 (194215'15312) delete d7518c2c/benchmark_data_storage0111-ib.pocstor_41526_object10151/head//24 by client.2174595.0:8823 0.000000 2014-08-13 18:06:06.552768 7f5c973a57a0 20 read_log 194215'15314 (192888'14911) clean a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by osd.20.0:2074 2014-08-12 15:36:45.277518 2014-08-13 18:06:06.552781 7f5c973a57a0 20 read_log 194215'15315 (194215'15314) delete a0a98c2c/benchmark_data_storage0111-ib.pocstor_2228_object24407/head//24 by client.2174595.0:8825 0.000000 2014-08-13 18:06:06.552792 7f5c973a57a0 20 read_log 194215'15316 (192834'13838) clean a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by osd.20.0:2076 2014-08-12 15:36:45.285712 2014-08-13 18:06:06.552804 7f5c973a57a0 20 read_log 194215'15317 (194215'15316) delete a0a8c2c/benchmark_data_storage0111-ib.pocstor_26659_object12736/head//24 by client.2174595.0:8827 0.000000 2014-08-13 18:06:06.552841 7f5c973a57a0 20 read_log 194215'15318 (193305'15011) clean fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by osd.20.0:2078 2014-08-12 15:36:45.295915 2014-08-13 18:06:06.552858 7f5c973a57a0 20 read_log 194215'15319 (194215'15318) delete fef19c2c/benchmark_data_storage0111-ib.pocstor_62621_object475/head//24 by client.2174595.0:8829 0.000000 2014-08-13 18:06:06.552869 7f5c973a57a0 20 read_log 194215'15320 (192888'14863) clean badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by osd.20.0:2080 2014-08-12 15:36:45.303574 2014-08-13 18:06:06.552881 7f5c973a57a0 20 read_log 194215'15321 (194215'15320) delete badb9c2c/benchmark_data_storage0111-ib.pocstor_2228_object5561/head//24 by client.2174595.0:8831 0.000000 2014-08-13 18:06:06.552892 7f5c973a57a0 20 read_log 194215'15322 (193305'15029) clean e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by osd.20.0:2082 2014-08-12 15:36:45.312310 2014-08-13 18:06:06.552904 7f5c973a57a0 20 read_log 194215'15323 (194215'15322) delete e73ac2c/benchmark_data_storage0111-ib.pocstor_62621_object23095/head//24 by client.2174595.0:8833 0.000000 2014-08-13 18:06:06.552915 7f5c973a57a0 20 read_log 194215'15324 (192888'14670) clean ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by osd.20.0:2084 2014-08-12 15:36:45.320480 2014-08-13 18:06:06.552927 7f5c973a57a0 20 read_log 194215'15325 (194215'15324) delete ddb6ac2c/benchmark_data_storage0111-ib.pocstor_41097_object14703/head//24 by client.2174595.0:8835 0.000000 2014-08-13 18:06:06.552938 7f5c973a57a0 20 read_log 194215'15326 (192888'14668) clean 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by osd.20.0:2086 2014-08-12 15:36:45.328462 2014-08-13 18:06:06.552951 7f5c973a57a0 20 read_log 194215'15327 (194215'15326) delete 9932bc2c/benchmark_data_storage0111-ib.pocstor_41097_object13594/head//24 by client.2174595.0:8837 0.000000 2014-08-13 18:06:06.552961 7f5c973a57a0 20 read_log 194215'15328 (192834'13885) clean 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by osd.20.0:2088 2014-08-12 15:36:45.335402 2014-08-13 18:06:06.552993 7f5c973a57a0 20 read_log 194215'15329 (194215'15328) delete 233cc2c/benchmark_data_storage0111-ib.pocstor_26659_object17672/head//24 by client.2174595.0:8839 0.000000 2014-08-13 18:06:06.553005 7f5c973a57a0 20 read_log 194215'15330 (192888'14664) clean 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by osd.20.0:2090 2014-08-12 15:36:45.342642 2014-08-13 18:06:06.553017 7f5c973a57a0 20 read_log 194215'15331 (194215'15330) delete 487cc2c/benchmark_data_storage0111-ib.pocstor_41097_object11507/head//24 by client.2174595.0:8841 0.000000 2014-08-13 18:06:06.553028 7f5c973a57a0 20 read_log 194215'15332 (192834'13772) clean 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by osd.20.0:2092 2014-08-12 15:36:45.350452 2014-08-13 18:06:06.553040 7f5c973a57a0 20 read_log 194215'15333 (194215'15332) delete 6d98cc2c/benchmark_data_storage0111-ib.pocstor_26659_object4212/head//24 by client.2174595.0:8843 0.000000 2014-08-13 18:06:06.553051 7f5c973a57a0 20 read_log 194217'15334 (194217'15334) modify 2c/hit_set_24.2c_archive_2014-08-12 15:32:25.737566_2014-08-12 17:25:03.718423/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 18:06:06.553064 7f5c973a57a0 20 read_log 194217'15335 (193305'14928) delete 2c/hit_set_24.2c_archive_2014-08-11 18:37:04.911332_2014-08-11 18:42:33.028272/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 18:06:06.553079 7f5c973a57a0 20 read_log 194217'15336 (0'0) promote 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by osd.20.0:4154 2014-08-12 17:25:03.734143 2014-08-13 18:06:06.553099 7f5c973a57a0 20 read_log 194217'15337 (194217'15336) modify 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by client.1487927.0:102 2014-08-12 17:25:03.710273 2014-08-13 18:06:06.553114 7f5c973a57a0 20 read_log 194217'15338 (0'0) promote a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by osd.20.0:4158 2014-08-12 17:25:03.872051 2014-08-13 18:06:06.553126 7f5c973a57a0 20 read_log 194217'15339 (194217'15338) modify a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by client.1487927.0:146 2014-08-12 17:25:03.864452 2014-08-13 18:06:06.553141 7f5c973a57a0 20 read_log 194370'15340 (0'0) promote e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by osd.20.0:4200 2014-08-12 20:30:19.536759 2014-08-13 18:06:06.553161 7f5c973a57a0 20 read_log 194370'15341 (194370'15340) modify e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by client.1508531.0:1373 2014-08-12 20:30:19.528413 2014-08-13 18:06:06.553174 7f5c973a57a0 20 read_log 194370'15342 (0'0) promote bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by osd.20.0:4202 2014-08-12 20:30:20.278092 2014-08-13 18:06:06.553186 7f5c973a57a0 20 read_log 194370'15343 (194370'15342) modify bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by client.1508531.0:1571 2014-08-12 20:30:20.269125 2014-08-13 18:06:06.553198 7f5c973a57a0 20 read_log 194370'15344 (0'0) promote 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by osd.20.0:4216 2014-08-12 20:30:22.042177 2014-08-13 18:06:06.553210 7f5c973a57a0 20 read_log 194370'15345 (194370'15344) modify 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by client.1508531.0:2033 2014-08-12 20:30:22.032396 2014-08-13 18:06:06.553223 7f5c973a57a0 20 read_log 194370'15346 (0'0) promote ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by osd.20.0:4236 2014-08-12 20:30:24.034883 2014-08-13 18:06:06.553235 7f5c973a57a0 20 read_log 194370'15347 (194370'15346) modify ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by client.1508531.0:2583 2014-08-12 20:30:24.026611 2014-08-13 18:06:06.553250 7f5c973a57a0 20 read_log 194370'15348 (0'0) promote 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by osd.20.0:4238 2014-08-12 20:30:24.464666 2014-08-13 18:06:06.553263 7f5c973a57a0 20 read_log 194370'15349 (194370'15348) modify 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by client.1508531.0:2702 2014-08-12 20:30:24.456839 2014-08-13 18:06:06.553275 7f5c973a57a0 20 read_log 194370'15350 (0'0) promote a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by osd.20.0:4260 2014-08-12 20:30:28.717201 2014-08-13 18:06:06.553311 7f5c973a57a0 20 read_log 194370'15351 (194370'15350) modify a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by client.1508531.0:3842 2014-08-12 20:30:28.710059 2014-08-13 18:06:06.553326 7f5c973a57a0 20 read_log 194370'15352 (0'0) promote 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by osd.20.0:4272 2014-08-12 20:30:30.667400 2014-08-13 18:06:06.553342 7f5c973a57a0 20 read_log 194370'15353 (194370'15352) modify 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by client.1508531.0:4358 2014-08-12 20:30:30.658820 2014-08-13 18:06:06.553356 7f5c973a57a0 20 read_log 194370'15354 (0'0) promote b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by osd.20.0:4288 2014-08-12 20:30:35.170349 2014-08-13 18:06:06.553368 7f5c973a57a0 20 read_log 194370'15355 (194370'15354) modify b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by client.1508531.0:5446 2014-08-12 20:30:35.162206 2014-08-13 18:06:06.553384 7f5c973a57a0 20 read_log 194370'15356 (0'0) promote f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by osd.20.0:4294 2014-08-12 20:30:37.799661 2014-08-13 18:06:06.553413 7f5c973a57a0 20 read_log 194370'15357 (194370'15356) modify f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by client.1508531.0:6016 2014-08-12 20:30:37.790483 2014-08-13 18:06:06.553432 7f5c973a57a0 20 read_log 194370'15358 (0'0) promote 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by osd.20.0:4308 2014-08-12 20:30:40.601390 2014-08-13 18:06:06.553448 7f5c973a57a0 20 read_log 194370'15359 (194370'15358) modify 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by client.1508531.0:6638 2014-08-12 20:30:40.590468 2014-08-13 18:06:06.553461 7f5c973a57a0 20 read_log 194370'15360 (0'0) promote 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by osd.20.0:4322 2014-08-12 20:30:43.151420 2014-08-13 18:06:06.553481 7f5c973a57a0 20 read_log 194370'15361 (194370'15360) modify 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by client.1508531.0:7156 2014-08-12 20:30:43.142664 2014-08-13 18:06:06.553497 7f5c973a57a0 20 read_log 194370'15362 (0'0) promote b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by osd.20.0:4326 2014-08-12 20:30:43.317583 2014-08-13 18:06:06.553509 7f5c973a57a0 20 read_log 194370'15363 (194370'15362) modify b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by client.1508531.0:7190 2014-08-12 20:30:43.308490 2014-08-13 18:06:06.553529 7f5c973a57a0 20 read_log 194370'15364 (0'0) promote 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by osd.20.0:4372 2014-08-12 20:30:53.223169 2014-08-13 18:06:06.553544 7f5c973a57a0 20 read_log 194370'15365 (194370'15364) modify 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by client.1508531.0:8983 2014-08-12 20:30:53.215484 2014-08-13 18:06:06.553557 7f5c973a57a0 20 read_log 194370'15366 (0'0) promote 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by osd.20.0:4376 2014-08-12 20:30:54.587347 2014-08-13 18:06:06.553577 7f5c973a57a0 20 read_log 194370'15367 (194370'15366) modify 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by client.1508531.0:9212 2014-08-12 20:30:54.579269 2014-08-13 18:06:06.553593 7f5c973a57a0 20 read_log 194370'15368 (0'0) promote 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by osd.20.0:4382 2014-08-12 20:30:55.184328 2014-08-13 18:06:06.553605 7f5c973a57a0 20 read_log 194370'15369 (194370'15368) modify 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by client.1508531.0:9311 2014-08-12 20:30:55.173988 2014-08-13 18:06:06.553623 7f5c973a57a0 20 read_log 194370'15370 (0'0) promote e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by osd.20.0:4394 2014-08-12 20:30:59.332921 2014-08-13 18:06:06.553639 7f5c973a57a0 20 read_log 194370'15371 (194370'15370) modify e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by client.1508531.0:10013 2014-08-12 20:30:59.324394 2014-08-13 18:06:06.553657 7f5c973a57a0 20 read_log 194370'15372 (0'0) promote b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by osd.20.0:4422 2014-08-12 20:31:05.215678 2014-08-13 18:06:06.553670 7f5c973a57a0 20 read_log 194370'15373 (194370'15372) modify b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by client.1508531.0:10992 2014-08-12 20:31:05.209034 2014-08-13 18:06:06.553683 7f5c973a57a0 20 read_log 194370'15374 (0'0) promote 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by osd.20.0:4448 2014-08-12 20:31:16.974350 2014-08-13 18:06:06.553695 7f5c973a57a0 20 read_log 194370'15375 (194370'15374) modify 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by client.1508531.0:12747 2014-08-12 20:31:16.966416 2014-08-13 18:06:06.553708 7f5c973a57a0 20 read_log 194370'15376 (0'0) promote 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by osd.20.0:4456 2014-08-12 20:31:18.607649 2014-08-13 18:06:06.553730 7f5c973a57a0 20 read_log 194370'15377 (194370'15376) modify 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by client.1508531.0:12920 2014-08-12 20:31:18.599908 2014-08-13 18:06:06.553745 7f5c973a57a0 20 read_log 194370'15378 (0'0) promote 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by osd.20.0:4466 2014-08-12 20:31:32.161593 2014-08-13 18:06:06.553758 7f5c973a57a0 20 read_log 194370'15379 (194370'15378) modify 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by client.1508531.0:13998 2014-08-12 20:31:32.153489 2014-08-13 18:06:06.553770 7f5c973a57a0 20 read_log 194377'15380 (0'0) promote e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by osd.20.0:4538 2014-08-12 20:33:09.241920 2014-08-13 18:06:06.553790 7f5c973a57a0 20 read_log 194377'15381 (194377'15380) modify e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by client.2231190.0:2972 2014-08-12 20:33:09.231970 2014-08-13 18:06:06.553815 7f5c973a57a0 20 read_log 194377'15382 (0'0) promote a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by osd.20.0:4570 2014-08-12 20:33:17.692054 2014-08-13 18:06:06.553827 7f5c973a57a0 20 read_log 194377'15383 (194377'15382) modify a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by client.2231190.0:4570 2014-08-12 20:33:17.683641 2014-08-13 18:06:06.553840 7f5c973a57a0 20 read_log 194377'15384 (0'0) promote de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by osd.20.0:4578 2014-08-12 20:33:18.942540 2014-08-13 18:06:06.553852 7f5c973a57a0 20 read_log 194377'15385 (194377'15384) modify de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by client.2231190.0:4744 2014-08-12 20:33:18.933995 2014-08-13 18:06:06.553864 7f5c973a57a0 20 read_log 194393'15386 (0'0) promote e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by osd.20.0:4664 2014-08-12 20:49:25.619826 2014-08-13 18:06:06.553876 7f5c973a57a0 20 read_log 194393'15387 (194393'15386) modify e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by client.1064182.0:43 2014-08-12 20:49:25.604653 2014-08-13 18:06:06.553889 7f5c973a57a0 20 read_log 194393'15388 (0'0) promote d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by osd.20.0:4675 2014-08-12 20:49:26.956715 2014-08-13 18:06:06.553901 7f5c973a57a0 20 read_log 194393'15389 (194393'15388) modify d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by client.1064182.0:318 2014-08-12 20:49:26.944410 2014-08-13 18:06:06.553915 7f5c973a57a0 20 read_log 194416'15390 (0'0) promote 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by osd.20.0:4903 2014-08-13 15:28:36.386970 2014-08-13 18:06:06.553927 7f5c973a57a0 20 read_log 194416'15391 (194416'15390) modify 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by client.2441958.0:1494 2014-08-13 15:28:36.375536 2014-08-13 18:06:06.553948 7f5c973a57a0 20 read_log 194416'15392 (0'0) promote 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by osd.20.0:4933 2014-08-13 15:28:39.365980 2014-08-13 18:06:06.553960 7f5c973a57a0 20 read_log 194416'15393 (194416'15392) modify 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by client.2441958.0:2288 2014-08-13 15:28:39.357910 2014-08-13 18:06:06.553973 7f5c973a57a0 20 read_log 194416'15394 (0'0) promote b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by osd.20.0:4981 2014-08-13 15:28:47.372315 2014-08-13 18:06:06.553985 7f5c973a57a0 20 read_log 194416'15395 (194416'15394) modify b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by client.2441958.0:4367 2014-08-13 15:28:47.365054 2014-08-13 18:06:06.554005 7f5c973a57a0 20 read_log 194416'15396 (0'0) promote dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by osd.20.0:4999 2014-08-13 15:28:51.343018 2014-08-13 18:06:06.554020 7f5c973a57a0 20 read_log 194416'15397 (194416'15396) modify dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by client.2441958.0:5380 2014-08-13 15:28:51.335442 2014-08-13 18:06:06.554032 7f5c973a57a0 20 read_log 194416'15398 (0'0) promote a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by osd.20.0:5017 2014-08-13 15:28:54.770300 2014-08-13 18:06:06.554045 7f5c973a57a0 20 read_log 194416'15399 (194416'15398) modify a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by client.2441958.0:6222 2014-08-13 15:28:54.761782 2014-08-13 18:06:06.554057 7f5c973a57a0 20 read_log 194416'15400 (0'0) promote eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by osd.20.0:5067 2014-08-13 15:29:03.748631 2014-08-13 18:06:06.554069 7f5c973a57a0 20 read_log 194416'15401 (194416'15400) modify eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by client.2441958.0:8589 2014-08-13 15:29:03.741402 2014-08-13 18:06:06.554081 7f5c973a57a0 20 read_log 194416'15402 (0'0) promote f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by osd.20.0:5095 2014-08-13 15:29:09.079007 2014-08-13 18:06:06.554095 7f5c973a57a0 20 read_log 194416'15403 (194416'15402) modify f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by client.2441958.0:9913 2014-08-13 15:29:09.071122 2014-08-13 18:06:06.554108 7f5c973a57a0 20 read_log 194416'15404 (0'0) promote f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by osd.20.0:5099 2014-08-13 15:29:09.431879 2014-08-13 18:06:06.554120 7f5c973a57a0 20 read_log 194416'15405 (194416'15404) modify f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by client.2441958.0:10002 2014-08-13 15:29:09.422426 2014-08-13 18:06:06.554132 7f5c973a57a0 20 read_log 194416'15406 (0'0) promote 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by osd.20.0:5105 2014-08-13 15:29:09.945666 2014-08-13 18:06:06.554144 7f5c973a57a0 20 read_log 194416'15407 (194416'15406) modify 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by client.2441958.0:10133 2014-08-13 15:29:09.937904 2014-08-13 18:06:06.554165 7f5c973a57a0 20 read_log 194416'15408 (0'0) promote 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by osd.20.0:5109 2014-08-13 15:29:10.597618 2014-08-13 18:06:06.554177 7f5c973a57a0 20 read_log 194416'15409 (194416'15408) modify 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by client.2441958.0:10279 2014-08-13 15:29:10.586826 2014-08-13 18:06:06.554190 7f5c973a57a0 20 read_log 194416'15410 (0'0) promote db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by osd.20.0:5175 2014-08-13 15:29:23.387230 2014-08-13 18:06:06.554202 7f5c973a57a0 20 read_log 194416'15411 (194416'15410) modify db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by client.2441958.0:13325 2014-08-13 15:29:23.378623 2014-08-13 18:06:06.554214 7f5c973a57a0 20 read_log 194416'15412 (0'0) promote ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by osd.20.0:5179 2014-08-13 15:29:25.662102 2014-08-13 18:06:06.554226 7f5c973a57a0 20 read_log 194416'15413 (194416'15412) modify ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by client.2441958.0:13859 2014-08-13 15:29:25.652448 2014-08-13 18:06:06.554240 7f5c973a57a0 20 read_log 194416'15414 (0'0) promote 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by osd.20.0:5191 2014-08-13 15:29:27.375636 2014-08-13 18:06:06.554252 7f5c973a57a0 20 read_log 194416'15415 (194416'15414) modify 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by client.2441958.0:14303 2014-08-13 15:29:27.368250 2014-08-13 18:06:06.554272 7f5c973a57a0 20 read_log 194416'15416 (0'0) promote 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by osd.20.0:5213 2014-08-13 15:29:31.715845 2014-08-13 18:06:06.554288 7f5c973a57a0 20 read_log 194416'15417 (194416'15416) modify 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by client.2441958.0:15323 2014-08-13 15:29:31.702841 2014-08-13 18:06:06.554300 7f5c973a57a0 20 read_log 194416'15418 (0'0) promote ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by osd.20.0:5217 2014-08-13 15:29:32.635883 2014-08-13 18:06:06.554313 7f5c973a57a0 20 read_log 194416'15419 (194416'15418) modify ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by client.2441958.0:15483 2014-08-13 15:29:32.626899 2014-08-13 18:06:06.554331 7f5c973a57a0 20 read_log 194416'15420 (0'0) promote 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by osd.20.0:5233 2014-08-13 15:29:37.244553 2014-08-13 18:06:06.554347 7f5c973a57a0 20 read_log 194416'15421 (194416'15420) modify 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by client.2441958.0:16262 2014-08-13 15:29:37.233451 2014-08-13 18:06:06.554359 7f5c973a57a0 20 read_log 194416'15422 (0'0) promote 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by osd.20.0:5241 2014-08-13 15:29:45.288896 2014-08-13 18:06:06.554371 7f5c973a57a0 20 read_log 194416'15423 (194416'15422) modify 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by client.2441958.0:16857 2014-08-13 15:29:45.277737 2014-08-13 18:06:06.554390 7f5c973a57a0 20 read_log 194422'15424 (0'0) promote 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by osd.20.0:5327 2014-08-13 15:32:20.502721 2014-08-13 18:06:06.554402 7f5c973a57a0 20 read_log 194422'15425 (194422'15424) modify 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by client.1148203.0:1300 2014-08-13 15:32:20.491615 2014-08-13 18:06:06.554423 7f5c973a57a0 20 read_log 194422'15426 (0'0) promote c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by osd.20.0:5359 2014-08-13 15:32:26.966964 2014-08-13 18:06:06.554442 7f5c973a57a0 20 read_log 194422'15427 (194422'15426) modify c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by client.1148203.0:2965 2014-08-13 15:32:26.958865 2014-08-13 18:06:06.554455 7f5c973a57a0 20 read_log 194422'15428 (0'0) promote 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by osd.20.0:5383 2014-08-13 15:32:31.317855 2014-08-13 18:06:06.554474 7f5c973a57a0 20 read_log 194422'15429 (194422'15428) modify 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by client.1148203.0:4082 2014-08-13 15:32:31.307985 2014-08-13 18:06:06.554490 7f5c973a57a0 20 read_log 194422'15430 (0'0) promote 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by osd.20.0:5393 2014-08-13 15:32:32.844747 2014-08-13 18:06:06.554502 7f5c973a57a0 20 read_log 194422'15431 (194422'15430) modify 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by client.1148203.0:4481 2014-08-13 15:32:32.836750 2014-08-13 18:06:06.554522 7f5c973a57a0 20 read_log 194422'15432 (0'0) promote 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by osd.20.0:5399 2014-08-13 15:32:33.750901 2014-08-13 18:06:06.554536 7f5c973a57a0 20 read_log 194422'15433 (194422'15432) modify 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by client.1148203.0:4714 2014-08-13 15:32:33.742155 2014-08-13 18:06:06.554548 7f5c973a57a0 20 read_log 194422'15434 (0'0) promote 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by osd.20.0:5403 2014-08-13 15:32:34.807661 2014-08-13 18:06:06.554561 7f5c973a57a0 20 read_log 194422'15435 (194422'15434) modify 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by client.1148203.0:4991 2014-08-13 15:32:34.798561 2014-08-13 18:06:06.554583 7f5c973a57a0 20 read_log 194422'15436 (0'0) promote 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by osd.20.0:5449 2014-08-13 15:32:47.898546 2014-08-13 18:06:06.554602 7f5c973a57a0 20 read_log 194422'15437 (194422'15436) modify 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by client.1148203.0:7388 2014-08-13 15:32:47.889207 2014-08-13 18:06:06.554614 7f5c973a57a0 20 read_log 194422'15438 (0'0) promote ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by osd.20.0:5461 2014-08-13 15:32:55.276201 2014-08-13 18:06:06.554627 7f5c973a57a0 20 read_log 194422'15439 (194422'15438) modify ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by client.1148203.0:8523 2014-08-13 15:32:55.268622 2014-08-13 18:06:06.554639 7f5c973a57a0 20 read_log 194422'15440 (0'0) promote 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by osd.20.0:5463 2014-08-13 15:32:55.665770 2014-08-13 18:06:06.554651 7f5c973a57a0 20 read_log 194422'15441 (194422'15440) modify 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by client.1148203.0:8576 2014-08-13 15:32:55.655971 2014-08-13 18:06:06.554663 7f5c973a57a0 20 read_log 194422'15442 (0'0) promote 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by osd.20.0:5481 2014-08-13 15:33:04.337860 2014-08-13 18:06:06.554675 7f5c973a57a0 20 read_log 194422'15443 (194422'15442) modify 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by client.1148203.0:9487 2014-08-13 15:33:04.327492 2014-08-13 18:06:06.554718 7f5c973a57a0 20 read_log 194422'15444 (0'0) promote 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by osd.20.0:5497 2014-08-13 15:33:16.735678 2014-08-13 18:06:06.554734 7f5c973a57a0 20 read_log 194422'15445 (194422'15444) modify 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by client.1148203.0:10239 2014-08-13 15:33:16.727346 2014-08-13 18:06:06.554751 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.555208 7f5c973a57a0 10 read_log checking for missing items over interval (194215'15333,194422'15445] 2014-08-13 18:06:06.555263 7f5c973a57a0 15 read_log missing 194422'15445 (194422'15444) modify 5414582c/benchmark_data_storage0111-ib.pocstor_41231_object10238/head//24 by client.1148203.0:10239 2014-08-13 15:33:16.727346 2014-08-13 18:06:06.555312 7f5c973a57a0 15 read_log missing 194422'15443 (194422'15442) modify 635ac82c/benchmark_data_storage0111-ib.pocstor_41231_object9486/head//24 by client.1148203.0:9487 2014-08-13 15:33:04.327492 2014-08-13 18:06:06.555354 7f5c973a57a0 15 read_log missing 194422'15441 (194422'15440) modify 5ac1bc2c/benchmark_data_storage0111-ib.pocstor_41231_object8575/head//24 by client.1148203.0:8576 2014-08-13 15:32:55.655971 2014-08-13 18:06:06.555390 7f5c973a57a0 15 read_log missing 194422'15439 (194422'15438) modify ef02ac2c/benchmark_data_storage0111-ib.pocstor_41231_object8522/head//24 by client.1148203.0:8523 2014-08-13 15:32:55.268622 2014-08-13 18:06:06.555447 7f5c973a57a0 15 read_log missing 194422'15437 (194422'15436) modify 7d0d442c/benchmark_data_storage0111-ib.pocstor_41231_object7387/head//24 by client.1148203.0:7388 2014-08-13 15:32:47.889207 2014-08-13 18:06:06.555486 7f5c973a57a0 15 read_log missing 194422'15435 (194422'15434) modify 2dd0842c/benchmark_data_storage0111-ib.pocstor_41231_object4990/head//24 by client.1148203.0:4991 2014-08-13 15:32:34.798561 2014-08-13 18:06:06.555522 7f5c973a57a0 15 read_log missing 194422'15433 (194422'15432) modify 96e75c2c/benchmark_data_storage0111-ib.pocstor_41231_object4713/head//24 by client.1148203.0:4714 2014-08-13 15:32:33.742155 2014-08-13 18:06:06.555557 7f5c973a57a0 15 read_log missing 194422'15431 (194422'15430) modify 59aa8c2c/benchmark_data_storage0111-ib.pocstor_41231_object4480/head//24 by client.1148203.0:4481 2014-08-13 15:32:32.836750 2014-08-13 18:06:06.555592 7f5c973a57a0 15 read_log missing 194422'15429 (194422'15428) modify 8b159c2c/benchmark_data_storage0111-ib.pocstor_41231_object4081/head//24 by client.1148203.0:4082 2014-08-13 15:32:31.307985 2014-08-13 18:06:06.555628 7f5c973a57a0 15 read_log missing 194422'15427 (194422'15426) modify c5fea02c/benchmark_data_storage0111-ib.pocstor_41231_object2964/head//24 by client.1148203.0:2965 2014-08-13 15:32:26.958865 2014-08-13 18:06:06.555663 7f5c973a57a0 15 read_log missing 194422'15425 (194422'15424) modify 2ade602c/benchmark_data_storage0111-ib.pocstor_41231_object1299/head//24 by client.1148203.0:1300 2014-08-13 15:32:20.491615 2014-08-13 18:06:06.555699 7f5c973a57a0 15 read_log missing 194416'15423 (194416'15422) modify 9428942c/benchmark_data_storage0111-ib.pocstor_19103_object16856/head//24 by client.2441958.0:16857 2014-08-13 15:29:45.277737 2014-08-13 18:06:06.555754 7f5c973a57a0 15 read_log missing 194416'15421 (194416'15420) modify 9f90c02c/benchmark_data_storage0111-ib.pocstor_19103_object16261/head//24 by client.2441958.0:16262 2014-08-13 15:29:37.233451 2014-08-13 18:06:06.555792 7f5c973a57a0 15 read_log missing 194416'15419 (194416'15418) modify ea22b02c/benchmark_data_storage0111-ib.pocstor_19103_object15482/head//24 by client.2441958.0:15483 2014-08-13 15:29:32.626899 2014-08-13 18:06:06.555827 7f5c973a57a0 15 read_log missing 194416'15417 (194416'15416) modify 9886602c/benchmark_data_storage0111-ib.pocstor_19103_object15322/head//24 by client.2441958.0:15323 2014-08-13 15:29:31.702841 2014-08-13 18:06:06.555863 7f5c973a57a0 15 read_log missing 194416'15415 (194416'15414) modify 5c2ed02c/benchmark_data_storage0111-ib.pocstor_19103_object14302/head//24 by client.2441958.0:14303 2014-08-13 15:29:27.368250 2014-08-13 18:06:06.555898 7f5c973a57a0 15 read_log missing 194416'15413 (194416'15412) modify ce59dc2c/benchmark_data_storage0111-ib.pocstor_19103_object13858/head//24 by client.2441958.0:13859 2014-08-13 15:29:25.652448 2014-08-13 18:06:06.555934 7f5c973a57a0 15 read_log missing 194416'15411 (194416'15410) modify db8fc02c/benchmark_data_storage0111-ib.pocstor_19103_object13324/head//24 by client.2441958.0:13325 2014-08-13 15:29:23.378623 2014-08-13 18:06:06.555971 7f5c973a57a0 15 read_log missing 194416'15409 (194416'15408) modify 76b4e02c/benchmark_data_storage0111-ib.pocstor_19103_object10278/head//24 by client.2441958.0:10279 2014-08-13 15:29:10.586826 2014-08-13 18:06:06.556006 7f5c973a57a0 15 read_log missing 194416'15407 (194416'15406) modify 9ae6c02c/benchmark_data_storage0111-ib.pocstor_19103_object10132/head//24 by client.2441958.0:10133 2014-08-13 15:29:09.937904 2014-08-13 18:06:06.556042 7f5c973a57a0 15 read_log missing 194416'15405 (194416'15404) modify f2996c2c/benchmark_data_storage0111-ib.pocstor_19103_object10001/head//24 by client.2441958.0:10002 2014-08-13 15:29:09.422426 2014-08-13 18:06:06.556078 7f5c973a57a0 15 read_log missing 194416'15403 (194416'15402) modify f622002c/benchmark_data_storage0111-ib.pocstor_19103_object9912/head//24 by client.2441958.0:9913 2014-08-13 15:29:09.071122 2014-08-13 18:06:06.556114 7f5c973a57a0 15 read_log missing 194416'15401 (194416'15400) modify eb1fc82c/benchmark_data_storage0111-ib.pocstor_19103_object8588/head//24 by client.2441958.0:8589 2014-08-13 15:29:03.741402 2014-08-13 18:06:06.556154 7f5c973a57a0 15 read_log missing 194416'15399 (194416'15398) modify a31b282c/benchmark_data_storage0111-ib.pocstor_19103_object6221/head//24 by client.2441958.0:6222 2014-08-13 15:28:54.761782 2014-08-13 18:06:06.556193 7f5c973a57a0 15 read_log missing 194416'15397 (194416'15396) modify dd1ec82c/benchmark_data_storage0111-ib.pocstor_19103_object5379/head//24 by client.2441958.0:5380 2014-08-13 15:28:51.335442 2014-08-13 18:06:06.556229 7f5c973a57a0 15 read_log missing 194416'15395 (194416'15394) modify b73be02c/benchmark_data_storage0111-ib.pocstor_19103_object4366/head//24 by client.2441958.0:4367 2014-08-13 15:28:47.365054 2014-08-13 18:06:06.556266 7f5c973a57a0 15 read_log missing 194416'15393 (194416'15392) modify 74227c2c/benchmark_data_storage0111-ib.pocstor_19103_object2287/head//24 by client.2441958.0:2288 2014-08-13 15:28:39.357910 2014-08-13 18:06:06.556306 7f5c973a57a0 15 read_log missing 194416'15391 (194416'15390) modify 3c693c2c/benchmark_data_storage0111-ib.pocstor_19103_object1493/head//24 by client.2441958.0:1494 2014-08-13 15:28:36.375536 2014-08-13 18:06:06.556347 7f5c973a57a0 15 read_log missing 194393'15389 (194393'15388) modify d1e7b02c/benchmark_data_storage0111-ib.pocstor_32482_object317/head//24 by client.1064182.0:318 2014-08-12 20:49:26.944410 2014-08-13 18:06:06.556384 7f5c973a57a0 15 read_log missing 194393'15387 (194393'15386) modify e128342c/benchmark_data_storage0111-ib.pocstor_32482_object42/head//24 by client.1064182.0:43 2014-08-12 20:49:25.604653 2014-08-13 18:06:06.556434 7f5c973a57a0 15 read_log missing 194377'15385 (194377'15384) modify de2f42c/benchmark_data_storage0111-ib.pocstor_25065_object4743/head//24 by client.2231190.0:4744 2014-08-12 20:33:18.933995 2014-08-13 18:06:06.556480 7f5c973a57a0 15 read_log missing 194377'15383 (194377'15382) modify a3879c2c/benchmark_data_storage0111-ib.pocstor_25065_object4569/head//24 by client.2231190.0:4570 2014-08-12 20:33:17.683641 2014-08-13 18:06:06.556518 7f5c973a57a0 15 read_log missing 194377'15381 (194377'15380) modify e32ce82c/benchmark_data_storage0111-ib.pocstor_25065_object2971/head//24 by client.2231190.0:2972 2014-08-12 20:33:09.231970 2014-08-13 18:06:06.556555 7f5c973a57a0 15 read_log missing 194370'15379 (194370'15378) modify 2c18882c/benchmark_data_storage0111-ib.pocstor_12495_object13997/head//24 by client.1508531.0:13998 2014-08-12 20:31:32.153489 2014-08-13 18:06:06.556591 7f5c973a57a0 15 read_log missing 194370'15377 (194370'15376) modify 4e1ab02c/benchmark_data_storage0111-ib.pocstor_12495_object12919/head//24 by client.1508531.0:12920 2014-08-12 20:31:18.599908 2014-08-13 18:06:06.556627 7f5c973a57a0 15 read_log missing 194370'15375 (194370'15374) modify 90db9c2c/benchmark_data_storage0111-ib.pocstor_12495_object12746/head//24 by client.1508531.0:12747 2014-08-12 20:31:16.966416 2014-08-13 18:06:06.556663 7f5c973a57a0 15 read_log missing 194370'15373 (194370'15372) modify b228642c/benchmark_data_storage0111-ib.pocstor_12495_object10991/head//24 by client.1508531.0:10992 2014-08-12 20:31:05.209034 2014-08-13 18:06:06.556699 7f5c973a57a0 15 read_log missing 194370'15371 (194370'15370) modify e531702c/benchmark_data_storage0111-ib.pocstor_12495_object10012/head//24 by client.1508531.0:10013 2014-08-12 20:30:59.324394 2014-08-13 18:06:06.556735 7f5c973a57a0 15 read_log missing 194370'15369 (194370'15368) modify 28f1c42c/benchmark_data_storage0111-ib.pocstor_12495_object9310/head//24 by client.1508531.0:9311 2014-08-12 20:30:55.173988 2014-08-13 18:06:06.556773 7f5c973a57a0 15 read_log missing 194370'15367 (194370'15366) modify 7886602c/benchmark_data_storage0111-ib.pocstor_12495_object9211/head//24 by client.1508531.0:9212 2014-08-12 20:30:54.579269 2014-08-13 18:06:06.556809 7f5c973a57a0 15 read_log missing 194370'15365 (194370'15364) modify 15d4e82c/benchmark_data_storage0111-ib.pocstor_12495_object8982/head//24 by client.1508531.0:8983 2014-08-12 20:30:53.215484 2014-08-13 18:06:06.556845 7f5c973a57a0 15 read_log missing 194370'15363 (194370'15362) modify b505302c/benchmark_data_storage0111-ib.pocstor_12495_object7189/head//24 by client.1508531.0:7190 2014-08-12 20:30:43.308490 2014-08-13 18:06:06.556899 7f5c973a57a0 15 read_log missing 194370'15361 (194370'15360) modify 7f37fc2c/benchmark_data_storage0111-ib.pocstor_12495_object7155/head//24 by client.1508531.0:7156 2014-08-12 20:30:43.142664 2014-08-13 18:06:06.556936 7f5c973a57a0 15 read_log missing 194370'15359 (194370'15358) modify 4c8dc42c/benchmark_data_storage0111-ib.pocstor_12495_object6637/head//24 by client.1508531.0:6638 2014-08-12 20:30:40.590468 2014-08-13 18:06:06.556972 7f5c973a57a0 15 read_log missing 194370'15357 (194370'15356) modify f893f82c/benchmark_data_storage0111-ib.pocstor_12495_object6015/head//24 by client.1508531.0:6016 2014-08-12 20:30:37.790483 2014-08-13 18:06:06.557008 7f5c973a57a0 15 read_log missing 194370'15355 (194370'15354) modify b187402c/benchmark_data_storage0111-ib.pocstor_12495_object5445/head//24 by client.1508531.0:5446 2014-08-12 20:30:35.162206 2014-08-13 18:06:06.557045 7f5c973a57a0 15 read_log missing 194370'15353 (194370'15352) modify 74e7782c/benchmark_data_storage0111-ib.pocstor_12495_object4357/head//24 by client.1508531.0:4358 2014-08-12 20:30:30.658820 2014-08-13 18:06:06.557081 7f5c973a57a0 15 read_log missing 194370'15351 (194370'15350) modify a09b02c/benchmark_data_storage0111-ib.pocstor_12495_object3841/head//24 by client.1508531.0:3842 2014-08-12 20:30:28.710059 2014-08-13 18:06:06.557118 7f5c973a57a0 15 read_log missing 194370'15349 (194370'15348) modify 9f47982c/benchmark_data_storage0111-ib.pocstor_12495_object2701/head//24 by client.1508531.0:2702 2014-08-12 20:30:24.456839 2014-08-13 18:06:06.557159 7f5c973a57a0 15 read_log missing 194370'15347 (194370'15346) modify ba4da42c/benchmark_data_storage0111-ib.pocstor_12495_object2582/head//24 by client.1508531.0:2583 2014-08-12 20:30:24.026611 2014-08-13 18:06:06.557196 7f5c973a57a0 15 read_log missing 194370'15345 (194370'15344) modify 8895602c/benchmark_data_storage0111-ib.pocstor_12495_object2032/head//24 by client.1508531.0:2033 2014-08-12 20:30:22.032396 2014-08-13 18:06:06.557232 7f5c973a57a0 15 read_log missing 194370'15343 (194370'15342) modify bf0e442c/benchmark_data_storage0111-ib.pocstor_12495_object1570/head//24 by client.1508531.0:1571 2014-08-12 20:30:20.269125 2014-08-13 18:06:06.557268 7f5c973a57a0 15 read_log missing 194370'15341 (194370'15340) modify e384e02c/benchmark_data_storage0111-ib.pocstor_12495_object1372/head//24 by client.1508531.0:1373 2014-08-12 20:30:19.528413 2014-08-13 18:06:06.557310 7f5c973a57a0 15 read_log missing 194217'15339 (194217'15338) modify a8a3482c/benchmark_data_storage0111-ib.pocstor_39693_object145/head//24 by client.1487927.0:146 2014-08-12 17:25:03.864452 2014-08-13 18:06:06.557347 7f5c973a57a0 15 read_log missing 194217'15337 (194217'15336) modify 547bac2c/benchmark_data_storage0111-ib.pocstor_39693_object101/head//24 by client.1487927.0:102 2014-08-12 17:25:03.710273 2014-08-13 18:06:06.557405 7f5c973a57a0 15 read_log missing 194217'15334 (194217'15334) modify 2c/hit_set_24.2c_archive_2014-08-12 15:32:25.737566_2014-08-12 17:25:03.718423/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.718675 2014-08-13 18:06:06.557422 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.557463 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 18:06:06.557482 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.016233 0 0.000000 2014-08-13 18:06:06.557495 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 18:06:06.557504 7f5c973a57a0 20 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=0 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194470 2014-08-13 18:06:06.557513 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] log((192718'12964,194422'15445], crt=194422'15445) 2014-08-13 18:06:06.557524 7f5c973a57a0 10 osd.4 194487 pgid 24.34 coll 24.34_head 2014-08-13 18:06:06.557710 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.34 2014-08-13 18:06:06.557724 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.557763 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.34(unlocked)] enter Initial 2014-08-13 18:06:06.557784 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.34(unlocked)] enter NotTrimming 2014-08-13 18:06:06.558020 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.558295 7f5c973a57a0 20 read_log 192718'11511 (192718'11510) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:76 2014-08-06 14:53:21.371745 2014-08-13 18:06:06.558317 7f5c973a57a0 20 read_log 192718'11512 (192718'11511) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:77 2014-08-06 14:53:21.398763 2014-08-13 18:06:06.558334 7f5c973a57a0 20 read_log 192718'11513 (192718'11512) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:78 2014-08-06 14:53:21.425979 2014-08-13 18:06:06.558349 7f5c973a57a0 20 read_log 192718'11514 (192718'11513) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:79 2014-08-06 14:53:21.454713 2014-08-13 18:06:06.558361 7f5c973a57a0 20 read_log 192718'11515 (192718'11514) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:80 2014-08-06 14:53:21.485370 2014-08-13 18:06:06.558373 7f5c973a57a0 20 read_log 192718'11516 (192718'11515) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:81 2014-08-06 14:53:21.513935 2014-08-13 18:06:06.558385 7f5c973a57a0 20 read_log 192718'11517 (192718'11516) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:82 2014-08-06 14:53:21.542619 2014-08-13 18:06:06.558397 7f5c973a57a0 20 read_log 192718'11518 (192718'11517) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:83 2014-08-06 14:53:21.570695 2014-08-13 18:06:06.558410 7f5c973a57a0 20 read_log 192718'11519 (192718'11518) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:84 2014-08-06 14:53:21.601072 2014-08-13 18:06:06.558432 7f5c973a57a0 20 read_log 192718'11520 (192718'11519) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:85 2014-08-06 14:53:21.631007 2014-08-13 18:06:06.558447 7f5c973a57a0 20 read_log 192718'11521 (192718'11520) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:86 2014-08-06 14:53:21.661151 2014-08-13 18:06:06.558459 7f5c973a57a0 20 read_log 192718'11522 (192718'11521) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:87 2014-08-06 14:53:21.690380 2014-08-13 18:06:06.558472 7f5c973a57a0 20 read_log 192718'11523 (192718'11522) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:88 2014-08-06 14:53:21.717771 2014-08-13 18:06:06.558484 7f5c973a57a0 20 read_log 192718'11524 (192718'11523) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:89 2014-08-06 14:53:21.745457 2014-08-13 18:06:06.558496 7f5c973a57a0 20 read_log 192718'11525 (192718'11524) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:90 2014-08-06 14:53:21.772985 2014-08-13 18:06:06.558508 7f5c973a57a0 20 read_log 192718'11526 (192718'11525) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:91 2014-08-06 14:53:21.801243 2014-08-13 18:06:06.558520 7f5c973a57a0 20 read_log 192718'11527 (192718'11526) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:92 2014-08-06 14:53:21.830745 2014-08-13 18:06:06.558532 7f5c973a57a0 20 read_log 192718'11528 (192718'11527) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:93 2014-08-06 14:53:21.857999 2014-08-13 18:06:06.558544 7f5c973a57a0 20 read_log 192718'11529 (192718'11528) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:94 2014-08-06 14:53:21.886302 2014-08-13 18:06:06.558556 7f5c973a57a0 20 read_log 192718'11530 (192718'11529) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:95 2014-08-06 14:53:21.913830 2014-08-13 18:06:06.558569 7f5c973a57a0 20 read_log 192718'11531 (192718'11530) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:96 2014-08-06 14:53:22.391659 2014-08-13 18:06:06.558581 7f5c973a57a0 20 read_log 192718'11532 (192718'11531) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:97 2014-08-06 14:53:22.419199 2014-08-13 18:06:06.558593 7f5c973a57a0 20 read_log 192718'11533 (192718'11532) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:98 2014-08-06 14:53:22.931082 2014-08-13 18:06:06.558605 7f5c973a57a0 20 read_log 192718'11534 (192718'11533) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:99 2014-08-06 14:53:22.961517 2014-08-13 18:06:06.558617 7f5c973a57a0 20 read_log 192718'11535 (192718'11534) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:100 2014-08-06 14:53:22.989073 2014-08-13 18:06:06.558629 7f5c973a57a0 20 read_log 192718'11536 (192718'11535) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:101 2014-08-06 14:53:23.014038 2014-08-13 18:06:06.558641 7f5c973a57a0 20 read_log 192718'11537 (192718'11536) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:102 2014-08-06 14:53:23.346989 2014-08-13 18:06:06.558654 7f5c973a57a0 20 read_log 192718'11538 (192718'11537) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:103 2014-08-06 14:53:23.374668 2014-08-13 18:06:06.558666 7f5c973a57a0 20 read_log 192718'11539 (192718'11538) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:104 2014-08-06 14:53:23.402932 2014-08-13 18:06:06.558678 7f5c973a57a0 20 read_log 192718'11540 (192718'11539) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:105 2014-08-06 14:53:23.428134 2014-08-13 18:06:06.558690 7f5c973a57a0 20 read_log 192718'11541 (192718'11540) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:106 2014-08-06 14:53:23.456545 2014-08-13 18:06:06.558702 7f5c973a57a0 20 read_log 192718'11542 (192718'11541) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:107 2014-08-06 14:53:23.488256 2014-08-13 18:06:06.558714 7f5c973a57a0 20 read_log 192718'11543 (192718'11542) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:108 2014-08-06 14:53:23.516400 2014-08-13 18:06:06.558733 7f5c973a57a0 20 read_log 192718'11544 (192718'11543) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:109 2014-08-06 14:53:23.545548 2014-08-13 18:06:06.558746 7f5c973a57a0 20 read_log 192718'11545 (192718'11544) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:110 2014-08-06 14:53:23.938902 2014-08-13 18:06:06.558760 7f5c973a57a0 20 read_log 192718'11546 (192718'11545) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:111 2014-08-06 14:53:23.963656 2014-08-13 18:06:06.558774 7f5c973a57a0 20 read_log 192718'11547 (192718'11546) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:112 2014-08-06 14:53:23.986963 2014-08-13 18:06:06.558786 7f5c973a57a0 20 read_log 192718'11548 (192718'11547) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:113 2014-08-06 14:53:24.009786 2014-08-13 18:06:06.558799 7f5c973a57a0 20 read_log 192718'11549 (192718'11548) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:114 2014-08-06 14:53:24.036954 2014-08-13 18:06:06.558811 7f5c973a57a0 20 read_log 192718'11550 (192718'11549) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:115 2014-08-06 14:53:24.064570 2014-08-13 18:06:06.558823 7f5c973a57a0 20 read_log 192718'11551 (192718'11550) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:116 2014-08-06 14:53:24.091849 2014-08-13 18:06:06.558838 7f5c973a57a0 20 read_log 192718'11552 (192718'11551) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:117 2014-08-06 14:53:24.119437 2014-08-13 18:06:06.558850 7f5c973a57a0 20 read_log 192718'11553 (192718'11552) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:118 2014-08-06 14:53:24.147002 2014-08-13 18:06:06.558864 7f5c973a57a0 20 read_log 192718'11554 (192718'11553) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:119 2014-08-06 14:53:24.173819 2014-08-13 18:06:06.558876 7f5c973a57a0 20 read_log 192718'11555 (192718'11554) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:120 2014-08-06 14:53:24.201400 2014-08-13 18:06:06.558888 7f5c973a57a0 20 read_log 192718'11556 (192718'11555) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:121 2014-08-06 14:53:24.229356 2014-08-13 18:06:06.558902 7f5c973a57a0 20 read_log 192718'11557 (192718'11556) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:122 2014-08-06 14:53:24.259697 2014-08-13 18:06:06.558914 7f5c973a57a0 20 read_log 192718'11558 (192718'11557) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:123 2014-08-06 14:53:24.292136 2014-08-13 18:06:06.558927 7f5c973a57a0 20 read_log 192718'11559 (192718'11558) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:124 2014-08-06 14:53:24.317970 2014-08-13 18:06:06.558941 7f5c973a57a0 20 read_log 192718'11560 (192718'11559) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:125 2014-08-06 14:53:24.343984 2014-08-13 18:06:06.558953 7f5c973a57a0 20 read_log 192718'11561 (192718'11560) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:126 2014-08-06 14:53:24.835101 2014-08-13 18:06:06.558965 7f5c973a57a0 20 read_log 192718'11562 (192718'11561) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:127 2014-08-06 14:53:24.860893 2014-08-13 18:06:06.558979 7f5c973a57a0 20 read_log 192718'11563 (192718'11562) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:128 2014-08-06 14:53:24.883734 2014-08-13 18:06:06.558991 7f5c973a57a0 20 read_log 192718'11564 (192718'11563) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:129 2014-08-06 14:53:24.905971 2014-08-13 18:06:06.559005 7f5c973a57a0 20 read_log 192718'11565 (192718'11564) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:130 2014-08-06 14:53:24.932079 2014-08-13 18:06:06.559018 7f5c973a57a0 20 read_log 192718'11566 (192718'11565) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:131 2014-08-06 14:53:24.959006 2014-08-13 18:06:06.559032 7f5c973a57a0 20 read_log 192718'11567 (192718'11566) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:132 2014-08-06 14:53:24.986719 2014-08-13 18:06:06.559052 7f5c973a57a0 20 read_log 192718'11568 (192718'11567) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:133 2014-08-06 14:53:25.012277 2014-08-13 18:06:06.559066 7f5c973a57a0 20 read_log 192718'11569 (192718'11568) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:134 2014-08-06 14:53:25.039512 2014-08-13 18:06:06.559079 7f5c973a57a0 20 read_log 192718'11570 (192718'11569) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:135 2014-08-06 14:53:25.065389 2014-08-13 18:06:06.559093 7f5c973a57a0 20 read_log 192718'11571 (192718'11570) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:136 2014-08-06 14:53:25.092805 2014-08-13 18:06:06.559107 7f5c973a57a0 20 read_log 192718'11572 (192718'11571) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:137 2014-08-06 14:53:25.119376 2014-08-13 18:06:06.559119 7f5c973a57a0 20 read_log 192718'11573 (192718'11572) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:138 2014-08-06 14:53:25.146917 2014-08-13 18:06:06.559132 7f5c973a57a0 20 read_log 192718'11574 (192718'11573) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:139 2014-08-06 14:53:25.173320 2014-08-13 18:06:06.559144 7f5c973a57a0 20 read_log 192718'11575 (192718'11574) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:140 2014-08-06 14:53:25.202479 2014-08-13 18:06:06.559168 7f5c973a57a0 20 read_log 192718'11576 (192718'11575) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:141 2014-08-06 14:53:25.240628 2014-08-13 18:06:06.559181 7f5c973a57a0 20 read_log 192718'11577 (192718'11576) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:142 2014-08-06 14:53:25.283176 2014-08-13 18:06:06.559193 7f5c973a57a0 20 read_log 192718'11578 (192718'11577) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:143 2014-08-06 14:53:25.328199 2014-08-13 18:06:06.559205 7f5c973a57a0 20 read_log 192718'11579 (192718'11578) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:144 2014-08-06 14:53:25.373374 2014-08-13 18:06:06.559219 7f5c973a57a0 20 read_log 192718'11580 (192718'11579) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:145 2014-08-06 14:53:25.417791 2014-08-13 18:06:06.559231 7f5c973a57a0 20 read_log 192718'11581 (192718'11580) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:146 2014-08-06 14:53:25.468000 2014-08-13 18:06:06.559245 7f5c973a57a0 20 read_log 192718'11582 (192718'11581) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:147 2014-08-06 14:53:25.719433 2014-08-13 18:06:06.559261 7f5c973a57a0 20 read_log 192718'11583 (192718'11582) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:148 2014-08-06 14:53:25.775296 2014-08-13 18:06:06.559275 7f5c973a57a0 20 read_log 192718'11584 (192718'11583) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:149 2014-08-06 14:53:25.828385 2014-08-13 18:06:06.559288 7f5c973a57a0 20 read_log 192718'11585 (192718'11584) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:150 2014-08-06 14:53:25.884760 2014-08-13 18:06:06.559300 7f5c973a57a0 20 read_log 192718'11586 (192718'11585) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:151 2014-08-06 14:53:25.931886 2014-08-13 18:06:06.559316 7f5c973a57a0 20 read_log 192718'11587 (192718'11586) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:152 2014-08-06 14:53:25.981505 2014-08-13 18:06:06.559329 7f5c973a57a0 20 read_log 192718'11588 (192718'11587) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:153 2014-08-06 14:53:26.031616 2014-08-13 18:06:06.559342 7f5c973a57a0 20 read_log 192718'11589 (192718'11588) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:154 2014-08-06 14:53:26.093416 2014-08-13 18:06:06.559357 7f5c973a57a0 20 read_log 192718'11590 (192718'11589) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:155 2014-08-06 14:53:26.175936 2014-08-13 18:06:06.559372 7f5c973a57a0 20 read_log 192718'11591 (192718'11590) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:156 2014-08-06 14:53:26.250331 2014-08-13 18:06:06.559407 7f5c973a57a0 20 read_log 192718'11592 (192718'11591) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:157 2014-08-06 14:53:26.491009 2014-08-13 18:06:06.559424 7f5c973a57a0 20 read_log 192718'11593 (192718'11592) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:158 2014-08-06 14:53:26.962507 2014-08-13 18:06:06.559444 7f5c973a57a0 20 read_log 192718'11594 (192718'11593) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:159 2014-08-06 14:53:26.991441 2014-08-13 18:06:06.559460 7f5c973a57a0 20 read_log 192718'11595 (192718'11594) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:160 2014-08-06 14:53:27.021332 2014-08-13 18:06:06.559472 7f5c973a57a0 20 read_log 192718'11596 (192718'11595) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:161 2014-08-06 14:53:27.047224 2014-08-13 18:06:06.559505 7f5c973a57a0 20 read_log 192718'11597 (192718'11596) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:162 2014-08-06 14:53:27.077768 2014-08-13 18:06:06.559521 7f5c973a57a0 20 read_log 192718'11598 (192718'11597) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:163 2014-08-06 14:53:27.107026 2014-08-13 18:06:06.559535 7f5c973a57a0 20 read_log 192718'11599 (192718'11598) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:164 2014-08-06 14:53:27.136121 2014-08-13 18:06:06.559548 7f5c973a57a0 20 read_log 192718'11600 (192718'11599) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:165 2014-08-06 14:53:27.164664 2014-08-13 18:06:06.559560 7f5c973a57a0 20 read_log 192718'11601 (192718'11600) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:166 2014-08-06 14:53:27.302901 2014-08-13 18:06:06.559573 7f5c973a57a0 20 read_log 192718'11602 (192718'11601) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:167 2014-08-06 14:53:27.412708 2014-08-13 18:06:06.559588 7f5c973a57a0 20 read_log 192718'11603 (192718'11602) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:168 2014-08-06 14:53:27.452649 2014-08-13 18:06:06.559602 7f5c973a57a0 20 read_log 192718'11604 (192718'11603) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:169 2014-08-06 14:53:27.500193 2014-08-13 18:06:06.559633 7f5c973a57a0 20 read_log 192718'11605 (192718'11604) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:170 2014-08-06 14:53:27.527560 2014-08-13 18:06:06.559649 7f5c973a57a0 20 read_log 192718'11606 (192718'11605) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:171 2014-08-06 14:53:27.554913 2014-08-13 18:06:06.559661 7f5c973a57a0 20 read_log 192718'11607 (192718'11606) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:172 2014-08-06 14:53:27.581615 2014-08-13 18:06:06.559675 7f5c973a57a0 20 read_log 192718'11608 (192718'11607) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:173 2014-08-06 14:53:27.609010 2014-08-13 18:06:06.559687 7f5c973a57a0 20 read_log 192718'11609 (192718'11608) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:174 2014-08-06 14:53:27.643073 2014-08-13 18:06:06.559703 7f5c973a57a0 20 read_log 192718'11610 (192718'11609) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:175 2014-08-06 14:53:27.669013 2014-08-13 18:06:06.559716 7f5c973a57a0 20 read_log 192718'11611 (192718'11610) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:176 2014-08-06 14:53:27.697754 2014-08-13 18:06:06.559728 7f5c973a57a0 20 read_log 192718'11612 (192718'11611) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:177 2014-08-06 14:53:27.724639 2014-08-13 18:06:06.559752 7f5c973a57a0 20 read_log 192718'11613 (192718'11612) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:178 2014-08-06 14:53:27.753862 2014-08-13 18:06:06.559765 7f5c973a57a0 20 read_log 192718'11614 (192718'11613) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:179 2014-08-06 14:53:28.082822 2014-08-13 18:06:06.559777 7f5c973a57a0 20 read_log 192718'11615 (192718'11614) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:180 2014-08-06 14:53:28.110588 2014-08-13 18:06:06.559799 7f5c973a57a0 20 read_log 192718'11616 (192718'11615) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:181 2014-08-06 14:53:28.136725 2014-08-13 18:06:06.559814 7f5c973a57a0 20 read_log 192718'11617 (192718'11616) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:182 2014-08-06 14:53:28.164101 2014-08-13 18:06:06.559827 7f5c973a57a0 20 read_log 192718'11618 (192718'11617) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:183 2014-08-06 14:53:28.189933 2014-08-13 18:06:06.559839 7f5c973a57a0 20 read_log 192718'11619 (192718'11618) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:184 2014-08-06 14:53:28.216291 2014-08-13 18:06:06.559851 7f5c973a57a0 20 read_log 192718'11620 (192718'11619) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:185 2014-08-06 14:53:28.241924 2014-08-13 18:06:06.559863 7f5c973a57a0 20 read_log 192718'11621 (192718'11620) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:186 2014-08-06 14:53:28.268270 2014-08-13 18:06:06.559876 7f5c973a57a0 20 read_log 192718'11622 (192718'11621) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:187 2014-08-06 14:53:28.294466 2014-08-13 18:06:06.559889 7f5c973a57a0 20 read_log 192718'11623 (192718'11622) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:188 2014-08-06 14:53:28.320210 2014-08-13 18:06:06.559902 7f5c973a57a0 20 read_log 192718'11624 (192718'11623) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:189 2014-08-06 14:53:28.483523 2014-08-13 18:06:06.559914 7f5c973a57a0 20 read_log 192718'11625 (192718'11624) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:190 2014-08-06 14:53:28.525776 2014-08-13 18:06:06.559926 7f5c973a57a0 20 read_log 192718'11626 (192718'11625) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:191 2014-08-06 14:53:28.571402 2014-08-13 18:06:06.559944 7f5c973a57a0 20 read_log 192718'11627 (192718'11626) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:192 2014-08-06 14:53:28.625285 2014-08-13 18:06:06.559957 7f5c973a57a0 20 read_log 192718'11628 (192718'11627) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:193 2014-08-06 14:53:28.684016 2014-08-13 18:06:06.559974 7f5c973a57a0 20 read_log 192718'11629 (192718'11628) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:194 2014-08-06 14:53:28.740778 2014-08-13 18:06:06.559987 7f5c973a57a0 20 read_log 192718'11630 (192718'11629) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:195 2014-08-06 14:53:28.784731 2014-08-13 18:06:06.560001 7f5c973a57a0 20 read_log 192718'11631 (192718'11630) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:196 2014-08-06 14:53:28.823441 2014-08-13 18:06:06.560014 7f5c973a57a0 20 read_log 192718'11632 (192718'11631) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:197 2014-08-06 14:53:28.894562 2014-08-13 18:06:06.560028 7f5c973a57a0 20 read_log 192718'11633 (192718'11632) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:198 2014-08-06 14:53:28.958436 2014-08-13 18:06:06.560040 7f5c973a57a0 20 read_log 192718'11634 (192718'11633) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:199 2014-08-06 14:53:29.352032 2014-08-13 18:06:06.560053 7f5c973a57a0 20 read_log 192718'11635 (192718'11634) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:200 2014-08-06 14:53:29.430234 2014-08-13 18:06:06.560068 7f5c973a57a0 20 read_log 192718'11636 (192718'11635) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:201 2014-08-06 14:53:29.497991 2014-08-13 18:06:06.560080 7f5c973a57a0 20 read_log 192718'11637 (192718'11636) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:202 2014-08-06 14:53:29.581245 2014-08-13 18:06:06.560094 7f5c973a57a0 20 read_log 192718'11638 (192718'11637) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:203 2014-08-06 14:53:29.642221 2014-08-13 18:06:06.560107 7f5c973a57a0 20 read_log 192718'11639 (192718'11638) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:204 2014-08-06 14:53:29.679868 2014-08-13 18:06:06.560155 7f5c973a57a0 20 read_log 192718'11640 (192718'11639) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:205 2014-08-06 14:53:29.719016 2014-08-13 18:06:06.560172 7f5c973a57a0 20 read_log 192718'11641 (192718'11640) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:206 2014-08-06 14:53:29.771344 2014-08-13 18:06:06.560184 7f5c973a57a0 20 read_log 192718'11642 (192718'11641) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:207 2014-08-06 14:53:29.820431 2014-08-13 18:06:06.560196 7f5c973a57a0 20 read_log 192718'11643 (192718'11642) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:208 2014-08-06 14:53:29.854351 2014-08-13 18:06:06.560209 7f5c973a57a0 20 read_log 192718'11644 (192718'11643) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:209 2014-08-06 14:53:30.245018 2014-08-13 18:06:06.560221 7f5c973a57a0 20 read_log 192718'11645 (192718'11644) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:210 2014-08-06 14:53:30.296335 2014-08-13 18:06:06.560233 7f5c973a57a0 20 read_log 192718'11646 (192718'11645) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:211 2014-08-06 14:53:30.342482 2014-08-13 18:06:06.560249 7f5c973a57a0 20 read_log 192718'11647 (192718'11646) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:212 2014-08-06 14:53:30.388593 2014-08-13 18:06:06.560264 7f5c973a57a0 20 read_log 192718'11648 (192718'11647) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:213 2014-08-06 14:53:30.436047 2014-08-13 18:06:06.560276 7f5c973a57a0 20 read_log 192718'11649 (192718'11648) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:214 2014-08-06 14:53:30.482345 2014-08-13 18:06:06.560288 7f5c973a57a0 20 read_log 192718'11650 (192718'11649) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:215 2014-08-06 14:53:30.532798 2014-08-13 18:06:06.560300 7f5c973a57a0 20 read_log 192718'11651 (192718'11650) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:216 2014-08-06 14:53:30.595892 2014-08-13 18:06:06.560312 7f5c973a57a0 20 read_log 192718'11652 (192718'11651) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:217 2014-08-06 14:53:30.652734 2014-08-13 18:06:06.560324 7f5c973a57a0 20 read_log 192718'11653 (192718'11652) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:218 2014-08-06 14:53:30.710371 2014-08-13 18:06:06.560336 7f5c973a57a0 20 read_log 192718'11654 (192718'11653) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:219 2014-08-06 14:53:31.015504 2014-08-13 18:06:06.560349 7f5c973a57a0 20 read_log 192718'11655 (192718'11654) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:220 2014-08-06 14:53:31.077847 2014-08-13 18:06:06.560362 7f5c973a57a0 20 read_log 192718'11656 (192718'11655) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:221 2014-08-06 14:53:31.129612 2014-08-13 18:06:06.560374 7f5c973a57a0 20 read_log 192718'11657 (192718'11656) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:222 2014-08-06 14:53:31.750931 2014-08-13 18:06:06.560386 7f5c973a57a0 20 read_log 192718'11658 (192718'11657) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:223 2014-08-06 14:53:31.778106 2014-08-13 18:06:06.560417 7f5c973a57a0 20 read_log 192718'11659 (192718'11658) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:224 2014-08-06 14:53:31.802553 2014-08-13 18:06:06.560433 7f5c973a57a0 20 read_log 192718'11660 (192718'11659) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:225 2014-08-06 14:53:31.829562 2014-08-13 18:06:06.560459 7f5c973a57a0 20 read_log 192718'11661 (192718'11660) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:226 2014-08-06 14:53:31.857361 2014-08-13 18:06:06.560475 7f5c973a57a0 20 read_log 192718'11662 (192718'11661) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:227 2014-08-06 14:53:31.882503 2014-08-13 18:06:06.560487 7f5c973a57a0 20 read_log 192718'11663 (192718'11662) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:228 2014-08-06 14:53:31.908271 2014-08-13 18:06:06.560514 7f5c973a57a0 20 read_log 192718'11664 (192718'11663) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:229 2014-08-06 14:53:32.109759 2014-08-13 18:06:06.560529 7f5c973a57a0 20 read_log 192718'11665 (192718'11664) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:230 2014-08-06 14:53:32.164501 2014-08-13 18:06:06.560550 7f5c973a57a0 20 read_log 192718'11666 (192718'11665) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:231 2014-08-06 14:53:32.222681 2014-08-13 18:06:06.560566 7f5c973a57a0 20 read_log 192718'11667 (192718'11666) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:232 2014-08-06 14:53:32.272173 2014-08-13 18:06:06.560582 7f5c973a57a0 20 read_log 192718'11668 (192718'11667) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:233 2014-08-06 14:53:32.314488 2014-08-13 18:06:06.560595 7f5c973a57a0 20 read_log 192718'11669 (192718'11668) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:234 2014-08-06 14:53:32.347183 2014-08-13 18:06:06.560608 7f5c973a57a0 20 read_log 192718'11670 (192718'11669) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:235 2014-08-06 14:53:32.373319 2014-08-13 18:06:06.560620 7f5c973a57a0 20 read_log 192718'11671 (192718'11670) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:236 2014-08-06 14:53:32.400125 2014-08-13 18:06:06.560645 7f5c973a57a0 20 read_log 192718'11672 (192718'11671) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:237 2014-08-06 14:53:32.428469 2014-08-13 18:06:06.560657 7f5c973a57a0 20 read_log 192718'11673 (192718'11672) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:238 2014-08-06 14:53:32.454484 2014-08-13 18:06:06.560670 7f5c973a57a0 20 read_log 192718'11674 (192718'11673) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:239 2014-08-06 14:53:32.707937 2014-08-13 18:06:06.560682 7f5c973a57a0 20 read_log 192718'11675 (192718'11674) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:240 2014-08-06 14:53:32.763339 2014-08-13 18:06:06.560694 7f5c973a57a0 20 read_log 192718'11676 (192718'11675) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:241 2014-08-06 14:53:32.813403 2014-08-13 18:06:06.560707 7f5c973a57a0 20 read_log 192718'11677 (192718'11676) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:242 2014-08-06 14:53:32.873666 2014-08-13 18:06:06.560719 7f5c973a57a0 20 read_log 192718'11678 (192718'11677) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:243 2014-08-06 14:53:32.931520 2014-08-13 18:06:06.560731 7f5c973a57a0 20 read_log 192718'11679 (192718'11678) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:244 2014-08-06 14:53:32.958435 2014-08-13 18:06:06.560743 7f5c973a57a0 20 read_log 192718'11680 (192718'11679) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:245 2014-08-06 14:53:33.001428 2014-08-13 18:06:06.560765 7f5c973a57a0 20 read_log 192718'11681 (192718'11680) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:246 2014-08-06 14:53:33.046407 2014-08-13 18:06:06.560778 7f5c973a57a0 20 read_log 192718'11682 (192718'11681) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:247 2014-08-06 14:53:33.087721 2014-08-13 18:06:06.560791 7f5c973a57a0 20 read_log 192718'11683 (192718'11682) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:248 2014-08-06 14:53:33.131679 2014-08-13 18:06:06.560803 7f5c973a57a0 20 read_log 192718'11684 (192718'11683) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:249 2014-08-06 14:53:33.172441 2014-08-13 18:06:06.560815 7f5c973a57a0 20 read_log 192718'11685 (192718'11684) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:250 2014-08-06 14:53:33.211658 2014-08-13 18:06:06.560827 7f5c973a57a0 20 read_log 192718'11686 (192718'11685) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:251 2014-08-06 14:53:33.459391 2014-08-13 18:06:06.560839 7f5c973a57a0 20 read_log 192718'11687 (192718'11686) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:252 2014-08-06 14:53:33.505091 2014-08-13 18:06:06.560861 7f5c973a57a0 20 read_log 192718'11688 (192718'11687) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:253 2014-08-06 14:53:33.533773 2014-08-13 18:06:06.560895 7f5c973a57a0 20 read_log 192718'11689 (192718'11688) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:254 2014-08-06 14:53:33.560693 2014-08-13 18:06:06.560910 7f5c973a57a0 20 read_log 192718'11690 (192718'11689) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:255 2014-08-06 14:53:33.586282 2014-08-13 18:06:06.560923 7f5c973a57a0 20 read_log 192718'11691 (192718'11690) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:256 2014-08-06 14:53:33.612985 2014-08-13 18:06:06.560935 7f5c973a57a0 20 read_log 192718'11692 (192718'11691) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:257 2014-08-06 14:53:33.639662 2014-08-13 18:06:06.560949 7f5c973a57a0 20 read_log 192718'11693 (192718'11692) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:258 2014-08-06 14:53:33.666363 2014-08-13 18:06:06.560962 7f5c973a57a0 20 read_log 192718'11694 (192718'11693) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:259 2014-08-06 14:53:33.695155 2014-08-13 18:06:06.560976 7f5c973a57a0 20 read_log 192718'11695 (192718'11694) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:260 2014-08-06 14:53:33.721666 2014-08-13 18:06:06.560988 7f5c973a57a0 20 read_log 192718'11696 (192718'11695) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:261 2014-08-06 14:53:34.174895 2014-08-13 18:06:06.561000 7f5c973a57a0 20 read_log 192718'11697 (192718'11696) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:262 2014-08-06 14:53:34.222590 2014-08-13 18:06:06.561013 7f5c973a57a0 20 read_log 192718'11698 (192718'11697) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:263 2014-08-06 14:53:34.266140 2014-08-13 18:06:06.561025 7f5c973a57a0 20 read_log 192718'11699 (192718'11698) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:264 2014-08-06 14:53:34.310235 2014-08-13 18:06:06.561037 7f5c973a57a0 20 read_log 192718'11700 (192718'11699) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:265 2014-08-06 14:53:34.353251 2014-08-13 18:06:06.561049 7f5c973a57a0 20 read_log 192718'11701 (192718'11700) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:266 2014-08-06 14:53:34.397088 2014-08-13 18:06:06.561063 7f5c973a57a0 20 read_log 192718'11702 (192718'11701) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:267 2014-08-06 14:53:34.440079 2014-08-13 18:06:06.561076 7f5c973a57a0 20 read_log 192718'11703 (192718'11702) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:268 2014-08-06 14:53:34.483607 2014-08-13 18:06:06.561090 7f5c973a57a0 20 read_log 192718'11704 (192718'11703) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:269 2014-08-06 14:53:34.526872 2014-08-13 18:06:06.561102 7f5c973a57a0 20 read_log 192718'11705 (192718'11704) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:270 2014-08-06 14:53:34.569721 2014-08-13 18:06:06.561116 7f5c973a57a0 20 read_log 192718'11706 (192718'11705) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:271 2014-08-06 14:53:35.000469 2014-08-13 18:06:06.561129 7f5c973a57a0 20 read_log 192718'11707 (192718'11706) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:272 2014-08-06 14:53:35.087793 2014-08-13 18:06:06.561143 7f5c973a57a0 20 read_log 192718'11708 (192718'11707) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:273 2014-08-06 14:53:35.152856 2014-08-13 18:06:06.561163 7f5c973a57a0 20 read_log 192718'11709 (192718'11708) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:274 2014-08-06 14:53:35.225882 2014-08-13 18:06:06.561176 7f5c973a57a0 20 read_log 192718'11710 (192718'11709) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:275 2014-08-06 14:53:35.302013 2014-08-13 18:06:06.561188 7f5c973a57a0 20 read_log 192718'11711 (192718'11710) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:276 2014-08-06 14:53:35.350984 2014-08-13 18:06:06.561207 7f5c973a57a0 20 read_log 192718'11712 (192718'11711) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:277 2014-08-06 14:53:35.411400 2014-08-13 18:06:06.561271 7f5c973a57a0 20 read_log 192718'11713 (192718'11712) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:278 2014-08-06 14:53:35.454367 2014-08-13 18:06:06.561290 7f5c973a57a0 20 read_log 192718'11714 (192718'11713) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:279 2014-08-06 14:53:35.497304 2014-08-13 18:06:06.561303 7f5c973a57a0 20 read_log 192718'11715 (192718'11714) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:280 2014-08-06 14:53:35.541532 2014-08-13 18:06:06.561315 7f5c973a57a0 20 read_log 192718'11716 (192718'11715) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:281 2014-08-06 14:53:36.021739 2014-08-13 18:06:06.561328 7f5c973a57a0 20 read_log 192718'11717 (192718'11716) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:282 2014-08-06 14:53:36.070341 2014-08-13 18:06:06.561340 7f5c973a57a0 20 read_log 192718'11718 (192718'11717) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:283 2014-08-06 14:53:36.113384 2014-08-13 18:06:06.561352 7f5c973a57a0 20 read_log 192718'11719 (192718'11718) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:284 2014-08-06 14:53:36.156845 2014-08-13 18:06:06.561364 7f5c973a57a0 20 read_log 192718'11720 (192718'11719) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:285 2014-08-06 14:53:36.199839 2014-08-13 18:06:06.561392 7f5c973a57a0 20 read_log 192718'11721 (192718'11720) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:286 2014-08-06 14:53:37.043944 2014-08-13 18:06:06.561406 7f5c973a57a0 20 read_log 192718'11722 (192718'11721) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:287 2014-08-06 14:53:37.069708 2014-08-13 18:06:06.561426 7f5c973a57a0 20 read_log 192718'11723 (192718'11722) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:288 2014-08-06 14:53:37.094381 2014-08-13 18:06:06.561442 7f5c973a57a0 20 read_log 192718'11724 (192718'11723) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:289 2014-08-06 14:53:37.118346 2014-08-13 18:06:06.561454 7f5c973a57a0 20 read_log 192718'11725 (192718'11724) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:290 2014-08-06 14:53:37.142055 2014-08-13 18:06:06.561466 7f5c973a57a0 20 read_log 192718'11726 (192718'11725) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:291 2014-08-06 14:53:37.167926 2014-08-13 18:06:06.561479 7f5c973a57a0 20 read_log 192718'11727 (192718'11726) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:292 2014-08-06 14:53:37.194857 2014-08-13 18:06:06.561492 7f5c973a57a0 20 read_log 192718'11728 (192718'11727) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:293 2014-08-06 14:53:37.221221 2014-08-13 18:06:06.561504 7f5c973a57a0 20 read_log 192718'11729 (192718'11728) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:294 2014-08-06 14:53:37.246904 2014-08-13 18:06:06.561516 7f5c973a57a0 20 read_log 192718'11730 (192718'11729) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:295 2014-08-06 14:53:37.272161 2014-08-13 18:06:06.561528 7f5c973a57a0 20 read_log 192718'11731 (192718'11730) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:296 2014-08-06 14:53:37.298120 2014-08-13 18:06:06.561540 7f5c973a57a0 20 read_log 192718'11732 (192718'11731) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:297 2014-08-06 14:53:37.324584 2014-08-13 18:06:06.561552 7f5c973a57a0 20 read_log 192718'11733 (192718'11732) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:298 2014-08-06 14:53:37.351272 2014-08-13 18:06:06.561564 7f5c973a57a0 20 read_log 192718'11734 (192718'11733) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:299 2014-08-06 14:53:37.377884 2014-08-13 18:06:06.561576 7f5c973a57a0 20 read_log 192718'11735 (192718'11734) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:300 2014-08-06 14:53:37.404177 2014-08-13 18:06:06.561596 7f5c973a57a0 20 read_log 192718'11736 (192718'11735) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:301 2014-08-06 14:53:37.632939 2014-08-13 18:06:06.561611 7f5c973a57a0 20 read_log 192718'11737 (192718'11736) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:302 2014-08-06 14:53:37.689419 2014-08-13 18:06:06.561623 7f5c973a57a0 20 read_log 192718'11738 (192718'11737) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:303 2014-08-06 14:53:37.717803 2014-08-13 18:06:06.561635 7f5c973a57a0 20 read_log 192718'11739 (192718'11738) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:304 2014-08-06 14:53:37.744250 2014-08-13 18:06:06.561649 7f5c973a57a0 20 read_log 192718'11740 (192718'11739) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:305 2014-08-06 14:53:37.771022 2014-08-13 18:06:06.561661 7f5c973a57a0 20 read_log 192718'11741 (192718'11740) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:306 2014-08-06 14:53:37.876730 2014-08-13 18:06:06.561673 7f5c973a57a0 20 read_log 192718'11742 (192718'11741) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:307 2014-08-06 14:53:37.956372 2014-08-13 18:06:06.561685 7f5c973a57a0 20 read_log 192718'11743 (192718'11742) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:308 2014-08-06 14:53:38.013556 2014-08-13 18:06:06.561697 7f5c973a57a0 20 read_log 192718'11744 (192718'11743) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:309 2014-08-06 14:53:38.047428 2014-08-13 18:06:06.561709 7f5c973a57a0 20 read_log 192718'11745 (192718'11744) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:310 2014-08-06 14:53:38.073303 2014-08-13 18:06:06.561721 7f5c973a57a0 20 read_log 192718'11746 (192718'11745) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:311 2014-08-06 14:53:38.099159 2014-08-13 18:06:06.561734 7f5c973a57a0 20 read_log 192718'11747 (192718'11746) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:312 2014-08-06 14:53:38.126837 2014-08-13 18:06:06.561746 7f5c973a57a0 20 read_log 192718'11748 (192718'11747) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:313 2014-08-06 14:53:38.153846 2014-08-13 18:06:06.561758 7f5c973a57a0 20 read_log 192718'11749 (192718'11748) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:314 2014-08-06 14:53:38.179355 2014-08-13 18:06:06.561774 7f5c973a57a0 20 read_log 192718'11750 (192718'11749) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:315 2014-08-06 14:53:38.204818 2014-08-13 18:06:06.561787 7f5c973a57a0 20 read_log 192718'11751 (192718'11750) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:316 2014-08-06 14:53:38.230210 2014-08-13 18:06:06.561799 7f5c973a57a0 20 read_log 192718'11752 (192718'11751) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:317 2014-08-06 14:53:38.260359 2014-08-13 18:06:06.561813 7f5c973a57a0 20 read_log 192718'11753 (192718'11752) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:318 2014-08-06 14:53:38.491939 2014-08-13 18:06:06.561825 7f5c973a57a0 20 read_log 192718'11754 (192718'11754) modify 34/hit_set_24.34_archive_2014-08-06 14:48:58.711039_2014-08-06 14:53:38.550251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 14:53:38.550776 2014-08-13 18:06:06.561840 7f5c973a57a0 20 read_log 192718'11755 (192718'11171) delete 34/hit_set_24.34_archive_2014-08-06 14:16:43.165223_2014-08-06 14:48:58.710801/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 14:53:38.550776 2014-08-13 18:06:06.561854 7f5c973a57a0 20 read_log 192718'11756 (192718'11753) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:319 2014-08-06 14:53:38.541793 2014-08-13 18:06:06.561868 7f5c973a57a0 20 read_log 192718'11757 (192718'11756) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:320 2014-08-06 14:53:38.608817 2014-08-13 18:06:06.561881 7f5c973a57a0 20 read_log 192718'11758 (192718'11757) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:321 2014-08-06 14:53:38.668325 2014-08-13 18:06:06.561924 7f5c973a57a0 20 read_log 192718'11759 (192718'11758) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:322 2014-08-06 14:53:38.694989 2014-08-13 18:06:06.561941 7f5c973a57a0 20 read_log 192718'11760 (192718'11759) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:323 2014-08-06 14:53:38.721162 2014-08-13 18:06:06.561955 7f5c973a57a0 20 read_log 192718'11761 (192718'11760) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:324 2014-08-06 14:53:38.747611 2014-08-13 18:06:06.561967 7f5c973a57a0 20 read_log 192718'11762 (192718'11761) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:325 2014-08-06 14:53:38.776637 2014-08-13 18:06:06.561986 7f5c973a57a0 20 read_log 192718'11763 (192718'11762) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:326 2014-08-06 14:53:38.802653 2014-08-13 18:06:06.562014 7f5c973a57a0 20 read_log 192718'11764 (192718'11763) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:327 2014-08-06 14:53:38.828907 2014-08-13 18:06:06.562049 7f5c973a57a0 20 read_log 192718'11765 (192718'11764) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:328 2014-08-06 14:53:39.123577 2014-08-13 18:06:06.562075 7f5c973a57a0 20 read_log 192718'11766 (192718'11765) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:329 2014-08-06 14:53:39.152900 2014-08-13 18:06:06.562092 7f5c973a57a0 20 read_log 192718'11767 (192718'11766) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:330 2014-08-06 14:53:39.181496 2014-08-13 18:06:06.562105 7f5c973a57a0 20 read_log 192718'11768 (192718'11767) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:331 2014-08-06 14:53:39.209792 2014-08-13 18:06:06.562129 7f5c973a57a0 20 read_log 192718'11769 (192718'11768) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:332 2014-08-06 14:53:39.238223 2014-08-13 18:06:06.562164 7f5c973a57a0 20 read_log 192718'11770 (192718'11769) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:333 2014-08-06 14:53:39.265568 2014-08-13 18:06:06.562220 7f5c973a57a0 20 read_log 192718'11771 (192718'11770) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:334 2014-08-06 14:53:39.292236 2014-08-13 18:06:06.562246 7f5c973a57a0 20 read_log 192718'11772 (192718'11771) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:335 2014-08-06 14:53:39.318798 2014-08-13 18:06:06.562259 7f5c973a57a0 20 read_log 192718'11773 (192718'11772) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:336 2014-08-06 14:53:39.345899 2014-08-13 18:06:06.562284 7f5c973a57a0 20 read_log 192718'11774 (192718'11773) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:337 2014-08-06 14:53:39.374743 2014-08-13 18:06:06.562301 7f5c973a57a0 20 read_log 192718'11775 (192718'11774) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:338 2014-08-06 14:53:39.661525 2014-08-13 18:06:06.562326 7f5c973a57a0 20 read_log 192718'11776 (192718'11775) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:339 2014-08-06 14:53:39.690156 2014-08-13 18:06:06.562349 7f5c973a57a0 20 read_log 192718'11777 (192718'11776) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:340 2014-08-06 14:53:39.717174 2014-08-13 18:06:06.562365 7f5c973a57a0 20 read_log 192718'11778 (192718'11777) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:341 2014-08-06 14:53:39.743553 2014-08-13 18:06:06.562383 7f5c973a57a0 20 read_log 192718'11779 (192718'11778) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:342 2014-08-06 14:53:39.771621 2014-08-13 18:06:06.562405 7f5c973a57a0 20 read_log 192718'11780 (192718'11779) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:343 2014-08-06 14:53:39.797313 2014-08-13 18:06:06.562421 7f5c973a57a0 20 read_log 192718'11781 (192718'11780) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:344 2014-08-06 14:53:39.823599 2014-08-13 18:06:06.562453 7f5c973a57a0 20 read_log 192718'11782 (192718'11781) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:345 2014-08-06 14:53:39.849737 2014-08-13 18:06:06.562481 7f5c973a57a0 20 read_log 192718'11783 (192718'11782) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:346 2014-08-06 14:53:39.875420 2014-08-13 18:06:06.562497 7f5c973a57a0 20 read_log 192718'11784 (192718'11783) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:347 2014-08-06 14:53:39.901789 2014-08-13 18:06:06.562509 7f5c973a57a0 20 read_log 192718'11785 (192718'11784) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:348 2014-08-06 14:53:39.927427 2014-08-13 18:06:06.562521 7f5c973a57a0 20 read_log 192718'11786 (192718'11785) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:349 2014-08-06 14:53:39.953894 2014-08-13 18:06:06.562535 7f5c973a57a0 20 read_log 192718'11787 (192718'11786) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:350 2014-08-06 14:53:40.365858 2014-08-13 18:06:06.562547 7f5c973a57a0 20 read_log 192718'11788 (192718'11787) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:351 2014-08-06 14:53:40.390943 2014-08-13 18:06:06.562560 7f5c973a57a0 20 read_log 192718'11789 (192718'11788) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:352 2014-08-06 14:53:40.418706 2014-08-13 18:06:06.562572 7f5c973a57a0 20 read_log 192718'11790 (192718'11789) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:353 2014-08-06 14:53:40.445374 2014-08-13 18:06:06.562584 7f5c973a57a0 20 read_log 192718'11791 (192718'11790) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:354 2014-08-06 14:53:40.473037 2014-08-13 18:06:06.562602 7f5c973a57a0 20 read_log 192718'11792 (192718'11791) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:355 2014-08-06 14:53:40.499574 2014-08-13 18:06:06.562616 7f5c973a57a0 20 read_log 192718'11793 (192718'11792) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:356 2014-08-06 14:53:40.525870 2014-08-13 18:06:06.562629 7f5c973a57a0 20 read_log 192718'11794 (192718'11793) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:357 2014-08-06 14:53:40.551530 2014-08-13 18:06:06.562655 7f5c973a57a0 20 read_log 192718'11795 (192718'11794) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:358 2014-08-06 14:53:40.579196 2014-08-13 18:06:06.562671 7f5c973a57a0 20 read_log 192718'11796 (192718'11795) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:359 2014-08-06 14:53:40.609908 2014-08-13 18:06:06.562713 7f5c973a57a0 20 read_log 192718'11797 (192718'11796) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:360 2014-08-06 14:53:40.636701 2014-08-13 18:06:06.562726 7f5c973a57a0 20 read_log 192718'11798 (192718'11797) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:361 2014-08-06 14:53:40.718449 2014-08-13 18:06:06.562739 7f5c973a57a0 20 read_log 192718'11799 (192718'11798) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:362 2014-08-06 14:53:40.799478 2014-08-13 18:06:06.562751 7f5c973a57a0 20 read_log 192718'11800 (192718'11799) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:363 2014-08-06 14:53:40.880461 2014-08-13 18:06:06.562775 7f5c973a57a0 20 read_log 192718'11801 (192718'11800) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:364 2014-08-06 14:53:40.933561 2014-08-13 18:06:06.562788 7f5c973a57a0 20 read_log 192718'11802 (192718'11801) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:365 2014-08-06 14:53:41.007887 2014-08-13 18:06:06.562801 7f5c973a57a0 20 read_log 192718'11803 (192718'11802) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:366 2014-08-06 14:53:41.089504 2014-08-13 18:06:06.562817 7f5c973a57a0 20 read_log 192718'11804 (192718'11803) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:367 2014-08-06 14:53:41.169838 2014-08-13 18:06:06.562830 7f5c973a57a0 20 read_log 192718'11805 (192718'11804) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:368 2014-08-06 14:53:41.616343 2014-08-13 18:06:06.562843 7f5c973a57a0 20 read_log 192718'11806 (192718'11805) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:369 2014-08-06 14:53:41.697452 2014-08-13 18:06:06.562889 7f5c973a57a0 20 read_log 192718'11807 (192718'11806) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:370 2014-08-06 14:53:41.778535 2014-08-13 18:06:06.562906 7f5c973a57a0 20 read_log 192718'11808 (192718'11807) modify 75dfec34/core.19312+32+2/head//24 by client.462194.0:371 2014-08-06 14:53:41.862025 2014-08-13 18:06:06.562918 7f5c973a57a0 20 read_log 192718'11809 (192718'11809) modify 34/hit_set_24.34_archive_2014-08-06 14:53:38.550781_2014-08-06 15:31:20.412990/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:31:20.413258 2014-08-13 18:06:06.562937 7f5c973a57a0 20 read_log 192718'11810 (192718'11754) delete 34/hit_set_24.34_archive_2014-08-06 14:48:58.711039_2014-08-06 14:53:38.550251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:31:20.413258 2014-08-13 18:06:06.562950 7f5c973a57a0 20 read_log 192718'11977 (192718'11808) clean 75dfec34/core.19312+32+2/head//24 by osd.325.0:33832 2014-08-06 15:31:53.476866 2014-08-13 18:06:06.562962 7f5c973a57a0 20 read_log 192718'12318 (192718'12318) modify 34/hit_set_24.34_archive_2014-08-06 15:31:20.413262_2014-08-06 15:50:05.551522/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:50:05.551751 2014-08-13 18:06:06.562974 7f5c973a57a0 20 read_log 192718'12319 (192718'11809) delete 34/hit_set_24.34_archive_2014-08-06 14:53:38.550781_2014-08-06 15:31:20.412990/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:50:05.551751 2014-08-13 18:06:06.562987 7f5c973a57a0 20 read_log 192803'12662 (192718'11977) delete 75dfec34/core.19312+32+2/head//24 by osd.325.0:35389 2014-08-11 10:27:14.259503 2014-08-13 18:06:06.562999 7f5c973a57a0 20 read_log 192834'12664 (192834'12664) modify 34/hit_set_24.34_archive_2014-08-11 10:24:43.621609_2014-08-11 15:23:36.202456/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.202664 2014-08-13 18:06:06.563011 7f5c973a57a0 20 read_log 192834'12665 (192718'12318) delete 34/hit_set_24.34_archive_2014-08-06 15:31:20.413262_2014-08-06 15:50:05.551522/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.202664 2014-08-13 18:06:06.563023 7f5c973a57a0 20 read_log 192834'12666 (0'0) promote 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by osd.325.0:36357 2014-08-11 15:23:36.206543 2014-08-13 18:06:06.563036 7f5c973a57a0 20 read_log 192834'12667 (192834'12666) modify 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.933448.0:134 2014-08-11 15:23:36.195923 2014-08-13 18:06:06.563048 7f5c973a57a0 20 read_log 192834'12668 (0'0) promote d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:36362 2014-08-11 15:23:37.163127 2014-08-13 18:06:06.563060 7f5c973a57a0 20 read_log 192834'12669 (192834'12668) modify d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by client.933448.0:395 2014-08-11 15:23:37.155417 2014-08-13 18:06:06.563073 7f5c973a57a0 20 read_log 192834'12720 (0'0) promote c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:36452 2014-08-11 15:23:51.206823 2014-08-13 18:06:06.563085 7f5c973a57a0 20 read_log 192834'12721 (192834'12720) modify c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by client.933448.0:3316 2014-08-11 15:23:51.199125 2014-08-13 18:06:06.563110 7f5c973a57a0 20 read_log 192834'12728 (0'0) promote 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:36464 2014-08-11 15:23:52.804960 2014-08-13 18:06:06.563123 7f5c973a57a0 20 read_log 192834'12729 (192834'12728) modify 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by client.933448.0:3622 2014-08-11 15:23:52.796696 2014-08-13 18:06:06.563136 7f5c973a57a0 20 read_log 192834'12738 (0'0) promote 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by osd.325.0:36476 2014-08-11 15:23:54.591706 2014-08-13 18:06:06.563165 7f5c973a57a0 20 read_log 192834'12739 (192834'12738) modify 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.933448.0:3935 2014-08-11 15:23:54.583107 2014-08-13 18:06:06.563184 7f5c973a57a0 20 read_log 192834'12750 (0'0) promote 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:36512 2014-08-11 15:24:02.823498 2014-08-13 18:06:06.563197 7f5c973a57a0 20 read_log 192834'12751 (192834'12750) modify 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by client.933448.0:5771 2014-08-11 15:24:02.816311 2014-08-13 18:06:06.563210 7f5c973a57a0 20 read_log 192834'12788 (0'0) promote 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by osd.325.0:36598 2014-08-11 15:24:16.250818 2014-08-13 18:06:06.563222 7f5c973a57a0 20 read_log 192834'12789 (192834'12788) modify 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.933448.0:8936 2014-08-11 15:24:16.242386 2014-08-13 18:06:06.563234 7f5c973a57a0 20 read_log 192834'12790 (0'0) promote 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by osd.325.0:36606 2014-08-11 15:24:18.065061 2014-08-13 18:06:06.563247 7f5c973a57a0 20 read_log 192834'12791 (192834'12790) modify 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.933448.0:9230 2014-08-11 15:24:18.057115 2014-08-13 18:06:06.563259 7f5c973a57a0 20 read_log 192834'12806 (0'0) promote e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by osd.325.0:36646 2014-08-11 15:24:25.575454 2014-08-13 18:06:06.563275 7f5c973a57a0 20 read_log 192834'12807 (192834'12806) modify e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.933448.0:10876 2014-08-11 15:24:25.567264 2014-08-13 18:06:06.563289 7f5c973a57a0 20 read_log 192834'12810 (0'0) promote c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by osd.325.0:36654 2014-08-11 15:24:26.875528 2014-08-13 18:06:06.563301 7f5c973a57a0 20 read_log 192834'12811 (192834'12810) modify c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.933448.0:11214 2014-08-11 15:24:26.867286 2014-08-13 18:06:06.563329 7f5c973a57a0 20 read_log 192834'12882 (0'0) promote c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:36788 2014-08-11 15:24:51.296830 2014-08-13 18:06:06.563344 7f5c973a57a0 20 read_log 192834'12883 (192834'12882) modify c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by client.933448.0:15788 2014-08-11 15:24:51.287928 2014-08-13 18:06:06.563362 7f5c973a57a0 20 read_log 192834'12906 (0'0) promote d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by osd.325.0:36838 2014-08-11 15:25:00.839701 2014-08-13 18:06:06.563381 7f5c973a57a0 20 read_log 192834'12907 (192834'12906) modify d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.933448.0:17636 2014-08-11 15:25:00.832090 2014-08-13 18:06:06.563394 7f5c973a57a0 20 read_log 192834'12908 (0'0) promote ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by osd.325.0:36840 2014-08-11 15:25:01.187700 2014-08-13 18:06:06.563406 7f5c973a57a0 20 read_log 192834'12909 (192834'12908) modify ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.933448.0:17719 2014-08-11 15:25:01.179217 2014-08-13 18:06:06.563419 7f5c973a57a0 20 read_log 192834'12914 (0'0) promote 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by osd.325.0:36856 2014-08-11 15:25:03.701143 2014-08-13 18:06:06.563441 7f5c973a57a0 20 read_log 192834'12915 (192834'12914) modify 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.933448.0:18108 2014-08-11 15:25:03.691216 2014-08-13 18:06:06.563460 7f5c973a57a0 20 read_log 192834'12940 (0'0) promote 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by osd.325.0:36914 2014-08-11 15:25:13.869994 2014-08-13 18:06:06.563486 7f5c973a57a0 20 read_log 192834'12941 (192834'12940) modify 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.933448.0:19895 2014-08-11 15:25:13.855426 2014-08-13 18:06:06.563508 7f5c973a57a0 20 read_log 192834'12952 (0'0) promote ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:36948 2014-08-11 15:25:23.513383 2014-08-13 18:06:06.563521 7f5c973a57a0 20 read_log 192834'12953 (192834'12952) modify ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by client.933448.0:20832 2014-08-11 15:25:23.498921 2014-08-13 18:06:06.563548 7f5c973a57a0 20 read_log 192834'12976 (192834'12667) modify 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.933448.0:22425 2014-08-11 15:25:37.855704 2014-08-13 18:06:06.563563 7f5c973a57a0 20 read_log 192834'12977 (192834'12669) modify d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by client.933448.0:22686 2014-08-11 15:25:38.467411 2014-08-13 18:06:06.563575 7f5c973a57a0 20 read_log 192834'13003 (192834'12721) modify c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by client.933448.0:25607 2014-08-11 15:25:42.261735 2014-08-13 18:06:06.563588 7f5c973a57a0 20 read_log 192834'13007 (192834'12729) modify 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by client.933448.0:25913 2014-08-11 15:25:42.713837 2014-08-13 18:06:06.563600 7f5c973a57a0 20 read_log 192834'13012 (192834'12739) modify 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.933448.0:26226 2014-08-11 15:25:43.138799 2014-08-13 18:06:06.563612 7f5c973a57a0 20 read_log 192834'13018 (192834'12751) modify 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by client.933448.0:28062 2014-08-11 15:25:45.447250 2014-08-13 18:06:06.563625 7f5c973a57a0 20 read_log 192834'13037 (192834'12789) modify 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.933448.0:31227 2014-08-11 15:25:49.816037 2014-08-13 18:06:06.563640 7f5c973a57a0 20 read_log 192834'13038 (192834'12791) modify 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.933448.0:31521 2014-08-11 15:25:50.150004 2014-08-13 18:06:06.563652 7f5c973a57a0 20 read_log 192834'13046 (192834'12807) modify e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.933448.0:33167 2014-08-11 15:25:52.932793 2014-08-13 18:06:06.563665 7f5c973a57a0 20 read_log 192834'13048 (192834'12811) modify c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.933448.0:33505 2014-08-11 15:25:53.563835 2014-08-13 18:06:06.563677 7f5c973a57a0 20 read_log 192834'13084 (192834'12883) modify c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by client.933448.0:38079 2014-08-11 15:26:00.468826 2014-08-13 18:06:06.563689 7f5c973a57a0 20 read_log 192834'13096 (192834'12907) modify d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.933448.0:39927 2014-08-11 15:26:02.170591 2014-08-13 18:06:06.563702 7f5c973a57a0 20 read_log 192834'13097 (192834'12909) modify ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.933448.0:40010 2014-08-11 15:26:02.218458 2014-08-13 18:06:06.563722 7f5c973a57a0 20 read_log 192834'13100 (192834'12915) modify 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.933448.0:40399 2014-08-11 15:26:02.599976 2014-08-13 18:06:06.563738 7f5c973a57a0 20 read_log 192834'13113 (192834'12941) modify 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.933448.0:42186 2014-08-11 15:26:03.179957 2014-08-13 18:06:06.563751 7f5c973a57a0 20 read_log 192834'13119 (192834'12953) modify ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by client.933448.0:43123 2014-08-11 15:26:03.508114 2014-08-13 18:06:06.563763 7f5c973a57a0 20 read_log 192834'13137 (0'0) promote 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by osd.325.0:37340 2014-08-11 15:26:15.620950 2014-08-13 18:06:06.563783 7f5c973a57a0 20 read_log 192834'13138 (192834'13137) modify 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.1883634.0:305 2014-08-11 15:26:15.611342 2014-08-13 18:06:06.563798 7f5c973a57a0 20 read_log 192834'13143 (0'0) promote e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by osd.325.0:37352 2014-08-11 15:26:16.653772 2014-08-13 18:06:06.563811 7f5c973a57a0 20 read_log 192834'13144 (192834'13143) modify e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.1883634.0:592 2014-08-11 15:26:16.645472 2014-08-13 18:06:06.563823 7f5c973a57a0 20 read_log 192834'13145 (0'0) promote bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by osd.325.0:37356 2014-08-11 15:26:17.523319 2014-08-13 18:06:06.563836 7f5c973a57a0 20 read_log 192834'13146 (192834'13145) modify bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.1883634.0:833 2014-08-11 15:26:17.514537 2014-08-13 18:06:06.563848 7f5c973a57a0 20 read_log 192834'13159 (0'0) promote bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:37376 2014-08-11 15:26:19.871321 2014-08-13 18:06:06.563860 7f5c973a57a0 20 read_log 192834'13160 (192834'13159) modify bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.1883634.0:1483 2014-08-11 15:26:19.863625 2014-08-13 18:06:06.563873 7f5c973a57a0 20 read_log 192834'13167 (0'0) promote 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:37386 2014-08-11 15:26:20.616570 2014-08-13 18:06:06.563885 7f5c973a57a0 20 read_log 192834'13168 (192834'13167) modify 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.1883634.0:1688 2014-08-11 15:26:20.609451 2014-08-13 18:06:06.563914 7f5c973a57a0 20 read_log 192834'13185 (0'0) promote acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by osd.325.0:37416 2014-08-11 15:26:23.058460 2014-08-13 18:06:06.563929 7f5c973a57a0 20 read_log 192834'13186 (192834'13185) modify acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.1883634.0:2322 2014-08-11 15:26:23.051509 2014-08-13 18:06:06.563942 7f5c973a57a0 20 read_log 192834'13191 (0'0) promote 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by osd.325.0:37426 2014-08-11 15:26:24.506735 2014-08-13 18:06:06.563954 7f5c973a57a0 20 read_log 192834'13192 (192834'13191) modify 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.1883634.0:2730 2014-08-11 15:26:24.497929 2014-08-13 18:06:06.563967 7f5c973a57a0 20 read_log 192834'13197 (0'0) promote 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by osd.325.0:37444 2014-08-11 15:26:26.251194 2014-08-13 18:06:06.563979 7f5c973a57a0 20 read_log 192834'13198 (192834'13197) modify 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.1883634.0:3222 2014-08-11 15:26:26.242599 2014-08-13 18:06:06.563994 7f5c973a57a0 20 read_log 192834'13207 (0'0) promote 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by osd.325.0:37470 2014-08-11 15:26:28.885502 2014-08-13 18:06:06.564006 7f5c973a57a0 20 read_log 192834'13208 (192834'13207) modify 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.1883634.0:3969 2014-08-11 15:26:28.877462 2014-08-13 18:06:06.564021 7f5c973a57a0 20 read_log 192834'13209 (0'0) promote 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by osd.325.0:37476 2014-08-11 15:26:29.088711 2014-08-13 18:06:06.564033 7f5c973a57a0 20 read_log 192834'13210 (192834'13209) modify 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.1883634.0:4026 2014-08-11 15:26:29.081371 2014-08-13 18:06:06.564048 7f5c973a57a0 20 read_log 192834'13213 (0'0) promote eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by osd.325.0:37488 2014-08-11 15:26:30.316213 2014-08-13 18:06:06.564069 7f5c973a57a0 20 read_log 192834'13214 (192834'13213) modify eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.1883634.0:4372 2014-08-11 15:26:30.306501 2014-08-13 18:06:06.564084 7f5c973a57a0 20 read_log 192834'13217 (0'0) promote 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by osd.325.0:37494 2014-08-11 15:26:30.614572 2014-08-13 18:06:06.564096 7f5c973a57a0 20 read_log 192834'13218 (192834'13217) modify 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.1883634.0:4456 2014-08-11 15:26:30.605825 2014-08-13 18:06:06.564109 7f5c973a57a0 20 read_log 192834'13221 (0'0) promote d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by osd.325.0:37520 2014-08-11 15:26:34.860915 2014-08-13 18:06:06.564121 7f5c973a57a0 20 read_log 192834'13222 (192834'13221) modify d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.1883634.0:5567 2014-08-11 15:26:34.854088 2014-08-13 18:06:06.564134 7f5c973a57a0 20 read_log 192834'13233 (0'0) promote 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by osd.325.0:37542 2014-08-11 15:26:35.893138 2014-08-13 18:06:06.564151 7f5c973a57a0 20 read_log 192834'13234 (192834'13233) modify 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.1883634.0:5857 2014-08-11 15:26:35.885475 2014-08-13 18:06:06.564166 7f5c973a57a0 20 read_log 192834'13237 (0'0) promote 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by osd.325.0:37546 2014-08-11 15:26:36.146514 2014-08-13 18:06:06.564178 7f5c973a57a0 20 read_log 192834'13238 (192834'13237) modify 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.1883634.0:5927 2014-08-11 15:26:36.138623 2014-08-13 18:06:06.564213 7f5c973a57a0 20 read_log 192834'13255 (0'0) promote e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by osd.325.0:37596 2014-08-11 15:26:40.940484 2014-08-13 18:06:06.564226 7f5c973a57a0 20 read_log 192834'13256 (192834'13255) modify e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.1883634.0:7136 2014-08-11 15:26:40.933369 2014-08-13 18:06:06.564244 7f5c973a57a0 20 read_log 192834'13267 (0'0) promote c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:37622 2014-08-11 15:26:43.777036 2014-08-13 18:06:06.564258 7f5c973a57a0 20 read_log 192834'13268 (192834'13267) modify c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.1883634.0:7807 2014-08-11 15:26:43.767811 2014-08-13 18:06:06.564271 7f5c973a57a0 20 read_log 192834'13269 (0'0) promote 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by osd.325.0:37626 2014-08-11 15:26:44.550547 2014-08-13 18:06:06.564286 7f5c973a57a0 20 read_log 192834'13270 (192834'13269) modify 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.1883634.0:7982 2014-08-11 15:26:44.541786 2014-08-13 18:06:06.564299 7f5c973a57a0 20 read_log 192834'13307 (0'0) promote b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by osd.325.0:37688 2014-08-11 15:26:54.218582 2014-08-13 18:06:06.564311 7f5c973a57a0 20 read_log 192834'13308 (192834'13307) modify b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.1883634.0:10242 2014-08-11 15:26:54.210945 2014-08-13 18:06:06.564324 7f5c973a57a0 20 read_log 192834'13313 (0'0) promote 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by osd.325.0:37698 2014-08-11 15:26:55.724019 2014-08-13 18:06:06.564340 7f5c973a57a0 20 read_log 192834'13314 (192834'13313) modify 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.1883634.0:10583 2014-08-11 15:26:55.715173 2014-08-13 18:06:06.564353 7f5c973a57a0 20 read_log 192834'13323 (0'0) promote 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:37716 2014-08-11 15:27:00.924119 2014-08-13 18:06:06.564391 7f5c973a57a0 20 read_log 192834'13324 (192834'13323) modify 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.1883634.0:11670 2014-08-11 15:27:00.915979 2014-08-13 18:06:06.564407 7f5c973a57a0 20 read_log 192834'13329 (0'0) promote 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:37722 2014-08-11 15:27:01.376545 2014-08-13 18:06:06.564419 7f5c973a57a0 20 read_log 192834'13330 (192834'13329) modify 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.1883634.0:11820 2014-08-11 15:27:01.368576 2014-08-13 18:06:06.564443 7f5c973a57a0 20 read_log 192834'13345 (0'0) promote be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by osd.325.0:37754 2014-08-11 15:27:05.273301 2014-08-13 18:06:06.564459 7f5c973a57a0 20 read_log 192834'13346 (192834'13345) modify be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.1883634.0:12654 2014-08-11 15:27:05.264502 2014-08-13 18:06:06.564472 7f5c973a57a0 20 read_log 192834'13379 (0'0) promote 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:37826 2014-08-11 15:27:13.925253 2014-08-13 18:06:06.564494 7f5c973a57a0 20 read_log 192834'13380 (192834'13379) modify 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.1883634.0:14603 2014-08-11 15:27:13.916419 2014-08-13 18:06:06.564510 7f5c973a57a0 20 read_log 192834'13417 (0'0) promote ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by osd.325.0:37916 2014-08-11 15:27:34.969228 2014-08-13 18:06:06.564523 7f5c973a57a0 20 read_log 192834'13418 (192834'13417) modify ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.1883634.0:17389 2014-08-11 15:27:34.958285 2014-08-13 18:06:06.564535 7f5c973a57a0 20 read_log 192834'13437 (0'0) promote af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by osd.325.0:37964 2014-08-11 15:27:48.268989 2014-08-13 18:06:06.564547 7f5c973a57a0 20 read_log 192834'13438 (192834'13437) modify af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.1883634.0:18917 2014-08-11 15:27:48.260158 2014-08-13 18:06:06.564560 7f5c973a57a0 20 read_log 192834'13441 (0'0) promote c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by osd.325.0:37968 2014-08-11 15:27:48.849331 2014-08-13 18:06:06.564575 7f5c973a57a0 20 read_log 192834'13442 (192834'13441) modify c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.1883634.0:19014 2014-08-11 15:27:48.840043 2014-08-13 18:06:06.564588 7f5c973a57a0 20 read_log 192834'13445 (0'0) promote 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by osd.325.0:37974 2014-08-11 15:27:50.099531 2014-08-13 18:06:06.564600 7f5c973a57a0 20 read_log 192834'13446 (192834'13445) modify 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.1883634.0:19245 2014-08-11 15:27:50.089180 2014-08-13 18:06:06.564612 7f5c973a57a0 20 read_log 192834'13463 (0'0) promote 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by osd.325.0:38008 2014-08-11 15:27:54.902252 2014-08-13 18:06:06.564625 7f5c973a57a0 20 read_log 192834'13464 (192834'13463) modify 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.1883634.0:19997 2014-08-11 15:27:54.892615 2014-08-13 18:06:06.564637 7f5c973a57a0 20 read_log 192834'13465 (0'0) promote 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by osd.325.0:38010 2014-08-11 15:27:54.916035 2014-08-13 18:06:06.564649 7f5c973a57a0 20 read_log 192834'13466 (192834'13465) modify 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.1883634.0:20002 2014-08-11 15:27:54.907899 2014-08-13 18:06:06.564661 7f5c973a57a0 20 read_log 192834'13499 (0'0) promote 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by osd.325.0:38122 2014-08-11 15:28:09.040453 2014-08-13 18:06:06.564681 7f5c973a57a0 20 read_log 192834'13500 (192834'13499) modify 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.1883634.0:22506 2014-08-11 15:28:09.030809 2014-08-13 18:06:06.564694 7f5c973a57a0 20 read_log 192834'13519 (192834'13003) clean c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:38187 2014-08-11 15:28:14.474340 2014-08-13 18:06:06.564708 7f5c973a57a0 20 read_log 192834'13520 (192834'12977) clean d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:38188 2014-08-11 15:28:14.477952 2014-08-13 18:06:06.564720 7f5c973a57a0 20 read_log 192834'13521 (192834'13119) clean ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:38189 2014-08-11 15:28:14.478298 2014-08-13 18:06:06.564734 7f5c973a57a0 20 read_log 192834'13522 (192834'13084) clean c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:38190 2014-08-11 15:28:14.491231 2014-08-13 18:06:06.564746 7f5c973a57a0 20 read_log 192834'13523 (192834'13018) clean 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:38191 2014-08-11 15:28:14.584747 2014-08-13 18:06:06.564758 7f5c973a57a0 20 read_log 192834'13524 (192834'13268) clean c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38192 2014-08-11 15:28:14.585093 2014-08-13 18:06:06.564771 7f5c973a57a0 20 read_log 192834'13525 (192834'13324) clean 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38193 2014-08-11 15:28:14.585334 2014-08-13 18:06:06.564783 7f5c973a57a0 20 read_log 192834'13526 (192834'13007) clean 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:38194 2014-08-11 15:28:14.602800 2014-08-13 18:06:06.564795 7f5c973a57a0 20 read_log 192834'13527 (192834'13330) clean 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38195 2014-08-11 15:28:14.803373 2014-08-13 18:06:06.564807 7f5c973a57a0 20 read_log 192834'13528 (192834'13380) clean 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38196 2014-08-11 15:28:14.803718 2014-08-13 18:06:06.564820 7f5c973a57a0 20 read_log 192834'13529 (192834'13160) clean bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38197 2014-08-11 15:28:14.803971 2014-08-13 18:06:06.564832 7f5c973a57a0 20 read_log 192834'13533 (192834'13168) clean 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38201 2014-08-11 15:28:15.062931 2014-08-13 18:06:06.564844 7f5c973a57a0 20 read_log 192834'13539 (192834'13519) delete c4680034/benchmark_data_storage0111-ib.pocstor_26659_object3315/head//24 by osd.325.0:38207 2014-08-11 15:28:15.240711 2014-08-13 18:06:06.564856 7f5c973a57a0 20 read_log 192834'13540 (192834'13524) delete c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38208 2014-08-11 15:28:15.241046 2014-08-13 18:06:06.564870 7f5c973a57a0 20 read_log 192834'13541 (192834'13520) delete d4c36034/benchmark_data_storage0111-ib.pocstor_26659_object394/head//24 by osd.325.0:38209 2014-08-11 15:28:15.243429 2014-08-13 18:06:06.564886 7f5c973a57a0 20 read_log 192834'13542 (192834'13521) delete ba9f7034/benchmark_data_storage0111-ib.pocstor_26659_object20831/head//24 by osd.325.0:38210 2014-08-11 15:28:15.243740 2014-08-13 18:06:06.564898 7f5c973a57a0 20 read_log 192834'13544 (192834'13522) delete c5028034/benchmark_data_storage0111-ib.pocstor_26659_object15787/head//24 by osd.325.0:38212 2014-08-11 15:28:15.250835 2014-08-13 18:06:06.564912 7f5c973a57a0 20 read_log 192834'13545 (192834'13525) delete 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38213 2014-08-11 15:28:15.251260 2014-08-13 18:06:06.564925 7f5c973a57a0 20 read_log 192834'13546 (192834'13527) delete 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38214 2014-08-11 15:28:15.254901 2014-08-13 18:06:06.564963 7f5c973a57a0 20 read_log 192834'13547 (192834'13523) delete 5fd4c034/benchmark_data_storage0111-ib.pocstor_26659_object5770/head//24 by osd.325.0:38215 2014-08-11 15:28:15.255145 2014-08-13 18:06:06.564979 7f5c973a57a0 20 read_log 192834'13548 (192834'13526) delete 928dc034/benchmark_data_storage0111-ib.pocstor_26659_object3621/head//24 by osd.325.0:38216 2014-08-11 15:28:15.261277 2014-08-13 18:06:06.564992 7f5c973a57a0 20 read_log 192834'13549 (192834'13528) delete 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38217 2014-08-11 15:28:15.261849 2014-08-13 18:06:06.565005 7f5c973a57a0 20 read_log 192834'13550 (192834'13529) delete bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38218 2014-08-11 15:28:15.265128 2014-08-13 18:06:06.565020 7f5c973a57a0 20 read_log 192834'13551 (192834'13533) delete 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38219 2014-08-11 15:28:15.265585 2014-08-13 18:06:06.565032 7f5c973a57a0 20 read_log 192834'13565 (192834'13138) modify 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.1883634.0:23629 2014-08-11 15:28:17.294559 2014-08-13 18:06:06.565045 7f5c973a57a0 20 read_log 192834'13568 (192834'13144) modify e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.1883634.0:23916 2014-08-11 15:28:18.077503 2014-08-13 18:06:06.565060 7f5c973a57a0 20 read_log 192834'13569 (192834'13146) modify bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.1883634.0:24157 2014-08-11 15:28:18.421687 2014-08-13 18:06:06.565072 7f5c973a57a0 20 read_log 192834'13576 (0'0) promote bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:38256 2014-08-11 15:28:19.583085 2014-08-13 18:06:06.565087 7f5c973a57a0 20 read_log 192834'13577 (192834'13576) modify bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.1883634.0:24807 2014-08-11 15:28:19.551421 2014-08-13 18:06:06.565102 7f5c973a57a0 20 read_log 192834'13581 (0'0) promote 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:38262 2014-08-11 15:28:19.954376 2014-08-13 18:06:06.565117 7f5c973a57a0 20 read_log 192834'13582 (192834'13581) modify 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.1883634.0:25012 2014-08-11 15:28:19.927770 2014-08-13 18:06:06.565129 7f5c973a57a0 20 read_log 192834'13591 (192834'13186) modify acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.1883634.0:25646 2014-08-11 15:28:21.670081 2014-08-13 18:06:06.565144 7f5c973a57a0 20 read_log 192834'13594 (192834'13192) modify 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.1883634.0:26054 2014-08-11 15:28:22.566345 2014-08-13 18:06:06.565374 7f5c973a57a0 20 read_log 192834'13597 (192834'13198) modify 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.1883634.0:26546 2014-08-11 15:28:23.244525 2014-08-13 18:06:06.565389 7f5c973a57a0 20 read_log 192834'13602 (192834'13208) modify 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.1883634.0:27293 2014-08-11 15:28:24.351028 2014-08-13 18:06:06.565414 7f5c973a57a0 20 read_log 192834'13603 (192834'13210) modify 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.1883634.0:27350 2014-08-11 15:28:24.395939 2014-08-13 18:06:06.565428 7f5c973a57a0 20 read_log 192834'13605 (192834'13214) modify eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.1883634.0:27696 2014-08-11 15:28:24.831542 2014-08-13 18:06:06.565440 7f5c973a57a0 20 read_log 192834'13607 (192834'13218) modify 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.1883634.0:27780 2014-08-11 15:28:24.970154 2014-08-13 18:06:06.565466 7f5c973a57a0 20 read_log 192834'13609 (192834'13222) modify d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.1883634.0:28891 2014-08-11 15:28:26.458499 2014-08-13 18:06:06.565490 7f5c973a57a0 20 read_log 192834'13615 (192834'13234) modify 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.1883634.0:29181 2014-08-11 15:28:26.826818 2014-08-13 18:06:06.565505 7f5c973a57a0 20 read_log 192834'13617 (192834'13238) modify 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.1883634.0:29251 2014-08-11 15:28:26.891325 2014-08-13 18:06:06.565535 7f5c973a57a0 20 read_log 192834'13626 (192834'13256) modify e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.1883634.0:30460 2014-08-11 15:28:28.658110 2014-08-13 18:06:06.565550 7f5c973a57a0 20 read_log 192834'13632 (0'0) promote c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:38385 2014-08-11 15:28:29.974667 2014-08-13 18:06:06.565563 7f5c973a57a0 20 read_log 192834'13633 (192834'13632) modify c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.1883634.0:31131 2014-08-11 15:28:29.940288 2014-08-13 18:06:06.565576 7f5c973a57a0 20 read_log 192834'13634 (192834'13270) modify 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.1883634.0:31306 2014-08-11 15:28:30.255515 2014-08-13 18:06:06.565589 7f5c973a57a0 20 read_log 192834'13653 (192834'13308) modify b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.1883634.0:33566 2014-08-11 15:28:33.491079 2014-08-13 18:06:06.565601 7f5c973a57a0 20 read_log 192834'13656 (192834'13314) modify 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.1883634.0:33907 2014-08-11 15:28:33.989691 2014-08-13 18:06:06.565613 7f5c973a57a0 20 read_log 192834'13661 (0'0) promote 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:38435 2014-08-11 15:28:35.803610 2014-08-13 18:06:06.565625 7f5c973a57a0 20 read_log 192834'13662 (192834'13661) modify 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.1883634.0:34994 2014-08-11 15:28:35.779699 2014-08-13 18:06:06.565638 7f5c973a57a0 20 read_log 192834'13665 (0'0) promote 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:38439 2014-08-11 15:28:36.119357 2014-08-13 18:06:06.565650 7f5c973a57a0 20 read_log 192834'13666 (192834'13665) modify 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.1883634.0:35144 2014-08-11 15:28:36.092811 2014-08-13 18:06:06.565662 7f5c973a57a0 20 read_log 192834'13668 (192834'13668) modify 34/hit_set_24.34_archive_2014-08-11 15:23:36.202668_2014-08-11 15:28:36.550941/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:36.551245 2014-08-13 18:06:06.565675 7f5c973a57a0 20 read_log 192834'13669 (192834'12664) delete 34/hit_set_24.34_archive_2014-08-11 10:24:43.621609_2014-08-11 15:23:36.202456/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:36.551245 2014-08-13 18:06:06.565701 7f5c973a57a0 20 read_log 192834'13676 (192834'13346) modify be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.1883634.0:35978 2014-08-11 15:28:37.421767 2014-08-13 18:06:06.565732 7f5c973a57a0 20 read_log 192834'13695 (0'0) promote 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:38501 2014-08-11 15:28:39.988239 2014-08-13 18:06:06.565745 7f5c973a57a0 20 read_log 192834'13696 (192834'13695) modify 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.1883634.0:37927 2014-08-11 15:28:39.963727 2014-08-13 18:06:06.565760 7f5c973a57a0 20 read_log 192834'13715 (192834'13418) modify ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.1883634.0:40713 2014-08-11 15:28:44.163345 2014-08-13 18:06:06.565784 7f5c973a57a0 20 read_log 192834'13725 (192834'13438) modify af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.1883634.0:42241 2014-08-11 15:28:46.179940 2014-08-13 18:06:06.565821 7f5c973a57a0 20 read_log 192834'13727 (192834'13442) modify c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.1883634.0:42338 2014-08-11 15:28:46.212260 2014-08-13 18:06:06.565837 7f5c973a57a0 20 read_log 192834'13729 (192834'13446) modify 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.1883634.0:42569 2014-08-11 15:28:46.284432 2014-08-13 18:06:06.565861 7f5c973a57a0 20 read_log 192834'13738 (192834'13464) modify 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.1883634.0:43321 2014-08-11 15:28:46.519268 2014-08-13 18:06:06.565886 7f5c973a57a0 20 read_log 192834'13739 (192834'13466) modify 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.1883634.0:43326 2014-08-11 15:28:46.521225 2014-08-13 18:06:06.565909 7f5c973a57a0 20 read_log 192834'13756 (192834'13500) modify 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.1883634.0:45830 2014-08-11 15:28:47.404291 2014-08-13 18:06:06.565922 7f5c973a57a0 20 read_log 192888'13766 (192888'13766) modify 34/hit_set_24.34_archive_2014-08-11 16:02:02.123078_2014-08-11 16:44:55.132899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.133197 2014-08-13 18:06:06.565936 7f5c973a57a0 20 read_log 192888'13767 (192834'13668) delete 34/hit_set_24.34_archive_2014-08-11 15:23:36.202668_2014-08-11 15:28:36.550941/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.133197 2014-08-13 18:06:06.565948 7f5c973a57a0 20 read_log 192888'13776 (0'0) promote ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by osd.325.0:41204 2014-08-11 16:45:03.225548 2014-08-13 18:06:06.565962 7f5c973a57a0 20 read_log 192888'13777 (192888'13776) modify ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.1315259.0:2095 2014-08-11 16:45:03.216217 2014-08-13 18:06:06.565985 7f5c973a57a0 20 read_log 192888'13780 (0'0) promote 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by osd.325.0:41220 2014-08-11 16:45:07.114793 2014-08-13 18:06:06.565998 7f5c973a57a0 20 read_log 192888'13781 (192888'13780) modify 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.1315259.0:3126 2014-08-11 16:45:07.106157 2014-08-13 18:06:06.566029 7f5c973a57a0 20 read_log 192888'13786 (0'0) promote e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by osd.325.0:41278 2014-08-11 16:45:22.725451 2014-08-13 18:06:06.566045 7f5c973a57a0 20 read_log 192888'13787 (192888'13786) modify e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.1315259.0:7367 2014-08-11 16:45:22.719005 2014-08-13 18:06:06.566058 7f5c973a57a0 20 read_log 192888'13792 (0'0) promote 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by osd.325.0:41312 2014-08-11 16:45:30.170176 2014-08-13 18:06:06.566072 7f5c973a57a0 20 read_log 192888'13793 (192888'13792) modify 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.1315259.0:9364 2014-08-11 16:45:30.162162 2014-08-13 18:06:06.566085 7f5c973a57a0 20 read_log 192888'13794 (0'0) promote f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by osd.325.0:41314 2014-08-11 16:45:30.498376 2014-08-13 18:06:06.566099 7f5c973a57a0 20 read_log 192888'13795 (192888'13794) modify f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.1315259.0:9428 2014-08-11 16:45:30.489392 2014-08-13 18:06:06.566111 7f5c973a57a0 20 read_log 192888'13800 (0'0) promote 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by osd.325.0:41328 2014-08-11 16:45:32.235728 2014-08-13 18:06:06.566126 7f5c973a57a0 20 read_log 192888'13801 (192888'13800) modify 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.1315259.0:9881 2014-08-11 16:45:32.228172 2014-08-13 18:06:06.566156 7f5c973a57a0 20 read_log 192888'13804 (0'0) promote a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by osd.325.0:41350 2014-08-11 16:45:41.176435 2014-08-13 18:06:06.566172 7f5c973a57a0 20 read_log 192888'13805 (192888'13804) modify a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.1315259.0:12125 2014-08-11 16:45:41.166092 2014-08-13 18:06:06.566185 7f5c973a57a0 20 read_log 192888'13808 (0'0) promote a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by osd.325.0:41362 2014-08-11 16:45:43.730156 2014-08-13 18:06:06.566197 7f5c973a57a0 20 read_log 192888'13809 (192888'13808) modify a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.1315259.0:12746 2014-08-11 16:45:43.721723 2014-08-13 18:06:06.566210 7f5c973a57a0 20 read_log 192888'13824 (0'0) promote 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by osd.325.0:41444 2014-08-11 16:46:03.397114 2014-08-13 18:06:06.566224 7f5c973a57a0 20 read_log 192888'13825 (192888'13824) modify 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.1315259.0:17756 2014-08-11 16:46:03.388532 2014-08-13 18:06:06.566236 7f5c973a57a0 20 read_log 192888'13826 (0'0) promote 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by osd.325.0:41470 2014-08-11 16:46:11.096695 2014-08-13 18:06:06.566251 7f5c973a57a0 20 read_log 192888'13827 (192888'13826) modify 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.1315259.0:19720 2014-08-11 16:46:11.087923 2014-08-13 18:06:06.566266 7f5c973a57a0 20 read_log 192888'13836 (0'0) promote 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by osd.325.0:41494 2014-08-11 16:46:13.923117 2014-08-13 18:06:06.566278 7f5c973a57a0 20 read_log 192888'13837 (192888'13836) modify 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.1315259.0:20487 2014-08-11 16:46:13.914844 2014-08-13 18:06:06.566291 7f5c973a57a0 20 read_log 192888'13844 (0'0) promote 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by osd.325.0:41520 2014-08-11 16:46:20.301034 2014-08-13 18:06:06.566303 7f5c973a57a0 20 read_log 192888'13845 (192888'13844) modify 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.1315259.0:22152 2014-08-11 16:46:20.292833 2014-08-13 18:06:06.566315 7f5c973a57a0 20 read_log 192888'13846 (0'0) promote ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by osd.325.0:41524 2014-08-11 16:46:20.945616 2014-08-13 18:06:06.566327 7f5c973a57a0 20 read_log 192888'13847 (192888'13846) modify ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.1315259.0:22328 2014-08-11 16:46:20.937522 2014-08-13 18:06:06.566339 7f5c973a57a0 20 read_log 192888'13848 (0'0) promote d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by osd.325.0:41526 2014-08-11 16:46:20.959256 2014-08-13 18:06:06.566351 7f5c973a57a0 20 read_log 192888'13849 (192888'13848) modify d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.1315259.0:22331 2014-08-11 16:46:20.948212 2014-08-13 18:06:06.566374 7f5c973a57a0 20 read_log 192888'13852 (0'0) promote 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by osd.325.0:41544 2014-08-11 16:46:23.958695 2014-08-13 18:06:06.566391 7f5c973a57a0 20 read_log 192888'13853 (192888'13852) modify 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.1315259.0:23065 2014-08-11 16:46:23.950295 2014-08-13 18:06:06.566403 7f5c973a57a0 20 read_log 192888'13856 (0'0) promote 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by osd.325.0:41564 2014-08-11 16:46:28.965343 2014-08-13 18:06:06.566432 7f5c973a57a0 20 read_log 192888'13857 (192888'13856) modify 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.1315259.0:24353 2014-08-11 16:46:28.956645 2014-08-13 18:06:06.566453 7f5c973a57a0 20 read_log 192888'13858 (0'0) promote 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by osd.325.0:41566 2014-08-11 16:46:28.995305 2014-08-13 18:06:06.566467 7f5c973a57a0 20 read_log 192888'13859 (192888'13858) modify 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.1315259.0:24361 2014-08-11 16:46:28.985585 2014-08-13 18:06:06.566489 7f5c973a57a0 20 read_log 192888'13860 (0'0) promote fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by osd.325.0:41570 2014-08-11 16:46:29.341322 2014-08-13 18:06:06.566501 7f5c973a57a0 20 read_log 192888'13861 (192888'13860) modify fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.1315259.0:24447 2014-08-11 16:46:29.332786 2014-08-13 18:06:06.566514 7f5c973a57a0 20 read_log 192888'13868 (0'0) promote 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by osd.325.0:41600 2014-08-11 16:46:36.822492 2014-08-13 18:06:06.566537 7f5c973a57a0 20 read_log 192888'13869 (192888'13868) modify 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.1315259.0:26254 2014-08-11 16:46:36.810318 2014-08-13 18:06:06.566550 7f5c973a57a0 20 read_log 192888'13872 (0'0) promote 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by osd.325.0:41616 2014-08-11 16:46:39.085616 2014-08-13 18:06:06.566562 7f5c973a57a0 20 read_log 192888'13873 (192888'13872) modify 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.1315259.0:26804 2014-08-11 16:46:39.077378 2014-08-13 18:06:06.566580 7f5c973a57a0 20 read_log 192888'13878 (0'0) promote da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by osd.325.0:41642 2014-08-11 16:46:44.960106 2014-08-13 18:06:06.566594 7f5c973a57a0 20 read_log 192888'13879 (192888'13878) modify da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.1315259.0:28234 2014-08-11 16:46:44.952650 2014-08-13 18:06:06.566607 7f5c973a57a0 20 read_log 192888'13882 (0'0) promote 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by osd.325.0:41660 2014-08-11 16:46:47.909597 2014-08-13 18:06:06.566619 7f5c973a57a0 20 read_log 192888'13883 (192888'13882) modify 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.1315259.0:28969 2014-08-11 16:46:47.898577 2014-08-13 18:06:06.566642 7f5c973a57a0 20 read_log 192888'13884 (0'0) promote aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by osd.325.0:41674 2014-08-11 16:46:49.926443 2014-08-13 18:06:06.566657 7f5c973a57a0 20 read_log 192888'13885 (192888'13884) modify aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.1315259.0:29450 2014-08-11 16:46:49.916371 2014-08-13 18:06:06.566670 7f5c973a57a0 20 read_log 192888'13886 (0'0) promote 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by osd.325.0:41682 2014-08-11 16:46:51.389969 2014-08-13 18:06:06.566682 7f5c973a57a0 20 read_log 192888'13887 (192888'13886) modify 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.1315259.0:29808 2014-08-11 16:46:51.382545 2014-08-13 18:06:06.566694 7f5c973a57a0 20 read_log 192888'13888 (0'0) promote aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by osd.325.0:41684 2014-08-11 16:46:52.354474 2014-08-13 18:06:06.566715 7f5c973a57a0 20 read_log 192888'13889 (192888'13888) modify aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.1315259.0:30065 2014-08-11 16:46:52.346807 2014-08-13 18:06:06.566738 7f5c973a57a0 20 read_log 192888'13894 (192888'13777) modify ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.1315259.0:32822 2014-08-11 16:46:56.479027 2014-08-13 18:06:06.566753 7f5c973a57a0 20 read_log 192888'13896 (192888'13781) modify 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.1315259.0:33853 2014-08-11 16:46:56.741739 2014-08-13 18:06:06.566774 7f5c973a57a0 20 read_log 192888'13899 (192888'13787) modify e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.1315259.0:38094 2014-08-11 16:46:57.855674 2014-08-13 18:06:06.566787 7f5c973a57a0 20 read_log 192888'13902 (192888'13793) modify 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.1315259.0:40091 2014-08-11 16:46:58.384975 2014-08-13 18:06:06.566799 7f5c973a57a0 20 read_log 192888'13903 (192888'13795) modify f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.1315259.0:40155 2014-08-11 16:46:58.404083 2014-08-13 18:06:06.566811 7f5c973a57a0 20 read_log 192888'13906 (192888'13801) modify 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.1315259.0:40608 2014-08-11 16:46:58.536116 2014-08-13 18:06:06.566823 7f5c973a57a0 20 read_log 192888'13908 (192888'13805) modify a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.1315259.0:42852 2014-08-11 16:46:59.122739 2014-08-13 18:06:06.566855 7f5c973a57a0 20 read_log 192888'13910 (192888'13809) modify a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.1315259.0:43473 2014-08-11 16:46:59.301445 2014-08-13 18:06:06.566871 7f5c973a57a0 20 read_log 192888'13918 (192888'13825) modify 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.1315259.0:48483 2014-08-11 16:47:00.775823 2014-08-13 18:06:06.566884 7f5c973a57a0 20 read_log 192888'13919 (192888'13827) modify 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.1315259.0:50447 2014-08-11 16:47:01.372700 2014-08-13 18:06:06.566896 7f5c973a57a0 20 read_log 192888'13924 (192888'13837) modify 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.1315259.0:51214 2014-08-11 16:47:01.612613 2014-08-13 18:06:06.566909 7f5c973a57a0 20 read_log 192888'13928 (192888'13845) modify 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.1315259.0:52879 2014-08-11 16:47:02.094252 2014-08-13 18:06:06.566921 7f5c973a57a0 20 read_log 192888'13929 (192888'13847) modify ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.1315259.0:53055 2014-08-11 16:47:02.142537 2014-08-13 18:06:06.566933 7f5c973a57a0 20 read_log 192888'13930 (192888'13849) modify d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.1315259.0:53058 2014-08-11 16:47:02.143193 2014-08-13 18:06:06.566945 7f5c973a57a0 20 read_log 192888'13932 (192888'13853) modify 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.1315259.0:53792 2014-08-11 16:47:02.363870 2014-08-13 18:06:06.566957 7f5c973a57a0 20 read_log 192888'13934 (192888'13857) modify 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.1315259.0:55080 2014-08-11 16:47:02.804344 2014-08-13 18:06:06.566969 7f5c973a57a0 20 read_log 192888'13935 (192888'13859) modify 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.1315259.0:55088 2014-08-11 16:47:02.807175 2014-08-13 18:06:06.566982 7f5c973a57a0 20 read_log 192888'13936 (192888'13861) modify fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.1315259.0:55174 2014-08-11 16:47:02.833347 2014-08-13 18:06:06.566994 7f5c973a57a0 20 read_log 192888'13940 (192888'13869) modify 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.1315259.0:56981 2014-08-11 16:47:03.372659 2014-08-13 18:06:06.567025 7f5c973a57a0 20 read_log 192888'13942 (192888'13873) modify 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.1315259.0:57531 2014-08-11 16:47:03.541536 2014-08-13 18:06:06.567040 7f5c973a57a0 20 read_log 192888'13945 (192888'13879) modify da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.1315259.0:58961 2014-08-11 16:47:03.958789 2014-08-13 18:06:06.567053 7f5c973a57a0 20 read_log 192888'13947 (192888'13883) modify 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.1315259.0:59696 2014-08-11 16:47:04.213409 2014-08-13 18:06:06.567075 7f5c973a57a0 20 read_log 192888'13948 (192888'13885) modify aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.1315259.0:60177 2014-08-11 16:47:04.370420 2014-08-13 18:06:06.567091 7f5c973a57a0 20 read_log 192888'13949 (192888'13887) modify 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.1315259.0:60535 2014-08-11 16:47:04.470210 2014-08-13 18:06:06.567103 7f5c973a57a0 20 read_log 192888'13950 (192888'13889) modify aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.1315259.0:60792 2014-08-11 16:47:04.542488 2014-08-13 18:06:06.567115 7f5c973a57a0 20 read_log 192888'13951 (192888'13951) modify 34/hit_set_24.34_archive_2014-08-11 16:44:55.133201_2014-08-11 17:05:14.852362/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.852638 2014-08-13 18:06:06.567140 7f5c973a57a0 20 read_log 192888'13952 (192888'13766) delete 34/hit_set_24.34_archive_2014-08-11 16:02:02.123078_2014-08-11 16:44:55.132899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.852638 2014-08-13 18:06:06.567165 7f5c973a57a0 20 read_log 192888'13955 (0'0) promote 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by osd.325.0:41968 2014-08-11 17:05:17.483027 2014-08-13 18:06:06.567178 7f5c973a57a0 20 read_log 192888'13956 (192888'13955) modify 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.942868.0:603 2014-08-11 17:05:17.474818 2014-08-13 18:06:06.567190 7f5c973a57a0 20 read_log 192888'13957 (0'0) promote aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by osd.325.0:41979 2014-08-11 17:05:18.921701 2014-08-13 18:06:06.567203 7f5c973a57a0 20 read_log 192888'13958 (192888'13957) modify aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.942868.0:957 2014-08-11 17:05:18.911644 2014-08-13 18:06:06.567218 7f5c973a57a0 20 read_log 192888'13967 (0'0) promote 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by osd.325.0:42036 2014-08-11 17:05:30.136811 2014-08-13 18:06:06.567231 7f5c973a57a0 20 read_log 192888'13968 (192888'13967) modify 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.942868.0:3837 2014-08-11 17:05:30.128369 2014-08-13 18:06:06.567243 7f5c973a57a0 20 read_log 192888'13969 (0'0) promote d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by osd.325.0:42060 2014-08-11 17:05:32.614452 2014-08-13 18:06:06.567256 7f5c973a57a0 20 read_log 192888'13970 (192888'13969) modify d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.942868.0:4508 2014-08-11 17:05:32.605964 2014-08-13 18:06:06.567274 7f5c973a57a0 20 read_log 192888'13975 (0'0) promote 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by osd.325.0:42108 2014-08-11 17:05:45.572580 2014-08-13 18:06:06.567287 7f5c973a57a0 20 read_log 192888'13976 (192888'13975) modify 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.942868.0:7929 2014-08-11 17:05:45.564407 2014-08-13 18:06:06.567299 7f5c973a57a0 20 read_log 192888'13977 (0'0) promote 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by osd.325.0:42110 2014-08-11 17:05:45.625064 2014-08-13 18:06:06.567315 7f5c973a57a0 20 read_log 192888'13978 (192888'13977) modify 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.942868.0:7944 2014-08-11 17:05:45.618097 2014-08-13 18:06:06.567330 7f5c973a57a0 20 read_log 192888'13981 (0'0) promote 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by osd.325.0:42126 2014-08-11 17:05:49.466104 2014-08-13 18:06:06.567352 7f5c973a57a0 20 read_log 192888'13982 (192888'13981) modify 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.942868.0:8984 2014-08-11 17:05:49.457056 2014-08-13 18:06:06.567402 7f5c973a57a0 20 read_log 192888'13985 (0'0) promote 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by osd.325.0:42164 2014-08-11 17:05:56.585904 2014-08-13 18:06:06.567419 7f5c973a57a0 20 read_log 192888'13986 (192888'13985) modify 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.942868.0:10941 2014-08-11 17:05:56.577607 2014-08-13 18:06:06.567431 7f5c973a57a0 20 read_log 192888'13987 (0'0) promote d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by osd.325.0:42184 2014-08-11 17:06:00.406601 2014-08-13 18:06:06.567455 7f5c973a57a0 20 read_log 192888'13988 (192888'13987) modify d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.942868.0:11971 2014-08-11 17:06:00.398551 2014-08-13 18:06:06.567468 7f5c973a57a0 20 read_log 192888'13991 (0'0) promote 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by osd.325.0:42212 2014-08-11 17:06:04.404024 2014-08-13 18:06:06.567481 7f5c973a57a0 20 read_log 192888'13992 (192888'13991) modify 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.942868.0:13059 2014-08-11 17:06:04.395783 2014-08-13 18:06:06.567494 7f5c973a57a0 20 read_log 192888'13993 (0'0) promote 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by osd.325.0:42222 2014-08-11 17:06:07.575289 2014-08-13 18:06:06.567506 7f5c973a57a0 20 read_log 192888'13994 (192888'13993) modify 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.942868.0:13853 2014-08-11 17:06:07.566720 2014-08-13 18:06:06.567518 7f5c973a57a0 20 read_log 192888'13997 (0'0) promote 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by osd.325.0:42238 2014-08-11 17:06:11.813054 2014-08-13 18:06:06.567535 7f5c973a57a0 20 read_log 192888'13998 (192888'13997) modify 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.942868.0:14895 2014-08-11 17:06:11.804229 2014-08-13 18:06:06.567548 7f5c973a57a0 20 read_log 192888'13999 (0'0) promote ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by osd.325.0:42250 2014-08-11 17:06:13.011641 2014-08-13 18:06:06.567560 7f5c973a57a0 20 read_log 192888'14000 (192888'13999) modify ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.942868.0:15202 2014-08-11 17:06:13.001241 2014-08-13 18:06:06.567572 7f5c973a57a0 20 read_log 192888'14003 (0'0) promote 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by osd.325.0:42270 2014-08-11 17:06:16.388644 2014-08-13 18:06:06.567584 7f5c973a57a0 20 read_log 192888'14004 (192888'14003) modify 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.942868.0:16072 2014-08-11 17:06:16.379378 2014-08-13 18:06:06.567596 7f5c973a57a0 20 read_log 192888'14007 (0'0) promote 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by osd.325.0:42294 2014-08-11 17:06:20.659334 2014-08-13 18:06:06.567609 7f5c973a57a0 20 read_log 192888'14008 (192888'14007) modify 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.942868.0:17158 2014-08-11 17:06:20.649920 2014-08-13 18:06:06.567621 7f5c973a57a0 20 read_log 192888'14009 (0'0) promote dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by osd.325.0:42298 2014-08-11 17:06:22.178318 2014-08-13 18:06:06.567636 7f5c973a57a0 20 read_log 192888'14010 (192888'14009) modify dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.942868.0:17560 2014-08-11 17:06:22.171320 2014-08-13 18:06:06.567649 7f5c973a57a0 20 read_log 192888'14011 (0'0) promote 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by osd.325.0:42318 2014-08-11 17:06:25.165994 2014-08-13 18:06:06.567661 7f5c973a57a0 20 read_log 192888'14012 (192888'14011) modify 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.942868.0:18352 2014-08-11 17:06:25.156221 2014-08-13 18:06:06.567693 7f5c973a57a0 20 read_log 192888'14013 (0'0) promote 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by osd.325.0:42322 2014-08-11 17:06:25.594173 2014-08-13 18:06:06.567709 7f5c973a57a0 20 read_log 192888'14014 (192888'14013) modify 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.942868.0:18467 2014-08-11 17:06:25.586179 2014-08-13 18:06:06.567722 7f5c973a57a0 20 read_log 192888'14015 (0'0) promote b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by osd.325.0:42326 2014-08-11 17:06:26.409435 2014-08-13 18:06:06.567734 7f5c973a57a0 20 read_log 192888'14016 (192888'14015) modify b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.942868.0:18682 2014-08-11 17:06:26.400579 2014-08-13 18:06:06.567746 7f5c973a57a0 20 read_log 192888'14017 (0'0) promote afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by osd.325.0:42338 2014-08-11 17:06:30.172064 2014-08-13 18:06:06.567758 7f5c973a57a0 20 read_log 192888'14018 (192888'14017) modify afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.942868.0:19686 2014-08-11 17:06:30.163483 2014-08-13 18:06:06.567771 7f5c973a57a0 20 read_log 192888'14019 (0'0) promote f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by osd.325.0:42346 2014-08-11 17:06:31.992911 2014-08-13 18:06:06.567783 7f5c973a57a0 20 read_log 192888'14020 (192888'14019) modify f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.942868.0:20169 2014-08-11 17:06:31.983404 2014-08-13 18:06:06.567795 7f5c973a57a0 20 read_log 192888'14021 (0'0) promote b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by osd.325.0:42358 2014-08-11 17:06:33.484075 2014-08-13 18:06:06.567808 7f5c973a57a0 20 read_log 192888'14022 (192888'14021) modify b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.942868.0:20547 2014-08-11 17:06:33.474245 2014-08-13 18:06:06.567823 7f5c973a57a0 20 read_log 192888'14023 (0'0) promote 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by osd.325.0:42364 2014-08-11 17:06:35.222159 2014-08-13 18:06:06.567835 7f5c973a57a0 20 read_log 192888'14024 (192888'14023) modify 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.942868.0:21006 2014-08-11 17:06:35.213284 2014-08-13 18:06:06.567847 7f5c973a57a0 20 read_log 192888'14025 (0'0) promote 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by osd.325.0:42368 2014-08-11 17:06:35.728586 2014-08-13 18:06:06.567859 7f5c973a57a0 20 read_log 192888'14026 (192888'14025) modify 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.942868.0:21137 2014-08-11 17:06:35.718903 2014-08-13 18:06:06.567874 7f5c973a57a0 20 read_log 192888'14029 (0'0) promote 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by osd.325.0:42378 2014-08-11 17:06:36.805887 2014-08-13 18:06:06.567887 7f5c973a57a0 20 read_log 192888'14030 (192888'14029) modify 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.942868.0:21417 2014-08-11 17:06:36.798097 2014-08-13 18:06:06.567900 7f5c973a57a0 20 read_log 192888'14031 (0'0) promote b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by osd.325.0:42380 2014-08-11 17:06:37.379137 2014-08-13 18:06:06.567914 7f5c973a57a0 20 read_log 192888'14032 (192888'14031) modify b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.942868.0:21571 2014-08-11 17:06:37.368920 2014-08-13 18:06:06.567926 7f5c973a57a0 20 read_log 192888'14037 (0'0) promote b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by osd.325.0:42438 2014-08-11 17:06:49.187131 2014-08-13 18:06:06.567938 7f5c973a57a0 20 read_log 192888'14038 (192888'14037) modify b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.942868.0:24668 2014-08-11 17:06:49.177825 2014-08-13 18:06:06.567961 7f5c973a57a0 20 read_log 192888'14039 (0'0) promote 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by osd.325.0:42440 2014-08-11 17:06:49.798357 2014-08-13 18:06:06.567976 7f5c973a57a0 20 read_log 192888'14040 (192888'14039) modify 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.942868.0:24834 2014-08-11 17:06:49.788888 2014-08-13 18:06:06.567989 7f5c973a57a0 20 read_log 192888'14041 (0'0) promote 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by osd.325.0:42458 2014-08-11 17:06:51.833551 2014-08-13 18:06:06.568003 7f5c973a57a0 20 read_log 192888'14042 (192888'14041) modify 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.942868.0:25374 2014-08-11 17:06:51.824651 2014-08-13 18:06:06.568015 7f5c973a57a0 20 read_log 192888'14043 (0'0) promote 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by osd.325.0:42460 2014-08-11 17:06:52.213821 2014-08-13 18:06:06.568029 7f5c973a57a0 20 read_log 192888'14044 (192888'14043) modify 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.942868.0:25476 2014-08-11 17:06:52.205502 2014-08-13 18:06:06.568043 7f5c973a57a0 20 read_log 192888'14045 (0'0) promote a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by osd.325.0:42468 2014-08-11 17:06:54.688180 2014-08-13 18:06:06.568057 7f5c973a57a0 20 read_log 192888'14046 (192888'14045) modify a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.942868.0:26066 2014-08-11 17:06:54.679117 2014-08-13 18:06:06.568070 7f5c973a57a0 20 read_log 192888'14049 (0'0) promote 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by osd.325.0:42488 2014-08-11 17:06:57.804947 2014-08-13 18:06:06.568084 7f5c973a57a0 20 read_log 192888'14050 (192888'14049) modify 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.942868.0:26884 2014-08-11 17:06:57.794638 2014-08-13 18:06:06.568098 7f5c973a57a0 20 read_log 192888'14051 (0'0) promote 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by osd.325.0:42490 2014-08-11 17:06:57.920508 2014-08-13 18:06:06.568112 7f5c973a57a0 20 read_log 192888'14052 (192888'14051) modify 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.942868.0:26916 2014-08-11 17:06:57.910888 2014-08-13 18:06:06.568124 7f5c973a57a0 20 read_log 192888'14053 (0'0) promote e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by osd.325.0:42496 2014-08-11 17:06:58.867378 2014-08-13 18:06:06.568137 7f5c973a57a0 20 read_log 192888'14054 (192888'14053) modify e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.942868.0:27174 2014-08-11 17:06:58.858282 2014-08-13 18:06:06.568158 7f5c973a57a0 20 read_log 192888'14055 (0'0) promote e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by osd.325.0:42514 2014-08-11 17:07:01.059380 2014-08-13 18:06:06.568172 7f5c973a57a0 20 read_log 192888'14056 (192888'14055) modify e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.942868.0:27749 2014-08-11 17:07:01.051225 2014-08-13 18:06:06.568185 7f5c973a57a0 20 read_log 192888'14063 (0'0) promote 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by osd.325.0:42568 2014-08-11 17:07:13.975232 2014-08-13 18:06:06.568197 7f5c973a57a0 20 read_log 192888'14064 (192888'14063) modify 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.942868.0:31044 2014-08-11 17:07:13.965349 2014-08-13 18:06:06.568209 7f5c973a57a0 20 read_log 192888'14066 (192888'13956) modify 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.942868.0:31869 2014-08-11 17:07:16.082077 2014-08-13 18:06:06.568222 7f5c973a57a0 20 read_log 192888'14067 (192888'13958) modify aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.942868.0:32223 2014-08-11 17:07:16.192158 2014-08-13 18:06:06.568244 7f5c973a57a0 20 read_log 192888'14072 (192888'13968) modify 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.942868.0:35103 2014-08-11 17:07:16.984859 2014-08-13 18:06:06.568260 7f5c973a57a0 20 read_log 192888'14073 (192888'13970) modify d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.942868.0:35774 2014-08-11 17:07:17.160851 2014-08-13 18:06:06.568272 7f5c973a57a0 20 read_log 192888'14076 (192888'13976) modify 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.942868.0:39195 2014-08-11 17:07:18.113142 2014-08-13 18:06:06.568284 7f5c973a57a0 20 read_log 192888'14077 (192888'13978) modify 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.942868.0:39210 2014-08-11 17:07:18.118746 2014-08-13 18:06:06.568301 7f5c973a57a0 20 read_log 192888'14079 (192888'13982) modify 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.942868.0:40250 2014-08-11 17:07:18.402282 2014-08-13 18:06:06.568335 7f5c973a57a0 20 read_log 192888'14081 (192888'13986) modify 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.942868.0:42207 2014-08-11 17:07:18.935502 2014-08-13 18:06:06.568354 7f5c973a57a0 20 read_log 192888'14082 (192888'13988) modify d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.942868.0:43237 2014-08-11 17:07:19.217169 2014-08-13 18:06:06.568376 7f5c973a57a0 20 read_log 192888'14084 (192888'13992) modify 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.942868.0:44325 2014-08-11 17:07:19.535749 2014-08-13 18:06:06.568395 7f5c973a57a0 20 read_log 192888'14085 (192888'13994) modify 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.942868.0:45119 2014-08-11 17:07:19.751046 2014-08-13 18:06:06.568409 7f5c973a57a0 20 read_log 192888'14087 (192888'13998) modify 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.942868.0:46161 2014-08-11 17:07:20.039116 2014-08-13 18:06:06.568429 7f5c973a57a0 20 read_log 192888'14088 (192888'14000) modify ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.942868.0:46468 2014-08-11 17:07:20.131420 2014-08-13 18:06:06.568463 7f5c973a57a0 20 read_log 192888'14090 (192888'14004) modify 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.942868.0:47338 2014-08-11 17:07:20.375451 2014-08-13 18:06:06.568490 7f5c973a57a0 20 read_log 192888'14092 (192888'14008) modify 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.942868.0:48424 2014-08-11 17:07:20.716059 2014-08-13 18:06:06.568503 7f5c973a57a0 20 read_log 192888'14093 (192888'14010) modify dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.942868.0:48826 2014-08-11 17:07:20.830387 2014-08-13 18:06:06.568518 7f5c973a57a0 20 read_log 192888'14094 (192888'14012) modify 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.942868.0:49618 2014-08-11 17:07:21.069658 2014-08-13 18:06:06.568531 7f5c973a57a0 20 read_log 192888'14095 (192888'14014) modify 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.942868.0:49733 2014-08-11 17:07:21.105046 2014-08-13 18:06:06.568544 7f5c973a57a0 20 read_log 192888'14096 (192888'14016) modify b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.942868.0:49948 2014-08-11 17:07:21.171232 2014-08-13 18:06:06.568556 7f5c973a57a0 20 read_log 192888'14097 (192888'14018) modify afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.942868.0:50952 2014-08-11 17:07:21.482590 2014-08-13 18:06:06.568581 7f5c973a57a0 20 read_log 192888'14098 (192888'14020) modify f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.942868.0:51435 2014-08-11 17:07:21.648490 2014-08-13 18:06:06.568594 7f5c973a57a0 20 read_log 192888'14099 (192888'14022) modify b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.942868.0:51813 2014-08-11 17:07:21.760172 2014-08-13 18:06:06.568615 7f5c973a57a0 20 read_log 192888'14100 (192888'14024) modify 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.942868.0:52272 2014-08-11 17:07:21.889317 2014-08-13 18:06:06.568627 7f5c973a57a0 20 read_log 192888'14101 (192888'14026) modify 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.942868.0:52403 2014-08-11 17:07:21.929550 2014-08-13 18:06:06.568639 7f5c973a57a0 20 read_log 192888'14103 (192888'14030) modify 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.942868.0:52683 2014-08-11 17:07:22.018787 2014-08-13 18:06:06.568652 7f5c973a57a0 20 read_log 192888'14104 (192888'14032) modify b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.942868.0:52837 2014-08-11 17:07:22.065225 2014-08-13 18:06:06.568664 7f5c973a57a0 20 read_log 192888'14107 (192888'14038) modify b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.942868.0:55934 2014-08-11 17:07:23.000304 2014-08-13 18:06:06.568676 7f5c973a57a0 20 read_log 192888'14108 (192888'14040) modify 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.942868.0:56100 2014-08-11 17:07:23.053218 2014-08-13 18:06:06.568689 7f5c973a57a0 20 read_log 192888'14109 (192888'14042) modify 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.942868.0:56640 2014-08-11 17:07:23.215269 2014-08-13 18:06:06.568701 7f5c973a57a0 20 read_log 192888'14110 (192888'14044) modify 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.942868.0:56742 2014-08-11 17:07:23.244998 2014-08-13 18:06:06.568714 7f5c973a57a0 20 read_log 192888'14111 (192888'14046) modify a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.942868.0:57332 2014-08-11 17:07:23.425329 2014-08-13 18:06:06.568726 7f5c973a57a0 20 read_log 192888'14113 (192888'14050) modify 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.942868.0:58150 2014-08-11 17:07:23.657057 2014-08-13 18:06:06.568738 7f5c973a57a0 20 read_log 192888'14114 (192888'14052) modify 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.942868.0:58182 2014-08-11 17:07:23.668807 2014-08-13 18:06:06.568751 7f5c973a57a0 20 read_log 192888'14115 (192888'14054) modify e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.942868.0:58440 2014-08-11 17:07:23.743641 2014-08-13 18:06:06.568763 7f5c973a57a0 20 read_log 192888'14116 (192888'14056) modify e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.942868.0:59015 2014-08-11 17:07:23.919817 2014-08-13 18:06:06.568775 7f5c973a57a0 20 read_log 192888'14120 (192888'14064) modify 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.942868.0:62310 2014-08-11 17:07:24.912843 2014-08-13 18:06:06.568787 7f5c973a57a0 20 read_log 193305'14121 (193305'14121) modify 34/hit_set_24.34_archive_2014-08-11 18:37:04.940322_2014-08-11 18:42:40.475022/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.475293 2014-08-13 18:06:06.568800 7f5c973a57a0 20 read_log 193305'14122 (192888'13951) delete 34/hit_set_24.34_archive_2014-08-11 16:44:55.133201_2014-08-11 17:05:14.852362/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.475293 2014-08-13 18:06:06.568812 7f5c973a57a0 20 read_log 193305'14123 (0'0) promote 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by osd.325.0:36 2014-08-11 18:42:40.789379 2014-08-13 18:06:06.568824 7f5c973a57a0 20 read_log 193305'14124 (193305'14123) modify 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.951337.0:1921 2014-08-11 18:42:40.470029 2014-08-13 18:06:06.568839 7f5c973a57a0 20 read_log 193305'14125 (0'0) promote 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by osd.325.0:50 2014-08-11 18:42:44.160651 2014-08-13 18:06:06.568861 7f5c973a57a0 20 read_log 193305'14126 (193305'14125) modify 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.951337.0:2937 2014-08-11 18:42:44.152798 2014-08-13 18:06:06.568874 7f5c973a57a0 20 read_log 193305'14127 (0'0) promote d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by osd.325.0:104 2014-08-11 18:42:56.364367 2014-08-13 18:06:06.568886 7f5c973a57a0 20 read_log 193305'14128 (193305'14127) modify d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.951337.0:6258 2014-08-11 18:42:56.356668 2014-08-13 18:06:06.568899 7f5c973a57a0 20 read_log 193305'14129 (0'0) promote b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by osd.325.0:110 2014-08-11 18:42:58.047160 2014-08-13 18:06:06.568911 7f5c973a57a0 20 read_log 193305'14130 (193305'14129) modify b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.951337.0:6724 2014-08-11 18:42:58.039624 2014-08-13 18:06:06.568923 7f5c973a57a0 20 read_log 193305'14131 (0'0) promote 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by osd.325.0:114 2014-08-11 18:42:58.417304 2014-08-13 18:06:06.568935 7f5c973a57a0 20 read_log 193305'14132 (193305'14131) modify 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.951337.0:6826 2014-08-11 18:42:58.408919 2014-08-13 18:06:06.568947 7f5c973a57a0 20 read_log 193305'14133 (0'0) promote 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by osd.325.0:126 2014-08-11 18:43:00.374239 2014-08-13 18:06:06.568963 7f5c973a57a0 20 read_log 193305'14134 (193305'14133) modify 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.951337.0:7369 2014-08-11 18:43:00.367151 2014-08-13 18:06:06.568975 7f5c973a57a0 20 read_log 193305'14135 (0'0) promote 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by osd.325.0:138 2014-08-11 18:43:04.626031 2014-08-13 18:06:06.568989 7f5c973a57a0 20 read_log 193305'14136 (193305'14135) modify 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.951337.0:8473 2014-08-11 18:43:04.617330 2014-08-13 18:06:06.569003 7f5c973a57a0 20 read_log 193305'14137 (0'0) promote 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by osd.325.0:146 2014-08-11 18:43:07.404195 2014-08-13 18:06:06.569015 7f5c973a57a0 20 read_log 193305'14138 (193305'14137) modify 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.951337.0:9213 2014-08-11 18:43:07.396514 2014-08-13 18:06:06.569031 7f5c973a57a0 20 read_log 193305'14139 (0'0) promote 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by osd.325.0:154 2014-08-11 18:43:08.882196 2014-08-13 18:06:06.569045 7f5c973a57a0 20 read_log 193305'14140 (193305'14139) modify 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.951337.0:9613 2014-08-11 18:43:08.872031 2014-08-13 18:06:06.569059 7f5c973a57a0 20 read_log 193305'14141 (0'0) promote cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by osd.325.0:158 2014-08-11 18:43:10.500576 2014-08-13 18:06:06.569074 7f5c973a57a0 20 read_log 193305'14142 (193305'14141) modify cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.951337.0:10057 2014-08-11 18:43:10.493514 2014-08-13 18:06:06.569089 7f5c973a57a0 20 read_log 193305'14143 (0'0) promote a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by osd.325.0:186 2014-08-11 18:43:16.619681 2014-08-13 18:06:06.569101 7f5c973a57a0 20 read_log 193305'14144 (193305'14143) modify a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.951337.0:11654 2014-08-11 18:43:16.611579 2014-08-13 18:06:06.569114 7f5c973a57a0 20 read_log 193305'14145 (0'0) promote e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by osd.325.0:204 2014-08-11 18:43:20.673946 2014-08-13 18:06:06.569135 7f5c973a57a0 20 read_log 193305'14146 (193305'14145) modify e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.951337.0:12742 2014-08-11 18:43:20.667152 2014-08-13 18:06:06.569155 7f5c973a57a0 20 read_log 193305'14147 (0'0) promote 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by osd.325.0:221 2014-08-11 18:43:23.995574 2014-08-13 18:06:06.569170 7f5c973a57a0 20 read_log 193305'14148 (193305'14147) modify 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.951337.0:13612 2014-08-11 18:43:23.984097 2014-08-13 18:06:06.569183 7f5c973a57a0 20 read_log 193305'14149 (0'0) promote 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by osd.325.0:225 2014-08-11 18:43:25.453120 2014-08-13 18:06:06.569195 7f5c973a57a0 20 read_log 193305'14150 (193305'14149) modify 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.951337.0:14020 2014-08-11 18:43:25.439836 2014-08-13 18:06:06.569207 7f5c973a57a0 20 read_log 193305'14151 (0'0) promote 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by osd.325.0:233 2014-08-11 18:43:26.668905 2014-08-13 18:06:06.569221 7f5c973a57a0 20 read_log 193305'14152 (193305'14151) modify 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.951337.0:14331 2014-08-11 18:43:26.660583 2014-08-13 18:06:06.569238 7f5c973a57a0 20 read_log 193305'14153 (0'0) promote 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by osd.325.0:256 2014-08-11 18:43:31.248568 2014-08-13 18:06:06.569252 7f5c973a57a0 20 read_log 193305'14154 (193305'14153) modify 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.951337.0:15665 2014-08-11 18:43:31.241310 2014-08-13 18:06:06.569265 7f5c973a57a0 20 read_log 193305'14155 (0'0) promote 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by osd.325.0:260 2014-08-11 18:43:31.765086 2014-08-13 18:06:06.569277 7f5c973a57a0 20 read_log 193305'14156 (193305'14155) modify 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.951337.0:15822 2014-08-11 18:43:31.754467 2014-08-13 18:06:06.569289 7f5c973a57a0 20 read_log 193305'14157 (0'0) promote 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by osd.325.0:264 2014-08-11 18:43:33.862421 2014-08-13 18:06:06.569305 7f5c973a57a0 20 read_log 193305'14158 (193305'14157) modify 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.951337.0:16403 2014-08-11 18:43:33.853089 2014-08-13 18:06:06.569320 7f5c973a57a0 20 read_log 193305'14159 (0'0) promote e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by osd.325.0:268 2014-08-11 18:43:34.496954 2014-08-13 18:06:06.569332 7f5c973a57a0 20 read_log 193305'14160 (193305'14159) modify e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.951337.0:16592 2014-08-11 18:43:34.487277 2014-08-13 18:06:06.569347 7f5c973a57a0 20 read_log 193305'14161 (0'0) promote 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by osd.325.0:272 2014-08-11 18:43:36.032652 2014-08-13 18:06:06.569370 7f5c973a57a0 20 read_log 193305'14162 (193305'14161) modify 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.951337.0:17021 2014-08-11 18:43:36.024623 2014-08-13 18:06:06.569386 7f5c973a57a0 20 read_log 193305'14163 (0'0) promote 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by osd.325.0:300 2014-08-11 18:43:44.528155 2014-08-13 18:06:06.569402 7f5c973a57a0 20 read_log 193305'14164 (193305'14163) modify 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.951337.0:19404 2014-08-11 18:43:44.520413 2014-08-13 18:06:06.569414 7f5c973a57a0 20 read_log 193305'14165 (0'0) promote dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by osd.325.0:304 2014-08-11 18:43:45.262872 2014-08-13 18:06:06.569451 7f5c973a57a0 20 read_log 193305'14166 (193305'14165) modify dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.951337.0:19616 2014-08-11 18:43:45.254685 2014-08-13 18:06:06.569467 7f5c973a57a0 20 read_log 193305'14167 (0'0) promote a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by osd.325.0:308 2014-08-11 18:43:45.730169 2014-08-13 18:06:06.569479 7f5c973a57a0 20 read_log 193305'14168 (193305'14167) modify a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.951337.0:19749 2014-08-11 18:43:45.721213 2014-08-13 18:06:06.569492 7f5c973a57a0 20 read_log 193305'14169 (0'0) promote 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by osd.325.0:314 2014-08-11 18:43:46.304030 2014-08-13 18:06:06.569513 7f5c973a57a0 20 read_log 193305'14170 (193305'14169) modify 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.951337.0:19915 2014-08-11 18:43:46.293920 2014-08-13 18:06:06.569529 7f5c973a57a0 20 read_log 193305'14171 (0'0) promote 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by osd.325.0:316 2014-08-11 18:43:47.054157 2014-08-13 18:06:06.569541 7f5c973a57a0 20 read_log 193305'14172 (193305'14171) modify 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.951337.0:20118 2014-08-11 18:43:47.045851 2014-08-13 18:06:06.569563 7f5c973a57a0 20 read_log 193305'14173 (0'0) promote ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by osd.325.0:320 2014-08-11 18:43:47.600773 2014-08-13 18:06:06.569581 7f5c973a57a0 20 read_log 193305'14174 (193305'14173) modify ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.951337.0:20255 2014-08-11 18:43:47.592872 2014-08-13 18:06:06.569607 7f5c973a57a0 20 read_log 193305'14175 (0'0) promote 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by osd.325.0:324 2014-08-11 18:43:47.863393 2014-08-13 18:06:06.569623 7f5c973a57a0 20 read_log 193305'14176 (193305'14175) modify 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.951337.0:20331 2014-08-11 18:43:47.854690 2014-08-13 18:06:06.569636 7f5c973a57a0 20 read_log 193305'14177 (0'0) promote 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by osd.325.0:340 2014-08-11 18:43:51.110074 2014-08-13 18:06:06.569648 7f5c973a57a0 20 read_log 193305'14178 (193305'14177) modify 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.951337.0:21250 2014-08-11 18:43:51.100374 2014-08-13 18:06:06.569660 7f5c973a57a0 20 read_log 193305'14179 (0'0) promote 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by osd.325.0:378 2014-08-11 18:43:58.191719 2014-08-13 18:06:06.569673 7f5c973a57a0 20 read_log 193305'14180 (193305'14179) modify 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.951337.0:23215 2014-08-11 18:43:58.183498 2014-08-13 18:06:06.569685 7f5c973a57a0 20 read_log 193305'14181 (0'0) promote 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by osd.325.0:392 2014-08-11 18:44:00.594004 2014-08-13 18:06:06.569697 7f5c973a57a0 20 read_log 193305'14182 (193305'14181) modify 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.951337.0:23880 2014-08-11 18:44:00.583948 2014-08-13 18:06:06.569709 7f5c973a57a0 20 read_log 193305'14183 (0'0) promote cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by osd.325.0:400 2014-08-11 18:44:01.533309 2014-08-13 18:06:06.569722 7f5c973a57a0 20 read_log 193305'14184 (193305'14183) modify cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.951337.0:24134 2014-08-11 18:44:01.522989 2014-08-13 18:06:06.569734 7f5c973a57a0 20 read_log 193305'14185 (0'0) promote be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by osd.325.0:426 2014-08-11 18:44:06.832000 2014-08-13 18:06:06.569756 7f5c973a57a0 20 read_log 193305'14186 (193305'14185) modify be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.951337.0:25591 2014-08-11 18:44:06.824258 2014-08-13 18:06:06.569772 7f5c973a57a0 20 read_log 193305'14187 (0'0) promote 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by osd.325.0:432 2014-08-11 18:44:09.426654 2014-08-13 18:06:06.569784 7f5c973a57a0 20 read_log 193305'14188 (193305'14187) modify 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.951337.0:26282 2014-08-11 18:44:09.416407 2014-08-13 18:06:06.569796 7f5c973a57a0 20 read_log 193305'14189 (0'0) promote 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by osd.325.0:440 2014-08-11 18:44:12.922734 2014-08-13 18:06:06.569809 7f5c973a57a0 20 read_log 193305'14190 (193305'14189) modify 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.951337.0:27232 2014-08-11 18:44:12.914438 2014-08-13 18:06:06.569839 7f5c973a57a0 20 read_log 193305'14191 (0'0) promote 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by osd.325.0:452 2014-08-11 18:44:18.340515 2014-08-13 18:06:06.569854 7f5c973a57a0 20 read_log 193305'14192 (193305'14191) modify 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.951337.0:28738 2014-08-11 18:44:18.331795 2014-08-13 18:06:06.569867 7f5c973a57a0 20 read_log 193305'14193 (0'0) promote f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by osd.325.0:460 2014-08-11 18:44:18.991635 2014-08-13 18:06:06.569880 7f5c973a57a0 20 read_log 193305'14194 (193305'14193) modify f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.951337.0:28902 2014-08-11 18:44:18.982974 2014-08-13 18:06:06.569892 7f5c973a57a0 20 read_log 193305'14195 (0'0) promote cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by osd.325.0:476 2014-08-11 18:44:20.746103 2014-08-13 18:06:06.569904 7f5c973a57a0 20 read_log 193305'14196 (193305'14195) modify cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.951337.0:29385 2014-08-11 18:44:20.737420 2014-08-13 18:06:06.569919 7f5c973a57a0 20 read_log 193305'14197 (193305'14124) modify 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.951337.0:34632 2014-08-11 18:44:35.103965 2014-08-13 18:06:06.569931 7f5c973a57a0 20 read_log 193305'14198 (193305'14126) modify 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.951337.0:35648 2014-08-11 18:44:35.689120 2014-08-13 18:06:06.569946 7f5c973a57a0 20 read_log 193305'14199 (193305'14128) modify d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.951337.0:38969 2014-08-11 18:44:37.754925 2014-08-13 18:06:06.569959 7f5c973a57a0 20 read_log 193305'14200 (193305'14130) modify b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.951337.0:39435 2014-08-11 18:44:38.005310 2014-08-13 18:06:06.569973 7f5c973a57a0 20 read_log 193305'14201 (193305'14132) modify 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.951337.0:39537 2014-08-11 18:44:38.063056 2014-08-13 18:06:06.569985 7f5c973a57a0 20 read_log 193305'14202 (193305'14134) modify 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.951337.0:40080 2014-08-11 18:44:38.278763 2014-08-13 18:06:06.569998 7f5c973a57a0 20 read_log 193305'14203 (193305'14136) modify 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.951337.0:41184 2014-08-11 18:44:38.660280 2014-08-13 18:06:06.570010 7f5c973a57a0 20 read_log 193305'14204 (193305'14138) modify 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.951337.0:41924 2014-08-11 18:44:38.927376 2014-08-13 18:06:06.570025 7f5c973a57a0 20 read_log 193305'14205 (193305'14140) modify 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.951337.0:42324 2014-08-11 18:44:39.189379 2014-08-13 18:06:06.570047 7f5c973a57a0 20 read_log 193305'14206 (193305'14142) modify cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.951337.0:42768 2014-08-11 18:44:39.373880 2014-08-13 18:06:06.570061 7f5c973a57a0 20 read_log 193305'14207 (193305'14144) modify a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.951337.0:44365 2014-08-11 18:44:39.863230 2014-08-13 18:06:06.570074 7f5c973a57a0 20 read_log 193305'14208 (193305'14146) modify e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.951337.0:45453 2014-08-11 18:44:40.205937 2014-08-13 18:06:06.570111 7f5c973a57a0 20 read_log 193305'14209 (193305'14148) modify 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.951337.0:46323 2014-08-11 18:44:40.452019 2014-08-13 18:06:06.570127 7f5c973a57a0 20 read_log 193305'14210 (193305'14150) modify 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.951337.0:46731 2014-08-11 18:44:40.573077 2014-08-13 18:06:06.570140 7f5c973a57a0 20 read_log 193305'14211 (193305'14152) modify 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.951337.0:47042 2014-08-11 18:44:40.659955 2014-08-13 18:06:06.570159 7f5c973a57a0 20 read_log 193305'14212 (193305'14154) modify 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.951337.0:48376 2014-08-11 18:44:41.037660 2014-08-13 18:06:06.570173 7f5c973a57a0 20 read_log 193305'14213 (193305'14156) modify 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.951337.0:48533 2014-08-11 18:44:41.080091 2014-08-13 18:06:06.570185 7f5c973a57a0 20 read_log 193305'14214 (193305'14158) modify 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.951337.0:49114 2014-08-11 18:44:41.257388 2014-08-13 18:06:06.570198 7f5c973a57a0 20 read_log 193305'14215 (193305'14160) modify e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.951337.0:49303 2014-08-11 18:44:41.324797 2014-08-13 18:06:06.570212 7f5c973a57a0 20 read_log 193305'14216 (193305'14162) modify 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.951337.0:49732 2014-08-11 18:44:41.452655 2014-08-13 18:06:06.570225 7f5c973a57a0 20 read_log 193305'14217 (193305'14164) modify 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.951337.0:52115 2014-08-11 18:44:42.137837 2014-08-13 18:06:06.570242 7f5c973a57a0 20 read_log 193305'14218 (193305'14166) modify dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.951337.0:52327 2014-08-11 18:44:42.203595 2014-08-13 18:06:06.570254 7f5c973a57a0 20 read_log 193305'14219 (193305'14168) modify a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.951337.0:52460 2014-08-11 18:44:42.239997 2014-08-13 18:06:06.570267 7f5c973a57a0 20 read_log 193305'14220 (193305'14170) modify 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.951337.0:52626 2014-08-11 18:44:42.287238 2014-08-13 18:06:06.570284 7f5c973a57a0 20 read_log 193305'14221 (193305'14172) modify 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.951337.0:52829 2014-08-11 18:44:42.353099 2014-08-13 18:06:06.570297 7f5c973a57a0 20 read_log 193305'14222 (193305'14174) modify ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.951337.0:52966 2014-08-11 18:44:42.392567 2014-08-13 18:06:06.570310 7f5c973a57a0 20 read_log 193305'14223 (193305'14176) modify 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.951337.0:53042 2014-08-11 18:44:42.413012 2014-08-13 18:06:06.570322 7f5c973a57a0 20 read_log 193305'14224 (193305'14178) modify 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.951337.0:53961 2014-08-11 18:44:42.684845 2014-08-13 18:06:06.570338 7f5c973a57a0 20 read_log 193305'14225 (193305'14180) modify 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.951337.0:55926 2014-08-11 18:44:43.270542 2014-08-13 18:06:06.570359 7f5c973a57a0 20 read_log 193305'14226 (193305'14182) modify 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.951337.0:56591 2014-08-11 18:44:43.463129 2014-08-13 18:06:06.570373 7f5c973a57a0 20 read_log 193305'14227 (193305'14184) modify cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.951337.0:56845 2014-08-11 18:44:43.550491 2014-08-13 18:06:06.570390 7f5c973a57a0 20 read_log 193305'14228 (193305'14186) modify be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.951337.0:58302 2014-08-11 18:44:44.002585 2014-08-13 18:06:06.570414 7f5c973a57a0 20 read_log 193305'14229 (193305'14188) modify 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.951337.0:58993 2014-08-11 18:44:44.204948 2014-08-13 18:06:06.570431 7f5c973a57a0 20 read_log 193305'14230 (193305'14190) modify 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.951337.0:59943 2014-08-11 18:44:44.472854 2014-08-13 18:06:06.570446 7f5c973a57a0 20 read_log 193305'14231 (193305'14192) modify 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.951337.0:61449 2014-08-11 18:44:44.924912 2014-08-13 18:06:06.570472 7f5c973a57a0 20 read_log 193305'14232 (193305'14194) modify f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.951337.0:61613 2014-08-11 18:44:44.977410 2014-08-13 18:06:06.570485 7f5c973a57a0 20 read_log 193305'14233 (193305'14196) modify cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.951337.0:62096 2014-08-11 18:44:45.124350 2014-08-13 18:06:06.570498 7f5c973a57a0 20 read_log 194215'14234 (192888'13947) clean 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by osd.325.0:868 2014-08-12 15:36:52.608185 2014-08-13 18:06:06.570519 7f5c973a57a0 20 read_log 194215'14235 (194215'14234) delete 7cf40034/benchmark_data_storage0111-ib.pocstor_41097_object28968/head//24 by client.2174595.0:10275 0.000000 2014-08-13 18:06:06.570531 7f5c973a57a0 20 read_log 194215'14236 (192834'13715) clean ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by osd.325.0:870 2014-08-12 15:36:52.620388 2014-08-13 18:06:06.570543 7f5c973a57a0 20 read_log 194215'14237 (194215'14236) delete ae570034/benchmark_data_storage0111-ib.pocstor_41526_object17388/head//24 by client.2174595.0:10277 0.000000 2014-08-13 18:06:06.570554 7f5c973a57a0 20 read_log 194215'14238 (192888'13899) clean e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by osd.325.0:872 2014-08-12 15:36:52.631030 2014-08-13 18:06:06.570573 7f5c973a57a0 20 read_log 194215'14239 (194215'14238) delete e20b0034/benchmark_data_storage0111-ib.pocstor_41097_object7366/head//24 by client.2174595.0:10279 0.000000 2014-08-13 18:06:06.570587 7f5c973a57a0 20 read_log 194215'14240 (192888'13929) clean ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by osd.325.0:874 2014-08-12 15:36:52.642154 2014-08-13 18:06:06.570608 7f5c973a57a0 20 read_log 194215'14241 (194215'14240) delete ad541034/benchmark_data_storage0111-ib.pocstor_41097_object22327/head//24 by client.2174595.0:10281 0.000000 2014-08-13 18:06:06.570622 7f5c973a57a0 20 read_log 194215'14242 (192888'14097) clean afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by osd.325.0:876 2014-08-12 15:36:52.652742 2014-08-13 18:06:06.570635 7f5c973a57a0 20 read_log 194215'14243 (194215'14242) delete afcc1034/benchmark_data_storage0111-ib.pocstor_2228_object19685/head//24 by client.2174595.0:10283 0.000000 2014-08-13 18:06:06.570646 7f5c973a57a0 20 read_log 194215'14244 (192888'14115) clean e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by osd.325.0:878 2014-08-12 15:36:52.662495 2014-08-13 18:06:06.570669 7f5c973a57a0 20 read_log 194215'14245 (194215'14244) delete e4d42034/benchmark_data_storage0111-ib.pocstor_2228_object27173/head//24 by client.2174595.0:10285 0.000000 2014-08-13 18:06:06.570683 7f5c973a57a0 20 read_log 194215'14246 (193305'14200) clean b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by osd.325.0:880 2014-08-12 15:36:52.686621 2014-08-13 18:06:06.570705 7f5c973a57a0 20 read_log 194215'14247 (194215'14246) delete b82a2034/benchmark_data_storage0111-ib.pocstor_62621_object6723/head//24 by client.2174595.0:10287 0.000000 2014-08-13 18:06:06.570720 7f5c973a57a0 20 read_log 194215'14248 (192888'14103) clean 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by osd.325.0:882 2014-08-12 15:36:52.698266 2014-08-13 18:06:06.570733 7f5c973a57a0 20 read_log 194215'14249 (194215'14248) delete 750c2034/benchmark_data_storage0111-ib.pocstor_2228_object21416/head//24 by client.2174595.0:10289 0.000000 2014-08-13 18:06:06.570750 7f5c973a57a0 20 read_log 194215'14250 (192888'13949) clean 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by osd.325.0:884 2014-08-12 15:36:52.708144 2014-08-13 18:06:06.570763 7f5c973a57a0 20 read_log 194215'14251 (194215'14250) delete 41463034/benchmark_data_storage0111-ib.pocstor_41097_object29807/head//24 by client.2174595.0:10291 0.000000 2014-08-13 18:06:06.570774 7f5c973a57a0 20 read_log 194215'14252 (192834'13738) clean 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by osd.325.0:886 2014-08-12 15:36:52.718951 2014-08-13 18:06:06.570786 7f5c973a57a0 20 read_log 194215'14253 (194215'14252) delete 864c3034/benchmark_data_storage0111-ib.pocstor_41526_object19996/head//24 by client.2174595.0:10293 0.000000 2014-08-13 18:06:06.570801 7f5c973a57a0 20 read_log 194215'14254 (192888'14076) clean 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by osd.325.0:888 2014-08-12 15:36:52.729964 2014-08-13 18:06:06.570825 7f5c973a57a0 20 read_log 194215'14255 (194215'14254) delete 224d3034/benchmark_data_storage0111-ib.pocstor_2228_object7928/head//24 by client.2174595.0:10295 0.000000 2014-08-13 18:06:06.570840 7f5c973a57a0 20 read_log 194215'14256 (193305'14233) clean cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by osd.325.0:890 2014-08-12 15:36:52.739675 2014-08-13 18:06:06.570852 7f5c973a57a0 20 read_log 194215'14257 (194215'14256) delete cba74034/benchmark_data_storage0111-ib.pocstor_62621_object29384/head//24 by client.2174595.0:10297 0.000000 2014-08-13 18:06:06.570864 7f5c973a57a0 20 read_log 194215'14258 (192888'14108) clean 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by osd.325.0:892 2014-08-12 15:36:52.751097 2014-08-13 18:06:06.570876 7f5c973a57a0 20 read_log 194215'14259 (194215'14258) delete 8a284034/benchmark_data_storage0111-ib.pocstor_2228_object24833/head//24 by client.2174595.0:10299 0.000000 2014-08-13 18:06:06.570887 7f5c973a57a0 20 read_log 194215'14260 (192888'14095) clean 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by osd.325.0:894 2014-08-12 15:36:52.763031 2014-08-13 18:06:06.570899 7f5c973a57a0 20 read_log 194215'14261 (194215'14260) delete 7a725034/benchmark_data_storage0111-ib.pocstor_2228_object18466/head//24 by client.2174595.0:10301 0.000000 2014-08-13 18:06:06.570910 7f5c973a57a0 20 read_log 194215'14262 (193305'14229) clean 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by osd.325.0:896 2014-08-12 15:36:52.775679 2014-08-13 18:06:06.570932 7f5c973a57a0 20 read_log 194215'14263 (194215'14262) delete 9d25034/benchmark_data_storage0111-ib.pocstor_62621_object26281/head//24 by client.2174595.0:10303 0.000000 2014-08-13 18:06:06.570944 7f5c973a57a0 20 read_log 194215'14264 (192888'13932) clean 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by osd.325.0:898 2014-08-12 15:36:52.787524 2014-08-13 18:06:06.570964 7f5c973a57a0 20 read_log 194215'14265 (194215'14264) delete 75795034/benchmark_data_storage0111-ib.pocstor_41097_object23064/head//24 by client.2174595.0:10305 0.000000 2014-08-13 18:06:06.570979 7f5c973a57a0 20 read_log 194215'14266 (192834'13633) clean c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by osd.325.0:900 2014-08-12 15:36:52.803725 2014-08-13 18:06:06.570992 7f5c973a57a0 20 read_log 194215'14267 (194215'14266) delete c18e5034/benchmark_data_storage0111-ib.pocstor_41526_object7806/head//24 by client.2174595.0:10307 0.000000 2014-08-13 18:06:06.571003 7f5c973a57a0 20 read_log 194215'14268 (192888'14104) clean b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by osd.325.0:902 2014-08-12 15:36:52.814725 2014-08-13 18:06:06.571015 7f5c973a57a0 20 read_log 194215'14269 (194215'14268) delete b88c6034/benchmark_data_storage0111-ib.pocstor_2228_object21570/head//24 by client.2174595.0:10309 0.000000 2014-08-13 18:06:06.571027 7f5c973a57a0 20 read_log 194215'14270 (193305'14227) clean cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by osd.325.0:904 2014-08-12 15:36:52.826788 2014-08-13 18:06:06.571039 7f5c973a57a0 20 read_log 194215'14271 (194215'14270) delete cc858034/benchmark_data_storage0111-ib.pocstor_62621_object24133/head//24 by client.2174595.0:10311 0.000000 2014-08-13 18:06:06.571050 7f5c973a57a0 20 read_log 194215'14272 (192888'14096) clean b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by osd.325.0:906 2014-08-12 15:36:52.836933 2014-08-13 18:06:06.571062 7f5c973a57a0 20 read_log 194215'14273 (194215'14272) delete b84f8034/benchmark_data_storage0111-ib.pocstor_2228_object18681/head//24 by client.2174595.0:10313 0.000000 2014-08-13 18:06:06.571073 7f5c973a57a0 20 read_log 194215'14274 (192834'13662) clean 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by osd.325.0:908 2014-08-12 15:36:52.850085 2014-08-13 18:06:06.571085 7f5c973a57a0 20 read_log 194215'14275 (194215'14274) delete 307b9034/benchmark_data_storage0111-ib.pocstor_41526_object11669/head//24 by client.2174595.0:10315 0.000000 2014-08-13 18:06:06.571096 7f5c973a57a0 20 read_log 194215'14276 (192888'13896) clean 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by osd.325.0:910 2014-08-12 15:36:52.859996 2014-08-13 18:06:06.571109 7f5c973a57a0 20 read_log 194215'14277 (194215'14276) delete 55e6a034/benchmark_data_storage0111-ib.pocstor_41097_object3125/head//24 by client.2174595.0:10317 0.000000 2014-08-13 18:06:06.571120 7f5c973a57a0 20 read_log 194215'14278 (192888'13910) clean a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by osd.325.0:912 2014-08-12 15:36:52.870755 2014-08-13 18:06:06.571132 7f5c973a57a0 20 read_log 194215'14279 (194215'14278) delete a81aa034/benchmark_data_storage0111-ib.pocstor_41097_object12745/head//24 by client.2174595.0:10319 0.000000 2014-08-13 18:06:06.571143 7f5c973a57a0 20 read_log 194215'14280 (193305'14218) clean dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by osd.325.0:914 2014-08-12 15:36:52.882437 2014-08-13 18:06:06.571164 7f5c973a57a0 20 read_log 194215'14281 (194215'14280) delete dc6fa034/benchmark_data_storage0111-ib.pocstor_62621_object19615/head//24 by client.2174595.0:10321 0.000000 2014-08-13 18:06:06.571175 7f5c973a57a0 20 read_log 194215'14282 (192834'13666) clean 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by osd.325.0:916 2014-08-12 15:36:52.896291 2014-08-13 18:06:06.571187 7f5c973a57a0 20 read_log 194215'14283 (194215'14282) delete 5e03b034/benchmark_data_storage0111-ib.pocstor_41526_object11819/head//24 by client.2174595.0:10323 0.000000 2014-08-13 18:06:06.571216 7f5c973a57a0 20 read_log 194215'14284 (192888'14092) clean 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by osd.325.0:918 2014-08-12 15:36:52.907716 2014-08-13 18:06:06.571252 7f5c973a57a0 20 read_log 194215'14285 (194215'14284) delete 2bdb034/benchmark_data_storage0111-ib.pocstor_2228_object17157/head//24 by client.2174595.0:10325 0.000000 2014-08-13 18:06:06.571266 7f5c973a57a0 20 read_log 194215'14286 (192888'13908) clean a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by osd.325.0:920 2014-08-12 15:36:52.918745 2014-08-13 18:06:06.571278 7f5c973a57a0 20 read_log 194215'14287 (194215'14286) delete a25eb034/benchmark_data_storage0111-ib.pocstor_41097_object12124/head//24 by client.2174595.0:10327 0.000000 2014-08-13 18:06:06.571293 7f5c973a57a0 20 read_log 194215'14288 (193305'14232) clean f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by osd.325.0:922 2014-08-12 15:36:52.929382 2014-08-13 18:06:06.571306 7f5c973a57a0 20 read_log 194215'14289 (194215'14288) delete f3f3c034/benchmark_data_storage0111-ib.pocstor_62621_object28901/head//24 by client.2174595.0:10329 0.000000 2014-08-13 18:06:06.571317 7f5c973a57a0 20 read_log 194215'14290 (192834'13756) clean 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by osd.325.0:924 2014-08-12 15:36:52.940959 2014-08-13 18:06:06.571329 7f5c973a57a0 20 read_log 194215'14291 (194215'14290) delete 103cc034/benchmark_data_storage0111-ib.pocstor_41526_object22505/head//24 by client.2174595.0:10331 0.000000 2014-08-13 18:06:06.571343 7f5c973a57a0 20 read_log 194215'14292 (192834'13696) clean 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by osd.325.0:926 2014-08-12 15:36:52.954838 2014-08-13 18:06:06.571358 7f5c973a57a0 20 read_log 194215'14293 (194215'14292) delete 7b32d034/benchmark_data_storage0111-ib.pocstor_41526_object14602/head//24 by client.2174595.0:10333 0.000000 2014-08-13 18:06:06.571381 7f5c973a57a0 20 read_log 194215'14294 (192888'13894) clean ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by osd.325.0:928 2014-08-12 15:36:52.966175 2014-08-13 18:06:06.571399 7f5c973a57a0 20 read_log 194215'14295 (194215'14294) delete ee46d034/benchmark_data_storage0111-ib.pocstor_41097_object2094/head//24 by client.2174595.0:10335 0.000000 2014-08-13 18:06:06.571417 7f5c973a57a0 20 read_log 194215'14296 (192834'13577) clean bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by osd.325.0:930 2014-08-12 15:36:52.980118 2014-08-13 18:06:06.571434 7f5c973a57a0 20 read_log 194215'14297 (194215'14296) delete bd1bd034/benchmark_data_storage0111-ib.pocstor_41526_object1482/head//24 by client.2174595.0:10337 0.000000 2014-08-13 18:06:06.571445 7f5c973a57a0 20 read_log 194215'14298 (193305'14222) clean ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by osd.325.0:932 2014-08-12 15:36:52.990650 2014-08-13 18:06:06.571458 7f5c973a57a0 20 read_log 194215'14299 (194215'14298) delete ada7e034/benchmark_data_storage0111-ib.pocstor_62621_object20254/head//24 by client.2174595.0:10339 0.000000 2014-08-13 18:06:06.571479 7f5c973a57a0 20 read_log 194215'14300 (192834'13582) clean 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by osd.325.0:934 2014-08-12 15:36:53.004133 2014-08-13 18:06:06.571492 7f5c973a57a0 20 read_log 194215'14301 (194215'14300) delete 340ce034/benchmark_data_storage0111-ib.pocstor_41526_object1687/head//24 by client.2174595.0:10341 0.000000 2014-08-13 18:06:06.571503 7f5c973a57a0 20 read_log 194215'14302 (192834'13565) clean 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by osd.325.0:936 2014-08-12 15:36:53.013874 2014-08-13 18:06:06.571516 7f5c973a57a0 20 read_log 194215'14303 (194215'14302) delete 83640434/benchmark_data_storage0111-ib.pocstor_41526_object304/head//24 by client.2174595.0:10343 0.000000 2014-08-13 18:06:06.571527 7f5c973a57a0 20 read_log 194215'14304 (192888'13924) clean 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by osd.325.0:938 2014-08-12 15:36:53.022162 2014-08-13 18:06:06.571583 7f5c973a57a0 20 read_log 194215'14305 (194215'14304) delete 26260434/benchmark_data_storage0111-ib.pocstor_41097_object20486/head//24 by client.2174595.0:10345 0.000000 2014-08-13 18:06:06.571602 7f5c973a57a0 20 read_log 194215'14306 (192888'14098) clean f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by osd.325.0:940 2014-08-12 15:36:53.028542 2014-08-13 18:06:06.571618 7f5c973a57a0 20 read_log 194215'14307 (194215'14306) delete f5291434/benchmark_data_storage0111-ib.pocstor_2228_object20168/head//24 by client.2174595.0:10347 0.000000 2014-08-13 18:06:06.571635 7f5c973a57a0 20 read_log 194215'14308 (193305'14221) clean 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by osd.325.0:942 2014-08-12 15:36:53.035451 2014-08-13 18:06:06.571648 7f5c973a57a0 20 read_log 194215'14309 (194215'14308) delete 195a1434/benchmark_data_storage0111-ib.pocstor_62621_object20117/head//24 by client.2174595.0:10349 0.000000 2014-08-13 18:06:06.571667 7f5c973a57a0 20 read_log 194215'14310 (193305'14210) clean 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by osd.325.0:944 2014-08-12 15:36:53.042360 2014-08-13 18:06:06.571683 7f5c973a57a0 20 read_log 194215'14311 (194215'14310) delete 4c0c1434/benchmark_data_storage0111-ib.pocstor_62621_object14019/head//24 by client.2174595.0:10351 0.000000 2014-08-13 18:06:06.571694 7f5c973a57a0 20 read_log 194215'14312 (192834'13037) clean 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by osd.325.0:946 2014-08-12 15:36:53.048592 2014-08-13 18:06:06.571708 7f5c973a57a0 20 read_log 194215'14313 (194215'14312) delete 175c1434/benchmark_data_storage0111-ib.pocstor_26659_object8935/head//24 by client.2174595.0:10353 0.000000 2014-08-13 18:06:06.571719 7f5c973a57a0 20 read_log 194215'14314 (193305'14211) clean 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by osd.325.0:948 2014-08-12 15:36:53.055461 2014-08-13 18:06:06.571731 7f5c973a57a0 20 read_log 194215'14315 (194215'14314) delete 77db2434/benchmark_data_storage0111-ib.pocstor_62621_object14330/head//24 by client.2174595.0:10355 0.000000 2014-08-13 18:06:06.571742 7f5c973a57a0 20 read_log 194215'14316 (192888'14088) clean ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by osd.325.0:950 2014-08-12 15:36:53.062243 2014-08-13 18:06:06.571755 7f5c973a57a0 20 read_log 194215'14317 (194215'14316) delete ca013434/benchmark_data_storage0111-ib.pocstor_2228_object15201/head//24 by client.2174595.0:10357 0.000000 2014-08-13 18:06:06.571766 7f5c973a57a0 20 read_log 194215'14318 (192888'13919) clean 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by osd.325.0:952 2014-08-12 15:36:53.069544 2014-08-13 18:06:06.571778 7f5c973a57a0 20 read_log 194215'14319 (194215'14318) delete 4d823434/benchmark_data_storage0111-ib.pocstor_41097_object19719/head//24 by client.2174595.0:10359 0.000000 2014-08-13 18:06:06.571789 7f5c973a57a0 20 read_log 194215'14320 (193305'14225) clean 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by osd.325.0:954 2014-08-12 15:36:53.077451 2014-08-13 18:06:06.571802 7f5c973a57a0 20 read_log 194215'14321 (194215'14320) delete 40bd3434/benchmark_data_storage0111-ib.pocstor_62621_object23214/head//24 by client.2174595.0:10361 0.000000 2014-08-13 18:06:06.571813 7f5c973a57a0 20 read_log 194215'14322 (193305'14203) clean 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by osd.325.0:956 2014-08-12 15:36:53.104038 2014-08-13 18:06:06.571825 7f5c973a57a0 20 read_log 194215'14323 (194215'14322) delete 3ca54434/benchmark_data_storage0111-ib.pocstor_62621_object8472/head//24 by client.2174595.0:10363 0.000000 2014-08-13 18:06:06.571836 7f5c973a57a0 20 read_log 194215'14324 (192834'13607) clean 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by osd.325.0:958 2014-08-12 15:36:53.112000 2014-08-13 18:06:06.571856 7f5c973a57a0 20 read_log 194215'14325 (194215'14324) delete 861f4434/benchmark_data_storage0111-ib.pocstor_41526_object4455/head//24 by client.2174595.0:10365 0.000000 2014-08-13 18:06:06.571867 7f5c973a57a0 20 read_log 194215'14326 (192888'14100) clean 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by osd.325.0:960 2014-08-12 15:36:53.119126 2014-08-13 18:06:06.571880 7f5c973a57a0 20 read_log 194215'14327 (194215'14326) delete 2db65434/benchmark_data_storage0111-ib.pocstor_2228_object21005/head//24 by client.2174595.0:10367 0.000000 2014-08-13 18:06:06.571891 7f5c973a57a0 20 read_log 194215'14328 (193305'14217) clean 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by osd.325.0:962 2014-08-12 15:36:53.127022 2014-08-13 18:06:06.571903 7f5c973a57a0 20 read_log 194215'14329 (194215'14328) delete 25366434/benchmark_data_storage0111-ib.pocstor_62621_object19403/head//24 by client.2174595.0:10369 0.000000 2014-08-13 18:06:06.571914 7f5c973a57a0 20 read_log 194215'14330 (192834'13729) clean 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by osd.325.0:964 2014-08-12 15:36:53.134759 2014-08-13 18:06:06.571926 7f5c973a57a0 20 read_log 194215'14331 (194215'14330) delete 5bf86434/benchmark_data_storage0111-ib.pocstor_41526_object19244/head//24 by client.2174595.0:10371 0.000000 2014-08-13 18:06:06.571937 7f5c973a57a0 20 read_log 194215'14332 (192834'13727) clean c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by osd.325.0:966 2014-08-12 15:36:53.144254 2014-08-13 18:06:06.571949 7f5c973a57a0 20 read_log 194215'14333 (194215'14332) delete c26e6434/benchmark_data_storage0111-ib.pocstor_41526_object19013/head//24 by client.2174595.0:10373 0.000000 2014-08-13 18:06:06.571962 7f5c973a57a0 20 read_log 194215'14334 (193305'14220) clean 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by osd.325.0:968 2014-08-12 15:36:53.168536 2014-08-13 18:06:06.571975 7f5c973a57a0 20 read_log 194215'14335 (194215'14334) delete 993f6434/benchmark_data_storage0111-ib.pocstor_62621_object19914/head//24 by client.2174595.0:10375 0.000000 2014-08-13 18:06:06.571985 7f5c973a57a0 20 read_log 194215'14336 (192834'13038) clean 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by osd.325.0:970 2014-08-12 15:36:53.180426 2014-08-13 18:06:06.571998 7f5c973a57a0 20 read_log 194215'14337 (194215'14336) delete 5fda7434/benchmark_data_storage0111-ib.pocstor_26659_object9229/head//24 by client.2174595.0:10377 0.000000 2014-08-13 18:06:06.572008 7f5c973a57a0 20 read_log 194215'14338 (192834'13096) clean d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by osd.325.0:972 2014-08-12 15:36:53.190656 2014-08-13 18:06:06.572021 7f5c973a57a0 20 read_log 194215'14339 (194215'14338) delete d85f7434/benchmark_data_storage0111-ib.pocstor_26659_object17635/head//24 by client.2174595.0:10379 0.000000 2014-08-13 18:06:06.572031 7f5c973a57a0 20 read_log 194215'14340 (193305'14209) clean 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by osd.325.0:974 2014-08-12 15:36:53.201363 2014-08-13 18:06:06.572044 7f5c973a57a0 20 read_log 194215'14341 (194215'14340) delete 3e778434/benchmark_data_storage0111-ib.pocstor_62621_object13611/head//24 by client.2174595.0:10381 0.000000 2014-08-13 18:06:06.572054 7f5c973a57a0 20 read_log 194215'14342 (192888'14077) clean 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by osd.325.0:976 2014-08-12 15:36:53.212542 2014-08-13 18:06:06.572067 7f5c973a57a0 20 read_log 194215'14343 (194215'14342) delete 8398434/benchmark_data_storage0111-ib.pocstor_2228_object7943/head//24 by client.2174595.0:10383 0.000000 2014-08-13 18:06:06.572077 7f5c973a57a0 20 read_log 194215'14344 (192888'14120) clean 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by osd.325.0:978 2014-08-12 15:36:53.222984 2014-08-13 18:06:06.572096 7f5c973a57a0 20 read_log 194215'14345 (194215'14344) delete 65898434/benchmark_data_storage0111-ib.pocstor_2228_object31043/head//24 by client.2174595.0:10385 0.000000 2014-08-13 18:06:06.572110 7f5c973a57a0 20 read_log 194215'14346 (193305'14199) clean d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by osd.325.0:980 2014-08-12 15:36:53.234248 2014-08-13 18:06:06.572123 7f5c973a57a0 20 read_log 194215'14347 (194215'14346) delete d1cc8434/benchmark_data_storage0111-ib.pocstor_62621_object6257/head//24 by client.2174595.0:10387 0.000000 2014-08-13 18:06:06.572134 7f5c973a57a0 20 read_log 194215'14348 (193305'14204) clean 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by osd.325.0:982 2014-08-12 15:36:53.245829 2014-08-13 18:06:06.572153 7f5c973a57a0 20 read_log 194215'14349 (194215'14348) delete 37c49434/benchmark_data_storage0111-ib.pocstor_62621_object9212/head//24 by client.2174595.0:10389 0.000000 2014-08-13 18:06:06.572167 7f5c973a57a0 20 read_log 194215'14350 (192888'14111) clean a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by osd.325.0:984 2014-08-12 15:36:53.256925 2014-08-13 18:06:06.572179 7f5c973a57a0 20 read_log 194215'14351 (194215'14350) delete a2899434/benchmark_data_storage0111-ib.pocstor_2228_object26065/head//24 by client.2174595.0:10391 0.000000 2014-08-13 18:06:06.572190 7f5c973a57a0 20 read_log 194215'14352 (193305'14228) clean be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by osd.325.0:986 2014-08-12 15:36:53.268819 2014-08-13 18:06:06.572203 7f5c973a57a0 20 read_log 194215'14353 (194215'14352) delete be8e9434/benchmark_data_storage0111-ib.pocstor_62621_object25590/head//24 by client.2174595.0:10393 0.000000 2014-08-13 18:06:06.572214 7f5c973a57a0 20 read_log 194215'14354 (192834'13097) clean ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by osd.325.0:988 2014-08-12 15:36:53.279323 2014-08-13 18:06:06.572228 7f5c973a57a0 20 read_log 194215'14355 (194215'14354) delete ec8a434/benchmark_data_storage0111-ib.pocstor_26659_object17718/head//24 by client.2174595.0:10395 0.000000 2014-08-13 18:06:06.572241 7f5c973a57a0 20 read_log 194215'14356 (193305'14226) clean 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by osd.325.0:990 2014-08-12 15:36:53.291128 2014-08-13 18:06:06.572254 7f5c973a57a0 20 read_log 194215'14357 (194215'14356) delete 412ba434/benchmark_data_storage0111-ib.pocstor_62621_object23879/head//24 by client.2174595.0:10397 0.000000 2014-08-13 18:06:06.572267 7f5c973a57a0 20 read_log 194215'14358 (192888'14087) clean 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by osd.325.0:992 2014-08-12 15:36:53.300910 2014-08-13 18:06:06.572282 7f5c973a57a0 20 read_log 194215'14359 (194215'14358) delete 6adda434/benchmark_data_storage0111-ib.pocstor_2228_object14894/head//24 by client.2174595.0:10399 0.000000 2014-08-13 18:06:06.572293 7f5c973a57a0 20 read_log 194215'14360 (192834'13012) clean 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by osd.325.0:994 2014-08-12 15:36:53.311433 2014-08-13 18:06:06.572306 7f5c973a57a0 20 read_log 194215'14361 (194215'14360) delete 7e3fb434/benchmark_data_storage0111-ib.pocstor_26659_object3934/head//24 by client.2174595.0:10401 0.000000 2014-08-13 18:06:06.572317 7f5c973a57a0 20 read_log 194215'14362 (192888'14081) clean 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by osd.325.0:996 2014-08-12 15:36:53.321308 2014-08-13 18:06:06.572332 7f5c973a57a0 20 read_log 194215'14363 (194215'14362) delete 5ab9c434/benchmark_data_storage0111-ib.pocstor_2228_object10940/head//24 by client.2174595.0:10403 0.000000 2014-08-13 18:06:06.572343 7f5c973a57a0 20 read_log 194215'14364 (192888'13918) clean 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by osd.325.0:998 2014-08-12 15:36:53.331494 2014-08-13 18:06:06.572391 7f5c973a57a0 20 read_log 194215'14365 (194215'14364) delete 8f0cc434/benchmark_data_storage0111-ib.pocstor_41097_object17755/head//24 by client.2174595.0:10405 0.000000 2014-08-13 18:06:06.572415 7f5c973a57a0 20 read_log 194215'14366 (193305'14207) clean a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by osd.325.0:1000 2014-08-12 15:36:53.343154 2014-08-13 18:06:06.572435 7f5c973a57a0 20 read_log 194215'14367 (194215'14366) delete a2c3d434/benchmark_data_storage0111-ib.pocstor_62621_object11653/head//24 by client.2174595.0:10407 0.000000 2014-08-13 18:06:06.572447 7f5c973a57a0 20 read_log 194215'14368 (193305'14205) clean 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by osd.325.0:1002 2014-08-12 15:36:53.355015 2014-08-13 18:06:06.572469 7f5c973a57a0 20 read_log 194215'14369 (194215'14368) delete 94f3d434/benchmark_data_storage0111-ib.pocstor_62621_object9612/head//24 by client.2174595.0:10409 0.000000 2014-08-13 18:06:06.572483 7f5c973a57a0 20 read_log 194215'14370 (192888'13934) clean 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by osd.325.0:1004 2014-08-12 15:36:53.365950 2014-08-13 18:06:06.572495 7f5c973a57a0 20 read_log 194215'14371 (194215'14370) delete 425dd434/benchmark_data_storage0111-ib.pocstor_41097_object24352/head//24 by client.2174595.0:10411 0.000000 2014-08-13 18:06:06.572513 7f5c973a57a0 20 read_log 194215'14372 (192888'13902) clean 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by osd.325.0:1006 2014-08-12 15:36:53.376145 2014-08-13 18:06:06.572530 7f5c973a57a0 20 read_log 194215'14373 (194215'14372) delete 56f8e434/benchmark_data_storage0111-ib.pocstor_41097_object9363/head//24 by client.2174595.0:10413 0.000000 2014-08-13 18:06:06.572541 7f5c973a57a0 20 read_log 194215'14374 (193305'14212) clean 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by osd.325.0:1008 2014-08-12 15:36:53.387257 2014-08-13 18:06:06.572563 7f5c973a57a0 20 read_log 194215'14375 (194215'14374) delete 4a4ae434/benchmark_data_storage0111-ib.pocstor_62621_object15664/head//24 by client.2174595.0:10415 0.000000 2014-08-13 18:06:06.572577 7f5c973a57a0 20 read_log 194215'14376 (193305'14224) clean 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by osd.325.0:1010 2014-08-12 15:36:53.409891 2014-08-13 18:06:06.572606 7f5c973a57a0 20 read_log 194215'14377 (194215'14376) delete 8abde434/benchmark_data_storage0111-ib.pocstor_62621_object21249/head//24 by client.2174595.0:10417 0.000000 2014-08-13 18:06:06.572623 7f5c973a57a0 20 read_log 194215'14378 (192834'13569) clean bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by osd.325.0:1012 2014-08-12 15:36:53.439714 2014-08-13 18:06:06.572636 7f5c973a57a0 20 read_log 194215'14379 (194215'14378) delete bf6cf434/benchmark_data_storage0111-ib.pocstor_41526_object832/head//24 by client.2174595.0:10419 0.000000 2014-08-13 18:06:06.572647 7f5c973a57a0 20 read_log 194215'14380 (193305'14214) clean 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by osd.325.0:1014 2014-08-12 15:36:53.452527 2014-08-13 18:06:06.572660 7f5c973a57a0 20 read_log 194215'14381 (194215'14380) delete 95ccf434/benchmark_data_storage0111-ib.pocstor_62621_object16402/head//24 by client.2174595.0:10421 0.000000 2014-08-13 18:06:06.572673 7f5c973a57a0 20 read_log 194215'14382 (193305'14213) clean 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by osd.325.0:1016 2014-08-12 15:36:53.463560 2014-08-13 18:06:06.572685 7f5c973a57a0 20 read_log 194215'14383 (194215'14382) delete 633d0834/benchmark_data_storage0111-ib.pocstor_62621_object15821/head//24 by client.2174595.0:10423 0.000000 2014-08-13 18:06:06.572696 7f5c973a57a0 20 read_log 194215'14384 (192834'13615) clean 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by osd.325.0:1018 2014-08-12 15:36:53.474314 2014-08-13 18:06:06.572716 7f5c973a57a0 20 read_log 194215'14385 (194215'14384) delete 24ca1834/benchmark_data_storage0111-ib.pocstor_41526_object5856/head//24 by client.2174595.0:10425 0.000000 2014-08-13 18:06:06.572728 7f5c973a57a0 20 read_log 194215'14386 (192888'14116) clean e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by osd.325.0:1020 2014-08-12 15:36:53.484462 2014-08-13 18:06:06.572740 7f5c973a57a0 20 read_log 194215'14387 (194215'14386) delete e9a62834/benchmark_data_storage0111-ib.pocstor_2228_object27748/head//24 by client.2174595.0:10427 0.000000 2014-08-13 18:06:06.572751 7f5c973a57a0 20 read_log 194215'14388 (192834'13626) clean e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by osd.325.0:1022 2014-08-12 15:36:53.495000 2014-08-13 18:06:06.572763 7f5c973a57a0 20 read_log 194215'14389 (194215'14388) delete e6203834/benchmark_data_storage0111-ib.pocstor_41526_object7135/head//24 by client.2174595.0:10429 0.000000 2014-08-13 18:06:06.572774 7f5c973a57a0 20 read_log 194215'14390 (193305'14198) clean 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by osd.325.0:1024 2014-08-12 15:36:53.507659 2014-08-13 18:06:06.572786 7f5c973a57a0 20 read_log 194215'14391 (194215'14390) delete 8f513834/benchmark_data_storage0111-ib.pocstor_62621_object2936/head//24 by client.2174595.0:10431 0.000000 2014-08-13 18:06:06.572797 7f5c973a57a0 20 read_log 194215'14392 (192834'13605) clean eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by osd.325.0:1026 2014-08-12 15:36:53.519733 2014-08-13 18:06:06.572810 7f5c973a57a0 20 read_log 194215'14393 (194215'14392) delete eb683834/benchmark_data_storage0111-ib.pocstor_41526_object4371/head//24 by client.2174595.0:10433 0.000000 2014-08-13 18:06:06.572821 7f5c973a57a0 20 read_log 194215'14394 (192834'13603) clean 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by osd.325.0:1028 2014-08-12 15:36:53.532163 2014-08-13 18:06:06.572833 7f5c973a57a0 20 read_log 194215'14395 (194215'14394) delete 5adc3834/benchmark_data_storage0111-ib.pocstor_41526_object4025/head//24 by client.2174595.0:10435 0.000000 2014-08-13 18:06:06.572844 7f5c973a57a0 20 read_log 194215'14396 (192888'13940) clean 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by osd.325.0:1030 2014-08-12 15:36:53.543466 2014-08-13 18:06:06.572856 7f5c973a57a0 20 read_log 194215'14397 (194215'14396) delete 46304834/benchmark_data_storage0111-ib.pocstor_41097_object26253/head//24 by client.2174595.0:10437 0.000000 2014-08-13 18:06:06.572867 7f5c973a57a0 20 read_log 194215'14398 (192834'13568) clean e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by osd.325.0:1032 2014-08-12 15:36:53.553550 2014-08-13 18:06:06.572880 7f5c973a57a0 20 read_log 194215'14399 (194215'14398) delete e7b94834/benchmark_data_storage0111-ib.pocstor_41526_object591/head//24 by client.2174595.0:10439 0.000000 2014-08-13 18:06:06.572890 7f5c973a57a0 20 read_log 194215'14400 (192834'13048) clean c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by osd.325.0:1034 2014-08-12 15:36:53.563805 2014-08-13 18:06:06.572903 7f5c973a57a0 20 read_log 194215'14401 (194215'14400) delete c95a4834/benchmark_data_storage0111-ib.pocstor_26659_object11213/head//24 by client.2174595.0:10441 0.000000 2014-08-13 18:06:06.572914 7f5c973a57a0 20 read_log 194215'14402 (192888'14107) clean b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by osd.325.0:1036 2014-08-12 15:36:53.574584 2014-08-13 18:06:06.572926 7f5c973a57a0 20 read_log 194215'14403 (194215'14402) delete b5985834/benchmark_data_storage0111-ib.pocstor_2228_object24667/head//24 by client.2174595.0:10443 0.000000 2014-08-13 18:06:06.572937 7f5c973a57a0 20 read_log 194215'14404 (192834'13609) clean d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by osd.325.0:1038 2014-08-12 15:36:53.584094 2014-08-13 18:06:06.572959 7f5c973a57a0 20 read_log 194215'14405 (194215'14404) delete d4ac7834/benchmark_data_storage0111-ib.pocstor_41526_object5566/head//24 by client.2174595.0:10445 0.000000 2014-08-13 18:06:06.572973 7f5c973a57a0 20 read_log 194215'14406 (193305'14201) clean 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by osd.325.0:1040 2014-08-12 15:36:53.594813 2014-08-13 18:06:06.572985 7f5c973a57a0 20 read_log 194215'14407 (194215'14406) delete 4d3f7834/benchmark_data_storage0111-ib.pocstor_62621_object6825/head//24 by client.2174595.0:10447 0.000000 2014-08-13 18:06:06.572996 7f5c973a57a0 20 read_log 194215'14408 (192834'13676) clean be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by osd.325.0:1042 2014-08-12 15:36:53.604998 2014-08-13 18:06:06.573011 7f5c973a57a0 20 read_log 194215'14409 (194215'14408) delete be278834/benchmark_data_storage0111-ib.pocstor_41526_object12653/head//24 by client.2174595.0:10449 0.000000 2014-08-13 18:06:06.573022 7f5c973a57a0 20 read_log 194215'14410 (192888'14090) clean 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by osd.325.0:1044 2014-08-12 15:36:53.616182 2014-08-13 18:06:06.573034 7f5c973a57a0 20 read_log 194215'14411 (194215'14410) delete 91fb8834/benchmark_data_storage0111-ib.pocstor_2228_object16071/head//24 by client.2174595.0:10451 0.000000 2014-08-13 18:06:06.573045 7f5c973a57a0 20 read_log 194215'14412 (192888'14099) clean b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by osd.325.0:1046 2014-08-12 15:36:53.627253 2014-08-13 18:06:06.573059 7f5c973a57a0 20 read_log 194215'14413 (194215'14412) delete b99c8834/benchmark_data_storage0111-ib.pocstor_2228_object20546/head//24 by client.2174595.0:10453 0.000000 2014-08-13 18:06:06.573070 7f5c973a57a0 20 read_log 194215'14414 (192888'14082) clean d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by osd.325.0:1048 2014-08-12 15:36:53.637033 2014-08-13 18:06:06.573083 7f5c973a57a0 20 read_log 194215'14415 (194215'14414) delete d1ee8834/benchmark_data_storage0111-ib.pocstor_2228_object11970/head//24 by client.2174595.0:10455 0.000000 2014-08-13 18:06:06.573094 7f5c973a57a0 20 read_log 194215'14416 (192888'13928) clean 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by osd.325.0:1050 2014-08-12 15:36:53.646985 2014-08-13 18:06:06.573108 7f5c973a57a0 20 read_log 194215'14417 (194215'14416) delete 80c2b834/benchmark_data_storage0111-ib.pocstor_41097_object22151/head//24 by client.2174595.0:10457 0.000000 2014-08-13 18:06:06.573119 7f5c973a57a0 20 read_log 194215'14418 (193305'14215) clean e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by osd.325.0:1052 2014-08-12 15:36:53.657970 2014-08-13 18:06:06.573131 7f5c973a57a0 20 read_log 194215'14419 (194215'14418) delete e437b834/benchmark_data_storage0111-ib.pocstor_62621_object16591/head//24 by client.2174595.0:10459 0.000000 2014-08-13 18:06:06.573144 7f5c973a57a0 20 read_log 194215'14420 (192888'14094) clean 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by osd.325.0:1054 2014-08-12 15:36:53.669952 2014-08-13 18:06:06.573165 7f5c973a57a0 20 read_log 194215'14421 (194215'14420) delete 6949b834/benchmark_data_storage0111-ib.pocstor_2228_object18351/head//24 by client.2174595.0:10461 0.000000 2014-08-13 18:06:06.573176 7f5c973a57a0 20 read_log 194215'14422 (192834'12976) clean 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by osd.325.0:1056 2014-08-12 15:36:53.681489 2014-08-13 18:06:06.573188 7f5c973a57a0 20 read_log 194215'14423 (194215'14422) delete 5eceb834/benchmark_data_storage0111-ib.pocstor_26659_object133/head//24 by client.2174595.0:10463 0.000000 2014-08-13 18:06:06.573199 7f5c973a57a0 20 read_log 194215'14424 (192888'14066) clean 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by osd.325.0:1058 2014-08-12 15:36:53.693118 2014-08-13 18:06:06.573221 7f5c973a57a0 20 read_log 194215'14425 (194215'14424) delete 5106c834/benchmark_data_storage0111-ib.pocstor_2228_object602/head//24 by client.2174595.0:10465 0.000000 2014-08-13 18:06:06.573235 7f5c973a57a0 20 read_log 194215'14426 (192834'13656) clean 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by osd.325.0:1060 2014-08-12 15:36:53.703211 2014-08-13 18:06:06.573248 7f5c973a57a0 20 read_log 194215'14427 (194215'14426) delete 58cc834/benchmark_data_storage0111-ib.pocstor_41526_object10582/head//24 by client.2174595.0:10467 0.000000 2014-08-13 18:06:06.573259 7f5c973a57a0 20 read_log 194215'14428 (193305'14206) clean cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by osd.325.0:1062 2014-08-12 15:36:53.713224 2014-08-13 18:06:06.573273 7f5c973a57a0 20 read_log 194215'14429 (194215'14428) delete cabec834/benchmark_data_storage0111-ib.pocstor_62621_object10056/head//24 by client.2174595.0:10469 0.000000 2014-08-13 18:06:06.573284 7f5c973a57a0 20 read_log 194215'14430 (192888'14072) clean 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by osd.325.0:1064 2014-08-12 15:36:53.723979 2014-08-13 18:06:06.573297 7f5c973a57a0 20 read_log 194215'14431 (194215'14430) delete 4afbd834/benchmark_data_storage0111-ib.pocstor_2228_object3836/head//24 by client.2174595.0:10471 0.000000 2014-08-13 18:06:06.573314 7f5c973a57a0 20 read_log 194215'14432 (192888'14110) clean 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by osd.325.0:1066 2014-08-12 15:36:53.736485 2014-08-13 18:06:06.573326 7f5c973a57a0 20 read_log 194215'14433 (194215'14432) delete 6631e834/benchmark_data_storage0111-ib.pocstor_2228_object25475/head//24 by client.2174595.0:10473 0.000000 2014-08-13 18:06:06.573337 7f5c973a57a0 20 read_log 194215'14434 (192888'14079) clean 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by osd.325.0:1068 2014-08-12 15:36:53.748142 2014-08-13 18:06:06.573352 7f5c973a57a0 20 read_log 194215'14435 (194215'14434) delete 50eee834/benchmark_data_storage0111-ib.pocstor_2228_object8983/head//24 by client.2174595.0:10475 0.000000 2014-08-13 18:06:06.573365 7f5c973a57a0 20 read_log 194215'14436 (192888'13936) clean fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by osd.325.0:1070 2014-08-12 15:36:53.759652 2014-08-13 18:06:06.573378 7f5c973a57a0 20 read_log 194215'14437 (194215'14436) delete fe55f834/benchmark_data_storage0111-ib.pocstor_41097_object24446/head//24 by client.2174595.0:10477 0.000000 2014-08-13 18:06:06.573389 7f5c973a57a0 20 read_log 194215'14438 (192888'13930) clean d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by osd.325.0:1072 2014-08-12 15:36:53.772580 2014-08-13 18:06:06.573407 7f5c973a57a0 20 read_log 194215'14439 (194215'14438) delete d40ff834/benchmark_data_storage0111-ib.pocstor_41097_object22330/head//24 by client.2174595.0:10479 0.000000 2014-08-13 18:06:06.573421 7f5c973a57a0 20 read_log 194215'14440 (192888'14113) clean 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by osd.325.0:1074 2014-08-12 15:36:53.784638 2014-08-13 18:06:06.573446 7f5c973a57a0 20 read_log 194215'14441 (194215'14440) delete 74010c34/benchmark_data_storage0111-ib.pocstor_2228_object26883/head//24 by client.2174595.0:10481 0.000000 2014-08-13 18:06:06.573467 7f5c973a57a0 20 read_log 194215'14442 (192888'13906) clean 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by osd.325.0:1076 2014-08-12 15:36:53.795750 2014-08-13 18:06:06.573480 7f5c973a57a0 20 read_log 194215'14443 (194215'14442) delete 1ac11c34/benchmark_data_storage0111-ib.pocstor_41097_object9880/head//24 by client.2174595.0:10483 0.000000 2014-08-13 18:06:06.573501 7f5c973a57a0 20 read_log 194215'14444 (192888'14114) clean 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by osd.325.0:1078 2014-08-12 15:36:53.805949 2014-08-13 18:06:06.573551 7f5c973a57a0 20 read_log 194215'14445 (194215'14444) delete 76f11c34/benchmark_data_storage0111-ib.pocstor_2228_object26915/head//24 by client.2174595.0:10485 0.000000 2014-08-13 18:06:06.573568 7f5c973a57a0 20 read_log 194215'14446 (192888'14073) clean d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by osd.325.0:1080 2014-08-12 15:36:53.817546 2014-08-13 18:06:06.573581 7f5c973a57a0 20 read_log 194215'14447 (194215'14446) delete d0f22c34/benchmark_data_storage0111-ib.pocstor_2228_object4507/head//24 by client.2174595.0:10487 0.000000 2014-08-13 18:06:06.573592 7f5c973a57a0 20 read_log 194215'14448 (192834'13046) clean e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by osd.325.0:1082 2014-08-12 15:36:53.828286 2014-08-13 18:06:06.573604 7f5c973a57a0 20 read_log 194215'14449 (194215'14448) delete e8f72c34/benchmark_data_storage0111-ib.pocstor_26659_object10875/head//24 by client.2174595.0:10489 0.000000 2014-08-13 18:06:06.573615 7f5c973a57a0 20 read_log 194215'14450 (193305'14208) clean e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by osd.325.0:1084 2014-08-12 15:36:53.841979 2014-08-13 18:06:06.573632 7f5c973a57a0 20 read_log 194215'14451 (194215'14450) delete e02b2c34/benchmark_data_storage0111-ib.pocstor_62621_object12741/head//24 by client.2174595.0:10491 0.000000 2014-08-13 18:06:06.573643 7f5c973a57a0 20 read_log 194215'14452 (192834'13725) clean af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by osd.325.0:1086 2014-08-12 15:36:53.854139 2014-08-13 18:06:06.573656 7f5c973a57a0 20 read_log 194215'14453 (194215'14452) delete af03c34/benchmark_data_storage0111-ib.pocstor_41526_object18916/head//24 by client.2174595.0:10493 0.000000 2014-08-13 18:06:06.573667 7f5c973a57a0 20 read_log 194215'14454 (192834'13617) clean 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by osd.325.0:1088 2014-08-12 15:36:53.864813 2014-08-13 18:06:06.573679 7f5c973a57a0 20 read_log 194215'14455 (194215'14454) delete 11d43c34/benchmark_data_storage0111-ib.pocstor_41526_object5926/head//24 by client.2174595.0:10495 0.000000 2014-08-13 18:06:06.573690 7f5c973a57a0 20 read_log 194215'14456 (192834'13602) clean 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by osd.325.0:1090 2014-08-12 15:36:53.875805 2014-08-13 18:06:06.573736 7f5c973a57a0 20 read_log 194215'14457 (194215'14456) delete 71c04c34/benchmark_data_storage0111-ib.pocstor_41526_object3968/head//24 by client.2174595.0:10497 0.000000 2014-08-13 18:06:06.573748 7f5c973a57a0 20 read_log 194215'14458 (192888'14084) clean 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by osd.325.0:1092 2014-08-12 15:36:53.886384 2014-08-13 18:06:06.573760 7f5c973a57a0 20 read_log 194215'14459 (194215'14458) delete 3a714c34/benchmark_data_storage0111-ib.pocstor_2228_object13058/head//24 by client.2174595.0:10499 0.000000 2014-08-13 18:06:06.573771 7f5c973a57a0 20 read_log 194215'14460 (192888'13948) clean aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by osd.325.0:1094 2014-08-12 15:36:53.897558 2014-08-13 18:06:06.573783 7f5c973a57a0 20 read_log 194215'14461 (194215'14460) delete aee74c34/benchmark_data_storage0111-ib.pocstor_41097_object29449/head//24 by client.2174595.0:10501 0.000000 2014-08-13 18:06:06.573794 7f5c973a57a0 20 read_log 194215'14462 (193305'14197) clean 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by osd.325.0:1096 2014-08-12 15:36:53.909458 2014-08-13 18:06:06.573807 7f5c973a57a0 20 read_log 194215'14463 (194215'14462) delete 5f126c34/benchmark_data_storage0111-ib.pocstor_62621_object1920/head//24 by client.2174595.0:10503 0.000000 2014-08-13 18:06:06.573817 7f5c973a57a0 20 read_log 194215'14464 (193305'14216) clean 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by osd.325.0:1098 2014-08-12 15:36:53.920040 2014-08-13 18:06:06.573837 7f5c973a57a0 20 read_log 194215'14465 (194215'14464) delete 47b67c34/benchmark_data_storage0111-ib.pocstor_62621_object17020/head//24 by client.2174595.0:10505 0.000000 2014-08-13 18:06:06.573848 7f5c973a57a0 20 read_log 194215'14466 (192888'13935) clean 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by osd.325.0:1100 2014-08-12 15:36:53.931253 2014-08-13 18:06:06.573861 7f5c973a57a0 20 read_log 194215'14467 (194215'14466) delete 6b477c34/benchmark_data_storage0111-ib.pocstor_41097_object24360/head//24 by client.2174595.0:10507 0.000000 2014-08-13 18:06:06.573872 7f5c973a57a0 20 read_log 194215'14468 (192888'13903) clean f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by osd.325.0:1102 2014-08-12 15:36:53.941995 2014-08-13 18:06:06.573884 7f5c973a57a0 20 read_log 194215'14469 (194215'14468) delete f5a28c34/benchmark_data_storage0111-ib.pocstor_41097_object9427/head//24 by client.2174595.0:10509 0.000000 2014-08-13 18:06:06.573911 7f5c973a57a0 20 read_log 194215'14470 (193305'14231) clean 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by osd.325.0:1104 2014-08-12 15:36:53.953755 2014-08-13 18:06:06.573927 7f5c973a57a0 20 read_log 194215'14471 (194215'14470) delete 4d778c34/benchmark_data_storage0111-ib.pocstor_62621_object28737/head//24 by client.2174595.0:10511 0.000000 2014-08-13 18:06:06.573938 7f5c973a57a0 20 read_log 194215'14472 (193305'14219) clean a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by osd.325.0:1106 2014-08-12 15:36:53.965170 2014-08-13 18:06:06.573951 7f5c973a57a0 20 read_log 194215'14473 (194215'14472) delete a8eb8c34/benchmark_data_storage0111-ib.pocstor_62621_object19748/head//24 by client.2174595.0:10513 0.000000 2014-08-13 18:06:06.573962 7f5c973a57a0 20 read_log 194215'14474 (192834'13739) clean 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by osd.325.0:1108 2014-08-12 15:36:53.976881 2014-08-13 18:06:06.573974 7f5c973a57a0 20 read_log 194215'14475 (194215'14474) delete 7f6f8c34/benchmark_data_storage0111-ib.pocstor_41526_object20001/head//24 by client.2174595.0:10515 0.000000 2014-08-13 18:06:06.573985 7f5c973a57a0 20 read_log 194215'14476 (192834'13634) clean 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by osd.325.0:1110 2014-08-12 15:36:53.988378 2014-08-13 18:06:06.573998 7f5c973a57a0 20 read_log 194215'14477 (194215'14476) delete 8c81ac34/benchmark_data_storage0111-ib.pocstor_41526_object7981/head//24 by client.2174595.0:10517 0.000000 2014-08-13 18:06:06.574009 7f5c973a57a0 20 read_log 194215'14478 (192888'13945) clean da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by osd.325.0:1112 2014-08-12 15:36:53.998819 2014-08-13 18:06:06.574024 7f5c973a57a0 20 read_log 194215'14479 (194215'14478) delete da73ac34/benchmark_data_storage0111-ib.pocstor_41097_object28233/head//24 by client.2174595.0:10519 0.000000 2014-08-13 18:06:06.574035 7f5c973a57a0 20 read_log 194215'14480 (192834'13653) clean b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by osd.325.0:1114 2014-08-12 15:36:54.009741 2014-08-13 18:06:06.574047 7f5c973a57a0 20 read_log 194215'14481 (194215'14480) delete b877ac34/benchmark_data_storage0111-ib.pocstor_41526_object10241/head//24 by client.2174595.0:10521 0.000000 2014-08-13 18:06:06.574058 7f5c973a57a0 20 read_log 194215'14482 (192888'14067) clean aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by osd.325.0:1116 2014-08-12 15:36:54.021792 2014-08-13 18:06:06.574072 7f5c973a57a0 20 read_log 194215'14483 (194215'14482) delete aeb0bc34/benchmark_data_storage0111-ib.pocstor_2228_object956/head//24 by client.2174595.0:10523 0.000000 2014-08-13 18:06:06.574084 7f5c973a57a0 20 read_log 194215'14484 (192834'13113) clean 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by osd.325.0:1118 2014-08-12 15:36:54.032752 2014-08-13 18:06:06.574371 7f5c973a57a0 20 read_log 194215'14485 (194215'14484) delete 6793bc34/benchmark_data_storage0111-ib.pocstor_26659_object19894/head//24 by client.2174595.0:10525 0.000000 2014-08-13 18:06:06.574386 7f5c973a57a0 20 read_log 194215'14486 (193305'14223) clean 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by osd.325.0:1120 2014-08-12 15:36:54.045219 2014-08-13 18:06:06.574399 7f5c973a57a0 20 read_log 194215'14487 (194215'14486) delete 44d6bc34/benchmark_data_storage0111-ib.pocstor_62621_object20330/head//24 by client.2174595.0:10527 0.000000 2014-08-13 18:06:06.574410 7f5c973a57a0 20 read_log 194215'14488 (193305'14202) clean 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by osd.325.0:1122 2014-08-12 15:36:54.056698 2014-08-13 18:06:06.574424 7f5c973a57a0 20 read_log 194215'14489 (194215'14488) delete 958bbc34/benchmark_data_storage0111-ib.pocstor_62621_object7368/head//24 by client.2174595.0:10529 0.000000 2014-08-13 18:06:06.574435 7f5c973a57a0 20 read_log 194215'14490 (192888'14093) clean dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by osd.325.0:1124 2014-08-12 15:36:54.071337 2014-08-13 18:06:06.574448 7f5c973a57a0 20 read_log 194215'14491 (194215'14490) delete dcbc34/benchmark_data_storage0111-ib.pocstor_2228_object17559/head//24 by client.2174595.0:10531 0.000000 2014-08-13 18:06:06.574463 7f5c973a57a0 20 read_log 194215'14492 (192834'13597) clean 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by osd.325.0:1126 2014-08-12 15:36:54.083425 2014-08-13 18:06:06.574478 7f5c973a57a0 20 read_log 194215'14493 (194215'14492) delete 18d8cc34/benchmark_data_storage0111-ib.pocstor_41526_object3221/head//24 by client.2174595.0:10533 0.000000 2014-08-13 18:06:06.574489 7f5c973a57a0 20 read_log 194215'14494 (192888'13950) clean aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by osd.325.0:1128 2014-08-12 15:36:54.093915 2014-08-13 18:06:06.574502 7f5c973a57a0 20 read_log 194215'14495 (194215'14494) delete aaebcc34/benchmark_data_storage0111-ib.pocstor_41097_object30064/head//24 by client.2174595.0:10535 0.000000 2014-08-13 18:06:06.574513 7f5c973a57a0 20 read_log 194215'14496 (192834'13591) clean acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by osd.325.0:1130 2014-08-12 15:36:54.109229 2014-08-13 18:06:06.574525 7f5c973a57a0 20 read_log 194215'14497 (194215'14496) delete acb5dc34/benchmark_data_storage0111-ib.pocstor_41526_object2321/head//24 by client.2174595.0:10537 0.000000 2014-08-13 18:06:06.574540 7f5c973a57a0 20 read_log 194215'14498 (192834'13100) clean 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by osd.325.0:1132 2014-08-12 15:36:54.121989 2014-08-13 18:06:06.574553 7f5c973a57a0 20 read_log 194215'14499 (194215'14498) delete 1498dc34/benchmark_data_storage0111-ib.pocstor_26659_object18107/head//24 by client.2174595.0:10539 0.000000 2014-08-13 18:06:06.574565 7f5c973a57a0 20 read_log 194215'14500 (192888'14101) clean 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by osd.325.0:1134 2014-08-12 15:36:54.132652 2014-08-13 18:06:06.574578 7f5c973a57a0 20 read_log 194215'14501 (194215'14500) delete 24dcdc34/benchmark_data_storage0111-ib.pocstor_2228_object21136/head//24 by client.2174595.0:10541 0.000000 2014-08-13 18:06:06.574604 7f5c973a57a0 20 read_log 194215'14502 (192834'13594) clean 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by osd.325.0:1136 2014-08-12 15:36:54.144685 2014-08-13 18:06:06.574620 7f5c973a57a0 20 read_log 194215'14503 (194215'14502) delete 60cedc34/benchmark_data_storage0111-ib.pocstor_41526_object2729/head//24 by client.2174595.0:10543 0.000000 2014-08-13 18:06:06.574635 7f5c973a57a0 20 read_log 194215'14504 (193305'14230) clean 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by osd.325.0:1138 2014-08-12 15:36:54.157053 2014-08-13 18:06:06.574685 7f5c973a57a0 20 read_log 194215'14505 (194215'14504) delete 4940fc34/benchmark_data_storage0111-ib.pocstor_62621_object27231/head//24 by client.2174595.0:10545 0.000000 2014-08-13 18:06:06.574725 7f5c973a57a0 20 read_log 194215'14506 (192888'14085) clean 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by osd.325.0:1140 2014-08-12 15:36:54.167797 2014-08-13 18:06:06.574760 7f5c973a57a0 20 read_log 194215'14507 (194215'14506) delete 5d52fc34/benchmark_data_storage0111-ib.pocstor_2228_object13852/head//24 by client.2174595.0:10547 0.000000 2014-08-13 18:06:06.574778 7f5c973a57a0 20 read_log 194215'14508 (192888'13942) clean 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by osd.325.0:1142 2014-08-12 15:36:54.179160 2014-08-13 18:06:06.574791 7f5c973a57a0 20 read_log 194215'14509 (194215'14508) delete 3444fc34/benchmark_data_storage0111-ib.pocstor_41097_object26803/head//24 by client.2174595.0:10549 0.000000 2014-08-13 18:06:06.574805 7f5c973a57a0 20 read_log 194215'14510 (192888'14109) clean 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by osd.325.0:1144 2014-08-12 15:36:54.191634 2014-08-13 18:06:06.574821 7f5c973a57a0 20 read_log 194215'14511 (194215'14510) delete 7a5efc34/benchmark_data_storage0111-ib.pocstor_2228_object25373/head//24 by client.2174595.0:10551 0.000000 2014-08-13 18:06:06.574851 7f5c973a57a0 20 read_log 194215'14512 (194215'14512) modify 34/hit_set_24.34_archive_2014-08-12 15:32:25.735509_2014-08-12 17:24:42.814554/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.814875 2014-08-13 18:06:06.574869 7f5c973a57a0 20 read_log 194215'14513 (193305'14121) delete 34/hit_set_24.34_archive_2014-08-11 18:37:04.940322_2014-08-11 18:42:40.475022/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.814875 2014-08-13 18:06:06.574894 7f5c973a57a0 20 read_log 194215'14514 (0'0) promote 47dd8c34/benchmark_data_storage0111-ib.pocstor_34628_object717/head//24 by osd.325.0:7024 2014-08-12 17:24:42.833517 2014-08-13 18:06:06.574914 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.575410 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.575451 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=0 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.575469 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=0 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] exit Initial 0.017707 0 0.000000 2014-08-13 18:06:06.575482 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=0 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:06.575491 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=0 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194487 2014-08-13 18:06:06.575500 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] log((192718'11510,194215'14514], crt=193305'14233) 2014-08-13 18:06:06.575510 7f5c973a57a0 10 osd.4 194487 pgid 24.3d coll 24.3d_head 2014-08-13 18:06:06.575683 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.3d 2014-08-13 18:06:06.575696 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.575740 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3d(unlocked)] enter Initial 2014-08-13 18:06:06.575752 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3d(unlocked)] enter NotTrimming 2014-08-13 18:06:06.575961 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.576095 7f5c973a57a0 20 read_log 192803'16166 (192718'15535) delete 3d/hit_set_24.3d_archive_2014-08-06 15:08:57.439794_2014-08-06 15:20:33.134523/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:52:38.941197 2014-08-13 18:06:06.576116 7f5c973a57a0 20 read_log 192803'16167 (0'0) promote 81bde03d/core.47650+10+2/head//24 by osd.4.0:18137 2014-08-11 12:53:03.341756 2014-08-13 18:06:06.576129 7f5c973a57a0 20 read_log 192803'16168 (192803'16167) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:1 2014-08-11 12:52:38.934293 2014-08-13 18:06:06.576142 7f5c973a57a0 20 read_log 192803'16169 (192803'16168) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:2 2014-08-11 12:53:09.149997 2014-08-13 18:06:06.576172 7f5c973a57a0 20 read_log 192803'16170 (192803'16169) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:3 2014-08-11 12:53:09.308293 2014-08-13 18:06:06.576185 7f5c973a57a0 20 read_log 192803'16171 (192803'16170) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:4 2014-08-11 12:53:09.429976 2014-08-13 18:06:06.576197 7f5c973a57a0 20 read_log 192803'16172 (192803'16171) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:5 2014-08-11 12:53:09.529269 2014-08-13 18:06:06.576209 7f5c973a57a0 20 read_log 192803'16173 (192803'16172) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:6 2014-08-11 12:53:09.556145 2014-08-13 18:06:06.576224 7f5c973a57a0 20 read_log 192803'16174 (192803'16173) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:7 2014-08-11 12:53:09.620832 2014-08-13 18:06:06.576236 7f5c973a57a0 20 read_log 192803'16175 (192803'16174) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:8 2014-08-11 12:53:09.647294 2014-08-13 18:06:06.576252 7f5c973a57a0 20 read_log 192803'16176 (192803'16175) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:9 2014-08-11 12:53:09.724633 2014-08-13 18:06:06.576266 7f5c973a57a0 20 read_log 192803'16177 (192803'16176) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:10 2014-08-11 12:53:09.750862 2014-08-13 18:06:06.576279 7f5c973a57a0 20 read_log 192803'16178 (192803'16177) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:11 2014-08-11 12:53:09.842022 2014-08-13 18:06:06.576294 7f5c973a57a0 20 read_log 192803'16179 (192803'16178) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:12 2014-08-11 12:53:09.868148 2014-08-13 18:06:06.576309 7f5c973a57a0 20 read_log 192803'16180 (192803'16179) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:13 2014-08-11 12:53:09.926787 2014-08-13 18:06:06.576321 7f5c973a57a0 20 read_log 192803'16181 (192803'16180) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:14 2014-08-11 12:53:09.952970 2014-08-13 18:06:06.576333 7f5c973a57a0 20 read_log 192803'16182 (192803'16181) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:15 2014-08-11 12:53:10.020818 2014-08-13 18:06:06.576346 7f5c973a57a0 20 read_log 192803'16183 (192803'16182) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:16 2014-08-11 12:53:10.048300 2014-08-13 18:06:06.576369 7f5c973a57a0 20 read_log 192803'16184 (192803'16183) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:17 2014-08-11 12:53:10.113637 2014-08-13 18:06:06.576385 7f5c973a57a0 20 read_log 192803'16185 (192803'16184) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:18 2014-08-11 12:53:10.134930 2014-08-13 18:06:06.576397 7f5c973a57a0 20 read_log 192803'16186 (192803'16185) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:19 2014-08-11 12:53:10.422526 2014-08-13 18:06:06.576418 7f5c973a57a0 20 read_log 192803'16187 (192803'16186) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:20 2014-08-11 12:53:10.450567 2014-08-13 18:06:06.576433 7f5c973a57a0 20 read_log 192803'16188 (192803'16187) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:21 2014-08-11 12:53:10.515845 2014-08-13 18:06:06.576446 7f5c973a57a0 20 read_log 192803'16189 (192803'16188) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:22 2014-08-11 12:53:10.891328 2014-08-13 18:06:06.576464 7f5c973a57a0 20 read_log 192803'16190 (192803'16189) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:23 2014-08-11 12:53:10.932960 2014-08-13 18:06:06.576476 7f5c973a57a0 20 read_log 192803'16191 (192803'16190) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:24 2014-08-11 12:53:10.970227 2014-08-13 18:06:06.576488 7f5c973a57a0 20 read_log 192803'16192 (192803'16191) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:25 2014-08-11 12:53:11.035408 2014-08-13 18:06:06.576500 7f5c973a57a0 20 read_log 192803'16193 (192803'16192) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:26 2014-08-11 12:53:11.058793 2014-08-13 18:06:06.576522 7f5c973a57a0 20 read_log 192803'16194 (192803'16193) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:27 2014-08-11 12:53:11.114810 2014-08-13 18:06:06.576537 7f5c973a57a0 20 read_log 192803'16195 (192803'16194) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:28 2014-08-11 12:53:11.137614 2014-08-13 18:06:06.576550 7f5c973a57a0 20 read_log 192803'16196 (192803'16195) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:29 2014-08-11 12:53:11.206086 2014-08-13 18:06:06.576562 7f5c973a57a0 20 read_log 192803'16197 (192803'16196) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:30 2014-08-11 12:53:11.232521 2014-08-13 18:06:06.576574 7f5c973a57a0 20 read_log 192803'16198 (192803'16197) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:31 2014-08-11 12:53:11.343572 2014-08-13 18:06:06.576586 7f5c973a57a0 20 read_log 192803'16199 (192803'16198) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:32 2014-08-11 12:53:11.617607 2014-08-13 18:06:06.576598 7f5c973a57a0 20 read_log 192803'16200 (192803'16199) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:33 2014-08-11 12:53:11.690478 2014-08-13 18:06:06.576610 7f5c973a57a0 20 read_log 192803'16201 (192803'16200) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:34 2014-08-11 12:53:11.718008 2014-08-13 18:06:06.576622 7f5c973a57a0 20 read_log 192803'16202 (192803'16201) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:35 2014-08-11 12:53:11.782179 2014-08-13 18:06:06.576634 7f5c973a57a0 20 read_log 192803'16203 (192803'16202) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:36 2014-08-11 12:53:11.808717 2014-08-13 18:06:06.576646 7f5c973a57a0 20 read_log 192803'16204 (192803'16203) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:37 2014-08-11 12:53:11.860744 2014-08-13 18:06:06.576658 7f5c973a57a0 20 read_log 192803'16205 (192803'16204) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:38 2014-08-11 12:53:11.894488 2014-08-13 18:06:06.576670 7f5c973a57a0 20 read_log 192803'16206 (192803'16205) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:39 2014-08-11 12:53:11.976918 2014-08-13 18:06:06.576682 7f5c973a57a0 20 read_log 192803'16207 (192803'16206) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:40 2014-08-11 12:53:12.002124 2014-08-13 18:06:06.576694 7f5c973a57a0 20 read_log 192803'16208 (192803'16207) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:41 2014-08-11 12:53:12.039820 2014-08-13 18:06:06.576725 7f5c973a57a0 20 read_log 192803'16209 (192803'16208) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:42 2014-08-11 12:53:12.440161 2014-08-13 18:06:06.576740 7f5c973a57a0 20 read_log 192803'16210 (192803'16209) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:43 2014-08-11 12:53:12.541883 2014-08-13 18:06:06.576753 7f5c973a57a0 20 read_log 192803'16211 (192803'16210) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:44 2014-08-11 12:53:12.566215 2014-08-13 18:06:06.576765 7f5c973a57a0 20 read_log 192803'16212 (192803'16211) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:45 2014-08-11 12:53:12.627677 2014-08-13 18:06:06.576777 7f5c973a57a0 20 read_log 192803'16213 (192803'16212) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:46 2014-08-11 12:53:12.657837 2014-08-13 18:06:06.576789 7f5c973a57a0 20 read_log 192803'16214 (192803'16213) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:47 2014-08-11 12:53:12.716735 2014-08-13 18:06:06.576801 7f5c973a57a0 20 read_log 192803'16215 (192803'16214) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:48 2014-08-11 12:53:12.742744 2014-08-13 18:06:06.576820 7f5c973a57a0 20 read_log 192803'16216 (192803'16215) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:49 2014-08-11 12:53:12.790931 2014-08-13 18:06:06.576832 7f5c973a57a0 20 read_log 192803'16217 (192803'16216) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:50 2014-08-11 12:53:12.816572 2014-08-13 18:06:06.576896 7f5c973a57a0 20 read_log 192803'16218 (192803'16217) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:51 2014-08-11 12:53:12.886053 2014-08-13 18:06:06.576913 7f5c973a57a0 20 read_log 192803'16219 (192803'16218) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:52 2014-08-11 12:53:13.216554 2014-08-13 18:06:06.576925 7f5c973a57a0 20 read_log 192803'16220 (192803'16219) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:53 2014-08-11 12:53:13.256556 2014-08-13 18:06:06.576939 7f5c973a57a0 20 read_log 192803'16221 (192803'16220) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:54 2014-08-11 12:53:13.282947 2014-08-13 18:06:06.576952 7f5c973a57a0 20 read_log 192803'16222 (192803'16221) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:55 2014-08-11 12:53:13.359808 2014-08-13 18:06:06.576966 7f5c973a57a0 20 read_log 192803'16223 (192803'16222) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:56 2014-08-11 12:53:13.385423 2014-08-13 18:06:06.576978 7f5c973a57a0 20 read_log 192803'16224 (192803'16223) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:57 2014-08-11 12:53:13.458956 2014-08-13 18:06:06.576992 7f5c973a57a0 20 read_log 192803'16225 (192803'16224) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:58 2014-08-11 12:53:13.484030 2014-08-13 18:06:06.577006 7f5c973a57a0 20 read_log 192803'16226 (192803'16225) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:59 2014-08-11 12:53:13.566591 2014-08-13 18:06:06.577019 7f5c973a57a0 20 read_log 192803'16227 (192803'16226) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:60 2014-08-11 12:53:13.593985 2014-08-13 18:06:06.577031 7f5c973a57a0 20 read_log 192803'16228 (192803'16227) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:61 2014-08-11 12:53:13.664073 2014-08-13 18:06:06.577045 7f5c973a57a0 20 read_log 192803'16229 (192803'16228) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:62 2014-08-11 12:53:13.975180 2014-08-13 18:06:06.577057 7f5c973a57a0 20 read_log 192803'16230 (192803'16229) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:63 2014-08-11 12:53:14.013106 2014-08-13 18:06:06.577072 7f5c973a57a0 20 read_log 192803'16231 (192803'16230) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:64 2014-08-11 12:53:14.055333 2014-08-13 18:06:06.577084 7f5c973a57a0 20 read_log 192803'16232 (192803'16231) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:65 2014-08-11 12:53:14.119762 2014-08-13 18:06:06.577098 7f5c973a57a0 20 read_log 192803'16233 (192803'16232) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:66 2014-08-11 12:53:14.145321 2014-08-13 18:06:06.577110 7f5c973a57a0 20 read_log 192803'16234 (192803'16233) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:67 2014-08-11 12:53:14.223820 2014-08-13 18:06:06.577124 7f5c973a57a0 20 read_log 192803'16235 (192803'16234) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:68 2014-08-11 12:53:14.250277 2014-08-13 18:06:06.577136 7f5c973a57a0 20 read_log 192803'16236 (192803'16235) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:69 2014-08-11 12:53:14.275807 2014-08-13 18:06:06.577154 7f5c973a57a0 20 read_log 192803'16237 (192803'16236) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:70 2014-08-11 12:53:14.300500 2014-08-13 18:06:06.577169 7f5c973a57a0 20 read_log 192803'16238 (192803'16237) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:71 2014-08-11 12:53:14.384316 2014-08-13 18:06:06.577182 7f5c973a57a0 20 read_log 192803'16239 (192803'16238) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:72 2014-08-11 12:53:14.753101 2014-08-13 18:06:06.577194 7f5c973a57a0 20 read_log 192803'16240 (192803'16239) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:73 2014-08-11 12:53:14.834096 2014-08-13 18:06:06.577206 7f5c973a57a0 20 read_log 192803'16241 (192803'16240) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:74 2014-08-11 12:53:14.877342 2014-08-13 18:06:06.577226 7f5c973a57a0 20 read_log 192803'16242 (192803'16241) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:75 2014-08-11 12:53:14.937237 2014-08-13 18:06:06.577241 7f5c973a57a0 20 read_log 192803'16243 (192803'16242) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:76 2014-08-11 12:53:14.963267 2014-08-13 18:06:06.577253 7f5c973a57a0 20 read_log 192803'16244 (192803'16243) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:77 2014-08-11 12:53:14.993830 2014-08-13 18:06:06.577267 7f5c973a57a0 20 read_log 192803'16245 (192803'16244) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:78 2014-08-11 12:53:15.036318 2014-08-13 18:06:06.577284 7f5c973a57a0 20 read_log 192803'16246 (192803'16245) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:79 2014-08-11 12:53:15.101187 2014-08-13 18:06:06.577300 7f5c973a57a0 20 read_log 192803'16247 (192803'16246) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:80 2014-08-11 12:53:15.128008 2014-08-13 18:06:06.577312 7f5c973a57a0 20 read_log 192803'16248 (192803'16247) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:81 2014-08-11 12:53:15.162392 2014-08-13 18:06:06.577324 7f5c973a57a0 20 read_log 192803'16249 (192803'16248) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:82 2014-08-11 12:53:15.200863 2014-08-13 18:06:06.577339 7f5c973a57a0 20 read_log 192803'16250 (192803'16249) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:83 2014-08-11 12:53:15.265261 2014-08-13 18:06:06.577352 7f5c973a57a0 20 read_log 192803'16251 (192803'16250) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:84 2014-08-11 12:53:15.703763 2014-08-13 18:06:06.577371 7f5c973a57a0 20 read_log 192803'16252 (192803'16251) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:85 2014-08-11 12:53:15.741336 2014-08-13 18:06:06.577383 7f5c973a57a0 20 read_log 192803'16253 (192803'16252) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:86 2014-08-11 12:53:15.780803 2014-08-13 18:06:06.577411 7f5c973a57a0 20 read_log 192803'16254 (192803'16253) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:87 2014-08-11 12:53:15.848785 2014-08-13 18:06:06.577428 7f5c973a57a0 20 read_log 192803'16255 (192803'16254) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:88 2014-08-11 12:53:15.876047 2014-08-13 18:06:06.577440 7f5c973a57a0 20 read_log 192803'16256 (192803'16255) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:89 2014-08-11 12:53:15.939770 2014-08-13 18:06:06.577452 7f5c973a57a0 20 read_log 192803'16257 (192803'16256) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:90 2014-08-11 12:53:15.966874 2014-08-13 18:06:06.577465 7f5c973a57a0 20 read_log 192803'16258 (192803'16257) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:91 2014-08-11 12:53:16.021513 2014-08-13 18:06:06.577477 7f5c973a57a0 20 read_log 192803'16259 (192803'16258) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:92 2014-08-11 12:53:16.058161 2014-08-13 18:06:06.577497 7f5c973a57a0 20 read_log 192803'16260 (192803'16259) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:93 2014-08-11 12:53:16.116873 2014-08-13 18:06:06.577510 7f5c973a57a0 20 read_log 192803'16261 (192803'16260) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:94 2014-08-11 12:53:16.430376 2014-08-13 18:06:06.577523 7f5c973a57a0 20 read_log 192803'16262 (192803'16261) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:95 2014-08-11 12:53:16.500234 2014-08-13 18:06:06.577567 7f5c973a57a0 20 read_log 192803'16263 (192803'16262) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:96 2014-08-11 12:53:16.528880 2014-08-13 18:06:06.577585 7f5c973a57a0 20 read_log 192803'16264 (192803'16263) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:97 2014-08-11 12:53:16.555832 2014-08-13 18:06:06.577597 7f5c973a57a0 20 read_log 192803'16265 (192803'16264) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:98 2014-08-11 12:53:16.577323 2014-08-13 18:06:06.577617 7f5c973a57a0 20 read_log 192803'16266 (192803'16265) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:99 2014-08-11 12:53:16.636980 2014-08-13 18:06:06.577632 7f5c973a57a0 20 read_log 192803'16267 (192803'16266) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:100 2014-08-11 12:53:16.658951 2014-08-13 18:06:06.577644 7f5c973a57a0 20 read_log 192803'16268 (192803'16267) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:101 2014-08-11 12:53:16.721705 2014-08-13 18:06:06.577657 7f5c973a57a0 20 read_log 192803'16269 (192803'16268) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:102 2014-08-11 12:53:16.748156 2014-08-13 18:06:06.577670 7f5c973a57a0 20 read_log 192803'16270 (192803'16269) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:103 2014-08-11 12:53:16.774213 2014-08-13 18:06:06.577682 7f5c973a57a0 20 read_log 192803'16271 (192803'16270) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:104 2014-08-11 12:53:17.013259 2014-08-13 18:06:06.577694 7f5c973a57a0 20 read_log 192803'16272 (192803'16271) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:105 2014-08-11 12:53:17.076765 2014-08-13 18:06:06.577706 7f5c973a57a0 20 read_log 192803'16273 (192803'16272) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:106 2014-08-11 12:53:17.100864 2014-08-13 18:06:06.577718 7f5c973a57a0 20 read_log 192803'16274 (192803'16273) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:107 2014-08-11 12:53:17.161869 2014-08-13 18:06:06.577730 7f5c973a57a0 20 read_log 192803'16275 (192803'16274) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:108 2014-08-11 12:53:17.187720 2014-08-13 18:06:06.577742 7f5c973a57a0 20 read_log 192803'16276 (192803'16275) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:109 2014-08-11 12:53:17.214317 2014-08-13 18:06:06.577754 7f5c973a57a0 20 read_log 192803'16277 (192803'16276) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:110 2014-08-11 12:53:17.238684 2014-08-13 18:06:06.577766 7f5c973a57a0 20 read_log 192803'16278 (192803'16277) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:111 2014-08-11 12:53:17.282322 2014-08-13 18:06:06.577778 7f5c973a57a0 20 read_log 192803'16279 (192803'16278) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:112 2014-08-11 12:53:17.307557 2014-08-13 18:06:06.577790 7f5c973a57a0 20 read_log 192803'16280 (192803'16279) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:113 2014-08-11 12:53:17.334339 2014-08-13 18:06:06.577803 7f5c973a57a0 20 read_log 192803'16281 (192803'16280) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:114 2014-08-11 12:53:17.595136 2014-08-13 18:06:06.577815 7f5c973a57a0 20 read_log 192803'16282 (192803'16281) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:115 2014-08-11 12:53:17.652062 2014-08-13 18:06:06.577827 7f5c973a57a0 20 read_log 192803'16283 (192803'16282) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:116 2014-08-11 12:53:17.678893 2014-08-13 18:06:06.577839 7f5c973a57a0 20 read_log 192803'16284 (192803'16283) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:117 2014-08-11 12:53:17.743581 2014-08-13 18:06:06.577851 7f5c973a57a0 20 read_log 192803'16285 (192803'16284) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:118 2014-08-11 12:53:17.769609 2014-08-13 18:06:06.577863 7f5c973a57a0 20 read_log 192803'16286 (192803'16285) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:119 2014-08-11 12:53:17.828212 2014-08-13 18:06:06.577875 7f5c973a57a0 20 read_log 192803'16287 (192803'16286) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:120 2014-08-11 12:53:17.855241 2014-08-13 18:06:06.577887 7f5c973a57a0 20 read_log 192803'16288 (192803'16287) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:121 2014-08-11 12:53:17.919891 2014-08-13 18:06:06.577899 7f5c973a57a0 20 read_log 192803'16289 (192803'16288) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:122 2014-08-11 12:53:17.946061 2014-08-13 18:06:06.577921 7f5c973a57a0 20 read_log 192803'16290 (192803'16289) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:123 2014-08-11 12:53:18.011313 2014-08-13 18:06:06.577936 7f5c973a57a0 20 read_log 192803'16291 (192803'16290) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:124 2014-08-11 12:53:18.128711 2014-08-13 18:06:06.577948 7f5c973a57a0 20 read_log 192803'16292 (192803'16291) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:125 2014-08-11 12:53:18.214666 2014-08-13 18:06:06.577963 7f5c973a57a0 20 read_log 192803'16293 (192803'16292) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:126 2014-08-11 12:53:18.241425 2014-08-13 18:06:06.577975 7f5c973a57a0 20 read_log 192803'16294 (192803'16293) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:127 2014-08-11 12:53:18.329826 2014-08-13 18:06:06.577988 7f5c973a57a0 20 read_log 192803'16295 (192803'16294) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:128 2014-08-11 12:53:18.359328 2014-08-13 18:06:06.578001 7f5c973a57a0 20 read_log 192803'16296 (192803'16295) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:129 2014-08-11 12:53:18.425544 2014-08-13 18:06:06.578013 7f5c973a57a0 20 read_log 192803'16297 (192803'16296) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:130 2014-08-11 12:53:18.447216 2014-08-13 18:06:06.578027 7f5c973a57a0 20 read_log 192803'16298 (192803'16297) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:131 2014-08-11 12:53:18.516961 2014-08-13 18:06:06.578042 7f5c973a57a0 20 read_log 192803'16299 (192803'16298) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:132 2014-08-11 12:53:18.542532 2014-08-13 18:06:06.578054 7f5c973a57a0 20 read_log 192803'16300 (192803'16299) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:133 2014-08-11 12:53:18.639185 2014-08-13 18:06:06.578068 7f5c973a57a0 20 read_log 192803'16301 (192803'16300) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:134 2014-08-11 12:53:18.696616 2014-08-13 18:06:06.578099 7f5c973a57a0 20 read_log 192803'16302 (192803'16301) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:135 2014-08-11 12:53:18.775680 2014-08-13 18:06:06.578114 7f5c973a57a0 20 read_log 192803'16303 (192803'16302) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:136 2014-08-11 12:53:18.801705 2014-08-13 18:06:06.578126 7f5c973a57a0 20 read_log 192803'16304 (192803'16303) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:137 2014-08-11 12:53:18.878857 2014-08-13 18:06:06.578140 7f5c973a57a0 20 read_log 192803'16305 (192803'16304) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:138 2014-08-11 12:53:18.905542 2014-08-13 18:06:06.578158 7f5c973a57a0 20 read_log 192803'16306 (192803'16305) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:139 2014-08-11 12:53:18.962980 2014-08-13 18:06:06.578173 7f5c973a57a0 20 read_log 192803'16307 (192803'16306) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:140 2014-08-11 12:53:18.986933 2014-08-13 18:06:06.578186 7f5c973a57a0 20 read_log 192803'16308 (192803'16307) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:141 2014-08-11 12:53:19.048188 2014-08-13 18:06:06.578220 7f5c973a57a0 20 read_log 192803'16309 (192803'16308) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:142 2014-08-11 12:53:19.074853 2014-08-13 18:06:06.578235 7f5c973a57a0 20 read_log 192803'16310 (192803'16309) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:143 2014-08-11 12:53:19.101265 2014-08-13 18:06:06.578247 7f5c973a57a0 20 read_log 192803'16311 (192803'16310) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:144 2014-08-11 12:53:19.207160 2014-08-13 18:06:06.578259 7f5c973a57a0 20 read_log 192803'16312 (192803'16311) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:145 2014-08-11 12:53:19.233707 2014-08-13 18:06:06.578273 7f5c973a57a0 20 read_log 192803'16313 (192803'16312) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:146 2014-08-11 12:53:19.259798 2014-08-13 18:06:06.578292 7f5c973a57a0 20 read_log 192803'16314 (192803'16313) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:147 2014-08-11 12:53:19.343621 2014-08-13 18:06:06.578307 7f5c973a57a0 20 read_log 192803'16315 (192803'16314) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:148 2014-08-11 12:53:19.370268 2014-08-13 18:06:06.578319 7f5c973a57a0 20 read_log 192803'16316 (192803'16315) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:149 2014-08-11 12:53:19.431809 2014-08-13 18:06:06.578331 7f5c973a57a0 20 read_log 192803'16317 (192803'16316) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:150 2014-08-11 12:53:19.452499 2014-08-13 18:06:06.578345 7f5c973a57a0 20 read_log 192803'16318 (192803'16317) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:151 2014-08-11 12:53:19.492107 2014-08-13 18:06:06.578357 7f5c973a57a0 20 read_log 192803'16319 (192803'16318) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:152 2014-08-11 12:53:19.516496 2014-08-13 18:06:06.578371 7f5c973a57a0 20 read_log 192803'16320 (192803'16319) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:153 2014-08-11 12:53:19.570492 2014-08-13 18:06:06.578384 7f5c973a57a0 20 read_log 192803'16321 (192803'16320) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:154 2014-08-11 12:53:19.733109 2014-08-13 18:06:06.578398 7f5c973a57a0 20 read_log 192803'16322 (192803'16321) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:155 2014-08-11 12:53:19.764513 2014-08-13 18:06:06.578411 7f5c973a57a0 20 read_log 192803'16323 (192803'16322) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:156 2014-08-11 12:53:19.792803 2014-08-13 18:06:06.578425 7f5c973a57a0 20 read_log 192803'16324 (192803'16323) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:157 2014-08-11 12:53:19.821841 2014-08-13 18:06:06.578437 7f5c973a57a0 20 read_log 192803'16325 (192803'16324) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:158 2014-08-11 12:53:19.919592 2014-08-13 18:06:06.578451 7f5c973a57a0 20 read_log 192803'16326 (192803'16325) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:159 2014-08-11 12:53:20.037234 2014-08-13 18:06:06.578463 7f5c973a57a0 20 read_log 192803'16327 (192803'16326) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:160 2014-08-11 12:53:20.070366 2014-08-13 18:06:06.578478 7f5c973a57a0 20 read_log 192803'16328 (192803'16327) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:161 2014-08-11 12:53:20.133265 2014-08-13 18:06:06.578491 7f5c973a57a0 20 read_log 192803'16329 (192803'16328) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:162 2014-08-11 12:53:20.159627 2014-08-13 18:06:06.578504 7f5c973a57a0 20 read_log 192803'16330 (192803'16329) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:163 2014-08-11 12:53:20.234424 2014-08-13 18:06:06.578516 7f5c973a57a0 20 read_log 192803'16331 (192803'16330) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:164 2014-08-11 12:53:20.260085 2014-08-13 18:06:06.578528 7f5c973a57a0 20 read_log 192803'16332 (192803'16331) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:165 2014-08-11 12:53:20.306447 2014-08-13 18:06:06.578540 7f5c973a57a0 20 read_log 192803'16333 (192803'16332) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:166 2014-08-11 12:53:20.332201 2014-08-13 18:06:06.578551 7f5c973a57a0 20 read_log 192803'16334 (192803'16333) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:167 2014-08-11 12:53:20.409692 2014-08-13 18:06:06.578566 7f5c973a57a0 20 read_log 192803'16335 (192803'16334) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:168 2014-08-11 12:53:20.437062 2014-08-13 18:06:06.578578 7f5c973a57a0 20 read_log 192803'16336 (192803'16335) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:169 2014-08-11 12:53:20.501814 2014-08-13 18:06:06.578592 7f5c973a57a0 20 read_log 192803'16337 (192803'16336) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:170 2014-08-11 12:53:20.668789 2014-08-13 18:06:06.578610 7f5c973a57a0 20 read_log 192803'16338 (192803'16337) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:171 2014-08-11 12:53:20.739359 2014-08-13 18:06:06.578625 7f5c973a57a0 20 read_log 192803'16339 (192803'16338) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:172 2014-08-11 12:53:20.783028 2014-08-13 18:06:06.578637 7f5c973a57a0 20 read_log 192803'16340 (192803'16339) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:173 2014-08-11 12:53:20.816340 2014-08-13 18:06:06.578651 7f5c973a57a0 20 read_log 192803'16341 (192803'16340) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:174 2014-08-11 12:53:20.842641 2014-08-13 18:06:06.578663 7f5c973a57a0 20 read_log 192803'16342 (192803'16341) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:175 2014-08-11 12:53:20.912013 2014-08-13 18:06:06.578677 7f5c973a57a0 20 read_log 192803'16343 (192803'16342) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:176 2014-08-11 12:53:20.934022 2014-08-13 18:06:06.578690 7f5c973a57a0 20 read_log 192803'16344 (192803'16343) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:177 2014-08-11 12:53:20.968716 2014-08-13 18:06:06.578703 7f5c973a57a0 20 read_log 192803'16345 (192803'16344) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:178 2014-08-11 12:53:20.993636 2014-08-13 18:06:06.578716 7f5c973a57a0 20 read_log 192803'16346 (192803'16345) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:179 2014-08-11 12:53:21.048268 2014-08-13 18:06:06.578730 7f5c973a57a0 20 read_log 192803'16347 (192803'16346) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:180 2014-08-11 12:53:21.241107 2014-08-13 18:06:06.578743 7f5c973a57a0 20 read_log 192803'16348 (192803'16347) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:181 2014-08-11 12:53:21.329314 2014-08-13 18:06:06.578755 7f5c973a57a0 20 read_log 192803'16349 (192803'16348) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:182 2014-08-11 12:53:21.356763 2014-08-13 18:06:06.578769 7f5c973a57a0 20 read_log 192803'16350 (192803'16349) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:183 2014-08-11 12:53:21.428007 2014-08-13 18:06:06.578784 7f5c973a57a0 20 read_log 192803'16351 (192803'16350) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:184 2014-08-11 12:53:21.450847 2014-08-13 18:06:06.578804 7f5c973a57a0 20 read_log 192803'16352 (192803'16351) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:185 2014-08-11 12:53:21.516743 2014-08-13 18:06:06.578816 7f5c973a57a0 20 read_log 192803'16353 (192803'16352) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:186 2014-08-11 12:53:21.542466 2014-08-13 18:06:06.578828 7f5c973a57a0 20 read_log 192803'16354 (192803'16353) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:187 2014-08-11 12:53:21.605362 2014-08-13 18:06:06.578840 7f5c973a57a0 20 read_log 192803'16355 (192803'16354) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:188 2014-08-11 12:53:21.631143 2014-08-13 18:06:06.578852 7f5c973a57a0 20 read_log 192803'16356 (192803'16355) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:189 2014-08-11 12:53:21.659546 2014-08-13 18:06:06.578866 7f5c973a57a0 20 read_log 192803'16357 (192803'16356) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:190 2014-08-11 12:53:21.752157 2014-08-13 18:06:06.578878 7f5c973a57a0 20 read_log 192803'16358 (192803'16357) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:191 2014-08-11 12:53:21.778473 2014-08-13 18:06:06.578890 7f5c973a57a0 20 read_log 192803'16359 (192803'16358) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:192 2014-08-11 12:53:21.804135 2014-08-13 18:06:06.578904 7f5c973a57a0 20 read_log 192803'16360 (192803'16359) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:193 2014-08-11 12:53:21.863720 2014-08-13 18:06:06.578916 7f5c973a57a0 20 read_log 192803'16361 (192803'16360) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:194 2014-08-11 12:53:21.889897 2014-08-13 18:06:06.578942 7f5c973a57a0 20 read_log 192803'16362 (192803'16361) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:195 2014-08-11 12:53:21.965185 2014-08-13 18:06:06.578957 7f5c973a57a0 20 read_log 192803'16363 (192803'16362) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:196 2014-08-11 12:53:21.990976 2014-08-13 18:06:06.578969 7f5c973a57a0 20 read_log 192803'16364 (192803'16363) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:197 2014-08-11 12:53:22.066394 2014-08-13 18:06:06.578984 7f5c973a57a0 20 read_log 192803'16365 (192803'16364) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:198 2014-08-11 12:53:22.094184 2014-08-13 18:06:06.578996 7f5c973a57a0 20 read_log 192803'16366 (192803'16365) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:199 2014-08-11 12:53:22.170469 2014-08-13 18:06:06.579009 7f5c973a57a0 20 read_log 192803'16367 (192803'16366) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:200 2014-08-11 12:53:22.274931 2014-08-13 18:06:06.579024 7f5c973a57a0 20 read_log 192803'16368 (192803'16367) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:201 2014-08-11 12:53:22.327497 2014-08-13 18:06:06.579036 7f5c973a57a0 20 read_log 192803'16369 (192803'16368) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:202 2014-08-11 12:53:22.353873 2014-08-13 18:06:06.579048 7f5c973a57a0 20 read_log 192803'16370 (192803'16369) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:203 2014-08-11 12:53:22.424998 2014-08-13 18:06:06.579062 7f5c973a57a0 20 read_log 192803'16371 (192803'16370) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:204 2014-08-11 12:53:22.451050 2014-08-13 18:06:06.579075 7f5c973a57a0 20 read_log 192803'16372 (192803'16371) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:205 2014-08-11 12:53:22.471589 2014-08-13 18:06:06.579089 7f5c973a57a0 20 read_log 192803'16373 (192803'16372) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:206 2014-08-11 12:53:22.494330 2014-08-13 18:06:06.579101 7f5c973a57a0 20 read_log 192803'16374 (192803'16373) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:207 2014-08-11 12:53:22.540204 2014-08-13 18:06:06.579115 7f5c973a57a0 20 read_log 192803'16375 (192803'16374) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:208 2014-08-11 12:53:22.566975 2014-08-13 18:06:06.579127 7f5c973a57a0 20 read_log 192803'16376 (192803'16375) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:209 2014-08-11 12:53:22.630812 2014-08-13 18:06:06.579139 7f5c973a57a0 20 read_log 192803'16377 (192803'16376) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:210 2014-08-11 12:53:22.748070 2014-08-13 18:06:06.579510 7f5c973a57a0 20 read_log 192803'16378 (192803'16377) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:211 2014-08-11 12:53:22.773561 2014-08-13 18:06:06.579532 7f5c973a57a0 20 read_log 192803'16379 (192803'16378) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:212 2014-08-11 12:53:22.798412 2014-08-13 18:06:06.579554 7f5c973a57a0 20 read_log 192803'16380 (192803'16379) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:213 2014-08-11 12:53:22.860739 2014-08-13 18:06:06.579570 7f5c973a57a0 20 read_log 192803'16381 (192803'16380) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:214 2014-08-11 12:53:22.886460 2014-08-13 18:06:06.579583 7f5c973a57a0 20 read_log 192803'16382 (192803'16381) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:215 2014-08-11 12:53:22.911615 2014-08-13 18:06:06.579608 7f5c973a57a0 20 read_log 192803'16383 (192803'16382) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:216 2014-08-11 12:53:22.937607 2014-08-13 18:06:06.579624 7f5c973a57a0 20 read_log 192803'16384 (192803'16383) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:217 2014-08-11 12:53:22.994578 2014-08-13 18:06:06.579637 7f5c973a57a0 20 read_log 192803'16385 (192803'16384) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:218 2014-08-11 12:53:23.020473 2014-08-13 18:06:06.579677 7f5c973a57a0 20 read_log 192803'16386 (192803'16385) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:219 2014-08-11 12:53:23.102186 2014-08-13 18:06:06.579693 7f5c973a57a0 20 read_log 192803'16387 (192803'16386) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:220 2014-08-11 12:53:23.266340 2014-08-13 18:06:06.579705 7f5c973a57a0 20 read_log 192803'16388 (192803'16387) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:221 2014-08-11 12:53:23.301336 2014-08-13 18:06:06.579717 7f5c973a57a0 20 read_log 192803'16389 (192803'16388) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:222 2014-08-11 12:53:23.327520 2014-08-13 18:06:06.579730 7f5c973a57a0 20 read_log 192803'16390 (192803'16389) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:223 2014-08-11 12:53:23.394259 2014-08-13 18:06:06.579742 7f5c973a57a0 20 read_log 192803'16391 (192803'16390) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:224 2014-08-11 12:53:23.420533 2014-08-13 18:06:06.579754 7f5c973a57a0 20 read_log 192803'16392 (192803'16391) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:225 2014-08-11 12:53:23.468345 2014-08-13 18:06:06.579766 7f5c973a57a0 20 read_log 192803'16393 (192803'16392) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:226 2014-08-11 12:53:23.494040 2014-08-13 18:06:06.579778 7f5c973a57a0 20 read_log 192803'16394 (192803'16393) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:227 2014-08-11 12:53:23.543339 2014-08-13 18:06:06.579826 7f5c973a57a0 20 read_log 192803'16395 (192803'16394) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:228 2014-08-11 12:53:23.569467 2014-08-13 18:06:06.579841 7f5c973a57a0 20 read_log 192803'16396 (192803'16395) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:229 2014-08-11 12:53:23.604468 2014-08-13 18:06:06.579853 7f5c973a57a0 20 read_log 192803'16397 (192803'16396) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:230 2014-08-11 12:53:23.789532 2014-08-13 18:06:06.579866 7f5c973a57a0 20 read_log 192803'16398 (192803'16397) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:231 2014-08-11 12:53:23.874588 2014-08-13 18:06:06.579878 7f5c973a57a0 20 read_log 192803'16399 (192803'16398) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:232 2014-08-11 12:53:23.900001 2014-08-13 18:06:06.579890 7f5c973a57a0 20 read_log 192803'16400 (192803'16399) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:233 2014-08-11 12:53:23.968171 2014-08-13 18:06:06.579902 7f5c973a57a0 20 read_log 192803'16401 (192803'16400) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:234 2014-08-11 12:53:23.994698 2014-08-13 18:06:06.579914 7f5c973a57a0 20 read_log 192803'16402 (192803'16401) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:235 2014-08-11 12:53:24.019855 2014-08-13 18:06:06.579926 7f5c973a57a0 20 read_log 192803'16403 (192803'16402) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:236 2014-08-11 12:53:24.065672 2014-08-13 18:06:06.579938 7f5c973a57a0 20 read_log 192803'16404 (192803'16403) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:237 2014-08-11 12:53:24.110490 2014-08-13 18:06:06.579950 7f5c973a57a0 20 read_log 192803'16405 (192803'16404) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:238 2014-08-11 12:53:24.136122 2014-08-13 18:06:06.580009 7f5c973a57a0 20 read_log 192803'16406 (192803'16405) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:239 2014-08-11 12:53:24.213446 2014-08-13 18:06:06.580026 7f5c973a57a0 20 read_log 192803'16407 (192803'16406) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:240 2014-08-11 12:53:24.385694 2014-08-13 18:06:06.580039 7f5c973a57a0 20 read_log 192803'16408 (192803'16407) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:241 2014-08-11 12:53:24.428666 2014-08-13 18:06:06.580052 7f5c973a57a0 20 read_log 192803'16409 (192803'16408) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:242 2014-08-11 12:53:24.463413 2014-08-13 18:06:06.580072 7f5c973a57a0 20 read_log 192803'16410 (192803'16409) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:243 2014-08-11 12:53:24.502416 2014-08-13 18:06:06.580086 7f5c973a57a0 20 read_log 192803'16411 (192803'16410) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:244 2014-08-11 12:53:24.541034 2014-08-13 18:06:06.580099 7f5c973a57a0 20 read_log 192803'16412 (192803'16411) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:245 2014-08-11 12:53:24.606516 2014-08-13 18:06:06.580111 7f5c973a57a0 20 read_log 192803'16413 (192803'16412) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:246 2014-08-11 12:53:24.631675 2014-08-13 18:06:06.580125 7f5c973a57a0 20 read_log 192803'16414 (192803'16413) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:247 2014-08-11 12:53:24.676639 2014-08-13 18:06:06.580138 7f5c973a57a0 20 read_log 192803'16415 (192803'16414) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:248 2014-08-11 12:53:24.739787 2014-08-13 18:06:06.580158 7f5c973a57a0 20 read_log 192803'16416 (192803'16415) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:249 2014-08-11 12:53:24.807420 2014-08-13 18:06:06.580173 7f5c973a57a0 20 read_log 192803'16417 (192803'16416) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:250 2014-08-11 12:53:25.151658 2014-08-13 18:06:06.580185 7f5c973a57a0 20 read_log 192803'16418 (192803'16417) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:251 2014-08-11 12:53:25.224836 2014-08-13 18:06:06.580198 7f5c973a57a0 20 read_log 192803'16419 (192803'16418) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:252 2014-08-11 12:53:25.267682 2014-08-13 18:06:06.580212 7f5c973a57a0 20 read_log 192803'16420 (192803'16419) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:253 2014-08-11 12:53:25.301831 2014-08-13 18:06:06.580225 7f5c973a57a0 20 read_log 192803'16421 (192803'16420) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:254 2014-08-11 12:53:25.336099 2014-08-13 18:06:06.580239 7f5c973a57a0 20 read_log 192803'16422 (192803'16421) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:255 2014-08-11 12:53:25.399491 2014-08-13 18:06:06.580251 7f5c973a57a0 20 read_log 192803'16423 (192803'16422) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:256 2014-08-11 12:53:25.427045 2014-08-13 18:06:06.580266 7f5c973a57a0 20 read_log 192803'16424 (192803'16423) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:257 2014-08-11 12:53:25.473947 2014-08-13 18:06:06.580279 7f5c973a57a0 20 read_log 192803'16425 (192803'16424) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:258 2014-08-11 12:53:25.502137 2014-08-13 18:06:06.580291 7f5c973a57a0 20 read_log 192803'16426 (192803'16425) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:259 2014-08-11 12:53:25.543867 2014-08-13 18:06:06.580303 7f5c973a57a0 20 read_log 192803'16427 (192803'16426) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:260 2014-08-11 12:53:25.569796 2014-08-13 18:06:06.580315 7f5c973a57a0 20 read_log 192803'16428 (192803'16427) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:261 2014-08-11 12:53:25.641692 2014-08-13 18:06:06.580327 7f5c973a57a0 20 read_log 192803'16429 (192803'16428) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:262 2014-08-11 12:53:26.046736 2014-08-13 18:06:06.580340 7f5c973a57a0 20 read_log 192803'16430 (192803'16429) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:263 2014-08-11 12:53:26.084894 2014-08-13 18:06:06.580352 7f5c973a57a0 20 read_log 192803'16431 (192803'16430) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:264 2014-08-11 12:53:26.119832 2014-08-13 18:06:06.580364 7f5c973a57a0 20 read_log 192803'16432 (192803'16431) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:265 2014-08-11 12:53:26.185824 2014-08-13 18:06:06.580376 7f5c973a57a0 20 read_log 192803'16433 (192803'16432) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:266 2014-08-11 12:53:26.213560 2014-08-13 18:06:06.580396 7f5c973a57a0 20 read_log 192803'16434 (192803'16433) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:267 2014-08-11 12:53:26.242913 2014-08-13 18:06:06.580409 7f5c973a57a0 20 read_log 192803'16435 (192803'16434) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:268 2014-08-11 12:53:26.270172 2014-08-13 18:06:06.580421 7f5c973a57a0 20 read_log 192803'16436 (192803'16435) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:269 2014-08-11 12:53:26.299784 2014-08-13 18:06:06.580433 7f5c973a57a0 20 read_log 192803'16437 (192803'16436) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:270 2014-08-11 12:53:26.330345 2014-08-13 18:06:06.580460 7f5c973a57a0 20 read_log 192803'16438 (192803'16437) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:271 2014-08-11 12:53:26.360397 2014-08-13 18:06:06.580477 7f5c973a57a0 20 read_log 192803'16439 (192803'16438) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:272 2014-08-11 12:53:26.739140 2014-08-13 18:06:06.580489 7f5c973a57a0 20 read_log 192803'16440 (192803'16439) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:273 2014-08-11 12:53:26.773128 2014-08-13 18:06:06.580512 7f5c973a57a0 20 read_log 192803'16441 (192803'16440) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:274 2014-08-11 12:53:26.808074 2014-08-13 18:06:06.580525 7f5c973a57a0 20 read_log 192803'16442 (192803'16441) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:275 2014-08-11 12:53:26.868398 2014-08-13 18:06:06.580537 7f5c973a57a0 20 read_log 192803'16443 (192803'16442) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:276 2014-08-11 12:53:26.895765 2014-08-13 18:06:06.580557 7f5c973a57a0 20 read_log 192803'16444 (192803'16443) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:277 2014-08-11 12:53:26.962335 2014-08-13 18:06:06.580572 7f5c973a57a0 20 read_log 192803'16445 (192803'16444) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:278 2014-08-11 12:53:26.989190 2014-08-13 18:06:06.580585 7f5c973a57a0 20 read_log 192803'16446 (192803'16445) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:279 2014-08-11 12:53:27.077046 2014-08-13 18:06:06.580608 7f5c973a57a0 20 read_log 192803'16447 (192803'16446) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:280 2014-08-11 12:53:27.101465 2014-08-13 18:06:06.580624 7f5c973a57a0 20 read_log 192803'16448 (192803'16447) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:281 2014-08-11 12:53:27.125234 2014-08-13 18:06:06.580636 7f5c973a57a0 20 read_log 192803'16449 (192803'16448) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:282 2014-08-11 12:53:27.479988 2014-08-13 18:06:06.580648 7f5c973a57a0 20 read_log 192803'16450 (192803'16449) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:283 2014-08-11 12:53:27.538797 2014-08-13 18:06:06.580660 7f5c973a57a0 20 read_log 192803'16451 (192803'16450) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:284 2014-08-11 12:53:27.567548 2014-08-13 18:06:06.580673 7f5c973a57a0 20 read_log 192803'16452 (192803'16451) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:285 2014-08-11 12:53:27.597271 2014-08-13 18:06:06.580685 7f5c973a57a0 20 read_log 192803'16453 (192803'16452) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:286 2014-08-11 12:53:27.634784 2014-08-13 18:06:06.580697 7f5c973a57a0 20 read_log 192803'16454 (192803'16453) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:287 2014-08-11 12:53:27.672494 2014-08-13 18:06:06.580709 7f5c973a57a0 20 read_log 192803'16455 (192803'16454) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:288 2014-08-11 12:53:27.710214 2014-08-13 18:06:06.580721 7f5c973a57a0 20 read_log 192803'16456 (192803'16455) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:289 2014-08-11 12:53:27.747569 2014-08-13 18:06:06.580733 7f5c973a57a0 20 read_log 192803'16457 (192803'16456) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:290 2014-08-11 12:53:27.785167 2014-08-13 18:06:06.580751 7f5c973a57a0 20 read_log 192803'16458 (192803'16457) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:291 2014-08-11 12:53:27.837986 2014-08-13 18:06:06.580764 7f5c973a57a0 20 read_log 192803'16459 (192803'16458) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:292 2014-08-11 12:53:28.236914 2014-08-13 18:06:06.580776 7f5c973a57a0 20 read_log 192803'16460 (192803'16459) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:293 2014-08-11 12:53:28.305931 2014-08-13 18:06:06.580790 7f5c973a57a0 20 read_log 192803'16461 (192803'16460) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:294 2014-08-11 12:53:28.332493 2014-08-13 18:06:06.580805 7f5c973a57a0 20 read_log 192803'16462 (192803'16461) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:295 2014-08-11 12:53:28.398614 2014-08-13 18:06:06.580818 7f5c973a57a0 20 read_log 192803'16463 (192803'16462) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:296 2014-08-11 12:53:28.425509 2014-08-13 18:06:06.580830 7f5c973a57a0 20 read_log 192803'16464 (192803'16463) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:297 2014-08-11 12:53:28.483435 2014-08-13 18:06:06.580842 7f5c973a57a0 20 read_log 192803'16465 (192803'16464) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:298 2014-08-11 12:53:28.509832 2014-08-13 18:06:06.580854 7f5c973a57a0 20 read_log 192803'16466 (192803'16465) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:299 2014-08-11 12:53:28.540004 2014-08-13 18:06:06.580866 7f5c973a57a0 20 read_log 192803'16467 (192803'16466) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:300 2014-08-11 12:53:28.588369 2014-08-13 18:06:06.580878 7f5c973a57a0 20 read_log 192803'16468 (192803'16467) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:301 2014-08-11 12:53:28.625029 2014-08-13 18:06:06.580890 7f5c973a57a0 20 read_log 192803'16469 (192803'16468) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:302 2014-08-11 12:53:28.971942 2014-08-13 18:06:06.580915 7f5c973a57a0 20 read_log 192803'16470 (192803'16469) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:303 2014-08-11 12:53:29.028031 2014-08-13 18:06:06.580928 7f5c973a57a0 20 read_log 192803'16471 (192803'16470) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:304 2014-08-11 12:53:29.055078 2014-08-13 18:06:06.580940 7f5c973a57a0 20 read_log 192803'16472 (192803'16471) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:305 2014-08-11 12:53:29.084226 2014-08-13 18:06:06.580952 7f5c973a57a0 20 read_log 192803'16473 (192803'16472) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:306 2014-08-11 12:53:29.121412 2014-08-13 18:06:06.580965 7f5c973a57a0 20 read_log 192803'16474 (192803'16473) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:307 2014-08-11 12:53:29.177879 2014-08-13 18:06:06.580979 7f5c973a57a0 20 read_log 192803'16475 (192803'16474) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:308 2014-08-11 12:53:29.204719 2014-08-13 18:06:06.580991 7f5c973a57a0 20 read_log 192803'16476 (192803'16475) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:309 2014-08-11 12:53:29.265051 2014-08-13 18:06:06.581003 7f5c973a57a0 20 read_log 192803'16477 (192803'16476) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:310 2014-08-11 12:53:29.291941 2014-08-13 18:06:06.581015 7f5c973a57a0 20 read_log 192803'16478 (192803'16477) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:311 2014-08-11 12:53:29.349156 2014-08-13 18:06:06.581027 7f5c973a57a0 20 read_log 192803'16479 (192803'16478) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:312 2014-08-11 12:53:29.666842 2014-08-13 18:06:06.581039 7f5c973a57a0 20 read_log 192803'16480 (192803'16479) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:313 2014-08-11 12:53:29.706069 2014-08-13 18:06:06.581055 7f5c973a57a0 20 read_log 192803'16481 (192803'16480) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:314 2014-08-11 12:53:29.740756 2014-08-13 18:06:06.581074 7f5c973a57a0 20 read_log 192803'16482 (192803'16481) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:315 2014-08-11 12:53:29.775359 2014-08-13 18:06:06.581088 7f5c973a57a0 20 read_log 192803'16483 (192803'16482) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:316 2014-08-11 12:53:29.810787 2014-08-13 18:06:06.581104 7f5c973a57a0 20 read_log 192803'16484 (192803'16483) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:317 2014-08-11 12:53:29.845282 2014-08-13 18:06:06.581124 7f5c973a57a0 20 read_log 192803'16485 (192803'16484) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:318 2014-08-11 12:53:29.879908 2014-08-13 18:06:06.581141 7f5c973a57a0 20 read_log 192803'16486 (192803'16485) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:319 2014-08-11 12:53:29.936687 2014-08-13 18:06:06.581164 7f5c973a57a0 20 read_log 192803'16487 (192803'16486) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:320 2014-08-11 12:53:29.961242 2014-08-13 18:06:06.581208 7f5c973a57a0 20 read_log 192803'16488 (192803'16487) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:321 2014-08-11 12:53:30.024930 2014-08-13 18:06:06.581237 7f5c973a57a0 20 read_log 192803'16489 (192803'16488) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:322 2014-08-11 12:53:30.367427 2014-08-13 18:06:06.581254 7f5c973a57a0 20 read_log 192803'16490 (192803'16489) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:323 2014-08-11 12:53:30.484253 2014-08-13 18:06:06.581269 7f5c973a57a0 20 read_log 192803'16491 (192803'16490) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:324 2014-08-11 12:53:30.513044 2014-08-13 18:06:06.581281 7f5c973a57a0 20 read_log 192803'16492 (192803'16491) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:325 2014-08-11 12:53:30.539571 2014-08-13 18:06:06.581294 7f5c973a57a0 20 read_log 192803'16493 (192803'16492) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:326 2014-08-11 12:53:30.566766 2014-08-13 18:06:06.581306 7f5c973a57a0 20 read_log 192803'16494 (192803'16493) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:327 2014-08-11 12:53:30.599092 2014-08-13 18:06:06.581327 7f5c973a57a0 20 read_log 192803'16495 (192803'16494) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:328 2014-08-11 12:53:30.633685 2014-08-13 18:06:06.581340 7f5c973a57a0 20 read_log 192803'16496 (192803'16495) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:329 2014-08-11 12:53:30.696074 2014-08-13 18:06:06.581352 7f5c973a57a0 20 read_log 192803'16497 (192803'16496) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:330 2014-08-11 12:53:30.722874 2014-08-13 18:06:06.581364 7f5c973a57a0 20 read_log 192803'16498 (192803'16497) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:331 2014-08-11 12:53:30.753715 2014-08-13 18:06:06.581376 7f5c973a57a0 20 read_log 192803'16499 (192803'16498) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:332 2014-08-11 12:53:30.778953 2014-08-13 18:06:06.581389 7f5c973a57a0 20 read_log 192803'16500 (192803'16499) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:333 2014-08-11 12:53:30.807231 2014-08-13 18:06:06.581401 7f5c973a57a0 20 read_log 192803'16501 (192803'16500) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:334 2014-08-11 12:53:31.224180 2014-08-13 18:06:06.581413 7f5c973a57a0 20 read_log 192803'16502 (192803'16501) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:335 2014-08-11 12:53:31.292045 2014-08-13 18:06:06.581425 7f5c973a57a0 20 read_log 192803'16503 (192803'16502) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:336 2014-08-11 12:53:31.319679 2014-08-13 18:06:06.581437 7f5c973a57a0 20 read_log 192803'16504 (192803'16503) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:337 2014-08-11 12:53:31.346848 2014-08-13 18:06:06.581450 7f5c973a57a0 20 read_log 192803'16505 (192803'16504) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:338 2014-08-11 12:53:31.374212 2014-08-13 18:06:06.581468 7f5c973a57a0 20 read_log 192803'16506 (192803'16505) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:339 2014-08-11 12:53:31.402027 2014-08-13 18:06:06.581481 7f5c973a57a0 20 read_log 192803'16507 (192803'16506) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:340 2014-08-11 12:53:31.433271 2014-08-13 18:06:06.581493 7f5c973a57a0 20 read_log 192803'16508 (192803'16507) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:341 2014-08-11 12:53:31.469874 2014-08-13 18:06:06.581505 7f5c973a57a0 20 read_log 192803'16509 (192803'16508) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:342 2014-08-11 12:53:31.507228 2014-08-13 18:06:06.581518 7f5c973a57a0 20 read_log 192803'16510 (192803'16509) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:343 2014-08-11 12:53:31.541046 2014-08-13 18:06:06.581530 7f5c973a57a0 20 read_log 192803'16511 (192803'16510) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:344 2014-08-11 12:53:31.923304 2014-08-13 18:06:06.581541 7f5c973a57a0 20 read_log 192803'16512 (192803'16511) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:345 2014-08-11 12:53:31.957073 2014-08-13 18:06:06.581553 7f5c973a57a0 20 read_log 192803'16513 (192803'16512) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:346 2014-08-11 12:53:31.993315 2014-08-13 18:06:06.581565 7f5c973a57a0 20 read_log 192803'16514 (192803'16513) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:347 2014-08-11 12:53:32.028278 2014-08-13 18:06:06.581577 7f5c973a57a0 20 read_log 192803'16515 (192803'16514) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:348 2014-08-11 12:53:32.061342 2014-08-13 18:06:06.581589 7f5c973a57a0 20 read_log 192803'16516 (192803'16515) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:349 2014-08-11 12:53:32.123898 2014-08-13 18:06:06.581601 7f5c973a57a0 20 read_log 192803'16517 (192803'16516) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:350 2014-08-11 12:53:32.150519 2014-08-13 18:06:06.581613 7f5c973a57a0 20 read_log 192803'16518 (192803'16517) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:351 2014-08-11 12:53:32.210117 2014-08-13 18:06:06.581625 7f5c973a57a0 20 read_log 192803'16519 (192803'16518) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:352 2014-08-11 12:53:32.282192 2014-08-13 18:06:06.581637 7f5c973a57a0 20 read_log 192803'16520 (192803'16519) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:353 2014-08-11 12:53:32.361118 2014-08-13 18:06:06.581650 7f5c973a57a0 20 read_log 192803'16521 (192803'16520) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:354 2014-08-11 12:53:32.615312 2014-08-13 18:06:06.581662 7f5c973a57a0 20 read_log 192803'16522 (192803'16521) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:355 2014-08-11 12:53:32.709928 2014-08-13 18:06:06.581674 7f5c973a57a0 20 read_log 192803'16523 (192803'16522) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:356 2014-08-11 12:53:32.737643 2014-08-13 18:06:06.581686 7f5c973a57a0 20 read_log 192803'16524 (192803'16523) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:357 2014-08-11 12:53:32.768143 2014-08-13 18:06:06.581701 7f5c973a57a0 20 read_log 192803'16525 (192803'16524) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:358 2014-08-11 12:53:32.794562 2014-08-13 18:06:06.581713 7f5c973a57a0 20 read_log 192803'16526 (192803'16525) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:359 2014-08-11 12:53:32.832581 2014-08-13 18:06:06.581728 7f5c973a57a0 20 read_log 192803'16527 (192803'16526) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:360 2014-08-11 12:53:32.862235 2014-08-13 18:06:06.581743 7f5c973a57a0 20 read_log 192803'16528 (192803'16527) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:361 2014-08-11 12:53:32.919707 2014-08-13 18:06:06.581755 7f5c973a57a0 20 read_log 192803'16529 (192803'16528) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:362 2014-08-11 12:53:32.942030 2014-08-13 18:06:06.581777 7f5c973a57a0 20 read_log 192803'16530 (192803'16529) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:363 2014-08-11 12:53:33.010769 2014-08-13 18:06:06.581792 7f5c973a57a0 20 read_log 192803'16531 (192803'16530) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:364 2014-08-11 12:53:33.312443 2014-08-13 18:06:06.581805 7f5c973a57a0 20 read_log 192803'16532 (192803'16531) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:365 2014-08-11 12:53:33.383666 2014-08-13 18:06:06.581819 7f5c973a57a0 20 read_log 192803'16533 (192803'16532) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:366 2014-08-11 12:53:33.410718 2014-08-13 18:06:06.581831 7f5c973a57a0 20 read_log 192803'16534 (192803'16533) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:367 2014-08-11 12:53:33.434195 2014-08-13 18:06:06.581845 7f5c973a57a0 20 read_log 192803'16535 (192803'16534) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:368 2014-08-11 12:53:33.458830 2014-08-13 18:06:06.581858 7f5c973a57a0 20 read_log 192803'16536 (192803'16535) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:369 2014-08-11 12:53:33.511937 2014-08-13 18:06:06.581871 7f5c973a57a0 20 read_log 192803'16537 (192803'16536) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:370 2014-08-11 12:53:33.538529 2014-08-13 18:06:06.581884 7f5c973a57a0 20 read_log 192803'16538 (192803'16537) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:371 2014-08-11 12:53:33.567269 2014-08-13 18:06:06.581898 7f5c973a57a0 20 read_log 192803'16539 (192803'16538) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:372 2014-08-11 12:53:33.596348 2014-08-13 18:06:06.581910 7f5c973a57a0 20 read_log 192803'16540 (192803'16539) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:373 2014-08-11 12:53:33.631163 2014-08-13 18:06:06.581924 7f5c973a57a0 20 read_log 192803'16541 (192803'16540) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:374 2014-08-11 12:53:34.049335 2014-08-13 18:06:06.581936 7f5c973a57a0 20 read_log 192803'16542 (192803'16541) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:375 2014-08-11 12:53:34.095147 2014-08-13 18:06:06.581998 7f5c973a57a0 20 read_log 192803'16543 (192803'16542) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:376 2014-08-11 12:53:34.130600 2014-08-13 18:06:06.582015 7f5c973a57a0 20 read_log 192803'16544 (192803'16543) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:377 2014-08-11 12:53:34.172082 2014-08-13 18:06:06.582030 7f5c973a57a0 20 read_log 192803'16545 (192803'16544) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:378 2014-08-11 12:53:34.212153 2014-08-13 18:06:06.582042 7f5c973a57a0 20 read_log 192803'16546 (192803'16545) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:379 2014-08-11 12:53:34.253384 2014-08-13 18:06:06.582057 7f5c973a57a0 20 read_log 192803'16547 (192803'16546) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:380 2014-08-11 12:53:34.299015 2014-08-13 18:06:06.582069 7f5c973a57a0 20 read_log 192803'16548 (192803'16547) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:381 2014-08-11 12:53:34.334693 2014-08-13 18:06:06.582083 7f5c973a57a0 20 read_log 192803'16549 (192803'16548) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:382 2014-08-11 12:53:34.376149 2014-08-13 18:06:06.582095 7f5c973a57a0 20 read_log 192803'16550 (192803'16549) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:383 2014-08-11 12:53:34.432228 2014-08-13 18:06:06.582110 7f5c973a57a0 20 read_log 192803'16551 (192803'16550) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:384 2014-08-11 12:53:34.881279 2014-08-13 18:06:06.582122 7f5c973a57a0 20 read_log 192803'16552 (192803'16551) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:385 2014-08-11 12:53:34.931583 2014-08-13 18:06:06.582137 7f5c973a57a0 20 read_log 192803'16553 (192803'16552) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:386 2014-08-11 12:53:34.975576 2014-08-13 18:06:06.582180 7f5c973a57a0 20 read_log 192803'16554 (192803'16553) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:387 2014-08-11 12:53:35.022712 2014-08-13 18:06:06.582196 7f5c973a57a0 20 read_log 192803'16555 (192803'16554) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:388 2014-08-11 12:53:35.073581 2014-08-13 18:06:06.582209 7f5c973a57a0 20 read_log 192803'16556 (192803'16555) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:389 2014-08-11 12:53:35.125382 2014-08-13 18:06:06.582223 7f5c973a57a0 20 read_log 192803'16557 (192803'16556) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:390 2014-08-11 12:53:35.176404 2014-08-13 18:06:06.582236 7f5c973a57a0 20 read_log 192803'16558 (192803'16557) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:391 2014-08-11 12:53:35.228628 2014-08-13 18:06:06.582249 7f5c973a57a0 20 read_log 192803'16559 (192803'16558) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:392 2014-08-11 12:53:35.279046 2014-08-13 18:06:06.582267 7f5c973a57a0 20 read_log 192803'16560 (192803'16559) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:393 2014-08-11 12:53:35.326187 2014-08-13 18:06:06.582279 7f5c973a57a0 20 read_log 192803'16561 (192803'16560) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:394 2014-08-11 12:53:35.526613 2014-08-13 18:06:06.582293 7f5c973a57a0 20 read_log 192803'16562 (192803'16561) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:395 2014-08-11 12:53:35.581129 2014-08-13 18:06:06.582309 7f5c973a57a0 20 read_log 192803'16563 (192803'16562) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:396 2014-08-11 12:53:35.632092 2014-08-13 18:06:06.582324 7f5c973a57a0 20 read_log 192803'16564 (192803'16563) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:397 2014-08-11 12:53:35.679020 2014-08-13 18:06:06.582336 7f5c973a57a0 20 read_log 192803'16565 (192803'16564) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:398 2014-08-11 12:53:35.729642 2014-08-13 18:06:06.582348 7f5c973a57a0 20 read_log 192803'16566 (192803'16565) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:399 2014-08-11 12:53:35.778043 2014-08-13 18:06:06.582365 7f5c973a57a0 20 read_log 192803'16567 (192803'16566) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:400 2014-08-11 12:53:35.828791 2014-08-13 18:06:06.582385 7f5c973a57a0 20 read_log 192803'16568 (192803'16567) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:401 2014-08-11 12:53:35.881548 2014-08-13 18:06:06.582404 7f5c973a57a0 20 read_log 192803'16569 (192803'16568) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:402 2014-08-11 12:53:35.930785 2014-08-13 18:06:06.582419 7f5c973a57a0 20 read_log 192803'16570 (192803'16569) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:403 2014-08-11 12:53:35.974872 2014-08-13 18:06:06.582439 7f5c973a57a0 20 read_log 192803'16571 (192803'16570) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:404 2014-08-11 12:53:36.024446 2014-08-13 18:06:06.582455 7f5c973a57a0 20 read_log 192803'16572 (192803'16571) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:405 2014-08-11 12:53:36.115408 2014-08-13 18:06:06.582467 7f5c973a57a0 20 read_log 192803'16573 (192803'16572) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:406 2014-08-11 12:53:36.160387 2014-08-13 18:06:06.582492 7f5c973a57a0 20 read_log 192803'16574 (192803'16573) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:407 2014-08-11 12:53:36.188270 2014-08-13 18:06:06.582509 7f5c973a57a0 20 read_log 192803'16575 (192803'16574) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:408 2014-08-11 12:53:36.216079 2014-08-13 18:06:06.582521 7f5c973a57a0 20 read_log 192803'16576 (192803'16575) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:409 2014-08-11 12:53:36.249294 2014-08-13 18:06:06.582533 7f5c973a57a0 20 read_log 192803'16577 (192803'16576) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:410 2014-08-11 12:53:36.276497 2014-08-13 18:06:06.582563 7f5c973a57a0 20 read_log 192803'16578 (192803'16577) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:411 2014-08-11 12:53:36.311052 2014-08-13 18:06:06.582579 7f5c973a57a0 20 read_log 192803'16579 (192803'16578) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:412 2014-08-11 12:53:36.361850 2014-08-13 18:06:06.582600 7f5c973a57a0 20 read_log 192803'16580 (192803'16579) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:413 2014-08-11 12:53:36.410681 2014-08-13 18:06:06.582630 7f5c973a57a0 20 read_log 192803'16581 (192803'16580) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:414 2014-08-11 12:53:36.459123 2014-08-13 18:06:06.582645 7f5c973a57a0 20 read_log 192803'16582 (192803'16581) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:415 2014-08-11 12:53:36.642107 2014-08-13 18:06:06.582666 7f5c973a57a0 20 read_log 192803'16583 (192803'16582) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:416 2014-08-11 12:53:36.692681 2014-08-13 18:06:06.582683 7f5c973a57a0 20 read_log 192803'16584 (192803'16583) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:417 2014-08-11 12:53:36.744354 2014-08-13 18:06:06.582695 7f5c973a57a0 20 read_log 192803'16585 (192803'16584) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:418 2014-08-11 12:53:36.794536 2014-08-13 18:06:06.582707 7f5c973a57a0 20 read_log 192803'16586 (192803'16585) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:419 2014-08-11 12:53:36.842157 2014-08-13 18:06:06.582720 7f5c973a57a0 20 read_log 192803'16587 (192803'16586) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:420 2014-08-11 12:53:36.892743 2014-08-13 18:06:06.582732 7f5c973a57a0 20 read_log 192803'16588 (192803'16587) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:421 2014-08-11 12:53:36.943028 2014-08-13 18:06:06.582744 7f5c973a57a0 20 read_log 192803'16589 (192803'16588) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:422 2014-08-11 12:53:36.989402 2014-08-13 18:06:06.582756 7f5c973a57a0 20 read_log 192803'16590 (192803'16589) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:423 2014-08-11 12:53:37.037547 2014-08-13 18:06:06.582774 7f5c973a57a0 20 read_log 192803'16591 (192803'16590) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:424 2014-08-11 12:53:37.105272 2014-08-13 18:06:06.582787 7f5c973a57a0 20 read_log 192803'16592 (192803'16591) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:425 2014-08-11 12:53:37.164887 2014-08-13 18:06:06.582799 7f5c973a57a0 20 read_log 192803'16593 (192803'16592) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:426 2014-08-11 12:53:37.217583 2014-08-13 18:06:06.582811 7f5c973a57a0 20 read_log 192803'16594 (192803'16593) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:427 2014-08-11 12:53:37.269170 2014-08-13 18:06:06.582826 7f5c973a57a0 20 read_log 192803'16595 (192803'16594) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:428 2014-08-11 12:53:37.323100 2014-08-13 18:06:06.582851 7f5c973a57a0 20 read_log 192803'16596 (192803'16595) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:429 2014-08-11 12:53:37.372437 2014-08-13 18:06:06.582864 7f5c973a57a0 20 read_log 192803'16597 (192803'16596) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:430 2014-08-11 12:53:37.424077 2014-08-13 18:06:06.582879 7f5c973a57a0 20 read_log 192803'16598 (192803'16597) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:431 2014-08-11 12:53:37.470770 2014-08-13 18:06:06.582894 7f5c973a57a0 20 read_log 192803'16599 (192803'16598) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:432 2014-08-11 12:53:37.513613 2014-08-13 18:06:06.582906 7f5c973a57a0 20 read_log 192803'16600 (192803'16599) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:433 2014-08-11 12:53:37.557722 2014-08-13 18:06:06.582918 7f5c973a57a0 20 read_log 192803'16601 (192803'16600) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:434 2014-08-11 12:53:37.601142 2014-08-13 18:06:06.582970 7f5c973a57a0 20 read_log 192803'16602 (192803'16601) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:435 2014-08-11 12:53:37.776787 2014-08-13 18:06:06.582988 7f5c973a57a0 20 read_log 192803'16603 (192803'16602) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:436 2014-08-11 12:53:37.826003 2014-08-13 18:06:06.583001 7f5c973a57a0 20 read_log 192803'16604 (192803'16603) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:437 2014-08-11 12:53:37.848408 2014-08-13 18:06:06.583013 7f5c973a57a0 20 read_log 192803'16605 (192803'16604) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:438 2014-08-11 12:53:37.872853 2014-08-13 18:06:06.583025 7f5c973a57a0 20 read_log 192803'16606 (192803'16605) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:439 2014-08-11 12:53:37.907468 2014-08-13 18:06:06.583038 7f5c973a57a0 20 read_log 192803'16607 (192803'16606) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:440 2014-08-11 12:53:37.935821 2014-08-13 18:06:06.583050 7f5c973a57a0 20 read_log 192803'16608 (192803'16607) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:441 2014-08-11 12:53:37.970043 2014-08-13 18:06:06.583062 7f5c973a57a0 20 read_log 192803'16609 (192803'16608) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:442 2014-08-11 12:53:37.998061 2014-08-13 18:06:06.583074 7f5c973a57a0 20 read_log 192803'16610 (192803'16609) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:443 2014-08-11 12:53:38.025500 2014-08-13 18:06:06.583087 7f5c973a57a0 20 read_log 192803'16611 (192803'16610) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:444 2014-08-11 12:53:38.060105 2014-08-13 18:06:06.583099 7f5c973a57a0 20 read_log 192803'16612 (192803'16611) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:445 2014-08-11 12:53:38.371026 2014-08-13 18:06:06.583111 7f5c973a57a0 20 read_log 192803'16613 (192803'16612) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:446 2014-08-11 12:53:38.402621 2014-08-13 18:06:06.583123 7f5c973a57a0 20 read_log 192803'16614 (192803'16613) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:447 2014-08-11 12:53:38.440522 2014-08-13 18:06:06.583135 7f5c973a57a0 20 read_log 192803'16615 (192803'16614) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:448 2014-08-11 12:53:38.468552 2014-08-13 18:06:06.583156 7f5c973a57a0 20 read_log 192803'16616 (192803'16615) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:449 2014-08-11 12:53:38.500067 2014-08-13 18:06:06.583170 7f5c973a57a0 20 read_log 192803'16617 (192803'16616) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:450 2014-08-11 12:53:38.526997 2014-08-13 18:06:06.583191 7f5c973a57a0 20 read_log 192803'16618 (192803'16617) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:451 2014-08-11 12:53:38.555151 2014-08-13 18:06:06.583207 7f5c973a57a0 20 read_log 192803'16619 (192803'16618) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:452 2014-08-11 12:53:38.589299 2014-08-13 18:06:06.583220 7f5c973a57a0 20 read_log 192803'16620 (192803'16619) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:453 2014-08-11 12:53:38.617638 2014-08-13 18:06:06.583235 7f5c973a57a0 20 read_log 192803'16621 (192803'16620) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:454 2014-08-11 12:53:38.652390 2014-08-13 18:06:06.583247 7f5c973a57a0 20 read_log 192803'16622 (192803'16621) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:455 2014-08-11 12:53:38.976666 2014-08-13 18:06:06.583259 7f5c973a57a0 20 read_log 192803'16623 (192803'16622) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:456 2014-08-11 12:53:39.006223 2014-08-13 18:06:06.583275 7f5c973a57a0 20 read_log 192803'16624 (192803'16623) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:457 2014-08-11 12:53:39.043265 2014-08-13 18:06:06.583290 7f5c973a57a0 20 read_log 192803'16625 (192803'16624) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:458 2014-08-11 12:53:39.071519 2014-08-13 18:06:06.583308 7f5c973a57a0 20 read_log 192803'16626 (192803'16625) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:459 2014-08-11 12:53:39.104475 2014-08-13 18:06:06.583341 7f5c973a57a0 20 read_log 192803'16627 (192803'16626) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:460 2014-08-11 12:53:39.132011 2014-08-13 18:06:06.583357 7f5c973a57a0 20 read_log 192803'16628 (192803'16627) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:461 2014-08-11 12:53:39.159441 2014-08-13 18:06:06.583373 7f5c973a57a0 20 read_log 192803'16629 (192803'16628) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:462 2014-08-11 12:53:39.195114 2014-08-13 18:06:06.583387 7f5c973a57a0 20 read_log 192803'16630 (192803'16629) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:463 2014-08-11 12:53:39.223018 2014-08-13 18:06:06.583400 7f5c973a57a0 20 read_log 192803'16631 (192803'16630) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:464 2014-08-11 12:53:39.260784 2014-08-13 18:06:06.583423 7f5c973a57a0 20 read_log 192803'16632 (192803'16631) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:465 2014-08-11 12:53:39.514692 2014-08-13 18:06:06.583436 7f5c973a57a0 20 read_log 192803'16633 (192803'16632) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:466 2014-08-11 12:53:39.544621 2014-08-13 18:06:06.583448 7f5c973a57a0 20 read_log 192803'16634 (192803'16633) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:467 2014-08-11 12:53:39.576998 2014-08-13 18:06:06.583460 7f5c973a57a0 20 read_log 192803'16635 (192803'16634) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:468 2014-08-11 12:53:39.606095 2014-08-13 18:06:06.583473 7f5c973a57a0 20 read_log 192803'16636 (192803'16635) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:469 2014-08-11 12:53:39.639074 2014-08-13 18:06:06.583485 7f5c973a57a0 20 read_log 192803'16637 (192803'16636) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:470 2014-08-11 12:53:39.666701 2014-08-13 18:06:06.583497 7f5c973a57a0 20 read_log 192803'16638 (192803'16637) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:471 2014-08-11 12:53:39.694107 2014-08-13 18:06:06.583509 7f5c973a57a0 20 read_log 192803'16639 (192803'16638) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:472 2014-08-11 12:53:39.723358 2014-08-13 18:06:06.583521 7f5c973a57a0 20 read_log 192803'16640 (192803'16639) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:473 2014-08-11 12:53:39.750470 2014-08-13 18:06:06.583533 7f5c973a57a0 20 read_log 192803'16641 (192803'16640) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:474 2014-08-11 12:53:39.784669 2014-08-13 18:06:06.583545 7f5c973a57a0 20 read_log 192803'16642 (192803'16641) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:475 2014-08-11 12:53:40.062147 2014-08-13 18:06:06.583557 7f5c973a57a0 20 read_log 192803'16643 (192803'16642) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:476 2014-08-11 12:53:40.091191 2014-08-13 18:06:06.583569 7f5c973a57a0 20 read_log 192803'16644 (192803'16643) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:477 2014-08-11 12:53:40.138435 2014-08-13 18:06:06.583581 7f5c973a57a0 20 read_log 192803'16645 (192803'16644) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:478 2014-08-11 12:53:40.166306 2014-08-13 18:06:06.583593 7f5c973a57a0 20 read_log 192803'16646 (192803'16645) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:479 2014-08-11 12:53:40.201809 2014-08-13 18:06:06.583605 7f5c973a57a0 20 read_log 192803'16647 (192803'16646) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:480 2014-08-11 12:53:40.230629 2014-08-13 18:06:06.583618 7f5c973a57a0 20 read_log 192803'16648 (192803'16647) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:481 2014-08-11 12:53:40.259115 2014-08-13 18:06:06.583630 7f5c973a57a0 20 read_log 192803'16649 (192803'16648) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:482 2014-08-11 12:53:40.288605 2014-08-13 18:06:06.583648 7f5c973a57a0 20 read_log 192803'16650 (192803'16649) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:483 2014-08-11 12:53:40.311543 2014-08-13 18:06:06.583660 7f5c973a57a0 20 read_log 192803'16651 (192803'16650) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:484 2014-08-11 12:53:40.337840 2014-08-13 18:06:06.583673 7f5c973a57a0 20 read_log 192803'16652 (192803'16651) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:485 2014-08-11 12:53:40.604597 2014-08-13 18:06:06.583685 7f5c973a57a0 20 read_log 192803'16653 (192803'16652) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:486 2014-08-11 12:53:40.665086 2014-08-13 18:06:06.583697 7f5c973a57a0 20 read_log 192803'16654 (192803'16653) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:487 2014-08-11 12:53:40.702420 2014-08-13 18:06:06.583709 7f5c973a57a0 20 read_log 192803'16655 (192803'16654) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:488 2014-08-11 12:53:40.731522 2014-08-13 18:06:06.583721 7f5c973a57a0 20 read_log 192803'16656 (192803'16655) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:489 2014-08-11 12:53:40.760481 2014-08-13 18:06:06.583733 7f5c973a57a0 20 read_log 192803'16657 (192803'16656) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:490 2014-08-11 12:53:40.788894 2014-08-13 18:06:06.583745 7f5c973a57a0 20 read_log 192803'16658 (192803'16657) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:491 2014-08-11 12:53:40.811793 2014-08-13 18:06:06.583757 7f5c973a57a0 20 read_log 192803'16659 (192803'16658) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:492 2014-08-11 12:53:40.882334 2014-08-13 18:06:06.583771 7f5c973a57a0 20 read_log 192803'16660 (192803'16659) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:493 2014-08-11 12:53:40.922434 2014-08-13 18:06:06.583783 7f5c973a57a0 20 read_log 192803'16661 (192803'16660) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:494 2014-08-11 12:53:40.960210 2014-08-13 18:06:06.583795 7f5c973a57a0 20 read_log 192803'16662 (192803'16661) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:495 2014-08-11 12:53:41.019327 2014-08-13 18:06:06.583809 7f5c973a57a0 20 read_log 192803'16663 (192803'16662) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:496 2014-08-11 12:53:41.061599 2014-08-13 18:06:06.583821 7f5c973a57a0 20 read_log 192803'16664 (192803'16663) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:497 2014-08-11 12:53:41.560077 2014-08-13 18:06:06.583833 7f5c973a57a0 20 read_log 192803'16665 (192803'16664) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:498 2014-08-11 12:53:41.590114 2014-08-13 18:06:06.583846 7f5c973a57a0 20 read_log 192803'16666 (192803'16665) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:499 2014-08-11 12:53:41.627305 2014-08-13 18:06:06.583858 7f5c973a57a0 20 read_log 192803'16667 (192803'16666) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:500 2014-08-11 12:53:41.654869 2014-08-13 18:06:06.583870 7f5c973a57a0 20 read_log 192803'16668 (192803'16667) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:501 2014-08-11 12:53:41.687069 2014-08-13 18:06:06.583885 7f5c973a57a0 20 read_log 192803'16669 (192803'16668) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:502 2014-08-11 12:53:41.732012 2014-08-13 18:06:06.583898 7f5c973a57a0 20 read_log 192803'16670 (192803'16669) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:503 2014-08-11 12:53:41.759707 2014-08-13 18:06:06.583910 7f5c973a57a0 20 read_log 192803'16671 (192803'16670) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:504 2014-08-11 12:53:41.804005 2014-08-13 18:06:06.583924 7f5c973a57a0 20 read_log 192803'16672 (192803'16671) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:505 2014-08-11 12:53:41.831109 2014-08-13 18:06:06.583936 7f5c973a57a0 20 read_log 192803'16673 (192803'16672) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:506 2014-08-11 12:53:41.864509 2014-08-13 18:06:06.583972 7f5c973a57a0 20 read_log 192803'16674 (192803'16673) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:507 2014-08-11 12:53:42.273231 2014-08-13 18:06:06.583988 7f5c973a57a0 20 read_log 192803'16675 (192803'16674) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:508 2014-08-11 12:53:42.307259 2014-08-13 18:06:06.584000 7f5c973a57a0 20 read_log 192803'16676 (192803'16675) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:509 2014-08-11 12:53:42.346470 2014-08-13 18:06:06.584013 7f5c973a57a0 20 read_log 192803'16677 (192803'16676) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:510 2014-08-11 12:53:42.377963 2014-08-13 18:06:06.584025 7f5c973a57a0 20 read_log 192803'16678 (192803'16677) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:511 2014-08-11 12:53:42.413786 2014-08-13 18:06:06.584037 7f5c973a57a0 20 read_log 192803'16679 (192803'16678) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:512 2014-08-11 12:53:42.509080 2014-08-13 18:06:06.584052 7f5c973a57a0 20 read_log 192803'16680 (192803'16679) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:513 2014-08-11 12:53:42.562471 2014-08-13 18:06:06.584064 7f5c973a57a0 20 read_log 192803'16681 (192803'16680) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:514 2014-08-11 12:53:42.600365 2014-08-13 18:06:06.584079 7f5c973a57a0 20 read_log 192803'16682 (192803'16681) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:515 2014-08-11 12:53:42.635350 2014-08-13 18:06:06.584092 7f5c973a57a0 20 read_log 192803'16683 (192803'16682) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:516 2014-08-11 12:53:42.672246 2014-08-13 18:06:06.584104 7f5c973a57a0 20 read_log 192803'16684 (192803'16683) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:517 2014-08-11 12:53:43.082119 2014-08-13 18:06:06.584116 7f5c973a57a0 20 read_log 192803'16685 (192803'16684) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:518 2014-08-11 12:53:43.110917 2014-08-13 18:06:06.584128 7f5c973a57a0 20 read_log 192803'16686 (192803'16685) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:519 2014-08-11 12:53:43.150596 2014-08-13 18:06:06.584166 7f5c973a57a0 20 read_log 192803'16687 (192803'16686) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:520 2014-08-11 12:53:43.180390 2014-08-13 18:06:06.584179 7f5c973a57a0 20 read_log 192803'16688 (192803'16687) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:521 2014-08-11 12:53:43.216557 2014-08-13 18:06:06.584192 7f5c973a57a0 20 read_log 192803'16689 (192803'16688) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:522 2014-08-11 12:53:43.255843 2014-08-13 18:06:06.584206 7f5c973a57a0 20 read_log 192803'16690 (192803'16689) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:523 2014-08-11 12:53:43.297249 2014-08-13 18:06:06.584218 7f5c973a57a0 20 read_log 192803'16691 (192803'16690) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:524 2014-08-11 12:53:43.337742 2014-08-13 18:06:06.584233 7f5c973a57a0 20 read_log 192803'16692 (192803'16691) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:525 2014-08-11 12:53:43.372497 2014-08-13 18:06:06.584245 7f5c973a57a0 20 read_log 192803'16693 (192803'16692) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:526 2014-08-11 12:53:43.409087 2014-08-13 18:06:06.584258 7f5c973a57a0 20 read_log 192803'16694 (192803'16693) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:527 2014-08-11 12:53:43.819985 2014-08-13 18:06:06.584274 7f5c973a57a0 20 read_log 192803'16695 (192803'16694) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:528 2014-08-11 12:53:43.848431 2014-08-13 18:06:06.584287 7f5c973a57a0 20 read_log 192803'16696 (192803'16695) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:529 2014-08-11 12:53:43.898321 2014-08-13 18:06:06.584299 7f5c973a57a0 20 read_log 192803'16697 (192803'16696) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:530 2014-08-11 12:53:43.929545 2014-08-13 18:06:06.584323 7f5c973a57a0 20 read_log 192803'16698 (192803'16697) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:531 2014-08-11 12:53:43.966036 2014-08-13 18:06:06.584338 7f5c973a57a0 20 read_log 192803'16699 (192803'16698) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:532 2014-08-11 12:53:44.011612 2014-08-13 18:06:06.584351 7f5c973a57a0 20 read_log 192803'16700 (192803'16699) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:533 2014-08-11 12:53:44.039472 2014-08-13 18:06:06.584367 7f5c973a57a0 20 read_log 192803'16701 (192803'16700) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:534 2014-08-11 12:53:44.092048 2014-08-13 18:06:06.584380 7f5c973a57a0 20 read_log 192803'16702 (192803'16701) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:535 2014-08-11 12:53:44.121326 2014-08-13 18:06:06.584409 7f5c973a57a0 20 read_log 192803'16703 (192803'16702) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:536 2014-08-11 12:53:44.149746 2014-08-13 18:06:06.584425 7f5c973a57a0 20 read_log 192803'16704 (192803'16703) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:537 2014-08-11 12:53:44.566869 2014-08-13 18:06:06.584437 7f5c973a57a0 20 read_log 192803'16705 (192803'16704) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:538 2014-08-11 12:53:44.595685 2014-08-13 18:06:06.584456 7f5c973a57a0 20 read_log 192803'16706 (192803'16705) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:539 2014-08-11 12:53:44.635157 2014-08-13 18:06:06.584470 7f5c973a57a0 20 read_log 192803'16707 (192803'16706) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:540 2014-08-11 12:53:44.674059 2014-08-13 18:06:06.584482 7f5c973a57a0 20 read_log 192803'16708 (192803'16707) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:541 2014-08-11 12:53:44.703278 2014-08-13 18:06:06.584495 7f5c973a57a0 20 read_log 192803'16709 (192803'16708) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:542 2014-08-11 12:53:44.742132 2014-08-13 18:06:06.584517 7f5c973a57a0 20 read_log 192803'16710 (192803'16709) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:543 2014-08-11 12:53:44.775718 2014-08-13 18:06:06.584529 7f5c973a57a0 20 read_log 192803'16711 (192803'16710) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:544 2014-08-11 12:53:44.808810 2014-08-13 18:06:06.584541 7f5c973a57a0 20 read_log 192803'16712 (192803'16711) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:545 2014-08-11 12:53:44.851986 2014-08-13 18:06:06.584554 7f5c973a57a0 20 read_log 192803'16713 (192803'16712) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:546 2014-08-11 12:53:44.893668 2014-08-13 18:06:06.584566 7f5c973a57a0 20 read_log 192803'16714 (192803'16713) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:547 2014-08-11 12:53:45.298983 2014-08-13 18:06:06.584578 7f5c973a57a0 20 read_log 192803'16715 (192803'16714) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:548 2014-08-11 12:53:45.327193 2014-08-13 18:06:06.584593 7f5c973a57a0 20 read_log 192803'16716 (192803'16715) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:549 2014-08-11 12:53:45.357600 2014-08-13 18:06:06.584605 7f5c973a57a0 20 read_log 192803'16717 (192803'16716) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:550 2014-08-11 12:53:45.396183 2014-08-13 18:06:06.584618 7f5c973a57a0 20 read_log 192803'16718 (192803'16717) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:551 2014-08-11 12:53:45.424669 2014-08-13 18:06:06.584630 7f5c973a57a0 20 read_log 192803'16719 (192803'16718) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:552 2014-08-11 12:53:45.469147 2014-08-13 18:06:06.584642 7f5c973a57a0 20 read_log 192803'16720 (192803'16719) modify 81bde03d/core.47650+10+2/head//24 by client.1283438.0:553 2014-08-11 12:53:45.493673 2014-08-13 18:06:06.584654 7f5c973a57a0 20 read_log 192803'16721 (192803'16720) clean 81bde03d/core.47650+10+2/head//24 by osd.4.0:18691 2014-08-11 12:55:06.554113 2014-08-13 18:06:06.584692 7f5c973a57a0 20 read_log 192803'16722 (192803'16721) delete 81bde03d/core.47650+10+2/head//24 by osd.4.0:18692 2014-08-11 12:55:09.349518 2014-08-13 18:06:06.584706 7f5c973a57a0 20 read_log 192834'16723 (192834'16723) modify 3d/hit_set_24.3d_archive_2014-08-11 12:52:38.941261_2014-08-11 15:23:36.570365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.570592 2014-08-13 18:06:06.584726 7f5c973a57a0 20 read_log 192834'16724 (192803'16165) delete 3d/hit_set_24.3d_archive_2014-08-11 10:24:43.584057_2014-08-11 12:52:38.940934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:36.570592 2014-08-13 18:06:06.584744 7f5c973a57a0 20 read_log 192834'16741 (0'0) promote 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by osd.4.0:18720 2014-08-11 15:23:39.328522 2014-08-13 18:06:06.584757 7f5c973a57a0 20 read_log 192834'16742 (192834'16741) modify 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.933448.0:958 2014-08-11 15:23:39.320690 2014-08-13 18:06:06.584769 7f5c973a57a0 20 read_log 192834'16787 (0'0) promote 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by osd.4.0:18766 2014-08-11 15:23:54.571565 2014-08-13 18:06:06.584782 7f5c973a57a0 20 read_log 192834'16788 (192834'16787) modify 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.933448.0:3932 2014-08-11 15:23:54.562641 2014-08-13 18:06:06.584794 7f5c973a57a0 20 read_log 192834'16791 (0'0) promote 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:18770 2014-08-11 15:23:54.945744 2014-08-13 18:06:06.584806 7f5c973a57a0 20 read_log 192834'16792 (192834'16791) modify 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by client.933448.0:3999 2014-08-11 15:23:54.938859 2014-08-13 18:06:06.584819 7f5c973a57a0 20 read_log 192834'16807 (0'0) promote 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:18786 2014-08-11 15:24:02.166372 2014-08-13 18:06:06.584831 7f5c973a57a0 20 read_log 192834'16808 (192834'16807) modify 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by client.933448.0:5597 2014-08-11 15:24:02.159339 2014-08-13 18:06:06.584843 7f5c973a57a0 20 read_log 192834'16829 (0'0) promote 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by osd.4.0:18808 2014-08-11 15:24:07.334638 2014-08-13 18:06:06.584856 7f5c973a57a0 20 read_log 192834'16830 (192834'16829) modify 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.933448.0:6927 2014-08-11 15:24:07.325617 2014-08-13 18:06:06.584868 7f5c973a57a0 20 read_log 192834'16839 (0'0) promote f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by osd.4.0:18818 2014-08-11 15:24:11.339202 2014-08-13 18:06:06.584880 7f5c973a57a0 20 read_log 192834'16840 (192834'16839) modify f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.933448.0:7770 2014-08-11 15:24:11.331914 2014-08-13 18:06:06.584892 7f5c973a57a0 20 read_log 192834'16885 (0'0) promote 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by osd.4.0:18864 2014-08-11 15:24:21.423659 2014-08-13 18:06:06.584905 7f5c973a57a0 20 read_log 192834'16886 (192834'16885) modify 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.933448.0:10047 2014-08-11 15:24:21.413849 2014-08-13 18:06:06.584917 7f5c973a57a0 20 read_log 192834'16887 (0'0) promote 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by osd.4.0:18866 2014-08-11 15:24:22.691548 2014-08-13 18:06:06.584929 7f5c973a57a0 20 read_log 192834'16888 (192834'16887) modify 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.933448.0:10332 2014-08-11 15:24:22.680116 2014-08-13 18:06:06.584949 7f5c973a57a0 20 read_log 192834'16889 (0'0) promote 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by osd.4.0:18868 2014-08-11 15:24:23.610327 2014-08-13 18:06:06.584962 7f5c973a57a0 20 read_log 192834'16890 (192834'16889) modify 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.933448.0:10526 2014-08-11 15:24:23.601067 2014-08-13 18:06:06.584975 7f5c973a57a0 20 read_log 192834'16901 (0'0) promote 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by osd.4.0:18880 2014-08-11 15:24:28.254812 2014-08-13 18:06:06.584987 7f5c973a57a0 20 read_log 192834'16902 (192834'16901) modify 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.933448.0:11548 2014-08-11 15:24:28.247538 2014-08-13 18:06:06.585002 7f5c973a57a0 20 read_log 192834'16959 (0'0) promote 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by osd.4.0:18938 2014-08-11 15:24:51.521230 2014-08-13 18:06:06.585015 7f5c973a57a0 20 read_log 192834'16960 (192834'16959) modify 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.933448.0:15838 2014-08-11 15:24:51.513156 2014-08-13 18:06:06.585029 7f5c973a57a0 20 read_log 192834'16979 (0'0) promote c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:18958 2014-08-11 15:24:59.312062 2014-08-13 18:06:06.585043 7f5c973a57a0 20 read_log 192834'16980 (192834'16979) modify c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by client.933448.0:17334 2014-08-11 15:24:59.304221 2014-08-13 18:06:06.585055 7f5c973a57a0 20 read_log 192834'16999 (0'0) promote 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by osd.4.0:18978 2014-08-11 15:25:06.272830 2014-08-13 18:06:06.585067 7f5c973a57a0 20 read_log 192834'17000 (192834'16999) modify 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.933448.0:18669 2014-08-11 15:25:06.263358 2014-08-13 18:06:06.585082 7f5c973a57a0 20 read_log 192834'17017 (0'0) promote ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:18996 2014-08-11 15:25:13.517677 2014-08-13 18:06:06.585094 7f5c973a57a0 20 read_log 192834'17018 (192834'17017) modify ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by client.933448.0:19809 2014-08-11 15:25:13.507884 2014-08-13 18:06:06.585107 7f5c973a57a0 20 read_log 192834'17025 (0'0) promote 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19004 2014-08-11 15:25:16.787107 2014-08-13 18:06:06.585121 7f5c973a57a0 20 read_log 192834'17026 (192834'17025) modify 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by client.933448.0:20139 2014-08-11 15:25:16.778486 2014-08-13 18:06:06.585134 7f5c973a57a0 20 read_log 192834'17063 (192834'16742) modify 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.933448.0:23249 2014-08-11 15:25:39.148826 2014-08-13 18:06:06.585153 7f5c973a57a0 20 read_log 192834'17086 (192834'16788) modify 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.933448.0:26223 2014-08-11 15:25:43.131286 2014-08-13 18:06:06.585168 7f5c973a57a0 20 read_log 192834'17088 (192834'16792) modify 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by client.933448.0:26290 2014-08-11 15:25:43.229811 2014-08-13 18:06:06.585181 7f5c973a57a0 20 read_log 192834'17096 (192834'16808) modify 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by client.933448.0:27888 2014-08-11 15:25:45.229875 2014-08-13 18:06:06.585195 7f5c973a57a0 20 read_log 192834'17107 (192834'16830) modify 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.933448.0:29218 2014-08-11 15:25:47.121765 2014-08-13 18:06:06.585208 7f5c973a57a0 20 read_log 192834'17112 (192834'16840) modify f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.933448.0:30061 2014-08-11 15:25:48.316419 2014-08-13 18:06:06.585233 7f5c973a57a0 20 read_log 192834'17135 (192834'16886) modify 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.933448.0:32338 2014-08-11 15:25:51.512288 2014-08-13 18:06:06.585248 7f5c973a57a0 20 read_log 192834'17136 (192834'16888) modify 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.933448.0:32623 2014-08-11 15:25:52.116867 2014-08-13 18:06:06.585264 7f5c973a57a0 20 read_log 192834'17137 (192834'16890) modify 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.933448.0:32817 2014-08-11 15:25:52.402448 2014-08-13 18:06:06.585278 7f5c973a57a0 20 read_log 192834'17143 (192834'16902) modify 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.933448.0:33839 2014-08-11 15:25:54.262153 2014-08-13 18:06:06.585297 7f5c973a57a0 20 read_log 192834'17172 (192834'16960) modify 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.933448.0:38129 2014-08-11 15:26:00.542281 2014-08-13 18:06:06.585315 7f5c973a57a0 20 read_log 192834'17182 (192834'16980) modify c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by client.933448.0:39625 2014-08-11 15:26:02.000371 2014-08-13 18:06:06.585351 7f5c973a57a0 20 read_log 192834'17192 (192834'17000) modify 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.933448.0:40960 2014-08-11 15:26:02.799318 2014-08-13 18:06:06.585371 7f5c973a57a0 20 read_log 192834'17201 (192834'17018) modify ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by client.933448.0:42100 2014-08-11 15:26:03.153625 2014-08-13 18:06:06.585393 7f5c973a57a0 20 read_log 192834'17205 (192834'17026) modify 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by client.933448.0:42430 2014-08-11 15:26:03.277304 2014-08-13 18:06:06.585412 7f5c973a57a0 20 read_log 192834'17222 (0'0) promote 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by osd.4.0:19201 2014-08-11 15:26:14.658560 2014-08-13 18:06:06.585425 7f5c973a57a0 20 read_log 192834'17223 (192834'17222) modify 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.1883634.0:97 2014-08-11 15:26:14.650082 2014-08-13 18:06:06.585444 7f5c973a57a0 20 read_log 192834'17226 (0'0) promote 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by osd.4.0:19205 2014-08-11 15:26:16.831773 2014-08-13 18:06:06.585461 7f5c973a57a0 20 read_log 192834'17227 (192834'17226) modify 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.1883634.0:642 2014-08-11 15:26:16.824046 2014-08-13 18:06:06.585474 7f5c973a57a0 20 read_log 192834'17258 (0'0) promote 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by osd.4.0:19237 2014-08-11 15:26:24.635586 2014-08-13 18:06:06.585486 7f5c973a57a0 20 read_log 192834'17259 (192834'17258) modify 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.1883634.0:2767 2014-08-11 15:26:24.626372 2014-08-13 18:06:06.585512 7f5c973a57a0 20 read_log 192834'17262 (0'0) promote d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by osd.4.0:19241 2014-08-11 15:26:26.394856 2014-08-13 18:06:06.585525 7f5c973a57a0 20 read_log 192834'17263 (192834'17262) modify d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.1883634.0:3264 2014-08-11 15:26:26.387203 2014-08-13 18:06:06.585538 7f5c973a57a0 20 read_log 192834'17274 (0'0) promote 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by osd.4.0:19253 2014-08-11 15:26:27.823878 2014-08-13 18:06:06.585550 7f5c973a57a0 20 read_log 192834'17275 (192834'17274) modify 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.1883634.0:3667 2014-08-11 15:26:27.814802 2014-08-13 18:06:06.585563 7f5c973a57a0 20 read_log 192834'17292 (0'0) promote 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19271 2014-08-11 15:26:32.750178 2014-08-13 18:06:06.585583 7f5c973a57a0 20 read_log 192834'17293 (192834'17292) modify 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.1883634.0:4968 2014-08-11 15:26:32.742056 2014-08-13 18:06:06.585598 7f5c973a57a0 20 read_log 192834'17318 (0'0) promote 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by osd.4.0:19297 2014-08-11 15:26:39.831754 2014-08-13 18:06:06.585611 7f5c973a57a0 20 read_log 192834'17319 (192834'17318) modify 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.1883634.0:6880 2014-08-11 15:26:39.820795 2014-08-13 18:06:06.585623 7f5c973a57a0 20 read_log 192834'17328 (0'0) promote 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by osd.4.0:19307 2014-08-11 15:26:41.446539 2014-08-13 18:06:06.585636 7f5c973a57a0 20 read_log 192834'17329 (192834'17328) modify 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.1883634.0:7270 2014-08-11 15:26:41.438433 2014-08-13 18:06:06.585655 7f5c973a57a0 20 read_log 192834'17332 (0'0) promote 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by osd.4.0:19311 2014-08-11 15:26:41.568606 2014-08-13 18:06:06.585672 7f5c973a57a0 20 read_log 192834'17333 (192834'17332) modify 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.1883634.0:7299 2014-08-11 15:26:41.560553 2014-08-13 18:06:06.585685 7f5c973a57a0 20 read_log 192834'17334 (0'0) promote 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by osd.4.0:19313 2014-08-11 15:26:41.769729 2014-08-13 18:06:06.585705 7f5c973a57a0 20 read_log 192834'17335 (192834'17334) modify 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.1883634.0:7341 2014-08-11 15:26:41.759457 2014-08-13 18:06:06.585721 7f5c973a57a0 20 read_log 192834'17352 (0'0) promote f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19331 2014-08-11 15:26:45.177189 2014-08-13 18:06:06.585733 7f5c973a57a0 20 read_log 192834'17353 (192834'17352) modify f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.1883634.0:8151 2014-08-11 15:26:45.165684 2014-08-13 18:06:06.585748 7f5c973a57a0 20 read_log 192834'17390 (0'0) promote 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19369 2014-08-11 15:26:52.691382 2014-08-13 18:06:06.585760 7f5c973a57a0 20 read_log 192834'17391 (192834'17390) modify 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.1883634.0:9853 2014-08-11 15:26:52.682402 2014-08-13 18:06:06.585775 7f5c973a57a0 20 read_log 192834'17456 (0'0) promote 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19435 2014-08-11 15:27:06.493918 2014-08-13 18:06:06.585787 7f5c973a57a0 20 read_log 192834'17457 (192834'17456) modify 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.1883634.0:12865 2014-08-11 15:27:06.484000 2014-08-13 18:06:06.585802 7f5c973a57a0 20 read_log 192834'17474 (0'0) promote 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by osd.4.0:19453 2014-08-11 15:27:09.886482 2014-08-13 18:06:06.585815 7f5c973a57a0 20 read_log 192834'17475 (192834'17474) modify 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.1883634.0:13497 2014-08-11 15:27:09.877448 2014-08-13 18:06:06.585829 7f5c973a57a0 20 read_log 192834'17486 (0'0) promote 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by osd.4.0:19465 2014-08-11 15:27:15.640401 2014-08-13 18:06:06.585842 7f5c973a57a0 20 read_log 192834'17487 (192834'17486) modify 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.1883634.0:14844 2014-08-11 15:27:15.632524 2014-08-13 18:06:06.585854 7f5c973a57a0 20 read_log 192834'17488 (0'0) promote 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by osd.4.0:19467 2014-08-11 15:27:16.884441 2014-08-13 18:06:06.585874 7f5c973a57a0 20 read_log 192834'17489 (192834'17488) modify 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.1883634.0:15065 2014-08-11 15:27:16.876357 2014-08-13 18:06:06.585889 7f5c973a57a0 20 read_log 192834'17504 (0'0) promote 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by osd.4.0:19483 2014-08-11 15:27:25.739966 2014-08-13 18:06:06.585901 7f5c973a57a0 20 read_log 192834'17505 (192834'17504) modify 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.1883634.0:16062 2014-08-11 15:27:25.730843 2014-08-13 18:06:06.585914 7f5c973a57a0 20 read_log 192834'17552 (0'0) promote e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by osd.4.0:19531 2014-08-11 15:27:46.594190 2014-08-13 18:06:06.585937 7f5c973a57a0 20 read_log 192834'17553 (192834'17552) modify e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.1883634.0:18587 2014-08-11 15:27:46.584688 2014-08-13 18:06:06.585950 7f5c973a57a0 20 read_log 192834'17588 (0'0) promote f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by osd.4.0:19567 2014-08-11 15:27:57.198999 2014-08-13 18:06:06.585963 7f5c973a57a0 20 read_log 192834'17589 (192834'17588) modify f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.1883634.0:20449 2014-08-11 15:27:57.190971 2014-08-13 18:06:06.585991 7f5c973a57a0 20 read_log 192834'17596 (0'0) promote 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by osd.4.0:19575 2014-08-11 15:28:01.957683 2014-08-13 18:06:06.586005 7f5c973a57a0 20 read_log 192834'17597 (192834'17596) modify 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.1883634.0:20904 2014-08-11 15:28:01.949410 2014-08-13 18:06:06.586017 7f5c973a57a0 20 read_log 192834'17604 (192834'17201) clean ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:19583 2014-08-11 15:28:04.379181 2014-08-13 18:06:06.586035 7f5c973a57a0 20 read_log 192834'17605 (192834'17182) clean c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:19584 2014-08-11 15:28:04.380338 2014-08-13 18:06:06.586048 7f5c973a57a0 20 read_log 192834'17606 (192834'17096) clean 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:19585 2014-08-11 15:28:04.386074 2014-08-13 18:06:06.586072 7f5c973a57a0 20 read_log 192834'17607 (192834'17205) clean 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19586 2014-08-11 15:28:04.414328 2014-08-13 18:06:06.586085 7f5c973a57a0 20 read_log 192834'17608 (192834'17088) clean 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:19587 2014-08-11 15:28:04.416848 2014-08-13 18:06:06.586098 7f5c973a57a0 20 read_log 192834'17609 (192834'17293) clean 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19588 2014-08-11 15:28:04.489517 2014-08-13 18:06:06.586110 7f5c973a57a0 20 read_log 192834'17610 (192834'17391) clean 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19589 2014-08-11 15:28:04.489825 2014-08-13 18:06:06.586125 7f5c973a57a0 20 read_log 192834'17611 (192834'17353) clean f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19590 2014-08-11 15:28:04.490938 2014-08-13 18:06:06.586137 7f5c973a57a0 20 read_log 192834'17624 (192834'17604) delete ff59103d/benchmark_data_storage0111-ib.pocstor_26659_object19808/head//24 by osd.4.0:19603 2014-08-11 15:28:05.387392 2014-08-13 18:06:06.586155 7f5c973a57a0 20 read_log 192834'17625 (192834'17605) delete c3ef303d/benchmark_data_storage0111-ib.pocstor_26659_object17333/head//24 by osd.4.0:19604 2014-08-11 15:28:05.392149 2014-08-13 18:06:06.586168 7f5c973a57a0 20 read_log 192834'17626 (192834'17609) delete 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19605 2014-08-11 15:28:05.394309 2014-08-13 18:06:06.586194 7f5c973a57a0 20 read_log 192834'17629 (192834'17606) delete 5cfaa03d/benchmark_data_storage0111-ib.pocstor_26659_object5596/head//24 by osd.4.0:19608 2014-08-11 15:28:05.399035 2014-08-13 18:06:06.586209 7f5c973a57a0 20 read_log 192834'17630 (192834'17607) delete 1472c03d/benchmark_data_storage0111-ib.pocstor_26659_object20138/head//24 by osd.4.0:19609 2014-08-11 15:28:05.401373 2014-08-13 18:06:06.586233 7f5c973a57a0 20 read_log 192834'17631 (192834'17610) delete 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19610 2014-08-11 15:28:05.401622 2014-08-13 18:06:06.586249 7f5c973a57a0 20 read_log 192834'17632 (192834'17608) delete 11ebe03d/benchmark_data_storage0111-ib.pocstor_26659_object3998/head//24 by osd.4.0:19611 2014-08-11 15:28:05.403853 2014-08-13 18:06:06.586261 7f5c973a57a0 20 read_log 192834'17633 (192834'17611) delete f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19612 2014-08-11 15:28:05.404107 2014-08-13 18:06:06.586274 7f5c973a57a0 20 read_log 192834'17650 (0'0) promote 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by osd.4.0:19629 2014-08-11 15:28:08.044681 2014-08-13 18:06:06.586293 7f5c973a57a0 20 read_log 192834'17651 (192834'17650) modify 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.1883634.0:22318 2014-08-11 15:28:08.035723 2014-08-13 18:06:06.586306 7f5c973a57a0 20 read_log 192834'17654 (0'0) promote 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by osd.4.0:19633 2014-08-11 15:28:11.779377 2014-08-13 18:06:06.586350 7f5c973a57a0 20 read_log 192834'17655 (192834'17654) modify 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.1883634.0:22932 2014-08-11 15:28:11.771636 2014-08-13 18:06:06.586369 7f5c973a57a0 20 read_log 192834'17660 (192834'17223) modify 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.1883634.0:23421 2014-08-11 15:28:16.311949 2014-08-13 18:06:06.586390 7f5c973a57a0 20 read_log 192834'17662 (192834'17457) clean 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19641 2014-08-11 15:28:16.395692 2014-08-13 18:06:06.586406 7f5c973a57a0 20 read_log 192834'17674 (192834'17662) delete 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19653 2014-08-11 15:28:16.821845 2014-08-13 18:06:06.586419 7f5c973a57a0 20 read_log 192834'17689 (192834'17227) modify 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.1883634.0:23966 2014-08-11 15:28:18.150896 2014-08-13 18:06:06.586431 7f5c973a57a0 20 read_log 192834'17706 (192834'17259) modify 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.1883634.0:26091 2014-08-11 15:28:22.601610 2014-08-13 18:06:06.586444 7f5c973a57a0 20 read_log 192834'17708 (192834'17263) modify d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.1883634.0:26588 2014-08-11 15:28:23.289083 2014-08-13 18:06:06.586456 7f5c973a57a0 20 read_log 192834'17716 (192834'17275) modify 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.1883634.0:26991 2014-08-11 15:28:23.871261 2014-08-13 18:06:06.586468 7f5c973a57a0 20 read_log 192834'17726 (0'0) promote 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:19705 2014-08-11 15:28:25.643844 2014-08-13 18:06:06.586481 7f5c973a57a0 20 read_log 192834'17727 (192834'17726) modify 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.1883634.0:28292 2014-08-11 15:28:25.613858 2014-08-13 18:06:06.586493 7f5c973a57a0 20 read_log 192834'17741 (192834'17319) modify 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.1883634.0:30204 2014-08-11 15:28:28.223943 2014-08-13 18:06:06.586506 7f5c973a57a0 20 read_log 192834'17746 (192834'17329) modify 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.1883634.0:30594 2014-08-11 15:28:28.915971 2014-08-13 18:06:06.586526 7f5c973a57a0 20 read_log 192834'17748 (192834'17333) modify 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.1883634.0:30623 2014-08-11 15:28:28.994963 2014-08-13 18:06:06.586539 7f5c973a57a0 20 read_log 192834'17749 (192834'17335) modify 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.1883634.0:30665 2014-08-11 15:28:29.056985 2014-08-13 18:06:06.586551 7f5c973a57a0 20 read_log 192834'17760 (0'0) promote f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:19739 2014-08-11 15:28:30.608070 2014-08-13 18:06:06.586563 7f5c973a57a0 20 read_log 192834'17761 (192834'17760) modify f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.1883634.0:31475 2014-08-11 15:28:30.581590 2014-08-13 18:06:06.586576 7f5c973a57a0 20 read_log 192834'17778 (192834'17778) modify 3d/hit_set_24.3d_archive_2014-08-11 15:23:36.570597_2014-08-11 15:28:32.651919/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:32.652610 2014-08-13 18:06:06.586589 7f5c973a57a0 20 read_log 192834'17779 (192834'16723) delete 3d/hit_set_24.3d_archive_2014-08-11 12:52:38.941261_2014-08-11 15:23:36.570365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:32.652610 2014-08-13 18:06:06.586603 7f5c973a57a0 20 read_log 192834'17783 (0'0) promote 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:19761 2014-08-11 15:28:32.883091 2014-08-13 18:06:06.586615 7f5c973a57a0 20 read_log 192834'17784 (192834'17783) modify 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.1883634.0:33177 2014-08-11 15:28:32.855093 2014-08-13 18:06:06.586628 7f5c973a57a0 20 read_log 192834'17817 (0'0) promote 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:19795 2014-08-11 15:28:37.730044 2014-08-13 18:06:06.586640 7f5c973a57a0 20 read_log 192834'17818 (192834'17817) modify 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.1883634.0:36189 2014-08-11 15:28:37.704734 2014-08-13 18:06:06.586652 7f5c973a57a0 20 read_log 192834'17827 (192834'17475) modify 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.1883634.0:36821 2014-08-11 15:28:38.451599 2014-08-13 18:06:06.586666 7f5c973a57a0 20 read_log 192834'17835 (192834'17487) modify 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.1883634.0:38168 2014-08-11 15:28:40.313681 2014-08-13 18:06:06.586678 7f5c973a57a0 20 read_log 192834'17836 (192834'17489) modify 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.1883634.0:38389 2014-08-11 15:28:40.572007 2014-08-13 18:06:06.586690 7f5c973a57a0 20 read_log 192834'17844 (192834'17505) modify 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.1883634.0:39386 2014-08-11 15:28:42.119497 2014-08-13 18:06:06.586702 7f5c973a57a0 20 read_log 192834'17868 (192834'17553) modify e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.1883634.0:41911 2014-08-11 15:28:45.573306 2014-08-13 18:06:06.586714 7f5c973a57a0 20 read_log 192834'17886 (192834'17589) modify f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.1883634.0:43773 2014-08-11 15:28:46.658664 2014-08-13 18:06:06.586727 7f5c973a57a0 20 read_log 192834'17890 (192834'17597) modify 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.1883634.0:44228 2014-08-11 15:28:46.799960 2014-08-13 18:06:06.586760 7f5c973a57a0 20 read_log 192834'17897 (192834'17651) modify 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.1883634.0:45642 2014-08-11 15:28:47.341577 2014-08-13 18:06:06.586772 7f5c973a57a0 20 read_log 192834'17899 (192834'17655) modify 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.1883634.0:46256 2014-08-11 15:28:47.538339 2014-08-13 18:06:06.586822 7f5c973a57a0 20 read_log 192888'17901 (192888'17901) modify 3d/hit_set_24.3d_archive_2014-08-11 16:02:02.114927_2014-08-11 16:44:56.253783/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.254066 2014-08-13 18:06:06.586838 7f5c973a57a0 20 read_log 192888'17902 (192834'17778) delete 3d/hit_set_24.3d_archive_2014-08-11 15:23:36.570597_2014-08-11 15:28:32.651919/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.254066 2014-08-13 18:06:06.586851 7f5c973a57a0 20 read_log 192888'17905 (0'0) promote c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by osd.4.0:21386 2014-08-11 16:44:56.463021 2014-08-13 18:06:06.586865 7f5c973a57a0 20 read_log 192888'17906 (192888'17905) modify c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.1315259.0:368 2014-08-11 16:44:56.454984 2014-08-13 18:06:06.586878 7f5c973a57a0 20 read_log 192888'17913 (0'0) promote 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by osd.4.0:21415 2014-08-11 16:45:00.433228 2014-08-13 18:06:06.586892 7f5c973a57a0 20 read_log 192888'17914 (192888'17913) modify 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.1315259.0:1355 2014-08-11 16:45:00.424399 2014-08-13 18:06:06.586905 7f5c973a57a0 20 read_log 192888'17915 (0'0) promote fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by osd.4.0:21419 2014-08-11 16:45:00.792426 2014-08-13 18:06:06.586917 7f5c973a57a0 20 read_log 192888'17916 (192888'17915) modify fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.1315259.0:1437 2014-08-11 16:45:00.781710 2014-08-13 18:06:06.586930 7f5c973a57a0 20 read_log 192888'17919 (0'0) promote 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by osd.4.0:21472 2014-08-11 16:45:06.785590 2014-08-13 18:06:06.586944 7f5c973a57a0 20 read_log 192888'17920 (192888'17919) modify 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.1315259.0:3051 2014-08-11 16:45:06.776409 2014-08-13 18:06:06.586957 7f5c973a57a0 20 read_log 192888'17925 (0'0) promote c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by osd.4.0:21524 2014-08-11 16:45:14.147077 2014-08-13 18:06:06.586971 7f5c973a57a0 20 read_log 192888'17926 (192888'17925) modify c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.1315259.0:5015 2014-08-11 16:45:14.139506 2014-08-13 18:06:06.586983 7f5c973a57a0 20 read_log 192888'17927 (0'0) promote 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by osd.4.0:21556 2014-08-11 16:45:17.240328 2014-08-13 18:06:06.586997 7f5c973a57a0 20 read_log 192888'17928 (192888'17927) modify 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.1315259.0:5848 2014-08-11 16:45:17.231647 2014-08-13 18:06:06.587010 7f5c973a57a0 20 read_log 192888'17933 (0'0) promote bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by osd.4.0:21594 2014-08-11 16:45:23.125534 2014-08-13 18:06:06.587024 7f5c973a57a0 20 read_log 192888'17934 (192888'17933) modify bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.1315259.0:7478 2014-08-11 16:45:23.117962 2014-08-13 18:06:06.587037 7f5c973a57a0 20 read_log 192888'17935 (0'0) promote d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by osd.4.0:21606 2014-08-11 16:45:26.040988 2014-08-13 18:06:06.587051 7f5c973a57a0 20 read_log 192888'17936 (192888'17935) modify d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.1315259.0:8276 2014-08-11 16:45:26.031534 2014-08-13 18:06:06.587064 7f5c973a57a0 20 read_log 192888'17941 (0'0) promote eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by osd.4.0:21658 2014-08-11 16:45:33.761590 2014-08-13 18:06:06.587087 7f5c973a57a0 20 read_log 192888'17942 (192888'17941) modify eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.1315259.0:10295 2014-08-11 16:45:33.752567 2014-08-13 18:06:06.587102 7f5c973a57a0 20 read_log 192888'17943 (0'0) promote b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by osd.4.0:21688 2014-08-11 16:45:36.712979 2014-08-13 18:06:06.587115 7f5c973a57a0 20 read_log 192888'17944 (192888'17943) modify b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.1315259.0:11024 2014-08-11 16:45:36.703544 2014-08-13 18:06:06.587129 7f5c973a57a0 20 read_log 192888'17945 (0'0) promote 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by osd.4.0:21692 2014-08-11 16:45:37.985590 2014-08-13 18:06:06.587142 7f5c973a57a0 20 read_log 192888'17946 (192888'17945) modify 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.1315259.0:11343 2014-08-11 16:45:37.977470 2014-08-13 18:06:06.587164 7f5c973a57a0 20 read_log 192888'17955 (0'0) promote 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by osd.4.0:21718 2014-08-11 16:45:41.594582 2014-08-13 18:06:06.587177 7f5c973a57a0 20 read_log 192888'17956 (192888'17955) modify 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.1315259.0:12232 2014-08-11 16:45:41.586100 2014-08-13 18:06:06.587191 7f5c973a57a0 20 read_log 192888'17957 (0'0) promote a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by osd.4.0:21740 2014-08-11 16:45:45.214273 2014-08-13 18:06:06.587204 7f5c973a57a0 20 read_log 192888'17958 (192888'17957) modify a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.1315259.0:13088 2014-08-11 16:45:45.204842 2014-08-13 18:06:06.587216 7f5c973a57a0 20 read_log 192888'17959 (0'0) promote 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by osd.4.0:21756 2014-08-11 16:45:48.138326 2014-08-13 18:06:06.587229 7f5c973a57a0 20 read_log 192888'17960 (192888'17959) modify 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.1315259.0:13829 2014-08-11 16:45:48.129634 2014-08-13 18:06:06.587243 7f5c973a57a0 20 read_log 192888'17961 (0'0) promote 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by osd.4.0:21758 2014-08-11 16:45:48.239976 2014-08-13 18:06:06.587259 7f5c973a57a0 20 read_log 192888'17962 (192888'17961) modify 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.1315259.0:13857 2014-08-11 16:45:48.233909 2014-08-13 18:06:06.587273 7f5c973a57a0 20 read_log 192888'17971 (0'0) promote 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by osd.4.0:21812 2014-08-11 16:45:56.239117 2014-08-13 18:06:06.587286 7f5c973a57a0 20 read_log 192888'17972 (192888'17971) modify 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.1315259.0:15949 2014-08-11 16:45:56.230164 2014-08-13 18:06:06.587298 7f5c973a57a0 20 read_log 192888'17973 (0'0) promote 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by osd.4.0:21814 2014-08-11 16:45:56.449710 2014-08-13 18:06:06.587311 7f5c973a57a0 20 read_log 192888'17974 (192888'17973) modify 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.1315259.0:16005 2014-08-11 16:45:56.440644 2014-08-13 18:06:06.587329 7f5c973a57a0 20 read_log 192888'17975 (0'0) promote 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by osd.4.0:21822 2014-08-11 16:45:57.098295 2014-08-13 18:06:06.587341 7f5c973a57a0 20 read_log 192888'17976 (192888'17975) modify 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.1315259.0:16176 2014-08-11 16:45:57.090281 2014-08-13 18:06:06.587354 7f5c973a57a0 20 read_log 192888'17977 (0'0) promote e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by osd.4.0:21826 2014-08-11 16:45:57.470932 2014-08-13 18:06:06.587385 7f5c973a57a0 20 read_log 192888'17978 (192888'17977) modify e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.1315259.0:16277 2014-08-11 16:45:57.462358 2014-08-13 18:06:06.587403 7f5c973a57a0 20 read_log 192888'17979 (0'0) promote 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by osd.4.0:21840 2014-08-11 16:45:59.898073 2014-08-13 18:06:06.587419 7f5c973a57a0 20 read_log 192888'17980 (192888'17979) modify 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.1315259.0:16893 2014-08-11 16:45:59.889699 2014-08-13 18:06:06.587441 7f5c973a57a0 20 read_log 192888'17981 (0'0) promote 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by osd.4.0:21850 2014-08-11 16:46:01.029567 2014-08-13 18:06:06.587454 7f5c973a57a0 20 read_log 192888'17982 (192888'17981) modify 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.1315259.0:17175 2014-08-11 16:46:01.020485 2014-08-13 18:06:06.587467 7f5c973a57a0 20 read_log 192888'17983 (0'0) promote ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by osd.4.0:21862 2014-08-11 16:46:02.242915 2014-08-13 18:06:06.587495 7f5c973a57a0 20 read_log 192888'17984 (192888'17983) modify ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.1315259.0:17482 2014-08-11 16:46:02.234908 2014-08-13 18:06:06.587508 7f5c973a57a0 20 read_log 192888'17985 (0'0) promote fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by osd.4.0:21914 2014-08-11 16:46:08.824924 2014-08-13 18:06:06.587526 7f5c973a57a0 20 read_log 192888'17986 (192888'17985) modify fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.1315259.0:19126 2014-08-11 16:46:08.817115 2014-08-13 18:06:06.587541 7f5c973a57a0 20 read_log 192888'17989 (0'0) promote 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by osd.4.0:21930 2014-08-11 16:46:11.785903 2014-08-13 18:06:06.587555 7f5c973a57a0 20 read_log 192888'17990 (192888'17989) modify 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.1315259.0:19907 2014-08-11 16:46:11.774640 2014-08-13 18:06:06.587567 7f5c973a57a0 20 read_log 192888'17991 (0'0) promote ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by osd.4.0:21934 2014-08-11 16:46:12.469506 2014-08-13 18:06:06.587580 7f5c973a57a0 20 read_log 192888'17992 (192888'17991) modify ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.1315259.0:20095 2014-08-11 16:46:12.460641 2014-08-13 18:06:06.587592 7f5c973a57a0 20 read_log 192888'17993 (0'0) promote fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by osd.4.0:21946 2014-08-11 16:46:14.375459 2014-08-13 18:06:06.587616 7f5c973a57a0 20 read_log 192888'17994 (192888'17993) modify fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.1315259.0:20591 2014-08-11 16:46:14.366699 2014-08-13 18:06:06.587631 7f5c973a57a0 20 read_log 192888'17995 (0'0) promote 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by osd.4.0:21960 2014-08-11 16:46:16.060062 2014-08-13 18:06:06.587644 7f5c973a57a0 20 read_log 192888'17996 (192888'17995) modify 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.1315259.0:21050 2014-08-11 16:46:16.051486 2014-08-13 18:06:06.587657 7f5c973a57a0 20 read_log 192888'18005 (0'0) promote 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by osd.4.0:22042 2014-08-11 16:46:28.070533 2014-08-13 18:06:06.587669 7f5c973a57a0 20 read_log 192888'18006 (192888'18005) modify 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.1315259.0:24112 2014-08-11 16:46:28.062675 2014-08-13 18:06:06.587682 7f5c973a57a0 20 read_log 192888'18007 (0'0) promote 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by osd.4.0:22050 2014-08-11 16:46:28.978097 2014-08-13 18:06:06.587701 7f5c973a57a0 20 read_log 192888'18008 (192888'18007) modify 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.1315259.0:24356 2014-08-11 16:46:28.967606 2014-08-13 18:06:06.587716 7f5c973a57a0 20 read_log 192888'18009 (0'0) promote 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by osd.4.0:22060 2014-08-11 16:46:29.926340 2014-08-13 18:06:06.587729 7f5c973a57a0 20 read_log 192888'18010 (192888'18009) modify 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.1315259.0:24601 2014-08-11 16:46:29.914132 2014-08-13 18:06:06.587741 7f5c973a57a0 20 read_log 192888'18011 (0'0) promote a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by osd.4.0:22070 2014-08-11 16:46:30.226812 2014-08-13 18:06:06.587754 7f5c973a57a0 20 read_log 192888'18012 (192888'18011) modify a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.1315259.0:24677 2014-08-11 16:46:30.216105 2014-08-13 18:06:06.587766 7f5c973a57a0 20 read_log 192888'18015 (0'0) promote 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by osd.4.0:22154 2014-08-11 16:46:45.081318 2014-08-13 18:06:06.587778 7f5c973a57a0 20 read_log 192888'18016 (192888'18015) modify 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.1315259.0:28265 2014-08-11 16:46:45.072892 2014-08-13 18:06:06.587791 7f5c973a57a0 20 read_log 192888'18017 (0'0) promote 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by osd.4.0:22162 2014-08-11 16:46:45.827070 2014-08-13 18:06:06.587803 7f5c973a57a0 20 read_log 192888'18018 (192888'18017) modify 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.1315259.0:28440 2014-08-11 16:46:45.817154 2014-08-13 18:06:06.587816 7f5c973a57a0 20 read_log 192888'18021 (0'0) promote 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by osd.4.0:22216 2014-08-11 16:46:53.841043 2014-08-13 18:06:06.587828 7f5c973a57a0 20 read_log 192888'18022 (192888'18021) modify 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.1315259.0:30461 2014-08-11 16:46:53.831802 2014-08-13 18:06:06.587840 7f5c973a57a0 20 read_log 192888'18024 (192888'17906) modify c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.1315259.0:31095 2014-08-11 16:46:55.995049 2014-08-13 18:06:06.587852 7f5c973a57a0 20 read_log 192888'18028 (192888'17914) modify 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.1315259.0:32082 2014-08-11 16:46:56.284807 2014-08-13 18:06:06.587865 7f5c973a57a0 20 read_log 192888'18029 (192888'17916) modify fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.1315259.0:32164 2014-08-11 16:46:56.304728 2014-08-13 18:06:06.587878 7f5c973a57a0 20 read_log 192888'18031 (192888'17920) modify 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.1315259.0:33778 2014-08-11 16:46:56.722757 2014-08-13 18:06:06.587890 7f5c973a57a0 20 read_log 192888'18034 (192888'17926) modify c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.1315259.0:35742 2014-08-11 16:46:57.235011 2014-08-13 18:06:06.587903 7f5c973a57a0 20 read_log 192888'18035 (192888'17928) modify 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.1315259.0:36575 2014-08-11 16:46:57.459284 2014-08-13 18:06:06.587915 7f5c973a57a0 20 read_log 192888'18038 (192888'17934) modify bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.1315259.0:38205 2014-08-11 16:46:57.885154 2014-08-13 18:06:06.587929 7f5c973a57a0 20 read_log 192888'18039 (192888'17936) modify d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.1315259.0:39003 2014-08-11 16:46:58.100923 2014-08-13 18:06:06.587942 7f5c973a57a0 20 read_log 192888'18042 (192888'17942) modify eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.1315259.0:41022 2014-08-11 16:46:58.639786 2014-08-13 18:06:06.587965 7f5c973a57a0 20 read_log 192888'18043 (192888'17944) modify b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.1315259.0:41751 2014-08-11 16:46:58.827161 2014-08-13 18:06:06.587981 7f5c973a57a0 20 read_log 192888'18044 (192888'17946) modify 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.1315259.0:42070 2014-08-11 16:46:58.911227 2014-08-13 18:06:06.587993 7f5c973a57a0 20 read_log 192888'18049 (192888'17956) modify 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.1315259.0:42959 2014-08-11 16:46:59.152633 2014-08-13 18:06:06.588008 7f5c973a57a0 20 read_log 192888'18050 (192888'17958) modify a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.1315259.0:43815 2014-08-11 16:46:59.396377 2014-08-13 18:06:06.588020 7f5c973a57a0 20 read_log 192888'18051 (192888'17960) modify 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.1315259.0:44556 2014-08-11 16:46:59.611618 2014-08-13 18:06:06.588035 7f5c973a57a0 20 read_log 192888'18052 (192888'17962) modify 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.1315259.0:44584 2014-08-11 16:46:59.619801 2014-08-13 18:06:06.588047 7f5c973a57a0 20 read_log 192888'18057 (192888'17972) modify 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.1315259.0:46676 2014-08-11 16:47:00.227419 2014-08-13 18:06:06.588060 7f5c973a57a0 20 read_log 192888'18058 (192888'17974) modify 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.1315259.0:46732 2014-08-11 16:47:00.243215 2014-08-13 18:06:06.588072 7f5c973a57a0 20 read_log 192888'18059 (192888'17976) modify 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.1315259.0:46903 2014-08-11 16:47:00.290863 2014-08-13 18:06:06.588087 7f5c973a57a0 20 read_log 192888'18060 (192888'17978) modify e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.1315259.0:47004 2014-08-11 16:47:00.319579 2014-08-13 18:06:06.588099 7f5c973a57a0 20 read_log 192888'18061 (192888'17980) modify 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.1315259.0:47620 2014-08-11 16:47:00.512319 2014-08-13 18:06:06.588113 7f5c973a57a0 20 read_log 192888'18062 (192888'17982) modify 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.1315259.0:47902 2014-08-11 16:47:00.602253 2014-08-13 18:06:06.588126 7f5c973a57a0 20 read_log 192888'18063 (192888'17984) modify ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.1315259.0:48209 2014-08-11 16:47:00.698020 2014-08-13 18:06:06.588140 7f5c973a57a0 20 read_log 192888'18064 (192888'17986) modify fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.1315259.0:49853 2014-08-11 16:47:01.191968 2014-08-13 18:06:06.588174 7f5c973a57a0 20 read_log 192888'18066 (192888'17990) modify 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.1315259.0:50634 2014-08-11 16:47:01.430670 2014-08-13 18:06:06.588189 7f5c973a57a0 20 read_log 192888'18067 (192888'17992) modify ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.1315259.0:50822 2014-08-11 16:47:01.488361 2014-08-13 18:06:06.588202 7f5c973a57a0 20 read_log 192888'18068 (192888'17994) modify fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.1315259.0:51318 2014-08-11 16:47:01.645821 2014-08-13 18:06:06.588217 7f5c973a57a0 20 read_log 192888'18069 (192888'17996) modify 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.1315259.0:51777 2014-08-11 16:47:01.779723 2014-08-13 18:06:06.588229 7f5c973a57a0 20 read_log 192888'18074 (192888'18006) modify 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.1315259.0:54839 2014-08-11 16:47:02.733103 2014-08-13 18:06:06.588243 7f5c973a57a0 20 read_log 192888'18075 (192888'18008) modify 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.1315259.0:55083 2014-08-11 16:47:02.805974 2014-08-13 18:06:06.588267 7f5c973a57a0 20 read_log 192888'18076 (192888'18010) modify 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.1315259.0:55328 2014-08-11 16:47:02.879450 2014-08-13 18:06:06.588282 7f5c973a57a0 20 read_log 192888'18077 (192888'18012) modify a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.1315259.0:55404 2014-08-11 16:47:02.902119 2014-08-13 18:06:06.588294 7f5c973a57a0 20 read_log 192888'18079 (192888'18016) modify 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.1315259.0:58992 2014-08-11 16:47:03.967250 2014-08-13 18:06:06.588308 7f5c973a57a0 20 read_log 192888'18080 (192888'18018) modify 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.1315259.0:59167 2014-08-11 16:47:04.024201 2014-08-13 18:06:06.588323 7f5c973a57a0 20 read_log 192888'18082 (192888'18022) modify 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.1315259.0:61188 2014-08-11 16:47:04.661871 2014-08-13 18:06:06.588339 7f5c973a57a0 20 read_log 192888'18083 (192888'18083) modify 3d/hit_set_24.3d_archive_2014-08-11 16:44:56.254254_2014-08-11 17:05:17.262043/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.262449 2014-08-13 18:06:06.588352 7f5c973a57a0 20 read_log 192888'18084 (192888'17901) delete 3d/hit_set_24.3d_archive_2014-08-11 16:02:02.114927_2014-08-11 16:44:56.253783/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.262449 2014-08-13 18:06:06.588364 7f5c973a57a0 20 read_log 192888'18085 (0'0) promote 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by osd.4.0:22660 2014-08-11 17:05:17.267300 2014-08-13 18:06:06.588376 7f5c973a57a0 20 read_log 192888'18086 (192888'18085) modify 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.942868.0:546 2014-08-11 17:05:17.256834 2014-08-13 18:06:06.588389 7f5c973a57a0 20 read_log 192888'18091 (0'0) promote 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by osd.4.0:22677 2014-08-11 17:05:19.797351 2014-08-13 18:06:06.588408 7f5c973a57a0 20 read_log 192888'18092 (192888'18091) modify 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.942868.0:1157 2014-08-11 17:05:19.789491 2014-08-13 18:06:06.588423 7f5c973a57a0 20 read_log 192888'18095 (0'0) promote 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by osd.4.0:22683 2014-08-11 17:05:20.534659 2014-08-13 18:06:06.588435 7f5c973a57a0 20 read_log 192888'18096 (192888'18095) modify 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.942868.0:1345 2014-08-11 17:05:20.525402 2014-08-13 18:06:06.588448 7f5c973a57a0 20 read_log 192888'18101 (0'0) promote dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by osd.4.0:22701 2014-08-11 17:05:24.380566 2014-08-13 18:06:06.588460 7f5c973a57a0 20 read_log 192888'18102 (192888'18101) modify dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.942868.0:2281 2014-08-11 17:05:24.371669 2014-08-13 18:06:06.588489 7f5c973a57a0 20 read_log 192888'18103 (0'0) promote b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by osd.4.0:22703 2014-08-11 17:05:24.391494 2014-08-13 18:06:06.588503 7f5c973a57a0 20 read_log 192888'18104 (192888'18103) modify b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.942868.0:2284 2014-08-11 17:05:24.382622 2014-08-13 18:06:06.588522 7f5c973a57a0 20 read_log 192888'18105 (0'0) promote db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by osd.4.0:22720 2014-08-11 17:05:26.623624 2014-08-13 18:06:06.588538 7f5c973a57a0 20 read_log 192888'18106 (192888'18105) modify db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.942868.0:2886 2014-08-11 17:05:26.614491 2014-08-13 18:06:06.588573 7f5c973a57a0 20 read_log 192888'18107 (0'0) promote a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by osd.4.0:22730 2014-08-11 17:05:28.062666 2014-08-13 18:06:06.588590 7f5c973a57a0 20 read_log 192888'18108 (192888'18107) modify a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.942868.0:3276 2014-08-11 17:05:28.054822 2014-08-13 18:06:06.588603 7f5c973a57a0 20 read_log 192888'18109 (0'0) promote 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by osd.4.0:22766 2014-08-11 17:05:31.895219 2014-08-13 18:06:06.588616 7f5c973a57a0 20 read_log 192888'18110 (192888'18109) modify 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.942868.0:4315 2014-08-11 17:05:31.886633 2014-08-13 18:06:06.588629 7f5c973a57a0 20 read_log 192888'18113 (0'0) promote 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by osd.4.0:22778 2014-08-11 17:05:33.383764 2014-08-13 18:06:06.588641 7f5c973a57a0 20 read_log 192888'18114 (192888'18113) modify 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.942868.0:4701 2014-08-11 17:05:33.375129 2014-08-13 18:06:06.588653 7f5c973a57a0 20 read_log 192888'18115 (0'0) promote 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by osd.4.0:22786 2014-08-11 17:05:33.841187 2014-08-13 18:06:06.588665 7f5c973a57a0 20 read_log 192888'18116 (192888'18115) modify 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.942868.0:4823 2014-08-11 17:05:33.833157 2014-08-13 18:06:06.588678 7f5c973a57a0 20 read_log 192888'18119 (0'0) promote 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by osd.4.0:22792 2014-08-11 17:05:34.506451 2014-08-13 18:06:06.588690 7f5c973a57a0 20 read_log 192888'18120 (192888'18119) modify 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.942868.0:4975 2014-08-11 17:05:34.497526 2014-08-13 18:06:06.588702 7f5c973a57a0 20 read_log 192888'18121 (0'0) promote cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by osd.4.0:22828 2014-08-11 17:05:39.850089 2014-08-13 18:06:06.588723 7f5c973a57a0 20 read_log 192888'18122 (192888'18121) modify cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.942868.0:6424 2014-08-11 17:05:39.841406 2014-08-13 18:06:06.588736 7f5c973a57a0 20 read_log 192888'18123 (0'0) promote 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by osd.4.0:22834 2014-08-11 17:05:40.375646 2014-08-13 18:06:06.588748 7f5c973a57a0 20 read_log 192888'18124 (192888'18123) modify 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.942868.0:6570 2014-08-11 17:05:40.365174 2014-08-13 18:06:06.588761 7f5c973a57a0 20 read_log 192888'18125 (0'0) promote e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by osd.4.0:22856 2014-08-11 17:05:43.515350 2014-08-13 18:06:06.588773 7f5c973a57a0 20 read_log 192888'18126 (192888'18125) modify e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.942868.0:7383 2014-08-11 17:05:43.506850 2014-08-13 18:06:06.588785 7f5c973a57a0 20 read_log 192888'18133 (0'0) promote 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by osd.4.0:22978 2014-08-11 17:06:03.404857 2014-08-13 18:06:06.588798 7f5c973a57a0 20 read_log 192888'18134 (192888'18133) modify 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.942868.0:12783 2014-08-11 17:06:03.394312 2014-08-13 18:06:06.588812 7f5c973a57a0 20 read_log 192888'18139 (0'0) promote 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by osd.4.0:22992 2014-08-11 17:06:04.859191 2014-08-13 18:06:06.588824 7f5c973a57a0 20 read_log 192888'18140 (192888'18139) modify 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.942868.0:13187 2014-08-11 17:06:04.851421 2014-08-13 18:06:06.588864 7f5c973a57a0 20 read_log 192888'18143 (0'0) promote 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by osd.4.0:23030 2014-08-11 17:06:11.782648 2014-08-13 18:06:06.588877 7f5c973a57a0 20 read_log 192888'18144 (192888'18143) modify 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.942868.0:14887 2014-08-11 17:06:11.773846 2014-08-13 18:06:06.588890 7f5c973a57a0 20 read_log 192888'18151 (0'0) promote db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by osd.4.0:23090 2014-08-11 17:06:20.725059 2014-08-13 18:06:06.588915 7f5c973a57a0 20 read_log 192888'18152 (192888'18151) modify db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.942868.0:17175 2014-08-11 17:06:20.714108 2014-08-13 18:06:06.588928 7f5c973a57a0 20 read_log 192888'18153 (0'0) promote f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by osd.4.0:23098 2014-08-11 17:06:21.574118 2014-08-13 18:06:06.588942 7f5c973a57a0 20 read_log 192888'18154 (192888'18153) modify f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.942868.0:17400 2014-08-11 17:06:21.566056 2014-08-13 18:06:06.588955 7f5c973a57a0 20 read_log 192888'18161 (0'0) promote 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by osd.4.0:23176 2014-08-11 17:06:31.947696 2014-08-13 18:06:06.588967 7f5c973a57a0 20 read_log 192888'18162 (192888'18161) modify 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.942868.0:20156 2014-08-11 17:06:31.936845 2014-08-13 18:06:06.588982 7f5c973a57a0 20 read_log 192888'18167 (0'0) promote 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by osd.4.0:23252 2014-08-11 17:06:43.173794 2014-08-13 18:06:06.588994 7f5c973a57a0 20 read_log 192888'18168 (192888'18167) modify 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.942868.0:23056 2014-08-11 17:06:43.164158 2014-08-13 18:06:06.589007 7f5c973a57a0 20 read_log 192888'18177 (0'0) promote eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by osd.4.0:23340 2014-08-11 17:06:52.488870 2014-08-13 18:06:06.589019 7f5c973a57a0 20 read_log 192888'18178 (192888'18177) modify eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.942868.0:25549 2014-08-11 17:06:52.480572 2014-08-13 18:06:06.589031 7f5c973a57a0 20 read_log 192888'18179 (0'0) promote 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by osd.4.0:23370 2014-08-11 17:06:58.820554 2014-08-13 18:06:06.589046 7f5c973a57a0 20 read_log 192888'18180 (192888'18179) modify 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.942868.0:27162 2014-08-11 17:06:58.812525 2014-08-13 18:06:06.589060 7f5c973a57a0 20 read_log 192888'18181 (0'0) promote c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by osd.4.0:23388 2014-08-11 17:07:00.934651 2014-08-13 18:06:06.589073 7f5c973a57a0 20 read_log 192888'18182 (192888'18181) modify c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.942868.0:27714 2014-08-11 17:07:00.926017 2014-08-13 18:06:06.589088 7f5c973a57a0 20 read_log 192888'18185 (0'0) promote a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by osd.4.0:23456 2014-08-11 17:07:12.238943 2014-08-13 18:06:06.589101 7f5c973a57a0 20 read_log 192888'18186 (192888'18185) modify a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.942868.0:30602 2014-08-11 17:07:12.228888 2014-08-13 18:06:06.589113 7f5c973a57a0 20 read_log 192888'18189 (192888'18086) modify 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.942868.0:31812 2014-08-11 17:07:16.065747 2014-08-13 18:06:06.589126 7f5c973a57a0 20 read_log 192888'18192 (192888'18092) modify 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.942868.0:32423 2014-08-11 17:07:16.249873 2014-08-13 18:06:06.589167 7f5c973a57a0 20 read_log 192888'18194 (192888'18096) modify 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.942868.0:32611 2014-08-11 17:07:16.300988 2014-08-13 18:06:06.589188 7f5c973a57a0 20 read_log 192888'18197 (192888'18102) modify dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.942868.0:33547 2014-08-11 17:07:16.546175 2014-08-13 18:06:06.589201 7f5c973a57a0 20 read_log 192888'18198 (192888'18104) modify b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.942868.0:33550 2014-08-11 17:07:16.546441 2014-08-13 18:06:06.589224 7f5c973a57a0 20 read_log 192888'18199 (192888'18106) modify db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.942868.0:34152 2014-08-11 17:07:16.712261 2014-08-13 18:06:06.589243 7f5c973a57a0 20 read_log 192888'18200 (192888'18108) modify a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.942868.0:34542 2014-08-11 17:07:16.819882 2014-08-13 18:06:06.589264 7f5c973a57a0 20 read_log 192888'18201 (192888'18110) modify 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.942868.0:35581 2014-08-11 17:07:17.110038 2014-08-13 18:06:06.589279 7f5c973a57a0 20 read_log 192888'18203 (192888'18114) modify 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.942868.0:35967 2014-08-11 17:07:17.209023 2014-08-13 18:06:06.589291 7f5c973a57a0 20 read_log 192888'18204 (192888'18116) modify 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.942868.0:36089 2014-08-11 17:07:17.238518 2014-08-13 18:06:06.589310 7f5c973a57a0 20 read_log 192888'18206 (192888'18120) modify 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.942868.0:36241 2014-08-11 17:07:17.277208 2014-08-13 18:06:06.589327 7f5c973a57a0 20 read_log 192888'18207 (192888'18122) modify cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.942868.0:37690 2014-08-11 17:07:17.680264 2014-08-13 18:06:06.589340 7f5c973a57a0 20 read_log 192888'18208 (192888'18124) modify 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.942868.0:37836 2014-08-11 17:07:17.720109 2014-08-13 18:06:06.589352 7f5c973a57a0 20 read_log 192888'18209 (192888'18126) modify e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.942868.0:38649 2014-08-11 17:07:17.950241 2014-08-13 18:06:06.589364 7f5c973a57a0 20 read_log 192888'18213 (192888'18134) modify 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.942868.0:44049 2014-08-11 17:07:19.451743 2014-08-13 18:06:06.589377 7f5c973a57a0 20 read_log 192888'18216 (192888'18140) modify 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.942868.0:44453 2014-08-11 17:07:19.569906 2014-08-13 18:06:06.589389 7f5c973a57a0 20 read_log 192888'18218 (192888'18144) modify 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.942868.0:46153 2014-08-11 17:07:20.035668 2014-08-13 18:06:06.589401 7f5c973a57a0 20 read_log 192888'18222 (192888'18152) modify db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.942868.0:48441 2014-08-11 17:07:20.719889 2014-08-13 18:06:06.589413 7f5c973a57a0 20 read_log 192888'18223 (192888'18154) modify f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.942868.0:48666 2014-08-11 17:07:20.782177 2014-08-13 18:06:06.589426 7f5c973a57a0 20 read_log 192888'18227 (192888'18162) modify 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.942868.0:51422 2014-08-11 17:07:21.645135 2014-08-13 18:06:06.589438 7f5c973a57a0 20 read_log 192888'18230 (192888'18168) modify 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.942868.0:54322 2014-08-11 17:07:22.525604 2014-08-13 18:06:06.589450 7f5c973a57a0 20 read_log 192888'18235 (192888'18178) modify eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.942868.0:56815 2014-08-11 17:07:23.267964 2014-08-13 18:06:06.589473 7f5c973a57a0 20 read_log 192888'18236 (192888'18180) modify 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.942868.0:58428 2014-08-11 17:07:23.740307 2014-08-13 18:06:06.589488 7f5c973a57a0 20 read_log 192888'18237 (192888'18182) modify c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.942868.0:58980 2014-08-11 17:07:23.908960 2014-08-13 18:06:06.589501 7f5c973a57a0 20 read_log 192888'18239 (192888'18186) modify a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.942868.0:61868 2014-08-11 17:07:24.780159 2014-08-13 18:06:06.589514 7f5c973a57a0 20 read_log 193305'18241 (193305'18241) modify 3d/hit_set_24.3d_archive_2014-08-11 18:37:04.904929_2014-08-11 18:42:35.578890/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:35.579189 2014-08-13 18:06:06.589526 7f5c973a57a0 20 read_log 193305'18242 (192888'18083) delete 3d/hit_set_24.3d_archive_2014-08-11 16:44:56.254254_2014-08-11 17:05:17.262043/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:35.579189 2014-08-13 18:06:06.589538 7f5c973a57a0 20 read_log 193305'18243 (0'0) promote 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by osd.4.0:2 2014-08-11 18:42:35.627695 2014-08-13 18:06:06.589550 7f5c973a57a0 20 read_log 193305'18244 (193305'18243) modify 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.951337.0:616 2014-08-11 18:42:35.571701 2014-08-13 18:06:06.589563 7f5c973a57a0 20 read_log 193305'18245 (0'0) promote 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by osd.4.0:4 2014-08-11 18:42:36.680419 2014-08-13 18:06:06.589599 7f5c973a57a0 20 read_log 193305'18246 (193305'18245) modify 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.951337.0:919 2014-08-11 18:42:36.673618 2014-08-13 18:06:06.589614 7f5c973a57a0 20 read_log 193305'18247 (0'0) promote 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by osd.4.0:9 2014-08-11 18:42:37.857728 2014-08-13 18:06:06.589627 7f5c973a57a0 20 read_log 193305'18248 (193305'18247) modify 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.951337.0:1245 2014-08-11 18:42:37.851448 2014-08-13 18:06:06.589639 7f5c973a57a0 20 read_log 193305'18249 (0'0) promote ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by osd.4.0:23 2014-08-11 18:42:40.452927 2014-08-13 18:06:06.589651 7f5c973a57a0 20 read_log 193305'18250 (193305'18249) modify ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.951337.0:1914 2014-08-11 18:42:40.445483 2014-08-13 18:06:06.589669 7f5c973a57a0 20 read_log 193305'18251 (0'0) promote 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by osd.4.0:51 2014-08-11 18:42:47.311767 2014-08-13 18:06:06.589681 7f5c973a57a0 20 read_log 193305'18252 (193305'18251) modify 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.951337.0:3805 2014-08-11 18:42:47.305036 2014-08-13 18:06:06.589695 7f5c973a57a0 20 read_log 193305'18253 (0'0) promote 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by osd.4.0:89 2014-08-11 18:42:55.181792 2014-08-13 18:06:06.589708 7f5c973a57a0 20 read_log 193305'18254 (193305'18253) modify 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.951337.0:5933 2014-08-11 18:42:55.173713 2014-08-13 18:06:06.589722 7f5c973a57a0 20 read_log 193305'18255 (0'0) promote bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by osd.4.0:93 2014-08-11 18:42:55.537740 2014-08-13 18:06:06.589736 7f5c973a57a0 20 read_log 193305'18256 (193305'18255) modify bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.951337.0:6029 2014-08-11 18:42:55.530490 2014-08-13 18:06:06.589756 7f5c973a57a0 20 read_log 193305'18257 (0'0) promote 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by osd.4.0:101 2014-08-11 18:42:56.741879 2014-08-13 18:06:06.589770 7f5c973a57a0 20 read_log 193305'18258 (193305'18257) modify 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.951337.0:6362 2014-08-11 18:42:56.733331 2014-08-13 18:06:06.589783 7f5c973a57a0 20 read_log 193305'18259 (0'0) promote 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by osd.4.0:107 2014-08-11 18:42:57.530495 2014-08-13 18:06:06.589795 7f5c973a57a0 20 read_log 193305'18260 (193305'18259) modify 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.951337.0:6580 2014-08-11 18:42:57.523865 2014-08-13 18:06:06.589809 7f5c973a57a0 20 read_log 193305'18261 (0'0) promote a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by osd.4.0:139 2014-08-11 18:43:04.370665 2014-08-13 18:06:06.589822 7f5c973a57a0 20 read_log 193305'18262 (193305'18261) modify a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.951337.0:8415 2014-08-11 18:43:04.360431 2014-08-13 18:06:06.589834 7f5c973a57a0 20 read_log 193305'18263 (0'0) promote 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by osd.4.0:171 2014-08-11 18:43:11.823420 2014-08-13 18:06:06.589846 7f5c973a57a0 20 read_log 193305'18264 (193305'18263) modify 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.951337.0:10413 2014-08-11 18:43:11.815148 2014-08-13 18:06:06.589858 7f5c973a57a0 20 read_log 193305'18265 (0'0) promote 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by osd.4.0:175 2014-08-11 18:43:13.758896 2014-08-13 18:06:06.589871 7f5c973a57a0 20 read_log 193305'18266 (193305'18265) modify 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.951337.0:10933 2014-08-11 18:43:13.748827 2014-08-13 18:06:06.589884 7f5c973a57a0 20 read_log 193305'18267 (0'0) promote 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by osd.4.0:181 2014-08-11 18:43:14.378604 2014-08-13 18:06:06.589899 7f5c973a57a0 20 read_log 193305'18268 (193305'18267) modify 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.951337.0:11099 2014-08-11 18:43:14.370536 2014-08-13 18:06:06.589911 7f5c973a57a0 20 read_log 193305'18269 (0'0) promote cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by osd.4.0:185 2014-08-11 18:43:14.988261 2014-08-13 18:06:06.589926 7f5c973a57a0 20 read_log 193305'18270 (193305'18269) modify cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.951337.0:11241 2014-08-11 18:43:14.980659 2014-08-13 18:06:06.589938 7f5c973a57a0 20 read_log 193305'18271 (0'0) promote ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by osd.4.0:187 2014-08-11 18:43:15.184336 2014-08-13 18:06:06.589952 7f5c973a57a0 20 read_log 193305'18272 (193305'18271) modify ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.951337.0:11277 2014-08-11 18:43:15.174869 2014-08-13 18:06:06.589967 7f5c973a57a0 20 read_log 193305'18273 (0'0) promote 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by osd.4.0:193 2014-08-11 18:43:16.492613 2014-08-13 18:06:06.589979 7f5c973a57a0 20 read_log 193305'18274 (193305'18273) modify 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.951337.0:11618 2014-08-11 18:43:16.484438 2014-08-13 18:06:06.589991 7f5c973a57a0 20 read_log 193305'18275 (0'0) promote 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by osd.4.0:215 2014-08-11 18:43:22.081417 2014-08-13 18:06:06.590004 7f5c973a57a0 20 read_log 193305'18276 (193305'18275) modify 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.951337.0:13123 2014-08-11 18:43:22.073614 2014-08-13 18:06:06.590025 7f5c973a57a0 20 read_log 193305'18277 (0'0) promote c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by osd.4.0:245 2014-08-11 18:43:29.972503 2014-08-13 18:06:06.590040 7f5c973a57a0 20 read_log 193305'18278 (193305'18277) modify c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.951337.0:15300 2014-08-11 18:43:29.964191 2014-08-13 18:06:06.590052 7f5c973a57a0 20 read_log 193305'18279 (0'0) promote 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by osd.4.0:249 2014-08-11 18:43:30.854709 2014-08-13 18:06:06.590064 7f5c973a57a0 20 read_log 193305'18280 (193305'18279) modify 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.951337.0:15561 2014-08-11 18:43:30.846758 2014-08-13 18:06:06.590079 7f5c973a57a0 20 read_log 193305'18281 (0'0) promote b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by osd.4.0:253 2014-08-11 18:43:32.067918 2014-08-13 18:06:06.590091 7f5c973a57a0 20 read_log 193305'18282 (193305'18281) modify b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.951337.0:15913 2014-08-11 18:43:32.058697 2014-08-13 18:06:06.590104 7f5c973a57a0 20 read_log 193305'18283 (0'0) promote 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by osd.4.0:265 2014-08-11 18:43:33.609276 2014-08-13 18:06:06.590116 7f5c973a57a0 20 read_log 193305'18284 (193305'18283) modify 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.951337.0:16329 2014-08-11 18:43:33.600704 2014-08-13 18:06:06.590128 7f5c973a57a0 20 read_log 193305'18285 (0'0) promote c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by osd.4.0:283 2014-08-11 18:43:36.194492 2014-08-13 18:06:06.590142 7f5c973a57a0 20 read_log 193305'18286 (193305'18285) modify c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.951337.0:17066 2014-08-11 18:43:36.186927 2014-08-13 18:06:06.590163 7f5c973a57a0 20 read_log 193305'18287 (0'0) promote a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by osd.4.0:341 2014-08-11 18:43:48.115912 2014-08-13 18:06:06.590175 7f5c973a57a0 20 read_log 193305'18288 (193305'18287) modify a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.951337.0:20405 2014-08-11 18:43:48.108000 2014-08-13 18:06:06.590188 7f5c973a57a0 20 read_log 193305'18289 (0'0) promote 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by osd.4.0:365 2014-08-11 18:43:53.694608 2014-08-13 18:06:06.590202 7f5c973a57a0 20 read_log 193305'18290 (193305'18289) modify 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.951337.0:21967 2014-08-11 18:43:53.685338 2014-08-13 18:06:06.590215 7f5c973a57a0 20 read_log 193305'18291 (0'0) promote cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by osd.4.0:371 2014-08-11 18:43:55.600623 2014-08-13 18:06:06.590229 7f5c973a57a0 20 read_log 193305'18292 (193305'18291) modify cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.951337.0:22482 2014-08-11 18:43:55.592374 2014-08-13 18:06:06.590241 7f5c973a57a0 20 read_log 193305'18293 (0'0) promote 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by osd.4.0:373 2014-08-11 18:43:55.641580 2014-08-13 18:06:06.590256 7f5c973a57a0 20 read_log 193305'18294 (193305'18293) modify 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.951337.0:22493 2014-08-11 18:43:55.630152 2014-08-13 18:06:06.590271 7f5c973a57a0 20 read_log 193305'18295 (0'0) promote 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by osd.4.0:395 2014-08-11 18:43:59.939904 2014-08-13 18:06:06.590284 7f5c973a57a0 20 read_log 193305'18296 (193305'18295) modify 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.951337.0:23700 2014-08-11 18:43:59.929720 2014-08-13 18:06:06.590309 7f5c973a57a0 20 read_log 193305'18297 (0'0) promote fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by osd.4.0:413 2014-08-11 18:44:05.147126 2014-08-13 18:06:06.590324 7f5c973a57a0 20 read_log 193305'18298 (193305'18297) modify fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.951337.0:25136 2014-08-11 18:44:05.139815 2014-08-13 18:06:06.590337 7f5c973a57a0 20 read_log 193305'18299 (0'0) promote e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by osd.4.0:415 2014-08-11 18:44:05.485896 2014-08-13 18:06:06.590355 7f5c973a57a0 20 read_log 193305'18300 (193305'18299) modify e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.951337.0:25232 2014-08-11 18:44:05.476915 2014-08-13 18:06:06.590369 7f5c973a57a0 20 read_log 193305'18301 (0'0) promote f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by osd.4.0:421 2014-08-11 18:44:07.449502 2014-08-13 18:06:06.590382 7f5c973a57a0 20 read_log 193305'18302 (193305'18301) modify f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.951337.0:25757 2014-08-11 18:44:07.439797 2014-08-13 18:06:06.590404 7f5c973a57a0 20 read_log 193305'18303 (0'0) promote 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by osd.4.0:423 2014-08-11 18:44:07.513771 2014-08-13 18:06:06.590417 7f5c973a57a0 20 read_log 193305'18304 (193305'18303) modify 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.951337.0:25773 2014-08-11 18:44:07.506058 2014-08-13 18:06:06.590430 7f5c973a57a0 20 read_log 193305'18305 (0'0) promote b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by osd.4.0:459 2014-08-11 18:44:15.711762 2014-08-13 18:06:06.590450 7f5c973a57a0 20 read_log 193305'18306 (193305'18305) modify b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.951337.0:28017 2014-08-11 18:44:15.704724 2014-08-13 18:06:06.590466 7f5c973a57a0 20 read_log 193305'18307 (0'0) promote 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by osd.4.0:465 2014-08-11 18:44:16.389850 2014-08-13 18:06:06.590478 7f5c973a57a0 20 read_log 193305'18308 (193305'18307) modify 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.951337.0:28208 2014-08-11 18:44:16.381563 2014-08-13 18:06:06.590498 7f5c973a57a0 20 read_log 193305'18309 (0'0) promote 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by osd.4.0:471 2014-08-11 18:44:18.142772 2014-08-13 18:06:06.590513 7f5c973a57a0 20 read_log 193305'18310 (193305'18309) modify 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.951337.0:28686 2014-08-11 18:44:18.135344 2014-08-13 18:06:06.590526 7f5c973a57a0 20 read_log 193305'18311 (0'0) promote 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by osd.4.0:485 2014-08-11 18:44:20.648948 2014-08-13 18:06:06.590545 7f5c973a57a0 20 read_log 193305'18312 (193305'18311) modify 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.951337.0:29359 2014-08-11 18:44:20.641816 2014-08-13 18:06:06.590561 7f5c973a57a0 20 read_log 193305'18313 (0'0) promote 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by osd.4.0:523 2014-08-11 18:44:29.794023 2014-08-13 18:06:06.590574 7f5c973a57a0 20 read_log 193305'18314 (193305'18313) modify 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.951337.0:31859 2014-08-11 18:44:29.786257 2014-08-13 18:06:06.590598 7f5c973a57a0 20 read_log 193305'18315 (0'0) promote db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by osd.4.0:531 2014-08-11 18:44:30.579872 2014-08-13 18:06:06.590614 7f5c973a57a0 20 read_log 193305'18316 (193305'18315) modify db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.951337.0:32091 2014-08-11 18:44:30.571599 2014-08-13 18:06:06.590634 7f5c973a57a0 20 read_log 193305'18317 (193305'18244) modify 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.951337.0:33327 2014-08-11 18:44:34.339876 2014-08-13 18:06:06.590649 7f5c973a57a0 20 read_log 193305'18318 (193305'18246) modify 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.951337.0:33630 2014-08-11 18:44:34.544380 2014-08-13 18:06:06.590661 7f5c973a57a0 20 read_log 193305'18319 (193305'18248) modify 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.951337.0:33956 2014-08-11 18:44:34.734283 2014-08-13 18:06:06.590674 7f5c973a57a0 20 read_log 193305'18320 (193305'18250) modify ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.951337.0:34625 2014-08-11 18:44:35.101470 2014-08-13 18:06:06.590686 7f5c973a57a0 20 read_log 193305'18321 (193305'18252) modify 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.951337.0:36516 2014-08-11 18:44:36.285520 2014-08-13 18:06:06.590698 7f5c973a57a0 20 read_log 193305'18322 (193305'18254) modify 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.951337.0:38644 2014-08-11 18:44:37.574840 2014-08-13 18:06:06.590710 7f5c973a57a0 20 read_log 193305'18323 (193305'18256) modify bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.951337.0:38740 2014-08-11 18:44:37.629460 2014-08-13 18:06:06.590725 7f5c973a57a0 20 read_log 193305'18324 (193305'18258) modify 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.951337.0:39073 2014-08-11 18:44:37.810862 2014-08-13 18:06:06.590738 7f5c973a57a0 20 read_log 193305'18325 (193305'18260) modify 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.951337.0:39291 2014-08-11 18:44:37.926628 2014-08-13 18:06:06.590750 7f5c973a57a0 20 read_log 193305'18326 (193305'18262) modify a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.951337.0:41126 2014-08-11 18:44:38.642102 2014-08-13 18:06:06.590762 7f5c973a57a0 20 read_log 193305'18327 (193305'18264) modify 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.951337.0:43124 2014-08-11 18:44:39.491754 2014-08-13 18:06:06.590775 7f5c973a57a0 20 read_log 193305'18328 (193305'18266) modify 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.951337.0:43644 2014-08-11 18:44:39.653371 2014-08-13 18:06:06.590787 7f5c973a57a0 20 read_log 193305'18329 (193305'18268) modify 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.951337.0:43810 2014-08-11 18:44:39.707556 2014-08-13 18:06:06.590799 7f5c973a57a0 20 read_log 193305'18330 (193305'18270) modify cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.951337.0:43952 2014-08-11 18:44:39.749228 2014-08-13 18:06:06.590811 7f5c973a57a0 20 read_log 193305'18331 (193305'18272) modify ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.951337.0:43988 2014-08-11 18:44:39.760620 2014-08-13 18:06:06.590823 7f5c973a57a0 20 read_log 193305'18332 (193305'18274) modify 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.951337.0:44329 2014-08-11 18:44:39.853623 2014-08-13 18:06:06.590836 7f5c973a57a0 20 read_log 193305'18333 (193305'18276) modify 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.951337.0:45834 2014-08-11 18:44:40.315073 2014-08-13 18:06:06.590848 7f5c973a57a0 20 read_log 193305'18334 (193305'18278) modify c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.951337.0:48011 2014-08-11 18:44:40.930420 2014-08-13 18:06:06.590860 7f5c973a57a0 20 read_log 193305'18335 (193305'18280) modify 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.951337.0:48272 2014-08-11 18:44:41.004591 2014-08-13 18:06:06.590872 7f5c973a57a0 20 read_log 193305'18336 (193305'18282) modify b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.951337.0:48624 2014-08-11 18:44:41.106124 2014-08-13 18:06:06.590907 7f5c973a57a0 20 read_log 193305'18337 (193305'18284) modify 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.951337.0:49040 2014-08-11 18:44:41.235068 2014-08-13 18:06:06.590923 7f5c973a57a0 20 read_log 193305'18338 (193305'18286) modify c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.951337.0:49777 2014-08-11 18:44:41.468519 2014-08-13 18:06:06.590952 7f5c973a57a0 20 read_log 193305'18339 (193305'18288) modify a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.951337.0:53116 2014-08-11 18:44:42.432183 2014-08-13 18:06:06.590967 7f5c973a57a0 20 read_log 193305'18340 (193305'18290) modify 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.951337.0:54678 2014-08-11 18:44:42.909512 2014-08-13 18:06:06.590979 7f5c973a57a0 20 read_log 193305'18341 (193305'18292) modify cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.951337.0:55193 2014-08-11 18:44:43.053403 2014-08-13 18:06:06.590992 7f5c973a57a0 20 read_log 193305'18342 (193305'18294) modify 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.951337.0:55204 2014-08-11 18:44:43.056249 2014-08-13 18:06:06.591004 7f5c973a57a0 20 read_log 193305'18343 (193305'18296) modify 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.951337.0:56411 2014-08-11 18:44:43.411100 2014-08-13 18:06:06.591016 7f5c973a57a0 20 read_log 193305'18344 (193305'18298) modify fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.951337.0:57847 2014-08-11 18:44:43.874368 2014-08-13 18:06:06.591029 7f5c973a57a0 20 read_log 193305'18345 (193305'18300) modify e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.951337.0:57943 2014-08-11 18:44:43.900621 2014-08-13 18:06:06.591041 7f5c973a57a0 20 read_log 193305'18346 (193305'18302) modify f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.951337.0:58468 2014-08-11 18:44:44.050429 2014-08-13 18:06:06.591053 7f5c973a57a0 20 read_log 193305'18347 (193305'18304) modify 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.951337.0:58484 2014-08-11 18:44:44.056072 2014-08-13 18:06:06.591065 7f5c973a57a0 20 read_log 193305'18348 (193305'18306) modify b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.951337.0:60728 2014-08-11 18:44:44.712485 2014-08-13 18:06:06.591078 7f5c973a57a0 20 read_log 193305'18349 (193305'18308) modify 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.951337.0:60919 2014-08-11 18:44:44.772383 2014-08-13 18:06:06.591090 7f5c973a57a0 20 read_log 193305'18350 (193305'18310) modify 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.951337.0:61397 2014-08-11 18:44:44.910219 2014-08-13 18:06:06.591102 7f5c973a57a0 20 read_log 193305'18351 (193305'18312) modify 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.951337.0:62070 2014-08-11 18:44:45.118106 2014-08-13 18:06:06.591114 7f5c973a57a0 20 read_log 193305'18352 (193305'18314) modify 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.951337.0:64570 2014-08-11 18:44:45.855216 2014-08-13 18:06:06.591126 7f5c973a57a0 20 read_log 193305'18353 (193305'18316) modify db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.951337.0:64802 2014-08-11 18:44:45.921775 2014-08-13 18:06:06.591139 7f5c973a57a0 20 read_log 194215'18354 (192888'18223) clean f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by osd.4.0:961 2014-08-12 15:37:01.766778 2014-08-13 18:06:06.591160 7f5c973a57a0 20 read_log 194215'18355 (194215'18354) delete f408003d/benchmark_data_storage0111-ib.pocstor_2228_object17399/head//24 by client.2174595.0:12038 0.000000 2014-08-13 18:06:06.591184 7f5c973a57a0 20 read_log 194215'18356 (192888'18051) clean 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by osd.4.0:963 2014-08-12 15:37:01.779391 2014-08-13 18:06:06.591199 7f5c973a57a0 20 read_log 194215'18357 (194215'18356) delete 6d81103d/benchmark_data_storage0111-ib.pocstor_41097_object13828/head//24 by client.2174595.0:12040 0.000000 2014-08-13 18:06:06.591211 7f5c973a57a0 20 read_log 194215'18358 (192834'17818) clean 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by osd.4.0:965 2014-08-12 15:37:01.811920 2014-08-13 18:06:06.591223 7f5c973a57a0 20 read_log 194215'18359 (194215'18358) delete 5467203d/benchmark_data_storage0111-ib.pocstor_41526_object12864/head//24 by client.2174595.0:12042 0.000000 2014-08-13 18:06:06.591236 7f5c973a57a0 20 read_log 194215'18360 (192888'18039) clean d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by osd.4.0:967 2014-08-12 15:37:01.828071 2014-08-13 18:06:06.591248 7f5c973a57a0 20 read_log 194215'18361 (194215'18360) delete d7be203d/benchmark_data_storage0111-ib.pocstor_41097_object8275/head//24 by client.2174595.0:12044 0.000000 2014-08-13 18:06:06.591262 7f5c973a57a0 20 read_log 194215'18362 (192888'18057) clean 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by osd.4.0:969 2014-08-12 15:37:01.835683 2014-08-13 18:06:06.591275 7f5c973a57a0 20 read_log 194215'18363 (194215'18362) delete 2ca5603d/benchmark_data_storage0111-ib.pocstor_41097_object15948/head//24 by client.2174595.0:12046 0.000000 2014-08-13 18:06:06.591286 7f5c973a57a0 20 read_log 194215'18364 (192888'18236) clean 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by osd.4.0:971 2014-08-12 15:37:01.843534 2014-08-13 18:06:06.591298 7f5c973a57a0 20 read_log 194215'18365 (194215'18364) delete 2a46603d/benchmark_data_storage0111-ib.pocstor_2228_object27161/head//24 by client.2174595.0:12048 0.000000 2014-08-13 18:06:06.591309 7f5c973a57a0 20 read_log 194215'18366 (193305'18323) clean bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by osd.4.0:973 2014-08-12 15:37:01.851894 2014-08-13 18:06:06.591321 7f5c973a57a0 20 read_log 194215'18367 (194215'18366) delete bcc603d/benchmark_data_storage0111-ib.pocstor_62621_object6028/head//24 by client.2174595.0:12050 0.000000 2014-08-13 18:06:06.591331 7f5c973a57a0 20 read_log 194215'18368 (193305'18339) clean a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by osd.4.0:975 2014-08-12 15:37:01.870577 2014-08-13 18:06:06.591344 7f5c973a57a0 20 read_log 194215'18369 (194215'18368) delete a058703d/benchmark_data_storage0111-ib.pocstor_62621_object20404/head//24 by client.2174595.0:12052 0.000000 2014-08-13 18:06:06.591354 7f5c973a57a0 20 read_log 194215'18370 (193305'18318) clean 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by osd.4.0:977 2014-08-12 15:37:01.878712 2014-08-13 18:06:06.591367 7f5c973a57a0 20 read_log 194215'18371 (194215'18370) delete 1268703d/benchmark_data_storage0111-ib.pocstor_62621_object918/head//24 by client.2174595.0:12054 0.000000 2014-08-13 18:06:06.591387 7f5c973a57a0 20 read_log 194215'18372 (192888'18080) clean 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by osd.4.0:979 2014-08-12 15:37:01.885807 2014-08-13 18:06:06.591404 7f5c973a57a0 20 read_log 194215'18373 (194215'18372) delete 2e73803d/benchmark_data_storage0111-ib.pocstor_41097_object28439/head//24 by client.2174595.0:12056 0.000000 2014-08-13 18:06:06.591415 7f5c973a57a0 20 read_log 194215'18374 (192888'18050) clean a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by osd.4.0:981 2014-08-12 15:37:01.894856 2014-08-13 18:06:06.591434 7f5c973a57a0 20 read_log 194215'18375 (194215'18374) delete a4e3803d/benchmark_data_storage0111-ib.pocstor_41097_object13087/head//24 by client.2174595.0:12058 0.000000 2014-08-13 18:06:06.591463 7f5c973a57a0 20 read_log 194215'18376 (192888'18076) clean 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by osd.4.0:983 2014-08-12 15:37:01.902929 2014-08-13 18:06:06.591487 7f5c973a57a0 20 read_log 194215'18377 (194215'18376) delete 1e6b803d/benchmark_data_storage0111-ib.pocstor_41097_object24600/head//24 by client.2174595.0:12060 0.000000 2014-08-13 18:06:06.591501 7f5c973a57a0 20 read_log 194215'18378 (193305'18335) clean 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by osd.4.0:985 2014-08-12 15:37:01.920171 2014-08-13 18:06:06.591514 7f5c973a57a0 20 read_log 194215'18379 (194215'18378) delete 76c4903d/benchmark_data_storage0111-ib.pocstor_62621_object15560/head//24 by client.2174595.0:12062 0.000000 2014-08-13 18:06:06.591538 7f5c973a57a0 20 read_log 194215'18380 (192888'18239) clean a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by osd.4.0:987 2014-08-12 15:37:01.930901 2014-08-13 18:06:06.591551 7f5c973a57a0 20 read_log 194215'18381 (194215'18380) delete a4cd903d/benchmark_data_storage0111-ib.pocstor_2228_object30601/head//24 by client.2174595.0:12064 0.000000 2014-08-13 18:06:06.591562 7f5c973a57a0 20 read_log 194215'18382 (192834'17727) clean 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by osd.4.0:989 2014-08-12 15:37:01.942754 2014-08-13 18:06:06.591583 7f5c973a57a0 20 read_log 194215'18383 (194215'18382) delete 18f903d/benchmark_data_storage0111-ib.pocstor_41526_object4967/head//24 by client.2174595.0:12066 0.000000 2014-08-13 18:06:06.591595 7f5c973a57a0 20 read_log 194215'18384 (193305'18350) clean 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by osd.4.0:991 2014-08-12 15:37:01.951216 2014-08-13 18:06:06.591611 7f5c973a57a0 20 read_log 194215'18385 (194215'18384) delete 118f903d/benchmark_data_storage0111-ib.pocstor_62621_object28685/head//24 by client.2174595.0:12068 0.000000 2014-08-13 18:06:06.591622 7f5c973a57a0 20 read_log 194215'18386 (193305'18331) clean ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by osd.4.0:993 2014-08-12 15:37:01.958812 2014-08-13 18:06:06.591646 7f5c973a57a0 20 read_log 194215'18387 (194215'18386) delete ad58b03d/benchmark_data_storage0111-ib.pocstor_62621_object11276/head//24 by client.2174595.0:12070 0.000000 2014-08-13 18:06:06.591658 7f5c973a57a0 20 read_log 194215'18388 (192888'18069) clean 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by osd.4.0:995 2014-08-12 15:37:01.966694 2014-08-13 18:06:06.591671 7f5c973a57a0 20 read_log 194215'18389 (194215'18388) delete 1b19b03d/benchmark_data_storage0111-ib.pocstor_41097_object21049/head//24 by client.2174595.0:12072 0.000000 2014-08-13 18:06:06.591688 7f5c973a57a0 20 read_log 194215'18390 (192888'18038) clean bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by osd.4.0:997 2014-08-12 15:37:01.976625 2014-08-13 18:06:06.591705 7f5c973a57a0 20 read_log 194215'18391 (194215'18390) delete bb8cb03d/benchmark_data_storage0111-ib.pocstor_41097_object7477/head//24 by client.2174595.0:12074 0.000000 2014-08-13 18:06:06.591717 7f5c973a57a0 20 read_log 194215'18392 (192888'18197) clean dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by osd.4.0:999 2014-08-12 15:37:01.984295 2014-08-13 18:06:06.591730 7f5c973a57a0 20 read_log 194215'18393 (194215'18392) delete dd27d03d/benchmark_data_storage0111-ib.pocstor_2228_object2280/head//24 by client.2174595.0:12076 0.000000 2014-08-13 18:06:06.591741 7f5c973a57a0 20 read_log 194215'18394 (192834'17784) clean 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by osd.4.0:1001 2014-08-12 15:37:01.996237 2014-08-13 18:06:06.591768 7f5c973a57a0 20 read_log 194215'18395 (194215'18394) delete 4cbcd03d/benchmark_data_storage0111-ib.pocstor_41526_object9852/head//24 by client.2174595.0:12078 0.000000 2014-08-13 18:06:06.591787 7f5c973a57a0 20 read_log 194215'18396 (192888'18200) clean a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by osd.4.0:1003 2014-08-12 15:37:02.019201 2014-08-13 18:06:06.591806 7f5c973a57a0 20 read_log 194215'18397 (194215'18396) delete a531e03d/benchmark_data_storage0111-ib.pocstor_2228_object3275/head//24 by client.2174595.0:12080 0.000000 2014-08-13 18:06:06.591817 7f5c973a57a0 20 read_log 194215'18398 (192888'18222) clean db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by osd.4.0:1005 2014-08-12 15:37:02.027941 2014-08-13 18:06:06.591830 7f5c973a57a0 20 read_log 194215'18399 (194215'18398) delete db4be03d/benchmark_data_storage0111-ib.pocstor_2228_object17174/head//24 by client.2174595.0:12082 0.000000 2014-08-13 18:06:06.591841 7f5c973a57a0 20 read_log 194215'18400 (193305'18347) clean 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by osd.4.0:1007 2014-08-12 15:37:02.037241 2014-08-13 18:06:06.591866 7f5c973a57a0 20 read_log 194215'18401 (194215'18400) delete 4550f03d/benchmark_data_storage0111-ib.pocstor_62621_object25772/head//24 by client.2174595.0:12084 0.000000 2014-08-13 18:06:06.591878 7f5c973a57a0 20 read_log 194215'18402 (192834'17761) clean f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by osd.4.0:1009 2014-08-12 15:37:02.048324 2014-08-13 18:06:06.591893 7f5c973a57a0 20 read_log 194215'18403 (194215'18402) delete f249f03d/benchmark_data_storage0111-ib.pocstor_41526_object8150/head//24 by client.2174595.0:12086 0.000000 2014-08-13 18:06:06.591905 7f5c973a57a0 20 read_log 194215'18404 (192888'18063) clean ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by osd.4.0:1011 2014-08-12 15:37:02.057052 2014-08-13 18:06:06.591917 7f5c973a57a0 20 read_log 194215'18405 (194215'18404) delete ea1a043d/benchmark_data_storage0111-ib.pocstor_41097_object17481/head//24 by client.2174595.0:12088 0.000000 2014-08-13 18:06:06.591928 7f5c973a57a0 20 read_log 194215'18406 (192888'18068) clean fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by osd.4.0:1013 2014-08-12 15:37:02.065804 2014-08-13 18:06:06.591946 7f5c973a57a0 20 read_log 194215'18407 (194215'18406) delete fc85243d/benchmark_data_storage0111-ib.pocstor_41097_object20590/head//24 by client.2174595.0:12090 0.000000 2014-08-13 18:06:06.591957 7f5c973a57a0 20 read_log 194215'18408 (192834'17143) clean 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by osd.4.0:1015 2014-08-12 15:37:02.079168 2014-08-13 18:06:06.591979 7f5c973a57a0 20 read_log 194215'18409 (194215'18408) delete 1752443d/benchmark_data_storage0111-ib.pocstor_26659_object11547/head//24 by client.2174595.0:12092 0.000000 2014-08-13 18:06:06.591993 7f5c973a57a0 20 read_log 194215'18410 (192834'17886) clean f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by osd.4.0:1017 2014-08-12 15:37:02.088136 2014-08-13 18:06:06.592006 7f5c973a57a0 20 read_log 194215'18411 (194215'18410) delete f23443d/benchmark_data_storage0111-ib.pocstor_41526_object20448/head//24 by client.2174595.0:12094 0.000000 2014-08-13 18:06:06.592017 7f5c973a57a0 20 read_log 194215'18412 (192888'18199) clean db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by osd.4.0:1019 2014-08-12 15:37:02.096196 2014-08-13 18:06:06.592030 7f5c973a57a0 20 read_log 194215'18413 (194215'18412) delete db33443d/benchmark_data_storage0111-ib.pocstor_2228_object2885/head//24 by client.2174595.0:12096 0.000000 2014-08-13 18:06:06.592044 7f5c973a57a0 20 read_log 194215'18414 (192834'17660) clean 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by osd.4.0:1021 2014-08-12 15:37:02.103915 2014-08-13 18:06:06.592056 7f5c973a57a0 20 read_log 194215'18415 (194215'18414) delete 93b4443d/benchmark_data_storage0111-ib.pocstor_41526_object96/head//24 by client.2174595.0:12098 0.000000 2014-08-13 18:06:06.592085 7f5c973a57a0 20 read_log 194215'18416 (193305'18342) clean 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by osd.4.0:1023 2014-08-12 15:37:02.112581 2014-08-13 18:06:06.592105 7f5c973a57a0 20 read_log 194215'18417 (194215'18416) delete 66bb443d/benchmark_data_storage0111-ib.pocstor_62621_object22492/head//24 by client.2174595.0:12100 0.000000 2014-08-13 18:06:06.592116 7f5c973a57a0 20 read_log 194215'18418 (192888'18029) clean fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by osd.4.0:1025 2014-08-12 15:37:02.119912 2014-08-13 18:06:06.592128 7f5c973a57a0 20 read_log 194215'18419 (194215'18418) delete fa81543d/benchmark_data_storage0111-ib.pocstor_41097_object1436/head//24 by client.2174595.0:12102 0.000000 2014-08-13 18:06:06.592142 7f5c973a57a0 20 read_log 194215'18420 (192834'17741) clean 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by osd.4.0:1027 2014-08-12 15:37:02.127355 2014-08-13 18:06:06.592160 7f5c973a57a0 20 read_log 194215'18421 (194215'18420) delete 6715543d/benchmark_data_storage0111-ib.pocstor_41526_object6879/head//24 by client.2174595.0:12104 0.000000 2014-08-13 18:06:06.592181 7f5c973a57a0 20 read_log 194215'18422 (192888'18062) clean 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by osd.4.0:1029 2014-08-12 15:37:02.145610 2014-08-13 18:06:06.592194 7f5c973a57a0 20 read_log 194215'18423 (194215'18422) delete 3597543d/benchmark_data_storage0111-ib.pocstor_41097_object17174/head//24 by client.2174595.0:12106 0.000000 2014-08-13 18:06:06.592205 7f5c973a57a0 20 read_log 194215'18424 (192888'18227) clean 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by osd.4.0:1031 2014-08-12 15:37:02.154869 2014-08-13 18:06:06.592217 7f5c973a57a0 20 read_log 194215'18425 (194215'18424) delete 1461743d/benchmark_data_storage0111-ib.pocstor_2228_object20155/head//24 by client.2174595.0:12108 0.000000 2014-08-13 18:06:06.592228 7f5c973a57a0 20 read_log 194215'18426 (192888'18059) clean 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by osd.4.0:1033 2014-08-12 15:37:02.164544 2014-08-13 18:06:06.592240 7f5c973a57a0 20 read_log 194215'18427 (194215'18426) delete 1eba743d/benchmark_data_storage0111-ib.pocstor_41097_object16175/head//24 by client.2174595.0:12110 0.000000 2014-08-13 18:06:06.592251 7f5c973a57a0 20 read_log 194215'18428 (192888'18028) clean 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by osd.4.0:1035 2014-08-12 15:37:02.173307 2014-08-13 18:06:06.592263 7f5c973a57a0 20 read_log 194215'18429 (194215'18428) delete 62af743d/benchmark_data_storage0111-ib.pocstor_41097_object1354/head//24 by client.2174595.0:12112 0.000000 2014-08-13 18:06:06.592274 7f5c973a57a0 20 read_log 194215'18430 (193305'18327) clean 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by osd.4.0:1037 2014-08-12 15:37:02.182168 2014-08-13 18:06:06.592286 7f5c973a57a0 20 read_log 194215'18431 (194215'18430) delete 7f48843d/benchmark_data_storage0111-ib.pocstor_62621_object10412/head//24 by client.2174595.0:12114 0.000000 2014-08-13 18:06:06.592320 7f5c973a57a0 20 read_log 194215'18432 (192888'18067) clean ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by osd.4.0:1039 2014-08-12 15:37:02.192336 2014-08-13 18:06:06.592336 7f5c973a57a0 20 read_log 194215'18433 (194215'18432) delete ee1b843d/benchmark_data_storage0111-ib.pocstor_41097_object20094/head//24 by client.2174595.0:12116 0.000000 2014-08-13 18:06:06.592347 7f5c973a57a0 20 read_log 194215'18434 (192888'18208) clean 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by osd.4.0:1041 2014-08-12 15:37:02.200741 2014-08-13 18:06:06.592363 7f5c973a57a0 20 read_log 194215'18435 (194215'18434) delete 49fc843d/benchmark_data_storage0111-ib.pocstor_2228_object6569/head//24 by client.2174595.0:12118 0.000000 2014-08-13 18:06:06.592382 7f5c973a57a0 20 read_log 194215'18436 (192888'18075) clean 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by osd.4.0:1043 2014-08-12 15:37:02.209144 2014-08-13 18:06:06.592395 7f5c973a57a0 20 read_log 194215'18437 (194215'18436) delete 542f943d/benchmark_data_storage0111-ib.pocstor_41097_object24355/head//24 by client.2174595.0:12120 0.000000 2014-08-13 18:06:06.592409 7f5c973a57a0 20 read_log 194215'18438 (192834'17844) clean 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by osd.4.0:1045 2014-08-12 15:37:02.216903 2014-08-13 18:06:06.592437 7f5c973a57a0 20 read_log 194215'18439 (194215'18438) delete 4690a43d/benchmark_data_storage0111-ib.pocstor_41526_object16061/head//24 by client.2174595.0:12122 0.000000 2014-08-13 18:06:06.592449 7f5c973a57a0 20 read_log 194215'18440 (192888'18198) clean b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by osd.4.0:1047 2014-08-12 15:37:02.225557 2014-08-13 18:06:06.592474 7f5c973a57a0 20 read_log 194215'18441 (194215'18440) delete b31a43d/benchmark_data_storage0111-ib.pocstor_2228_object2283/head//24 by client.2174595.0:12124 0.000000 2014-08-13 18:06:06.592485 7f5c973a57a0 20 read_log 194215'18442 (193305'18352) clean 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by osd.4.0:1049 2014-08-12 15:37:02.234344 2014-08-13 18:06:06.592498 7f5c973a57a0 20 read_log 194215'18443 (194215'18442) delete 131ca43d/benchmark_data_storage0111-ib.pocstor_62621_object31858/head//24 by client.2174595.0:12126 0.000000 2014-08-13 18:06:06.592509 7f5c973a57a0 20 read_log 194215'18444 (192888'18031) clean 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by osd.4.0:1051 2014-08-12 15:37:02.241818 2014-08-13 18:06:06.592521 7f5c973a57a0 20 read_log 194215'18445 (194215'18444) delete 20f1b43d/benchmark_data_storage0111-ib.pocstor_41097_object3050/head//24 by client.2174595.0:12128 0.000000 2014-08-13 18:06:06.592556 7f5c973a57a0 20 read_log 194215'18446 (192888'18061) clean 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by osd.4.0:1053 2014-08-12 15:37:02.249278 2014-08-13 18:06:06.592601 7f5c973a57a0 20 read_log 194215'18447 (194215'18446) delete 9040c43d/benchmark_data_storage0111-ib.pocstor_41097_object16892/head//24 by client.2174595.0:12130 0.000000 2014-08-13 18:06:06.592613 7f5c973a57a0 20 read_log 194215'18448 (192888'18207) clean cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by osd.4.0:1055 2014-08-12 15:37:02.257179 2014-08-13 18:06:06.592629 7f5c973a57a0 20 read_log 194215'18449 (194215'18448) delete cda3c43d/benchmark_data_storage0111-ib.pocstor_2228_object6423/head//24 by client.2174595.0:12132 0.000000 2014-08-13 18:06:06.592640 7f5c973a57a0 20 read_log 194215'18450 (192888'18216) clean 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by osd.4.0:1057 2014-08-12 15:37:02.265073 2014-08-13 18:06:06.592652 7f5c973a57a0 20 read_log 194215'18451 (194215'18450) delete 9936c43d/benchmark_data_storage0111-ib.pocstor_2228_object13186/head//24 by client.2174595.0:12134 0.000000 2014-08-13 18:06:06.592663 7f5c973a57a0 20 read_log 194215'18452 (192888'18049) clean 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by osd.4.0:1059 2014-08-12 15:37:02.273190 2014-08-13 18:06:06.592675 7f5c973a57a0 20 read_log 194215'18453 (194215'18452) delete 7d46c43d/benchmark_data_storage0111-ib.pocstor_41097_object12231/head//24 by client.2174595.0:12136 0.000000 2014-08-13 18:06:06.592686 7f5c973a57a0 20 read_log 194215'18454 (193305'18334) clean c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by osd.4.0:1061 2014-08-12 15:37:02.282158 2014-08-13 18:06:06.592699 7f5c973a57a0 20 read_log 194215'18455 (194215'18454) delete c896c43d/benchmark_data_storage0111-ib.pocstor_62621_object15299/head//24 by client.2174595.0:12138 0.000000 2014-08-13 18:06:06.592717 7f5c973a57a0 20 read_log 194215'18456 (193305'18348) clean b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by osd.4.0:1063 2014-08-12 15:37:02.290968 2014-08-13 18:06:06.592730 7f5c973a57a0 20 read_log 194215'18457 (194215'18456) delete b506d43d/benchmark_data_storage0111-ib.pocstor_62621_object28016/head//24 by client.2174595.0:12140 0.000000 2014-08-13 18:06:06.592741 7f5c973a57a0 20 read_log 194215'18458 (192888'18079) clean 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by osd.4.0:1065 2014-08-12 15:37:02.300164 2014-08-13 18:06:06.592753 7f5c973a57a0 20 read_log 194215'18459 (194215'18458) delete 2baed43d/benchmark_data_storage0111-ib.pocstor_41097_object28264/head//24 by client.2174595.0:12142 0.000000 2014-08-13 18:06:06.592772 7f5c973a57a0 20 read_log 194215'18460 (193305'18319) clean 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by osd.4.0:1067 2014-08-12 15:37:02.308636 2014-08-13 18:06:06.592786 7f5c973a57a0 20 read_log 194215'18461 (194215'18460) delete 9c26e43d/benchmark_data_storage0111-ib.pocstor_62621_object1244/head//24 by client.2174595.0:12144 0.000000 2014-08-13 18:06:06.592798 7f5c973a57a0 20 read_log 194215'18462 (192834'17107) clean 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by osd.4.0:1069 2014-08-12 15:37:02.341275 2014-08-13 18:06:06.592810 7f5c973a57a0 20 read_log 194215'18463 (194215'18462) delete 81c9e43d/benchmark_data_storage0111-ib.pocstor_26659_object6926/head//24 by client.2174595.0:12146 0.000000 2014-08-13 18:06:06.592821 7f5c973a57a0 20 read_log 194215'18464 (192888'18189) clean 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by osd.4.0:1071 2014-08-12 15:37:02.350146 2014-08-13 18:06:06.592833 7f5c973a57a0 20 read_log 194215'18465 (194215'18464) delete 611be43d/benchmark_data_storage0111-ib.pocstor_2228_object545/head//24 by client.2174595.0:12148 0.000000 2014-08-13 18:06:06.592844 7f5c973a57a0 20 read_log 194215'18466 (193305'18322) clean 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by osd.4.0:1073 2014-08-12 15:37:02.359231 2014-08-13 18:06:06.592856 7f5c973a57a0 20 read_log 194215'18467 (194215'18466) delete 571fe43d/benchmark_data_storage0111-ib.pocstor_62621_object5932/head//24 by client.2174595.0:12150 0.000000 2014-08-13 18:06:06.592867 7f5c973a57a0 20 read_log 194215'18468 (192888'18204) clean 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by osd.4.0:1075 2014-08-12 15:37:02.367070 2014-08-13 18:06:06.592879 7f5c973a57a0 20 read_log 194215'18469 (194215'18468) delete 5b14f43d/benchmark_data_storage0111-ib.pocstor_2228_object4822/head//24 by client.2174595.0:12152 0.000000 2014-08-13 18:06:06.592890 7f5c973a57a0 20 read_log 194215'18470 (192834'17135) clean 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by osd.4.0:1077 2014-08-12 15:37:02.375240 2014-08-13 18:06:06.592902 7f5c973a57a0 20 read_log 194215'18471 (194215'18470) delete 3feaf43d/benchmark_data_storage0111-ib.pocstor_26659_object10046/head//24 by client.2174595.0:12154 0.000000 2014-08-13 18:06:06.592913 7f5c973a57a0 20 read_log 194215'18472 (192888'18042) clean eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by osd.4.0:1079 2014-08-12 15:37:02.383461 2014-08-13 18:06:06.592925 7f5c973a57a0 20 read_log 194215'18473 (194215'18472) delete eba1083d/benchmark_data_storage0111-ib.pocstor_41097_object10294/head//24 by client.2174595.0:12156 0.000000 2014-08-13 18:06:06.592936 7f5c973a57a0 20 read_log 194215'18474 (192888'18074) clean 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by osd.4.0:1081 2014-08-12 15:37:02.413267 2014-08-13 18:06:06.592948 7f5c973a57a0 20 read_log 194215'18475 (194215'18474) delete 8b4083d/benchmark_data_storage0111-ib.pocstor_41097_object24111/head//24 by client.2174595.0:12158 0.000000 2014-08-13 18:06:06.592991 7f5c973a57a0 20 read_log 194215'18476 (192834'17868) clean e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by osd.4.0:1083 2014-08-12 15:37:02.420938 2014-08-13 18:06:06.593005 7f5c973a57a0 20 read_log 194215'18477 (194215'18476) delete e75b083d/benchmark_data_storage0111-ib.pocstor_41526_object18586/head//24 by client.2174595.0:12160 0.000000 2014-08-13 18:06:06.593017 7f5c973a57a0 20 read_log 194215'18478 (192888'18044) clean 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by osd.4.0:1085 2014-08-12 15:37:02.429860 2014-08-13 18:06:06.593029 7f5c973a57a0 20 read_log 194215'18479 (194215'18478) delete 34b2183d/benchmark_data_storage0111-ib.pocstor_41097_object11342/head//24 by client.2174595.0:12162 0.000000 2014-08-13 18:06:06.593040 7f5c973a57a0 20 read_log 194215'18480 (192834'17136) clean 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by osd.4.0:1087 2014-08-12 15:37:02.438798 2014-08-13 18:06:06.593052 7f5c973a57a0 20 read_log 194215'18481 (194215'18480) delete 3260483d/benchmark_data_storage0111-ib.pocstor_26659_object10331/head//24 by client.2174595.0:12164 0.000000 2014-08-13 18:06:06.593063 7f5c973a57a0 20 read_log 194215'18482 (192888'18043) clean b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by osd.4.0:1089 2014-08-12 15:37:02.453149 2014-08-13 18:06:06.593075 7f5c973a57a0 20 read_log 194215'18483 (194215'18482) delete b71f483d/benchmark_data_storage0111-ib.pocstor_41097_object11023/head//24 by client.2174595.0:12166 0.000000 2014-08-13 18:06:06.593086 7f5c973a57a0 20 read_log 194215'18484 (193305'18351) clean 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by osd.4.0:1091 2014-08-12 15:37:02.461437 2014-08-13 18:06:06.593101 7f5c973a57a0 20 read_log 194215'18485 (194215'18484) delete 745f583d/benchmark_data_storage0111-ib.pocstor_62621_object29358/head//24 by client.2174595.0:12168 0.000000 2014-08-13 18:06:06.593113 7f5c973a57a0 20 read_log 194215'18486 (193305'18346) clean f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by osd.4.0:1093 2014-08-12 15:37:02.470504 2014-08-13 18:06:06.593125 7f5c973a57a0 20 read_log 194215'18487 (194215'18486) delete f7683d/benchmark_data_storage0111-ib.pocstor_62621_object25756/head//24 by client.2174595.0:12170 0.000000 2014-08-13 18:06:06.593136 7f5c973a57a0 20 read_log 194215'18488 (193305'18343) clean 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by osd.4.0:1095 2014-08-12 15:37:02.479878 2014-08-13 18:06:06.593157 7f5c973a57a0 20 read_log 194215'18489 (194215'18488) delete 9ccc683d/benchmark_data_storage0111-ib.pocstor_62621_object23699/head//24 by client.2174595.0:12172 0.000000 2014-08-13 18:06:06.593168 7f5c973a57a0 20 read_log 194215'18490 (193305'18340) clean 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by osd.4.0:1097 2014-08-12 15:37:02.489734 2014-08-13 18:06:06.593181 7f5c973a57a0 20 read_log 194215'18491 (194215'18490) delete 7f0f683d/benchmark_data_storage0111-ib.pocstor_62621_object21966/head//24 by client.2174595.0:12174 0.000000 2014-08-13 18:06:06.593195 7f5c973a57a0 20 read_log 194215'18492 (192834'17689) clean 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by osd.4.0:1099 2014-08-12 15:37:02.498004 2014-08-13 18:06:06.593207 7f5c973a57a0 20 read_log 194215'18493 (194215'18492) delete 119783d/benchmark_data_storage0111-ib.pocstor_41526_object641/head//24 by client.2174595.0:12176 0.000000 2014-08-13 18:06:06.593218 7f5c973a57a0 20 read_log 194215'18494 (193305'18337) clean 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by osd.4.0:1101 2014-08-12 15:37:02.506158 2014-08-13 18:06:06.593246 7f5c973a57a0 20 read_log 194215'18495 (194215'18494) delete 693f783d/benchmark_data_storage0111-ib.pocstor_62621_object16328/head//24 by client.2174595.0:12178 0.000000 2014-08-13 18:06:06.593267 7f5c973a57a0 20 read_log 194215'18496 (193305'18336) clean b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by osd.4.0:1103 2014-08-12 15:37:02.514067 2014-08-13 18:06:06.593291 7f5c973a57a0 20 read_log 194215'18497 (194215'18496) delete b422883d/benchmark_data_storage0111-ib.pocstor_62621_object15912/head//24 by client.2174595.0:12180 0.000000 2014-08-13 18:06:06.593303 7f5c973a57a0 20 read_log 194215'18498 (192834'17706) clean 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by osd.4.0:1105 2014-08-12 15:37:02.532379 2014-08-13 18:06:06.593315 7f5c973a57a0 20 read_log 194215'18499 (194215'18498) delete 9f15883d/benchmark_data_storage0111-ib.pocstor_41526_object2766/head//24 by client.2174595.0:12182 0.000000 2014-08-13 18:06:06.593335 7f5c973a57a0 20 read_log 194215'18500 (192888'18082) clean 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by osd.4.0:1107 2014-08-12 15:37:02.542050 2014-08-13 18:06:06.593350 7f5c973a57a0 20 read_log 194215'18501 (194215'18500) delete 9620983d/benchmark_data_storage0111-ib.pocstor_41097_object30460/head//24 by client.2174595.0:12184 0.000000 2014-08-13 18:06:06.593361 7f5c973a57a0 20 read_log 194215'18502 (192834'17749) clean 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by osd.4.0:1109 2014-08-12 15:37:02.551334 2014-08-13 18:06:06.593382 7f5c973a57a0 20 read_log 194215'18503 (194215'18502) delete 7593983d/benchmark_data_storage0111-ib.pocstor_41526_object7340/head//24 by client.2174595.0:12186 0.000000 2014-08-13 18:06:06.593393 7f5c973a57a0 20 read_log 194215'18504 (193305'18349) clean 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by osd.4.0:1111 2014-08-12 15:37:02.558801 2014-08-13 18:06:06.593413 7f5c973a57a0 20 read_log 194215'18505 (194215'18504) delete 8e94983d/benchmark_data_storage0111-ib.pocstor_62621_object28207/head//24 by client.2174595.0:12188 0.000000 2014-08-13 18:06:06.593424 7f5c973a57a0 20 read_log 194215'18506 (193305'18332) clean 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by osd.4.0:1113 2014-08-12 15:37:02.566321 2014-08-13 18:06:06.593437 7f5c973a57a0 20 read_log 194215'18507 (194215'18506) delete 22b8983d/benchmark_data_storage0111-ib.pocstor_62621_object11617/head//24 by client.2174595.0:12190 0.000000 2014-08-13 18:06:06.593448 7f5c973a57a0 20 read_log 194215'18508 (193305'18338) clean c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by osd.4.0:1115 2014-08-12 15:37:02.574227 2014-08-13 18:06:06.593460 7f5c973a57a0 20 read_log 194215'18509 (194215'18508) delete c6e8a83d/benchmark_data_storage0111-ib.pocstor_62621_object17065/head//24 by client.2174595.0:12192 0.000000 2014-08-13 18:06:06.593471 7f5c973a57a0 20 read_log 194215'18510 (192834'17890) clean 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by osd.4.0:1117 2014-08-12 15:37:02.582277 2014-08-13 18:06:06.593483 7f5c973a57a0 20 read_log 194215'18511 (194215'18510) delete 9071b83d/benchmark_data_storage0111-ib.pocstor_41526_object20903/head//24 by client.2174595.0:12194 0.000000 2014-08-13 18:06:06.593494 7f5c973a57a0 20 read_log 194215'18512 (193305'18328) clean 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by osd.4.0:1119 2014-08-12 15:37:02.616040 2014-08-13 18:06:06.593506 7f5c973a57a0 20 read_log 194215'18513 (194215'18512) delete 8bc2c83d/benchmark_data_storage0111-ib.pocstor_62621_object10932/head//24 by client.2174595.0:12196 0.000000 2014-08-13 18:06:06.593517 7f5c973a57a0 20 read_log 194215'18514 (193305'18326) clean a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by osd.4.0:1121 2014-08-12 15:37:02.626012 2014-08-13 18:06:06.593529 7f5c973a57a0 20 read_log 194215'18515 (194215'18514) delete a595c83d/benchmark_data_storage0111-ib.pocstor_62621_object8414/head//24 by client.2174595.0:12198 0.000000 2014-08-13 18:06:06.593547 7f5c973a57a0 20 read_log 194215'18516 (192834'17112) clean f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by osd.4.0:1123 2014-08-12 15:37:02.634701 2014-08-13 18:06:06.593560 7f5c973a57a0 20 read_log 194215'18517 (194215'18516) delete f907c83d/benchmark_data_storage0111-ib.pocstor_26659_object7769/head//24 by client.2174595.0:12200 0.000000 2014-08-13 18:06:06.593571 7f5c973a57a0 20 read_log 194215'18518 (192888'18206) clean 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by osd.4.0:1125 2014-08-12 15:37:02.652792 2014-08-13 18:06:06.593583 7f5c973a57a0 20 read_log 194215'18519 (194215'18518) delete 735cc83d/benchmark_data_storage0111-ib.pocstor_2228_object4974/head//24 by client.2174595.0:12202 0.000000 2014-08-13 18:06:06.593594 7f5c973a57a0 20 read_log 194215'18520 (193305'18344) clean fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by osd.4.0:1127 2014-08-12 15:37:02.660799 2014-08-13 18:06:06.593606 7f5c973a57a0 20 read_log 194215'18521 (194215'18520) delete fd15d83d/benchmark_data_storage0111-ib.pocstor_62621_object25135/head//24 by client.2174595.0:12204 0.000000 2014-08-13 18:06:06.593617 7f5c973a57a0 20 read_log 194215'18522 (192888'18209) clean e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by osd.4.0:1129 2014-08-12 15:37:02.670602 2014-08-13 18:06:06.593629 7f5c973a57a0 20 read_log 194215'18523 (194215'18522) delete e0dbd83d/benchmark_data_storage0111-ib.pocstor_2228_object7382/head//24 by client.2174595.0:12206 0.000000 2014-08-13 18:06:06.593639 7f5c973a57a0 20 read_log 194215'18524 (192834'17827) clean 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by osd.4.0:1131 2014-08-12 15:37:02.678315 2014-08-13 18:06:06.593670 7f5c973a57a0 20 read_log 194215'18525 (194215'18524) delete 272fd83d/benchmark_data_storage0111-ib.pocstor_41526_object13496/head//24 by client.2174595.0:12208 0.000000 2014-08-13 18:06:06.593681 7f5c973a57a0 20 read_log 194215'18526 (192888'18230) clean 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by osd.4.0:1133 2014-08-12 15:37:02.685349 2014-08-13 18:06:06.593694 7f5c973a57a0 20 read_log 194215'18527 (194215'18526) delete 1a4be83d/benchmark_data_storage0111-ib.pocstor_2228_object23055/head//24 by client.2174595.0:12210 0.000000 2014-08-13 18:06:06.593705 7f5c973a57a0 20 read_log 194215'18528 (192888'18064) clean fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by osd.4.0:1135 2014-08-12 15:37:02.693546 2014-08-13 18:06:06.593717 7f5c973a57a0 20 read_log 194215'18529 (194215'18528) delete fbd1f83d/benchmark_data_storage0111-ib.pocstor_41097_object19125/head//24 by client.2174595.0:12212 0.000000 2014-08-13 18:06:06.593728 7f5c973a57a0 20 read_log 194215'18530 (192834'17172) clean 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by osd.4.0:1137 2014-08-12 15:37:02.716647 2014-08-13 18:06:06.593740 7f5c973a57a0 20 read_log 194215'18531 (194215'18530) delete 18a3f83d/benchmark_data_storage0111-ib.pocstor_26659_object15837/head//24 by client.2174595.0:12214 0.000000 2014-08-13 18:06:06.593754 7f5c973a57a0 20 read_log 194215'18532 (192888'18060) clean e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by osd.4.0:1139 2014-08-12 15:37:02.726140 2014-08-13 18:06:06.593767 7f5c973a57a0 20 read_log 194215'18533 (194215'18532) delete e1a8f83d/benchmark_data_storage0111-ib.pocstor_41097_object16276/head//24 by client.2174595.0:12216 0.000000 2014-08-13 18:06:06.593780 7f5c973a57a0 20 read_log 194215'18534 (192834'17746) clean 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by osd.4.0:1141 2014-08-12 15:37:02.734727 2014-08-13 18:06:06.593792 7f5c973a57a0 20 read_log 194215'18535 (194215'18534) delete 4c3af83d/benchmark_data_storage0111-ib.pocstor_41526_object7269/head//24 by client.2174595.0:12218 0.000000 2014-08-13 18:06:06.593812 7f5c973a57a0 20 read_log 194215'18536 (192888'18052) clean 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by osd.4.0:1143 2014-08-12 15:37:02.742385 2014-08-13 18:06:06.593827 7f5c973a57a0 20 read_log 194215'18537 (194215'18536) delete 2b9ef83d/benchmark_data_storage0111-ib.pocstor_41097_object13856/head//24 by client.2174595.0:12220 0.000000 2014-08-13 18:06:06.593838 7f5c973a57a0 20 read_log 194215'18538 (193305'18330) clean cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by osd.4.0:1145 2014-08-12 15:37:02.750312 2014-08-13 18:06:06.593850 7f5c973a57a0 20 read_log 194215'18539 (194215'18538) delete cd020c3d/benchmark_data_storage0111-ib.pocstor_62621_object11240/head//24 by client.2174595.0:12222 0.000000 2014-08-13 18:06:06.593861 7f5c973a57a0 20 read_log 194215'18540 (192888'18035) clean 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by osd.4.0:1147 2014-08-12 15:37:02.773783 2014-08-13 18:06:06.593875 7f5c973a57a0 20 read_log 194215'18541 (194215'18540) delete 7b860c3d/benchmark_data_storage0111-ib.pocstor_41097_object5847/head//24 by client.2174595.0:12224 0.000000 2014-08-13 18:06:06.593886 7f5c973a57a0 20 read_log 194215'18542 (192834'17086) clean 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by osd.4.0:1149 2014-08-12 15:37:02.783017 2014-08-13 18:06:06.593899 7f5c973a57a0 20 read_log 194215'18543 (194215'18542) delete 67870c3d/benchmark_data_storage0111-ib.pocstor_26659_object3931/head//24 by client.2174595.0:12226 0.000000 2014-08-13 18:06:06.593910 7f5c973a57a0 20 read_log 194215'18544 (192834'17748) clean 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by osd.4.0:1151 2014-08-12 15:37:02.790729 2014-08-13 18:06:06.593922 7f5c973a57a0 20 read_log 194215'18545 (194215'18544) delete 8a1b0c3d/benchmark_data_storage0111-ib.pocstor_41526_object7298/head//24 by client.2174595.0:12228 0.000000 2014-08-13 18:06:06.593933 7f5c973a57a0 20 read_log 194215'18546 (193305'18317) clean 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by osd.4.0:1153 2014-08-12 15:37:02.800528 2014-08-13 18:06:06.593952 7f5c973a57a0 20 read_log 194215'18547 (194215'18546) delete 3d001c3d/benchmark_data_storage0111-ib.pocstor_62621_object615/head//24 by client.2174595.0:12230 0.000000 2014-08-13 18:06:06.593963 7f5c973a57a0 20 read_log 194215'18548 (193305'18324) clean 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by osd.4.0:1155 2014-08-12 15:37:02.809078 2014-08-13 18:06:06.593981 7f5c973a57a0 20 read_log 194215'18549 (194215'18548) delete 7c1a1c3d/benchmark_data_storage0111-ib.pocstor_62621_object6361/head//24 by client.2174595.0:12232 0.000000 2014-08-13 18:06:06.593992 7f5c973a57a0 20 read_log 194215'18550 (192834'17708) clean d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by osd.4.0:1157 2014-08-12 15:37:02.817771 2014-08-13 18:06:06.594004 7f5c973a57a0 20 read_log 194215'18551 (194215'18550) delete d7dd1c3d/benchmark_data_storage0111-ib.pocstor_41526_object3263/head//24 by client.2174595.0:12234 0.000000 2014-08-13 18:06:06.594017 7f5c973a57a0 20 read_log 194215'18552 (192888'18213) clean 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by osd.4.0:1159 2014-08-12 15:37:02.825476 2014-08-13 18:06:06.594030 7f5c973a57a0 20 read_log 194215'18553 (194215'18552) delete 86b62c3d/benchmark_data_storage0111-ib.pocstor_2228_object12782/head//24 by client.2174595.0:12236 0.000000 2014-08-13 18:06:06.594043 7f5c973a57a0 20 read_log 194215'18554 (192888'18201) clean 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by osd.4.0:1161 2014-08-12 15:37:02.852203 2014-08-13 18:06:06.594055 7f5c973a57a0 20 read_log 194215'18555 (194215'18554) delete 68d92c3d/benchmark_data_storage0111-ib.pocstor_2228_object4314/head//24 by client.2174595.0:12238 0.000000 2014-08-13 18:06:06.594077 7f5c973a57a0 20 read_log 194215'18556 (192834'17137) clean 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by osd.4.0:1163 2014-08-12 15:37:02.861156 2014-08-13 18:06:06.594092 7f5c973a57a0 20 read_log 194215'18557 (194215'18556) delete 5b1d2c3d/benchmark_data_storage0111-ib.pocstor_26659_object10525/head//24 by client.2174595.0:12240 0.000000 2014-08-13 18:06:06.594103 7f5c973a57a0 20 read_log 194215'18558 (193305'18353) clean db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by osd.4.0:1165 2014-08-12 15:37:02.869845 2014-08-13 18:06:06.594116 7f5c973a57a0 20 read_log 194215'18559 (194215'18558) delete db513c3d/benchmark_data_storage0111-ib.pocstor_62621_object32090/head//24 by client.2174595.0:12242 0.000000 2014-08-13 18:06:06.594129 7f5c973a57a0 20 read_log 194215'18560 (192834'17836) clean 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by osd.4.0:1167 2014-08-12 15:37:02.878329 2014-08-13 18:06:06.594142 7f5c973a57a0 20 read_log 194215'18561 (194215'18560) delete 6b4f4c3d/benchmark_data_storage0111-ib.pocstor_41526_object15064/head//24 by client.2174595.0:12244 0.000000 2014-08-13 18:06:06.594160 7f5c973a57a0 20 read_log 194215'18562 (192888'18235) clean eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by osd.4.0:1169 2014-08-12 15:37:02.887101 2014-08-13 18:06:06.594173 7f5c973a57a0 20 read_log 194215'18563 (194215'18562) delete eb565c3d/benchmark_data_storage0111-ib.pocstor_2228_object25548/head//24 by client.2174595.0:12246 0.000000 2014-08-13 18:06:06.594184 7f5c973a57a0 20 read_log 194215'18564 (193305'18320) clean ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by osd.4.0:1171 2014-08-12 15:37:02.895564 2014-08-13 18:06:06.594196 7f5c973a57a0 20 read_log 194215'18565 (194215'18564) delete ae6e5c3d/benchmark_data_storage0111-ib.pocstor_62621_object1913/head//24 by client.2174595.0:12248 0.000000 2014-08-13 18:06:06.594207 7f5c973a57a0 20 read_log 194215'18566 (192834'17063) clean 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by osd.4.0:1173 2014-08-12 15:37:02.904688 2014-08-13 18:06:06.594219 7f5c973a57a0 20 read_log 194215'18567 (194215'18566) delete 3bb66c3d/benchmark_data_storage0111-ib.pocstor_26659_object957/head//24 by client.2174595.0:12250 0.000000 2014-08-13 18:06:06.594230 7f5c973a57a0 20 read_log 194215'18568 (192834'17899) clean 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by osd.4.0:1175 2014-08-12 15:37:02.914372 2014-08-13 18:06:06.594245 7f5c973a57a0 20 read_log 194215'18569 (194215'18568) delete 9b376c3d/benchmark_data_storage0111-ib.pocstor_41526_object22931/head//24 by client.2174595.0:12252 0.000000 2014-08-13 18:06:06.594259 7f5c973a57a0 20 read_log 194215'18570 (192888'18024) clean c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by osd.4.0:1177 2014-08-12 15:37:02.923266 2014-08-13 18:06:06.594274 7f5c973a57a0 20 read_log 194215'18571 (194215'18570) delete c2e86c3d/benchmark_data_storage0111-ib.pocstor_41097_object367/head//24 by client.2174595.0:12254 0.000000 2014-08-13 18:06:06.594285 7f5c973a57a0 20 read_log 194215'18572 (192834'17716) clean 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by osd.4.0:1179 2014-08-12 15:37:02.940487 2014-08-13 18:06:06.594297 7f5c973a57a0 20 read_log 194215'18573 (194215'18572) delete 14d6c3d/benchmark_data_storage0111-ib.pocstor_41526_object3666/head//24 by client.2174595.0:12256 0.000000 2014-08-13 18:06:06.594313 7f5c973a57a0 20 read_log 194215'18574 (192834'17192) clean 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by osd.4.0:1181 2014-08-12 15:37:02.961715 2014-08-13 18:06:06.594326 7f5c973a57a0 20 read_log 194215'18575 (194215'18574) delete 27317c3d/benchmark_data_storage0111-ib.pocstor_26659_object18668/head//24 by client.2174595.0:12258 0.000000 2014-08-13 18:06:06.594344 7f5c973a57a0 20 read_log 194215'18576 (192888'18077) clean a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by osd.4.0:1183 2014-08-12 15:37:02.970770 2014-08-13 18:06:06.594362 7f5c973a57a0 20 read_log 194215'18577 (194215'18576) delete a1037c3d/benchmark_data_storage0111-ib.pocstor_41097_object24676/head//24 by client.2174595.0:12260 0.000000 2014-08-13 18:06:06.594374 7f5c973a57a0 20 read_log 194215'18578 (192888'18194) clean 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by osd.4.0:1185 2014-08-12 15:37:02.978963 2014-08-13 18:06:06.594403 7f5c973a57a0 20 read_log 194215'18579 (194215'18578) delete 9c877c3d/benchmark_data_storage0111-ib.pocstor_2228_object1344/head//24 by client.2174595.0:12262 0.000000 2014-08-13 18:06:06.594416 7f5c973a57a0 20 read_log 194215'18580 (193305'18329) clean 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by osd.4.0:1187 2014-08-12 15:37:02.986258 2014-08-13 18:06:06.594433 7f5c973a57a0 20 read_log 194215'18581 (194215'18580) delete 68da7c3d/benchmark_data_storage0111-ib.pocstor_62621_object11098/head//24 by client.2174595.0:12264 0.000000 2014-08-13 18:06:06.594447 7f5c973a57a0 20 read_log 194215'18582 (193305'18325) clean 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by osd.4.0:1189 2014-08-12 15:37:02.995171 2014-08-13 18:06:06.594460 7f5c973a57a0 20 read_log 194215'18583 (194215'18582) delete 70e88c3d/benchmark_data_storage0111-ib.pocstor_62621_object6579/head//24 by client.2174595.0:12266 0.000000 2014-08-13 18:06:06.594471 7f5c973a57a0 20 read_log 194215'18584 (192888'18218) clean 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by osd.4.0:1191 2014-08-12 15:37:03.002944 2014-08-13 18:06:06.594483 7f5c973a57a0 20 read_log 194215'18585 (194215'18584) delete 9daa8c3d/benchmark_data_storage0111-ib.pocstor_2228_object14886/head//24 by client.2174595.0:12268 0.000000 2014-08-13 18:06:06.594502 7f5c973a57a0 20 read_log 194215'18586 (193305'18321) clean 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by osd.4.0:1193 2014-08-12 15:37:03.010354 2014-08-13 18:06:06.594516 7f5c973a57a0 20 read_log 194215'18587 (194215'18586) delete 799f8c3d/benchmark_data_storage0111-ib.pocstor_62621_object3804/head//24 by client.2174595.0:12270 0.000000 2014-08-13 18:06:06.594527 7f5c973a57a0 20 read_log 194215'18588 (192834'17897) clean 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by osd.4.0:1195 2014-08-12 15:37:03.018437 2014-08-13 18:06:06.594546 7f5c973a57a0 20 read_log 194215'18589 (194215'18588) delete 43189c3d/benchmark_data_storage0111-ib.pocstor_41526_object22317/head//24 by client.2174595.0:12272 0.000000 2014-08-13 18:06:06.594561 7f5c973a57a0 20 read_log 194215'18590 (192888'18203) clean 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by osd.4.0:1197 2014-08-12 15:37:03.030592 2014-08-13 18:06:06.594574 7f5c973a57a0 20 read_log 194215'18591 (194215'18590) delete 8df3bc3d/benchmark_data_storage0111-ib.pocstor_2228_object4700/head//24 by client.2174595.0:12274 0.000000 2014-08-13 18:06:06.594597 7f5c973a57a0 20 read_log 194215'18592 (193305'18333) clean 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by osd.4.0:1199 2014-08-12 15:37:03.061077 2014-08-13 18:06:06.594613 7f5c973a57a0 20 read_log 194215'18593 (194215'18592) delete 9d0bcc3d/benchmark_data_storage0111-ib.pocstor_62621_object13122/head//24 by client.2174595.0:12276 0.000000 2014-08-13 18:06:06.594625 7f5c973a57a0 20 read_log 194215'18594 (192888'18192) clean 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by osd.4.0:1201 2014-08-12 15:37:03.071722 2014-08-13 18:06:06.594637 7f5c973a57a0 20 read_log 194215'18595 (194215'18594) delete 70f2dc3d/benchmark_data_storage0111-ib.pocstor_2228_object1156/head//24 by client.2174595.0:12278 0.000000 2014-08-13 18:06:06.594656 7f5c973a57a0 20 read_log 194215'18596 (192888'18058) clean 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by osd.4.0:1203 2014-08-12 15:37:03.079988 2014-08-13 18:06:06.594671 7f5c973a57a0 20 read_log 194215'18597 (194215'18596) delete 3464ec3d/benchmark_data_storage0111-ib.pocstor_41097_object16004/head//24 by client.2174595.0:12280 0.000000 2014-08-13 18:06:06.594690 7f5c973a57a0 20 read_log 194215'18598 (192888'18066) clean 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by osd.4.0:1205 2014-08-12 15:37:03.087543 2014-08-13 18:06:06.594706 7f5c973a57a0 20 read_log 194215'18599 (194215'18598) delete 85eaec3d/benchmark_data_storage0111-ib.pocstor_41097_object19906/head//24 by client.2174595.0:12282 0.000000 2014-08-13 18:06:06.594717 7f5c973a57a0 20 read_log 194215'18600 (192888'18237) clean c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by osd.4.0:1207 2014-08-12 15:37:03.095270 2014-08-13 18:06:06.594730 7f5c973a57a0 20 read_log 194215'18601 (194215'18600) delete c32eec3d/benchmark_data_storage0111-ib.pocstor_2228_object27713/head//24 by client.2174595.0:12284 0.000000 2014-08-13 18:06:06.594741 7f5c973a57a0 20 read_log 194215'18602 (193305'18341) clean cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by osd.4.0:1209 2014-08-12 15:37:03.102213 2014-08-13 18:06:06.594753 7f5c973a57a0 20 read_log 194215'18603 (194215'18602) delete cce6fc3d/benchmark_data_storage0111-ib.pocstor_62621_object22481/head//24 by client.2174595.0:12286 0.000000 2014-08-13 18:06:06.594768 7f5c973a57a0 20 read_log 194215'18604 (192834'17835) clean 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by osd.4.0:1211 2014-08-12 15:37:03.110132 2014-08-13 18:06:06.594781 7f5c973a57a0 20 read_log 194215'18605 (194215'18604) delete 3197fc3d/benchmark_data_storage0111-ib.pocstor_41526_object14843/head//24 by client.2174595.0:12288 0.000000 2014-08-13 18:06:06.594792 7f5c973a57a0 20 read_log 194215'18606 (192888'18034) clean c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by osd.4.0:1213 2014-08-12 15:37:03.117352 2014-08-13 18:06:06.594804 7f5c973a57a0 20 read_log 194215'18607 (194215'18606) delete c6bcfc3d/benchmark_data_storage0111-ib.pocstor_41097_object5014/head//24 by client.2174595.0:12290 0.000000 2014-08-13 18:06:06.594815 7f5c973a57a0 20 read_log 194215'18608 (193305'18345) clean e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by osd.4.0:1215 2014-08-12 15:37:03.125730 2014-08-13 18:06:06.594827 7f5c973a57a0 20 read_log 194215'18609 (194215'18608) delete e80ffc3d/benchmark_data_storage0111-ib.pocstor_62621_object25231/head//24 by client.2174595.0:12292 0.000000 2014-08-13 18:06:06.594839 7f5c973a57a0 20 read_log 194215'18610 (194215'18610) modify 3d/hit_set_24.3d_archive_2014-08-12 15:32:25.863067_2014-08-12 17:24:40.629052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.630222 2014-08-13 18:06:06.594851 7f5c973a57a0 20 read_log 194215'18611 (193305'18241) delete 3d/hit_set_24.3d_archive_2014-08-11 18:37:04.904929_2014-08-11 18:42:35.578890/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.630222 2014-08-13 18:06:06.594864 7f5c973a57a0 20 read_log 194215'18612 (0'0) promote 2fbabc3d/benchmark_data_storage0111-ib.pocstor_34628_object187/head//24 by osd.4.0:5229 2014-08-12 17:24:40.677366 2014-08-13 18:06:06.594876 7f5c973a57a0 20 read_log 194215'18613 (194215'18612) modify 2fbabc3d/benchmark_data_storage0111-ib.pocstor_34628_object187/head//24 by client.2193759.0:188 2014-08-12 17:24:40.620380 2014-08-13 18:06:06.594889 7f5c973a57a0 20 read_log 194215'18614 (0'0) promote ff60cc3d/benchmark_data_storage0111-ib.pocstor_34628_object1470/head//24 by osd.4.0:5248 2014-08-12 17:24:46.823842 2014-08-13 18:06:06.594929 7f5c973a57a0 20 read_log 194215'18615 (194215'18614) modify ff60cc3d/benchmark_data_storage0111-ib.pocstor_34628_object1470/head//24 by client.2193759.0:1471 2014-08-12 17:24:46.814795 2014-08-13 18:06:06.594945 7f5c973a57a0 20 read_log 194370'18616 (0'0) promote 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by osd.20.0:4204 2014-08-12 20:30:20.439055 2014-08-13 18:06:06.594958 7f5c973a57a0 20 read_log 194370'18617 (194370'18616) modify 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by client.1508531.0:1616 2014-08-12 20:30:20.430889 2014-08-13 18:06:06.594973 7f5c973a57a0 20 read_log 194370'18618 (0'0) promote cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by osd.20.0:4250 2014-08-12 20:30:27.367180 2014-08-13 18:06:06.594985 7f5c973a57a0 20 read_log 194370'18619 (194370'18618) modify cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by client.1508531.0:3488 2014-08-12 20:30:27.359030 2014-08-13 18:06:06.594997 7f5c973a57a0 20 read_log 194370'18620 (0'0) promote 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by osd.20.0:4266 2014-08-12 20:30:29.758622 2014-08-13 18:06:06.595009 7f5c973a57a0 20 read_log 194370'18621 (194370'18620) modify 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by client.1508531.0:4127 2014-08-12 20:30:29.751699 2014-08-13 18:06:06.595021 7f5c973a57a0 20 read_log 194370'18622 (0'0) promote 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by osd.20.0:4310 2014-08-12 20:30:41.002392 2014-08-13 18:06:06.595034 7f5c973a57a0 20 read_log 194370'18623 (194370'18622) modify 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by client.1508531.0:6728 2014-08-12 20:30:40.992116 2014-08-13 18:06:06.595046 7f5c973a57a0 20 read_log 194370'18624 (0'0) promote fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by osd.20.0:4358 2014-08-12 20:30:51.154468 2014-08-13 18:06:06.595058 7f5c973a57a0 20 read_log 194370'18625 (194370'18624) modify fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by client.1508531.0:8648 2014-08-12 20:30:51.141855 2014-08-13 18:06:06.595070 7f5c973a57a0 20 read_log 194370'18626 (0'0) promote cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by osd.20.0:4380 2014-08-12 20:30:55.175714 2014-08-13 18:06:06.595084 7f5c973a57a0 20 read_log 194370'18627 (194370'18626) modify cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by client.1508531.0:9310 2014-08-12 20:30:55.167609 2014-08-13 18:06:06.595097 7f5c973a57a0 20 read_log 194370'18628 (0'0) promote f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by osd.20.0:4388 2014-08-12 20:30:57.263858 2014-08-13 18:06:06.595109 7f5c973a57a0 20 read_log 194370'18629 (194370'18628) modify f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by client.1508531.0:9659 2014-08-12 20:30:57.255203 2014-08-13 18:06:06.595121 7f5c973a57a0 20 read_log 194370'18630 (0'0) promote 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by osd.20.0:4398 2014-08-12 20:31:00.147671 2014-08-13 18:06:06.595133 7f5c973a57a0 20 read_log 194370'18631 (194370'18630) modify 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by client.1508531.0:10152 2014-08-12 20:31:00.138476 2014-08-13 18:06:06.595154 7f5c973a57a0 20 read_log 194370'18632 (0'0) promote fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by osd.20.0:4404 2014-08-12 20:31:00.957385 2014-08-13 18:06:06.595167 7f5c973a57a0 20 read_log 194370'18633 (194370'18632) modify fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by client.1508531.0:10288 2014-08-12 20:31:00.946368 2014-08-13 18:06:06.595180 7f5c973a57a0 20 read_log 194370'18634 (0'0) promote d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by osd.20.0:4420 2014-08-12 20:31:04.922557 2014-08-13 18:06:06.595202 7f5c973a57a0 20 read_log 194370'18635 (194370'18634) modify d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by client.1508531.0:10941 2014-08-12 20:31:04.914356 2014-08-13 18:06:06.595217 7f5c973a57a0 20 read_log 194370'18636 (0'0) promote 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by osd.20.0:4458 2014-08-12 20:31:19.062773 2014-08-13 18:06:06.595230 7f5c973a57a0 20 read_log 194370'18637 (194370'18636) modify 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by client.1508531.0:12968 2014-08-12 20:31:19.055563 2014-08-13 18:06:06.595242 7f5c973a57a0 20 read_log 194370'18638 (0'0) promote 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by osd.20.0:4460 2014-08-12 20:31:19.808728 2014-08-13 18:06:06.595259 7f5c973a57a0 20 read_log 194370'18639 (194370'18638) modify 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by client.1508531.0:13048 2014-08-12 20:31:19.800706 2014-08-13 18:06:06.595271 7f5c973a57a0 20 read_log 194370'18640 (0'0) promote 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by osd.20.0:4462 2014-08-12 20:31:22.383879 2014-08-13 18:06:06.595284 7f5c973a57a0 20 read_log 194370'18641 (194370'18640) modify 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by client.1508531.0:13314 2014-08-12 20:31:22.374937 2014-08-13 18:06:06.595299 7f5c973a57a0 20 read_log 194377'18642 (0'0) promote 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by osd.20.0:4472 2014-08-12 20:32:58.011238 2014-08-13 18:06:06.595314 7f5c973a57a0 20 read_log 194377'18643 (194377'18642) modify 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by client.2231190.0:204 2014-08-12 20:32:58.004900 2014-08-13 18:06:06.595327 7f5c973a57a0 20 read_log 194377'18644 (0'0) promote a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by osd.20.0:4490 2014-08-12 20:33:00.913860 2014-08-13 18:06:06.595339 7f5c973a57a0 20 read_log 194377'18645 (194377'18644) modify a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by client.2231190.0:896 2014-08-12 20:33:00.907409 2014-08-13 18:06:06.595357 7f5c973a57a0 20 read_log 194377'18646 (0'0) promote cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by osd.20.0:4496 2014-08-12 20:33:02.030736 2014-08-13 18:06:06.595370 7f5c973a57a0 20 read_log 194377'18647 (194377'18646) modify cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by client.2231190.0:1192 2014-08-12 20:33:02.023014 2014-08-13 18:06:06.595402 7f5c973a57a0 20 read_log 194377'18648 (0'0) promote 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by osd.20.0:4504 2014-08-12 20:33:03.021960 2014-08-13 18:06:06.595416 7f5c973a57a0 20 read_log 194377'18649 (194377'18648) modify 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by client.2231190.0:1453 2014-08-12 20:33:03.014348 2014-08-13 18:06:06.595442 7f5c973a57a0 20 read_log 194377'18650 (0'0) promote 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by osd.20.0:4514 2014-08-12 20:33:06.215836 2014-08-13 18:06:06.595455 7f5c973a57a0 20 read_log 194377'18651 (194377'18650) modify 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by client.2231190.0:2241 2014-08-12 20:33:06.209363 2014-08-13 18:06:06.595467 7f5c973a57a0 20 read_log 194377'18652 (0'0) promote cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by osd.20.0:4536 2014-08-12 20:33:08.897041 2014-08-13 18:06:06.595482 7f5c973a57a0 20 read_log 194377'18653 (194377'18652) modify cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by client.2231190.0:2905 2014-08-12 20:33:08.890178 2014-08-13 18:06:06.595494 7f5c973a57a0 20 read_log 194377'18654 (0'0) promote 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by osd.20.0:4540 2014-08-12 20:33:09.682593 2014-08-13 18:06:06.595517 7f5c973a57a0 20 read_log 194377'18655 (194377'18654) modify 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by client.2231190.0:3092 2014-08-12 20:33:09.673777 2014-08-13 18:06:06.595540 7f5c973a57a0 20 read_log 194377'18656 (0'0) promote 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by osd.20.0:4558 2014-08-12 20:33:14.126715 2014-08-13 18:06:06.595556 7f5c973a57a0 20 read_log 194377'18657 (194377'18656) modify 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by client.2231190.0:4036 2014-08-12 20:33:14.118216 2014-08-13 18:06:06.595569 7f5c973a57a0 20 read_log 194377'18658 (0'0) promote 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by osd.20.0:4576 2014-08-12 20:33:18.891187 2014-08-13 18:06:06.595588 7f5c973a57a0 20 read_log 194377'18659 (194377'18658) modify 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by client.2231190.0:4736 2014-08-12 20:33:18.883461 2014-08-13 18:06:06.595603 7f5c973a57a0 20 read_log 194383'18660 (0'0) promote a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by osd.20.0:4614 2014-08-12 20:34:03.570988 2014-08-13 18:06:06.595615 7f5c973a57a0 20 read_log 194383'18661 (194383'18660) modify a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by client.2231190.0:6394 2014-08-12 20:34:03.564216 2014-08-13 18:06:06.595627 7f5c973a57a0 20 read_log 194383'18662 (0'0) promote abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by osd.20.0:4620 2014-08-12 20:34:17.976519 2014-08-13 18:06:06.595640 7f5c973a57a0 20 read_log 194383'18663 (194383'18662) modify abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by client.2231190.0:7104 2014-08-12 20:34:17.967809 2014-08-13 18:06:06.595652 7f5c973a57a0 20 read_log 194393'18664 (194393'18664) modify 3d/hit_set_24.3d_archive_2014-08-12 20:37:08.991137_2014-08-12 20:47:27.683414/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.684567 2014-08-13 18:06:06.595664 7f5c973a57a0 20 read_log 194393'18665 (194215'18610) delete 3d/hit_set_24.3d_archive_2014-08-12 15:32:25.863067_2014-08-12 17:24:40.629052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.684567 2014-08-13 18:06:06.595689 7f5c973a57a0 20 read_log 194393'18666 (0'0) promote 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by osd.20.0:4648 2014-08-12 20:47:27.689016 2014-08-13 18:06:06.595705 7f5c973a57a0 20 read_log 194393'18667 (194393'18666) modify 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by client.1064095.0:409 2014-08-12 20:47:27.678974 2014-08-13 18:06:06.595717 7f5c973a57a0 20 read_log 194393'18668 (0'0) promote 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by osd.20.0:4653 2014-08-12 20:47:28.148894 2014-08-13 18:06:06.595741 7f5c973a57a0 20 read_log 194393'18669 (194393'18668) modify 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by client.1064095.0:516 2014-08-12 20:47:28.139439 2014-08-13 18:06:06.595756 7f5c973a57a0 20 read_log 194393'18670 (0'0) promote d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by osd.20.0:4673 2014-08-12 20:49:26.743759 2014-08-13 18:06:06.595772 7f5c973a57a0 20 read_log 194393'18671 (194393'18670) modify d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by client.1064182.0:270 2014-08-12 20:49:26.734751 2014-08-13 18:06:06.595785 7f5c973a57a0 20 read_log 194416'18672 (194416'18672) modify 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 18:06:06.595798 7f5c973a57a0 20 read_log 194416'18673 (194393'18664) delete 3d/hit_set_24.3d_archive_2014-08-12 20:37:08.991137_2014-08-12 20:47:27.683414/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 18:06:06.595820 7f5c973a57a0 20 read_log 194416'18674 (0'0) promote 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by osd.20.0:4862 2014-08-13 15:28:30.882411 2014-08-13 18:06:06.595844 7f5c973a57a0 20 read_log 194416'18675 (194416'18674) modify 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by client.2441958.0:179 2014-08-13 15:28:30.869401 2014-08-13 18:06:06.595860 7f5c973a57a0 20 read_log 194416'18676 (0'0) promote 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by osd.20.0:4867 2014-08-13 15:28:31.482821 2014-08-13 18:06:06.595873 7f5c973a57a0 20 read_log 194416'18677 (194416'18676) modify 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by client.2441958.0:306 2014-08-13 15:28:31.473163 2014-08-13 18:06:06.595885 7f5c973a57a0 20 read_log 194416'18678 (0'0) promote d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by osd.20.0:4927 2014-08-13 15:28:39.112470 2014-08-13 18:06:06.595897 7f5c973a57a0 20 read_log 194416'18679 (194416'18678) modify d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by client.2441958.0:2221 2014-08-13 15:28:39.104322 2014-08-13 18:06:06.595910 7f5c973a57a0 20 read_log 194416'18680 (0'0) promote 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by osd.20.0:4997 2014-08-13 15:28:51.030775 2014-08-13 18:06:06.595922 7f5c973a57a0 20 read_log 194416'18681 (194416'18680) modify 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by client.2441958.0:5296 2014-08-13 15:28:51.021478 2014-08-13 18:06:06.595934 7f5c973a57a0 20 read_log 194416'18682 (0'0) promote 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by osd.20.0:5007 2014-08-13 15:28:52.934138 2014-08-13 18:06:06.595946 7f5c973a57a0 20 read_log 194416'18683 (194416'18682) modify 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by client.2441958.0:5788 2014-08-13 15:28:52.927293 2014-08-13 18:06:06.595958 7f5c973a57a0 20 read_log 194416'18684 (0'0) promote b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by osd.20.0:5027 2014-08-13 15:28:56.474887 2014-08-13 18:06:06.595970 7f5c973a57a0 20 read_log 194416'18685 (194416'18684) modify b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by client.2441958.0:6649 2014-08-13 15:28:56.466596 2014-08-13 18:06:06.595982 7f5c973a57a0 20 read_log 194416'18686 (0'0) promote edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by osd.20.0:5041 2014-08-13 15:28:59.609677 2014-08-13 18:06:06.595994 7f5c973a57a0 20 read_log 194416'18687 (194416'18686) modify edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by client.2441958.0:7488 2014-08-13 15:28:59.601755 2014-08-13 18:06:06.596006 7f5c973a57a0 20 read_log 194416'18688 (0'0) promote c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by osd.20.0:5057 2014-08-13 15:29:02.620130 2014-08-13 18:06:06.596018 7f5c973a57a0 20 read_log 194416'18689 (194416'18688) modify c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by client.2441958.0:8290 2014-08-13 15:29:02.611334 2014-08-13 18:06:06.596034 7f5c973a57a0 20 read_log 194416'18690 (0'0) promote 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by osd.20.0:5065 2014-08-13 15:29:03.156511 2014-08-13 18:06:06.596047 7f5c973a57a0 20 read_log 194416'18691 (194416'18690) modify 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by client.2441958.0:8429 2014-08-13 15:29:03.146674 2014-08-13 18:06:06.596059 7f5c973a57a0 20 read_log 194416'18692 (0'0) promote 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by osd.20.0:5093 2014-08-13 15:29:08.855277 2014-08-13 18:06:06.596071 7f5c973a57a0 20 read_log 194416'18693 (194416'18692) modify 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by client.2441958.0:9868 2014-08-13 15:29:08.847037 2014-08-13 18:06:06.596090 7f5c973a57a0 20 read_log 194416'18694 (0'0) promote ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by osd.20.0:5121 2014-08-13 15:29:12.011260 2014-08-13 18:06:06.596106 7f5c973a57a0 20 read_log 194416'18695 (194416'18694) modify ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by client.2441958.0:10653 2014-08-13 15:29:12.001679 2014-08-13 18:06:06.596118 7f5c973a57a0 20 read_log 194416'18696 (0'0) promote 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by osd.20.0:5123 2014-08-13 15:29:12.325346 2014-08-13 18:06:06.596131 7f5c973a57a0 20 read_log 194416'18697 (194416'18696) modify 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by client.2441958.0:10738 2014-08-13 15:29:12.316890 2014-08-13 18:06:06.596143 7f5c973a57a0 20 read_log 194416'18698 (0'0) promote ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by osd.20.0:5127 2014-08-13 15:29:12.981978 2014-08-13 18:06:06.596182 7f5c973a57a0 20 read_log 194416'18699 (194416'18698) modify ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by client.2441958.0:10908 2014-08-13 15:29:12.972969 2014-08-13 18:06:06.596198 7f5c973a57a0 20 read_log 194416'18700 (0'0) promote 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by osd.20.0:5147 2014-08-13 15:29:18.329536 2014-08-13 18:06:06.596211 7f5c973a57a0 20 read_log 194416'18701 (194416'18700) modify 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by client.2441958.0:12160 2014-08-13 15:29:18.318594 2014-08-13 18:06:06.596223 7f5c973a57a0 20 read_log 194416'18702 (0'0) promote 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by osd.20.0:5169 2014-08-13 15:29:22.749406 2014-08-13 18:06:06.596238 7f5c973a57a0 20 read_log 194416'18703 (194416'18702) modify 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by client.2441958.0:13182 2014-08-13 15:29:22.740393 2014-08-13 18:06:06.596251 7f5c973a57a0 20 read_log 194416'18704 (0'0) promote 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by osd.20.0:5177 2014-08-13 15:29:24.831349 2014-08-13 18:06:06.596268 7f5c973a57a0 20 read_log 194416'18705 (194416'18704) modify 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by client.2441958.0:13660 2014-08-13 15:29:24.821470 2014-08-13 18:06:06.596284 7f5c973a57a0 20 read_log 194416'18706 (0'0) promote 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by osd.20.0:5203 2014-08-13 15:29:29.423198 2014-08-13 18:06:06.596296 7f5c973a57a0 20 read_log 194416'18707 (194416'18706) modify 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by client.2441958.0:14808 2014-08-13 15:29:29.411886 2014-08-13 18:06:06.596309 7f5c973a57a0 20 read_log 194416'18708 (0'0) promote ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by osd.20.0:5229 2014-08-13 15:29:35.594271 2014-08-13 18:06:06.596321 7f5c973a57a0 20 read_log 194416'18709 (194416'18708) modify ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by client.2441958.0:16003 2014-08-13 15:29:35.584326 2014-08-13 18:06:06.596341 7f5c973a57a0 20 read_log 194416'18710 (0'0) promote 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by osd.20.0:5237 2014-08-13 15:29:40.938939 2014-08-13 18:06:06.596389 7f5c973a57a0 20 read_log 194416'18711 (194416'18710) modify 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by client.2441958.0:16550 2014-08-13 15:29:40.931938 2014-08-13 18:06:06.596406 7f5c973a57a0 20 read_log 194416'18712 (0'0) promote aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by osd.20.0:5253 2014-08-13 15:29:49.069530 2014-08-13 18:06:06.596429 7f5c973a57a0 20 read_log 194416'18713 (194416'18712) modify aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by client.2441958.0:17121 2014-08-13 15:29:49.061090 2014-08-13 18:06:06.596450 7f5c973a57a0 20 read_log 194416'18714 (0'0) promote 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by osd.20.0:5279 2014-08-13 15:29:57.943443 2014-08-13 18:06:06.596473 7f5c973a57a0 20 read_log 194416'18715 (194416'18714) modify 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by client.2441958.0:17752 2014-08-13 15:29:57.934316 2014-08-13 18:06:06.596488 7f5c973a57a0 20 read_log 194422'18716 (0'0) promote 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by osd.20.0:5295 2014-08-13 15:32:15.015699 2014-08-13 18:06:06.596501 7f5c973a57a0 20 read_log 194422'18717 (194422'18716) modify 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by client.1148203.0:34 2014-08-13 15:32:15.007722 2014-08-13 18:06:06.596514 7f5c973a57a0 20 read_log 194422'18718 (0'0) promote 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by osd.20.0:5331 2014-08-13 15:32:20.755966 2014-08-13 18:06:06.596526 7f5c973a57a0 20 read_log 194422'18719 (194422'18718) modify 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by client.1148203.0:1368 2014-08-13 15:32:20.746738 2014-08-13 18:06:06.596538 7f5c973a57a0 20 read_log 194422'18720 (0'0) promote f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by osd.20.0:5349 2014-08-13 15:32:24.537306 2014-08-13 18:06:06.596554 7f5c973a57a0 20 read_log 194422'18721 (194422'18720) modify f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by client.1148203.0:2339 2014-08-13 15:32:24.525408 2014-08-13 18:06:06.596567 7f5c973a57a0 20 read_log 194422'18722 (0'0) promote 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by osd.20.0:5367 2014-08-13 15:32:29.056340 2014-08-13 18:06:06.596579 7f5c973a57a0 20 read_log 194422'18723 (194422'18722) modify 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by client.1148203.0:3490 2014-08-13 15:32:29.047446 2014-08-13 18:06:06.596591 7f5c973a57a0 20 read_log 194422'18724 (0'0) promote e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by osd.20.0:5425 2014-08-13 15:32:40.724564 2014-08-13 18:06:06.596603 7f5c973a57a0 20 read_log 194422'18725 (194422'18724) modify e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by client.1148203.0:6106 2014-08-13 15:32:40.713670 2014-08-13 18:06:06.596615 7f5c973a57a0 20 read_log 194422'18726 (0'0) promote 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by osd.20.0:5427 2014-08-13 15:32:42.428687 2014-08-13 18:06:06.596628 7f5c973a57a0 20 read_log 194422'18727 (194422'18726) modify 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by client.1148203.0:6422 2014-08-13 15:32:42.419805 2014-08-13 18:06:06.596640 7f5c973a57a0 20 read_log 194422'18728 (0'0) promote 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by osd.20.0:5469 2014-08-13 15:32:59.205436 2014-08-13 18:06:06.596652 7f5c973a57a0 20 read_log 194422'18729 (194422'18728) modify 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by client.1148203.0:8962 2014-08-13 15:32:59.195194 2014-08-13 18:06:06.596664 7f5c973a57a0 20 read_log 194422'18730 (0'0) promote 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by osd.20.0:5505 2014-08-13 15:33:26.090665 2014-08-13 18:06:06.596685 7f5c973a57a0 20 read_log 194422'18731 (194422'18730) modify 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by client.1148203.0:10577 2014-08-13 15:33:26.082626 2014-08-13 18:06:06.597431 7f5c973a57a0 10 read_log checking for missing items over interval (194370'18616,194422'18731] 2014-08-13 18:06:06.597487 7f5c973a57a0 15 read_log missing 194422'18731 (194422'18730) modify 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 by client.1148203.0:10577 2014-08-13 15:33:26.082626 2014-08-13 18:06:06.597533 7f5c973a57a0 15 read_log missing 194422'18729 (194422'18728) modify 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 by client.1148203.0:8962 2014-08-13 15:32:59.195194 2014-08-13 18:06:06.597574 7f5c973a57a0 15 read_log missing 194422'18727 (194422'18726) modify 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 by client.1148203.0:6422 2014-08-13 15:32:42.419805 2014-08-13 18:06:06.597615 7f5c973a57a0 15 read_log missing 194422'18725 (194422'18724) modify e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 by client.1148203.0:6106 2014-08-13 15:32:40.713670 2014-08-13 18:06:06.597655 7f5c973a57a0 15 read_log missing 194422'18723 (194422'18722) modify 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 by client.1148203.0:3490 2014-08-13 15:32:29.047446 2014-08-13 18:06:06.597695 7f5c973a57a0 15 read_log missing 194422'18721 (194422'18720) modify f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 by client.1148203.0:2339 2014-08-13 15:32:24.525408 2014-08-13 18:06:06.597735 7f5c973a57a0 15 read_log missing 194422'18719 (194422'18718) modify 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 by client.1148203.0:1368 2014-08-13 15:32:20.746738 2014-08-13 18:06:06.597774 7f5c973a57a0 15 read_log missing 194422'18717 (194422'18716) modify 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 by client.1148203.0:34 2014-08-13 15:32:15.007722 2014-08-13 18:06:06.597815 7f5c973a57a0 15 read_log missing 194416'18715 (194416'18714) modify 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 by client.2441958.0:17752 2014-08-13 15:29:57.934316 2014-08-13 18:06:06.597855 7f5c973a57a0 15 read_log missing 194416'18713 (194416'18712) modify aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 by client.2441958.0:17121 2014-08-13 15:29:49.061090 2014-08-13 18:06:06.597895 7f5c973a57a0 15 read_log missing 194416'18711 (194416'18710) modify 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 by client.2441958.0:16550 2014-08-13 15:29:40.931938 2014-08-13 18:06:06.597935 7f5c973a57a0 15 read_log missing 194416'18709 (194416'18708) modify ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 by client.2441958.0:16003 2014-08-13 15:29:35.584326 2014-08-13 18:06:06.597975 7f5c973a57a0 15 read_log missing 194416'18707 (194416'18706) modify 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 by client.2441958.0:14808 2014-08-13 15:29:29.411886 2014-08-13 18:06:06.598015 7f5c973a57a0 15 read_log missing 194416'18705 (194416'18704) modify 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 by client.2441958.0:13660 2014-08-13 15:29:24.821470 2014-08-13 18:06:06.598055 7f5c973a57a0 15 read_log missing 194416'18703 (194416'18702) modify 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 by client.2441958.0:13182 2014-08-13 15:29:22.740393 2014-08-13 18:06:06.598095 7f5c973a57a0 15 read_log missing 194416'18701 (194416'18700) modify 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 by client.2441958.0:12160 2014-08-13 15:29:18.318594 2014-08-13 18:06:06.598135 7f5c973a57a0 15 read_log missing 194416'18699 (194416'18698) modify ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 by client.2441958.0:10908 2014-08-13 15:29:12.972969 2014-08-13 18:06:06.598182 7f5c973a57a0 15 read_log missing 194416'18697 (194416'18696) modify 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 by client.2441958.0:10738 2014-08-13 15:29:12.316890 2014-08-13 18:06:06.598223 7f5c973a57a0 15 read_log missing 194416'18695 (194416'18694) modify ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 by client.2441958.0:10653 2014-08-13 15:29:12.001679 2014-08-13 18:06:06.598267 7f5c973a57a0 15 read_log missing 194416'18693 (194416'18692) modify 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 by client.2441958.0:9868 2014-08-13 15:29:08.847037 2014-08-13 18:06:06.598312 7f5c973a57a0 15 read_log missing 194416'18691 (194416'18690) modify 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 by client.2441958.0:8429 2014-08-13 15:29:03.146674 2014-08-13 18:06:06.598363 7f5c973a57a0 15 read_log missing 194416'18689 (194416'18688) modify c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 by client.2441958.0:8290 2014-08-13 15:29:02.611334 2014-08-13 18:06:06.598415 7f5c973a57a0 15 read_log missing 194416'18687 (194416'18686) modify edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 by client.2441958.0:7488 2014-08-13 15:28:59.601755 2014-08-13 18:06:06.598457 7f5c973a57a0 15 read_log missing 194416'18685 (194416'18684) modify b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 by client.2441958.0:6649 2014-08-13 15:28:56.466596 2014-08-13 18:06:06.598498 7f5c973a57a0 15 read_log missing 194416'18683 (194416'18682) modify 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 by client.2441958.0:5788 2014-08-13 15:28:52.927293 2014-08-13 18:06:06.598538 7f5c973a57a0 15 read_log missing 194416'18681 (194416'18680) modify 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 by client.2441958.0:5296 2014-08-13 15:28:51.021478 2014-08-13 18:06:06.598578 7f5c973a57a0 15 read_log missing 194416'18679 (194416'18678) modify d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 by client.2441958.0:2221 2014-08-13 15:28:39.104322 2014-08-13 18:06:06.598619 7f5c973a57a0 15 read_log missing 194416'18677 (194416'18676) modify 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 by client.2441958.0:306 2014-08-13 15:28:31.473163 2014-08-13 18:06:06.598660 7f5c973a57a0 15 read_log missing 194416'18675 (194416'18674) modify 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 by client.2441958.0:179 2014-08-13 15:28:30.869401 2014-08-13 18:06:06.598704 7f5c973a57a0 15 read_log missing 194416'18672 (194416'18672) modify 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:30.876509 2014-08-13 18:06:06.598741 7f5c973a57a0 15 read_log missing 194393'18671 (194393'18670) modify d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 by client.1064182.0:270 2014-08-12 20:49:26.734751 2014-08-13 18:06:06.598781 7f5c973a57a0 15 read_log missing 194393'18669 (194393'18668) modify 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 by client.1064095.0:516 2014-08-12 20:47:28.139439 2014-08-13 18:06:06.598822 7f5c973a57a0 15 read_log missing 194393'18667 (194393'18666) modify 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 by client.1064095.0:409 2014-08-12 20:47:27.678974 2014-08-13 18:06:06.598865 7f5c973a57a0 15 read_log missing 194383'18663 (194383'18662) modify abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 by client.2231190.0:7104 2014-08-12 20:34:17.967809 2014-08-13 18:06:06.598905 7f5c973a57a0 15 read_log missing 194383'18661 (194383'18660) modify a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 by client.2231190.0:6394 2014-08-12 20:34:03.564216 2014-08-13 18:06:06.598946 7f5c973a57a0 15 read_log missing 194377'18659 (194377'18658) modify 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 by client.2231190.0:4736 2014-08-12 20:33:18.883461 2014-08-13 18:06:06.598986 7f5c973a57a0 15 read_log missing 194377'18657 (194377'18656) modify 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 by client.2231190.0:4036 2014-08-12 20:33:14.118216 2014-08-13 18:06:06.599026 7f5c973a57a0 15 read_log missing 194377'18655 (194377'18654) modify 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 by client.2231190.0:3092 2014-08-12 20:33:09.673777 2014-08-13 18:06:06.599067 7f5c973a57a0 15 read_log missing 194377'18653 (194377'18652) modify cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 by client.2231190.0:2905 2014-08-12 20:33:08.890178 2014-08-13 18:06:06.599107 7f5c973a57a0 15 read_log missing 194377'18651 (194377'18650) modify 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 by client.2231190.0:2241 2014-08-12 20:33:06.209363 2014-08-13 18:06:06.599152 7f5c973a57a0 15 read_log missing 194377'18649 (194377'18648) modify 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 by client.2231190.0:1453 2014-08-12 20:33:03.014348 2014-08-13 18:06:06.599194 7f5c973a57a0 15 read_log missing 194377'18647 (194377'18646) modify cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 by client.2231190.0:1192 2014-08-12 20:33:02.023014 2014-08-13 18:06:06.599235 7f5c973a57a0 15 read_log missing 194377'18645 (194377'18644) modify a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 by client.2231190.0:896 2014-08-12 20:33:00.907409 2014-08-13 18:06:06.599278 7f5c973a57a0 15 read_log missing 194377'18643 (194377'18642) modify 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 by client.2231190.0:204 2014-08-12 20:32:58.004900 2014-08-13 18:06:06.599319 7f5c973a57a0 15 read_log missing 194370'18641 (194370'18640) modify 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 by client.1508531.0:13314 2014-08-12 20:31:22.374937 2014-08-13 18:06:06.599360 7f5c973a57a0 15 read_log missing 194370'18639 (194370'18638) modify 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 by client.1508531.0:13048 2014-08-12 20:31:19.800706 2014-08-13 18:06:06.599416 7f5c973a57a0 15 read_log missing 194370'18637 (194370'18636) modify 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 by client.1508531.0:12968 2014-08-12 20:31:19.055563 2014-08-13 18:06:06.599468 7f5c973a57a0 15 read_log missing 194370'18635 (194370'18634) modify d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 by client.1508531.0:10941 2014-08-12 20:31:04.914356 2014-08-13 18:06:06.599511 7f5c973a57a0 15 read_log missing 194370'18633 (194370'18632) modify fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 by client.1508531.0:10288 2014-08-12 20:31:00.946368 2014-08-13 18:06:06.599552 7f5c973a57a0 15 read_log missing 194370'18631 (194370'18630) modify 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 by client.1508531.0:10152 2014-08-12 20:31:00.138476 2014-08-13 18:06:06.599592 7f5c973a57a0 15 read_log missing 194370'18629 (194370'18628) modify f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 by client.1508531.0:9659 2014-08-12 20:30:57.255203 2014-08-13 18:06:06.599632 7f5c973a57a0 15 read_log missing 194370'18627 (194370'18626) modify cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 by client.1508531.0:9310 2014-08-12 20:30:55.167609 2014-08-13 18:06:06.599672 7f5c973a57a0 15 read_log missing 194370'18625 (194370'18624) modify fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 by client.1508531.0:8648 2014-08-12 20:30:51.141855 2014-08-13 18:06:06.599713 7f5c973a57a0 15 read_log missing 194370'18623 (194370'18622) modify 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 by client.1508531.0:6728 2014-08-12 20:30:40.992116 2014-08-13 18:06:06.599781 7f5c973a57a0 15 read_log missing 194370'18621 (194370'18620) modify 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 by client.1508531.0:4127 2014-08-12 20:30:29.751699 2014-08-13 18:06:06.599835 7f5c973a57a0 15 read_log missing 194370'18619 (194370'18618) modify cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 by client.1508531.0:3488 2014-08-12 20:30:27.359030 2014-08-13 18:06:06.599908 7f5c973a57a0 15 read_log missing 194370'18617 (194370'18616) modify 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 by client.1508531.0:1616 2014-08-12 20:30:20.430889 2014-08-13 18:06:06.599925 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.599974 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_loaded 2014-08-13 18:06:06.599995 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Initial 0.024255 0 0.000000 2014-08-13 18:06:06.600012 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset 2014-08-13 18:06:06.600021 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=0 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194487 2014-08-13 18:06:06.600039 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 18:06:06.600052 7f5c973a57a0 10 osd.4 194487 pgid 24.5e coll 24.5e_head 2014-08-13 18:06:06.600256 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.5e 2014-08-13 18:06:06.600270 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.600303 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.5e(unlocked)] enter Initial 2014-08-13 18:06:06.600315 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.5e(unlocked)] enter NotTrimming 2014-08-13 18:06:06.600516 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.600651 7f5c973a57a0 20 read_log 192718'14373 (192718'13737) delete 5e/hit_set_24.5e_archive_2014-08-06 14:24:01.267758_2014-08-06 14:50:06.074726/head/.ceph-internal/24 by unknown.0.0:0 2014-08-06 15:00:54.701437 2014-08-13 18:06:06.600674 7f5c973a57a0 20 read_log 192803'14718 (192803'14718) modify 5e/hit_set_24.5e_archive_2014-08-11 10:24:43.601998_2014-08-11 12:48:05.517931/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:48:05.518598 2014-08-13 18:06:06.600687 7f5c973a57a0 20 read_log 192803'14719 (192718'14372) delete 5e/hit_set_24.5e_archive_2014-08-06 14:50:06.074945_2014-08-06 15:00:54.701195/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 12:48:05.518598 2014-08-13 18:06:06.600755 7f5c973a57a0 20 read_log 192803'15090 (0'0) promote a181605e/core.46996+10+2/head//24 by osd.289.0:51828 2014-08-11 12:51:33.017661 2014-08-13 18:06:06.600773 7f5c973a57a0 20 read_log 192803'15091 (192803'15090) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:1 2014-08-11 12:51:28.148875 2014-08-13 18:06:06.600786 7f5c973a57a0 20 read_log 192803'15092 (192803'15091) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:2 2014-08-11 12:51:33.930260 2014-08-13 18:06:06.600798 7f5c973a57a0 20 read_log 192803'15093 (192803'15092) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:3 2014-08-11 12:51:34.383482 2014-08-13 18:06:06.600810 7f5c973a57a0 20 read_log 192803'15094 (192803'15093) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:4 2014-08-11 12:51:34.413145 2014-08-13 18:06:06.600822 7f5c973a57a0 20 read_log 192803'15095 (192803'15094) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:5 2014-08-11 12:51:34.471494 2014-08-13 18:06:06.600834 7f5c973a57a0 20 read_log 192803'15096 (192803'15095) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:6 2014-08-11 12:51:34.499111 2014-08-13 18:06:06.600846 7f5c973a57a0 20 read_log 192803'15097 (192803'15096) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:7 2014-08-11 12:51:34.556731 2014-08-13 18:06:06.600858 7f5c973a57a0 20 read_log 192803'15098 (192803'15097) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:8 2014-08-11 12:51:34.583284 2014-08-13 18:06:06.600870 7f5c973a57a0 20 read_log 192803'15099 (192803'15098) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:9 2014-08-11 12:51:34.652838 2014-08-13 18:06:06.600882 7f5c973a57a0 20 read_log 192803'15100 (192803'15099) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:10 2014-08-11 12:51:34.679367 2014-08-13 18:06:06.600894 7f5c973a57a0 20 read_log 192803'15101 (192803'15100) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:11 2014-08-11 12:51:34.756048 2014-08-13 18:06:06.600906 7f5c973a57a0 20 read_log 192803'15102 (192803'15101) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:12 2014-08-11 12:51:34.886668 2014-08-13 18:06:06.600918 7f5c973a57a0 20 read_log 192803'15103 (192803'15102) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:13 2014-08-11 12:51:34.955906 2014-08-13 18:06:06.600930 7f5c973a57a0 20 read_log 192803'15104 (192803'15103) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:14 2014-08-11 12:51:34.986833 2014-08-13 18:06:06.600942 7f5c973a57a0 20 read_log 192803'15105 (192803'15104) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:15 2014-08-11 12:51:35.108219 2014-08-13 18:06:06.600954 7f5c973a57a0 20 read_log 192803'15106 (192803'15105) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:16 2014-08-11 12:51:35.129258 2014-08-13 18:06:06.600966 7f5c973a57a0 20 read_log 192803'15107 (192803'15106) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:17 2014-08-11 12:51:35.152078 2014-08-13 18:06:06.600978 7f5c973a57a0 20 read_log 192803'15108 (192803'15107) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:18 2014-08-11 12:51:35.191769 2014-08-13 18:06:06.600990 7f5c973a57a0 20 read_log 192803'15109 (192803'15108) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:19 2014-08-11 12:51:35.216401 2014-08-13 18:06:06.601002 7f5c973a57a0 20 read_log 192803'15110 (192803'15109) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:20 2014-08-11 12:51:35.240751 2014-08-13 18:06:06.601014 7f5c973a57a0 20 read_log 192803'15111 (192803'15110) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:21 2014-08-11 12:51:35.298213 2014-08-13 18:06:06.601026 7f5c973a57a0 20 read_log 192803'15112 (192803'15111) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:22 2014-08-11 12:51:35.718802 2014-08-13 18:06:06.601038 7f5c973a57a0 20 read_log 192803'15113 (192803'15112) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:23 2014-08-11 12:51:35.796988 2014-08-13 18:06:06.601057 7f5c973a57a0 20 read_log 192803'15114 (192803'15113) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:24 2014-08-11 12:51:35.823828 2014-08-13 18:06:06.601069 7f5c973a57a0 20 read_log 192803'15115 (192803'15114) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:25 2014-08-11 12:51:35.862002 2014-08-13 18:06:06.601081 7f5c973a57a0 20 read_log 192803'15116 (192803'15115) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:26 2014-08-11 12:51:35.908241 2014-08-13 18:06:06.601093 7f5c973a57a0 20 read_log 192803'15117 (192803'15116) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:27 2014-08-11 12:51:35.954386 2014-08-13 18:06:06.601105 7f5c973a57a0 20 read_log 192803'15118 (192803'15117) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:28 2014-08-11 12:51:36.000446 2014-08-13 18:06:06.601117 7f5c973a57a0 20 read_log 192803'15119 (192803'15118) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:29 2014-08-11 12:51:36.046029 2014-08-13 18:06:06.601130 7f5c973a57a0 20 read_log 192803'15120 (192803'15119) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:30 2014-08-11 12:51:36.089736 2014-08-13 18:06:06.601142 7f5c973a57a0 20 read_log 192803'15121 (192803'15120) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:31 2014-08-11 12:51:36.148976 2014-08-13 18:06:06.601185 7f5c973a57a0 20 read_log 192803'15122 (192803'15121) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:32 2014-08-11 12:51:36.666026 2014-08-13 18:06:06.601200 7f5c973a57a0 20 read_log 192803'15123 (192803'15122) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:33 2014-08-11 12:51:36.712683 2014-08-13 18:06:06.601213 7f5c973a57a0 20 read_log 192803'15124 (192803'15123) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:34 2014-08-11 12:51:36.759756 2014-08-13 18:06:06.601232 7f5c973a57a0 20 read_log 192803'15125 (192803'15124) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:35 2014-08-11 12:51:36.806560 2014-08-13 18:06:06.601247 7f5c973a57a0 20 read_log 192803'15126 (192803'15125) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:36 2014-08-11 12:51:36.859590 2014-08-13 18:06:06.601262 7f5c973a57a0 20 read_log 192803'15127 (192803'15126) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:37 2014-08-11 12:51:36.906385 2014-08-13 18:06:06.601275 7f5c973a57a0 20 read_log 192803'15128 (192803'15127) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:38 2014-08-11 12:51:36.954201 2014-08-13 18:06:06.601287 7f5c973a57a0 20 read_log 192803'15129 (192803'15128) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:39 2014-08-11 12:51:37.081535 2014-08-13 18:06:06.601299 7f5c973a57a0 20 read_log 192803'15130 (192803'15129) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:40 2014-08-11 12:51:37.230490 2014-08-13 18:06:06.601311 7f5c973a57a0 20 read_log 192803'15131 (192803'15130) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:41 2014-08-11 12:51:37.388095 2014-08-13 18:06:06.601323 7f5c973a57a0 20 read_log 192803'15132 (192803'15131) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:42 2014-08-11 12:51:37.594120 2014-08-13 18:06:06.601335 7f5c973a57a0 20 read_log 192803'15133 (192803'15132) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:43 2014-08-11 12:51:37.651853 2014-08-13 18:06:06.601352 7f5c973a57a0 20 read_log 192803'15134 (192803'15133) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:44 2014-08-11 12:51:37.695980 2014-08-13 18:06:06.601365 7f5c973a57a0 20 read_log 192803'15135 (192803'15134) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:45 2014-08-11 12:51:37.745008 2014-08-13 18:06:06.601377 7f5c973a57a0 20 read_log 192803'15136 (192803'15135) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:46 2014-08-11 12:51:37.795263 2014-08-13 18:06:06.601393 7f5c973a57a0 20 read_log 192803'15137 (192803'15136) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:47 2014-08-11 12:51:37.842348 2014-08-13 18:06:06.601428 7f5c973a57a0 20 read_log 192803'15138 (192803'15137) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:48 2014-08-11 12:51:37.883945 2014-08-13 18:06:06.601445 7f5c973a57a0 20 read_log 192803'15139 (192803'15138) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:49 2014-08-11 12:51:37.927824 2014-08-13 18:06:06.601472 7f5c973a57a0 20 read_log 192803'15140 (192803'15139) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:50 2014-08-11 12:51:37.977290 2014-08-13 18:06:06.601485 7f5c973a57a0 20 read_log 192803'15141 (192803'15140) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:51 2014-08-11 12:51:38.027134 2014-08-13 18:06:06.601508 7f5c973a57a0 20 read_log 192803'15142 (192803'15141) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:52 2014-08-11 12:51:38.075175 2014-08-13 18:06:06.601525 7f5c973a57a0 20 read_log 192803'15143 (192803'15142) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:53 2014-08-11 12:51:38.567081 2014-08-13 18:06:06.601537 7f5c973a57a0 20 read_log 192803'15144 (192803'15143) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:54 2014-08-11 12:51:38.618119 2014-08-13 18:06:06.601550 7f5c973a57a0 20 read_log 192803'15145 (192803'15144) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:55 2014-08-11 12:51:38.667519 2014-08-13 18:06:06.601564 7f5c973a57a0 20 read_log 192803'15146 (192803'15145) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:56 2014-08-11 12:51:38.717249 2014-08-13 18:06:06.601576 7f5c973a57a0 20 read_log 192803'15147 (192803'15146) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:57 2014-08-11 12:51:38.767026 2014-08-13 18:06:06.601589 7f5c973a57a0 20 read_log 192803'15148 (192803'15147) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:58 2014-08-11 12:51:38.831233 2014-08-13 18:06:06.601601 7f5c973a57a0 20 read_log 192803'15149 (192803'15148) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:59 2014-08-11 12:51:38.858206 2014-08-13 18:06:06.601613 7f5c973a57a0 20 read_log 192803'15150 (192803'15149) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:60 2014-08-11 12:51:38.891154 2014-08-13 18:06:06.601625 7f5c973a57a0 20 read_log 192803'15151 (192803'15150) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:61 2014-08-11 12:51:38.916702 2014-08-13 18:06:06.601645 7f5c973a57a0 20 read_log 192803'15152 (192803'15151) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:62 2014-08-11 12:51:38.946230 2014-08-13 18:06:06.601658 7f5c973a57a0 20 read_log 192803'15153 (192803'15152) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:63 2014-08-11 12:51:39.455207 2014-08-13 18:06:06.601696 7f5c973a57a0 20 read_log 192803'15154 (192803'15153) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:64 2014-08-11 12:51:39.538286 2014-08-13 18:06:06.601713 7f5c973a57a0 20 read_log 192803'15155 (192803'15154) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:65 2014-08-11 12:51:39.623434 2014-08-13 18:06:06.601725 7f5c973a57a0 20 read_log 192803'15156 (192803'15155) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:66 2014-08-11 12:51:39.705746 2014-08-13 18:06:06.601738 7f5c973a57a0 20 read_log 192803'15157 (192803'15156) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:67 2014-08-11 12:51:39.783124 2014-08-13 18:06:06.601750 7f5c973a57a0 20 read_log 192803'15158 (192803'15157) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:68 2014-08-11 12:51:39.874953 2014-08-13 18:06:06.601762 7f5c973a57a0 20 read_log 192803'15159 (192803'15158) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:69 2014-08-11 12:51:39.955526 2014-08-13 18:06:06.601774 7f5c973a57a0 20 read_log 192803'15160 (192803'15159) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:70 2014-08-11 12:51:40.040125 2014-08-13 18:06:06.601786 7f5c973a57a0 20 read_log 192803'15161 (192803'15160) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:71 2014-08-11 12:51:40.117788 2014-08-13 18:06:06.601806 7f5c973a57a0 20 read_log 192803'15162 (192803'15161) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:72 2014-08-11 12:51:40.200092 2014-08-13 18:06:06.601821 7f5c973a57a0 20 read_log 192803'15163 (192803'15162) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:73 2014-08-11 12:51:40.445578 2014-08-13 18:06:06.601833 7f5c973a57a0 20 read_log 192803'15164 (192803'15163) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:74 2014-08-11 12:51:40.473317 2014-08-13 18:06:06.601846 7f5c973a57a0 20 read_log 192803'15165 (192803'15164) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:75 2014-08-11 12:51:40.502496 2014-08-13 18:06:06.601858 7f5c973a57a0 20 read_log 192803'15166 (192803'15165) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:76 2014-08-11 12:51:40.529351 2014-08-13 18:06:06.601870 7f5c973a57a0 20 read_log 192803'15167 (192803'15166) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:77 2014-08-11 12:51:40.555503 2014-08-13 18:06:06.601882 7f5c973a57a0 20 read_log 192803'15168 (192803'15167) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:78 2014-08-11 12:51:40.588176 2014-08-13 18:06:06.601894 7f5c973a57a0 20 read_log 192803'15169 (192803'15168) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:79 2014-08-11 12:51:40.614409 2014-08-13 18:06:06.601906 7f5c973a57a0 20 read_log 192803'15170 (192803'15169) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:80 2014-08-11 12:51:40.645163 2014-08-13 18:06:06.601918 7f5c973a57a0 20 read_log 192803'15171 (192803'15170) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:81 2014-08-11 12:51:40.681089 2014-08-13 18:06:06.601930 7f5c973a57a0 20 read_log 192803'15172 (192803'15171) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:82 2014-08-11 12:51:40.728585 2014-08-13 18:06:06.601944 7f5c973a57a0 20 read_log 192803'15173 (192803'15172) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:83 2014-08-11 12:51:41.193614 2014-08-13 18:06:06.601957 7f5c973a57a0 20 read_log 192803'15174 (192803'15173) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:84 2014-08-11 12:51:41.223323 2014-08-13 18:06:06.601969 7f5c973a57a0 20 read_log 192803'15175 (192803'15174) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:85 2014-08-11 12:51:41.258193 2014-08-13 18:06:06.601983 7f5c973a57a0 20 read_log 192803'15176 (192803'15175) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:86 2014-08-11 12:51:41.286695 2014-08-13 18:06:06.601995 7f5c973a57a0 20 read_log 192803'15177 (192803'15176) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:87 2014-08-11 12:51:41.313936 2014-08-13 18:06:06.602007 7f5c973a57a0 20 read_log 192803'15178 (192803'15177) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:88 2014-08-11 12:51:41.344205 2014-08-13 18:06:06.602019 7f5c973a57a0 20 read_log 192803'15179 (192803'15178) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:89 2014-08-11 12:51:41.371658 2014-08-13 18:06:06.602031 7f5c973a57a0 20 read_log 192803'15180 (192803'15179) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:90 2014-08-11 12:51:41.408570 2014-08-13 18:06:06.602045 7f5c973a57a0 20 read_log 192803'15181 (192803'15180) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:91 2014-08-11 12:51:41.435537 2014-08-13 18:06:06.602058 7f5c973a57a0 20 read_log 192803'15182 (192803'15181) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:92 2014-08-11 12:51:41.461828 2014-08-13 18:06:06.602072 7f5c973a57a0 20 read_log 192803'15183 (192803'15182) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:93 2014-08-11 12:51:41.749054 2014-08-13 18:06:06.602084 7f5c973a57a0 20 read_log 192803'15184 (192803'15183) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:94 2014-08-11 12:51:41.778586 2014-08-13 18:06:06.602098 7f5c973a57a0 20 read_log 192803'15185 (192803'15184) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:95 2014-08-11 12:51:41.813766 2014-08-13 18:06:06.602116 7f5c973a57a0 20 read_log 192803'15186 (192803'15185) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:96 2014-08-11 12:51:41.841566 2014-08-13 18:06:06.602131 7f5c973a57a0 20 read_log 192803'15187 (192803'15186) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:97 2014-08-11 12:51:41.869467 2014-08-13 18:06:06.602143 7f5c973a57a0 20 read_log 192803'15188 (192803'15187) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:98 2014-08-11 12:51:41.907960 2014-08-13 18:06:06.602169 7f5c973a57a0 20 read_log 192803'15189 (192803'15188) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:99 2014-08-11 12:51:41.935927 2014-08-13 18:06:06.602182 7f5c973a57a0 20 read_log 192803'15190 (192803'15189) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:100 2014-08-11 12:51:41.964647 2014-08-13 18:06:06.602196 7f5c973a57a0 20 read_log 192803'15191 (192803'15190) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:101 2014-08-11 12:51:41.991298 2014-08-13 18:06:06.602214 7f5c973a57a0 20 read_log 192803'15192 (192803'15191) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:102 2014-08-11 12:51:42.021812 2014-08-13 18:06:06.602236 7f5c973a57a0 20 read_log 192803'15193 (192803'15192) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:103 2014-08-11 12:51:42.313044 2014-08-13 18:06:06.602250 7f5c973a57a0 20 read_log 192803'15194 (192803'15193) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:104 2014-08-11 12:51:42.344301 2014-08-13 18:06:06.602264 7f5c973a57a0 20 read_log 192803'15195 (192803'15194) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:105 2014-08-11 12:51:42.378036 2014-08-13 18:06:06.602279 7f5c973a57a0 20 read_log 192803'15196 (192803'15195) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:106 2014-08-11 12:51:42.407567 2014-08-13 18:06:06.602291 7f5c973a57a0 20 read_log 192803'15197 (192803'15196) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:107 2014-08-11 12:51:42.433160 2014-08-13 18:06:06.602303 7f5c973a57a0 20 read_log 192803'15198 (192803'15197) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:108 2014-08-11 12:51:42.463423 2014-08-13 18:06:06.602315 7f5c973a57a0 20 read_log 192803'15199 (192803'15198) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:109 2014-08-11 12:51:42.489784 2014-08-13 18:06:06.602328 7f5c973a57a0 20 read_log 192803'15200 (192803'15199) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:110 2014-08-11 12:51:42.519781 2014-08-13 18:06:06.602340 7f5c973a57a0 20 read_log 192803'15201 (192803'15200) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:111 2014-08-11 12:51:42.544088 2014-08-13 18:06:06.602352 7f5c973a57a0 20 read_log 192803'15202 (192803'15201) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:112 2014-08-11 12:51:42.569853 2014-08-13 18:06:06.602369 7f5c973a57a0 20 read_log 192803'15203 (192803'15202) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:113 2014-08-11 12:51:42.859576 2014-08-13 18:06:06.602381 7f5c973a57a0 20 read_log 192803'15204 (192803'15203) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:114 2014-08-11 12:51:42.888261 2014-08-13 18:06:06.602393 7f5c973a57a0 20 read_log 192803'15205 (192803'15204) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:115 2014-08-11 12:51:42.924610 2014-08-13 18:06:06.602417 7f5c973a57a0 20 read_log 192803'15206 (192803'15205) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:116 2014-08-11 12:51:42.951731 2014-08-13 18:06:06.602433 7f5c973a57a0 20 read_log 192803'15207 (192803'15206) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:117 2014-08-11 12:51:42.978054 2014-08-13 18:06:06.602446 7f5c973a57a0 20 read_log 192803'15208 (192803'15207) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:118 2014-08-11 12:51:43.008757 2014-08-13 18:06:06.602473 7f5c973a57a0 20 read_log 192803'15209 (192803'15208) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:119 2014-08-11 12:51:43.037252 2014-08-13 18:06:06.602496 7f5c973a57a0 20 read_log 192803'15210 (192803'15209) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:120 2014-08-11 12:51:43.067240 2014-08-13 18:06:06.602511 7f5c973a57a0 20 read_log 192803'15211 (192803'15210) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:121 2014-08-11 12:51:43.096791 2014-08-13 18:06:06.602524 7f5c973a57a0 20 read_log 192803'15212 (192803'15211) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:122 2014-08-11 12:51:43.123787 2014-08-13 18:06:06.602536 7f5c973a57a0 20 read_log 192803'15213 (192803'15212) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:123 2014-08-11 12:51:43.156897 2014-08-13 18:06:06.602557 7f5c973a57a0 20 read_log 192803'15214 (192803'15213) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:124 2014-08-11 12:51:43.427360 2014-08-13 18:06:06.602573 7f5c973a57a0 20 read_log 192803'15215 (192803'15214) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:125 2014-08-11 12:51:43.463522 2014-08-13 18:06:06.602585 7f5c973a57a0 20 read_log 192803'15216 (192803'15215) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:126 2014-08-11 12:51:43.490840 2014-08-13 18:06:06.602597 7f5c973a57a0 20 read_log 192803'15217 (192803'15216) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:127 2014-08-11 12:51:43.522163 2014-08-13 18:06:06.602610 7f5c973a57a0 20 read_log 192803'15218 (192803'15217) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:128 2014-08-11 12:51:43.556598 2014-08-13 18:06:06.602622 7f5c973a57a0 20 read_log 192803'15219 (192803'15218) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:129 2014-08-11 12:51:43.585877 2014-08-13 18:06:06.602639 7f5c973a57a0 20 read_log 192803'15220 (192803'15219) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:130 2014-08-11 12:51:43.619930 2014-08-13 18:06:06.602662 7f5c973a57a0 20 read_log 192803'15221 (192803'15220) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:131 2014-08-11 12:51:43.648088 2014-08-13 18:06:06.602681 7f5c973a57a0 20 read_log 192803'15222 (192803'15221) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:132 2014-08-11 12:51:43.674881 2014-08-13 18:06:06.602695 7f5c973a57a0 20 read_log 192803'15223 (192803'15222) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:133 2014-08-11 12:51:43.704609 2014-08-13 18:06:06.602707 7f5c973a57a0 20 read_log 192803'15224 (192803'15223) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:134 2014-08-11 12:51:43.940761 2014-08-13 18:06:06.602719 7f5c973a57a0 20 read_log 192803'15225 (192803'15224) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:135 2014-08-11 12:51:43.977275 2014-08-13 18:06:06.602734 7f5c973a57a0 20 read_log 192803'15226 (192803'15225) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:136 2014-08-11 12:51:44.004970 2014-08-13 18:06:06.602747 7f5c973a57a0 20 read_log 192803'15227 (192803'15226) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:137 2014-08-11 12:51:44.032293 2014-08-13 18:06:06.602759 7f5c973a57a0 20 read_log 192803'15228 (192803'15227) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:138 2014-08-11 12:51:44.059751 2014-08-13 18:06:06.602782 7f5c973a57a0 20 read_log 192803'15229 (192803'15228) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:139 2014-08-11 12:51:44.082608 2014-08-13 18:06:06.602798 7f5c973a57a0 20 read_log 192803'15230 (192803'15229) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:140 2014-08-11 12:51:44.132956 2014-08-13 18:06:06.602810 7f5c973a57a0 20 read_log 192803'15231 (192803'15230) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:141 2014-08-11 12:51:44.155206 2014-08-13 18:06:06.602822 7f5c973a57a0 20 read_log 192803'15232 (192803'15231) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:142 2014-08-11 12:51:44.192450 2014-08-13 18:06:06.602840 7f5c973a57a0 20 read_log 192803'15233 (192803'15232) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:143 2014-08-11 12:51:44.225561 2014-08-13 18:06:06.602859 7f5c973a57a0 20 read_log 192803'15234 (192803'15233) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:144 2014-08-11 12:51:44.462087 2014-08-13 18:06:06.602876 7f5c973a57a0 20 read_log 192803'15235 (192803'15234) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:145 2014-08-11 12:51:44.495255 2014-08-13 18:06:06.602889 7f5c973a57a0 20 read_log 192803'15236 (192803'15235) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:146 2014-08-11 12:51:44.520700 2014-08-13 18:06:06.602911 7f5c973a57a0 20 read_log 192803'15237 (192803'15236) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:147 2014-08-11 12:51:44.547456 2014-08-13 18:06:06.602930 7f5c973a57a0 20 read_log 192803'15238 (192803'15237) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:148 2014-08-11 12:51:44.583774 2014-08-13 18:06:06.602945 7f5c973a57a0 20 read_log 192803'15239 (192803'15238) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:149 2014-08-11 12:51:44.610919 2014-08-13 18:06:06.602957 7f5c973a57a0 20 read_log 192803'15240 (192803'15239) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:150 2014-08-11 12:51:44.638647 2014-08-13 18:06:06.602969 7f5c973a57a0 20 read_log 192803'15241 (192803'15240) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:151 2014-08-11 12:51:44.663876 2014-08-13 18:06:06.602984 7f5c973a57a0 20 read_log 192803'15242 (192803'15241) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:152 2014-08-11 12:51:44.695669 2014-08-13 18:06:06.602996 7f5c973a57a0 20 read_log 192803'15243 (192803'15242) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:153 2014-08-11 12:51:44.736393 2014-08-13 18:06:06.603008 7f5c973a57a0 20 read_log 192803'15244 (192803'15243) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:154 2014-08-11 12:51:44.983272 2014-08-13 18:06:06.603031 7f5c973a57a0 20 read_log 192803'15245 (192803'15244) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:155 2014-08-11 12:51:45.012355 2014-08-13 18:06:06.603044 7f5c973a57a0 20 read_log 192803'15246 (192803'15245) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:156 2014-08-11 12:51:45.045155 2014-08-13 18:06:06.603077 7f5c973a57a0 20 read_log 192803'15247 (192803'15246) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:157 2014-08-11 12:51:45.069567 2014-08-13 18:06:06.603092 7f5c973a57a0 20 read_log 192803'15248 (192803'15247) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:158 2014-08-11 12:51:45.099702 2014-08-13 18:06:06.603105 7f5c973a57a0 20 read_log 192803'15249 (192803'15248) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:159 2014-08-11 12:51:45.129360 2014-08-13 18:06:06.603117 7f5c973a57a0 20 read_log 192803'15250 (192803'15249) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:160 2014-08-11 12:51:45.156266 2014-08-13 18:06:06.603129 7f5c973a57a0 20 read_log 192803'15251 (192803'15250) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:161 2014-08-11 12:51:45.190255 2014-08-13 18:06:06.603142 7f5c973a57a0 20 read_log 192803'15252 (192803'15251) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:162 2014-08-11 12:51:45.218098 2014-08-13 18:06:06.603191 7f5c973a57a0 20 read_log 192803'15253 (192803'15252) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:163 2014-08-11 12:51:45.253114 2014-08-13 18:06:06.603222 7f5c973a57a0 20 read_log 192803'15254 (192803'15253) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:164 2014-08-11 12:51:45.495913 2014-08-13 18:06:06.603236 7f5c973a57a0 20 read_log 192803'15255 (192803'15254) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:165 2014-08-11 12:51:45.524677 2014-08-13 18:06:06.603261 7f5c973a57a0 20 read_log 192803'15256 (192803'15255) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:166 2014-08-11 12:51:45.558582 2014-08-13 18:06:06.603275 7f5c973a57a0 20 read_log 192803'15257 (192803'15256) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:167 2014-08-11 12:51:45.582147 2014-08-13 18:06:06.603345 7f5c973a57a0 20 read_log 192803'15258 (192803'15257) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:168 2014-08-11 12:51:45.614941 2014-08-13 18:06:06.603361 7f5c973a57a0 20 read_log 192803'15259 (192803'15258) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:169 2014-08-11 12:51:45.637976 2014-08-13 18:06:06.603374 7f5c973a57a0 20 read_log 192803'15260 (192803'15259) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:170 2014-08-11 12:51:45.660468 2014-08-13 18:06:06.603396 7f5c973a57a0 20 read_log 192803'15261 (192803'15260) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:171 2014-08-11 12:51:45.690353 2014-08-13 18:06:06.603409 7f5c973a57a0 20 read_log 192803'15262 (192803'15261) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:172 2014-08-11 12:51:45.717250 2014-08-13 18:06:06.603450 7f5c973a57a0 20 read_log 192803'15263 (192803'15262) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:173 2014-08-11 12:51:45.753957 2014-08-13 18:06:06.603487 7f5c973a57a0 20 read_log 192803'15264 (192803'15263) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:174 2014-08-11 12:51:46.012871 2014-08-13 18:06:06.603501 7f5c973a57a0 20 read_log 192803'15265 (192803'15264) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:175 2014-08-11 12:51:46.041959 2014-08-13 18:06:06.603523 7f5c973a57a0 20 read_log 192803'15266 (192803'15265) modify a181605e/core.46996+10+2/head//24 by client.1855404.0:176 2014-08-11 12:51:46.082156 2014-08-13 18:06:06.603541 7f5c973a57a0 20 read_log 192834'15269 (192834'15269) modify 5e/hit_set_24.5e_archive_2014-08-11 12:48:05.518603_2014-08-11 15:23:37.081884/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:37.082085 2014-08-13 18:06:06.603557 7f5c973a57a0 20 read_log 192834'15270 (192803'14718) delete 5e/hit_set_24.5e_archive_2014-08-11 10:24:43.601998_2014-08-11 12:48:05.517931/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:23:37.082085 2014-08-13 18:06:06.603569 7f5c973a57a0 20 read_log 192834'15295 (0'0) promote 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by osd.289.0:52117 2014-08-11 15:23:42.581980 2014-08-13 18:06:06.603600 7f5c973a57a0 20 read_log 192834'15296 (192834'15295) modify 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.933448.0:1712 2014-08-11 15:23:42.571637 2014-08-13 18:06:06.603628 7f5c973a57a0 20 read_log 192834'15297 (0'0) promote 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:52127 2014-08-11 15:23:43.365951 2014-08-13 18:06:06.603651 7f5c973a57a0 20 read_log 192834'15298 (192834'15297) modify 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by client.933448.0:1844 2014-08-11 15:23:43.357131 2014-08-13 18:06:06.603667 7f5c973a57a0 20 read_log 192834'15305 (0'0) promote e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by osd.289.0:52149 2014-08-11 15:23:44.834375 2014-08-13 18:06:06.603680 7f5c973a57a0 20 read_log 192834'15306 (192834'15305) modify e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.933448.0:2067 2014-08-11 15:23:44.826459 2014-08-13 18:06:06.603703 7f5c973a57a0 20 read_log 192834'15327 (0'0) promote 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:52193 2014-08-11 15:23:50.068420 2014-08-13 18:06:06.603716 7f5c973a57a0 20 read_log 192834'15328 (192834'15327) modify 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by client.933448.0:3078 2014-08-11 15:23:50.047334 2014-08-13 18:06:06.603728 7f5c973a57a0 20 read_log 192834'15343 (0'0) promote fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by osd.289.0:52265 2014-08-11 15:23:57.307866 2014-08-13 18:06:06.603744 7f5c973a57a0 20 read_log 192834'15344 (192834'15343) modify fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.933448.0:4505 2014-08-11 15:23:57.299725 2014-08-13 18:06:06.603768 7f5c973a57a0 20 read_log 192834'15345 (0'0) promote 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:52271 2014-08-11 15:23:57.812384 2014-08-13 18:06:06.603781 7f5c973a57a0 20 read_log 192834'15346 (192834'15345) modify 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by client.933448.0:4595 2014-08-11 15:23:57.803189 2014-08-13 18:06:06.603793 7f5c973a57a0 20 read_log 192834'15347 (192803'15266) clean a181605e/core.46996+10+2/head//24 by osd.289.0:52279 2014-08-11 15:23:58.247387 2014-08-13 18:06:06.603806 7f5c973a57a0 20 read_log 192834'15348 (0'0) promote df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by osd.289.0:52280 2014-08-11 15:23:58.249582 2014-08-13 18:06:06.603818 7f5c973a57a0 20 read_log 192834'15349 (192834'15348) modify df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.933448.0:4703 2014-08-11 15:23:58.241533 2014-08-13 18:06:06.603830 7f5c973a57a0 20 read_log 192834'15352 (192834'15347) delete a181605e/core.46996+10+2/head//24 by osd.289.0:52288 2014-08-11 15:23:59.377241 2014-08-13 18:06:06.603842 7f5c973a57a0 20 read_log 192834'15369 (0'0) promote 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by osd.289.0:52329 2014-08-11 15:24:02.472772 2014-08-13 18:06:06.603854 7f5c973a57a0 20 read_log 192834'15370 (192834'15369) modify 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.933448.0:5678 2014-08-11 15:24:02.463426 2014-08-13 18:06:06.603867 7f5c973a57a0 20 read_log 192834'15411 (0'0) promote db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by osd.289.0:52457 2014-08-11 15:24:15.047999 2014-08-13 18:06:06.603879 7f5c973a57a0 20 read_log 192834'15412 (192834'15411) modify db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.933448.0:8688 2014-08-11 15:24:15.035509 2014-08-13 18:06:06.603891 7f5c973a57a0 20 read_log 192834'15413 (0'0) promote 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by osd.289.0:52461 2014-08-11 15:24:15.122133 2014-08-13 18:06:06.603907 7f5c973a57a0 20 read_log 192834'15414 (192834'15413) modify 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.933448.0:8708 2014-08-11 15:24:15.113940 2014-08-13 18:06:06.603919 7f5c973a57a0 20 read_log 192834'15417 (0'0) promote d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by osd.289.0:52475 2014-08-11 15:24:15.935584 2014-08-13 18:06:06.603931 7f5c973a57a0 20 read_log 192834'15418 (192834'15417) modify d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.933448.0:8906 2014-08-11 15:24:15.926246 2014-08-13 18:06:06.603943 7f5c973a57a0 20 read_log 192834'15419 (0'0) promote 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by osd.289.0:52487 2014-08-11 15:24:17.520503 2014-08-13 18:06:06.603955 7f5c973a57a0 20 read_log 192834'15420 (192834'15419) modify 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.933448.0:9090 2014-08-11 15:24:17.512392 2014-08-13 18:06:06.603968 7f5c973a57a0 20 read_log 192834'15423 (0'0) promote 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by osd.289.0:52497 2014-08-11 15:24:18.253862 2014-08-13 18:06:06.603980 7f5c973a57a0 20 read_log 192834'15424 (192834'15423) modify 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.933448.0:9279 2014-08-11 15:24:18.245606 2014-08-13 18:06:06.603992 7f5c973a57a0 20 read_log 192834'15439 (0'0) promote 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by osd.289.0:52587 2014-08-11 15:24:28.169430 2014-08-13 18:06:06.604004 7f5c973a57a0 20 read_log 192834'15440 (192834'15439) modify 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.933448.0:11525 2014-08-11 15:24:28.161528 2014-08-13 18:06:06.604024 7f5c973a57a0 20 read_log 192834'15467 (0'0) promote e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by osd.289.0:52711 2014-08-11 15:24:39.464419 2014-08-13 18:06:06.604040 7f5c973a57a0 20 read_log 192834'15468 (192834'15467) modify e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.933448.0:13651 2014-08-11 15:24:39.456155 2014-08-13 18:06:06.604053 7f5c973a57a0 20 read_log 192834'15515 (0'0) promote 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by osd.289.0:52980 2014-08-11 15:24:57.433124 2014-08-13 18:06:06.604065 7f5c973a57a0 20 read_log 192834'15516 (192834'15515) modify 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.933448.0:16853 2014-08-11 15:24:57.422510 2014-08-13 18:06:06.604077 7f5c973a57a0 20 read_log 192834'15534 (0'0) promote 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by osd.289.0:53045 2014-08-11 15:25:03.640900 2014-08-13 18:06:06.604092 7f5c973a57a0 20 read_log 192834'15535 (192834'15534) modify 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.933448.0:18059 2014-08-11 15:25:03.309225 2014-08-13 18:06:06.604106 7f5c973a57a0 20 read_log 192834'15537 (0'0) promote 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by osd.289.0:53094 2014-08-11 15:25:08.703300 2014-08-13 18:06:06.604120 7f5c973a57a0 20 read_log 192834'15538 (192834'15537) modify 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.933448.0:19059 2014-08-11 15:25:08.332658 2014-08-13 18:06:06.604133 7f5c973a57a0 20 read_log 192834'15567 (0'0) promote cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by osd.289.0:53376 2014-08-11 15:25:31.751128 2014-08-13 18:06:06.604154 7f5c973a57a0 20 read_log 192834'15568 (192834'15567) modify cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.933448.0:21602 2014-08-11 15:25:31.179036 2014-08-13 18:06:06.604170 7f5c973a57a0 20 read_log 192834'15585 (192834'15296) modify 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.933448.0:24003 2014-08-11 15:25:40.165000 2014-08-13 18:06:06.604182 7f5c973a57a0 20 read_log 192834'15586 (192834'15298) modify 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by client.933448.0:24135 2014-08-11 15:25:40.318705 2014-08-13 18:06:06.604195 7f5c973a57a0 20 read_log 192834'15590 (192834'15306) modify e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.933448.0:24358 2014-08-11 15:25:40.673433 2014-08-13 18:06:06.604207 7f5c973a57a0 20 read_log 192834'15601 (192834'15328) modify 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by client.933448.0:25369 2014-08-11 15:25:42.033345 2014-08-13 18:06:06.604223 7f5c973a57a0 20 read_log 192834'15609 (192834'15344) modify fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.933448.0:26796 2014-08-11 15:25:43.768760 2014-08-13 18:06:06.604236 7f5c973a57a0 20 read_log 192834'15610 (192834'15346) modify 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by client.933448.0:26886 2014-08-11 15:25:43.840895 2014-08-13 18:06:06.604249 7f5c973a57a0 20 read_log 192834'15611 (192834'15349) modify df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.933448.0:26994 2014-08-11 15:25:43.985609 2014-08-13 18:06:06.604264 7f5c973a57a0 20 read_log 192834'15621 (192834'15370) modify 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.933448.0:27969 2014-08-11 15:25:45.321803 2014-08-13 18:06:06.604279 7f5c973a57a0 20 read_log 192834'15642 (192834'15412) modify db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.933448.0:30979 2014-08-11 15:25:49.473939 2014-08-13 18:06:06.604292 7f5c973a57a0 20 read_log 192834'15643 (192834'15414) modify 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.933448.0:30999 2014-08-11 15:25:49.483423 2014-08-13 18:06:06.604316 7f5c973a57a0 20 read_log 192834'15645 (192834'15418) modify d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.933448.0:31197 2014-08-11 15:25:49.783756 2014-08-13 18:06:06.604331 7f5c973a57a0 20 read_log 192834'15646 (192834'15420) modify 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.933448.0:31381 2014-08-11 15:25:49.989824 2014-08-13 18:06:06.604344 7f5c973a57a0 20 read_log 192834'15648 (192834'15424) modify 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.933448.0:31570 2014-08-11 15:25:50.199046 2014-08-13 18:06:06.604360 7f5c973a57a0 20 read_log 192834'15656 (192834'15440) modify 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.933448.0:33816 2014-08-11 15:25:54.233560 2014-08-13 18:06:06.604373 7f5c973a57a0 20 read_log 192834'15670 (192834'15468) modify e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.933448.0:35942 2014-08-11 15:25:57.031249 2014-08-13 18:06:06.604392 7f5c973a57a0 20 read_log 192834'15694 (192834'15516) modify 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.933448.0:39144 2014-08-11 15:26:01.622184 2014-08-13 18:06:06.604409 7f5c973a57a0 20 read_log 192834'15704 (192834'15535) modify 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.933448.0:40350 2014-08-11 15:26:02.583450 2014-08-13 18:06:06.604424 7f5c973a57a0 20 read_log 192834'15705 (192834'15538) modify 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.933448.0:41350 2014-08-11 15:26:02.917189 2014-08-13 18:06:06.604445 7f5c973a57a0 20 read_log 192834'15720 (192834'15568) modify cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.933448.0:43893 2014-08-11 15:26:03.754790 2014-08-13 18:06:06.604460 7f5c973a57a0 20 read_log 192834'15729 (0'0) promote 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by osd.289.0:54389 2014-08-11 15:26:16.605256 2014-08-13 18:06:06.604473 7f5c973a57a0 20 read_log 192834'15730 (192834'15729) modify 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.1883634.0:579 2014-08-11 15:26:16.596541 2014-08-13 18:06:06.604493 7f5c973a57a0 20 read_log 192834'15745 (0'0) promote 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by osd.289.0:54429 2014-08-11 15:26:21.693416 2014-08-13 18:06:06.604506 7f5c973a57a0 20 read_log 192834'15746 (192834'15745) modify 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.1883634.0:1954 2014-08-11 15:26:21.686018 2014-08-13 18:06:06.604518 7f5c973a57a0 20 read_log 192834'15765 (0'0) promote cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:54489 2014-08-11 15:26:25.993049 2014-08-13 18:06:06.604536 7f5c973a57a0 20 read_log 192834'15766 (192834'15765) modify cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by client.1883634.0:3149 2014-08-11 15:26:25.984546 2014-08-13 18:06:06.604550 7f5c973a57a0 20 read_log 192834'15795 (0'0) promote a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by osd.289.0:54599 2014-08-11 15:26:34.065524 2014-08-13 18:06:06.604563 7f5c973a57a0 20 read_log 192834'15796 (192834'15795) modify a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.1883634.0:5342 2014-08-11 15:26:34.057539 2014-08-13 18:06:06.604575 7f5c973a57a0 20 read_log 192834'15815 (0'0) promote fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by osd.289.0:54677 2014-08-11 15:26:39.647566 2014-08-13 18:06:06.604602 7f5c973a57a0 20 read_log 192834'15816 (192834'15815) modify fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.1883634.0:6820 2014-08-11 15:26:39.614193 2014-08-13 18:06:06.604615 7f5c973a57a0 20 read_log 192834'15827 (0'0) promote bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by osd.289.0:54703 2014-08-11 15:26:42.061184 2014-08-13 18:06:06.604638 7f5c973a57a0 20 read_log 192834'15828 (192834'15827) modify bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.1883634.0:7408 2014-08-11 15:26:42.052017 2014-08-13 18:06:06.604670 7f5c973a57a0 20 read_log 192834'15869 (0'0) promote 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by osd.289.0:55140 2014-08-11 15:26:54.923756 2014-08-13 18:06:06.604685 7f5c973a57a0 20 read_log 192834'15870 (192834'15869) modify 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.1883634.0:10399 2014-08-11 15:26:54.915696 2014-08-13 18:06:06.604697 7f5c973a57a0 20 read_log 192834'15893 (0'0) promote 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by osd.289.0:55357 2014-08-11 15:27:01.387196 2014-08-13 18:06:06.604710 7f5c973a57a0 20 read_log 192834'15894 (192834'15893) modify 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.1883634.0:11812 2014-08-11 15:27:01.343735 2014-08-13 18:06:06.604722 7f5c973a57a0 20 read_log 192834'15903 (0'0) promote da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by osd.289.0:55415 2014-08-11 15:27:06.247124 2014-08-13 18:06:06.604735 7f5c973a57a0 20 read_log 192834'15904 (192834'15903) modify da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.1883634.0:12829 2014-08-11 15:27:06.239621 2014-08-13 18:06:06.604748 7f5c973a57a0 20 read_log 192834'15909 (0'0) promote 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:55453 2014-08-11 15:27:07.351048 2014-08-13 18:06:06.604760 7f5c973a57a0 20 read_log 192834'15910 (192834'15909) modify 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by client.1883634.0:13046 2014-08-11 15:27:07.341666 2014-08-13 18:06:06.604773 7f5c973a57a0 20 read_log 192834'15913 (0'0) promote 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:55545 2014-08-11 15:27:10.061375 2014-08-13 18:06:06.604785 7f5c973a57a0 20 read_log 192834'15914 (192834'15913) modify 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by client.1883634.0:13552 2014-08-11 15:27:10.052171 2014-08-13 18:06:06.604797 7f5c973a57a0 20 read_log 192834'15927 (0'0) promote cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by osd.289.0:55597 2014-08-11 15:27:14.175079 2014-08-13 18:06:06.604809 7f5c973a57a0 20 read_log 192834'15928 (192834'15927) modify cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.1883634.0:14635 2014-08-11 15:27:14.164837 2014-08-13 18:06:06.604821 7f5c973a57a0 20 read_log 192834'15939 (0'0) promote 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:55656 2014-08-11 15:27:20.710803 2014-08-13 18:06:06.604833 7f5c973a57a0 20 read_log 192834'15940 (192834'15939) modify 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by client.1883634.0:15568 2014-08-11 15:27:20.347562 2014-08-13 18:06:06.604845 7f5c973a57a0 20 read_log 192834'15943 (0'0) promote d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by osd.289.0:55661 2014-08-11 15:27:20.945819 2014-08-13 18:06:06.604858 7f5c973a57a0 20 read_log 192834'15944 (192834'15943) modify d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.1883634.0:15601 2014-08-11 15:27:20.796894 2014-08-13 18:06:06.604870 7f5c973a57a0 20 read_log 192834'15951 (0'0) promote 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by osd.289.0:55727 2014-08-11 15:27:28.630170 2014-08-13 18:06:06.604882 7f5c973a57a0 20 read_log 192834'15952 (192834'15951) modify 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.1883634.0:16644 2014-08-11 15:27:28.620972 2014-08-13 18:06:06.604894 7f5c973a57a0 20 read_log 192834'16007 (0'0) promote 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by osd.289.0:56234 2014-08-11 15:27:56.050634 2014-08-13 18:06:06.604914 7f5c973a57a0 20 read_log 192834'16010 (192834'16007) modify 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.1883634.0:20093 2014-08-11 15:27:55.206213 2014-08-13 18:06:06.604929 7f5c973a57a0 20 read_log 192834'16029 (0'0) promote 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by osd.289.0:56363 2014-08-11 15:28:03.795286 2014-08-13 18:06:06.604941 7f5c973a57a0 20 read_log 192834'16030 (192834'16029) modify 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.1883634.0:21323 2014-08-11 15:28:03.785039 2014-08-13 18:06:06.604956 7f5c973a57a0 20 read_log 192834'16047 (0'0) promote 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by osd.289.0:56439 2014-08-11 15:28:07.916215 2014-08-13 18:06:06.604968 7f5c973a57a0 20 read_log 192834'16048 (192834'16047) modify 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.1883634.0:22298 2014-08-11 15:28:07.906780 2014-08-13 18:06:06.604983 7f5c973a57a0 20 read_log 192834'16062 (192834'15730) modify 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.1883634.0:23903 2014-08-11 15:28:18.063093 2014-08-13 18:06:06.604995 7f5c973a57a0 20 read_log 192834'16070 (192834'15746) modify 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.1883634.0:25278 2014-08-11 15:28:20.450628 2014-08-13 18:06:06.605009 7f5c973a57a0 20 read_log 192834'16080 (192834'15766) modify cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by client.1883634.0:26473 2014-08-11 15:28:23.175466 2014-08-13 18:06:06.605022 7f5c973a57a0 20 read_log 192834'16095 (192834'15796) modify a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.1883634.0:28666 2014-08-11 15:28:26.117023 2014-08-13 18:06:06.605036 7f5c973a57a0 20 read_log 192834'16105 (192834'15816) modify fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.1883634.0:30144 2014-08-11 15:28:28.163152 2014-08-13 18:06:06.605048 7f5c973a57a0 20 read_log 192834'16111 (192834'15828) modify bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.1883634.0:30732 2014-08-11 15:28:29.229322 2014-08-13 18:06:06.605061 7f5c973a57a0 20 read_log 192834'16132 (192834'15870) modify 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.1883634.0:33723 2014-08-11 15:28:33.719119 2014-08-13 18:06:06.605073 7f5c973a57a0 20 read_log 192834'16144 (192834'15894) modify 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.1883634.0:35136 2014-08-11 15:28:36.066344 2014-08-13 18:06:06.605085 7f5c973a57a0 20 read_log 192834'16148 (192834'16148) modify 5e/hit_set_24.5e_archive_2014-08-11 15:23:37.082089_2014-08-11 15:28:37.297779/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:37.298018 2014-08-13 18:06:06.605099 7f5c973a57a0 20 read_log 192834'16149 (192834'15269) delete 5e/hit_set_24.5e_archive_2014-08-11 12:48:05.518603_2014-08-11 15:23:37.081884/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 15:28:37.298018 2014-08-13 18:06:06.605112 7f5c973a57a0 20 read_log 192834'16151 (192834'15904) modify da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.1883634.0:36153 2014-08-11 15:28:37.652886 2014-08-13 18:06:06.605124 7f5c973a57a0 20 read_log 192834'16154 (192834'15910) modify 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by client.1883634.0:36370 2014-08-11 15:28:37.952304 2014-08-13 18:06:06.605138 7f5c973a57a0 20 read_log 192834'16156 (192834'15914) modify 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by client.1883634.0:36876 2014-08-11 15:28:38.526662 2014-08-13 18:06:06.605159 7f5c973a57a0 20 read_log 192834'16163 (192834'15928) modify cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.1883634.0:37959 2014-08-11 15:28:40.020890 2014-08-13 18:06:06.605181 7f5c973a57a0 20 read_log 192834'16169 (192834'15940) modify 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by client.1883634.0:38892 2014-08-11 15:28:41.250112 2014-08-13 18:06:06.605195 7f5c973a57a0 20 read_log 192834'16171 (192834'15944) modify d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.1883634.0:38925 2014-08-11 15:28:41.284398 2014-08-13 18:06:06.605208 7f5c973a57a0 20 read_log 192834'16175 (192834'15952) modify 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.1883634.0:39968 2014-08-11 15:28:42.985654 2014-08-13 18:06:06.605223 7f5c973a57a0 20 read_log 192834'16203 (192834'16010) modify 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.1883634.0:43417 2014-08-11 15:28:46.547267 2014-08-13 18:06:06.605238 7f5c973a57a0 20 read_log 192834'16214 (192834'16030) modify 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.1883634.0:44647 2014-08-11 15:28:46.915324 2014-08-13 18:06:06.605252 7f5c973a57a0 20 read_log 192834'16223 (192834'16048) modify 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.1883634.0:45622 2014-08-11 15:28:47.335549 2014-08-13 18:06:06.605265 7f5c973a57a0 20 read_log 192888'16229 (192888'16229) modify 5e/hit_set_24.5e_archive_2014-08-11 16:02:02.174496_2014-08-11 16:44:57.949927/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.950147 2014-08-13 18:06:06.605277 7f5c973a57a0 20 read_log 192888'16230 (192834'16148) delete 5e/hit_set_24.5e_archive_2014-08-11 15:23:37.082089_2014-08-11 15:28:37.297779/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.950147 2014-08-13 18:06:06.605289 7f5c973a57a0 20 read_log 192888'16235 (0'0) promote d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by osd.289.0:58607 2014-08-11 16:45:02.224217 2014-08-13 18:06:06.605305 7f5c973a57a0 20 read_log 192888'16236 (192888'16235) modify d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.1315259.0:1824 2014-08-11 16:45:02.216620 2014-08-13 18:06:06.605320 7f5c973a57a0 20 read_log 192888'16237 (0'0) promote 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by osd.289.0:58613 2014-08-11 16:45:02.561952 2014-08-13 18:06:06.605333 7f5c973a57a0 20 read_log 192888'16238 (192888'16237) modify 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.1315259.0:1915 2014-08-11 16:45:02.553046 2014-08-13 18:06:06.605348 7f5c973a57a0 20 read_log 192888'16239 (0'0) promote 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by osd.289.0:58705 2014-08-11 16:45:16.983742 2014-08-13 18:06:06.605360 7f5c973a57a0 20 read_log 192888'16240 (192888'16239) modify 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.1315259.0:5778 2014-08-11 16:45:16.975357 2014-08-13 18:06:06.605385 7f5c973a57a0 20 read_log 192888'16247 (0'0) promote 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:58725 2014-08-11 16:45:19.366169 2014-08-13 18:06:06.605401 7f5c973a57a0 20 read_log 192888'16248 (192888'16247) modify 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by client.1315259.0:6438 2014-08-11 16:45:19.358092 2014-08-13 18:06:06.605416 7f5c973a57a0 20 read_log 192888'16257 (0'0) promote bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by osd.289.0:58785 2014-08-11 16:45:29.097856 2014-08-13 18:06:06.605435 7f5c973a57a0 20 read_log 192888'16258 (192888'16257) modify bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.1315259.0:9064 2014-08-11 16:45:29.090278 2014-08-13 18:06:06.605451 7f5c973a57a0 20 read_log 192888'16261 (0'0) promote a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:58795 2014-08-11 16:45:30.581054 2014-08-13 18:06:06.605500 7f5c973a57a0 20 read_log 192888'16262 (192888'16261) modify a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by client.1315259.0:9448 2014-08-11 16:45:30.572157 2014-08-13 18:06:06.605517 7f5c973a57a0 20 read_log 192888'16271 (0'0) promote 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by osd.289.0:58819 2014-08-11 16:45:36.344037 2014-08-13 18:06:06.605547 7f5c973a57a0 20 read_log 192888'16272 (192888'16271) modify 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.1315259.0:10931 2014-08-11 16:45:36.334786 2014-08-13 18:06:06.605563 7f5c973a57a0 20 read_log 192888'16273 (0'0) promote be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by osd.289.0:58823 2014-08-11 16:45:36.977441 2014-08-13 18:06:06.605576 7f5c973a57a0 20 read_log 192888'16274 (192888'16273) modify be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.1315259.0:11089 2014-08-11 16:45:36.970864 2014-08-13 18:06:06.605604 7f5c973a57a0 20 read_log 192888'16275 (0'0) promote 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by osd.289.0:58845 2014-08-11 16:45:41.700699 2014-08-13 18:06:06.605617 7f5c973a57a0 20 read_log 192888'16276 (192888'16275) modify 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.1315259.0:12260 2014-08-11 16:45:41.693138 2014-08-13 18:06:06.605629 7f5c973a57a0 20 read_log 192888'16283 (0'0) promote 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by osd.289.0:58859 2014-08-11 16:45:44.436103 2014-08-13 18:06:06.605642 7f5c973a57a0 20 read_log 192888'16284 (192888'16283) modify 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.1315259.0:12905 2014-08-11 16:45:44.428863 2014-08-13 18:06:06.605654 7f5c973a57a0 20 read_log 192888'16287 (0'0) promote e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by osd.289.0:58867 2014-08-11 16:45:45.122324 2014-08-13 18:06:06.605666 7f5c973a57a0 20 read_log 192888'16288 (192888'16287) modify e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.1315259.0:13065 2014-08-11 16:45:45.113113 2014-08-13 18:06:06.605679 7f5c973a57a0 20 read_log 192888'16289 (0'0) promote 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by osd.289.0:58871 2014-08-11 16:45:45.801271 2014-08-13 18:06:06.605691 7f5c973a57a0 20 read_log 192888'16290 (192888'16289) modify 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.1315259.0:13238 2014-08-11 16:45:45.790728 2014-08-13 18:06:06.605703 7f5c973a57a0 20 read_log 192888'16291 (0'0) promote d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by osd.289.0:58875 2014-08-11 16:45:45.923528 2014-08-13 18:06:06.605715 7f5c973a57a0 20 read_log 192888'16292 (192888'16291) modify d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.1315259.0:13270 2014-08-11 16:45:45.912169 2014-08-13 18:06:06.605727 7f5c973a57a0 20 read_log 192888'16295 (0'0) promote 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by osd.289.0:58881 2014-08-11 16:45:47.889717 2014-08-13 18:06:06.605739 7f5c973a57a0 20 read_log 192888'16296 (192888'16295) modify 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.1315259.0:13762 2014-08-11 16:45:47.881662 2014-08-13 18:06:06.605751 7f5c973a57a0 20 read_log 192888'16299 (0'0) promote 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by osd.289.0:58911 2014-08-11 16:45:53.883084 2014-08-13 18:06:06.605764 7f5c973a57a0 20 read_log 192888'16300 (192888'16299) modify 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.1315259.0:15345 2014-08-11 16:45:53.875124 2014-08-13 18:06:06.605776 7f5c973a57a0 20 read_log 192888'16301 (0'0) promote 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by osd.289.0:58935 2014-08-11 16:45:59.279291 2014-08-13 18:06:06.605795 7f5c973a57a0 20 read_log 192888'16302 (192888'16301) modify 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.1315259.0:16730 2014-08-11 16:45:59.270806 2014-08-13 18:06:06.605808 7f5c973a57a0 20 read_log 192888'16305 (0'0) promote 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by osd.289.0:58945 2014-08-11 16:46:00.234643 2014-08-13 18:06:06.605829 7f5c973a57a0 20 read_log 192888'16306 (192888'16305) modify 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.1315259.0:16976 2014-08-11 16:46:00.225503 2014-08-13 18:06:06.605845 7f5c973a57a0 20 read_log 192888'16307 (0'0) promote 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by osd.289.0:58947 2014-08-11 16:46:00.356730 2014-08-13 18:06:06.605858 7f5c973a57a0 20 read_log 192888'16308 (192888'16307) modify 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.1315259.0:17000 2014-08-11 16:46:00.348630 2014-08-13 18:06:06.605870 7f5c973a57a0 20 read_log 192888'16309 (0'0) promote 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by osd.289.0:58961 2014-08-11 16:46:03.904274 2014-08-13 18:06:06.605882 7f5c973a57a0 20 read_log 192888'16310 (192888'16309) modify 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.1315259.0:17879 2014-08-11 16:46:03.895214 2014-08-13 18:06:06.605895 7f5c973a57a0 20 read_log 192888'16317 (0'0) promote ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by osd.289.0:58999 2014-08-11 16:46:13.141423 2014-08-13 18:06:06.605907 7f5c973a57a0 20 read_log 192888'16318 (192888'16317) modify ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.1315259.0:20277 2014-08-11 16:46:13.132097 2014-08-13 18:06:06.605919 7f5c973a57a0 20 read_log 192888'16319 (0'0) promote 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by osd.289.0:59017 2014-08-11 16:46:16.464926 2014-08-13 18:06:06.605931 7f5c973a57a0 20 read_log 192888'16320 (192888'16319) modify 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.1315259.0:21160 2014-08-11 16:46:16.454651 2014-08-13 18:06:06.605943 7f5c973a57a0 20 read_log 192888'16327 (0'0) promote 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by osd.289.0:59065 2014-08-11 16:46:28.504274 2014-08-13 18:06:06.605955 7f5c973a57a0 20 read_log 192888'16328 (192888'16327) modify 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.1315259.0:24231 2014-08-11 16:46:28.497227 2014-08-13 18:06:06.605967 7f5c973a57a0 20 read_log 192888'16329 (0'0) promote 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:59089 2014-08-11 16:46:32.308198 2014-08-13 18:06:06.605979 7f5c973a57a0 20 read_log 192888'16330 (192888'16329) modify 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by client.1315259.0:25200 2014-08-11 16:46:32.295225 2014-08-13 18:06:06.605991 7f5c973a57a0 20 read_log 192888'16331 (0'0) promote 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:59093 2014-08-11 16:46:33.082691 2014-08-13 18:06:06.606022 7f5c973a57a0 20 read_log 192888'16332 (192888'16331) modify 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by client.1315259.0:25385 2014-08-11 16:46:33.072615 2014-08-13 18:06:06.606038 7f5c973a57a0 20 read_log 192888'16333 (0'0) promote abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by osd.289.0:59117 2014-08-11 16:46:37.643636 2014-08-13 18:06:06.606050 7f5c973a57a0 20 read_log 192888'16334 (192888'16333) modify abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.1315259.0:26436 2014-08-11 16:46:37.635532 2014-08-13 18:06:06.606065 7f5c973a57a0 20 read_log 192888'16337 (0'0) promote bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by osd.289.0:59135 2014-08-11 16:46:41.273907 2014-08-13 18:06:06.606084 7f5c973a57a0 20 read_log 192888'16338 (192888'16337) modify bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.1315259.0:27310 2014-08-11 16:46:41.265343 2014-08-13 18:06:06.606099 7f5c973a57a0 20 read_log 192888'16339 (0'0) promote 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by osd.289.0:59137 2014-08-11 16:46:41.370576 2014-08-13 18:06:06.606111 7f5c973a57a0 20 read_log 192888'16340 (192888'16339) modify 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.1315259.0:27333 2014-08-11 16:46:41.355114 2014-08-13 18:06:06.606126 7f5c973a57a0 20 read_log 192888'16341 (0'0) promote 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:59141 2014-08-11 16:46:41.768267 2014-08-13 18:06:06.606138 7f5c973a57a0 20 read_log 192888'16342 (192888'16341) modify 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by client.1315259.0:27437 2014-08-11 16:46:41.759469 2014-08-13 18:06:06.606157 7f5c973a57a0 20 read_log 192888'16343 (0'0) promote dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by osd.289.0:59155 2014-08-11 16:46:43.004666 2014-08-13 18:06:06.606172 7f5c973a57a0 20 read_log 192888'16344 (192888'16343) modify dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.1315259.0:27754 2014-08-11 16:46:42.996425 2014-08-13 18:06:06.606185 7f5c973a57a0 20 read_log 192888'16347 (0'0) promote 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:59161 2014-08-11 16:46:44.404343 2014-08-13 18:06:06.606197 7f5c973a57a0 20 read_log 192888'16348 (192888'16347) modify 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by client.1315259.0:28113 2014-08-11 16:46:44.394288 2014-08-13 18:06:06.606209 7f5c973a57a0 20 read_log 192888'16353 (0'0) promote 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by osd.289.0:59167 2014-08-11 16:46:45.964771 2014-08-13 18:06:06.606224 7f5c973a57a0 20 read_log 192888'16354 (192888'16353) modify 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.1315259.0:28464 2014-08-11 16:46:45.955841 2014-08-13 18:06:06.606239 7f5c973a57a0 20 read_log 192888'16357 (0'0) promote 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:59179 2014-08-11 16:46:47.385676 2014-08-13 18:06:06.606251 7f5c973a57a0 20 read_log 192888'16358 (192888'16357) modify 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by client.1315259.0:28832 2014-08-11 16:46:47.377417 2014-08-13 18:06:06.606264 7f5c973a57a0 20 read_log 192888'16359 (0'0) promote 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by osd.289.0:59183 2014-08-11 16:46:47.595462 2014-08-13 18:06:06.606275 7f5c973a57a0 20 read_log 192888'16360 (192888'16359) modify 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.1315259.0:28887 2014-08-11 16:46:47.588052 2014-08-13 18:06:06.606291 7f5c973a57a0 20 read_log 192888'16365 (0'0) promote 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by osd.289.0:59197 2014-08-11 16:46:51.586904 2014-08-13 18:06:06.606303 7f5c973a57a0 20 read_log 192888'16366 (192888'16365) modify 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.1315259.0:29858 2014-08-11 16:46:51.579316 2014-08-13 18:06:06.606315 7f5c973a57a0 20 read_log 192888'16369 (192888'16236) modify d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.1315259.0:32551 2014-08-11 16:46:56.408217 2014-08-13 18:06:06.606327 7f5c973a57a0 20 read_log 192888'16370 (192888'16238) modify 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.1315259.0:32642 2014-08-11 16:46:56.432730 2014-08-13 18:06:06.606342 7f5c973a57a0 20 read_log 192888'16371 (192888'16240) modify 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.1315259.0:36505 2014-08-11 16:46:57.439893 2014-08-13 18:06:06.606365 7f5c973a57a0 20 read_log 192888'16375 (192888'16248) modify 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by client.1315259.0:37165 2014-08-11 16:46:57.612278 2014-08-13 18:06:06.606380 7f5c973a57a0 20 read_log 192888'16380 (192888'16258) modify bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.1315259.0:39791 2014-08-11 16:46:58.303982 2014-08-13 18:06:06.606402 7f5c973a57a0 20 read_log 192888'16382 (192888'16262) modify a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by client.1315259.0:40175 2014-08-11 16:46:58.411187 2014-08-13 18:06:06.606414 7f5c973a57a0 20 read_log 192888'16387 (192888'16272) modify 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.1315259.0:41658 2014-08-11 16:46:58.802744 2014-08-13 18:06:06.606440 7f5c973a57a0 20 read_log 192888'16388 (192888'16274) modify be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.1315259.0:41816 2014-08-11 16:46:58.845865 2014-08-13 18:06:06.606453 7f5c973a57a0 20 read_log 192888'16389 (192888'16276) modify 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.1315259.0:42987 2014-08-11 16:46:59.159820 2014-08-13 18:06:06.606466 7f5c973a57a0 20 read_log 192888'16393 (192888'16284) modify 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.1315259.0:43632 2014-08-11 16:46:59.344544 2014-08-13 18:06:06.606488 7f5c973a57a0 20 read_log 192888'16395 (192888'16288) modify e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.1315259.0:43792 2014-08-11 16:46:59.389746 2014-08-13 18:06:06.606501 7f5c973a57a0 20 read_log 192888'16396 (192888'16290) modify 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.1315259.0:43965 2014-08-11 16:46:59.441253 2014-08-13 18:06:06.606513 7f5c973a57a0 20 read_log 192888'16397 (192888'16292) modify d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.1315259.0:43997 2014-08-11 16:46:59.451702 2014-08-13 18:06:06.606534 7f5c973a57a0 20 read_log 192888'16399 (192888'16296) modify 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.1315259.0:44489 2014-08-11 16:46:59.593022 2014-08-13 18:06:06.606550 7f5c973a57a0 20 read_log 192888'16401 (192888'16300) modify 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.1315259.0:46072 2014-08-11 16:47:00.052186 2014-08-13 18:06:06.606562 7f5c973a57a0 20 read_log 192888'16402 (192888'16302) modify 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.1315259.0:47457 2014-08-11 16:47:00.461523 2014-08-13 18:06:06.606582 7f5c973a57a0 20 read_log 192888'16404 (192888'16306) modify 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.1315259.0:47703 2014-08-11 16:47:00.541404 2014-08-13 18:06:06.606606 7f5c973a57a0 20 read_log 192888'16405 (192888'16308) modify 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.1315259.0:47727 2014-08-11 16:47:00.548841 2014-08-13 18:06:06.606619 7f5c973a57a0 20 read_log 192888'16406 (192888'16310) modify 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.1315259.0:48606 2014-08-11 16:47:00.811560 2014-08-13 18:06:06.606631 7f5c973a57a0 20 read_log 192888'16410 (192888'16318) modify ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.1315259.0:51004 2014-08-11 16:47:01.547227 2014-08-13 18:06:06.606644 7f5c973a57a0 20 read_log 192888'16411 (192888'16320) modify 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.1315259.0:51887 2014-08-11 16:47:01.813112 2014-08-13 18:06:06.606656 7f5c973a57a0 20 read_log 192888'16415 (192888'16328) modify 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.1315259.0:54958 2014-08-11 16:47:02.769139 2014-08-13 18:06:06.606677 7f5c973a57a0 20 read_log 192888'16416 (192888'16330) modify 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by client.1315259.0:55927 2014-08-11 16:47:03.054582 2014-08-13 18:06:06.606707 7f5c973a57a0 20 read_log 192888'16417 (192888'16332) modify 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by client.1315259.0:56112 2014-08-11 16:47:03.108902 2014-08-13 18:06:06.606723 7f5c973a57a0 20 read_log 192888'16418 (192888'16334) modify abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.1315259.0:57163 2014-08-11 16:47:03.429301 2014-08-13 18:06:06.606742 7f5c973a57a0 20 read_log 192888'16420 (192888'16338) modify bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.1315259.0:58037 2014-08-11 16:47:03.684938 2014-08-13 18:06:06.606760 7f5c973a57a0 20 read_log 192888'16421 (192888'16340) modify 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.1315259.0:58060 2014-08-11 16:47:03.691422 2014-08-13 18:06:06.606781 7f5c973a57a0 20 read_log 192888'16422 (192888'16342) modify 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by client.1315259.0:58164 2014-08-11 16:47:03.723457 2014-08-13 18:06:06.606797 7f5c973a57a0 20 read_log 192888'16423 (192888'16344) modify dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.1315259.0:58481 2014-08-11 16:47:03.815361 2014-08-13 18:06:06.606810 7f5c973a57a0 20 read_log 192888'16425 (192888'16348) modify 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by client.1315259.0:58840 2014-08-11 16:47:03.923358 2014-08-13 18:06:06.606822 7f5c973a57a0 20 read_log 192888'16428 (192888'16354) modify 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.1315259.0:59191 2014-08-11 16:47:04.029430 2014-08-13 18:06:06.606835 7f5c973a57a0 20 read_log 192888'16430 (192888'16358) modify 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by client.1315259.0:59559 2014-08-11 16:47:04.159786 2014-08-13 18:06:06.606847 7f5c973a57a0 20 read_log 192888'16431 (192888'16360) modify 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.1315259.0:59614 2014-08-11 16:47:04.182780 2014-08-13 18:06:06.606859 7f5c973a57a0 20 read_log 192888'16434 (192888'16366) modify 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.1315259.0:60585 2014-08-11 16:47:04.483820 2014-08-13 18:06:06.606871 7f5c973a57a0 20 read_log 192888'16435 (192888'16435) modify 5e/hit_set_24.5e_archive_2014-08-11 16:44:57.950151_2014-08-11 17:05:17.864913/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.865135 2014-08-13 18:06:06.606884 7f5c973a57a0 20 read_log 192888'16436 (192888'16229) delete 5e/hit_set_24.5e_archive_2014-08-11 16:02:02.174496_2014-08-11 16:44:57.949927/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:17.865135 2014-08-13 18:06:06.606896 7f5c973a57a0 20 read_log 192888'16441 (0'0) promote ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by osd.289.0:59575 2014-08-11 17:05:25.931326 2014-08-13 18:06:06.606908 7f5c973a57a0 20 read_log 192888'16442 (192888'16441) modify ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.942868.0:2699 2014-08-11 17:05:25.924315 2014-08-13 18:06:06.606920 7f5c973a57a0 20 read_log 192888'16443 (0'0) promote e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by osd.289.0:59581 2014-08-11 17:05:27.926967 2014-08-13 18:06:06.606932 7f5c973a57a0 20 read_log 192888'16444 (192888'16443) modify e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.942868.0:3239 2014-08-11 17:05:27.917895 2014-08-13 18:06:06.606945 7f5c973a57a0 20 read_log 192888'16453 (0'0) promote 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by osd.289.0:59697 2014-08-11 17:05:52.558911 2014-08-13 18:06:06.606957 7f5c973a57a0 20 read_log 192888'16454 (192888'16453) modify 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.942868.0:9821 2014-08-11 17:05:52.550432 2014-08-13 18:06:06.606976 7f5c973a57a0 20 read_log 192888'16459 (0'0) promote edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by osd.289.0:59731 2014-08-11 17:05:58.088047 2014-08-13 18:06:06.606989 7f5c973a57a0 20 read_log 192888'16460 (192888'16459) modify edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.942868.0:11339 2014-08-11 17:05:58.079898 2014-08-13 18:06:06.607006 7f5c973a57a0 20 read_log 192888'16461 (0'0) promote 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by osd.289.0:59733 2014-08-11 17:05:58.109620 2014-08-13 18:06:06.607019 7f5c973a57a0 20 read_log 192888'16462 (192888'16461) modify 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.942868.0:11345 2014-08-11 17:05:58.101944 2014-08-13 18:06:06.607031 7f5c973a57a0 20 read_log 192888'16463 (0'0) promote 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by osd.289.0:59735 2014-08-11 17:05:58.542574 2014-08-13 18:06:06.607043 7f5c973a57a0 20 read_log 192888'16464 (192888'16463) modify 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.942868.0:11464 2014-08-11 17:05:58.534488 2014-08-13 18:06:06.607055 7f5c973a57a0 20 read_log 192888'16465 (0'0) promote d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by osd.289.0:59741 2014-08-11 17:05:59.849564 2014-08-13 18:06:06.607067 7f5c973a57a0 20 read_log 192888'16466 (192888'16465) modify d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.942868.0:11817 2014-08-11 17:05:59.841072 2014-08-13 18:06:06.607091 7f5c973a57a0 20 read_log 192888'16469 (0'0) promote 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by osd.289.0:59755 2014-08-11 17:06:02.614900 2014-08-13 18:06:06.607104 7f5c973a57a0 20 read_log 192888'16470 (192888'16469) modify 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.942868.0:12589 2014-08-11 17:06:02.606382 2014-08-13 18:06:06.607117 7f5c973a57a0 20 read_log 192888'16475 (0'0) promote 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by osd.289.0:59775 2014-08-11 17:06:06.472642 2014-08-13 18:06:06.607129 7f5c973a57a0 20 read_log 192888'16476 (192888'16475) modify 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.942868.0:13577 2014-08-11 17:06:06.464147 2014-08-13 18:06:06.607141 7f5c973a57a0 20 read_log 192888'16479 (0'0) promote 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by osd.289.0:59787 2014-08-11 17:06:08.549408 2014-08-13 18:06:06.607159 7f5c973a57a0 20 read_log 192888'16480 (192888'16479) modify 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.942868.0:14078 2014-08-11 17:06:08.541220 2014-08-13 18:06:06.607184 7f5c973a57a0 20 read_log 192888'16481 (0'0) promote 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by osd.289.0:59795 2014-08-11 17:06:10.853143 2014-08-13 18:06:06.607197 7f5c973a57a0 20 read_log 192888'16482 (192888'16481) modify 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.942868.0:14647 2014-08-11 17:06:10.844443 2014-08-13 18:06:06.607209 7f5c973a57a0 20 read_log 192888'16483 (0'0) promote 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by osd.289.0:59797 2014-08-11 17:06:11.031839 2014-08-13 18:06:06.607222 7f5c973a57a0 20 read_log 192888'16484 (192888'16483) modify 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.942868.0:14694 2014-08-11 17:06:11.024312 2014-08-13 18:06:06.607237 7f5c973a57a0 20 read_log 192888'16487 (0'0) promote eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by osd.289.0:59831 2014-08-11 17:06:18.259522 2014-08-13 18:06:06.607249 7f5c973a57a0 20 read_log 192888'16488 (192888'16487) modify eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.942868.0:16536 2014-08-11 17:06:18.249107 2014-08-13 18:06:06.607271 7f5c973a57a0 20 read_log 192888'16489 (0'0) promote 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by osd.289.0:59835 2014-08-11 17:06:19.261080 2014-08-13 18:06:06.607290 7f5c973a57a0 20 read_log 192888'16490 (192888'16489) modify 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.942868.0:16800 2014-08-11 17:06:19.251260 2014-08-13 18:06:06.607303 7f5c973a57a0 20 read_log 192888'16499 (0'0) promote 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by osd.289.0:59867 2014-08-11 17:06:28.083323 2014-08-13 18:06:06.607315 7f5c973a57a0 20 read_log 192888'16500 (192888'16499) modify 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.942868.0:19127 2014-08-11 17:06:28.074467 2014-08-13 18:06:06.607327 7f5c973a57a0 20 read_log 192888'16505 (0'0) promote 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by osd.289.0:59905 2014-08-11 17:06:38.217706 2014-08-13 18:06:06.607345 7f5c973a57a0 20 read_log 192888'16506 (192888'16505) modify 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.942868.0:21797 2014-08-11 17:06:38.208130 2014-08-13 18:06:06.607357 7f5c973a57a0 20 read_log 192888'16507 (0'0) promote 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by osd.289.0:59911 2014-08-11 17:06:39.874383 2014-08-13 18:06:06.607369 7f5c973a57a0 20 read_log 192888'16508 (192888'16507) modify 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.942868.0:22229 2014-08-11 17:06:39.863297 2014-08-13 18:06:06.607393 7f5c973a57a0 20 read_log 192888'16509 (0'0) promote 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by osd.289.0:59921 2014-08-11 17:06:41.228441 2014-08-13 18:06:06.607408 7f5c973a57a0 20 read_log 192888'16510 (192888'16509) modify 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.942868.0:22559 2014-08-11 17:06:41.219089 2014-08-13 18:06:06.607446 7f5c973a57a0 20 read_log 192888'16515 (0'0) promote ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by osd.289.0:59949 2014-08-11 17:06:46.563774 2014-08-13 18:06:06.607462 7f5c973a57a0 20 read_log 192888'16516 (192888'16515) modify ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.942868.0:23960 2014-08-11 17:06:46.555674 2014-08-13 18:06:06.607474 7f5c973a57a0 20 read_log 192888'16527 (0'0) promote 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by osd.289.0:60019 2014-08-11 17:07:02.977283 2014-08-13 18:06:06.607497 7f5c973a57a0 20 read_log 192888'16528 (192888'16527) modify 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.942868.0:28269 2014-08-11 17:07:02.969843 2014-08-13 18:06:06.607510 7f5c973a57a0 20 read_log 192888'16533 (0'0) promote 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by osd.289.0:60035 2014-08-11 17:07:07.118681 2014-08-13 18:06:06.607522 7f5c973a57a0 20 read_log 192888'16534 (192888'16533) modify 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.942868.0:29339 2014-08-11 17:07:07.109202 2014-08-13 18:06:06.607535 7f5c973a57a0 20 read_log 192888'16537 (0'0) promote b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by osd.289.0:60043 2014-08-11 17:07:08.290672 2014-08-13 18:06:06.607553 7f5c973a57a0 20 read_log 192888'16538 (192888'16537) modify b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.942868.0:29618 2014-08-11 17:07:08.283107 2014-08-13 18:06:06.607565 7f5c973a57a0 20 read_log 192888'16547 (192888'16442) modify ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.942868.0:33965 2014-08-11 17:07:16.660561 2014-08-13 18:06:06.607577 7f5c973a57a0 20 read_log 192888'16548 (192888'16444) modify e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.942868.0:34505 2014-08-11 17:07:16.807626 2014-08-13 18:06:06.607617 7f5c973a57a0 20 read_log 192888'16553 (192888'16454) modify 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.942868.0:41087 2014-08-11 17:07:18.635237 2014-08-13 18:06:06.607633 7f5c973a57a0 20 read_log 192888'16556 (192888'16460) modify edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.942868.0:42605 2014-08-11 17:07:19.043003 2014-08-13 18:06:06.607645 7f5c973a57a0 20 read_log 192888'16557 (192888'16462) modify 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.942868.0:42611 2014-08-11 17:07:19.044268 2014-08-13 18:06:06.607657 7f5c973a57a0 20 read_log 192888'16558 (192888'16464) modify 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.942868.0:42730 2014-08-11 17:07:19.077358 2014-08-13 18:06:06.607670 7f5c973a57a0 20 read_log 192888'16559 (192888'16466) modify d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.942868.0:43083 2014-08-11 17:07:19.172797 2014-08-13 18:06:06.607682 7f5c973a57a0 20 read_log 192888'16561 (192888'16470) modify 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.942868.0:43855 2014-08-11 17:07:19.396078 2014-08-13 18:06:06.607706 7f5c973a57a0 20 read_log 192888'16564 (192888'16476) modify 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.942868.0:44843 2014-08-11 17:07:19.677189 2014-08-13 18:06:06.607719 7f5c973a57a0 20 read_log 192888'16566 (192888'16480) modify 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.942868.0:45344 2014-08-11 17:07:19.812314 2014-08-13 18:06:06.607731 7f5c973a57a0 20 read_log 192888'16567 (192888'16482) modify 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.942868.0:45913 2014-08-11 17:07:19.969488 2014-08-13 18:06:06.607744 7f5c973a57a0 20 read_log 192888'16568 (192888'16484) modify 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.942868.0:45960 2014-08-11 17:07:19.982180 2014-08-13 18:06:06.607756 7f5c973a57a0 20 read_log 192888'16570 (192888'16488) modify eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.942868.0:47802 2014-08-11 17:07:20.523667 2014-08-13 18:06:06.607768 7f5c973a57a0 20 read_log 192888'16571 (192888'16490) modify 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.942868.0:48066 2014-08-11 17:07:20.612008 2014-08-13 18:06:06.607780 7f5c973a57a0 20 read_log 192888'16576 (192888'16500) modify 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.942868.0:50393 2014-08-11 17:07:21.305271 2014-08-13 18:06:06.607792 7f5c973a57a0 20 read_log 192888'16579 (192888'16506) modify 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.942868.0:53063 2014-08-11 17:07:22.137300 2014-08-13 18:06:06.607805 7f5c973a57a0 20 read_log 192888'16580 (192888'16508) modify 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.942868.0:53495 2014-08-11 17:07:22.285758 2014-08-13 18:06:06.607817 7f5c973a57a0 20 read_log 192888'16581 (192888'16510) modify 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.942868.0:53825 2014-08-11 17:07:22.387263 2014-08-13 18:06:06.607829 7f5c973a57a0 20 read_log 192888'16584 (192888'16516) modify ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.942868.0:55226 2014-08-11 17:07:22.779504 2014-08-13 18:06:06.607841 7f5c973a57a0 20 read_log 192888'16590 (192888'16528) modify 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.942868.0:59535 2014-08-11 17:07:24.074025 2014-08-13 18:06:06.607853 7f5c973a57a0 20 read_log 192888'16593 (192888'16534) modify 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.942868.0:60605 2014-08-11 17:07:24.387846 2014-08-13 18:06:06.607866 7f5c973a57a0 20 read_log 192888'16595 (192888'16538) modify b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.942868.0:60884 2014-08-11 17:07:24.474119 2014-08-13 18:06:06.607885 7f5c973a57a0 20 read_log 193305'16599 (193305'16599) modify 5e/hit_set_24.5e_archive_2014-08-11 18:37:04.920337_2014-08-11 18:42:44.060681/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:44.060900 2014-08-13 18:06:06.607898 7f5c973a57a0 20 read_log 193305'16600 (192888'16435) delete 5e/hit_set_24.5e_archive_2014-08-11 16:44:57.950151_2014-08-11 17:05:17.864913/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:44.060900 2014-08-13 18:06:06.607910 7f5c973a57a0 20 read_log 193305'16601 (0'0) promote 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:63 2014-08-11 18:42:44.078599 2014-08-13 18:06:06.607925 7f5c973a57a0 20 read_log 193305'16602 (193305'16601) modify 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by client.951337.0:2910 2014-08-11 18:42:44.055502 2014-08-13 18:06:06.607937 7f5c973a57a0 20 read_log 193305'16603 (0'0) promote 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by osd.289.0:79 2014-08-11 18:42:47.212126 2014-08-13 18:06:06.607951 7f5c973a57a0 20 read_log 193305'16604 (193305'16603) modify 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.951337.0:3778 2014-08-11 18:42:47.205716 2014-08-13 18:06:06.607964 7f5c973a57a0 20 read_log 193305'16605 (0'0) promote 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by osd.289.0:81 2014-08-11 18:42:47.909133 2014-08-13 18:06:06.607976 7f5c973a57a0 20 read_log 193305'16606 (193305'16605) modify 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.951337.0:3968 2014-08-11 18:42:47.902320 2014-08-13 18:06:06.607988 7f5c973a57a0 20 read_log 193305'16607 (0'0) promote b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:83 2014-08-11 18:42:47.915014 2014-08-13 18:06:06.608000 7f5c973a57a0 20 read_log 193305'16608 (193305'16607) modify b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by client.951337.0:3969 2014-08-11 18:42:47.906916 2014-08-13 18:06:06.608014 7f5c973a57a0 20 read_log 193305'16609 (0'0) promote afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by osd.289.0:101 2014-08-11 18:42:52.696387 2014-08-13 18:06:06.608027 7f5c973a57a0 20 read_log 193305'16610 (193305'16609) modify afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.951337.0:5248 2014-08-11 18:42:52.687743 2014-08-13 18:06:06.608041 7f5c973a57a0 20 read_log 193305'16611 (0'0) promote f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by osd.289.0:109 2014-08-11 18:42:53.492965 2014-08-13 18:06:06.608053 7f5c973a57a0 20 read_log 193305'16612 (193305'16611) modify f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.951337.0:5468 2014-08-11 18:42:53.483833 2014-08-13 18:06:06.608069 7f5c973a57a0 20 read_log 193305'16613 (0'0) promote 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by osd.289.0:137 2014-08-11 18:42:59.378188 2014-08-13 18:06:06.608083 7f5c973a57a0 20 read_log 193305'16614 (193305'16613) modify 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.951337.0:7092 2014-08-11 18:42:59.370396 2014-08-13 18:06:06.608095 7f5c973a57a0 20 read_log 193305'16615 (0'0) promote 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by osd.289.0:167 2014-08-11 18:43:05.461047 2014-08-13 18:06:06.608109 7f5c973a57a0 20 read_log 193305'16616 (193305'16615) modify 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.951337.0:8701 2014-08-11 18:43:05.452908 2014-08-13 18:06:06.608122 7f5c973a57a0 20 read_log 193305'16617 (0'0) promote 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by osd.289.0:183 2014-08-11 18:43:09.187542 2014-08-13 18:06:06.608151 7f5c973a57a0 20 read_log 193305'16618 (193305'16617) modify 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.951337.0:9696 2014-08-11 18:43:09.180362 2014-08-13 18:06:06.608167 7f5c973a57a0 20 read_log 193305'16619 (0'0) promote e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by osd.289.0:199 2014-08-11 18:43:11.787802 2014-08-13 18:06:06.608180 7f5c973a57a0 20 read_log 193305'16620 (193305'16619) modify e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.951337.0:10404 2014-08-11 18:43:11.780832 2014-08-13 18:06:06.608194 7f5c973a57a0 20 read_log 193305'16621 (0'0) promote 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by osd.289.0:211 2014-08-11 18:43:13.903802 2014-08-13 18:06:06.608207 7f5c973a57a0 20 read_log 193305'16622 (193305'16621) modify 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.951337.0:10972 2014-08-11 18:43:13.895681 2014-08-13 18:06:06.608221 7f5c973a57a0 20 read_log 193305'16623 (0'0) promote cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by osd.289.0:369 2014-08-11 18:43:46.314563 2014-08-13 18:06:06.608234 7f5c973a57a0 20 read_log 193305'16624 (193305'16623) modify cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.951337.0:19919 2014-08-11 18:43:46.307102 2014-08-13 18:06:06.608251 7f5c973a57a0 20 read_log 193305'16625 (0'0) promote 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by osd.289.0:375 2014-08-11 18:43:47.650604 2014-08-13 18:06:06.608264 7f5c973a57a0 20 read_log 193305'16626 (193305'16625) modify 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.951337.0:20270 2014-08-11 18:43:47.643650 2014-08-13 18:06:06.608276 7f5c973a57a0 20 read_log 193305'16627 (0'0) promote c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:383 2014-08-11 18:43:49.388687 2014-08-13 18:06:06.608294 7f5c973a57a0 20 read_log 193305'16628 (193305'16627) modify c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by client.951337.0:20772 2014-08-11 18:43:49.379423 2014-08-13 18:06:06.608307 7f5c973a57a0 20 read_log 193305'16629 (0'0) promote 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by osd.289.0:397 2014-08-11 18:43:50.477192 2014-08-13 18:06:06.608320 7f5c973a57a0 20 read_log 193305'16630 (193305'16629) modify 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.951337.0:21069 2014-08-11 18:43:50.467674 2014-08-13 18:06:06.608332 7f5c973a57a0 20 read_log 193305'16631 (0'0) promote f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:407 2014-08-11 18:43:53.047596 2014-08-13 18:06:06.608351 7f5c973a57a0 20 read_log 193305'16632 (193305'16631) modify f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by client.951337.0:21794 2014-08-11 18:43:53.039061 2014-08-13 18:06:06.608376 7f5c973a57a0 20 read_log 193305'16633 (0'0) promote a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by osd.289.0:417 2014-08-11 18:43:55.378473 2014-08-13 18:06:06.608395 7f5c973a57a0 20 read_log 193305'16634 (193305'16633) modify a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.951337.0:22424 2014-08-11 18:43:55.367113 2014-08-13 18:06:06.608408 7f5c973a57a0 20 read_log 193305'16635 (0'0) promote b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by osd.289.0:423 2014-08-11 18:43:56.071866 2014-08-13 18:06:06.608428 7f5c973a57a0 20 read_log 193305'16636 (193305'16635) modify b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.951337.0:22617 2014-08-11 18:43:56.062406 2014-08-13 18:06:06.608443 7f5c973a57a0 20 read_log 193305'16637 (0'0) promote a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by osd.289.0:437 2014-08-11 18:43:59.929680 2014-08-13 18:06:06.608473 7f5c973a57a0 20 read_log 193305'16638 (193305'16637) modify a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.951337.0:23698 2014-08-11 18:43:59.922431 2014-08-13 18:06:06.608490 7f5c973a57a0 20 read_log 193305'16639 (0'0) promote 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by osd.289.0:439 2014-08-11 18:44:00.188408 2014-08-13 18:06:06.608502 7f5c973a57a0 20 read_log 193305'16640 (193305'16639) modify 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.951337.0:23771 2014-08-11 18:44:00.179739 2014-08-13 18:06:06.608515 7f5c973a57a0 20 read_log 193305'16641 (0'0) promote 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:539 2014-08-11 18:44:18.355045 2014-08-13 18:06:06.608527 7f5c973a57a0 20 read_log 193305'16642 (193305'16641) modify 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by client.951337.0:28741 2014-08-11 18:44:18.347341 2014-08-13 18:06:06.608546 7f5c973a57a0 20 read_log 193305'16643 (0'0) promote 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by osd.289.0:543 2014-08-11 18:44:20.225603 2014-08-13 18:06:06.608563 7f5c973a57a0 20 read_log 193305'16644 (193305'16643) modify 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.951337.0:29243 2014-08-11 18:44:20.217584 2014-08-13 18:06:06.608583 7f5c973a57a0 20 read_log 193305'16645 (0'0) promote 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by osd.289.0:549 2014-08-11 18:44:20.981093 2014-08-13 18:06:06.608595 7f5c973a57a0 20 read_log 193305'16646 (193305'16645) modify 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.951337.0:29448 2014-08-11 18:44:20.973834 2014-08-13 18:06:06.608608 7f5c973a57a0 20 read_log 193305'16647 (0'0) promote 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by osd.289.0:585 2014-08-11 18:44:28.251735 2014-08-13 18:06:06.608620 7f5c973a57a0 20 read_log 193305'16648 (193305'16647) modify 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.951337.0:31406 2014-08-11 18:44:28.244026 2014-08-13 18:06:06.608632 7f5c973a57a0 20 read_log 193305'16649 (193305'16602) modify 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by client.951337.0:35621 2014-08-11 18:44:35.679997 2014-08-13 18:06:06.608646 7f5c973a57a0 20 read_log 193305'16650 (193305'16604) modify 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.951337.0:36489 2014-08-11 18:44:36.260175 2014-08-13 18:06:06.608666 7f5c973a57a0 20 read_log 193305'16651 (193305'16606) modify 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.951337.0:36679 2014-08-11 18:44:36.470180 2014-08-13 18:06:06.608679 7f5c973a57a0 20 read_log 193305'16652 (193305'16608) modify b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by client.951337.0:36680 2014-08-11 18:44:36.470548 2014-08-13 18:06:06.608692 7f5c973a57a0 20 read_log 193305'16653 (193305'16610) modify afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.951337.0:37959 2014-08-11 18:44:37.141182 2014-08-13 18:06:06.608704 7f5c973a57a0 20 read_log 193305'16654 (193305'16612) modify f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.951337.0:38179 2014-08-11 18:44:37.274837 2014-08-13 18:06:06.608721 7f5c973a57a0 20 read_log 193305'16655 (193305'16614) modify 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.951337.0:39803 2014-08-11 18:44:38.189991 2014-08-13 18:06:06.608733 7f5c973a57a0 20 read_log 193305'16656 (193305'16616) modify 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.951337.0:41412 2014-08-11 18:44:38.745788 2014-08-13 18:06:06.608745 7f5c973a57a0 20 read_log 193305'16657 (193305'16618) modify 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.951337.0:42407 2014-08-11 18:44:39.214426 2014-08-13 18:06:06.608778 7f5c973a57a0 20 read_log 193305'16658 (193305'16620) modify e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.951337.0:43115 2014-08-11 18:44:39.487617 2014-08-13 18:06:06.608795 7f5c973a57a0 20 read_log 193305'16659 (193305'16622) modify 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.951337.0:43683 2014-08-11 18:44:39.665713 2014-08-13 18:06:06.608807 7f5c973a57a0 20 read_log 193305'16660 (193305'16624) modify cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.951337.0:52630 2014-08-11 18:44:42.288576 2014-08-13 18:06:06.608820 7f5c973a57a0 20 read_log 193305'16661 (193305'16626) modify 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.951337.0:52981 2014-08-11 18:44:42.396718 2014-08-13 18:06:06.608835 7f5c973a57a0 20 read_log 193305'16662 (193305'16628) modify c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by client.951337.0:53483 2014-08-11 18:44:42.544071 2014-08-13 18:06:06.608848 7f5c973a57a0 20 read_log 193305'16663 (193305'16630) modify 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.951337.0:53780 2014-08-11 18:44:42.634215 2014-08-13 18:06:06.608860 7f5c973a57a0 20 read_log 193305'16664 (193305'16632) modify f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by client.951337.0:54505 2014-08-11 18:44:42.858637 2014-08-13 18:06:06.608872 7f5c973a57a0 20 read_log 193305'16665 (193305'16634) modify a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.951337.0:55135 2014-08-11 18:44:43.035090 2014-08-13 18:06:06.608885 7f5c973a57a0 20 read_log 193305'16666 (193305'16636) modify b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.951337.0:55328 2014-08-11 18:44:43.094291 2014-08-13 18:06:06.608897 7f5c973a57a0 20 read_log 193305'16667 (193305'16638) modify a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.951337.0:56409 2014-08-11 18:44:43.410881 2014-08-13 18:06:06.608909 7f5c973a57a0 20 read_log 193305'16668 (193305'16640) modify 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.951337.0:56482 2014-08-11 18:44:43.431809 2014-08-13 18:06:06.608921 7f5c973a57a0 20 read_log 193305'16669 (193305'16642) modify 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by client.951337.0:61452 2014-08-11 18:44:44.925234 2014-08-13 18:06:06.608933 7f5c973a57a0 20 read_log 193305'16670 (193305'16644) modify 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.951337.0:61954 2014-08-11 18:44:45.081029 2014-08-13 18:06:06.608947 7f5c973a57a0 20 read_log 193305'16671 (193305'16646) modify 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.951337.0:62159 2014-08-11 18:44:45.142798 2014-08-13 18:06:06.608959 7f5c973a57a0 20 read_log 193305'16672 (193305'16648) modify 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.951337.0:64117 2014-08-11 18:44:45.708754 2014-08-13 18:06:06.608971 7f5c973a57a0 20 read_log 194209'16673 (192834'16169) clean 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:1303 2014-08-12 14:06:31.960131 2014-08-13 18:06:06.608984 7f5c973a57a0 20 read_log 194209'16674 (193305'16652) clean b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:1304 2014-08-12 14:06:31.960667 2014-08-13 18:06:06.608996 7f5c973a57a0 20 read_log 194209'16675 (192888'16422) clean 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:1305 2014-08-12 14:06:31.961366 2014-08-13 18:06:06.609009 7f5c973a57a0 20 read_log 194209'16676 (193305'16662) clean c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:1306 2014-08-12 14:06:31.961686 2014-08-13 18:06:06.609031 7f5c973a57a0 20 read_log 194209'16677 (193305'16669) clean 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:1307 2014-08-12 14:06:31.968627 2014-08-13 18:06:06.609047 7f5c973a57a0 20 read_log 194209'16678 (192834'15601) clean 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:1308 2014-08-12 14:06:31.971488 2014-08-13 18:06:06.609059 7f5c973a57a0 20 read_log 194209'16679 (192888'16382) clean a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:1309 2014-08-12 14:06:31.979168 2014-08-13 18:06:06.609075 7f5c973a57a0 20 read_log 194209'16680 (192834'15586) clean 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:1310 2014-08-12 14:06:31.979447 2014-08-13 18:06:06.609092 7f5c973a57a0 20 read_log 194209'16681 (192834'16156) clean 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:1311 2014-08-12 14:06:31.979724 2014-08-13 18:06:06.609105 7f5c973a57a0 20 read_log 194209'16682 (192834'16080) clean cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:1312 2014-08-12 14:06:31.980682 2014-08-13 18:06:06.609119 7f5c973a57a0 20 read_log 194209'16683 (192888'16430) clean 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:1313 2014-08-12 14:06:31.987486 2014-08-13 18:06:06.609132 7f5c973a57a0 20 read_log 194209'16684 (192888'16425) clean 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:1314 2014-08-12 14:06:31.990235 2014-08-13 18:06:06.609162 7f5c973a57a0 20 read_log 194209'16685 (192888'16553) clean 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by osd.289.0:1315 2014-08-12 14:06:31.993074 2014-08-13 18:06:06.609209 7f5c973a57a0 20 read_log 194209'16686 (192888'16417) clean 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:1316 2014-08-12 14:06:31.995865 2014-08-13 18:06:06.609237 7f5c973a57a0 20 read_log 194209'16687 (192888'16548) clean e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by osd.289.0:1317 2014-08-12 14:06:31.998749 2014-08-13 18:06:06.609253 7f5c973a57a0 20 read_log 194209'16688 (192834'16154) clean 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:1318 2014-08-12 14:06:32.001584 2014-08-13 18:06:06.609266 7f5c973a57a0 20 read_log 194209'16689 (192888'16571) clean 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by osd.289.0:1319 2014-08-12 14:06:32.004333 2014-08-13 18:06:06.609279 7f5c973a57a0 20 read_log 194209'16690 (192888'16568) clean 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by osd.289.0:1320 2014-08-12 14:06:32.009637 2014-08-13 18:06:06.609298 7f5c973a57a0 20 read_log 194209'16691 (193305'16664) clean f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:1321 2014-08-12 14:06:32.009905 2014-08-13 18:06:06.609310 7f5c973a57a0 20 read_log 194209'16692 (192888'16375) clean 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:1322 2014-08-12 14:06:32.012773 2014-08-13 18:06:06.609323 7f5c973a57a0 20 read_log 194209'16693 (193305'16649) clean 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:1323 2014-08-12 14:06:32.015825 2014-08-13 18:06:06.609339 7f5c973a57a0 20 read_log 194209'16694 (192888'16556) clean edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by osd.289.0:1324 2014-08-12 14:06:32.020858 2014-08-13 18:06:06.609353 7f5c973a57a0 20 read_log 194209'16695 (192888'16416) clean 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:1325 2014-08-12 14:06:32.021145 2014-08-13 18:06:06.609374 7f5c973a57a0 20 read_log 194209'16696 (192834'15610) clean 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:1326 2014-08-12 14:06:32.024132 2014-08-13 18:06:06.609403 7f5c973a57a0 20 read_log 194209'16697 (194209'16673) delete 981085e/benchmark_data_storage0111-ib.pocstor_41526_object15567/head//24 by osd.289.0:1327 2014-08-12 14:06:32.054808 2014-08-13 18:06:06.609427 7f5c973a57a0 20 read_log 194209'16698 (194209'16674) delete b8a6085e/benchmark_data_storage0111-ib.pocstor_62621_object3968/head//24 by osd.289.0:1328 2014-08-12 14:06:32.055191 2014-08-13 18:06:06.609440 7f5c973a57a0 20 read_log 194209'16699 (194209'16675) delete 6c49085e/benchmark_data_storage0111-ib.pocstor_41097_object27436/head//24 by osd.289.0:1329 2014-08-12 14:06:32.057959 2014-08-13 18:06:06.609453 7f5c973a57a0 20 read_log 194209'16700 (194209'16676) delete c04b085e/benchmark_data_storage0111-ib.pocstor_62621_object20771/head//24 by osd.289.0:1330 2014-08-12 14:06:32.058251 2014-08-13 18:06:06.609475 7f5c973a57a0 20 read_log 194209'16701 (194209'16677) delete 4994185e/benchmark_data_storage0111-ib.pocstor_62621_object28740/head//24 by osd.289.0:1331 2014-08-12 14:06:32.060942 2014-08-13 18:06:06.609488 7f5c973a57a0 20 read_log 194209'16702 (194209'16678) delete 638185e/benchmark_data_storage0111-ib.pocstor_26659_object3077/head//24 by osd.289.0:1332 2014-08-12 14:06:32.061233 2014-08-13 18:06:06.609501 7f5c973a57a0 20 read_log 194209'16703 (194209'16679) delete a140385e/benchmark_data_storage0111-ib.pocstor_41097_object9447/head//24 by osd.289.0:1333 2014-08-12 14:06:32.064185 2014-08-13 18:06:06.609520 7f5c973a57a0 20 read_log 194209'16704 (194209'16680) delete 143e385e/benchmark_data_storage0111-ib.pocstor_26659_object1843/head//24 by osd.289.0:1334 2014-08-12 14:06:32.064546 2014-08-13 18:06:06.609536 7f5c973a57a0 20 read_log 194209'16705 (194209'16681) delete 3a64485e/benchmark_data_storage0111-ib.pocstor_41526_object13551/head//24 by osd.289.0:1335 2014-08-12 14:06:32.067383 2014-08-13 18:06:06.609549 7f5c973a57a0 20 read_log 194209'16706 (194209'16682) delete cf77485e/benchmark_data_storage0111-ib.pocstor_41526_object3148/head//24 by osd.289.0:1336 2014-08-12 14:06:32.067725 2014-08-13 18:06:06.609562 7f5c973a57a0 20 read_log 194209'16707 (194209'16683) delete 2f01585e/benchmark_data_storage0111-ib.pocstor_41097_object28831/head//24 by osd.289.0:1337 2014-08-12 14:06:32.070758 2014-08-13 18:06:06.609579 7f5c973a57a0 20 read_log 194209'16708 (194209'16684) delete 38c7685e/benchmark_data_storage0111-ib.pocstor_41097_object28112/head//24 by osd.289.0:1338 2014-08-12 14:06:32.071088 2014-08-13 18:06:06.609592 7f5c973a57a0 20 read_log 194209'16709 (194209'16686) delete 9e3d685e/benchmark_data_storage0111-ib.pocstor_41097_object25384/head//24 by osd.289.0:1339 2014-08-12 14:06:32.073637 2014-08-13 18:06:06.609605 7f5c973a57a0 20 read_log 194209'16710 (194209'16688) delete 7d9a785e/benchmark_data_storage0111-ib.pocstor_41526_object13045/head//24 by osd.289.0:1340 2014-08-12 14:06:32.074264 2014-08-13 18:06:06.609623 7f5c973a57a0 20 read_log 194209'16711 (194209'16691) delete f981885e/benchmark_data_storage0111-ib.pocstor_62621_object21793/head//24 by osd.289.0:1341 2014-08-12 14:06:32.075046 2014-08-13 18:06:06.609640 7f5c973a57a0 20 read_log 194209'16712 (194209'16692) delete 1385885e/benchmark_data_storage0111-ib.pocstor_41097_object6437/head//24 by osd.289.0:1342 2014-08-12 14:06:32.077637 2014-08-13 18:06:06.609652 7f5c973a57a0 20 read_log 194209'16713 (194209'16693) delete 7cea985e/benchmark_data_storage0111-ib.pocstor_62621_object2909/head//24 by osd.289.0:1343 2014-08-12 14:06:32.077960 2014-08-13 18:06:06.609665 7f5c973a57a0 20 read_log 194209'16714 (194209'16695) delete 2d40c85e/benchmark_data_storage0111-ib.pocstor_41097_object25199/head//24 by osd.289.0:1344 2014-08-12 14:06:32.078751 2014-08-13 18:06:06.609677 7f5c973a57a0 20 read_log 194209'16715 (194209'16696) delete 9ee4c85e/benchmark_data_storage0111-ib.pocstor_26659_object4594/head//24 by osd.289.0:1345 2014-08-12 14:06:32.081300 2014-08-13 18:06:06.609690 7f5c973a57a0 20 read_log 194209'16716 (193305'16670) clean 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by osd.289.0:1346 2014-08-12 14:06:32.128800 2014-08-13 18:06:06.609727 7f5c973a57a0 20 read_log 194215'16717 (194215'16717) modify 5e/hit_set_24.5e_archive_2014-08-12 15:32:25.855993_2014-08-12 15:37:32.210894/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:37:32.211130 2014-08-13 18:06:06.609743 7f5c973a57a0 20 read_log 194215'16718 (193305'16599) delete 5e/hit_set_24.5e_archive_2014-08-11 18:37:04.920337_2014-08-11 18:42:44.060681/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:37:32.211130 2014-08-13 18:06:06.609756 7f5c973a57a0 20 read_log 194215'16719 (192888'16558) clean 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by osd.289.0:1740 2014-08-12 15:37:32.215651 2014-08-13 18:06:06.609769 7f5c973a57a0 20 read_log 194215'16720 (194215'16719) delete 3880005e/benchmark_data_storage0111-ib.pocstor_2228_object11463/head//24 by client.2174595.0:18609 0.000000 2014-08-13 18:06:06.609780 7f5c973a57a0 20 read_log 194215'16721 (193305'16658) clean e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by osd.289.0:1742 2014-08-12 15:37:32.223366 2014-08-13 18:06:06.609792 7f5c973a57a0 20 read_log 194215'16722 (194215'16721) delete e106005e/benchmark_data_storage0111-ib.pocstor_62621_object10403/head//24 by client.2174595.0:18611 0.000000 2014-08-13 18:06:06.609803 7f5c973a57a0 20 read_log 194215'16723 (192888'16566) clean 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by osd.289.0:1744 2014-08-12 15:37:32.230482 2014-08-13 18:06:06.609815 7f5c973a57a0 20 read_log 194215'16724 (194215'16723) delete 82c6005e/benchmark_data_storage0111-ib.pocstor_2228_object14077/head//24 by client.2174595.0:18613 0.000000 2014-08-13 18:06:06.609826 7f5c973a57a0 20 read_log 194215'16725 (192834'15643) clean 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by osd.289.0:1746 2014-08-12 15:37:32.237249 2014-08-13 18:06:06.609839 7f5c973a57a0 20 read_log 194215'16726 (194215'16725) delete 446b005e/benchmark_data_storage0111-ib.pocstor_26659_object8707/head//24 by client.2174595.0:18615 0.000000 2014-08-13 18:06:06.609849 7f5c973a57a0 20 read_log 194215'16727 (192888'16580) clean 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by osd.289.0:1748 2014-08-12 15:37:32.244179 2014-08-13 18:06:06.609861 7f5c973a57a0 20 read_log 194215'16728 (194215'16727) delete 1a8b005e/benchmark_data_storage0111-ib.pocstor_2228_object22228/head//24 by client.2174595.0:18617 0.000000 2014-08-13 18:06:06.609872 7f5c973a57a0 20 read_log 194215'16729 (192834'16151) clean da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by osd.289.0:1750 2014-08-12 15:37:32.251086 2014-08-13 18:06:06.609885 7f5c973a57a0 20 read_log 194215'16730 (194215'16729) delete da9d105e/benchmark_data_storage0111-ib.pocstor_41526_object12828/head//24 by client.2174595.0:18619 0.000000 2014-08-13 18:06:06.609895 7f5c973a57a0 20 read_log 194215'16731 (193305'16666) clean b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by osd.289.0:1752 2014-08-12 15:37:32.258423 2014-08-13 18:06:06.609908 7f5c973a57a0 20 read_log 194215'16732 (194215'16731) delete b046205e/benchmark_data_storage0111-ib.pocstor_62621_object22616/head//24 by client.2174595.0:18621 0.000000 2014-08-13 18:06:06.609918 7f5c973a57a0 20 read_log 194215'16733 (192834'16062) clean 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by osd.289.0:1754 2014-08-12 15:37:32.265042 2014-08-13 18:06:06.609931 7f5c973a57a0 20 read_log 194215'16734 (194215'16733) delete 83ea305e/benchmark_data_storage0111-ib.pocstor_41526_object578/head//24 by client.2174595.0:18623 0.000000 2014-08-13 18:06:06.609942 7f5c973a57a0 20 read_log 194215'16735 (193305'16667) clean a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by osd.289.0:1756 2014-08-12 15:37:32.271612 2014-08-13 18:06:06.609964 7f5c973a57a0 20 read_log 194215'16736 (194215'16735) delete a5ba405e/benchmark_data_storage0111-ib.pocstor_62621_object23697/head//24 by client.2174595.0:18625 0.000000 2014-08-13 18:06:06.609978 7f5c973a57a0 20 read_log 194215'16737 (192888'16564) clean 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by osd.289.0:1758 2014-08-12 15:37:32.278906 2014-08-13 18:06:06.609990 7f5c973a57a0 20 read_log 194215'16738 (194215'16737) delete 1ace405e/benchmark_data_storage0111-ib.pocstor_2228_object13576/head//24 by client.2174595.0:18627 0.000000 2014-08-13 18:06:06.610001 7f5c973a57a0 20 read_log 194215'16739 (193305'16672) clean 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by osd.289.0:1760 2014-08-12 15:37:32.286015 2014-08-13 18:06:06.610015 7f5c973a57a0 20 read_log 194215'16740 (194215'16739) delete 8c51505e/benchmark_data_storage0111-ib.pocstor_62621_object31405/head//24 by client.2174595.0:18629 0.000000 2014-08-13 18:06:06.610026 7f5c973a57a0 20 read_log 194215'16741 (192888'16389) clean 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by osd.289.0:1762 2014-08-12 15:37:32.293269 2014-08-13 18:06:06.610039 7f5c973a57a0 20 read_log 194215'16742 (194215'16741) delete 4132505e/benchmark_data_storage0111-ib.pocstor_41097_object12259/head//24 by client.2174595.0:18631 0.000000 2014-08-13 18:06:06.610051 7f5c973a57a0 20 read_log 194215'16743 (192888'16401) clean 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by osd.289.0:1764 2014-08-12 15:37:32.301042 2014-08-13 18:06:06.610066 7f5c973a57a0 20 read_log 194215'16744 (194215'16743) delete 11ce605e/benchmark_data_storage0111-ib.pocstor_41097_object15344/head//24 by client.2174595.0:18633 0.000000 2014-08-13 18:06:06.610077 7f5c973a57a0 20 read_log 194215'16745 (192888'16388) clean be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by osd.289.0:1766 2014-08-12 15:37:32.308028 2014-08-13 18:06:06.610089 7f5c973a57a0 20 read_log 194215'16746 (194215'16745) delete be61705e/benchmark_data_storage0111-ib.pocstor_41097_object11088/head//24 by client.2174595.0:18635 0.000000 2014-08-13 18:06:06.610100 7f5c973a57a0 20 read_log 194215'16747 (192834'15585) clean 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by osd.289.0:1768 2014-08-12 15:37:32.314277 2014-08-13 18:06:06.610112 7f5c973a57a0 20 read_log 194215'16748 (194215'16747) delete 8143905e/benchmark_data_storage0111-ib.pocstor_26659_object1711/head//24 by client.2174595.0:18637 0.000000 2014-08-13 18:06:06.610126 7f5c973a57a0 20 read_log 194215'16749 (192834'15704) clean 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by osd.289.0:1770 2014-08-12 15:37:32.320749 2014-08-13 18:06:06.610140 7f5c973a57a0 20 read_log 194215'16750 (194215'16749) delete 76d8905e/benchmark_data_storage0111-ib.pocstor_26659_object18058/head//24 by client.2174595.0:18639 0.000000 2014-08-13 18:06:06.610157 7f5c973a57a0 20 read_log 194215'16751 (192834'15611) clean df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by osd.289.0:1772 2014-08-12 15:37:32.328046 2014-08-13 18:06:06.610172 7f5c973a57a0 20 read_log 194215'16752 (194215'16751) delete df32a05e/benchmark_data_storage0111-ib.pocstor_26659_object4702/head//24 by client.2174595.0:18641 0.000000 2014-08-13 18:06:06.610183 7f5c973a57a0 20 read_log 194215'16753 (192834'15705) clean 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by osd.289.0:1774 2014-08-12 15:37:32.335680 2014-08-13 18:06:06.610195 7f5c973a57a0 20 read_log 194215'16754 (194215'16753) delete 71ccb05e/benchmark_data_storage0111-ib.pocstor_26659_object19058/head//24 by client.2174595.0:18643 0.000000 2014-08-13 18:06:06.610242 7f5c973a57a0 20 read_log 194215'16755 (192888'16428) clean 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by osd.289.0:1776 2014-08-12 15:37:32.342957 2014-08-13 18:06:06.610266 7f5c973a57a0 20 read_log 194215'16756 (194215'16755) delete 20e4d05e/benchmark_data_storage0111-ib.pocstor_41097_object28463/head//24 by client.2174595.0:18645 0.000000 2014-08-13 18:06:06.610279 7f5c973a57a0 20 read_log 194215'16757 (192888'16581) clean 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by osd.289.0:1778 2014-08-12 15:37:32.350107 2014-08-13 18:06:06.610296 7f5c973a57a0 20 read_log 194215'16758 (194215'16757) delete 9f6cd05e/benchmark_data_storage0111-ib.pocstor_2228_object22558/head//24 by client.2174595.0:18647 0.000000 2014-08-13 18:06:06.610307 7f5c973a57a0 20 read_log 194215'16759 (192834'15656) clean 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by osd.289.0:1780 2014-08-12 15:37:32.357296 2014-08-13 18:06:06.610319 7f5c973a57a0 20 read_log 194215'16760 (194215'16759) delete 4ca3e05e/benchmark_data_storage0111-ib.pocstor_26659_object11524/head//24 by client.2174595.0:18649 0.000000 2014-08-13 18:06:06.610330 7f5c973a57a0 20 read_log 194215'16761 (192888'16434) clean 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by osd.289.0:1782 2014-08-12 15:37:32.364598 2014-08-13 18:06:06.610348 7f5c973a57a0 20 read_log 194215'16762 (194215'16761) delete 6a68e05e/benchmark_data_storage0111-ib.pocstor_41097_object29857/head//24 by client.2174595.0:18651 0.000000 2014-08-13 18:06:06.610371 7f5c973a57a0 20 read_log 194215'16763 (193305'16661) clean 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by osd.289.0:1784 2014-08-12 15:37:32.371499 2014-08-13 18:06:06.610390 7f5c973a57a0 20 read_log 194215'16764 (194215'16763) delete 88e5f05e/benchmark_data_storage0111-ib.pocstor_62621_object20269/head//24 by client.2174595.0:18653 0.000000 2014-08-13 18:06:06.610401 7f5c973a57a0 20 read_log 194215'16765 (192888'16418) clean abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by osd.289.0:1786 2014-08-12 15:37:32.378481 2014-08-13 18:06:06.610444 7f5c973a57a0 20 read_log 194215'16766 (194215'16765) delete abfbf05e/benchmark_data_storage0111-ib.pocstor_41097_object26435/head//24 by client.2174595.0:18655 0.000000 2014-08-13 18:06:06.610456 7f5c973a57a0 20 read_log 194215'16767 (192888'16371) clean 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by osd.289.0:1788 2014-08-12 15:37:32.385711 2014-08-13 18:06:06.610482 7f5c973a57a0 20 read_log 194215'16768 (194215'16767) delete 2b9ef05e/benchmark_data_storage0111-ib.pocstor_41097_object5777/head//24 by client.2174595.0:18657 0.000000 2014-08-13 18:06:06.610493 7f5c973a57a0 20 read_log 194215'16769 (192888'16557) clean 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by osd.289.0:1790 2014-08-12 15:37:32.393299 2014-08-13 18:06:06.610508 7f5c973a57a0 20 read_log 194215'16770 (194215'16769) delete 91ae045e/benchmark_data_storage0111-ib.pocstor_2228_object11344/head//24 by client.2174595.0:18659 0.000000 2014-08-13 18:06:06.610519 7f5c973a57a0 20 read_log 194215'16771 (192834'15645) clean d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by osd.289.0:1792 2014-08-12 15:37:32.399910 2014-08-13 18:06:06.610531 7f5c973a57a0 20 read_log 194215'16772 (194215'16771) delete d8a5245e/benchmark_data_storage0111-ib.pocstor_26659_object8905/head//24 by client.2174595.0:18661 0.000000 2014-08-13 18:06:06.610542 7f5c973a57a0 20 read_log 194215'16773 (192888'16547) clean ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by osd.289.0:1794 2014-08-12 15:37:32.406967 2014-08-13 18:06:06.610562 7f5c973a57a0 20 read_log 194215'16774 (194215'16773) delete ea26245e/benchmark_data_storage0111-ib.pocstor_2228_object2698/head//24 by client.2174595.0:18663 0.000000 2014-08-13 18:06:06.610581 7f5c973a57a0 20 read_log 194215'16775 (192834'16105) clean fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by osd.289.0:1796 2014-08-12 15:37:32.413663 2014-08-13 18:06:06.610602 7f5c973a57a0 20 read_log 194215'16776 (194215'16775) delete fe07245e/benchmark_data_storage0111-ib.pocstor_41526_object6819/head//24 by client.2174595.0:18665 0.000000 2014-08-13 18:06:06.610615 7f5c973a57a0 20 read_log 194215'16777 (192834'16144) clean 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by osd.289.0:1798 2014-08-12 15:37:32.421093 2014-08-13 18:06:06.610628 7f5c973a57a0 20 read_log 194215'16778 (194215'16777) delete 61df245e/benchmark_data_storage0111-ib.pocstor_41526_object11811/head//24 by client.2174595.0:18667 0.000000 2014-08-13 18:06:06.610639 7f5c973a57a0 20 read_log 194215'16779 (192888'16402) clean 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by osd.289.0:1800 2014-08-12 15:37:32.428437 2014-08-13 18:06:06.610651 7f5c973a57a0 20 read_log 194215'16780 (194215'16779) delete 62e3345e/benchmark_data_storage0111-ib.pocstor_41097_object16729/head//24 by client.2174595.0:18669 0.000000 2014-08-13 18:06:06.610662 7f5c973a57a0 20 read_log 194215'16781 (192888'16567) clean 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by osd.289.0:1802 2014-08-12 15:37:32.435394 2014-08-13 18:06:06.610674 7f5c973a57a0 20 read_log 194215'16782 (194215'16781) delete 6617345e/benchmark_data_storage0111-ib.pocstor_2228_object14646/head//24 by client.2174595.0:18671 0.000000 2014-08-13 18:06:06.610685 7f5c973a57a0 20 read_log 194215'16783 (192834'16223) clean 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by osd.289.0:1804 2014-08-12 15:37:32.442313 2014-08-13 18:06:06.610699 7f5c973a57a0 20 read_log 194215'16784 (194215'16783) delete 172445e/benchmark_data_storage0111-ib.pocstor_41526_object22297/head//24 by client.2174595.0:18673 0.000000 2014-08-13 18:06:06.610710 7f5c973a57a0 20 read_log 194215'16785 (193305'16671) clean 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by osd.289.0:1806 2014-08-12 15:37:32.449360 2014-08-13 18:06:06.610722 7f5c973a57a0 20 read_log 194215'16786 (194215'16785) delete 4713445e/benchmark_data_storage0111-ib.pocstor_62621_object29447/head//24 by client.2174595.0:18675 0.000000 2014-08-13 18:06:06.610733 7f5c973a57a0 20 read_log 194215'16787 (192834'16175) clean 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by osd.289.0:1808 2014-08-12 15:37:32.456906 2014-08-13 18:06:06.610745 7f5c973a57a0 20 read_log 194215'16788 (194215'16787) delete 4c1a445e/benchmark_data_storage0111-ib.pocstor_41526_object16643/head//24 by client.2174595.0:18677 0.000000 2014-08-13 18:06:06.610755 7f5c973a57a0 20 read_log 194215'16789 (193305'16651) clean 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by osd.289.0:1810 2014-08-12 15:37:32.464095 2014-08-13 18:06:06.610768 7f5c973a57a0 20 read_log 194215'16790 (194215'16789) delete 547d445e/benchmark_data_storage0111-ib.pocstor_62621_object3967/head//24 by client.2174595.0:18679 0.000000 2014-08-13 18:06:06.610778 7f5c973a57a0 20 read_log 194215'16791 (192834'16171) clean d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by osd.289.0:1812 2014-08-12 15:37:32.471059 2014-08-13 18:06:06.610791 7f5c973a57a0 20 read_log 194215'16792 (194215'16791) delete d6db545e/benchmark_data_storage0111-ib.pocstor_41526_object15600/head//24 by client.2174595.0:18681 0.000000 2014-08-13 18:06:06.610811 7f5c973a57a0 20 read_log 194215'16793 (192888'16579) clean 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by osd.289.0:1814 2014-08-12 15:37:32.478745 2014-08-13 18:06:06.610824 7f5c973a57a0 20 read_log 194215'16794 (194215'16793) delete 907645e/benchmark_data_storage0111-ib.pocstor_2228_object21796/head//24 by client.2174595.0:18683 0.000000 2014-08-13 18:06:06.610835 7f5c973a57a0 20 read_log 194215'16795 (192888'16405) clean 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by osd.289.0:1816 2014-08-12 15:37:32.485320 2014-08-13 18:06:06.610858 7f5c973a57a0 20 read_log 194215'16796 (194215'16795) delete 28c7645e/benchmark_data_storage0111-ib.pocstor_41097_object16999/head//24 by client.2174595.0:18685 0.000000 2014-08-13 18:06:06.610872 7f5c973a57a0 20 read_log 194215'16797 (192834'16132) clean 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by osd.289.0:1818 2014-08-12 15:37:32.492177 2014-08-13 18:06:06.610884 7f5c973a57a0 20 read_log 194215'16798 (194215'16797) delete 910d645e/benchmark_data_storage0111-ib.pocstor_41526_object10398/head//24 by client.2174595.0:18687 0.000000 2014-08-13 18:06:06.610895 7f5c973a57a0 20 read_log 194215'16799 (192834'15646) clean 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by osd.289.0:1820 2014-08-12 15:37:32.499073 2014-08-13 18:06:06.610908 7f5c973a57a0 20 read_log 194215'16800 (194215'16799) delete 1c18845e/benchmark_data_storage0111-ib.pocstor_26659_object9089/head//24 by client.2174595.0:18689 0.000000 2014-08-13 18:06:06.610918 7f5c973a57a0 20 read_log 194215'16801 (192888'16393) clean 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by osd.289.0:1822 2014-08-12 15:37:32.506632 2014-08-13 18:06:06.610930 7f5c973a57a0 20 read_log 194215'16802 (194215'16801) delete 9ec9845e/benchmark_data_storage0111-ib.pocstor_41097_object12904/head//24 by client.2174595.0:18691 0.000000 2014-08-13 18:06:06.610944 7f5c973a57a0 20 read_log 194215'16803 (192888'16396) clean 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by osd.289.0:1824 2014-08-12 15:37:32.513793 2014-08-13 18:06:06.610956 7f5c973a57a0 20 read_log 194215'16804 (194215'16803) delete 3c5b845e/benchmark_data_storage0111-ib.pocstor_41097_object13237/head//24 by client.2174595.0:18693 0.000000 2014-08-13 18:06:06.610967 7f5c973a57a0 20 read_log 194215'16805 (193305'16653) clean afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by osd.289.0:1826 2014-08-12 15:37:32.521767 2014-08-13 18:06:06.610980 7f5c973a57a0 20 read_log 194215'16806 (194215'16805) delete afa6945e/benchmark_data_storage0111-ib.pocstor_62621_object5247/head//24 by client.2174595.0:18695 0.000000 2014-08-13 18:06:06.610994 7f5c973a57a0 20 read_log 194215'16807 (193305'16654) clean f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by osd.289.0:1828 2014-08-12 15:37:32.529091 2014-08-13 18:06:06.611007 7f5c973a57a0 20 read_log 194215'16808 (194215'16807) delete f588945e/benchmark_data_storage0111-ib.pocstor_62621_object5467/head//24 by client.2174595.0:18697 0.000000 2014-08-13 18:06:06.611018 7f5c973a57a0 20 read_log 194215'16809 (192888'16561) clean 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by osd.289.0:1830 2014-08-12 15:37:32.535902 2014-08-13 18:06:06.611030 7f5c973a57a0 20 read_log 194215'16810 (194215'16809) delete 182ca45e/benchmark_data_storage0111-ib.pocstor_2228_object12588/head//24 by client.2174595.0:18699 0.000000 2014-08-13 18:06:06.611044 7f5c973a57a0 20 read_log 194215'16811 (192834'16163) clean cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by osd.289.0:1832 2014-08-12 15:37:32.542802 2014-08-13 18:06:06.611057 7f5c973a57a0 20 read_log 194215'16812 (194215'16811) delete cc21b45e/benchmark_data_storage0111-ib.pocstor_41526_object14634/head//24 by client.2174595.0:18701 0.000000 2014-08-13 18:06:06.611069 7f5c973a57a0 20 read_log 194215'16813 (192834'16203) clean 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by osd.289.0:1834 2014-08-12 15:37:32.550050 2014-08-13 18:06:06.611093 7f5c973a57a0 20 read_log 194215'16814 (194215'16813) delete 82dcb45e/benchmark_data_storage0111-ib.pocstor_41526_object20092/head//24 by client.2174595.0:18703 0.000000 2014-08-13 18:06:06.611105 7f5c973a57a0 20 read_log 194215'16815 (193305'16659) clean 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by osd.289.0:1836 2014-08-12 15:37:32.556668 2014-08-13 18:06:06.611132 7f5c973a57a0 20 read_log 194215'16816 (194215'16815) delete 26fcb45e/benchmark_data_storage0111-ib.pocstor_62621_object10971/head//24 by client.2174595.0:18705 0.000000 2014-08-13 18:06:06.611145 7f5c973a57a0 20 read_log 194215'16817 (192888'16593) clean 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by osd.289.0:1838 2014-08-12 15:37:32.563447 2014-08-13 18:06:06.611163 7f5c973a57a0 20 read_log 194215'16818 (194215'16817) delete 95a2c45e/benchmark_data_storage0111-ib.pocstor_2228_object29338/head//24 by client.2174595.0:18707 0.000000 2014-08-13 18:06:06.611183 7f5c973a57a0 20 read_log 194215'16819 (192888'16576) clean 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by osd.289.0:1840 2014-08-12 15:37:32.570745 2014-08-13 18:06:06.611196 7f5c973a57a0 20 read_log 194215'16820 (194215'16819) delete 346c45e/benchmark_data_storage0111-ib.pocstor_2228_object19126/head//24 by client.2174595.0:18709 0.000000 2014-08-13 18:06:06.611207 7f5c973a57a0 20 read_log 194215'16821 (192888'16370) clean 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by osd.289.0:1842 2014-08-12 15:37:32.578296 2014-08-13 18:06:06.611220 7f5c973a57a0 20 read_log 194215'16822 (194215'16821) delete 401ec45e/benchmark_data_storage0111-ib.pocstor_41097_object1914/head//24 by client.2174595.0:18711 0.000000 2014-08-13 18:06:06.611238 7f5c973a57a0 20 read_log 194215'16823 (193305'16655) clean 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by osd.289.0:1844 2014-08-12 15:37:32.585499 2014-08-13 18:06:06.611257 7f5c973a57a0 20 read_log 194215'16824 (194215'16823) delete 5c00e45e/benchmark_data_storage0111-ib.pocstor_62621_object7091/head//24 by client.2174595.0:18713 0.000000 2014-08-13 18:06:06.611268 7f5c973a57a0 20 read_log 194215'16825 (192888'16431) clean 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by osd.289.0:1846 2014-08-12 15:37:32.592882 2014-08-13 18:06:06.611298 7f5c973a57a0 20 read_log 194215'16826 (194215'16825) delete 8228e45e/benchmark_data_storage0111-ib.pocstor_41097_object28886/head//24 by client.2174595.0:18715 0.000000 2014-08-13 18:06:06.611312 7f5c973a57a0 20 read_log 194215'16827 (194209'16685) delete 4fca685e/benchmark_data_storage0111-ib.pocstor_2228_object9820/head//24 by client.2174595.0:18717 0.000000 2014-08-13 18:06:06.611323 7f5c973a57a0 20 read_log 194215'16828 (194209'16687) delete e500785e/benchmark_data_storage0111-ib.pocstor_2228_object3238/head//24 by client.2174595.0:18719 0.000000 2014-08-13 18:06:06.611334 7f5c973a57a0 20 read_log 194215'16829 (194209'16689) delete 31bb785e/benchmark_data_storage0111-ib.pocstor_2228_object16799/head//24 by client.2174595.0:18721 0.000000 2014-08-13 18:06:06.611348 7f5c973a57a0 20 read_log 194215'16830 (194209'16690) delete 22bc785e/benchmark_data_storage0111-ib.pocstor_2228_object14693/head//24 by client.2174595.0:18723 0.000000 2014-08-13 18:06:06.611454 7f5c973a57a0 20 read_log 194215'16831 (194209'16694) delete edb7b85e/benchmark_data_storage0111-ib.pocstor_2228_object11338/head//24 by client.2174595.0:18725 0.000000 2014-08-13 18:06:06.611500 7f5c973a57a0 20 read_log 194215'16832 (194209'16716) delete 1615c85e/benchmark_data_storage0111-ib.pocstor_62621_object29242/head//24 by client.2174595.0:18727 0.000000 2014-08-13 18:06:06.611553 7f5c973a57a0 20 read_log 194215'16833 (192888'16406) clean 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by osd.289.0:1854 2014-08-12 15:37:32.632279 2014-08-13 18:06:06.611568 7f5c973a57a0 20 read_log 194215'16834 (194215'16833) delete 8c38c85e/benchmark_data_storage0111-ib.pocstor_41097_object17878/head//24 by client.2174595.0:18729 0.000000 2014-08-13 18:06:06.611579 7f5c973a57a0 20 read_log 194215'16835 (192888'16423) clean dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by osd.289.0:1856 2014-08-12 15:37:32.639631 2014-08-13 18:06:06.611608 7f5c973a57a0 20 read_log 194215'16836 (194215'16835) delete dc88c85e/benchmark_data_storage0111-ib.pocstor_41097_object27753/head//24 by client.2174595.0:18731 0.000000 2014-08-13 18:06:06.611636 7f5c973a57a0 20 read_log 194215'16837 (192834'15670) clean e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by osd.289.0:1858 2014-08-12 15:37:32.646420 2014-08-13 18:06:06.611663 7f5c973a57a0 20 read_log 194215'16838 (194215'16837) delete e8ebc85e/benchmark_data_storage0111-ib.pocstor_26659_object13650/head//24 by client.2174595.0:18733 0.000000 2014-08-13 18:06:06.611675 7f5c973a57a0 20 read_log 194215'16839 (192834'15648) clean 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by osd.289.0:1860 2014-08-12 15:37:32.653330 2014-08-13 18:06:06.611691 7f5c973a57a0 20 read_log 194215'16840 (194215'16839) delete 84dbd85e/benchmark_data_storage0111-ib.pocstor_26659_object9278/head//24 by client.2174595.0:18735 0.000000 2014-08-13 18:06:06.611705 7f5c973a57a0 20 read_log 194215'16841 (192888'16415) clean 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by osd.289.0:1862 2014-08-12 15:37:32.660429 2014-08-13 18:06:06.611721 7f5c973a57a0 20 read_log 194215'16842 (194215'16841) delete 84cfd85e/benchmark_data_storage0111-ib.pocstor_41097_object24230/head//24 by client.2174595.0:18737 0.000000 2014-08-13 18:06:06.611732 7f5c973a57a0 20 read_log 194215'16843 (193305'16668) clean 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by osd.289.0:1864 2014-08-12 15:37:32.667744 2014-08-13 18:06:06.611745 7f5c973a57a0 20 read_log 194215'16844 (194215'16843) delete 1417e85e/benchmark_data_storage0111-ib.pocstor_62621_object23770/head//24 by client.2174595.0:18739 0.000000 2014-08-13 18:06:06.611755 7f5c973a57a0 20 read_log 194215'16845 (192888'16559) clean d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by osd.289.0:1866 2014-08-12 15:37:32.674854 2014-08-13 18:06:06.611768 7f5c973a57a0 20 read_log 194215'16846 (194215'16845) delete d6f8e85e/benchmark_data_storage0111-ib.pocstor_2228_object11816/head//24 by client.2174595.0:18741 0.000000 2014-08-13 18:06:06.611779 7f5c973a57a0 20 read_log 194215'16847 (192888'16584) clean ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by osd.289.0:1868 2014-08-12 15:37:32.681667 2014-08-13 18:06:06.611810 7f5c973a57a0 20 read_log 194215'16848 (194215'16847) delete ab2df85e/benchmark_data_storage0111-ib.pocstor_2228_object23959/head//24 by client.2174595.0:18743 0.000000 2014-08-13 18:06:06.611829 7f5c973a57a0 20 read_log 194215'16849 (193305'16665) clean a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by osd.289.0:1870 2014-08-12 15:37:32.688391 2014-08-13 18:06:06.611844 7f5c973a57a0 20 read_log 194215'16850 (194215'16849) delete a2400c5e/benchmark_data_storage0111-ib.pocstor_62621_object22423/head//24 by client.2174595.0:18745 0.000000 2014-08-13 18:06:06.611859 7f5c973a57a0 20 read_log 194215'16851 (192888'16395) clean e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by osd.289.0:1872 2014-08-12 15:37:32.695606 2014-08-13 18:06:06.611872 7f5c973a57a0 20 read_log 194215'16852 (194215'16851) delete e4391c5e/benchmark_data_storage0111-ib.pocstor_41097_object13064/head//24 by client.2174595.0:18747 0.000000 2014-08-13 18:06:06.611893 7f5c973a57a0 20 read_log 194215'16853 (192834'15642) clean db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by osd.289.0:1874 2014-08-12 15:37:32.702979 2014-08-13 18:06:06.611910 7f5c973a57a0 20 read_log 194215'16854 (194215'16853) delete db2d2c5e/benchmark_data_storage0111-ib.pocstor_26659_object8687/head//24 by client.2174595.0:18749 0.000000 2014-08-13 18:06:06.611921 7f5c973a57a0 20 read_log 194215'16855 (192834'15694) clean 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by osd.289.0:1876 2014-08-12 15:37:32.709891 2014-08-13 18:06:06.611965 7f5c973a57a0 20 read_log 194215'16856 (194215'16855) delete 1d613c5e/benchmark_data_storage0111-ib.pocstor_26659_object16852/head//24 by client.2174595.0:18751 0.000000 2014-08-13 18:06:06.611995 7f5c973a57a0 20 read_log 194215'16857 (192834'15590) clean e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by osd.289.0:1878 2014-08-12 15:37:32.717047 2014-08-13 18:06:06.612011 7f5c973a57a0 20 read_log 194215'16858 (194215'16857) delete e1924c5e/benchmark_data_storage0111-ib.pocstor_26659_object2066/head//24 by client.2174595.0:18753 0.000000 2014-08-13 18:06:06.612022 7f5c973a57a0 20 read_log 194215'16859 (193305'16660) clean cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by osd.289.0:1880 2014-08-12 15:37:32.724650 2014-08-13 18:06:06.612035 7f5c973a57a0 20 read_log 194215'16860 (194215'16859) delete cb084c5e/benchmark_data_storage0111-ib.pocstor_62621_object19918/head//24 by client.2174595.0:18755 0.000000 2014-08-13 18:06:06.612046 7f5c973a57a0 20 read_log 194215'16861 (192834'16111) clean bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by osd.289.0:1882 2014-08-12 15:37:32.732206 2014-08-13 18:06:06.612058 7f5c973a57a0 20 read_log 194215'16862 (194215'16861) delete bdea4c5e/benchmark_data_storage0111-ib.pocstor_41526_object7407/head//24 by client.2174595.0:18757 0.000000 2014-08-13 18:06:06.612069 7f5c973a57a0 20 read_log 194215'16863 (192834'16070) clean 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by osd.289.0:1884 2014-08-12 15:37:32.739303 2014-08-13 18:06:06.612081 7f5c973a57a0 20 read_log 194215'16864 (194215'16863) delete 77c05c5e/benchmark_data_storage0111-ib.pocstor_41526_object1953/head//24 by client.2174595.0:18759 0.000000 2014-08-13 18:06:06.612092 7f5c973a57a0 20 read_log 194215'16865 (192834'16214) clean 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by osd.289.0:1886 2014-08-12 15:37:32.746036 2014-08-13 18:06:06.612104 7f5c973a57a0 20 read_log 194215'16866 (194215'16865) delete 86f95c5e/benchmark_data_storage0111-ib.pocstor_41526_object21322/head//24 by client.2174595.0:18761 0.000000 2014-08-13 18:06:06.612115 7f5c973a57a0 20 read_log 194215'16867 (193305'16650) clean 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by osd.289.0:1888 2014-08-12 15:37:32.752968 2014-08-13 18:06:06.612127 7f5c973a57a0 20 read_log 194215'16868 (194215'16867) delete 8f7f5c5e/benchmark_data_storage0111-ib.pocstor_62621_object3777/head//24 by client.2174595.0:18763 0.000000 2014-08-13 18:06:06.612138 7f5c973a57a0 20 read_log 194215'16869 (192888'16404) clean 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by osd.289.0:1890 2014-08-12 15:37:32.759680 2014-08-13 18:06:06.612157 7f5c973a57a0 20 read_log 194215'16870 (194215'16869) delete 80856c5e/benchmark_data_storage0111-ib.pocstor_41097_object16975/head//24 by client.2174595.0:18765 0.000000 2014-08-13 18:06:06.612172 7f5c973a57a0 20 read_log 194215'16871 (192834'15621) clean 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by osd.289.0:1892 2014-08-12 15:37:32.766788 2014-08-13 18:06:06.612185 7f5c973a57a0 20 read_log 194215'16872 (194215'16871) delete 586e6c5e/benchmark_data_storage0111-ib.pocstor_26659_object5677/head//24 by client.2174595.0:18767 0.000000 2014-08-13 18:06:06.612196 7f5c973a57a0 20 read_log 194215'16873 (192888'16397) clean d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by osd.289.0:1894 2014-08-12 15:37:32.773234 2014-08-13 18:06:06.612211 7f5c973a57a0 20 read_log 194215'16874 (194215'16873) delete d5f67c5e/benchmark_data_storage0111-ib.pocstor_41097_object13269/head//24 by client.2174595.0:18769 0.000000 2014-08-13 18:06:06.612222 7f5c973a57a0 20 read_log 194215'16875 (192888'16399) clean 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by osd.289.0:1896 2014-08-12 15:37:32.779912 2014-08-13 18:06:06.612241 7f5c973a57a0 20 read_log 194215'16876 (194215'16875) delete 2268c5e/benchmark_data_storage0111-ib.pocstor_41097_object13761/head//24 by client.2174595.0:18771 0.000000 2014-08-13 18:06:06.612256 7f5c973a57a0 20 read_log 194215'16877 (192888'16369) clean d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by osd.289.0:1898 2014-08-12 15:37:32.786045 2014-08-13 18:06:06.612269 7f5c973a57a0 20 read_log 194215'16878 (194215'16877) delete d02c8c5e/benchmark_data_storage0111-ib.pocstor_41097_object1823/head//24 by client.2174595.0:18773 0.000000 2014-08-13 18:06:06.612280 7f5c973a57a0 20 read_log 194215'16879 (192888'16387) clean 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by osd.289.0:1900 2014-08-12 15:37:32.793007 2014-08-13 18:06:06.612292 7f5c973a57a0 20 read_log 194215'16880 (194215'16879) delete 7e9d8c5e/benchmark_data_storage0111-ib.pocstor_41097_object10930/head//24 by client.2174595.0:18775 0.000000 2014-08-13 18:06:06.612306 7f5c973a57a0 20 read_log 194215'16881 (192888'16411) clean 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by osd.289.0:1902 2014-08-12 15:37:32.802197 2014-08-13 18:06:06.612318 7f5c973a57a0 20 read_log 194215'16882 (194215'16881) delete 4ee2ac5e/benchmark_data_storage0111-ib.pocstor_41097_object21159/head//24 by client.2174595.0:18777 0.000000 2014-08-13 18:06:06.612329 7f5c973a57a0 20 read_log 194215'16883 (192888'16420) clean bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by osd.289.0:1904 2014-08-12 15:37:32.809610 2014-08-13 18:06:06.612342 7f5c973a57a0 20 read_log 194215'16884 (194215'16883) delete bea8ac5e/benchmark_data_storage0111-ib.pocstor_41097_object27309/head//24 by client.2174595.0:18779 0.000000 2014-08-13 18:06:06.612356 7f5c973a57a0 20 read_log 194215'16885 (192888'16410) clean ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by osd.289.0:1906 2014-08-12 15:37:32.817103 2014-08-13 18:06:06.612369 7f5c973a57a0 20 read_log 194215'16886 (194215'16885) delete ed34bc5e/benchmark_data_storage0111-ib.pocstor_41097_object20276/head//24 by client.2174595.0:18781 0.000000 2014-08-13 18:06:06.612390 7f5c973a57a0 20 read_log 194215'16887 (193305'16656) clean 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by osd.289.0:1908 2014-08-12 15:37:32.824253 2014-08-13 18:06:06.612410 7f5c973a57a0 20 read_log 194215'16888 (194215'16887) delete 334abc5e/benchmark_data_storage0111-ib.pocstor_62621_object8700/head//24 by client.2174595.0:18783 0.000000 2014-08-13 18:06:06.612426 7f5c973a57a0 20 read_log 194215'16889 (193305'16657) clean 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by osd.289.0:1910 2014-08-12 15:37:32.830765 2014-08-13 18:06:06.612441 7f5c973a57a0 20 read_log 194215'16890 (194215'16889) delete 14ccbc5e/benchmark_data_storage0111-ib.pocstor_62621_object9695/head//24 by client.2174595.0:18785 0.000000 2014-08-13 18:06:06.612453 7f5c973a57a0 20 read_log 194215'16891 (192834'15720) clean cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by osd.289.0:1912 2014-08-12 15:37:32.837545 2014-08-13 18:06:06.612465 7f5c973a57a0 20 read_log 194215'16892 (194215'16891) delete cec9cc5e/benchmark_data_storage0111-ib.pocstor_26659_object21601/head//24 by client.2174595.0:18787 0.000000 2014-08-13 18:06:06.612486 7f5c973a57a0 20 read_log 194215'16893 (193305'16663) clean 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by osd.289.0:1914 2014-08-12 15:37:32.844717 2014-08-13 18:06:06.612500 7f5c973a57a0 20 read_log 194215'16894 (194215'16893) delete 1220dc5e/benchmark_data_storage0111-ib.pocstor_62621_object21068/head//24 by client.2174595.0:18789 0.000000 2014-08-13 18:06:06.612511 7f5c973a57a0 20 read_log 194215'16895 (192888'16421) clean 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by osd.289.0:1916 2014-08-12 15:37:32.851990 2014-08-13 18:06:06.612540 7f5c973a57a0 20 read_log 194215'16896 (194215'16895) delete 4d77dc5e/benchmark_data_storage0111-ib.pocstor_41097_object27332/head//24 by client.2174595.0:18791 0.000000 2014-08-13 18:06:06.612554 7f5c973a57a0 20 read_log 194215'16897 (192834'16095) clean a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by osd.289.0:1918 2014-08-12 15:37:32.859422 2014-08-13 18:06:06.612567 7f5c973a57a0 20 read_log 194215'16898 (194215'16897) delete a06edc5e/benchmark_data_storage0111-ib.pocstor_41526_object5341/head//24 by client.2174595.0:18793 0.000000 2014-08-13 18:06:06.612584 7f5c973a57a0 20 read_log 194215'16899 (192888'16570) clean eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by osd.289.0:1920 2014-08-12 15:37:32.866299 2014-08-13 18:06:06.612608 7f5c973a57a0 20 read_log 194215'16900 (194215'16899) delete eac0ec5e/benchmark_data_storage0111-ib.pocstor_2228_object16535/head//24 by client.2174595.0:18795 0.000000 2014-08-13 18:06:06.612619 7f5c973a57a0 20 read_log 194215'16901 (192888'16380) clean bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by osd.289.0:1922 2014-08-12 15:37:32.874164 2014-08-13 18:06:06.612632 7f5c973a57a0 20 read_log 194215'16902 (194215'16901) delete bf4bec5e/benchmark_data_storage0111-ib.pocstor_41097_object9063/head//24 by client.2174595.0:18797 0.000000 2014-08-13 18:06:06.612643 7f5c973a57a0 20 read_log 194215'16903 (192834'15609) clean fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by osd.289.0:1924 2014-08-12 15:37:32.881603 2014-08-13 18:06:06.612655 7f5c973a57a0 20 read_log 194215'16904 (194215'16903) delete fb05fc5e/benchmark_data_storage0111-ib.pocstor_26659_object4504/head//24 by client.2174595.0:18799 0.000000 2014-08-13 18:06:06.612666 7f5c973a57a0 20 read_log 194215'16905 (192888'16590) clean 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by osd.289.0:1926 2014-08-12 15:37:32.889643 2014-08-13 18:06:06.612678 7f5c973a57a0 20 read_log 194215'16906 (194215'16905) delete 5ad7fc5e/benchmark_data_storage0111-ib.pocstor_2228_object28268/head//24 by client.2174595.0:18801 0.000000 2014-08-13 18:06:06.612689 7f5c973a57a0 20 read_log 194215'16907 (192888'16595) clean b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by osd.289.0:1928 2014-08-12 15:37:32.897113 2014-08-13 18:06:06.612701 7f5c973a57a0 20 read_log 194215'16908 (194215'16907) delete b71dfc5e/benchmark_data_storage0111-ib.pocstor_2228_object29617/head//24 by client.2174595.0:18803 0.000000 2014-08-13 18:06:06.612721 7f5c973a57a0 20 read_log 194217'16909 (194217'16909) modify 5e/hit_set_24.5e_archive_2014-08-12 15:37:32.211135_2014-08-12 17:25:03.987810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 18:06:06.612739 7f5c973a57a0 20 read_log 194217'16910 (194215'16717) delete 5e/hit_set_24.5e_archive_2014-08-12 15:32:25.855993_2014-08-12 15:37:32.210894/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 18:06:06.612759 7f5c973a57a0 20 read_log 194217'16911 (0'0) promote 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by osd.289.0:3614 2014-08-12 17:25:03.991195 2014-08-13 18:06:06.612776 7f5c973a57a0 20 read_log 194217'16912 (194217'16911) modify 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by client.1487927.0:177 2014-08-12 17:25:03.982296 2014-08-13 18:06:06.612793 7f5c973a57a0 20 read_log 194370'16913 (0'0) promote 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by osd.289.0:3630 2014-08-12 20:30:15.306267 2014-08-13 18:06:06.612806 7f5c973a57a0 20 read_log 194370'16914 (194370'16913) modify 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by client.1508531.0:276 2014-08-12 20:30:15.299339 2014-08-13 18:06:06.612832 7f5c973a57a0 20 read_log 194370'16915 (0'0) promote 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by osd.289.0:3632 2014-08-12 20:30:15.412195 2014-08-13 18:06:06.612848 7f5c973a57a0 20 read_log 194370'16916 (194370'16915) modify 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by client.1508531.0:303 2014-08-12 20:30:15.404611 2014-08-13 18:06:06.612861 7f5c973a57a0 20 read_log 194370'16917 (0'0) promote 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by osd.289.0:3644 2014-08-12 20:30:16.477939 2014-08-13 18:06:06.612873 7f5c973a57a0 20 read_log 194370'16918 (194370'16917) modify 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by client.1508531.0:545 2014-08-12 20:30:16.471772 2014-08-13 18:06:06.612900 7f5c973a57a0 20 read_log 194370'16919 (0'0) promote 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by osd.289.0:3662 2014-08-12 20:30:18.701982 2014-08-13 18:06:06.612916 7f5c973a57a0 20 read_log 194370'16920 (194370'16919) modify 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by client.1508531.0:1158 2014-08-12 20:30:18.692375 2014-08-13 18:06:06.612929 7f5c973a57a0 20 read_log 194370'16921 (0'0) promote 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by osd.289.0:3666 2014-08-12 20:30:19.445130 2014-08-13 18:06:06.612941 7f5c973a57a0 20 read_log 194370'16922 (194370'16921) modify 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by client.1508531.0:1349 2014-08-12 20:30:19.437629 2014-08-13 18:06:06.612953 7f5c973a57a0 20 read_log 194370'16923 (0'0) promote c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by osd.289.0:3674 2014-08-12 20:30:21.042607 2014-08-13 18:06:06.612965 7f5c973a57a0 20 read_log 194370'16924 (194370'16923) modify c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by client.1508531.0:1783 2014-08-12 20:30:21.032416 2014-08-13 18:06:06.612978 7f5c973a57a0 20 read_log 194370'16925 (0'0) promote c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by osd.289.0:3676 2014-08-12 20:30:21.088164 2014-08-13 18:06:06.612990 7f5c973a57a0 20 read_log 194370'16926 (194370'16925) modify c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by client.1508531.0:1795 2014-08-12 20:30:21.077929 2014-08-13 18:06:06.613002 7f5c973a57a0 20 read_log 194370'16927 (0'0) promote 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by osd.289.0:3750 2014-08-12 20:30:39.341360 2014-08-13 18:06:06.613014 7f5c973a57a0 20 read_log 194370'16928 (194370'16927) modify 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by client.1508531.0:6370 2014-08-12 20:30:39.333782 2014-08-13 18:06:06.613036 7f5c973a57a0 20 read_log 194370'16929 (0'0) promote 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by osd.289.0:3770 2014-08-12 20:30:43.532904 2014-08-13 18:06:06.613049 7f5c973a57a0 20 read_log 194370'16930 (194370'16929) modify 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by client.1508531.0:7235 2014-08-12 20:30:43.520542 2014-08-13 18:06:06.613061 7f5c973a57a0 20 read_log 194370'16931 (0'0) promote d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by osd.289.0:3776 2014-08-12 20:30:44.573781 2014-08-13 18:06:06.613073 7f5c973a57a0 20 read_log 194370'16932 (194370'16931) modify d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by client.1508531.0:7443 2014-08-12 20:30:44.563223 2014-08-13 18:06:06.613086 7f5c973a57a0 20 read_log 194370'16933 (0'0) promote d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by osd.289.0:3792 2014-08-12 20:30:50.836699 2014-08-13 18:06:06.613098 7f5c973a57a0 20 read_log 194370'16934 (194370'16933) modify d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by client.1508531.0:8596 2014-08-12 20:30:50.829702 2014-08-13 18:06:06.613118 7f5c973a57a0 20 read_log 194370'16935 (0'0) promote eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by osd.289.0:3816 2014-08-12 20:30:58.707828 2014-08-13 18:06:06.613133 7f5c973a57a0 20 read_log 194370'16936 (194370'16935) modify eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by client.1508531.0:9903 2014-08-12 20:30:58.699187 2014-08-13 18:06:06.613154 7f5c973a57a0 20 read_log 194370'16937 (0'0) promote aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by osd.289.0:3824 2014-08-12 20:31:00.847698 2014-08-13 18:06:06.613170 7f5c973a57a0 20 read_log 194370'16938 (194370'16937) modify aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by client.1508531.0:10270 2014-08-12 20:31:00.838695 2014-08-13 18:06:06.613182 7f5c973a57a0 20 read_log 194370'16939 (0'0) promote 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by osd.289.0:3830 2014-08-12 20:31:02.605182 2014-08-13 18:06:06.613195 7f5c973a57a0 20 read_log 194370'16940 (194370'16939) modify 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by client.1508531.0:10561 2014-08-12 20:31:02.595811 2014-08-13 18:06:06.613231 7f5c973a57a0 20 read_log 194370'16941 (0'0) promote 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by osd.289.0:3876 2014-08-12 20:31:19.931553 2014-08-13 18:06:06.613247 7f5c973a57a0 20 read_log 194370'16942 (194370'16941) modify 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by client.1508531.0:13061 2014-08-12 20:31:19.922980 2014-08-13 18:06:06.613259 7f5c973a57a0 20 read_log 194370'16943 (0'0) promote 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by osd.289.0:3894 2014-08-12 20:31:30.063409 2014-08-13 18:06:06.613271 7f5c973a57a0 20 read_log 194370'16944 (194370'16943) modify 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by client.1508531.0:13921 2014-08-12 20:31:30.054866 2014-08-13 18:06:06.613287 7f5c973a57a0 20 read_log 194377'16945 (0'0) promote da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by osd.289.0:3898 2014-08-12 20:32:57.928655 2014-08-13 18:06:06.613300 7f5c973a57a0 20 read_log 194377'16946 (194377'16945) modify da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by client.2231190.0:182 2014-08-12 20:32:57.922328 2014-08-13 18:06:06.613312 7f5c973a57a0 20 read_log 194377'16947 (0'0) promote 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by osd.289.0:3902 2014-08-12 20:32:58.158659 2014-08-13 18:06:06.613327 7f5c973a57a0 20 read_log 194377'16948 (194377'16947) modify 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by client.2231190.0:243 2014-08-12 20:32:58.153044 2014-08-13 18:06:06.613340 7f5c973a57a0 20 read_log 194377'16949 (0'0) promote cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by osd.289.0:3932 2014-08-12 20:33:04.336264 2014-08-13 18:06:06.613361 7f5c973a57a0 20 read_log 194377'16950 (194377'16949) modify cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by client.2231190.0:1778 2014-08-12 20:33:04.328244 2014-08-13 18:06:06.613377 7f5c973a57a0 20 read_log 194377'16951 (0'0) promote 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by osd.289.0:3948 2014-08-12 20:33:06.994988 2014-08-13 18:06:06.613399 7f5c973a57a0 20 read_log 194377'16952 (194377'16951) modify 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by client.2231190.0:2443 2014-08-12 20:33:06.984933 2014-08-13 18:06:06.613415 7f5c973a57a0 20 read_log 194377'16953 (0'0) promote 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by osd.289.0:3958 2014-08-12 20:33:08.787056 2014-08-13 18:06:06.613428 7f5c973a57a0 20 read_log 194377'16954 (194377'16953) modify 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by client.2231190.0:2875 2014-08-12 20:33:08.779446 2014-08-13 18:06:06.613448 7f5c973a57a0 20 read_log 194377'16955 (0'0) promote 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by osd.289.0:3978 2014-08-12 20:33:12.084534 2014-08-13 18:06:06.613463 7f5c973a57a0 20 read_log 194377'16956 (194377'16955) modify 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by client.2231190.0:3644 2014-08-12 20:33:12.077222 2014-08-13 18:06:06.613478 7f5c973a57a0 20 read_log 194377'16957 (0'0) promote 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by osd.289.0:3984 2014-08-12 20:33:13.879580 2014-08-13 18:06:06.613491 7f5c973a57a0 20 read_log 194377'16958 (194377'16957) modify 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by client.2231190.0:3993 2014-08-12 20:33:13.871436 2014-08-13 18:06:06.613503 7f5c973a57a0 20 read_log 194383'16959 (0'0) promote b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by osd.289.0:4044 2014-08-12 20:34:30.647679 2014-08-13 18:06:06.613528 7f5c973a57a0 20 read_log 194383'16960 (194383'16959) modify b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by client.2231190.0:7569 2014-08-12 20:34:30.638584 2014-08-13 18:06:06.613543 7f5c973a57a0 20 read_log 194393'16961 (0'0) promote 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by osd.289.0:4072 2014-08-12 20:49:26.719414 2014-08-13 18:06:06.613556 7f5c973a57a0 20 read_log 194393'16962 (194393'16961) modify 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by client.1064182.0:265 2014-08-12 20:49:26.711262 2014-08-13 18:06:06.613568 7f5c973a57a0 20 read_log 194393'16963 (0'0) promote c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by osd.289.0:4093 2014-08-12 20:52:56.726134 2014-08-13 18:06:06.613581 7f5c973a57a0 20 read_log 194393'16964 (194393'16963) modify c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by client.2235069.0:85 2014-08-12 20:52:56.719683 2014-08-13 18:06:06.613593 7f5c973a57a0 20 read_log 194416'16965 (0'0) promote 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by osd.289.0:4281 2014-08-13 15:28:35.502228 2014-08-13 18:06:06.613605 7f5c973a57a0 20 read_log 194416'16966 (194416'16965) modify 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by client.2441958.0:1260 2014-08-13 15:28:35.492974 2014-08-13 18:06:06.613623 7f5c973a57a0 20 read_log 194416'16967 (0'0) promote 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by osd.289.0:4314 2014-08-13 15:28:42.456735 2014-08-13 18:06:06.613638 7f5c973a57a0 20 read_log 194416'16968 (194416'16967) modify 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by client.2441958.0:3101 2014-08-13 15:28:42.448145 2014-08-13 18:06:06.613650 7f5c973a57a0 20 read_log 194416'16969 (0'0) promote 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by osd.289.0:4334 2014-08-13 15:28:46.553242 2014-08-13 18:06:06.613662 7f5c973a57a0 20 read_log 194416'16970 (194416'16969) modify 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by client.2441958.0:4159 2014-08-13 15:28:46.543697 2014-08-13 18:06:06.613675 7f5c973a57a0 20 read_log 194416'16971 (0'0) promote 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by osd.289.0:4340 2014-08-13 15:28:47.641557 2014-08-13 18:06:06.613687 7f5c973a57a0 20 read_log 194416'16972 (194416'16971) modify 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by client.2441958.0:4439 2014-08-13 15:28:47.634245 2014-08-13 18:06:06.613699 7f5c973a57a0 20 read_log 194416'16973 (0'0) promote 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by osd.289.0:4342 2014-08-13 15:28:48.797392 2014-08-13 18:06:06.613721 7f5c973a57a0 20 read_log 194416'16974 (194416'16973) modify 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by client.2441958.0:4710 2014-08-13 15:28:48.788974 2014-08-13 18:06:06.613734 7f5c973a57a0 20 read_log 194416'16975 (0'0) promote 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by osd.289.0:4348 2014-08-13 15:28:49.959265 2014-08-13 18:06:06.613746 7f5c973a57a0 20 read_log 194416'16976 (194416'16975) modify 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by client.2441958.0:5018 2014-08-13 15:28:49.950732 2014-08-13 18:06:06.613758 7f5c973a57a0 20 read_log 194416'16977 (0'0) promote 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by osd.289.0:4358 2014-08-13 15:28:51.486788 2014-08-13 18:06:06.613770 7f5c973a57a0 20 read_log 194416'16978 (194416'16977) modify 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by client.2441958.0:5418 2014-08-13 15:28:51.478489 2014-08-13 18:06:06.613782 7f5c973a57a0 20 read_log 194416'16979 (0'0) promote 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by osd.289.0:4365 2014-08-13 15:28:52.833508 2014-08-13 18:06:06.613794 7f5c973a57a0 20 read_log 194416'16980 (194416'16979) modify 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by client.2441958.0:5763 2014-08-13 15:28:52.824627 2014-08-13 18:06:06.613807 7f5c973a57a0 20 read_log 194416'16981 (0'0) promote 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by osd.289.0:4371 2014-08-13 15:28:53.238124 2014-08-13 18:06:06.613819 7f5c973a57a0 20 read_log 194416'16982 (194416'16981) modify 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by client.2441958.0:5869 2014-08-13 15:28:53.229339 2014-08-13 18:06:06.613831 7f5c973a57a0 20 read_log 194416'16983 (0'0) promote 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by osd.289.0:4385 2014-08-13 15:28:55.628268 2014-08-13 18:06:06.613843 7f5c973a57a0 20 read_log 194416'16984 (194416'16983) modify 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by client.2441958.0:6434 2014-08-13 15:28:55.619015 2014-08-13 18:06:06.613855 7f5c973a57a0 20 read_log 194416'16985 (0'0) promote 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by osd.289.0:4399 2014-08-13 15:28:59.726840 2014-08-13 18:06:06.613866 7f5c973a57a0 20 read_log 194416'16986 (194416'16985) modify 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by client.2441958.0:7520 2014-08-13 15:28:59.718424 2014-08-13 18:06:06.613878 7f5c973a57a0 20 read_log 194416'16987 (0'0) promote 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by osd.289.0:4405 2014-08-13 15:29:00.669794 2014-08-13 18:06:06.613890 7f5c973a57a0 20 read_log 194416'16988 (194416'16987) modify 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by client.2441958.0:7771 2014-08-13 15:29:00.661735 2014-08-13 18:06:06.613902 7f5c973a57a0 20 read_log 194416'16989 (0'0) promote a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by osd.289.0:4421 2014-08-13 15:29:05.020705 2014-08-13 18:06:06.613915 7f5c973a57a0 20 read_log 194416'16990 (194416'16989) modify a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by client.2441958.0:8899 2014-08-13 15:29:05.011810 2014-08-13 18:06:06.613927 7f5c973a57a0 20 read_log 194416'16991 (0'0) promote e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by osd.289.0:4473 2014-08-13 15:29:17.814759 2014-08-13 18:06:06.613939 7f5c973a57a0 20 read_log 194416'16992 (194416'16991) modify e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by client.2441958.0:12046 2014-08-13 15:29:17.806207 2014-08-13 18:06:06.613951 7f5c973a57a0 20 read_log 194416'16993 (0'0) promote 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by osd.289.0:4475 2014-08-13 15:29:17.842191 2014-08-13 18:06:06.613988 7f5c973a57a0 20 read_log 194416'16994 (194416'16993) modify 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by client.2441958.0:12052 2014-08-13 15:29:17.834093 2014-08-13 18:06:06.614005 7f5c973a57a0 20 read_log 194416'16995 (0'0) promote 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by osd.289.0:4487 2014-08-13 15:29:20.235649 2014-08-13 18:06:06.614017 7f5c973a57a0 20 read_log 194416'16996 (194416'16995) modify 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by client.2441958.0:12621 2014-08-13 15:29:20.226221 2014-08-13 18:06:06.614029 7f5c973a57a0 20 read_log 194416'16997 (0'0) promote 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by osd.289.0:4503 2014-08-13 15:29:24.175799 2014-08-13 18:06:06.614044 7f5c973a57a0 20 read_log 194416'16998 (194416'16997) modify 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by client.2441958.0:13516 2014-08-13 15:29:24.166796 2014-08-13 18:06:06.614056 7f5c973a57a0 20 read_log 194416'16999 (0'0) promote 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by osd.289.0:4535 2014-08-13 15:29:34.392361 2014-08-13 18:06:06.614070 7f5c973a57a0 20 read_log 194416'17000 (194416'16999) modify 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by client.2441958.0:15803 2014-08-13 15:29:34.382066 2014-08-13 18:06:06.614084 7f5c973a57a0 20 read_log 194416'17001 (0'0) promote 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by osd.289.0:4543 2014-08-13 15:29:39.440499 2014-08-13 18:06:06.614098 7f5c973a57a0 20 read_log 194416'17002 (194416'17001) modify 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by client.2441958.0:16443 2014-08-13 15:29:39.431217 2014-08-13 18:06:06.614110 7f5c973a57a0 20 read_log 194416'17003 (0'0) promote 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by osd.289.0:4545 2014-08-13 15:29:40.828203 2014-08-13 18:06:06.614122 7f5c973a57a0 20 read_log 194416'17004 (194416'17003) modify 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by client.2441958.0:16542 2014-08-13 15:29:40.818618 2014-08-13 18:06:06.614137 7f5c973a57a0 20 read_log 194416'17005 (0'0) promote 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by osd.289.0:4551 2014-08-13 15:29:43.801767 2014-08-13 18:06:06.614154 7f5c973a57a0 20 read_log 194416'17006 (194416'17005) modify 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by client.2441958.0:16754 2014-08-13 15:29:43.791717 2014-08-13 18:06:06.614169 7f5c973a57a0 20 read_log 194416'17007 (0'0) promote 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by osd.289.0:4555 2014-08-13 15:29:46.223807 2014-08-13 18:06:06.614182 7f5c973a57a0 20 read_log 194416'17008 (194416'17007) modify 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by client.2441958.0:16921 2014-08-13 15:29:46.214245 2014-08-13 18:06:06.614194 7f5c973a57a0 20 read_log 194416'17009 (0'0) promote 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by osd.289.0:4559 2014-08-13 15:29:47.577791 2014-08-13 18:06:06.614206 7f5c973a57a0 20 read_log 194416'17010 (194416'17009) modify 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by client.2441958.0:17016 2014-08-13 15:29:47.568548 2014-08-13 18:06:06.614218 7f5c973a57a0 20 read_log 194416'17011 (0'0) promote d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by osd.289.0:4573 2014-08-13 15:29:56.281401 2014-08-13 18:06:06.614231 7f5c973a57a0 20 read_log 194416'17012 (194416'17011) modify d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by client.2441958.0:17630 2014-08-13 15:29:56.272694 2014-08-13 18:06:06.614243 7f5c973a57a0 20 read_log 194416'17013 (0'0) promote 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by osd.289.0:4589 2014-08-13 15:30:14.384554 2014-08-13 18:06:06.614269 7f5c973a57a0 20 read_log 194416'17014 (194416'17013) modify 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by client.2441958.0:18547 2014-08-13 15:30:14.376062 2014-08-13 18:06:06.614284 7f5c973a57a0 20 read_log 194422'17015 (0'0) promote f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by osd.289.0:4595 2014-08-13 15:32:15.554127 2014-08-13 18:06:06.614297 7f5c973a57a0 20 read_log 194422'17016 (194422'17015) modify f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by client.1148203.0:137 2014-08-13 15:32:15.545108 2014-08-13 18:06:06.614312 7f5c973a57a0 20 read_log 194422'17017 (0'0) promote 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by osd.289.0:4609 2014-08-13 15:32:21.003831 2014-08-13 18:06:06.614325 7f5c973a57a0 20 read_log 194422'17018 (194422'17017) modify 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by client.1148203.0:1435 2014-08-13 15:32:20.994549 2014-08-13 18:06:06.614337 7f5c973a57a0 20 read_log 194422'17019 (0'0) promote a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by osd.289.0:4621 2014-08-13 15:32:23.189497 2014-08-13 18:06:06.614349 7f5c973a57a0 20 read_log 194422'17020 (194422'17019) modify a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by client.1148203.0:1986 2014-08-13 15:32:23.176283 2014-08-13 18:06:06.614366 7f5c973a57a0 20 read_log 194422'17021 (0'0) promote 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by osd.289.0:4633 2014-08-13 15:32:25.011040 2014-08-13 18:06:06.614389 7f5c973a57a0 20 read_log 194422'17022 (194422'17021) modify 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by client.1148203.0:2464 2014-08-13 15:32:25.001257 2014-08-13 18:06:06.614405 7f5c973a57a0 20 read_log 194422'17023 (0'0) promote 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by osd.289.0:4655 2014-08-13 15:32:31.279919 2014-08-13 18:06:06.614417 7f5c973a57a0 20 read_log 194422'17024 (194422'17023) modify 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by client.1148203.0:4071 2014-08-13 15:32:31.269670 2014-08-13 18:06:06.614437 7f5c973a57a0 20 read_log 194422'17025 (0'0) promote e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by osd.289.0:4669 2014-08-13 15:32:36.032657 2014-08-13 18:06:06.614450 7f5c973a57a0 20 read_log 194422'17026 (194422'17025) modify e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by client.1148203.0:5235 2014-08-13 15:32:36.023870 2014-08-13 18:06:06.614463 7f5c973a57a0 20 read_log 194422'17027 (0'0) promote addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by osd.289.0:4677 2014-08-13 15:32:37.076109 2014-08-13 18:06:06.614509 7f5c973a57a0 20 read_log 194422'17028 (194422'17027) modify addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by client.1148203.0:5448 2014-08-13 15:32:37.068280 2014-08-13 18:06:06.614524 7f5c973a57a0 20 read_log 194422'17029 (0'0) promote d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by osd.289.0:4683 2014-08-13 15:32:38.282723 2014-08-13 18:06:06.614536 7f5c973a57a0 20 read_log 194422'17030 (194422'17029) modify d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by client.1148203.0:5659 2014-08-13 15:32:38.273121 2014-08-13 18:06:06.614549 7f5c973a57a0 20 read_log 194422'17031 (0'0) promote ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by osd.289.0:4721 2014-08-13 15:32:51.426396 2014-08-13 18:06:06.614561 7f5c973a57a0 20 read_log 194422'17032 (194422'17031) modify ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by client.1148203.0:7956 2014-08-13 15:32:51.417220 2014-08-13 18:06:06.614573 7f5c973a57a0 20 read_log 194422'17033 (0'0) promote 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by osd.289.0:4763 2014-08-13 15:33:17.064308 2014-08-13 18:06:06.614624 7f5c973a57a0 20 read_log 194422'17034 (194422'17033) modify 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by client.1148203.0:10251 2014-08-13 15:33:17.055164 2014-08-13 18:06:06.614645 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.615101 7f5c973a57a0 10 read_log checking for missing items over interval (194215'16908,194422'17034] 2014-08-13 18:06:06.615159 7f5c973a57a0 15 read_log missing 194422'17034 (194422'17033) modify 9535a85e/benchmark_data_storage0111-ib.pocstor_41231_object10250/head//24 by client.1148203.0:10251 2014-08-13 15:33:17.055164 2014-08-13 18:06:06.615201 7f5c973a57a0 15 read_log missing 194422'17032 (194422'17031) modify ff7b5c5e/benchmark_data_storage0111-ib.pocstor_41231_object7955/head//24 by client.1148203.0:7956 2014-08-13 15:32:51.417220 2014-08-13 18:06:06.615242 7f5c973a57a0 15 read_log missing 194422'17030 (194422'17029) modify d70d245e/benchmark_data_storage0111-ib.pocstor_41231_object5658/head//24 by client.1148203.0:5659 2014-08-13 15:32:38.273121 2014-08-13 18:06:06.615283 7f5c973a57a0 15 read_log missing 194422'17028 (194422'17027) modify addc085e/benchmark_data_storage0111-ib.pocstor_41231_object5447/head//24 by client.1148203.0:5448 2014-08-13 15:32:37.068280 2014-08-13 18:06:06.615320 7f5c973a57a0 15 read_log missing 194422'17026 (194422'17025) modify e4dd45e/benchmark_data_storage0111-ib.pocstor_41231_object5234/head//24 by client.1148203.0:5235 2014-08-13 15:32:36.023870 2014-08-13 18:06:06.615367 7f5c973a57a0 15 read_log missing 194422'17024 (194422'17023) modify 2d6e945e/benchmark_data_storage0111-ib.pocstor_41231_object4070/head//24 by client.1148203.0:4071 2014-08-13 15:32:31.269670 2014-08-13 18:06:06.615405 7f5c973a57a0 15 read_log missing 194422'17022 (194422'17021) modify 267be05e/benchmark_data_storage0111-ib.pocstor_41231_object2463/head//24 by client.1148203.0:2464 2014-08-13 15:32:25.001257 2014-08-13 18:06:06.615441 7f5c973a57a0 15 read_log missing 194422'17020 (194422'17019) modify a49dc5e/benchmark_data_storage0111-ib.pocstor_41231_object1985/head//24 by client.1148203.0:1986 2014-08-13 15:32:23.176283 2014-08-13 18:06:06.615477 7f5c973a57a0 15 read_log missing 194422'17018 (194422'17017) modify 6f738c5e/benchmark_data_storage0111-ib.pocstor_41231_object1434/head//24 by client.1148203.0:1435 2014-08-13 15:32:20.994549 2014-08-13 18:06:06.615513 7f5c973a57a0 15 read_log missing 194422'17016 (194422'17015) modify f15cf45e/benchmark_data_storage0111-ib.pocstor_41231_object136/head//24 by client.1148203.0:137 2014-08-13 15:32:15.545108 2014-08-13 18:06:06.615548 7f5c973a57a0 15 read_log missing 194416'17014 (194416'17013) modify 32ec2c5e/benchmark_data_storage0111-ib.pocstor_19103_object18546/head//24 by client.2441958.0:18547 2014-08-13 15:30:14.376062 2014-08-13 18:06:06.615585 7f5c973a57a0 15 read_log missing 194416'17012 (194416'17011) modify d1d47c5e/benchmark_data_storage0111-ib.pocstor_19103_object17629/head//24 by client.2441958.0:17630 2014-08-13 15:29:56.272694 2014-08-13 18:06:06.615620 7f5c973a57a0 15 read_log missing 194416'17010 (194416'17009) modify 31a8c05e/benchmark_data_storage0111-ib.pocstor_19103_object17015/head//24 by client.2441958.0:17016 2014-08-13 15:29:47.568548 2014-08-13 18:06:06.615656 7f5c973a57a0 15 read_log missing 194416'17008 (194416'17007) modify 88c6c5e/benchmark_data_storage0111-ib.pocstor_19103_object16920/head//24 by client.2441958.0:16921 2014-08-13 15:29:46.214245 2014-08-13 18:06:06.615692 7f5c973a57a0 15 read_log missing 194416'17006 (194416'17005) modify 4f56505e/benchmark_data_storage0111-ib.pocstor_19103_object16753/head//24 by client.2441958.0:16754 2014-08-13 15:29:43.791717 2014-08-13 18:06:06.615727 7f5c973a57a0 15 read_log missing 194416'17004 (194416'17003) modify 1a1cb85e/benchmark_data_storage0111-ib.pocstor_19103_object16541/head//24 by client.2441958.0:16542 2014-08-13 15:29:40.818618 2014-08-13 18:06:06.615766 7f5c973a57a0 15 read_log missing 194416'17002 (194416'17001) modify 692fdc5e/benchmark_data_storage0111-ib.pocstor_19103_object16442/head//24 by client.2441958.0:16443 2014-08-13 15:29:39.431217 2014-08-13 18:06:06.615803 7f5c973a57a0 15 read_log missing 194416'17000 (194416'16999) modify 5374b85e/benchmark_data_storage0111-ib.pocstor_19103_object15802/head//24 by client.2441958.0:15803 2014-08-13 15:29:34.382066 2014-08-13 18:06:06.615839 7f5c973a57a0 15 read_log missing 194416'16998 (194416'16997) modify 92f4d45e/benchmark_data_storage0111-ib.pocstor_19103_object13515/head//24 by client.2441958.0:13516 2014-08-13 15:29:24.166796 2014-08-13 18:06:06.615876 7f5c973a57a0 15 read_log missing 194416'16996 (194416'16995) modify 6844805e/benchmark_data_storage0111-ib.pocstor_19103_object12620/head//24 by client.2441958.0:12621 2014-08-13 15:29:20.226221 2014-08-13 18:06:06.615912 7f5c973a57a0 15 read_log missing 194416'16994 (194416'16993) modify 3a4c745e/benchmark_data_storage0111-ib.pocstor_19103_object12051/head//24 by client.2441958.0:12052 2014-08-13 15:29:17.834093 2014-08-13 18:06:06.615948 7f5c973a57a0 15 read_log missing 194416'16992 (194416'16991) modify e605105e/benchmark_data_storage0111-ib.pocstor_19103_object12045/head//24 by client.2441958.0:12046 2014-08-13 15:29:17.806207 2014-08-13 18:06:06.616033 7f5c973a57a0 15 read_log missing 194416'16990 (194416'16989) modify a964985e/benchmark_data_storage0111-ib.pocstor_19103_object8898/head//24 by client.2441958.0:8899 2014-08-13 15:29:05.011810 2014-08-13 18:06:06.616071 7f5c973a57a0 15 read_log missing 194416'16988 (194416'16987) modify 3f836c5e/benchmark_data_storage0111-ib.pocstor_19103_object7770/head//24 by client.2441958.0:7771 2014-08-13 15:29:00.661735 2014-08-13 18:06:06.616107 7f5c973a57a0 15 read_log missing 194416'16986 (194416'16985) modify 2eaa505e/benchmark_data_storage0111-ib.pocstor_19103_object7519/head//24 by client.2441958.0:7520 2014-08-13 15:28:59.718424 2014-08-13 18:06:06.616143 7f5c973a57a0 15 read_log missing 194416'16984 (194416'16983) modify 6a81445e/benchmark_data_storage0111-ib.pocstor_19103_object6433/head//24 by client.2441958.0:6434 2014-08-13 15:28:55.619015 2014-08-13 18:06:06.616186 7f5c973a57a0 15 read_log missing 194416'16982 (194416'16981) modify 8bbaf45e/benchmark_data_storage0111-ib.pocstor_19103_object5868/head//24 by client.2441958.0:5869 2014-08-13 15:28:53.229339 2014-08-13 18:06:06.616224 7f5c973a57a0 15 read_log missing 194416'16980 (194416'16979) modify 43b6505e/benchmark_data_storage0111-ib.pocstor_19103_object5762/head//24 by client.2441958.0:5763 2014-08-13 15:28:52.824627 2014-08-13 18:06:06.616264 7f5c973a57a0 15 read_log missing 194416'16978 (194416'16977) modify 1277605e/benchmark_data_storage0111-ib.pocstor_19103_object5417/head//24 by client.2441958.0:5418 2014-08-13 15:28:51.478489 2014-08-13 18:06:06.616308 7f5c973a57a0 15 read_log missing 194416'16976 (194416'16975) modify 1a36e05e/benchmark_data_storage0111-ib.pocstor_19103_object5017/head//24 by client.2441958.0:5018 2014-08-13 15:28:49.950732 2014-08-13 18:06:06.616348 7f5c973a57a0 15 read_log missing 194416'16974 (194416'16973) modify 270e945e/benchmark_data_storage0111-ib.pocstor_19103_object4709/head//24 by client.2441958.0:4710 2014-08-13 15:28:48.788974 2014-08-13 18:06:06.616396 7f5c973a57a0 15 read_log missing 194416'16972 (194416'16971) modify 48e59c5e/benchmark_data_storage0111-ib.pocstor_19103_object4438/head//24 by client.2441958.0:4439 2014-08-13 15:28:47.634245 2014-08-13 18:06:06.616444 7f5c973a57a0 15 read_log missing 194416'16970 (194416'16969) modify 4650785e/benchmark_data_storage0111-ib.pocstor_19103_object4158/head//24 by client.2441958.0:4159 2014-08-13 15:28:46.543697 2014-08-13 18:06:06.616495 7f5c973a57a0 15 read_log missing 194416'16968 (194416'16967) modify 9e358c5e/benchmark_data_storage0111-ib.pocstor_19103_object3100/head//24 by client.2441958.0:3101 2014-08-13 15:28:42.448145 2014-08-13 18:06:06.616533 7f5c973a57a0 15 read_log missing 194416'16966 (194416'16965) modify 9cc76c5e/benchmark_data_storage0111-ib.pocstor_19103_object1259/head//24 by client.2441958.0:1260 2014-08-13 15:28:35.492974 2014-08-13 18:06:06.616569 7f5c973a57a0 15 read_log missing 194393'16964 (194393'16963) modify c71d345e/benchmark_data_storage0101-ib.pocstor_6819_object84/head//24 by client.2235069.0:85 2014-08-12 20:52:56.719683 2014-08-13 18:06:06.616609 7f5c973a57a0 15 read_log missing 194393'16962 (194393'16961) modify 1c06c05e/benchmark_data_storage0111-ib.pocstor_32482_object264/head//24 by client.1064182.0:265 2014-08-12 20:49:26.711262 2014-08-13 18:06:06.616645 7f5c973a57a0 15 read_log missing 194383'16960 (194383'16959) modify b248985e/benchmark_data_storage0111-ib.pocstor_25065_object7568/head//24 by client.2231190.0:7569 2014-08-12 20:34:30.638584 2014-08-13 18:06:06.616682 7f5c973a57a0 15 read_log missing 194377'16958 (194377'16957) modify 27c3545e/benchmark_data_storage0111-ib.pocstor_25065_object3992/head//24 by client.2231190.0:3993 2014-08-12 20:33:13.871436 2014-08-13 18:06:06.616718 7f5c973a57a0 15 read_log missing 194377'16956 (194377'16955) modify 22893c5e/benchmark_data_storage0111-ib.pocstor_25065_object3643/head//24 by client.2231190.0:3644 2014-08-12 20:33:12.077222 2014-08-13 18:06:06.616754 7f5c973a57a0 15 read_log missing 194377'16954 (194377'16953) modify 17c58c5e/benchmark_data_storage0111-ib.pocstor_25065_object2874/head//24 by client.2231190.0:2875 2014-08-12 20:33:08.779446 2014-08-13 18:06:06.616817 7f5c973a57a0 15 read_log missing 194377'16952 (194377'16951) modify 995b85e/benchmark_data_storage0111-ib.pocstor_25065_object2442/head//24 by client.2231190.0:2443 2014-08-12 20:33:06.984933 2014-08-13 18:06:06.616856 7f5c973a57a0 15 read_log missing 194377'16950 (194377'16949) modify cc0e585e/benchmark_data_storage0111-ib.pocstor_25065_object1777/head//24 by client.2231190.0:1778 2014-08-12 20:33:04.328244 2014-08-13 18:06:06.616894 7f5c973a57a0 15 read_log missing 194377'16948 (194377'16947) modify 446e205e/benchmark_data_storage0111-ib.pocstor_25065_object242/head//24 by client.2231190.0:243 2014-08-12 20:32:58.153044 2014-08-13 18:06:06.616931 7f5c973a57a0 15 read_log missing 194377'16946 (194377'16945) modify da2e905e/benchmark_data_storage0111-ib.pocstor_25065_object181/head//24 by client.2231190.0:182 2014-08-12 20:32:57.922328 2014-08-13 18:06:06.616978 7f5c973a57a0 15 read_log missing 194370'16944 (194370'16943) modify 401bf45e/benchmark_data_storage0111-ib.pocstor_12495_object13920/head//24 by client.1508531.0:13921 2014-08-12 20:31:30.054866 2014-08-13 18:06:06.617018 7f5c973a57a0 15 read_log missing 194370'16942 (194370'16941) modify 6902e45e/benchmark_data_storage0111-ib.pocstor_12495_object13060/head//24 by client.1508531.0:13061 2014-08-12 20:31:19.922980 2014-08-13 18:06:06.617055 7f5c973a57a0 15 read_log missing 194370'16940 (194370'16939) modify 482c745e/benchmark_data_storage0111-ib.pocstor_12495_object10560/head//24 by client.1508531.0:10561 2014-08-12 20:31:02.595811 2014-08-13 18:06:06.617091 7f5c973a57a0 15 read_log missing 194370'16938 (194370'16937) modify aff2ec5e/benchmark_data_storage0111-ib.pocstor_12495_object10269/head//24 by client.1508531.0:10270 2014-08-12 20:31:00.838695 2014-08-13 18:06:06.617128 7f5c973a57a0 15 read_log missing 194370'16936 (194370'16935) modify eabb485e/benchmark_data_storage0111-ib.pocstor_12495_object9902/head//24 by client.1508531.0:9903 2014-08-12 20:30:58.699187 2014-08-13 18:06:06.617176 7f5c973a57a0 15 read_log missing 194370'16934 (194370'16933) modify d5bcd85e/benchmark_data_storage0111-ib.pocstor_12495_object8595/head//24 by client.1508531.0:8596 2014-08-12 20:30:50.829702 2014-08-13 18:06:06.617213 7f5c973a57a0 15 read_log missing 194370'16932 (194370'16931) modify d545705e/benchmark_data_storage0111-ib.pocstor_12495_object7442/head//24 by client.1508531.0:7443 2014-08-12 20:30:44.563223 2014-08-13 18:06:06.617252 7f5c973a57a0 15 read_log missing 194370'16930 (194370'16929) modify 297ec05e/benchmark_data_storage0111-ib.pocstor_12495_object7234/head//24 by client.1508531.0:7235 2014-08-12 20:30:43.520542 2014-08-13 18:06:06.617289 7f5c973a57a0 15 read_log missing 194370'16928 (194370'16927) modify 17908c5e/benchmark_data_storage0111-ib.pocstor_12495_object6369/head//24 by client.1508531.0:6370 2014-08-12 20:30:39.333782 2014-08-13 18:06:06.617329 7f5c973a57a0 15 read_log missing 194370'16926 (194370'16925) modify c6ecc5e/benchmark_data_storage0111-ib.pocstor_12495_object1794/head//24 by client.1508531.0:1795 2014-08-12 20:30:21.077929 2014-08-13 18:06:06.617377 7f5c973a57a0 15 read_log missing 194370'16924 (194370'16923) modify c08b485e/benchmark_data_storage0111-ib.pocstor_12495_object1782/head//24 by client.1508531.0:1783 2014-08-12 20:30:21.032416 2014-08-13 18:06:06.617424 7f5c973a57a0 15 read_log missing 194370'16922 (194370'16921) modify 200ff45e/benchmark_data_storage0111-ib.pocstor_12495_object1348/head//24 by client.1508531.0:1349 2014-08-12 20:30:19.437629 2014-08-13 18:06:06.617462 7f5c973a57a0 15 read_log missing 194370'16920 (194370'16919) modify 5e69f85e/benchmark_data_storage0111-ib.pocstor_12495_object1157/head//24 by client.1508531.0:1158 2014-08-12 20:30:18.692375 2014-08-13 18:06:06.617498 7f5c973a57a0 15 read_log missing 194370'16918 (194370'16917) modify 9961745e/benchmark_data_storage0111-ib.pocstor_12495_object544/head//24 by client.1508531.0:545 2014-08-12 20:30:16.471772 2014-08-13 18:06:06.617535 7f5c973a57a0 15 read_log missing 194370'16916 (194370'16915) modify 104fdc5e/benchmark_data_storage0111-ib.pocstor_12495_object302/head//24 by client.1508531.0:303 2014-08-12 20:30:15.404611 2014-08-13 18:06:06.617593 7f5c973a57a0 15 read_log missing 194370'16914 (194370'16913) modify 35db505e/benchmark_data_storage0111-ib.pocstor_12495_object275/head//24 by client.1508531.0:276 2014-08-12 20:30:15.299339 2014-08-13 18:06:06.617659 7f5c973a57a0 15 read_log missing 194217'16912 (194217'16911) modify 3f62f45e/benchmark_data_storage0111-ib.pocstor_39693_object176/head//24 by client.1487927.0:177 2014-08-12 17:25:03.982296 2014-08-13 18:06:06.617725 7f5c973a57a0 15 read_log missing 194217'16909 (194217'16909) modify 5e/hit_set_24.5e_archive_2014-08-12 15:37:32.211135_2014-08-12 17:25:03.987810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.988024 2014-08-13 18:06:06.617744 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.617773 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=0 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive m=63] handle_loaded 2014-08-13 18:06:06.617816 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=0 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Initial 0.017513 0 0.000000 2014-08-13 18:06:06.617837 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=0 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Reset 2014-08-13 18:06:06.617846 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=0 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194487 2014-08-13 18:06:06.617856 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log((192718'14372,194422'17034], crt=194422'17034) 2014-08-13 18:06:06.617876 7f5c973a57a0 10 osd.4 194487 pgid 24.9b coll 24.9b_head 2014-08-13 18:06:06.618045 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.9b 2014-08-13 18:06:06.618059 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.618092 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.9b(unlocked)] enter Initial 2014-08-13 18:06:06.618102 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.9b(unlocked)] enter NotTrimming 2014-08-13 18:06:06.618254 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.618403 7f5c973a57a0 20 read_log 192834'15074 (192834'15040) delete 24e5989b/benchmark_data_storage0111-ib.pocstor_26659_object6053/head//24 by osd.321.0:54113 2014-08-11 15:26:51.117161 2014-08-13 18:06:06.618434 7f5c973a57a0 20 read_log 192834'15121 (192834'14622) clean 1f6c209b/benchmark_data_storage0111-ib.pocstor_26659_object11198/head//24 by osd.321.0:54225 2014-08-11 15:26:56.367995 2014-08-13 18:06:06.618447 7f5c973a57a0 20 read_log 192834'15125 (192834'14634) clean 53cb349b/benchmark_data_storage0111-ib.pocstor_26659_object11501/head//24 by osd.321.0:54229 2014-08-11 15:26:56.387299 2014-08-13 18:06:06.618460 7f5c973a57a0 20 read_log 192834'15144 (192834'15121) delete 1f6c209b/benchmark_data_storage0111-ib.pocstor_26659_object11198/head//24 by osd.321.0:54248 2014-08-11 15:26:56.496688 2014-08-13 18:06:06.618473 7f5c973a57a0 20 read_log 192834'15148 (192834'15125) delete 53cb349b/benchmark_data_storage0111-ib.pocstor_26659_object11501/head//24 by osd.321.0:54252 2014-08-11 15:26:56.513641 2014-08-13 18:06:06.618485 7f5c973a57a0 20 read_log 192834'15157 (0'0) promote 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:54269 2014-08-11 15:26:57.334648 2014-08-13 18:06:06.618500 7f5c973a57a0 20 read_log 192834'15158 (192834'15157) modify 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by client.1883634.0:10869 2014-08-11 15:26:57.324935 2014-08-13 18:06:06.618513 7f5c973a57a0 20 read_log 192834'15179 (0'0) promote fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:54320 2014-08-11 15:26:59.683665 2014-08-13 18:06:06.618525 7f5c973a57a0 20 read_log 192834'15180 (192834'15179) modify fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by client.1883634.0:11404 2014-08-11 15:26:59.675811 2014-08-13 18:06:06.618538 7f5c973a57a0 20 read_log 192834'15205 (192834'14700) clean 6e5d549b/benchmark_data_storage0111-ib.pocstor_26659_object14626/head//24 by osd.321.0:54376 2014-08-11 15:27:01.696299 2014-08-13 18:06:06.618550 7f5c973a57a0 20 read_log 192834'15209 (192834'14733) clean 229b1c9b/benchmark_data_storage0111-ib.pocstor_26659_object16797/head//24 by osd.321.0:54380 2014-08-11 15:27:01.717278 2014-08-13 18:06:06.618562 7f5c973a57a0 20 read_log 192834'15210 (192834'14725) clean 87ff3c9b/benchmark_data_storage0111-ib.pocstor_26659_object16245/head//24 by osd.321.0:54381 2014-08-11 15:27:01.722389 2014-08-13 18:06:06.618575 7f5c973a57a0 20 read_log 192834'15211 (192834'14719) clean 47997c9b/benchmark_data_storage0111-ib.pocstor_26659_object15459/head//24 by osd.321.0:54382 2014-08-11 15:27:01.726323 2014-08-13 18:06:06.618587 7f5c973a57a0 20 read_log 192834'15247 (192834'14748) clean 378a109b/benchmark_data_storage0111-ib.pocstor_26659_object18844/head//24 by osd.321.0:54470 2014-08-11 15:27:06.912626 2014-08-13 18:06:06.618599 7f5c973a57a0 20 read_log 192834'15251 (192834'14756) clean 1ad5249b/benchmark_data_storage0111-ib.pocstor_26659_object19362/head//24 by osd.321.0:54474 2014-08-11 15:27:06.928977 2014-08-13 18:06:06.618611 7f5c973a57a0 20 read_log 192834'15252 (192834'14772) clean d3e8e49b/benchmark_data_storage0111-ib.pocstor_26659_object21507/head//24 by osd.321.0:54475 2014-08-11 15:27:06.937264 2014-08-13 18:06:06.618623 7f5c973a57a0 20 read_log 192834'15257 (192834'14741) clean 4dc89b/benchmark_data_storage0111-ib.pocstor_26659_object18019/head//24 by osd.321.0:54480 2014-08-11 15:27:06.958398 2014-08-13 18:06:06.618635 7f5c973a57a0 20 read_log 192834'15263 (192834'14739) clean 5a133c9b/benchmark_data_storage0111-ib.pocstor_26659_object17781/head//24 by osd.321.0:54486 2014-08-11 15:27:06.978421 2014-08-13 18:06:06.618647 7f5c973a57a0 20 read_log 192834'15272 (0'0) promote c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:54511 2014-08-11 15:27:08.303890 2014-08-13 18:06:06.618659 7f5c973a57a0 20 read_log 192834'15273 (192834'15272) modify c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by client.1883634.0:13261 2014-08-11 15:27:08.294647 2014-08-13 18:06:06.618671 7f5c973a57a0 20 read_log 192834'15298 (0'0) promote 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:54609 2014-08-11 15:27:16.206527 2014-08-13 18:06:06.618709 7f5c973a57a0 20 read_log 192834'15299 (192834'15298) modify 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by client.1883634.0:14936 2014-08-11 15:27:16.197910 2014-08-13 18:06:06.618723 7f5c973a57a0 20 read_log 192834'15305 (192834'14784) clean 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:54630 2014-08-11 15:27:17.681435 2014-08-13 18:06:06.618735 7f5c973a57a0 20 read_log 192834'15313 (0'0) promote dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:54652 2014-08-11 15:27:19.874486 2014-08-13 18:06:06.618748 7f5c973a57a0 20 read_log 192834'15314 (192834'15313) modify dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by client.1883634.0:15512 2014-08-11 15:27:19.865279 2014-08-13 18:06:06.618760 7f5c973a57a0 20 read_log 192834'15315 (0'0) promote 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:54654 2014-08-11 15:27:19.914332 2014-08-13 18:06:06.618772 7f5c973a57a0 20 read_log 192834'15316 (192834'15315) modify 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by client.1883634.0:15519 2014-08-11 15:27:19.904827 2014-08-13 18:06:06.618784 7f5c973a57a0 20 read_log 192834'15338 (192834'15305) delete 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:54713 2014-08-11 15:27:24.346555 2014-08-13 18:06:06.618796 7f5c973a57a0 20 read_log 192834'15353 (192834'14788) clean 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:54728 2014-08-11 15:27:25.059012 2014-08-13 18:06:06.618809 7f5c973a57a0 20 read_log 192834'15355 (192834'15353) delete 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:54731 2014-08-11 15:27:25.250349 2014-08-13 18:06:06.618821 7f5c973a57a0 20 read_log 192834'15365 (0'0) promote b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:54754 2014-08-11 15:27:26.278155 2014-08-13 18:06:06.618833 7f5c973a57a0 20 read_log 192834'15366 (192834'15365) modify b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by client.1883634.0:16161 2014-08-11 15:27:26.266643 2014-08-13 18:06:06.618845 7f5c973a57a0 20 read_log 192834'15389 (192834'14828) clean 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:54824 2014-08-11 15:27:31.262919 2014-08-13 18:06:06.618858 7f5c973a57a0 20 read_log 192834'15402 (192834'15389) delete 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:54838 2014-08-11 15:27:31.380947 2014-08-13 18:06:06.618870 7f5c973a57a0 20 read_log 192834'15406 (192834'14842) clean 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:54843 2014-08-11 15:27:31.542523 2014-08-13 18:06:06.618882 7f5c973a57a0 20 read_log 192834'15421 (192834'15406) delete 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:54892 2014-08-11 15:27:36.826042 2014-08-13 18:06:06.618896 7f5c973a57a0 20 read_log 192834'15443 (0'0) promote 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:54924 2014-08-11 15:27:38.427173 2014-08-13 18:06:06.618908 7f5c973a57a0 20 read_log 192834'15444 (192834'15443) modify 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by client.1883634.0:17680 2014-08-11 15:27:38.417513 2014-08-13 18:06:06.618920 7f5c973a57a0 20 read_log 192834'15470 (192834'15263) delete 5a133c9b/benchmark_data_storage0111-ib.pocstor_26659_object17781/head//24 by osd.321.0:54975 2014-08-11 15:27:42.675815 2014-08-13 18:06:06.618933 7f5c973a57a0 20 read_log 192834'15471 (192834'15210) delete 87ff3c9b/benchmark_data_storage0111-ib.pocstor_26659_object16245/head//24 by osd.321.0:54976 2014-08-11 15:27:42.679700 2014-08-13 18:06:06.618948 7f5c973a57a0 20 read_log 192834'15472 (192834'15211) delete 47997c9b/benchmark_data_storage0111-ib.pocstor_26659_object15459/head//24 by osd.321.0:54977 2014-08-11 15:27:42.683576 2014-08-13 18:06:06.618971 7f5c973a57a0 20 read_log 192834'15476 (192834'14878) clean 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:54981 2014-08-11 15:27:42.695193 2014-08-13 18:06:06.618986 7f5c973a57a0 20 read_log 192834'15521 (192834'15247) delete 378a109b/benchmark_data_storage0111-ib.pocstor_26659_object18844/head//24 by osd.321.0:55026 2014-08-11 15:27:42.851990 2014-08-13 18:06:06.618999 7f5c973a57a0 20 read_log 192834'15527 (192834'15251) delete 1ad5249b/benchmark_data_storage0111-ib.pocstor_26659_object19362/head//24 by osd.321.0:55032 2014-08-11 15:27:42.871981 2014-08-13 18:06:06.619011 7f5c973a57a0 20 read_log 192834'15528 (192834'15205) delete 6e5d549b/benchmark_data_storage0111-ib.pocstor_26659_object14626/head//24 by osd.321.0:55033 2014-08-11 15:27:42.874904 2014-08-13 18:06:06.619024 7f5c973a57a0 20 read_log 192834'15529 (192834'15252) delete d3e8e49b/benchmark_data_storage0111-ib.pocstor_26659_object21507/head//24 by osd.321.0:55034 2014-08-11 15:27:42.877841 2014-08-13 18:06:06.619036 7f5c973a57a0 20 read_log 192834'15535 (192834'15257) delete 4dc89b/benchmark_data_storage0111-ib.pocstor_26659_object18019/head//24 by osd.321.0:55040 2014-08-11 15:27:42.895172 2014-08-13 18:06:06.619048 7f5c973a57a0 20 read_log 192834'15543 (192834'15209) delete 229b1c9b/benchmark_data_storage0111-ib.pocstor_26659_object16797/head//24 by osd.321.0:55048 2014-08-11 15:27:42.915419 2014-08-13 18:06:06.619061 7f5c973a57a0 20 read_log 192834'15544 (192834'15476) delete 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:55049 2014-08-11 15:27:42.918447 2014-08-13 18:06:06.619075 7f5c973a57a0 20 read_log 192834'15571 (192834'15007) clean 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:55112 2014-08-11 15:27:48.337477 2014-08-13 18:06:06.619087 7f5c973a57a0 20 read_log 192834'15589 (192834'15571) delete 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:55148 2014-08-11 15:27:53.267500 2014-08-13 18:06:06.619099 7f5c973a57a0 20 read_log 192834'15596 (0'0) promote 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:55161 2014-08-11 15:27:53.589147 2014-08-13 18:06:06.619112 7f5c973a57a0 20 read_log 192834'15597 (192834'15596) modify 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by client.1883634.0:19626 2014-08-11 15:27:53.581136 2014-08-13 18:06:06.619126 7f5c973a57a0 20 read_log 192834'15602 (0'0) promote b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:55169 2014-08-11 15:27:54.357228 2014-08-13 18:06:06.619138 7f5c973a57a0 20 read_log 192834'15603 (192834'15602) modify b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by client.1883634.0:19850 2014-08-11 15:27:54.347328 2014-08-13 18:06:06.619172 7f5c973a57a0 20 read_log 192834'15608 (0'0) promote 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:55179 2014-08-11 15:27:54.903864 2014-08-13 18:06:06.619187 7f5c973a57a0 20 read_log 192834'15609 (192834'15608) modify 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by client.1883634.0:19998 2014-08-11 15:27:54.895406 2014-08-13 18:06:06.619200 7f5c973a57a0 20 read_log 192834'15640 (192834'15158) clean 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:55239 2014-08-11 15:27:58.703983 2014-08-13 18:06:06.619212 7f5c973a57a0 20 read_log 192834'15646 (192834'15640) delete 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:55245 2014-08-11 15:27:58.735483 2014-08-13 18:06:06.619237 7f5c973a57a0 20 read_log 192834'15671 (192834'15180) clean fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:55324 2014-08-11 15:28:04.104809 2014-08-13 18:06:06.619251 7f5c973a57a0 20 read_log 192834'15682 (192834'15671) delete fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:55335 2014-08-11 15:28:04.152246 2014-08-13 18:06:06.619276 7f5c973a57a0 20 read_log 192834'15695 (0'0) promote 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:55460 2014-08-11 15:28:10.359363 2014-08-13 18:06:06.619291 7f5c973a57a0 20 read_log 192834'15696 (192834'15695) modify 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by client.1883634.0:22624 2014-08-11 15:28:09.971569 2014-08-13 18:06:06.619304 7f5c973a57a0 20 read_log 192834'15710 (192834'15273) clean c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:55573 2014-08-11 15:28:14.584971 2014-08-13 18:06:06.619316 7f5c973a57a0 20 read_log 192834'15723 (192834'15710) delete c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:55586 2014-08-11 15:28:15.715581 2014-08-13 18:06:06.619329 7f5c973a57a0 20 read_log 192834'15736 (0'0) promote 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:55624 2014-08-11 15:28:18.036412 2014-08-13 18:06:06.619341 7f5c973a57a0 20 read_log 192834'15737 (192834'15736) modify 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by client.1883634.0:23715 2014-08-11 15:28:17.548271 2014-08-13 18:06:06.619357 7f5c973a57a0 20 read_log 192834'15740 (0'0) promote 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:55654 2014-08-11 15:28:18.895727 2014-08-13 18:06:06.619370 7f5c973a57a0 20 read_log 192834'15741 (192834'15740) modify 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by client.1883634.0:24250 2014-08-11 15:28:18.648664 2014-08-13 18:06:06.619394 7f5c973a57a0 20 read_log 192834'15768 (192834'15314) clean dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:55767 2014-08-11 15:28:21.668229 2014-08-13 18:06:06.619407 7f5c973a57a0 20 read_log 192834'15770 (192834'15316) clean 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:55770 2014-08-11 15:28:21.746071 2014-08-13 18:06:06.619419 7f5c973a57a0 20 read_log 192834'15773 (192834'15299) clean 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:55773 2014-08-11 15:28:21.756277 2014-08-13 18:06:06.619440 7f5c973a57a0 20 read_log 192834'15780 (192834'15768) delete dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:55784 2014-08-11 15:28:21.812808 2014-08-13 18:06:06.619457 7f5c973a57a0 20 read_log 192834'15783 (192834'15770) delete 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:55787 2014-08-11 15:28:21.852271 2014-08-13 18:06:06.619470 7f5c973a57a0 20 read_log 192834'15784 (192834'15773) delete 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:55788 2014-08-11 15:28:21.856317 2014-08-13 18:06:06.619488 7f5c973a57a0 20 read_log 192834'15813 (0'0) promote 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:55849 2014-08-11 15:28:23.841212 2014-08-13 18:06:06.619500 7f5c973a57a0 20 read_log 192834'15814 (192834'15813) modify 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by client.1883634.0:26946 2014-08-11 15:28:23.812876 2014-08-13 18:06:06.619526 7f5c973a57a0 20 read_log 192834'15827 (0'0) promote 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:55899 2014-08-11 15:28:25.200596 2014-08-13 18:06:06.619543 7f5c973a57a0 20 read_log 192834'15828 (192834'15827) modify 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by client.1883634.0:27948 2014-08-11 15:28:25.176538 2014-08-13 18:06:06.619556 7f5c973a57a0 20 read_log 192834'15860 (192834'15366) clean b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:55965 2014-08-11 15:28:27.173206 2014-08-13 18:06:06.619569 7f5c973a57a0 20 read_log 192834'15869 (0'0) promote 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:55997 2014-08-11 15:28:28.024373 2014-08-13 18:06:06.619598 7f5c973a57a0 20 read_log 192834'15870 (192834'15869) modify 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by client.1883634.0:30038 2014-08-11 15:28:27.993618 2014-08-13 18:06:06.619615 7f5c973a57a0 20 read_log 192834'15895 (0'0) promote 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56065 2014-08-11 15:28:31.179506 2014-08-13 18:06:06.619628 7f5c973a57a0 20 read_log 192834'15896 (192834'15895) modify 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by client.1883634.0:31941 2014-08-11 15:28:31.154016 2014-08-13 18:06:06.619640 7f5c973a57a0 20 read_log 192834'15908 (192834'15860) delete b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56103 2014-08-11 15:28:32.356287 2014-08-13 18:06:06.619653 7f5c973a57a0 20 read_log 192834'15941 (0'0) promote 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56174 2014-08-11 15:28:34.378710 2014-08-13 18:06:06.619668 7f5c973a57a0 20 read_log 192834'15942 (192834'15941) modify 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by client.1883634.0:34193 2014-08-11 15:28:34.351417 2014-08-13 18:06:06.619700 7f5c973a57a0 20 read_log 192834'15961 (0'0) promote fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56209 2014-08-11 15:28:35.250255 2014-08-13 18:06:06.619715 7f5c973a57a0 20 read_log 192834'15962 (192834'15961) modify fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by client.1883634.0:34728 2014-08-11 15:28:35.227977 2014-08-13 18:06:06.619728 7f5c973a57a0 20 read_log 192834'15990 (0'0) promote c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56293 2014-08-11 15:28:38.201404 2014-08-13 18:06:06.619740 7f5c973a57a0 20 read_log 192834'15991 (192834'15990) modify c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by client.1883634.0:36585 2014-08-11 15:28:38.178737 2014-08-13 18:06:06.619752 7f5c973a57a0 20 read_log 192834'16016 (0'0) promote 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56356 2014-08-11 15:28:40.454713 2014-08-13 18:06:06.619765 7f5c973a57a0 20 read_log 192834'16017 (192834'16016) modify 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by client.1883634.0:38260 2014-08-11 15:28:40.420894 2014-08-13 18:06:06.619777 7f5c973a57a0 20 read_log 192834'16026 (0'0) promote dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56380 2014-08-11 15:28:41.219275 2014-08-13 18:06:06.619789 7f5c973a57a0 20 read_log 192834'16027 (192834'16026) modify dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by client.1883634.0:38836 2014-08-11 15:28:41.194331 2014-08-13 18:06:06.619802 7f5c973a57a0 20 read_log 192834'16028 (0'0) promote 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56383 2014-08-11 15:28:41.230935 2014-08-13 18:06:06.619814 7f5c973a57a0 20 read_log 192834'16029 (192834'16028) modify 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by client.1883634.0:38843 2014-08-11 15:28:41.202067 2014-08-13 18:06:06.619826 7f5c973a57a0 20 read_log 192834'16050 (0'0) promote b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56429 2014-08-11 15:28:42.232668 2014-08-13 18:06:06.619847 7f5c973a57a0 20 read_log 192834'16051 (192834'16050) modify b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by client.1883634.0:39485 2014-08-11 15:28:42.209952 2014-08-13 18:06:06.619861 7f5c973a57a0 20 read_log 192834'16059 (192834'15444) clean 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56451 2014-08-11 15:28:42.846587 2014-08-13 18:06:06.619873 7f5c973a57a0 20 read_log 192834'16063 (192834'16059) delete 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56455 2014-08-11 15:28:42.933504 2014-08-13 18:06:06.619897 7f5c973a57a0 20 read_log 192834'16082 (0'0) promote 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56508 2014-08-11 15:28:44.524548 2014-08-13 18:06:06.619910 7f5c973a57a0 20 read_log 192834'16083 (192834'16082) modify 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by client.1883634.0:41004 2014-08-11 15:28:44.500967 2014-08-13 18:06:06.619922 7f5c973a57a0 20 read_log 192834'16102 (192834'15597) modify 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by client.1883634.0:42950 2014-08-11 15:28:46.412808 2014-08-13 18:06:06.619935 7f5c973a57a0 20 read_log 192834'16105 (192834'15603) modify b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by client.1883634.0:43174 2014-08-11 15:28:46.477021 2014-08-13 18:06:06.619947 7f5c973a57a0 20 read_log 192834'16108 (192834'15609) modify 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by client.1883634.0:43322 2014-08-11 15:28:46.520061 2014-08-13 18:06:06.619960 7f5c973a57a0 20 read_log 192834'16124 (192834'15696) modify 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by client.1883634.0:45948 2014-08-11 15:28:47.442630 2014-08-13 18:06:06.619972 7f5c973a57a0 20 read_log 192834'16130 (192834'15737) clean 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:56695 2014-08-11 15:29:18.667655 2014-08-13 18:06:06.619984 7f5c973a57a0 20 read_log 192834'16131 (192834'15741) clean 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:56696 2014-08-11 15:29:18.678428 2014-08-13 18:06:06.619997 7f5c973a57a0 20 read_log 192834'16134 (192834'16130) delete 6b81689b/benchmark_data_storage0111-ib.pocstor_41526_object390/head//24 by osd.321.0:56699 2014-08-11 15:29:18.735417 2014-08-13 18:06:06.620011 7f5c973a57a0 20 read_log 192834'16135 (192834'16131) delete 7e37ac9b/benchmark_data_storage0111-ib.pocstor_41526_object925/head//24 by osd.321.0:56700 2014-08-11 15:29:18.741423 2014-08-13 18:06:06.620025 7f5c973a57a0 20 read_log 192834'16154 (192834'15814) clean 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:56719 2014-08-11 15:29:23.915302 2014-08-13 18:06:06.620037 7f5c973a57a0 20 read_log 192834'16174 (192834'16154) delete 8189889b/benchmark_data_storage0111-ib.pocstor_41526_object3621/head//24 by osd.321.0:56739 2014-08-11 15:29:24.033769 2014-08-13 18:06:06.620050 7f5c973a57a0 20 read_log 192834'16197 (192834'15828) clean 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:56762 2014-08-11 15:29:29.223406 2014-08-13 18:06:06.620062 7f5c973a57a0 20 read_log 192834'16204 (192834'15870) clean 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:56769 2014-08-11 15:29:29.251085 2014-08-13 18:06:06.620077 7f5c973a57a0 20 read_log 192834'16225 (192834'16197) delete 19e9f49b/benchmark_data_storage0111-ib.pocstor_41526_object4623/head//24 by osd.321.0:56790 2014-08-11 15:29:29.356293 2014-08-13 18:06:06.620091 7f5c973a57a0 20 read_log 192834'16231 (192834'16204) delete 4875c9b/benchmark_data_storage0111-ib.pocstor_41526_object6713/head//24 by osd.321.0:56796 2014-08-11 15:29:29.380973 2014-08-13 18:06:06.620105 7f5c973a57a0 20 read_log 192834'16255 (192834'15942) clean 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56820 2014-08-11 15:29:34.532726 2014-08-13 18:06:06.620118 7f5c973a57a0 20 read_log 192834'16261 (192834'15896) clean 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56826 2014-08-11 15:29:34.551303 2014-08-13 18:06:06.620132 7f5c973a57a0 20 read_log 192834'16282 (192834'16255) delete 7323389b/benchmark_data_storage0111-ib.pocstor_41526_object10868/head//24 by osd.321.0:56847 2014-08-11 15:29:34.616265 2014-08-13 18:06:06.620145 7f5c973a57a0 20 read_log 192834'16286 (192834'16261) delete 79322c9b/benchmark_data_storage0111-ib.pocstor_41526_object8616/head//24 by osd.321.0:56851 2014-08-11 15:29:34.624557 2014-08-13 18:06:06.620176 7f5c973a57a0 20 read_log 192834'16304 (192834'15991) clean c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56869 2014-08-11 15:29:39.727824 2014-08-13 18:06:06.620191 7f5c973a57a0 20 read_log 192834'16309 (192834'15962) clean fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56874 2014-08-11 15:29:39.746112 2014-08-13 18:06:06.620204 7f5c973a57a0 20 read_log 192834'16333 (192834'16304) delete c29e709b/benchmark_data_storage0111-ib.pocstor_41526_object13260/head//24 by osd.321.0:56898 2014-08-11 15:29:39.809613 2014-08-13 18:06:06.620218 7f5c973a57a0 20 read_log 192834'16337 (192834'16309) delete fb2549b/benchmark_data_storage0111-ib.pocstor_41526_object11403/head//24 by osd.321.0:56902 2014-08-11 15:29:39.817818 2014-08-13 18:06:06.620230 7f5c973a57a0 20 read_log 192834'16367 (192834'16027) clean dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56932 2014-08-11 15:29:44.919861 2014-08-13 18:06:06.620246 7f5c973a57a0 20 read_log 192834'16372 (192834'16083) clean 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56937 2014-08-11 15:29:44.936612 2014-08-13 18:06:06.620258 7f5c973a57a0 20 read_log 192834'16389 (192834'16367) delete dcb8549b/benchmark_data_storage0111-ib.pocstor_41526_object15511/head//24 by osd.321.0:56954 2014-08-11 15:29:44.964210 2014-08-13 18:06:06.620270 7f5c973a57a0 20 read_log 192834'16390 (192834'16372) delete 55f3a49b/benchmark_data_storage0111-ib.pocstor_41526_object17679/head//24 by osd.321.0:56955 2014-08-11 15:29:44.964464 2014-08-13 18:06:06.620283 7f5c973a57a0 20 read_log 192834'16395 (192834'16029) clean 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56960 2014-08-11 15:29:44.980958 2014-08-13 18:06:06.620300 7f5c973a57a0 20 read_log 192834'16396 (192834'16051) clean b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56961 2014-08-11 15:29:44.986162 2014-08-13 18:06:06.620313 7f5c973a57a0 20 read_log 192834'16397 (192834'16017) clean 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56962 2014-08-11 15:29:44.991458 2014-08-13 18:06:06.620326 7f5c973a57a0 20 read_log 192834'16404 (192834'16395) delete 13553c9b/benchmark_data_storage0111-ib.pocstor_41526_object15518/head//24 by osd.321.0:56969 2014-08-11 15:29:45.012510 2014-08-13 18:06:06.620338 7f5c973a57a0 20 read_log 192834'16407 (192834'16396) delete b5c37c9b/benchmark_data_storage0111-ib.pocstor_41526_object16160/head//24 by osd.321.0:56972 2014-08-11 15:29:45.015429 2014-08-13 18:06:06.620354 7f5c973a57a0 20 read_log 192834'16409 (192834'16397) delete 1358ec9b/benchmark_data_storage0111-ib.pocstor_41526_object14935/head//24 by osd.321.0:56974 2014-08-11 15:29:45.017453 2014-08-13 18:06:06.620366 7f5c973a57a0 20 read_log 192834'16462 (192834'16105) clean b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:57027 2014-08-11 15:29:50.114227 2014-08-13 18:06:06.620378 7f5c973a57a0 20 read_log 192834'16463 (192834'16124) clean 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:57028 2014-08-11 15:29:50.115651 2014-08-13 18:06:06.620406 7f5c973a57a0 20 read_log 192834'16465 (192834'16102) clean 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:57030 2014-08-11 15:29:50.117334 2014-08-13 18:06:06.620419 7f5c973a57a0 20 read_log 192834'16469 (192834'16462) delete b09b249b/benchmark_data_storage0111-ib.pocstor_41526_object19849/head//24 by osd.321.0:57034 2014-08-11 15:29:50.123636 2014-08-13 18:06:06.620441 7f5c973a57a0 20 read_log 192834'16470 (192834'16463) delete 6e82549b/benchmark_data_storage0111-ib.pocstor_41526_object22623/head//24 by osd.321.0:57035 2014-08-11 15:29:50.124921 2014-08-13 18:06:06.620457 7f5c973a57a0 20 read_log 192834'16471 (192834'16465) delete 5af3c49b/benchmark_data_storage0111-ib.pocstor_41526_object19625/head//24 by osd.321.0:57036 2014-08-11 15:29:50.126004 2014-08-13 18:06:06.620478 7f5c973a57a0 20 read_log 192834'16480 (192834'16108) clean 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:57045 2014-08-11 15:29:50.139357 2014-08-13 18:06:06.620493 7f5c973a57a0 20 read_log 192834'16484 (192834'16480) delete 75705c9b/benchmark_data_storage0111-ib.pocstor_41526_object19997/head//24 by osd.321.0:57049 2014-08-11 15:29:50.143491 2014-08-13 18:06:06.620505 7f5c973a57a0 20 read_log 192888'16490 (192888'16490) modify 9b/hit_set_24.9b_archive_2014-08-11 16:02:39.145403_2014-08-11 16:44:55.575365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.575861 2014-08-13 18:06:06.620527 7f5c973a57a0 20 read_log 192888'16491 (0'0) promote 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by osd.4.0:21381 2014-08-11 16:44:55.581901 2014-08-13 18:06:06.620543 7f5c973a57a0 20 read_log 192888'16492 (192888'16491) modify 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.1315259.0:161 2014-08-11 16:44:55.569607 2014-08-13 18:06:06.620556 7f5c973a57a0 20 read_log 192888'16495 (0'0) promote 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by osd.4.0:21430 2014-08-11 16:45:01.483541 2014-08-13 18:06:06.620581 7f5c973a57a0 20 read_log 192888'16496 (192888'16495) modify 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.1315259.0:1624 2014-08-11 16:45:01.475095 2014-08-13 18:06:06.620597 7f5c973a57a0 20 read_log 192888'16497 (0'0) promote ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by osd.4.0:21454 2014-08-11 16:45:04.788555 2014-08-13 18:06:06.620610 7f5c973a57a0 20 read_log 192888'16498 (192888'16497) modify ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.1315259.0:2464 2014-08-11 16:45:04.577662 2014-08-13 18:06:06.620623 7f5c973a57a0 20 read_log 192888'16499 (0'0) promote ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by osd.4.0:21466 2014-08-11 16:45:05.560019 2014-08-13 18:06:06.620635 7f5c973a57a0 20 read_log 192888'16500 (192888'16499) modify ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.1315259.0:2723 2014-08-11 16:45:05.551958 2014-08-13 18:06:06.620647 7f5c973a57a0 20 read_log 192888'16501 (0'0) promote 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by osd.4.0:21470 2014-08-11 16:45:06.752673 2014-08-13 18:06:06.620672 7f5c973a57a0 20 read_log 192888'16502 (192888'16501) modify 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.1315259.0:3045 2014-08-11 16:45:06.743502 2014-08-13 18:06:06.620686 7f5c973a57a0 20 read_log 192888'16507 (0'0) promote 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by osd.4.0:21496 2014-08-11 16:45:11.339353 2014-08-13 18:06:06.620698 7f5c973a57a0 20 read_log 192888'16508 (192888'16507) modify 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.1315259.0:4241 2014-08-11 16:45:11.330318 2014-08-13 18:06:06.620710 7f5c973a57a0 20 read_log 192888'16509 (0'0) promote 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by osd.4.0:21520 2014-08-11 16:45:13.486295 2014-08-13 18:06:06.620731 7f5c973a57a0 20 read_log 192888'16510 (192888'16509) modify 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.1315259.0:4833 2014-08-11 16:45:13.478515 2014-08-13 18:06:06.620746 7f5c973a57a0 20 read_log 192888'16511 (0'0) promote 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by osd.4.0:21534 2014-08-11 16:45:14.809563 2014-08-13 18:06:06.620758 7f5c973a57a0 20 read_log 192888'16512 (192888'16511) modify 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.1315259.0:5179 2014-08-11 16:45:14.801701 2014-08-13 18:06:06.620796 7f5c973a57a0 20 read_log 192888'16523 (0'0) promote 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by osd.4.0:21598 2014-08-11 16:45:24.414853 2014-08-13 18:06:06.620812 7f5c973a57a0 20 read_log 192888'16524 (192888'16523) modify 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.1315259.0:7825 2014-08-11 16:45:24.401578 2014-08-13 18:06:06.620825 7f5c973a57a0 20 read_log 192888'16525 (0'0) promote 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by osd.4.0:21608 2014-08-11 16:45:26.195805 2014-08-13 18:06:06.620837 7f5c973a57a0 20 read_log 192888'16526 (192888'16525) modify 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.1315259.0:8319 2014-08-11 16:45:26.187092 2014-08-13 18:06:06.620849 7f5c973a57a0 20 read_log 192888'16527 (0'0) promote a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by osd.4.0:21616 2014-08-11 16:45:27.047706 2014-08-13 18:06:06.620861 7f5c973a57a0 20 read_log 192888'16528 (192888'16527) modify a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.1315259.0:8520 2014-08-11 16:45:27.038518 2014-08-13 18:06:06.620874 7f5c973a57a0 20 read_log 192888'16545 (0'0) promote 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by osd.4.0:21728 2014-08-11 16:45:43.052477 2014-08-13 18:06:06.620886 7f5c973a57a0 20 read_log 192888'16546 (192888'16545) modify 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.1315259.0:12572 2014-08-11 16:45:43.043414 2014-08-13 18:06:06.620898 7f5c973a57a0 20 read_log 192888'16547 (0'0) promote 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by osd.4.0:21734 2014-08-11 16:45:44.181570 2014-08-13 18:06:06.620910 7f5c973a57a0 20 read_log 192888'16548 (192888'16547) modify 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.1315259.0:12837 2014-08-11 16:45:44.172547 2014-08-13 18:06:06.620922 7f5c973a57a0 20 read_log 192888'16549 (0'0) promote def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by osd.4.0:21772 2014-08-11 16:45:50.853714 2014-08-13 18:06:06.620934 7f5c973a57a0 20 read_log 192888'16550 (192888'16549) modify def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.1315259.0:14543 2014-08-11 16:45:50.845591 2014-08-13 18:06:06.620946 7f5c973a57a0 20 read_log 192888'16551 (0'0) promote cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by osd.4.0:21796 2014-08-11 16:45:54.503206 2014-08-13 18:06:06.620958 7f5c973a57a0 20 read_log 192888'16552 (192888'16551) modify cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.1315259.0:15489 2014-08-11 16:45:54.494515 2014-08-13 18:06:06.620971 7f5c973a57a0 20 read_log 192888'16559 (0'0) promote 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by osd.4.0:21818 2014-08-11 16:45:56.945931 2014-08-13 18:06:06.620982 7f5c973a57a0 20 read_log 192888'16560 (192888'16559) modify 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.1315259.0:16136 2014-08-11 16:45:56.935113 2014-08-13 18:06:06.620995 7f5c973a57a0 20 read_log 192888'16563 (0'0) promote 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by osd.4.0:21848 2014-08-11 16:46:00.984611 2014-08-13 18:06:06.621007 7f5c973a57a0 20 read_log 192888'16564 (192888'16563) modify 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.1315259.0:17163 2014-08-11 16:46:00.977207 2014-08-13 18:06:06.621019 7f5c973a57a0 20 read_log 192888'16567 (0'0) promote fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by osd.4.0:21856 2014-08-11 16:46:01.445634 2014-08-13 18:06:06.621031 7f5c973a57a0 20 read_log 192888'16568 (192888'16567) modify fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.1315259.0:17287 2014-08-11 16:46:01.436405 2014-08-13 18:06:06.621071 7f5c973a57a0 20 read_log 192888'16573 (0'0) promote e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by osd.4.0:21950 2014-08-11 16:46:15.024795 2014-08-13 18:06:06.621088 7f5c973a57a0 20 read_log 192888'16574 (192888'16573) modify e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.1315259.0:20768 2014-08-11 16:46:15.016704 2014-08-13 18:06:06.621100 7f5c973a57a0 20 read_log 192888'16577 (0'0) promote 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by osd.4.0:21976 2014-08-11 16:46:17.701039 2014-08-13 18:06:06.621115 7f5c973a57a0 20 read_log 192888'16578 (192888'16577) modify 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.1315259.0:21473 2014-08-11 16:46:17.692741 2014-08-13 18:06:06.621130 7f5c973a57a0 20 read_log 192888'16581 (0'0) promote 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by osd.4.0:22000 2014-08-11 16:46:21.319355 2014-08-13 18:06:06.621142 7f5c973a57a0 20 read_log 192888'16582 (192888'16581) modify 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.1315259.0:22411 2014-08-11 16:46:21.310832 2014-08-13 18:06:06.621183 7f5c973a57a0 20 read_log 192888'16587 (0'0) promote e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by osd.4.0:22034 2014-08-11 16:46:27.560326 2014-08-13 18:06:06.621196 7f5c973a57a0 20 read_log 192888'16588 (192888'16587) modify e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.1315259.0:23980 2014-08-11 16:46:27.551427 2014-08-13 18:06:06.621237 7f5c973a57a0 20 read_log 192888'16589 (0'0) promote 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by osd.4.0:22046 2014-08-11 16:46:28.737472 2014-08-13 18:06:06.621253 7f5c973a57a0 20 read_log 192888'16590 (192888'16589) modify 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.1315259.0:24291 2014-08-11 16:46:28.728519 2014-08-13 18:06:06.621265 7f5c973a57a0 20 read_log 192888'16593 (0'0) promote 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by osd.4.0:22084 2014-08-11 16:46:32.378159 2014-08-13 18:06:06.621278 7f5c973a57a0 20 read_log 192888'16594 (192888'16593) modify 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.1315259.0:25220 2014-08-11 16:46:32.367678 2014-08-13 18:06:06.621294 7f5c973a57a0 20 read_log 192888'16605 (0'0) promote 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by osd.4.0:22144 2014-08-11 16:46:41.907962 2014-08-13 18:06:06.621308 7f5c973a57a0 20 read_log 192888'16606 (192888'16605) modify 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.1315259.0:27473 2014-08-11 16:46:41.900184 2014-08-13 18:06:06.621321 7f5c973a57a0 20 read_log 192888'16611 (0'0) promote bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by osd.4.0:22178 2014-08-11 16:46:50.933223 2014-08-13 18:06:06.621337 7f5c973a57a0 20 read_log 192888'16612 (192888'16611) modify bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.1315259.0:29686 2014-08-11 16:46:50.924551 2014-08-13 18:06:06.621352 7f5c973a57a0 20 read_log 192888'16617 (0'0) promote 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by osd.4.0:22220 2014-08-11 16:46:54.408647 2014-08-13 18:06:06.621365 7f5c973a57a0 20 read_log 192888'16618 (192888'16617) modify 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.1315259.0:30608 2014-08-11 16:46:54.400124 2014-08-13 18:06:06.621389 7f5c973a57a0 20 read_log 192888'16619 (192888'16492) modify 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.1315259.0:30888 2014-08-11 16:46:55.928284 2014-08-13 18:06:06.621406 7f5c973a57a0 20 read_log 192888'16621 (192888'16496) modify 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.1315259.0:32351 2014-08-11 16:46:56.355066 2014-08-13 18:06:06.621439 7f5c973a57a0 20 read_log 192888'16622 (192888'16498) modify ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.1315259.0:33191 2014-08-11 16:46:56.574598 2014-08-13 18:06:06.621456 7f5c973a57a0 20 read_log 192888'16623 (192888'16500) modify ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.1315259.0:33450 2014-08-11 16:46:56.640201 2014-08-13 18:06:06.621469 7f5c973a57a0 20 read_log 192888'16624 (192888'16502) modify 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.1315259.0:33772 2014-08-11 16:46:56.721047 2014-08-13 18:06:06.621489 7f5c973a57a0 20 read_log 192888'16627 (192888'16508) modify 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.1315259.0:34968 2014-08-11 16:46:57.033749 2014-08-13 18:06:06.621505 7f5c973a57a0 20 read_log 192888'16628 (192888'16510) modify 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.1315259.0:35560 2014-08-11 16:46:57.191644 2014-08-13 18:06:06.621518 7f5c973a57a0 20 read_log 192888'16629 (192888'16512) modify 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.1315259.0:35906 2014-08-11 16:46:57.274923 2014-08-13 18:06:06.621530 7f5c973a57a0 20 read_log 192888'16635 (192888'16524) modify 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.1315259.0:38552 2014-08-11 16:46:57.979937 2014-08-13 18:06:06.621542 7f5c973a57a0 20 read_log 192888'16636 (192888'16526) modify 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.1315259.0:39046 2014-08-11 16:46:58.111536 2014-08-13 18:06:06.621555 7f5c973a57a0 20 read_log 192888'16637 (192888'16528) modify a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.1315259.0:39247 2014-08-11 16:46:58.162473 2014-08-13 18:06:06.621570 7f5c973a57a0 20 read_log 192888'16646 (192888'16546) modify 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.1315259.0:43299 2014-08-11 16:46:59.249675 2014-08-13 18:06:06.621583 7f5c973a57a0 20 read_log 192888'16647 (192888'16548) modify 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.1315259.0:43564 2014-08-11 16:46:59.326369 2014-08-13 18:06:06.621595 7f5c973a57a0 20 read_log 192888'16648 (192888'16550) modify def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.1315259.0:45270 2014-08-11 16:46:59.820029 2014-08-13 18:06:06.621608 7f5c973a57a0 20 read_log 192888'16649 (192888'16552) modify cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.1315259.0:46216 2014-08-11 16:47:00.094619 2014-08-13 18:06:06.621620 7f5c973a57a0 20 read_log 192888'16653 (192888'16560) modify 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.1315259.0:46863 2014-08-11 16:47:00.279805 2014-08-13 18:06:06.621639 7f5c973a57a0 20 read_log 192888'16655 (192888'16564) modify 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.1315259.0:47890 2014-08-11 16:47:00.599029 2014-08-13 18:06:06.621655 7f5c973a57a0 20 read_log 192888'16657 (192888'16568) modify fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.1315259.0:48014 2014-08-11 16:47:00.645284 2014-08-13 18:06:06.621668 7f5c973a57a0 20 read_log 192888'16660 (192888'16574) modify e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.1315259.0:51495 2014-08-11 16:47:01.697480 2014-08-13 18:06:06.621681 7f5c973a57a0 20 read_log 192888'16662 (192888'16578) modify 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.1315259.0:52200 2014-08-11 16:47:01.904483 2014-08-13 18:06:06.621693 7f5c973a57a0 20 read_log 192888'16664 (192888'16582) modify 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.1315259.0:53138 2014-08-11 16:47:02.166007 2014-08-13 18:06:06.621705 7f5c973a57a0 20 read_log 192888'16667 (192888'16588) modify e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.1315259.0:54707 2014-08-11 16:47:02.659072 2014-08-13 18:06:06.621728 7f5c973a57a0 20 read_log 192888'16668 (192888'16590) modify 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.1315259.0:55018 2014-08-11 16:47:02.788508 2014-08-13 18:06:06.621743 7f5c973a57a0 20 read_log 192888'16670 (192888'16594) modify 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.1315259.0:55947 2014-08-11 16:47:03.060907 2014-08-13 18:06:06.621756 7f5c973a57a0 20 read_log 192888'16676 (192888'16606) modify 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.1315259.0:58200 2014-08-11 16:47:03.733487 2014-08-13 18:06:06.621778 7f5c973a57a0 20 read_log 192888'16679 (192888'16612) modify bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.1315259.0:60413 2014-08-11 16:47:04.437455 2014-08-13 18:06:06.621793 7f5c973a57a0 20 read_log 192888'16682 (192888'16618) modify 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.1315259.0:61335 2014-08-11 16:47:04.707176 2014-08-13 18:06:06.621806 7f5c973a57a0 20 read_log 192888'16683 (192888'16683) modify 9b/hit_set_24.9b_archive_2014-08-11 16:44:55.575969_2014-08-11 17:05:18.967052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:18.967559 2014-08-13 18:06:06.621819 7f5c973a57a0 20 read_log 192888'16684 (192888'16490) delete 9b/hit_set_24.9b_archive_2014-08-11 16:02:39.145403_2014-08-11 16:44:55.575365/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:18.967559 2014-08-13 18:06:06.621831 7f5c973a57a0 20 read_log 192888'16685 (0'0) promote ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by osd.4.0:22667 2014-08-11 17:05:18.972798 2014-08-13 18:06:06.621846 7f5c973a57a0 20 read_log 192888'16686 (192888'16685) modify ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.942868.0:970 2014-08-11 17:05:18.961432 2014-08-13 18:06:06.621859 7f5c973a57a0 20 read_log 192888'16689 (0'0) promote 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by osd.4.0:22689 2014-08-11 17:05:21.620176 2014-08-13 18:06:06.621871 7f5c973a57a0 20 read_log 192888'16690 (192888'16689) modify 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.942868.0:1596 2014-08-11 17:05:21.610399 2014-08-13 18:06:06.621884 7f5c973a57a0 20 read_log 192888'16691 (0'0) promote bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by osd.4.0:22691 2014-08-11 17:05:22.075070 2014-08-13 18:06:06.621900 7f5c973a57a0 20 read_log 192888'16692 (192888'16691) modify bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.942868.0:1709 2014-08-11 17:05:22.067725 2014-08-13 18:06:06.621915 7f5c973a57a0 20 read_log 192888'16699 (0'0) promote e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by osd.4.0:22734 2014-08-11 17:05:28.610686 2014-08-13 18:06:06.621927 7f5c973a57a0 20 read_log 192888'16700 (192888'16699) modify e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.942868.0:3425 2014-08-11 17:05:28.602752 2014-08-13 18:06:06.621962 7f5c973a57a0 20 read_log 192888'16705 (0'0) promote bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by osd.4.0:22754 2014-08-11 17:05:30.566472 2014-08-13 18:06:06.621991 7f5c973a57a0 20 read_log 192888'16706 (192888'16705) modify bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.942868.0:3954 2014-08-11 17:05:30.559746 2014-08-13 18:06:06.622006 7f5c973a57a0 20 read_log 192888'16707 (0'0) promote ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by osd.4.0:22784 2014-08-11 17:05:33.758902 2014-08-13 18:06:06.622029 7f5c973a57a0 20 read_log 192888'16708 (192888'16707) modify ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.942868.0:4801 2014-08-11 17:05:33.751332 2014-08-13 18:06:06.622053 7f5c973a57a0 20 read_log 192888'16709 (0'0) promote 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by osd.4.0:22790 2014-08-11 17:05:34.208586 2014-08-13 18:06:06.622070 7f5c973a57a0 20 read_log 192888'16710 (192888'16709) modify 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.942868.0:4903 2014-08-11 17:05:34.200526 2014-08-13 18:06:06.622083 7f5c973a57a0 20 read_log 192888'16713 (0'0) promote 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by osd.4.0:22810 2014-08-11 17:05:36.930786 2014-08-13 18:06:06.622095 7f5c973a57a0 20 read_log 192888'16714 (192888'16713) modify 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.942868.0:5618 2014-08-11 17:05:36.922962 2014-08-13 18:06:06.622108 7f5c973a57a0 20 read_log 192888'16721 (0'0) promote 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by osd.4.0:22854 2014-08-11 17:05:43.415846 2014-08-13 18:06:06.622120 7f5c973a57a0 20 read_log 192888'16722 (192888'16721) modify 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.942868.0:7356 2014-08-11 17:05:43.406311 2014-08-13 18:06:06.622132 7f5c973a57a0 20 read_log 192888'16725 (0'0) promote 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by osd.4.0:22908 2014-08-11 17:05:52.716042 2014-08-13 18:06:06.622144 7f5c973a57a0 20 read_log 192888'16726 (192888'16725) modify 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.942868.0:9864 2014-08-11 17:05:52.707977 2014-08-13 18:06:06.622177 7f5c973a57a0 20 read_log 192888'16729 (0'0) promote 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by osd.4.0:22930 2014-08-11 17:05:55.322844 2014-08-13 18:06:06.622190 7f5c973a57a0 20 read_log 192888'16730 (192888'16729) modify 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.942868.0:10588 2014-08-11 17:05:55.314947 2014-08-13 18:06:06.622203 7f5c973a57a0 20 read_log 192888'16731 (0'0) promote cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by osd.4.0:22944 2014-08-11 17:05:57.720776 2014-08-13 18:06:06.622216 7f5c973a57a0 20 read_log 192888'16732 (192888'16731) modify cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.942868.0:11236 2014-08-11 17:05:57.711472 2014-08-13 18:06:06.622231 7f5c973a57a0 20 read_log 192888'16741 (0'0) promote e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by osd.4.0:23020 2014-08-11 17:06:09.910660 2014-08-13 18:06:06.622254 7f5c973a57a0 20 read_log 192888'16742 (192888'16741) modify e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.942868.0:14416 2014-08-11 17:06:09.901594 2014-08-13 18:06:06.622270 7f5c973a57a0 20 read_log 192888'16743 (0'0) promote 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by osd.4.0:23022 2014-08-11 17:06:10.640869 2014-08-13 18:06:06.622286 7f5c973a57a0 20 read_log 192888'16744 (192888'16743) modify 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.942868.0:14592 2014-08-11 17:06:10.632078 2014-08-13 18:06:06.622301 7f5c973a57a0 20 read_log 192888'16757 (0'0) promote 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by osd.4.0:23124 2014-08-11 17:06:24.041643 2014-08-13 18:06:06.622313 7f5c973a57a0 20 read_log 192888'16758 (192888'16757) modify 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.942868.0:18055 2014-08-11 17:06:24.032527 2014-08-13 18:06:06.622326 7f5c973a57a0 20 read_log 192888'16759 (0'0) promote ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by osd.4.0:23160 2014-08-11 17:06:30.456178 2014-08-13 18:06:06.622343 7f5c973a57a0 20 read_log 192888'16760 (192888'16759) modify ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.942868.0:19760 2014-08-11 17:06:30.447113 2014-08-13 18:06:06.622371 7f5c973a57a0 20 read_log 192888'16761 (0'0) promote bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by osd.4.0:23178 2014-08-11 17:06:32.565707 2014-08-13 18:06:06.622393 7f5c973a57a0 20 read_log 192888'16762 (192888'16761) modify bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.942868.0:20323 2014-08-11 17:06:32.556918 2014-08-13 18:06:06.622406 7f5c973a57a0 20 read_log 192888'16765 (0'0) promote 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by osd.4.0:23188 2014-08-11 17:06:35.412377 2014-08-13 18:06:06.622418 7f5c973a57a0 20 read_log 192888'16766 (192888'16765) modify 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.942868.0:21057 2014-08-11 17:06:35.403705 2014-08-13 18:06:06.622430 7f5c973a57a0 20 read_log 192888'16773 (0'0) promote 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by osd.4.0:23216 2014-08-11 17:06:38.276877 2014-08-13 18:06:06.622442 7f5c973a57a0 20 read_log 192888'16774 (192888'16773) modify 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.942868.0:21811 2014-08-11 17:06:38.269046 2014-08-13 18:06:06.622455 7f5c973a57a0 20 read_log 192888'16775 (0'0) promote 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by osd.4.0:23234 2014-08-11 17:06:41.431866 2014-08-13 18:06:06.622478 7f5c973a57a0 20 read_log 192888'16776 (192888'16775) modify 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.942868.0:22612 2014-08-11 17:06:41.423880 2014-08-13 18:06:06.622493 7f5c973a57a0 20 read_log 192888'16783 (0'0) promote 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by osd.4.0:23302 2014-08-11 17:06:48.387243 2014-08-13 18:06:06.622505 7f5c973a57a0 20 read_log 192888'16784 (192888'16783) modify 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.942868.0:24451 2014-08-11 17:06:48.378678 2014-08-13 18:06:06.622528 7f5c973a57a0 20 read_log 192888'16787 (0'0) promote efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by osd.4.0:23332 2014-08-11 17:06:51.522486 2014-08-13 18:06:06.622542 7f5c973a57a0 20 read_log 192888'16788 (192888'16787) modify efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.942868.0:25293 2014-08-11 17:06:51.512762 2014-08-13 18:06:06.622554 7f5c973a57a0 20 read_log 192888'16795 (0'0) promote 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by osd.4.0:23382 2014-08-11 17:07:00.611648 2014-08-13 18:06:06.622566 7f5c973a57a0 20 read_log 192888'16796 (192888'16795) modify 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.942868.0:27639 2014-08-11 17:07:00.601718 2014-08-13 18:06:06.622597 7f5c973a57a0 20 read_log 192888'16797 (0'0) promote c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by osd.4.0:23394 2014-08-11 17:07:01.701078 2014-08-13 18:06:06.622618 7f5c973a57a0 20 read_log 192888'16798 (192888'16797) modify c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.942868.0:27920 2014-08-11 17:07:01.692805 2014-08-13 18:06:06.622631 7f5c973a57a0 20 read_log 192888'16803 (0'0) promote 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by osd.4.0:23436 2014-08-11 17:07:09.757402 2014-08-13 18:06:06.622643 7f5c973a57a0 20 read_log 192888'16804 (192888'16803) modify 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.942868.0:29985 2014-08-11 17:07:09.747032 2014-08-13 18:06:06.622655 7f5c973a57a0 20 read_log 192888'16805 (0'0) promote e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by osd.4.0:23450 2014-08-11 17:07:11.402054 2014-08-13 18:06:06.622667 7f5c973a57a0 20 read_log 192888'16806 (192888'16805) modify e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.942868.0:30392 2014-08-11 17:07:11.393287 2014-08-13 18:06:06.622690 7f5c973a57a0 20 read_log 192888'16811 (192888'16686) modify ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.942868.0:32236 2014-08-11 17:07:16.195017 2014-08-13 18:06:06.622705 7f5c973a57a0 20 read_log 192888'16813 (192888'16690) modify 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.942868.0:32862 2014-08-11 17:07:16.363979 2014-08-13 18:06:06.622718 7f5c973a57a0 20 read_log 192888'16814 (192888'16692) modify bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.942868.0:32975 2014-08-11 17:07:16.392657 2014-08-13 18:06:06.622730 7f5c973a57a0 20 read_log 192888'16818 (192888'16700) modify e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.942868.0:34691 2014-08-11 17:07:16.865761 2014-08-13 18:06:06.622743 7f5c973a57a0 20 read_log 192888'16821 (192888'16706) modify bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.942868.0:35220 2014-08-11 17:07:17.018744 2014-08-13 18:06:06.622755 7f5c973a57a0 20 read_log 192888'16822 (192888'16708) modify ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.942868.0:36067 2014-08-11 17:07:17.233702 2014-08-13 18:06:06.622767 7f5c973a57a0 20 read_log 192888'16823 (192888'16710) modify 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.942868.0:36169 2014-08-11 17:07:17.260112 2014-08-13 18:06:06.622779 7f5c973a57a0 20 read_log 192888'16825 (192888'16714) modify 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.942868.0:36884 2014-08-11 17:07:17.458618 2014-08-13 18:06:06.622791 7f5c973a57a0 20 read_log 192888'16829 (192888'16722) modify 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.942868.0:38622 2014-08-11 17:07:17.940788 2014-08-13 18:06:06.622803 7f5c973a57a0 20 read_log 192888'16831 (192888'16726) modify 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.942868.0:41130 2014-08-11 17:07:18.650076 2014-08-13 18:06:06.622815 7f5c973a57a0 20 read_log 192888'16833 (192888'16730) modify 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.942868.0:41854 2014-08-11 17:07:18.840959 2014-08-13 18:06:06.622828 7f5c973a57a0 20 read_log 192888'16834 (192888'16732) modify cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.942868.0:42502 2014-08-11 17:07:19.014254 2014-08-13 18:06:06.622840 7f5c973a57a0 20 read_log 192888'16839 (192888'16742) modify e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.942868.0:45682 2014-08-11 17:07:19.905192 2014-08-13 18:06:06.622853 7f5c973a57a0 20 read_log 192888'16840 (192888'16744) modify 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.942868.0:45858 2014-08-11 17:07:19.954430 2014-08-13 18:06:06.622865 7f5c973a57a0 20 read_log 192888'16847 (192888'16758) modify 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.942868.0:49321 2014-08-11 17:07:20.977700 2014-08-13 18:06:06.622878 7f5c973a57a0 20 read_log 192888'16848 (192888'16760) modify ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.942868.0:51026 2014-08-11 17:07:21.507165 2014-08-13 18:06:06.622890 7f5c973a57a0 20 read_log 192888'16849 (192888'16762) modify bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.942868.0:51589 2014-08-11 17:07:21.690314 2014-08-13 18:06:06.622902 7f5c973a57a0 20 read_log 192888'16851 (192888'16766) modify 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.942868.0:52323 2014-08-11 17:07:21.903144 2014-08-13 18:06:06.622916 7f5c973a57a0 20 read_log 192888'16855 (192888'16774) modify 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.942868.0:53077 2014-08-11 17:07:22.140725 2014-08-13 18:06:06.622929 7f5c973a57a0 20 read_log 192888'16856 (192888'16776) modify 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.942868.0:53878 2014-08-11 17:07:22.403187 2014-08-13 18:06:06.622964 7f5c973a57a0 20 read_log 192888'16860 (192888'16784) modify 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.942868.0:55717 2014-08-11 17:07:22.932737 2014-08-13 18:06:06.622980 7f5c973a57a0 20 read_log 192888'16862 (192888'16788) modify efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.942868.0:56559 2014-08-11 17:07:23.191514 2014-08-13 18:06:06.622993 7f5c973a57a0 20 read_log 192888'16866 (192888'16796) modify 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.942868.0:58905 2014-08-11 17:07:23.886567 2014-08-13 18:06:06.623007 7f5c973a57a0 20 read_log 192888'16867 (192888'16798) modify c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.942868.0:59186 2014-08-11 17:07:23.970039 2014-08-13 18:06:06.623019 7f5c973a57a0 20 read_log 192888'16870 (192888'16804) modify 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.942868.0:61251 2014-08-11 17:07:24.584930 2014-08-13 18:06:06.623034 7f5c973a57a0 20 read_log 192888'16871 (192888'16806) modify e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.942868.0:61658 2014-08-11 17:07:24.710255 2014-08-13 18:06:06.623046 7f5c973a57a0 20 read_log 193305'16874 (193305'16874) modify 9b/hit_set_24.9b_archive_2014-08-11 18:37:04.904270_2014-08-11 18:42:39.916705/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:39.916923 2014-08-13 18:06:06.623059 7f5c973a57a0 20 read_log 193305'16875 (192888'16683) delete 9b/hit_set_24.9b_archive_2014-08-11 16:44:55.575969_2014-08-11 17:05:18.967052/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:39.916923 2014-08-13 18:06:06.623071 7f5c973a57a0 20 read_log 193305'16876 (0'0) promote 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by osd.4.0:21 2014-08-11 18:42:39.945347 2014-08-13 18:06:06.623086 7f5c973a57a0 20 read_log 193305'16877 (193305'16876) modify 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.951337.0:1779 2014-08-11 18:42:39.911239 2014-08-13 18:06:06.623098 7f5c973a57a0 20 read_log 193305'16878 (0'0) promote da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by osd.4.0:28 2014-08-11 18:42:40.890732 2014-08-13 18:06:06.623112 7f5c973a57a0 20 read_log 193305'16879 (193305'16878) modify da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.951337.0:2036 2014-08-11 18:42:40.884250 2014-08-13 18:06:06.623125 7f5c973a57a0 20 read_log 193305'16880 (0'0) promote 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by osd.4.0:43 2014-08-11 18:42:43.940584 2014-08-13 18:06:06.623138 7f5c973a57a0 20 read_log 193305'16881 (193305'16880) modify 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.951337.0:2876 2014-08-11 18:42:43.933265 2014-08-13 18:06:06.623158 7f5c973a57a0 20 read_log 193305'16882 (0'0) promote 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by osd.4.0:53 2014-08-11 18:42:47.435949 2014-08-13 18:06:06.623173 7f5c973a57a0 20 read_log 193305'16883 (193305'16882) modify 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.951337.0:3838 2014-08-11 18:42:47.428679 2014-08-13 18:06:06.623186 7f5c973a57a0 20 read_log 193305'16884 (0'0) promote e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by osd.4.0:55 2014-08-11 18:42:48.232307 2014-08-13 18:06:06.623198 7f5c973a57a0 20 read_log 193305'16885 (193305'16884) modify e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.951337.0:4057 2014-08-11 18:42:48.224447 2014-08-13 18:06:06.623211 7f5c973a57a0 20 read_log 193305'16886 (0'0) promote 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by osd.4.0:57 2014-08-11 18:42:48.436760 2014-08-13 18:06:06.623231 7f5c973a57a0 20 read_log 193305'16887 (193305'16886) modify 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.951337.0:4113 2014-08-11 18:42:48.429005 2014-08-13 18:06:06.623246 7f5c973a57a0 20 read_log 193305'16888 (0'0) promote 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by osd.4.0:113 2014-08-11 18:42:59.535690 2014-08-13 18:06:06.623259 7f5c973a57a0 20 read_log 193305'16889 (193305'16888) modify 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.951337.0:7135 2014-08-11 18:42:59.525590 2014-08-13 18:06:06.623277 7f5c973a57a0 20 read_log 193305'16890 (0'0) promote 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by osd.4.0:129 2014-08-11 18:43:02.222602 2014-08-13 18:06:06.623289 7f5c973a57a0 20 read_log 193305'16891 (193305'16890) modify 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.951337.0:7879 2014-08-11 18:43:02.214874 2014-08-13 18:06:06.623302 7f5c973a57a0 20 read_log 193305'16892 (0'0) promote b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by osd.4.0:155 2014-08-11 18:43:08.551901 2014-08-13 18:06:06.623317 7f5c973a57a0 20 read_log 193305'16893 (193305'16892) modify b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.951337.0:9523 2014-08-11 18:43:08.544108 2014-08-13 18:06:06.623330 7f5c973a57a0 20 read_log 193305'16894 (0'0) promote 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by osd.4.0:165 2014-08-11 18:43:10.482916 2014-08-13 18:06:06.623342 7f5c973a57a0 20 read_log 193305'16895 (193305'16894) modify 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.951337.0:10052 2014-08-11 18:43:10.475665 2014-08-13 18:06:06.623364 7f5c973a57a0 20 read_log 193305'16896 (0'0) promote 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by osd.4.0:173 2014-08-11 18:43:13.503107 2014-08-13 18:06:06.623379 7f5c973a57a0 20 read_log 193305'16897 (193305'16896) modify 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.951337.0:10865 2014-08-11 18:43:13.494945 2014-08-13 18:06:06.623401 7f5c973a57a0 20 read_log 193305'16898 (0'0) promote 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by osd.4.0:195 2014-08-11 18:43:16.495178 2014-08-13 18:06:06.623418 7f5c973a57a0 20 read_log 193305'16899 (193305'16898) modify 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.951337.0:11619 2014-08-11 18:43:16.487933 2014-08-13 18:06:06.623433 7f5c973a57a0 20 read_log 193305'16900 (0'0) promote 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by osd.4.0:219 2014-08-11 18:43:23.568978 2014-08-13 18:06:06.623456 7f5c973a57a0 20 read_log 193305'16901 (193305'16900) modify 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.951337.0:13502 2014-08-11 18:43:23.562646 2014-08-13 18:06:06.623469 7f5c973a57a0 20 read_log 193305'16902 (0'0) promote 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by osd.4.0:233 2014-08-11 18:43:26.822692 2014-08-13 18:06:06.623481 7f5c973a57a0 20 read_log 193305'16903 (193305'16902) modify 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.951337.0:14378 2014-08-11 18:43:26.815012 2014-08-13 18:06:06.623504 7f5c973a57a0 20 read_log 193305'16904 (0'0) promote 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by osd.4.0:239 2014-08-11 18:43:29.291999 2014-08-13 18:06:06.623517 7f5c973a57a0 20 read_log 193305'16905 (193305'16904) modify 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.951337.0:15096 2014-08-11 18:43:29.284821 2014-08-13 18:06:06.623530 7f5c973a57a0 20 read_log 193305'16906 (0'0) promote 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by osd.4.0:247 2014-08-11 18:43:30.263105 2014-08-13 18:06:06.623560 7f5c973a57a0 20 read_log 193305'16907 (193305'16906) modify 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.951337.0:15381 2014-08-11 18:43:30.254664 2014-08-13 18:06:06.623577 7f5c973a57a0 20 read_log 193305'16908 (0'0) promote baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by osd.4.0:263 2014-08-11 18:43:33.548506 2014-08-13 18:06:06.623601 7f5c973a57a0 20 read_log 193305'16909 (193305'16908) modify baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.951337.0:16311 2014-08-11 18:43:33.540497 2014-08-13 18:06:06.623617 7f5c973a57a0 20 read_log 193305'16910 (0'0) promote 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by osd.4.0:289 2014-08-11 18:43:36.563396 2014-08-13 18:06:06.623629 7f5c973a57a0 20 read_log 193305'16911 (193305'16910) modify 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.951337.0:17174 2014-08-11 18:43:36.555372 2014-08-13 18:06:06.623642 7f5c973a57a0 20 read_log 193305'16912 (0'0) promote 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by osd.4.0:291 2014-08-11 18:43:36.792137 2014-08-13 18:06:06.623654 7f5c973a57a0 20 read_log 193305'16913 (193305'16912) modify 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.951337.0:17244 2014-08-11 18:43:36.785395 2014-08-13 18:06:06.623666 7f5c973a57a0 20 read_log 193305'16914 (0'0) promote ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by osd.4.0:293 2014-08-11 18:43:37.104486 2014-08-13 18:06:06.623679 7f5c973a57a0 20 read_log 193305'16915 (193305'16914) modify ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.951337.0:17339 2014-08-11 18:43:37.097077 2014-08-13 18:06:06.623692 7f5c973a57a0 20 read_log 193305'16916 (0'0) promote 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by osd.4.0:305 2014-08-11 18:43:39.599624 2014-08-13 18:06:06.623704 7f5c973a57a0 20 read_log 193305'16917 (193305'16916) modify 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.951337.0:18066 2014-08-11 18:43:39.589107 2014-08-13 18:06:06.623716 7f5c973a57a0 20 read_log 193305'16918 (0'0) promote e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by osd.4.0:327 2014-08-11 18:43:43.705573 2014-08-13 18:06:06.623728 7f5c973a57a0 20 read_log 193305'16919 (193305'16918) modify e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.951337.0:19195 2014-08-11 18:43:43.697602 2014-08-13 18:06:06.623740 7f5c973a57a0 20 read_log 193305'16920 (0'0) promote bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by osd.4.0:329 2014-08-11 18:43:44.037601 2014-08-13 18:06:06.623752 7f5c973a57a0 20 read_log 193305'16921 (193305'16920) modify bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.951337.0:19274 2014-08-11 18:43:44.030086 2014-08-13 18:06:06.623765 7f5c973a57a0 20 read_log 193305'16922 (0'0) promote 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by osd.4.0:357 2014-08-11 18:43:52.546066 2014-08-13 18:06:06.623777 7f5c973a57a0 20 read_log 193305'16923 (193305'16922) modify 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.951337.0:21661 2014-08-11 18:43:52.539035 2014-08-13 18:06:06.623789 7f5c973a57a0 20 read_log 193305'16924 (0'0) promote 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by osd.4.0:361 2014-08-11 18:43:53.172007 2014-08-13 18:06:06.623801 7f5c973a57a0 20 read_log 193305'16925 (193305'16924) modify 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.951337.0:21824 2014-08-11 18:43:53.162001 2014-08-13 18:06:06.623813 7f5c973a57a0 20 read_log 193305'16926 (0'0) promote 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by osd.4.0:375 2014-08-11 18:43:55.695314 2014-08-13 18:06:06.623833 7f5c973a57a0 20 read_log 193305'16927 (193305'16926) modify 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.951337.0:22509 2014-08-11 18:43:55.686776 2014-08-13 18:06:06.623846 7f5c973a57a0 20 read_log 193305'16928 (0'0) promote 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by osd.4.0:391 2014-08-11 18:43:57.788696 2014-08-13 18:06:06.623866 7f5c973a57a0 20 read_log 193305'16929 (193305'16928) modify 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.951337.0:23100 2014-08-11 18:43:57.781052 2014-08-13 18:06:06.623882 7f5c973a57a0 20 read_log 193305'16930 (0'0) promote bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by osd.4.0:403 2014-08-11 18:44:02.034981 2014-08-13 18:06:06.623895 7f5c973a57a0 20 read_log 193305'16931 (193305'16930) modify bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.951337.0:24275 2014-08-11 18:44:02.026404 2014-08-13 18:06:06.623907 7f5c973a57a0 20 read_log 193305'16932 (0'0) promote 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by osd.4.0:409 2014-08-11 18:44:04.369233 2014-08-13 18:06:06.623920 7f5c973a57a0 20 read_log 193305'16933 (193305'16932) modify 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.951337.0:24915 2014-08-11 18:44:04.360674 2014-08-13 18:06:06.623934 7f5c973a57a0 20 read_log 193305'16934 (0'0) promote 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by osd.4.0:425 2014-08-11 18:44:08.564579 2014-08-13 18:06:06.623949 7f5c973a57a0 20 read_log 193305'16935 (193305'16934) modify 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.951337.0:26042 2014-08-11 18:44:08.556988 2014-08-13 18:06:06.623961 7f5c973a57a0 20 read_log 193305'16936 (0'0) promote 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by osd.4.0:431 2014-08-11 18:44:09.511400 2014-08-13 18:06:06.624046 7f5c973a57a0 20 read_log 193305'16937 (193305'16936) modify 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.951337.0:26307 2014-08-11 18:44:09.503337 2014-08-13 18:06:06.624064 7f5c973a57a0 20 read_log 193305'16938 (0'0) promote bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by osd.4.0:441 2014-08-11 18:44:11.183938 2014-08-13 18:06:06.624076 7f5c973a57a0 20 read_log 193305'16939 (193305'16938) modify bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.951337.0:26776 2014-08-11 18:44:11.173117 2014-08-13 18:06:06.624089 7f5c973a57a0 20 read_log 193305'16940 (0'0) promote 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by osd.4.0:443 2014-08-11 18:44:11.368213 2014-08-13 18:06:06.624101 7f5c973a57a0 20 read_log 193305'16941 (193305'16940) modify 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.951337.0:26828 2014-08-11 18:44:11.359412 2014-08-13 18:06:06.624113 7f5c973a57a0 20 read_log 193305'16942 (0'0) promote 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by osd.4.0:445 2014-08-11 18:44:11.396123 2014-08-13 18:06:06.624128 7f5c973a57a0 20 read_log 193305'16943 (193305'16942) modify 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.951337.0:26836 2014-08-11 18:44:11.387380 2014-08-13 18:06:06.624140 7f5c973a57a0 20 read_log 193305'16944 (0'0) promote b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by osd.4.0:497 2014-08-11 18:44:23.872355 2014-08-13 18:06:06.624163 7f5c973a57a0 20 read_log 193305'16945 (193305'16944) modify b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.951337.0:30216 2014-08-11 18:44:23.863648 2014-08-13 18:06:06.624176 7f5c973a57a0 20 read_log 193305'16946 (0'0) promote 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by osd.4.0:503 2014-08-11 18:44:26.080734 2014-08-13 18:06:06.624201 7f5c973a57a0 20 read_log 193305'16947 (193305'16946) modify 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.951337.0:30791 2014-08-11 18:44:26.071039 2014-08-13 18:06:06.624216 7f5c973a57a0 20 read_log 193305'16948 (0'0) promote 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by osd.4.0:533 2014-08-11 18:44:30.697342 2014-08-13 18:06:06.624228 7f5c973a57a0 20 read_log 193305'16949 (193305'16948) modify 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.951337.0:32125 2014-08-11 18:44:30.689994 2014-08-13 18:06:06.624241 7f5c973a57a0 20 read_log 193305'16950 (0'0) promote be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by osd.4.0:535 2014-08-11 18:44:30.731453 2014-08-13 18:06:06.624256 7f5c973a57a0 20 read_log 193305'16951 (193305'16950) modify be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.951337.0:32135 2014-08-11 18:44:30.724267 2014-08-13 18:06:06.624271 7f5c973a57a0 20 read_log 193305'16952 (0'0) promote 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by osd.4.0:539 2014-08-11 18:44:31.034869 2014-08-13 18:06:06.624283 7f5c973a57a0 20 read_log 193305'16953 (193305'16952) modify 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.951337.0:32216 2014-08-11 18:44:31.026260 2014-08-13 18:06:06.624296 7f5c973a57a0 20 read_log 193305'16954 (193305'16877) modify 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.951337.0:34490 2014-08-11 18:44:35.027593 2014-08-13 18:06:06.624308 7f5c973a57a0 20 read_log 193305'16955 (193305'16879) modify da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.951337.0:34747 2014-08-11 18:44:35.166935 2014-08-13 18:06:06.624320 7f5c973a57a0 20 read_log 193305'16956 (193305'16881) modify 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.951337.0:35587 2014-08-11 18:44:35.669508 2014-08-13 18:06:06.624332 7f5c973a57a0 20 read_log 193305'16957 (193305'16883) modify 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.951337.0:36549 2014-08-11 18:44:36.318001 2014-08-13 18:06:06.624349 7f5c973a57a0 20 read_log 193305'16958 (193305'16885) modify e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.951337.0:36768 2014-08-11 18:44:36.535112 2014-08-13 18:06:06.624362 7f5c973a57a0 20 read_log 193305'16959 (193305'16887) modify 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.951337.0:36824 2014-08-11 18:44:36.557272 2014-08-13 18:06:06.624382 7f5c973a57a0 20 read_log 193305'16960 (193305'16889) modify 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.951337.0:39846 2014-08-11 18:44:38.205272 2014-08-13 18:06:06.624402 7f5c973a57a0 20 read_log 193305'16961 (193305'16891) modify 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.951337.0:40590 2014-08-11 18:44:38.456609 2014-08-13 18:06:06.624414 7f5c973a57a0 20 read_log 193305'16962 (193305'16893) modify b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.951337.0:42234 2014-08-11 18:44:39.159082 2014-08-13 18:06:06.624434 7f5c973a57a0 20 read_log 193305'16963 (193305'16895) modify 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.951337.0:42763 2014-08-11 18:44:39.371984 2014-08-13 18:06:06.624453 7f5c973a57a0 20 read_log 193305'16964 (193305'16897) modify 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.951337.0:43576 2014-08-11 18:44:39.631631 2014-08-13 18:06:06.624466 7f5c973a57a0 20 read_log 193305'16965 (193305'16899) modify 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.951337.0:44330 2014-08-11 18:44:39.853658 2014-08-13 18:06:06.624489 7f5c973a57a0 20 read_log 193305'16966 (193305'16901) modify 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.951337.0:46213 2014-08-11 18:44:40.421654 2014-08-13 18:06:06.624512 7f5c973a57a0 20 read_log 193305'16967 (193305'16903) modify 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.951337.0:47089 2014-08-11 18:44:40.671382 2014-08-13 18:06:06.624527 7f5c973a57a0 20 read_log 193305'16968 (193305'16905) modify 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.951337.0:47807 2014-08-11 18:44:40.870799 2014-08-13 18:06:06.624539 7f5c973a57a0 20 read_log 193305'16969 (193305'16907) modify 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.951337.0:48092 2014-08-11 18:44:40.950773 2014-08-13 18:06:06.624552 7f5c973a57a0 20 read_log 193305'16970 (193305'16909) modify baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.951337.0:49022 2014-08-11 18:44:41.228766 2014-08-13 18:06:06.624564 7f5c973a57a0 20 read_log 193305'16971 (193305'16911) modify 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.951337.0:49885 2014-08-11 18:44:41.506848 2014-08-13 18:06:06.624576 7f5c973a57a0 20 read_log 193305'16972 (193305'16913) modify 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.951337.0:49955 2014-08-11 18:44:41.525178 2014-08-13 18:06:06.624588 7f5c973a57a0 20 read_log 193305'16973 (193305'16915) modify ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.951337.0:50050 2014-08-11 18:44:41.551453 2014-08-13 18:06:06.624609 7f5c973a57a0 20 read_log 193305'16974 (193305'16917) modify 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.951337.0:50777 2014-08-11 18:44:41.758112 2014-08-13 18:06:06.624622 7f5c973a57a0 20 read_log 193305'16975 (193305'16919) modify e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.951337.0:51906 2014-08-11 18:44:42.083576 2014-08-13 18:06:06.624634 7f5c973a57a0 20 read_log 193305'16976 (193305'16921) modify bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.951337.0:51985 2014-08-11 18:44:42.102828 2014-08-13 18:06:06.624647 7f5c973a57a0 20 read_log 193305'16977 (193305'16923) modify 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.951337.0:54372 2014-08-11 18:44:42.813466 2014-08-13 18:06:06.624659 7f5c973a57a0 20 read_log 193305'16978 (193305'16925) modify 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.951337.0:54535 2014-08-11 18:44:42.866356 2014-08-13 18:06:06.624671 7f5c973a57a0 20 read_log 193305'16979 (193305'16927) modify 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.951337.0:55220 2014-08-11 18:44:43.060423 2014-08-13 18:06:06.624684 7f5c973a57a0 20 read_log 193305'16980 (193305'16929) modify 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.951337.0:55811 2014-08-11 18:44:43.238051 2014-08-13 18:06:06.624696 7f5c973a57a0 20 read_log 193305'16981 (193305'16931) modify bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.951337.0:56986 2014-08-11 18:44:43.593001 2014-08-13 18:06:06.624708 7f5c973a57a0 20 read_log 193305'16982 (193305'16933) modify 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.951337.0:57626 2014-08-11 18:44:43.811499 2014-08-13 18:06:06.624720 7f5c973a57a0 20 read_log 193305'16983 (193305'16935) modify 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.951337.0:58753 2014-08-11 18:44:44.132441 2014-08-13 18:06:06.624732 7f5c973a57a0 20 read_log 193305'16984 (193305'16937) modify 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.951337.0:59018 2014-08-11 18:44:44.211540 2014-08-13 18:06:06.624745 7f5c973a57a0 20 read_log 193305'16985 (193305'16939) modify bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.951337.0:59487 2014-08-11 18:44:44.340653 2014-08-13 18:06:06.624757 7f5c973a57a0 20 read_log 193305'16986 (193305'16941) modify 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.951337.0:59539 2014-08-11 18:44:44.356882 2014-08-13 18:06:06.624777 7f5c973a57a0 20 read_log 193305'16987 (193305'16943) modify 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.951337.0:59547 2014-08-11 18:44:44.358266 2014-08-13 18:06:06.624792 7f5c973a57a0 20 read_log 193305'16988 (193305'16945) modify b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.951337.0:62927 2014-08-11 18:44:45.364342 2014-08-13 18:06:06.624805 7f5c973a57a0 20 read_log 193305'16989 (193305'16947) modify 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.951337.0:63502 2014-08-11 18:44:45.526590 2014-08-13 18:06:06.624817 7f5c973a57a0 20 read_log 193305'16990 (193305'16949) modify 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.951337.0:64836 2014-08-11 18:44:45.935299 2014-08-13 18:06:06.624829 7f5c973a57a0 20 read_log 193305'16991 (193305'16951) modify be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.951337.0:64846 2014-08-11 18:44:45.936177 2014-08-13 18:06:06.624841 7f5c973a57a0 20 read_log 193305'16992 (193305'16953) modify 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.951337.0:64927 2014-08-11 18:44:45.961220 2014-08-13 18:06:06.624854 7f5c973a57a0 20 read_log 194215'16993 (194215'16993) modify 9b/hit_set_24.9b_archive_2014-08-12 15:32:25.864657_2014-08-12 15:38:31.718183/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:31.718441 2014-08-13 18:06:06.624866 7f5c973a57a0 20 read_log 194215'16994 (193305'16874) delete 9b/hit_set_24.9b_archive_2014-08-11 18:37:04.904270_2014-08-11 18:42:39.916705/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:31.718441 2014-08-13 18:06:06.624878 7f5c973a57a0 20 read_log 194215'16995 (192888'16811) clean ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by osd.4.0:1218 2014-08-12 15:38:31.723310 2014-08-13 18:06:06.624890 7f5c973a57a0 20 read_log 194215'16996 (194215'16995) delete ca46009b/benchmark_data_storage0111-ib.pocstor_2228_object969/head//24 by client.2174595.0:31352 0.000000 2014-08-13 18:06:06.624902 7f5c973a57a0 20 read_log 194215'16997 (193305'16959) clean 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by osd.4.0:1220 2014-08-12 15:38:31.765829 2014-08-13 18:06:06.624919 7f5c973a57a0 20 read_log 194215'16998 (194215'16997) delete 73cb009b/benchmark_data_storage0111-ib.pocstor_62621_object4112/head//24 by client.2174595.0:31354 0.000000 2014-08-13 18:06:06.624932 7f5c973a57a0 20 read_log 194215'16999 (193305'16956) clean 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by osd.4.0:1222 2014-08-12 15:38:31.774738 2014-08-13 18:06:06.624950 7f5c973a57a0 20 read_log 194215'17000 (194215'16999) delete 9bed009b/benchmark_data_storage0111-ib.pocstor_62621_object2875/head//24 by client.2174595.0:31356 0.000000 2014-08-13 18:06:06.624961 7f5c973a57a0 20 read_log 194215'17001 (192888'16664) clean 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by osd.4.0:1224 2014-08-12 15:38:31.783784 2014-08-13 18:06:06.624975 7f5c973a57a0 20 read_log 194215'17002 (194215'17001) delete 9443209b/benchmark_data_storage0111-ib.pocstor_41097_object22410/head//24 by client.2174595.0:31358 0.000000 2014-08-13 18:06:06.624990 7f5c973a57a0 20 read_log 194215'17003 (192888'16662) clean 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by osd.4.0:1226 2014-08-12 15:38:31.791960 2014-08-13 18:06:06.625002 7f5c973a57a0 20 read_log 194215'17004 (194215'17003) delete 89d0309b/benchmark_data_storage0111-ib.pocstor_41097_object21472/head//24 by client.2174595.0:31360 0.000000 2014-08-13 18:06:06.625016 7f5c973a57a0 20 read_log 194215'17005 (193305'16961) clean 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by osd.4.0:1228 2014-08-12 15:38:31.801209 2014-08-13 18:06:06.625057 7f5c973a57a0 20 read_log 194215'17006 (194215'17005) delete 9ca3309b/benchmark_data_storage0111-ib.pocstor_62621_object7878/head//24 by client.2174595.0:31362 0.000000 2014-08-13 18:06:06.625071 7f5c973a57a0 20 read_log 194215'17007 (192888'16862) clean efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by osd.4.0:1230 2014-08-12 15:38:31.826654 2014-08-13 18:06:06.625084 7f5c973a57a0 20 read_log 194215'17008 (194215'17007) delete efc6309b/benchmark_data_storage0111-ib.pocstor_2228_object25292/head//24 by client.2174595.0:31364 0.000000 2014-08-13 18:06:06.625096 7f5c973a57a0 20 read_log 194215'17009 (192888'16839) clean e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by osd.4.0:1232 2014-08-12 15:38:31.836138 2014-08-13 18:06:06.625110 7f5c973a57a0 20 read_log 194215'17010 (194215'17009) delete e353409b/benchmark_data_storage0111-ib.pocstor_2228_object14415/head//24 by client.2174595.0:31366 0.000000 2014-08-13 18:06:06.625121 7f5c973a57a0 20 read_log 194215'17011 (193305'16985) clean bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by osd.4.0:1234 2014-08-12 15:38:31.844546 2014-08-13 18:06:06.625133 7f5c973a57a0 20 read_log 194215'17012 (194215'17011) delete bf5e409b/benchmark_data_storage0111-ib.pocstor_62621_object26775/head//24 by client.2174595.0:31368 0.000000 2014-08-13 18:06:06.625144 7f5c973a57a0 20 read_log 194215'17013 (192888'16648) clean def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by osd.4.0:1236 2014-08-12 15:38:31.852863 2014-08-13 18:06:06.625164 7f5c973a57a0 20 read_log 194215'17014 (194215'17013) delete def2509b/benchmark_data_storage0111-ib.pocstor_41097_object14542/head//24 by client.2174595.0:31370 0.000000 2014-08-13 18:06:06.625175 7f5c973a57a0 20 read_log 194215'17015 (193305'16957) clean 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by osd.4.0:1238 2014-08-12 15:38:31.861378 2014-08-13 18:06:06.625188 7f5c973a57a0 20 read_log 194215'17016 (194215'17015) delete 63e6509b/benchmark_data_storage0111-ib.pocstor_62621_object3837/head//24 by client.2174595.0:31372 0.000000 2014-08-13 18:06:06.625199 7f5c973a57a0 20 read_log 194215'17017 (192888'16655) clean 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by osd.4.0:1240 2014-08-12 15:38:31.869387 2014-08-13 18:06:06.625211 7f5c973a57a0 20 read_log 194215'17018 (194215'17017) delete 9efc509b/benchmark_data_storage0111-ib.pocstor_41097_object17162/head//24 by client.2174595.0:31374 0.000000 2014-08-13 18:06:06.625224 7f5c973a57a0 20 read_log 194215'17019 (193305'16958) clean e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by osd.4.0:1242 2014-08-12 15:38:31.877987 2014-08-13 18:06:06.625238 7f5c973a57a0 20 read_log 194215'17020 (194215'17019) delete e887709b/benchmark_data_storage0111-ib.pocstor_62621_object4056/head//24 by client.2174595.0:31376 0.000000 2014-08-13 18:06:06.625254 7f5c973a57a0 20 read_log 194215'17021 (193305'16982) clean 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by osd.4.0:1244 2014-08-12 15:38:31.886898 2014-08-13 18:06:06.625266 7f5c973a57a0 20 read_log 194215'17022 (194215'17021) delete 3a4f809b/benchmark_data_storage0111-ib.pocstor_62621_object24914/head//24 by client.2174595.0:31378 0.000000 2014-08-13 18:06:06.625277 7f5c973a57a0 20 read_log 194215'17023 (192888'16840) clean 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by osd.4.0:1246 2014-08-12 15:38:31.895022 2014-08-13 18:06:06.625292 7f5c973a57a0 20 read_log 194215'17024 (194215'17023) delete 61f4b09b/benchmark_data_storage0111-ib.pocstor_2228_object14591/head//24 by client.2174595.0:31380 0.000000 2014-08-13 18:06:06.625303 7f5c973a57a0 20 read_log 194215'17025 (192888'16679) clean bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by osd.4.0:1248 2014-08-12 15:38:31.915279 2014-08-13 18:06:06.625326 7f5c973a57a0 20 read_log 194215'17026 (194215'17025) delete bd0fb09b/benchmark_data_storage0111-ib.pocstor_41097_object29685/head//24 by client.2174595.0:31382 0.000000 2014-08-13 18:06:06.625343 7f5c973a57a0 20 read_log 194215'17027 (192888'16676) clean 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by osd.4.0:1250 2014-08-12 15:38:31.923291 2014-08-13 18:06:06.625391 7f5c973a57a0 20 read_log 194215'17028 (194215'17027) delete 136fc09b/benchmark_data_storage0111-ib.pocstor_41097_object27472/head//24 by client.2174595.0:31384 0.000000 2014-08-13 18:06:06.625406 7f5c973a57a0 20 read_log 194215'17029 (192888'16834) clean cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by osd.4.0:1252 2014-08-12 15:38:31.930907 2014-08-13 18:06:06.625419 7f5c973a57a0 20 read_log 194215'17030 (194215'17029) delete cde0d09b/benchmark_data_storage0111-ib.pocstor_2228_object11235/head//24 by client.2174595.0:31386 0.000000 2014-08-13 18:06:06.625438 7f5c973a57a0 20 read_log 194215'17031 (193305'16955) clean da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by osd.4.0:1254 2014-08-12 15:38:31.940216 2014-08-13 18:06:06.625451 7f5c973a57a0 20 read_log 194215'17032 (194215'17031) delete da4bd09b/benchmark_data_storage0111-ib.pocstor_62621_object2035/head//24 by client.2174595.0:31388 0.000000 2014-08-13 18:06:06.625462 7f5c973a57a0 20 read_log 194215'17033 (193305'16980) clean 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by osd.4.0:1256 2014-08-12 15:38:31.948960 2014-08-13 18:06:06.625485 7f5c973a57a0 20 read_log 194215'17034 (194215'17033) delete 76a3e09b/benchmark_data_storage0111-ib.pocstor_62621_object23099/head//24 by client.2174595.0:31390 0.000000 2014-08-13 18:06:06.625499 7f5c973a57a0 20 read_log 194215'17035 (192888'16848) clean ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by osd.4.0:1258 2014-08-12 15:38:31.957415 2014-08-13 18:06:06.625512 7f5c973a57a0 20 read_log 194215'17036 (194215'17035) delete ac6ce09b/benchmark_data_storage0111-ib.pocstor_2228_object19759/head//24 by client.2174595.0:31392 0.000000 2014-08-13 18:06:06.625523 7f5c973a57a0 20 read_log 194215'17037 (193305'16974) clean 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by osd.4.0:1260 2014-08-12 15:38:31.965692 2014-08-13 18:06:06.625535 7f5c973a57a0 20 read_log 194215'17038 (194215'17037) delete 7c86f09b/benchmark_data_storage0111-ib.pocstor_62621_object18065/head//24 by client.2174595.0:31394 0.000000 2014-08-13 18:06:06.625546 7f5c973a57a0 20 read_log 194215'17039 (192888'16851) clean 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by osd.4.0:1262 2014-08-12 15:38:31.975226 2014-08-13 18:06:06.625559 7f5c973a57a0 20 read_log 194215'17040 (194215'17039) delete 2acdf09b/benchmark_data_storage0111-ib.pocstor_2228_object21056/head//24 by client.2174595.0:31396 0.000000 2014-08-13 18:06:06.625572 7f5c973a57a0 20 read_log 194215'17041 (193305'16970) clean baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by osd.4.0:1264 2014-08-12 15:38:31.982937 2014-08-13 18:06:06.625586 7f5c973a57a0 20 read_log 194215'17042 (194215'17041) delete baf3049b/benchmark_data_storage0111-ib.pocstor_62621_object16310/head//24 by client.2174595.0:31398 0.000000 2014-08-13 18:06:06.625597 7f5c973a57a0 20 read_log 194215'17043 (192888'16649) clean cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by osd.4.0:1266 2014-08-12 15:38:31.991780 2014-08-13 18:06:06.625609 7f5c973a57a0 20 read_log 194215'17044 (194215'17043) delete cd34049b/benchmark_data_storage0111-ib.pocstor_41097_object15488/head//24 by client.2174595.0:31400 0.000000 2014-08-13 18:06:06.625621 7f5c973a57a0 20 read_log 194215'17045 (192888'16814) clean bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by osd.4.0:1268 2014-08-12 15:38:32.014012 2014-08-13 18:06:06.625641 7f5c973a57a0 20 read_log 194215'17046 (194215'17045) delete bcfc049b/benchmark_data_storage0111-ib.pocstor_2228_object1708/head//24 by client.2174595.0:31402 0.000000 2014-08-13 18:06:06.625655 7f5c973a57a0 20 read_log 194215'17047 (192888'16635) clean 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by osd.4.0:1270 2014-08-12 15:38:32.030280 2014-08-13 18:06:06.625678 7f5c973a57a0 20 read_log 194215'17048 (194215'17047) delete 1efb249b/benchmark_data_storage0111-ib.pocstor_41097_object7824/head//24 by client.2174595.0:31404 0.000000 2014-08-13 18:06:06.625690 7f5c973a57a0 20 read_log 194215'17049 (193305'16966) clean 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by osd.4.0:1272 2014-08-12 15:38:32.038918 2014-08-13 18:06:06.625703 7f5c973a57a0 20 read_log 194215'17050 (194215'17049) delete 449349b/benchmark_data_storage0111-ib.pocstor_62621_object13501/head//24 by client.2174595.0:31406 0.000000 2014-08-13 18:06:06.625722 7f5c973a57a0 20 read_log 194215'17051 (193305'16988) clean b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by osd.4.0:1274 2014-08-12 15:38:32.047364 2014-08-13 18:06:06.625737 7f5c973a57a0 20 read_log 194215'17052 (194215'17051) delete b522549b/benchmark_data_storage0111-ib.pocstor_62621_object30215/head//24 by client.2174595.0:31408 0.000000 2014-08-13 18:06:06.625748 7f5c973a57a0 20 read_log 194215'17053 (192888'16860) clean 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by osd.4.0:1276 2014-08-12 15:38:32.055413 2014-08-13 18:06:06.625761 7f5c973a57a0 20 read_log 194215'17054 (194215'17053) delete 6b49649b/benchmark_data_storage0111-ib.pocstor_2228_object24450/head//24 by client.2174595.0:31410 0.000000 2014-08-13 18:06:06.625772 7f5c973a57a0 20 read_log 194215'17055 (192888'16849) clean bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by osd.4.0:1278 2014-08-12 15:38:32.063029 2014-08-13 18:06:06.625784 7f5c973a57a0 20 read_log 194215'17056 (194215'17055) delete bb8c749b/benchmark_data_storage0111-ib.pocstor_2228_object20322/head//24 by client.2174595.0:31412 0.000000 2014-08-13 18:06:06.625795 7f5c973a57a0 20 read_log 194215'17057 (192888'16855) clean 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by osd.4.0:1280 2014-08-12 15:38:32.070571 2014-08-13 18:06:06.625810 7f5c973a57a0 20 read_log 194215'17058 (194215'17057) delete 6523849b/benchmark_data_storage0111-ib.pocstor_2228_object21810/head//24 by client.2174595.0:31414 0.000000 2014-08-13 18:06:06.625821 7f5c973a57a0 20 read_log 194215'17059 (192888'16823) clean 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by osd.4.0:1282 2014-08-12 15:38:32.078405 2014-08-13 18:06:06.625833 7f5c973a57a0 20 read_log 194215'17060 (194215'17059) delete 74c3849b/benchmark_data_storage0111-ib.pocstor_2228_object4902/head//24 by client.2174595.0:31416 0.000000 2014-08-13 18:06:06.625844 7f5c973a57a0 20 read_log 194215'17061 (193305'16977) clean 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by osd.4.0:1284 2014-08-12 15:38:32.085833 2014-08-13 18:06:06.625857 7f5c973a57a0 20 read_log 194215'17062 (194215'17061) delete 659b849b/benchmark_data_storage0111-ib.pocstor_62621_object21660/head//24 by client.2174595.0:31418 0.000000 2014-08-13 18:06:06.625868 7f5c973a57a0 20 read_log 194215'17063 (192888'16867) clean c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by osd.4.0:1286 2014-08-12 15:38:32.094246 2014-08-13 18:06:06.625880 7f5c973a57a0 20 read_log 194215'17064 (194215'17063) delete c79d949b/benchmark_data_storage0111-ib.pocstor_2228_object27919/head//24 by client.2174595.0:31420 0.000000 2014-08-13 18:06:06.625895 7f5c973a57a0 20 read_log 194215'17065 (192888'16667) clean e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by osd.4.0:1288 2014-08-12 15:38:32.101534 2014-08-13 18:06:06.625918 7f5c973a57a0 20 read_log 194215'17066 (194215'17065) delete e565a49b/benchmark_data_storage0111-ib.pocstor_41097_object23979/head//24 by client.2174595.0:31422 0.000000 2014-08-13 18:06:06.625944 7f5c973a57a0 20 read_log 194215'17067 (193305'16986) clean 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by osd.4.0:1290 2014-08-12 15:38:32.109966 2014-08-13 18:06:06.625960 7f5c973a57a0 20 read_log 194215'17068 (194215'17067) delete 88d6a49b/benchmark_data_storage0111-ib.pocstor_62621_object26827/head//24 by client.2174595.0:31424 0.000000 2014-08-13 18:06:06.625977 7f5c973a57a0 20 read_log 194215'17069 (192888'16831) clean 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by osd.4.0:1292 2014-08-12 15:38:32.118180 2014-08-13 18:06:06.625996 7f5c973a57a0 20 read_log 194215'17070 (194215'17069) delete 5faea49b/benchmark_data_storage0111-ib.pocstor_2228_object9863/head//24 by client.2174595.0:31426 0.000000 2014-08-13 18:06:06.626007 7f5c973a57a0 20 read_log 194215'17071 (192888'16636) clean 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by osd.4.0:1294 2014-08-12 15:38:32.126295 2014-08-13 18:06:06.626020 7f5c973a57a0 20 read_log 194215'17072 (194215'17071) delete 9031c49b/benchmark_data_storage0111-ib.pocstor_41097_object8318/head//24 by client.2174595.0:31428 0.000000 2014-08-13 18:06:06.626031 7f5c973a57a0 20 read_log 194215'17073 (192888'16637) clean a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by osd.4.0:1296 2014-08-12 15:38:32.135833 2014-08-13 18:06:06.626043 7f5c973a57a0 20 read_log 194215'17074 (194215'17073) delete a6cad49b/benchmark_data_storage0111-ib.pocstor_41097_object8519/head//24 by client.2174595.0:31430 0.000000 2014-08-13 18:06:06.626053 7f5c973a57a0 20 read_log 194215'17075 (193305'16981) clean bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by osd.4.0:1298 2014-08-12 15:38:32.144611 2014-08-13 18:06:06.626065 7f5c973a57a0 20 read_log 194215'17076 (194215'17075) delete bbead49b/benchmark_data_storage0111-ib.pocstor_62621_object24274/head//24 by client.2174595.0:31432 0.000000 2014-08-13 18:06:06.626085 7f5c973a57a0 20 read_log 194215'17077 (193305'16992) clean 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by osd.4.0:1300 2014-08-12 15:38:32.164229 2014-08-13 18:06:06.626098 7f5c973a57a0 20 read_log 194215'17078 (194215'17077) delete 9ea7e49b/benchmark_data_storage0111-ib.pocstor_62621_object32215/head//24 by client.2174595.0:31434 0.000000 2014-08-13 18:06:06.626109 7f5c973a57a0 20 read_log 194215'17079 (193305'16965) clean 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by osd.4.0:1302 2014-08-12 15:38:32.172728 2014-08-13 18:06:06.626131 7f5c973a57a0 20 read_log 194215'17080 (194215'17079) delete 73c4f49b/benchmark_data_storage0111-ib.pocstor_62621_object11618/head//24 by client.2174595.0:31436 0.000000 2014-08-13 18:06:06.626143 7f5c973a57a0 20 read_log 194215'17081 (192888'16623) clean ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by osd.4.0:1304 2014-08-12 15:38:32.180392 2014-08-13 18:06:06.626164 7f5c973a57a0 20 read_log 194215'17082 (194215'17081) delete ddeaf49b/benchmark_data_storage0111-ib.pocstor_41097_object2722/head//24 by client.2174595.0:31438 0.000000 2014-08-13 18:06:06.626179 7f5c973a57a0 20 read_log 194215'17083 (192888'16856) clean 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by osd.4.0:1306 2014-08-12 15:38:32.189187 2014-08-13 18:06:06.626192 7f5c973a57a0 20 read_log 194215'17084 (194215'17083) delete 65bef49b/benchmark_data_storage0111-ib.pocstor_2228_object22611/head//24 by client.2174595.0:31440 0.000000 2014-08-13 18:06:06.626203 7f5c973a57a0 20 read_log 194215'17085 (192888'16866) clean 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by osd.4.0:1308 2014-08-12 15:38:32.197039 2014-08-13 18:06:06.626237 7f5c973a57a0 20 read_log 194215'17086 (194215'17085) delete 4eaa089b/benchmark_data_storage0111-ib.pocstor_2228_object27638/head//24 by client.2174595.0:31442 0.000000 2014-08-13 18:06:06.626252 7f5c973a57a0 20 read_log 194215'17087 (193305'16962) clean b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by osd.4.0:1310 2014-08-12 15:38:32.204839 2014-08-13 18:06:06.626265 7f5c973a57a0 20 read_log 194215'17088 (194215'17087) delete b7f189b/benchmark_data_storage0111-ib.pocstor_62621_object9522/head//24 by client.2174595.0:31444 0.000000 2014-08-13 18:06:06.626276 7f5c973a57a0 20 read_log 194215'17089 (193305'16991) clean be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by osd.4.0:1312 2014-08-12 15:38:32.212337 2014-08-13 18:06:06.626288 7f5c973a57a0 20 read_log 194215'17090 (194215'17089) delete be17489b/benchmark_data_storage0111-ib.pocstor_62621_object32134/head//24 by client.2174595.0:31446 0.000000 2014-08-13 18:06:06.626347 7f5c973a57a0 20 read_log 194215'17091 (192888'16629) clean 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by osd.4.0:1314 2014-08-12 15:38:32.221426 2014-08-13 18:06:06.626370 7f5c973a57a0 20 read_log 194215'17092 (194215'17091) delete 3fbd489b/benchmark_data_storage0111-ib.pocstor_41097_object5178/head//24 by client.2174595.0:31448 0.000000 2014-08-13 18:06:06.626384 7f5c973a57a0 20 read_log 194215'17093 (192888'16847) clean 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by osd.4.0:1316 2014-08-12 15:38:32.229452 2014-08-13 18:06:06.626397 7f5c973a57a0 20 read_log 194215'17094 (194215'17093) delete 250c689b/benchmark_data_storage0111-ib.pocstor_2228_object18054/head//24 by client.2174595.0:31450 0.000000 2014-08-13 18:06:06.626408 7f5c973a57a0 20 read_log 194215'17095 (192888'16619) clean 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by osd.4.0:1318 2014-08-12 15:38:32.238435 2014-08-13 18:06:06.626420 7f5c973a57a0 20 read_log 194215'17096 (194215'17095) delete 7e8d789b/benchmark_data_storage0111-ib.pocstor_41097_object160/head//24 by client.2174595.0:31452 0.000000 2014-08-13 18:06:06.626431 7f5c973a57a0 20 read_log 194215'17097 (192888'16833) clean 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by osd.4.0:1320 2014-08-12 15:38:32.246224 2014-08-13 18:06:06.626444 7f5c973a57a0 20 read_log 194215'17098 (194215'17097) delete 7f39889b/benchmark_data_storage0111-ib.pocstor_2228_object10587/head//24 by client.2174595.0:31454 0.000000 2014-08-13 18:06:06.626454 7f5c973a57a0 20 read_log 194215'17099 (192888'16627) clean 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by osd.4.0:1322 2014-08-12 15:38:32.254634 2014-08-13 18:06:06.626467 7f5c973a57a0 20 read_log 194215'17100 (194215'17099) delete 9b92989b/benchmark_data_storage0111-ib.pocstor_41097_object4240/head//24 by client.2174595.0:31456 0.000000 2014-08-13 18:06:06.626478 7f5c973a57a0 20 read_log 194215'17101 (193305'16987) clean 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by osd.4.0:1324 2014-08-12 15:38:32.261454 2014-08-13 18:06:06.626490 7f5c973a57a0 20 read_log 194215'17102 (194215'17101) delete 7150a89b/benchmark_data_storage0111-ib.pocstor_62621_object26835/head//24 by client.2174595.0:31458 0.000000 2014-08-13 18:06:06.626501 7f5c973a57a0 20 read_log 194215'17103 (192888'16871) clean e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by osd.4.0:1326 2014-08-12 15:38:32.268904 2014-08-13 18:06:06.626514 7f5c973a57a0 20 read_log 194215'17104 (194215'17103) delete e342a89b/benchmark_data_storage0111-ib.pocstor_2228_object30391/head//24 by client.2174595.0:31460 0.000000 2014-08-13 18:06:06.626524 7f5c973a57a0 20 read_log 194215'17105 (192888'16657) clean fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by osd.4.0:1328 2014-08-12 15:38:32.285781 2014-08-13 18:06:06.626544 7f5c973a57a0 20 read_log 194215'17106 (194215'17105) delete fba3a89b/benchmark_data_storage0111-ib.pocstor_41097_object17286/head//24 by client.2174595.0:31462 0.000000 2014-08-13 18:06:06.626555 7f5c973a57a0 20 read_log 194215'17107 (193305'16963) clean 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by osd.4.0:1330 2014-08-12 15:38:32.294127 2014-08-13 18:06:06.626568 7f5c973a57a0 20 read_log 194215'17108 (194215'17107) delete 777ea89b/benchmark_data_storage0111-ib.pocstor_62621_object10051/head//24 by client.2174595.0:31464 0.000000 2014-08-13 18:06:06.626578 7f5c973a57a0 20 read_log 194215'17109 (193305'16990) clean 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by osd.4.0:1332 2014-08-12 15:38:32.301704 2014-08-13 18:06:06.626591 7f5c973a57a0 20 read_log 194215'17110 (194215'17109) delete 4476b89b/benchmark_data_storage0111-ib.pocstor_62621_object32124/head//24 by client.2174595.0:31466 0.000000 2014-08-13 18:06:06.626601 7f5c973a57a0 20 read_log 194215'17111 (193305'16964) clean 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by osd.4.0:1334 2014-08-12 15:38:32.309668 2014-08-13 18:06:06.626614 7f5c973a57a0 20 read_log 194215'17112 (194215'17111) delete 6cafb89b/benchmark_data_storage0111-ib.pocstor_62621_object10864/head//24 by client.2174595.0:31468 0.000000 2014-08-13 18:06:06.626624 7f5c973a57a0 20 read_log 194215'17113 (193305'16983) clean 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by osd.4.0:1336 2014-08-12 15:38:32.316656 2014-08-13 18:06:06.626636 7f5c973a57a0 20 read_log 194215'17114 (194215'17113) delete 4d7c89b/benchmark_data_storage0111-ib.pocstor_62621_object26041/head//24 by client.2174595.0:31470 0.000000 2014-08-13 18:06:06.626647 7f5c973a57a0 20 read_log 194215'17115 (192888'16825) clean 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by osd.4.0:1338 2014-08-12 15:38:32.324276 2014-08-13 18:06:06.626659 7f5c973a57a0 20 read_log 194215'17116 (194215'17115) delete 41cac89b/benchmark_data_storage0111-ib.pocstor_2228_object5617/head//24 by client.2174595.0:31472 0.000000 2014-08-13 18:06:06.626670 7f5c973a57a0 20 read_log 194215'17117 (193305'16967) clean 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by osd.4.0:1340 2014-08-12 15:38:32.331882 2014-08-13 18:06:06.626682 7f5c973a57a0 20 read_log 194215'17118 (194215'17117) delete 4eadd89b/benchmark_data_storage0111-ib.pocstor_62621_object14377/head//24 by client.2174595.0:31474 0.000000 2014-08-13 18:06:06.626693 7f5c973a57a0 20 read_log 194215'17119 (192888'16646) clean 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by osd.4.0:1342 2014-08-12 15:38:32.340753 2014-08-13 18:06:06.626705 7f5c973a57a0 20 read_log 194215'17120 (194215'17119) delete 70eae89b/benchmark_data_storage0111-ib.pocstor_41097_object12571/head//24 by client.2174595.0:31476 0.000000 2014-08-13 18:06:06.626734 7f5c973a57a0 20 read_log 194215'17121 (192888'16624) clean 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by osd.4.0:1344 2014-08-12 15:38:32.349085 2014-08-13 18:06:06.626750 7f5c973a57a0 20 read_log 194215'17122 (194215'17121) delete 93d20c9b/benchmark_data_storage0111-ib.pocstor_41097_object3044/head//24 by client.2174595.0:31478 0.000000 2014-08-13 18:06:06.626761 7f5c973a57a0 20 read_log 194215'17123 (193305'16989) clean 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by osd.4.0:1346 2014-08-12 15:38:32.356894 2014-08-13 18:06:06.626773 7f5c973a57a0 20 read_log 194215'17124 (194215'17123) delete 87651c9b/benchmark_data_storage0111-ib.pocstor_62621_object30790/head//24 by client.2174595.0:31480 0.000000 2014-08-13 18:06:06.626786 7f5c973a57a0 20 read_log 194215'17125 (192888'16682) clean 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by osd.4.0:1348 2014-08-12 15:38:32.397317 2014-08-13 18:06:06.626805 7f5c973a57a0 20 read_log 194215'17126 (194215'17125) delete 419d1c9b/benchmark_data_storage0111-ib.pocstor_41097_object30607/head//24 by client.2174595.0:31482 0.000000 2014-08-13 18:06:06.626818 7f5c973a57a0 20 read_log 194215'17127 (193305'16972) clean 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by osd.4.0:1350 2014-08-12 15:38:32.405053 2014-08-13 18:06:06.626831 7f5c973a57a0 20 read_log 194215'17128 (194215'17127) delete 521c2c9b/benchmark_data_storage0111-ib.pocstor_62621_object17243/head//24 by client.2174595.0:31484 0.000000 2014-08-13 18:06:06.626843 7f5c973a57a0 20 read_log 194215'17129 (193305'16954) clean 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by osd.4.0:1352 2014-08-12 15:38:32.413815 2014-08-13 18:06:06.626858 7f5c973a57a0 20 read_log 194215'17130 (194215'17129) delete 40e33c9b/benchmark_data_storage0111-ib.pocstor_62621_object1778/head//24 by client.2174595.0:31486 0.000000 2014-08-13 18:06:06.626869 7f5c973a57a0 20 read_log 194215'17131 (192888'16822) clean ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by osd.4.0:1354 2014-08-12 15:38:32.422603 2014-08-13 18:06:06.626881 7f5c973a57a0 20 read_log 194215'17132 (194215'17131) delete ed874c9b/benchmark_data_storage0111-ib.pocstor_2228_object4800/head//24 by client.2174595.0:31488 0.000000 2014-08-13 18:06:06.626894 7f5c973a57a0 20 read_log 194215'17133 (192888'16813) clean 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by osd.4.0:1356 2014-08-12 15:38:32.430715 2014-08-13 18:06:06.626906 7f5c973a57a0 20 read_log 194215'17134 (194215'17133) delete 711c4c9b/benchmark_data_storage0111-ib.pocstor_2228_object1595/head//24 by client.2174595.0:31490 0.000000 2014-08-13 18:06:06.626917 7f5c973a57a0 20 read_log 194215'17135 (192888'16829) clean 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by osd.4.0:1358 2014-08-12 15:38:32.438272 2014-08-13 18:06:06.626930 7f5c973a57a0 20 read_log 194215'17136 (194215'17135) delete 915f4c9b/benchmark_data_storage0111-ib.pocstor_2228_object7355/head//24 by client.2174595.0:31492 0.000000 2014-08-13 18:06:06.626941 7f5c973a57a0 20 read_log 194215'17137 (192888'16621) clean 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by osd.4.0:1360 2014-08-12 15:38:32.463860 2014-08-13 18:06:06.626955 7f5c973a57a0 20 read_log 194215'17138 (194215'17137) delete 64e35c9b/benchmark_data_storage0111-ib.pocstor_41097_object1623/head//24 by client.2174595.0:31494 0.000000 2014-08-13 18:06:06.626966 7f5c973a57a0 20 read_log 194215'17139 (192888'16670) clean 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by osd.4.0:1362 2014-08-12 15:38:32.472343 2014-08-13 18:06:06.626978 7f5c973a57a0 20 read_log 194215'17140 (194215'17139) delete 45655c9b/benchmark_data_storage0111-ib.pocstor_41097_object25219/head//24 by client.2174595.0:31496 0.000000 2014-08-13 18:06:06.626992 7f5c973a57a0 20 read_log 194215'17141 (193305'16978) clean 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by osd.4.0:1364 2014-08-12 15:38:32.479893 2014-08-13 18:06:06.627004 7f5c973a57a0 20 read_log 194215'17142 (194215'17141) delete 2af36c9b/benchmark_data_storage0111-ib.pocstor_62621_object21823/head//24 by client.2174595.0:31498 0.000000 2014-08-13 18:06:06.627017 7f5c973a57a0 20 read_log 194215'17143 (193305'16969) clean 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by osd.4.0:1366 2014-08-12 15:38:32.488572 2014-08-13 18:06:06.627031 7f5c973a57a0 20 read_log 194215'17144 (194215'17143) delete 75b56c9b/benchmark_data_storage0111-ib.pocstor_62621_object15380/head//24 by client.2174595.0:31500 0.000000 2014-08-13 18:06:06.627042 7f5c973a57a0 20 read_log 194215'17145 (193305'16973) clean ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by osd.4.0:1368 2014-08-12 15:38:32.496764 2014-08-13 18:06:06.627078 7f5c973a57a0 20 read_log 194215'17146 (194215'17145) delete ed927c9b/benchmark_data_storage0111-ib.pocstor_62621_object17338/head//24 by client.2174595.0:31502 0.000000 2014-08-13 18:06:06.627093 7f5c973a57a0 20 read_log 194215'17147 (192888'16622) clean ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by osd.4.0:1370 2014-08-12 15:38:32.504695 2014-08-13 18:06:06.627105 7f5c973a57a0 20 read_log 194215'17148 (194215'17147) delete ec4e7c9b/benchmark_data_storage0111-ib.pocstor_41097_object2463/head//24 by client.2174595.0:31504 0.000000 2014-08-13 18:06:06.627117 7f5c973a57a0 20 read_log 194215'17149 (193305'16976) clean bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by osd.4.0:1372 2014-08-12 15:38:32.511906 2014-08-13 18:06:06.627132 7f5c973a57a0 20 read_log 194215'17150 (194215'17149) delete bcda8c9b/benchmark_data_storage0111-ib.pocstor_62621_object19273/head//24 by client.2174595.0:31506 0.000000 2014-08-13 18:06:06.627143 7f5c973a57a0 20 read_log 194215'17151 (193305'16971) clean 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by osd.4.0:1374 2014-08-12 15:38:32.520071 2014-08-13 18:06:06.627163 7f5c973a57a0 20 read_log 194215'17152 (194215'17151) delete 9b7f8c9b/benchmark_data_storage0111-ib.pocstor_62621_object17173/head//24 by client.2174595.0:31508 0.000000 2014-08-13 18:06:06.627174 7f5c973a57a0 20 read_log 194215'17153 (193305'16975) clean e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by osd.4.0:1376 2014-08-12 15:38:32.529123 2014-08-13 18:06:06.627186 7f5c973a57a0 20 read_log 194215'17154 (194215'17153) delete e5739c9b/benchmark_data_storage0111-ib.pocstor_62621_object19194/head//24 by client.2174595.0:31510 0.000000 2014-08-13 18:06:06.627197 7f5c973a57a0 20 read_log 194215'17155 (193305'16968) clean 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by osd.4.0:1378 2014-08-12 15:38:32.538705 2014-08-13 18:06:06.627211 7f5c973a57a0 20 read_log 194215'17156 (194215'17155) delete 3e959c9b/benchmark_data_storage0111-ib.pocstor_62621_object15095/head//24 by client.2174595.0:31512 0.000000 2014-08-13 18:06:06.627222 7f5c973a57a0 20 read_log 194215'17157 (192888'16653) clean 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by osd.4.0:1380 2014-08-12 15:38:32.546135 2014-08-13 18:06:06.627235 7f5c973a57a0 20 read_log 194215'17158 (194215'17157) delete 4049ac9b/benchmark_data_storage0111-ib.pocstor_41097_object16135/head//24 by client.2174595.0:31514 0.000000 2014-08-13 18:06:06.627246 7f5c973a57a0 20 read_log 194215'17159 (192888'16821) clean bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by osd.4.0:1382 2014-08-12 15:38:32.555002 2014-08-13 18:06:06.627258 7f5c973a57a0 20 read_log 194215'17160 (194215'17159) delete bb4cac9b/benchmark_data_storage0111-ib.pocstor_2228_object3953/head//24 by client.2174595.0:31516 0.000000 2014-08-13 18:06:06.627272 7f5c973a57a0 20 read_log 194215'17161 (192888'16818) clean e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by osd.4.0:1384 2014-08-12 15:38:32.563564 2014-08-13 18:06:06.627284 7f5c973a57a0 20 read_log 194215'17162 (194215'17161) delete e73fbc9b/benchmark_data_storage0111-ib.pocstor_2228_object3424/head//24 by client.2174595.0:31518 0.000000 2014-08-13 18:06:06.627295 7f5c973a57a0 20 read_log 194215'17163 (192888'16870) clean 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by osd.4.0:1386 2014-08-12 15:38:32.571644 2014-08-13 18:06:06.627308 7f5c973a57a0 20 read_log 194215'17164 (194215'17163) delete 4b7edc9b/benchmark_data_storage0111-ib.pocstor_2228_object29984/head//24 by client.2174595.0:31520 0.000000 2014-08-13 18:06:06.627322 7f5c973a57a0 20 read_log 194215'17165 (192888'16668) clean 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by osd.4.0:1388 2014-08-12 15:38:32.578978 2014-08-13 18:06:06.627349 7f5c973a57a0 20 read_log 194215'17166 (194215'17165) delete 3c00ec9b/benchmark_data_storage0111-ib.pocstor_41097_object24290/head//24 by client.2174595.0:31522 0.000000 2014-08-13 18:06:06.627367 7f5c973a57a0 20 read_log 194215'17167 (193305'16984) clean 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by osd.4.0:1390 2014-08-12 15:38:32.586331 2014-08-13 18:06:06.627380 7f5c973a57a0 20 read_log 194215'17168 (194215'17167) delete 9145ec9b/benchmark_data_storage0111-ib.pocstor_62621_object26306/head//24 by client.2174595.0:31524 0.000000 2014-08-13 18:06:06.627391 7f5c973a57a0 20 read_log 194215'17169 (192888'16660) clean e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by osd.4.0:1392 2014-08-12 15:38:32.594193 2014-08-13 18:06:06.627417 7f5c973a57a0 20 read_log 194215'17170 (194215'17169) delete e076ec9b/benchmark_data_storage0111-ib.pocstor_41097_object20767/head//24 by client.2174595.0:31526 0.000000 2014-08-13 18:06:06.627429 7f5c973a57a0 20 read_log 194215'17171 (192888'16647) clean 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by osd.4.0:1394 2014-08-12 15:38:32.601523 2014-08-13 18:06:06.627448 7f5c973a57a0 20 read_log 194215'17172 (194215'17171) delete 3ae6ec9b/benchmark_data_storage0111-ib.pocstor_41097_object12836/head//24 by client.2174595.0:31528 0.000000 2014-08-13 18:06:06.627463 7f5c973a57a0 20 read_log 194215'17173 (192888'16628) clean 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by osd.4.0:1396 2014-08-12 15:38:32.609453 2014-08-13 18:06:06.627476 7f5c973a57a0 20 read_log 194215'17174 (194215'17173) delete 4da8fc9b/benchmark_data_storage0111-ib.pocstor_41097_object4832/head//24 by client.2174595.0:31530 0.000000 2014-08-13 18:06:06.627487 7f5c973a57a0 20 read_log 194215'17175 (193305'16960) clean 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by osd.4.0:1398 2014-08-12 15:38:32.621336 2014-08-13 18:06:06.627506 7f5c973a57a0 20 read_log 194215'17176 (194215'17175) delete 4ceafc9b/benchmark_data_storage0111-ib.pocstor_62621_object7134/head//24 by client.2174595.0:31532 0.000000 2014-08-13 18:06:06.627518 7f5c973a57a0 20 read_log 194215'17177 (193305'16979) clean 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by osd.4.0:1400 2014-08-12 15:38:32.629679 2014-08-13 18:06:06.627531 7f5c973a57a0 20 read_log 194215'17178 (194215'17177) delete 42cefc9b/benchmark_data_storage0111-ib.pocstor_62621_object22508/head//24 by client.2174595.0:31534 0.000000 2014-08-13 18:06:06.627542 7f5c973a57a0 20 read_log 194215'17179 (194215'17179) modify 9b/hit_set_24.9b_archive_2014-08-12 15:38:31.718448_2014-08-12 17:24:40.634730/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.635537 2014-08-13 18:06:06.627564 7f5c973a57a0 20 read_log 194215'17180 (194215'16993) delete 9b/hit_set_24.9b_archive_2014-08-12 15:32:25.864657_2014-08-12 15:38:31.718183/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.635537 2014-08-13 18:06:06.627577 7f5c973a57a0 20 read_log 194215'17181 (0'0) promote fb5d6c9b/benchmark_data_storage0111-ib.pocstor_34628_object188/head//24 by osd.4.0:5227 2014-08-12 17:24:40.640704 2014-08-13 18:06:06.627590 7f5c973a57a0 20 read_log 194215'17182 (194215'17181) modify fb5d6c9b/benchmark_data_storage0111-ib.pocstor_34628_object188/head//24 by client.2193759.0:189 2014-08-12 17:24:40.626295 2014-08-13 18:06:06.627619 7f5c973a57a0 20 read_log 194215'17183 (0'0) promote e686789b/benchmark_data_storage0111-ib.pocstor_34628_object818/head//24 by osd.4.0:5243 2014-08-12 17:24:43.085924 2014-08-13 18:06:06.627632 7f5c973a57a0 20 read_log 194215'17184 (194215'17183) modify e686789b/benchmark_data_storage0111-ib.pocstor_34628_object818/head//24 by client.2193759.0:819 2014-08-12 17:24:43.078725 2014-08-13 18:06:06.627656 7f5c973a57a0 20 read_log 194370'17185 (0'0) promote 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by osd.365.0:3664 2014-08-12 20:30:14.749730 2014-08-13 18:06:06.627671 7f5c973a57a0 20 read_log 194370'17186 (194370'17185) modify 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by client.1508531.0:157 2014-08-12 20:30:14.740139 2014-08-13 18:06:06.627684 7f5c973a57a0 20 read_log 194370'17187 (0'0) promote 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by osd.365.0:3676 2014-08-12 20:30:18.893137 2014-08-13 18:06:06.627696 7f5c973a57a0 20 read_log 194370'17188 (194370'17187) modify 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by client.1508531.0:1199 2014-08-12 20:30:18.882929 2014-08-13 18:06:06.627717 7f5c973a57a0 20 read_log 194370'17189 (0'0) promote b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by osd.365.0:3712 2014-08-12 20:30:27.743527 2014-08-13 18:06:06.627733 7f5c973a57a0 20 read_log 194370'17190 (194370'17189) modify b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by client.1508531.0:3590 2014-08-12 20:30:27.734195 2014-08-13 18:06:06.627747 7f5c973a57a0 20 read_log 194370'17191 (0'0) promote c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by osd.365.0:3738 2014-08-12 20:30:32.110372 2014-08-13 18:06:06.627759 7f5c973a57a0 20 read_log 194370'17192 (194370'17191) modify c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by client.1508531.0:4723 2014-08-12 20:30:32.103291 2014-08-13 18:06:06.627772 7f5c973a57a0 20 read_log 194370'17193 (0'0) promote e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by osd.365.0:3762 2014-08-12 20:30:39.243150 2014-08-13 18:06:06.627789 7f5c973a57a0 20 read_log 194370'17194 (194370'17193) modify e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by client.1508531.0:6348 2014-08-12 20:30:39.236432 2014-08-13 18:06:06.627801 7f5c973a57a0 20 read_log 194370'17195 (0'0) promote 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by osd.365.0:3792 2014-08-12 20:30:49.478892 2014-08-13 18:06:06.627813 7f5c973a57a0 20 read_log 194370'17196 (194370'17195) modify 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by client.1508531.0:8355 2014-08-12 20:30:49.471690 2014-08-13 18:06:06.627826 7f5c973a57a0 20 read_log 194370'17197 (0'0) promote 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by osd.365.0:3796 2014-08-12 20:30:50.324880 2014-08-13 18:06:06.627838 7f5c973a57a0 20 read_log 194370'17198 (194370'17197) modify 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by client.1508531.0:8511 2014-08-12 20:30:50.316538 2014-08-13 18:06:06.627851 7f5c973a57a0 20 read_log 194370'17199 (0'0) promote 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by osd.365.0:3820 2014-08-12 20:30:54.681070 2014-08-13 18:06:06.627863 7f5c973a57a0 20 read_log 194370'17200 (194370'17199) modify 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by client.1508531.0:9228 2014-08-12 20:30:54.674352 2014-08-13 18:06:06.627877 7f5c973a57a0 20 read_log 194370'17201 (0'0) promote efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by osd.365.0:3838 2014-08-12 20:30:58.700578 2014-08-13 18:06:06.627889 7f5c973a57a0 20 read_log 194370'17202 (194370'17201) modify efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by client.1508531.0:9902 2014-08-12 20:30:58.691845 2014-08-13 18:06:06.627901 7f5c973a57a0 20 read_log 194370'17203 (0'0) promote 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by osd.365.0:3856 2014-08-12 20:31:03.577724 2014-08-13 18:06:06.627914 7f5c973a57a0 20 read_log 194370'17204 (194370'17203) modify 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by client.1508531.0:10711 2014-08-12 20:31:03.568643 2014-08-13 18:06:06.627936 7f5c973a57a0 20 read_log 194370'17205 (0'0) promote 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by osd.365.0:3862 2014-08-12 20:31:05.678584 2014-08-13 18:06:06.627952 7f5c973a57a0 20 read_log 194370'17206 (194370'17205) modify 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by client.1508531.0:11067 2014-08-12 20:31:05.669937 2014-08-13 18:06:06.627965 7f5c973a57a0 20 read_log 194370'17207 (0'0) promote ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by osd.365.0:3870 2014-08-12 20:31:06.442885 2014-08-13 18:06:06.627977 7f5c973a57a0 20 read_log 194370'17208 (194370'17207) modify ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by client.1508531.0:11194 2014-08-12 20:31:06.434067 2014-08-13 18:06:06.627989 7f5c973a57a0 20 read_log 194370'17209 (0'0) promote 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by osd.365.0:3884 2014-08-12 20:31:15.819769 2014-08-13 18:06:06.628001 7f5c973a57a0 20 read_log 194370'17210 (194370'17209) modify 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by client.1508531.0:12615 2014-08-12 20:31:15.811325 2014-08-13 18:06:06.628014 7f5c973a57a0 20 read_log 194370'17211 (0'0) promote ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by osd.365.0:3900 2014-08-12 20:31:21.990397 2014-08-13 18:06:06.628026 7f5c973a57a0 20 read_log 194370'17212 (194370'17211) modify ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by client.1508531.0:13272 2014-08-12 20:31:21.981379 2014-08-13 18:06:06.628038 7f5c973a57a0 20 read_log 194370'17213 (0'0) promote d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by osd.365.0:3914 2014-08-12 20:31:28.961810 2014-08-13 18:06:06.628069 7f5c973a57a0 20 read_log 194370'17214 (194370'17213) modify d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by client.1508531.0:13881 2014-08-12 20:31:28.953568 2014-08-13 18:06:06.628084 7f5c973a57a0 20 read_log 194370'17215 (0'0) promote b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by osd.365.0:3916 2014-08-12 20:31:32.051201 2014-08-13 18:06:06.628097 7f5c973a57a0 20 read_log 194370'17216 (194370'17215) modify b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by client.1508531.0:13994 2014-08-12 20:31:32.042820 2014-08-13 18:06:06.628111 7f5c973a57a0 20 read_log 194377'17217 (0'0) promote d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by osd.291.0:3899 2014-08-12 20:33:00.827904 2014-08-13 18:06:06.628126 7f5c973a57a0 20 read_log 194377'17218 (194377'17217) modify d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by client.2231190.0:872 2014-08-12 20:33:00.818408 2014-08-13 18:06:06.628166 7f5c973a57a0 20 read_log 194377'17219 (0'0) promote b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by osd.291.0:3905 2014-08-12 20:33:02.345171 2014-08-13 18:06:06.628183 7f5c973a57a0 20 read_log 194377'17220 (194377'17219) modify b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by client.2231190.0:1273 2014-08-12 20:33:02.338556 2014-08-13 18:06:06.628196 7f5c973a57a0 20 read_log 194377'17221 (0'0) promote 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by osd.291.0:3907 2014-08-12 20:33:02.642836 2014-08-13 18:06:06.628210 7f5c973a57a0 20 read_log 194377'17222 (194377'17221) modify 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by client.2231190.0:1353 2014-08-12 20:33:02.636079 2014-08-13 18:06:06.628223 7f5c973a57a0 20 read_log 194377'17223 (0'0) promote 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by osd.291.0:3911 2014-08-12 20:33:03.151302 2014-08-13 18:06:06.628243 7f5c973a57a0 20 read_log 194377'17224 (194377'17223) modify 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by client.2231190.0:1487 2014-08-12 20:33:03.144670 2014-08-13 18:06:06.628261 7f5c973a57a0 20 read_log 194377'17225 (0'0) promote b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by osd.291.0:3917 2014-08-12 20:33:05.227326 2014-08-13 18:06:06.628273 7f5c973a57a0 20 read_log 194377'17226 (194377'17225) modify b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by client.2231190.0:1996 2014-08-12 20:33:05.220164 2014-08-13 18:06:06.628286 7f5c973a57a0 20 read_log 194377'17227 (0'0) promote 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by osd.291.0:3931 2014-08-12 20:33:07.879263 2014-08-13 18:06:06.628298 7f5c973a57a0 20 read_log 194377'17228 (194377'17227) modify 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by client.2231190.0:2636 2014-08-12 20:33:07.870982 2014-08-13 18:06:06.628316 7f5c973a57a0 20 read_log 194377'17229 (0'0) promote 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by osd.291.0:3937 2014-08-12 20:33:09.638339 2014-08-13 18:06:06.628328 7f5c973a57a0 20 read_log 194377'17230 (194377'17229) modify 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by client.2231190.0:3080 2014-08-12 20:33:09.629893 2014-08-13 18:06:06.628341 7f5c973a57a0 20 read_log 194377'17231 (0'0) promote 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by osd.291.0:3959 2014-08-12 20:33:16.259252 2014-08-13 18:06:06.628357 7f5c973a57a0 20 read_log 194377'17232 (194377'17231) modify 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by client.2231190.0:4379 2014-08-12 20:33:16.251492 2014-08-13 18:06:06.628372 7f5c973a57a0 20 read_log 194377'17233 (0'0) promote ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by osd.291.0:3965 2014-08-12 20:33:16.689810 2014-08-13 18:06:06.628397 7f5c973a57a0 20 read_log 194377'17234 (194377'17233) modify ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by client.2231190.0:4436 2014-08-12 20:33:16.682348 2014-08-13 18:06:06.628416 7f5c973a57a0 20 read_log 194377'17235 (0'0) promote 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by osd.291.0:3973 2014-08-12 20:33:19.532772 2014-08-13 18:06:06.628435 7f5c973a57a0 20 read_log 194377'17236 (194377'17235) modify 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by client.2231190.0:4807 2014-08-12 20:33:19.522292 2014-08-13 18:06:06.628455 7f5c973a57a0 20 read_log 194378'17237 (0'0) promote 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by osd.291.0:3977 2014-08-12 20:33:20.480003 2014-08-13 18:06:06.628467 7f5c973a57a0 20 read_log 194378'17238 (194378'17237) modify 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by client.2231190.0:4866 2014-08-12 20:33:20.470704 2014-08-13 18:06:06.628488 7f5c973a57a0 20 read_log 194380'17239 (0'0) promote f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by osd.291.0:3987 2014-08-12 20:33:27.746925 2014-08-13 18:06:06.628504 7f5c973a57a0 20 read_log 194380'17240 (194380'17239) modify f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by client.2231190.0:5343 2014-08-12 20:33:27.738227 2014-08-13 18:06:06.628516 7f5c973a57a0 20 read_log 194383'17241 (0'0) promote ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by osd.291.0:4007 2014-08-12 20:34:03.135985 2014-08-13 18:06:06.628529 7f5c973a57a0 20 read_log 194383'17242 (194383'17241) modify ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by client.2231190.0:6319 2014-08-12 20:34:03.127803 2014-08-13 18:06:06.628557 7f5c973a57a0 20 read_log 194383'17243 (0'0) promote 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by osd.291.0:4025 2014-08-12 20:34:24.543516 2014-08-13 18:06:06.628573 7f5c973a57a0 20 read_log 194383'17244 (194383'17243) modify 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by client.2231190.0:7346 2014-08-12 20:34:24.535485 2014-08-13 18:06:06.628592 7f5c973a57a0 20 read_log 194416'17245 (194416'17245) modify 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 18:06:06.628607 7f5c973a57a0 20 read_log 194416'17246 (194215'17179) delete 9b/hit_set_24.9b_archive_2014-08-12 15:38:31.718448_2014-08-12 17:24:40.634730/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 18:06:06.628620 7f5c973a57a0 20 read_log 194416'17247 (0'0) promote 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by osd.365.0:4305 2014-08-13 15:28:33.802418 2014-08-13 18:06:06.628640 7f5c973a57a0 20 read_log 194416'17248 (194416'17247) modify 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by client.2441958.0:826 2014-08-13 15:28:33.793568 2014-08-13 18:06:06.628662 7f5c973a57a0 20 read_log 194416'17249 (0'0) promote 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by osd.365.0:4355 2014-08-13 15:28:41.298245 2014-08-13 18:06:06.628678 7f5c973a57a0 20 read_log 194416'17250 (194416'17249) modify 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by client.2441958.0:2791 2014-08-13 15:28:41.288527 2014-08-13 18:06:06.628693 7f5c973a57a0 20 read_log 194416'17251 (0'0) promote a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by osd.365.0:4367 2014-08-13 15:28:43.919462 2014-08-13 18:06:06.628705 7f5c973a57a0 20 read_log 194416'17252 (194416'17251) modify a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by client.2441958.0:3487 2014-08-13 15:28:43.908733 2014-08-13 18:06:06.628717 7f5c973a57a0 20 read_log 194416'17253 (0'0) promote 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by osd.365.0:4407 2014-08-13 15:28:53.741369 2014-08-13 18:06:06.628730 7f5c973a57a0 20 read_log 194416'17254 (194416'17253) modify 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by client.2441958.0:6000 2014-08-13 15:28:53.730275 2014-08-13 18:06:06.628742 7f5c973a57a0 20 read_log 194416'17255 (0'0) promote 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by osd.365.0:4423 2014-08-13 15:28:57.206841 2014-08-13 18:06:06.628754 7f5c973a57a0 20 read_log 194416'17256 (194416'17255) modify 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by client.2441958.0:6845 2014-08-13 15:28:57.199118 2014-08-13 18:06:06.628770 7f5c973a57a0 20 read_log 194416'17257 (0'0) promote b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by osd.365.0:4435 2014-08-13 15:29:00.948687 2014-08-13 18:06:06.628782 7f5c973a57a0 20 read_log 194416'17258 (194416'17257) modify b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by client.2441958.0:7845 2014-08-13 15:29:00.939980 2014-08-13 18:06:06.628794 7f5c973a57a0 20 read_log 194416'17259 (0'0) promote 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by osd.365.0:4439 2014-08-13 15:29:01.158329 2014-08-13 18:06:06.628806 7f5c973a57a0 20 read_log 194416'17260 (194416'17259) modify 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by client.2441958.0:7901 2014-08-13 15:29:01.151151 2014-08-13 18:06:06.628819 7f5c973a57a0 20 read_log 194416'17261 (0'0) promote 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by osd.365.0:4445 2014-08-13 15:29:02.707665 2014-08-13 18:06:06.628841 7f5c973a57a0 20 read_log 194416'17262 (194416'17261) modify 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by client.2441958.0:8310 2014-08-13 15:29:02.701038 2014-08-13 18:06:06.628857 7f5c973a57a0 20 read_log 194416'17263 (0'0) promote a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by osd.365.0:4469 2014-08-13 15:29:06.850953 2014-08-13 18:06:06.628870 7f5c973a57a0 20 read_log 194416'17264 (194416'17263) modify a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by client.2441958.0:9362 2014-08-13 15:29:06.842730 2014-08-13 18:06:06.628882 7f5c973a57a0 20 read_log 194416'17265 (0'0) promote 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by osd.365.0:4475 2014-08-13 15:29:08.380752 2014-08-13 18:06:06.628903 7f5c973a57a0 20 read_log 194416'17266 (194416'17265) modify 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by client.2441958.0:9760 2014-08-13 15:29:08.372051 2014-08-13 18:06:06.628919 7f5c973a57a0 20 read_log 194416'17267 (0'0) promote c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by osd.365.0:4489 2014-08-13 15:29:10.510787 2014-08-13 18:06:06.628932 7f5c973a57a0 20 read_log 194416'17268 (194416'17267) modify c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by client.2441958.0:10262 2014-08-13 15:29:10.502047 2014-08-13 18:06:06.628945 7f5c973a57a0 20 read_log 194416'17269 (0'0) promote f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by osd.365.0:4515 2014-08-13 15:29:16.039284 2014-08-13 18:06:06.628957 7f5c973a57a0 20 read_log 194416'17270 (194416'17269) modify f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by client.2441958.0:11634 2014-08-13 15:29:16.031715 2014-08-13 18:06:06.628969 7f5c973a57a0 20 read_log 194416'17271 (0'0) promote a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by osd.365.0:4523 2014-08-13 15:29:18.417979 2014-08-13 18:06:06.628984 7f5c973a57a0 20 read_log 194416'17272 (194416'17271) modify a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by client.2441958.0:12184 2014-08-13 15:29:18.409753 2014-08-13 18:06:06.628998 7f5c973a57a0 20 read_log 194416'17273 (0'0) promote 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by osd.365.0:4531 2014-08-13 15:29:19.203121 2014-08-13 18:06:06.629011 7f5c973a57a0 20 read_log 194416'17274 (194416'17273) modify 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by client.2441958.0:12371 2014-08-13 15:29:19.195539 2014-08-13 18:06:06.629023 7f5c973a57a0 20 read_log 194416'17275 (0'0) promote d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by osd.365.0:4539 2014-08-13 15:29:20.920212 2014-08-13 18:06:06.629039 7f5c973a57a0 20 read_log 194416'17276 (194416'17275) modify d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by client.2441958.0:12775 2014-08-13 15:29:20.910681 2014-08-13 18:06:06.629052 7f5c973a57a0 20 read_log 194416'17277 (0'0) promote 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by osd.365.0:4541 2014-08-13 15:29:21.060874 2014-08-13 18:06:06.629064 7f5c973a57a0 20 read_log 194416'17278 (194416'17277) modify 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by client.2441958.0:12802 2014-08-13 15:29:21.053764 2014-08-13 18:06:06.629079 7f5c973a57a0 20 read_log 194416'17279 (0'0) promote f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by osd.365.0:4547 2014-08-13 15:29:22.800945 2014-08-13 18:06:06.629091 7f5c973a57a0 20 read_log 194416'17280 (194416'17279) modify f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by client.2441958.0:13196 2014-08-13 15:29:22.791824 2014-08-13 18:06:06.629115 7f5c973a57a0 20 read_log 194416'17281 (0'0) promote fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by osd.365.0:4551 2014-08-13 15:29:23.506297 2014-08-13 18:06:06.629139 7f5c973a57a0 20 read_log 194416'17282 (194416'17281) modify fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by client.2441958.0:13357 2014-08-13 15:29:23.498808 2014-08-13 18:06:06.629173 7f5c973a57a0 20 read_log 194416'17283 (0'0) promote 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by osd.365.0:4625 2014-08-13 15:29:58.203831 2014-08-13 18:06:06.629186 7f5c973a57a0 20 read_log 194416'17284 (194416'17283) modify 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by client.2441958.0:17771 2014-08-13 15:29:58.194289 2014-08-13 18:06:06.629199 7f5c973a57a0 20 read_log 194422'17285 (0'0) promote 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by osd.365.0:4651 2014-08-13 15:32:15.458090 2014-08-13 18:06:06.629222 7f5c973a57a0 20 read_log 194422'17286 (194422'17285) modify 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by client.1148203.0:121 2014-08-13 15:32:15.450657 2014-08-13 18:06:06.629235 7f5c973a57a0 20 read_log 194422'17287 (0'0) promote 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by osd.365.0:4659 2014-08-13 15:32:18.970767 2014-08-13 18:06:06.629248 7f5c973a57a0 20 read_log 194422'17288 (194422'17287) modify 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by client.1148203.0:913 2014-08-13 15:32:18.960547 2014-08-13 18:06:06.629260 7f5c973a57a0 20 read_log 194422'17289 (0'0) promote 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by osd.365.0:4663 2014-08-13 15:32:19.444995 2014-08-13 18:06:06.629273 7f5c973a57a0 20 read_log 194422'17290 (194422'17289) modify 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by client.1148203.0:1039 2014-08-13 15:32:19.437462 2014-08-13 18:06:06.629285 7f5c973a57a0 20 read_log 194422'17291 (0'0) promote 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by osd.365.0:4673 2014-08-13 15:32:20.973955 2014-08-13 18:06:06.629303 7f5c973a57a0 20 read_log 194422'17292 (194422'17291) modify 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by client.1148203.0:1427 2014-08-13 15:32:20.965536 2014-08-13 18:06:06.629316 7f5c973a57a0 20 read_log 194422'17293 (0'0) promote f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by osd.365.0:4707 2014-08-13 15:32:29.249344 2014-08-13 18:06:06.629328 7f5c973a57a0 20 read_log 194422'17294 (194422'17293) modify f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by client.1148203.0:3540 2014-08-13 15:32:29.240088 2014-08-13 18:06:06.629340 7f5c973a57a0 20 read_log 194422'17295 (0'0) promote 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by osd.365.0:4737 2014-08-13 15:32:35.191923 2014-08-13 18:06:06.629353 7f5c973a57a0 20 read_log 194422'17296 (194422'17295) modify 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by client.1148203.0:5077 2014-08-13 15:32:35.182165 2014-08-13 18:06:06.629365 7f5c973a57a0 20 read_log 194422'17297 (0'0) promote 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by osd.365.0:4743 2014-08-13 15:32:37.108092 2014-08-13 18:06:06.629377 7f5c973a57a0 20 read_log 194422'17298 (194422'17297) modify 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by client.1148203.0:5453 2014-08-13 15:32:37.097117 2014-08-13 18:06:06.629390 7f5c973a57a0 20 read_log 194422'17299 (0'0) promote 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by osd.365.0:4835 2014-08-13 15:33:05.261973 2014-08-13 18:06:06.629402 7f5c973a57a0 20 read_log 194422'17300 (194422'17299) modify 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by client.1148203.0:9580 2014-08-13 15:33:05.253527 2014-08-13 18:06:06.629414 7f5c973a57a0 20 read_log 194422'17301 (0'0) promote a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by osd.365.0:4839 2014-08-13 15:33:09.485388 2014-08-13 18:06:06.629463 7f5c973a57a0 20 read_log 194422'17302 (194422'17301) modify a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by client.1148203.0:9917 2014-08-13 15:33:09.476174 2014-08-13 18:06:06.629485 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.629865 7f5c973a57a0 10 read_log checking for missing items over interval (194370'17185,194422'17302] 2014-08-13 18:06:06.629918 7f5c973a57a0 15 read_log missing 194422'17302 (194422'17301) modify a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 by client.1148203.0:9917 2014-08-13 15:33:09.476174 2014-08-13 18:06:06.629966 7f5c973a57a0 15 read_log missing 194422'17300 (194422'17299) modify 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 by client.1148203.0:9580 2014-08-13 15:33:05.253527 2014-08-13 18:06:06.630003 7f5c973a57a0 15 read_log missing 194422'17298 (194422'17297) modify 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 by client.1148203.0:5453 2014-08-13 15:32:37.097117 2014-08-13 18:06:06.630039 7f5c973a57a0 15 read_log missing 194422'17296 (194422'17295) modify 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 by client.1148203.0:5077 2014-08-13 15:32:35.182165 2014-08-13 18:06:06.630074 7f5c973a57a0 15 read_log missing 194422'17294 (194422'17293) modify f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 by client.1148203.0:3540 2014-08-13 15:32:29.240088 2014-08-13 18:06:06.630110 7f5c973a57a0 15 read_log missing 194422'17292 (194422'17291) modify 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 by client.1148203.0:1427 2014-08-13 15:32:20.965536 2014-08-13 18:06:06.630150 7f5c973a57a0 15 read_log missing 194422'17290 (194422'17289) modify 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 by client.1148203.0:1039 2014-08-13 15:32:19.437462 2014-08-13 18:06:06.630188 7f5c973a57a0 15 read_log missing 194422'17288 (194422'17287) modify 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 by client.1148203.0:913 2014-08-13 15:32:18.960547 2014-08-13 18:06:06.630224 7f5c973a57a0 15 read_log missing 194422'17286 (194422'17285) modify 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 by client.1148203.0:121 2014-08-13 15:32:15.450657 2014-08-13 18:06:06.630263 7f5c973a57a0 15 read_log missing 194416'17284 (194416'17283) modify 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 by client.2441958.0:17771 2014-08-13 15:29:58.194289 2014-08-13 18:06:06.630304 7f5c973a57a0 15 read_log missing 194416'17282 (194416'17281) modify fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 by client.2441958.0:13357 2014-08-13 15:29:23.498808 2014-08-13 18:06:06.630341 7f5c973a57a0 15 read_log missing 194416'17280 (194416'17279) modify f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 by client.2441958.0:13196 2014-08-13 15:29:22.791824 2014-08-13 18:06:06.630388 7f5c973a57a0 15 read_log missing 194416'17278 (194416'17277) modify 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 by client.2441958.0:12802 2014-08-13 15:29:21.053764 2014-08-13 18:06:06.630435 7f5c973a57a0 15 read_log missing 194416'17276 (194416'17275) modify d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 by client.2441958.0:12775 2014-08-13 15:29:20.910681 2014-08-13 18:06:06.630473 7f5c973a57a0 15 read_log missing 194416'17274 (194416'17273) modify 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 by client.2441958.0:12371 2014-08-13 15:29:19.195539 2014-08-13 18:06:06.630509 7f5c973a57a0 15 read_log missing 194416'17272 (194416'17271) modify a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 by client.2441958.0:12184 2014-08-13 15:29:18.409753 2014-08-13 18:06:06.630545 7f5c973a57a0 15 read_log missing 194416'17270 (194416'17269) modify f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 by client.2441958.0:11634 2014-08-13 15:29:16.031715 2014-08-13 18:06:06.630581 7f5c973a57a0 15 read_log missing 194416'17268 (194416'17267) modify c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 by client.2441958.0:10262 2014-08-13 15:29:10.502047 2014-08-13 18:06:06.630616 7f5c973a57a0 15 read_log missing 194416'17266 (194416'17265) modify 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 by client.2441958.0:9760 2014-08-13 15:29:08.372051 2014-08-13 18:06:06.630652 7f5c973a57a0 15 read_log missing 194416'17264 (194416'17263) modify a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 by client.2441958.0:9362 2014-08-13 15:29:06.842730 2014-08-13 18:06:06.630687 7f5c973a57a0 15 read_log missing 194416'17262 (194416'17261) modify 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 by client.2441958.0:8310 2014-08-13 15:29:02.701038 2014-08-13 18:06:06.630723 7f5c973a57a0 15 read_log missing 194416'17260 (194416'17259) modify 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 by client.2441958.0:7901 2014-08-13 15:29:01.151151 2014-08-13 18:06:06.630758 7f5c973a57a0 15 read_log missing 194416'17258 (194416'17257) modify b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 by client.2441958.0:7845 2014-08-13 15:29:00.939980 2014-08-13 18:06:06.630794 7f5c973a57a0 15 read_log missing 194416'17256 (194416'17255) modify 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 by client.2441958.0:6845 2014-08-13 15:28:57.199118 2014-08-13 18:06:06.630829 7f5c973a57a0 15 read_log missing 194416'17254 (194416'17253) modify 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 by client.2441958.0:6000 2014-08-13 15:28:53.730275 2014-08-13 18:06:06.630865 7f5c973a57a0 15 read_log missing 194416'17252 (194416'17251) modify a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 by client.2441958.0:3487 2014-08-13 15:28:43.908733 2014-08-13 18:06:06.630901 7f5c973a57a0 15 read_log missing 194416'17250 (194416'17249) modify 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 by client.2441958.0:2791 2014-08-13 15:28:41.288527 2014-08-13 18:06:06.630936 7f5c973a57a0 15 read_log missing 194416'17248 (194416'17247) modify 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 by client.2441958.0:826 2014-08-13 15:28:33.793568 2014-08-13 18:06:06.630975 7f5c973a57a0 15 read_log missing 194416'17245 (194416'17245) modify 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.799272 2014-08-13 18:06:06.631010 7f5c973a57a0 15 read_log missing 194383'17244 (194383'17243) modify 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 by client.2231190.0:7346 2014-08-12 20:34:24.535485 2014-08-13 18:06:06.631046 7f5c973a57a0 15 read_log missing 194383'17242 (194383'17241) modify ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 by client.2231190.0:6319 2014-08-12 20:34:03.127803 2014-08-13 18:06:06.631081 7f5c973a57a0 15 read_log missing 194380'17240 (194380'17239) modify f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 by client.2231190.0:5343 2014-08-12 20:33:27.738227 2014-08-13 18:06:06.631117 7f5c973a57a0 15 read_log missing 194378'17238 (194378'17237) modify 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 by client.2231190.0:4866 2014-08-12 20:33:20.470704 2014-08-13 18:06:06.631158 7f5c973a57a0 15 read_log missing 194377'17236 (194377'17235) modify 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 by client.2231190.0:4807 2014-08-12 20:33:19.522292 2014-08-13 18:06:06.631195 7f5c973a57a0 15 read_log missing 194377'17234 (194377'17233) modify ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 by client.2231190.0:4436 2014-08-12 20:33:16.682348 2014-08-13 18:06:06.631230 7f5c973a57a0 15 read_log missing 194377'17232 (194377'17231) modify 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 by client.2231190.0:4379 2014-08-12 20:33:16.251492 2014-08-13 18:06:06.631272 7f5c973a57a0 15 read_log missing 194377'17230 (194377'17229) modify 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 by client.2231190.0:3080 2014-08-12 20:33:09.629893 2014-08-13 18:06:06.631314 7f5c973a57a0 15 read_log missing 194377'17228 (194377'17227) modify 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 by client.2231190.0:2636 2014-08-12 20:33:07.870982 2014-08-13 18:06:06.631360 7f5c973a57a0 15 read_log missing 194377'17226 (194377'17225) modify b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 by client.2231190.0:1996 2014-08-12 20:33:05.220164 2014-08-13 18:06:06.631407 7f5c973a57a0 15 read_log missing 194377'17224 (194377'17223) modify 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 by client.2231190.0:1487 2014-08-12 20:33:03.144670 2014-08-13 18:06:06.631447 7f5c973a57a0 15 read_log missing 194377'17222 (194377'17221) modify 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 by client.2231190.0:1353 2014-08-12 20:33:02.636079 2014-08-13 18:06:06.631483 7f5c973a57a0 15 read_log missing 194377'17220 (194377'17219) modify b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 by client.2231190.0:1273 2014-08-12 20:33:02.338556 2014-08-13 18:06:06.631519 7f5c973a57a0 15 read_log missing 194377'17218 (194377'17217) modify d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 by client.2231190.0:872 2014-08-12 20:33:00.818408 2014-08-13 18:06:06.631554 7f5c973a57a0 15 read_log missing 194370'17216 (194370'17215) modify b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 by client.1508531.0:13994 2014-08-12 20:31:32.042820 2014-08-13 18:06:06.631590 7f5c973a57a0 15 read_log missing 194370'17214 (194370'17213) modify d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 by client.1508531.0:13881 2014-08-12 20:31:28.953568 2014-08-13 18:06:06.631626 7f5c973a57a0 15 read_log missing 194370'17212 (194370'17211) modify ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 by client.1508531.0:13272 2014-08-12 20:31:21.981379 2014-08-13 18:06:06.631662 7f5c973a57a0 15 read_log missing 194370'17210 (194370'17209) modify 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 by client.1508531.0:12615 2014-08-12 20:31:15.811325 2014-08-13 18:06:06.631697 7f5c973a57a0 15 read_log missing 194370'17208 (194370'17207) modify ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 by client.1508531.0:11194 2014-08-12 20:31:06.434067 2014-08-13 18:06:06.631733 7f5c973a57a0 15 read_log missing 194370'17206 (194370'17205) modify 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 by client.1508531.0:11067 2014-08-12 20:31:05.669937 2014-08-13 18:06:06.631769 7f5c973a57a0 15 read_log missing 194370'17204 (194370'17203) modify 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 by client.1508531.0:10711 2014-08-12 20:31:03.568643 2014-08-13 18:06:06.631804 7f5c973a57a0 15 read_log missing 194370'17202 (194370'17201) modify efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 by client.1508531.0:9902 2014-08-12 20:30:58.691845 2014-08-13 18:06:06.631840 7f5c973a57a0 15 read_log missing 194370'17200 (194370'17199) modify 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 by client.1508531.0:9228 2014-08-12 20:30:54.674352 2014-08-13 18:06:06.631876 7f5c973a57a0 15 read_log missing 194370'17198 (194370'17197) modify 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 by client.1508531.0:8511 2014-08-12 20:30:50.316538 2014-08-13 18:06:06.631911 7f5c973a57a0 15 read_log missing 194370'17196 (194370'17195) modify 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 by client.1508531.0:8355 2014-08-12 20:30:49.471690 2014-08-13 18:06:06.631946 7f5c973a57a0 15 read_log missing 194370'17194 (194370'17193) modify e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 by client.1508531.0:6348 2014-08-12 20:30:39.236432 2014-08-13 18:06:06.631982 7f5c973a57a0 15 read_log missing 194370'17192 (194370'17191) modify c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 by client.1508531.0:4723 2014-08-12 20:30:32.103291 2014-08-13 18:06:06.632017 7f5c973a57a0 15 read_log missing 194370'17190 (194370'17189) modify b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 by client.1508531.0:3590 2014-08-12 20:30:27.734195 2014-08-13 18:06:06.632053 7f5c973a57a0 15 read_log missing 194370'17188 (194370'17187) modify 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 by client.1508531.0:1199 2014-08-12 20:30:18.882929 2014-08-13 18:06:06.632089 7f5c973a57a0 15 read_log missing 194370'17186 (194370'17185) modify 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 by client.1508531.0:157 2014-08-12 20:30:14.740139 2014-08-13 18:06:06.632100 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.632127 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_loaded 2014-08-13 18:06:06.632150 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.014058 0 0.000000 2014-08-13 18:06:06.632164 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 18:06:06.632174 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=0 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194487 2014-08-13 18:06:06.632205 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 18:06:06.632221 7f5c973a57a0 10 osd.4 194487 pgid 24.b6 coll 24.b6_head 2014-08-13 18:06:06.632374 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.b6 2014-08-13 18:06:06.632397 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.632434 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.b6(unlocked)] enter Initial 2014-08-13 18:06:06.632455 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.b6(unlocked)] enter NotTrimming 2014-08-13 18:06:06.632663 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.632766 7f5c973a57a0 20 read_log 192834'13334 (192834'12844) clean 55a900b6/benchmark_data_storage0111-ib.pocstor_26659_object933/head//24 by osd.21.0:14814 2014-08-11 15:26:42.582577 2014-08-13 18:06:06.632788 7f5c973a57a0 20 read_log 192834'13335 (192834'12889) clean a64550b6/benchmark_data_storage0111-ib.pocstor_26659_object2499/head//24 by osd.21.0:14815 2014-08-11 15:26:42.593255 2014-08-13 18:06:06.632801 7f5c973a57a0 20 read_log 192834'13336 (192834'12838) clean 4cf370b6/benchmark_data_storage0111-ib.pocstor_26659_object379/head//24 by osd.21.0:14816 2014-08-11 15:26:42.593528 2014-08-13 18:06:06.632814 7f5c973a57a0 20 read_log 192834'13337 (192834'12860) clean ae2280b6/benchmark_data_storage0111-ib.pocstor_26659_object1456/head//24 by osd.21.0:14817 2014-08-11 15:26:42.598633 2014-08-13 18:06:06.632826 7f5c973a57a0 20 read_log 192834'13339 (192834'12840) clean a24c74b6/benchmark_data_storage0111-ib.pocstor_26659_object588/head//24 by osd.21.0:14819 2014-08-11 15:26:42.616367 2014-08-13 18:06:06.632839 7f5c973a57a0 20 read_log 192834'13347 (192834'12881) clean c77ddcb6/benchmark_data_storage0111-ib.pocstor_26659_object1932/head//24 by osd.21.0:14827 2014-08-11 15:26:42.673764 2014-08-13 18:06:06.632852 7f5c973a57a0 20 read_log 192834'13368 (192834'13334) delete 55a900b6/benchmark_data_storage0111-ib.pocstor_26659_object933/head//24 by osd.21.0:14848 2014-08-11 15:26:42.784469 2014-08-13 18:06:06.632864 7f5c973a57a0 20 read_log 192834'13369 (192834'13335) delete a64550b6/benchmark_data_storage0111-ib.pocstor_26659_object2499/head//24 by osd.21.0:14849 2014-08-11 15:26:42.789640 2014-08-13 18:06:06.632877 7f5c973a57a0 20 read_log 192834'13370 (192834'13336) delete 4cf370b6/benchmark_data_storage0111-ib.pocstor_26659_object379/head//24 by osd.21.0:14850 2014-08-11 15:26:42.790095 2014-08-13 18:06:06.632890 7f5c973a57a0 20 read_log 192834'13371 (192834'13337) delete ae2280b6/benchmark_data_storage0111-ib.pocstor_26659_object1456/head//24 by osd.21.0:14851 2014-08-11 15:26:42.790567 2014-08-13 18:06:06.632902 7f5c973a57a0 20 read_log 192834'13373 (192834'13339) delete a24c74b6/benchmark_data_storage0111-ib.pocstor_26659_object588/head//24 by osd.21.0:14853 2014-08-11 15:26:42.810180 2014-08-13 18:06:06.632915 7f5c973a57a0 20 read_log 192834'13380 (192834'13347) delete c77ddcb6/benchmark_data_storage0111-ib.pocstor_26659_object1932/head//24 by osd.21.0:14860 2014-08-11 15:26:42.848554 2014-08-13 18:06:06.632927 7f5c973a57a0 20 read_log 192834'13404 (192834'12986) clean 769644b6/benchmark_data_storage0111-ib.pocstor_26659_object7061/head//24 by osd.21.0:14884 2014-08-11 15:26:48.218285 2014-08-13 18:06:06.632940 7f5c973a57a0 20 read_log 192834'13405 (192834'12941) clean 82cd4b6/benchmark_data_storage0111-ib.pocstor_26659_object5672/head//24 by osd.21.0:14885 2014-08-11 15:26:48.223187 2014-08-13 18:06:06.632952 7f5c973a57a0 20 read_log 192834'13412 (192834'12937) clean 249bd8b6/benchmark_data_storage0111-ib.pocstor_26659_object5569/head//24 by osd.21.0:14892 2014-08-11 15:26:48.260309 2014-08-13 18:06:06.632965 7f5c973a57a0 20 read_log 192834'13420 (192834'12984) clean f4feecb6/benchmark_data_storage0111-ib.pocstor_26659_object7018/head//24 by osd.21.0:14900 2014-08-11 15:26:48.297357 2014-08-13 18:06:06.632977 7f5c973a57a0 20 read_log 192834'13435 (192834'13404) delete 769644b6/benchmark_data_storage0111-ib.pocstor_26659_object7061/head//24 by osd.21.0:14915 2014-08-11 15:26:48.409480 2014-08-13 18:06:06.632989 7f5c973a57a0 20 read_log 192834'13436 (192834'13405) delete 82cd4b6/benchmark_data_storage0111-ib.pocstor_26659_object5672/head//24 by osd.21.0:14916 2014-08-11 15:26:48.410134 2014-08-13 18:06:06.633014 7f5c973a57a0 20 read_log 192834'13443 (192834'13412) delete 249bd8b6/benchmark_data_storage0111-ib.pocstor_26659_object5569/head//24 by osd.21.0:14923 2014-08-11 15:26:48.436869 2014-08-13 18:06:06.633030 7f5c973a57a0 20 read_log 192834'13450 (192834'13420) delete f4feecb6/benchmark_data_storage0111-ib.pocstor_26659_object7018/head//24 by osd.21.0:14930 2014-08-11 15:26:48.459530 2014-08-13 18:06:06.633043 7f5c973a57a0 20 read_log 192834'13461 (0'0) promote d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:14941 2014-08-11 15:26:49.871391 2014-08-13 18:06:06.633056 7f5c973a57a0 20 read_log 192834'13462 (192834'13461) modify d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by client.1883634.0:9192 2014-08-11 15:26:49.862663 2014-08-13 18:06:06.633068 7f5c973a57a0 20 read_log 192834'13467 (0'0) promote a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:14947 2014-08-11 15:26:51.012300 2014-08-13 18:06:06.633081 7f5c973a57a0 20 read_log 192834'13468 (192834'13467) modify a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by client.1883634.0:9481 2014-08-11 15:26:51.004045 2014-08-13 18:06:06.633093 7f5c973a57a0 20 read_log 192834'13492 (192834'13006) clean 47c04b6/benchmark_data_storage0111-ib.pocstor_26659_object8617/head//24 by osd.21.0:14972 2014-08-11 15:26:53.902775 2014-08-13 18:06:06.633106 7f5c973a57a0 20 read_log 192834'13493 (192834'13010) clean 2d1744b6/benchmark_data_storage0111-ib.pocstor_26659_object9010/head//24 by osd.21.0:14973 2014-08-11 15:26:53.903078 2014-08-13 18:06:06.633118 7f5c973a57a0 20 read_log 192834'13494 (192834'13050) clean 499e48b6/benchmark_data_storage0111-ib.pocstor_26659_object10391/head//24 by osd.21.0:14974 2014-08-11 15:26:53.911522 2014-08-13 18:06:06.633131 7f5c973a57a0 20 read_log 192834'13498 (192834'13012) clean ebd77cb6/benchmark_data_storage0111-ib.pocstor_26659_object9086/head//24 by osd.21.0:14978 2014-08-11 15:26:53.927614 2014-08-13 18:06:06.633143 7f5c973a57a0 20 read_log 192834'13509 (192834'13062) clean 99bdfcb6/benchmark_data_storage0111-ib.pocstor_26659_object11367/head//24 by osd.21.0:14989 2014-08-11 15:26:53.988650 2014-08-13 18:06:06.633168 7f5c973a57a0 20 read_log 192834'13519 (192834'13492) delete 47c04b6/benchmark_data_storage0111-ib.pocstor_26659_object8617/head//24 by osd.21.0:14999 2014-08-11 15:26:54.024071 2014-08-13 18:06:06.633181 7f5c973a57a0 20 read_log 192834'13520 (192834'13493) delete 2d1744b6/benchmark_data_storage0111-ib.pocstor_26659_object9010/head//24 by osd.21.0:15000 2014-08-11 15:26:54.027709 2014-08-13 18:06:06.633196 7f5c973a57a0 20 read_log 192834'13521 (192834'13494) delete 499e48b6/benchmark_data_storage0111-ib.pocstor_26659_object10391/head//24 by osd.21.0:15001 2014-08-11 15:26:54.036463 2014-08-13 18:06:06.633209 7f5c973a57a0 20 read_log 192834'13525 (192834'13498) delete ebd77cb6/benchmark_data_storage0111-ib.pocstor_26659_object9086/head//24 by osd.21.0:15005 2014-08-11 15:26:54.048380 2014-08-13 18:06:06.633221 7f5c973a57a0 20 read_log 192834'13526 (192834'13509) delete 99bdfcb6/benchmark_data_storage0111-ib.pocstor_26659_object11367/head//24 by osd.21.0:15006 2014-08-11 15:26:54.051748 2014-08-13 18:06:06.633238 7f5c973a57a0 20 read_log 192834'13560 (192834'13124) clean 69ff30b6/benchmark_data_storage0111-ib.pocstor_26659_object14659/head//24 by osd.21.0:15040 2014-08-11 15:26:59.202307 2014-08-13 18:06:06.633250 7f5c973a57a0 20 read_log 192834'13565 (192834'13116) clean ec4608b6/benchmark_data_storage0111-ib.pocstor_26659_object14361/head//24 by osd.21.0:15045 2014-08-11 15:26:59.222470 2014-08-13 18:06:06.633262 7f5c973a57a0 20 read_log 192834'13566 (192834'13094) clean 44d2c8b6/benchmark_data_storage0111-ib.pocstor_26659_object13186/head//24 by osd.21.0:15046 2014-08-11 15:26:59.225839 2014-08-13 18:06:06.633275 7f5c973a57a0 20 read_log 192834'13569 (192834'13122) clean 8d588cb6/benchmark_data_storage0111-ib.pocstor_26659_object14462/head//24 by osd.21.0:15049 2014-08-11 15:26:59.234425 2014-08-13 18:06:06.633295 7f5c973a57a0 20 read_log 192834'13570 (192834'13108) clean 6868dcb6/benchmark_data_storage0111-ib.pocstor_26659_object14042/head//24 by osd.21.0:15050 2014-08-11 15:26:59.241697 2014-08-13 18:06:06.633310 7f5c973a57a0 20 read_log 192834'13587 (192834'13560) delete 69ff30b6/benchmark_data_storage0111-ib.pocstor_26659_object14659/head//24 by osd.21.0:15067 2014-08-11 15:26:59.321378 2014-08-13 18:06:06.633326 7f5c973a57a0 20 read_log 192834'13594 (192834'13565) delete ec4608b6/benchmark_data_storage0111-ib.pocstor_26659_object14361/head//24 by osd.21.0:15074 2014-08-11 15:26:59.342615 2014-08-13 18:06:06.633338 7f5c973a57a0 20 read_log 192834'13595 (192834'13566) delete 44d2c8b6/benchmark_data_storage0111-ib.pocstor_26659_object13186/head//24 by osd.21.0:15075 2014-08-11 15:26:59.343546 2014-08-13 18:06:06.633353 7f5c973a57a0 20 read_log 192834'13598 (192834'13569) delete 8d588cb6/benchmark_data_storage0111-ib.pocstor_26659_object14462/head//24 by osd.21.0:15078 2014-08-11 15:26:59.355847 2014-08-13 18:06:06.633368 7f5c973a57a0 20 read_log 192834'13599 (192834'13570) delete 6868dcb6/benchmark_data_storage0111-ib.pocstor_26659_object14042/head//24 by osd.21.0:15079 2014-08-11 15:26:59.356493 2014-08-13 18:06:06.633386 7f5c973a57a0 20 read_log 192834'13652 (192834'13169) clean 49f38b6/benchmark_data_storage0111-ib.pocstor_26659_object18009/head//24 by osd.21.0:15132 2014-08-11 15:27:04.685396 2014-08-13 18:06:06.633401 7f5c973a57a0 20 read_log 192834'13653 (192834'13175) clean a6c858b6/benchmark_data_storage0111-ib.pocstor_26659_object18518/head//24 by osd.21.0:15133 2014-08-11 15:27:04.688362 2014-08-13 18:06:06.633414 7f5c973a57a0 20 read_log 192834'13654 (192834'13190) clean bbb98b6/benchmark_data_storage0111-ib.pocstor_26659_object19705/head//24 by osd.21.0:15134 2014-08-11 15:27:04.694200 2014-08-13 18:06:06.633426 7f5c973a57a0 20 read_log 192834'13655 (192834'13138) clean 330d58b6/benchmark_data_storage0111-ib.pocstor_26659_object15824/head//24 by osd.21.0:15135 2014-08-11 15:27:04.694461 2014-08-13 18:06:06.633454 7f5c973a57a0 20 read_log 192834'13657 (192834'13164) clean 635cc8b6/benchmark_data_storage0111-ib.pocstor_26659_object17522/head//24 by osd.21.0:15137 2014-08-11 15:27:04.700445 2014-08-13 18:06:06.633471 7f5c973a57a0 20 read_log 192834'13666 (192834'13172) clean bd143cb6/benchmark_data_storage0111-ib.pocstor_26659_object18307/head//24 by osd.21.0:15146 2014-08-11 15:27:04.734066 2014-08-13 18:06:06.633484 7f5c973a57a0 20 read_log 192834'13707 (192834'13652) delete 49f38b6/benchmark_data_storage0111-ib.pocstor_26659_object18009/head//24 by osd.21.0:15187 2014-08-11 15:27:04.805702 2014-08-13 18:06:06.633507 7f5c973a57a0 20 read_log 192834'13708 (192834'13653) delete a6c858b6/benchmark_data_storage0111-ib.pocstor_26659_object18518/head//24 by osd.21.0:15188 2014-08-11 15:27:04.806033 2014-08-13 18:06:06.633520 7f5c973a57a0 20 read_log 192834'13709 (192834'13655) delete 330d58b6/benchmark_data_storage0111-ib.pocstor_26659_object15824/head//24 by osd.21.0:15189 2014-08-11 15:27:04.808886 2014-08-13 18:06:06.633533 7f5c973a57a0 20 read_log 192834'13710 (192834'13654) delete bbb98b6/benchmark_data_storage0111-ib.pocstor_26659_object19705/head//24 by osd.21.0:15190 2014-08-11 15:27:04.809231 2014-08-13 18:06:06.633553 7f5c973a57a0 20 read_log 192834'13711 (192834'13657) delete 635cc8b6/benchmark_data_storage0111-ib.pocstor_26659_object17522/head//24 by osd.21.0:15191 2014-08-11 15:27:04.809509 2014-08-13 18:06:06.633568 7f5c973a57a0 20 read_log 192834'13722 (192834'13666) delete bd143cb6/benchmark_data_storage0111-ib.pocstor_26659_object18307/head//24 by osd.21.0:15202 2014-08-11 15:27:04.820860 2014-08-13 18:06:06.633581 7f5c973a57a0 20 read_log 192834'13736 (0'0) promote ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15216 2014-08-11 15:27:09.560786 2014-08-13 18:06:06.633593 7f5c973a57a0 20 read_log 192834'13737 (192834'13736) modify ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by client.1883634.0:13429 2014-08-11 15:27:09.550352 2014-08-13 18:06:06.633636 7f5c973a57a0 20 read_log 192834'13761 (0'0) promote 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15240 2014-08-11 15:27:16.424309 2014-08-13 18:06:06.633652 7f5c973a57a0 20 read_log 192834'13762 (192834'13761) modify 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by client.1883634.0:14954 2014-08-11 15:27:16.417123 2014-08-13 18:06:06.633665 7f5c973a57a0 20 read_log 192834'13765 (0'0) promote dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15244 2014-08-11 15:27:16.964951 2014-08-13 18:06:06.633677 7f5c973a57a0 20 read_log 192834'13766 (192834'13765) modify dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by client.1883634.0:15086 2014-08-11 15:27:16.955575 2014-08-13 18:06:06.633690 7f5c973a57a0 20 read_log 192834'13782 (192834'13213) clean 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15261 2014-08-11 15:27:20.153149 2014-08-13 18:06:06.633703 7f5c973a57a0 20 read_log 192834'13798 (192834'13782) delete 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15277 2014-08-11 15:27:25.483564 2014-08-13 18:06:06.633715 7f5c973a57a0 20 read_log 192834'13815 (192834'13241) clean 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15294 2014-08-11 15:27:25.584524 2014-08-13 18:06:06.633728 7f5c973a57a0 20 read_log 192834'13818 (192834'13815) delete 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15297 2014-08-11 15:27:25.590842 2014-08-13 18:06:06.633740 7f5c973a57a0 20 read_log 192834'13830 (0'0) promote 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15309 2014-08-11 15:27:27.554556 2014-08-13 18:06:06.633752 7f5c973a57a0 20 read_log 192834'13831 (192834'13830) modify 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by client.1883634.0:16441 2014-08-11 15:27:27.545577 2014-08-13 18:06:06.633765 7f5c973a57a0 20 read_log 192834'13848 (0'0) promote d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15327 2014-08-11 15:27:30.092782 2014-08-13 18:06:06.633777 7f5c973a57a0 20 read_log 192834'13849 (192834'13848) modify d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by client.1883634.0:16955 2014-08-11 15:27:30.083388 2014-08-13 18:06:06.633789 7f5c973a57a0 20 read_log 192834'13859 (192834'13271) clean 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15338 2014-08-11 15:27:30.895242 2014-08-13 18:06:06.633802 7f5c973a57a0 20 read_log 192834'13869 (192834'13859) delete 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15348 2014-08-11 15:27:30.940519 2014-08-13 18:06:06.633817 7f5c973a57a0 20 read_log 192834'13874 (0'0) promote ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15353 2014-08-11 15:27:34.114581 2014-08-13 18:06:06.633829 7f5c973a57a0 20 read_log 192834'13875 (192834'13874) modify ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by client.1883634.0:17289 2014-08-11 15:27:34.105681 2014-08-13 18:06:06.633842 7f5c973a57a0 20 read_log 192834'13901 (0'0) promote 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:15380 2014-08-11 15:27:40.064768 2014-08-13 18:06:06.633854 7f5c973a57a0 20 read_log 192834'13902 (192834'13901) modify 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by client.1883634.0:17929 2014-08-11 15:27:40.055936 2014-08-13 18:06:06.633874 7f5c973a57a0 20 read_log 192834'13935 (0'0) promote 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:15414 2014-08-11 15:27:49.550899 2014-08-13 18:06:06.633887 7f5c973a57a0 20 read_log 192834'13936 (192834'13935) modify 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by client.1883634.0:19102 2014-08-11 15:27:49.543600 2014-08-13 18:06:06.633911 7f5c973a57a0 20 read_log 192834'13948 (192834'13462) clean d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15427 2014-08-11 15:27:51.826015 2014-08-13 18:06:06.633926 7f5c973a57a0 20 read_log 192834'13949 (192834'13948) delete d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15428 2014-08-11 15:27:51.833700 2014-08-13 18:06:06.633939 7f5c973a57a0 20 read_log 192834'13950 (192834'13468) clean a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15429 2014-08-11 15:27:51.865418 2014-08-13 18:06:06.633953 7f5c973a57a0 20 read_log 192834'13964 (0'0) promote 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:15443 2014-08-11 15:27:55.392401 2014-08-13 18:06:06.633965 7f5c973a57a0 20 read_log 192834'13965 (192834'13964) modify 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by client.1883634.0:20135 2014-08-11 15:27:55.383147 2014-08-13 18:06:06.633980 7f5c973a57a0 20 read_log 192834'13970 (0'0) promote 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:15449 2014-08-11 15:27:56.267530 2014-08-13 18:06:06.633994 7f5c973a57a0 20 read_log 192834'13971 (192834'13970) modify 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by client.1883634.0:20341 2014-08-11 15:27:56.257674 2014-08-13 18:06:06.634007 7f5c973a57a0 20 read_log 192834'13972 (192834'13950) delete a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15451 2014-08-11 15:27:56.864442 2014-08-13 18:06:06.634021 7f5c973a57a0 20 read_log 192834'14038 (192834'13737) clean ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15517 2014-08-11 15:28:12.564298 2014-08-13 18:06:06.634034 7f5c973a57a0 20 read_log 192834'14048 (192834'14038) delete ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15527 2014-08-11 15:28:12.669545 2014-08-13 18:06:06.634048 7f5c973a57a0 20 read_log 192834'14069 (192834'13766) clean dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15548 2014-08-11 15:28:17.906447 2014-08-13 18:06:06.634061 7f5c973a57a0 20 read_log 192834'14070 (0'0) promote 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15549 2014-08-11 15:28:17.938466 2014-08-13 18:06:06.634075 7f5c973a57a0 20 read_log 192834'14071 (192834'13762) clean 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15550 2014-08-11 15:28:17.938747 2014-08-13 18:06:06.634089 7f5c973a57a0 20 read_log 192834'14072 (192834'14070) modify 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by client.1883634.0:23815 2014-08-11 15:28:17.857683 2014-08-13 18:06:06.634105 7f5c973a57a0 20 read_log 192834'14099 (0'0) promote 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15578 2014-08-11 15:28:20.395840 2014-08-13 18:06:06.634117 7f5c973a57a0 20 read_log 192834'14100 (192834'14099) modify 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by client.1883634.0:25238 2014-08-11 15:28:20.366028 2014-08-13 18:06:06.634130 7f5c973a57a0 20 read_log 192834'14118 (192834'14069) delete dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15597 2014-08-11 15:28:22.892244 2014-08-13 18:06:06.634167 7f5c973a57a0 20 read_log 192834'14119 (192834'14071) delete 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15598 2014-08-11 15:28:22.896160 2014-08-13 18:06:06.634184 7f5c973a57a0 20 read_log 192834'14133 (0'0) promote 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15612 2014-08-11 15:28:24.302539 2014-08-13 18:06:06.634196 7f5c973a57a0 20 read_log 192834'14134 (192834'14133) modify 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by client.1883634.0:27242 2014-08-11 15:28:24.273240 2014-08-13 18:06:06.634219 7f5c973a57a0 20 read_log 192834'14178 (192834'13831) clean 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15657 2014-08-11 15:28:28.189577 2014-08-13 18:06:06.634234 7f5c973a57a0 20 read_log 192834'14186 (192834'14178) delete 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15665 2014-08-11 15:28:28.236474 2014-08-13 18:06:06.634247 7f5c973a57a0 20 read_log 192834'14207 (0'0) promote d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15686 2014-08-11 15:28:31.992190 2014-08-13 18:06:06.634264 7f5c973a57a0 20 read_log 192834'14208 (192834'14207) modify d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by client.1883634.0:32516 2014-08-11 15:28:31.958593 2014-08-13 18:06:06.634279 7f5c973a57a0 20 read_log 192834'14213 (0'0) promote a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15692 2014-08-11 15:28:32.368941 2014-08-13 18:06:06.634291 7f5c973a57a0 20 read_log 192834'14214 (192834'14213) modify a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by client.1883634.0:32805 2014-08-11 15:28:32.343211 2014-08-13 18:06:06.634304 7f5c973a57a0 20 read_log 192834'14242 (192834'13849) clean d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15721 2014-08-11 15:28:33.724350 2014-08-13 18:06:06.634316 7f5c973a57a0 20 read_log 192834'14272 (0'0) promote ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:15751 2014-08-11 15:28:38.398943 2014-08-13 18:06:06.634329 7f5c973a57a0 20 read_log 192834'14273 (192834'14272) modify ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by client.1883634.0:36753 2014-08-11 15:28:38.367459 2014-08-13 18:06:06.634341 7f5c973a57a0 20 read_log 192834'14275 (192834'14242) delete d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15754 2014-08-11 15:28:38.677270 2014-08-13 18:06:06.634361 7f5c973a57a0 20 read_log 192834'14279 (192834'13875) clean ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15758 2014-08-11 15:28:38.781680 2014-08-13 18:06:06.634375 7f5c973a57a0 20 read_log 192834'14299 (0'0) promote 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:15778 2014-08-11 15:28:40.469982 2014-08-13 18:06:06.634387 7f5c973a57a0 20 read_log 192834'14300 (192834'14299) modify 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by client.1883634.0:38278 2014-08-11 15:28:40.442266 2014-08-13 18:06:06.634411 7f5c973a57a0 20 read_log 192834'14303 (0'0) promote dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:15782 2014-08-11 15:28:40.632774 2014-08-13 18:06:06.634424 7f5c973a57a0 20 read_log 192834'14304 (192834'14303) modify dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by client.1883634.0:38410 2014-08-11 15:28:40.607496 2014-08-13 18:06:06.634436 7f5c973a57a0 20 read_log 192834'14317 (0'0) promote 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:15796 2014-08-11 15:28:42.564704 2014-08-13 18:06:06.634461 7f5c973a57a0 20 read_log 192834'14318 (192834'14317) modify 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by client.1883634.0:39765 2014-08-11 15:28:42.535121 2014-08-13 18:06:06.634474 7f5c973a57a0 20 read_log 192834'14335 (0'0) promote d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:15814 2014-08-11 15:28:43.492699 2014-08-13 18:06:06.634486 7f5c973a57a0 20 read_log 192834'14336 (192834'14335) modify d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by client.1883634.0:40279 2014-08-11 15:28:43.468856 2014-08-13 18:06:06.634512 7f5c973a57a0 20 read_log 192834'14342 (192834'14279) delete ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15821 2014-08-11 15:28:43.774574 2014-08-13 18:06:06.634534 7f5c973a57a0 20 read_log 192834'14345 (192834'13902) clean 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:15824 2014-08-11 15:28:44.076870 2014-08-13 18:06:06.634549 7f5c973a57a0 20 read_log 192834'14346 (0'0) promote ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:15825 2014-08-11 15:28:44.087715 2014-08-13 18:06:06.634562 7f5c973a57a0 20 read_log 192834'14348 (192834'14346) modify ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by client.1883634.0:40613 2014-08-11 15:28:44.050415 2014-08-13 18:06:06.634574 7f5c973a57a0 20 read_log 192834'14354 (192834'14345) modify 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by client.1883634.0:41253 2014-08-11 15:28:44.817939 2014-08-13 18:06:06.634587 7f5c973a57a0 20 read_log 192834'14359 (192834'13936) modify 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by client.1883634.0:42426 2014-08-11 15:28:46.240555 2014-08-13 18:06:06.634599 7f5c973a57a0 20 read_log 192834'14368 (192834'13965) modify 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by client.1883634.0:43459 2014-08-11 15:28:46.561255 2014-08-13 18:06:06.634611 7f5c973a57a0 20 read_log 192834'14371 (192834'13971) modify 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by client.1883634.0:43665 2014-08-11 15:28:46.624796 2014-08-13 18:06:06.634624 7f5c973a57a0 20 read_log 192834'14395 (192834'14072) clean 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15874 2014-08-11 15:29:19.316108 2014-08-13 18:06:06.634645 7f5c973a57a0 20 read_log 192834'14401 (192834'14395) delete 9e6d1cb6/benchmark_data_storage0111-ib.pocstor_41526_object490/head//24 by osd.21.0:15880 2014-08-11 15:29:19.364777 2014-08-13 18:06:06.634658 7f5c973a57a0 20 read_log 192834'14419 (192834'14134) clean 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15898 2014-08-11 15:29:24.508420 2014-08-13 18:06:06.634671 7f5c973a57a0 20 read_log 192834'14425 (192834'14100) clean 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15904 2014-08-11 15:29:24.528486 2014-08-13 18:06:06.634683 7f5c973a57a0 20 read_log 192834'14444 (192834'14419) delete 4be804b6/benchmark_data_storage0111-ib.pocstor_41526_object3917/head//24 by osd.21.0:15923 2014-08-11 15:29:24.599879 2014-08-13 18:06:06.634696 7f5c973a57a0 20 read_log 192834'14450 (192834'14425) delete 1cec6cb6/benchmark_data_storage0111-ib.pocstor_41526_object1913/head//24 by osd.21.0:15929 2014-08-11 15:29:24.613478 2014-08-13 18:06:06.634708 7f5c973a57a0 20 read_log 192834'14510 (192834'14208) clean d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:15989 2014-08-11 15:29:34.926212 2014-08-13 18:06:06.634721 7f5c973a57a0 20 read_log 192834'14511 (192834'14214) clean a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:15990 2014-08-11 15:29:34.926532 2014-08-13 18:06:06.634733 7f5c973a57a0 20 read_log 192834'14524 (192834'14510) delete d62c90b6/benchmark_data_storage0111-ib.pocstor_41526_object9191/head//24 by osd.21.0:16003 2014-08-11 15:29:34.973217 2014-08-13 18:06:06.634746 7f5c973a57a0 20 read_log 192834'14525 (192834'14511) delete a79fa0b6/benchmark_data_storage0111-ib.pocstor_41526_object9480/head//24 by osd.21.0:16004 2014-08-11 15:29:34.975387 2014-08-13 18:06:06.634758 7f5c973a57a0 20 read_log 192834'14541 (192834'14273) clean ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:16020 2014-08-11 15:29:40.044619 2014-08-13 18:06:06.634771 7f5c973a57a0 20 read_log 192834'14561 (192834'14541) delete ea5430b6/benchmark_data_storage0111-ib.pocstor_41526_object13428/head//24 by osd.21.0:16040 2014-08-11 15:29:40.099076 2014-08-13 18:06:06.634783 7f5c973a57a0 20 read_log 192834'14586 (192834'14348) clean ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:16065 2014-08-11 15:29:45.179258 2014-08-13 18:06:06.634806 7f5c973a57a0 20 read_log 192834'14592 (192834'14354) clean 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:16071 2014-08-11 15:29:45.193556 2014-08-13 18:06:06.634828 7f5c973a57a0 20 read_log 192834'14593 (192834'14336) clean d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:16072 2014-08-11 15:29:45.197052 2014-08-13 18:06:06.634843 7f5c973a57a0 20 read_log 192834'14594 (192834'14318) clean 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:16073 2014-08-11 15:29:45.200363 2014-08-13 18:06:06.634855 7f5c973a57a0 20 read_log 192834'14596 (192834'14304) clean dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:16075 2014-08-11 15:29:45.207242 2014-08-13 18:06:06.634868 7f5c973a57a0 20 read_log 192834'14597 (192834'14300) clean 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:16076 2014-08-11 15:29:45.210707 2014-08-13 18:06:06.634881 7f5c973a57a0 20 read_log 192834'14613 (192834'14586) delete ae5980b6/benchmark_data_storage0111-ib.pocstor_41526_object17288/head//24 by osd.21.0:16092 2014-08-11 15:29:45.233557 2014-08-13 18:06:06.634893 7f5c973a57a0 20 read_log 192834'14618 (192834'14592) delete 364c54b6/benchmark_data_storage0111-ib.pocstor_41526_object17928/head//24 by osd.21.0:16097 2014-08-11 15:29:45.238583 2014-08-13 18:06:06.634908 7f5c973a57a0 20 read_log 192834'14619 (192834'14593) delete d35ec4b6/benchmark_data_storage0111-ib.pocstor_41526_object16954/head//24 by osd.21.0:16098 2014-08-11 15:29:45.239292 2014-08-13 18:06:06.634921 7f5c973a57a0 20 read_log 192834'14620 (192834'14594) delete 8434d4b6/benchmark_data_storage0111-ib.pocstor_41526_object16440/head//24 by osd.21.0:16099 2014-08-11 15:29:45.240868 2014-08-13 18:06:06.634933 7f5c973a57a0 20 read_log 192834'14622 (192834'14596) delete dbc718b6/benchmark_data_storage0111-ib.pocstor_41526_object15085/head//24 by osd.21.0:16101 2014-08-11 15:29:45.242039 2014-08-13 18:06:06.634946 7f5c973a57a0 20 read_log 192834'14623 (192834'14597) delete 286da8b6/benchmark_data_storage0111-ib.pocstor_41526_object14953/head//24 by osd.21.0:16102 2014-08-11 15:29:45.244382 2014-08-13 18:06:06.634958 7f5c973a57a0 20 read_log 192834'14642 (192834'14359) clean 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:16121 2014-08-11 15:29:50.280145 2014-08-13 18:06:06.634970 7f5c973a57a0 20 read_log 192834'14660 (192834'14642) delete 7c4b00b6/benchmark_data_storage0111-ib.pocstor_41526_object19101/head//24 by osd.21.0:16139 2014-08-11 15:29:50.298221 2014-08-13 18:06:06.634983 7f5c973a57a0 20 read_log 192834'14666 (192834'14368) clean 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:16145 2014-08-11 15:29:50.307407 2014-08-13 18:06:06.634995 7f5c973a57a0 20 read_log 192834'14675 (192834'14666) delete 428494b6/benchmark_data_storage0111-ib.pocstor_41526_object20134/head//24 by osd.21.0:16154 2014-08-11 15:29:50.314819 2014-08-13 18:06:06.635007 7f5c973a57a0 20 read_log 192834'14682 (192834'14371) clean 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:16161 2014-08-11 15:29:50.321980 2014-08-13 18:06:06.635020 7f5c973a57a0 20 read_log 192834'14687 (192834'14682) delete 23329cb6/benchmark_data_storage0111-ib.pocstor_41526_object20340/head//24 by osd.21.0:16166 2014-08-11 15:29:50.327358 2014-08-13 18:06:06.635040 7f5c973a57a0 20 read_log 192888'14690 (192888'14690) modify b6/hit_set_24.b6_archive_2014-08-11 16:02:27.751742_2014-08-11 16:44:57.822375/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.822933 2014-08-13 18:06:06.635054 7f5c973a57a0 20 read_log 192888'14701 (0'0) promote 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by osd.4.0:21464 2014-08-11 16:45:05.409770 2014-08-13 18:06:06.635074 7f5c973a57a0 20 read_log 192888'14702 (192888'14701) modify 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.1315259.0:2683 2014-08-11 16:45:05.402670 2014-08-13 18:06:06.635104 7f5c973a57a0 20 read_log 192888'14703 (0'0) promote 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by osd.4.0:21500 2014-08-11 16:45:11.786111 2014-08-13 18:06:06.635117 7f5c973a57a0 20 read_log 192888'14704 (192888'14703) modify 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.1315259.0:4364 2014-08-11 16:45:11.777501 2014-08-13 18:06:06.635137 7f5c973a57a0 20 read_log 192888'14705 (0'0) promote 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by osd.4.0:21504 2014-08-11 16:45:12.137164 2014-08-13 18:06:06.635161 7f5c973a57a0 20 read_log 192888'14706 (192888'14705) modify 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.1315259.0:4461 2014-08-11 16:45:12.128694 2014-08-13 18:06:06.635174 7f5c973a57a0 20 read_log 192888'14709 (0'0) promote c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by osd.4.0:21508 2014-08-11 16:45:12.450511 2014-08-13 18:06:06.635186 7f5c973a57a0 20 read_log 192888'14710 (192888'14709) modify c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.1315259.0:4546 2014-08-11 16:45:12.441798 2014-08-13 18:06:06.635207 7f5c973a57a0 20 read_log 192888'14715 (0'0) promote 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by osd.4.0:21546 2014-08-11 16:45:16.045119 2014-08-13 18:06:06.635225 7f5c973a57a0 20 read_log 192888'14716 (192888'14715) modify 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.1315259.0:5518 2014-08-11 16:45:16.035342 2014-08-13 18:06:06.635239 7f5c973a57a0 20 read_log 192888'14719 (0'0) promote aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by osd.4.0:21566 2014-08-11 16:45:18.783660 2014-08-13 18:06:06.635251 7f5c973a57a0 20 read_log 192888'14720 (192888'14719) modify aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.1315259.0:6277 2014-08-11 16:45:18.775492 2014-08-13 18:06:06.635264 7f5c973a57a0 20 read_log 192888'14721 (0'0) promote f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by osd.4.0:21622 2014-08-11 16:45:27.434372 2014-08-13 18:06:06.635278 7f5c973a57a0 20 read_log 192888'14722 (192888'14721) modify f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.1315259.0:8627 2014-08-11 16:45:27.425153 2014-08-13 18:06:06.635291 7f5c973a57a0 20 read_log 192888'14725 (0'0) promote a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by osd.4.0:21652 2014-08-11 16:45:32.006393 2014-08-13 18:06:06.635310 7f5c973a57a0 20 read_log 192888'14726 (192888'14725) modify a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.1315259.0:9816 2014-08-11 16:45:31.997783 2014-08-13 18:06:06.635323 7f5c973a57a0 20 read_log 192888'14739 (0'0) promote 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by osd.4.0:21750 2014-08-11 16:45:47.014484 2014-08-13 18:06:06.635346 7f5c973a57a0 20 read_log 192888'14740 (192888'14739) modify 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.1315259.0:13530 2014-08-11 16:45:47.003185 2014-08-13 18:06:06.635362 7f5c973a57a0 20 read_log 192888'14745 (0'0) promote 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by osd.4.0:21780 2014-08-11 16:45:52.607763 2014-08-13 18:06:06.635375 7f5c973a57a0 20 read_log 192888'14746 (192888'14745) modify 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.1315259.0:15013 2014-08-11 16:45:52.597880 2014-08-13 18:06:06.635388 7f5c973a57a0 20 read_log 192888'14749 (0'0) promote 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by osd.4.0:21866 2014-08-11 16:46:03.212155 2014-08-13 18:06:06.635411 7f5c973a57a0 20 read_log 192888'14750 (192888'14749) modify 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.1315259.0:17709 2014-08-11 16:46:03.204008 2014-08-13 18:06:06.635427 7f5c973a57a0 20 read_log 192888'14751 (0'0) promote 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by osd.4.0:21868 2014-08-11 16:46:03.420732 2014-08-13 18:06:06.635448 7f5c973a57a0 20 read_log 192888'14752 (192888'14751) modify 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.1315259.0:17762 2014-08-11 16:46:03.411561 2014-08-13 18:06:06.635465 7f5c973a57a0 20 read_log 192888'14757 (0'0) promote 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by osd.4.0:21904 2014-08-11 16:46:07.952459 2014-08-13 18:06:06.635478 7f5c973a57a0 20 read_log 192888'14758 (192888'14757) modify 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.1315259.0:18942 2014-08-11 16:46:07.943567 2014-08-13 18:06:06.635491 7f5c973a57a0 20 read_log 192888'14759 (0'0) promote 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by osd.4.0:21908 2014-08-11 16:46:08.476804 2014-08-13 18:06:06.635503 7f5c973a57a0 20 read_log 192888'14760 (192888'14759) modify 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.1315259.0:19052 2014-08-11 16:46:08.466460 2014-08-13 18:06:06.635522 7f5c973a57a0 20 read_log 192888'14761 (0'0) promote d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by osd.4.0:21938 2014-08-11 16:46:13.100679 2014-08-13 18:06:06.635535 7f5c973a57a0 20 read_log 192888'14762 (192888'14761) modify d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.1315259.0:20266 2014-08-11 16:46:13.092080 2014-08-13 18:06:06.635548 7f5c973a57a0 20 read_log 192888'14763 (0'0) promote 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by osd.4.0:21942 2014-08-11 16:46:13.359338 2014-08-13 18:06:06.635560 7f5c973a57a0 20 read_log 192888'14764 (192888'14763) modify 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.1315259.0:20336 2014-08-11 16:46:13.349574 2014-08-13 18:06:06.635591 7f5c973a57a0 20 read_log 192888'14779 (0'0) promote fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by osd.4.0:22032 2014-08-11 16:46:26.959638 2014-08-13 18:06:06.635606 7f5c973a57a0 20 read_log 192888'14780 (192888'14779) modify fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.1315259.0:23822 2014-08-11 16:46:26.951072 2014-08-13 18:06:06.635619 7f5c973a57a0 20 read_log 192888'14789 (0'0) promote 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by osd.4.0:22074 2014-08-11 16:46:30.756897 2014-08-13 18:06:06.635631 7f5c973a57a0 20 read_log 192888'14790 (192888'14789) modify 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.1315259.0:24796 2014-08-11 16:46:30.750037 2014-08-13 18:06:06.635644 7f5c973a57a0 20 read_log 192888'14795 (0'0) promote 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by osd.4.0:22134 2014-08-11 16:46:39.588936 2014-08-13 18:06:06.635656 7f5c973a57a0 20 read_log 192888'14796 (192888'14795) modify 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.1315259.0:26931 2014-08-11 16:46:39.579318 2014-08-13 18:06:06.635668 7f5c973a57a0 20 read_log 192888'14801 (0'0) promote 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by osd.4.0:22170 2014-08-11 16:46:48.689171 2014-08-13 18:06:06.635681 7f5c973a57a0 20 read_log 192888'14802 (192888'14801) modify 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.1315259.0:29171 2014-08-11 16:46:48.680986 2014-08-13 18:06:06.635693 7f5c973a57a0 20 read_log 192888'14803 (0'0) promote 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by osd.4.0:22176 2014-08-11 16:46:50.271004 2014-08-13 18:06:06.635730 7f5c973a57a0 20 read_log 192888'14804 (192888'14803) modify 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.1315259.0:29533 2014-08-11 16:46:50.262555 2014-08-13 18:06:06.635743 7f5c973a57a0 20 read_log 192888'14814 (192888'14702) modify 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.1315259.0:33410 2014-08-11 16:46:56.630025 2014-08-13 18:06:06.635756 7f5c973a57a0 20 read_log 192888'14815 (192888'14704) modify 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.1315259.0:35091 2014-08-11 16:46:57.066653 2014-08-13 18:06:06.635769 7f5c973a57a0 20 read_log 192888'14816 (192888'14706) modify 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.1315259.0:35188 2014-08-11 16:46:57.095514 2014-08-13 18:06:06.635782 7f5c973a57a0 20 read_log 192888'14818 (192888'14710) modify c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.1315259.0:35273 2014-08-11 16:46:57.120762 2014-08-13 18:06:06.635794 7f5c973a57a0 20 read_log 192888'14821 (192888'14716) modify 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.1315259.0:36245 2014-08-11 16:46:57.373025 2014-08-13 18:06:06.635810 7f5c973a57a0 20 read_log 192888'14823 (192888'14720) modify aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.1315259.0:37004 2014-08-11 16:46:57.569190 2014-08-13 18:06:06.635823 7f5c973a57a0 20 read_log 192888'14824 (192888'14722) modify f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.1315259.0:39354 2014-08-11 16:46:58.189203 2014-08-13 18:06:06.635835 7f5c973a57a0 20 read_log 192888'14826 (192888'14726) modify a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.1315259.0:40543 2014-08-11 16:46:58.519129 2014-08-13 18:06:06.635847 7f5c973a57a0 20 read_log 192888'14833 (192888'14740) modify 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.1315259.0:44257 2014-08-11 16:46:59.528427 2014-08-13 18:06:06.635862 7f5c973a57a0 20 read_log 192888'14836 (192888'14746) modify 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.1315259.0:45740 2014-08-11 16:46:59.955783 2014-08-13 18:06:06.635875 7f5c973a57a0 20 read_log 192888'14838 (192888'14750) modify 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.1315259.0:48436 2014-08-11 16:47:00.763339 2014-08-13 18:06:06.635889 7f5c973a57a0 20 read_log 192888'14839 (192888'14752) modify 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.1315259.0:48489 2014-08-11 16:47:00.777604 2014-08-13 18:06:06.635902 7f5c973a57a0 20 read_log 192888'14842 (192888'14758) modify 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.1315259.0:49669 2014-08-11 16:47:01.137578 2014-08-13 18:06:06.635914 7f5c973a57a0 20 read_log 192888'14843 (192888'14760) modify 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.1315259.0:49779 2014-08-11 16:47:01.170689 2014-08-13 18:06:06.635927 7f5c973a57a0 20 read_log 192888'14844 (192888'14762) modify d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.1315259.0:50993 2014-08-11 16:47:01.543674 2014-08-13 18:06:06.635939 7f5c973a57a0 20 read_log 192888'14845 (192888'14764) modify 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.1315259.0:51063 2014-08-11 16:47:01.568141 2014-08-13 18:06:06.635954 7f5c973a57a0 20 read_log 192888'14853 (192888'14780) modify fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.1315259.0:54549 2014-08-11 16:47:02.598201 2014-08-13 18:06:06.635967 7f5c973a57a0 20 read_log 192888'14858 (192888'14790) modify 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.1315259.0:55523 2014-08-11 16:47:02.935489 2014-08-13 18:06:06.635981 7f5c973a57a0 20 read_log 192888'14861 (192888'14796) modify 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.1315259.0:57658 2014-08-11 16:47:03.576744 2014-08-13 18:06:06.636001 7f5c973a57a0 20 read_log 192888'14864 (192888'14802) modify 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.1315259.0:59898 2014-08-11 16:47:04.277608 2014-08-13 18:06:06.636016 7f5c973a57a0 20 read_log 192888'14865 (192888'14804) modify 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.1315259.0:60260 2014-08-11 16:47:04.395260 2014-08-13 18:06:06.636028 7f5c973a57a0 20 read_log 192888'14868 (192888'14868) modify b6/hit_set_24.b6_archive_2014-08-11 16:44:57.823070_2014-08-11 17:05:15.028214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.028548 2014-08-13 18:06:06.636043 7f5c973a57a0 20 read_log 192888'14869 (192888'14690) delete b6/hit_set_24.b6_archive_2014-08-11 16:02:27.751742_2014-08-11 16:44:57.822375/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.028548 2014-08-13 18:06:06.636059 7f5c973a57a0 20 read_log 192888'14874 (0'0) promote d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by osd.4.0:22712 2014-08-11 17:05:25.826181 2014-08-13 18:06:06.636071 7f5c973a57a0 20 read_log 192888'14875 (192888'14874) modify d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.942868.0:2670 2014-08-11 17:05:25.818065 2014-08-13 18:06:06.636086 7f5c973a57a0 20 read_log 192888'14878 (0'0) promote a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by osd.4.0:22728 2014-08-11 17:05:28.051620 2014-08-13 18:06:06.636098 7f5c973a57a0 20 read_log 192888'14879 (192888'14878) modify a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.942868.0:3273 2014-08-11 17:05:28.043688 2014-08-13 18:06:06.636113 7f5c973a57a0 20 read_log 192888'14884 (0'0) promote a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by osd.4.0:22776 2014-08-11 17:05:32.603579 2014-08-13 18:06:06.636125 7f5c973a57a0 20 read_log 192888'14885 (192888'14884) modify a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.942868.0:4505 2014-08-11 17:05:32.595158 2014-08-13 18:06:06.636140 7f5c973a57a0 20 read_log 192888'14886 (0'0) promote 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by osd.4.0:22806 2014-08-11 17:05:36.812424 2014-08-13 18:06:06.636158 7f5c973a57a0 20 read_log 192888'14887 (192888'14886) modify 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.942868.0:5586 2014-08-11 17:05:36.805050 2014-08-13 18:06:06.636173 7f5c973a57a0 20 read_log 192888'14888 (0'0) promote 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by osd.4.0:22824 2014-08-11 17:05:39.247033 2014-08-13 18:06:06.636186 7f5c973a57a0 20 read_log 192888'14889 (192888'14888) modify 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.942868.0:6258 2014-08-11 17:05:39.240567 2014-08-13 18:06:06.636198 7f5c973a57a0 20 read_log 192888'14890 (0'0) promote 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by osd.4.0:22840 2014-08-11 17:05:41.141836 2014-08-13 18:06:06.636211 7f5c973a57a0 20 read_log 192888'14891 (192888'14890) modify 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.942868.0:6783 2014-08-11 17:05:41.130767 2014-08-13 18:06:06.636225 7f5c973a57a0 20 read_log 192888'14894 (0'0) promote 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by osd.4.0:22886 2014-08-11 17:05:48.058684 2014-08-13 18:06:06.636238 7f5c973a57a0 20 read_log 192888'14895 (192888'14894) modify 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.942868.0:8617 2014-08-11 17:05:48.049769 2014-08-13 18:06:06.636252 7f5c973a57a0 20 read_log 192888'14898 (0'0) promote ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by osd.4.0:22926 2014-08-11 17:05:55.183907 2014-08-13 18:06:06.636277 7f5c973a57a0 20 read_log 192888'14899 (192888'14898) modify ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.942868.0:10549 2014-08-11 17:05:55.175570 2014-08-13 18:06:06.636292 7f5c973a57a0 20 read_log 192888'14900 (0'0) promote 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by osd.4.0:22948 2014-08-11 17:05:58.546578 2014-08-13 18:06:06.636308 7f5c973a57a0 20 read_log 192888'14901 (192888'14900) modify 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.942868.0:11465 2014-08-11 17:05:58.538142 2014-08-13 18:06:06.636323 7f5c973a57a0 20 read_log 192888'14902 (0'0) promote a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by osd.4.0:22962 2014-08-11 17:06:01.398686 2014-08-13 18:06:06.636336 7f5c973a57a0 20 read_log 192888'14903 (192888'14902) modify a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.942868.0:12249 2014-08-11 17:06:01.389020 2014-08-13 18:06:06.636348 7f5c973a57a0 20 read_log 192888'14904 (0'0) promote 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by osd.4.0:22966 2014-08-11 17:06:01.871922 2014-08-13 18:06:06.636367 7f5c973a57a0 20 read_log 192888'14905 (192888'14904) modify 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.942868.0:12381 2014-08-11 17:06:01.862473 2014-08-13 18:06:06.636390 7f5c973a57a0 20 read_log 192888'14912 (0'0) promote 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by osd.4.0:23026 2014-08-11 17:06:11.503401 2014-08-13 18:06:06.636410 7f5c973a57a0 20 read_log 192888'14913 (192888'14912) modify 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.942868.0:14815 2014-08-11 17:06:11.494499 2014-08-13 18:06:06.636429 7f5c973a57a0 20 read_log 192888'14914 (0'0) promote 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by osd.4.0:23078 2014-08-11 17:06:19.931983 2014-08-13 18:06:06.636445 7f5c973a57a0 20 read_log 192888'14915 (192888'14914) modify 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.942868.0:16965 2014-08-11 17:06:19.923270 2014-08-13 18:06:06.636458 7f5c973a57a0 20 read_log 192888'14916 (0'0) promote 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by osd.4.0:23096 2014-08-11 17:06:21.381673 2014-08-13 18:06:06.636470 7f5c973a57a0 20 read_log 192888'14917 (192888'14916) modify 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.942868.0:17348 2014-08-11 17:06:21.370306 2014-08-13 18:06:06.636493 7f5c973a57a0 20 read_log 192888'14920 (0'0) promote 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by osd.4.0:23138 2014-08-11 17:06:27.418074 2014-08-13 18:06:06.636506 7f5c973a57a0 20 read_log 192888'14921 (192888'14920) modify 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.942868.0:18953 2014-08-11 17:06:27.410061 2014-08-13 18:06:06.636518 7f5c973a57a0 20 read_log 192888'14926 (0'0) promote 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by osd.4.0:23150 2014-08-11 17:06:29.536553 2014-08-13 18:06:06.636541 7f5c973a57a0 20 read_log 192888'14927 (192888'14926) modify 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.942868.0:19517 2014-08-11 17:06:29.528414 2014-08-13 18:06:06.636554 7f5c973a57a0 20 read_log 192888'14928 (0'0) promote e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by osd.4.0:23152 2014-08-11 17:06:29.569157 2014-08-13 18:06:06.636567 7f5c973a57a0 20 read_log 192888'14929 (192888'14928) modify e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.942868.0:19526 2014-08-11 17:06:29.560339 2014-08-13 18:06:06.636595 7f5c973a57a0 20 read_log 192888'14930 (0'0) promote 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by osd.4.0:23162 2014-08-11 17:06:30.602831 2014-08-13 18:06:06.636620 7f5c973a57a0 20 read_log 192888'14931 (192888'14930) modify 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.942868.0:19798 2014-08-11 17:06:30.593767 2014-08-13 18:06:06.636636 7f5c973a57a0 20 read_log 192888'14934 (0'0) promote 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by osd.4.0:23194 2014-08-11 17:06:35.683952 2014-08-13 18:06:06.636648 7f5c973a57a0 20 read_log 192888'14935 (192888'14934) modify 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.942868.0:21128 2014-08-11 17:06:35.675477 2014-08-13 18:06:06.636661 7f5c973a57a0 20 read_log 192888'14938 (0'0) promote 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by osd.4.0:23214 2014-08-11 17:06:38.189695 2014-08-13 18:06:06.636673 7f5c973a57a0 20 read_log 192888'14939 (192888'14938) modify 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.942868.0:21790 2014-08-11 17:06:38.183130 2014-08-13 18:06:06.636686 7f5c973a57a0 20 read_log 192888'14944 (0'0) promote 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by osd.4.0:23236 2014-08-11 17:06:41.445686 2014-08-13 18:06:06.636698 7f5c973a57a0 20 read_log 192888'14945 (192888'14944) modify 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.942868.0:22616 2014-08-11 17:06:41.437806 2014-08-13 18:06:06.636711 7f5c973a57a0 20 read_log 192888'14948 (0'0) promote ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by osd.4.0:23280 2014-08-11 17:06:46.874368 2014-08-13 18:06:06.636723 7f5c973a57a0 20 read_log 192888'14949 (192888'14948) modify ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.942868.0:24043 2014-08-11 17:06:46.864634 2014-08-13 18:06:06.636735 7f5c973a57a0 20 read_log 192888'14950 (0'0) promote d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by osd.4.0:23310 2014-08-11 17:06:49.329177 2014-08-13 18:06:06.636749 7f5c973a57a0 20 read_log 192888'14951 (192888'14950) modify d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.942868.0:24707 2014-08-11 17:06:49.320526 2014-08-13 18:06:06.636762 7f5c973a57a0 20 read_log 192888'14954 (0'0) promote edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by osd.4.0:23334 2014-08-11 17:06:51.652565 2014-08-13 18:06:06.636774 7f5c973a57a0 20 read_log 192888'14955 (192888'14954) modify edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.942868.0:25324 2014-08-11 17:06:51.642588 2014-08-13 18:06:06.636787 7f5c973a57a0 20 read_log 192888'14958 (0'0) promote 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by osd.4.0:23362 2014-08-11 17:06:57.195033 2014-08-13 18:06:06.636799 7f5c973a57a0 20 read_log 192888'14959 (192888'14958) modify 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.942868.0:26738 2014-08-11 17:06:57.185923 2014-08-13 18:06:06.636811 7f5c973a57a0 20 read_log 192888'14960 (0'0) promote 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by osd.4.0:23366 2014-08-11 17:06:58.507967 2014-08-13 18:06:06.636824 7f5c973a57a0 20 read_log 192888'14961 (192888'14960) modify 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.942868.0:27076 2014-08-11 17:06:58.498683 2014-08-13 18:06:06.636837 7f5c973a57a0 20 read_log 192888'14962 (0'0) promote 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by osd.4.0:23376 2014-08-11 17:06:59.781997 2014-08-13 18:06:06.636849 7f5c973a57a0 20 read_log 192888'14963 (192888'14962) modify 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.942868.0:27426 2014-08-11 17:06:59.773654 2014-08-13 18:06:06.636861 7f5c973a57a0 20 read_log 192888'14966 (0'0) promote 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by osd.4.0:23392 2014-08-11 17:07:01.536730 2014-08-13 18:06:06.636880 7f5c973a57a0 20 read_log 192888'14967 (192888'14966) modify 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.942868.0:27875 2014-08-11 17:07:01.525856 2014-08-13 18:06:06.636894 7f5c973a57a0 20 read_log 192888'14968 (0'0) promote d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by osd.4.0:23396 2014-08-11 17:07:03.003942 2014-08-13 18:06:06.636909 7f5c973a57a0 20 read_log 192888'14969 (192888'14968) modify d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.942868.0:28276 2014-08-11 17:07:02.994869 2014-08-13 18:06:06.636921 7f5c973a57a0 20 read_log 192888'14970 (0'0) promote f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by osd.4.0:23404 2014-08-11 17:07:03.815431 2014-08-13 18:06:06.636934 7f5c973a57a0 20 read_log 192888'14971 (192888'14970) modify f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.942868.0:28496 2014-08-11 17:07:03.806089 2014-08-13 18:06:06.636948 7f5c973a57a0 20 read_log 192888'14974 (0'0) promote 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by osd.4.0:23420 2014-08-11 17:07:06.508847 2014-08-13 18:06:06.636962 7f5c973a57a0 20 read_log 192888'14975 (192888'14974) modify 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.942868.0:29185 2014-08-11 17:07:06.499490 2014-08-13 18:06:06.636975 7f5c973a57a0 20 read_log 192888'14988 (192888'14875) modify d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.942868.0:33936 2014-08-11 17:07:16.652094 2014-08-13 18:06:06.636989 7f5c973a57a0 20 read_log 192888'14990 (192888'14879) modify a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.942868.0:34539 2014-08-11 17:07:16.818800 2014-08-13 18:06:06.637022 7f5c973a57a0 20 read_log 192888'14993 (192888'14885) modify a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.942868.0:35771 2014-08-11 17:07:17.160060 2014-08-13 18:06:06.637038 7f5c973a57a0 20 read_log 192888'14994 (192888'14887) modify 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.942868.0:36852 2014-08-11 17:07:17.450395 2014-08-13 18:06:06.637050 7f5c973a57a0 20 read_log 192888'14995 (192888'14889) modify 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.942868.0:37524 2014-08-11 17:07:17.633137 2014-08-13 18:06:06.637065 7f5c973a57a0 20 read_log 192888'14996 (192888'14891) modify 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.942868.0:38049 2014-08-11 17:07:17.777571 2014-08-13 18:06:06.637077 7f5c973a57a0 20 read_log 192888'14998 (192888'14895) modify 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.942868.0:39883 2014-08-11 17:07:18.310216 2014-08-13 18:06:06.637090 7f5c973a57a0 20 read_log 192888'15000 (192888'14899) modify ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.942868.0:41815 2014-08-11 17:07:18.831064 2014-08-13 18:06:06.637102 7f5c973a57a0 20 read_log 192888'15001 (192888'14901) modify 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.942868.0:42731 2014-08-11 17:07:19.077431 2014-08-13 18:06:06.637122 7f5c973a57a0 20 read_log 192888'15002 (192888'14903) modify a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.942868.0:43515 2014-08-11 17:07:19.293534 2014-08-13 18:06:06.637135 7f5c973a57a0 20 read_log 192888'15003 (192888'14905) modify 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.942868.0:43647 2014-08-11 17:07:19.332183 2014-08-13 18:06:06.637158 7f5c973a57a0 20 read_log 192888'15007 (192888'14913) modify 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.942868.0:46081 2014-08-11 17:07:20.015459 2014-08-13 18:06:06.637173 7f5c973a57a0 20 read_log 192888'15008 (192888'14915) modify 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.942868.0:48231 2014-08-11 17:07:20.661825 2014-08-13 18:06:06.637193 7f5c973a57a0 20 read_log 192888'15009 (192888'14917) modify 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.942868.0:48614 2014-08-11 17:07:20.766166 2014-08-13 18:06:06.637208 7f5c973a57a0 20 read_log 192888'15011 (192888'14921) modify 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.942868.0:50219 2014-08-11 17:07:21.251884 2014-08-13 18:06:06.637221 7f5c973a57a0 20 read_log 192888'15014 (192888'14927) modify 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.942868.0:50783 2014-08-11 17:07:21.431039 2014-08-13 18:06:06.637234 7f5c973a57a0 20 read_log 192888'15015 (192888'14929) modify e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.942868.0:50792 2014-08-11 17:07:21.434279 2014-08-13 18:06:06.637249 7f5c973a57a0 20 read_log 192888'15016 (192888'14931) modify 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.942868.0:51064 2014-08-11 17:07:21.520475 2014-08-13 18:06:06.637261 7f5c973a57a0 20 read_log 192888'15018 (192888'14935) modify 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.942868.0:52394 2014-08-11 17:07:21.924644 2014-08-13 18:06:06.637277 7f5c973a57a0 20 read_log 192888'15020 (192888'14939) modify 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.942868.0:53056 2014-08-11 17:07:22.134108 2014-08-13 18:06:06.637292 7f5c973a57a0 20 read_log 192888'15023 (192888'14945) modify 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.942868.0:53882 2014-08-11 17:07:22.404192 2014-08-13 18:06:06.637304 7f5c973a57a0 20 read_log 192888'15025 (192888'14949) modify ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.942868.0:55309 2014-08-11 17:07:22.808490 2014-08-13 18:06:06.637318 7f5c973a57a0 20 read_log 192888'15026 (192888'14951) modify d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.942868.0:55973 2014-08-11 17:07:23.013407 2014-08-13 18:06:06.637330 7f5c973a57a0 20 read_log 192888'15028 (192888'14955) modify edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.942868.0:56590 2014-08-11 17:07:23.199855 2014-08-13 18:06:06.637349 7f5c973a57a0 20 read_log 192888'15030 (192888'14959) modify 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.942868.0:58004 2014-08-11 17:07:23.615104 2014-08-13 18:06:06.637361 7f5c973a57a0 20 read_log 192888'15031 (192888'14961) modify 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.942868.0:58342 2014-08-11 17:07:23.712254 2014-08-13 18:06:06.637374 7f5c973a57a0 20 read_log 192888'15032 (192888'14963) modify 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.942868.0:58692 2014-08-11 17:07:23.821281 2014-08-13 18:06:06.637402 7f5c973a57a0 20 read_log 192888'15034 (192888'14967) modify 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.942868.0:59141 2014-08-11 17:07:23.956493 2014-08-13 18:06:06.637419 7f5c973a57a0 20 read_log 192888'15035 (192888'14969) modify d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.942868.0:59542 2014-08-11 17:07:24.075182 2014-08-13 18:06:06.637435 7f5c973a57a0 20 read_log 192888'15036 (192888'14971) modify f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.942868.0:59762 2014-08-11 17:07:24.142878 2014-08-13 18:06:06.637458 7f5c973a57a0 20 read_log 192888'15038 (192888'14975) modify 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.942868.0:60451 2014-08-11 17:07:24.344284 2014-08-13 18:06:06.637474 7f5c973a57a0 20 read_log 193305'15044 (193305'15044) modify b6/hit_set_24.b6_archive_2014-08-11 18:37:04.909790_2014-08-11 18:42:37.869502/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.869737 2014-08-13 18:06:06.637504 7f5c973a57a0 20 read_log 193305'15045 (192888'14868) delete b6/hit_set_24.b6_archive_2014-08-11 16:44:57.823070_2014-08-11 17:05:15.028214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.869737 2014-08-13 18:06:06.637521 7f5c973a57a0 20 read_log 193305'15046 (0'0) promote 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by osd.4.0:12 2014-08-11 18:42:37.903343 2014-08-13 18:06:06.637534 7f5c973a57a0 20 read_log 193305'15047 (193305'15046) modify 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.951337.0:1248 2014-08-11 18:42:37.863310 2014-08-13 18:06:06.637554 7f5c973a57a0 20 read_log 193305'15048 (0'0) promote e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by osd.4.0:35 2014-08-11 18:42:42.057993 2014-08-13 18:06:06.637571 7f5c973a57a0 20 read_log 193305'15049 (193305'15048) modify e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.951337.0:2358 2014-08-11 18:42:42.051033 2014-08-13 18:06:06.637584 7f5c973a57a0 20 read_log 193305'15050 (0'0) promote d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by osd.4.0:37 2014-08-11 18:42:42.443128 2014-08-13 18:06:06.637596 7f5c973a57a0 20 read_log 193305'15051 (193305'15050) modify d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.951337.0:2463 2014-08-11 18:42:42.432532 2014-08-13 18:06:06.637626 7f5c973a57a0 20 read_log 193305'15052 (0'0) promote 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by osd.4.0:41 2014-08-11 18:42:43.792019 2014-08-13 18:06:06.637641 7f5c973a57a0 20 read_log 193305'15053 (193305'15052) modify 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.951337.0:2836 2014-08-11 18:42:43.785300 2014-08-13 18:06:06.637654 7f5c973a57a0 20 read_log 193305'15054 (0'0) promote 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by osd.4.0:45 2014-08-11 18:42:44.772532 2014-08-13 18:06:06.637666 7f5c973a57a0 20 read_log 193305'15055 (193305'15054) modify 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.951337.0:3103 2014-08-11 18:42:44.765714 2014-08-13 18:06:06.637679 7f5c973a57a0 20 read_log 193305'15056 (0'0) promote 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by osd.4.0:61 2014-08-11 18:42:49.737259 2014-08-13 18:06:06.637691 7f5c973a57a0 20 read_log 193305'15057 (193305'15056) modify 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.951337.0:4459 2014-08-11 18:42:49.729113 2014-08-13 18:06:06.637704 7f5c973a57a0 20 read_log 193305'15058 (0'0) promote 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by osd.4.0:69 2014-08-11 18:42:50.870661 2014-08-13 18:06:06.637753 7f5c973a57a0 20 read_log 193305'15059 (193305'15058) modify 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.951337.0:4770 2014-08-11 18:42:50.862483 2014-08-13 18:06:06.637773 7f5c973a57a0 20 read_log 193305'15060 (0'0) promote 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by osd.4.0:71 2014-08-11 18:42:52.107574 2014-08-13 18:06:06.637786 7f5c973a57a0 20 read_log 193305'15061 (193305'15060) modify 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.951337.0:5086 2014-08-11 18:42:52.099666 2014-08-13 18:06:06.637799 7f5c973a57a0 20 read_log 193305'15062 (0'0) promote b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by osd.4.0:85 2014-08-11 18:42:54.268125 2014-08-13 18:06:06.637816 7f5c973a57a0 20 read_log 193305'15063 (193305'15062) modify b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.951337.0:5681 2014-08-11 18:42:54.260287 2014-08-13 18:06:06.637865 7f5c973a57a0 20 read_log 193305'15064 (0'0) promote 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by osd.4.0:119 2014-08-11 18:43:00.907416 2014-08-13 18:06:06.637882 7f5c973a57a0 20 read_log 193305'15065 (193305'15064) modify 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.951337.0:7515 2014-08-11 18:43:00.900205 2014-08-13 18:06:06.637895 7f5c973a57a0 20 read_log 193305'15066 (0'0) promote 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by osd.4.0:137 2014-08-11 18:43:04.344548 2014-08-13 18:06:06.637908 7f5c973a57a0 20 read_log 193305'15067 (193305'15066) modify 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.951337.0:8410 2014-08-11 18:43:04.333582 2014-08-13 18:06:06.637920 7f5c973a57a0 20 read_log 193305'15068 (0'0) promote 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by osd.4.0:147 2014-08-11 18:43:06.786242 2014-08-13 18:06:06.637933 7f5c973a57a0 20 read_log 193305'15069 (193305'15068) modify 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.951337.0:9042 2014-08-11 18:43:06.779080 2014-08-13 18:06:06.637945 7f5c973a57a0 20 read_log 193305'15070 (0'0) promote e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by osd.4.0:157 2014-08-11 18:43:08.605928 2014-08-13 18:06:06.637958 7f5c973a57a0 20 read_log 193305'15071 (193305'15070) modify e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.951337.0:9538 2014-08-11 18:43:08.598413 2014-08-13 18:06:06.637970 7f5c973a57a0 20 read_log 193305'15072 (0'0) promote f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by osd.4.0:225 2014-08-11 18:43:25.575445 2014-08-13 18:06:06.637982 7f5c973a57a0 20 read_log 193305'15073 (193305'15072) modify f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.951337.0:14058 2014-08-11 18:43:25.567009 2014-08-13 18:06:06.637995 7f5c973a57a0 20 read_log 193305'15074 (0'0) promote f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by osd.4.0:235 2014-08-11 18:43:28.966755 2014-08-13 18:06:06.638007 7f5c973a57a0 20 read_log 193305'15075 (193305'15074) modify f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.951337.0:14996 2014-08-11 18:43:28.959057 2014-08-13 18:06:06.638020 7f5c973a57a0 20 read_log 193305'15076 (0'0) promote 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by osd.4.0:237 2014-08-11 18:43:29.100651 2014-08-13 18:06:06.638032 7f5c973a57a0 20 read_log 193305'15077 (193305'15076) modify 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.951337.0:15037 2014-08-11 18:43:29.093778 2014-08-13 18:06:06.638044 7f5c973a57a0 20 read_log 193305'15078 (0'0) promote 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by osd.4.0:257 2014-08-11 18:43:32.439417 2014-08-13 18:06:06.638056 7f5c973a57a0 20 read_log 193305'15079 (193305'15078) modify 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.951337.0:16009 2014-08-11 18:43:32.430312 2014-08-13 18:06:06.638069 7f5c973a57a0 20 read_log 193305'15080 (0'0) promote decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by osd.4.0:271 2014-08-11 18:43:34.153950 2014-08-13 18:06:06.638081 7f5c973a57a0 20 read_log 193305'15081 (193305'15080) modify decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.951337.0:16488 2014-08-11 18:43:34.145672 2014-08-13 18:06:06.638093 7f5c973a57a0 20 read_log 193305'15082 (0'0) promote fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by osd.4.0:275 2014-08-11 18:43:34.439904 2014-08-13 18:06:06.638106 7f5c973a57a0 20 read_log 193305'15083 (193305'15082) modify fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.951337.0:16575 2014-08-11 18:43:34.431607 2014-08-13 18:06:06.638125 7f5c973a57a0 20 read_log 193305'15084 (0'0) promote e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by osd.4.0:281 2014-08-11 18:43:35.749899 2014-08-13 18:06:06.638141 7f5c973a57a0 20 read_log 193305'15085 (193305'15084) modify e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.951337.0:16939 2014-08-11 18:43:35.741388 2014-08-13 18:06:06.638161 7f5c973a57a0 20 read_log 193305'15086 (0'0) promote 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by osd.4.0:287 2014-08-11 18:43:36.540813 2014-08-13 18:06:06.638174 7f5c973a57a0 20 read_log 193305'15087 (193305'15086) modify 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.951337.0:17167 2014-08-11 18:43:36.533583 2014-08-13 18:06:06.638186 7f5c973a57a0 20 read_log 193305'15088 (0'0) promote dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by osd.4.0:309 2014-08-11 18:43:40.824344 2014-08-13 18:06:06.638199 7f5c973a57a0 20 read_log 193305'15089 (193305'15088) modify dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.951337.0:18410 2014-08-11 18:43:40.814504 2014-08-13 18:06:06.638211 7f5c973a57a0 20 read_log 193305'15090 (0'0) promote 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by osd.4.0:323 2014-08-11 18:43:43.467498 2014-08-13 18:06:06.638223 7f5c973a57a0 20 read_log 193305'15091 (193305'15090) modify 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.951337.0:19129 2014-08-11 18:43:43.459675 2014-08-13 18:06:06.638237 7f5c973a57a0 20 read_log 193305'15092 (0'0) promote f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by osd.4.0:369 2014-08-11 18:43:54.405738 2014-08-13 18:06:06.638250 7f5c973a57a0 20 read_log 193305'15093 (193305'15092) modify f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.951337.0:22155 2014-08-11 18:43:54.397798 2014-08-13 18:06:06.638264 7f5c973a57a0 20 read_log 193305'15094 (0'0) promote 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by osd.4.0:377 2014-08-11 18:43:55.799629 2014-08-13 18:06:06.638277 7f5c973a57a0 20 read_log 193305'15095 (193305'15094) modify 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.951337.0:22539 2014-08-11 18:43:55.791744 2014-08-13 18:06:06.638293 7f5c973a57a0 20 read_log 193305'15096 (0'0) promote 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by osd.4.0:405 2014-08-11 18:44:02.955808 2014-08-13 18:06:06.638308 7f5c973a57a0 20 read_log 193305'15097 (193305'15096) modify 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.951337.0:24534 2014-08-11 18:44:02.948465 2014-08-13 18:06:06.638321 7f5c973a57a0 20 read_log 193305'15098 (0'0) promote 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by osd.4.0:411 2014-08-11 18:44:04.970222 2014-08-13 18:06:06.638337 7f5c973a57a0 20 read_log 193305'15099 (193305'15098) modify 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.951337.0:25085 2014-08-11 18:44:04.962756 2014-08-13 18:06:06.638352 7f5c973a57a0 20 read_log 193305'15100 (0'0) promote 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by osd.4.0:437 2014-08-11 18:44:10.781459 2014-08-13 18:06:06.638365 7f5c973a57a0 20 read_log 193305'15101 (193305'15100) modify 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.951337.0:26663 2014-08-11 18:44:10.773653 2014-08-13 18:06:06.638377 7f5c973a57a0 20 read_log 193305'15102 (0'0) promote 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by osd.4.0:461 2014-08-11 18:44:16.017878 2014-08-13 18:06:06.638392 7f5c973a57a0 20 read_log 193305'15103 (193305'15102) modify 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.951337.0:28103 2014-08-11 18:44:16.009650 2014-08-13 18:06:06.638428 7f5c973a57a0 20 read_log 193305'15104 (0'0) promote 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by osd.4.0:475 2014-08-11 18:44:18.798250 2014-08-13 18:06:06.638445 7f5c973a57a0 20 read_log 193305'15105 (193305'15104) modify 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.951337.0:28847 2014-08-11 18:44:18.790590 2014-08-13 18:06:06.638458 7f5c973a57a0 20 read_log 193305'15106 (0'0) promote fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by osd.4.0:483 2014-08-11 18:44:20.239013 2014-08-13 18:06:06.638480 7f5c973a57a0 20 read_log 193305'15107 (193305'15106) modify fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.951337.0:29247 2014-08-11 18:44:20.232618 2014-08-13 18:06:06.638512 7f5c973a57a0 20 read_log 193305'15108 (0'0) promote ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by osd.4.0:493 2014-08-11 18:44:22.926795 2014-08-13 18:06:06.638527 7f5c973a57a0 20 read_log 193305'15109 (193305'15108) modify ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.951337.0:29981 2014-08-11 18:44:22.918542 2014-08-13 18:06:06.638540 7f5c973a57a0 20 read_log 193305'15110 (0'0) promote f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by osd.4.0:501 2014-08-11 18:44:26.037398 2014-08-13 18:06:06.638553 7f5c973a57a0 20 read_log 193305'15111 (193305'15110) modify f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.951337.0:30780 2014-08-11 18:44:26.029501 2014-08-13 18:06:06.638566 7f5c973a57a0 20 read_log 193305'15112 (0'0) promote 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by osd.4.0:521 2014-08-11 18:44:29.761906 2014-08-13 18:06:06.638578 7f5c973a57a0 20 read_log 193305'15113 (193305'15112) modify 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.951337.0:31849 2014-08-11 18:44:29.754208 2014-08-13 18:06:06.638591 7f5c973a57a0 20 read_log 193305'15114 (0'0) promote f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by osd.4.0:551 2014-08-11 18:44:32.107362 2014-08-13 18:06:06.638603 7f5c973a57a0 20 read_log 193305'15115 (193305'15114) modify f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.951337.0:32507 2014-08-11 18:44:32.100040 2014-08-13 18:06:06.638615 7f5c973a57a0 20 read_log 193305'15116 (193305'15047) modify 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.951337.0:33959 2014-08-11 18:44:34.736096 2014-08-13 18:06:06.638628 7f5c973a57a0 20 read_log 193305'15117 (193305'15049) modify e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.951337.0:35069 2014-08-11 18:44:35.361204 2014-08-13 18:06:06.638640 7f5c973a57a0 20 read_log 193305'15118 (193305'15051) modify d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.951337.0:35174 2014-08-11 18:44:35.446878 2014-08-13 18:06:06.638653 7f5c973a57a0 20 read_log 193305'15119 (193305'15053) modify 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.951337.0:35547 2014-08-11 18:44:35.651710 2014-08-13 18:06:06.638675 7f5c973a57a0 20 read_log 193305'15120 (193305'15055) modify 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.951337.0:35814 2014-08-11 18:44:35.757083 2014-08-13 18:06:06.638693 7f5c973a57a0 20 read_log 193305'15121 (193305'15057) modify 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.951337.0:37170 2014-08-11 18:44:36.745932 2014-08-13 18:06:06.638706 7f5c973a57a0 20 read_log 193305'15122 (193305'15059) modify 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.951337.0:37481 2014-08-11 18:44:36.933476 2014-08-13 18:06:06.638719 7f5c973a57a0 20 read_log 193305'15123 (193305'15061) modify 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.951337.0:37797 2014-08-11 18:44:37.064962 2014-08-13 18:06:06.638739 7f5c973a57a0 20 read_log 193305'15124 (193305'15063) modify b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.951337.0:38392 2014-08-11 18:44:37.392796 2014-08-13 18:06:06.638754 7f5c973a57a0 20 read_log 193305'15125 (193305'15065) modify 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.951337.0:40226 2014-08-11 18:44:38.341507 2014-08-13 18:06:06.638766 7f5c973a57a0 20 read_log 193305'15126 (193305'15067) modify 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.951337.0:41121 2014-08-11 18:44:38.641525 2014-08-13 18:06:06.638779 7f5c973a57a0 20 read_log 193305'15127 (193305'15069) modify 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.951337.0:41753 2014-08-11 18:44:38.861102 2014-08-13 18:06:06.638791 7f5c973a57a0 20 read_log 193305'15128 (193305'15071) modify e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.951337.0:42249 2014-08-11 18:44:39.163319 2014-08-13 18:06:06.638809 7f5c973a57a0 20 read_log 193305'15129 (193305'15073) modify f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.951337.0:46769 2014-08-11 18:44:40.584033 2014-08-13 18:06:06.638822 7f5c973a57a0 20 read_log 193305'15130 (193305'15075) modify f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.951337.0:47707 2014-08-11 18:44:40.843160 2014-08-13 18:06:06.638835 7f5c973a57a0 20 read_log 193305'15131 (193305'15077) modify 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.951337.0:47748 2014-08-11 18:44:40.854987 2014-08-13 18:06:06.638847 7f5c973a57a0 20 read_log 193305'15132 (193305'15079) modify 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.951337.0:48720 2014-08-11 18:44:41.135222 2014-08-13 18:06:06.638859 7f5c973a57a0 20 read_log 193305'15133 (193305'15081) modify decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.951337.0:49199 2014-08-11 18:44:41.287105 2014-08-13 18:06:06.638872 7f5c973a57a0 20 read_log 193305'15134 (193305'15083) modify fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.951337.0:49286 2014-08-11 18:44:41.320341 2014-08-13 18:06:06.638887 7f5c973a57a0 20 read_log 193305'15135 (193305'15085) modify e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.951337.0:49650 2014-08-11 18:44:41.428024 2014-08-13 18:06:06.638899 7f5c973a57a0 20 read_log 193305'15136 (193305'15087) modify 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.951337.0:49878 2014-08-11 18:44:41.504342 2014-08-13 18:06:06.638912 7f5c973a57a0 20 read_log 193305'15137 (193305'15089) modify dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.951337.0:51121 2014-08-11 18:44:41.858596 2014-08-13 18:06:06.638933 7f5c973a57a0 20 read_log 193305'15138 (193305'15091) modify 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.951337.0:51840 2014-08-11 18:44:42.062841 2014-08-13 18:06:06.638949 7f5c973a57a0 20 read_log 193305'15139 (193305'15093) modify f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.951337.0:54866 2014-08-11 18:44:42.960800 2014-08-13 18:06:06.638977 7f5c973a57a0 20 read_log 193305'15140 (193305'15095) modify 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.951337.0:55250 2014-08-11 18:44:43.068617 2014-08-13 18:06:06.638993 7f5c973a57a0 20 read_log 193305'15141 (193305'15097) modify 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.951337.0:57245 2014-08-11 18:44:43.688975 2014-08-13 18:06:06.639006 7f5c973a57a0 20 read_log 193305'15142 (193305'15099) modify 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.951337.0:57796 2014-08-11 18:44:43.862502 2014-08-13 18:06:06.639030 7f5c973a57a0 20 read_log 193305'15143 (193305'15101) modify 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.951337.0:59374 2014-08-11 18:44:44.309973 2014-08-13 18:06:06.639053 7f5c973a57a0 20 read_log 193305'15144 (193305'15103) modify 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.951337.0:60814 2014-08-11 18:44:44.742621 2014-08-13 18:06:06.639068 7f5c973a57a0 20 read_log 193305'15145 (193305'15105) modify 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.951337.0:61558 2014-08-11 18:44:44.962707 2014-08-13 18:06:06.639084 7f5c973a57a0 20 read_log 193305'15146 (193305'15107) modify fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.951337.0:61958 2014-08-11 18:44:45.085374 2014-08-13 18:06:06.639109 7f5c973a57a0 20 read_log 193305'15147 (193305'15109) modify ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.951337.0:62692 2014-08-11 18:44:45.301193 2014-08-13 18:06:06.639122 7f5c973a57a0 20 read_log 193305'15148 (193305'15111) modify f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.951337.0:63491 2014-08-11 18:44:45.522995 2014-08-13 18:06:06.639135 7f5c973a57a0 20 read_log 193305'15149 (193305'15113) modify 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.951337.0:64560 2014-08-11 18:44:45.854132 2014-08-13 18:06:06.639158 7f5c973a57a0 20 read_log 193305'15150 (193305'15115) modify f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.951337.0:65218 2014-08-11 18:44:46.053440 2014-08-13 18:06:06.639174 7f5c973a57a0 20 read_log 194215'15151 (194215'15151) modify b6/hit_set_24.b6_archive_2014-08-12 15:32:25.864388_2014-08-12 15:38:57.656047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:57.656323 2014-08-13 18:06:06.639195 7f5c973a57a0 20 read_log 194215'15152 (193305'15044) delete b6/hit_set_24.b6_archive_2014-08-11 18:37:04.909790_2014-08-11 18:42:37.869502/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:38:57.656323 2014-08-13 18:06:06.639211 7f5c973a57a0 20 read_log 194215'15153 (192888'15028) clean edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by osd.4.0:1403 2014-08-12 15:38:57.668821 2014-08-13 18:06:06.639226 7f5c973a57a0 20 read_log 194215'15154 (194215'15153) delete edde20b6/benchmark_data_storage0111-ib.pocstor_2228_object25323/head//24 by client.2174595.0:37141 0.000000 2014-08-13 18:06:06.639238 7f5c973a57a0 20 read_log 194215'15155 (192888'15015) clean e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by osd.4.0:1405 2014-08-12 15:38:57.678186 2014-08-13 18:06:06.639250 7f5c973a57a0 20 read_log 194215'15156 (194215'15155) delete e13330b6/benchmark_data_storage0111-ib.pocstor_2228_object19525/head//24 by client.2174595.0:37143 0.000000 2014-08-13 18:06:06.639265 7f5c973a57a0 20 read_log 194215'15157 (193305'15117) clean e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by osd.4.0:1407 2014-08-12 15:38:57.686239 2014-08-13 18:06:06.639277 7f5c973a57a0 20 read_log 194215'15158 (194215'15157) delete e44540b6/benchmark_data_storage0111-ib.pocstor_62621_object2357/head//24 by client.2174595.0:37145 0.000000 2014-08-13 18:06:06.639288 7f5c973a57a0 20 read_log 194215'15159 (193305'15118) clean d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by osd.4.0:1409 2014-08-12 15:38:57.695490 2014-08-13 18:06:06.639301 7f5c973a57a0 20 read_log 194215'15160 (194215'15159) delete d1650b6/benchmark_data_storage0111-ib.pocstor_62621_object2462/head//24 by client.2174595.0:37147 0.000000 2014-08-13 18:06:06.639315 7f5c973a57a0 20 read_log 194215'15161 (192888'14864) clean 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by osd.4.0:1411 2014-08-12 15:38:57.703466 2014-08-13 18:06:06.639329 7f5c973a57a0 20 read_log 194215'15162 (194215'15161) delete 841160b6/benchmark_data_storage0111-ib.pocstor_41097_object29170/head//24 by client.2174595.0:37149 0.000000 2014-08-13 18:06:06.639360 7f5c973a57a0 20 read_log 194215'15163 (192888'15002) clean a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by osd.4.0:1413 2014-08-12 15:38:57.711308 2014-08-13 18:06:06.639379 7f5c973a57a0 20 read_log 194215'15164 (194215'15163) delete a36360b6/benchmark_data_storage0111-ib.pocstor_2228_object12248/head//24 by client.2174595.0:37151 0.000000 2014-08-13 18:06:06.639390 7f5c973a57a0 20 read_log 194215'15165 (192888'14994) clean 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by osd.4.0:1415 2014-08-12 15:38:57.719804 2014-08-13 18:06:06.639414 7f5c973a57a0 20 read_log 194215'15166 (194215'15165) delete 4b2d60b6/benchmark_data_storage0111-ib.pocstor_2228_object5585/head//24 by client.2174595.0:37153 0.000000 2014-08-13 18:06:06.639429 7f5c973a57a0 20 read_log 194215'15167 (193305'15133) clean decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by osd.4.0:1417 2014-08-12 15:38:57.747336 2014-08-13 18:06:06.639442 7f5c973a57a0 20 read_log 194215'15168 (194215'15167) delete decf70b6/benchmark_data_storage0111-ib.pocstor_62621_object16487/head//24 by client.2174595.0:37155 0.000000 2014-08-13 18:06:06.639453 7f5c973a57a0 20 read_log 194215'15169 (192888'14824) clean f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by osd.4.0:1419 2014-08-12 15:38:57.755711 2014-08-13 18:06:06.639465 7f5c973a57a0 20 read_log 194215'15170 (194215'15169) delete f4fa90b6/benchmark_data_storage0111-ib.pocstor_41097_object8626/head//24 by client.2174595.0:37157 0.000000 2014-08-13 18:06:06.639487 7f5c973a57a0 20 read_log 194215'15171 (193305'15136) clean 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by osd.4.0:1421 2014-08-12 15:38:57.765143 2014-08-13 18:06:06.639501 7f5c973a57a0 20 read_log 194215'15172 (194215'15171) delete 356e90b6/benchmark_data_storage0111-ib.pocstor_62621_object17166/head//24 by client.2174595.0:37159 0.000000 2014-08-13 18:06:06.639512 7f5c973a57a0 20 read_log 194215'15173 (192888'15030) clean 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by osd.4.0:1423 2014-08-12 15:38:57.774522 2014-08-13 18:06:06.639524 7f5c973a57a0 20 read_log 194215'15174 (194215'15173) delete 81aa0b6/benchmark_data_storage0111-ib.pocstor_2228_object26737/head//24 by client.2174595.0:37161 0.000000 2014-08-13 18:06:06.639535 7f5c973a57a0 20 read_log 194215'15175 (193305'15123) clean 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by osd.4.0:1425 2014-08-12 15:38:57.783729 2014-08-13 18:06:06.639548 7f5c973a57a0 20 read_log 194215'15176 (194215'15175) delete 5e6ba0b6/benchmark_data_storage0111-ib.pocstor_62621_object5085/head//24 by client.2174595.0:37163 0.000000 2014-08-13 18:06:06.639559 7f5c973a57a0 20 read_log 194215'15177 (192888'15034) clean 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by osd.4.0:1427 2014-08-12 15:38:57.792408 2014-08-13 18:06:06.639571 7f5c973a57a0 20 read_log 194215'15178 (194215'15177) delete 5eb0b0b6/benchmark_data_storage0111-ib.pocstor_2228_object27874/head//24 by client.2174595.0:37165 0.000000 2014-08-13 18:06:06.639582 7f5c973a57a0 20 read_log 194215'15179 (192888'14823) clean aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by osd.4.0:1429 2014-08-12 15:38:57.801706 2014-08-13 18:06:06.639594 7f5c973a57a0 20 read_log 194215'15180 (194215'15179) delete aaa2b0b6/benchmark_data_storage0111-ib.pocstor_41097_object6276/head//24 by client.2174595.0:37167 0.000000 2014-08-13 18:06:06.639605 7f5c973a57a0 20 read_log 194215'15181 (192888'14833) clean 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by osd.4.0:1431 2014-08-12 15:38:57.809961 2014-08-13 18:06:06.639618 7f5c973a57a0 20 read_log 194215'15182 (194215'15181) delete 2b73b0b6/benchmark_data_storage0111-ib.pocstor_41097_object13529/head//24 by client.2174595.0:37169 0.000000 2014-08-13 18:06:06.639636 7f5c973a57a0 20 read_log 194215'15183 (193305'15131) clean 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by osd.4.0:1433 2014-08-12 15:38:57.818730 2014-08-13 18:06:06.639649 7f5c973a57a0 20 read_log 194215'15184 (194215'15183) delete 8151c0b6/benchmark_data_storage0111-ib.pocstor_62621_object15036/head//24 by client.2174595.0:37171 0.000000 2014-08-13 18:06:06.639660 7f5c973a57a0 20 read_log 194215'15185 (193305'15138) clean 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by osd.4.0:1435 2014-08-12 15:38:57.827425 2014-08-13 18:06:06.639673 7f5c973a57a0 20 read_log 194215'15186 (194215'15185) delete 233bc0b6/benchmark_data_storage0111-ib.pocstor_62621_object19128/head//24 by client.2174595.0:37173 0.000000 2014-08-13 18:06:06.639684 7f5c973a57a0 20 read_log 194215'15187 (192888'15007) clean 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by osd.4.0:1437 2014-08-12 15:38:57.835508 2014-08-13 18:06:06.639696 7f5c973a57a0 20 read_log 194215'15188 (194215'15187) delete 3fc6d0b6/benchmark_data_storage0111-ib.pocstor_2228_object14814/head//24 by client.2174595.0:37175 0.000000 2014-08-13 18:06:06.639707 7f5c973a57a0 20 read_log 194215'15189 (192888'14865) clean 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by osd.4.0:1439 2014-08-12 15:38:57.876942 2014-08-13 18:06:06.639741 7f5c973a57a0 20 read_log 194215'15190 (194215'15189) delete 3b27d0b6/benchmark_data_storage0111-ib.pocstor_41097_object29532/head//24 by client.2174595.0:37177 0.000000 2014-08-13 18:06:06.639753 7f5c973a57a0 20 read_log 194215'15191 (193305'15129) clean f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by osd.4.0:1441 2014-08-12 15:38:57.886338 2014-08-13 18:06:06.639765 7f5c973a57a0 20 read_log 194215'15192 (194215'15191) delete f2fef0b6/benchmark_data_storage0111-ib.pocstor_62621_object14057/head//24 by client.2174595.0:37179 0.000000 2014-08-13 18:06:06.639777 7f5c973a57a0 20 read_log 194215'15193 (192888'14988) clean d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by osd.4.0:1443 2014-08-12 15:38:57.895832 2014-08-13 18:06:06.639789 7f5c973a57a0 20 read_log 194215'15194 (194215'15193) delete d7f304b6/benchmark_data_storage0111-ib.pocstor_2228_object2669/head//24 by client.2174595.0:37181 0.000000 2014-08-13 18:06:06.639800 7f5c973a57a0 20 read_log 194215'15195 (193305'15124) clean b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by osd.4.0:1445 2014-08-12 15:38:57.904759 2014-08-13 18:06:06.639812 7f5c973a57a0 20 read_log 194215'15196 (194215'15195) delete b65d04b6/benchmark_data_storage0111-ib.pocstor_62621_object5680/head//24 by client.2174595.0:37183 0.000000 2014-08-13 18:06:06.639823 7f5c973a57a0 20 read_log 194215'15197 (192888'14814) clean 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by osd.4.0:1447 2014-08-12 15:38:57.914716 2014-08-13 18:06:06.639836 7f5c973a57a0 20 read_log 194215'15198 (194215'15197) delete 206614b6/benchmark_data_storage0111-ib.pocstor_41097_object2682/head//24 by client.2174595.0:37185 0.000000 2014-08-13 18:06:06.639847 7f5c973a57a0 20 read_log 194215'15199 (193305'15127) clean 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by osd.4.0:1449 2014-08-12 15:38:57.923559 2014-08-13 18:06:06.639859 7f5c973a57a0 20 read_log 194215'15200 (194215'15199) delete 9ac244b6/benchmark_data_storage0111-ib.pocstor_62621_object9041/head//24 by client.2174595.0:37187 0.000000 2014-08-13 18:06:06.639871 7f5c973a57a0 20 read_log 194215'15201 (192888'15001) clean 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by osd.4.0:1451 2014-08-12 15:38:57.932098 2014-08-13 18:06:06.639886 7f5c973a57a0 20 read_log 194215'15202 (194215'15201) delete 5e3174b6/benchmark_data_storage0111-ib.pocstor_2228_object11464/head//24 by client.2174595.0:37189 0.000000 2014-08-13 18:06:06.639924 7f5c973a57a0 20 read_log 194215'15203 (192888'14995) clean 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by osd.4.0:1453 2014-08-12 15:38:57.941184 2014-08-13 18:06:06.639940 7f5c973a57a0 20 read_log 194215'15204 (194215'15203) delete 1cb584b6/benchmark_data_storage0111-ib.pocstor_2228_object6257/head//24 by client.2174595.0:37191 0.000000 2014-08-13 18:06:06.639952 7f5c973a57a0 20 read_log 194215'15205 (192888'15032) clean 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by osd.4.0:1455 2014-08-12 15:38:57.951395 2014-08-13 18:06:06.639979 7f5c973a57a0 20 read_log 194215'15206 (194215'15205) delete 4c0684b6/benchmark_data_storage0111-ib.pocstor_2228_object27425/head//24 by client.2174595.0:37193 0.000000 2014-08-13 18:06:06.639993 7f5c973a57a0 20 read_log 194215'15207 (192888'14839) clean 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by osd.4.0:1457 2014-08-12 15:38:57.960463 2014-08-13 18:06:06.640005 7f5c973a57a0 20 read_log 194215'15208 (194215'15207) delete 628394b6/benchmark_data_storage0111-ib.pocstor_41097_object17761/head//24 by client.2174595.0:37195 0.000000 2014-08-13 18:06:06.640016 7f5c973a57a0 20 read_log 194215'15209 (193305'15119) clean 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by osd.4.0:1459 2014-08-12 15:38:57.970213 2014-08-13 18:06:06.640031 7f5c973a57a0 20 read_log 194215'15210 (194215'15209) delete 9b2994b6/benchmark_data_storage0111-ib.pocstor_62621_object2835/head//24 by client.2174595.0:37197 0.000000 2014-08-13 18:06:06.640042 7f5c973a57a0 20 read_log 194215'15211 (192888'14853) clean fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by osd.4.0:1461 2014-08-12 15:38:57.978808 2014-08-13 18:06:06.640055 7f5c973a57a0 20 read_log 194215'15212 (194215'15211) delete fe4d94b6/benchmark_data_storage0111-ib.pocstor_41097_object23821/head//24 by client.2174595.0:37199 0.000000 2014-08-13 18:06:06.640067 7f5c973a57a0 20 read_log 194215'15213 (192888'15026) clean d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by osd.4.0:1463 2014-08-12 15:38:57.985882 2014-08-13 18:06:06.640080 7f5c973a57a0 20 read_log 194215'15214 (194215'15213) delete d7e4b4b6/benchmark_data_storage0111-ib.pocstor_2228_object24706/head//24 by client.2174595.0:37201 0.000000 2014-08-13 18:06:06.640093 7f5c973a57a0 20 read_log 194215'15215 (193305'15145) clean 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by osd.4.0:1465 2014-08-12 15:38:57.993755 2014-08-13 18:06:06.640106 7f5c973a57a0 20 read_log 194215'15216 (194215'15215) delete 42f7b4b6/benchmark_data_storage0111-ib.pocstor_62621_object28846/head//24 by client.2174595.0:37203 0.000000 2014-08-13 18:06:06.640117 7f5c973a57a0 20 read_log 194215'15217 (192888'15009) clean 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by osd.4.0:1467 2014-08-12 15:38:58.002083 2014-08-13 18:06:06.640130 7f5c973a57a0 20 read_log 194215'15218 (194215'15217) delete 9ba9b4b6/benchmark_data_storage0111-ib.pocstor_2228_object17347/head//24 by client.2174595.0:37205 0.000000 2014-08-13 18:06:06.640141 7f5c973a57a0 20 read_log 194215'15219 (193305'15140) clean 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by osd.4.0:1469 2014-08-12 15:38:58.009976 2014-08-13 18:06:06.640161 7f5c973a57a0 20 read_log 194215'15220 (194215'15219) delete 5142c4b6/benchmark_data_storage0111-ib.pocstor_62621_object22538/head//24 by client.2174595.0:37207 0.000000 2014-08-13 18:06:06.640173 7f5c973a57a0 20 read_log 194215'15221 (193305'15134) clean fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by osd.4.0:1471 2014-08-12 15:38:58.018429 2014-08-13 18:06:06.640186 7f5c973a57a0 20 read_log 194215'15222 (194215'15221) delete fe8fd4b6/benchmark_data_storage0111-ib.pocstor_62621_object16574/head//24 by client.2174595.0:37209 0.000000 2014-08-13 18:06:06.640211 7f5c973a57a0 20 read_log 194215'15223 (193305'15150) clean f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by osd.4.0:1473 2014-08-12 15:38:58.026679 2014-08-13 18:06:06.640227 7f5c973a57a0 20 read_log 194215'15224 (194215'15223) delete f38cf4b6/benchmark_data_storage0111-ib.pocstor_62621_object32506/head//24 by client.2174595.0:37211 0.000000 2014-08-13 18:06:06.640239 7f5c973a57a0 20 read_log 194215'15225 (192888'15020) clean 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by osd.4.0:1475 2014-08-12 15:38:58.035374 2014-08-13 18:06:06.640253 7f5c973a57a0 20 read_log 194215'15226 (194215'15225) delete 8ff008b6/benchmark_data_storage0111-ib.pocstor_2228_object21789/head//24 by client.2174595.0:37213 0.000000 2014-08-13 18:06:06.640265 7f5c973a57a0 20 read_log 194215'15227 (192888'15031) clean 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by osd.4.0:1477 2014-08-12 15:38:58.043817 2014-08-13 18:06:06.640285 7f5c973a57a0 20 read_log 194215'15228 (194215'15227) delete 3ca808b6/benchmark_data_storage0111-ib.pocstor_2228_object27075/head//24 by client.2174595.0:37215 0.000000 2014-08-13 18:06:06.640297 7f5c973a57a0 20 read_log 194215'15229 (192888'15018) clean 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by osd.4.0:1479 2014-08-12 15:38:58.053127 2014-08-13 18:06:06.640309 7f5c973a57a0 20 read_log 194215'15230 (194215'15229) delete 484c08b6/benchmark_data_storage0111-ib.pocstor_2228_object21127/head//24 by client.2174595.0:37217 0.000000 2014-08-13 18:06:06.640323 7f5c973a57a0 20 read_log 194215'15231 (193305'15143) clean 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by osd.4.0:1481 2014-08-12 15:38:58.063224 2014-08-13 18:06:06.640336 7f5c973a57a0 20 read_log 194215'15232 (194215'15231) delete 39fc08b6/benchmark_data_storage0111-ib.pocstor_62621_object26662/head//24 by client.2174595.0:37219 0.000000 2014-08-13 18:06:06.640348 7f5c973a57a0 20 read_log 194215'15233 (193305'15148) clean f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by osd.4.0:1483 2014-08-12 15:38:58.071336 2014-08-13 18:06:06.640360 7f5c973a57a0 20 read_log 194215'15234 (194215'15233) delete f6c218b6/benchmark_data_storage0111-ib.pocstor_62621_object30779/head//24 by client.2174595.0:37221 0.000000 2014-08-13 18:06:06.640374 7f5c973a57a0 20 read_log 194215'15235 (193305'15142) clean 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by osd.4.0:1485 2014-08-12 15:38:58.079702 2014-08-13 18:06:06.640389 7f5c973a57a0 20 read_log 194215'15236 (194215'15235) delete 55ca28b6/benchmark_data_storage0111-ib.pocstor_62621_object25084/head//24 by client.2174595.0:37223 0.000000 2014-08-13 18:06:06.640414 7f5c973a57a0 20 read_log 194215'15237 (192888'15025) clean ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by osd.4.0:1487 2014-08-12 15:38:58.087507 2014-08-13 18:06:06.640434 7f5c973a57a0 20 read_log 194215'15238 (194215'15237) delete ce4b28b6/benchmark_data_storage0111-ib.pocstor_2228_object24042/head//24 by client.2174595.0:37225 0.000000 2014-08-13 18:06:06.640464 7f5c973a57a0 20 read_log 194215'15239 (192888'15008) clean 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by osd.4.0:1489 2014-08-12 15:38:58.096135 2014-08-13 18:06:06.640481 7f5c973a57a0 20 read_log 194215'15240 (194215'15239) delete 9f1238b6/benchmark_data_storage0111-ib.pocstor_2228_object16964/head//24 by client.2174595.0:37227 0.000000 2014-08-13 18:06:06.640492 7f5c973a57a0 20 read_log 194215'15241 (192888'14861) clean 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by osd.4.0:1491 2014-08-12 15:38:58.104188 2014-08-13 18:06:06.640515 7f5c973a57a0 20 read_log 194215'15242 (194215'15241) delete 975d58b6/benchmark_data_storage0111-ib.pocstor_41097_object26930/head//24 by client.2174595.0:37229 0.000000 2014-08-13 18:06:06.640534 7f5c973a57a0 20 read_log 194215'15243 (193305'15139) clean f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by osd.4.0:1493 2014-08-12 15:38:58.113005 2014-08-13 18:06:06.640555 7f5c973a57a0 20 read_log 194215'15244 (194215'15243) delete f21078b6/benchmark_data_storage0111-ib.pocstor_62621_object22154/head//24 by client.2174595.0:37231 0.000000 2014-08-13 18:06:06.640567 7f5c973a57a0 20 read_log 194215'15245 (192888'14843) clean 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by osd.4.0:1495 2014-08-12 15:38:58.122612 2014-08-13 18:06:06.640580 7f5c973a57a0 20 read_log 194215'15246 (194215'15245) delete 181578b6/benchmark_data_storage0111-ib.pocstor_41097_object19051/head//24 by client.2174595.0:37233 0.000000 2014-08-13 18:06:06.640591 7f5c973a57a0 20 read_log 194215'15247 (193305'15137) clean dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by osd.4.0:1497 2014-08-12 15:38:58.131445 2014-08-13 18:06:06.640607 7f5c973a57a0 20 read_log 194215'15248 (194215'15247) delete dbf388b6/benchmark_data_storage0111-ib.pocstor_62621_object18409/head//24 by client.2174595.0:37235 0.000000 2014-08-13 18:06:06.640618 7f5c973a57a0 20 read_log 194215'15249 (193305'15125) clean 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by osd.4.0:1499 2014-08-12 15:38:58.139631 2014-08-13 18:06:06.640631 7f5c973a57a0 20 read_log 194215'15250 (194215'15249) delete 982888b6/benchmark_data_storage0111-ib.pocstor_62621_object7514/head//24 by client.2174595.0:37237 0.000000 2014-08-13 18:06:06.640642 7f5c973a57a0 20 read_log 194215'15251 (193305'15128) clean e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by osd.4.0:1501 2014-08-12 15:38:58.174456 2014-08-13 18:06:06.640661 7f5c973a57a0 20 read_log 194215'15252 (194215'15251) delete e70698b6/benchmark_data_storage0111-ib.pocstor_62621_object9537/head//24 by client.2174595.0:37239 0.000000 2014-08-13 18:06:06.640674 7f5c973a57a0 20 read_log 194215'15253 (192888'15016) clean 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by osd.4.0:1503 2014-08-12 15:38:58.182506 2014-08-13 18:06:06.640686 7f5c973a57a0 20 read_log 194215'15254 (194215'15253) delete 72cfb8b6/benchmark_data_storage0111-ib.pocstor_2228_object19797/head//24 by client.2174595.0:37241 0.000000 2014-08-13 18:06:06.640697 7f5c973a57a0 20 read_log 194215'15255 (193305'15135) clean e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by osd.4.0:1505 2014-08-12 15:38:58.191665 2014-08-13 18:06:06.640710 7f5c973a57a0 20 read_log 194215'15256 (194215'15255) delete e962c8b6/benchmark_data_storage0111-ib.pocstor_62621_object16938/head//24 by client.2174595.0:37243 0.000000 2014-08-13 18:06:06.640723 7f5c973a57a0 20 read_log 194215'15257 (193305'15126) clean 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by osd.4.0:1507 2014-08-12 15:38:58.216603 2014-08-13 18:06:06.640736 7f5c973a57a0 20 read_log 194215'15258 (194215'15257) delete 88eac8b6/benchmark_data_storage0111-ib.pocstor_62621_object8409/head//24 by client.2174595.0:37245 0.000000 2014-08-13 18:06:06.640747 7f5c973a57a0 20 read_log 194215'15259 (193305'15122) clean 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by osd.4.0:1509 2014-08-12 15:38:58.226718 2014-08-13 18:06:06.640759 7f5c973a57a0 20 read_log 194215'15260 (194215'15259) delete 67ffc8b6/benchmark_data_storage0111-ib.pocstor_62621_object4769/head//24 by client.2174595.0:37247 0.000000 2014-08-13 18:06:06.640770 7f5c973a57a0 20 read_log 194215'15261 (193305'15146) clean fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by osd.4.0:1511 2014-08-12 15:38:58.236759 2014-08-13 18:06:06.640783 7f5c973a57a0 20 read_log 194215'15262 (194215'15261) delete fac2d8b6/benchmark_data_storage0111-ib.pocstor_62621_object29246/head//24 by client.2174595.0:37249 0.000000 2014-08-13 18:06:06.640801 7f5c973a57a0 20 read_log 194215'15263 (193305'15130) clean f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by osd.4.0:1513 2014-08-12 15:38:58.245829 2014-08-13 18:06:06.640813 7f5c973a57a0 20 read_log 194215'15264 (194215'15263) delete f55dd8b6/benchmark_data_storage0111-ib.pocstor_62621_object14995/head//24 by client.2174595.0:37251 0.000000 2014-08-13 18:06:06.640825 7f5c973a57a0 20 read_log 194215'15265 (192888'14993) clean a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by osd.4.0:1515 2014-08-12 15:38:58.254554 2014-08-13 18:06:06.640837 7f5c973a57a0 20 read_log 194215'15266 (194215'15265) delete a535e8b6/benchmark_data_storage0111-ib.pocstor_2228_object4504/head//24 by client.2174595.0:37253 0.000000 2014-08-13 18:06:06.640848 7f5c973a57a0 20 read_log 194215'15267 (192888'14826) clean a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by osd.4.0:1517 2014-08-12 15:38:58.262537 2014-08-13 18:06:06.640860 7f5c973a57a0 20 read_log 194215'15268 (194215'15267) delete a3cae8b6/benchmark_data_storage0111-ib.pocstor_41097_object9815/head//24 by client.2174595.0:37255 0.000000 2014-08-13 18:06:06.640871 7f5c973a57a0 20 read_log 194215'15269 (192888'14990) clean a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by osd.4.0:1519 2014-08-12 15:38:58.270226 2014-08-13 18:06:06.640883 7f5c973a57a0 20 read_log 194215'15270 (194215'15269) delete a62be8b6/benchmark_data_storage0111-ib.pocstor_2228_object3272/head//24 by client.2174595.0:37257 0.000000 2014-08-13 18:06:06.640894 7f5c973a57a0 20 read_log 194215'15271 (192888'14998) clean 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by osd.4.0:1521 2014-08-12 15:38:58.286524 2014-08-13 18:06:06.640906 7f5c973a57a0 20 read_log 194215'15272 (194215'15271) delete 429ee8b6/benchmark_data_storage0111-ib.pocstor_2228_object8616/head//24 by client.2174595.0:37259 0.000000 2014-08-13 18:06:06.640917 7f5c973a57a0 20 read_log 194215'15273 (192888'14816) clean 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by osd.4.0:1523 2014-08-12 15:38:58.294823 2014-08-13 18:06:06.640930 7f5c973a57a0 20 read_log 194215'15274 (194215'15273) delete 8fa3f8b6/benchmark_data_storage0111-ib.pocstor_41097_object4460/head//24 by client.2174595.0:37261 0.000000 2014-08-13 18:06:06.640940 7f5c973a57a0 20 read_log 194215'15275 (193305'15116) clean 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by osd.4.0:1525 2014-08-12 15:38:58.303280 2014-08-13 18:06:06.640953 7f5c973a57a0 20 read_log 194215'15276 (194215'15275) delete 12c3f8b6/benchmark_data_storage0111-ib.pocstor_62621_object1247/head//24 by client.2174595.0:37263 0.000000 2014-08-13 18:06:06.640964 7f5c973a57a0 20 read_log 194215'15277 (192888'14836) clean 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by osd.4.0:1527 2014-08-12 15:38:58.311520 2014-08-13 18:06:06.640976 7f5c973a57a0 20 read_log 194215'15278 (194215'15277) delete 62c7f8b6/benchmark_data_storage0111-ib.pocstor_41097_object15012/head//24 by client.2174595.0:37265 0.000000 2014-08-13 18:06:06.640987 7f5c973a57a0 20 read_log 194215'15279 (193305'15120) clean 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by osd.4.0:1529 2014-08-12 15:38:58.320115 2014-08-13 18:06:06.640999 7f5c973a57a0 20 read_log 194215'15280 (194215'15279) delete 3db8f8b6/benchmark_data_storage0111-ib.pocstor_62621_object3102/head//24 by client.2174595.0:37267 0.000000 2014-08-13 18:06:06.641013 7f5c973a57a0 20 read_log 194215'15281 (192888'14842) clean 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by osd.4.0:1531 2014-08-12 15:38:58.329161 2014-08-13 18:06:06.641026 7f5c973a57a0 20 read_log 194215'15282 (194215'15281) delete 1e8f8b6/benchmark_data_storage0111-ib.pocstor_41097_object18941/head//24 by client.2174595.0:37269 0.000000 2014-08-13 18:06:06.641043 7f5c973a57a0 20 read_log 194215'15283 (192888'14815) clean 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by osd.4.0:1533 2014-08-12 15:38:58.338138 2014-08-13 18:06:06.641058 7f5c973a57a0 20 read_log 194215'15284 (194215'15283) delete 7bf30cb6/benchmark_data_storage0111-ib.pocstor_41097_object4363/head//24 by client.2174595.0:37271 0.000000 2014-08-13 18:06:06.641069 7f5c973a57a0 20 read_log 194215'15285 (192888'14818) clean c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by osd.4.0:1535 2014-08-12 15:38:58.347261 2014-08-13 18:06:06.641082 7f5c973a57a0 20 read_log 194215'15286 (194215'15285) delete c5390cb6/benchmark_data_storage0111-ib.pocstor_41097_object4545/head//24 by client.2174595.0:37273 0.000000 2014-08-13 18:06:06.641094 7f5c973a57a0 20 read_log 194215'15287 (192888'14821) clean 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by osd.4.0:1537 2014-08-12 15:38:58.356845 2014-08-13 18:06:06.641108 7f5c973a57a0 20 read_log 194215'15288 (194215'15287) delete 95061cb6/benchmark_data_storage0111-ib.pocstor_41097_object5517/head//24 by client.2174595.0:37275 0.000000 2014-08-13 18:06:06.641119 7f5c973a57a0 20 read_log 194215'15289 (192888'14845) clean 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by osd.4.0:1539 2014-08-12 15:38:58.364730 2014-08-13 18:06:06.641133 7f5c973a57a0 20 read_log 194215'15290 (194215'15289) delete 8cb61cb6/benchmark_data_storage0111-ib.pocstor_41097_object20335/head//24 by client.2174595.0:37277 0.000000 2014-08-13 18:06:06.641144 7f5c973a57a0 20 read_log 194215'15291 (192888'15003) clean 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by osd.4.0:1541 2014-08-12 15:38:58.372851 2014-08-13 18:06:06.641167 7f5c973a57a0 20 read_log 194215'15292 (194215'15291) delete 31e71cb6/benchmark_data_storage0111-ib.pocstor_2228_object12380/head//24 by client.2174595.0:37279 0.000000 2014-08-13 18:06:06.641179 7f5c973a57a0 20 read_log 194215'15293 (192888'14838) clean 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by osd.4.0:1543 2014-08-12 15:38:58.380390 2014-08-13 18:06:06.641191 7f5c973a57a0 20 read_log 194215'15294 (194215'15293) delete 823b1cb6/benchmark_data_storage0111-ib.pocstor_41097_object17708/head//24 by client.2174595.0:37281 0.000000 2014-08-13 18:06:06.641204 7f5c973a57a0 20 read_log 194215'15295 (192888'15000) clean ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by osd.4.0:1545 2014-08-12 15:38:58.390547 2014-08-13 18:06:06.641217 7f5c973a57a0 20 read_log 194215'15296 (194215'15295) delete ec6d1cb6/benchmark_data_storage0111-ib.pocstor_2228_object10548/head//24 by client.2174595.0:37283 0.000000 2014-08-13 18:06:06.641230 7f5c973a57a0 20 read_log 194215'15297 (193305'15144) clean 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by osd.4.0:1547 2014-08-12 15:38:58.399686 2014-08-13 18:06:06.641243 7f5c973a57a0 20 read_log 194215'15298 (194215'15297) delete 9052cb6/benchmark_data_storage0111-ib.pocstor_62621_object28102/head//24 by client.2174595.0:37285 0.000000 2014-08-13 18:06:06.641274 7f5c973a57a0 20 read_log 194215'15299 (192888'14858) clean 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by osd.4.0:1549 2014-08-12 15:38:58.427096 2014-08-13 18:06:06.641290 7f5c973a57a0 20 read_log 194215'15300 (194215'15299) delete 8cea2cb6/benchmark_data_storage0111-ib.pocstor_41097_object24795/head//24 by client.2174595.0:37287 0.000000 2014-08-13 18:06:06.641301 7f5c973a57a0 20 read_log 194215'15301 (193305'15121) clean 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by osd.4.0:1551 2014-08-12 15:38:58.436635 2014-08-13 18:06:06.641316 7f5c973a57a0 20 read_log 194215'15302 (194215'15301) delete 205b2cb6/benchmark_data_storage0111-ib.pocstor_62621_object4458/head//24 by client.2174595.0:37289 0.000000 2014-08-13 18:06:06.641338 7f5c973a57a0 20 read_log 194215'15303 (192888'15023) clean 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by osd.4.0:1553 2014-08-12 15:38:58.445032 2014-08-13 18:06:06.641353 7f5c973a57a0 20 read_log 194215'15304 (194215'15303) delete 367b3cb6/benchmark_data_storage0111-ib.pocstor_2228_object22615/head//24 by client.2174595.0:37291 0.000000 2014-08-13 18:06:06.641365 7f5c973a57a0 20 read_log 194215'15305 (193305'15149) clean 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by osd.4.0:1555 2014-08-12 15:38:58.453379 2014-08-13 18:06:06.641377 7f5c973a57a0 20 read_log 194215'15306 (194215'15305) delete 84e55cb6/benchmark_data_storage0111-ib.pocstor_62621_object31848/head//24 by client.2174595.0:37293 0.000000 2014-08-13 18:06:06.641388 7f5c973a57a0 20 read_log 194215'15307 (192888'15014) clean 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by osd.4.0:1557 2014-08-12 15:38:58.461365 2014-08-13 18:06:06.641401 7f5c973a57a0 20 read_log 194215'15308 (194215'15307) delete 501f5cb6/benchmark_data_storage0111-ib.pocstor_2228_object19516/head//24 by client.2174595.0:37295 0.000000 2014-08-13 18:06:06.641416 7f5c973a57a0 20 read_log 194215'15309 (192888'15011) clean 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by osd.4.0:1559 2014-08-12 15:38:58.468593 2014-08-13 18:06:06.641438 7f5c973a57a0 20 read_log 194215'15310 (194215'15309) delete 4e4f6cb6/benchmark_data_storage0111-ib.pocstor_2228_object18952/head//24 by client.2174595.0:37297 0.000000 2014-08-13 18:06:06.641456 7f5c973a57a0 20 read_log 194215'15311 (192888'15036) clean f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by osd.4.0:1561 2014-08-12 15:38:58.477665 2014-08-13 18:06:06.641471 7f5c973a57a0 20 read_log 194215'15312 (194215'15311) delete f08e8cb6/benchmark_data_storage0111-ib.pocstor_2228_object28495/head//24 by client.2174595.0:37299 0.000000 2014-08-13 18:06:06.641483 7f5c973a57a0 20 read_log 194215'15313 (192888'15035) clean d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by osd.4.0:1563 2014-08-12 15:38:58.485360 2014-08-13 18:06:06.641516 7f5c973a57a0 20 read_log 194215'15314 (194215'15313) delete d52f9cb6/benchmark_data_storage0111-ib.pocstor_2228_object28275/head//24 by client.2174595.0:37301 0.000000 2014-08-13 18:06:06.641531 7f5c973a57a0 20 read_log 194215'15315 (193305'15132) clean 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by osd.4.0:1565 2014-08-12 15:38:58.495111 2014-08-13 18:06:06.641544 7f5c973a57a0 20 read_log 194215'15316 (194215'15315) delete 44eeccb6/benchmark_data_storage0111-ib.pocstor_62621_object16008/head//24 by client.2174595.0:37303 0.000000 2014-08-13 18:06:06.641555 7f5c973a57a0 20 read_log 194215'15317 (192888'14996) clean 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by osd.4.0:1567 2014-08-12 15:38:58.504474 2014-08-13 18:06:06.641567 7f5c973a57a0 20 read_log 194215'15318 (194215'15317) delete 589adcb6/benchmark_data_storage0111-ib.pocstor_2228_object6782/head//24 by client.2174595.0:37305 0.000000 2014-08-13 18:06:06.641579 7f5c973a57a0 20 read_log 194215'15319 (193305'15147) clean ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by osd.4.0:1569 2014-08-12 15:38:58.513240 2014-08-13 18:06:06.641591 7f5c973a57a0 20 read_log 194215'15320 (194215'15319) delete ff3ddcb6/benchmark_data_storage0111-ib.pocstor_62621_object29980/head//24 by client.2174595.0:37307 0.000000 2014-08-13 18:06:06.641610 7f5c973a57a0 20 read_log 194215'15321 (192888'15038) clean 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by osd.4.0:1571 2014-08-12 15:38:58.520583 2014-08-13 18:06:06.641626 7f5c973a57a0 20 read_log 194215'15322 (194215'15321) delete 90c3ecb6/benchmark_data_storage0111-ib.pocstor_2228_object29184/head//24 by client.2174595.0:37309 0.000000 2014-08-13 18:06:06.641653 7f5c973a57a0 20 read_log 194215'15323 (192888'14844) clean d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by osd.4.0:1573 2014-08-12 15:38:58.529404 2014-08-13 18:06:06.641670 7f5c973a57a0 20 read_log 194215'15324 (194215'15323) delete d9a8ecb6/benchmark_data_storage0111-ib.pocstor_41097_object20265/head//24 by client.2174595.0:37311 0.000000 2014-08-13 18:06:06.641687 7f5c973a57a0 20 read_log 194215'15325 (193305'15141) clean 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by osd.4.0:1575 2014-08-12 15:38:58.538415 2014-08-13 18:06:06.641710 7f5c973a57a0 20 read_log 194215'15326 (194215'15325) delete 9f89fcb6/benchmark_data_storage0111-ib.pocstor_62621_object24533/head//24 by client.2174595.0:37313 0.000000 2014-08-13 18:06:06.641724 7f5c973a57a0 20 read_log 194215'15327 (194215'15327) modify b6/hit_set_24.b6_archive_2014-08-12 15:38:57.656327_2014-08-12 17:24:42.218256/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.219403 2014-08-13 18:06:06.641737 7f5c973a57a0 20 read_log 194215'15328 (194215'15151) delete b6/hit_set_24.b6_archive_2014-08-12 15:32:25.864388_2014-08-12 15:38:57.656047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.219403 2014-08-13 18:06:06.641757 7f5c973a57a0 20 read_log 194215'15329 (0'0) promote e189c8b6/benchmark_data_storage0111-ib.pocstor_34628_object591/head//24 by osd.4.0:5238 2014-08-12 17:24:42.225151 2014-08-13 18:06:06.641773 7f5c973a57a0 20 read_log 194215'15330 (194215'15329) modify e189c8b6/benchmark_data_storage0111-ib.pocstor_34628_object591/head//24 by client.2193759.0:592 2014-08-12 17:24:42.212099 2014-08-13 18:06:06.641786 7f5c973a57a0 20 read_log 194216'15331 (0'0) promote 16e6e8b6/benchmark_data_storage0111-ib.pocstor_39693_object25/head//24 by osd.4.0:5252 2014-08-12 17:25:03.272073 2014-08-13 18:06:06.641799 7f5c973a57a0 20 read_log 194216'15332 (194216'15331) modify 16e6e8b6/benchmark_data_storage0111-ib.pocstor_39693_object25/head//24 by client.1487927.0:26 2014-08-12 17:25:03.260972 2014-08-13 18:06:06.641812 7f5c973a57a0 20 read_log 194216'15333 (0'0) promote 36319cb6/benchmark_data_storage0111-ib.pocstor_39693_object30/head//24 by osd.4.0:5254 2014-08-12 17:25:03.300761 2014-08-13 18:06:06.641824 7f5c973a57a0 20 read_log 194216'15334 (194216'15333) modify 36319cb6/benchmark_data_storage0111-ib.pocstor_39693_object30/head//24 by client.1487927.0:31 2014-08-12 17:25:03.293269 2014-08-13 18:06:06.641837 7f5c973a57a0 20 read_log 194370'15335 (0'0) promote 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by osd.327.0:1623 2014-08-12 20:30:15.938917 2014-08-13 18:06:06.641852 7f5c973a57a0 20 read_log 194370'15336 (194370'15335) modify 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by client.1508531.0:410 2014-08-12 20:30:15.927529 2014-08-13 18:06:06.641864 7f5c973a57a0 20 read_log 194370'15337 (0'0) promote fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by osd.327.0:1631 2014-08-12 20:30:18.646776 2014-08-13 18:06:06.641877 7f5c973a57a0 20 read_log 194370'15338 (194370'15337) modify fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by client.1508531.0:1144 2014-08-12 20:30:18.640063 2014-08-13 18:06:06.641897 7f5c973a57a0 20 read_log 194370'15339 (0'0) promote e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by osd.327.0:1635 2014-08-12 20:30:19.404330 2014-08-13 18:06:06.641913 7f5c973a57a0 20 read_log 194370'15340 (194370'15339) modify e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by client.1508531.0:1338 2014-08-12 20:30:19.397975 2014-08-13 18:06:06.641925 7f5c973a57a0 20 read_log 194370'15341 (0'0) promote 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by osd.327.0:1637 2014-08-12 20:30:19.453885 2014-08-13 18:06:06.641965 7f5c973a57a0 20 read_log 194370'15342 (194370'15341) modify 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by client.1508531.0:1352 2014-08-12 20:30:19.448773 2014-08-13 18:06:06.641981 7f5c973a57a0 20 read_log 194370'15343 (0'0) promote 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by osd.327.0:1647 2014-08-12 20:30:22.819535 2014-08-13 18:06:06.641993 7f5c973a57a0 20 read_log 194370'15344 (194370'15343) modify 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by client.1508531.0:2249 2014-08-12 20:30:22.811865 2014-08-13 18:06:06.642006 7f5c973a57a0 20 read_log 194370'15345 (0'0) promote b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by osd.327.0:1655 2014-08-12 20:30:28.364565 2014-08-13 18:06:06.642018 7f5c973a57a0 20 read_log 194370'15346 (194370'15345) modify b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by client.1508531.0:3744 2014-08-12 20:30:28.357002 2014-08-13 18:06:06.642031 7f5c973a57a0 20 read_log 194370'15347 (0'0) promote 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by osd.327.0:1657 2014-08-12 20:30:29.595741 2014-08-13 18:06:06.642043 7f5c973a57a0 20 read_log 194370'15348 (194370'15347) modify 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by client.1508531.0:4081 2014-08-12 20:30:29.589242 2014-08-13 18:06:06.642055 7f5c973a57a0 20 read_log 194370'15349 (0'0) promote 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by osd.327.0:1659 2014-08-12 20:30:29.601178 2014-08-13 18:06:06.642068 7f5c973a57a0 20 read_log 194370'15350 (194370'15349) modify 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by client.1508531.0:4082 2014-08-12 20:30:29.592706 2014-08-13 18:06:06.642080 7f5c973a57a0 20 read_log 194370'15351 (0'0) promote c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by osd.327.0:1661 2014-08-12 20:30:29.740578 2014-08-13 18:06:06.642092 7f5c973a57a0 20 read_log 194370'15352 (194370'15351) modify c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by client.1508531.0:4122 2014-08-12 20:30:29.734240 2014-08-13 18:06:06.642105 7f5c973a57a0 20 read_log 194370'15353 (0'0) promote 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by osd.327.0:1685 2014-08-12 20:30:40.074130 2014-08-13 18:06:06.642117 7f5c973a57a0 20 read_log 194370'15354 (194370'15353) modify 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by client.1508531.0:6520 2014-08-12 20:30:40.062956 2014-08-13 18:06:06.642129 7f5c973a57a0 20 read_log 194370'15355 (0'0) promote 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by osd.327.0:1691 2014-08-12 20:30:44.837155 2014-08-13 18:06:06.642143 7f5c973a57a0 20 read_log 194370'15356 (194370'15355) modify 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by client.1508531.0:7494 2014-08-12 20:30:44.827180 2014-08-13 18:06:06.642163 7f5c973a57a0 20 read_log 194370'15357 (0'0) promote 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by osd.327.0:1697 2014-08-12 20:30:50.203149 2014-08-13 18:06:06.642175 7f5c973a57a0 20 read_log 194370'15358 (194370'15357) modify 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by client.1508531.0:8494 2014-08-12 20:30:50.192733 2014-08-13 18:06:06.642188 7f5c973a57a0 20 read_log 194370'15359 (0'0) promote b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by osd.327.0:1715 2014-08-12 20:31:02.215713 2014-08-13 18:06:06.642201 7f5c973a57a0 20 read_log 194370'15360 (194370'15359) modify b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by client.1508531.0:10498 2014-08-12 20:31:02.203948 2014-08-13 18:06:06.642213 7f5c973a57a0 20 read_log 194370'15361 (0'0) promote dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by osd.327.0:1719 2014-08-12 20:31:04.537008 2014-08-13 18:06:06.642237 7f5c973a57a0 20 read_log 194370'15362 (194370'15361) modify dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by client.1508531.0:10874 2014-08-12 20:31:04.526909 2014-08-13 18:06:06.642253 7f5c973a57a0 20 read_log 194370'15363 (0'0) promote e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by osd.327.0:1733 2014-08-12 20:31:21.631971 2014-08-13 18:06:06.642266 7f5c973a57a0 20 read_log 194370'15364 (194370'15363) modify e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by client.1508531.0:13235 2014-08-12 20:31:21.622526 2014-08-13 18:06:06.642278 7f5c973a57a0 20 read_log 194370'15365 (0'0) promote b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by osd.327.0:1739 2014-08-12 20:31:24.446726 2014-08-13 18:06:06.642291 7f5c973a57a0 20 read_log 194370'15366 (194370'15365) modify b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by client.1508531.0:13533 2014-08-12 20:31:24.438250 2014-08-13 18:06:06.642303 7f5c973a57a0 20 read_log 194370'15367 (0'0) promote fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by osd.327.0:1747 2014-08-12 20:31:27.309446 2014-08-13 18:06:06.642321 7f5c973a57a0 20 read_log 194370'15368 (194370'15367) modify fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by client.1508531.0:13802 2014-08-12 20:31:27.300048 2014-08-13 18:06:06.642334 7f5c973a57a0 20 read_log 194377'15369 (0'0) promote cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by osd.17.0:5412 2014-08-12 20:32:59.757231 2014-08-13 18:06:06.642347 7f5c973a57a0 20 read_log 194377'15370 (194377'15369) modify cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by client.2231190.0:592 2014-08-12 20:32:59.748956 2014-08-13 18:06:06.642362 7f5c973a57a0 20 read_log 194377'15371 (0'0) promote a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by osd.17.0:5422 2014-08-12 20:33:01.245813 2014-08-13 18:06:06.642377 7f5c973a57a0 20 read_log 194377'15372 (194377'15371) modify a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by client.2231190.0:985 2014-08-12 20:33:01.237895 2014-08-13 18:06:06.642390 7f5c973a57a0 20 read_log 194377'15373 (0'0) promote 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by osd.17.0:5426 2014-08-12 20:33:02.370901 2014-08-13 18:06:06.642406 7f5c973a57a0 20 read_log 194377'15374 (194377'15373) modify 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by client.2231190.0:1280 2014-08-12 20:33:02.364322 2014-08-13 18:06:06.642421 7f5c973a57a0 20 read_log 194377'15375 (0'0) promote 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by osd.17.0:5452 2014-08-12 20:33:06.970592 2014-08-13 18:06:06.642440 7f5c973a57a0 20 read_log 194377'15376 (194377'15375) modify 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by client.2231190.0:2437 2014-08-12 20:33:06.962690 2014-08-13 18:06:06.642459 7f5c973a57a0 20 read_log 194377'15377 (0'0) promote 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by osd.17.0:5482 2014-08-12 20:33:13.599024 2014-08-13 18:06:06.642472 7f5c973a57a0 20 read_log 194377'15378 (194377'15377) modify 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by client.2231190.0:3944 2014-08-12 20:33:13.590004 2014-08-13 18:06:06.642495 7f5c973a57a0 20 read_log 194377'15379 (0'0) promote 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by osd.17.0:5506 2014-08-12 20:33:19.504028 2014-08-13 18:06:06.642511 7f5c973a57a0 20 read_log 194377'15380 (194377'15379) modify 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by client.2231190.0:4805 2014-08-12 20:33:19.495440 2014-08-13 18:06:06.642523 7f5c973a57a0 20 read_log 194380'15381 (0'0) promote 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by osd.17.0:5510 2014-08-12 20:33:27.400731 2014-08-13 18:06:06.642546 7f5c973a57a0 20 read_log 194380'15382 (194380'15381) modify 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by client.2231190.0:5318 2014-08-12 20:33:27.391945 2014-08-13 18:06:06.642562 7f5c973a57a0 20 read_log 194383'15383 (0'0) promote 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by osd.17.0:5526 2014-08-12 20:34:00.632129 2014-08-13 18:06:06.642575 7f5c973a57a0 20 read_log 194383'15384 (194383'15383) modify 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by client.2231190.0:5832 2014-08-12 20:34:00.622639 2014-08-13 18:06:06.642599 7f5c973a57a0 20 read_log 194393'15385 (194393'15385) modify b6/hit_set_24.b6_archive_2014-08-12 20:37:09.067202_2014-08-12 20:52:32.940397/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.941630 2014-08-13 18:06:06.642613 7f5c973a57a0 20 read_log 194393'15386 (194215'15327) delete b6/hit_set_24.b6_archive_2014-08-12 15:38:57.656327_2014-08-12 17:24:42.218256/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.941630 2014-08-13 18:06:06.642625 7f5c973a57a0 20 read_log 194393'15387 (0'0) promote 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by osd.327.0:1789 2014-08-12 20:52:32.947640 2014-08-13 18:06:06.642646 7f5c973a57a0 20 read_log 194393'15388 (194393'15387) modify 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by client.1064422.0:105 2014-08-12 20:52:32.928439 2014-08-13 18:06:06.642661 7f5c973a57a0 20 read_log 194393'15389 (0'0) promote 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by osd.327.0:1793 2014-08-12 20:52:33.176956 2014-08-13 18:06:06.642674 7f5c973a57a0 20 read_log 194393'15390 (194393'15389) modify 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by client.1064422.0:163 2014-08-12 20:52:33.171180 2014-08-13 18:06:06.642686 7f5c973a57a0 20 read_log 194396'15391 (0'0) promote 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by osd.327.0:1801 2014-08-12 20:53:00.273218 2014-08-13 18:06:06.642713 7f5c973a57a0 20 read_log 194396'15392 (194396'15391) modify 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by client.2235069.0:278 2014-08-12 20:53:00.005661 2014-08-13 18:06:06.642726 7f5c973a57a0 20 read_log 194416'15393 (194416'15393) modify b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 18:06:06.642739 7f5c973a57a0 20 read_log 194416'15394 (194393'15385) delete b6/hit_set_24.b6_archive_2014-08-12 20:37:09.067202_2014-08-12 20:52:32.940397/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 18:06:06.642752 7f5c973a57a0 20 read_log 194416'15395 (0'0) promote 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by osd.327.0:1929 2014-08-13 15:28:32.216485 2014-08-13 18:06:06.642764 7f5c973a57a0 20 read_log 194416'15396 (194416'15395) modify 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by client.2441958.0:456 2014-08-13 15:28:32.200243 2014-08-13 18:06:06.642777 7f5c973a57a0 20 read_log 194416'15397 (0'0) promote 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by osd.327.0:1934 2014-08-13 15:28:37.790278 2014-08-13 18:06:06.642797 7f5c973a57a0 20 read_log 194416'15398 (194416'15397) modify 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by client.2441958.0:1866 2014-08-13 15:28:37.782074 2014-08-13 18:06:06.642810 7f5c973a57a0 20 read_log 194416'15399 (0'0) promote c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by osd.327.0:1944 2014-08-13 15:28:39.841187 2014-08-13 18:06:06.642823 7f5c973a57a0 20 read_log 194416'15400 (194416'15399) modify c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by client.2441958.0:2410 2014-08-13 15:28:39.832496 2014-08-13 18:06:06.642857 7f5c973a57a0 20 read_log 194416'15401 (0'0) promote 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by osd.327.0:1946 2014-08-13 15:28:41.727376 2014-08-13 18:06:06.642874 7f5c973a57a0 20 read_log 194416'15402 (194416'15401) modify 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by client.2441958.0:2906 2014-08-13 15:28:41.719393 2014-08-13 18:06:06.642897 7f5c973a57a0 20 read_log 194416'15403 (0'0) promote d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by osd.327.0:1952 2014-08-13 15:28:45.807321 2014-08-13 18:06:06.642911 7f5c973a57a0 20 read_log 194416'15404 (194416'15403) modify d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by client.2441958.0:3967 2014-08-13 15:28:45.798504 2014-08-13 18:06:06.642924 7f5c973a57a0 20 read_log 194416'15405 (0'0) promote 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by osd.327.0:1956 2014-08-13 15:28:51.717904 2014-08-13 18:06:06.642939 7f5c973a57a0 20 read_log 194416'15406 (194416'15405) modify 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by client.2441958.0:5479 2014-08-13 15:28:51.708570 2014-08-13 18:06:06.642958 7f5c973a57a0 20 read_log 194416'15407 (0'0) promote 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by osd.327.0:1958 2014-08-13 15:28:52.976478 2014-08-13 18:06:06.642972 7f5c973a57a0 20 read_log 194416'15408 (194416'15407) modify 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by client.2441958.0:5799 2014-08-13 15:28:52.967429 2014-08-13 18:06:06.642987 7f5c973a57a0 20 read_log 194416'15409 (0'0) promote bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by osd.327.0:1966 2014-08-13 15:28:59.632837 2014-08-13 18:06:06.643000 7f5c973a57a0 20 read_log 194416'15410 (194416'15409) modify bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by client.2441958.0:7494 2014-08-13 15:28:59.623501 2014-08-13 18:06:06.643013 7f5c973a57a0 20 read_log 194416'15411 (0'0) promote aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by osd.327.0:1970 2014-08-13 15:29:01.575004 2014-08-13 18:06:06.643025 7f5c973a57a0 20 read_log 194416'15412 (194416'15411) modify aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by client.2441958.0:8011 2014-08-13 15:29:01.566132 2014-08-13 18:06:06.643041 7f5c973a57a0 20 read_log 194416'15413 (0'0) promote 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by osd.327.0:1974 2014-08-13 15:29:02.993201 2014-08-13 18:06:06.643053 7f5c973a57a0 20 read_log 194416'15414 (194416'15413) modify 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by client.2441958.0:8386 2014-08-13 15:29:02.986273 2014-08-13 18:06:06.643066 7f5c973a57a0 20 read_log 194416'15415 (0'0) promote 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by osd.327.0:1980 2014-08-13 15:29:06.514683 2014-08-13 18:06:06.643090 7f5c973a57a0 20 read_log 194416'15416 (194416'15415) modify 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by client.2441958.0:9272 2014-08-13 15:29:06.507029 2014-08-13 18:06:06.643103 7f5c973a57a0 20 read_log 194416'15417 (0'0) promote ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by osd.327.0:1982 2014-08-13 15:29:06.962728 2014-08-13 18:06:06.643116 7f5c973a57a0 20 read_log 194416'15418 (194416'15417) modify ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by client.2441958.0:9391 2014-08-13 15:29:06.953835 2014-08-13 18:06:06.643129 7f5c973a57a0 20 read_log 194416'15419 (0'0) promote 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by osd.327.0:1992 2014-08-13 15:29:11.741815 2014-08-13 18:06:06.643144 7f5c973a57a0 20 read_log 194416'15420 (194416'15419) modify 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by client.2441958.0:10581 2014-08-13 15:29:11.731226 2014-08-13 18:06:06.643171 7f5c973a57a0 20 read_log 194416'15421 (0'0) promote 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by osd.327.0:2000 2014-08-13 15:29:13.525661 2014-08-13 18:06:06.643188 7f5c973a57a0 20 read_log 194416'15422 (194416'15421) modify 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by client.2441958.0:11054 2014-08-13 15:29:13.516980 2014-08-13 18:06:06.643213 7f5c973a57a0 20 read_log 194416'15423 (0'0) promote a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by osd.327.0:2002 2014-08-13 15:29:15.854398 2014-08-13 18:06:06.643226 7f5c973a57a0 20 read_log 194416'15424 (194416'15423) modify a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by client.2441958.0:11584 2014-08-13 15:29:15.843703 2014-08-13 18:06:06.643243 7f5c973a57a0 20 read_log 194416'15425 (0'0) promote 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by osd.327.0:2004 2014-08-13 15:29:17.088086 2014-08-13 18:06:06.643256 7f5c973a57a0 20 read_log 194416'15426 (194416'15425) modify 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by client.2441958.0:11893 2014-08-13 15:29:17.077699 2014-08-13 18:06:06.643277 7f5c973a57a0 20 read_log 194416'15427 (0'0) promote 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by osd.327.0:2014 2014-08-13 15:29:25.123245 2014-08-13 18:06:06.643293 7f5c973a57a0 20 read_log 194416'15428 (194416'15427) modify 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by client.2441958.0:13729 2014-08-13 15:29:25.112446 2014-08-13 18:06:06.643306 7f5c973a57a0 20 read_log 194416'15429 (0'0) promote d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by osd.327.0:2018 2014-08-13 15:29:26.626288 2014-08-13 18:06:06.643319 7f5c973a57a0 20 read_log 194416'15430 (194416'15429) modify d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by client.2441958.0:14110 2014-08-13 15:29:26.616854 2014-08-13 18:06:06.643332 7f5c973a57a0 20 read_log 194416'15431 (0'0) promote ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by osd.327.0:2022 2014-08-13 15:29:30.499997 2014-08-13 18:06:06.643344 7f5c973a57a0 20 read_log 194416'15432 (194416'15431) modify ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by client.2441958.0:15047 2014-08-13 15:29:30.488617 2014-08-13 18:06:06.643356 7f5c973a57a0 20 read_log 194416'15433 (0'0) promote e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by osd.327.0:2024 2014-08-13 15:29:30.755553 2014-08-13 18:06:06.643369 7f5c973a57a0 20 read_log 194416'15434 (194416'15433) modify e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by client.2441958.0:15104 2014-08-13 15:29:30.745232 2014-08-13 18:06:06.643390 7f5c973a57a0 20 read_log 194416'15435 (0'0) promote 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by osd.327.0:2026 2014-08-13 15:29:32.497149 2014-08-13 18:06:06.643407 7f5c973a57a0 20 read_log 194416'15436 (194416'15435) modify 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by client.2441958.0:15457 2014-08-13 15:29:32.489527 2014-08-13 18:06:06.643419 7f5c973a57a0 20 read_log 194416'15437 (0'0) promote 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by osd.327.0:2040 2014-08-13 15:30:00.320034 2014-08-13 18:06:06.643432 7f5c973a57a0 20 read_log 194416'15438 (194416'15437) modify 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by client.2441958.0:17918 2014-08-13 15:30:00.308877 2014-08-13 18:06:06.643445 7f5c973a57a0 20 read_log 194416'15439 (0'0) promote 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by osd.327.0:2042 2014-08-13 15:30:00.483514 2014-08-13 18:06:06.643457 7f5c973a57a0 20 read_log 194416'15440 (194416'15439) modify 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by client.2441958.0:17930 2014-08-13 15:30:00.474134 2014-08-13 18:06:06.643478 7f5c973a57a0 20 read_log 194422'15441 (0'0) promote 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by osd.327.0:2066 2014-08-13 15:32:26.590505 2014-08-13 18:06:06.643493 7f5c973a57a0 20 read_log 194422'15442 (194422'15441) modify 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by client.1148203.0:2867 2014-08-13 15:32:26.581398 2014-08-13 18:06:06.643508 7f5c973a57a0 20 read_log 194422'15443 (0'0) promote b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by osd.327.0:2072 2014-08-13 15:32:30.001397 2014-08-13 18:06:06.643521 7f5c973a57a0 20 read_log 194422'15444 (194422'15443) modify b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by client.1148203.0:3736 2014-08-13 15:32:29.992429 2014-08-13 18:06:06.643540 7f5c973a57a0 20 read_log 194422'15445 (0'0) promote be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by osd.327.0:2074 2014-08-13 15:32:30.689989 2014-08-13 18:06:06.643556 7f5c973a57a0 20 read_log 194422'15446 (194422'15445) modify be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by client.1148203.0:3922 2014-08-13 15:32:30.682233 2014-08-13 18:06:06.643569 7f5c973a57a0 20 read_log 194422'15447 (0'0) promote bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by osd.327.0:2088 2014-08-13 15:32:50.050377 2014-08-13 18:06:06.643581 7f5c973a57a0 20 read_log 194422'15448 (194422'15447) modify bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by client.1148203.0:7727 2014-08-13 15:32:50.037724 2014-08-13 18:06:06.643594 7f5c973a57a0 20 read_log 194422'15449 (0'0) promote 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by osd.327.0:2100 2014-08-13 15:33:05.440521 2014-08-13 18:06:06.643612 7f5c973a57a0 20 read_log 194422'15450 (194422'15449) modify 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by client.1148203.0:9599 2014-08-13 15:33:05.430555 2014-08-13 18:06:06.643637 7f5c973a57a0 20 read_log 194422'15451 (0'0) promote c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by osd.327.0:2102 2014-08-13 15:33:07.223246 2014-08-13 18:06:06.643656 7f5c973a57a0 20 read_log 194422'15452 (194422'15451) modify c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by client.1148203.0:9755 2014-08-13 15:33:07.212684 2014-08-13 18:06:06.643676 7f5c973a57a0 20 read_log 194422'15453 (0'0) promote 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by osd.327.0:2106 2014-08-13 15:33:26.017816 2014-08-13 18:06:06.643692 7f5c973a57a0 20 read_log 194422'15454 (194422'15453) modify 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by client.1148203.0:10574 2014-08-13 15:33:26.009369 2014-08-13 18:06:06.643709 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.644096 7f5c973a57a0 10 read_log checking for missing items over interval (194370'15335,194422'15454] 2014-08-13 18:06:06.644151 7f5c973a57a0 15 read_log missing 194422'15454 (194422'15453) modify 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 by client.1148203.0:10574 2014-08-13 15:33:26.009369 2014-08-13 18:06:06.644193 7f5c973a57a0 15 read_log missing 194422'15452 (194422'15451) modify c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 by client.1148203.0:9755 2014-08-13 15:33:07.212684 2014-08-13 18:06:06.644229 7f5c973a57a0 15 read_log missing 194422'15450 (194422'15449) modify 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 by client.1148203.0:9599 2014-08-13 15:33:05.430555 2014-08-13 18:06:06.644269 7f5c973a57a0 15 read_log missing 194422'15448 (194422'15447) modify bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 by client.1148203.0:7727 2014-08-13 15:32:50.037724 2014-08-13 18:06:06.644308 7f5c973a57a0 15 read_log missing 194422'15446 (194422'15445) modify be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 by client.1148203.0:3922 2014-08-13 15:32:30.682233 2014-08-13 18:06:06.644345 7f5c973a57a0 15 read_log missing 194422'15444 (194422'15443) modify b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 by client.1148203.0:3736 2014-08-13 15:32:29.992429 2014-08-13 18:06:06.644390 7f5c973a57a0 15 read_log missing 194422'15442 (194422'15441) modify 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 by client.1148203.0:2867 2014-08-13 15:32:26.581398 2014-08-13 18:06:06.644429 7f5c973a57a0 15 read_log missing 194416'15440 (194416'15439) modify 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 by client.2441958.0:17930 2014-08-13 15:30:00.474134 2014-08-13 18:06:06.644465 7f5c973a57a0 15 read_log missing 194416'15438 (194416'15437) modify 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 by client.2441958.0:17918 2014-08-13 15:30:00.308877 2014-08-13 18:06:06.644501 7f5c973a57a0 15 read_log missing 194416'15436 (194416'15435) modify 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 by client.2441958.0:15457 2014-08-13 15:29:32.489527 2014-08-13 18:06:06.644538 7f5c973a57a0 15 read_log missing 194416'15434 (194416'15433) modify e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 by client.2441958.0:15104 2014-08-13 15:29:30.745232 2014-08-13 18:06:06.644574 7f5c973a57a0 15 read_log missing 194416'15432 (194416'15431) modify ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 by client.2441958.0:15047 2014-08-13 15:29:30.488617 2014-08-13 18:06:06.644610 7f5c973a57a0 15 read_log missing 194416'15430 (194416'15429) modify d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 by client.2441958.0:14110 2014-08-13 15:29:26.616854 2014-08-13 18:06:06.644646 7f5c973a57a0 15 read_log missing 194416'15428 (194416'15427) modify 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 by client.2441958.0:13729 2014-08-13 15:29:25.112446 2014-08-13 18:06:06.644682 7f5c973a57a0 15 read_log missing 194416'15426 (194416'15425) modify 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 by client.2441958.0:11893 2014-08-13 15:29:17.077699 2014-08-13 18:06:06.644717 7f5c973a57a0 15 read_log missing 194416'15424 (194416'15423) modify a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 by client.2441958.0:11584 2014-08-13 15:29:15.843703 2014-08-13 18:06:06.644753 7f5c973a57a0 15 read_log missing 194416'15422 (194416'15421) modify 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 by client.2441958.0:11054 2014-08-13 15:29:13.516980 2014-08-13 18:06:06.644790 7f5c973a57a0 15 read_log missing 194416'15420 (194416'15419) modify 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 by client.2441958.0:10581 2014-08-13 15:29:11.731226 2014-08-13 18:06:06.644826 7f5c973a57a0 15 read_log missing 194416'15418 (194416'15417) modify ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 by client.2441958.0:9391 2014-08-13 15:29:06.953835 2014-08-13 18:06:06.644862 7f5c973a57a0 15 read_log missing 194416'15416 (194416'15415) modify 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 by client.2441958.0:9272 2014-08-13 15:29:06.507029 2014-08-13 18:06:06.644897 7f5c973a57a0 15 read_log missing 194416'15414 (194416'15413) modify 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 by client.2441958.0:8386 2014-08-13 15:29:02.986273 2014-08-13 18:06:06.644933 7f5c973a57a0 15 read_log missing 194416'15412 (194416'15411) modify aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 by client.2441958.0:8011 2014-08-13 15:29:01.566132 2014-08-13 18:06:06.644969 7f5c973a57a0 15 read_log missing 194416'15410 (194416'15409) modify bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 by client.2441958.0:7494 2014-08-13 15:28:59.623501 2014-08-13 18:06:06.645004 7f5c973a57a0 15 read_log missing 194416'15408 (194416'15407) modify 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 by client.2441958.0:5799 2014-08-13 15:28:52.967429 2014-08-13 18:06:06.645040 7f5c973a57a0 15 read_log missing 194416'15406 (194416'15405) modify 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 by client.2441958.0:5479 2014-08-13 15:28:51.708570 2014-08-13 18:06:06.645075 7f5c973a57a0 15 read_log missing 194416'15404 (194416'15403) modify d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 by client.2441958.0:3967 2014-08-13 15:28:45.798504 2014-08-13 18:06:06.645132 7f5c973a57a0 15 read_log missing 194416'15402 (194416'15401) modify 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 by client.2441958.0:2906 2014-08-13 15:28:41.719393 2014-08-13 18:06:06.645179 7f5c973a57a0 15 read_log missing 194416'15400 (194416'15399) modify c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 by client.2441958.0:2410 2014-08-13 15:28:39.832496 2014-08-13 18:06:06.645216 7f5c973a57a0 15 read_log missing 194416'15398 (194416'15397) modify 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 by client.2441958.0:1866 2014-08-13 15:28:37.782074 2014-08-13 18:06:06.645255 7f5c973a57a0 15 read_log missing 194416'15396 (194416'15395) modify 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 by client.2441958.0:456 2014-08-13 15:28:32.200243 2014-08-13 18:06:06.645301 7f5c973a57a0 15 read_log missing 194416'15393 (194416'15393) modify b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.209492 2014-08-13 18:06:06.645337 7f5c973a57a0 15 read_log missing 194396'15392 (194396'15391) modify 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 by client.2235069.0:278 2014-08-12 20:53:00.005661 2014-08-13 18:06:06.645385 7f5c973a57a0 15 read_log missing 194393'15390 (194393'15389) modify 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 by client.1064422.0:163 2014-08-12 20:52:33.171180 2014-08-13 18:06:06.645473 7f5c973a57a0 15 read_log missing 194393'15388 (194393'15387) modify 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 by client.1064422.0:105 2014-08-12 20:52:32.928439 2014-08-13 18:06:06.645524 7f5c973a57a0 15 read_log missing 194383'15384 (194383'15383) modify 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 by client.2231190.0:5832 2014-08-12 20:34:00.622639 2014-08-13 18:06:06.645569 7f5c973a57a0 15 read_log missing 194380'15382 (194380'15381) modify 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 by client.2231190.0:5318 2014-08-12 20:33:27.391945 2014-08-13 18:06:06.645621 7f5c973a57a0 15 read_log missing 194377'15380 (194377'15379) modify 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 by client.2231190.0:4805 2014-08-12 20:33:19.495440 2014-08-13 18:06:06.645663 7f5c973a57a0 15 read_log missing 194377'15378 (194377'15377) modify 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 by client.2231190.0:3944 2014-08-12 20:33:13.590004 2014-08-13 18:06:06.645700 7f5c973a57a0 15 read_log missing 194377'15376 (194377'15375) modify 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 by client.2231190.0:2437 2014-08-12 20:33:06.962690 2014-08-13 18:06:06.645736 7f5c973a57a0 15 read_log missing 194377'15374 (194377'15373) modify 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 by client.2231190.0:1280 2014-08-12 20:33:02.364322 2014-08-13 18:06:06.645794 7f5c973a57a0 15 read_log missing 194377'15372 (194377'15371) modify a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 by client.2231190.0:985 2014-08-12 20:33:01.237895 2014-08-13 18:06:06.645835 7f5c973a57a0 15 read_log missing 194377'15370 (194377'15369) modify cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 by client.2231190.0:592 2014-08-12 20:32:59.748956 2014-08-13 18:06:06.645892 7f5c973a57a0 15 read_log missing 194370'15368 (194370'15367) modify fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 by client.1508531.0:13802 2014-08-12 20:31:27.300048 2014-08-13 18:06:06.645955 7f5c973a57a0 15 read_log missing 194370'15366 (194370'15365) modify b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 by client.1508531.0:13533 2014-08-12 20:31:24.438250 2014-08-13 18:06:06.645995 7f5c973a57a0 15 read_log missing 194370'15364 (194370'15363) modify e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 by client.1508531.0:13235 2014-08-12 20:31:21.622526 2014-08-13 18:06:06.646038 7f5c973a57a0 15 read_log missing 194370'15362 (194370'15361) modify dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 by client.1508531.0:10874 2014-08-12 20:31:04.526909 2014-08-13 18:06:06.646090 7f5c973a57a0 15 read_log missing 194370'15360 (194370'15359) modify b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 by client.1508531.0:10498 2014-08-12 20:31:02.203948 2014-08-13 18:06:06.646126 7f5c973a57a0 15 read_log missing 194370'15358 (194370'15357) modify 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 by client.1508531.0:8494 2014-08-12 20:30:50.192733 2014-08-13 18:06:06.646169 7f5c973a57a0 15 read_log missing 194370'15356 (194370'15355) modify 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 by client.1508531.0:7494 2014-08-12 20:30:44.827180 2014-08-13 18:06:06.646206 7f5c973a57a0 15 read_log missing 194370'15354 (194370'15353) modify 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 by client.1508531.0:6520 2014-08-12 20:30:40.062956 2014-08-13 18:06:06.646254 7f5c973a57a0 15 read_log missing 194370'15352 (194370'15351) modify c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 by client.1508531.0:4122 2014-08-12 20:30:29.734240 2014-08-13 18:06:06.646297 7f5c973a57a0 15 read_log missing 194370'15350 (194370'15349) modify 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 by client.1508531.0:4082 2014-08-12 20:30:29.592706 2014-08-13 18:06:06.646339 7f5c973a57a0 15 read_log missing 194370'15348 (194370'15347) modify 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 by client.1508531.0:4081 2014-08-12 20:30:29.589242 2014-08-13 18:06:06.646376 7f5c973a57a0 15 read_log missing 194370'15346 (194370'15345) modify b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 by client.1508531.0:3744 2014-08-12 20:30:28.357002 2014-08-13 18:06:06.646442 7f5c973a57a0 15 read_log missing 194370'15344 (194370'15343) modify 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 by client.1508531.0:2249 2014-08-12 20:30:22.811865 2014-08-13 18:06:06.646481 7f5c973a57a0 15 read_log missing 194370'15342 (194370'15341) modify 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 by client.1508531.0:1352 2014-08-12 20:30:19.448773 2014-08-13 18:06:06.646517 7f5c973a57a0 15 read_log missing 194370'15340 (194370'15339) modify e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 by client.1508531.0:1338 2014-08-12 20:30:19.397975 2014-08-13 18:06:06.646553 7f5c973a57a0 15 read_log missing 194370'15338 (194370'15337) modify fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 by client.1508531.0:1144 2014-08-12 20:30:18.640063 2014-08-13 18:06:06.646588 7f5c973a57a0 15 read_log missing 194370'15336 (194370'15335) modify 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 by client.1508531.0:410 2014-08-12 20:30:15.927529 2014-08-13 18:06:06.646604 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.646631 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_loaded 2014-08-13 18:06:06.646650 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Initial 0.014217 0 0.000000 2014-08-13 18:06:06.646663 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 18:06:06.646673 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=0 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194487 2014-08-13 18:06:06.646683 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 18:06:06.646694 7f5c973a57a0 10 osd.4 194487 pgid 24.cd coll 24.cd_head 2014-08-13 18:06:06.646852 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.cd 2014-08-13 18:06:06.646865 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.646907 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.cd(unlocked)] enter Initial 2014-08-13 18:06:06.646919 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.cd(unlocked)] enter NotTrimming 2014-08-13 18:06:06.647076 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.647215 7f5c973a57a0 20 read_log 192718'12886 (192718'12885) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:250 2014-08-06 14:19:56.760918 2014-08-13 18:06:06.647237 7f5c973a57a0 20 read_log 192718'12887 (192718'12886) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:251 2014-08-06 14:19:56.791441 2014-08-13 18:06:06.647250 7f5c973a57a0 20 read_log 192718'12888 (192718'12887) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:251 2014-08-06 14:19:56.807918 2014-08-13 18:06:06.647263 7f5c973a57a0 20 read_log 192718'12889 (192718'12888) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:252 2014-08-06 14:19:56.875582 2014-08-13 18:06:06.647279 7f5c973a57a0 20 read_log 192718'12890 (192718'12889) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:252 2014-08-06 14:19:56.962008 2014-08-13 18:06:06.647294 7f5c973a57a0 20 read_log 192718'12891 (192718'12890) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:253 2014-08-06 14:19:56.980626 2014-08-13 18:06:06.647306 7f5c973a57a0 20 read_log 192718'12892 (192718'12891) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:253 2014-08-06 14:19:56.994968 2014-08-13 18:06:06.647319 7f5c973a57a0 20 read_log 192718'12893 (192718'12892) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:254 2014-08-06 14:19:57.010129 2014-08-13 18:06:06.647333 7f5c973a57a0 20 read_log 192718'12894 (192718'12893) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:254 2014-08-06 14:19:57.027442 2014-08-13 18:06:06.647346 7f5c973a57a0 20 read_log 192718'12895 (192718'12894) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:255 2014-08-06 14:19:57.098108 2014-08-13 18:06:06.647358 7f5c973a57a0 20 read_log 192718'12896 (192718'12895) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:255 2014-08-06 14:19:57.098331 2014-08-13 18:06:06.647393 7f5c973a57a0 20 read_log 192718'12897 (192718'12896) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:256 2014-08-06 14:19:57.151004 2014-08-13 18:06:06.647410 7f5c973a57a0 20 read_log 192718'12898 (192718'12897) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:256 2014-08-06 14:19:57.203773 2014-08-13 18:06:06.647422 7f5c973a57a0 20 read_log 192718'12899 (192718'12898) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:257 2014-08-06 14:19:57.413574 2014-08-13 18:06:06.647435 7f5c973a57a0 20 read_log 192718'12900 (192718'12899) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:257 2014-08-06 14:19:57.505038 2014-08-13 18:06:06.647447 7f5c973a57a0 20 read_log 192718'12901 (192718'12900) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:258 2014-08-06 14:19:57.525061 2014-08-13 18:06:06.647464 7f5c973a57a0 20 read_log 192718'12902 (192718'12901) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:258 2014-08-06 14:19:57.551645 2014-08-13 18:06:06.647479 7f5c973a57a0 20 read_log 192718'12903 (192718'12902) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:259 2014-08-06 14:19:57.593093 2014-08-13 18:06:06.647492 7f5c973a57a0 20 read_log 192718'12904 (192718'12903) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:259 2014-08-06 14:19:57.593061 2014-08-13 18:06:06.647504 7f5c973a57a0 20 read_log 192718'12905 (192718'12904) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:260 2014-08-06 14:19:57.825570 2014-08-13 18:06:06.647534 7f5c973a57a0 20 read_log 192718'12906 (192718'12905) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:260 2014-08-06 14:19:57.886668 2014-08-13 18:06:06.647550 7f5c973a57a0 20 read_log 192718'12907 (192718'12906) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:261 2014-08-06 14:19:57.963221 2014-08-13 18:06:06.647562 7f5c973a57a0 20 read_log 192718'12908 (192718'12907) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:261 2014-08-06 14:19:57.963338 2014-08-13 18:06:06.647574 7f5c973a57a0 20 read_log 192718'12909 (192718'12908) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:262 2014-08-06 14:19:58.061780 2014-08-13 18:06:06.647587 7f5c973a57a0 20 read_log 192718'12910 (192718'12909) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:262 2014-08-06 14:19:58.127951 2014-08-13 18:06:06.647599 7f5c973a57a0 20 read_log 192718'12911 (192718'12910) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:263 2014-08-06 14:19:58.161768 2014-08-13 18:06:06.647611 7f5c973a57a0 20 read_log 192718'12912 (192718'12911) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:263 2014-08-06 14:19:58.193512 2014-08-13 18:06:06.647623 7f5c973a57a0 20 read_log 192718'12913 (192718'12912) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:264 2014-08-06 14:19:58.243489 2014-08-13 18:06:06.647645 7f5c973a57a0 20 read_log 192718'12914 (192718'12913) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:264 2014-08-06 14:19:58.337256 2014-08-13 18:06:06.647661 7f5c973a57a0 20 read_log 192718'12915 (192718'12914) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:265 2014-08-06 14:19:58.600951 2014-08-13 18:06:06.647673 7f5c973a57a0 20 read_log 192718'12916 (192718'12915) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:265 2014-08-06 14:19:58.641740 2014-08-13 18:06:06.647710 7f5c973a57a0 20 read_log 192718'12917 (192718'12916) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:266 2014-08-06 14:19:58.751607 2014-08-13 18:06:06.647726 7f5c973a57a0 20 read_log 192718'12918 (192718'12917) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:266 2014-08-06 14:19:59.207061 2014-08-13 18:06:06.647738 7f5c973a57a0 20 read_log 192718'12919 (192718'12918) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:267 2014-08-06 14:19:59.360766 2014-08-13 18:06:06.647750 7f5c973a57a0 20 read_log 192718'12920 (192718'12919) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:267 2014-08-06 14:19:59.695766 2014-08-13 18:06:06.647770 7f5c973a57a0 20 read_log 192718'12921 (192718'12920) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:268 2014-08-06 14:19:59.786818 2014-08-13 18:06:06.647784 7f5c973a57a0 20 read_log 192718'12922 (192718'12921) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:268 2014-08-06 14:19:59.992924 2014-08-13 18:06:06.647797 7f5c973a57a0 20 read_log 192718'12923 (192718'12922) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:269 2014-08-06 14:20:00.148926 2014-08-13 18:06:06.647809 7f5c973a57a0 20 read_log 192718'12924 (192718'12923) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:269 2014-08-06 14:20:00.259181 2014-08-13 18:06:06.647821 7f5c973a57a0 20 read_log 192718'12925 (192718'12924) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:270 2014-08-06 14:20:00.337829 2014-08-13 18:06:06.647833 7f5c973a57a0 20 read_log 192718'12926 (192718'12925) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:270 2014-08-06 14:20:00.672571 2014-08-13 18:06:06.647845 7f5c973a57a0 20 read_log 192718'12927 (192718'12926) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:271 2014-08-06 14:20:00.751163 2014-08-13 18:06:06.647857 7f5c973a57a0 20 read_log 192718'12928 (192718'12927) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:271 2014-08-06 14:20:00.863751 2014-08-13 18:06:06.647869 7f5c973a57a0 20 read_log 192718'12929 (192718'12928) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:272 2014-08-06 14:20:00.923558 2014-08-13 18:06:06.647881 7f5c973a57a0 20 read_log 192718'12930 (192718'12929) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:272 2014-08-06 14:20:00.943084 2014-08-13 18:06:06.647893 7f5c973a57a0 20 read_log 192718'12931 (192718'12930) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:273 2014-08-06 14:20:00.980367 2014-08-13 18:06:06.647905 7f5c973a57a0 20 read_log 192718'12932 (192718'12931) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:273 2014-08-06 14:20:01.010928 2014-08-13 18:06:06.647917 7f5c973a57a0 20 read_log 192718'12933 (192718'12932) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:274 2014-08-06 14:20:01.038947 2014-08-13 18:06:06.647929 7f5c973a57a0 20 read_log 192718'12934 (192718'12933) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:274 2014-08-06 14:20:01.073429 2014-08-13 18:06:06.647941 7f5c973a57a0 20 read_log 192718'12935 (192718'12934) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:275 2014-08-06 14:20:01.135949 2014-08-13 18:06:06.647953 7f5c973a57a0 20 read_log 192718'12936 (192718'12935) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:275 2014-08-06 14:20:01.363661 2014-08-13 18:06:06.647965 7f5c973a57a0 20 read_log 192718'12937 (192718'12936) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:276 2014-08-06 14:20:01.425602 2014-08-13 18:06:06.647977 7f5c973a57a0 20 read_log 192718'12938 (192718'12937) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:276 2014-08-06 14:20:01.503083 2014-08-13 18:06:06.647989 7f5c973a57a0 20 read_log 192718'12939 (192718'12938) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:277 2014-08-06 14:20:01.618705 2014-08-13 18:06:06.648001 7f5c973a57a0 20 read_log 192718'12940 (192718'12939) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:277 2014-08-06 14:20:01.657661 2014-08-13 18:06:06.648013 7f5c973a57a0 20 read_log 192718'12941 (192718'12940) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:278 2014-08-06 14:20:01.765604 2014-08-13 18:06:06.648025 7f5c973a57a0 20 read_log 192718'12942 (192718'12941) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:278 2014-08-06 14:20:01.890004 2014-08-13 18:06:06.648040 7f5c973a57a0 20 read_log 192718'12943 (192718'12942) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:279 2014-08-06 14:20:02.003530 2014-08-13 18:06:06.648074 7f5c973a57a0 20 read_log 192718'12944 (192718'12943) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:279 2014-08-06 14:20:02.031348 2014-08-13 18:06:06.648093 7f5c973a57a0 20 read_log 192718'12945 (192718'12944) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:280 2014-08-06 14:20:02.083653 2014-08-13 18:06:06.648108 7f5c973a57a0 20 read_log 192718'12946 (192718'12945) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:280 2014-08-06 14:20:02.123033 2014-08-13 18:06:06.648120 7f5c973a57a0 20 read_log 192718'12947 (192718'12946) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:281 2014-08-06 14:20:02.182567 2014-08-13 18:06:06.648133 7f5c973a57a0 20 read_log 192718'12948 (192718'12947) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:281 2014-08-06 14:20:02.404137 2014-08-13 18:06:06.648154 7f5c973a57a0 20 read_log 192718'12949 (192718'12948) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:282 2014-08-06 14:20:02.448903 2014-08-13 18:06:06.648170 7f5c973a57a0 20 read_log 192718'12950 (192718'12949) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:282 2014-08-06 14:20:02.529782 2014-08-13 18:06:06.648182 7f5c973a57a0 20 read_log 192718'12951 (192718'12950) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:283 2014-08-06 14:20:02.613534 2014-08-13 18:06:06.648194 7f5c973a57a0 20 read_log 192718'12952 (192718'12951) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:283 2014-08-06 14:20:02.722147 2014-08-13 18:06:06.648208 7f5c973a57a0 20 read_log 192718'12953 (192718'12952) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:284 2014-08-06 14:20:02.794140 2014-08-13 18:06:06.648221 7f5c973a57a0 20 read_log 192718'12954 (192718'12953) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:284 2014-08-06 14:20:02.866153 2014-08-13 18:06:06.648233 7f5c973a57a0 20 read_log 192718'12955 (192718'12954) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:285 2014-08-06 14:20:02.930733 2014-08-13 18:06:06.648247 7f5c973a57a0 20 read_log 192718'12956 (192718'12955) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:285 2014-08-06 14:20:03.001145 2014-08-13 18:06:06.648259 7f5c973a57a0 20 read_log 192718'12957 (192718'12956) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:286 2014-08-06 14:20:03.067173 2014-08-13 18:06:06.648274 7f5c973a57a0 20 read_log 192718'12958 (192718'12957) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:286 2014-08-06 14:20:03.296994 2014-08-13 18:06:06.648289 7f5c973a57a0 20 read_log 192718'12959 (192718'12958) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:287 2014-08-06 14:20:03.402762 2014-08-13 18:06:06.648301 7f5c973a57a0 20 read_log 192718'12960 (192718'12959) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:287 2014-08-06 14:20:03.645512 2014-08-13 18:06:06.648313 7f5c973a57a0 20 read_log 192718'12961 (192718'12960) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:288 2014-08-06 14:20:04.476787 2014-08-13 18:06:06.648329 7f5c973a57a0 20 read_log 192718'12962 (192718'12961) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:288 2014-08-06 14:20:04.699644 2014-08-13 18:06:06.648342 7f5c973a57a0 20 read_log 192718'12963 (192718'12962) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:289 2014-08-06 14:20:04.741850 2014-08-13 18:06:06.648354 7f5c973a57a0 20 read_log 192718'12964 (192718'12963) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:289 2014-08-06 14:20:04.793174 2014-08-13 18:06:06.648369 7f5c973a57a0 20 read_log 192718'12965 (192718'12964) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:290 2014-08-06 14:20:04.813337 2014-08-13 18:06:06.648381 7f5c973a57a0 20 read_log 192718'12966 (192718'12965) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:290 2014-08-06 14:20:04.837382 2014-08-13 18:06:06.648404 7f5c973a57a0 20 read_log 192718'12967 (192718'12966) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:291 2014-08-06 14:20:04.895681 2014-08-13 18:06:06.648423 7f5c973a57a0 20 read_log 192718'12968 (192718'12967) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:291 2014-08-06 14:20:04.903477 2014-08-13 18:06:06.648456 7f5c973a57a0 20 read_log 192718'12969 (192718'12968) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:292 2014-08-06 14:20:04.942194 2014-08-13 18:06:06.648473 7f5c973a57a0 20 read_log 192718'12970 (192718'12969) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:292 2014-08-06 14:20:05.007546 2014-08-13 18:06:06.648486 7f5c973a57a0 20 read_log 192718'12971 (192718'12970) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:293 2014-08-06 14:20:05.075085 2014-08-13 18:06:06.648498 7f5c973a57a0 20 read_log 192718'12972 (192718'12971) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:293 2014-08-06 14:20:05.175712 2014-08-13 18:06:06.648511 7f5c973a57a0 20 read_log 192718'12973 (192718'12972) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:294 2014-08-06 14:20:05.190112 2014-08-13 18:06:06.648523 7f5c973a57a0 20 read_log 192718'12974 (192718'12973) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:294 2014-08-06 14:20:05.216057 2014-08-13 18:06:06.648535 7f5c973a57a0 20 read_log 192718'12975 (192718'12974) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:295 2014-08-06 14:20:05.256400 2014-08-13 18:06:06.648555 7f5c973a57a0 20 read_log 192718'12976 (192718'12975) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:295 2014-08-06 14:20:05.256547 2014-08-13 18:06:06.648572 7f5c973a57a0 20 read_log 192718'12977 (192718'12976) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:296 2014-08-06 14:20:05.290774 2014-08-13 18:06:06.648585 7f5c973a57a0 20 read_log 192718'12978 (192718'12977) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:296 2014-08-06 14:20:05.304407 2014-08-13 18:06:06.648601 7f5c973a57a0 20 read_log 192718'12979 (192718'12978) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:297 2014-08-06 14:20:05.351273 2014-08-13 18:06:06.648616 7f5c973a57a0 20 read_log 192718'12980 (192718'12979) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:297 2014-08-06 14:20:05.354027 2014-08-13 18:06:06.648628 7f5c973a57a0 20 read_log 192718'12981 (192718'12980) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:298 2014-08-06 14:20:05.468130 2014-08-13 18:06:06.648641 7f5c973a57a0 20 read_log 192718'12982 (192718'12981) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:298 2014-08-06 14:20:05.487720 2014-08-13 18:06:06.648653 7f5c973a57a0 20 read_log 192718'12983 (192718'12982) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:299 2014-08-06 14:20:05.501101 2014-08-13 18:06:06.648665 7f5c973a57a0 20 read_log 192718'12984 (192718'12983) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:299 2014-08-06 14:20:05.530302 2014-08-13 18:06:06.648677 7f5c973a57a0 20 read_log 192718'12985 (192718'12984) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:300 2014-08-06 14:20:05.590970 2014-08-13 18:06:06.648689 7f5c973a57a0 20 read_log 192718'12986 (192718'12985) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:300 2014-08-06 14:20:05.590677 2014-08-13 18:06:06.648701 7f5c973a57a0 20 read_log 192718'12987 (192718'12986) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:301 2014-08-06 14:20:05.633807 2014-08-13 18:06:06.648713 7f5c973a57a0 20 read_log 192718'12988 (192718'12987) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:301 2014-08-06 14:20:05.633807 2014-08-13 18:06:06.648725 7f5c973a57a0 20 read_log 192718'12989 (192718'12988) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:302 2014-08-06 14:20:05.660625 2014-08-13 18:06:06.648738 7f5c973a57a0 20 read_log 192718'12990 (192718'12989) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:302 2014-08-06 14:20:05.669423 2014-08-13 18:06:06.648749 7f5c973a57a0 20 read_log 192718'12991 (192718'12990) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:303 2014-08-06 14:20:05.936009 2014-08-13 18:06:06.648762 7f5c973a57a0 20 read_log 192718'12992 (192718'12991) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:303 2014-08-06 14:20:05.941043 2014-08-13 18:06:06.648789 7f5c973a57a0 20 read_log 192718'12993 (192718'12992) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:304 2014-08-06 14:20:05.980207 2014-08-13 18:06:06.648803 7f5c973a57a0 20 read_log 192718'12994 (192718'12993) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:304 2014-08-06 14:20:06.006070 2014-08-13 18:06:06.648816 7f5c973a57a0 20 read_log 192718'12995 (192718'12994) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:305 2014-08-06 14:20:06.050766 2014-08-13 18:06:06.648828 7f5c973a57a0 20 read_log 192718'12996 (192718'12995) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:305 2014-08-06 14:20:06.076547 2014-08-13 18:06:06.648840 7f5c973a57a0 20 read_log 192718'12997 (192718'12996) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:306 2014-08-06 14:20:06.095923 2014-08-13 18:06:06.648852 7f5c973a57a0 20 read_log 192718'12998 (192718'12997) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:306 2014-08-06 14:20:06.120395 2014-08-13 18:06:06.648867 7f5c973a57a0 20 read_log 192718'12999 (192718'12998) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:307 2014-08-06 14:20:06.174717 2014-08-13 18:06:06.648882 7f5c973a57a0 20 read_log 192718'13000 (192718'12999) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:307 2014-08-06 14:20:06.174949 2014-08-13 18:06:06.648895 7f5c973a57a0 20 read_log 192718'13001 (192718'13000) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:308 2014-08-06 14:20:06.200841 2014-08-13 18:06:06.648911 7f5c973a57a0 20 read_log 192718'13002 (192718'13001) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:308 2014-08-06 14:20:06.217992 2014-08-13 18:06:06.648925 7f5c973a57a0 20 read_log 192718'13003 (192718'13002) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:309 2014-08-06 14:20:06.276018 2014-08-13 18:06:06.648938 7f5c973a57a0 20 read_log 192718'13004 (192718'13003) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:309 2014-08-06 14:20:06.276051 2014-08-13 18:06:06.648950 7f5c973a57a0 20 read_log 192718'13005 (192718'13004) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:310 2014-08-06 14:20:06.309307 2014-08-13 18:06:06.648973 7f5c973a57a0 20 read_log 192718'13006 (192718'13005) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:310 2014-08-06 14:20:06.399140 2014-08-13 18:06:06.648989 7f5c973a57a0 20 read_log 192718'13007 (192718'13006) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:311 2014-08-06 14:20:06.489160 2014-08-13 18:06:06.649005 7f5c973a57a0 20 read_log 192718'13008 (192718'13007) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:311 2014-08-06 14:20:06.554912 2014-08-13 18:06:06.649020 7f5c973a57a0 20 read_log 192718'13009 (192718'13008) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:312 2014-08-06 14:20:06.574163 2014-08-13 18:06:06.649056 7f5c973a57a0 20 read_log 192718'13010 (192718'13009) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:312 2014-08-06 14:20:06.600473 2014-08-13 18:06:06.649072 7f5c973a57a0 20 read_log 192718'13011 (192718'13010) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:313 2014-08-06 14:20:06.609504 2014-08-13 18:06:06.649085 7f5c973a57a0 20 read_log 192718'13012 (192718'13011) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:313 2014-08-06 14:20:06.627689 2014-08-13 18:06:06.649101 7f5c973a57a0 20 read_log 192718'13013 (192718'13012) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:314 2014-08-06 14:20:06.636894 2014-08-13 18:06:06.649114 7f5c973a57a0 20 read_log 192718'13014 (192718'13013) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:314 2014-08-06 14:20:06.658284 2014-08-13 18:06:06.649126 7f5c973a57a0 20 read_log 192718'13015 (192718'13014) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:315 2014-08-06 14:20:06.668907 2014-08-13 18:06:06.649171 7f5c973a57a0 20 read_log 192718'13016 (192718'13015) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:315 2014-08-06 14:20:06.693763 2014-08-13 18:06:06.649197 7f5c973a57a0 20 read_log 192718'13017 (192718'13016) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:316 2014-08-06 14:20:06.703144 2014-08-13 18:06:06.649215 7f5c973a57a0 20 read_log 192718'13018 (192718'13017) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:316 2014-08-06 14:20:06.780033 2014-08-13 18:06:06.649231 7f5c973a57a0 20 read_log 192718'13019 (192718'13018) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:317 2014-08-06 14:20:06.793939 2014-08-13 18:06:06.649246 7f5c973a57a0 20 read_log 192718'13020 (192718'13019) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:317 2014-08-06 14:20:06.815555 2014-08-13 18:06:06.649262 7f5c973a57a0 20 read_log 192718'13021 (192718'13020) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:318 2014-08-06 14:20:06.828865 2014-08-13 18:06:06.649277 7f5c973a57a0 20 read_log 192718'13022 (192718'13021) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:318 2014-08-06 14:20:06.846423 2014-08-13 18:06:06.649289 7f5c973a57a0 20 read_log 192718'13023 (192718'13022) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:319 2014-08-06 14:20:06.901604 2014-08-13 18:06:06.649302 7f5c973a57a0 20 read_log 192718'13024 (192718'13023) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:319 2014-08-06 14:20:06.901684 2014-08-13 18:06:06.649316 7f5c973a57a0 20 read_log 192718'13025 (192718'13024) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:320 2014-08-06 14:20:07.057175 2014-08-13 18:06:06.649329 7f5c973a57a0 20 read_log 192718'13026 (192718'13025) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:320 2014-08-06 14:20:07.089741 2014-08-13 18:06:06.649341 7f5c973a57a0 20 read_log 192718'13027 (192718'13026) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:321 2014-08-06 14:20:07.144198 2014-08-13 18:06:06.649362 7f5c973a57a0 20 read_log 192718'13028 (192718'13027) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:321 2014-08-06 14:20:07.273928 2014-08-13 18:06:06.649379 7f5c973a57a0 20 read_log 192718'13029 (192718'13028) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:322 2014-08-06 14:20:07.291482 2014-08-13 18:06:06.649391 7f5c973a57a0 20 read_log 192718'13030 (192718'13029) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:322 2014-08-06 14:20:07.317834 2014-08-13 18:06:06.649403 7f5c973a57a0 20 read_log 192718'13031 (192718'13030) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:323 2014-08-06 14:20:07.388201 2014-08-13 18:06:06.649416 7f5c973a57a0 20 read_log 192718'13032 (192718'13031) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:323 2014-08-06 14:20:07.388663 2014-08-13 18:06:06.649428 7f5c973a57a0 20 read_log 192718'13033 (192718'13032) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:324 2014-08-06 14:20:07.419091 2014-08-13 18:06:06.649440 7f5c973a57a0 20 read_log 192718'13034 (192718'13033) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:324 2014-08-06 14:20:07.426311 2014-08-13 18:06:06.649452 7f5c973a57a0 20 read_log 192718'13035 (192718'13034) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:325 2014-08-06 14:20:07.465989 2014-08-13 18:06:06.649464 7f5c973a57a0 20 read_log 192718'13036 (192718'13035) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:325 2014-08-06 14:20:07.477567 2014-08-13 18:06:06.649477 7f5c973a57a0 20 read_log 192718'13037 (192718'13036) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:326 2014-08-06 14:20:07.579013 2014-08-13 18:06:06.649489 7f5c973a57a0 20 read_log 192718'13038 (192718'13037) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:326 2014-08-06 14:20:07.589471 2014-08-13 18:06:06.649502 7f5c973a57a0 20 read_log 192718'13039 (192718'13038) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:327 2014-08-06 14:20:07.611334 2014-08-13 18:06:06.649515 7f5c973a57a0 20 read_log 192718'13040 (192718'13039) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:327 2014-08-06 14:20:07.619952 2014-08-13 18:06:06.649558 7f5c973a57a0 20 read_log 192718'13041 (192718'13040) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:328 2014-08-06 14:20:07.739858 2014-08-13 18:06:06.649574 7f5c973a57a0 20 read_log 192718'13042 (192718'13041) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:328 2014-08-06 14:20:07.751268 2014-08-13 18:06:06.649587 7f5c973a57a0 20 read_log 192718'13043 (192718'13042) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:329 2014-08-06 14:20:07.806577 2014-08-13 18:06:06.649599 7f5c973a57a0 20 read_log 192718'13044 (192718'13043) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:329 2014-08-06 14:20:07.806586 2014-08-13 18:06:06.649611 7f5c973a57a0 20 read_log 192718'13045 (192718'13044) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:330 2014-08-06 14:20:07.838454 2014-08-13 18:06:06.649623 7f5c973a57a0 20 read_log 192718'13046 (192718'13045) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:330 2014-08-06 14:20:07.845244 2014-08-13 18:06:06.649635 7f5c973a57a0 20 read_log 192718'13047 (192718'13046) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:331 2014-08-06 14:20:07.901697 2014-08-13 18:06:06.649648 7f5c973a57a0 20 read_log 192718'13048 (192718'13047) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:331 2014-08-06 14:20:07.923359 2014-08-13 18:06:06.649660 7f5c973a57a0 20 read_log 192718'13049 (192718'13048) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:332 2014-08-06 14:20:07.937178 2014-08-13 18:06:06.649672 7f5c973a57a0 20 read_log 192718'13050 (192718'13049) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:332 2014-08-06 14:20:07.959890 2014-08-13 18:06:06.649685 7f5c973a57a0 20 read_log 192718'13051 (192718'13050) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:333 2014-08-06 14:20:07.971420 2014-08-13 18:06:06.649700 7f5c973a57a0 20 read_log 192718'13052 (192718'13051) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:333 2014-08-06 14:20:07.993926 2014-08-13 18:06:06.649712 7f5c973a57a0 20 read_log 192718'13053 (192718'13052) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:334 2014-08-06 14:20:08.007654 2014-08-13 18:06:06.649726 7f5c973a57a0 20 read_log 192718'13054 (192718'13053) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:334 2014-08-06 14:20:08.028620 2014-08-13 18:06:06.649739 7f5c973a57a0 20 read_log 192718'13055 (192718'13054) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:335 2014-08-06 14:20:08.075338 2014-08-13 18:06:06.649752 7f5c973a57a0 20 read_log 192718'13056 (192718'13055) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:335 2014-08-06 14:20:08.075446 2014-08-13 18:06:06.649766 7f5c973a57a0 20 read_log 192718'13057 (192718'13056) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:336 2014-08-06 14:20:08.165676 2014-08-13 18:06:06.649780 7f5c973a57a0 20 read_log 192718'13058 (192718'13057) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:336 2014-08-06 14:20:08.189365 2014-08-13 18:06:06.649792 7f5c973a57a0 20 read_log 192718'13059 (192718'13058) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:337 2014-08-06 14:20:08.264223 2014-08-13 18:06:06.649805 7f5c973a57a0 20 read_log 192718'13060 (192718'13059) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:337 2014-08-06 14:20:08.339552 2014-08-13 18:06:06.649817 7f5c973a57a0 20 read_log 192718'13061 (192718'13060) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:338 2014-08-06 14:20:08.362492 2014-08-13 18:06:06.649831 7f5c973a57a0 20 read_log 192718'13062 (192718'13061) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:338 2014-08-06 14:20:08.380778 2014-08-13 18:06:06.649843 7f5c973a57a0 20 read_log 192718'13063 (192718'13062) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:339 2014-08-06 14:20:08.417732 2014-08-13 18:06:06.649857 7f5c973a57a0 20 read_log 192718'13064 (192718'13063) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:339 2014-08-06 14:20:08.434376 2014-08-13 18:06:06.649875 7f5c973a57a0 20 read_log 192718'13065 (192718'13064) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:340 2014-08-06 14:20:08.461221 2014-08-13 18:06:06.649890 7f5c973a57a0 20 read_log 192718'13066 (192718'13065) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:340 2014-08-06 14:20:08.477700 2014-08-13 18:06:06.649903 7f5c973a57a0 20 read_log 192718'13067 (192718'13066) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:341 2014-08-06 14:20:08.564697 2014-08-13 18:06:06.649918 7f5c973a57a0 20 read_log 192718'13068 (192718'13067) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:341 2014-08-06 14:20:08.590690 2014-08-13 18:06:06.649930 7f5c973a57a0 20 read_log 192718'13069 (192718'13068) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:342 2014-08-06 14:20:08.602875 2014-08-13 18:06:06.649945 7f5c973a57a0 20 read_log 192718'13070 (192718'13069) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:342 2014-08-06 14:20:08.624540 2014-08-13 18:06:06.649957 7f5c973a57a0 20 read_log 192718'13071 (192718'13070) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:343 2014-08-06 14:20:08.635698 2014-08-13 18:06:06.649971 7f5c973a57a0 20 read_log 192718'13072 (192718'13071) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:343 2014-08-06 14:20:08.659770 2014-08-13 18:06:06.649984 7f5c973a57a0 20 read_log 192718'13073 (192718'13072) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:344 2014-08-06 14:20:08.676278 2014-08-13 18:06:06.649998 7f5c973a57a0 20 read_log 192718'13074 (192718'13073) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:344 2014-08-06 14:20:08.699706 2014-08-13 18:06:06.650010 7f5c973a57a0 20 read_log 192718'13075 (192718'13074) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:345 2014-08-06 14:20:08.711488 2014-08-13 18:06:06.650024 7f5c973a57a0 20 read_log 192718'13076 (192718'13075) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:345 2014-08-06 14:20:08.730302 2014-08-13 18:06:06.650037 7f5c973a57a0 20 read_log 192718'13077 (192718'13076) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:346 2014-08-06 14:20:08.749711 2014-08-13 18:06:06.650051 7f5c973a57a0 20 read_log 192718'13078 (192718'13077) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:346 2014-08-06 14:20:08.811847 2014-08-13 18:06:06.650065 7f5c973a57a0 20 read_log 192718'13079 (192718'13078) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:347 2014-08-06 14:20:08.825709 2014-08-13 18:06:06.650078 7f5c973a57a0 20 read_log 192718'13080 (192718'13079) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:347 2014-08-06 14:20:08.953242 2014-08-13 18:06:06.650092 7f5c973a57a0 20 read_log 192718'13081 (192718'13080) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:348 2014-08-06 14:20:08.964048 2014-08-13 18:06:06.650104 7f5c973a57a0 20 read_log 192718'13082 (192718'13081) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:348 2014-08-06 14:20:08.984023 2014-08-13 18:06:06.650116 7f5c973a57a0 20 read_log 192718'13083 (192718'13082) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:349 2014-08-06 14:20:09.039562 2014-08-13 18:06:06.650129 7f5c973a57a0 20 read_log 192718'13084 (192718'13083) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:349 2014-08-06 14:20:09.039713 2014-08-13 18:06:06.650143 7f5c973a57a0 20 read_log 192718'13085 (192718'13084) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:350 2014-08-06 14:20:09.079962 2014-08-13 18:06:06.650168 7f5c973a57a0 20 read_log 192718'13086 (192718'13085) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:350 2014-08-06 14:20:09.080224 2014-08-13 18:06:06.650180 7f5c973a57a0 20 read_log 192718'13087 (192718'13086) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:351 2014-08-06 14:20:09.140757 2014-08-13 18:06:06.650195 7f5c973a57a0 20 read_log 192718'13088 (192718'13087) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:351 2014-08-06 14:20:09.140779 2014-08-13 18:06:06.650213 7f5c973a57a0 20 read_log 192718'13089 (192718'13088) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:352 2014-08-06 14:20:09.204070 2014-08-13 18:06:06.650233 7f5c973a57a0 20 read_log 192718'13090 (192718'13089) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:352 2014-08-06 14:20:09.204127 2014-08-13 18:06:06.650246 7f5c973a57a0 20 read_log 192718'13091 (192718'13090) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:353 2014-08-06 14:20:09.269091 2014-08-13 18:06:06.650263 7f5c973a57a0 20 read_log 192718'13092 (192718'13091) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:353 2014-08-06 14:20:09.269219 2014-08-13 18:06:06.650278 7f5c973a57a0 20 read_log 192718'13093 (192718'13092) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:354 2014-08-06 14:20:09.299962 2014-08-13 18:06:06.650290 7f5c973a57a0 20 read_log 192718'13094 (192718'13093) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:354 2014-08-06 14:20:09.301975 2014-08-13 18:06:06.650307 7f5c973a57a0 20 read_log 192718'13095 (192718'13094) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:355 2014-08-06 14:20:09.395279 2014-08-13 18:06:06.650323 7f5c973a57a0 20 read_log 192718'13096 (192718'13095) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:355 2014-08-06 14:20:09.395285 2014-08-13 18:06:06.650335 7f5c973a57a0 20 read_log 192718'13097 (192718'13096) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:356 2014-08-06 14:20:09.425471 2014-08-13 18:06:06.650348 7f5c973a57a0 20 read_log 192718'13098 (192718'13097) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:356 2014-08-06 14:20:09.428378 2014-08-13 18:06:06.650360 7f5c973a57a0 20 read_log 192718'13099 (192718'13098) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:357 2014-08-06 14:20:09.490485 2014-08-13 18:06:06.650372 7f5c973a57a0 20 read_log 192718'13100 (192718'13099) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:357 2014-08-06 14:20:09.490592 2014-08-13 18:06:06.650408 7f5c973a57a0 20 read_log 192718'13101 (192718'13100) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:358 2014-08-06 14:20:09.536306 2014-08-13 18:06:06.650424 7f5c973a57a0 20 read_log 192718'13102 (192718'13101) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:358 2014-08-06 14:20:09.537096 2014-08-13 18:06:06.650438 7f5c973a57a0 20 read_log 192718'13103 (192718'13102) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:359 2014-08-06 14:20:09.571163 2014-08-13 18:06:06.650470 7f5c973a57a0 20 read_log 192718'13104 (192718'13103) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:359 2014-08-06 14:20:09.572033 2014-08-13 18:06:06.650486 7f5c973a57a0 20 read_log 192718'13105 (192718'13104) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:360 2014-08-06 14:20:09.611825 2014-08-13 18:06:06.650515 7f5c973a57a0 20 read_log 192718'13106 (192718'13105) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:360 2014-08-06 14:20:09.611729 2014-08-13 18:06:06.650532 7f5c973a57a0 20 read_log 192718'13107 (192718'13106) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:361 2014-08-06 14:20:10.000184 2014-08-13 18:06:06.650552 7f5c973a57a0 20 read_log 192718'13108 (192718'13107) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:361 2014-08-06 14:20:10.000178 2014-08-13 18:06:06.650568 7f5c973a57a0 20 read_log 192718'13109 (192718'13108) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:362 2014-08-06 14:20:10.094805 2014-08-13 18:06:06.650626 7f5c973a57a0 20 read_log 192718'13110 (192718'13109) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:362 2014-08-06 14:20:10.094808 2014-08-13 18:06:06.650643 7f5c973a57a0 20 read_log 192718'13111 (192718'13110) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:363 2014-08-06 14:20:10.351913 2014-08-13 18:06:06.650655 7f5c973a57a0 20 read_log 192718'13112 (192718'13111) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:363 2014-08-06 14:20:10.351991 2014-08-13 18:06:06.650687 7f5c973a57a0 20 read_log 192718'13113 (192718'13112) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:364 2014-08-06 14:20:10.431603 2014-08-13 18:06:06.650702 7f5c973a57a0 20 read_log 192718'13114 (192718'13113) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:364 2014-08-06 14:20:10.431432 2014-08-13 18:06:06.650715 7f5c973a57a0 20 read_log 192718'13115 (192718'13114) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:365 2014-08-06 14:20:10.582076 2014-08-13 18:06:06.650727 7f5c973a57a0 20 read_log 192718'13116 (192718'13115) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:365 2014-08-06 14:20:10.582588 2014-08-13 18:06:06.650739 7f5c973a57a0 20 read_log 192718'13117 (192718'13116) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:366 2014-08-06 14:20:10.637194 2014-08-13 18:06:06.650751 7f5c973a57a0 20 read_log 192718'13118 (192718'13117) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:366 2014-08-06 14:20:10.637194 2014-08-13 18:06:06.650763 7f5c973a57a0 20 read_log 192718'13119 (192718'13118) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:367 2014-08-06 14:20:10.713664 2014-08-13 18:06:06.650775 7f5c973a57a0 20 read_log 192718'13120 (192718'13119) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:367 2014-08-06 14:20:10.713433 2014-08-13 18:06:06.650787 7f5c973a57a0 20 read_log 192718'13121 (192718'13120) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:368 2014-08-06 14:20:10.743622 2014-08-13 18:06:06.650799 7f5c973a57a0 20 read_log 192718'13122 (192718'13121) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:368 2014-08-06 14:20:10.745499 2014-08-13 18:06:06.650811 7f5c973a57a0 20 read_log 192718'13123 (192718'13122) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:369 2014-08-06 14:20:10.914536 2014-08-13 18:06:06.650823 7f5c973a57a0 20 read_log 192718'13124 (192718'13123) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:369 2014-08-06 14:20:10.914468 2014-08-13 18:06:06.650835 7f5c973a57a0 20 read_log 192718'13125 (192718'13124) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:370 2014-08-06 14:20:10.947506 2014-08-13 18:06:06.650847 7f5c973a57a0 20 read_log 192718'13126 (192718'13125) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:370 2014-08-06 14:20:10.948018 2014-08-13 18:06:06.650859 7f5c973a57a0 20 read_log 192718'13127 (192718'13126) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:371 2014-08-06 14:20:10.980271 2014-08-13 18:06:06.650871 7f5c973a57a0 20 read_log 192718'13128 (192718'13127) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:371 2014-08-06 14:20:10.982385 2014-08-13 18:06:06.650906 7f5c973a57a0 20 read_log 192718'13129 (192718'13128) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:372 2014-08-06 14:20:11.011108 2014-08-13 18:06:06.650919 7f5c973a57a0 20 read_log 192718'13130 (192718'13129) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:372 2014-08-06 14:20:11.013235 2014-08-13 18:06:06.650931 7f5c973a57a0 20 read_log 192718'13131 (192718'13130) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:373 2014-08-06 14:20:11.069912 2014-08-13 18:06:06.650943 7f5c973a57a0 20 read_log 192718'13132 (192718'13131) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:373 2014-08-06 14:20:11.069767 2014-08-13 18:06:06.650955 7f5c973a57a0 20 read_log 192718'13133 (192718'13132) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:374 2014-08-06 14:20:11.100851 2014-08-13 18:06:06.650967 7f5c973a57a0 20 read_log 192718'13134 (192718'13133) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:374 2014-08-06 14:20:11.103083 2014-08-13 18:06:06.650979 7f5c973a57a0 20 read_log 192718'13135 (192718'13134) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:375 2014-08-06 14:20:11.128235 2014-08-13 18:06:06.650991 7f5c973a57a0 20 read_log 192718'13136 (192718'13135) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:375 2014-08-06 14:20:11.129689 2014-08-13 18:06:06.651012 7f5c973a57a0 20 read_log 192718'13137 (192718'13136) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:376 2014-08-06 14:20:11.154357 2014-08-13 18:06:06.651027 7f5c973a57a0 20 read_log 192718'13138 (192718'13137) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:376 2014-08-06 14:20:11.159111 2014-08-13 18:06:06.651040 7f5c973a57a0 20 read_log 192718'13139 (192718'13138) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:377 2014-08-06 14:20:11.190607 2014-08-13 18:06:06.651052 7f5c973a57a0 20 read_log 192718'13140 (192718'13139) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:377 2014-08-06 14:20:11.190954 2014-08-13 18:06:06.651066 7f5c973a57a0 20 read_log 192718'13141 (192718'13140) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:378 2014-08-06 14:20:11.220630 2014-08-13 18:06:06.651079 7f5c973a57a0 20 read_log 192718'13142 (192718'13141) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:378 2014-08-06 14:20:11.222915 2014-08-13 18:06:06.651091 7f5c973a57a0 20 read_log 192718'13143 (192718'13142) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:379 2014-08-06 14:20:11.251209 2014-08-13 18:06:06.651105 7f5c973a57a0 20 read_log 192718'13144 (192718'13143) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:379 2014-08-06 14:20:11.253554 2014-08-13 18:06:06.651117 7f5c973a57a0 20 read_log 192718'13145 (192718'13144) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:380 2014-08-06 14:20:11.281940 2014-08-13 18:06:06.651131 7f5c973a57a0 20 read_log 192718'13146 (192718'13145) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:380 2014-08-06 14:20:11.284026 2014-08-13 18:06:06.651150 7f5c973a57a0 20 read_log 192718'13147 (192718'13146) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:381 2014-08-06 14:20:11.315082 2014-08-13 18:06:06.651166 7f5c973a57a0 20 read_log 192718'13148 (192718'13147) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:381 2014-08-06 14:20:11.317394 2014-08-13 18:06:06.651178 7f5c973a57a0 20 read_log 192718'13149 (192718'13148) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:382 2014-08-06 14:20:11.346358 2014-08-13 18:06:06.651193 7f5c973a57a0 20 read_log 192718'13150 (192718'13149) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:382 2014-08-06 14:20:11.348366 2014-08-13 18:06:06.651205 7f5c973a57a0 20 read_log 192718'13151 (192718'13150) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:383 2014-08-06 14:20:11.402720 2014-08-13 18:06:06.651219 7f5c973a57a0 20 read_log 192718'13152 (192718'13151) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:383 2014-08-06 14:20:11.401728 2014-08-13 18:06:06.651231 7f5c973a57a0 20 read_log 192718'13153 (192718'13152) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:384 2014-08-06 14:20:11.451756 2014-08-13 18:06:06.651244 7f5c973a57a0 20 read_log 192718'13154 (192718'13153) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:384 2014-08-06 14:20:11.453180 2014-08-13 18:06:06.651256 7f5c973a57a0 20 read_log 192718'13155 (192718'13154) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:385 2014-08-06 14:20:11.508806 2014-08-13 18:06:06.651271 7f5c973a57a0 20 read_log 192718'13156 (192718'13155) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:385 2014-08-06 14:20:11.514143 2014-08-13 18:06:06.651283 7f5c973a57a0 20 read_log 192718'13157 (192718'13156) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:386 2014-08-06 14:20:11.567447 2014-08-13 18:06:06.651296 7f5c973a57a0 20 read_log 192718'13158 (192718'13157) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:386 2014-08-06 14:20:11.567521 2014-08-13 18:06:06.651308 7f5c973a57a0 20 read_log 192718'13159 (192718'13158) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:387 2014-08-06 14:20:11.615442 2014-08-13 18:06:06.651320 7f5c973a57a0 20 read_log 192718'13160 (192718'13159) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:387 2014-08-06 14:20:11.614314 2014-08-13 18:06:06.651338 7f5c973a57a0 20 read_log 192718'13161 (192718'13160) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:388 2014-08-06 14:20:11.662576 2014-08-13 18:06:06.651353 7f5c973a57a0 20 read_log 192718'13162 (192718'13161) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:388 2014-08-06 14:20:11.662637 2014-08-13 18:06:06.651365 7f5c973a57a0 20 read_log 192718'13163 (192718'13162) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:389 2014-08-06 14:20:11.739290 2014-08-13 18:06:06.651377 7f5c973a57a0 20 read_log 192718'13164 (192718'13163) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:389 2014-08-06 14:20:11.739303 2014-08-13 18:06:06.651402 7f5c973a57a0 20 read_log 192718'13165 (192718'13164) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:390 2014-08-06 14:20:11.792856 2014-08-13 18:06:06.651422 7f5c973a57a0 20 read_log 192718'13166 (192718'13165) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:390 2014-08-06 14:20:11.795751 2014-08-13 18:06:06.651448 7f5c973a57a0 20 read_log 192718'13167 (192718'13166) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:391 2014-08-06 14:20:11.846207 2014-08-13 18:06:06.651461 7f5c973a57a0 20 read_log 192718'13168 (192718'13167) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:391 2014-08-06 14:20:11.848697 2014-08-13 18:06:06.651479 7f5c973a57a0 20 read_log 192718'13169 (192718'13168) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:392 2014-08-06 14:20:12.103843 2014-08-13 18:06:06.651491 7f5c973a57a0 20 read_log 192718'13170 (192718'13169) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:392 2014-08-06 14:20:12.103852 2014-08-13 18:06:06.651504 7f5c973a57a0 20 read_log 192718'13171 (192718'13170) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:393 2014-08-06 14:20:12.156596 2014-08-13 18:06:06.651516 7f5c973a57a0 20 read_log 192718'13172 (192718'13171) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:393 2014-08-06 14:20:12.156630 2014-08-13 18:06:06.651528 7f5c973a57a0 20 read_log 192718'13173 (192718'13172) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:394 2014-08-06 14:20:12.206735 2014-08-13 18:06:06.651540 7f5c973a57a0 20 read_log 192718'13174 (192718'13173) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:394 2014-08-06 14:20:12.235698 2014-08-13 18:06:06.651552 7f5c973a57a0 20 read_log 192718'13175 (192718'13174) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:395 2014-08-06 14:20:12.438917 2014-08-13 18:06:06.651564 7f5c973a57a0 20 read_log 192718'13176 (192718'13175) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:395 2014-08-06 14:20:12.467530 2014-08-13 18:06:06.651577 7f5c973a57a0 20 read_log 192718'13177 (192718'13176) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:396 2014-08-06 14:20:12.547791 2014-08-13 18:06:06.651588 7f5c973a57a0 20 read_log 192718'13178 (192718'13177) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:396 2014-08-06 14:20:12.607150 2014-08-13 18:06:06.651600 7f5c973a57a0 20 read_log 192718'13179 (192718'13178) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:397 2014-08-06 14:20:12.668453 2014-08-13 18:06:06.651620 7f5c973a57a0 20 read_log 192718'13180 (192718'13179) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:397 2014-08-06 14:20:12.678935 2014-08-13 18:06:06.651634 7f5c973a57a0 20 read_log 192718'13181 (192718'13180) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:398 2014-08-06 14:20:12.745267 2014-08-13 18:06:06.651646 7f5c973a57a0 20 read_log 192718'13182 (192718'13181) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:398 2014-08-06 14:20:12.751988 2014-08-13 18:06:06.651659 7f5c973a57a0 20 read_log 192718'13183 (192718'13182) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:399 2014-08-06 14:20:13.044516 2014-08-13 18:06:06.651671 7f5c973a57a0 20 read_log 192718'13184 (192718'13183) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:399 2014-08-06 14:20:13.106547 2014-08-13 18:06:06.651689 7f5c973a57a0 20 read_log 192718'13185 (192718'13184) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:400 2014-08-06 14:20:13.186056 2014-08-13 18:06:06.651711 7f5c973a57a0 20 read_log 192718'13186 (192718'13185) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:400 2014-08-06 14:20:13.425274 2014-08-13 18:06:06.651728 7f5c973a57a0 20 read_log 192718'13187 (192718'13186) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:401 2014-08-06 14:20:13.615007 2014-08-13 18:06:06.651741 7f5c973a57a0 20 read_log 192718'13188 (192718'13187) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:401 2014-08-06 14:20:13.618620 2014-08-13 18:06:06.651753 7f5c973a57a0 20 read_log 192718'13189 (192718'13188) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:402 2014-08-06 14:20:13.697778 2014-08-13 18:06:06.651765 7f5c973a57a0 20 read_log 192718'13190 (192718'13189) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:402 2014-08-06 14:20:13.737969 2014-08-13 18:06:06.651778 7f5c973a57a0 20 read_log 192718'13191 (192718'13190) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:403 2014-08-06 14:20:13.808267 2014-08-13 18:06:06.651790 7f5c973a57a0 20 read_log 192718'13192 (192718'13191) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:403 2014-08-06 14:20:13.890650 2014-08-13 18:06:06.651802 7f5c973a57a0 20 read_log 192718'13193 (192718'13192) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:404 2014-08-06 14:20:13.975790 2014-08-13 18:06:06.651814 7f5c973a57a0 20 read_log 192718'13194 (192718'13193) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:404 2014-08-06 14:20:14.079804 2014-08-13 18:06:06.651826 7f5c973a57a0 20 read_log 192718'13195 (192718'13194) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:405 2014-08-06 14:20:14.127946 2014-08-13 18:06:06.651839 7f5c973a57a0 20 read_log 192718'13196 (192718'13195) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:405 2014-08-06 14:20:14.127851 2014-08-13 18:06:06.651851 7f5c973a57a0 20 read_log 192718'13197 (192718'13196) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:406 2014-08-06 14:20:14.172724 2014-08-13 18:06:06.651863 7f5c973a57a0 20 read_log 192718'13198 (192718'13197) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:406 2014-08-06 14:20:14.172866 2014-08-13 18:06:06.651875 7f5c973a57a0 20 read_log 192718'13199 (192718'13198) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:407 2014-08-06 14:20:14.199763 2014-08-13 18:06:06.651887 7f5c973a57a0 20 read_log 192718'13200 (192718'13199) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:407 2014-08-06 14:20:14.201338 2014-08-13 18:06:06.651899 7f5c973a57a0 20 read_log 192718'13201 (192718'13200) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:408 2014-08-06 14:20:14.229077 2014-08-13 18:06:06.651911 7f5c973a57a0 20 read_log 192718'13202 (192718'13201) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:408 2014-08-06 14:20:14.247520 2014-08-13 18:06:06.651943 7f5c973a57a0 20 read_log 192718'13203 (192718'13202) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:409 2014-08-06 14:20:14.272784 2014-08-13 18:06:06.651958 7f5c973a57a0 20 read_log 192718'13204 (192718'13203) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:409 2014-08-06 14:20:14.365455 2014-08-13 18:06:06.651971 7f5c973a57a0 20 read_log 192718'13205 (192718'13204) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:410 2014-08-06 14:20:14.385654 2014-08-13 18:06:06.651985 7f5c973a57a0 20 read_log 192718'13206 (192718'13205) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:410 2014-08-06 14:20:14.407133 2014-08-13 18:06:06.651998 7f5c973a57a0 20 read_log 192718'13207 (192718'13206) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:411 2014-08-06 14:20:14.444147 2014-08-13 18:06:06.652012 7f5c973a57a0 20 read_log 192718'13208 (192718'13207) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:411 2014-08-06 14:20:14.448299 2014-08-13 18:06:06.652049 7f5c973a57a0 20 read_log 192718'13209 (192718'13208) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:412 2014-08-06 14:20:14.494147 2014-08-13 18:06:06.652065 7f5c973a57a0 20 read_log 192718'13210 (192718'13209) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:412 2014-08-06 14:20:14.494176 2014-08-13 18:06:06.652077 7f5c973a57a0 20 read_log 192718'13211 (192718'13210) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:413 2014-08-06 14:20:14.551139 2014-08-13 18:06:06.652092 7f5c973a57a0 20 read_log 192718'13212 (192718'13211) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:413 2014-08-06 14:20:14.565780 2014-08-13 18:06:06.652105 7f5c973a57a0 20 read_log 192718'13213 (192718'13212) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:414 2014-08-06 14:20:14.632375 2014-08-13 18:06:06.652119 7f5c973a57a0 20 read_log 192718'13214 (192718'13213) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:414 2014-08-06 14:20:14.632434 2014-08-13 18:06:06.652131 7f5c973a57a0 20 read_log 192718'13215 (192718'13214) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:415 2014-08-06 14:20:14.680961 2014-08-13 18:06:06.652150 7f5c973a57a0 20 read_log 192718'13216 (192718'13215) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:415 2014-08-06 14:20:14.681021 2014-08-13 18:06:06.652166 7f5c973a57a0 20 read_log 192718'13217 (192718'13216) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:416 2014-08-06 14:20:14.731058 2014-08-13 18:06:06.652178 7f5c973a57a0 20 read_log 192718'13218 (192718'13217) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:416 2014-08-06 14:20:14.731074 2014-08-13 18:06:06.652192 7f5c973a57a0 20 read_log 192718'13219 (192718'13218) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:417 2014-08-06 14:20:14.782515 2014-08-13 18:06:06.652205 7f5c973a57a0 20 read_log 192718'13220 (192718'13219) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:417 2014-08-06 14:20:14.781787 2014-08-13 18:06:06.652219 7f5c973a57a0 20 read_log 192718'13221 (192718'13220) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:418 2014-08-06 14:20:14.832517 2014-08-13 18:06:06.652231 7f5c973a57a0 20 read_log 192718'13222 (192718'13221) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:418 2014-08-06 14:20:14.832561 2014-08-13 18:06:06.652246 7f5c973a57a0 20 read_log 192718'13223 (192718'13222) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:419 2014-08-06 14:20:14.882486 2014-08-13 18:06:06.652260 7f5c973a57a0 20 read_log 192718'13224 (192718'13223) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:419 2014-08-06 14:20:14.882751 2014-08-13 18:06:06.652276 7f5c973a57a0 20 read_log 192718'13225 (192718'13224) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:420 2014-08-06 14:20:14.941376 2014-08-13 18:06:06.652288 7f5c973a57a0 20 read_log 192718'13226 (192718'13225) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:420 2014-08-06 14:20:14.941317 2014-08-13 18:06:06.652300 7f5c973a57a0 20 read_log 192718'13227 (192718'13226) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:421 2014-08-06 14:20:14.994070 2014-08-13 18:06:06.652312 7f5c973a57a0 20 read_log 192718'13228 (192718'13227) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:421 2014-08-06 14:20:14.994051 2014-08-13 18:06:06.652325 7f5c973a57a0 20 read_log 192718'13229 (192718'13228) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:422 2014-08-06 14:20:15.046136 2014-08-13 18:06:06.652337 7f5c973a57a0 20 read_log 192718'13230 (192718'13229) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:422 2014-08-06 14:20:15.046169 2014-08-13 18:06:06.652349 7f5c973a57a0 20 read_log 192718'13231 (192718'13230) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:423 2014-08-06 14:20:15.094654 2014-08-13 18:06:06.652366 7f5c973a57a0 20 read_log 192718'13232 (192718'13231) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:423 2014-08-06 14:20:15.094553 2014-08-13 18:06:06.652385 7f5c973a57a0 20 read_log 192718'13233 (192718'13232) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:424 2014-08-06 14:20:15.145974 2014-08-13 18:06:06.652407 7f5c973a57a0 20 read_log 192718'13234 (192718'13233) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:424 2014-08-06 14:20:15.145912 2014-08-13 18:06:06.652423 7f5c973a57a0 20 read_log 192718'13235 (192718'13234) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:425 2014-08-06 14:20:15.195629 2014-08-13 18:06:06.652435 7f5c973a57a0 20 read_log 192718'13236 (192718'13235) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:425 2014-08-06 14:20:15.195589 2014-08-13 18:06:06.652464 7f5c973a57a0 20 read_log 192718'13237 (192718'13236) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:426 2014-08-06 14:20:15.239049 2014-08-13 18:06:06.652476 7f5c973a57a0 20 read_log 192718'13238 (192718'13237) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:426 2014-08-06 14:20:15.239021 2014-08-13 18:06:06.652489 7f5c973a57a0 20 read_log 192718'13239 (192718'13238) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:427 2014-08-06 14:20:15.293310 2014-08-13 18:06:06.652501 7f5c973a57a0 20 read_log 192718'13240 (192718'13239) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:427 2014-08-06 14:20:15.293310 2014-08-13 18:06:06.652514 7f5c973a57a0 20 read_log 192718'13241 (192718'13240) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:428 2014-08-06 14:20:15.345473 2014-08-13 18:06:06.652526 7f5c973a57a0 20 read_log 192718'13242 (192718'13241) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:428 2014-08-06 14:20:15.345644 2014-08-13 18:06:06.652547 7f5c973a57a0 20 read_log 192718'13243 (192718'13242) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:429 2014-08-06 14:20:15.404278 2014-08-13 18:06:06.652563 7f5c973a57a0 20 read_log 192718'13244 (192718'13243) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:429 2014-08-06 14:20:15.404545 2014-08-13 18:06:06.652575 7f5c973a57a0 20 read_log 192718'13245 (192718'13244) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:430 2014-08-06 14:20:15.456003 2014-08-13 18:06:06.652588 7f5c973a57a0 20 read_log 192718'13246 (192718'13245) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:430 2014-08-06 14:20:15.456027 2014-08-13 18:06:06.652600 7f5c973a57a0 20 read_log 192718'13247 (192718'13246) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:431 2014-08-06 14:20:15.512751 2014-08-13 18:06:06.652612 7f5c973a57a0 20 read_log 192718'13248 (192718'13247) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:431 2014-08-06 14:20:15.513907 2014-08-13 18:06:06.652629 7f5c973a57a0 20 read_log 192718'13249 (192718'13248) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:432 2014-08-06 14:20:15.564486 2014-08-13 18:06:06.652642 7f5c973a57a0 20 read_log 192718'13250 (192718'13249) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:432 2014-08-06 14:20:15.564445 2014-08-13 18:06:06.652661 7f5c973a57a0 20 read_log 192718'13251 (192718'13250) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:433 2014-08-06 14:20:15.616748 2014-08-13 18:06:06.652677 7f5c973a57a0 20 read_log 192718'13252 (192718'13251) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:433 2014-08-06 14:20:15.630699 2014-08-13 18:06:06.652689 7f5c973a57a0 20 read_log 192718'13253 (192718'13252) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:434 2014-08-06 14:20:15.665899 2014-08-13 18:06:06.652701 7f5c973a57a0 20 read_log 192718'13254 (192718'13253) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:434 2014-08-06 14:20:15.665848 2014-08-13 18:06:06.652713 7f5c973a57a0 20 read_log 192718'13255 (192718'13254) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:435 2014-08-06 14:20:15.717277 2014-08-13 18:06:06.652725 7f5c973a57a0 20 read_log 192718'13256 (192718'13255) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:435 2014-08-06 14:20:15.717310 2014-08-13 18:06:06.652744 7f5c973a57a0 20 read_log 192718'13257 (192718'13256) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:436 2014-08-06 14:20:15.768892 2014-08-13 18:06:06.652759 7f5c973a57a0 20 read_log 192718'13258 (192718'13257) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:436 2014-08-06 14:20:15.769144 2014-08-13 18:06:06.652771 7f5c973a57a0 20 read_log 192718'13259 (192718'13258) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:437 2014-08-06 14:20:15.798405 2014-08-13 18:06:06.652783 7f5c973a57a0 20 read_log 192718'13260 (192718'13259) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:437 2014-08-06 14:20:15.800263 2014-08-13 18:06:06.652795 7f5c973a57a0 20 read_log 192718'13261 (192718'13260) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:438 2014-08-06 14:20:15.828327 2014-08-13 18:06:06.652807 7f5c973a57a0 20 read_log 192718'13262 (192718'13261) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:438 2014-08-06 14:20:15.829748 2014-08-13 18:06:06.652819 7f5c973a57a0 20 read_log 192718'13263 (192718'13262) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:439 2014-08-06 14:20:15.938156 2014-08-13 18:06:06.652831 7f5c973a57a0 20 read_log 192718'13264 (192718'13263) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:439 2014-08-06 14:20:15.985137 2014-08-13 18:06:06.652843 7f5c973a57a0 20 read_log 192718'13265 (192718'13264) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:440 2014-08-06 14:20:16.014234 2014-08-13 18:06:06.652856 7f5c973a57a0 20 read_log 192718'13266 (192718'13265) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:440 2014-08-06 14:20:16.038986 2014-08-13 18:06:06.652868 7f5c973a57a0 20 read_log 192718'13267 (192718'13266) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:441 2014-08-06 14:20:16.076662 2014-08-13 18:06:06.652880 7f5c973a57a0 20 read_log 192718'13268 (192718'13267) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:441 2014-08-06 14:20:16.090390 2014-08-13 18:06:06.652892 7f5c973a57a0 20 read_log 192718'13269 (192718'13268) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:442 2014-08-06 14:20:16.139959 2014-08-13 18:06:06.652904 7f5c973a57a0 20 read_log 192718'13270 (192718'13269) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:442 2014-08-06 14:20:16.225427 2014-08-13 18:06:06.652916 7f5c973a57a0 20 read_log 192718'13271 (192718'13270) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:443 2014-08-06 14:20:16.302484 2014-08-13 18:06:06.652928 7f5c973a57a0 20 read_log 192718'13272 (192718'13271) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:443 2014-08-06 14:20:16.328123 2014-08-13 18:06:06.652940 7f5c973a57a0 20 read_log 192718'13273 (192718'13272) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:444 2014-08-06 14:20:16.391595 2014-08-13 18:06:06.652952 7f5c973a57a0 20 read_log 192718'13274 (192718'13273) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:444 2014-08-06 14:20:16.410360 2014-08-13 18:06:06.652964 7f5c973a57a0 20 read_log 192718'13275 (192718'13274) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:445 2014-08-06 14:20:16.430529 2014-08-13 18:06:06.652976 7f5c973a57a0 20 read_log 192718'13276 (192718'13275) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:445 2014-08-06 14:20:16.443579 2014-08-13 18:06:06.652991 7f5c973a57a0 20 read_log 192718'13277 (192718'13276) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:446 2014-08-06 14:20:16.460460 2014-08-13 18:06:06.653003 7f5c973a57a0 20 read_log 192718'13278 (192718'13277) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:446 2014-08-06 14:20:16.474248 2014-08-13 18:06:06.653016 7f5c973a57a0 20 read_log 192718'13279 (192718'13278) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:447 2014-08-06 14:20:16.514000 2014-08-13 18:06:06.653028 7f5c973a57a0 20 read_log 192718'13280 (192718'13279) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:447 2014-08-06 14:20:16.537587 2014-08-13 18:06:06.653046 7f5c973a57a0 20 read_log 192718'13281 (192718'13280) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:448 2014-08-06 14:20:16.565310 2014-08-13 18:06:06.653061 7f5c973a57a0 20 read_log 192718'13282 (192718'13281) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:448 2014-08-06 14:20:16.590346 2014-08-13 18:06:06.653074 7f5c973a57a0 20 read_log 192718'13283 (192718'13282) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:449 2014-08-06 14:20:16.623875 2014-08-13 18:06:06.653088 7f5c973a57a0 20 read_log 192718'13284 (192718'13283) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:449 2014-08-06 14:20:16.779209 2014-08-13 18:06:06.653100 7f5c973a57a0 20 read_log 192718'13285 (192718'13284) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:450 2014-08-06 14:20:16.881824 2014-08-13 18:06:06.653112 7f5c973a57a0 20 read_log 192718'13286 (192718'13285) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:450 2014-08-06 14:20:17.005386 2014-08-13 18:06:06.653126 7f5c973a57a0 20 read_log 192718'13287 (192718'13286) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:451 2014-08-06 14:20:17.119132 2014-08-13 18:06:06.653139 7f5c973a57a0 20 read_log 192718'13288 (192718'13287) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:451 2014-08-06 14:20:17.165051 2014-08-13 18:06:06.653164 7f5c973a57a0 20 read_log 192718'13289 (192718'13288) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:452 2014-08-06 14:20:17.209092 2014-08-13 18:06:06.653179 7f5c973a57a0 20 read_log 192718'13290 (192718'13289) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:452 2014-08-06 14:20:17.238485 2014-08-13 18:06:06.653191 7f5c973a57a0 20 read_log 192718'13291 (192718'13290) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:453 2014-08-06 14:20:17.279006 2014-08-13 18:06:06.653205 7f5c973a57a0 20 read_log 192718'13292 (192718'13291) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:453 2014-08-06 14:20:17.319177 2014-08-13 18:06:06.653218 7f5c973a57a0 20 read_log 192718'13293 (192718'13292) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:454 2014-08-06 14:20:17.433741 2014-08-13 18:06:06.653231 7f5c973a57a0 20 read_log 192718'13294 (192718'13293) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:454 2014-08-06 14:20:17.639748 2014-08-13 18:06:06.653244 7f5c973a57a0 20 read_log 192718'13295 (192718'13294) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:455 2014-08-06 14:20:17.730929 2014-08-13 18:06:06.653261 7f5c973a57a0 20 read_log 192718'13296 (192718'13295) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:455 2014-08-06 14:20:17.893304 2014-08-13 18:06:06.653277 7f5c973a57a0 20 read_log 192718'13297 (192718'13296) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:456 2014-08-06 14:20:18.021243 2014-08-13 18:06:06.653289 7f5c973a57a0 20 read_log 192718'13298 (192718'13297) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:456 2014-08-06 14:20:18.091822 2014-08-13 18:06:06.653301 7f5c973a57a0 20 read_log 192718'13299 (192718'13298) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:457 2014-08-06 14:20:18.138525 2014-08-13 18:06:06.653317 7f5c973a57a0 20 read_log 192718'13300 (192718'13299) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:457 2014-08-06 14:20:18.198240 2014-08-13 18:06:06.653330 7f5c973a57a0 20 read_log 192718'13301 (192718'13300) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:458 2014-08-06 14:20:18.312164 2014-08-13 18:06:06.653344 7f5c973a57a0 20 read_log 192718'13302 (192718'13301) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:458 2014-08-06 14:20:18.423236 2014-08-13 18:06:06.653360 7f5c973a57a0 20 read_log 192718'13303 (192718'13302) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:459 2014-08-06 14:20:18.866107 2014-08-13 18:06:06.653373 7f5c973a57a0 20 read_log 192718'13304 (192718'13303) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:459 2014-08-06 14:20:19.084168 2014-08-13 18:06:06.653410 7f5c973a57a0 20 read_log 192718'13305 (192718'13304) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:460 2014-08-06 14:20:19.154382 2014-08-13 18:06:06.653425 7f5c973a57a0 20 read_log 192718'13306 (192718'13305) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:460 2014-08-06 14:20:19.236461 2014-08-13 18:06:06.653449 7f5c973a57a0 20 read_log 192718'13307 (192718'13306) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:461 2014-08-06 14:20:19.290831 2014-08-13 18:06:06.653462 7f5c973a57a0 20 read_log 192718'13308 (192718'13307) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:461 2014-08-06 14:20:19.314630 2014-08-13 18:06:06.653475 7f5c973a57a0 20 read_log 192718'13309 (192718'13308) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:462 2014-08-06 14:20:19.350155 2014-08-13 18:06:06.653497 7f5c973a57a0 20 read_log 192718'13310 (192718'13309) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:462 2014-08-06 14:20:19.368028 2014-08-13 18:06:06.653510 7f5c973a57a0 20 read_log 192718'13311 (192718'13310) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:463 2014-08-06 14:20:19.394650 2014-08-13 18:06:06.653522 7f5c973a57a0 20 read_log 192718'13312 (192718'13311) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:463 2014-08-06 14:20:19.418823 2014-08-13 18:06:06.653534 7f5c973a57a0 20 read_log 192718'13313 (192718'13312) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:464 2014-08-06 14:20:19.449666 2014-08-13 18:06:06.653546 7f5c973a57a0 20 read_log 192718'13314 (192718'13313) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:464 2014-08-06 14:20:19.600916 2014-08-13 18:06:06.653559 7f5c973a57a0 20 read_log 192718'13315 (192718'13314) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:465 2014-08-06 14:20:19.657144 2014-08-13 18:06:06.653571 7f5c973a57a0 20 read_log 192718'13316 (192718'13315) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:465 2014-08-06 14:20:19.708572 2014-08-13 18:06:06.653583 7f5c973a57a0 20 read_log 192718'13317 (192718'13316) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:466 2014-08-06 14:20:19.763741 2014-08-13 18:06:06.653602 7f5c973a57a0 20 read_log 192718'13318 (192718'13317) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:466 2014-08-06 14:20:19.787162 2014-08-13 18:06:06.653616 7f5c973a57a0 20 read_log 192718'13319 (192718'13318) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:467 2014-08-06 14:20:19.823020 2014-08-13 18:06:06.653628 7f5c973a57a0 20 read_log 192718'13320 (192718'13319) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:467 2014-08-06 14:20:19.840519 2014-08-13 18:06:06.653640 7f5c973a57a0 20 read_log 192718'13321 (192718'13320) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:468 2014-08-06 14:20:19.866867 2014-08-13 18:06:06.653657 7f5c973a57a0 20 read_log 192718'13322 (192718'13321) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:468 2014-08-06 14:20:19.891767 2014-08-13 18:06:06.653670 7f5c973a57a0 20 read_log 192718'13323 (192718'13322) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:469 2014-08-06 14:20:19.927191 2014-08-13 18:06:06.653682 7f5c973a57a0 20 read_log 192718'13324 (192718'13323) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:469 2014-08-06 14:20:20.064167 2014-08-13 18:06:06.653695 7f5c973a57a0 20 read_log 192718'13325 (192718'13324) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:470 2014-08-06 14:20:20.142122 2014-08-13 18:06:06.653707 7f5c973a57a0 20 read_log 192718'13326 (192718'13325) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:470 2014-08-06 14:20:20.315482 2014-08-13 18:06:06.653730 7f5c973a57a0 20 read_log 192718'13327 (192718'13326) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:471 2014-08-06 14:20:20.363447 2014-08-13 18:06:06.653743 7f5c973a57a0 20 read_log 192718'13328 (192718'13327) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:471 2014-08-06 14:20:20.413264 2014-08-13 18:06:06.653762 7f5c973a57a0 20 read_log 192718'13329 (192718'13328) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:472 2014-08-06 14:20:20.532506 2014-08-13 18:06:06.653777 7f5c973a57a0 20 read_log 192718'13330 (192718'13329) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:472 2014-08-06 14:20:20.587426 2014-08-13 18:06:06.653789 7f5c973a57a0 20 read_log 192718'13331 (192718'13330) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:473 2014-08-06 14:20:20.621479 2014-08-13 18:06:06.653802 7f5c973a57a0 20 read_log 192718'13332 (192718'13331) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:473 2014-08-06 14:20:20.660494 2014-08-13 18:06:06.653814 7f5c973a57a0 20 read_log 192718'13333 (192718'13332) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:474 2014-08-06 14:20:20.813856 2014-08-13 18:06:06.653826 7f5c973a57a0 20 read_log 192718'13334 (192718'13333) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:474 2014-08-06 14:20:20.940826 2014-08-13 18:06:06.653838 7f5c973a57a0 20 read_log 192718'13335 (192718'13334) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:475 2014-08-06 14:20:20.962441 2014-08-13 18:06:06.653853 7f5c973a57a0 20 read_log 192718'13336 (192718'13335) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:475 2014-08-06 14:20:20.981178 2014-08-13 18:06:06.653865 7f5c973a57a0 20 read_log 192718'13337 (192718'13336) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:476 2014-08-06 14:20:21.001847 2014-08-13 18:06:06.653878 7f5c973a57a0 20 read_log 192718'13338 (192718'13337) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:476 2014-08-06 14:20:21.031990 2014-08-13 18:06:06.653890 7f5c973a57a0 20 read_log 192718'13339 (192718'13338) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:477 2014-08-06 14:20:21.068641 2014-08-13 18:06:06.653902 7f5c973a57a0 20 read_log 192718'13340 (192718'13339) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:477 2014-08-06 14:20:21.078172 2014-08-13 18:06:06.653914 7f5c973a57a0 20 read_log 192718'13341 (192718'13340) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:478 2014-08-06 14:20:21.115904 2014-08-13 18:06:06.653926 7f5c973a57a0 20 read_log 192718'13342 (192718'13341) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:478 2014-08-06 14:20:21.136795 2014-08-13 18:06:06.653938 7f5c973a57a0 20 read_log 192718'13343 (192718'13342) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:479 2014-08-06 14:20:21.172279 2014-08-13 18:06:06.653950 7f5c973a57a0 20 read_log 192718'13344 (192718'13343) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:479 2014-08-06 14:20:21.281631 2014-08-13 18:06:06.653962 7f5c973a57a0 20 read_log 192718'13345 (192718'13344) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:480 2014-08-06 14:20:21.303459 2014-08-13 18:06:06.653981 7f5c973a57a0 20 read_log 192718'13346 (192718'13345) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:480 2014-08-06 14:20:21.322075 2014-08-13 18:06:06.653994 7f5c973a57a0 20 read_log 192718'13347 (192718'13346) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:481 2014-08-06 14:20:21.343701 2014-08-13 18:06:06.654007 7f5c973a57a0 20 read_log 192718'13348 (192718'13347) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:481 2014-08-06 14:20:21.362298 2014-08-13 18:06:06.654031 7f5c973a57a0 20 read_log 192718'13349 (192718'13348) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:482 2014-08-06 14:20:21.434772 2014-08-13 18:06:06.654048 7f5c973a57a0 20 read_log 192718'13350 (192718'13349) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:482 2014-08-06 14:20:21.456347 2014-08-13 18:06:06.654061 7f5c973a57a0 20 read_log 192718'13351 (192718'13350) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:483 2014-08-06 14:20:21.494203 2014-08-13 18:06:06.654083 7f5c973a57a0 20 read_log 192718'13352 (192718'13351) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:483 2014-08-06 14:20:21.514921 2014-08-13 18:06:06.654116 7f5c973a57a0 20 read_log 192718'13353 (192718'13352) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:484 2014-08-06 14:20:21.543789 2014-08-13 18:06:06.654132 7f5c973a57a0 20 read_log 192718'13354 (192718'13353) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:484 2014-08-06 14:20:21.576556 2014-08-13 18:06:06.654156 7f5c973a57a0 20 read_log 192718'13355 (192718'13354) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:485 2014-08-06 14:20:21.598401 2014-08-13 18:06:06.654174 7f5c973a57a0 20 read_log 192718'13356 (192718'13355) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:485 2014-08-06 14:20:21.629285 2014-08-13 18:06:06.654187 7f5c973a57a0 20 read_log 192718'13357 (192718'13356) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:486 2014-08-06 14:20:21.669828 2014-08-13 18:06:06.654199 7f5c973a57a0 20 read_log 192718'13358 (192718'13357) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:486 2014-08-06 14:20:21.722981 2014-08-13 18:06:06.654212 7f5c973a57a0 20 read_log 192718'13359 (192718'13358) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:487 2014-08-06 14:20:21.756947 2014-08-13 18:06:06.654224 7f5c973a57a0 20 read_log 192718'13360 (192718'13359) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:487 2014-08-06 14:20:21.765643 2014-08-13 18:06:06.654236 7f5c973a57a0 20 read_log 192718'13361 (192718'13360) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:488 2014-08-06 14:20:21.795073 2014-08-13 18:06:06.654250 7f5c973a57a0 20 read_log 192718'13362 (192718'13361) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:488 2014-08-06 14:20:21.818161 2014-08-13 18:06:06.654265 7f5c973a57a0 20 read_log 192718'13363 (192718'13362) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:489 2014-08-06 14:20:21.881334 2014-08-13 18:06:06.654279 7f5c973a57a0 20 read_log 192718'13364 (192718'13363) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:489 2014-08-06 14:20:22.048758 2014-08-13 18:06:06.654292 7f5c973a57a0 20 read_log 192718'13365 (192718'13364) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:490 2014-08-06 14:20:22.109540 2014-08-13 18:06:06.654304 7f5c973a57a0 20 read_log 192718'13366 (192718'13365) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:490 2014-08-06 14:20:22.160904 2014-08-13 18:06:06.654316 7f5c973a57a0 20 read_log 192718'13367 (192718'13366) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:491 2014-08-06 14:20:22.207335 2014-08-13 18:06:06.654337 7f5c973a57a0 20 read_log 192718'13368 (192718'13367) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:491 2014-08-06 14:20:22.250512 2014-08-13 18:06:06.654351 7f5c973a57a0 20 read_log 192718'13369 (192718'13368) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:492 2014-08-06 14:20:22.286245 2014-08-13 18:06:06.654363 7f5c973a57a0 20 read_log 192718'13370 (192718'13369) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:492 2014-08-06 14:20:22.302424 2014-08-13 18:06:06.654382 7f5c973a57a0 20 read_log 192718'13371 (192718'13370) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:493 2014-08-06 14:20:22.325248 2014-08-13 18:06:06.654398 7f5c973a57a0 20 read_log 192718'13372 (192718'13371) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:493 2014-08-06 14:20:22.348834 2014-08-13 18:06:06.654410 7f5c973a57a0 20 read_log 192718'13373 (192718'13372) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:494 2014-08-06 14:20:22.405785 2014-08-13 18:06:06.654433 7f5c973a57a0 20 read_log 192718'13374 (192718'13373) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:494 2014-08-06 14:20:22.541010 2014-08-13 18:06:06.654449 7f5c973a57a0 20 read_log 192718'13375 (192718'13374) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:495 2014-08-06 14:20:22.595949 2014-08-13 18:06:06.654461 7f5c973a57a0 20 read_log 192718'13376 (192718'13375) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:495 2014-08-06 14:20:22.777152 2014-08-13 18:06:06.654504 7f5c973a57a0 20 read_log 192718'13377 (192718'13376) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:496 2014-08-06 14:20:22.870953 2014-08-13 18:06:06.654521 7f5c973a57a0 20 read_log 192718'13378 (192718'13377) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:496 2014-08-06 14:20:22.922780 2014-08-13 18:06:06.654537 7f5c973a57a0 20 read_log 192718'13379 (192718'13378) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:497 2014-08-06 14:20:22.972179 2014-08-13 18:06:06.654561 7f5c973a57a0 20 read_log 192718'13380 (192718'13379) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:497 2014-08-06 14:20:22.994578 2014-08-13 18:06:06.654574 7f5c973a57a0 20 read_log 192718'13381 (192718'13380) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:498 2014-08-06 14:20:23.067651 2014-08-13 18:06:06.654587 7f5c973a57a0 20 read_log 192718'13382 (192718'13381) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:498 2014-08-06 14:20:23.101849 2014-08-13 18:06:06.654599 7f5c973a57a0 20 read_log 192718'13383 (192718'13382) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:499 2014-08-06 14:20:23.147542 2014-08-13 18:06:06.654612 7f5c973a57a0 20 read_log 192718'13384 (192718'13383) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:499 2014-08-06 14:20:23.372579 2014-08-13 18:06:06.654624 7f5c973a57a0 20 read_log 192718'13385 (192718'13384) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:500 2014-08-06 14:20:23.520053 2014-08-13 18:06:06.654636 7f5c973a57a0 20 read_log 192718'13388 (192718'13385) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:500 2014-08-06 14:20:23.593520 2014-08-13 18:06:06.654649 7f5c973a57a0 20 read_log 192718'13389 (192718'13388) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:501 2014-08-06 14:20:23.714941 2014-08-13 18:06:06.654661 7f5c973a57a0 20 read_log 192718'13390 (192718'13389) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:501 2014-08-06 14:20:23.769108 2014-08-13 18:06:06.654674 7f5c973a57a0 20 read_log 192718'13391 (192718'13390) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:502 2014-08-06 14:20:24.210977 2014-08-13 18:06:06.654687 7f5c973a57a0 20 read_log 192718'13392 (192718'13391) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:502 2014-08-06 14:20:24.252633 2014-08-13 18:06:06.654699 7f5c973a57a0 20 read_log 192718'13393 (192718'13392) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:503 2014-08-06 14:20:24.280064 2014-08-13 18:06:06.654720 7f5c973a57a0 20 read_log 192718'13394 (192718'13393) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:503 2014-08-06 14:20:24.334842 2014-08-13 18:06:06.654736 7f5c973a57a0 20 read_log 192718'13395 (192718'13394) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:504 2014-08-06 14:20:24.449686 2014-08-13 18:06:06.654749 7f5c973a57a0 20 read_log 192718'13396 (192718'13395) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:504 2014-08-06 14:20:24.511926 2014-08-13 18:06:06.654761 7f5c973a57a0 20 read_log 192718'13397 (192718'13396) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:505 2014-08-06 14:20:24.753746 2014-08-13 18:06:06.654773 7f5c973a57a0 20 read_log 192718'13398 (192718'13397) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:505 2014-08-06 14:20:24.791125 2014-08-13 18:06:06.654804 7f5c973a57a0 20 read_log 192718'13399 (192718'13398) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:506 2014-08-06 14:20:24.836681 2014-08-13 18:06:06.654819 7f5c973a57a0 20 read_log 192718'13400 (192718'13399) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:506 2014-08-06 14:20:24.906573 2014-08-13 18:06:06.654831 7f5c973a57a0 20 read_log 192718'13401 (192718'13400) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:507 2014-08-06 14:20:25.075479 2014-08-13 18:06:06.654844 7f5c973a57a0 20 read_log 192718'13402 (192718'13401) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:507 2014-08-06 14:20:25.117971 2014-08-13 18:06:06.654863 7f5c973a57a0 20 read_log 192718'13403 (192718'13402) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:508 2014-08-06 14:20:25.167049 2014-08-13 18:06:06.654877 7f5c973a57a0 20 read_log 192718'13404 (192718'13403) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:508 2014-08-06 14:20:25.191495 2014-08-13 18:06:06.654890 7f5c973a57a0 20 read_log 192718'13405 (192718'13404) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:509 2014-08-06 14:20:25.226747 2014-08-13 18:06:06.654902 7f5c973a57a0 20 read_log 192718'13406 (192718'13405) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:509 2014-08-06 14:20:25.348127 2014-08-13 18:06:06.654914 7f5c973a57a0 20 read_log 192718'13407 (192718'13406) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:510 2014-08-06 14:20:25.397085 2014-08-13 18:06:06.654926 7f5c973a57a0 20 read_log 192718'13408 (192718'13407) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:510 2014-08-06 14:20:25.421465 2014-08-13 18:06:06.654939 7f5c973a57a0 20 read_log 192718'13409 (192718'13408) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:511 2014-08-06 14:20:25.476340 2014-08-13 18:06:06.654951 7f5c973a57a0 20 read_log 192718'13410 (192718'13409) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:511 2014-08-06 14:20:25.672362 2014-08-13 18:06:06.654963 7f5c973a57a0 20 read_log 192718'13411 (192718'13410) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:512 2014-08-06 14:20:25.719235 2014-08-13 18:06:06.654975 7f5c973a57a0 20 read_log 192718'13412 (192718'13411) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:512 2014-08-06 14:20:25.738373 2014-08-13 18:06:06.654988 7f5c973a57a0 20 read_log 192718'13413 (192718'13412) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:513 2014-08-06 14:20:25.777508 2014-08-13 18:06:06.655002 7f5c973a57a0 20 read_log 192718'13414 (192718'13413) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:513 2014-08-06 14:20:25.804636 2014-08-13 18:06:06.655015 7f5c973a57a0 20 read_log 192718'13415 (192718'13414) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:514 2014-08-06 14:20:25.860788 2014-08-13 18:06:06.655027 7f5c973a57a0 20 read_log 192718'13416 (192718'13415) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:514 2014-08-06 14:20:25.975697 2014-08-13 18:06:06.655039 7f5c973a57a0 20 read_log 192718'13417 (192718'13416) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:515 2014-08-06 14:20:26.009905 2014-08-13 18:06:06.655051 7f5c973a57a0 20 read_log 192718'13418 (192718'13417) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:515 2014-08-06 14:20:26.036843 2014-08-13 18:06:06.655063 7f5c973a57a0 20 read_log 192718'13419 (192718'13418) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:516 2014-08-06 14:20:26.081636 2014-08-13 18:06:06.655077 7f5c973a57a0 20 read_log 192718'13420 (192718'13419) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:516 2014-08-06 14:20:26.094363 2014-08-13 18:06:06.655109 7f5c973a57a0 20 read_log 192718'13421 (192718'13420) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:517 2014-08-06 14:20:26.134549 2014-08-13 18:06:06.655122 7f5c973a57a0 20 read_log 192718'13422 (192718'13421) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:517 2014-08-06 14:20:26.144087 2014-08-13 18:06:06.655136 7f5c973a57a0 20 read_log 192718'13423 (192718'13422) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:518 2014-08-06 14:20:26.179724 2014-08-13 18:06:06.655173 7f5c973a57a0 20 read_log 192718'13424 (192718'13423) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:518 2014-08-06 14:20:26.393002 2014-08-13 18:06:06.655190 7f5c973a57a0 20 read_log 192718'13425 (192718'13424) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:519 2014-08-06 14:20:26.436136 2014-08-13 18:06:06.655202 7f5c973a57a0 20 read_log 192718'13426 (192718'13425) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:519 2014-08-06 14:20:26.563209 2014-08-13 18:06:06.655224 7f5c973a57a0 20 read_log 192718'13427 (192718'13426) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:520 2014-08-06 14:20:26.664051 2014-08-13 18:06:06.655239 7f5c973a57a0 20 read_log 192718'13428 (192718'13427) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:520 2014-08-06 14:20:26.712815 2014-08-13 18:06:06.655251 7f5c973a57a0 20 read_log 192718'13429 (192718'13428) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:521 2014-08-06 14:20:26.750518 2014-08-13 18:06:06.655266 7f5c973a57a0 20 read_log 192718'13430 (192718'13429) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:521 2014-08-06 14:20:26.769964 2014-08-13 18:06:06.655278 7f5c973a57a0 20 read_log 192718'13431 (192718'13430) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:522 2014-08-06 14:20:26.816315 2014-08-13 18:06:06.655292 7f5c973a57a0 20 read_log 192718'13432 (192718'13431) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:522 2014-08-06 14:20:26.830586 2014-08-13 18:06:06.655308 7f5c973a57a0 20 read_log 192718'13433 (192718'13432) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:523 2014-08-06 14:20:26.861963 2014-08-13 18:06:06.655320 7f5c973a57a0 20 read_log 192718'13434 (192718'13433) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:523 2014-08-06 14:20:26.880525 2014-08-13 18:06:06.655332 7f5c973a57a0 20 read_log 192718'13435 (192718'13434) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:524 2014-08-06 14:20:26.950470 2014-08-13 18:06:06.655345 7f5c973a57a0 20 read_log 192718'13436 (192718'13435) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:524 2014-08-06 14:20:26.991472 2014-08-13 18:06:06.655362 7f5c973a57a0 20 read_log 192718'13437 (192718'13436) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:525 2014-08-06 14:20:27.009449 2014-08-13 18:06:06.655375 7f5c973a57a0 20 read_log 192718'13438 (192718'13437) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:525 2014-08-06 14:20:27.030752 2014-08-13 18:06:06.655387 7f5c973a57a0 20 read_log 192718'13439 (192718'13438) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:526 2014-08-06 14:20:27.067960 2014-08-13 18:06:06.655404 7f5c973a57a0 20 read_log 192718'13440 (192718'13439) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:526 2014-08-06 14:20:27.102008 2014-08-13 18:06:06.655416 7f5c973a57a0 20 read_log 192718'13441 (192718'13440) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:527 2014-08-06 14:20:27.207336 2014-08-13 18:06:06.655441 7f5c973a57a0 20 read_log 192718'13442 (192718'13441) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:527 2014-08-06 14:20:27.446459 2014-08-13 18:06:06.655460 7f5c973a57a0 20 read_log 192718'13443 (192718'13442) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:528 2014-08-06 14:20:27.485087 2014-08-13 18:06:06.655480 7f5c973a57a0 20 read_log 192718'13444 (192718'13443) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:528 2014-08-06 14:20:27.522120 2014-08-13 18:06:06.655495 7f5c973a57a0 20 read_log 192718'13445 (192718'13444) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:529 2014-08-06 14:20:27.569212 2014-08-13 18:06:06.655508 7f5c973a57a0 20 read_log 192718'13446 (192718'13445) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:529 2014-08-06 14:20:27.599378 2014-08-13 18:06:06.655527 7f5c973a57a0 20 read_log 192718'13447 (192718'13446) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:530 2014-08-06 14:20:27.637596 2014-08-13 18:06:06.655542 7f5c973a57a0 20 read_log 192718'13448 (192718'13447) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:530 2014-08-06 14:20:27.674712 2014-08-13 18:06:06.655555 7f5c973a57a0 20 read_log 192718'13449 (192718'13448) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:531 2014-08-06 14:20:27.714108 2014-08-13 18:06:06.655575 7f5c973a57a0 20 read_log 192718'13450 (192718'13449) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:531 2014-08-06 14:20:27.748999 2014-08-13 18:06:06.655597 7f5c973a57a0 20 read_log 192718'13451 (192718'13450) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:532 2014-08-06 14:20:27.796504 2014-08-13 18:06:06.655611 7f5c973a57a0 20 read_log 192718'13452 (192718'13451) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:532 2014-08-06 14:20:27.919445 2014-08-13 18:06:06.655624 7f5c973a57a0 20 read_log 192718'13453 (192718'13452) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:533 2014-08-06 14:20:27.961700 2014-08-13 18:06:06.655645 7f5c973a57a0 20 read_log 192718'13454 (192718'13453) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:533 2014-08-06 14:20:27.995143 2014-08-13 18:06:06.655658 7f5c973a57a0 20 read_log 192718'13455 (192718'13454) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:534 2014-08-06 14:20:28.053182 2014-08-13 18:06:06.655678 7f5c973a57a0 20 read_log 192718'13456 (192718'13455) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:534 2014-08-06 14:20:28.268286 2014-08-13 18:06:06.655694 7f5c973a57a0 20 read_log 192718'13457 (192718'13456) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:535 2014-08-06 14:20:28.290926 2014-08-13 18:06:06.655707 7f5c973a57a0 20 read_log 192718'13458 (192718'13457) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:535 2014-08-06 14:20:28.341313 2014-08-13 18:06:06.655719 7f5c973a57a0 20 read_log 192718'13459 (192718'13458) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:536 2014-08-06 14:20:28.384376 2014-08-13 18:06:06.655732 7f5c973a57a0 20 read_log 192718'13460 (192718'13459) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:536 2014-08-06 14:20:28.419676 2014-08-13 18:06:06.655744 7f5c973a57a0 20 read_log 192718'13461 (192718'13460) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:537 2014-08-06 14:20:28.463673 2014-08-13 18:06:06.655756 7f5c973a57a0 20 read_log 192718'13462 (192718'13461) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:537 2014-08-06 14:20:28.551931 2014-08-13 18:06:06.655781 7f5c973a57a0 20 read_log 192718'13463 (192718'13462) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:538 2014-08-06 14:20:28.566900 2014-08-13 18:06:06.655797 7f5c973a57a0 20 read_log 192718'13464 (192718'13463) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:538 2014-08-06 14:20:28.592885 2014-08-13 18:06:06.655810 7f5c973a57a0 20 read_log 192718'13465 (192718'13464) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:539 2014-08-06 14:20:28.614370 2014-08-13 18:06:06.655823 7f5c973a57a0 20 read_log 192718'13466 (192718'13465) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:539 2014-08-06 14:20:28.635217 2014-08-13 18:06:06.655839 7f5c973a57a0 20 read_log 192718'13467 (192718'13466) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:540 2014-08-06 14:20:28.679220 2014-08-13 18:06:06.655851 7f5c973a57a0 20 read_log 192718'13468 (192718'13467) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:540 2014-08-06 14:20:28.820308 2014-08-13 18:06:06.655863 7f5c973a57a0 20 read_log 192718'13469 (192718'13468) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:541 2014-08-06 14:20:28.841805 2014-08-13 18:06:06.655886 7f5c973a57a0 20 read_log 192718'13470 (192718'13469) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:541 2014-08-06 14:20:28.863588 2014-08-13 18:06:06.655903 7f5c973a57a0 20 read_log 192718'13471 (192718'13470) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:542 2014-08-06 14:20:28.885568 2014-08-13 18:06:06.655915 7f5c973a57a0 20 read_log 192718'13472 (192718'13471) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:542 2014-08-06 14:20:29.026316 2014-08-13 18:06:06.655931 7f5c973a57a0 20 read_log 192718'13473 (192718'13472) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:543 2014-08-06 14:20:29.056754 2014-08-13 18:06:06.655946 7f5c973a57a0 20 read_log 192718'13474 (192718'13473) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:543 2014-08-06 14:20:29.073678 2014-08-13 18:06:06.655968 7f5c973a57a0 20 read_log 192718'13475 (192718'13474) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:544 2014-08-06 14:20:29.099106 2014-08-13 18:06:06.655987 7f5c973a57a0 20 read_log 192718'13476 (192718'13475) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:544 2014-08-06 14:20:29.127153 2014-08-13 18:06:06.656009 7f5c973a57a0 20 read_log 192718'13477 (192718'13476) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:545 2014-08-06 14:20:29.183520 2014-08-13 18:06:06.656028 7f5c973a57a0 20 read_log 192718'13478 (192718'13477) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:545 2014-08-06 14:20:29.185933 2014-08-13 18:06:06.656040 7f5c973a57a0 20 read_log 192718'13479 (192718'13478) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:546 2014-08-06 14:20:29.215487 2014-08-13 18:06:06.656053 7f5c973a57a0 20 read_log 192718'13480 (192718'13479) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:546 2014-08-06 14:20:29.238911 2014-08-13 18:06:06.656065 7f5c973a57a0 20 read_log 192718'13481 (192718'13480) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:547 2014-08-06 14:20:29.272901 2014-08-13 18:06:06.656077 7f5c973a57a0 20 read_log 192718'13482 (192718'13481) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:547 2014-08-06 14:20:29.442239 2014-08-13 18:06:06.656090 7f5c973a57a0 20 read_log 192718'13483 (192718'13482) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:548 2014-08-06 14:20:29.580341 2014-08-13 18:06:06.656102 7f5c973a57a0 20 read_log 192718'13484 (192718'13483) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:548 2014-08-06 14:20:29.593043 2014-08-13 18:06:06.656114 7f5c973a57a0 20 read_log 192718'13485 (192718'13484) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:549 2014-08-06 14:20:29.617563 2014-08-13 18:06:06.656130 7f5c973a57a0 20 read_log 192718'13486 (192718'13485) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:549 2014-08-06 14:20:29.635197 2014-08-13 18:06:06.656142 7f5c973a57a0 20 read_log 192718'13487 (192718'13486) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:550 2014-08-06 14:20:29.664027 2014-08-13 18:06:06.656171 7f5c973a57a0 20 read_log 192718'13488 (192718'13487) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:550 2014-08-06 14:20:29.671844 2014-08-13 18:06:06.656197 7f5c973a57a0 20 read_log 192718'13489 (192718'13488) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:551 2014-08-06 14:20:29.698319 2014-08-13 18:06:06.656212 7f5c973a57a0 20 read_log 192718'13490 (192718'13489) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:551 2014-08-06 14:20:29.717411 2014-08-13 18:06:06.656233 7f5c973a57a0 20 read_log 192718'13491 (192718'13490) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:552 2014-08-06 14:20:29.761434 2014-08-13 18:06:06.656271 7f5c973a57a0 20 read_log 192718'13492 (192718'13491) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:552 2014-08-06 14:20:29.839956 2014-08-13 18:06:06.656286 7f5c973a57a0 20 read_log 192718'13493 (192718'13492) modify 407860cd/core.47650+11+2/head//24 by client.408433.0:553 2014-08-06 14:20:29.886066 2014-08-13 18:06:06.656299 7f5c973a57a0 20 read_log 192718'13494 (192718'13493) modify 407860cd/core.47650+11+2/head//24 by client.456998.0:553 2014-08-06 14:20:29.912573 2014-08-13 18:06:06.656321 7f5c973a57a0 20 read_log 192718'13837 (192718'13494) clean 407860cd/core.47650+11+2/head//24 by osd.371.0:25752 2014-08-06 14:52:03.375393 2014-08-13 18:06:06.656338 7f5c973a57a0 20 read_log 192798'14100 (192718'13837) delete 407860cd/core.47650+11+2/head//24 by osd.371.0:26877 2014-08-11 10:24:43.645971 2014-08-13 18:06:06.656351 7f5c973a57a0 20 read_log 192834'14113 (0'0) promote adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.371.0:26970 2014-08-11 15:23:36.476590 2014-08-13 18:06:06.656364 7f5c973a57a0 20 read_log 192834'14114 (192834'14113) modify adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by client.933448.0:207 2014-08-11 15:23:36.470563 2014-08-13 18:06:06.656397 7f5c973a57a0 20 read_log 192834'14125 (0'0) promote fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by osd.371.0:26994 2014-08-11 15:23:40.726370 2014-08-13 18:06:06.656414 7f5c973a57a0 20 read_log 192834'14126 (192834'14125) modify fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.933448.0:1305 2014-08-11 15:23:40.718631 2014-08-13 18:06:06.656427 7f5c973a57a0 20 read_log 192834'14209 (0'0) promote 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by osd.371.0:27134 2014-08-11 15:24:04.371951 2014-08-13 18:06:06.656440 7f5c973a57a0 20 read_log 192834'14210 (192834'14209) modify 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.933448.0:6165 2014-08-11 15:24:04.363883 2014-08-13 18:06:06.656452 7f5c973a57a0 20 read_log 192834'14233 (0'0) promote 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by osd.371.0:27178 2014-08-11 15:24:10.446720 2014-08-13 18:06:06.656465 7f5c973a57a0 20 read_log 192834'14234 (192834'14233) modify 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.933448.0:7535 2014-08-11 15:24:10.438788 2014-08-13 18:06:06.656477 7f5c973a57a0 20 read_log 192834'14239 (0'0) promote dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.371.0:27192 2014-08-11 15:24:12.261211 2014-08-13 18:06:06.656489 7f5c973a57a0 20 read_log 192834'14240 (192834'14239) modify dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by client.933448.0:8008 2014-08-11 15:24:12.252875 2014-08-13 18:06:06.656501 7f5c973a57a0 20 read_log 192834'14255 (0'0) promote 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by osd.371.0:27224 2014-08-11 15:24:15.205984 2014-08-13 18:06:06.656514 7f5c973a57a0 20 read_log 192834'14256 (192834'14255) modify 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.933448.0:8731 2014-08-11 15:24:15.198975 2014-08-13 18:06:06.656526 7f5c973a57a0 20 read_log 192834'14257 (0'0) promote d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by osd.371.0:27230 2014-08-11 15:24:15.946565 2014-08-13 18:06:06.656538 7f5c973a57a0 20 read_log 192834'14258 (192834'14257) modify d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.933448.0:8908 2014-08-11 15:24:15.936147 2014-08-13 18:06:06.656553 7f5c973a57a0 20 read_log 192834'14279 (0'0) promote 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by osd.371.0:27284 2014-08-11 15:24:22.602297 2014-08-13 18:06:06.656566 7f5c973a57a0 20 read_log 192834'14280 (192834'14279) modify 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.933448.0:10326 2014-08-11 15:24:22.594703 2014-08-13 18:06:06.656578 7f5c973a57a0 20 read_log 192834'14327 (0'0) promote 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.371.0:27374 2014-08-11 15:24:41.001314 2014-08-13 18:06:06.656591 7f5c973a57a0 20 read_log 192834'14328 (192834'14327) modify 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by client.933448.0:14065 2014-08-11 15:24:40.991189 2014-08-13 18:06:06.656603 7f5c973a57a0 20 read_log 192834'14359 (0'0) promote 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by osd.371.0:27422 2014-08-11 15:24:48.398025 2014-08-13 18:06:06.656615 7f5c973a57a0 20 read_log 192834'14360 (192834'14359) modify 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.933448.0:15096 2014-08-11 15:24:48.390776 2014-08-13 18:06:06.656628 7f5c973a57a0 20 read_log 192834'14369 (0'0) promote ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by osd.371.0:27436 2014-08-11 15:24:50.397593 2014-08-13 18:06:06.656640 7f5c973a57a0 20 read_log 192834'14370 (192834'14369) modify ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.933448.0:15618 2014-08-11 15:24:50.390068 2014-08-13 18:06:06.656660 7f5c973a57a0 20 read_log 192834'14393 (0'0) promote f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by osd.371.0:27478 2014-08-11 15:24:58.470355 2014-08-13 18:06:06.656672 7f5c973a57a0 20 read_log 192834'14394 (192834'14393) modify f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.933448.0:17107 2014-08-11 15:24:58.461362 2014-08-13 18:06:06.656685 7f5c973a57a0 20 read_log 192834'14397 (0'0) promote cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.371.0:27482 2014-08-11 15:24:58.782364 2014-08-13 18:06:06.656697 7f5c973a57a0 20 read_log 192834'14398 (192834'14397) modify cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by client.933448.0:17191 2014-08-11 15:24:58.773499 2014-08-13 18:06:06.656709 7f5c973a57a0 20 read_log 192834'14401 (0'0) promote 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by osd.371.0:27490 2014-08-11 15:24:59.964564 2014-08-13 18:06:06.656721 7f5c973a57a0 20 read_log 192834'14402 (192834'14401) modify 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.933448.0:17506 2014-08-11 15:24:59.956609 2014-08-13 18:06:06.656733 7f5c973a57a0 20 read_log 192834'14407 (0'0) promote a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.371.0:27502 2014-08-11 15:25:04.546574 2014-08-13 18:06:06.656748 7f5c973a57a0 20 read_log 192834'14408 (192834'14407) modify a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by client.933448.0:18272 2014-08-11 15:25:04.538140 2014-08-13 18:06:06.656761 7f5c973a57a0 20 read_log 192834'14437 (0'0) promote 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by osd.371.0:27564 2014-08-11 15:25:20.554251 2014-08-13 18:06:06.656775 7f5c973a57a0 20 read_log 192834'14438 (192834'14437) modify 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.933448.0:20354 2014-08-11 15:25:20.545362 2014-08-13 18:06:06.656788 7f5c973a57a0 20 read_log 192834'14447 (0'0) promote e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by osd.371.0:27578 2014-08-11 15:25:25.469700 2014-08-13 18:06:06.656800 7f5c973a57a0 20 read_log 192834'14448 (192834'14447) modify e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.933448.0:20965 2014-08-11 15:25:25.461251 2014-08-13 18:06:06.656813 7f5c973a57a0 20 read_log 192834'14467 (0'0) promote 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by osd.371.0:27614 2014-08-11 15:25:31.784867 2014-08-13 18:06:06.656826 7f5c973a57a0 20 read_log 192834'14468 (192834'14467) modify 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.933448.0:21759 2014-08-11 15:25:31.776610 2014-08-13 18:06:06.656842 7f5c973a57a0 20 read_log 192834'14474 (192834'14114) modify adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by client.933448.0:22498 2014-08-11 15:25:37.979493 2014-08-13 18:06:06.656854 7f5c973a57a0 20 read_log 192834'14480 (192834'14126) modify fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.933448.0:23596 2014-08-11 15:25:39.536561 2014-08-13 18:06:06.656869 7f5c973a57a0 20 read_log 192834'14522 (192834'14210) modify 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.933448.0:28456 2014-08-11 15:25:46.255252 2014-08-13 18:06:06.656881 7f5c973a57a0 20 read_log 192834'14534 (192834'14234) modify 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.933448.0:29826 2014-08-11 15:25:47.953637 2014-08-13 18:06:06.656895 7f5c973a57a0 20 read_log 192834'14537 (192834'14240) modify dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by client.933448.0:30299 2014-08-11 15:25:48.698855 2014-08-13 18:06:06.656908 7f5c973a57a0 20 read_log 192834'14545 (192834'14256) modify 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.933448.0:31022 2014-08-11 15:25:49.502666 2014-08-13 18:06:06.656953 7f5c973a57a0 20 read_log 192834'14546 (192834'14258) modify d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.933448.0:31199 2014-08-11 15:25:49.785049 2014-08-13 18:06:06.656969 7f5c973a57a0 20 read_log 192834'14557 (192834'14280) modify 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.933448.0:32617 2014-08-11 15:25:52.107426 2014-08-13 18:06:06.656982 7f5c973a57a0 20 read_log 192834'14581 (192834'14328) modify 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by client.933448.0:36356 2014-08-11 15:25:57.605793 2014-08-13 18:06:06.656996 7f5c973a57a0 20 read_log 192834'14597 (192834'14360) modify 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.933448.0:37387 2014-08-11 15:25:59.496529 2014-08-13 18:06:06.657013 7f5c973a57a0 20 read_log 192834'14602 (192834'14370) modify ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.933448.0:37909 2014-08-11 15:26:00.261738 2014-08-13 18:06:06.657028 7f5c973a57a0 20 read_log 192834'14614 (192834'14394) modify f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.933448.0:39398 2014-08-11 15:26:01.845574 2014-08-13 18:06:06.657041 7f5c973a57a0 20 read_log 192834'14616 (192834'14398) modify cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by client.933448.0:39482 2014-08-11 15:26:01.914512 2014-08-13 18:06:06.657055 7f5c973a57a0 20 read_log 192834'14618 (192834'14402) modify 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.933448.0:39797 2014-08-11 15:26:02.089110 2014-08-13 18:06:06.657068 7f5c973a57a0 20 read_log 192834'14621 (192834'14408) modify a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by client.933448.0:40563 2014-08-11 15:26:02.653475 2014-08-13 18:06:06.657080 7f5c973a57a0 20 read_log 192834'14636 (192834'14438) modify 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.933448.0:42645 2014-08-11 15:26:03.341725 2014-08-13 18:06:06.657093 7f5c973a57a0 20 read_log 192834'14641 (192834'14448) modify e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.933448.0:43256 2014-08-11 15:26:03.552578 2014-08-13 18:06:06.657107 7f5c973a57a0 20 read_log 192834'14651 (192834'14468) modify 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.933448.0:44050 2014-08-11 15:26:03.801023 2014-08-13 18:06:06.657120 7f5c973a57a0 20 read_log 192834'14661 (0'0) promote 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by osd.371.0:27983 2014-08-11 15:26:16.512241 2014-08-13 18:06:06.657134 7f5c973a57a0 20 read_log 192834'14662 (192834'14661) modify 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.1883634.0:553 2014-08-11 15:26:16.504642 2014-08-13 18:06:06.657156 7f5c973a57a0 20 read_log 192834'14691 (0'0) promote b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.371.0:28029 2014-08-11 15:26:21.917766 2014-08-13 18:06:06.657172 7f5c973a57a0 20 read_log 192834'14692 (192834'14691) modify b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by client.1883634.0:2015 2014-08-11 15:26:21.910447 2014-08-13 18:06:06.657187 7f5c973a57a0 20 read_log 192834'14693 (0'0) promote 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by osd.371.0:28031 2014-08-11 15:26:21.973118 2014-08-13 18:06:06.657201 7f5c973a57a0 20 read_log 192834'14694 (192834'14693) modify 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.1883634.0:2030 2014-08-11 15:26:21.964709 2014-08-13 18:06:06.657214 7f5c973a57a0 20 read_log 192834'14713 (0'0) promote 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by osd.371.0:28077 2014-08-11 15:26:28.877830 2014-08-13 18:06:06.657233 7f5c973a57a0 20 read_log 192834'14714 (192834'14713) modify 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.1883634.0:3967 2014-08-11 15:26:28.870023 2014-08-13 18:06:06.657253 7f5c973a57a0 20 read_log 192834'14719 (0'0) promote fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by osd.371.0:28085 2014-08-11 15:26:30.336520 2014-08-13 18:06:06.657271 7f5c973a57a0 20 read_log 192834'14720 (192834'14719) modify fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.1883634.0:4378 2014-08-11 15:26:30.328730 2014-08-13 18:06:06.657284 7f5c973a57a0 20 read_log 192834'14723 (0'0) promote daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by osd.371.0:28089 2014-08-11 15:26:31.568873 2014-08-13 18:06:06.657299 7f5c973a57a0 20 read_log 192834'14724 (192834'14723) modify daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.1883634.0:4696 2014-08-11 15:26:31.559954 2014-08-13 18:06:06.657311 7f5c973a57a0 20 read_log 192834'14731 (0'0) promote 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by osd.371.0:28101 2014-08-11 15:26:32.687351 2014-08-13 18:06:06.657327 7f5c973a57a0 20 read_log 192834'14732 (192834'14731) modify 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.1883634.0:4950 2014-08-11 15:26:32.679765 2014-08-13 18:06:06.657342 7f5c973a57a0 20 read_log 192834'14737 (0'0) promote f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by osd.371.0:28111 2014-08-11 15:26:34.006556 2014-08-13 18:06:06.657355 7f5c973a57a0 20 read_log 192834'14738 (192834'14737) modify f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.1883634.0:5325 2014-08-11 15:26:33.998672 2014-08-13 18:06:06.657367 7f5c973a57a0 20 read_log 192834'14739 (0'0) promote f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.371.0:28117 2014-08-11 15:26:34.268277 2014-08-13 18:06:06.657383 7f5c973a57a0 20 read_log 192834'14740 (192834'14739) modify f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by client.1883634.0:5399 2014-08-11 15:26:34.259521 2014-08-13 18:06:06.657411 7f5c973a57a0 20 read_log 192834'14753 (0'0) promote 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.371.0:28141 2014-08-11 15:26:36.424734 2014-08-13 18:06:06.657428 7f5c973a57a0 20 read_log 192834'14754 (192834'14753) modify 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by client.1883634.0:6004 2014-08-11 15:26:36.416947 2014-08-13 18:06:06.657451 7f5c973a57a0 20 read_log 192834'14757 (0'0) promote 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by osd.371.0:28149 2014-08-11 15:26:36.775215 2014-08-13 18:06:06.657466 7f5c973a57a0 20 read_log 192834'14758 (192834'14757) modify 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.1883634.0:6102 2014-08-11 15:26:36.767279 2014-08-13 18:06:06.657479 7f5c973a57a0 20 read_log 192834'14775 (0'0) promote 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.371.0:28191 2014-08-11 15:26:41.754236 2014-08-13 18:06:06.657499 7f5c973a57a0 20 read_log 192834'14776 (192834'14775) modify 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by client.1883634.0:7338 2014-08-11 15:26:41.745566 2014-08-13 18:06:06.657514 7f5c973a57a0 20 read_log 192834'14779 (0'0) promote 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.371.0:28209 2014-08-11 15:26:44.748599 2014-08-13 18:06:06.657527 7f5c973a57a0 20 read_log 192834'14780 (192834'14779) modify 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by client.1883634.0:7985 2014-08-11 15:26:44.552450 2014-08-13 18:06:06.657556 7f5c973a57a0 20 read_log 192834'14781 (0'0) promote e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by osd.371.0:28213 2014-08-11 15:26:44.761462 2014-08-13 18:06:06.657569 7f5c973a57a0 20 read_log 192834'14782 (192834'14781) modify e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.1883634.0:8042 2014-08-11 15:26:44.751319 2014-08-13 18:06:06.657597 7f5c973a57a0 20 read_log 192834'14785 (0'0) promote 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by osd.371.0:28227 2014-08-11 15:26:47.108199 2014-08-13 18:06:06.657621 7f5c973a57a0 20 read_log 192834'14786 (192834'14785) modify 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.1883634.0:8613 2014-08-11 15:26:47.100531 2014-08-13 18:06:06.657634 7f5c973a57a0 20 read_log 192834'14789 (0'0) promote 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by osd.371.0:28231 2014-08-11 15:26:47.834976 2014-08-13 18:06:06.657646 7f5c973a57a0 20 read_log 192834'14790 (192834'14789) modify 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.1883634.0:8774 2014-08-11 15:26:47.826458 2014-08-13 18:06:06.657659 7f5c973a57a0 20 read_log 192834'14795 (0'0) promote 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by osd.371.0:28247 2014-08-11 15:26:50.264600 2014-08-13 18:06:06.657679 7f5c973a57a0 20 read_log 192834'14796 (192834'14795) modify 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.1883634.0:9302 2014-08-11 15:26:50.256237 2014-08-13 18:06:06.657714 7f5c973a57a0 20 read_log 192834'14799 (0'0) promote b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by osd.371.0:28253 2014-08-11 15:26:50.839723 2014-08-13 18:06:06.657729 7f5c973a57a0 20 read_log 192834'14800 (192834'14799) modify b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.1883634.0:9431 2014-08-11 15:26:50.831895 2014-08-13 18:06:06.657742 7f5c973a57a0 20 read_log 192834'14807 (0'0) promote 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by osd.371.0:28267 2014-08-11 15:26:53.635092 2014-08-13 18:06:06.657754 7f5c973a57a0 20 read_log 192834'14808 (192834'14807) modify 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.1883634.0:10084 2014-08-11 15:26:53.625139 2014-08-13 18:06:06.657767 7f5c973a57a0 20 read_log 192834'14819 (0'0) promote bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by osd.371.0:28283 2014-08-11 15:26:58.891794 2014-08-13 18:06:06.657779 7f5c973a57a0 20 read_log 192834'14820 (192834'14819) modify bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.1883634.0:11221 2014-08-11 15:26:58.882409 2014-08-13 18:06:06.657795 7f5c973a57a0 20 read_log 192834'14835 (0'0) promote d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by osd.371.0:28333 2014-08-11 15:27:10.056405 2014-08-13 18:06:06.657807 7f5c973a57a0 20 read_log 192834'14836 (192834'14835) modify d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.1883634.0:13550 2014-08-11 15:27:10.046300 2014-08-13 18:06:06.657819 7f5c973a57a0 20 read_log 192834'14847 (0'0) promote d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by osd.371.0:28351 2014-08-11 15:27:12.004729 2014-08-13 18:06:06.657832 7f5c973a57a0 20 read_log 192834'14848 (192834'14847) modify d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.1883634.0:14118 2014-08-11 15:27:11.996057 2014-08-13 18:06:06.657844 7f5c973a57a0 20 read_log 192834'14853 (0'0) promote baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.371.0:28367 2014-08-11 15:27:15.160422 2014-08-13 18:06:06.657856 7f5c973a57a0 20 read_log 192834'14854 (192834'14853) modify baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by client.1883634.0:14732 2014-08-11 15:27:15.151222 2014-08-13 18:06:06.657868 7f5c973a57a0 20 read_log 192834'14865 (0'0) promote ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by osd.371.0:28389 2014-08-11 15:27:19.239125 2014-08-13 18:06:06.657880 7f5c973a57a0 20 read_log 192834'14866 (192834'14865) modify ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.1883634.0:15356 2014-08-11 15:27:19.231026 2014-08-13 18:06:06.657905 7f5c973a57a0 20 read_log 192834'14883 (0'0) promote 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by osd.371.0:28417 2014-08-11 15:27:27.404182 2014-08-13 18:06:06.657921 7f5c973a57a0 20 read_log 192834'14884 (192834'14883) modify 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.1883634.0:16406 2014-08-11 15:27:27.394412 2014-08-13 18:06:06.657936 7f5c973a57a0 20 read_log 192834'14899 (0'0) promote e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by osd.371.0:28439 2014-08-11 15:27:30.235251 2014-08-13 18:06:06.657949 7f5c973a57a0 20 read_log 192834'14900 (192834'14899) modify e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.1883634.0:16985 2014-08-11 15:27:30.225156 2014-08-13 18:06:06.657972 7f5c973a57a0 20 read_log 192834'14915 (0'0) promote bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.371.0:28463 2014-08-11 15:27:39.441241 2014-08-13 18:06:06.657988 7f5c973a57a0 20 read_log 192834'14916 (192834'14915) modify bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by client.1883634.0:17872 2014-08-11 15:27:39.431291 2014-08-13 18:06:06.658001 7f5c973a57a0 20 read_log 192834'14945 (0'0) promote 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by osd.371.0:28521 2014-08-11 15:27:53.418387 2014-08-13 18:06:06.658013 7f5c973a57a0 20 read_log 192834'14946 (192834'14945) modify 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.1883634.0:19563 2014-08-11 15:27:53.303507 2014-08-13 18:06:06.658025 7f5c973a57a0 20 read_log 192834'14981 (0'0) promote 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by osd.371.0:28597 2014-08-11 15:28:11.737448 2014-08-13 18:06:06.658038 7f5c973a57a0 20 read_log 192834'14982 (192834'14981) modify 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.1883634.0:22919 2014-08-11 15:28:11.726077 2014-08-13 18:06:06.658050 7f5c973a57a0 20 read_log 192834'14995 (192834'14662) modify 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.1883634.0:23877 2014-08-11 15:28:18.037685 2014-08-13 18:06:06.658065 7f5c973a57a0 20 read_log 192834'15010 (192834'14692) modify b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by client.1883634.0:25339 2014-08-11 15:28:20.653685 2014-08-13 18:06:06.658078 7f5c973a57a0 20 read_log 192834'15011 (192834'14694) modify 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.1883634.0:25354 2014-08-11 15:28:20.792278 2014-08-13 18:06:06.658090 7f5c973a57a0 20 read_log 192834'15021 (192834'14714) modify 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.1883634.0:27291 2014-08-11 15:28:24.350807 2014-08-13 18:06:06.658103 7f5c973a57a0 20 read_log 192834'15024 (192834'14720) modify fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.1883634.0:27702 2014-08-11 15:28:24.842691 2014-08-13 18:06:06.658119 7f5c973a57a0 20 read_log 192834'15026 (192834'14724) modify daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.1883634.0:28020 2014-08-11 15:28:25.275118 2014-08-13 18:06:06.658131 7f5c973a57a0 20 read_log 192834'15030 (192834'14732) modify 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.1883634.0:28274 2014-08-11 15:28:25.592652 2014-08-13 18:06:06.658144 7f5c973a57a0 20 read_log 192834'15033 (192834'14738) modify f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.1883634.0:28649 2014-08-11 15:28:26.092327 2014-08-13 18:06:06.658167 7f5c973a57a0 20 read_log 192834'15034 (192834'14740) modify f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by client.1883634.0:28723 2014-08-11 15:28:26.189028 2014-08-13 18:06:06.658182 7f5c973a57a0 20 read_log 192834'15041 (192834'14754) modify 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by client.1883634.0:29328 2014-08-11 15:28:26.985587 2014-08-13 18:06:06.658218 7f5c973a57a0 20 read_log 192834'15043 (192834'14758) modify 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.1883634.0:29426 2014-08-11 15:28:27.232894 2014-08-13 18:06:06.658235 7f5c973a57a0 20 read_log 192834'15052 (192834'14776) modify 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by client.1883634.0:30662 2014-08-11 15:28:29.052680 2014-08-13 18:06:06.658258 7f5c973a57a0 20 read_log 192834'15054 (192834'14780) modify 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by client.1883634.0:31309 2014-08-11 15:28:30.259843 2014-08-13 18:06:06.658274 7f5c973a57a0 20 read_log 192834'15055 (192834'14782) modify e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.1883634.0:31366 2014-08-11 15:28:30.377789 2014-08-13 18:06:06.658287 7f5c973a57a0 20 read_log 192834'15057 (192834'14786) modify 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.1883634.0:31937 2014-08-11 15:28:31.151168 2014-08-13 18:06:06.658301 7f5c973a57a0 20 read_log 192834'15059 (192834'14790) modify 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.1883634.0:32098 2014-08-11 15:28:31.349478 2014-08-13 18:06:06.658314 7f5c973a57a0 20 read_log 192834'15062 (192834'14796) modify 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.1883634.0:32626 2014-08-11 15:28:32.082331 2014-08-13 18:06:06.658327 7f5c973a57a0 20 read_log 192834'15064 (192834'14800) modify b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.1883634.0:32755 2014-08-11 15:28:32.271261 2014-08-13 18:06:06.658339 7f5c973a57a0 20 read_log 192834'15068 (192834'14808) modify 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.1883634.0:33408 2014-08-11 15:28:33.131967 2014-08-13 18:06:06.658351 7f5c973a57a0 20 read_log 192834'15074 (192834'14820) modify bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.1883634.0:34545 2014-08-11 15:28:34.982526 2014-08-13 18:06:06.658363 7f5c973a57a0 20 read_log 192834'15084 (192834'14836) modify d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.1883634.0:36874 2014-08-11 15:28:38.521178 2014-08-13 18:06:06.658379 7f5c973a57a0 20 read_log 192834'15090 (192834'14848) modify d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.1883634.0:37442 2014-08-11 15:28:39.361362 2014-08-13 18:06:06.658392 7f5c973a57a0 20 read_log 192834'15093 (192834'14854) modify baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by client.1883634.0:38056 2014-08-11 15:28:40.164630 2014-08-13 18:06:06.658404 7f5c973a57a0 20 read_log 192834'15099 (192834'14866) modify ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.1883634.0:38680 2014-08-11 15:28:41.019987 2014-08-13 18:06:06.658416 7f5c973a57a0 20 read_log 192834'15108 (192834'14884) modify 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.1883634.0:39730 2014-08-11 15:28:42.488426 2014-08-13 18:06:06.658428 7f5c973a57a0 20 read_log 192834'15116 (192834'14900) modify e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.1883634.0:40309 2014-08-11 15:28:43.506503 2014-08-13 18:06:06.658441 7f5c973a57a0 20 read_log 192834'15124 (192834'14916) modify bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by client.1883634.0:41196 2014-08-11 15:28:44.740042 2014-08-13 18:06:06.658453 7f5c973a57a0 20 read_log 192834'15139 (192834'14946) modify 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.1883634.0:42887 2014-08-11 15:28:46.390930 2014-08-13 18:06:06.658465 7f5c973a57a0 20 read_log 192834'15157 (192834'14982) modify 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.1883634.0:46243 2014-08-11 15:28:47.533949 2014-08-13 18:06:06.658478 7f5c973a57a0 20 read_log 192888'15162 (192888'15162) modify cd/hit_set_24.cd_archive_2014-08-11 16:02:34.118089_2014-08-11 16:44:57.980567/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:57.980782 2014-08-13 18:06:06.658498 7f5c973a57a0 20 read_log 192888'15163 (0'0) promote 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by osd.6.0:1291 2014-08-11 16:44:57.984730 2014-08-13 18:06:06.658522 7f5c973a57a0 20 read_log 192888'15164 (192888'15163) modify 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.1315259.0:710 2014-08-11 16:44:57.976007 2014-08-13 18:06:06.658538 7f5c973a57a0 20 read_log 192888'15165 (0'0) promote a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by osd.6.0:1303 2014-08-11 16:45:02.811331 2014-08-13 18:06:06.658551 7f5c973a57a0 20 read_log 192888'15166 (192888'15165) modify a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.1315259.0:1983 2014-08-11 16:45:02.804616 2014-08-13 18:06:06.658563 7f5c973a57a0 20 read_log 192888'15167 (0'0) promote 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:1307 2014-08-11 16:45:04.887503 2014-08-13 18:06:06.658576 7f5c973a57a0 20 read_log 192888'15168 (192888'15167) modify 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by client.1315259.0:2546 2014-08-11 16:45:04.881128 2014-08-13 18:06:06.658588 7f5c973a57a0 20 read_log 192888'15169 (0'0) promote dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by osd.6.0:1311 2014-08-11 16:45:05.043607 2014-08-13 18:06:06.658600 7f5c973a57a0 20 read_log 192888'15170 (192888'15169) modify dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.1315259.0:2587 2014-08-11 16:45:05.037134 2014-08-13 18:06:06.658613 7f5c973a57a0 20 read_log 192888'15175 (0'0) promote a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by osd.6.0:1321 2014-08-11 16:45:12.427412 2014-08-13 18:06:06.658625 7f5c973a57a0 20 read_log 192888'15176 (192888'15175) modify a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.1315259.0:4540 2014-08-11 16:45:12.419249 2014-08-13 18:06:06.658637 7f5c973a57a0 20 read_log 192888'15177 (0'0) promote d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by osd.6.0:1323 2014-08-11 16:45:14.564319 2014-08-13 18:06:06.658649 7f5c973a57a0 20 read_log 192888'15178 (192888'15177) modify d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.1315259.0:5112 2014-08-11 16:45:14.556390 2014-08-13 18:06:06.658662 7f5c973a57a0 20 read_log 192888'15191 (0'0) promote e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by osd.6.0:1367 2014-08-11 16:45:39.527180 2014-08-13 18:06:06.658674 7f5c973a57a0 20 read_log 192888'15192 (192888'15191) modify e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.1315259.0:11739 2014-08-11 16:45:39.519574 2014-08-13 18:06:06.658686 7f5c973a57a0 20 read_log 192888'15203 (0'0) promote 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by osd.6.0:1399 2014-08-11 16:45:53.171214 2014-08-13 18:06:06.658699 7f5c973a57a0 20 read_log 192888'15204 (192888'15203) modify 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.1315259.0:15156 2014-08-11 16:45:53.163519 2014-08-13 18:06:06.658713 7f5c973a57a0 20 read_log 192888'15213 (0'0) promote f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by osd.6.0:1409 2014-08-11 16:45:59.015641 2014-08-13 18:06:06.658728 7f5c973a57a0 20 read_log 192888'15214 (192888'15213) modify f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.1315259.0:16659 2014-08-11 16:45:59.008496 2014-08-13 18:06:06.658742 7f5c973a57a0 20 read_log 192888'15221 (0'0) promote 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:1419 2014-08-11 16:46:07.962131 2014-08-13 18:06:06.658755 7f5c973a57a0 20 read_log 192888'15222 (192888'15221) modify 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by client.1315259.0:18945 2014-08-11 16:46:07.954327 2014-08-13 18:06:06.658778 7f5c973a57a0 20 read_log 192888'15227 (0'0) promote a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by osd.6.0:1425 2014-08-11 16:46:11.558924 2014-08-13 18:06:06.658794 7f5c973a57a0 20 read_log 192888'15228 (192888'15227) modify a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.1315259.0:19846 2014-08-11 16:46:11.551765 2014-08-13 18:06:06.658807 7f5c973a57a0 20 read_log 192888'15235 (0'0) promote 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by osd.6.0:1433 2014-08-11 16:46:18.526363 2014-08-13 18:06:06.658821 7f5c973a57a0 20 read_log 192888'15236 (192888'15235) modify 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.1315259.0:21696 2014-08-11 16:46:18.518513 2014-08-13 18:06:06.658833 7f5c973a57a0 20 read_log 192888'15239 (0'0) promote e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by osd.6.0:1437 2014-08-11 16:46:19.167623 2014-08-13 18:06:06.658848 7f5c973a57a0 20 read_log 192888'15240 (192888'15239) modify e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.1315259.0:21869 2014-08-11 16:46:19.160102 2014-08-13 18:06:06.658861 7f5c973a57a0 20 read_log 192888'15249 (0'0) promote 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:1457 2014-08-11 16:46:34.133558 2014-08-13 18:06:06.658873 7f5c973a57a0 20 read_log 192888'15250 (192888'15249) modify 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by client.1315259.0:25633 2014-08-11 16:46:34.124109 2014-08-13 18:06:06.658887 7f5c973a57a0 20 read_log 192888'15251 (0'0) promote 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:1465 2014-08-11 16:46:39.111666 2014-08-13 18:06:06.658902 7f5c973a57a0 20 read_log 192888'15252 (192888'15251) modify 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by client.1315259.0:26811 2014-08-11 16:46:39.103959 2014-08-13 18:06:06.658918 7f5c973a57a0 20 read_log 192888'15253 (0'0) promote 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by osd.6.0:1469 2014-08-11 16:46:42.295977 2014-08-13 18:06:06.658932 7f5c973a57a0 20 read_log 192888'15254 (192888'15253) modify 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.1315259.0:27573 2014-08-11 16:46:42.288439 2014-08-13 18:06:06.658945 7f5c973a57a0 20 read_log 192888'15255 (0'0) promote 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by osd.6.0:1471 2014-08-11 16:46:43.513034 2014-08-13 18:06:06.658959 7f5c973a57a0 20 read_log 192888'15256 (192888'15255) modify 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.1315259.0:27885 2014-08-11 16:46:43.505253 2014-08-13 18:06:06.658972 7f5c973a57a0 20 read_log 192888'15257 (0'0) promote d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by osd.6.0:1475 2014-08-11 16:46:46.348974 2014-08-13 18:06:06.658985 7f5c973a57a0 20 read_log 192888'15258 (192888'15257) modify d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.1315259.0:28561 2014-08-11 16:46:46.342019 2014-08-13 18:06:06.658999 7f5c973a57a0 20 read_log 192888'15259 (0'0) promote ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by osd.6.0:1485 2014-08-11 16:46:50.146443 2014-08-13 18:06:06.659012 7f5c973a57a0 20 read_log 192888'15260 (192888'15259) modify ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.1315259.0:29500 2014-08-11 16:46:50.140126 2014-08-13 18:06:06.659025 7f5c973a57a0 20 read_log 192888'15261 (0'0) promote 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by osd.6.0:1487 2014-08-11 16:46:50.236077 2014-08-13 18:06:06.659040 7f5c973a57a0 20 read_log 192888'15262 (192888'15261) modify 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.1315259.0:29524 2014-08-11 16:46:50.229242 2014-08-13 18:06:06.659079 7f5c973a57a0 20 read_log 192888'15263 (0'0) promote 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by osd.6.0:1489 2014-08-11 16:46:50.462485 2014-08-13 18:06:06.659095 7f5c973a57a0 20 read_log 192888'15264 (192888'15263) modify 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.1315259.0:29583 2014-08-11 16:46:50.454838 2014-08-13 18:06:06.659107 7f5c973a57a0 20 read_log 192888'15267 (0'0) promote 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:1497 2014-08-11 16:46:54.526811 2014-08-13 18:06:06.659121 7f5c973a57a0 20 read_log 192888'15268 (192888'15267) modify 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by client.1315259.0:30640 2014-08-11 16:46:54.519778 2014-08-13 18:06:06.659134 7f5c973a57a0 20 read_log 192888'15269 (192888'15164) modify 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.1315259.0:31437 2014-08-11 16:46:56.095665 2014-08-13 18:06:06.659153 7f5c973a57a0 20 read_log 192888'15270 (192888'15166) modify a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.1315259.0:32710 2014-08-11 16:46:56.449283 2014-08-13 18:06:06.659166 7f5c973a57a0 20 read_log 192888'15271 (192888'15168) modify 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by client.1315259.0:33273 2014-08-11 16:46:56.594069 2014-08-13 18:06:06.659179 7f5c973a57a0 20 read_log 192888'15272 (192888'15170) modify dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.1315259.0:33314 2014-08-11 16:46:56.605325 2014-08-13 18:06:06.659212 7f5c973a57a0 20 read_log 192888'15275 (192888'15176) modify a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.1315259.0:35267 2014-08-11 16:46:57.118314 2014-08-13 18:06:06.659228 7f5c973a57a0 20 read_log 192888'15276 (192888'15178) modify d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.1315259.0:35839 2014-08-11 16:46:57.258935 2014-08-13 18:06:06.659240 7f5c973a57a0 20 read_log 192888'15283 (192888'15192) modify e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.1315259.0:42466 2014-08-11 16:46:59.020163 2014-08-13 18:06:06.659257 7f5c973a57a0 20 read_log 192888'15289 (192888'15204) modify 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.1315259.0:45883 2014-08-11 16:46:59.999416 2014-08-13 18:06:06.659272 7f5c973a57a0 20 read_log 192888'15294 (192888'15214) modify f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.1315259.0:47386 2014-08-11 16:47:00.441597 2014-08-13 18:06:06.659285 7f5c973a57a0 20 read_log 192888'15298 (192888'15222) modify 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by client.1315259.0:49672 2014-08-11 16:47:01.139619 2014-08-13 18:06:06.659299 7f5c973a57a0 20 read_log 192888'15301 (192888'15228) modify a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.1315259.0:50573 2014-08-11 16:47:01.412235 2014-08-13 18:06:06.659317 7f5c973a57a0 20 read_log 192888'15305 (192888'15236) modify 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.1315259.0:52423 2014-08-11 16:47:01.969318 2014-08-13 18:06:06.659330 7f5c973a57a0 20 read_log 192888'15307 (192888'15240) modify e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.1315259.0:52596 2014-08-11 16:47:02.017567 2014-08-13 18:06:06.659342 7f5c973a57a0 20 read_log 192888'15312 (192888'15250) modify 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by client.1315259.0:56360 2014-08-11 16:47:03.181680 2014-08-13 18:06:06.659358 7f5c973a57a0 20 read_log 192888'15313 (192888'15252) modify 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by client.1315259.0:57538 2014-08-11 16:47:03.542420 2014-08-13 18:06:06.659372 7f5c973a57a0 20 read_log 192888'15314 (192888'15254) modify 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.1315259.0:58300 2014-08-11 16:47:03.761194 2014-08-13 18:06:06.659405 7f5c973a57a0 20 read_log 192888'15315 (192888'15256) modify 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.1315259.0:58612 2014-08-11 16:47:03.853339 2014-08-13 18:06:06.659422 7f5c973a57a0 20 read_log 192888'15316 (192888'15258) modify d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.1315259.0:59288 2014-08-11 16:47:04.066933 2014-08-13 18:06:06.659445 7f5c973a57a0 20 read_log 192888'15317 (192888'15260) modify ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.1315259.0:60227 2014-08-11 16:47:04.386566 2014-08-13 18:06:06.659458 7f5c973a57a0 20 read_log 192888'15318 (192888'15262) modify 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.1315259.0:60251 2014-08-11 16:47:04.392415 2014-08-13 18:06:06.659471 7f5c973a57a0 20 read_log 192888'15319 (192888'15264) modify 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.1315259.0:60310 2014-08-11 16:47:04.408163 2014-08-13 18:06:06.659491 7f5c973a57a0 20 read_log 192888'15321 (192888'15268) modify 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by client.1315259.0:61367 2014-08-11 16:47:04.717233 2014-08-13 18:06:06.659504 7f5c973a57a0 20 read_log 192888'15322 (192888'15322) modify cd/hit_set_24.cd_archive_2014-08-11 16:44:57.980787_2014-08-11 17:05:16.882120/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.882347 2014-08-13 18:06:06.659517 7f5c973a57a0 20 read_log 192888'15323 (192888'15162) delete cd/hit_set_24.cd_archive_2014-08-11 16:02:34.118089_2014-08-11 16:44:57.980567/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.882347 2014-08-13 18:06:06.659537 7f5c973a57a0 20 read_log 192888'15324 (0'0) promote d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by osd.6.0:1614 2014-08-11 17:05:16.885330 2014-08-13 18:06:06.659553 7f5c973a57a0 20 read_log 192888'15325 (192888'15324) modify d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.942868.0:447 2014-08-11 17:05:16.876716 2014-08-13 18:06:06.659566 7f5c973a57a0 20 read_log 192888'15334 (0'0) promote 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by osd.6.0:1634 2014-08-11 17:05:31.810239 2014-08-13 18:06:06.659578 7f5c973a57a0 20 read_log 192888'15335 (192888'15334) modify 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.942868.0:4292 2014-08-11 17:05:31.802605 2014-08-13 18:06:06.659596 7f5c973a57a0 20 read_log 192888'15336 (0'0) promote 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by osd.6.0:1638 2014-08-11 17:05:33.354405 2014-08-13 18:06:06.659609 7f5c973a57a0 20 read_log 192888'15337 (192888'15336) modify 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.942868.0:4695 2014-08-11 17:05:33.347048 2014-08-13 18:06:06.659622 7f5c973a57a0 20 read_log 192888'15342 (0'0) promote 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by osd.6.0:1656 2014-08-11 17:05:41.667635 2014-08-13 18:06:06.659634 7f5c973a57a0 20 read_log 192888'15343 (192888'15342) modify 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.942868.0:6917 2014-08-11 17:05:41.656361 2014-08-13 18:06:06.659649 7f5c973a57a0 20 read_log 192888'15346 (0'0) promote 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:1666 2014-08-11 17:05:45.873070 2014-08-13 18:06:06.659662 7f5c973a57a0 20 read_log 192888'15347 (192888'15346) modify 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by client.942868.0:8013 2014-08-11 17:05:45.865075 2014-08-13 18:06:06.659675 7f5c973a57a0 20 read_log 192888'15354 (0'0) promote 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by osd.6.0:1674 2014-08-11 17:05:49.858991 2014-08-13 18:06:06.659697 7f5c973a57a0 20 read_log 192888'15355 (192888'15354) modify 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.942868.0:9094 2014-08-11 17:05:49.853198 2014-08-13 18:06:06.659713 7f5c973a57a0 20 read_log 192888'15358 (0'0) promote bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by osd.6.0:1682 2014-08-11 17:05:51.572770 2014-08-13 18:06:06.659725 7f5c973a57a0 20 read_log 192888'15359 (192888'15358) modify bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.942868.0:9549 2014-08-11 17:05:51.565029 2014-08-13 18:06:06.659738 7f5c973a57a0 20 read_log 192888'15360 (0'0) promote 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by osd.6.0:1684 2014-08-11 17:05:51.854970 2014-08-13 18:06:06.659762 7f5c973a57a0 20 read_log 192888'15361 (192888'15360) modify 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.942868.0:9627 2014-08-11 17:05:51.847632 2014-08-13 18:06:06.659775 7f5c973a57a0 20 read_log 192888'15364 (0'0) promote 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by osd.6.0:1694 2014-08-11 17:05:56.202866 2014-08-13 18:06:06.659787 7f5c973a57a0 20 read_log 192888'15365 (192888'15364) modify 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.942868.0:10834 2014-08-11 17:05:56.195643 2014-08-13 18:06:06.659800 7f5c973a57a0 20 read_log 192888'15366 (0'0) promote 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by osd.6.0:1698 2014-08-11 17:05:59.325041 2014-08-13 18:06:06.659813 7f5c973a57a0 20 read_log 192888'15367 (192888'15366) modify 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.942868.0:11683 2014-08-11 17:05:59.318291 2014-08-13 18:06:06.659825 7f5c973a57a0 20 read_log 192888'15368 (0'0) promote 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by osd.6.0:1700 2014-08-11 17:06:00.128563 2014-08-13 18:06:06.659838 7f5c973a57a0 20 read_log 192888'15369 (192888'15368) modify 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.942868.0:11893 2014-08-11 17:06:00.121250 2014-08-13 18:06:06.659850 7f5c973a57a0 20 read_log 192888'15374 (0'0) promote f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:1708 2014-08-11 17:06:03.808380 2014-08-13 18:06:06.659862 7f5c973a57a0 20 read_log 192888'15375 (192888'15374) modify f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by client.942868.0:12895 2014-08-11 17:06:03.801492 2014-08-13 18:06:06.659874 7f5c973a57a0 20 read_log 192888'15376 (0'0) promote da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by osd.6.0:1710 2014-08-11 17:06:04.705753 2014-08-13 18:06:06.659886 7f5c973a57a0 20 read_log 192888'15377 (192888'15376) modify da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.942868.0:13144 2014-08-11 17:06:04.698517 2014-08-13 18:06:06.659899 7f5c973a57a0 20 read_log 192888'15378 (0'0) promote 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by osd.6.0:1724 2014-08-11 17:06:12.130556 2014-08-13 18:06:06.659911 7f5c973a57a0 20 read_log 192888'15379 (192888'15378) modify 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.942868.0:14977 2014-08-11 17:06:12.123719 2014-08-13 18:06:06.659923 7f5c973a57a0 20 read_log 192888'15380 (0'0) promote 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by osd.6.0:1726 2014-08-11 17:06:13.246123 2014-08-13 18:06:06.659936 7f5c973a57a0 20 read_log 192888'15381 (192888'15380) modify 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.942868.0:15262 2014-08-11 17:06:13.238221 2014-08-13 18:06:06.659950 7f5c973a57a0 20 read_log 192888'15382 (0'0) promote 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by osd.6.0:1736 2014-08-11 17:06:22.056339 2014-08-13 18:06:06.659970 7f5c973a57a0 20 read_log 192888'15383 (192888'15382) modify 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.942868.0:17528 2014-08-11 17:06:22.049245 2014-08-13 18:06:06.659985 7f5c973a57a0 20 read_log 192888'15384 (0'0) promote 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:1738 2014-08-11 17:06:25.821010 2014-08-13 18:06:06.659997 7f5c973a57a0 20 read_log 192888'15385 (192888'15384) modify 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by client.942868.0:18526 2014-08-11 17:06:25.813580 2014-08-13 18:06:06.660012 7f5c973a57a0 20 read_log 192888'15388 (0'0) promote 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:1742 2014-08-11 17:06:27.613351 2014-08-13 18:06:06.660024 7f5c973a57a0 20 read_log 192888'15389 (192888'15388) modify 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by client.942868.0:19003 2014-08-11 17:06:27.604916 2014-08-13 18:06:06.660038 7f5c973a57a0 20 read_log 192888'15390 (0'0) promote 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by osd.6.0:1744 2014-08-11 17:06:30.695742 2014-08-13 18:06:06.660051 7f5c973a57a0 20 read_log 192888'15391 (192888'15390) modify 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.942868.0:19824 2014-08-11 17:06:30.688216 2014-08-13 18:06:06.660063 7f5c973a57a0 20 read_log 192888'15402 (0'0) promote e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by osd.6.0:1768 2014-08-11 17:06:38.903230 2014-08-13 18:06:06.660075 7f5c973a57a0 20 read_log 192888'15403 (192888'15402) modify e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.942868.0:21980 2014-08-11 17:06:38.896300 2014-08-13 18:06:06.660088 7f5c973a57a0 20 read_log 192888'15404 (0'0) promote d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by osd.6.0:1770 2014-08-11 17:06:39.426853 2014-08-13 18:06:06.660102 7f5c973a57a0 20 read_log 192888'15405 (192888'15404) modify d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.942868.0:22123 2014-08-11 17:06:39.419437 2014-08-13 18:06:06.660114 7f5c973a57a0 20 read_log 192888'15408 (0'0) promote ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by osd.6.0:1776 2014-08-11 17:06:42.211245 2014-08-13 18:06:06.660128 7f5c973a57a0 20 read_log 192888'15409 (192888'15408) modify ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.942868.0:22825 2014-08-11 17:06:42.203981 2014-08-13 18:06:06.660141 7f5c973a57a0 20 read_log 192888'15412 (0'0) promote ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:1780 2014-08-11 17:06:43.580701 2014-08-13 18:06:06.660160 7f5c973a57a0 20 read_log 192888'15413 (192888'15412) modify ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by client.942868.0:23164 2014-08-11 17:06:43.574231 2014-08-13 18:06:06.660173 7f5c973a57a0 20 read_log 192888'15416 (0'0) promote 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:1784 2014-08-11 17:06:45.608406 2014-08-13 18:06:06.660185 7f5c973a57a0 20 read_log 192888'15417 (192888'15416) modify 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by client.942868.0:23702 2014-08-11 17:06:45.601126 2014-08-13 18:06:06.660197 7f5c973a57a0 20 read_log 192888'15422 (0'0) promote 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by osd.6.0:1792 2014-08-11 17:06:48.842229 2014-08-13 18:06:06.660211 7f5c973a57a0 20 read_log 192888'15423 (192888'15422) modify 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.942868.0:24575 2014-08-11 17:06:48.837014 2014-08-13 18:06:06.660224 7f5c973a57a0 20 read_log 192888'15424 (0'0) promote 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by osd.6.0:1798 2014-08-11 17:06:56.808283 2014-08-13 18:06:06.660245 7f5c973a57a0 20 read_log 192888'15425 (192888'15424) modify 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.942868.0:26632 2014-08-11 17:06:56.801293 2014-08-13 18:06:06.660260 7f5c973a57a0 20 read_log 192888'15426 (0'0) promote dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by osd.6.0:1808 2014-08-11 17:06:59.633368 2014-08-13 18:06:06.660275 7f5c973a57a0 20 read_log 192888'15427 (192888'15426) modify dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.942868.0:27385 2014-08-11 17:06:59.625493 2014-08-13 18:06:06.660288 7f5c973a57a0 20 read_log 192888'15432 (0'0) promote 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by osd.6.0:1834 2014-08-11 17:07:11.019150 2014-08-13 18:06:06.660300 7f5c973a57a0 20 read_log 192888'15433 (192888'15432) modify 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.942868.0:30295 2014-08-11 17:07:11.011931 2014-08-13 18:06:06.660316 7f5c973a57a0 20 read_log 192888'15434 (0'0) promote d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:1838 2014-08-11 17:07:14.279657 2014-08-13 18:06:06.660328 7f5c973a57a0 20 read_log 192888'15435 (192888'15434) modify d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by client.942868.0:31124 2014-08-11 17:07:14.271810 2014-08-13 18:06:06.660340 7f5c973a57a0 20 read_log 192888'15436 (192888'15325) modify d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.942868.0:31713 2014-08-11 17:07:16.036133 2014-08-13 18:06:06.660353 7f5c973a57a0 20 read_log 192888'15441 (192888'15335) modify 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.942868.0:35558 2014-08-11 17:07:17.103824 2014-08-13 18:06:06.660365 7f5c973a57a0 20 read_log 192888'15442 (192888'15337) modify 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.942868.0:35961 2014-08-11 17:07:17.206816 2014-08-13 18:06:06.660377 7f5c973a57a0 20 read_log 192888'15445 (192888'15343) modify 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.942868.0:38183 2014-08-11 17:07:17.816231 2014-08-13 18:06:06.660398 7f5c973a57a0 20 read_log 192888'15447 (192888'15347) modify 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by client.942868.0:39279 2014-08-11 17:07:18.137715 2014-08-13 18:06:06.660414 7f5c973a57a0 20 read_log 192888'15451 (192888'15355) modify 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.942868.0:40360 2014-08-11 17:07:18.429592 2014-08-13 18:06:06.660426 7f5c973a57a0 20 read_log 192888'15453 (192888'15359) modify bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.942868.0:40815 2014-08-11 17:07:18.554453 2014-08-13 18:06:06.660451 7f5c973a57a0 20 read_log 192888'15454 (192888'15361) modify 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.942868.0:40893 2014-08-11 17:07:18.574844 2014-08-13 18:06:06.660467 7f5c973a57a0 20 read_log 192888'15456 (192888'15365) modify 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.942868.0:42100 2014-08-11 17:07:18.905739 2014-08-13 18:06:06.660480 7f5c973a57a0 20 read_log 192888'15457 (192888'15367) modify 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.942868.0:42949 2014-08-11 17:07:19.136104 2014-08-13 18:06:06.660500 7f5c973a57a0 20 read_log 192888'15458 (192888'15369) modify 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.942868.0:43159 2014-08-11 17:07:19.196037 2014-08-13 18:06:06.660517 7f5c973a57a0 20 read_log 192888'15461 (192888'15375) modify f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by client.942868.0:44161 2014-08-11 17:07:19.487549 2014-08-13 18:06:06.660529 7f5c973a57a0 20 read_log 192888'15462 (192888'15377) modify da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.942868.0:44410 2014-08-11 17:07:19.559049 2014-08-13 18:06:06.660566 7f5c973a57a0 20 read_log 192888'15463 (192888'15379) modify 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.942868.0:46243 2014-08-11 17:07:20.065633 2014-08-13 18:06:06.660583 7f5c973a57a0 20 read_log 192888'15464 (192888'15381) modify 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.942868.0:46528 2014-08-11 17:07:20.149099 2014-08-13 18:06:06.660600 7f5c973a57a0 20 read_log 192888'15465 (192888'15383) modify 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.942868.0:48794 2014-08-11 17:07:20.819233 2014-08-13 18:06:06.660616 7f5c973a57a0 20 read_log 192888'15466 (192888'15385) modify 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by client.942868.0:49792 2014-08-11 17:07:21.123059 2014-08-13 18:06:06.660634 7f5c973a57a0 20 read_log 192888'15468 (192888'15389) modify 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by client.942868.0:50269 2014-08-11 17:07:21.265447 2014-08-13 18:06:06.660647 7f5c973a57a0 20 read_log 192888'15469 (192888'15391) modify 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.942868.0:51090 2014-08-11 17:07:21.528505 2014-08-13 18:06:06.660659 7f5c973a57a0 20 read_log 192888'15475 (192888'15403) modify e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.942868.0:53246 2014-08-11 17:07:22.197644 2014-08-13 18:06:06.660672 7f5c973a57a0 20 read_log 192888'15476 (192888'15405) modify d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.942868.0:53389 2014-08-11 17:07:22.247965 2014-08-13 18:06:06.660684 7f5c973a57a0 20 read_log 192888'15478 (192888'15409) modify ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.942868.0:54091 2014-08-11 17:07:22.461397 2014-08-13 18:06:06.660696 7f5c973a57a0 20 read_log 192888'15480 (192888'15413) modify ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by client.942868.0:54430 2014-08-11 17:07:22.554534 2014-08-13 18:06:06.660709 7f5c973a57a0 20 read_log 192888'15482 (192888'15417) modify 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by client.942868.0:54968 2014-08-11 17:07:22.706277 2014-08-13 18:06:06.660721 7f5c973a57a0 20 read_log 192888'15485 (192888'15423) modify 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.942868.0:55841 2014-08-11 17:07:22.972775 2014-08-13 18:06:06.660733 7f5c973a57a0 20 read_log 192888'15486 (192888'15425) modify 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.942868.0:57898 2014-08-11 17:07:23.586382 2014-08-13 18:06:06.660746 7f5c973a57a0 20 read_log 192888'15487 (192888'15427) modify dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.942868.0:58651 2014-08-11 17:07:23.809555 2014-08-13 18:06:06.660758 7f5c973a57a0 20 read_log 192888'15490 (192888'15433) modify 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.942868.0:61561 2014-08-11 17:07:24.680622 2014-08-13 18:06:06.660770 7f5c973a57a0 20 read_log 192888'15491 (192888'15435) modify d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by client.942868.0:62390 2014-08-11 17:07:24.934421 2014-08-13 18:06:06.660782 7f5c973a57a0 20 read_log 193305'15492 (193305'15492) modify cd/hit_set_24.cd_archive_2014-08-11 18:37:04.943507_2014-08-11 18:42:36.902876/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.903321 2014-08-13 18:06:06.660795 7f5c973a57a0 20 read_log 193305'15493 (192888'15322) delete cd/hit_set_24.cd_archive_2014-08-11 16:44:57.980787_2014-08-11 17:05:16.882120/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.903321 2014-08-13 18:06:06.660807 7f5c973a57a0 20 read_log 193305'15494 (0'0) promote e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:4 2014-08-11 18:42:36.926482 2014-08-13 18:06:06.660827 7f5c973a57a0 20 read_log 193305'15495 (193305'15494) modify e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by client.951337.0:981 2014-08-11 18:42:36.898841 2014-08-13 18:06:06.660842 7f5c973a57a0 20 read_log 193305'15496 (0'0) promote e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by osd.6.0:6 2014-08-11 18:42:36.927330 2014-08-13 18:06:06.660855 7f5c973a57a0 20 read_log 193305'15497 (193305'15496) modify e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.951337.0:983 2014-08-11 18:42:36.905811 2014-08-13 18:06:06.660867 7f5c973a57a0 20 read_log 193305'15498 (0'0) promote 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by osd.6.0:10 2014-08-11 18:42:40.448617 2014-08-13 18:06:06.660887 7f5c973a57a0 20 read_log 193305'15499 (193305'15498) modify 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.951337.0:1913 2014-08-11 18:42:40.442025 2014-08-13 18:06:06.660900 7f5c973a57a0 20 read_log 193305'15500 (0'0) promote d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by osd.6.0:17 2014-08-11 18:42:43.061959 2014-08-13 18:06:06.660913 7f5c973a57a0 20 read_log 193305'15501 (193305'15500) modify d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.951337.0:2635 2014-08-11 18:42:43.055621 2014-08-13 18:06:06.660925 7f5c973a57a0 20 read_log 193305'15502 (0'0) promote f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by osd.6.0:21 2014-08-11 18:42:44.301399 2014-08-13 18:06:06.660937 7f5c973a57a0 20 read_log 193305'15503 (193305'15502) modify f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.951337.0:2976 2014-08-11 18:42:44.294613 2014-08-13 18:06:06.660949 7f5c973a57a0 20 read_log 193305'15504 (0'0) promote 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:35 2014-08-11 18:42:55.827476 2014-08-13 18:06:06.660961 7f5c973a57a0 20 read_log 193305'15505 (193305'15504) modify 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by client.951337.0:6109 2014-08-11 18:42:55.818242 2014-08-13 18:06:06.660976 7f5c973a57a0 20 read_log 193305'15506 (0'0) promote 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:39 2014-08-11 18:42:57.498207 2014-08-13 18:06:06.660988 7f5c973a57a0 20 read_log 193305'15507 (193305'15506) modify 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by client.951337.0:6571 2014-08-11 18:42:57.490010 2014-08-13 18:06:06.661003 7f5c973a57a0 20 read_log 193305'15508 (0'0) promote 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by osd.6.0:47 2014-08-11 18:43:03.313376 2014-08-13 18:06:06.661018 7f5c973a57a0 20 read_log 193305'15509 (193305'15508) modify 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.951337.0:8175 2014-08-11 18:43:03.303280 2014-08-13 18:06:06.661031 7f5c973a57a0 20 read_log 193305'15510 (0'0) promote da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:51 2014-08-11 18:43:05.772640 2014-08-13 18:06:06.661045 7f5c973a57a0 20 read_log 193305'15511 (193305'15510) modify da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by client.951337.0:8779 2014-08-11 18:43:05.765411 2014-08-13 18:06:06.661060 7f5c973a57a0 20 read_log 193305'15512 (0'0) promote 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:53 2014-08-11 18:43:07.663883 2014-08-13 18:06:06.661072 7f5c973a57a0 20 read_log 193305'15513 (193305'15512) modify 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by client.951337.0:9285 2014-08-11 18:43:07.657511 2014-08-13 18:06:06.661086 7f5c973a57a0 20 read_log 193305'15514 (0'0) promote 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:57 2014-08-11 18:43:08.349300 2014-08-13 18:06:06.661107 7f5c973a57a0 20 read_log 193305'15515 (193305'15514) modify 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by client.951337.0:9468 2014-08-11 18:43:08.342094 2014-08-13 18:06:06.661122 7f5c973a57a0 20 read_log 193305'15516 (0'0) promote a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by osd.6.0:65 2014-08-11 18:43:12.602268 2014-08-13 18:06:06.661135 7f5c973a57a0 20 read_log 193305'15517 (193305'15516) modify a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.951337.0:10622 2014-08-11 18:43:12.595214 2014-08-13 18:06:06.661158 7f5c973a57a0 20 read_log 193305'15518 (0'0) promote dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by osd.6.0:69 2014-08-11 18:43:14.085572 2014-08-13 18:06:06.661173 7f5c973a57a0 20 read_log 193305'15519 (193305'15518) modify dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.951337.0:11020 2014-08-11 18:43:14.075760 2014-08-13 18:06:06.661186 7f5c973a57a0 20 read_log 193305'15520 (0'0) promote 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:71 2014-08-11 18:43:15.841663 2014-08-13 18:06:06.661199 7f5c973a57a0 20 read_log 193305'15521 (193305'15520) modify 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by client.951337.0:11443 2014-08-11 18:43:15.833237 2014-08-13 18:06:06.661211 7f5c973a57a0 20 read_log 193305'15522 (0'0) promote 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by osd.6.0:80 2014-08-11 18:43:17.958735 2014-08-13 18:06:06.661223 7f5c973a57a0 20 read_log 193305'15523 (193305'15522) modify 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.951337.0:12018 2014-08-11 18:43:17.951291 2014-08-13 18:06:06.661236 7f5c973a57a0 20 read_log 193305'15524 (0'0) promote 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by osd.6.0:82 2014-08-11 18:43:21.110679 2014-08-13 18:06:06.661248 7f5c973a57a0 20 read_log 193305'15525 (193305'15524) modify 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.951337.0:12859 2014-08-11 18:43:21.103976 2014-08-13 18:06:06.661261 7f5c973a57a0 20 read_log 193305'15526 (0'0) promote da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by osd.6.0:84 2014-08-11 18:43:25.150463 2014-08-13 18:06:06.661276 7f5c973a57a0 20 read_log 193305'15527 (193305'15526) modify da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.951337.0:13934 2014-08-11 18:43:25.140910 2014-08-13 18:06:06.661289 7f5c973a57a0 20 read_log 193305'15528 (0'0) promote eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:92 2014-08-11 18:43:27.094127 2014-08-13 18:06:06.661301 7f5c973a57a0 20 read_log 193305'15529 (193305'15528) modify eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by client.951337.0:14461 2014-08-11 18:43:27.087528 2014-08-13 18:06:06.661313 7f5c973a57a0 20 read_log 193305'15530 (0'0) promote db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by osd.6.0:96 2014-08-11 18:43:29.017437 2014-08-13 18:06:06.661326 7f5c973a57a0 20 read_log 193305'15531 (193305'15530) modify db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.951337.0:15010 2014-08-11 18:43:29.004947 2014-08-13 18:06:06.661338 7f5c973a57a0 20 read_log 193305'15532 (0'0) promote 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:98 2014-08-11 18:43:33.516706 2014-08-13 18:06:06.661350 7f5c973a57a0 20 read_log 193305'15533 (193305'15532) modify 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by client.951337.0:16301 2014-08-11 18:43:33.508755 2014-08-13 18:06:06.661362 7f5c973a57a0 20 read_log 193305'15534 (0'0) promote 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:100 2014-08-11 18:43:35.665089 2014-08-13 18:06:06.661384 7f5c973a57a0 20 read_log 193305'15535 (193305'15534) modify 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by client.951337.0:16920 2014-08-11 18:43:35.657157 2014-08-13 18:06:06.661409 7f5c973a57a0 20 read_log 193305'15536 (0'0) promote 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by osd.6.0:104 2014-08-11 18:43:36.034807 2014-08-13 18:06:06.661432 7f5c973a57a0 20 read_log 193305'15537 (193305'15536) modify 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.951337.0:17022 2014-08-11 18:43:36.027478 2014-08-13 18:06:06.661455 7f5c973a57a0 20 read_log 193305'15538 (0'0) promote 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by osd.6.0:108 2014-08-11 18:43:38.236503 2014-08-13 18:06:06.661470 7f5c973a57a0 20 read_log 193305'15539 (193305'15538) modify 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.951337.0:17684 2014-08-11 18:43:38.229708 2014-08-13 18:06:06.661483 7f5c973a57a0 20 read_log 193305'15540 (0'0) promote 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:112 2014-08-11 18:43:42.504735 2014-08-13 18:06:06.661507 7f5c973a57a0 20 read_log 193305'15541 (193305'15540) modify 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by client.951337.0:18885 2014-08-11 18:43:42.496612 2014-08-13 18:06:06.661520 7f5c973a57a0 20 read_log 193305'15542 (0'0) promote 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:118 2014-08-11 18:43:47.741915 2014-08-13 18:06:06.661533 7f5c973a57a0 20 read_log 193305'15543 (193305'15542) modify 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by client.951337.0:20296 2014-08-11 18:43:47.732414 2014-08-13 18:06:06.661545 7f5c973a57a0 20 read_log 193305'15544 (0'0) promote 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:120 2014-08-11 18:43:51.888641 2014-08-13 18:06:06.661557 7f5c973a57a0 20 read_log 193305'15545 (193305'15544) modify 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by client.951337.0:21467 2014-08-11 18:43:51.881425 2014-08-13 18:06:06.661570 7f5c973a57a0 20 read_log 193305'15546 (0'0) promote e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by osd.6.0:128 2014-08-11 18:43:57.805777 2014-08-13 18:06:06.661582 7f5c973a57a0 20 read_log 193305'15547 (193305'15546) modify e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.951337.0:23105 2014-08-11 18:43:57.798458 2014-08-13 18:06:06.661594 7f5c973a57a0 20 read_log 193305'15548 (0'0) promote 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by osd.6.0:138 2014-08-11 18:44:06.822851 2014-08-13 18:06:06.661606 7f5c973a57a0 20 read_log 193305'15549 (193305'15548) modify 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.951337.0:25589 2014-08-11 18:44:06.815563 2014-08-13 18:06:06.661628 7f5c973a57a0 20 read_log 193305'15550 (0'0) promote 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by osd.6.0:144 2014-08-11 18:44:10.392901 2014-08-13 18:06:06.661642 7f5c973a57a0 20 read_log 193305'15551 (193305'15550) modify 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.951337.0:26557 2014-08-11 18:44:10.385017 2014-08-13 18:06:06.661654 7f5c973a57a0 20 read_log 193305'15552 (0'0) promote aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by osd.6.0:146 2014-08-11 18:44:10.732854 2014-08-13 18:06:06.661667 7f5c973a57a0 20 read_log 193305'15553 (193305'15552) modify aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.951337.0:26649 2014-08-11 18:44:10.724564 2014-08-13 18:06:06.661679 7f5c973a57a0 20 read_log 193305'15554 (0'0) promote 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:150 2014-08-11 18:44:11.355989 2014-08-13 18:06:06.661698 7f5c973a57a0 20 read_log 193305'15555 (193305'15554) modify 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by client.951337.0:26824 2014-08-11 18:44:11.345742 2014-08-13 18:06:06.661713 7f5c973a57a0 20 read_log 193305'15556 (0'0) promote 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by osd.6.0:152 2014-08-11 18:44:12.869699 2014-08-13 18:06:06.661726 7f5c973a57a0 20 read_log 193305'15557 (193305'15556) modify 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.951337.0:27218 2014-08-11 18:44:12.859720 2014-08-13 18:06:06.661738 7f5c973a57a0 20 read_log 193305'15558 (0'0) promote b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:154 2014-08-11 18:44:13.025345 2014-08-13 18:06:06.661750 7f5c973a57a0 20 read_log 193305'15559 (193305'15558) modify b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by client.951337.0:27260 2014-08-11 18:44:13.017520 2014-08-13 18:06:06.661763 7f5c973a57a0 20 read_log 193305'15560 (0'0) promote 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by osd.6.0:156 2014-08-11 18:44:14.399127 2014-08-13 18:06:06.661775 7f5c973a57a0 20 read_log 193305'15561 (193305'15560) modify 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.951337.0:27646 2014-08-11 18:44:14.390173 2014-08-13 18:06:06.661797 7f5c973a57a0 20 read_log 193305'15562 (0'0) promote e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:162 2014-08-11 18:44:21.820513 2014-08-13 18:06:06.661810 7f5c973a57a0 20 read_log 193305'15563 (193305'15562) modify e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by client.951337.0:29684 2014-08-11 18:44:21.814306 2014-08-13 18:06:06.661823 7f5c973a57a0 20 read_log 193305'15564 (0'0) promote ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:164 2014-08-11 18:44:22.094535 2014-08-13 18:06:06.661835 7f5c973a57a0 20 read_log 193305'15565 (193305'15564) modify ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by client.951337.0:29753 2014-08-11 18:44:22.088313 2014-08-13 18:06:06.661850 7f5c973a57a0 20 read_log 193305'15566 (0'0) promote a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:166 2014-08-11 18:44:24.092571 2014-08-13 18:06:06.661863 7f5c973a57a0 20 read_log 193305'15567 (193305'15566) modify a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by client.951337.0:30270 2014-08-11 18:44:24.084289 2014-08-13 18:06:06.661875 7f5c973a57a0 20 read_log 193305'15568 (0'0) promote 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by osd.6.0:168 2014-08-11 18:44:25.377964 2014-08-13 18:06:06.661887 7f5c973a57a0 20 read_log 193305'15569 (193305'15568) modify 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.951337.0:30598 2014-08-11 18:44:25.369933 2014-08-13 18:06:06.661900 7f5c973a57a0 20 read_log 193305'15570 (0'0) promote 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:170 2014-08-11 18:44:28.561574 2014-08-13 18:06:06.661940 7f5c973a57a0 20 read_log 193305'15571 (193305'15570) modify 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by client.951337.0:31499 2014-08-11 18:44:28.554908 2014-08-13 18:06:06.661956 7f5c973a57a0 20 read_log 193305'15572 (193305'15495) modify e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by client.951337.0:33692 2014-08-11 18:44:34.583624 2014-08-13 18:06:06.661968 7f5c973a57a0 20 read_log 193305'15573 (193305'15497) modify e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.951337.0:33694 2014-08-11 18:44:34.584468 2014-08-13 18:06:06.661981 7f5c973a57a0 20 read_log 193305'15574 (193305'15499) modify 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.951337.0:34624 2014-08-11 18:44:35.099666 2014-08-13 18:06:06.662003 7f5c973a57a0 20 read_log 193305'15575 (193305'15501) modify d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.951337.0:35346 2014-08-11 18:44:35.521853 2014-08-13 18:06:06.662023 7f5c973a57a0 20 read_log 193305'15576 (193305'15503) modify f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.951337.0:35687 2014-08-11 18:44:35.703275 2014-08-13 18:06:06.662038 7f5c973a57a0 20 read_log 193305'15577 (193305'15505) modify 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by client.951337.0:38820 2014-08-11 18:44:37.677494 2014-08-13 18:06:06.662050 7f5c973a57a0 20 read_log 193305'15578 (193305'15507) modify 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by client.951337.0:39282 2014-08-11 18:44:37.921499 2014-08-13 18:06:06.662062 7f5c973a57a0 20 read_log 193305'15579 (193305'15509) modify 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.951337.0:40886 2014-08-11 18:44:38.567551 2014-08-13 18:06:06.662083 7f5c973a57a0 20 read_log 193305'15580 (193305'15511) modify da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by client.951337.0:41490 2014-08-11 18:44:38.769094 2014-08-13 18:06:06.662100 7f5c973a57a0 20 read_log 193305'15581 (193305'15513) modify 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by client.951337.0:41996 2014-08-11 18:44:38.976321 2014-08-13 18:06:06.662116 7f5c973a57a0 20 read_log 193305'15582 (193305'15515) modify 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by client.951337.0:42179 2014-08-11 18:44:39.136016 2014-08-13 18:06:06.662136 7f5c973a57a0 20 read_log 193305'15583 (193305'15517) modify a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.951337.0:43333 2014-08-11 18:44:39.557821 2014-08-13 18:06:06.662157 7f5c973a57a0 20 read_log 193305'15584 (193305'15519) modify dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.951337.0:43731 2014-08-11 18:44:39.680661 2014-08-13 18:06:06.662170 7f5c973a57a0 20 read_log 193305'15585 (193305'15521) modify 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by client.951337.0:44154 2014-08-11 18:44:39.807167 2014-08-13 18:06:06.662182 7f5c973a57a0 20 read_log 193305'15586 (193305'15523) modify 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.951337.0:44729 2014-08-11 18:44:39.971916 2014-08-13 18:06:06.662197 7f5c973a57a0 20 read_log 193305'15587 (193305'15525) modify 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.951337.0:45570 2014-08-11 18:44:40.237983 2014-08-13 18:06:06.662209 7f5c973a57a0 20 read_log 193305'15588 (193305'15527) modify da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.951337.0:46645 2014-08-11 18:44:40.546293 2014-08-13 18:06:06.662224 7f5c973a57a0 20 read_log 193305'15589 (193305'15529) modify eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by client.951337.0:47172 2014-08-11 18:44:40.693345 2014-08-13 18:06:06.662236 7f5c973a57a0 20 read_log 193305'15590 (193305'15531) modify db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.951337.0:47721 2014-08-11 18:44:40.847707 2014-08-13 18:06:06.662249 7f5c973a57a0 20 read_log 193305'15591 (193305'15533) modify 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by client.951337.0:49012 2014-08-11 18:44:41.227499 2014-08-13 18:06:06.662261 7f5c973a57a0 20 read_log 193305'15592 (193305'15535) modify 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by client.951337.0:49631 2014-08-11 18:44:41.423723 2014-08-13 18:06:06.662273 7f5c973a57a0 20 read_log 193305'15593 (193305'15537) modify 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.951337.0:49733 2014-08-11 18:44:41.452764 2014-08-13 18:06:06.662285 7f5c973a57a0 20 read_log 193305'15594 (193305'15539) modify 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.951337.0:50395 2014-08-11 18:44:41.652887 2014-08-13 18:06:06.662308 7f5c973a57a0 20 read_log 193305'15595 (193305'15541) modify 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by client.951337.0:51596 2014-08-11 18:44:41.996706 2014-08-13 18:06:06.662333 7f5c973a57a0 20 read_log 193305'15596 (193305'15543) modify 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by client.951337.0:53007 2014-08-11 18:44:42.403396 2014-08-13 18:06:06.662369 7f5c973a57a0 20 read_log 193305'15597 (193305'15545) modify 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by client.951337.0:54178 2014-08-11 18:44:42.751202 2014-08-13 18:06:06.662386 7f5c973a57a0 20 read_log 193305'15598 (193305'15547) modify e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.951337.0:55816 2014-08-11 18:44:43.241411 2014-08-13 18:06:06.662398 7f5c973a57a0 20 read_log 193305'15599 (193305'15549) modify 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.951337.0:58300 2014-08-11 18:44:44.000080 2014-08-13 18:06:06.662411 7f5c973a57a0 20 read_log 193305'15600 (193305'15551) modify 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.951337.0:59268 2014-08-11 18:44:44.277736 2014-08-13 18:06:06.662434 7f5c973a57a0 20 read_log 193305'15601 (193305'15553) modify aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.951337.0:59360 2014-08-11 18:44:44.304020 2014-08-13 18:06:06.662447 7f5c973a57a0 20 read_log 193305'15602 (193305'15555) modify 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by client.951337.0:59535 2014-08-11 18:44:44.354491 2014-08-13 18:06:06.662460 7f5c973a57a0 20 read_log 193305'15603 (193305'15557) modify 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.951337.0:59929 2014-08-11 18:44:44.469215 2014-08-13 18:06:06.662482 7f5c973a57a0 20 read_log 193305'15604 (193305'15559) modify b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by client.951337.0:59971 2014-08-11 18:44:44.481152 2014-08-13 18:06:06.662495 7f5c973a57a0 20 read_log 193305'15605 (193305'15561) modify 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.951337.0:60357 2014-08-11 18:44:44.605610 2014-08-13 18:06:06.662508 7f5c973a57a0 20 read_log 193305'15606 (193305'15563) modify e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by client.951337.0:62395 2014-08-11 18:44:45.214982 2014-08-13 18:06:06.662520 7f5c973a57a0 20 read_log 193305'15607 (193305'15565) modify ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by client.951337.0:62464 2014-08-11 18:44:45.234587 2014-08-13 18:06:06.662533 7f5c973a57a0 20 read_log 193305'15608 (193305'15567) modify a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by client.951337.0:62981 2014-08-11 18:44:45.378711 2014-08-13 18:06:06.662545 7f5c973a57a0 20 read_log 193305'15609 (193305'15569) modify 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.951337.0:63309 2014-08-11 18:44:45.467753 2014-08-13 18:06:06.662557 7f5c973a57a0 20 read_log 193305'15610 (193305'15571) modify 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by client.951337.0:64210 2014-08-11 18:44:45.736754 2014-08-13 18:06:06.662570 7f5c973a57a0 20 read_log 194209'15611 (193305'15596) clean 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:458 2014-08-12 14:07:27.199692 2014-08-13 18:06:06.662582 7f5c973a57a0 20 read_log 194209'15612 (192888'15461) clean f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:459 2014-08-12 14:07:27.199982 2014-08-13 18:06:06.662595 7f5c973a57a0 20 read_log 194209'15613 (193305'15610) clean 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:460 2014-08-12 14:07:27.200546 2014-08-13 18:06:06.662607 7f5c973a57a0 20 read_log 194209'15614 (192888'15466) clean 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:461 2014-08-12 14:07:27.200954 2014-08-13 18:06:06.662627 7f5c973a57a0 20 read_log 194209'15615 (192888'15271) clean 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:462 2014-08-12 14:07:27.270171 2014-08-13 18:06:06.662642 7f5c973a57a0 20 read_log 194209'15616 (193305'15589) clean eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:463 2014-08-12 14:07:27.273295 2014-08-13 18:06:06.662655 7f5c973a57a0 20 read_log 194209'15617 (193305'15608) clean a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:464 2014-08-12 14:07:27.276442 2014-08-13 18:06:06.662668 7f5c973a57a0 20 read_log 194209'15618 (192834'14537) clean dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.6.0:465 2014-08-12 14:07:27.279717 2014-08-13 18:06:06.662680 7f5c973a57a0 20 read_log 194209'15619 (192888'15480) clean ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:466 2014-08-12 14:07:27.284230 2014-08-13 18:06:06.662692 7f5c973a57a0 20 read_log 194209'15620 (192834'14616) clean cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.6.0:467 2014-08-12 14:07:27.287455 2014-08-13 18:06:06.662705 7f5c973a57a0 20 read_log 194209'15621 (193305'15606) clean e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:468 2014-08-12 14:07:27.293128 2014-08-13 18:06:06.662717 7f5c973a57a0 20 read_log 194209'15622 (192834'14621) clean a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.6.0:469 2014-08-12 14:07:27.293470 2014-08-13 18:06:06.662729 7f5c973a57a0 20 read_log 194209'15623 (193305'15585) clean 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:470 2014-08-12 14:07:27.296594 2014-08-13 18:06:06.662742 7f5c973a57a0 20 read_log 194209'15624 (192834'15010) clean b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.6.0:471 2014-08-12 14:07:27.299767 2014-08-13 18:06:06.662771 7f5c973a57a0 20 read_log 194209'15625 (192834'15034) clean f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.6.0:472 2014-08-12 14:07:27.303198 2014-08-13 18:06:06.662787 7f5c973a57a0 20 read_log 194209'15626 (193305'15572) clean e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:473 2014-08-12 14:07:27.306155 2014-08-13 18:06:06.662800 7f5c973a57a0 20 read_log 194209'15627 (192888'15447) clean 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:474 2014-08-12 14:07:27.309348 2014-08-13 18:06:06.662812 7f5c973a57a0 20 read_log 194209'15628 (193305'15597) clean 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:475 2014-08-12 14:07:27.312460 2014-08-13 18:06:06.662825 7f5c973a57a0 20 read_log 194209'15629 (192834'14581) clean 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.6.0:476 2014-08-12 14:07:27.315579 2014-08-13 18:06:06.662837 7f5c973a57a0 20 read_log 194209'15630 (192834'14474) clean adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.6.0:477 2014-08-12 14:07:27.318771 2014-08-13 18:06:06.662852 7f5c973a57a0 20 read_log 194209'15631 (193305'15591) clean 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:478 2014-08-12 14:07:27.321922 2014-08-13 18:06:06.662864 7f5c973a57a0 20 read_log 194209'15632 (192834'15093) clean baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.6.0:479 2014-08-12 14:07:27.325105 2014-08-13 18:06:06.662878 7f5c973a57a0 20 read_log 194209'15633 (193305'15607) clean ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:480 2014-08-12 14:07:27.328278 2014-08-13 18:06:06.662891 7f5c973a57a0 20 read_log 194209'15634 (193305'15581) clean 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:481 2014-08-12 14:07:27.331506 2014-08-13 18:06:06.662913 7f5c973a57a0 20 read_log 194209'15635 (192834'15054) clean 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.6.0:482 2014-08-12 14:07:27.334847 2014-08-13 18:06:06.662928 7f5c973a57a0 20 read_log 194209'15636 (193305'15592) clean 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:483 2014-08-12 14:07:27.338026 2014-08-13 18:06:06.662941 7f5c973a57a0 20 read_log 194209'15637 (193305'15602) clean 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:484 2014-08-12 14:07:27.341146 2014-08-13 18:06:06.662955 7f5c973a57a0 20 read_log 194209'15638 (193305'15577) clean 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:485 2014-08-12 14:07:27.343956 2014-08-13 18:06:06.662968 7f5c973a57a0 20 read_log 194209'15639 (192834'15124) clean bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.6.0:486 2014-08-12 14:07:27.346983 2014-08-13 18:06:06.662980 7f5c973a57a0 20 read_log 194209'15640 (193305'15604) clean b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:487 2014-08-12 14:07:27.350026 2014-08-13 18:06:06.662992 7f5c973a57a0 20 read_log 194209'15641 (192888'15298) clean 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:488 2014-08-12 14:07:27.353029 2014-08-13 18:06:06.663005 7f5c973a57a0 20 read_log 194209'15642 (192888'15321) clean 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:489 2014-08-12 14:07:27.356458 2014-08-13 18:06:06.663019 7f5c973a57a0 20 read_log 194209'15643 (193305'15582) clean 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:490 2014-08-12 14:07:27.359584 2014-08-13 18:06:06.663032 7f5c973a57a0 20 read_log 194209'15644 (192834'15041) clean 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.6.0:491 2014-08-12 14:07:27.362524 2014-08-13 18:06:06.663046 7f5c973a57a0 20 read_log 194209'15645 (192834'15052) clean 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.6.0:492 2014-08-12 14:07:27.365474 2014-08-13 18:06:06.663059 7f5c973a57a0 20 read_log 194209'15646 (192888'15482) clean 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:493 2014-08-12 14:07:27.368538 2014-08-13 18:06:06.663071 7f5c973a57a0 20 read_log 194209'15647 (193305'15578) clean 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:494 2014-08-12 14:07:27.371691 2014-08-13 18:06:06.663083 7f5c973a57a0 20 read_log 194209'15648 (192888'15313) clean 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:495 2014-08-12 14:07:27.377180 2014-08-13 18:06:06.663096 7f5c973a57a0 20 read_log 194209'15649 (192888'15312) clean 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:496 2014-08-12 14:07:27.377440 2014-08-13 18:06:06.663110 7f5c973a57a0 20 read_log 194209'15650 (193305'15580) clean da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:497 2014-08-12 14:07:27.380385 2014-08-13 18:06:06.663123 7f5c973a57a0 20 read_log 194209'15651 (192888'15491) clean d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:498 2014-08-12 14:07:27.383447 2014-08-13 18:06:06.663137 7f5c973a57a0 20 read_log 194209'15652 (192888'15468) clean 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:499 2014-08-12 14:07:27.386736 2014-08-13 18:06:06.663183 7f5c973a57a0 20 read_log 194209'15653 (192888'15456) clean 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by osd.6.0:500 2014-08-12 14:07:27.389799 2014-08-13 18:06:06.663201 7f5c973a57a0 20 read_log 194209'15654 (194209'15611) delete 4caf80cd/benchmark_data_storage0111-ib.pocstor_62621_object20295/head//24 by osd.6.0:501 2014-08-12 14:07:27.422055 2014-08-13 18:06:06.663245 7f5c973a57a0 20 read_log 194209'15655 (194209'15613) delete 665690cd/benchmark_data_storage0111-ib.pocstor_62621_object31498/head//24 by osd.6.0:502 2014-08-12 14:07:27.425434 2014-08-13 18:06:06.663262 7f5c973a57a0 20 read_log 194209'15656 (194209'15615) delete 9b94b0cd/benchmark_data_storage0111-ib.pocstor_41097_object2545/head//24 by osd.6.0:503 2014-08-12 14:07:27.428743 2014-08-13 18:06:06.663274 7f5c973a57a0 20 read_log 194209'15657 (194209'15616) delete eac3c0cd/benchmark_data_storage0111-ib.pocstor_62621_object14460/head//24 by osd.6.0:504 2014-08-12 14:07:27.431936 2014-08-13 18:06:06.663289 7f5c973a57a0 20 read_log 194209'15658 (194209'15617) delete a2f4e0cd/benchmark_data_storage0111-ib.pocstor_62621_object30269/head//24 by osd.6.0:505 2014-08-12 14:07:27.435070 2014-08-13 18:06:06.663304 7f5c973a57a0 20 read_log 194209'15659 (193305'15595) clean 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:506 2014-08-12 14:07:27.435396 2014-08-13 18:06:06.663317 7f5c973a57a0 20 read_log 194209'15660 (194209'15618) delete dcb3f0cd/benchmark_data_storage0111-ib.pocstor_26659_object8007/head//24 by osd.6.0:507 2014-08-12 14:07:27.438798 2014-08-13 18:06:06.663329 7f5c973a57a0 20 read_log 194209'15661 (194209'15620) delete cc9914cd/benchmark_data_storage0111-ib.pocstor_26659_object17190/head//24 by osd.6.0:508 2014-08-12 14:07:27.441683 2014-08-13 18:06:06.663341 7f5c973a57a0 20 read_log 194209'15662 (194209'15621) delete e6e224cd/benchmark_data_storage0111-ib.pocstor_62621_object29683/head//24 by osd.6.0:509 2014-08-12 14:07:27.442059 2014-08-13 18:06:06.663357 7f5c973a57a0 20 read_log 194209'15663 (194209'15622) delete a7d234cd/benchmark_data_storage0111-ib.pocstor_26659_object18271/head//24 by osd.6.0:510 2014-08-12 14:07:27.442395 2014-08-13 18:06:06.663391 7f5c973a57a0 20 read_log 194209'15664 (194209'15623) delete 788534cd/benchmark_data_storage0111-ib.pocstor_62621_object11442/head//24 by osd.6.0:511 2014-08-12 14:07:27.445487 2014-08-13 18:06:06.663406 7f5c973a57a0 20 read_log 194209'15665 (194209'15624) delete b3fa34cd/benchmark_data_storage0111-ib.pocstor_41526_object2014/head//24 by osd.6.0:512 2014-08-12 14:07:27.445850 2014-08-13 18:06:06.663432 7f5c973a57a0 20 read_log 194209'15666 (194209'15625) delete f6bb34cd/benchmark_data_storage0111-ib.pocstor_41526_object5398/head//24 by osd.6.0:513 2014-08-12 14:07:27.446728 2014-08-13 18:06:06.663454 7f5c973a57a0 20 read_log 194209'15667 (194209'15626) delete e37e34cd/benchmark_data_storage0111-ib.pocstor_62621_object980/head//24 by osd.6.0:514 2014-08-12 14:07:27.450837 2014-08-13 18:06:06.663466 7f5c973a57a0 20 read_log 194209'15668 (194209'15628) delete 6f4454cd/benchmark_data_storage0111-ib.pocstor_62621_object21466/head//24 by osd.6.0:515 2014-08-12 14:07:27.451290 2014-08-13 18:06:06.663479 7f5c973a57a0 20 read_log 194209'15669 (194209'15629) delete 785f54cd/benchmark_data_storage0111-ib.pocstor_26659_object14064/head//24 by osd.6.0:516 2014-08-12 14:07:27.451814 2014-08-13 18:06:06.663503 7f5c973a57a0 20 read_log 194209'15670 (194209'15630) delete adc964cd/benchmark_data_storage0111-ib.pocstor_26659_object206/head//24 by osd.6.0:517 2014-08-12 14:07:27.454929 2014-08-13 18:06:06.663522 7f5c973a57a0 20 read_log 194209'15671 (194209'15631) delete 8e6b64cd/benchmark_data_storage0111-ib.pocstor_62621_object16300/head//24 by osd.6.0:518 2014-08-12 14:07:27.455234 2014-08-13 18:06:06.663535 7f5c973a57a0 20 read_log 194209'15672 (194209'15632) delete baa374cd/benchmark_data_storage0111-ib.pocstor_41526_object14731/head//24 by osd.6.0:519 2014-08-12 14:07:27.455512 2014-08-13 18:06:06.663560 7f5c973a57a0 20 read_log 194209'15673 (194209'15633) delete ebd574cd/benchmark_data_storage0111-ib.pocstor_62621_object29752/head//24 by osd.6.0:520 2014-08-12 14:07:27.458778 2014-08-13 18:06:06.663577 7f5c973a57a0 20 read_log 194209'15674 (194209'15634) delete 560694cd/benchmark_data_storage0111-ib.pocstor_62621_object9284/head//24 by osd.6.0:521 2014-08-12 14:07:27.459086 2014-08-13 18:06:06.663597 7f5c973a57a0 20 read_log 194209'15675 (194209'15635) delete 15f794cd/benchmark_data_storage0111-ib.pocstor_41526_object7984/head//24 by osd.6.0:522 2014-08-12 14:07:27.459396 2014-08-13 18:06:06.663612 7f5c973a57a0 20 read_log 194209'15676 (194209'15636) delete 875c94cd/benchmark_data_storage0111-ib.pocstor_62621_object16919/head//24 by osd.6.0:523 2014-08-12 14:07:27.462795 2014-08-13 18:06:06.663624 7f5c973a57a0 20 read_log 194209'15677 (194209'15637) delete 5e2da4cd/benchmark_data_storage0111-ib.pocstor_62621_object26823/head//24 by osd.6.0:524 2014-08-12 14:07:27.463119 2014-08-13 18:06:06.663637 7f5c973a57a0 20 read_log 194209'15678 (194209'15638) delete 3355b4cd/benchmark_data_storage0111-ib.pocstor_62621_object6108/head//24 by osd.6.0:525 2014-08-12 14:07:27.463407 2014-08-13 18:06:06.663649 7f5c973a57a0 20 read_log 194209'15679 (194209'15639) delete bac6c4cd/benchmark_data_storage0111-ib.pocstor_41526_object17871/head//24 by osd.6.0:526 2014-08-12 14:07:27.466570 2014-08-13 18:06:06.663661 7f5c973a57a0 20 read_log 194209'15680 (194209'15640) delete b71ac4cd/benchmark_data_storage0111-ib.pocstor_62621_object27259/head//24 by osd.6.0:527 2014-08-12 14:07:27.466868 2014-08-13 18:06:06.663674 7f5c973a57a0 20 read_log 194209'15681 (194209'15641) delete 2aafc4cd/benchmark_data_storage0111-ib.pocstor_41097_object18944/head//24 by osd.6.0:528 2014-08-12 14:07:27.467143 2014-08-13 18:06:06.663686 7f5c973a57a0 20 read_log 194209'15682 (194209'15643) delete 83f5d4cd/benchmark_data_storage0111-ib.pocstor_62621_object9467/head//24 by osd.6.0:529 2014-08-12 14:07:27.470476 2014-08-13 18:06:06.663698 7f5c973a57a0 20 read_log 194209'15683 (194209'15644) delete 6b0dd4cd/benchmark_data_storage0111-ib.pocstor_41526_object6003/head//24 by osd.6.0:530 2014-08-12 14:07:27.470793 2014-08-13 18:06:06.663719 7f5c973a57a0 20 read_log 194209'15684 (194209'15645) delete 2ca4e4cd/benchmark_data_storage0111-ib.pocstor_41526_object7337/head//24 by osd.6.0:531 2014-08-12 14:07:27.471407 2014-08-13 18:06:06.663735 7f5c973a57a0 20 read_log 194209'15685 (194209'15647) delete 8407f4cd/benchmark_data_storage0111-ib.pocstor_62621_object6570/head//24 by osd.6.0:532 2014-08-12 14:07:27.474590 2014-08-13 18:06:06.663747 7f5c973a57a0 20 read_log 194209'15686 (194209'15648) delete 3a0a08cd/benchmark_data_storage0111-ib.pocstor_41097_object26810/head//24 by osd.6.0:533 2014-08-12 14:07:27.474946 2014-08-13 18:06:06.663760 7f5c973a57a0 20 read_log 194209'15687 (194209'15649) delete 101b08cd/benchmark_data_storage0111-ib.pocstor_41097_object25632/head//24 by osd.6.0:534 2014-08-12 14:07:27.475489 2014-08-13 18:06:06.663772 7f5c973a57a0 20 read_log 194209'15688 (194209'15650) delete da7b08cd/benchmark_data_storage0111-ib.pocstor_62621_object8778/head//24 by osd.6.0:535 2014-08-12 14:07:27.478252 2014-08-13 18:06:06.663785 7f5c973a57a0 20 read_log 194209'15689 (194209'15659) delete 707e18cd/benchmark_data_storage0111-ib.pocstor_62621_object18884/head//24 by osd.6.0:536 2014-08-12 14:07:27.478729 2014-08-13 18:06:06.663797 7f5c973a57a0 20 read_log 194209'15690 (194209'15612) delete f9b490cd/benchmark_data_storage0111-ib.pocstor_2228_object12894/head//24 by osd.6.0:599 2014-08-12 14:10:13.376915 2014-08-13 18:06:06.663809 7f5c973a57a0 20 read_log 194209'15691 (194209'15614) delete 805a90cd/benchmark_data_storage0111-ib.pocstor_2228_object18525/head//24 by osd.6.0:600 2014-08-12 14:10:13.377288 2014-08-13 18:06:06.663821 7f5c973a57a0 20 read_log 194209'15692 (194209'15619) delete ec1914cd/benchmark_data_storage0111-ib.pocstor_2228_object23163/head//24 by osd.6.0:601 2014-08-12 14:10:13.377596 2014-08-13 18:06:06.663834 7f5c973a57a0 20 read_log 194209'15693 (194209'15627) delete 2454cd/benchmark_data_storage0111-ib.pocstor_2228_object8012/head//24 by osd.6.0:602 2014-08-12 14:10:13.377859 2014-08-13 18:06:06.663846 7f5c973a57a0 20 read_log 194209'15694 (194209'15642) delete 24d3d4cd/benchmark_data_storage0111-ib.pocstor_41097_object30639/head//24 by osd.6.0:603 2014-08-12 14:10:13.380553 2014-08-13 18:06:06.663869 7f5c973a57a0 20 read_log 194209'15695 (194209'15646) delete 6a95f4cd/benchmark_data_storage0111-ib.pocstor_2228_object23701/head//24 by osd.6.0:604 2014-08-12 14:10:13.380906 2014-08-13 18:06:06.663885 7f5c973a57a0 20 read_log 194209'15696 (194209'15651) delete d3d118cd/benchmark_data_storage0111-ib.pocstor_2228_object31123/head//24 by osd.6.0:605 2014-08-12 14:10:13.381699 2014-08-13 18:06:06.663897 7f5c973a57a0 20 read_log 194209'15697 (194209'15652) delete 430818cd/benchmark_data_storage0111-ib.pocstor_2228_object19002/head//24 by osd.6.0:606 2014-08-12 14:10:13.381995 2014-08-13 18:06:06.663910 7f5c973a57a0 20 read_log 194215'15698 (194215'15698) modify cd/hit_set_24.cd_archive_2014-08-12 15:32:25.874029_2014-08-12 15:39:20.336162/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:20.336405 2014-08-13 18:06:06.663922 7f5c973a57a0 20 read_log 194215'15699 (193305'15492) delete cd/hit_set_24.cd_archive_2014-08-11 18:37:04.943507_2014-08-11 18:42:36.902876/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:20.336405 2014-08-13 18:06:06.663935 7f5c973a57a0 20 read_log 194215'15700 (193305'15598) clean e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by osd.6.0:815 2014-08-12 15:39:20.340162 2014-08-13 18:06:06.663947 7f5c973a57a0 20 read_log 194215'15701 (194215'15700) delete e4d800cd/benchmark_data_storage0111-ib.pocstor_62621_object23104/head//24 by client.2174595.0:41908 0.000000 2014-08-13 18:06:06.663958 7f5c973a57a0 20 read_log 194215'15702 (192888'15475) clean e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by osd.6.0:817 2014-08-12 15:39:20.347804 2014-08-13 18:06:06.663973 7f5c973a57a0 20 read_log 194215'15703 (194215'15702) delete e89c00cd/benchmark_data_storage0111-ib.pocstor_2228_object21979/head//24 by client.2174595.0:41910 0.000000 2014-08-13 18:06:06.663984 7f5c973a57a0 20 read_log 194215'15704 (192834'14636) clean 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by osd.6.0:819 2014-08-12 15:39:20.354460 2014-08-13 18:06:06.663997 7f5c973a57a0 20 read_log 194215'15705 (194215'15704) delete 7ec010cd/benchmark_data_storage0111-ib.pocstor_26659_object20353/head//24 by client.2174595.0:41912 0.000000 2014-08-13 18:06:06.664010 7f5c973a57a0 20 read_log 194215'15706 (193305'15605) clean 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by osd.6.0:821 2014-08-12 15:39:20.360528 2014-08-13 18:06:06.664023 7f5c973a57a0 20 read_log 194215'15707 (194215'15706) delete 2eaf10cd/benchmark_data_storage0111-ib.pocstor_62621_object27645/head//24 by client.2174595.0:41914 0.000000 2014-08-13 18:06:06.664036 7f5c973a57a0 20 read_log 194215'15708 (192888'15318) clean 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by osd.6.0:823 2014-08-12 15:39:20.367308 2014-08-13 18:06:06.664050 7f5c973a57a0 20 read_log 194215'15709 (194215'15708) delete 44d930cd/benchmark_data_storage0111-ib.pocstor_41097_object29523/head//24 by client.2174595.0:41916 0.000000 2014-08-13 18:06:06.664061 7f5c973a57a0 20 read_log 194215'15710 (192888'15272) clean dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by osd.6.0:825 2014-08-12 15:39:20.373354 2014-08-13 18:06:06.664076 7f5c973a57a0 20 read_log 194215'15711 (194215'15710) delete dded30cd/benchmark_data_storage0111-ib.pocstor_41097_object2586/head//24 by client.2174595.0:41918 0.000000 2014-08-13 18:06:06.664087 7f5c973a57a0 20 read_log 194215'15712 (192888'15275) clean a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by osd.6.0:827 2014-08-12 15:39:20.379222 2014-08-13 18:06:06.664101 7f5c973a57a0 20 read_log 194215'15713 (194215'15712) delete a1ce30cd/benchmark_data_storage0111-ib.pocstor_41097_object4539/head//24 by client.2174595.0:41920 0.000000 2014-08-13 18:06:06.664119 7f5c973a57a0 20 read_log 194215'15714 (192888'15276) clean d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by osd.6.0:829 2014-08-12 15:39:20.385350 2014-08-13 18:06:06.664134 7f5c973a57a0 20 read_log 194215'15715 (194215'15714) delete d69240cd/benchmark_data_storage0111-ib.pocstor_41097_object5111/head//24 by client.2174595.0:41922 0.000000 2014-08-13 18:06:06.664150 7f5c973a57a0 20 read_log 194215'15716 (192834'14546) clean d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by osd.6.0:831 2014-08-12 15:39:20.391286 2014-08-13 18:06:06.664165 7f5c973a57a0 20 read_log 194215'15717 (194215'15716) delete d01540cd/benchmark_data_storage0111-ib.pocstor_26659_object8907/head//24 by client.2174595.0:41924 0.000000 2014-08-13 18:06:06.664176 7f5c973a57a0 20 read_log 194215'15718 (192888'15462) clean da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by osd.6.0:833 2014-08-12 15:39:20.397438 2014-08-13 18:06:06.664189 7f5c973a57a0 20 read_log 194215'15719 (194215'15718) delete da4940cd/benchmark_data_storage0111-ib.pocstor_2228_object13143/head//24 by client.2174595.0:41926 0.000000 2014-08-13 18:06:06.664200 7f5c973a57a0 20 read_log 194215'15720 (192888'15307) clean e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by osd.6.0:835 2014-08-12 15:39:20.405318 2014-08-13 18:06:06.664212 7f5c973a57a0 20 read_log 194215'15721 (194215'15720) delete e62b40cd/benchmark_data_storage0111-ib.pocstor_41097_object21868/head//24 by client.2174595.0:41928 0.000000 2014-08-13 18:06:06.664223 7f5c973a57a0 20 read_log 194215'15722 (192834'14557) clean 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by osd.6.0:837 2014-08-12 15:39:20.411992 2014-08-13 18:06:06.664238 7f5c973a57a0 20 read_log 194215'15723 (194215'15722) delete 387350cd/benchmark_data_storage0111-ib.pocstor_26659_object10325/head//24 by client.2174595.0:41930 0.000000 2014-08-13 18:06:06.664249 7f5c973a57a0 20 read_log 194215'15724 (192834'15084) clean d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by osd.6.0:839 2014-08-12 15:39:20.417777 2014-08-13 18:06:06.664262 7f5c973a57a0 20 read_log 194215'15725 (194215'15724) delete d4fa50cd/benchmark_data_storage0111-ib.pocstor_41526_object13549/head//24 by client.2174595.0:41932 0.000000 2014-08-13 18:06:06.664280 7f5c973a57a0 20 read_log 194215'15726 (192834'15030) clean 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by osd.6.0:841 2014-08-12 15:39:20.423798 2014-08-13 18:06:06.664292 7f5c973a57a0 20 read_log 194215'15727 (194215'15726) delete 72d460cd/benchmark_data_storage0111-ib.pocstor_41526_object4949/head//24 by client.2174595.0:41934 0.000000 2014-08-13 18:06:06.664304 7f5c973a57a0 20 read_log 194215'15728 (192834'14534) clean 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by osd.6.0:843 2014-08-12 15:39:20.430699 2014-08-13 18:06:06.664316 7f5c973a57a0 20 read_log 194215'15729 (194215'15728) delete 72f170cd/benchmark_data_storage0111-ib.pocstor_26659_object7534/head//24 by client.2174595.0:41936 0.000000 2014-08-13 18:06:06.664327 7f5c973a57a0 20 read_log 194215'15730 (192834'14651) clean 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by osd.6.0:845 2014-08-12 15:39:20.436890 2014-08-13 18:06:06.664339 7f5c973a57a0 20 read_log 194215'15731 (194215'15730) delete 62ea70cd/benchmark_data_storage0111-ib.pocstor_26659_object21758/head//24 by client.2174595.0:41938 0.000000 2014-08-13 18:06:06.664350 7f5c973a57a0 20 read_log 194215'15732 (193305'15586) clean 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by osd.6.0:847 2014-08-12 15:39:20.443093 2014-08-13 18:06:06.664363 7f5c973a57a0 20 read_log 194215'15733 (194215'15732) delete 4b0d70cd/benchmark_data_storage0111-ib.pocstor_62621_object12017/head//24 by client.2174595.0:41940 0.000000 2014-08-13 18:06:06.664381 7f5c973a57a0 20 read_log 194215'15734 (193305'15590) clean db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by osd.6.0:849 2014-08-12 15:39:20.450186 2014-08-13 18:06:06.664403 7f5c973a57a0 20 read_log 194215'15735 (194215'15734) delete db9e70cd/benchmark_data_storage0111-ib.pocstor_62621_object15009/head//24 by client.2174595.0:41942 0.000000 2014-08-13 18:06:06.664417 7f5c973a57a0 20 read_log 194215'15736 (193305'15579) clean 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by osd.6.0:851 2014-08-12 15:39:20.456641 2014-08-13 18:06:06.664430 7f5c973a57a0 20 read_log 194215'15737 (194215'15736) delete 152080cd/benchmark_data_storage0111-ib.pocstor_62621_object8174/head//24 by client.2174595.0:41944 0.000000 2014-08-13 18:06:06.664457 7f5c973a57a0 20 read_log 194215'15738 (192888'15289) clean 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by osd.6.0:853 2014-08-12 15:39:20.462259 2014-08-13 18:06:06.664471 7f5c973a57a0 20 read_log 194215'15739 (194215'15738) delete 18c580cd/benchmark_data_storage0111-ib.pocstor_41097_object15155/head//24 by client.2174595.0:41946 0.000000 2014-08-13 18:06:06.664485 7f5c973a57a0 20 read_log 194215'15740 (194209'15653) delete 131818cd/benchmark_data_storage0111-ib.pocstor_2228_object10833/head//24 by client.2174595.0:41948 0.000000 2014-08-13 18:06:06.664501 7f5c973a57a0 20 read_log 194215'15741 (192834'15055) clean e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by osd.6.0:856 2014-08-12 15:39:20.472296 2014-08-13 18:06:06.664516 7f5c973a57a0 20 read_log 194215'15742 (194215'15741) delete e33328cd/benchmark_data_storage0111-ib.pocstor_41526_object8041/head//24 by client.2174595.0:41950 0.000000 2014-08-13 18:06:06.664527 7f5c973a57a0 20 read_log 194215'15743 (193305'15600) clean 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by osd.6.0:858 2014-08-12 15:39:20.478681 2014-08-13 18:06:06.664552 7f5c973a57a0 20 read_log 194215'15744 (194215'15743) delete 935628cd/benchmark_data_storage0111-ib.pocstor_62621_object26556/head//24 by client.2174595.0:41952 0.000000 2014-08-13 18:06:06.664567 7f5c973a57a0 20 read_log 194215'15745 (192888'15315) clean 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by osd.6.0:860 2014-08-12 15:39:20.485451 2014-08-13 18:06:06.664580 7f5c973a57a0 20 read_log 194215'15746 (194215'15745) delete 405938cd/benchmark_data_storage0111-ib.pocstor_41097_object27884/head//24 by client.2174595.0:41954 0.000000 2014-08-13 18:06:06.664591 7f5c973a57a0 20 read_log 194215'15747 (192888'15442) clean 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by osd.6.0:862 2014-08-12 15:39:20.492013 2014-08-13 18:06:06.664603 7f5c973a57a0 20 read_log 194215'15748 (194215'15747) delete 5a4a38cd/benchmark_data_storage0111-ib.pocstor_2228_object4694/head//24 by client.2174595.0:41956 0.000000 2014-08-13 18:06:06.664614 7f5c973a57a0 20 read_log 194215'15749 (193305'15583) clean a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by osd.6.0:864 2014-08-12 15:39:20.498474 2014-08-13 18:06:06.664626 7f5c973a57a0 20 read_log 194215'15750 (194215'15749) delete a2ad48cd/benchmark_data_storage0111-ib.pocstor_62621_object10621/head//24 by client.2174595.0:41958 0.000000 2014-08-13 18:06:06.664642 7f5c973a57a0 20 read_log 194215'15751 (192888'15319) clean 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by osd.6.0:866 2014-08-12 15:39:20.505107 2014-08-13 18:06:06.664654 7f5c973a57a0 20 read_log 194215'15752 (194215'15751) delete 8bde48cd/benchmark_data_storage0111-ib.pocstor_41097_object29582/head//24 by client.2174595.0:41960 0.000000 2014-08-13 18:06:06.664677 7f5c973a57a0 20 read_log 194215'15753 (192888'15478) clean ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by osd.6.0:868 2014-08-12 15:39:20.511583 2014-08-13 18:06:06.664711 7f5c973a57a0 20 read_log 194215'15754 (194215'15753) delete ce0258cd/benchmark_data_storage0111-ib.pocstor_2228_object22824/head//24 by client.2174595.0:41962 0.000000 2014-08-13 18:06:06.664726 7f5c973a57a0 20 read_log 194215'15755 (192888'15454) clean 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by osd.6.0:870 2014-08-12 15:39:20.518087 2014-08-13 18:06:06.664742 7f5c973a57a0 20 read_log 194215'15756 (194215'15755) delete 860558cd/benchmark_data_storage0111-ib.pocstor_2228_object9626/head//24 by client.2174595.0:41964 0.000000 2014-08-13 18:06:06.664774 7f5c973a57a0 20 read_log 194215'15757 (192888'15463) clean 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by osd.6.0:872 2014-08-12 15:39:20.524304 2014-08-13 18:06:06.664792 7f5c973a57a0 20 read_log 194215'15758 (194215'15757) delete 461c58cd/benchmark_data_storage0111-ib.pocstor_2228_object14976/head//24 by client.2174595.0:41966 0.000000 2014-08-13 18:06:06.664813 7f5c973a57a0 20 read_log 194215'15759 (192834'15116) clean e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by osd.6.0:874 2014-08-12 15:39:20.530543 2014-08-13 18:06:06.664827 7f5c973a57a0 20 read_log 194215'15760 (194215'15759) delete e75368cd/benchmark_data_storage0111-ib.pocstor_41526_object16984/head//24 by client.2174595.0:41968 0.000000 2014-08-13 18:06:06.664838 7f5c973a57a0 20 read_log 194215'15761 (193305'15603) clean 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by osd.6.0:876 2014-08-12 15:39:20.537407 2014-08-13 18:06:06.664852 7f5c973a57a0 20 read_log 194215'15762 (194215'15761) delete 7ff368cd/benchmark_data_storage0111-ib.pocstor_62621_object27217/head//24 by client.2174595.0:41970 0.000000 2014-08-13 18:06:06.664863 7f5c973a57a0 20 read_log 194215'15763 (192834'14480) clean fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by osd.6.0:878 2014-08-12 15:39:20.544151 2014-08-13 18:06:06.664875 7f5c973a57a0 20 read_log 194215'15764 (194215'15763) delete fcc868cd/benchmark_data_storage0111-ib.pocstor_26659_object1304/head//24 by client.2174595.0:41972 0.000000 2014-08-13 18:06:06.664886 7f5c973a57a0 20 read_log 194215'15765 (192834'15090) clean d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by osd.6.0:880 2014-08-12 15:39:20.550638 2014-08-13 18:06:06.664909 7f5c973a57a0 20 read_log 194215'15766 (194215'15765) delete d43288cd/benchmark_data_storage0111-ib.pocstor_41526_object14117/head//24 by client.2174595.0:41974 0.000000 2014-08-13 18:06:06.664920 7f5c973a57a0 20 read_log 194215'15767 (192834'14641) clean e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by osd.6.0:882 2014-08-12 15:39:20.556965 2014-08-13 18:06:06.664933 7f5c973a57a0 20 read_log 194215'15768 (194215'15767) delete e77288cd/benchmark_data_storage0111-ib.pocstor_26659_object20964/head//24 by client.2174595.0:41976 0.000000 2014-08-13 18:06:06.664944 7f5c973a57a0 20 read_log 194215'15769 (192888'15465) clean 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by osd.6.0:884 2014-08-12 15:39:20.563203 2014-08-13 18:06:06.664956 7f5c973a57a0 20 read_log 194215'15770 (194215'15769) delete 3f3688cd/benchmark_data_storage0111-ib.pocstor_2228_object17527/head//24 by client.2174595.0:41978 0.000000 2014-08-13 18:06:06.664967 7f5c973a57a0 20 read_log 194215'15771 (193305'15573) clean e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by osd.6.0:886 2014-08-12 15:39:20.569936 2014-08-13 18:06:06.664979 7f5c973a57a0 20 read_log 194215'15772 (194215'15771) delete e04a88cd/benchmark_data_storage0111-ib.pocstor_62621_object982/head//24 by client.2174595.0:41980 0.000000 2014-08-13 18:06:06.664990 7f5c973a57a0 20 read_log 194215'15773 (192888'15314) clean 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by osd.6.0:888 2014-08-12 15:39:20.576288 2014-08-13 18:06:06.665010 7f5c973a57a0 20 read_log 194215'15774 (194215'15773) delete 3e3198cd/benchmark_data_storage0111-ib.pocstor_41097_object27572/head//24 by client.2174595.0:41982 0.000000 2014-08-13 18:06:06.665021 7f5c973a57a0 20 read_log 194215'15775 (192834'15064) clean b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by osd.6.0:890 2014-08-12 15:39:20.582848 2014-08-13 18:06:06.665033 7f5c973a57a0 20 read_log 194215'15776 (194215'15775) delete b01d98cd/benchmark_data_storage0111-ib.pocstor_41526_object9430/head//24 by client.2174595.0:41984 0.000000 2014-08-13 18:06:06.665044 7f5c973a57a0 20 read_log 194215'15777 (192834'15139) clean 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by osd.6.0:892 2014-08-12 15:39:20.589326 2014-08-13 18:06:06.665056 7f5c973a57a0 20 read_log 194215'15778 (194215'15777) delete 4903a8cd/benchmark_data_storage0111-ib.pocstor_41526_object19562/head//24 by client.2174595.0:41986 0.000000 2014-08-13 18:06:06.665067 7f5c973a57a0 20 read_log 194215'15779 (192888'15476) clean d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by osd.6.0:894 2014-08-12 15:39:20.595548 2014-08-13 18:06:06.665079 7f5c973a57a0 20 read_log 194215'15780 (194215'15779) delete d4f4a8cd/benchmark_data_storage0111-ib.pocstor_2228_object22122/head//24 by client.2174595.0:41988 0.000000 2014-08-13 18:06:06.665090 7f5c973a57a0 20 read_log 194215'15781 (192834'15043) clean 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by osd.6.0:896 2014-08-12 15:39:20.602638 2014-08-13 18:06:06.665102 7f5c973a57a0 20 read_log 194215'15782 (194215'15781) delete 960aa8cd/benchmark_data_storage0111-ib.pocstor_41526_object6101/head//24 by client.2174595.0:41990 0.000000 2014-08-13 18:06:06.665113 7f5c973a57a0 20 read_log 194215'15783 (193305'15574) clean 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by osd.6.0:898 2014-08-12 15:39:20.609498 2014-08-13 18:06:06.665125 7f5c973a57a0 20 read_log 194215'15784 (194215'15783) delete 7581b8cd/benchmark_data_storage0111-ib.pocstor_62621_object1912/head//24 by client.2174595.0:41992 0.000000 2014-08-13 18:06:06.665135 7f5c973a57a0 20 read_log 194215'15785 (193305'15599) clean 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by osd.6.0:900 2014-08-12 15:39:20.615731 2014-08-13 18:06:06.665156 7f5c973a57a0 20 read_log 194215'15786 (194215'15785) delete 2167b8cd/benchmark_data_storage0111-ib.pocstor_62621_object25588/head//24 by client.2174595.0:41994 0.000000 2014-08-13 18:06:06.665168 7f5c973a57a0 20 read_log 194215'15787 (193305'15594) clean 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by osd.6.0:902 2014-08-12 15:39:20.621844 2014-08-13 18:06:06.665181 7f5c973a57a0 20 read_log 194215'15788 (194215'15787) delete 957cb8cd/benchmark_data_storage0111-ib.pocstor_62621_object17683/head//24 by client.2174595.0:41996 0.000000 2014-08-13 18:06:06.665192 7f5c973a57a0 20 read_log 194215'15789 (192834'15026) clean daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by osd.6.0:904 2014-08-12 15:39:20.628147 2014-08-13 18:06:06.665204 7f5c973a57a0 20 read_log 194215'15790 (194215'15789) delete daecc8cd/benchmark_data_storage0111-ib.pocstor_41526_object4695/head//24 by client.2174595.0:41998 0.000000 2014-08-13 18:06:06.665215 7f5c973a57a0 20 read_log 194215'15791 (192888'15445) clean 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by osd.6.0:906 2014-08-12 15:39:20.635094 2014-08-13 18:06:06.665227 7f5c973a57a0 20 read_log 194215'15792 (194215'15791) delete 1bedc8cd/benchmark_data_storage0111-ib.pocstor_2228_object6916/head//24 by client.2174595.0:42000 0.000000 2014-08-13 18:06:06.665238 7f5c973a57a0 20 read_log 194215'15793 (192834'15074) clean bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by osd.6.0:908 2014-08-12 15:39:20.641975 2014-08-13 18:06:06.665276 7f5c973a57a0 20 read_log 194215'15794 (194215'15793) delete bcd0d8cd/benchmark_data_storage0111-ib.pocstor_41526_object11220/head//24 by client.2174595.0:42002 0.000000 2014-08-13 18:06:06.665291 7f5c973a57a0 20 read_log 194215'15795 (192834'14597) clean 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by osd.6.0:910 2014-08-12 15:39:20.648350 2014-08-13 18:06:06.665307 7f5c973a57a0 20 read_log 194215'15796 (194215'15795) delete 43f0d8cd/benchmark_data_storage0111-ib.pocstor_26659_object15095/head//24 by client.2174595.0:42004 0.000000 2014-08-13 18:06:06.665319 7f5c973a57a0 20 read_log 194215'15797 (192888'15451) clean 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by osd.6.0:912 2014-08-12 15:39:20.654642 2014-08-13 18:06:06.665334 7f5c973a57a0 20 read_log 194215'15798 (194215'15797) delete 76f2d8cd/benchmark_data_storage0111-ib.pocstor_2228_object9093/head//24 by client.2174595.0:42006 0.000000 2014-08-13 18:06:06.665346 7f5c973a57a0 20 read_log 194215'15799 (192834'14995) clean 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by osd.6.0:914 2014-08-12 15:39:20.661160 2014-08-13 18:06:06.665361 7f5c973a57a0 20 read_log 194215'15800 (194215'15799) delete 1785d8cd/benchmark_data_storage0111-ib.pocstor_41526_object552/head//24 by client.2174595.0:42008 0.000000 2014-08-13 18:06:06.665372 7f5c973a57a0 20 read_log 194215'15801 (193305'15588) clean da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by osd.6.0:916 2014-08-12 15:39:20.667342 2014-08-13 18:06:06.665385 7f5c973a57a0 20 read_log 194215'15802 (194215'15801) delete da1ad8cd/benchmark_data_storage0111-ib.pocstor_62621_object13933/head//24 by client.2174595.0:42010 0.000000 2014-08-13 18:06:06.665405 7f5c973a57a0 20 read_log 194215'15803 (192834'15108) clean 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by osd.6.0:918 2014-08-12 15:39:20.674483 2014-08-13 18:06:06.665418 7f5c973a57a0 20 read_log 194215'15804 (194215'15803) delete 669ad8cd/benchmark_data_storage0111-ib.pocstor_41526_object16405/head//24 by client.2174595.0:42012 0.000000 2014-08-13 18:06:06.665429 7f5c973a57a0 20 read_log 194215'15805 (192834'14522) clean 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by osd.6.0:920 2014-08-12 15:39:20.681126 2014-08-13 18:06:06.665449 7f5c973a57a0 20 read_log 194215'15806 (194215'15805) delete 40aae8cd/benchmark_data_storage0111-ib.pocstor_26659_object6164/head//24 by client.2174595.0:42014 0.000000 2014-08-13 18:06:06.665463 7f5c973a57a0 20 read_log 194215'15807 (193305'15593) clean 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by osd.6.0:922 2014-08-12 15:39:20.687532 2014-08-13 18:06:06.665476 7f5c973a57a0 20 read_log 194215'15808 (194215'15807) delete 9d0ce8cd/benchmark_data_storage0111-ib.pocstor_62621_object17021/head//24 by client.2174595.0:42016 0.000000 2014-08-13 18:06:06.665494 7f5c973a57a0 20 read_log 194215'15809 (192834'15024) clean fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by osd.6.0:924 2014-08-12 15:39:20.694381 2014-08-13 18:06:06.665510 7f5c973a57a0 20 read_log 194215'15810 (194215'15809) delete fd1de8cd/benchmark_data_storage0111-ib.pocstor_41526_object4377/head//24 by client.2174595.0:42018 0.000000 2014-08-13 18:06:06.665521 7f5c973a57a0 20 read_log 194215'15811 (192888'15458) clean 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by osd.6.0:926 2014-08-12 15:39:20.701257 2014-08-13 18:06:06.665534 7f5c973a57a0 20 read_log 194215'15812 (194215'15811) delete 12c1f8cd/benchmark_data_storage0111-ib.pocstor_2228_object11892/head//24 by client.2174595.0:42020 0.000000 2014-08-13 18:06:06.665545 7f5c973a57a0 20 read_log 194215'15813 (192834'15033) clean f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by osd.6.0:928 2014-08-12 15:39:20.708080 2014-08-13 18:06:06.665578 7f5c973a57a0 20 read_log 194215'15814 (194215'15813) delete f907f8cd/benchmark_data_storage0111-ib.pocstor_41526_object5324/head//24 by client.2174595.0:42022 0.000000 2014-08-13 18:06:06.665593 7f5c973a57a0 20 read_log 194215'15815 (192834'15021) clean 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by osd.6.0:930 2014-08-12 15:39:20.715286 2014-08-13 18:06:06.665606 7f5c973a57a0 20 read_log 194215'15816 (194215'15815) delete 21e9f8cd/benchmark_data_storage0111-ib.pocstor_41526_object3966/head//24 by client.2174595.0:42024 0.000000 2014-08-13 18:06:06.665617 7f5c973a57a0 20 read_log 194215'15817 (192888'15486) clean 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by osd.6.0:932 2014-08-12 15:39:20.721851 2014-08-13 18:06:06.665632 7f5c973a57a0 20 read_log 194215'15818 (194215'15817) delete 388ff8cd/benchmark_data_storage0111-ib.pocstor_2228_object26631/head//24 by client.2174595.0:42026 0.000000 2014-08-13 18:06:06.665643 7f5c973a57a0 20 read_log 194215'15819 (192888'15441) clean 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by osd.6.0:934 2014-08-12 15:39:20.728952 2014-08-13 18:06:06.665655 7f5c973a57a0 20 read_log 194215'15820 (194215'15819) delete 87970ccd/benchmark_data_storage0111-ib.pocstor_2228_object4291/head//24 by client.2174595.0:42028 0.000000 2014-08-13 18:06:06.665666 7f5c973a57a0 20 read_log 194215'15821 (192888'15269) clean 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by osd.6.0:936 2014-08-12 15:39:20.735612 2014-08-13 18:06:06.665687 7f5c973a57a0 20 read_log 194215'15822 (194215'15821) delete 9bba0ccd/benchmark_data_storage0111-ib.pocstor_41097_object709/head//24 by client.2174595.0:42030 0.000000 2014-08-13 18:06:06.665698 7f5c973a57a0 20 read_log 194215'15823 (192888'15436) clean d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by osd.6.0:938 2014-08-12 15:39:20.742274 2014-08-13 18:06:06.665711 7f5c973a57a0 20 read_log 194215'15824 (194215'15823) delete d5c91ccd/benchmark_data_storage0111-ib.pocstor_2228_object446/head//24 by client.2174595.0:42032 0.000000 2014-08-13 18:06:06.665722 7f5c973a57a0 20 read_log 194215'15825 (192888'15316) clean d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by osd.6.0:940 2014-08-12 15:39:20.749539 2014-08-13 18:06:06.665735 7f5c973a57a0 20 read_log 194215'15826 (194215'15825) delete d25f1ccd/benchmark_data_storage0111-ib.pocstor_41097_object28560/head//24 by client.2174595.0:42034 0.000000 2014-08-13 18:06:06.665746 7f5c973a57a0 20 read_log 194215'15827 (192834'15062) clean 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by osd.6.0:942 2014-08-12 15:39:20.756527 2014-08-13 18:06:06.665758 7f5c973a57a0 20 read_log 194215'15828 (194215'15827) delete 87b22ccd/benchmark_data_storage0111-ib.pocstor_41526_object9301/head//24 by client.2174595.0:42036 0.000000 2014-08-13 18:06:06.665769 7f5c973a57a0 20 read_log 194215'15829 (192834'14614) clean f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by osd.6.0:944 2014-08-12 15:39:20.763069 2014-08-13 18:06:06.665781 7f5c973a57a0 20 read_log 194215'15830 (194215'15829) delete f0362ccd/benchmark_data_storage0111-ib.pocstor_26659_object17106/head//24 by client.2174595.0:42038 0.000000 2014-08-13 18:06:06.665794 7f5c973a57a0 20 read_log 194215'15831 (192888'15305) clean 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by osd.6.0:946 2014-08-12 15:39:20.769957 2014-08-13 18:06:06.665807 7f5c973a57a0 20 read_log 194215'15832 (194215'15831) delete 5f4e2ccd/benchmark_data_storage0111-ib.pocstor_41097_object21695/head//24 by client.2174595.0:42040 0.000000 2014-08-13 18:06:06.665818 7f5c973a57a0 20 read_log 194215'15833 (193305'15609) clean 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by osd.6.0:948 2014-08-12 15:39:20.777056 2014-08-13 18:06:06.665840 7f5c973a57a0 20 read_log 194215'15834 (194215'15833) delete 9e7f2ccd/benchmark_data_storage0111-ib.pocstor_62621_object30597/head//24 by client.2174595.0:42042 0.000000 2014-08-13 18:06:06.665855 7f5c973a57a0 20 read_log 194215'15835 (192834'15099) clean ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by osd.6.0:950 2014-08-12 15:39:20.784025 2014-08-13 18:06:06.665868 7f5c973a57a0 20 read_log 194215'15836 (194215'15835) delete ae553ccd/benchmark_data_storage0111-ib.pocstor_41526_object15355/head//24 by client.2174595.0:42044 0.000000 2014-08-13 18:06:06.665878 7f5c973a57a0 20 read_log 194215'15837 (192888'15270) clean a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by osd.6.0:952 2014-08-12 15:39:20.806152 2014-08-13 18:06:06.665891 7f5c973a57a0 20 read_log 194215'15838 (194215'15837) delete a3693ccd/benchmark_data_storage0111-ib.pocstor_41097_object1982/head//24 by client.2174595.0:42046 0.000000 2014-08-13 18:06:06.665901 7f5c973a57a0 20 read_log 194215'15839 (192888'15294) clean f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by osd.6.0:954 2014-08-12 15:39:20.813777 2014-08-13 18:06:06.665914 7f5c973a57a0 20 read_log 194215'15840 (194215'15839) delete f7494ccd/benchmark_data_storage0111-ib.pocstor_41097_object16658/head//24 by client.2174595.0:42048 0.000000 2014-08-13 18:06:06.665935 7f5c973a57a0 20 read_log 194215'15841 (192834'15059) clean 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by osd.6.0:956 2014-08-12 15:39:20.820536 2014-08-13 18:06:06.665948 7f5c973a57a0 20 read_log 194215'15842 (194215'15841) delete 4aff4ccd/benchmark_data_storage0111-ib.pocstor_41526_object8773/head//24 by client.2174595.0:42050 0.000000 2014-08-13 18:06:06.665959 7f5c973a57a0 20 read_log 194215'15843 (192888'15490) clean 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by osd.6.0:958 2014-08-12 15:39:20.827925 2014-08-13 18:06:06.665971 7f5c973a57a0 20 read_log 194215'15844 (194215'15843) delete 8a305ccd/benchmark_data_storage0111-ib.pocstor_2228_object30294/head//24 by client.2174595.0:42052 0.000000 2014-08-13 18:06:06.665982 7f5c973a57a0 20 read_log 194215'15845 (192834'15068) clean 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by osd.6.0:960 2014-08-12 15:39:20.834851 2014-08-13 18:06:06.665994 7f5c973a57a0 20 read_log 194215'15846 (194215'15845) delete 34315ccd/benchmark_data_storage0111-ib.pocstor_41526_object10083/head//24 by client.2174595.0:42054 0.000000 2014-08-13 18:06:06.666005 7f5c973a57a0 20 read_log 194215'15847 (192888'15464) clean 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by osd.6.0:962 2014-08-12 15:39:20.841695 2014-08-13 18:06:06.666017 7f5c973a57a0 20 read_log 194215'15848 (194215'15847) delete 32d65ccd/benchmark_data_storage0111-ib.pocstor_2228_object15261/head//24 by client.2174595.0:42056 0.000000 2014-08-13 18:06:06.666031 7f5c973a57a0 20 read_log 194215'15849 (192834'14618) clean 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by osd.6.0:964 2014-08-12 15:39:20.848365 2014-08-13 18:06:06.666045 7f5c973a57a0 20 read_log 194215'15850 (194215'15849) delete 10885ccd/benchmark_data_storage0111-ib.pocstor_26659_object17505/head//24 by client.2174595.0:42058 0.000000 2014-08-13 18:06:06.666056 7f5c973a57a0 20 read_log 194215'15851 (192888'15469) clean 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by osd.6.0:966 2014-08-12 15:39:20.854815 2014-08-13 18:06:06.666094 7f5c973a57a0 20 read_log 194215'15852 (194215'15851) delete 8e7b5ccd/benchmark_data_storage0111-ib.pocstor_2228_object19823/head//24 by client.2174595.0:42060 0.000000 2014-08-13 18:06:06.666110 7f5c973a57a0 20 read_log 194215'15853 (192888'15457) clean 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by osd.6.0:968 2014-08-12 15:39:20.861458 2014-08-13 18:06:06.666134 7f5c973a57a0 20 read_log 194215'15854 (194215'15853) delete 5b6f5ccd/benchmark_data_storage0111-ib.pocstor_2228_object11682/head//24 by client.2174595.0:42062 0.000000 2014-08-13 18:06:06.666166 7f5c973a57a0 20 read_log 194215'15855 (193305'15587) clean 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by osd.6.0:970 2014-08-12 15:39:20.868723 2014-08-13 18:06:06.666183 7f5c973a57a0 20 read_log 194215'15856 (194215'15855) delete 12a6ccd/benchmark_data_storage0111-ib.pocstor_62621_object12858/head//24 by client.2174595.0:42064 0.000000 2014-08-13 18:06:06.666199 7f5c973a57a0 20 read_log 194215'15857 (192888'15487) clean dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by osd.6.0:972 2014-08-12 15:39:20.875611 2014-08-13 18:06:06.666215 7f5c973a57a0 20 read_log 194215'15858 (194215'15857) delete dbcd6ccd/benchmark_data_storage0111-ib.pocstor_2228_object27384/head//24 by client.2174595.0:42066 0.000000 2014-08-13 18:06:06.666227 7f5c973a57a0 20 read_log 194215'15859 (192888'15317) clean ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by osd.6.0:974 2014-08-12 15:39:20.882409 2014-08-13 18:06:06.666267 7f5c973a57a0 20 read_log 194215'15860 (194215'15859) delete ad557ccd/benchmark_data_storage0111-ib.pocstor_41097_object29499/head//24 by client.2174595.0:42068 0.000000 2014-08-13 18:06:06.666282 7f5c973a57a0 20 read_log 194215'15861 (192888'15301) clean a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by osd.6.0:976 2014-08-12 15:39:20.888805 2014-08-13 18:06:06.666312 7f5c973a57a0 20 read_log 194215'15862 (194215'15861) delete a34d7ccd/benchmark_data_storage0111-ib.pocstor_41097_object19845/head//24 by client.2174595.0:42070 0.000000 2014-08-13 18:06:06.666327 7f5c973a57a0 20 read_log 194215'15863 (193305'15576) clean f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by osd.6.0:978 2014-08-12 15:39:20.895309 2014-08-13 18:06:06.666343 7f5c973a57a0 20 read_log 194215'15864 (194215'15863) delete f8728ccd/benchmark_data_storage0111-ib.pocstor_62621_object2975/head//24 by client.2174595.0:42072 0.000000 2014-08-13 18:06:06.666367 7f5c973a57a0 20 read_log 194215'15865 (192888'15453) clean bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by osd.6.0:980 2014-08-12 15:39:20.902144 2014-08-13 18:06:06.666384 7f5c973a57a0 20 read_log 194215'15866 (194215'15865) delete bb809ccd/benchmark_data_storage0111-ib.pocstor_2228_object9548/head//24 by client.2174595.0:42074 0.000000 2014-08-13 18:06:06.666396 7f5c973a57a0 20 read_log 194215'15867 (193305'15584) clean dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by osd.6.0:982 2014-08-12 15:39:20.908753 2014-08-13 18:06:06.666408 7f5c973a57a0 20 read_log 194215'15868 (194215'15867) delete dd199ccd/benchmark_data_storage0111-ib.pocstor_62621_object11019/head//24 by client.2174595.0:42076 0.000000 2014-08-13 18:06:06.666419 7f5c973a57a0 20 read_log 194215'15869 (192834'15011) clean 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by osd.6.0:984 2014-08-12 15:39:20.915616 2014-08-13 18:06:06.666431 7f5c973a57a0 20 read_log 194215'15870 (194215'15869) delete 5814accd/benchmark_data_storage0111-ib.pocstor_41526_object2029/head//24 by client.2174595.0:42078 0.000000 2014-08-13 18:06:06.666442 7f5c973a57a0 20 read_log 194215'15871 (192888'15485) clean 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by osd.6.0:986 2014-08-12 15:39:20.922440 2014-08-13 18:06:06.666454 7f5c973a57a0 20 read_log 194215'15872 (194215'15871) delete 1ad8accd/benchmark_data_storage0111-ib.pocstor_2228_object24574/head//24 by client.2174595.0:42080 0.000000 2014-08-13 18:06:06.666465 7f5c973a57a0 20 read_log 194215'15873 (192834'15157) clean 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by osd.6.0:988 2014-08-12 15:39:20.929179 2014-08-13 18:06:06.666486 7f5c973a57a0 20 read_log 194215'15874 (194215'15873) delete 4c90bccd/benchmark_data_storage0111-ib.pocstor_41526_object22918/head//24 by client.2174595.0:42082 0.000000 2014-08-13 18:06:06.666498 7f5c973a57a0 20 read_log 194215'15875 (192888'15283) clean e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by osd.6.0:990 2014-08-12 15:39:20.936228 2014-08-13 18:06:06.666511 7f5c973a57a0 20 read_log 194215'15876 (194215'15875) delete e022dccd/benchmark_data_storage0111-ib.pocstor_41097_object11738/head//24 by client.2174595.0:42084 0.000000 2014-08-13 18:06:06.666522 7f5c973a57a0 20 read_log 194215'15877 (192834'14602) clean ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by osd.6.0:992 2014-08-12 15:39:20.942864 2014-08-13 18:06:06.666534 7f5c973a57a0 20 read_log 194215'15878 (194215'15877) delete ec88dccd/benchmark_data_storage0111-ib.pocstor_26659_object15617/head//24 by client.2174595.0:42086 0.000000 2014-08-13 18:06:06.666545 7f5c973a57a0 20 read_log 194215'15879 (193305'15601) clean aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by osd.6.0:994 2014-08-12 15:39:20.950045 2014-08-13 18:06:06.666557 7f5c973a57a0 20 read_log 194215'15880 (194215'15879) delete aed3eccd/benchmark_data_storage0111-ib.pocstor_62621_object26648/head//24 by client.2174595.0:42088 0.000000 2014-08-13 18:06:06.666568 7f5c973a57a0 20 read_log 194215'15881 (193305'15575) clean d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by osd.6.0:996 2014-08-12 15:39:20.957126 2014-08-13 18:06:06.666580 7f5c973a57a0 20 read_log 194215'15882 (194215'15881) delete d238eccd/benchmark_data_storage0111-ib.pocstor_62621_object2634/head//24 by client.2174595.0:42090 0.000000 2014-08-13 18:06:06.666590 7f5c973a57a0 20 read_log 194215'15883 (192834'15057) clean 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by osd.6.0:998 2014-08-12 15:39:20.963782 2014-08-13 18:06:06.666602 7f5c973a57a0 20 read_log 194215'15884 (194215'15883) delete 13f0fccd/benchmark_data_storage0111-ib.pocstor_41526_object8612/head//24 by client.2174595.0:42092 0.000000 2014-08-13 18:06:06.666613 7f5c973a57a0 20 read_log 194215'15885 (192834'14545) clean 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by osd.6.0:1000 2014-08-12 15:39:20.970054 2014-08-13 18:06:06.666625 7f5c973a57a0 20 read_log 194215'15886 (194215'15885) delete 3487fccd/benchmark_data_storage0111-ib.pocstor_26659_object8730/head//24 by client.2174595.0:42094 0.000000 2014-08-13 18:06:06.666636 7f5c973a57a0 20 read_log 194217'15887 (194217'15887) modify cd/hit_set_24.cd_archive_2014-08-12 15:39:20.336577_2014-08-12 17:25:05.320436/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 18:06:06.666648 7f5c973a57a0 20 read_log 194217'15888 (194215'15698) delete cd/hit_set_24.cd_archive_2014-08-12 15:32:25.874029_2014-08-12 15:39:20.336162/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 18:06:06.666661 7f5c973a57a0 20 read_log 194217'15889 (0'0) promote 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by osd.6.0:1568 2014-08-12 17:25:05.324213 2014-08-13 18:06:06.666674 7f5c973a57a0 20 read_log 194217'15890 (194217'15889) modify 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by client.1487927.0:412 2014-08-12 17:25:05.313391 2014-08-13 18:06:06.666686 7f5c973a57a0 20 read_log 194370'15891 (0'0) promote 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by osd.6.0:1599 2014-08-12 20:30:25.525467 2014-08-13 18:06:06.666700 7f5c973a57a0 20 read_log 194370'15892 (194370'15891) modify 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by client.1508531.0:2994 2014-08-12 20:30:25.517860 2014-08-13 18:06:06.666723 7f5c973a57a0 20 read_log 194370'15893 (0'0) promote f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by osd.6.0:1605 2014-08-12 20:30:28.438433 2014-08-13 18:06:06.666736 7f5c973a57a0 20 read_log 194370'15894 (194370'15893) modify f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by client.1508531.0:3765 2014-08-12 20:30:28.431666 2014-08-13 18:06:06.666749 7f5c973a57a0 20 read_log 194370'15895 (0'0) promote 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by osd.6.0:1623 2014-08-12 20:30:41.566511 2014-08-13 18:06:06.666761 7f5c973a57a0 20 read_log 194370'15896 (194370'15895) modify 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by client.1508531.0:6839 2014-08-12 20:30:41.557825 2014-08-13 18:06:06.666774 7f5c973a57a0 20 read_log 194370'15897 (0'0) promote 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by osd.6.0:1643 2014-08-12 20:30:58.540988 2014-08-13 18:06:06.666786 7f5c973a57a0 20 read_log 194370'15898 (194370'15897) modify 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by client.1508531.0:9874 2014-08-12 20:30:58.532474 2014-08-13 18:06:06.666801 7f5c973a57a0 20 read_log 194370'15899 (0'0) promote a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by osd.6.0:1647 2014-08-12 20:31:06.764327 2014-08-13 18:06:06.666814 7f5c973a57a0 20 read_log 194370'15900 (194370'15899) modify a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by client.1508531.0:11245 2014-08-12 20:31:06.755136 2014-08-13 18:06:06.666828 7f5c973a57a0 20 read_log 194370'15901 (0'0) promote a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by osd.6.0:1651 2014-08-12 20:31:10.396334 2014-08-13 18:06:06.666841 7f5c973a57a0 20 read_log 194370'15902 (194370'15901) modify a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by client.1508531.0:11854 2014-08-12 20:31:10.388241 2014-08-13 18:06:06.666853 7f5c973a57a0 20 read_log 194370'15903 (0'0) promote 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by osd.6.0:1665 2014-08-12 20:31:23.235796 2014-08-13 18:06:06.666865 7f5c973a57a0 20 read_log 194370'15904 (194370'15903) modify 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by client.1508531.0:13405 2014-08-12 20:31:23.228485 2014-08-13 18:06:06.666880 7f5c973a57a0 20 read_log 194377'15905 (0'0) promote e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by osd.6.0:1671 2014-08-12 20:32:58.710442 2014-08-13 18:06:06.666892 7f5c973a57a0 20 read_log 194377'15906 (194377'15905) modify e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by client.2231190.0:361 2014-08-12 20:32:58.702360 2014-08-13 18:06:06.666906 7f5c973a57a0 20 read_log 194377'15907 (0'0) promote 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by osd.6.0:1673 2014-08-12 20:33:00.037487 2014-08-13 18:06:06.666919 7f5c973a57a0 20 read_log 194377'15908 (194377'15907) modify 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by client.2231190.0:666 2014-08-12 20:33:00.029776 2014-08-13 18:06:06.666934 7f5c973a57a0 20 read_log 194377'15909 (0'0) promote b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by osd.6.0:1683 2014-08-12 20:33:06.009473 2014-08-13 18:06:06.666946 7f5c973a57a0 20 read_log 194377'15910 (194377'15909) modify b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by client.2231190.0:2186 2014-08-12 20:33:06.003220 2014-08-13 18:06:06.666960 7f5c973a57a0 20 read_log 194377'15911 (0'0) promote 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by osd.6.0:1685 2014-08-12 20:33:06.471091 2014-08-13 18:06:06.666973 7f5c973a57a0 20 read_log 194377'15912 (194377'15911) modify 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by client.2231190.0:2302 2014-08-12 20:33:06.464047 2014-08-13 18:06:06.666997 7f5c973a57a0 20 read_log 194377'15913 (0'0) promote 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by osd.6.0:1701 2014-08-12 20:33:09.365716 2014-08-13 18:06:06.667012 7f5c973a57a0 20 read_log 194377'15914 (194377'15913) modify 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by client.2231190.0:3007 2014-08-12 20:33:09.359555 2014-08-13 18:06:06.667025 7f5c973a57a0 20 read_log 194377'15915 (0'0) promote a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by osd.6.0:1711 2014-08-12 20:33:11.084556 2014-08-13 18:06:06.667039 7f5c973a57a0 20 read_log 194377'15916 (194377'15915) modify a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by client.2231190.0:3446 2014-08-12 20:33:11.076919 2014-08-13 18:06:06.667052 7f5c973a57a0 20 read_log 194377'15917 (0'0) promote 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by osd.6.0:1713 2014-08-12 20:33:11.599861 2014-08-13 18:06:06.667066 7f5c973a57a0 20 read_log 194377'15918 (194377'15917) modify 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by client.2231190.0:3547 2014-08-12 20:33:11.591730 2014-08-13 18:06:06.667084 7f5c973a57a0 20 read_log 194377'15919 (0'0) promote 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by osd.6.0:1721 2014-08-12 20:33:16.240465 2014-08-13 18:06:06.667098 7f5c973a57a0 20 read_log 194377'15920 (194377'15919) modify 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by client.2231190.0:4376 2014-08-12 20:33:16.232582 2014-08-13 18:06:06.667112 7f5c973a57a0 20 read_log 194380'15921 (0'0) promote 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by osd.6.0:1729 2014-08-12 20:33:27.663787 2014-08-13 18:06:06.667125 7f5c973a57a0 20 read_log 194380'15922 (194380'15921) modify 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by client.2231190.0:5337 2014-08-12 20:33:27.655777 2014-08-13 18:06:06.667137 7f5c973a57a0 20 read_log 194383'15923 (0'0) promote bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by osd.6.0:1735 2014-08-12 20:34:06.936028 2014-08-13 18:06:06.667157 7f5c973a57a0 20 read_log 194383'15924 (194383'15923) modify bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by client.2231190.0:6691 2014-08-12 20:34:06.927281 2014-08-13 18:06:06.667172 7f5c973a57a0 20 read_log 194383'15925 (0'0) promote 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by osd.6.0:1737 2014-08-12 20:34:07.672346 2014-08-13 18:06:06.667185 7f5c973a57a0 20 read_log 194383'15926 (194383'15925) modify 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by client.2231190.0:6724 2014-08-12 20:34:07.664179 2014-08-13 18:06:06.667200 7f5c973a57a0 20 read_log 194393'15927 (0'0) promote 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by osd.6.0:1751 2014-08-12 20:49:26.130157 2014-08-13 18:06:06.667212 7f5c973a57a0 20 read_log 194393'15928 (194393'15927) modify 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by client.1064182.0:140 2014-08-12 20:49:26.124270 2014-08-13 18:06:06.667227 7f5c973a57a0 20 read_log 194416'15929 (0'0) promote ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by osd.6.0:1880 2014-08-13 15:28:31.344921 2014-08-13 18:06:06.667240 7f5c973a57a0 20 read_log 194416'15930 (194416'15929) modify ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by client.2441958.0:279 2014-08-13 15:28:31.335616 2014-08-13 18:06:06.667257 7f5c973a57a0 20 read_log 194416'15931 (0'0) promote 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by osd.6.0:1885 2014-08-13 15:28:33.027558 2014-08-13 18:06:06.667281 7f5c973a57a0 20 read_log 194416'15932 (194416'15931) modify 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by client.2441958.0:639 2014-08-13 15:28:33.018056 2014-08-13 18:06:06.667297 7f5c973a57a0 20 read_log 194416'15933 (0'0) promote 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by osd.6.0:1887 2014-08-13 15:28:34.899061 2014-08-13 18:06:06.667313 7f5c973a57a0 20 read_log 194416'15934 (194416'15933) modify 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by client.2441958.0:1101 2014-08-13 15:28:34.891565 2014-08-13 18:06:06.667328 7f5c973a57a0 20 read_log 194416'15935 (0'0) promote affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by osd.6.0:1897 2014-08-13 15:28:42.137168 2014-08-13 18:06:06.667340 7f5c973a57a0 20 read_log 194416'15936 (194416'15935) modify affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by client.2441958.0:3017 2014-08-13 15:28:42.129743 2014-08-13 18:06:06.667353 7f5c973a57a0 20 read_log 194416'15937 (0'0) promote 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by osd.6.0:1909 2014-08-13 15:28:49.814240 2014-08-13 18:06:06.667378 7f5c973a57a0 20 read_log 194416'15938 (194416'15937) modify 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by client.2441958.0:4978 2014-08-13 15:28:49.805250 2014-08-13 18:06:06.667395 7f5c973a57a0 20 read_log 194416'15939 (0'0) promote e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by osd.6.0:1913 2014-08-13 15:28:51.454527 2014-08-13 18:06:06.667411 7f5c973a57a0 20 read_log 194416'15940 (194416'15939) modify e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by client.2441958.0:5409 2014-08-13 15:28:51.445994 2014-08-13 18:06:06.667433 7f5c973a57a0 20 read_log 194416'15941 (0'0) promote abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by osd.6.0:1917 2014-08-13 15:28:55.148821 2014-08-13 18:06:06.667446 7f5c973a57a0 20 read_log 194416'15942 (194416'15941) modify abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by client.2441958.0:6304 2014-08-13 15:28:55.140350 2014-08-13 18:06:06.667459 7f5c973a57a0 20 read_log 194416'15943 (0'0) promote aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by osd.6.0:1929 2014-08-13 15:29:02.033402 2014-08-13 18:06:06.667478 7f5c973a57a0 20 read_log 194416'15944 (194416'15943) modify aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by client.2441958.0:8132 2014-08-13 15:29:02.024936 2014-08-13 18:06:06.667494 7f5c973a57a0 20 read_log 194416'15945 (0'0) promote b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by osd.6.0:1933 2014-08-13 15:29:03.462587 2014-08-13 18:06:06.667506 7f5c973a57a0 20 read_log 194416'15946 (194416'15945) modify b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by client.2441958.0:8512 2014-08-13 15:29:03.456402 2014-08-13 18:06:06.667525 7f5c973a57a0 20 read_log 194416'15947 (0'0) promote a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by osd.6.0:1935 2014-08-13 15:29:03.494696 2014-08-13 18:06:06.667540 7f5c973a57a0 20 read_log 194416'15948 (194416'15947) modify a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by client.2441958.0:8521 2014-08-13 15:29:03.488161 2014-08-13 18:06:06.667553 7f5c973a57a0 20 read_log 194416'15949 (0'0) promote 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by osd.6.0:1937 2014-08-13 15:29:03.782091 2014-08-13 18:06:06.667566 7f5c973a57a0 20 read_log 194416'15950 (194416'15949) modify 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by client.2441958.0:8598 2014-08-13 15:29:03.773943 2014-08-13 18:06:06.667596 7f5c973a57a0 20 read_log 194416'15951 (0'0) promote dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by osd.6.0:1941 2014-08-13 15:29:06.950672 2014-08-13 18:06:06.667621 7f5c973a57a0 20 read_log 194416'15952 (194416'15951) modify dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by client.2441958.0:9388 2014-08-13 15:29:06.942961 2014-08-13 18:06:06.667655 7f5c973a57a0 20 read_log 194416'15953 (0'0) promote 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by osd.6.0:1943 2014-08-13 15:29:10.732496 2014-08-13 18:06:06.667671 7f5c973a57a0 20 read_log 194416'15954 (194416'15953) modify 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by client.2441958.0:10313 2014-08-13 15:29:10.724222 2014-08-13 18:06:06.667693 7f5c973a57a0 20 read_log 194416'15955 (0'0) promote 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by osd.6.0:1945 2014-08-13 15:29:10.911215 2014-08-13 18:06:06.667709 7f5c973a57a0 20 read_log 194416'15956 (194416'15955) modify 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by client.2441958.0:10359 2014-08-13 15:29:10.901520 2014-08-13 18:06:06.667722 7f5c973a57a0 20 read_log 194416'15957 (0'0) promote d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by osd.6.0:1947 2014-08-13 15:29:11.147372 2014-08-13 18:06:06.667735 7f5c973a57a0 20 read_log 194416'15958 (194416'15957) modify d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by client.2441958.0:10422 2014-08-13 15:29:11.139455 2014-08-13 18:06:06.667747 7f5c973a57a0 20 read_log 194416'15959 (0'0) promote 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by osd.6.0:1953 2014-08-13 15:29:16.798127 2014-08-13 18:06:06.667759 7f5c973a57a0 20 read_log 194416'15960 (194416'15959) modify 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by client.2441958.0:11823 2014-08-13 15:29:16.790522 2014-08-13 18:06:06.667775 7f5c973a57a0 20 read_log 194416'15961 (0'0) promote 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by osd.6.0:1975 2014-08-13 15:29:34.201290 2014-08-13 18:06:06.667787 7f5c973a57a0 20 read_log 194416'15962 (194416'15961) modify 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by client.2441958.0:15771 2014-08-13 15:29:34.192971 2014-08-13 18:06:06.667800 7f5c973a57a0 20 read_log 194416'15963 (0'0) promote dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by osd.6.0:1977 2014-08-13 15:29:34.431122 2014-08-13 18:06:06.667812 7f5c973a57a0 20 read_log 194416'15964 (194416'15963) modify dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by client.2441958.0:15810 2014-08-13 15:29:34.422385 2014-08-13 18:06:06.667824 7f5c973a57a0 20 read_log 194416'15965 (0'0) promote 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by osd.6.0:1987 2014-08-13 15:29:58.847379 2014-08-13 18:06:06.667836 7f5c973a57a0 20 read_log 194416'15966 (194416'15965) modify 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by client.2441958.0:17816 2014-08-13 15:29:58.837982 2014-08-13 18:06:06.667848 7f5c973a57a0 20 read_log 194416'15967 (0'0) promote 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by osd.6.0:1989 2014-08-13 15:30:01.417168 2014-08-13 18:06:06.667861 7f5c973a57a0 20 read_log 194416'15968 (194416'15967) modify 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by client.2441958.0:17996 2014-08-13 15:30:01.408574 2014-08-13 18:06:06.667873 7f5c973a57a0 20 read_log 194422'15969 (0'0) promote 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by osd.6.0:1993 2014-08-13 15:32:20.951313 2014-08-13 18:06:06.667885 7f5c973a57a0 20 read_log 194422'15970 (194422'15969) modify 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by client.1148203.0:1421 2014-08-13 15:32:20.943748 2014-08-13 18:06:06.667912 7f5c973a57a0 20 read_log 194422'15971 (0'0) promote bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by osd.6.0:1999 2014-08-13 15:32:25.701996 2014-08-13 18:06:06.667930 7f5c973a57a0 20 read_log 194422'15972 (194422'15971) modify bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by client.1148203.0:2648 2014-08-13 15:32:25.694476 2014-08-13 18:06:06.667943 7f5c973a57a0 20 read_log 194422'15973 (0'0) promote b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by osd.6.0:2003 2014-08-13 15:32:26.834606 2014-08-13 18:06:06.667955 7f5c973a57a0 20 read_log 194422'15974 (194422'15973) modify b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by client.1148203.0:2868 2014-08-13 15:32:26.586165 2014-08-13 18:06:06.667967 7f5c973a57a0 20 read_log 194422'15975 (0'0) promote a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by osd.6.0:2009 2014-08-13 15:32:34.957558 2014-08-13 18:06:06.667980 7f5c973a57a0 20 read_log 194422'15976 (194422'15975) modify a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by client.1148203.0:5028 2014-08-13 15:32:34.948621 2014-08-13 18:06:06.667992 7f5c973a57a0 20 read_log 194422'15977 (0'0) promote 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by osd.6.0:2013 2014-08-13 15:32:38.313723 2014-08-13 18:06:06.668004 7f5c973a57a0 20 read_log 194422'15978 (194422'15977) modify 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by client.1148203.0:5665 2014-08-13 15:32:38.306136 2014-08-13 18:06:06.668016 7f5c973a57a0 20 read_log 194422'15979 (0'0) promote f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by osd.6.0:2025 2014-08-13 15:32:49.374130 2014-08-13 18:06:06.668028 7f5c973a57a0 20 read_log 194422'15980 (194422'15979) modify f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by client.1148203.0:7622 2014-08-13 15:32:49.364629 2014-08-13 18:06:06.668046 7f5c973a57a0 20 read_log 194422'15981 (0'0) promote 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by osd.6.0:2031 2014-08-13 15:32:52.463487 2014-08-13 18:06:06.668058 7f5c973a57a0 20 read_log 194422'15982 (194422'15981) modify 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by client.1148203.0:8117 2014-08-13 15:32:52.454672 2014-08-13 18:06:06.668070 7f5c973a57a0 20 read_log 194422'15983 (0'0) promote bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by osd.6.0:2035 2014-08-13 15:32:57.693618 2014-08-13 18:06:06.668082 7f5c973a57a0 20 read_log 194422'15984 (194422'15983) modify bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by client.1148203.0:8807 2014-08-13 15:32:57.684987 2014-08-13 18:06:06.668094 7f5c973a57a0 20 read_log 194422'15985 (0'0) promote 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by osd.6.0:2037 2014-08-13 15:33:01.164622 2014-08-13 18:06:06.668107 7f5c973a57a0 20 read_log 194422'15986 (194422'15985) modify 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by client.1148203.0:9164 2014-08-13 15:33:01.155232 2014-08-13 18:06:06.668121 7f5c973a57a0 20 read_log 194422'15987 (0'0) promote 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by osd.6.0:2039 2014-08-13 15:33:07.403937 2014-08-13 18:06:06.668134 7f5c973a57a0 20 read_log 194422'15988 (194422'15987) modify 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by client.1148203.0:9769 2014-08-13 15:33:07.394203 2014-08-13 18:06:06.668153 7f5c973a57a0 20 read_log 194422'15989 (0'0) promote acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by osd.6.0:2041 2014-08-13 15:33:12.330934 2014-08-13 18:06:06.668168 7f5c973a57a0 20 read_log 194422'15990 (194422'15989) modify acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by client.1148203.0:10083 2014-08-13 15:33:12.322811 2014-08-13 18:06:06.668188 7f5c973a57a0 20 read_log 194422'15991 (0'0) promote ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by osd.6.0:2043 2014-08-13 15:33:19.817059 2014-08-13 18:06:06.668202 7f5c973a57a0 20 read_log 194422'15992 (194422'15991) modify ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by client.1148203.0:10351 2014-08-13 15:33:19.809562 2014-08-13 18:06:06.668219 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.668832 7f5c973a57a0 10 read_log checking for missing items over interval (194215'15886,194422'15992] 2014-08-13 18:06:06.668883 7f5c973a57a0 15 read_log missing 194422'15992 (194422'15991) modify ad9cf8cd/benchmark_data_storage0111-ib.pocstor_41231_object10350/head//24 by client.1148203.0:10351 2014-08-13 15:33:19.809562 2014-08-13 18:06:06.668923 7f5c973a57a0 15 read_log missing 194422'15990 (194422'15989) modify acc958cd/benchmark_data_storage0111-ib.pocstor_41231_object10082/head//24 by client.1148203.0:10083 2014-08-13 15:33:12.322811 2014-08-13 18:06:06.668959 7f5c973a57a0 15 read_log missing 194422'15988 (194422'15987) modify 53fb9ccd/benchmark_data_storage0111-ib.pocstor_41231_object9768/head//24 by client.1148203.0:9769 2014-08-13 15:33:07.394203 2014-08-13 18:06:06.668995 7f5c973a57a0 15 read_log missing 194422'15986 (194422'15985) modify 2874f0cd/benchmark_data_storage0111-ib.pocstor_41231_object9163/head//24 by client.1148203.0:9164 2014-08-13 15:33:01.155232 2014-08-13 18:06:06.669030 7f5c973a57a0 15 read_log missing 194422'15984 (194422'15983) modify bb6000cd/benchmark_data_storage0111-ib.pocstor_41231_object8806/head//24 by client.1148203.0:8807 2014-08-13 15:32:57.684987 2014-08-13 18:06:06.669067 7f5c973a57a0 15 read_log missing 194422'15982 (194422'15981) modify 42592ccd/benchmark_data_storage0111-ib.pocstor_41231_object8116/head//24 by client.1148203.0:8117 2014-08-13 15:32:52.454672 2014-08-13 18:06:06.669102 7f5c973a57a0 15 read_log missing 194422'15980 (194422'15979) modify f06a90cd/benchmark_data_storage0111-ib.pocstor_41231_object7621/head//24 by client.1148203.0:7622 2014-08-13 15:32:49.364629 2014-08-13 18:06:06.669138 7f5c973a57a0 15 read_log missing 194422'15978 (194422'15977) modify 4a2f7ccd/benchmark_data_storage0111-ib.pocstor_41231_object5664/head//24 by client.1148203.0:5665 2014-08-13 15:32:38.306136 2014-08-13 18:06:06.669186 7f5c973a57a0 15 read_log missing 194422'15976 (194422'15975) modify a68d14cd/benchmark_data_storage0111-ib.pocstor_41231_object5027/head//24 by client.1148203.0:5028 2014-08-13 15:32:34.948621 2014-08-13 18:06:06.669225 7f5c973a57a0 15 read_log missing 194422'15974 (194422'15973) modify b0d92ccd/benchmark_data_storage0111-ib.pocstor_41231_object2867/head//24 by client.1148203.0:2868 2014-08-13 15:32:26.586165 2014-08-13 18:06:06.669261 7f5c973a57a0 15 read_log missing 194422'15972 (194422'15971) modify bcaaa4cd/benchmark_data_storage0111-ib.pocstor_41231_object2647/head//24 by client.1148203.0:2648 2014-08-13 15:32:25.694476 2014-08-13 18:06:06.669297 7f5c973a57a0 15 read_log missing 194422'15970 (194422'15969) modify 70c198cd/benchmark_data_storage0111-ib.pocstor_41231_object1420/head//24 by client.1148203.0:1421 2014-08-13 15:32:20.943748 2014-08-13 18:06:06.669344 7f5c973a57a0 15 read_log missing 194416'15968 (194416'15967) modify 37e0c8cd/benchmark_data_storage0111-ib.pocstor_19103_object17995/head//24 by client.2441958.0:17996 2014-08-13 15:30:01.408574 2014-08-13 18:06:06.669390 7f5c973a57a0 15 read_log missing 194416'15966 (194416'15965) modify 3e8daccd/benchmark_data_storage0111-ib.pocstor_19103_object17815/head//24 by client.2441958.0:17816 2014-08-13 15:29:58.837982 2014-08-13 18:06:06.669433 7f5c973a57a0 15 read_log missing 194416'15964 (194416'15963) modify dbcc58cd/benchmark_data_storage0111-ib.pocstor_19103_object15809/head//24 by client.2441958.0:15810 2014-08-13 15:29:34.422385 2014-08-13 18:06:06.669469 7f5c973a57a0 15 read_log missing 194416'15962 (194416'15961) modify 666634cd/benchmark_data_storage0111-ib.pocstor_19103_object15770/head//24 by client.2441958.0:15771 2014-08-13 15:29:34.192971 2014-08-13 18:06:06.669504 7f5c973a57a0 15 read_log missing 194416'15960 (194416'15959) modify 6fe938cd/benchmark_data_storage0111-ib.pocstor_19103_object11822/head//24 by client.2441958.0:11823 2014-08-13 15:29:16.790522 2014-08-13 18:06:06.669540 7f5c973a57a0 15 read_log missing 194416'15958 (194416'15957) modify d4b008cd/benchmark_data_storage0111-ib.pocstor_19103_object10421/head//24 by client.2441958.0:10422 2014-08-13 15:29:11.139455 2014-08-13 18:06:06.669576 7f5c973a57a0 15 read_log missing 194416'15956 (194416'15955) modify 3a2900cd/benchmark_data_storage0111-ib.pocstor_19103_object10358/head//24 by client.2441958.0:10359 2014-08-13 15:29:10.901520 2014-08-13 18:06:06.669614 7f5c973a57a0 15 read_log missing 194416'15954 (194416'15953) modify 307d0ccd/benchmark_data_storage0111-ib.pocstor_19103_object10312/head//24 by client.2441958.0:10313 2014-08-13 15:29:10.724222 2014-08-13 18:06:06.669649 7f5c973a57a0 15 read_log missing 194416'15952 (194416'15951) modify dbffc8cd/benchmark_data_storage0111-ib.pocstor_19103_object9387/head//24 by client.2441958.0:9388 2014-08-13 15:29:06.942961 2014-08-13 18:06:06.669684 7f5c973a57a0 15 read_log missing 194416'15950 (194416'15949) modify 7520e0cd/benchmark_data_storage0111-ib.pocstor_19103_object8597/head//24 by client.2441958.0:8598 2014-08-13 15:29:03.773943 2014-08-13 18:06:06.669720 7f5c973a57a0 15 read_log missing 194416'15948 (194416'15947) modify a7174cd/benchmark_data_storage0111-ib.pocstor_19103_object8520/head//24 by client.2441958.0:8521 2014-08-13 15:29:03.488161 2014-08-13 18:06:06.669755 7f5c973a57a0 15 read_log missing 194416'15946 (194416'15945) modify b88204cd/benchmark_data_storage0111-ib.pocstor_19103_object8511/head//24 by client.2441958.0:8512 2014-08-13 15:29:03.456402 2014-08-13 18:06:06.669790 7f5c973a57a0 15 read_log missing 194416'15944 (194416'15943) modify aa9008cd/benchmark_data_storage0111-ib.pocstor_19103_object8131/head//24 by client.2441958.0:8132 2014-08-13 15:29:02.024936 2014-08-13 18:06:06.669825 7f5c973a57a0 15 read_log missing 194416'15942 (194416'15941) modify abe2e4cd/benchmark_data_storage0111-ib.pocstor_19103_object6303/head//24 by client.2441958.0:6304 2014-08-13 15:28:55.140350 2014-08-13 18:06:06.669860 7f5c973a57a0 15 read_log missing 194416'15940 (194416'15939) modify e1ef08cd/benchmark_data_storage0111-ib.pocstor_19103_object5408/head//24 by client.2441958.0:5409 2014-08-13 15:28:51.445994 2014-08-13 18:06:06.669895 7f5c973a57a0 15 read_log missing 194416'15938 (194416'15937) modify 8ad678cd/benchmark_data_storage0111-ib.pocstor_19103_object4977/head//24 by client.2441958.0:4978 2014-08-13 15:28:49.805250 2014-08-13 18:06:06.669930 7f5c973a57a0 15 read_log missing 194416'15936 (194416'15935) modify affaf0cd/benchmark_data_storage0111-ib.pocstor_19103_object3016/head//24 by client.2441958.0:3017 2014-08-13 15:28:42.129743 2014-08-13 18:06:06.669966 7f5c973a57a0 15 read_log missing 194416'15934 (194416'15933) modify 2e5938cd/benchmark_data_storage0111-ib.pocstor_19103_object1100/head//24 by client.2441958.0:1101 2014-08-13 15:28:34.891565 2014-08-13 18:06:06.670001 7f5c973a57a0 15 read_log missing 194416'15932 (194416'15931) modify 54b7fccd/benchmark_data_storage0111-ib.pocstor_19103_object638/head//24 by client.2441958.0:639 2014-08-13 15:28:33.018056 2014-08-13 18:06:06.670036 7f5c973a57a0 15 read_log missing 194416'15930 (194416'15929) modify ca6e30cd/benchmark_data_storage0111-ib.pocstor_19103_object278/head//24 by client.2441958.0:279 2014-08-13 15:28:31.335616 2014-08-13 18:06:06.670071 7f5c973a57a0 15 read_log missing 194393'15928 (194393'15927) modify 1b07b8cd/benchmark_data_storage0111-ib.pocstor_32482_object139/head//24 by client.1064182.0:140 2014-08-12 20:49:26.124270 2014-08-13 18:06:06.670106 7f5c973a57a0 15 read_log missing 194383'15926 (194383'15925) modify 69e92ccd/benchmark_data_storage0111-ib.pocstor_25065_object6723/head//24 by client.2231190.0:6724 2014-08-12 20:34:07.664179 2014-08-13 18:06:06.670142 7f5c973a57a0 15 read_log missing 194383'15924 (194383'15923) modify bf2814cd/benchmark_data_storage0111-ib.pocstor_25065_object6690/head//24 by client.2231190.0:6691 2014-08-12 20:34:06.927281 2014-08-13 18:06:06.670183 7f5c973a57a0 15 read_log missing 194380'15922 (194380'15921) modify 2395b0cd/benchmark_data_storage0111-ib.pocstor_25065_object5336/head//24 by client.2231190.0:5337 2014-08-12 20:33:27.655777 2014-08-13 18:06:06.670219 7f5c973a57a0 15 read_log missing 194377'15920 (194377'15919) modify 24af78cd/benchmark_data_storage0111-ib.pocstor_25065_object4375/head//24 by client.2231190.0:4376 2014-08-12 20:33:16.232582 2014-08-13 18:06:06.670258 7f5c973a57a0 15 read_log missing 194377'15918 (194377'15917) modify 1706d0cd/benchmark_data_storage0111-ib.pocstor_25065_object3546/head//24 by client.2231190.0:3547 2014-08-12 20:33:11.591730 2014-08-13 18:06:06.670299 7f5c973a57a0 15 read_log missing 194377'15916 (194377'15915) modify a0f6e8cd/benchmark_data_storage0111-ib.pocstor_25065_object3445/head//24 by client.2231190.0:3446 2014-08-12 20:33:11.076919 2014-08-13 18:06:06.670335 7f5c973a57a0 15 read_log missing 194377'15914 (194377'15913) modify 6f1894cd/benchmark_data_storage0111-ib.pocstor_25065_object3006/head//24 by client.2231190.0:3007 2014-08-12 20:33:09.359555 2014-08-13 18:06:06.670381 7f5c973a57a0 15 read_log missing 194377'15912 (194377'15911) modify 16df74cd/benchmark_data_storage0111-ib.pocstor_25065_object2301/head//24 by client.2231190.0:2302 2014-08-12 20:33:06.464047 2014-08-13 18:06:06.670427 7f5c973a57a0 15 read_log missing 194377'15910 (194377'15909) modify b29fc4cd/benchmark_data_storage0111-ib.pocstor_25065_object2185/head//24 by client.2231190.0:2186 2014-08-12 20:33:06.003220 2014-08-13 18:06:06.670465 7f5c973a57a0 15 read_log missing 194377'15908 (194377'15907) modify 8d12e0cd/benchmark_data_storage0111-ib.pocstor_25065_object665/head//24 by client.2231190.0:666 2014-08-12 20:33:00.029776 2014-08-13 18:06:06.670501 7f5c973a57a0 15 read_log missing 194377'15906 (194377'15905) modify e5b1e8cd/benchmark_data_storage0111-ib.pocstor_25065_object360/head//24 by client.2231190.0:361 2014-08-12 20:32:58.702360 2014-08-13 18:06:06.670537 7f5c973a57a0 15 read_log missing 194370'15904 (194370'15903) modify 4aa030cd/benchmark_data_storage0111-ib.pocstor_12495_object13404/head//24 by client.1508531.0:13405 2014-08-12 20:31:23.228485 2014-08-13 18:06:06.670584 7f5c973a57a0 15 read_log missing 194370'15902 (194370'15901) modify a7239ccd/benchmark_data_storage0111-ib.pocstor_12495_object11853/head//24 by client.1508531.0:11854 2014-08-12 20:31:10.388241 2014-08-13 18:06:06.670625 7f5c973a57a0 15 read_log missing 194370'15900 (194370'15899) modify a5f108cd/benchmark_data_storage0111-ib.pocstor_12495_object11244/head//24 by client.1508531.0:11245 2014-08-12 20:31:06.755136 2014-08-13 18:06:06.670662 7f5c973a57a0 15 read_log missing 194370'15898 (194370'15897) modify 43c588cd/benchmark_data_storage0111-ib.pocstor_12495_object9873/head//24 by client.1508531.0:9874 2014-08-12 20:30:58.532474 2014-08-13 18:06:06.670702 7f5c973a57a0 15 read_log missing 194370'15896 (194370'15895) modify 13e86ccd/benchmark_data_storage0111-ib.pocstor_12495_object6838/head//24 by client.1508531.0:6839 2014-08-12 20:30:41.557825 2014-08-13 18:06:06.670747 7f5c973a57a0 15 read_log missing 194370'15894 (194370'15893) modify f1d408cd/benchmark_data_storage0111-ib.pocstor_12495_object3764/head//24 by client.1508531.0:3765 2014-08-12 20:30:28.431666 2014-08-13 18:06:06.670784 7f5c973a57a0 15 read_log missing 194370'15892 (194370'15891) modify 2f6128cd/benchmark_data_storage0111-ib.pocstor_12495_object2993/head//24 by client.1508531.0:2994 2014-08-12 20:30:25.517860 2014-08-13 18:06:06.670821 7f5c973a57a0 15 read_log missing 194217'15890 (194217'15889) modify 2c5adccd/benchmark_data_storage0111-ib.pocstor_39693_object411/head//24 by client.1487927.0:412 2014-08-12 17:25:05.313391 2014-08-13 18:06:06.670881 7f5c973a57a0 15 read_log missing 194217'15887 (194217'15887) modify cd/hit_set_24.cd_archive_2014-08-12 15:39:20.336577_2014-08-12 17:25:05.320436/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:05.320667 2014-08-13 18:06:06.670898 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.670934 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive m=53] handle_loaded 2014-08-13 18:06:06.670953 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Initial 0.024046 0 0.000000 2014-08-13 18:06:06.670966 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Reset 2014-08-13 18:06:06.670975 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194487 2014-08-13 18:06:06.670985 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log((192718'12885,194422'15992], crt=194422'15992) 2014-08-13 18:06:06.670995 7f5c973a57a0 10 osd.4 194487 pgid 24.e2 coll 24.e2_head 2014-08-13 18:06:06.671154 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.e2 2014-08-13 18:06:06.671168 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.671202 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.e2(unlocked)] enter Initial 2014-08-13 18:06:06.671214 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.e2(unlocked)] enter NotTrimming 2014-08-13 18:06:06.671406 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.671559 7f5c973a57a0 20 read_log 192834'11894 (192834'11881) delete 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:12720 2014-08-11 15:24:42.480271 2014-08-13 18:06:06.671581 7f5c973a57a0 20 read_log 192834'11895 (192834'11882) delete 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:12721 2014-08-11 15:24:42.481508 2014-08-13 18:06:06.671595 7f5c973a57a0 20 read_log 192834'11901 (0'0) promote f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:12727 2014-08-11 15:24:46.244603 2014-08-13 18:06:06.671608 7f5c973a57a0 20 read_log 192834'11902 (192834'11901) modify f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by client.933448.0:14578 2014-08-11 15:24:46.234876 2014-08-13 18:06:06.671620 7f5c973a57a0 20 read_log 192834'11920 (192834'11669) clean 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:12746 2014-08-11 15:24:48.012738 2014-08-13 18:06:06.671633 7f5c973a57a0 20 read_log 192834'11922 (192834'11675) clean f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:12748 2014-08-11 15:24:48.337400 2014-08-13 18:06:06.671649 7f5c973a57a0 20 read_log 192834'11923 (192834'11677) clean ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:12749 2014-08-11 15:24:48.337629 2014-08-13 18:06:06.671670 7f5c973a57a0 20 read_log 192834'11933 (192834'11920) delete 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:12759 2014-08-11 15:24:48.365401 2014-08-13 18:06:06.671685 7f5c973a57a0 20 read_log 192834'11934 (192834'11922) delete f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:12760 2014-08-11 15:24:48.367939 2014-08-13 18:06:06.671698 7f5c973a57a0 20 read_log 192834'11935 (192834'11923) delete ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:12761 2014-08-11 15:24:48.370644 2014-08-13 18:06:06.671710 7f5c973a57a0 20 read_log 192834'11948 (0'0) promote 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:12774 2014-08-11 15:24:51.148696 2014-08-13 18:06:06.671723 7f5c973a57a0 20 read_log 192834'11949 (192834'11948) modify 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by client.933448.0:15763 2014-08-11 15:24:51.138951 2014-08-13 18:06:06.671735 7f5c973a57a0 20 read_log 192834'11965 (192834'11685) clean 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:12791 2014-08-11 15:24:53.645520 2014-08-13 18:06:06.671747 7f5c973a57a0 20 read_log 192834'11967 (192834'11693) clean 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:12793 2014-08-11 15:24:53.720878 2014-08-13 18:06:06.671760 7f5c973a57a0 20 read_log 192834'11975 (192834'11965) delete 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:12801 2014-08-11 15:24:53.747375 2014-08-13 18:06:06.671772 7f5c973a57a0 20 read_log 192834'11977 (192834'11967) delete 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:12803 2014-08-11 15:24:53.752224 2014-08-13 18:06:06.671786 7f5c973a57a0 20 read_log 192834'11981 (192834'11691) clean 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:12807 2014-08-11 15:24:53.775538 2014-08-13 18:06:06.671816 7f5c973a57a0 20 read_log 192834'11989 (0'0) promote cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:12815 2014-08-11 15:24:56.112339 2014-08-13 18:06:06.671829 7f5c973a57a0 20 read_log 192834'11990 (192834'11989) modify cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by client.933448.0:16529 2014-08-11 15:24:56.102816 2014-08-13 18:06:06.671842 7f5c973a57a0 20 read_log 192834'12008 (192834'11981) delete 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:12834 2014-08-11 15:24:59.169597 2014-08-13 18:06:06.671871 7f5c973a57a0 20 read_log 192834'12026 (0'0) promote e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:12852 2014-08-11 15:25:01.626757 2014-08-13 18:06:06.671884 7f5c973a57a0 20 read_log 192834'12027 (192834'12026) modify e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by client.933448.0:17788 2014-08-11 15:25:01.617803 2014-08-13 18:06:06.671896 7f5c973a57a0 20 read_log 192834'12098 (192834'11777) clean b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:12924 2014-08-11 15:25:15.169008 2014-08-13 18:06:06.671909 7f5c973a57a0 20 read_log 192834'12106 (192834'12098) delete b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:12932 2014-08-11 15:25:15.196894 2014-08-13 18:06:06.671921 7f5c973a57a0 20 read_log 192834'12110 (192834'11771) clean c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:12936 2014-08-11 15:25:15.233747 2014-08-13 18:06:06.671934 7f5c973a57a0 20 read_log 192834'12111 (192834'11767) clean 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:12937 2014-08-11 15:25:15.242215 2014-08-13 18:06:06.671946 7f5c973a57a0 20 read_log 192834'12127 (192834'12110) delete c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:12953 2014-08-11 15:25:20.607061 2014-08-13 18:06:06.671969 7f5c973a57a0 20 read_log 192834'12128 (192834'12111) delete 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:12954 2014-08-11 15:25:20.609450 2014-08-13 18:06:06.671982 7f5c973a57a0 20 read_log 192834'12138 (0'0) promote 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:12964 2014-08-11 15:25:20.825324 2014-08-13 18:06:06.671994 7f5c973a57a0 20 read_log 192834'12139 (192834'12138) modify 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by client.933448.0:20412 2014-08-11 15:25:20.815098 2014-08-13 18:06:06.672007 7f5c973a57a0 20 read_log 192834'12162 (192834'11811) clean 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:12988 2014-08-11 15:25:26.032514 2014-08-13 18:06:06.672019 7f5c973a57a0 20 read_log 192834'12164 (192834'11809) clean dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:12990 2014-08-11 15:25:26.225669 2014-08-13 18:06:06.672032 7f5c973a57a0 20 read_log 192834'12173 (192834'12162) delete 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:12999 2014-08-11 15:25:31.141974 2014-08-13 18:06:06.672044 7f5c973a57a0 20 read_log 192834'12175 (0'0) promote 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:13001 2014-08-11 15:25:31.277248 2014-08-13 18:06:06.672056 7f5c973a57a0 20 read_log 192834'12176 (192834'12175) modify 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by client.933448.0:21626 2014-08-11 15:25:31.268683 2014-08-13 18:06:06.672068 7f5c973a57a0 20 read_log 192834'12178 (192834'12164) delete dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13004 2014-08-11 15:25:31.335438 2014-08-13 18:06:06.672080 7f5c973a57a0 20 read_log 192834'12182 (192834'11813) clean de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13008 2014-08-11 15:25:31.349715 2014-08-13 18:06:06.672093 7f5c973a57a0 20 read_log 192834'12187 (192834'12182) delete de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13013 2014-08-11 15:25:31.365224 2014-08-13 18:06:06.672105 7f5c973a57a0 20 read_log 192834'12188 (192834'11823) clean 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13014 2014-08-11 15:25:31.404855 2014-08-13 18:06:06.672117 7f5c973a57a0 20 read_log 192834'12211 (192834'12188) delete 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13037 2014-08-11 15:25:36.754293 2014-08-13 18:06:06.672132 7f5c973a57a0 20 read_log 192834'12244 (0'0) promote 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13070 2014-08-11 15:25:38.730752 2014-08-13 18:06:06.672145 7f5c973a57a0 20 read_log 192834'12245 (192834'12244) modify 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by client.933448.0:22846 2014-08-11 15:25:38.701922 2014-08-13 18:06:06.672165 7f5c973a57a0 20 read_log 192834'12248 (0'0) promote 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13074 2014-08-11 15:25:38.773959 2014-08-13 18:06:06.672177 7f5c973a57a0 20 read_log 192834'12249 (192834'12248) modify 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by client.933448.0:22874 2014-08-11 15:25:38.737517 2014-08-13 18:06:06.672190 7f5c973a57a0 20 read_log 192834'12282 (0'0) promote 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13108 2014-08-11 15:25:40.959716 2014-08-13 18:06:06.672202 7f5c973a57a0 20 read_log 192834'12283 (192834'12282) modify 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by client.933448.0:24481 2014-08-11 15:25:40.923294 2014-08-13 18:06:06.672214 7f5c973a57a0 20 read_log 192834'12288 (0'0) promote f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13114 2014-08-11 15:25:41.241345 2014-08-13 18:06:06.672239 7f5c973a57a0 20 read_log 192834'12289 (192834'12288) modify f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by client.933448.0:24728 2014-08-11 15:25:41.223914 2014-08-13 18:06:06.672274 7f5c973a57a0 20 read_log 192834'12290 (0'0) promote ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13116 2014-08-11 15:25:41.664430 2014-08-13 18:06:06.672290 7f5c973a57a0 20 read_log 192834'12291 (192834'12290) modify ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by client.933448.0:24998 2014-08-11 15:25:41.634649 2014-08-13 18:06:06.672303 7f5c973a57a0 20 read_log 192834'12302 (0'0) promote 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13128 2014-08-11 15:25:42.193066 2014-08-13 18:06:06.672315 7f5c973a57a0 20 read_log 192834'12303 (192834'12302) modify 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by client.933448.0:25525 2014-08-11 15:25:42.173695 2014-08-13 18:06:06.672330 7f5c973a57a0 20 read_log 192834'12308 (0'0) promote 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13134 2014-08-11 15:25:42.317501 2014-08-13 18:06:06.672342 7f5c973a57a0 20 read_log 192834'12309 (192834'12308) modify 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by client.933448.0:25640 2014-08-11 15:25:42.296339 2014-08-13 18:06:06.672362 7f5c973a57a0 20 read_log 192834'12310 (0'0) promote 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13136 2014-08-11 15:25:42.335327 2014-08-13 18:06:06.672374 7f5c973a57a0 20 read_log 192834'12311 (192834'12310) modify 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by client.933448.0:25654 2014-08-11 15:25:42.313722 2014-08-13 18:06:06.672396 7f5c973a57a0 20 read_log 192834'12364 (192834'11902) clean f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13190 2014-08-11 15:25:47.012290 2014-08-13 18:06:06.672413 7f5c973a57a0 20 read_log 192834'12369 (192834'12364) delete f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13195 2014-08-11 15:25:47.044538 2014-08-13 18:06:06.672426 7f5c973a57a0 20 read_log 192834'12393 (0'0) promote 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13219 2014-08-11 15:25:48.826772 2014-08-13 18:06:06.672448 7f5c973a57a0 20 read_log 192834'12394 (192834'12393) modify 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by client.933448.0:30391 2014-08-11 15:25:48.805086 2014-08-13 18:06:06.672465 7f5c973a57a0 20 read_log 192834'12397 (0'0) promote c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13223 2014-08-11 15:25:49.119938 2014-08-13 18:06:06.672477 7f5c973a57a0 20 read_log 192834'12398 (192834'12397) modify c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by client.933448.0:30654 2014-08-11 15:25:49.094761 2014-08-13 18:06:06.672490 7f5c973a57a0 20 read_log 192834'12403 (0'0) promote b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13229 2014-08-11 15:25:49.346101 2014-08-13 18:06:06.672502 7f5c973a57a0 20 read_log 192834'12404 (192834'12403) modify b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by client.933448.0:30854 2014-08-11 15:25:49.313997 2014-08-13 18:06:06.672515 7f5c973a57a0 20 read_log 192834'12450 (192834'11949) clean 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13276 2014-08-11 15:25:52.579690 2014-08-13 18:06:06.672527 7f5c973a57a0 20 read_log 192834'12452 (192834'12450) delete 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13278 2014-08-11 15:25:52.598666 2014-08-13 18:06:06.672539 7f5c973a57a0 20 read_log 192834'12458 (0'0) promote dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13284 2014-08-11 15:25:52.840064 2014-08-13 18:06:06.672560 7f5c973a57a0 20 read_log 192834'12459 (192834'12458) modify dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by client.933448.0:33067 2014-08-11 15:25:52.809547 2014-08-13 18:06:06.672575 7f5c973a57a0 20 read_log 192834'12460 (0'0) promote 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:13286 2014-08-11 15:25:53.100320 2014-08-13 18:06:06.672587 7f5c973a57a0 20 read_log 192834'12461 (192834'12460) modify 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by client.933448.0:33230 2014-08-11 15:25:53.060737 2014-08-13 18:06:06.672600 7f5c973a57a0 20 read_log 192834'12462 (0'0) promote de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13288 2014-08-11 15:25:53.443742 2014-08-13 18:06:06.672612 7f5c973a57a0 20 read_log 192834'12463 (192834'12462) modify de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by client.933448.0:33376 2014-08-11 15:25:53.415317 2014-08-13 18:06:06.672624 7f5c973a57a0 20 read_log 192834'12472 (0'0) promote 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13298 2014-08-11 15:25:54.453692 2014-08-13 18:06:06.672636 7f5c973a57a0 20 read_log 192834'12473 (192834'12472) modify 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by client.933448.0:33964 2014-08-11 15:25:54.427518 2014-08-13 18:06:06.672648 7f5c973a57a0 20 read_log 192834'12523 (192834'11990) clean cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13349 2014-08-11 15:25:57.912690 2014-08-13 18:06:06.672661 7f5c973a57a0 20 read_log 192834'12530 (192834'12523) delete cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13356 2014-08-11 15:25:57.968613 2014-08-13 18:06:06.672674 7f5c973a57a0 20 read_log 192834'12536 (0'0) promote f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13362 2014-08-11 15:25:58.409647 2014-08-13 18:06:06.672686 7f5c973a57a0 20 read_log 192834'12537 (192834'12536) modify f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by client.933448.0:36869 2014-08-11 15:25:58.384189 2014-08-13 18:06:06.672698 7f5c973a57a0 20 read_log 192834'12556 (0'0) promote 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13382 2014-08-11 15:26:00.462414 2014-08-13 18:06:06.672710 7f5c973a57a0 20 read_log 192834'12557 (192834'12556) modify 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by client.933448.0:38054 2014-08-11 15:26:00.438527 2014-08-13 18:06:06.672722 7f5c973a57a0 20 read_log 192834'12573 (0'0) promote cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13399 2014-08-11 15:26:01.350230 2014-08-13 18:06:06.672734 7f5c973a57a0 20 read_log 192834'12574 (192834'12573) modify cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by client.933448.0:38820 2014-08-11 15:26:01.303761 2014-08-13 18:06:06.672747 7f5c973a57a0 20 read_log 192834'12586 (192834'12027) modify e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by client.933448.0:40079 2014-08-11 15:26:02.274540 2014-08-13 18:06:06.672759 7f5c973a57a0 20 read_log 192834'12605 (192834'12139) modify 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by client.933448.0:42703 2014-08-11 15:26:03.363313 2014-08-13 18:06:06.672771 7f5c973a57a0 20 read_log 192834'12614 (192834'12176) modify 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by client.933448.0:43917 2014-08-11 15:26:03.761254 2014-08-13 18:06:06.672783 7f5c973a57a0 20 read_log 192834'12667 (0'0) promote 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:13493 2014-08-11 15:26:23.572465 2014-08-13 18:06:06.672795 7f5c973a57a0 20 read_log 192834'12668 (192834'12667) modify 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by client.1883634.0:2466 2014-08-11 15:26:23.564624 2014-08-13 18:06:06.672818 7f5c973a57a0 20 read_log 192834'12693 (0'0) promote 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:13519 2014-08-11 15:26:29.356691 2014-08-13 18:06:06.672831 7f5c973a57a0 20 read_log 192834'12694 (192834'12693) modify 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by client.1883634.0:4101 2014-08-11 15:26:29.347366 2014-08-13 18:06:06.672843 7f5c973a57a0 20 read_log 192834'12697 (0'0) promote 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:13523 2014-08-11 15:26:29.914589 2014-08-13 18:06:06.672855 7f5c973a57a0 20 read_log 192834'12698 (192834'12697) modify 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by client.1883634.0:4259 2014-08-11 15:26:29.907247 2014-08-13 18:06:06.672868 7f5c973a57a0 20 read_log 192834'12719 (0'0) promote e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:13545 2014-08-11 15:26:36.067449 2014-08-13 18:06:06.672881 7f5c973a57a0 20 read_log 192834'12720 (192834'12719) modify e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by client.1883634.0:5904 2014-08-11 15:26:36.059974 2014-08-13 18:06:06.672893 7f5c973a57a0 20 read_log 192834'12734 (192834'12245) clean 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13560 2014-08-11 15:26:38.782265 2014-08-13 18:06:06.672906 7f5c973a57a0 20 read_log 192834'12735 (192834'12249) clean 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13561 2014-08-11 15:26:38.782532 2014-08-13 18:06:06.672921 7f5c973a57a0 20 read_log 192834'12736 (0'0) promote 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:13562 2014-08-11 15:26:38.828569 2014-08-13 18:06:06.672933 7f5c973a57a0 20 read_log 192834'12738 (192834'12736) modify 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by client.1883634.0:6546 2014-08-11 15:26:38.661999 2014-08-13 18:06:06.672948 7f5c973a57a0 20 read_log 192834'12742 (192834'12734) delete 5f21ace2/benchmark_data_storage0111-ib.pocstor_26659_object554/head//24 by osd.396.0:13568 2014-08-11 15:26:38.847355 2014-08-13 18:06:06.672960 7f5c973a57a0 20 read_log 192834'12743 (192834'12735) delete 8453ece2/benchmark_data_storage0111-ib.pocstor_26659_object582/head//24 by osd.396.0:13569 2014-08-11 15:26:38.849750 2014-08-13 18:06:06.672973 7f5c973a57a0 20 read_log 192834'12785 (192834'12303) clean 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13611 2014-08-11 15:26:44.121150 2014-08-13 18:06:06.672985 7f5c973a57a0 20 read_log 192834'12786 (192834'12283) clean 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13612 2014-08-11 15:26:44.121373 2014-08-13 18:06:06.672999 7f5c973a57a0 20 read_log 192834'12789 (192834'12289) clean f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13615 2014-08-11 15:26:44.132877 2014-08-13 18:06:06.673012 7f5c973a57a0 20 read_log 192834'12790 (192834'12311) clean 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13616 2014-08-11 15:26:44.138194 2014-08-13 18:06:06.673025 7f5c973a57a0 20 read_log 192834'12795 (192834'12291) clean ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13621 2014-08-11 15:26:44.159677 2014-08-13 18:06:06.673040 7f5c973a57a0 20 read_log 192834'12823 (192834'12785) delete 8f6c20e2/benchmark_data_storage0111-ib.pocstor_26659_object3233/head//24 by osd.396.0:13649 2014-08-11 15:26:44.272782 2014-08-13 18:06:06.673053 7f5c973a57a0 20 read_log 192834'12824 (192834'12786) delete 5a7770e2/benchmark_data_storage0111-ib.pocstor_26659_object2189/head//24 by osd.396.0:13650 2014-08-11 15:26:44.275607 2014-08-13 18:06:06.673067 7f5c973a57a0 20 read_log 192834'12827 (192834'12789) delete f0564e2/benchmark_data_storage0111-ib.pocstor_26659_object2436/head//24 by osd.396.0:13653 2014-08-11 15:26:44.288312 2014-08-13 18:06:06.673087 7f5c973a57a0 20 read_log 192834'12828 (192834'12790) delete 4ff0e4e2/benchmark_data_storage0111-ib.pocstor_26659_object3362/head//24 by osd.396.0:13654 2014-08-11 15:26:44.291609 2014-08-13 18:06:06.673102 7f5c973a57a0 20 read_log 192834'12832 (192834'12309) clean 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13658 2014-08-11 15:26:44.301067 2014-08-13 18:06:06.673114 7f5c973a57a0 20 read_log 192834'12833 (192834'12795) delete ac7668e2/benchmark_data_storage0111-ib.pocstor_26659_object2706/head//24 by osd.396.0:13659 2014-08-11 15:26:44.307789 2014-08-13 18:06:06.673128 7f5c973a57a0 20 read_log 192834'12838 (192834'12832) delete 70872ce2/benchmark_data_storage0111-ib.pocstor_26659_object3348/head//24 by osd.396.0:13664 2014-08-11 15:26:44.324022 2014-08-13 18:06:06.673142 7f5c973a57a0 20 read_log 192834'12843 (0'0) promote 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:13669 2014-08-11 15:26:45.030039 2014-08-13 18:06:06.673166 7f5c973a57a0 20 read_log 192834'12844 (192834'12843) modify 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by client.1883634.0:8120 2014-08-11 15:26:45.021769 2014-08-13 18:06:06.673180 7f5c973a57a0 20 read_log 192834'12871 (192834'12404) clean b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13697 2014-08-11 15:26:49.471035 2014-08-13 18:06:06.673192 7f5c973a57a0 20 read_log 192834'12881 (192834'12398) clean c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13707 2014-08-11 15:26:49.507064 2014-08-13 18:06:06.673207 7f5c973a57a0 20 read_log 192834'12882 (192834'12394) clean 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13708 2014-08-11 15:26:49.512141 2014-08-13 18:06:06.673220 7f5c973a57a0 20 read_log 192834'12899 (192834'12871) delete b6a130e2/benchmark_data_storage0111-ib.pocstor_26659_object8562/head//24 by osd.396.0:13725 2014-08-11 15:26:49.577015 2014-08-13 18:06:06.673234 7f5c973a57a0 20 read_log 192834'12907 (192834'12881) delete c4557ce2/benchmark_data_storage0111-ib.pocstor_26659_object8362/head//24 by osd.396.0:13733 2014-08-11 15:26:49.606447 2014-08-13 18:06:06.673247 7f5c973a57a0 20 read_log 192834'12908 (192834'12882) delete 26f98ce2/benchmark_data_storage0111-ib.pocstor_26659_object8099/head//24 by osd.396.0:13734 2014-08-11 15:26:49.608334 2014-08-13 18:06:06.673259 7f5c973a57a0 20 read_log 192834'12921 (0'0) promote b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:13747 2014-08-11 15:26:53.423263 2014-08-13 18:06:06.673277 7f5c973a57a0 20 read_log 192834'12922 (192834'12921) modify b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by client.1883634.0:10024 2014-08-11 15:26:53.412919 2014-08-13 18:06:06.673290 7f5c973a57a0 20 read_log 192834'12946 (192834'12461) clean 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:13771 2014-08-11 15:26:54.721063 2014-08-13 18:06:06.673304 7f5c973a57a0 20 read_log 192834'12949 (192834'12463) clean de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:13774 2014-08-11 15:26:54.735825 2014-08-13 18:06:06.673317 7f5c973a57a0 20 read_log 192834'12950 (192834'12459) clean dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:13775 2014-08-11 15:26:54.736034 2014-08-13 18:06:06.673329 7f5c973a57a0 20 read_log 192834'12954 (192834'12473) clean 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:13779 2014-08-11 15:26:54.799138 2014-08-13 18:06:06.673345 7f5c973a57a0 20 read_log 192834'12958 (0'0) promote edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:13783 2014-08-11 15:26:57.390145 2014-08-13 18:06:06.673360 7f5c973a57a0 20 read_log 192834'12959 (192834'12958) modify edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by client.1883634.0:10880 2014-08-11 15:26:57.379281 2014-08-13 18:06:06.673413 7f5c973a57a0 20 read_log 192834'12985 (192834'12537) clean f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:13810 2014-08-11 15:26:59.885306 2014-08-13 18:06:06.673431 7f5c973a57a0 20 read_log 192834'13004 (0'0) promote e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:13829 2014-08-11 15:27:03.896579 2014-08-13 18:06:06.673454 7f5c973a57a0 20 read_log 192834'13005 (192834'13004) modify e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by client.1883634.0:12363 2014-08-11 15:27:03.684932 2014-08-13 18:06:06.673468 7f5c973a57a0 20 read_log 192834'13038 (192834'12557) clean 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:13863 2014-08-11 15:27:05.052012 2014-08-13 18:06:06.673480 7f5c973a57a0 20 read_log 192834'13043 (192834'12614) clean 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:13868 2014-08-11 15:27:05.065041 2014-08-13 18:06:06.673503 7f5c973a57a0 20 read_log 192834'13045 (192834'12586) clean e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:13870 2014-08-11 15:27:05.066382 2014-08-13 18:06:06.673523 7f5c973a57a0 20 read_log 192834'13050 (192834'12605) clean 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:13875 2014-08-11 15:27:05.087787 2014-08-13 18:06:06.673542 7f5c973a57a0 20 read_log 192834'13051 (192834'12574) clean cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:13876 2014-08-11 15:27:05.087943 2014-08-13 18:06:06.673558 7f5c973a57a0 20 read_log 192834'13102 (0'0) promote f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:13927 2014-08-11 15:27:15.417551 2014-08-13 18:06:06.673570 7f5c973a57a0 20 read_log 192834'13103 (192834'13102) modify f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by client.1883634.0:14797 2014-08-11 15:27:15.409345 2014-08-13 18:06:06.673583 7f5c973a57a0 20 read_log 192834'13104 (0'0) promote 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:13929 2014-08-11 15:27:16.054483 2014-08-13 18:06:06.673595 7f5c973a57a0 20 read_log 192834'13105 (192834'13104) modify 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by client.1883634.0:14915 2014-08-11 15:27:16.045312 2014-08-13 18:06:06.673607 7f5c973a57a0 20 read_log 192834'13106 (0'0) promote de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:13931 2014-08-11 15:27:17.353349 2014-08-13 18:06:06.673619 7f5c973a57a0 20 read_log 192834'13107 (192834'13106) modify de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by client.1883634.0:15160 2014-08-11 15:27:17.345621 2014-08-13 18:06:06.673633 7f5c973a57a0 20 read_log 192834'13179 (192834'12946) delete 735540e2/benchmark_data_storage0111-ib.pocstor_26659_object10938/head//24 by osd.396.0:14004 2014-08-11 15:27:26.380290 2014-08-13 18:06:06.673646 7f5c973a57a0 20 read_log 192834'13180 (192834'13038) delete 72e890e2/benchmark_data_storage0111-ib.pocstor_26659_object15762/head//24 by osd.396.0:14005 2014-08-11 15:27:26.385049 2014-08-13 18:06:06.673658 7f5c973a57a0 20 read_log 192834'13189 (192834'12985) delete f6d0a4e2/benchmark_data_storage0111-ib.pocstor_26659_object14577/head//24 by osd.396.0:14014 2014-08-11 15:27:26.406977 2014-08-13 18:06:06.673670 7f5c973a57a0 20 read_log 192834'13190 (192834'13043) delete 472bf4e2/benchmark_data_storage0111-ib.pocstor_26659_object21625/head//24 by osd.396.0:14015 2014-08-11 15:27:26.409873 2014-08-13 18:06:06.673683 7f5c973a57a0 20 read_log 192834'13191 (192834'13045) delete e83cf4e2/benchmark_data_storage0111-ib.pocstor_26659_object17787/head//24 by osd.396.0:14016 2014-08-11 15:27:26.412625 2014-08-13 18:06:06.673695 7f5c973a57a0 20 read_log 192834'13199 (192834'12668) clean 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14024 2014-08-11 15:27:26.529200 2014-08-13 18:06:06.673718 7f5c973a57a0 20 read_log 192834'13202 (192834'12949) delete de0608e2/benchmark_data_storage0111-ib.pocstor_26659_object11084/head//24 by osd.396.0:14027 2014-08-11 15:27:26.537119 2014-08-13 18:06:06.673733 7f5c973a57a0 20 read_log 192834'13203 (192834'12950) delete dcf648e2/benchmark_data_storage0111-ib.pocstor_26659_object10775/head//24 by osd.396.0:14028 2014-08-11 15:27:26.539946 2014-08-13 18:06:06.673746 7f5c973a57a0 20 read_log 192834'13204 (192834'13051) delete cee58e2/benchmark_data_storage0111-ib.pocstor_26659_object16528/head//24 by osd.396.0:14029 2014-08-11 15:27:26.540191 2014-08-13 18:06:06.673758 7f5c973a57a0 20 read_log 192834'13205 (192834'13050) delete 6fd7a8e2/benchmark_data_storage0111-ib.pocstor_26659_object20411/head//24 by osd.396.0:14030 2014-08-11 15:27:26.542645 2014-08-13 18:06:06.673770 7f5c973a57a0 20 read_log 192834'13222 (192834'12954) delete 82fd3ce2/benchmark_data_storage0111-ib.pocstor_26659_object11672/head//24 by osd.396.0:14047 2014-08-11 15:27:26.562893 2014-08-13 18:06:06.673802 7f5c973a57a0 20 read_log 192834'13230 (0'0) promote 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14055 2014-08-11 15:27:26.693357 2014-08-13 18:06:06.673818 7f5c973a57a0 20 read_log 192834'13231 (192834'13230) modify 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by client.1883634.0:16254 2014-08-11 15:27:26.644300 2014-08-13 18:06:06.673830 7f5c973a57a0 20 read_log 192834'13280 (192834'13199) delete 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14105 2014-08-11 15:27:26.888992 2014-08-13 18:06:06.673843 7f5c973a57a0 20 read_log 192834'13319 (192834'12694) clean 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14144 2014-08-11 15:27:32.292572 2014-08-13 18:06:06.673855 7f5c973a57a0 20 read_log 192834'13326 (192834'12698) clean 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14151 2014-08-11 15:27:32.336205 2014-08-13 18:06:06.673868 7f5c973a57a0 20 read_log 192834'13327 (192834'13319) delete 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14152 2014-08-11 15:27:32.340983 2014-08-13 18:06:06.673880 7f5c973a57a0 20 read_log 192834'13328 (192834'13326) delete 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14153 2014-08-11 15:27:32.343117 2014-08-13 18:06:06.673892 7f5c973a57a0 20 read_log 192834'13332 (0'0) promote db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14157 2014-08-11 15:27:32.946951 2014-08-13 18:06:06.673904 7f5c973a57a0 20 read_log 192834'13333 (192834'13332) modify db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by client.1883634.0:17185 2014-08-11 15:27:32.935181 2014-08-13 18:06:06.673917 7f5c973a57a0 20 read_log 192834'13345 (192834'12720) clean e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14170 2014-08-11 15:27:37.483313 2014-08-13 18:06:06.673929 7f5c973a57a0 20 read_log 192834'13353 (192834'13345) delete e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14178 2014-08-11 15:27:37.626690 2014-08-13 18:06:06.673941 7f5c973a57a0 20 read_log 192834'13361 (0'0) promote a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:14186 2014-08-11 15:27:39.257397 2014-08-13 18:06:06.673955 7f5c973a57a0 20 read_log 192834'13362 (192834'13361) modify a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by client.1883634.0:17833 2014-08-11 15:27:39.247974 2014-08-13 18:06:06.673968 7f5c973a57a0 20 read_log 192834'13378 (192834'12738) clean 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14203 2014-08-11 15:27:42.840550 2014-08-13 18:06:06.673983 7f5c973a57a0 20 read_log 192834'13383 (0'0) promote 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14208 2014-08-11 15:27:44.900332 2014-08-13 18:06:06.674004 7f5c973a57a0 20 read_log 192834'13384 (192834'13383) modify 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by client.1883634.0:18505 2014-08-11 15:27:44.889797 2014-08-13 18:06:06.674019 7f5c973a57a0 20 read_log 192834'13395 (192834'13378) delete 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14220 2014-08-11 15:27:47.887184 2014-08-13 18:06:06.674032 7f5c973a57a0 20 read_log 192834'13397 (192834'12844) clean 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14222 2014-08-11 15:27:47.952664 2014-08-13 18:06:06.674046 7f5c973a57a0 20 read_log 192834'13403 (192834'13397) delete 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14228 2014-08-11 15:27:47.970500 2014-08-13 18:06:06.674060 7f5c973a57a0 20 read_log 192834'13416 (0'0) promote 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:14241 2014-08-11 15:27:51.415615 2014-08-13 18:06:06.674072 7f5c973a57a0 20 read_log 192834'13417 (192834'13416) modify 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by client.1883634.0:19343 2014-08-11 15:27:51.400874 2014-08-13 18:06:06.674087 7f5c973a57a0 20 read_log 192834'13434 (0'0) promote 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:14259 2014-08-11 15:27:54.933338 2014-08-13 18:06:06.674101 7f5c973a57a0 20 read_log 192834'13435 (192834'13434) modify 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by client.1883634.0:20008 2014-08-11 15:27:54.926814 2014-08-13 18:06:06.674113 7f5c973a57a0 20 read_log 192834'13449 (192834'12959) clean edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14274 2014-08-11 15:27:58.362391 2014-08-13 18:06:06.674128 7f5c973a57a0 20 read_log 192834'13451 (192834'12922) clean b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14276 2014-08-11 15:27:58.375850 2014-08-13 18:06:06.674154 7f5c973a57a0 20 read_log 192834'13458 (192834'13449) delete edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14283 2014-08-11 15:28:03.326806 2014-08-13 18:06:06.674169 7f5c973a57a0 20 read_log 192834'13463 (192834'13451) delete b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14288 2014-08-11 15:28:03.439580 2014-08-13 18:06:06.674182 7f5c973a57a0 20 read_log 192834'13478 (0'0) promote db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:14303 2014-08-11 15:28:06.962276 2014-08-13 18:06:06.674196 7f5c973a57a0 20 read_log 192834'13479 (192834'13478) modify db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by client.1883634.0:22159 2014-08-11 15:28:06.953090 2014-08-13 18:06:06.674208 7f5c973a57a0 20 read_log 192834'13501 (192834'13005) clean e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14326 2014-08-11 15:28:08.749010 2014-08-13 18:06:06.674223 7f5c973a57a0 20 read_log 192834'13514 (0'0) promote cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:14339 2014-08-11 15:28:12.845985 2014-08-13 18:06:06.674235 7f5c973a57a0 20 read_log 192834'13515 (192834'13514) modify cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by client.1883634.0:23125 2014-08-11 15:28:12.835770 2014-08-13 18:06:06.674249 7f5c973a57a0 20 read_log 192834'13523 (192834'13501) delete e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14348 2014-08-11 15:28:13.937216 2014-08-13 18:06:06.674265 7f5c973a57a0 20 read_log 192834'13564 (192834'13105) clean 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14389 2014-08-11 15:28:19.162060 2014-08-13 18:06:06.674279 7f5c973a57a0 20 read_log 192834'13568 (192834'13564) delete 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14393 2014-08-11 15:28:19.178197 2014-08-13 18:06:06.674299 7f5c973a57a0 20 read_log 192834'13569 (192834'13107) clean de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14394 2014-08-11 15:28:19.189537 2014-08-13 18:06:06.674314 7f5c973a57a0 20 read_log 192834'13570 (192834'13103) clean f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14395 2014-08-11 15:28:19.199741 2014-08-13 18:06:06.674329 7f5c973a57a0 20 read_log 192834'13599 (0'0) promote 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14424 2014-08-11 15:28:21.901978 2014-08-13 18:06:06.674341 7f5c973a57a0 20 read_log 192834'13600 (192834'13599) modify 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by client.1883634.0:25790 2014-08-11 15:28:21.881267 2014-08-13 18:06:06.674357 7f5c973a57a0 20 read_log 192834'13622 (192834'13569) delete de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14447 2014-08-11 15:28:24.308379 2014-08-13 18:06:06.674369 7f5c973a57a0 20 read_log 192834'13623 (192834'13570) delete f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14448 2014-08-11 15:28:24.310555 2014-08-13 18:06:06.674390 7f5c973a57a0 20 read_log 192834'13635 (0'0) promote 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14460 2014-08-11 15:28:24.488334 2014-08-13 18:06:06.674407 7f5c973a57a0 20 read_log 192834'13636 (192834'13635) modify 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by client.1883634.0:27425 2014-08-11 15:28:24.459483 2014-08-13 18:06:06.674420 7f5c973a57a0 20 read_log 192834'13639 (0'0) promote 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14464 2014-08-11 15:28:24.676334 2014-08-13 18:06:06.674448 7f5c973a57a0 20 read_log 192834'13640 (192834'13639) modify 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by client.1883634.0:27583 2014-08-11 15:28:24.644949 2014-08-13 18:06:06.674465 7f5c973a57a0 20 read_log 192834'13661 (0'0) promote e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14486 2014-08-11 15:28:26.899026 2014-08-13 18:06:06.674478 7f5c973a57a0 20 read_log 192834'13662 (192834'13661) modify e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by client.1883634.0:29228 2014-08-11 15:28:26.873620 2014-08-13 18:06:06.674500 7f5c973a57a0 20 read_log 192834'13665 (0'0) promote 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14490 2014-08-11 15:28:27.808809 2014-08-13 18:06:06.674513 7f5c973a57a0 20 read_log 192834'13666 (192834'13665) modify 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by client.1883634.0:29870 2014-08-11 15:28:27.774317 2014-08-13 18:06:06.674526 7f5c973a57a0 20 read_log 192834'13681 (192834'13231) clean 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14506 2014-08-11 15:28:29.558507 2014-08-13 18:06:06.674549 7f5c973a57a0 20 read_log 192834'13687 (192834'13681) delete 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14512 2014-08-11 15:28:29.583201 2014-08-13 18:06:06.674562 7f5c973a57a0 20 read_log 192834'13700 (0'0) promote 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14525 2014-08-11 15:28:30.548722 2014-08-13 18:06:06.674574 7f5c973a57a0 20 read_log 192834'13701 (192834'13700) modify 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by client.1883634.0:31444 2014-08-11 15:28:30.519282 2014-08-13 18:06:06.674587 7f5c973a57a0 20 read_log 192834'13722 (0'0) promote b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14547 2014-08-11 15:28:33.071199 2014-08-13 18:06:06.674604 7f5c973a57a0 20 read_log 192834'13723 (192834'13722) modify b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by client.1883634.0:33348 2014-08-11 15:28:33.043002 2014-08-13 18:06:06.674636 7f5c973a57a0 20 read_log 192834'13732 (0'0) promote edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14557 2014-08-11 15:28:34.387218 2014-08-13 18:06:06.674652 7f5c973a57a0 20 read_log 192834'13733 (192834'13732) modify edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by client.1883634.0:34204 2014-08-11 15:28:34.359742 2014-08-13 18:06:06.674665 7f5c973a57a0 20 read_log 192834'13740 (192834'13333) clean db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14565 2014-08-11 15:28:34.715826 2014-08-13 18:06:06.674678 7f5c973a57a0 20 read_log 192834'13743 (192834'13740) delete db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14568 2014-08-11 15:28:34.730358 2014-08-13 18:06:06.674690 7f5c973a57a0 20 read_log 192834'13760 (0'0) promote e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14585 2014-08-11 15:28:37.027270 2014-08-13 18:06:06.674702 7f5c973a57a0 20 read_log 192834'13761 (192834'13760) modify e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by client.1883634.0:35687 2014-08-11 15:28:37.002594 2014-08-13 18:06:06.674714 7f5c973a57a0 20 read_log 192834'13808 (192834'13362) clean a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:14633 2014-08-11 15:28:39.988510 2014-08-13 18:06:06.674726 7f5c973a57a0 20 read_log 192834'13813 (0'0) promote f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14638 2014-08-11 15:28:40.270373 2014-08-13 18:06:06.674750 7f5c973a57a0 20 read_log 192834'13814 (192834'13813) modify f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by client.1883634.0:38121 2014-08-11 15:28:40.241382 2014-08-13 18:06:06.674766 7f5c973a57a0 20 read_log 192834'13815 (0'0) promote 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14640 2014-08-11 15:28:40.423257 2014-08-13 18:06:06.674779 7f5c973a57a0 20 read_log 192834'13816 (192834'13815) modify 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by client.1883634.0:38239 2014-08-11 15:28:40.399029 2014-08-13 18:06:06.674792 7f5c973a57a0 20 read_log 192834'13817 (0'0) promote de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14642 2014-08-11 15:28:40.726272 2014-08-13 18:06:06.674804 7f5c973a57a0 20 read_log 192834'13818 (192834'13817) modify de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by client.1883634.0:38484 2014-08-11 15:28:40.693349 2014-08-13 18:06:06.674816 7f5c973a57a0 20 read_log 192834'13831 (0'0) promote 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14656 2014-08-11 15:28:42.354107 2014-08-13 18:06:06.674828 7f5c973a57a0 20 read_log 192834'13832 (192834'13831) modify 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by client.1883634.0:39578 2014-08-11 15:28:42.323350 2014-08-13 18:06:06.674841 7f5c973a57a0 20 read_log 192834'13853 (0'0) promote db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14678 2014-08-11 15:28:43.887974 2014-08-13 18:06:06.674853 7f5c973a57a0 20 read_log 192834'13854 (192834'13853) modify db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by client.1883634.0:40509 2014-08-11 15:28:43.852982 2014-08-13 18:06:06.674865 7f5c973a57a0 20 read_log 192834'13862 (192834'13808) modify a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by client.1883634.0:41157 2014-08-11 15:28:44.696809 2014-08-13 18:06:06.674877 7f5c973a57a0 20 read_log 192834'13868 (192834'13384) clean 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14693 2014-08-11 15:28:45.133723 2014-08-13 18:06:06.674911 7f5c973a57a0 20 read_log 192834'13871 (192834'13868) delete 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14696 2014-08-11 15:28:45.175179 2014-08-13 18:06:06.674964 7f5c973a57a0 20 read_log 192834'13878 (0'0) promote 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14703 2014-08-11 15:28:45.545404 2014-08-13 18:06:06.674981 7f5c973a57a0 20 read_log 192834'13879 (192834'13878) modify 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by client.1883634.0:41829 2014-08-11 15:28:45.523032 2014-08-13 18:06:06.674994 7f5c973a57a0 20 read_log 192834'13887 (192834'13417) modify 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by client.1883634.0:42667 2014-08-11 15:28:46.317169 2014-08-13 18:06:06.675006 7f5c973a57a0 20 read_log 192834'13890 (192834'13435) modify 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by client.1883634.0:43332 2014-08-11 15:28:46.522659 2014-08-13 18:06:06.675019 7f5c973a57a0 20 read_log 192834'13899 (192834'13479) modify db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by client.1883634.0:45483 2014-08-11 15:28:47.292523 2014-08-13 18:06:06.675031 7f5c973a57a0 20 read_log 192834'13907 (192834'13515) modify cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by client.1883634.0:46449 2014-08-11 15:28:47.596292 2014-08-13 18:06:06.675043 7f5c973a57a0 20 read_log 192834'13957 (192834'13600) clean 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14782 2014-08-11 15:29:25.765155 2014-08-13 18:06:06.675056 7f5c973a57a0 20 read_log 192834'13963 (192834'13636) clean 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14788 2014-08-11 15:29:25.786234 2014-08-13 18:06:06.675068 7f5c973a57a0 20 read_log 192834'13964 (192834'13640) clean 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14789 2014-08-11 15:29:25.791380 2014-08-13 18:06:06.675080 7f5c973a57a0 20 read_log 192834'13985 (192834'13957) delete 8ae950e2/benchmark_data_storage0111-ib.pocstor_41526_object2465/head//24 by osd.396.0:14810 2014-08-11 15:29:25.836936 2014-08-13 18:06:06.675092 7f5c973a57a0 20 read_log 192834'13991 (192834'13963) delete 8a012ce2/benchmark_data_storage0111-ib.pocstor_41526_object4100/head//24 by osd.396.0:14816 2014-08-11 15:29:25.854789 2014-08-13 18:06:06.675104 7f5c973a57a0 20 read_log 192834'13992 (192834'13964) delete 3cd07ce2/benchmark_data_storage0111-ib.pocstor_41526_object4258/head//24 by osd.396.0:14817 2014-08-11 15:29:25.856470 2014-08-13 18:06:06.675117 7f5c973a57a0 20 read_log 192834'14004 (192834'13701) clean 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14829 2014-08-11 15:29:30.921610 2014-08-13 18:06:06.675129 7f5c973a57a0 20 read_log 192834'14009 (192834'13666) clean 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14834 2014-08-11 15:29:30.937720 2014-08-13 18:06:06.675141 7f5c973a57a0 20 read_log 192834'14010 (192834'13662) clean e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14835 2014-08-11 15:29:30.943217 2014-08-13 18:06:06.675162 7f5c973a57a0 20 read_log 192834'14020 (192834'14004) delete 8ee9f0e2/benchmark_data_storage0111-ib.pocstor_41526_object8119/head//24 by osd.396.0:14845 2014-08-11 15:29:30.967513 2014-08-13 18:06:06.675175 7f5c973a57a0 20 read_log 192834'14025 (192834'14009) delete 5c4088e2/benchmark_data_storage0111-ib.pocstor_41526_object6545/head//24 by osd.396.0:14850 2014-08-11 15:29:30.977244 2014-08-13 18:06:06.675187 7f5c973a57a0 20 read_log 192834'14026 (192834'14010) delete e4d6e8e2/benchmark_data_storage0111-ib.pocstor_41526_object5903/head//24 by osd.396.0:14851 2014-08-11 15:29:30.978731 2014-08-13 18:06:06.675244 7f5c973a57a0 20 read_log 192834'14044 (192834'13733) clean edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14869 2014-08-11 15:29:36.050181 2014-08-13 18:06:06.675264 7f5c973a57a0 20 read_log 192834'14050 (192834'13723) clean b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14875 2014-08-11 15:29:36.067538 2014-08-13 18:06:06.675284 7f5c973a57a0 20 read_log 192834'14064 (192834'14044) delete edeff0e2/benchmark_data_storage0111-ib.pocstor_41526_object10879/head//24 by osd.396.0:14889 2014-08-11 15:29:36.091967 2014-08-13 18:06:06.675303 7f5c973a57a0 20 read_log 192834'14069 (192834'14050) delete b847cce2/benchmark_data_storage0111-ib.pocstor_41526_object10023/head//24 by osd.396.0:14894 2014-08-11 15:29:36.101936 2014-08-13 18:06:06.675322 7f5c973a57a0 20 read_log 192834'14091 (192834'13761) clean e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14916 2014-08-11 15:29:41.186439 2014-08-13 18:06:06.675335 7f5c973a57a0 20 read_log 192834'14092 (192834'13816) clean 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14917 2014-08-11 15:29:41.189771 2014-08-13 18:06:06.675348 7f5c973a57a0 20 read_log 192834'14099 (192834'13818) clean de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14924 2014-08-11 15:29:41.208156 2014-08-13 18:06:06.675366 7f5c973a57a0 20 read_log 192834'14100 (192834'13814) clean f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14925 2014-08-11 15:29:41.208354 2014-08-13 18:06:06.675379 7f5c973a57a0 20 read_log 192834'14122 (192834'14091) delete e46c14e2/benchmark_data_storage0111-ib.pocstor_41526_object12362/head//24 by osd.396.0:14947 2014-08-11 15:29:41.237411 2014-08-13 18:06:06.675401 7f5c973a57a0 20 read_log 192834'14123 (192834'14092) delete 7492f8e2/benchmark_data_storage0111-ib.pocstor_41526_object14914/head//24 by osd.396.0:14948 2014-08-11 15:29:41.238847 2014-08-13 18:06:06.675419 7f5c973a57a0 20 read_log 192834'14128 (192834'14099) delete de64ece2/benchmark_data_storage0111-ib.pocstor_41526_object15159/head//24 by osd.396.0:14953 2014-08-11 15:29:41.244491 2014-08-13 18:06:06.675432 7f5c973a57a0 20 read_log 192834'14129 (192834'14100) delete f748fce2/benchmark_data_storage0111-ib.pocstor_41526_object14796/head//24 by osd.396.0:14954 2014-08-11 15:29:41.245520 2014-08-13 18:06:06.675457 7f5c973a57a0 20 read_log 192834'14147 (192834'13854) clean db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14972 2014-08-11 15:29:46.293628 2014-08-13 18:06:06.675470 7f5c973a57a0 20 read_log 192834'14148 (192834'13879) clean 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14973 2014-08-11 15:29:46.294164 2014-08-13 18:06:06.675482 7f5c973a57a0 20 read_log 192834'14166 (192834'14147) delete db7960e2/benchmark_data_storage0111-ib.pocstor_41526_object17184/head//24 by osd.396.0:14991 2014-08-11 15:29:46.320026 2014-08-13 18:06:06.675497 7f5c973a57a0 20 read_log 192834'14167 (192834'14148) delete 6d49e0e2/benchmark_data_storage0111-ib.pocstor_41526_object18504/head//24 by osd.396.0:14992 2014-08-11 15:29:46.320906 2014-08-13 18:06:06.675509 7f5c973a57a0 20 read_log 192834'14172 (192834'13832) clean 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:14997 2014-08-11 15:29:46.325109 2014-08-13 18:06:06.675522 7f5c973a57a0 20 read_log 192834'14179 (192834'13862) clean a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:15004 2014-08-11 15:29:46.335057 2014-08-13 18:06:06.675534 7f5c973a57a0 20 read_log 192834'14184 (192834'14172) delete 447e4e2/benchmark_data_storage0111-ib.pocstor_41526_object16253/head//24 by osd.396.0:15009 2014-08-11 15:29:46.344826 2014-08-13 18:06:06.675554 7f5c973a57a0 20 read_log 192834'14190 (192834'14179) delete a2d8e8e2/benchmark_data_storage0111-ib.pocstor_41526_object17832/head//24 by osd.396.0:15015 2014-08-11 15:29:46.349471 2014-08-13 18:06:06.675570 7f5c973a57a0 20 read_log 192834'14194 (192834'13887) clean 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:15019 2014-08-11 15:29:46.352122 2014-08-13 18:06:06.675589 7f5c973a57a0 20 read_log 192834'14199 (192834'14194) delete 41a73ce2/benchmark_data_storage0111-ib.pocstor_41526_object19342/head//24 by osd.396.0:15024 2014-08-11 15:29:46.354349 2014-08-13 18:06:06.675637 7f5c973a57a0 20 read_log 192834'14222 (192834'13890) clean 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:15047 2014-08-11 15:29:51.378668 2014-08-13 18:06:06.675655 7f5c973a57a0 20 read_log 192834'14226 (192834'14222) delete 70bd0e2/benchmark_data_storage0111-ib.pocstor_41526_object20007/head//24 by osd.396.0:15051 2014-08-11 15:29:51.382346 2014-08-13 18:06:06.675668 7f5c973a57a0 20 read_log 192834'14230 (192834'13907) clean cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:15055 2014-08-11 15:29:51.384125 2014-08-13 18:06:06.675683 7f5c973a57a0 20 read_log 192834'14234 (192834'14230) delete cbbf8e2/benchmark_data_storage0111-ib.pocstor_41526_object23124/head//24 by osd.396.0:15059 2014-08-11 15:29:51.387927 2014-08-13 18:06:06.675698 7f5c973a57a0 20 read_log 192834'14238 (192834'13899) clean db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:15063 2014-08-11 15:29:51.389911 2014-08-13 18:06:06.675711 7f5c973a57a0 20 read_log 192834'14239 (192834'14238) delete db489ce2/benchmark_data_storage0111-ib.pocstor_41526_object22158/head//24 by osd.396.0:15064 2014-08-11 15:29:56.390254 2014-08-13 18:06:06.675732 7f5c973a57a0 20 read_log 192888'14240 (192888'14240) modify e2/hit_set_24.e2_archive_2014-08-11 16:02:27.891633_2014-08-11 16:44:58.212047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.212323 2014-08-13 18:06:06.675748 7f5c973a57a0 20 read_log 192888'14241 (0'0) promote 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by osd.346.0:15307 2014-08-11 16:44:58.220761 2014-08-13 18:06:06.675760 7f5c973a57a0 20 read_log 192888'14242 (192888'14241) modify 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.1315259.0:772 2014-08-11 16:44:58.208320 2014-08-13 18:06:06.675773 7f5c973a57a0 20 read_log 192888'14243 (0'0) promote 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by osd.346.0:15321 2014-08-11 16:45:01.382128 2014-08-13 18:06:06.675785 7f5c973a57a0 20 read_log 192888'14244 (192888'14243) modify 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.1315259.0:1597 2014-08-11 16:45:01.375256 2014-08-13 18:06:06.675798 7f5c973a57a0 20 read_log 192888'14249 (0'0) promote e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by osd.346.0:15341 2014-08-11 16:45:07.924623 2014-08-13 18:06:06.675810 7f5c973a57a0 20 read_log 192888'14250 (192888'14249) modify e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.1315259.0:3320 2014-08-11 16:45:07.916878 2014-08-13 18:06:06.675822 7f5c973a57a0 20 read_log 192888'14253 (0'0) promote ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by osd.346.0:15347 2014-08-11 16:45:09.148299 2014-08-13 18:06:06.675834 7f5c973a57a0 20 read_log 192888'14254 (192888'14253) modify ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.1315259.0:3645 2014-08-11 16:45:09.140922 2014-08-13 18:06:06.675847 7f5c973a57a0 20 read_log 192888'14259 (0'0) promote 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by osd.346.0:15357 2014-08-11 16:45:13.563006 2014-08-13 18:06:06.675859 7f5c973a57a0 20 read_log 192888'14260 (192888'14259) modify 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.1315259.0:4854 2014-08-11 16:45:13.555397 2014-08-13 18:06:06.675871 7f5c973a57a0 20 read_log 192888'14265 (0'0) promote 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by osd.346.0:15377 2014-08-11 16:45:22.666173 2014-08-13 18:06:06.675883 7f5c973a57a0 20 read_log 192888'14266 (192888'14265) modify 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.1315259.0:7350 2014-08-11 16:45:22.657483 2014-08-13 18:06:06.675905 7f5c973a57a0 20 read_log 192888'14275 (0'0) promote 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by osd.346.0:15405 2014-08-11 16:45:33.022348 2014-08-13 18:06:06.675921 7f5c973a57a0 20 read_log 192888'14276 (192888'14275) modify 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.1315259.0:10101 2014-08-11 16:45:33.014965 2014-08-13 18:06:06.675934 7f5c973a57a0 20 read_log 192888'14279 (0'0) promote 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by osd.346.0:15409 2014-08-11 16:45:33.833103 2014-08-13 18:06:06.675946 7f5c973a57a0 20 read_log 192888'14280 (192888'14279) modify 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.1315259.0:10313 2014-08-11 16:45:33.823651 2014-08-13 18:06:06.675968 7f5c973a57a0 20 read_log 192888'14281 (0'0) promote 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by osd.346.0:15421 2014-08-11 16:45:36.493103 2014-08-13 18:06:06.675983 7f5c973a57a0 20 read_log 192888'14282 (192888'14281) modify 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.1315259.0:10969 2014-08-11 16:45:36.486433 2014-08-13 18:06:06.675996 7f5c973a57a0 20 read_log 192888'14287 (0'0) promote f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by osd.346.0:15445 2014-08-11 16:45:41.424337 2014-08-13 18:06:06.676008 7f5c973a57a0 20 read_log 192888'14288 (192888'14287) modify f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.1315259.0:12189 2014-08-11 16:45:41.416352 2014-08-13 18:06:06.676020 7f5c973a57a0 20 read_log 192888'14289 (0'0) promote f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by osd.346.0:15451 2014-08-11 16:45:45.526859 2014-08-13 18:06:06.676035 7f5c973a57a0 20 read_log 192888'14290 (192888'14289) modify f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.1315259.0:13170 2014-08-11 16:45:45.519395 2014-08-13 18:06:06.676048 7f5c973a57a0 20 read_log 192888'14293 (0'0) promote 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by osd.346.0:15473 2014-08-11 16:45:51.146361 2014-08-13 18:06:06.676060 7f5c973a57a0 20 read_log 192888'14294 (192888'14293) modify 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.1315259.0:14624 2014-08-11 16:45:51.139475 2014-08-13 18:06:06.676072 7f5c973a57a0 20 read_log 192888'14297 (0'0) promote adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by osd.346.0:15479 2014-08-11 16:45:52.323637 2014-08-13 18:06:06.676087 7f5c973a57a0 20 read_log 192888'14298 (192888'14297) modify adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.1315259.0:14940 2014-08-11 16:45:52.317241 2014-08-13 18:06:06.676100 7f5c973a57a0 20 read_log 192888'14299 (0'0) promote 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by osd.346.0:15485 2014-08-11 16:45:54.116869 2014-08-13 18:06:06.676112 7f5c973a57a0 20 read_log 192888'14300 (192888'14299) modify 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.1315259.0:15397 2014-08-11 16:45:54.109177 2014-08-13 18:06:06.676124 7f5c973a57a0 20 read_log 192888'14301 (0'0) promote d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by osd.346.0:15497 2014-08-11 16:46:00.547141 2014-08-13 18:06:06.676140 7f5c973a57a0 20 read_log 192888'14302 (192888'14301) modify d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.1315259.0:17045 2014-08-11 16:46:00.539501 2014-08-13 18:06:06.676170 7f5c973a57a0 20 read_log 192888'14303 (0'0) promote 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by osd.346.0:15501 2014-08-11 16:46:01.643980 2014-08-13 18:06:06.676186 7f5c973a57a0 20 read_log 192888'14304 (192888'14303) modify 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.1315259.0:17333 2014-08-11 16:46:01.635925 2014-08-13 18:06:06.676215 7f5c973a57a0 20 read_log 192888'14309 (0'0) promote 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by osd.346.0:15527 2014-08-11 16:46:12.187606 2014-08-13 18:06:06.676232 7f5c973a57a0 20 read_log 192888'14310 (192888'14309) modify 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.1315259.0:20017 2014-08-11 16:46:12.179805 2014-08-13 18:06:06.676252 7f5c973a57a0 20 read_log 192888'14311 (0'0) promote 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by osd.346.0:15529 2014-08-11 16:46:12.493679 2014-08-13 18:06:06.676269 7f5c973a57a0 20 read_log 192888'14312 (192888'14311) modify 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.1315259.0:20102 2014-08-11 16:46:12.485655 2014-08-13 18:06:06.676281 7f5c973a57a0 20 read_log 192888'14313 (0'0) promote 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by osd.346.0:15537 2014-08-11 16:46:14.218231 2014-08-13 18:06:06.676294 7f5c973a57a0 20 read_log 192888'14314 (192888'14313) modify 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.1315259.0:20551 2014-08-11 16:46:14.210052 2014-08-13 18:06:06.676306 7f5c973a57a0 20 read_log 192888'14317 (0'0) promote 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by osd.346.0:15551 2014-08-11 16:46:16.565124 2014-08-13 18:06:06.676318 7f5c973a57a0 20 read_log 192888'14318 (192888'14317) modify 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.1315259.0:21185 2014-08-11 16:46:16.558657 2014-08-13 18:06:06.676331 7f5c973a57a0 20 read_log 192888'14321 (0'0) promote 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by osd.346.0:15577 2014-08-11 16:46:22.099091 2014-08-13 18:06:06.676344 7f5c973a57a0 20 read_log 192888'14322 (192888'14321) modify 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.1315259.0:22613 2014-08-11 16:46:22.091347 2014-08-13 18:06:06.676359 7f5c973a57a0 20 read_log 192888'14323 (0'0) promote 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by osd.346.0:15579 2014-08-11 16:46:22.230226 2014-08-13 18:06:06.676372 7f5c973a57a0 20 read_log 192888'14324 (192888'14323) modify 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.1315259.0:22648 2014-08-11 16:46:22.223382 2014-08-13 18:06:06.676384 7f5c973a57a0 20 read_log 192888'14325 (0'0) promote 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by osd.346.0:15589 2014-08-11 16:46:25.340402 2014-08-13 18:06:06.676397 7f5c973a57a0 20 read_log 192888'14326 (192888'14325) modify 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.1315259.0:23433 2014-08-11 16:46:25.332969 2014-08-13 18:06:06.676409 7f5c973a57a0 20 read_log 192888'14327 (0'0) promote 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by osd.346.0:15593 2014-08-11 16:46:26.964728 2014-08-13 18:06:06.676421 7f5c973a57a0 20 read_log 192888'14328 (192888'14327) modify 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.1315259.0:23824 2014-08-11 16:46:26.958339 2014-08-13 18:06:06.676434 7f5c973a57a0 20 read_log 192888'14329 (0'0) promote 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by osd.346.0:15603 2014-08-11 16:46:28.923960 2014-08-13 18:06:06.676446 7f5c973a57a0 20 read_log 192888'14330 (192888'14329) modify 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.1315259.0:24343 2014-08-11 16:46:28.916433 2014-08-13 18:06:06.676459 7f5c973a57a0 20 read_log 192888'14343 (0'0) promote 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by osd.346.0:15641 2014-08-11 16:46:38.921893 2014-08-13 18:06:06.676472 7f5c973a57a0 20 read_log 192888'14344 (192888'14343) modify 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.1315259.0:26763 2014-08-11 16:46:38.913145 2014-08-13 18:06:06.676492 7f5c973a57a0 20 read_log 192888'14345 (0'0) promote 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by osd.346.0:15649 2014-08-11 16:46:43.418819 2014-08-13 18:06:06.676505 7f5c973a57a0 20 read_log 192888'14346 (192888'14345) modify 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.1315259.0:27861 2014-08-11 16:46:43.411757 2014-08-13 18:06:06.676517 7f5c973a57a0 20 read_log 192888'14353 (0'0) promote e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by osd.346.0:15663 2014-08-11 16:46:47.791083 2014-08-13 18:06:06.676529 7f5c973a57a0 20 read_log 192888'14354 (192888'14353) modify e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.1315259.0:28939 2014-08-11 16:46:47.783903 2014-08-13 18:06:06.676541 7f5c973a57a0 20 read_log 192888'14355 (0'0) promote e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by osd.346.0:15665 2014-08-11 16:46:48.098995 2014-08-13 18:06:06.676553 7f5c973a57a0 20 read_log 192888'14356 (192888'14355) modify e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.1315259.0:29020 2014-08-11 16:46:48.091124 2014-08-13 18:06:06.676566 7f5c973a57a0 20 read_log 192888'14357 (0'0) promote be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by osd.346.0:15667 2014-08-11 16:46:48.535653 2014-08-13 18:06:06.676578 7f5c973a57a0 20 read_log 192888'14358 (192888'14357) modify be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.1315259.0:29135 2014-08-11 16:46:48.528348 2014-08-13 18:06:06.676590 7f5c973a57a0 20 read_log 192888'14359 (0'0) promote 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by osd.346.0:15671 2014-08-11 16:46:51.597242 2014-08-13 18:06:06.676602 7f5c973a57a0 20 read_log 192888'14360 (192888'14359) modify 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.1315259.0:29861 2014-08-11 16:46:51.590119 2014-08-13 18:06:06.676614 7f5c973a57a0 20 read_log 192888'14363 (192888'14242) modify 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.1315259.0:31499 2014-08-11 16:46:56.112604 2014-08-13 18:06:06.676626 7f5c973a57a0 20 read_log 192888'14364 (192888'14244) modify 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.1315259.0:32324 2014-08-11 16:46:56.347881 2014-08-13 18:06:06.676641 7f5c973a57a0 20 read_log 192888'14367 (192888'14250) modify e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.1315259.0:34047 2014-08-11 16:46:56.791363 2014-08-13 18:06:06.676656 7f5c973a57a0 20 read_log 192888'14369 (192888'14254) modify ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.1315259.0:34372 2014-08-11 16:46:56.876738 2014-08-13 18:06:06.676669 7f5c973a57a0 20 read_log 192888'14372 (192888'14260) modify 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.1315259.0:35581 2014-08-11 16:46:57.195973 2014-08-13 18:06:06.676684 7f5c973a57a0 20 read_log 192888'14375 (192888'14266) modify 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.1315259.0:38077 2014-08-11 16:46:57.851185 2014-08-13 18:06:06.676699 7f5c973a57a0 20 read_log 192888'14380 (192888'14276) modify 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.1315259.0:40828 2014-08-11 16:46:58.591115 2014-08-13 18:06:06.676711 7f5c973a57a0 20 read_log 192888'14382 (192888'14280) modify 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.1315259.0:41040 2014-08-11 16:46:58.643844 2014-08-13 18:06:06.676745 7f5c973a57a0 20 read_log 192888'14383 (192888'14282) modify 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.1315259.0:41696 2014-08-11 16:46:58.812860 2014-08-13 18:06:06.676761 7f5c973a57a0 20 read_log 192888'14386 (192888'14288) modify f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.1315259.0:42916 2014-08-11 16:46:59.140411 2014-08-13 18:06:06.676785 7f5c973a57a0 20 read_log 192888'14387 (192888'14290) modify f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.1315259.0:43897 2014-08-11 16:46:59.421663 2014-08-13 18:06:06.676800 7f5c973a57a0 20 read_log 192888'14389 (192888'14294) modify 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.1315259.0:45351 2014-08-11 16:46:59.844300 2014-08-13 18:06:06.676813 7f5c973a57a0 20 read_log 192888'14391 (192888'14298) modify adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.1315259.0:45667 2014-08-11 16:46:59.936500 2014-08-13 18:06:06.676827 7f5c973a57a0 20 read_log 192888'14392 (192888'14300) modify 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.1315259.0:46124 2014-08-11 16:47:00.066566 2014-08-13 18:06:06.676840 7f5c973a57a0 20 read_log 192888'14393 (192888'14302) modify d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.1315259.0:47772 2014-08-11 16:47:00.561556 2014-08-13 18:06:06.676854 7f5c973a57a0 20 read_log 192888'14394 (192888'14304) modify 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.1315259.0:48060 2014-08-11 16:47:00.657103 2014-08-13 18:06:06.676866 7f5c973a57a0 20 read_log 192888'14397 (192888'14310) modify 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.1315259.0:50744 2014-08-11 16:47:01.463338 2014-08-13 18:06:06.676879 7f5c973a57a0 20 read_log 192888'14398 (192888'14312) modify 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.1315259.0:50829 2014-08-11 16:47:01.492323 2014-08-13 18:06:06.676891 7f5c973a57a0 20 read_log 192888'14399 (192888'14314) modify 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.1315259.0:51278 2014-08-11 16:47:01.632857 2014-08-13 18:06:06.676906 7f5c973a57a0 20 read_log 192888'14401 (192888'14318) modify 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.1315259.0:51912 2014-08-11 16:47:01.818802 2014-08-13 18:06:06.676918 7f5c973a57a0 20 read_log 192888'14403 (192888'14322) modify 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.1315259.0:53340 2014-08-11 16:47:02.224993 2014-08-13 18:06:06.676932 7f5c973a57a0 20 read_log 192888'14404 (192888'14324) modify 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.1315259.0:53375 2014-08-11 16:47:02.236239 2014-08-13 18:06:06.676945 7f5c973a57a0 20 read_log 192888'14405 (192888'14326) modify 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.1315259.0:54160 2014-08-11 16:47:02.484343 2014-08-13 18:06:06.676959 7f5c973a57a0 20 read_log 192888'14406 (192888'14328) modify 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.1315259.0:54551 2014-08-11 16:47:02.598721 2014-08-13 18:06:06.676972 7f5c973a57a0 20 read_log 192888'14407 (192888'14330) modify 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.1315259.0:55070 2014-08-11 16:47:02.802046 2014-08-13 18:06:06.676986 7f5c973a57a0 20 read_log 192888'14414 (192888'14344) modify 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.1315259.0:57490 2014-08-11 16:47:03.528529 2014-08-13 18:06:06.676998 7f5c973a57a0 20 read_log 192888'14415 (192888'14346) modify 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.1315259.0:58588 2014-08-11 16:47:03.847109 2014-08-13 18:06:06.677012 7f5c973a57a0 20 read_log 192888'14419 (192888'14354) modify e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.1315259.0:59666 2014-08-11 16:47:04.200732 2014-08-13 18:06:06.677025 7f5c973a57a0 20 read_log 192888'14420 (192888'14356) modify e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.1315259.0:59747 2014-08-11 16:47:04.229343 2014-08-13 18:06:06.677039 7f5c973a57a0 20 read_log 192888'14421 (192888'14358) modify be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.1315259.0:59862 2014-08-11 16:47:04.264084 2014-08-13 18:06:06.677058 7f5c973a57a0 20 read_log 192888'14422 (192888'14360) modify 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.1315259.0:60588 2014-08-11 16:47:04.484567 2014-08-13 18:06:06.677073 7f5c973a57a0 20 read_log 192888'14424 (192888'14424) modify e2/hit_set_24.e2_archive_2014-08-11 16:44:58.212327_2014-08-11 17:05:19.923836/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:19.924066 2014-08-13 18:06:06.677086 7f5c973a57a0 20 read_log 192888'14425 (192888'14240) delete e2/hit_set_24.e2_archive_2014-08-11 16:02:27.891633_2014-08-11 16:44:58.212047/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:19.924066 2014-08-13 18:06:06.677101 7f5c973a57a0 20 read_log 192888'14426 (0'0) promote cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by osd.346.0:15884 2014-08-11 17:05:19.927328 2014-08-13 18:06:06.677113 7f5c973a57a0 20 read_log 192888'14427 (192888'14426) modify cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.942868.0:1189 2014-08-11 17:05:19.916121 2014-08-13 18:06:06.677127 7f5c973a57a0 20 read_log 192888'14428 (0'0) promote 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by osd.346.0:15900 2014-08-11 17:05:25.816168 2014-08-13 18:06:06.677140 7f5c973a57a0 20 read_log 192888'14429 (192888'14428) modify 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.942868.0:2668 2014-08-11 17:05:25.810896 2014-08-13 18:06:06.677161 7f5c973a57a0 20 read_log 192888'14430 (0'0) promote c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by osd.346.0:15902 2014-08-11 17:05:26.175331 2014-08-13 18:06:06.677177 7f5c973a57a0 20 read_log 192888'14431 (192888'14430) modify c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.942868.0:2765 2014-08-11 17:05:26.169086 2014-08-13 18:06:06.677190 7f5c973a57a0 20 read_log 192888'14434 (0'0) promote fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by osd.346.0:15930 2014-08-11 17:05:35.185393 2014-08-13 18:06:06.677202 7f5c973a57a0 20 read_log 192888'14435 (192888'14434) modify fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.942868.0:5149 2014-08-11 17:05:35.175804 2014-08-13 18:06:06.677216 7f5c973a57a0 20 read_log 192888'14436 (0'0) promote 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by osd.346.0:15932 2014-08-11 17:05:36.714196 2014-08-13 18:06:06.677229 7f5c973a57a0 20 read_log 192888'14437 (192888'14436) modify 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.942868.0:5559 2014-08-11 17:05:36.704368 2014-08-13 18:06:06.677244 7f5c973a57a0 20 read_log 192888'14442 (0'0) promote e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by osd.346.0:15942 2014-08-11 17:05:44.678615 2014-08-13 18:06:06.677259 7f5c973a57a0 20 read_log 192888'14443 (192888'14442) modify e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.942868.0:7682 2014-08-11 17:05:44.671600 2014-08-13 18:06:06.677274 7f5c973a57a0 20 read_log 192888'14444 (0'0) promote d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by osd.346.0:15944 2014-08-11 17:05:45.365194 2014-08-13 18:06:06.677286 7f5c973a57a0 20 read_log 192888'14445 (192888'14444) modify d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.942868.0:7872 2014-08-11 17:05:45.359365 2014-08-13 18:06:06.677301 7f5c973a57a0 20 read_log 192888'14446 (0'0) promote 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by osd.346.0:15948 2014-08-11 17:05:47.548916 2014-08-13 18:06:06.677319 7f5c973a57a0 20 read_log 192888'14447 (192888'14446) modify 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.942868.0:8477 2014-08-11 17:05:47.542121 2014-08-13 18:06:06.677338 7f5c973a57a0 20 read_log 192888'14448 (0'0) promote 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by osd.346.0:15968 2014-08-11 17:05:55.414664 2014-08-13 18:06:06.677356 7f5c973a57a0 20 read_log 192888'14449 (192888'14448) modify 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.942868.0:10614 2014-08-11 17:05:55.407532 2014-08-13 18:06:06.677369 7f5c973a57a0 20 read_log 192888'14450 (0'0) promote 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by osd.346.0:15972 2014-08-11 17:05:57.183003 2014-08-13 18:06:06.677391 7f5c973a57a0 20 read_log 192888'14451 (192888'14450) modify 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.942868.0:11108 2014-08-11 17:05:57.175916 2014-08-13 18:06:06.677411 7f5c973a57a0 20 read_log 192888'14454 (0'0) promote 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by osd.346.0:15990 2014-08-11 17:06:04.384060 2014-08-13 18:06:06.677432 7f5c973a57a0 20 read_log 192888'14455 (192888'14454) modify 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.942868.0:13054 2014-08-11 17:06:04.377479 2014-08-13 18:06:06.677448 7f5c973a57a0 20 read_log 192888'14456 (0'0) promote be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by osd.346.0:16012 2014-08-11 17:06:08.689581 2014-08-13 18:06:06.677461 7f5c973a57a0 20 read_log 192888'14457 (192888'14456) modify be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.942868.0:14113 2014-08-11 17:06:08.681754 2014-08-13 18:06:06.677473 7f5c973a57a0 20 read_log 192888'14458 (0'0) promote 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by osd.346.0:16014 2014-08-11 17:06:08.748006 2014-08-13 18:06:06.677486 7f5c973a57a0 20 read_log 192888'14459 (192888'14458) modify 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.942868.0:14127 2014-08-11 17:06:08.740653 2014-08-13 18:06:06.677506 7f5c973a57a0 20 read_log 192888'14460 (0'0) promote 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by osd.346.0:16020 2014-08-11 17:06:10.215158 2014-08-13 18:06:06.677521 7f5c973a57a0 20 read_log 192888'14461 (192888'14460) modify 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.942868.0:14492 2014-08-11 17:06:10.207604 2014-08-13 18:06:06.677534 7f5c973a57a0 20 read_log 192888'14466 (0'0) promote dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by osd.346.0:16032 2014-08-11 17:06:13.254358 2014-08-13 18:06:06.677555 7f5c973a57a0 20 read_log 192888'14467 (192888'14466) modify dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.942868.0:15264 2014-08-11 17:06:13.246262 2014-08-13 18:06:06.677568 7f5c973a57a0 20 read_log 192888'14468 (0'0) promote 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by osd.346.0:16034 2014-08-11 17:06:13.311533 2014-08-13 18:06:06.677581 7f5c973a57a0 20 read_log 192888'14469 (192888'14468) modify 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.942868.0:15279 2014-08-11 17:06:13.304203 2014-08-13 18:06:06.677602 7f5c973a57a0 20 read_log 192888'14480 (0'0) promote 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by osd.346.0:16086 2014-08-11 17:06:30.339435 2014-08-13 18:06:06.677615 7f5c973a57a0 20 read_log 192888'14481 (192888'14480) modify 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.942868.0:19729 2014-08-11 17:06:30.332446 2014-08-13 18:06:06.677627 7f5c973a57a0 20 read_log 192888'14484 (0'0) promote 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by osd.346.0:16090 2014-08-11 17:06:31.578336 2014-08-13 18:06:06.677639 7f5c973a57a0 20 read_log 192888'14485 (192888'14484) modify 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.942868.0:20058 2014-08-11 17:06:31.571838 2014-08-13 18:06:06.677681 7f5c973a57a0 20 read_log 192888'14488 (0'0) promote 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by osd.346.0:16096 2014-08-11 17:06:34.447418 2014-08-13 18:06:06.677699 7f5c973a57a0 20 read_log 192888'14489 (192888'14488) modify 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.942868.0:20800 2014-08-11 17:06:34.437522 2014-08-13 18:06:06.677717 7f5c973a57a0 20 read_log 192888'14490 (0'0) promote 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by osd.346.0:16104 2014-08-11 17:06:36.138613 2014-08-13 18:06:06.677730 7f5c973a57a0 20 read_log 192888'14491 (192888'14490) modify 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.942868.0:21238 2014-08-11 17:06:36.131166 2014-08-13 18:06:06.677751 7f5c973a57a0 20 read_log 192888'14492 (0'0) promote d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by osd.346.0:16112 2014-08-11 17:06:37.549262 2014-08-13 18:06:06.677771 7f5c973a57a0 20 read_log 192888'14493 (192888'14492) modify d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.942868.0:21618 2014-08-11 17:06:37.541317 2014-08-13 18:06:06.677786 7f5c973a57a0 20 read_log 192888'14498 (0'0) promote c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by osd.346.0:16136 2014-08-11 17:06:42.384692 2014-08-13 18:06:06.677799 7f5c973a57a0 20 read_log 192888'14499 (192888'14498) modify c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.942868.0:22873 2014-08-11 17:06:42.378050 2014-08-13 18:06:06.677811 7f5c973a57a0 20 read_log 192888'14500 (0'0) promote 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by osd.346.0:16138 2014-08-11 17:06:44.798455 2014-08-13 18:06:06.677823 7f5c973a57a0 20 read_log 192888'14501 (192888'14500) modify 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.942868.0:23481 2014-08-11 17:06:44.791283 2014-08-13 18:06:06.677836 7f5c973a57a0 20 read_log 192888'14502 (0'0) promote d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by osd.346.0:16140 2014-08-11 17:06:45.184296 2014-08-13 18:06:06.677848 7f5c973a57a0 20 read_log 192888'14503 (192888'14502) modify d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.942868.0:23587 2014-08-11 17:06:45.177357 2014-08-13 18:06:06.677860 7f5c973a57a0 20 read_log 192888'14508 (0'0) promote 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by osd.346.0:16158 2014-08-11 17:06:49.433511 2014-08-13 18:06:06.677880 7f5c973a57a0 20 read_log 192888'14509 (192888'14508) modify 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.942868.0:24735 2014-08-11 17:06:49.425851 2014-08-13 18:06:06.677895 7f5c973a57a0 20 read_log 192888'14516 (0'0) promote 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by osd.346.0:16184 2014-08-11 17:06:54.705552 2014-08-13 18:06:06.677907 7f5c973a57a0 20 read_log 192888'14517 (192888'14516) modify 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.942868.0:26070 2014-08-11 17:06:54.697484 2014-08-13 18:06:06.677920 7f5c973a57a0 20 read_log 192888'14522 (0'0) promote 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by osd.346.0:16208 2014-08-11 17:07:05.853970 2014-08-13 18:06:06.677932 7f5c973a57a0 20 read_log 192888'14523 (192888'14522) modify 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.942868.0:29048 2014-08-11 17:07:05.847289 2014-08-13 18:06:06.677944 7f5c973a57a0 20 read_log 192888'14526 (0'0) promote 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by osd.346.0:16228 2014-08-11 17:07:12.030187 2014-08-13 18:06:06.677957 7f5c973a57a0 20 read_log 192888'14527 (192888'14526) modify 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.942868.0:30553 2014-08-11 17:07:12.023268 2014-08-13 18:06:06.677977 7f5c973a57a0 20 read_log 192888'14528 (0'0) promote 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by osd.346.0:16234 2014-08-11 17:07:14.438998 2014-08-13 18:06:06.677992 7f5c973a57a0 20 read_log 192888'14529 (192888'14528) modify 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.942868.0:31166 2014-08-11 17:07:14.432020 2014-08-13 18:06:06.678005 7f5c973a57a0 20 read_log 192888'14530 (192888'14427) modify cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.942868.0:32455 2014-08-11 17:07:16.259453 2014-08-13 18:06:06.678017 7f5c973a57a0 20 read_log 192888'14531 (192888'14429) modify 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.942868.0:33934 2014-08-11 17:07:16.651909 2014-08-13 18:06:06.678030 7f5c973a57a0 20 read_log 192888'14532 (192888'14431) modify c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.942868.0:34031 2014-08-11 17:07:16.679184 2014-08-13 18:06:06.678042 7f5c973a57a0 20 read_log 192888'14534 (192888'14435) modify fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.942868.0:36415 2014-08-11 17:07:17.321712 2014-08-13 18:06:06.678075 7f5c973a57a0 20 read_log 192888'14535 (192888'14437) modify 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.942868.0:36825 2014-08-11 17:07:17.443173 2014-08-13 18:06:06.678090 7f5c973a57a0 20 read_log 192888'14538 (192888'14443) modify e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.942868.0:38948 2014-08-11 17:07:18.036663 2014-08-13 18:06:06.678103 7f5c973a57a0 20 read_log 192888'14539 (192888'14445) modify d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.942868.0:39138 2014-08-11 17:07:18.094245 2014-08-13 18:06:06.678115 7f5c973a57a0 20 read_log 192888'14540 (192888'14447) modify 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.942868.0:39743 2014-08-11 17:07:18.270873 2014-08-13 18:06:06.678128 7f5c973a57a0 20 read_log 192888'14541 (192888'14449) modify 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.942868.0:41880 2014-08-11 17:07:18.847459 2014-08-13 18:06:06.678140 7f5c973a57a0 20 read_log 192888'14542 (192888'14451) modify 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.942868.0:42374 2014-08-11 17:07:18.981708 2014-08-13 18:06:06.678160 7f5c973a57a0 20 read_log 192888'14544 (192888'14455) modify 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.942868.0:44320 2014-08-11 17:07:19.534162 2014-08-13 18:06:06.678173 7f5c973a57a0 20 read_log 192888'14545 (192888'14457) modify be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.942868.0:45379 2014-08-11 17:07:19.821254 2014-08-13 18:06:06.678186 7f5c973a57a0 20 read_log 192888'14546 (192888'14459) modify 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.942868.0:45393 2014-08-11 17:07:19.826472 2014-08-13 18:06:06.678200 7f5c973a57a0 20 read_log 192888'14547 (192888'14461) modify 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.942868.0:45758 2014-08-11 17:07:19.926863 2014-08-13 18:06:06.678212 7f5c973a57a0 20 read_log 192888'14550 (192888'14467) modify dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.942868.0:46530 2014-08-11 17:07:20.149754 2014-08-13 18:06:06.678227 7f5c973a57a0 20 read_log 192888'14551 (192888'14469) modify 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.942868.0:46545 2014-08-11 17:07:20.153322 2014-08-13 18:06:06.678240 7f5c973a57a0 20 read_log 192888'14557 (192888'14481) modify 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.942868.0:50995 2014-08-11 17:07:21.496413 2014-08-13 18:06:06.678252 7f5c973a57a0 20 read_log 192888'14559 (192888'14485) modify 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.942868.0:51324 2014-08-11 17:07:21.608988 2014-08-13 18:06:06.678277 7f5c973a57a0 20 read_log 192888'14561 (192888'14489) modify 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.942868.0:52066 2014-08-11 17:07:21.831703 2014-08-13 18:06:06.678293 7f5c973a57a0 20 read_log 192888'14562 (192888'14491) modify 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.942868.0:52504 2014-08-11 17:07:21.960744 2014-08-13 18:06:06.678305 7f5c973a57a0 20 read_log 192888'14563 (192888'14493) modify d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.942868.0:52884 2014-08-11 17:07:22.079657 2014-08-13 18:06:06.678324 7f5c973a57a0 20 read_log 192888'14566 (192888'14499) modify c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.942868.0:54139 2014-08-11 17:07:22.473398 2014-08-13 18:06:06.678336 7f5c973a57a0 20 read_log 192888'14567 (192888'14501) modify 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.942868.0:54747 2014-08-11 17:07:22.645641 2014-08-13 18:06:06.678349 7f5c973a57a0 20 read_log 192888'14568 (192888'14503) modify d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.942868.0:54853 2014-08-11 17:07:22.675361 2014-08-13 18:06:06.678364 7f5c973a57a0 20 read_log 192888'14571 (192888'14509) modify 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.942868.0:56001 2014-08-11 17:07:23.021556 2014-08-13 18:06:06.678377 7f5c973a57a0 20 read_log 192888'14575 (192888'14517) modify 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.942868.0:57336 2014-08-11 17:07:23.426189 2014-08-13 18:06:06.678389 7f5c973a57a0 20 read_log 192888'14578 (192888'14523) modify 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.942868.0:60314 2014-08-11 17:07:24.305112 2014-08-13 18:06:06.678439 7f5c973a57a0 20 read_log 192888'14580 (192888'14527) modify 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.942868.0:61819 2014-08-11 17:07:24.757652 2014-08-13 18:06:06.678465 7f5c973a57a0 20 read_log 192888'14581 (192888'14529) modify 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.942868.0:62432 2014-08-11 17:07:24.945282 2014-08-13 18:06:06.678478 7f5c973a57a0 20 read_log 193305'14582 (193305'14582) modify e2/hit_set_24.e2_archive_2014-08-11 18:37:04.930706_2014-08-11 18:42:38.411584/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.411808 2014-08-13 18:06:06.678498 7f5c973a57a0 20 read_log 193305'14583 (192888'14424) delete e2/hit_set_24.e2_archive_2014-08-11 16:44:58.212327_2014-08-11 17:05:19.923836/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.411808 2014-08-13 18:06:06.678514 7f5c973a57a0 20 read_log 193305'14584 (0'0) promote cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by osd.346.0:7 2014-08-11 18:42:38.423756 2014-08-13 18:06:06.678527 7f5c973a57a0 20 read_log 193305'14585 (193305'14584) modify cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.951337.0:1397 2014-08-11 18:42:38.405792 2014-08-13 18:06:06.678539 7f5c973a57a0 20 read_log 193305'14586 (0'0) promote ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by osd.346.0:21 2014-08-11 18:42:45.510208 2014-08-13 18:06:06.678552 7f5c973a57a0 20 read_log 193305'14587 (193305'14586) modify ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.951337.0:3307 2014-08-11 18:42:45.504144 2014-08-13 18:06:06.678564 7f5c973a57a0 20 read_log 193305'14588 (0'0) promote 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by osd.346.0:30 2014-08-11 18:42:50.212274 2014-08-13 18:06:06.678576 7f5c973a57a0 20 read_log 193305'14589 (193305'14588) modify 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.951337.0:4588 2014-08-11 18:42:50.203978 2014-08-13 18:06:06.678602 7f5c973a57a0 20 read_log 193305'14590 (0'0) promote 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by osd.346.0:46 2014-08-11 18:42:57.688680 2014-08-13 18:06:06.678617 7f5c973a57a0 20 read_log 193305'14591 (193305'14590) modify 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.951337.0:6624 2014-08-11 18:42:57.681845 2014-08-13 18:06:06.678630 7f5c973a57a0 20 read_log 193305'14592 (0'0) promote 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by osd.346.0:50 2014-08-11 18:42:58.551518 2014-08-13 18:06:06.678642 7f5c973a57a0 20 read_log 193305'14593 (193305'14592) modify 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.951337.0:6864 2014-08-11 18:42:58.544688 2014-08-13 18:06:06.678654 7f5c973a57a0 20 read_log 193305'14594 (0'0) promote 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by osd.346.0:64 2014-08-11 18:43:05.758005 2014-08-13 18:06:06.678666 7f5c973a57a0 20 read_log 193305'14595 (193305'14594) modify 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.951337.0:8775 2014-08-11 18:43:05.750436 2014-08-13 18:06:06.678679 7f5c973a57a0 20 read_log 193305'14596 (0'0) promote 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by osd.346.0:68 2014-08-11 18:43:11.275728 2014-08-13 18:06:06.678691 7f5c973a57a0 20 read_log 193305'14597 (193305'14596) modify 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.951337.0:10272 2014-08-11 18:43:11.267336 2014-08-13 18:06:06.678703 7f5c973a57a0 20 read_log 193305'14598 (0'0) promote 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by osd.346.0:74 2014-08-11 18:43:13.866365 2014-08-13 18:06:06.678715 7f5c973a57a0 20 read_log 193305'14599 (193305'14598) modify 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.951337.0:10962 2014-08-11 18:43:13.859088 2014-08-13 18:06:06.678728 7f5c973a57a0 20 read_log 193305'14600 (0'0) promote b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by osd.346.0:76 2014-08-11 18:43:14.595096 2014-08-13 18:06:06.678740 7f5c973a57a0 20 read_log 193305'14601 (193305'14600) modify b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.951337.0:11141 2014-08-11 18:43:14.587300 2014-08-13 18:06:06.678752 7f5c973a57a0 20 read_log 193305'14602 (0'0) promote 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by osd.346.0:80 2014-08-11 18:43:16.010454 2014-08-13 18:06:06.678773 7f5c973a57a0 20 read_log 193305'14603 (193305'14602) modify 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.951337.0:11490 2014-08-11 18:43:16.004045 2014-08-13 18:06:06.678789 7f5c973a57a0 20 read_log 193305'14604 (0'0) promote b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by osd.346.0:86 2014-08-11 18:43:19.304544 2014-08-13 18:06:06.678802 7f5c973a57a0 20 read_log 193305'14605 (193305'14604) modify b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.951337.0:12387 2014-08-11 18:43:19.297877 2014-08-13 18:06:06.678815 7f5c973a57a0 20 read_log 193305'14606 (0'0) promote 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by osd.346.0:96 2014-08-11 18:43:21.761682 2014-08-13 18:06:06.678827 7f5c973a57a0 20 read_log 193305'14607 (193305'14606) modify 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.951337.0:13035 2014-08-11 18:43:21.753652 2014-08-13 18:06:06.678839 7f5c973a57a0 20 read_log 193305'14608 (0'0) promote 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by osd.346.0:102 2014-08-11 18:43:22.758827 2014-08-13 18:06:06.678851 7f5c973a57a0 20 read_log 193305'14609 (193305'14608) modify 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.951337.0:13304 2014-08-11 18:43:22.751632 2014-08-13 18:06:06.678871 7f5c973a57a0 20 read_log 193305'14610 (0'0) promote d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by osd.346.0:116 2014-08-11 18:43:29.178749 2014-08-13 18:06:06.678886 7f5c973a57a0 20 read_log 193305'14611 (193305'14610) modify d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.951337.0:15061 2014-08-11 18:43:29.170910 2014-08-13 18:06:06.678898 7f5c973a57a0 20 read_log 193305'14612 (0'0) promote e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by osd.346.0:122 2014-08-11 18:43:29.937057 2014-08-13 18:06:06.678911 7f5c973a57a0 20 read_log 193305'14613 (193305'14612) modify e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.951337.0:15289 2014-08-11 18:43:29.929949 2014-08-13 18:06:06.678925 7f5c973a57a0 20 read_log 193305'14614 (0'0) promote b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by osd.346.0:136 2014-08-11 18:43:37.267541 2014-08-13 18:06:06.678937 7f5c973a57a0 20 read_log 193305'14615 (193305'14614) modify b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.951337.0:17388 2014-08-11 18:43:37.260852 2014-08-13 18:06:06.678949 7f5c973a57a0 20 read_log 193305'14616 (0'0) promote 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by osd.346.0:150 2014-08-11 18:43:43.085639 2014-08-13 18:06:06.678962 7f5c973a57a0 20 read_log 193305'14617 (193305'14616) modify 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.951337.0:19034 2014-08-11 18:43:43.077642 2014-08-13 18:06:06.678976 7f5c973a57a0 20 read_log 193305'14618 (0'0) promote 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by osd.346.0:154 2014-08-11 18:43:46.844480 2014-08-13 18:06:06.678988 7f5c973a57a0 20 read_log 193305'14619 (193305'14618) modify 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.951337.0:20062 2014-08-11 18:43:46.836360 2014-08-13 18:06:06.679002 7f5c973a57a0 20 read_log 193305'14620 (0'0) promote eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by osd.346.0:160 2014-08-11 18:43:47.959317 2014-08-13 18:06:06.679015 7f5c973a57a0 20 read_log 193305'14621 (193305'14620) modify eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.951337.0:20360 2014-08-11 18:43:47.952888 2014-08-13 18:06:06.679030 7f5c973a57a0 20 read_log 193305'14622 (0'0) promote 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by osd.346.0:162 2014-08-11 18:43:50.209000 2014-08-13 18:06:06.679042 7f5c973a57a0 20 read_log 193305'14623 (193305'14622) modify 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.951337.0:21002 2014-08-11 18:43:50.201439 2014-08-13 18:06:06.679056 7f5c973a57a0 20 read_log 193305'14624 (0'0) promote a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by osd.346.0:164 2014-08-11 18:43:50.828111 2014-08-13 18:06:06.679069 7f5c973a57a0 20 read_log 193305'14625 (193305'14624) modify a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.951337.0:21170 2014-08-11 18:43:50.819497 2014-08-13 18:06:06.679083 7f5c973a57a0 20 read_log 193305'14626 (0'0) promote e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by osd.346.0:166 2014-08-11 18:43:51.017953 2014-08-13 18:06:06.679095 7f5c973a57a0 20 read_log 193305'14627 (193305'14626) modify e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.951337.0:21224 2014-08-11 18:43:51.010872 2014-08-13 18:06:06.679109 7f5c973a57a0 20 read_log 193305'14628 (0'0) promote d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by osd.346.0:168 2014-08-11 18:43:52.095892 2014-08-13 18:06:06.679122 7f5c973a57a0 20 read_log 193305'14629 (193305'14628) modify d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.951337.0:21527 2014-08-11 18:43:52.088055 2014-08-13 18:06:06.679143 7f5c973a57a0 20 read_log 193305'14630 (0'0) promote ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by osd.346.0:170 2014-08-11 18:43:53.414794 2014-08-13 18:06:06.679164 7f5c973a57a0 20 read_log 193305'14631 (193305'14630) modify ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.951337.0:21896 2014-08-11 18:43:53.408259 2014-08-13 18:06:06.679177 7f5c973a57a0 20 read_log 193305'14632 (0'0) promote 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by osd.346.0:172 2014-08-11 18:43:55.229361 2014-08-13 18:06:06.679189 7f5c973a57a0 20 read_log 193305'14633 (193305'14632) modify 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.951337.0:22388 2014-08-11 18:43:55.222188 2014-08-13 18:06:06.679217 7f5c973a57a0 20 read_log 193305'14634 (0'0) promote cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by osd.346.0:174 2014-08-11 18:43:57.395887 2014-08-13 18:06:06.679234 7f5c973a57a0 20 read_log 193305'14635 (193305'14634) modify cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.951337.0:22987 2014-08-11 18:43:57.388819 2014-08-13 18:06:06.679269 7f5c973a57a0 20 read_log 193305'14636 (0'0) promote 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by osd.346.0:178 2014-08-11 18:44:01.287948 2014-08-13 18:06:06.679298 7f5c973a57a0 20 read_log 193305'14637 (193305'14636) modify 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.951337.0:24066 2014-08-11 18:44:01.280030 2014-08-13 18:06:06.679315 7f5c973a57a0 20 read_log 193305'14638 (0'0) promote 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by osd.346.0:188 2014-08-11 18:44:04.156740 2014-08-13 18:06:06.679333 7f5c973a57a0 20 read_log 193305'14639 (193305'14638) modify 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.951337.0:24869 2014-08-11 18:44:04.148178 2014-08-13 18:06:06.679346 7f5c973a57a0 20 read_log 193305'14640 (0'0) promote 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by osd.346.0:198 2014-08-11 18:44:12.424088 2014-08-13 18:06:06.679401 7f5c973a57a0 20 read_log 193305'14641 (193305'14640) modify 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.951337.0:27097 2014-08-11 18:44:12.415841 2014-08-13 18:06:06.679418 7f5c973a57a0 20 read_log 193305'14642 (0'0) promote 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by osd.346.0:202 2014-08-11 18:44:13.148869 2014-08-13 18:06:06.679431 7f5c973a57a0 20 read_log 193305'14643 (193305'14642) modify 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.951337.0:27295 2014-08-11 18:44:13.141203 2014-08-13 18:06:06.679451 7f5c973a57a0 20 read_log 193305'14644 (0'0) promote b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by osd.346.0:206 2014-08-11 18:44:16.038688 2014-08-13 18:06:06.679464 7f5c973a57a0 20 read_log 193305'14645 (193305'14644) modify b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.951337.0:28108 2014-08-11 18:44:16.028298 2014-08-13 18:06:06.679503 7f5c973a57a0 20 read_log 193305'14646 (0'0) promote 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by osd.346.0:239 2014-08-11 18:44:30.506208 2014-08-13 18:06:06.679523 7f5c973a57a0 20 read_log 193305'14647 (193305'14646) modify 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.951337.0:32069 2014-08-11 18:44:30.499427 2014-08-13 18:06:06.679536 7f5c973a57a0 20 read_log 193305'14648 (0'0) promote fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by osd.346.0:245 2014-08-11 18:44:32.281068 2014-08-13 18:06:06.679550 7f5c973a57a0 20 read_log 193305'14649 (193305'14648) modify fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.951337.0:32556 2014-08-11 18:44:32.273712 2014-08-13 18:06:06.679590 7f5c973a57a0 20 read_log 193305'14650 (193305'14585) modify cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.951337.0:34108 2014-08-11 18:44:34.798588 2014-08-13 18:06:06.679606 7f5c973a57a0 20 read_log 193305'14651 (193305'14587) modify ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.951337.0:36018 2014-08-11 18:44:35.876263 2014-08-13 18:06:06.679619 7f5c973a57a0 20 read_log 193305'14652 (193305'14589) modify 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.951337.0:37299 2014-08-11 18:44:36.844671 2014-08-13 18:06:06.679631 7f5c973a57a0 20 read_log 193305'14653 (193305'14591) modify 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.951337.0:39335 2014-08-11 18:44:37.952822 2014-08-13 18:06:06.679644 7f5c973a57a0 20 read_log 193305'14654 (193305'14593) modify 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.951337.0:39575 2014-08-11 18:44:38.086264 2014-08-13 18:06:06.679656 7f5c973a57a0 20 read_log 193305'14655 (193305'14595) modify 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.951337.0:41486 2014-08-11 18:44:38.768691 2014-08-13 18:06:06.679668 7f5c973a57a0 20 read_log 193305'14656 (193305'14597) modify 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.951337.0:42983 2014-08-11 18:44:39.444175 2014-08-13 18:06:06.679680 7f5c973a57a0 20 read_log 193305'14657 (193305'14599) modify 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.951337.0:43673 2014-08-11 18:44:39.661467 2014-08-13 18:06:06.679701 7f5c973a57a0 20 read_log 193305'14658 (193305'14601) modify b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.951337.0:43852 2014-08-11 18:44:39.718105 2014-08-13 18:06:06.679716 7f5c973a57a0 20 read_log 193305'14659 (193305'14603) modify 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.951337.0:44201 2014-08-11 18:44:39.820587 2014-08-13 18:06:06.679728 7f5c973a57a0 20 read_log 193305'14660 (193305'14605) modify b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.951337.0:45098 2014-08-11 18:44:40.089011 2014-08-13 18:06:06.679741 7f5c973a57a0 20 read_log 193305'14661 (193305'14607) modify 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.951337.0:45746 2014-08-11 18:44:40.292293 2014-08-13 18:06:06.679753 7f5c973a57a0 20 read_log 193305'14662 (193305'14609) modify 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.951337.0:46015 2014-08-11 18:44:40.366697 2014-08-13 18:06:06.679766 7f5c973a57a0 20 read_log 193305'14663 (193305'14611) modify d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.951337.0:47772 2014-08-11 18:44:40.861713 2014-08-13 18:06:06.679782 7f5c973a57a0 20 read_log 193305'14664 (193305'14613) modify e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.951337.0:48000 2014-08-11 18:44:40.926778 2014-08-13 18:06:06.679795 7f5c973a57a0 20 read_log 193305'14665 (193305'14615) modify b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.951337.0:50099 2014-08-11 18:44:41.565226 2014-08-13 18:06:06.679807 7f5c973a57a0 20 read_log 193305'14666 (193305'14617) modify 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.951337.0:51745 2014-08-11 18:44:42.038572 2014-08-13 18:06:06.679822 7f5c973a57a0 20 read_log 193305'14667 (193305'14619) modify 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.951337.0:52773 2014-08-11 18:44:42.333648 2014-08-13 18:06:06.679835 7f5c973a57a0 20 read_log 193305'14668 (193305'14621) modify eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.951337.0:53071 2014-08-11 18:44:42.420768 2014-08-13 18:06:06.679847 7f5c973a57a0 20 read_log 193305'14669 (193305'14623) modify 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.951337.0:53713 2014-08-11 18:44:42.610951 2014-08-13 18:06:06.679884 7f5c973a57a0 20 read_log 193305'14670 (193305'14625) modify a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.951337.0:53881 2014-08-11 18:44:42.663724 2014-08-13 18:06:06.679907 7f5c973a57a0 20 read_log 193305'14671 (193305'14627) modify e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.951337.0:53935 2014-08-11 18:44:42.678222 2014-08-13 18:06:06.679921 7f5c973a57a0 20 read_log 193305'14672 (193305'14629) modify d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.951337.0:54238 2014-08-11 18:44:42.772739 2014-08-13 18:06:06.679934 7f5c973a57a0 20 read_log 193305'14673 (193305'14631) modify ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.951337.0:54607 2014-08-11 18:44:42.887853 2014-08-13 18:06:06.679946 7f5c973a57a0 20 read_log 193305'14674 (193305'14633) modify 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.951337.0:55099 2014-08-11 18:44:43.026159 2014-08-13 18:06:06.679971 7f5c973a57a0 20 read_log 193305'14675 (193305'14635) modify cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.951337.0:55698 2014-08-11 18:44:43.204627 2014-08-13 18:06:06.679984 7f5c973a57a0 20 read_log 193305'14676 (193305'14637) modify 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.951337.0:56777 2014-08-11 18:44:43.529085 2014-08-13 18:06:06.680003 7f5c973a57a0 20 read_log 193305'14677 (193305'14639) modify 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.951337.0:57580 2014-08-11 18:44:43.798718 2014-08-13 18:06:06.680017 7f5c973a57a0 20 read_log 193305'14678 (193305'14641) modify 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.951337.0:59808 2014-08-11 18:44:44.432841 2014-08-13 18:06:06.680030 7f5c973a57a0 20 read_log 193305'14679 (193305'14643) modify 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.951337.0:60006 2014-08-11 18:44:44.492307 2014-08-13 18:06:06.680042 7f5c973a57a0 20 read_log 193305'14680 (193305'14645) modify b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.951337.0:60819 2014-08-11 18:44:44.744444 2014-08-13 18:06:06.680054 7f5c973a57a0 20 read_log 193305'14681 (193305'14647) modify 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.951337.0:64780 2014-08-11 18:44:45.915738 2014-08-13 18:06:06.680067 7f5c973a57a0 20 read_log 193305'14682 (193305'14649) modify fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.951337.0:65267 2014-08-11 18:44:46.069420 2014-08-13 18:06:06.680079 7f5c973a57a0 20 read_log 194215'14683 (194215'14683) modify e2/hit_set_24.e2_archive_2014-08-12 15:32:25.865165_2014-08-12 15:39:39.946124/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:39.946382 2014-08-13 18:06:06.680091 7f5c973a57a0 20 read_log 194215'14684 (193305'14582) delete e2/hit_set_24.e2_archive_2014-08-11 18:37:04.930706_2014-08-11 18:42:38.411584/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:39:39.946382 2014-08-13 18:06:06.680104 7f5c973a57a0 20 read_log 194215'14685 (192888'14398) clean 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by osd.346.0:958 2014-08-12 15:39:39.951822 2014-08-13 18:06:06.680116 7f5c973a57a0 20 read_log 194215'14686 (194215'14685) delete 23800e2/benchmark_data_storage0111-ib.pocstor_41097_object20101/head//24 by client.2174595.0:46101 0.000000 2014-08-13 18:06:06.680127 7f5c973a57a0 20 read_log 194215'14687 (192888'14563) clean d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by osd.346.0:960 2014-08-12 15:39:39.972593 2014-08-13 18:06:06.680140 7f5c973a57a0 20 read_log 194215'14688 (194215'14687) delete d6f010e2/benchmark_data_storage0111-ib.pocstor_2228_object21617/head//24 by client.2174595.0:46103 0.000000 2014-08-13 18:06:06.680167 7f5c973a57a0 20 read_log 194215'14689 (193305'14672) clean d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by osd.346.0:962 2014-08-12 15:39:39.980161 2014-08-13 18:06:06.680181 7f5c973a57a0 20 read_log 194215'14690 (194215'14689) delete d09410e2/benchmark_data_storage0111-ib.pocstor_62621_object21526/head//24 by client.2174595.0:46105 0.000000 2014-08-13 18:06:06.680192 7f5c973a57a0 20 read_log 194215'14691 (192888'14530) clean cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by osd.346.0:964 2014-08-12 15:39:39.987619 2014-08-13 18:06:06.680204 7f5c973a57a0 20 read_log 194215'14692 (194215'14691) delete cfad10e2/benchmark_data_storage0111-ib.pocstor_2228_object1188/head//24 by client.2174595.0:46107 0.000000 2014-08-13 18:06:06.680215 7f5c973a57a0 20 read_log 194215'14693 (193305'14659) clean 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by osd.346.0:966 2014-08-12 15:39:39.994868 2014-08-13 18:06:06.680227 7f5c973a57a0 20 read_log 194215'14694 (194215'14693) delete 475c20e2/benchmark_data_storage0111-ib.pocstor_62621_object11489/head//24 by client.2174595.0:46109 0.000000 2014-08-13 18:06:06.680238 7f5c973a57a0 20 read_log 194215'14695 (193305'14671) clean e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by osd.346.0:968 2014-08-12 15:39:40.001584 2014-08-13 18:06:06.680250 7f5c973a57a0 20 read_log 194215'14696 (194215'14695) delete e83b40e2/benchmark_data_storage0111-ib.pocstor_62621_object21223/head//24 by client.2174595.0:46111 0.000000 2014-08-13 18:06:06.680261 7f5c973a57a0 20 read_log 194215'14697 (192888'14389) clean 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by osd.346.0:970 2014-08-12 15:39:40.008683 2014-08-13 18:06:06.680273 7f5c973a57a0 20 read_log 194215'14698 (194215'14697) delete 5d0660e2/benchmark_data_storage0111-ib.pocstor_41097_object14623/head//24 by client.2174595.0:46113 0.000000 2014-08-13 18:06:06.680284 7f5c973a57a0 20 read_log 194215'14699 (193305'14675) clean cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by osd.346.0:972 2014-08-12 15:39:40.015678 2014-08-13 18:06:06.680296 7f5c973a57a0 20 read_log 194215'14700 (194215'14699) delete cd8a60e2/benchmark_data_storage0111-ib.pocstor_62621_object22986/head//24 by client.2174595.0:46115 0.000000 2014-08-13 18:06:06.680307 7f5c973a57a0 20 read_log 194215'14701 (192888'14534) clean fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by osd.346.0:974 2014-08-12 15:39:40.022506 2014-08-13 18:06:06.680320 7f5c973a57a0 20 read_log 194215'14702 (194215'14701) delete fc4080e2/benchmark_data_storage0111-ib.pocstor_2228_object5148/head//24 by client.2174595.0:46117 0.000000 2014-08-13 18:06:06.680333 7f5c973a57a0 20 read_log 194215'14703 (193305'14670) clean a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by osd.346.0:976 2014-08-12 15:39:40.029553 2014-08-13 18:06:06.680346 7f5c973a57a0 20 read_log 194215'14704 (194215'14703) delete a5fc80e2/benchmark_data_storage0111-ib.pocstor_62621_object21169/head//24 by client.2174595.0:46119 0.000000 2014-08-13 18:06:06.680357 7f5c973a57a0 20 read_log 194215'14705 (193305'14652) clean 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by osd.346.0:978 2014-08-12 15:39:40.036619 2014-08-13 18:06:06.680370 7f5c973a57a0 20 read_log 194215'14706 (194215'14705) delete 6f4690e2/benchmark_data_storage0111-ib.pocstor_62621_object4587/head//24 by client.2174595.0:46121 0.000000 2014-08-13 18:06:06.680380 7f5c973a57a0 20 read_log 194215'14707 (192888'14567) clean 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by osd.346.0:980 2014-08-12 15:39:40.044419 2014-08-13 18:06:06.680393 7f5c973a57a0 20 read_log 194215'14708 (194215'14707) delete 519690e2/benchmark_data_storage0111-ib.pocstor_2228_object23480/head//24 by client.2174595.0:46123 0.000000 2014-08-13 18:06:06.680410 7f5c973a57a0 20 read_log 194215'14709 (192888'14561) clean 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by osd.346.0:982 2014-08-12 15:39:40.051223 2014-08-13 18:06:06.680428 7f5c973a57a0 20 read_log 194215'14710 (194215'14709) delete 63a5a0e2/benchmark_data_storage0111-ib.pocstor_2228_object20799/head//24 by client.2174595.0:46125 0.000000 2014-08-13 18:06:06.680439 7f5c973a57a0 20 read_log 194215'14711 (192888'14420) clean e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by osd.346.0:984 2014-08-12 15:39:40.058362 2014-08-13 18:06:06.680451 7f5c973a57a0 20 read_log 194215'14712 (194215'14711) delete e29da0e2/benchmark_data_storage0111-ib.pocstor_41097_object29019/head//24 by client.2174595.0:46127 0.000000 2014-08-13 18:06:06.680472 7f5c973a57a0 20 read_log 194215'14713 (193305'14676) clean 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by osd.346.0:986 2014-08-12 15:39:40.065305 2014-08-13 18:06:06.680485 7f5c973a57a0 20 read_log 194215'14714 (194215'14713) delete 709b0e2/benchmark_data_storage0111-ib.pocstor_62621_object24065/head//24 by client.2174595.0:46129 0.000000 2014-08-13 18:06:06.680496 7f5c973a57a0 20 read_log 194215'14715 (192888'14380) clean 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by osd.346.0:988 2014-08-12 15:39:40.072547 2014-08-13 18:06:06.680524 7f5c973a57a0 20 read_log 194215'14716 (194215'14715) delete 4ecb0e2/benchmark_data_storage0111-ib.pocstor_41097_object10100/head//24 by client.2174595.0:46131 0.000000 2014-08-13 18:06:06.680536 7f5c973a57a0 20 read_log 194215'14717 (192888'14535) clean 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by osd.346.0:990 2014-08-12 15:39:40.079567 2014-08-13 18:06:06.680580 7f5c973a57a0 20 read_log 194215'14718 (194215'14717) delete 5d6eb0e2/benchmark_data_storage0111-ib.pocstor_2228_object5558/head//24 by client.2174595.0:46133 0.000000 2014-08-13 18:06:06.680593 7f5c973a57a0 20 read_log 194215'14719 (192888'14397) clean 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by osd.346.0:992 2014-08-12 15:39:40.086495 2014-08-13 18:06:06.680605 7f5c973a57a0 20 read_log 194215'14720 (194215'14719) delete 233fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object20016/head//24 by client.2174595.0:46135 0.000000 2014-08-13 18:06:06.680616 7f5c973a57a0 20 read_log 194215'14721 (192888'14394) clean 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by osd.346.0:994 2014-08-12 15:39:40.093456 2014-08-13 18:06:06.680629 7f5c973a57a0 20 read_log 194215'14722 (194215'14721) delete 724fb0e2/benchmark_data_storage0111-ib.pocstor_41097_object17332/head//24 by client.2174595.0:46137 0.000000 2014-08-13 18:06:06.680648 7f5c973a57a0 20 read_log 194215'14723 (192888'14551) clean 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by osd.346.0:996 2014-08-12 15:39:40.100841 2014-08-13 18:06:06.680662 7f5c973a57a0 20 read_log 194215'14724 (194215'14723) delete 9905c0e2/benchmark_data_storage0111-ib.pocstor_2228_object15278/head//24 by client.2174595.0:46139 0.000000 2014-08-13 18:06:06.680675 7f5c973a57a0 20 read_log 194215'14725 (193305'14674) clean 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by osd.346.0:998 2014-08-12 15:39:40.107798 2014-08-13 18:06:06.680688 7f5c973a57a0 20 read_log 194215'14726 (194215'14725) delete 720ac0e2/benchmark_data_storage0111-ib.pocstor_62621_object22387/head//24 by client.2174595.0:46141 0.000000 2014-08-13 18:06:06.680699 7f5c973a57a0 20 read_log 194215'14727 (192888'14566) clean c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by osd.346.0:1000 2014-08-12 15:39:40.114965 2014-08-13 18:06:06.680711 7f5c973a57a0 20 read_log 194215'14728 (194215'14727) delete c865d0e2/benchmark_data_storage0111-ib.pocstor_2228_object22872/head//24 by client.2174595.0:46143 0.000000 2014-08-13 18:06:06.680729 7f5c973a57a0 20 read_log 194215'14729 (192888'14575) clean 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by osd.346.0:1002 2014-08-12 15:39:40.122085 2014-08-13 18:06:06.680742 7f5c973a57a0 20 read_log 194215'14730 (194215'14729) delete 23f7d0e2/benchmark_data_storage0111-ib.pocstor_2228_object26069/head//24 by client.2174595.0:46145 0.000000 2014-08-13 18:06:06.680753 7f5c973a57a0 20 read_log 194215'14731 (192888'14550) clean dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by osd.346.0:1004 2014-08-12 15:39:40.128853 2014-08-13 18:06:06.680765 7f5c973a57a0 20 read_log 194215'14732 (194215'14731) delete dabfd0e2/benchmark_data_storage0111-ib.pocstor_2228_object15263/head//24 by client.2174595.0:46147 0.000000 2014-08-13 18:06:06.680776 7f5c973a57a0 20 read_log 194215'14733 (192888'14578) clean 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by osd.346.0:1006 2014-08-12 15:39:40.136034 2014-08-13 18:06:06.680789 7f5c973a57a0 20 read_log 194215'14734 (194215'14733) delete 663ce0e2/benchmark_data_storage0111-ib.pocstor_2228_object29047/head//24 by client.2174595.0:46149 0.000000 2014-08-13 18:06:06.680808 7f5c973a57a0 20 read_log 194215'14735 (193305'14680) clean b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by osd.346.0:1008 2014-08-12 15:39:40.143639 2014-08-13 18:06:06.680821 7f5c973a57a0 20 read_log 194215'14736 (194215'14735) delete b1a2f0e2/benchmark_data_storage0111-ib.pocstor_62621_object28107/head//24 by client.2174595.0:46151 0.000000 2014-08-13 18:06:06.680832 7f5c973a57a0 20 read_log 194215'14737 (193305'14656) clean 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by osd.346.0:1010 2014-08-12 15:39:40.151150 2014-08-13 18:06:06.680845 7f5c973a57a0 20 read_log 194215'14738 (194215'14737) delete 8083f0e2/benchmark_data_storage0111-ib.pocstor_62621_object10271/head//24 by client.2174595.0:46153 0.000000 2014-08-13 18:06:06.680856 7f5c973a57a0 20 read_log 194215'14739 (192888'14363) clean 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by osd.346.0:1012 2014-08-12 15:39:40.158691 2014-08-13 18:06:06.680868 7f5c973a57a0 20 read_log 194215'14740 (194215'14739) delete 4cfaf0e2/benchmark_data_storage0111-ib.pocstor_41097_object771/head//24 by client.2174595.0:46155 0.000000 2014-08-13 18:06:06.680879 7f5c973a57a0 20 read_log 194215'14741 (192888'14532) clean c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by osd.346.0:1014 2014-08-12 15:39:40.166311 2014-08-13 18:06:06.680891 7f5c973a57a0 20 read_log 194215'14742 (194215'14741) delete c74404e2/benchmark_data_storage0111-ib.pocstor_2228_object2764/head//24 by client.2174595.0:46157 0.000000 2014-08-13 18:06:06.680904 7f5c973a57a0 20 read_log 194215'14743 (192888'14415) clean 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by osd.346.0:1016 2014-08-12 15:39:40.173040 2014-08-13 18:06:06.680917 7f5c973a57a0 20 read_log 194215'14744 (194215'14743) delete 3a6904e2/benchmark_data_storage0111-ib.pocstor_41097_object27860/head//24 by client.2174595.0:46159 0.000000 2014-08-13 18:06:06.680927 7f5c973a57a0 20 read_log 194215'14745 (192888'14387) clean f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by osd.346.0:1018 2014-08-12 15:39:40.180374 2014-08-13 18:06:06.680940 7f5c973a57a0 20 read_log 194215'14746 (194215'14745) delete f51c04e2/benchmark_data_storage0111-ib.pocstor_41097_object13169/head//24 by client.2174595.0:46161 0.000000 2014-08-13 18:06:06.680951 7f5c973a57a0 20 read_log 194215'14747 (192888'14557) clean 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by osd.346.0:1020 2014-08-12 15:39:40.186798 2014-08-13 18:06:06.680963 7f5c973a57a0 20 read_log 194215'14748 (194215'14747) delete 55f114e2/benchmark_data_storage0111-ib.pocstor_2228_object19728/head//24 by client.2174595.0:46163 0.000000 2014-08-13 18:06:06.680981 7f5c973a57a0 20 read_log 194215'14749 (193305'14662) clean 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by osd.346.0:1022 2014-08-12 15:39:40.193776 2014-08-13 18:06:06.680994 7f5c973a57a0 20 read_log 194215'14750 (194215'14749) delete 77e414e2/benchmark_data_storage0111-ib.pocstor_62621_object13303/head//24 by client.2174595.0:46165 0.000000 2014-08-13 18:06:06.681005 7f5c973a57a0 20 read_log 194215'14751 (192888'14546) clean 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by osd.346.0:1024 2014-08-12 15:39:40.200998 2014-08-13 18:06:06.681022 7f5c973a57a0 20 read_log 194215'14752 (194215'14751) delete 4ab024e2/benchmark_data_storage0111-ib.pocstor_2228_object14126/head//24 by client.2174595.0:46167 0.000000 2014-08-13 18:06:06.681033 7f5c973a57a0 20 read_log 194215'14753 (193305'14658) clean b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by osd.346.0:1026 2014-08-12 15:39:40.207451 2014-08-13 18:06:06.681045 7f5c973a57a0 20 read_log 194215'14754 (194215'14753) delete b37e24e2/benchmark_data_storage0111-ib.pocstor_62621_object11140/head//24 by client.2174595.0:46169 0.000000 2014-08-13 18:06:06.681058 7f5c973a57a0 20 read_log 194215'14755 (193305'14661) clean 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by osd.346.0:1028 2014-08-12 15:39:40.214060 2014-08-13 18:06:06.681071 7f5c973a57a0 20 read_log 194215'14756 (194215'14755) delete 1aa534e2/benchmark_data_storage0111-ib.pocstor_62621_object13034/head//24 by client.2174595.0:46171 0.000000 2014-08-13 18:06:06.681082 7f5c973a57a0 20 read_log 194215'14757 (193305'14663) clean d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by osd.346.0:1030 2014-08-12 15:39:40.220535 2014-08-13 18:06:06.681098 7f5c973a57a0 20 read_log 194215'14758 (194215'14757) delete d8264e2/benchmark_data_storage0111-ib.pocstor_62621_object15060/head//24 by client.2174595.0:46173 0.000000 2014-08-13 18:06:06.681109 7f5c973a57a0 20 read_log 194215'14759 (192888'14414) clean 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by osd.346.0:1032 2014-08-12 15:39:40.226981 2014-08-13 18:06:06.681123 7f5c973a57a0 20 read_log 194215'14760 (194215'14759) delete 3ce664e2/benchmark_data_storage0111-ib.pocstor_41097_object26762/head//24 by client.2174595.0:46175 0.000000 2014-08-13 18:06:06.681134 7f5c973a57a0 20 read_log 194215'14761 (192888'14386) clean f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by osd.346.0:1034 2014-08-12 15:39:40.233319 2014-08-13 18:06:06.681157 7f5c973a57a0 20 read_log 194215'14762 (194215'14761) delete f94964e2/benchmark_data_storage0111-ib.pocstor_41097_object12188/head//24 by client.2174595.0:46177 0.000000 2014-08-13 18:06:06.681171 7f5c973a57a0 20 read_log 194215'14763 (193305'14678) clean 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by osd.346.0:1036 2014-08-12 15:39:40.240258 2014-08-13 18:06:06.681184 7f5c973a57a0 20 read_log 194215'14764 (194215'14763) delete 7ab174e2/benchmark_data_storage0111-ib.pocstor_62621_object27096/head//24 by client.2174595.0:46179 0.000000 2014-08-13 18:06:06.681195 7f5c973a57a0 20 read_log 194215'14765 (192888'14405) clean 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by osd.346.0:1038 2014-08-12 15:39:40.246728 2014-08-13 18:06:06.681207 7f5c973a57a0 20 read_log 194215'14766 (194215'14765) delete 8d4574e2/benchmark_data_storage0111-ib.pocstor_41097_object23432/head//24 by client.2174595.0:46181 0.000000 2014-08-13 18:06:06.681218 7f5c973a57a0 20 read_log 194215'14767 (192888'14580) clean 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by osd.346.0:1040 2014-08-12 15:39:40.253264 2014-08-13 18:06:06.681233 7f5c973a57a0 20 read_log 194215'14768 (194215'14767) delete 251974e2/benchmark_data_storage0111-ib.pocstor_2228_object30552/head//24 by client.2174595.0:46183 0.000000 2014-08-13 18:06:06.681254 7f5c973a57a0 20 read_log 194215'14769 (192888'14369) clean ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by osd.346.0:1042 2014-08-12 15:39:40.259955 2014-08-13 18:06:06.681269 7f5c973a57a0 20 read_log 194215'14770 (194215'14769) delete ac9794e2/benchmark_data_storage0111-ib.pocstor_41097_object3644/head//24 by client.2174595.0:46185 0.000000 2014-08-13 18:06:06.681280 7f5c973a57a0 20 read_log 194215'14771 (193305'14660) clean b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by osd.346.0:1044 2014-08-12 15:39:40.267492 2014-08-13 18:06:06.681293 7f5c973a57a0 20 read_log 194215'14772 (194215'14771) delete b731c4e2/benchmark_data_storage0111-ib.pocstor_62621_object12386/head//24 by client.2174595.0:46187 0.000000 2014-08-13 18:06:06.681307 7f5c973a57a0 20 read_log 194215'14773 (192888'14372) clean 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by osd.346.0:1046 2014-08-12 15:39:40.275202 2014-08-13 18:06:06.681320 7f5c973a57a0 20 read_log 194215'14774 (194215'14773) delete 9783c4e2/benchmark_data_storage0111-ib.pocstor_41097_object4853/head//24 by client.2174595.0:46189 0.000000 2014-08-13 18:06:06.681331 7f5c973a57a0 20 read_log 194215'14775 (193305'14657) clean 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by osd.346.0:1048 2014-08-12 15:39:40.283051 2014-08-13 18:06:06.681346 7f5c973a57a0 20 read_log 194215'14776 (194215'14775) delete 1ca1d4e2/benchmark_data_storage0111-ib.pocstor_62621_object10961/head//24 by client.2174595.0:46191 0.000000 2014-08-13 18:06:06.681357 7f5c973a57a0 20 read_log 194215'14777 (192888'14531) clean 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by osd.346.0:1050 2014-08-12 15:39:40.289920 2014-08-13 18:06:06.681369 7f5c973a57a0 20 read_log 194215'14778 (194215'14777) delete 878d4e2/benchmark_data_storage0111-ib.pocstor_2228_object2667/head//24 by client.2174595.0:46193 0.000000 2014-08-13 18:06:06.681380 7f5c973a57a0 20 read_log 194215'14779 (193305'14669) clean 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by osd.346.0:1052 2014-08-12 15:39:40.297244 2014-08-13 18:06:06.681396 7f5c973a57a0 20 read_log 194215'14780 (194215'14779) delete 8dad4e2/benchmark_data_storage0111-ib.pocstor_62621_object21001/head//24 by client.2174595.0:46195 0.000000 2014-08-13 18:06:06.681407 7f5c973a57a0 20 read_log 194215'14781 (192888'14403) clean 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by osd.346.0:1054 2014-08-12 15:39:40.304969 2014-08-13 18:06:06.681419 7f5c973a57a0 20 read_log 194215'14782 (194215'14781) delete 36a1f4e2/benchmark_data_storage0111-ib.pocstor_41097_object22612/head//24 by client.2174595.0:46197 0.000000 2014-08-13 18:06:06.681446 7f5c973a57a0 20 read_log 194215'14783 (192888'14382) clean 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by osd.346.0:1056 2014-08-12 15:39:40.312577 2014-08-13 18:06:06.681459 7f5c973a57a0 20 read_log 194215'14784 (194215'14783) delete 2b708e2/benchmark_data_storage0111-ib.pocstor_41097_object10312/head//24 by client.2174595.0:46199 0.000000 2014-08-13 18:06:06.681471 7f5c973a57a0 20 read_log 194215'14785 (192888'14540) clean 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by osd.346.0:1058 2014-08-12 15:39:40.319943 2014-08-13 18:06:06.681494 7f5c973a57a0 20 read_log 194215'14786 (194215'14785) delete 3d7a08e2/benchmark_data_storage0111-ib.pocstor_2228_object8476/head//24 by client.2174595.0:46201 0.000000 2014-08-13 18:06:06.681505 7f5c973a57a0 20 read_log 194215'14787 (192888'14547) clean 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by osd.346.0:1060 2014-08-12 15:39:40.326706 2014-08-13 18:06:06.681518 7f5c973a57a0 20 read_log 194215'14788 (194215'14787) delete 570118e2/benchmark_data_storage0111-ib.pocstor_2228_object14491/head//24 by client.2174595.0:46203 0.000000 2014-08-13 18:06:06.681543 7f5c973a57a0 20 read_log 194215'14789 (192888'14367) clean e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by osd.346.0:1062 2014-08-12 15:39:40.333434 2014-08-13 18:06:06.681560 7f5c973a57a0 20 read_log 194215'14790 (194215'14789) delete e83518e2/benchmark_data_storage0111-ib.pocstor_41097_object3319/head//24 by client.2174595.0:46205 0.000000 2014-08-13 18:06:06.681571 7f5c973a57a0 20 read_log 194215'14791 (193305'14679) clean 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by osd.346.0:1064 2014-08-12 15:39:40.339890 2014-08-13 18:06:06.681594 7f5c973a57a0 20 read_log 194215'14792 (194215'14791) delete 2f0428e2/benchmark_data_storage0111-ib.pocstor_62621_object27294/head//24 by client.2174595.0:46207 0.000000 2014-08-13 18:06:06.681605 7f5c973a57a0 20 read_log 194215'14793 (193305'14667) clean 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by osd.346.0:1066 2014-08-12 15:39:40.346960 2014-08-13 18:06:06.681621 7f5c973a57a0 20 read_log 194215'14794 (194215'14793) delete 3e1528e2/benchmark_data_storage0111-ib.pocstor_62621_object20061/head//24 by client.2174595.0:46209 0.000000 2014-08-13 18:06:06.681636 7f5c973a57a0 20 read_log 194215'14795 (192888'14421) clean be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by osd.346.0:1068 2014-08-12 15:39:40.354496 2014-08-13 18:06:06.681649 7f5c973a57a0 20 read_log 194215'14796 (194215'14795) delete be2758e2/benchmark_data_storage0111-ib.pocstor_41097_object29134/head//24 by client.2174595.0:46211 0.000000 2014-08-13 18:06:06.681660 7f5c973a57a0 20 read_log 194215'14797 (192888'14545) clean be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by osd.346.0:1070 2014-08-12 15:39:40.360736 2014-08-13 18:06:06.681690 7f5c973a57a0 20 read_log 194215'14798 (194215'14797) delete be5858e2/benchmark_data_storage0111-ib.pocstor_2228_object14112/head//24 by client.2174595.0:46213 0.000000 2014-08-13 18:06:06.681702 7f5c973a57a0 20 read_log 194215'14799 (192888'14404) clean 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by osd.346.0:1072 2014-08-12 15:39:40.367221 2014-08-13 18:06:06.681714 7f5c973a57a0 20 read_log 194215'14800 (194215'14799) delete 317368e2/benchmark_data_storage0111-ib.pocstor_41097_object22647/head//24 by client.2174595.0:46215 0.000000 2014-08-13 18:06:06.681725 7f5c973a57a0 20 read_log 194215'14801 (192888'14375) clean 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by osd.346.0:1074 2014-08-12 15:39:40.374250 2014-08-13 18:06:06.681738 7f5c973a57a0 20 read_log 194215'14802 (194215'14801) delete 10a568e2/benchmark_data_storage0111-ib.pocstor_41097_object7349/head//24 by client.2174595.0:46217 0.000000 2014-08-13 18:06:06.681749 7f5c973a57a0 20 read_log 194215'14803 (192888'14407) clean 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by osd.346.0:1076 2014-08-12 15:39:40.382136 2014-08-13 18:06:06.681761 7f5c973a57a0 20 read_log 194215'14804 (194215'14803) delete 6e2178e2/benchmark_data_storage0111-ib.pocstor_41097_object24342/head//24 by client.2174595.0:46219 0.000000 2014-08-13 18:06:06.681772 7f5c973a57a0 20 read_log 194215'14805 (193305'14664) clean e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by osd.346.0:1078 2014-08-12 15:39:40.389037 2014-08-13 18:06:06.681784 7f5c973a57a0 20 read_log 194215'14806 (194215'14805) delete e7b678e2/benchmark_data_storage0111-ib.pocstor_62621_object15288/head//24 by client.2174595.0:46221 0.000000 2014-08-13 18:06:06.681795 7f5c973a57a0 20 read_log 194215'14807 (192888'14401) clean 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by osd.346.0:1080 2014-08-12 15:39:40.396351 2014-08-13 18:06:06.681807 7f5c973a57a0 20 read_log 194215'14808 (194215'14807) delete 954098e2/benchmark_data_storage0111-ib.pocstor_41097_object21184/head//24 by client.2174595.0:46223 0.000000 2014-08-13 18:06:06.681844 7f5c973a57a0 20 read_log 194215'14809 (192888'14406) clean 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by osd.346.0:1082 2014-08-12 15:39:40.402863 2014-08-13 18:06:06.681860 7f5c973a57a0 20 read_log 194215'14810 (194215'14809) delete 61a4b8e2/benchmark_data_storage0111-ib.pocstor_41097_object23823/head//24 by client.2174595.0:46225 0.000000 2014-08-13 18:06:06.681871 7f5c973a57a0 20 read_log 194215'14811 (192888'14393) clean d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by osd.346.0:1084 2014-08-12 15:39:40.409621 2014-08-13 18:06:06.681883 7f5c973a57a0 20 read_log 194215'14812 (194215'14811) delete d3f9b8e2/benchmark_data_storage0111-ib.pocstor_41097_object17044/head//24 by client.2174595.0:46227 0.000000 2014-08-13 18:06:06.681894 7f5c973a57a0 20 read_log 194215'14813 (193305'14653) clean 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by osd.346.0:1086 2014-08-12 15:39:40.416490 2014-08-13 18:06:06.681906 7f5c973a57a0 20 read_log 194215'14814 (194215'14813) delete 5402c8e2/benchmark_data_storage0111-ib.pocstor_62621_object6623/head//24 by client.2174595.0:46229 0.000000 2014-08-13 18:06:06.681917 7f5c973a57a0 20 read_log 194215'14815 (192888'14383) clean 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by osd.346.0:1088 2014-08-12 15:39:40.423247 2014-08-13 18:06:06.681930 7f5c973a57a0 20 read_log 194215'14816 (194215'14815) delete 2949c8e2/benchmark_data_storage0111-ib.pocstor_41097_object10968/head//24 by client.2174595.0:46231 0.000000 2014-08-13 18:06:06.681940 7f5c973a57a0 20 read_log 194215'14817 (193305'14665) clean b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by osd.346.0:1090 2014-08-12 15:39:40.430513 2014-08-13 18:06:06.681953 7f5c973a57a0 20 read_log 194215'14818 (194215'14817) delete b3a7d8e2/benchmark_data_storage0111-ib.pocstor_62621_object17387/head//24 by client.2174595.0:46233 0.000000 2014-08-13 18:06:06.681963 7f5c973a57a0 20 read_log 194215'14819 (193305'14651) clean ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by osd.346.0:1092 2014-08-12 15:39:40.437357 2014-08-13 18:06:06.681976 7f5c973a57a0 20 read_log 194215'14820 (194215'14819) delete ca28d8e2/benchmark_data_storage0111-ib.pocstor_62621_object3306/head//24 by client.2174595.0:46235 0.000000 2014-08-13 18:06:06.681986 7f5c973a57a0 20 read_log 194215'14821 (192888'14559) clean 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by osd.346.0:1094 2014-08-12 15:39:40.444192 2014-08-13 18:06:06.681999 7f5c973a57a0 20 read_log 194215'14822 (194215'14821) delete 978de8e2/benchmark_data_storage0111-ib.pocstor_2228_object20057/head//24 by client.2174595.0:46237 0.000000 2014-08-13 18:06:06.682010 7f5c973a57a0 20 read_log 194215'14823 (193305'14682) clean fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by osd.346.0:1096 2014-08-12 15:39:40.450910 2014-08-13 18:06:06.682022 7f5c973a57a0 20 read_log 194215'14824 (194215'14823) delete fc9fe8e2/benchmark_data_storage0111-ib.pocstor_62621_object32555/head//24 by client.2174595.0:46239 0.000000 2014-08-13 18:06:06.682032 7f5c973a57a0 20 read_log 194215'14825 (192888'14581) clean 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by osd.346.0:1098 2014-08-12 15:39:40.457344 2014-08-13 18:06:06.682045 7f5c973a57a0 20 read_log 194215'14826 (194215'14825) delete 7bb4f8e2/benchmark_data_storage0111-ib.pocstor_2228_object31165/head//24 by client.2174595.0:46241 0.000000 2014-08-13 18:06:06.682057 7f5c973a57a0 20 read_log 194215'14827 (192888'14392) clean 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by osd.346.0:1100 2014-08-12 15:39:40.463974 2014-08-13 18:06:06.682069 7f5c973a57a0 20 read_log 194215'14828 (194215'14827) delete 14140ce2/benchmark_data_storage0111-ib.pocstor_41097_object15396/head//24 by client.2174595.0:46243 0.000000 2014-08-13 18:06:06.682092 7f5c973a57a0 20 read_log 194215'14829 (192888'14541) clean 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by osd.346.0:1102 2014-08-12 15:39:40.470955 2014-08-13 18:06:06.682107 7f5c973a57a0 20 read_log 194215'14830 (194215'14829) delete 98c40ce2/benchmark_data_storage0111-ib.pocstor_2228_object10613/head//24 by client.2174595.0:46245 0.000000 2014-08-13 18:06:06.682118 7f5c973a57a0 20 read_log 194215'14831 (192888'14571) clean 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by osd.346.0:1104 2014-08-12 15:39:40.478529 2014-08-13 18:06:06.682130 7f5c973a57a0 20 read_log 194215'14832 (194215'14831) delete 96260ce2/benchmark_data_storage0111-ib.pocstor_2228_object24734/head//24 by client.2174595.0:46247 0.000000 2014-08-13 18:06:06.682144 7f5c973a57a0 20 read_log 194215'14833 (193305'14677) clean 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by osd.346.0:1106 2014-08-12 15:39:40.485891 2014-08-13 18:06:06.682164 7f5c973a57a0 20 read_log 194215'14834 (194215'14833) delete 8f661ce2/benchmark_data_storage0111-ib.pocstor_62621_object24868/head//24 by client.2174595.0:46249 0.000000 2014-08-13 18:06:06.682175 7f5c973a57a0 20 read_log 194215'14835 (193305'14681) clean 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by osd.346.0:1108 2014-08-12 15:39:40.493040 2014-08-13 18:06:06.682188 7f5c973a57a0 20 read_log 194215'14836 (194215'14835) delete 31002ce2/benchmark_data_storage0111-ib.pocstor_62621_object32068/head//24 by client.2174595.0:46251 0.000000 2014-08-13 18:06:06.682199 7f5c973a57a0 20 read_log 194215'14837 (192888'14539) clean d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by osd.346.0:1110 2014-08-12 15:39:40.500779 2014-08-13 18:06:06.682211 7f5c973a57a0 20 read_log 194215'14838 (194215'14837) delete d6e93ce2/benchmark_data_storage0111-ib.pocstor_2228_object7871/head//24 by client.2174595.0:46253 0.000000 2014-08-13 18:06:06.682222 7f5c973a57a0 20 read_log 194215'14839 (193305'14655) clean 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by osd.346.0:1112 2014-08-12 15:39:40.508536 2014-08-13 18:06:06.682234 7f5c973a57a0 20 read_log 194215'14840 (194215'14839) delete 25074ce2/benchmark_data_storage0111-ib.pocstor_62621_object8774/head//24 by client.2174595.0:46255 0.000000 2014-08-13 18:06:06.682245 7f5c973a57a0 20 read_log 194215'14841 (193305'14654) clean 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by osd.346.0:1114 2014-08-12 15:39:40.515965 2014-08-13 18:06:06.682260 7f5c973a57a0 20 read_log 194215'14842 (194215'14841) delete 968e4ce2/benchmark_data_storage0111-ib.pocstor_62621_object6863/head//24 by client.2174595.0:46257 0.000000 2014-08-13 18:06:06.682271 7f5c973a57a0 20 read_log 194215'14843 (192888'14391) clean adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by osd.346.0:1116 2014-08-12 15:39:40.523549 2014-08-13 18:06:06.682286 7f5c973a57a0 20 read_log 194215'14844 (194215'14843) delete adae5ce2/benchmark_data_storage0111-ib.pocstor_41097_object14939/head//24 by client.2174595.0:46259 0.000000 2014-08-13 18:06:06.682302 7f5c973a57a0 20 read_log 194215'14845 (193305'14650) clean cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by osd.346.0:1118 2014-08-12 15:39:40.530934 2014-08-13 18:06:06.682315 7f5c973a57a0 20 read_log 194215'14846 (194215'14845) delete cc8c8ce2/benchmark_data_storage0111-ib.pocstor_62621_object1396/head//24 by client.2174595.0:46261 0.000000 2014-08-13 18:06:06.682326 7f5c973a57a0 20 read_log 194215'14847 (192888'14562) clean 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by osd.346.0:1120 2014-08-12 15:39:40.538766 2014-08-13 18:06:06.682359 7f5c973a57a0 20 read_log 194215'14848 (194215'14847) delete 7c729ce2/benchmark_data_storage0111-ib.pocstor_2228_object21237/head//24 by client.2174595.0:46263 0.000000 2014-08-13 18:06:06.682380 7f5c973a57a0 20 read_log 194215'14849 (192888'14568) clean d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by osd.346.0:1122 2014-08-12 15:39:40.546397 2014-08-13 18:06:06.682398 7f5c973a57a0 20 read_log 194215'14850 (194215'14849) delete d4279ce2/benchmark_data_storage0111-ib.pocstor_2228_object23586/head//24 by client.2174595.0:46265 0.000000 2014-08-13 18:06:06.682410 7f5c973a57a0 20 read_log 194215'14851 (192888'14399) clean 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by osd.346.0:1124 2014-08-12 15:39:40.553187 2014-08-13 18:06:06.682438 7f5c973a57a0 20 read_log 194215'14852 (194215'14851) delete 9422bce2/benchmark_data_storage0111-ib.pocstor_41097_object20550/head//24 by client.2174595.0:46267 0.000000 2014-08-13 18:06:06.682450 7f5c973a57a0 20 read_log 194215'14853 (192888'14419) clean e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by osd.346.0:1126 2014-08-12 15:39:40.560420 2014-08-13 18:06:06.682463 7f5c973a57a0 20 read_log 194215'14854 (194215'14853) delete e603bce2/benchmark_data_storage0111-ib.pocstor_41097_object28938/head//24 by client.2174595.0:46269 0.000000 2014-08-13 18:06:06.682474 7f5c973a57a0 20 read_log 194215'14855 (192888'14422) clean 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by osd.346.0:1128 2014-08-12 15:39:40.568555 2014-08-13 18:06:06.682496 7f5c973a57a0 20 read_log 194215'14856 (194215'14855) delete 2096bce2/benchmark_data_storage0111-ib.pocstor_41097_object29860/head//24 by client.2174595.0:46271 0.000000 2014-08-13 18:06:06.682507 7f5c973a57a0 20 read_log 194215'14857 (193305'14666) clean 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by osd.346.0:1130 2014-08-12 15:39:40.576183 2014-08-13 18:06:06.682519 7f5c973a57a0 20 read_log 194215'14858 (194215'14857) delete 10ebce2/benchmark_data_storage0111-ib.pocstor_62621_object19033/head//24 by client.2174595.0:46273 0.000000 2014-08-13 18:06:06.682538 7f5c973a57a0 20 read_log 194215'14859 (193305'14668) clean eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by osd.346.0:1132 2014-08-12 15:39:40.583411 2014-08-13 18:06:06.682553 7f5c973a57a0 20 read_log 194215'14860 (194215'14859) delete eaf5dce2/benchmark_data_storage0111-ib.pocstor_62621_object20359/head//24 by client.2174595.0:46275 0.000000 2014-08-13 18:06:06.682564 7f5c973a57a0 20 read_log 194215'14861 (192888'14538) clean e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by osd.346.0:1134 2014-08-12 15:39:40.590973 2014-08-13 18:06:06.682585 7f5c973a57a0 20 read_log 194215'14862 (194215'14861) delete e3c6dce2/benchmark_data_storage0111-ib.pocstor_2228_object7681/head//24 by client.2174595.0:46277 0.000000 2014-08-13 18:06:06.682599 7f5c973a57a0 20 read_log 194215'14863 (193305'14673) clean ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by osd.346.0:1136 2014-08-12 15:39:40.597702 2014-08-13 18:06:06.682612 7f5c973a57a0 20 read_log 194215'14864 (194215'14863) delete ccc5ece2/benchmark_data_storage0111-ib.pocstor_62621_object21895/head//24 by client.2174595.0:46279 0.000000 2014-08-13 18:06:06.682629 7f5c973a57a0 20 read_log 194215'14865 (192888'14544) clean 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by osd.346.0:1138 2014-08-12 15:39:40.604898 2014-08-13 18:06:06.682645 7f5c973a57a0 20 read_log 194215'14866 (194215'14865) delete 1e50fce2/benchmark_data_storage0111-ib.pocstor_2228_object13053/head//24 by client.2174595.0:46281 0.000000 2014-08-13 18:06:06.682664 7f5c973a57a0 20 read_log 194215'14867 (192888'14542) clean 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by osd.346.0:1140 2014-08-12 15:39:40.611349 2014-08-13 18:06:06.682680 7f5c973a57a0 20 read_log 194215'14868 (194215'14867) delete 4e75fce2/benchmark_data_storage0111-ib.pocstor_2228_object11107/head//24 by client.2174595.0:46283 0.000000 2014-08-13 18:06:06.682699 7f5c973a57a0 20 read_log 194215'14869 (192888'14364) clean 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by osd.346.0:1142 2014-08-12 15:39:40.618546 2014-08-13 18:06:06.682714 7f5c973a57a0 20 read_log 194215'14870 (194215'14869) delete 2edefce2/benchmark_data_storage0111-ib.pocstor_41097_object1596/head//24 by client.2174595.0:46285 0.000000 2014-08-13 18:06:06.682726 7f5c973a57a0 20 read_log 194215'14871 (194215'14871) modify e2/hit_set_24.e2_archive_2014-08-12 15:39:39.946574_2014-08-12 17:24:43.501290/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.501524 2014-08-13 18:06:06.682738 7f5c973a57a0 20 read_log 194215'14872 (194215'14683) delete e2/hit_set_24.e2_archive_2014-08-12 15:32:25.865165_2014-08-12 15:39:39.946124/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.501524 2014-08-13 18:06:06.682750 7f5c973a57a0 20 read_log 194215'14873 (0'0) promote 2d679ce2/benchmark_data_storage0111-ib.pocstor_34628_object920/head//24 by osd.346.0:3550 2014-08-12 17:24:43.506498 2014-08-13 18:06:06.682763 7f5c973a57a0 20 read_log 194215'14874 (194215'14873) modify 2d679ce2/benchmark_data_storage0111-ib.pocstor_34628_object920/head//24 by client.2193759.0:921 2014-08-12 17:24:43.495413 2014-08-13 18:06:06.682787 7f5c973a57a0 20 read_log 194215'14875 (0'0) promote e999dce2/benchmark_data_storage0111-ib.pocstor_34628_object973/head//24 by osd.346.0:3552 2014-08-12 17:24:43.695936 2014-08-13 18:06:06.682805 7f5c973a57a0 20 read_log 194215'14876 (194215'14875) modify e999dce2/benchmark_data_storage0111-ib.pocstor_34628_object973/head//24 by client.2193759.0:974 2014-08-12 17:24:43.687533 2014-08-13 18:06:06.682818 7f5c973a57a0 20 read_log 194215'14877 (0'0) promote fb8c64e2/benchmark_data_storage0111-ib.pocstor_34628_object1367/head//24 by osd.346.0:3556 2014-08-12 17:24:45.817529 2014-08-13 18:06:06.682831 7f5c973a57a0 20 read_log 194215'14878 (194215'14877) modify fb8c64e2/benchmark_data_storage0111-ib.pocstor_34628_object1367/head//24 by client.2193759.0:1368 2014-08-12 17:24:45.808540 2014-08-13 18:06:06.682843 7f5c973a57a0 20 read_log 194370'14879 (0'0) promote 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by osd.346.0:3582 2014-08-12 20:30:17.686159 2014-08-13 18:06:06.682858 7f5c973a57a0 20 read_log 194370'14880 (194370'14879) modify 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by client.1508531.0:878 2014-08-12 20:30:17.678096 2014-08-13 18:06:06.682871 7f5c973a57a0 20 read_log 194370'14881 (0'0) promote b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by osd.346.0:3602 2014-08-12 20:30:27.647062 2014-08-13 18:06:06.682898 7f5c973a57a0 20 read_log 194370'14882 (194370'14881) modify b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by client.1508531.0:3564 2014-08-12 20:30:27.639554 2014-08-13 18:06:06.682913 7f5c973a57a0 20 read_log 194370'14883 (0'0) promote 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by osd.346.0:3614 2014-08-12 20:30:33.167472 2014-08-13 18:06:06.682926 7f5c973a57a0 20 read_log 194370'14884 (194370'14883) modify 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by client.1508531.0:4984 2014-08-12 20:30:33.159380 2014-08-13 18:06:06.682947 7f5c973a57a0 20 read_log 194370'14885 (0'0) promote c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by osd.346.0:3616 2014-08-12 20:30:33.509835 2014-08-13 18:06:06.682963 7f5c973a57a0 20 read_log 194370'14886 (194370'14885) modify c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by client.1508531.0:5058 2014-08-12 20:30:33.502529 2014-08-13 18:06:06.682976 7f5c973a57a0 20 read_log 194370'14887 (0'0) promote db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by osd.346.0:3618 2014-08-12 20:30:33.970323 2014-08-13 18:06:06.683003 7f5c973a57a0 20 read_log 194370'14888 (194370'14887) modify db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by client.1508531.0:5176 2014-08-12 20:30:33.962698 2014-08-13 18:06:06.683019 7f5c973a57a0 20 read_log 194370'14889 (0'0) promote 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by osd.346.0:3620 2014-08-12 20:30:36.792082 2014-08-13 18:06:06.683046 7f5c973a57a0 20 read_log 194370'14890 (194370'14889) modify 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by client.1508531.0:5802 2014-08-12 20:30:36.783911 2014-08-13 18:06:06.683060 7f5c973a57a0 20 read_log 194370'14891 (0'0) promote aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by osd.346.0:3624 2014-08-12 20:30:38.611728 2014-08-13 18:06:06.683072 7f5c973a57a0 20 read_log 194370'14892 (194370'14891) modify aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by client.1508531.0:6194 2014-08-12 20:30:38.604425 2014-08-13 18:06:06.683085 7f5c973a57a0 20 read_log 194370'14893 (0'0) promote 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by osd.346.0:3626 2014-08-12 20:30:40.408272 2014-08-13 18:06:06.683097 7f5c973a57a0 20 read_log 194370'14894 (194370'14893) modify 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by client.1508531.0:6591 2014-08-12 20:30:40.400131 2014-08-13 18:06:06.683109 7f5c973a57a0 20 read_log 194370'14895 (0'0) promote 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by osd.346.0:3630 2014-08-12 20:30:43.446901 2014-08-13 18:06:06.683121 7f5c973a57a0 20 read_log 194370'14896 (194370'14895) modify 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by client.1508531.0:7217 2014-08-12 20:30:43.438717 2014-08-13 18:06:06.683134 7f5c973a57a0 20 read_log 194370'14897 (0'0) promote 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by osd.346.0:3640 2014-08-12 20:30:50.674363 2014-08-13 18:06:06.683151 7f5c973a57a0 20 read_log 194370'14898 (194370'14897) modify 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by client.1508531.0:8568 2014-08-12 20:30:50.666014 2014-08-13 18:06:06.683164 7f5c973a57a0 20 read_log 194370'14899 (0'0) promote bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by osd.346.0:3650 2014-08-12 20:31:00.519839 2014-08-13 18:06:06.683177 7f5c973a57a0 20 read_log 194370'14900 (194370'14899) modify bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by client.1508531.0:10214 2014-08-12 20:31:00.508225 2014-08-13 18:06:06.683190 7f5c973a57a0 20 read_log 194370'14901 (0'0) promote b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by osd.346.0:3654 2014-08-12 20:31:02.313507 2014-08-13 18:06:06.683202 7f5c973a57a0 20 read_log 194370'14902 (194370'14901) modify b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by client.1508531.0:10516 2014-08-12 20:31:02.307184 2014-08-13 18:06:06.683217 7f5c973a57a0 20 read_log 194370'14903 (0'0) promote b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by osd.346.0:3658 2014-08-12 20:31:03.603179 2014-08-13 18:06:06.683229 7f5c973a57a0 20 read_log 194370'14904 (194370'14903) modify b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by client.1508531.0:10715 2014-08-12 20:31:03.591547 2014-08-13 18:06:06.683241 7f5c973a57a0 20 read_log 194370'14905 (0'0) promote b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by osd.346.0:3660 2014-08-12 20:31:04.879706 2014-08-13 18:06:06.683254 7f5c973a57a0 20 read_log 194370'14906 (194370'14905) modify b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by client.1508531.0:10933 2014-08-12 20:31:04.869721 2014-08-13 18:06:06.683266 7f5c973a57a0 20 read_log 194370'14907 (0'0) promote 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by osd.346.0:3672 2014-08-12 20:31:11.122409 2014-08-13 18:06:06.683288 7f5c973a57a0 20 read_log 194370'14908 (194370'14907) modify 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by client.1508531.0:11967 2014-08-12 20:31:11.111281 2014-08-13 18:06:06.683304 7f5c973a57a0 20 read_log 194370'14909 (0'0) promote 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by osd.346.0:3674 2014-08-12 20:31:14.392210 2014-08-13 18:06:06.683325 7f5c973a57a0 20 read_log 194370'14910 (194370'14909) modify 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by client.1508531.0:12416 2014-08-12 20:31:14.383025 2014-08-13 18:06:06.683345 7f5c973a57a0 20 read_log 194370'14911 (0'0) promote 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by osd.346.0:3678 2014-08-12 20:31:15.089493 2014-08-13 18:06:06.683360 7f5c973a57a0 20 read_log 194370'14912 (194370'14911) modify 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by client.1508531.0:12511 2014-08-12 20:31:15.080922 2014-08-13 18:06:06.683373 7f5c973a57a0 20 read_log 194370'14913 (0'0) promote e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by osd.346.0:3680 2014-08-12 20:31:21.062559 2014-08-13 18:06:06.683394 7f5c973a57a0 20 read_log 194370'14914 (194370'14913) modify e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by client.1508531.0:13176 2014-08-12 20:31:21.054330 2014-08-13 18:06:06.683411 7f5c973a57a0 20 read_log 194370'14915 (0'0) promote 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by osd.346.0:3682 2014-08-12 20:31:21.839139 2014-08-13 18:06:06.683423 7f5c973a57a0 20 read_log 194370'14916 (194370'14915) modify 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by client.1508531.0:13258 2014-08-12 20:31:21.830476 2014-08-13 18:06:06.683439 7f5c973a57a0 20 read_log 194377'14917 (0'0) promote 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by osd.346.0:3686 2014-08-12 20:32:57.802897 2014-08-13 18:06:06.683454 7f5c973a57a0 20 read_log 194377'14918 (194377'14917) modify 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by client.2231190.0:153 2014-08-12 20:32:57.792894 2014-08-13 18:06:06.683472 7f5c973a57a0 20 read_log 194377'14919 (0'0) promote c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by osd.346.0:3702 2014-08-12 20:33:02.103922 2014-08-13 18:06:06.683487 7f5c973a57a0 20 read_log 194377'14920 (194377'14919) modify c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by client.2231190.0:1209 2014-08-12 20:33:02.094827 2014-08-13 18:06:06.683500 7f5c973a57a0 20 read_log 194377'14921 (0'0) promote 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by osd.346.0:3710 2014-08-12 20:33:08.015672 2014-08-13 18:06:06.683515 7f5c973a57a0 20 read_log 194377'14922 (194377'14921) modify 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by client.2231190.0:2671 2014-08-12 20:33:08.007800 2014-08-13 18:06:06.683528 7f5c973a57a0 20 read_log 194377'14923 (0'0) promote 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by osd.346.0:3714 2014-08-12 20:33:08.889973 2014-08-13 18:06:06.683549 7f5c973a57a0 20 read_log 194377'14924 (194377'14923) modify 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by client.2231190.0:2903 2014-08-12 20:33:08.882724 2014-08-13 18:06:06.683567 7f5c973a57a0 20 read_log 194377'14925 (0'0) promote cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by osd.346.0:3716 2014-08-12 20:33:09.489008 2014-08-13 18:06:06.683580 7f5c973a57a0 20 read_log 194377'14926 (194377'14925) modify cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by client.2231190.0:3040 2014-08-12 20:33:09.482049 2014-08-13 18:06:06.683592 7f5c973a57a0 20 read_log 194377'14927 (0'0) promote f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by osd.346.0:3722 2014-08-12 20:33:10.673358 2014-08-13 18:06:06.683618 7f5c973a57a0 20 read_log 194377'14928 (194377'14927) modify f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by client.2231190.0:3360 2014-08-12 20:33:10.665803 2014-08-13 18:06:06.683633 7f5c973a57a0 20 read_log 194380'14929 (0'0) promote 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by osd.346.0:3736 2014-08-12 20:33:23.603589 2014-08-13 18:06:06.683654 7f5c973a57a0 20 read_log 194380'14930 (194380'14929) modify 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by client.2231190.0:5043 2014-08-12 20:33:23.594993 2014-08-13 18:06:06.683672 7f5c973a57a0 20 read_log 194380'14931 (0'0) promote bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by osd.346.0:3738 2014-08-12 20:33:24.371968 2014-08-13 18:06:06.683685 7f5c973a57a0 20 read_log 194380'14932 (194380'14931) modify bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by client.2231190.0:5097 2014-08-12 20:33:24.362870 2014-08-13 18:06:06.683697 7f5c973a57a0 20 read_log 194383'14933 (0'0) promote 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by osd.346.0:3754 2014-08-12 20:34:33.235487 2014-08-13 18:06:06.683712 7f5c973a57a0 20 read_log 194383'14934 (194383'14933) modify 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by client.2231190.0:7665 2014-08-12 20:34:33.226508 2014-08-13 18:06:06.683725 7f5c973a57a0 20 read_log 194393'14935 (194393'14935) modify e2/hit_set_24.e2_archive_2014-08-12 20:37:07.990544_2014-08-12 20:47:26.960810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.961032 2014-08-13 18:06:06.683738 7f5c973a57a0 20 read_log 194393'14936 (194215'14871) delete e2/hit_set_24.e2_archive_2014-08-12 15:39:39.946574_2014-08-12 17:24:43.501290/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.961032 2014-08-13 18:06:06.683761 7f5c973a57a0 20 read_log 194393'14937 (0'0) promote 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by osd.346.0:3763 2014-08-12 20:47:26.964219 2014-08-13 18:06:06.683777 7f5c973a57a0 20 read_log 194393'14938 (194393'14937) modify 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by client.1064095.0:249 2014-08-12 20:47:26.955489 2014-08-13 18:06:06.683790 7f5c973a57a0 20 read_log 194393'14939 (0'0) promote 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by osd.346.0:3765 2014-08-12 20:47:28.709802 2014-08-13 18:06:06.683816 7f5c973a57a0 20 read_log 194393'14940 (194393'14939) modify 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by client.1064095.0:615 2014-08-12 20:47:28.700564 2014-08-13 18:06:06.683846 7f5c973a57a0 20 read_log 194393'14941 (194393'14941) modify e2/hit_set_24.e2_archive_2014-08-12 20:47:26.961037_2014-08-12 20:52:32.946845/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.947226 2014-08-13 18:06:06.683861 7f5c973a57a0 20 read_log 194393'14942 (194393'14935) delete e2/hit_set_24.e2_archive_2014-08-12 20:37:07.990544_2014-08-12 20:47:26.960810/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:32.947226 2014-08-13 18:06:06.683884 7f5c973a57a0 20 read_log 194393'14943 (0'0) promote 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by osd.346.0:3776 2014-08-12 20:52:32.951655 2014-08-13 18:06:06.683900 7f5c973a57a0 20 read_log 194393'14944 (194393'14943) modify 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by client.1064422.0:108 2014-08-12 20:52:32.942063 2014-08-13 18:06:06.683913 7f5c973a57a0 20 read_log 194393'14945 (0'0) promote 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by osd.346.0:3778 2014-08-12 20:52:33.018064 2014-08-13 18:06:06.683929 7f5c973a57a0 20 read_log 194393'14946 (194393'14945) modify 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by client.1064422.0:125 2014-08-12 20:52:33.010604 2014-08-13 18:06:06.683974 7f5c973a57a0 20 read_log 194393'14947 (0'0) promote c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by osd.346.0:3780 2014-08-12 20:52:56.623445 2014-08-13 18:06:06.683991 7f5c973a57a0 20 read_log 194393'14948 (194393'14947) modify c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by client.2235069.0:61 2014-08-12 20:52:56.613072 2014-08-13 18:06:06.684016 7f5c973a57a0 20 read_log 194393'14949 (0'0) promote ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by osd.346.0:3782 2014-08-12 20:52:56.989152 2014-08-13 18:06:06.684029 7f5c973a57a0 20 read_log 194393'14950 (194393'14949) modify ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by client.2235069.0:148 2014-08-12 20:52:56.980170 2014-08-13 18:06:06.684046 7f5c973a57a0 20 read_log 194393'14951 (0'0) promote 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by osd.346.0:3786 2014-08-12 20:52:57.712450 2014-08-13 18:06:06.684059 7f5c973a57a0 20 read_log 194393'14952 (194393'14951) modify 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by client.2235069.0:260 2014-08-12 20:52:57.702244 2014-08-13 18:06:06.684071 7f5c973a57a0 20 read_log 194416'14953 (194416'14953) modify e2/hit_set_24.e2_archive_2014-08-12 20:52:32.947277_2014-08-13 15:28:33.527799/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 18:06:06.684084 7f5c973a57a0 20 read_log 194416'14954 (194393'14941) delete e2/hit_set_24.e2_archive_2014-08-12 20:47:26.961037_2014-08-12 20:52:32.946845/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 18:06:06.684096 7f5c973a57a0 20 read_log 194416'14955 (0'0) promote 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by osd.346.0:3935 2014-08-13 15:28:33.530674 2014-08-13 18:06:06.684108 7f5c973a57a0 20 read_log 194416'14956 (194416'14955) modify 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by client.2441958.0:765 2014-08-13 15:28:33.521857 2014-08-13 18:06:06.684121 7f5c973a57a0 20 read_log 194416'14957 (0'0) promote 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by osd.346.0:3939 2014-08-13 15:28:35.607515 2014-08-13 18:06:06.684141 7f5c973a57a0 20 read_log 194416'14958 (194416'14957) modify 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by client.2441958.0:1289 2014-08-13 15:28:35.598104 2014-08-13 18:06:06.684162 7f5c973a57a0 20 read_log 194416'14959 (0'0) promote 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by osd.346.0:3944 2014-08-13 15:28:36.844152 2014-08-13 18:06:06.684175 7f5c973a57a0 20 read_log 194416'14960 (194416'14959) modify 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by client.2441958.0:1616 2014-08-13 15:28:36.837034 2014-08-13 18:06:06.684187 7f5c973a57a0 20 read_log 194416'14961 (0'0) promote 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by osd.346.0:3954 2014-08-13 15:28:42.638007 2014-08-13 18:06:06.684200 7f5c973a57a0 20 read_log 194416'14962 (194416'14961) modify 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by client.2441958.0:3150 2014-08-13 15:28:42.629635 2014-08-13 18:06:06.684212 7f5c973a57a0 20 read_log 194416'14963 (0'0) promote 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by osd.346.0:3956 2014-08-13 15:28:42.651643 2014-08-13 18:06:06.684227 7f5c973a57a0 20 read_log 194416'14964 (194416'14963) modify 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by client.2441958.0:3154 2014-08-13 15:28:42.643803 2014-08-13 18:06:06.684239 7f5c973a57a0 20 read_log 194416'14965 (0'0) promote f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by osd.346.0:3964 2014-08-13 15:28:45.677995 2014-08-13 18:06:06.684259 7f5c973a57a0 20 read_log 194416'14966 (194416'14965) modify f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by client.2441958.0:3932 2014-08-13 15:28:45.670425 2014-08-13 18:06:06.684277 7f5c973a57a0 20 read_log 194416'14967 (0'0) promote 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by osd.346.0:3966 2014-08-13 15:28:46.835923 2014-08-13 18:06:06.684290 7f5c973a57a0 20 read_log 194416'14968 (194416'14967) modify 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by client.2441958.0:4223 2014-08-13 15:28:46.828611 2014-08-13 18:06:06.684302 7f5c973a57a0 20 read_log 194416'14969 (0'0) promote 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by osd.346.0:3970 2014-08-13 15:28:47.334968 2014-08-13 18:06:06.684314 7f5c973a57a0 20 read_log 194416'14970 (194416'14969) modify 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by client.2441958.0:4357 2014-08-13 15:28:47.328858 2014-08-13 18:06:06.684332 7f5c973a57a0 20 read_log 194416'14971 (0'0) promote 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by osd.346.0:3980 2014-08-13 15:28:51.005013 2014-08-13 18:06:06.684353 7f5c973a57a0 20 read_log 194416'14972 (194416'14971) modify 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by client.2441958.0:5289 2014-08-13 15:28:50.995585 2014-08-13 18:06:06.684372 7f5c973a57a0 20 read_log 194416'14973 (0'0) promote 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by osd.346.0:3998 2014-08-13 15:28:57.693112 2014-08-13 18:06:06.684384 7f5c973a57a0 20 read_log 194416'14974 (194416'14973) modify 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by client.2441958.0:6976 2014-08-13 15:28:57.682311 2014-08-13 18:06:06.684408 7f5c973a57a0 20 read_log 194416'14975 (0'0) promote c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by osd.346.0:4004 2014-08-13 15:29:01.052099 2014-08-13 18:06:06.684421 7f5c973a57a0 20 read_log 194416'14976 (194416'14975) modify c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by client.2441958.0:7872 2014-08-13 15:29:01.043368 2014-08-13 18:06:06.684433 7f5c973a57a0 20 read_log 194416'14977 (0'0) promote a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by osd.346.0:4008 2014-08-13 15:29:02.435944 2014-08-13 18:06:06.684448 7f5c973a57a0 20 read_log 194416'14978 (194416'14977) modify a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by client.2441958.0:8240 2014-08-13 15:29:02.426744 2014-08-13 18:06:06.684484 7f5c973a57a0 20 read_log 194416'14979 (0'0) promote ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by osd.346.0:4016 2014-08-13 15:29:07.697311 2014-08-13 18:06:06.684497 7f5c973a57a0 20 read_log 194416'14980 (194416'14979) modify ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by client.2441958.0:9582 2014-08-13 15:29:07.689454 2014-08-13 18:06:06.684518 7f5c973a57a0 20 read_log 194416'14981 (0'0) promote 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by osd.346.0:4018 2014-08-13 15:29:08.082009 2014-08-13 18:06:06.684534 7f5c973a57a0 20 read_log 194416'14982 (194416'14981) modify 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by client.2441958.0:9681 2014-08-13 15:29:08.072489 2014-08-13 18:06:06.684547 7f5c973a57a0 20 read_log 194416'14983 (0'0) promote 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by osd.346.0:4020 2014-08-13 15:29:14.578390 2014-08-13 18:06:06.684559 7f5c973a57a0 20 read_log 194416'14984 (194416'14983) modify 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by client.2441958.0:11271 2014-08-13 15:29:14.563606 2014-08-13 18:06:06.684572 7f5c973a57a0 20 read_log 194416'14985 (0'0) promote baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by osd.346.0:4022 2014-08-13 15:29:15.199098 2014-08-13 18:06:06.684591 7f5c973a57a0 20 read_log 194416'14986 (194416'14985) modify baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by client.2441958.0:11427 2014-08-13 15:29:15.190979 2014-08-13 18:06:06.684606 7f5c973a57a0 20 read_log 194416'14987 (0'0) promote f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by osd.346.0:4032 2014-08-13 15:29:20.960503 2014-08-13 18:06:06.684619 7f5c973a57a0 20 read_log 194416'14988 (194416'14987) modify f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by client.2441958.0:12783 2014-08-13 15:29:20.951520 2014-08-13 18:06:06.684631 7f5c973a57a0 20 read_log 194416'14989 (0'0) promote b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by osd.346.0:4036 2014-08-13 15:29:26.478944 2014-08-13 18:06:06.684643 7f5c973a57a0 20 read_log 194416'14990 (194416'14989) modify b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by client.2441958.0:14070 2014-08-13 15:29:26.470333 2014-08-13 18:06:06.684656 7f5c973a57a0 20 read_log 194416'14991 (0'0) promote 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by osd.346.0:4040 2014-08-13 15:29:26.815135 2014-08-13 18:06:06.684668 7f5c973a57a0 20 read_log 194416'14992 (194416'14991) modify 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by client.2441958.0:14159 2014-08-13 15:29:26.807165 2014-08-13 18:06:06.684680 7f5c973a57a0 20 read_log 194416'14993 (0'0) promote 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by osd.346.0:4042 2014-08-13 15:29:27.024088 2014-08-13 18:06:06.684692 7f5c973a57a0 20 read_log 194416'14994 (194416'14993) modify 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by client.2441958.0:14212 2014-08-13 15:29:27.015024 2014-08-13 18:06:06.684705 7f5c973a57a0 20 read_log 194416'14995 (0'0) promote 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by osd.346.0:4060 2014-08-13 15:29:43.046581 2014-08-13 18:06:06.684718 7f5c973a57a0 20 read_log 194416'14996 (194416'14995) modify 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by client.2441958.0:16700 2014-08-13 15:29:43.037136 2014-08-13 18:06:06.684730 7f5c973a57a0 20 read_log 194422'14997 (0'0) promote 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by osd.346.0:4076 2014-08-13 15:32:15.767448 2014-08-13 18:06:06.684744 7f5c973a57a0 20 read_log 194422'14998 (194422'14997) modify 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by client.1148203.0:172 2014-08-13 15:32:15.757634 2014-08-13 18:06:06.684756 7f5c973a57a0 20 read_log 194422'14999 (0'0) promote 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by osd.346.0:4082 2014-08-13 15:32:18.409546 2014-08-13 18:06:06.684768 7f5c973a57a0 20 read_log 194422'15000 (194422'14999) modify 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by client.1148203.0:766 2014-08-13 15:32:18.402124 2014-08-13 18:06:06.684799 7f5c973a57a0 20 read_log 194422'15001 (0'0) promote 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by osd.346.0:4090 2014-08-13 15:32:21.744796 2014-08-13 18:06:06.684816 7f5c973a57a0 20 read_log 194422'15002 (194422'15001) modify 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by client.1148203.0:1621 2014-08-13 15:32:21.736020 2014-08-13 18:06:06.684829 7f5c973a57a0 20 read_log 194422'15003 (0'0) promote deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by osd.346.0:4104 2014-08-13 15:32:26.264497 2014-08-13 18:06:06.684841 7f5c973a57a0 20 read_log 194422'15004 (194422'15003) modify deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by client.1148203.0:2794 2014-08-13 15:32:26.256164 2014-08-13 18:06:06.684864 7f5c973a57a0 20 read_log 194422'15005 (0'0) promote 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by osd.346.0:4106 2014-08-13 15:32:26.283030 2014-08-13 18:06:06.684879 7f5c973a57a0 20 read_log 194422'15006 (194422'15005) modify 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by client.1148203.0:2799 2014-08-13 15:32:26.273946 2014-08-13 18:06:06.684892 7f5c973a57a0 20 read_log 194422'15007 (0'0) promote 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by osd.346.0:4114 2014-08-13 15:32:30.977799 2014-08-13 18:06:06.684905 7f5c973a57a0 20 read_log 194422'15008 (194422'15007) modify 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by client.1148203.0:3996 2014-08-13 15:32:30.971523 2014-08-13 18:06:06.684923 7f5c973a57a0 20 read_log 194422'15009 (0'0) promote f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by osd.346.0:4118 2014-08-13 15:32:34.425789 2014-08-13 18:06:06.684937 7f5c973a57a0 20 read_log 194422'15010 (194422'15009) modify f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by client.1148203.0:4891 2014-08-13 15:32:34.417029 2014-08-13 18:06:06.684950 7f5c973a57a0 20 read_log 194422'15011 (0'0) promote b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by osd.346.0:4156 2014-08-13 15:33:02.939383 2014-08-13 18:06:06.684964 7f5c973a57a0 20 read_log 194422'15012 (194422'15011) modify b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by client.1148203.0:9345 2014-08-13 15:33:02.932357 2014-08-13 18:06:06.684976 7f5c973a57a0 20 read_log 194422'15013 (0'0) promote d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by osd.346.0:4162 2014-08-13 15:33:04.859400 2014-08-13 18:06:06.684989 7f5c973a57a0 20 read_log 194422'15014 (194422'15013) modify d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by client.1148203.0:9539 2014-08-13 15:33:04.848878 2014-08-13 18:06:06.685005 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.685532 7f5c973a57a0 10 read_log checking for missing items over interval (194370'14879,194422'15014] 2014-08-13 18:06:06.685581 7f5c973a57a0 15 read_log missing 194422'15014 (194422'15013) modify d7f4dce2/benchmark_data_storage0111-ib.pocstor_41231_object9538/head//24 by client.1148203.0:9539 2014-08-13 15:33:04.848878 2014-08-13 18:06:06.685621 7f5c973a57a0 15 read_log missing 194422'15012 (194422'15011) modify b06ed8e2/benchmark_data_storage0111-ib.pocstor_41231_object9344/head//24 by client.1148203.0:9345 2014-08-13 15:33:02.932357 2014-08-13 18:06:06.685658 7f5c973a57a0 15 read_log missing 194422'15010 (194422'15009) modify f1506ce2/benchmark_data_storage0111-ib.pocstor_41231_object4890/head//24 by client.1148203.0:4891 2014-08-13 15:32:34.417029 2014-08-13 18:06:06.685694 7f5c973a57a0 15 read_log missing 194422'15008 (194422'15007) modify 1a3fdce2/benchmark_data_storage0111-ib.pocstor_41231_object3995/head//24 by client.1148203.0:3996 2014-08-13 15:32:30.971523 2014-08-13 18:06:06.685730 7f5c973a57a0 15 read_log missing 194422'15006 (194422'15005) modify 995584e2/benchmark_data_storage0111-ib.pocstor_41231_object2798/head//24 by client.1148203.0:2799 2014-08-13 15:32:26.273946 2014-08-13 18:06:06.685765 7f5c973a57a0 15 read_log missing 194422'15004 (194422'15003) modify deaf0ce2/benchmark_data_storage0111-ib.pocstor_41231_object2793/head//24 by client.1148203.0:2794 2014-08-13 15:32:26.256164 2014-08-13 18:06:06.685801 7f5c973a57a0 15 read_log missing 194422'15002 (194422'15001) modify 1b603ce2/benchmark_data_storage0111-ib.pocstor_41231_object1620/head//24 by client.1148203.0:1621 2014-08-13 15:32:21.736020 2014-08-13 18:06:06.685837 7f5c973a57a0 15 read_log missing 194422'15000 (194422'14999) modify 480bce2/benchmark_data_storage0111-ib.pocstor_41231_object765/head//24 by client.1148203.0:766 2014-08-13 15:32:18.402124 2014-08-13 18:06:06.685873 7f5c973a57a0 15 read_log missing 194422'14998 (194422'14997) modify 255d4ce2/benchmark_data_storage0111-ib.pocstor_41231_object171/head//24 by client.1148203.0:172 2014-08-13 15:32:15.757634 2014-08-13 18:06:06.685908 7f5c973a57a0 15 read_log missing 194416'14996 (194416'14995) modify 1c3c00e2/benchmark_data_storage0111-ib.pocstor_19103_object16699/head//24 by client.2441958.0:16700 2014-08-13 15:29:43.037136 2014-08-13 18:06:06.685944 7f5c973a57a0 15 read_log missing 194416'14994 (194416'14993) modify 9ecd3ce2/benchmark_data_storage0111-ib.pocstor_19103_object14211/head//24 by client.2441958.0:14212 2014-08-13 15:29:27.015024 2014-08-13 18:06:06.685982 7f5c973a57a0 15 read_log missing 194416'14992 (194416'14991) modify 469554e2/benchmark_data_storage0111-ib.pocstor_19103_object14158/head//24 by client.2441958.0:14159 2014-08-13 15:29:26.807165 2014-08-13 18:06:06.686017 7f5c973a57a0 15 read_log missing 194416'14990 (194416'14989) modify b6a4a0e2/benchmark_data_storage0111-ib.pocstor_19103_object14069/head//24 by client.2441958.0:14070 2014-08-13 15:29:26.470333 2014-08-13 18:06:06.686053 7f5c973a57a0 15 read_log missing 194416'14988 (194416'14987) modify f97980e2/benchmark_data_storage0111-ib.pocstor_19103_object12782/head//24 by client.2441958.0:12783 2014-08-13 15:29:20.951520 2014-08-13 18:06:06.686095 7f5c973a57a0 15 read_log missing 194416'14986 (194416'14985) modify baab74e2/benchmark_data_storage0111-ib.pocstor_19103_object11426/head//24 by client.2441958.0:11427 2014-08-13 15:29:15.190979 2014-08-13 18:06:06.686131 7f5c973a57a0 15 read_log missing 194416'14984 (194416'14983) modify 48c618e2/benchmark_data_storage0111-ib.pocstor_19103_object11270/head//24 by client.2441958.0:11271 2014-08-13 15:29:14.563606 2014-08-13 18:06:06.686180 7f5c973a57a0 15 read_log missing 194416'14982 (194416'14981) modify 8d4464e2/benchmark_data_storage0111-ib.pocstor_19103_object9680/head//24 by client.2441958.0:9681 2014-08-13 15:29:08.072489 2014-08-13 18:06:06.686218 7f5c973a57a0 15 read_log missing 194416'14980 (194416'14979) modify ebeee8e2/benchmark_data_storage0111-ib.pocstor_19103_object9581/head//24 by client.2441958.0:9582 2014-08-13 15:29:07.689454 2014-08-13 18:06:06.686266 7f5c973a57a0 15 read_log missing 194416'14978 (194416'14977) modify a6ce58e2/benchmark_data_storage0111-ib.pocstor_19103_object8239/head//24 by client.2441958.0:8240 2014-08-13 15:29:02.426744 2014-08-13 18:06:06.686302 7f5c973a57a0 15 read_log missing 194416'14976 (194416'14975) modify c6b454e2/benchmark_data_storage0111-ib.pocstor_19103_object7871/head//24 by client.2441958.0:7872 2014-08-13 15:29:01.043368 2014-08-13 18:06:06.686338 7f5c973a57a0 15 read_log missing 194416'14974 (194416'14973) modify 658590e2/benchmark_data_storage0111-ib.pocstor_19103_object6975/head//24 by client.2441958.0:6976 2014-08-13 15:28:57.682311 2014-08-13 18:06:06.686387 7f5c973a57a0 15 read_log missing 194416'14972 (194416'14971) modify 19fc4e2/benchmark_data_storage0111-ib.pocstor_19103_object5288/head//24 by client.2441958.0:5289 2014-08-13 15:28:50.995585 2014-08-13 18:06:06.686437 7f5c973a57a0 15 read_log missing 194416'14970 (194416'14969) modify 6a2520e2/benchmark_data_storage0111-ib.pocstor_19103_object4356/head//24 by client.2441958.0:4357 2014-08-13 15:28:47.328858 2014-08-13 18:06:06.686475 7f5c973a57a0 15 read_log missing 194416'14968 (194416'14967) modify 1b6350e2/benchmark_data_storage0111-ib.pocstor_19103_object4222/head//24 by client.2441958.0:4223 2014-08-13 15:28:46.828611 2014-08-13 18:06:06.686511 7f5c973a57a0 15 read_log missing 194416'14966 (194416'14965) modify f74858e2/benchmark_data_storage0111-ib.pocstor_19103_object3931/head//24 by client.2441958.0:3932 2014-08-13 15:28:45.670425 2014-08-13 18:06:06.686547 7f5c973a57a0 15 read_log missing 194416'14964 (194416'14963) modify 1621e4e2/benchmark_data_storage0111-ib.pocstor_19103_object3153/head//24 by client.2441958.0:3154 2014-08-13 15:28:42.643803 2014-08-13 18:06:06.686583 7f5c973a57a0 15 read_log missing 194416'14962 (194416'14961) modify 5a1578e2/benchmark_data_storage0111-ib.pocstor_19103_object3149/head//24 by client.2441958.0:3150 2014-08-13 15:28:42.629635 2014-08-13 18:06:06.686619 7f5c973a57a0 15 read_log missing 194416'14960 (194416'14959) modify 529ac0e2/benchmark_data_storage0111-ib.pocstor_19103_object1615/head//24 by client.2441958.0:1616 2014-08-13 15:28:36.837034 2014-08-13 18:06:06.686655 7f5c973a57a0 15 read_log missing 194416'14958 (194416'14957) modify 29bb1ce2/benchmark_data_storage0111-ib.pocstor_19103_object1288/head//24 by client.2441958.0:1289 2014-08-13 15:28:35.598104 2014-08-13 18:06:06.686690 7f5c973a57a0 15 read_log missing 194416'14956 (194416'14955) modify 35a728e2/benchmark_data_storage0111-ib.pocstor_19103_object764/head//24 by client.2441958.0:765 2014-08-13 15:28:33.521857 2014-08-13 18:06:06.686730 7f5c973a57a0 15 read_log missing 194416'14953 (194416'14953) modify e2/hit_set_24.e2_archive_2014-08-12 20:52:32.947277_2014-08-13 15:28:33.527799/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:33.528066 2014-08-13 18:06:06.686766 7f5c973a57a0 15 read_log missing 194393'14952 (194393'14951) modify 308af0e2/benchmark_data_storage0101-ib.pocstor_6819_object259/head//24 by client.2235069.0:260 2014-08-12 20:52:57.702244 2014-08-13 18:06:06.686802 7f5c973a57a0 15 read_log missing 194393'14950 (194393'14949) modify ae3f80e2/benchmark_data_storage0101-ib.pocstor_6819_object147/head//24 by client.2235069.0:148 2014-08-12 20:52:56.980170 2014-08-13 18:06:06.686838 7f5c973a57a0 15 read_log missing 194393'14948 (194393'14947) modify c0cb9ce2/benchmark_data_storage0101-ib.pocstor_6819_object60/head//24 by client.2235069.0:61 2014-08-12 20:52:56.613072 2014-08-13 18:06:06.686874 7f5c973a57a0 15 read_log missing 194393'14946 (194393'14945) modify 76c3bce2/benchmark_data_storage0111-ib.pocstor_48304_object124/head//24 by client.1064422.0:125 2014-08-12 20:52:33.010604 2014-08-13 18:06:06.686910 7f5c973a57a0 15 read_log missing 194393'14944 (194393'14943) modify 126254e2/benchmark_data_storage0111-ib.pocstor_48304_object107/head//24 by client.1064422.0:108 2014-08-12 20:52:32.942063 2014-08-13 18:06:06.686949 7f5c973a57a0 15 read_log missing 194393'14940 (194393'14939) modify 1fc6c8e2/benchmark_data_storage0111-ib.pocstor_24170_object614/head//24 by client.1064095.0:615 2014-08-12 20:47:28.700564 2014-08-13 18:06:06.686985 7f5c973a57a0 15 read_log missing 194393'14938 (194393'14937) modify 150b84e2/benchmark_data_storage0111-ib.pocstor_24170_object248/head//24 by client.1064095.0:249 2014-08-12 20:47:26.955489 2014-08-13 18:06:06.687025 7f5c973a57a0 15 read_log missing 194383'14934 (194383'14933) modify 605ee0e2/benchmark_data_storage0111-ib.pocstor_25065_object7664/head//24 by client.2231190.0:7665 2014-08-12 20:34:33.226508 2014-08-13 18:06:06.687061 7f5c973a57a0 15 read_log missing 194380'14932 (194380'14931) modify bff720e2/benchmark_data_storage0111-ib.pocstor_25065_object5096/head//24 by client.2231190.0:5097 2014-08-12 20:33:24.362870 2014-08-13 18:06:06.687097 7f5c973a57a0 15 read_log missing 194380'14930 (194380'14929) modify 4791e8e2/benchmark_data_storage0111-ib.pocstor_25065_object5042/head//24 by client.2231190.0:5043 2014-08-12 20:33:23.594993 2014-08-13 18:06:06.687133 7f5c973a57a0 15 read_log missing 194377'14928 (194377'14927) modify f4e08ce2/benchmark_data_storage0111-ib.pocstor_25065_object3359/head//24 by client.2231190.0:3360 2014-08-12 20:33:10.665803 2014-08-13 18:06:06.687175 7f5c973a57a0 15 read_log missing 194377'14926 (194377'14925) modify cf080ce2/benchmark_data_storage0111-ib.pocstor_25065_object3039/head//24 by client.2231190.0:3040 2014-08-12 20:33:09.482049 2014-08-13 18:06:06.687211 7f5c973a57a0 15 read_log missing 194377'14924 (194377'14923) modify 2cda4e2/benchmark_data_storage0111-ib.pocstor_25065_object2902/head//24 by client.2231190.0:2903 2014-08-12 20:33:08.882724 2014-08-13 18:06:06.687250 7f5c973a57a0 15 read_log missing 194377'14922 (194377'14921) modify 6adeece2/benchmark_data_storage0111-ib.pocstor_25065_object2670/head//24 by client.2231190.0:2671 2014-08-12 20:33:08.007800 2014-08-13 18:06:06.687287 7f5c973a57a0 15 read_log missing 194377'14920 (194377'14919) modify c231e8e2/benchmark_data_storage0111-ib.pocstor_25065_object1208/head//24 by client.2231190.0:1209 2014-08-12 20:33:02.094827 2014-08-13 18:06:06.687328 7f5c973a57a0 15 read_log missing 194377'14918 (194377'14917) modify 63056ce2/benchmark_data_storage0111-ib.pocstor_25065_object152/head//24 by client.2231190.0:153 2014-08-12 20:32:57.792894 2014-08-13 18:06:06.687365 7f5c973a57a0 15 read_log missing 194370'14916 (194370'14915) modify 16040e2/benchmark_data_storage0111-ib.pocstor_12495_object13257/head//24 by client.1508531.0:13258 2014-08-12 20:31:21.830476 2014-08-13 18:06:06.687422 7f5c973a57a0 15 read_log missing 194370'14914 (194370'14913) modify e0cd0e2/benchmark_data_storage0111-ib.pocstor_12495_object13175/head//24 by client.1508531.0:13176 2014-08-12 20:31:21.054330 2014-08-13 18:06:06.687462 7f5c973a57a0 15 read_log missing 194370'14912 (194370'14911) modify 29c470e2/benchmark_data_storage0111-ib.pocstor_12495_object12510/head//24 by client.1508531.0:12511 2014-08-12 20:31:15.080922 2014-08-13 18:06:06.687511 7f5c973a57a0 15 read_log missing 194370'14910 (194370'14909) modify 37e58e2/benchmark_data_storage0111-ib.pocstor_12495_object12415/head//24 by client.1508531.0:12416 2014-08-12 20:31:14.383025 2014-08-13 18:06:06.687548 7f5c973a57a0 15 read_log missing 194370'14908 (194370'14907) modify 74c248e2/benchmark_data_storage0111-ib.pocstor_12495_object11966/head//24 by client.1508531.0:11967 2014-08-12 20:31:11.111281 2014-08-13 18:06:06.687585 7f5c973a57a0 15 read_log missing 194370'14906 (194370'14905) modify b53bb8e2/benchmark_data_storage0111-ib.pocstor_12495_object10932/head//24 by client.1508531.0:10933 2014-08-12 20:31:04.869721 2014-08-13 18:06:06.687622 7f5c973a57a0 15 read_log missing 194370'14904 (194370'14903) modify b0e3a8e2/benchmark_data_storage0111-ib.pocstor_12495_object10714/head//24 by client.1508531.0:10715 2014-08-12 20:31:03.591547 2014-08-13 18:06:06.687658 7f5c973a57a0 15 read_log missing 194370'14902 (194370'14901) modify b50840e2/benchmark_data_storage0111-ib.pocstor_12495_object10515/head//24 by client.1508531.0:10516 2014-08-12 20:31:02.307184 2014-08-13 18:06:06.687694 7f5c973a57a0 15 read_log missing 194370'14900 (194370'14899) modify bad4d8e2/benchmark_data_storage0111-ib.pocstor_12495_object10213/head//24 by client.1508531.0:10214 2014-08-12 20:31:00.508225 2014-08-13 18:06:06.687731 7f5c973a57a0 15 read_log missing 194370'14898 (194370'14897) modify 2668ce2/benchmark_data_storage0111-ib.pocstor_12495_object8567/head//24 by client.1508531.0:8568 2014-08-12 20:30:50.666014 2014-08-13 18:06:06.687767 7f5c973a57a0 15 read_log missing 194370'14896 (194370'14895) modify 51c82ce2/benchmark_data_storage0111-ib.pocstor_12495_object7216/head//24 by client.1508531.0:7217 2014-08-12 20:30:43.438717 2014-08-13 18:06:06.687803 7f5c973a57a0 15 read_log missing 194370'14894 (194370'14893) modify 1e2e90e2/benchmark_data_storage0111-ib.pocstor_12495_object6590/head//24 by client.1508531.0:6591 2014-08-12 20:30:40.400131 2014-08-13 18:06:06.687843 7f5c973a57a0 15 read_log missing 194370'14892 (194370'14891) modify aa1678e2/benchmark_data_storage0111-ib.pocstor_12495_object6193/head//24 by client.1508531.0:6194 2014-08-12 20:30:38.604425 2014-08-13 18:06:06.687880 7f5c973a57a0 15 read_log missing 194370'14890 (194370'14889) modify 568fbce2/benchmark_data_storage0111-ib.pocstor_12495_object5801/head//24 by client.1508531.0:5802 2014-08-12 20:30:36.783911 2014-08-13 18:06:06.687916 7f5c973a57a0 15 read_log missing 194370'14888 (194370'14887) modify db5ae8e2/benchmark_data_storage0111-ib.pocstor_12495_object5175/head//24 by client.1508531.0:5176 2014-08-12 20:30:33.962698 2014-08-13 18:06:06.687952 7f5c973a57a0 15 read_log missing 194370'14886 (194370'14885) modify c9253ce2/benchmark_data_storage0111-ib.pocstor_12495_object5057/head//24 by client.1508531.0:5058 2014-08-12 20:30:33.502529 2014-08-13 18:06:06.687988 7f5c973a57a0 15 read_log missing 194370'14884 (194370'14883) modify 46e8a8e2/benchmark_data_storage0111-ib.pocstor_12495_object4983/head//24 by client.1508531.0:4984 2014-08-12 20:30:33.159380 2014-08-13 18:06:06.688025 7f5c973a57a0 15 read_log missing 194370'14882 (194370'14881) modify b404e8e2/benchmark_data_storage0111-ib.pocstor_12495_object3563/head//24 by client.1508531.0:3564 2014-08-12 20:30:27.639554 2014-08-13 18:06:06.688061 7f5c973a57a0 15 read_log missing 194370'14880 (194370'14879) modify 23ae90e2/benchmark_data_storage0111-ib.pocstor_12495_object877/head//24 by client.1508531.0:878 2014-08-12 20:30:17.678096 2014-08-13 18:06:06.688073 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.688102 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive m=66] handle_loaded 2014-08-13 18:06:06.688120 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Initial 0.016919 0 0.000000 2014-08-13 18:06:06.688133 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset 2014-08-13 18:06:06.688142 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=0 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194487 2014-08-13 18:06:06.688158 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] log((192834'11882,194422'15014], crt=194422'15014) 2014-08-13 18:06:06.688170 7f5c973a57a0 10 osd.4 194487 pgid 24.136 coll 24.136_head 2014-08-13 18:06:06.688314 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.136 2014-08-13 18:06:06.688331 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.688365 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.136(unlocked)] enter Initial 2014-08-13 18:06:06.688377 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.136(unlocked)] enter NotTrimming 2014-08-13 18:06:06.688557 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.688694 7f5c973a57a0 20 read_log 192834'12418 (0'0) promote 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:13898 2014-08-11 15:24:19.282007 2014-08-13 18:06:06.688716 7f5c973a57a0 20 read_log 192834'12419 (192834'12418) modify 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by client.933448.0:9549 2014-08-11 15:24:19.274646 2014-08-13 18:06:06.688729 7f5c973a57a0 20 read_log 192834'12428 (0'0) promote bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:13908 2014-08-11 15:24:20.927952 2014-08-13 18:06:06.688742 7f5c973a57a0 20 read_log 192834'12429 (192834'12428) modify bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by client.933448.0:9935 2014-08-11 15:24:20.919848 2014-08-13 18:06:06.688755 7f5c973a57a0 20 read_log 192834'12430 (0'0) promote 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:13910 2014-08-11 15:24:22.347154 2014-08-13 18:06:06.688769 7f5c973a57a0 20 read_log 192834'12431 (192834'12430) modify 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by client.933448.0:10270 2014-08-11 15:24:22.338905 2014-08-13 18:06:06.688782 7f5c973a57a0 20 read_log 192834'12452 (0'0) promote aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:13932 2014-08-11 15:24:29.226630 2014-08-13 18:06:06.688795 7f5c973a57a0 20 read_log 192834'12453 (192834'12452) modify aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by client.933448.0:11750 2014-08-11 15:24:29.217974 2014-08-13 18:06:06.688807 7f5c973a57a0 20 read_log 192834'12474 (192834'12263) clean fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:13954 2014-08-11 15:24:36.059608 2014-08-13 18:06:06.688820 7f5c973a57a0 20 read_log 192834'12481 (0'0) promote dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:13961 2014-08-11 15:24:40.092709 2014-08-13 18:06:06.688832 7f5c973a57a0 20 read_log 192834'12482 (192834'12481) modify dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by client.933448.0:13824 2014-08-11 15:24:40.084916 2014-08-13 18:06:06.688845 7f5c973a57a0 20 read_log 192834'12489 (192834'12474) delete fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:13969 2014-08-11 15:24:40.998768 2014-08-13 18:06:06.688857 7f5c973a57a0 20 read_log 192834'12491 (192834'12283) clean 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:13971 2014-08-11 15:24:41.156576 2014-08-13 18:06:06.688870 7f5c973a57a0 20 read_log 192834'12495 (192834'12281) clean fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:13975 2014-08-11 15:24:41.254199 2014-08-13 18:06:06.688885 7f5c973a57a0 20 read_log 192834'12503 (192834'12491) delete 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:13983 2014-08-11 15:24:41.550408 2014-08-13 18:06:06.688898 7f5c973a57a0 20 read_log 192834'12507 (192834'12495) delete fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:13987 2014-08-11 15:24:41.572638 2014-08-13 18:06:06.688910 7f5c973a57a0 20 read_log 192834'12523 (0'0) promote 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14003 2014-08-11 15:24:44.720195 2014-08-13 18:06:06.688922 7f5c973a57a0 20 read_log 192834'12524 (192834'12523) modify 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by client.933448.0:14321 2014-08-11 15:24:44.710863 2014-08-13 18:06:06.688935 7f5c973a57a0 20 read_log 192834'12537 (192834'12306) clean 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14017 2014-08-11 15:24:46.951872 2014-08-13 18:06:06.688959 7f5c973a57a0 20 read_log 192834'12544 (192834'12537) delete 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14024 2014-08-11 15:24:47.000408 2014-08-13 18:06:06.688975 7f5c973a57a0 20 read_log 192834'12582 (192834'12325) clean 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14062 2014-08-11 15:24:52.500534 2014-08-13 18:06:06.688988 7f5c973a57a0 20 read_log 192834'12591 (192834'12582) delete 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14071 2014-08-11 15:24:57.438039 2014-08-13 18:06:06.689000 7f5c973a57a0 20 read_log 192834'12593 (192834'12337) clean 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14073 2014-08-11 15:24:57.590221 2014-08-13 18:06:06.689012 7f5c973a57a0 20 read_log 192834'12600 (192834'12593) delete 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14080 2014-08-11 15:24:57.623851 2014-08-13 18:06:06.689025 7f5c973a57a0 20 read_log 192834'12617 (0'0) promote 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14097 2014-08-11 15:25:01.307384 2014-08-13 18:06:06.689037 7f5c973a57a0 20 read_log 192834'12618 (192834'12617) modify 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by client.933448.0:17741 2014-08-11 15:25:01.299749 2014-08-13 18:06:06.689049 7f5c973a57a0 20 read_log 192834'12651 (0'0) promote b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:14131 2014-08-11 15:25:04.680329 2014-08-13 18:06:06.689062 7f5c973a57a0 20 read_log 192834'12652 (192834'12651) modify b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by client.933448.0:18303 2014-08-11 15:25:04.672494 2014-08-13 18:06:06.689074 7f5c973a57a0 20 read_log 192834'12675 (192834'12379) clean 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14155 2014-08-11 15:25:08.396549 2014-08-13 18:06:06.689086 7f5c973a57a0 20 read_log 192834'12689 (192834'12675) delete 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14169 2014-08-11 15:25:08.589376 2014-08-13 18:06:06.689099 7f5c973a57a0 20 read_log 192834'12705 (0'0) promote 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:14185 2014-08-11 15:25:11.750881 2014-08-13 18:06:06.689113 7f5c973a57a0 20 read_log 192834'12706 (192834'12705) modify 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by client.933448.0:19387 2014-08-11 15:25:11.742661 2014-08-13 18:06:06.689126 7f5c973a57a0 20 read_log 192834'12755 (0'0) promote 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:14235 2014-08-11 15:25:23.435859 2014-08-13 18:06:06.689140 7f5c973a57a0 20 read_log 192834'12756 (192834'12755) modify 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by client.933448.0:20813 2014-08-11 15:25:23.428206 2014-08-13 18:06:06.689165 7f5c973a57a0 20 read_log 192834'12757 (192834'12419) clean 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14237 2014-08-11 15:25:24.363178 2014-08-13 18:06:06.689178 7f5c973a57a0 20 read_log 192834'12761 (192834'12429) clean bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14241 2014-08-11 15:25:24.424223 2014-08-13 18:06:06.689190 7f5c973a57a0 20 read_log 192834'12762 (192834'12431) clean 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14242 2014-08-11 15:25:24.473416 2014-08-13 18:06:06.689203 7f5c973a57a0 20 read_log 192834'12767 (192834'12757) delete 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14247 2014-08-11 15:25:24.542586 2014-08-13 18:06:06.689215 7f5c973a57a0 20 read_log 192834'12771 (192834'12761) delete bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14251 2014-08-11 15:25:24.555241 2014-08-13 18:06:06.689254 7f5c973a57a0 20 read_log 192834'12772 (192834'12762) delete 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14252 2014-08-11 15:25:24.559863 2014-08-13 18:06:06.689270 7f5c973a57a0 20 read_log 192834'12785 (192834'12453) clean aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14265 2014-08-11 15:25:29.755103 2014-08-13 18:06:06.689282 7f5c973a57a0 20 read_log 192834'12792 (192834'12785) delete aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14272 2014-08-11 15:25:29.874932 2014-08-13 18:06:06.689299 7f5c973a57a0 20 read_log 192834'12812 (0'0) promote ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:14292 2014-08-11 15:25:35.049658 2014-08-13 18:06:06.689311 7f5c973a57a0 20 read_log 192834'12813 (192834'12812) modify ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by client.933448.0:22199 2014-08-11 15:25:34.961761 2014-08-13 18:06:06.689327 7f5c973a57a0 20 read_log 192834'12829 (0'0) promote fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14309 2014-08-11 15:25:37.691044 2014-08-13 18:06:06.689339 7f5c973a57a0 20 read_log 192834'12830 (192834'12829) modify fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by client.933448.0:22341 2014-08-11 15:25:37.662942 2014-08-13 18:06:06.689352 7f5c973a57a0 20 read_log 192834'12847 (0'0) promote fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14327 2014-08-11 15:25:39.264994 2014-08-13 18:06:06.689364 7f5c973a57a0 20 read_log 192834'12848 (192834'12847) modify fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by client.933448.0:23337 2014-08-11 15:25:39.234974 2014-08-13 18:06:06.689383 7f5c973a57a0 20 read_log 192834'12849 (0'0) promote 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14329 2014-08-11 15:25:39.397138 2014-08-13 18:06:06.689396 7f5c973a57a0 20 read_log 192834'12850 (192834'12849) modify 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by client.933448.0:23453 2014-08-11 15:25:39.367061 2014-08-13 18:06:06.689408 7f5c973a57a0 20 read_log 192834'12868 (192834'12482) clean dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14348 2014-08-11 15:25:40.254649 2014-08-13 18:06:06.689420 7f5c973a57a0 20 read_log 192834'12872 (192834'12868) delete dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14352 2014-08-11 15:25:40.280984 2014-08-13 18:06:06.689446 7f5c973a57a0 20 read_log 192834'12884 (0'0) promote 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14364 2014-08-11 15:25:40.551240 2014-08-13 18:06:06.689459 7f5c973a57a0 20 read_log 192834'12885 (192834'12884) modify 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by client.933448.0:24283 2014-08-11 15:25:40.522655 2014-08-13 18:06:06.689471 7f5c973a57a0 20 read_log 192834'12902 (0'0) promote 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14382 2014-08-11 15:25:42.633086 2014-08-13 18:06:06.689492 7f5c973a57a0 20 read_log 192834'12903 (192834'12902) modify 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by client.933448.0:25838 2014-08-11 15:25:42.604701 2014-08-13 18:06:06.689507 7f5c973a57a0 20 read_log 192834'12914 (0'0) promote 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14394 2014-08-11 15:25:43.796880 2014-08-13 18:06:06.689520 7f5c973a57a0 20 read_log 192834'12915 (192834'12914) modify 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by client.933448.0:26799 2014-08-11 15:25:43.770287 2014-08-13 18:06:06.689547 7f5c973a57a0 20 read_log 192834'12946 (192834'12524) clean 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14426 2014-08-11 15:25:45.467919 2014-08-13 18:06:06.689574 7f5c973a57a0 20 read_log 192834'12954 (192834'12946) delete 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14434 2014-08-11 15:25:45.609143 2014-08-13 18:06:06.689589 7f5c973a57a0 20 read_log 192834'12979 (0'0) promote 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14459 2014-08-11 15:25:47.139797 2014-08-13 18:06:06.689601 7f5c973a57a0 20 read_log 192834'12980 (192834'12979) modify 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by client.933448.0:29207 2014-08-11 15:25:47.114359 2014-08-13 18:06:06.689626 7f5c973a57a0 20 read_log 192834'13019 (0'0) promote 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14499 2014-08-11 15:25:50.708999 2014-08-13 18:06:06.689639 7f5c973a57a0 20 read_log 192834'13020 (192834'13019) modify 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by client.933448.0:31840 2014-08-11 15:25:50.682700 2014-08-13 18:06:06.689651 7f5c973a57a0 20 read_log 192834'13041 (0'0) promote bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14521 2014-08-11 15:25:51.311295 2014-08-13 18:06:06.689664 7f5c973a57a0 20 read_log 192834'13042 (192834'13041) modify bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by client.933448.0:32226 2014-08-11 15:25:51.279533 2014-08-13 18:06:06.689676 7f5c973a57a0 20 read_log 192834'13043 (0'0) promote 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14523 2014-08-11 15:25:52.047611 2014-08-13 18:06:06.689691 7f5c973a57a0 20 read_log 192834'13044 (192834'13043) modify 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by client.933448.0:32561 2014-08-11 15:25:52.017984 2014-08-13 18:06:06.689711 7f5c973a57a0 20 read_log 192834'13065 (0'0) promote aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:14545 2014-08-11 15:25:54.548809 2014-08-13 18:06:06.689734 7f5c973a57a0 20 read_log 192834'13066 (192834'13065) modify aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by client.933448.0:34041 2014-08-11 15:25:54.510434 2014-08-13 18:06:06.689747 7f5c973a57a0 20 read_log 192834'13101 (0'0) promote dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:14581 2014-08-11 15:25:57.281638 2014-08-13 18:06:06.689759 7f5c973a57a0 20 read_log 192834'13102 (192834'13101) modify dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by client.933448.0:36115 2014-08-11 15:25:57.256477 2014-08-13 18:06:06.689772 7f5c973a57a0 20 read_log 192834'13113 (0'0) promote 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:14593 2014-08-11 15:25:58.091419 2014-08-13 18:06:06.689784 7f5c973a57a0 20 read_log 192834'13114 (192834'13113) modify 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by client.933448.0:36612 2014-08-11 15:25:58.067626 2014-08-13 18:06:06.689796 7f5c973a57a0 20 read_log 192834'13145 (192834'12618) clean 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14625 2014-08-11 15:26:01.532619 2014-08-13 18:06:06.689821 7f5c973a57a0 20 read_log 192834'13149 (192834'13145) delete 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14629 2014-08-11 15:26:01.652295 2014-08-13 18:06:06.689837 7f5c973a57a0 20 read_log 192834'13167 (0'0) promote 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:14647 2014-08-11 15:26:02.257841 2014-08-13 18:06:06.689850 7f5c973a57a0 20 read_log 192834'13168 (192834'13167) modify 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by client.933448.0:40032 2014-08-11 15:26:02.231771 2014-08-13 18:06:06.689863 7f5c973a57a0 20 read_log 192834'13171 (192834'12652) modify b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by client.933448.0:40594 2014-08-11 15:26:02.662421 2014-08-13 18:06:06.689891 7f5c973a57a0 20 read_log 192834'13188 (192834'12706) modify 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by client.933448.0:41678 2014-08-11 15:26:03.018534 2014-08-13 18:06:06.689912 7f5c973a57a0 20 read_log 192834'13197 (192834'12756) modify 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by client.933448.0:43104 2014-08-11 15:26:03.503064 2014-08-13 18:06:06.689925 7f5c973a57a0 20 read_log 192834'13204 (192834'12813) modify ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by client.933448.0:44490 2014-08-11 15:26:03.960541 2014-08-13 18:06:06.689937 7f5c973a57a0 20 read_log 192834'13216 (0'0) promote 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:14696 2014-08-11 15:26:16.863024 2014-08-13 18:06:06.689950 7f5c973a57a0 20 read_log 192834'13217 (192834'13216) modify 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by client.1883634.0:651 2014-08-11 15:26:16.855277 2014-08-13 18:06:06.689962 7f5c973a57a0 20 read_log 192834'13228 (0'0) promote 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:14708 2014-08-11 15:26:20.208238 2014-08-13 18:06:06.689974 7f5c973a57a0 20 read_log 192834'13229 (192834'13228) modify 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by client.1883634.0:1577 2014-08-11 15:26:20.200892 2014-08-13 18:06:06.689987 7f5c973a57a0 20 read_log 192834'13230 (0'0) promote bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:14710 2014-08-11 15:26:20.527134 2014-08-13 18:06:06.689999 7f5c973a57a0 20 read_log 192834'13231 (192834'13230) modify bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by client.1883634.0:1663 2014-08-11 15:26:20.518306 2014-08-13 18:06:06.690011 7f5c973a57a0 20 read_log 192834'13272 (0'0) promote c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:14752 2014-08-11 15:26:29.154756 2014-08-13 18:06:06.690023 7f5c973a57a0 20 read_log 192834'13273 (192834'13272) modify c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by client.1883634.0:4045 2014-08-11 15:26:29.147716 2014-08-13 18:06:06.690035 7f5c973a57a0 20 read_log 192834'13276 (0'0) promote 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:14756 2014-08-11 15:26:29.970140 2014-08-13 18:06:06.690047 7f5c973a57a0 20 read_log 192834'13277 (192834'13276) modify 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by client.1883634.0:4275 2014-08-11 15:26:29.962730 2014-08-13 18:06:06.690059 7f5c973a57a0 20 read_log 192834'13278 (0'0) promote 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:14758 2014-08-11 15:26:30.194159 2014-08-13 18:06:06.690071 7f5c973a57a0 20 read_log 192834'13279 (192834'13278) modify 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by client.1883634.0:4339 2014-08-11 15:26:30.186725 2014-08-13 18:06:06.690084 7f5c973a57a0 20 read_log 192834'13306 (0'0) promote 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:14786 2014-08-11 15:26:38.540099 2014-08-13 18:06:06.690096 7f5c973a57a0 20 read_log 192834'13307 (192834'13306) modify 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by client.1883634.0:6508 2014-08-11 15:26:38.531222 2014-08-13 18:06:06.690108 7f5c973a57a0 20 read_log 192834'13310 (0'0) promote 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:14790 2014-08-11 15:26:40.837927 2014-08-13 18:06:06.690120 7f5c973a57a0 20 read_log 192834'13311 (192834'13310) modify 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by client.1883634.0:7106 2014-08-11 15:26:40.830906 2014-08-13 18:06:06.690133 7f5c973a57a0 20 read_log 192834'13318 (192834'12885) clean 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14798 2014-08-11 15:26:42.486454 2014-08-13 18:06:06.690174 7f5c973a57a0 20 read_log 192834'13322 (192834'12830) clean fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14802 2014-08-11 15:26:42.520966 2014-08-13 18:06:06.690199 7f5c973a57a0 20 read_log 192834'13323 (192834'12850) clean 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14803 2014-08-11 15:26:42.521248 2014-08-13 18:06:06.690215 7f5c973a57a0 20 read_log 192834'13329 (192834'12848) clean fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14809 2014-08-11 15:26:42.556159 2014-08-13 18:06:06.690228 7f5c973a57a0 20 read_log 192834'13349 (192834'13318) delete 99eec136/benchmark_data_storage0111-ib.pocstor_26659_object1991/head//24 by osd.21.0:14829 2014-08-11 15:26:42.693153 2014-08-13 18:06:06.690240 7f5c973a57a0 20 read_log 192834'13355 (192834'13322) delete fcc81536/benchmark_data_storage0111-ib.pocstor_26659_object49/head//24 by osd.21.0:14835 2014-08-11 15:26:42.728305 2014-08-13 18:06:06.690253 7f5c973a57a0 20 read_log 192834'13356 (192834'13323) delete 80767536/benchmark_data_storage0111-ib.pocstor_26659_object1161/head//24 by osd.21.0:14836 2014-08-11 15:26:42.734380 2014-08-13 18:06:06.690267 7f5c973a57a0 20 read_log 192834'13361 (192834'12903) clean 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14841 2014-08-11 15:26:42.764257 2014-08-13 18:06:06.690282 7f5c973a57a0 20 read_log 192834'13363 (192834'13329) delete fc1cdd36/benchmark_data_storage0111-ib.pocstor_26659_object1045/head//24 by osd.21.0:14843 2014-08-11 15:26:42.775718 2014-08-13 18:06:06.690297 7f5c973a57a0 20 read_log 192834'13384 (192834'13361) delete 49312536/benchmark_data_storage0111-ib.pocstor_26659_object3546/head//24 by osd.21.0:14864 2014-08-11 15:26:42.884276 2014-08-13 18:06:06.690309 7f5c973a57a0 20 read_log 192834'13387 (0'0) promote 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:14867 2014-08-11 15:26:43.277616 2014-08-13 18:06:06.690322 7f5c973a57a0 20 read_log 192834'13388 (192834'13387) modify 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by client.1883634.0:7684 2014-08-11 15:26:43.269813 2014-08-13 18:06:06.690338 7f5c973a57a0 20 read_log 192834'13398 (192834'12915) clean 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14878 2014-08-11 15:26:48.169887 2014-08-13 18:06:06.690351 7f5c973a57a0 20 read_log 192834'13399 (192834'12980) clean 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14879 2014-08-11 15:26:48.170231 2014-08-13 18:06:06.690366 7f5c973a57a0 20 read_log 192834'13427 (192834'13398) delete 6cd77936/benchmark_data_storage0111-ib.pocstor_26659_object4507/head//24 by osd.21.0:14907 2014-08-11 15:26:48.363275 2014-08-13 18:06:06.690382 7f5c973a57a0 20 read_log 192834'13430 (192834'13399) delete 73a936/benchmark_data_storage0111-ib.pocstor_26659_object6915/head//24 by osd.21.0:14910 2014-08-11 15:26:48.369185 2014-08-13 18:06:06.690395 7f5c973a57a0 20 read_log 192834'13474 (192834'13020) clean 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14954 2014-08-11 15:26:53.799572 2014-08-13 18:06:06.690407 7f5c973a57a0 20 read_log 192834'13479 (192834'13042) clean bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14959 2014-08-11 15:26:53.820989 2014-08-13 18:06:06.690420 7f5c973a57a0 20 read_log 192834'13481 (192834'13044) clean 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14961 2014-08-11 15:26:53.834148 2014-08-13 18:06:06.690446 7f5c973a57a0 20 read_log 192834'13500 (192834'13474) delete 1fada136/benchmark_data_storage0111-ib.pocstor_26659_object9548/head//24 by osd.21.0:14980 2014-08-11 15:26:53.944512 2014-08-13 18:06:06.690462 7f5c973a57a0 20 read_log 192834'13505 (192834'13479) delete bc63b536/benchmark_data_storage0111-ib.pocstor_26659_object9934/head//24 by osd.21.0:14985 2014-08-11 15:26:53.970510 2014-08-13 18:06:06.690490 7f5c973a57a0 20 read_log 192834'13507 (192834'13481) delete 5e397936/benchmark_data_storage0111-ib.pocstor_26659_object10269/head//24 by osd.21.0:14987 2014-08-11 15:26:53.983309 2014-08-13 18:06:06.690508 7f5c973a57a0 20 read_log 192834'13540 (0'0) promote b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15020 2014-08-11 15:26:58.233964 2014-08-13 18:06:06.690521 7f5c973a57a0 20 read_log 192834'13541 (192834'13540) modify b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by client.1883634.0:11085 2014-08-11 15:26:58.226196 2014-08-13 18:06:06.690545 7f5c973a57a0 20 read_log 192834'13551 (192834'13102) clean dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:15031 2014-08-11 15:26:59.158127 2014-08-13 18:06:06.690559 7f5c973a57a0 20 read_log 192834'13555 (192834'13066) clean aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:15035 2014-08-11 15:26:59.179556 2014-08-13 18:06:06.690574 7f5c973a57a0 20 read_log 192834'13556 (192834'13114) clean 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:15036 2014-08-11 15:26:59.183337 2014-08-13 18:06:06.690602 7f5c973a57a0 20 read_log 192834'13578 (192834'13551) delete dd2a4936/benchmark_data_storage0111-ib.pocstor_26659_object13823/head//24 by osd.21.0:15058 2014-08-11 15:26:59.285659 2014-08-13 18:06:06.690615 7f5c973a57a0 20 read_log 192834'13582 (192834'13555) delete aa306d36/benchmark_data_storage0111-ib.pocstor_26659_object11749/head//24 by osd.21.0:15062 2014-08-11 15:26:59.302575 2014-08-13 18:06:06.690628 7f5c973a57a0 20 read_log 192834'13583 (192834'13556) delete 753bbd36/benchmark_data_storage0111-ib.pocstor_26659_object14320/head//24 by osd.21.0:15063 2014-08-11 15:26:59.306342 2014-08-13 18:06:06.690643 7f5c973a57a0 20 read_log 192834'13624 (192834'13188) clean 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:15104 2014-08-11 15:27:04.592638 2014-08-13 18:06:06.690658 7f5c973a57a0 20 read_log 192834'13632 (192834'13168) clean 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:15112 2014-08-11 15:27:04.616969 2014-08-13 18:06:06.690678 7f5c973a57a0 20 read_log 192834'13636 (192834'13204) clean ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:15116 2014-08-11 15:27:04.632024 2014-08-13 18:06:06.690698 7f5c973a57a0 20 read_log 192834'13637 (192834'13197) clean 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:15117 2014-08-11 15:27:04.635321 2014-08-13 18:06:06.690711 7f5c973a57a0 20 read_log 192834'13638 (192834'13171) clean b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:15118 2014-08-11 15:27:04.639248 2014-08-13 18:06:06.690723 7f5c973a57a0 20 read_log 192834'13680 (192834'13624) delete 66c89536/benchmark_data_storage0111-ib.pocstor_26659_object19386/head//24 by osd.21.0:15160 2014-08-11 15:27:04.763924 2014-08-13 18:06:06.690739 7f5c973a57a0 20 read_log 192834'13687 (192834'13632) delete 1d51e936/benchmark_data_storage0111-ib.pocstor_26659_object17740/head//24 by osd.21.0:15167 2014-08-11 15:27:04.777246 2014-08-13 18:06:06.690751 7f5c973a57a0 20 read_log 192834'13692 (192834'13636) delete ff623d36/benchmark_data_storage0111-ib.pocstor_26659_object22198/head//24 by osd.21.0:15172 2014-08-11 15:27:04.786504 2014-08-13 18:06:06.690764 7f5c973a57a0 20 read_log 192834'13693 (192834'13637) delete 6f0a4d36/benchmark_data_storage0111-ib.pocstor_26659_object20812/head//24 by osd.21.0:15173 2014-08-11 15:27:04.786878 2014-08-13 18:06:06.690776 7f5c973a57a0 20 read_log 192834'13694 (192834'13638) delete b9a2ad36/benchmark_data_storage0111-ib.pocstor_26659_object18302/head//24 by osd.21.0:15174 2014-08-11 15:27:04.787217 2014-08-13 18:06:06.690799 7f5c973a57a0 20 read_log 192834'13732 (0'0) promote 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15212 2014-08-11 15:27:07.014357 2014-08-13 18:06:06.690823 7f5c973a57a0 20 read_log 192834'13733 (192834'13732) modify 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by client.1883634.0:12962 2014-08-11 15:27:07.005624 2014-08-13 18:06:06.690844 7f5c973a57a0 20 read_log 192834'13748 (0'0) promote fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15228 2014-08-11 15:27:12.205432 2014-08-13 18:06:06.690857 7f5c973a57a0 20 read_log 192834'13749 (192834'13748) modify fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by client.1883634.0:14181 2014-08-11 15:27:12.198117 2014-08-13 18:06:06.690869 7f5c973a57a0 20 read_log 192834'13769 (192834'13217) clean 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15248 2014-08-11 15:27:20.009986 2014-08-13 18:06:06.690884 7f5c973a57a0 20 read_log 192834'13775 (192834'13769) delete 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15254 2014-08-11 15:27:20.094644 2014-08-13 18:06:06.690897 7f5c973a57a0 20 read_log 192834'13787 (192834'13229) clean 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15266 2014-08-11 15:27:25.270105 2014-08-13 18:06:06.690921 7f5c973a57a0 20 read_log 192834'13788 (192834'13231) clean bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15267 2014-08-11 15:27:25.270796 2014-08-13 18:06:06.690934 7f5c973a57a0 20 read_log 192834'13802 (192834'13787) delete 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15281 2014-08-11 15:27:25.535635 2014-08-13 18:06:06.690946 7f5c973a57a0 20 read_log 192834'13803 (192834'13788) delete bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15282 2014-08-11 15:27:25.541276 2014-08-13 18:06:06.690959 7f5c973a57a0 20 read_log 192834'13822 (0'0) promote 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15301 2014-08-11 15:27:26.657363 2014-08-13 18:06:06.690977 7f5c973a57a0 20 read_log 192834'13823 (192834'13822) modify 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by client.1883634.0:16256 2014-08-11 15:27:26.650085 2014-08-13 18:06:06.690993 7f5c973a57a0 20 read_log 192834'13826 (0'0) promote 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15305 2014-08-11 15:27:27.130044 2014-08-13 18:06:06.691006 7f5c973a57a0 20 read_log 192834'13827 (192834'13826) modify 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by client.1883634.0:16341 2014-08-11 15:27:27.121203 2014-08-13 18:06:06.691019 7f5c973a57a0 20 read_log 192834'13840 (0'0) promote cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15319 2014-08-11 15:27:29.220152 2014-08-13 18:06:06.691031 7f5c973a57a0 20 read_log 192834'13841 (192834'13840) modify cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by client.1883634.0:16764 2014-08-11 15:27:29.212198 2014-08-13 18:06:06.691043 7f5c973a57a0 20 read_log 192834'13842 (0'0) promote f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15321 2014-08-11 15:27:29.460252 2014-08-13 18:06:06.691055 7f5c973a57a0 20 read_log 192834'13843 (192834'13842) modify f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by client.1883634.0:16816 2014-08-11 15:27:29.451646 2014-08-13 18:06:06.691068 7f5c973a57a0 20 read_log 192834'13844 (0'0) promote f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15323 2014-08-11 15:27:29.599433 2014-08-13 18:06:06.691080 7f5c973a57a0 20 read_log 192834'13845 (192834'13844) modify f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by client.1883634.0:16842 2014-08-11 15:27:29.589944 2014-08-13 18:06:06.691092 7f5c973a57a0 20 read_log 192834'13853 (192834'13273) clean c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15332 2014-08-11 15:27:30.724336 2014-08-13 18:06:06.691112 7f5c973a57a0 20 read_log 192834'13854 (192834'13279) clean 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15333 2014-08-11 15:27:30.725216 2014-08-13 18:06:06.691127 7f5c973a57a0 20 read_log 192834'13857 (192834'13277) clean 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15336 2014-08-11 15:27:30.816144 2014-08-13 18:06:06.691140 7f5c973a57a0 20 read_log 192834'13861 (192834'13853) delete c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15340 2014-08-11 15:27:30.908256 2014-08-13 18:06:06.691158 7f5c973a57a0 20 read_log 192834'13862 (192834'13854) delete 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15341 2014-08-11 15:27:30.915000 2014-08-13 18:06:06.691173 7f5c973a57a0 20 read_log 192834'13865 (192834'13857) delete 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15344 2014-08-11 15:27:30.929356 2014-08-13 18:06:06.691185 7f5c973a57a0 20 read_log 192834'13905 (192834'13307) clean 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15384 2014-08-11 15:27:41.416738 2014-08-13 18:06:06.691198 7f5c973a57a0 20 read_log 192834'13910 (192834'13905) delete 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15389 2014-08-11 15:27:41.454668 2014-08-13 18:06:06.691210 7f5c973a57a0 20 read_log 192834'13913 (192834'13311) clean 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15392 2014-08-11 15:27:41.525475 2014-08-13 18:06:06.691222 7f5c973a57a0 20 read_log 192834'13919 (0'0) promote 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:15398 2014-08-11 15:27:46.410427 2014-08-13 18:06:06.691234 7f5c973a57a0 20 read_log 192834'13920 (192834'13919) modify 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by client.1883634.0:18550 2014-08-11 15:27:46.400894 2014-08-13 18:06:06.691247 7f5c973a57a0 20 read_log 192834'13925 (192834'13913) delete 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15404 2014-08-11 15:27:46.616644 2014-08-13 18:06:06.691263 7f5c973a57a0 20 read_log 192834'13927 (192834'13388) clean 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15406 2014-08-11 15:27:46.630123 2014-08-13 18:06:06.691278 7f5c973a57a0 20 read_log 192834'13931 (192834'13927) delete 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15410 2014-08-11 15:27:46.643973 2014-08-13 18:06:06.691290 7f5c973a57a0 20 read_log 192834'13933 (0'0) promote e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:15412 2014-08-11 15:27:47.887198 2014-08-13 18:06:06.691303 7f5c973a57a0 20 read_log 192834'13934 (192834'13933) modify e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by client.1883634.0:18856 2014-08-11 15:27:47.878517 2014-08-13 18:06:06.691315 7f5c973a57a0 20 read_log 192834'13952 (0'0) promote 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:15431 2014-08-11 15:27:52.963088 2014-08-13 18:06:06.691331 7f5c973a57a0 20 read_log 192834'13953 (192834'13952) modify 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by client.1883634.0:19497 2014-08-11 15:27:52.954620 2014-08-13 18:06:06.691344 7f5c973a57a0 20 read_log 192834'13993 (192834'13541) clean b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15472 2014-08-11 15:28:02.230971 2014-08-13 18:06:06.691356 7f5c973a57a0 20 read_log 192834'13999 (192834'13993) delete b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15478 2014-08-11 15:28:02.257656 2014-08-13 18:06:06.691369 7f5c973a57a0 20 read_log 192834'14004 (0'0) promote dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:15483 2014-08-11 15:28:03.803402 2014-08-13 18:06:06.691425 7f5c973a57a0 20 read_log 192834'14005 (192834'14004) modify dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by client.1883634.0:21326 2014-08-11 15:28:03.794049 2014-08-13 18:06:06.691443 7f5c973a57a0 20 read_log 192834'14016 (192834'13733) clean 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15495 2014-08-11 15:28:07.412325 2014-08-13 18:06:06.691462 7f5c973a57a0 20 read_log 192834'14019 (192834'14016) delete 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15498 2014-08-11 15:28:07.422644 2014-08-13 18:06:06.691479 7f5c973a57a0 20 read_log 192834'14037 (192834'13749) clean fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15516 2014-08-11 15:28:12.550306 2014-08-13 18:06:06.691492 7f5c973a57a0 20 read_log 192834'14047 (192834'14037) delete fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15526 2014-08-11 15:28:12.665014 2014-08-13 18:06:06.691505 7f5c973a57a0 20 read_log 192834'14075 (0'0) promote 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15554 2014-08-11 15:28:18.188358 2014-08-13 18:06:06.691527 7f5c973a57a0 20 read_log 192834'14076 (192834'14075) modify 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by client.1883634.0:23975 2014-08-11 15:28:18.160840 2014-08-13 18:06:06.691540 7f5c973a57a0 20 read_log 192834'14087 (0'0) promote 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15566 2014-08-11 15:28:19.759050 2014-08-13 18:06:06.691553 7f5c973a57a0 20 read_log 192834'14088 (192834'14087) modify 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by client.1883634.0:24901 2014-08-11 15:28:19.732481 2014-08-13 18:06:06.691565 7f5c973a57a0 20 read_log 192834'14089 (0'0) promote bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15568 2014-08-11 15:28:19.890344 2014-08-13 18:06:06.691578 7f5c973a57a0 20 read_log 192834'14090 (192834'14089) modify bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by client.1883634.0:24987 2014-08-11 15:28:19.865440 2014-08-13 18:06:06.691595 7f5c973a57a0 20 read_log 192834'14135 (0'0) promote c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15614 2014-08-11 15:28:24.446003 2014-08-13 18:06:06.691608 7f5c973a57a0 20 read_log 192834'14136 (192834'14135) modify c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by client.1883634.0:27369 2014-08-11 15:28:24.419778 2014-08-13 18:06:06.691623 7f5c973a57a0 20 read_log 192834'14139 (0'0) promote 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15618 2014-08-11 15:28:24.702334 2014-08-13 18:06:06.691636 7f5c973a57a0 20 read_log 192834'14140 (192834'14139) modify 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by client.1883634.0:27599 2014-08-11 15:28:24.673644 2014-08-13 18:06:06.691648 7f5c973a57a0 20 read_log 192834'14141 (0'0) promote 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15620 2014-08-11 15:28:24.799560 2014-08-13 18:06:06.691660 7f5c973a57a0 20 read_log 192834'14142 (192834'14141) modify 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by client.1883634.0:27663 2014-08-11 15:28:24.768689 2014-08-13 18:06:06.691673 7f5c973a57a0 20 read_log 192834'14169 (0'0) promote 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15648 2014-08-11 15:28:27.760475 2014-08-13 18:06:06.691688 7f5c973a57a0 20 read_log 192834'14170 (192834'14169) modify 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by client.1883634.0:29832 2014-08-11 15:28:27.731672 2014-08-13 18:06:06.691701 7f5c973a57a0 20 read_log 192834'14175 (192834'13823) clean 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15654 2014-08-11 15:28:28.057740 2014-08-13 18:06:06.691721 7f5c973a57a0 20 read_log 192834'14176 (192834'13827) clean 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15655 2014-08-11 15:28:28.085509 2014-08-13 18:06:06.691746 7f5c973a57a0 20 read_log 192834'14183 (192834'14175) delete 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15662 2014-08-11 15:28:28.216863 2014-08-13 18:06:06.691759 7f5c973a57a0 20 read_log 192834'14184 (192834'14176) delete 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15663 2014-08-11 15:28:28.221008 2014-08-13 18:06:06.691772 7f5c973a57a0 20 read_log 192834'14189 (0'0) promote 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15668 2014-08-11 15:28:28.655567 2014-08-13 18:06:06.691784 7f5c973a57a0 20 read_log 192834'14190 (192834'14189) modify 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by client.1883634.0:30430 2014-08-11 15:28:28.627326 2014-08-13 18:06:06.691796 7f5c973a57a0 20 read_log 192834'14199 (0'0) promote 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15678 2014-08-11 15:28:29.742267 2014-08-13 18:06:06.691808 7f5c973a57a0 20 read_log 192834'14200 (192834'14199) modify 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by client.1883634.0:31008 2014-08-11 15:28:29.715699 2014-08-13 18:06:06.691821 7f5c973a57a0 20 read_log 192834'14220 (192834'13843) clean f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15699 2014-08-11 15:28:33.450009 2014-08-13 18:06:06.691833 7f5c973a57a0 20 read_log 192834'14222 (192834'13841) clean cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15701 2014-08-11 15:28:33.451178 2014-08-13 18:06:06.691845 7f5c973a57a0 20 read_log 192834'14225 (192834'13845) clean f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15704 2014-08-11 15:28:33.546151 2014-08-13 18:06:06.691857 7f5c973a57a0 20 read_log 192834'14230 (192834'14220) delete f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15709 2014-08-11 15:28:33.571736 2014-08-13 18:06:06.691870 7f5c973a57a0 20 read_log 192834'14232 (192834'14222) delete cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15711 2014-08-11 15:28:33.583530 2014-08-13 18:06:06.691882 7f5c973a57a0 20 read_log 192834'14233 (192834'14225) delete f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15712 2014-08-11 15:28:33.593194 2014-08-13 18:06:06.691894 7f5c973a57a0 20 read_log 192834'14248 (0'0) promote b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15727 2014-08-11 15:28:34.679520 2014-08-13 18:06:06.691925 7f5c973a57a0 20 read_log 192834'14249 (192834'14248) modify b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by client.1883634.0:34409 2014-08-11 15:28:34.652350 2014-08-13 18:06:06.691958 7f5c973a57a0 20 read_log 192834'14268 (0'0) promote 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:15747 2014-08-11 15:28:37.862958 2014-08-13 18:06:06.691970 7f5c973a57a0 20 read_log 192834'14269 (192834'14268) modify 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by client.1883634.0:36286 2014-08-11 15:28:37.833319 2014-08-13 18:06:06.691985 7f5c973a57a0 20 read_log 192834'14289 (0'0) promote fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:15768 2014-08-11 15:28:39.483446 2014-08-13 18:06:06.691997 7f5c973a57a0 20 read_log 192834'14290 (192834'14289) modify fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by client.1883634.0:37505 2014-08-11 15:28:39.446720 2014-08-13 18:06:06.692011 7f5c973a57a0 20 read_log 192834'14309 (0'0) promote 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:15788 2014-08-11 15:28:42.356531 2014-08-13 18:06:06.692033 7f5c973a57a0 20 read_log 192834'14310 (192834'14309) modify 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by client.1883634.0:39580 2014-08-11 15:28:42.329208 2014-08-13 18:06:06.692048 7f5c973a57a0 20 read_log 192834'14313 (0'0) promote 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:15792 2014-08-11 15:28:42.452662 2014-08-13 18:06:06.692060 7f5c973a57a0 20 read_log 192834'14314 (192834'14313) modify 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by client.1883634.0:39665 2014-08-11 15:28:42.414195 2014-08-13 18:06:06.692074 7f5c973a57a0 20 read_log 192834'14327 (0'0) promote cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:15806 2014-08-11 15:28:43.176310 2014-08-13 18:06:06.692088 7f5c973a57a0 20 read_log 192834'14328 (192834'14327) modify cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by client.1883634.0:40088 2014-08-11 15:28:43.146358 2014-08-13 18:06:06.692100 7f5c973a57a0 20 read_log 192834'14329 (0'0) promote f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:15808 2014-08-11 15:28:43.262364 2014-08-13 18:06:06.692113 7f5c973a57a0 20 read_log 192834'14330 (192834'14329) modify f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by client.1883634.0:40140 2014-08-11 15:28:43.233934 2014-08-13 18:06:06.692127 7f5c973a57a0 20 read_log 192834'14331 (0'0) promote f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:15810 2014-08-11 15:28:43.307737 2014-08-13 18:06:06.692139 7f5c973a57a0 20 read_log 192834'14332 (192834'14331) modify f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by client.1883634.0:40166 2014-08-11 15:28:43.283009 2014-08-13 18:06:06.692159 7f5c973a57a0 20 read_log 192834'14357 (192834'13920) modify 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by client.1883634.0:41874 2014-08-11 15:28:45.558637 2014-08-13 18:06:06.692172 7f5c973a57a0 20 read_log 192834'14358 (192834'13934) modify e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by client.1883634.0:42180 2014-08-11 15:28:46.162057 2014-08-13 18:06:06.692185 7f5c973a57a0 20 read_log 192834'14362 (192834'13953) modify 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by client.1883634.0:42821 2014-08-11 15:28:46.368950 2014-08-13 18:06:06.692197 7f5c973a57a0 20 read_log 192834'14375 (192834'14005) modify dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by client.1883634.0:44650 2014-08-11 15:28:46.916100 2014-08-13 18:06:06.692211 7f5c973a57a0 20 read_log 192834'14388 (192834'14076) clean 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15867 2014-08-11 15:29:19.254297 2014-08-13 18:06:06.692224 7f5c973a57a0 20 read_log 192834'14396 (192834'14388) delete 38405536/benchmark_data_storage0111-ib.pocstor_41526_object650/head//24 by osd.21.0:15875 2014-08-11 15:29:19.320673 2014-08-13 18:06:06.692236 7f5c973a57a0 20 read_log 192834'14402 (192834'14136) clean c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15881 2014-08-11 15:29:24.435246 2014-08-13 18:06:06.692249 7f5c973a57a0 20 read_log 192834'14404 (192834'14088) clean 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15883 2014-08-11 15:29:24.442074 2014-08-13 18:06:06.692261 7f5c973a57a0 20 read_log 192834'14405 (192834'14090) clean bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15884 2014-08-11 15:29:24.442301 2014-08-13 18:06:06.692276 7f5c973a57a0 20 read_log 192834'14427 (192834'14402) delete c65b1136/benchmark_data_storage0111-ib.pocstor_41526_object4044/head//24 by osd.21.0:15906 2014-08-11 15:29:24.539314 2014-08-13 18:06:06.692292 7f5c973a57a0 20 read_log 192834'14429 (192834'14404) delete 32081536/benchmark_data_storage0111-ib.pocstor_41526_object1576/head//24 by osd.21.0:15908 2014-08-11 15:29:24.550825 2014-08-13 18:06:06.692315 7f5c973a57a0 20 read_log 192834'14430 (192834'14405) delete bd587536/benchmark_data_storage0111-ib.pocstor_41526_object1662/head//24 by osd.21.0:15909 2014-08-11 15:29:24.554786 2014-08-13 18:06:06.692334 7f5c973a57a0 20 read_log 192834'14460 (192834'14170) clean 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15939 2014-08-11 15:29:29.719598 2014-08-13 18:06:06.692347 7f5c973a57a0 20 read_log 192834'14461 (192834'14142) clean 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15940 2014-08-11 15:29:29.725283 2014-08-13 18:06:06.692360 7f5c973a57a0 20 read_log 192834'14468 (192834'14200) clean 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15947 2014-08-11 15:29:29.748375 2014-08-13 18:06:06.692373 7f5c973a57a0 20 read_log 192834'14469 (192834'14140) clean 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15948 2014-08-11 15:29:29.754633 2014-08-13 18:06:06.692389 7f5c973a57a0 20 read_log 192834'14470 (192834'14190) clean 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15949 2014-08-11 15:29:29.755020 2014-08-13 18:06:06.692402 7f5c973a57a0 20 read_log 192834'14483 (192834'14460) delete 7954536/benchmark_data_storage0111-ib.pocstor_41526_object6507/head//24 by osd.21.0:15962 2014-08-11 15:29:29.798417 2014-08-13 18:06:06.692431 7f5c973a57a0 20 read_log 192834'14484 (192834'14461) delete 1d1c9536/benchmark_data_storage0111-ib.pocstor_41526_object4338/head//24 by osd.21.0:15963 2014-08-11 15:29:29.801766 2014-08-13 18:06:06.692444 7f5c973a57a0 20 read_log 192834'14491 (192834'14468) delete 73470d36/benchmark_data_storage0111-ib.pocstor_41526_object7683/head//24 by osd.21.0:15970 2014-08-11 15:29:29.822667 2014-08-13 18:06:06.692456 7f5c973a57a0 20 read_log 192834'14492 (192834'14469) delete 62edbd36/benchmark_data_storage0111-ib.pocstor_41526_object4274/head//24 by osd.21.0:15971 2014-08-11 15:29:29.825377 2014-08-13 18:06:06.692476 7f5c973a57a0 20 read_log 192834'14493 (192834'14470) delete 2bfcd36/benchmark_data_storage0111-ib.pocstor_41526_object7105/head//24 by osd.21.0:15972 2014-08-11 15:29:29.828145 2014-08-13 18:06:06.692489 7f5c973a57a0 20 read_log 192834'14506 (192834'14249) clean b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15985 2014-08-11 15:29:34.911828 2014-08-13 18:06:06.692502 7f5c973a57a0 20 read_log 192834'14520 (192834'14506) delete b7cc6936/benchmark_data_storage0111-ib.pocstor_41526_object11084/head//24 by osd.21.0:15999 2014-08-11 15:29:34.961135 2014-08-13 18:06:06.692522 7f5c973a57a0 20 read_log 192834'14532 (192834'14269) clean 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:16011 2014-08-11 15:29:40.016106 2014-08-13 18:06:06.692535 7f5c973a57a0 20 read_log 192834'14537 (192834'14290) clean fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:16016 2014-08-11 15:29:40.032764 2014-08-13 18:06:06.692548 7f5c973a57a0 20 read_log 192834'14552 (192834'14532) delete 421e0136/benchmark_data_storage0111-ib.pocstor_41526_object12961/head//24 by osd.21.0:16031 2014-08-11 15:29:40.075336 2014-08-13 18:06:06.692567 7f5c973a57a0 20 read_log 192834'14557 (192834'14537) delete fcc53d36/benchmark_data_storage0111-ib.pocstor_41526_object14180/head//24 by osd.21.0:16036 2014-08-11 15:29:40.090168 2014-08-13 18:06:06.692581 7f5c973a57a0 20 read_log 192834'14576 (192834'14330) clean f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:16055 2014-08-11 15:29:45.153743 2014-08-13 18:06:06.692593 7f5c973a57a0 20 read_log 192834'14579 (192834'14328) clean cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:16058 2014-08-11 15:29:45.161286 2014-08-13 18:06:06.692606 7f5c973a57a0 20 read_log 192834'14581 (192834'14310) clean 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:16060 2014-08-11 15:29:45.163109 2014-08-13 18:06:06.692639 7f5c973a57a0 20 read_log 192834'14582 (192834'14314) clean 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:16061 2014-08-11 15:29:45.165147 2014-08-13 18:06:06.692653 7f5c973a57a0 20 read_log 192834'14583 (192834'14332) clean f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:16062 2014-08-11 15:29:45.170133 2014-08-13 18:06:06.692669 7f5c973a57a0 20 read_log 192834'14602 (192834'14576) delete f1cb1136/benchmark_data_storage0111-ib.pocstor_41526_object16815/head//24 by osd.21.0:16081 2014-08-11 15:29:45.220465 2014-08-13 18:06:06.692689 7f5c973a57a0 20 read_log 192834'14606 (192834'14579) delete cbc46536/benchmark_data_storage0111-ib.pocstor_41526_object16763/head//24 by osd.21.0:16085 2014-08-11 15:29:45.225400 2014-08-13 18:06:06.692705 7f5c973a57a0 20 read_log 192834'14608 (192834'14581) delete 71103936/benchmark_data_storage0111-ib.pocstor_41526_object16255/head//24 by osd.21.0:16087 2014-08-11 15:29:45.227852 2014-08-13 18:06:06.692718 7f5c973a57a0 20 read_log 192834'14609 (192834'14582) delete 8dfa7936/benchmark_data_storage0111-ib.pocstor_41526_object16340/head//24 by osd.21.0:16088 2014-08-11 15:29:45.228280 2014-08-13 18:06:06.692730 7f5c973a57a0 20 read_log 192834'14610 (192834'14583) delete f94ccd36/benchmark_data_storage0111-ib.pocstor_41526_object16841/head//24 by osd.21.0:16089 2014-08-11 15:29:45.230406 2014-08-13 18:06:06.692743 7f5c973a57a0 20 read_log 192834'14628 (192834'14358) clean e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:16107 2014-08-11 15:29:50.259508 2014-08-13 18:06:06.692755 7f5c973a57a0 20 read_log 192834'14629 (192834'14357) clean 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:16108 2014-08-11 15:29:50.260151 2014-08-13 18:06:06.692767 7f5c973a57a0 20 read_log 192834'14634 (192834'14362) clean 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:16113 2014-08-11 15:29:50.267181 2014-08-13 18:06:06.692779 7f5c973a57a0 20 read_log 192834'14635 (192834'14375) clean dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:16114 2014-08-11 15:29:50.268776 2014-08-13 18:06:06.692807 7f5c973a57a0 20 read_log 192834'14643 (192834'14628) delete e39e0136/benchmark_data_storage0111-ib.pocstor_41526_object18855/head//24 by osd.21.0:16122 2014-08-11 15:29:50.283268 2014-08-13 18:06:06.692822 7f5c973a57a0 20 read_log 192834'14646 (192834'14629) delete 86d84136/benchmark_data_storage0111-ib.pocstor_41526_object18549/head//24 by osd.21.0:16125 2014-08-11 15:29:50.285639 2014-08-13 18:06:06.692834 7f5c973a57a0 20 read_log 192834'14651 (192834'14634) delete 359d0536/benchmark_data_storage0111-ib.pocstor_41526_object19496/head//24 by osd.21.0:16130 2014-08-11 15:29:50.291084 2014-08-13 18:06:06.692850 7f5c973a57a0 20 read_log 192834'14652 (192834'14635) delete dfc24536/benchmark_data_storage0111-ib.pocstor_41526_object21325/head//24 by osd.21.0:16131 2014-08-11 15:29:50.291353 2014-08-13 18:06:06.692863 7f5c973a57a0 20 read_log 192888'14690 (192888'14690) modify 136/hit_set_24.136_archive_2014-08-11 16:02:49.190600_2014-08-11 16:44:58.717710/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.718577 2014-08-13 18:06:06.692876 7f5c973a57a0 20 read_log 192888'14691 (0'0) promote d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by osd.4.0:21405 2014-08-11 16:44:58.722761 2014-08-13 18:06:06.692889 7f5c973a57a0 20 read_log 192888'14692 (192888'14691) modify d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.1315259.0:907 2014-08-11 16:44:58.712700 2014-08-13 18:06:06.692912 7f5c973a57a0 20 read_log 192888'14699 (0'0) promote 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by osd.4.0:21490 2014-08-11 16:45:10.414474 2014-08-13 18:06:06.692934 7f5c973a57a0 20 read_log 192888'14700 (192888'14699) modify 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.1315259.0:3986 2014-08-11 16:45:10.402820 2014-08-13 18:06:06.692951 7f5c973a57a0 20 read_log 192888'14709 (0'0) promote d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by osd.4.0:21530 2014-08-11 16:45:14.586515 2014-08-13 18:06:06.692963 7f5c973a57a0 20 read_log 192888'14710 (192888'14709) modify d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.1315259.0:5118 2014-08-11 16:45:14.578130 2014-08-13 18:06:06.692976 7f5c973a57a0 20 read_log 192888'14711 (0'0) promote 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by osd.4.0:21532 2014-08-11 16:45:14.705578 2014-08-13 18:06:06.692991 7f5c973a57a0 20 read_log 192888'14712 (192888'14711) modify 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.1315259.0:5150 2014-08-11 16:45:14.697865 2014-08-13 18:06:06.693003 7f5c973a57a0 20 read_log 192888'14715 (0'0) promote 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by osd.4.0:21554 2014-08-11 16:45:17.045726 2014-08-13 18:06:06.693016 7f5c973a57a0 20 read_log 192888'14716 (192888'14715) modify 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.1315259.0:5794 2014-08-11 16:45:17.033330 2014-08-13 18:06:06.693028 7f5c973a57a0 20 read_log 192888'14717 (0'0) promote 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by osd.4.0:21570 2014-08-11 16:45:19.261820 2014-08-13 18:06:06.693051 7f5c973a57a0 20 read_log 192888'14718 (192888'14717) modify 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.1315259.0:6409 2014-08-11 16:45:19.252967 2014-08-13 18:06:06.693069 7f5c973a57a0 20 read_log 192888'14723 (0'0) promote f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by osd.4.0:21596 2014-08-11 16:45:23.961112 2014-08-13 18:06:06.693082 7f5c973a57a0 20 read_log 192888'14724 (192888'14723) modify f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.1315259.0:7710 2014-08-11 16:45:23.950175 2014-08-13 18:06:06.693097 7f5c973a57a0 20 read_log 192888'14725 (0'0) promote fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by osd.4.0:21618 2014-08-11 16:45:27.126301 2014-08-13 18:06:06.693119 7f5c973a57a0 20 read_log 192888'14726 (192888'14725) modify fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.1315259.0:8542 2014-08-11 16:45:27.117895 2014-08-13 18:06:06.693136 7f5c973a57a0 20 read_log 192888'14735 (0'0) promote bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by osd.4.0:21678 2014-08-11 16:45:35.747447 2014-08-13 18:06:06.693156 7f5c973a57a0 20 read_log 192888'14736 (192888'14735) modify bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.1315259.0:10782 2014-08-11 16:45:35.738505 2014-08-13 18:06:06.693169 7f5c973a57a0 20 read_log 192888'14741 (0'0) promote b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by osd.4.0:21738 2014-08-11 16:45:44.956458 2014-08-13 18:06:06.693182 7f5c973a57a0 20 read_log 192888'14742 (192888'14741) modify b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.1315259.0:13022 2014-08-11 16:45:44.947879 2014-08-13 18:06:06.693194 7f5c973a57a0 20 read_log 192888'14743 (0'0) promote 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by osd.4.0:21762 2014-08-11 16:45:49.158999 2014-08-13 18:06:06.693231 7f5c973a57a0 20 read_log 192888'14744 (192888'14743) modify 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.1315259.0:14101 2014-08-11 16:45:49.149678 2014-08-13 18:06:06.693247 7f5c973a57a0 20 read_log 192888'14745 (0'0) promote 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by osd.4.0:21770 2014-08-11 16:45:50.824551 2014-08-13 18:06:06.693269 7f5c973a57a0 20 read_log 192888'14746 (192888'14745) modify 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.1315259.0:14535 2014-08-11 16:45:50.816381 2014-08-13 18:06:06.693288 7f5c973a57a0 20 read_log 192888'14751 (0'0) promote 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by osd.4.0:21802 2014-08-11 16:45:55.654601 2014-08-13 18:06:06.693301 7f5c973a57a0 20 read_log 192888'14752 (192888'14751) modify 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.1315259.0:15795 2014-08-11 16:45:55.644691 2014-08-13 18:06:06.693315 7f5c973a57a0 20 read_log 192888'14753 (0'0) promote de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by osd.4.0:21844 2014-08-11 16:46:00.692846 2014-08-13 18:06:06.693328 7f5c973a57a0 20 read_log 192888'14754 (192888'14753) modify de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.1315259.0:17084 2014-08-11 16:46:00.684857 2014-08-13 18:06:06.693358 7f5c973a57a0 20 read_log 192888'14755 (0'0) promote 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by osd.4.0:21878 2014-08-11 16:46:04.754933 2014-08-13 18:06:06.693374 7f5c973a57a0 20 read_log 192888'14756 (192888'14755) modify 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.1315259.0:18088 2014-08-11 16:46:04.746767 2014-08-13 18:06:06.693391 7f5c973a57a0 20 read_log 192888'14761 (0'0) promote f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by osd.4.0:21910 2014-08-11 16:46:08.512692 2014-08-13 18:06:06.693422 7f5c973a57a0 20 read_log 192888'14762 (192888'14761) modify f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.1315259.0:19058 2014-08-11 16:46:08.502735 2014-08-13 18:06:06.693441 7f5c973a57a0 20 read_log 192888'14765 (0'0) promote f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by osd.4.0:21920 2014-08-11 16:46:10.436109 2014-08-13 18:06:06.693461 7f5c973a57a0 20 read_log 192888'14766 (192888'14765) modify f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.1315259.0:19541 2014-08-11 16:46:10.426708 2014-08-13 18:06:06.693478 7f5c973a57a0 20 read_log 192888'14767 (0'0) promote 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by osd.4.0:21932 2014-08-11 16:46:12.044100 2014-08-13 18:06:06.693493 7f5c973a57a0 20 read_log 192888'14768 (192888'14767) modify 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.1315259.0:19978 2014-08-11 16:46:12.035433 2014-08-13 18:06:06.693506 7f5c973a57a0 20 read_log 192888'14771 (0'0) promote 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by osd.4.0:21948 2014-08-11 16:46:14.966086 2014-08-13 18:06:06.693518 7f5c973a57a0 20 read_log 192888'14772 (192888'14771) modify 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.1315259.0:20752 2014-08-11 16:46:14.957070 2014-08-13 18:06:06.693531 7f5c973a57a0 20 read_log 192888'14773 (0'0) promote cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by osd.4.0:21964 2014-08-11 16:46:16.388696 2014-08-13 18:06:06.693543 7f5c973a57a0 20 read_log 192888'14774 (192888'14773) modify cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.1315259.0:21139 2014-08-11 16:46:16.378753 2014-08-13 18:06:06.693555 7f5c973a57a0 20 read_log 192888'14775 (0'0) promote d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by osd.4.0:21982 2014-08-11 16:46:19.267371 2014-08-13 18:06:06.693567 7f5c973a57a0 20 read_log 192888'14776 (192888'14775) modify d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.1315259.0:21896 2014-08-11 16:46:19.259496 2014-08-13 18:06:06.693580 7f5c973a57a0 20 read_log 192888'14777 (0'0) promote 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by osd.4.0:22026 2014-08-11 16:46:26.229908 2014-08-13 18:06:06.693625 7f5c973a57a0 20 read_log 192888'14778 (192888'14777) modify 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.1315259.0:23626 2014-08-11 16:46:26.221699 2014-08-13 18:06:06.693643 7f5c973a57a0 20 read_log 192888'14781 (0'0) promote ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by osd.4.0:22094 2014-08-11 16:46:33.590742 2014-08-13 18:06:06.693655 7f5c973a57a0 20 read_log 192888'14782 (192888'14781) modify ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.1315259.0:25515 2014-08-11 16:46:33.582714 2014-08-13 18:06:06.693668 7f5c973a57a0 20 read_log 192888'14785 (0'0) promote 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by osd.4.0:22112 2014-08-11 16:46:37.009214 2014-08-13 18:06:06.693680 7f5c973a57a0 20 read_log 192888'14786 (192888'14785) modify 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.1315259.0:26303 2014-08-11 16:46:37.000753 2014-08-13 18:06:06.693692 7f5c973a57a0 20 read_log 192888'14789 (0'0) promote 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by osd.4.0:22130 2014-08-11 16:46:39.288439 2014-08-13 18:06:06.693705 7f5c973a57a0 20 read_log 192888'14790 (192888'14789) modify 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.1315259.0:26856 2014-08-11 16:46:39.280206 2014-08-13 18:06:06.693717 7f5c973a57a0 20 read_log 192888'14791 (0'0) promote 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by osd.4.0:22138 2014-08-11 16:46:40.808934 2014-08-13 18:06:06.693729 7f5c973a57a0 20 read_log 192888'14792 (192888'14791) modify 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.1315259.0:27190 2014-08-11 16:46:40.800424 2014-08-13 18:06:06.693741 7f5c973a57a0 20 read_log 192888'14797 (0'0) promote de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by osd.4.0:22160 2014-08-11 16:46:45.435962 2014-08-13 18:06:06.693753 7f5c973a57a0 20 read_log 192888'14798 (192888'14797) modify de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.1315259.0:28354 2014-08-11 16:46:45.428112 2014-08-13 18:06:06.693765 7f5c973a57a0 20 read_log 192888'14801 (0'0) promote 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by osd.4.0:22206 2014-08-11 16:46:53.077404 2014-08-13 18:06:06.693778 7f5c973a57a0 20 read_log 192888'14802 (192888'14801) modify 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.1315259.0:30259 2014-08-11 16:46:53.069164 2014-08-13 18:06:06.693790 7f5c973a57a0 20 read_log 192888'14805 (192888'14692) modify d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.1315259.0:31634 2014-08-11 16:46:56.152826 2014-08-13 18:06:06.693802 7f5c973a57a0 20 read_log 192888'14809 (192888'14700) modify 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.1315259.0:34713 2014-08-11 16:46:56.967895 2014-08-13 18:06:06.693823 7f5c973a57a0 20 read_log 192888'14814 (192888'14710) modify d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.1315259.0:35845 2014-08-11 16:46:57.261069 2014-08-13 18:06:06.693838 7f5c973a57a0 20 read_log 192888'14815 (192888'14712) modify 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.1315259.0:35877 2014-08-11 16:46:57.268588 2014-08-13 18:06:06.693850 7f5c973a57a0 20 read_log 192888'14817 (192888'14716) modify 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.1315259.0:36521 2014-08-11 16:46:57.444707 2014-08-13 18:06:06.693863 7f5c973a57a0 20 read_log 192888'14818 (192888'14718) modify 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.1315259.0:37136 2014-08-11 16:46:57.603716 2014-08-13 18:06:06.693875 7f5c973a57a0 20 read_log 192888'14821 (192888'14724) modify f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.1315259.0:38437 2014-08-11 16:46:57.947169 2014-08-13 18:06:06.693895 7f5c973a57a0 20 read_log 192888'14822 (192888'14726) modify fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.1315259.0:39269 2014-08-11 16:46:58.168996 2014-08-13 18:06:06.693907 7f5c973a57a0 20 read_log 192888'14827 (192888'14736) modify bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.1315259.0:41509 2014-08-11 16:46:58.763963 2014-08-13 18:06:06.693920 7f5c973a57a0 20 read_log 192888'14830 (192888'14742) modify b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.1315259.0:43749 2014-08-11 16:46:59.378358 2014-08-13 18:06:06.693932 7f5c973a57a0 20 read_log 192888'14831 (192888'14744) modify 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.1315259.0:44828 2014-08-11 16:46:59.688620 2014-08-13 18:06:06.693944 7f5c973a57a0 20 read_log 192888'14832 (192888'14746) modify 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.1315259.0:45262 2014-08-11 16:46:59.816251 2014-08-13 18:06:06.693959 7f5c973a57a0 20 read_log 192888'14835 (192888'14752) modify 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.1315259.0:46522 2014-08-11 16:47:00.183303 2014-08-13 18:06:06.693972 7f5c973a57a0 20 read_log 192888'14836 (192888'14754) modify de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.1315259.0:47811 2014-08-11 16:47:00.577208 2014-08-13 18:06:06.693984 7f5c973a57a0 20 read_log 192888'14837 (192888'14756) modify 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.1315259.0:48815 2014-08-11 16:47:00.873874 2014-08-13 18:06:06.694000 7f5c973a57a0 20 read_log 192888'14840 (192888'14762) modify f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.1315259.0:49785 2014-08-11 16:47:01.172149 2014-08-13 18:06:06.694017 7f5c973a57a0 20 read_log 192888'14842 (192888'14766) modify f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.1315259.0:50268 2014-08-11 16:47:01.315666 2014-08-13 18:06:06.694029 7f5c973a57a0 20 read_log 192888'14843 (192888'14768) modify 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.1315259.0:50705 2014-08-11 16:47:01.452292 2014-08-13 18:06:06.694041 7f5c973a57a0 20 read_log 192888'14845 (192888'14772) modify 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.1315259.0:51479 2014-08-11 16:47:01.691909 2014-08-13 18:06:06.694057 7f5c973a57a0 20 read_log 192888'14846 (192888'14774) modify cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.1315259.0:51866 2014-08-11 16:47:01.805766 2014-08-13 18:06:06.694070 7f5c973a57a0 20 read_log 192888'14847 (192888'14776) modify d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.1315259.0:52623 2014-08-11 16:47:02.025953 2014-08-13 18:06:06.694082 7f5c973a57a0 20 read_log 192888'14848 (192888'14778) modify 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.1315259.0:54353 2014-08-11 16:47:02.543323 2014-08-13 18:06:06.694094 7f5c973a57a0 20 read_log 192888'14850 (192888'14782) modify ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.1315259.0:56242 2014-08-11 16:47:03.144404 2014-08-13 18:06:06.694118 7f5c973a57a0 20 read_log 192888'14852 (192888'14786) modify 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.1315259.0:57030 2014-08-11 16:47:03.390023 2014-08-13 18:06:06.694131 7f5c973a57a0 20 read_log 192888'14854 (192888'14790) modify 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.1315259.0:57583 2014-08-11 16:47:03.554640 2014-08-13 18:06:06.694144 7f5c973a57a0 20 read_log 192888'14855 (192888'14792) modify 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.1315259.0:57917 2014-08-11 16:47:03.649092 2014-08-13 18:06:06.694173 7f5c973a57a0 20 read_log 192888'14858 (192888'14798) modify de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.1315259.0:59081 2014-08-11 16:47:03.993833 2014-08-13 18:06:06.694202 7f5c973a57a0 20 read_log 192888'14860 (192888'14802) modify 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.1315259.0:60986 2014-08-11 16:47:04.599846 2014-08-13 18:06:06.694219 7f5c973a57a0 20 read_log 192888'14862 (192888'14862) modify 136/hit_set_24.136_archive_2014-08-11 16:44:58.718710_2014-08-11 17:05:15.819389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.819628 2014-08-13 18:06:06.694232 7f5c973a57a0 20 read_log 192888'14863 (192888'14690) delete 136/hit_set_24.136_archive_2014-08-11 16:02:49.190600_2014-08-11 16:44:58.717710/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.819628 2014-08-13 18:06:06.694244 7f5c973a57a0 20 read_log 192888'14864 (0'0) promote 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by osd.4.0:22651 2014-08-11 17:05:15.824490 2014-08-13 18:06:06.694257 7f5c973a57a0 20 read_log 192888'14865 (192888'14864) modify 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.942868.0:201 2014-08-11 17:05:15.813467 2014-08-13 18:06:06.694286 7f5c973a57a0 20 read_log 192888'14870 (0'0) promote ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by osd.4.0:22716 2014-08-11 17:05:25.976498 2014-08-13 18:06:06.694299 7f5c973a57a0 20 read_log 192888'14871 (192888'14870) modify ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.942868.0:2711 2014-08-11 17:05:25.967623 2014-08-13 18:06:06.694311 7f5c973a57a0 20 read_log 192888'14874 (0'0) promote cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by osd.4.0:22752 2014-08-11 17:05:30.458283 2014-08-13 18:06:06.694327 7f5c973a57a0 20 read_log 192888'14875 (192888'14874) modify cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.942868.0:3924 2014-08-11 17:05:30.448644 2014-08-13 18:06:06.694342 7f5c973a57a0 20 read_log 192888'14876 (0'0) promote d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by osd.4.0:22756 2014-08-11 17:05:31.292454 2014-08-13 18:06:06.694355 7f5c973a57a0 20 read_log 192888'14877 (192888'14876) modify d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.942868.0:4151 2014-08-11 17:05:31.284720 2014-08-13 18:06:06.694381 7f5c973a57a0 20 read_log 192888'14878 (0'0) promote 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by osd.4.0:22770 2014-08-11 17:05:32.042389 2014-08-13 18:06:06.694395 7f5c973a57a0 20 read_log 192888'14879 (192888'14878) modify 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.942868.0:4355 2014-08-11 17:05:32.034783 2014-08-13 18:06:06.694407 7f5c973a57a0 20 read_log 192888'14882 (0'0) promote 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by osd.4.0:22796 2014-08-11 17:05:36.126236 2014-08-13 18:06:06.694419 7f5c973a57a0 20 read_log 192888'14883 (192888'14882) modify 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.942868.0:5402 2014-08-11 17:05:36.118119 2014-08-13 18:06:06.694437 7f5c973a57a0 20 read_log 192888'14890 (0'0) promote b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by osd.4.0:22836 2014-08-11 17:05:40.585835 2014-08-13 18:06:06.694450 7f5c973a57a0 20 read_log 192888'14891 (192888'14890) modify b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.942868.0:6628 2014-08-11 17:05:40.575227 2014-08-13 18:06:06.694473 7f5c973a57a0 20 read_log 192888'14898 (0'0) promote 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by osd.4.0:22896 2014-08-11 17:05:50.050596 2014-08-13 18:06:06.694489 7f5c973a57a0 20 read_log 192888'14899 (192888'14898) modify 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.942868.0:9147 2014-08-11 17:05:50.041369 2014-08-13 18:06:06.694509 7f5c973a57a0 20 read_log 192888'14900 (0'0) promote 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by osd.4.0:22900 2014-08-11 17:05:50.739871 2014-08-13 18:06:06.694527 7f5c973a57a0 20 read_log 192888'14901 (192888'14900) modify 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.942868.0:9330 2014-08-11 17:05:50.729715 2014-08-13 18:06:06.694540 7f5c973a57a0 20 read_log 192888'14902 (0'0) promote b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by osd.4.0:22918 2014-08-11 17:05:54.495245 2014-08-13 18:06:06.694552 7f5c973a57a0 20 read_log 192888'14903 (192888'14902) modify b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.942868.0:10357 2014-08-11 17:05:54.484795 2014-08-13 18:06:06.694573 7f5c973a57a0 20 read_log 192888'14908 (0'0) promote b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by osd.4.0:22928 2014-08-11 17:05:55.293341 2014-08-13 18:06:06.694589 7f5c973a57a0 20 read_log 192888'14909 (192888'14908) modify b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.942868.0:10580 2014-08-11 17:05:55.285062 2014-08-13 18:06:06.694602 7f5c973a57a0 20 read_log 192888'14912 (0'0) promote 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by osd.4.0:22974 2014-08-11 17:06:03.090657 2014-08-13 18:06:06.694614 7f5c973a57a0 20 read_log 192888'14913 (192888'14912) modify 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.942868.0:12721 2014-08-11 17:06:03.079712 2014-08-13 18:06:06.694627 7f5c973a57a0 20 read_log 192888'14916 (0'0) promote 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by osd.4.0:23036 2014-08-11 17:06:13.589716 2014-08-13 18:06:06.694639 7f5c973a57a0 20 read_log 192888'14917 (192888'14916) modify 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.942868.0:15350 2014-08-11 17:06:13.578192 2014-08-13 18:06:06.694651 7f5c973a57a0 20 read_log 192888'14922 (0'0) promote d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by osd.4.0:23064 2014-08-11 17:06:17.643145 2014-08-13 18:06:06.694664 7f5c973a57a0 20 read_log 192888'14923 (192888'14922) modify d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.942868.0:16398 2014-08-11 17:06:17.632836 2014-08-13 18:06:06.694676 7f5c973a57a0 20 read_log 192888'14924 (0'0) promote 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by osd.4.0:23068 2014-08-11 17:06:18.733220 2014-08-13 18:06:06.694688 7f5c973a57a0 20 read_log 192888'14925 (192888'14924) modify 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.942868.0:16661 2014-08-11 17:06:18.725521 2014-08-13 18:06:06.694700 7f5c973a57a0 20 read_log 192888'14926 (0'0) promote 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by osd.4.0:23080 2014-08-11 17:06:20.054930 2014-08-13 18:06:06.694712 7f5c973a57a0 20 read_log 192888'14927 (192888'14926) modify 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.942868.0:16998 2014-08-11 17:06:20.046703 2014-08-13 18:06:06.694725 7f5c973a57a0 20 read_log 192888'14928 (0'0) promote 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by osd.4.0:23082 2014-08-11 17:06:20.320454 2014-08-13 18:06:06.694737 7f5c973a57a0 20 read_log 192888'14929 (192888'14928) modify 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.942868.0:17068 2014-08-11 17:06:20.311629 2014-08-13 18:06:06.694749 7f5c973a57a0 20 read_log 192888'14930 (0'0) promote 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by osd.4.0:23102 2014-08-11 17:06:22.293012 2014-08-13 18:06:06.694762 7f5c973a57a0 20 read_log 192888'14931 (192888'14930) modify 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.942868.0:17590 2014-08-11 17:06:22.283891 2014-08-13 18:06:06.694799 7f5c973a57a0 20 read_log 192888'14932 (0'0) promote 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by osd.4.0:23106 2014-08-11 17:06:22.737968 2014-08-13 18:06:06.694815 7f5c973a57a0 20 read_log 192888'14933 (192888'14932) modify 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.942868.0:17708 2014-08-11 17:06:22.729526 2014-08-13 18:06:06.694844 7f5c973a57a0 20 read_log 192888'14934 (0'0) promote f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by osd.4.0:23126 2014-08-11 17:06:25.907190 2014-08-13 18:06:06.694860 7f5c973a57a0 20 read_log 192888'14935 (192888'14934) modify f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.942868.0:18549 2014-08-11 17:06:25.897516 2014-08-13 18:06:06.694872 7f5c973a57a0 20 read_log 192888'14936 (0'0) promote cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by osd.4.0:23146 2014-08-11 17:06:28.687524 2014-08-13 18:06:06.694885 7f5c973a57a0 20 read_log 192888'14937 (192888'14936) modify cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.942868.0:19288 2014-08-11 17:06:28.678528 2014-08-13 18:06:06.694897 7f5c973a57a0 20 read_log 192888'14940 (0'0) promote 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by osd.4.0:23166 2014-08-11 17:06:31.093834 2014-08-13 18:06:06.694909 7f5c973a57a0 20 read_log 192888'14941 (192888'14940) modify 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.942868.0:19927 2014-08-11 17:06:31.085201 2014-08-13 18:06:06.694922 7f5c973a57a0 20 read_log 192888'14944 (0'0) promote 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by osd.4.0:23184 2014-08-11 17:06:34.051456 2014-08-13 18:06:06.694934 7f5c973a57a0 20 read_log 192888'14945 (192888'14944) modify 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.942868.0:20698 2014-08-11 17:06:34.042873 2014-08-13 18:06:06.694946 7f5c973a57a0 20 read_log 192888'14950 (0'0) promote e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by osd.4.0:23218 2014-08-11 17:06:38.660949 2014-08-13 18:06:06.694958 7f5c973a57a0 20 read_log 192888'14951 (192888'14950) modify e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.942868.0:21915 2014-08-11 17:06:38.653278 2014-08-13 18:06:06.694971 7f5c973a57a0 20 read_log 192888'14952 (0'0) promote f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by osd.4.0:23232 2014-08-11 17:06:41.072538 2014-08-13 18:06:06.694983 7f5c973a57a0 20 read_log 192888'14953 (192888'14952) modify f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.942868.0:22516 2014-08-11 17:06:41.064647 2014-08-13 18:06:06.694996 7f5c973a57a0 20 read_log 192888'14956 (0'0) promote fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by osd.4.0:23242 2014-08-11 17:06:41.993908 2014-08-13 18:06:06.695011 7f5c973a57a0 20 read_log 192888'14957 (192888'14956) modify fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.942868.0:22765 2014-08-11 17:06:41.985848 2014-08-13 18:06:06.695024 7f5c973a57a0 20 read_log 192888'14962 (0'0) promote 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by osd.4.0:23278 2014-08-11 17:06:46.569583 2014-08-13 18:06:06.695038 7f5c973a57a0 20 read_log 192888'14963 (192888'14962) modify 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.942868.0:23961 2014-08-11 17:06:46.560526 2014-08-13 18:06:06.695050 7f5c973a57a0 20 read_log 192888'14968 (0'0) promote f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by osd.4.0:23372 2014-08-11 17:06:59.026632 2014-08-13 18:06:06.695064 7f5c973a57a0 20 read_log 192888'14969 (192888'14968) modify f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.942868.0:27218 2014-08-11 17:06:59.016692 2014-08-13 18:06:06.695087 7f5c973a57a0 20 read_log 192888'14970 (0'0) promote 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by osd.4.0:23380 2014-08-11 17:07:00.411505 2014-08-13 18:06:06.695102 7f5c973a57a0 20 read_log 192888'14971 (192888'14970) modify 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.942868.0:27596 2014-08-11 17:07:00.402458 2014-08-13 18:06:06.695114 7f5c973a57a0 20 read_log 192888'14972 (0'0) promote 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by osd.4.0:23416 2014-08-11 17:07:06.304442 2014-08-13 18:06:06.695129 7f5c973a57a0 20 read_log 192888'14973 (192888'14972) modify 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.942868.0:29146 2014-08-11 17:07:06.294439 2014-08-13 18:06:06.695142 7f5c973a57a0 20 read_log 192888'14978 (0'0) promote 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by osd.4.0:23452 2014-08-11 17:07:11.741174 2014-08-13 18:06:06.695162 7f5c973a57a0 20 read_log 192888'14979 (192888'14978) modify 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.942868.0:30478 2014-08-11 17:07:11.732510 2014-08-13 18:06:06.695174 7f5c973a57a0 20 read_log 192888'14982 (192888'14865) modify 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.942868.0:31467 2014-08-11 17:07:15.934797 2014-08-13 18:06:06.695187 7f5c973a57a0 20 read_log 192888'14985 (192888'14871) modify ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.942868.0:33977 2014-08-11 17:07:16.663792 2014-08-13 18:06:06.695203 7f5c973a57a0 20 read_log 192888'14987 (192888'14875) modify cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.942868.0:35190 2014-08-11 17:07:17.009973 2014-08-13 18:06:06.695215 7f5c973a57a0 20 read_log 192888'14988 (192888'14877) modify d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.942868.0:35417 2014-08-11 17:07:17.069364 2014-08-13 18:06:06.695230 7f5c973a57a0 20 read_log 192888'14989 (192888'14879) modify 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.942868.0:35621 2014-08-11 17:07:17.121603 2014-08-13 18:06:06.695245 7f5c973a57a0 20 read_log 192888'14991 (192888'14883) modify 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.942868.0:36668 2014-08-11 17:07:17.403369 2014-08-13 18:06:06.695258 7f5c973a57a0 20 read_log 192888'14995 (192888'14891) modify b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.942868.0:37894 2014-08-11 17:07:17.737166 2014-08-13 18:06:06.695270 7f5c973a57a0 20 read_log 192888'14999 (192888'14899) modify 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.942868.0:40413 2014-08-11 17:07:18.444316 2014-08-13 18:06:06.695282 7f5c973a57a0 20 read_log 192888'15000 (192888'14901) modify 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.942868.0:40596 2014-08-11 17:07:18.495439 2014-08-13 18:06:06.695299 7f5c973a57a0 20 read_log 192888'15001 (192888'14903) modify b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.942868.0:41623 2014-08-11 17:07:18.778204 2014-08-13 18:06:06.695311 7f5c973a57a0 20 read_log 192888'15004 (192888'14909) modify b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.942868.0:41846 2014-08-11 17:07:18.838175 2014-08-13 18:06:06.695324 7f5c973a57a0 20 read_log 192888'15006 (192888'14913) modify 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.942868.0:43987 2014-08-11 17:07:19.435520 2014-08-13 18:06:06.695339 7f5c973a57a0 20 read_log 192888'15008 (192888'14917) modify 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.942868.0:46616 2014-08-11 17:07:20.172366 2014-08-13 18:06:06.695354 7f5c973a57a0 20 read_log 192888'15011 (192888'14923) modify d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.942868.0:47664 2014-08-11 17:07:20.481595 2014-08-13 18:06:06.695387 7f5c973a57a0 20 read_log 192888'15012 (192888'14925) modify 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.942868.0:47927 2014-08-11 17:07:20.568369 2014-08-13 18:06:06.695403 7f5c973a57a0 20 read_log 192888'15013 (192888'14927) modify 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.942868.0:48264 2014-08-11 17:07:20.670579 2014-08-13 18:06:06.695416 7f5c973a57a0 20 read_log 192888'15014 (192888'14929) modify 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.942868.0:48334 2014-08-11 17:07:20.690068 2014-08-13 18:06:06.695429 7f5c973a57a0 20 read_log 192888'15015 (192888'14931) modify 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.942868.0:48856 2014-08-11 17:07:20.837791 2014-08-13 18:06:06.695441 7f5c973a57a0 20 read_log 192888'15016 (192888'14933) modify 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.942868.0:48974 2014-08-11 17:07:20.871996 2014-08-13 18:06:06.695453 7f5c973a57a0 20 read_log 192888'15017 (192888'14935) modify f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.942868.0:49815 2014-08-11 17:07:21.132037 2014-08-13 18:06:06.695466 7f5c973a57a0 20 read_log 192888'15018 (192888'14937) modify cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.942868.0:50554 2014-08-11 17:07:21.360446 2014-08-13 18:06:06.695478 7f5c973a57a0 20 read_log 192888'15020 (192888'14941) modify 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.942868.0:51193 2014-08-11 17:07:21.569287 2014-08-13 18:06:06.695490 7f5c973a57a0 20 read_log 192888'15022 (192888'14945) modify 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.942868.0:51964 2014-08-11 17:07:21.802840 2014-08-13 18:06:06.695511 7f5c973a57a0 20 read_log 192888'15025 (192888'14951) modify e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.942868.0:53181 2014-08-11 17:07:22.175450 2014-08-13 18:06:06.695527 7f5c973a57a0 20 read_log 192888'15026 (192888'14953) modify f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.942868.0:53782 2014-08-11 17:07:22.371808 2014-08-13 18:06:06.695540 7f5c973a57a0 20 read_log 192888'15028 (192888'14957) modify fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.942868.0:54031 2014-08-11 17:07:22.445687 2014-08-13 18:06:06.695563 7f5c973a57a0 20 read_log 192888'15031 (192888'14963) modify 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.942868.0:55227 2014-08-11 17:07:22.779573 2014-08-13 18:06:06.695604 7f5c973a57a0 20 read_log 192888'15034 (192888'14969) modify f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.942868.0:58484 2014-08-11 17:07:23.757261 2014-08-13 18:06:06.695619 7f5c973a57a0 20 read_log 192888'15035 (192888'14971) modify 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.942868.0:58862 2014-08-11 17:07:23.872381 2014-08-13 18:06:06.695632 7f5c973a57a0 20 read_log 192888'15036 (192888'14973) modify 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.942868.0:60412 2014-08-11 17:07:24.332756 2014-08-13 18:06:06.695652 7f5c973a57a0 20 read_log 192888'15039 (192888'14979) modify 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.942868.0:61744 2014-08-11 17:07:24.733870 2014-08-13 18:06:06.695665 7f5c973a57a0 20 read_log 193305'15041 (193305'15041) modify 136/hit_set_24.136_archive_2014-08-11 18:37:04.911155_2014-08-11 18:42:38.276778/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.276999 2014-08-13 18:06:06.695686 7f5c973a57a0 20 read_log 193305'15042 (192888'14862) delete 136/hit_set_24.136_archive_2014-08-11 16:44:58.718710_2014-08-11 17:05:15.819389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:38.276999 2014-08-13 18:06:06.695729 7f5c973a57a0 20 read_log 193305'15043 (0'0) promote 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by osd.4.0:18 2014-08-11 18:42:38.311783 2014-08-13 18:06:06.695746 7f5c973a57a0 20 read_log 193305'15044 (193305'15043) modify 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.951337.0:1360 2014-08-11 18:42:38.270697 2014-08-13 18:06:06.695758 7f5c973a57a0 20 read_log 193305'15045 (0'0) promote fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by osd.4.0:39 2014-08-11 18:42:42.796079 2014-08-13 18:06:06.695771 7f5c973a57a0 20 read_log 193305'15046 (193305'15045) modify fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.951337.0:2561 2014-08-11 18:42:42.788439 2014-08-13 18:06:06.695783 7f5c973a57a0 20 read_log 193305'15047 (0'0) promote 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by osd.4.0:49 2014-08-11 18:42:47.155282 2014-08-13 18:06:06.695795 7f5c973a57a0 20 read_log 193305'15048 (193305'15047) modify 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.951337.0:3762 2014-08-11 18:42:47.147819 2014-08-13 18:06:06.695808 7f5c973a57a0 20 read_log 193305'15049 (0'0) promote eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by osd.4.0:59 2014-08-11 18:42:49.602003 2014-08-13 18:06:06.695820 7f5c973a57a0 20 read_log 193305'15050 (193305'15049) modify eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.951337.0:4423 2014-08-11 18:42:49.594371 2014-08-13 18:06:06.695832 7f5c973a57a0 20 read_log 193305'15051 (0'0) promote 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by osd.4.0:77 2014-08-11 18:42:52.819201 2014-08-13 18:06:06.695844 7f5c973a57a0 20 read_log 193305'15052 (193305'15051) modify 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.951337.0:5281 2014-08-11 18:42:52.809676 2014-08-13 18:06:06.695857 7f5c973a57a0 20 read_log 193305'15053 (0'0) promote 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by osd.4.0:81 2014-08-11 18:42:53.477155 2014-08-13 18:06:06.695869 7f5c973a57a0 20 read_log 193305'15054 (193305'15053) modify 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.951337.0:5464 2014-08-11 18:42:53.469492 2014-08-13 18:06:06.695885 7f5c973a57a0 20 read_log 193305'15055 (0'0) promote cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by osd.4.0:83 2014-08-11 18:42:53.603123 2014-08-13 18:06:06.695897 7f5c973a57a0 20 read_log 193305'15056 (193305'15055) modify cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.951337.0:5499 2014-08-11 18:42:53.595935 2014-08-13 18:06:06.695909 7f5c973a57a0 20 read_log 193305'15057 (0'0) promote 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by osd.4.0:91 2014-08-11 18:42:55.500508 2014-08-13 18:06:06.695921 7f5c973a57a0 20 read_log 193305'15058 (193305'15057) modify 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.951337.0:6019 2014-08-11 18:42:55.492431 2014-08-13 18:06:06.695935 7f5c973a57a0 20 read_log 193305'15059 (0'0) promote fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by osd.4.0:95 2014-08-11 18:42:56.017071 2014-08-13 18:06:06.695950 7f5c973a57a0 20 read_log 193305'15060 (193305'15059) modify fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.951337.0:6162 2014-08-11 18:42:56.008809 2014-08-13 18:06:06.695962 7f5c973a57a0 20 read_log 193305'15061 (0'0) promote 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by osd.4.0:97 2014-08-11 18:42:56.074783 2014-08-13 18:06:06.695975 7f5c973a57a0 20 read_log 193305'15062 (193305'15061) modify 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.951337.0:6178 2014-08-11 18:42:56.067236 2014-08-13 18:06:06.695998 7f5c973a57a0 20 read_log 193305'15063 (0'0) promote 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by osd.4.0:105 2014-08-11 18:42:57.125679 2014-08-13 18:06:06.696013 7f5c973a57a0 20 read_log 193305'15064 (193305'15063) modify 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.951337.0:6468 2014-08-11 18:42:57.118956 2014-08-13 18:06:06.696026 7f5c973a57a0 20 read_log 193305'15065 (0'0) promote 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by osd.4.0:115 2014-08-11 18:42:59.604539 2014-08-13 18:06:06.696041 7f5c973a57a0 20 read_log 193305'15066 (193305'15065) modify 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.951337.0:7155 2014-08-11 18:42:59.596647 2014-08-13 18:06:06.696054 7f5c973a57a0 20 read_log 193305'15067 (0'0) promote fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by osd.4.0:121 2014-08-11 18:43:01.108747 2014-08-13 18:06:06.696069 7f5c973a57a0 20 read_log 193305'15068 (193305'15067) modify fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.951337.0:7570 2014-08-11 18:43:01.101756 2014-08-13 18:06:06.696082 7f5c973a57a0 20 read_log 193305'15069 (0'0) promote 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by osd.4.0:151 2014-08-11 18:43:07.278217 2014-08-13 18:06:06.696096 7f5c973a57a0 20 read_log 193305'15070 (193305'15069) modify 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.951337.0:9178 2014-08-11 18:43:07.270127 2014-08-13 18:06:06.696108 7f5c973a57a0 20 read_log 193305'15071 (0'0) promote 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by osd.4.0:177 2014-08-11 18:43:13.799041 2014-08-13 18:06:06.696123 7f5c973a57a0 20 read_log 193305'15072 (193305'15071) modify 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.951337.0:10944 2014-08-11 18:43:13.791219 2014-08-13 18:06:06.696135 7f5c973a57a0 20 read_log 193305'15073 (0'0) promote 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by osd.4.0:183 2014-08-11 18:43:14.942553 2014-08-13 18:06:06.696154 7f5c973a57a0 20 read_log 193305'15074 (193305'15073) modify 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.951337.0:11230 2014-08-11 18:43:14.935644 2014-08-13 18:06:06.696169 7f5c973a57a0 20 read_log 193305'15075 (0'0) promote e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by osd.4.0:191 2014-08-11 18:43:15.947021 2014-08-13 18:06:06.696182 7f5c973a57a0 20 read_log 193305'15076 (193305'15075) modify e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.951337.0:11472 2014-08-11 18:43:15.937013 2014-08-13 18:06:06.696195 7f5c973a57a0 20 read_log 193305'15077 (0'0) promote a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by osd.4.0:207 2014-08-11 18:43:19.365193 2014-08-13 18:06:06.696209 7f5c973a57a0 20 read_log 193305'15078 (193305'15077) modify a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.951337.0:12403 2014-08-11 18:43:19.357697 2014-08-13 18:06:06.696243 7f5c973a57a0 20 read_log 193305'15079 (0'0) promote 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by osd.4.0:209 2014-08-11 18:43:20.198221 2014-08-13 18:06:06.696258 7f5c973a57a0 20 read_log 193305'15080 (193305'15079) modify 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.951337.0:12627 2014-08-11 18:43:20.189502 2014-08-13 18:06:06.696271 7f5c973a57a0 20 read_log 193305'15081 (0'0) promote 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by osd.4.0:267 2014-08-11 18:43:33.691573 2014-08-13 18:06:06.696287 7f5c973a57a0 20 read_log 193305'15082 (193305'15081) modify 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.951337.0:16353 2014-08-11 18:43:33.683619 2014-08-13 18:06:06.696308 7f5c973a57a0 20 read_log 193305'15083 (0'0) promote b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by osd.4.0:269 2014-08-11 18:43:33.771749 2014-08-13 18:06:06.696323 7f5c973a57a0 20 read_log 193305'15084 (193305'15083) modify b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.951337.0:16377 2014-08-11 18:43:33.764292 2014-08-13 18:06:06.696339 7f5c973a57a0 20 read_log 193305'15085 (0'0) promote 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by osd.4.0:297 2014-08-11 18:43:38.341555 2014-08-13 18:06:06.696354 7f5c973a57a0 20 read_log 193305'15086 (193305'15085) modify 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.951337.0:17716 2014-08-11 18:43:38.333431 2014-08-13 18:06:06.696378 7f5c973a57a0 20 read_log 193305'15087 (0'0) promote 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by osd.4.0:299 2014-08-11 18:43:38.975639 2014-08-13 18:06:06.696397 7f5c973a57a0 20 read_log 193305'15088 (193305'15087) modify 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.951337.0:17888 2014-08-11 18:43:38.967446 2014-08-13 18:06:06.696418 7f5c973a57a0 20 read_log 193305'15089 (0'0) promote 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by osd.4.0:313 2014-08-11 18:43:41.408822 2014-08-13 18:06:06.696433 7f5c973a57a0 20 read_log 193305'15090 (193305'15089) modify 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.951337.0:18582 2014-08-11 18:43:41.400157 2014-08-13 18:06:06.696446 7f5c973a57a0 20 read_log 193305'15091 (0'0) promote 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by osd.4.0:317 2014-08-11 18:43:41.787535 2014-08-13 18:06:06.696465 7f5c973a57a0 20 read_log 193305'15092 (193305'15091) modify 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.951337.0:18693 2014-08-11 18:43:41.779202 2014-08-13 18:06:06.696481 7f5c973a57a0 20 read_log 193305'15093 (0'0) promote c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by osd.4.0:347 2014-08-11 18:43:49.073360 2014-08-13 18:06:06.696493 7f5c973a57a0 20 read_log 193305'15094 (193305'15093) modify c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.951337.0:20681 2014-08-11 18:43:49.064897 2014-08-13 18:06:06.696513 7f5c973a57a0 20 read_log 193305'15095 (0'0) promote d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by osd.4.0:349 2014-08-11 18:43:49.594168 2014-08-13 18:06:06.696527 7f5c973a57a0 20 read_log 193305'15096 (193305'15095) modify d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.951337.0:20830 2014-08-11 18:43:49.585499 2014-08-13 18:06:06.696543 7f5c973a57a0 20 read_log 193305'15097 (0'0) promote 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by osd.4.0:363 2014-08-11 18:43:53.641904 2014-08-13 18:06:06.696555 7f5c973a57a0 20 read_log 193305'15098 (193305'15097) modify 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.951337.0:21953 2014-08-11 18:43:53.633662 2014-08-13 18:06:06.696577 7f5c973a57a0 20 read_log 193305'15099 (0'0) promote a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by osd.4.0:387 2014-08-11 18:43:57.040427 2014-08-13 18:06:06.696593 7f5c973a57a0 20 read_log 193305'15100 (193305'15099) modify a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.951337.0:22883 2014-08-11 18:43:57.029244 2014-08-13 18:06:06.696606 7f5c973a57a0 20 read_log 193305'15101 (0'0) promote f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by osd.4.0:389 2014-08-11 18:43:57.157043 2014-08-13 18:06:06.696618 7f5c973a57a0 20 read_log 193305'15102 (193305'15101) modify f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.951337.0:22918 2014-08-11 18:43:57.149422 2014-08-13 18:06:06.696638 7f5c973a57a0 20 read_log 193305'15103 (0'0) promote 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by osd.4.0:393 2014-08-11 18:43:58.516762 2014-08-13 18:06:06.696653 7f5c973a57a0 20 read_log 193305'15104 (193305'15103) modify 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.951337.0:23308 2014-08-11 18:43:58.508196 2014-08-13 18:06:06.696666 7f5c973a57a0 20 read_log 193305'15105 (0'0) promote 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by osd.4.0:433 2014-08-11 18:44:10.243562 2014-08-13 18:06:06.696678 7f5c973a57a0 20 read_log 193305'15106 (193305'15105) modify 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.951337.0:26516 2014-08-11 18:44:10.237426 2014-08-13 18:06:06.696702 7f5c973a57a0 20 read_log 193305'15107 (0'0) promote b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by osd.4.0:467 2014-08-11 18:44:17.099629 2014-08-13 18:06:06.696715 7f5c973a57a0 20 read_log 193305'15108 (193305'15107) modify b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.951337.0:28403 2014-08-11 18:44:17.091711 2014-08-13 18:06:06.696734 7f5c973a57a0 20 read_log 193305'15109 (0'0) promote daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by osd.4.0:489 2014-08-11 18:44:21.790134 2014-08-13 18:06:06.696749 7f5c973a57a0 20 read_log 193305'15110 (193305'15109) modify daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.951337.0:29675 2014-08-11 18:44:21.782752 2014-08-13 18:06:06.696762 7f5c973a57a0 20 read_log 193305'15111 (0'0) promote cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by osd.4.0:505 2014-08-11 18:44:26.451931 2014-08-13 18:06:06.696777 7f5c973a57a0 20 read_log 193305'15112 (193305'15111) modify cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.951337.0:30892 2014-08-11 18:44:26.444605 2014-08-13 18:06:06.696790 7f5c973a57a0 20 read_log 193305'15113 (0'0) promote 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by osd.4.0:507 2014-08-11 18:44:26.883037 2014-08-13 18:06:06.696814 7f5c973a57a0 20 read_log 193305'15114 (193305'15113) modify 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.951337.0:31017 2014-08-11 18:44:26.874272 2014-08-13 18:06:06.696830 7f5c973a57a0 20 read_log 193305'15115 (0'0) promote 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by osd.4.0:527 2014-08-11 18:44:30.089672 2014-08-13 18:06:06.696843 7f5c973a57a0 20 read_log 193305'15116 (193305'15115) modify 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.951337.0:31946 2014-08-11 18:44:30.081335 2014-08-13 18:06:06.696855 7f5c973a57a0 20 read_log 193305'15117 (0'0) promote 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by osd.4.0:537 2014-08-11 18:44:30.738098 2014-08-13 18:06:06.696869 7f5c973a57a0 20 read_log 193305'15118 (193305'15117) modify 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.951337.0:32137 2014-08-11 18:44:30.731458 2014-08-13 18:06:06.696882 7f5c973a57a0 20 read_log 193305'15119 (0'0) promote 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by osd.4.0:545 2014-08-11 18:44:31.643084 2014-08-13 18:06:06.696903 7f5c973a57a0 20 read_log 193305'15120 (193305'15119) modify 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.951337.0:32370 2014-08-11 18:44:31.630100 2014-08-13 18:06:06.696919 7f5c973a57a0 20 read_log 193305'15121 (0'0) promote ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by osd.4.0:547 2014-08-11 18:44:31.954494 2014-08-13 18:06:06.696932 7f5c973a57a0 20 read_log 193305'15122 (193305'15121) modify ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.951337.0:32461 2014-08-11 18:44:31.946280 2014-08-13 18:06:06.696954 7f5c973a57a0 20 read_log 193305'15123 (193305'15044) modify 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.951337.0:34071 2014-08-11 18:44:34.783420 2014-08-13 18:06:06.696970 7f5c973a57a0 20 read_log 193305'15124 (193305'15046) modify fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.951337.0:35272 2014-08-11 18:44:35.486049 2014-08-13 18:06:06.696983 7f5c973a57a0 20 read_log 193305'15125 (193305'15048) modify 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.951337.0:36473 2014-08-11 18:44:36.246961 2014-08-13 18:06:06.696995 7f5c973a57a0 20 read_log 193305'15126 (193305'15050) modify eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.951337.0:37134 2014-08-11 18:44:36.727062 2014-08-13 18:06:06.697007 7f5c973a57a0 20 read_log 193305'15127 (193305'15052) modify 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.951337.0:37992 2014-08-11 18:44:37.186043 2014-08-13 18:06:06.697020 7f5c973a57a0 20 read_log 193305'15128 (193305'15054) modify 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.951337.0:38175 2014-08-11 18:44:37.274138 2014-08-13 18:06:06.697032 7f5c973a57a0 20 read_log 193305'15129 (193305'15056) modify cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.951337.0:38210 2014-08-11 18:44:37.288348 2014-08-13 18:06:06.697044 7f5c973a57a0 20 read_log 193305'15130 (193305'15058) modify 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.951337.0:38730 2014-08-11 18:44:37.623799 2014-08-13 18:06:06.697056 7f5c973a57a0 20 read_log 193305'15131 (193305'15060) modify fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.951337.0:38873 2014-08-11 18:44:37.705993 2014-08-13 18:06:06.697068 7f5c973a57a0 20 read_log 193305'15132 (193305'15062) modify 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.951337.0:38889 2014-08-11 18:44:37.716624 2014-08-13 18:06:06.697080 7f5c973a57a0 20 read_log 193305'15133 (193305'15064) modify 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.951337.0:39179 2014-08-11 18:44:37.867600 2014-08-13 18:06:06.697093 7f5c973a57a0 20 read_log 193305'15134 (193305'15066) modify 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.951337.0:39866 2014-08-11 18:44:38.209860 2014-08-13 18:06:06.697107 7f5c973a57a0 20 read_log 193305'15135 (193305'15068) modify fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.951337.0:40281 2014-08-11 18:44:38.361379 2014-08-13 18:06:06.697119 7f5c973a57a0 20 read_log 193305'15136 (193305'15070) modify 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.951337.0:41889 2014-08-11 18:44:38.911536 2014-08-13 18:06:06.697132 7f5c973a57a0 20 read_log 193305'15137 (193305'15072) modify 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.951337.0:43655 2014-08-11 18:44:39.656646 2014-08-13 18:06:06.697192 7f5c973a57a0 20 read_log 193305'15138 (193305'15074) modify 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.951337.0:43941 2014-08-11 18:44:39.743464 2014-08-13 18:06:06.697212 7f5c973a57a0 20 read_log 193305'15139 (193305'15076) modify e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.951337.0:44183 2014-08-11 18:44:39.815446 2014-08-13 18:06:06.697225 7f5c973a57a0 20 read_log 193305'15140 (193305'15078) modify a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.951337.0:45114 2014-08-11 18:44:40.093470 2014-08-13 18:06:06.697243 7f5c973a57a0 20 read_log 193305'15141 (193305'15080) modify 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.951337.0:45338 2014-08-11 18:44:40.166973 2014-08-13 18:06:06.697256 7f5c973a57a0 20 read_log 193305'15142 (193305'15082) modify 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.951337.0:49064 2014-08-11 18:44:41.239633 2014-08-13 18:06:06.697275 7f5c973a57a0 20 read_log 193305'15143 (193305'15084) modify b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.951337.0:49088 2014-08-11 18:44:41.251711 2014-08-13 18:06:06.697294 7f5c973a57a0 20 read_log 193305'15144 (193305'15086) modify 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.951337.0:50427 2014-08-11 18:44:41.661417 2014-08-13 18:06:06.697307 7f5c973a57a0 20 read_log 193305'15145 (193305'15088) modify 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.951337.0:50599 2014-08-11 18:44:41.711520 2014-08-13 18:06:06.697319 7f5c973a57a0 20 read_log 193305'15146 (193305'15090) modify 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.951337.0:51293 2014-08-11 18:44:41.907724 2014-08-13 18:06:06.697335 7f5c973a57a0 20 read_log 193305'15147 (193305'15092) modify 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.951337.0:51404 2014-08-11 18:44:41.941828 2014-08-13 18:06:06.697359 7f5c973a57a0 20 read_log 193305'15148 (193305'15094) modify c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.951337.0:53392 2014-08-11 18:44:42.517377 2014-08-13 18:06:06.697375 7f5c973a57a0 20 read_log 193305'15149 (193305'15096) modify d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.951337.0:53541 2014-08-11 18:44:42.559971 2014-08-13 18:06:06.697390 7f5c973a57a0 20 read_log 193305'15150 (193305'15098) modify 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.951337.0:54664 2014-08-11 18:44:42.905459 2014-08-13 18:06:06.697412 7f5c973a57a0 20 read_log 193305'15151 (193305'15100) modify a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.951337.0:55594 2014-08-11 18:44:43.172363 2014-08-13 18:06:06.697425 7f5c973a57a0 20 read_log 193305'15152 (193305'15102) modify f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.951337.0:55629 2014-08-11 18:44:43.183070 2014-08-13 18:06:06.697438 7f5c973a57a0 20 read_log 193305'15153 (193305'15104) modify 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.951337.0:56019 2014-08-11 18:44:43.295230 2014-08-13 18:06:06.697450 7f5c973a57a0 20 read_log 193305'15154 (193305'15106) modify 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.951337.0:59227 2014-08-11 18:44:44.267428 2014-08-13 18:06:06.697470 7f5c973a57a0 20 read_log 193305'15155 (193305'15108) modify b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.951337.0:61114 2014-08-11 18:44:44.827932 2014-08-13 18:06:06.697486 7f5c973a57a0 20 read_log 193305'15156 (193305'15110) modify daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.951337.0:62386 2014-08-11 18:44:45.212195 2014-08-13 18:06:06.697499 7f5c973a57a0 20 read_log 193305'15157 (193305'15112) modify cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.951337.0:63603 2014-08-11 18:44:45.557607 2014-08-13 18:06:06.697511 7f5c973a57a0 20 read_log 193305'15158 (193305'15114) modify 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.951337.0:63728 2014-08-11 18:44:45.593564 2014-08-13 18:06:06.697523 7f5c973a57a0 20 read_log 193305'15159 (193305'15116) modify 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.951337.0:64657 2014-08-11 18:44:45.881088 2014-08-13 18:06:06.697543 7f5c973a57a0 20 read_log 193305'15160 (193305'15118) modify 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.951337.0:64848 2014-08-11 18:44:45.937664 2014-08-13 18:06:06.697565 7f5c973a57a0 20 read_log 193305'15161 (193305'15120) modify 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.951337.0:65081 2014-08-11 18:44:46.011027 2014-08-13 18:06:06.697580 7f5c973a57a0 20 read_log 193305'15162 (193305'15122) modify ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.951337.0:65172 2014-08-11 18:44:46.041880 2014-08-13 18:06:06.697601 7f5c973a57a0 20 read_log 194215'15163 (194215'15163) modify 136/hit_set_24.136_archive_2014-08-12 15:32:25.865410_2014-08-12 15:40:54.805843/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:40:54.806500 2014-08-13 18:06:06.697619 7f5c973a57a0 20 read_log 194215'15164 (193305'15041) delete 136/hit_set_24.136_archive_2014-08-11 18:37:04.911155_2014-08-11 18:42:38.276778/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:40:54.806500 2014-08-13 18:06:06.697631 7f5c973a57a0 20 read_log 194215'15165 (192888'15020) clean 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by osd.4.0:1578 2014-08-12 15:40:54.812453 2014-08-13 18:06:06.697656 7f5c973a57a0 20 read_log 194215'15166 (194215'15165) delete 46f20136/benchmark_data_storage0111-ib.pocstor_2228_object19926/head//24 by client.2174595.0:63211 0.000000 2014-08-13 18:06:06.697668 7f5c973a57a0 20 read_log 194215'15167 (193305'15158) clean 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by osd.4.0:1580 2014-08-12 15:40:54.860249 2014-08-13 18:06:06.697680 7f5c973a57a0 20 read_log 194215'15168 (194215'15167) delete 72e40136/benchmark_data_storage0111-ib.pocstor_62621_object31016/head//24 by client.2174595.0:63213 0.000000 2014-08-13 18:06:06.697691 7f5c973a57a0 20 read_log 194215'15169 (192888'14814) clean d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by osd.4.0:1582 2014-08-12 15:40:54.868405 2014-08-13 18:06:06.697704 7f5c973a57a0 20 read_log 194215'15170 (194215'15169) delete d5160136/benchmark_data_storage0111-ib.pocstor_41097_object5117/head//24 by client.2174595.0:63215 0.000000 2014-08-13 18:06:06.697715 7f5c973a57a0 20 read_log 194215'15171 (193305'15143) clean b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by osd.4.0:1584 2014-08-12 15:40:54.875727 2014-08-13 18:06:06.697744 7f5c973a57a0 20 read_log 194215'15172 (194215'15171) delete b0e42136/benchmark_data_storage0111-ib.pocstor_62621_object16376/head//24 by client.2174595.0:63217 0.000000 2014-08-13 18:06:06.697758 7f5c973a57a0 20 read_log 194215'15173 (193305'15156) clean daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by osd.4.0:1586 2014-08-12 15:40:54.915635 2014-08-13 18:06:06.697771 7f5c973a57a0 20 read_log 194215'15174 (194215'15173) delete daba2136/benchmark_data_storage0111-ib.pocstor_62621_object29674/head//24 by client.2174595.0:63219 0.000000 2014-08-13 18:06:06.697781 7f5c973a57a0 20 read_log 194215'15175 (192888'14818) clean 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by osd.4.0:1588 2014-08-12 15:40:54.924333 2014-08-13 18:06:06.697794 7f5c973a57a0 20 read_log 194215'15176 (194215'15175) delete 8e154136/benchmark_data_storage0111-ib.pocstor_41097_object6408/head//24 by client.2174595.0:63221 0.000000 2014-08-13 18:06:06.697805 7f5c973a57a0 20 read_log 194215'15177 (192888'14999) clean 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by osd.4.0:1590 2014-08-12 15:40:54.933248 2014-08-13 18:06:06.697817 7f5c973a57a0 20 read_log 194215'15178 (194215'15177) delete 15254136/benchmark_data_storage0111-ib.pocstor_2228_object9146/head//24 by client.2174595.0:63223 0.000000 2014-08-13 18:06:06.697828 7f5c973a57a0 20 read_log 194215'15179 (192888'14832) clean 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by osd.4.0:1592 2014-08-12 15:40:54.941943 2014-08-13 18:06:06.697840 7f5c973a57a0 20 read_log 194215'15180 (194215'15179) delete 74986136/benchmark_data_storage0111-ib.pocstor_41097_object14534/head//24 by client.2174595.0:63225 0.000000 2014-08-13 18:06:06.697851 7f5c973a57a0 20 read_log 194215'15181 (193305'15135) clean fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by osd.4.0:1594 2014-08-12 15:40:54.950533 2014-08-13 18:06:06.697927 7f5c973a57a0 20 read_log 194215'15182 (194215'15181) delete fae86136/benchmark_data_storage0111-ib.pocstor_62621_object7569/head//24 by client.2174595.0:63227 0.000000 2014-08-13 18:06:06.697943 7f5c973a57a0 20 read_log 194215'15183 (193305'15141) clean 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by osd.4.0:1596 2014-08-12 15:40:54.959264 2014-08-13 18:06:06.697956 7f5c973a57a0 20 read_log 194215'15184 (194215'15183) delete 4d2f6136/benchmark_data_storage0111-ib.pocstor_62621_object12626/head//24 by client.2174595.0:63229 0.000000 2014-08-13 18:06:06.697967 7f5c973a57a0 20 read_log 194215'15185 (193305'15138) clean 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by osd.4.0:1598 2014-08-12 15:40:54.967567 2014-08-13 18:06:06.697980 7f5c973a57a0 20 read_log 194215'15186 (194215'15185) delete 5c7c8136/benchmark_data_storage0111-ib.pocstor_62621_object11229/head//24 by client.2174595.0:63231 0.000000 2014-08-13 18:06:06.697991 7f5c973a57a0 20 read_log 194215'15187 (193305'15159) clean 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by osd.4.0:1600 2014-08-12 15:40:54.993535 2014-08-13 18:06:06.698003 7f5c973a57a0 20 read_log 194215'15188 (194215'15187) delete 1dfa9136/benchmark_data_storage0111-ib.pocstor_62621_object31945/head//24 by client.2174595.0:63233 0.000000 2014-08-13 18:06:06.698014 7f5c973a57a0 20 read_log 194215'15189 (193305'15154) clean 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by osd.4.0:1602 2014-08-12 15:40:55.032202 2014-08-13 18:06:06.698026 7f5c973a57a0 20 read_log 194215'15190 (194215'15189) delete 9e42a136/benchmark_data_storage0111-ib.pocstor_62621_object26515/head//24 by client.2174595.0:63235 0.000000 2014-08-13 18:06:06.698037 7f5c973a57a0 20 read_log 194215'15191 (192888'14843) clean 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by osd.4.0:1604 2014-08-12 15:40:55.041324 2014-08-13 18:06:06.698049 7f5c973a57a0 20 read_log 194215'15192 (194215'15191) delete 3c8aa136/benchmark_data_storage0111-ib.pocstor_41097_object19977/head//24 by client.2174595.0:63237 0.000000 2014-08-13 18:06:06.698060 7f5c973a57a0 20 read_log 194215'15193 (192888'15035) clean 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by osd.4.0:1606 2014-08-12 15:40:55.049360 2014-08-13 18:06:06.698072 7f5c973a57a0 20 read_log 194215'15194 (194215'15193) delete 5b94b136/benchmark_data_storage0111-ib.pocstor_2228_object27595/head//24 by client.2174595.0:63239 0.000000 2014-08-13 18:06:06.698083 7f5c973a57a0 20 read_log 194215'15195 (192888'15013) clean 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by osd.4.0:1608 2014-08-12 15:40:55.057066 2014-08-13 18:06:06.698095 7f5c973a57a0 20 read_log 194215'15196 (194215'15195) delete 8553d136/benchmark_data_storage0111-ib.pocstor_2228_object16997/head//24 by client.2174595.0:63241 0.000000 2014-08-13 18:06:06.698106 7f5c973a57a0 20 read_log 194215'15197 (193305'15144) clean 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by osd.4.0:1610 2014-08-12 15:40:55.065117 2014-08-13 18:06:06.698118 7f5c973a57a0 20 read_log 194215'15198 (194215'15197) delete 9563d136/benchmark_data_storage0111-ib.pocstor_62621_object17715/head//24 by client.2174595.0:63243 0.000000 2014-08-13 18:06:06.698129 7f5c973a57a0 20 read_log 194215'15199 (192888'15001) clean b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by osd.4.0:1612 2014-08-12 15:40:55.074588 2014-08-13 18:06:06.698141 7f5c973a57a0 20 read_log 194215'15200 (194215'15199) delete b92e136/benchmark_data_storage0111-ib.pocstor_2228_object10356/head//24 by client.2174595.0:63245 0.000000 2014-08-13 18:06:06.698162 7f5c973a57a0 20 read_log 194215'15201 (193305'15152) clean f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by osd.4.0:1614 2014-08-12 15:40:55.106909 2014-08-13 18:06:06.698185 7f5c973a57a0 20 read_log 194215'15202 (194215'15201) delete f3abe136/benchmark_data_storage0111-ib.pocstor_62621_object22917/head//24 by client.2174595.0:63247 0.000000 2014-08-13 18:06:06.698197 7f5c973a57a0 20 read_log 194215'15203 (192888'14860) clean 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by osd.4.0:1616 2014-08-12 15:40:55.114586 2014-08-13 18:06:06.698209 7f5c973a57a0 20 read_log 194215'15204 (194215'15203) delete 8d9ce136/benchmark_data_storage0111-ib.pocstor_41097_object30258/head//24 by client.2174595.0:63249 0.000000 2014-08-13 18:06:06.698220 7f5c973a57a0 20 read_log 194215'15205 (193305'15130) clean 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by osd.4.0:1618 2014-08-12 15:40:55.169581 2014-08-13 18:06:06.698232 7f5c973a57a0 20 read_log 194215'15206 (194215'15205) delete 47ee136/benchmark_data_storage0111-ib.pocstor_62621_object6018/head//24 by client.2174595.0:63251 0.000000 2014-08-13 18:06:06.698246 7f5c973a57a0 20 read_log 194215'15207 (193305'15160) clean 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by osd.4.0:1620 2014-08-12 15:40:55.178388 2014-08-13 18:06:06.698259 7f5c973a57a0 20 read_log 194215'15208 (194215'15207) delete 2e59f136/benchmark_data_storage0111-ib.pocstor_62621_object32136/head//24 by client.2174595.0:63253 0.000000 2014-08-13 18:06:06.698270 7f5c973a57a0 20 read_log 194215'15209 (193305'15124) clean fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by osd.4.0:1622 2014-08-12 15:40:55.226444 2014-08-13 18:06:06.698282 7f5c973a57a0 20 read_log 194215'15210 (194215'15209) delete fd3f0536/benchmark_data_storage0111-ib.pocstor_62621_object2560/head//24 by client.2174595.0:63255 0.000000 2014-08-13 18:06:06.698296 7f5c973a57a0 20 read_log 194215'15211 (192888'14831) clean 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by osd.4.0:1624 2014-08-12 15:40:55.235026 2014-08-13 18:06:06.698309 7f5c973a57a0 20 read_log 194215'15212 (194215'15211) delete 45561536/benchmark_data_storage0111-ib.pocstor_41097_object14100/head//24 by client.2174595.0:63257 0.000000 2014-08-13 18:06:06.698319 7f5c973a57a0 20 read_log 194215'15213 (192888'15016) clean 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by osd.4.0:1626 2014-08-12 15:40:55.241989 2014-08-13 18:06:06.698332 7f5c973a57a0 20 read_log 194215'15214 (194215'15213) delete 3e171536/benchmark_data_storage0111-ib.pocstor_2228_object17707/head//24 by client.2174595.0:63259 0.000000 2014-08-13 18:06:06.698347 7f5c973a57a0 20 read_log 194215'15215 (193305'15127) clean 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by osd.4.0:1628 2014-08-12 15:40:55.249841 2014-08-13 18:06:06.698360 7f5c973a57a0 20 read_log 194215'15216 (194215'15215) delete 95091536/benchmark_data_storage0111-ib.pocstor_62621_object5280/head//24 by client.2174595.0:63261 0.000000 2014-08-13 18:06:06.698387 7f5c973a57a0 20 read_log 194215'15217 (193305'15157) clean cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by osd.4.0:1630 2014-08-12 15:40:55.257513 2014-08-13 18:06:06.698403 7f5c973a57a0 20 read_log 194215'15218 (194215'15217) delete cf8a1536/benchmark_data_storage0111-ib.pocstor_62621_object30891/head//24 by client.2174595.0:63263 0.000000 2014-08-13 18:06:06.698424 7f5c973a57a0 20 read_log 194215'15219 (192888'14842) clean f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by osd.4.0:1632 2014-08-12 15:40:55.264721 2014-08-13 18:06:06.698441 7f5c973a57a0 20 read_log 194215'15220 (194215'15219) delete f2c12536/benchmark_data_storage0111-ib.pocstor_41097_object19540/head//24 by client.2174595.0:63265 0.000000 2014-08-13 18:06:06.698452 7f5c973a57a0 20 read_log 194215'15221 (193305'15145) clean 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by osd.4.0:1634 2014-08-12 15:40:55.272342 2014-08-13 18:06:06.698472 7f5c973a57a0 20 read_log 194215'15222 (194215'15221) delete 84722536/benchmark_data_storage0111-ib.pocstor_62621_object17887/head//24 by client.2174595.0:63267 0.000000 2014-08-13 18:06:06.698486 7f5c973a57a0 20 read_log 194215'15223 (192888'14815) clean 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by osd.4.0:1636 2014-08-12 15:40:55.279940 2014-08-13 18:06:06.698498 7f5c973a57a0 20 read_log 194215'15224 (194215'15223) delete 7ff82536/benchmark_data_storage0111-ib.pocstor_41097_object5149/head//24 by client.2174595.0:63269 0.000000 2014-08-13 18:06:06.698517 7f5c973a57a0 20 read_log 194215'15225 (192888'15017) clean f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by osd.4.0:1638 2014-08-12 15:40:55.288071 2014-08-13 18:06:06.698534 7f5c973a57a0 20 read_log 194215'15226 (194215'15225) delete f2503536/benchmark_data_storage0111-ib.pocstor_2228_object18548/head//24 by client.2174595.0:63271 0.000000 2014-08-13 18:06:06.698546 7f5c973a57a0 20 read_log 194215'15227 (193305'15132) clean 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by osd.4.0:1640 2014-08-12 15:40:55.296326 2014-08-13 18:06:06.698558 7f5c973a57a0 20 read_log 194215'15228 (194215'15227) delete 9e623536/benchmark_data_storage0111-ib.pocstor_62621_object6177/head//24 by client.2174595.0:63273 0.000000 2014-08-13 18:06:06.698575 7f5c973a57a0 20 read_log 194215'15229 (192888'15039) clean 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by osd.4.0:1642 2014-08-12 15:40:55.303906 2014-08-13 18:06:06.698590 7f5c973a57a0 20 read_log 194215'15230 (194215'15229) delete 89bb3536/benchmark_data_storage0111-ib.pocstor_2228_object30477/head//24 by client.2174595.0:63275 0.000000 2014-08-13 18:06:06.698601 7f5c973a57a0 20 read_log 194215'15231 (192888'14827) clean bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by osd.4.0:1644 2014-08-12 15:40:55.312714 2014-08-13 18:06:06.698614 7f5c973a57a0 20 read_log 194215'15232 (194215'15231) delete bb9a5536/benchmark_data_storage0111-ib.pocstor_41097_object10781/head//24 by client.2174595.0:63277 0.000000 2014-08-13 18:06:06.698633 7f5c973a57a0 20 read_log 194215'15233 (192888'14840) clean f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by osd.4.0:1646 2014-08-12 15:40:55.320697 2014-08-13 18:06:06.698646 7f5c973a57a0 20 read_log 194215'15234 (194215'15233) delete f0ec5536/benchmark_data_storage0111-ib.pocstor_41097_object19057/head//24 by client.2174595.0:63279 0.000000 2014-08-13 18:06:06.698662 7f5c973a57a0 20 read_log 194215'15235 (192888'14836) clean de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by osd.4.0:1648 2014-08-12 15:40:55.330822 2014-08-13 18:06:06.698677 7f5c973a57a0 20 read_log 194215'15236 (194215'15235) delete de0e5536/benchmark_data_storage0111-ib.pocstor_41097_object17083/head//24 by client.2174595.0:63281 0.000000 2014-08-13 18:06:06.698695 7f5c973a57a0 20 read_log 194215'15237 (193305'15162) clean ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by osd.4.0:1650 2014-08-12 15:40:55.340018 2014-08-13 18:06:06.698714 7f5c973a57a0 20 read_log 194215'15238 (194215'15237) delete ae757536/benchmark_data_storage0111-ib.pocstor_62621_object32460/head//24 by client.2174595.0:63283 0.000000 2014-08-13 18:06:06.698726 7f5c973a57a0 20 read_log 194215'15239 (193305'15123) clean 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by osd.4.0:1652 2014-08-12 15:40:55.348547 2014-08-13 18:06:06.698738 7f5c973a57a0 20 read_log 194215'15240 (194215'15239) delete 8c167536/benchmark_data_storage0111-ib.pocstor_62621_object1359/head//24 by client.2174595.0:63285 0.000000 2014-08-13 18:06:06.698749 7f5c973a57a0 20 read_log 194215'15241 (193305'15131) clean fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by osd.4.0:1654 2014-08-12 15:40:55.357787 2014-08-13 18:06:06.698776 7f5c973a57a0 20 read_log 194215'15242 (194215'15241) delete fb267536/benchmark_data_storage0111-ib.pocstor_62621_object6161/head//24 by client.2174595.0:63287 0.000000 2014-08-13 18:06:06.698790 7f5c973a57a0 20 read_log 194215'15243 (192888'14817) clean 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by osd.4.0:1656 2014-08-12 15:40:55.366050 2014-08-13 18:06:06.698815 7f5c973a57a0 20 read_log 194215'15244 (194215'15243) delete 65dc7536/benchmark_data_storage0111-ib.pocstor_41097_object5793/head//24 by client.2174595.0:63289 0.000000 2014-08-13 18:06:06.698829 7f5c973a57a0 20 read_log 194215'15245 (192888'15022) clean 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by osd.4.0:1658 2014-08-12 15:40:55.407521 2014-08-13 18:06:06.698842 7f5c973a57a0 20 read_log 194215'15246 (194215'15245) delete 97b68536/benchmark_data_storage0111-ib.pocstor_2228_object20697/head//24 by client.2174595.0:63291 0.000000 2014-08-13 18:06:06.698856 7f5c973a57a0 20 read_log 194215'15247 (192888'14995) clean b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by osd.4.0:1660 2014-08-12 15:40:55.421818 2014-08-13 18:06:06.698872 7f5c973a57a0 20 read_log 194215'15248 (194215'15247) delete b7659536/benchmark_data_storage0111-ib.pocstor_2228_object6627/head//24 by client.2174595.0:63293 0.000000 2014-08-13 18:06:06.698883 7f5c973a57a0 20 read_log 194215'15249 (192888'15014) clean 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by osd.4.0:1662 2014-08-12 15:40:55.429479 2014-08-13 18:06:06.698896 7f5c973a57a0 20 read_log 194215'15250 (194215'15249) delete 73089536/benchmark_data_storage0111-ib.pocstor_2228_object17067/head//24 by client.2174595.0:63295 0.000000 2014-08-13 18:06:06.698918 7f5c973a57a0 20 read_log 194215'15251 (193305'15126) clean eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by osd.4.0:1664 2014-08-12 15:40:55.436833 2014-08-13 18:06:06.698934 7f5c973a57a0 20 read_log 194215'15252 (194215'15251) delete eec89536/benchmark_data_storage0111-ib.pocstor_62621_object4422/head//24 by client.2174595.0:63297 0.000000 2014-08-13 18:06:06.698945 7f5c973a57a0 20 read_log 194215'15253 (192888'14805) clean d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by osd.4.0:1666 2014-08-12 15:40:55.444535 2014-08-13 18:06:06.698961 7f5c973a57a0 20 read_log 194215'15254 (194215'15253) delete d6f0c536/benchmark_data_storage0111-ib.pocstor_41097_object906/head//24 by client.2174595.0:63299 0.000000 2014-08-13 18:06:06.698972 7f5c973a57a0 20 read_log 194215'15255 (192888'15036) clean 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by osd.4.0:1668 2014-08-12 15:40:55.451769 2014-08-13 18:06:06.698984 7f5c973a57a0 20 read_log 194215'15256 (194215'15255) delete 838ad536/benchmark_data_storage0111-ib.pocstor_2228_object29145/head//24 by client.2174595.0:63301 0.000000 2014-08-13 18:06:06.698995 7f5c973a57a0 20 read_log 194215'15257 (193305'15134) clean 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by osd.4.0:1670 2014-08-12 15:40:55.459166 2014-08-13 18:06:06.699011 7f5c973a57a0 20 read_log 194215'15258 (194215'15257) delete 9dd5e536/benchmark_data_storage0111-ib.pocstor_62621_object7154/head//24 by client.2174595.0:63303 0.000000 2014-08-13 18:06:06.699022 7f5c973a57a0 20 read_log 194215'15259 (192888'15034) clean f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by osd.4.0:1672 2014-08-12 15:40:55.466283 2014-08-13 18:06:06.699034 7f5c973a57a0 20 read_log 194215'15260 (194215'15259) delete f664f536/benchmark_data_storage0111-ib.pocstor_2228_object27217/head//24 by client.2174595.0:63305 0.000000 2014-08-13 18:06:06.699057 7f5c973a57a0 20 read_log 194215'15261 (192888'14982) clean 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by osd.4.0:1674 2014-08-12 15:40:55.473935 2014-08-13 18:06:06.699086 7f5c973a57a0 20 read_log 194215'15262 (194215'15261) delete 6ea4f536/benchmark_data_storage0111-ib.pocstor_2228_object200/head//24 by client.2174595.0:63307 0.000000 2014-08-13 18:06:06.699385 7f5c973a57a0 20 read_log 194215'15263 (193305'15142) clean 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by osd.4.0:1676 2014-08-12 15:40:55.482340 2014-08-13 18:06:06.699399 7f5c973a57a0 20 read_log 194215'15264 (194215'15263) delete 2c1e1936/benchmark_data_storage0111-ib.pocstor_62621_object16352/head//24 by client.2174595.0:63309 0.000000 2014-08-13 18:06:06.699412 7f5c973a57a0 20 read_log 194215'15265 (192888'15028) clean fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by osd.4.0:1678 2014-08-12 15:40:55.497953 2014-08-13 18:06:06.699429 7f5c973a57a0 20 read_log 194215'15266 (194215'15265) delete fffe1936/benchmark_data_storage0111-ib.pocstor_2228_object22764/head//24 by client.2174595.0:63311 0.000000 2014-08-13 18:06:06.699440 7f5c973a57a0 20 read_log 194215'15267 (192888'15031) clean 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by osd.4.0:1680 2014-08-12 15:40:55.506684 2014-08-13 18:06:06.699453 7f5c973a57a0 20 read_log 194215'15268 (194215'15267) delete 65162936/benchmark_data_storage0111-ib.pocstor_2228_object23960/head//24 by client.2174595.0:63313 0.000000 2014-08-13 18:06:06.699464 7f5c973a57a0 20 read_log 194215'15269 (193305'15149) clean d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by osd.4.0:1682 2014-08-12 15:40:55.523238 2014-08-13 18:06:06.699476 7f5c973a57a0 20 read_log 194215'15270 (194215'15269) delete d6ce4936/benchmark_data_storage0111-ib.pocstor_62621_object20829/head//24 by client.2174595.0:63315 0.000000 2014-08-13 18:06:06.699487 7f5c973a57a0 20 read_log 194215'15271 (193305'15133) clean 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by osd.4.0:1684 2014-08-12 15:40:55.561811 2014-08-13 18:06:06.699499 7f5c973a57a0 20 read_log 194215'15272 (194215'15271) delete 2945936/benchmark_data_storage0111-ib.pocstor_62621_object6467/head//24 by client.2174595.0:63317 0.000000 2014-08-13 18:06:06.699510 7f5c973a57a0 20 read_log 194215'15273 (192888'14988) clean d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by osd.4.0:1686 2014-08-12 15:40:55.571371 2014-08-13 18:06:06.699522 7f5c973a57a0 20 read_log 194215'15274 (194215'15273) delete d99b5936/benchmark_data_storage0111-ib.pocstor_2228_object4150/head//24 by client.2174595.0:63319 0.000000 2014-08-13 18:06:06.699540 7f5c973a57a0 20 read_log 194215'15275 (192888'14822) clean fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by osd.4.0:1688 2014-08-12 15:40:55.601344 2014-08-13 18:06:06.699558 7f5c973a57a0 20 read_log 194215'15276 (194215'15275) delete fc846936/benchmark_data_storage0111-ib.pocstor_41097_object8541/head//24 by client.2174595.0:63321 0.000000 2014-08-13 18:06:06.699573 7f5c973a57a0 20 read_log 194215'15277 (192888'14987) clean cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by osd.4.0:1690 2014-08-12 15:40:55.610900 2014-08-13 18:06:06.699586 7f5c973a57a0 20 read_log 194215'15278 (194215'15277) delete cbd86936/benchmark_data_storage0111-ib.pocstor_2228_object3923/head//24 by client.2174595.0:63323 0.000000 2014-08-13 18:06:06.699599 7f5c973a57a0 20 read_log 194215'15279 (193305'15148) clean c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by osd.4.0:1692 2014-08-12 15:40:55.634120 2014-08-13 18:06:06.699612 7f5c973a57a0 20 read_log 194215'15280 (194215'15279) delete c5aa6936/benchmark_data_storage0111-ib.pocstor_62621_object20680/head//24 by client.2174595.0:63325 0.000000 2014-08-13 18:06:06.699623 7f5c973a57a0 20 read_log 194215'15281 (192888'14991) clean 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by osd.4.0:1694 2014-08-12 15:40:55.642346 2014-08-13 18:06:06.699643 7f5c973a57a0 20 read_log 194215'15282 (194215'15281) delete 4bfc8936/benchmark_data_storage0111-ib.pocstor_2228_object5401/head//24 by client.2174595.0:63327 0.000000 2014-08-13 18:06:06.699657 7f5c973a57a0 20 read_log 194215'15283 (192888'15015) clean 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by osd.4.0:1696 2014-08-12 15:40:55.651416 2014-08-13 18:06:06.699670 7f5c973a57a0 20 read_log 194215'15284 (194215'15283) delete 92fd8936/benchmark_data_storage0111-ib.pocstor_2228_object17589/head//24 by client.2174595.0:63329 0.000000 2014-08-13 18:06:06.699681 7f5c973a57a0 20 read_log 194215'15285 (193305'15153) clean 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by osd.4.0:1698 2014-08-12 15:40:55.661040 2014-08-13 18:06:06.699696 7f5c973a57a0 20 read_log 194215'15286 (194215'15285) delete 8c919936/benchmark_data_storage0111-ib.pocstor_62621_object23307/head//24 by client.2174595.0:63331 0.000000 2014-08-13 18:06:06.699707 7f5c973a57a0 20 read_log 194215'15287 (193305'15139) clean e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by osd.4.0:1700 2014-08-12 15:40:55.669062 2014-08-13 18:06:06.699720 7f5c973a57a0 20 read_log 194215'15288 (194215'15287) delete e6229936/benchmark_data_storage0111-ib.pocstor_62621_object11471/head//24 by client.2174595.0:63333 0.000000 2014-08-13 18:06:06.699731 7f5c973a57a0 20 read_log 194215'15289 (192888'15008) clean 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by osd.4.0:1702 2014-08-12 15:40:55.677340 2014-08-13 18:06:06.699743 7f5c973a57a0 20 read_log 194215'15290 (194215'15289) delete 9e9d9936/benchmark_data_storage0111-ib.pocstor_2228_object15349/head//24 by client.2174595.0:63335 0.000000 2014-08-13 18:06:06.699754 7f5c973a57a0 20 read_log 194215'15291 (192888'14821) clean f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by osd.4.0:1704 2014-08-12 15:40:55.686000 2014-08-13 18:06:06.699766 7f5c973a57a0 20 read_log 194215'15292 (194215'15291) delete f644a936/benchmark_data_storage0111-ib.pocstor_41097_object7709/head//24 by client.2174595.0:63337 0.000000 2014-08-13 18:06:06.699777 7f5c973a57a0 20 read_log 194215'15293 (192888'15006) clean 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by osd.4.0:1706 2014-08-12 15:40:55.695103 2014-08-13 18:06:06.699789 7f5c973a57a0 20 read_log 194215'15294 (194215'15293) delete 1057a936/benchmark_data_storage0111-ib.pocstor_2228_object12720/head//24 by client.2174595.0:63339 0.000000 2014-08-13 18:06:06.699800 7f5c973a57a0 20 read_log 194215'15295 (193305'15125) clean 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by osd.4.0:1708 2014-08-12 15:40:55.704519 2014-08-13 18:06:06.699812 7f5c973a57a0 20 read_log 194215'15296 (194215'15295) delete 6348b936/benchmark_data_storage0111-ib.pocstor_62621_object3761/head//24 by client.2174595.0:63341 0.000000 2014-08-13 18:06:06.699823 7f5c973a57a0 20 read_log 194215'15297 (193305'15136) clean 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by osd.4.0:1710 2014-08-12 15:40:55.713219 2014-08-13 18:06:06.699835 7f5c973a57a0 20 read_log 194215'15298 (194215'15297) delete 85acb936/benchmark_data_storage0111-ib.pocstor_62621_object9177/head//24 by client.2174595.0:63343 0.000000 2014-08-13 18:06:06.699846 7f5c973a57a0 20 read_log 194215'15299 (193305'15146) clean 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by osd.4.0:1712 2014-08-12 15:40:55.721495 2014-08-13 18:06:06.699858 7f5c973a57a0 20 read_log 194215'15300 (194215'15299) delete 5299c936/benchmark_data_storage0111-ib.pocstor_62621_object18581/head//24 by client.2174595.0:63345 0.000000 2014-08-13 18:06:06.699869 7f5c973a57a0 20 read_log 194215'15301 (192888'14855) clean 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by osd.4.0:1714 2014-08-12 15:40:55.729061 2014-08-13 18:06:06.699890 7f5c973a57a0 20 read_log 194215'15302 (194215'15301) delete 57eac936/benchmark_data_storage0111-ib.pocstor_41097_object27189/head//24 by client.2174595.0:63347 0.000000 2014-08-13 18:06:06.699904 7f5c973a57a0 20 read_log 194215'15303 (192888'14845) clean 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by osd.4.0:1716 2014-08-12 15:40:55.735852 2014-08-13 18:06:06.699917 7f5c973a57a0 20 read_log 194215'15304 (194215'15303) delete 611d936/benchmark_data_storage0111-ib.pocstor_41097_object20751/head//24 by client.2174595.0:63349 0.000000 2014-08-13 18:06:06.699928 7f5c973a57a0 20 read_log 194215'15305 (192888'15004) clean b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by osd.4.0:1718 2014-08-12 15:40:55.744503 2014-08-13 18:06:06.699942 7f5c973a57a0 20 read_log 194215'15306 (194215'15305) delete b8ebd936/benchmark_data_storage0111-ib.pocstor_2228_object10579/head//24 by client.2174595.0:63351 0.000000 2014-08-13 18:06:06.699953 7f5c973a57a0 20 read_log 194215'15307 (193305'15129) clean cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by osd.4.0:1720 2014-08-12 15:40:55.752179 2014-08-13 18:06:06.699965 7f5c973a57a0 20 read_log 194215'15308 (194215'15307) delete cb3e936/benchmark_data_storage0111-ib.pocstor_62621_object5498/head//24 by client.2174595.0:63353 0.000000 2014-08-13 18:06:06.699976 7f5c973a57a0 20 read_log 194215'15309 (192888'15011) clean d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by osd.4.0:1722 2014-08-12 15:40:55.760216 2014-08-13 18:06:06.699990 7f5c973a57a0 20 read_log 194215'15310 (194215'15309) delete d21be936/benchmark_data_storage0111-ib.pocstor_2228_object16397/head//24 by client.2174595.0:63355 0.000000 2014-08-13 18:06:06.700001 7f5c973a57a0 20 read_log 194215'15311 (193305'15161) clean 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by osd.4.0:1724 2014-08-12 15:40:55.767523 2014-08-13 18:06:06.700014 7f5c973a57a0 20 read_log 194215'15312 (194215'15311) delete 8b0cf936/benchmark_data_storage0111-ib.pocstor_62621_object32369/head//24 by client.2174595.0:63357 0.000000 2014-08-13 18:06:06.700026 7f5c973a57a0 20 read_log 194215'15313 (192888'14809) clean 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by osd.4.0:1726 2014-08-12 15:40:55.792504 2014-08-13 18:06:06.700041 7f5c973a57a0 20 read_log 194215'15314 (194215'15313) delete 5f4ef936/benchmark_data_storage0111-ib.pocstor_41097_object3985/head//24 by client.2174595.0:63359 0.000000 2014-08-13 18:06:06.700051 7f5c973a57a0 20 read_log 194215'15315 (192888'15025) clean e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by osd.4.0:1728 2014-08-12 15:40:55.801252 2014-08-13 18:06:06.700064 7f5c973a57a0 20 read_log 194215'15316 (194215'15315) delete e4f30d36/benchmark_data_storage0111-ib.pocstor_2228_object21914/head//24 by client.2174595.0:63361 0.000000 2014-08-13 18:06:06.700074 7f5c973a57a0 20 read_log 194215'15317 (193305'15147) clean 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by osd.4.0:1730 2014-08-12 15:40:55.809369 2014-08-13 18:06:06.700087 7f5c973a57a0 20 read_log 194215'15318 (194215'15317) delete 4ed61d36/benchmark_data_storage0111-ib.pocstor_62621_object18692/head//24 by client.2174595.0:63363 0.000000 2014-08-13 18:06:06.700097 7f5c973a57a0 20 read_log 194215'15319 (192888'15012) clean 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by osd.4.0:1732 2014-08-12 15:40:55.818123 2014-08-13 18:06:06.700110 7f5c973a57a0 20 read_log 194215'15320 (194215'15319) delete 2ab72d36/benchmark_data_storage0111-ib.pocstor_2228_object16660/head//24 by client.2174595.0:63365 0.000000 2014-08-13 18:06:06.700123 7f5c973a57a0 20 read_log 194215'15321 (192888'15026) clean f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by osd.4.0:1734 2014-08-12 15:40:55.826204 2014-08-13 18:06:06.700166 7f5c973a57a0 20 read_log 194215'15322 (194215'15321) delete f7d82d36/benchmark_data_storage0111-ib.pocstor_2228_object22515/head//24 by client.2174595.0:63367 0.000000 2014-08-13 18:06:06.700181 7f5c973a57a0 20 read_log 194215'15323 (192888'14985) clean ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by osd.4.0:1736 2014-08-12 15:40:55.834074 2014-08-13 18:06:06.700193 7f5c973a57a0 20 read_log 194215'15324 (194215'15323) delete ad4b2d36/benchmark_data_storage0111-ib.pocstor_2228_object2710/head//24 by client.2174595.0:63369 0.000000 2014-08-13 18:06:06.700204 7f5c973a57a0 20 read_log 194215'15325 (192888'14852) clean 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by osd.4.0:1738 2014-08-12 15:40:55.842274 2014-08-13 18:06:06.700218 7f5c973a57a0 20 read_log 194215'15326 (194215'15325) delete 59843d36/benchmark_data_storage0111-ib.pocstor_41097_object26302/head//24 by client.2174595.0:63371 0.000000 2014-08-13 18:06:06.700229 7f5c973a57a0 20 read_log 194215'15327 (192888'15018) clean cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by osd.4.0:1740 2014-08-12 15:40:55.849672 2014-08-13 18:06:06.700247 7f5c973a57a0 20 read_log 194215'15328 (194215'15327) delete cc304d36/benchmark_data_storage0111-ib.pocstor_2228_object19287/head//24 by client.2174595.0:63373 0.000000 2014-08-13 18:06:06.700258 7f5c973a57a0 20 read_log 194215'15329 (192888'14846) clean cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by osd.4.0:1742 2014-08-12 15:40:55.859192 2014-08-13 18:06:06.700271 7f5c973a57a0 20 read_log 194215'15330 (194215'15329) delete cba74d36/benchmark_data_storage0111-ib.pocstor_41097_object21138/head//24 by client.2174595.0:63375 0.000000 2014-08-13 18:06:06.700282 7f5c973a57a0 20 read_log 194215'15331 (193305'15137) clean 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by osd.4.0:1744 2014-08-12 15:40:55.868271 2014-08-13 18:06:06.700299 7f5c973a57a0 20 read_log 194215'15332 (194215'15331) delete 77715d36/benchmark_data_storage0111-ib.pocstor_62621_object10943/head//24 by client.2174595.0:63377 0.000000 2014-08-13 18:06:06.700311 7f5c973a57a0 20 read_log 194215'15333 (192888'14837) clean 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by osd.4.0:1746 2014-08-12 15:40:55.876768 2014-08-13 18:06:06.700323 7f5c973a57a0 20 read_log 194215'15334 (194215'15333) delete 8dcf5d36/benchmark_data_storage0111-ib.pocstor_41097_object18087/head//24 by client.2174595.0:63379 0.000000 2014-08-13 18:06:06.700336 7f5c973a57a0 20 read_log 194215'15335 (192888'14848) clean 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by osd.4.0:1748 2014-08-12 15:40:55.885182 2014-08-13 18:06:06.700349 7f5c973a57a0 20 read_log 194215'15336 (194215'15335) delete 14fa6d36/benchmark_data_storage0111-ib.pocstor_41097_object23625/head//24 by client.2174595.0:63381 0.000000 2014-08-13 18:06:06.700373 7f5c973a57a0 20 read_log 194215'15337 (193305'15151) clean a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by osd.4.0:1750 2014-08-12 15:40:55.894693 2014-08-13 18:06:06.700393 7f5c973a57a0 20 read_log 194215'15338 (194215'15337) delete a2ff7d36/benchmark_data_storage0111-ib.pocstor_62621_object22882/head//24 by client.2174595.0:63383 0.000000 2014-08-13 18:06:06.700404 7f5c973a57a0 20 read_log 194215'15339 (192888'15000) clean 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by osd.4.0:1752 2014-08-12 15:40:55.903896 2014-08-13 18:06:06.700417 7f5c973a57a0 20 read_log 194215'15340 (194215'15339) delete 37408d36/benchmark_data_storage0111-ib.pocstor_2228_object9329/head//24 by client.2174595.0:63385 0.000000 2014-08-13 18:06:06.700438 7f5c973a57a0 20 read_log 194215'15341 (192888'14847) clean d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by osd.4.0:1754 2014-08-12 15:40:55.911668 2014-08-13 18:06:06.700459 7f5c973a57a0 20 read_log 194215'15342 (194215'15341) delete d97a8d36/benchmark_data_storage0111-ib.pocstor_41097_object21895/head//24 by client.2174595.0:63387 0.000000 2014-08-13 18:06:06.700483 7f5c973a57a0 20 read_log 194215'15343 (193305'15140) clean a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by osd.4.0:1756 2014-08-12 15:40:55.918997 2014-08-13 18:06:06.700498 7f5c973a57a0 20 read_log 194215'15344 (194215'15343) delete a2cc8d36/benchmark_data_storage0111-ib.pocstor_62621_object12402/head//24 by client.2174595.0:63389 0.000000 2014-08-13 18:06:06.700510 7f5c973a57a0 20 read_log 194215'15345 (193305'15128) clean 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by osd.4.0:1758 2014-08-12 15:40:55.926143 2014-08-13 18:06:06.700529 7f5c973a57a0 20 read_log 194215'15346 (194215'15345) delete 8b739d36/benchmark_data_storage0111-ib.pocstor_62621_object5463/head//24 by client.2174595.0:63391 0.000000 2014-08-13 18:06:06.700543 7f5c973a57a0 20 read_log 194215'15347 (192888'14835) clean 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by osd.4.0:1760 2014-08-12 15:40:55.933898 2014-08-13 18:06:06.700556 7f5c973a57a0 20 read_log 194215'15348 (194215'15347) delete 6c61ad36/benchmark_data_storage0111-ib.pocstor_41097_object15794/head//24 by client.2174595.0:63393 0.000000 2014-08-13 18:06:06.700572 7f5c973a57a0 20 read_log 194215'15349 (192888'14989) clean 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by osd.4.0:1762 2014-08-12 15:40:55.942124 2014-08-13 18:06:06.700586 7f5c973a57a0 20 read_log 194215'15350 (194215'15349) delete 9d42ad36/benchmark_data_storage0111-ib.pocstor_2228_object4354/head//24 by client.2174595.0:63395 0.000000 2014-08-13 18:06:06.700597 7f5c973a57a0 20 read_log 194215'15351 (192888'14858) clean de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by osd.4.0:1764 2014-08-12 15:40:55.950572 2014-08-13 18:06:06.700610 7f5c973a57a0 20 read_log 194215'15352 (194215'15351) delete de32bd36/benchmark_data_storage0111-ib.pocstor_41097_object28353/head//24 by client.2174595.0:63397 0.000000 2014-08-13 18:06:06.700621 7f5c973a57a0 20 read_log 194215'15353 (192888'14854) clean 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by osd.4.0:1766 2014-08-12 15:40:55.959839 2014-08-13 18:06:06.700633 7f5c973a57a0 20 read_log 194215'15354 (194215'15353) delete 3871cd36/benchmark_data_storage0111-ib.pocstor_41097_object26855/head//24 by client.2174595.0:63399 0.000000 2014-08-13 18:06:06.700644 7f5c973a57a0 20 read_log 194215'15355 (193305'15155) clean b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by osd.4.0:1768 2014-08-12 15:40:55.968808 2014-08-13 18:06:06.700656 7f5c973a57a0 20 read_log 194215'15356 (194215'15355) delete b58cdd36/benchmark_data_storage0111-ib.pocstor_62621_object28402/head//24 by client.2174595.0:63401 0.000000 2014-08-13 18:06:06.700667 7f5c973a57a0 20 read_log 194215'15357 (192888'14850) clean ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by osd.4.0:1770 2014-08-12 15:40:55.977509 2014-08-13 18:06:06.700679 7f5c973a57a0 20 read_log 194215'15358 (194215'15357) delete ea52ed36/benchmark_data_storage0111-ib.pocstor_41097_object25514/head//24 by client.2174595.0:63403 0.000000 2014-08-13 18:06:06.700690 7f5c973a57a0 20 read_log 194215'15359 (192888'14830) clean b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by osd.4.0:1772 2014-08-12 15:40:55.986061 2014-08-13 18:06:06.700702 7f5c973a57a0 20 read_log 194215'15360 (194215'15359) delete b0eeed36/benchmark_data_storage0111-ib.pocstor_41097_object13021/head//24 by client.2174595.0:63405 0.000000 2014-08-13 18:06:06.700713 7f5c973a57a0 20 read_log 194215'15361 (193305'15150) clean 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by osd.4.0:1774 2014-08-12 15:40:55.994146 2014-08-13 18:06:06.700734 7f5c973a57a0 20 read_log 194215'15362 (194215'15361) delete 1021fd36/benchmark_data_storage0111-ib.pocstor_62621_object21952/head//24 by client.2174595.0:63407 0.000000 2014-08-13 18:06:06.700746 7f5c973a57a0 20 read_log 194215'15363 (194215'15363) modify 136/hit_set_24.136_archive_2014-08-12 15:40:54.806616_2014-08-12 17:24:43.007231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.007988 2014-08-13 18:06:06.700759 7f5c973a57a0 20 read_log 194215'15364 (194215'15163) delete 136/hit_set_24.136_archive_2014-08-12 15:32:25.865410_2014-08-12 15:40:54.805843/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:43.007988 2014-08-13 18:06:06.700771 7f5c973a57a0 20 read_log 194215'15365 (0'0) promote ed8e8936/benchmark_data_storage0111-ib.pocstor_34628_object796/head//24 by osd.4.0:5241 2014-08-12 17:24:43.013546 2014-08-13 18:06:06.700784 7f5c973a57a0 20 read_log 194215'15366 (194215'15365) modify ed8e8936/benchmark_data_storage0111-ib.pocstor_34628_object796/head//24 by client.2193759.0:797 2014-08-12 17:24:43.002165 2014-08-13 18:06:06.700796 7f5c973a57a0 20 read_log 194370'15367 (0'0) promote d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by osd.50.0:4028 2014-08-12 20:30:14.686260 2014-08-13 18:06:06.700809 7f5c973a57a0 20 read_log 194370'15368 (194370'15367) modify d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by client.1508531.0:142 2014-08-12 20:30:14.678612 2014-08-13 18:06:06.700830 7f5c973a57a0 20 read_log 194370'15369 (0'0) promote e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by osd.50.0:4056 2014-08-12 20:30:18.139744 2014-08-13 18:06:06.700846 7f5c973a57a0 20 read_log 194370'15370 (194370'15369) modify e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by client.1508531.0:1005 2014-08-12 20:30:18.134242 2014-08-13 18:06:06.700859 7f5c973a57a0 20 read_log 194370'15371 (0'0) promote f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by osd.50.0:4078 2014-08-12 20:30:23.559797 2014-08-13 18:06:06.700915 7f5c973a57a0 20 read_log 194370'15372 (194370'15371) modify f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by client.1508531.0:2453 2014-08-12 20:30:23.553305 2014-08-13 18:06:06.700934 7f5c973a57a0 20 read_log 194370'15373 (0'0) promote 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by osd.50.0:4080 2014-08-12 20:30:23.658210 2014-08-13 18:06:06.700947 7f5c973a57a0 20 read_log 194370'15374 (194370'15373) modify 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by client.1508531.0:2480 2014-08-12 20:30:23.652101 2014-08-13 18:06:06.700959 7f5c973a57a0 20 read_log 194370'15375 (0'0) promote 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by osd.50.0:4136 2014-08-12 20:30:35.020903 2014-08-13 18:06:06.700971 7f5c973a57a0 20 read_log 194370'15376 (194370'15375) modify 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by client.1508531.0:5413 2014-08-12 20:30:35.010202 2014-08-13 18:06:06.700984 7f5c973a57a0 20 read_log 194370'15377 (0'0) promote 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by osd.50.0:4138 2014-08-12 20:30:35.088069 2014-08-13 18:06:06.700996 7f5c973a57a0 20 read_log 194370'15378 (194370'15377) modify 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by client.1508531.0:5429 2014-08-12 20:30:35.078738 2014-08-13 18:06:06.701008 7f5c973a57a0 20 read_log 194370'15379 (0'0) promote 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by osd.50.0:4140 2014-08-12 20:30:35.293102 2014-08-13 18:06:06.701025 7f5c973a57a0 20 read_log 194370'15380 (194370'15379) modify 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by client.1508531.0:5473 2014-08-12 20:30:35.284477 2014-08-13 18:06:06.701061 7f5c973a57a0 20 read_log 194370'15381 (0'0) promote ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by osd.50.0:4164 2014-08-12 20:30:41.421064 2014-08-13 18:06:06.701078 7f5c973a57a0 20 read_log 194370'15382 (194370'15381) modify ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by client.1508531.0:6812 2014-08-12 20:30:41.412373 2014-08-13 18:06:06.701099 7f5c973a57a0 20 read_log 194370'15383 (0'0) promote 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by osd.50.0:4192 2014-08-12 20:30:50.128567 2014-08-13 18:06:06.701114 7f5c973a57a0 20 read_log 194370'15384 (194370'15383) modify 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by client.1508531.0:8483 2014-08-12 20:30:50.118720 2014-08-13 18:06:06.701127 7f5c973a57a0 20 read_log 194370'15385 (0'0) promote 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by osd.50.0:4214 2014-08-12 20:30:57.232882 2014-08-13 18:06:06.701139 7f5c973a57a0 20 read_log 194370'15386 (194370'15385) modify 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by client.1508531.0:9654 2014-08-12 20:30:57.223661 2014-08-13 18:06:06.701158 7f5c973a57a0 20 read_log 194370'15387 (0'0) promote 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by osd.50.0:4232 2014-08-12 20:31:01.288857 2014-08-13 18:06:06.701173 7f5c973a57a0 20 read_log 194370'15388 (194370'15387) modify 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by client.1508531.0:10344 2014-08-12 20:31:01.281710 2014-08-13 18:06:06.701186 7f5c973a57a0 20 read_log 194370'15389 (0'0) promote cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by osd.50.0:4238 2014-08-12 20:31:02.622586 2014-08-13 18:06:06.701198 7f5c973a57a0 20 read_log 194370'15390 (194370'15389) modify cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by client.1508531.0:10564 2014-08-12 20:31:02.614352 2014-08-13 18:06:06.701210 7f5c973a57a0 20 read_log 194370'15391 (0'0) promote c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by osd.50.0:4262 2014-08-12 20:31:09.245325 2014-08-13 18:06:06.701222 7f5c973a57a0 20 read_log 194370'15392 (194370'15391) modify c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by client.1508531.0:11654 2014-08-12 20:31:09.235996 2014-08-13 18:06:06.701237 7f5c973a57a0 20 read_log 194377'15393 (0'0) promote e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by osd.50.0:4333 2014-08-12 20:33:00.397153 2014-08-13 18:06:06.701250 7f5c973a57a0 20 read_log 194377'15394 (194377'15393) modify e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by client.2231190.0:757 2014-08-12 20:33:00.390011 2014-08-13 18:06:06.701262 7f5c973a57a0 20 read_log 194377'15395 (0'0) promote 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by osd.50.0:4336 2014-08-12 20:33:00.548732 2014-08-13 18:06:06.701301 7f5c973a57a0 20 read_log 194377'15396 (194377'15395) modify 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by client.2231190.0:780 2014-08-12 20:33:00.477364 2014-08-13 18:06:06.701314 7f5c973a57a0 20 read_log 194377'15397 (0'0) promote 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by osd.50.0:4370 2014-08-12 20:33:06.822164 2014-08-13 18:06:06.701327 7f5c973a57a0 20 read_log 194377'15398 (194377'15397) modify 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by client.2231190.0:2397 2014-08-12 20:33:06.812923 2014-08-13 18:06:06.701366 7f5c973a57a0 20 read_log 194377'15399 (0'0) promote 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by osd.50.0:4414 2014-08-12 20:33:15.143469 2014-08-13 18:06:06.701391 7f5c973a57a0 20 read_log 194377'15400 (194377'15399) modify 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by client.2231190.0:4206 2014-08-12 20:33:15.133839 2014-08-13 18:06:06.701416 7f5c973a57a0 20 read_log 194377'15401 (0'0) promote 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by osd.50.0:4430 2014-08-12 20:33:18.845231 2014-08-13 18:06:06.701432 7f5c973a57a0 20 read_log 194377'15402 (194377'15401) modify 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by client.2231190.0:4729 2014-08-12 20:33:18.837895 2014-08-13 18:06:06.701444 7f5c973a57a0 20 read_log 194378'15403 (0'0) promote d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by osd.50.0:4434 2014-08-12 20:33:20.387373 2014-08-13 18:06:06.701460 7f5c973a57a0 20 read_log 194378'15404 (194378'15403) modify d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by client.2231190.0:4860 2014-08-12 20:33:20.379484 2014-08-13 18:06:06.701474 7f5c973a57a0 20 read_log 194380'15405 (0'0) promote d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by osd.50.0:4438 2014-08-12 20:33:23.997984 2014-08-13 18:06:06.701487 7f5c973a57a0 20 read_log 194380'15406 (194380'15405) modify d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by client.2231190.0:5071 2014-08-12 20:33:23.989565 2014-08-13 18:06:06.701499 7f5c973a57a0 20 read_log 194380'15407 (0'0) promote 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by osd.50.0:4440 2014-08-12 20:33:24.053255 2014-08-13 18:06:06.701511 7f5c973a57a0 20 read_log 194380'15408 (194380'15407) modify 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by client.2231190.0:5075 2014-08-12 20:33:24.042859 2014-08-13 18:06:06.701523 7f5c973a57a0 20 read_log 194380'15409 (0'0) promote 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by osd.50.0:4442 2014-08-12 20:33:24.585352 2014-08-13 18:06:06.701535 7f5c973a57a0 20 read_log 194380'15410 (194380'15409) modify 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by client.2231190.0:5113 2014-08-12 20:33:24.579796 2014-08-13 18:06:06.701547 7f5c973a57a0 20 read_log 194393'15411 (194393'15411) modify 136/hit_set_24.136_archive_2014-08-12 20:37:08.028004_2014-08-12 20:49:27.348465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.349262 2014-08-13 18:06:06.701560 7f5c973a57a0 20 read_log 194393'15412 (194215'15363) delete 136/hit_set_24.136_archive_2014-08-12 15:40:54.806616_2014-08-12 17:24:43.007231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.349262 2014-08-13 18:06:06.701573 7f5c973a57a0 20 read_log 194393'15413 (0'0) promote 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by osd.50.0:4544 2014-08-12 20:49:27.354418 2014-08-13 18:06:06.701585 7f5c973a57a0 20 read_log 194393'15414 (194393'15413) modify 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by client.1064182.0:397 2014-08-12 20:49:27.342558 2014-08-13 18:06:06.701598 7f5c973a57a0 20 read_log 194416'15415 (194416'15415) modify 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 18:06:06.701610 7f5c973a57a0 20 read_log 194416'15416 (194393'15411) delete 136/hit_set_24.136_archive_2014-08-12 20:37:08.028004_2014-08-12 20:49:27.348465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 18:06:06.701622 7f5c973a57a0 20 read_log 194416'15417 (0'0) promote f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by osd.50.0:4759 2014-08-13 15:28:40.337941 2014-08-13 18:06:06.701634 7f5c973a57a0 20 read_log 194416'15418 (194416'15417) modify f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by client.2441958.0:2541 2014-08-13 15:28:40.328510 2014-08-13 18:06:06.701647 7f5c973a57a0 20 read_log 194416'15419 (0'0) promote aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by osd.50.0:4789 2014-08-13 15:28:48.305669 2014-08-13 18:06:06.701672 7f5c973a57a0 20 read_log 194416'15420 (194416'15419) modify aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by client.2441958.0:4581 2014-08-13 15:28:48.295597 2014-08-13 18:06:06.701688 7f5c973a57a0 20 read_log 194416'15421 (0'0) promote 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by osd.50.0:4819 2014-08-13 15:28:54.031854 2014-08-13 18:06:06.701701 7f5c973a57a0 20 read_log 194416'15422 (194416'15421) modify 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by client.2441958.0:6064 2014-08-13 15:28:54.020440 2014-08-13 18:06:06.701713 7f5c973a57a0 20 read_log 194416'15423 (0'0) promote 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by osd.50.0:4827 2014-08-13 15:28:56.025679 2014-08-13 18:06:06.701726 7f5c973a57a0 20 read_log 194416'15424 (194416'15423) modify 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by client.2441958.0:6543 2014-08-13 15:28:56.016868 2014-08-13 18:06:06.701738 7f5c973a57a0 20 read_log 194416'15425 (0'0) promote 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by osd.50.0:4833 2014-08-13 15:28:58.647096 2014-08-13 18:06:06.701750 7f5c973a57a0 20 read_log 194416'15426 (194416'15425) modify 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by client.2441958.0:7233 2014-08-13 15:28:58.639937 2014-08-13 18:06:06.701762 7f5c973a57a0 20 read_log 194416'15427 (0'0) promote 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by osd.50.0:4853 2014-08-13 15:29:02.666952 2014-08-13 18:06:06.701774 7f5c973a57a0 20 read_log 194416'15428 (194416'15427) modify 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by client.2441958.0:8300 2014-08-13 15:29:02.659806 2014-08-13 18:06:06.701793 7f5c973a57a0 20 read_log 194416'15429 (0'0) promote af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by osd.50.0:4867 2014-08-13 15:29:06.450451 2014-08-13 18:06:06.701805 7f5c973a57a0 20 read_log 194416'15430 (194416'15429) modify af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by client.2441958.0:9254 2014-08-13 15:29:06.441323 2014-08-13 18:06:06.701820 7f5c973a57a0 20 read_log 194416'15431 (0'0) promote 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by osd.50.0:4877 2014-08-13 15:29:10.265851 2014-08-13 18:06:06.701832 7f5c973a57a0 20 read_log 194416'15432 (194416'15431) modify 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by client.2441958.0:10207 2014-08-13 15:29:10.255722 2014-08-13 18:06:06.701846 7f5c973a57a0 20 read_log 194416'15433 (0'0) promote 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by osd.50.0:4903 2014-08-13 15:29:14.775895 2014-08-13 18:06:06.701858 7f5c973a57a0 20 read_log 194416'15434 (194416'15433) modify 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by client.2441958.0:11325 2014-08-13 15:29:14.766401 2014-08-13 18:06:06.701871 7f5c973a57a0 20 read_log 194416'15435 (0'0) promote b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by osd.50.0:4907 2014-08-13 15:29:15.843710 2014-08-13 18:06:06.701883 7f5c973a57a0 20 read_log 194416'15436 (194416'15435) modify b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by client.2441958.0:11582 2014-08-13 15:29:15.836349 2014-08-13 18:06:06.701897 7f5c973a57a0 20 read_log 194416'15437 (0'0) promote 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by osd.50.0:4919 2014-08-13 15:29:17.552781 2014-08-13 18:06:06.701909 7f5c973a57a0 20 read_log 194416'15438 (194416'15437) modify 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by client.2441958.0:11991 2014-08-13 15:29:17.545175 2014-08-13 18:06:06.701924 7f5c973a57a0 20 read_log 194416'15439 (0'0) promote fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by osd.50.0:4953 2014-08-13 15:29:24.480841 2014-08-13 18:06:06.701943 7f5c973a57a0 20 read_log 194416'15440 (194416'15439) modify fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by client.2441958.0:13587 2014-08-13 15:29:24.471552 2014-08-13 18:06:06.701958 7f5c973a57a0 20 read_log 194416'15441 (0'0) promote 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by osd.50.0:4979 2014-08-13 15:29:31.665871 2014-08-13 18:06:06.701970 7f5c973a57a0 20 read_log 194416'15442 (194416'15441) modify 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by client.2441958.0:15311 2014-08-13 15:29:31.656206 2014-08-13 18:06:06.701984 7f5c973a57a0 20 read_log 194416'15443 (0'0) promote 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by osd.50.0:4983 2014-08-13 15:29:32.279595 2014-08-13 18:06:06.701997 7f5c973a57a0 20 read_log 194416'15444 (194416'15443) modify 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by client.2441958.0:15419 2014-08-13 15:29:32.269452 2014-08-13 18:06:06.702011 7f5c973a57a0 20 read_log 194416'15445 (0'0) promote a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by osd.50.0:5019 2014-08-13 15:29:46.036094 2014-08-13 18:06:06.702023 7f5c973a57a0 20 read_log 194416'15446 (194416'15445) modify a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by client.2441958.0:16908 2014-08-13 15:29:46.025271 2014-08-13 18:06:06.702038 7f5c973a57a0 20 read_log 194422'15447 (0'0) promote 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by osd.50.0:5053 2014-08-13 15:32:19.133039 2014-08-13 18:06:06.702050 7f5c973a57a0 20 read_log 194422'15448 (194422'15447) modify 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by client.1148203.0:956 2014-08-13 15:32:19.125424 2014-08-13 18:06:06.702065 7f5c973a57a0 20 read_log 194422'15449 (0'0) promote 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by osd.50.0:5061 2014-08-13 15:32:19.718056 2014-08-13 18:06:06.702077 7f5c973a57a0 20 read_log 194422'15450 (194422'15449) modify 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by client.1148203.0:1112 2014-08-13 15:32:19.709026 2014-08-13 18:06:06.702090 7f5c973a57a0 20 read_log 194422'15451 (0'0) promote 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by osd.50.0:5073 2014-08-13 15:32:23.772106 2014-08-13 18:06:06.702104 7f5c973a57a0 20 read_log 194422'15452 (194422'15451) modify 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by client.1148203.0:2140 2014-08-13 15:32:23.763153 2014-08-13 18:06:06.702117 7f5c973a57a0 20 read_log 194422'15453 (0'0) promote 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by osd.50.0:5079 2014-08-13 15:32:25.821181 2014-08-13 18:06:06.702129 7f5c973a57a0 20 read_log 194422'15454 (194422'15453) modify 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by client.1148203.0:2680 2014-08-13 15:32:25.813684 2014-08-13 18:06:06.702141 7f5c973a57a0 20 read_log 194422'15455 (0'0) promote 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by osd.50.0:5097 2014-08-13 15:32:29.907712 2014-08-13 18:06:06.702161 7f5c973a57a0 20 read_log 194422'15456 (194422'15455) modify 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by client.1148203.0:3710 2014-08-13 15:32:29.898750 2014-08-13 18:06:06.702174 7f5c973a57a0 20 read_log 194422'15457 (0'0) promote 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by osd.50.0:5101 2014-08-13 15:32:30.200218 2014-08-13 18:06:06.702186 7f5c973a57a0 20 read_log 194422'15458 (194422'15457) modify 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by client.1148203.0:3790 2014-08-13 15:32:30.192225 2014-08-13 18:06:06.702199 7f5c973a57a0 20 read_log 194422'15459 (0'0) promote e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by osd.50.0:5113 2014-08-13 15:32:31.725484 2014-08-13 18:06:06.702237 7f5c973a57a0 20 read_log 194422'15460 (194422'15459) modify e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by client.1148203.0:4192 2014-08-13 15:32:31.717484 2014-08-13 18:06:06.702257 7f5c973a57a0 20 read_log 194422'15461 (0'0) promote 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by osd.50.0:5115 2014-08-13 15:32:31.791282 2014-08-13 18:06:06.702269 7f5c973a57a0 20 read_log 194422'15462 (194422'15461) modify 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by client.1148203.0:4197 2014-08-13 15:32:31.736979 2014-08-13 18:06:06.702282 7f5c973a57a0 20 read_log 194422'15463 (0'0) promote 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by osd.50.0:5157 2014-08-13 15:32:48.143485 2014-08-13 18:06:06.702295 7f5c973a57a0 20 read_log 194422'15464 (194422'15463) modify 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by client.1148203.0:7426 2014-08-13 15:32:48.134944 2014-08-13 18:06:06.702310 7f5c973a57a0 20 read_log 194422'15465 (0'0) promote e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by osd.50.0:5179 2014-08-13 15:32:55.378322 2014-08-13 18:06:06.702325 7f5c973a57a0 20 read_log 194422'15466 (194422'15465) modify e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by client.1148203.0:8538 2014-08-13 15:32:55.369835 2014-08-13 18:06:06.702338 7f5c973a57a0 20 read_log 194422'15467 (0'0) promote 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by osd.50.0:5185 2014-08-13 15:32:57.125714 2014-08-13 18:06:06.702351 7f5c973a57a0 20 read_log 194422'15468 (194422'15467) modify 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by client.1148203.0:8749 2014-08-13 15:32:57.115446 2014-08-13 18:06:06.702363 7f5c973a57a0 20 read_log 194422'15469 (0'0) promote 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by osd.50.0:5199 2014-08-13 15:33:04.743917 2014-08-13 18:06:06.702389 7f5c973a57a0 20 read_log 194422'15470 (194422'15469) modify 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by client.1148203.0:9527 2014-08-13 15:33:04.733257 2014-08-13 18:06:06.702411 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.702875 7f5c973a57a0 10 read_log checking for missing items over interval (194370'15367,194422'15470] 2014-08-13 18:06:06.702922 7f5c973a57a0 15 read_log missing 194422'15470 (194422'15469) modify 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 by client.1148203.0:9527 2014-08-13 15:33:04.733257 2014-08-13 18:06:06.702963 7f5c973a57a0 15 read_log missing 194422'15468 (194422'15467) modify 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 by client.1148203.0:8749 2014-08-13 15:32:57.115446 2014-08-13 18:06:06.702999 7f5c973a57a0 15 read_log missing 194422'15466 (194422'15465) modify e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 by client.1148203.0:8538 2014-08-13 15:32:55.369835 2014-08-13 18:06:06.703035 7f5c973a57a0 15 read_log missing 194422'15464 (194422'15463) modify 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 by client.1148203.0:7426 2014-08-13 15:32:48.134944 2014-08-13 18:06:06.703071 7f5c973a57a0 15 read_log missing 194422'15462 (194422'15461) modify 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 by client.1148203.0:4197 2014-08-13 15:32:31.736979 2014-08-13 18:06:06.703107 7f5c973a57a0 15 read_log missing 194422'15460 (194422'15459) modify e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 by client.1148203.0:4192 2014-08-13 15:32:31.717484 2014-08-13 18:06:06.703142 7f5c973a57a0 15 read_log missing 194422'15458 (194422'15457) modify 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 by client.1148203.0:3790 2014-08-13 15:32:30.192225 2014-08-13 18:06:06.703185 7f5c973a57a0 15 read_log missing 194422'15456 (194422'15455) modify 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 by client.1148203.0:3710 2014-08-13 15:32:29.898750 2014-08-13 18:06:06.703221 7f5c973a57a0 15 read_log missing 194422'15454 (194422'15453) modify 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 by client.1148203.0:2680 2014-08-13 15:32:25.813684 2014-08-13 18:06:06.703257 7f5c973a57a0 15 read_log missing 194422'15452 (194422'15451) modify 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 by client.1148203.0:2140 2014-08-13 15:32:23.763153 2014-08-13 18:06:06.703297 7f5c973a57a0 15 read_log missing 194422'15450 (194422'15449) modify 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 by client.1148203.0:1112 2014-08-13 15:32:19.709026 2014-08-13 18:06:06.703337 7f5c973a57a0 15 read_log missing 194422'15448 (194422'15447) modify 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 by client.1148203.0:956 2014-08-13 15:32:19.125424 2014-08-13 18:06:06.703375 7f5c973a57a0 15 read_log missing 194416'15446 (194416'15445) modify a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 by client.2441958.0:16908 2014-08-13 15:29:46.025271 2014-08-13 18:06:06.703411 7f5c973a57a0 15 read_log missing 194416'15444 (194416'15443) modify 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 by client.2441958.0:15419 2014-08-13 15:29:32.269452 2014-08-13 18:06:06.703458 7f5c973a57a0 15 read_log missing 194416'15442 (194416'15441) modify 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 by client.2441958.0:15311 2014-08-13 15:29:31.656206 2014-08-13 18:06:06.703496 7f5c973a57a0 15 read_log missing 194416'15440 (194416'15439) modify fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 by client.2441958.0:13587 2014-08-13 15:29:24.471552 2014-08-13 18:06:06.703532 7f5c973a57a0 15 read_log missing 194416'15438 (194416'15437) modify 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 by client.2441958.0:11991 2014-08-13 15:29:17.545175 2014-08-13 18:06:06.703579 7f5c973a57a0 15 read_log missing 194416'15436 (194416'15435) modify b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 by client.2441958.0:11582 2014-08-13 15:29:15.836349 2014-08-13 18:06:06.703616 7f5c973a57a0 15 read_log missing 194416'15434 (194416'15433) modify 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 by client.2441958.0:11325 2014-08-13 15:29:14.766401 2014-08-13 18:06:06.703652 7f5c973a57a0 15 read_log missing 194416'15432 (194416'15431) modify 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 by client.2441958.0:10207 2014-08-13 15:29:10.255722 2014-08-13 18:06:06.703688 7f5c973a57a0 15 read_log missing 194416'15430 (194416'15429) modify af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 by client.2441958.0:9254 2014-08-13 15:29:06.441323 2014-08-13 18:06:06.703724 7f5c973a57a0 15 read_log missing 194416'15428 (194416'15427) modify 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 by client.2441958.0:8300 2014-08-13 15:29:02.659806 2014-08-13 18:06:06.703760 7f5c973a57a0 15 read_log missing 194416'15426 (194416'15425) modify 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 by client.2441958.0:7233 2014-08-13 15:28:58.639937 2014-08-13 18:06:06.703796 7f5c973a57a0 15 read_log missing 194416'15424 (194416'15423) modify 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 by client.2441958.0:6543 2014-08-13 15:28:56.016868 2014-08-13 18:06:06.703834 7f5c973a57a0 15 read_log missing 194416'15422 (194416'15421) modify 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 by client.2441958.0:6064 2014-08-13 15:28:54.020440 2014-08-13 18:06:06.703870 7f5c973a57a0 15 read_log missing 194416'15420 (194416'15419) modify aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 by client.2441958.0:4581 2014-08-13 15:28:48.295597 2014-08-13 18:06:06.703906 7f5c973a57a0 15 read_log missing 194416'15418 (194416'15417) modify f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 by client.2441958.0:2541 2014-08-13 15:28:40.328510 2014-08-13 18:06:06.703945 7f5c973a57a0 15 read_log missing 194416'15415 (194416'15415) modify 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:40.334225 2014-08-13 18:06:06.703981 7f5c973a57a0 15 read_log missing 194393'15414 (194393'15413) modify 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 by client.1064182.0:397 2014-08-12 20:49:27.342558 2014-08-13 18:06:06.704021 7f5c973a57a0 15 read_log missing 194380'15410 (194380'15409) modify 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 by client.2231190.0:5113 2014-08-12 20:33:24.579796 2014-08-13 18:06:06.704057 7f5c973a57a0 15 read_log missing 194380'15408 (194380'15407) modify 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 by client.2231190.0:5075 2014-08-12 20:33:24.042859 2014-08-13 18:06:06.704093 7f5c973a57a0 15 read_log missing 194380'15406 (194380'15405) modify d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 by client.2231190.0:5071 2014-08-12 20:33:23.989565 2014-08-13 18:06:06.704129 7f5c973a57a0 15 read_log missing 194378'15404 (194378'15403) modify d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 by client.2231190.0:4860 2014-08-12 20:33:20.379484 2014-08-13 18:06:06.704170 7f5c973a57a0 15 read_log missing 194377'15402 (194377'15401) modify 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 by client.2231190.0:4729 2014-08-12 20:33:18.837895 2014-08-13 18:06:06.704207 7f5c973a57a0 15 read_log missing 194377'15400 (194377'15399) modify 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 by client.2231190.0:4206 2014-08-12 20:33:15.133839 2014-08-13 18:06:06.704243 7f5c973a57a0 15 read_log missing 194377'15398 (194377'15397) modify 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 by client.2231190.0:2397 2014-08-12 20:33:06.812923 2014-08-13 18:06:06.704282 7f5c973a57a0 15 read_log missing 194377'15396 (194377'15395) modify 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 by client.2231190.0:780 2014-08-12 20:33:00.477364 2014-08-13 18:06:06.704319 7f5c973a57a0 15 read_log missing 194377'15394 (194377'15393) modify e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 by client.2231190.0:757 2014-08-12 20:33:00.390011 2014-08-13 18:06:06.704360 7f5c973a57a0 15 read_log missing 194370'15392 (194370'15391) modify c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 by client.1508531.0:11654 2014-08-12 20:31:09.235996 2014-08-13 18:06:06.704400 7f5c973a57a0 15 read_log missing 194370'15390 (194370'15389) modify cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 by client.1508531.0:10564 2014-08-12 20:31:02.614352 2014-08-13 18:06:06.704437 7f5c973a57a0 15 read_log missing 194370'15388 (194370'15387) modify 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 by client.1508531.0:10344 2014-08-12 20:31:01.281710 2014-08-13 18:06:06.704473 7f5c973a57a0 15 read_log missing 194370'15386 (194370'15385) modify 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 by client.1508531.0:9654 2014-08-12 20:30:57.223661 2014-08-13 18:06:06.704520 7f5c973a57a0 15 read_log missing 194370'15384 (194370'15383) modify 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 by client.1508531.0:8483 2014-08-12 20:30:50.118720 2014-08-13 18:06:06.704570 7f5c973a57a0 15 read_log missing 194370'15382 (194370'15381) modify ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 by client.1508531.0:6812 2014-08-12 20:30:41.412373 2014-08-13 18:06:06.704615 7f5c973a57a0 15 read_log missing 194370'15380 (194370'15379) modify 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 by client.1508531.0:5473 2014-08-12 20:30:35.284477 2014-08-13 18:06:06.704653 7f5c973a57a0 15 read_log missing 194370'15378 (194370'15377) modify 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 by client.1508531.0:5429 2014-08-12 20:30:35.078738 2014-08-13 18:06:06.704689 7f5c973a57a0 15 read_log missing 194370'15376 (194370'15375) modify 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 by client.1508531.0:5413 2014-08-12 20:30:35.010202 2014-08-13 18:06:06.704724 7f5c973a57a0 15 read_log missing 194370'15374 (194370'15373) modify 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 by client.1508531.0:2480 2014-08-12 20:30:23.652101 2014-08-13 18:06:06.704760 7f5c973a57a0 15 read_log missing 194370'15372 (194370'15371) modify f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 by client.1508531.0:2453 2014-08-12 20:30:23.553305 2014-08-13 18:06:06.704801 7f5c973a57a0 15 read_log missing 194370'15370 (194370'15369) modify e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 by client.1508531.0:1005 2014-08-12 20:30:18.134242 2014-08-13 18:06:06.704837 7f5c973a57a0 15 read_log missing 194370'15368 (194370'15367) modify d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 by client.1508531.0:142 2014-08-12 20:30:14.678612 2014-08-13 18:06:06.704848 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.704878 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_loaded 2014-08-13 18:06:06.704934 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Initial 0.016569 0 0.000000 2014-08-13 18:06:06.704948 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset 2014-08-13 18:06:06.704958 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=0 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194487 2014-08-13 18:06:06.704968 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 18:06:06.704979 7f5c973a57a0 10 osd.4 194487 pgid 24.181 coll 24.181_head 2014-08-13 18:06:06.705138 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.181 2014-08-13 18:06:06.705160 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.705195 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.181(unlocked)] enter Initial 2014-08-13 18:06:06.705207 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.181(unlocked)] enter NotTrimming 2014-08-13 18:06:06.705375 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.705510 7f5c973a57a0 20 read_log 192834'17290 (0'0) promote 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18226 2014-08-11 15:26:56.142374 2014-08-13 18:06:06.705532 7f5c973a57a0 20 read_log 192834'17291 (192834'17290) modify 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by client.1883634.0:10673 2014-08-11 15:26:56.133982 2014-08-13 18:06:06.705546 7f5c973a57a0 20 read_log 192834'17292 (0'0) promote 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18228 2014-08-11 15:26:56.271737 2014-08-13 18:06:06.705558 7f5c973a57a0 20 read_log 192834'17293 (192834'17292) modify 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by client.1883634.0:10701 2014-08-11 15:26:56.263816 2014-08-13 18:06:06.705571 7f5c973a57a0 20 read_log 192834'17324 (192834'16876) clean 491e0581/benchmark_data_storage0111-ib.pocstor_26659_object12533/head//24 by osd.13.0:18260 2014-08-11 15:26:57.995293 2014-08-13 18:06:06.705583 7f5c973a57a0 20 read_log 192834'17325 (192834'16839) clean 3b031581/benchmark_data_storage0111-ib.pocstor_26659_object11165/head//24 by osd.13.0:18261 2014-08-11 15:26:58.015503 2014-08-13 18:06:06.705596 7f5c973a57a0 20 read_log 192834'17326 (192834'16878) clean ee95581/benchmark_data_storage0111-ib.pocstor_26659_object12648/head//24 by osd.13.0:18262 2014-08-11 15:26:58.016079 2014-08-13 18:06:06.705608 7f5c973a57a0 20 read_log 192834'17330 (192834'16802) clean 593e2981/benchmark_data_storage0111-ib.pocstor_26659_object10540/head//24 by osd.13.0:18266 2014-08-11 15:26:58.063987 2014-08-13 18:06:06.705639 7f5c973a57a0 20 read_log 192834'17331 (192834'16904) clean c8cfa981/benchmark_data_storage0111-ib.pocstor_26659_object14036/head//24 by osd.13.0:18267 2014-08-11 15:26:58.064306 2014-08-13 18:06:06.705654 7f5c973a57a0 20 read_log 192834'17333 (192834'16896) clean a1011d81/benchmark_data_storage0111-ib.pocstor_26659_object13209/head//24 by osd.13.0:18269 2014-08-11 15:26:58.089682 2014-08-13 18:06:06.705667 7f5c973a57a0 20 read_log 192834'17334 (192834'16804) clean 8185dd81/benchmark_data_storage0111-ib.pocstor_26659_object10573/head//24 by osd.13.0:18270 2014-08-11 15:26:58.090253 2014-08-13 18:06:06.705679 7f5c973a57a0 20 read_log 192834'17336 (192834'16910) clean 11593181/benchmark_data_storage0111-ib.pocstor_26659_object14300/head//24 by osd.13.0:18272 2014-08-11 15:26:58.328812 2014-08-13 18:06:06.705692 7f5c973a57a0 20 read_log 192834'17346 (0'0) promote d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18282 2014-08-11 15:26:59.899134 2014-08-13 18:06:06.705704 7f5c973a57a0 20 read_log 192834'17347 (192834'17346) modify d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by client.1883634.0:11454 2014-08-11 15:26:59.889020 2014-08-13 18:06:06.705716 7f5c973a57a0 20 read_log 192834'17350 (0'0) promote dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18286 2014-08-11 15:27:03.202913 2014-08-13 18:06:06.705728 7f5c973a57a0 20 read_log 192834'17351 (192834'17350) modify dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by client.1883634.0:12251 2014-08-11 15:27:03.194397 2014-08-13 18:06:06.705740 7f5c973a57a0 20 read_log 192834'17414 (192834'17336) delete 11593181/benchmark_data_storage0111-ib.pocstor_26659_object14300/head//24 by osd.13.0:18350 2014-08-11 15:27:05.574640 2014-08-13 18:06:06.705756 7f5c973a57a0 20 read_log 192834'17415 (192834'17155) delete fd12b181/benchmark_data_storage0111-ib.pocstor_26659_object323/head//24 by osd.13.0:18351 2014-08-11 15:27:05.581366 2014-08-13 18:06:06.705769 7f5c973a57a0 20 read_log 192834'17417 (192834'16992) clean 329f4181/benchmark_data_storage0111-ib.pocstor_26659_object19253/head//24 by osd.13.0:18353 2014-08-11 15:27:05.587839 2014-08-13 18:06:06.705781 7f5c973a57a0 20 read_log 192834'17418 (192834'16941) clean 7de6b181/benchmark_data_storage0111-ib.pocstor_26659_object15116/head//24 by osd.13.0:18354 2014-08-11 15:27:05.594408 2014-08-13 18:06:06.705793 7f5c973a57a0 20 read_log 192834'17437 (192834'17324) delete 491e0581/benchmark_data_storage0111-ib.pocstor_26659_object12533/head//24 by osd.13.0:18373 2014-08-11 15:27:05.886660 2014-08-13 18:06:06.705805 7f5c973a57a0 20 read_log 192834'17438 (192834'17325) delete 3b031581/benchmark_data_storage0111-ib.pocstor_26659_object11165/head//24 by osd.13.0:18374 2014-08-11 15:27:05.892558 2014-08-13 18:06:06.705817 7f5c973a57a0 20 read_log 192834'17439 (192834'17253) delete 706e1581/benchmark_data_storage0111-ib.pocstor_26659_object7704/head//24 by osd.13.0:18375 2014-08-11 15:27:05.898428 2014-08-13 18:06:06.705829 7f5c973a57a0 20 read_log 192834'17440 (192834'17268) delete 8e503581/benchmark_data_storage0111-ib.pocstor_26659_object10189/head//24 by osd.13.0:18376 2014-08-11 15:27:05.903809 2014-08-13 18:06:06.705842 7f5c973a57a0 20 read_log 192834'17441 (192834'17202) delete 31575581/benchmark_data_storage0111-ib.pocstor_26659_object5439/head//24 by osd.13.0:18377 2014-08-11 15:27:05.908642 2014-08-13 18:06:06.705854 7f5c973a57a0 20 read_log 192834'17442 (192834'17326) delete ee95581/benchmark_data_storage0111-ib.pocstor_26659_object12648/head//24 by osd.13.0:18378 2014-08-11 15:27:05.914337 2014-08-13 18:06:06.705866 7f5c973a57a0 20 read_log 192834'17443 (192834'17254) delete e0a48581/benchmark_data_storage0111-ib.pocstor_26659_object9186/head//24 by osd.13.0:18379 2014-08-11 15:27:05.925461 2014-08-13 18:06:06.705878 7f5c973a57a0 20 read_log 192834'17444 (192834'16968) clean 67cc5581/benchmark_data_storage0111-ib.pocstor_26659_object16450/head//24 by osd.13.0:18380 2014-08-11 15:27:05.931672 2014-08-13 18:06:06.705897 7f5c973a57a0 20 read_log 192834'17446 (192834'17003) clean 3968d581/benchmark_data_storage0111-ib.pocstor_26659_object20233/head//24 by osd.13.0:18382 2014-08-11 15:27:05.947068 2014-08-13 18:06:06.705912 7f5c973a57a0 20 read_log 192834'17447 (192834'16945) clean 49b7e581/benchmark_data_storage0111-ib.pocstor_26659_object15256/head//24 by osd.13.0:18383 2014-08-11 15:27:05.947539 2014-08-13 18:06:06.705925 7f5c973a57a0 20 read_log 192834'17468 (192834'17159) delete aa372981/benchmark_data_storage0111-ib.pocstor_26659_object1476/head//24 by osd.13.0:18404 2014-08-11 15:27:06.141493 2014-08-13 18:06:06.705937 7f5c973a57a0 20 read_log 192834'17469 (192834'17330) delete 593e2981/benchmark_data_storage0111-ib.pocstor_26659_object10540/head//24 by osd.13.0:18405 2014-08-11 15:27:06.148804 2014-08-13 18:06:06.705949 7f5c973a57a0 20 read_log 192834'17470 (192834'17260) delete 19476981/benchmark_data_storage0111-ib.pocstor_26659_object9234/head//24 by osd.13.0:18406 2014-08-11 15:27:06.154281 2014-08-13 18:06:06.705961 7f5c973a57a0 20 read_log 192834'17471 (192834'17331) delete c8cfa981/benchmark_data_storage0111-ib.pocstor_26659_object14036/head//24 by osd.13.0:18407 2014-08-11 15:27:06.154787 2014-08-13 18:06:06.705973 7f5c973a57a0 20 read_log 192834'17474 (192834'17021) clean 78ef3981/benchmark_data_storage0111-ib.pocstor_26659_object22036/head//24 by osd.13.0:18410 2014-08-11 15:27:06.171822 2014-08-13 18:06:06.705985 7f5c973a57a0 20 read_log 192834'17475 (192834'16949) clean 1cc2b981/benchmark_data_storage0111-ib.pocstor_26659_object15351/head//24 by osd.13.0:18411 2014-08-11 15:27:06.172418 2014-08-13 18:06:06.705998 7f5c973a57a0 20 read_log 192834'17476 (192834'16994) clean 4f75d981/benchmark_data_storage0111-ib.pocstor_26659_object19627/head//24 by osd.13.0:18412 2014-08-11 15:27:06.172747 2014-08-13 18:06:06.706010 7f5c973a57a0 20 read_log 192834'17492 (192834'17333) delete a1011d81/benchmark_data_storage0111-ib.pocstor_26659_object13209/head//24 by osd.13.0:18428 2014-08-11 15:27:06.660662 2014-08-13 18:06:06.706022 7f5c973a57a0 20 read_log 192834'17493 (192834'17210) delete d19a2d81/benchmark_data_storage0111-ib.pocstor_26659_object5812/head//24 by osd.13.0:18429 2014-08-11 15:27:06.666226 2014-08-13 18:06:06.706034 7f5c973a57a0 20 read_log 192834'17494 (192834'17211) delete 6c4d5d81/benchmark_data_storage0111-ib.pocstor_26659_object5334/head//24 by osd.13.0:18430 2014-08-11 15:27:06.668752 2014-08-13 18:06:06.706046 7f5c973a57a0 20 read_log 192834'17495 (192834'17263) delete 1e677d81/benchmark_data_storage0111-ib.pocstor_26659_object9002/head//24 by osd.13.0:18431 2014-08-11 15:27:06.669104 2014-08-13 18:06:06.706058 7f5c973a57a0 20 read_log 192834'17496 (192834'17023) clean e2272d81/benchmark_data_storage0111-ib.pocstor_26659_object22061/head//24 by osd.13.0:18432 2014-08-11 15:27:06.672119 2014-08-13 18:06:06.706070 7f5c973a57a0 20 read_log 192834'17497 (192834'17212) delete 9deccd81/benchmark_data_storage0111-ib.pocstor_26659_object3029/head//24 by osd.13.0:18433 2014-08-11 15:27:06.674717 2014-08-13 18:06:06.706082 7f5c973a57a0 20 read_log 192834'17498 (192834'16981) clean c1638d81/benchmark_data_storage0111-ib.pocstor_26659_object17673/head//24 by osd.13.0:18434 2014-08-11 15:27:06.674982 2014-08-13 18:06:06.706095 7f5c973a57a0 20 read_log 192834'17499 (192834'17018) clean 14cbbd81/benchmark_data_storage0111-ib.pocstor_26659_object21700/head//24 by osd.13.0:18435 2014-08-11 15:27:06.675226 2014-08-13 18:06:06.706107 7f5c973a57a0 20 read_log 192834'17500 (192834'17334) delete 8185dd81/benchmark_data_storage0111-ib.pocstor_26659_object10573/head//24 by osd.13.0:18436 2014-08-11 15:27:06.679826 2014-08-13 18:06:06.706119 7f5c973a57a0 20 read_log 192834'17501 (192834'17213) delete f2b7fd81/benchmark_data_storage0111-ib.pocstor_26659_object4829/head//24 by osd.13.0:18437 2014-08-11 15:27:06.684253 2014-08-13 18:06:06.706131 7f5c973a57a0 20 read_log 192834'17502 (192834'17214) delete f6aefd81/benchmark_data_storage0111-ib.pocstor_26659_object4912/head//24 by osd.13.0:18438 2014-08-11 15:27:06.688303 2014-08-13 18:06:06.706180 7f5c973a57a0 20 read_log 192834'17597 (192834'17417) delete 329f4181/benchmark_data_storage0111-ib.pocstor_26659_object19253/head//24 by osd.13.0:18533 2014-08-11 15:27:07.288394 2014-08-13 18:06:06.706197 7f5c973a57a0 20 read_log 192834'17598 (192834'17418) delete 7de6b181/benchmark_data_storage0111-ib.pocstor_26659_object15116/head//24 by osd.13.0:18534 2014-08-11 15:27:07.292042 2014-08-13 18:06:06.706211 7f5c973a57a0 20 read_log 192834'17604 (192834'17444) delete 67cc5581/benchmark_data_storage0111-ib.pocstor_26659_object16450/head//24 by osd.13.0:18540 2014-08-11 15:27:07.315677 2014-08-13 18:06:06.706223 7f5c973a57a0 20 read_log 192834'17605 (192834'17446) delete 3968d581/benchmark_data_storage0111-ib.pocstor_26659_object20233/head//24 by osd.13.0:18541 2014-08-11 15:27:07.319783 2014-08-13 18:06:06.706238 7f5c973a57a0 20 read_log 192834'17606 (192834'17447) delete 49b7e581/benchmark_data_storage0111-ib.pocstor_26659_object15256/head//24 by osd.13.0:18542 2014-08-11 15:27:07.323144 2014-08-13 18:06:06.706250 7f5c973a57a0 20 read_log 192834'17615 (192834'17474) delete 78ef3981/benchmark_data_storage0111-ib.pocstor_26659_object22036/head//24 by osd.13.0:18551 2014-08-11 15:27:07.353216 2014-08-13 18:06:06.706265 7f5c973a57a0 20 read_log 192834'17616 (192834'17475) delete 1cc2b981/benchmark_data_storage0111-ib.pocstor_26659_object15351/head//24 by osd.13.0:18552 2014-08-11 15:27:07.357016 2014-08-13 18:06:06.706278 7f5c973a57a0 20 read_log 192834'17617 (192834'17476) delete 4f75d981/benchmark_data_storage0111-ib.pocstor_26659_object19627/head//24 by osd.13.0:18553 2014-08-11 15:27:07.360260 2014-08-13 18:06:06.706294 7f5c973a57a0 20 read_log 192834'17627 (192834'17496) delete e2272d81/benchmark_data_storage0111-ib.pocstor_26659_object22061/head//24 by osd.13.0:18563 2014-08-11 15:27:07.373066 2014-08-13 18:06:06.706306 7f5c973a57a0 20 read_log 192834'17628 (192834'17498) delete c1638d81/benchmark_data_storage0111-ib.pocstor_26659_object17673/head//24 by osd.13.0:18564 2014-08-11 15:27:07.373878 2014-08-13 18:06:06.706319 7f5c973a57a0 20 read_log 192834'17629 (192834'17499) delete 14cbbd81/benchmark_data_storage0111-ib.pocstor_26659_object21700/head//24 by osd.13.0:18565 2014-08-11 15:27:07.376663 2014-08-13 18:06:06.706334 7f5c973a57a0 20 read_log 192834'17649 (0'0) promote b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18585 2014-08-11 15:27:10.249546 2014-08-13 18:06:06.706346 7f5c973a57a0 20 read_log 192834'17650 (192834'17649) modify b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by client.1883634.0:13611 2014-08-11 15:27:10.240825 2014-08-13 18:06:06.706359 7f5c973a57a0 20 read_log 192834'17669 (0'0) promote f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18605 2014-08-11 15:27:14.852985 2014-08-13 18:06:06.706371 7f5c973a57a0 20 read_log 192834'17670 (192834'17669) modify f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by client.1883634.0:14710 2014-08-11 15:27:14.844768 2014-08-13 18:06:06.706384 7f5c973a57a0 20 read_log 192834'17691 (192834'17030) clean 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:18627 2014-08-11 15:27:19.111902 2014-08-13 18:06:06.706396 7f5c973a57a0 20 read_log 192834'17699 (192834'17691) delete 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:18635 2014-08-11 15:27:19.178885 2014-08-13 18:06:06.706409 7f5c973a57a0 20 read_log 192834'17711 (0'0) promote f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:18647 2014-08-11 15:27:22.198168 2014-08-13 18:06:06.706421 7f5c973a57a0 20 read_log 192834'17712 (192834'17711) modify f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by client.1883634.0:15891 2014-08-11 15:27:22.189467 2014-08-13 18:06:06.706434 7f5c973a57a0 20 read_log 192834'17743 (192834'17070) clean d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:18679 2014-08-11 15:27:26.378659 2014-08-13 18:06:06.706455 7f5c973a57a0 20 read_log 192834'17744 (0'0) promote 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:18680 2014-08-11 15:27:26.612915 2014-08-13 18:06:06.706471 7f5c973a57a0 20 read_log 192834'17745 (192834'17744) modify 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by client.1883634.0:16241 2014-08-11 15:27:26.604648 2014-08-13 18:06:06.706484 7f5c973a57a0 20 read_log 192834'17748 (0'0) promote 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:18684 2014-08-11 15:27:27.334259 2014-08-13 18:06:06.706516 7f5c973a57a0 20 read_log 192834'17749 (192834'17748) modify 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by client.1883634.0:16383 2014-08-11 15:27:27.326581 2014-08-13 18:06:06.706533 7f5c973a57a0 20 read_log 192834'17754 (0'0) promote 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:18690 2014-08-11 15:27:28.448576 2014-08-13 18:06:06.706556 7f5c973a57a0 20 read_log 192834'17755 (192834'17754) modify 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by client.1883634.0:16612 2014-08-11 15:27:28.440537 2014-08-13 18:06:06.706571 7f5c973a57a0 20 read_log 192834'17760 (0'0) promote 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:18696 2014-08-11 15:27:30.275302 2014-08-13 18:06:06.706584 7f5c973a57a0 20 read_log 192834'17761 (192834'17760) modify 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by client.1883634.0:16994 2014-08-11 15:27:30.267363 2014-08-13 18:06:06.706603 7f5c973a57a0 20 read_log 192834'17771 (192834'17072) clean 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:18707 2014-08-11 15:27:32.925390 2014-08-13 18:06:06.706616 7f5c973a57a0 20 read_log 192834'17772 (192834'17098) clean ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:18708 2014-08-11 15:27:32.926582 2014-08-13 18:06:06.706629 7f5c973a57a0 20 read_log 192834'17773 (192834'17092) clean 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:18709 2014-08-11 15:27:32.936848 2014-08-13 18:06:06.706648 7f5c973a57a0 20 read_log 192834'17774 (192834'17743) delete d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:18710 2014-08-11 15:27:33.896114 2014-08-13 18:06:06.706664 7f5c973a57a0 20 read_log 192834'17775 (192834'17086) clean e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:18711 2014-08-11 15:27:33.974366 2014-08-13 18:06:06.706677 7f5c973a57a0 20 read_log 192834'17776 (192834'17076) clean 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:18712 2014-08-11 15:27:33.975119 2014-08-13 18:06:06.706713 7f5c973a57a0 20 read_log 192834'17796 (192834'17771) delete 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:18732 2014-08-11 15:27:35.493146 2014-08-13 18:06:06.706728 7f5c973a57a0 20 read_log 192834'17797 (192834'17772) delete ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:18733 2014-08-11 15:27:35.497531 2014-08-13 18:06:06.706741 7f5c973a57a0 20 read_log 192834'17798 (192834'17773) delete 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:18734 2014-08-11 15:27:35.498141 2014-08-13 18:06:06.706753 7f5c973a57a0 20 read_log 192834'17799 (192834'17775) delete e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:18735 2014-08-11 15:27:35.505568 2014-08-13 18:06:06.706766 7f5c973a57a0 20 read_log 192834'17800 (192834'17776) delete 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:18736 2014-08-11 15:27:35.509900 2014-08-13 18:06:06.706778 7f5c973a57a0 20 read_log 192834'17825 (0'0) promote 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:18761 2014-08-11 15:27:39.222838 2014-08-13 18:06:06.706798 7f5c973a57a0 20 read_log 192834'17826 (192834'17825) modify 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by client.1883634.0:17825 2014-08-11 15:27:39.214203 2014-08-13 18:06:06.706813 7f5c973a57a0 20 read_log 192834'17852 (192834'17138) clean 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:18788 2014-08-11 15:27:41.906046 2014-08-13 18:06:06.706825 7f5c973a57a0 20 read_log 192834'17853 (192834'17146) clean 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:18789 2014-08-11 15:27:41.906313 2014-08-13 18:06:06.706838 7f5c973a57a0 20 read_log 192834'17869 (192834'17852) delete 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:18805 2014-08-11 15:27:47.158942 2014-08-13 18:06:06.706850 7f5c973a57a0 20 read_log 192834'17870 (192834'17853) delete 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:18806 2014-08-11 15:27:47.167264 2014-08-13 18:06:06.706862 7f5c973a57a0 20 read_log 192834'17879 (192834'17218) clean b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:18815 2014-08-11 15:27:47.308165 2014-08-13 18:06:06.706874 7f5c973a57a0 20 read_log 192834'17890 (0'0) promote 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:18826 2014-08-11 15:27:51.423418 2014-08-13 18:06:06.706886 7f5c973a57a0 20 read_log 192834'17891 (192834'17890) modify 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by client.1883634.0:19346 2014-08-11 15:27:51.414708 2014-08-13 18:06:06.706898 7f5c973a57a0 20 read_log 192834'17898 (192834'17879) delete b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:18834 2014-08-11 15:27:52.884975 2014-08-13 18:06:06.706911 7f5c973a57a0 20 read_log 192834'17913 (192834'17228) clean 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:18849 2014-08-11 15:27:53.105535 2014-08-13 18:06:06.706924 7f5c973a57a0 20 read_log 192834'17917 (192834'17913) delete 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:18853 2014-08-11 15:27:53.318603 2014-08-13 18:06:06.706936 7f5c973a57a0 20 read_log 192834'17927 (0'0) promote 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:18863 2014-08-11 15:27:54.229847 2014-08-13 18:06:06.706948 7f5c973a57a0 20 read_log 192834'17928 (192834'17927) modify 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by client.1883634.0:19815 2014-08-11 15:27:54.220537 2014-08-13 18:06:06.706960 7f5c973a57a0 20 read_log 192834'17941 (192834'17293) clean 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18877 2014-08-11 15:28:00.536037 2014-08-13 18:06:06.706982 7f5c973a57a0 20 read_log 192834'17942 (192834'17347) clean d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18878 2014-08-11 15:28:00.760494 2014-08-13 18:06:06.706996 7f5c973a57a0 20 read_log 192834'17947 (192834'17291) clean 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18883 2014-08-11 15:28:00.888031 2014-08-13 18:06:06.707011 7f5c973a57a0 20 read_log 192834'17955 (192834'17941) delete 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:18891 2014-08-11 15:28:00.930925 2014-08-13 18:06:06.707023 7f5c973a57a0 20 read_log 192834'17956 (192834'17942) delete d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:18892 2014-08-11 15:28:00.934581 2014-08-13 18:06:06.707037 7f5c973a57a0 20 read_log 192834'17963 (192834'17947) delete 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:18899 2014-08-11 15:28:00.958447 2014-08-13 18:06:06.707050 7f5c973a57a0 20 read_log 192834'17974 (0'0) promote 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:18910 2014-08-11 15:28:01.506362 2014-08-13 18:06:06.707074 7f5c973a57a0 20 read_log 192834'17975 (192834'17974) modify 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by client.1883634.0:20787 2014-08-11 15:28:01.498456 2014-08-13 18:06:06.707089 7f5c973a57a0 20 read_log 192834'17988 (0'0) promote 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:18924 2014-08-11 15:28:05.402856 2014-08-13 18:06:06.707101 7f5c973a57a0 20 read_log 192834'17989 (192834'17988) modify 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by client.1883634.0:21825 2014-08-11 15:28:05.395611 2014-08-13 18:06:06.707117 7f5c973a57a0 20 read_log 192834'18007 (192834'17351) clean dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18943 2014-08-11 15:28:06.592044 2014-08-13 18:06:06.707129 7f5c973a57a0 20 read_log 192834'18023 (192834'18007) delete dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:18959 2014-08-11 15:28:11.679808 2014-08-13 18:06:06.707141 7f5c973a57a0 20 read_log 192834'18026 (192834'17650) clean b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18962 2014-08-11 15:28:11.785582 2014-08-13 18:06:06.707162 7f5c973a57a0 20 read_log 192834'18033 (192834'18026) delete b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:18969 2014-08-11 15:28:11.829757 2014-08-13 18:06:06.707174 7f5c973a57a0 20 read_log 192834'18046 (192834'17670) clean f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18982 2014-08-11 15:28:17.147079 2014-08-13 18:06:06.707191 7f5c973a57a0 20 read_log 192834'18060 (192834'18046) delete f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:18996 2014-08-11 15:28:17.476690 2014-08-13 18:06:06.707204 7f5c973a57a0 20 read_log 192834'18069 (0'0) promote 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19005 2014-08-11 15:28:17.535118 2014-08-13 18:06:06.707239 7f5c973a57a0 20 read_log 192834'18071 (192834'18069) modify 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by client.1883634.0:23683 2014-08-11 15:28:17.443004 2014-08-13 18:06:06.707254 7f5c973a57a0 20 read_log 192834'18118 (192834'17712) clean f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19054 2014-08-11 15:28:22.809678 2014-08-13 18:06:06.707267 7f5c973a57a0 20 read_log 192834'18123 (0'0) promote d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19059 2014-08-11 15:28:23.248839 2014-08-13 18:06:06.707292 7f5c973a57a0 20 read_log 192834'18124 (192834'18123) modify d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by client.1883634.0:26520 2014-08-11 15:28:23.209939 2014-08-13 18:06:06.707305 7f5c973a57a0 20 read_log 192834'18125 (0'0) promote 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19061 2014-08-11 15:28:23.367455 2014-08-13 18:06:06.707317 7f5c973a57a0 20 read_log 192834'18126 (192834'18125) modify 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by client.1883634.0:26627 2014-08-11 15:28:23.340438 2014-08-13 18:06:06.707340 7f5c973a57a0 20 read_log 192834'18129 (0'0) promote 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19065 2014-08-11 15:28:23.604963 2014-08-13 18:06:06.707353 7f5c973a57a0 20 read_log 192834'18130 (192834'18129) modify 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by client.1883634.0:26777 2014-08-11 15:28:23.569751 2014-08-13 18:06:06.707365 7f5c973a57a0 20 read_log 192834'18139 (0'0) promote e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19075 2014-08-11 15:28:24.861779 2014-08-13 18:06:06.707377 7f5c973a57a0 20 read_log 192834'18140 (192834'18139) modify e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by client.1883634.0:27699 2014-08-11 15:28:24.836221 2014-08-13 18:06:06.707397 7f5c973a57a0 20 read_log 192834'18145 (0'0) promote 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19081 2014-08-11 15:28:25.498048 2014-08-13 18:06:06.707412 7f5c973a57a0 20 read_log 192834'18146 (192834'18145) modify 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by client.1883634.0:28174 2014-08-11 15:28:25.470648 2014-08-13 18:06:06.707424 7f5c973a57a0 20 read_log 192834'18151 (0'0) promote ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19087 2014-08-11 15:28:25.663805 2014-08-13 18:06:06.707437 7f5c973a57a0 20 read_log 192834'18152 (192834'18151) modify ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by client.1883634.0:28310 2014-08-11 15:28:25.632503 2014-08-13 18:06:06.707449 7f5c973a57a0 20 read_log 192834'18189 (0'0) promote 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19125 2014-08-11 15:28:27.540624 2014-08-13 18:06:06.707461 7f5c973a57a0 20 read_log 192834'18190 (192834'18189) modify 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by client.1883634.0:29615 2014-08-11 15:28:27.506757 2014-08-13 18:06:06.707473 7f5c973a57a0 20 read_log 192834'18195 (192834'18118) delete f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19131 2014-08-11 15:28:27.940960 2014-08-13 18:06:06.707485 7f5c973a57a0 20 read_log 192834'18196 (192834'17745) clean 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19132 2014-08-11 15:28:27.949986 2014-08-13 18:06:06.707497 7f5c973a57a0 20 read_log 192834'18200 (192834'18196) delete 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19136 2014-08-11 15:28:28.001102 2014-08-13 18:06:06.707510 7f5c973a57a0 20 read_log 192834'18202 (192834'17749) clean 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19138 2014-08-11 15:28:28.052582 2014-08-13 18:06:06.707522 7f5c973a57a0 20 read_log 192834'18210 (0'0) promote 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19146 2014-08-11 15:28:28.293825 2014-08-13 18:06:06.707534 7f5c973a57a0 20 read_log 192834'18211 (192834'18210) modify 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by client.1883634.0:30232 2014-08-11 15:28:28.263090 2014-08-13 18:06:06.707546 7f5c973a57a0 20 read_log 192834'18226 (0'0) promote b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19162 2014-08-11 15:28:31.174202 2014-08-13 18:06:06.707565 7f5c973a57a0 20 read_log 192834'18227 (192834'18226) modify b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by client.1883634.0:31932 2014-08-11 15:28:31.146398 2014-08-13 18:06:06.707580 7f5c973a57a0 20 read_log 192834'18236 (0'0) promote 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19172 2014-08-11 15:28:31.467115 2014-08-13 18:06:06.707595 7f5c973a57a0 20 read_log 192834'18237 (192834'18236) modify 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by client.1883634.0:32161 2014-08-11 15:28:31.430490 2014-08-13 18:06:06.707608 7f5c973a57a0 20 read_log 192834'18258 (192834'17761) clean 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19194 2014-08-11 15:28:33.203353 2014-08-13 18:06:06.707621 7f5c973a57a0 20 read_log 192834'18260 (192834'18202) delete 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19196 2014-08-11 15:28:33.217425 2014-08-13 18:06:06.707633 7f5c973a57a0 20 read_log 192834'18264 (192834'18258) delete 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19200 2014-08-11 15:28:33.265469 2014-08-13 18:06:06.707648 7f5c973a57a0 20 read_log 192834'18265 (192834'17755) clean 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19201 2014-08-11 15:28:33.336936 2014-08-13 18:06:06.707671 7f5c973a57a0 20 read_log 192834'18279 (0'0) promote 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19215 2014-08-11 15:28:34.142627 2014-08-13 18:06:06.707686 7f5c973a57a0 20 read_log 192834'18280 (192834'18279) modify 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by client.1883634.0:33997 2014-08-11 15:28:34.104537 2014-08-13 18:06:06.707698 7f5c973a57a0 20 read_log 192834'18281 (0'0) promote 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19217 2014-08-11 15:28:34.370684 2014-08-13 18:06:06.707710 7f5c973a57a0 20 read_log 192834'18282 (192834'18281) modify 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by client.1883634.0:34025 2014-08-11 15:28:34.138651 2014-08-13 18:06:06.707725 7f5c973a57a0 20 read_log 192834'18293 (0'0) promote d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19229 2014-08-11 15:28:35.321339 2014-08-13 18:06:06.707737 7f5c973a57a0 20 read_log 192834'18294 (192834'18293) modify d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by client.1883634.0:34778 2014-08-11 15:28:35.287194 2014-08-13 18:06:06.707749 7f5c973a57a0 20 read_log 192834'18297 (0'0) promote dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19233 2014-08-11 15:28:36.896926 2014-08-13 18:06:06.707761 7f5c973a57a0 20 read_log 192834'18298 (192834'18297) modify dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by client.1883634.0:35575 2014-08-11 15:28:36.863215 2014-08-13 18:06:06.707776 7f5c973a57a0 20 read_log 192834'18322 (192834'18265) delete 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19258 2014-08-11 15:28:38.481727 2014-08-13 18:06:06.707789 7f5c973a57a0 20 read_log 192834'18328 (0'0) promote b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19264 2014-08-11 15:28:38.681633 2014-08-13 18:06:06.707803 7f5c973a57a0 20 read_log 192834'18329 (192834'18328) modify b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by client.1883634.0:36935 2014-08-11 15:28:38.610177 2014-08-13 18:06:06.707815 7f5c973a57a0 20 read_log 192834'18348 (0'0) promote f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19284 2014-08-11 15:28:40.160628 2014-08-13 18:06:06.707829 7f5c973a57a0 20 read_log 192834'18349 (192834'18348) modify f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by client.1883634.0:38034 2014-08-11 15:28:40.127642 2014-08-13 18:06:06.707841 7f5c973a57a0 20 read_log 192834'18372 (0'0) promote f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19308 2014-08-11 15:28:41.974147 2014-08-13 18:06:06.707855 7f5c973a57a0 20 read_log 192834'18373 (192834'18372) modify f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by client.1883634.0:39215 2014-08-11 15:28:41.772740 2014-08-13 18:06:06.707868 7f5c973a57a0 20 read_log 192834'18378 (0'0) promote 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19314 2014-08-11 15:28:42.341059 2014-08-13 18:06:06.707882 7f5c973a57a0 20 read_log 192834'18379 (192834'18378) modify 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by client.1883634.0:39565 2014-08-11 15:28:42.308984 2014-08-13 18:06:06.707895 7f5c973a57a0 20 read_log 192834'18382 (0'0) promote 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19318 2014-08-11 15:28:42.495623 2014-08-13 18:06:06.707907 7f5c973a57a0 20 read_log 192834'18383 (192834'18382) modify 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by client.1883634.0:39707 2014-08-11 15:28:42.458028 2014-08-13 18:06:06.707921 7f5c973a57a0 20 read_log 192834'18388 (0'0) promote 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19324 2014-08-11 15:28:42.864371 2014-08-13 18:06:06.707940 7f5c973a57a0 20 read_log 192834'18389 (192834'18388) modify 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by client.1883634.0:39936 2014-08-11 15:28:42.838307 2014-08-13 18:06:06.707955 7f5c973a57a0 20 read_log 192834'18394 (0'0) promote 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19330 2014-08-11 15:28:43.552174 2014-08-13 18:06:06.707967 7f5c973a57a0 20 read_log 192834'18395 (192834'18394) modify 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by client.1883634.0:40318 2014-08-11 15:28:43.519407 2014-08-13 18:06:06.707982 7f5c973a57a0 20 read_log 192834'18404 (192834'17826) clean 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19340 2014-08-11 15:28:43.849176 2014-08-13 18:06:06.707995 7f5c973a57a0 20 read_log 192834'18408 (192834'18404) delete 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19344 2014-08-11 15:28:43.936807 2014-08-13 18:06:06.708009 7f5c973a57a0 20 read_log 192834'18419 (0'0) promote 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19355 2014-08-11 15:28:44.706532 2014-08-13 18:06:06.708021 7f5c973a57a0 20 read_log 192834'18420 (192834'18419) modify 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by client.1883634.0:41149 2014-08-11 15:28:44.680902 2014-08-13 18:06:06.708050 7f5c973a57a0 20 read_log 192834'18437 (192834'17891) modify 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by client.1883634.0:42670 2014-08-11 15:28:46.317733 2014-08-13 18:06:06.708065 7f5c973a57a0 20 read_log 192834'18443 (192834'17928) modify 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by client.1883634.0:43139 2014-08-11 15:28:46.465951 2014-08-13 18:06:06.708097 7f5c973a57a0 20 read_log 192834'18449 (192834'17975) modify 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by client.1883634.0:44111 2014-08-11 15:28:46.765006 2014-08-13 18:06:06.708109 7f5c973a57a0 20 read_log 192834'18456 (192834'17989) modify 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by client.1883634.0:45149 2014-08-11 15:28:47.174714 2014-08-13 18:06:06.708124 7f5c973a57a0 20 read_log 192834'18486 (192834'18071) clean 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19422 2014-08-11 15:29:19.742758 2014-08-13 18:06:06.708136 7f5c973a57a0 20 read_log 192834'18492 (192834'18486) delete 2ffd7d81/benchmark_data_storage0111-ib.pocstor_41526_object358/head//24 by osd.13.0:19428 2014-08-11 15:29:19.780150 2014-08-13 18:06:06.708156 7f5c973a57a0 20 read_log 192834'18503 (192834'18126) clean 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19439 2014-08-11 15:29:24.942869 2014-08-13 18:06:06.708170 7f5c973a57a0 20 read_log 192834'18506 (192834'18140) clean e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19442 2014-08-11 15:29:24.955659 2014-08-13 18:06:06.708185 7f5c973a57a0 20 read_log 192834'18507 (192834'18124) clean d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19443 2014-08-11 15:29:24.959602 2014-08-13 18:06:06.708197 7f5c973a57a0 20 read_log 192834'18508 (192834'18130) clean 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19444 2014-08-11 15:29:24.968301 2014-08-13 18:06:06.708210 7f5c973a57a0 20 read_log 192834'18522 (192834'18503) delete 4c584181/benchmark_data_storage0111-ib.pocstor_41526_object3302/head//24 by osd.13.0:19458 2014-08-11 15:29:25.043841 2014-08-13 18:06:06.708222 7f5c973a57a0 20 read_log 192834'18525 (192834'18506) delete e968a581/benchmark_data_storage0111-ib.pocstor_41526_object4374/head//24 by osd.13.0:19461 2014-08-11 15:29:25.058735 2014-08-13 18:06:06.708234 7f5c973a57a0 20 read_log 192834'18526 (192834'18507) delete d209a581/benchmark_data_storage0111-ib.pocstor_41526_object3195/head//24 by osd.13.0:19462 2014-08-11 15:29:25.059094 2014-08-13 18:06:06.708272 7f5c973a57a0 20 read_log 192834'18527 (192834'18508) delete 79d7c581/benchmark_data_storage0111-ib.pocstor_41526_object3452/head//24 by osd.13.0:19463 2014-08-11 15:29:25.062848 2014-08-13 18:06:06.708288 7f5c973a57a0 20 read_log 192834'18547 (192834'18152) clean ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19483 2014-08-11 15:29:30.301602 2014-08-13 18:06:06.708300 7f5c973a57a0 20 read_log 192834'18548 (192834'18146) clean 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19484 2014-08-11 15:29:30.304081 2014-08-13 18:06:06.708320 7f5c973a57a0 20 read_log 192834'18557 (192834'18190) clean 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19493 2014-08-11 15:29:30.350732 2014-08-13 18:06:06.708333 7f5c973a57a0 20 read_log 192834'18575 (192834'18547) delete ba684181/benchmark_data_storage0111-ib.pocstor_41526_object4985/head//24 by osd.13.0:19511 2014-08-11 15:29:30.446223 2014-08-13 18:06:06.708345 7f5c973a57a0 20 read_log 192834'18576 (192834'18548) delete 966cd181/benchmark_data_storage0111-ib.pocstor_41526_object4849/head//24 by osd.13.0:19512 2014-08-11 15:29:30.450294 2014-08-13 18:06:06.708357 7f5c973a57a0 20 read_log 192834'18578 (192834'18211) clean 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19514 2014-08-11 15:29:30.454205 2014-08-13 18:06:06.708370 7f5c973a57a0 20 read_log 192834'18586 (192834'18557) delete 1555b981/benchmark_data_storage0111-ib.pocstor_41526_object6290/head//24 by osd.13.0:19522 2014-08-11 15:29:30.488665 2014-08-13 18:06:06.708382 7f5c973a57a0 20 read_log 192834'18590 (192834'18578) delete 8e3c0d81/benchmark_data_storage0111-ib.pocstor_41526_object6907/head//24 by osd.13.0:19526 2014-08-11 15:29:30.502121 2014-08-13 18:06:06.708396 7f5c973a57a0 20 read_log 192834'18609 (192834'18227) clean b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19545 2014-08-11 15:29:35.635765 2014-08-13 18:06:06.708409 7f5c973a57a0 20 read_log 192834'18610 (192834'18282) clean 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19546 2014-08-11 15:29:35.641672 2014-08-13 18:06:06.708421 7f5c973a57a0 20 read_log 192834'18611 (192834'18294) clean d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19547 2014-08-11 15:29:35.643307 2014-08-13 18:06:06.708433 7f5c973a57a0 20 read_log 192834'18618 (192834'18237) clean 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19554 2014-08-11 15:29:35.669634 2014-08-13 18:06:06.708457 7f5c973a57a0 20 read_log 192834'18619 (192834'18280) clean 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19555 2014-08-11 15:29:35.675810 2014-08-13 18:06:06.708476 7f5c973a57a0 20 read_log 192834'18642 (192834'18609) delete b4192581/benchmark_data_storage0111-ib.pocstor_41526_object8607/head//24 by osd.13.0:19578 2014-08-11 15:29:35.740101 2014-08-13 18:06:06.708500 7f5c973a57a0 20 read_log 192834'18643 (192834'18610) delete 5013a581/benchmark_data_storage0111-ib.pocstor_41526_object10700/head//24 by osd.13.0:19579 2014-08-11 15:29:35.740471 2014-08-13 18:06:06.708513 7f5c973a57a0 20 read_log 192834'18644 (192834'18611) delete d65da581/benchmark_data_storage0111-ib.pocstor_41526_object11453/head//24 by osd.13.0:19580 2014-08-11 15:29:35.743664 2014-08-13 18:06:06.708532 7f5c973a57a0 20 read_log 192834'18651 (192834'18618) delete 361c8981/benchmark_data_storage0111-ib.pocstor_41526_object8836/head//24 by osd.13.0:19587 2014-08-11 15:29:35.755881 2014-08-13 18:06:06.708554 7f5c973a57a0 20 read_log 192834'18652 (192834'18619) delete 722cb981/benchmark_data_storage0111-ib.pocstor_41526_object10672/head//24 by osd.13.0:19588 2014-08-11 15:29:35.758390 2014-08-13 18:06:06.708574 7f5c973a57a0 20 read_log 192834'18675 (192834'18349) clean f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19611 2014-08-11 15:29:40.869422 2014-08-13 18:06:06.708609 7f5c973a57a0 20 read_log 192834'18680 (192834'18329) clean b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19616 2014-08-11 15:29:40.885978 2014-08-13 18:06:06.708625 7f5c973a57a0 20 read_log 192834'18681 (192834'18298) clean dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19617 2014-08-11 15:29:40.888777 2014-08-13 18:06:06.708638 7f5c973a57a0 20 read_log 192834'18704 (192834'18675) delete f6c86181/benchmark_data_storage0111-ib.pocstor_41526_object14709/head//24 by osd.13.0:19640 2014-08-11 15:29:40.950666 2014-08-13 18:06:06.708650 7f5c973a57a0 20 read_log 192834'18709 (192834'18680) delete b3746981/benchmark_data_storage0111-ib.pocstor_41526_object13610/head//24 by osd.13.0:19645 2014-08-11 15:29:40.963561 2014-08-13 18:06:06.708670 7f5c973a57a0 20 read_log 192834'18710 (192834'18681) delete dd34e981/benchmark_data_storage0111-ib.pocstor_41526_object12250/head//24 by osd.13.0:19646 2014-08-11 15:29:40.965832 2014-08-13 18:06:06.708684 7f5c973a57a0 20 read_log 192834'18736 (192834'18395) clean 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19672 2014-08-11 15:29:46.062186 2014-08-13 18:06:06.708697 7f5c973a57a0 20 read_log 192834'18737 (192834'18379) clean 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19673 2014-08-11 15:29:46.062942 2014-08-13 18:06:06.708713 7f5c973a57a0 20 read_log 192834'18738 (192834'18389) clean 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19674 2014-08-11 15:29:46.063783 2014-08-13 18:06:06.708726 7f5c973a57a0 20 read_log 192834'18739 (192834'18373) clean f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19675 2014-08-11 15:29:46.069817 2014-08-13 18:06:06.708739 7f5c973a57a0 20 read_log 192834'18744 (192834'18420) clean 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19680 2014-08-11 15:29:46.087995 2014-08-13 18:06:06.708756 7f5c973a57a0 20 read_log 192834'18745 (192834'18383) clean 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19681 2014-08-11 15:29:46.089234 2014-08-13 18:06:06.708772 7f5c973a57a0 20 read_log 192834'18766 (192834'18736) delete 79ab4181/benchmark_data_storage0111-ib.pocstor_41526_object16993/head//24 by osd.13.0:19702 2014-08-11 15:29:46.116656 2014-08-13 18:06:06.708784 7f5c973a57a0 20 read_log 192834'18767 (192834'18737) delete 8e3b6181/benchmark_data_storage0111-ib.pocstor_41526_object16240/head//24 by osd.13.0:19703 2014-08-11 15:29:46.118608 2014-08-13 18:06:06.708805 7f5c973a57a0 20 read_log 192834'18768 (192834'18738) delete 5f5bc181/benchmark_data_storage0111-ib.pocstor_41526_object16611/head//24 by osd.13.0:19704 2014-08-11 15:29:46.118910 2014-08-13 18:06:06.708825 7f5c973a57a0 20 read_log 192834'18769 (192834'18739) delete f09b1581/benchmark_data_storage0111-ib.pocstor_41526_object15890/head//24 by osd.13.0:19705 2014-08-11 15:29:46.119858 2014-08-13 18:06:06.708837 7f5c973a57a0 20 read_log 192834'18774 (192834'18744) delete 8ba00981/benchmark_data_storage0111-ib.pocstor_41526_object17824/head//24 by osd.13.0:19710 2014-08-11 15:29:46.124957 2014-08-13 18:06:06.708849 7f5c973a57a0 20 read_log 192834'18775 (192834'18745) delete 1eef0981/benchmark_data_storage0111-ib.pocstor_41526_object16382/head//24 by osd.13.0:19711 2014-08-11 15:29:46.125486 2014-08-13 18:06:06.708865 7f5c973a57a0 20 read_log 192834'18797 (192834'18437) clean 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:19733 2014-08-11 15:29:51.177950 2014-08-13 18:06:06.708887 7f5c973a57a0 20 read_log 192834'18815 (192834'18797) delete 9b955181/benchmark_data_storage0111-ib.pocstor_41526_object19345/head//24 by osd.13.0:19751 2014-08-11 15:29:51.198713 2014-08-13 18:06:06.708903 7f5c973a57a0 20 read_log 192834'18819 (192834'18443) clean 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:19755 2014-08-11 15:29:51.203207 2014-08-13 18:06:06.708924 7f5c973a57a0 20 read_log 192834'18820 (192834'18456) clean 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:19756 2014-08-11 15:29:51.203548 2014-08-13 18:06:06.708939 7f5c973a57a0 20 read_log 192834'18823 (192834'18449) clean 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:19759 2014-08-11 15:29:51.211103 2014-08-13 18:06:06.708951 7f5c973a57a0 20 read_log 192834'18827 (192834'18819) delete 8b3a3581/benchmark_data_storage0111-ib.pocstor_41526_object19814/head//24 by osd.13.0:19763 2014-08-11 15:29:51.216504 2014-08-13 18:06:06.708964 7f5c973a57a0 20 read_log 192834'18828 (192834'18820) delete 5bdfe581/benchmark_data_storage0111-ib.pocstor_41526_object21824/head//24 by osd.13.0:19764 2014-08-11 15:29:51.217678 2014-08-13 18:06:06.708976 7f5c973a57a0 20 read_log 192834'18831 (192834'18823) delete 8c427981/benchmark_data_storage0111-ib.pocstor_41526_object20786/head//24 by osd.13.0:19767 2014-08-11 15:29:51.220434 2014-08-13 18:06:06.708988 7f5c973a57a0 20 read_log 192888'18852 (192888'18852) modify 181/hit_set_24.181_archive_2014-08-11 16:02:27.752096_2014-08-11 16:44:54.931211/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:54.931521 2014-08-13 18:06:06.709001 7f5c973a57a0 20 read_log 192888'18853 (0'0) promote 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by osd.4.0:21378 2014-08-11 16:44:54.937105 2014-08-13 18:06:06.709013 7f5c973a57a0 20 read_log 192888'18854 (192888'18853) modify 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.1315259.0:22 2014-08-11 16:44:54.922103 2014-08-13 18:06:06.709026 7f5c973a57a0 20 read_log 192888'18855 (0'0) promote 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by osd.4.0:21391 2014-08-11 16:44:57.877717 2014-08-13 18:06:06.709038 7f5c973a57a0 20 read_log 192888'18856 (192888'18855) modify 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.1315259.0:682 2014-08-11 16:44:57.867837 2014-08-13 18:06:06.709050 7f5c973a57a0 20 read_log 192888'18859 (0'0) promote 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by osd.4.0:21409 2014-08-11 16:44:59.717900 2014-08-13 18:06:06.709063 7f5c973a57a0 20 read_log 192888'18860 (192888'18859) modify 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.1315259.0:1178 2014-08-11 16:44:59.711004 2014-08-13 18:06:06.709075 7f5c973a57a0 20 read_log 192888'18865 (0'0) promote 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by osd.4.0:21448 2014-08-11 16:45:03.750369 2014-08-13 18:06:06.709087 7f5c973a57a0 20 read_log 192888'18866 (192888'18865) modify 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.1315259.0:2238 2014-08-11 16:45:03.742783 2014-08-13 18:06:06.709099 7f5c973a57a0 20 read_log 192888'18867 (0'0) promote 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by osd.4.0:21462 2014-08-11 16:45:05.394582 2014-08-13 18:06:06.709111 7f5c973a57a0 20 read_log 192888'18868 (192888'18867) modify 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.1315259.0:2679 2014-08-11 16:45:05.387181 2014-08-13 18:06:06.709124 7f5c973a57a0 20 read_log 192888'18871 (0'0) promote 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by osd.4.0:21492 2014-08-11 16:45:10.870313 2014-08-13 18:06:06.709136 7f5c973a57a0 20 read_log 192888'18872 (192888'18871) modify 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.1315259.0:4112 2014-08-11 16:45:10.861190 2014-08-13 18:06:06.709156 7f5c973a57a0 20 read_log 192888'18873 (0'0) promote 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by osd.4.0:21514 2014-08-11 16:45:13.039211 2014-08-13 18:06:06.709177 7f5c973a57a0 20 read_log 192888'18874 (192888'18873) modify 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.1315259.0:4709 2014-08-11 16:45:13.029352 2014-08-13 18:06:06.709190 7f5c973a57a0 20 read_log 192888'18877 (0'0) promote b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by osd.4.0:21540 2014-08-11 16:45:15.437137 2014-08-13 18:06:06.709202 7f5c973a57a0 20 read_log 192888'18878 (192888'18877) modify b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.1315259.0:5353 2014-08-11 16:45:15.429633 2014-08-13 18:06:06.709215 7f5c973a57a0 20 read_log 192888'18879 (0'0) promote 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by osd.4.0:21552 2014-08-11 16:45:16.761594 2014-08-13 18:06:06.709227 7f5c973a57a0 20 read_log 192888'18880 (192888'18879) modify 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.1315259.0:5717 2014-08-11 16:45:16.754026 2014-08-13 18:06:06.709239 7f5c973a57a0 20 read_log 192888'18881 (0'0) promote 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by osd.4.0:21564 2014-08-11 16:45:18.427841 2014-08-13 18:06:06.709251 7f5c973a57a0 20 read_log 192888'18882 (192888'18881) modify 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.1315259.0:6178 2014-08-11 16:45:18.418877 2014-08-13 18:06:06.709264 7f5c973a57a0 20 read_log 192888'18893 (0'0) promote 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by osd.4.0:21668 2014-08-11 16:45:34.518316 2014-08-13 18:06:06.709276 7f5c973a57a0 20 read_log 192888'18894 (192888'18893) modify 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.1315259.0:10476 2014-08-11 16:45:34.507904 2014-08-13 18:06:06.709288 7f5c973a57a0 20 read_log 192888'18897 (0'0) promote e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by osd.4.0:21710 2014-08-11 16:45:40.073139 2014-08-13 18:06:06.709303 7f5c973a57a0 20 read_log 192888'18898 (192888'18897) modify e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.1315259.0:11844 2014-08-11 16:45:40.064915 2014-08-13 18:06:06.709316 7f5c973a57a0 20 read_log 192888'18901 (0'0) promote 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by osd.4.0:21744 2014-08-11 16:45:45.846724 2014-08-13 18:06:06.709329 7f5c973a57a0 20 read_log 192888'18902 (192888'18901) modify 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.1315259.0:13250 2014-08-11 16:45:45.837072 2014-08-13 18:06:06.709344 7f5c973a57a0 20 read_log 192888'18903 (0'0) promote 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by osd.4.0:21768 2014-08-11 16:45:50.726893 2014-08-13 18:06:06.709357 7f5c973a57a0 20 read_log 192888'18904 (192888'18903) modify 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.1315259.0:14508 2014-08-11 16:45:50.717693 2014-08-13 18:06:06.709369 7f5c973a57a0 20 read_log 192888'18907 (0'0) promote 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by osd.4.0:21838 2014-08-11 16:45:59.297974 2014-08-13 18:06:06.709381 7f5c973a57a0 20 read_log 192888'18908 (192888'18907) modify 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.1315259.0:16735 2014-08-11 16:45:59.289249 2014-08-13 18:06:06.709397 7f5c973a57a0 20 read_log 192888'18909 (0'0) promote f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by osd.4.0:21854 2014-08-11 16:46:01.192888 2014-08-13 18:06:06.709409 7f5c973a57a0 20 read_log 192888'18910 (192888'18909) modify f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.1315259.0:17219 2014-08-11 16:46:01.183945 2014-08-13 18:06:06.709431 7f5c973a57a0 20 read_log 192888'18913 (0'0) promote f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by osd.4.0:21864 2014-08-11 16:46:02.793027 2014-08-13 18:06:06.709460 7f5c973a57a0 20 read_log 192888'18914 (192888'18913) modify f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.1315259.0:17603 2014-08-11 16:46:02.781789 2014-08-13 18:06:06.709487 7f5c973a57a0 20 read_log 192888'18915 (0'0) promote 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by osd.4.0:21870 2014-08-11 16:46:03.854827 2014-08-13 18:06:06.709502 7f5c973a57a0 20 read_log 192888'18916 (192888'18915) modify 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.1315259.0:17869 2014-08-11 16:46:03.845744 2014-08-13 18:06:06.709522 7f5c973a57a0 20 read_log 192888'18917 (0'0) promote e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by osd.4.0:21886 2014-08-11 16:46:06.470385 2014-08-13 18:06:06.709538 7f5c973a57a0 20 read_log 192888'18918 (192888'18917) modify e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.1315259.0:18550 2014-08-11 16:46:06.462494 2014-08-13 18:06:06.709551 7f5c973a57a0 20 read_log 192888'18919 (0'0) promote 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by osd.4.0:21890 2014-08-11 16:46:06.607946 2014-08-13 18:06:06.709563 7f5c973a57a0 20 read_log 192888'18920 (192888'18919) modify 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.1315259.0:18585 2014-08-11 16:46:06.600866 2014-08-13 18:06:06.709576 7f5c973a57a0 20 read_log 192888'18921 (0'0) promote ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by osd.4.0:21924 2014-08-11 16:46:10.797472 2014-08-13 18:06:06.709594 7f5c973a57a0 20 read_log 192888'18922 (192888'18921) modify ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.1315259.0:19639 2014-08-11 16:46:10.789479 2014-08-13 18:06:06.709607 7f5c973a57a0 20 read_log 192888'18925 (0'0) promote 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by osd.4.0:21956 2014-08-11 16:46:15.730649 2014-08-13 18:06:06.709623 7f5c973a57a0 20 read_log 192888'18926 (192888'18925) modify 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.1315259.0:20960 2014-08-11 16:46:15.722267 2014-08-13 18:06:06.709638 7f5c973a57a0 20 read_log 192888'18929 (0'0) promote 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by osd.4.0:21970 2014-08-11 16:46:17.118564 2014-08-13 18:06:06.709650 7f5c973a57a0 20 read_log 192888'18930 (192888'18929) modify 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.1315259.0:21333 2014-08-11 16:46:17.110201 2014-08-13 18:06:06.709663 7f5c973a57a0 20 read_log 192888'18935 (0'0) promote d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by osd.4.0:22004 2014-08-11 16:46:21.855538 2014-08-13 18:06:06.709675 7f5c973a57a0 20 read_log 192888'18936 (192888'18935) modify d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.1315259.0:22548 2014-08-11 16:46:21.846745 2014-08-13 18:06:06.709687 7f5c973a57a0 20 read_log 192888'18937 (0'0) promote 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by osd.4.0:22024 2014-08-11 16:46:26.140211 2014-08-13 18:06:06.709699 7f5c973a57a0 20 read_log 192888'18938 (192888'18937) modify 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.1315259.0:23601 2014-08-11 16:46:26.130444 2014-08-13 18:06:06.709711 7f5c973a57a0 20 read_log 192888'18939 (0'0) promote 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by osd.4.0:22028 2014-08-11 16:46:26.643186 2014-08-13 18:06:06.709723 7f5c973a57a0 20 read_log 192888'18940 (192888'18939) modify 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.1315259.0:23736 2014-08-11 16:46:26.632987 2014-08-13 18:06:06.709736 7f5c973a57a0 20 read_log 192888'18943 (0'0) promote 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by osd.4.0:22040 2014-08-11 16:46:28.051821 2014-08-13 18:06:06.709755 7f5c973a57a0 20 read_log 192888'18944 (192888'18943) modify 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.1315259.0:24107 2014-08-11 16:46:28.044596 2014-08-13 18:06:06.709781 7f5c973a57a0 20 read_log 192888'18945 (0'0) promote 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by osd.4.0:22054 2014-08-11 16:46:29.204523 2014-08-13 18:06:06.709794 7f5c973a57a0 20 read_log 192888'18946 (192888'18945) modify 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.1315259.0:24417 2014-08-11 16:46:29.195317 2014-08-13 18:06:06.709807 7f5c973a57a0 20 read_log 192888'18949 (0'0) promote b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by osd.4.0:22080 2014-08-11 16:46:32.005925 2014-08-13 18:06:06.709819 7f5c973a57a0 20 read_log 192888'18950 (192888'18949) modify b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.1315259.0:25130 2014-08-11 16:46:31.996994 2014-08-13 18:06:06.709832 7f5c973a57a0 20 read_log 192888'18953 (0'0) promote 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by osd.4.0:22086 2014-08-11 16:46:32.539382 2014-08-13 18:06:06.709844 7f5c973a57a0 20 read_log 192888'18954 (192888'18953) modify 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.1315259.0:25254 2014-08-11 16:46:32.530900 2014-08-13 18:06:06.709856 7f5c973a57a0 20 read_log 192888'18955 (0'0) promote f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by osd.4.0:22118 2014-08-11 16:46:37.821290 2014-08-13 18:06:06.709868 7f5c973a57a0 20 read_log 192888'18956 (192888'18955) modify f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.1315259.0:26482 2014-08-11 16:46:37.813093 2014-08-13 18:06:06.709880 7f5c973a57a0 20 read_log 192888'18957 (0'0) promote 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by osd.4.0:22126 2014-08-11 16:46:38.452089 2014-08-13 18:06:06.709892 7f5c973a57a0 20 read_log 192888'18958 (192888'18957) modify 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.1315259.0:26643 2014-08-11 16:46:38.442666 2014-08-13 18:06:06.709904 7f5c973a57a0 20 read_log 192888'18959 (0'0) promote 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by osd.4.0:22140 2014-08-11 16:46:41.469562 2014-08-13 18:06:06.709917 7f5c973a57a0 20 read_log 192888'18960 (192888'18959) modify 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.1315259.0:27361 2014-08-11 16:46:41.462625 2014-08-13 18:06:06.709929 7f5c973a57a0 20 read_log 192888'18971 (192888'18854) modify 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.1315259.0:30749 2014-08-11 16:46:55.876395 2014-08-13 18:06:06.709941 7f5c973a57a0 20 read_log 192888'18972 (192888'18856) modify 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.1315259.0:31409 2014-08-11 16:46:56.084993 2014-08-13 18:06:06.709956 7f5c973a57a0 20 read_log 192888'18974 (192888'18860) modify 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.1315259.0:31905 2014-08-11 16:46:56.232884 2014-08-13 18:06:06.709969 7f5c973a57a0 20 read_log 192888'18977 (192888'18866) modify 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.1315259.0:32965 2014-08-11 16:46:56.519248 2014-08-13 18:06:06.709983 7f5c973a57a0 20 read_log 192888'18978 (192888'18868) modify 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.1315259.0:33406 2014-08-11 16:46:56.628896 2014-08-13 18:06:06.709995 7f5c973a57a0 20 read_log 192888'18980 (192888'18872) modify 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.1315259.0:34839 2014-08-11 16:46:56.998787 2014-08-13 18:06:06.710010 7f5c973a57a0 20 read_log 192888'18981 (192888'18874) modify 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.1315259.0:35436 2014-08-11 16:46:57.163755 2014-08-13 18:06:06.710031 7f5c973a57a0 20 read_log 192888'18983 (192888'18878) modify b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.1315259.0:36080 2014-08-11 16:46:57.330117 2014-08-13 18:06:06.710046 7f5c973a57a0 20 read_log 192888'18984 (192888'18880) modify 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.1315259.0:36444 2014-08-11 16:46:57.424640 2014-08-13 18:06:06.710058 7f5c973a57a0 20 read_log 192888'18985 (192888'18882) modify 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.1315259.0:36905 2014-08-11 16:46:57.542387 2014-08-13 18:06:06.710072 7f5c973a57a0 20 read_log 192888'18991 (192888'18894) modify 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.1315259.0:41203 2014-08-11 16:46:58.686202 2014-08-13 18:06:06.710086 7f5c973a57a0 20 read_log 192888'18993 (192888'18898) modify e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.1315259.0:42571 2014-08-11 16:46:59.046820 2014-08-13 18:06:06.710099 7f5c973a57a0 20 read_log 192888'18995 (192888'18902) modify 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.1315259.0:43977 2014-08-11 16:46:59.446683 2014-08-13 18:06:06.710111 7f5c973a57a0 20 read_log 192888'18996 (192888'18904) modify 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.1315259.0:45235 2014-08-11 16:46:59.808334 2014-08-13 18:06:06.710123 7f5c973a57a0 20 read_log 192888'18998 (192888'18908) modify 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.1315259.0:47462 2014-08-11 16:47:00.466027 2014-08-13 18:06:06.710136 7f5c973a57a0 20 read_log 192888'18999 (192888'18910) modify f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.1315259.0:47946 2014-08-11 16:47:00.618540 2014-08-13 18:06:06.710155 7f5c973a57a0 20 read_log 192888'19001 (192888'18914) modify f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.1315259.0:48330 2014-08-11 16:47:00.732050 2014-08-13 18:06:06.710170 7f5c973a57a0 20 read_log 192888'19002 (192888'18916) modify 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.1315259.0:48596 2014-08-11 16:47:00.808301 2014-08-13 18:06:06.710183 7f5c973a57a0 20 read_log 192888'19003 (192888'18918) modify e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.1315259.0:49277 2014-08-11 16:47:01.013446 2014-08-13 18:06:06.710197 7f5c973a57a0 20 read_log 192888'19004 (192888'18920) modify 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.1315259.0:49312 2014-08-11 16:47:01.024216 2014-08-13 18:06:06.710210 7f5c973a57a0 20 read_log 192888'19005 (192888'18922) modify ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.1315259.0:50366 2014-08-11 16:47:01.347761 2014-08-13 18:06:06.710224 7f5c973a57a0 20 read_log 192888'19007 (192888'18926) modify 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.1315259.0:51687 2014-08-11 16:47:01.754335 2014-08-13 18:06:06.710236 7f5c973a57a0 20 read_log 192888'19009 (192888'18930) modify 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.1315259.0:52060 2014-08-11 16:47:01.860336 2014-08-13 18:06:06.710250 7f5c973a57a0 20 read_log 192888'19012 (192888'18936) modify d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.1315259.0:53275 2014-08-11 16:47:02.206715 2014-08-13 18:06:06.710265 7f5c973a57a0 20 read_log 192888'19013 (192888'18938) modify 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.1315259.0:54328 2014-08-11 16:47:02.537066 2014-08-13 18:06:06.710279 7f5c973a57a0 20 read_log 192888'19014 (192888'18940) modify 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.1315259.0:54463 2014-08-11 16:47:02.574040 2014-08-13 18:06:06.710292 7f5c973a57a0 20 read_log 192888'19016 (192888'18944) modify 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.1315259.0:54834 2014-08-11 16:47:02.729468 2014-08-13 18:06:06.710335 7f5c973a57a0 20 read_log 192888'19017 (192888'18946) modify 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.1315259.0:55144 2014-08-11 16:47:02.824600 2014-08-13 18:06:06.710356 7f5c973a57a0 20 read_log 192888'19019 (192888'18950) modify b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.1315259.0:55857 2014-08-11 16:47:03.035764 2014-08-13 18:06:06.710370 7f5c973a57a0 20 read_log 192888'19021 (192888'18954) modify 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.1315259.0:55981 2014-08-11 16:47:03.072397 2014-08-13 18:06:06.710394 7f5c973a57a0 20 read_log 192888'19022 (192888'18956) modify f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.1315259.0:57209 2014-08-11 16:47:03.442791 2014-08-13 18:06:06.710414 7f5c973a57a0 20 read_log 192888'19023 (192888'18958) modify 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.1315259.0:57370 2014-08-11 16:47:03.491842 2014-08-13 18:06:06.710426 7f5c973a57a0 20 read_log 192888'19024 (192888'18960) modify 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.1315259.0:58088 2014-08-11 16:47:03.698029 2014-08-13 18:06:06.710446 7f5c973a57a0 20 read_log 192888'19030 (192888'19030) modify 181/hit_set_24.181_archive_2014-08-11 16:44:54.931606_2014-08-11 17:05:14.856635/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.856830 2014-08-13 18:06:06.710463 7f5c973a57a0 20 read_log 192888'19031 (192888'18852) delete 181/hit_set_24.181_archive_2014-08-11 16:02:27.752096_2014-08-11 16:44:54.931211/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:14.856830 2014-08-13 18:06:06.710475 7f5c973a57a0 20 read_log 192888'19034 (0'0) promote 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by osd.4.0:22653 2014-08-11 17:05:16.000765 2014-08-13 18:06:06.710499 7f5c973a57a0 20 read_log 192888'19035 (192888'19034) modify 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.942868.0:245 2014-08-11 17:05:15.993154 2014-08-13 18:06:06.710514 7f5c973a57a0 20 read_log 192888'19036 (0'0) promote e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by osd.4.0:22655 2014-08-11 17:05:17.182423 2014-08-13 18:06:06.710527 7f5c973a57a0 20 read_log 192888'19037 (192888'19036) modify e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.942868.0:524 2014-08-11 17:05:17.172611 2014-08-13 18:06:06.710562 7f5c973a57a0 20 read_log 192888'19040 (0'0) promote 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by osd.4.0:22681 2014-08-11 17:05:20.290443 2014-08-13 18:06:06.710578 7f5c973a57a0 20 read_log 192888'19041 (192888'19040) modify 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.942868.0:1283 2014-08-11 17:05:20.282570 2014-08-13 18:06:06.710592 7f5c973a57a0 20 read_log 192888'19042 (0'0) promote 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by osd.4.0:22687 2014-08-11 17:05:21.442398 2014-08-13 18:06:06.710604 7f5c973a57a0 20 read_log 192888'19043 (192888'19042) modify 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.942868.0:1564 2014-08-11 17:05:21.434083 2014-08-13 18:06:06.710617 7f5c973a57a0 20 read_log 192888'19046 (0'0) promote dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by osd.4.0:22726 2014-08-11 17:05:27.832090 2014-08-13 18:06:06.710629 7f5c973a57a0 20 read_log 192888'19047 (192888'19046) modify dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.942868.0:3213 2014-08-11 17:05:27.823106 2014-08-13 18:06:06.710651 7f5c973a57a0 20 read_log 192888'19048 (0'0) promote a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by osd.4.0:22742 2014-08-11 17:05:29.289712 2014-08-13 18:06:06.710676 7f5c973a57a0 20 read_log 192888'19049 (192888'19048) modify a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.942868.0:3607 2014-08-11 17:05:29.280803 2014-08-13 18:06:06.710691 7f5c973a57a0 20 read_log 192888'19050 (0'0) promote 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by osd.4.0:22750 2014-08-11 17:05:29.994874 2014-08-13 18:06:06.710703 7f5c973a57a0 20 read_log 192888'19051 (192888'19050) modify 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.942868.0:3798 2014-08-11 17:05:29.986171 2014-08-13 18:06:06.710716 7f5c973a57a0 20 read_log 192888'19060 (0'0) promote 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by osd.4.0:22844 2014-08-11 17:05:41.863144 2014-08-13 18:06:06.710728 7f5c973a57a0 20 read_log 192888'19061 (192888'19060) modify 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.942868.0:6959 2014-08-11 17:05:41.853656 2014-08-13 18:06:06.710741 7f5c973a57a0 20 read_log 192888'19066 (0'0) promote e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by osd.4.0:22862 2014-08-11 17:05:44.915316 2014-08-13 18:06:06.710753 7f5c973a57a0 20 read_log 192888'19067 (192888'19066) modify e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.942868.0:7747 2014-08-11 17:05:44.906210 2014-08-13 18:06:06.710765 7f5c973a57a0 20 read_log 192888'19070 (0'0) promote 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by osd.4.0:22882 2014-08-11 17:05:47.640604 2014-08-13 18:06:06.710780 7f5c973a57a0 20 read_log 192888'19071 (192888'19070) modify 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.942868.0:8502 2014-08-11 17:05:47.632050 2014-08-13 18:06:06.710793 7f5c973a57a0 20 read_log 192888'19072 (0'0) promote 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by osd.4.0:22894 2014-08-11 17:05:49.989404 2014-08-13 18:06:06.710805 7f5c973a57a0 20 read_log 192888'19073 (192888'19072) modify 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.942868.0:9130 2014-08-11 17:05:49.980541 2014-08-13 18:06:06.710817 7f5c973a57a0 20 read_log 192888'19074 (0'0) promote fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by osd.4.0:22904 2014-08-11 17:05:52.544037 2014-08-13 18:06:06.710829 7f5c973a57a0 20 read_log 192888'19075 (192888'19074) modify fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.942868.0:9817 2014-08-11 17:05:52.535766 2014-08-13 18:06:06.710842 7f5c973a57a0 20 read_log 192888'19080 (0'0) promote fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by osd.4.0:22916 2014-08-11 17:05:54.105014 2014-08-13 18:06:06.710854 7f5c973a57a0 20 read_log 192888'19081 (192888'19080) modify fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.942868.0:10249 2014-08-11 17:05:54.095545 2014-08-13 18:06:06.710868 7f5c973a57a0 20 read_log 192888'19088 (0'0) promote 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by osd.4.0:22956 2014-08-11 17:06:00.324855 2014-08-13 18:06:06.710883 7f5c973a57a0 20 read_log 192888'19089 (192888'19088) modify 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.942868.0:11947 2014-08-11 17:06:00.312616 2014-08-13 18:06:06.710903 7f5c973a57a0 20 read_log 192888'19092 (0'0) promote 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by osd.4.0:22982 2014-08-11 17:06:04.329890 2014-08-13 18:06:06.710919 7f5c973a57a0 20 read_log 192888'19093 (192888'19092) modify 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.942868.0:13038 2014-08-11 17:06:04.320975 2014-08-13 18:06:06.710932 7f5c973a57a0 20 read_log 192888'19094 (0'0) promote 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by osd.4.0:22996 2014-08-11 17:06:05.543846 2014-08-13 18:06:06.710966 7f5c973a57a0 20 read_log 192888'19095 (192888'19094) modify 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.942868.0:13358 2014-08-11 17:06:05.534696 2014-08-13 18:06:06.710983 7f5c973a57a0 20 read_log 192888'19096 (0'0) promote 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by osd.4.0:23002 2014-08-11 17:06:06.175031 2014-08-13 18:06:06.711003 7f5c973a57a0 20 read_log 192888'19097 (192888'19096) modify 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.942868.0:13502 2014-08-11 17:06:06.165502 2014-08-13 18:06:06.711049 7f5c973a57a0 20 read_log 192888'19098 (0'0) promote 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by osd.4.0:23004 2014-08-11 17:06:06.332082 2014-08-13 18:06:06.711065 7f5c973a57a0 20 read_log 192888'19099 (192888'19098) modify 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.942868.0:13542 2014-08-11 17:06:06.323761 2014-08-13 18:06:06.711078 7f5c973a57a0 20 read_log 192888'19102 (0'0) promote 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by osd.4.0:23042 2014-08-11 17:06:14.095190 2014-08-13 18:06:06.711097 7f5c973a57a0 20 read_log 192888'19103 (192888'19102) modify 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.942868.0:15462 2014-08-11 17:06:14.015989 2014-08-13 18:06:06.711114 7f5c973a57a0 20 read_log 192888'19104 (0'0) promote e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by osd.4.0:23046 2014-08-11 17:06:14.230488 2014-08-13 18:06:06.711127 7f5c973a57a0 20 read_log 192888'19105 (192888'19104) modify e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.942868.0:15514 2014-08-11 17:06:14.219954 2014-08-13 18:06:06.711139 7f5c973a57a0 20 read_log 192888'19106 (0'0) promote 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by osd.4.0:23050 2014-08-11 17:06:14.619314 2014-08-13 18:06:06.711176 7f5c973a57a0 20 read_log 192888'19107 (192888'19106) modify 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.942868.0:15615 2014-08-11 17:06:14.609285 2014-08-13 18:06:06.711192 7f5c973a57a0 20 read_log 192888'19110 (0'0) promote ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by osd.4.0:23084 2014-08-11 17:06:20.522440 2014-08-13 18:06:06.711205 7f5c973a57a0 20 read_log 192888'19111 (192888'19110) modify ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.942868.0:17122 2014-08-11 17:06:20.513004 2014-08-13 18:06:06.711217 7f5c973a57a0 20 read_log 192888'19114 (0'0) promote 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by osd.4.0:23118 2014-08-11 17:06:23.715555 2014-08-13 18:06:06.711232 7f5c973a57a0 20 read_log 192888'19115 (192888'19114) modify 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.942868.0:17969 2014-08-11 17:06:23.707330 2014-08-13 18:06:06.711247 7f5c973a57a0 20 read_log 192888'19116 (0'0) promote 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by osd.4.0:23128 2014-08-11 17:06:26.323103 2014-08-13 18:06:06.711261 7f5c973a57a0 20 read_log 192888'19117 (192888'19116) modify 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.942868.0:18659 2014-08-11 17:06:26.315017 2014-08-13 18:06:06.711274 7f5c973a57a0 20 read_log 192888'19118 (0'0) promote efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by osd.4.0:23132 2014-08-11 17:06:26.803553 2014-08-13 18:06:06.711286 7f5c973a57a0 20 read_log 192888'19119 (192888'19118) modify efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.942868.0:18788 2014-08-11 17:06:26.794709 2014-08-13 18:06:06.711298 7f5c973a57a0 20 read_log 192888'19120 (0'0) promote e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by osd.4.0:23148 2014-08-11 17:06:29.495322 2014-08-13 18:06:06.711324 7f5c973a57a0 20 read_log 192888'19121 (192888'19120) modify e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.942868.0:19506 2014-08-11 17:06:29.487098 2014-08-13 18:06:06.711340 7f5c973a57a0 20 read_log 192888'19122 (0'0) promote 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by osd.4.0:23190 2014-08-11 17:06:35.477560 2014-08-13 18:06:06.711363 7f5c973a57a0 20 read_log 192888'19123 (192888'19122) modify 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.942868.0:21075 2014-08-11 17:06:35.469246 2014-08-13 18:06:06.711377 7f5c973a57a0 20 read_log 192888'19126 (0'0) promote 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by osd.4.0:23204 2014-08-11 17:06:37.303695 2014-08-13 18:06:06.711389 7f5c973a57a0 20 read_log 192888'19127 (192888'19126) modify 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.942868.0:21551 2014-08-11 17:06:37.293758 2014-08-13 18:06:06.711401 7f5c973a57a0 20 read_log 192888'19130 (0'0) promote 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by osd.4.0:23254 2014-08-11 17:06:43.234463 2014-08-13 18:06:06.711413 7f5c973a57a0 20 read_log 192888'19131 (192888'19130) modify 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.942868.0:23073 2014-08-11 17:06:43.225830 2014-08-13 18:06:06.711426 7f5c973a57a0 20 read_log 192888'19140 (0'0) promote e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by osd.4.0:23308 2014-08-11 17:06:49.278156 2014-08-13 18:06:06.711438 7f5c973a57a0 20 read_log 192888'19141 (192888'19140) modify e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.942868.0:24693 2014-08-11 17:06:49.269414 2014-08-13 18:06:06.711450 7f5c973a57a0 20 read_log 192888'19146 (0'0) promote d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by osd.4.0:23352 2014-08-11 17:06:55.227311 2014-08-13 18:06:06.711463 7f5c973a57a0 20 read_log 192888'19147 (192888'19146) modify d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.942868.0:26205 2014-08-11 17:06:55.218805 2014-08-13 18:06:06.711475 7f5c973a57a0 20 read_log 192888'19148 (0'0) promote 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by osd.4.0:23354 2014-08-11 17:06:56.487206 2014-08-13 18:06:06.711487 7f5c973a57a0 20 read_log 192888'19149 (192888'19148) modify 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.942868.0:26546 2014-08-11 17:06:56.477792 2014-08-13 18:06:06.711500 7f5c973a57a0 20 read_log 192888'19152 (0'0) promote d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by osd.4.0:23402 2014-08-11 17:07:03.633151 2014-08-13 18:06:06.711512 7f5c973a57a0 20 read_log 192888'19153 (192888'19152) modify d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.942868.0:28448 2014-08-11 17:07:03.625720 2014-08-13 18:06:06.711524 7f5c973a57a0 20 read_log 192888'19154 (0'0) promote 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by osd.4.0:23418 2014-08-11 17:07:06.402141 2014-08-13 18:06:06.711536 7f5c973a57a0 20 read_log 192888'19155 (192888'19154) modify 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.942868.0:29164 2014-08-11 17:07:06.391625 2014-08-13 18:06:06.711549 7f5c973a57a0 20 read_log 192888'19156 (0'0) promote be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by osd.4.0:23440 2014-08-11 17:07:10.267519 2014-08-13 18:06:06.711561 7f5c973a57a0 20 read_log 192888'19157 (192888'19156) modify be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.942868.0:30107 2014-08-11 17:07:10.256861 2014-08-13 18:06:06.711573 7f5c973a57a0 20 read_log 192888'19160 (0'0) promote e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by osd.4.0:23478 2014-08-11 17:07:13.780629 2014-08-13 18:06:06.711593 7f5c973a57a0 20 read_log 192888'19161 (192888'19160) modify e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.942868.0:30994 2014-08-11 17:07:13.772385 2014-08-13 18:06:06.711608 7f5c973a57a0 20 read_log 192888'19163 (192888'19035) modify 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.942868.0:31511 2014-08-11 17:07:15.949001 2014-08-13 18:06:06.711621 7f5c973a57a0 20 read_log 192888'19164 (192888'19037) modify e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.942868.0:31790 2014-08-11 17:07:16.058110 2014-08-13 18:06:06.711633 7f5c973a57a0 20 read_log 192888'19166 (192888'19041) modify 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.942868.0:32549 2014-08-11 17:07:16.284983 2014-08-13 18:06:06.711645 7f5c973a57a0 20 read_log 192888'19167 (192888'19043) modify 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.942868.0:32830 2014-08-11 17:07:16.355709 2014-08-13 18:06:06.711658 7f5c973a57a0 20 read_log 192888'19169 (192888'19047) modify dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.942868.0:34479 2014-08-11 17:07:16.801692 2014-08-13 18:06:06.711670 7f5c973a57a0 20 read_log 192888'19170 (192888'19049) modify a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.942868.0:34873 2014-08-11 17:07:16.919664 2014-08-13 18:06:06.711684 7f5c973a57a0 20 read_log 192888'19171 (192888'19051) modify 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.942868.0:35064 2014-08-11 17:07:16.975715 2014-08-13 18:06:06.711697 7f5c973a57a0 20 read_log 192888'19176 (192888'19061) modify 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.942868.0:38225 2014-08-11 17:07:17.826568 2014-08-13 18:06:06.711712 7f5c973a57a0 20 read_log 192888'19179 (192888'19067) modify e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.942868.0:39013 2014-08-11 17:07:18.055422 2014-08-13 18:06:06.711724 7f5c973a57a0 20 read_log 192888'19181 (192888'19071) modify 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.942868.0:39768 2014-08-11 17:07:18.277904 2014-08-13 18:06:06.711738 7f5c973a57a0 20 read_log 192888'19182 (192888'19073) modify 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.942868.0:40396 2014-08-11 17:07:18.439382 2014-08-13 18:06:06.711751 7f5c973a57a0 20 read_log 192888'19183 (192888'19075) modify fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.942868.0:41083 2014-08-11 17:07:18.634662 2014-08-13 18:06:06.711765 7f5c973a57a0 20 read_log 192888'19186 (192888'19081) modify fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.942868.0:41515 2014-08-11 17:07:18.749169 2014-08-13 18:06:06.711778 7f5c973a57a0 20 read_log 192888'19190 (192888'19089) modify 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.942868.0:43213 2014-08-11 17:07:19.209757 2014-08-13 18:06:06.711792 7f5c973a57a0 20 read_log 192888'19192 (192888'19093) modify 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.942868.0:44304 2014-08-11 17:07:19.529156 2014-08-13 18:06:06.711804 7f5c973a57a0 20 read_log 192888'19193 (192888'19095) modify 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.942868.0:44624 2014-08-11 17:07:19.618360 2014-08-13 18:06:06.711819 7f5c973a57a0 20 read_log 192888'19194 (192888'19097) modify 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.942868.0:44768 2014-08-11 17:07:19.657375 2014-08-13 18:06:06.711833 7f5c973a57a0 20 read_log 192888'19195 (192888'19099) modify 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.942868.0:44808 2014-08-11 17:07:19.668315 2014-08-13 18:06:06.711847 7f5c973a57a0 20 read_log 192888'19197 (192888'19103) modify 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.942868.0:46728 2014-08-11 17:07:20.203299 2014-08-13 18:06:06.711869 7f5c973a57a0 20 read_log 192888'19198 (192888'19105) modify e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.942868.0:46780 2014-08-11 17:07:20.216609 2014-08-13 18:06:06.711884 7f5c973a57a0 20 read_log 192888'19199 (192888'19107) modify 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.942868.0:46881 2014-08-11 17:07:20.243783 2014-08-13 18:06:06.711896 7f5c973a57a0 20 read_log 192888'19201 (192888'19111) modify ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.942868.0:48388 2014-08-11 17:07:20.705306 2014-08-13 18:06:06.711909 7f5c973a57a0 20 read_log 192888'19203 (192888'19115) modify 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.942868.0:49235 2014-08-11 17:07:20.950348 2014-08-13 18:06:06.711921 7f5c973a57a0 20 read_log 192888'19204 (192888'19117) modify 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.942868.0:49925 2014-08-11 17:07:21.164554 2014-08-13 18:06:06.711935 7f5c973a57a0 20 read_log 192888'19205 (192888'19119) modify efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.942868.0:50054 2014-08-11 17:07:21.201692 2014-08-13 18:06:06.711948 7f5c973a57a0 20 read_log 192888'19206 (192888'19121) modify e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.942868.0:50772 2014-08-11 17:07:21.426146 2014-08-13 18:06:06.711962 7f5c973a57a0 20 read_log 192888'19207 (192888'19123) modify 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.942868.0:52341 2014-08-11 17:07:21.908353 2014-08-13 18:06:06.711975 7f5c973a57a0 20 read_log 192888'19209 (192888'19127) modify 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.942868.0:52817 2014-08-11 17:07:22.059200 2014-08-13 18:06:06.711989 7f5c973a57a0 20 read_log 192888'19211 (192888'19131) modify 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.942868.0:54339 2014-08-11 17:07:22.529625 2014-08-13 18:06:06.712003 7f5c973a57a0 20 read_log 192888'19216 (192888'19141) modify e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.942868.0:55959 2014-08-11 17:07:23.009373 2014-08-13 18:06:06.712017 7f5c973a57a0 20 read_log 192888'19219 (192888'19147) modify d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.942868.0:57471 2014-08-11 17:07:23.465636 2014-08-13 18:06:06.712030 7f5c973a57a0 20 read_log 192888'19220 (192888'19149) modify 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.942868.0:57812 2014-08-11 17:07:23.561913 2014-08-13 18:06:06.712044 7f5c973a57a0 20 read_log 192888'19222 (192888'19153) modify d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.942868.0:59714 2014-08-11 17:07:24.127237 2014-08-13 18:06:06.712057 7f5c973a57a0 20 read_log 192888'19223 (192888'19155) modify 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.942868.0:60430 2014-08-11 17:07:24.337643 2014-08-13 18:06:06.712071 7f5c973a57a0 20 read_log 192888'19224 (192888'19157) modify be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.942868.0:61373 2014-08-11 17:07:24.620557 2014-08-13 18:06:06.712083 7f5c973a57a0 20 read_log 192888'19226 (192888'19161) modify e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.942868.0:62260 2014-08-11 17:07:24.897906 2014-08-13 18:06:06.712098 7f5c973a57a0 20 read_log 193305'19227 (193305'19227) modify 181/hit_set_24.181_archive_2014-08-11 18:37:04.912085_2014-08-11 18:42:37.391795/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.392158 2014-08-13 18:06:06.712112 7f5c973a57a0 20 read_log 193305'19228 (192888'19030) delete 181/hit_set_24.181_archive_2014-08-11 16:44:54.931606_2014-08-11 17:05:14.856635/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.392158 2014-08-13 18:06:06.712131 7f5c973a57a0 20 read_log 193305'19229 (0'0) promote 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by osd.4.0:7 2014-08-11 18:42:37.414235 2014-08-13 18:06:06.712150 7f5c973a57a0 20 read_log 193305'19230 (193305'19229) modify 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.951337.0:1116 2014-08-11 18:42:37.386510 2014-08-13 18:06:06.712166 7f5c973a57a0 20 read_log 193305'19231 (0'0) promote 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by osd.4.0:65 2014-08-11 18:42:50.623266 2014-08-13 18:06:06.712178 7f5c973a57a0 20 read_log 193305'19232 (193305'19231) modify 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.951337.0:4701 2014-08-11 18:42:50.615171 2014-08-13 18:06:06.712191 7f5c973a57a0 20 read_log 193305'19233 (0'0) promote 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by osd.4.0:103 2014-08-11 18:42:56.888876 2014-08-13 18:06:06.712203 7f5c973a57a0 20 read_log 193305'19234 (193305'19233) modify 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.951337.0:6403 2014-08-11 18:42:56.880978 2014-08-13 18:06:06.712215 7f5c973a57a0 20 read_log 193305'19235 (0'0) promote be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by osd.4.0:117 2014-08-11 18:42:59.752968 2014-08-13 18:06:06.712229 7f5c973a57a0 20 read_log 193305'19236 (193305'19235) modify be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.951337.0:7196 2014-08-11 18:42:59.745109 2014-08-13 18:06:06.712242 7f5c973a57a0 20 read_log 193305'19237 (0'0) promote 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by osd.4.0:141 2014-08-11 18:43:04.570027 2014-08-13 18:06:06.712256 7f5c973a57a0 20 read_log 193305'19238 (193305'19237) modify 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.951337.0:8456 2014-08-11 18:43:04.557561 2014-08-13 18:06:06.712268 7f5c973a57a0 20 read_log 193305'19239 (0'0) promote 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by osd.4.0:143 2014-08-11 18:43:04.647499 2014-08-13 18:06:06.712283 7f5c973a57a0 20 read_log 193305'19240 (193305'19239) modify 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.951337.0:8479 2014-08-11 18:43:04.639167 2014-08-13 18:06:06.712295 7f5c973a57a0 20 read_log 193305'19241 (0'0) promote 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by osd.4.0:145 2014-08-11 18:43:06.730763 2014-08-13 18:06:06.712312 7f5c973a57a0 20 read_log 193305'19242 (193305'19241) modify 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.951337.0:9026 2014-08-11 18:43:06.722341 2014-08-13 18:06:06.712327 7f5c973a57a0 20 read_log 193305'19243 (0'0) promote 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by osd.4.0:149 2014-08-11 18:43:07.117839 2014-08-13 18:06:06.712340 7f5c973a57a0 20 read_log 193305'19244 (193305'19243) modify 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.951337.0:9134 2014-08-11 18:43:07.109874 2014-08-13 18:06:06.712354 7f5c973a57a0 20 read_log 193305'19245 (0'0) promote 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by osd.4.0:163 2014-08-11 18:43:10.317581 2014-08-13 18:06:06.712372 7f5c973a57a0 20 read_log 193305'19246 (193305'19245) modify 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.951337.0:10006 2014-08-11 18:43:10.309378 2014-08-13 18:06:06.712385 7f5c973a57a0 20 read_log 193305'19247 (0'0) promote 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by osd.4.0:179 2014-08-11 18:43:13.923431 2014-08-13 18:06:06.712399 7f5c973a57a0 20 read_log 193305'19248 (193305'19247) modify 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.951337.0:10977 2014-08-11 18:43:13.914869 2014-08-13 18:06:06.712422 7f5c973a57a0 20 read_log 193305'19249 (0'0) promote eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by osd.4.0:197 2014-08-11 18:43:16.816858 2014-08-13 18:06:06.712437 7f5c973a57a0 20 read_log 193305'19250 (193305'19249) modify eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.951337.0:11709 2014-08-11 18:43:16.809120 2014-08-13 18:06:06.712462 7f5c973a57a0 20 read_log 193305'19251 (0'0) promote 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by osd.4.0:243 2014-08-11 18:43:29.536038 2014-08-13 18:06:06.712475 7f5c973a57a0 20 read_log 193305'19252 (193305'19251) modify 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.951337.0:15169 2014-08-11 18:43:29.529550 2014-08-13 18:06:06.712513 7f5c973a57a0 20 read_log 193305'19253 (0'0) promote 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by osd.4.0:255 2014-08-11 18:43:32.251220 2014-08-13 18:06:06.712529 7f5c973a57a0 20 read_log 193305'19254 (193305'19253) modify 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.951337.0:15965 2014-08-11 18:43:32.242778 2014-08-13 18:06:06.712548 7f5c973a57a0 20 read_log 193305'19255 (0'0) promote 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by osd.4.0:259 2014-08-11 18:43:32.910096 2014-08-13 18:06:06.712560 7f5c973a57a0 20 read_log 193305'19256 (193305'19255) modify 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.951337.0:16148 2014-08-11 18:43:32.902502 2014-08-13 18:06:06.712575 7f5c973a57a0 20 read_log 193305'19257 (0'0) promote 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by osd.4.0:273 2014-08-11 18:43:34.231520 2014-08-13 18:06:06.712587 7f5c973a57a0 20 read_log 193305'19258 (193305'19257) modify 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.951337.0:16511 2014-08-11 18:43:34.222243 2014-08-13 18:06:06.712602 7f5c973a57a0 20 read_log 193305'19259 (0'0) promote 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by osd.4.0:277 2014-08-11 18:43:35.004791 2014-08-13 18:06:06.712614 7f5c973a57a0 20 read_log 193305'19260 (193305'19259) modify 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.951337.0:16744 2014-08-11 18:43:34.996459 2014-08-13 18:06:06.712627 7f5c973a57a0 20 read_log 193305'19261 (0'0) promote c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by osd.4.0:295 2014-08-11 18:43:37.947641 2014-08-13 18:06:06.712639 7f5c973a57a0 20 read_log 193305'19262 (193305'19261) modify c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.951337.0:17597 2014-08-11 18:43:37.940973 2014-08-13 18:06:06.712660 7f5c973a57a0 20 read_log 193305'19263 (0'0) promote fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by osd.4.0:335 2014-08-11 18:43:46.701167 2014-08-13 18:06:06.712675 7f5c973a57a0 20 read_log 193305'19264 (193305'19263) modify fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.951337.0:20021 2014-08-11 18:43:46.692250 2014-08-13 18:06:06.712688 7f5c973a57a0 20 read_log 193305'19265 (0'0) promote 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by osd.4.0:351 2014-08-11 18:43:49.877673 2014-08-13 18:06:06.712710 7f5c973a57a0 20 read_log 193305'19266 (193305'19265) modify 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.951337.0:20911 2014-08-11 18:43:49.869894 2014-08-13 18:06:06.712725 7f5c973a57a0 20 read_log 193305'19267 (0'0) promote 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by osd.4.0:359 2014-08-11 18:43:52.549222 2014-08-13 18:06:06.712738 7f5c973a57a0 20 read_log 193305'19268 (193305'19267) modify 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.951337.0:21662 2014-08-11 18:43:52.541745 2014-08-13 18:06:06.712761 7f5c973a57a0 20 read_log 193305'19269 (0'0) promote 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by osd.4.0:381 2014-08-11 18:43:55.936000 2014-08-13 18:06:06.712776 7f5c973a57a0 20 read_log 193305'19270 (193305'19269) modify 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.951337.0:22578 2014-08-11 18:43:55.925642 2014-08-13 18:06:06.712789 7f5c973a57a0 20 read_log 193305'19271 (0'0) promote 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by osd.4.0:407 2014-08-11 18:44:03.019451 2014-08-13 18:06:06.712801 7f5c973a57a0 20 read_log 193305'19272 (193305'19271) modify 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.951337.0:24549 2014-08-11 18:44:03.011034 2014-08-13 18:06:06.712814 7f5c973a57a0 20 read_log 193305'19273 (0'0) promote b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by osd.4.0:427 2014-08-11 18:44:08.640304 2014-08-13 18:06:06.712826 7f5c973a57a0 20 read_log 193305'19274 (193305'19273) modify b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.951337.0:26063 2014-08-11 18:44:08.632247 2014-08-13 18:06:06.712838 7f5c973a57a0 20 read_log 193305'19275 (0'0) promote c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by osd.4.0:435 2014-08-11 18:44:10.359018 2014-08-13 18:06:06.712851 7f5c973a57a0 20 read_log 193305'19276 (193305'19275) modify c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.951337.0:26548 2014-08-11 18:44:10.352627 2014-08-13 18:06:06.712863 7f5c973a57a0 20 read_log 193305'19277 (0'0) promote 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by osd.4.0:449 2014-08-11 18:44:11.539208 2014-08-13 18:06:06.712875 7f5c973a57a0 20 read_log 193305'19278 (193305'19277) modify 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.951337.0:26875 2014-08-11 18:44:11.528771 2014-08-13 18:06:06.712887 7f5c973a57a0 20 read_log 193305'19279 (0'0) promote c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by osd.4.0:451 2014-08-11 18:44:12.518839 2014-08-13 18:06:06.712899 7f5c973a57a0 20 read_log 193305'19280 (193305'19279) modify c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.951337.0:27124 2014-08-11 18:44:12.510131 2014-08-13 18:06:06.712912 7f5c973a57a0 20 read_log 193305'19281 (0'0) promote 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by osd.4.0:513 2014-08-11 18:44:28.776138 2014-08-13 18:06:06.712933 7f5c973a57a0 20 read_log 193305'19282 (193305'19281) modify 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.951337.0:31562 2014-08-11 18:44:28.766694 2014-08-13 18:06:06.712948 7f5c973a57a0 20 read_log 193305'19283 (0'0) promote d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by osd.4.0:515 2014-08-11 18:44:29.296022 2014-08-13 18:06:06.712961 7f5c973a57a0 20 read_log 193305'19284 (193305'19283) modify d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.951337.0:31712 2014-08-11 18:44:29.287665 2014-08-13 18:06:06.712973 7f5c973a57a0 20 read_log 193305'19285 (0'0) promote 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by osd.4.0:525 2014-08-11 18:44:30.070715 2014-08-13 18:06:06.712986 7f5c973a57a0 20 read_log 193305'19286 (193305'19285) modify 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.951337.0:31940 2014-08-11 18:44:30.061779 2014-08-13 18:06:06.713000 7f5c973a57a0 20 read_log 193305'19287 (193305'19230) modify 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.951337.0:33827 2014-08-11 18:44:34.672154 2014-08-13 18:06:06.713013 7f5c973a57a0 20 read_log 193305'19288 (193305'19232) modify 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.951337.0:37412 2014-08-11 18:44:36.899211 2014-08-13 18:06:06.713034 7f5c973a57a0 20 read_log 193305'19289 (193305'19234) modify 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.951337.0:39114 2014-08-11 18:44:37.831940 2014-08-13 18:06:06.713050 7f5c973a57a0 20 read_log 193305'19290 (193305'19236) modify be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.951337.0:39907 2014-08-11 18:44:38.223729 2014-08-13 18:06:06.713062 7f5c973a57a0 20 read_log 193305'19291 (193305'19238) modify 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.951337.0:41167 2014-08-11 18:44:38.655216 2014-08-13 18:06:06.713075 7f5c973a57a0 20 read_log 193305'19292 (193305'19240) modify 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.951337.0:41190 2014-08-11 18:44:38.662732 2014-08-13 18:06:06.713087 7f5c973a57a0 20 read_log 193305'19293 (193305'19242) modify 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.951337.0:41737 2014-08-11 18:44:38.853048 2014-08-13 18:06:06.713099 7f5c973a57a0 20 read_log 193305'19294 (193305'19244) modify 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.951337.0:41845 2014-08-11 18:44:38.891962 2014-08-13 18:06:06.713113 7f5c973a57a0 20 read_log 193305'19295 (193305'19246) modify 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.951337.0:42717 2014-08-11 18:44:39.360400 2014-08-13 18:06:06.713126 7f5c973a57a0 20 read_log 193305'19296 (193305'19248) modify 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.951337.0:43688 2014-08-11 18:44:39.666300 2014-08-13 18:06:06.713140 7f5c973a57a0 20 read_log 193305'19297 (193305'19250) modify eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.951337.0:44420 2014-08-11 18:44:39.876758 2014-08-13 18:06:06.713167 7f5c973a57a0 20 read_log 193305'19298 (193305'19252) modify 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.951337.0:47880 2014-08-11 18:44:40.891796 2014-08-13 18:06:06.713181 7f5c973a57a0 20 read_log 193305'19299 (193305'19254) modify 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.951337.0:48676 2014-08-11 18:44:41.121410 2014-08-13 18:06:06.713196 7f5c973a57a0 20 read_log 193305'19300 (193305'19256) modify 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.951337.0:48859 2014-08-11 18:44:41.177409 2014-08-13 18:06:06.713214 7f5c973a57a0 20 read_log 193305'19301 (193305'19258) modify 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.951337.0:49222 2014-08-11 18:44:41.298585 2014-08-13 18:06:06.713227 7f5c973a57a0 20 read_log 193305'19302 (193305'19260) modify 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.951337.0:49455 2014-08-11 18:44:41.370780 2014-08-13 18:06:06.713240 7f5c973a57a0 20 read_log 193305'19303 (193305'19262) modify c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.951337.0:50308 2014-08-11 18:44:41.628174 2014-08-13 18:06:06.713254 7f5c973a57a0 20 read_log 193305'19304 (193305'19264) modify fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.951337.0:52732 2014-08-11 18:44:42.321549 2014-08-13 18:06:06.713267 7f5c973a57a0 20 read_log 193305'19305 (193305'19266) modify 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.951337.0:53622 2014-08-11 18:44:42.584211 2014-08-13 18:06:06.713282 7f5c973a57a0 20 read_log 193305'19306 (193305'19268) modify 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.951337.0:54373 2014-08-11 18:44:42.813535 2014-08-13 18:06:06.713297 7f5c973a57a0 20 read_log 193305'19307 (193305'19270) modify 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.951337.0:55289 2014-08-11 18:44:43.079263 2014-08-13 18:06:06.713309 7f5c973a57a0 20 read_log 193305'19308 (193305'19272) modify 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.951337.0:57260 2014-08-11 18:44:43.693702 2014-08-13 18:06:06.713329 7f5c973a57a0 20 read_log 193305'19309 (193305'19274) modify b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.951337.0:58774 2014-08-11 18:44:44.138463 2014-08-13 18:06:06.713344 7f5c973a57a0 20 read_log 193305'19310 (193305'19276) modify c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.951337.0:59259 2014-08-11 18:44:44.275811 2014-08-13 18:06:06.713356 7f5c973a57a0 20 read_log 193305'19311 (193305'19278) modify 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.951337.0:59586 2014-08-11 18:44:44.370047 2014-08-13 18:06:06.713369 7f5c973a57a0 20 read_log 193305'19312 (193305'19280) modify c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.951337.0:59835 2014-08-11 18:44:44.440651 2014-08-13 18:06:06.713381 7f5c973a57a0 20 read_log 193305'19313 (193305'19282) modify 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.951337.0:64273 2014-08-11 18:44:45.753606 2014-08-13 18:06:06.713393 7f5c973a57a0 20 read_log 193305'19314 (193305'19284) modify d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.951337.0:64423 2014-08-11 18:44:45.802345 2014-08-13 18:06:06.713413 7f5c973a57a0 20 read_log 193305'19315 (193305'19286) modify 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.951337.0:64651 2014-08-11 18:44:45.880380 2014-08-13 18:06:06.713427 7f5c973a57a0 20 read_log 194215'19316 (194215'19316) modify 181/hit_set_24.181_archive_2014-08-12 15:32:25.863362_2014-08-12 15:42:02.778991/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:42:02.779435 2014-08-13 18:06:06.713442 7f5c973a57a0 20 read_log 194215'19317 (193305'19227) delete 181/hit_set_24.181_archive_2014-08-11 18:37:04.912085_2014-08-11 18:42:37.391795/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:42:02.779435 2014-08-13 18:06:06.713467 7f5c973a57a0 20 read_log 194215'19318 (192888'19016) clean 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by osd.4.0:1777 2014-08-12 15:42:02.792250 2014-08-13 18:06:06.713483 7f5c973a57a0 20 read_log 194215'19319 (194215'19318) delete 49b50181/benchmark_data_storage0111-ib.pocstor_41097_object24106/head//24 by client.2174595.0:78382 0.000000 2014-08-13 18:06:06.713495 7f5c973a57a0 20 read_log 194215'19320 (192888'19176) clean 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by osd.4.0:1779 2014-08-12 15:42:02.817495 2014-08-13 18:06:06.713514 7f5c973a57a0 20 read_log 194215'19321 (194215'19320) delete 4a8c0181/benchmark_data_storage0111-ib.pocstor_2228_object6958/head//24 by client.2174595.0:78384 0.000000 2014-08-13 18:06:06.713529 7f5c973a57a0 20 read_log 194215'19322 (192888'19005) clean ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by osd.4.0:1781 2014-08-12 15:42:02.825705 2014-08-13 18:06:06.713542 7f5c973a57a0 20 read_log 194215'19323 (194215'19322) delete ff1d1181/benchmark_data_storage0111-ib.pocstor_41097_object19638/head//24 by client.2174595.0:78386 0.000000 2014-08-13 18:06:06.713562 7f5c973a57a0 20 read_log 194215'19324 (193305'19292) clean 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by osd.4.0:1783 2014-08-12 15:42:02.834177 2014-08-13 18:06:06.713578 7f5c973a57a0 20 read_log 194215'19325 (194215'19324) delete 22b34181/benchmark_data_storage0111-ib.pocstor_62621_object8478/head//24 by client.2174595.0:78388 0.000000 2014-08-13 18:06:06.713589 7f5c973a57a0 20 read_log 194215'19326 (193305'19288) clean 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by osd.4.0:1785 2014-08-12 15:42:02.842492 2014-08-13 18:06:06.713602 7f5c973a57a0 20 read_log 194215'19327 (194215'19326) delete 46065181/benchmark_data_storage0111-ib.pocstor_62621_object4700/head//24 by client.2174595.0:78390 0.000000 2014-08-13 18:06:06.713623 7f5c973a57a0 20 read_log 194215'19328 (192888'19186) clean fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by osd.4.0:1787 2014-08-12 15:42:02.850866 2014-08-13 18:06:06.713638 7f5c973a57a0 20 read_log 194215'19329 (194215'19328) delete fc75181/benchmark_data_storage0111-ib.pocstor_2228_object10248/head//24 by client.2174595.0:78392 0.000000 2014-08-13 18:06:06.713649 7f5c973a57a0 20 read_log 194215'19330 (193305'19308) clean 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by osd.4.0:1789 2014-08-12 15:42:02.860751 2014-08-13 18:06:06.713661 7f5c973a57a0 20 read_log 194215'19331 (194215'19330) delete 4f7a5181/benchmark_data_storage0111-ib.pocstor_62621_object24548/head//24 by client.2174595.0:78394 0.000000 2014-08-13 18:06:06.713672 7f5c973a57a0 20 read_log 194215'19332 (193305'19310) clean c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by osd.4.0:1791 2014-08-12 15:42:02.869496 2014-08-13 18:06:06.713693 7f5c973a57a0 20 read_log 194215'19333 (194215'19332) delete c6166181/benchmark_data_storage0111-ib.pocstor_62621_object26547/head//24 by client.2174595.0:78396 0.000000 2014-08-13 18:06:06.713705 7f5c973a57a0 20 read_log 194215'19334 (192888'19198) clean e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by osd.4.0:1793 2014-08-12 15:42:02.878153 2014-08-13 18:06:06.713718 7f5c973a57a0 20 read_log 194215'19335 (194215'19334) delete e7b57181/benchmark_data_storage0111-ib.pocstor_2228_object15513/head//24 by client.2174595.0:78398 0.000000 2014-08-13 18:06:06.713729 7f5c973a57a0 20 read_log 194215'19336 (192888'19224) clean be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by osd.4.0:1795 2014-08-12 15:42:02.886428 2014-08-13 18:06:06.713741 7f5c973a57a0 20 read_log 194215'19337 (194215'19336) delete be967181/benchmark_data_storage0111-ib.pocstor_2228_object30106/head//24 by client.2174595.0:78400 0.000000 2014-08-13 18:06:06.713752 7f5c973a57a0 20 read_log 194215'19338 (192888'19017) clean 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by osd.4.0:1797 2014-08-12 15:42:02.895484 2014-08-13 18:06:06.713764 7f5c973a57a0 20 read_log 194215'19339 (194215'19338) delete 41c29181/benchmark_data_storage0111-ib.pocstor_41097_object24416/head//24 by client.2174595.0:78402 0.000000 2014-08-13 18:06:06.713775 7f5c973a57a0 20 read_log 194215'19340 (192888'19201) clean ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by osd.4.0:1799 2014-08-12 15:42:02.904116 2014-08-13 18:06:06.713787 7f5c973a57a0 20 read_log 194215'19341 (194215'19340) delete ce189181/benchmark_data_storage0111-ib.pocstor_2228_object17121/head//24 by client.2174595.0:78404 0.000000 2014-08-13 18:06:06.713798 7f5c973a57a0 20 read_log 194215'19342 (192888'19021) clean 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by osd.4.0:1801 2014-08-12 15:42:02.912300 2014-08-13 18:06:06.713810 7f5c973a57a0 20 read_log 194215'19343 (194215'19342) delete 4edb9181/benchmark_data_storage0111-ib.pocstor_41097_object25253/head//24 by client.2174595.0:78406 0.000000 2014-08-13 18:06:06.713821 7f5c973a57a0 20 read_log 194215'19344 (193305'19291) clean 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by osd.4.0:1803 2014-08-12 15:42:02.947296 2014-08-13 18:06:06.713836 7f5c973a57a0 20 read_log 194215'19345 (194215'19344) delete 250aa181/benchmark_data_storage0111-ib.pocstor_62621_object8455/head//24 by client.2174595.0:78408 0.000000 2014-08-13 18:06:06.713865 7f5c973a57a0 20 read_log 194215'19346 (192888'19003) clean e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by osd.4.0:1805 2014-08-12 15:42:02.956194 2014-08-13 18:06:06.713881 7f5c973a57a0 20 read_log 194215'19347 (194215'19346) delete e1e5c181/benchmark_data_storage0111-ib.pocstor_41097_object18549/head//24 by client.2174595.0:78410 0.000000 2014-08-13 18:06:06.713899 7f5c973a57a0 20 read_log 194215'19348 (192888'18974) clean 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by osd.4.0:1807 2014-08-12 15:42:02.964745 2014-08-13 18:06:06.713912 7f5c973a57a0 20 read_log 194215'19349 (194215'19348) delete 1209c181/benchmark_data_storage0111-ib.pocstor_41097_object1177/head//24 by client.2174595.0:78412 0.000000 2014-08-13 18:06:06.713923 7f5c973a57a0 20 read_log 194215'19350 (192888'19013) clean 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by osd.4.0:1809 2014-08-12 15:42:02.973050 2014-08-13 18:06:06.713936 7f5c973a57a0 20 read_log 194215'19351 (194215'19350) delete 5ddbc181/benchmark_data_storage0111-ib.pocstor_41097_object23600/head//24 by client.2174595.0:78414 0.000000 2014-08-13 18:06:06.713947 7f5c973a57a0 20 read_log 194215'19352 (192888'19220) clean 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by osd.4.0:1811 2014-08-12 15:42:02.983226 2014-08-13 18:06:06.713959 7f5c973a57a0 20 read_log 194215'19353 (194215'19352) delete 5712d181/benchmark_data_storage0111-ib.pocstor_2228_object26545/head//24 by client.2174595.0:78416 0.000000 2014-08-13 18:06:06.713970 7f5c973a57a0 20 read_log 194215'19354 (192888'19024) clean 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by osd.4.0:1813 2014-08-12 15:42:02.991751 2014-08-13 18:06:06.713982 7f5c973a57a0 20 read_log 194215'19355 (194215'19354) delete 3b4d181/benchmark_data_storage0111-ib.pocstor_41097_object27360/head//24 by client.2174595.0:78418 0.000000 2014-08-13 18:06:06.713993 7f5c973a57a0 20 read_log 194215'19356 (192888'18985) clean 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by osd.4.0:1815 2014-08-12 15:42:02.999471 2014-08-13 18:06:06.714006 7f5c973a57a0 20 read_log 194215'19357 (194215'19356) delete 83e3e181/benchmark_data_storage0111-ib.pocstor_41097_object6177/head//24 by client.2174595.0:78420 0.000000 2014-08-13 18:06:06.714019 7f5c973a57a0 20 read_log 194215'19358 (192888'19203) clean 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by osd.4.0:1817 2014-08-12 15:42:03.007328 2014-08-13 18:06:06.714032 7f5c973a57a0 20 read_log 194215'19359 (194215'19358) delete 935f1581/benchmark_data_storage0111-ib.pocstor_2228_object17968/head//24 by client.2174595.0:78422 0.000000 2014-08-13 18:06:06.714047 7f5c973a57a0 20 read_log 194215'19360 (192888'19211) clean 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by osd.4.0:1819 2014-08-12 15:42:03.014677 2014-08-13 18:06:06.714062 7f5c973a57a0 20 read_log 194215'19361 (194215'19360) delete 59f82581/benchmark_data_storage0111-ib.pocstor_2228_object23072/head//24 by client.2174595.0:78424 0.000000 2014-08-13 18:06:06.714073 7f5c973a57a0 20 read_log 194215'19362 (192888'19009) clean 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by osd.4.0:1821 2014-08-12 15:42:03.024077 2014-08-13 18:06:06.714088 7f5c973a57a0 20 read_log 194215'19363 (194215'19362) delete 6f8a2581/benchmark_data_storage0111-ib.pocstor_41097_object21332/head//24 by client.2174595.0:78426 0.000000 2014-08-13 18:06:06.714099 7f5c973a57a0 20 read_log 194215'19364 (192888'18971) clean 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by osd.4.0:1823 2014-08-12 15:42:03.032097 2014-08-13 18:06:06.714113 7f5c973a57a0 20 read_log 194215'19365 (194215'19364) delete 4a7c2581/benchmark_data_storage0111-ib.pocstor_41097_object21/head//24 by client.2174595.0:78428 0.000000 2014-08-13 18:06:06.714126 7f5c973a57a0 20 read_log 194215'19366 (193305'19314) clean d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by osd.4.0:1825 2014-08-12 15:42:03.055264 2014-08-13 18:06:06.714141 7f5c973a57a0 20 read_log 194215'19367 (194215'19366) delete d03d2581/benchmark_data_storage0111-ib.pocstor_62621_object31711/head//24 by client.2174595.0:78430 0.000000 2014-08-13 18:06:06.714175 7f5c973a57a0 20 read_log 194215'19368 (192888'19022) clean f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by osd.4.0:1827 2014-08-12 15:42:03.064395 2014-08-13 18:06:06.714192 7f5c973a57a0 20 read_log 194215'19369 (194215'19368) delete f4d43581/benchmark_data_storage0111-ib.pocstor_41097_object26481/head//24 by client.2174595.0:78432 0.000000 2014-08-13 18:06:06.714203 7f5c973a57a0 20 read_log 194215'19370 (193305'19293) clean 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by osd.4.0:1829 2014-08-12 15:42:03.089076 2014-08-13 18:06:06.714216 7f5c973a57a0 20 read_log 194215'19371 (194215'19370) delete 641c4581/benchmark_data_storage0111-ib.pocstor_62621_object9025/head//24 by client.2174595.0:78434 0.000000 2014-08-13 18:06:06.714239 7f5c973a57a0 20 read_log 194215'19372 (192888'19219) clean d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by osd.4.0:1831 2014-08-12 15:42:03.097755 2014-08-13 18:06:06.714252 7f5c973a57a0 20 read_log 194215'19373 (194215'19372) delete d28c4581/benchmark_data_storage0111-ib.pocstor_2228_object26204/head//24 by client.2174595.0:78436 0.000000 2014-08-13 18:06:06.714265 7f5c973a57a0 20 read_log 194215'19374 (192888'18972) clean 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by osd.4.0:1833 2014-08-12 15:42:03.106868 2014-08-13 18:06:06.714280 7f5c973a57a0 20 read_log 194215'19375 (194215'19374) delete 4fd15581/benchmark_data_storage0111-ib.pocstor_41097_object681/head//24 by client.2174595.0:78438 0.000000 2014-08-13 18:06:06.714291 7f5c973a57a0 20 read_log 194215'19376 (193305'19294) clean 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by osd.4.0:1835 2014-08-12 15:42:03.116274 2014-08-13 18:06:06.714310 7f5c973a57a0 20 read_log 194215'19377 (194215'19376) delete 2f787581/benchmark_data_storage0111-ib.pocstor_62621_object9133/head//24 by client.2174595.0:78440 0.000000 2014-08-13 18:06:06.714321 7f5c973a57a0 20 read_log 194215'19378 (192888'19199) clean 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by osd.4.0:1837 2014-08-12 15:42:03.125354 2014-08-13 18:06:06.714333 7f5c973a57a0 20 read_log 194215'19379 (194215'19378) delete 1c3c7581/benchmark_data_storage0111-ib.pocstor_2228_object15614/head//24 by client.2174595.0:78442 0.000000 2014-08-13 18:06:06.714344 7f5c973a57a0 20 read_log 194215'19380 (192888'19014) clean 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by osd.4.0:1839 2014-08-12 15:42:03.134064 2014-08-13 18:06:06.714356 7f5c973a57a0 20 read_log 194215'19381 (194215'19380) delete 807a8581/benchmark_data_storage0111-ib.pocstor_41097_object23735/head//24 by client.2174595.0:78444 0.000000 2014-08-13 18:06:06.714372 7f5c973a57a0 20 read_log 194215'19382 (193305'19301) clean 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by osd.4.0:1841 2014-08-12 15:42:03.143215 2014-08-13 18:06:06.714384 7f5c973a57a0 20 read_log 194215'19383 (194215'19382) delete 67d99581/benchmark_data_storage0111-ib.pocstor_62621_object16510/head//24 by client.2174595.0:78446 0.000000 2014-08-13 18:06:06.714405 7f5c973a57a0 20 read_log 194215'19384 (192888'18991) clean 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by osd.4.0:1843 2014-08-12 15:42:03.151261 2014-08-13 18:06:06.714424 7f5c973a57a0 20 read_log 194215'19385 (194215'19384) delete 7ee99581/benchmark_data_storage0111-ib.pocstor_41097_object10475/head//24 by client.2174595.0:78448 0.000000 2014-08-13 18:06:06.714436 7f5c973a57a0 20 read_log 194215'19386 (192888'19183) clean fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by osd.4.0:1845 2014-08-12 15:42:03.161390 2014-08-13 18:06:06.714455 7f5c973a57a0 20 read_log 194215'19387 (194215'19386) delete fcb3b581/benchmark_data_storage0111-ib.pocstor_2228_object9816/head//24 by client.2174595.0:78450 0.000000 2014-08-13 18:06:06.714506 7f5c973a57a0 20 read_log 194215'19388 (192888'19023) clean 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by osd.4.0:1847 2014-08-12 15:42:03.169991 2014-08-13 18:06:06.714523 7f5c973a57a0 20 read_log 194215'19389 (194215'19388) delete 5aa7b581/benchmark_data_storage0111-ib.pocstor_41097_object26642/head//24 by client.2174595.0:78452 0.000000 2014-08-13 18:06:06.714535 7f5c973a57a0 20 read_log 194215'19390 (193305'19298) clean 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by osd.4.0:1849 2014-08-12 15:42:03.190911 2014-08-13 18:06:06.714554 7f5c973a57a0 20 read_log 194215'19391 (194215'19390) delete 4624c581/benchmark_data_storage0111-ib.pocstor_62621_object15168/head//24 by client.2174595.0:78454 0.000000 2014-08-13 18:06:06.714566 7f5c973a57a0 20 read_log 194215'19392 (192888'19216) clean e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by osd.4.0:1851 2014-08-12 15:42:03.200498 2014-08-13 18:06:06.714579 7f5c973a57a0 20 read_log 194215'19393 (194215'19392) delete e0e7c581/benchmark_data_storage0111-ib.pocstor_2228_object24692/head//24 by client.2174595.0:78456 0.000000 2014-08-13 18:06:06.714590 7f5c973a57a0 20 read_log 194215'19394 (192888'18999) clean f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by osd.4.0:1853 2014-08-12 15:42:03.209635 2014-08-13 18:06:06.714613 7f5c973a57a0 20 read_log 194215'19395 (194215'19394) delete f469d581/benchmark_data_storage0111-ib.pocstor_41097_object17218/head//24 by client.2174595.0:78458 0.000000 2014-08-13 18:06:06.714624 7f5c973a57a0 20 read_log 194215'19396 (193305'19289) clean 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by osd.4.0:1855 2014-08-12 15:42:03.218587 2014-08-13 18:06:06.714641 7f5c973a57a0 20 read_log 194215'19397 (194215'19396) delete 528fd581/benchmark_data_storage0111-ib.pocstor_62621_object6402/head//24 by client.2174595.0:78460 0.000000 2014-08-13 18:06:06.714654 7f5c973a57a0 20 read_log 194215'19398 (192888'18977) clean 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by osd.4.0:1857 2014-08-12 15:42:03.227078 2014-08-13 18:06:06.714667 7f5c973a57a0 20 read_log 194215'19399 (194215'19398) delete 1981e581/benchmark_data_storage0111-ib.pocstor_41097_object2237/head//24 by client.2174595.0:78462 0.000000 2014-08-13 18:06:06.714678 7f5c973a57a0 20 read_log 194215'19400 (193305'19309) clean b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by osd.4.0:1859 2014-08-12 15:42:03.236441 2014-08-13 18:06:06.714690 7f5c973a57a0 20 read_log 194215'19401 (194215'19400) delete b11ee581/benchmark_data_storage0111-ib.pocstor_62621_object26062/head//24 by client.2174595.0:78464 0.000000 2014-08-13 18:06:06.714701 7f5c973a57a0 20 read_log 194215'19402 (192888'19194) clean 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by osd.4.0:1861 2014-08-12 15:42:03.244814 2014-08-13 18:06:06.714713 7f5c973a57a0 20 read_log 194215'19403 (194215'19402) delete 4d21f581/benchmark_data_storage0111-ib.pocstor_2228_object13501/head//24 by client.2174595.0:78466 0.000000 2014-08-13 18:06:06.714724 7f5c973a57a0 20 read_log 194215'19404 (192888'19204) clean 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by osd.4.0:1863 2014-08-12 15:42:03.252965 2014-08-13 18:06:06.714736 7f5c973a57a0 20 read_log 194215'19405 (194215'19404) delete 3954f581/benchmark_data_storage0111-ib.pocstor_2228_object18658/head//24 by client.2174595.0:78468 0.000000 2014-08-13 18:06:06.714747 7f5c973a57a0 20 read_log 194215'19406 (192888'18981) clean 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by osd.4.0:1865 2014-08-12 15:42:03.262330 2014-08-13 18:06:06.714760 7f5c973a57a0 20 read_log 194215'19407 (194215'19406) delete 4736f581/benchmark_data_storage0111-ib.pocstor_41097_object4708/head//24 by client.2174595.0:78470 0.000000 2014-08-13 18:06:06.714780 7f5c973a57a0 20 read_log 194215'19408 (192888'19169) clean dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by osd.4.0:1867 2014-08-12 15:42:03.270176 2014-08-13 18:06:06.714796 7f5c973a57a0 20 read_log 194215'19409 (194215'19408) delete dae9f581/benchmark_data_storage0111-ib.pocstor_2228_object3212/head//24 by client.2174595.0:78472 0.000000 2014-08-13 18:06:06.714807 7f5c973a57a0 20 read_log 194215'19410 (193305'19306) clean 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by osd.4.0:1869 2014-08-12 15:42:03.278244 2014-08-13 18:06:06.714819 7f5c973a57a0 20 read_log 194215'19411 (194215'19410) delete 19640981/benchmark_data_storage0111-ib.pocstor_62621_object21661/head//24 by client.2174595.0:78474 0.000000 2014-08-13 18:06:06.714839 7f5c973a57a0 20 read_log 194215'19412 (192888'19193) clean 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by osd.4.0:1871 2014-08-12 15:42:03.287850 2014-08-13 18:06:06.714853 7f5c973a57a0 20 read_log 194215'19413 (194215'19412) delete 88771981/benchmark_data_storage0111-ib.pocstor_2228_object13357/head//24 by client.2174595.0:78476 0.000000 2014-08-13 18:06:06.714864 7f5c973a57a0 20 read_log 194215'19414 (192888'18983) clean b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by osd.4.0:1873 2014-08-12 15:42:03.333740 2014-08-13 18:06:06.714876 7f5c973a57a0 20 read_log 194215'19415 (194215'19414) delete b3281981/benchmark_data_storage0111-ib.pocstor_41097_object5352/head//24 by client.2174595.0:78478 0.000000 2014-08-13 18:06:06.714887 7f5c973a57a0 20 read_log 194215'19416 (192888'19019) clean b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by osd.4.0:1875 2014-08-12 15:42:03.342318 2014-08-13 18:06:06.714899 7f5c973a57a0 20 read_log 194215'19417 (194215'19416) delete b7ab1981/benchmark_data_storage0111-ib.pocstor_41097_object25129/head//24 by client.2174595.0:78480 0.000000 2014-08-13 18:06:06.714910 7f5c973a57a0 20 read_log 194215'19418 (192888'19170) clean a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by osd.4.0:1877 2014-08-12 15:42:03.350536 2014-08-13 18:06:06.714922 7f5c973a57a0 20 read_log 194215'19419 (194215'19418) delete a0733981/benchmark_data_storage0111-ib.pocstor_2228_object3606/head//24 by client.2174595.0:78482 0.000000 2014-08-13 18:06:06.714935 7f5c973a57a0 20 read_log 194215'19420 (192888'19001) clean f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by osd.4.0:1879 2014-08-12 15:42:03.391147 2014-08-13 18:06:06.714968 7f5c973a57a0 20 read_log 194215'19421 (194215'19420) delete f5a53981/benchmark_data_storage0111-ib.pocstor_41097_object17602/head//24 by client.2174595.0:78484 0.000000 2014-08-13 18:06:06.714984 7f5c973a57a0 20 read_log 194215'19422 (193305'19307) clean 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by osd.4.0:1881 2014-08-12 15:42:03.399966 2014-08-13 18:06:06.714998 7f5c973a57a0 20 read_log 194215'19423 (194215'19422) delete 10265981/benchmark_data_storage0111-ib.pocstor_62621_object22577/head//24 by client.2174595.0:78486 0.000000 2014-08-13 18:06:06.715009 7f5c973a57a0 20 read_log 194215'19424 (193305'19287) clean 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by osd.4.0:1883 2014-08-12 15:42:03.408110 2014-08-13 18:06:06.715022 7f5c973a57a0 20 read_log 194215'19425 (194215'19424) delete 77006981/benchmark_data_storage0111-ib.pocstor_62621_object1115/head//24 by client.2174595.0:78488 0.000000 2014-08-13 18:06:06.715033 7f5c973a57a0 20 read_log 194215'19426 (192888'19167) clean 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by osd.4.0:1885 2014-08-12 15:42:03.416773 2014-08-13 18:06:06.715045 7f5c973a57a0 20 read_log 194215'19427 (194215'19426) delete 36556981/benchmark_data_storage0111-ib.pocstor_2228_object1563/head//24 by client.2174595.0:78490 0.000000 2014-08-13 18:06:06.715072 7f5c973a57a0 20 read_log 194215'19428 (192888'18998) clean 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by osd.4.0:1887 2014-08-12 15:42:03.424982 2014-08-13 18:06:06.715088 7f5c973a57a0 20 read_log 194215'19429 (194215'19428) delete 67e17981/benchmark_data_storage0111-ib.pocstor_41097_object16734/head//24 by client.2174595.0:78492 0.000000 2014-08-13 18:06:06.715099 7f5c973a57a0 20 read_log 194215'19430 (192888'19207) clean 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by osd.4.0:1889 2014-08-12 15:42:03.433536 2014-08-13 18:06:06.715123 7f5c973a57a0 20 read_log 194215'19431 (194215'19430) delete 4e0d7981/benchmark_data_storage0111-ib.pocstor_2228_object21074/head//24 by client.2174595.0:78494 0.000000 2014-08-13 18:06:06.715134 7f5c973a57a0 20 read_log 194215'19432 (192888'19195) clean 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by osd.4.0:1891 2014-08-12 15:42:03.442337 2014-08-13 18:06:06.715151 7f5c973a57a0 20 read_log 194215'19433 (194215'19432) delete 17108981/benchmark_data_storage0111-ib.pocstor_2228_object13541/head//24 by client.2174595.0:78496 0.000000 2014-08-13 18:06:06.715163 7f5c973a57a0 20 read_log 194215'19434 (192888'19007) clean 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by osd.4.0:1893 2014-08-12 15:42:03.452247 2014-08-13 18:06:06.715176 7f5c973a57a0 20 read_log 194215'19435 (194215'19434) delete 22358981/benchmark_data_storage0111-ib.pocstor_41097_object20959/head//24 by client.2174595.0:78498 0.000000 2014-08-13 18:06:06.715187 7f5c973a57a0 20 read_log 194215'19436 (193305'19312) clean c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by osd.4.0:1895 2014-08-12 15:42:03.461070 2014-08-13 18:06:06.715199 7f5c973a57a0 20 read_log 194215'19437 (194215'19436) delete c3e59981/benchmark_data_storage0111-ib.pocstor_62621_object27123/head//24 by client.2174595.0:78500 0.000000 2014-08-13 18:06:06.715210 7f5c973a57a0 20 read_log 194215'19438 (192888'19163) clean 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by osd.4.0:1897 2014-08-12 15:42:03.469493 2014-08-13 18:06:06.715244 7f5c973a57a0 20 read_log 194215'19439 (194215'19438) delete 468c9981/benchmark_data_storage0111-ib.pocstor_2228_object244/head//24 by client.2174595.0:78502 0.000000 2014-08-13 18:06:06.715259 7f5c973a57a0 20 read_log 194215'19440 (193305'19296) clean 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by osd.4.0:1899 2014-08-12 15:42:03.479493 2014-08-13 18:06:06.715274 7f5c973a57a0 20 read_log 194215'19441 (194215'19440) delete 444da981/benchmark_data_storage0111-ib.pocstor_62621_object10976/head//24 by client.2174595.0:78504 0.000000 2014-08-13 18:06:06.715288 7f5c973a57a0 20 read_log 194215'19442 (192888'19209) clean 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by osd.4.0:1901 2014-08-12 15:42:03.487344 2014-08-13 18:06:06.715300 7f5c973a57a0 20 read_log 194215'19443 (194215'19442) delete 52cea981/benchmark_data_storage0111-ib.pocstor_2228_object21550/head//24 by client.2174595.0:78506 0.000000 2014-08-13 18:06:06.715311 7f5c973a57a0 20 read_log 194215'19444 (193305'19295) clean 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by osd.4.0:1903 2014-08-12 15:42:03.495520 2014-08-13 18:06:06.715323 7f5c973a57a0 20 read_log 194215'19445 (194215'19444) delete 90e7c981/benchmark_data_storage0111-ib.pocstor_62621_object10005/head//24 by client.2174595.0:78508 0.000000 2014-08-13 18:06:06.715347 7f5c973a57a0 20 read_log 194215'19446 (192888'18984) clean 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by osd.4.0:1905 2014-08-12 15:42:03.503831 2014-08-13 18:06:06.715360 7f5c973a57a0 20 read_log 194215'19447 (194215'19446) delete 2ffec981/benchmark_data_storage0111-ib.pocstor_41097_object5716/head//24 by client.2174595.0:78510 0.000000 2014-08-13 18:06:06.715379 7f5c973a57a0 20 read_log 194215'19448 (193305'19315) clean 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by osd.4.0:1907 2014-08-12 15:42:03.511551 2014-08-13 18:06:06.715394 7f5c973a57a0 20 read_log 194215'19449 (194215'19448) delete 303ae981/benchmark_data_storage0111-ib.pocstor_62621_object31939/head//24 by client.2174595.0:78512 0.000000 2014-08-13 18:06:06.715405 7f5c973a57a0 20 read_log 194215'19450 (193305'19311) clean 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by osd.4.0:1909 2014-08-12 15:42:03.520627 2014-08-13 18:06:06.715418 7f5c973a57a0 20 read_log 194215'19451 (194215'19450) delete 13b3f981/benchmark_data_storage0111-ib.pocstor_62621_object26874/head//24 by client.2174595.0:78514 0.000000 2014-08-13 18:06:06.715428 7f5c973a57a0 20 read_log 194215'19452 (192888'18995) clean 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by osd.4.0:1911 2014-08-12 15:42:03.528525 2014-08-13 18:06:06.715441 7f5c973a57a0 20 read_log 194215'19453 (194215'19452) delete 87970d81/benchmark_data_storage0111-ib.pocstor_41097_object13249/head//24 by client.2174595.0:78516 0.000000 2014-08-13 18:06:06.715451 7f5c973a57a0 20 read_log 194215'19454 (192888'18978) clean 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by osd.4.0:1913 2014-08-12 15:42:03.549893 2014-08-13 18:06:06.715465 7f5c973a57a0 20 read_log 194215'19455 (194215'19454) delete 8bc61d81/benchmark_data_storage0111-ib.pocstor_41097_object2678/head//24 by client.2174595.0:78518 0.000000 2014-08-13 18:06:06.715476 7f5c973a57a0 20 read_log 194215'19456 (192888'19192) clean 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by osd.4.0:1915 2014-08-12 15:42:03.558894 2014-08-13 18:06:06.715488 7f5c973a57a0 20 read_log 194215'19457 (194215'19456) delete 58762d81/benchmark_data_storage0111-ib.pocstor_2228_object13037/head//24 by client.2174595.0:78520 0.000000 2014-08-13 18:06:06.715499 7f5c973a57a0 20 read_log 194215'19458 (193305'19290) clean be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by osd.4.0:1917 2014-08-12 15:42:03.567317 2014-08-13 18:06:06.715511 7f5c973a57a0 20 read_log 194215'19459 (194215'19458) delete be753d81/benchmark_data_storage0111-ib.pocstor_62621_object7195/head//24 by client.2174595.0:78522 0.000000 2014-08-13 18:06:06.715522 7f5c973a57a0 20 read_log 194215'19460 (192888'18993) clean e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by osd.4.0:1919 2014-08-12 15:42:03.583268 2014-08-13 18:06:06.715534 7f5c973a57a0 20 read_log 194215'19461 (194215'19460) delete e4993d81/benchmark_data_storage0111-ib.pocstor_41097_object11843/head//24 by client.2174595.0:78524 0.000000 2014-08-13 18:06:06.715545 7f5c973a57a0 20 read_log 194215'19462 (193305'19313) clean 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by osd.4.0:1921 2014-08-12 15:42:03.621444 2014-08-13 18:06:06.715558 7f5c973a57a0 20 read_log 194215'19463 (194215'19462) delete 51dc3d81/benchmark_data_storage0111-ib.pocstor_62621_object31561/head//24 by client.2174595.0:78526 0.000000 2014-08-13 18:06:06.715568 7f5c973a57a0 20 read_log 194215'19464 (192888'19223) clean 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by osd.4.0:1923 2014-08-12 15:42:03.632567 2014-08-13 18:06:06.715581 7f5c973a57a0 20 read_log 194215'19465 (194215'19464) delete 3f774d81/benchmark_data_storage0111-ib.pocstor_2228_object29163/head//24 by client.2174595.0:78528 0.000000 2014-08-13 18:06:06.715592 7f5c973a57a0 20 read_log 194215'19466 (192888'18996) clean 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by osd.4.0:1925 2014-08-12 15:42:03.642046 2014-08-13 18:06:06.715604 7f5c973a57a0 20 read_log 194215'19467 (194215'19466) delete 79f25d81/benchmark_data_storage0111-ib.pocstor_41097_object14507/head//24 by client.2174595.0:78530 0.000000 2014-08-13 18:06:06.715624 7f5c973a57a0 20 read_log 194215'19468 (192888'19222) clean d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by osd.4.0:1927 2014-08-12 15:42:03.650019 2014-08-13 18:06:06.715639 7f5c973a57a0 20 read_log 194215'19469 (194215'19468) delete d3d45d81/benchmark_data_storage0111-ib.pocstor_2228_object28447/head//24 by client.2174595.0:78532 0.000000 2014-08-13 18:06:06.715651 7f5c973a57a0 20 read_log 194215'19470 (192888'19181) clean 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by osd.4.0:1929 2014-08-12 15:42:03.657986 2014-08-13 18:06:06.715665 7f5c973a57a0 20 read_log 194215'19471 (194215'19470) delete 654b5d81/benchmark_data_storage0111-ib.pocstor_2228_object8501/head//24 by client.2174595.0:78534 0.000000 2014-08-13 18:06:06.715676 7f5c973a57a0 20 read_log 194215'19472 (193305'19304) clean fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by osd.4.0:1931 2014-08-12 15:42:03.666503 2014-08-13 18:06:06.715689 7f5c973a57a0 20 read_log 194215'19473 (194215'19472) delete fb926d81/benchmark_data_storage0111-ib.pocstor_62621_object20020/head//24 by client.2174595.0:78536 0.000000 2014-08-13 18:06:06.715700 7f5c973a57a0 20 read_log 194215'19474 (193305'19303) clean c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by osd.4.0:1933 2014-08-12 15:42:03.674105 2014-08-13 18:06:06.715712 7f5c973a57a0 20 read_log 194215'19475 (194215'19474) delete c38d6d81/benchmark_data_storage0111-ib.pocstor_62621_object17596/head//24 by client.2174595.0:78538 0.000000 2014-08-13 18:06:06.715724 7f5c973a57a0 20 read_log 194215'19476 (193305'19297) clean eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by osd.4.0:1935 2014-08-12 15:42:03.682807 2014-08-13 18:06:06.715739 7f5c973a57a0 20 read_log 194215'19477 (194215'19476) delete eab37d81/benchmark_data_storage0111-ib.pocstor_62621_object11708/head//24 by client.2174595.0:78540 0.000000 2014-08-13 18:06:06.715750 7f5c973a57a0 20 read_log 194215'19478 (192888'19179) clean e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by osd.4.0:1937 2014-08-12 15:42:03.691401 2014-08-13 18:06:06.715764 7f5c973a57a0 20 read_log 194215'19479 (194215'19478) delete e4357d81/benchmark_data_storage0111-ib.pocstor_2228_object7746/head//24 by client.2174595.0:78542 0.000000 2014-08-13 18:06:06.715774 7f5c973a57a0 20 read_log 194215'19480 (192888'19012) clean d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by osd.4.0:1939 2014-08-12 15:42:03.699012 2014-08-13 18:06:06.715789 7f5c973a57a0 20 read_log 194215'19481 (194215'19480) delete d26b7d81/benchmark_data_storage0111-ib.pocstor_41097_object22547/head//24 by client.2174595.0:78544 0.000000 2014-08-13 18:06:06.715800 7f5c973a57a0 20 read_log 194215'19482 (192888'19205) clean efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by osd.4.0:1941 2014-08-12 15:42:03.707183 2014-08-13 18:06:06.715814 7f5c973a57a0 20 read_log 194215'19483 (194215'19482) delete efdf7d81/benchmark_data_storage0111-ib.pocstor_2228_object18787/head//24 by client.2174595.0:78546 0.000000 2014-08-13 18:06:06.715826 7f5c973a57a0 20 read_log 194215'19484 (192888'19190) clean 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by osd.4.0:1943 2014-08-12 15:42:03.715814 2014-08-13 18:06:06.715838 7f5c973a57a0 20 read_log 194215'19485 (194215'19484) delete 24b28d81/benchmark_data_storage0111-ib.pocstor_2228_object11946/head//24 by client.2174595.0:78548 0.000000 2014-08-13 18:06:06.715851 7f5c973a57a0 20 read_log 194215'19486 (192888'19182) clean 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by osd.4.0:1945 2014-08-12 15:42:03.723991 2014-08-13 18:06:06.715863 7f5c973a57a0 20 read_log 194215'19487 (194215'19486) delete 97098d81/benchmark_data_storage0111-ib.pocstor_2228_object9129/head//24 by client.2174595.0:78550 0.000000 2014-08-13 18:06:06.715881 7f5c973a57a0 20 read_log 194215'19488 (193305'19305) clean 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by osd.4.0:1947 2014-08-12 15:42:03.733040 2014-08-13 18:06:06.715895 7f5c973a57a0 20 read_log 194215'19489 (194215'19488) delete 2469d81/benchmark_data_storage0111-ib.pocstor_62621_object20910/head//24 by client.2174595.0:78552 0.000000 2014-08-13 18:06:06.715906 7f5c973a57a0 20 read_log 194215'19490 (192888'19002) clean 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by osd.4.0:1949 2014-08-12 15:42:03.741537 2014-08-13 18:06:06.715919 7f5c973a57a0 20 read_log 194215'19491 (194215'19490) delete 3db99d81/benchmark_data_storage0111-ib.pocstor_41097_object17868/head//24 by client.2174595.0:78554 0.000000 2014-08-13 18:06:06.715930 7f5c973a57a0 20 read_log 194215'19492 (192888'19171) clean 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by osd.4.0:1951 2014-08-12 15:42:03.749987 2014-08-13 18:06:06.715942 7f5c973a57a0 20 read_log 194215'19493 (194215'19492) delete 457b9d81/benchmark_data_storage0111-ib.pocstor_2228_object3797/head//24 by client.2174595.0:78556 0.000000 2014-08-13 18:06:06.715953 7f5c973a57a0 20 read_log 194215'19494 (192888'19226) clean e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by osd.4.0:1953 2014-08-12 15:42:03.781877 2014-08-13 18:06:06.715965 7f5c973a57a0 20 read_log 194215'19495 (194215'19494) delete e504ad81/benchmark_data_storage0111-ib.pocstor_2228_object30993/head//24 by client.2174595.0:78558 0.000000 2014-08-13 18:06:06.715976 7f5c973a57a0 20 read_log 194215'19496 (193305'19299) clean 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by osd.4.0:1955 2014-08-12 15:42:03.790909 2014-08-13 18:06:06.715988 7f5c973a57a0 20 read_log 194215'19497 (194215'19496) delete 6e3bd81/benchmark_data_storage0111-ib.pocstor_62621_object15964/head//24 by client.2174595.0:78560 0.000000 2014-08-13 18:06:06.715999 7f5c973a57a0 20 read_log 194215'19498 (192888'19004) clean 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by osd.4.0:1957 2014-08-12 15:42:03.798879 2014-08-13 18:06:06.716013 7f5c973a57a0 20 read_log 194215'19499 (194215'19498) delete 68fbd81/benchmark_data_storage0111-ib.pocstor_41097_object18584/head//24 by client.2174595.0:78562 0.000000 2014-08-13 18:06:06.716024 7f5c973a57a0 20 read_log 194215'19500 (192888'19166) clean 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by osd.4.0:1959 2014-08-12 15:42:03.807284 2014-08-13 18:06:06.716037 7f5c973a57a0 20 read_log 194215'19501 (194215'19500) delete 1e6ccd81/benchmark_data_storage0111-ib.pocstor_2228_object1282/head//24 by client.2174595.0:78564 0.000000 2014-08-13 18:06:06.716050 7f5c973a57a0 20 read_log 194215'19502 (193305'19300) clean 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by osd.4.0:1961 2014-08-12 15:42:03.815401 2014-08-13 18:06:06.716064 7f5c973a57a0 20 read_log 194215'19503 (194215'19502) delete 2d92dd81/benchmark_data_storage0111-ib.pocstor_62621_object16147/head//24 by client.2174595.0:78566 0.000000 2014-08-13 18:06:06.716075 7f5c973a57a0 20 read_log 194215'19504 (193305'19302) clean 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by osd.4.0:1963 2014-08-12 15:42:03.825342 2014-08-13 18:06:06.716089 7f5c973a57a0 20 read_log 194215'19505 (194215'19504) delete 76f1ed81/benchmark_data_storage0111-ib.pocstor_62621_object16743/head//24 by client.2174595.0:78568 0.000000 2014-08-13 18:06:06.716100 7f5c973a57a0 20 read_log 194215'19506 (192888'19206) clean e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by osd.4.0:1965 2014-08-12 15:42:03.833221 2014-08-13 18:06:06.716112 7f5c973a57a0 20 read_log 194215'19507 (194215'19506) delete e1f5ed81/benchmark_data_storage0111-ib.pocstor_2228_object19505/head//24 by client.2174595.0:78570 0.000000 2014-08-13 18:06:06.716132 7f5c973a57a0 20 read_log 194215'19508 (192888'18980) clean 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by osd.4.0:1967 2014-08-12 15:42:03.842093 2014-08-13 18:06:06.716152 7f5c973a57a0 20 read_log 194215'19509 (194215'19508) delete 4e91fd81/benchmark_data_storage0111-ib.pocstor_41097_object4111/head//24 by client.2174595.0:78572 0.000000 2014-08-13 18:06:06.716172 7f5c973a57a0 20 read_log 194215'19510 (192888'19197) clean 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by osd.4.0:1969 2014-08-12 15:42:03.850658 2014-08-13 18:06:06.716187 7f5c973a57a0 20 read_log 194215'19511 (194215'19510) delete 8ffafd81/benchmark_data_storage0111-ib.pocstor_2228_object15461/head//24 by client.2174595.0:78574 0.000000 2014-08-13 18:06:06.716199 7f5c973a57a0 20 read_log 194215'19512 (192888'19164) clean e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by osd.4.0:1971 2014-08-12 15:42:03.858539 2014-08-13 18:06:06.716213 7f5c973a57a0 20 read_log 194215'19513 (194215'19512) delete e9fefd81/benchmark_data_storage0111-ib.pocstor_2228_object523/head//24 by client.2174595.0:78576 0.000000 2014-08-13 18:06:06.716224 7f5c973a57a0 20 read_log 194215'19514 (194215'19514) modify 181/hit_set_24.181_archive_2014-08-12 15:42:02.779513_2014-08-12 17:24:44.874495/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.874798 2014-08-13 18:06:06.716237 7f5c973a57a0 20 read_log 194215'19515 (194215'19316) delete 181/hit_set_24.181_archive_2014-08-12 15:32:25.863362_2014-08-12 15:42:02.778991/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.874798 2014-08-13 18:06:06.716249 7f5c973a57a0 20 read_log 194215'19516 (0'0) promote ebdcd581/benchmark_data_storage0111-ib.pocstor_34628_object1261/head//24 by osd.4.0:5246 2014-08-12 17:24:44.880964 2014-08-13 18:06:06.716263 7f5c973a57a0 20 read_log 194215'19517 (194215'19516) modify ebdcd581/benchmark_data_storage0111-ib.pocstor_34628_object1261/head//24 by client.2193759.0:1262 2014-08-12 17:24:44.864634 2014-08-13 18:06:06.716276 7f5c973a57a0 20 read_log 194215'19518 (0'0) promote baad81/benchmark_data_storage0111-ib.pocstor_34628_object1494/head//24 by osd.4.0:5250 2014-08-12 17:24:47.139114 2014-08-13 18:06:06.716293 7f5c973a57a0 20 read_log 194215'19519 (194215'19518) modify baad81/benchmark_data_storage0111-ib.pocstor_34628_object1494/head//24 by client.2193759.0:1495 2014-08-12 17:24:47.129419 2014-08-13 18:06:06.716306 7f5c973a57a0 20 read_log 194370'19520 (0'0) promote c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by osd.320.0:1625 2014-08-12 20:30:17.915230 2014-08-13 18:06:06.716321 7f5c973a57a0 20 read_log 194370'19521 (194370'19520) modify c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by client.1508531.0:942 2014-08-12 20:30:17.906537 2014-08-13 18:06:06.716334 7f5c973a57a0 20 read_log 194370'19522 (0'0) promote 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by osd.320.0:1631 2014-08-12 20:30:20.614233 2014-08-13 18:06:06.716348 7f5c973a57a0 20 read_log 194370'19523 (194370'19522) modify 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by client.1508531.0:1665 2014-08-12 20:30:20.606550 2014-08-13 18:06:06.716361 7f5c973a57a0 20 read_log 194370'19524 (0'0) promote b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by osd.320.0:1639 2014-08-12 20:30:22.470096 2014-08-13 18:06:06.716375 7f5c973a57a0 20 read_log 194370'19525 (194370'19524) modify b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by client.1508531.0:2152 2014-08-12 20:30:22.461892 2014-08-13 18:06:06.716398 7f5c973a57a0 20 read_log 194370'19526 (0'0) promote 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by osd.320.0:1647 2014-08-12 20:30:24.118172 2014-08-13 18:06:06.716458 7f5c973a57a0 20 read_log 194370'19527 (194370'19526) modify 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by client.1508531.0:2607 2014-08-12 20:30:24.111564 2014-08-13 18:06:06.716476 7f5c973a57a0 20 read_log 194370'19528 (0'0) promote 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by osd.320.0:1651 2014-08-12 20:30:24.482966 2014-08-13 18:06:06.716489 7f5c973a57a0 20 read_log 194370'19529 (194370'19528) modify 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by client.1508531.0:2707 2014-08-12 20:30:24.474364 2014-08-13 18:06:06.716501 7f5c973a57a0 20 read_log 194370'19530 (0'0) promote 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by osd.320.0:1653 2014-08-12 20:30:24.773221 2014-08-13 18:06:06.716520 7f5c973a57a0 20 read_log 194370'19531 (194370'19530) modify 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by client.1508531.0:2787 2014-08-12 20:30:24.764784 2014-08-13 18:06:06.716571 7f5c973a57a0 20 read_log 194370'19532 (0'0) promote 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by osd.320.0:1663 2014-08-12 20:30:27.238370 2014-08-13 18:06:06.716588 7f5c973a57a0 20 read_log 194370'19533 (194370'19532) modify 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by client.1508531.0:3456 2014-08-12 20:30:27.230589 2014-08-13 18:06:06.716601 7f5c973a57a0 20 read_log 194370'19534 (0'0) promote e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by osd.320.0:1665 2014-08-12 20:30:28.763841 2014-08-13 18:06:06.716615 7f5c973a57a0 20 read_log 194370'19535 (194370'19534) modify e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by client.1508531.0:3854 2014-08-12 20:30:28.755859 2014-08-13 18:06:06.716628 7f5c973a57a0 20 read_log 194370'19536 (0'0) promote b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by osd.320.0:1673 2014-08-12 20:30:30.376805 2014-08-13 18:06:06.716640 7f5c973a57a0 20 read_log 194370'19537 (194370'19536) modify b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by client.1508531.0:4277 2014-08-12 20:30:30.369189 2014-08-13 18:06:06.716671 7f5c973a57a0 20 read_log 194370'19538 (0'0) promote d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by osd.320.0:1675 2014-08-12 20:30:30.577055 2014-08-13 18:06:06.716687 7f5c973a57a0 20 read_log 194370'19539 (194370'19538) modify d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by client.1508531.0:4333 2014-08-12 20:30:30.570437 2014-08-13 18:06:06.716700 7f5c973a57a0 20 read_log 194370'19540 (0'0) promote 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by osd.320.0:1695 2014-08-12 20:30:35.976511 2014-08-13 18:06:06.716712 7f5c973a57a0 20 read_log 194370'19541 (194370'19540) modify 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by client.1508531.0:5623 2014-08-12 20:30:35.968041 2014-08-13 18:06:06.716724 7f5c973a57a0 20 read_log 194370'19542 (0'0) promote 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by osd.320.0:1733 2014-08-12 20:31:08.641394 2014-08-13 18:06:06.716736 7f5c973a57a0 20 read_log 194370'19543 (194370'19542) modify 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by client.1508531.0:11554 2014-08-12 20:31:08.632409 2014-08-13 18:06:06.716749 7f5c973a57a0 20 read_log 194377'19544 (0'0) promote 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by osd.320.0:1753 2014-08-12 20:32:57.876969 2014-08-13 18:06:06.716761 7f5c973a57a0 20 read_log 194377'19545 (194377'19544) modify 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by client.2231190.0:170 2014-08-12 20:32:57.868261 2014-08-13 18:06:06.716774 7f5c973a57a0 20 read_log 194377'19546 (0'0) promote a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by osd.320.0:1755 2014-08-12 20:32:58.064153 2014-08-13 18:06:06.716794 7f5c973a57a0 20 read_log 194377'19547 (194377'19546) modify a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by client.2231190.0:218 2014-08-12 20:32:58.056121 2014-08-13 18:06:06.716809 7f5c973a57a0 20 read_log 194377'19548 (0'0) promote 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by osd.320.0:1759 2014-08-12 20:32:59.189241 2014-08-13 18:06:06.716822 7f5c973a57a0 20 read_log 194377'19549 (194377'19548) modify 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by client.2231190.0:463 2014-08-12 20:32:59.179609 2014-08-13 18:06:06.716834 7f5c973a57a0 20 read_log 194377'19550 (0'0) promote 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by osd.320.0:1761 2014-08-12 20:32:59.678978 2014-08-13 18:06:06.716846 7f5c973a57a0 20 read_log 194377'19551 (194377'19550) modify 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by client.2231190.0:575 2014-08-12 20:32:59.671033 2014-08-13 18:06:06.716859 7f5c973a57a0 20 read_log 194377'19552 (0'0) promote 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by osd.320.0:1783 2014-08-12 20:33:07.125458 2014-08-13 18:06:06.716871 7f5c973a57a0 20 read_log 194377'19553 (194377'19552) modify 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by client.2231190.0:2479 2014-08-12 20:33:07.118227 2014-08-13 18:06:06.716884 7f5c973a57a0 20 read_log 194383'19554 (0'0) promote d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by osd.320.0:1821 2014-08-12 20:34:01.175476 2014-08-13 18:06:06.716896 7f5c973a57a0 20 read_log 194383'19555 (194383'19554) modify d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by client.2231190.0:5953 2014-08-12 20:34:01.166996 2014-08-13 18:06:06.716908 7f5c973a57a0 20 read_log 194383'19556 (0'0) promote 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by osd.320.0:1825 2014-08-12 20:34:02.428053 2014-08-13 18:06:06.716920 7f5c973a57a0 20 read_log 194383'19557 (194383'19556) modify 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by client.2231190.0:6194 2014-08-12 20:34:02.418244 2014-08-13 18:06:06.716933 7f5c973a57a0 20 read_log 194383'19558 (0'0) promote a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by osd.320.0:1829 2014-08-12 20:34:06.630301 2014-08-13 18:06:06.716945 7f5c973a57a0 20 read_log 194383'19559 (194383'19558) modify a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by client.2231190.0:6670 2014-08-12 20:34:06.621019 2014-08-13 18:06:06.716957 7f5c973a57a0 20 read_log 194393'19560 (0'0) promote 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by osd.320.0:1856 2014-08-12 20:52:32.756836 2014-08-13 18:06:06.716970 7f5c973a57a0 20 read_log 194393'19561 (194393'19560) modify 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by client.1064422.0:64 2014-08-12 20:52:32.746227 2014-08-13 18:06:06.716982 7f5c973a57a0 20 read_log 194416'19562 (0'0) promote cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by osd.320.0:1982 2014-08-13 15:28:31.486966 2014-08-13 18:06:06.716997 7f5c973a57a0 20 read_log 194416'19563 (194416'19562) modify cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by client.2441958.0:307 2014-08-13 15:28:31.477555 2014-08-13 18:06:06.717009 7f5c973a57a0 20 read_log 194416'19564 (0'0) promote 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by osd.320.0:1984 2014-08-13 15:28:32.180083 2014-08-13 18:06:06.717024 7f5c973a57a0 20 read_log 194416'19565 (194416'19564) modify 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by client.2441958.0:450 2014-08-13 15:28:32.171608 2014-08-13 18:06:06.717043 7f5c973a57a0 20 read_log 194416'19566 (0'0) promote 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by osd.320.0:1993 2014-08-13 15:28:34.173447 2014-08-13 18:06:06.717058 7f5c973a57a0 20 read_log 194416'19567 (194416'19566) modify 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by client.2441958.0:924 2014-08-13 15:28:34.164486 2014-08-13 18:06:06.717070 7f5c973a57a0 20 read_log 194416'19568 (0'0) promote 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by osd.320.0:1999 2014-08-13 15:28:36.812163 2014-08-13 18:06:06.717083 7f5c973a57a0 20 read_log 194416'19569 (194416'19568) modify 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by client.2441958.0:1607 2014-08-13 15:28:36.804176 2014-08-13 18:06:06.717097 7f5c973a57a0 20 read_log 194416'19570 (0'0) promote 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by osd.320.0:2001 2014-08-13 15:28:38.765628 2014-08-13 18:06:06.717110 7f5c973a57a0 20 read_log 194416'19571 (194416'19570) modify 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by client.2441958.0:2128 2014-08-13 15:28:38.757016 2014-08-13 18:06:06.717122 7f5c973a57a0 20 read_log 194416'19572 (0'0) promote c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by osd.320.0:2018 2014-08-13 15:28:48.445764 2014-08-13 18:06:06.717134 7f5c973a57a0 20 read_log 194416'19573 (194416'19572) modify c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by client.2441958.0:4616 2014-08-13 15:28:48.437627 2014-08-13 18:06:06.717152 7f5c973a57a0 20 read_log 194416'19574 (0'0) promote ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by osd.320.0:2032 2014-08-13 15:28:56.047454 2014-08-13 18:06:06.717165 7f5c973a57a0 20 read_log 194416'19575 (194416'19574) modify ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by client.2441958.0:6548 2014-08-13 15:28:56.037366 2014-08-13 18:06:06.717178 7f5c973a57a0 20 read_log 194416'19576 (0'0) promote b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by osd.320.0:2040 2014-08-13 15:29:01.049211 2014-08-13 18:06:06.717190 7f5c973a57a0 20 read_log 194416'19577 (194416'19576) modify b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by client.2441958.0:7871 2014-08-13 15:29:01.039824 2014-08-13 18:06:06.717212 7f5c973a57a0 20 read_log 194416'19578 (0'0) promote 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by osd.320.0:2042 2014-08-13 15:29:10.111280 2014-08-13 18:06:06.717225 7f5c973a57a0 20 read_log 194416'19579 (194416'19578) modify 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by client.2441958.0:10175 2014-08-13 15:29:10.101106 2014-08-13 18:06:06.717239 7f5c973a57a0 20 read_log 194416'19580 (0'0) promote 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by osd.320.0:2064 2014-08-13 15:29:15.188096 2014-08-13 18:06:06.717252 7f5c973a57a0 20 read_log 194416'19581 (194416'19580) modify 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by client.2441958.0:11424 2014-08-13 15:29:15.179344 2014-08-13 18:06:06.717266 7f5c973a57a0 20 read_log 194416'19582 (0'0) promote b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by osd.320.0:2070 2014-08-13 15:29:18.333781 2014-08-13 18:06:06.717279 7f5c973a57a0 20 read_log 194416'19583 (194416'19582) modify b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by client.2441958.0:12162 2014-08-13 15:29:18.326088 2014-08-13 18:06:06.717296 7f5c973a57a0 20 read_log 194416'19584 (0'0) promote 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by osd.320.0:2072 2014-08-13 15:29:19.052488 2014-08-13 18:06:06.717311 7f5c973a57a0 20 read_log 194416'19585 (194416'19584) modify 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by client.2441958.0:12334 2014-08-13 15:29:19.044709 2014-08-13 18:06:06.717334 7f5c973a57a0 20 read_log 194416'19586 (0'0) promote 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by osd.320.0:2074 2014-08-13 15:29:19.453734 2014-08-13 18:06:06.717349 7f5c973a57a0 20 read_log 194416'19587 (194416'19586) modify 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by client.2441958.0:12437 2014-08-13 15:29:19.445291 2014-08-13 18:06:06.717361 7f5c973a57a0 20 read_log 194416'19588 (0'0) promote 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by osd.320.0:2076 2014-08-13 15:29:21.833855 2014-08-13 18:06:06.717373 7f5c973a57a0 20 read_log 194416'19589 (194416'19588) modify 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by client.2441958.0:12987 2014-08-13 15:29:21.825229 2014-08-13 18:06:06.717386 7f5c973a57a0 20 read_log 194416'19590 (0'0) promote 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by osd.320.0:2078 2014-08-13 15:29:22.879104 2014-08-13 18:06:06.717398 7f5c973a57a0 20 read_log 194416'19591 (194416'19590) modify 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by client.2441958.0:13216 2014-08-13 15:29:22.868567 2014-08-13 18:06:06.717420 7f5c973a57a0 20 read_log 194416'19592 (0'0) promote 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by osd.320.0:2084 2014-08-13 15:29:25.487783 2014-08-13 18:06:06.717437 7f5c973a57a0 20 read_log 194416'19593 (194416'19592) modify 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by client.2441958.0:13813 2014-08-13 15:29:25.476784 2014-08-13 18:06:06.717450 7f5c973a57a0 20 read_log 194416'19594 (0'0) promote a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by osd.320.0:2100 2014-08-13 15:29:41.211718 2014-08-13 18:06:06.717462 7f5c973a57a0 20 read_log 194416'19595 (194416'19594) modify a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by client.2441958.0:16568 2014-08-13 15:29:41.201401 2014-08-13 18:06:06.717490 7f5c973a57a0 20 read_log 194416'19596 (0'0) promote 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by osd.320.0:2102 2014-08-13 15:29:46.866231 2014-08-13 18:06:06.717503 7f5c973a57a0 20 read_log 194416'19597 (194416'19596) modify 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by client.2441958.0:16965 2014-08-13 15:29:46.855751 2014-08-13 18:06:06.717522 7f5c973a57a0 20 read_log 194416'19598 (0'0) promote 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by osd.320.0:2116 2014-08-13 15:30:08.730007 2014-08-13 18:06:06.717538 7f5c973a57a0 20 read_log 194416'19599 (194416'19598) modify 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by client.2441958.0:18342 2014-08-13 15:30:08.722319 2014-08-13 18:06:06.717551 7f5c973a57a0 20 read_log 194422'19600 (0'0) promote de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by osd.320.0:2130 2014-08-13 15:32:26.228968 2014-08-13 18:06:06.717563 7f5c973a57a0 20 read_log 194422'19601 (194422'19600) modify de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by client.1148203.0:2786 2014-08-13 15:32:26.218465 2014-08-13 18:06:06.717585 7f5c973a57a0 20 read_log 194422'19602 (0'0) promote 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by osd.320.0:2140 2014-08-13 15:32:32.608460 2014-08-13 18:06:06.717598 7f5c973a57a0 20 read_log 194422'19603 (194422'19602) modify 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by client.1148203.0:4421 2014-08-13 15:32:32.599714 2014-08-13 18:06:06.717611 7f5c973a57a0 20 read_log 194422'19604 (0'0) promote 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by osd.320.0:2142 2014-08-13 15:32:35.559977 2014-08-13 18:06:06.717631 7f5c973a57a0 20 read_log 194422'19605 (194422'19604) modify 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by client.1148203.0:5146 2014-08-13 15:32:35.550402 2014-08-13 18:06:06.717649 7f5c973a57a0 20 read_log 194422'19606 (0'0) promote c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by osd.320.0:2148 2014-08-13 15:32:42.224499 2014-08-13 18:06:06.717662 7f5c973a57a0 20 read_log 194422'19607 (194422'19606) modify c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by client.1148203.0:6381 2014-08-13 15:32:42.215412 2014-08-13 18:06:06.717674 7f5c973a57a0 20 read_log 194422'19608 (0'0) promote 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by osd.320.0:2160 2014-08-13 15:32:50.176861 2014-08-13 18:06:06.717686 7f5c973a57a0 20 read_log 194422'19609 (194422'19608) modify 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by client.1148203.0:7748 2014-08-13 15:32:50.168241 2014-08-13 18:06:06.717698 7f5c973a57a0 20 read_log 194422'19610 (0'0) promote ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by osd.320.0:2162 2014-08-13 15:32:51.917737 2014-08-13 18:06:06.717710 7f5c973a57a0 20 read_log 194422'19611 (194422'19610) modify ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by client.1148203.0:8035 2014-08-13 15:32:51.909955 2014-08-13 18:06:06.717723 7f5c973a57a0 20 read_log 194422'19612 (0'0) promote dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by osd.320.0:2168 2014-08-13 15:32:54.836968 2014-08-13 18:06:06.717735 7f5c973a57a0 20 read_log 194422'19613 (194422'19612) modify dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by client.1148203.0:8463 2014-08-13 15:32:54.827337 2014-08-13 18:06:06.717762 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.718237 7f5c973a57a0 10 read_log checking for missing items over interval (194370'19520,194422'19613] 2014-08-13 18:06:06.718288 7f5c973a57a0 15 read_log missing 194422'19613 (194422'19612) modify dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 by client.1148203.0:8463 2014-08-13 15:32:54.827337 2014-08-13 18:06:06.718330 7f5c973a57a0 15 read_log missing 194422'19611 (194422'19610) modify ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 by client.1148203.0:8035 2014-08-13 15:32:51.909955 2014-08-13 18:06:06.718370 7f5c973a57a0 15 read_log missing 194422'19609 (194422'19608) modify 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 by client.1148203.0:7748 2014-08-13 15:32:50.168241 2014-08-13 18:06:06.718418 7f5c973a57a0 15 read_log missing 194422'19607 (194422'19606) modify c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 by client.1148203.0:6381 2014-08-13 15:32:42.215412 2014-08-13 18:06:06.718456 7f5c973a57a0 15 read_log missing 194422'19605 (194422'19604) modify 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 by client.1148203.0:5146 2014-08-13 15:32:35.550402 2014-08-13 18:06:06.718492 7f5c973a57a0 15 read_log missing 194422'19603 (194422'19602) modify 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 by client.1148203.0:4421 2014-08-13 15:32:32.599714 2014-08-13 18:06:06.718528 7f5c973a57a0 15 read_log missing 194422'19601 (194422'19600) modify de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 by client.1148203.0:2786 2014-08-13 15:32:26.218465 2014-08-13 18:06:06.718563 7f5c973a57a0 15 read_log missing 194416'19599 (194416'19598) modify 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 by client.2441958.0:18342 2014-08-13 15:30:08.722319 2014-08-13 18:06:06.718599 7f5c973a57a0 15 read_log missing 194416'19597 (194416'19596) modify 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 by client.2441958.0:16965 2014-08-13 15:29:46.855751 2014-08-13 18:06:06.718635 7f5c973a57a0 15 read_log missing 194416'19595 (194416'19594) modify a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 by client.2441958.0:16568 2014-08-13 15:29:41.201401 2014-08-13 18:06:06.718670 7f5c973a57a0 15 read_log missing 194416'19593 (194416'19592) modify 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 by client.2441958.0:13813 2014-08-13 15:29:25.476784 2014-08-13 18:06:06.718706 7f5c973a57a0 15 read_log missing 194416'19591 (194416'19590) modify 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 by client.2441958.0:13216 2014-08-13 15:29:22.868567 2014-08-13 18:06:06.718742 7f5c973a57a0 15 read_log missing 194416'19589 (194416'19588) modify 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 by client.2441958.0:12987 2014-08-13 15:29:21.825229 2014-08-13 18:06:06.718778 7f5c973a57a0 15 read_log missing 194416'19587 (194416'19586) modify 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 by client.2441958.0:12437 2014-08-13 15:29:19.445291 2014-08-13 18:06:06.718815 7f5c973a57a0 15 read_log missing 194416'19585 (194416'19584) modify 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 by client.2441958.0:12334 2014-08-13 15:29:19.044709 2014-08-13 18:06:06.718852 7f5c973a57a0 15 read_log missing 194416'19583 (194416'19582) modify b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 by client.2441958.0:12162 2014-08-13 15:29:18.326088 2014-08-13 18:06:06.718887 7f5c973a57a0 15 read_log missing 194416'19581 (194416'19580) modify 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 by client.2441958.0:11424 2014-08-13 15:29:15.179344 2014-08-13 18:06:06.718923 7f5c973a57a0 15 read_log missing 194416'19579 (194416'19578) modify 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 by client.2441958.0:10175 2014-08-13 15:29:10.101106 2014-08-13 18:06:06.718958 7f5c973a57a0 15 read_log missing 194416'19577 (194416'19576) modify b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 by client.2441958.0:7871 2014-08-13 15:29:01.039824 2014-08-13 18:06:06.718994 7f5c973a57a0 15 read_log missing 194416'19575 (194416'19574) modify ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 by client.2441958.0:6548 2014-08-13 15:28:56.037366 2014-08-13 18:06:06.719030 7f5c973a57a0 15 read_log missing 194416'19573 (194416'19572) modify c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 by client.2441958.0:4616 2014-08-13 15:28:48.437627 2014-08-13 18:06:06.719066 7f5c973a57a0 15 read_log missing 194416'19571 (194416'19570) modify 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 by client.2441958.0:2128 2014-08-13 15:28:38.757016 2014-08-13 18:06:06.719101 7f5c973a57a0 15 read_log missing 194416'19569 (194416'19568) modify 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 by client.2441958.0:1607 2014-08-13 15:28:36.804176 2014-08-13 18:06:06.719137 7f5c973a57a0 15 read_log missing 194416'19567 (194416'19566) modify 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 by client.2441958.0:924 2014-08-13 15:28:34.164486 2014-08-13 18:06:06.719179 7f5c973a57a0 15 read_log missing 194416'19565 (194416'19564) modify 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 by client.2441958.0:450 2014-08-13 15:28:32.171608 2014-08-13 18:06:06.719215 7f5c973a57a0 15 read_log missing 194416'19563 (194416'19562) modify cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 by client.2441958.0:307 2014-08-13 15:28:31.477555 2014-08-13 18:06:06.719251 7f5c973a57a0 15 read_log missing 194393'19561 (194393'19560) modify 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 by client.1064422.0:64 2014-08-12 20:52:32.746227 2014-08-13 18:06:06.719290 7f5c973a57a0 15 read_log missing 194383'19559 (194383'19558) modify a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 by client.2231190.0:6670 2014-08-12 20:34:06.621019 2014-08-13 18:06:06.719328 7f5c973a57a0 15 read_log missing 194383'19557 (194383'19556) modify 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 by client.2231190.0:6194 2014-08-12 20:34:02.418244 2014-08-13 18:06:06.719369 7f5c973a57a0 15 read_log missing 194383'19555 (194383'19554) modify d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 by client.2231190.0:5953 2014-08-12 20:34:01.166996 2014-08-13 18:06:06.719405 7f5c973a57a0 15 read_log missing 194377'19553 (194377'19552) modify 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 by client.2231190.0:2479 2014-08-12 20:33:07.118227 2014-08-13 18:06:06.719457 7f5c973a57a0 15 read_log missing 194377'19551 (194377'19550) modify 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 by client.2231190.0:575 2014-08-12 20:32:59.671033 2014-08-13 18:06:06.719502 7f5c973a57a0 15 read_log missing 194377'19549 (194377'19548) modify 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 by client.2231190.0:463 2014-08-12 20:32:59.179609 2014-08-13 18:06:06.719549 7f5c973a57a0 15 read_log missing 194377'19547 (194377'19546) modify a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 by client.2231190.0:218 2014-08-12 20:32:58.056121 2014-08-13 18:06:06.719586 7f5c973a57a0 15 read_log missing 194377'19545 (194377'19544) modify 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 by client.2231190.0:170 2014-08-12 20:32:57.868261 2014-08-13 18:06:06.719621 7f5c973a57a0 15 read_log missing 194370'19543 (194370'19542) modify 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 by client.1508531.0:11554 2014-08-12 20:31:08.632409 2014-08-13 18:06:06.719673 7f5c973a57a0 15 read_log missing 194370'19541 (194370'19540) modify 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 by client.1508531.0:5623 2014-08-12 20:30:35.968041 2014-08-13 18:06:06.719711 7f5c973a57a0 15 read_log missing 194370'19539 (194370'19538) modify d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 by client.1508531.0:4333 2014-08-12 20:30:30.570437 2014-08-13 18:06:06.719751 7f5c973a57a0 15 read_log missing 194370'19537 (194370'19536) modify b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 by client.1508531.0:4277 2014-08-12 20:30:30.369189 2014-08-13 18:06:06.719787 7f5c973a57a0 15 read_log missing 194370'19535 (194370'19534) modify e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 by client.1508531.0:3854 2014-08-12 20:30:28.755859 2014-08-13 18:06:06.719835 7f5c973a57a0 15 read_log missing 194370'19533 (194370'19532) modify 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 by client.1508531.0:3456 2014-08-12 20:30:27.230589 2014-08-13 18:06:06.719874 7f5c973a57a0 15 read_log missing 194370'19531 (194370'19530) modify 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 by client.1508531.0:2787 2014-08-12 20:30:24.764784 2014-08-13 18:06:06.719910 7f5c973a57a0 15 read_log missing 194370'19529 (194370'19528) modify 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 by client.1508531.0:2707 2014-08-12 20:30:24.474364 2014-08-13 18:06:06.719947 7f5c973a57a0 15 read_log missing 194370'19527 (194370'19526) modify 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 by client.1508531.0:2607 2014-08-12 20:30:24.111564 2014-08-13 18:06:06.719983 7f5c973a57a0 15 read_log missing 194370'19525 (194370'19524) modify b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 by client.1508531.0:2152 2014-08-12 20:30:22.461892 2014-08-13 18:06:06.720018 7f5c973a57a0 15 read_log missing 194370'19523 (194370'19522) modify 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 by client.1508531.0:1665 2014-08-12 20:30:20.606550 2014-08-13 18:06:06.720054 7f5c973a57a0 15 read_log missing 194370'19521 (194370'19520) modify c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 by client.1508531.0:942 2014-08-12 20:30:17.906537 2014-08-13 18:06:06.720065 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.720094 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_loaded 2014-08-13 18:06:06.720112 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] exit Initial 0.014917 0 0.000000 2014-08-13 18:06:06.720125 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] enter Reset 2014-08-13 18:06:06.720134 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=0 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] set_last_peering_reset 194487 2014-08-13 18:06:06.720145 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] log((192834'17288,194422'19613], crt=194422'19613) 2014-08-13 18:06:06.720162 7f5c973a57a0 10 osd.4 194487 pgid 24.1bb coll 24.1bb_head 2014-08-13 18:06:06.720304 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.1bb 2014-08-13 18:06:06.720330 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.720381 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1bb(unlocked)] enter Initial 2014-08-13 18:06:06.720393 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1bb(unlocked)] enter NotTrimming 2014-08-13 18:06:06.720539 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.720649 7f5c973a57a0 20 read_log 192718'9696 (192718'9695) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:5 2014-08-06 14:51:11.570229 2014-08-13 18:06:06.720682 7f5c973a57a0 20 read_log 192718'9697 (192718'9696) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:6 2014-08-06 14:51:11.597604 2014-08-13 18:06:06.720698 7f5c973a57a0 20 read_log 192718'9698 (192718'9697) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:7 2014-08-06 14:51:11.653681 2014-08-13 18:06:06.720710 7f5c973a57a0 20 read_log 192718'9699 (192718'9698) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:8 2014-08-06 14:51:11.680474 2014-08-13 18:06:06.720723 7f5c973a57a0 20 read_log 192718'9700 (192718'9699) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:9 2014-08-06 14:51:11.775367 2014-08-13 18:06:06.720735 7f5c973a57a0 20 read_log 192718'9701 (192718'9700) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:10 2014-08-06 14:51:11.801363 2014-08-13 18:06:06.720769 7f5c973a57a0 20 read_log 192718'9702 (192718'9701) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:11 2014-08-06 14:51:11.826722 2014-08-13 18:06:06.720784 7f5c973a57a0 20 read_log 192718'9703 (192718'9702) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:12 2014-08-06 14:51:11.979621 2014-08-13 18:06:06.720797 7f5c973a57a0 20 read_log 192718'9704 (192718'9703) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:13 2014-08-06 14:51:12.044660 2014-08-13 18:06:06.720809 7f5c973a57a0 20 read_log 192718'9705 (192718'9704) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:14 2014-08-06 14:51:12.069439 2014-08-13 18:06:06.720821 7f5c973a57a0 20 read_log 192718'9706 (192718'9705) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:15 2014-08-06 14:51:12.129877 2014-08-13 18:06:06.720833 7f5c973a57a0 20 read_log 192718'9707 (192718'9706) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:16 2014-08-06 14:51:12.152051 2014-08-13 18:06:06.720845 7f5c973a57a0 20 read_log 192718'9708 (192718'9707) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:17 2014-08-06 14:51:12.212285 2014-08-13 18:06:06.720857 7f5c973a57a0 20 read_log 192718'9709 (192718'9708) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:18 2014-08-06 14:51:12.235832 2014-08-13 18:06:06.720870 7f5c973a57a0 20 read_log 192718'9710 (192718'9709) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:19 2014-08-06 14:51:12.303282 2014-08-13 18:06:06.720882 7f5c973a57a0 20 read_log 192718'9711 (192718'9710) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:20 2014-08-06 14:51:12.327205 2014-08-13 18:06:06.720894 7f5c973a57a0 20 read_log 192718'9712 (192718'9711) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:21 2014-08-06 14:51:12.391765 2014-08-13 18:06:06.720906 7f5c973a57a0 20 read_log 192718'9713 (192718'9712) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:22 2014-08-06 14:51:12.416635 2014-08-13 18:06:06.720918 7f5c973a57a0 20 read_log 192718'9714 (192718'9713) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:23 2014-08-06 14:51:12.477402 2014-08-13 18:06:06.720930 7f5c973a57a0 20 read_log 192718'9715 (192718'9714) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:24 2014-08-06 14:51:12.499198 2014-08-13 18:06:06.720942 7f5c973a57a0 20 read_log 192718'9716 (192718'9715) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:25 2014-08-06 14:51:12.589531 2014-08-13 18:06:06.720954 7f5c973a57a0 20 read_log 192718'9717 (192718'9716) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:26 2014-08-06 14:51:12.625294 2014-08-13 18:06:06.720966 7f5c973a57a0 20 read_log 192718'9718 (192718'9717) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:27 2014-08-06 14:51:12.647996 2014-08-13 18:06:06.720978 7f5c973a57a0 20 read_log 192718'9719 (192718'9718) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:28 2014-08-06 14:51:12.672217 2014-08-13 18:06:06.720990 7f5c973a57a0 20 read_log 192718'9720 (192718'9719) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:29 2014-08-06 14:51:12.726388 2014-08-13 18:06:06.721024 7f5c973a57a0 20 read_log 192718'9721 (192718'9720) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:30 2014-08-06 14:51:12.751381 2014-08-13 18:06:06.721040 7f5c973a57a0 20 read_log 192718'9722 (192718'9721) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:31 2014-08-06 14:51:12.841734 2014-08-13 18:06:06.721052 7f5c973a57a0 20 read_log 192718'9723 (192718'9722) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:32 2014-08-06 14:51:12.867699 2014-08-13 18:06:06.721064 7f5c973a57a0 20 read_log 192718'9724 (192718'9723) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:33 2014-08-06 14:51:12.952203 2014-08-13 18:06:06.721076 7f5c973a57a0 20 read_log 192718'9725 (192718'9724) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:34 2014-08-06 14:51:12.978052 2014-08-13 18:06:06.721088 7f5c973a57a0 20 read_log 192718'9726 (192718'9725) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:35 2014-08-06 14:51:13.001808 2014-08-13 18:06:06.721101 7f5c973a57a0 20 read_log 192718'9727 (192718'9726) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:36 2014-08-06 14:51:13.025358 2014-08-13 18:06:06.721115 7f5c973a57a0 20 read_log 192718'9728 (192718'9727) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:37 2014-08-06 14:51:13.206163 2014-08-13 18:06:06.721127 7f5c973a57a0 20 read_log 192718'9729 (192718'9728) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:38 2014-08-06 14:51:13.229839 2014-08-13 18:06:06.721141 7f5c973a57a0 20 read_log 192718'9730 (192718'9729) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:39 2014-08-06 14:51:13.317738 2014-08-13 18:06:06.721162 7f5c973a57a0 20 read_log 192718'9731 (192718'9730) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:40 2014-08-06 14:51:13.343139 2014-08-13 18:06:06.721175 7f5c973a57a0 20 read_log 192718'9732 (192718'9731) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:41 2014-08-06 14:51:13.418396 2014-08-13 18:06:06.721189 7f5c973a57a0 20 read_log 192718'9733 (192718'9732) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:42 2014-08-06 14:51:13.442477 2014-08-13 18:06:06.721206 7f5c973a57a0 20 read_log 192718'9734 (192718'9733) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:43 2014-08-06 14:51:13.465916 2014-08-13 18:06:06.721220 7f5c973a57a0 20 read_log 192718'9735 (192718'9734) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:44 2014-08-06 14:51:13.489250 2014-08-13 18:06:06.721232 7f5c973a57a0 20 read_log 192718'9736 (192718'9735) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:45 2014-08-06 14:51:13.545183 2014-08-13 18:06:06.721245 7f5c973a57a0 20 read_log 192718'9737 (192718'9736) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:46 2014-08-06 14:51:13.569762 2014-08-13 18:06:06.721260 7f5c973a57a0 20 read_log 192718'9738 (192718'9737) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:47 2014-08-06 14:51:13.592102 2014-08-13 18:06:06.721273 7f5c973a57a0 20 read_log 192718'9739 (192718'9738) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:48 2014-08-06 14:51:13.614915 2014-08-13 18:06:06.721285 7f5c973a57a0 20 read_log 192718'9740 (192718'9739) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:49 2014-08-06 14:51:13.695425 2014-08-13 18:06:06.721297 7f5c973a57a0 20 read_log 192718'9741 (192718'9740) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:50 2014-08-06 14:51:13.718871 2014-08-13 18:06:06.721309 7f5c973a57a0 20 read_log 192718'9742 (192718'9741) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:51 2014-08-06 14:51:13.911742 2014-08-13 18:06:06.721325 7f5c973a57a0 20 read_log 192718'9743 (192718'9742) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:52 2014-08-06 14:51:13.937172 2014-08-13 18:06:06.721340 7f5c973a57a0 20 read_log 192718'9744 (192718'9743) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:53 2014-08-06 14:51:13.961359 2014-08-13 18:06:06.721359 7f5c973a57a0 20 read_log 192718'9745 (192718'9744) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:54 2014-08-06 14:51:13.983699 2014-08-13 18:06:06.721374 7f5c973a57a0 20 read_log 192718'9746 (192718'9745) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:55 2014-08-06 14:51:14.061329 2014-08-13 18:06:06.721401 7f5c973a57a0 20 read_log 192718'9747 (192718'9746) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:56 2014-08-06 14:51:14.133308 2014-08-13 18:06:06.721417 7f5c973a57a0 20 read_log 192718'9748 (192718'9747) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:57 2014-08-06 14:51:14.201953 2014-08-13 18:06:06.721429 7f5c973a57a0 20 read_log 192718'9749 (192718'9748) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:58 2014-08-06 14:51:14.224902 2014-08-13 18:06:06.721452 7f5c973a57a0 20 read_log 192718'9750 (192718'9749) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:59 2014-08-06 14:51:14.248547 2014-08-13 18:06:06.721464 7f5c973a57a0 20 read_log 192718'9751 (192718'9750) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:60 2014-08-06 14:51:14.271249 2014-08-13 18:06:06.721491 7f5c973a57a0 20 read_log 192718'9752 (192718'9751) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:61 2014-08-06 14:51:14.358673 2014-08-13 18:06:06.721506 7f5c973a57a0 20 read_log 192718'9753 (192718'9752) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:62 2014-08-06 14:51:14.385336 2014-08-13 18:06:06.721519 7f5c973a57a0 20 read_log 192718'9754 (192718'9753) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:63 2014-08-06 14:51:14.467799 2014-08-13 18:06:06.721531 7f5c973a57a0 20 read_log 192718'9755 (192718'9754) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:64 2014-08-06 14:51:14.494113 2014-08-13 18:06:06.721543 7f5c973a57a0 20 read_log 192718'9756 (192718'9755) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:65 2014-08-06 14:51:14.595811 2014-08-13 18:06:06.721555 7f5c973a57a0 20 read_log 192718'9757 (192718'9756) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:66 2014-08-06 14:51:14.621650 2014-08-13 18:06:06.721572 7f5c973a57a0 20 read_log 192718'9758 (192718'9757) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:67 2014-08-06 14:51:14.687018 2014-08-13 18:06:06.721593 7f5c973a57a0 20 read_log 192718'9759 (192718'9758) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:68 2014-08-06 14:51:14.733342 2014-08-13 18:06:06.721609 7f5c973a57a0 20 read_log 192718'9760 (192718'9759) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:69 2014-08-06 14:51:14.757059 2014-08-13 18:06:06.721625 7f5c973a57a0 20 read_log 192718'9761 (192718'9760) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:70 2014-08-06 14:51:14.782451 2014-08-13 18:06:06.721644 7f5c973a57a0 20 read_log 192718'9762 (192718'9761) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:71 2014-08-06 14:51:14.805746 2014-08-13 18:06:06.721657 7f5c973a57a0 20 read_log 192718'9763 (192718'9762) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:72 2014-08-06 14:51:14.832474 2014-08-13 18:06:06.721672 7f5c973a57a0 20 read_log 192718'9764 (192718'9763) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:73 2014-08-06 14:51:14.883320 2014-08-13 18:06:06.721690 7f5c973a57a0 20 read_log 192718'9765 (192718'9764) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:74 2014-08-06 14:51:14.935358 2014-08-13 18:06:06.721703 7f5c973a57a0 20 read_log 192718'9766 (192718'9765) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:75 2014-08-06 14:51:15.009004 2014-08-13 18:06:06.721718 7f5c973a57a0 20 read_log 192718'9767 (192718'9766) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:76 2014-08-06 14:51:15.068489 2014-08-13 18:06:06.721733 7f5c973a57a0 20 read_log 192718'9768 (192718'9767) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:77 2014-08-06 14:51:15.154761 2014-08-13 18:06:06.721755 7f5c973a57a0 20 read_log 192718'9769 (192718'9768) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:78 2014-08-06 14:51:15.205767 2014-08-13 18:06:06.721776 7f5c973a57a0 20 read_log 192718'9770 (192718'9769) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:79 2014-08-06 14:51:15.248762 2014-08-13 18:06:06.721788 7f5c973a57a0 20 read_log 192718'9771 (192718'9770) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:80 2014-08-06 14:51:15.290874 2014-08-13 18:06:06.721800 7f5c973a57a0 20 read_log 192718'9772 (192718'9771) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:81 2014-08-06 14:51:15.341078 2014-08-13 18:06:06.721815 7f5c973a57a0 20 read_log 192718'9773 (192718'9772) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:82 2014-08-06 14:51:15.391499 2014-08-13 18:06:06.721828 7f5c973a57a0 20 read_log 192718'9774 (192718'9773) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:83 2014-08-06 14:51:15.452023 2014-08-13 18:06:06.721841 7f5c973a57a0 20 read_log 192718'9775 (192718'9774) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:84 2014-08-06 14:51:15.499944 2014-08-13 18:06:06.721853 7f5c973a57a0 20 read_log 192718'9776 (192718'9775) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:85 2014-08-06 14:51:15.561114 2014-08-13 18:06:06.721866 7f5c973a57a0 20 read_log 192718'9777 (192718'9776) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:86 2014-08-06 14:51:15.605101 2014-08-13 18:06:06.721889 7f5c973a57a0 20 read_log 192718'9778 (192718'9777) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:87 2014-08-06 14:51:15.656542 2014-08-13 18:06:06.721901 7f5c973a57a0 20 read_log 192718'9779 (192718'9778) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:88 2014-08-06 14:51:15.708122 2014-08-13 18:06:06.721914 7f5c973a57a0 20 read_log 192718'9780 (192718'9779) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:89 2014-08-06 14:51:15.772837 2014-08-13 18:06:06.721926 7f5c973a57a0 20 read_log 192718'9781 (192718'9780) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:90 2014-08-06 14:51:15.856918 2014-08-13 18:06:06.721938 7f5c973a57a0 20 read_log 192718'9782 (192718'9781) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:91 2014-08-06 14:51:19.303214 2014-08-13 18:06:06.721950 7f5c973a57a0 20 read_log 192718'9783 (192718'9782) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:92 2014-08-06 14:51:19.331714 2014-08-13 18:06:06.721962 7f5c973a57a0 20 read_log 192718'9784 (192718'9783) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:93 2014-08-06 14:51:19.366245 2014-08-13 18:06:06.721974 7f5c973a57a0 20 read_log 192718'9785 (192718'9784) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:94 2014-08-06 14:51:19.393851 2014-08-13 18:06:06.721986 7f5c973a57a0 20 read_log 192718'9786 (192718'9785) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:95 2014-08-06 14:51:19.525371 2014-08-13 18:06:06.721998 7f5c973a57a0 20 read_log 192718'9787 (192718'9786) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:96 2014-08-06 14:51:19.576047 2014-08-13 18:06:06.722010 7f5c973a57a0 20 read_log 192718'9788 (192718'9787) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:97 2014-08-06 14:51:19.603030 2014-08-13 18:06:06.722022 7f5c973a57a0 20 read_log 192718'9789 (192718'9788) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:98 2014-08-06 14:51:19.796585 2014-08-13 18:06:06.722034 7f5c973a57a0 20 read_log 192718'9790 (192718'9789) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:99 2014-08-06 14:51:19.970715 2014-08-13 18:06:06.722054 7f5c973a57a0 20 read_log 192718'9791 (192718'9790) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:100 2014-08-06 14:51:19.998178 2014-08-13 18:06:06.722069 7f5c973a57a0 20 read_log 192718'9792 (192718'9791) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:101 2014-08-06 14:51:20.044751 2014-08-13 18:06:06.722088 7f5c973a57a0 20 read_log 192718'9793 (192718'9792) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:102 2014-08-06 14:51:20.111210 2014-08-13 18:06:06.722104 7f5c973a57a0 20 read_log 192718'9794 (192718'9793) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:103 2014-08-06 14:51:20.281296 2014-08-13 18:06:06.722134 7f5c973a57a0 20 read_log 192718'9795 (192718'9794) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:104 2014-08-06 14:51:20.306003 2014-08-13 18:06:06.722157 7f5c973a57a0 20 read_log 192718'9796 (192718'9795) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:105 2014-08-06 14:51:20.336247 2014-08-13 18:06:06.722170 7f5c973a57a0 20 read_log 192718'9797 (192718'9796) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:106 2014-08-06 14:51:20.371802 2014-08-13 18:06:06.722185 7f5c973a57a0 20 read_log 192718'9798 (192718'9797) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:107 2014-08-06 14:51:20.767198 2014-08-13 18:06:06.722197 7f5c973a57a0 20 read_log 192718'9799 (192718'9798) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:108 2014-08-06 14:51:20.813758 2014-08-13 18:06:06.722209 7f5c973a57a0 20 read_log 192718'9800 (192718'9799) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:109 2014-08-06 14:51:20.839343 2014-08-13 18:06:06.722221 7f5c973a57a0 20 read_log 192718'9801 (192718'9800) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:110 2014-08-06 14:51:20.978255 2014-08-13 18:06:06.722233 7f5c973a57a0 20 read_log 192718'9802 (192718'9801) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:111 2014-08-06 14:51:21.012120 2014-08-13 18:06:06.722251 7f5c973a57a0 20 read_log 192718'9803 (192718'9802) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:112 2014-08-06 14:51:21.147593 2014-08-13 18:06:06.722264 7f5c973a57a0 20 read_log 192718'9804 (192718'9803) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:113 2014-08-06 14:51:21.184801 2014-08-13 18:06:06.722276 7f5c973a57a0 20 read_log 192718'9805 (192718'9804) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:114 2014-08-06 14:51:21.211724 2014-08-13 18:06:06.722291 7f5c973a57a0 20 read_log 192718'9806 (192718'9805) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:115 2014-08-06 14:51:21.241501 2014-08-13 18:06:06.722306 7f5c973a57a0 20 read_log 192718'9807 (192718'9806) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:116 2014-08-06 14:51:21.468656 2014-08-13 18:06:06.722318 7f5c973a57a0 20 read_log 192718'9808 (192718'9807) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:117 2014-08-06 14:51:21.498800 2014-08-13 18:06:06.722330 7f5c973a57a0 20 read_log 192718'9809 (192718'9808) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:118 2014-08-06 14:51:21.539990 2014-08-13 18:06:06.722346 7f5c973a57a0 20 read_log 192718'9810 (192718'9809) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:119 2014-08-06 14:51:21.566708 2014-08-13 18:06:06.722370 7f5c973a57a0 20 read_log 192718'9811 (192718'9810) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:120 2014-08-06 14:51:21.606489 2014-08-13 18:06:06.722387 7f5c973a57a0 20 read_log 192718'9812 (192718'9811) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:121 2014-08-06 14:51:21.796414 2014-08-13 18:06:06.722406 7f5c973a57a0 20 read_log 192718'9813 (192718'9812) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:122 2014-08-06 14:51:22.161288 2014-08-13 18:06:06.722421 7f5c973a57a0 20 read_log 192718'9814 (192718'9813) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:123 2014-08-06 14:51:22.199283 2014-08-13 18:06:06.722434 7f5c973a57a0 20 read_log 192718'9815 (192718'9814) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:124 2014-08-06 14:51:22.231580 2014-08-13 18:06:06.722446 7f5c973a57a0 20 read_log 192718'9816 (192718'9815) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:125 2014-08-06 14:51:22.274521 2014-08-13 18:06:06.722475 7f5c973a57a0 20 read_log 192718'9817 (192718'9816) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:126 2014-08-06 14:51:22.393088 2014-08-13 18:06:06.722492 7f5c973a57a0 20 read_log 192718'9818 (192718'9817) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:127 2014-08-06 14:51:22.421795 2014-08-13 18:06:06.722504 7f5c973a57a0 20 read_log 192718'9819 (192718'9818) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:128 2014-08-06 14:51:22.551853 2014-08-13 18:06:06.722516 7f5c973a57a0 20 read_log 192718'9820 (192718'9819) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:129 2014-08-06 14:51:22.576463 2014-08-13 18:06:06.722529 7f5c973a57a0 20 read_log 192718'9821 (192718'9820) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:130 2014-08-06 14:51:22.602377 2014-08-13 18:06:06.722545 7f5c973a57a0 20 read_log 192718'9822 (192718'9821) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:131 2014-08-06 14:51:22.715852 2014-08-13 18:06:06.722558 7f5c973a57a0 20 read_log 192718'9823 (192718'9822) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:132 2014-08-06 14:51:22.863429 2014-08-13 18:06:06.722570 7f5c973a57a0 20 read_log 192718'9824 (192718'9823) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:133 2014-08-06 14:51:22.900727 2014-08-13 18:06:06.722582 7f5c973a57a0 20 read_log 192718'9825 (192718'9824) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:134 2014-08-06 14:51:22.937819 2014-08-13 18:06:06.722594 7f5c973a57a0 20 read_log 192718'9826 (192718'9825) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:135 2014-08-06 14:51:22.997844 2014-08-13 18:06:06.722606 7f5c973a57a0 20 read_log 192718'9827 (192718'9826) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:136 2014-08-06 14:51:23.157507 2014-08-13 18:06:06.722618 7f5c973a57a0 20 read_log 192718'9828 (192718'9827) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:137 2014-08-06 14:51:23.183973 2014-08-13 18:06:06.722630 7f5c973a57a0 20 read_log 192718'9829 (192718'9828) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:138 2014-08-06 14:51:23.306132 2014-08-13 18:06:06.722642 7f5c973a57a0 20 read_log 192718'9830 (192718'9829) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:139 2014-08-06 14:51:23.335649 2014-08-13 18:06:06.722654 7f5c973a57a0 20 read_log 192718'9831 (192718'9830) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:140 2014-08-06 14:51:23.366506 2014-08-13 18:06:06.722666 7f5c973a57a0 20 read_log 192718'9832 (192718'9831) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:141 2014-08-06 14:51:23.453847 2014-08-13 18:06:06.722678 7f5c973a57a0 20 read_log 192718'9833 (192718'9832) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:142 2014-08-06 14:51:23.600013 2014-08-13 18:06:06.722690 7f5c973a57a0 20 read_log 192718'9834 (192718'9833) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:143 2014-08-06 14:51:23.640648 2014-08-13 18:06:06.722702 7f5c973a57a0 20 read_log 192718'9835 (192718'9834) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:144 2014-08-06 14:51:23.669549 2014-08-13 18:06:06.722714 7f5c973a57a0 20 read_log 192718'9836 (192718'9835) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:145 2014-08-06 14:51:23.838736 2014-08-13 18:06:06.722726 7f5c973a57a0 20 read_log 192718'9837 (192718'9836) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:146 2014-08-06 14:51:24.039390 2014-08-13 18:06:06.722738 7f5c973a57a0 20 read_log 192718'9838 (192718'9837) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:147 2014-08-06 14:51:24.066889 2014-08-13 18:06:06.722750 7f5c973a57a0 20 read_log 192718'9839 (192718'9838) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:148 2014-08-06 14:51:24.109291 2014-08-13 18:06:06.722762 7f5c973a57a0 20 read_log 192718'9840 (192718'9839) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:149 2014-08-06 14:51:24.166112 2014-08-13 18:06:06.722790 7f5c973a57a0 20 read_log 192718'9841 (192718'9840) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:150 2014-08-06 14:51:24.324455 2014-08-13 18:06:06.722807 7f5c973a57a0 20 read_log 192718'9842 (192718'9841) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:151 2014-08-06 14:51:24.358059 2014-08-13 18:06:06.722819 7f5c973a57a0 20 read_log 192718'9843 (192718'9842) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:152 2014-08-06 14:51:24.549623 2014-08-13 18:06:06.722831 7f5c973a57a0 20 read_log 192718'9844 (192718'9843) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:153 2014-08-06 14:51:24.583303 2014-08-13 18:06:06.722853 7f5c973a57a0 20 read_log 192718'9845 (192718'9844) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:154 2014-08-06 14:51:24.629149 2014-08-13 18:06:06.722865 7f5c973a57a0 20 read_log 192718'9846 (192718'9845) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:155 2014-08-06 14:51:24.855637 2014-08-13 18:06:06.722878 7f5c973a57a0 20 read_log 192718'9847 (192718'9846) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:156 2014-08-06 14:51:24.887613 2014-08-13 18:06:06.722890 7f5c973a57a0 20 read_log 192718'9848 (192718'9847) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:157 2014-08-06 14:51:24.915650 2014-08-13 18:06:06.722902 7f5c973a57a0 20 read_log 192718'9849 (192718'9848) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:158 2014-08-06 14:51:24.945462 2014-08-13 18:06:06.722914 7f5c973a57a0 20 read_log 192718'9850 (192718'9849) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:159 2014-08-06 14:51:24.971432 2014-08-13 18:06:06.722926 7f5c973a57a0 20 read_log 192718'9851 (192718'9850) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:160 2014-08-06 14:51:24.995565 2014-08-13 18:06:06.722940 7f5c973a57a0 20 read_log 192718'9852 (192718'9851) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:161 2014-08-06 14:51:25.025328 2014-08-13 18:06:06.722952 7f5c973a57a0 20 read_log 192718'9853 (192718'9852) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:162 2014-08-06 14:51:25.153566 2014-08-13 18:06:06.722967 7f5c973a57a0 20 read_log 192718'9854 (192718'9853) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:163 2014-08-06 14:51:25.300566 2014-08-13 18:06:06.722979 7f5c973a57a0 20 read_log 192718'9855 (192718'9854) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:164 2014-08-06 14:51:25.329505 2014-08-13 18:06:06.722993 7f5c973a57a0 20 read_log 192718'9856 (192718'9855) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:165 2014-08-06 14:51:25.360517 2014-08-13 18:06:06.723005 7f5c973a57a0 20 read_log 192718'9857 (192718'9856) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:166 2014-08-06 14:51:25.391367 2014-08-13 18:06:06.723022 7f5c973a57a0 20 read_log 192718'9858 (192718'9857) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:167 2014-08-06 14:51:25.543055 2014-08-13 18:06:06.723034 7f5c973a57a0 20 read_log 192718'9859 (192718'9858) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:168 2014-08-06 14:51:25.723032 2014-08-13 18:06:06.723046 7f5c973a57a0 20 read_log 192718'9860 (192718'9859) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:169 2014-08-06 14:51:25.750977 2014-08-13 18:06:06.723059 7f5c973a57a0 20 read_log 192718'9861 (192718'9860) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:170 2014-08-06 14:51:25.776152 2014-08-13 18:06:06.723073 7f5c973a57a0 20 read_log 192718'9862 (192718'9861) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:171 2014-08-06 14:51:25.806821 2014-08-13 18:06:06.723085 7f5c973a57a0 20 read_log 192718'9863 (192718'9862) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:172 2014-08-06 14:51:25.847958 2014-08-13 18:06:06.723097 7f5c973a57a0 20 read_log 192718'9864 (192718'9863) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:173 2014-08-06 14:51:26.006887 2014-08-13 18:06:06.723121 7f5c973a57a0 20 read_log 192718'9865 (192718'9864) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:174 2014-08-06 14:51:26.033012 2014-08-13 18:06:06.723136 7f5c973a57a0 20 read_log 192718'9866 (192718'9865) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:175 2014-08-06 14:51:26.058666 2014-08-13 18:06:06.723153 7f5c973a57a0 20 read_log 192718'9867 (192718'9866) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:176 2014-08-06 14:51:26.093614 2014-08-13 18:06:06.723168 7f5c973a57a0 20 read_log 192718'9868 (192718'9867) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:177 2014-08-06 14:51:26.117815 2014-08-13 18:06:06.723180 7f5c973a57a0 20 read_log 192718'9869 (192718'9868) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:178 2014-08-06 14:51:26.160042 2014-08-13 18:06:06.723193 7f5c973a57a0 20 read_log 192718'9870 (192718'9869) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:179 2014-08-06 14:51:26.185401 2014-08-13 18:06:06.723207 7f5c973a57a0 20 read_log 192718'9871 (192718'9870) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:180 2014-08-06 14:51:26.406770 2014-08-13 18:06:06.723219 7f5c973a57a0 20 read_log 192718'9872 (192718'9871) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:181 2014-08-06 14:51:26.444029 2014-08-13 18:06:06.723231 7f5c973a57a0 20 read_log 192718'9873 (192718'9872) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:182 2014-08-06 14:51:26.476147 2014-08-13 18:06:06.723245 7f5c973a57a0 20 read_log 192718'9874 (192718'9873) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:183 2014-08-06 14:51:26.536014 2014-08-13 18:06:06.723260 7f5c973a57a0 20 read_log 192718'9875 (192718'9874) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:184 2014-08-06 14:51:26.715145 2014-08-13 18:06:06.723274 7f5c973a57a0 20 read_log 192718'9876 (192718'9875) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:185 2014-08-06 14:51:26.833460 2014-08-13 18:06:06.723286 7f5c973a57a0 20 read_log 192718'9877 (192718'9876) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:186 2014-08-06 14:51:26.882576 2014-08-13 18:06:06.723298 7f5c973a57a0 20 read_log 192718'9878 (192718'9877) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:187 2014-08-06 14:51:26.923674 2014-08-13 18:06:06.723313 7f5c973a57a0 20 read_log 192718'9879 (192718'9878) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:188 2014-08-06 14:51:26.963566 2014-08-13 18:06:06.723326 7f5c973a57a0 20 read_log 192718'9880 (192718'9879) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:189 2014-08-06 14:51:27.473340 2014-08-13 18:06:06.723338 7f5c973a57a0 20 read_log 192718'9881 (192718'9880) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:190 2014-08-06 14:51:27.637534 2014-08-13 18:06:06.723354 7f5c973a57a0 20 read_log 192718'9882 (192718'9881) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:191 2014-08-06 14:51:28.188279 2014-08-13 18:06:06.723366 7f5c973a57a0 20 read_log 192718'9883 (192718'9882) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:192 2014-08-06 14:51:28.962986 2014-08-13 18:06:06.723378 7f5c973a57a0 20 read_log 192718'9884 (192718'9883) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:193 2014-08-06 14:51:29.149277 2014-08-13 18:06:06.723406 7f5c973a57a0 20 read_log 192718'9885 (192718'9884) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:194 2014-08-06 14:51:29.183877 2014-08-13 18:06:06.723421 7f5c973a57a0 20 read_log 192718'9886 (192718'9885) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:195 2014-08-06 14:51:29.712400 2014-08-13 18:06:06.723443 7f5c973a57a0 20 read_log 192718'9887 (192718'9886) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:196 2014-08-06 14:51:30.333811 2014-08-13 18:06:06.723480 7f5c973a57a0 20 read_log 192718'9888 (192718'9887) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:197 2014-08-06 14:51:30.492833 2014-08-13 18:06:06.723529 7f5c973a57a0 20 read_log 192718'9889 (192718'9888) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:198 2014-08-06 14:51:30.651491 2014-08-13 18:06:06.723546 7f5c973a57a0 20 read_log 192718'9890 (192718'9889) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:199 2014-08-06 14:51:31.183686 2014-08-13 18:06:06.723559 7f5c973a57a0 20 read_log 192718'9891 (192718'9890) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:200 2014-08-06 14:51:31.669534 2014-08-13 18:06:06.723571 7f5c973a57a0 20 read_log 192718'9892 (192718'9891) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:201 2014-08-06 14:51:31.868342 2014-08-13 18:06:06.723593 7f5c973a57a0 20 read_log 192718'9893 (192718'9892) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:202 2014-08-06 14:51:32.072946 2014-08-13 18:06:06.723608 7f5c973a57a0 20 read_log 192718'9894 (192718'9893) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:203 2014-08-06 14:51:32.666981 2014-08-13 18:06:06.723621 7f5c973a57a0 20 read_log 192718'9895 (192718'9894) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:204 2014-08-06 14:51:34.486489 2014-08-13 18:06:06.723633 7f5c973a57a0 20 read_log 192718'9896 (192718'9895) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:205 2014-08-06 14:51:34.604728 2014-08-13 18:06:06.723645 7f5c973a57a0 20 read_log 192718'9897 (192718'9896) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:206 2014-08-06 14:51:34.965504 2014-08-13 18:06:06.723657 7f5c973a57a0 20 read_log 192718'9898 (192718'9897) modify 32c071bb/core.58593+95+2/head//24 by client.412318.0:207 2014-08-06 14:51:35.233551 2014-08-13 18:06:06.723680 7f5c973a57a0 20 read_log 192802'10243 (192718'9898) clean 32c071bb/core.58593+95+2/head//24 by osd.373.0:11971 2014-08-11 10:25:00.978538 2014-08-13 18:06:06.723694 7f5c973a57a0 20 read_log 192802'10244 (192802'10243) delete 32c071bb/core.58593+95+2/head//24 by osd.373.0:11972 2014-08-11 10:25:02.862111 2014-08-13 18:06:06.723707 7f5c973a57a0 20 read_log 192834'10518 (0'0) promote d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by osd.373.0:12325 2014-08-11 15:23:36.173587 2014-08-13 18:06:06.723719 7f5c973a57a0 20 read_log 192834'10519 (192834'10518) modify d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.933448.0:125 2014-08-11 15:23:36.161343 2014-08-13 18:06:06.723732 7f5c973a57a0 20 read_log 192834'10538 (0'0) promote c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by osd.373.0:12345 2014-08-11 15:23:38.870934 2014-08-13 18:06:06.723745 7f5c973a57a0 20 read_log 192834'10539 (192834'10538) modify c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.933448.0:839 2014-08-11 15:23:38.863269 2014-08-13 18:06:06.723762 7f5c973a57a0 20 read_log 192834'10544 (0'0) promote 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by osd.373.0:12351 2014-08-11 15:23:39.594380 2014-08-13 18:06:06.723775 7f5c973a57a0 20 read_log 192834'10545 (192834'10544) modify 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.933448.0:1028 2014-08-11 15:23:39.587821 2014-08-13 18:06:06.723787 7f5c973a57a0 20 read_log 192834'10577 (0'0) promote e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by osd.373.0:12384 2014-08-11 15:23:46.813383 2014-08-13 18:06:06.723799 7f5c973a57a0 20 read_log 192834'10578 (192834'10577) modify e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.933448.0:2438 2014-08-11 15:23:46.786896 2014-08-13 18:06:06.723812 7f5c973a57a0 20 read_log 192834'10596 (0'0) promote bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by osd.373.0:12403 2014-08-11 15:23:53.786649 2014-08-13 18:06:06.723824 7f5c973a57a0 20 read_log 192834'10597 (192834'10596) modify bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.933448.0:3827 2014-08-11 15:23:53.753419 2014-08-13 18:06:06.723844 7f5c973a57a0 20 read_log 192834'10608 (0'0) promote ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by osd.373.0:12415 2014-08-11 15:23:59.176341 2014-08-13 18:06:06.723857 7f5c973a57a0 20 read_log 192834'10609 (192834'10608) modify ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.933448.0:4878 2014-08-11 15:23:59.167274 2014-08-13 18:06:06.723869 7f5c973a57a0 20 read_log 192834'10642 (0'0) promote b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by osd.373.0:12449 2014-08-11 15:24:06.249942 2014-08-13 18:06:06.723881 7f5c973a57a0 20 read_log 192834'10643 (192834'10642) modify b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.933448.0:6654 2014-08-11 15:24:06.242303 2014-08-13 18:06:06.723894 7f5c973a57a0 20 read_log 192834'10646 (0'0) promote ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by osd.373.0:12453 2014-08-11 15:24:06.700016 2014-08-13 18:06:06.723906 7f5c973a57a0 20 read_log 192834'10647 (192834'10646) modify ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.933448.0:6772 2014-08-11 15:24:06.691857 2014-08-13 18:06:06.723919 7f5c973a57a0 20 read_log 192834'10668 (0'0) promote f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by osd.373.0:12475 2014-08-11 15:24:11.481453 2014-08-13 18:06:06.723931 7f5c973a57a0 20 read_log 192834'10669 (192834'10668) modify f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.933448.0:7808 2014-08-11 15:24:11.474783 2014-08-13 18:06:06.723943 7f5c973a57a0 20 read_log 192834'10684 (0'0) promote c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by osd.373.0:12491 2014-08-11 15:24:15.606739 2014-08-13 18:06:06.723955 7f5c973a57a0 20 read_log 192834'10685 (192834'10684) modify c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.933448.0:8837 2014-08-11 15:24:15.598834 2014-08-13 18:06:06.723967 7f5c973a57a0 20 read_log 192834'10688 (0'0) promote cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by osd.373.0:12495 2014-08-11 15:24:17.461211 2014-08-13 18:06:06.723982 7f5c973a57a0 20 read_log 192834'10689 (192834'10688) modify cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.933448.0:9074 2014-08-11 15:24:17.454593 2014-08-13 18:06:06.723997 7f5c973a57a0 20 read_log 192834'10698 (0'0) promote e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by osd.373.0:12505 2014-08-11 15:24:18.759082 2014-08-13 18:06:06.724009 7f5c973a57a0 20 read_log 192834'10699 (192834'10698) modify e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.933448.0:9419 2014-08-11 15:24:18.751768 2014-08-13 18:06:06.724022 7f5c973a57a0 20 read_log 192834'10704 (0'0) promote b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by osd.373.0:12511 2014-08-11 15:24:19.734582 2014-08-13 18:06:06.724034 7f5c973a57a0 20 read_log 192834'10705 (192834'10704) modify b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.933448.0:9663 2014-08-11 15:24:19.728104 2014-08-13 18:06:06.724046 7f5c973a57a0 20 read_log 192834'10708 (0'0) promote 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by osd.373.0:12515 2014-08-11 15:24:20.909771 2014-08-13 18:06:06.724059 7f5c973a57a0 20 read_log 192834'10709 (192834'10708) modify 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.933448.0:9930 2014-08-11 15:24:20.900925 2014-08-13 18:06:06.724074 7f5c973a57a0 20 read_log 192834'10712 (0'0) promote c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by osd.373.0:12519 2014-08-11 15:24:22.050722 2014-08-13 18:06:06.724086 7f5c973a57a0 20 read_log 192834'10713 (192834'10712) modify c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.933448.0:10194 2014-08-11 15:24:22.043450 2014-08-13 18:06:06.724111 7f5c973a57a0 20 read_log 192834'10714 (0'0) promote 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by osd.373.0:12521 2014-08-11 15:24:22.764161 2014-08-13 18:06:06.724126 7f5c973a57a0 20 read_log 192834'10715 (192834'10714) modify 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.933448.0:10342 2014-08-11 15:24:22.755024 2014-08-13 18:06:06.724139 7f5c973a57a0 20 read_log 192834'10744 (0'0) promote 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by osd.373.0:12551 2014-08-11 15:24:31.217190 2014-08-13 18:06:06.724161 7f5c973a57a0 20 read_log 192834'10745 (192834'10744) modify 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.933448.0:12134 2014-08-11 15:24:31.209838 2014-08-13 18:06:06.724174 7f5c973a57a0 20 read_log 192834'10778 (0'0) promote 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by osd.373.0:12585 2014-08-11 15:24:47.920516 2014-08-13 18:06:06.724187 7f5c973a57a0 20 read_log 192834'10779 (192834'10778) modify 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.933448.0:14979 2014-08-11 15:24:47.911925 2014-08-13 18:06:06.724199 7f5c973a57a0 20 read_log 192834'10788 (0'0) promote 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by osd.373.0:12595 2014-08-11 15:24:49.911588 2014-08-13 18:06:06.724212 7f5c973a57a0 20 read_log 192834'10789 (192834'10788) modify 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.933448.0:15507 2014-08-11 15:24:49.903752 2014-08-13 18:06:06.724224 7f5c973a57a0 20 read_log 192834'10802 (0'0) promote 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by osd.373.0:12609 2014-08-11 15:24:52.575819 2014-08-13 18:06:06.724236 7f5c973a57a0 20 read_log 192834'10803 (192834'10802) modify 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.933448.0:16033 2014-08-11 15:24:52.358432 2014-08-13 18:06:06.724253 7f5c973a57a0 20 read_log 192834'10882 (192834'10519) modify d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.933448.0:22416 2014-08-11 15:25:37.837977 2014-08-13 18:06:06.724269 7f5c973a57a0 20 read_log 192834'10892 (192834'10539) modify c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.933448.0:23130 2014-08-11 15:25:39.041513 2014-08-13 18:06:06.724281 7f5c973a57a0 20 read_log 192834'10895 (192834'10545) modify 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.933448.0:23319 2014-08-11 15:25:39.223281 2014-08-13 18:06:06.724293 7f5c973a57a0 20 read_log 192834'10912 (192834'10578) modify e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.933448.0:24729 2014-08-11 15:25:41.224272 2014-08-13 18:06:06.724309 7f5c973a57a0 20 read_log 192834'10921 (192834'10597) modify bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.933448.0:26118 2014-08-11 15:25:42.989934 2014-08-13 18:06:06.724321 7f5c973a57a0 20 read_log 192834'10926 (192834'10609) modify ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.933448.0:27169 2014-08-11 15:25:44.181224 2014-08-13 18:06:06.724333 7f5c973a57a0 20 read_log 192834'10943 (192834'10643) modify b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.933448.0:28945 2014-08-11 15:25:46.866149 2014-08-13 18:06:06.724349 7f5c973a57a0 20 read_log 192834'10945 (192834'10647) modify ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.933448.0:29063 2014-08-11 15:25:46.965168 2014-08-13 18:06:06.724362 7f5c973a57a0 20 read_log 192834'10956 (192834'10669) modify f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.933448.0:30099 2014-08-11 15:25:48.370737 2014-08-13 18:06:06.724382 7f5c973a57a0 20 read_log 192834'10964 (192834'10685) modify c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.933448.0:31128 2014-08-11 15:25:49.680249 2014-08-13 18:06:06.724410 7f5c973a57a0 20 read_log 192834'10966 (192834'10689) modify cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.933448.0:31365 2014-08-11 15:25:49.974738 2014-08-13 18:06:06.724433 7f5c973a57a0 20 read_log 192834'10971 (192834'10699) modify e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.933448.0:31710 2014-08-11 15:25:50.419626 2014-08-13 18:06:06.724449 7f5c973a57a0 20 read_log 192834'10974 (192834'10705) modify b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.933448.0:31954 2014-08-11 15:25:50.872526 2014-08-13 18:06:06.724461 7f5c973a57a0 20 read_log 192834'10976 (192834'10709) modify 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.933448.0:32221 2014-08-11 15:25:51.263213 2014-08-13 18:06:06.724480 7f5c973a57a0 20 read_log 192834'10978 (192834'10713) modify c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.933448.0:32485 2014-08-11 15:25:51.895350 2014-08-13 18:06:06.724496 7f5c973a57a0 20 read_log 192834'10979 (192834'10715) modify 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.933448.0:32633 2014-08-11 15:25:52.136414 2014-08-13 18:06:06.724509 7f5c973a57a0 20 read_log 192834'10994 (192834'10745) modify 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.933448.0:34425 2014-08-11 15:25:54.995856 2014-08-13 18:06:06.724528 7f5c973a57a0 20 read_log 192834'11011 (192834'10779) modify 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.933448.0:37270 2014-08-11 15:25:59.241877 2014-08-13 18:06:06.724544 7f5c973a57a0 20 read_log 192834'11016 (192834'10789) modify 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.933448.0:37798 2014-08-11 15:26:00.072755 2014-08-13 18:06:06.724557 7f5c973a57a0 20 read_log 192834'11023 (192834'10803) modify 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.933448.0:38324 2014-08-11 15:26:00.758785 2014-08-13 18:06:06.724570 7f5c973a57a0 20 read_log 192834'11063 (0'0) promote f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by osd.373.0:12870 2014-08-11 15:26:14.531449 2014-08-13 18:06:06.724586 7f5c973a57a0 20 read_log 192834'11064 (192834'11063) modify f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.1883634.0:74 2014-08-11 15:26:14.521605 2014-08-13 18:06:06.724601 7f5c973a57a0 20 read_log 192834'11075 (0'0) promote a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by osd.373.0:12882 2014-08-11 15:26:18.759636 2014-08-13 18:06:06.724613 7f5c973a57a0 20 read_log 192834'11076 (192834'11075) modify a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.1883634.0:1175 2014-08-11 15:26:18.750764 2014-08-13 18:06:06.724633 7f5c973a57a0 20 read_log 192834'11093 (0'0) promote 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by osd.373.0:12900 2014-08-11 15:26:24.859237 2014-08-13 18:06:06.724647 7f5c973a57a0 20 read_log 192834'11094 (192834'11093) modify 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.1883634.0:2831 2014-08-11 15:26:24.850752 2014-08-13 18:06:06.724659 7f5c973a57a0 20 read_log 192834'11103 (0'0) promote ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.373.0:12910 2014-08-11 15:26:29.423452 2014-08-13 18:06:06.724671 7f5c973a57a0 20 read_log 192834'11104 (192834'11103) modify ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.1883634.0:4121 2014-08-11 15:26:29.415740 2014-08-13 18:06:06.724684 7f5c973a57a0 20 read_log 192834'11111 (0'0) promote 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by osd.373.0:12918 2014-08-11 15:26:31.666131 2014-08-13 18:06:06.724696 7f5c973a57a0 20 read_log 192834'11112 (192834'11111) modify 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.1883634.0:4721 2014-08-11 15:26:31.659131 2014-08-13 18:06:06.724716 7f5c973a57a0 20 read_log 192834'11129 (0'0) promote bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by osd.373.0:12936 2014-08-11 15:26:34.672265 2014-08-13 18:06:06.724731 7f5c973a57a0 20 read_log 192834'11130 (192834'11129) modify bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.1883634.0:5513 2014-08-11 15:26:34.664094 2014-08-13 18:06:06.724743 7f5c973a57a0 20 read_log 192834'11147 (0'0) promote 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:12954 2014-08-11 15:26:37.945386 2014-08-13 18:06:06.724756 7f5c973a57a0 20 read_log 192834'11148 (192834'11147) modify 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.1883634.0:6393 2014-08-11 15:26:37.938043 2014-08-13 18:06:06.724768 7f5c973a57a0 20 read_log 192834'11199 (0'0) promote a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by osd.373.0:13006 2014-08-11 15:26:46.760513 2014-08-13 18:06:06.724780 7f5c973a57a0 20 read_log 192834'11200 (192834'11199) modify a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.1883634.0:8536 2014-08-11 15:26:46.753206 2014-08-13 18:06:06.724801 7f5c973a57a0 20 read_log 192834'11207 (0'0) promote 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by osd.373.0:13014 2014-08-11 15:26:49.096819 2014-08-13 18:06:06.724817 7f5c973a57a0 20 read_log 192834'11208 (192834'11207) modify 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.1883634.0:9053 2014-08-11 15:26:49.089815 2014-08-13 18:06:06.724829 7f5c973a57a0 20 read_log 192834'11225 (0'0) promote 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13032 2014-08-11 15:26:52.515584 2014-08-13 18:06:06.724842 7f5c973a57a0 20 read_log 192834'11226 (192834'11225) modify 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.1883634.0:9821 2014-08-11 15:26:52.506138 2014-08-13 18:06:06.724857 7f5c973a57a0 20 read_log 192834'11231 (0'0) promote 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13038 2014-08-11 15:26:56.371467 2014-08-13 18:06:06.724869 7f5c973a57a0 20 read_log 192834'11232 (192834'11231) modify 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.1883634.0:10722 2014-08-11 15:26:56.362788 2014-08-13 18:06:06.724886 7f5c973a57a0 20 read_log 192834'11241 (0'0) promote 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13048 2014-08-11 15:26:59.394020 2014-08-13 18:06:06.724899 7f5c973a57a0 20 read_log 192834'11242 (192834'11241) modify 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.1883634.0:11328 2014-08-11 15:26:59.385357 2014-08-13 18:06:06.724911 7f5c973a57a0 20 read_log 192834'11263 (0'0) promote 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by osd.373.0:13070 2014-08-11 15:27:06.169443 2014-08-13 18:06:06.724928 7f5c973a57a0 20 read_log 192834'11264 (192834'11263) modify 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.1883634.0:12803 2014-08-11 15:27:06.161641 2014-08-13 18:06:06.724940 7f5c973a57a0 20 read_log 192834'11273 (0'0) promote 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13080 2014-08-11 15:27:09.995907 2014-08-13 18:06:06.724953 7f5c973a57a0 20 read_log 192834'11274 (192834'11273) modify 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.1883634.0:13532 2014-08-11 15:27:09.987777 2014-08-13 18:06:06.724965 7f5c973a57a0 20 read_log 192834'11305 (0'0) promote 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by osd.373.0:13112 2014-08-11 15:27:19.988389 2014-08-13 18:06:06.724982 7f5c973a57a0 20 read_log 192834'11306 (192834'11305) modify 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.1883634.0:15531 2014-08-11 15:27:19.979134 2014-08-13 18:06:06.725004 7f5c973a57a0 20 read_log 192834'11309 (0'0) promote 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by osd.373.0:13116 2014-08-11 15:27:21.443821 2014-08-13 18:06:06.725032 7f5c973a57a0 20 read_log 192834'11310 (192834'11309) modify 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.1883634.0:15750 2014-08-11 15:27:21.434918 2014-08-13 18:06:06.725046 7f5c973a57a0 20 read_log 192834'11323 (0'0) promote ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by osd.373.0:13130 2014-08-11 15:27:27.101938 2014-08-13 18:06:06.725058 7f5c973a57a0 20 read_log 192834'11324 (192834'11323) modify ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.1883634.0:16336 2014-08-11 15:27:27.092626 2014-08-13 18:06:06.725081 7f5c973a57a0 20 read_log 192834'11347 (0'0) promote f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by osd.373.0:13154 2014-08-11 15:27:36.760480 2014-08-13 18:06:06.725094 7f5c973a57a0 20 read_log 192834'11348 (192834'11347) modify f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.1883634.0:17582 2014-08-11 15:27:36.750764 2014-08-13 18:06:06.725106 7f5c973a57a0 20 read_log 192834'11361 (0'0) promote e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by osd.373.0:13168 2014-08-11 15:27:44.822786 2014-08-13 18:06:06.725129 7f5c973a57a0 20 read_log 192834'11362 (192834'11361) modify e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.1883634.0:18488 2014-08-11 15:27:44.815393 2014-08-13 18:06:06.725142 7f5c973a57a0 20 read_log 192834'11371 (0'0) promote 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by osd.373.0:13178 2014-08-11 15:27:47.991888 2014-08-13 18:06:06.725162 7f5c973a57a0 20 read_log 192834'11372 (192834'11371) modify 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.1883634.0:18880 2014-08-11 15:27:47.980740 2014-08-13 18:06:06.725175 7f5c973a57a0 20 read_log 192834'11375 (0'0) promote eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by osd.373.0:13182 2014-08-11 15:27:49.976816 2014-08-13 18:06:06.725190 7f5c973a57a0 20 read_log 192834'11376 (192834'11375) modify eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.1883634.0:19206 2014-08-11 15:27:49.966867 2014-08-13 18:06:06.725205 7f5c973a57a0 20 read_log 192834'11391 (0'0) promote e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by osd.373.0:13197 2014-08-11 15:27:55.322916 2014-08-13 18:06:06.725217 7f5c973a57a0 20 read_log 192834'11392 (192834'11391) modify e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.1883634.0:20118 2014-08-11 15:27:55.314967 2014-08-13 18:06:06.725230 7f5c973a57a0 20 read_log 192834'11403 (0'0) promote b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by osd.373.0:13209 2014-08-11 15:27:57.009185 2014-08-13 18:06:06.725245 7f5c973a57a0 20 read_log 192834'11404 (192834'11403) modify b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.1883634.0:20439 2014-08-11 15:27:56.999045 2014-08-13 18:06:06.725258 7f5c973a57a0 20 read_log 192834'11439 (0'0) promote 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by osd.373.0:13245 2014-08-11 15:28:10.436394 2014-08-13 18:06:06.725270 7f5c973a57a0 20 read_log 192834'11440 (192834'11439) modify 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.1883634.0:22667 2014-08-11 15:28:10.426978 2014-08-13 18:06:06.725282 7f5c973a57a0 20 read_log 192834'11447 (192834'11064) modify f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.1883634.0:23398 2014-08-11 15:28:16.267977 2014-08-13 18:06:06.725295 7f5c973a57a0 20 read_log 192834'11448 (192834'10892) clean c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by osd.373.0:13254 2014-08-11 15:28:16.302954 2014-08-13 18:06:06.725328 7f5c973a57a0 20 read_log 192834'11449 (192834'11011) clean 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by osd.373.0:13255 2014-08-11 15:28:16.304265 2014-08-13 18:06:06.725344 7f5c973a57a0 20 read_log 192834'11451 (192834'11232) clean 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13257 2014-08-11 15:28:16.421765 2014-08-13 18:06:06.725356 7f5c973a57a0 20 read_log 192834'11452 (192834'11242) clean 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13258 2014-08-11 15:28:16.422215 2014-08-13 18:06:06.725368 7f5c973a57a0 20 read_log 192834'11453 (192834'11274) clean 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13259 2014-08-11 15:28:16.423152 2014-08-13 18:06:06.725381 7f5c973a57a0 20 read_log 192834'11454 (192834'10943) clean b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by osd.373.0:13260 2014-08-11 15:28:16.432021 2014-08-13 18:06:06.725393 7f5c973a57a0 20 read_log 192834'11455 (192834'11226) clean 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13261 2014-08-11 15:28:16.432323 2014-08-13 18:06:06.725406 7f5c973a57a0 20 read_log 192834'11456 (192834'10964) clean c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by osd.373.0:13262 2014-08-11 15:28:16.436294 2014-08-13 18:06:06.725418 7f5c973a57a0 20 read_log 192834'11457 (192834'10921) clean bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by osd.373.0:13263 2014-08-11 15:28:16.440424 2014-08-13 18:06:06.725430 7f5c973a57a0 20 read_log 192834'11458 (192834'11148) clean 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13264 2014-08-11 15:28:16.510075 2014-08-13 18:06:06.725442 7f5c973a57a0 20 read_log 192834'11461 (192834'11451) delete 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13267 2014-08-11 15:28:16.753297 2014-08-13 18:06:06.725454 7f5c973a57a0 20 read_log 192834'11462 (192834'11452) delete 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13268 2014-08-11 15:28:16.759060 2014-08-13 18:06:06.725466 7f5c973a57a0 20 read_log 192834'11463 (192834'11453) delete 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13269 2014-08-11 15:28:16.762956 2014-08-13 18:06:06.725478 7f5c973a57a0 20 read_log 192834'11464 (192834'11455) delete 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13270 2014-08-11 15:28:16.766905 2014-08-13 18:06:06.725491 7f5c973a57a0 20 read_log 192834'11465 (192834'11458) delete 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13271 2014-08-11 15:28:16.771113 2014-08-13 18:06:06.725503 7f5c973a57a0 20 read_log 192834'11466 (192834'11104) clean ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.373.0:13272 2014-08-11 15:28:16.948197 2014-08-13 18:06:06.725515 7f5c973a57a0 20 read_log 192834'11473 (192834'11076) modify a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.1883634.0:24499 2014-08-11 15:28:19.142943 2014-08-13 18:06:06.725527 7f5c973a57a0 20 read_log 192834'11482 (192834'11094) modify 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.1883634.0:26155 2014-08-11 15:28:22.684851 2014-08-13 18:06:06.725539 7f5c973a57a0 20 read_log 192834'11487 (192834'11466) modify ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.1883634.0:27445 2014-08-11 15:28:24.476961 2014-08-13 18:06:06.725552 7f5c973a57a0 20 read_log 192834'11491 (192834'11112) modify 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.1883634.0:28045 2014-08-11 15:28:25.301791 2014-08-13 18:06:06.725564 7f5c973a57a0 20 read_log 192834'11500 (192834'11130) modify bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.1883634.0:28837 2014-08-11 15:28:26.360935 2014-08-13 18:06:06.725629 7f5c973a57a0 20 read_log 192834'11509 (0'0) promote 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.373.0:13315 2014-08-11 15:28:27.636238 2014-08-13 18:06:06.725646 7f5c973a57a0 20 read_log 192834'11510 (192834'11509) modify 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.1883634.0:29717 2014-08-11 15:28:27.603446 2014-08-13 18:06:06.725659 7f5c973a57a0 20 read_log 192834'11536 (192834'11200) modify a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.1883634.0:31860 2014-08-11 15:28:31.050127 2014-08-13 18:06:06.725672 7f5c973a57a0 20 read_log 192834'11540 (192834'11208) modify 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.1883634.0:32377 2014-08-11 15:28:31.747207 2014-08-13 18:06:06.725686 7f5c973a57a0 20 read_log 192834'11549 (0'0) promote 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.373.0:13355 2014-08-11 15:28:32.850543 2014-08-13 18:06:06.725699 7f5c973a57a0 20 read_log 192834'11550 (192834'11549) modify 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.1883634.0:33145 2014-08-11 15:28:32.823058 2014-08-13 18:06:06.725711 7f5c973a57a0 20 read_log 192834'11553 (0'0) promote 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.373.0:13359 2014-08-11 15:28:34.195532 2014-08-13 18:06:06.725724 7f5c973a57a0 20 read_log 192834'11554 (192834'11553) modify 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.1883634.0:34046 2014-08-11 15:28:34.166416 2014-08-13 18:06:06.725736 7f5c973a57a0 20 read_log 192834'11559 (0'0) promote 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.373.0:13365 2014-08-11 15:28:35.152350 2014-08-13 18:06:06.725748 7f5c973a57a0 20 read_log 192834'11560 (192834'11559) modify 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.1883634.0:34652 2014-08-11 15:28:35.122947 2014-08-13 18:06:06.725762 7f5c973a57a0 20 read_log 192834'11571 (192834'11264) modify 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.1883634.0:36127 2014-08-11 15:28:37.591873 2014-08-13 18:06:06.725775 7f5c973a57a0 20 read_log 192834'11576 (0'0) promote 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.373.0:13382 2014-08-11 15:28:38.527256 2014-08-13 18:06:06.725787 7f5c973a57a0 20 read_log 192834'11577 (192834'11576) modify 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.1883634.0:36856 2014-08-11 15:28:38.493179 2014-08-13 18:06:06.725808 7f5c973a57a0 20 read_log 192834'11593 (192834'11306) modify 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.1883634.0:38855 2014-08-11 15:28:41.215681 2014-08-13 18:06:06.725823 7f5c973a57a0 20 read_log 192834'11595 (192834'11310) modify 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.1883634.0:39074 2014-08-11 15:28:41.532334 2014-08-13 18:06:06.725835 7f5c973a57a0 20 read_log 192834'11602 (192834'11324) modify ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.1883634.0:39660 2014-08-11 15:28:42.409587 2014-08-13 18:06:06.725849 7f5c973a57a0 20 read_log 192834'11614 (192834'11348) modify f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.1883634.0:40906 2014-08-11 15:28:44.416508 2014-08-13 18:06:06.725862 7f5c973a57a0 20 read_log 192834'11621 (192834'11362) modify e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.1883634.0:41812 2014-08-11 15:28:45.510004 2014-08-13 18:06:06.725875 7f5c973a57a0 20 read_log 192834'11626 (192834'11372) modify 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.1883634.0:42204 2014-08-11 15:28:46.169697 2014-08-13 18:06:06.725888 7f5c973a57a0 20 read_log 192834'11628 (192834'11376) modify eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.1883634.0:42530 2014-08-11 15:28:46.273333 2014-08-13 18:06:06.725911 7f5c973a57a0 20 read_log 192834'11635 (192834'11392) modify e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.1883634.0:43442 2014-08-11 15:28:46.555327 2014-08-13 18:06:06.725926 7f5c973a57a0 20 read_log 192834'11641 (192834'11404) modify b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.1883634.0:43763 2014-08-11 15:28:46.655912 2014-08-13 18:06:06.725939 7f5c973a57a0 20 read_log 192834'11659 (192834'11440) modify 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.1883634.0:45991 2014-08-11 15:28:47.458842 2014-08-13 18:06:06.725953 7f5c973a57a0 20 read_log 192888'11663 (192888'11663) modify 1bb/hit_set_24.1bb_archive_2014-08-11 16:02:29.007052_2014-08-11 16:44:58.224868/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:58.225627 2014-08-13 18:06:06.725968 7f5c973a57a0 20 read_log 192888'11664 (0'0) promote 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by osd.4.0:21394 2014-08-11 16:44:58.231342 2014-08-13 18:06:06.725982 7f5c973a57a0 20 read_log 192888'11665 (192888'11664) modify 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.1315259.0:775 2014-08-11 16:44:58.219246 2014-08-13 18:06:06.725995 7f5c973a57a0 20 read_log 192888'11672 (0'0) promote 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by osd.4.0:21426 2014-08-11 16:45:01.307538 2014-08-13 18:06:06.726008 7f5c973a57a0 20 read_log 192888'11673 (192888'11672) modify 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.1315259.0:1576 2014-08-11 16:45:01.298901 2014-08-13 18:06:06.726022 7f5c973a57a0 20 read_log 192888'11674 (0'0) promote 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by osd.4.0:21436 2014-08-11 16:45:02.238717 2014-08-13 18:06:06.726035 7f5c973a57a0 20 read_log 192888'11675 (192888'11674) modify 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.1315259.0:1828 2014-08-11 16:45:02.230872 2014-08-13 18:06:06.726049 7f5c973a57a0 20 read_log 192888'11680 (0'0) promote 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by osd.4.0:21468 2014-08-11 16:45:06.718512 2014-08-13 18:06:06.726062 7f5c973a57a0 20 read_log 192888'11681 (192888'11680) modify 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.1315259.0:3036 2014-08-11 16:45:06.710251 2014-08-13 18:06:06.726074 7f5c973a57a0 20 read_log 192888'11682 (0'0) promote a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by osd.4.0:21522 2014-08-11 16:45:13.950025 2014-08-13 18:06:06.726086 7f5c973a57a0 20 read_log 192888'11683 (192888'11682) modify a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.1315259.0:4960 2014-08-11 16:45:13.941066 2014-08-13 18:06:06.726099 7f5c973a57a0 20 read_log 192888'11686 (0'0) promote cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by osd.4.0:21550 2014-08-11 16:45:16.549559 2014-08-13 18:06:06.726113 7f5c973a57a0 20 read_log 192888'11687 (192888'11686) modify cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.1315259.0:5658 2014-08-11 16:45:16.541187 2014-08-13 18:06:06.726128 7f5c973a57a0 20 read_log 192888'11698 (0'0) promote f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by osd.4.0:21610 2014-08-11 16:45:26.526727 2014-08-13 18:06:06.726140 7f5c973a57a0 20 read_log 192888'11699 (192888'11698) modify f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.1315259.0:8402 2014-08-11 16:45:26.517483 2014-08-13 18:06:06.726164 7f5c973a57a0 20 read_log 192888'11704 (0'0) promote 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by osd.4.0:21684 2014-08-11 16:45:36.207886 2014-08-13 18:06:06.726184 7f5c973a57a0 20 read_log 192888'11705 (192888'11704) modify 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.1315259.0:10897 2014-08-11 16:45:36.199399 2014-08-13 18:06:06.726199 7f5c973a57a0 20 read_log 192888'11714 (0'0) promote cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by osd.4.0:21742 2014-08-11 16:45:45.448358 2014-08-13 18:06:06.726211 7f5c973a57a0 20 read_log 192888'11715 (192888'11714) modify cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.1315259.0:13149 2014-08-11 16:45:45.439154 2014-08-13 18:06:06.726226 7f5c973a57a0 20 read_log 192888'11716 (0'0) promote 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by osd.4.0:21746 2014-08-11 16:45:46.123528 2014-08-13 18:06:06.726255 7f5c973a57a0 20 read_log 192888'11717 (192888'11716) modify 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.1315259.0:13324 2014-08-11 16:45:46.115310 2014-08-13 18:06:06.726270 7f5c973a57a0 20 read_log 192888'11718 (0'0) promote 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by osd.4.0:21752 2014-08-11 16:45:47.112025 2014-08-13 18:06:06.726282 7f5c973a57a0 20 read_log 192888'11719 (192888'11718) modify 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.1315259.0:13557 2014-08-11 16:45:47.104722 2014-08-13 18:06:06.726297 7f5c973a57a0 20 read_log 192888'11726 (0'0) promote 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by osd.4.0:21792 2014-08-11 16:45:54.038131 2014-08-13 18:06:06.726312 7f5c973a57a0 20 read_log 192888'11727 (192888'11726) modify 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.1315259.0:15381 2014-08-11 16:45:54.027412 2014-08-13 18:06:06.726327 7f5c973a57a0 20 read_log 192888'11734 (0'0) promote b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by osd.4.0:21830 2014-08-11 16:45:57.591447 2014-08-13 18:06:06.726339 7f5c973a57a0 20 read_log 192888'11735 (192888'11734) modify b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.1315259.0:16309 2014-08-11 16:45:57.583229 2014-08-13 18:06:06.726354 7f5c973a57a0 20 read_log 192888'11736 (0'0) promote 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by osd.4.0:21832 2014-08-11 16:45:57.905721 2014-08-13 18:06:06.726371 7f5c973a57a0 20 read_log 192888'11737 (192888'11736) modify 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.1315259.0:16393 2014-08-11 16:45:57.896978 2014-08-13 18:06:06.726384 7f5c973a57a0 20 read_log 192888'11738 (0'0) promote b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by osd.4.0:21842 2014-08-11 16:46:00.254837 2014-08-13 18:06:06.726396 7f5c973a57a0 20 read_log 192888'11739 (192888'11738) modify b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.1315259.0:16980 2014-08-11 16:46:00.246782 2014-08-13 18:06:06.726409 7f5c973a57a0 20 read_log 192888'11740 (0'0) promote 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by osd.4.0:21858 2014-08-11 16:46:01.481287 2014-08-13 18:06:06.726421 7f5c973a57a0 20 read_log 192888'11741 (192888'11740) modify 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.1315259.0:17297 2014-08-11 16:46:01.473429 2014-08-13 18:06:06.726446 7f5c973a57a0 20 read_log 192888'11746 (0'0) promote f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by osd.4.0:21884 2014-08-11 16:46:06.440322 2014-08-13 18:06:06.726462 7f5c973a57a0 20 read_log 192888'11747 (192888'11746) modify f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.1315259.0:18542 2014-08-11 16:46:06.432691 2014-08-13 18:06:06.726475 7f5c973a57a0 20 read_log 192888'11748 (0'0) promote f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by osd.4.0:21888 2014-08-11 16:46:06.503060 2014-08-13 18:06:06.726518 7f5c973a57a0 20 read_log 192888'11749 (192888'11748) modify f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.1315259.0:18556 2014-08-11 16:46:06.495623 2014-08-13 18:06:06.726545 7f5c973a57a0 20 read_log 192888'11750 (0'0) promote 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by osd.4.0:21906 2014-08-11 16:46:08.047075 2014-08-13 18:06:06.726560 7f5c973a57a0 20 read_log 192888'11751 (192888'11750) modify 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.1315259.0:18968 2014-08-11 16:46:08.038898 2014-08-13 18:06:06.726573 7f5c973a57a0 20 read_log 192888'11752 (0'0) promote 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by osd.4.0:21912 2014-08-11 16:46:08.583468 2014-08-13 18:06:06.726597 7f5c973a57a0 20 read_log 192888'11753 (192888'11752) modify 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.1315259.0:19071 2014-08-11 16:46:08.573989 2014-08-13 18:06:06.726610 7f5c973a57a0 20 read_log 192888'11758 (0'0) promote ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by osd.4.0:21994 2014-08-11 16:46:20.096803 2014-08-13 18:06:06.726622 7f5c973a57a0 20 read_log 192888'11759 (192888'11758) modify ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.1315259.0:22102 2014-08-11 16:46:20.089659 2014-08-13 18:06:06.726637 7f5c973a57a0 20 read_log 192888'11760 (0'0) promote 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by osd.4.0:22008 2014-08-11 16:46:22.103186 2014-08-13 18:06:06.726649 7f5c973a57a0 20 read_log 192888'11761 (192888'11760) modify 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.1315259.0:22614 2014-08-11 16:46:22.094941 2014-08-13 18:06:06.726662 7f5c973a57a0 20 read_log 192888'11764 (0'0) promote be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by osd.4.0:22014 2014-08-11 16:46:22.767415 2014-08-13 18:06:06.726674 7f5c973a57a0 20 read_log 192888'11765 (192888'11764) modify be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.1315259.0:22792 2014-08-11 16:46:22.757663 2014-08-13 18:06:06.726686 7f5c973a57a0 20 read_log 192888'11766 (0'0) promote b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by osd.4.0:22044 2014-08-11 16:46:28.198353 2014-08-13 18:06:06.726701 7f5c973a57a0 20 read_log 192888'11767 (192888'11766) modify b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.1315259.0:24147 2014-08-11 16:46:28.188138 2014-08-13 18:06:06.726714 7f5c973a57a0 20 read_log 192888'11774 (0'0) promote 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by osd.4.0:22096 2014-08-11 16:46:34.767534 2014-08-13 18:06:06.726726 7f5c973a57a0 20 read_log 192888'11775 (192888'11774) modify 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.1315259.0:25788 2014-08-11 16:46:34.759072 2014-08-13 18:06:06.726738 7f5c973a57a0 20 read_log 192888'11784 (0'0) promote 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by osd.4.0:22168 2014-08-11 16:46:48.420798 2014-08-13 18:06:06.726750 7f5c973a57a0 20 read_log 192888'11785 (192888'11784) modify 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.1315259.0:29105 2014-08-11 16:46:48.413155 2014-08-13 18:06:06.726763 7f5c973a57a0 20 read_log 192888'11788 (0'0) promote 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by osd.4.0:22184 2014-08-11 16:46:51.267681 2014-08-13 18:06:06.726775 7f5c973a57a0 20 read_log 192888'11789 (192888'11788) modify 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.1315259.0:29776 2014-08-11 16:46:51.258168 2014-08-13 18:06:06.726787 7f5c973a57a0 20 read_log 192888'11792 (0'0) promote f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by osd.4.0:22202 2014-08-11 16:46:53.026474 2014-08-13 18:06:06.726819 7f5c973a57a0 20 read_log 192888'11793 (192888'11792) modify f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.1315259.0:30247 2014-08-11 16:46:53.016351 2014-08-13 18:06:06.726835 7f5c973a57a0 20 read_log 192888'11796 (192888'11665) modify 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.1315259.0:31502 2014-08-11 16:46:56.113980 2014-08-13 18:06:06.726852 7f5c973a57a0 20 read_log 192888'11800 (192888'11673) modify 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.1315259.0:32303 2014-08-11 16:46:56.342675 2014-08-13 18:06:06.726864 7f5c973a57a0 20 read_log 192888'11801 (192888'11675) modify 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.1315259.0:32555 2014-08-11 16:46:56.409038 2014-08-13 18:06:06.726877 7f5c973a57a0 20 read_log 192888'11804 (192888'11681) modify 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.1315259.0:33763 2014-08-11 16:46:56.718564 2014-08-13 18:06:06.726889 7f5c973a57a0 20 read_log 192888'11805 (192888'11683) modify a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.1315259.0:35687 2014-08-11 16:46:57.222547 2014-08-13 18:06:06.726901 7f5c973a57a0 20 read_log 192888'11807 (192888'11687) modify cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.1315259.0:36385 2014-08-11 16:46:57.409456 2014-08-13 18:06:06.726913 7f5c973a57a0 20 read_log 192888'11813 (192888'11699) modify f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.1315259.0:39129 2014-08-11 16:46:58.133015 2014-08-13 18:06:06.726925 7f5c973a57a0 20 read_log 192888'11816 (192888'11705) modify 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.1315259.0:41624 2014-08-11 16:46:58.794369 2014-08-13 18:06:06.726938 7f5c973a57a0 20 read_log 192888'11821 (192888'11715) modify cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.1315259.0:43876 2014-08-11 16:46:59.415895 2014-08-13 18:06:06.726950 7f5c973a57a0 20 read_log 192888'11822 (192888'11717) modify 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.1315259.0:44051 2014-08-11 16:46:59.466981 2014-08-13 18:06:06.726962 7f5c973a57a0 20 read_log 192888'11823 (192888'11719) modify 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.1315259.0:44284 2014-08-11 16:46:59.535014 2014-08-13 18:06:06.726974 7f5c973a57a0 20 read_log 192888'11827 (192888'11727) modify 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.1315259.0:46108 2014-08-11 16:47:00.062461 2014-08-13 18:06:06.726986 7f5c973a57a0 20 read_log 192888'11831 (192888'11735) modify b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.1315259.0:47036 2014-08-11 16:47:00.328690 2014-08-13 18:06:06.726999 7f5c973a57a0 20 read_log 192888'11832 (192888'11737) modify 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.1315259.0:47120 2014-08-11 16:47:00.356372 2014-08-13 18:06:06.727011 7f5c973a57a0 20 read_log 192888'11833 (192888'11739) modify b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.1315259.0:47707 2014-08-11 16:47:00.541775 2014-08-13 18:06:06.727023 7f5c973a57a0 20 read_log 192888'11834 (192888'11741) modify 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.1315259.0:48024 2014-08-11 16:47:00.647057 2014-08-13 18:06:06.727035 7f5c973a57a0 20 read_log 192888'11837 (192888'11747) modify f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.1315259.0:49269 2014-08-11 16:47:01.008684 2014-08-13 18:06:06.727050 7f5c973a57a0 20 read_log 192888'11838 (192888'11749) modify f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.1315259.0:49283 2014-08-11 16:47:01.014019 2014-08-13 18:06:06.727062 7f5c973a57a0 20 read_log 192888'11839 (192888'11751) modify 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.1315259.0:49695 2014-08-11 16:47:01.146123 2014-08-13 18:06:06.727083 7f5c973a57a0 20 read_log 192888'11840 (192888'11753) modify 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.1315259.0:49798 2014-08-11 16:47:01.176641 2014-08-13 18:06:06.727098 7f5c973a57a0 20 read_log 192888'11843 (192888'11759) modify ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.1315259.0:52829 2014-08-11 16:47:02.081540 2014-08-13 18:06:06.727111 7f5c973a57a0 20 read_log 192888'11844 (192888'11761) modify 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.1315259.0:53341 2014-08-11 16:47:02.225747 2014-08-13 18:06:06.727125 7f5c973a57a0 20 read_log 192888'11846 (192888'11765) modify be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.1315259.0:53519 2014-08-11 16:47:02.277176 2014-08-13 18:06:06.727137 7f5c973a57a0 20 read_log 192888'11847 (192888'11767) modify b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.1315259.0:54874 2014-08-11 16:47:02.743164 2014-08-13 18:06:06.727157 7f5c973a57a0 20 read_log 192888'11851 (192888'11775) modify 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.1315259.0:56515 2014-08-11 16:47:03.231029 2014-08-13 18:06:06.727170 7f5c973a57a0 20 read_log 192888'11856 (192888'11785) modify 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.1315259.0:59832 2014-08-11 16:47:04.253844 2014-08-13 18:06:06.727185 7f5c973a57a0 20 read_log 192888'11858 (192888'11789) modify 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.1315259.0:60503 2014-08-11 16:47:04.461426 2014-08-13 18:06:06.727197 7f5c973a57a0 20 read_log 192888'11860 (192888'11793) modify f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.1315259.0:60974 2014-08-11 16:47:04.596900 2014-08-13 18:06:06.727210 7f5c973a57a0 20 read_log 192888'11862 (192888'11862) modify 1bb/hit_set_24.1bb_archive_2014-08-11 16:44:58.225727_2014-08-11 17:05:25.545358/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:25.545591 2014-08-13 18:06:06.727224 7f5c973a57a0 20 read_log 192888'11863 (192888'11663) delete 1bb/hit_set_24.1bb_archive_2014-08-11 16:02:29.007052_2014-08-11 16:44:58.224868/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:25.545591 2014-08-13 18:06:06.727237 7f5c973a57a0 20 read_log 192888'11864 (0'0) promote 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by osd.4.0:22708 2014-08-11 17:05:25.550802 2014-08-13 18:06:06.727249 7f5c973a57a0 20 read_log 192888'11865 (192888'11864) modify 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.942868.0:2595 2014-08-11 17:05:25.539243 2014-08-13 18:06:06.727262 7f5c973a57a0 20 read_log 192888'11866 (0'0) promote e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by osd.4.0:22736 2014-08-11 17:05:28.658274 2014-08-13 18:06:06.727274 7f5c973a57a0 20 read_log 192888'11867 (192888'11866) modify e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.942868.0:3437 2014-08-11 17:05:28.647956 2014-08-13 18:06:06.727291 7f5c973a57a0 20 read_log 192888'11870 (0'0) promote c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by osd.4.0:22768 2014-08-11 17:05:32.011179 2014-08-13 18:06:06.727306 7f5c973a57a0 20 read_log 192888'11871 (192888'11870) modify c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.942868.0:4346 2014-08-11 17:05:32.002248 2014-08-13 18:06:06.727318 7f5c973a57a0 20 read_log 192888'11872 (0'0) promote 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by osd.4.0:22820 2014-08-11 17:05:38.738237 2014-08-13 18:06:06.727330 7f5c973a57a0 20 read_log 192888'11873 (192888'11872) modify 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.942868.0:6116 2014-08-11 17:05:38.729381 2014-08-13 18:06:06.727354 7f5c973a57a0 20 read_log 192888'11880 (0'0) promote 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by osd.4.0:22884 2014-08-11 17:05:47.672866 2014-08-13 18:06:06.727369 7f5c973a57a0 20 read_log 192888'11881 (192888'11880) modify 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.942868.0:8510 2014-08-11 17:05:47.661201 2014-08-13 18:06:06.727387 7f5c973a57a0 20 read_log 192888'11882 (0'0) promote 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by osd.4.0:22892 2014-08-11 17:05:49.916267 2014-08-13 18:06:06.727402 7f5c973a57a0 20 read_log 192888'11883 (192888'11882) modify 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.942868.0:9109 2014-08-11 17:05:49.906604 2014-08-13 18:06:06.727429 7f5c973a57a0 20 read_log 192888'11884 (0'0) promote b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by osd.4.0:22912 2014-08-11 17:05:52.903118 2014-08-13 18:06:06.727445 7f5c973a57a0 20 read_log 192888'11885 (192888'11884) modify b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.942868.0:9916 2014-08-11 17:05:52.895634 2014-08-13 18:06:06.727458 7f5c973a57a0 20 read_log 192888'11886 (0'0) promote bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by osd.4.0:22932 2014-08-11 17:05:55.660735 2014-08-13 18:06:06.727481 7f5c973a57a0 20 read_log 192888'11887 (192888'11886) modify bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.942868.0:10683 2014-08-11 17:05:55.653661 2014-08-13 18:06:06.727497 7f5c973a57a0 20 read_log 192888'11892 (0'0) promote cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by osd.4.0:22942 2014-08-11 17:05:57.414968 2014-08-13 18:06:06.727515 7f5c973a57a0 20 read_log 192888'11893 (192888'11892) modify cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.942868.0:11156 2014-08-11 17:05:57.406340 2014-08-13 18:06:06.727531 7f5c973a57a0 20 read_log 192888'11896 (0'0) promote 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by osd.4.0:22958 2014-08-11 17:06:00.973274 2014-08-13 18:06:06.727544 7f5c973a57a0 20 read_log 192888'11897 (192888'11896) modify 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.942868.0:12130 2014-08-11 17:06:00.964049 2014-08-13 18:06:06.727557 7f5c973a57a0 20 read_log 192888'11898 (0'0) promote 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by osd.4.0:22964 2014-08-11 17:06:01.692019 2014-08-13 18:06:06.727575 7f5c973a57a0 20 read_log 192888'11899 (192888'11898) modify 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.942868.0:12331 2014-08-11 17:06:01.683659 2014-08-13 18:06:06.727587 7f5c973a57a0 20 read_log 192888'11900 (0'0) promote 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by osd.4.0:22968 2014-08-11 17:06:02.223124 2014-08-13 18:06:06.727600 7f5c973a57a0 20 read_log 192888'11901 (192888'11900) modify 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.942868.0:12480 2014-08-11 17:06:02.215147 2014-08-13 18:06:06.727624 7f5c973a57a0 20 read_log 192888'11902 (0'0) promote 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by osd.4.0:23000 2014-08-11 17:06:06.139079 2014-08-13 18:06:06.727639 7f5c973a57a0 20 read_log 192888'11903 (192888'11902) modify 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.942868.0:13493 2014-08-11 17:06:06.130281 2014-08-13 18:06:06.727679 7f5c973a57a0 20 read_log 192888'11904 (0'0) promote 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by osd.4.0:23006 2014-08-11 17:06:07.852919 2014-08-13 18:06:06.727704 7f5c973a57a0 20 read_log 192888'11905 (192888'11904) modify 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.942868.0:13922 2014-08-11 17:06:07.843027 2014-08-13 18:06:06.727744 7f5c973a57a0 20 read_log 192888'11906 (0'0) promote 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by osd.4.0:23010 2014-08-11 17:06:08.651023 2014-08-13 18:06:06.727760 7f5c973a57a0 20 read_log 192888'11907 (192888'11906) modify 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.942868.0:14103 2014-08-11 17:06:08.641314 2014-08-13 18:06:06.727773 7f5c973a57a0 20 read_log 192888'11912 (0'0) promote 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by osd.4.0:23028 2014-08-11 17:06:11.525726 2014-08-13 18:06:06.727785 7f5c973a57a0 20 read_log 192888'11913 (192888'11912) modify 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.942868.0:14821 2014-08-11 17:06:11.517433 2014-08-13 18:06:06.727797 7f5c973a57a0 20 read_log 192888'11918 (0'0) promote 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by osd.4.0:23088 2014-08-11 17:06:20.636199 2014-08-13 18:06:06.727810 7f5c973a57a0 20 read_log 192888'11919 (192888'11918) modify 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.942868.0:17152 2014-08-11 17:06:20.627339 2014-08-13 18:06:06.727822 7f5c973a57a0 20 read_log 192888'11920 (0'0) promote 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by osd.4.0:23094 2014-08-11 17:06:21.274779 2014-08-13 18:06:06.727834 7f5c973a57a0 20 read_log 192888'11921 (192888'11920) modify 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.942868.0:17320 2014-08-11 17:06:21.265953 2014-08-13 18:06:06.727846 7f5c973a57a0 20 read_log 192888'11924 (0'0) promote 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by osd.4.0:23108 2014-08-11 17:06:22.868704 2014-08-13 18:06:06.727858 7f5c973a57a0 20 read_log 192888'11925 (192888'11924) modify 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.942868.0:17742 2014-08-11 17:06:22.858455 2014-08-13 18:06:06.727870 7f5c973a57a0 20 read_log 192888'11928 (0'0) promote c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by osd.4.0:23136 2014-08-11 17:06:27.338192 2014-08-13 18:06:06.727882 7f5c973a57a0 20 read_log 192888'11929 (192888'11928) modify c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.942868.0:18931 2014-08-11 17:06:27.329604 2014-08-13 18:06:06.727894 7f5c973a57a0 20 read_log 192888'11932 (0'0) promote d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by osd.4.0:23186 2014-08-11 17:06:34.416083 2014-08-13 18:06:06.727906 7f5c973a57a0 20 read_log 192888'11933 (192888'11932) modify d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.942868.0:20792 2014-08-11 17:06:34.407975 2014-08-13 18:06:06.727928 7f5c973a57a0 20 read_log 192888'11938 (0'0) promote d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by osd.4.0:23266 2014-08-11 17:06:45.128382 2014-08-13 18:06:06.727940 7f5c973a57a0 20 read_log 192888'11939 (192888'11938) modify d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.942868.0:23571 2014-08-11 17:06:45.119954 2014-08-13 18:06:06.727953 7f5c973a57a0 20 read_log 192888'11944 (0'0) promote 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by osd.4.0:23298 2014-08-11 17:06:48.052290 2014-08-13 18:06:06.727965 7f5c973a57a0 20 read_log 192888'11945 (192888'11944) modify 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.942868.0:24364 2014-08-11 17:06:48.042837 2014-08-13 18:06:06.727977 7f5c973a57a0 20 read_log 192888'11946 (0'0) promote a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by osd.4.0:23300 2014-08-11 17:06:48.300162 2014-08-13 18:06:06.727989 7f5c973a57a0 20 read_log 192888'11947 (192888'11946) modify a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.942868.0:24428 2014-08-11 17:06:48.290724 2014-08-13 18:06:06.728008 7f5c973a57a0 20 read_log 192888'11948 (0'0) promote fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by osd.4.0:23318 2014-08-11 17:06:49.746408 2014-08-13 18:06:06.728024 7f5c973a57a0 20 read_log 192888'11949 (192888'11948) modify fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.942868.0:24820 2014-08-11 17:06:49.737654 2014-08-13 18:06:06.728036 7f5c973a57a0 20 read_log 192888'11950 (0'0) promote a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by osd.4.0:23320 2014-08-11 17:06:49.784824 2014-08-13 18:06:06.728048 7f5c973a57a0 20 read_log 192888'11951 (192888'11950) modify a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.942868.0:24830 2014-08-11 17:06:49.774436 2014-08-13 18:06:06.728060 7f5c973a57a0 20 read_log 192888'11952 (0'0) promote bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by osd.4.0:23324 2014-08-11 17:06:50.693715 2014-08-13 18:06:06.728072 7f5c973a57a0 20 read_log 192888'11953 (192888'11952) modify bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.942868.0:25075 2014-08-11 17:06:50.681555 2014-08-13 18:06:06.728085 7f5c973a57a0 20 read_log 192888'11956 (0'0) promote 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by osd.4.0:23338 2014-08-11 17:06:52.370971 2014-08-13 18:06:06.728097 7f5c973a57a0 20 read_log 192888'11957 (192888'11956) modify 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.942868.0:25517 2014-08-11 17:06:52.362503 2014-08-13 18:06:06.728109 7f5c973a57a0 20 read_log 192888'11958 (0'0) promote ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by osd.4.0:23346 2014-08-11 17:06:54.094439 2014-08-13 18:06:06.728121 7f5c973a57a0 20 read_log 192888'11959 (192888'11958) modify ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.942868.0:25916 2014-08-11 17:06:54.085777 2014-08-13 18:06:06.728133 7f5c973a57a0 20 read_log 192888'11960 (0'0) promote 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by osd.4.0:23348 2014-08-11 17:06:54.905964 2014-08-13 18:06:06.728149 7f5c973a57a0 20 read_log 192888'11961 (192888'11960) modify 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.942868.0:26118 2014-08-11 17:06:54.897100 2014-08-13 18:06:06.728163 7f5c973a57a0 20 read_log 192888'11964 (0'0) promote 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by osd.4.0:23368 2014-08-11 17:06:58.769938 2014-08-13 18:06:06.728175 7f5c973a57a0 20 read_log 192888'11965 (192888'11964) modify 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.942868.0:27148 2014-08-11 17:06:58.762138 2014-08-13 18:06:06.728188 7f5c973a57a0 20 read_log 192888'11966 (0'0) promote dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by osd.4.0:23374 2014-08-11 17:06:59.687533 2014-08-13 18:06:06.728202 7f5c973a57a0 20 read_log 192888'11967 (192888'11966) modify dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.942868.0:27400 2014-08-11 17:06:59.679140 2014-08-13 18:06:06.728215 7f5c973a57a0 20 read_log 192888'11970 (0'0) promote 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by osd.4.0:23386 2014-08-11 17:07:00.673588 2014-08-13 18:06:06.728229 7f5c973a57a0 20 read_log 192888'11971 (192888'11970) modify 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.942868.0:27650 2014-08-11 17:07:00.664200 2014-08-13 18:06:06.728242 7f5c973a57a0 20 read_log 192888'11972 (0'0) promote d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by osd.4.0:23400 2014-08-11 17:07:03.630731 2014-08-13 18:06:06.728255 7f5c973a57a0 20 read_log 192888'11973 (192888'11972) modify d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.942868.0:28447 2014-08-11 17:07:03.622238 2014-08-13 18:06:06.728281 7f5c973a57a0 20 read_log 192888'11974 (0'0) promote d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by osd.4.0:23406 2014-08-11 17:07:04.359977 2014-08-13 18:06:06.728297 7f5c973a57a0 20 read_log 192888'11975 (192888'11974) modify d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.942868.0:28643 2014-08-11 17:07:04.350520 2014-08-13 18:06:06.728309 7f5c973a57a0 20 read_log 192888'11976 (0'0) promote 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by osd.4.0:23414 2014-08-11 17:07:05.126053 2014-08-13 18:06:06.728322 7f5c973a57a0 20 read_log 192888'11977 (192888'11976) modify 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.942868.0:28852 2014-08-11 17:07:05.117512 2014-08-13 18:06:06.728340 7f5c973a57a0 20 read_log 192888'11980 (0'0) promote 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by osd.4.0:23438 2014-08-11 17:07:09.794505 2014-08-13 18:06:06.728353 7f5c973a57a0 20 read_log 192888'11981 (192888'11980) modify 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.942868.0:29995 2014-08-11 17:07:09.785352 2014-08-13 18:06:06.728365 7f5c973a57a0 20 read_log 192888'11986 (192888'11865) modify 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.942868.0:33861 2014-08-11 17:07:16.632374 2014-08-13 18:06:06.728381 7f5c973a57a0 20 read_log 192888'11987 (192888'11867) modify e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.942868.0:34703 2014-08-11 17:07:16.869072 2014-08-13 18:06:06.728395 7f5c973a57a0 20 read_log 192888'11989 (192888'11871) modify c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.942868.0:35612 2014-08-11 17:07:17.118579 2014-08-13 18:06:06.728416 7f5c973a57a0 20 read_log 192888'11990 (192888'11873) modify 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.942868.0:37382 2014-08-11 17:07:17.593795 2014-08-13 18:06:06.728436 7f5c973a57a0 20 read_log 192888'11994 (192888'11881) modify 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.942868.0:39776 2014-08-11 17:07:18.279856 2014-08-13 18:06:06.728448 7f5c973a57a0 20 read_log 192888'11995 (192888'11883) modify 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.942868.0:40375 2014-08-11 17:07:18.433050 2014-08-13 18:06:06.728468 7f5c973a57a0 20 read_log 192888'11996 (192888'11885) modify b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.942868.0:41182 2014-08-11 17:07:18.663115 2014-08-13 18:06:06.728481 7f5c973a57a0 20 read_log 192888'11997 (192888'11887) modify bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.942868.0:41949 2014-08-11 17:07:18.864825 2014-08-13 18:06:06.728494 7f5c973a57a0 20 read_log 192888'12000 (192888'11893) modify cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.942868.0:42422 2014-08-11 17:07:18.992873 2014-08-13 18:06:06.728511 7f5c973a57a0 20 read_log 192888'12002 (192888'11897) modify 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.942868.0:43396 2014-08-11 17:07:19.260866 2014-08-13 18:06:06.728526 7f5c973a57a0 20 read_log 192888'12003 (192888'11899) modify 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.942868.0:43597 2014-08-11 17:07:19.318313 2014-08-13 18:06:06.728539 7f5c973a57a0 20 read_log 192888'12004 (192888'11901) modify 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.942868.0:43746 2014-08-11 17:07:19.362158 2014-08-13 18:06:06.728558 7f5c973a57a0 20 read_log 192888'12005 (192888'11903) modify 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.942868.0:44759 2014-08-11 17:07:19.655477 2014-08-13 18:06:06.728574 7f5c973a57a0 20 read_log 192888'12006 (192888'11905) modify 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.942868.0:45188 2014-08-11 17:07:19.769490 2014-08-13 18:06:06.728594 7f5c973a57a0 20 read_log 192888'12007 (192888'11907) modify 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.942868.0:45369 2014-08-11 17:07:19.818310 2014-08-13 18:06:06.728621 7f5c973a57a0 20 read_log 192888'12010 (192888'11913) modify 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.942868.0:46087 2014-08-11 17:07:20.016559 2014-08-13 18:06:06.728637 7f5c973a57a0 20 read_log 192888'12013 (192888'11919) modify 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.942868.0:48418 2014-08-11 17:07:20.715055 2014-08-13 18:06:06.728650 7f5c973a57a0 20 read_log 192888'12014 (192888'11921) modify 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.942868.0:48586 2014-08-11 17:07:20.758583 2014-08-13 18:06:06.728665 7f5c973a57a0 20 read_log 192888'12016 (192888'11925) modify 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.942868.0:49008 2014-08-11 17:07:20.881773 2014-08-13 18:06:06.728677 7f5c973a57a0 20 read_log 192888'12018 (192888'11929) modify c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.942868.0:50197 2014-08-11 17:07:21.244821 2014-08-13 18:06:06.728690 7f5c973a57a0 20 read_log 192888'12020 (192888'11933) modify d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.942868.0:52058 2014-08-11 17:07:21.828846 2014-08-13 18:06:06.728730 7f5c973a57a0 20 read_log 192888'12023 (192888'11939) modify d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.942868.0:54837 2014-08-11 17:07:22.671208 2014-08-13 18:06:06.728748 7f5c973a57a0 20 read_log 192888'12026 (192888'11945) modify 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.942868.0:55630 2014-08-11 17:07:22.905405 2014-08-13 18:06:06.728761 7f5c973a57a0 20 read_log 192888'12027 (192888'11947) modify a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.942868.0:55694 2014-08-11 17:07:22.925739 2014-08-13 18:06:06.728773 7f5c973a57a0 20 read_log 192888'12028 (192888'11949) modify fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.942868.0:56086 2014-08-11 17:07:23.045357 2014-08-13 18:06:06.728786 7f5c973a57a0 20 read_log 192888'12029 (192888'11951) modify a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.942868.0:56096 2014-08-11 17:07:23.049234 2014-08-13 18:06:06.728798 7f5c973a57a0 20 read_log 192888'12030 (192888'11953) modify bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.942868.0:56341 2014-08-11 17:07:23.121807 2014-08-13 18:06:06.728810 7f5c973a57a0 20 read_log 192888'12032 (192888'11957) modify 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.942868.0:56783 2014-08-11 17:07:23.257591 2014-08-13 18:06:06.728822 7f5c973a57a0 20 read_log 192888'12033 (192888'11959) modify ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.942868.0:57182 2014-08-11 17:07:23.380512 2014-08-13 18:06:06.728834 7f5c973a57a0 20 read_log 192888'12034 (192888'11961) modify 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.942868.0:57384 2014-08-11 17:07:23.440310 2014-08-13 18:06:06.728847 7f5c973a57a0 20 read_log 192888'12036 (192888'11965) modify 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.942868.0:58414 2014-08-11 17:07:23.736170 2014-08-13 18:06:06.728859 7f5c973a57a0 20 read_log 192888'12037 (192888'11967) modify dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.942868.0:58666 2014-08-11 17:07:23.812909 2014-08-13 18:06:06.728871 7f5c973a57a0 20 read_log 192888'12039 (192888'11971) modify 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.942868.0:58916 2014-08-11 17:07:23.889480 2014-08-13 18:06:06.728883 7f5c973a57a0 20 read_log 192888'12040 (192888'11973) modify d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.942868.0:59713 2014-08-11 17:07:24.127168 2014-08-13 18:06:06.728903 7f5c973a57a0 20 read_log 192888'12041 (192888'11975) modify d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.942868.0:59909 2014-08-11 17:07:24.184316 2014-08-13 18:06:06.728916 7f5c973a57a0 20 read_log 192888'12042 (192888'11977) modify 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.942868.0:60118 2014-08-11 17:07:24.246018 2014-08-13 18:06:06.728928 7f5c973a57a0 20 read_log 192888'12044 (192888'11981) modify 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.942868.0:61261 2014-08-11 17:07:24.587748 2014-08-13 18:06:06.728941 7f5c973a57a0 20 read_log 193305'12047 (193305'12047) modify 1bb/hit_set_24.1bb_archive_2014-08-11 18:37:04.914122_2014-08-11 18:42:37.986871/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.987093 2014-08-13 18:06:06.728953 7f5c973a57a0 20 read_log 193305'12048 (192888'11862) delete 1bb/hit_set_24.1bb_archive_2014-08-11 16:44:58.225727_2014-08-11 17:05:25.545358/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.987093 2014-08-13 18:06:06.728965 7f5c973a57a0 20 read_log 193305'12049 (0'0) promote 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by osd.4.0:15 2014-08-11 18:42:38.026009 2014-08-13 18:06:06.728977 7f5c973a57a0 20 read_log 193305'12050 (193305'12049) modify 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.951337.0:1280 2014-08-11 18:42:37.980089 2014-08-13 18:06:06.729004 7f5c973a57a0 20 read_log 193305'12051 (0'0) promote 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by osd.4.0:33 2014-08-11 18:42:41.229264 2014-08-13 18:06:06.729016 7f5c973a57a0 20 read_log 193305'12052 (193305'12051) modify 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.951337.0:2129 2014-08-11 18:42:41.221561 2014-08-13 18:06:06.729029 7f5c973a57a0 20 read_log 193305'12053 (0'0) promote 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by osd.4.0:63 2014-08-11 18:42:49.809111 2014-08-13 18:06:06.729041 7f5c973a57a0 20 read_log 193305'12054 (193305'12053) modify 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.951337.0:4479 2014-08-11 18:42:49.802550 2014-08-13 18:06:06.729072 7f5c973a57a0 20 read_log 193305'12055 (0'0) promote cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by osd.4.0:67 2014-08-11 18:42:50.636458 2014-08-13 18:06:06.729088 7f5c973a57a0 20 read_log 193305'12056 (193305'12055) modify cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.951337.0:4705 2014-08-11 18:42:50.629019 2014-08-13 18:06:06.729101 7f5c973a57a0 20 read_log 193305'12057 (0'0) promote 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by osd.4.0:111 2014-08-11 18:42:58.663345 2014-08-13 18:06:06.729113 7f5c973a57a0 20 read_log 193305'12058 (193305'12057) modify 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.951337.0:6894 2014-08-11 18:42:58.653891 2014-08-13 18:06:06.729129 7f5c973a57a0 20 read_log 193305'12059 (0'0) promote b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by osd.4.0:123 2014-08-11 18:43:01.155315 2014-08-13 18:06:06.729142 7f5c973a57a0 20 read_log 193305'12060 (193305'12059) modify b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.951337.0:7583 2014-08-11 18:43:01.148102 2014-08-13 18:06:06.729160 7f5c973a57a0 20 read_log 193305'12061 (0'0) promote 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by osd.4.0:167 2014-08-11 18:43:10.670107 2014-08-13 18:06:06.729175 7f5c973a57a0 20 read_log 193305'12062 (193305'12061) modify 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.951337.0:10104 2014-08-11 18:43:10.662692 2014-08-13 18:06:06.729205 7f5c973a57a0 20 read_log 193305'12063 (0'0) promote 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by osd.4.0:211 2014-08-11 18:43:21.566919 2014-08-13 18:06:06.729221 7f5c973a57a0 20 read_log 193305'12064 (193305'12063) modify 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.951337.0:12982 2014-08-11 18:43:21.558135 2014-08-13 18:06:06.729237 7f5c973a57a0 20 read_log 193305'12065 (0'0) promote 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by osd.4.0:217 2014-08-11 18:43:22.828418 2014-08-13 18:06:06.729259 7f5c973a57a0 20 read_log 193305'12066 (193305'12065) modify 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.951337.0:13320 2014-08-11 18:43:22.820344 2014-08-13 18:06:06.729273 7f5c973a57a0 20 read_log 193305'12067 (0'0) promote d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by osd.4.0:223 2014-08-11 18:43:24.855939 2014-08-13 18:06:06.729285 7f5c973a57a0 20 read_log 193305'12068 (193305'12067) modify d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.951337.0:13849 2014-08-11 18:43:24.848444 2014-08-13 18:06:06.729308 7f5c973a57a0 20 read_log 193305'12069 (0'0) promote 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by osd.4.0:229 2014-08-11 18:43:26.370081 2014-08-13 18:06:06.729324 7f5c973a57a0 20 read_log 193305'12070 (193305'12069) modify 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.951337.0:14240 2014-08-11 18:43:26.362734 2014-08-13 18:06:06.729336 7f5c973a57a0 20 read_log 193305'12071 (0'0) promote 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by osd.4.0:231 2014-08-11 18:43:26.526152 2014-08-13 18:06:06.729348 7f5c973a57a0 20 read_log 193305'12072 (193305'12071) modify 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.951337.0:14288 2014-08-11 18:43:26.518369 2014-08-13 18:06:06.729361 7f5c973a57a0 20 read_log 193305'12073 (0'0) promote c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by osd.4.0:261 2014-08-11 18:43:33.227509 2014-08-13 18:06:06.729380 7f5c973a57a0 20 read_log 193305'12074 (193305'12073) modify c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.951337.0:16227 2014-08-11 18:43:33.217510 2014-08-13 18:06:06.729402 7f5c973a57a0 20 read_log 193305'12075 (0'0) promote e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by osd.4.0:285 2014-08-11 18:43:36.452413 2014-08-13 18:06:06.729415 7f5c973a57a0 20 read_log 193305'12076 (193305'12075) modify e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.951337.0:17140 2014-08-11 18:43:36.442932 2014-08-13 18:06:06.729427 7f5c973a57a0 20 read_log 193305'12077 (0'0) promote 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by osd.4.0:307 2014-08-11 18:43:39.825570 2014-08-13 18:06:06.729442 7f5c973a57a0 20 read_log 193305'12078 (193305'12077) modify 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.951337.0:18130 2014-08-11 18:43:39.814482 2014-08-13 18:06:06.729455 7f5c973a57a0 20 read_log 193305'12079 (0'0) promote a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by osd.4.0:315 2014-08-11 18:43:41.733560 2014-08-13 18:06:06.729469 7f5c973a57a0 20 read_log 193305'12080 (193305'12079) modify a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.951337.0:18677 2014-08-11 18:43:41.724114 2014-08-13 18:06:06.729481 7f5c973a57a0 20 read_log 193305'12081 (0'0) promote 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by osd.4.0:325 2014-08-11 18:43:43.624138 2014-08-13 18:06:06.729505 7f5c973a57a0 20 read_log 193305'12082 (193305'12081) modify 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.951337.0:19171 2014-08-11 18:43:43.617250 2014-08-13 18:06:06.729528 7f5c973a57a0 20 read_log 193305'12083 (0'0) promote 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by osd.4.0:331 2014-08-11 18:43:45.894510 2014-08-13 18:06:06.729546 7f5c973a57a0 20 read_log 193305'12084 (193305'12083) modify 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.951337.0:19797 2014-08-11 18:43:45.886968 2014-08-13 18:06:06.729559 7f5c973a57a0 20 read_log 193305'12085 (0'0) promote a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by osd.4.0:333 2014-08-11 18:43:46.139474 2014-08-13 18:06:06.729580 7f5c973a57a0 20 read_log 193305'12086 (193305'12085) modify a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.951337.0:19868 2014-08-11 18:43:46.132956 2014-08-13 18:06:06.729599 7f5c973a57a0 20 read_log 193305'12087 (0'0) promote 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by osd.4.0:345 2014-08-11 18:43:49.066255 2014-08-13 18:06:06.729612 7f5c973a57a0 20 read_log 193305'12088 (193305'12087) modify 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.951337.0:20679 2014-08-11 18:43:49.058618 2014-08-13 18:06:06.729624 7f5c973a57a0 20 read_log 193305'12089 (0'0) promote 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by osd.4.0:385 2014-08-11 18:43:56.506626 2014-08-13 18:06:06.729639 7f5c973a57a0 20 read_log 193305'12090 (193305'12089) modify 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.951337.0:22732 2014-08-11 18:43:56.497823 2014-08-13 18:06:06.729651 7f5c973a57a0 20 read_log 193305'12091 (0'0) promote 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by osd.4.0:447 2014-08-11 18:44:11.416562 2014-08-13 18:06:06.729664 7f5c973a57a0 20 read_log 193305'12092 (193305'12091) modify 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.951337.0:26841 2014-08-11 18:44:11.406100 2014-08-13 18:06:06.729681 7f5c973a57a0 20 read_log 193305'12093 (0'0) promote 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by osd.4.0:457 2014-08-11 18:44:14.003452 2014-08-13 18:06:06.729705 7f5c973a57a0 20 read_log 193305'12094 (193305'12093) modify 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.951337.0:27533 2014-08-11 18:44:13.996081 2014-08-13 18:06:06.729718 7f5c973a57a0 20 read_log 193305'12095 (0'0) promote 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by osd.4.0:463 2014-08-11 18:44:16.180664 2014-08-13 18:06:06.729730 7f5c973a57a0 20 read_log 193305'12096 (193305'12095) modify 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.951337.0:28149 2014-08-11 18:44:16.173356 2014-08-13 18:06:06.729746 7f5c973a57a0 20 read_log 193305'12097 (0'0) promote 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by osd.4.0:477 2014-08-11 18:44:19.315825 2014-08-13 18:06:06.729759 7f5c973a57a0 20 read_log 193305'12098 (193305'12097) modify 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.951337.0:28992 2014-08-11 18:44:19.307168 2014-08-13 18:06:06.729771 7f5c973a57a0 20 read_log 193305'12099 (0'0) promote 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by osd.4.0:481 2014-08-11 18:44:20.123729 2014-08-13 18:06:06.729793 7f5c973a57a0 20 read_log 193305'12100 (193305'12099) modify 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.951337.0:29215 2014-08-11 18:44:20.115801 2014-08-13 18:06:06.729809 7f5c973a57a0 20 read_log 193305'12101 (0'0) promote eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by osd.4.0:491 2014-08-11 18:44:22.748683 2014-08-13 18:06:06.729822 7f5c973a57a0 20 read_log 193305'12102 (193305'12101) modify eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.951337.0:29932 2014-08-11 18:44:22.741284 2014-08-13 18:06:06.729847 7f5c973a57a0 20 read_log 193305'12103 (0'0) promote fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by osd.4.0:499 2014-08-11 18:44:24.257088 2014-08-13 18:06:06.729863 7f5c973a57a0 20 read_log 193305'12104 (193305'12103) modify fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.951337.0:30315 2014-08-11 18:44:24.249432 2014-08-13 18:06:06.729875 7f5c973a57a0 20 read_log 193305'12105 (0'0) promote f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by osd.4.0:509 2014-08-11 18:44:28.123078 2014-08-13 18:06:06.729887 7f5c973a57a0 20 read_log 193305'12106 (193305'12105) modify f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.951337.0:31367 2014-08-11 18:44:28.114410 2014-08-13 18:06:06.729900 7f5c973a57a0 20 read_log 193305'12107 (0'0) promote ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by osd.4.0:517 2014-08-11 18:44:29.336942 2014-08-13 18:06:06.729912 7f5c973a57a0 20 read_log 193305'12108 (193305'12107) modify ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.951337.0:31725 2014-08-11 18:44:29.329961 2014-08-13 18:06:06.729924 7f5c973a57a0 20 read_log 193305'12109 (193305'12050) modify 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.951337.0:33991 2014-08-11 18:44:34.750830 2014-08-13 18:06:06.729936 7f5c973a57a0 20 read_log 193305'12110 (193305'12052) modify 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.951337.0:34840 2014-08-11 18:44:35.216510 2014-08-13 18:06:06.729948 7f5c973a57a0 20 read_log 193305'12111 (193305'12054) modify 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.951337.0:37190 2014-08-11 18:44:36.755779 2014-08-13 18:06:06.729960 7f5c973a57a0 20 read_log 193305'12112 (193305'12056) modify cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.951337.0:37416 2014-08-11 18:44:36.900744 2014-08-13 18:06:06.729973 7f5c973a57a0 20 read_log 193305'12113 (193305'12058) modify 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.951337.0:39605 2014-08-11 18:44:38.108753 2014-08-13 18:06:06.729995 7f5c973a57a0 20 read_log 193305'12114 (193305'12060) modify b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.951337.0:40294 2014-08-11 18:44:38.365393 2014-08-13 18:06:06.730008 7f5c973a57a0 20 read_log 193305'12115 (193305'12062) modify 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.951337.0:42815 2014-08-11 18:44:39.389359 2014-08-13 18:06:06.730020 7f5c973a57a0 20 read_log 193305'12116 (193305'12064) modify 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.951337.0:45693 2014-08-11 18:44:40.273962 2014-08-13 18:06:06.730032 7f5c973a57a0 20 read_log 193305'12117 (193305'12066) modify 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.951337.0:46031 2014-08-11 18:44:40.370671 2014-08-13 18:06:06.730045 7f5c973a57a0 20 read_log 193305'12118 (193305'12068) modify d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.951337.0:46560 2014-08-11 18:44:40.514455 2014-08-13 18:06:06.730057 7f5c973a57a0 20 read_log 193305'12119 (193305'12070) modify 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.951337.0:46951 2014-08-11 18:44:40.635424 2014-08-13 18:06:06.730071 7f5c973a57a0 20 read_log 193305'12120 (193305'12072) modify 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.951337.0:46999 2014-08-11 18:44:40.647881 2014-08-13 18:06:06.730084 7f5c973a57a0 20 read_log 193305'12121 (193305'12074) modify c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.951337.0:48938 2014-08-11 18:44:41.201616 2014-08-13 18:06:06.730096 7f5c973a57a0 20 read_log 193305'12122 (193305'12076) modify e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.951337.0:49851 2014-08-11 18:44:41.493099 2014-08-13 18:06:06.730118 7f5c973a57a0 20 read_log 193305'12123 (193305'12078) modify 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.951337.0:50841 2014-08-11 18:44:41.775246 2014-08-13 18:06:06.730134 7f5c973a57a0 20 read_log 193305'12124 (193305'12080) modify a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.951337.0:51388 2014-08-11 18:44:41.938036 2014-08-13 18:06:06.730157 7f5c973a57a0 20 read_log 193305'12125 (193305'12082) modify 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.951337.0:51882 2014-08-11 18:44:42.077316 2014-08-13 18:06:06.730173 7f5c973a57a0 20 read_log 193305'12126 (193305'12084) modify 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.951337.0:52508 2014-08-11 18:44:42.253145 2014-08-13 18:06:06.730186 7f5c973a57a0 20 read_log 193305'12127 (193305'12086) modify a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.951337.0:52579 2014-08-11 18:44:42.275137 2014-08-13 18:06:06.730198 7f5c973a57a0 20 read_log 193305'12128 (193305'12088) modify 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.951337.0:53390 2014-08-11 18:44:42.517158 2014-08-13 18:06:06.730211 7f5c973a57a0 20 read_log 193305'12129 (193305'12090) modify 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.951337.0:55443 2014-08-11 18:44:43.124528 2014-08-13 18:06:06.730223 7f5c973a57a0 20 read_log 193305'12130 (193305'12092) modify 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.951337.0:59552 2014-08-11 18:44:44.360577 2014-08-13 18:06:06.730235 7f5c973a57a0 20 read_log 193305'12131 (193305'12094) modify 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.951337.0:60244 2014-08-11 18:44:44.569371 2014-08-13 18:06:06.730269 7f5c973a57a0 20 read_log 193305'12132 (193305'12096) modify 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.951337.0:60860 2014-08-11 18:44:44.755311 2014-08-13 18:06:06.730285 7f5c973a57a0 20 read_log 193305'12133 (193305'12098) modify 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.951337.0:61703 2014-08-11 18:44:45.006724 2014-08-13 18:06:06.730299 7f5c973a57a0 20 read_log 193305'12134 (193305'12100) modify 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.951337.0:61926 2014-08-11 18:44:45.074299 2014-08-13 18:06:06.730312 7f5c973a57a0 20 read_log 193305'12135 (193305'12102) modify eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.951337.0:62643 2014-08-11 18:44:45.287657 2014-08-13 18:06:06.730324 7f5c973a57a0 20 read_log 193305'12136 (193305'12104) modify fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.951337.0:63026 2014-08-11 18:44:45.391044 2014-08-13 18:06:06.730342 7f5c973a57a0 20 read_log 193305'12137 (193305'12106) modify f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.951337.0:64078 2014-08-11 18:44:45.696891 2014-08-13 18:06:06.730355 7f5c973a57a0 20 read_log 193305'12138 (193305'12108) modify ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.951337.0:64436 2014-08-11 18:44:45.807003 2014-08-13 18:06:06.730367 7f5c973a57a0 20 read_log 194215'12139 (194215'12139) modify 1bb/hit_set_24.1bb_archive_2014-08-12 15:32:25.863957_2014-08-12 15:43:01.047258/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:01.047846 2014-08-13 18:06:06.730383 7f5c973a57a0 20 read_log 194215'12140 (193305'12047) delete 1bb/hit_set_24.1bb_archive_2014-08-11 18:37:04.914122_2014-08-11 18:42:37.986871/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:01.047846 2014-08-13 18:06:06.730398 7f5c973a57a0 20 read_log 194215'12141 (192834'11016) clean 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by osd.4.0:1974 2014-08-12 15:43:01.062473 2014-08-13 18:06:06.730432 7f5c973a57a0 20 read_log 194215'12142 (194215'12141) delete 19c301bb/benchmark_data_storage0111-ib.pocstor_26659_object15506/head//24 by client.2174595.0:91136 0.000000 2014-08-13 18:06:06.730447 7f5c973a57a0 20 read_log 194215'12143 (192888'12003) clean 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by osd.4.0:1976 2014-08-12 15:43:01.082963 2014-08-13 18:06:06.730470 7f5c973a57a0 20 read_log 194215'12144 (194215'12143) delete 4ed701bb/benchmark_data_storage0111-ib.pocstor_2228_object12330/head//24 by client.2174595.0:91138 0.000000 2014-08-13 18:06:06.730482 7f5c973a57a0 20 read_log 194215'12145 (192834'11626) clean 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by osd.4.0:1978 2014-08-12 15:43:01.092092 2014-08-13 18:06:06.730495 7f5c973a57a0 20 read_log 194215'12146 (194215'12145) delete 166d01bb/benchmark_data_storage0111-ib.pocstor_41526_object18879/head//24 by client.2174595.0:91140 0.000000 2014-08-13 18:06:06.730513 7f5c973a57a0 20 read_log 194215'12147 (193305'12129) clean 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by osd.4.0:1980 2014-08-12 15:43:01.101072 2014-08-13 18:06:06.730548 7f5c973a57a0 20 read_log 194215'12148 (194215'12147) delete 79c011bb/benchmark_data_storage0111-ib.pocstor_62621_object22731/head//24 by client.2174595.0:91142 0.000000 2014-08-13 18:06:06.730571 7f5c973a57a0 20 read_log 194215'12149 (192888'12018) clean c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by osd.4.0:1982 2014-08-12 15:43:01.125229 2014-08-13 18:06:06.730584 7f5c973a57a0 20 read_log 194215'12150 (194215'12149) delete c0c211bb/benchmark_data_storage0111-ib.pocstor_2228_object18930/head//24 by client.2174595.0:91144 0.000000 2014-08-13 18:06:06.730595 7f5c973a57a0 20 read_log 194215'12151 (192834'11500) clean bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by osd.4.0:1984 2014-08-12 15:43:01.132822 2014-08-13 18:06:06.730611 7f5c973a57a0 20 read_log 194215'12152 (194215'12151) delete bb7811bb/benchmark_data_storage0111-ib.pocstor_41526_object5512/head//24 by client.2174595.0:91146 0.000000 2014-08-13 18:06:06.730622 7f5c973a57a0 20 read_log 194215'12153 (192888'12036) clean 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by osd.4.0:1986 2014-08-12 15:43:01.159084 2014-08-13 18:06:06.730634 7f5c973a57a0 20 read_log 194215'12154 (194215'12153) delete 3dc11bb/benchmark_data_storage0111-ib.pocstor_2228_object27147/head//24 by client.2174595.0:91148 0.000000 2014-08-13 18:06:06.730645 7f5c973a57a0 20 read_log 194215'12155 (193305'12131) clean 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by osd.4.0:1988 2014-08-12 15:43:01.169163 2014-08-13 18:06:06.730657 7f5c973a57a0 20 read_log 194215'12156 (194215'12155) delete 91d421bb/benchmark_data_storage0111-ib.pocstor_62621_object27532/head//24 by client.2174595.0:91150 0.000000 2014-08-13 18:06:06.730668 7f5c973a57a0 20 read_log 194215'12157 (192834'10956) clean f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by osd.4.0:1990 2014-08-12 15:43:01.177916 2014-08-13 18:06:06.730680 7f5c973a57a0 20 read_log 194215'12158 (194215'12157) delete f2c621bb/benchmark_data_storage0111-ib.pocstor_26659_object7807/head//24 by client.2174595.0:91152 0.000000 2014-08-13 18:06:06.730691 7f5c973a57a0 20 read_log 194215'12159 (192888'12013) clean 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by osd.4.0:1992 2014-08-12 15:43:01.199038 2014-08-13 18:06:06.730704 7f5c973a57a0 20 read_log 194215'12160 (194215'12159) delete 7af031bb/benchmark_data_storage0111-ib.pocstor_2228_object17151/head//24 by client.2174595.0:91154 0.000000 2014-08-13 18:06:06.730714 7f5c973a57a0 20 read_log 194215'12161 (193305'12112) clean cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by osd.4.0:1994 2014-08-12 15:43:01.207468 2014-08-13 18:06:06.730735 7f5c973a57a0 20 read_log 194215'12162 (194215'12161) delete cc7431bb/benchmark_data_storage0111-ib.pocstor_62621_object4704/head//24 by client.2174595.0:91156 0.000000 2014-08-13 18:06:06.730746 7f5c973a57a0 20 read_log 194215'12163 (193305'12119) clean 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by osd.4.0:1996 2014-08-12 15:43:01.215417 2014-08-13 18:06:06.730759 7f5c973a57a0 20 read_log 194215'12164 (194215'12163) delete 298441bb/benchmark_data_storage0111-ib.pocstor_62621_object14239/head//24 by client.2174595.0:91158 0.000000 2014-08-13 18:06:06.730769 7f5c973a57a0 20 read_log 194215'12165 (192888'11796) clean 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by osd.4.0:1998 2014-08-12 15:43:01.224201 2014-08-13 18:06:06.730781 7f5c973a57a0 20 read_log 194215'12166 (194215'12165) delete 8a441bb/benchmark_data_storage0111-ib.pocstor_41097_object774/head//24 by client.2174595.0:91160 0.000000 2014-08-13 18:06:06.730792 7f5c973a57a0 20 read_log 194215'12167 (192888'11833) clean b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by osd.4.0:2000 2014-08-12 15:43:01.232796 2014-08-13 18:06:06.730804 7f5c973a57a0 20 read_log 194215'12168 (194215'12167) delete b75d41bb/benchmark_data_storage0111-ib.pocstor_41097_object16979/head//24 by client.2174595.0:91162 0.000000 2014-08-13 18:06:06.730815 7f5c973a57a0 20 read_log 194215'12169 (193305'12125) clean 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by osd.4.0:2002 2014-08-12 15:43:01.271760 2014-08-13 18:06:06.730827 7f5c973a57a0 20 read_log 194215'12170 (194215'12169) delete 776e51bb/benchmark_data_storage0111-ib.pocstor_62621_object19170/head//24 by client.2174595.0:91164 0.000000 2014-08-13 18:06:06.730838 7f5c973a57a0 20 read_log 194215'12171 (192888'11989) clean c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by osd.4.0:2004 2014-08-12 15:43:01.280633 2014-08-13 18:06:06.730850 7f5c973a57a0 20 read_log 194215'12172 (194215'12171) delete c7e071bb/benchmark_data_storage0111-ib.pocstor_2228_object4345/head//24 by client.2174595.0:91166 0.000000 2014-08-13 18:06:06.730860 7f5c973a57a0 20 read_log 194215'12173 (192888'11800) clean 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by osd.4.0:2006 2014-08-12 15:43:01.289852 2014-08-13 18:06:06.730873 7f5c973a57a0 20 read_log 194215'12174 (194215'12173) delete 3ba171bb/benchmark_data_storage0111-ib.pocstor_41097_object1575/head//24 by client.2174595.0:91168 0.000000 2014-08-13 18:06:06.730883 7f5c973a57a0 20 read_log 194215'12175 (192834'10882) clean d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by osd.4.0:2008 2014-08-12 15:43:01.317002 2014-08-13 18:06:06.730895 7f5c973a57a0 20 read_log 194215'12176 (194215'12175) delete d82371bb/benchmark_data_storage0111-ib.pocstor_26659_object124/head//24 by client.2174595.0:91170 0.000000 2014-08-13 18:06:06.730906 7f5c973a57a0 20 read_log 194215'12177 (193305'12114) clean b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by osd.4.0:2010 2014-08-12 15:43:01.325368 2014-08-13 18:06:06.730918 7f5c973a57a0 20 read_log 194215'12178 (194215'12177) delete b95371bb/benchmark_data_storage0111-ib.pocstor_62621_object7582/head//24 by client.2174595.0:91172 0.000000 2014-08-13 18:06:06.730929 7f5c973a57a0 20 read_log 194215'12179 (192834'10912) clean e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by osd.4.0:2012 2014-08-12 15:43:01.358444 2014-08-13 18:06:06.730941 7f5c973a57a0 20 read_log 194215'12180 (194215'12179) delete e79571bb/benchmark_data_storage0111-ib.pocstor_26659_object2437/head//24 by client.2174595.0:91174 0.000000 2014-08-13 18:06:06.730952 7f5c973a57a0 20 read_log 194215'12181 (192834'11536) clean a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by osd.4.0:2014 2014-08-12 15:43:01.366447 2014-08-13 18:06:06.730974 7f5c973a57a0 20 read_log 194215'12182 (194215'12181) delete a01771bb/benchmark_data_storage0111-ib.pocstor_41526_object8535/head//24 by client.2174595.0:91176 0.000000 2014-08-13 18:06:06.730988 7f5c973a57a0 20 read_log 194215'12183 (193305'12124) clean a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by osd.4.0:2016 2014-08-12 15:43:01.375336 2014-08-13 18:06:06.731001 7f5c973a57a0 20 read_log 194215'12184 (194215'12183) delete a7fa71bb/benchmark_data_storage0111-ib.pocstor_62621_object18676/head//24 by client.2174595.0:91178 0.000000 2014-08-13 18:06:06.731014 7f5c973a57a0 20 read_log 194215'12185 (192888'11996) clean b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by osd.4.0:2018 2014-08-12 15:43:01.383416 2014-08-13 18:06:06.731027 7f5c973a57a0 20 read_log 194215'12186 (194215'12185) delete b5db71bb/benchmark_data_storage0111-ib.pocstor_2228_object9915/head//24 by client.2174595.0:91180 0.000000 2014-08-13 18:06:06.731039 7f5c973a57a0 20 read_log 194215'12187 (192888'11839) clean 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by osd.4.0:2020 2014-08-12 15:43:01.390428 2014-08-13 18:06:06.731054 7f5c973a57a0 20 read_log 194215'12188 (194215'12187) delete 1ed181bb/benchmark_data_storage0111-ib.pocstor_41097_object18967/head//24 by client.2174595.0:91182 0.000000 2014-08-13 18:06:06.731065 7f5c973a57a0 20 read_log 194215'12189 (192834'11641) clean b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by osd.4.0:2022 2014-08-12 15:43:01.399080 2014-08-13 18:06:06.731077 7f5c973a57a0 20 read_log 194215'12190 (194215'12189) delete b8581bb/benchmark_data_storage0111-ib.pocstor_41526_object20438/head//24 by client.2174595.0:91184 0.000000 2014-08-13 18:06:06.731090 7f5c973a57a0 20 read_log 194215'12191 (192834'11635) clean e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by osd.4.0:2024 2014-08-12 15:43:01.415487 2014-08-13 18:06:06.731102 7f5c973a57a0 20 read_log 194215'12192 (194215'12191) delete e29b81bb/benchmark_data_storage0111-ib.pocstor_41526_object20117/head//24 by client.2174595.0:91186 0.000000 2014-08-13 18:06:06.731113 7f5c973a57a0 20 read_log 194215'12193 (192888'11816) clean 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by osd.4.0:2026 2014-08-12 15:43:01.423103 2014-08-13 18:06:06.731125 7f5c973a57a0 20 read_log 194215'12194 (194215'12193) delete 40a291bb/benchmark_data_storage0111-ib.pocstor_41097_object10896/head//24 by client.2174595.0:91188 0.000000 2014-08-13 18:06:06.731136 7f5c973a57a0 20 read_log 194215'12195 (192834'10974) clean b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by osd.4.0:2028 2014-08-12 15:43:01.431392 2014-08-13 18:06:06.731154 7f5c973a57a0 20 read_log 194215'12196 (194215'12195) delete b8a5a1bb/benchmark_data_storage0111-ib.pocstor_26659_object9662/head//24 by client.2174595.0:91190 0.000000 2014-08-13 18:06:06.731168 7f5c973a57a0 20 read_log 194215'12197 (192834'10966) clean cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by osd.4.0:2030 2014-08-12 15:43:01.439942 2014-08-13 18:06:06.731180 7f5c973a57a0 20 read_log 194215'12198 (194215'12197) delete cb1ba1bb/benchmark_data_storage0111-ib.pocstor_26659_object9073/head//24 by client.2174595.0:91192 0.000000 2014-08-13 18:06:06.731191 7f5c973a57a0 20 read_log 194215'12199 (192834'10976) clean 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by osd.4.0:2032 2014-08-12 15:43:01.447709 2014-08-13 18:06:06.731204 7f5c973a57a0 20 read_log 194215'12200 (194215'12199) delete 30c5c1bb/benchmark_data_storage0111-ib.pocstor_26659_object9929/head//24 by client.2174595.0:91194 0.000000 2014-08-13 18:06:06.731215 7f5c973a57a0 20 read_log 194215'12201 (193305'12109) clean 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by osd.4.0:2034 2014-08-12 15:43:01.475831 2014-08-13 18:06:06.731236 7f5c973a57a0 20 read_log 194215'12202 (194215'12201) delete 5d29c1bb/benchmark_data_storage0111-ib.pocstor_62621_object1279/head//24 by client.2174595.0:91196 0.000000 2014-08-13 18:06:06.731249 7f5c973a57a0 20 read_log 194215'12203 (193305'12134) clean 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by osd.4.0:2036 2014-08-12 15:43:01.484596 2014-08-13 18:06:06.731261 7f5c973a57a0 20 read_log 194215'12204 (194215'12203) delete 8fdc1bb/benchmark_data_storage0111-ib.pocstor_62621_object29214/head//24 by client.2174595.0:91198 0.000000 2014-08-13 18:06:06.731272 7f5c973a57a0 20 read_log 194215'12205 (193305'12118) clean d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by osd.4.0:2038 2014-08-12 15:43:01.493062 2014-08-13 18:06:06.731287 7f5c973a57a0 20 read_log 194215'12206 (194215'12205) delete d892d1bb/benchmark_data_storage0111-ib.pocstor_62621_object13848/head//24 by client.2174595.0:91200 0.000000 2014-08-13 18:06:06.731298 7f5c973a57a0 20 read_log 194215'12207 (192834'10979) clean 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by osd.4.0:2040 2014-08-12 15:43:01.502501 2014-08-13 18:06:06.731311 7f5c973a57a0 20 read_log 194215'12208 (194215'12207) delete 4ee6d1bb/benchmark_data_storage0111-ib.pocstor_26659_object10341/head//24 by client.2174595.0:91202 0.000000 2014-08-13 18:06:06.731321 7f5c973a57a0 20 read_log 194215'12209 (192888'11856) clean 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by osd.4.0:2042 2014-08-12 15:43:01.510973 2014-08-13 18:06:06.731337 7f5c973a57a0 20 read_log 194215'12210 (194215'12209) delete 2e08d1bb/benchmark_data_storage0111-ib.pocstor_41097_object29104/head//24 by client.2174595.0:91204 0.000000 2014-08-13 18:06:06.731348 7f5c973a57a0 20 read_log 194215'12211 (192888'11805) clean a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by osd.4.0:2044 2014-08-12 15:43:01.520792 2014-08-13 18:06:06.731360 7f5c973a57a0 20 read_log 194215'12212 (194215'12211) delete a71ad1bb/benchmark_data_storage0111-ib.pocstor_41097_object4959/head//24 by client.2174595.0:91206 0.000000 2014-08-13 18:06:06.731371 7f5c973a57a0 20 read_log 194215'12213 (192888'11846) clean be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by osd.4.0:2046 2014-08-12 15:43:01.529006 2014-08-13 18:06:06.731386 7f5c973a57a0 20 read_log 194215'12214 (194215'12213) delete be53e1bb/benchmark_data_storage0111-ib.pocstor_41097_object22791/head//24 by client.2174595.0:91208 0.000000 2014-08-13 18:06:06.731397 7f5c973a57a0 20 read_log 194215'12215 (192834'10895) clean 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by osd.4.0:2048 2014-08-12 15:43:01.537214 2014-08-13 18:06:06.731419 7f5c973a57a0 20 read_log 194215'12216 (194215'12215) delete 61e3e1bb/benchmark_data_storage0111-ib.pocstor_26659_object1027/head//24 by client.2174595.0:91210 0.000000 2014-08-13 18:06:06.731437 7f5c973a57a0 20 read_log 194215'12217 (193305'12117) clean 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by osd.4.0:2050 2014-08-12 15:43:01.546200 2014-08-13 18:06:06.731450 7f5c973a57a0 20 read_log 194215'12218 (194215'12217) delete 3b17e1bb/benchmark_data_storage0111-ib.pocstor_62621_object13319/head//24 by client.2174595.0:91212 0.000000 2014-08-13 18:06:06.731471 7f5c973a57a0 20 read_log 194215'12219 (192888'12002) clean 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by osd.4.0:2052 2014-08-12 15:43:01.554017 2014-08-13 18:06:06.731484 7f5c973a57a0 20 read_log 194215'12220 (194215'12219) delete 4ec8e1bb/benchmark_data_storage0111-ib.pocstor_2228_object12129/head//24 by client.2174595.0:91214 0.000000 2014-08-13 18:06:06.731495 7f5c973a57a0 20 read_log 194215'12221 (193305'12127) clean a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by osd.4.0:2054 2014-08-12 15:43:01.562008 2014-08-13 18:06:06.731520 7f5c973a57a0 20 read_log 194215'12222 (194215'12221) delete a249e1bb/benchmark_data_storage0111-ib.pocstor_62621_object19867/head//24 by client.2174595.0:91216 0.000000 2014-08-13 18:06:06.731534 7f5c973a57a0 20 read_log 194215'12223 (192888'11832) clean 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by osd.4.0:2056 2014-08-12 15:43:01.570499 2014-08-13 18:06:06.731547 7f5c973a57a0 20 read_log 194215'12224 (194215'12223) delete 756be1bb/benchmark_data_storage0111-ib.pocstor_41097_object16392/head//24 by client.2174595.0:91218 0.000000 2014-08-13 18:06:06.731565 7f5c973a57a0 20 read_log 194215'12225 (192888'11843) clean ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by osd.4.0:2058 2014-08-12 15:43:01.578588 2014-08-13 18:06:06.731581 7f5c973a57a0 20 read_log 194215'12226 (194215'12225) delete ac39f1bb/benchmark_data_storage0111-ib.pocstor_41097_object22101/head//24 by client.2174595.0:91220 0.000000 2014-08-13 18:06:06.731592 7f5c973a57a0 20 read_log 194215'12227 (192834'11447) clean f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by osd.4.0:2060 2014-08-12 15:43:01.586578 2014-08-13 18:06:06.731615 7f5c973a57a0 20 read_log 194215'12228 (194215'12227) delete f8d005bb/benchmark_data_storage0111-ib.pocstor_41526_object73/head//24 by client.2174595.0:91222 0.000000 2014-08-13 18:06:06.731630 7f5c973a57a0 20 read_log 194215'12229 (193305'12137) clean f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by osd.4.0:2062 2014-08-12 15:43:01.594942 2014-08-13 18:06:06.731642 7f5c973a57a0 20 read_log 194215'12230 (194215'12229) delete f60305bb/benchmark_data_storage0111-ib.pocstor_62621_object31366/head//24 by client.2174595.0:91224 0.000000 2014-08-13 18:06:06.731653 7f5c973a57a0 20 read_log 194215'12231 (192888'12005) clean 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by osd.4.0:2064 2014-08-12 15:43:01.603052 2014-08-13 18:06:06.731666 7f5c973a57a0 20 read_log 194215'12232 (194215'12231) delete 30e305bb/benchmark_data_storage0111-ib.pocstor_2228_object13492/head//24 by client.2174595.0:91226 0.000000 2014-08-13 18:06:06.731676 7f5c973a57a0 20 read_log 194215'12233 (192834'10945) clean ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by osd.4.0:2066 2014-08-12 15:43:01.610619 2014-08-13 18:06:06.731689 7f5c973a57a0 20 read_log 194215'12234 (194215'12233) delete ac9b05bb/benchmark_data_storage0111-ib.pocstor_26659_object6771/head//24 by client.2174595.0:91228 0.000000 2014-08-13 18:06:06.731699 7f5c973a57a0 20 read_log 194215'12235 (193305'12138) clean ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by osd.4.0:2068 2014-08-12 15:43:01.618313 2014-08-13 18:06:06.731712 7f5c973a57a0 20 read_log 194215'12236 (194215'12235) delete ab0415bb/benchmark_data_storage0111-ib.pocstor_62621_object31724/head//24 by client.2174595.0:91230 0.000000 2014-08-13 18:06:06.731722 7f5c973a57a0 20 read_log 194215'12237 (192888'12026) clean 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by osd.4.0:2070 2014-08-12 15:43:01.627948 2014-08-13 18:06:06.731734 7f5c973a57a0 20 read_log 194215'12238 (194215'12237) delete 4fc325bb/benchmark_data_storage0111-ib.pocstor_2228_object24363/head//24 by client.2174595.0:91232 0.000000 2014-08-13 18:06:06.731745 7f5c973a57a0 20 read_log 194215'12239 (192834'11448) delete c34925bb/benchmark_data_storage0111-ib.pocstor_26659_object838/head//24 by client.2174595.0:91234 0.000000 2014-08-13 18:06:06.731756 7f5c973a57a0 20 read_log 194215'12240 (192834'11449) delete 87bf25bb/benchmark_data_storage0111-ib.pocstor_26659_object14978/head//24 by client.2174595.0:91236 0.000000 2014-08-13 18:06:06.731769 7f5c973a57a0 20 read_log 194215'12241 (193305'12136) clean fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by osd.4.0:2074 2014-08-12 15:43:01.649557 2014-08-13 18:06:06.731819 7f5c973a57a0 20 read_log 194215'12242 (194215'12241) delete fa4135bb/benchmark_data_storage0111-ib.pocstor_62621_object30314/head//24 by client.2174595.0:91238 0.000000 2014-08-13 18:06:06.731834 7f5c973a57a0 20 read_log 194215'12243 (192888'12042) clean 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by osd.4.0:2076 2014-08-12 15:43:01.657739 2014-08-13 18:06:06.731847 7f5c973a57a0 20 read_log 194215'12244 (194215'12243) delete 234935bb/benchmark_data_storage0111-ib.pocstor_2228_object28851/head//24 by client.2174595.0:91240 0.000000 2014-08-13 18:06:06.731858 7f5c973a57a0 20 read_log 194215'12245 (192834'11595) clean 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by osd.4.0:2078 2014-08-12 15:43:01.666949 2014-08-13 18:06:06.731879 7f5c973a57a0 20 read_log 194215'12246 (194215'12245) delete 987e45bb/benchmark_data_storage0111-ib.pocstor_41526_object15749/head//24 by client.2174595.0:91242 0.000000 2014-08-13 18:06:06.731893 7f5c973a57a0 20 read_log 194215'12247 (192834'11593) clean 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by osd.4.0:2080 2014-08-12 15:43:01.692863 2014-08-13 18:06:06.731906 7f5c973a57a0 20 read_log 194215'12248 (194215'12247) delete 4be45bb/benchmark_data_storage0111-ib.pocstor_41526_object15530/head//24 by client.2174595.0:91244 0.000000 2014-08-13 18:06:06.731925 7f5c973a57a0 20 read_log 194215'12249 (192834'11554) clean 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by osd.4.0:2082 2014-08-12 15:43:01.709869 2014-08-13 18:06:06.731940 7f5c973a57a0 20 read_log 194215'12250 (194215'12249) delete 76de45bb/benchmark_data_storage0111-ib.pocstor_41526_object10721/head//24 by client.2174595.0:91246 0.000000 2014-08-13 18:06:06.731951 7f5c973a57a0 20 read_log 194215'12251 (192834'11560) clean 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by osd.4.0:2084 2014-08-12 15:43:01.721246 2014-08-13 18:06:06.731964 7f5c973a57a0 20 read_log 194215'12252 (194215'12251) delete 9e8055bb/benchmark_data_storage0111-ib.pocstor_41526_object11327/head//24 by client.2174595.0:91248 0.000000 2014-08-13 18:06:06.731975 7f5c973a57a0 20 read_log 194215'12253 (192888'11838) clean f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by osd.4.0:2086 2014-08-12 15:43:01.729438 2014-08-13 18:06:06.731987 7f5c973a57a0 20 read_log 194215'12254 (194215'12253) delete f4f255bb/benchmark_data_storage0111-ib.pocstor_41097_object18555/head//24 by client.2174595.0:91250 0.000000 2014-08-13 18:06:06.732000 7f5c973a57a0 20 read_log 194215'12255 (192888'11807) clean cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by osd.4.0:2088 2014-08-12 15:43:01.737263 2014-08-13 18:06:06.732012 7f5c973a57a0 20 read_log 194215'12256 (194215'12255) delete cad355bb/benchmark_data_storage0111-ib.pocstor_41097_object5657/head//24 by client.2174595.0:91252 0.000000 2014-08-13 18:06:06.732025 7f5c973a57a0 20 read_log 194215'12257 (192888'12020) clean d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by osd.4.0:2090 2014-08-12 15:43:01.745848 2014-08-13 18:06:06.732040 7f5c973a57a0 20 read_log 194215'12258 (194215'12257) delete d5bc55bb/benchmark_data_storage0111-ib.pocstor_2228_object20791/head//24 by client.2174595.0:91254 0.000000 2014-08-13 18:06:06.732055 7f5c973a57a0 20 read_log 194215'12259 (192888'12033) clean ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by osd.4.0:2092 2014-08-12 15:43:01.754735 2014-08-13 18:06:06.732067 7f5c973a57a0 20 read_log 194215'12260 (194215'12259) delete ae3665bb/benchmark_data_storage0111-ib.pocstor_2228_object25915/head//24 by client.2174595.0:91256 0.000000 2014-08-13 18:06:06.732078 7f5c973a57a0 20 read_log 194215'12261 (192834'11577) clean 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by osd.4.0:2094 2014-08-12 15:43:01.766498 2014-08-13 18:06:06.732100 7f5c973a57a0 20 read_log 194215'12262 (194215'12261) delete 62f665bb/benchmark_data_storage0111-ib.pocstor_41526_object13531/head//24 by client.2174595.0:91258 0.000000 2014-08-13 18:06:06.732115 7f5c973a57a0 20 read_log 194215'12263 (192888'12044) clean 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by osd.4.0:2096 2014-08-12 15:43:01.775332 2014-08-13 18:06:06.732127 7f5c973a57a0 20 read_log 194215'12264 (194215'12263) delete 4aad65bb/benchmark_data_storage0111-ib.pocstor_2228_object29994/head//24 by client.2174595.0:91260 0.000000 2014-08-13 18:06:06.732138 7f5c973a57a0 20 read_log 194215'12265 (193305'12126) clean 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by osd.4.0:2098 2014-08-12 15:43:01.782695 2014-08-13 18:06:06.732156 7f5c973a57a0 20 read_log 194215'12266 (194215'12265) delete 664075bb/benchmark_data_storage0111-ib.pocstor_62621_object19796/head//24 by client.2174595.0:91262 0.000000 2014-08-13 18:06:06.732170 7f5c973a57a0 20 read_log 194215'12267 (192888'11801) clean 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by osd.4.0:2100 2014-08-12 15:43:01.791244 2014-08-13 18:06:06.732194 7f5c973a57a0 20 read_log 194215'12268 (194215'12267) delete 201e75bb/benchmark_data_storage0111-ib.pocstor_41097_object1827/head//24 by client.2174595.0:91264 0.000000 2014-08-13 18:06:06.732212 7f5c973a57a0 20 read_log 194215'12269 (192888'12010) clean 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by osd.4.0:2102 2014-08-12 15:43:01.800773 2014-08-13 18:06:06.732232 7f5c973a57a0 20 read_log 194215'12270 (194215'12269) delete 15ba85bb/benchmark_data_storage0111-ib.pocstor_2228_object14820/head//24 by client.2174595.0:91266 0.000000 2014-08-13 18:06:06.732244 7f5c973a57a0 20 read_log 194215'12271 (192888'11823) clean 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by osd.4.0:2104 2014-08-12 15:43:01.809029 2014-08-13 18:06:06.732260 7f5c973a57a0 20 read_log 194215'12272 (194215'12271) delete 773c85bb/benchmark_data_storage0111-ib.pocstor_41097_object13556/head//24 by client.2174595.0:91268 0.000000 2014-08-13 18:06:06.732277 7f5c973a57a0 20 read_log 194215'12273 (192888'11990) clean 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by osd.4.0:2106 2014-08-12 15:43:01.818077 2014-08-13 18:06:06.732291 7f5c973a57a0 20 read_log 194215'12274 (194215'12273) delete 4d8195bb/benchmark_data_storage0111-ib.pocstor_2228_object6115/head//24 by client.2174595.0:91270 0.000000 2014-08-13 18:06:06.732305 7f5c973a57a0 20 read_log 194215'12275 (192834'11550) clean 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by osd.4.0:2108 2014-08-12 15:43:01.831228 2014-08-13 18:06:06.732317 7f5c973a57a0 20 read_log 194215'12276 (194215'12275) delete 210595bb/benchmark_data_storage0111-ib.pocstor_41526_object9820/head//24 by client.2174595.0:91272 0.000000 2014-08-13 18:06:06.732328 7f5c973a57a0 20 read_log 194215'12277 (192834'11659) clean 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by osd.4.0:2110 2014-08-12 15:43:01.839735 2014-08-13 18:06:06.732346 7f5c973a57a0 20 read_log 194215'12278 (194215'12277) delete 8640a5bb/benchmark_data_storage0111-ib.pocstor_41526_object22666/head//24 by client.2174595.0:91274 0.000000 2014-08-13 18:06:06.732358 7f5c973a57a0 20 read_log 194215'12279 (192888'12028) clean fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by osd.4.0:2112 2014-08-12 15:43:01.847838 2014-08-13 18:06:06.732370 7f5c973a57a0 20 read_log 194215'12280 (194215'12279) delete fbeca5bb/benchmark_data_storage0111-ib.pocstor_2228_object24819/head//24 by client.2174595.0:91276 0.000000 2014-08-13 18:06:06.732381 7f5c973a57a0 20 read_log 194215'12281 (192888'12016) clean 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by osd.4.0:2114 2014-08-12 15:43:01.856134 2014-08-13 18:06:06.732405 7f5c973a57a0 20 read_log 194215'12282 (194215'12281) delete 177ea5bb/benchmark_data_storage0111-ib.pocstor_2228_object17741/head//24 by client.2174595.0:91278 0.000000 2014-08-13 18:06:06.732416 7f5c973a57a0 20 read_log 194215'12283 (192834'11454) delete b990b5bb/benchmark_data_storage0111-ib.pocstor_26659_object6653/head//24 by client.2174595.0:91280 0.000000 2014-08-13 18:06:06.732427 7f5c973a57a0 20 read_log 194215'12284 (193305'12120) clean 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by osd.4.0:2117 2014-08-12 15:43:01.868452 2014-08-13 18:06:06.732443 7f5c973a57a0 20 read_log 194215'12285 (194215'12284) delete 3b21c5bb/benchmark_data_storage0111-ib.pocstor_62621_object14287/head//24 by client.2174595.0:91282 0.000000 2014-08-13 18:06:06.732454 7f5c973a57a0 20 read_log 194215'12286 (192834'11456) delete c275c5bb/benchmark_data_storage0111-ib.pocstor_26659_object8836/head//24 by client.2174595.0:91284 0.000000 2014-08-13 18:06:06.732477 7f5c973a57a0 20 read_log 194215'12287 (193305'12133) clean 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by osd.4.0:2120 2014-08-12 15:43:01.881741 2014-08-13 18:06:06.732513 7f5c973a57a0 20 read_log 194215'12288 (194215'12287) delete 51ddc5bb/benchmark_data_storage0111-ib.pocstor_62621_object28991/head//24 by client.2174595.0:91286 0.000000 2014-08-13 18:06:06.732528 7f5c973a57a0 20 read_log 194215'12289 (192888'11987) clean e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by osd.4.0:2122 2014-08-12 15:43:01.890084 2014-08-13 18:06:06.732546 7f5c973a57a0 20 read_log 194215'12290 (194215'12289) delete e8d1d5bb/benchmark_data_storage0111-ib.pocstor_2228_object3436/head//24 by client.2174595.0:91288 0.000000 2014-08-13 18:06:06.732557 7f5c973a57a0 20 read_log 194215'12291 (193305'12116) clean 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by osd.4.0:2124 2014-08-12 15:43:01.898213 2014-08-13 18:06:06.732570 7f5c973a57a0 20 read_log 194215'12292 (194215'12291) delete 23c3d5bb/benchmark_data_storage0111-ib.pocstor_62621_object12981/head//24 by client.2174595.0:91290 0.000000 2014-08-13 18:06:06.732590 7f5c973a57a0 20 read_log 194215'12293 (193305'12111) clean 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by osd.4.0:2126 2014-08-12 15:43:01.906248 2014-08-13 18:06:06.732603 7f5c973a57a0 20 read_log 194215'12294 (194215'12293) delete 3308d5bb/benchmark_data_storage0111-ib.pocstor_62621_object4478/head//24 by client.2174595.0:91292 0.000000 2014-08-13 18:06:06.732614 7f5c973a57a0 20 read_log 194215'12295 (192834'11457) delete bc16e5bb/benchmark_data_storage0111-ib.pocstor_26659_object3826/head//24 by client.2174595.0:91294 0.000000 2014-08-13 18:06:06.732625 7f5c973a57a0 20 read_log 194215'12296 (192834'11510) clean 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by osd.4.0:2129 2014-08-12 15:43:01.923826 2014-08-13 18:06:06.732652 7f5c973a57a0 20 read_log 194215'12297 (194215'12296) delete 4528e5bb/benchmark_data_storage0111-ib.pocstor_41526_object6392/head//24 by client.2174595.0:91296 0.000000 2014-08-13 18:06:06.732664 7f5c973a57a0 20 read_log 194215'12298 (192888'12014) clean 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by osd.4.0:2131 2014-08-12 15:43:01.933176 2014-08-13 18:06:06.732676 7f5c973a57a0 20 read_log 194215'12299 (194215'12298) delete 643ff5bb/benchmark_data_storage0111-ib.pocstor_2228_object17319/head//24 by client.2174595.0:91298 0.000000 2014-08-13 18:06:06.732688 7f5c973a57a0 20 read_log 194215'12300 (192834'11487) clean ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by osd.4.0:2133 2014-08-12 15:43:01.944654 2014-08-13 18:06:06.732700 7f5c973a57a0 20 read_log 194215'12301 (194215'12300) delete ef3ff5bb/benchmark_data_storage0111-ib.pocstor_41526_object4120/head//24 by client.2174595.0:91300 0.000000 2014-08-13 18:06:06.732718 7f5c973a57a0 20 read_log 194215'12302 (192888'11837) clean f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by osd.4.0:2135 2014-08-12 15:43:01.952827 2014-08-13 18:06:06.732733 7f5c973a57a0 20 read_log 194215'12303 (194215'12302) delete f3aa09bb/benchmark_data_storage0111-ib.pocstor_41097_object18541/head//24 by client.2174595.0:91302 0.000000 2014-08-13 18:06:06.732744 7f5c973a57a0 20 read_log 194215'12304 (192888'11834) clean 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by osd.4.0:2137 2014-08-12 15:43:01.960696 2014-08-13 18:06:06.732757 7f5c973a57a0 20 read_log 194215'12305 (194215'12304) delete 20da09bb/benchmark_data_storage0111-ib.pocstor_41097_object17296/head//24 by client.2174595.0:91304 0.000000 2014-08-13 18:06:06.732768 7f5c973a57a0 20 read_log 194215'12306 (192888'11994) clean 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by osd.4.0:2139 2014-08-12 15:43:01.987108 2014-08-13 18:06:06.732780 7f5c973a57a0 20 read_log 194215'12307 (194215'12306) delete 552519bb/benchmark_data_storage0111-ib.pocstor_2228_object8509/head//24 by client.2174595.0:91306 0.000000 2014-08-13 18:06:06.732791 7f5c973a57a0 20 read_log 194215'12308 (192888'12041) clean d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by osd.4.0:2141 2014-08-12 15:43:01.995639 2014-08-13 18:06:06.732803 7f5c973a57a0 20 read_log 194215'12309 (194215'12308) delete d4ae19bb/benchmark_data_storage0111-ib.pocstor_2228_object28642/head//24 by client.2174595.0:91308 0.000000 2014-08-13 18:06:06.732814 7f5c973a57a0 20 read_log 194215'12310 (193305'12115) clean 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by osd.4.0:2143 2014-08-12 15:43:02.003868 2014-08-13 18:06:06.732826 7f5c973a57a0 20 read_log 194215'12311 (194215'12310) delete 850229bb/benchmark_data_storage0111-ib.pocstor_62621_object10103/head//24 by client.2174595.0:91310 0.000000 2014-08-13 18:06:06.732837 7f5c973a57a0 20 read_log 194215'12312 (192834'11628) clean eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by osd.4.0:2145 2014-08-12 15:43:02.012919 2014-08-13 18:06:06.732849 7f5c973a57a0 20 read_log 194215'12313 (194215'12312) delete eb0829bb/benchmark_data_storage0111-ib.pocstor_41526_object19205/head//24 by client.2174595.0:91312 0.000000 2014-08-13 18:06:06.732860 7f5c973a57a0 20 read_log 194215'12314 (193305'12123) clean 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by osd.4.0:2147 2014-08-12 15:43:02.021342 2014-08-13 18:06:06.732872 7f5c973a57a0 20 read_log 194215'12315 (194215'12314) delete 63fa39bb/benchmark_data_storage0111-ib.pocstor_62621_object18129/head//24 by client.2174595.0:91314 0.000000 2014-08-13 18:06:06.732883 7f5c973a57a0 20 read_log 194215'12316 (192888'11997) clean bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by osd.4.0:2149 2014-08-12 15:43:02.029783 2014-08-13 18:06:06.732895 7f5c973a57a0 20 read_log 194215'12317 (194215'12316) delete bd8e49bb/benchmark_data_storage0111-ib.pocstor_2228_object10682/head//24 by client.2174595.0:91316 0.000000 2014-08-13 18:06:06.732906 7f5c973a57a0 20 read_log 194215'12318 (192888'11851) clean 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by osd.4.0:2151 2014-08-12 15:43:02.037265 2014-08-13 18:06:06.732918 7f5c973a57a0 20 read_log 194215'12319 (194215'12318) delete 216159bb/benchmark_data_storage0111-ib.pocstor_41097_object25787/head//24 by client.2174595.0:91318 0.000000 2014-08-13 18:06:06.732929 7f5c973a57a0 20 read_log 194215'12320 (193305'12122) clean e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by osd.4.0:2153 2014-08-12 15:43:02.046180 2014-08-13 18:06:06.732941 7f5c973a57a0 20 read_log 194215'12321 (194215'12320) delete e1f259bb/benchmark_data_storage0111-ib.pocstor_62621_object17139/head//24 by client.2174595.0:91320 0.000000 2014-08-13 18:06:06.732962 7f5c973a57a0 20 read_log 194215'12322 (192888'11860) clean f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by osd.4.0:2155 2014-08-12 15:43:02.054124 2014-08-13 18:06:06.732975 7f5c973a57a0 20 read_log 194215'12323 (194215'12322) delete f8d759bb/benchmark_data_storage0111-ib.pocstor_41097_object30246/head//24 by client.2174595.0:91322 0.000000 2014-08-13 18:06:06.732990 7f5c973a57a0 20 read_log 194215'12324 (192834'10994) clean 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by osd.4.0:2157 2014-08-12 15:43:02.063668 2014-08-13 18:06:06.733003 7f5c973a57a0 20 read_log 194215'12325 (194215'12324) delete 335069bb/benchmark_data_storage0111-ib.pocstor_26659_object12133/head//24 by client.2174595.0:91324 0.000000 2014-08-13 18:06:06.733013 7f5c973a57a0 20 read_log 194215'12326 (192888'11822) clean 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by osd.4.0:2159 2014-08-12 15:43:02.072405 2014-08-13 18:06:06.733028 7f5c973a57a0 20 read_log 194215'12327 (194215'12326) delete 4e2169bb/benchmark_data_storage0111-ib.pocstor_41097_object13323/head//24 by client.2174595.0:91326 0.000000 2014-08-13 18:06:06.733039 7f5c973a57a0 20 read_log 194215'12328 (192834'11482) clean 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by osd.4.0:2161 2014-08-12 15:43:02.081532 2014-08-13 18:06:06.733052 7f5c973a57a0 20 read_log 194215'12329 (194215'12328) delete 425d69bb/benchmark_data_storage0111-ib.pocstor_41526_object2830/head//24 by client.2174595.0:91328 0.000000 2014-08-13 18:06:06.733064 7f5c973a57a0 20 read_log 194215'12330 (193305'12113) clean 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by osd.4.0:2163 2014-08-12 15:43:02.089847 2014-08-13 18:06:06.733078 7f5c973a57a0 20 read_log 194215'12331 (194215'12330) delete 94a579bb/benchmark_data_storage0111-ib.pocstor_62621_object6893/head//24 by client.2174595.0:91330 0.000000 2014-08-13 18:06:06.733089 7f5c973a57a0 20 read_log 194215'12332 (193305'12132) clean 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by osd.4.0:2165 2014-08-12 15:43:02.098139 2014-08-13 18:06:06.733102 7f5c973a57a0 20 read_log 194215'12333 (194215'12332) delete 4f8d79bb/benchmark_data_storage0111-ib.pocstor_62621_object28148/head//24 by client.2174595.0:91332 0.000000 2014-08-13 18:06:06.733134 7f5c973a57a0 20 read_log 194215'12334 (192888'12000) clean cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by osd.4.0:2167 2014-08-12 15:43:02.108194 2014-08-13 18:06:06.733155 7f5c973a57a0 20 read_log 194215'12335 (194215'12334) delete cf9089bb/benchmark_data_storage0111-ib.pocstor_2228_object11155/head//24 by client.2174595.0:91334 0.000000 2014-08-13 18:06:06.733169 7f5c973a57a0 20 read_log 194215'12336 (192888'12030) clean bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by osd.4.0:2169 2014-08-12 15:43:02.116334 2014-08-13 18:06:06.733181 7f5c973a57a0 20 read_log 194215'12337 (194215'12336) delete bd9099bb/benchmark_data_storage0111-ib.pocstor_2228_object25074/head//24 by client.2174595.0:91336 0.000000 2014-08-13 18:06:06.733192 7f5c973a57a0 20 read_log 194215'12338 (192834'10978) clean c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by osd.4.0:2171 2014-08-12 15:43:02.124821 2014-08-13 18:06:06.733206 7f5c973a57a0 20 read_log 194215'12339 (194215'12338) delete c9b299bb/benchmark_data_storage0111-ib.pocstor_26659_object10193/head//24 by client.2174595.0:91338 0.000000 2014-08-13 18:06:06.733217 7f5c973a57a0 20 read_log 194215'12340 (192888'12037) clean dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by osd.4.0:2173 2014-08-12 15:43:02.133316 2014-08-13 18:06:06.733230 7f5c973a57a0 20 read_log 194215'12341 (194215'12340) delete dc3c99bb/benchmark_data_storage0111-ib.pocstor_2228_object27399/head//24 by client.2174595.0:91340 0.000000 2014-08-13 18:06:06.733250 7f5c973a57a0 20 read_log 194215'12342 (192888'12006) clean 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by osd.4.0:2175 2014-08-12 15:43:02.141708 2014-08-13 18:06:06.733265 7f5c973a57a0 20 read_log 194215'12343 (194215'12342) delete 7a6a9bb/benchmark_data_storage0111-ib.pocstor_2228_object13921/head//24 by client.2174595.0:91342 0.000000 2014-08-13 18:06:06.733276 7f5c973a57a0 20 read_log 194215'12344 (192888'12032) clean 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by osd.4.0:2177 2014-08-12 15:43:02.182758 2014-08-13 18:06:06.733293 7f5c973a57a0 20 read_log 194215'12345 (194215'12344) delete 1528a9bb/benchmark_data_storage0111-ib.pocstor_2228_object25516/head//24 by client.2174595.0:91344 0.000000 2014-08-13 18:06:06.733304 7f5c973a57a0 20 read_log 194215'12346 (192888'12029) clean a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by osd.4.0:2179 2014-08-12 15:43:02.191047 2014-08-13 18:06:06.733316 7f5c973a57a0 20 read_log 194215'12347 (194215'12346) delete a268a9bb/benchmark_data_storage0111-ib.pocstor_2228_object24829/head//24 by client.2174595.0:91346 0.000000 2014-08-13 18:06:06.733327 7f5c973a57a0 20 read_log 194215'12348 (193305'12128) clean 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by osd.4.0:2181 2014-08-12 15:43:02.199031 2014-08-13 18:06:06.733343 7f5c973a57a0 20 read_log 194215'12349 (194215'12348) delete 11fb9bb/benchmark_data_storage0111-ib.pocstor_62621_object20678/head//24 by client.2174595.0:91348 0.000000 2014-08-13 18:06:06.733355 7f5c973a57a0 20 read_log 194215'12350 (192888'12023) clean d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by osd.4.0:2183 2014-08-12 15:43:02.208199 2014-08-13 18:06:06.733368 7f5c973a57a0 20 read_log 194215'12351 (194215'12350) delete d934c9bb/benchmark_data_storage0111-ib.pocstor_2228_object23570/head//24 by client.2174595.0:91350 0.000000 2014-08-13 18:06:06.733395 7f5c973a57a0 20 read_log 194215'12352 (192888'11840) clean 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by osd.4.0:2185 2014-08-12 15:43:02.216071 2014-08-13 18:06:06.733411 7f5c973a57a0 20 read_log 194215'12353 (194215'12352) delete 8a35c9bb/benchmark_data_storage0111-ib.pocstor_41097_object19070/head//24 by client.2174595.0:91352 0.000000 2014-08-13 18:06:06.733431 7f5c973a57a0 20 read_log 194215'12354 (192888'12007) clean 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by osd.4.0:2187 2014-08-12 15:43:02.224138 2014-08-13 18:06:06.733446 7f5c973a57a0 20 read_log 194215'12355 (194215'12354) delete 3438c9bb/benchmark_data_storage0111-ib.pocstor_2228_object14102/head//24 by client.2174595.0:91354 0.000000 2014-08-13 18:06:06.733457 7f5c973a57a0 20 read_log 194215'12356 (192888'12027) clean a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by osd.4.0:2189 2014-08-12 15:43:02.231932 2014-08-13 18:06:06.733469 7f5c973a57a0 20 read_log 194215'12357 (194215'12356) delete a094d9bb/benchmark_data_storage0111-ib.pocstor_2228_object24427/head//24 by client.2174595.0:91356 0.000000 2014-08-13 18:06:06.733491 7f5c973a57a0 20 read_log 194215'12358 (192888'11995) clean 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by osd.4.0:2191 2014-08-12 15:43:02.240202 2014-08-13 18:06:06.733504 7f5c973a57a0 20 read_log 194215'12359 (194215'12358) delete 6307d9bb/benchmark_data_storage0111-ib.pocstor_2228_object9108/head//24 by client.2174595.0:91358 0.000000 2014-08-13 18:06:06.733515 7f5c973a57a0 20 read_log 194215'12360 (192834'11571) clean 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by osd.4.0:2193 2014-08-12 15:43:02.248134 2014-08-13 18:06:06.733528 7f5c973a57a0 20 read_log 194215'12361 (194215'12360) delete 2f0ed9bb/benchmark_data_storage0111-ib.pocstor_41526_object12802/head//24 by client.2174595.0:91360 0.000000 2014-08-13 18:06:06.733545 7f5c973a57a0 20 read_log 194215'12362 (192834'11023) clean 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by osd.4.0:2195 2014-08-12 15:43:02.257437 2014-08-13 18:06:06.733561 7f5c973a57a0 20 read_log 194215'12363 (194215'12362) delete 52b1e9bb/benchmark_data_storage0111-ib.pocstor_26659_object16032/head//24 by client.2174595.0:91362 0.000000 2014-08-13 18:06:06.733575 7f5c973a57a0 20 read_log 194215'12364 (192888'11847) clean b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by osd.4.0:2197 2014-08-12 15:43:02.266524 2014-08-13 18:06:06.733588 7f5c973a57a0 20 read_log 194215'12365 (194215'12364) delete b6ff9bb/benchmark_data_storage0111-ib.pocstor_41097_object24146/head//24 by client.2174595.0:91364 0.000000 2014-08-13 18:06:06.733598 7f5c973a57a0 20 read_log 194215'12366 (192888'12034) clean 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by osd.4.0:2199 2014-08-12 15:43:02.276059 2014-08-13 18:06:06.733611 7f5c973a57a0 20 read_log 194215'12367 (194215'12366) delete 31731dbb/benchmark_data_storage0111-ib.pocstor_2228_object26117/head//24 by client.2174595.0:91366 0.000000 2014-08-13 18:06:06.733621 7f5c973a57a0 20 read_log 194215'12368 (192834'11540) clean 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by osd.4.0:2201 2014-08-12 15:43:02.284961 2014-08-13 18:06:06.733641 7f5c973a57a0 20 read_log 194215'12369 (194215'12368) delete 24482dbb/benchmark_data_storage0111-ib.pocstor_41526_object9052/head//24 by client.2174595.0:91368 0.000000 2014-08-13 18:06:06.733655 7f5c973a57a0 20 read_log 194215'12370 (192888'12004) clean 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by osd.4.0:2203 2014-08-12 15:43:02.294104 2014-08-13 18:06:06.733668 7f5c973a57a0 20 read_log 194215'12371 (194215'12370) delete 19e83dbb/benchmark_data_storage0111-ib.pocstor_2228_object12479/head//24 by client.2174595.0:91370 0.000000 2014-08-13 18:06:06.733679 7f5c973a57a0 20 read_log 194215'12372 (192888'11827) clean 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by osd.4.0:2205 2014-08-12 15:43:02.301929 2014-08-13 18:06:06.733691 7f5c973a57a0 20 read_log 194215'12373 (194215'12372) delete 471c4dbb/benchmark_data_storage0111-ib.pocstor_41097_object15380/head//24 by client.2174595.0:91372 0.000000 2014-08-13 18:06:06.733702 7f5c973a57a0 20 read_log 194215'12374 (192888'11986) clean 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by osd.4.0:2207 2014-08-12 15:43:02.310145 2014-08-13 18:06:06.733714 7f5c973a57a0 20 read_log 194215'12375 (194215'12374) delete 71205dbb/benchmark_data_storage0111-ib.pocstor_2228_object2594/head//24 by client.2174595.0:91374 0.000000 2014-08-13 18:06:06.733725 7f5c973a57a0 20 read_log 194215'12376 (192888'11844) clean 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by osd.4.0:2209 2014-08-12 15:43:02.318327 2014-08-13 18:06:06.733737 7f5c973a57a0 20 read_log 194215'12377 (194215'12376) delete 88055dbb/benchmark_data_storage0111-ib.pocstor_41097_object22613/head//24 by client.2174595.0:91376 0.000000 2014-08-13 18:06:06.733748 7f5c973a57a0 20 read_log 194215'12378 (192888'11831) clean b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by osd.4.0:2211 2014-08-12 15:43:02.325768 2014-08-13 18:06:06.733760 7f5c973a57a0 20 read_log 194215'12379 (194215'12378) delete b9095dbb/benchmark_data_storage0111-ib.pocstor_41097_object16308/head//24 by client.2174595.0:91378 0.000000 2014-08-13 18:06:06.733771 7f5c973a57a0 20 read_log 194215'12380 (193305'12130) clean 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by osd.4.0:2213 2014-08-12 15:43:02.335031 2014-08-13 18:06:06.733783 7f5c973a57a0 20 read_log 194215'12381 (194215'12380) delete 1b2c7dbb/benchmark_data_storage0111-ib.pocstor_62621_object26840/head//24 by client.2174595.0:91380 0.000000 2014-08-13 18:06:06.733821 7f5c973a57a0 20 read_log 194215'12382 (192834'10971) clean e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by osd.4.0:2215 2014-08-12 15:43:02.343191 2014-08-13 18:06:06.733837 7f5c973a57a0 20 read_log 194215'12383 (194215'12382) delete e3eb8dbb/benchmark_data_storage0111-ib.pocstor_26659_object9418/head//24 by client.2174595.0:91382 0.000000 2014-08-13 18:06:06.733848 7f5c973a57a0 20 read_log 194215'12384 (192834'11602) clean ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by osd.4.0:2217 2014-08-12 15:43:02.350462 2014-08-13 18:06:06.733861 7f5c973a57a0 20 read_log 194215'12385 (194215'12384) delete ae219dbb/benchmark_data_storage0111-ib.pocstor_41526_object16335/head//24 by client.2174595.0:91384 0.000000 2014-08-13 18:06:06.733872 7f5c973a57a0 20 read_log 194215'12386 (192888'11821) clean cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by osd.4.0:2219 2014-08-12 15:43:02.357831 2014-08-13 18:06:06.733884 7f5c973a57a0 20 read_log 194215'12387 (194215'12386) delete cd49dbb/benchmark_data_storage0111-ib.pocstor_41097_object13148/head//24 by client.2174595.0:91386 0.000000 2014-08-13 18:06:06.733897 7f5c973a57a0 20 read_log 194215'12388 (192888'11813) clean f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by osd.4.0:2221 2014-08-12 15:43:02.366070 2014-08-13 18:06:06.733910 7f5c973a57a0 20 read_log 194215'12389 (194215'12388) delete f82e9dbb/benchmark_data_storage0111-ib.pocstor_41097_object8401/head//24 by client.2174595.0:91388 0.000000 2014-08-13 18:06:06.733922 7f5c973a57a0 20 read_log 194215'12390 (192834'10926) clean ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by osd.4.0:2223 2014-08-12 15:43:02.373453 2014-08-13 18:06:06.733935 7f5c973a57a0 20 read_log 194215'12391 (194215'12390) delete ef1bdbb/benchmark_data_storage0111-ib.pocstor_26659_object4877/head//24 by client.2174595.0:91390 0.000000 2014-08-13 18:06:06.733948 7f5c973a57a0 20 read_log 194215'12392 (192834'11614) clean f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by osd.4.0:2225 2014-08-12 15:43:02.382388 2014-08-13 18:06:06.733960 7f5c973a57a0 20 read_log 194215'12393 (194215'12392) delete f279bdbb/benchmark_data_storage0111-ib.pocstor_41526_object17581/head//24 by client.2174595.0:91392 0.000000 2014-08-13 18:06:06.733973 7f5c973a57a0 20 read_log 194215'12394 (192834'11621) clean e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by osd.4.0:2227 2014-08-12 15:43:02.391049 2014-08-13 18:06:06.733985 7f5c973a57a0 20 read_log 194215'12395 (194215'12394) delete e76bbdbb/benchmark_data_storage0111-ib.pocstor_41526_object18487/head//24 by client.2174595.0:91394 0.000000 2014-08-13 18:06:06.733996 7f5c973a57a0 20 read_log 194215'12396 (192834'11473) clean a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by osd.4.0:2229 2014-08-12 15:43:02.399533 2014-08-13 18:06:06.734008 7f5c973a57a0 20 read_log 194215'12397 (194215'12396) delete a3fbbdbb/benchmark_data_storage0111-ib.pocstor_41526_object1174/head//24 by client.2174595.0:91396 0.000000 2014-08-13 18:06:06.734040 7f5c973a57a0 20 read_log 194215'12398 (193305'12110) clean 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by osd.4.0:2231 2014-08-12 15:43:02.407094 2014-08-13 18:06:06.734053 7f5c973a57a0 20 read_log 194215'12399 (194215'12398) delete 709cbdbb/benchmark_data_storage0111-ib.pocstor_62621_object2128/head//24 by client.2174595.0:91398 0.000000 2014-08-13 18:06:06.734064 7f5c973a57a0 20 read_log 194215'12400 (192888'11804) clean 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by osd.4.0:2233 2014-08-12 15:43:02.430858 2014-08-13 18:06:06.734078 7f5c973a57a0 20 read_log 194215'12401 (194215'12400) delete 6eb7cdbb/benchmark_data_storage0111-ib.pocstor_41097_object3035/head//24 by client.2174595.0:91400 0.000000 2014-08-13 18:06:06.734097 7f5c973a57a0 20 read_log 194215'12402 (193305'12121) clean c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by osd.4.0:2235 2014-08-12 15:43:02.438996 2014-08-13 18:06:06.734112 7f5c973a57a0 20 read_log 194215'12403 (194215'12402) delete c397edbb/benchmark_data_storage0111-ib.pocstor_62621_object16226/head//24 by client.2174595.0:91402 0.000000 2014-08-13 18:06:06.734123 7f5c973a57a0 20 read_log 194215'12404 (193305'12135) clean eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by osd.4.0:2237 2014-08-12 15:43:02.447111 2014-08-13 18:06:06.734138 7f5c973a57a0 20 read_log 194215'12405 (194215'12404) delete eaa8edbb/benchmark_data_storage0111-ib.pocstor_62621_object29931/head//24 by client.2174595.0:91404 0.000000 2014-08-13 18:06:06.734162 7f5c973a57a0 20 read_log 194215'12406 (192834'11491) clean 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by osd.4.0:2239 2014-08-12 15:43:02.455272 2014-08-13 18:06:06.734175 7f5c973a57a0 20 read_log 194215'12407 (194215'12406) delete 2eb0fdbb/benchmark_data_storage0111-ib.pocstor_41526_object4720/head//24 by client.2174595.0:91406 0.000000 2014-08-13 18:06:06.734186 7f5c973a57a0 20 read_log 194215'12408 (192888'12040) clean d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by osd.4.0:2241 2014-08-12 15:43:02.464606 2014-08-13 18:06:06.734198 7f5c973a57a0 20 read_log 194215'12409 (194215'12408) delete d201fdbb/benchmark_data_storage0111-ib.pocstor_2228_object28446/head//24 by client.2174595.0:91408 0.000000 2014-08-13 18:06:06.734209 7f5c973a57a0 20 read_log 194215'12410 (192888'12039) clean 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by osd.4.0:2243 2014-08-12 15:43:02.473186 2014-08-13 18:06:06.734224 7f5c973a57a0 20 read_log 194215'12411 (194215'12410) delete 4924fdbb/benchmark_data_storage0111-ib.pocstor_2228_object27649/head//24 by client.2174595.0:91410 0.000000 2014-08-13 18:06:06.734237 7f5c973a57a0 20 read_log 194215'12412 (192888'11858) clean 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by osd.4.0:2245 2014-08-12 15:43:02.481162 2014-08-13 18:06:06.734252 7f5c973a57a0 20 read_log 194215'12413 (194215'12412) delete 2d3afdbb/benchmark_data_storage0111-ib.pocstor_41097_object29775/head//24 by client.2174595.0:91412 0.000000 2014-08-13 18:06:06.734263 7f5c973a57a0 20 read_log 194215'12414 (194215'12414) modify 1bb/hit_set_24.1bb_archive_2014-08-12 15:43:01.048059_2014-08-12 17:24:40.865594/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.866303 2014-08-13 18:06:06.734276 7f5c973a57a0 20 read_log 194215'12415 (194215'12139) delete 1bb/hit_set_24.1bb_archive_2014-08-12 15:32:25.863957_2014-08-12 15:43:01.047258/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.866303 2014-08-13 18:06:06.734291 7f5c973a57a0 20 read_log 194215'12416 (0'0) promote 323fcdbb/benchmark_data_storage0111-ib.pocstor_34628_object247/head//24 by osd.4.0:5232 2014-08-12 17:24:40.871311 2014-08-13 18:06:06.734304 7f5c973a57a0 20 read_log 194215'12417 (194215'12416) modify 323fcdbb/benchmark_data_storage0111-ib.pocstor_34628_object247/head//24 by client.2193759.0:248 2014-08-12 17:24:40.858439 2014-08-13 18:06:06.734316 7f5c973a57a0 20 read_log 194217'12418 (0'0) promote be386dbb/benchmark_data_storage0111-ib.pocstor_39693_object71/head//24 by osd.4.0:5256 2014-08-12 17:25:03.578296 2014-08-13 18:06:06.734329 7f5c973a57a0 20 read_log 194217'12419 (194217'12418) modify be386dbb/benchmark_data_storage0111-ib.pocstor_39693_object71/head//24 by client.1487927.0:72 2014-08-12 17:25:03.562697 2014-08-13 18:06:06.734347 7f5c973a57a0 20 read_log 194370'12420 (0'0) promote 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by osd.332.0:3733 2014-08-12 20:30:17.571032 2014-08-13 18:06:06.734367 7f5c973a57a0 20 read_log 194370'12421 (194370'12420) modify 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by client.1508531.0:846 2014-08-12 20:30:17.563335 2014-08-13 18:06:06.734397 7f5c973a57a0 20 read_log 194370'12422 (0'0) promote fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by osd.332.0:3735 2014-08-12 20:30:17.738491 2014-08-13 18:06:06.734410 7f5c973a57a0 20 read_log 194370'12423 (194370'12422) modify fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by client.1508531.0:893 2014-08-12 20:30:17.732009 2014-08-13 18:06:06.734432 7f5c973a57a0 20 read_log 194370'12424 (0'0) promote 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by osd.332.0:3739 2014-08-12 20:30:18.221906 2014-08-13 18:06:06.734448 7f5c973a57a0 20 read_log 194370'12425 (194370'12424) modify 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by client.1508531.0:1027 2014-08-12 20:30:18.215574 2014-08-13 18:06:06.734461 7f5c973a57a0 20 read_log 194370'12426 (0'0) promote 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by osd.332.0:3743 2014-08-12 20:30:18.815967 2014-08-13 18:06:06.734499 7f5c973a57a0 20 read_log 194370'12427 (194370'12426) modify 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by client.1508531.0:1182 2014-08-12 20:30:18.808413 2014-08-13 18:06:06.734515 7f5c973a57a0 20 read_log 194370'12428 (0'0) promote 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by osd.332.0:3781 2014-08-12 20:30:24.474696 2014-08-13 18:06:06.734527 7f5c973a57a0 20 read_log 194370'12429 (194370'12428) modify 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by client.1508531.0:2705 2014-08-12 20:30:24.467345 2014-08-13 18:06:06.734540 7f5c973a57a0 20 read_log 194370'12430 (0'0) promote aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by osd.332.0:3809 2014-08-12 20:30:27.725673 2014-08-13 18:06:06.734552 7f5c973a57a0 20 read_log 194370'12431 (194370'12430) modify aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by client.1508531.0:3585 2014-08-12 20:30:27.716306 2014-08-13 18:06:06.734568 7f5c973a57a0 20 read_log 194370'12432 (0'0) promote 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by osd.332.0:3879 2014-08-12 20:30:38.978130 2014-08-13 18:06:06.734581 7f5c973a57a0 20 read_log 194370'12433 (194370'12432) modify 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by client.1508531.0:6282 2014-08-12 20:30:38.971071 2014-08-13 18:06:06.734593 7f5c973a57a0 20 read_log 194370'12434 (0'0) promote 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by osd.332.0:3889 2014-08-12 20:30:42.847116 2014-08-13 18:06:06.734613 7f5c973a57a0 20 read_log 194370'12435 (194370'12434) modify 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by client.1508531.0:7087 2014-08-12 20:30:42.838696 2014-08-13 18:06:06.734626 7f5c973a57a0 20 read_log 194370'12436 (0'0) promote 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by osd.332.0:3899 2014-08-12 20:30:44.638187 2014-08-13 18:06:06.734642 7f5c973a57a0 20 read_log 194370'12437 (194370'12436) modify 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by client.1508531.0:7455 2014-08-12 20:30:44.631213 2014-08-13 18:06:06.734654 7f5c973a57a0 20 read_log 194370'12438 (0'0) promote f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by osd.332.0:3913 2014-08-12 20:30:49.212601 2014-08-13 18:06:06.734667 7f5c973a57a0 20 read_log 194370'12439 (194370'12438) modify f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by client.1508531.0:8308 2014-08-12 20:30:49.206145 2014-08-13 18:06:06.734679 7f5c973a57a0 20 read_log 194370'12440 (0'0) promote 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by osd.332.0:3925 2014-08-12 20:30:52.140583 2014-08-13 18:06:06.734699 7f5c973a57a0 20 read_log 194370'12441 (194370'12440) modify 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by client.1508531.0:8814 2014-08-12 20:30:52.131128 2014-08-13 18:06:06.734715 7f5c973a57a0 20 read_log 194370'12442 (0'0) promote 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by osd.332.0:3965 2014-08-12 20:31:01.991803 2014-08-13 18:06:06.734727 7f5c973a57a0 20 read_log 194370'12443 (194370'12442) modify 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by client.1508531.0:10461 2014-08-12 20:31:01.983217 2014-08-13 18:06:06.734740 7f5c973a57a0 20 read_log 194370'12444 (0'0) promote eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by osd.332.0:3969 2014-08-12 20:31:02.883903 2014-08-13 18:06:06.734752 7f5c973a57a0 20 read_log 194370'12445 (194370'12444) modify eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by client.1508531.0:10604 2014-08-12 20:31:02.874965 2014-08-13 18:06:06.734781 7f5c973a57a0 20 read_log 194370'12446 (0'0) promote 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by osd.332.0:3989 2014-08-12 20:31:11.747534 2014-08-13 18:06:06.734798 7f5c973a57a0 20 read_log 194370'12447 (194370'12446) modify 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by client.1508531.0:12053 2014-08-12 20:31:11.739138 2014-08-13 18:06:06.734810 7f5c973a57a0 20 read_log 194370'12448 (0'0) promote 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by osd.332.0:3999 2014-08-12 20:31:14.764086 2014-08-13 18:06:06.734822 7f5c973a57a0 20 read_log 194370'12449 (194370'12448) modify 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by client.1508531.0:12467 2014-08-12 20:31:14.755486 2014-08-13 18:06:06.734835 7f5c973a57a0 20 read_log 194370'12450 (0'0) promote 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by osd.332.0:4011 2014-08-12 20:31:20.422594 2014-08-13 18:06:06.734847 7f5c973a57a0 20 read_log 194370'12451 (194370'12450) modify 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by client.1508531.0:13107 2014-08-12 20:31:20.413293 2014-08-13 18:06:06.734859 7f5c973a57a0 20 read_log 194377'12452 (0'0) promote efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by osd.332.0:4049 2014-08-12 20:33:00.376451 2014-08-13 18:06:06.734872 7f5c973a57a0 20 read_log 194377'12453 (194377'12452) modify efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by client.2231190.0:751 2014-08-12 20:33:00.367237 2014-08-13 18:06:06.734884 7f5c973a57a0 20 read_log 194377'12454 (0'0) promote ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by osd.332.0:4059 2014-08-12 20:33:02.566410 2014-08-13 18:06:06.734899 7f5c973a57a0 20 read_log 194377'12455 (194377'12454) modify ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by client.2231190.0:1331 2014-08-12 20:33:02.557160 2014-08-13 18:06:06.734911 7f5c973a57a0 20 read_log 194377'12456 (0'0) promote b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by osd.332.0:4071 2014-08-12 20:33:06.779043 2014-08-13 18:06:06.734924 7f5c973a57a0 20 read_log 194377'12457 (194377'12456) modify b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by client.2231190.0:2386 2014-08-12 20:33:06.773113 2014-08-13 18:06:06.734936 7f5c973a57a0 20 read_log 194377'12458 (0'0) promote c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by osd.332.0:4085 2014-08-12 20:33:09.554340 2014-08-13 18:06:06.734951 7f5c973a57a0 20 read_log 194377'12459 (194377'12458) modify c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by client.2231190.0:3058 2014-08-12 20:33:09.547018 2014-08-13 18:06:06.734963 7f5c973a57a0 20 read_log 194377'12460 (0'0) promote ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by osd.332.0:4097 2014-08-12 20:33:11.455609 2014-08-13 18:06:06.734983 7f5c973a57a0 20 read_log 194377'12461 (194377'12460) modify ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by client.2231190.0:3520 2014-08-12 20:33:11.447247 2014-08-13 18:06:06.734998 7f5c973a57a0 20 read_log 194397'12462 (194397'12462) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:37:08.016110_2014-08-12 20:53:01.910165/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:01.910318 2014-08-13 18:06:06.735011 7f5c973a57a0 20 read_log 194397'12463 (194215'12414) delete 1bb/hit_set_24.1bb_archive_2014-08-12 15:43:01.048059_2014-08-12 17:24:40.865594/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:01.910318 2014-08-13 18:06:06.735025 7f5c973a57a0 20 read_log 194397'12464 (0'0) promote b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by osd.332.0:4247 2014-08-12 20:53:01.913586 2014-08-13 18:06:06.735038 7f5c973a57a0 20 read_log 194397'12465 (194397'12464) modify b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by client.2235069.0:565 2014-08-12 20:53:01.904209 2014-08-13 18:06:06.735052 7f5c973a57a0 20 read_log 194416'12466 (194416'12466) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 18:06:06.735065 7f5c973a57a0 20 read_log 194416'12467 (194397'12462) delete 1bb/hit_set_24.1bb_archive_2014-08-12 20:37:08.016110_2014-08-12 20:53:01.910165/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 18:06:06.735079 7f5c973a57a0 20 read_log 194416'12468 (0'0) promote 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by osd.332.0:4385 2014-08-13 15:28:36.887696 2014-08-13 18:06:06.735091 7f5c973a57a0 20 read_log 194416'12469 (194416'12468) modify 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by client.2441958.0:1627 2014-08-13 15:28:36.876856 2014-08-13 18:06:06.735106 7f5c973a57a0 20 read_log 194416'12470 (0'0) promote 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by osd.332.0:4409 2014-08-13 15:28:41.435570 2014-08-13 18:06:06.735118 7f5c973a57a0 20 read_log 194416'12471 (194416'12470) modify 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by client.2441958.0:2829 2014-08-13 15:28:41.426800 2014-08-13 18:06:06.735132 7f5c973a57a0 20 read_log 194416'12472 (0'0) promote 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by osd.332.0:4419 2014-08-13 15:28:42.410760 2014-08-13 18:06:06.735144 7f5c973a57a0 20 read_log 194416'12473 (194416'12472) modify 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by client.2441958.0:3088 2014-08-13 15:28:42.401137 2014-08-13 18:06:06.735162 7f5c973a57a0 20 read_log 194416'12474 (0'0) promote 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by osd.332.0:4423 2014-08-13 15:28:42.509249 2014-08-13 18:06:06.735175 7f5c973a57a0 20 read_log 194416'12475 (194416'12474) modify 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by client.2441958.0:3115 2014-08-13 15:28:42.499330 2014-08-13 18:06:06.735187 7f5c973a57a0 20 read_log 194416'12476 (0'0) promote 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by osd.332.0:4515 2014-08-13 15:28:58.599923 2014-08-13 18:06:06.735199 7f5c973a57a0 20 read_log 194416'12477 (194416'12476) modify 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by client.2441958.0:7219 2014-08-13 15:28:58.589366 2014-08-13 18:06:06.735212 7f5c973a57a0 20 read_log 194416'12478 (0'0) promote d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by osd.332.0:4545 2014-08-13 15:29:03.283742 2014-08-13 18:06:06.735226 7f5c973a57a0 20 read_log 194416'12479 (194416'12478) modify d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by client.2441958.0:8464 2014-08-13 15:29:03.275090 2014-08-13 18:06:06.735249 7f5c973a57a0 20 read_log 194416'12480 (0'0) promote 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by osd.332.0:4561 2014-08-13 15:29:04.641188 2014-08-13 18:06:06.735264 7f5c973a57a0 20 read_log 194416'12481 (194416'12480) modify 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by client.2441958.0:8807 2014-08-13 15:29:04.626173 2014-08-13 18:06:06.735276 7f5c973a57a0 20 read_log 194416'12482 (0'0) promote 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by osd.332.0:4597 2014-08-13 15:29:14.568670 2014-08-13 18:06:06.735289 7f5c973a57a0 20 read_log 194416'12483 (194416'12482) modify 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by client.2441958.0:11269 2014-08-13 15:29:14.556475 2014-08-13 18:06:06.735304 7f5c973a57a0 20 read_log 194416'12484 (0'0) promote f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by osd.332.0:4625 2014-08-13 15:29:19.200191 2014-08-13 18:06:06.735317 7f5c973a57a0 20 read_log 194416'12485 (194416'12484) modify f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by client.2441958.0:12370 2014-08-13 15:29:19.191812 2014-08-13 18:06:06.735330 7f5c973a57a0 20 read_log 194416'12486 (0'0) promote 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by osd.332.0:4631 2014-08-13 15:29:21.418124 2014-08-13 18:06:06.735342 7f5c973a57a0 20 read_log 194416'12487 (194416'12486) modify 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by client.2441958.0:12895 2014-08-13 15:29:21.410797 2014-08-13 18:06:06.735358 7f5c973a57a0 20 read_log 194416'12488 (0'0) promote bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by osd.332.0:4643 2014-08-13 15:29:23.128925 2014-08-13 18:06:06.735371 7f5c973a57a0 20 read_log 194416'12489 (194416'12488) modify bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by client.2441958.0:13269 2014-08-13 15:29:23.120531 2014-08-13 18:06:06.735392 7f5c973a57a0 20 read_log 194416'12490 (0'0) promote 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by osd.332.0:4673 2014-08-13 15:29:27.741021 2014-08-13 18:06:06.735409 7f5c973a57a0 20 read_log 194416'12491 (194416'12490) modify 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by client.2441958.0:14397 2014-08-13 15:29:27.730476 2014-08-13 18:06:06.735424 7f5c973a57a0 20 read_log 194416'12492 (0'0) promote 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by osd.332.0:4679 2014-08-13 15:29:29.604493 2014-08-13 18:06:06.735449 7f5c973a57a0 20 read_log 194416'12493 (194416'12492) modify 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by client.2441958.0:14844 2014-08-13 15:29:29.596571 2014-08-13 18:06:06.735462 7f5c973a57a0 20 read_log 194416'12494 (0'0) promote 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by osd.332.0:4691 2014-08-13 15:29:32.421036 2014-08-13 18:06:06.735474 7f5c973a57a0 20 read_log 194416'12495 (194416'12494) modify 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by client.2441958.0:15445 2014-08-13 15:29:32.411875 2014-08-13 18:06:06.735489 7f5c973a57a0 20 read_log 194416'12496 (0'0) promote cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by osd.332.0:4699 2014-08-13 15:29:33.263855 2014-08-13 18:06:06.735505 7f5c973a57a0 20 read_log 194416'12497 (194416'12496) modify cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by client.2441958.0:15605 2014-08-13 15:29:33.255247 2014-08-13 18:06:06.735518 7f5c973a57a0 20 read_log 194416'12498 (0'0) promote c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by osd.332.0:4749 2014-08-13 15:30:00.202521 2014-08-13 18:06:06.735538 7f5c973a57a0 20 read_log 194416'12499 (194416'12498) modify c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by client.2441958.0:17910 2014-08-13 15:30:00.193255 2014-08-13 18:06:06.735554 7f5c973a57a0 20 read_log 194422'12500 (0'0) promote 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by osd.332.0:4791 2014-08-13 15:32:19.666593 2014-08-13 18:06:06.735566 7f5c973a57a0 20 read_log 194422'12501 (194422'12500) modify 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by client.1148203.0:1098 2014-08-13 15:32:19.658149 2014-08-13 18:06:06.735579 7f5c973a57a0 20 read_log 194422'12502 (0'0) promote adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by osd.332.0:4799 2014-08-13 15:32:21.421131 2014-08-13 18:06:06.735598 7f5c973a57a0 20 read_log 194422'12503 (194422'12502) modify adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by client.1148203.0:1548 2014-08-13 15:32:21.413428 2014-08-13 18:06:06.735611 7f5c973a57a0 20 read_log 194422'12504 (0'0) promote 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by osd.332.0:4855 2014-08-13 15:32:33.463926 2014-08-13 18:06:06.735623 7f5c973a57a0 20 read_log 194422'12505 (194422'12504) modify 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by client.1148203.0:4640 2014-08-13 15:32:33.455244 2014-08-13 18:06:06.735646 7f5c973a57a0 20 read_log 194422'12506 (0'0) promote edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by osd.332.0:4871 2014-08-13 15:32:37.126742 2014-08-13 18:06:06.735659 7f5c973a57a0 20 read_log 194422'12507 (194422'12506) modify edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by client.1148203.0:5457 2014-08-13 15:32:37.118627 2014-08-13 18:06:06.735672 7f5c973a57a0 20 read_log 194422'12508 (0'0) promote 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by osd.332.0:4889 2014-08-13 15:32:41.239656 2014-08-13 18:06:06.735684 7f5c973a57a0 20 read_log 194422'12509 (194422'12508) modify 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by client.1148203.0:6204 2014-08-13 15:32:41.230821 2014-08-13 18:06:06.735696 7f5c973a57a0 20 read_log 194422'12510 (0'0) promote 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by osd.332.0:4893 2014-08-13 15:32:42.035695 2014-08-13 18:06:06.735709 7f5c973a57a0 20 read_log 194422'12511 (194422'12510) modify 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by client.1148203.0:6344 2014-08-13 15:32:42.025183 2014-08-13 18:06:06.735723 7f5c973a57a0 20 read_log 194422'12512 (0'0) promote f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by osd.332.0:4905 2014-08-13 15:32:46.041594 2014-08-13 18:06:06.735735 7f5c973a57a0 20 read_log 194422'12513 (194422'12512) modify f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by client.1148203.0:7082 2014-08-13 15:32:46.034022 2014-08-13 18:06:06.735747 7f5c973a57a0 20 read_log 194422'12514 (0'0) promote 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by osd.332.0:4961 2014-08-13 15:33:09.893219 2014-08-13 18:06:06.735759 7f5c973a57a0 20 read_log 194422'12515 (194422'12514) modify 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by client.1148203.0:9945 2014-08-13 15:33:09.884120 2014-08-13 18:06:06.735772 7f5c973a57a0 20 read_log 194422'12516 (0'0) promote a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by osd.332.0:4965 2014-08-13 15:33:22.152590 2014-08-13 18:06:06.735784 7f5c973a57a0 20 read_log 194422'12517 (194422'12516) modify a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by client.1148203.0:10435 2014-08-13 15:33:22.144771 2014-08-13 18:06:06.735801 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.736287 7f5c973a57a0 10 read_log checking for missing items over interval (194370'12420,194422'12517] 2014-08-13 18:06:06.736336 7f5c973a57a0 15 read_log missing 194422'12517 (194422'12516) modify a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 by client.1148203.0:10435 2014-08-13 15:33:22.144771 2014-08-13 18:06:06.736376 7f5c973a57a0 15 read_log missing 194422'12515 (194422'12514) modify 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 by client.1148203.0:9945 2014-08-13 15:33:09.884120 2014-08-13 18:06:06.736413 7f5c973a57a0 15 read_log missing 194422'12513 (194422'12512) modify f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 by client.1148203.0:7082 2014-08-13 15:32:46.034022 2014-08-13 18:06:06.736449 7f5c973a57a0 15 read_log missing 194422'12511 (194422'12510) modify 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 by client.1148203.0:6344 2014-08-13 15:32:42.025183 2014-08-13 18:06:06.736485 7f5c973a57a0 15 read_log missing 194422'12509 (194422'12508) modify 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 by client.1148203.0:6204 2014-08-13 15:32:41.230821 2014-08-13 18:06:06.736520 7f5c973a57a0 15 read_log missing 194422'12507 (194422'12506) modify edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 by client.1148203.0:5457 2014-08-13 15:32:37.118627 2014-08-13 18:06:06.736556 7f5c973a57a0 15 read_log missing 194422'12505 (194422'12504) modify 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 by client.1148203.0:4640 2014-08-13 15:32:33.455244 2014-08-13 18:06:06.736591 7f5c973a57a0 15 read_log missing 194422'12503 (194422'12502) modify adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 by client.1148203.0:1548 2014-08-13 15:32:21.413428 2014-08-13 18:06:06.736627 7f5c973a57a0 15 read_log missing 194422'12501 (194422'12500) modify 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 by client.1148203.0:1098 2014-08-13 15:32:19.658149 2014-08-13 18:06:06.736662 7f5c973a57a0 15 read_log missing 194416'12499 (194416'12498) modify c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 by client.2441958.0:17910 2014-08-13 15:30:00.193255 2014-08-13 18:06:06.736698 7f5c973a57a0 15 read_log missing 194416'12497 (194416'12496) modify cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 by client.2441958.0:15605 2014-08-13 15:29:33.255247 2014-08-13 18:06:06.736733 7f5c973a57a0 15 read_log missing 194416'12495 (194416'12494) modify 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 by client.2441958.0:15445 2014-08-13 15:29:32.411875 2014-08-13 18:06:06.736768 7f5c973a57a0 15 read_log missing 194416'12493 (194416'12492) modify 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 by client.2441958.0:14844 2014-08-13 15:29:29.596571 2014-08-13 18:06:06.736804 7f5c973a57a0 15 read_log missing 194416'12491 (194416'12490) modify 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 by client.2441958.0:14397 2014-08-13 15:29:27.730476 2014-08-13 18:06:06.736840 7f5c973a57a0 15 read_log missing 194416'12489 (194416'12488) modify bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 by client.2441958.0:13269 2014-08-13 15:29:23.120531 2014-08-13 18:06:06.736876 7f5c973a57a0 15 read_log missing 194416'12487 (194416'12486) modify 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 by client.2441958.0:12895 2014-08-13 15:29:21.410797 2014-08-13 18:06:06.736911 7f5c973a57a0 15 read_log missing 194416'12485 (194416'12484) modify f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 by client.2441958.0:12370 2014-08-13 15:29:19.191812 2014-08-13 18:06:06.736946 7f5c973a57a0 15 read_log missing 194416'12483 (194416'12482) modify 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 by client.2441958.0:11269 2014-08-13 15:29:14.556475 2014-08-13 18:06:06.736982 7f5c973a57a0 15 read_log missing 194416'12481 (194416'12480) modify 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 by client.2441958.0:8807 2014-08-13 15:29:04.626173 2014-08-13 18:06:06.737017 7f5c973a57a0 15 read_log missing 194416'12479 (194416'12478) modify d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 by client.2441958.0:8464 2014-08-13 15:29:03.275090 2014-08-13 18:06:06.737053 7f5c973a57a0 15 read_log missing 194416'12477 (194416'12476) modify 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 by client.2441958.0:7219 2014-08-13 15:28:58.589366 2014-08-13 18:06:06.737089 7f5c973a57a0 15 read_log missing 194416'12475 (194416'12474) modify 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 by client.2441958.0:3115 2014-08-13 15:28:42.499330 2014-08-13 18:06:06.737124 7f5c973a57a0 15 read_log missing 194416'12473 (194416'12472) modify 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 by client.2441958.0:3088 2014-08-13 15:28:42.401137 2014-08-13 18:06:06.737176 7f5c973a57a0 15 read_log missing 194416'12471 (194416'12470) modify 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 by client.2441958.0:2829 2014-08-13 15:28:41.426800 2014-08-13 18:06:06.737215 7f5c973a57a0 15 read_log missing 194416'12469 (194416'12468) modify 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 by client.2441958.0:1627 2014-08-13 15:28:36.876856 2014-08-13 18:06:06.737269 7f5c973a57a0 15 read_log missing 194416'12466 (194416'12466) modify 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:36.883498 2014-08-13 18:06:06.737310 7f5c973a57a0 15 read_log missing 194397'12465 (194397'12464) modify b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 by client.2235069.0:565 2014-08-12 20:53:01.904209 2014-08-13 18:06:06.737350 7f5c973a57a0 15 read_log missing 194377'12461 (194377'12460) modify ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 by client.2231190.0:3520 2014-08-12 20:33:11.447247 2014-08-13 18:06:06.737400 7f5c973a57a0 15 read_log missing 194377'12459 (194377'12458) modify c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 by client.2231190.0:3058 2014-08-12 20:33:09.547018 2014-08-13 18:06:06.737447 7f5c973a57a0 15 read_log missing 194377'12457 (194377'12456) modify b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 by client.2231190.0:2386 2014-08-12 20:33:06.773113 2014-08-13 18:06:06.737484 7f5c973a57a0 15 read_log missing 194377'12455 (194377'12454) modify ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 by client.2231190.0:1331 2014-08-12 20:33:02.557160 2014-08-13 18:06:06.737520 7f5c973a57a0 15 read_log missing 194377'12453 (194377'12452) modify efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 by client.2231190.0:751 2014-08-12 20:33:00.367237 2014-08-13 18:06:06.737556 7f5c973a57a0 15 read_log missing 194370'12451 (194370'12450) modify 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 by client.1508531.0:13107 2014-08-12 20:31:20.413293 2014-08-13 18:06:06.737591 7f5c973a57a0 15 read_log missing 194370'12449 (194370'12448) modify 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 by client.1508531.0:12467 2014-08-12 20:31:14.755486 2014-08-13 18:06:06.737627 7f5c973a57a0 15 read_log missing 194370'12447 (194370'12446) modify 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 by client.1508531.0:12053 2014-08-12 20:31:11.739138 2014-08-13 18:06:06.737669 7f5c973a57a0 15 read_log missing 194370'12445 (194370'12444) modify eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 by client.1508531.0:10604 2014-08-12 20:31:02.874965 2014-08-13 18:06:06.737706 7f5c973a57a0 15 read_log missing 194370'12443 (194370'12442) modify 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 by client.1508531.0:10461 2014-08-12 20:31:01.983217 2014-08-13 18:06:06.737764 7f5c973a57a0 15 read_log missing 194370'12441 (194370'12440) modify 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 by client.1508531.0:8814 2014-08-12 20:30:52.131128 2014-08-13 18:06:06.737800 7f5c973a57a0 15 read_log missing 194370'12439 (194370'12438) modify f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 by client.1508531.0:8308 2014-08-12 20:30:49.206145 2014-08-13 18:06:06.737836 7f5c973a57a0 15 read_log missing 194370'12437 (194370'12436) modify 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 by client.1508531.0:7455 2014-08-12 20:30:44.631213 2014-08-13 18:06:06.737872 7f5c973a57a0 15 read_log missing 194370'12435 (194370'12434) modify 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 by client.1508531.0:7087 2014-08-12 20:30:42.838696 2014-08-13 18:06:06.737907 7f5c973a57a0 15 read_log missing 194370'12433 (194370'12432) modify 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 by client.1508531.0:6282 2014-08-12 20:30:38.971071 2014-08-13 18:06:06.737943 7f5c973a57a0 15 read_log missing 194370'12431 (194370'12430) modify aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 by client.1508531.0:3585 2014-08-12 20:30:27.716306 2014-08-13 18:06:06.737978 7f5c973a57a0 15 read_log missing 194370'12429 (194370'12428) modify 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 by client.1508531.0:2705 2014-08-12 20:30:24.467345 2014-08-13 18:06:06.738014 7f5c973a57a0 15 read_log missing 194370'12427 (194370'12426) modify 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 by client.1508531.0:1182 2014-08-12 20:30:18.808413 2014-08-13 18:06:06.738050 7f5c973a57a0 15 read_log missing 194370'12425 (194370'12424) modify 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 by client.1508531.0:1027 2014-08-12 20:30:18.215574 2014-08-13 18:06:06.738086 7f5c973a57a0 15 read_log missing 194370'12423 (194370'12422) modify fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 by client.1508531.0:893 2014-08-12 20:30:17.732009 2014-08-13 18:06:06.738122 7f5c973a57a0 15 read_log missing 194370'12421 (194370'12420) modify 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 by client.1508531.0:846 2014-08-12 20:30:17.563335 2014-08-13 18:06:06.738133 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.738168 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_loaded 2014-08-13 18:06:06.738187 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Initial 0.017807 0 0.000000 2014-08-13 18:06:06.738220 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset 2014-08-13 18:06:06.738234 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=0 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194487 2014-08-13 18:06:06.738244 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 18:06:06.738255 7f5c973a57a0 10 osd.4 194487 pgid 24.1de coll 24.1de_head 2014-08-13 18:06:06.738445 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.1de 2014-08-13 18:06:06.738459 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.738492 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1de(unlocked)] enter Initial 2014-08-13 18:06:06.738504 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1de(unlocked)] enter NotTrimming 2014-08-13 18:06:06.738681 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.738813 7f5c973a57a0 20 read_log 192834'15291 (0'0) promote b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by osd.289.0:52105 2014-08-11 15:23:40.933638 2014-08-13 18:06:06.738835 7f5c973a57a0 20 read_log 192834'15292 (192834'15291) modify b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.933448.0:1358 2014-08-11 15:23:40.925381 2014-08-13 18:06:06.738849 7f5c973a57a0 20 read_log 192834'15299 (0'0) promote 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.289.0:52129 2014-08-11 15:23:43.401617 2014-08-13 18:06:06.738862 7f5c973a57a0 20 read_log 192834'15300 (192834'15299) modify 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by client.933448.0:1853 2014-08-11 15:23:43.393271 2014-08-13 18:06:06.738874 7f5c973a57a0 20 read_log 192834'15317 (0'0) promote 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by osd.289.0:52181 2014-08-11 15:23:49.371321 2014-08-13 18:06:06.738886 7f5c973a57a0 20 read_log 192834'15318 (192834'15317) modify 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.933448.0:2925 2014-08-11 15:23:49.356984 2014-08-13 18:06:06.738899 7f5c973a57a0 20 read_log 192834'15333 (0'0) promote 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by osd.289.0:52225 2014-08-11 15:23:53.304362 2014-08-13 18:06:06.738911 7f5c973a57a0 20 read_log 192834'15334 (192834'15333) modify 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.933448.0:3715 2014-08-11 15:23:53.272115 2014-08-13 18:06:06.738923 7f5c973a57a0 20 read_log 192834'15339 (0'0) promote 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by osd.289.0:52257 2014-08-11 15:23:56.052408 2014-08-13 18:06:06.738935 7f5c973a57a0 20 read_log 192834'15340 (192834'15339) modify 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.933448.0:4249 2014-08-11 15:23:56.045664 2014-08-13 18:06:06.738948 7f5c973a57a0 20 read_log 192834'15365 (0'0) promote 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.289.0:52319 2014-08-11 15:24:01.102588 2014-08-13 18:06:06.738960 7f5c973a57a0 20 read_log 192834'15366 (192834'15365) modify 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by client.933448.0:5353 2014-08-11 15:24:01.091198 2014-08-13 18:06:06.738972 7f5c973a57a0 20 read_log 192834'15377 (0'0) promote 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by osd.289.0:52359 2014-08-11 15:24:05.279182 2014-08-13 18:06:06.738984 7f5c973a57a0 20 read_log 192834'15378 (192834'15377) modify 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.933448.0:6401 2014-08-11 15:24:05.271713 2014-08-13 18:06:06.738997 7f5c973a57a0 20 read_log 192834'15381 (0'0) promote e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.289.0:52371 2014-08-11 15:24:06.061720 2014-08-13 18:06:06.739009 7f5c973a57a0 20 read_log 192834'15382 (192834'15381) modify e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by client.933448.0:6605 2014-08-11 15:24:06.053539 2014-08-13 18:06:06.739021 7f5c973a57a0 20 read_log 192834'15415 (0'0) promote d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by osd.289.0:52467 2014-08-11 15:24:15.658697 2014-08-13 18:06:06.739033 7f5c973a57a0 20 read_log 192834'15416 (192834'15415) modify d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.933448.0:8851 2014-08-11 15:24:15.650934 2014-08-13 18:06:06.739053 7f5c973a57a0 20 read_log 192834'15447 (0'0) promote 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by osd.289.0:52603 2014-08-11 15:24:29.705542 2014-08-13 18:06:06.739066 7f5c973a57a0 20 read_log 192834'15448 (192834'15447) modify 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.933448.0:11868 2014-08-11 15:24:29.698018 2014-08-13 18:06:06.739079 7f5c973a57a0 20 read_log 192834'15455 (0'0) promote 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.289.0:52665 2014-08-11 15:24:35.385250 2014-08-13 18:06:06.739091 7f5c973a57a0 20 read_log 192834'15456 (192834'15455) modify 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by client.933448.0:13104 2014-08-11 15:24:35.375933 2014-08-13 18:06:06.739103 7f5c973a57a0 20 read_log 192834'15465 (0'0) promote 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.289.0:52699 2014-08-11 15:24:39.017046 2014-08-13 18:06:06.739115 7f5c973a57a0 20 read_log 192834'15466 (192834'15465) modify 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by client.933448.0:13532 2014-08-11 15:24:39.007817 2014-08-13 18:06:06.739130 7f5c973a57a0 20 read_log 192834'15473 (0'0) promote 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.289.0:52727 2014-08-11 15:24:41.179691 2014-08-13 18:06:06.739143 7f5c973a57a0 20 read_log 192834'15474 (192834'15473) modify 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by client.933448.0:14088 2014-08-11 15:24:41.170226 2014-08-13 18:06:06.739164 7f5c973a57a0 20 read_log 192834'15481 (0'0) promote 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.289.0:52802 2014-08-11 15:24:46.475202 2014-08-13 18:06:06.739179 7f5c973a57a0 20 read_log 192834'15482 (192834'15481) modify 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by client.933448.0:14533 2014-08-11 15:24:46.052447 2014-08-13 18:06:06.739192 7f5c973a57a0 20 read_log 192834'15483 (0'0) promote ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by osd.289.0:52844 2014-08-11 15:24:48.403098 2014-08-13 18:06:06.739206 7f5c973a57a0 20 read_log 192834'15484 (192834'15483) modify ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.933448.0:15097 2014-08-11 15:24:48.394353 2014-08-13 18:06:06.739219 7f5c973a57a0 20 read_log 192834'15485 (0'0) promote de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by osd.289.0:52846 2014-08-11 15:24:48.647186 2014-08-13 18:06:06.739233 7f5c973a57a0 20 read_log 192834'15486 (192834'15485) modify de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.933448.0:15165 2014-08-11 15:24:48.638711 2014-08-13 18:06:06.739246 7f5c973a57a0 20 read_log 192834'15487 (0'0) promote 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by osd.289.0:52852 2014-08-11 15:24:48.916548 2014-08-13 18:06:06.739262 7f5c973a57a0 20 read_log 192834'15488 (192834'15487) modify 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.933448.0:15239 2014-08-11 15:24:48.908341 2014-08-13 18:06:06.739275 7f5c973a57a0 20 read_log 192834'15491 (0'0) promote 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by osd.289.0:52858 2014-08-11 15:24:49.131206 2014-08-13 18:06:06.739289 7f5c973a57a0 20 read_log 192834'15492 (192834'15491) modify 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.933448.0:15297 2014-08-11 15:24:49.122458 2014-08-13 18:06:06.739301 7f5c973a57a0 20 read_log 192834'15497 (0'0) promote cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by osd.289.0:52890 2014-08-11 15:24:50.970062 2014-08-13 18:06:06.739313 7f5c973a57a0 20 read_log 192834'15498 (192834'15497) modify cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.933448.0:15659 2014-08-11 15:24:50.549880 2014-08-13 18:06:06.739333 7f5c973a57a0 20 read_log 192834'15499 (0'0) promote 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by osd.289.0:52908 2014-08-11 15:24:52.853969 2014-08-13 18:06:06.739348 7f5c973a57a0 20 read_log 192834'15500 (192834'15499) modify 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.933448.0:16029 2014-08-11 15:24:52.342486 2014-08-13 18:06:06.739361 7f5c973a57a0 20 read_log 192834'15527 (0'0) promote 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.289.0:53002 2014-08-11 15:25:00.611616 2014-08-13 18:06:06.739382 7f5c973a57a0 20 read_log 192834'15528 (192834'15527) modify 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by client.933448.0:17525 2014-08-11 15:25:00.045602 2014-08-13 18:06:06.739399 7f5c973a57a0 20 read_log 192834'15547 (0'0) promote ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.289.0:53186 2014-08-11 15:25:14.020232 2014-08-13 18:06:06.739428 7f5c973a57a0 20 read_log 192834'15548 (192834'15547) modify ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by client.933448.0:19926 2014-08-11 15:25:14.011015 2014-08-13 18:06:06.739444 7f5c973a57a0 20 read_log 192834'15551 (0'0) promote 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by osd.289.0:53262 2014-08-11 15:25:20.897336 2014-08-13 18:06:06.739457 7f5c973a57a0 20 read_log 192834'15552 (192834'15551) modify 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.933448.0:20431 2014-08-11 15:25:20.888994 2014-08-13 18:06:06.739470 7f5c973a57a0 20 read_log 192834'15582 (192834'15290) modify 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by client.933448.0:23601 2014-08-11 15:25:39.541328 2014-08-13 18:06:06.739482 7f5c973a57a0 20 read_log 192834'15583 (192834'15292) modify b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.933448.0:23649 2014-08-11 15:25:39.621900 2014-08-13 18:06:06.739502 7f5c973a57a0 20 read_log 192834'15587 (192834'15300) modify 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by client.933448.0:24144 2014-08-11 15:25:40.326923 2014-08-13 18:06:06.739518 7f5c973a57a0 20 read_log 192834'15596 (192834'15318) modify 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.933448.0:25216 2014-08-11 15:25:41.892075 2014-08-13 18:06:06.739530 7f5c973a57a0 20 read_log 192834'15604 (192834'15334) modify 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.933448.0:26006 2014-08-11 15:25:42.839602 2014-08-13 18:06:06.739543 7f5c973a57a0 20 read_log 192834'15607 (192834'15340) modify 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.933448.0:26540 2014-08-11 15:25:43.519558 2014-08-13 18:06:06.739560 7f5c973a57a0 20 read_log 192834'15619 (192834'15366) modify 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by client.933448.0:27644 2014-08-11 15:25:44.744662 2014-08-13 18:06:06.739573 7f5c973a57a0 20 read_log 192834'15625 (192834'15378) modify 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.933448.0:28692 2014-08-11 15:25:46.599687 2014-08-13 18:06:06.739585 7f5c973a57a0 20 read_log 192834'15627 (192834'15382) modify e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by client.933448.0:28896 2014-08-11 15:25:46.811940 2014-08-13 18:06:06.739597 7f5c973a57a0 20 read_log 192834'15644 (192834'15416) modify d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.933448.0:31142 2014-08-11 15:25:49.691424 2014-08-13 18:06:06.739621 7f5c973a57a0 20 read_log 192834'15660 (192834'15448) modify 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.933448.0:34159 2014-08-11 15:25:54.655997 2014-08-13 18:06:06.739634 7f5c973a57a0 20 read_log 192834'15664 (192834'15456) modify 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by client.933448.0:35395 2014-08-11 15:25:56.198227 2014-08-13 18:06:06.739666 7f5c973a57a0 20 read_log 192834'15669 (192834'15466) modify 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by client.933448.0:35823 2014-08-11 15:25:56.818352 2014-08-13 18:06:06.739683 7f5c973a57a0 20 read_log 192834'15673 (192834'15474) modify 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by client.933448.0:36379 2014-08-11 15:25:57.632340 2014-08-13 18:06:06.739695 7f5c973a57a0 20 read_log 192834'15677 (192834'15482) modify 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by client.933448.0:36824 2014-08-11 15:25:58.344725 2014-08-13 18:06:06.739713 7f5c973a57a0 20 read_log 192834'15678 (192834'15484) modify ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.933448.0:37388 2014-08-11 15:25:59.496820 2014-08-13 18:06:06.739736 7f5c973a57a0 20 read_log 192834'15679 (192834'15486) modify de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.933448.0:37456 2014-08-11 15:25:59.580782 2014-08-13 18:06:06.739752 7f5c973a57a0 20 read_log 192834'15680 (192834'15488) modify 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.933448.0:37530 2014-08-11 15:25:59.662431 2014-08-13 18:06:06.739765 7f5c973a57a0 20 read_log 192834'15682 (192834'15492) modify 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.933448.0:37588 2014-08-11 15:25:59.725507 2014-08-13 18:06:06.739777 7f5c973a57a0 20 read_log 192834'15685 (192834'15498) modify cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.933448.0:37950 2014-08-11 15:26:00.302554 2014-08-13 18:06:06.739789 7f5c973a57a0 20 read_log 192834'15686 (192834'15500) modify 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.933448.0:38320 2014-08-11 15:26:00.753748 2014-08-13 18:06:06.739802 7f5c973a57a0 20 read_log 192834'15700 (192834'15528) modify 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by client.933448.0:39816 2014-08-11 15:26:02.102046 2014-08-13 18:06:06.739814 7f5c973a57a0 20 read_log 192834'15710 (192834'15548) modify ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by client.933448.0:42217 2014-08-11 15:26:03.191883 2014-08-13 18:06:06.739832 7f5c973a57a0 20 read_log 192834'15712 (192834'15552) modify 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.933448.0:42722 2014-08-11 15:26:03.367333 2014-08-13 18:06:06.739847 7f5c973a57a0 20 read_log 192834'15733 (0'0) promote 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by osd.289.0:54405 2014-08-11 15:26:18.707854 2014-08-13 18:06:06.739860 7f5c973a57a0 20 read_log 192834'15734 (192834'15733) modify 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.1883634.0:1161 2014-08-11 15:26:18.699367 2014-08-13 18:06:06.739872 7f5c973a57a0 20 read_log 192834'15739 (0'0) promote 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by osd.289.0:54419 2014-08-11 15:26:20.650244 2014-08-13 18:06:06.739884 7f5c973a57a0 20 read_log 192834'15740 (192834'15739) modify 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.1883634.0:1697 2014-08-11 15:26:20.641990 2014-08-13 18:06:06.739897 7f5c973a57a0 20 read_log 192834'15743 (0'0) promote c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by osd.289.0:54425 2014-08-11 15:26:21.311131 2014-08-13 18:06:06.739909 7f5c973a57a0 20 read_log 192834'15744 (192834'15743) modify c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.1883634.0:1851 2014-08-11 15:26:21.302753 2014-08-13 18:06:06.739921 7f5c973a57a0 20 read_log 192834'15775 (0'0) promote 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by osd.289.0:54529 2014-08-11 15:26:29.458366 2014-08-13 18:06:06.739933 7f5c973a57a0 20 read_log 192834'15776 (192834'15775) modify 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.1883634.0:4130 2014-08-11 15:26:29.447995 2014-08-13 18:06:06.739952 7f5c973a57a0 20 read_log 192834'15777 (0'0) promote 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by osd.289.0:54547 2014-08-11 15:26:30.344448 2014-08-13 18:06:06.739968 7f5c973a57a0 20 read_log 192834'15778 (192834'15777) modify 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.1883634.0:4380 2014-08-11 15:26:30.335888 2014-08-13 18:06:06.739980 7f5c973a57a0 20 read_log 192834'15789 (0'0) promote 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.289.0:54591 2014-08-11 15:26:33.267772 2014-08-13 18:06:06.739993 7f5c973a57a0 20 read_log 192834'15790 (192834'15789) modify 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by client.1883634.0:5114 2014-08-11 15:26:33.257770 2014-08-13 18:06:06.740005 7f5c973a57a0 20 read_log 192834'15797 (0'0) promote bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by osd.289.0:54601 2014-08-11 15:26:34.101653 2014-08-13 18:06:06.740017 7f5c973a57a0 20 read_log 192834'15798 (192834'15797) modify bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.1883634.0:5352 2014-08-11 15:26:34.093094 2014-08-13 18:06:06.740029 7f5c973a57a0 20 read_log 192834'15803 (0'0) promote 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by osd.289.0:54621 2014-08-11 15:26:36.635543 2014-08-13 18:06:06.740041 7f5c973a57a0 20 read_log 192834'15804 (192834'15803) modify 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.1883634.0:6063 2014-08-11 15:26:36.626941 2014-08-13 18:06:06.740053 7f5c973a57a0 20 read_log 192834'15805 (0'0) promote 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by osd.289.0:54623 2014-08-11 15:26:36.844058 2014-08-13 18:06:06.740065 7f5c973a57a0 20 read_log 192834'15806 (192834'15805) modify 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.1883634.0:6121 2014-08-11 15:26:36.836093 2014-08-13 18:06:06.740077 7f5c973a57a0 20 read_log 192834'15807 (0'0) promote e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by osd.289.0:54627 2014-08-11 15:26:37.227569 2014-08-13 18:06:06.740089 7f5c973a57a0 20 read_log 192834'15808 (192834'15807) modify e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.1883634.0:6228 2014-08-11 15:26:37.218600 2014-08-13 18:06:06.740123 7f5c973a57a0 20 read_log 192834'15811 (0'0) promote 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by osd.289.0:54635 2014-08-11 15:26:38.764384 2014-08-13 18:06:06.740139 7f5c973a57a0 20 read_log 192834'15812 (192834'15811) modify 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.1883634.0:6573 2014-08-11 15:26:38.754359 2014-08-13 18:06:06.740160 7f5c973a57a0 20 read_log 192834'15813 (0'0) promote e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by osd.289.0:54674 2014-08-11 15:26:39.503976 2014-08-13 18:06:06.740175 7f5c973a57a0 20 read_log 192834'15814 (192834'15813) modify e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.1883634.0:6718 2014-08-11 15:26:39.259959 2014-08-13 18:06:06.740188 7f5c973a57a0 20 read_log 192834'15837 (0'0) promote b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by osd.289.0:54816 2014-08-11 15:26:44.183129 2014-08-13 18:06:06.740202 7f5c973a57a0 20 read_log 192834'15838 (192834'15837) modify b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.1883634.0:7878 2014-08-11 15:26:44.174527 2014-08-13 18:06:06.740214 7f5c973a57a0 20 read_log 192834'15849 (0'0) promote e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by osd.289.0:54910 2014-08-11 15:26:47.921115 2014-08-13 18:06:06.740226 7f5c973a57a0 20 read_log 192834'15850 (192834'15849) modify e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.1883634.0:8797 2014-08-11 15:26:47.912751 2014-08-13 18:06:06.740249 7f5c973a57a0 20 read_log 192834'15861 (0'0) promote 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.289.0:55074 2014-08-11 15:26:51.727226 2014-08-13 18:06:06.740265 7f5c973a57a0 20 read_log 192834'15862 (192834'15861) modify 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by client.1883634.0:9643 2014-08-11 15:26:51.718381 2014-08-13 18:06:06.740277 7f5c973a57a0 20 read_log 192834'15867 (0'0) promote 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.289.0:55095 2014-08-11 15:26:53.373731 2014-08-13 18:06:06.740289 7f5c973a57a0 20 read_log 192834'15868 (192834'15867) modify 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by client.1883634.0:10010 2014-08-11 15:26:53.365862 2014-08-13 18:06:06.740306 7f5c973a57a0 20 read_log 192834'15871 (0'0) promote 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by osd.289.0:55145 2014-08-11 15:26:55.152694 2014-08-13 18:06:06.740318 7f5c973a57a0 20 read_log 192834'15872 (192834'15871) modify 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.1883634.0:10465 2014-08-11 15:26:55.144383 2014-08-13 18:06:06.740333 7f5c973a57a0 20 read_log 192834'15891 (0'0) promote 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.289.0:55354 2014-08-11 15:27:01.385257 2014-08-13 18:06:06.740349 7f5c973a57a0 20 read_log 192834'15892 (192834'15891) modify 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by client.1883634.0:11808 2014-08-11 15:27:01.333206 2014-08-13 18:06:06.740361 7f5c973a57a0 20 read_log 192834'15897 (0'0) promote 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.289.0:55365 2014-08-11 15:27:01.655703 2014-08-13 18:06:06.740376 7f5c973a57a0 20 read_log 192834'15898 (192834'15897) modify 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by client.1883634.0:11903 2014-08-11 15:27:01.647448 2014-08-13 18:06:06.740389 7f5c973a57a0 20 read_log 192834'15901 (0'0) promote 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by osd.289.0:55401 2014-08-11 15:27:04.633621 2014-08-13 18:06:06.740405 7f5c973a57a0 20 read_log 192834'15902 (192834'15901) modify 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.1883634.0:12580 2014-08-11 15:27:04.625607 2014-08-13 18:06:06.740417 7f5c973a57a0 20 read_log 192834'15929 (0'0) promote b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.289.0:55609 2014-08-11 15:27:16.283381 2014-08-13 18:06:06.740441 7f5c973a57a0 20 read_log 192834'15930 (192834'15929) modify b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by client.1883634.0:14942 2014-08-11 15:27:16.273615 2014-08-13 18:06:06.740458 7f5c973a57a0 20 read_log 192834'15969 (0'0) promote ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by osd.289.0:55933 2014-08-11 15:27:36.815571 2014-08-13 18:06:06.740471 7f5c973a57a0 20 read_log 192834'15970 (192834'15969) modify ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.1883634.0:17531 2014-08-11 15:27:36.140755 2014-08-13 18:06:06.740491 7f5c973a57a0 20 read_log 192834'15981 (0'0) promote c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.289.0:56062 2014-08-11 15:27:42.484935 2014-08-13 18:06:06.740506 7f5c973a57a0 20 read_log 192834'15982 (192834'15981) modify c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by client.1883634.0:18212 2014-08-11 15:27:42.189347 2014-08-13 18:06:06.740519 7f5c973a57a0 20 read_log 192834'15983 (0'0) promote 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by osd.289.0:56066 2014-08-11 15:27:42.485947 2014-08-13 18:06:06.740539 7f5c973a57a0 20 read_log 192834'15986 (192834'15983) modify 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.1883634.0:18226 2014-08-11 15:27:42.292021 2014-08-13 18:06:06.740596 7f5c973a57a0 20 read_log 192834'16015 (0'0) promote 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.289.0:56320 2014-08-11 15:28:01.006273 2014-08-13 18:06:06.740613 7f5c973a57a0 20 read_log 192834'16016 (192834'16015) modify 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by client.1883634.0:20663 2014-08-11 15:28:00.998106 2014-08-13 18:06:06.740631 7f5c973a57a0 20 read_log 192834'16021 (0'0) promote bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by osd.289.0:56332 2014-08-11 15:28:02.220078 2014-08-13 18:06:06.740644 7f5c973a57a0 20 read_log 192834'16022 (192834'16021) modify bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.1883634.0:20985 2014-08-11 15:28:02.209818 2014-08-13 18:06:06.740656 7f5c973a57a0 20 read_log 192834'16023 (0'0) promote 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by osd.289.0:56351 2014-08-11 15:28:03.128530 2014-08-13 18:06:06.740678 7f5c973a57a0 20 read_log 192834'16024 (192834'16023) modify 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.1883634.0:21170 2014-08-11 15:28:03.119360 2014-08-13 18:06:06.740693 7f5c973a57a0 20 read_log 192834'16035 (0'0) promote f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by osd.289.0:56408 2014-08-11 15:28:05.110596 2014-08-13 18:06:06.740706 7f5c973a57a0 20 read_log 192834'16036 (192834'16035) modify f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.1883634.0:21731 2014-08-11 15:28:05.100524 2014-08-13 18:06:06.740718 7f5c973a57a0 20 read_log 192834'16053 (0'0) promote dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.289.0:56497 2014-08-11 15:28:10.995115 2014-08-13 18:06:06.740730 7f5c973a57a0 20 read_log 192834'16054 (192834'16053) modify dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by client.1883634.0:22771 2014-08-11 15:28:10.984468 2014-08-13 18:06:06.740742 7f5c973a57a0 20 read_log 192834'16057 (0'0) promote 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by osd.289.0:56501 2014-08-11 15:28:11.404695 2014-08-13 18:06:06.740754 7f5c973a57a0 20 read_log 192834'16058 (192834'16057) modify 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.1883634.0:22854 2014-08-11 15:28:11.395067 2014-08-13 18:06:06.740766 7f5c973a57a0 20 read_log 192834'16064 (192834'15734) modify 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.1883634.0:24485 2014-08-11 15:28:19.117282 2014-08-13 18:06:06.740778 7f5c973a57a0 20 read_log 192834'16067 (192834'15740) modify 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.1883634.0:25021 2014-08-11 15:28:19.948513 2014-08-13 18:06:06.740790 7f5c973a57a0 20 read_log 192834'16069 (192834'15744) modify c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.1883634.0:25175 2014-08-11 15:28:20.295779 2014-08-13 18:06:06.740802 7f5c973a57a0 20 read_log 192834'16085 (192834'15776) modify 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.1883634.0:27454 2014-08-11 15:28:24.481588 2014-08-13 18:06:06.740815 7f5c973a57a0 20 read_log 192834'16086 (192834'15778) modify 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.1883634.0:27704 2014-08-11 15:28:24.844068 2014-08-13 18:06:06.740827 7f5c973a57a0 20 read_log 192834'16092 (192834'15790) modify 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by client.1883634.0:28438 2014-08-11 15:28:25.810373 2014-08-13 18:06:06.740840 7f5c973a57a0 20 read_log 192834'16096 (192834'15798) modify bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.1883634.0:28676 2014-08-11 15:28:26.128929 2014-08-13 18:06:06.740852 7f5c973a57a0 20 read_log 192834'16099 (192834'15804) modify 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.1883634.0:29387 2014-08-11 15:28:27.080615 2014-08-13 18:06:06.740872 7f5c973a57a0 20 read_log 192834'16100 (192834'15806) modify 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.1883634.0:29445 2014-08-11 15:28:27.280511 2014-08-13 18:06:06.740884 7f5c973a57a0 20 read_log 192834'16101 (192834'15808) modify e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.1883634.0:29552 2014-08-11 15:28:27.419922 2014-08-13 18:06:06.740896 7f5c973a57a0 20 read_log 192834'16103 (192834'15812) modify 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.1883634.0:29897 2014-08-11 15:28:27.805577 2014-08-13 18:06:06.740908 7f5c973a57a0 20 read_log 192834'16104 (192834'15814) modify e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.1883634.0:30042 2014-08-11 15:28:27.997594 2014-08-13 18:06:06.740920 7f5c973a57a0 20 read_log 192834'16116 (192834'15838) modify b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.1883634.0:31202 2014-08-11 15:28:30.092629 2014-08-13 18:06:06.740934 7f5c973a57a0 20 read_log 192834'16122 (192834'15850) modify e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.1883634.0:32121 2014-08-11 15:28:31.370096 2014-08-13 18:06:06.740946 7f5c973a57a0 20 read_log 192834'16128 (192834'15862) modify 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by client.1883634.0:32967 2014-08-11 15:28:32.611718 2014-08-13 18:06:06.740958 7f5c973a57a0 20 read_log 192834'16131 (192834'15868) modify 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by client.1883634.0:33334 2014-08-11 15:28:33.034439 2014-08-13 18:06:06.740971 7f5c973a57a0 20 read_log 192834'16133 (192834'15872) modify 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.1883634.0:33789 2014-08-11 15:28:33.801448 2014-08-13 18:06:06.740983 7f5c973a57a0 20 read_log 192834'16143 (192834'15892) modify 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by client.1883634.0:35132 2014-08-11 15:28:36.060916 2014-08-13 18:06:06.740995 7f5c973a57a0 20 read_log 192834'16146 (192834'15898) modify 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by client.1883634.0:35227 2014-08-11 15:28:36.283478 2014-08-13 18:06:06.741007 7f5c973a57a0 20 read_log 192834'16150 (192834'15902) modify 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.1883634.0:35904 2014-08-11 15:28:37.295930 2014-08-13 18:06:06.741019 7f5c973a57a0 20 read_log 192834'16164 (192834'15930) modify b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by client.1883634.0:38266 2014-08-11 15:28:40.426091 2014-08-13 18:06:06.741033 7f5c973a57a0 20 read_log 192834'16184 (192834'15970) modify ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.1883634.0:40855 2014-08-11 15:28:44.336970 2014-08-13 18:06:06.741046 7f5c973a57a0 20 read_log 192834'16191 (192834'15982) modify c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by client.1883634.0:41536 2014-08-11 15:28:45.248191 2014-08-13 18:06:06.741058 7f5c973a57a0 20 read_log 192834'16192 (192834'15986) modify 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.1883634.0:41550 2014-08-11 15:28:45.266531 2014-08-13 18:06:06.741070 7f5c973a57a0 20 read_log 192834'16207 (192834'16016) modify 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by client.1883634.0:43987 2014-08-11 15:28:46.724669 2014-08-13 18:06:06.741082 7f5c973a57a0 20 read_log 192834'16210 (192834'16022) modify bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.1883634.0:44309 2014-08-11 15:28:46.824341 2014-08-13 18:06:06.741094 7f5c973a57a0 20 read_log 192834'16211 (192834'16024) modify 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.1883634.0:44494 2014-08-11 15:28:46.873704 2014-08-13 18:06:06.741106 7f5c973a57a0 20 read_log 192834'16217 (192834'16036) modify f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.1883634.0:45055 2014-08-11 15:28:47.135120 2014-08-13 18:06:06.741125 7f5c973a57a0 20 read_log 192834'16226 (192834'16054) modify dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by client.1883634.0:46095 2014-08-11 15:28:47.489765 2014-08-13 18:06:06.741141 7f5c973a57a0 20 read_log 192834'16228 (192834'16058) modify 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.1883634.0:46178 2014-08-11 15:28:47.513293 2014-08-13 18:06:06.741162 7f5c973a57a0 20 read_log 192888'16229 (192888'16229) modify 1de/hit_set_24.1de_archive_2014-08-11 16:02:27.862431_2014-08-11 16:44:55.175023/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:55.175327 2014-08-13 18:06:06.741175 7f5c973a57a0 20 read_log 192888'16234 (0'0) promote 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by osd.364.0:19362 2014-08-11 16:45:02.365991 2014-08-13 18:06:06.741188 7f5c973a57a0 20 read_log 192888'16235 (192888'16234) modify 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.1315259.0:1862 2014-08-11 16:45:02.357604 2014-08-13 18:06:06.741200 7f5c973a57a0 20 read_log 192888'16244 (0'0) promote 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by osd.364.0:19402 2014-08-11 16:45:10.217293 2014-08-13 18:06:06.741212 7f5c973a57a0 20 read_log 192888'16245 (192888'16244) modify 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.1315259.0:3934 2014-08-11 16:45:10.209837 2014-08-13 18:06:06.741224 7f5c973a57a0 20 read_log 192888'16248 (0'0) promote e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by osd.364.0:19416 2014-08-11 16:45:12.506789 2014-08-13 18:06:06.741236 7f5c973a57a0 20 read_log 192888'16249 (192888'16248) modify e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.1315259.0:4562 2014-08-11 16:45:12.499663 2014-08-13 18:06:06.741249 7f5c973a57a0 20 read_log 192888'16250 (0'0) promote 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:19430 2014-08-11 16:45:14.484377 2014-08-13 18:06:06.741263 7f5c973a57a0 20 read_log 192888'16251 (192888'16250) modify 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by client.1315259.0:5090 2014-08-11 16:45:14.476281 2014-08-13 18:06:06.741276 7f5c973a57a0 20 read_log 192888'16252 (0'0) promote 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by osd.364.0:19434 2014-08-11 16:45:15.153767 2014-08-13 18:06:06.741288 7f5c973a57a0 20 read_log 192888'16253 (192888'16252) modify 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.1315259.0:5274 2014-08-11 16:45:15.146353 2014-08-13 18:06:06.741300 7f5c973a57a0 20 read_log 192888'16254 (0'0) promote 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:19440 2014-08-11 16:45:15.643385 2014-08-13 18:06:06.741315 7f5c973a57a0 20 read_log 192888'16255 (192888'16254) modify 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by client.1315259.0:5409 2014-08-11 16:45:15.635592 2014-08-13 18:06:06.741327 7f5c973a57a0 20 read_log 192888'16258 (0'0) promote 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:19452 2014-08-11 16:45:17.030728 2014-08-13 18:06:06.741340 7f5c973a57a0 20 read_log 192888'16259 (192888'16258) modify 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by client.1315259.0:5791 2014-08-11 16:45:17.022785 2014-08-13 18:06:06.741352 7f5c973a57a0 20 read_log 192888'16262 (0'0) promote b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by osd.364.0:19502 2014-08-11 16:45:25.488980 2014-08-13 18:06:06.741368 7f5c973a57a0 20 read_log 192888'16263 (192888'16262) modify b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.1315259.0:8124 2014-08-11 16:45:25.481414 2014-08-13 18:06:06.741390 7f5c973a57a0 20 read_log 192888'16266 (0'0) promote 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by osd.364.0:19532 2014-08-11 16:45:30.002986 2014-08-13 18:06:06.741403 7f5c973a57a0 20 read_log 192888'16267 (192888'16266) modify 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.1315259.0:9317 2014-08-11 16:45:29.995421 2014-08-13 18:06:06.741419 7f5c973a57a0 20 read_log 192888'16268 (0'0) promote 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by osd.364.0:19558 2014-08-11 16:45:32.552281 2014-08-13 18:06:06.741432 7f5c973a57a0 20 read_log 192888'16269 (192888'16268) modify 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.1315259.0:9970 2014-08-11 16:45:32.545851 2014-08-13 18:06:06.741444 7f5c973a57a0 20 read_log 192888'16270 (0'0) promote 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by osd.364.0:19574 2014-08-11 16:45:36.817974 2014-08-13 18:06:06.741456 7f5c973a57a0 20 read_log 192888'16271 (192888'16270) modify 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.1315259.0:11050 2014-08-11 16:45:36.807954 2014-08-13 18:06:06.741498 7f5c973a57a0 20 read_log 192888'16272 (0'0) promote e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by osd.364.0:19580 2014-08-11 16:45:38.510500 2014-08-13 18:06:06.741524 7f5c973a57a0 20 read_log 192888'16273 (192888'16272) modify e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.1315259.0:11478 2014-08-11 16:45:38.502880 2014-08-13 18:06:06.741540 7f5c973a57a0 20 read_log 192888'16280 (0'0) promote bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:19630 2014-08-11 16:45:44.929039 2014-08-13 18:06:06.741553 7f5c973a57a0 20 read_log 192888'16281 (192888'16280) modify bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by client.1315259.0:13015 2014-08-11 16:45:44.921303 2014-08-13 18:06:06.741573 7f5c973a57a0 20 read_log 192888'16282 (0'0) promote 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by osd.364.0:19642 2014-08-11 16:45:46.677924 2014-08-13 18:06:06.741590 7f5c973a57a0 20 read_log 192888'16283 (192888'16282) modify 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.1315259.0:13450 2014-08-11 16:45:46.669664 2014-08-13 18:06:06.741607 7f5c973a57a0 20 read_log 192888'16292 (0'0) promote 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by osd.364.0:19700 2014-08-11 16:45:55.975387 2014-08-13 18:06:06.741629 7f5c973a57a0 20 read_log 192888'16293 (192888'16292) modify 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.1315259.0:15879 2014-08-11 16:45:55.965966 2014-08-13 18:06:06.741642 7f5c973a57a0 20 read_log 192888'16294 (0'0) promote 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by osd.364.0:19720 2014-08-11 16:45:59.912751 2014-08-13 18:06:06.741654 7f5c973a57a0 20 read_log 192888'16295 (192888'16294) modify 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.1315259.0:16897 2014-08-11 16:45:59.904944 2014-08-13 18:06:06.741667 7f5c973a57a0 20 read_log 192888'16298 (0'0) promote 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by osd.364.0:19752 2014-08-11 16:46:04.229801 2014-08-13 18:06:06.741679 7f5c973a57a0 20 read_log 192888'16299 (192888'16298) modify 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.1315259.0:17949 2014-08-11 16:46:04.222145 2014-08-13 18:06:06.741692 7f5c973a57a0 20 read_log 192888'16300 (0'0) promote 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by osd.364.0:19770 2014-08-11 16:46:08.009256 2014-08-13 18:06:06.741704 7f5c973a57a0 20 read_log 192888'16301 (192888'16300) modify 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.1315259.0:18958 2014-08-11 16:46:08.002035 2014-08-13 18:06:06.741724 7f5c973a57a0 20 read_log 192888'16302 (0'0) promote 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by osd.364.0:19786 2014-08-11 16:46:10.260014 2014-08-13 18:06:06.741739 7f5c973a57a0 20 read_log 192888'16303 (192888'16302) modify 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.1315259.0:19494 2014-08-11 16:46:10.252744 2014-08-13 18:06:06.741751 7f5c973a57a0 20 read_log 192888'16310 (0'0) promote 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:19830 2014-08-11 16:46:18.621241 2014-08-13 18:06:06.741763 7f5c973a57a0 20 read_log 192888'16311 (192888'16310) modify 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by client.1315259.0:21722 2014-08-11 16:46:18.613444 2014-08-13 18:06:06.741775 7f5c973a57a0 20 read_log 192888'16312 (0'0) promote de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by osd.364.0:19838 2014-08-11 16:46:20.689649 2014-08-13 18:06:06.741787 7f5c973a57a0 20 read_log 192888'16313 (192888'16312) modify de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.1315259.0:22258 2014-08-11 16:46:20.681703 2014-08-13 18:06:06.741799 7f5c973a57a0 20 read_log 192888'16318 (0'0) promote 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:19862 2014-08-11 16:46:24.434089 2014-08-13 18:06:06.741811 7f5c973a57a0 20 read_log 192888'16319 (192888'16318) modify 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by client.1315259.0:23191 2014-08-11 16:46:24.425606 2014-08-13 18:06:06.741824 7f5c973a57a0 20 read_log 192888'16320 (0'0) promote e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by osd.364.0:19876 2014-08-11 16:46:27.010744 2014-08-13 18:06:06.741836 7f5c973a57a0 20 read_log 192888'16321 (192888'16320) modify e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.1315259.0:23836 2014-08-11 16:46:27.001698 2014-08-13 18:06:06.741848 7f5c973a57a0 20 read_log 192888'16326 (0'0) promote c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:19914 2014-08-11 16:46:33.075963 2014-08-13 18:06:06.741860 7f5c973a57a0 20 read_log 192888'16327 (192888'16326) modify c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by client.1315259.0:25384 2014-08-11 16:46:33.069018 2014-08-13 18:06:06.741872 7f5c973a57a0 20 read_log 192888'16328 (0'0) promote 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:19916 2014-08-11 16:46:33.097385 2014-08-13 18:06:06.741884 7f5c973a57a0 20 read_log 192888'16329 (192888'16328) modify 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by client.1315259.0:25390 2014-08-11 16:46:33.090563 2014-08-13 18:06:06.741896 7f5c973a57a0 20 read_log 192888'16334 (0'0) promote 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by osd.364.0:19982 2014-08-11 16:46:44.641020 2014-08-13 18:06:06.741908 7f5c973a57a0 20 read_log 192888'16335 (192888'16334) modify 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.1315259.0:28172 2014-08-11 16:46:44.632411 2014-08-13 18:06:06.741929 7f5c973a57a0 20 read_log 192888'16336 (0'0) promote 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by osd.364.0:20026 2014-08-11 16:46:52.636669 2014-08-13 18:06:06.741943 7f5c973a57a0 20 read_log 192888'16337 (192888'16336) modify 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.1315259.0:30141 2014-08-11 16:46:52.628733 2014-08-13 18:06:06.741956 7f5c973a57a0 20 read_log 192888'16344 (192888'16235) modify 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.1315259.0:32589 2014-08-11 16:46:56.418727 2014-08-13 18:06:06.741968 7f5c973a57a0 20 read_log 192888'16349 (192888'16245) modify 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.1315259.0:34661 2014-08-11 16:46:56.952571 2014-08-13 18:06:06.741987 7f5c973a57a0 20 read_log 192888'16351 (192888'16249) modify e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.1315259.0:35289 2014-08-11 16:46:57.125816 2014-08-13 18:06:06.742011 7f5c973a57a0 20 read_log 192888'16352 (192888'16251) modify 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by client.1315259.0:35817 2014-08-11 16:46:57.254062 2014-08-13 18:06:06.742023 7f5c973a57a0 20 read_log 192888'16353 (192888'16253) modify 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.1315259.0:36001 2014-08-11 16:46:57.300781 2014-08-13 18:06:06.742036 7f5c973a57a0 20 read_log 192888'16354 (192888'16255) modify 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by client.1315259.0:36136 2014-08-11 16:46:57.345860 2014-08-13 18:06:06.742051 7f5c973a57a0 20 read_log 192888'16356 (192888'16259) modify 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by client.1315259.0:36518 2014-08-11 16:46:57.444282 2014-08-13 18:06:06.742066 7f5c973a57a0 20 read_log 192888'16358 (192888'16263) modify b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.1315259.0:38851 2014-08-11 16:46:58.061259 2014-08-13 18:06:06.742079 7f5c973a57a0 20 read_log 192888'16360 (192888'16267) modify 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.1315259.0:40044 2014-08-11 16:46:58.372533 2014-08-13 18:06:06.742091 7f5c973a57a0 20 read_log 192888'16361 (192888'16269) modify 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.1315259.0:40697 2014-08-11 16:46:58.556751 2014-08-13 18:06:06.742107 7f5c973a57a0 20 read_log 192888'16362 (192888'16271) modify 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.1315259.0:41777 2014-08-11 16:46:58.834777 2014-08-13 18:06:06.742129 7f5c973a57a0 20 read_log 192888'16363 (192888'16273) modify e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.1315259.0:42205 2014-08-11 16:46:58.946852 2014-08-13 18:06:06.742141 7f5c973a57a0 20 read_log 192888'16367 (192888'16281) modify bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by client.1315259.0:43742 2014-08-11 16:46:59.375212 2014-08-13 18:06:06.742163 7f5c973a57a0 20 read_log 192888'16368 (192888'16283) modify 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.1315259.0:44177 2014-08-11 16:46:59.503315 2014-08-13 18:06:06.742176 7f5c973a57a0 20 read_log 192888'16373 (192888'16293) modify 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.1315259.0:46606 2014-08-11 16:47:00.206167 2014-08-13 18:06:06.742188 7f5c973a57a0 20 read_log 192888'16374 (192888'16295) modify 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.1315259.0:47624 2014-08-11 16:47:00.516747 2014-08-13 18:06:06.742210 7f5c973a57a0 20 read_log 192888'16376 (192888'16299) modify 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.1315259.0:48676 2014-08-11 16:47:00.833176 2014-08-13 18:06:06.742226 7f5c973a57a0 20 read_log 192888'16377 (192888'16301) modify 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.1315259.0:49685 2014-08-11 16:47:01.142954 2014-08-13 18:06:06.742239 7f5c973a57a0 20 read_log 192888'16378 (192888'16303) modify 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.1315259.0:50221 2014-08-11 16:47:01.300075 2014-08-13 18:06:06.742259 7f5c973a57a0 20 read_log 192888'16382 (192888'16311) modify 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by client.1315259.0:52449 2014-08-11 16:47:01.976544 2014-08-13 18:06:06.742274 7f5c973a57a0 20 read_log 192888'16383 (192888'16313) modify de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.1315259.0:52985 2014-08-11 16:47:02.123022 2014-08-13 18:06:06.742287 7f5c973a57a0 20 read_log 192888'16386 (192888'16319) modify 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by client.1315259.0:53918 2014-08-11 16:47:02.402943 2014-08-13 18:06:06.742318 7f5c973a57a0 20 read_log 192888'16387 (192888'16321) modify e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.1315259.0:54563 2014-08-11 16:47:02.602145 2014-08-13 18:06:06.742340 7f5c973a57a0 20 read_log 192888'16390 (192888'16327) modify c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by client.1315259.0:56111 2014-08-11 16:47:03.108800 2014-08-13 18:06:06.742364 7f5c973a57a0 20 read_log 192888'16391 (192888'16329) modify 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by client.1315259.0:56117 2014-08-11 16:47:03.110276 2014-08-13 18:06:06.742379 7f5c973a57a0 20 read_log 192888'16394 (192888'16335) modify 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.1315259.0:58899 2014-08-11 16:47:03.941157 2014-08-13 18:06:06.742401 7f5c973a57a0 20 read_log 192888'16395 (192888'16337) modify 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.1315259.0:60868 2014-08-11 16:47:04.564442 2014-08-13 18:06:06.742418 7f5c973a57a0 20 read_log 192888'16398 (192888'16398) modify 1de/hit_set_24.1de_archive_2014-08-11 16:44:55.175331_2014-08-11 17:05:15.139431/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.139657 2014-08-13 18:06:06.742431 7f5c973a57a0 20 read_log 192888'16399 (192888'16229) delete 1de/hit_set_24.1de_archive_2014-08-11 16:02:27.862431_2014-08-11 16:44:55.175023/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.139657 2014-08-13 18:06:06.742443 7f5c973a57a0 20 read_log 192888'16406 (0'0) promote 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:20439 2014-08-11 17:05:18.937678 2014-08-13 18:06:06.742456 7f5c973a57a0 20 read_log 192888'16407 (192888'16406) modify 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by client.942868.0:962 2014-08-11 17:05:18.930939 2014-08-13 18:06:06.742468 7f5c973a57a0 20 read_log 192888'16422 (0'0) promote ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by osd.364.0:20495 2014-08-11 17:05:24.797291 2014-08-13 18:06:06.742481 7f5c973a57a0 20 read_log 192888'16423 (192888'16422) modify ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.942868.0:2393 2014-08-11 17:05:24.790409 2014-08-13 18:06:06.742493 7f5c973a57a0 20 read_log 192888'16428 (0'0) promote cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by osd.364.0:20548 2014-08-11 17:05:33.528128 2014-08-13 18:06:06.742505 7f5c973a57a0 20 read_log 192888'16429 (192888'16428) modify cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.942868.0:4739 2014-08-11 17:05:33.520182 2014-08-13 18:06:06.742517 7f5c973a57a0 20 read_log 192888'16430 (0'0) promote 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by osd.364.0:20558 2014-08-11 17:05:34.246892 2014-08-13 18:06:06.742529 7f5c973a57a0 20 read_log 192888'16431 (192888'16430) modify 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.942868.0:4913 2014-08-11 17:05:34.238843 2014-08-13 18:06:06.742542 7f5c973a57a0 20 read_log 192888'16432 (0'0) promote 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by osd.364.0:20560 2014-08-11 17:05:35.011449 2014-08-13 18:06:06.742554 7f5c973a57a0 20 read_log 192888'16433 (192888'16432) modify 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.942868.0:5103 2014-08-11 17:05:35.004378 2014-08-13 18:06:06.742566 7f5c973a57a0 20 read_log 192888'16440 (0'0) promote 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by osd.364.0:20584 2014-08-11 17:05:37.764615 2014-08-13 18:06:06.742580 7f5c973a57a0 20 read_log 192888'16441 (192888'16440) modify 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.942868.0:5846 2014-08-11 17:05:37.755472 2014-08-13 18:06:06.742616 7f5c973a57a0 20 read_log 192888'16444 (0'0) promote 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by osd.364.0:20626 2014-08-11 17:05:46.758850 2014-08-13 18:06:06.742631 7f5c973a57a0 20 read_log 192888'16445 (192888'16444) modify 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.942868.0:8258 2014-08-11 17:05:46.751066 2014-08-13 18:06:06.742644 7f5c973a57a0 20 read_log 192888'16448 (0'0) promote 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:20634 2014-08-11 17:05:48.042259 2014-08-13 18:06:06.742656 7f5c973a57a0 20 read_log 192888'16449 (192888'16448) modify 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by client.942868.0:8613 2014-08-11 17:05:48.033997 2014-08-13 18:06:06.742669 7f5c973a57a0 20 read_log 192888'16454 (0'0) promote b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:20690 2014-08-11 17:05:56.269991 2014-08-13 18:06:06.742681 7f5c973a57a0 20 read_log 192888'16455 (192888'16454) modify b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by client.942868.0:10853 2014-08-11 17:05:56.263272 2014-08-13 18:06:06.742693 7f5c973a57a0 20 read_log 192888'16460 (0'0) promote 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:20720 2014-08-11 17:06:00.982692 2014-08-13 18:06:06.742705 7f5c973a57a0 20 read_log 192888'16461 (192888'16460) modify 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by client.942868.0:12133 2014-08-11 17:06:00.975755 2014-08-13 18:06:06.742717 7f5c973a57a0 20 read_log 192888'16468 (0'0) promote ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by osd.364.0:20780 2014-08-11 17:06:13.826025 2014-08-13 18:06:06.742729 7f5c973a57a0 20 read_log 192888'16469 (192888'16468) modify ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.942868.0:15411 2014-08-11 17:06:13.816897 2014-08-13 18:06:06.742741 7f5c973a57a0 20 read_log 192888'16470 (0'0) promote 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:20784 2014-08-11 17:06:14.540096 2014-08-13 18:06:06.742753 7f5c973a57a0 20 read_log 192888'16471 (192888'16470) modify 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by client.942868.0:15595 2014-08-11 17:06:14.532350 2014-08-13 18:06:06.742765 7f5c973a57a0 20 read_log 192888'16474 (0'0) promote 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by osd.364.0:20820 2014-08-11 17:06:20.165524 2014-08-13 18:06:06.742777 7f5c973a57a0 20 read_log 192888'16475 (192888'16474) modify 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.942868.0:17027 2014-08-11 17:06:20.157502 2014-08-13 18:06:06.742789 7f5c973a57a0 20 read_log 192888'16476 (0'0) promote c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by osd.364.0:20838 2014-08-11 17:06:22.533295 2014-08-13 18:06:06.742801 7f5c973a57a0 20 read_log 192888'16477 (192888'16476) modify c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.942868.0:17654 2014-08-11 17:06:22.525712 2014-08-13 18:06:06.742813 7f5c973a57a0 20 read_log 192888'16478 (0'0) promote a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by osd.364.0:20854 2014-08-11 17:06:24.656726 2014-08-13 18:06:06.742825 7f5c973a57a0 20 read_log 192888'16479 (192888'16478) modify a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.942868.0:18216 2014-08-11 17:06:24.648836 2014-08-13 18:06:06.742837 7f5c973a57a0 20 read_log 192888'16486 (0'0) promote 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:20874 2014-08-11 17:06:26.991702 2014-08-13 18:06:06.742849 7f5c973a57a0 20 read_log 192888'16487 (192888'16486) modify 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by client.942868.0:18839 2014-08-11 17:06:26.984106 2014-08-13 18:06:06.742885 7f5c973a57a0 20 read_log 192888'16490 (0'0) promote d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by osd.364.0:20900 2014-08-11 17:06:31.560696 2014-08-13 18:06:06.742901 7f5c973a57a0 20 read_log 192888'16491 (192888'16490) modify d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.942868.0:20053 2014-08-11 17:06:31.553712 2014-08-13 18:06:06.742914 7f5c973a57a0 20 read_log 192888'16492 (0'0) promote 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by osd.364.0:20908 2014-08-11 17:06:32.446494 2014-08-13 18:06:06.742926 7f5c973a57a0 20 read_log 192888'16493 (192888'16492) modify 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.942868.0:20291 2014-08-11 17:06:32.439294 2014-08-13 18:06:06.742940 7f5c973a57a0 20 read_log 192888'16502 (0'0) promote a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:20954 2014-08-11 17:06:40.072326 2014-08-13 18:06:06.742952 7f5c973a57a0 20 read_log 192888'16503 (192888'16502) modify a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by client.942868.0:22282 2014-08-11 17:06:40.065628 2014-08-13 18:06:06.742965 7f5c973a57a0 20 read_log 192888'16504 (0'0) promote 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by osd.364.0:20964 2014-08-11 17:06:40.571476 2014-08-13 18:06:06.742979 7f5c973a57a0 20 read_log 192888'16505 (192888'16504) modify 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.942868.0:22403 2014-08-11 17:06:40.563489 2014-08-13 18:06:06.742991 7f5c973a57a0 20 read_log 192888'16508 (0'0) promote 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:21002 2014-08-11 17:06:46.309230 2014-08-13 18:06:06.743005 7f5c973a57a0 20 read_log 192888'16509 (192888'16508) modify 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by client.942868.0:23890 2014-08-11 17:06:46.301650 2014-08-13 18:06:06.743018 7f5c973a57a0 20 read_log 192888'16510 (0'0) promote 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:21014 2014-08-11 17:06:48.132069 2014-08-13 18:06:06.743030 7f5c973a57a0 20 read_log 192888'16511 (192888'16510) modify 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by client.942868.0:24387 2014-08-11 17:06:48.124661 2014-08-13 18:06:06.743042 7f5c973a57a0 20 read_log 192888'16514 (0'0) promote 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by osd.364.0:21022 2014-08-11 17:06:49.291121 2014-08-13 18:06:06.743054 7f5c973a57a0 20 read_log 192888'16515 (192888'16514) modify 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.942868.0:24697 2014-08-11 17:06:49.284245 2014-08-13 18:06:06.743068 7f5c973a57a0 20 read_log 192888'16516 (0'0) promote eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by osd.364.0:21028 2014-08-11 17:06:49.436781 2014-08-13 18:06:06.743080 7f5c973a57a0 20 read_log 192888'16517 (192888'16516) modify eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.942868.0:24736 2014-08-11 17:06:49.429485 2014-08-13 18:06:06.743094 7f5c973a57a0 20 read_log 192888'16518 (0'0) promote 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by osd.364.0:21066 2014-08-11 17:06:55.593921 2014-08-13 18:06:06.743107 7f5c973a57a0 20 read_log 192888'16519 (192888'16518) modify 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.942868.0:26306 2014-08-11 17:06:55.587243 2014-08-13 18:06:06.743121 7f5c973a57a0 20 read_log 192888'16522 (0'0) promote 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by osd.364.0:21094 2014-08-11 17:07:01.623097 2014-08-13 18:06:06.743133 7f5c973a57a0 20 read_log 192888'16523 (192888'16522) modify 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.942868.0:27899 2014-08-11 17:07:01.615714 2014-08-13 18:06:06.743160 7f5c973a57a0 20 read_log 192888'16526 (0'0) promote 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by osd.364.0:21136 2014-08-11 17:07:08.980189 2014-08-13 18:06:06.743176 7f5c973a57a0 20 read_log 192888'16527 (192888'16526) modify 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.942868.0:29793 2014-08-11 17:07:08.973343 2014-08-13 18:06:06.743189 7f5c973a57a0 20 read_log 192888'16535 (192888'16407) modify 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by client.942868.0:32228 2014-08-11 17:07:16.194477 2014-08-13 18:06:06.743203 7f5c973a57a0 20 read_log 192888'16543 (192888'16423) modify ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.942868.0:33659 2014-08-11 17:07:16.575315 2014-08-13 18:06:06.743216 7f5c973a57a0 20 read_log 192888'16546 (192888'16429) modify cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.942868.0:36005 2014-08-11 17:07:17.218100 2014-08-13 18:06:06.743230 7f5c973a57a0 20 read_log 192888'16547 (192888'16431) modify 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.942868.0:36179 2014-08-11 17:07:17.260815 2014-08-13 18:06:06.743242 7f5c973a57a0 20 read_log 192888'16548 (192888'16433) modify 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.942868.0:36369 2014-08-11 17:07:17.310466 2014-08-13 18:06:06.743260 7f5c973a57a0 20 read_log 192888'16552 (192888'16441) modify 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.942868.0:37112 2014-08-11 17:07:17.518995 2014-08-13 18:06:06.743274 7f5c973a57a0 20 read_log 192888'16554 (192888'16445) modify 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.942868.0:39524 2014-08-11 17:07:18.205816 2014-08-13 18:06:06.743286 7f5c973a57a0 20 read_log 192888'16556 (192888'16449) modify 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by client.942868.0:39879 2014-08-11 17:07:18.307556 2014-08-13 18:06:06.743298 7f5c973a57a0 20 read_log 192888'16559 (192888'16455) modify b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by client.942868.0:42119 2014-08-11 17:07:18.910739 2014-08-13 18:06:06.743314 7f5c973a57a0 20 read_log 192888'16562 (192888'16461) modify 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by client.942868.0:43399 2014-08-11 17:07:19.262028 2014-08-13 18:06:06.743326 7f5c973a57a0 20 read_log 192888'16566 (192888'16469) modify ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.942868.0:46677 2014-08-11 17:07:20.188625 2014-08-13 18:06:06.743338 7f5c973a57a0 20 read_log 192888'16567 (192888'16471) modify 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by client.942868.0:46861 2014-08-11 17:07:20.238902 2014-08-13 18:06:06.743354 7f5c973a57a0 20 read_log 192888'16569 (192888'16475) modify 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.942868.0:48293 2014-08-11 17:07:20.678616 2014-08-13 18:06:06.743371 7f5c973a57a0 20 read_log 192888'16570 (192888'16477) modify c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.942868.0:48920 2014-08-11 17:07:20.857758 2014-08-13 18:06:06.743400 7f5c973a57a0 20 read_log 192888'16571 (192888'16479) modify a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.942868.0:49482 2014-08-11 17:07:21.026204 2014-08-13 18:06:06.743416 7f5c973a57a0 20 read_log 192888'16575 (192888'16487) modify 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by client.942868.0:50105 2014-08-11 17:07:21.218616 2014-08-13 18:06:06.743436 7f5c973a57a0 20 read_log 192888'16577 (192888'16491) modify d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.942868.0:51319 2014-08-11 17:07:21.607241 2014-08-13 18:06:06.743451 7f5c973a57a0 20 read_log 192888'16578 (192888'16493) modify 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.942868.0:51557 2014-08-11 17:07:21.681049 2014-08-13 18:06:06.743478 7f5c973a57a0 20 read_log 192888'16583 (192888'16503) modify a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by client.942868.0:53548 2014-08-11 17:07:22.301462 2014-08-13 18:06:06.743494 7f5c973a57a0 20 read_log 192888'16584 (192888'16505) modify 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.942868.0:53669 2014-08-11 17:07:22.338824 2014-08-13 18:06:06.743507 7f5c973a57a0 20 read_log 192888'16586 (192888'16509) modify 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by client.942868.0:55156 2014-08-11 17:07:22.760166 2014-08-13 18:06:06.743519 7f5c973a57a0 20 read_log 192888'16587 (192888'16511) modify 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by client.942868.0:55653 2014-08-11 17:07:22.912653 2014-08-13 18:06:06.743543 7f5c973a57a0 20 read_log 192888'16589 (192888'16515) modify 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.942868.0:55963 2014-08-11 17:07:23.009726 2014-08-13 18:06:06.743555 7f5c973a57a0 20 read_log 192888'16590 (192888'16517) modify eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.942868.0:56002 2014-08-11 17:07:23.021591 2014-08-13 18:06:06.743568 7f5c973a57a0 20 read_log 192888'16591 (192888'16519) modify 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.942868.0:57572 2014-08-11 17:07:23.493667 2014-08-13 18:06:06.743596 7f5c973a57a0 20 read_log 192888'16593 (192888'16523) modify 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.942868.0:59165 2014-08-11 17:07:23.964641 2014-08-13 18:06:06.743609 7f5c973a57a0 20 read_log 192888'16595 (192888'16527) modify 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.942868.0:61059 2014-08-11 17:07:24.526657 2014-08-13 18:06:06.743622 7f5c973a57a0 20 read_log 193305'16598 (193305'16598) modify 1de/hit_set_24.1de_archive_2014-08-11 18:37:04.915039_2014-08-11 18:42:33.427516/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.427747 2014-08-13 18:06:06.743634 7f5c973a57a0 20 read_log 193305'16599 (192888'16398) delete 1de/hit_set_24.1de_archive_2014-08-11 16:44:55.175331_2014-08-11 17:05:15.139431/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:33.427747 2014-08-13 18:06:06.743647 7f5c973a57a0 20 read_log 193305'16600 (0'0) promote 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by osd.364.0:5 2014-08-11 18:42:33.565205 2014-08-13 18:06:06.743659 7f5c973a57a0 20 read_log 193305'16601 (193305'16600) modify 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.951337.0:130 2014-08-11 18:42:33.420297 2014-08-13 18:06:06.743671 7f5c973a57a0 20 read_log 193305'16602 (0'0) promote b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:24 2014-08-11 18:42:36.888556 2014-08-13 18:06:06.743696 7f5c973a57a0 20 read_log 193305'16603 (193305'16602) modify b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by client.951337.0:976 2014-08-11 18:42:36.880780 2014-08-13 18:06:06.743709 7f5c973a57a0 20 read_log 193305'16604 (0'0) promote 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:28 2014-08-11 18:42:37.125821 2014-08-13 18:06:06.743721 7f5c973a57a0 20 read_log 193305'16605 (193305'16604) modify 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by client.951337.0:1042 2014-08-11 18:42:37.118434 2014-08-13 18:06:06.743733 7f5c973a57a0 20 read_log 193305'16606 (0'0) promote 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by osd.364.0:38 2014-08-11 18:42:39.259338 2014-08-13 18:06:06.743745 7f5c973a57a0 20 read_log 193305'16607 (193305'16606) modify 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.951337.0:1597 2014-08-11 18:42:39.251620 2014-08-13 18:06:06.743768 7f5c973a57a0 20 read_log 193305'16608 (0'0) promote a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by osd.364.0:50 2014-08-11 18:42:40.724812 2014-08-13 18:06:06.743784 7f5c973a57a0 20 read_log 193305'16609 (193305'16608) modify a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.951337.0:1990 2014-08-11 18:42:40.717969 2014-08-13 18:06:06.743796 7f5c973a57a0 20 read_log 193305'16610 (0'0) promote 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by osd.364.0:91 2014-08-11 18:42:56.237810 2014-08-13 18:06:06.743809 7f5c973a57a0 20 read_log 193305'16611 (193305'16610) modify 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.951337.0:6223 2014-08-11 18:42:56.230859 2014-08-13 18:06:06.743821 7f5c973a57a0 20 read_log 193305'16612 (0'0) promote af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:109 2014-08-11 18:43:00.118164 2014-08-13 18:06:06.743833 7f5c973a57a0 20 read_log 193305'16613 (193305'16612) modify af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by client.951337.0:7298 2014-08-11 18:43:00.110326 2014-08-13 18:06:06.743867 7f5c973a57a0 20 read_log 193305'16614 (0'0) promote 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by osd.364.0:111 2014-08-11 18:43:01.064481 2014-08-13 18:06:06.743884 7f5c973a57a0 20 read_log 193305'16615 (193305'16614) modify 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.951337.0:7557 2014-08-11 18:43:01.055850 2014-08-13 18:06:06.743896 7f5c973a57a0 20 read_log 193305'16616 (0'0) promote 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by osd.364.0:159 2014-08-11 18:43:14.321888 2014-08-13 18:06:06.743909 7f5c973a57a0 20 read_log 193305'16617 (193305'16616) modify 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.951337.0:11084 2014-08-11 18:43:14.314494 2014-08-13 18:06:06.743921 7f5c973a57a0 20 read_log 193305'16618 (0'0) promote 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by osd.364.0:183 2014-08-11 18:43:21.326837 2014-08-13 18:06:06.743933 7f5c973a57a0 20 read_log 193305'16619 (193305'16618) modify 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.951337.0:12919 2014-08-11 18:43:21.318193 2014-08-13 18:06:06.743946 7f5c973a57a0 20 read_log 193305'16620 (0'0) promote d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by osd.364.0:185 2014-08-11 18:43:22.042479 2014-08-13 18:06:06.743958 7f5c973a57a0 20 read_log 193305'16621 (193305'16620) modify d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.951337.0:13112 2014-08-11 18:43:22.033021 2014-08-13 18:06:06.743971 7f5c973a57a0 20 read_log 193305'16622 (0'0) promote 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by osd.364.0:191 2014-08-11 18:43:23.621399 2014-08-13 18:06:06.743983 7f5c973a57a0 20 read_log 193305'16623 (193305'16622) modify 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.951337.0:13515 2014-08-11 18:43:23.608926 2014-08-13 18:06:06.743995 7f5c973a57a0 20 read_log 193305'16624 (0'0) promote 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by osd.364.0:209 2014-08-11 18:43:26.828802 2014-08-13 18:06:06.744008 7f5c973a57a0 20 read_log 193305'16625 (193305'16624) modify 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.951337.0:14380 2014-08-11 18:43:26.821989 2014-08-13 18:06:06.744020 7f5c973a57a0 20 read_log 193305'16626 (0'0) promote 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:252 2014-08-11 18:43:39.782750 2014-08-13 18:06:06.744032 7f5c973a57a0 20 read_log 193305'16627 (193305'16626) modify 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by client.951337.0:18118 2014-08-11 18:43:39.773301 2014-08-13 18:06:06.744054 7f5c973a57a0 20 read_log 193305'16628 (0'0) promote 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:256 2014-08-11 18:43:40.896165 2014-08-13 18:06:06.744069 7f5c973a57a0 20 read_log 193305'16629 (193305'16628) modify 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by client.951337.0:18431 2014-08-11 18:43:40.888150 2014-08-13 18:06:06.744082 7f5c973a57a0 20 read_log 193305'16630 (0'0) promote 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by osd.364.0:260 2014-08-11 18:43:40.992499 2014-08-13 18:06:06.744096 7f5c973a57a0 20 read_log 193305'16631 (193305'16630) modify 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.951337.0:18460 2014-08-11 18:43:40.984957 2014-08-13 18:06:06.744109 7f5c973a57a0 20 read_log 193305'16632 (0'0) promote d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:286 2014-08-11 18:43:49.918793 2014-08-13 18:06:06.744123 7f5c973a57a0 20 read_log 193305'16633 (193305'16632) modify d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by client.951337.0:20923 2014-08-11 18:43:49.912169 2014-08-13 18:06:06.744136 7f5c973a57a0 20 read_log 193305'16634 (0'0) promote f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:322 2014-08-11 18:43:58.359432 2014-08-13 18:06:06.744171 7f5c973a57a0 20 read_log 193305'16635 (193305'16634) modify f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by client.951337.0:23264 2014-08-11 18:43:58.351129 2014-08-13 18:06:06.744188 7f5c973a57a0 20 read_log 193305'16636 (0'0) promote 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by osd.364.0:340 2014-08-11 18:44:01.044268 2014-08-13 18:06:06.744201 7f5c973a57a0 20 read_log 193305'16637 (193305'16636) modify 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.951337.0:24004 2014-08-11 18:44:01.034545 2014-08-13 18:06:06.744213 7f5c973a57a0 20 read_log 193305'16638 (0'0) promote 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:344 2014-08-11 18:44:01.637172 2014-08-13 18:06:06.744236 7f5c973a57a0 20 read_log 193305'16639 (193305'16638) modify 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by client.951337.0:24162 2014-08-11 18:44:01.627871 2014-08-13 18:06:06.744254 7f5c973a57a0 20 read_log 193305'16640 (0'0) promote 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by osd.364.0:390 2014-08-11 18:44:09.407700 2014-08-13 18:06:06.744267 7f5c973a57a0 20 read_log 193305'16641 (193305'16640) modify 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.951337.0:26277 2014-08-11 18:44:09.399033 2014-08-13 18:06:06.744298 7f5c973a57a0 20 read_log 193305'16642 (0'0) promote 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:398 2014-08-11 18:44:10.407722 2014-08-13 18:06:06.744314 7f5c973a57a0 20 read_log 193305'16643 (193305'16642) modify 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by client.951337.0:26561 2014-08-11 18:44:10.399675 2014-08-13 18:06:06.744327 7f5c973a57a0 20 read_log 193305'16644 (0'0) promote 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by osd.364.0:404 2014-08-11 18:44:14.032514 2014-08-13 18:06:06.744339 7f5c973a57a0 20 read_log 193305'16645 (193305'16644) modify 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.951337.0:27541 2014-08-11 18:44:14.024928 2014-08-13 18:06:06.744356 7f5c973a57a0 20 read_log 193305'16646 (0'0) promote 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by osd.364.0:406 2014-08-11 18:44:14.583690 2014-08-13 18:06:06.744381 7f5c973a57a0 20 read_log 193305'16647 (193305'16646) modify 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.951337.0:27698 2014-08-11 18:44:14.575030 2014-08-13 18:06:06.744409 7f5c973a57a0 20 read_log 193305'16648 (0'0) promote d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:424 2014-08-11 18:44:17.990246 2014-08-13 18:06:06.744432 7f5c973a57a0 20 read_log 193305'16649 (193305'16648) modify d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by client.951337.0:28642 2014-08-11 18:44:17.982711 2014-08-13 18:06:06.744448 7f5c973a57a0 20 read_log 193305'16650 (0'0) promote e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by osd.364.0:430 2014-08-11 18:44:18.563664 2014-08-13 18:06:06.744461 7f5c973a57a0 20 read_log 193305'16651 (193305'16650) modify e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.951337.0:28786 2014-08-11 18:44:18.553845 2014-08-13 18:06:06.744480 7f5c973a57a0 20 read_log 193305'16652 (0'0) promote b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:442 2014-08-11 18:44:20.594027 2014-08-13 18:06:06.744496 7f5c973a57a0 20 read_log 193305'16653 (193305'16652) modify b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by client.951337.0:29343 2014-08-11 18:44:20.586383 2014-08-13 18:06:06.744508 7f5c973a57a0 20 read_log 193305'16654 (0'0) promote 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by osd.364.0:444 2014-08-11 18:44:20.819641 2014-08-13 18:06:06.744521 7f5c973a57a0 20 read_log 193305'16655 (193305'16654) modify 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.951337.0:29406 2014-08-11 18:44:20.813127 2014-08-13 18:06:06.744533 7f5c973a57a0 20 read_log 193305'16656 (0'0) promote 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by osd.364.0:452 2014-08-11 18:44:22.431884 2014-08-13 18:06:06.744545 7f5c973a57a0 20 read_log 193305'16657 (193305'16656) modify 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.951337.0:29844 2014-08-11 18:44:22.425079 2014-08-13 18:06:06.744559 7f5c973a57a0 20 read_log 193305'16658 (0'0) promote 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by osd.364.0:480 2014-08-11 18:44:28.376804 2014-08-13 18:06:06.744572 7f5c973a57a0 20 read_log 193305'16659 (193305'16658) modify 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.951337.0:31444 2014-08-11 18:44:28.369690 2014-08-13 18:06:06.744594 7f5c973a57a0 20 read_log 193305'16660 (0'0) promote d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by osd.364.0:486 2014-08-11 18:44:30.591503 2014-08-13 18:06:06.744611 7f5c973a57a0 20 read_log 193305'16661 (193305'16660) modify d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.951337.0:32094 2014-08-11 18:44:30.583611 2014-08-13 18:06:06.744623 7f5c973a57a0 20 read_log 193305'16662 (193305'16601) modify 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.951337.0:32841 2014-08-11 18:44:33.898900 2014-08-13 18:06:06.744635 7f5c973a57a0 20 read_log 193305'16663 (193305'16603) modify b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by client.951337.0:33687 2014-08-11 18:44:34.580089 2014-08-13 18:06:06.744648 7f5c973a57a0 20 read_log 193305'16664 (193305'16605) modify 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by client.951337.0:33753 2014-08-11 18:44:34.625191 2014-08-13 18:06:06.744660 7f5c973a57a0 20 read_log 193305'16665 (193305'16607) modify 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.951337.0:34308 2014-08-11 18:44:34.916223 2014-08-13 18:06:06.744672 7f5c973a57a0 20 read_log 193305'16666 (193305'16609) modify a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.951337.0:34701 2014-08-11 18:44:35.142659 2014-08-13 18:06:06.744684 7f5c973a57a0 20 read_log 193305'16667 (193305'16611) modify 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.951337.0:38934 2014-08-11 18:44:37.741047 2014-08-13 18:06:06.744720 7f5c973a57a0 20 read_log 193305'16668 (193305'16613) modify af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by client.951337.0:40009 2014-08-11 18:44:38.260123 2014-08-13 18:06:06.744736 7f5c973a57a0 20 read_log 193305'16669 (193305'16615) modify 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.951337.0:40268 2014-08-11 18:44:38.355307 2014-08-13 18:06:06.744749 7f5c973a57a0 20 read_log 193305'16670 (193305'16617) modify 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.951337.0:43795 2014-08-11 18:44:39.700289 2014-08-13 18:06:06.744761 7f5c973a57a0 20 read_log 193305'16671 (193305'16619) modify 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.951337.0:45630 2014-08-11 18:44:40.255101 2014-08-13 18:06:06.744774 7f5c973a57a0 20 read_log 193305'16672 (193305'16621) modify d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.951337.0:45823 2014-08-11 18:44:40.312112 2014-08-13 18:06:06.744786 7f5c973a57a0 20 read_log 193305'16673 (193305'16623) modify 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.951337.0:46226 2014-08-11 18:44:40.424384 2014-08-13 18:06:06.744798 7f5c973a57a0 20 read_log 193305'16674 (193305'16625) modify 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.951337.0:47091 2014-08-11 18:44:40.671485 2014-08-13 18:06:06.744810 7f5c973a57a0 20 read_log 193305'16675 (193305'16627) modify 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by client.951337.0:50829 2014-08-11 18:44:41.773591 2014-08-13 18:06:06.744822 7f5c973a57a0 20 read_log 193305'16676 (193305'16629) modify 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by client.951337.0:51142 2014-08-11 18:44:41.865926 2014-08-13 18:06:06.744834 7f5c973a57a0 20 read_log 193305'16677 (193305'16631) modify 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.951337.0:51171 2014-08-11 18:44:41.873828 2014-08-13 18:06:06.744846 7f5c973a57a0 20 read_log 193305'16678 (193305'16633) modify d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by client.951337.0:53634 2014-08-11 18:44:42.587204 2014-08-13 18:06:06.744858 7f5c973a57a0 20 read_log 193305'16679 (193305'16635) modify f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by client.951337.0:55975 2014-08-11 18:44:43.283626 2014-08-13 18:06:06.744870 7f5c973a57a0 20 read_log 193305'16680 (193305'16637) modify 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.951337.0:56715 2014-08-11 18:44:43.507995 2014-08-13 18:06:06.744882 7f5c973a57a0 20 read_log 193305'16681 (193305'16639) modify 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by client.951337.0:56873 2014-08-11 18:44:43.558522 2014-08-13 18:06:06.744896 7f5c973a57a0 20 read_log 193305'16682 (193305'16641) modify 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.951337.0:58988 2014-08-11 18:44:44.204622 2014-08-13 18:06:06.744909 7f5c973a57a0 20 read_log 193305'16683 (193305'16643) modify 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by client.951337.0:59272 2014-08-11 18:44:44.278097 2014-08-13 18:06:06.744921 7f5c973a57a0 20 read_log 193305'16684 (193305'16645) modify 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.951337.0:60252 2014-08-11 18:44:44.573356 2014-08-13 18:06:06.744933 7f5c973a57a0 20 read_log 193305'16685 (193305'16647) modify 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.951337.0:60409 2014-08-11 18:44:44.622451 2014-08-13 18:06:06.744947 7f5c973a57a0 20 read_log 193305'16686 (193305'16649) modify d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by client.951337.0:61353 2014-08-11 18:44:44.900210 2014-08-13 18:06:06.744960 7f5c973a57a0 20 read_log 193305'16687 (193305'16651) modify e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.951337.0:61497 2014-08-11 18:44:44.938268 2014-08-13 18:06:06.744984 7f5c973a57a0 20 read_log 193305'16688 (193305'16653) modify b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by client.951337.0:62054 2014-08-11 18:44:45.114203 2014-08-13 18:06:06.744999 7f5c973a57a0 20 read_log 193305'16689 (193305'16655) modify 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.951337.0:62117 2014-08-11 18:44:45.130191 2014-08-13 18:06:06.745012 7f5c973a57a0 20 read_log 193305'16690 (193305'16657) modify 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.951337.0:62555 2014-08-11 18:44:45.261835 2014-08-13 18:06:06.745026 7f5c973a57a0 20 read_log 193305'16691 (193305'16659) modify 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.951337.0:64155 2014-08-11 18:44:45.721145 2014-08-13 18:06:06.745039 7f5c973a57a0 20 read_log 193305'16692 (193305'16661) modify d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.951337.0:64805 2014-08-11 18:44:45.924653 2014-08-13 18:06:06.745053 7f5c973a57a0 20 read_log 194209'16693 (192834'16164) clean b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.364.0:1161 2014-08-12 14:06:37.171341 2014-08-13 18:06:06.745066 7f5c973a57a0 20 read_log 194209'16694 (192888'16587) clean 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:1162 2014-08-12 14:06:37.172011 2014-08-13 18:06:06.745078 7f5c973a57a0 20 read_log 194209'16695 (192888'16556) clean 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:1163 2014-08-12 14:06:37.172433 2014-08-13 18:06:06.745090 7f5c973a57a0 20 read_log 194209'16696 (192834'15677) clean 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.364.0:1164 2014-08-12 14:06:37.172924 2014-08-13 18:06:06.745104 7f5c973a57a0 20 read_log 194209'16697 (192888'16562) clean 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:1165 2014-08-12 14:06:37.183262 2014-08-13 18:06:06.745119 7f5c973a57a0 20 read_log 194209'16698 (192834'15619) clean 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.364.0:1166 2014-08-12 14:06:37.183548 2014-08-13 18:06:06.745136 7f5c973a57a0 20 read_log 194209'16699 (192834'15673) clean 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.364.0:1167 2014-08-12 14:06:37.186788 2014-08-13 18:06:06.745155 7f5c973a57a0 20 read_log 194209'16700 (193305'16668) clean af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:1168 2014-08-12 14:06:37.190384 2014-08-13 18:06:06.745170 7f5c973a57a0 20 read_log 194209'16701 (192888'16390) clean c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:1169 2014-08-12 14:06:37.193657 2014-08-13 18:06:06.745182 7f5c973a57a0 20 read_log 194209'16702 (192888'16386) clean 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:1170 2014-08-12 14:06:37.196992 2014-08-13 18:06:06.745195 7f5c973a57a0 20 read_log 194209'16703 (192888'16583) clean a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:1171 2014-08-12 14:06:37.200305 2014-08-13 18:06:06.745209 7f5c973a57a0 20 read_log 194209'16704 (192834'16131) clean 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.364.0:1172 2014-08-12 14:06:37.206235 2014-08-13 18:06:06.745221 7f5c973a57a0 20 read_log 194209'16705 (192834'15587) clean 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.364.0:1173 2014-08-12 14:06:37.206514 2014-08-13 18:06:06.745235 7f5c973a57a0 20 read_log 194209'16706 (192834'16146) clean 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.364.0:1174 2014-08-12 14:06:37.209720 2014-08-13 18:06:06.745258 7f5c973a57a0 20 read_log 194209'16707 (193305'16679) clean f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:1175 2014-08-12 14:06:37.213057 2014-08-13 18:06:06.745273 7f5c973a57a0 20 read_log 194209'16708 (192888'16367) clean bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:1176 2014-08-12 14:06:37.216381 2014-08-13 18:06:06.745286 7f5c973a57a0 20 read_log 194209'16709 (192888'16352) clean 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:1177 2014-08-12 14:06:37.219708 2014-08-13 18:06:06.745302 7f5c973a57a0 20 read_log 194209'16710 (193305'16663) clean b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:1178 2014-08-12 14:06:37.223069 2014-08-13 18:06:06.745314 7f5c973a57a0 20 read_log 194209'16711 (193305'16676) clean 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:1179 2014-08-12 14:06:37.226308 2014-08-13 18:06:06.745326 7f5c973a57a0 20 read_log 194209'16712 (193305'16688) clean b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:1180 2014-08-12 14:06:37.229514 2014-08-13 18:06:06.745338 7f5c973a57a0 20 read_log 194209'16713 (192834'16191) clean c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.364.0:1181 2014-08-12 14:06:37.235306 2014-08-13 18:06:06.745354 7f5c973a57a0 20 read_log 194209'16714 (192888'16586) clean 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:1182 2014-08-12 14:06:37.235818 2014-08-13 18:06:06.745367 7f5c973a57a0 20 read_log 194209'16715 (193305'16664) clean 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:1183 2014-08-12 14:06:37.239009 2014-08-13 18:06:06.745392 7f5c973a57a0 20 read_log 194209'16716 (192888'16382) clean 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:1184 2014-08-12 14:06:37.242159 2014-08-13 18:06:06.745410 7f5c973a57a0 20 read_log 194209'16717 (192834'16207) clean 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.364.0:1185 2014-08-12 14:06:37.245363 2014-08-13 18:06:06.745430 7f5c973a57a0 20 read_log 194209'16718 (193305'16686) clean d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:1186 2014-08-12 14:06:37.248618 2014-08-13 18:06:06.745446 7f5c973a57a0 20 read_log 194209'16719 (192834'16226) clean dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.364.0:1187 2014-08-12 14:06:37.251854 2014-08-13 18:06:06.745458 7f5c973a57a0 20 read_log 194209'16720 (192834'15627) clean e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.364.0:1188 2014-08-12 14:06:37.257606 2014-08-13 18:06:06.745480 7f5c973a57a0 20 read_log 194209'16721 (192834'15700) clean 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.364.0:1189 2014-08-12 14:06:37.257876 2014-08-13 18:06:06.745496 7f5c973a57a0 20 read_log 194209'16722 (192888'16559) clean b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:1190 2014-08-12 14:06:37.261061 2014-08-13 18:06:06.745509 7f5c973a57a0 20 read_log 194209'16723 (192834'16092) clean 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.364.0:1191 2014-08-12 14:06:37.267324 2014-08-13 18:06:06.745524 7f5c973a57a0 20 read_log 194209'16724 (193305'16675) clean 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:1192 2014-08-12 14:06:37.267571 2014-08-13 18:06:06.745536 7f5c973a57a0 20 read_log 194209'16725 (192888'16356) clean 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:1193 2014-08-12 14:06:37.270763 2014-08-13 18:06:06.745549 7f5c973a57a0 20 read_log 194209'16726 (192834'16143) clean 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.364.0:1194 2014-08-12 14:06:37.274031 2014-08-13 18:06:06.745581 7f5c973a57a0 20 read_log 194209'16727 (192888'16575) clean 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:1195 2014-08-12 14:06:37.277331 2014-08-13 18:06:06.745599 7f5c973a57a0 20 read_log 194209'16728 (193305'16678) clean d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:1196 2014-08-12 14:06:37.280497 2014-08-13 18:06:06.745611 7f5c973a57a0 20 read_log 194209'16729 (193305'16681) clean 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:1197 2014-08-12 14:06:37.283796 2014-08-13 18:06:06.745625 7f5c973a57a0 20 read_log 194209'16730 (192834'15710) clean ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.364.0:1198 2014-08-12 14:06:37.287339 2014-08-13 18:06:06.745637 7f5c973a57a0 20 read_log 194209'16731 (193305'16683) clean 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:1199 2014-08-12 14:06:37.293105 2014-08-13 18:06:06.745649 7f5c973a57a0 20 read_log 194209'16732 (192834'15669) clean 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.364.0:1200 2014-08-12 14:06:37.296336 2014-08-13 18:06:06.745662 7f5c973a57a0 20 read_log 194209'16733 (192888'16535) clean 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:1201 2014-08-12 14:06:37.296646 2014-08-13 18:06:06.745674 7f5c973a57a0 20 read_log 194209'16734 (192888'16354) clean 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:1202 2014-08-12 14:06:37.300218 2014-08-13 18:06:06.745711 7f5c973a57a0 20 read_log 194209'16735 (192834'15664) clean 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.364.0:1203 2014-08-12 14:06:37.303917 2014-08-13 18:06:06.745726 7f5c973a57a0 20 read_log 194209'16736 (192888'16567) clean 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:1204 2014-08-12 14:06:37.307110 2014-08-13 18:06:06.745738 7f5c973a57a0 20 read_log 194209'16737 (192888'16391) clean 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:1205 2014-08-12 14:06:37.310394 2014-08-13 18:06:06.745751 7f5c973a57a0 20 read_log 194209'16738 (194209'16693) delete b39d35de/benchmark_data_storage0111-ib.pocstor_41526_object14941/head//24 by osd.364.0:1206 2014-08-12 14:06:37.343051 2014-08-13 18:06:06.745767 7f5c973a57a0 20 read_log 194209'16739 (194209'16698) delete 23f365de/benchmark_data_storage0111-ib.pocstor_26659_object5352/head//24 by osd.364.0:1207 2014-08-12 14:06:37.343934 2014-08-13 18:06:06.745780 7f5c973a57a0 20 read_log 194209'16740 (194209'16699) delete 60e765de/benchmark_data_storage0111-ib.pocstor_26659_object14087/head//24 by osd.364.0:1208 2014-08-12 14:06:37.347381 2014-08-13 18:06:06.745792 7f5c973a57a0 20 read_log 194209'16741 (194209'16700) delete af9b65de/benchmark_data_storage0111-ib.pocstor_62621_object7297/head//24 by osd.364.0:1209 2014-08-12 14:06:37.347988 2014-08-13 18:06:06.745804 7f5c973a57a0 20 read_log 194209'16742 (194209'16701) delete c40475de/benchmark_data_storage0111-ib.pocstor_41097_object25383/head//24 by osd.364.0:1210 2014-08-12 14:06:37.351416 2014-08-13 18:06:06.745817 7f5c973a57a0 20 read_log 194209'16743 (194209'16702) delete 5bd75de/benchmark_data_storage0111-ib.pocstor_41097_object23190/head//24 by osd.364.0:1211 2014-08-12 14:06:37.351777 2014-08-13 18:06:06.745829 7f5c973a57a0 20 read_log 194209'16744 (192834'16128) clean 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.364.0:1212 2014-08-12 14:06:37.352008 2014-08-13 18:06:06.745841 7f5c973a57a0 20 read_log 194209'16745 (194209'16704) delete 5ba185de/benchmark_data_storage0111-ib.pocstor_41526_object10009/head//24 by osd.364.0:1213 2014-08-12 14:06:37.370454 2014-08-13 18:06:06.745853 7f5c973a57a0 20 read_log 194209'16746 (194209'16705) delete 1f4485de/benchmark_data_storage0111-ib.pocstor_26659_object1852/head//24 by osd.364.0:1214 2014-08-12 14:06:37.371196 2014-08-13 18:06:06.745877 7f5c973a57a0 20 read_log 194209'16747 (194209'16706) delete 6c5b85de/benchmark_data_storage0111-ib.pocstor_41526_object11902/head//24 by osd.364.0:1215 2014-08-12 14:06:37.374025 2014-08-13 18:06:06.745890 7f5c973a57a0 20 read_log 194209'16748 (194209'16707) delete f2cb85de/benchmark_data_storage0111-ib.pocstor_62621_object23263/head//24 by osd.364.0:1216 2014-08-12 14:06:37.374401 2014-08-13 18:06:06.745903 7f5c973a57a0 20 read_log 194209'16749 (194209'16708) delete bb14a5de/benchmark_data_storage0111-ib.pocstor_41097_object13014/head//24 by osd.364.0:1217 2014-08-12 14:06:37.375255 2014-08-13 18:06:06.745916 7f5c973a57a0 20 read_log 194209'16750 (194209'16709) delete 27d5a5de/benchmark_data_storage0111-ib.pocstor_41097_object5089/head//24 by osd.364.0:1218 2014-08-12 14:06:37.378531 2014-08-13 18:06:06.745935 7f5c973a57a0 20 read_log 194209'16751 (194209'16710) delete b946a5de/benchmark_data_storage0111-ib.pocstor_62621_object975/head//24 by osd.364.0:1219 2014-08-12 14:06:37.378953 2014-08-13 18:06:06.745958 7f5c973a57a0 20 read_log 194209'16752 (194209'16711) delete 9537a5de/benchmark_data_storage0111-ib.pocstor_62621_object18430/head//24 by osd.364.0:1220 2014-08-12 14:06:37.379584 2014-08-13 18:06:06.745971 7f5c973a57a0 20 read_log 194209'16753 (194209'16712) delete b2a1b5de/benchmark_data_storage0111-ib.pocstor_62621_object29342/head//24 by osd.364.0:1221 2014-08-12 14:06:37.406219 2014-08-13 18:06:06.745984 7f5c973a57a0 20 read_log 194209'16754 (194209'16713) delete c012b5de/benchmark_data_storage0111-ib.pocstor_41526_object18211/head//24 by osd.364.0:1222 2014-08-12 14:06:37.406939 2014-08-13 18:06:06.745996 7f5c973a57a0 20 read_log 194209'16755 (194209'16715) delete 7223d5de/benchmark_data_storage0111-ib.pocstor_62621_object1041/head//24 by osd.364.0:1223 2014-08-12 14:06:37.407381 2014-08-13 18:06:06.746009 7f5c973a57a0 20 read_log 194209'16756 (194209'16716) delete 7ee4d5de/benchmark_data_storage0111-ib.pocstor_41097_object21721/head//24 by osd.364.0:1224 2014-08-12 14:06:37.411157 2014-08-13 18:06:06.746023 7f5c973a57a0 20 read_log 194209'16757 (194209'16718) delete d0dbe5de/benchmark_data_storage0111-ib.pocstor_62621_object28641/head//24 by osd.364.0:1225 2014-08-12 14:06:37.411905 2014-08-13 18:06:06.746036 7f5c973a57a0 20 read_log 194209'16758 (194209'16719) delete dd5ce5de/benchmark_data_storage0111-ib.pocstor_41526_object22770/head//24 by osd.364.0:1226 2014-08-12 14:06:37.412222 2014-08-13 18:06:06.746048 7f5c973a57a0 20 read_log 194209'16759 (194209'16720) delete e13ee5de/benchmark_data_storage0111-ib.pocstor_26659_object6604/head//24 by osd.364.0:1227 2014-08-12 14:06:37.415587 2014-08-13 18:06:06.746064 7f5c973a57a0 20 read_log 194209'16760 (194209'16721) delete 6e4ee5de/benchmark_data_storage0111-ib.pocstor_26659_object17524/head//24 by osd.364.0:1228 2014-08-12 14:06:37.415907 2014-08-13 18:06:06.746077 7f5c973a57a0 20 read_log 194209'16761 (194209'16723) delete 38baf5de/benchmark_data_storage0111-ib.pocstor_41526_object5113/head//24 by osd.364.0:1229 2014-08-12 14:06:37.416600 2014-08-13 18:06:06.746089 7f5c973a57a0 20 read_log 194209'16762 (194209'16724) delete 188cf5de/benchmark_data_storage0111-ib.pocstor_62621_object18117/head//24 by osd.364.0:1230 2014-08-12 14:06:37.419609 2014-08-13 18:06:06.746101 7f5c973a57a0 20 read_log 194209'16763 (194209'16725) delete 424009de/benchmark_data_storage0111-ib.pocstor_41097_object5790/head//24 by osd.364.0:1231 2014-08-12 14:06:37.419916 2014-08-13 18:06:06.746117 7f5c973a57a0 20 read_log 194209'16764 (194209'16726) delete 5a0109de/benchmark_data_storage0111-ib.pocstor_41526_object11807/head//24 by osd.364.0:1232 2014-08-12 14:06:37.420789 2014-08-13 18:06:06.746129 7f5c973a57a0 20 read_log 194209'16765 (194209'16728) delete d0ad09de/benchmark_data_storage0111-ib.pocstor_62621_object20922/head//24 by osd.364.0:1233 2014-08-12 14:06:37.423854 2014-08-13 18:06:06.746155 7f5c973a57a0 20 read_log 194209'16766 (194209'16729) delete 6cd919de/benchmark_data_storage0111-ib.pocstor_62621_object24161/head//24 by osd.364.0:1234 2014-08-12 14:06:37.424157 2014-08-13 18:06:06.746185 7f5c973a57a0 20 read_log 194209'16767 (194209'16730) delete ec129de/benchmark_data_storage0111-ib.pocstor_26659_object19925/head//24 by osd.364.0:1235 2014-08-12 14:06:37.424892 2014-08-13 18:06:06.746203 7f5c973a57a0 20 read_log 194209'16768 (194209'16731) delete 60cb29de/benchmark_data_storage0111-ib.pocstor_62621_object26560/head//24 by osd.364.0:1236 2014-08-12 14:06:37.428018 2014-08-13 18:06:06.746216 7f5c973a57a0 20 read_log 194209'16769 (194209'16732) delete 605d29de/benchmark_data_storage0111-ib.pocstor_26659_object13531/head//24 by osd.364.0:1237 2014-08-12 14:06:37.428338 2014-08-13 18:06:06.746235 7f5c973a57a0 20 read_log 194209'16770 (194209'16734) delete 753e39de/benchmark_data_storage0111-ib.pocstor_41097_object5408/head//24 by osd.364.0:1238 2014-08-12 14:06:37.429069 2014-08-13 18:06:06.746252 7f5c973a57a0 20 read_log 194209'16771 (194209'16735) delete 775349de/benchmark_data_storage0111-ib.pocstor_26659_object13103/head//24 by osd.364.0:1239 2014-08-12 14:06:37.432132 2014-08-13 18:06:06.746265 7f5c973a57a0 20 read_log 194209'16772 (194209'16737) delete 186269de/benchmark_data_storage0111-ib.pocstor_41097_object25389/head//24 by osd.364.0:1240 2014-08-12 14:06:37.432537 2014-08-13 18:06:06.746282 7f5c973a57a0 20 read_log 194209'16773 (194209'16744) delete 50e669de/benchmark_data_storage0111-ib.pocstor_41526_object9642/head//24 by osd.364.0:1241 2014-08-12 14:06:37.432751 2014-08-13 18:06:06.746295 7f5c973a57a0 20 read_log 194209'16774 (194209'16722) delete b414f5de/benchmark_data_storage0111-ib.pocstor_2228_object10852/head//24 by osd.364.0:1583 2014-08-12 14:08:53.345943 2014-08-13 18:06:06.746307 7f5c973a57a0 20 read_log 194209'16775 (194209'16727) delete 980609de/benchmark_data_storage0111-ib.pocstor_2228_object18838/head//24 by osd.364.0:1584 2014-08-12 14:08:53.346392 2014-08-13 18:06:06.746323 7f5c973a57a0 20 read_log 194209'16776 (194209'16733) delete 5639de/benchmark_data_storage0111-ib.pocstor_2228_object961/head//24 by osd.364.0:1585 2014-08-12 14:08:53.346829 2014-08-13 18:06:06.746335 7f5c973a57a0 20 read_log 194209'16777 (194209'16736) delete 44ad59de/benchmark_data_storage0111-ib.pocstor_2228_object15594/head//24 by osd.364.0:1586 2014-08-12 14:08:53.347190 2014-08-13 18:06:06.746347 7f5c973a57a0 20 read_log 194209'16778 (194209'16694) delete 6d4445de/benchmark_data_storage0111-ib.pocstor_2228_object24386/head//24 by osd.364.0:1587 2014-08-12 14:08:53.377509 2014-08-13 18:06:06.746360 7f5c973a57a0 20 read_log 194209'16779 (194209'16695) delete 181b45de/benchmark_data_storage0111-ib.pocstor_2228_object8612/head//24 by osd.364.0:1588 2014-08-12 14:08:53.377867 2014-08-13 18:06:06.746372 7f5c973a57a0 20 read_log 194209'16780 (194209'16696) delete 111955de/benchmark_data_storage0111-ib.pocstor_26659_object14532/head//24 by osd.364.0:1589 2014-08-12 14:08:53.378234 2014-08-13 18:06:06.746384 7f5c973a57a0 20 read_log 194209'16781 (194209'16697) delete 4ccc55de/benchmark_data_storage0111-ib.pocstor_2228_object12132/head//24 by osd.364.0:1590 2014-08-12 14:08:53.378943 2014-08-13 18:06:06.746407 7f5c973a57a0 20 read_log 194209'16782 (194209'16703) delete a93e75de/benchmark_data_storage0111-ib.pocstor_2228_object22281/head//24 by osd.364.0:1591 2014-08-12 14:08:53.381510 2014-08-13 18:06:06.746420 7f5c973a57a0 20 read_log 194209'16783 (194209'16714) delete 4549b5de/benchmark_data_storage0111-ib.pocstor_2228_object23889/head//24 by osd.364.0:1592 2014-08-12 14:08:53.382410 2014-08-13 18:06:06.746432 7f5c973a57a0 20 read_log 194209'16784 (194209'16717) delete 9258d5de/benchmark_data_storage0111-ib.pocstor_41526_object20662/head//24 by osd.364.0:1593 2014-08-12 14:08:53.382739 2014-08-13 18:06:06.746445 7f5c973a57a0 20 read_log 194215'16785 (194215'16785) modify 1de/hit_set_24.1de_archive_2014-08-12 15:32:25.736820_2014-08-12 15:43:36.831465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:36.831705 2014-08-13 18:06:06.746466 7f5c973a57a0 20 read_log 194215'16786 (193305'16598) delete 1de/hit_set_24.1de_archive_2014-08-11 18:37:04.915039_2014-08-11 18:42:33.427516/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:36.831705 2014-08-13 18:06:06.746481 7f5c973a57a0 20 read_log 194215'16787 (192834'15607) clean 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by osd.364.0:2385 2014-08-12 15:43:36.834694 2014-08-13 18:06:06.746494 7f5c973a57a0 20 read_log 194215'16788 (194215'16787) delete 4b1101de/benchmark_data_storage0111-ib.pocstor_26659_object4248/head//24 by client.2174595.0:98983 0.000000 2014-08-13 18:06:06.746505 7f5c973a57a0 20 read_log 194215'16789 (192888'16571) clean a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by osd.364.0:2387 2014-08-12 15:43:36.852620 2014-08-13 18:06:06.746517 7f5c973a57a0 20 read_log 194215'16790 (194215'16789) delete a7a01de/benchmark_data_storage0111-ib.pocstor_2228_object18215/head//24 by client.2174595.0:98985 0.000000 2014-08-13 18:06:06.746528 7f5c973a57a0 20 read_log 194215'16791 (192834'16210) clean bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by osd.364.0:2389 2014-08-12 15:43:36.861140 2014-08-13 18:06:06.746541 7f5c973a57a0 20 read_log 194215'16792 (194215'16791) delete bedd11de/benchmark_data_storage0111-ib.pocstor_41526_object20984/head//24 by client.2174595.0:98987 0.000000 2014-08-13 18:06:06.746552 7f5c973a57a0 20 read_log 194215'16793 (192888'16361) clean 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by osd.364.0:2391 2014-08-12 15:43:36.867906 2014-08-13 18:06:06.746564 7f5c973a57a0 20 read_log 194215'16794 (194215'16793) delete 43f11de/benchmark_data_storage0111-ib.pocstor_41097_object9969/head//24 by client.2174595.0:98989 0.000000 2014-08-13 18:06:06.746575 7f5c973a57a0 20 read_log 194215'16795 (193305'16673) clean 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by osd.364.0:2393 2014-08-12 15:43:36.874370 2014-08-13 18:06:06.746587 7f5c973a57a0 20 read_log 194215'16796 (194215'16795) delete 8b8621de/benchmark_data_storage0111-ib.pocstor_62621_object13514/head//24 by client.2174595.0:98991 0.000000 2014-08-13 18:06:06.746597 7f5c973a57a0 20 read_log 194215'16797 (192888'16591) clean 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by osd.364.0:2395 2014-08-12 15:43:36.882843 2014-08-13 18:06:06.746610 7f5c973a57a0 20 read_log 194215'16798 (194215'16797) delete 271731de/benchmark_data_storage0111-ib.pocstor_2228_object26305/head//24 by client.2174595.0:98993 0.000000 2014-08-13 18:06:06.746620 7f5c973a57a0 20 read_log 194215'16799 (192834'16100) clean 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by osd.364.0:2397 2014-08-12 15:43:36.889168 2014-08-13 18:06:06.746633 7f5c973a57a0 20 read_log 194215'16800 (194215'16799) delete 61a541de/benchmark_data_storage0111-ib.pocstor_41526_object6120/head//24 by client.2174595.0:98995 0.000000 2014-08-13 18:06:06.746643 7f5c973a57a0 20 read_log 194215'16801 (192888'16569) clean 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by osd.364.0:2399 2014-08-12 15:43:36.895786 2014-08-13 18:06:06.746656 7f5c973a57a0 20 read_log 194215'16802 (194215'16801) delete 793b41de/benchmark_data_storage0111-ib.pocstor_2228_object17026/head//24 by client.2174595.0:98997 0.000000 2014-08-13 18:06:06.746666 7f5c973a57a0 20 read_log 194215'16803 (193305'16682) clean 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by osd.364.0:2401 2014-08-12 15:43:36.902434 2014-08-13 18:06:06.746679 7f5c973a57a0 20 read_log 194215'16804 (194215'16803) delete 215561de/benchmark_data_storage0111-ib.pocstor_62621_object26276/head//24 by client.2174595.0:98999 0.000000 2014-08-13 18:06:06.746719 7f5c973a57a0 20 read_log 194215'16805 (192888'16570) clean c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by osd.364.0:2403 2014-08-12 15:43:36.908718 2014-08-13 18:06:06.746735 7f5c973a57a0 20 read_log 194215'16806 (194215'16805) delete c87961de/benchmark_data_storage0111-ib.pocstor_2228_object17653/head//24 by client.2174595.0:99001 0.000000 2014-08-13 18:06:06.746747 7f5c973a57a0 20 read_log 194215'16807 (192888'16395) clean 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by osd.364.0:2405 2014-08-12 15:43:36.915065 2014-08-13 18:06:06.746759 7f5c973a57a0 20 read_log 194215'16808 (194215'16807) delete 1ad481de/benchmark_data_storage0111-ib.pocstor_41097_object30140/head//24 by client.2174595.0:99003 0.000000 2014-08-13 18:06:06.746770 7f5c973a57a0 20 read_log 194215'16809 (193305'16667) clean 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by osd.364.0:2407 2014-08-12 15:43:36.921107 2014-08-13 18:06:06.746784 7f5c973a57a0 20 read_log 194215'16810 (194215'16809) delete 41491de/benchmark_data_storage0111-ib.pocstor_62621_object6222/head//24 by client.2174595.0:99005 0.000000 2014-08-13 18:06:06.746795 7f5c973a57a0 20 read_log 194215'16811 (192888'16368) clean 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by osd.364.0:2409 2014-08-12 15:43:36.927084 2014-08-13 18:06:06.746808 7f5c973a57a0 20 read_log 194215'16812 (194215'16811) delete 732791de/benchmark_data_storage0111-ib.pocstor_41097_object13449/head//24 by client.2174595.0:99007 0.000000 2014-08-13 18:06:06.746821 7f5c973a57a0 20 read_log 194215'16813 (193305'16670) clean 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by osd.364.0:2411 2014-08-12 15:43:36.933348 2014-08-13 18:06:06.746833 7f5c973a57a0 20 read_log 194215'16814 (194215'16813) delete 645b91de/benchmark_data_storage0111-ib.pocstor_62621_object11083/head//24 by client.2174595.0:99009 0.000000 2014-08-13 18:06:06.746844 7f5c973a57a0 20 read_log 194215'16815 (192888'16353) clean 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by osd.364.0:2413 2014-08-12 15:43:36.939717 2014-08-13 18:06:06.746857 7f5c973a57a0 20 read_log 194215'16816 (194215'16815) delete 40ce91de/benchmark_data_storage0111-ib.pocstor_41097_object5273/head//24 by client.2174595.0:99011 0.000000 2014-08-13 18:06:06.746868 7f5c973a57a0 20 read_log 194215'16817 (192834'15660) clean 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by osd.364.0:2415 2014-08-12 15:43:36.945734 2014-08-13 18:06:06.746883 7f5c973a57a0 20 read_log 194215'16818 (194215'16817) delete 9d4ea1de/benchmark_data_storage0111-ib.pocstor_26659_object11867/head//24 by client.2174595.0:99013 0.000000 2014-08-13 18:06:06.746894 7f5c973a57a0 20 read_log 194215'16819 (192834'15604) clean 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by osd.364.0:2417 2014-08-12 15:43:36.952111 2014-08-13 18:06:06.746907 7f5c973a57a0 20 read_log 194215'16820 (194215'16819) delete 8b2fa1de/benchmark_data_storage0111-ib.pocstor_26659_object3714/head//24 by client.2174595.0:99015 0.000000 2014-08-13 18:06:06.746918 7f5c973a57a0 20 read_log 194215'16821 (192888'16547) clean 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by osd.364.0:2419 2014-08-12 15:43:36.958420 2014-08-13 18:06:06.746933 7f5c973a57a0 20 read_log 194215'16822 (194215'16821) delete 561fb1de/benchmark_data_storage0111-ib.pocstor_2228_object4912/head//24 by client.2174595.0:99017 0.000000 2014-08-13 18:06:06.746944 7f5c973a57a0 20 read_log 194215'16823 (192834'16217) clean f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by osd.364.0:2421 2014-08-12 15:43:36.964318 2014-08-13 18:06:06.746956 7f5c973a57a0 20 read_log 194215'16824 (194215'16823) delete f38cd1de/benchmark_data_storage0111-ib.pocstor_41526_object21730/head//24 by client.2174595.0:99019 0.000000 2014-08-13 18:06:06.746977 7f5c973a57a0 20 read_log 194215'16825 (192888'16589) clean 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by osd.364.0:2423 2014-08-12 15:43:36.970492 2014-08-13 18:06:06.746991 7f5c973a57a0 20 read_log 194215'16826 (194215'16825) delete 45a2e1de/benchmark_data_storage0111-ib.pocstor_2228_object24696/head//24 by client.2174595.0:99021 0.000000 2014-08-13 18:06:06.747003 7f5c973a57a0 20 read_log 194215'16827 (192834'15644) clean d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by osd.364.0:2425 2014-08-12 15:43:36.976413 2014-08-13 18:06:06.747030 7f5c973a57a0 20 read_log 194215'16828 (194215'16827) delete d5e3e1de/benchmark_data_storage0111-ib.pocstor_26659_object8850/head//24 by client.2174595.0:99023 0.000000 2014-08-13 18:06:06.747044 7f5c973a57a0 20 read_log 194215'16829 (192834'15686) clean 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by osd.364.0:2427 2014-08-12 15:43:36.982404 2014-08-13 18:06:06.747056 7f5c973a57a0 20 read_log 194215'16830 (194215'16829) delete 536405de/benchmark_data_storage0111-ib.pocstor_26659_object16028/head//24 by client.2174595.0:99025 0.000000 2014-08-13 18:06:06.747067 7f5c973a57a0 20 read_log 194215'16831 (192834'16116) clean b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by osd.364.0:2429 2014-08-12 15:43:36.989617 2014-08-13 18:06:06.747082 7f5c973a57a0 20 read_log 194215'16832 (194215'16831) delete b31a05de/benchmark_data_storage0111-ib.pocstor_41526_object7877/head//24 by client.2174595.0:99027 0.000000 2014-08-13 18:06:06.747093 7f5c973a57a0 20 read_log 194215'16833 (192834'16086) clean 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by osd.364.0:2431 2014-08-12 15:43:36.997175 2014-08-13 18:06:06.747107 7f5c973a57a0 20 read_log 194215'16834 (194215'16833) delete 2cfc05de/benchmark_data_storage0111-ib.pocstor_41526_object4379/head//24 by client.2174595.0:99029 0.000000 2014-08-13 18:06:06.747118 7f5c973a57a0 20 read_log 194215'16835 (192834'15583) clean b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by osd.364.0:2433 2014-08-12 15:43:37.003248 2014-08-13 18:06:06.747131 7f5c973a57a0 20 read_log 194215'16836 (194215'16835) delete b7ad05de/benchmark_data_storage0111-ib.pocstor_26659_object1357/head//24 by client.2174595.0:99031 0.000000 2014-08-13 18:06:06.747142 7f5c973a57a0 20 read_log 194215'16837 (193305'16680) clean 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by osd.364.0:2435 2014-08-12 15:43:37.009512 2014-08-13 18:06:06.747162 7f5c973a57a0 20 read_log 194215'16838 (194215'16837) delete 2615de/benchmark_data_storage0111-ib.pocstor_62621_object24003/head//24 by client.2174595.0:99033 0.000000 2014-08-13 18:06:06.747173 7f5c973a57a0 20 read_log 194215'16839 (192888'16578) clean 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by osd.364.0:2437 2014-08-12 15:43:37.026721 2014-08-13 18:06:06.747186 7f5c973a57a0 20 read_log 194215'16840 (194215'16839) delete 5e5915de/benchmark_data_storage0111-ib.pocstor_2228_object20290/head//24 by client.2174595.0:99035 0.000000 2014-08-13 18:06:06.747197 7f5c973a57a0 20 read_log 194215'16841 (192888'16344) clean 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by osd.364.0:2439 2014-08-12 15:43:37.050496 2014-08-13 18:06:06.747211 7f5c973a57a0 20 read_log 194215'16842 (194215'16841) delete 2e6e15de/benchmark_data_storage0111-ib.pocstor_41097_object1861/head//24 by client.2174595.0:99037 0.000000 2014-08-13 18:06:06.747225 7f5c973a57a0 20 read_log 194215'16843 (192834'15625) clean 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by osd.364.0:2441 2014-08-12 15:43:37.057871 2014-08-13 18:06:06.747238 7f5c973a57a0 20 read_log 194215'16844 (194215'16843) delete 70025de/benchmark_data_storage0111-ib.pocstor_26659_object6400/head//24 by client.2174595.0:99039 0.000000 2014-08-13 18:06:06.747256 7f5c973a57a0 20 read_log 194215'16845 (192834'16085) clean 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by osd.364.0:2443 2014-08-12 15:43:37.065393 2014-08-13 18:06:06.747274 7f5c973a57a0 20 read_log 194215'16846 (194215'16845) delete 80e225de/benchmark_data_storage0111-ib.pocstor_41526_object4129/head//24 by client.2174595.0:99041 0.000000 2014-08-13 18:06:06.747288 7f5c973a57a0 20 read_log 194215'16847 (192888'16351) clean e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by osd.364.0:2445 2014-08-12 15:43:37.072400 2014-08-13 18:06:06.747303 7f5c973a57a0 20 read_log 194215'16848 (194215'16847) delete e7825de/benchmark_data_storage0111-ib.pocstor_41097_object4561/head//24 by client.2174595.0:99043 0.000000 2014-08-13 18:06:06.747317 7f5c973a57a0 20 read_log 194215'16849 (193305'16665) clean 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by osd.364.0:2447 2014-08-12 15:43:37.079409 2014-08-13 18:06:06.747332 7f5c973a57a0 20 read_log 194215'16850 (194215'16849) delete 4bf825de/benchmark_data_storage0111-ib.pocstor_62621_object1596/head//24 by client.2174595.0:99045 0.000000 2014-08-13 18:06:06.747343 7f5c973a57a0 20 read_log 194215'16851 (193305'16691) clean 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by osd.364.0:2449 2014-08-12 15:43:37.086554 2014-08-13 18:06:06.747356 7f5c973a57a0 20 read_log 194215'16852 (194215'16851) delete 950035de/benchmark_data_storage0111-ib.pocstor_62621_object31443/head//24 by client.2174595.0:99047 0.000000 2014-08-13 18:06:06.747370 7f5c973a57a0 20 read_log 194215'16853 (192834'16064) clean 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by osd.364.0:2451 2014-08-12 15:43:37.093816 2014-08-13 18:06:06.747385 7f5c973a57a0 20 read_log 194215'16854 (194215'16853) delete 88fb69de/benchmark_data_storage0111-ib.pocstor_41526_object1160/head//24 by client.2174595.0:99049 0.000000 2014-08-13 18:06:06.747409 7f5c973a57a0 20 read_log 194215'16855 (192888'16548) clean 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by osd.364.0:2453 2014-08-12 15:43:37.112037 2014-08-13 18:06:06.747429 7f5c973a57a0 20 read_log 194215'16856 (194215'16855) delete 543f79de/benchmark_data_storage0111-ib.pocstor_2228_object5102/head//24 by client.2174595.0:99051 0.000000 2014-08-13 18:06:06.747447 7f5c973a57a0 20 read_log 194215'16857 (192834'15712) clean 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by osd.364.0:2455 2014-08-12 15:43:37.119283 2014-08-13 18:06:06.747468 7f5c973a57a0 20 read_log 194215'16858 (194215'16857) delete 524f79de/benchmark_data_storage0111-ib.pocstor_26659_object20430/head//24 by client.2174595.0:99053 0.000000 2014-08-13 18:06:06.747479 7f5c973a57a0 20 read_log 194215'16859 (192834'16228) clean 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by osd.364.0:2457 2014-08-12 15:43:37.126415 2014-08-13 18:06:06.747492 7f5c973a57a0 20 read_log 194215'16860 (194215'16859) delete 9e6a99de/benchmark_data_storage0111-ib.pocstor_41526_object22853/head//24 by client.2174595.0:99055 0.000000 2014-08-13 18:06:06.747512 7f5c973a57a0 20 read_log 194215'16861 (193305'16677) clean 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by osd.364.0:2459 2014-08-12 15:43:37.133499 2014-08-13 18:06:06.747525 7f5c973a57a0 20 read_log 194215'16862 (194215'16861) delete 89cb99de/benchmark_data_storage0111-ib.pocstor_62621_object18459/head//24 by client.2174595.0:99057 0.000000 2014-08-13 18:06:06.747536 7f5c973a57a0 20 read_log 194215'16863 (192888'16378) clean 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by osd.364.0:2461 2014-08-12 15:43:37.166335 2014-08-13 18:06:06.747557 7f5c973a57a0 20 read_log 194215'16864 (194215'16863) delete 271d99de/benchmark_data_storage0111-ib.pocstor_41097_object19493/head//24 by client.2174595.0:99059 0.000000 2014-08-13 18:06:06.747581 7f5c973a57a0 20 read_log 194215'16865 (192834'16192) clean 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by osd.364.0:2463 2014-08-12 15:43:37.173462 2014-08-13 18:06:06.747604 7f5c973a57a0 20 read_log 194215'16866 (194215'16865) delete 2d84a9de/benchmark_data_storage0111-ib.pocstor_41526_object18225/head//24 by client.2174595.0:99061 0.000000 2014-08-13 18:06:06.747626 7f5c973a57a0 20 read_log 194215'16867 (192888'16374) clean 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by osd.364.0:2465 2014-08-12 15:43:37.180551 2014-08-13 18:06:06.747642 7f5c973a57a0 20 read_log 194215'16868 (194215'16867) delete 24eca9de/benchmark_data_storage0111-ib.pocstor_41097_object16896/head//24 by client.2174595.0:99063 0.000000 2014-08-13 18:06:06.747653 7f5c973a57a0 20 read_log 194215'16869 (192888'16554) clean 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by osd.364.0:2467 2014-08-12 15:43:37.208816 2014-08-13 18:06:06.747666 7f5c973a57a0 20 read_log 194215'16870 (194215'16869) delete 962fa9de/benchmark_data_storage0111-ib.pocstor_2228_object8257/head//24 by client.2174595.0:99065 0.000000 2014-08-13 18:06:06.747677 7f5c973a57a0 20 read_log 194215'16871 (193305'16666) clean a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by osd.364.0:2469 2014-08-12 15:43:37.216012 2014-08-13 18:06:06.747689 7f5c973a57a0 20 read_log 194215'16872 (194215'16871) delete a4c2d9de/benchmark_data_storage0111-ib.pocstor_62621_object1989/head//24 by client.2174595.0:99067 0.000000 2014-08-13 18:06:06.747700 7f5c973a57a0 20 read_log 194215'16873 (192888'16349) clean 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by osd.364.0:2471 2014-08-12 15:43:37.223279 2014-08-13 18:06:06.747712 7f5c973a57a0 20 read_log 194215'16874 (194215'16873) delete 5962e9de/benchmark_data_storage0111-ib.pocstor_41097_object3933/head//24 by client.2174595.0:99069 0.000000 2014-08-13 18:06:06.747723 7f5c973a57a0 20 read_log 194215'16875 (192888'16362) clean 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by osd.364.0:2473 2014-08-12 15:43:37.230084 2014-08-13 18:06:06.747742 7f5c973a57a0 20 read_log 194215'16876 (194215'16875) delete 1306e9de/benchmark_data_storage0111-ib.pocstor_41097_object11049/head//24 by client.2174595.0:99071 0.000000 2014-08-13 18:06:06.747755 7f5c973a57a0 20 read_log 194215'16877 (192888'16595) clean 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by osd.364.0:2475 2014-08-12 15:43:37.237193 2014-08-13 18:06:06.747768 7f5c973a57a0 20 read_log 194215'16878 (194215'16877) delete 5c36e9de/benchmark_data_storage0111-ib.pocstor_2228_object29792/head//24 by client.2174595.0:99073 0.000000 2014-08-13 18:06:06.747785 7f5c973a57a0 20 read_log 194215'16879 (192834'15582) clean 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by osd.364.0:2477 2014-08-12 15:43:37.244398 2014-08-13 18:06:06.747798 7f5c973a57a0 20 read_log 194215'16880 (194215'16879) delete 7b7ee9de/benchmark_data_storage0111-ib.pocstor_26659_object1309/head//24 by client.2174595.0:99075 0.000000 2014-08-13 18:06:06.747809 7f5c973a57a0 20 read_log 194215'16881 (193305'16669) clean 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by osd.364.0:2479 2014-08-12 15:43:37.250970 2014-08-13 18:06:06.747821 7f5c973a57a0 20 read_log 194215'16882 (194215'16881) delete 8108f9de/benchmark_data_storage0111-ib.pocstor_62621_object7556/head//24 by client.2174595.0:99077 0.000000 2014-08-13 18:06:06.747833 7f5c973a57a0 20 read_log 194215'16883 (192834'16104) clean e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by osd.364.0:2481 2014-08-12 15:43:37.258693 2014-08-13 18:06:06.747856 7f5c973a57a0 20 read_log 194215'16884 (194215'16883) delete e1daf9de/benchmark_data_storage0111-ib.pocstor_41526_object6717/head//24 by client.2174595.0:99079 0.000000 2014-08-13 18:06:06.747875 7f5c973a57a0 20 read_log 194215'16885 (193305'16687) clean e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by osd.364.0:2483 2014-08-12 15:43:37.265824 2014-08-13 18:06:06.747890 7f5c973a57a0 20 read_log 194215'16886 (194215'16885) delete e9860dde/benchmark_data_storage0111-ib.pocstor_62621_object28785/head//24 by client.2174595.0:99081 0.000000 2014-08-13 18:06:06.747901 7f5c973a57a0 20 read_log 194215'16887 (193305'16685) clean 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by osd.364.0:2485 2014-08-12 15:43:37.273453 2014-08-13 18:06:06.747914 7f5c973a57a0 20 read_log 194215'16888 (194215'16887) delete 567f0dde/benchmark_data_storage0111-ib.pocstor_62621_object27697/head//24 by client.2174595.0:99083 0.000000 2014-08-13 18:06:06.747925 7f5c973a57a0 20 read_log 194215'16889 (192888'16577) clean d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by osd.364.0:2487 2014-08-12 15:43:37.281077 2014-08-13 18:06:06.747937 7f5c973a57a0 20 read_log 194215'16890 (194215'16889) delete d4f21dde/benchmark_data_storage0111-ib.pocstor_2228_object20052/head//24 by client.2174595.0:99085 0.000000 2014-08-13 18:06:06.747954 7f5c973a57a0 20 read_log 194215'16891 (192834'16069) clean c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by osd.364.0:2489 2014-08-12 15:43:37.288161 2014-08-13 18:06:06.747978 7f5c973a57a0 20 read_log 194215'16892 (194215'16891) delete c2d41dde/benchmark_data_storage0111-ib.pocstor_41526_object1850/head//24 by client.2174595.0:99087 0.000000 2014-08-13 18:06:06.747989 7f5c973a57a0 20 read_log 194215'16893 (192834'16096) clean bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by osd.364.0:2491 2014-08-12 15:43:37.295189 2014-08-13 18:06:06.748001 7f5c973a57a0 20 read_log 194215'16894 (194215'16893) delete bcab1dde/benchmark_data_storage0111-ib.pocstor_41526_object5351/head//24 by client.2174595.0:99089 0.000000 2014-08-13 18:06:06.748012 7f5c973a57a0 20 read_log 194215'16895 (193305'16684) clean 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by osd.364.0:2493 2014-08-12 15:43:37.302203 2014-08-13 18:06:06.748028 7f5c973a57a0 20 read_log 194215'16896 (194215'16895) delete 220e1dde/benchmark_data_storage0111-ib.pocstor_62621_object27540/head//24 by client.2174595.0:99091 0.000000 2014-08-13 18:06:06.748040 7f5c973a57a0 20 read_log 194215'16897 (192888'16584) clean 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by osd.364.0:2495 2014-08-12 15:43:37.309418 2014-08-13 18:06:06.748052 7f5c973a57a0 20 read_log 194215'16898 (194215'16897) delete 61422dde/benchmark_data_storage0111-ib.pocstor_2228_object22402/head//24 by client.2174595.0:99093 0.000000 2014-08-13 18:06:06.748063 7f5c973a57a0 20 read_log 194215'16899 (192888'16552) clean 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by osd.364.0:2497 2014-08-12 15:43:37.316332 2014-08-13 18:06:06.748217 7f5c973a57a0 20 read_log 194215'16900 (194215'16899) delete 83b22dde/benchmark_data_storage0111-ib.pocstor_2228_object5845/head//24 by client.2174595.0:99095 0.000000 2014-08-13 18:06:06.748232 7f5c973a57a0 20 read_log 194215'16901 (192888'16363) clean e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by osd.364.0:2499 2014-08-12 15:43:37.323240 2014-08-13 18:06:06.748245 7f5c973a57a0 20 read_log 194215'16902 (194215'16901) delete e81e2dde/benchmark_data_storage0111-ib.pocstor_41097_object11477/head//24 by client.2174595.0:99097 0.000000 2014-08-13 18:06:06.748257 7f5c973a57a0 20 read_log 194215'16903 (193305'16692) clean d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by osd.364.0:2501 2014-08-12 15:43:37.330019 2014-08-13 18:06:06.748271 7f5c973a57a0 20 read_log 194215'16904 (194215'16903) delete d03f2dde/benchmark_data_storage0111-ib.pocstor_62621_object32093/head//24 by client.2174595.0:99099 0.000000 2014-08-13 18:06:06.748290 7f5c973a57a0 20 read_log 194215'16905 (192834'16103) clean 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by osd.364.0:2503 2014-08-12 15:43:37.337442 2014-08-13 18:06:06.748305 7f5c973a57a0 20 read_log 194215'16906 (194215'16905) delete 8c603dde/benchmark_data_storage0111-ib.pocstor_41526_object6572/head//24 by client.2174595.0:99101 0.000000 2014-08-13 18:06:06.748316 7f5c973a57a0 20 read_log 194215'16907 (192888'16590) clean eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by osd.364.0:2505 2014-08-12 15:43:37.344871 2014-08-13 18:06:06.748328 7f5c973a57a0 20 read_log 194215'16908 (194215'16907) delete eec24dde/benchmark_data_storage0111-ib.pocstor_2228_object24735/head//24 by client.2174595.0:99103 0.000000 2014-08-13 18:06:06.748339 7f5c973a57a0 20 read_log 194215'16909 (192834'15682) clean 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by osd.364.0:2507 2014-08-12 15:43:37.352010 2014-08-13 18:06:06.748351 7f5c973a57a0 20 read_log 194215'16910 (194215'16909) delete 52e84dde/benchmark_data_storage0111-ib.pocstor_26659_object15296/head//24 by client.2174595.0:99105 0.000000 2014-08-13 18:06:06.748362 7f5c973a57a0 20 read_log 194215'16911 (192834'16211) clean 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by osd.364.0:2509 2014-08-12 15:43:37.358892 2014-08-13 18:06:06.748375 7f5c973a57a0 20 read_log 194215'16912 (194215'16911) delete 65845dde/benchmark_data_storage0111-ib.pocstor_41526_object21169/head//24 by client.2174595.0:99107 0.000000 2014-08-13 18:06:06.748385 7f5c973a57a0 20 read_log 194215'16913 (192888'16387) clean e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by osd.364.0:2511 2014-08-12 15:43:37.365836 2014-08-13 18:06:06.748397 7f5c973a57a0 20 read_log 194215'16914 (194215'16913) delete e1226dde/benchmark_data_storage0111-ib.pocstor_41097_object23835/head//24 by client.2174595.0:99109 0.000000 2014-08-13 18:06:06.748421 7f5c973a57a0 20 read_log 194215'16915 (192888'16383) clean de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by osd.364.0:2513 2014-08-12 15:43:37.372994 2014-08-13 18:06:06.748434 7f5c973a57a0 20 read_log 194215'16916 (194215'16915) delete de446dde/benchmark_data_storage0111-ib.pocstor_41097_object22257/head//24 by client.2174595.0:99111 0.000000 2014-08-13 18:06:06.748445 7f5c973a57a0 20 read_log 194215'16917 (192834'16099) clean 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by osd.364.0:2515 2014-08-12 15:43:37.380476 2014-08-13 18:06:06.748457 7f5c973a57a0 20 read_log 194215'16918 (194215'16917) delete 6d986dde/benchmark_data_storage0111-ib.pocstor_41526_object6062/head//24 by client.2174595.0:99113 0.000000 2014-08-13 18:06:06.748468 7f5c973a57a0 20 read_log 194215'16919 (192834'15685) clean cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by osd.364.0:2517 2014-08-12 15:43:37.402787 2014-08-13 18:06:06.748480 7f5c973a57a0 20 read_log 194215'16920 (194215'16919) delete cd496dde/benchmark_data_storage0111-ib.pocstor_26659_object15658/head//24 by client.2174595.0:99115 0.000000 2014-08-13 18:06:06.748493 7f5c973a57a0 20 read_log 194215'16921 (192888'16360) clean 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by osd.364.0:2519 2014-08-12 15:43:37.410434 2014-08-13 18:06:06.748506 7f5c973a57a0 20 read_log 194215'16922 (194215'16921) delete 52ed6dde/benchmark_data_storage0111-ib.pocstor_41097_object9316/head//24 by client.2174595.0:99117 0.000000 2014-08-13 18:06:06.748517 7f5c973a57a0 20 read_log 194215'16923 (193305'16689) clean 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by osd.364.0:2521 2014-08-12 15:43:37.417720 2014-08-13 18:06:06.748529 7f5c973a57a0 20 read_log 194215'16924 (194215'16923) delete 1fc47dde/benchmark_data_storage0111-ib.pocstor_62621_object29405/head//24 by client.2174595.0:99119 0.000000 2014-08-13 18:06:06.748550 7f5c973a57a0 20 read_log 194215'16925 (193305'16672) clean d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by osd.364.0:2523 2014-08-12 15:43:37.424741 2014-08-13 18:06:06.748565 7f5c973a57a0 20 read_log 194215'16926 (194215'16925) delete d1fa7dde/benchmark_data_storage0111-ib.pocstor_62621_object13111/head//24 by client.2174595.0:99121 0.000000 2014-08-13 18:06:06.748576 7f5c973a57a0 20 read_log 194215'16927 (193305'16662) clean 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by osd.364.0:2525 2014-08-12 15:43:37.431284 2014-08-13 18:06:06.748589 7f5c973a57a0 20 read_log 194215'16928 (194215'16927) delete 81cf7dde/benchmark_data_storage0111-ib.pocstor_62621_object129/head//24 by client.2174595.0:99123 0.000000 2014-08-13 18:06:06.748600 7f5c973a57a0 20 read_log 194215'16929 (192888'16593) clean 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by osd.364.0:2527 2014-08-12 15:43:37.437420 2014-08-13 18:06:06.748612 7f5c973a57a0 20 read_log 194215'16930 (194215'16929) delete 34608dde/benchmark_data_storage0111-ib.pocstor_2228_object27898/head//24 by client.2174595.0:99125 0.000000 2014-08-13 18:06:06.748623 7f5c973a57a0 20 read_log 194215'16931 (193305'16671) clean 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by osd.364.0:2529 2014-08-12 15:43:37.443931 2014-08-13 18:06:06.748635 7f5c973a57a0 20 read_log 194215'16932 (194215'16931) delete 62028dde/benchmark_data_storage0111-ib.pocstor_62621_object12918/head//24 by client.2174595.0:99127 0.000000 2014-08-13 18:06:06.748646 7f5c973a57a0 20 read_log 194215'16933 (192834'16067) clean 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by osd.364.0:2531 2014-08-12 15:43:37.451078 2014-08-13 18:06:06.748658 7f5c973a57a0 20 read_log 194215'16934 (194215'16933) delete 60528dde/benchmark_data_storage0111-ib.pocstor_41526_object1696/head//24 by client.2174595.0:99129 0.000000 2014-08-13 18:06:06.748669 7f5c973a57a0 20 read_log 194215'16935 (192888'16543) clean ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by osd.364.0:2533 2014-08-12 15:43:37.458389 2014-08-13 18:06:06.748681 7f5c973a57a0 20 read_log 194215'16936 (194215'16935) delete ac5c8dde/benchmark_data_storage0111-ib.pocstor_2228_object2392/head//24 by client.2174595.0:99131 0.000000 2014-08-13 18:06:06.748713 7f5c973a57a0 20 read_log 194215'16937 (192834'15596) clean 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by osd.364.0:2535 2014-08-12 15:43:37.464702 2014-08-13 18:06:06.748728 7f5c973a57a0 20 read_log 194215'16938 (194215'16937) delete 677f8dde/benchmark_data_storage0111-ib.pocstor_26659_object2924/head//24 by client.2174595.0:99133 0.000000 2014-08-13 18:06:06.748739 7f5c973a57a0 20 read_log 194215'16939 (192888'16358) clean b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by osd.364.0:2537 2014-08-12 15:43:37.471314 2014-08-13 18:06:06.748752 7f5c973a57a0 20 read_log 194215'16940 (194215'16939) delete b6dd9dde/benchmark_data_storage0111-ib.pocstor_41097_object8123/head//24 by client.2174595.0:99135 0.000000 2014-08-13 18:06:06.748763 7f5c973a57a0 20 read_log 194215'16941 (192888'16546) clean cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by osd.364.0:2539 2014-08-12 15:43:37.478570 2014-08-13 18:06:06.748777 7f5c973a57a0 20 read_log 194215'16942 (194215'16941) delete cddbadde/benchmark_data_storage0111-ib.pocstor_2228_object4738/head//24 by client.2174595.0:99137 0.000000 2014-08-13 18:06:06.748789 7f5c973a57a0 20 read_log 194215'16943 (193305'16690) clean 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by osd.364.0:2541 2014-08-12 15:43:37.485702 2014-08-13 18:06:06.748803 7f5c973a57a0 20 read_log 194215'16944 (194215'16943) delete 5975bdde/benchmark_data_storage0111-ib.pocstor_62621_object29843/head//24 by client.2174595.0:99139 0.000000 2014-08-13 18:06:06.748836 7f5c973a57a0 20 read_log 194215'16945 (192888'16394) clean 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by osd.364.0:2543 2014-08-12 15:43:37.493129 2014-08-13 18:06:06.748852 7f5c973a57a0 20 read_log 194215'16946 (194215'16945) delete 51b6bdde/benchmark_data_storage0111-ib.pocstor_41097_object28171/head//24 by client.2174595.0:99141 0.000000 2014-08-13 18:06:06.748863 7f5c973a57a0 20 read_log 194215'16947 (192888'16373) clean 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by osd.364.0:2545 2014-08-12 15:43:37.499872 2014-08-13 18:06:06.748876 7f5c973a57a0 20 read_log 194215'16948 (194215'16947) delete 2987bdde/benchmark_data_storage0111-ib.pocstor_41097_object15878/head//24 by client.2174595.0:99143 0.000000 2014-08-13 18:06:06.748888 7f5c973a57a0 20 read_log 194215'16949 (192888'16566) clean ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by osd.364.0:2547 2014-08-12 15:43:37.525331 2014-08-13 18:06:06.748903 7f5c973a57a0 20 read_log 194215'16950 (194215'16949) delete ecf9bdde/benchmark_data_storage0111-ib.pocstor_2228_object15410/head//24 by client.2174595.0:99145 0.000000 2014-08-13 18:06:06.748914 7f5c973a57a0 20 read_log 194215'16951 (192834'15678) clean ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by osd.364.0:2549 2014-08-12 15:43:37.533470 2014-08-13 18:06:06.748926 7f5c973a57a0 20 read_log 194215'16952 (194215'16951) delete ae0cdde/benchmark_data_storage0111-ib.pocstor_26659_object15096/head//24 by client.2174595.0:99147 0.000000 2014-08-13 18:06:06.748937 7f5c973a57a0 20 read_log 194215'16953 (192834'16133) clean 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by osd.364.0:2551 2014-08-12 15:43:37.540724 2014-08-13 18:06:06.748952 7f5c973a57a0 20 read_log 194215'16954 (194215'16953) delete 6bb4cdde/benchmark_data_storage0111-ib.pocstor_41526_object10464/head//24 by client.2174595.0:99149 0.000000 2014-08-13 18:06:06.748963 7f5c973a57a0 20 read_log 194215'16955 (192888'16376) clean 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by osd.364.0:2553 2014-08-12 15:43:37.547755 2014-08-13 18:06:06.748975 7f5c973a57a0 20 read_log 194215'16956 (194215'16955) delete 2965cdde/benchmark_data_storage0111-ib.pocstor_41097_object17948/head//24 by client.2174595.0:99151 0.000000 2014-08-13 18:06:06.748986 7f5c973a57a0 20 read_log 194215'16957 (192888'16377) clean 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by osd.364.0:2555 2014-08-12 15:43:37.555248 2014-08-13 18:06:06.748998 7f5c973a57a0 20 read_log 194215'16958 (194215'16957) delete 1766cdde/benchmark_data_storage0111-ib.pocstor_41097_object18957/head//24 by client.2174595.0:99153 0.000000 2014-08-13 18:06:06.749009 7f5c973a57a0 20 read_log 194215'16959 (193305'16674) clean 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by osd.364.0:2557 2014-08-12 15:43:37.562846 2014-08-13 18:06:06.749023 7f5c973a57a0 20 read_log 194215'16960 (194215'16959) delete 7f67cdde/benchmark_data_storage0111-ib.pocstor_62621_object14379/head//24 by client.2174595.0:99155 0.000000 2014-08-13 18:06:06.749037 7f5c973a57a0 20 read_log 194215'16961 (192834'16101) clean e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by osd.364.0:2559 2014-08-12 15:43:37.570539 2014-08-13 18:06:06.749049 7f5c973a57a0 20 read_log 194215'16962 (194215'16961) delete e6eacdde/benchmark_data_storage0111-ib.pocstor_41526_object6227/head//24 by client.2174595.0:99157 0.000000 2014-08-13 18:06:06.749063 7f5c973a57a0 20 read_log 194215'16963 (192834'16150) clean 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by osd.364.0:2561 2014-08-12 15:43:37.579861 2014-08-13 18:06:06.749075 7f5c973a57a0 20 read_log 194215'16964 (194215'16963) delete 593ecdde/benchmark_data_storage0111-ib.pocstor_41526_object12579/head//24 by client.2174595.0:99159 0.000000 2014-08-13 18:06:06.749095 7f5c973a57a0 20 read_log 194215'16965 (192834'15680) clean 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by osd.364.0:2563 2014-08-12 15:43:37.586744 2014-08-13 18:06:06.749110 7f5c973a57a0 20 read_log 194215'16966 (194215'16965) delete 60f8ddde/benchmark_data_storage0111-ib.pocstor_26659_object15238/head//24 by client.2174595.0:99161 0.000000 2014-08-13 18:06:06.749121 7f5c973a57a0 20 read_log 194215'16967 (192834'15679) clean de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by osd.364.0:2565 2014-08-12 15:43:37.593397 2014-08-13 18:06:06.749134 7f5c973a57a0 20 read_log 194215'16968 (194215'16967) delete de4eddde/benchmark_data_storage0111-ib.pocstor_26659_object15164/head//24 by client.2174595.0:99163 0.000000 2014-08-13 18:06:06.749145 7f5c973a57a0 20 read_log 194215'16969 (192834'16184) clean ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by osd.364.0:2567 2014-08-12 15:43:37.601086 2014-08-13 18:06:06.749165 7f5c973a57a0 20 read_log 194215'16970 (194215'16969) delete ed70edde/benchmark_data_storage0111-ib.pocstor_41526_object17530/head//24 by client.2174595.0:99165 0.000000 2014-08-13 18:06:06.749176 7f5c973a57a0 20 read_log 194215'16971 (192834'16122) clean e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by osd.364.0:2569 2014-08-12 15:43:37.608288 2014-08-13 18:06:06.749188 7f5c973a57a0 20 read_log 194215'16972 (194215'16971) delete e9f0fdde/benchmark_data_storage0111-ib.pocstor_41526_object8796/head//24 by client.2174595.0:99167 0.000000 2014-08-13 18:06:06.749200 7f5c973a57a0 20 read_log 194215'16973 (194215'16973) modify 1de/hit_set_24.1de_archive_2014-08-12 15:43:36.831710_2014-08-12 17:24:44.125385/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.125667 2014-08-13 18:06:06.749212 7f5c973a57a0 20 read_log 194215'16974 (194215'16785) delete 1de/hit_set_24.1de_archive_2014-08-12 15:32:25.736820_2014-08-12 15:43:36.831465/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.125667 2014-08-13 18:06:06.749227 7f5c973a57a0 20 read_log 194215'16975 (0'0) promote f036cdde/benchmark_data_storage0111-ib.pocstor_34628_object1089/head//24 by osd.364.0:3492 2014-08-12 17:24:44.131079 2014-08-13 18:06:06.749239 7f5c973a57a0 20 read_log 194215'16976 (194215'16975) modify f036cdde/benchmark_data_storage0111-ib.pocstor_34628_object1089/head//24 by client.2193759.0:1090 2014-08-12 17:24:44.118362 2014-08-13 18:06:06.749252 7f5c973a57a0 20 read_log 194370'16977 (0'0) promote 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by osd.364.0:3558 2014-08-12 20:30:28.204970 2014-08-13 18:06:06.749267 7f5c973a57a0 20 read_log 194370'16978 (194370'16977) modify 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by client.1508531.0:3700 2014-08-12 20:30:28.195281 2014-08-13 18:06:06.749279 7f5c973a57a0 20 read_log 194370'16979 (0'0) promote db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by osd.364.0:3566 2014-08-12 20:30:29.883521 2014-08-13 18:06:06.749291 7f5c973a57a0 20 read_log 194370'16980 (194370'16979) modify db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by client.1508531.0:4154 2014-08-12 20:30:29.875338 2014-08-13 18:06:06.749304 7f5c973a57a0 20 read_log 194370'16981 (0'0) promote d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by osd.364.0:3574 2014-08-12 20:30:31.498401 2014-08-13 18:06:06.749319 7f5c973a57a0 20 read_log 194370'16982 (194370'16981) modify d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by client.1508531.0:4570 2014-08-12 20:30:31.488804 2014-08-13 18:06:06.749331 7f5c973a57a0 20 read_log 194370'16983 (0'0) promote 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by osd.364.0:3576 2014-08-12 20:30:31.875612 2014-08-13 18:06:06.749353 7f5c973a57a0 20 read_log 194370'16984 (194370'16983) modify 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by client.1508531.0:4665 2014-08-12 20:30:31.867906 2014-08-13 18:06:06.749369 7f5c973a57a0 20 read_log 194370'16985 (0'0) promote b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by osd.364.0:3578 2014-08-12 20:30:32.470251 2014-08-13 18:06:06.749384 7f5c973a57a0 20 read_log 194370'16986 (194370'16985) modify b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by client.1508531.0:4822 2014-08-12 20:30:32.463242 2014-08-13 18:06:06.749406 7f5c973a57a0 20 read_log 194370'16987 (0'0) promote df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by osd.364.0:3678 2014-08-12 20:31:08.658202 2014-08-13 18:06:06.749425 7f5c973a57a0 20 read_log 194370'16988 (194370'16987) modify df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by client.1508531.0:11557 2014-08-12 20:31:08.650240 2014-08-13 18:06:06.749438 7f5c973a57a0 20 read_log 194370'16989 (0'0) promote 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by osd.364.0:3682 2014-08-12 20:31:09.954591 2014-08-13 18:06:06.749456 7f5c973a57a0 20 read_log 194370'16990 (194370'16989) modify 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by client.1508531.0:11778 2014-08-12 20:31:09.947609 2014-08-13 18:06:06.749468 7f5c973a57a0 20 read_log 194370'16991 (0'0) promote 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by osd.364.0:3692 2014-08-12 20:31:11.065603 2014-08-13 18:06:06.749481 7f5c973a57a0 20 read_log 194370'16992 (194370'16991) modify 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by client.1508531.0:11959 2014-08-12 20:31:11.056711 2014-08-13 18:06:06.749493 7f5c973a57a0 20 read_log 194370'16993 (0'0) promote f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by osd.364.0:3696 2014-08-12 20:31:11.584470 2014-08-13 18:06:06.749513 7f5c973a57a0 20 read_log 194370'16994 (194370'16993) modify f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by client.1508531.0:12031 2014-08-12 20:31:11.574016 2014-08-13 18:06:06.749526 7f5c973a57a0 20 read_log 194370'16995 (0'0) promote 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by osd.364.0:3698 2014-08-12 20:31:11.605710 2014-08-13 18:06:06.749538 7f5c973a57a0 20 read_log 194370'16996 (194370'16995) modify 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by client.1508531.0:12034 2014-08-12 20:31:11.595130 2014-08-13 18:06:06.749559 7f5c973a57a0 20 read_log 194370'16997 (0'0) promote 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by osd.364.0:3708 2014-08-12 20:31:16.871592 2014-08-13 18:06:06.749572 7f5c973a57a0 20 read_log 194370'16998 (194370'16997) modify 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by client.1508531.0:12736 2014-08-12 20:31:16.863845 2014-08-13 18:06:06.749585 7f5c973a57a0 20 read_log 194370'16999 (0'0) promote 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by osd.364.0:3710 2014-08-12 20:31:22.856582 2014-08-13 18:06:06.749601 7f5c973a57a0 20 read_log 194370'17000 (194370'16999) modify 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by client.1508531.0:13364 2014-08-12 20:31:22.848815 2014-08-13 18:06:06.749614 7f5c973a57a0 20 read_log 194370'17001 (0'0) promote 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by osd.364.0:3716 2014-08-12 20:31:25.947380 2014-08-13 18:06:06.749626 7f5c973a57a0 20 read_log 194370'17002 (194370'17001) modify 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by client.1508531.0:13689 2014-08-12 20:31:25.940307 2014-08-13 18:06:06.749639 7f5c973a57a0 20 read_log 194370'17003 (0'0) promote 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by osd.364.0:3722 2014-08-12 20:31:32.135148 2014-08-13 18:06:06.749666 7f5c973a57a0 20 read_log 194370'17004 (194370'17003) modify 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by client.1508531.0:13997 2014-08-12 20:31:32.126438 2014-08-13 18:06:06.749679 7f5c973a57a0 20 read_log 194377'17005 (0'0) promote 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by osd.364.0:3750 2014-08-12 20:33:02.971956 2014-08-13 18:06:06.749691 7f5c973a57a0 20 read_log 194377'17006 (194377'17005) modify 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by client.2231190.0:1440 2014-08-12 20:33:02.964206 2014-08-13 18:06:06.749703 7f5c973a57a0 20 read_log 194383'17007 (0'0) promote 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by osd.364.0:3846 2014-08-12 20:34:24.795228 2014-08-13 18:06:06.749716 7f5c973a57a0 20 read_log 194383'17008 (194383'17007) modify 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by client.2231190.0:7355 2014-08-12 20:34:24.786699 2014-08-13 18:06:06.749728 7f5c973a57a0 20 read_log 194383'17009 (0'0) promote 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by osd.364.0:3850 2014-08-12 20:34:29.642453 2014-08-13 18:06:06.749740 7f5c973a57a0 20 read_log 194383'17010 (194383'17009) modify 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by client.2231190.0:7532 2014-08-12 20:34:29.634334 2014-08-13 18:06:06.749752 7f5c973a57a0 20 read_log 194393'17011 (194393'17011) modify 1de/hit_set_24.1de_archive_2014-08-12 20:37:09.033964_2014-08-12 20:49:27.577081/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.577309 2014-08-13 18:06:06.749764 7f5c973a57a0 20 read_log 194393'17012 (194215'16973) delete 1de/hit_set_24.1de_archive_2014-08-12 15:43:36.831710_2014-08-12 17:24:44.125385/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.577309 2014-08-13 18:06:06.749777 7f5c973a57a0 20 read_log 194393'17013 (0'0) promote ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by osd.364.0:3856 2014-08-12 20:49:27.580084 2014-08-13 18:06:06.749789 7f5c973a57a0 20 read_log 194393'17014 (194393'17013) modify ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by client.1064182.0:425 2014-08-12 20:49:27.571281 2014-08-13 18:06:06.749801 7f5c973a57a0 20 read_log 194416'17015 (194416'17015) modify 1de/hit_set_24.1de_archive_2014-08-12 20:49:27.577314_2014-08-13 15:28:31.498257/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 18:06:06.749813 7f5c973a57a0 20 read_log 194416'17016 (194393'17011) delete 1de/hit_set_24.1de_archive_2014-08-12 20:37:09.033964_2014-08-12 20:49:27.577081/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 18:06:06.749825 7f5c973a57a0 20 read_log 194416'17017 (0'0) promote 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by osd.364.0:4024 2014-08-13 15:28:31.501740 2014-08-13 18:06:06.749837 7f5c973a57a0 20 read_log 194416'17018 (194416'17017) modify 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by client.2441958.0:310 2014-08-13 15:28:31.491025 2014-08-13 18:06:06.749850 7f5c973a57a0 20 read_log 194416'17019 (0'0) promote 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by osd.364.0:4043 2014-08-13 15:28:35.426964 2014-08-13 18:06:06.749862 7f5c973a57a0 20 read_log 194416'17020 (194416'17019) modify 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by client.2441958.0:1240 2014-08-13 15:28:35.419433 2014-08-13 18:06:06.749874 7f5c973a57a0 20 read_log 194416'17021 (0'0) promote b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by osd.364.0:4070 2014-08-13 15:28:41.285953 2014-08-13 18:06:06.749886 7f5c973a57a0 20 read_log 194416'17022 (194416'17021) modify b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by client.2441958.0:2788 2014-08-13 15:28:41.277463 2014-08-13 18:06:06.749915 7f5c973a57a0 20 read_log 194416'17023 (0'0) promote da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by osd.364.0:4074 2014-08-13 15:28:41.565171 2014-08-13 18:06:06.749928 7f5c973a57a0 20 read_log 194416'17024 (194416'17023) modify da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by client.2441958.0:2863 2014-08-13 15:28:41.558080 2014-08-13 18:06:06.749940 7f5c973a57a0 20 read_log 194416'17025 (0'0) promote 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by osd.364.0:4086 2014-08-13 15:28:43.716695 2014-08-13 18:06:06.749955 7f5c973a57a0 20 read_log 194416'17026 (194416'17025) modify 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by client.2441958.0:3435 2014-08-13 15:28:43.708507 2014-08-13 18:06:06.749972 7f5c973a57a0 20 read_log 194416'17027 (0'0) promote 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by osd.364.0:4126 2014-08-13 15:28:53.917553 2014-08-13 18:06:06.749987 7f5c973a57a0 20 read_log 194416'17028 (194416'17027) modify 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by client.2441958.0:6041 2014-08-13 15:28:53.907907 2014-08-13 18:06:06.750011 7f5c973a57a0 20 read_log 194416'17029 (0'0) promote 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by osd.364.0:4128 2014-08-13 15:28:54.748039 2014-08-13 18:06:06.750042 7f5c973a57a0 20 read_log 194416'17030 (194416'17029) modify 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by client.2441958.0:6218 2014-08-13 15:28:54.739638 2014-08-13 18:06:06.750075 7f5c973a57a0 20 read_log 194416'17031 (0'0) promote 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by osd.364.0:4158 2014-08-13 15:29:04.781669 2014-08-13 18:06:06.750092 7f5c973a57a0 20 read_log 194416'17032 (194416'17031) modify 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by client.2441958.0:8846 2014-08-13 15:29:04.772064 2014-08-13 18:06:06.750104 7f5c973a57a0 20 read_log 194416'17033 (0'0) promote 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by osd.364.0:4162 2014-08-13 15:29:05.686961 2014-08-13 18:06:06.750119 7f5c973a57a0 20 read_log 194416'17034 (194416'17033) modify 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by client.2441958.0:9054 2014-08-13 15:29:05.677607 2014-08-13 18:06:06.750131 7f5c973a57a0 20 read_log 194416'17035 (0'0) promote 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by osd.364.0:4184 2014-08-13 15:29:13.941911 2014-08-13 18:06:06.750143 7f5c973a57a0 20 read_log 194416'17036 (194416'17035) modify 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by client.2441958.0:11141 2014-08-13 15:29:13.933397 2014-08-13 18:06:06.750161 7f5c973a57a0 20 read_log 194416'17037 (0'0) promote aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by osd.364.0:4190 2014-08-13 15:29:14.613548 2014-08-13 18:06:06.750174 7f5c973a57a0 20 read_log 194416'17038 (194416'17037) modify aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by client.2441958.0:11283 2014-08-13 15:29:14.606987 2014-08-13 18:06:06.750187 7f5c973a57a0 20 read_log 194416'17039 (0'0) promote a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by osd.364.0:4196 2014-08-13 15:29:16.675756 2014-08-13 18:06:06.750207 7f5c973a57a0 20 read_log 194416'17040 (194416'17039) modify a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by client.2441958.0:11789 2014-08-13 15:29:16.666335 2014-08-13 18:06:06.750224 7f5c973a57a0 20 read_log 194416'17041 (0'0) promote c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by osd.364.0:4208 2014-08-13 15:29:19.591320 2014-08-13 18:06:06.750242 7f5c973a57a0 20 read_log 194416'17042 (194416'17041) modify c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by client.2441958.0:12471 2014-08-13 15:29:19.578362 2014-08-13 18:06:06.750272 7f5c973a57a0 20 read_log 194416'17043 (0'0) promote 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by osd.364.0:4226 2014-08-13 15:29:25.085099 2014-08-13 18:06:06.750287 7f5c973a57a0 20 read_log 194416'17044 (194416'17043) modify 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by client.2441958.0:13719 2014-08-13 15:29:25.075531 2014-08-13 18:06:06.750310 7f5c973a57a0 20 read_log 194416'17045 (0'0) promote 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by osd.364.0:4290 2014-08-13 15:30:13.915299 2014-08-13 18:06:06.750323 7f5c973a57a0 20 read_log 194416'17046 (194416'17045) modify 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by client.2441958.0:18529 2014-08-13 15:30:13.906249 2014-08-13 18:06:06.750336 7f5c973a57a0 20 read_log 194422'17047 (0'0) promote 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by osd.364.0:4292 2014-08-13 15:32:15.581218 2014-08-13 18:06:06.750348 7f5c973a57a0 20 read_log 194422'17048 (194422'17047) modify 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by client.1148203.0:142 2014-08-13 15:32:15.571168 2014-08-13 18:06:06.750360 7f5c973a57a0 20 read_log 194422'17049 (0'0) promote 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by osd.364.0:4296 2014-08-13 15:32:16.377171 2014-08-13 18:06:06.750372 7f5c973a57a0 20 read_log 194422'17050 (194422'17049) modify 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by client.1148203.0:300 2014-08-13 15:32:16.369060 2014-08-13 18:06:06.750385 7f5c973a57a0 20 read_log 194422'17051 (0'0) promote 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by osd.364.0:4308 2014-08-13 15:32:18.166647 2014-08-13 18:06:06.750397 7f5c973a57a0 20 read_log 194422'17052 (194422'17051) modify 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by client.1148203.0:702 2014-08-13 15:32:18.157676 2014-08-13 18:06:06.750409 7f5c973a57a0 20 read_log 194422'17053 (0'0) promote d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by osd.364.0:4314 2014-08-13 15:32:20.015280 2014-08-13 18:06:06.750421 7f5c973a57a0 20 read_log 194422'17054 (194422'17053) modify d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by client.1148203.0:1189 2014-08-13 15:32:20.006353 2014-08-13 18:06:06.750434 7f5c973a57a0 20 read_log 194422'17055 (0'0) promote 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by osd.364.0:4334 2014-08-13 15:32:25.908685 2014-08-13 18:06:06.750454 7f5c973a57a0 20 read_log 194422'17056 (194422'17055) modify 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by client.1148203.0:2704 2014-08-13 15:32:25.901830 2014-08-13 18:06:06.750468 7f5c973a57a0 20 read_log 194422'17057 (0'0) promote d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by osd.364.0:4348 2014-08-13 15:32:29.709011 2014-08-13 18:06:06.750480 7f5c973a57a0 20 read_log 194422'17058 (194422'17057) modify d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by client.1148203.0:3659 2014-08-13 15:32:29.701406 2014-08-13 18:06:06.750492 7f5c973a57a0 20 read_log 194422'17059 (0'0) promote 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by osd.364.0:4350 2014-08-13 15:32:29.902858 2014-08-13 18:06:06.750504 7f5c973a57a0 20 read_log 194422'17060 (194422'17059) modify 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by client.1148203.0:3709 2014-08-13 15:32:29.894894 2014-08-13 18:06:06.750516 7f5c973a57a0 20 read_log 194422'17061 (0'0) promote 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by osd.364.0:4372 2014-08-13 15:32:38.401185 2014-08-13 18:06:06.750528 7f5c973a57a0 20 read_log 194422'17062 (194422'17061) modify 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by client.1148203.0:5681 2014-08-13 15:32:38.391351 2014-08-13 18:06:06.750542 7f5c973a57a0 20 read_log 194422'17063 (0'0) promote 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by osd.364.0:4386 2014-08-13 15:32:43.094141 2014-08-13 18:06:06.750554 7f5c973a57a0 20 read_log 194422'17064 (194422'17063) modify 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by client.1148203.0:6548 2014-08-13 15:32:43.085134 2014-08-13 18:06:06.750567 7f5c973a57a0 20 read_log 194422'17065 (0'0) promote dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by osd.364.0:4400 2014-08-13 15:32:48.919635 2014-08-13 18:06:06.750579 7f5c973a57a0 20 read_log 194422'17066 (194422'17065) modify dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by client.1148203.0:7549 2014-08-13 15:32:48.910443 2014-08-13 18:06:06.750591 7f5c973a57a0 20 read_log 194422'17067 (0'0) promote 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by osd.364.0:4406 2014-08-13 15:32:51.270747 2014-08-13 18:06:06.750603 7f5c973a57a0 20 read_log 194422'17068 (194422'17067) modify 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by client.1148203.0:7929 2014-08-13 15:32:51.261634 2014-08-13 18:06:06.750615 7f5c973a57a0 20 read_log 194422'17069 (0'0) promote a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by osd.364.0:4414 2014-08-13 15:32:52.703908 2014-08-13 18:06:06.750630 7f5c973a57a0 20 read_log 194422'17070 (194422'17069) modify a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by client.1148203.0:8156 2014-08-13 15:32:52.695197 2014-08-13 18:06:06.750642 7f5c973a57a0 20 read_log 194422'17071 (0'0) promote e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by osd.364.0:4416 2014-08-13 15:32:52.819821 2014-08-13 18:06:06.750657 7f5c973a57a0 20 read_log 194422'17072 (194422'17071) modify e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by client.1148203.0:8174 2014-08-13 15:32:52.812513 2014-08-13 18:06:06.750670 7f5c973a57a0 20 read_log 194422'17073 (0'0) promote 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by osd.364.0:4437 2014-08-13 15:33:21.760279 2014-08-13 18:06:06.750684 7f5c973a57a0 20 read_log 194422'17074 (194422'17073) modify 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by client.1148203.0:10420 2014-08-13 15:33:21.751200 2014-08-13 18:06:06.750700 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.751102 7f5c973a57a0 10 read_log checking for missing items over interval (194370'16977,194422'17074] 2014-08-13 18:06:06.751154 7f5c973a57a0 15 read_log missing 194422'17074 (194422'17073) modify 2b0bfdde/benchmark_data_storage0111-ib.pocstor_41231_object10419/head//24 by client.1148203.0:10420 2014-08-13 15:33:21.751200 2014-08-13 18:06:06.751195 7f5c973a57a0 15 read_log missing 194422'17072 (194422'17071) modify e47995de/benchmark_data_storage0111-ib.pocstor_41231_object8173/head//24 by client.1148203.0:8174 2014-08-13 15:32:52.812513 2014-08-13 18:06:06.751231 7f5c973a57a0 15 read_log missing 194422'17070 (194422'17069) modify a49ebdde/benchmark_data_storage0111-ib.pocstor_41231_object8155/head//24 by client.1148203.0:8156 2014-08-13 15:32:52.695197 2014-08-13 18:06:06.751271 7f5c973a57a0 15 read_log missing 194422'17068 (194422'17067) modify 7fb515de/benchmark_data_storage0111-ib.pocstor_41231_object7928/head//24 by client.1148203.0:7929 2014-08-13 15:32:51.261634 2014-08-13 18:06:06.751307 7f5c973a57a0 15 read_log missing 194422'17066 (194422'17065) modify dd7b65de/benchmark_data_storage0111-ib.pocstor_41231_object7548/head//24 by client.1148203.0:7549 2014-08-13 15:32:48.910443 2014-08-13 18:06:06.751348 7f5c973a57a0 15 read_log missing 194422'17064 (194422'17063) modify 34ebe5de/benchmark_data_storage0111-ib.pocstor_41231_object6547/head//24 by client.1148203.0:6548 2014-08-13 15:32:43.085134 2014-08-13 18:06:06.751384 7f5c973a57a0 15 read_log missing 194422'17062 (194422'17061) modify 2d431de/benchmark_data_storage0111-ib.pocstor_41231_object5680/head//24 by client.1148203.0:5681 2014-08-13 15:32:38.391351 2014-08-13 18:06:06.751437 7f5c973a57a0 15 read_log missing 194422'17060 (194422'17059) modify 71c0c5de/benchmark_data_storage0111-ib.pocstor_41231_object3708/head//24 by client.1148203.0:3709 2014-08-13 15:32:29.894894 2014-08-13 18:06:06.751485 7f5c973a57a0 15 read_log missing 194422'17058 (194422'17057) modify d3e735de/benchmark_data_storage0111-ib.pocstor_41231_object3658/head//24 by client.1148203.0:3659 2014-08-13 15:32:29.701406 2014-08-13 18:06:06.751522 7f5c973a57a0 15 read_log missing 194422'17056 (194422'17055) modify 1f331dde/benchmark_data_storage0111-ib.pocstor_41231_object2703/head//24 by client.1148203.0:2704 2014-08-13 15:32:25.901830 2014-08-13 18:06:06.751557 7f5c973a57a0 15 read_log missing 194422'17054 (194422'17053) modify d9fa99de/benchmark_data_storage0111-ib.pocstor_41231_object1188/head//24 by client.1148203.0:1189 2014-08-13 15:32:20.006353 2014-08-13 18:06:06.751603 7f5c973a57a0 15 read_log missing 194422'17052 (194422'17051) modify 49c439de/benchmark_data_storage0111-ib.pocstor_41231_object701/head//24 by client.1148203.0:702 2014-08-13 15:32:18.157676 2014-08-13 18:06:06.751639 7f5c973a57a0 15 read_log missing 194422'17050 (194422'17049) modify 1564b5de/benchmark_data_storage0111-ib.pocstor_41231_object299/head//24 by client.1148203.0:300 2014-08-13 15:32:16.369060 2014-08-13 18:06:06.751675 7f5c973a57a0 15 read_log missing 194422'17048 (194422'17047) modify 457559de/benchmark_data_storage0111-ib.pocstor_41231_object141/head//24 by client.1148203.0:142 2014-08-13 15:32:15.571168 2014-08-13 18:06:06.751710 7f5c973a57a0 15 read_log missing 194416'17046 (194416'17045) modify 2c42cdde/benchmark_data_storage0111-ib.pocstor_19103_object18528/head//24 by client.2441958.0:18529 2014-08-13 15:30:13.906249 2014-08-13 18:06:06.751745 7f5c973a57a0 15 read_log missing 194416'17044 (194416'17043) modify 87de09de/benchmark_data_storage0111-ib.pocstor_19103_object13718/head//24 by client.2441958.0:13719 2014-08-13 15:29:25.075531 2014-08-13 18:06:06.751781 7f5c973a57a0 15 read_log missing 194416'17042 (194416'17041) modify c7e8b1de/benchmark_data_storage0111-ib.pocstor_19103_object12470/head//24 by client.2441958.0:12471 2014-08-13 15:29:19.578362 2014-08-13 18:06:06.751816 7f5c973a57a0 15 read_log missing 194416'17040 (194416'17039) modify a89d45de/benchmark_data_storage0111-ib.pocstor_19103_object11788/head//24 by client.2441958.0:11789 2014-08-13 15:29:16.666335 2014-08-13 18:06:06.751851 7f5c973a57a0 15 read_log missing 194416'17038 (194416'17037) modify aceef9de/benchmark_data_storage0111-ib.pocstor_19103_object11282/head//24 by client.2441958.0:11283 2014-08-13 15:29:14.606987 2014-08-13 18:06:06.751887 7f5c973a57a0 15 read_log missing 194416'17036 (194416'17035) modify 1b0cb9de/benchmark_data_storage0111-ib.pocstor_19103_object11140/head//24 by client.2441958.0:11141 2014-08-13 15:29:13.933397 2014-08-13 18:06:06.751922 7f5c973a57a0 15 read_log missing 194416'17034 (194416'17033) modify 804df9de/benchmark_data_storage0111-ib.pocstor_19103_object9053/head//24 by client.2441958.0:9054 2014-08-13 15:29:05.677607 2014-08-13 18:06:06.751957 7f5c973a57a0 15 read_log missing 194416'17032 (194416'17031) modify 6ed095de/benchmark_data_storage0111-ib.pocstor_19103_object8845/head//24 by client.2441958.0:8846 2014-08-13 15:29:04.772064 2014-08-13 18:06:06.751992 7f5c973a57a0 15 read_log missing 194416'17030 (194416'17029) modify 353cadde/benchmark_data_storage0111-ib.pocstor_19103_object6217/head//24 by client.2441958.0:6218 2014-08-13 15:28:54.739638 2014-08-13 18:06:06.752027 7f5c973a57a0 15 read_log missing 194416'17028 (194416'17027) modify 9f1a1dde/benchmark_data_storage0111-ib.pocstor_19103_object6040/head//24 by client.2441958.0:6041 2014-08-13 15:28:53.907907 2014-08-13 18:06:06.752063 7f5c973a57a0 15 read_log missing 194416'17026 (194416'17025) modify 47c8d5de/benchmark_data_storage0111-ib.pocstor_19103_object3434/head//24 by client.2441958.0:3435 2014-08-13 15:28:43.708507 2014-08-13 18:06:06.752098 7f5c973a57a0 15 read_log missing 194416'17024 (194416'17023) modify da9e39de/benchmark_data_storage0111-ib.pocstor_19103_object2862/head//24 by client.2441958.0:2863 2014-08-13 15:28:41.558080 2014-08-13 18:06:06.752133 7f5c973a57a0 15 read_log missing 194416'17022 (194416'17021) modify b7e269de/benchmark_data_storage0111-ib.pocstor_19103_object2787/head//24 by client.2441958.0:2788 2014-08-13 15:28:41.277463 2014-08-13 18:06:06.752174 7f5c973a57a0 15 read_log missing 194416'17020 (194416'17019) modify 15c819de/benchmark_data_storage0111-ib.pocstor_19103_object1239/head//24 by client.2441958.0:1240 2014-08-13 15:28:35.419433 2014-08-13 18:06:06.752211 7f5c973a57a0 15 read_log missing 194416'17018 (194416'17017) modify 3c54b9de/benchmark_data_storage0111-ib.pocstor_19103_object309/head//24 by client.2441958.0:310 2014-08-13 15:28:31.491025 2014-08-13 18:06:06.752253 7f5c973a57a0 15 read_log missing 194416'17015 (194416'17015) modify 1de/hit_set_24.1de_archive_2014-08-12 20:49:27.577314_2014-08-13 15:28:31.498257/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.498468 2014-08-13 18:06:06.752289 7f5c973a57a0 15 read_log missing 194393'17014 (194393'17013) modify ef74b5de/benchmark_data_storage0111-ib.pocstor_32482_object424/head//24 by client.1064182.0:425 2014-08-12 20:49:27.571281 2014-08-13 18:06:06.752328 7f5c973a57a0 15 read_log missing 194383'17010 (194383'17009) modify 47adadde/benchmark_data_storage0111-ib.pocstor_25065_object7531/head//24 by client.2231190.0:7532 2014-08-12 20:34:29.634334 2014-08-13 18:06:06.752372 7f5c973a57a0 15 read_log missing 194383'17008 (194383'17007) modify 6ab4d1de/benchmark_data_storage0111-ib.pocstor_25065_object7354/head//24 by client.2231190.0:7355 2014-08-12 20:34:24.786699 2014-08-13 18:06:06.752418 7f5c973a57a0 15 read_log missing 194377'17006 (194377'17005) modify 730dc1de/benchmark_data_storage0111-ib.pocstor_25065_object1439/head//24 by client.2231190.0:1440 2014-08-12 20:33:02.964206 2014-08-13 18:06:06.752462 7f5c973a57a0 15 read_log missing 194370'17004 (194370'17003) modify 303259de/benchmark_data_storage0111-ib.pocstor_12495_object13996/head//24 by client.1508531.0:13997 2014-08-12 20:31:32.126438 2014-08-13 18:06:06.752499 7f5c973a57a0 15 read_log missing 194370'17002 (194370'17001) modify 6d6d9dde/benchmark_data_storage0111-ib.pocstor_12495_object13688/head//24 by client.1508531.0:13689 2014-08-12 20:31:25.940307 2014-08-13 18:06:06.752535 7f5c973a57a0 15 read_log missing 194370'17000 (194370'16999) modify 3c9e19de/benchmark_data_storage0111-ib.pocstor_12495_object13363/head//24 by client.1508531.0:13364 2014-08-12 20:31:22.848815 2014-08-13 18:06:06.752571 7f5c973a57a0 15 read_log missing 194370'16998 (194370'16997) modify 46d635de/benchmark_data_storage0111-ib.pocstor_12495_object12735/head//24 by client.1508531.0:12736 2014-08-12 20:31:16.863845 2014-08-13 18:06:06.752607 7f5c973a57a0 15 read_log missing 194370'16996 (194370'16995) modify 5b8f39de/benchmark_data_storage0111-ib.pocstor_12495_object12033/head//24 by client.1508531.0:12034 2014-08-12 20:31:11.595130 2014-08-13 18:06:06.752643 7f5c973a57a0 15 read_log missing 194370'16994 (194370'16993) modify f254c1de/benchmark_data_storage0111-ib.pocstor_12495_object12030/head//24 by client.1508531.0:12031 2014-08-12 20:31:11.574016 2014-08-13 18:06:06.752679 7f5c973a57a0 15 read_log missing 194370'16992 (194370'16991) modify 6d78a5de/benchmark_data_storage0111-ib.pocstor_12495_object11958/head//24 by client.1508531.0:11959 2014-08-12 20:31:11.056711 2014-08-13 18:06:06.752714 7f5c973a57a0 15 read_log missing 194370'16990 (194370'16989) modify 5cdc2dde/benchmark_data_storage0111-ib.pocstor_12495_object11777/head//24 by client.1508531.0:11778 2014-08-12 20:31:09.947609 2014-08-13 18:06:06.752750 7f5c973a57a0 15 read_log missing 194370'16988 (194370'16987) modify df6ca9de/benchmark_data_storage0111-ib.pocstor_12495_object11556/head//24 by client.1508531.0:11557 2014-08-12 20:31:08.650240 2014-08-13 18:06:06.752785 7f5c973a57a0 15 read_log missing 194370'16986 (194370'16985) modify b0ca95de/benchmark_data_storage0111-ib.pocstor_12495_object4821/head//24 by client.1508531.0:4822 2014-08-12 20:30:32.463242 2014-08-13 18:06:06.752820 7f5c973a57a0 15 read_log missing 194370'16984 (194370'16983) modify 8bff9dde/benchmark_data_storage0111-ib.pocstor_12495_object4664/head//24 by client.1508531.0:4665 2014-08-12 20:30:31.867906 2014-08-13 18:06:06.752855 7f5c973a57a0 15 read_log missing 194370'16982 (194370'16981) modify d220f5de/benchmark_data_storage0111-ib.pocstor_12495_object4569/head//24 by client.1508531.0:4570 2014-08-12 20:30:31.488804 2014-08-13 18:06:06.752891 7f5c973a57a0 15 read_log missing 194370'16980 (194370'16979) modify db7af1de/benchmark_data_storage0111-ib.pocstor_12495_object4153/head//24 by client.1508531.0:4154 2014-08-12 20:30:29.875338 2014-08-13 18:06:06.752926 7f5c973a57a0 15 read_log missing 194370'16978 (194370'16977) modify 34959de/benchmark_data_storage0111-ib.pocstor_12495_object3699/head//24 by client.1508531.0:3700 2014-08-12 20:30:28.195281 2014-08-13 18:06:06.752937 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.752964 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive m=48] handle_loaded 2014-08-13 18:06:06.752982 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Initial 0.014490 0 0.000000 2014-08-13 18:06:06.752994 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset 2014-08-13 18:06:06.753004 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=0 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194487 2014-08-13 18:06:06.753014 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] log((192834'15290,194422'17074], crt=194422'17074) 2014-08-13 18:06:06.753025 7f5c973a57a0 10 osd.4 194487 pgid 24.1e4 coll 24.1e4_head 2014-08-13 18:06:06.753184 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.1e4 2014-08-13 18:06:06.753196 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.753228 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e4(unlocked)] enter Initial 2014-08-13 18:06:06.753239 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1e4(unlocked)] enter NotTrimming 2014-08-13 18:06:06.753445 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.753607 7f5c973a57a0 20 read_log 192834'16490 (192834'15999) clean 8b8129e4/benchmark_data_storage0111-ib.pocstor_26659_object7495/head//24 by osd.376.0:18837 2014-08-11 15:26:47.861429 2014-08-13 18:06:06.753629 7f5c973a57a0 20 read_log 192834'16493 (192834'16490) delete 8b8129e4/benchmark_data_storage0111-ib.pocstor_26659_object7495/head//24 by osd.376.0:18840 2014-08-11 15:26:47.930074 2014-08-13 18:06:06.753642 7f5c973a57a0 20 read_log 192834'16494 (192834'16001) clean 652a51e4/benchmark_data_storage0111-ib.pocstor_26659_object7550/head//24 by osd.376.0:18841 2014-08-11 15:26:48.059873 2014-08-13 18:06:06.753655 7f5c973a57a0 20 read_log 192834'16528 (192834'16494) delete 652a51e4/benchmark_data_storage0111-ib.pocstor_26659_object7550/head//24 by osd.376.0:18875 2014-08-11 15:26:53.162875 2014-08-13 18:06:06.753668 7f5c973a57a0 20 read_log 192834'16529 (192834'16088) clean 8f1151e4/benchmark_data_storage0111-ib.pocstor_26659_object10963/head//24 by osd.376.0:18876 2014-08-11 15:26:53.163334 2014-08-13 18:06:06.753681 7f5c973a57a0 20 read_log 192834'16530 (192834'16070) clean 5e9ab1e4/benchmark_data_storage0111-ib.pocstor_26659_object9782/head//24 by osd.376.0:18877 2014-08-11 15:26:53.175662 2014-08-13 18:06:06.753696 7f5c973a57a0 20 read_log 192834'16532 (192834'16033) clean 4e24d5e4/benchmark_data_storage0111-ib.pocstor_26659_object8815/head//24 by osd.376.0:18879 2014-08-11 15:26:53.184051 2014-08-13 18:06:06.753708 7f5c973a57a0 20 read_log 192834'16540 (192834'16011) clean b49e0de4/benchmark_data_storage0111-ib.pocstor_26659_object7890/head//24 by osd.376.0:18887 2014-08-11 15:26:53.227715 2014-08-13 18:06:06.753720 7f5c973a57a0 20 read_log 192834'16541 (192834'16017) clean 6a5f6de4/benchmark_data_storage0111-ib.pocstor_26659_object8143/head//24 by osd.376.0:18888 2014-08-11 15:26:53.232422 2014-08-13 18:06:06.753764 7f5c973a57a0 20 read_log 192834'16563 (192834'16529) delete 8f1151e4/benchmark_data_storage0111-ib.pocstor_26659_object10963/head//24 by osd.376.0:18910 2014-08-11 15:26:53.314990 2014-08-13 18:06:06.753781 7f5c973a57a0 20 read_log 192834'16564 (192834'16530) delete 5e9ab1e4/benchmark_data_storage0111-ib.pocstor_26659_object9782/head//24 by osd.376.0:18911 2014-08-11 15:26:53.318901 2014-08-13 18:06:06.753793 7f5c973a57a0 20 read_log 192834'16566 (192834'16532) delete 4e24d5e4/benchmark_data_storage0111-ib.pocstor_26659_object8815/head//24 by osd.376.0:18913 2014-08-11 15:26:53.324690 2014-08-13 18:06:06.753806 7f5c973a57a0 20 read_log 192834'16574 (192834'16540) delete b49e0de4/benchmark_data_storage0111-ib.pocstor_26659_object7890/head//24 by osd.376.0:18921 2014-08-11 15:26:53.347774 2014-08-13 18:06:06.753818 7f5c973a57a0 20 read_log 192834'16575 (192834'16541) delete 6a5f6de4/benchmark_data_storage0111-ib.pocstor_26659_object8143/head//24 by osd.376.0:18922 2014-08-11 15:26:53.348696 2014-08-13 18:06:06.753831 7f5c973a57a0 20 read_log 192834'16609 (192834'16094) clean 7dd549e4/benchmark_data_storage0111-ib.pocstor_26659_object11222/head//24 by osd.376.0:18956 2014-08-11 15:26:58.507301 2014-08-13 18:06:06.753844 7f5c973a57a0 20 read_log 192834'16610 (192834'16151) clean 6fb079e4/benchmark_data_storage0111-ib.pocstor_26659_object14301/head//24 by osd.376.0:18957 2014-08-11 15:26:58.513974 2014-08-13 18:06:06.753856 7f5c973a57a0 20 read_log 192834'16611 (192834'16135) clean 4e95d9e4/benchmark_data_storage0111-ib.pocstor_26659_object13710/head//24 by osd.376.0:18958 2014-08-11 15:26:58.517977 2014-08-13 18:06:06.753868 7f5c973a57a0 20 read_log 192834'16632 (192834'16609) delete 7dd549e4/benchmark_data_storage0111-ib.pocstor_26659_object11222/head//24 by osd.376.0:18979 2014-08-11 15:26:58.623016 2014-08-13 18:06:06.753880 7f5c973a57a0 20 read_log 192834'16633 (192834'16610) delete 6fb079e4/benchmark_data_storage0111-ib.pocstor_26659_object14301/head//24 by osd.376.0:18980 2014-08-11 15:26:58.626477 2014-08-13 18:06:06.753894 7f5c973a57a0 20 read_log 192834'16634 (192834'16611) delete 4e95d9e4/benchmark_data_storage0111-ib.pocstor_26659_object13710/head//24 by osd.376.0:18981 2014-08-11 15:26:58.631088 2014-08-13 18:06:06.753907 7f5c973a57a0 20 read_log 192834'16644 (0'0) promote ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:18991 2014-08-11 15:27:01.312408 2014-08-13 18:06:06.753921 7f5c973a57a0 20 read_log 192834'16645 (192834'16644) modify ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by client.1883634.0:11798 2014-08-11 15:27:01.303391 2014-08-13 18:06:06.753934 7f5c973a57a0 20 read_log 192834'16656 (0'0) promote ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19003 2014-08-11 15:27:03.576334 2014-08-13 18:06:06.753946 7f5c973a57a0 20 read_log 192834'16657 (192834'16656) modify ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by client.1883634.0:12332 2014-08-11 15:27:03.568932 2014-08-13 18:06:06.753959 7f5c973a57a0 20 read_log 192834'16693 (192834'16242) clean 3aa285e4/benchmark_data_storage0111-ib.pocstor_26659_object20665/head//24 by osd.376.0:19040 2014-08-11 15:27:03.871504 2014-08-13 18:06:06.753971 7f5c973a57a0 20 read_log 192834'16705 (192834'16230) clean 43f20de4/benchmark_data_storage0111-ib.pocstor_26659_object19129/head//24 by osd.376.0:19052 2014-08-11 15:27:03.920400 2014-08-13 18:06:06.753984 7f5c973a57a0 20 read_log 192834'16706 (192834'16247) clean 710f1de4/benchmark_data_storage0111-ib.pocstor_26659_object21474/head//24 by osd.376.0:19053 2014-08-11 15:27:03.923690 2014-08-13 18:06:06.753996 7f5c973a57a0 20 read_log 192834'16707 (192834'16172) clean e9685de4/benchmark_data_storage0111-ib.pocstor_26659_object15613/head//24 by osd.376.0:19054 2014-08-11 15:27:03.932435 2014-08-13 18:06:06.754008 7f5c973a57a0 20 read_log 192834'16708 (192834'16217) clean d41a5de4/benchmark_data_storage0111-ib.pocstor_26659_object17013/head//24 by osd.376.0:19055 2014-08-11 15:27:03.934378 2014-08-13 18:06:06.754029 7f5c973a57a0 20 read_log 192834'16748 (192834'16693) delete 3aa285e4/benchmark_data_storage0111-ib.pocstor_26659_object20665/head//24 by osd.376.0:19095 2014-08-11 15:27:04.029255 2014-08-13 18:06:06.754045 7f5c973a57a0 20 read_log 192834'16760 (192834'16705) delete 43f20de4/benchmark_data_storage0111-ib.pocstor_26659_object19129/head//24 by osd.376.0:19107 2014-08-11 15:27:04.048439 2014-08-13 18:06:06.754058 7f5c973a57a0 20 read_log 192834'16761 (192834'16706) delete 710f1de4/benchmark_data_storage0111-ib.pocstor_26659_object21474/head//24 by osd.376.0:19108 2014-08-11 15:27:04.048798 2014-08-13 18:06:06.754070 7f5c973a57a0 20 read_log 192834'16762 (192834'16707) delete e9685de4/benchmark_data_storage0111-ib.pocstor_26659_object15613/head//24 by osd.376.0:19109 2014-08-11 15:27:04.049381 2014-08-13 18:06:06.754085 7f5c973a57a0 20 read_log 192834'16763 (192834'16708) delete d41a5de4/benchmark_data_storage0111-ib.pocstor_26659_object17013/head//24 by osd.376.0:19110 2014-08-11 15:27:04.052067 2014-08-13 18:06:06.754116 7f5c973a57a0 20 read_log 192834'16772 (0'0) promote b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19119 2014-08-11 15:27:04.630613 2014-08-13 18:06:06.754129 7f5c973a57a0 20 read_log 192834'16773 (192834'16772) modify b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by client.1883634.0:12579 2014-08-11 15:27:04.621518 2014-08-13 18:06:06.754142 7f5c973a57a0 20 read_log 192834'16784 (0'0) promote c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19131 2014-08-11 15:27:09.893547 2014-08-13 18:06:06.754166 7f5c973a57a0 20 read_log 192834'16785 (192834'16784) modify c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by client.1883634.0:13500 2014-08-11 15:27:09.886556 2014-08-13 18:06:06.754179 7f5c973a57a0 20 read_log 192834'16796 (0'0) promote 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19142 2014-08-11 15:27:10.822345 2014-08-13 18:06:06.754191 7f5c973a57a0 20 read_log 192834'16797 (192834'16796) modify 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by client.1883634.0:13794 2014-08-11 15:27:10.814860 2014-08-13 18:06:06.754206 7f5c973a57a0 20 read_log 192834'16820 (192834'16253) clean 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19166 2014-08-11 15:27:19.409687 2014-08-13 18:06:06.754219 7f5c973a57a0 20 read_log 192834'16833 (192834'16255) clean cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19179 2014-08-11 15:27:19.723091 2014-08-13 18:06:06.754233 7f5c973a57a0 20 read_log 192834'16836 (192834'16820) delete 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19182 2014-08-11 15:27:19.741014 2014-08-13 18:06:06.754266 7f5c973a57a0 20 read_log 192834'16841 (192834'16833) delete cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19187 2014-08-11 15:27:19.767551 2014-08-13 18:06:06.754282 7f5c973a57a0 20 read_log 192834'16867 (0'0) promote faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19213 2014-08-11 15:27:27.709173 2014-08-13 18:06:06.754294 7f5c973a57a0 20 read_log 192834'16868 (192834'16867) modify faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by client.1883634.0:16468 2014-08-11 15:27:27.687708 2014-08-13 18:06:06.754307 7f5c973a57a0 20 read_log 192834'16871 (0'0) promote b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19217 2014-08-11 15:27:28.422589 2014-08-13 18:06:06.754321 7f5c973a57a0 20 read_log 192834'16872 (192834'16871) modify b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by client.1883634.0:16603 2014-08-11 15:27:28.414830 2014-08-13 18:06:06.754334 7f5c973a57a0 20 read_log 192834'16907 (192834'16317) clean 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19253 2014-08-11 15:27:35.686832 2014-08-13 18:06:06.754354 7f5c973a57a0 20 read_log 192834'16909 (192834'16907) delete 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19255 2014-08-11 15:27:35.702949 2014-08-13 18:06:06.754369 7f5c973a57a0 20 read_log 192834'16915 (0'0) promote c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19261 2014-08-11 15:27:38.785079 2014-08-13 18:06:06.754381 7f5c973a57a0 20 read_log 192834'16916 (192834'16915) modify c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by client.1883634.0:17735 2014-08-11 15:27:38.776108 2014-08-13 18:06:06.754393 7f5c973a57a0 20 read_log 192834'16954 (192834'16418) clean a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19300 2014-08-11 15:27:46.171596 2014-08-13 18:06:06.754425 7f5c973a57a0 20 read_log 192834'16955 (192834'16351) clean 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19301 2014-08-11 15:27:46.187454 2014-08-13 18:06:06.754438 7f5c973a57a0 20 read_log 192834'16958 (192834'16353) clean 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19304 2014-08-11 15:27:46.262566 2014-08-13 18:06:06.754461 7f5c973a57a0 20 read_log 192834'16961 (192834'16414) clean a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19307 2014-08-11 15:27:46.314392 2014-08-13 18:06:06.754477 7f5c973a57a0 20 read_log 192834'16964 (192834'16954) delete a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19310 2014-08-11 15:27:46.324663 2014-08-13 18:06:06.754489 7f5c973a57a0 20 read_log 192834'16965 (192834'16955) delete 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19311 2014-08-11 15:27:46.328271 2014-08-13 18:06:06.754509 7f5c973a57a0 20 read_log 192834'16968 (192834'16958) delete 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19314 2014-08-11 15:27:46.339635 2014-08-13 18:06:06.754525 7f5c973a57a0 20 read_log 192834'16969 (192834'16961) delete a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19315 2014-08-11 15:27:46.343372 2014-08-13 18:06:06.754538 7f5c973a57a0 20 read_log 192834'16979 (0'0) promote ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:19325 2014-08-11 15:27:48.637354 2014-08-13 18:06:06.754550 7f5c973a57a0 20 read_log 192834'16980 (192834'16979) modify ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by client.1883634.0:18985 2014-08-11 15:27:48.628369 2014-08-13 18:06:06.754563 7f5c973a57a0 20 read_log 192834'17001 (0'0) promote 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:19347 2014-08-11 15:27:53.723432 2014-08-13 18:06:06.754575 7f5c973a57a0 20 read_log 192834'17002 (192834'17001) modify 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by client.1883634.0:19662 2014-08-11 15:27:53.715724 2014-08-13 18:06:06.754593 7f5c973a57a0 20 read_log 192834'17019 (0'0) promote 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by osd.376.0:19365 2014-08-11 15:28:01.637810 2014-08-13 18:06:06.754611 7f5c973a57a0 20 read_log 192834'17020 (192834'17019) modify 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.1883634.0:20814 2014-08-11 15:28:01.628647 2014-08-13 18:06:06.754624 7f5c973a57a0 20 read_log 192834'17026 (192834'16645) clean ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19372 2014-08-11 15:28:01.930424 2014-08-13 18:06:06.754637 7f5c973a57a0 20 read_log 192834'17037 (192834'17026) delete ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19383 2014-08-11 15:28:02.201856 2014-08-13 18:06:06.754649 7f5c973a57a0 20 read_log 192834'17052 (0'0) promote 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:19398 2014-08-11 15:28:04.148687 2014-08-13 18:06:06.754672 7f5c973a57a0 20 read_log 192834'17053 (192834'17052) modify 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by client.1883634.0:21426 2014-08-11 15:28:04.139645 2014-08-13 18:06:06.754694 7f5c973a57a0 20 read_log 192834'17065 (192834'16773) clean b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19411 2014-08-11 15:28:07.412361 2014-08-13 18:06:06.754707 7f5c973a57a0 20 read_log 192834'17066 (192834'16657) clean ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19412 2014-08-11 15:28:07.431144 2014-08-13 18:06:06.754720 7f5c973a57a0 20 read_log 192834'17079 (0'0) promote ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:19425 2014-08-11 15:28:11.376531 2014-08-13 18:06:06.754732 7f5c973a57a0 20 read_log 192834'17080 (192834'17079) modify ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by client.1883634.0:22850 2014-08-11 15:28:11.367862 2014-08-13 18:06:06.754757 7f5c973a57a0 20 read_log 192834'17091 (192834'17065) delete b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19437 2014-08-11 15:28:12.791715 2014-08-13 18:06:06.754770 7f5c973a57a0 20 read_log 192834'17092 (192834'17066) delete ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19438 2014-08-11 15:28:12.798702 2014-08-13 18:06:06.754783 7f5c973a57a0 20 read_log 192834'17093 (192834'16797) clean 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19439 2014-08-11 15:28:12.802150 2014-08-13 18:06:06.754795 7f5c973a57a0 20 read_log 192834'17101 (192834'17093) delete 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19447 2014-08-11 15:28:12.838726 2014-08-13 18:06:06.754807 7f5c973a57a0 20 read_log 192834'17102 (192834'16785) clean c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19448 2014-08-11 15:28:12.887330 2014-08-13 18:06:06.754820 7f5c973a57a0 20 read_log 192834'17105 (0'0) promote 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:19451 2014-08-11 15:28:13.252449 2014-08-13 18:06:06.754832 7f5c973a57a0 20 read_log 192834'17106 (192834'17105) modify 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by client.1883634.0:23171 2014-08-11 15:28:13.244038 2014-08-13 18:06:06.754848 7f5c973a57a0 20 read_log 192834'17107 (0'0) promote 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:19453 2014-08-11 15:28:13.300441 2014-08-13 18:06:06.754865 7f5c973a57a0 20 read_log 192834'17108 (192834'17107) modify 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by client.1883634.0:23179 2014-08-11 15:28:13.291872 2014-08-13 18:06:06.754878 7f5c973a57a0 20 read_log 192834'17113 (0'0) promote 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19459 2014-08-11 15:28:16.236487 2014-08-13 18:06:06.754893 7f5c973a57a0 20 read_log 192834'17114 (192834'17113) modify 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by client.1883634.0:23343 2014-08-11 15:28:16.204144 2014-08-13 18:06:06.754906 7f5c973a57a0 20 read_log 192834'17115 (0'0) promote cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19461 2014-08-11 15:28:16.384055 2014-08-13 18:06:06.754918 7f5c973a57a0 20 read_log 192834'17116 (192834'17115) modify cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by client.1883634.0:23440 2014-08-11 15:28:16.360524 2014-08-13 18:06:06.754931 7f5c973a57a0 20 read_log 192834'17130 (192834'17102) delete c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19476 2014-08-11 15:28:17.981088 2014-08-13 18:06:06.754946 7f5c973a57a0 20 read_log 192834'17201 (0'0) promote 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19547 2014-08-11 15:28:26.103379 2014-08-13 18:06:06.754994 7f5c973a57a0 20 read_log 192834'17202 (192834'17201) modify 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by client.1883634.0:28611 2014-08-11 15:28:26.035642 2014-08-13 18:06:06.755025 7f5c973a57a0 20 read_log 192834'17240 (192834'16868) clean faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19586 2014-08-11 15:28:28.711273 2014-08-13 18:06:06.755056 7f5c973a57a0 20 read_log 192834'17242 (192834'17240) delete faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19588 2014-08-11 15:28:28.719370 2014-08-13 18:06:06.755096 7f5c973a57a0 20 read_log 192834'17246 (192834'16872) clean b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19592 2014-08-11 15:28:28.778307 2014-08-13 18:06:06.755113 7f5c973a57a0 20 read_log 192834'17247 (192834'17246) delete b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19593 2014-08-11 15:28:28.780856 2014-08-13 18:06:06.755136 7f5c973a57a0 20 read_log 192834'17249 (0'0) promote 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19595 2014-08-11 15:28:28.984745 2014-08-13 18:06:06.755155 7f5c973a57a0 20 read_log 192834'17250 (192834'17249) modify 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by client.1883634.0:30612 2014-08-11 15:28:28.952860 2014-08-13 18:06:06.755172 7f5c973a57a0 20 read_log 192834'17251 (0'0) promote 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19597 2014-08-11 15:28:29.092782 2014-08-13 18:06:06.755192 7f5c973a57a0 20 read_log 192834'17252 (192834'17251) modify 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by client.1883634.0:30672 2014-08-11 15:28:29.064254 2014-08-13 18:06:06.755229 7f5c973a57a0 20 read_log 192834'17261 (0'0) promote a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19607 2014-08-11 15:28:30.420688 2014-08-13 18:06:06.755274 7f5c973a57a0 20 read_log 192834'17262 (192834'17261) modify a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by client.1883634.0:31380 2014-08-11 15:28:30.395882 2014-08-13 18:06:06.755296 7f5c973a57a0 20 read_log 192834'17265 (0'0) promote a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19611 2014-08-11 15:28:30.857820 2014-08-13 18:06:06.755315 7f5c973a57a0 20 read_log 192834'17266 (192834'17265) modify a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by client.1883634.0:31677 2014-08-11 15:28:30.826313 2014-08-13 18:06:06.755330 7f5c973a57a0 20 read_log 192834'17310 (0'0) promote ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19656 2014-08-11 15:28:36.087509 2014-08-13 18:06:06.755343 7f5c973a57a0 20 read_log 192834'17311 (192834'17310) modify ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by client.1883634.0:35122 2014-08-11 15:28:36.047511 2014-08-13 18:06:06.755355 7f5c973a57a0 20 read_log 192834'17322 (0'0) promote ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19668 2014-08-11 15:28:36.985354 2014-08-13 18:06:06.755384 7f5c973a57a0 20 read_log 192834'17323 (192834'17322) modify ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by client.1883634.0:35656 2014-08-11 15:28:36.961937 2014-08-13 18:06:06.755400 7f5c973a57a0 20 read_log 192834'17328 (0'0) promote b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19674 2014-08-11 15:28:37.326179 2014-08-13 18:06:06.755423 7f5c973a57a0 20 read_log 192834'17329 (192834'17328) modify b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by client.1883634.0:35903 2014-08-11 15:28:37.295325 2014-08-13 18:06:06.755443 7f5c973a57a0 20 read_log 192834'17340 (0'0) promote c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19686 2014-08-11 15:28:38.484617 2014-08-13 18:06:06.755466 7f5c973a57a0 20 read_log 192834'17341 (192834'17340) modify c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by client.1883634.0:36824 2014-08-11 15:28:38.456979 2014-08-13 18:06:06.755492 7f5c973a57a0 20 read_log 192834'17352 (0'0) promote 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19698 2014-08-11 15:28:38.992190 2014-08-13 18:06:06.755505 7f5c973a57a0 20 read_log 192834'17353 (192834'17352) modify 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by client.1883634.0:37118 2014-08-11 15:28:38.860677 2014-08-13 18:06:06.755520 7f5c973a57a0 20 read_log 192834'17355 (192834'16916) clean c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19701 2014-08-11 15:28:39.065968 2014-08-13 18:06:06.755533 7f5c973a57a0 20 read_log 192834'17386 (0'0) promote faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:19732 2014-08-11 15:28:42.605117 2014-08-13 18:06:06.755549 7f5c973a57a0 20 read_log 192834'17387 (192834'17386) modify faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by client.1883634.0:39792 2014-08-11 15:28:42.576655 2014-08-13 18:06:06.755561 7f5c973a57a0 20 read_log 192834'17390 (0'0) promote b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:19736 2014-08-11 15:28:42.825990 2014-08-13 18:06:06.755576 7f5c973a57a0 20 read_log 192834'17391 (192834'17390) modify b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by client.1883634.0:39927 2014-08-11 15:28:42.800645 2014-08-13 18:06:06.755588 7f5c973a57a0 20 read_log 192834'17402 (192834'17355) delete c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19748 2014-08-11 15:28:44.197904 2014-08-13 18:06:06.755608 7f5c973a57a0 20 read_log 192834'17411 (0'0) promote c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:19757 2014-08-11 15:28:44.594850 2014-08-13 18:06:06.755624 7f5c973a57a0 20 read_log 192834'17412 (192834'17411) modify c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by client.1883634.0:41059 2014-08-11 15:28:44.568519 2014-08-13 18:06:06.755637 7f5c973a57a0 20 read_log 192834'17425 (192834'16980) modify ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by client.1883634.0:42309 2014-08-11 15:28:46.204070 2014-08-13 18:06:06.755649 7f5c973a57a0 20 read_log 192834'17429 (192834'17002) modify 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by client.1883634.0:42986 2014-08-11 15:28:46.423089 2014-08-13 18:06:06.755663 7f5c973a57a0 20 read_log 192834'17433 (192834'17020) modify 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.1883634.0:44138 2014-08-11 15:28:46.773129 2014-08-13 18:06:06.755676 7f5c973a57a0 20 read_log 192834'17438 (192834'17053) modify 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by client.1883634.0:44750 2014-08-11 15:28:46.945642 2014-08-13 18:06:06.755688 7f5c973a57a0 20 read_log 192834'17446 (192834'17080) modify ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by client.1883634.0:46174 2014-08-11 15:28:47.512546 2014-08-13 18:06:06.755701 7f5c973a57a0 20 read_log 192834'17448 (192834'17106) modify 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by client.1883634.0:46495 2014-08-11 15:28:47.610881 2014-08-13 18:06:06.755713 7f5c973a57a0 20 read_log 192834'17449 (192834'17108) modify 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by client.1883634.0:46503 2014-08-11 15:28:47.613451 2014-08-13 18:06:06.755725 7f5c973a57a0 20 read_log 192834'17460 (192834'17114) clean 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19806 2014-08-11 15:29:19.774214 2014-08-13 18:06:06.755738 7f5c973a57a0 20 read_log 192834'17470 (192834'17116) clean cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19816 2014-08-11 15:29:19.845534 2014-08-13 18:06:06.755760 7f5c973a57a0 20 read_log 192834'17475 (192834'17460) delete 81d3f1e4/benchmark_data_storage0111-ib.pocstor_41526_object18/head//24 by osd.376.0:19821 2014-08-11 15:29:19.861850 2014-08-13 18:06:06.755775 7f5c973a57a0 20 read_log 192834'17480 (192834'17470) delete cf79de4/benchmark_data_storage0111-ib.pocstor_41526_object115/head//24 by osd.376.0:19826 2014-08-11 15:29:19.891183 2014-08-13 18:06:06.755788 7f5c973a57a0 20 read_log 192834'17524 (192834'17250) clean 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19870 2014-08-11 15:29:30.224456 2014-08-13 18:06:06.755800 7f5c973a57a0 20 read_log 192834'17527 (192834'17252) clean 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19873 2014-08-11 15:29:30.238163 2014-08-13 18:06:06.755813 7f5c973a57a0 20 read_log 192834'17528 (192834'17202) clean 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19874 2014-08-11 15:29:30.243113 2014-08-13 18:06:06.755825 7f5c973a57a0 20 read_log 192834'17552 (192834'17524) delete 130551e4/benchmark_data_storage0111-ib.pocstor_41526_object7287/head//24 by osd.376.0:19898 2014-08-11 15:29:30.314713 2014-08-13 18:06:06.755837 7f5c973a57a0 20 read_log 192834'17555 (192834'17527) delete 5d3219e4/benchmark_data_storage0111-ib.pocstor_41526_object7347/head//24 by osd.376.0:19901 2014-08-11 15:29:30.326343 2014-08-13 18:06:06.755870 7f5c973a57a0 20 read_log 192834'17556 (192834'17528) delete 7e8e79e4/benchmark_data_storage0111-ib.pocstor_41526_object5286/head//24 by osd.376.0:19902 2014-08-11 15:29:30.329214 2014-08-13 18:06:06.755885 7f5c973a57a0 20 read_log 192834'17574 (192834'17266) clean a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19920 2014-08-11 15:29:35.426326 2014-08-13 18:06:06.755898 7f5c973a57a0 20 read_log 192834'17582 (192834'17262) clean a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19928 2014-08-11 15:29:35.453736 2014-08-13 18:06:06.755910 7f5c973a57a0 20 read_log 192834'17596 (192834'17574) delete a1fd21e4/benchmark_data_storage0111-ib.pocstor_41526_object8352/head//24 by osd.376.0:19942 2014-08-11 15:29:35.489699 2014-08-13 18:06:06.755922 7f5c973a57a0 20 read_log 192834'17603 (192834'17582) delete a9b6c9e4/benchmark_data_storage0111-ib.pocstor_41526_object8055/head//24 by osd.376.0:19949 2014-08-11 15:29:35.502589 2014-08-13 18:06:06.755935 7f5c973a57a0 20 read_log 192834'17619 (192834'17353) clean 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19965 2014-08-11 15:29:40.569744 2014-08-13 18:06:06.755947 7f5c973a57a0 20 read_log 192834'17620 (192834'17311) clean ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19966 2014-08-11 15:29:40.573455 2014-08-13 18:06:06.755960 7f5c973a57a0 20 read_log 192834'17628 (192834'17329) clean b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:19974 2014-08-11 15:29:40.593251 2014-08-13 18:06:06.755974 7f5c973a57a0 20 read_log 192834'17629 (192834'17341) clean c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:19975 2014-08-11 15:29:40.597189 2014-08-13 18:06:06.755987 7f5c973a57a0 20 read_log 192834'17640 (192834'17323) clean ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:19986 2014-08-11 15:29:40.619878 2014-08-13 18:06:06.756001 7f5c973a57a0 20 read_log 192834'17647 (192834'17619) delete 113121e4/benchmark_data_storage0111-ib.pocstor_41526_object13793/head//24 by osd.376.0:19993 2014-08-11 15:29:40.632314 2014-08-13 18:06:06.756013 7f5c973a57a0 20 read_log 192834'17648 (192834'17620) delete ec091e4/benchmark_data_storage0111-ib.pocstor_41526_object11797/head//24 by osd.376.0:19994 2014-08-11 15:29:40.632801 2014-08-13 18:06:06.756028 7f5c973a57a0 20 read_log 192834'17656 (192834'17628) delete b51719e4/benchmark_data_storage0111-ib.pocstor_41526_object12578/head//24 by osd.376.0:20002 2014-08-11 15:29:40.644802 2014-08-13 18:06:06.756063 7f5c973a57a0 20 read_log 192834'17657 (192834'17629) delete c1ac39e4/benchmark_data_storage0111-ib.pocstor_41526_object13499/head//24 by osd.376.0:20003 2014-08-11 15:29:40.646693 2014-08-13 18:06:06.756079 7f5c973a57a0 20 read_log 192834'17660 (192834'17640) delete ef50fde4/benchmark_data_storage0111-ib.pocstor_41526_object12331/head//24 by osd.376.0:20006 2014-08-11 15:29:40.652904 2014-08-13 18:06:06.756092 7f5c973a57a0 20 read_log 192834'17681 (192834'17391) clean b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:20027 2014-08-11 15:29:45.724742 2014-08-13 18:06:06.756106 7f5c973a57a0 20 read_log 192834'17689 (192834'17387) clean faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:20035 2014-08-11 15:29:45.738487 2014-08-13 18:06:06.756119 7f5c973a57a0 20 read_log 192834'17695 (192834'17412) clean c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:20041 2014-08-11 15:29:45.745526 2014-08-13 18:06:06.756131 7f5c973a57a0 20 read_log 192834'17703 (192834'17681) delete b7ff79e4/benchmark_data_storage0111-ib.pocstor_41526_object16602/head//24 by osd.376.0:20049 2014-08-11 15:29:45.755200 2014-08-13 18:06:06.756144 7f5c973a57a0 20 read_log 192834'17706 (192834'17689) delete faa35de4/benchmark_data_storage0111-ib.pocstor_41526_object16467/head//24 by osd.376.0:20052 2014-08-11 15:29:45.759468 2014-08-13 18:06:06.756165 7f5c973a57a0 20 read_log 192834'17707 (192834'17695) delete c739dde4/benchmark_data_storage0111-ib.pocstor_41526_object17734/head//24 by osd.376.0:20053 2014-08-11 15:29:45.759827 2014-08-13 18:06:06.756177 7f5c973a57a0 20 read_log 192834'17739 (192834'17449) clean 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:20085 2014-08-11 15:29:50.810472 2014-08-13 18:06:06.756190 7f5c973a57a0 20 read_log 192834'17740 (192834'17438) clean 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:20086 2014-08-11 15:29:50.810917 2014-08-13 18:06:06.756204 7f5c973a57a0 20 read_log 192834'17748 (192834'17739) delete 178021e4/benchmark_data_storage0111-ib.pocstor_41526_object23178/head//24 by osd.376.0:20094 2014-08-11 15:29:50.822155 2014-08-13 18:06:06.756219 7f5c973a57a0 20 read_log 192834'17749 (192834'17740) delete 91e51e4/benchmark_data_storage0111-ib.pocstor_41526_object21425/head//24 by osd.376.0:20095 2014-08-11 15:29:50.822456 2014-08-13 18:06:06.756231 7f5c973a57a0 20 read_log 192834'17757 (192834'17446) clean ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:20103 2014-08-11 15:29:50.829348 2014-08-13 18:06:06.756246 7f5c973a57a0 20 read_log 192834'17758 (192834'17448) clean 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:20104 2014-08-11 15:29:50.829786 2014-08-13 18:06:06.756258 7f5c973a57a0 20 read_log 192834'17759 (192834'17429) clean 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:20105 2014-08-11 15:29:50.830411 2014-08-13 18:06:06.756272 7f5c973a57a0 20 read_log 192834'17761 (192834'17757) delete ce559e4/benchmark_data_storage0111-ib.pocstor_41526_object22849/head//24 by osd.376.0:20107 2014-08-11 15:29:50.834783 2014-08-13 18:06:06.756288 7f5c973a57a0 20 read_log 192834'17762 (192834'17425) clean ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:20108 2014-08-11 15:29:50.835046 2014-08-13 18:06:06.756301 7f5c973a57a0 20 read_log 192834'17763 (192834'17758) delete 618679e4/benchmark_data_storage0111-ib.pocstor_41526_object23170/head//24 by osd.376.0:20109 2014-08-11 15:29:50.835668 2014-08-13 18:06:06.756313 7f5c973a57a0 20 read_log 192834'17764 (192834'17759) delete 38bec9e4/benchmark_data_storage0111-ib.pocstor_41526_object19661/head//24 by osd.376.0:20110 2014-08-11 15:29:50.836194 2014-08-13 18:06:06.756325 7f5c973a57a0 20 read_log 192834'17765 (192834'17762) delete ef33de4/benchmark_data_storage0111-ib.pocstor_41526_object18984/head//24 by osd.376.0:20111 2014-08-11 15:29:50.837287 2014-08-13 18:06:06.756346 7f5c973a57a0 20 read_log 192888'17766 (192888'17766) modify 1e4/hit_set_24.1e4_archive_2014-08-11 16:03:08.430974_2014-08-11 16:45:02.244560/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:45:02.244796 2014-08-13 18:06:06.756362 7f5c973a57a0 20 read_log 192888'17767 (0'0) promote a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by osd.31.0:2131 2014-08-11 16:45:02.249826 2014-08-13 18:06:06.756374 7f5c973a57a0 20 read_log 192888'17768 (192888'17767) modify a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.1315259.0:1830 2014-08-11 16:45:02.238156 2014-08-13 18:06:06.756387 7f5c973a57a0 20 read_log 192888'17769 (0'0) promote a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by osd.31.0:2135 2014-08-11 16:45:03.925028 2014-08-13 18:06:06.756399 7f5c973a57a0 20 read_log 192888'17770 (192888'17769) modify a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.1315259.0:2285 2014-08-11 16:45:03.918533 2014-08-13 18:06:06.756434 7f5c973a57a0 20 read_log 192888'17781 (0'0) promote 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by osd.31.0:2197 2014-08-11 16:45:15.701034 2014-08-13 18:06:06.756447 7f5c973a57a0 20 read_log 192888'17782 (192888'17781) modify 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.1315259.0:5425 2014-08-11 16:45:15.693713 2014-08-13 18:06:06.756479 7f5c973a57a0 20 read_log 192888'17785 (0'0) promote 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by osd.31.0:2213 2014-08-11 16:45:20.945382 2014-08-13 18:06:06.756492 7f5c973a57a0 20 read_log 192888'17786 (192888'17785) modify 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.1315259.0:6875 2014-08-11 16:45:20.937715 2014-08-13 18:06:06.756512 7f5c973a57a0 20 read_log 192888'17795 (0'0) promote b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by osd.31.0:2241 2014-08-11 16:45:27.396639 2014-08-13 18:06:06.756528 7f5c973a57a0 20 read_log 192888'17796 (192888'17795) modify b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.1315259.0:8617 2014-08-11 16:45:27.389129 2014-08-13 18:06:06.756541 7f5c973a57a0 20 read_log 192888'17797 (0'0) promote e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by osd.31.0:2243 2014-08-11 16:45:27.486089 2014-08-13 18:06:06.756553 7f5c973a57a0 20 read_log 192888'17798 (192888'17797) modify e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.1315259.0:8642 2014-08-11 16:45:27.478748 2014-08-13 18:06:06.756568 7f5c973a57a0 20 read_log 192888'17811 (0'0) promote b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by osd.31.0:2277 2014-08-11 16:45:35.297960 2014-08-13 18:06:06.756583 7f5c973a57a0 20 read_log 192888'17812 (192888'17811) modify b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.1315259.0:10671 2014-08-11 16:45:35.290568 2014-08-13 18:06:06.756596 7f5c973a57a0 20 read_log 192888'17813 (0'0) promote 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by osd.31.0:2283 2014-08-11 16:45:36.150203 2014-08-13 18:06:06.756608 7f5c973a57a0 20 read_log 192888'17814 (192888'17813) modify 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.1315259.0:10883 2014-08-11 16:45:36.143304 2014-08-13 18:06:06.756620 7f5c973a57a0 20 read_log 192888'17827 (0'0) promote a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by osd.31.0:2333 2014-08-11 16:45:46.545990 2014-08-13 18:06:06.756632 7f5c973a57a0 20 read_log 192888'17828 (192888'17827) modify a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.1315259.0:13426 2014-08-11 16:45:46.538181 2014-08-13 18:06:06.756652 7f5c973a57a0 20 read_log 192888'17831 (0'0) promote 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by osd.31.0:2355 2014-08-11 16:45:52.679911 2014-08-13 18:06:06.756667 7f5c973a57a0 20 read_log 192888'17832 (192888'17831) modify 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.1315259.0:15033 2014-08-11 16:45:52.671670 2014-08-13 18:06:06.756679 7f5c973a57a0 20 read_log 192888'17835 (0'0) promote 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by osd.31.0:2381 2014-08-11 16:45:56.717866 2014-08-13 18:06:06.756691 7f5c973a57a0 20 read_log 192888'17836 (192888'17835) modify 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.1315259.0:16075 2014-08-11 16:45:56.706773 2014-08-13 18:06:06.756704 7f5c973a57a0 20 read_log 192888'17837 (0'0) promote 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by osd.31.0:2387 2014-08-11 16:45:58.608924 2014-08-13 18:06:06.756716 7f5c973a57a0 20 read_log 192888'17838 (192888'17837) modify 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.1315259.0:16551 2014-08-11 16:45:58.600675 2014-08-13 18:06:06.756728 7f5c973a57a0 20 read_log 192888'17839 (0'0) promote e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by osd.31.0:2391 2014-08-11 16:45:59.907606 2014-08-13 18:06:06.756740 7f5c973a57a0 20 read_log 192888'17840 (192888'17839) modify e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.1315259.0:16895 2014-08-11 16:45:59.897207 2014-08-13 18:06:06.756753 7f5c973a57a0 20 read_log 192888'17841 (0'0) promote bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by osd.31.0:2393 2014-08-11 16:46:00.681743 2014-08-13 18:06:06.756765 7f5c973a57a0 20 read_log 192888'17842 (192888'17841) modify bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.1315259.0:17081 2014-08-11 16:46:00.674152 2014-08-13 18:06:06.756777 7f5c973a57a0 20 read_log 192888'17843 (0'0) promote 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by osd.31.0:2395 2014-08-11 16:46:00.953151 2014-08-13 18:06:06.756789 7f5c973a57a0 20 read_log 192888'17844 (192888'17843) modify 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.1315259.0:17154 2014-08-11 16:46:00.945287 2014-08-13 18:06:06.756801 7f5c973a57a0 20 read_log 192888'17849 (0'0) promote f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by osd.31.0:2431 2014-08-11 16:46:11.056852 2014-08-13 18:06:06.756813 7f5c973a57a0 20 read_log 192888'17850 (192888'17849) modify f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.1315259.0:19709 2014-08-11 16:46:11.048223 2014-08-13 18:06:06.756825 7f5c973a57a0 20 read_log 192888'17851 (0'0) promote e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by osd.31.0:2443 2014-08-11 16:46:13.260717 2014-08-13 18:06:06.756837 7f5c973a57a0 20 read_log 192888'17852 (192888'17851) modify e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.1315259.0:20309 2014-08-11 16:46:13.251914 2014-08-13 18:06:06.756850 7f5c973a57a0 20 read_log 192888'17865 (0'0) promote 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by osd.31.0:2487 2014-08-11 16:46:18.767931 2014-08-13 18:06:06.756862 7f5c973a57a0 20 read_log 192888'17866 (192888'17865) modify 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.1315259.0:21760 2014-08-11 16:46:18.759811 2014-08-13 18:06:06.756874 7f5c973a57a0 20 read_log 192888'17867 (0'0) promote 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by osd.31.0:2489 2014-08-11 16:46:18.771884 2014-08-13 18:06:06.756886 7f5c973a57a0 20 read_log 192888'17868 (192888'17867) modify 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.1315259.0:21761 2014-08-11 16:46:18.763293 2014-08-13 18:06:06.756910 7f5c973a57a0 20 read_log 192888'17871 (0'0) promote d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by osd.31.0:2499 2014-08-11 16:46:22.435297 2014-08-13 18:06:06.756925 7f5c973a57a0 20 read_log 192888'17872 (192888'17871) modify d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.1315259.0:22704 2014-08-11 16:46:22.427213 2014-08-13 18:06:06.756938 7f5c973a57a0 20 read_log 192888'17873 (0'0) promote 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by osd.31.0:2509 2014-08-11 16:46:26.731426 2014-08-13 18:06:06.756952 7f5c973a57a0 20 read_log 192888'17874 (192888'17873) modify 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.1315259.0:23760 2014-08-11 16:46:26.722464 2014-08-13 18:06:06.756964 7f5c973a57a0 20 read_log 192888'17875 (0'0) promote 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by osd.31.0:2511 2014-08-11 16:46:27.611115 2014-08-13 18:06:06.756976 7f5c973a57a0 20 read_log 192888'17876 (192888'17875) modify 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.1315259.0:23993 2014-08-11 16:46:27.601262 2014-08-13 18:06:06.756988 7f5c973a57a0 20 read_log 192888'17877 (0'0) promote c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by osd.31.0:2521 2014-08-11 16:46:28.912507 2014-08-13 18:06:06.757000 7f5c973a57a0 20 read_log 192888'17878 (192888'17877) modify c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.1315259.0:24340 2014-08-11 16:46:28.905836 2014-08-13 18:06:06.757015 7f5c973a57a0 20 read_log 192888'17879 (0'0) promote 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by osd.31.0:2523 2014-08-11 16:46:30.408840 2014-08-13 18:06:06.757027 7f5c973a57a0 20 read_log 192888'17880 (192888'17879) modify 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.1315259.0:24711 2014-08-11 16:46:30.401385 2014-08-13 18:06:06.757042 7f5c973a57a0 20 read_log 192888'17883 (0'0) promote 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by osd.31.0:2539 2014-08-11 16:46:35.933006 2014-08-13 18:06:06.757054 7f5c973a57a0 20 read_log 192888'17884 (192888'17883) modify 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.1315259.0:26054 2014-08-11 16:46:35.923801 2014-08-13 18:06:06.757068 7f5c973a57a0 20 read_log 192888'17885 (0'0) promote 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by osd.31.0:2553 2014-08-11 16:46:38.241385 2014-08-13 18:06:06.757080 7f5c973a57a0 20 read_log 192888'17886 (192888'17885) modify 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.1315259.0:26590 2014-08-11 16:46:38.233633 2014-08-13 18:06:06.757095 7f5c973a57a0 20 read_log 192888'17893 (0'0) promote 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by osd.31.0:2569 2014-08-11 16:46:43.733048 2014-08-13 18:06:06.757107 7f5c973a57a0 20 read_log 192888'17894 (192888'17893) modify 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.1315259.0:27942 2014-08-11 16:46:43.726080 2014-08-13 18:06:06.757121 7f5c973a57a0 20 read_log 192888'17897 (0'0) promote 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by osd.31.0:2585 2014-08-11 16:46:45.749028 2014-08-13 18:06:06.757134 7f5c973a57a0 20 read_log 192888'17898 (192888'17897) modify 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.1315259.0:28426 2014-08-11 16:46:45.740544 2014-08-13 18:06:06.757153 7f5c973a57a0 20 read_log 192888'17901 (0'0) promote 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by osd.31.0:2593 2014-08-11 16:46:46.828513 2014-08-13 18:06:06.757168 7f5c973a57a0 20 read_log 192888'17902 (192888'17901) modify 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.1315259.0:28687 2014-08-11 16:46:46.820907 2014-08-13 18:06:06.757188 7f5c973a57a0 20 read_log 192888'17903 (0'0) promote bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by osd.31.0:2601 2014-08-11 16:46:50.252011 2014-08-13 18:06:06.757221 7f5c973a57a0 20 read_log 192888'17904 (192888'17903) modify bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.1315259.0:29528 2014-08-11 16:46:50.244491 2014-08-13 18:06:06.757236 7f5c973a57a0 20 read_log 192888'17911 (192888'17768) modify a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.1315259.0:32557 2014-08-11 16:46:56.409629 2014-08-13 18:06:06.757249 7f5c973a57a0 20 read_log 192888'17912 (192888'17770) modify a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.1315259.0:33012 2014-08-11 16:46:56.531923 2014-08-13 18:06:06.757261 7f5c973a57a0 20 read_log 192888'17918 (192888'17782) modify 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.1315259.0:36152 2014-08-11 16:46:57.350510 2014-08-13 18:06:06.757274 7f5c973a57a0 20 read_log 192888'17920 (192888'17786) modify 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.1315259.0:37602 2014-08-11 16:46:57.727081 2014-08-13 18:06:06.757292 7f5c973a57a0 20 read_log 192888'17925 (192888'17796) modify b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.1315259.0:39344 2014-08-11 16:46:58.187124 2014-08-13 18:06:06.757304 7f5c973a57a0 20 read_log 192888'17926 (192888'17798) modify e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.1315259.0:39369 2014-08-11 16:46:58.192959 2014-08-13 18:06:06.757317 7f5c973a57a0 20 read_log 192888'17933 (192888'17812) modify b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.1315259.0:41398 2014-08-11 16:46:58.735798 2014-08-13 18:06:06.757332 7f5c973a57a0 20 read_log 192888'17934 (192888'17814) modify 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.1315259.0:41610 2014-08-11 16:46:58.790061 2014-08-13 18:06:06.757347 7f5c973a57a0 20 read_log 192888'17941 (192888'17828) modify a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.1315259.0:44153 2014-08-11 16:46:59.496286 2014-08-13 18:06:06.757359 7f5c973a57a0 20 read_log 192888'17943 (192888'17832) modify 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.1315259.0:45760 2014-08-11 16:46:59.961449 2014-08-13 18:06:06.757372 7f5c973a57a0 20 read_log 192888'17945 (192888'17836) modify 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.1315259.0:46802 2014-08-11 16:47:00.262295 2014-08-13 18:06:06.757384 7f5c973a57a0 20 read_log 192888'17946 (192888'17838) modify 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.1315259.0:47278 2014-08-11 16:47:00.403793 2014-08-13 18:06:06.757407 7f5c973a57a0 20 read_log 192888'17947 (192888'17840) modify e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.1315259.0:47622 2014-08-11 16:47:00.513336 2014-08-13 18:06:06.757423 7f5c973a57a0 20 read_log 192888'17948 (192888'17842) modify bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.1315259.0:47808 2014-08-11 16:47:00.572772 2014-08-13 18:06:06.757436 7f5c973a57a0 20 read_log 192888'17949 (192888'17844) modify 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.1315259.0:47881 2014-08-11 16:47:00.597560 2014-08-13 18:06:06.757448 7f5c973a57a0 20 read_log 192888'17952 (192888'17850) modify f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.1315259.0:50436 2014-08-11 16:47:01.369122 2014-08-13 18:06:06.757472 7f5c973a57a0 20 read_log 192888'17953 (192888'17852) modify e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.1315259.0:51036 2014-08-11 16:47:01.559397 2014-08-13 18:06:06.757485 7f5c973a57a0 20 read_log 192888'17960 (192888'17866) modify 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.1315259.0:52487 2014-08-11 16:47:01.987003 2014-08-13 18:06:06.757521 7f5c973a57a0 20 read_log 192888'17961 (192888'17868) modify 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.1315259.0:52488 2014-08-11 16:47:01.987109 2014-08-13 18:06:06.757537 7f5c973a57a0 20 read_log 192888'17963 (192888'17872) modify d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.1315259.0:53431 2014-08-11 16:47:02.249621 2014-08-13 18:06:06.757557 7f5c973a57a0 20 read_log 192888'17964 (192888'17874) modify 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.1315259.0:54487 2014-08-11 16:47:02.579560 2014-08-13 18:06:06.757574 7f5c973a57a0 20 read_log 192888'17965 (192888'17876) modify 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.1315259.0:54720 2014-08-11 16:47:02.665717 2014-08-13 18:06:06.757587 7f5c973a57a0 20 read_log 192888'17966 (192888'17878) modify c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.1315259.0:55067 2014-08-11 16:47:02.801493 2014-08-13 18:06:06.757599 7f5c973a57a0 20 read_log 192888'17967 (192888'17880) modify 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.1315259.0:55438 2014-08-11 16:47:02.912795 2014-08-13 18:06:06.757612 7f5c973a57a0 20 read_log 192888'17969 (192888'17884) modify 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.1315259.0:56781 2014-08-11 16:47:03.310461 2014-08-13 18:06:06.757632 7f5c973a57a0 20 read_log 192888'17970 (192888'17886) modify 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.1315259.0:57317 2014-08-11 16:47:03.474301 2014-08-13 18:06:06.757648 7f5c973a57a0 20 read_log 192888'17974 (192888'17894) modify 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.1315259.0:58669 2014-08-11 16:47:03.869734 2014-08-13 18:06:06.757660 7f5c973a57a0 20 read_log 192888'17976 (192888'17898) modify 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.1315259.0:59153 2014-08-11 16:47:04.017876 2014-08-13 18:06:06.757676 7f5c973a57a0 20 read_log 192888'17978 (192888'17902) modify 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.1315259.0:59414 2014-08-11 16:47:04.107668 2014-08-13 18:06:06.757691 7f5c973a57a0 20 read_log 192888'17979 (192888'17904) modify bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.1315259.0:60255 2014-08-11 16:47:04.394547 2014-08-13 18:06:06.757704 7f5c973a57a0 20 read_log 192888'17983 (192888'17983) modify 1e4/hit_set_24.1e4_archive_2014-08-11 16:45:02.244800_2014-08-11 17:05:16.516717/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.516943 2014-08-13 18:06:06.757717 7f5c973a57a0 20 read_log 192888'17984 (192888'17766) delete 1e4/hit_set_24.1e4_archive_2014-08-11 16:03:08.430974_2014-08-11 16:45:02.244560/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:16.516943 2014-08-13 18:06:06.757751 7f5c973a57a0 20 read_log 192888'17985 (0'0) promote c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by osd.31.0:2894 2014-08-11 17:05:16.520366 2014-08-13 18:06:06.757763 7f5c973a57a0 20 read_log 192888'17986 (192888'17985) modify c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.942868.0:359 2014-08-11 17:05:16.509882 2014-08-13 18:06:06.757776 7f5c973a57a0 20 read_log 192888'17987 (0'0) promote 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by osd.31.0:2899 2014-08-11 17:05:17.931406 2014-08-13 18:06:06.757788 7f5c973a57a0 20 read_log 192888'17988 (192888'17987) modify 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.942868.0:722 2014-08-11 17:05:17.923502 2014-08-13 18:06:06.757812 7f5c973a57a0 20 read_log 192888'17989 (0'0) promote 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by osd.31.0:2901 2014-08-11 17:05:18.704982 2014-08-13 18:06:06.757833 7f5c973a57a0 20 read_log 192888'17990 (192888'17989) modify 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.942868.0:901 2014-08-11 17:05:18.697764 2014-08-13 18:06:06.757848 7f5c973a57a0 20 read_log 192888'17993 (0'0) promote d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by osd.31.0:2914 2014-08-11 17:05:20.035892 2014-08-13 18:06:06.757861 7f5c973a57a0 20 read_log 192888'17994 (192888'17993) modify d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.942868.0:1218 2014-08-11 17:05:20.027053 2014-08-13 18:06:06.757873 7f5c973a57a0 20 read_log 192888'17997 (0'0) promote abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by osd.31.0:2944 2014-08-11 17:05:31.250967 2014-08-13 18:06:06.757885 7f5c973a57a0 20 read_log 192888'17998 (192888'17997) modify abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.942868.0:4140 2014-08-11 17:05:31.243738 2014-08-13 18:06:06.757898 7f5c973a57a0 20 read_log 192888'18003 (0'0) promote d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by osd.31.0:2972 2014-08-11 17:05:39.982775 2014-08-13 18:06:06.757910 7f5c973a57a0 20 read_log 192888'18004 (192888'18003) modify d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.942868.0:6462 2014-08-11 17:05:39.977011 2014-08-13 18:06:06.757922 7f5c973a57a0 20 read_log 192888'18009 (0'0) promote b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by osd.31.0:2998 2014-08-11 17:05:46.794550 2014-08-13 18:06:06.757935 7f5c973a57a0 20 read_log 192888'18010 (192888'18009) modify b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.942868.0:8267 2014-08-11 17:05:46.783250 2014-08-13 18:06:06.757947 7f5c973a57a0 20 read_log 192888'18015 (0'0) promote 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by osd.31.0:3026 2014-08-11 17:05:51.874354 2014-08-13 18:06:06.757959 7f5c973a57a0 20 read_log 192888'18016 (192888'18015) modify 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.942868.0:9632 2014-08-11 17:05:51.866563 2014-08-13 18:06:06.757972 7f5c973a57a0 20 read_log 192888'18017 (0'0) promote 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by osd.31.0:3028 2014-08-11 17:05:52.189616 2014-08-13 18:06:06.757984 7f5c973a57a0 20 read_log 192888'18018 (192888'18017) modify 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.942868.0:9720 2014-08-11 17:05:52.182881 2014-08-13 18:06:06.757997 7f5c973a57a0 20 read_log 192888'18019 (0'0) promote 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by osd.31.0:3032 2014-08-11 17:05:53.004278 2014-08-13 18:06:06.758009 7f5c973a57a0 20 read_log 192888'18020 (192888'18019) modify 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.942868.0:9944 2014-08-11 17:05:52.996689 2014-08-13 18:06:06.758021 7f5c973a57a0 20 read_log 192888'18021 (0'0) promote 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by osd.31.0:3056 2014-08-11 17:05:58.906902 2014-08-13 18:06:06.758033 7f5c973a57a0 20 read_log 192888'18022 (192888'18021) modify 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.942868.0:11566 2014-08-11 17:05:58.900323 2014-08-13 18:06:06.758048 7f5c973a57a0 20 read_log 192888'18025 (0'0) promote e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by osd.31.0:3066 2014-08-11 17:06:02.031120 2014-08-13 18:06:06.758061 7f5c973a57a0 20 read_log 192888'18026 (192888'18025) modify e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.942868.0:12426 2014-08-11 17:06:02.023702 2014-08-13 18:06:06.758075 7f5c973a57a0 20 read_log 192888'18027 (0'0) promote db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by osd.31.0:3068 2014-08-11 17:06:02.190415 2014-08-13 18:06:06.758111 7f5c973a57a0 20 read_log 192888'18028 (192888'18027) modify db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.942868.0:12471 2014-08-11 17:06:02.183824 2014-08-13 18:06:06.758127 7f5c973a57a0 20 read_log 192888'18033 (0'0) promote e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by osd.31.0:3078 2014-08-11 17:06:04.506098 2014-08-13 18:06:06.758139 7f5c973a57a0 20 read_log 192888'18034 (192888'18033) modify e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.942868.0:13088 2014-08-11 17:06:04.498535 2014-08-13 18:06:06.758164 7f5c973a57a0 20 read_log 192888'18037 (0'0) promote c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by osd.31.0:3090 2014-08-11 17:06:08.232513 2014-08-13 18:06:06.758179 7f5c973a57a0 20 read_log 192888'18038 (192888'18037) modify c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.942868.0:14009 2014-08-11 17:06:08.224609 2014-08-13 18:06:06.758192 7f5c973a57a0 20 read_log 192888'18041 (0'0) promote 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by osd.31.0:3118 2014-08-11 17:06:15.584015 2014-08-13 18:06:06.758204 7f5c973a57a0 20 read_log 192888'18042 (192888'18041) modify 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.942868.0:15864 2014-08-11 17:06:15.576443 2014-08-13 18:06:06.758216 7f5c973a57a0 20 read_log 192888'18043 (0'0) promote c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by osd.31.0:3122 2014-08-11 17:06:16.620605 2014-08-13 18:06:06.758230 7f5c973a57a0 20 read_log 192888'18044 (192888'18043) modify c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.942868.0:16133 2014-08-11 17:06:16.613363 2014-08-13 18:06:06.758243 7f5c973a57a0 20 read_log 192888'18047 (0'0) promote 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by osd.31.0:3140 2014-08-11 17:06:19.850972 2014-08-13 18:06:06.758257 7f5c973a57a0 20 read_log 192888'18048 (192888'18047) modify 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.942868.0:16944 2014-08-11 17:06:19.843399 2014-08-13 18:06:06.758274 7f5c973a57a0 20 read_log 192888'18049 (0'0) promote 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by osd.31.0:3144 2014-08-11 17:06:20.941598 2014-08-13 18:06:06.758289 7f5c973a57a0 20 read_log 192888'18050 (192888'18049) modify 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.942868.0:17232 2014-08-11 17:06:20.933591 2014-08-13 18:06:06.758302 7f5c973a57a0 20 read_log 192888'18053 (0'0) promote e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by osd.31.0:3152 2014-08-11 17:06:22.978611 2014-08-13 18:06:06.758314 7f5c973a57a0 20 read_log 192888'18054 (192888'18053) modify e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.942868.0:17772 2014-08-11 17:06:22.970337 2014-08-13 18:06:06.758326 7f5c973a57a0 20 read_log 192888'18057 (0'0) promote 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by osd.31.0:3172 2014-08-11 17:06:28.344584 2014-08-13 18:06:06.758338 7f5c973a57a0 20 read_log 192888'18058 (192888'18057) modify 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.942868.0:19197 2014-08-11 17:06:28.336943 2014-08-13 18:06:06.758354 7f5c973a57a0 20 read_log 192888'18059 (0'0) promote 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by osd.31.0:3192 2014-08-11 17:06:31.183532 2014-08-13 18:06:06.758369 7f5c973a57a0 20 read_log 192888'18060 (192888'18059) modify 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.942868.0:19950 2014-08-11 17:06:31.172853 2014-08-13 18:06:06.758382 7f5c973a57a0 20 read_log 192888'18071 (0'0) promote ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by osd.31.0:3246 2014-08-11 17:06:51.076772 2014-08-13 18:06:06.758408 7f5c973a57a0 20 read_log 192888'18072 (192888'18071) modify ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.942868.0:25178 2014-08-11 17:06:51.069971 2014-08-13 18:06:06.758423 7f5c973a57a0 20 read_log 192888'18075 (0'0) promote 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by osd.31.0:3252 2014-08-11 17:06:52.606360 2014-08-13 18:06:06.758436 7f5c973a57a0 20 read_log 192888'18076 (192888'18075) modify 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.942868.0:25581 2014-08-11 17:06:52.599199 2014-08-13 18:06:06.758448 7f5c973a57a0 20 read_log 192888'18077 (0'0) promote 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by osd.31.0:3260 2014-08-11 17:06:54.846843 2014-08-13 18:06:06.758470 7f5c973a57a0 20 read_log 192888'18078 (192888'18077) modify 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.942868.0:26102 2014-08-11 17:06:54.838639 2014-08-13 18:06:06.758484 7f5c973a57a0 20 read_log 192888'18079 (0'0) promote 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by osd.31.0:3264 2014-08-11 17:06:55.006914 2014-08-13 18:06:06.758496 7f5c973a57a0 20 read_log 192888'18080 (192888'18079) modify 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.942868.0:26147 2014-08-11 17:06:54.999961 2014-08-13 18:06:06.758519 7f5c973a57a0 20 read_log 192888'18081 (0'0) promote 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by osd.31.0:3268 2014-08-11 17:06:56.109580 2014-08-13 18:06:06.758532 7f5c973a57a0 20 read_log 192888'18082 (192888'18081) modify 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.942868.0:26443 2014-08-11 17:06:56.101921 2014-08-13 18:06:06.758547 7f5c973a57a0 20 read_log 192888'18089 (0'0) promote 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by osd.31.0:3294 2014-08-11 17:07:01.184082 2014-08-13 18:06:06.758562 7f5c973a57a0 20 read_log 192888'18090 (192888'18089) modify 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.942868.0:27779 2014-08-11 17:07:01.174097 2014-08-13 18:06:06.758574 7f5c973a57a0 20 read_log 192888'18099 (0'0) promote 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by osd.31.0:3324 2014-08-11 17:07:08.453502 2014-08-13 18:06:06.758588 7f5c973a57a0 20 read_log 192888'18100 (192888'18099) modify 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.942868.0:29660 2014-08-11 17:07:08.445512 2014-08-13 18:06:06.758600 7f5c973a57a0 20 read_log 192888'18101 (0'0) promote e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by osd.31.0:3330 2014-08-11 17:07:09.821007 2014-08-13 18:06:06.758612 7f5c973a57a0 20 read_log 192888'18102 (192888'18101) modify e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.942868.0:30002 2014-08-11 17:07:09.811834 2014-08-13 18:06:06.758627 7f5c973a57a0 20 read_log 192888'18103 (0'0) promote b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by osd.31.0:3332 2014-08-11 17:07:10.853302 2014-08-13 18:06:06.758642 7f5c973a57a0 20 read_log 192888'18104 (192888'18103) modify b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.942868.0:30252 2014-08-11 17:07:10.845105 2014-08-13 18:06:06.758654 7f5c973a57a0 20 read_log 192888'18105 (192888'17986) modify c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.942868.0:31625 2014-08-11 17:07:16.001800 2014-08-13 18:06:06.758667 7f5c973a57a0 20 read_log 192888'18106 (192888'17988) modify 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.942868.0:31988 2014-08-11 17:07:16.118175 2014-08-13 18:06:06.758679 7f5c973a57a0 20 read_log 192888'18107 (192888'17990) modify 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.942868.0:32167 2014-08-11 17:07:16.172482 2014-08-13 18:06:06.758707 7f5c973a57a0 20 read_log 192888'18109 (192888'17994) modify d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.942868.0:32484 2014-08-11 17:07:16.267295 2014-08-13 18:06:06.758724 7f5c973a57a0 20 read_log 192888'18111 (192888'17998) modify abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.942868.0:35406 2014-08-11 17:07:17.065860 2014-08-13 18:06:06.758737 7f5c973a57a0 20 read_log 192888'18114 (192888'18004) modify d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.942868.0:37728 2014-08-11 17:07:17.690511 2014-08-13 18:06:06.758749 7f5c973a57a0 20 read_log 192888'18117 (192888'18010) modify b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.942868.0:39533 2014-08-11 17:07:18.209032 2014-08-13 18:06:06.758761 7f5c973a57a0 20 read_log 192888'18120 (192888'18016) modify 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.942868.0:40898 2014-08-11 17:07:18.576245 2014-08-13 18:06:06.758774 7f5c973a57a0 20 read_log 192888'18121 (192888'18018) modify 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.942868.0:40986 2014-08-11 17:07:18.600033 2014-08-13 18:06:06.758786 7f5c973a57a0 20 read_log 192888'18122 (192888'18020) modify 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.942868.0:41210 2014-08-11 17:07:18.671024 2014-08-13 18:06:06.758798 7f5c973a57a0 20 read_log 192888'18123 (192888'18022) modify 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.942868.0:42832 2014-08-11 17:07:19.104278 2014-08-13 18:06:06.758810 7f5c973a57a0 20 read_log 192888'18125 (192888'18026) modify e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.942868.0:43692 2014-08-11 17:07:19.348155 2014-08-13 18:06:06.758823 7f5c973a57a0 20 read_log 192888'18126 (192888'18028) modify db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.942868.0:43737 2014-08-11 17:07:19.360506 2014-08-13 18:06:06.758835 7f5c973a57a0 20 read_log 192888'18129 (192888'18034) modify e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.942868.0:44354 2014-08-11 17:07:19.544499 2014-08-13 18:06:06.758847 7f5c973a57a0 20 read_log 192888'18131 (192888'18038) modify c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.942868.0:45275 2014-08-11 17:07:19.793275 2014-08-13 18:06:06.758859 7f5c973a57a0 20 read_log 192888'18133 (192888'18042) modify 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.942868.0:47130 2014-08-11 17:07:20.317891 2014-08-13 18:06:06.758874 7f5c973a57a0 20 read_log 192888'18134 (192888'18044) modify c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.942868.0:47399 2014-08-11 17:07:20.393492 2014-08-13 18:06:06.758886 7f5c973a57a0 20 read_log 192888'18136 (192888'18048) modify 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.942868.0:48210 2014-08-11 17:07:20.655744 2014-08-13 18:06:06.758900 7f5c973a57a0 20 read_log 192888'18137 (192888'18050) modify 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.942868.0:48498 2014-08-11 17:07:20.735290 2014-08-13 18:06:06.758912 7f5c973a57a0 20 read_log 192888'18139 (192888'18054) modify e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.942868.0:49038 2014-08-11 17:07:20.890049 2014-08-13 18:06:06.758927 7f5c973a57a0 20 read_log 192888'18141 (192888'18058) modify 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.942868.0:50463 2014-08-11 17:07:21.326204 2014-08-13 18:06:06.758939 7f5c973a57a0 20 read_log 192888'18142 (192888'18060) modify 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.942868.0:51216 2014-08-11 17:07:21.574852 2014-08-13 18:06:06.758954 7f5c973a57a0 20 read_log 192888'18148 (192888'18072) modify ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.942868.0:56444 2014-08-11 17:07:23.150459 2014-08-13 18:06:06.758974 7f5c973a57a0 20 read_log 192888'18150 (192888'18076) modify 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.942868.0:56847 2014-08-11 17:07:23.277623 2014-08-13 18:06:06.758989 7f5c973a57a0 20 read_log 192888'18151 (192888'18078) modify 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.942868.0:57368 2014-08-11 17:07:23.435994 2014-08-13 18:06:06.759001 7f5c973a57a0 20 read_log 192888'18152 (192888'18080) modify 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.942868.0:57413 2014-08-11 17:07:23.448283 2014-08-13 18:06:06.759014 7f5c973a57a0 20 read_log 192888'18153 (192888'18082) modify 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.942868.0:57709 2014-08-11 17:07:23.534515 2014-08-13 18:06:06.759026 7f5c973a57a0 20 read_log 192888'18157 (192888'18090) modify 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.942868.0:59045 2014-08-11 17:07:23.927890 2014-08-13 18:06:06.759040 7f5c973a57a0 20 read_log 192888'18162 (192888'18100) modify 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.942868.0:60926 2014-08-11 17:07:24.485568 2014-08-13 18:06:06.759053 7f5c973a57a0 20 read_log 192888'18163 (192888'18102) modify e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.942868.0:61268 2014-08-11 17:07:24.589518 2014-08-13 18:06:06.759067 7f5c973a57a0 20 read_log 192888'18164 (192888'18104) modify b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.942868.0:61518 2014-08-11 17:07:24.668425 2014-08-13 18:06:06.759079 7f5c973a57a0 20 read_log 193305'18165 (193305'18165) modify 1e4/hit_set_24.1e4_archive_2014-08-11 18:37:04.917636_2014-08-11 18:42:34.237231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:34.237452 2014-08-13 18:06:06.759094 7f5c973a57a0 20 read_log 193305'18166 (192888'17983) delete 1e4/hit_set_24.1e4_archive_2014-08-11 16:45:02.244800_2014-08-11 17:05:16.516717/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:34.237452 2014-08-13 18:06:06.759107 7f5c973a57a0 20 read_log 193305'18167 (0'0) promote 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by osd.31.0:7 2014-08-11 18:42:34.267899 2014-08-13 18:06:06.759121 7f5c973a57a0 20 read_log 193305'18168 (193305'18167) modify 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.951337.0:331 2014-08-11 18:42:34.231959 2014-08-13 18:06:06.759134 7f5c973a57a0 20 read_log 193305'18169 (0'0) promote c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by osd.31.0:16 2014-08-11 18:42:37.010307 2014-08-13 18:06:06.759153 7f5c973a57a0 20 read_log 193305'18170 (193305'18169) modify c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.951337.0:1010 2014-08-11 18:42:37.003295 2014-08-13 18:06:06.759168 7f5c973a57a0 20 read_log 193305'18171 (0'0) promote d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by osd.31.0:58 2014-08-11 18:42:46.630320 2014-08-13 18:06:06.759181 7f5c973a57a0 20 read_log 193305'18172 (193305'18171) modify d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.951337.0:3618 2014-08-11 18:42:46.623841 2014-08-13 18:06:06.759193 7f5c973a57a0 20 read_log 193305'18173 (0'0) promote 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by osd.31.0:64 2014-08-11 18:42:47.990527 2014-08-13 18:06:06.759205 7f5c973a57a0 20 read_log 193305'18174 (193305'18173) modify 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.951337.0:3990 2014-08-11 18:42:47.982858 2014-08-13 18:06:06.759220 7f5c973a57a0 20 read_log 193305'18175 (0'0) promote 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by osd.31.0:66 2014-08-11 18:42:48.559598 2014-08-13 18:06:06.759241 7f5c973a57a0 20 read_log 193305'18176 (193305'18175) modify 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.951337.0:4147 2014-08-11 18:42:48.551890 2014-08-13 18:06:06.759277 7f5c973a57a0 20 read_log 193305'18177 (0'0) promote d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by osd.31.0:70 2014-08-11 18:42:48.912991 2014-08-13 18:06:06.759294 7f5c973a57a0 20 read_log 193305'18178 (193305'18177) modify d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.951337.0:4244 2014-08-11 18:42:48.905748 2014-08-13 18:06:06.759307 7f5c973a57a0 20 read_log 193305'18179 (0'0) promote f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by osd.31.0:74 2014-08-11 18:42:49.095467 2014-08-13 18:06:06.759319 7f5c973a57a0 20 read_log 193305'18180 (193305'18179) modify f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.951337.0:4294 2014-08-11 18:42:49.086941 2014-08-13 18:06:06.759332 7f5c973a57a0 20 read_log 193305'18181 (0'0) promote 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by osd.31.0:106 2014-08-11 18:42:56.453014 2014-08-13 18:06:06.759345 7f5c973a57a0 20 read_log 193305'18182 (193305'18181) modify 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.951337.0:6282 2014-08-11 18:42:56.445764 2014-08-13 18:06:06.759359 7f5c973a57a0 20 read_log 193305'18183 (0'0) promote d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by osd.31.0:138 2014-08-11 18:43:11.196706 2014-08-13 18:06:06.759372 7f5c973a57a0 20 read_log 193305'18184 (193305'18183) modify d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.951337.0:10250 2014-08-11 18:43:11.188237 2014-08-13 18:06:06.759384 7f5c973a57a0 20 read_log 193305'18185 (0'0) promote 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by osd.31.0:158 2014-08-11 18:43:18.505721 2014-08-13 18:06:06.759396 7f5c973a57a0 20 read_log 193305'18186 (193305'18185) modify 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.951337.0:12168 2014-08-11 18:43:18.498819 2014-08-13 18:06:06.759418 7f5c973a57a0 20 read_log 193305'18187 (0'0) promote bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by osd.31.0:160 2014-08-11 18:43:20.297074 2014-08-13 18:06:06.759431 7f5c973a57a0 20 read_log 193305'18188 (193305'18187) modify bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.951337.0:12654 2014-08-11 18:43:20.288746 2014-08-13 18:06:06.759444 7f5c973a57a0 20 read_log 193305'18189 (0'0) promote 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by osd.31.0:162 2014-08-11 18:43:22.848666 2014-08-13 18:06:06.759471 7f5c973a57a0 20 read_log 193305'18190 (193305'18189) modify 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.951337.0:13326 2014-08-11 18:43:22.841204 2014-08-13 18:06:06.759487 7f5c973a57a0 20 read_log 193305'18191 (0'0) promote 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by osd.31.0:164 2014-08-11 18:43:22.978077 2014-08-13 18:06:06.759509 7f5c973a57a0 20 read_log 193305'18192 (193305'18191) modify 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.951337.0:13362 2014-08-11 18:43:22.969970 2014-08-13 18:06:06.759525 7f5c973a57a0 20 read_log 193305'18193 (0'0) promote 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by osd.31.0:170 2014-08-11 18:43:23.432693 2014-08-13 18:06:06.759542 7f5c973a57a0 20 read_log 193305'18194 (193305'18193) modify 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.951337.0:13462 2014-08-11 18:43:23.425495 2014-08-13 18:06:06.759558 7f5c973a57a0 20 read_log 193305'18195 (0'0) promote 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by osd.31.0:174 2014-08-11 18:43:24.301995 2014-08-13 18:06:06.759578 7f5c973a57a0 20 read_log 193305'18196 (193305'18195) modify 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.951337.0:13694 2014-08-11 18:43:24.293864 2014-08-13 18:06:06.759593 7f5c973a57a0 20 read_log 193305'18197 (0'0) promote c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by osd.31.0:188 2014-08-11 18:43:27.459047 2014-08-13 18:06:06.759615 7f5c973a57a0 20 read_log 193305'18198 (193305'18197) modify c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.951337.0:14567 2014-08-11 18:43:27.450155 2014-08-13 18:06:06.759634 7f5c973a57a0 20 read_log 193305'18199 (0'0) promote c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by osd.31.0:192 2014-08-11 18:43:29.690539 2014-08-13 18:06:06.759654 7f5c973a57a0 20 read_log 193305'18200 (193305'18199) modify c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.951337.0:15216 2014-08-11 18:43:29.682105 2014-08-13 18:06:06.759670 7f5c973a57a0 20 read_log 193305'18201 (0'0) promote 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by osd.31.0:204 2014-08-11 18:43:35.132495 2014-08-13 18:06:06.759683 7f5c973a57a0 20 read_log 193305'18202 (193305'18201) modify 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.951337.0:16783 2014-08-11 18:43:35.122772 2014-08-13 18:06:06.759696 7f5c973a57a0 20 read_log 193305'18203 (0'0) promote 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by osd.31.0:208 2014-08-11 18:43:38.257614 2014-08-13 18:06:06.759708 7f5c973a57a0 20 read_log 193305'18204 (193305'18203) modify 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.951337.0:17691 2014-08-11 18:43:38.251229 2014-08-13 18:06:06.759725 7f5c973a57a0 20 read_log 193305'18205 (0'0) promote d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by osd.31.0:210 2014-08-11 18:43:39.344924 2014-08-13 18:06:06.759739 7f5c973a57a0 20 read_log 193305'18206 (193305'18205) modify d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.951337.0:17989 2014-08-11 18:43:39.336680 2014-08-13 18:06:06.759752 7f5c973a57a0 20 read_log 193305'18207 (0'0) promote f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by osd.31.0:232 2014-08-11 18:43:44.460325 2014-08-13 18:06:06.759767 7f5c973a57a0 20 read_log 193305'18208 (193305'18207) modify f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.951337.0:19385 2014-08-11 18:43:44.452360 2014-08-13 18:06:06.759780 7f5c973a57a0 20 read_log 193305'18209 (0'0) promote a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by osd.31.0:246 2014-08-11 18:43:45.924327 2014-08-13 18:06:06.759804 7f5c973a57a0 20 read_log 193305'18210 (193305'18209) modify a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.951337.0:19806 2014-08-11 18:43:45.917125 2014-08-13 18:06:06.759821 7f5c973a57a0 20 read_log 193305'18211 (0'0) promote 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by osd.31.0:250 2014-08-11 18:43:47.442440 2014-08-13 18:06:06.759835 7f5c973a57a0 20 read_log 193305'18212 (193305'18211) modify 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.951337.0:20210 2014-08-11 18:43:47.435108 2014-08-13 18:06:06.759848 7f5c973a57a0 20 read_log 193305'18213 (0'0) promote 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by osd.31.0:262 2014-08-11 18:43:49.693622 2014-08-13 18:06:06.759860 7f5c973a57a0 20 read_log 193305'18214 (193305'18213) modify 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.951337.0:20858 2014-08-11 18:43:49.686252 2014-08-13 18:06:06.759878 7f5c973a57a0 20 read_log 193305'18215 (0'0) promote d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by osd.31.0:272 2014-08-11 18:43:50.976101 2014-08-13 18:06:06.759904 7f5c973a57a0 20 read_log 193305'18216 (193305'18215) modify d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.951337.0:21211 2014-08-11 18:43:50.966081 2014-08-13 18:06:06.759925 7f5c973a57a0 20 read_log 193305'18217 (0'0) promote 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by osd.31.0:276 2014-08-11 18:43:51.579189 2014-08-13 18:06:06.759937 7f5c973a57a0 20 read_log 193305'18218 (193305'18217) modify 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.951337.0:21385 2014-08-11 18:43:51.571974 2014-08-13 18:06:06.759950 7f5c973a57a0 20 read_log 193305'18219 (0'0) promote 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by osd.31.0:290 2014-08-11 18:43:54.288271 2014-08-13 18:06:06.759965 7f5c973a57a0 20 read_log 193305'18220 (193305'18219) modify 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.951337.0:22121 2014-08-11 18:43:54.280878 2014-08-13 18:06:06.759978 7f5c973a57a0 20 read_log 193305'18221 (0'0) promote 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by osd.31.0:306 2014-08-11 18:43:59.586444 2014-08-13 18:06:06.759991 7f5c973a57a0 20 read_log 193305'18222 (193305'18221) modify 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.951337.0:23600 2014-08-11 18:43:59.579087 2014-08-13 18:06:06.760012 7f5c973a57a0 20 read_log 193305'18223 (0'0) promote 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by osd.31.0:319 2014-08-11 18:44:04.114112 2014-08-13 18:06:06.760029 7f5c973a57a0 20 read_log 193305'18224 (193305'18223) modify 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.951337.0:24860 2014-08-11 18:44:04.106803 2014-08-13 18:06:06.760062 7f5c973a57a0 20 read_log 193305'18225 (0'0) promote 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by osd.31.0:351 2014-08-11 18:44:14.198462 2014-08-13 18:06:06.760077 7f5c973a57a0 20 read_log 193305'18226 (193305'18225) modify 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.951337.0:27589 2014-08-11 18:44:14.188725 2014-08-13 18:06:06.760090 7f5c973a57a0 20 read_log 193305'18227 (0'0) promote 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by osd.31.0:369 2014-08-11 18:44:18.849172 2014-08-13 18:06:06.760102 7f5c973a57a0 20 read_log 193305'18228 (193305'18227) modify 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.951337.0:28861 2014-08-11 18:44:18.841566 2014-08-13 18:06:06.760115 7f5c973a57a0 20 read_log 193305'18229 (0'0) promote d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by osd.31.0:377 2014-08-11 18:44:24.711304 2014-08-13 18:06:06.760127 7f5c973a57a0 20 read_log 193305'18230 (193305'18229) modify d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.951337.0:30428 2014-08-11 18:44:24.703177 2014-08-13 18:06:06.760139 7f5c973a57a0 20 read_log 193305'18231 (0'0) promote 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by osd.31.0:383 2014-08-11 18:44:25.710961 2014-08-13 18:06:06.760157 7f5c973a57a0 20 read_log 193305'18232 (193305'18231) modify 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.951337.0:30685 2014-08-11 18:44:25.701617 2014-08-13 18:06:06.760172 7f5c973a57a0 20 read_log 193305'18233 (0'0) promote b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by osd.31.0:391 2014-08-11 18:44:28.184473 2014-08-13 18:06:06.760193 7f5c973a57a0 20 read_log 193305'18234 (193305'18233) modify b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.951337.0:31386 2014-08-11 18:44:28.177650 2014-08-13 18:06:06.760209 7f5c973a57a0 20 read_log 193305'18235 (0'0) promote d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by osd.31.0:405 2014-08-11 18:44:32.237113 2014-08-13 18:06:06.760247 7f5c973a57a0 20 read_log 193305'18236 (193305'18235) modify d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.951337.0:32543 2014-08-11 18:44:32.227612 2014-08-13 18:06:06.760269 7f5c973a57a0 20 read_log 193305'18237 (193305'18168) modify 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.951337.0:33042 2014-08-11 18:44:34.098242 2014-08-13 18:06:06.760282 7f5c973a57a0 20 read_log 193305'18238 (193305'18170) modify c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.951337.0:33721 2014-08-11 18:44:34.605577 2014-08-13 18:06:06.760294 7f5c973a57a0 20 read_log 193305'18239 (193305'18172) modify d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.951337.0:36329 2014-08-11 18:44:36.133685 2014-08-13 18:06:06.760312 7f5c973a57a0 20 read_log 193305'18240 (193305'18174) modify 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.951337.0:36701 2014-08-11 18:44:36.482159 2014-08-13 18:06:06.760325 7f5c973a57a0 20 read_log 193305'18241 (193305'18176) modify 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.951337.0:36858 2014-08-11 18:44:36.571751 2014-08-13 18:06:06.760337 7f5c973a57a0 20 read_log 193305'18242 (193305'18178) modify d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.951337.0:36955 2014-08-11 18:44:36.620095 2014-08-13 18:06:06.760349 7f5c973a57a0 20 read_log 193305'18243 (193305'18180) modify f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.951337.0:37005 2014-08-11 18:44:36.648648 2014-08-13 18:06:06.760362 7f5c973a57a0 20 read_log 193305'18244 (193305'18182) modify 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.951337.0:38993 2014-08-11 18:44:37.766648 2014-08-13 18:06:06.760374 7f5c973a57a0 20 read_log 193305'18245 (193305'18184) modify d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.951337.0:42961 2014-08-11 18:44:39.436466 2014-08-13 18:06:06.760386 7f5c973a57a0 20 read_log 193305'18246 (193305'18186) modify 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.951337.0:44879 2014-08-11 18:44:40.022832 2014-08-13 18:06:06.760410 7f5c973a57a0 20 read_log 193305'18247 (193305'18188) modify bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.951337.0:45365 2014-08-11 18:44:40.176410 2014-08-13 18:06:06.760423 7f5c973a57a0 20 read_log 193305'18248 (193305'18190) modify 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.951337.0:46037 2014-08-11 18:44:40.371712 2014-08-13 18:06:06.760436 7f5c973a57a0 20 read_log 193305'18249 (193305'18192) modify 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.951337.0:46073 2014-08-11 18:44:40.381311 2014-08-13 18:06:06.760448 7f5c973a57a0 20 read_log 193305'18250 (193305'18194) modify 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.951337.0:46173 2014-08-11 18:44:40.410404 2014-08-13 18:06:06.760461 7f5c973a57a0 20 read_log 193305'18251 (193305'18196) modify 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.951337.0:46405 2014-08-11 18:44:40.473923 2014-08-13 18:06:06.760473 7f5c973a57a0 20 read_log 193305'18252 (193305'18198) modify c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.951337.0:47278 2014-08-11 18:44:40.723650 2014-08-13 18:06:06.760486 7f5c973a57a0 20 read_log 193305'18253 (193305'18200) modify c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.951337.0:47927 2014-08-11 18:44:40.905370 2014-08-13 18:06:06.760498 7f5c973a57a0 20 read_log 193305'18254 (193305'18202) modify 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.951337.0:49494 2014-08-11 18:44:41.386948 2014-08-13 18:06:06.760510 7f5c973a57a0 20 read_log 193305'18255 (193305'18204) modify 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.951337.0:50402 2014-08-11 18:44:41.654749 2014-08-13 18:06:06.760533 7f5c973a57a0 20 read_log 193305'18256 (193305'18206) modify d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.951337.0:50700 2014-08-11 18:44:41.738138 2014-08-13 18:06:06.760549 7f5c973a57a0 20 read_log 193305'18257 (193305'18208) modify f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.951337.0:52096 2014-08-11 18:44:42.132883 2014-08-13 18:06:06.760562 7f5c973a57a0 20 read_log 193305'18258 (193305'18210) modify a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.951337.0:52517 2014-08-11 18:44:42.254449 2014-08-13 18:06:06.760574 7f5c973a57a0 20 read_log 193305'18259 (193305'18212) modify 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.951337.0:52921 2014-08-11 18:44:42.380469 2014-08-13 18:06:06.760586 7f5c973a57a0 20 read_log 193305'18260 (193305'18214) modify 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.951337.0:53569 2014-08-11 18:44:42.566863 2014-08-13 18:06:06.760599 7f5c973a57a0 20 read_log 193305'18261 (193305'18216) modify d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.951337.0:53922 2014-08-11 18:44:42.674633 2014-08-13 18:06:06.760611 7f5c973a57a0 20 read_log 193305'18262 (193305'18218) modify 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.951337.0:54096 2014-08-11 18:44:42.729849 2014-08-13 18:06:06.760623 7f5c973a57a0 20 read_log 193305'18263 (193305'18220) modify 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.951337.0:54832 2014-08-11 18:44:42.951656 2014-08-13 18:06:06.760635 7f5c973a57a0 20 read_log 193305'18264 (193305'18222) modify 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.951337.0:56311 2014-08-11 18:44:43.381186 2014-08-13 18:06:06.760647 7f5c973a57a0 20 read_log 193305'18265 (193305'18224) modify 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.951337.0:57571 2014-08-11 18:44:43.796585 2014-08-13 18:06:06.760659 7f5c973a57a0 20 read_log 193305'18266 (193305'18226) modify 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.951337.0:60300 2014-08-11 18:44:44.589001 2014-08-13 18:06:06.760671 7f5c973a57a0 20 read_log 193305'18267 (193305'18228) modify 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.951337.0:61572 2014-08-11 18:44:44.964334 2014-08-13 18:06:06.760683 7f5c973a57a0 20 read_log 193305'18268 (193305'18230) modify d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.951337.0:63139 2014-08-11 18:44:45.420569 2014-08-13 18:06:06.760696 7f5c973a57a0 20 read_log 193305'18269 (193305'18232) modify 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.951337.0:63396 2014-08-11 18:44:45.497304 2014-08-13 18:06:06.760710 7f5c973a57a0 20 read_log 193305'18270 (193305'18234) modify b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.951337.0:64097 2014-08-11 18:44:45.703309 2014-08-13 18:06:06.760722 7f5c973a57a0 20 read_log 193305'18271 (193305'18236) modify d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.951337.0:65254 2014-08-11 18:44:46.063902 2014-08-13 18:06:06.760735 7f5c973a57a0 20 read_log 194215'18272 (194215'18272) modify 1e4/hit_set_24.1e4_archive_2014-08-12 15:32:25.782637_2014-08-12 15:43:43.172386/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:43.172618 2014-08-13 18:06:06.760747 7f5c973a57a0 20 read_log 194215'18273 (193305'18165) delete 1e4/hit_set_24.1e4_archive_2014-08-11 18:37:04.917636_2014-08-11 18:42:34.237231/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:43.172618 2014-08-13 18:06:06.760760 7f5c973a57a0 20 read_log 194215'18274 (192888'17969) clean 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by osd.31.0:1598 2014-08-12 15:43:43.176821 2014-08-13 18:06:06.760779 7f5c973a57a0 20 read_log 194215'18275 (194215'18274) delete 94b201e4/benchmark_data_storage0111-ib.pocstor_41097_object26053/head//24 by client.2174595.0:100171 0.000000 2014-08-13 18:06:06.760791 7f5c973a57a0 20 read_log 194215'18276 (192888'18142) clean 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by osd.31.0:1600 2014-08-12 15:43:43.249820 2014-08-13 18:06:06.760803 7f5c973a57a0 20 read_log 194215'18277 (194215'18276) delete 1a3b01e4/benchmark_data_storage0111-ib.pocstor_2228_object19949/head//24 by client.2174595.0:100173 0.000000 2014-08-13 18:06:06.760814 7f5c973a57a0 20 read_log 194215'18278 (193305'18250) clean 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by osd.31.0:1602 2014-08-12 15:43:43.258606 2014-08-13 18:06:06.760829 7f5c973a57a0 20 read_log 194215'18279 (194215'18278) delete 95df01e4/benchmark_data_storage0111-ib.pocstor_62621_object13461/head//24 by client.2174595.0:100175 0.000000 2014-08-13 18:06:06.760840 7f5c973a57a0 20 read_log 194215'18280 (193305'18240) clean 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by osd.31.0:1604 2014-08-12 15:43:43.267221 2014-08-13 18:06:06.760852 7f5c973a57a0 20 read_log 194215'18281 (194215'18280) delete 3aeb11e4/benchmark_data_storage0111-ib.pocstor_62621_object3989/head//24 by client.2174595.0:100177 0.000000 2014-08-13 18:06:06.760865 7f5c973a57a0 20 read_log 194215'18282 (192888'17934) clean 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by osd.31.0:1606 2014-08-12 15:43:43.274964 2014-08-13 18:06:06.760878 7f5c973a57a0 20 read_log 194215'18283 (194215'18282) delete 3ba931e4/benchmark_data_storage0111-ib.pocstor_41097_object10882/head//24 by client.2174595.0:100179 0.000000 2014-08-13 18:06:06.760889 7f5c973a57a0 20 read_log 194215'18284 (193305'18258) clean a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by osd.31.0:1608 2014-08-12 15:43:43.282043 2014-08-13 18:06:06.760903 7f5c973a57a0 20 read_log 194215'18285 (194215'18284) delete a08b41e4/benchmark_data_storage0111-ib.pocstor_62621_object19805/head//24 by client.2174595.0:100181 0.000000 2014-08-13 18:06:06.760914 7f5c973a57a0 20 read_log 194215'18286 (193305'18242) clean d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by osd.31.0:1610 2014-08-12 15:43:43.289905 2014-08-13 18:06:06.760926 7f5c973a57a0 20 read_log 194215'18287 (194215'18286) delete d7ee51e4/benchmark_data_storage0111-ib.pocstor_62621_object4243/head//24 by client.2174595.0:100183 0.000000 2014-08-13 18:06:06.760937 7f5c973a57a0 20 read_log 194215'18288 (192888'18120) clean 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by osd.31.0:1612 2014-08-12 15:43:43.297640 2014-08-13 18:06:06.760951 7f5c973a57a0 20 read_log 194215'18289 (194215'18288) delete 666961e4/benchmark_data_storage0111-ib.pocstor_2228_object9631/head//24 by client.2174595.0:100185 0.000000 2014-08-13 18:06:06.760962 7f5c973a57a0 20 read_log 194215'18290 (192888'18111) clean abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by osd.31.0:1614 2014-08-12 15:43:43.304954 2014-08-13 18:06:06.760974 7f5c973a57a0 20 read_log 194215'18291 (194215'18290) delete abe61e4/benchmark_data_storage0111-ib.pocstor_2228_object4139/head//24 by client.2174595.0:100187 0.000000 2014-08-13 18:06:06.760987 7f5c973a57a0 20 read_log 194215'18292 (192888'18162) clean 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by osd.31.0:1616 2014-08-12 15:43:43.312458 2014-08-13 18:06:06.761000 7f5c973a57a0 20 read_log 194215'18293 (194215'18292) delete 7ae271e4/benchmark_data_storage0111-ib.pocstor_2228_object29659/head//24 by client.2174595.0:100189 0.000000 2014-08-13 18:06:06.761013 7f5c973a57a0 20 read_log 194215'18294 (192888'18122) clean 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by osd.31.0:1618 2014-08-12 15:43:43.319512 2014-08-13 18:06:06.761036 7f5c973a57a0 20 read_log 194215'18295 (194215'18294) delete 7ab771e4/benchmark_data_storage0111-ib.pocstor_2228_object9943/head//24 by client.2174595.0:100191 0.000000 2014-08-13 18:06:06.761050 7f5c973a57a0 20 read_log 194215'18296 (192888'17970) clean 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by osd.31.0:1620 2014-08-12 15:43:43.327227 2014-08-13 18:06:06.761062 7f5c973a57a0 20 read_log 194215'18297 (194215'18296) delete 780381e4/benchmark_data_storage0111-ib.pocstor_41097_object26589/head//24 by client.2174595.0:100193 0.000000 2014-08-13 18:06:06.761073 7f5c973a57a0 20 read_log 194215'18298 (192888'17964) clean 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by osd.31.0:1622 2014-08-12 15:43:43.334500 2014-08-13 18:06:06.761088 7f5c973a57a0 20 read_log 194215'18299 (194215'18298) delete 837a1e4/benchmark_data_storage0111-ib.pocstor_41097_object23759/head//24 by client.2174595.0:100195 0.000000 2014-08-13 18:06:06.761098 7f5c973a57a0 20 read_log 194215'18300 (192888'18133) clean 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by osd.31.0:1624 2014-08-12 15:43:43.341493 2014-08-13 18:06:06.761111 7f5c973a57a0 20 read_log 194215'18301 (194215'18300) delete 88f5b1e4/benchmark_data_storage0111-ib.pocstor_2228_object15863/head//24 by client.2174595.0:100197 0.000000 2014-08-13 18:06:06.761124 7f5c973a57a0 20 read_log 194215'18302 (192888'18131) clean c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by osd.31.0:1626 2014-08-12 15:43:43.348859 2014-08-13 18:06:06.761136 7f5c973a57a0 20 read_log 194215'18303 (194215'18302) delete c1fcc1e4/benchmark_data_storage0111-ib.pocstor_2228_object14008/head//24 by client.2174595.0:100199 0.000000 2014-08-13 18:06:06.761154 7f5c973a57a0 20 read_log 194215'18304 (192888'17952) clean f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by osd.31.0:1628 2014-08-12 15:43:43.355994 2014-08-13 18:06:06.761169 7f5c973a57a0 20 read_log 194215'18305 (194215'18304) delete f39d1e4/benchmark_data_storage0111-ib.pocstor_41097_object19708/head//24 by client.2174595.0:100201 0.000000 2014-08-13 18:06:06.761180 7f5c973a57a0 20 read_log 194215'18306 (192888'18125) clean e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by osd.31.0:1630 2014-08-12 15:43:43.362854 2014-08-13 18:06:06.761193 7f5c973a57a0 20 read_log 194215'18307 (194215'18306) delete e4cbe1e4/benchmark_data_storage0111-ib.pocstor_2228_object12425/head//24 by client.2174595.0:100203 0.000000 2014-08-13 18:06:06.761204 7f5c973a57a0 20 read_log 194215'18308 (193305'18257) clean f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by osd.31.0:1632 2014-08-12 15:43:43.370039 2014-08-13 18:06:06.761216 7f5c973a57a0 20 read_log 194215'18309 (194215'18308) delete f961f1e4/benchmark_data_storage0111-ib.pocstor_62621_object19384/head//24 by client.2174595.0:100205 0.000000 2014-08-13 18:06:06.761227 7f5c973a57a0 20 read_log 194215'18310 (193305'18269) clean 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by osd.31.0:1634 2014-08-12 15:43:43.377110 2014-08-13 18:06:06.761244 7f5c973a57a0 20 read_log 194215'18311 (194215'18310) delete 688af1e4/benchmark_data_storage0111-ib.pocstor_62621_object30684/head//24 by client.2174595.0:100207 0.000000 2014-08-13 18:06:06.761257 7f5c973a57a0 20 read_log 194215'18312 (193305'18267) clean 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by osd.31.0:1636 2014-08-12 15:43:43.384371 2014-08-13 18:06:06.761270 7f5c973a57a0 20 read_log 194215'18313 (194215'18312) delete 6db705e4/benchmark_data_storage0111-ib.pocstor_62621_object28860/head//24 by client.2174595.0:100209 0.000000 2014-08-13 18:06:06.761283 7f5c973a57a0 20 read_log 194215'18314 (192888'18134) clean c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by osd.31.0:1638 2014-08-12 15:43:43.391807 2014-08-13 18:06:06.761327 7f5c973a57a0 20 read_log 194215'18315 (194215'18314) delete c10b05e4/benchmark_data_storage0111-ib.pocstor_2228_object16132/head//24 by client.2174595.0:100211 0.000000 2014-08-13 18:06:06.761341 7f5c973a57a0 20 read_log 194215'18316 (193305'18251) clean 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by osd.31.0:1640 2014-08-12 15:43:43.399533 2014-08-13 18:06:06.761354 7f5c973a57a0 20 read_log 194215'18317 (194215'18316) delete 113c05e4/benchmark_data_storage0111-ib.pocstor_62621_object13693/head//24 by client.2174595.0:100213 0.000000 2014-08-13 18:06:06.761395 7f5c973a57a0 20 read_log 194215'18318 (192888'18106) clean 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by osd.31.0:1642 2014-08-12 15:43:43.407404 2014-08-13 18:06:06.761411 7f5c973a57a0 20 read_log 194215'18319 (194215'18318) delete 42b015e4/benchmark_data_storage0111-ib.pocstor_2228_object721/head//24 by client.2174595.0:100215 0.000000 2014-08-13 18:06:06.761434 7f5c973a57a0 20 read_log 194215'18320 (193305'18261) clean d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by osd.31.0:1644 2014-08-12 15:43:43.414842 2014-08-13 18:06:06.761450 7f5c973a57a0 20 read_log 194215'18321 (194215'18320) delete d8d215e4/benchmark_data_storage0111-ib.pocstor_62621_object21210/head//24 by client.2174595.0:100217 0.000000 2014-08-13 18:06:06.761481 7f5c973a57a0 20 read_log 194215'18322 (192888'17948) clean bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by osd.31.0:1646 2014-08-12 15:43:43.421997 2014-08-13 18:06:06.761498 7f5c973a57a0 20 read_log 194215'18323 (194215'18322) delete bcf315e4/benchmark_data_storage0111-ib.pocstor_41097_object17080/head//24 by client.2174595.0:100219 0.000000 2014-08-13 18:06:06.761509 7f5c973a57a0 20 read_log 194215'18324 (193305'18239) clean d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by osd.31.0:1648 2014-08-12 15:43:43.428559 2014-08-13 18:06:06.761529 7f5c973a57a0 20 read_log 194215'18325 (194215'18324) delete d0d415e4/benchmark_data_storage0111-ib.pocstor_62621_object3617/head//24 by client.2174595.0:100221 0.000000 2014-08-13 18:06:06.761543 7f5c973a57a0 20 read_log 194215'18326 (192888'18136) clean 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by osd.31.0:1650 2014-08-12 15:43:43.436226 2014-08-13 18:06:06.761556 7f5c973a57a0 20 read_log 194215'18327 (194215'18326) delete 12cd15e4/benchmark_data_storage0111-ib.pocstor_2228_object16943/head//24 by client.2174595.0:100223 0.000000 2014-08-13 18:06:06.761567 7f5c973a57a0 20 read_log 194215'18328 (193305'18271) clean d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by osd.31.0:1652 2014-08-12 15:43:43.443653 2014-08-13 18:06:06.761579 7f5c973a57a0 20 read_log 194215'18329 (194215'18328) delete d92a35e4/benchmark_data_storage0111-ib.pocstor_62621_object32542/head//24 by client.2174595.0:100225 0.000000 2014-08-13 18:06:06.761590 7f5c973a57a0 20 read_log 194215'18330 (193305'18266) clean 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by osd.31.0:1654 2014-08-12 15:43:43.450780 2014-08-13 18:06:06.761602 7f5c973a57a0 20 read_log 194215'18331 (194215'18330) delete 230f35e4/benchmark_data_storage0111-ib.pocstor_62621_object27588/head//24 by client.2174595.0:100227 0.000000 2014-08-13 18:06:06.761613 7f5c973a57a0 20 read_log 194215'18332 (192888'18121) clean 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by osd.31.0:1656 2014-08-12 15:43:43.458096 2014-08-13 18:06:06.761625 7f5c973a57a0 20 read_log 194215'18333 (194215'18332) delete 533145e4/benchmark_data_storage0111-ib.pocstor_2228_object9719/head//24 by client.2174595.0:100229 0.000000 2014-08-13 18:06:06.761636 7f5c973a57a0 20 read_log 194215'18334 (193305'18253) clean c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by osd.31.0:1658 2014-08-12 15:43:43.465662 2014-08-13 18:06:06.761656 7f5c973a57a0 20 read_log 194215'18335 (194215'18334) delete c41a55e4/benchmark_data_storage0111-ib.pocstor_62621_object15215/head//24 by client.2174595.0:100231 0.000000 2014-08-13 18:06:06.761667 7f5c973a57a0 20 read_log 194215'18336 (192888'17965) clean 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by osd.31.0:1660 2014-08-12 15:43:43.473350 2014-08-13 18:06:06.761679 7f5c973a57a0 20 read_log 194215'18337 (194215'18336) delete 386c55e4/benchmark_data_storage0111-ib.pocstor_41097_object23992/head//24 by client.2174595.0:100233 0.000000 2014-08-13 18:06:06.761690 7f5c973a57a0 20 read_log 194215'18338 (192888'18126) clean db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by osd.31.0:1662 2014-08-12 15:43:43.482005 2014-08-13 18:06:06.761712 7f5c973a57a0 20 read_log 194215'18339 (194215'18338) delete db2865e4/benchmark_data_storage0111-ib.pocstor_2228_object12470/head//24 by client.2174595.0:100235 0.000000 2014-08-13 18:06:06.761724 7f5c973a57a0 20 read_log 194215'18340 (192888'17976) clean 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by osd.31.0:1664 2014-08-12 15:43:43.490042 2014-08-13 18:06:06.761737 7f5c973a57a0 20 read_log 194215'18341 (194215'18340) delete 2e5875e4/benchmark_data_storage0111-ib.pocstor_41097_object28425/head//24 by client.2174595.0:100237 0.000000 2014-08-13 18:06:06.761748 7f5c973a57a0 20 read_log 194215'18342 (192888'17978) clean 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by osd.31.0:1666 2014-08-12 15:43:43.497442 2014-08-13 18:06:06.761760 7f5c973a57a0 20 read_log 194215'18343 (194215'18342) delete 89d875e4/benchmark_data_storage0111-ib.pocstor_41097_object28686/head//24 by client.2174595.0:100239 0.000000 2014-08-13 18:06:06.761771 7f5c973a57a0 20 read_log 194215'18344 (192888'17912) clean a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by osd.31.0:1668 2014-08-12 15:43:43.504933 2014-08-13 18:06:06.761783 7f5c973a57a0 20 read_log 194215'18345 (194215'18344) delete a51975e4/benchmark_data_storage0111-ib.pocstor_41097_object2284/head//24 by client.2174595.0:100241 0.000000 2014-08-13 18:06:06.761794 7f5c973a57a0 20 read_log 194215'18346 (193305'18243) clean f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by osd.31.0:1670 2014-08-12 15:43:43.512493 2014-08-13 18:06:06.761806 7f5c973a57a0 20 read_log 194215'18347 (194215'18346) delete f01d75e4/benchmark_data_storage0111-ib.pocstor_62621_object4293/head//24 by client.2174595.0:100243 0.000000 2014-08-13 18:06:06.761817 7f5c973a57a0 20 read_log 194215'18348 (192888'17947) clean e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by osd.31.0:1672 2014-08-12 15:43:43.519915 2014-08-13 18:06:06.761830 7f5c973a57a0 20 read_log 194215'18349 (194215'18348) delete e0d285e4/benchmark_data_storage0111-ib.pocstor_41097_object16894/head//24 by client.2174595.0:100245 0.000000 2014-08-13 18:06:06.761841 7f5c973a57a0 20 read_log 194215'18350 (192888'18141) clean 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by osd.31.0:1674 2014-08-12 15:43:43.527567 2014-08-13 18:06:06.761853 7f5c973a57a0 20 read_log 194215'18351 (194215'18350) delete 294685e4/benchmark_data_storage0111-ib.pocstor_2228_object19196/head//24 by client.2174595.0:100247 0.000000 2014-08-13 18:06:06.761864 7f5c973a57a0 20 read_log 194215'18352 (193305'18245) clean d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by osd.31.0:1676 2014-08-12 15:43:43.534506 2014-08-13 18:06:06.761876 7f5c973a57a0 20 read_log 194215'18353 (194215'18352) delete d9795e4/benchmark_data_storage0111-ib.pocstor_62621_object10249/head//24 by client.2174595.0:100249 0.000000 2014-08-13 18:06:06.761886 7f5c973a57a0 20 read_log 194215'18354 (193305'18248) clean 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by osd.31.0:1678 2014-08-12 15:43:43.541650 2014-08-13 18:06:06.761908 7f5c973a57a0 20 read_log 194215'18355 (194215'18354) delete 262c95e4/benchmark_data_storage0111-ib.pocstor_62621_object13325/head//24 by client.2174595.0:100251 0.000000 2014-08-13 18:06:06.761923 7f5c973a57a0 20 read_log 194215'18356 (193305'18260) clean 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by osd.31.0:1680 2014-08-12 15:43:43.549083 2014-08-13 18:06:06.761935 7f5c973a57a0 20 read_log 194215'18357 (194215'18356) delete 7513a5e4/benchmark_data_storage0111-ib.pocstor_62621_object20857/head//24 by client.2174595.0:100253 0.000000 2014-08-13 18:06:06.761946 7f5c973a57a0 20 read_log 194215'18358 (192888'17961) clean 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by osd.31.0:1682 2014-08-12 15:43:43.556931 2014-08-13 18:06:06.761958 7f5c973a57a0 20 read_log 194215'18359 (194215'18358) delete 7f63a5e4/benchmark_data_storage0111-ib.pocstor_41097_object21760/head//24 by client.2174595.0:100255 0.000000 2014-08-13 18:06:06.761969 7f5c973a57a0 20 read_log 194215'18360 (193305'18264) clean 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by osd.31.0:1684 2014-08-12 15:43:43.565284 2014-08-13 18:06:06.761984 7f5c973a57a0 20 read_log 194215'18361 (194215'18360) delete 5857a5e4/benchmark_data_storage0111-ib.pocstor_62621_object23599/head//24 by client.2174595.0:100257 0.000000 2014-08-13 18:06:06.761994 7f5c973a57a0 20 read_log 194215'18362 (192888'17925) clean b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by osd.31.0:1686 2014-08-12 15:43:43.573307 2014-08-13 18:06:06.762007 7f5c973a57a0 20 read_log 194215'18363 (194215'18362) delete b62ea5e4/benchmark_data_storage0111-ib.pocstor_41097_object8616/head//24 by client.2174595.0:100259 0.000000 2014-08-13 18:06:06.762020 7f5c973a57a0 20 read_log 194215'18364 (193305'18265) clean 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by osd.31.0:1688 2014-08-12 15:43:43.580553 2014-08-13 18:06:06.762032 7f5c973a57a0 20 read_log 194215'18365 (194215'18364) delete 1c88b5e4/benchmark_data_storage0111-ib.pocstor_62621_object24859/head//24 by client.2174595.0:100261 0.000000 2014-08-13 18:06:06.762044 7f5c973a57a0 20 read_log 194215'18366 (192888'18157) clean 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by osd.31.0:1690 2014-08-12 15:43:43.587991 2014-08-13 18:06:06.762058 7f5c973a57a0 20 read_log 194215'18367 (194215'18366) delete 7caab5e4/benchmark_data_storage0111-ib.pocstor_2228_object27778/head//24 by client.2174595.0:100263 0.000000 2014-08-13 18:06:06.762069 7f5c973a57a0 20 read_log 194215'18368 (192888'18139) clean e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by osd.31.0:1692 2014-08-12 15:43:43.595155 2014-08-13 18:06:06.762081 7f5c973a57a0 20 read_log 194215'18369 (194215'18368) delete e33cc5e4/benchmark_data_storage0111-ib.pocstor_2228_object17771/head//24 by client.2174595.0:100265 0.000000 2014-08-13 18:06:06.762092 7f5c973a57a0 20 read_log 194215'18370 (193305'18256) clean d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by osd.31.0:1694 2014-08-12 15:43:43.602326 2014-08-13 18:06:06.762107 7f5c973a57a0 20 read_log 194215'18371 (194215'18370) delete d3acc5e4/benchmark_data_storage0111-ib.pocstor_62621_object17988/head//24 by client.2174595.0:100267 0.000000 2014-08-13 18:06:06.762118 7f5c973a57a0 20 read_log 194215'18372 (192888'17960) clean 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by osd.31.0:1696 2014-08-12 15:43:43.609468 2014-08-13 18:06:06.762130 7f5c973a57a0 20 read_log 194215'18373 (194215'18372) delete 646cd5e4/benchmark_data_storage0111-ib.pocstor_41097_object21759/head//24 by client.2174595.0:100269 0.000000 2014-08-13 18:06:06.762141 7f5c973a57a0 20 read_log 194215'18374 (192888'17949) clean 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by osd.31.0:1698 2014-08-12 15:43:43.616898 2014-08-13 18:06:06.762172 7f5c973a57a0 20 read_log 194215'18375 (194215'18374) delete 97f5e5e4/benchmark_data_storage0111-ib.pocstor_41097_object17153/head//24 by client.2174595.0:100271 0.000000 2014-08-13 18:06:06.762187 7f5c973a57a0 20 read_log 194215'18376 (193305'18241) clean 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by osd.31.0:1700 2014-08-12 15:43:43.623453 2014-08-13 18:06:06.762199 7f5c973a57a0 20 read_log 194215'18377 (194215'18376) delete 285ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object4146/head//24 by client.2174595.0:100273 0.000000 2014-08-13 18:06:06.762210 7f5c973a57a0 20 read_log 194215'18378 (193305'18244) clean 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by osd.31.0:1702 2014-08-12 15:43:43.630486 2014-08-13 18:06:06.762223 7f5c973a57a0 20 read_log 194215'18379 (194215'18378) delete 6a5ee5e4/benchmark_data_storage0111-ib.pocstor_62621_object6281/head//24 by client.2174595.0:100275 0.000000 2014-08-13 18:06:06.762234 7f5c973a57a0 20 read_log 194215'18380 (192888'18107) clean 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by osd.31.0:1704 2014-08-12 15:43:43.637886 2014-08-13 18:06:06.762255 7f5c973a57a0 20 read_log 194215'18381 (194215'18380) delete 251df5e4/benchmark_data_storage0111-ib.pocstor_2228_object900/head//24 by client.2174595.0:100277 0.000000 2014-08-13 18:06:06.762266 7f5c973a57a0 20 read_log 194215'18382 (192888'18151) clean 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by osd.31.0:1706 2014-08-12 15:43:43.644750 2014-08-13 18:06:06.762279 7f5c973a57a0 20 read_log 194215'18383 (194215'18382) delete 37c819e4/benchmark_data_storage0111-ib.pocstor_2228_object26101/head//24 by client.2174595.0:100279 0.000000 2014-08-13 18:06:06.762290 7f5c973a57a0 20 read_log 194215'18384 (192888'17945) clean 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by osd.31.0:1708 2014-08-12 15:43:43.651904 2014-08-13 18:06:06.762302 7f5c973a57a0 20 read_log 194215'18385 (194215'18384) delete 741a29e4/benchmark_data_storage0111-ib.pocstor_41097_object16074/head//24 by client.2174595.0:100281 0.000000 2014-08-13 18:06:06.762313 7f5c973a57a0 20 read_log 194215'18386 (193305'18249) clean 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by osd.31.0:1710 2014-08-12 15:43:43.659442 2014-08-13 18:06:06.762325 7f5c973a57a0 20 read_log 194215'18387 (194215'18386) delete 67db29e4/benchmark_data_storage0111-ib.pocstor_62621_object13361/head//24 by client.2174595.0:100283 0.000000 2014-08-13 18:06:06.762336 7f5c973a57a0 20 read_log 194215'18388 (192888'17979) clean bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by osd.31.0:1712 2014-08-12 15:43:43.666799 2014-08-13 18:06:06.762348 7f5c973a57a0 20 read_log 194215'18389 (194215'18388) delete bca739e4/benchmark_data_storage0111-ib.pocstor_41097_object29527/head//24 by client.2174595.0:100285 0.000000 2014-08-13 18:06:06.762367 7f5c973a57a0 20 read_log 194215'18390 (192888'17966) clean c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by osd.31.0:1714 2014-08-12 15:43:43.674568 2014-08-13 18:06:06.762384 7f5c973a57a0 20 read_log 194215'18391 (194215'18390) delete c7c649e4/benchmark_data_storage0111-ib.pocstor_41097_object24339/head//24 by client.2174595.0:100287 0.000000 2014-08-13 18:06:06.762395 7f5c973a57a0 20 read_log 194215'18392 (192888'18164) clean b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by osd.31.0:1716 2014-08-12 15:43:43.682380 2014-08-13 18:06:06.762413 7f5c973a57a0 20 read_log 194215'18393 (194215'18392) delete b9ce49e4/benchmark_data_storage0111-ib.pocstor_2228_object30251/head//24 by client.2174595.0:100289 0.000000 2014-08-13 18:06:06.762435 7f5c973a57a0 20 read_log 194215'18394 (193305'18246) clean 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by osd.31.0:1718 2014-08-12 15:43:43.689808 2014-08-13 18:06:06.762455 7f5c973a57a0 20 read_log 194215'18395 (194215'18394) delete 917559e4/benchmark_data_storage0111-ib.pocstor_62621_object12167/head//24 by client.2174595.0:100291 0.000000 2014-08-13 18:06:06.762477 7f5c973a57a0 20 read_log 194215'18396 (192888'18153) clean 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by osd.31.0:1720 2014-08-12 15:43:43.697003 2014-08-13 18:06:06.762490 7f5c973a57a0 20 read_log 194215'18397 (194215'18396) delete 2c7469e4/benchmark_data_storage0111-ib.pocstor_2228_object26442/head//24 by client.2174595.0:100293 0.000000 2014-08-13 18:06:06.762501 7f5c973a57a0 20 read_log 194215'18398 (192888'17920) clean 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by osd.31.0:1722 2014-08-12 15:43:43.704647 2014-08-13 18:06:06.762527 7f5c973a57a0 20 read_log 194215'18399 (194215'18398) delete 8c5289e4/benchmark_data_storage0111-ib.pocstor_41097_object6874/head//24 by client.2174595.0:100295 0.000000 2014-08-13 18:06:06.762539 7f5c973a57a0 20 read_log 194215'18400 (192888'17911) clean a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by osd.31.0:1724 2014-08-12 15:43:43.712202 2014-08-13 18:06:06.762551 7f5c973a57a0 20 read_log 194215'18401 (194215'18400) delete a96399e4/benchmark_data_storage0111-ib.pocstor_41097_object1829/head//24 by client.2174595.0:100297 0.000000 2014-08-13 18:06:06.762562 7f5c973a57a0 20 read_log 194215'18402 (193305'18252) clean c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by osd.31.0:1726 2014-08-12 15:43:43.718863 2014-08-13 18:06:06.762586 7f5c973a57a0 20 read_log 194215'18403 (194215'18402) delete c253b9e4/benchmark_data_storage0111-ib.pocstor_62621_object14566/head//24 by client.2174595.0:100299 0.000000 2014-08-13 18:06:06.762601 7f5c973a57a0 20 read_log 194215'18404 (193305'18238) clean c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by osd.31.0:1728 2014-08-12 15:43:43.725832 2014-08-13 18:06:06.762616 7f5c973a57a0 20 read_log 194215'18405 (194215'18404) delete c2aeb9e4/benchmark_data_storage0111-ib.pocstor_62621_object1009/head//24 by client.2174595.0:100301 0.000000 2014-08-13 18:06:06.762627 7f5c973a57a0 20 read_log 194215'18406 (193305'18270) clean b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by osd.31.0:1730 2014-08-12 15:43:43.733562 2014-08-13 18:06:06.762639 7f5c973a57a0 20 read_log 194215'18407 (194215'18406) delete b4d0c9e4/benchmark_data_storage0111-ib.pocstor_62621_object31385/head//24 by client.2174595.0:100303 0.000000 2014-08-13 18:06:06.762650 7f5c973a57a0 20 read_log 194215'18408 (193305'18255) clean 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by osd.31.0:1732 2014-08-12 15:43:43.740834 2014-08-13 18:06:06.762665 7f5c973a57a0 20 read_log 194215'18409 (194215'18408) delete 3177c9e4/benchmark_data_storage0111-ib.pocstor_62621_object17690/head//24 by client.2174595.0:100305 0.000000 2014-08-13 18:06:06.762676 7f5c973a57a0 20 read_log 194215'18410 (192888'18163) clean e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by osd.31.0:1734 2014-08-12 15:43:43.748073 2014-08-13 18:06:06.762707 7f5c973a57a0 20 read_log 194215'18411 (194215'18410) delete e4ed9e4/benchmark_data_storage0111-ib.pocstor_2228_object30001/head//24 by client.2174595.0:100307 0.000000 2014-08-13 18:06:06.762731 7f5c973a57a0 20 read_log 194215'18412 (192888'17967) clean 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by osd.31.0:1736 2014-08-12 15:43:43.755565 2014-08-13 18:06:06.762744 7f5c973a57a0 20 read_log 194215'18413 (194215'18412) delete 1d43e9e4/benchmark_data_storage0111-ib.pocstor_41097_object24710/head//24 by client.2174595.0:100309 0.000000 2014-08-13 18:06:06.762755 7f5c973a57a0 20 read_log 194215'18414 (193305'18259) clean 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by osd.31.0:1738 2014-08-12 15:43:43.762529 2014-08-13 18:06:06.762780 7f5c973a57a0 20 read_log 194215'18415 (194215'18414) delete 4ad4e9e4/benchmark_data_storage0111-ib.pocstor_62621_object20209/head//24 by client.2174595.0:100311 0.000000 2014-08-13 18:06:06.762794 7f5c973a57a0 20 read_log 194215'18416 (192888'18109) clean d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by osd.31.0:1740 2014-08-12 15:43:43.769492 2014-08-13 18:06:06.762816 7f5c973a57a0 20 read_log 194215'18417 (194215'18416) delete d768f9e4/benchmark_data_storage0111-ib.pocstor_2228_object1217/head//24 by client.2174595.0:100313 0.000000 2014-08-13 18:06:06.762830 7f5c973a57a0 20 read_log 194215'18418 (192888'17974) clean 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by osd.31.0:1742 2014-08-12 15:43:43.776037 2014-08-13 18:06:06.762843 7f5c973a57a0 20 read_log 194215'18419 (194215'18418) delete 43260de4/benchmark_data_storage0111-ib.pocstor_41097_object27941/head//24 by client.2174595.0:100315 0.000000 2014-08-13 18:06:06.762854 7f5c973a57a0 20 read_log 194215'18420 (192888'18148) clean ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by osd.31.0:1744 2014-08-12 15:43:43.783018 2014-08-13 18:06:06.762866 7f5c973a57a0 20 read_log 194215'18421 (194215'18420) delete ff101de4/benchmark_data_storage0111-ib.pocstor_2228_object25177/head//24 by client.2174595.0:100317 0.000000 2014-08-13 18:06:06.762877 7f5c973a57a0 20 read_log 194215'18422 (193305'18262) clean 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by osd.31.0:1746 2014-08-12 15:43:43.790266 2014-08-13 18:06:06.762889 7f5c973a57a0 20 read_log 194215'18423 (194215'18422) delete 8e261de4/benchmark_data_storage0111-ib.pocstor_62621_object21384/head//24 by client.2174595.0:100319 0.000000 2014-08-13 18:06:06.762900 7f5c973a57a0 20 read_log 194215'18424 (192888'17926) clean e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by osd.31.0:1748 2014-08-12 15:43:43.797872 2014-08-13 18:06:06.762912 7f5c973a57a0 20 read_log 194215'18425 (194215'18424) delete e9902de4/benchmark_data_storage0111-ib.pocstor_41097_object8641/head//24 by client.2174595.0:100321 0.000000 2014-08-13 18:06:06.762923 7f5c973a57a0 20 read_log 194215'18426 (192888'18114) clean d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by osd.31.0:1750 2014-08-12 15:43:43.804848 2014-08-13 18:06:06.762936 7f5c973a57a0 20 read_log 194215'18427 (194215'18426) delete d4952de4/benchmark_data_storage0111-ib.pocstor_2228_object6461/head//24 by client.2174595.0:100323 0.000000 2014-08-13 18:06:06.762947 7f5c973a57a0 20 read_log 194215'18428 (192888'17953) clean e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by osd.31.0:1752 2014-08-12 15:43:43.812866 2014-08-13 18:06:06.762960 7f5c973a57a0 20 read_log 194215'18429 (194215'18428) delete e67a2de4/benchmark_data_storage0111-ib.pocstor_41097_object20308/head//24 by client.2174595.0:100325 0.000000 2014-08-13 18:06:06.762972 7f5c973a57a0 20 read_log 194215'18430 (192888'17943) clean 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by osd.31.0:1754 2014-08-12 15:43:43.820773 2014-08-13 18:06:06.762985 7f5c973a57a0 20 read_log 194215'18431 (194215'18430) delete 8a924de4/benchmark_data_storage0111-ib.pocstor_41097_object15032/head//24 by client.2174595.0:100327 0.000000 2014-08-13 18:06:06.763003 7f5c973a57a0 20 read_log 194215'18432 (193305'18247) clean bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by osd.31.0:1756 2014-08-12 15:43:43.828648 2014-08-13 18:06:06.763016 7f5c973a57a0 20 read_log 194215'18433 (194215'18432) delete bb0c4de4/benchmark_data_storage0111-ib.pocstor_62621_object12653/head//24 by client.2174595.0:100329 0.000000 2014-08-13 18:06:06.763027 7f5c973a57a0 20 read_log 194215'18434 (192888'17918) clean 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by osd.31.0:1758 2014-08-12 15:43:43.835375 2014-08-13 18:06:06.763049 7f5c973a57a0 20 read_log 194215'18435 (194215'18434) delete 57dd4de4/benchmark_data_storage0111-ib.pocstor_41097_object5424/head//24 by client.2174595.0:100331 0.000000 2014-08-13 18:06:06.763063 7f5c973a57a0 20 read_log 194215'18436 (192888'18137) clean 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by osd.31.0:1760 2014-08-12 15:43:43.842121 2014-08-13 18:06:06.763076 7f5c973a57a0 20 read_log 194215'18437 (194215'18436) delete 8d15de4/benchmark_data_storage0111-ib.pocstor_2228_object17231/head//24 by client.2174595.0:100333 0.000000 2014-08-13 18:06:06.763087 7f5c973a57a0 20 read_log 194215'18438 (192834'17433) clean 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by osd.31.0:1762 2014-08-12 15:43:43.848946 2014-08-13 18:06:06.763099 7f5c973a57a0 20 read_log 194215'18439 (194215'18438) delete 1dd75de4/benchmark_data_storage0111-ib.pocstor_41526_object20813/head//24 by client.2174595.0:100335 0.000000 2014-08-13 18:06:06.763119 7f5c973a57a0 20 read_log 194215'18440 (192888'18123) clean 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by osd.31.0:1764 2014-08-12 15:43:43.856049 2014-08-13 18:06:06.763133 7f5c973a57a0 20 read_log 194215'18441 (194215'18440) delete 8d006de4/benchmark_data_storage0111-ib.pocstor_2228_object11565/head//24 by client.2174595.0:100337 0.000000 2014-08-13 18:06:06.763144 7f5c973a57a0 20 read_log 194215'18442 (192888'18117) clean b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by osd.31.0:1766 2014-08-12 15:43:43.862953 2014-08-13 18:06:06.763164 7f5c973a57a0 20 read_log 194215'18443 (194215'18442) delete b6226de4/benchmark_data_storage0111-ib.pocstor_2228_object8266/head//24 by client.2174595.0:100339 0.000000 2014-08-13 18:06:06.763175 7f5c973a57a0 20 read_log 194215'18444 (192888'17933) clean b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by osd.31.0:1768 2014-08-12 15:43:43.869724 2014-08-13 18:06:06.763188 7f5c973a57a0 20 read_log 194215'18445 (194215'18444) delete b2736de4/benchmark_data_storage0111-ib.pocstor_41097_object10670/head//24 by client.2174595.0:100341 0.000000 2014-08-13 18:06:06.763202 7f5c973a57a0 20 read_log 194215'18446 (192888'17963) clean d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by osd.31.0:1770 2014-08-12 15:43:43.876164 2014-08-13 18:06:06.763214 7f5c973a57a0 20 read_log 194215'18447 (194215'18446) delete d1b36de4/benchmark_data_storage0111-ib.pocstor_41097_object22703/head//24 by client.2174595.0:100343 0.000000 2014-08-13 18:06:06.763225 7f5c973a57a0 20 read_log 194215'18448 (192888'18105) clean c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by osd.31.0:1772 2014-08-12 15:43:43.883207 2014-08-13 18:06:06.763237 7f5c973a57a0 20 read_log 194215'18449 (194215'18448) delete c7ee6de4/benchmark_data_storage0111-ib.pocstor_2228_object358/head//24 by client.2174595.0:100345 0.000000 2014-08-13 18:06:06.763252 7f5c973a57a0 20 read_log 194215'18450 (193305'18268) clean d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by osd.31.0:1774 2014-08-12 15:43:43.890484 2014-08-13 18:06:06.763264 7f5c973a57a0 20 read_log 194215'18451 (194215'18450) delete d12d7de4/benchmark_data_storage0111-ib.pocstor_62621_object30427/head//24 by client.2174595.0:100347 0.000000 2014-08-13 18:06:06.763275 7f5c973a57a0 20 read_log 194215'18452 (192888'17946) clean 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by osd.31.0:1776 2014-08-12 15:43:43.897520 2014-08-13 18:06:06.763288 7f5c973a57a0 20 read_log 194215'18453 (194215'18452) delete 3dcbade4/benchmark_data_storage0111-ib.pocstor_41097_object16550/head//24 by client.2174595.0:100349 0.000000 2014-08-13 18:06:06.763304 7f5c973a57a0 20 read_log 194215'18454 (192888'18129) clean e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by osd.31.0:1778 2014-08-12 15:43:43.904426 2014-08-13 18:06:06.763332 7f5c973a57a0 20 read_log 194215'18455 (194215'18454) delete e87ebde4/benchmark_data_storage0111-ib.pocstor_2228_object13087/head//24 by client.2174595.0:100351 0.000000 2014-08-13 18:06:06.763351 7f5c973a57a0 20 read_log 194215'18456 (192888'18152) clean 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by osd.31.0:1780 2014-08-12 15:43:43.911985 2014-08-13 18:06:06.763364 7f5c973a57a0 20 read_log 194215'18457 (194215'18456) delete 443acde4/benchmark_data_storage0111-ib.pocstor_2228_object26146/head//24 by client.2174595.0:100353 0.000000 2014-08-13 18:06:06.763382 7f5c973a57a0 20 read_log 194215'18458 (192888'18150) clean 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by osd.31.0:1782 2014-08-12 15:43:43.919809 2014-08-13 18:06:06.763395 7f5c973a57a0 20 read_log 194215'18459 (194215'18458) delete 30b6dde4/benchmark_data_storage0111-ib.pocstor_2228_object25580/head//24 by client.2174595.0:100355 0.000000 2014-08-13 18:06:06.763406 7f5c973a57a0 20 read_log 194215'18460 (193305'18254) clean 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by osd.31.0:1784 2014-08-12 15:43:43.927228 2014-08-13 18:06:06.763425 7f5c973a57a0 20 read_log 194215'18461 (194215'18460) delete 93a8dde4/benchmark_data_storage0111-ib.pocstor_62621_object16782/head//24 by client.2174595.0:100357 0.000000 2014-08-13 18:06:06.763440 7f5c973a57a0 20 read_log 194215'18462 (193305'18237) clean 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by osd.31.0:1786 2014-08-12 15:43:43.934613 2014-08-13 18:06:06.763452 7f5c973a57a0 20 read_log 194215'18463 (194215'18462) delete 19cfede4/benchmark_data_storage0111-ib.pocstor_62621_object330/head//24 by client.2174595.0:100359 0.000000 2014-08-13 18:06:06.763463 7f5c973a57a0 20 read_log 194215'18464 (193305'18263) clean 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by osd.31.0:1788 2014-08-12 15:43:43.941918 2014-08-13 18:06:06.763476 7f5c973a57a0 20 read_log 194215'18465 (194215'18464) delete 359dfde4/benchmark_data_storage0111-ib.pocstor_62621_object22120/head//24 by client.2174595.0:100361 0.000000 2014-08-13 18:06:06.763487 7f5c973a57a0 20 read_log 194215'18466 (192888'17941) clean a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by osd.31.0:1790 2014-08-12 15:43:43.948996 2014-08-13 18:06:06.763499 7f5c973a57a0 20 read_log 194215'18467 (194215'18466) delete a24ffde4/benchmark_data_storage0111-ib.pocstor_41097_object13425/head//24 by client.2174595.0:100363 0.000000 2014-08-13 18:06:06.763510 7f5c973a57a0 20 read_log 194215'18468 (194215'18468) modify 1e4/hit_set_24.1e4_archive_2014-08-12 15:43:43.172622_2014-08-12 17:24:41.973907/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.974133 2014-08-13 18:06:06.763525 7f5c973a57a0 20 read_log 194215'18469 (194215'18272) delete 1e4/hit_set_24.1e4_archive_2014-08-12 15:32:25.782637_2014-08-12 15:43:43.172386/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.974133 2014-08-13 18:06:06.763538 7f5c973a57a0 20 read_log 194215'18470 (0'0) promote b9d7cde4/benchmark_data_storage0111-ib.pocstor_34628_object522/head//24 by osd.31.0:2686 2014-08-12 17:24:41.978763 2014-08-13 18:06:06.763550 7f5c973a57a0 20 read_log 194215'18471 (194215'18470) modify b9d7cde4/benchmark_data_storage0111-ib.pocstor_34628_object522/head//24 by client.2193759.0:523 2014-08-12 17:24:41.968629 2014-08-13 18:06:06.763563 7f5c973a57a0 20 read_log 194215'18472 (0'0) promote b0c31de4/benchmark_data_storage0111-ib.pocstor_34628_object1046/head//24 by osd.31.0:2699 2014-08-12 17:24:43.968374 2014-08-13 18:06:06.763575 7f5c973a57a0 20 read_log 194215'18473 (194215'18472) modify b0c31de4/benchmark_data_storage0111-ib.pocstor_34628_object1046/head//24 by client.2193759.0:1047 2014-08-12 17:24:43.957085 2014-08-13 18:06:06.763595 7f5c973a57a0 20 read_log 194217'18474 (0'0) promote 878cf9e4/benchmark_data_storage0111-ib.pocstor_39693_object68/head//24 by osd.31.0:2703 2014-08-12 17:25:03.584198 2014-08-13 18:06:06.763611 7f5c973a57a0 20 read_log 194217'18475 (194217'18474) modify 878cf9e4/benchmark_data_storage0111-ib.pocstor_39693_object68/head//24 by client.1487927.0:69 2014-08-12 17:25:03.546057 2014-08-13 18:06:06.763624 7f5c973a57a0 20 read_log 194370'18476 (0'0) promote e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by osd.31.0:2719 2014-08-12 20:30:14.450742 2014-08-13 18:06:06.763644 7f5c973a57a0 20 read_log 194370'18477 (194370'18476) modify e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by client.1508531.0:93 2014-08-12 20:30:14.442892 2014-08-13 18:06:06.763657 7f5c973a57a0 20 read_log 194370'18478 (0'0) promote d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by osd.31.0:2723 2014-08-12 20:30:15.427107 2014-08-13 18:06:06.763670 7f5c973a57a0 20 read_log 194370'18479 (194370'18478) modify d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by client.1508531.0:305 2014-08-12 20:30:15.416579 2014-08-13 18:06:06.763683 7f5c973a57a0 20 read_log 194370'18480 (0'0) promote ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by osd.31.0:2751 2014-08-12 20:30:21.605382 2014-08-13 18:06:06.763695 7f5c973a57a0 20 read_log 194370'18481 (194370'18480) modify ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by client.1508531.0:1938 2014-08-12 20:30:21.598856 2014-08-13 18:06:06.763707 7f5c973a57a0 20 read_log 194370'18482 (0'0) promote 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by osd.31.0:2765 2014-08-12 20:30:27.705091 2014-08-13 18:06:06.763719 7f5c973a57a0 20 read_log 194370'18483 (194370'18482) modify 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by client.1508531.0:3580 2014-08-12 20:30:27.698415 2014-08-13 18:06:06.763732 7f5c973a57a0 20 read_log 194370'18484 (0'0) promote e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by osd.31.0:2787 2014-08-12 20:30:33.356088 2014-08-13 18:06:06.763744 7f5c973a57a0 20 read_log 194370'18485 (194370'18484) modify e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by client.1508531.0:5021 2014-08-12 20:30:33.348586 2014-08-13 18:06:06.763756 7f5c973a57a0 20 read_log 194370'18486 (0'0) promote 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by osd.31.0:2807 2014-08-12 20:30:39.224805 2014-08-13 18:06:06.763768 7f5c973a57a0 20 read_log 194370'18487 (194370'18486) modify 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by client.1508531.0:6343 2014-08-12 20:30:39.217114 2014-08-13 18:06:06.763780 7f5c973a57a0 20 read_log 194370'18488 (0'0) promote 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by osd.31.0:2833 2014-08-12 20:30:52.362569 2014-08-13 18:06:06.763795 7f5c973a57a0 20 read_log 194370'18489 (194370'18488) modify 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by client.1508531.0:8851 2014-08-12 20:30:52.354717 2014-08-13 18:06:06.763807 7f5c973a57a0 20 read_log 194370'18490 (0'0) promote d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by osd.31.0:2835 2014-08-12 20:30:55.329290 2014-08-13 18:06:06.763821 7f5c973a57a0 20 read_log 194370'18491 (194370'18490) modify d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by client.1508531.0:9336 2014-08-12 20:30:55.321023 2014-08-13 18:06:06.763834 7f5c973a57a0 20 read_log 194370'18492 (0'0) promote 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by osd.31.0:2859 2014-08-12 20:31:11.150528 2014-08-13 18:06:06.763848 7f5c973a57a0 20 read_log 194370'18493 (194370'18492) modify 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by client.1508531.0:11971 2014-08-12 20:31:11.141221 2014-08-13 18:06:06.763868 7f5c973a57a0 20 read_log 194377'18494 (0'0) promote 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by osd.31.0:2883 2014-08-12 20:32:57.195553 2014-08-13 18:06:06.763883 7f5c973a57a0 20 read_log 194377'18495 (194377'18494) modify 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by client.2231190.0:23 2014-08-12 20:32:57.185894 2014-08-13 18:06:06.763896 7f5c973a57a0 20 read_log 194377'18496 (0'0) promote f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by osd.31.0:2917 2014-08-12 20:33:05.880103 2014-08-13 18:06:06.763910 7f5c973a57a0 20 read_log 194377'18497 (194377'18496) modify f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by client.2231190.0:2150 2014-08-12 20:33:05.869661 2014-08-13 18:06:06.763922 7f5c973a57a0 20 read_log 194380'18498 (0'0) promote 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by osd.31.0:2935 2014-08-12 20:33:23.490700 2014-08-13 18:06:06.763934 7f5c973a57a0 20 read_log 194380'18499 (194380'18498) modify 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by client.2231190.0:5035 2014-08-12 20:33:23.482176 2014-08-13 18:06:06.763947 7f5c973a57a0 20 read_log 194382'18500 (0'0) promote cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by osd.31.0:2951 2014-08-12 20:33:59.544580 2014-08-13 18:06:06.763961 7f5c973a57a0 20 read_log 194382'18501 (194382'18500) modify cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by client.2231190.0:5682 2014-08-12 20:33:59.536043 2014-08-13 18:06:06.763973 7f5c973a57a0 20 read_log 194383'18502 (0'0) promote bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by osd.31.0:2963 2014-08-12 20:34:03.551306 2014-08-13 18:06:06.763986 7f5c973a57a0 20 read_log 194383'18503 (194383'18502) modify bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by client.2231190.0:6390 2014-08-12 20:34:03.543343 2014-08-13 18:06:06.764022 7f5c973a57a0 20 read_log 194383'18504 (0'0) promote fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by osd.31.0:2977 2014-08-12 20:34:23.099986 2014-08-13 18:06:06.764037 7f5c973a57a0 20 read_log 194383'18505 (194383'18504) modify fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by client.2231190.0:7292 2014-08-12 20:34:23.091293 2014-08-13 18:06:06.764050 7f5c973a57a0 20 read_log 194393'18506 (0'0) promote 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by osd.31.0:2999 2014-08-12 20:49:25.782689 2014-08-13 18:06:06.764064 7f5c973a57a0 20 read_log 194393'18507 (194393'18506) modify 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by client.1064182.0:37 2014-08-12 20:49:25.574065 2014-08-13 18:06:06.764077 7f5c973a57a0 20 read_log 194416'18508 (0'0) promote 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by osd.31.0:3151 2014-08-13 15:28:33.652423 2014-08-13 18:06:06.764091 7f5c973a57a0 20 read_log 194416'18509 (194416'18508) modify 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by client.2441958.0:788 2014-08-13 15:28:33.643812 2014-08-13 18:06:06.764103 7f5c973a57a0 20 read_log 194416'18510 (0'0) promote fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by osd.31.0:3160 2014-08-13 15:28:36.604011 2014-08-13 18:06:06.764116 7f5c973a57a0 20 read_log 194416'18511 (194416'18510) modify fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by client.2441958.0:1552 2014-08-13 15:28:36.596389 2014-08-13 18:06:06.764130 7f5c973a57a0 20 read_log 194416'18512 (0'0) promote 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by osd.31.0:3166 2014-08-13 15:28:39.783912 2014-08-13 18:06:06.764170 7f5c973a57a0 20 read_log 194416'18513 (194416'18512) modify 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by client.2441958.0:2395 2014-08-13 15:28:39.775215 2014-08-13 18:06:06.764186 7f5c973a57a0 20 read_log 194416'18514 (0'0) promote ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by osd.31.0:3191 2014-08-13 15:28:49.476368 2014-08-13 18:06:06.764199 7f5c973a57a0 20 read_log 194416'18515 (194416'18514) modify ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by client.2441958.0:4890 2014-08-13 15:28:49.468609 2014-08-13 18:06:06.764213 7f5c973a57a0 20 read_log 194416'18516 (0'0) promote a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by osd.31.0:3229 2014-08-13 15:29:00.207103 2014-08-13 18:06:06.764225 7f5c973a57a0 20 read_log 194416'18517 (194416'18516) modify a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by client.2441958.0:7648 2014-08-13 15:29:00.199648 2014-08-13 18:06:06.764238 7f5c973a57a0 20 read_log 194416'18518 (0'0) promote 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by osd.31.0:3231 2014-08-13 15:29:00.319737 2014-08-13 18:06:06.764252 7f5c973a57a0 20 read_log 194416'18519 (194416'18518) modify 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by client.2441958.0:7678 2014-08-13 15:29:00.311236 2014-08-13 18:06:06.764267 7f5c973a57a0 20 read_log 194416'18520 (0'0) promote 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by osd.31.0:3239 2014-08-13 15:29:03.131875 2014-08-13 18:06:06.764279 7f5c973a57a0 20 read_log 194416'18521 (194416'18520) modify 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by client.2441958.0:8424 2014-08-13 15:29:03.124637 2014-08-13 18:06:06.764292 7f5c973a57a0 20 read_log 194416'18522 (0'0) promote 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by osd.31.0:3263 2014-08-13 15:29:10.631590 2014-08-13 18:06:06.764307 7f5c973a57a0 20 read_log 194416'18523 (194416'18522) modify 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by client.2441958.0:10287 2014-08-13 15:29:10.623621 2014-08-13 18:06:06.764320 7f5c973a57a0 20 read_log 194416'18524 (0'0) promote a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by osd.31.0:3271 2014-08-13 15:29:14.538227 2014-08-13 18:06:06.764332 7f5c973a57a0 20 read_log 194416'18525 (194416'18524) modify a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by client.2441958.0:11262 2014-08-13 15:29:14.530778 2014-08-13 18:06:06.764344 7f5c973a57a0 20 read_log 194416'18526 (0'0) promote 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by osd.31.0:3291 2014-08-13 15:29:19.404865 2014-08-13 18:06:06.764362 7f5c973a57a0 20 read_log 194416'18527 (194416'18526) modify 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by client.2441958.0:12424 2014-08-13 15:29:19.397388 2014-08-13 18:06:06.764384 7f5c973a57a0 20 read_log 194416'18528 (0'0) promote 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by osd.31.0:3293 2014-08-13 15:29:20.407581 2014-08-13 18:06:06.764404 7f5c973a57a0 20 read_log 194416'18529 (194416'18528) modify 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by client.2441958.0:12661 2014-08-13 15:29:20.398665 2014-08-13 18:06:06.764419 7f5c973a57a0 20 read_log 194416'18530 (0'0) promote c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by osd.31.0:3299 2014-08-13 15:29:21.286211 2014-08-13 18:06:06.764444 7f5c973a57a0 20 read_log 194416'18531 (194416'18530) modify c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by client.2441958.0:12860 2014-08-13 15:29:21.278560 2014-08-13 18:06:06.764457 7f5c973a57a0 20 read_log 194416'18532 (0'0) promote a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by osd.31.0:3301 2014-08-13 15:29:21.520683 2014-08-13 18:06:06.764483 7f5c973a57a0 20 read_log 194416'18533 (194416'18532) modify a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by client.2441958.0:12921 2014-08-13 15:29:21.511554 2014-08-13 18:06:06.764499 7f5c973a57a0 20 read_log 194416'18534 (0'0) promote 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by osd.31.0:3303 2014-08-13 15:29:22.370597 2014-08-13 18:06:06.764512 7f5c973a57a0 20 read_log 194416'18535 (194416'18534) modify 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by client.2441958.0:13087 2014-08-13 15:29:22.362326 2014-08-13 18:06:06.764526 7f5c973a57a0 20 read_log 194416'18536 (0'0) promote 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by osd.31.0:3319 2014-08-13 15:29:26.351879 2014-08-13 18:06:06.764539 7f5c973a57a0 20 read_log 194416'18537 (194416'18536) modify 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by client.2441958.0:14037 2014-08-13 15:29:26.343605 2014-08-13 18:06:06.764551 7f5c973a57a0 20 read_log 194416'18538 (0'0) promote cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by osd.31.0:3323 2014-08-13 15:29:27.696510 2014-08-13 18:06:06.764563 7f5c973a57a0 20 read_log 194416'18539 (194416'18538) modify cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by client.2441958.0:14386 2014-08-13 15:29:27.688209 2014-08-13 18:06:06.764588 7f5c973a57a0 20 read_log 194416'18540 (0'0) promote 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by osd.31.0:3325 2014-08-13 15:29:27.720906 2014-08-13 18:06:06.764603 7f5c973a57a0 20 read_log 194416'18541 (194416'18540) modify 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by client.2441958.0:14390 2014-08-13 15:29:27.703558 2014-08-13 18:06:06.764616 7f5c973a57a0 20 read_log 194416'18542 (0'0) promote 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by osd.31.0:3335 2014-08-13 15:29:28.840896 2014-08-13 18:06:06.764628 7f5c973a57a0 20 read_log 194416'18543 (194416'18542) modify 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by client.2441958.0:14663 2014-08-13 15:29:28.831766 2014-08-13 18:06:06.764640 7f5c973a57a0 20 read_log 194416'18544 (0'0) promote dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by osd.31.0:3341 2014-08-13 15:29:29.768284 2014-08-13 18:06:06.764652 7f5c973a57a0 20 read_log 194416'18545 (194416'18544) modify dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by client.2441958.0:14881 2014-08-13 15:29:29.757749 2014-08-13 18:06:06.764665 7f5c973a57a0 20 read_log 194422'18546 (0'0) promote a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by osd.31.0:3379 2014-08-13 15:32:18.068140 2014-08-13 18:06:06.764678 7f5c973a57a0 20 read_log 194422'18547 (194422'18546) modify a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by client.1148203.0:677 2014-08-13 15:32:18.061160 2014-08-13 18:06:06.764690 7f5c973a57a0 20 read_log 194422'18548 (0'0) promote fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by osd.31.0:3383 2014-08-13 15:32:19.172976 2014-08-13 18:06:06.764711 7f5c973a57a0 20 read_log 194422'18549 (194422'18548) modify fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by client.1148203.0:967 2014-08-13 15:32:19.166187 2014-08-13 18:06:06.764727 7f5c973a57a0 20 read_log 194422'18550 (0'0) promote 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by osd.31.0:3421 2014-08-13 15:32:27.870777 2014-08-13 18:06:06.764740 7f5c973a57a0 20 read_log 194422'18551 (194422'18550) modify 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by client.1148203.0:3204 2014-08-13 15:32:27.863636 2014-08-13 18:06:06.764754 7f5c973a57a0 20 read_log 194422'18552 (0'0) promote c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by osd.31.0:3423 2014-08-13 15:32:29.573280 2014-08-13 18:06:06.764771 7f5c973a57a0 20 read_log 194422'18553 (194422'18552) modify c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by client.1148203.0:3622 2014-08-13 15:32:29.564248 2014-08-13 18:06:06.764797 7f5c973a57a0 20 read_log 194422'18554 (0'0) promote 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by osd.31.0:3447 2014-08-13 15:32:39.301084 2014-08-13 18:06:06.764810 7f5c973a57a0 20 read_log 194422'18555 (194422'18554) modify 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by client.1148203.0:5849 2014-08-13 15:32:39.293513 2014-08-13 18:06:06.764823 7f5c973a57a0 20 read_log 194422'18556 (0'0) promote 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by osd.31.0:3451 2014-08-13 15:32:40.821318 2014-08-13 18:06:06.764835 7f5c973a57a0 20 read_log 194422'18557 (194422'18556) modify 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by client.1148203.0:6128 2014-08-13 15:32:40.813701 2014-08-13 18:06:06.764847 7f5c973a57a0 20 read_log 194422'18558 (0'0) promote d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by osd.31.0:3455 2014-08-13 15:32:45.294557 2014-08-13 18:06:06.764859 7f5c973a57a0 20 read_log 194422'18559 (194422'18558) modify d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by client.1148203.0:6965 2014-08-13 15:32:45.286396 2014-08-13 18:06:06.764871 7f5c973a57a0 20 read_log 194422'18560 (0'0) promote a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by osd.31.0:3461 2014-08-13 15:32:47.070933 2014-08-13 18:06:06.764883 7f5c973a57a0 20 read_log 194422'18561 (194422'18560) modify a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by client.1148203.0:7254 2014-08-13 15:32:47.062071 2014-08-13 18:06:06.764896 7f5c973a57a0 20 read_log 194422'18562 (0'0) promote c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by osd.31.0:3475 2014-08-13 15:32:57.016344 2014-08-13 18:06:06.764908 7f5c973a57a0 20 read_log 194422'18563 (194422'18562) modify c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by client.1148203.0:8737 2014-08-13 15:32:57.007687 2014-08-13 18:06:06.764920 7f5c973a57a0 20 read_log 194422'18564 (0'0) promote 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by osd.31.0:3485 2014-08-13 15:33:00.349920 2014-08-13 18:06:06.764932 7f5c973a57a0 20 read_log 194422'18565 (194422'18564) modify 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by client.1148203.0:9077 2014-08-13 15:33:00.341640 2014-08-13 18:06:06.764945 7f5c973a57a0 20 read_log 194422'18566 (0'0) promote 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by osd.31.0:3503 2014-08-13 15:33:23.376678 2014-08-13 18:06:06.764957 7f5c973a57a0 20 read_log 194422'18567 (194422'18566) modify 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by client.1148203.0:10479 2014-08-13 15:33:23.366178 2014-08-13 18:06:06.764973 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.765401 7f5c973a57a0 10 read_log checking for missing items over interval (194370'18476,194422'18567] 2014-08-13 18:06:06.765448 7f5c973a57a0 15 read_log missing 194422'18567 (194422'18566) modify 430adde4/benchmark_data_storage0111-ib.pocstor_41231_object10478/head//24 by client.1148203.0:10479 2014-08-13 15:33:23.366178 2014-08-13 18:06:06.765489 7f5c973a57a0 15 read_log missing 194422'18565 (194422'18564) modify 6e6edde4/benchmark_data_storage0111-ib.pocstor_41231_object9076/head//24 by client.1148203.0:9077 2014-08-13 15:33:00.341640 2014-08-13 18:06:06.765525 7f5c973a57a0 15 read_log missing 194422'18563 (194422'18562) modify c153ede4/benchmark_data_storage0111-ib.pocstor_41231_object8736/head//24 by client.1148203.0:8737 2014-08-13 15:32:57.007687 2014-08-13 18:06:06.765561 7f5c973a57a0 15 read_log missing 194422'18561 (194422'18560) modify a34c01e4/benchmark_data_storage0111-ib.pocstor_41231_object7253/head//24 by client.1148203.0:7254 2014-08-13 15:32:47.062071 2014-08-13 18:06:06.765597 7f5c973a57a0 15 read_log missing 194422'18559 (194422'18558) modify d6a6de4/benchmark_data_storage0111-ib.pocstor_41231_object6964/head//24 by client.1148203.0:6965 2014-08-13 15:32:45.286396 2014-08-13 18:06:06.765633 7f5c973a57a0 15 read_log missing 194422'18557 (194422'18556) modify 45bad9e4/benchmark_data_storage0111-ib.pocstor_41231_object6127/head//24 by client.1148203.0:6128 2014-08-13 15:32:40.813701 2014-08-13 18:06:06.765668 7f5c973a57a0 15 read_log missing 194422'18555 (194422'18554) modify 6191a9e4/benchmark_data_storage0111-ib.pocstor_41231_object5848/head//24 by client.1148203.0:5849 2014-08-13 15:32:39.293513 2014-08-13 18:06:06.765704 7f5c973a57a0 15 read_log missing 194422'18553 (194422'18552) modify c31d39e4/benchmark_data_storage0111-ib.pocstor_41231_object3621/head//24 by client.1148203.0:3622 2014-08-13 15:32:29.564248 2014-08-13 18:06:06.765739 7f5c973a57a0 15 read_log missing 194422'18551 (194422'18550) modify 419759e4/benchmark_data_storage0111-ib.pocstor_41231_object3203/head//24 by client.1148203.0:3204 2014-08-13 15:32:27.863636 2014-08-13 18:06:06.765775 7f5c973a57a0 15 read_log missing 194422'18549 (194422'18548) modify fbd065e4/benchmark_data_storage0111-ib.pocstor_41231_object966/head//24 by client.1148203.0:967 2014-08-13 15:32:19.166187 2014-08-13 18:06:06.765813 7f5c973a57a0 15 read_log missing 194422'18547 (194422'18546) modify a04ac1e4/benchmark_data_storage0111-ib.pocstor_41231_object676/head//24 by client.1148203.0:677 2014-08-13 15:32:18.061160 2014-08-13 18:06:06.765850 7f5c973a57a0 15 read_log missing 194416'18545 (194416'18544) modify dbf749e4/benchmark_data_storage0111-ib.pocstor_19103_object14880/head//24 by client.2441958.0:14881 2014-08-13 15:29:29.757749 2014-08-13 18:06:06.765885 7f5c973a57a0 15 read_log missing 194416'18543 (194416'18542) modify 3dcee9e4/benchmark_data_storage0111-ib.pocstor_19103_object14662/head//24 by client.2441958.0:14663 2014-08-13 15:29:28.831766 2014-08-13 18:06:06.765921 7f5c973a57a0 15 read_log missing 194416'18541 (194416'18540) modify 7f9509e4/benchmark_data_storage0111-ib.pocstor_19103_object14389/head//24 by client.2441958.0:14390 2014-08-13 15:29:27.703558 2014-08-13 18:06:06.765955 7f5c973a57a0 15 read_log missing 194416'18539 (194416'18538) modify cacbbde4/benchmark_data_storage0111-ib.pocstor_19103_object14385/head//24 by client.2441958.0:14386 2014-08-13 15:29:27.688209 2014-08-13 18:06:06.765990 7f5c973a57a0 15 read_log missing 194416'18537 (194416'18536) modify 1c57fde4/benchmark_data_storage0111-ib.pocstor_19103_object14036/head//24 by client.2441958.0:14037 2014-08-13 15:29:26.343605 2014-08-13 18:06:06.766025 7f5c973a57a0 15 read_log missing 194416'18535 (194416'18534) modify 1ff355e4/benchmark_data_storage0111-ib.pocstor_19103_object13086/head//24 by client.2441958.0:13087 2014-08-13 15:29:22.362326 2014-08-13 18:06:06.766060 7f5c973a57a0 15 read_log missing 194416'18533 (194416'18532) modify a16441e4/benchmark_data_storage0111-ib.pocstor_19103_object12920/head//24 by client.2441958.0:12921 2014-08-13 15:29:21.511554 2014-08-13 18:06:06.766096 7f5c973a57a0 15 read_log missing 194416'18531 (194416'18530) modify c002de4/benchmark_data_storage0111-ib.pocstor_19103_object12859/head//24 by client.2441958.0:12860 2014-08-13 15:29:21.278560 2014-08-13 18:06:06.766132 7f5c973a57a0 15 read_log missing 194416'18529 (194416'18528) modify 735391e4/benchmark_data_storage0111-ib.pocstor_19103_object12660/head//24 by client.2441958.0:12661 2014-08-13 15:29:20.398665 2014-08-13 18:06:06.766176 7f5c973a57a0 15 read_log missing 194416'18527 (194416'18526) modify 3906b5e4/benchmark_data_storage0111-ib.pocstor_19103_object12423/head//24 by client.2441958.0:12424 2014-08-13 15:29:19.397388 2014-08-13 18:06:06.766214 7f5c973a57a0 15 read_log missing 194416'18525 (194416'18524) modify a67c1de4/benchmark_data_storage0111-ib.pocstor_19103_object11261/head//24 by client.2441958.0:11262 2014-08-13 15:29:14.530778 2014-08-13 18:06:06.766254 7f5c973a57a0 15 read_log missing 194416'18523 (194416'18522) modify 59353de4/benchmark_data_storage0111-ib.pocstor_19103_object10286/head//24 by client.2441958.0:10287 2014-08-13 15:29:10.623621 2014-08-13 18:06:06.766295 7f5c973a57a0 15 read_log missing 194416'18521 (194416'18520) modify 29baade4/benchmark_data_storage0111-ib.pocstor_19103_object8423/head//24 by client.2441958.0:8424 2014-08-13 15:29:03.124637 2014-08-13 18:06:06.766332 7f5c973a57a0 15 read_log missing 194416'18519 (194416'18518) modify 85bdb5e4/benchmark_data_storage0111-ib.pocstor_19103_object7677/head//24 by client.2441958.0:7678 2014-08-13 15:29:00.311236 2014-08-13 18:06:06.766379 7f5c973a57a0 15 read_log missing 194416'18517 (194416'18516) modify a1ca21e4/benchmark_data_storage0111-ib.pocstor_19103_object7647/head//24 by client.2441958.0:7648 2014-08-13 15:29:00.199648 2014-08-13 18:06:06.766418 7f5c973a57a0 15 read_log missing 194416'18515 (194416'18514) modify ee2291e4/benchmark_data_storage0111-ib.pocstor_19103_object4889/head//24 by client.2441958.0:4890 2014-08-13 15:28:49.468609 2014-08-13 18:06:06.766454 7f5c973a57a0 15 read_log missing 194416'18513 (194416'18512) modify 6bc961e4/benchmark_data_storage0111-ib.pocstor_19103_object2394/head//24 by client.2441958.0:2395 2014-08-13 15:28:39.775215 2014-08-13 18:06:06.766490 7f5c973a57a0 15 read_log missing 194416'18511 (194416'18510) modify fccb09e4/benchmark_data_storage0111-ib.pocstor_19103_object1551/head//24 by client.2441958.0:1552 2014-08-13 15:28:36.596389 2014-08-13 18:06:06.766532 7f5c973a57a0 15 read_log missing 194416'18509 (194416'18508) modify 46231e4/benchmark_data_storage0111-ib.pocstor_19103_object787/head//24 by client.2441958.0:788 2014-08-13 15:28:33.643812 2014-08-13 18:06:06.766569 7f5c973a57a0 15 read_log missing 194393'18507 (194393'18506) modify 42a359e4/benchmark_data_storage0111-ib.pocstor_32482_object36/head//24 by client.1064182.0:37 2014-08-12 20:49:25.574065 2014-08-13 18:06:06.766605 7f5c973a57a0 15 read_log missing 194383'18505 (194383'18504) modify fd1541e4/benchmark_data_storage0111-ib.pocstor_25065_object7291/head//24 by client.2231190.0:7292 2014-08-12 20:34:23.091293 2014-08-13 18:06:06.766640 7f5c973a57a0 15 read_log missing 194383'18503 (194383'18502) modify bb23f1e4/benchmark_data_storage0111-ib.pocstor_25065_object6389/head//24 by client.2231190.0:6390 2014-08-12 20:34:03.543343 2014-08-13 18:06:06.766698 7f5c973a57a0 15 read_log missing 194382'18501 (194382'18500) modify cc70f5e4/benchmark_data_storage0111-ib.pocstor_25065_object5681/head//24 by client.2231190.0:5682 2014-08-12 20:33:59.536043 2014-08-13 18:06:06.766735 7f5c973a57a0 15 read_log missing 194380'18499 (194380'18498) modify 43bbe1e4/benchmark_data_storage0111-ib.pocstor_25065_object5034/head//24 by client.2231190.0:5035 2014-08-12 20:33:23.482176 2014-08-13 18:06:06.766771 7f5c973a57a0 15 read_log missing 194377'18497 (194377'18496) modify f5d2b1e4/benchmark_data_storage0111-ib.pocstor_25065_object2149/head//24 by client.2231190.0:2150 2014-08-12 20:33:05.869661 2014-08-13 18:06:06.766807 7f5c973a57a0 15 read_log missing 194377'18495 (194377'18494) modify 83735e4/benchmark_data_storage0111-ib.pocstor_25065_object22/head//24 by client.2231190.0:23 2014-08-12 20:32:57.185894 2014-08-13 18:06:06.766843 7f5c973a57a0 15 read_log missing 194370'18493 (194370'18492) modify 286d21e4/benchmark_data_storage0111-ib.pocstor_12495_object11970/head//24 by client.1508531.0:11971 2014-08-12 20:31:11.141221 2014-08-13 18:06:06.766880 7f5c973a57a0 15 read_log missing 194370'18491 (194370'18490) modify d39af5e4/benchmark_data_storage0111-ib.pocstor_12495_object9335/head//24 by client.1508531.0:9336 2014-08-12 20:30:55.321023 2014-08-13 18:06:06.766916 7f5c973a57a0 15 read_log missing 194370'18489 (194370'18488) modify 217abde4/benchmark_data_storage0111-ib.pocstor_12495_object8850/head//24 by client.1508531.0:8851 2014-08-12 20:30:52.354717 2014-08-13 18:06:06.766952 7f5c973a57a0 15 read_log missing 194370'18487 (194370'18486) modify 9a811e4/benchmark_data_storage0111-ib.pocstor_12495_object6342/head//24 by client.1508531.0:6343 2014-08-12 20:30:39.217114 2014-08-13 18:06:06.766988 7f5c973a57a0 15 read_log missing 194370'18485 (194370'18484) modify e99e11e4/benchmark_data_storage0111-ib.pocstor_12495_object5020/head//24 by client.1508531.0:5021 2014-08-12 20:30:33.348586 2014-08-13 18:06:06.767024 7f5c973a57a0 15 read_log missing 194370'18483 (194370'18482) modify 62ec91e4/benchmark_data_storage0111-ib.pocstor_12495_object3579/head//24 by client.1508531.0:3580 2014-08-12 20:30:27.698415 2014-08-13 18:06:06.767059 7f5c973a57a0 15 read_log missing 194370'18481 (194370'18480) modify ffb6e1e4/benchmark_data_storage0111-ib.pocstor_12495_object1937/head//24 by client.1508531.0:1938 2014-08-12 20:30:21.598856 2014-08-13 18:06:06.767095 7f5c973a57a0 15 read_log missing 194370'18479 (194370'18478) modify d35031e4/benchmark_data_storage0111-ib.pocstor_12495_object304/head//24 by client.1508531.0:305 2014-08-12 20:30:15.416579 2014-08-13 18:06:06.767131 7f5c973a57a0 15 read_log missing 194370'18477 (194370'18476) modify e3d8f1e4/benchmark_data_storage0111-ib.pocstor_12495_object92/head//24 by client.1508531.0:93 2014-08-12 20:30:14.442892 2014-08-13 18:06:06.767151 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.767177 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=0 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive m=46] handle_loaded 2014-08-13 18:06:06.767194 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=0 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Initial 0.013966 0 0.000000 2014-08-13 18:06:06.767207 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=0 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Reset 2014-08-13 18:06:06.767216 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=0 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194487 2014-08-13 18:06:06.767226 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log((192834'16484,194422'18567], crt=194422'18567) 2014-08-13 18:06:06.767240 7f5c973a57a0 10 osd.4 194487 pgid 24.1e6 coll 24.1e6_head 2014-08-13 18:06:06.767402 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.1e6 2014-08-13 18:06:06.767422 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.767472 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e6(unlocked)] enter Initial 2014-08-13 18:06:06.767492 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1e6(unlocked)] enter NotTrimming 2014-08-13 18:06:06.767647 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.767778 7f5c973a57a0 20 read_log 192834'15882 (0'0) promote 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by osd.364.0:17853 2014-08-11 15:23:39.586904 2014-08-13 18:06:06.767799 7f5c973a57a0 20 read_log 192834'15883 (192834'15882) modify 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.933448.0:1026 2014-08-11 15:23:39.580290 2014-08-13 18:06:06.767813 7f5c973a57a0 20 read_log 192834'15894 (0'0) promote 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by osd.364.0:17865 2014-08-11 15:23:43.322273 2014-08-13 18:06:06.767826 7f5c973a57a0 20 read_log 192834'15895 (192834'15894) modify 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.933448.0:1832 2014-08-11 15:23:43.312343 2014-08-13 18:06:06.767839 7f5c973a57a0 20 read_log 192834'15902 (0'0) promote 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by osd.364.0:17873 2014-08-11 15:23:49.780370 2014-08-13 18:06:06.767851 7f5c973a57a0 20 read_log 192834'15903 (192834'15902) modify 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.933448.0:3010 2014-08-11 15:23:49.772085 2014-08-13 18:06:06.767863 7f5c973a57a0 20 read_log 192834'15918 (0'0) promote 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by osd.364.0:17889 2014-08-11 15:23:55.304268 2014-08-13 18:06:06.767875 7f5c973a57a0 20 read_log 192834'15919 (192834'15918) modify 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.933448.0:4059 2014-08-11 15:23:55.296455 2014-08-13 18:06:06.767888 7f5c973a57a0 20 read_log 192834'15920 (0'0) promote 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by osd.364.0:17891 2014-08-11 15:23:55.464693 2014-08-13 18:06:06.767900 7f5c973a57a0 20 read_log 192834'15921 (192834'15920) modify 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.933448.0:4100 2014-08-11 15:23:55.456664 2014-08-13 18:06:06.767912 7f5c973a57a0 20 read_log 192834'15922 (0'0) promote 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by osd.364.0:17893 2014-08-11 15:23:56.084202 2014-08-13 18:06:06.767924 7f5c973a57a0 20 read_log 192834'15923 (192834'15922) modify 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.933448.0:4254 2014-08-11 15:23:56.077638 2014-08-13 18:06:06.767937 7f5c973a57a0 20 read_log 192834'15930 (0'0) promote b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by osd.364.0:17901 2014-08-11 15:24:02.129142 2014-08-13 18:06:06.767949 7f5c973a57a0 20 read_log 192834'15931 (192834'15930) modify b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.933448.0:5590 2014-08-11 15:24:02.120743 2014-08-13 18:06:06.767961 7f5c973a57a0 20 read_log 192834'15934 (0'0) promote f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by osd.364.0:17905 2014-08-11 15:24:02.670617 2014-08-13 18:06:06.767973 7f5c973a57a0 20 read_log 192834'15935 (192834'15934) modify f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.933448.0:5730 2014-08-11 15:24:02.661350 2014-08-13 18:06:06.767986 7f5c973a57a0 20 read_log 192834'15952 (0'0) promote 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by osd.364.0:17923 2014-08-11 15:24:09.267619 2014-08-13 18:06:06.768015 7f5c973a57a0 20 read_log 192834'15953 (192834'15952) modify 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.933448.0:7234 2014-08-11 15:24:09.260092 2014-08-13 18:06:06.768032 7f5c973a57a0 20 read_log 192834'15984 (0'0) promote dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by osd.364.0:17955 2014-08-11 15:24:16.262325 2014-08-13 18:06:06.768044 7f5c973a57a0 20 read_log 192834'15985 (192834'15984) modify dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.933448.0:8938 2014-08-11 15:24:16.254647 2014-08-13 18:06:06.768056 7f5c973a57a0 20 read_log 192834'15992 (0'0) promote 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by osd.364.0:17963 2014-08-11 15:24:17.397424 2014-08-13 18:06:06.768069 7f5c973a57a0 20 read_log 192834'15993 (192834'15992) modify 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.933448.0:9059 2014-08-11 15:24:17.389655 2014-08-13 18:06:06.768081 7f5c973a57a0 20 read_log 192834'16002 (0'0) promote c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by osd.364.0:17973 2014-08-11 15:24:19.384870 2014-08-13 18:06:06.768093 7f5c973a57a0 20 read_log 192834'16003 (192834'16002) modify c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.933448.0:9571 2014-08-11 15:24:19.376173 2014-08-13 18:06:06.768108 7f5c973a57a0 20 read_log 192834'16016 (0'0) promote e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by osd.364.0:17987 2014-08-11 15:24:22.418231 2014-08-13 18:06:06.768121 7f5c973a57a0 20 read_log 192834'16017 (192834'16016) modify e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.933448.0:10287 2014-08-11 15:24:22.405262 2014-08-13 18:06:06.768133 7f5c973a57a0 20 read_log 192834'16046 (0'0) promote af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by osd.364.0:18017 2014-08-11 15:24:32.428319 2014-08-13 18:06:06.768151 7f5c973a57a0 20 read_log 192834'16047 (192834'16046) modify af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.933448.0:12413 2014-08-11 15:24:32.251592 2014-08-13 18:06:06.768165 7f5c973a57a0 20 read_log 192834'16050 (0'0) promote 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by osd.364.0:18021 2014-08-11 15:24:33.842704 2014-08-13 18:06:06.768177 7f5c973a57a0 20 read_log 192834'16051 (192834'16050) modify 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.933448.0:12785 2014-08-11 15:24:33.834503 2014-08-13 18:06:06.768190 7f5c973a57a0 20 read_log 192834'16068 (0'0) promote f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by osd.364.0:18039 2014-08-11 15:24:40.242968 2014-08-13 18:06:06.768202 7f5c973a57a0 20 read_log 192834'16069 (192834'16068) modify f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.933448.0:13865 2014-08-11 15:24:40.234958 2014-08-13 18:06:06.768214 7f5c973a57a0 20 read_log 192834'16076 (0'0) promote 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by osd.364.0:18047 2014-08-11 15:24:45.542179 2014-08-13 18:06:06.768226 7f5c973a57a0 20 read_log 192834'16077 (192834'16076) modify 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.933448.0:14446 2014-08-11 15:24:45.534415 2014-08-13 18:06:06.768242 7f5c973a57a0 20 read_log 192834'16084 (0'0) promote 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by osd.364.0:18055 2014-08-11 15:24:47.863403 2014-08-13 18:06:06.768258 7f5c973a57a0 20 read_log 192834'16085 (192834'16084) modify 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.933448.0:14964 2014-08-11 15:24:47.856554 2014-08-13 18:06:06.768270 7f5c973a57a0 20 read_log 192834'16098 (0'0) promote 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by osd.364.0:18069 2014-08-11 15:24:52.941515 2014-08-13 18:06:06.768291 7f5c973a57a0 20 read_log 192834'16099 (192834'16098) modify 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.933448.0:16084 2014-08-11 15:24:52.929345 2014-08-13 18:06:06.768306 7f5c973a57a0 20 read_log 192834'16102 (0'0) promote b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by osd.364.0:18073 2014-08-11 15:24:55.790808 2014-08-13 18:06:06.768319 7f5c973a57a0 20 read_log 192834'16103 (192834'16102) modify b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.933448.0:16455 2014-08-11 15:24:55.783520 2014-08-13 18:06:06.768331 7f5c973a57a0 20 read_log 192834'16112 (0'0) promote 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by osd.364.0:18083 2014-08-11 15:24:57.721227 2014-08-13 18:06:06.768343 7f5c973a57a0 20 read_log 192834'16113 (192834'16112) modify 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.933448.0:16920 2014-08-11 15:24:57.714232 2014-08-13 18:06:06.768356 7f5c973a57a0 20 read_log 192834'16130 (0'0) promote 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by osd.364.0:18101 2014-08-11 15:25:05.292172 2014-08-13 18:06:06.768382 7f5c973a57a0 20 read_log 192834'16131 (192834'16130) modify 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.933448.0:18432 2014-08-11 15:25:05.285860 2014-08-13 18:06:06.768402 7f5c973a57a0 20 read_log 192834'16148 (0'0) promote e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by osd.364.0:18119 2014-08-11 15:25:13.939729 2014-08-13 18:06:06.768427 7f5c973a57a0 20 read_log 192834'16149 (192834'16148) modify e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.933448.0:19910 2014-08-11 15:25:13.932531 2014-08-13 18:06:06.768441 7f5c973a57a0 20 read_log 192834'16172 (0'0) promote afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by osd.364.0:18143 2014-08-11 15:25:28.699526 2014-08-13 18:06:06.768453 7f5c973a57a0 20 read_log 192834'16173 (192834'16172) modify afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.933448.0:21357 2014-08-11 15:25:28.692663 2014-08-13 18:06:06.768477 7f5c973a57a0 20 read_log 192834'16195 (192834'15883) modify 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.933448.0:23317 2014-08-11 15:25:39.220819 2014-08-13 18:06:06.768493 7f5c973a57a0 20 read_log 192834'16201 (192834'15895) modify 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.933448.0:24123 2014-08-11 15:25:40.307083 2014-08-13 18:06:06.768505 7f5c973a57a0 20 read_log 192834'16205 (192834'15903) modify 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.933448.0:25301 2014-08-11 15:25:41.980288 2014-08-13 18:06:06.768518 7f5c973a57a0 20 read_log 192834'16212 (192834'15919) modify 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.933448.0:26350 2014-08-11 15:25:43.290252 2014-08-13 18:06:06.768530 7f5c973a57a0 20 read_log 192834'16213 (192834'15921) modify 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.933448.0:26391 2014-08-11 15:25:43.341983 2014-08-13 18:06:06.768562 7f5c973a57a0 20 read_log 192834'16214 (192834'15923) modify 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.933448.0:26545 2014-08-11 15:25:43.524028 2014-08-13 18:06:06.768575 7f5c973a57a0 20 read_log 192834'16218 (192834'15931) modify b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.933448.0:27881 2014-08-11 15:25:45.223632 2014-08-13 18:06:06.768587 7f5c973a57a0 20 read_log 192834'16220 (192834'15935) modify f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.933448.0:28021 2014-08-11 15:25:45.393622 2014-08-13 18:06:06.768601 7f5c973a57a0 20 read_log 192834'16229 (192834'15953) modify 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.933448.0:29525 2014-08-11 15:25:47.429831 2014-08-13 18:06:06.768669 7f5c973a57a0 20 read_log 192834'16245 (192834'15985) modify dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.933448.0:31229 2014-08-11 15:25:49.820651 2014-08-13 18:06:06.768692 7f5c973a57a0 20 read_log 192834'16249 (192834'15993) modify 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.933448.0:31350 2014-08-11 15:25:49.955965 2014-08-13 18:06:06.768722 7f5c973a57a0 20 read_log 192834'16254 (192834'16003) modify c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.933448.0:31862 2014-08-11 15:25:50.702478 2014-08-13 18:06:06.768747 7f5c973a57a0 20 read_log 192834'16261 (192834'16017) modify e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.933448.0:32578 2014-08-11 15:25:52.040119 2014-08-13 18:06:06.768763 7f5c973a57a0 20 read_log 192834'16276 (192834'16047) modify af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.933448.0:34704 2014-08-11 15:25:55.380170 2014-08-13 18:06:06.768840 7f5c973a57a0 20 read_log 192834'16278 (192834'16051) modify 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.933448.0:35076 2014-08-11 15:25:55.776916 2014-08-13 18:06:06.768881 7f5c973a57a0 20 read_log 192834'16287 (192834'16069) modify f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.933448.0:36156 2014-08-11 15:25:57.304427 2014-08-13 18:06:06.769064 7f5c973a57a0 20 read_log 192834'16291 (192834'16077) modify 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.933448.0:36737 2014-08-11 15:25:58.255174 2014-08-13 18:06:06.769101 7f5c973a57a0 20 read_log 192834'16295 (192834'16085) modify 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.933448.0:37255 2014-08-11 15:25:59.212294 2014-08-13 18:06:06.769115 7f5c973a57a0 20 read_log 192834'16302 (192834'16099) modify 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.933448.0:38375 2014-08-11 15:26:00.827633 2014-08-13 18:06:06.769131 7f5c973a57a0 20 read_log 192834'16304 (192834'16103) modify b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.933448.0:38746 2014-08-11 15:26:01.211878 2014-08-13 18:06:06.769144 7f5c973a57a0 20 read_log 192834'16309 (192834'16113) modify 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.933448.0:39211 2014-08-11 15:26:01.672923 2014-08-13 18:06:06.769170 7f5c973a57a0 20 read_log 192834'16318 (192834'16131) modify 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.933448.0:40723 2014-08-11 15:26:02.710004 2014-08-13 18:06:06.769184 7f5c973a57a0 20 read_log 192834'16327 (192834'16149) modify e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.933448.0:42201 2014-08-11 15:26:03.187455 2014-08-13 18:06:06.769197 7f5c973a57a0 20 read_log 192834'16339 (192834'16173) modify afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.933448.0:43648 2014-08-11 15:26:03.685002 2014-08-13 18:06:06.769210 7f5c973a57a0 20 read_log 192834'16367 (0'0) promote a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by osd.364.0:18338 2014-08-11 15:26:18.652144 2014-08-13 18:06:06.769226 7f5c973a57a0 20 read_log 192834'16368 (192834'16367) modify a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.1883634.0:1146 2014-08-11 15:26:18.644718 2014-08-13 18:06:06.769243 7f5c973a57a0 20 read_log 192834'16391 (0'0) promote ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by osd.364.0:18362 2014-08-11 15:26:24.194229 2014-08-13 18:06:06.769255 7f5c973a57a0 20 read_log 192834'16392 (192834'16391) modify ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.1883634.0:2642 2014-08-11 15:26:24.186026 2014-08-13 18:06:06.769268 7f5c973a57a0 20 read_log 192834'16401 (0'0) promote 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by osd.364.0:18372 2014-08-11 15:26:27.375534 2014-08-13 18:06:06.769293 7f5c973a57a0 20 read_log 192834'16402 (192834'16401) modify 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.1883634.0:3542 2014-08-11 15:26:27.369036 2014-08-13 18:06:06.769306 7f5c973a57a0 20 read_log 192834'16413 (0'0) promote 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by osd.364.0:18384 2014-08-11 15:26:29.542361 2014-08-13 18:06:06.769318 7f5c973a57a0 20 read_log 192834'16414 (192834'16413) modify 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.1883634.0:4154 2014-08-11 15:26:29.534485 2014-08-13 18:06:06.769330 7f5c973a57a0 20 read_log 192834'16419 (0'0) promote 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by osd.364.0:18390 2014-08-11 15:26:31.070202 2014-08-13 18:06:06.769353 7f5c973a57a0 20 read_log 192834'16420 (192834'16419) modify 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.1883634.0:4562 2014-08-11 15:26:31.062838 2014-08-13 18:06:06.769366 7f5c973a57a0 20 read_log 192834'16447 (0'0) promote 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by osd.364.0:18418 2014-08-11 15:26:36.335015 2014-08-13 18:06:06.769378 7f5c973a57a0 20 read_log 192834'16448 (192834'16447) modify 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.1883634.0:5979 2014-08-11 15:26:36.325899 2014-08-13 18:06:06.769394 7f5c973a57a0 20 read_log 192834'16467 (0'0) promote f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by osd.364.0:18438 2014-08-11 15:26:42.003939 2014-08-13 18:06:06.769407 7f5c973a57a0 20 read_log 192834'16468 (192834'16467) modify f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.1883634.0:7394 2014-08-11 15:26:41.996874 2014-08-13 18:06:06.769436 7f5c973a57a0 20 read_log 192834'16471 (0'0) promote 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by osd.364.0:18442 2014-08-11 15:26:43.029553 2014-08-13 18:06:06.769456 7f5c973a57a0 20 read_log 192834'16472 (192834'16471) modify 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.1883634.0:7622 2014-08-11 15:26:43.021221 2014-08-13 18:06:06.769481 7f5c973a57a0 20 read_log 192834'16477 (0'0) promote d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by osd.364.0:18448 2014-08-11 15:26:45.635271 2014-08-13 18:06:06.769498 7f5c973a57a0 20 read_log 192834'16478 (192834'16477) modify d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.1883634.0:8276 2014-08-11 15:26:45.626360 2014-08-13 18:06:06.769514 7f5c973a57a0 20 read_log 192834'16501 (0'0) promote da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by osd.364.0:18472 2014-08-11 15:26:51.199698 2014-08-13 18:06:06.769526 7f5c973a57a0 20 read_log 192834'16502 (192834'16501) modify da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.1883634.0:9534 2014-08-11 15:26:51.191829 2014-08-13 18:06:06.769539 7f5c973a57a0 20 read_log 192834'16515 (0'0) promote c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by osd.364.0:18486 2014-08-11 15:26:57.113244 2014-08-13 18:06:06.769569 7f5c973a57a0 20 read_log 192834'16516 (192834'16515) modify c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.1883634.0:10823 2014-08-11 15:26:57.103972 2014-08-13 18:06:06.769583 7f5c973a57a0 20 read_log 192834'16547 (0'0) promote c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by osd.364.0:18518 2014-08-11 15:27:08.836329 2014-08-13 18:06:06.769598 7f5c973a57a0 20 read_log 192834'16548 (192834'16547) modify c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.1883634.0:13319 2014-08-11 15:27:08.829179 2014-08-13 18:06:06.769619 7f5c973a57a0 20 read_log 192834'16553 (0'0) promote 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by osd.364.0:18524 2014-08-11 15:27:10.324141 2014-08-13 18:06:06.769656 7f5c973a57a0 20 read_log 192834'16554 (192834'16553) modify 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.1883634.0:13635 2014-08-11 15:27:10.317673 2014-08-13 18:06:06.769670 7f5c973a57a0 20 read_log 192834'16559 (0'0) promote c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by osd.364.0:18530 2014-08-11 15:27:12.071038 2014-08-13 18:06:06.769682 7f5c973a57a0 20 read_log 192834'16560 (192834'16559) modify c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.1883634.0:14139 2014-08-11 15:27:12.064150 2014-08-13 18:06:06.769695 7f5c973a57a0 20 read_log 192834'16561 (0'0) promote 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by osd.364.0:18532 2014-08-11 15:27:12.561770 2014-08-13 18:06:06.769716 7f5c973a57a0 20 read_log 192834'16562 (192834'16561) modify 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.1883634.0:14278 2014-08-11 15:27:12.554349 2014-08-13 18:06:06.769729 7f5c973a57a0 20 read_log 192834'16587 (0'0) promote 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by osd.364.0:18558 2014-08-11 15:27:21.461374 2014-08-13 18:06:06.769742 7f5c973a57a0 20 read_log 192834'16588 (192834'16587) modify 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.1883634.0:15756 2014-08-11 15:27:21.453911 2014-08-13 18:06:06.769758 7f5c973a57a0 20 read_log 192834'16599 (0'0) promote 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by osd.364.0:18570 2014-08-11 15:27:29.935836 2014-08-13 18:06:06.769771 7f5c973a57a0 20 read_log 192834'16600 (192834'16599) modify 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.1883634.0:16922 2014-08-11 15:27:29.927352 2014-08-13 18:06:06.769783 7f5c973a57a0 20 read_log 192834'16627 (0'0) promote 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by osd.364.0:18598 2014-08-11 15:27:54.067547 2014-08-13 18:06:06.769795 7f5c973a57a0 20 read_log 192834'16628 (192834'16627) modify 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.1883634.0:19768 2014-08-11 15:27:54.060325 2014-08-13 18:06:06.769807 7f5c973a57a0 20 read_log 192834'16637 (0'0) promote 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by osd.364.0:18608 2014-08-11 15:28:02.819015 2014-08-13 18:06:06.769819 7f5c973a57a0 20 read_log 192834'16638 (192834'16637) modify 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.1883634.0:21103 2014-08-11 15:28:02.809836 2014-08-13 18:06:06.769831 7f5c973a57a0 20 read_log 192834'16639 (0'0) promote f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by osd.364.0:18610 2014-08-11 15:28:02.942581 2014-08-13 18:06:06.769843 7f5c973a57a0 20 read_log 192834'16640 (192834'16639) modify f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.1883634.0:21125 2014-08-11 15:28:02.934566 2014-08-13 18:06:06.769855 7f5c973a57a0 20 read_log 192834'16653 (0'0) promote 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by osd.364.0:18624 2014-08-11 15:28:04.329567 2014-08-13 18:06:06.769867 7f5c973a57a0 20 read_log 192834'16654 (192834'16653) modify 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.1883634.0:21487 2014-08-11 15:28:04.321602 2014-08-13 18:06:06.769879 7f5c973a57a0 20 read_log 192834'16657 (0'0) promote 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by osd.364.0:18628 2014-08-11 15:28:05.831506 2014-08-13 18:06:06.769891 7f5c973a57a0 20 read_log 192834'16658 (192834'16657) modify 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.1883634.0:21934 2014-08-11 15:28:05.822858 2014-08-13 18:06:06.769903 7f5c973a57a0 20 read_log 192834'16698 (192834'16368) modify a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.1883634.0:24470 2014-08-11 15:28:19.091751 2014-08-13 18:06:06.769944 7f5c973a57a0 20 read_log 192834'16710 (192834'16392) modify ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.1883634.0:25966 2014-08-11 15:28:22.422382 2014-08-13 18:06:06.769957 7f5c973a57a0 20 read_log 192834'16715 (192834'16402) modify 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.1883634.0:26866 2014-08-11 15:28:23.681913 2014-08-13 18:06:06.769970 7f5c973a57a0 20 read_log 192834'16721 (192834'16414) modify 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.1883634.0:27478 2014-08-11 15:28:24.506889 2014-08-13 18:06:06.769982 7f5c973a57a0 20 read_log 192834'16724 (192834'16420) modify 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.1883634.0:27886 2014-08-11 15:28:25.089688 2014-08-13 18:06:06.769994 7f5c973a57a0 20 read_log 192834'16738 (192834'16448) modify 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.1883634.0:29303 2014-08-11 15:28:26.954655 2014-08-13 18:06:06.770006 7f5c973a57a0 20 read_log 192834'16748 (192834'16468) modify f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.1883634.0:30718 2014-08-11 15:28:29.191728 2014-08-13 18:06:06.770018 7f5c973a57a0 20 read_log 192834'16750 (192834'16472) modify 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.1883634.0:30946 2014-08-11 15:28:29.623850 2014-08-13 18:06:06.770030 7f5c973a57a0 20 read_log 192834'16753 (192834'16478) modify d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.1883634.0:31600 2014-08-11 15:28:30.744187 2014-08-13 18:06:06.770042 7f5c973a57a0 20 read_log 192834'16765 (192834'16502) modify da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.1883634.0:32858 2014-08-11 15:28:32.429009 2014-08-13 18:06:06.770054 7f5c973a57a0 20 read_log 192834'16774 (192834'16516) modify c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.1883634.0:34147 2014-08-11 15:28:34.295829 2014-08-13 18:06:06.770066 7f5c973a57a0 20 read_log 192834'16790 (192834'16548) modify c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.1883634.0:36643 2014-08-11 15:28:38.245882 2014-08-13 18:06:06.770079 7f5c973a57a0 20 read_log 192834'16793 (192834'16554) modify 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.1883634.0:36959 2014-08-11 15:28:38.631608 2014-08-13 18:06:06.770091 7f5c973a57a0 20 read_log 192834'16796 (192834'16560) modify c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.1883634.0:37463 2014-08-11 15:28:39.390165 2014-08-13 18:06:06.770103 7f5c973a57a0 20 read_log 192834'16797 (192834'16562) modify 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.1883634.0:37602 2014-08-11 15:28:39.561928 2014-08-13 18:06:06.770115 7f5c973a57a0 20 read_log 192834'16810 (192834'16588) modify 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.1883634.0:39080 2014-08-11 15:28:41.540857 2014-08-13 18:06:06.770127 7f5c973a57a0 20 read_log 192834'16816 (192834'16600) modify 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.1883634.0:40246 2014-08-11 15:28:43.423233 2014-08-13 18:06:06.770139 7f5c973a57a0 20 read_log 192834'16830 (192834'16628) modify 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.1883634.0:43092 2014-08-11 15:28:46.452820 2014-08-13 18:06:06.770162 7f5c973a57a0 20 read_log 192834'16835 (192834'16638) modify 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.1883634.0:44427 2014-08-11 15:28:46.855800 2014-08-13 18:06:06.770175 7f5c973a57a0 20 read_log 192834'16836 (192834'16640) modify f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.1883634.0:44449 2014-08-11 15:28:46.860759 2014-08-13 18:06:06.770188 7f5c973a57a0 20 read_log 192834'16843 (192834'16654) modify 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.1883634.0:44811 2014-08-11 15:28:46.966212 2014-08-13 18:06:06.770217 7f5c973a57a0 20 read_log 192834'16845 (192834'16658) modify 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.1883634.0:45258 2014-08-11 15:28:47.217296 2014-08-13 18:06:06.770231 7f5c973a57a0 20 read_log 192888'16860 (192888'16860) modify 1e6/hit_set_24.1e6_archive_2014-08-11 16:02:37.098356_2014-08-11 16:44:56.372952/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:44:56.373364 2014-08-13 18:06:06.770244 7f5c973a57a0 20 read_log 192888'16861 (0'0) promote c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by osd.285.0:17347 2014-08-11 16:44:56.377297 2014-08-13 18:06:06.770256 7f5c973a57a0 20 read_log 192888'16862 (192888'16861) modify c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.1315259.0:347 2014-08-11 16:44:56.366883 2014-08-13 18:06:06.770269 7f5c973a57a0 20 read_log 192888'16867 (0'0) promote a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by osd.285.0:17483 2014-08-11 16:45:15.246403 2014-08-13 18:06:06.770282 7f5c973a57a0 20 read_log 192888'16868 (192888'16867) modify a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.1315259.0:5299 2014-08-11 16:45:15.234639 2014-08-13 18:06:06.770294 7f5c973a57a0 20 read_log 192888'16871 (0'0) promote d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by osd.285.0:17517 2014-08-11 16:45:17.601460 2014-08-13 18:06:06.770306 7f5c973a57a0 20 read_log 192888'16872 (192888'16871) modify d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.1315259.0:5949 2014-08-11 16:45:17.593514 2014-08-13 18:06:06.770320 7f5c973a57a0 20 read_log 192888'16877 (0'0) promote bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by osd.285.0:17549 2014-08-11 16:45:23.967820 2014-08-13 18:06:06.770332 7f5c973a57a0 20 read_log 192888'16878 (192888'16877) modify bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.1315259.0:7713 2014-08-11 16:45:23.960694 2014-08-13 18:06:06.770344 7f5c973a57a0 20 read_log 192888'16879 (0'0) promote c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by osd.285.0:17571 2014-08-11 16:45:27.165831 2014-08-13 18:06:06.770356 7f5c973a57a0 20 read_log 192888'16880 (192888'16879) modify c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.1315259.0:8553 2014-08-11 16:45:27.158099 2014-08-13 18:06:06.770368 7f5c973a57a0 20 read_log 192888'16889 (0'0) promote 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by osd.285.0:17665 2014-08-11 16:45:41.386895 2014-08-13 18:06:06.770380 7f5c973a57a0 20 read_log 192888'16890 (192888'16889) modify 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.1315259.0:12179 2014-08-11 16:45:41.377529 2014-08-13 18:06:06.770411 7f5c973a57a0 20 read_log 192888'16893 (0'0) promote 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by osd.285.0:17693 2014-08-11 16:45:46.025159 2014-08-13 18:06:06.770423 7f5c973a57a0 20 read_log 192888'16894 (192888'16893) modify 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.1315259.0:13298 2014-08-11 16:45:46.015949 2014-08-13 18:06:06.770435 7f5c973a57a0 20 read_log 192888'16895 (0'0) promote 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by osd.285.0:17703 2014-08-11 16:45:47.442687 2014-08-13 18:06:06.770447 7f5c973a57a0 20 read_log 192888'16896 (192888'16895) modify 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.1315259.0:13644 2014-08-11 16:45:47.432657 2014-08-13 18:06:06.770460 7f5c973a57a0 20 read_log 192888'16897 (0'0) promote eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by osd.285.0:17705 2014-08-11 16:45:47.466839 2014-08-13 18:06:06.770479 7f5c973a57a0 20 read_log 192888'16898 (192888'16897) modify eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.1315259.0:13651 2014-08-11 16:45:47.458536 2014-08-13 18:06:06.770492 7f5c973a57a0 20 read_log 192888'16899 (0'0) promote 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by osd.285.0:17725 2014-08-11 16:45:50.599573 2014-08-13 18:06:06.770504 7f5c973a57a0 20 read_log 192888'16900 (192888'16899) modify 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.1315259.0:14474 2014-08-11 16:45:50.592230 2014-08-13 18:06:06.770517 7f5c973a57a0 20 read_log 192888'16901 (0'0) promote 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by osd.285.0:17737 2014-08-11 16:45:53.764623 2014-08-13 18:06:06.770529 7f5c973a57a0 20 read_log 192888'16902 (192888'16901) modify 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.1315259.0:15314 2014-08-11 16:45:53.756762 2014-08-13 18:06:06.770541 7f5c973a57a0 20 read_log 192888'16905 (0'0) promote 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by osd.285.0:17753 2014-08-11 16:45:56.759000 2014-08-13 18:06:06.770553 7f5c973a57a0 20 read_log 192888'16906 (192888'16905) modify 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.1315259.0:16087 2014-08-11 16:45:56.751058 2014-08-13 18:06:06.770565 7f5c973a57a0 20 read_log 192888'16907 (0'0) promote 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by osd.285.0:17755 2014-08-11 16:45:56.936952 2014-08-13 18:06:06.770577 7f5c973a57a0 20 read_log 192888'16908 (192888'16907) modify 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.1315259.0:16134 2014-08-11 16:45:56.927851 2014-08-13 18:06:06.770589 7f5c973a57a0 20 read_log 192888'16911 (0'0) promote a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by osd.285.0:17771 2014-08-11 16:45:59.124546 2014-08-13 18:06:06.770601 7f5c973a57a0 20 read_log 192888'16912 (192888'16911) modify a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.1315259.0:16688 2014-08-11 16:45:59.115774 2014-08-13 18:06:06.770613 7f5c973a57a0 20 read_log 192888'16913 (0'0) promote a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by osd.285.0:17801 2014-08-11 16:46:02.056317 2014-08-13 18:06:06.770625 7f5c973a57a0 20 read_log 192888'16914 (192888'16913) modify a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.1315259.0:17435 2014-08-11 16:46:02.049291 2014-08-13 18:06:06.770638 7f5c973a57a0 20 read_log 192888'16917 (0'0) promote d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by osd.285.0:17835 2014-08-11 16:46:06.682392 2014-08-13 18:06:06.770650 7f5c973a57a0 20 read_log 192888'16918 (192888'16917) modify d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.1315259.0:18605 2014-08-11 16:46:06.673751 2014-08-13 18:06:06.770662 7f5c973a57a0 20 read_log 192888'16919 (0'0) promote 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by osd.285.0:17837 2014-08-11 16:46:07.251479 2014-08-13 18:06:06.770674 7f5c973a57a0 20 read_log 192888'16920 (192888'16919) modify 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.1315259.0:18757 2014-08-11 16:46:07.242916 2014-08-13 18:06:06.770686 7f5c973a57a0 20 read_log 192888'16921 (0'0) promote 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by osd.285.0:17845 2014-08-11 16:46:08.551554 2014-08-13 18:06:06.770698 7f5c973a57a0 20 read_log 192888'16922 (192888'16921) modify 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.1315259.0:19065 2014-08-11 16:46:08.541925 2014-08-13 18:06:06.770710 7f5c973a57a0 20 read_log 192888'16923 (0'0) promote 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by osd.285.0:17881 2014-08-11 16:46:13.784874 2014-08-13 18:06:06.770728 7f5c973a57a0 20 read_log 192888'16924 (192888'16923) modify 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.1315259.0:20452 2014-08-11 16:46:13.774763 2014-08-13 18:06:06.770741 7f5c973a57a0 20 read_log 192888'16933 (0'0) promote f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by osd.285.0:17931 2014-08-11 16:46:19.451710 2014-08-13 18:06:06.770753 7f5c973a57a0 20 read_log 192888'16934 (192888'16933) modify f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.1315259.0:21944 2014-08-11 16:46:19.441152 2014-08-13 18:06:06.770765 7f5c973a57a0 20 read_log 192888'16935 (0'0) promote 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by osd.285.0:17941 2014-08-11 16:46:20.974145 2014-08-13 18:06:06.770777 7f5c973a57a0 20 read_log 192888'16936 (192888'16935) modify 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.1315259.0:22336 2014-08-11 16:46:20.965754 2014-08-13 18:06:06.770789 7f5c973a57a0 20 read_log 192888'16937 (0'0) promote 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by osd.285.0:17943 2014-08-11 16:46:21.131233 2014-08-13 18:06:06.770801 7f5c973a57a0 20 read_log 192888'16938 (192888'16937) modify 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.1315259.0:22373 2014-08-11 16:46:21.121968 2014-08-13 18:06:06.770813 7f5c973a57a0 20 read_log 192888'16941 (0'0) promote 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by osd.285.0:17973 2014-08-11 16:46:24.556338 2014-08-13 18:06:06.770825 7f5c973a57a0 20 read_log 192888'16942 (192888'16941) modify 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.1315259.0:23223 2014-08-11 16:46:24.548392 2014-08-13 18:06:06.770837 7f5c973a57a0 20 read_log 192888'16945 (0'0) promote efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by osd.285.0:18001 2014-08-11 16:46:30.543883 2014-08-13 18:06:06.770849 7f5c973a57a0 20 read_log 192888'16946 (192888'16945) modify efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.1315259.0:24740 2014-08-11 16:46:30.536690 2014-08-13 18:06:06.770861 7f5c973a57a0 20 read_log 192888'16947 (0'0) promote ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by osd.285.0:18011 2014-08-11 16:46:31.433916 2014-08-13 18:06:06.770873 7f5c973a57a0 20 read_log 192888'16948 (192888'16947) modify ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.1315259.0:24978 2014-08-11 16:46:31.423850 2014-08-13 18:06:06.770885 7f5c973a57a0 20 read_log 192888'16951 (0'0) promote ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by osd.285.0:18063 2014-08-11 16:46:40.907289 2014-08-13 18:06:06.770896 7f5c973a57a0 20 read_log 192888'16952 (192888'16951) modify ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.1315259.0:27215 2014-08-11 16:46:40.899033 2014-08-13 18:06:06.770909 7f5c973a57a0 20 read_log 192888'16953 (0'0) promote 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by osd.285.0:18065 2014-08-11 16:46:40.966866 2014-08-13 18:06:06.770920 7f5c973a57a0 20 read_log 192888'16954 (192888'16953) modify 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.1315259.0:27231 2014-08-11 16:46:40.960169 2014-08-13 18:06:06.770932 7f5c973a57a0 20 read_log 192888'16955 (0'0) promote f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by osd.285.0:18085 2014-08-11 16:46:44.401533 2014-08-13 18:06:06.770944 7f5c973a57a0 20 read_log 192888'16956 (192888'16955) modify f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.1315259.0:28112 2014-08-11 16:46:44.390280 2014-08-13 18:06:06.770956 7f5c973a57a0 20 read_log 192888'16957 (0'0) promote 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by osd.285.0:18111 2014-08-11 16:46:47.211898 2014-08-13 18:06:06.770979 7f5c973a57a0 20 read_log 192888'16958 (192888'16957) modify 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.1315259.0:28787 2014-08-11 16:46:47.203907 2014-08-13 18:06:06.770992 7f5c973a57a0 20 read_log 192888'16961 (0'0) promote 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by osd.285.0:18119 2014-08-11 16:46:48.337251 2014-08-13 18:06:06.771005 7f5c973a57a0 20 read_log 192888'16962 (192888'16961) modify 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.1315259.0:29083 2014-08-11 16:46:48.329620 2014-08-13 18:06:06.771017 7f5c973a57a0 20 read_log 192888'16967 (192888'16862) modify c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.1315259.0:31074 2014-08-11 16:46:55.989683 2014-08-13 18:06:06.771029 7f5c973a57a0 20 read_log 192888'16970 (192888'16868) modify a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.1315259.0:36026 2014-08-11 16:46:57.306102 2014-08-13 18:06:06.771041 7f5c973a57a0 20 read_log 192888'16972 (192888'16872) modify d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.1315259.0:36676 2014-08-11 16:46:57.483766 2014-08-13 18:06:06.771053 7f5c973a57a0 20 read_log 192888'16975 (192888'16878) modify bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.1315259.0:38440 2014-08-11 16:46:57.947564 2014-08-13 18:06:06.771065 7f5c973a57a0 20 read_log 192888'16976 (192888'16880) modify c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.1315259.0:39280 2014-08-11 16:46:58.170762 2014-08-13 18:06:06.771078 7f5c973a57a0 20 read_log 192888'16981 (192888'16890) modify 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.1315259.0:42906 2014-08-11 16:46:59.137963 2014-08-13 18:06:06.771090 7f5c973a57a0 20 read_log 192888'16983 (192888'16894) modify 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.1315259.0:44025 2014-08-11 16:46:59.458355 2014-08-13 18:06:06.771102 7f5c973a57a0 20 read_log 192888'16984 (192888'16896) modify 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.1315259.0:44371 2014-08-11 16:46:59.559320 2014-08-13 18:06:06.771115 7f5c973a57a0 20 read_log 192888'16985 (192888'16898) modify eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.1315259.0:44378 2014-08-11 16:46:59.563053 2014-08-13 18:06:06.771127 7f5c973a57a0 20 read_log 192888'16986 (192888'16900) modify 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.1315259.0:45201 2014-08-11 16:46:59.798363 2014-08-13 18:06:06.771139 7f5c973a57a0 20 read_log 192888'16987 (192888'16902) modify 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.1315259.0:46041 2014-08-11 16:47:00.043752 2014-08-13 18:06:06.771175 7f5c973a57a0 20 read_log 192888'16989 (192888'16906) modify 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.1315259.0:46814 2014-08-11 16:47:00.266498 2014-08-13 18:06:06.771189 7f5c973a57a0 20 read_log 192888'16990 (192888'16908) modify 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.1315259.0:46861 2014-08-11 16:47:00.278778 2014-08-13 18:06:06.771201 7f5c973a57a0 20 read_log 192888'16992 (192888'16912) modify a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.1315259.0:47415 2014-08-11 16:47:00.448947 2014-08-13 18:06:06.771213 7f5c973a57a0 20 read_log 192888'16993 (192888'16914) modify a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.1315259.0:48162 2014-08-11 16:47:00.686068 2014-08-13 18:06:06.771225 7f5c973a57a0 20 read_log 192888'16995 (192888'16918) modify d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.1315259.0:49332 2014-08-11 16:47:01.029137 2014-08-13 18:06:06.771237 7f5c973a57a0 20 read_log 192888'16996 (192888'16920) modify 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.1315259.0:49484 2014-08-11 16:47:01.077579 2014-08-13 18:06:06.771257 7f5c973a57a0 20 read_log 192888'16997 (192888'16922) modify 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.1315259.0:49792 2014-08-11 16:47:01.174936 2014-08-13 18:06:06.771273 7f5c973a57a0 20 read_log 192888'16998 (192888'16924) modify 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.1315259.0:51179 2014-08-11 16:47:01.602397 2014-08-13 18:06:06.771287 7f5c973a57a0 20 read_log 192888'17003 (192888'16934) modify f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.1315259.0:52671 2014-08-11 16:47:02.037196 2014-08-13 18:06:06.771299 7f5c973a57a0 20 read_log 192888'17004 (192888'16936) modify 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.1315259.0:53063 2014-08-11 16:47:02.144044 2014-08-13 18:06:06.771312 7f5c973a57a0 20 read_log 192888'17005 (192888'16938) modify 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.1315259.0:53100 2014-08-11 16:47:02.155663 2014-08-13 18:06:06.771328 7f5c973a57a0 20 read_log 192888'17007 (192888'16942) modify 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.1315259.0:53950 2014-08-11 16:47:02.412788 2014-08-13 18:06:06.771340 7f5c973a57a0 20 read_log 192888'17009 (192888'16946) modify efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.1315259.0:55467 2014-08-11 16:47:02.920864 2014-08-13 18:06:06.771353 7f5c973a57a0 20 read_log 192888'17010 (192888'16948) modify ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.1315259.0:55705 2014-08-11 16:47:02.988952 2014-08-13 18:06:06.771365 7f5c973a57a0 20 read_log 192888'17012 (192888'16952) modify ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.1315259.0:57942 2014-08-11 16:47:03.657097 2014-08-13 18:06:06.771377 7f5c973a57a0 20 read_log 192888'17013 (192888'16954) modify 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.1315259.0:57958 2014-08-11 16:47:03.661641 2014-08-13 18:06:06.771390 7f5c973a57a0 20 read_log 192888'17014 (192888'16956) modify f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.1315259.0:58839 2014-08-11 16:47:03.923128 2014-08-13 18:06:06.771412 7f5c973a57a0 20 read_log 192888'17015 (192888'16958) modify 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.1315259.0:59514 2014-08-11 16:47:04.142581 2014-08-13 18:06:06.771426 7f5c973a57a0 20 read_log 192888'17017 (192888'16962) modify 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.1315259.0:59810 2014-08-11 16:47:04.247670 2014-08-13 18:06:06.771439 7f5c973a57a0 20 read_log 192888'17020 (192888'17020) modify 1e6/hit_set_24.1e6_archive_2014-08-11 16:44:56.373369_2014-08-11 17:05:20.065317/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:20.065812 2014-08-13 18:06:06.771464 7f5c973a57a0 20 read_log 192888'17021 (192888'16860) delete 1e6/hit_set_24.1e6_archive_2014-08-11 16:02:37.098356_2014-08-11 16:44:56.372952/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:20.065812 2014-08-13 18:06:06.771478 7f5c973a57a0 20 read_log 192888'17026 (0'0) promote 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by osd.285.0:18666 2014-08-11 17:05:28.309286 2014-08-13 18:06:06.771491 7f5c973a57a0 20 read_log 192888'17027 (192888'17026) modify 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.942868.0:3342 2014-08-11 17:05:28.299783 2014-08-13 18:06:06.771511 7f5c973a57a0 20 read_log 192888'17030 (0'0) promote 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by osd.285.0:18710 2014-08-11 17:05:34.674115 2014-08-13 18:06:06.771528 7f5c973a57a0 20 read_log 192888'17031 (192888'17030) modify 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.942868.0:5012 2014-08-11 17:05:34.666630 2014-08-13 18:06:06.771577 7f5c973a57a0 20 read_log 192888'17032 (0'0) promote c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by osd.285.0:18754 2014-08-11 17:05:40.829286 2014-08-13 18:06:06.771591 7f5c973a57a0 20 read_log 192888'17033 (192888'17032) modify c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.942868.0:6697 2014-08-11 17:05:40.821428 2014-08-13 18:06:06.771618 7f5c973a57a0 20 read_log 192888'17036 (0'0) promote 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by osd.285.0:18770 2014-08-11 17:05:42.845006 2014-08-13 18:06:06.771631 7f5c973a57a0 20 read_log 192888'17037 (192888'17036) modify 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.942868.0:7221 2014-08-11 17:05:42.837205 2014-08-13 18:06:06.771644 7f5c973a57a0 20 read_log 192888'17048 (0'0) promote e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by osd.285.0:18832 2014-08-11 17:05:51.139002 2014-08-13 18:06:06.771663 7f5c973a57a0 20 read_log 192888'17049 (192888'17048) modify e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.942868.0:9429 2014-08-11 17:05:51.131053 2014-08-13 18:06:06.771677 7f5c973a57a0 20 read_log 192888'17050 (0'0) promote ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by osd.285.0:18840 2014-08-11 17:05:52.017587 2014-08-13 18:06:06.771690 7f5c973a57a0 20 read_log 192888'17051 (192888'17050) modify ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.942868.0:9672 2014-08-11 17:05:52.010440 2014-08-13 18:06:06.771704 7f5c973a57a0 20 read_log 192888'17052 (0'0) promote 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by osd.285.0:18876 2014-08-11 17:05:57.055940 2014-08-13 18:06:06.771718 7f5c973a57a0 20 read_log 192888'17053 (192888'17052) modify 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.942868.0:11072 2014-08-11 17:05:57.048164 2014-08-13 18:06:06.771730 7f5c973a57a0 20 read_log 192888'17060 (0'0) promote e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by osd.285.0:18940 2014-08-11 17:06:03.410233 2014-08-13 18:06:06.771743 7f5c973a57a0 20 read_log 192888'17061 (192888'17060) modify e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.942868.0:12785 2014-08-11 17:06:03.401712 2014-08-13 18:06:06.771761 7f5c973a57a0 20 read_log 192888'17062 (0'0) promote e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by osd.285.0:18942 2014-08-11 17:06:03.511343 2014-08-13 18:06:06.771775 7f5c973a57a0 20 read_log 192888'17063 (192888'17062) modify e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.942868.0:12813 2014-08-11 17:06:03.502776 2014-08-13 18:06:06.771794 7f5c973a57a0 20 read_log 192888'17066 (0'0) promote 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by osd.285.0:18970 2014-08-11 17:06:07.419818 2014-08-13 18:06:06.771807 7f5c973a57a0 20 read_log 192888'17067 (192888'17066) modify 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.942868.0:13814 2014-08-11 17:06:07.413217 2014-08-13 18:06:06.771828 7f5c973a57a0 20 read_log 192888'17068 (0'0) promote fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by osd.285.0:18982 2014-08-11 17:06:09.285037 2014-08-13 18:06:06.771841 7f5c973a57a0 20 read_log 192888'17069 (192888'17068) modify fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.942868.0:14260 2014-08-11 17:06:09.277470 2014-08-13 18:06:06.771854 7f5c973a57a0 20 read_log 192888'17072 (0'0) promote 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by osd.285.0:19050 2014-08-11 17:06:17.130461 2014-08-13 18:06:06.771870 7f5c973a57a0 20 read_log 192888'17073 (192888'17072) modify 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.942868.0:16267 2014-08-11 17:06:17.121618 2014-08-13 18:06:06.771890 7f5c973a57a0 20 read_log 192888'17074 (0'0) promote 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by osd.285.0:19052 2014-08-11 17:06:17.717660 2014-08-13 18:06:06.771911 7f5c973a57a0 20 read_log 192888'17075 (192888'17074) modify 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.942868.0:16418 2014-08-11 17:06:17.709359 2014-08-13 18:06:06.771924 7f5c973a57a0 20 read_log 192888'17078 (0'0) promote 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by osd.285.0:19070 2014-08-11 17:06:21.027538 2014-08-13 18:06:06.771937 7f5c973a57a0 20 read_log 192888'17079 (192888'17078) modify 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.942868.0:17255 2014-08-11 17:06:21.020081 2014-08-13 18:06:06.771949 7f5c973a57a0 20 read_log 192888'17080 (0'0) promote 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by osd.285.0:19086 2014-08-11 17:06:24.642826 2014-08-13 18:06:06.771967 7f5c973a57a0 20 read_log 192888'17081 (192888'17080) modify 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.942868.0:18213 2014-08-11 17:06:24.635209 2014-08-13 18:06:06.771980 7f5c973a57a0 20 read_log 192888'17082 (0'0) promote d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by osd.285.0:19100 2014-08-11 17:06:26.584360 2014-08-13 18:06:06.771997 7f5c973a57a0 20 read_log 192888'17083 (192888'17082) modify d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.942868.0:18729 2014-08-11 17:06:26.573925 2014-08-13 18:06:06.772010 7f5c973a57a0 20 read_log 192888'17086 (0'0) promote e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by osd.285.0:19134 2014-08-11 17:06:30.564960 2014-08-13 18:06:06.772023 7f5c973a57a0 20 read_log 192888'17087 (192888'17086) modify e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.942868.0:19788 2014-08-11 17:06:30.557573 2014-08-13 18:06:06.772038 7f5c973a57a0 20 read_log 192888'17090 (0'0) promote 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by osd.285.0:19154 2014-08-11 17:06:32.280847 2014-08-13 18:06:06.772051 7f5c973a57a0 20 read_log 192888'17091 (192888'17090) modify 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.942868.0:20246 2014-08-11 17:06:32.273552 2014-08-13 18:06:06.772069 7f5c973a57a0 20 read_log 192888'17094 (0'0) promote 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by osd.285.0:19184 2014-08-11 17:06:35.997964 2014-08-13 18:06:06.772085 7f5c973a57a0 20 read_log 192888'17095 (192888'17094) modify 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.942868.0:21200 2014-08-11 17:06:35.989775 2014-08-13 18:06:06.772098 7f5c973a57a0 20 read_log 192888'17096 (0'0) promote e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by osd.285.0:19200 2014-08-11 17:06:38.600024 2014-08-13 18:06:06.772112 7f5c973a57a0 20 read_log 192888'17097 (192888'17096) modify e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.942868.0:21898 2014-08-11 17:06:38.592416 2014-08-13 18:06:06.772125 7f5c973a57a0 20 read_log 192888'17100 (0'0) promote a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by osd.285.0:19208 2014-08-11 17:06:40.429124 2014-08-13 18:06:06.772140 7f5c973a57a0 20 read_log 192888'17101 (192888'17100) modify a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.942868.0:22375 2014-08-11 17:06:40.419533 2014-08-13 18:06:06.772167 7f5c973a57a0 20 read_log 192888'17102 (0'0) promote 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by osd.285.0:19210 2014-08-11 17:06:40.854576 2014-08-13 18:06:06.772184 7f5c973a57a0 20 read_log 192888'17103 (192888'17102) modify 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.942868.0:22470 2014-08-11 17:06:40.845626 2014-08-13 18:06:06.772204 7f5c973a57a0 20 read_log 192888'17104 (0'0) promote a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by osd.285.0:19216 2014-08-11 17:06:41.147793 2014-08-13 18:06:06.772217 7f5c973a57a0 20 read_log 192888'17105 (192888'17104) modify a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.942868.0:22537 2014-08-11 17:06:41.140438 2014-08-13 18:06:06.772233 7f5c973a57a0 20 read_log 192888'17108 (0'0) promote 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by osd.285.0:19232 2014-08-11 17:06:42.206695 2014-08-13 18:06:06.772245 7f5c973a57a0 20 read_log 192888'17109 (192888'17108) modify 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.942868.0:22823 2014-08-11 17:06:42.196158 2014-08-13 18:06:06.772258 7f5c973a57a0 20 read_log 192888'17112 (0'0) promote 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by osd.285.0:19260 2014-08-11 17:06:46.715883 2014-08-13 18:06:06.772270 7f5c973a57a0 20 read_log 192888'17113 (192888'17112) modify 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.942868.0:24001 2014-08-11 17:06:46.708763 2014-08-13 18:06:06.772285 7f5c973a57a0 20 read_log 192888'17114 (0'0) promote 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by osd.285.0:19262 2014-08-11 17:06:47.203568 2014-08-13 18:06:06.772298 7f5c973a57a0 20 read_log 192888'17115 (192888'17114) modify 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.942868.0:24132 2014-08-11 17:06:47.194942 2014-08-13 18:06:06.772311 7f5c973a57a0 20 read_log 192888'17120 (0'0) promote 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by osd.285.0:19296 2014-08-11 17:06:51.320297 2014-08-13 18:06:06.772323 7f5c973a57a0 20 read_log 192888'17121 (192888'17120) modify 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.942868.0:25244 2014-08-11 17:06:51.313116 2014-08-13 18:06:06.772339 7f5c973a57a0 20 read_log 192888'17122 (0'0) promote e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by osd.285.0:19300 2014-08-11 17:06:51.407257 2014-08-13 18:06:06.772351 7f5c973a57a0 20 read_log 192888'17123 (192888'17122) modify e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.942868.0:25267 2014-08-11 17:06:51.398396 2014-08-13 18:06:06.772364 7f5c973a57a0 20 read_log 192888'17126 (0'0) promote fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by osd.285.0:19324 2014-08-11 17:06:54.545625 2014-08-13 18:06:06.772379 7f5c973a57a0 20 read_log 192888'17127 (192888'17126) modify fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.942868.0:26037 2014-08-11 17:06:54.536278 2014-08-13 18:06:06.772392 7f5c973a57a0 20 read_log 192888'17136 (0'0) promote 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by osd.285.0:19388 2014-08-11 17:07:01.169182 2014-08-13 18:06:06.772404 7f5c973a57a0 20 read_log 192888'17137 (192888'17136) modify 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.942868.0:27775 2014-08-11 17:07:01.159166 2014-08-13 18:06:06.772416 7f5c973a57a0 20 read_log 192888'17142 (0'0) promote 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by osd.285.0:19440 2014-08-11 17:07:08.092026 2014-08-13 18:06:06.772431 7f5c973a57a0 20 read_log 192888'17143 (192888'17142) modify 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.942868.0:29568 2014-08-11 17:07:08.085757 2014-08-13 18:06:06.772444 7f5c973a57a0 20 read_log 192888'17144 (0'0) promote 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by osd.285.0:19486 2014-08-11 17:07:13.182233 2014-08-13 18:06:06.772456 7f5c973a57a0 20 read_log 192888'17145 (192888'17144) modify 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.942868.0:30839 2014-08-11 17:07:13.173476 2014-08-13 18:06:06.772483 7f5c973a57a0 20 read_log 192888'17152 (192888'17027) modify 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.942868.0:34608 2014-08-11 17:07:16.841879 2014-08-13 18:06:06.772497 7f5c973a57a0 20 read_log 192888'17154 (192888'17031) modify 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.942868.0:36278 2014-08-11 17:07:17.286774 2014-08-13 18:06:06.772510 7f5c973a57a0 20 read_log 192888'17155 (192888'17033) modify c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.942868.0:37963 2014-08-11 17:07:17.755263 2014-08-13 18:06:06.772537 7f5c973a57a0 20 read_log 192888'17157 (192888'17037) modify 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.942868.0:38487 2014-08-11 17:07:17.900344 2014-08-13 18:06:06.772552 7f5c973a57a0 20 read_log 192888'17163 (192888'17049) modify e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.942868.0:40695 2014-08-11 17:07:18.522764 2014-08-13 18:06:06.772565 7f5c973a57a0 20 read_log 192888'17164 (192888'17051) modify ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.942868.0:40938 2014-08-11 17:07:18.587213 2014-08-13 18:06:06.772580 7f5c973a57a0 20 read_log 192888'17165 (192888'17053) modify 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.942868.0:42338 2014-08-11 17:07:18.972198 2014-08-13 18:06:06.772593 7f5c973a57a0 20 read_log 192888'17169 (192888'17061) modify e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.942868.0:44051 2014-08-11 17:07:19.452412 2014-08-13 18:06:06.772626 7f5c973a57a0 20 read_log 192888'17170 (192888'17063) modify e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.942868.0:44079 2014-08-11 17:07:19.461156 2014-08-13 18:06:06.772640 7f5c973a57a0 20 read_log 192888'17172 (192888'17067) modify 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.942868.0:45080 2014-08-11 17:07:19.741398 2014-08-13 18:06:06.772652 7f5c973a57a0 20 read_log 192888'17173 (192888'17069) modify fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.942868.0:45526 2014-08-11 17:07:19.862233 2014-08-13 18:06:06.772665 7f5c973a57a0 20 read_log 192888'17175 (192888'17073) modify 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.942868.0:47533 2014-08-11 17:07:20.436686 2014-08-13 18:06:06.772680 7f5c973a57a0 20 read_log 192888'17176 (192888'17075) modify 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.942868.0:47684 2014-08-11 17:07:20.486431 2014-08-13 18:06:06.772738 7f5c973a57a0 20 read_log 192888'17178 (192888'17079) modify 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.942868.0:48521 2014-08-11 17:07:20.743766 2014-08-13 18:06:06.772751 7f5c973a57a0 20 read_log 192888'17179 (192888'17081) modify 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.942868.0:49479 2014-08-11 17:07:21.025674 2014-08-13 18:06:06.772764 7f5c973a57a0 20 read_log 192888'17180 (192888'17083) modify d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.942868.0:49995 2014-08-11 17:07:21.184424 2014-08-13 18:06:06.772780 7f5c973a57a0 20 read_log 192888'17182 (192888'17087) modify e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.942868.0:51054 2014-08-11 17:07:21.517090 2014-08-13 18:06:06.772792 7f5c973a57a0 20 read_log 192888'17184 (192888'17091) modify 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.942868.0:51512 2014-08-11 17:07:21.669555 2014-08-13 18:06:06.772805 7f5c973a57a0 20 read_log 192888'17186 (192888'17095) modify 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.942868.0:52466 2014-08-11 17:07:21.950076 2014-08-13 18:06:06.772820 7f5c973a57a0 20 read_log 192888'17187 (192888'17097) modify e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.942868.0:53164 2014-08-11 17:07:22.170976 2014-08-13 18:06:06.772841 7f5c973a57a0 20 read_log 192888'17189 (192888'17101) modify a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.942868.0:53641 2014-08-11 17:07:22.329396 2014-08-13 18:06:06.772854 7f5c973a57a0 20 read_log 192888'17190 (192888'17103) modify 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.942868.0:53736 2014-08-11 17:07:22.359201 2014-08-13 18:06:06.772870 7f5c973a57a0 20 read_log 192888'17191 (192888'17105) modify a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.942868.0:53803 2014-08-11 17:07:22.378460 2014-08-13 18:06:06.772883 7f5c973a57a0 20 read_log 192888'17193 (192888'17109) modify 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.942868.0:54089 2014-08-11 17:07:22.460614 2014-08-13 18:06:06.772895 7f5c973a57a0 20 read_log 192888'17195 (192888'17113) modify 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.942868.0:55267 2014-08-11 17:07:22.794958 2014-08-13 18:06:06.772907 7f5c973a57a0 20 read_log 192888'17196 (192888'17115) modify 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.942868.0:55398 2014-08-11 17:07:22.834641 2014-08-13 18:06:06.772923 7f5c973a57a0 20 read_log 192888'17199 (192888'17121) modify 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.942868.0:56510 2014-08-11 17:07:23.174812 2014-08-13 18:06:06.772935 7f5c973a57a0 20 read_log 192888'17200 (192888'17123) modify e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.942868.0:56533 2014-08-11 17:07:23.182095 2014-08-13 18:06:06.772948 7f5c973a57a0 20 read_log 192888'17202 (192888'17127) modify fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.942868.0:57303 2014-08-11 17:07:23.417442 2014-08-13 18:06:06.772960 7f5c973a57a0 20 read_log 192888'17207 (192888'17137) modify 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.942868.0:59041 2014-08-11 17:07:23.926655 2014-08-13 18:06:06.772976 7f5c973a57a0 20 read_log 192888'17210 (192888'17143) modify 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.942868.0:60834 2014-08-11 17:07:24.459271 2014-08-13 18:06:06.772988 7f5c973a57a0 20 read_log 192888'17211 (192888'17145) modify 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.942868.0:62105 2014-08-11 17:07:24.850186 2014-08-13 18:06:06.773001 7f5c973a57a0 20 read_log 193305'17214 (193305'17214) modify 1e6/hit_set_24.1e6_archive_2014-08-11 18:37:04.946502_2014-08-11 18:42:36.225526/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.225780 2014-08-13 18:06:06.773016 7f5c973a57a0 20 read_log 193305'17215 (192888'17020) delete 1e6/hit_set_24.1e6_archive_2014-08-11 16:44:56.373369_2014-08-11 17:05:20.065317/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:36.225780 2014-08-13 18:06:06.773028 7f5c973a57a0 20 read_log 193305'17216 (0'0) promote 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by osd.285.0:18 2014-08-11 18:42:36.245508 2014-08-13 18:06:06.773041 7f5c973a57a0 20 read_log 193305'17217 (193305'17216) modify 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.951337.0:793 2014-08-11 18:42:36.219952 2014-08-13 18:06:06.773053 7f5c973a57a0 20 read_log 193305'17218 (0'0) promote 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by osd.285.0:20 2014-08-11 18:42:36.687986 2014-08-13 18:06:06.773150 7f5c973a57a0 20 read_log 193305'17219 (193305'17218) modify 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.951337.0:921 2014-08-11 18:42:36.680667 2014-08-13 18:06:06.773175 7f5c973a57a0 20 read_log 193305'17220 (0'0) promote c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by osd.285.0:38 2014-08-11 18:42:38.659106 2014-08-13 18:06:06.773209 7f5c973a57a0 20 read_log 193305'17221 (193305'17220) modify c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.951337.0:1443 2014-08-11 18:42:38.651305 2014-08-13 18:06:06.773222 7f5c973a57a0 20 read_log 193305'17222 (0'0) promote 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by osd.285.0:55 2014-08-11 18:42:43.267553 2014-08-13 18:06:06.773235 7f5c973a57a0 20 read_log 193305'17223 (193305'17222) modify 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.951337.0:2691 2014-08-11 18:42:43.260413 2014-08-13 18:06:06.773247 7f5c973a57a0 20 read_log 193305'17224 (0'0) promote b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by osd.285.0:97 2014-08-11 18:42:51.388150 2014-08-13 18:06:06.773263 7f5c973a57a0 20 read_log 193305'17225 (193305'17224) modify b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.951337.0:4906 2014-08-11 18:42:51.380904 2014-08-13 18:06:06.773285 7f5c973a57a0 20 read_log 193305'17226 (0'0) promote 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by osd.285.0:195 2014-08-11 18:43:07.428532 2014-08-13 18:06:06.773298 7f5c973a57a0 20 read_log 193305'17227 (193305'17226) modify 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.951337.0:9220 2014-08-11 18:43:07.421347 2014-08-13 18:06:06.773325 7f5c973a57a0 20 read_log 193305'17228 (0'0) promote e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by osd.285.0:235 2014-08-11 18:43:14.746897 2014-08-13 18:06:06.773338 7f5c973a57a0 20 read_log 193305'17229 (193305'17228) modify e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.951337.0:11177 2014-08-11 18:43:14.738344 2014-08-13 18:06:06.773382 7f5c973a57a0 20 read_log 193305'17230 (0'0) promote 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by osd.285.0:241 2014-08-11 18:43:16.682911 2014-08-13 18:06:06.773417 7f5c973a57a0 20 read_log 193305'17231 (193305'17230) modify 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.951337.0:11672 2014-08-11 18:43:16.675212 2014-08-13 18:06:06.773441 7f5c973a57a0 20 read_log 193305'17232 (0'0) promote bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by osd.285.0:297 2014-08-11 18:43:27.721113 2014-08-13 18:06:06.773463 7f5c973a57a0 20 read_log 193305'17233 (193305'17232) modify bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.951337.0:14636 2014-08-11 18:43:27.714529 2014-08-13 18:06:06.773479 7f5c973a57a0 20 read_log 193305'17234 (0'0) promote dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by osd.285.0:349 2014-08-11 18:43:35.622704 2014-08-13 18:06:06.773495 7f5c973a57a0 20 read_log 193305'17235 (193305'17234) modify dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.951337.0:16911 2014-08-11 18:43:35.614950 2014-08-13 18:06:06.773510 7f5c973a57a0 20 read_log 193305'17236 (0'0) promote 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by osd.285.0:369 2014-08-11 18:43:38.051525 2014-08-13 18:06:06.773523 7f5c973a57a0 20 read_log 193305'17237 (193305'17236) modify 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.951337.0:17627 2014-08-11 18:43:38.042598 2014-08-13 18:06:06.773535 7f5c973a57a0 20 read_log 193305'17238 (0'0) promote d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by osd.285.0:405 2014-08-11 18:43:42.926337 2014-08-13 18:06:06.773548 7f5c973a57a0 20 read_log 193305'17239 (193305'17238) modify d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.951337.0:18995 2014-08-11 18:43:42.918365 2014-08-13 18:06:06.773560 7f5c973a57a0 20 read_log 193305'17240 (0'0) promote 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by osd.285.0:455 2014-08-11 18:43:49.845211 2014-08-13 18:06:06.773584 7f5c973a57a0 20 read_log 193305'17241 (193305'17240) modify 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.951337.0:20901 2014-08-11 18:43:49.836904 2014-08-13 18:06:06.773599 7f5c973a57a0 20 read_log 193305'17242 (0'0) promote 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by osd.285.0:467 2014-08-11 18:43:51.375171 2014-08-13 18:06:06.773612 7f5c973a57a0 20 read_log 193305'17243 (193305'17242) modify 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.951337.0:21327 2014-08-11 18:43:51.367334 2014-08-13 18:06:06.773624 7f5c973a57a0 20 read_log 193305'17244 (0'0) promote e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by osd.285.0:527 2014-08-11 18:44:00.932045 2014-08-13 18:06:06.773636 7f5c973a57a0 20 read_log 193305'17245 (193305'17244) modify e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.951337.0:23975 2014-08-11 18:44:00.924981 2014-08-13 18:06:06.773649 7f5c973a57a0 20 read_log 193305'17246 (0'0) promote 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by osd.285.0:587 2014-08-11 18:44:12.355565 2014-08-13 18:06:06.773661 7f5c973a57a0 20 read_log 193305'17247 (193305'17246) modify 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.951337.0:27078 2014-08-11 18:44:12.346972 2014-08-13 18:06:06.773673 7f5c973a57a0 20 read_log 193305'17248 (0'0) promote b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by osd.285.0:648 2014-08-11 18:44:22.174244 2014-08-13 18:06:06.773685 7f5c973a57a0 20 read_log 193305'17249 (193305'17248) modify b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.951337.0:29774 2014-08-11 18:44:22.165460 2014-08-13 18:06:06.773698 7f5c973a57a0 20 read_log 193305'17250 (0'0) promote b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by osd.285.0:682 2014-08-11 18:44:28.228903 2014-08-13 18:06:06.773710 7f5c973a57a0 20 read_log 193305'17251 (193305'17250) modify b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.951337.0:31398 2014-08-11 18:44:28.216762 2014-08-13 18:06:06.773722 7f5c973a57a0 20 read_log 193305'17252 (0'0) promote ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by osd.285.0:698 2014-08-11 18:44:29.835774 2014-08-13 18:06:06.773734 7f5c973a57a0 20 read_log 193305'17253 (193305'17252) modify ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.951337.0:31871 2014-08-11 18:44:29.827640 2014-08-13 18:06:06.773747 7f5c973a57a0 20 read_log 193305'17254 (193305'17217) modify 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.951337.0:33504 2014-08-11 18:44:34.459297 2014-08-13 18:06:06.773759 7f5c973a57a0 20 read_log 193305'17255 (193305'17219) modify 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.951337.0:33632 2014-08-11 18:44:34.544580 2014-08-13 18:06:06.773771 7f5c973a57a0 20 read_log 193305'17256 (193305'17221) modify c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.951337.0:34154 2014-08-11 18:44:34.823522 2014-08-13 18:06:06.773783 7f5c973a57a0 20 read_log 193305'17257 (193305'17223) modify 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.951337.0:35402 2014-08-11 18:44:35.551024 2014-08-13 18:06:06.773808 7f5c973a57a0 20 read_log 193305'17258 (193305'17225) modify b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.951337.0:37617 2014-08-11 18:44:36.993950 2014-08-13 18:06:06.773821 7f5c973a57a0 20 read_log 193305'17259 (193305'17227) modify 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.951337.0:41931 2014-08-11 18:44:38.930732 2014-08-13 18:06:06.773834 7f5c973a57a0 20 read_log 193305'17260 (193305'17229) modify e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.951337.0:43888 2014-08-11 18:44:39.727959 2014-08-13 18:06:06.773879 7f5c973a57a0 20 read_log 193305'17261 (193305'17231) modify 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.951337.0:44383 2014-08-11 18:44:39.867808 2014-08-13 18:06:06.773896 7f5c973a57a0 20 read_log 193305'17262 (193305'17233) modify bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.951337.0:47347 2014-08-11 18:44:40.741735 2014-08-13 18:06:06.773909 7f5c973a57a0 20 read_log 193305'17263 (193305'17235) modify dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.951337.0:49622 2014-08-11 18:44:41.422275 2014-08-13 18:06:06.773921 7f5c973a57a0 20 read_log 193305'17264 (193305'17237) modify 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.951337.0:50338 2014-08-11 18:44:41.638354 2014-08-13 18:06:06.773936 7f5c973a57a0 20 read_log 193305'17265 (193305'17239) modify d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.951337.0:51706 2014-08-11 18:44:42.027698 2014-08-13 18:06:06.773949 7f5c973a57a0 20 read_log 193305'17266 (193305'17241) modify 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.951337.0:53612 2014-08-11 18:44:42.582040 2014-08-13 18:06:06.773962 7f5c973a57a0 20 read_log 193305'17267 (193305'17243) modify 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.951337.0:54038 2014-08-11 18:44:42.711547 2014-08-13 18:06:06.773974 7f5c973a57a0 20 read_log 193305'17268 (193305'17245) modify e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.951337.0:56686 2014-08-11 18:44:43.497517 2014-08-13 18:06:06.773986 7f5c973a57a0 20 read_log 193305'17269 (193305'17247) modify 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.951337.0:59789 2014-08-11 18:44:44.427621 2014-08-13 18:06:06.774004 7f5c973a57a0 20 read_log 193305'17270 (193305'17249) modify b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.951337.0:62485 2014-08-11 18:44:45.241112 2014-08-13 18:06:06.774017 7f5c973a57a0 20 read_log 193305'17271 (193305'17251) modify b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.951337.0:64109 2014-08-11 18:44:45.706482 2014-08-13 18:06:06.774029 7f5c973a57a0 20 read_log 193305'17272 (193305'17253) modify ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.951337.0:64582 2014-08-11 18:44:45.858719 2014-08-13 18:06:06.774045 7f5c973a57a0 20 read_log 194215'17273 (194215'17273) modify 1e6/hit_set_24.1e6_archive_2014-08-12 15:32:25.782585_2014-08-12 15:43:44.587553/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:44.587824 2014-08-13 18:06:06.774060 7f5c973a57a0 20 read_log 194215'17274 (193305'17214) delete 1e6/hit_set_24.1e6_archive_2014-08-11 18:37:04.946502_2014-08-11 18:42:36.225526/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:43:44.587824 2014-08-13 18:06:06.774073 7f5c973a57a0 20 read_log 194215'17275 (192834'16843) clean 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by osd.285.0:2823 2014-08-12 15:43:44.592339 2014-08-13 18:06:06.774100 7f5c973a57a0 20 read_log 194215'17276 (194215'17275) delete 5a2601e6/benchmark_data_storage0111-ib.pocstor_41526_object21486/head//24 by client.2174595.0:100553 0.000000 2014-08-13 18:06:06.774113 7f5c973a57a0 20 read_log 194215'17277 (192888'16997) clean 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by osd.285.0:2825 2014-08-12 15:43:44.624433 2014-08-13 18:06:06.774132 7f5c973a57a0 20 read_log 194215'17278 (194215'17277) delete 5b1a01e6/benchmark_data_storage0111-ib.pocstor_41097_object19064/head//24 by client.2174595.0:100555 0.000000 2014-08-13 18:06:06.774155 7f5c973a57a0 20 read_log 194215'17279 (192834'16276) clean af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by osd.285.0:2827 2014-08-12 15:43:44.633023 2014-08-13 18:06:06.774209 7f5c973a57a0 20 read_log 194215'17280 (194215'17279) delete af0811e6/benchmark_data_storage0111-ib.pocstor_26659_object12412/head//24 by client.2174595.0:100557 0.000000 2014-08-13 18:06:06.774224 7f5c973a57a0 20 read_log 194215'17281 (193305'17264) clean 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by osd.285.0:2829 2014-08-12 15:43:44.659064 2014-08-13 18:06:06.774237 7f5c973a57a0 20 read_log 194215'17282 (194215'17281) delete 5a0421e6/benchmark_data_storage0111-ib.pocstor_62621_object17626/head//24 by client.2174595.0:100559 0.000000 2014-08-13 18:06:06.774278 7f5c973a57a0 20 read_log 194215'17283 (192834'16715) clean 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by osd.285.0:2831 2014-08-12 15:43:44.667039 2014-08-13 18:06:06.774295 7f5c973a57a0 20 read_log 194215'17284 (194215'17283) delete 5d5521e6/benchmark_data_storage0111-ib.pocstor_41526_object3541/head//24 by client.2174595.0:100561 0.000000 2014-08-13 18:06:06.774306 7f5c973a57a0 20 read_log 194215'17285 (192888'17013) clean 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by osd.285.0:2833 2014-08-12 15:43:44.695886 2014-08-13 18:06:06.774319 7f5c973a57a0 20 read_log 194215'17286 (194215'17285) delete 460231e6/benchmark_data_storage0111-ib.pocstor_41097_object27230/head//24 by client.2174595.0:100563 0.000000 2014-08-13 18:06:06.774330 7f5c973a57a0 20 read_log 194215'17287 (192888'17007) clean 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by osd.285.0:2835 2014-08-12 15:43:44.703714 2014-08-13 18:06:06.774343 7f5c973a57a0 20 read_log 194215'17288 (194215'17287) delete 1ec651e6/benchmark_data_storage0111-ib.pocstor_41097_object23222/head//24 by client.2174595.0:100565 0.000000 2014-08-13 18:06:06.774353 7f5c973a57a0 20 read_log 194215'17289 (192834'16304) clean b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by osd.285.0:2837 2014-08-12 15:43:44.711223 2014-08-13 18:06:06.774366 7f5c973a57a0 20 read_log 194215'17290 (194215'17289) delete b24661e6/benchmark_data_storage0111-ib.pocstor_26659_object16454/head//24 by client.2174595.0:100567 0.000000 2014-08-13 18:06:06.774376 7f5c973a57a0 20 read_log 194215'17291 (192834'16302) clean 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by osd.285.0:2839 2014-08-12 15:43:44.719591 2014-08-13 18:06:06.774388 7f5c973a57a0 20 read_log 194215'17292 (194215'17291) delete 290a61e6/benchmark_data_storage0111-ib.pocstor_26659_object16083/head//24 by client.2174595.0:100569 0.000000 2014-08-13 18:06:06.774399 7f5c973a57a0 20 read_log 194215'17293 (192834'16738) clean 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by osd.285.0:2841 2014-08-12 15:43:44.727403 2014-08-13 18:06:06.774411 7f5c973a57a0 20 read_log 194215'17294 (194215'17293) delete 804e61e6/benchmark_data_storage0111-ib.pocstor_41526_object5978/head//24 by client.2174595.0:100571 0.000000 2014-08-13 18:06:06.774422 7f5c973a57a0 20 read_log 194215'17295 (192888'16975) clean bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by osd.285.0:2843 2014-08-12 15:43:44.735691 2014-08-13 18:06:06.774435 7f5c973a57a0 20 read_log 194215'17296 (194215'17295) delete bc9271e6/benchmark_data_storage0111-ib.pocstor_41097_object7712/head//24 by client.2174595.0:100573 0.000000 2014-08-13 18:06:06.774445 7f5c973a57a0 20 read_log 194215'17297 (192888'17017) clean 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by osd.285.0:2845 2014-08-12 15:43:44.743902 2014-08-13 18:06:06.774457 7f5c973a57a0 20 read_log 194215'17298 (194215'17297) delete 2be671e6/benchmark_data_storage0111-ib.pocstor_41097_object29082/head//24 by client.2174595.0:100575 0.000000 2014-08-13 18:06:06.774468 7f5c973a57a0 20 read_log 194215'17299 (192834'16721) clean 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by osd.285.0:2847 2014-08-12 15:43:44.751565 2014-08-13 18:06:06.774490 7f5c973a57a0 20 read_log 194215'17300 (194215'17299) delete 447771e6/benchmark_data_storage0111-ib.pocstor_41526_object4153/head//24 by client.2174595.0:100577 0.000000 2014-08-13 18:06:06.774502 7f5c973a57a0 20 read_log 194215'17301 (192888'17155) clean c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by osd.285.0:2849 2014-08-12 15:43:44.760010 2014-08-13 18:06:06.774515 7f5c973a57a0 20 read_log 194215'17302 (194215'17301) delete c5f291e6/benchmark_data_storage0111-ib.pocstor_2228_object6696/head//24 by client.2174595.0:100579 0.000000 2014-08-13 18:06:06.774526 7f5c973a57a0 20 read_log 194215'17303 (192888'16996) clean 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by osd.285.0:2851 2014-08-12 15:43:44.787605 2014-08-13 18:06:06.774538 7f5c973a57a0 20 read_log 194215'17304 (194215'17303) delete 5bdb91e6/benchmark_data_storage0111-ib.pocstor_41097_object18756/head//24 by client.2174595.0:100581 0.000000 2014-08-13 18:06:06.774549 7f5c973a57a0 20 read_log 194215'17305 (192888'16990) clean 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by osd.285.0:2853 2014-08-12 15:43:44.801514 2014-08-13 18:06:06.774562 7f5c973a57a0 20 read_log 194215'17306 (194215'17305) delete 4049b1e6/benchmark_data_storage0111-ib.pocstor_41097_object16133/head//24 by client.2174595.0:100583 0.000000 2014-08-13 18:06:06.774573 7f5c973a57a0 20 read_log 194215'17307 (192834'16213) clean 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by osd.285.0:2855 2014-08-12 15:43:44.809686 2014-08-13 18:06:06.774585 7f5c973a57a0 20 read_log 194215'17308 (194215'17307) delete 2cdb1e6/benchmark_data_storage0111-ib.pocstor_26659_object4099/head//24 by client.2174595.0:100585 0.000000 2014-08-13 18:06:06.774596 7f5c973a57a0 20 read_log 194215'17309 (192888'17187) clean e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by osd.285.0:2857 2014-08-12 15:43:44.818120 2014-08-13 18:06:06.774608 7f5c973a57a0 20 read_log 194215'17310 (194215'17309) delete e271c1e6/benchmark_data_storage0111-ib.pocstor_2228_object21897/head//24 by client.2174595.0:100587 0.000000 2014-08-13 18:06:06.774619 7f5c973a57a0 20 read_log 194215'17311 (193305'17268) clean e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by osd.285.0:2859 2014-08-12 15:43:44.826307 2014-08-13 18:06:06.774631 7f5c973a57a0 20 read_log 194215'17312 (194215'17311) delete e981c1e6/benchmark_data_storage0111-ib.pocstor_62621_object23974/head//24 by client.2174595.0:100589 0.000000 2014-08-13 18:06:06.774642 7f5c973a57a0 20 read_log 194215'17313 (193305'17255) clean 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by osd.285.0:2861 2014-08-12 15:43:44.834048 2014-08-13 18:06:06.774654 7f5c973a57a0 20 read_log 194215'17314 (194215'17313) delete 7983c1e6/benchmark_data_storage0111-ib.pocstor_62621_object920/head//24 by client.2174595.0:100591 0.000000 2014-08-13 18:06:06.774665 7f5c973a57a0 20 read_log 194215'17315 (192888'16984) clean 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by osd.285.0:2863 2014-08-12 15:43:44.841604 2014-08-13 18:06:06.774677 7f5c973a57a0 20 read_log 194215'17316 (194215'17315) delete 6144c1e6/benchmark_data_storage0111-ib.pocstor_41097_object13643/head//24 by client.2174595.0:100593 0.000000 2014-08-13 18:06:06.774688 7f5c973a57a0 20 read_log 194215'17317 (192834'16287) clean f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by osd.285.0:2865 2014-08-12 15:43:44.849506 2014-08-13 18:06:06.774700 7f5c973a57a0 20 read_log 194215'17318 (194215'17317) delete f5a6c1e6/benchmark_data_storage0111-ib.pocstor_26659_object13864/head//24 by client.2174595.0:100595 0.000000 2014-08-13 18:06:06.774711 7f5c973a57a0 20 read_log 194215'17319 (192888'17211) clean 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by osd.285.0:2867 2014-08-12 15:43:44.876595 2014-08-13 18:06:06.774733 7f5c973a57a0 20 read_log 194215'17320 (194215'17319) delete 15f9c1e6/benchmark_data_storage0111-ib.pocstor_2228_object30838/head//24 by client.2174595.0:100597 0.000000 2014-08-13 18:06:06.774747 7f5c973a57a0 20 read_log 194215'17321 (192888'17202) clean fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by osd.285.0:2869 2014-08-12 15:43:44.884458 2014-08-13 18:06:06.774782 7f5c973a57a0 20 read_log 194215'17322 (194215'17321) delete fecc1e6/benchmark_data_storage0111-ib.pocstor_2228_object26036/head//24 by client.2174595.0:100599 0.000000 2014-08-13 18:06:06.774797 7f5c973a57a0 20 read_log 194215'17323 (192888'17152) clean 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by osd.285.0:2871 2014-08-12 15:43:44.893249 2014-08-13 18:06:06.774812 7f5c973a57a0 20 read_log 194215'17324 (194215'17323) delete 6ac3e1e6/benchmark_data_storage0111-ib.pocstor_2228_object3341/head//24 by client.2174595.0:100601 0.000000 2014-08-13 18:06:06.774823 7f5c973a57a0 20 read_log 194215'17325 (192888'17015) clean 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by osd.285.0:2873 2014-08-12 15:43:44.901339 2014-08-13 18:06:06.774836 7f5c973a57a0 20 read_log 194215'17326 (194215'17325) delete 8ca1f1e6/benchmark_data_storage0111-ib.pocstor_41097_object28786/head//24 by client.2174595.0:100603 0.000000 2014-08-13 18:06:06.774847 7f5c973a57a0 20 read_log 194215'17327 (192888'16972) clean d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by osd.285.0:2875 2014-08-12 15:43:44.909553 2014-08-13 18:06:06.774859 7f5c973a57a0 20 read_log 194215'17328 (194215'17327) delete d1b5f1e6/benchmark_data_storage0111-ib.pocstor_41097_object5948/head//24 by client.2174595.0:100605 0.000000 2014-08-13 18:06:06.774873 7f5c973a57a0 20 read_log 194215'17329 (192834'16229) clean 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by osd.285.0:2877 2014-08-12 15:43:44.918659 2014-08-13 18:06:06.774888 7f5c973a57a0 20 read_log 194215'17330 (194215'17329) delete 2019f1e6/benchmark_data_storage0111-ib.pocstor_26659_object7233/head//24 by client.2174595.0:100607 0.000000 2014-08-13 18:06:06.774899 7f5c973a57a0 20 read_log 194215'17331 (192834'16220) clean f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by osd.285.0:2879 2014-08-12 15:43:44.926908 2014-08-13 18:06:06.774911 7f5c973a57a0 20 read_log 194215'17332 (194215'17331) delete f61bf1e6/benchmark_data_storage0111-ib.pocstor_26659_object5729/head//24 by client.2174595.0:100609 0.000000 2014-08-13 18:06:06.774924 7f5c973a57a0 20 read_log 194215'17333 (192834'16810) clean 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by osd.285.0:2881 2014-08-12 15:43:44.934470 2014-08-13 18:06:06.774937 7f5c973a57a0 20 read_log 194215'17334 (194215'17333) delete 308cf1e6/benchmark_data_storage0111-ib.pocstor_41526_object15755/head//24 by client.2174595.0:100611 0.000000 2014-08-13 18:06:06.774949 7f5c973a57a0 20 read_log 194215'17335 (192888'17180) clean d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by osd.285.0:2883 2014-08-12 15:43:44.943467 2014-08-13 18:06:06.774963 7f5c973a57a0 20 read_log 194215'17336 (194215'17335) delete d52df1e6/benchmark_data_storage0111-ib.pocstor_2228_object18728/head//24 by client.2174595.0:100613 0.000000 2014-08-13 18:06:06.774974 7f5c973a57a0 20 read_log 194215'17337 (193305'17256) clean c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by osd.285.0:2885 2014-08-12 15:43:44.951227 2014-08-13 18:06:06.774987 7f5c973a57a0 20 read_log 194215'17338 (194215'17337) delete c4b315e6/benchmark_data_storage0111-ib.pocstor_62621_object1442/head//24 by client.2174595.0:100615 0.000000 2014-08-13 18:06:06.775001 7f5c973a57a0 20 read_log 194215'17339 (192888'17210) clean 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by osd.285.0:2887 2014-08-12 15:43:44.960497 2014-08-13 18:06:06.775022 7f5c973a57a0 20 read_log 194215'17340 (194215'17339) delete 547715e6/benchmark_data_storage0111-ib.pocstor_2228_object29567/head//24 by client.2174595.0:100617 0.000000 2014-08-13 18:06:06.775036 7f5c973a57a0 20 read_log 194215'17341 (193305'17267) clean 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by osd.285.0:2889 2014-08-12 15:43:44.970372 2014-08-13 18:06:06.775048 7f5c973a57a0 20 read_log 194215'17342 (194215'17341) delete 568125e6/benchmark_data_storage0111-ib.pocstor_62621_object21326/head//24 by client.2174595.0:100619 0.000000 2014-08-13 18:06:06.775061 7f5c973a57a0 20 read_log 194215'17343 (192888'17003) clean f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by osd.285.0:2891 2014-08-12 15:43:44.978818 2014-08-13 18:06:06.775075 7f5c973a57a0 20 read_log 194215'17344 (194215'17343) delete f74b25e6/benchmark_data_storage0111-ib.pocstor_41097_object21943/head//24 by client.2174595.0:100621 0.000000 2014-08-13 18:06:06.775086 7f5c973a57a0 20 read_log 194215'17345 (193305'17263) clean dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by osd.285.0:2893 2014-08-12 15:43:44.986267 2014-08-13 18:06:06.775099 7f5c973a57a0 20 read_log 194215'17346 (194215'17345) delete dbac25e6/benchmark_data_storage0111-ib.pocstor_62621_object16910/head//24 by client.2174595.0:100623 0.000000 2014-08-13 18:06:06.775110 7f5c973a57a0 20 read_log 194215'17347 (192888'17190) clean 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by osd.285.0:2895 2014-08-12 15:43:44.994718 2014-08-13 18:06:06.775122 7f5c973a57a0 20 read_log 194215'17348 (194215'17347) delete 1dc045e6/benchmark_data_storage0111-ib.pocstor_2228_object22469/head//24 by client.2174595.0:100625 0.000000 2014-08-13 18:06:06.775134 7f5c973a57a0 20 read_log 194215'17349 (192888'17012) clean ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by osd.285.0:2897 2014-08-12 15:43:45.002941 2014-08-13 18:06:06.775159 7f5c973a57a0 20 read_log 194215'17350 (194215'17349) delete ae0445e6/benchmark_data_storage0111-ib.pocstor_41097_object27214/head//24 by client.2174595.0:100627 0.000000 2014-08-13 18:06:06.775173 7f5c973a57a0 20 read_log 194215'17351 (193305'17258) clean b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by osd.285.0:2899 2014-08-12 15:43:45.010691 2014-08-13 18:06:06.775206 7f5c973a57a0 20 read_log 194215'17352 (194215'17351) delete b78445e6/benchmark_data_storage0111-ib.pocstor_62621_object4905/head//24 by client.2174595.0:100629 0.000000 2014-08-13 18:06:06.775217 7f5c973a57a0 20 read_log 194215'17353 (192888'16985) clean eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by osd.285.0:2901 2014-08-12 15:43:45.037182 2014-08-13 18:06:06.775229 7f5c973a57a0 20 read_log 194215'17354 (194215'17353) delete eb2555e6/benchmark_data_storage0111-ib.pocstor_41097_object13650/head//24 by client.2174595.0:100631 0.000000 2014-08-13 18:06:06.775240 7f5c973a57a0 20 read_log 194215'17355 (192888'17157) clean 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by osd.285.0:2903 2014-08-12 15:43:45.047036 2014-08-13 18:06:06.775254 7f5c973a57a0 20 read_log 194215'17356 (194215'17355) delete 83c655e6/benchmark_data_storage0111-ib.pocstor_2228_object7220/head//24 by client.2174595.0:100633 0.000000 2014-08-13 18:06:06.775271 7f5c973a57a0 20 read_log 194215'17357 (192888'17199) clean 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by osd.285.0:2905 2014-08-12 15:43:45.055226 2014-08-13 18:06:06.775286 7f5c973a57a0 20 read_log 194215'17358 (194215'17357) delete 62b765e6/benchmark_data_storage0111-ib.pocstor_2228_object25243/head//24 by client.2174595.0:100635 0.000000 2014-08-13 18:06:06.775297 7f5c973a57a0 20 read_log 194215'17359 (192888'17196) clean 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by osd.285.0:2907 2014-08-12 15:43:45.064125 2014-08-13 18:06:06.775323 7f5c973a57a0 20 read_log 194215'17360 (194215'17359) delete 299b65e6/benchmark_data_storage0111-ib.pocstor_2228_object24131/head//24 by client.2174595.0:100637 0.000000 2014-08-13 18:06:06.775337 7f5c973a57a0 20 read_log 194215'17361 (192888'16983) clean 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by osd.285.0:2909 2014-08-12 15:43:45.072834 2014-08-13 18:06:06.775350 7f5c973a57a0 20 read_log 194215'17362 (194215'17361) delete 945975e6/benchmark_data_storage0111-ib.pocstor_41097_object13297/head//24 by client.2174595.0:100639 0.000000 2014-08-13 18:06:06.775364 7f5c973a57a0 20 read_log 194215'17363 (192834'16205) clean 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by osd.285.0:2911 2014-08-12 15:43:45.094029 2014-08-13 18:06:06.775379 7f5c973a57a0 20 read_log 194215'17364 (194215'17363) delete 3e8c75e6/benchmark_data_storage0111-ib.pocstor_26659_object3009/head//24 by client.2174595.0:100641 0.000000 2014-08-13 18:06:06.775400 7f5c973a57a0 20 read_log 194215'17365 (192834'16710) clean ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by osd.285.0:2913 2014-08-12 15:43:45.101252 2014-08-13 18:06:06.775420 7f5c973a57a0 20 read_log 194215'17366 (194215'17365) delete ccac75e6/benchmark_data_storage0111-ib.pocstor_41526_object2641/head//24 by client.2174595.0:100643 0.000000 2014-08-13 18:06:06.775431 7f5c973a57a0 20 read_log 194215'17367 (192888'16986) clean 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by osd.285.0:2915 2014-08-12 15:43:45.108970 2014-08-13 18:06:06.775449 7f5c973a57a0 20 read_log 194215'17368 (194215'17367) delete 75bf75e6/benchmark_data_storage0111-ib.pocstor_41097_object14473/head//24 by client.2174595.0:100645 0.000000 2014-08-13 18:06:06.775463 7f5c973a57a0 20 read_log 194215'17369 (192834'16748) clean f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by osd.285.0:2917 2014-08-12 15:43:45.116722 2014-08-13 18:06:06.775475 7f5c973a57a0 20 read_log 194215'17370 (194215'17369) delete f0cc85e6/benchmark_data_storage0111-ib.pocstor_41526_object7393/head//24 by client.2174595.0:100647 0.000000 2014-08-13 18:06:06.775493 7f5c973a57a0 20 read_log 194215'17371 (192888'17186) clean 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by osd.285.0:2919 2014-08-12 15:43:45.124996 2014-08-13 18:06:06.775509 7f5c973a57a0 20 read_log 194215'17372 (194215'17371) delete 3f9595e6/benchmark_data_storage0111-ib.pocstor_2228_object21199/head//24 by client.2174595.0:100649 0.000000 2014-08-13 18:06:06.775521 7f5c973a57a0 20 read_log 194215'17373 (192834'16195) clean 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by osd.285.0:2921 2014-08-12 15:43:45.133195 2014-08-13 18:06:06.775533 7f5c973a57a0 20 read_log 194215'17374 (194215'17373) delete 88c695e6/benchmark_data_storage0111-ib.pocstor_26659_object1025/head//24 by client.2174595.0:100651 0.000000 2014-08-13 18:06:06.775556 7f5c973a57a0 20 read_log 194215'17375 (192888'16967) clean c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by osd.285.0:2923 2014-08-12 15:43:45.140885 2014-08-13 18:06:06.775590 7f5c973a57a0 20 read_log 194215'17376 (194215'17375) delete c1795e6/benchmark_data_storage0111-ib.pocstor_41097_object346/head//24 by client.2174595.0:100653 0.000000 2014-08-13 18:06:06.775611 7f5c973a57a0 20 read_log 194215'17377 (192834'16790) clean c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by osd.285.0:2925 2014-08-12 15:43:45.148400 2014-08-13 18:06:06.775624 7f5c973a57a0 20 read_log 194215'17378 (194215'17377) delete c58a95e6/benchmark_data_storage0111-ib.pocstor_41526_object13318/head//24 by client.2174595.0:100655 0.000000 2014-08-13 18:06:06.775635 7f5c973a57a0 20 read_log 194215'17379 (192834'16218) clean b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by osd.285.0:2927 2014-08-12 15:43:45.155865 2014-08-13 18:06:06.775664 7f5c973a57a0 20 read_log 194215'17380 (194215'17379) delete b7eca5e6/benchmark_data_storage0111-ib.pocstor_26659_object5589/head//24 by client.2174595.0:100657 0.000000 2014-08-13 18:06:06.775680 7f5c973a57a0 20 read_log 194215'17381 (192834'16278) clean 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by osd.285.0:2929 2014-08-12 15:43:45.162904 2014-08-13 18:06:06.775693 7f5c973a57a0 20 read_log 194215'17382 (194215'17381) delete 15b2c5e6/benchmark_data_storage0111-ib.pocstor_26659_object12784/head//24 by client.2174595.0:100659 0.000000 2014-08-13 18:06:06.775704 7f5c973a57a0 20 read_log 194215'17383 (192888'17009) clean efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by osd.285.0:2931 2014-08-12 15:43:45.171731 2014-08-13 18:06:06.775716 7f5c973a57a0 20 read_log 194215'17384 (194215'17383) delete efbac5e6/benchmark_data_storage0111-ib.pocstor_41097_object24739/head//24 by client.2174595.0:100661 0.000000 2014-08-13 18:06:06.775727 7f5c973a57a0 20 read_log 194215'17385 (192834'16339) clean afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by osd.285.0:2933 2014-08-12 15:43:45.180409 2014-08-13 18:06:06.775739 7f5c973a57a0 20 read_log 194215'17386 (194215'17385) delete afffc5e6/benchmark_data_storage0111-ib.pocstor_26659_object21356/head//24 by client.2174595.0:100663 0.000000 2014-08-13 18:06:06.775750 7f5c973a57a0 20 read_log 194215'17387 (192834'16830) clean 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by osd.285.0:2935 2014-08-12 15:43:45.188219 2014-08-13 18:06:06.775762 7f5c973a57a0 20 read_log 194215'17388 (194215'17387) delete 8373d5e6/benchmark_data_storage0111-ib.pocstor_41526_object19767/head//24 by client.2174595.0:100665 0.000000 2014-08-13 18:06:06.775776 7f5c973a57a0 20 read_log 194215'17389 (192888'17195) clean 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by osd.285.0:2937 2014-08-12 15:43:45.222729 2014-08-13 18:06:06.775789 7f5c973a57a0 20 read_log 194215'17390 (194215'17389) delete 342bd5e6/benchmark_data_storage0111-ib.pocstor_2228_object24000/head//24 by client.2174595.0:100667 0.000000 2014-08-13 18:06:06.775800 7f5c973a57a0 20 read_log 194215'17391 (192834'16765) clean da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by osd.285.0:2939 2014-08-12 15:43:45.230738 2014-08-13 18:06:06.775812 7f5c973a57a0 20 read_log 194215'17392 (194215'17391) delete da09e5e6/benchmark_data_storage0111-ib.pocstor_41526_object9533/head//24 by client.2174595.0:100669 0.000000 2014-08-13 18:06:06.775831 7f5c973a57a0 20 read_log 194215'17393 (192888'17010) clean ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by osd.285.0:2941 2014-08-12 15:43:45.239239 2014-08-13 18:06:06.775847 7f5c973a57a0 20 read_log 194215'17394 (194215'17393) delete ea1ae5e6/benchmark_data_storage0111-ib.pocstor_41097_object24977/head//24 by client.2174595.0:100671 0.000000 2014-08-13 18:06:06.775858 7f5c973a57a0 20 read_log 194215'17395 (192834'16698) clean a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by osd.285.0:2943 2014-08-12 15:43:45.262532 2014-08-13 18:06:06.775871 7f5c973a57a0 20 read_log 194215'17396 (194215'17395) delete a500f5e6/benchmark_data_storage0111-ib.pocstor_41526_object1145/head//24 by client.2174595.0:100673 0.000000 2014-08-13 18:06:06.775882 7f5c973a57a0 20 read_log 194215'17397 (192834'16797) clean 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by osd.285.0:2945 2014-08-12 15:43:45.272139 2014-08-13 18:06:06.775895 7f5c973a57a0 20 read_log 194215'17398 (194215'17397) delete 6e2af5e6/benchmark_data_storage0111-ib.pocstor_41526_object14277/head//24 by client.2174595.0:100675 0.000000 2014-08-13 18:06:06.775906 7f5c973a57a0 20 read_log 194215'17399 (192834'16327) clean e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by osd.285.0:2947 2014-08-12 15:43:45.302952 2014-08-13 18:06:06.775944 7f5c973a57a0 20 read_log 194215'17400 (194215'17399) delete e7fa09e6/benchmark_data_storage0111-ib.pocstor_26659_object19909/head//24 by client.2174595.0:100677 0.000000 2014-08-13 18:06:06.775958 7f5c973a57a0 20 read_log 194215'17401 (192888'17172) clean 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by osd.285.0:2949 2014-08-12 15:43:45.313335 2014-08-13 18:06:06.775971 7f5c973a57a0 20 read_log 194215'17402 (194215'17401) delete 8d2819e6/benchmark_data_storage0111-ib.pocstor_2228_object13813/head//24 by client.2174595.0:100679 0.000000 2014-08-13 18:06:06.775982 7f5c973a57a0 20 read_log 194215'17403 (192888'17163) clean e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by osd.285.0:2951 2014-08-12 15:43:45.321425 2014-08-13 18:06:06.775994 7f5c973a57a0 20 read_log 194215'17404 (194215'17403) delete e1bc19e6/benchmark_data_storage0111-ib.pocstor_2228_object9428/head//24 by client.2174595.0:100681 0.000000 2014-08-13 18:06:06.776005 7f5c973a57a0 20 read_log 194215'17405 (193305'17270) clean b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by osd.285.0:2953 2014-08-12 15:43:45.346963 2014-08-13 18:06:06.776018 7f5c973a57a0 20 read_log 194215'17406 (194215'17405) delete b0929e6/benchmark_data_storage0111-ib.pocstor_62621_object29773/head//24 by client.2174595.0:100683 0.000000 2014-08-13 18:06:06.776028 7f5c973a57a0 20 read_log 194215'17407 (192888'16976) clean c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by osd.285.0:2955 2014-08-12 15:43:45.355489 2014-08-13 18:06:06.776040 7f5c973a57a0 20 read_log 194215'17408 (194215'17407) delete c1c239e6/benchmark_data_storage0111-ib.pocstor_41097_object8552/head//24 by client.2174595.0:100685 0.000000 2014-08-13 18:06:06.776051 7f5c973a57a0 20 read_log 194215'17409 (192888'16992) clean a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by osd.285.0:2957 2014-08-12 15:43:45.364635 2014-08-13 18:06:06.776063 7f5c973a57a0 20 read_log 194215'17410 (194215'17409) delete a00f39e6/benchmark_data_storage0111-ib.pocstor_41097_object16687/head//24 by client.2174595.0:100687 0.000000 2014-08-13 18:06:06.776074 7f5c973a57a0 20 read_log 194215'17411 (192888'17164) clean ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by osd.285.0:2959 2014-08-12 15:43:45.373698 2014-08-13 18:06:06.776087 7f5c973a57a0 20 read_log 194215'17412 (194215'17411) delete ed7f39e6/benchmark_data_storage0111-ib.pocstor_2228_object9671/head//24 by client.2174595.0:100689 0.000000 2014-08-13 18:06:06.776098 7f5c973a57a0 20 read_log 194215'17413 (192888'16998) clean 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by osd.285.0:2961 2014-08-12 15:43:45.383128 2014-08-13 18:06:06.776110 7f5c973a57a0 20 read_log 194215'17414 (194215'17413) delete 79849e6/benchmark_data_storage0111-ib.pocstor_41097_object20451/head//24 by client.2174595.0:100691 0.000000 2014-08-13 18:06:06.776123 7f5c973a57a0 20 read_log 194215'17415 (192834'16214) clean 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by osd.285.0:2963 2014-08-12 15:43:45.402305 2014-08-13 18:06:06.776137 7f5c973a57a0 20 read_log 194215'17416 (194215'17415) delete 423159e6/benchmark_data_storage0111-ib.pocstor_26659_object4253/head//24 by client.2174595.0:100693 0.000000 2014-08-13 18:06:06.776154 7f5c973a57a0 20 read_log 194215'17417 (192888'17004) clean 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by osd.285.0:2965 2014-08-12 15:43:45.410408 2014-08-13 18:06:06.776169 7f5c973a57a0 20 read_log 194215'17418 (194215'17417) delete 7c2359e6/benchmark_data_storage0111-ib.pocstor_41097_object22335/head//24 by client.2174595.0:100695 0.000000 2014-08-13 18:06:06.776181 7f5c973a57a0 20 read_log 194215'17419 (192888'16995) clean d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by osd.285.0:2967 2014-08-12 15:43:45.418929 2014-08-13 18:06:06.776200 7f5c973a57a0 20 read_log 194215'17420 (194215'17419) delete d79359e6/benchmark_data_storage0111-ib.pocstor_41097_object18604/head//24 by client.2174595.0:100697 0.000000 2014-08-13 18:06:06.776213 7f5c973a57a0 20 read_log 194215'17421 (193305'17261) clean 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by osd.285.0:2969 2014-08-12 15:43:45.427273 2014-08-13 18:06:06.776226 7f5c973a57a0 20 read_log 194215'17422 (194215'17421) delete 6df459e6/benchmark_data_storage0111-ib.pocstor_62621_object11671/head//24 by client.2174595.0:100699 0.000000 2014-08-13 18:06:06.776237 7f5c973a57a0 20 read_log 194215'17423 (193305'17272) clean ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by osd.285.0:2971 2014-08-12 15:43:45.435639 2014-08-13 18:06:06.776251 7f5c973a57a0 20 read_log 194215'17424 (194215'17423) delete ea2069e6/benchmark_data_storage0111-ib.pocstor_62621_object31870/head//24 by client.2174595.0:100701 0.000000 2014-08-13 18:06:06.776265 7f5c973a57a0 20 read_log 194215'17425 (192834'16245) clean dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by osd.285.0:2973 2014-08-12 15:43:45.444189 2014-08-13 18:06:06.776278 7f5c973a57a0 20 read_log 194215'17426 (194215'17425) delete dd1479e6/benchmark_data_storage0111-ib.pocstor_26659_object8937/head//24 by client.2174595.0:100703 0.000000 2014-08-13 18:06:06.776288 7f5c973a57a0 20 read_log 194215'17427 (192834'16774) clean c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by osd.285.0:2975 2014-08-12 15:43:45.452405 2014-08-13 18:06:06.776301 7f5c973a57a0 20 read_log 194215'17428 (194215'17427) delete c9ab79e6/benchmark_data_storage0111-ib.pocstor_41526_object10822/head//24 by client.2174595.0:100705 0.000000 2014-08-13 18:06:06.776311 7f5c973a57a0 20 read_log 194215'17429 (192834'16816) clean 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by osd.285.0:2977 2014-08-12 15:43:45.460567 2014-08-13 18:06:06.776327 7f5c973a57a0 20 read_log 194215'17430 (194215'17429) delete 7a6c79e6/benchmark_data_storage0111-ib.pocstor_41526_object16921/head//24 by client.2174595.0:100707 0.000000 2014-08-13 18:06:06.776338 7f5c973a57a0 20 read_log 194215'17431 (192888'17191) clean a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by osd.285.0:2979 2014-08-12 15:43:45.469302 2014-08-13 18:06:06.776351 7f5c973a57a0 20 read_log 194215'17432 (194215'17431) delete a48189e6/benchmark_data_storage0111-ib.pocstor_2228_object22536/head//24 by client.2174595.0:100709 0.000000 2014-08-13 18:06:06.776362 7f5c973a57a0 20 read_log 194215'17433 (193305'17254) clean 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by osd.285.0:2981 2014-08-12 15:43:45.477308 2014-08-13 18:06:06.776378 7f5c973a57a0 20 read_log 194215'17434 (194215'17433) delete 79ba89e6/benchmark_data_storage0111-ib.pocstor_62621_object792/head//24 by client.2174595.0:100711 0.000000 2014-08-13 18:06:06.776388 7f5c973a57a0 20 read_log 194215'17435 (192834'16201) clean 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by osd.285.0:2983 2014-08-12 15:43:45.485662 2014-08-13 18:06:06.776416 7f5c973a57a0 20 read_log 194215'17436 (194215'17435) delete 88099e6/benchmark_data_storage0111-ib.pocstor_26659_object1831/head//24 by client.2174595.0:100713 0.000000 2014-08-13 18:06:06.776431 7f5c973a57a0 20 read_log 194215'17437 (192888'17154) clean 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by osd.285.0:2985 2014-08-12 15:43:45.494587 2014-08-13 18:06:06.776453 7f5c973a57a0 20 read_log 194215'17438 (194215'17437) delete 18f299e6/benchmark_data_storage0111-ib.pocstor_2228_object5011/head//24 by client.2174595.0:100715 0.000000 2014-08-13 18:06:06.776468 7f5c973a57a0 20 read_log 194215'17439 (192834'16750) clean 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by osd.285.0:2987 2014-08-12 15:43:45.502572 2014-08-13 18:06:06.776490 7f5c973a57a0 20 read_log 194215'17440 (194215'17439) delete 7e6899e6/benchmark_data_storage0111-ib.pocstor_41526_object7621/head//24 by client.2174595.0:100717 0.000000 2014-08-13 18:06:06.776514 7f5c973a57a0 20 read_log 194215'17441 (192888'16989) clean 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by osd.285.0:2989 2014-08-12 15:43:45.529043 2014-08-13 18:06:06.776527 7f5c973a57a0 20 read_log 194215'17442 (194215'17441) delete 314a99e6/benchmark_data_storage0111-ib.pocstor_41097_object16086/head//24 by client.2174595.0:100719 0.000000 2014-08-13 18:06:06.776539 7f5c973a57a0 20 read_log 194215'17443 (192888'17176) clean 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by osd.285.0:2991 2014-08-12 15:43:45.538065 2014-08-13 18:06:06.776551 7f5c973a57a0 20 read_log 194215'17444 (194215'17443) delete 1a12a9e6/benchmark_data_storage0111-ib.pocstor_2228_object16417/head//24 by client.2174595.0:100721 0.000000 2014-08-13 18:06:06.776562 7f5c973a57a0 20 read_log 194215'17445 (192888'17005) clean 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by osd.285.0:2993 2014-08-12 15:43:45.546055 2014-08-13 18:06:06.776574 7f5c973a57a0 20 read_log 194215'17446 (194215'17445) delete 7593a9e6/benchmark_data_storage0111-ib.pocstor_41097_object22372/head//24 by client.2174595.0:100723 0.000000 2014-08-13 18:06:06.776585 7f5c973a57a0 20 read_log 194215'17447 (193305'17259) clean 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by osd.285.0:2995 2014-08-12 15:43:45.555250 2014-08-13 18:06:06.776603 7f5c973a57a0 20 read_log 194215'17448 (194215'17447) delete 7d84b9e6/benchmark_data_storage0111-ib.pocstor_62621_object9219/head//24 by client.2174595.0:100725 0.000000 2014-08-13 18:06:06.776614 7f5c973a57a0 20 read_log 194215'17449 (192834'16845) clean 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by osd.285.0:2997 2014-08-12 15:43:45.563625 2014-08-13 18:06:06.776634 7f5c973a57a0 20 read_log 194215'17450 (194215'17449) delete 2578b9e6/benchmark_data_storage0111-ib.pocstor_41526_object21933/head//24 by client.2174595.0:100727 0.000000 2014-08-13 18:06:06.776650 7f5c973a57a0 20 read_log 194215'17451 (192834'16796) clean c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by osd.285.0:2999 2014-08-12 15:43:45.572256 2014-08-13 18:06:06.776663 7f5c973a57a0 20 read_log 194215'17452 (194215'17451) delete c904c9e6/benchmark_data_storage0111-ib.pocstor_41526_object14138/head//24 by client.2174595.0:100729 0.000000 2014-08-13 18:06:06.776677 7f5c973a57a0 20 read_log 194215'17453 (192888'16993) clean a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by osd.285.0:3001 2014-08-12 15:43:45.614869 2014-08-13 18:06:06.776692 7f5c973a57a0 20 read_log 194215'17454 (194215'17453) delete a5a8c9e6/benchmark_data_storage0111-ib.pocstor_41097_object17434/head//24 by client.2174595.0:100731 0.000000 2014-08-13 18:06:06.776703 7f5c973a57a0 20 read_log 194215'17455 (192834'16212) clean 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by osd.285.0:3003 2014-08-12 15:43:45.623073 2014-08-13 18:06:06.776726 7f5c973a57a0 20 read_log 194215'17456 (194215'17455) delete 9a3e9e6/benchmark_data_storage0111-ib.pocstor_26659_object4058/head//24 by client.2174595.0:100733 0.000000 2014-08-13 18:06:06.776740 7f5c973a57a0 20 read_log 194215'17457 (192888'16981) clean 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by osd.285.0:3005 2014-08-12 15:43:45.631157 2014-08-13 18:06:06.776760 7f5c973a57a0 20 read_log 194215'17458 (194215'17457) delete 8478f9e6/benchmark_data_storage0111-ib.pocstor_41097_object12178/head//24 by client.2174595.0:100735 0.000000 2014-08-13 18:06:06.776775 7f5c973a57a0 20 read_log 194215'17459 (192834'16261) clean e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by osd.285.0:3007 2014-08-12 15:43:45.638643 2014-08-13 18:06:06.776808 7f5c973a57a0 20 read_log 194215'17460 (194215'17459) delete e9ef9e6/benchmark_data_storage0111-ib.pocstor_26659_object10286/head//24 by client.2174595.0:100737 0.000000 2014-08-13 18:06:06.776827 7f5c973a57a0 20 read_log 194215'17461 (192888'17014) clean f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by osd.285.0:3009 2014-08-12 15:43:45.646167 2014-08-13 18:06:06.776842 7f5c973a57a0 20 read_log 194215'17462 (194215'17461) delete f1cff9e6/benchmark_data_storage0111-ib.pocstor_41097_object28111/head//24 by client.2174595.0:100739 0.000000 2014-08-13 18:06:06.776853 7f5c973a57a0 20 read_log 194215'17463 (192888'16987) clean 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by osd.285.0:3011 2014-08-12 15:43:45.670883 2014-08-13 18:06:06.776868 7f5c973a57a0 20 read_log 194215'17464 (194215'17463) delete 87a60de6/benchmark_data_storage0111-ib.pocstor_41097_object15313/head//24 by client.2174595.0:100741 0.000000 2014-08-13 18:06:06.776879 7f5c973a57a0 20 read_log 194215'17465 (192834'16249) clean 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by osd.285.0:3013 2014-08-12 15:43:45.679581 2014-08-13 18:06:06.776891 7f5c973a57a0 20 read_log 194215'17466 (194215'17465) delete 6eb31de6/benchmark_data_storage0111-ib.pocstor_26659_object9058/head//24 by client.2174595.0:100743 0.000000 2014-08-13 18:06:06.776902 7f5c973a57a0 20 read_log 194215'17467 (192888'17200) clean e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by osd.285.0:3015 2014-08-12 15:43:45.689120 2014-08-13 18:06:06.776914 7f5c973a57a0 20 read_log 194215'17468 (194215'17467) delete e4bc1de6/benchmark_data_storage0111-ib.pocstor_2228_object25266/head//24 by client.2174595.0:100745 0.000000 2014-08-13 18:06:06.776942 7f5c973a57a0 20 read_log 194215'17469 (193305'17265) clean d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by osd.285.0:3017 2014-08-12 15:43:45.698038 2014-08-13 18:06:06.776957 7f5c973a57a0 20 read_log 194215'17470 (194215'17469) delete d4642de6/benchmark_data_storage0111-ib.pocstor_62621_object18994/head//24 by client.2174595.0:100747 0.000000 2014-08-13 18:06:06.776968 7f5c973a57a0 20 read_log 194215'17471 (192888'17189) clean a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by osd.285.0:3019 2014-08-12 15:43:45.713776 2014-08-13 18:06:06.776981 7f5c973a57a0 20 read_log 194215'17472 (194215'17471) delete a5072de6/benchmark_data_storage0111-ib.pocstor_2228_object22374/head//24 by client.2174595.0:100749 0.000000 2014-08-13 18:06:06.776992 7f5c973a57a0 20 read_log 194215'17473 (192888'17207) clean 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by osd.285.0:3021 2014-08-12 15:43:45.721341 2014-08-13 18:06:06.777004 7f5c973a57a0 20 read_log 194215'17474 (194215'17473) delete 96053de6/benchmark_data_storage0111-ib.pocstor_2228_object27774/head//24 by client.2174595.0:100751 0.000000 2014-08-13 18:06:06.777015 7f5c973a57a0 20 read_log 194215'17475 (192888'17193) clean 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by osd.285.0:3023 2014-08-12 15:43:45.730042 2014-08-13 18:06:06.777028 7f5c973a57a0 20 read_log 194215'17476 (194215'17475) delete 2d853de6/benchmark_data_storage0111-ib.pocstor_2228_object22822/head//24 by client.2174595.0:100753 0.000000 2014-08-13 18:06:06.777039 7f5c973a57a0 20 read_log 194215'17477 (192888'17178) clean 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by osd.285.0:3025 2014-08-12 15:43:45.740294 2014-08-13 18:06:06.777051 7f5c973a57a0 20 read_log 194215'17478 (194215'17477) delete 63163de6/benchmark_data_storage0111-ib.pocstor_2228_object17254/head//24 by client.2174595.0:100755 0.000000 2014-08-13 18:06:06.777062 7f5c973a57a0 20 read_log 194215'17479 (192834'16254) clean c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by osd.285.0:3027 2014-08-12 15:43:45.748782 2014-08-13 18:06:06.777081 7f5c973a57a0 20 read_log 194215'17480 (194215'17479) delete c19a3de6/benchmark_data_storage0111-ib.pocstor_26659_object9570/head//24 by client.2174595.0:100757 0.000000 2014-08-13 18:06:06.777092 7f5c973a57a0 20 read_log 194215'17481 (192834'16753) clean d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by osd.285.0:3029 2014-08-12 15:43:45.757452 2014-08-13 18:06:06.777105 7f5c973a57a0 20 read_log 194215'17482 (194215'17481) delete d9544de6/benchmark_data_storage0111-ib.pocstor_41526_object8275/head//24 by client.2174595.0:100759 0.000000 2014-08-13 18:06:06.777116 7f5c973a57a0 20 read_log 194215'17483 (193305'17266) clean 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by osd.285.0:3031 2014-08-12 15:43:45.766117 2014-08-13 18:06:06.777128 7f5c973a57a0 20 read_log 194215'17484 (194215'17483) delete 1b1a4de6/benchmark_data_storage0111-ib.pocstor_62621_object20900/head//24 by client.2174595.0:100761 0.000000 2014-08-13 18:06:06.777154 7f5c973a57a0 20 read_log 194215'17485 (193305'17269) clean 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by osd.285.0:3033 2014-08-12 15:43:45.787496 2014-08-13 18:06:06.777170 7f5c973a57a0 20 read_log 194215'17486 (194215'17485) delete 953f4de6/benchmark_data_storage0111-ib.pocstor_62621_object27077/head//24 by client.2174595.0:100763 0.000000 2014-08-13 18:06:06.777182 7f5c973a57a0 20 read_log 194215'17487 (192834'16291) clean 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by osd.285.0:3035 2014-08-12 15:43:45.796683 2014-08-13 18:06:06.777201 7f5c973a57a0 20 read_log 194215'17488 (194215'17487) delete 7aef4de6/benchmark_data_storage0111-ib.pocstor_26659_object14445/head//24 by client.2174595.0:100765 0.000000 2014-08-13 18:06:06.777214 7f5c973a57a0 20 read_log 194215'17489 (193305'17262) clean bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by osd.285.0:3037 2014-08-12 15:43:45.830400 2014-08-13 18:06:06.777226 7f5c973a57a0 20 read_log 194215'17490 (194215'17489) delete bd3f6de6/benchmark_data_storage0111-ib.pocstor_62621_object14635/head//24 by client.2174595.0:100767 0.000000 2014-08-13 18:06:06.777237 7f5c973a57a0 20 read_log 194215'17491 (192834'16836) clean f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by osd.285.0:3039 2014-08-12 15:43:45.840311 2014-08-13 18:06:06.777250 7f5c973a57a0 20 read_log 194215'17492 (194215'17491) delete f9c17de6/benchmark_data_storage0111-ib.pocstor_41526_object21124/head//24 by client.2174595.0:100769 0.000000 2014-08-13 18:06:06.777260 7f5c973a57a0 20 read_log 194215'17493 (192834'16835) clean 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by osd.285.0:3041 2014-08-12 15:43:45.850390 2014-08-13 18:06:06.777273 7f5c973a57a0 20 read_log 194215'17494 (194215'17493) delete 9ea77de6/benchmark_data_storage0111-ib.pocstor_41526_object21102/head//24 by client.2174595.0:100771 0.000000 2014-08-13 18:06:06.777286 7f5c973a57a0 20 read_log 194215'17495 (192888'17165) clean 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by osd.285.0:3043 2014-08-12 15:43:45.871633 2014-08-13 18:06:06.777299 7f5c973a57a0 20 read_log 194215'17496 (194215'17495) delete 2897de6/benchmark_data_storage0111-ib.pocstor_2228_object11071/head//24 by client.2174595.0:100773 0.000000 2014-08-13 18:06:06.777310 7f5c973a57a0 20 read_log 194215'17497 (192888'17179) clean 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by osd.285.0:3045 2014-08-12 15:43:45.879855 2014-08-13 18:06:06.777322 7f5c973a57a0 20 read_log 194215'17498 (194215'17497) delete 936c7de6/benchmark_data_storage0111-ib.pocstor_2228_object18212/head//24 by client.2174595.0:100775 0.000000 2014-08-13 18:06:06.777337 7f5c973a57a0 20 read_log 194215'17499 (192888'17184) clean 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by osd.285.0:3047 2014-08-12 15:43:45.889702 2014-08-13 18:06:06.777361 7f5c973a57a0 20 read_log 194215'17500 (194215'17499) delete 923e7de6/benchmark_data_storage0111-ib.pocstor_2228_object20245/head//24 by client.2174595.0:100777 0.000000 2014-08-13 18:06:06.777375 7f5c973a57a0 20 read_log 194215'17501 (192834'16724) clean 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by osd.285.0:3049 2014-08-12 15:43:45.898121 2014-08-13 18:06:06.777391 7f5c973a57a0 20 read_log 194215'17502 (194215'17501) delete 64b09de6/benchmark_data_storage0111-ib.pocstor_41526_object4561/head//24 by client.2174595.0:100779 0.000000 2014-08-13 18:06:06.777402 7f5c973a57a0 20 read_log 194215'17503 (192888'17170) clean e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by osd.285.0:3051 2014-08-12 15:43:45.922795 2014-08-13 18:06:06.777415 7f5c973a57a0 20 read_log 194215'17504 (194215'17503) delete e2169de6/benchmark_data_storage0111-ib.pocstor_2228_object12812/head//24 by client.2174595.0:100781 0.000000 2014-08-13 18:06:06.777426 7f5c973a57a0 20 read_log 194215'17505 (192888'17173) clean fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by osd.285.0:3053 2014-08-12 15:43:45.932047 2014-08-13 18:06:06.777444 7f5c973a57a0 20 read_log 194215'17506 (194215'17505) delete fdab9de6/benchmark_data_storage0111-ib.pocstor_2228_object14259/head//24 by client.2174595.0:100783 0.000000 2014-08-13 18:06:06.777455 7f5c973a57a0 20 read_log 194215'17507 (193305'17271) clean b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by osd.285.0:3055 2014-08-12 15:43:45.941086 2014-08-13 18:06:06.777468 7f5c973a57a0 20 read_log 194215'17508 (194215'17507) delete b2cd9de6/benchmark_data_storage0111-ib.pocstor_62621_object31397/head//24 by client.2174595.0:100785 0.000000 2014-08-13 18:06:06.777515 7f5c973a57a0 20 read_log 194215'17509 (192888'17175) clean 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by osd.285.0:3057 2014-08-12 15:43:45.950348 2014-08-13 18:06:06.777529 7f5c973a57a0 20 read_log 194215'17510 (194215'17509) delete 21aeade6/benchmark_data_storage0111-ib.pocstor_2228_object16266/head//24 by client.2174595.0:100787 0.000000 2014-08-13 18:06:06.777540 7f5c973a57a0 20 read_log 194215'17511 (192834'16318) clean 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by osd.285.0:3059 2014-08-12 15:43:45.958667 2014-08-13 18:06:06.777565 7f5c973a57a0 20 read_log 194215'17512 (194215'17511) delete 2cefade6/benchmark_data_storage0111-ib.pocstor_26659_object18431/head//24 by client.2174595.0:100789 0.000000 2014-08-13 18:06:06.777577 7f5c973a57a0 20 read_log 194215'17513 (192888'17169) clean e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by osd.285.0:3061 2014-08-12 15:43:45.968100 2014-08-13 18:06:06.777589 7f5c973a57a0 20 read_log 194215'17514 (194215'17513) delete e002bde6/benchmark_data_storage0111-ib.pocstor_2228_object12784/head//24 by client.2174595.0:100791 0.000000 2014-08-13 18:06:06.777614 7f5c973a57a0 20 read_log 194215'17515 (192834'16793) clean 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by osd.285.0:3063 2014-08-12 15:43:45.977317 2014-08-13 18:06:06.777627 7f5c973a57a0 20 read_log 194215'17516 (194215'17515) delete 2efabde6/benchmark_data_storage0111-ib.pocstor_41526_object13634/head//24 by client.2174595.0:100793 0.000000 2014-08-13 18:06:06.777638 7f5c973a57a0 20 read_log 194215'17517 (192834'16295) clean 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by osd.285.0:3065 2014-08-12 15:43:45.984879 2014-08-13 18:06:06.777651 7f5c973a57a0 20 read_log 194215'17518 (194215'17517) delete 9471cde6/benchmark_data_storage0111-ib.pocstor_26659_object14963/head//24 by client.2174595.0:100795 0.000000 2014-08-13 18:06:06.777661 7f5c973a57a0 20 read_log 194215'17519 (193305'17260) clean e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by osd.285.0:3067 2014-08-12 15:43:46.008737 2014-08-13 18:06:06.777682 7f5c973a57a0 20 read_log 194215'17520 (194215'17519) delete e107cde6/benchmark_data_storage0111-ib.pocstor_62621_object11176/head//24 by client.2174595.0:100797 0.000000 2014-08-13 18:06:06.777696 7f5c973a57a0 20 read_log 194215'17521 (192888'17182) clean e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by osd.285.0:3069 2014-08-12 15:43:46.017163 2014-08-13 18:06:06.777709 7f5c973a57a0 20 read_log 194215'17522 (194215'17521) delete e9e6ede6/benchmark_data_storage0111-ib.pocstor_2228_object19787/head//24 by client.2174595.0:100799 0.000000 2014-08-13 18:06:06.777720 7f5c973a57a0 20 read_log 194215'17523 (193305'17257) clean 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by osd.285.0:3071 2014-08-12 15:43:46.025336 2014-08-13 18:06:06.777732 7f5c973a57a0 20 read_log 194215'17524 (194215'17523) delete 4998ede6/benchmark_data_storage0111-ib.pocstor_62621_object2690/head//24 by client.2174595.0:100801 0.000000 2014-08-13 18:06:06.777743 7f5c973a57a0 20 read_log 194215'17525 (192834'16309) clean 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by osd.285.0:3073 2014-08-12 15:43:46.033855 2014-08-13 18:06:06.777756 7f5c973a57a0 20 read_log 194215'17526 (194215'17525) delete 4439fde6/benchmark_data_storage0111-ib.pocstor_26659_object16919/head//24 by client.2174595.0:100803 0.000000 2014-08-13 18:06:06.777767 7f5c973a57a0 20 read_log 194215'17527 (192888'16970) clean a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by osd.285.0:3075 2014-08-12 15:43:46.042207 2014-08-13 18:06:06.777779 7f5c973a57a0 20 read_log 194215'17528 (194215'17527) delete a549fde6/benchmark_data_storage0111-ib.pocstor_41097_object5298/head//24 by client.2174595.0:100805 0.000000 2014-08-13 18:06:06.777793 7f5c973a57a0 20 read_log 194215'17529 (194215'17529) modify 1e6/hit_set_24.1e6_archive_2014-08-12 15:43:44.587829_2014-08-12 17:24:40.436910/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.437356 2014-08-13 18:06:06.777806 7f5c973a57a0 20 read_log 194215'17530 (194215'17273) delete 1e6/hit_set_24.1e6_archive_2014-08-12 15:32:25.782585_2014-08-12 15:43:44.587553/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.437356 2014-08-13 18:06:06.777818 7f5c973a57a0 20 read_log 194215'17531 (0'0) promote 23a635e6/benchmark_data_storage0111-ib.pocstor_34628_object151/head//24 by osd.285.0:6158 2014-08-12 17:24:40.442940 2014-08-13 18:06:06.777831 7f5c973a57a0 20 read_log 194215'17532 (194215'17531) modify 23a635e6/benchmark_data_storage0111-ib.pocstor_34628_object151/head//24 by client.2193759.0:152 2014-08-12 17:24:40.428426 2014-08-13 18:06:06.777843 7f5c973a57a0 20 read_log 194215'17533 (0'0) promote a6e755e6/benchmark_data_storage0111-ib.pocstor_34628_object861/head//24 by osd.285.0:6191 2014-08-12 17:24:43.239335 2014-08-13 18:06:06.777856 7f5c973a57a0 20 read_log 194215'17534 (194215'17533) modify a6e755e6/benchmark_data_storage0111-ib.pocstor_34628_object861/head//24 by client.2193759.0:862 2014-08-12 17:24:43.232558 2014-08-13 18:06:06.777868 7f5c973a57a0 20 read_log 194215'17535 (0'0) promote 79b31e6/benchmark_data_storage0111-ib.pocstor_34628_object1071/head//24 by osd.285.0:6195 2014-08-12 17:24:44.057625 2014-08-13 18:06:06.777880 7f5c973a57a0 20 read_log 194215'17536 (194215'17535) modify 79b31e6/benchmark_data_storage0111-ib.pocstor_34628_object1071/head//24 by client.2193759.0:1072 2014-08-12 17:24:44.051082 2014-08-13 18:06:06.777893 7f5c973a57a0 20 read_log 194215'17537 (0'0) promote 234a49e6/benchmark_data_storage0111-ib.pocstor_34628_object1143/head//24 by osd.285.0:6197 2014-08-12 17:24:44.330931 2014-08-13 18:06:06.777907 7f5c973a57a0 20 read_log 194215'17538 (194215'17537) modify 234a49e6/benchmark_data_storage0111-ib.pocstor_34628_object1143/head//24 by client.2193759.0:1144 2014-08-12 17:24:44.324387 2014-08-13 18:06:06.777943 7f5c973a57a0 20 read_log 194215'17539 (0'0) promote 739665e6/benchmark_data_storage0111-ib.pocstor_34628_object1445/head//24 by osd.285.0:6207 2014-08-12 17:24:46.551733 2014-08-13 18:06:06.777959 7f5c973a57a0 20 read_log 194215'17540 (194215'17539) modify 739665e6/benchmark_data_storage0111-ib.pocstor_34628_object1445/head//24 by client.2193759.0:1446 2014-08-12 17:24:46.538512 2014-08-13 18:06:06.777971 7f5c973a57a0 20 read_log 194216'17541 (0'0) promote 429831e6/benchmark_data_storage0111-ib.pocstor_39693_object6/head//24 by osd.285.0:6211 2014-08-12 17:25:03.177318 2014-08-13 18:06:06.777986 7f5c973a57a0 20 read_log 194216'17542 (194216'17541) modify 429831e6/benchmark_data_storage0111-ib.pocstor_39693_object6/head//24 by client.1487927.0:7 2014-08-12 17:25:03.160232 2014-08-13 18:06:06.778000 7f5c973a57a0 20 read_log 194370'17543 (0'0) promote 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by osd.285.0:6234 2014-08-12 20:30:17.198376 2014-08-13 18:06:06.778014 7f5c973a57a0 20 read_log 194370'17544 (194370'17543) modify 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by client.1508531.0:743 2014-08-12 20:30:17.192859 2014-08-13 18:06:06.778027 7f5c973a57a0 20 read_log 194370'17545 (0'0) promote 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by osd.285.0:6238 2014-08-12 20:30:17.300018 2014-08-13 18:06:06.778041 7f5c973a57a0 20 read_log 194370'17546 (194370'17545) modify 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by client.1508531.0:771 2014-08-12 20:30:17.293344 2014-08-13 18:06:06.778054 7f5c973a57a0 20 read_log 194370'17547 (0'0) promote 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by osd.285.0:6264 2014-08-12 20:30:21.630458 2014-08-13 18:06:06.778066 7f5c973a57a0 20 read_log 194370'17548 (194370'17547) modify 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by client.1508531.0:1945 2014-08-12 20:30:21.623294 2014-08-13 18:06:06.778078 7f5c973a57a0 20 read_log 194370'17549 (0'0) promote d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by osd.285.0:6294 2014-08-12 20:30:26.872631 2014-08-13 18:06:06.778092 7f5c973a57a0 20 read_log 194370'17550 (194370'17549) modify d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by client.1508531.0:3362 2014-08-12 20:30:26.866144 2014-08-13 18:06:06.778105 7f5c973a57a0 20 read_log 194370'17551 (0'0) promote 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by osd.285.0:6300 2014-08-12 20:30:28.756668 2014-08-13 18:06:06.778119 7f5c973a57a0 20 read_log 194370'17552 (194370'17551) modify 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by client.1508531.0:3852 2014-08-12 20:30:28.748728 2014-08-13 18:06:06.778131 7f5c973a57a0 20 read_log 194370'17553 (0'0) promote 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by osd.285.0:6326 2014-08-12 20:30:33.176959 2014-08-13 18:06:06.778150 7f5c973a57a0 20 read_log 194370'17554 (194370'17553) modify 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by client.1508531.0:4986 2014-08-12 20:30:33.168763 2014-08-13 18:06:06.778166 7f5c973a57a0 20 read_log 194370'17555 (0'0) promote b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by osd.285.0:6334 2014-08-12 20:30:33.999532 2014-08-13 18:06:06.778178 7f5c973a57a0 20 read_log 194370'17556 (194370'17555) modify b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by client.1508531.0:5182 2014-08-12 20:30:33.991823 2014-08-13 18:06:06.778191 7f5c973a57a0 20 read_log 194370'17557 (0'0) promote d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by osd.285.0:6358 2014-08-12 20:30:37.932283 2014-08-13 18:06:06.778213 7f5c973a57a0 20 read_log 194370'17558 (194370'17557) modify d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by client.1508531.0:6043 2014-08-12 20:30:37.924085 2014-08-13 18:06:06.778228 7f5c973a57a0 20 read_log 194370'17559 (0'0) promote 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by osd.285.0:6362 2014-08-12 20:30:39.292374 2014-08-13 18:06:06.778243 7f5c973a57a0 20 read_log 194370'17560 (194370'17559) modify 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by client.1508531.0:6359 2014-08-12 20:30:39.285755 2014-08-13 18:06:06.778256 7f5c973a57a0 20 read_log 194370'17561 (0'0) promote 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by osd.285.0:6364 2014-08-12 20:30:39.937017 2014-08-13 18:06:06.778286 7f5c973a57a0 20 read_log 194370'17562 (194370'17561) modify 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by client.1508531.0:6493 2014-08-12 20:30:39.927587 2014-08-13 18:06:06.778301 7f5c973a57a0 20 read_log 194370'17563 (0'0) promote 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by osd.285.0:6370 2014-08-12 20:30:41.251881 2014-08-13 18:06:06.778317 7f5c973a57a0 20 read_log 194370'17564 (194370'17563) modify 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by client.1508531.0:6777 2014-08-12 20:30:41.243264 2014-08-13 18:06:06.778329 7f5c973a57a0 20 read_log 194370'17565 (0'0) promote 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by osd.285.0:6426 2014-08-12 20:30:54.950682 2014-08-13 18:06:06.778341 7f5c973a57a0 20 read_log 194370'17566 (194370'17565) modify 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by client.1508531.0:9270 2014-08-12 20:30:54.943986 2014-08-13 18:06:06.778356 7f5c973a57a0 20 read_log 194377'17567 (0'0) promote baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by osd.285.0:6542 2014-08-12 20:33:00.374306 2014-08-13 18:06:06.778369 7f5c973a57a0 20 read_log 194377'17568 (194377'17567) modify baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by client.2231190.0:750 2014-08-12 20:33:00.363477 2014-08-13 18:06:06.778382 7f5c973a57a0 20 read_log 194377'17569 (0'0) promote 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by osd.285.0:6552 2014-08-12 20:33:02.432625 2014-08-13 18:06:06.778394 7f5c973a57a0 20 read_log 194377'17570 (194377'17569) modify 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by client.2231190.0:1297 2014-08-12 20:33:02.426220 2014-08-13 18:06:06.778410 7f5c973a57a0 20 read_log 194377'17571 (0'0) promote 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by osd.285.0:6572 2014-08-12 20:33:06.347750 2014-08-13 18:06:06.778423 7f5c973a57a0 20 read_log 194377'17572 (194377'17571) modify 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by client.2231190.0:2270 2014-08-12 20:33:06.340532 2014-08-13 18:06:06.778435 7f5c973a57a0 20 read_log 194380'17573 (0'0) promote 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by osd.285.0:6638 2014-08-12 20:33:22.396527 2014-08-13 18:06:06.778447 7f5c973a57a0 20 read_log 194380'17574 (194380'17573) modify 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by client.2231190.0:4956 2014-08-12 20:33:22.387767 2014-08-13 18:06:06.778463 7f5c973a57a0 20 read_log 194380'17575 (0'0) promote 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by osd.285.0:6650 2014-08-12 20:33:26.161972 2014-08-13 18:06:06.778475 7f5c973a57a0 20 read_log 194380'17576 (194380'17575) modify 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by client.2231190.0:5227 2014-08-12 20:33:26.151470 2014-08-13 18:06:06.778495 7f5c973a57a0 20 read_log 194383'17577 (0'0) promote e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by osd.285.0:6662 2014-08-12 20:34:01.239061 2014-08-13 18:06:06.778508 7f5c973a57a0 20 read_log 194383'17578 (194383'17577) modify e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by client.2231190.0:5965 2014-08-12 20:34:01.231283 2014-08-13 18:06:06.778530 7f5c973a57a0 20 read_log 194383'17579 (0'0) promote a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by osd.285.0:6678 2014-08-12 20:34:08.806925 2014-08-13 18:06:06.778543 7f5c973a57a0 20 read_log 194383'17580 (194383'17579) modify a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by client.2231190.0:6767 2014-08-12 20:34:08.797961 2014-08-13 18:06:06.778556 7f5c973a57a0 20 read_log 194383'17581 (0'0) promote 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by osd.285.0:6694 2014-08-12 20:34:32.456987 2014-08-13 18:06:06.778575 7f5c973a57a0 20 read_log 194383'17582 (194383'17581) modify 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by client.2231190.0:7636 2014-08-12 20:34:32.449118 2014-08-13 18:06:06.778589 7f5c973a57a0 20 read_log 194393'17583 (0'0) promote c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by osd.285.0:6733 2014-08-12 20:49:26.428594 2014-08-13 18:06:06.778601 7f5c973a57a0 20 read_log 194393'17584 (194393'17583) modify c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by client.1064182.0:200 2014-08-12 20:49:26.420766 2014-08-13 18:06:06.778624 7f5c973a57a0 20 read_log 194393'17585 (0'0) promote 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by osd.285.0:6742 2014-08-12 20:49:27.680263 2014-08-13 18:06:06.778643 7f5c973a57a0 20 read_log 194393'17586 (194393'17585) modify 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by client.1064182.0:434 2014-08-12 20:49:27.672142 2014-08-13 18:06:06.778656 7f5c973a57a0 20 read_log 194393'17587 (0'0) promote 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by osd.285.0:6752 2014-08-12 20:52:33.478035 2014-08-13 18:06:06.778674 7f5c973a57a0 20 read_log 194393'17588 (194393'17587) modify 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by client.1064422.0:233 2014-08-12 20:52:33.470274 2014-08-13 18:06:06.778687 7f5c973a57a0 20 read_log 194416'17589 (0'0) promote 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by osd.285.0:6881 2014-08-13 15:28:29.954653 2014-08-13 18:06:06.778699 7f5c973a57a0 20 read_log 194416'17590 (194416'17589) modify 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by client.2441958.0:4 2014-08-13 15:28:29.919716 2014-08-13 18:06:06.778712 7f5c973a57a0 20 read_log 194416'17591 (0'0) promote 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by osd.285.0:6898 2014-08-13 15:28:34.290031 2014-08-13 18:06:06.778724 7f5c973a57a0 20 read_log 194416'17592 (194416'17591) modify 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by client.2441958.0:945 2014-08-13 15:28:34.269526 2014-08-13 18:06:06.778737 7f5c973a57a0 20 read_log 194416'17593 (0'0) promote 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by osd.285.0:6942 2014-08-13 15:28:41.002911 2014-08-13 18:06:06.778749 7f5c973a57a0 20 read_log 194416'17594 (194416'17593) modify 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by client.2441958.0:2714 2014-08-13 15:28:40.995272 2014-08-13 18:06:06.778762 7f5c973a57a0 20 read_log 194416'17595 (0'0) promote 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by osd.285.0:6952 2014-08-13 15:28:41.670103 2014-08-13 18:06:06.778790 7f5c973a57a0 20 read_log 194416'17596 (194416'17595) modify 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by client.2441958.0:2891 2014-08-13 15:28:41.661418 2014-08-13 18:06:06.778807 7f5c973a57a0 20 read_log 194416'17597 (0'0) promote 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by osd.285.0:6986 2014-08-13 15:28:48.780468 2014-08-13 18:06:06.778826 7f5c973a57a0 20 read_log 194416'17598 (194416'17597) modify 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by client.2441958.0:4705 2014-08-13 15:28:48.770243 2014-08-13 18:06:06.778842 7f5c973a57a0 20 read_log 194416'17599 (0'0) promote ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by osd.285.0:7006 2014-08-13 15:28:51.781837 2014-08-13 18:06:06.778854 7f5c973a57a0 20 read_log 194416'17600 (194416'17599) modify ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by client.2441958.0:5497 2014-08-13 15:28:51.775367 2014-08-13 18:06:06.778870 7f5c973a57a0 20 read_log 194416'17601 (0'0) promote c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by osd.285.0:7010 2014-08-13 15:28:52.158566 2014-08-13 18:06:06.778892 7f5c973a57a0 20 read_log 194416'17602 (194416'17601) modify c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by client.2441958.0:5594 2014-08-13 15:28:52.148545 2014-08-13 18:06:06.778908 7f5c973a57a0 20 read_log 194416'17603 (0'0) promote 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by osd.285.0:7024 2014-08-13 15:28:56.470640 2014-08-13 18:06:06.778924 7f5c973a57a0 20 read_log 194416'17604 (194416'17603) modify 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by client.2441958.0:6648 2014-08-13 15:28:56.462954 2014-08-13 18:06:06.778936 7f5c973a57a0 20 read_log 194416'17605 (0'0) promote 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by osd.285.0:7034 2014-08-13 15:28:59.166125 2014-08-13 18:06:06.778948 7f5c973a57a0 20 read_log 194416'17606 (194416'17605) modify 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by client.2441958.0:7371 2014-08-13 15:28:59.157077 2014-08-13 18:06:06.778961 7f5c973a57a0 20 read_log 194416'17607 (0'0) promote 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by osd.285.0:7070 2014-08-13 15:29:05.568880 2014-08-13 18:06:06.778981 7f5c973a57a0 20 read_log 194416'17608 (194416'17607) modify 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by client.2441958.0:9022 2014-08-13 15:29:05.558364 2014-08-13 18:06:06.778995 7f5c973a57a0 20 read_log 194416'17609 (0'0) promote 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by osd.285.0:7096 2014-08-13 15:29:07.899630 2014-08-13 18:06:06.779007 7f5c973a57a0 20 read_log 194416'17610 (194416'17609) modify 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by client.2441958.0:9635 2014-08-13 15:29:07.892021 2014-08-13 18:06:06.779022 7f5c973a57a0 20 read_log 194416'17611 (0'0) promote 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by osd.285.0:7098 2014-08-13 15:29:08.249448 2014-08-13 18:06:06.779035 7f5c973a57a0 20 read_log 194416'17612 (194416'17611) modify 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by client.2441958.0:9727 2014-08-13 15:29:08.241468 2014-08-13 18:06:06.779047 7f5c973a57a0 20 read_log 194416'17613 (0'0) promote c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by osd.285.0:7114 2014-08-13 15:29:11.345668 2014-08-13 18:06:06.779060 7f5c973a57a0 20 read_log 194416'17614 (194416'17613) modify c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by client.2441958.0:10475 2014-08-13 15:29:11.336396 2014-08-13 18:06:06.779075 7f5c973a57a0 20 read_log 194416'17615 (0'0) promote 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by osd.285.0:7136 2014-08-13 15:29:16.427040 2014-08-13 18:06:06.779104 7f5c973a57a0 20 read_log 194416'17616 (194416'17615) modify 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by client.2441958.0:11725 2014-08-13 15:29:16.417986 2014-08-13 18:06:06.779121 7f5c973a57a0 20 read_log 194416'17617 (0'0) promote d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by osd.285.0:7146 2014-08-13 15:29:17.758852 2014-08-13 18:06:06.779136 7f5c973a57a0 20 read_log 194416'17618 (194416'17617) modify d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by client.2441958.0:12034 2014-08-13 15:29:17.749689 2014-08-13 18:06:06.779170 7f5c973a57a0 20 read_log 194416'17619 (0'0) promote 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by osd.285.0:7154 2014-08-13 15:29:19.103144 2014-08-13 18:06:06.779193 7f5c973a57a0 20 read_log 194416'17620 (194416'17619) modify 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by client.2441958.0:12344 2014-08-13 15:29:19.093009 2014-08-13 18:06:06.779214 7f5c973a57a0 20 read_log 194416'17621 (0'0) promote 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by osd.285.0:7252 2014-08-13 15:29:41.736838 2014-08-13 18:06:06.779229 7f5c973a57a0 20 read_log 194416'17622 (194416'17621) modify 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by client.2441958.0:16606 2014-08-13 15:29:41.727382 2014-08-13 18:06:06.779241 7f5c973a57a0 20 read_log 194416'17623 (0'0) promote 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by osd.285.0:7254 2014-08-13 15:29:42.050800 2014-08-13 18:06:06.779254 7f5c973a57a0 20 read_log 194416'17624 (194416'17623) modify 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by client.2441958.0:16630 2014-08-13 15:29:42.041487 2014-08-13 18:06:06.779272 7f5c973a57a0 20 read_log 194416'17625 (0'0) promote a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by osd.285.0:7276 2014-08-13 15:29:53.367892 2014-08-13 18:06:06.779284 7f5c973a57a0 20 read_log 194416'17626 (194416'17625) modify a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by client.2441958.0:17425 2014-08-13 15:29:53.358926 2014-08-13 18:06:06.779297 7f5c973a57a0 20 read_log 194416'17627 (0'0) promote 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by osd.285.0:7284 2014-08-13 15:29:58.352010 2014-08-13 18:06:06.779324 7f5c973a57a0 20 read_log 194416'17628 (194416'17627) modify 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by client.2441958.0:17781 2014-08-13 15:29:58.341408 2014-08-13 18:06:06.779337 7f5c973a57a0 20 read_log 194422'17629 (0'0) promote dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by osd.285.0:7336 2014-08-13 15:32:20.791483 2014-08-13 18:06:06.779350 7f5c973a57a0 20 read_log 194422'17630 (194422'17629) modify dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by client.1148203.0:1378 2014-08-13 15:32:20.782756 2014-08-13 18:06:06.779365 7f5c973a57a0 20 read_log 194422'17631 (0'0) promote ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by osd.285.0:7386 2014-08-13 15:32:29.962551 2014-08-13 18:06:06.779380 7f5c973a57a0 20 read_log 194422'17632 (194422'17631) modify ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by client.1148203.0:3725 2014-08-13 15:32:29.953694 2014-08-13 18:06:06.779393 7f5c973a57a0 20 read_log 194422'17633 (0'0) promote a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by osd.285.0:7406 2014-08-13 15:32:34.472931 2014-08-13 18:06:06.779408 7f5c973a57a0 20 read_log 194422'17634 (194422'17633) modify a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by client.1148203.0:4904 2014-08-13 15:32:34.465187 2014-08-13 18:06:06.779438 7f5c973a57a0 20 read_log 194422'17635 (0'0) promote 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by osd.285.0:7458 2014-08-13 15:32:46.004223 2014-08-13 18:06:06.779458 7f5c973a57a0 20 read_log 194422'17636 (194422'17635) modify 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by client.1148203.0:7076 2014-08-13 15:32:45.995195 2014-08-13 18:06:06.779473 7f5c973a57a0 20 read_log 194422'17637 (0'0) promote 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by osd.285.0:7460 2014-08-13 15:32:46.067143 2014-08-13 18:06:06.779486 7f5c973a57a0 20 read_log 194422'17638 (194422'17637) modify 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by client.1148203.0:7087 2014-08-13 15:32:46.058576 2014-08-13 18:06:06.779499 7f5c973a57a0 20 read_log 194422'17639 (0'0) promote fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by osd.285.0:7495 2014-08-13 15:32:55.403501 2014-08-13 18:06:06.779513 7f5c973a57a0 20 read_log 194422'17640 (194422'17639) modify fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by client.1148203.0:8541 2014-08-13 15:32:55.392841 2014-08-13 18:06:06.779526 7f5c973a57a0 20 read_log 194422'17641 (0'0) promote 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by osd.285.0:7517 2014-08-13 15:33:02.095563 2014-08-13 18:06:06.779549 7f5c973a57a0 20 read_log 194422'17642 (194422'17641) modify 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by client.1148203.0:9258 2014-08-13 15:33:02.084968 2014-08-13 18:06:06.779567 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.779935 7f5c973a57a0 10 read_log checking for missing items over interval (194370'17543,194422'17642] 2014-08-13 18:06:06.779984 7f5c973a57a0 15 read_log missing 194422'17642 (194422'17641) modify 598eb1e6/benchmark_data_storage0111-ib.pocstor_41231_object9257/head//24 by client.1148203.0:9258 2014-08-13 15:33:02.084968 2014-08-13 18:06:06.780025 7f5c973a57a0 15 read_log missing 194422'17640 (194422'17639) modify fc4b21e6/benchmark_data_storage0111-ib.pocstor_41231_object8540/head//24 by client.1148203.0:8541 2014-08-13 15:32:55.392841 2014-08-13 18:06:06.780061 7f5c973a57a0 15 read_log missing 194422'17638 (194422'17637) modify 9fe805e6/benchmark_data_storage0111-ib.pocstor_41231_object7086/head//24 by client.1148203.0:7087 2014-08-13 15:32:46.058576 2014-08-13 18:06:06.780097 7f5c973a57a0 15 read_log missing 194422'17636 (194422'17635) modify 5a5475e6/benchmark_data_storage0111-ib.pocstor_41231_object7075/head//24 by client.1148203.0:7076 2014-08-13 15:32:45.995195 2014-08-13 18:06:06.780132 7f5c973a57a0 15 read_log missing 194422'17634 (194422'17633) modify a22b09e6/benchmark_data_storage0111-ib.pocstor_41231_object4903/head//24 by client.1148203.0:4904 2014-08-13 15:32:34.465187 2014-08-13 18:06:06.780173 7f5c973a57a0 15 read_log missing 194422'17632 (194422'17631) modify ac20fde6/benchmark_data_storage0111-ib.pocstor_41231_object3724/head//24 by client.1148203.0:3725 2014-08-13 15:32:29.953694 2014-08-13 18:06:06.780213 7f5c973a57a0 15 read_log missing 194422'17630 (194422'17629) modify dd351e6/benchmark_data_storage0111-ib.pocstor_41231_object1377/head//24 by client.1148203.0:1378 2014-08-13 15:32:20.782756 2014-08-13 18:06:06.780250 7f5c973a57a0 15 read_log missing 194416'17628 (194416'17627) modify 6e9cdde6/benchmark_data_storage0111-ib.pocstor_19103_object17780/head//24 by client.2441958.0:17781 2014-08-13 15:29:58.341408 2014-08-13 18:06:06.780291 7f5c973a57a0 15 read_log missing 194416'17626 (194416'17625) modify a0d799e6/benchmark_data_storage0111-ib.pocstor_19103_object17424/head//24 by client.2441958.0:17425 2014-08-13 15:29:53.358926 2014-08-13 18:06:06.780327 7f5c973a57a0 15 read_log missing 194416'17624 (194416'17623) modify 9cb755e6/benchmark_data_storage0111-ib.pocstor_19103_object16629/head//24 by client.2441958.0:16630 2014-08-13 15:29:42.041487 2014-08-13 18:06:06.780374 7f5c973a57a0 15 read_log missing 194416'17622 (194416'17621) modify 89470de6/benchmark_data_storage0111-ib.pocstor_19103_object16605/head//24 by client.2441958.0:16606 2014-08-13 15:29:41.727382 2014-08-13 18:06:06.780438 7f5c973a57a0 15 read_log missing 194416'17620 (194416'17619) modify 27c3fde6/benchmark_data_storage0111-ib.pocstor_19103_object12343/head//24 by client.2441958.0:12344 2014-08-13 15:29:19.093009 2014-08-13 18:06:06.780479 7f5c973a57a0 15 read_log missing 194416'17618 (194416'17617) modify d22e2de6/benchmark_data_storage0111-ib.pocstor_19103_object12033/head//24 by client.2441958.0:12034 2014-08-13 15:29:17.749689 2014-08-13 18:06:06.780515 7f5c973a57a0 15 read_log missing 194416'17616 (194416'17615) modify 9bc891e6/benchmark_data_storage0111-ib.pocstor_19103_object11724/head//24 by client.2441958.0:11725 2014-08-13 15:29:16.417986 2014-08-13 18:06:06.780551 7f5c973a57a0 15 read_log missing 194416'17614 (194416'17613) modify c29f45e6/benchmark_data_storage0111-ib.pocstor_19103_object10474/head//24 by client.2441958.0:10475 2014-08-13 15:29:11.336396 2014-08-13 18:06:06.780586 7f5c973a57a0 15 read_log missing 194416'17612 (194416'17611) modify 6799fde6/benchmark_data_storage0111-ib.pocstor_19103_object9726/head//24 by client.2441958.0:9727 2014-08-13 15:29:08.241468 2014-08-13 18:06:06.780621 7f5c973a57a0 15 read_log missing 194416'17610 (194416'17609) modify 39859de6/benchmark_data_storage0111-ib.pocstor_19103_object9634/head//24 by client.2441958.0:9635 2014-08-13 15:29:07.892021 2014-08-13 18:06:06.780656 7f5c973a57a0 15 read_log missing 194416'17608 (194416'17607) modify 2d4045e6/benchmark_data_storage0111-ib.pocstor_19103_object9021/head//24 by client.2441958.0:9022 2014-08-13 15:29:05.558364 2014-08-13 18:06:06.780692 7f5c973a57a0 15 read_log missing 194416'17606 (194416'17605) modify 1b83fde6/benchmark_data_storage0111-ib.pocstor_19103_object7370/head//24 by client.2441958.0:7371 2014-08-13 15:28:59.157077 2014-08-13 18:06:06.780727 7f5c973a57a0 15 read_log missing 194416'17604 (194416'17603) modify 27e3b5e6/benchmark_data_storage0111-ib.pocstor_19103_object6647/head//24 by client.2441958.0:6648 2014-08-13 15:28:56.462954 2014-08-13 18:06:06.780763 7f5c973a57a0 15 read_log missing 194416'17602 (194416'17601) modify c02831e6/benchmark_data_storage0111-ib.pocstor_19103_object5593/head//24 by client.2441958.0:5594 2014-08-13 15:28:52.148545 2014-08-13 18:06:06.780819 7f5c973a57a0 15 read_log missing 194416'17600 (194416'17599) modify ec4745e6/benchmark_data_storage0111-ib.pocstor_19103_object5496/head//24 by client.2441958.0:5497 2014-08-13 15:28:51.775367 2014-08-13 18:06:06.780858 7f5c973a57a0 15 read_log missing 194416'17598 (194416'17597) modify 1856a1e6/benchmark_data_storage0111-ib.pocstor_19103_object4704/head//24 by client.2441958.0:4705 2014-08-13 15:28:48.770243 2014-08-13 18:06:06.780894 7f5c973a57a0 15 read_log missing 194416'17596 (194416'17595) modify 240121e6/benchmark_data_storage0111-ib.pocstor_19103_object2890/head//24 by client.2441958.0:2891 2014-08-13 15:28:41.661418 2014-08-13 18:06:06.780930 7f5c973a57a0 15 read_log missing 194416'17594 (194416'17593) modify 2b3bd1e6/benchmark_data_storage0111-ib.pocstor_19103_object2713/head//24 by client.2441958.0:2714 2014-08-13 15:28:40.995272 2014-08-13 18:06:06.780966 7f5c973a57a0 15 read_log missing 194416'17592 (194416'17591) modify 39bb1de6/benchmark_data_storage0111-ib.pocstor_19103_object944/head//24 by client.2441958.0:945 2014-08-13 15:28:34.269526 2014-08-13 18:06:06.781002 7f5c973a57a0 15 read_log missing 194416'17590 (194416'17589) modify 5ee1d5e6/benchmark_data_storage0111-ib.pocstor_19103_object3/head//24 by client.2441958.0:4 2014-08-13 15:28:29.919716 2014-08-13 18:06:06.781038 7f5c973a57a0 15 read_log missing 194393'17588 (194393'17587) modify 7b394de6/benchmark_data_storage0111-ib.pocstor_48304_object232/head//24 by client.1064422.0:233 2014-08-12 20:52:33.470274 2014-08-13 18:06:06.781074 7f5c973a57a0 15 read_log missing 194393'17586 (194393'17585) modify 897b35e6/benchmark_data_storage0111-ib.pocstor_32482_object433/head//24 by client.1064182.0:434 2014-08-12 20:49:27.672142 2014-08-13 18:06:06.781110 7f5c973a57a0 15 read_log missing 194393'17584 (194393'17583) modify c41b81e6/benchmark_data_storage0111-ib.pocstor_32482_object199/head//24 by client.1064182.0:200 2014-08-12 20:49:26.420766 2014-08-13 18:06:06.781150 7f5c973a57a0 15 read_log missing 194383'17582 (194383'17581) modify 6dab85e6/benchmark_data_storage0111-ib.pocstor_25065_object7635/head//24 by client.2231190.0:7636 2014-08-12 20:34:32.449118 2014-08-13 18:06:06.781188 7f5c973a57a0 15 read_log missing 194383'17580 (194383'17579) modify a7c5f1e6/benchmark_data_storage0111-ib.pocstor_25065_object6766/head//24 by client.2231190.0:6767 2014-08-12 20:34:08.797961 2014-08-13 18:06:06.781225 7f5c973a57a0 15 read_log missing 194383'17578 (194383'17577) modify e55de1e6/benchmark_data_storage0111-ib.pocstor_25065_object5964/head//24 by client.2231190.0:5965 2014-08-12 20:34:01.231283 2014-08-13 18:06:06.781264 7f5c973a57a0 15 read_log missing 194380'17576 (194380'17575) modify 6526ade6/benchmark_data_storage0111-ib.pocstor_25065_object5226/head//24 by client.2231190.0:5227 2014-08-12 20:33:26.151470 2014-08-13 18:06:06.781300 7f5c973a57a0 15 read_log missing 194380'17574 (194380'17573) modify 299c05e6/benchmark_data_storage0111-ib.pocstor_25065_object4955/head//24 by client.2231190.0:4956 2014-08-12 20:33:22.387767 2014-08-13 18:06:06.781342 7f5c973a57a0 15 read_log missing 194377'17572 (194377'17571) modify 86c6d1e6/benchmark_data_storage0111-ib.pocstor_25065_object2269/head//24 by client.2231190.0:2270 2014-08-12 20:33:06.340532 2014-08-13 18:06:06.781393 7f5c973a57a0 15 read_log missing 194377'17570 (194377'17569) modify 3b3c31e6/benchmark_data_storage0111-ib.pocstor_25065_object1296/head//24 by client.2231190.0:1297 2014-08-12 20:33:02.426220 2014-08-13 18:06:06.781475 7f5c973a57a0 15 read_log missing 194377'17568 (194377'17567) modify baa0c1e6/benchmark_data_storage0111-ib.pocstor_25065_object749/head//24 by client.2231190.0:750 2014-08-12 20:33:00.363477 2014-08-13 18:06:06.781523 7f5c973a57a0 15 read_log missing 194370'17566 (194370'17565) modify 3e6179e6/benchmark_data_storage0111-ib.pocstor_12495_object9269/head//24 by client.1508531.0:9270 2014-08-12 20:30:54.943986 2014-08-13 18:06:06.781560 7f5c973a57a0 15 read_log missing 194370'17564 (194370'17563) modify 542745e6/benchmark_data_storage0111-ib.pocstor_12495_object6776/head//24 by client.1508531.0:6777 2014-08-12 20:30:41.243264 2014-08-13 18:06:06.781596 7f5c973a57a0 15 read_log missing 194370'17562 (194370'17561) modify 9c0e55e6/benchmark_data_storage0111-ib.pocstor_12495_object6492/head//24 by client.1508531.0:6493 2014-08-12 20:30:39.927587 2014-08-13 18:06:06.781632 7f5c973a57a0 15 read_log missing 194370'17560 (194370'17559) modify 83d235e6/benchmark_data_storage0111-ib.pocstor_12495_object6358/head//24 by client.1508531.0:6359 2014-08-12 20:30:39.285755 2014-08-13 18:06:06.781668 7f5c973a57a0 15 read_log missing 194370'17558 (194370'17557) modify d51a5e6/benchmark_data_storage0111-ib.pocstor_12495_object6042/head//24 by client.1508531.0:6043 2014-08-12 20:30:37.924085 2014-08-13 18:06:06.781704 7f5c973a57a0 15 read_log missing 194370'17556 (194370'17555) modify b7a881e6/benchmark_data_storage0111-ib.pocstor_12495_object5181/head//24 by client.1508531.0:5182 2014-08-12 20:30:33.991823 2014-08-13 18:06:06.781740 7f5c973a57a0 15 read_log missing 194370'17554 (194370'17553) modify 5c869e6/benchmark_data_storage0111-ib.pocstor_12495_object4985/head//24 by client.1508531.0:4986 2014-08-12 20:30:33.168763 2014-08-13 18:06:06.781776 7f5c973a57a0 15 read_log missing 194370'17552 (194370'17551) modify 4972f9e6/benchmark_data_storage0111-ib.pocstor_12495_object3851/head//24 by client.1508531.0:3852 2014-08-12 20:30:28.748728 2014-08-13 18:06:06.781812 7f5c973a57a0 15 read_log missing 194370'17550 (194370'17549) modify d70719e6/benchmark_data_storage0111-ib.pocstor_12495_object3361/head//24 by client.1508531.0:3362 2014-08-12 20:30:26.866144 2014-08-13 18:06:06.781847 7f5c973a57a0 15 read_log missing 194370'17548 (194370'17547) modify 161f45e6/benchmark_data_storage0111-ib.pocstor_12495_object1944/head//24 by client.1508531.0:1945 2014-08-12 20:30:21.623294 2014-08-13 18:06:06.781883 7f5c973a57a0 15 read_log missing 194370'17546 (194370'17545) modify 6e7c4de6/benchmark_data_storage0111-ib.pocstor_12495_object770/head//24 by client.1508531.0:771 2014-08-12 20:30:17.293344 2014-08-13 18:06:06.781918 7f5c973a57a0 15 read_log missing 194370'17544 (194370'17543) modify 17c859e6/benchmark_data_storage0111-ib.pocstor_12495_object742/head//24 by client.1508531.0:743 2014-08-12 20:30:17.192859 2014-08-13 18:06:06.781929 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.781955 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive m=50] handle_loaded 2014-08-13 18:06:06.781972 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.014501 0 0.000000 2014-08-13 18:06:06.781985 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 18:06:06.781994 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=0 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194487 2014-08-13 18:06:06.782003 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log((192718'14692,194422'17642], crt=194422'17642) 2014-08-13 18:06:06.782015 7f5c973a57a0 10 osd.4 194487 pgid 24.1f7 coll 24.1f7_head 2014-08-13 18:06:06.782157 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.1f7 2014-08-13 18:06:06.782170 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.782214 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1f7(unlocked)] enter Initial 2014-08-13 18:06:06.782226 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1f7(unlocked)] enter NotTrimming 2014-08-13 18:06:06.782449 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.782602 7f5c973a57a0 20 read_log 192718'12873 (192718'12872) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:3 2014-08-06 15:24:47.114884 2014-08-13 18:06:06.782635 7f5c973a57a0 20 read_log 192718'12874 (192718'12873) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:4 2014-08-06 15:24:47.424123 2014-08-13 18:06:06.782651 7f5c973a57a0 20 read_log 192718'12875 (192718'12874) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:5 2014-08-06 15:24:47.595699 2014-08-13 18:06:06.782664 7f5c973a57a0 20 read_log 192718'12876 (192718'12875) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:6 2014-08-06 15:24:48.028351 2014-08-13 18:06:06.782677 7f5c973a57a0 20 read_log 192718'12877 (192718'12876) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:7 2014-08-06 15:24:48.614236 2014-08-13 18:06:06.782689 7f5c973a57a0 20 read_log 192718'12878 (192718'12877) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:8 2014-08-06 15:24:49.060913 2014-08-13 18:06:06.782710 7f5c973a57a0 20 read_log 192718'12879 (192718'12878) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:9 2014-08-06 15:24:49.507639 2014-08-13 18:06:06.782749 7f5c973a57a0 20 read_log 192718'12880 (192718'12879) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:10 2014-08-06 15:24:50.158398 2014-08-13 18:06:06.782768 7f5c973a57a0 20 read_log 192718'12881 (192718'12880) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:11 2014-08-06 15:24:50.421065 2014-08-13 18:06:06.782783 7f5c973a57a0 20 read_log 192718'12882 (192718'12881) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:12 2014-08-06 15:24:50.657631 2014-08-13 18:06:06.782795 7f5c973a57a0 20 read_log 192718'12883 (192718'12882) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:13 2014-08-06 15:24:50.836072 2014-08-13 18:06:06.782807 7f5c973a57a0 20 read_log 192718'12884 (192718'12883) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:14 2014-08-06 15:24:50.900548 2014-08-13 18:06:06.782819 7f5c973a57a0 20 read_log 192718'12885 (192718'12884) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:15 2014-08-06 15:24:50.929959 2014-08-13 18:06:06.782832 7f5c973a57a0 20 read_log 192718'12886 (192718'12885) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:16 2014-08-06 15:24:50.960414 2014-08-13 18:06:06.782844 7f5c973a57a0 20 read_log 192718'12887 (192718'12886) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:17 2014-08-06 15:24:50.985093 2014-08-13 18:06:06.782856 7f5c973a57a0 20 read_log 192718'12888 (192718'12887) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:18 2014-08-06 15:24:51.010332 2014-08-13 18:06:06.782868 7f5c973a57a0 20 read_log 192718'12889 (192718'12888) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:19 2014-08-06 15:24:51.036022 2014-08-13 18:06:06.782881 7f5c973a57a0 20 read_log 192718'12890 (192718'12889) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:20 2014-08-06 15:24:51.060228 2014-08-13 18:06:06.782904 7f5c973a57a0 20 read_log 192718'12891 (192718'12890) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:21 2014-08-06 15:24:51.084232 2014-08-13 18:06:06.782917 7f5c973a57a0 20 read_log 192718'12892 (192718'12891) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:22 2014-08-06 15:24:51.108502 2014-08-13 18:06:06.782932 7f5c973a57a0 20 read_log 192718'12893 (192718'12892) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:23 2014-08-06 15:24:51.133094 2014-08-13 18:06:06.782944 7f5c973a57a0 20 read_log 192718'12894 (192718'12893) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:24 2014-08-06 15:24:51.156710 2014-08-13 18:06:06.782956 7f5c973a57a0 20 read_log 192718'12895 (192718'12894) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:25 2014-08-06 15:24:51.230774 2014-08-13 18:06:06.782972 7f5c973a57a0 20 read_log 192718'12896 (192718'12895) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:26 2014-08-06 15:24:51.255363 2014-08-13 18:06:06.782984 7f5c973a57a0 20 read_log 192718'12897 (192718'12896) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:27 2014-08-06 15:24:51.279009 2014-08-13 18:06:06.782996 7f5c973a57a0 20 read_log 192718'12898 (192718'12897) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:28 2014-08-06 15:24:51.302621 2014-08-13 18:06:06.783008 7f5c973a57a0 20 read_log 192718'12899 (192718'12898) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:29 2014-08-06 15:24:51.326614 2014-08-13 18:06:06.783021 7f5c973a57a0 20 read_log 192718'12900 (192718'12899) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:30 2014-08-06 15:24:51.350464 2014-08-13 18:06:06.783033 7f5c973a57a0 20 read_log 192718'12901 (192718'12900) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:31 2014-08-06 15:24:51.389035 2014-08-13 18:06:06.783045 7f5c973a57a0 20 read_log 192718'12902 (192718'12901) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:32 2014-08-06 15:24:51.414094 2014-08-13 18:06:06.783067 7f5c973a57a0 20 read_log 192718'12903 (192718'12902) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:33 2014-08-06 15:24:51.437762 2014-08-13 18:06:06.783080 7f5c973a57a0 20 read_log 192718'12904 (192718'12903) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:34 2014-08-06 15:24:51.462561 2014-08-13 18:06:06.783092 7f5c973a57a0 20 read_log 192718'12905 (192718'12904) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:35 2014-08-06 15:24:51.573919 2014-08-13 18:06:06.783104 7f5c973a57a0 20 read_log 192718'12906 (192718'12905) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:36 2014-08-06 15:24:51.599477 2014-08-13 18:06:06.783117 7f5c973a57a0 20 read_log 192718'12907 (192718'12906) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:37 2014-08-06 15:24:51.643384 2014-08-13 18:06:06.783129 7f5c973a57a0 20 read_log 192718'12908 (192718'12907) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:38 2014-08-06 15:24:51.687272 2014-08-13 18:06:06.783141 7f5c973a57a0 20 read_log 192718'12909 (192718'12908) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:39 2014-08-06 15:24:51.731104 2014-08-13 18:06:06.783163 7f5c973a57a0 20 read_log 192718'12910 (192718'12909) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:40 2014-08-06 15:24:51.773411 2014-08-13 18:06:06.783176 7f5c973a57a0 20 read_log 192718'12911 (192718'12910) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:41 2014-08-06 15:24:51.818387 2014-08-13 18:06:06.783188 7f5c973a57a0 20 read_log 192718'12912 (192718'12911) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:42 2014-08-06 15:24:51.862979 2014-08-13 18:06:06.783200 7f5c973a57a0 20 read_log 192718'12913 (192718'12912) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:43 2014-08-06 15:24:51.907051 2014-08-13 18:06:06.783214 7f5c973a57a0 20 read_log 192718'12914 (192718'12913) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:44 2014-08-06 15:24:52.052791 2014-08-13 18:06:06.783226 7f5c973a57a0 20 read_log 192718'12915 (192718'12914) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:45 2014-08-06 15:24:52.104726 2014-08-13 18:06:06.783238 7f5c973a57a0 20 read_log 192718'12916 (192718'12915) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:46 2014-08-06 15:24:52.156812 2014-08-13 18:06:06.783250 7f5c973a57a0 20 read_log 192718'12917 (192718'12916) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:47 2014-08-06 15:24:52.212763 2014-08-13 18:06:06.783262 7f5c973a57a0 20 read_log 192718'12918 (192718'12917) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:48 2014-08-06 15:24:52.261099 2014-08-13 18:06:06.783275 7f5c973a57a0 20 read_log 192718'12919 (192718'12918) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:49 2014-08-06 15:24:52.312303 2014-08-13 18:06:06.783287 7f5c973a57a0 20 read_log 192718'12920 (192718'12919) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:50 2014-08-06 15:24:52.354953 2014-08-13 18:06:06.783302 7f5c973a57a0 20 read_log 192718'12921 (192718'12920) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:51 2014-08-06 15:24:52.407051 2014-08-13 18:06:06.783314 7f5c973a57a0 20 read_log 192718'12922 (192718'12921) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:52 2014-08-06 15:24:52.450846 2014-08-13 18:06:06.783326 7f5c973a57a0 20 read_log 192718'12923 (192718'12922) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:53 2014-08-06 15:24:52.502022 2014-08-13 18:06:06.783338 7f5c973a57a0 20 read_log 192718'12924 (192718'12923) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:54 2014-08-06 15:24:52.541358 2014-08-13 18:06:06.783358 7f5c973a57a0 20 read_log 192718'12925 (192718'12924) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:55 2014-08-06 15:24:52.572327 2014-08-13 18:06:06.783370 7f5c973a57a0 20 read_log 192718'12926 (192718'12925) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:56 2014-08-06 15:24:52.602817 2014-08-13 18:06:06.783412 7f5c973a57a0 20 read_log 192718'12927 (192718'12926) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:57 2014-08-06 15:24:52.645849 2014-08-13 18:06:06.783441 7f5c973a57a0 20 read_log 192718'12928 (192718'12927) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:58 2014-08-06 15:24:52.696786 2014-08-13 18:06:06.783457 7f5c973a57a0 20 read_log 192718'12929 (192718'12928) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:59 2014-08-06 15:24:52.739794 2014-08-13 18:06:06.783470 7f5c973a57a0 20 read_log 192718'12930 (192718'12929) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:60 2014-08-06 15:24:52.781021 2014-08-13 18:06:06.783492 7f5c973a57a0 20 read_log 192718'12931 (192718'12930) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:61 2014-08-06 15:24:52.823192 2014-08-13 18:06:06.783505 7f5c973a57a0 20 read_log 192718'12932 (192718'12931) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:62 2014-08-06 15:24:52.868251 2014-08-13 18:06:06.783517 7f5c973a57a0 20 read_log 192718'12933 (192718'12932) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:63 2014-08-06 15:24:52.910387 2014-08-13 18:06:06.783537 7f5c973a57a0 20 read_log 192718'12934 (192718'12933) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:64 2014-08-06 15:24:52.953257 2014-08-13 18:06:06.783553 7f5c973a57a0 20 read_log 192718'12935 (192718'12934) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:65 2014-08-06 15:24:52.994503 2014-08-13 18:06:06.783565 7f5c973a57a0 20 read_log 192718'12936 (192718'12935) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:66 2014-08-06 15:24:53.019589 2014-08-13 18:06:06.783578 7f5c973a57a0 20 read_log 192718'12937 (192718'12936) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:67 2014-08-06 15:24:53.052020 2014-08-13 18:06:06.783590 7f5c973a57a0 20 read_log 192718'12938 (192718'12937) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:68 2014-08-06 15:24:53.077756 2014-08-13 18:06:06.783602 7f5c973a57a0 20 read_log 192718'12939 (192718'12938) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:69 2014-08-06 15:24:53.109563 2014-08-13 18:06:06.783614 7f5c973a57a0 20 read_log 192718'12940 (192718'12939) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:70 2014-08-06 15:24:53.140771 2014-08-13 18:06:06.783627 7f5c973a57a0 20 read_log 192718'12941 (192718'12940) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:71 2014-08-06 15:24:53.166270 2014-08-13 18:06:06.783639 7f5c973a57a0 20 read_log 192718'12942 (192718'12941) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:72 2014-08-06 15:24:53.215080 2014-08-13 18:06:06.783651 7f5c973a57a0 20 read_log 192718'12943 (192718'12942) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:73 2014-08-06 15:24:53.245523 2014-08-13 18:06:06.783663 7f5c973a57a0 20 read_log 192718'12944 (192718'12943) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:74 2014-08-06 15:24:53.274287 2014-08-13 18:06:06.783675 7f5c973a57a0 20 read_log 192718'12945 (192718'12944) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:75 2014-08-06 15:24:53.302027 2014-08-13 18:06:06.783687 7f5c973a57a0 20 read_log 192718'12946 (192718'12945) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:76 2014-08-06 15:24:53.335025 2014-08-13 18:06:06.783699 7f5c973a57a0 20 read_log 192718'12947 (192718'12946) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:77 2014-08-06 15:24:53.373354 2014-08-13 18:06:06.783712 7f5c973a57a0 20 read_log 192718'12948 (192718'12947) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:78 2014-08-06 15:24:53.401568 2014-08-13 18:06:06.783724 7f5c973a57a0 20 read_log 192718'12949 (192718'12948) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:79 2014-08-06 15:24:53.426974 2014-08-13 18:06:06.783736 7f5c973a57a0 20 read_log 192718'12950 (192718'12949) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:80 2014-08-06 15:24:53.458959 2014-08-13 18:06:06.783755 7f5c973a57a0 20 read_log 192718'12951 (192718'12950) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:81 2014-08-06 15:24:53.483675 2014-08-13 18:06:06.783767 7f5c973a57a0 20 read_log 192718'12952 (192718'12951) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:82 2014-08-06 15:24:53.510015 2014-08-13 18:06:06.783779 7f5c973a57a0 20 read_log 192718'12953 (192718'12952) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:83 2014-08-06 15:24:53.533872 2014-08-13 18:06:06.783791 7f5c973a57a0 20 read_log 192718'12954 (192718'12953) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:84 2014-08-06 15:24:53.558366 2014-08-13 18:06:06.783804 7f5c973a57a0 20 read_log 192718'12955 (192718'12954) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:85 2014-08-06 15:24:53.591179 2014-08-13 18:06:06.783816 7f5c973a57a0 20 read_log 192718'12956 (192718'12955) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:86 2014-08-06 15:24:53.622795 2014-08-13 18:06:06.783828 7f5c973a57a0 20 read_log 192718'12957 (192718'12956) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:87 2014-08-06 15:24:53.658412 2014-08-13 18:06:06.783840 7f5c973a57a0 20 read_log 192718'12958 (192718'12957) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:88 2014-08-06 15:24:53.683738 2014-08-13 18:06:06.783852 7f5c973a57a0 20 read_log 192718'12959 (192718'12958) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:89 2014-08-06 15:24:53.712818 2014-08-13 18:06:06.783864 7f5c973a57a0 20 read_log 192718'12960 (192718'12959) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:90 2014-08-06 15:24:53.744611 2014-08-13 18:06:06.783876 7f5c973a57a0 20 read_log 192718'12961 (192718'12960) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:91 2014-08-06 15:24:53.766915 2014-08-13 18:06:06.783891 7f5c973a57a0 20 read_log 192718'12962 (192718'12961) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:92 2014-08-06 15:24:53.802135 2014-08-13 18:06:06.783903 7f5c973a57a0 20 read_log 192718'12963 (192718'12962) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:93 2014-08-06 15:24:53.837191 2014-08-13 18:06:06.783917 7f5c973a57a0 20 read_log 192718'12964 (192718'12963) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:94 2014-08-06 15:24:53.899762 2014-08-13 18:06:06.783951 7f5c973a57a0 20 read_log 192718'12965 (192718'12964) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:95 2014-08-06 15:24:53.932235 2014-08-13 18:06:06.783967 7f5c973a57a0 20 read_log 192718'12966 (192718'12965) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:96 2014-08-06 15:24:53.962983 2014-08-13 18:06:06.783980 7f5c973a57a0 20 read_log 192718'12967 (192718'12966) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:97 2014-08-06 15:24:53.995196 2014-08-13 18:06:06.783994 7f5c973a57a0 20 read_log 192718'12968 (192718'12967) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:98 2014-08-06 15:24:54.034257 2014-08-13 18:06:06.784007 7f5c973a57a0 20 read_log 192718'12969 (192718'12968) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:99 2014-08-06 15:24:54.059904 2014-08-13 18:06:06.784021 7f5c973a57a0 20 read_log 192718'12970 (192718'12969) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:100 2014-08-06 15:24:54.096752 2014-08-13 18:06:06.784033 7f5c973a57a0 20 read_log 192718'12971 (192718'12970) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:101 2014-08-06 15:24:54.124311 2014-08-13 18:06:06.784047 7f5c973a57a0 20 read_log 192718'12972 (192718'12971) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:102 2014-08-06 15:24:54.150508 2014-08-13 18:06:06.784075 7f5c973a57a0 20 read_log 192718'12973 (192718'12972) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:103 2014-08-06 15:24:54.180616 2014-08-13 18:06:06.784089 7f5c973a57a0 20 read_log 192718'12974 (192718'12973) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:104 2014-08-06 15:24:54.205055 2014-08-13 18:06:06.784111 7f5c973a57a0 20 read_log 192718'12975 (192718'12974) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:105 2014-08-06 15:24:54.239803 2014-08-13 18:06:06.784126 7f5c973a57a0 20 read_log 192718'12976 (192718'12975) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:106 2014-08-06 15:24:54.267614 2014-08-13 18:06:06.784138 7f5c973a57a0 20 read_log 192718'12977 (192718'12976) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:107 2014-08-06 15:24:54.296078 2014-08-13 18:06:06.784159 7f5c973a57a0 20 read_log 192718'12978 (192718'12977) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:108 2014-08-06 15:24:54.321972 2014-08-13 18:06:06.784174 7f5c973a57a0 20 read_log 192718'12979 (192718'12978) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:109 2014-08-06 15:24:54.353636 2014-08-13 18:06:06.784194 7f5c973a57a0 20 read_log 192718'12980 (192718'12979) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:110 2014-08-06 15:24:54.386511 2014-08-13 18:06:06.784207 7f5c973a57a0 20 read_log 192718'12981 (192718'12980) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:111 2014-08-06 15:24:54.410818 2014-08-13 18:06:06.784219 7f5c973a57a0 20 read_log 192718'12982 (192718'12981) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:112 2014-08-06 15:24:54.452686 2014-08-13 18:06:06.784231 7f5c973a57a0 20 read_log 192718'12983 (192718'12982) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:113 2014-08-06 15:24:54.479632 2014-08-13 18:06:06.784245 7f5c973a57a0 20 read_log 192718'12984 (192718'12983) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:114 2014-08-06 15:24:54.537663 2014-08-13 18:06:06.784257 7f5c973a57a0 20 read_log 192718'12985 (192718'12984) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:115 2014-08-06 15:24:54.570157 2014-08-13 18:06:06.784270 7f5c973a57a0 20 read_log 192718'12986 (192718'12985) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:116 2014-08-06 15:24:54.598680 2014-08-13 18:06:06.784284 7f5c973a57a0 20 read_log 192718'12987 (192718'12986) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:117 2014-08-06 15:24:54.639901 2014-08-13 18:06:06.784299 7f5c973a57a0 20 read_log 192718'12988 (192718'12987) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:118 2014-08-06 15:24:54.667717 2014-08-13 18:06:06.784311 7f5c973a57a0 20 read_log 192718'12989 (192718'12988) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:119 2014-08-06 15:24:54.731431 2014-08-13 18:06:06.784351 7f5c973a57a0 20 read_log 192718'12990 (192718'12989) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:120 2014-08-06 15:24:54.763905 2014-08-13 18:06:06.784366 7f5c973a57a0 20 read_log 192718'12991 (192718'12990) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:121 2014-08-06 15:24:54.784962 2014-08-13 18:06:06.784378 7f5c973a57a0 20 read_log 192718'12992 (192718'12991) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:122 2014-08-06 15:24:54.824008 2014-08-13 18:06:06.784390 7f5c973a57a0 20 read_log 192718'12993 (192718'12992) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:123 2014-08-06 15:24:54.847003 2014-08-13 18:06:06.784408 7f5c973a57a0 20 read_log 192718'12994 (192718'12993) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:124 2014-08-06 15:24:54.875523 2014-08-13 18:06:06.784420 7f5c973a57a0 20 read_log 192718'12995 (192718'12994) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:125 2014-08-06 15:24:55.241111 2014-08-13 18:06:06.784445 7f5c973a57a0 20 read_log 192718'12996 (192718'12995) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:126 2014-08-06 15:24:55.337976 2014-08-13 18:06:06.784458 7f5c973a57a0 20 read_log 192718'12997 (192718'12996) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:127 2014-08-06 15:24:55.508412 2014-08-13 18:06:06.784477 7f5c973a57a0 20 read_log 192718'12998 (192718'12997) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:128 2014-08-06 15:24:55.542707 2014-08-13 18:06:06.784515 7f5c973a57a0 20 read_log 192718'12999 (192718'12998) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:129 2014-08-06 15:24:55.577165 2014-08-13 18:06:06.784539 7f5c973a57a0 20 read_log 192718'13000 (192718'12999) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:130 2014-08-06 15:24:55.607515 2014-08-13 18:06:06.784552 7f5c973a57a0 20 read_log 192718'13001 (192718'13000) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:131 2014-08-06 15:24:55.634278 2014-08-13 18:06:06.784565 7f5c973a57a0 20 read_log 192718'13002 (192718'13001) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:132 2014-08-06 15:24:55.805528 2014-08-13 18:06:06.784577 7f5c973a57a0 20 read_log 192718'13003 (192718'13002) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:133 2014-08-06 15:24:55.947411 2014-08-13 18:06:06.784589 7f5c973a57a0 20 read_log 192718'13004 (192718'13003) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:134 2014-08-06 15:24:55.972566 2014-08-13 18:06:06.784602 7f5c973a57a0 20 read_log 192718'13005 (192718'13004) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:135 2014-08-06 15:24:56.421414 2014-08-13 18:06:06.784614 7f5c973a57a0 20 read_log 192718'13006 (192718'13005) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:136 2014-08-06 15:24:56.451611 2014-08-13 18:06:06.784626 7f5c973a57a0 20 read_log 192718'13007 (192718'13006) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:137 2014-08-06 15:24:56.702159 2014-08-13 18:06:06.784638 7f5c973a57a0 20 read_log 192718'13008 (192718'13007) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:138 2014-08-06 15:24:56.737239 2014-08-13 18:06:06.784650 7f5c973a57a0 20 read_log 192718'13009 (192718'13008) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:139 2014-08-06 15:24:56.766385 2014-08-13 18:06:06.784662 7f5c973a57a0 20 read_log 192718'13010 (192718'13009) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:140 2014-08-06 15:24:56.859552 2014-08-13 18:06:06.784674 7f5c973a57a0 20 read_log 192718'13011 (192718'13010) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:141 2014-08-06 15:24:56.969870 2014-08-13 18:06:06.784686 7f5c973a57a0 20 read_log 192718'13012 (192718'13011) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:142 2014-08-06 15:24:57.018994 2014-08-13 18:06:06.784698 7f5c973a57a0 20 read_log 192718'13013 (192718'13012) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:143 2014-08-06 15:24:57.305959 2014-08-13 18:06:06.784710 7f5c973a57a0 20 read_log 192718'13014 (192718'13013) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:144 2014-08-06 15:24:57.360675 2014-08-13 18:06:06.784722 7f5c973a57a0 20 read_log 192718'13015 (192718'13014) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:145 2014-08-06 15:24:57.679353 2014-08-13 18:06:06.784734 7f5c973a57a0 20 read_log 192718'13016 (192718'13015) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:146 2014-08-06 15:24:57.707449 2014-08-13 18:06:06.784746 7f5c973a57a0 20 read_log 192718'13017 (192718'13016) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:147 2014-08-06 15:24:57.748457 2014-08-13 18:06:06.784758 7f5c973a57a0 20 read_log 192718'13018 (192718'13017) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:148 2014-08-06 15:24:57.843826 2014-08-13 18:06:06.784770 7f5c973a57a0 20 read_log 192718'13019 (192718'13018) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:149 2014-08-06 15:24:57.880124 2014-08-13 18:06:06.784783 7f5c973a57a0 20 read_log 192718'13020 (192718'13019) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:150 2014-08-06 15:24:58.017593 2014-08-13 18:06:06.784795 7f5c973a57a0 20 read_log 192718'13021 (192718'13020) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:151 2014-08-06 15:24:58.043514 2014-08-13 18:06:06.784807 7f5c973a57a0 20 read_log 192718'13022 (192718'13021) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:152 2014-08-06 15:24:58.122123 2014-08-13 18:06:06.784825 7f5c973a57a0 20 read_log 192718'13023 (192718'13022) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:153 2014-08-06 15:24:58.238607 2014-08-13 18:06:06.784840 7f5c973a57a0 20 read_log 192718'13024 (192718'13023) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:154 2014-08-06 15:24:58.261422 2014-08-13 18:06:06.784852 7f5c973a57a0 20 read_log 192718'13025 (192718'13024) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:155 2014-08-06 15:24:58.954801 2014-08-13 18:06:06.784867 7f5c973a57a0 20 read_log 192718'13026 (192718'13025) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:156 2014-08-06 15:24:59.190463 2014-08-13 18:06:06.784911 7f5c973a57a0 20 read_log 192718'13027 (192718'13026) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:157 2014-08-06 15:24:59.237926 2014-08-13 18:06:06.784928 7f5c973a57a0 20 read_log 192718'13028 (192718'13027) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:158 2014-08-06 15:24:59.262128 2014-08-13 18:06:06.784941 7f5c973a57a0 20 read_log 192718'13029 (192718'13028) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:159 2014-08-06 15:24:59.406584 2014-08-13 18:06:06.784955 7f5c973a57a0 20 read_log 192718'13030 (192718'13029) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:160 2014-08-06 15:24:59.438109 2014-08-13 18:06:06.784968 7f5c973a57a0 20 read_log 192718'13031 (192718'13030) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:161 2014-08-06 15:24:59.466531 2014-08-13 18:06:06.784982 7f5c973a57a0 20 read_log 192718'13032 (192718'13031) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:162 2014-08-06 15:24:59.645755 2014-08-13 18:06:06.784999 7f5c973a57a0 20 read_log 192718'13033 (192718'13032) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:163 2014-08-06 15:24:59.674300 2014-08-13 18:06:06.785014 7f5c973a57a0 20 read_log 192718'13034 (192718'13033) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:164 2014-08-06 15:24:59.706738 2014-08-13 18:06:06.785026 7f5c973a57a0 20 read_log 192718'13035 (192718'13034) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:165 2014-08-06 15:25:00.385904 2014-08-13 18:06:06.785038 7f5c973a57a0 20 read_log 192718'13036 (192718'13035) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:166 2014-08-06 15:25:00.412808 2014-08-13 18:06:06.785051 7f5c973a57a0 20 read_log 192718'13037 (192718'13036) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:167 2014-08-06 15:25:00.466025 2014-08-13 18:06:06.785063 7f5c973a57a0 20 read_log 192718'13038 (192718'13037) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:168 2014-08-06 15:25:00.637563 2014-08-13 18:06:06.785077 7f5c973a57a0 20 read_log 192718'13039 (192718'13038) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:169 2014-08-06 15:25:00.665982 2014-08-13 18:06:06.785089 7f5c973a57a0 20 read_log 192718'13040 (192718'13039) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:170 2014-08-06 15:25:00.710207 2014-08-13 18:06:06.785102 7f5c973a57a0 20 read_log 192718'13041 (192718'13040) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:171 2014-08-06 15:25:00.878477 2014-08-13 18:06:06.785117 7f5c973a57a0 20 read_log 192718'13042 (192718'13041) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:172 2014-08-06 15:25:00.916571 2014-08-13 18:06:06.785129 7f5c973a57a0 20 read_log 192718'13043 (192718'13042) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:173 2014-08-06 15:25:00.947094 2014-08-13 18:06:06.785143 7f5c973a57a0 20 read_log 192718'13044 (192718'13043) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:174 2014-08-06 15:25:01.176849 2014-08-13 18:06:06.785164 7f5c973a57a0 20 read_log 192718'13045 (192718'13044) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:175 2014-08-06 15:25:01.681116 2014-08-13 18:06:06.785176 7f5c973a57a0 20 read_log 192718'13046 (192718'13045) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:176 2014-08-06 15:25:01.710062 2014-08-13 18:06:06.785201 7f5c973a57a0 20 read_log 192718'13047 (192718'13046) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:177 2014-08-06 15:25:01.810633 2014-08-13 18:06:06.785216 7f5c973a57a0 20 read_log 192718'13048 (192718'13047) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:178 2014-08-06 15:25:01.928976 2014-08-13 18:06:06.785229 7f5c973a57a0 20 read_log 192718'13051 (192718'13048) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:179 2014-08-06 15:25:01.961540 2014-08-13 18:06:06.785243 7f5c973a57a0 20 read_log 192718'13052 (192718'13051) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:180 2014-08-06 15:25:02.069954 2014-08-13 18:06:06.785256 7f5c973a57a0 20 read_log 192718'13053 (192718'13052) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:181 2014-08-06 15:25:02.172992 2014-08-13 18:06:06.785273 7f5c973a57a0 20 read_log 192718'13054 (192718'13053) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:182 2014-08-06 15:25:02.204544 2014-08-13 18:06:06.785286 7f5c973a57a0 20 read_log 192718'13055 (192718'13054) modify 30d03df7/core.29770+97+2/head//24 by client.416500.0:183 2014-08-06 15:25:02.449339 2014-08-13 18:06:06.785300 7f5c973a57a0 20 read_log 192797'13316 (192718'13055) clean 30d03df7/core.29770+97+2/head//24 by osd.306.0:27365 2014-08-11 10:24:24.831086 2014-08-13 18:06:06.785313 7f5c973a57a0 20 read_log 192798'13321 (192797'13316) delete 30d03df7/core.29770+97+2/head//24 by osd.306.0:27374 2014-08-11 10:24:45.664487 2014-08-13 18:06:06.785328 7f5c973a57a0 20 read_log 192834'13631 (0'0) promote 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by osd.306.0:27889 2014-08-11 15:23:37.056859 2014-08-13 18:06:06.785341 7f5c973a57a0 20 read_log 192834'13632 (192834'13631) modify 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.933448.0:366 2014-08-11 15:23:37.049493 2014-08-13 18:06:06.785354 7f5c973a57a0 20 read_log 192834'13641 (0'0) promote e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by osd.306.0:27905 2014-08-11 15:23:39.830975 2014-08-13 18:06:06.785366 7f5c973a57a0 20 read_log 192834'13642 (192834'13641) modify e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.933448.0:1079 2014-08-11 15:23:39.822726 2014-08-13 18:06:06.785378 7f5c973a57a0 20 read_log 192834'13657 (0'0) promote c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by osd.306.0:27947 2014-08-11 15:23:44.957210 2014-08-13 18:06:06.785390 7f5c973a57a0 20 read_log 192834'13658 (192834'13657) modify c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.933448.0:2100 2014-08-11 15:23:44.951160 2014-08-13 18:06:06.785411 7f5c973a57a0 20 read_log 192834'13663 (0'0) promote 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by osd.306.0:27961 2014-08-11 15:23:46.584785 2014-08-13 18:06:06.785427 7f5c973a57a0 20 read_log 192834'13664 (192834'13663) modify 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.933448.0:2397 2014-08-11 15:23:46.573721 2014-08-13 18:06:06.785440 7f5c973a57a0 20 read_log 192834'13677 (0'0) promote d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by osd.306.0:27993 2014-08-11 15:23:55.429322 2014-08-13 18:06:06.785466 7f5c973a57a0 20 read_log 192834'13678 (192834'13677) modify d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.933448.0:4091 2014-08-11 15:23:55.421713 2014-08-13 18:06:06.785513 7f5c973a57a0 20 read_log 192834'13683 (0'0) promote 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by osd.306.0:28005 2014-08-11 15:23:56.289758 2014-08-13 18:06:06.785530 7f5c973a57a0 20 read_log 192834'13684 (192834'13683) modify 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.933448.0:4306 2014-08-11 15:23:56.281240 2014-08-13 18:06:06.785560 7f5c973a57a0 20 read_log 192834'13687 (0'0) promote 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by osd.306.0:28013 2014-08-11 15:23:58.634559 2014-08-13 18:06:06.785576 7f5c973a57a0 20 read_log 192834'13688 (192834'13687) modify 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.933448.0:4798 2014-08-11 15:23:58.627088 2014-08-13 18:06:06.785589 7f5c973a57a0 20 read_log 192834'13747 (0'0) promote 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by osd.306.0:28133 2014-08-11 15:24:12.112130 2014-08-13 18:06:06.785602 7f5c973a57a0 20 read_log 192834'13748 (192834'13747) modify 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.933448.0:7975 2014-08-11 15:24:12.104539 2014-08-13 18:06:06.785614 7f5c973a57a0 20 read_log 192834'13751 (0'0) promote db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by osd.306.0:28137 2014-08-11 15:24:13.015061 2014-08-13 18:06:06.785630 7f5c973a57a0 20 read_log 192834'13752 (192834'13751) modify db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.933448.0:8175 2014-08-11 15:24:13.007610 2014-08-13 18:06:06.785645 7f5c973a57a0 20 read_log 192834'13753 (0'0) promote 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by osd.306.0:28141 2014-08-11 15:24:13.316405 2014-08-13 18:06:06.785668 7f5c973a57a0 20 read_log 192834'13754 (192834'13753) modify 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.933448.0:8255 2014-08-11 15:24:13.307187 2014-08-13 18:06:06.785681 7f5c973a57a0 20 read_log 192834'13785 (0'0) promote 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by osd.306.0:28211 2014-08-11 15:24:21.258568 2014-08-13 18:06:06.785693 7f5c973a57a0 20 read_log 192834'13786 (192834'13785) modify 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.933448.0:10022 2014-08-11 15:24:21.249744 2014-08-13 18:06:06.785706 7f5c973a57a0 20 read_log 192834'13809 (0'0) promote bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by osd.306.0:28265 2014-08-11 15:24:30.991308 2014-08-13 18:06:06.785718 7f5c973a57a0 20 read_log 192834'13810 (192834'13809) modify bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.933448.0:12075 2014-08-11 15:24:30.983429 2014-08-13 18:06:06.785731 7f5c973a57a0 20 read_log 192834'13811 (0'0) promote a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by osd.306.0:28273 2014-08-11 15:24:32.343374 2014-08-13 18:06:06.785743 7f5c973a57a0 20 read_log 192834'13812 (192834'13811) modify a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.933448.0:12436 2014-08-11 15:24:32.336062 2014-08-13 18:06:06.785755 7f5c973a57a0 20 read_log 192834'13813 (0'0) promote e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by osd.306.0:28275 2014-08-11 15:24:32.445051 2014-08-13 18:06:06.785768 7f5c973a57a0 20 read_log 192834'13814 (192834'13813) modify e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.933448.0:12463 2014-08-11 15:24:32.437908 2014-08-13 18:06:06.785780 7f5c973a57a0 20 read_log 192834'13825 (0'0) promote d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by osd.306.0:28295 2014-08-11 15:24:35.210764 2014-08-13 18:06:06.785792 7f5c973a57a0 20 read_log 192834'13826 (192834'13825) modify d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.933448.0:13061 2014-08-11 15:24:35.203166 2014-08-13 18:06:06.785805 7f5c973a57a0 20 read_log 192834'13853 (0'0) promote c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by osd.306.0:28341 2014-08-11 15:24:45.508319 2014-08-13 18:06:06.785817 7f5c973a57a0 20 read_log 192834'13854 (192834'13853) modify c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.933448.0:14442 2014-08-11 15:24:45.500005 2014-08-13 18:06:06.785836 7f5c973a57a0 20 read_log 192834'13855 (0'0) promote 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by osd.306.0:28343 2014-08-11 15:24:45.932779 2014-08-13 18:06:06.785852 7f5c973a57a0 20 read_log 192834'13856 (192834'13855) modify 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.933448.0:14510 2014-08-11 15:24:45.923024 2014-08-13 18:06:06.785864 7f5c973a57a0 20 read_log 192834'13861 (0'0) promote ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by osd.306.0:28353 2014-08-11 15:24:47.738909 2014-08-13 18:06:06.785877 7f5c973a57a0 20 read_log 192834'13862 (192834'13861) modify ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.933448.0:14930 2014-08-11 15:24:47.731311 2014-08-13 18:06:06.785889 7f5c973a57a0 20 read_log 192834'13863 (0'0) promote e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by osd.306.0:28355 2014-08-11 15:24:47.845686 2014-08-13 18:06:06.785901 7f5c973a57a0 20 read_log 192834'13864 (192834'13863) modify e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.933448.0:14959 2014-08-11 15:24:47.837487 2014-08-13 18:06:06.785913 7f5c973a57a0 20 read_log 192834'13877 (0'0) promote 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by osd.306.0:28371 2014-08-11 15:24:49.766481 2014-08-13 18:06:06.785925 7f5c973a57a0 20 read_log 192834'13878 (192834'13877) modify 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.933448.0:15468 2014-08-11 15:24:49.759293 2014-08-13 18:06:06.785938 7f5c973a57a0 20 read_log 192834'13899 (0'0) promote 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by osd.306.0:28427 2014-08-11 15:24:59.769526 2014-08-13 18:06:06.785950 7f5c973a57a0 20 read_log 192834'13900 (192834'13899) modify 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.933448.0:17456 2014-08-11 15:24:59.759982 2014-08-13 18:06:06.785962 7f5c973a57a0 20 read_log 192834'13901 (0'0) promote 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by osd.306.0:28429 2014-08-11 15:24:59.918308 2014-08-13 18:06:06.785975 7f5c973a57a0 20 read_log 192834'13902 (192834'13901) modify 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.933448.0:17494 2014-08-11 15:24:59.910248 2014-08-13 18:06:06.785989 7f5c973a57a0 20 read_log 192834'13909 (0'0) promote 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by osd.306.0:28455 2014-08-11 15:25:04.552690 2014-08-13 18:06:06.786003 7f5c973a57a0 20 read_log 192834'13910 (192834'13909) modify 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.933448.0:18274 2014-08-11 15:25:04.545560 2014-08-13 18:06:06.786016 7f5c973a57a0 20 read_log 192834'13915 (0'0) promote 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by osd.306.0:28461 2014-08-11 15:25:05.345427 2014-08-13 18:06:06.786030 7f5c973a57a0 20 read_log 192834'13916 (192834'13915) modify 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.933448.0:18446 2014-08-11 15:25:05.337622 2014-08-13 18:06:06.786047 7f5c973a57a0 20 read_log 192834'13919 (0'0) promote 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by osd.306.0:28467 2014-08-11 15:25:07.336858 2014-08-13 18:06:06.786061 7f5c973a57a0 20 read_log 192834'13920 (192834'13919) modify 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.933448.0:18884 2014-08-11 15:25:07.329123 2014-08-13 18:06:06.786074 7f5c973a57a0 20 read_log 192834'13927 (0'0) promote ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by osd.306.0:28475 2014-08-11 15:25:08.673661 2014-08-13 18:06:06.786088 7f5c973a57a0 20 read_log 192834'13928 (192834'13927) modify ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.933448.0:19114 2014-08-11 15:25:08.665552 2014-08-13 18:06:06.786110 7f5c973a57a0 20 read_log 192834'13937 (0'0) promote 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by osd.306.0:28491 2014-08-11 15:25:12.822583 2014-08-13 18:06:06.786125 7f5c973a57a0 20 read_log 192834'13938 (192834'13937) modify 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.933448.0:19631 2014-08-11 15:25:12.814961 2014-08-13 18:06:06.786138 7f5c973a57a0 20 read_log 192834'13965 (192834'13632) modify 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.933448.0:22657 2014-08-11 15:25:38.404194 2014-08-13 18:06:06.786157 7f5c973a57a0 20 read_log 192834'13970 (192834'13642) modify e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.933448.0:23370 2014-08-11 15:25:39.281880 2014-08-13 18:06:06.786173 7f5c973a57a0 20 read_log 192834'13978 (192834'13658) modify c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.933448.0:24391 2014-08-11 15:25:40.747225 2014-08-13 18:06:06.786185 7f5c973a57a0 20 read_log 192834'13981 (192834'13664) modify 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.933448.0:24688 2014-08-11 15:25:41.177779 2014-08-13 18:06:06.786200 7f5c973a57a0 20 read_log 192834'13988 (192834'13678) modify d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.933448.0:26382 2014-08-11 15:25:43.333975 2014-08-13 18:06:06.786212 7f5c973a57a0 20 read_log 192834'13990 (192834'13684) modify 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.933448.0:26597 2014-08-11 15:25:43.573048 2014-08-13 18:06:06.786225 7f5c973a57a0 20 read_log 192834'13992 (192834'13688) modify 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.933448.0:27089 2014-08-11 15:25:44.079622 2014-08-13 18:06:06.786239 7f5c973a57a0 20 read_log 192834'14022 (192834'13748) modify 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.933448.0:30266 2014-08-11 15:25:48.662667 2014-08-13 18:06:06.786251 7f5c973a57a0 20 read_log 192834'14024 (192834'13752) modify db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.933448.0:30466 2014-08-11 15:25:48.889852 2014-08-13 18:06:06.786266 7f5c973a57a0 20 read_log 192834'14025 (192834'13754) modify 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.933448.0:30546 2014-08-11 15:25:48.974647 2014-08-13 18:06:06.786279 7f5c973a57a0 20 read_log 192834'14041 (192834'13786) modify 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.933448.0:32313 2014-08-11 15:25:51.455784 2014-08-13 18:06:06.786292 7f5c973a57a0 20 read_log 192834'14053 (192834'13810) modify bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.933448.0:34366 2014-08-11 15:25:54.936359 2014-08-13 18:06:06.786304 7f5c973a57a0 20 read_log 192834'14054 (192834'13812) modify a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.933448.0:34727 2014-08-11 15:25:55.394304 2014-08-13 18:06:06.786320 7f5c973a57a0 20 read_log 192834'14055 (192834'13814) modify e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.933448.0:34754 2014-08-11 15:25:55.407554 2014-08-13 18:06:06.786332 7f5c973a57a0 20 read_log 192834'14061 (192834'13826) modify d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.933448.0:35352 2014-08-11 15:25:56.130057 2014-08-13 18:06:06.786349 7f5c973a57a0 20 read_log 192834'14075 (192834'13854) modify c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.933448.0:36733 2014-08-11 15:25:58.248474 2014-08-13 18:06:06.786364 7f5c973a57a0 20 read_log 192834'14076 (192834'13856) modify 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.933448.0:36801 2014-08-11 15:25:58.325606 2014-08-13 18:06:06.786376 7f5c973a57a0 20 read_log 192834'14079 (192834'13862) modify ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.933448.0:37221 2014-08-11 15:25:59.154352 2014-08-13 18:06:06.786399 7f5c973a57a0 20 read_log 192834'14080 (192834'13864) modify e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.933448.0:37250 2014-08-11 15:25:59.206499 2014-08-13 18:06:06.786424 7f5c973a57a0 20 read_log 192834'14087 (192834'13878) modify 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.933448.0:37759 2014-08-11 15:25:59.967102 2014-08-13 18:06:06.786437 7f5c973a57a0 20 read_log 192834'14098 (192834'13900) modify 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.933448.0:39747 2014-08-11 15:26:02.057163 2014-08-13 18:06:06.786458 7f5c973a57a0 20 read_log 192834'14099 (192834'13902) modify 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.933448.0:39785 2014-08-11 15:26:02.079253 2014-08-13 18:06:06.786475 7f5c973a57a0 20 read_log 192834'14103 (192834'13910) modify 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.933448.0:40565 2014-08-11 15:26:02.653932 2014-08-13 18:06:06.786488 7f5c973a57a0 20 read_log 192834'14106 (192834'13916) modify 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.933448.0:40737 2014-08-11 15:26:02.714206 2014-08-13 18:06:06.786503 7f5c973a57a0 20 read_log 192834'14108 (192834'13920) modify 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.933448.0:41175 2014-08-11 15:26:02.860236 2014-08-13 18:06:06.786515 7f5c973a57a0 20 read_log 192834'14112 (192834'13928) modify ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.933448.0:41405 2014-08-11 15:26:02.934285 2014-08-13 18:06:06.786529 7f5c973a57a0 20 read_log 192834'14117 (192834'13938) modify 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.933448.0:41922 2014-08-11 15:26:03.094669 2014-08-13 18:06:06.786542 7f5c973a57a0 20 read_log 192834'14158 (0'0) promote 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by osd.306.0:28956 2014-08-11 15:26:21.407453 2014-08-13 18:06:06.786554 7f5c973a57a0 20 read_log 192834'14159 (192834'14158) modify 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.1883634.0:1877 2014-08-11 15:26:21.399255 2014-08-13 18:06:06.786566 7f5c973a57a0 20 read_log 192834'14162 (0'0) promote f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by osd.306.0:28964 2014-08-11 15:26:23.177186 2014-08-13 18:06:06.786579 7f5c973a57a0 20 read_log 192834'14163 (192834'14162) modify f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.1883634.0:2355 2014-08-11 15:26:23.168935 2014-08-13 18:06:06.786591 7f5c973a57a0 20 read_log 192834'14174 (0'0) promote 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by osd.306.0:28978 2014-08-11 15:26:25.110169 2014-08-13 18:06:06.786603 7f5c973a57a0 20 read_log 192834'14175 (192834'14174) modify 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.1883634.0:2902 2014-08-11 15:26:25.102157 2014-08-13 18:06:06.786615 7f5c973a57a0 20 read_log 192834'14194 (0'0) promote 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by osd.306.0:29016 2014-08-11 15:26:31.099945 2014-08-13 18:06:06.786627 7f5c973a57a0 20 read_log 192834'14195 (192834'14194) modify 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.1883634.0:4570 2014-08-11 15:26:31.091595 2014-08-13 18:06:06.786669 7f5c973a57a0 20 read_log 192834'14198 (0'0) promote 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by osd.306.0:29024 2014-08-11 15:26:31.589800 2014-08-13 18:06:06.786702 7f5c973a57a0 20 read_log 192834'14199 (192834'14198) modify 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.1883634.0:4701 2014-08-11 15:26:31.580430 2014-08-13 18:06:06.786715 7f5c973a57a0 20 read_log 192834'14220 (0'0) promote 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by osd.306.0:29074 2014-08-11 15:26:37.159596 2014-08-13 18:06:06.786751 7f5c973a57a0 20 read_log 192834'14221 (192834'14220) modify 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.1883634.0:6210 2014-08-11 15:26:37.152952 2014-08-13 18:06:06.786767 7f5c973a57a0 20 read_log 192834'14222 (0'0) promote cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by osd.306.0:29078 2014-08-11 15:26:37.329855 2014-08-13 18:06:06.786780 7f5c973a57a0 20 read_log 192834'14223 (192834'14222) modify cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.1883634.0:6253 2014-08-11 15:26:37.322353 2014-08-13 18:06:06.786792 7f5c973a57a0 20 read_log 192834'14236 (0'0) promote 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by osd.306.0:29102 2014-08-11 15:26:39.809506 2014-08-13 18:06:06.786805 7f5c973a57a0 20 read_log 192834'14237 (192834'14236) modify 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.1883634.0:6874 2014-08-11 15:26:39.799952 2014-08-13 18:06:06.786817 7f5c973a57a0 20 read_log 192834'14256 (0'0) promote 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by osd.306.0:29146 2014-08-11 15:26:46.328249 2014-08-13 18:06:06.786830 7f5c973a57a0 20 read_log 192834'14257 (192834'14256) modify 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.1883634.0:8442 2014-08-11 15:26:46.319832 2014-08-13 18:06:06.786844 7f5c973a57a0 20 read_log 192834'14286 (0'0) promote 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by osd.306.0:29192 2014-08-11 15:26:50.680179 2014-08-13 18:06:06.786856 7f5c973a57a0 20 read_log 192834'14287 (192834'14286) modify 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.1883634.0:9386 2014-08-11 15:26:50.672991 2014-08-13 18:06:06.786871 7f5c973a57a0 20 read_log 192834'14304 (0'0) promote 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by osd.306.0:29224 2014-08-11 15:26:56.695920 2014-08-13 18:06:06.786883 7f5c973a57a0 20 read_log 192834'14305 (192834'14304) modify 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.1883634.0:10775 2014-08-11 15:26:56.688379 2014-08-13 18:06:06.786911 7f5c973a57a0 20 read_log 192834'14326 (0'0) promote c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by osd.306.0:29280 2014-08-11 15:27:01.680021 2014-08-13 18:06:06.786926 7f5c973a57a0 20 read_log 192834'14327 (192834'14326) modify c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.1883634.0:11909 2014-08-11 15:27:01.670971 2014-08-13 18:06:06.786938 7f5c973a57a0 20 read_log 192834'14366 (0'0) promote 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by osd.306.0:29382 2014-08-11 15:27:19.346336 2014-08-13 18:06:06.786955 7f5c973a57a0 20 read_log 192834'14367 (192834'14366) modify 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.1883634.0:15391 2014-08-11 15:27:19.337817 2014-08-13 18:06:06.786970 7f5c973a57a0 20 read_log 192834'14370 (0'0) promote 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by osd.306.0:29386 2014-08-11 15:27:19.538697 2014-08-13 18:06:06.786982 7f5c973a57a0 20 read_log 192834'14371 (192834'14370) modify 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.1883634.0:15439 2014-08-11 15:27:19.529201 2014-08-13 18:06:06.786997 7f5c973a57a0 20 read_log 192834'14380 (0'0) promote 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by osd.306.0:29412 2014-08-11 15:27:25.355117 2014-08-13 18:06:06.787009 7f5c973a57a0 20 read_log 192834'14381 (192834'14380) modify 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.1883634.0:16029 2014-08-11 15:27:25.346109 2014-08-13 18:06:06.787023 7f5c973a57a0 20 read_log 192834'14394 (0'0) promote 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by osd.306.0:29450 2014-08-11 15:27:38.676586 2014-08-13 18:06:06.787046 7f5c973a57a0 20 read_log 192834'14395 (192834'14394) modify 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.1883634.0:17715 2014-08-11 15:27:38.666866 2014-08-13 18:06:06.787061 7f5c973a57a0 20 read_log 192834'14402 (0'0) promote 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by osd.306.0:29458 2014-08-11 15:27:39.399405 2014-08-13 18:06:06.787074 7f5c973a57a0 20 read_log 192834'14403 (192834'14402) modify 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.1883634.0:17864 2014-08-11 15:27:39.388286 2014-08-13 18:06:06.787088 7f5c973a57a0 20 read_log 192834'14430 (0'0) promote 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by osd.306.0:29510 2014-08-11 15:27:54.109913 2014-08-13 18:06:06.787105 7f5c973a57a0 20 read_log 192834'14431 (192834'14430) modify 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.1883634.0:19780 2014-08-11 15:27:54.100162 2014-08-13 18:06:06.787120 7f5c973a57a0 20 read_log 192834'14442 (0'0) promote 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by osd.306.0:29528 2014-08-11 15:27:55.994888 2014-08-13 18:06:06.787132 7f5c973a57a0 20 read_log 192834'14443 (192834'14442) modify 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.1883634.0:20278 2014-08-11 15:27:55.985794 2014-08-13 18:06:06.787152 7f5c973a57a0 20 read_log 192834'14450 (0'0) promote 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by osd.306.0:29548 2014-08-11 15:28:02.509009 2014-08-13 18:06:06.787167 7f5c973a57a0 20 read_log 192834'14451 (192834'14450) modify 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.1883634.0:21047 2014-08-11 15:28:02.499500 2014-08-13 18:06:06.787180 7f5c973a57a0 20 read_log 192834'14454 (0'0) promote d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by osd.306.0:29558 2014-08-11 15:28:04.051727 2014-08-13 18:06:06.787192 7f5c973a57a0 20 read_log 192834'14455 (192834'14454) modify d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.1883634.0:21394 2014-08-11 15:28:04.043438 2014-08-13 18:06:06.787205 7f5c973a57a0 20 read_log 192834'14464 (0'0) promote 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by osd.306.0:29578 2014-08-11 15:28:07.460423 2014-08-13 18:06:06.787217 7f5c973a57a0 20 read_log 192834'14465 (192834'14464) modify 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.1883634.0:22226 2014-08-11 15:28:07.450812 2014-08-13 18:06:06.787231 7f5c973a57a0 20 read_log 192834'14492 (192834'14159) modify 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.1883634.0:25201 2014-08-11 15:28:20.332407 2014-08-13 18:06:06.787246 7f5c973a57a0 20 read_log 192834'14494 (192834'14163) modify f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.1883634.0:25679 2014-08-11 15:28:21.725131 2014-08-13 18:06:06.787261 7f5c973a57a0 20 read_log 192834'14500 (192834'14175) modify 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.1883634.0:26226 2014-08-11 15:28:22.809462 2014-08-13 18:06:06.787276 7f5c973a57a0 20 read_log 192834'14510 (192834'14195) modify 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.1883634.0:27894 2014-08-11 15:28:25.094621 2014-08-13 18:06:06.787289 7f5c973a57a0 20 read_log 192834'14512 (192834'14199) modify 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.1883634.0:28025 2014-08-11 15:28:25.280648 2014-08-13 18:06:06.787301 7f5c973a57a0 20 read_log 192834'14523 (192834'14221) modify 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.1883634.0:29534 2014-08-11 15:28:27.398918 2014-08-13 18:06:06.787315 7f5c973a57a0 20 read_log 192834'14524 (192834'14223) modify cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.1883634.0:29577 2014-08-11 15:28:27.459899 2014-08-13 18:06:06.787335 7f5c973a57a0 20 read_log 192834'14531 (192834'14237) modify 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.1883634.0:30198 2014-08-11 15:28:28.217969 2014-08-13 18:06:06.787350 7f5c973a57a0 20 read_log 192834'14541 (192834'14257) modify 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.1883634.0:31766 2014-08-11 15:28:30.935481 2014-08-13 18:06:06.787363 7f5c973a57a0 20 read_log 192834'14556 (192834'14287) modify 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.1883634.0:32710 2014-08-11 15:28:32.223735 2014-08-13 18:06:06.787375 7f5c973a57a0 20 read_log 192834'14565 (192834'14305) modify 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.1883634.0:34099 2014-08-11 15:28:34.229560 2014-08-13 18:06:06.787388 7f5c973a57a0 20 read_log 192834'14576 (192834'14327) modify c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.1883634.0:35233 2014-08-11 15:28:36.301181 2014-08-13 18:06:06.787400 7f5c973a57a0 20 read_log 192834'14596 (192834'14367) modify 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.1883634.0:38715 2014-08-11 15:28:41.050944 2014-08-13 18:06:06.787412 7f5c973a57a0 20 read_log 192834'14598 (192834'14371) modify 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.1883634.0:38763 2014-08-11 15:28:41.109922 2014-08-13 18:06:06.787426 7f5c973a57a0 20 read_log 192834'14603 (192834'14381) modify 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.1883634.0:39353 2014-08-11 15:28:42.048074 2014-08-13 18:06:06.787439 7f5c973a57a0 20 read_log 192834'14610 (192834'14395) modify 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.1883634.0:41039 2014-08-11 15:28:44.541057 2014-08-13 18:06:06.787469 7f5c973a57a0 20 read_log 192834'14614 (192834'14403) modify 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.1883634.0:41188 2014-08-11 15:28:44.728164 2014-08-13 18:06:06.787485 7f5c973a57a0 20 read_log 192834'14628 (192834'14431) modify 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.1883634.0:43104 2014-08-11 15:28:46.456383 2014-08-13 18:06:06.787498 7f5c973a57a0 20 read_log 192834'14634 (192834'14443) modify 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.1883634.0:43602 2014-08-11 15:28:46.604603 2014-08-13 18:06:06.787517 7f5c973a57a0 20 read_log 192834'14638 (192834'14451) modify 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.1883634.0:44371 2014-08-11 15:28:46.840803 2014-08-13 18:06:06.787535 7f5c973a57a0 20 read_log 192834'14640 (192834'14455) modify d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.1883634.0:44718 2014-08-11 15:28:46.937466 2014-08-13 18:06:06.787548 7f5c973a57a0 20 read_log 192834'14645 (192834'14465) modify 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.1883634.0:45550 2014-08-11 15:28:47.313851 2014-08-13 18:06:06.787570 7f5c973a57a0 20 read_log 192888'14651 (192888'14651) modify 1f7/hit_set_24.1f7_archive_2014-08-11 16:02:38.015319_2014-08-11 16:45:01.130851/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 16:45:01.131472 2014-08-13 18:06:06.787586 7f5c973a57a0 20 read_log 192888'14652 (0'0) promote eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by osd.4.0:21424 2014-08-11 16:45:01.137451 2014-08-13 18:06:06.787599 7f5c973a57a0 20 read_log 192888'14653 (192888'14652) modify eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.1315259.0:1529 2014-08-11 16:45:01.124452 2014-08-13 18:06:06.787615 7f5c973a57a0 20 read_log 192888'14656 (0'0) promote ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by osd.4.0:21444 2014-08-11 16:45:03.640506 2014-08-13 18:06:06.787637 7f5c973a57a0 20 read_log 192888'14657 (192888'14656) modify ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.1315259.0:2208 2014-08-11 16:45:03.631863 2014-08-13 18:06:06.787652 7f5c973a57a0 20 read_log 192888'14662 (0'0) promote 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by osd.4.0:21484 2014-08-11 16:45:08.748402 2014-08-13 18:06:06.787665 7f5c973a57a0 20 read_log 192888'14663 (192888'14662) modify 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.1315259.0:3538 2014-08-11 16:45:08.737839 2014-08-13 18:06:06.787677 7f5c973a57a0 20 read_log 192888'14666 (0'0) promote d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by osd.4.0:21494 2014-08-11 16:45:11.104741 2014-08-13 18:06:06.787689 7f5c973a57a0 20 read_log 192888'14667 (192888'14666) modify d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.1315259.0:4176 2014-08-11 16:45:11.094155 2014-08-13 18:06:06.787702 7f5c973a57a0 20 read_log 192888'14668 (0'0) promote 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by osd.4.0:21512 2014-08-11 16:45:12.989530 2014-08-13 18:06:06.787714 7f5c973a57a0 20 read_log 192888'14669 (192888'14668) modify 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.1315259.0:4696 2014-08-11 16:45:12.980696 2014-08-13 18:06:06.787726 7f5c973a57a0 20 read_log 192888'14670 (0'0) promote 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by osd.4.0:21558 2014-08-11 16:45:17.400736 2014-08-13 18:06:06.787739 7f5c973a57a0 20 read_log 192888'14671 (192888'14670) modify 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.1315259.0:5893 2014-08-11 16:45:17.392092 2014-08-13 18:06:06.787760 7f5c973a57a0 20 read_log 192888'14674 (0'0) promote e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by osd.4.0:21588 2014-08-11 16:45:22.248854 2014-08-13 18:06:06.787776 7f5c973a57a0 20 read_log 192888'14675 (192888'14674) modify e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.1315259.0:7235 2014-08-11 16:45:22.241220 2014-08-13 18:06:06.787789 7f5c973a57a0 20 read_log 192888'14682 (0'0) promote afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by osd.4.0:21634 2014-08-11 16:45:28.393684 2014-08-13 18:06:06.787801 7f5c973a57a0 20 read_log 192888'14683 (192888'14682) modify afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.1315259.0:8869 2014-08-11 16:45:28.384940 2014-08-13 18:06:06.787813 7f5c973a57a0 20 read_log 192888'14684 (0'0) promote 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by osd.4.0:21650 2014-08-11 16:45:31.671899 2014-08-13 18:06:06.787825 7f5c973a57a0 20 read_log 192888'14685 (192888'14684) modify 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.1315259.0:9723 2014-08-11 16:45:31.663840 2014-08-13 18:06:06.787838 7f5c973a57a0 20 read_log 192888'14688 (0'0) promote da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by osd.4.0:21664 2014-08-11 16:45:34.107026 2014-08-13 18:06:06.787850 7f5c973a57a0 20 read_log 192888'14689 (192888'14688) modify da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.1315259.0:10376 2014-08-11 16:45:34.098362 2014-08-13 18:06:06.787863 7f5c973a57a0 20 read_log 192888'14700 (0'0) promote 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by osd.4.0:21766 2014-08-11 16:45:49.645245 2014-08-13 18:06:06.787877 7f5c973a57a0 20 read_log 192888'14701 (192888'14700) modify 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.1315259.0:14214 2014-08-11 16:45:49.637371 2014-08-13 18:06:06.787890 7f5c973a57a0 20 read_log 192888'14706 (0'0) promote 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by osd.4.0:21834 2014-08-11 16:45:58.238915 2014-08-13 18:06:06.787919 7f5c973a57a0 20 read_log 192888'14707 (192888'14706) modify 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.1315259.0:16476 2014-08-11 16:45:58.229869 2014-08-13 18:06:06.787936 7f5c973a57a0 20 read_log 192888'14722 (0'0) promote c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by osd.4.0:21928 2014-08-11 16:46:11.232489 2014-08-13 18:06:06.787949 7f5c973a57a0 20 read_log 192888'14723 (192888'14722) modify c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.1315259.0:19757 2014-08-11 16:46:11.224215 2014-08-13 18:06:06.787964 7f5c973a57a0 20 read_log 192888'14738 (0'0) promote 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by osd.4.0:22052 2014-08-11 16:46:29.139060 2014-08-13 18:06:06.787976 7f5c973a57a0 20 read_log 192888'14739 (192888'14738) modify 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.1315259.0:24401 2014-08-11 16:46:29.129144 2014-08-13 18:06:06.787989 7f5c973a57a0 20 read_log 192888'14746 (0'0) promote 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by osd.4.0:22114 2014-08-11 16:46:37.504991 2014-08-13 18:06:06.788001 7f5c973a57a0 20 read_log 192888'14747 (192888'14746) modify 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.1315259.0:26402 2014-08-11 16:46:37.494558 2014-08-13 18:06:06.788015 7f5c973a57a0 20 read_log 192888'14748 (0'0) promote 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by osd.4.0:22116 2014-08-11 16:46:37.671491 2014-08-13 18:06:06.788030 7f5c973a57a0 20 read_log 192888'14749 (192888'14748) modify 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.1315259.0:26443 2014-08-11 16:46:37.662481 2014-08-13 18:06:06.788043 7f5c973a57a0 20 read_log 192888'14752 (0'0) promote e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by osd.4.0:22180 2014-08-11 16:46:50.944697 2014-08-13 18:06:06.788055 7f5c973a57a0 20 read_log 192888'14753 (192888'14752) modify e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.1315259.0:29689 2014-08-11 16:46:50.935440 2014-08-13 18:06:06.788067 7f5c973a57a0 20 read_log 192888'14754 (0'0) promote b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by osd.4.0:22186 2014-08-11 16:46:51.331977 2014-08-13 18:06:06.788079 7f5c973a57a0 20 read_log 192888'14755 (192888'14754) modify b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.1315259.0:29792 2014-08-11 16:46:51.323542 2014-08-13 18:06:06.788092 7f5c973a57a0 20 read_log 192888'14756 (0'0) promote 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by osd.4.0:22188 2014-08-11 16:46:51.396382 2014-08-13 18:06:06.788104 7f5c973a57a0 20 read_log 192888'14757 (192888'14756) modify 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.1315259.0:29809 2014-08-11 16:46:51.386100 2014-08-13 18:06:06.788121 7f5c973a57a0 20 read_log 192888'14760 (192888'14653) modify eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.1315259.0:32256 2014-08-11 16:46:56.329289 2014-08-13 18:06:06.788136 7f5c973a57a0 20 read_log 192888'14762 (192888'14657) modify ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.1315259.0:32935 2014-08-11 16:46:56.509221 2014-08-13 18:06:06.788160 7f5c973a57a0 20 read_log 192888'14765 (192888'14663) modify 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.1315259.0:34265 2014-08-11 16:46:56.848338 2014-08-13 18:06:06.788176 7f5c973a57a0 20 read_log 192888'14767 (192888'14667) modify d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.1315259.0:34903 2014-08-11 16:46:57.016692 2014-08-13 18:06:06.788189 7f5c973a57a0 20 read_log 192888'14768 (192888'14669) modify 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.1315259.0:35423 2014-08-11 16:46:57.160350 2014-08-13 18:06:06.788212 7f5c973a57a0 20 read_log 192888'14769 (192888'14671) modify 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.1315259.0:36620 2014-08-11 16:46:57.470481 2014-08-13 18:06:06.788237 7f5c973a57a0 20 read_log 192888'14771 (192888'14675) modify e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.1315259.0:37962 2014-08-11 16:46:57.821838 2014-08-13 18:06:06.788251 7f5c973a57a0 20 read_log 192888'14775 (192888'14683) modify afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.1315259.0:39596 2014-08-11 16:46:58.253180 2014-08-13 18:06:06.788263 7f5c973a57a0 20 read_log 192888'14776 (192888'14685) modify 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.1315259.0:40450 2014-08-11 16:46:58.489696 2014-08-13 18:06:06.788286 7f5c973a57a0 20 read_log 192888'14778 (192888'14689) modify da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.1315259.0:41103 2014-08-11 16:46:58.659832 2014-08-13 18:06:06.788331 7f5c973a57a0 20 read_log 192888'14784 (192888'14701) modify 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.1315259.0:44941 2014-08-11 16:46:59.723797 2014-08-13 18:06:06.788348 7f5c973a57a0 20 read_log 192888'14787 (192888'14707) modify 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.1315259.0:47203 2014-08-11 16:47:00.381989 2014-08-13 18:06:06.788371 7f5c973a57a0 20 read_log 192888'14795 (192888'14723) modify c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.1315259.0:50484 2014-08-11 16:47:01.383722 2014-08-13 18:06:06.788387 7f5c973a57a0 20 read_log 192888'14803 (192888'14739) modify 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.1315259.0:55128 2014-08-11 16:47:02.820551 2014-08-13 18:06:06.788400 7f5c973a57a0 20 read_log 192888'14807 (192888'14747) modify 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.1315259.0:57129 2014-08-11 16:47:03.417012 2014-08-13 18:06:06.788413 7f5c973a57a0 20 read_log 192888'14808 (192888'14749) modify 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.1315259.0:57170 2014-08-11 16:47:03.431832 2014-08-13 18:06:06.788425 7f5c973a57a0 20 read_log 192888'14810 (192888'14753) modify e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.1315259.0:60416 2014-08-11 16:47:04.437860 2014-08-13 18:06:06.788438 7f5c973a57a0 20 read_log 192888'14811 (192888'14755) modify b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.1315259.0:60519 2014-08-11 16:47:04.466312 2014-08-13 18:06:06.788450 7f5c973a57a0 20 read_log 192888'14812 (192888'14757) modify 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.1315259.0:60536 2014-08-11 16:47:04.470431 2014-08-13 18:06:06.788462 7f5c973a57a0 20 read_log 192888'14814 (192888'14814) modify 1f7/hit_set_24.1f7_archive_2014-08-11 16:45:01.131609_2014-08-11 17:05:15.751274/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.751520 2014-08-13 18:06:06.788477 7f5c973a57a0 20 read_log 192888'14815 (192888'14651) delete 1f7/hit_set_24.1f7_archive_2014-08-11 16:02:38.015319_2014-08-11 16:45:01.130851/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 17:05:15.751520 2014-08-13 18:06:06.788489 7f5c973a57a0 20 read_log 192888'14818 (0'0) promote 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by osd.4.0:22657 2014-08-11 17:05:17.232592 2014-08-13 18:06:06.788502 7f5c973a57a0 20 read_log 192888'14819 (192888'14818) modify 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.942868.0:538 2014-08-11 17:05:17.225955 2014-08-13 18:06:06.788514 7f5c973a57a0 20 read_log 192888'14820 (0'0) promote b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by osd.4.0:22673 2014-08-11 17:05:19.614261 2014-08-13 18:06:06.788534 7f5c973a57a0 20 read_log 192888'14821 (192888'14820) modify b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.942868.0:1113 2014-08-11 17:05:19.605145 2014-08-13 18:06:06.788549 7f5c973a57a0 20 read_log 192888'14826 (0'0) promote d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by osd.4.0:22732 2014-08-11 17:05:28.543938 2014-08-13 18:06:06.788562 7f5c973a57a0 20 read_log 192888'14827 (192888'14826) modify d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.942868.0:3406 2014-08-11 17:05:28.533546 2014-08-13 18:06:06.788574 7f5c973a57a0 20 read_log 192888'14828 (0'0) promote aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by osd.4.0:22746 2014-08-11 17:05:29.553338 2014-08-13 18:06:06.788587 7f5c973a57a0 20 read_log 192888'14829 (192888'14828) modify aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.942868.0:3679 2014-08-11 17:05:29.544629 2014-08-13 18:06:06.788599 7f5c973a57a0 20 read_log 192888'14838 (0'0) promote 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by osd.4.0:22804 2014-08-11 17:05:36.699043 2014-08-13 18:06:06.788611 7f5c973a57a0 20 read_log 192888'14839 (192888'14838) modify 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.942868.0:5555 2014-08-11 17:05:36.689188 2014-08-13 18:06:06.788624 7f5c973a57a0 20 read_log 192888'14840 (0'0) promote 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by osd.4.0:22822 2014-08-11 17:05:38.818034 2014-08-13 18:06:06.788636 7f5c973a57a0 20 read_log 192888'14841 (192888'14840) modify 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.942868.0:6139 2014-08-11 17:05:38.810729 2014-08-13 18:06:06.788648 7f5c973a57a0 20 read_log 192888'14844 (0'0) promote aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by osd.4.0:22838 2014-08-11 17:05:40.767160 2014-08-13 18:06:06.788660 7f5c973a57a0 20 read_log 192888'14845 (192888'14844) modify aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.942868.0:6679 2014-08-11 17:05:40.757606 2014-08-13 18:06:06.788672 7f5c973a57a0 20 read_log 192888'14846 (0'0) promote 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by osd.4.0:22842 2014-08-11 17:05:41.412893 2014-08-13 18:06:06.788684 7f5c973a57a0 20 read_log 192888'14847 (192888'14846) modify 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.942868.0:6858 2014-08-11 17:05:41.402121 2014-08-13 18:06:06.788697 7f5c973a57a0 20 read_log 192888'14850 (0'0) promote 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by osd.4.0:22864 2014-08-11 17:05:45.310125 2014-08-13 18:06:06.788709 7f5c973a57a0 20 read_log 192888'14851 (192888'14850) modify 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.942868.0:7856 2014-08-11 17:05:45.302618 2014-08-13 18:06:06.788721 7f5c973a57a0 20 read_log 192888'14852 (0'0) promote ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by osd.4.0:22870 2014-08-11 17:05:45.946651 2014-08-13 18:06:06.788733 7f5c973a57a0 20 read_log 192888'14853 (192888'14852) modify ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.942868.0:8033 2014-08-11 17:05:45.938110 2014-08-13 18:06:06.788746 7f5c973a57a0 20 read_log 192888'14860 (0'0) promote 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by osd.4.0:22940 2014-08-11 17:05:56.974245 2014-08-13 18:06:06.788758 7f5c973a57a0 20 read_log 192888'14861 (192888'14860) modify 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.942868.0:11050 2014-08-11 17:05:56.967386 2014-08-13 18:06:06.788770 7f5c973a57a0 20 read_log 192888'14862 (0'0) promote ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by osd.4.0:22980 2014-08-11 17:06:03.937206 2014-08-13 18:06:06.788808 7f5c973a57a0 20 read_log 192888'14863 (192888'14862) modify ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.942868.0:12930 2014-08-11 17:06:03.928616 2014-08-13 18:06:06.788821 7f5c973a57a0 20 read_log 192888'14874 (0'0) promote 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by osd.4.0:23076 2014-08-11 17:06:19.780915 2014-08-13 18:06:06.788834 7f5c973a57a0 20 read_log 192888'14875 (192888'14874) modify 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.942868.0:16925 2014-08-11 17:06:19.772447 2014-08-13 18:06:06.788846 7f5c973a57a0 20 read_log 192888'14876 (0'0) promote ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by osd.4.0:23100 2014-08-11 17:06:21.945987 2014-08-13 18:06:06.788859 7f5c973a57a0 20 read_log 192888'14877 (192888'14876) modify ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.942868.0:17498 2014-08-11 17:06:21.937416 2014-08-13 18:06:06.788874 7f5c973a57a0 20 read_log 192888'14878 (0'0) promote fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by osd.4.0:23110 2014-08-11 17:06:23.041178 2014-08-13 18:06:06.788886 7f5c973a57a0 20 read_log 192888'14879 (192888'14878) modify fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.942868.0:17789 2014-08-11 17:06:23.033636 2014-08-13 18:06:06.788901 7f5c973a57a0 20 read_log 192888'14880 (0'0) promote 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by osd.4.0:23114 2014-08-11 17:06:23.123137 2014-08-13 18:06:06.788913 7f5c973a57a0 20 read_log 192888'14881 (192888'14880) modify 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.942868.0:17802 2014-08-11 17:06:23.082988 2014-08-13 18:06:06.788925 7f5c973a57a0 20 read_log 192888'14884 (0'0) promote c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by osd.4.0:23156 2014-08-11 17:06:30.000437 2014-08-13 18:06:06.788938 7f5c973a57a0 20 read_log 192888'14885 (192888'14884) modify c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.942868.0:19640 2014-08-11 17:06:29.991526 2014-08-13 18:06:06.788952 7f5c973a57a0 20 read_log 192888'14886 (0'0) promote 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by osd.4.0:23164 2014-08-11 17:06:30.618152 2014-08-13 18:06:06.788965 7f5c973a57a0 20 read_log 192888'14887 (192888'14886) modify 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.942868.0:19802 2014-08-11 17:06:30.608262 2014-08-13 18:06:06.788979 7f5c973a57a0 20 read_log 192888'14888 (0'0) promote a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by osd.4.0:23168 2014-08-11 17:06:31.153709 2014-08-13 18:06:06.788991 7f5c973a57a0 20 read_log 192888'14889 (192888'14888) modify a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.942868.0:19943 2014-08-11 17:06:31.144512 2014-08-13 18:06:06.789005 7f5c973a57a0 20 read_log 192888'14900 (0'0) promote e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by osd.4.0:23258 2014-08-11 17:06:44.500430 2014-08-13 18:06:06.789022 7f5c973a57a0 20 read_log 192888'14901 (192888'14900) modify e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.942868.0:23400 2014-08-11 17:06:44.493770 2014-08-13 18:06:06.789055 7f5c973a57a0 20 read_log 192888'14902 (0'0) promote 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by osd.4.0:23262 2014-08-11 17:06:44.706016 2014-08-13 18:06:06.789068 7f5c973a57a0 20 read_log 192888'14903 (192888'14902) modify 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.942868.0:23456 2014-08-11 17:06:44.698521 2014-08-13 18:06:06.789083 7f5c973a57a0 20 read_log 192888'14904 (0'0) promote 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by osd.4.0:23264 2014-08-11 17:06:45.114518 2014-08-13 18:06:06.789102 7f5c973a57a0 20 read_log 192888'14905 (192888'14904) modify 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.942868.0:23567 2014-08-11 17:06:45.105658 2014-08-13 18:06:06.789117 7f5c973a57a0 20 read_log 192888'14908 (0'0) promote ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by osd.4.0:23284 2014-08-11 17:06:47.065367 2014-08-13 18:06:06.789130 7f5c973a57a0 20 read_log 192888'14909 (192888'14908) modify ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.942868.0:24096 2014-08-11 17:06:47.056802 2014-08-13 18:06:06.789144 7f5c973a57a0 20 read_log 192888'14914 (0'0) promote a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by osd.4.0:23322 2014-08-11 17:06:50.193248 2014-08-13 18:06:06.789171 7f5c973a57a0 20 read_log 192888'14915 (192888'14914) modify a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.942868.0:24942 2014-08-11 17:06:50.185171 2014-08-13 18:06:06.789184 7f5c973a57a0 20 read_log 192888'14916 (0'0) promote e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by osd.4.0:23364 2014-08-11 17:06:57.272548 2014-08-13 18:06:06.789198 7f5c973a57a0 20 read_log 192888'14917 (192888'14916) modify e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.942868.0:26759 2014-08-11 17:06:57.264487 2014-08-13 18:06:06.789211 7f5c973a57a0 20 read_log 192888'14918 (0'0) promote 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by osd.4.0:23408 2014-08-11 17:07:04.778465 2014-08-13 18:06:06.789223 7f5c973a57a0 20 read_log 192888'14919 (192888'14918) modify 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.942868.0:28758 2014-08-11 17:07:04.769415 2014-08-13 18:06:06.789235 7f5c973a57a0 20 read_log 192888'14920 (0'0) promote 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by osd.4.0:23424 2014-08-11 17:07:06.981522 2014-08-13 18:06:06.789248 7f5c973a57a0 20 read_log 192888'14921 (192888'14920) modify 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.942868.0:29306 2014-08-11 17:07:06.973191 2014-08-13 18:06:06.789266 7f5c973a57a0 20 read_log 192888'14926 (0'0) promote 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by osd.4.0:23458 2014-08-11 17:07:12.294353 2014-08-13 18:06:06.789280 7f5c973a57a0 20 read_log 192888'14927 (192888'14926) modify 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.942868.0:30616 2014-08-11 17:07:12.285702 2014-08-13 18:06:06.789294 7f5c973a57a0 20 read_log 192888'14928 (0'0) promote c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by osd.4.0:23462 2014-08-11 17:07:12.485101 2014-08-13 18:06:06.789310 7f5c973a57a0 20 read_log 192888'14929 (192888'14928) modify c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.942868.0:30662 2014-08-11 17:07:12.474532 2014-08-13 18:06:06.789325 7f5c973a57a0 20 read_log 192888'14930 (0'0) promote f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by osd.4.0:23474 2014-08-11 17:07:13.276466 2014-08-13 18:06:06.789337 7f5c973a57a0 20 read_log 192888'14931 (192888'14930) modify f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.942868.0:30864 2014-08-11 17:07:13.267493 2014-08-13 18:06:06.789352 7f5c973a57a0 20 read_log 192888'14932 (0'0) promote e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by osd.4.0:23476 2014-08-11 17:07:13.602559 2014-08-13 18:06:06.789367 7f5c973a57a0 20 read_log 192888'14933 (192888'14932) modify e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.942868.0:30948 2014-08-11 17:07:13.594572 2014-08-13 18:06:06.789380 7f5c973a57a0 20 read_log 192888'14935 (192888'14819) modify 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.942868.0:31804 2014-08-11 17:07:16.062574 2014-08-13 18:06:06.789403 7f5c973a57a0 20 read_log 192888'14936 (192888'14821) modify b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.942868.0:32379 2014-08-11 17:07:16.237557 2014-08-13 18:06:06.789419 7f5c973a57a0 20 read_log 192888'14939 (192888'14827) modify d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.942868.0:34672 2014-08-11 17:07:16.861140 2014-08-13 18:06:06.789431 7f5c973a57a0 20 read_log 192888'14940 (192888'14829) modify aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.942868.0:34945 2014-08-11 17:07:16.939951 2014-08-13 18:06:06.789444 7f5c973a57a0 20 read_log 192888'14945 (192888'14839) modify 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.942868.0:36821 2014-08-11 17:07:17.442503 2014-08-13 18:06:06.789466 7f5c973a57a0 20 read_log 192888'14946 (192888'14841) modify 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.942868.0:37405 2014-08-11 17:07:17.600445 2014-08-13 18:06:06.789482 7f5c973a57a0 20 read_log 192888'14948 (192888'14845) modify aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.942868.0:37945 2014-08-11 17:07:17.750289 2014-08-13 18:06:06.789495 7f5c973a57a0 20 read_log 192888'14949 (192888'14847) modify 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.942868.0:38124 2014-08-11 17:07:17.798448 2014-08-13 18:06:06.789514 7f5c973a57a0 20 read_log 192888'14951 (192888'14851) modify 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.942868.0:39122 2014-08-11 17:07:18.087802 2014-08-13 18:06:06.789528 7f5c973a57a0 20 read_log 192888'14952 (192888'14853) modify ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.942868.0:39299 2014-08-11 17:07:18.142708 2014-08-13 18:06:06.789541 7f5c973a57a0 20 read_log 192888'14956 (192888'14861) modify 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.942868.0:42316 2014-08-11 17:07:18.966394 2014-08-13 18:06:06.789559 7f5c973a57a0 20 read_log 192888'14957 (192888'14863) modify ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.942868.0:44196 2014-08-11 17:07:19.497197 2014-08-13 18:06:06.789575 7f5c973a57a0 20 read_log 192888'14963 (192888'14875) modify 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.942868.0:48191 2014-08-11 17:07:20.650144 2014-08-13 18:06:06.789595 7f5c973a57a0 20 read_log 192888'14964 (192888'14877) modify ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.942868.0:48764 2014-08-11 17:07:20.810594 2014-08-13 18:06:06.789618 7f5c973a57a0 20 read_log 192888'14965 (192888'14879) modify fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.942868.0:49055 2014-08-11 17:07:20.894083 2014-08-13 18:06:06.789632 7f5c973a57a0 20 read_log 192888'14966 (192888'14881) modify 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.942868.0:49068 2014-08-11 17:07:20.897643 2014-08-13 18:06:06.789644 7f5c973a57a0 20 read_log 192888'14968 (192888'14885) modify c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.942868.0:50906 2014-08-11 17:07:21.468683 2014-08-13 18:06:06.789657 7f5c973a57a0 20 read_log 192888'14969 (192888'14887) modify 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.942868.0:51068 2014-08-11 17:07:21.521317 2014-08-13 18:06:06.789669 7f5c973a57a0 20 read_log 192888'14970 (192888'14889) modify a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.942868.0:51209 2014-08-11 17:07:21.573517 2014-08-13 18:06:06.789682 7f5c973a57a0 20 read_log 192888'14976 (192888'14901) modify e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.942868.0:54666 2014-08-11 17:07:22.622521 2014-08-13 18:06:06.789712 7f5c973a57a0 20 read_log 192888'14977 (192888'14903) modify 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.942868.0:54722 2014-08-11 17:07:22.637972 2014-08-13 18:06:06.789746 7f5c973a57a0 20 read_log 192888'14978 (192888'14905) modify 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.942868.0:54833 2014-08-11 17:07:22.668688 2014-08-13 18:06:06.789768 7f5c973a57a0 20 read_log 192888'14980 (192888'14909) modify ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.942868.0:55362 2014-08-11 17:07:22.823609 2014-08-13 18:06:06.789783 7f5c973a57a0 20 read_log 192888'14983 (192888'14915) modify a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.942868.0:56208 2014-08-11 17:07:23.081292 2014-08-13 18:06:06.789796 7f5c973a57a0 20 read_log 192888'14984 (192888'14917) modify e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.942868.0:58025 2014-08-11 17:07:23.621993 2014-08-13 18:06:06.789820 7f5c973a57a0 20 read_log 192888'14985 (192888'14919) modify 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.942868.0:60024 2014-08-11 17:07:24.219467 2014-08-13 18:06:06.789832 7f5c973a57a0 20 read_log 192888'14986 (192888'14921) modify 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.942868.0:60572 2014-08-11 17:07:24.379311 2014-08-13 18:06:06.789845 7f5c973a57a0 20 read_log 192888'14989 (192888'14927) modify 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.942868.0:61882 2014-08-11 17:07:24.784729 2014-08-13 18:06:06.789857 7f5c973a57a0 20 read_log 192888'14990 (192888'14929) modify c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.942868.0:61928 2014-08-11 17:07:24.797523 2014-08-13 18:06:06.789869 7f5c973a57a0 20 read_log 192888'14991 (192888'14931) modify f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.942868.0:62130 2014-08-11 17:07:24.857221 2014-08-13 18:06:06.789882 7f5c973a57a0 20 read_log 192888'14992 (192888'14933) modify e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.942868.0:62214 2014-08-11 17:07:24.885517 2014-08-13 18:06:06.789894 7f5c973a57a0 20 read_log 193305'14993 (193305'14993) modify 1f7/hit_set_24.1f7_archive_2014-08-11 18:37:04.910739_2014-08-11 18:42:41.143618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.143869 2014-08-13 18:06:06.789907 7f5c973a57a0 20 read_log 193305'14994 (192888'14814) delete 1f7/hit_set_24.1f7_archive_2014-08-11 16:45:01.131609_2014-08-11 17:05:15.751274/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.143869 2014-08-13 18:06:06.789920 7f5c973a57a0 20 read_log 193305'14995 (0'0) promote d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by osd.4.0:31 2014-08-11 18:42:41.222357 2014-08-13 18:06:06.789932 7f5c973a57a0 20 read_log 193305'14996 (193305'14995) modify d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.951337.0:2106 2014-08-11 18:42:41.138103 2014-08-13 18:06:06.789944 7f5c973a57a0 20 read_log 193305'14997 (0'0) promote 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by osd.4.0:47 2014-08-11 18:42:45.283409 2014-08-13 18:06:06.789957 7f5c973a57a0 20 read_log 193305'14998 (193305'14997) modify 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.951337.0:3243 2014-08-11 18:42:45.273764 2014-08-13 18:06:06.789969 7f5c973a57a0 20 read_log 193305'14999 (0'0) promote ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by osd.4.0:73 2014-08-11 18:42:52.320494 2014-08-13 18:06:06.789981 7f5c973a57a0 20 read_log 193305'15000 (193305'14999) modify ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.951337.0:5144 2014-08-11 18:42:52.310978 2014-08-13 18:06:06.789996 7f5c973a57a0 20 read_log 193305'15001 (0'0) promote a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by osd.4.0:75 2014-08-11 18:42:52.499244 2014-08-13 18:06:06.790016 7f5c973a57a0 20 read_log 193305'15002 (193305'15001) modify a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.951337.0:5194 2014-08-11 18:42:52.492771 2014-08-13 18:06:06.790031 7f5c973a57a0 20 read_log 193305'15003 (0'0) promote 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by osd.4.0:79 2014-08-11 18:42:53.382798 2014-08-13 18:06:06.790054 7f5c973a57a0 20 read_log 193305'15004 (193305'15003) modify 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.951337.0:5438 2014-08-11 18:42:53.375125 2014-08-13 18:06:06.790068 7f5c973a57a0 20 read_log 193305'15005 (0'0) promote cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by osd.4.0:87 2014-08-11 18:42:54.323998 2014-08-13 18:06:06.790080 7f5c973a57a0 20 read_log 193305'15006 (193305'15005) modify cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.951337.0:5696 2014-08-11 18:42:54.317314 2014-08-13 18:06:06.790092 7f5c973a57a0 20 read_log 193305'15007 (0'0) promote 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by osd.4.0:99 2014-08-11 18:42:56.249677 2014-08-13 18:06:06.790105 7f5c973a57a0 20 read_log 193305'15008 (193305'15007) modify 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.951337.0:6226 2014-08-11 18:42:56.241508 2014-08-13 18:06:06.790117 7f5c973a57a0 20 read_log 193305'15009 (0'0) promote 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by osd.4.0:109 2014-08-11 18:42:58.169244 2014-08-13 18:06:06.790129 7f5c973a57a0 20 read_log 193305'15010 (193305'15009) modify 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.951337.0:6757 2014-08-11 18:42:58.161290 2014-08-13 18:06:06.790141 7f5c973a57a0 20 read_log 193305'15011 (0'0) promote 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by osd.4.0:125 2014-08-11 18:43:01.278500 2014-08-13 18:06:06.790161 7f5c973a57a0 20 read_log 193305'15012 (193305'15011) modify 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.951337.0:7617 2014-08-11 18:43:01.271122 2014-08-13 18:06:06.790174 7f5c973a57a0 20 read_log 193305'15013 (0'0) promote 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by osd.4.0:127 2014-08-11 18:43:01.722052 2014-08-13 18:06:06.790189 7f5c973a57a0 20 read_log 193305'15014 (193305'15013) modify 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.951337.0:7740 2014-08-11 18:43:01.714852 2014-08-13 18:06:06.790201 7f5c973a57a0 20 read_log 193305'15015 (0'0) promote 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by osd.4.0:169 2014-08-11 18:43:10.984877 2014-08-13 18:06:06.790213 7f5c973a57a0 20 read_log 193305'15016 (193305'15015) modify 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.951337.0:10191 2014-08-11 18:43:10.976643 2014-08-13 18:06:06.790226 7f5c973a57a0 20 read_log 193305'15017 (0'0) promote 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by osd.4.0:189 2014-08-11 18:43:15.418084 2014-08-13 18:06:06.790238 7f5c973a57a0 20 read_log 193305'15018 (193305'15017) modify 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.951337.0:11332 2014-08-11 18:43:15.410732 2014-08-13 18:06:06.790250 7f5c973a57a0 20 read_log 193305'15019 (0'0) promote bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by osd.4.0:251 2014-08-11 18:43:31.736696 2014-08-13 18:06:06.790263 7f5c973a57a0 20 read_log 193305'15020 (193305'15019) modify bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.951337.0:15815 2014-08-11 18:43:31.729666 2014-08-13 18:06:06.790275 7f5c973a57a0 20 read_log 193305'15021 (0'0) promote b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by osd.4.0:279 2014-08-11 18:43:35.693000 2014-08-13 18:06:06.790295 7f5c973a57a0 20 read_log 193305'15022 (193305'15021) modify b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.951337.0:16926 2014-08-11 18:43:35.683871 2014-08-13 18:06:06.790313 7f5c973a57a0 20 read_log 193305'15023 (0'0) promote f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by osd.4.0:321 2014-08-11 18:43:43.252502 2014-08-13 18:06:06.790325 7f5c973a57a0 20 read_log 193305'15024 (193305'15023) modify f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.951337.0:19072 2014-08-11 18:43:43.245115 2014-08-13 18:06:06.790340 7f5c973a57a0 20 read_log 193305'15025 (0'0) promote 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by osd.4.0:337 2014-08-11 18:43:46.952101 2014-08-13 18:06:06.790357 7f5c973a57a0 20 read_log 193305'15026 (193305'15025) modify 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.951337.0:20091 2014-08-11 18:43:46.944424 2014-08-13 18:06:06.790372 7f5c973a57a0 20 read_log 193305'15027 (0'0) promote 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by osd.4.0:353 2014-08-11 18:43:51.774124 2014-08-13 18:06:06.790384 7f5c973a57a0 20 read_log 193305'15028 (193305'15027) modify 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.951337.0:21434 2014-08-11 18:43:51.766763 2014-08-13 18:06:06.790397 7f5c973a57a0 20 read_log 193305'15029 (0'0) promote ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by osd.4.0:355 2014-08-11 18:43:52.308846 2014-08-13 18:06:06.790413 7f5c973a57a0 20 read_log 193305'15030 (193305'15029) modify ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.951337.0:21590 2014-08-11 18:43:52.302416 2014-08-13 18:06:06.790425 7f5c973a57a0 20 read_log 193305'15031 (0'0) promote 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by osd.4.0:367 2014-08-11 18:43:53.947261 2014-08-13 18:06:06.790437 7f5c973a57a0 20 read_log 193305'15032 (193305'15031) modify 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.951337.0:22040 2014-08-11 18:43:53.939735 2014-08-13 18:06:06.790453 7f5c973a57a0 20 read_log 193305'15033 (0'0) promote e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by osd.4.0:401 2014-08-11 18:44:01.496760 2014-08-13 18:06:06.790468 7f5c973a57a0 20 read_log 193305'15034 (193305'15033) modify e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.951337.0:24124 2014-08-11 18:44:01.488379 2014-08-13 18:06:06.790480 7f5c973a57a0 20 read_log 193305'15035 (0'0) promote 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by osd.4.0:417 2014-08-11 18:44:05.809263 2014-08-13 18:06:06.790493 7f5c973a57a0 20 read_log 193305'15036 (193305'15035) modify 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.951337.0:25320 2014-08-11 18:44:05.801152 2014-08-13 18:06:06.790507 7f5c973a57a0 20 read_log 193305'15037 (0'0) promote 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by osd.4.0:439 2014-08-11 18:44:10.928508 2014-08-13 18:06:06.790520 7f5c973a57a0 20 read_log 193305'15038 (193305'15037) modify 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.951337.0:26705 2014-08-11 18:44:10.920478 2014-08-13 18:06:06.790550 7f5c973a57a0 20 read_log 193305'15039 (0'0) promote e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by osd.4.0:479 2014-08-11 18:44:19.453838 2014-08-13 18:06:06.790563 7f5c973a57a0 20 read_log 193305'15040 (193305'15039) modify e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.951337.0:29031 2014-08-11 18:44:19.446713 2014-08-13 18:06:06.790583 7f5c973a57a0 20 read_log 193305'15041 (0'0) promote cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by osd.4.0:487 2014-08-11 18:44:20.941237 2014-08-13 18:06:06.790609 7f5c973a57a0 20 read_log 193305'15042 (193305'15041) modify cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.951337.0:29438 2014-08-11 18:44:20.934369 2014-08-13 18:06:06.790635 7f5c973a57a0 20 read_log 193305'15043 (0'0) promote d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by osd.4.0:519 2014-08-11 18:44:29.677422 2014-08-13 18:06:06.790652 7f5c973a57a0 20 read_log 193305'15044 (193305'15043) modify d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.951337.0:31824 2014-08-11 18:44:29.669878 2014-08-13 18:06:06.790665 7f5c973a57a0 20 read_log 193305'15045 (0'0) promote 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by osd.4.0:549 2014-08-11 18:44:32.070950 2014-08-13 18:06:06.790685 7f5c973a57a0 20 read_log 193305'15046 (193305'15045) modify 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.951337.0:32495 2014-08-11 18:44:32.059692 2014-08-13 18:06:06.790701 7f5c973a57a0 20 read_log 193305'15047 (193305'14996) modify d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.951337.0:34817 2014-08-11 18:44:35.208564 2014-08-13 18:06:06.790714 7f5c973a57a0 20 read_log 193305'15048 (193305'14998) modify 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.951337.0:35954 2014-08-11 18:44:35.827425 2014-08-13 18:06:06.790727 7f5c973a57a0 20 read_log 193305'15049 (193305'15000) modify ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.951337.0:37855 2014-08-11 18:44:37.085215 2014-08-13 18:06:06.790739 7f5c973a57a0 20 read_log 193305'15050 (193305'15002) modify a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.951337.0:37905 2014-08-11 18:44:37.109933 2014-08-13 18:06:06.790752 7f5c973a57a0 20 read_log 193305'15051 (193305'15004) modify 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.951337.0:38149 2014-08-11 18:44:37.262858 2014-08-13 18:06:06.790764 7f5c973a57a0 20 read_log 193305'15052 (193305'15006) modify cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.951337.0:38407 2014-08-11 18:44:37.410498 2014-08-13 18:06:06.790776 7f5c973a57a0 20 read_log 193305'15053 (193305'15008) modify 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.951337.0:38937 2014-08-11 18:44:37.741804 2014-08-13 18:06:06.790789 7f5c973a57a0 20 read_log 193305'15054 (193305'15010) modify 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.951337.0:39468 2014-08-11 18:44:38.027184 2014-08-13 18:06:06.790801 7f5c973a57a0 20 read_log 193305'15055 (193305'15012) modify 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.951337.0:40328 2014-08-11 18:44:38.376904 2014-08-13 18:06:06.790813 7f5c973a57a0 20 read_log 193305'15056 (193305'15014) modify 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.951337.0:40451 2014-08-11 18:44:38.418583 2014-08-13 18:06:06.790826 7f5c973a57a0 20 read_log 193305'15057 (193305'15016) modify 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.951337.0:42902 2014-08-11 18:44:39.417558 2014-08-13 18:06:06.790838 7f5c973a57a0 20 read_log 193305'15058 (193305'15018) modify 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.951337.0:44043 2014-08-11 18:44:39.777976 2014-08-13 18:06:06.790850 7f5c973a57a0 20 read_log 193305'15059 (193305'15020) modify bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.951337.0:48526 2014-08-11 18:44:41.079157 2014-08-13 18:06:06.790863 7f5c973a57a0 20 read_log 193305'15060 (193305'15022) modify b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.951337.0:49637 2014-08-11 18:44:41.425291 2014-08-13 18:06:06.790883 7f5c973a57a0 20 read_log 193305'15061 (193305'15024) modify f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.951337.0:51783 2014-08-11 18:44:42.048338 2014-08-13 18:06:06.790923 7f5c973a57a0 20 read_log 193305'15062 (193305'15026) modify 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.951337.0:52802 2014-08-11 18:44:42.345286 2014-08-13 18:06:06.790938 7f5c973a57a0 20 read_log 193305'15063 (193305'15028) modify 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.951337.0:54145 2014-08-11 18:44:42.742455 2014-08-13 18:06:06.790957 7f5c973a57a0 20 read_log 193305'15064 (193305'15030) modify ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.951337.0:54301 2014-08-11 18:44:42.791353 2014-08-13 18:06:06.790970 7f5c973a57a0 20 read_log 193305'15065 (193305'15032) modify 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.951337.0:54751 2014-08-11 18:44:42.928778 2014-08-13 18:06:06.790982 7f5c973a57a0 20 read_log 193305'15066 (193305'15034) modify e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.951337.0:56835 2014-08-11 18:44:43.545384 2014-08-13 18:06:06.790995 7f5c973a57a0 20 read_log 193305'15067 (193305'15036) modify 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.951337.0:58031 2014-08-11 18:44:43.927930 2014-08-13 18:06:06.791007 7f5c973a57a0 20 read_log 193305'15068 (193305'15038) modify 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.951337.0:59416 2014-08-11 18:44:44.321043 2014-08-13 18:06:06.791019 7f5c973a57a0 20 read_log 193305'15069 (193305'15040) modify e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.951337.0:61742 2014-08-11 18:44:45.015522 2014-08-13 18:06:06.791032 7f5c973a57a0 20 read_log 193305'15070 (193305'15042) modify cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.951337.0:62149 2014-08-11 18:44:45.140090 2014-08-13 18:06:06.791044 7f5c973a57a0 20 read_log 193305'15071 (193305'15044) modify d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.951337.0:64535 2014-08-11 18:44:45.841030 2014-08-13 18:06:06.791056 7f5c973a57a0 20 read_log 193305'15072 (193305'15046) modify 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.951337.0:65206 2014-08-11 18:44:46.050385 2014-08-13 18:06:06.791080 7f5c973a57a0 20 read_log 194215'15073 (194215'15073) modify 1f7/hit_set_24.1f7_archive_2014-08-12 15:32:25.864872_2014-08-12 15:44:02.507691/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:02.507944 2014-08-13 18:06:06.791095 7f5c973a57a0 20 read_log 194215'15074 (193305'14993) delete 1f7/hit_set_24.1f7_archive_2014-08-11 18:37:04.910739_2014-08-11 18:42:41.143618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:02.507944 2014-08-13 18:06:06.791154 7f5c973a57a0 20 read_log 194215'15075 (192888'14966) clean 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by osd.4.0:2248 2014-08-12 15:44:02.512493 2014-08-13 18:06:06.791172 7f5c973a57a0 20 read_log 194215'15076 (194215'15075) delete 4e3b01f7/benchmark_data_storage0111-ib.pocstor_2228_object17801/head//24 by client.2174595.0:104012 0.000000 2014-08-13 18:06:06.791183 7f5c973a57a0 20 read_log 194215'15077 (192834'14576) clean c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by osd.4.0:2250 2014-08-12 15:44:02.541271 2014-08-13 18:06:06.791196 7f5c973a57a0 20 read_log 194215'15078 (194215'15077) delete c6dc01f7/benchmark_data_storage0111-ib.pocstor_41526_object11908/head//24 by client.2174595.0:104014 0.000000 2014-08-13 18:06:06.791207 7f5c973a57a0 20 read_log 194215'15079 (192834'13970) clean e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by osd.4.0:2252 2014-08-12 15:44:02.550334 2014-08-13 18:06:06.791222 7f5c973a57a0 20 read_log 194215'15080 (194215'15079) delete e45521f7/benchmark_data_storage0111-ib.pocstor_26659_object1078/head//24 by client.2174595.0:104016 0.000000 2014-08-13 18:06:06.791242 7f5c973a57a0 20 read_log 194215'15081 (192888'14812) clean 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by osd.4.0:2254 2014-08-12 15:44:02.586098 2014-08-13 18:06:06.791257 7f5c973a57a0 20 read_log 194215'15082 (194215'15081) delete 7be721f7/benchmark_data_storage0111-ib.pocstor_41097_object29808/head//24 by client.2174595.0:104018 0.000000 2014-08-13 18:06:06.791272 7f5c973a57a0 20 read_log 194215'15083 (192834'14103) clean 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by osd.4.0:2256 2014-08-12 15:44:02.595563 2014-08-13 18:06:06.791284 7f5c973a57a0 20 read_log 194215'15084 (194215'15083) delete 9cb441f7/benchmark_data_storage0111-ib.pocstor_26659_object18273/head//24 by client.2174595.0:104020 0.000000 2014-08-13 18:06:06.791295 7f5c973a57a0 20 read_log 194215'15085 (192834'14634) clean 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by osd.4.0:2258 2014-08-12 15:44:02.605767 2014-08-13 18:06:06.791307 7f5c973a57a0 20 read_log 194215'15086 (194215'15085) delete 8b7551f7/benchmark_data_storage0111-ib.pocstor_41526_object20277/head//24 by client.2174595.0:104022 0.000000 2014-08-13 18:06:06.791322 7f5c973a57a0 20 read_log 194215'15087 (193305'15060) clean b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by osd.4.0:2260 2014-08-12 15:44:02.614268 2014-08-13 18:06:06.791337 7f5c973a57a0 20 read_log 194215'15088 (194215'15087) delete b39651f7/benchmark_data_storage0111-ib.pocstor_62621_object16925/head//24 by client.2174595.0:104024 0.000000 2014-08-13 18:06:06.791348 7f5c973a57a0 20 read_log 194215'15089 (192834'14603) clean 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by osd.4.0:2262 2014-08-12 15:44:02.622526 2014-08-13 18:06:06.791363 7f5c973a57a0 20 read_log 194215'15090 (194215'15089) delete 726d61f7/benchmark_data_storage0111-ib.pocstor_41526_object16028/head//24 by client.2174595.0:104026 0.000000 2014-08-13 18:06:06.791374 7f5c973a57a0 20 read_log 194215'15091 (192888'14978) clean 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by osd.4.0:2264 2014-08-12 15:44:02.630356 2014-08-13 18:06:06.791399 7f5c973a57a0 20 read_log 194215'15092 (194215'15091) delete 641a71f7/benchmark_data_storage0111-ib.pocstor_2228_object23566/head//24 by client.2174595.0:104028 0.000000 2014-08-13 18:06:06.791414 7f5c973a57a0 20 read_log 194215'15093 (192888'14940) clean aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by osd.4.0:2266 2014-08-12 15:44:02.667077 2014-08-13 18:06:06.791427 7f5c973a57a0 20 read_log 194215'15094 (194215'15093) delete aa2081f7/benchmark_data_storage0111-ib.pocstor_2228_object3678/head//24 by client.2174595.0:104030 0.000000 2014-08-13 18:06:06.791445 7f5c973a57a0 20 read_log 194215'15095 (192888'14970) clean a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by osd.4.0:2268 2014-08-12 15:44:02.675458 2014-08-13 18:06:06.791461 7f5c973a57a0 20 read_log 194215'15096 (194215'15095) delete a3c591f7/benchmark_data_storage0111-ib.pocstor_2228_object19942/head//24 by client.2174595.0:104032 0.000000 2014-08-13 18:06:06.791503 7f5c973a57a0 20 read_log 194215'15097 (192834'14614) clean 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by osd.4.0:2270 2014-08-12 15:44:02.683377 2014-08-13 18:06:06.791517 7f5c973a57a0 20 read_log 194215'15098 (194215'15097) delete 8b7791f7/benchmark_data_storage0111-ib.pocstor_41526_object17863/head//24 by client.2174595.0:104034 0.000000 2014-08-13 18:06:06.791528 7f5c973a57a0 20 read_log 194215'15099 (192834'14054) clean a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by osd.4.0:2272 2014-08-12 15:44:02.691485 2014-08-13 18:06:06.791558 7f5c973a57a0 20 read_log 194215'15100 (194215'15099) delete a015b1f7/benchmark_data_storage0111-ib.pocstor_26659_object12435/head//24 by client.2174595.0:104036 0.000000 2014-08-13 18:06:06.791580 7f5c973a57a0 20 read_log 194215'15101 (192888'14991) clean f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by osd.4.0:2274 2014-08-12 15:44:02.700061 2014-08-13 18:06:06.791596 7f5c973a57a0 20 read_log 194215'15102 (194215'15101) delete f138b1f7/benchmark_data_storage0111-ib.pocstor_2228_object30863/head//24 by client.2174595.0:104038 0.000000 2014-08-13 18:06:06.791607 7f5c973a57a0 20 read_log 194215'15103 (192888'14803) clean 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by osd.4.0:2276 2014-08-12 15:44:02.708816 2014-08-13 18:06:06.791620 7f5c973a57a0 20 read_log 194215'15104 (194215'15103) delete 7fbeb1f7/benchmark_data_storage0111-ib.pocstor_41097_object24400/head//24 by client.2174595.0:104040 0.000000 2014-08-13 18:06:06.791630 7f5c973a57a0 20 read_log 194215'15105 (192834'14510) clean 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by osd.4.0:2278 2014-08-12 15:44:02.717168 2014-08-13 18:06:06.791643 7f5c973a57a0 20 read_log 194215'15106 (194215'15105) delete 66d1c1f7/benchmark_data_storage0111-ib.pocstor_41526_object4569/head//24 by client.2174595.0:104042 0.000000 2014-08-13 18:06:06.791654 7f5c973a57a0 20 read_log 194215'15107 (192834'14098) clean 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by osd.4.0:2280 2014-08-12 15:44:02.735265 2014-08-13 18:06:06.791666 7f5c973a57a0 20 read_log 194215'15108 (194215'15107) delete 3ae2c1f7/benchmark_data_storage0111-ib.pocstor_26659_object17455/head//24 by client.2174595.0:104044 0.000000 2014-08-13 18:06:06.791677 7f5c973a57a0 20 read_log 194215'15109 (192888'14775) clean afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by osd.4.0:2282 2014-08-12 15:44:02.743935 2014-08-13 18:06:06.791689 7f5c973a57a0 20 read_log 194215'15110 (194215'15109) delete afb6c1f7/benchmark_data_storage0111-ib.pocstor_41097_object8868/head//24 by client.2174595.0:104046 0.000000 2014-08-13 18:06:06.791700 7f5c973a57a0 20 read_log 194215'15111 (193305'15059) clean bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by osd.4.0:2284 2014-08-12 15:44:02.753086 2014-08-13 18:06:06.791712 7f5c973a57a0 20 read_log 194215'15112 (194215'15111) delete bcd6c1f7/benchmark_data_storage0111-ib.pocstor_62621_object15814/head//24 by client.2174595.0:104048 0.000000 2014-08-13 18:06:06.791723 7f5c973a57a0 20 read_log 194215'15113 (193305'15047) clean d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by osd.4.0:2286 2014-08-12 15:44:02.761976 2014-08-13 18:06:06.791735 7f5c973a57a0 20 read_log 194215'15114 (194215'15113) delete d275d1f7/benchmark_data_storage0111-ib.pocstor_62621_object2105/head//24 by client.2174595.0:104050 0.000000 2014-08-13 18:06:06.791746 7f5c973a57a0 20 read_log 194215'15115 (192888'14963) clean 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by osd.4.0:2288 2014-08-12 15:44:02.771311 2014-08-13 18:06:06.791758 7f5c973a57a0 20 read_log 194215'15116 (194215'15115) delete 8839d1f7/benchmark_data_storage0111-ib.pocstor_2228_object16924/head//24 by client.2174595.0:104052 0.000000 2014-08-13 18:06:06.791769 7f5c973a57a0 20 read_log 194215'15117 (192888'14976) clean e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by osd.4.0:2290 2014-08-12 15:44:02.780392 2014-08-13 18:06:06.791781 7f5c973a57a0 20 read_log 194215'15118 (194215'15117) delete e240e1f7/benchmark_data_storage0111-ib.pocstor_2228_object23399/head//24 by client.2174595.0:104054 0.000000 2014-08-13 18:06:06.791792 7f5c973a57a0 20 read_log 194215'15119 (192834'14076) clean 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by osd.4.0:2292 2014-08-12 15:44:02.789265 2014-08-13 18:06:06.791804 7f5c973a57a0 20 read_log 194215'15120 (194215'15119) delete 19e5f1f7/benchmark_data_storage0111-ib.pocstor_26659_object14509/head//24 by client.2174595.0:104056 0.000000 2014-08-13 18:06:06.791822 7f5c973a57a0 20 read_log 194215'15121 (192834'14041) clean 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by osd.4.0:2294 2014-08-12 15:44:02.798142 2014-08-13 18:06:06.791835 7f5c973a57a0 20 read_log 194215'15122 (194215'15121) delete 2718f1f7/benchmark_data_storage0111-ib.pocstor_26659_object10021/head//24 by client.2174595.0:104058 0.000000 2014-08-13 18:06:06.791846 7f5c973a57a0 20 read_log 194215'15123 (192834'14075) clean c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by osd.4.0:2296 2014-08-12 15:44:02.806616 2014-08-13 18:06:06.791858 7f5c973a57a0 20 read_log 194215'15124 (194215'15123) delete c2cff1f7/benchmark_data_storage0111-ib.pocstor_26659_object14441/head//24 by client.2174595.0:104060 0.000000 2014-08-13 18:06:06.791869 7f5c973a57a0 20 read_log 194215'15125 (192834'14112) clean ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by osd.4.0:2298 2014-08-12 15:44:02.836751 2014-08-13 18:06:06.791881 7f5c973a57a0 20 read_log 194215'15126 (194215'15125) delete ea0105f7/benchmark_data_storage0111-ib.pocstor_26659_object19113/head//24 by client.2174595.0:104062 0.000000 2014-08-13 18:06:06.791892 7f5c973a57a0 20 read_log 194215'15127 (192834'14494) clean f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by osd.4.0:2300 2014-08-12 15:44:02.846685 2014-08-13 18:06:06.791904 7f5c973a57a0 20 read_log 194215'15128 (194215'15127) delete f36905f7/benchmark_data_storage0111-ib.pocstor_41526_object2354/head//24 by client.2174595.0:104064 0.000000 2014-08-13 18:06:06.791915 7f5c973a57a0 20 read_log 194215'15129 (192834'14500) clean 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by osd.4.0:2302 2014-08-12 15:44:02.857282 2014-08-13 18:06:06.791927 7f5c973a57a0 20 read_log 194215'15130 (194215'15129) delete 863d05f7/benchmark_data_storage0111-ib.pocstor_41526_object2901/head//24 by client.2174595.0:104066 0.000000 2014-08-13 18:06:06.791938 7f5c973a57a0 20 read_log 194215'15131 (192888'14948) clean aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by osd.4.0:2304 2014-08-12 15:44:02.866193 2014-08-13 18:06:06.791951 7f5c973a57a0 20 read_log 194215'15132 (194215'15131) delete aa7715f7/benchmark_data_storage0111-ib.pocstor_2228_object6678/head//24 by client.2174595.0:104068 0.000000 2014-08-13 18:06:06.791961 7f5c973a57a0 20 read_log 194215'15133 (192834'14610) clean 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by osd.4.0:2306 2014-08-12 15:44:02.874731 2014-08-13 18:06:06.791974 7f5c973a57a0 20 read_log 194215'15134 (194215'15133) delete 194815f7/benchmark_data_storage0111-ib.pocstor_41526_object17714/head//24 by client.2174595.0:104070 0.000000 2014-08-13 18:06:06.791984 7f5c973a57a0 20 read_log 194215'15135 (192888'14810) clean e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by osd.4.0:2308 2014-08-12 15:44:02.883159 2014-08-13 18:06:06.791997 7f5c973a57a0 20 read_log 194215'15136 (194215'15135) delete e81f15f7/benchmark_data_storage0111-ib.pocstor_41097_object29688/head//24 by client.2174595.0:104072 0.000000 2014-08-13 18:06:06.792007 7f5c973a57a0 20 read_log 194215'15137 (192834'14024) clean db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by osd.4.0:2310 2014-08-12 15:44:02.891621 2014-08-13 18:06:06.792019 7f5c973a57a0 20 read_log 194215'15138 (194215'15137) delete db335f7/benchmark_data_storage0111-ib.pocstor_26659_object8174/head//24 by client.2174595.0:104074 0.000000 2014-08-13 18:06:06.792032 7f5c973a57a0 20 read_log 194215'15139 (192834'14512) clean 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by osd.4.0:2312 2014-08-12 15:44:02.900289 2014-08-13 18:06:06.792045 7f5c973a57a0 20 read_log 194215'15140 (194215'15139) delete 688735f7/benchmark_data_storage0111-ib.pocstor_41526_object4700/head//24 by client.2174595.0:104076 0.000000 2014-08-13 18:06:06.792065 7f5c973a57a0 20 read_log 194215'15141 (192888'14949) clean 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by osd.4.0:2314 2014-08-12 15:44:02.908203 2014-08-13 18:06:06.792080 7f5c973a57a0 20 read_log 194215'15142 (194215'15141) delete 64b835f7/benchmark_data_storage0111-ib.pocstor_2228_object6857/head//24 by client.2174595.0:104078 0.000000 2014-08-13 18:06:06.792091 7f5c973a57a0 20 read_log 194215'15143 (192834'14598) clean 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by osd.4.0:2316 2014-08-12 15:44:02.916460 2014-08-13 18:06:06.792103 7f5c973a57a0 20 read_log 194215'15144 (194215'15143) delete 31cb35f7/benchmark_data_storage0111-ib.pocstor_41526_object15438/head//24 by client.2174595.0:104080 0.000000 2014-08-13 18:06:06.792115 7f5c973a57a0 20 read_log 194215'15145 (192834'14108) clean 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by osd.4.0:2318 2014-08-12 15:44:02.924655 2014-08-13 18:06:06.792129 7f5c973a57a0 20 read_log 194215'15146 (194215'15145) delete 7e5a45f7/benchmark_data_storage0111-ib.pocstor_26659_object18883/head//24 by client.2174595.0:104082 0.000000 2014-08-13 18:06:06.792140 7f5c973a57a0 20 read_log 194215'15147 (192834'14645) clean 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by osd.4.0:2320 2014-08-12 15:44:02.932540 2014-08-13 18:06:06.792161 7f5c973a57a0 20 read_log 194215'15148 (194215'15147) delete 8fda55f7/benchmark_data_storage0111-ib.pocstor_41526_object22225/head//24 by client.2174595.0:104084 0.000000 2014-08-13 18:06:06.792173 7f5c973a57a0 20 read_log 194215'15149 (193305'15053) clean 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by osd.4.0:2322 2014-08-12 15:44:02.940118 2014-08-13 18:06:06.792186 7f5c973a57a0 20 read_log 194215'15150 (194215'15149) delete 5cf965f7/benchmark_data_storage0111-ib.pocstor_62621_object6225/head//24 by client.2174595.0:104086 0.000000 2014-08-13 18:06:06.792197 7f5c973a57a0 20 read_log 194215'15151 (192834'14565) clean 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by osd.4.0:2324 2014-08-12 15:44:02.948004 2014-08-13 18:06:06.792209 7f5c973a57a0 20 read_log 194215'15152 (194215'15151) delete 641e65f7/benchmark_data_storage0111-ib.pocstor_41526_object10774/head//24 by client.2174595.0:104088 0.000000 2014-08-13 18:06:06.792220 7f5c973a57a0 20 read_log 194215'15153 (192888'14957) clean ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by osd.4.0:2326 2014-08-12 15:44:02.955977 2014-08-13 18:06:06.792232 7f5c973a57a0 20 read_log 194215'15154 (194215'15153) delete ee65f7/benchmark_data_storage0111-ib.pocstor_2228_object12929/head//24 by client.2174595.0:104090 0.000000 2014-08-13 18:06:06.792246 7f5c973a57a0 20 read_log 194215'15155 (192888'14939) clean d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by osd.4.0:2328 2014-08-12 15:44:02.972360 2014-08-13 18:06:06.792261 7f5c973a57a0 20 read_log 194215'15156 (194215'15155) delete d8f65f7/benchmark_data_storage0111-ib.pocstor_2228_object3405/head//24 by client.2174595.0:104092 0.000000 2014-08-13 18:06:06.792272 7f5c973a57a0 20 read_log 194215'15157 (192888'14936) clean b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by osd.4.0:2330 2014-08-12 15:44:02.980570 2014-08-13 18:06:06.792284 7f5c973a57a0 20 read_log 194215'15158 (194215'15157) delete b4685f7/benchmark_data_storage0111-ib.pocstor_2228_object1112/head//24 by client.2174595.0:104094 0.000000 2014-08-13 18:06:06.792298 7f5c973a57a0 20 read_log 194215'15159 (192834'13978) clean c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by osd.4.0:2332 2014-08-12 15:44:02.989452 2014-08-13 18:06:06.792311 7f5c973a57a0 20 read_log 194215'15160 (194215'15159) delete c6685f7/benchmark_data_storage0111-ib.pocstor_26659_object2099/head//24 by client.2174595.0:104096 0.000000 2014-08-13 18:06:06.792331 7f5c973a57a0 20 read_log 194215'15161 (192834'14524) clean cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by osd.4.0:2334 2014-08-12 15:44:02.997215 2014-08-13 18:06:06.792350 7f5c973a57a0 20 read_log 194215'15162 (194215'15161) delete cedc85f7/benchmark_data_storage0111-ib.pocstor_41526_object6252/head//24 by client.2174595.0:104098 0.000000 2014-08-13 18:06:06.792361 7f5c973a57a0 20 read_log 194215'15163 (192888'14951) clean 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by osd.4.0:2336 2014-08-12 15:44:03.032392 2014-08-13 18:06:06.792386 7f5c973a57a0 20 read_log 194215'15164 (194215'15163) delete 48ae95f7/benchmark_data_storage0111-ib.pocstor_2228_object7855/head//24 by client.2174595.0:104100 0.000000 2014-08-13 18:06:06.792404 7f5c973a57a0 20 read_log 194215'15165 (192834'14640) clean d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by osd.4.0:2338 2014-08-12 15:44:03.042609 2014-08-13 18:06:06.792423 7f5c973a57a0 20 read_log 194215'15166 (194215'15165) delete d11a5f7/benchmark_data_storage0111-ib.pocstor_41526_object21393/head//24 by client.2174595.0:104102 0.000000 2014-08-13 18:06:06.792438 7f5c973a57a0 20 read_log 194215'15167 (192834'14087) clean 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by osd.4.0:2340 2014-08-12 15:44:03.051650 2014-08-13 18:06:06.792477 7f5c973a57a0 20 read_log 194215'15168 (194215'15167) delete 4e73a5f7/benchmark_data_storage0111-ib.pocstor_26659_object15467/head//24 by client.2174595.0:104104 0.000000 2014-08-13 18:06:06.792492 7f5c973a57a0 20 read_log 194215'15169 (192888'14952) clean ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by osd.4.0:2342 2014-08-12 15:44:03.060366 2014-08-13 18:06:06.792505 7f5c973a57a0 20 read_log 194215'15170 (194215'15169) delete ffd8a5f7/benchmark_data_storage0111-ib.pocstor_2228_object8032/head//24 by client.2174595.0:104106 0.000000 2014-08-13 18:06:06.792523 7f5c973a57a0 20 read_log 194215'15171 (192888'14968) clean c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by osd.4.0:2344 2014-08-12 15:44:03.068676 2014-08-13 18:06:06.792537 7f5c973a57a0 20 read_log 194215'15172 (194215'15171) delete c356b5f7/benchmark_data_storage0111-ib.pocstor_2228_object19639/head//24 by client.2174595.0:104108 0.000000 2014-08-13 18:06:06.792548 7f5c973a57a0 20 read_log 194215'15173 (193305'15055) clean 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by osd.4.0:2346 2014-08-12 15:44:03.077376 2014-08-13 18:06:06.792561 7f5c973a57a0 20 read_log 194215'15174 (194215'15173) delete 72fbb5f7/benchmark_data_storage0111-ib.pocstor_62621_object7616/head//24 by client.2174595.0:104110 0.000000 2014-08-13 18:06:06.792587 7f5c973a57a0 20 read_log 194215'15175 (192834'14628) clean 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by osd.4.0:2348 2014-08-12 15:44:03.086032 2014-08-13 18:06:06.792600 7f5c973a57a0 20 read_log 194215'15176 (194215'15175) delete 27e1c5f7/benchmark_data_storage0111-ib.pocstor_41526_object19779/head//24 by client.2174595.0:104112 0.000000 2014-08-13 18:06:06.792611 7f5c973a57a0 20 read_log 194215'15177 (193305'15051) clean 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by osd.4.0:2350 2014-08-12 15:44:03.095004 2014-08-13 18:06:06.792624 7f5c973a57a0 20 read_log 194215'15178 (194215'15177) delete 9a92c5f7/benchmark_data_storage0111-ib.pocstor_62621_object5437/head//24 by client.2174595.0:104114 0.000000 2014-08-13 18:06:06.792635 7f5c973a57a0 20 read_log 194215'15179 (193305'15072) clean 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by osd.4.0:2352 2014-08-12 15:44:03.103807 2014-08-13 18:06:06.792647 7f5c973a57a0 20 read_log 194215'15180 (194215'15179) delete 803cc5f7/benchmark_data_storage0111-ib.pocstor_62621_object32494/head//24 by client.2174595.0:104116 0.000000 2014-08-13 18:06:06.792666 7f5c973a57a0 20 read_log 194215'15181 (192834'14079) clean ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by osd.4.0:2354 2014-08-12 15:44:03.111259 2014-08-13 18:06:06.792681 7f5c973a57a0 20 read_log 194215'15182 (194215'15181) delete ec97d5f7/benchmark_data_storage0111-ib.pocstor_26659_object14929/head//24 by client.2174595.0:104118 0.000000 2014-08-13 18:06:06.792692 7f5c973a57a0 20 read_log 194215'15183 (192834'14106) clean 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by osd.4.0:2356 2014-08-12 15:44:03.118558 2014-08-13 18:06:06.792705 7f5c973a57a0 20 read_log 194215'15184 (194215'15183) delete 49b0e5f7/benchmark_data_storage0111-ib.pocstor_26659_object18445/head//24 by client.2174595.0:104120 0.000000 2014-08-13 18:06:06.792715 7f5c973a57a0 20 read_log 194215'15185 (192888'14989) clean 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by osd.4.0:2358 2014-08-12 15:44:03.126172 2014-08-13 18:06:06.792728 7f5c973a57a0 20 read_log 194215'15186 (194215'15185) delete 1697e5f7/benchmark_data_storage0111-ib.pocstor_2228_object30615/head//24 by client.2174595.0:104122 0.000000 2014-08-13 18:06:06.792739 7f5c973a57a0 20 read_log 194215'15187 (192834'14596) clean 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by osd.4.0:2360 2014-08-12 15:44:03.133598 2014-08-13 18:06:06.792751 7f5c973a57a0 20 read_log 194215'15188 (194215'15187) delete 9e09e5f7/benchmark_data_storage0111-ib.pocstor_41526_object15390/head//24 by client.2174595.0:104124 0.000000 2014-08-13 18:06:06.792762 7f5c973a57a0 20 read_log 194215'15189 (193305'15048) clean 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by osd.4.0:2362 2014-08-12 15:44:03.141831 2014-08-13 18:06:06.792774 7f5c973a57a0 20 read_log 194215'15190 (194215'15189) delete 8bf1f5f7/benchmark_data_storage0111-ib.pocstor_62621_object3242/head//24 by client.2174595.0:104126 0.000000 2014-08-13 18:06:06.792785 7f5c973a57a0 20 read_log 194215'15191 (192888'14767) clean d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by osd.4.0:2364 2014-08-12 15:44:03.149479 2014-08-13 18:06:06.792797 7f5c973a57a0 20 read_log 194215'15192 (194215'15191) delete d677f5f7/benchmark_data_storage0111-ib.pocstor_41097_object4175/head//24 by client.2174595.0:104128 0.000000 2014-08-13 18:06:06.792808 7f5c973a57a0 20 read_log 194215'15193 (192888'14946) clean 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by osd.4.0:2366 2014-08-12 15:44:03.158866 2014-08-13 18:06:06.792820 7f5c973a57a0 20 read_log 194215'15194 (194215'15193) delete 51bff5f7/benchmark_data_storage0111-ib.pocstor_2228_object6138/head//24 by client.2174595.0:104130 0.000000 2014-08-13 18:06:06.792831 7f5c973a57a0 20 read_log 194215'15195 (193305'15070) clean cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by osd.4.0:2368 2014-08-12 15:44:03.167449 2014-08-13 18:06:06.792843 7f5c973a57a0 20 read_log 194215'15196 (194215'15195) delete cfac19f7/benchmark_data_storage0111-ib.pocstor_62621_object29437/head//24 by client.2174595.0:104132 0.000000 2014-08-13 18:06:06.792854 7f5c973a57a0 20 read_log 194215'15197 (192834'14061) clean d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by osd.4.0:2370 2014-08-12 15:44:03.175283 2014-08-13 18:06:06.792887 7f5c973a57a0 20 read_log 194215'15198 (194215'15197) delete d82e29f7/benchmark_data_storage0111-ib.pocstor_26659_object13060/head//24 by client.2174595.0:104134 0.000000 2014-08-13 18:06:06.792902 7f5c973a57a0 20 read_log 194215'15199 (192834'14541) clean 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by osd.4.0:2372 2014-08-12 15:44:03.183456 2014-08-13 18:06:06.792915 7f5c973a57a0 20 read_log 194215'15200 (194215'15199) delete 986139f7/benchmark_data_storage0111-ib.pocstor_41526_object8441/head//24 by client.2174595.0:104136 0.000000 2014-08-13 18:06:06.792953 7f5c973a57a0 20 read_log 194215'15201 (192888'14769) clean 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by osd.4.0:2374 2014-08-12 15:44:03.192452 2014-08-13 18:06:06.792969 7f5c973a57a0 20 read_log 194215'15202 (194215'15201) delete 2de549f7/benchmark_data_storage0111-ib.pocstor_41097_object5892/head//24 by client.2174595.0:104138 0.000000 2014-08-13 18:06:06.792980 7f5c973a57a0 20 read_log 194215'15203 (192888'14778) clean da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by osd.4.0:2376 2014-08-12 15:44:03.200895 2014-08-13 18:06:06.792998 7f5c973a57a0 20 read_log 194215'15204 (194215'15203) delete da7949f7/benchmark_data_storage0111-ib.pocstor_41097_object10375/head//24 by client.2174595.0:104140 0.000000 2014-08-13 18:06:06.793009 7f5c973a57a0 20 read_log 194215'15205 (192888'14992) clean e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by osd.4.0:2378 2014-08-12 15:44:03.208990 2014-08-13 18:06:06.793022 7f5c973a57a0 20 read_log 194215'15206 (194215'15205) delete e8de49f7/benchmark_data_storage0111-ib.pocstor_2228_object30947/head//24 by client.2174595.0:104142 0.000000 2014-08-13 18:06:06.793034 7f5c973a57a0 20 read_log 194215'15207 (192834'14492) clean 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by osd.4.0:2380 2014-08-12 15:44:03.217426 2014-08-13 18:06:06.793047 7f5c973a57a0 20 read_log 194215'15208 (194215'15207) delete 8c7059f7/benchmark_data_storage0111-ib.pocstor_41526_object1876/head//24 by client.2174595.0:104144 0.000000 2014-08-13 18:06:06.793060 7f5c973a57a0 20 read_log 194215'15209 (192888'14768) clean 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by osd.4.0:2382 2014-08-12 15:44:03.226590 2014-08-13 18:06:06.793073 7f5c973a57a0 20 read_log 194215'15210 (194215'15209) delete 2a2459f7/benchmark_data_storage0111-ib.pocstor_41097_object4695/head//24 by client.2174595.0:104146 0.000000 2014-08-13 18:06:06.793084 7f5c973a57a0 20 read_log 194215'15211 (193305'15068) clean 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by osd.4.0:2384 2014-08-12 15:44:03.235713 2014-08-13 18:06:06.793096 7f5c973a57a0 20 read_log 194215'15212 (194215'15211) delete 435d59f7/benchmark_data_storage0111-ib.pocstor_62621_object26704/head//24 by client.2174595.0:104148 0.000000 2014-08-13 18:06:06.793109 7f5c973a57a0 20 read_log 194215'15213 (193305'15052) clean cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by osd.4.0:2386 2014-08-12 15:44:03.244646 2014-08-13 18:06:06.793122 7f5c973a57a0 20 read_log 194215'15214 (194215'15213) delete cb1369f7/benchmark_data_storage0111-ib.pocstor_62621_object5695/head//24 by client.2174595.0:104150 0.000000 2014-08-13 18:06:06.793135 7f5c973a57a0 20 read_log 194215'15215 (192888'14935) clean 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by osd.4.0:2388 2014-08-12 15:44:03.253110 2014-08-13 18:06:06.793153 7f5c973a57a0 20 read_log 194215'15216 (194215'15215) delete 894569f7/benchmark_data_storage0111-ib.pocstor_2228_object537/head//24 by client.2174595.0:104152 0.000000 2014-08-13 18:06:06.793167 7f5c973a57a0 20 read_log 194215'15217 (192888'14776) clean 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by osd.4.0:2390 2014-08-12 15:44:03.260910 2014-08-13 18:06:06.793179 7f5c973a57a0 20 read_log 194215'15218 (194215'15217) delete 889289f7/benchmark_data_storage0111-ib.pocstor_41097_object9722/head//24 by client.2174595.0:104154 0.000000 2014-08-13 18:06:06.793190 7f5c973a57a0 20 read_log 194215'15219 (192888'14811) clean b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by osd.4.0:2392 2014-08-12 15:44:03.269464 2014-08-13 18:06:06.793205 7f5c973a57a0 20 read_log 194215'15220 (194215'15219) delete b8e489f7/benchmark_data_storage0111-ib.pocstor_41097_object29791/head//24 by client.2174595.0:104156 0.000000 2014-08-13 18:06:06.793225 7f5c973a57a0 20 read_log 194215'15221 (192888'14956) clean 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by osd.4.0:2394 2014-08-12 15:44:03.291809 2014-08-13 18:06:06.793240 7f5c973a57a0 20 read_log 194215'15222 (194215'15221) delete 6a2889f7/benchmark_data_storage0111-ib.pocstor_2228_object11049/head//24 by client.2174595.0:104158 0.000000 2014-08-13 18:06:06.793254 7f5c973a57a0 20 read_log 194215'15223 (193305'15062) clean 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by osd.4.0:2396 2014-08-12 15:44:03.299478 2014-08-13 18:06:06.793267 7f5c973a57a0 20 read_log 194215'15224 (194215'15223) delete 438599f7/benchmark_data_storage0111-ib.pocstor_62621_object20090/head//24 by client.2174595.0:104160 0.000000 2014-08-13 18:06:06.793278 7f5c973a57a0 20 read_log 194215'15225 (193305'15065) clean 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by osd.4.0:2398 2014-08-12 15:44:03.306949 2014-08-13 18:06:06.793290 7f5c973a57a0 20 read_log 194215'15226 (194215'15225) delete 6799f7/benchmark_data_storage0111-ib.pocstor_62621_object22039/head//24 by client.2174595.0:104162 0.000000 2014-08-13 18:06:06.793300 7f5c973a57a0 20 read_log 194215'15227 (193305'15067) clean 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by osd.4.0:2400 2014-08-12 15:44:03.315203 2014-08-13 18:06:06.793313 7f5c973a57a0 20 read_log 194215'15228 (194215'15227) delete 1be899f7/benchmark_data_storage0111-ib.pocstor_62621_object25319/head//24 by client.2174595.0:104164 0.000000 2014-08-13 18:06:06.793324 7f5c973a57a0 20 read_log 194215'15229 (192834'14638) clean 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by osd.4.0:2402 2014-08-12 15:44:03.322606 2014-08-13 18:06:06.793336 7f5c973a57a0 20 read_log 194215'15230 (194215'15229) delete 48ba99f7/benchmark_data_storage0111-ib.pocstor_41526_object21046/head//24 by client.2174595.0:104166 0.000000 2014-08-13 18:06:06.793347 7f5c973a57a0 20 read_log 194215'15231 (193305'15049) clean ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by osd.4.0:2404 2014-08-12 15:44:03.330642 2014-08-13 18:06:06.793359 7f5c973a57a0 20 read_log 194215'15232 (194215'15231) delete ae90a9f7/benchmark_data_storage0111-ib.pocstor_62621_object5143/head//24 by client.2174595.0:104168 0.000000 2014-08-13 18:06:06.793370 7f5c973a57a0 20 read_log 194215'15233 (192834'14099) clean 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by osd.4.0:2406 2014-08-12 15:44:03.338339 2014-08-13 18:06:06.793391 7f5c973a57a0 20 read_log 194215'15234 (194215'15233) delete 79d0a9f7/benchmark_data_storage0111-ib.pocstor_26659_object17493/head//24 by client.2174595.0:104170 0.000000 2014-08-13 18:06:06.793409 7f5c973a57a0 20 read_log 194215'15235 (193305'15066) clean e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by osd.4.0:2408 2014-08-12 15:44:03.346674 2014-08-13 18:06:06.793460 7f5c973a57a0 20 read_log 194215'15236 (194215'15235) delete e712a9f7/benchmark_data_storage0111-ib.pocstor_62621_object24123/head//24 by client.2174595.0:104172 0.000000 2014-08-13 18:06:06.793472 7f5c973a57a0 20 read_log 194215'15237 (193305'15063) clean 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by osd.4.0:2410 2014-08-12 15:44:03.354170 2014-08-13 18:06:06.793504 7f5c973a57a0 20 read_log 194215'15238 (194215'15237) delete 423da9f7/benchmark_data_storage0111-ib.pocstor_62621_object21433/head//24 by client.2174595.0:104174 0.000000 2014-08-13 18:06:06.793533 7f5c973a57a0 20 read_log 194215'15239 (192888'14784) clean 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by osd.4.0:2412 2014-08-12 15:44:03.362518 2014-08-13 18:06:06.793560 7f5c973a57a0 20 read_log 194215'15240 (194215'15239) delete 247b9f7/benchmark_data_storage0111-ib.pocstor_41097_object14213/head//24 by client.2174595.0:104176 0.000000 2014-08-13 18:06:06.793580 7f5c973a57a0 20 read_log 194215'15241 (193305'15054) clean 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by osd.4.0:2414 2014-08-12 15:44:03.369904 2014-08-13 18:06:06.793643 7f5c973a57a0 20 read_log 194215'15242 (194215'15241) delete 11e8b9f7/benchmark_data_storage0111-ib.pocstor_62621_object6756/head//24 by client.2174595.0:104178 0.000000 2014-08-13 18:06:06.793662 7f5c973a57a0 20 read_log 194215'15243 (192888'14969) clean 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by osd.4.0:2416 2014-08-12 15:44:03.377827 2014-08-13 18:06:06.793688 7f5c973a57a0 20 read_log 194215'15244 (194215'15243) delete 7646c9f7/benchmark_data_storage0111-ib.pocstor_2228_object19801/head//24 by client.2174595.0:104180 0.000000 2014-08-13 18:06:06.793705 7f5c973a57a0 20 read_log 194215'15245 (192834'14053) clean bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by osd.4.0:2418 2014-08-12 15:44:03.387091 2014-08-13 18:06:06.793737 7f5c973a57a0 20 read_log 194215'15246 (194215'15245) delete bb54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object12074/head//24 by client.2174595.0:104182 0.000000 2014-08-13 18:06:06.793794 7f5c973a57a0 20 read_log 194215'15247 (192834'13992) clean 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by osd.4.0:2420 2014-08-12 15:44:03.395359 2014-08-13 18:06:06.793814 7f5c973a57a0 20 read_log 194215'15248 (194215'15247) delete 5e54e9f7/benchmark_data_storage0111-ib.pocstor_26659_object4797/head//24 by client.2174595.0:104184 0.000000 2014-08-13 18:06:06.793829 7f5c973a57a0 20 read_log 194215'15249 (192834'14117) clean 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by osd.4.0:2422 2014-08-12 15:44:03.403634 2014-08-13 18:06:06.793844 7f5c973a57a0 20 read_log 194215'15250 (194215'15249) delete 2888e9f7/benchmark_data_storage0111-ib.pocstor_26659_object19630/head//24 by client.2174595.0:104186 0.000000 2014-08-13 18:06:06.793855 7f5c973a57a0 20 read_log 194215'15251 (192834'14055) clean e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by osd.4.0:2424 2014-08-12 15:44:03.412255 2014-08-13 18:06:06.793884 7f5c973a57a0 20 read_log 194215'15252 (194215'15251) delete e827f9f7/benchmark_data_storage0111-ib.pocstor_26659_object12462/head//24 by client.2174595.0:104188 0.000000 2014-08-13 18:06:06.793899 7f5c973a57a0 20 read_log 194215'15253 (192888'14787) clean 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by osd.4.0:2426 2014-08-12 15:44:03.420859 2014-08-13 18:06:06.793942 7f5c973a57a0 20 read_log 194215'15254 (194215'15253) delete 923af9f7/benchmark_data_storage0111-ib.pocstor_41097_object16475/head//24 by client.2174595.0:104190 0.000000 2014-08-13 18:06:06.793954 7f5c973a57a0 20 read_log 194215'15255 (192888'14760) clean eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by osd.4.0:2428 2014-08-12 15:44:03.429002 2014-08-13 18:06:06.793969 7f5c973a57a0 20 read_log 194215'15256 (194215'15255) delete eccf9f7/benchmark_data_storage0111-ib.pocstor_41097_object1528/head//24 by client.2174595.0:104192 0.000000 2014-08-13 18:06:06.793980 7f5c973a57a0 20 read_log 194215'15257 (192888'14795) clean c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by osd.4.0:2430 2014-08-12 15:44:03.437346 2014-08-13 18:06:06.793994 7f5c973a57a0 20 read_log 194215'15258 (194215'15257) delete c74b0df7/benchmark_data_storage0111-ib.pocstor_41097_object19756/head//24 by client.2174595.0:104194 0.000000 2014-08-13 18:06:06.794005 7f5c973a57a0 20 read_log 194215'15259 (192888'14980) clean ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by osd.4.0:2432 2014-08-12 15:44:03.445462 2014-08-13 18:06:06.794018 7f5c973a57a0 20 read_log 194215'15260 (194215'15259) delete ab651df7/benchmark_data_storage0111-ib.pocstor_2228_object24095/head//24 by client.2174595.0:104196 0.000000 2014-08-13 18:06:06.794056 7f5c973a57a0 20 read_log 194215'15261 (193305'15050) clean a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by osd.4.0:2434 2014-08-12 15:44:03.454869 2014-08-13 18:06:06.794072 7f5c973a57a0 20 read_log 194215'15262 (194215'15261) delete a7912df7/benchmark_data_storage0111-ib.pocstor_62621_object5193/head//24 by client.2174595.0:104198 0.000000 2014-08-13 18:06:06.794084 7f5c973a57a0 20 read_log 194215'15263 (192834'14080) clean e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by osd.4.0:2436 2014-08-12 15:44:03.463002 2014-08-13 18:06:06.794096 7f5c973a57a0 20 read_log 194215'15264 (194215'15263) delete e9432df7/benchmark_data_storage0111-ib.pocstor_26659_object14958/head//24 by client.2174595.0:104200 0.000000 2014-08-13 18:06:06.794107 7f5c973a57a0 20 read_log 194215'15265 (193305'15057) clean 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by osd.4.0:2438 2014-08-12 15:44:03.471356 2014-08-13 18:06:06.794120 7f5c973a57a0 20 read_log 194215'15266 (194215'15265) delete 28682df7/benchmark_data_storage0111-ib.pocstor_62621_object10190/head//24 by client.2174595.0:104202 0.000000 2014-08-13 18:06:06.794131 7f5c973a57a0 20 read_log 194215'15267 (192888'14765) clean 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by osd.4.0:2440 2014-08-12 15:44:03.479308 2014-08-13 18:06:06.794143 7f5c973a57a0 20 read_log 194215'15268 (194215'15267) delete 34373df7/benchmark_data_storage0111-ib.pocstor_41097_object3537/head//24 by client.2174595.0:104204 0.000000 2014-08-13 18:06:06.794162 7f5c973a57a0 20 read_log 194215'15269 (192888'14985) clean 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by osd.4.0:2442 2014-08-12 15:44:03.488081 2014-08-13 18:06:06.794175 7f5c973a57a0 20 read_log 194215'15270 (194215'15269) delete 66fd3df7/benchmark_data_storage0111-ib.pocstor_2228_object28757/head//24 by client.2174595.0:104206 0.000000 2014-08-13 18:06:06.794186 7f5c973a57a0 20 read_log 194215'15271 (192834'14523) clean 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by osd.4.0:2444 2014-08-12 15:44:03.495459 2014-08-13 18:06:06.794199 7f5c973a57a0 20 read_log 194215'15272 (194215'15271) delete 2edf3df7/benchmark_data_storage0111-ib.pocstor_41526_object6209/head//24 by client.2174595.0:104208 0.000000 2014-08-13 18:06:06.794210 7f5c973a57a0 20 read_log 194215'15273 (192834'13981) clean 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by osd.4.0:2446 2014-08-12 15:44:03.503297 2014-08-13 18:06:06.794222 7f5c973a57a0 20 read_log 194215'15274 (194215'15273) delete 16994df7/benchmark_data_storage0111-ib.pocstor_26659_object2396/head//24 by client.2174595.0:104210 0.000000 2014-08-13 18:06:06.794233 7f5c973a57a0 20 read_log 194215'15275 (192888'14965) clean fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by osd.4.0:2448 2014-08-12 15:44:03.525853 2014-08-13 18:06:06.794245 7f5c973a57a0 20 read_log 194215'15276 (194215'15275) delete fbc95df7/benchmark_data_storage0111-ib.pocstor_2228_object17788/head//24 by client.2174595.0:104212 0.000000 2014-08-13 18:06:06.794259 7f5c973a57a0 20 read_log 194215'15277 (192888'14984) clean e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by osd.4.0:2450 2014-08-12 15:44:03.576684 2014-08-13 18:06:06.794272 7f5c973a57a0 20 read_log 194215'15278 (194215'15277) delete e37d5df7/benchmark_data_storage0111-ib.pocstor_2228_object26758/head//24 by client.2174595.0:104214 0.000000 2014-08-13 18:06:06.794283 7f5c973a57a0 20 read_log 194215'15279 (193305'15071) clean d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by osd.4.0:2452 2014-08-12 15:44:03.586079 2014-08-13 18:06:06.794295 7f5c973a57a0 20 read_log 194215'15280 (194215'15279) delete d53e5df7/benchmark_data_storage0111-ib.pocstor_62621_object31823/head//24 by client.2174595.0:104216 0.000000 2014-08-13 18:06:06.794320 7f5c973a57a0 20 read_log 194215'15281 (192834'14556) clean 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by osd.4.0:2454 2014-08-12 15:44:03.595278 2014-08-13 18:06:06.794335 7f5c973a57a0 20 read_log 194215'15282 (194215'15281) delete 436df7/benchmark_data_storage0111-ib.pocstor_41526_object9385/head//24 by client.2174595.0:104218 0.000000 2014-08-13 18:06:06.794347 7f5c973a57a0 20 read_log 194215'15283 (192888'14771) clean e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by osd.4.0:2456 2014-08-12 15:44:03.603813 2014-08-13 18:06:06.794362 7f5c973a57a0 20 read_log 194215'15284 (194215'15283) delete e6e36df7/benchmark_data_storage0111-ib.pocstor_41097_object7234/head//24 by client.2174595.0:104220 0.000000 2014-08-13 18:06:06.794374 7f5c973a57a0 20 read_log 194215'15285 (192888'14977) clean 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by osd.4.0:2458 2014-08-12 15:44:03.611762 2014-08-13 18:06:06.794392 7f5c973a57a0 20 read_log 194215'15286 (194215'15285) delete 3f7d7df7/benchmark_data_storage0111-ib.pocstor_2228_object23455/head//24 by client.2174595.0:104222 0.000000 2014-08-13 18:06:06.794410 7f5c973a57a0 20 read_log 194215'15287 (193305'15061) clean f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by osd.4.0:2460 2014-08-12 15:44:03.620225 2014-08-13 18:06:06.794425 7f5c973a57a0 20 read_log 194215'15288 (194215'15287) delete f1778df7/benchmark_data_storage0111-ib.pocstor_62621_object19071/head//24 by client.2174595.0:104224 0.000000 2014-08-13 18:06:06.794444 7f5c973a57a0 20 read_log 194215'15289 (192834'14022) clean 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by osd.4.0:2462 2014-08-12 15:44:03.628456 2014-08-13 18:06:06.794460 7f5c973a57a0 20 read_log 194215'15290 (194215'15289) delete 2f5a8df7/benchmark_data_storage0111-ib.pocstor_26659_object7974/head//24 by client.2174595.0:104226 0.000000 2014-08-13 18:06:06.794471 7f5c973a57a0 20 read_log 194215'15291 (192888'14986) clean 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by osd.4.0:2464 2014-08-12 15:44:03.637126 2014-08-13 18:06:06.794490 7f5c973a57a0 20 read_log 194215'15292 (194215'15291) delete 9c809df7/benchmark_data_storage0111-ib.pocstor_2228_object29305/head//24 by client.2174595.0:104228 0.000000 2014-08-13 18:06:06.794503 7f5c973a57a0 20 read_log 194215'15293 (192888'14945) clean 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by osd.4.0:2466 2014-08-12 15:44:03.648734 2014-08-13 18:06:06.794516 7f5c973a57a0 20 read_log 194215'15294 (194215'15293) delete 74979df7/benchmark_data_storage0111-ib.pocstor_2228_object5554/head//24 by client.2174595.0:104230 0.000000 2014-08-13 18:06:06.794527 7f5c973a57a0 20 read_log 194215'15295 (192888'14762) clean ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by osd.4.0:2468 2014-08-12 15:44:03.657030 2014-08-13 18:06:06.794539 7f5c973a57a0 20 read_log 194215'15296 (194215'15295) delete ee3badf7/benchmark_data_storage0111-ib.pocstor_41097_object2207/head//24 by client.2174595.0:104232 0.000000 2014-08-13 18:06:06.794550 7f5c973a57a0 20 read_log 194215'15297 (192888'14990) clean c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by osd.4.0:2470 2014-08-12 15:44:03.665812 2014-08-13 18:06:06.794563 7f5c973a57a0 20 read_log 194215'15298 (194215'15297) delete c56cadf7/benchmark_data_storage0111-ib.pocstor_2228_object30661/head//24 by client.2174595.0:104234 0.000000 2014-08-13 18:06:06.794574 7f5c973a57a0 20 read_log 194215'15299 (192888'14808) clean 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by osd.4.0:2472 2014-08-12 15:44:03.674249 2014-08-13 18:06:06.794606 7f5c973a57a0 20 read_log 194215'15300 (194215'15299) delete 28afadf7/benchmark_data_storage0111-ib.pocstor_41097_object26442/head//24 by client.2174595.0:104236 0.000000 2014-08-13 18:06:06.794626 7f5c973a57a0 20 read_log 194215'15301 (193305'15058) clean 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by osd.4.0:2474 2014-08-12 15:44:03.682645 2014-08-13 18:06:06.794644 7f5c973a57a0 20 read_log 194215'15302 (194215'15301) delete 948ebdf7/benchmark_data_storage0111-ib.pocstor_62621_object11331/head//24 by client.2174595.0:104238 0.000000 2014-08-13 18:06:06.794661 7f5c973a57a0 20 read_log 194215'15303 (192834'14531) clean 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by osd.4.0:2476 2014-08-12 15:44:03.691028 2014-08-13 18:06:06.794676 7f5c973a57a0 20 read_log 194215'15304 (194215'15303) delete 8ba1cdf7/benchmark_data_storage0111-ib.pocstor_41526_object6873/head//24 by client.2174595.0:104240 0.000000 2014-08-13 18:06:06.794687 7f5c973a57a0 20 read_log 194215'15305 (193305'15069) clean e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by osd.4.0:2478 2014-08-12 15:44:03.699811 2014-08-13 18:06:06.794703 7f5c973a57a0 20 read_log 194215'15306 (194215'15305) delete e3e4ddf7/benchmark_data_storage0111-ib.pocstor_62621_object29030/head//24 by client.2174595.0:104242 0.000000 2014-08-13 18:06:06.794714 7f5c973a57a0 20 read_log 194215'15307 (192834'13990) clean 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by osd.4.0:2480 2014-08-12 15:44:03.708085 2014-08-13 18:06:06.794726 7f5c973a57a0 20 read_log 194215'15308 (194215'15307) delete 480cddf7/benchmark_data_storage0111-ib.pocstor_26659_object4305/head//24 by client.2174595.0:104244 0.000000 2014-08-13 18:06:06.794740 7f5c973a57a0 20 read_log 194215'15309 (193305'15064) clean ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by osd.4.0:2482 2014-08-12 15:44:03.716547 2014-08-13 18:06:06.794754 7f5c973a57a0 20 read_log 194215'15310 (194215'15309) delete ef0eddf7/benchmark_data_storage0111-ib.pocstor_62621_object21589/head//24 by client.2174595.0:104246 0.000000 2014-08-13 18:06:06.794765 7f5c973a57a0 20 read_log 194215'15311 (192834'13988) clean d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by osd.4.0:2484 2014-08-12 15:44:03.725946 2014-08-13 18:06:06.794778 7f5c973a57a0 20 read_log 194215'15312 (194215'15311) delete d701edf7/benchmark_data_storage0111-ib.pocstor_26659_object4090/head//24 by client.2174595.0:104248 0.000000 2014-08-13 18:06:06.794792 7f5c973a57a0 20 read_log 194215'15313 (192888'14983) clean a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by osd.4.0:2486 2014-08-12 15:44:03.734490 2014-08-13 18:06:06.794804 7f5c973a57a0 20 read_log 194215'15314 (194215'15313) delete a381edf7/benchmark_data_storage0111-ib.pocstor_2228_object24941/head//24 by client.2174595.0:104250 0.000000 2014-08-13 18:06:06.794822 7f5c973a57a0 20 read_log 194215'15315 (192834'14025) clean 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by osd.4.0:2488 2014-08-12 15:44:03.742220 2014-08-13 18:06:06.794839 7f5c973a57a0 20 read_log 194215'15316 (194215'15315) delete 557edf7/benchmark_data_storage0111-ib.pocstor_26659_object8254/head//24 by client.2174595.0:104252 0.000000 2014-08-13 18:06:06.794853 7f5c973a57a0 20 read_log 194215'15317 (193305'15056) clean 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by osd.4.0:2490 2014-08-12 15:44:03.749699 2014-08-13 18:06:06.794866 7f5c973a57a0 20 read_log 194215'15318 (194215'15317) delete 2a4dedf7/benchmark_data_storage0111-ib.pocstor_62621_object7739/head//24 by client.2174595.0:104254 0.000000 2014-08-13 18:06:06.794877 7f5c973a57a0 20 read_log 194215'15319 (192888'14807) clean 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by osd.4.0:2492 2014-08-12 15:44:03.758293 2014-08-13 18:06:06.794904 7f5c973a57a0 20 read_log 194215'15320 (194215'15319) delete 4100fdf7/benchmark_data_storage0111-ib.pocstor_41097_object26401/head//24 by client.2174595.0:104256 0.000000 2014-08-13 18:06:06.794923 7f5c973a57a0 20 read_log 194215'15321 (192888'14964) clean ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by osd.4.0:2494 2014-08-12 15:44:03.825292 2014-08-13 18:06:06.794949 7f5c973a57a0 20 read_log 194215'15322 (194215'15321) delete ff89fdf7/benchmark_data_storage0111-ib.pocstor_2228_object17497/head//24 by client.2174595.0:104258 0.000000 2014-08-13 18:06:06.794964 7f5c973a57a0 20 read_log 194215'15323 (192834'13965) clean 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by osd.4.0:2496 2014-08-12 15:44:03.833042 2014-08-13 18:06:06.794977 7f5c973a57a0 20 read_log 194215'15324 (194215'15323) delete 237dfdf7/benchmark_data_storage0111-ib.pocstor_26659_object365/head//24 by client.2174595.0:104260 0.000000 2014-08-13 18:06:06.794988 7f5c973a57a0 20 read_log 194215'15325 (194215'15325) modify 1f7/hit_set_24.1f7_archive_2014-08-12 15:44:02.507949_2014-08-12 17:24:41.967015/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.967257 2014-08-13 18:06:06.795001 7f5c973a57a0 20 read_log 194215'15326 (194215'15073) delete 1f7/hit_set_24.1f7_archive_2014-08-12 15:32:25.864872_2014-08-12 15:44:02.507691/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.967257 2014-08-13 18:06:06.795013 7f5c973a57a0 20 read_log 194215'15327 (0'0) promote b9f149f7/benchmark_data_storage0111-ib.pocstor_34628_object520/head//24 by osd.4.0:5235 2014-08-12 17:24:41.971524 2014-08-13 18:06:06.795025 7f5c973a57a0 20 read_log 194215'15328 (194215'15327) modify b9f149f7/benchmark_data_storage0111-ib.pocstor_34628_object520/head//24 by client.2193759.0:521 2014-08-12 17:24:41.962038 2014-08-13 18:06:06.795038 7f5c973a57a0 20 read_log 194370'15329 (0'0) promote 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by osd.55.0:4020 2014-08-12 20:30:16.135870 2014-08-13 18:06:06.795050 7f5c973a57a0 20 read_log 194370'15330 (194370'15329) modify 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by client.1508531.0:462 2014-08-12 20:30:16.127883 2014-08-13 18:06:06.795063 7f5c973a57a0 20 read_log 194370'15331 (0'0) promote 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by osd.55.0:4026 2014-08-12 20:30:17.500712 2014-08-13 18:06:06.795075 7f5c973a57a0 20 read_log 194370'15332 (194370'15331) modify 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by client.1508531.0:827 2014-08-12 20:30:17.494531 2014-08-13 18:06:06.795087 7f5c973a57a0 20 read_log 194370'15333 (0'0) promote de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by osd.55.0:4028 2014-08-12 20:30:17.555150 2014-08-13 18:06:06.795107 7f5c973a57a0 20 read_log 194370'15334 (194370'15333) modify de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by client.1508531.0:842 2014-08-12 20:30:17.548893 2014-08-13 18:06:06.795123 7f5c973a57a0 20 read_log 194370'15335 (0'0) promote 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by osd.55.0:4038 2014-08-12 20:30:21.338572 2014-08-13 18:06:06.795136 7f5c973a57a0 20 read_log 194370'15336 (194370'15335) modify 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by client.1508531.0:1864 2014-08-12 20:30:21.330546 2014-08-13 18:06:06.795153 7f5c973a57a0 20 read_log 194370'15337 (0'0) promote 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by osd.55.0:4040 2014-08-12 20:30:23.057662 2014-08-13 18:06:06.795168 7f5c973a57a0 20 read_log 194370'15338 (194370'15337) modify 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by client.1508531.0:2315 2014-08-12 20:30:23.049895 2014-08-13 18:06:06.795181 7f5c973a57a0 20 read_log 194370'15339 (0'0) promote 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by osd.55.0:4042 2014-08-12 20:30:23.073993 2014-08-13 18:06:06.795230 7f5c973a57a0 20 read_log 194370'15340 (194370'15339) modify 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by client.1508531.0:2319 2014-08-12 20:30:23.067777 2014-08-13 18:06:06.795247 7f5c973a57a0 20 read_log 194370'15341 (0'0) promote 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by osd.55.0:4056 2014-08-12 20:30:25.398683 2014-08-13 18:06:06.795264 7f5c973a57a0 20 read_log 194370'15342 (194370'15341) modify 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by client.1508531.0:2960 2014-08-12 20:30:25.392825 2014-08-13 18:06:06.795279 7f5c973a57a0 20 read_log 194370'15343 (0'0) promote fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by osd.55.0:4072 2014-08-12 20:30:30.136613 2014-08-13 18:06:06.795291 7f5c973a57a0 20 read_log 194370'15344 (194370'15343) modify fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by client.1508531.0:4210 2014-08-12 20:30:30.127847 2014-08-13 18:06:06.795304 7f5c973a57a0 20 read_log 194370'15345 (0'0) promote 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by osd.55.0:4076 2014-08-12 20:30:31.136903 2014-08-13 18:06:06.795324 7f5c973a57a0 20 read_log 194370'15346 (194370'15345) modify 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by client.1508531.0:4487 2014-08-12 20:30:31.129603 2014-08-13 18:06:06.795340 7f5c973a57a0 20 read_log 194370'15347 (0'0) promote 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by osd.55.0:4098 2014-08-12 20:30:35.447295 2014-08-13 18:06:06.795353 7f5c973a57a0 20 read_log 194370'15348 (194370'15347) modify 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by client.1508531.0:5513 2014-08-12 20:30:35.439859 2014-08-13 18:06:06.795365 7f5c973a57a0 20 read_log 194370'15349 (0'0) promote 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by osd.55.0:4116 2014-08-12 20:30:41.806880 2014-08-13 18:06:06.795388 7f5c973a57a0 20 read_log 194370'15350 (194370'15349) modify 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by client.1508531.0:6878 2014-08-12 20:30:41.798480 2014-08-13 18:06:06.795401 7f5c973a57a0 20 read_log 194370'15351 (0'0) promote af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by osd.55.0:4170 2014-08-12 20:31:00.759786 2014-08-13 18:06:06.795413 7f5c973a57a0 20 read_log 194370'15352 (194370'15351) modify af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by client.1508531.0:10256 2014-08-12 20:31:00.751788 2014-08-13 18:06:06.795426 7f5c973a57a0 20 read_log 194370'15353 (0'0) promote 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by osd.55.0:4180 2014-08-12 20:31:06.788082 2014-08-13 18:06:06.795457 7f5c973a57a0 20 read_log 194370'15354 (194370'15353) modify 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by client.1508531.0:11249 2014-08-12 20:31:06.779426 2014-08-13 18:06:06.795473 7f5c973a57a0 20 read_log 194370'15355 (0'0) promote e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by osd.55.0:4188 2014-08-12 20:31:10.650071 2014-08-13 18:06:06.795485 7f5c973a57a0 20 read_log 194370'15356 (194370'15355) modify e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by client.1508531.0:11899 2014-08-12 20:31:10.642433 2014-08-13 18:06:06.795498 7f5c973a57a0 20 read_log 194370'15357 (0'0) promote fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by osd.55.0:4206 2014-08-12 20:31:23.668578 2014-08-13 18:06:06.795510 7f5c973a57a0 20 read_log 194370'15358 (194370'15357) modify fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by client.1508531.0:13451 2014-08-12 20:31:23.660793 2014-08-13 18:06:06.795522 7f5c973a57a0 20 read_log 194370'15359 (0'0) promote 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by osd.55.0:4208 2014-08-12 20:31:23.792998 2014-08-13 18:06:06.795543 7f5c973a57a0 20 read_log 194370'15360 (194370'15359) modify 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by client.1508531.0:13464 2014-08-12 20:31:23.785576 2014-08-13 18:06:06.795556 7f5c973a57a0 20 read_log 194377'15361 (0'0) promote de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by osd.55.0:4226 2014-08-12 20:33:01.996518 2014-08-13 18:06:06.795568 7f5c973a57a0 20 read_log 194377'15362 (194377'15361) modify de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by client.2231190.0:1185 2014-08-12 20:33:01.990904 2014-08-13 18:06:06.795581 7f5c973a57a0 20 read_log 194377'15363 (0'0) promote 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by osd.55.0:4238 2014-08-12 20:33:04.895801 2014-08-13 18:06:06.795593 7f5c973a57a0 20 read_log 194377'15364 (194377'15363) modify 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by client.2231190.0:1925 2014-08-12 20:33:04.888178 2014-08-13 18:06:06.795606 7f5c973a57a0 20 read_log 194377'15365 (0'0) promote 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by osd.55.0:4240 2014-08-12 20:33:06.121891 2014-08-13 18:06:06.795618 7f5c973a57a0 20 read_log 194377'15366 (194377'15365) modify 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by client.2231190.0:2216 2014-08-12 20:33:06.114672 2014-08-13 18:06:06.795630 7f5c973a57a0 20 read_log 194377'15367 (0'0) promote a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by osd.55.0:4272 2014-08-12 20:33:17.458328 2014-08-13 18:06:06.795642 7f5c973a57a0 20 read_log 194377'15368 (194377'15367) modify a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by client.2231190.0:4540 2014-08-12 20:33:17.449352 2014-08-13 18:06:06.795655 7f5c973a57a0 20 read_log 194377'15369 (0'0) promote d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by osd.55.0:4274 2014-08-12 20:33:17.788024 2014-08-13 18:06:06.795667 7f5c973a57a0 20 read_log 194377'15370 (194377'15369) modify d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by client.2231190.0:4583 2014-08-12 20:33:17.781221 2014-08-13 18:06:06.795679 7f5c973a57a0 20 read_log 194378'15371 (0'0) promote 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by osd.55.0:4278 2014-08-12 20:33:19.648517 2014-08-13 18:06:06.795691 7f5c973a57a0 20 read_log 194378'15372 (194378'15371) modify 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by client.2231190.0:4814 2014-08-12 20:33:19.607282 2014-08-13 18:06:06.795703 7f5c973a57a0 20 read_log 194383'15373 (0'0) promote 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by osd.55.0:4288 2014-08-12 20:34:02.044306 2014-08-13 18:06:06.795715 7f5c973a57a0 20 read_log 194383'15374 (194383'15373) modify 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by client.2231190.0:6122 2014-08-12 20:34:02.036743 2014-08-13 18:06:06.795727 7f5c973a57a0 20 read_log 194393'15375 (194393'15375) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:37:08.012293_2014-08-12 20:47:26.399042/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.399479 2014-08-13 18:06:06.795740 7f5c973a57a0 20 read_log 194393'15376 (194215'15325) delete 1f7/hit_set_24.1f7_archive_2014-08-12 15:44:02.507949_2014-08-12 17:24:41.967015/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.399479 2014-08-13 18:06:06.795752 7f5c973a57a0 20 read_log 194393'15377 (0'0) promote 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by osd.55.0:4301 2014-08-12 20:47:26.404510 2014-08-13 18:06:06.795765 7f5c973a57a0 20 read_log 194393'15378 (194393'15377) modify 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by client.1064095.0:105 2014-08-12 20:47:26.390525 2014-08-13 18:06:06.795785 7f5c973a57a0 20 read_log 194393'15379 (0'0) promote 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by osd.55.0:4309 2014-08-12 20:47:27.511552 2014-08-13 18:06:06.795800 7f5c973a57a0 20 read_log 194393'15380 (194393'15379) modify 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by client.1064095.0:364 2014-08-12 20:47:27.505831 2014-08-13 18:06:06.795813 7f5c973a57a0 20 read_log 194393'15381 (0'0) promote e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by osd.55.0:4318 2014-08-12 20:47:28.299641 2014-08-13 18:06:06.795827 7f5c973a57a0 20 read_log 194393'15382 (194393'15381) modify e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by client.1064095.0:544 2014-08-12 20:47:28.293346 2014-08-13 18:06:06.795841 7f5c973a57a0 20 read_log 194393'15383 (0'0) promote 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by osd.55.0:4327 2014-08-12 20:49:26.154719 2014-08-13 18:06:06.795855 7f5c973a57a0 20 read_log 194393'15384 (194393'15383) modify 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by client.1064182.0:146 2014-08-12 20:49:26.147080 2014-08-13 18:06:06.795870 7f5c973a57a0 20 read_log 194393'15385 (0'0) promote 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by osd.55.0:4329 2014-08-12 20:49:26.212573 2014-08-13 18:06:06.795882 7f5c973a57a0 20 read_log 194393'15386 (194393'15385) modify 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by client.1064182.0:159 2014-08-12 20:49:26.205261 2014-08-13 18:06:06.795896 7f5c973a57a0 20 read_log 194396'15387 (194396'15387) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:47:26.399484_2014-08-12 20:53:00.360133/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.360468 2014-08-13 18:06:06.795909 7f5c973a57a0 20 read_log 194396'15388 (194393'15375) delete 1f7/hit_set_24.1f7_archive_2014-08-12 20:37:08.012293_2014-08-12 20:47:26.399042/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.360468 2014-08-13 18:06:06.795923 7f5c973a57a0 20 read_log 194396'15389 (0'0) promote a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by osd.55.0:4353 2014-08-12 20:53:00.365713 2014-08-13 18:06:06.795936 7f5c973a57a0 20 read_log 194396'15390 (194396'15389) modify a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by client.2235069.0:309 2014-08-12 20:53:00.353284 2014-08-13 18:06:06.795948 7f5c973a57a0 20 read_log 194396'15391 (0'0) promote 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by osd.55.0:4355 2014-08-12 20:53:00.537643 2014-08-13 18:06:06.795960 7f5c973a57a0 20 read_log 194396'15392 (194396'15391) modify 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by client.2235069.0:341 2014-08-12 20:53:00.529259 2014-08-13 18:06:06.795973 7f5c973a57a0 20 read_log 194396'15393 (0'0) promote da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by osd.55.0:4359 2014-08-12 20:53:00.807310 2014-08-13 18:06:06.795987 7f5c973a57a0 20 read_log 194396'15394 (194396'15393) modify da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by client.2235069.0:406 2014-08-12 20:53:00.798343 2014-08-13 18:06:06.796000 7f5c973a57a0 20 read_log 194416'15395 (194416'15395) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 18:06:06.796014 7f5c973a57a0 20 read_log 194416'15396 (194396'15387) delete 1f7/hit_set_24.1f7_archive_2014-08-12 20:47:26.399484_2014-08-12 20:53:00.360133/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 18:06:06.796027 7f5c973a57a0 20 read_log 194416'15397 (0'0) promote 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by osd.55.0:4497 2014-08-13 15:28:31.203329 2014-08-13 18:06:06.796048 7f5c973a57a0 20 read_log 194416'15398 (194416'15397) modify 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by client.2441958.0:246 2014-08-13 15:28:31.194786 2014-08-13 18:06:06.796061 7f5c973a57a0 20 read_log 194416'15399 (0'0) promote 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by osd.55.0:4509 2014-08-13 15:28:34.820408 2014-08-13 18:06:06.796073 7f5c973a57a0 20 read_log 194416'15400 (194416'15399) modify 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by client.2441958.0:1080 2014-08-13 15:28:34.814563 2014-08-13 18:06:06.796085 7f5c973a57a0 20 read_log 194416'15401 (0'0) promote 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by osd.55.0:4521 2014-08-13 15:28:41.689171 2014-08-13 18:06:06.796100 7f5c973a57a0 20 read_log 194416'15402 (194416'15401) modify 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by client.2441958.0:2896 2014-08-13 15:28:41.679680 2014-08-13 18:06:06.796112 7f5c973a57a0 20 read_log 194416'15403 (0'0) promote 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by osd.55.0:4535 2014-08-13 15:28:45.234493 2014-08-13 18:06:06.796124 7f5c973a57a0 20 read_log 194416'15404 (194416'15403) modify 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by client.2441958.0:3815 2014-08-13 15:28:45.226875 2014-08-13 18:06:06.796137 7f5c973a57a0 20 read_log 194416'15405 (0'0) promote f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by osd.55.0:4539 2014-08-13 15:28:46.778918 2014-08-13 18:06:06.796154 7f5c973a57a0 20 read_log 194416'15406 (194416'15405) modify f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by client.2441958.0:4208 2014-08-13 15:28:46.771328 2014-08-13 18:06:06.796167 7f5c973a57a0 20 read_log 194416'15407 (0'0) promote 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by osd.55.0:4577 2014-08-13 15:28:56.287463 2014-08-13 18:06:06.796181 7f5c973a57a0 20 read_log 194416'15408 (194416'15407) modify 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by client.2441958.0:6599 2014-08-13 15:28:56.276634 2014-08-13 18:06:06.796197 7f5c973a57a0 20 read_log 194416'15409 (0'0) promote ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by osd.55.0:4585 2014-08-13 15:28:59.769628 2014-08-13 18:06:06.796210 7f5c973a57a0 20 read_log 194416'15410 (194416'15409) modify ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by client.2441958.0:7532 2014-08-13 15:28:59.762703 2014-08-13 18:06:06.796244 7f5c973a57a0 20 read_log 194416'15411 (0'0) promote 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by osd.55.0:4589 2014-08-13 15:29:00.834441 2014-08-13 18:06:06.796256 7f5c973a57a0 20 read_log 194416'15412 (194416'15411) modify 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by client.2441958.0:7815 2014-08-13 15:29:00.827799 2014-08-13 18:06:06.796269 7f5c973a57a0 20 read_log 194416'15413 (0'0) promote 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by osd.55.0:4595 2014-08-13 15:29:02.236837 2014-08-13 18:06:06.796287 7f5c973a57a0 20 read_log 194416'15414 (194416'15413) modify 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by client.2441958.0:8187 2014-08-13 15:29:02.229787 2014-08-13 18:06:06.796300 7f5c973a57a0 20 read_log 194416'15415 (0'0) promote 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by osd.55.0:4599 2014-08-13 15:29:03.181613 2014-08-13 18:06:06.796313 7f5c973a57a0 20 read_log 194416'15416 (194416'15415) modify 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by client.2441958.0:8436 2014-08-13 15:29:03.172525 2014-08-13 18:06:06.796328 7f5c973a57a0 20 read_log 194416'15417 (0'0) promote f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by osd.55.0:4601 2014-08-13 15:29:03.395038 2014-08-13 18:06:06.796348 7f5c973a57a0 20 read_log 194416'15418 (194416'15417) modify f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by client.2441958.0:8493 2014-08-13 15:29:03.387213 2014-08-13 18:06:06.796361 7f5c973a57a0 20 read_log 194416'15419 (0'0) promote 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by osd.55.0:4605 2014-08-13 15:29:05.181385 2014-08-13 18:06:06.796373 7f5c973a57a0 20 read_log 194416'15420 (194416'15419) modify 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by client.2441958.0:8935 2014-08-13 15:29:05.172817 2014-08-13 18:06:06.796385 7f5c973a57a0 20 read_log 194416'15421 (0'0) promote 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by osd.55.0:4607 2014-08-13 15:29:06.092684 2014-08-13 18:06:06.796397 7f5c973a57a0 20 read_log 194416'15422 (194416'15421) modify 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by client.2441958.0:9161 2014-08-13 15:29:06.085354 2014-08-13 18:06:06.796414 7f5c973a57a0 20 read_log 194416'15423 (0'0) promote 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by osd.55.0:4645 2014-08-13 15:29:15.558144 2014-08-13 18:06:06.796436 7f5c973a57a0 20 read_log 194416'15424 (194416'15423) modify 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by client.2441958.0:11513 2014-08-13 15:29:15.547610 2014-08-13 18:06:06.796449 7f5c973a57a0 20 read_log 194416'15425 (0'0) promote 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by osd.55.0:4671 2014-08-13 15:29:29.495715 2014-08-13 18:06:06.796461 7f5c973a57a0 20 read_log 194416'15426 (194416'15425) modify 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by client.2441958.0:14822 2014-08-13 15:29:29.486067 2014-08-13 18:06:06.796489 7f5c973a57a0 20 read_log 194416'15427 (0'0) promote d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by osd.55.0:4699 2014-08-13 15:29:48.541983 2014-08-13 18:06:06.796504 7f5c973a57a0 20 read_log 194416'15428 (194416'15427) modify d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by client.2441958.0:17084 2014-08-13 15:29:48.533177 2014-08-13 18:06:06.796523 7f5c973a57a0 20 read_log 194416'15429 (0'0) promote b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by osd.55.0:4701 2014-08-13 15:29:49.591754 2014-08-13 18:06:06.796539 7f5c973a57a0 20 read_log 194416'15430 (194416'15429) modify b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by client.2441958.0:17158 2014-08-13 15:29:49.582821 2014-08-13 18:06:06.796552 7f5c973a57a0 20 read_log 194416'15431 (0'0) promote 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by osd.55.0:4711 2014-08-13 15:29:58.502359 2014-08-13 18:06:06.796565 7f5c973a57a0 20 read_log 194416'15432 (194416'15431) modify 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by client.2441958.0:17792 2014-08-13 15:29:58.493248 2014-08-13 18:06:06.796577 7f5c973a57a0 20 read_log 194422'15433 (0'0) promote 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by osd.55.0:4719 2014-08-13 15:32:15.783067 2014-08-13 18:06:06.796589 7f5c973a57a0 20 read_log 194422'15434 (194422'15433) modify 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by client.1148203.0:175 2014-08-13 15:32:15.775000 2014-08-13 18:06:06.796602 7f5c973a57a0 20 read_log 194422'15435 (0'0) promote e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by osd.55.0:4721 2014-08-13 15:32:15.853831 2014-08-13 18:06:06.796614 7f5c973a57a0 20 read_log 194422'15436 (194422'15435) modify e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by client.1148203.0:188 2014-08-13 15:32:15.845579 2014-08-13 18:06:06.796634 7f5c973a57a0 20 read_log 194422'15437 (0'0) promote 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by osd.55.0:4757 2014-08-13 15:32:27.339857 2014-08-13 18:06:06.796656 7f5c973a57a0 20 read_log 194422'15438 (194422'15437) modify 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by client.1148203.0:3063 2014-08-13 15:32:27.331306 2014-08-13 18:06:06.796685 7f5c973a57a0 20 read_log 194422'15439 (0'0) promote b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by osd.55.0:4767 2014-08-13 15:32:29.204202 2014-08-13 18:06:06.796700 7f5c973a57a0 20 read_log 194422'15440 (194422'15439) modify b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by client.1148203.0:3528 2014-08-13 15:32:29.195794 2014-08-13 18:06:06.796713 7f5c973a57a0 20 read_log 194422'15441 (0'0) promote aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by osd.55.0:4773 2014-08-13 15:32:32.003065 2014-08-13 18:06:06.796734 7f5c973a57a0 20 read_log 194422'15442 (194422'15441) modify aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by client.1148203.0:4266 2014-08-13 15:32:31.993510 2014-08-13 18:06:06.796750 7f5c973a57a0 20 read_log 194422'15443 (0'0) promote 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by osd.55.0:4781 2014-08-13 15:32:39.266538 2014-08-13 18:06:06.796762 7f5c973a57a0 20 read_log 194422'15444 (194422'15443) modify 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by client.1148203.0:5841 2014-08-13 15:32:39.257570 2014-08-13 18:06:06.796782 7f5c973a57a0 20 read_log 194422'15445 (0'0) promote 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by osd.55.0:4783 2014-08-13 15:32:39.679792 2014-08-13 18:06:06.796800 7f5c973a57a0 20 read_log 194422'15446 (194422'15445) modify 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by client.1148203.0:5918 2014-08-13 15:32:39.669700 2014-08-13 18:06:06.796830 7f5c973a57a0 20 read_log 194422'15447 (0'0) promote be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by osd.55.0:4787 2014-08-13 15:32:41.686432 2014-08-13 18:06:06.796845 7f5c973a57a0 20 read_log 194422'15448 (194422'15447) modify be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by client.1148203.0:6280 2014-08-13 15:32:41.678072 2014-08-13 18:06:06.796864 7f5c973a57a0 20 read_log 194422'15449 (0'0) promote 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by osd.55.0:4789 2014-08-13 15:32:42.406474 2014-08-13 18:06:06.796877 7f5c973a57a0 20 read_log 194422'15450 (194422'15449) modify 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by client.1148203.0:6417 2014-08-13 15:32:42.397066 2014-08-13 18:06:06.796889 7f5c973a57a0 20 read_log 194422'15451 (0'0) promote ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by osd.55.0:4793 2014-08-13 15:32:44.530865 2014-08-13 18:06:06.796901 7f5c973a57a0 20 read_log 194422'15452 (194422'15451) modify ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by client.1148203.0:6823 2014-08-13 15:32:44.523271 2014-08-13 18:06:06.796914 7f5c973a57a0 20 read_log 194422'15453 (0'0) promote 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by osd.55.0:4816 2014-08-13 15:32:58.048819 2014-08-13 18:06:06.796926 7f5c973a57a0 20 read_log 194422'15454 (194422'15453) modify 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by client.1148203.0:8845 2014-08-13 15:32:58.038562 2014-08-13 18:06:06.796939 7f5c973a57a0 20 read_log 194422'15455 (0'0) promote 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by osd.55.0:4840 2014-08-13 15:33:18.693626 2014-08-13 18:06:06.796951 7f5c973a57a0 20 read_log 194422'15456 (194422'15455) modify 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by client.1148203.0:10309 2014-08-13 15:33:18.684448 2014-08-13 18:06:06.796969 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.797417 7f5c973a57a0 10 read_log checking for missing items over interval (194370'15329,194422'15456] 2014-08-13 18:06:06.797465 7f5c973a57a0 15 read_log missing 194422'15456 (194422'15455) modify 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 by client.1148203.0:10309 2014-08-13 15:33:18.684448 2014-08-13 18:06:06.797506 7f5c973a57a0 15 read_log missing 194422'15454 (194422'15453) modify 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 by client.1148203.0:8845 2014-08-13 15:32:58.038562 2014-08-13 18:06:06.797543 7f5c973a57a0 15 read_log missing 194422'15452 (194422'15451) modify ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 by client.1148203.0:6823 2014-08-13 15:32:44.523271 2014-08-13 18:06:06.797579 7f5c973a57a0 15 read_log missing 194422'15450 (194422'15449) modify 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 by client.1148203.0:6417 2014-08-13 15:32:42.397066 2014-08-13 18:06:06.797615 7f5c973a57a0 15 read_log missing 194422'15448 (194422'15447) modify be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 by client.1148203.0:6280 2014-08-13 15:32:41.678072 2014-08-13 18:06:06.797651 7f5c973a57a0 15 read_log missing 194422'15446 (194422'15445) modify 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 by client.1148203.0:5918 2014-08-13 15:32:39.669700 2014-08-13 18:06:06.797687 7f5c973a57a0 15 read_log missing 194422'15444 (194422'15443) modify 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 by client.1148203.0:5841 2014-08-13 15:32:39.257570 2014-08-13 18:06:06.797723 7f5c973a57a0 15 read_log missing 194422'15442 (194422'15441) modify aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 by client.1148203.0:4266 2014-08-13 15:32:31.993510 2014-08-13 18:06:06.797759 7f5c973a57a0 15 read_log missing 194422'15440 (194422'15439) modify b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 by client.1148203.0:3528 2014-08-13 15:32:29.195794 2014-08-13 18:06:06.797794 7f5c973a57a0 15 read_log missing 194422'15438 (194422'15437) modify 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 by client.1148203.0:3063 2014-08-13 15:32:27.331306 2014-08-13 18:06:06.797830 7f5c973a57a0 15 read_log missing 194422'15436 (194422'15435) modify e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 by client.1148203.0:188 2014-08-13 15:32:15.845579 2014-08-13 18:06:06.797866 7f5c973a57a0 15 read_log missing 194422'15434 (194422'15433) modify 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 by client.1148203.0:175 2014-08-13 15:32:15.775000 2014-08-13 18:06:06.797902 7f5c973a57a0 15 read_log missing 194416'15432 (194416'15431) modify 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 by client.2441958.0:17792 2014-08-13 15:29:58.493248 2014-08-13 18:06:06.797937 7f5c973a57a0 15 read_log missing 194416'15430 (194416'15429) modify b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 by client.2441958.0:17158 2014-08-13 15:29:49.582821 2014-08-13 18:06:06.797973 7f5c973a57a0 15 read_log missing 194416'15428 (194416'15427) modify d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 by client.2441958.0:17084 2014-08-13 15:29:48.533177 2014-08-13 18:06:06.798009 7f5c973a57a0 15 read_log missing 194416'15426 (194416'15425) modify 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 by client.2441958.0:14822 2014-08-13 15:29:29.486067 2014-08-13 18:06:06.798045 7f5c973a57a0 15 read_log missing 194416'15424 (194416'15423) modify 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 by client.2441958.0:11513 2014-08-13 15:29:15.547610 2014-08-13 18:06:06.798080 7f5c973a57a0 15 read_log missing 194416'15422 (194416'15421) modify 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 by client.2441958.0:9161 2014-08-13 15:29:06.085354 2014-08-13 18:06:06.798116 7f5c973a57a0 15 read_log missing 194416'15420 (194416'15419) modify 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 by client.2441958.0:8935 2014-08-13 15:29:05.172817 2014-08-13 18:06:06.798158 7f5c973a57a0 15 read_log missing 194416'15418 (194416'15417) modify f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 by client.2441958.0:8493 2014-08-13 15:29:03.387213 2014-08-13 18:06:06.798195 7f5c973a57a0 15 read_log missing 194416'15416 (194416'15415) modify 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 by client.2441958.0:8436 2014-08-13 15:29:03.172525 2014-08-13 18:06:06.798231 7f5c973a57a0 15 read_log missing 194416'15414 (194416'15413) modify 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 by client.2441958.0:8187 2014-08-13 15:29:02.229787 2014-08-13 18:06:06.798271 7f5c973a57a0 15 read_log missing 194416'15412 (194416'15411) modify 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 by client.2441958.0:7815 2014-08-13 15:29:00.827799 2014-08-13 18:06:06.798312 7f5c973a57a0 15 read_log missing 194416'15410 (194416'15409) modify ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 by client.2441958.0:7532 2014-08-13 15:28:59.762703 2014-08-13 18:06:06.798349 7f5c973a57a0 15 read_log missing 194416'15408 (194416'15407) modify 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 by client.2441958.0:6599 2014-08-13 15:28:56.276634 2014-08-13 18:06:06.798397 7f5c973a57a0 15 read_log missing 194416'15406 (194416'15405) modify f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 by client.2441958.0:4208 2014-08-13 15:28:46.771328 2014-08-13 18:06:06.798438 7f5c973a57a0 15 read_log missing 194416'15404 (194416'15403) modify 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 by client.2441958.0:3815 2014-08-13 15:28:45.226875 2014-08-13 18:06:06.798483 7f5c973a57a0 15 read_log missing 194416'15402 (194416'15401) modify 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 by client.2441958.0:2896 2014-08-13 15:28:41.679680 2014-08-13 18:06:06.798521 7f5c973a57a0 15 read_log missing 194416'15400 (194416'15399) modify 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 by client.2441958.0:1080 2014-08-13 15:28:34.814563 2014-08-13 18:06:06.798599 7f5c973a57a0 15 read_log missing 194416'15398 (194416'15397) modify 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 by client.2441958.0:246 2014-08-13 15:28:31.194786 2014-08-13 18:06:06.798685 7f5c973a57a0 15 read_log missing 194416'15395 (194416'15395) modify 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.199898 2014-08-13 18:06:06.798742 7f5c973a57a0 15 read_log missing 194396'15394 (194396'15393) modify da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 by client.2235069.0:406 2014-08-12 20:53:00.798343 2014-08-13 18:06:06.798800 7f5c973a57a0 15 read_log missing 194396'15392 (194396'15391) modify 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 by client.2235069.0:341 2014-08-12 20:53:00.529259 2014-08-13 18:06:06.798843 7f5c973a57a0 15 read_log missing 194396'15390 (194396'15389) modify a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 by client.2235069.0:309 2014-08-12 20:53:00.353284 2014-08-13 18:06:06.798883 7f5c973a57a0 15 read_log missing 194393'15386 (194393'15385) modify 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 by client.1064182.0:159 2014-08-12 20:49:26.205261 2014-08-13 18:06:06.798953 7f5c973a57a0 15 read_log missing 194393'15384 (194393'15383) modify 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 by client.1064182.0:146 2014-08-12 20:49:26.147080 2014-08-13 18:06:06.799018 7f5c973a57a0 15 read_log missing 194393'15382 (194393'15381) modify e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 by client.1064095.0:544 2014-08-12 20:47:28.293346 2014-08-13 18:06:06.799070 7f5c973a57a0 15 read_log missing 194393'15380 (194393'15379) modify 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 by client.1064095.0:364 2014-08-12 20:47:27.505831 2014-08-13 18:06:06.799126 7f5c973a57a0 15 read_log missing 194393'15378 (194393'15377) modify 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 by client.1064095.0:105 2014-08-12 20:47:26.390525 2014-08-13 18:06:06.799182 7f5c973a57a0 15 read_log missing 194383'15374 (194383'15373) modify 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 by client.2231190.0:6122 2014-08-12 20:34:02.036743 2014-08-13 18:06:06.799220 7f5c973a57a0 15 read_log missing 194378'15372 (194378'15371) modify 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 by client.2231190.0:4814 2014-08-12 20:33:19.607282 2014-08-13 18:06:06.799254 7f5c973a57a0 15 read_log missing 194377'15370 (194377'15369) modify d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 by client.2231190.0:4583 2014-08-12 20:33:17.781221 2014-08-13 18:06:06.799292 7f5c973a57a0 15 read_log missing 194377'15368 (194377'15367) modify a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 by client.2231190.0:4540 2014-08-12 20:33:17.449352 2014-08-13 18:06:06.799355 7f5c973a57a0 15 read_log missing 194377'15366 (194377'15365) modify 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 by client.2231190.0:2216 2014-08-12 20:33:06.114672 2014-08-13 18:06:06.799404 7f5c973a57a0 15 read_log missing 194377'15364 (194377'15363) modify 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 by client.2231190.0:1925 2014-08-12 20:33:04.888178 2014-08-13 18:06:06.799478 7f5c973a57a0 15 read_log missing 194377'15362 (194377'15361) modify de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 by client.2231190.0:1185 2014-08-12 20:33:01.990904 2014-08-13 18:06:06.799531 7f5c973a57a0 15 read_log missing 194370'15360 (194370'15359) modify 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 by client.1508531.0:13464 2014-08-12 20:31:23.785576 2014-08-13 18:06:06.799591 7f5c973a57a0 15 read_log missing 194370'15358 (194370'15357) modify fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 by client.1508531.0:13451 2014-08-12 20:31:23.660793 2014-08-13 18:06:06.799634 7f5c973a57a0 15 read_log missing 194370'15356 (194370'15355) modify e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 by client.1508531.0:11899 2014-08-12 20:31:10.642433 2014-08-13 18:06:06.799672 7f5c973a57a0 15 read_log missing 194370'15354 (194370'15353) modify 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 by client.1508531.0:11249 2014-08-12 20:31:06.779426 2014-08-13 18:06:06.799708 7f5c973a57a0 15 read_log missing 194370'15352 (194370'15351) modify af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 by client.1508531.0:10256 2014-08-12 20:31:00.751788 2014-08-13 18:06:06.799745 7f5c973a57a0 15 read_log missing 194370'15350 (194370'15349) modify 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 by client.1508531.0:6878 2014-08-12 20:30:41.798480 2014-08-13 18:06:06.799781 7f5c973a57a0 15 read_log missing 194370'15348 (194370'15347) modify 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 by client.1508531.0:5513 2014-08-12 20:30:35.439859 2014-08-13 18:06:06.799817 7f5c973a57a0 15 read_log missing 194370'15346 (194370'15345) modify 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 by client.1508531.0:4487 2014-08-12 20:30:31.129603 2014-08-13 18:06:06.799854 7f5c973a57a0 15 read_log missing 194370'15344 (194370'15343) modify fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 by client.1508531.0:4210 2014-08-12 20:30:30.127847 2014-08-13 18:06:06.799890 7f5c973a57a0 15 read_log missing 194370'15342 (194370'15341) modify 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 by client.1508531.0:2960 2014-08-12 20:30:25.392825 2014-08-13 18:06:06.799926 7f5c973a57a0 15 read_log missing 194370'15340 (194370'15339) modify 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 by client.1508531.0:2319 2014-08-12 20:30:23.067777 2014-08-13 18:06:06.799962 7f5c973a57a0 15 read_log missing 194370'15338 (194370'15337) modify 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 by client.1508531.0:2315 2014-08-12 20:30:23.049895 2014-08-13 18:06:06.799999 7f5c973a57a0 15 read_log missing 194370'15336 (194370'15335) modify 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 by client.1508531.0:1864 2014-08-12 20:30:21.330546 2014-08-13 18:06:06.800037 7f5c973a57a0 15 read_log missing 194370'15334 (194370'15333) modify de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 by client.1508531.0:842 2014-08-12 20:30:17.548893 2014-08-13 18:06:06.800074 7f5c973a57a0 15 read_log missing 194370'15332 (194370'15331) modify 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 by client.1508531.0:827 2014-08-12 20:30:17.494531 2014-08-13 18:06:06.800110 7f5c973a57a0 15 read_log missing 194370'15330 (194370'15329) modify 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 by client.1508531.0:462 2014-08-12 20:30:16.127883 2014-08-13 18:06:06.800122 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.800158 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_loaded 2014-08-13 18:06:06.800177 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Initial 0.017963 0 0.000000 2014-08-13 18:06:06.800190 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset 2014-08-13 18:06:06.800200 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=0 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194487 2014-08-13 18:06:06.800210 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] log((192718'12872,194422'15456], crt=194422'15456) 2014-08-13 18:06:06.800221 7f5c973a57a0 10 osd.4 194487 pgid 24.206 coll 24.206_head 2014-08-13 18:06:06.800414 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.206 2014-08-13 18:06:06.800427 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.800460 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.206(unlocked)] enter Initial 2014-08-13 18:06:06.800472 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.206(unlocked)] enter NotTrimming 2014-08-13 18:06:06.800615 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.800746 7f5c973a57a0 20 read_log 192834'12572 (0'0) promote 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.397.0:13176 2014-08-11 15:23:36.292094 2014-08-13 18:06:06.800767 7f5c973a57a0 20 read_log 192834'12573 (192834'12572) modify 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by client.933448.0:157 2014-08-11 15:23:36.283359 2014-08-13 18:06:06.800781 7f5c973a57a0 20 read_log 192834'12580 (0'0) promote 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by osd.397.0:13184 2014-08-11 15:23:38.977732 2014-08-13 18:06:06.800795 7f5c973a57a0 20 read_log 192834'12581 (192834'12580) modify 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.933448.0:867 2014-08-11 15:23:38.970238 2014-08-13 18:06:06.800808 7f5c973a57a0 20 read_log 192834'12582 (0'0) promote fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by osd.397.0:13186 2014-08-11 15:23:39.034300 2014-08-13 18:06:06.800821 7f5c973a57a0 20 read_log 192834'12583 (192834'12582) modify fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.933448.0:881 2014-08-11 15:23:39.025075 2014-08-13 18:06:06.800833 7f5c973a57a0 20 read_log 192834'12620 (0'0) promote 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by osd.397.0:13224 2014-08-11 15:23:56.441661 2014-08-13 18:06:06.800846 7f5c973a57a0 20 read_log 192834'12621 (192834'12620) modify 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.933448.0:4345 2014-08-11 15:23:56.433697 2014-08-13 18:06:06.800858 7f5c973a57a0 20 read_log 192834'12646 (0'0) promote f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.397.0:13250 2014-08-11 15:24:02.510034 2014-08-13 18:06:06.800870 7f5c973a57a0 20 read_log 192834'12647 (192834'12646) modify f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by client.933448.0:5688 2014-08-11 15:24:02.501955 2014-08-13 18:06:06.800883 7f5c973a57a0 20 read_log 192834'12654 (0'0) promote 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.397.0:13258 2014-08-11 15:24:04.016107 2014-08-13 18:06:06.800895 7f5c973a57a0 20 read_log 192834'12655 (192834'12654) modify 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by client.933448.0:6074 2014-08-11 15:24:04.008004 2014-08-13 18:06:06.800907 7f5c973a57a0 20 read_log 192834'12658 (0'0) promote ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by osd.397.0:13262 2014-08-11 15:24:05.085447 2014-08-13 18:06:06.800920 7f5c973a57a0 20 read_log 192834'12659 (192834'12658) modify ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.933448.0:6350 2014-08-11 15:24:05.076859 2014-08-13 18:06:06.800932 7f5c973a57a0 20 read_log 192834'12666 (0'0) promote ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by osd.397.0:13270 2014-08-11 15:24:06.726396 2014-08-13 18:06:06.800945 7f5c973a57a0 20 read_log 192834'12667 (192834'12666) modify ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.933448.0:6779 2014-08-11 15:24:06.718687 2014-08-13 18:06:06.800957 7f5c973a57a0 20 read_log 192834'12678 (0'0) promote ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by osd.397.0:13282 2014-08-11 15:24:09.396510 2014-08-13 18:06:06.800969 7f5c973a57a0 20 read_log 192834'12679 (192834'12678) modify ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.933448.0:7266 2014-08-11 15:24:09.388147 2014-08-13 18:06:06.800981 7f5c973a57a0 20 read_log 192834'12706 (0'0) promote b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by osd.397.0:13310 2014-08-11 15:24:17.829208 2014-08-13 18:06:06.801002 7f5c973a57a0 20 read_log 192834'12707 (192834'12706) modify b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.933448.0:9174 2014-08-11 15:24:17.821001 2014-08-13 18:06:06.801015 7f5c973a57a0 20 read_log 192834'12708 (0'0) promote f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by osd.397.0:13312 2014-08-11 15:24:18.263837 2014-08-13 18:06:06.801027 7f5c973a57a0 20 read_log 192834'12709 (192834'12708) modify f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.933448.0:9282 2014-08-11 15:24:18.255965 2014-08-13 18:06:06.801039 7f5c973a57a0 20 read_log 192834'12712 (0'0) promote c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.397.0:13316 2014-08-11 15:24:19.706646 2014-08-13 18:06:06.801052 7f5c973a57a0 20 read_log 192834'12713 (192834'12712) modify c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by client.933448.0:9658 2014-08-11 15:24:19.699716 2014-08-13 18:06:06.801064 7f5c973a57a0 20 read_log 192834'12740 (0'0) promote 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by osd.397.0:13344 2014-08-11 15:24:26.176671 2014-08-13 18:06:06.801077 7f5c973a57a0 20 read_log 192834'12741 (192834'12740) modify 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.933448.0:11027 2014-08-11 15:24:26.169559 2014-08-13 18:06:06.801090 7f5c973a57a0 20 read_log 192834'12744 (0'0) promote 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.397.0:13348 2014-08-11 15:24:27.355703 2014-08-13 18:06:06.801102 7f5c973a57a0 20 read_log 192834'12745 (192834'12744) modify 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by client.933448.0:11341 2014-08-11 15:24:27.347861 2014-08-13 18:06:06.801114 7f5c973a57a0 20 read_log 192834'12750 (0'0) promote 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by osd.397.0:13354 2014-08-11 15:24:29.279796 2014-08-13 18:06:06.801126 7f5c973a57a0 20 read_log 192834'12751 (192834'12750) modify 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.933448.0:11764 2014-08-11 15:24:29.270029 2014-08-13 18:06:06.801139 7f5c973a57a0 20 read_log 192834'12784 (0'0) promote 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.397.0:13388 2014-08-11 15:24:40.086023 2014-08-13 18:06:06.801161 7f5c973a57a0 20 read_log 192834'12785 (192834'12784) modify 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by client.933448.0:13822 2014-08-11 15:24:40.077718 2014-08-13 18:06:06.801174 7f5c973a57a0 20 read_log 192834'12820 (0'0) promote 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by osd.397.0:13424 2014-08-11 15:24:52.453670 2014-08-13 18:06:06.801189 7f5c973a57a0 20 read_log 192834'12821 (192834'12820) modify 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.933448.0:16041 2014-08-11 15:24:52.444616 2014-08-13 18:06:06.801201 7f5c973a57a0 20 read_log 192834'12824 (0'0) promote abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.397.0:13428 2014-08-11 15:24:54.442317 2014-08-13 18:06:06.801238 7f5c973a57a0 20 read_log 192834'12825 (192834'12824) modify abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by client.933448.0:16381 2014-08-11 15:24:54.431989 2014-08-13 18:06:06.801251 7f5c973a57a0 20 read_log 192834'12836 (0'0) promote 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by osd.397.0:13440 2014-08-11 15:25:00.926777 2014-08-13 18:06:06.801268 7f5c973a57a0 20 read_log 192834'12837 (192834'12836) modify 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.933448.0:17657 2014-08-11 15:25:00.919059 2014-08-13 18:06:06.801280 7f5c973a57a0 20 read_log 192834'12846 (0'0) promote 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by osd.397.0:13450 2014-08-11 15:25:03.279511 2014-08-13 18:06:06.801301 7f5c973a57a0 20 read_log 192834'12847 (192834'12846) modify 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.933448.0:18052 2014-08-11 15:25:03.270428 2014-08-13 18:06:06.801316 7f5c973a57a0 20 read_log 192834'12856 (0'0) promote b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by osd.397.0:13460 2014-08-11 15:25:12.768675 2014-08-13 18:06:06.801328 7f5c973a57a0 20 read_log 192834'12857 (192834'12856) modify b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.933448.0:19616 2014-08-11 15:25:12.758610 2014-08-13 18:06:06.801341 7f5c973a57a0 20 read_log 192834'12872 (0'0) promote f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by osd.397.0:13476 2014-08-11 15:25:16.449680 2014-08-13 18:06:06.801353 7f5c973a57a0 20 read_log 192834'12873 (192834'12872) modify f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.933448.0:20090 2014-08-11 15:25:16.442653 2014-08-13 18:06:06.801365 7f5c973a57a0 20 read_log 192834'12874 (0'0) promote 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by osd.397.0:13478 2014-08-11 15:25:20.427158 2014-08-13 18:06:06.801377 7f5c973a57a0 20 read_log 192834'12875 (192834'12874) modify 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.933448.0:20338 2014-08-11 15:25:20.416986 2014-08-13 18:06:06.801401 7f5c973a57a0 20 read_log 192834'12886 (0'0) promote f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.397.0:13490 2014-08-11 15:25:28.610796 2014-08-13 18:06:06.801417 7f5c973a57a0 20 read_log 192834'12887 (192834'12886) modify f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by client.933448.0:21341 2014-08-11 15:25:28.601922 2014-08-13 18:06:06.801443 7f5c973a57a0 20 read_log 192834'12898 (0'0) promote 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by osd.397.0:13502 2014-08-11 15:25:32.390453 2014-08-13 18:06:06.801459 7f5c973a57a0 20 read_log 192834'12899 (192834'12898) modify 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.933448.0:21859 2014-08-11 15:25:32.380445 2014-08-13 18:06:06.801471 7f5c973a57a0 20 read_log 192834'12905 (192834'12573) modify 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by client.933448.0:22448 2014-08-11 15:25:37.885867 2014-08-13 18:06:06.801492 7f5c973a57a0 20 read_log 192834'12909 (192834'12581) modify 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.933448.0:23158 2014-08-11 15:25:39.067040 2014-08-13 18:06:06.801512 7f5c973a57a0 20 read_log 192834'12910 (192834'12583) modify fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.933448.0:23172 2014-08-11 15:25:39.075022 2014-08-13 18:06:06.801527 7f5c973a57a0 20 read_log 192834'12929 (192834'12621) modify 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.933448.0:26636 2014-08-11 15:25:43.609441 2014-08-13 18:06:06.801539 7f5c973a57a0 20 read_log 192834'12942 (192834'12647) modify f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by client.933448.0:27979 2014-08-11 15:25:45.334539 2014-08-13 18:06:06.801560 7f5c973a57a0 20 read_log 192834'12946 (192834'12655) modify 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by client.933448.0:28365 2014-08-11 15:25:46.026239 2014-08-13 18:06:06.801575 7f5c973a57a0 20 read_log 192834'12948 (192834'12659) modify ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.933448.0:28641 2014-08-11 15:25:46.550910 2014-08-13 18:06:06.801592 7f5c973a57a0 20 read_log 192834'12952 (192834'12667) modify ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.933448.0:29070 2014-08-11 15:25:46.975386 2014-08-13 18:06:06.801607 7f5c973a57a0 20 read_log 192834'12958 (192834'12679) modify ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.933448.0:29557 2014-08-11 15:25:47.475069 2014-08-13 18:06:06.801630 7f5c973a57a0 20 read_log 192834'12972 (192834'12707) modify b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.933448.0:31465 2014-08-11 15:25:50.093790 2014-08-13 18:06:06.801645 7f5c973a57a0 20 read_log 192834'12973 (192834'12709) modify f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.933448.0:31573 2014-08-11 15:25:50.203460 2014-08-13 18:06:06.801662 7f5c973a57a0 20 read_log 192834'12975 (192834'12713) modify c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by client.933448.0:31949 2014-08-11 15:25:50.868576 2014-08-13 18:06:06.801687 7f5c973a57a0 20 read_log 192834'12989 (192834'12741) modify 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.933448.0:33318 2014-08-11 15:25:53.268093 2014-08-13 18:06:06.801700 7f5c973a57a0 20 read_log 192834'12991 (192834'12745) modify 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by client.933448.0:33632 2014-08-11 15:25:53.858207 2014-08-13 18:06:06.801716 7f5c973a57a0 20 read_log 192834'12994 (192834'12751) modify 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.933448.0:34055 2014-08-11 15:25:54.531476 2014-08-13 18:06:06.801731 7f5c973a57a0 20 read_log 192834'13011 (192834'12785) modify 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by client.933448.0:36113 2014-08-11 15:25:57.253491 2014-08-13 18:06:06.801743 7f5c973a57a0 20 read_log 192834'13029 (192834'12821) modify 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.933448.0:38332 2014-08-11 15:26:00.768615 2014-08-13 18:06:06.801758 7f5c973a57a0 20 read_log 192834'13031 (192834'12825) modify abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by client.933448.0:38672 2014-08-11 15:26:01.136462 2014-08-13 18:06:06.801781 7f5c973a57a0 20 read_log 192834'13037 (192834'12837) modify 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.933448.0:39948 2014-08-11 15:26:02.179821 2014-08-13 18:06:06.801797 7f5c973a57a0 20 read_log 192834'13042 (192834'12847) modify 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.933448.0:40343 2014-08-11 15:26:02.577424 2014-08-13 18:06:06.801812 7f5c973a57a0 20 read_log 192834'13047 (192834'12857) modify b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.933448.0:41907 2014-08-11 15:26:03.089624 2014-08-13 18:06:06.801825 7f5c973a57a0 20 read_log 192834'13055 (192834'12873) modify f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.933448.0:42381 2014-08-11 15:26:03.259615 2014-08-13 18:06:06.801839 7f5c973a57a0 20 read_log 192834'13056 (192834'12875) modify 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.933448.0:42629 2014-08-11 15:26:03.337575 2014-08-13 18:06:06.801852 7f5c973a57a0 20 read_log 192834'13062 (192834'12887) modify f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by client.933448.0:43632 2014-08-11 15:26:03.679997 2014-08-13 18:06:06.801868 7f5c973a57a0 20 read_log 192834'13068 (192834'12899) modify 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.933448.0:44150 2014-08-11 15:26:03.831226 2014-08-13 18:06:06.801885 7f5c973a57a0 20 read_log 192834'13087 (0'0) promote 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by osd.397.0:13691 2014-08-11 15:26:17.351914 2014-08-13 18:06:06.801902 7f5c973a57a0 20 read_log 192834'13088 (192834'13087) modify 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.1883634.0:786 2014-08-11 15:26:17.345072 2014-08-13 18:06:06.801920 7f5c973a57a0 20 read_log 192834'13107 (0'0) promote 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.397.0:13711 2014-08-11 15:26:24.429495 2014-08-13 18:06:06.801950 7f5c973a57a0 20 read_log 192834'13108 (192834'13107) modify 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by client.1883634.0:2708 2014-08-11 15:26:24.419719 2014-08-13 18:06:06.801977 7f5c973a57a0 20 read_log 192834'13113 (0'0) promote fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by osd.397.0:13717 2014-08-11 15:26:26.679019 2014-08-13 18:06:06.801992 7f5c973a57a0 20 read_log 192834'13114 (192834'13113) modify fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.1883634.0:3343 2014-08-11 15:26:26.668944 2014-08-13 18:06:06.802008 7f5c973a57a0 20 read_log 192834'13123 (0'0) promote 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.397.0:13727 2014-08-11 15:26:28.581488 2014-08-13 18:06:06.802021 7f5c973a57a0 20 read_log 192834'13124 (192834'13123) modify 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by client.1883634.0:3883 2014-08-11 15:26:28.573150 2014-08-13 18:06:06.802043 7f5c973a57a0 20 read_log 192834'13145 (0'0) promote deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by osd.397.0:13749 2014-08-11 15:26:35.340848 2014-08-13 18:06:06.802058 7f5c973a57a0 20 read_log 192834'13146 (192834'13145) modify deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.1883634.0:5701 2014-08-11 15:26:35.332635 2014-08-13 18:06:06.802071 7f5c973a57a0 20 read_log 192834'13161 (0'0) promote a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by osd.397.0:13765 2014-08-11 15:26:38.666266 2014-08-13 18:06:06.802083 7f5c973a57a0 20 read_log 192834'13162 (192834'13161) modify a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.1883634.0:6545 2014-08-11 15:26:38.658175 2014-08-13 18:06:06.802096 7f5c973a57a0 20 read_log 192834'13219 (0'0) promote ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by osd.397.0:13823 2014-08-11 15:26:50.117795 2014-08-13 18:06:06.802108 7f5c973a57a0 20 read_log 192834'13220 (192834'13219) modify ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.1883634.0:9261 2014-08-11 15:26:50.110444 2014-08-13 18:06:06.802121 7f5c973a57a0 20 read_log 192834'13225 (0'0) promote 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by osd.397.0:13829 2014-08-11 15:26:52.324894 2014-08-13 18:06:06.802133 7f5c973a57a0 20 read_log 192834'13226 (192834'13225) modify 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.1883634.0:9772 2014-08-11 15:26:52.314487 2014-08-13 18:06:06.802150 7f5c973a57a0 20 read_log 192834'13233 (0'0) promote a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.397.0:13837 2014-08-11 15:26:52.850147 2014-08-13 18:06:06.802164 7f5c973a57a0 20 read_log 192834'13234 (192834'13233) modify a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by client.1883634.0:9890 2014-08-11 15:26:52.840690 2014-08-13 18:06:06.802176 7f5c973a57a0 20 read_log 192834'13265 (0'0) promote f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by osd.397.0:13869 2014-08-11 15:27:03.128920 2014-08-13 18:06:06.802189 7f5c973a57a0 20 read_log 192834'13266 (192834'13265) modify f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.1883634.0:12233 2014-08-11 15:27:03.120081 2014-08-13 18:06:06.802201 7f5c973a57a0 20 read_log 192834'13301 (0'0) promote 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by osd.397.0:13905 2014-08-11 15:27:14.376058 2014-08-13 18:06:06.802213 7f5c973a57a0 20 read_log 192834'13302 (192834'13301) modify 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.1883634.0:14665 2014-08-11 15:27:14.366524 2014-08-13 18:06:06.802226 7f5c973a57a0 20 read_log 192834'13319 (0'0) promote bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.397.0:13923 2014-08-11 15:27:21.305538 2014-08-13 18:06:06.802247 7f5c973a57a0 20 read_log 192834'13320 (192834'13319) modify bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by client.1883634.0:15709 2014-08-11 15:27:21.296983 2014-08-13 18:06:06.802299 7f5c973a57a0 20 read_log 192834'13325 (0'0) promote f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by osd.397.0:13929 2014-08-11 15:27:22.771728 2014-08-13 18:06:06.802316 7f5c973a57a0 20 read_log 192834'13326 (192834'13325) modify f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.1883634.0:15912 2014-08-11 15:27:22.760464 2014-08-13 18:06:06.802332 7f5c973a57a0 20 read_log 192834'13335 (0'0) promote dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.397.0:13939 2014-08-11 15:27:27.720177 2014-08-13 18:06:06.802347 7f5c973a57a0 20 read_log 192834'13336 (192834'13335) modify dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by client.1883634.0:16473 2014-08-11 15:27:27.709759 2014-08-13 18:06:06.802360 7f5c973a57a0 20 read_log 192834'13337 (0'0) promote 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.397.0:13941 2014-08-11 15:27:27.957168 2014-08-13 18:06:06.802372 7f5c973a57a0 20 read_log 192834'13338 (192834'13337) modify 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by client.1883634.0:16516 2014-08-11 15:27:27.947366 2014-08-13 18:06:06.802384 7f5c973a57a0 20 read_log 192834'13353 (0'0) promote 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by osd.397.0:13957 2014-08-11 15:27:34.077274 2014-08-13 18:06:06.802407 7f5c973a57a0 20 read_log 192834'13354 (192834'13353) modify 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.1883634.0:17283 2014-08-11 15:27:34.064084 2014-08-13 18:06:06.802421 7f5c973a57a0 20 read_log 192834'13383 (0'0) promote cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.397.0:13987 2014-08-11 15:27:48.527285 2014-08-13 18:06:06.802433 7f5c973a57a0 20 read_log 192834'13384 (192834'13383) modify cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by client.1883634.0:18969 2014-08-11 15:27:48.519399 2014-08-13 18:06:06.802446 7f5c973a57a0 20 read_log 192834'13397 (0'0) promote 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.397.0:14001 2014-08-11 15:27:55.410558 2014-08-13 18:06:06.802464 7f5c973a57a0 20 read_log 192834'13398 (192834'13397) modify 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by client.1883634.0:20138 2014-08-11 15:27:55.397145 2014-08-13 18:06:06.802480 7f5c973a57a0 20 read_log 192834'13401 (0'0) promote ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.397.0:14005 2014-08-11 15:27:55.798071 2014-08-13 18:06:06.802493 7f5c973a57a0 20 read_log 192834'13402 (192834'13401) modify ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by client.1883634.0:20232 2014-08-11 15:27:55.785793 2014-08-13 18:06:06.802505 7f5c973a57a0 20 read_log 192834'13405 (0'0) promote ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.397.0:14009 2014-08-11 15:28:01.367752 2014-08-13 18:06:06.802517 7f5c973a57a0 20 read_log 192834'13406 (192834'13405) modify ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by client.1883634.0:20749 2014-08-11 15:28:01.359528 2014-08-13 18:06:06.802530 7f5c973a57a0 20 read_log 192834'13409 (0'0) promote c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by osd.397.0:14013 2014-08-11 15:28:02.729265 2014-08-13 18:06:06.802542 7f5c973a57a0 20 read_log 192834'13410 (192834'13409) modify c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.1883634.0:21087 2014-08-11 15:28:02.716417 2014-08-13 18:06:06.802554 7f5c973a57a0 20 read_log 192834'13415 (0'0) promote 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by osd.397.0:14019 2014-08-11 15:28:04.230688 2014-08-13 18:06:06.802567 7f5c973a57a0 20 read_log 192834'13416 (192834'13415) modify 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.1883634.0:21454 2014-08-11 15:28:04.219970 2014-08-13 18:06:06.802590 7f5c973a57a0 20 read_log 192834'13421 (0'0) promote d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by osd.397.0:14025 2014-08-11 15:28:04.968324 2014-08-13 18:06:06.802605 7f5c973a57a0 20 read_log 192834'13422 (192834'13421) modify d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.1883634.0:21688 2014-08-11 15:28:04.959134 2014-08-13 18:06:06.802618 7f5c973a57a0 20 read_log 192834'13433 (0'0) promote eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by osd.397.0:14037 2014-08-11 15:28:11.421057 2014-08-13 18:06:06.802630 7f5c973a57a0 20 read_log 192834'13434 (192834'13433) modify eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.1883634.0:22855 2014-08-11 15:28:11.406764 2014-08-13 18:06:06.802643 7f5c973a57a0 20 read_log 192834'13445 (192834'13088) modify 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.1883634.0:24110 2014-08-11 15:28:18.366418 2014-08-13 18:06:06.802655 7f5c973a57a0 20 read_log 192834'13455 (192834'13108) modify 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by client.1883634.0:26032 2014-08-11 15:28:22.526748 2014-08-13 18:06:06.802667 7f5c973a57a0 20 read_log 192834'13458 (192834'13114) modify fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.1883634.0:26667 2014-08-11 15:28:23.391700 2014-08-13 18:06:06.802680 7f5c973a57a0 20 read_log 192834'13463 (192834'13124) modify 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by client.1883634.0:27207 2014-08-11 15:28:24.193205 2014-08-13 18:06:06.802692 7f5c973a57a0 20 read_log 192834'13474 (192834'13146) modify deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.1883634.0:29025 2014-08-11 15:28:26.639314 2014-08-13 18:06:06.802711 7f5c973a57a0 20 read_log 192834'13482 (192834'13162) modify a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.1883634.0:29869 2014-08-11 15:28:27.773696 2014-08-13 18:06:06.802724 7f5c973a57a0 20 read_log 192834'13511 (192834'13220) modify ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.1883634.0:32585 2014-08-11 15:28:32.049640 2014-08-13 18:06:06.802736 7f5c973a57a0 20 read_log 192834'13514 (192834'13226) modify 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.1883634.0:33096 2014-08-11 15:28:32.772315 2014-08-13 18:06:06.802748 7f5c973a57a0 20 read_log 192834'13518 (192834'13234) modify a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by client.1883634.0:33214 2014-08-11 15:28:32.906792 2014-08-13 18:06:06.802761 7f5c973a57a0 20 read_log 192834'13536 (192834'13266) modify f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.1883634.0:35557 2014-08-11 15:28:36.826573 2014-08-13 18:06:06.802773 7f5c973a57a0 20 read_log 192834'13554 (192834'13302) modify 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.1883634.0:37989 2014-08-11 15:28:40.065714 2014-08-13 18:06:06.802785 7f5c973a57a0 20 read_log 192834'13563 (192834'13320) modify bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by client.1883634.0:39033 2014-08-11 15:28:41.502004 2014-08-13 18:06:06.802801 7f5c973a57a0 20 read_log 192834'13566 (192834'13326) modify f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.1883634.0:39236 2014-08-11 15:28:41.898560 2014-08-13 18:06:06.802815 7f5c973a57a0 20 read_log 192834'13571 (192834'13336) modify dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by client.1883634.0:39797 2014-08-11 15:28:42.585177 2014-08-13 18:06:06.802828 7f5c973a57a0 20 read_log 192834'13572 (192834'13338) modify 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by client.1883634.0:39840 2014-08-11 15:28:42.645135 2014-08-13 18:06:06.802842 7f5c973a57a0 20 read_log 192834'13580 (192834'13354) modify 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.1883634.0:40607 2014-08-11 15:28:44.047491 2014-08-13 18:06:06.802862 7f5c973a57a0 20 read_log 192834'13595 (192834'13384) modify cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by client.1883634.0:42293 2014-08-11 15:28:46.199551 2014-08-13 18:06:06.802877 7f5c973a57a0 20 read_log 192834'13602 (192834'13398) modify 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by client.1883634.0:43462 2014-08-11 15:28:46.562106 2014-08-13 18:06:06.802890 7f5c973a57a0 20 read_log 192834'13604 (192834'13402) modify ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by client.1883634.0:43556 2014-08-11 15:28:46.590186 2014-08-13 18:06:06.802904 7f5c973a57a0 20 read_log 192834'13606 (192834'13406) modify ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by client.1883634.0:44073 2014-08-11 15:28:46.752107 2014-08-13 18:06:06.802917 7f5c973a57a0 20 read_log 192834'13608 (192834'13410) modify c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.1883634.0:44411 2014-08-11 15:28:46.851919 2014-08-13 18:06:06.802931 7f5c973a57a0 20 read_log 192834'13611 (192834'13416) modify 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.1883634.0:44778 2014-08-11 15:28:46.954960 2014-08-13 18:06:06.802943 7f5c973a57a0 20 read_log 192834'13614 (192834'13422) modify d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.1883634.0:45012 2014-08-11 15:28:47.121415 2014-08-13 18:06:06.802956 7f5c973a57a0 20 read_log 192834'13620 (192834'13434) modify eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.1883634.0:46179 2014-08-11 15:28:47.513603 2014-08-13 18:06:06.802971 7f5c973a57a0 20 read_log 192888'13624 (0'0) promote d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.397.0:14464 2014-08-11 16:44:54.982436 2014-08-13 18:06:06.802984 7f5c973a57a0 20 read_log 192888'13625 (192888'13624) modify d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by client.1315259.0:29 2014-08-11 16:44:54.969680 2014-08-13 18:06:06.802998 7f5c973a57a0 20 read_log 192888'13626 (0'0) promote 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by osd.397.0:14466 2014-08-11 16:44:57.675315 2014-08-13 18:06:06.803011 7f5c973a57a0 20 read_log 192888'13627 (192888'13626) modify 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.1315259.0:630 2014-08-11 16:44:57.666114 2014-08-13 18:06:06.803023 7f5c973a57a0 20 read_log 192888'13630 (0'0) promote 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by osd.397.0:14470 2014-08-11 16:44:57.814678 2014-08-13 18:06:06.803035 7f5c973a57a0 20 read_log 192888'13631 (192888'13630) modify 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.1315259.0:666 2014-08-11 16:44:57.805689 2014-08-13 18:06:06.803070 7f5c973a57a0 20 read_log 192888'13636 (0'0) promote a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by osd.397.0:14476 2014-08-11 16:45:06.280489 2014-08-13 18:06:06.803086 7f5c973a57a0 20 read_log 192888'13637 (192888'13636) modify a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.1315259.0:2918 2014-08-11 16:45:06.271296 2014-08-13 18:06:06.803098 7f5c973a57a0 20 read_log 192888'13638 (0'0) promote 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by osd.397.0:14478 2014-08-11 16:45:11.281264 2014-08-13 18:06:06.803113 7f5c973a57a0 20 read_log 192888'13639 (192888'13638) modify 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.1315259.0:4225 2014-08-11 16:45:11.271335 2014-08-13 18:06:06.803125 7f5c973a57a0 20 read_log 192888'13640 (0'0) promote c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by osd.397.0:14480 2014-08-11 16:45:14.341911 2014-08-13 18:06:06.803140 7f5c973a57a0 20 read_log 192888'13641 (192888'13640) modify c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.1315259.0:5055 2014-08-11 16:45:14.332248 2014-08-13 18:06:06.803168 7f5c973a57a0 20 read_log 192888'13658 (0'0) promote 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by osd.397.0:14498 2014-08-11 16:45:33.512633 2014-08-13 18:06:06.803183 7f5c973a57a0 20 read_log 192888'13659 (192888'13658) modify 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.1315259.0:10230 2014-08-11 16:45:33.500817 2014-08-13 18:06:06.803195 7f5c973a57a0 20 read_log 192888'13662 (0'0) promote d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.397.0:14502 2014-08-11 16:45:35.311398 2014-08-13 18:06:06.803210 7f5c973a57a0 20 read_log 192888'13663 (192888'13662) modify d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by client.1315259.0:10673 2014-08-11 16:45:35.298624 2014-08-13 18:06:06.803238 7f5c973a57a0 20 read_log 192888'13664 (0'0) promote 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by osd.397.0:14504 2014-08-11 16:45:37.571848 2014-08-13 18:06:06.803265 7f5c973a57a0 20 read_log 192888'13665 (192888'13664) modify 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.1315259.0:11237 2014-08-11 16:45:37.562112 2014-08-13 18:06:06.803279 7f5c973a57a0 20 read_log 192888'13672 (0'0) promote ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by osd.397.0:14512 2014-08-11 16:45:45.355405 2014-08-13 18:06:06.803291 7f5c973a57a0 20 read_log 192888'13673 (192888'13672) modify ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.1315259.0:13125 2014-08-11 16:45:45.347675 2014-08-13 18:06:06.803311 7f5c973a57a0 20 read_log 192888'13676 (0'0) promote 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by osd.397.0:14516 2014-08-11 16:45:48.044757 2014-08-13 18:06:06.803327 7f5c973a57a0 20 read_log 192888'13677 (192888'13676) modify 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.1315259.0:13803 2014-08-11 16:45:48.033140 2014-08-13 18:06:06.803339 7f5c973a57a0 20 read_log 192888'13680 (0'0) promote 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by osd.397.0:14520 2014-08-11 16:45:50.763016 2014-08-13 18:06:06.803355 7f5c973a57a0 20 read_log 192888'13681 (192888'13680) modify 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.1315259.0:14518 2014-08-11 16:45:50.753878 2014-08-13 18:06:06.803400 7f5c973a57a0 20 read_log 192888'13682 (0'0) promote dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by osd.397.0:14522 2014-08-11 16:45:50.951650 2014-08-13 18:06:06.803418 7f5c973a57a0 20 read_log 192888'13683 (192888'13682) modify dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.1315259.0:14569 2014-08-11 16:45:50.940855 2014-08-13 18:06:06.803445 7f5c973a57a0 20 read_log 192888'13684 (0'0) promote 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by osd.397.0:14524 2014-08-11 16:45:56.834501 2014-08-13 18:06:06.803458 7f5c973a57a0 20 read_log 192888'13685 (192888'13684) modify 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.1315259.0:16106 2014-08-11 16:45:56.820910 2014-08-13 18:06:06.803471 7f5c973a57a0 20 read_log 192888'13686 (0'0) promote 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.397.0:14526 2014-08-11 16:45:58.442984 2014-08-13 18:06:06.803490 7f5c973a57a0 20 read_log 192888'13687 (192888'13686) modify 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by client.1315259.0:16516 2014-08-11 16:45:58.433557 2014-08-13 18:06:06.803511 7f5c973a57a0 20 read_log 192888'13688 (0'0) promote 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.397.0:14528 2014-08-11 16:45:59.989603 2014-08-13 18:06:06.803524 7f5c973a57a0 20 read_log 192888'13689 (192888'13688) modify 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by client.1315259.0:16918 2014-08-11 16:45:59.981347 2014-08-13 18:06:06.803557 7f5c973a57a0 20 read_log 192888'13692 (0'0) promote 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.397.0:14532 2014-08-11 16:46:01.196953 2014-08-13 18:06:06.803573 7f5c973a57a0 20 read_log 192888'13693 (192888'13692) modify 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by client.1315259.0:17220 2014-08-11 16:46:01.187898 2014-08-13 18:06:06.803585 7f5c973a57a0 20 read_log 192888'13698 (0'0) promote 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by osd.397.0:14538 2014-08-11 16:46:05.866780 2014-08-13 18:06:06.803607 7f5c973a57a0 20 read_log 192888'13699 (192888'13698) modify 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.1315259.0:18387 2014-08-11 16:46:05.857153 2014-08-13 18:06:06.803620 7f5c973a57a0 20 read_log 192888'13702 (0'0) promote e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.397.0:14542 2014-08-11 16:46:09.726901 2014-08-13 18:06:06.803632 7f5c973a57a0 20 read_log 192888'13703 (192888'13702) modify e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by client.1315259.0:19351 2014-08-11 16:46:09.717207 2014-08-13 18:06:06.803645 7f5c973a57a0 20 read_log 192888'13704 (0'0) promote 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by osd.397.0:14544 2014-08-11 16:46:10.423331 2014-08-13 18:06:06.803657 7f5c973a57a0 20 read_log 192888'13705 (192888'13704) modify 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.1315259.0:19537 2014-08-11 16:46:10.411872 2014-08-13 18:06:06.803669 7f5c973a57a0 20 read_log 192888'13706 (0'0) promote dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by osd.397.0:14546 2014-08-11 16:46:12.479490 2014-08-13 18:06:06.803681 7f5c973a57a0 20 read_log 192888'13707 (192888'13706) modify dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.1315259.0:20098 2014-08-11 16:46:12.471540 2014-08-13 18:06:06.803697 7f5c973a57a0 20 read_log 192888'13708 (0'0) promote e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.397.0:14548 2014-08-11 16:46:13.653648 2014-08-13 18:06:06.803710 7f5c973a57a0 20 read_log 192888'13709 (192888'13708) modify e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by client.1315259.0:20417 2014-08-11 16:46:13.645500 2014-08-13 18:06:06.803722 7f5c973a57a0 20 read_log 192888'13712 (0'0) promote 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by osd.397.0:14552 2014-08-11 16:46:18.608838 2014-08-13 18:06:06.803757 7f5c973a57a0 20 read_log 192888'13713 (192888'13712) modify 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.1315259.0:21718 2014-08-11 16:46:18.599272 2014-08-13 18:06:06.803770 7f5c973a57a0 20 read_log 192888'13714 (0'0) promote c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by osd.397.0:14554 2014-08-11 16:46:20.227503 2014-08-13 18:06:06.803782 7f5c973a57a0 20 read_log 192888'13715 (192888'13714) modify c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.1315259.0:22134 2014-08-11 16:46:20.218526 2014-08-13 18:06:06.803794 7f5c973a57a0 20 read_log 192888'13718 (0'0) promote da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.397.0:14558 2014-08-11 16:46:22.825005 2014-08-13 18:06:06.803806 7f5c973a57a0 20 read_log 192888'13719 (192888'13718) modify da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by client.1315259.0:22807 2014-08-11 16:46:22.814296 2014-08-13 18:06:06.803819 7f5c973a57a0 20 read_log 192888'13724 (0'0) promote a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.397.0:14564 2014-08-11 16:46:34.518209 2014-08-13 18:06:06.803831 7f5c973a57a0 20 read_log 192888'13725 (192888'13724) modify a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by client.1315259.0:25731 2014-08-11 16:46:34.507619 2014-08-13 18:06:06.803851 7f5c973a57a0 20 read_log 192888'13728 (0'0) promote 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by osd.397.0:14568 2014-08-11 16:46:43.153881 2014-08-13 18:06:06.803863 7f5c973a57a0 20 read_log 192888'13729 (192888'13728) modify 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.1315259.0:27792 2014-08-11 16:46:43.143223 2014-08-13 18:06:06.803876 7f5c973a57a0 20 read_log 192888'13730 (0'0) promote b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by osd.397.0:14570 2014-08-11 16:46:43.163899 2014-08-13 18:06:06.803888 7f5c973a57a0 20 read_log 192888'13731 (192888'13730) modify b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.1315259.0:27794 2014-08-11 16:46:43.151411 2014-08-13 18:06:06.803901 7f5c973a57a0 20 read_log 192888'13732 (0'0) promote f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by osd.397.0:14572 2014-08-11 16:46:43.270125 2014-08-13 18:06:06.803912 7f5c973a57a0 20 read_log 192888'13733 (192888'13732) modify f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.1315259.0:27822 2014-08-11 16:46:43.259760 2014-08-13 18:06:06.803925 7f5c973a57a0 20 read_log 192888'13734 (0'0) promote d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by osd.397.0:14574 2014-08-11 16:46:43.492662 2014-08-13 18:06:06.803937 7f5c973a57a0 20 read_log 192888'13735 (192888'13734) modify d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.1315259.0:27879 2014-08-11 16:46:43.481843 2014-08-13 18:06:06.803949 7f5c973a57a0 20 read_log 192888'13736 (0'0) promote bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by osd.397.0:14576 2014-08-11 16:46:44.566136 2014-08-13 18:06:06.803961 7f5c973a57a0 20 read_log 192888'13737 (192888'13736) modify bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.1315259.0:28156 2014-08-11 16:46:44.559270 2014-08-13 18:06:06.803973 7f5c973a57a0 20 read_log 192888'13740 (0'0) promote df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by osd.397.0:14580 2014-08-11 16:46:50.336045 2014-08-13 18:06:06.803985 7f5c973a57a0 20 read_log 192888'13741 (192888'13740) modify df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.1315259.0:29550 2014-08-11 16:46:50.327553 2014-08-13 18:06:06.803997 7f5c973a57a0 20 read_log 192888'13742 (0'0) promote 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by osd.397.0:14582 2014-08-11 16:46:51.076923 2014-08-13 18:06:06.804010 7f5c973a57a0 20 read_log 192888'13743 (192888'13742) modify 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.1315259.0:29725 2014-08-11 16:46:51.067525 2014-08-13 18:06:06.804022 7f5c973a57a0 20 read_log 192888'13752 (192888'13625) modify d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by client.1315259.0:30756 2014-08-11 16:46:55.877658 2014-08-13 18:06:06.804034 7f5c973a57a0 20 read_log 192888'13753 (192888'13627) modify 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.1315259.0:31357 2014-08-11 16:46:56.070590 2014-08-13 18:06:06.804046 7f5c973a57a0 20 read_log 192888'13755 (192888'13631) modify 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.1315259.0:31393 2014-08-11 16:46:56.080061 2014-08-13 18:06:06.804059 7f5c973a57a0 20 read_log 192888'13758 (192888'13637) modify a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.1315259.0:33645 2014-08-11 16:46:56.688863 2014-08-13 18:06:06.804071 7f5c973a57a0 20 read_log 192888'13759 (192888'13639) modify 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.1315259.0:34952 2014-08-11 16:46:57.028595 2014-08-13 18:06:06.804083 7f5c973a57a0 20 read_log 192888'13760 (192888'13641) modify c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.1315259.0:35782 2014-08-11 16:46:57.244721 2014-08-13 18:06:06.804101 7f5c973a57a0 20 read_log 192888'13769 (192888'13659) modify 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.1315259.0:40957 2014-08-11 16:46:58.624706 2014-08-13 18:06:06.804114 7f5c973a57a0 20 read_log 192888'13771 (192888'13663) modify d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by client.1315259.0:41400 2014-08-11 16:46:58.736545 2014-08-13 18:06:06.804127 7f5c973a57a0 20 read_log 192888'13772 (192888'13665) modify 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.1315259.0:41964 2014-08-11 16:46:58.882090 2014-08-13 18:06:06.804139 7f5c973a57a0 20 read_log 192888'13776 (192888'13673) modify ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.1315259.0:43852 2014-08-11 16:46:59.410666 2014-08-13 18:06:06.804158 7f5c973a57a0 20 read_log 192888'13778 (192888'13677) modify 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.1315259.0:44530 2014-08-11 16:46:59.603721 2014-08-13 18:06:06.804171 7f5c973a57a0 20 read_log 192888'13780 (192888'13681) modify 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.1315259.0:45245 2014-08-11 16:46:59.811513 2014-08-13 18:06:06.804183 7f5c973a57a0 20 read_log 192888'13781 (192888'13683) modify dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.1315259.0:45296 2014-08-11 16:46:59.829908 2014-08-13 18:06:06.804196 7f5c973a57a0 20 read_log 192888'13782 (192888'13685) modify 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.1315259.0:46833 2014-08-11 16:47:00.270962 2014-08-13 18:06:06.804208 7f5c973a57a0 20 read_log 192888'13783 (192888'13687) modify 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by client.1315259.0:47243 2014-08-11 16:47:00.393381 2014-08-13 18:06:06.804220 7f5c973a57a0 20 read_log 192888'13784 (192888'13689) modify 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by client.1315259.0:47645 2014-08-11 16:47:00.522645 2014-08-13 18:06:06.804232 7f5c973a57a0 20 read_log 192888'13786 (192888'13693) modify 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by client.1315259.0:47947 2014-08-11 16:47:00.618658 2014-08-13 18:06:06.804244 7f5c973a57a0 20 read_log 192888'13789 (192888'13699) modify 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.1315259.0:49114 2014-08-11 16:47:00.962539 2014-08-13 18:06:06.804256 7f5c973a57a0 20 read_log 192888'13791 (192888'13703) modify e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by client.1315259.0:50078 2014-08-11 16:47:01.256226 2014-08-13 18:06:06.804271 7f5c973a57a0 20 read_log 192888'13792 (192888'13705) modify 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.1315259.0:50264 2014-08-11 16:47:01.314121 2014-08-13 18:06:06.804284 7f5c973a57a0 20 read_log 192888'13793 (192888'13707) modify dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.1315259.0:50825 2014-08-11 16:47:01.489217 2014-08-13 18:06:06.804296 7f5c973a57a0 20 read_log 192888'13794 (192888'13709) modify e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by client.1315259.0:51144 2014-08-11 16:47:01.592417 2014-08-13 18:06:06.804308 7f5c973a57a0 20 read_log 192888'13796 (192888'13713) modify 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.1315259.0:52445 2014-08-11 16:47:01.975970 2014-08-13 18:06:06.804321 7f5c973a57a0 20 read_log 192888'13797 (192888'13715) modify c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.1315259.0:52861 2014-08-11 16:47:02.089616 2014-08-13 18:06:06.804336 7f5c973a57a0 20 read_log 192888'13799 (192888'13719) modify da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by client.1315259.0:53534 2014-08-11 16:47:02.285108 2014-08-13 18:06:06.804349 7f5c973a57a0 20 read_log 192888'13802 (192888'13725) modify a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by client.1315259.0:56458 2014-08-11 16:47:03.211073 2014-08-13 18:06:06.804408 7f5c973a57a0 20 read_log 192888'13804 (192888'13729) modify 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.1315259.0:58519 2014-08-11 16:47:03.825914 2014-08-13 18:06:06.804424 7f5c973a57a0 20 read_log 192888'13805 (192888'13731) modify b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.1315259.0:58521 2014-08-11 16:47:03.826452 2014-08-13 18:06:06.804450 7f5c973a57a0 20 read_log 192888'13806 (192888'13733) modify f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.1315259.0:58549 2014-08-11 16:47:03.834615 2014-08-13 18:06:06.804466 7f5c973a57a0 20 read_log 192888'13807 (192888'13735) modify d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.1315259.0:58606 2014-08-11 16:47:03.851781 2014-08-13 18:06:06.804479 7f5c973a57a0 20 read_log 192888'13808 (192888'13737) modify bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.1315259.0:58883 2014-08-11 16:47:03.936632 2014-08-13 18:06:06.804499 7f5c973a57a0 20 read_log 192888'13810 (192888'13741) modify df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.1315259.0:60277 2014-08-11 16:47:04.399787 2014-08-13 18:06:06.804514 7f5c973a57a0 20 read_log 192888'13811 (192888'13743) modify 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.1315259.0:60452 2014-08-11 16:47:04.447578 2014-08-13 18:06:06.804527 7f5c973a57a0 20 read_log 192888'13818 (0'0) promote 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by osd.397.0:14657 2014-08-11 17:05:17.690008 2014-08-13 18:06:06.804546 7f5c973a57a0 20 read_log 192888'13819 (192888'13818) modify 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.942868.0:657 2014-08-11 17:05:17.678545 2014-08-13 18:06:06.804562 7f5c973a57a0 20 read_log 192888'13820 (0'0) promote 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.397.0:14659 2014-08-11 17:05:21.130444 2014-08-13 18:06:06.804575 7f5c973a57a0 20 read_log 192888'13821 (192888'13820) modify 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by client.942868.0:1497 2014-08-11 17:05:21.123224 2014-08-13 18:06:06.804590 7f5c973a57a0 20 read_log 192888'13828 (0'0) promote 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by osd.397.0:14667 2014-08-11 17:05:26.815988 2014-08-13 18:06:06.804615 7f5c973a57a0 20 read_log 192888'13829 (192888'13828) modify 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.942868.0:2939 2014-08-11 17:05:26.808260 2014-08-13 18:06:06.804631 7f5c973a57a0 20 read_log 192888'13830 (0'0) promote 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by osd.397.0:14669 2014-08-11 17:05:28.482525 2014-08-13 18:06:06.804644 7f5c973a57a0 20 read_log 192888'13831 (192888'13830) modify 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.942868.0:3389 2014-08-11 17:05:28.471785 2014-08-13 18:06:06.804656 7f5c973a57a0 20 read_log 192888'13832 (0'0) promote d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by osd.397.0:14671 2014-08-11 17:05:33.263246 2014-08-13 18:06:06.804668 7f5c973a57a0 20 read_log 192888'13833 (192888'13832) modify d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.942868.0:4675 2014-08-11 17:05:33.254577 2014-08-13 18:06:06.804681 7f5c973a57a0 20 read_log 192888'13834 (0'0) promote 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.397.0:14673 2014-08-11 17:05:33.651103 2014-08-13 18:06:06.804693 7f5c973a57a0 20 read_log 192888'13835 (192888'13834) modify 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by client.942868.0:4772 2014-08-11 17:05:33.643227 2014-08-13 18:06:06.804706 7f5c973a57a0 20 read_log 192888'13838 (0'0) promote 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.397.0:14677 2014-08-11 17:05:39.302713 2014-08-13 18:06:06.804725 7f5c973a57a0 20 read_log 192888'13839 (192888'13838) modify 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by client.942868.0:6273 2014-08-11 17:05:39.294126 2014-08-13 18:06:06.804740 7f5c973a57a0 20 read_log 192888'13840 (0'0) promote 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by osd.397.0:14679 2014-08-11 17:05:39.854552 2014-08-13 18:06:06.804753 7f5c973a57a0 20 read_log 192888'13841 (192888'13840) modify 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.942868.0:6425 2014-08-11 17:05:39.844940 2014-08-13 18:06:06.804765 7f5c973a57a0 20 read_log 192888'13842 (0'0) promote 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.397.0:14681 2014-08-11 17:05:40.321285 2014-08-13 18:06:06.804777 7f5c973a57a0 20 read_log 192888'13843 (192888'13842) modify 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by client.942868.0:6555 2014-08-11 17:05:40.312320 2014-08-13 18:06:06.804807 7f5c973a57a0 20 read_log 192888'13844 (0'0) promote 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.397.0:14683 2014-08-11 17:05:47.612984 2014-08-13 18:06:06.804822 7f5c973a57a0 20 read_log 192888'13845 (192888'13844) modify 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by client.942868.0:8494 2014-08-11 17:05:47.603625 2014-08-13 18:06:06.804835 7f5c973a57a0 20 read_log 192888'13848 (0'0) promote 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.397.0:14687 2014-08-11 17:05:48.965752 2014-08-13 18:06:06.804847 7f5c973a57a0 20 read_log 192888'13849 (192888'13848) modify 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by client.942868.0:8845 2014-08-11 17:05:48.955129 2014-08-13 18:06:06.804859 7f5c973a57a0 20 read_log 192888'13850 (0'0) promote 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by osd.397.0:14689 2014-08-11 17:05:50.329993 2014-08-13 18:06:06.804871 7f5c973a57a0 20 read_log 192888'13851 (192888'13850) modify 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.942868.0:9223 2014-08-11 17:05:50.319043 2014-08-13 18:06:06.804885 7f5c973a57a0 20 read_log 192888'13854 (0'0) promote 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.397.0:14693 2014-08-11 17:05:51.967949 2014-08-13 18:06:06.804897 7f5c973a57a0 20 read_log 192888'13855 (192888'13854) modify 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by client.942868.0:9657 2014-08-11 17:05:51.956728 2014-08-13 18:06:06.804909 7f5c973a57a0 20 read_log 192888'13856 (0'0) promote ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.397.0:14695 2014-08-11 17:05:52.665385 2014-08-13 18:06:06.804922 7f5c973a57a0 20 read_log 192888'13857 (192888'13856) modify ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by client.942868.0:9849 2014-08-11 17:05:52.654162 2014-08-13 18:06:06.804934 7f5c973a57a0 20 read_log 192888'13858 (0'0) promote 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by osd.397.0:14697 2014-08-11 17:05:52.713490 2014-08-13 18:06:06.804946 7f5c973a57a0 20 read_log 192888'13859 (192888'13858) modify 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.942868.0:9863 2014-08-11 17:05:52.704271 2014-08-13 18:06:06.804958 7f5c973a57a0 20 read_log 192888'13860 (0'0) promote 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by osd.397.0:14699 2014-08-11 17:05:52.930280 2014-08-13 18:06:06.804971 7f5c973a57a0 20 read_log 192888'13861 (192888'13860) modify 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.942868.0:9923 2014-08-11 17:05:52.920296 2014-08-13 18:06:06.804983 7f5c973a57a0 20 read_log 192888'13868 (0'0) promote 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.397.0:14707 2014-08-11 17:05:55.335121 2014-08-13 18:06:06.805002 7f5c973a57a0 20 read_log 192888'13869 (192888'13868) modify 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by client.942868.0:10591 2014-08-11 17:05:55.325428 2014-08-13 18:06:06.805017 7f5c973a57a0 20 read_log 192888'13876 (0'0) promote 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by osd.397.0:14715 2014-08-11 17:06:00.419626 2014-08-13 18:06:06.805030 7f5c973a57a0 20 read_log 192888'13877 (192888'13876) modify 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.942868.0:11974 2014-08-11 17:06:00.408940 2014-08-13 18:06:06.805044 7f5c973a57a0 20 read_log 192888'13886 (0'0) promote 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.397.0:14725 2014-08-11 17:06:16.051915 2014-08-13 18:06:06.805057 7f5c973a57a0 20 read_log 192888'13887 (192888'13886) modify 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by client.942868.0:15985 2014-08-11 17:06:16.043482 2014-08-13 18:06:06.805071 7f5c973a57a0 20 read_log 192888'13890 (0'0) promote c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by osd.397.0:14729 2014-08-11 17:06:20.796566 2014-08-13 18:06:06.805084 7f5c973a57a0 20 read_log 192888'13891 (192888'13890) modify c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.942868.0:17194 2014-08-11 17:06:20.785182 2014-08-13 18:06:06.805098 7f5c973a57a0 20 read_log 192888'13892 (0'0) promote d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by osd.397.0:14731 2014-08-11 17:06:23.270681 2014-08-13 18:06:06.805111 7f5c973a57a0 20 read_log 192888'13893 (192888'13892) modify d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.942868.0:17850 2014-08-11 17:06:23.262069 2014-08-13 18:06:06.805124 7f5c973a57a0 20 read_log 192888'13896 (0'0) promote 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.397.0:14735 2014-08-11 17:06:25.353748 2014-08-13 18:06:06.805136 7f5c973a57a0 20 read_log 192888'13897 (192888'13896) modify 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by client.942868.0:18402 2014-08-11 17:06:25.345935 2014-08-13 18:06:06.805159 7f5c973a57a0 20 read_log 192888'13898 (0'0) promote caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by osd.397.0:14737 2014-08-11 17:06:26.294115 2014-08-13 18:06:06.805175 7f5c973a57a0 20 read_log 192888'13899 (192888'13898) modify caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.942868.0:18650 2014-08-11 17:06:26.281925 2014-08-13 18:06:06.805187 7f5c973a57a0 20 read_log 192888'13902 (0'0) promote 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by osd.397.0:14741 2014-08-11 17:06:28.566697 2014-08-13 18:06:06.805200 7f5c973a57a0 20 read_log 192888'13903 (192888'13902) modify 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.942868.0:19256 2014-08-11 17:06:28.557218 2014-08-13 18:06:06.805212 7f5c973a57a0 20 read_log 192888'13908 (0'0) promote f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by osd.397.0:14747 2014-08-11 17:06:34.160839 2014-08-13 18:06:06.805227 7f5c973a57a0 20 read_log 192888'13909 (192888'13908) modify f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.942868.0:20724 2014-08-11 17:06:34.149718 2014-08-13 18:06:06.805239 7f5c973a57a0 20 read_log 192888'13912 (0'0) promote 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.397.0:14751 2014-08-11 17:06:34.633066 2014-08-13 18:06:06.805257 7f5c973a57a0 20 read_log 192888'13913 (192888'13912) modify 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by client.942868.0:20850 2014-08-11 17:06:34.623585 2014-08-13 18:06:06.805269 7f5c973a57a0 20 read_log 192888'13914 (0'0) promote b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.397.0:14753 2014-08-11 17:06:37.754375 2014-08-13 18:06:06.805295 7f5c973a57a0 20 read_log 192888'13915 (192888'13914) modify b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by client.942868.0:21674 2014-08-11 17:06:37.746402 2014-08-13 18:06:06.805311 7f5c973a57a0 20 read_log 192888'13916 (0'0) promote 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.397.0:14755 2014-08-11 17:06:38.591304 2014-08-13 18:06:06.805324 7f5c973a57a0 20 read_log 192888'13917 (192888'13916) modify 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by client.942868.0:21895 2014-08-11 17:06:38.581491 2014-08-13 18:06:06.805336 7f5c973a57a0 20 read_log 192888'13918 (0'0) promote dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.397.0:14757 2014-08-11 17:06:40.282421 2014-08-13 18:06:06.805352 7f5c973a57a0 20 read_log 192888'13919 (192888'13918) modify dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by client.942868.0:22336 2014-08-11 17:06:40.273835 2014-08-13 18:06:06.805367 7f5c973a57a0 20 read_log 192888'13922 (0'0) promote 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by osd.397.0:14761 2014-08-11 17:06:50.610266 2014-08-13 18:06:06.805388 7f5c973a57a0 20 read_log 192888'13923 (192888'13922) modify 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.942868.0:25054 2014-08-11 17:06:50.601548 2014-08-13 18:06:06.805407 7f5c973a57a0 20 read_log 192888'13930 (0'0) promote c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by osd.397.0:14769 2014-08-11 17:06:56.900790 2014-08-13 18:06:06.805420 7f5c973a57a0 20 read_log 192888'13931 (192888'13930) modify c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.942868.0:26657 2014-08-11 17:06:56.891754 2014-08-13 18:06:06.805440 7f5c973a57a0 20 read_log 192888'13932 (0'0) promote 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by osd.397.0:14771 2014-08-11 17:07:03.695508 2014-08-13 18:06:06.805456 7f5c973a57a0 20 read_log 192888'13933 (192888'13932) modify 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.942868.0:28464 2014-08-11 17:07:03.684227 2014-08-13 18:06:06.805469 7f5c973a57a0 20 read_log 192888'13934 (0'0) promote 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.397.0:14773 2014-08-11 17:07:05.104880 2014-08-13 18:06:06.805488 7f5c973a57a0 20 read_log 192888'13935 (192888'13934) modify 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by client.942868.0:28846 2014-08-11 17:07:05.096653 2014-08-13 18:06:06.805504 7f5c973a57a0 20 read_log 192888'13936 (0'0) promote d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by osd.397.0:14775 2014-08-11 17:07:13.607392 2014-08-13 18:06:06.805516 7f5c973a57a0 20 read_log 192888'13937 (192888'13936) modify d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.942868.0:30949 2014-08-11 17:07:13.598312 2014-08-13 18:06:06.805537 7f5c973a57a0 20 read_log 192888'13938 (192888'13819) modify 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.942868.0:31923 2014-08-11 17:07:16.098614 2014-08-13 18:06:06.805553 7f5c973a57a0 20 read_log 192888'13939 (192888'13821) modify 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by client.942868.0:32763 2014-08-11 17:07:16.339213 2014-08-13 18:06:06.805566 7f5c973a57a0 20 read_log 192888'13943 (192888'13829) modify 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.942868.0:34205 2014-08-11 17:07:16.725700 2014-08-13 18:06:06.805581 7f5c973a57a0 20 read_log 192888'13944 (192888'13831) modify 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.942868.0:34655 2014-08-11 17:07:16.854194 2014-08-13 18:06:06.805596 7f5c973a57a0 20 read_log 192888'13945 (192888'13833) modify d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.942868.0:35941 2014-08-11 17:07:17.202434 2014-08-13 18:06:06.805616 7f5c973a57a0 20 read_log 192888'13946 (192888'13835) modify 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by client.942868.0:36038 2014-08-11 17:07:17.226719 2014-08-13 18:06:06.805631 7f5c973a57a0 20 read_log 192888'13948 (192888'13839) modify 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by client.942868.0:37539 2014-08-11 17:07:17.637342 2014-08-13 18:06:06.805643 7f5c973a57a0 20 read_log 192888'13949 (192888'13841) modify 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.942868.0:37691 2014-08-11 17:07:17.680397 2014-08-13 18:06:06.805656 7f5c973a57a0 20 read_log 192888'13950 (192888'13843) modify 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by client.942868.0:37821 2014-08-11 17:07:17.715687 2014-08-13 18:06:06.805668 7f5c973a57a0 20 read_log 192888'13951 (192888'13845) modify 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by client.942868.0:39760 2014-08-11 17:07:18.275853 2014-08-13 18:06:06.805680 7f5c973a57a0 20 read_log 192888'13953 (192888'13849) modify 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by client.942868.0:40111 2014-08-11 17:07:18.368286 2014-08-13 18:06:06.805692 7f5c973a57a0 20 read_log 192888'13954 (192888'13851) modify 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.942868.0:40489 2014-08-11 17:07:18.466233 2014-08-13 18:06:06.805704 7f5c973a57a0 20 read_log 192888'13956 (192888'13855) modify 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by client.942868.0:40923 2014-08-11 17:07:18.583317 2014-08-13 18:06:06.805718 7f5c973a57a0 20 read_log 192888'13957 (192888'13857) modify ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by client.942868.0:41115 2014-08-11 17:07:18.646246 2014-08-13 18:06:06.805730 7f5c973a57a0 20 read_log 192888'13958 (192888'13859) modify 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.942868.0:41129 2014-08-11 17:07:18.649861 2014-08-13 18:06:06.805742 7f5c973a57a0 20 read_log 192888'13959 (192888'13861) modify 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.942868.0:41189 2014-08-11 17:07:18.665294 2014-08-13 18:06:06.805763 7f5c973a57a0 20 read_log 192888'13963 (192888'13869) modify 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by client.942868.0:41857 2014-08-11 17:07:18.842741 2014-08-13 18:06:06.805777 7f5c973a57a0 20 read_log 192888'13967 (192888'13877) modify 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.942868.0:43240 2014-08-11 17:07:19.217401 2014-08-13 18:06:06.805790 7f5c973a57a0 20 read_log 192888'13972 (192888'13887) modify 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by client.942868.0:47251 2014-08-11 17:07:20.351567 2014-08-13 18:06:06.805802 7f5c973a57a0 20 read_log 192888'13974 (192888'13891) modify c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.942868.0:48460 2014-08-11 17:07:20.725612 2014-08-13 18:06:06.805814 7f5c973a57a0 20 read_log 192888'13975 (192888'13893) modify d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.942868.0:49116 2014-08-11 17:07:20.913876 2014-08-13 18:06:06.805826 7f5c973a57a0 20 read_log 192888'13977 (192888'13897) modify 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by client.942868.0:49668 2014-08-11 17:07:21.084501 2014-08-13 18:06:06.805841 7f5c973a57a0 20 read_log 192888'13978 (192888'13899) modify caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.942868.0:49916 2014-08-11 17:07:21.162265 2014-08-13 18:06:06.805853 7f5c973a57a0 20 read_log 192888'13980 (192888'13903) modify 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.942868.0:50522 2014-08-11 17:07:21.348883 2014-08-13 18:06:06.805865 7f5c973a57a0 20 read_log 192888'13983 (192888'13909) modify f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.942868.0:51990 2014-08-11 17:07:21.810880 2014-08-13 18:06:06.805889 7f5c973a57a0 20 read_log 192888'13985 (192888'13913) modify 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by client.942868.0:52116 2014-08-11 17:07:21.845706 2014-08-13 18:06:06.805904 7f5c973a57a0 20 read_log 192888'13986 (192888'13915) modify b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by client.942868.0:52940 2014-08-11 17:07:22.097157 2014-08-13 18:06:06.805917 7f5c973a57a0 20 read_log 192888'13987 (192888'13917) modify 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by client.942868.0:53161 2014-08-11 17:07:22.170492 2014-08-13 18:06:06.805929 7f5c973a57a0 20 read_log 192888'13988 (192888'13919) modify dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by client.942868.0:53602 2014-08-11 17:07:22.318416 2014-08-13 18:06:06.805945 7f5c973a57a0 20 read_log 192888'13990 (192888'13923) modify 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.942868.0:56320 2014-08-11 17:07:23.113320 2014-08-13 18:06:06.805967 7f5c973a57a0 20 read_log 192888'13994 (192888'13931) modify c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.942868.0:57923 2014-08-11 17:07:23.592548 2014-08-13 18:06:06.805986 7f5c973a57a0 20 read_log 192888'13995 (192888'13933) modify 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.942868.0:59730 2014-08-11 17:07:24.133175 2014-08-13 18:06:06.805999 7f5c973a57a0 20 read_log 192888'13996 (192888'13935) modify 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by client.942868.0:60112 2014-08-11 17:07:24.244978 2014-08-13 18:06:06.806020 7f5c973a57a0 20 read_log 192888'13997 (192888'13937) modify d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.942868.0:62215 2014-08-11 17:07:24.885609 2014-08-13 18:06:06.806035 7f5c973a57a0 20 read_log 193305'13998 (0'0) promote 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by osd.343.0:31 2014-08-11 18:42:38.048812 2014-08-13 18:06:06.806081 7f5c973a57a0 20 read_log 193305'13999 (193305'13998) modify 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.951337.0:1297 2014-08-11 18:42:38.042090 2014-08-13 18:06:06.806094 7f5c973a57a0 20 read_log 193305'14000 (0'0) promote 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:109 2014-08-11 18:42:49.141048 2014-08-13 18:06:06.806106 7f5c973a57a0 20 read_log 193305'14001 (193305'14000) modify 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by client.951337.0:4307 2014-08-11 18:42:49.134695 2014-08-13 18:06:06.806120 7f5c973a57a0 20 read_log 193305'14002 (0'0) promote 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:133 2014-08-11 18:42:52.288739 2014-08-13 18:06:06.806133 7f5c973a57a0 20 read_log 193305'14003 (193305'14002) modify 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by client.951337.0:5136 2014-08-11 18:42:52.281338 2014-08-13 18:06:06.806161 7f5c973a57a0 20 read_log 193305'14004 (0'0) promote 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by osd.343.0:141 2014-08-11 18:42:54.231465 2014-08-13 18:06:06.806177 7f5c973a57a0 20 read_log 193305'14005 (193305'14004) modify 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.951337.0:5671 2014-08-11 18:42:54.224232 2014-08-13 18:06:06.806190 7f5c973a57a0 20 read_log 193305'14006 (0'0) promote 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by osd.343.0:143 2014-08-11 18:42:54.508615 2014-08-13 18:06:06.806204 7f5c973a57a0 20 read_log 193305'14007 (193305'14006) modify 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.951337.0:5747 2014-08-11 18:42:54.501929 2014-08-13 18:06:06.806227 7f5c973a57a0 20 read_log 193305'14008 (0'0) promote 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by osd.343.0:151 2014-08-11 18:42:56.848301 2014-08-13 18:06:06.806254 7f5c973a57a0 20 read_log 193305'14009 (193305'14008) modify 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.951337.0:6392 2014-08-11 18:42:56.840863 2014-08-13 18:06:06.806274 7f5c973a57a0 20 read_log 193305'14010 (0'0) promote 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by osd.343.0:169 2014-08-11 18:43:00.981777 2014-08-13 18:06:06.806286 7f5c973a57a0 20 read_log 193305'14011 (193305'14010) modify 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.951337.0:7535 2014-08-11 18:43:00.974716 2014-08-13 18:06:06.806299 7f5c973a57a0 20 read_log 193305'14012 (0'0) promote 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:177 2014-08-11 18:43:02.541105 2014-08-13 18:06:06.806314 7f5c973a57a0 20 read_log 193305'14013 (193305'14012) modify 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by client.951337.0:7967 2014-08-11 18:43:02.533611 2014-08-13 18:06:06.806334 7f5c973a57a0 20 read_log 193305'14014 (0'0) promote 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by osd.343.0:187 2014-08-11 18:43:05.605905 2014-08-13 18:06:06.806350 7f5c973a57a0 20 read_log 193305'14015 (193305'14014) modify 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.951337.0:8735 2014-08-11 18:43:05.598818 2014-08-13 18:06:06.806363 7f5c973a57a0 20 read_log 193305'14016 (0'0) promote a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by osd.343.0:193 2014-08-11 18:43:07.328681 2014-08-13 18:06:06.806376 7f5c973a57a0 20 read_log 193305'14017 (193305'14016) modify a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.951337.0:9192 2014-08-11 18:43:07.320220 2014-08-13 18:06:06.806396 7f5c973a57a0 20 read_log 193305'14018 (0'0) promote 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:235 2014-08-11 18:43:15.369894 2014-08-13 18:06:06.806410 7f5c973a57a0 20 read_log 193305'14019 (193305'14018) modify 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by client.951337.0:11319 2014-08-11 18:43:15.363106 2014-08-13 18:06:06.806422 7f5c973a57a0 20 read_log 193305'14020 (0'0) promote de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:263 2014-08-11 18:43:20.879278 2014-08-13 18:06:06.806435 7f5c973a57a0 20 read_log 193305'14021 (193305'14020) modify de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by client.951337.0:12796 2014-08-11 18:43:20.871168 2014-08-13 18:06:06.806447 7f5c973a57a0 20 read_log 193305'14022 (0'0) promote a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:269 2014-08-11 18:43:22.984970 2014-08-13 18:06:06.806463 7f5c973a57a0 20 read_log 193305'14023 (193305'14022) modify a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by client.951337.0:13364 2014-08-11 18:43:22.977364 2014-08-13 18:06:06.806505 7f5c973a57a0 20 read_log 193305'14024 (0'0) promote f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by osd.343.0:280 2014-08-11 18:43:24.183269 2014-08-13 18:06:06.806520 7f5c973a57a0 20 read_log 193305'14025 (193305'14024) modify f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.951337.0:13661 2014-08-11 18:43:24.176422 2014-08-13 18:06:06.806533 7f5c973a57a0 20 read_log 193305'14026 (0'0) promote 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:331 2014-08-11 18:43:33.497560 2014-08-13 18:06:06.806545 7f5c973a57a0 20 read_log 193305'14027 (193305'14026) modify 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by client.951337.0:16295 2014-08-11 18:43:33.488144 2014-08-13 18:06:06.806558 7f5c973a57a0 20 read_log 193305'14028 (0'0) promote 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by osd.343.0:355 2014-08-11 18:43:37.736789 2014-08-13 18:06:06.806594 7f5c973a57a0 20 read_log 193305'14029 (193305'14028) modify 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.951337.0:17532 2014-08-11 18:43:37.730110 2014-08-13 18:06:06.806610 7f5c973a57a0 20 read_log 193305'14030 (0'0) promote 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by osd.343.0:367 2014-08-11 18:43:38.451643 2014-08-13 18:06:06.806623 7f5c973a57a0 20 read_log 193305'14031 (193305'14030) modify 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.951337.0:17750 2014-08-11 18:43:38.442836 2014-08-13 18:06:06.806635 7f5c973a57a0 20 read_log 193305'14032 (0'0) promote 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:379 2014-08-11 18:43:42.638035 2014-08-13 18:06:06.806647 7f5c973a57a0 20 read_log 193305'14033 (193305'14032) modify 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by client.951337.0:18913 2014-08-11 18:43:42.630298 2014-08-13 18:06:06.806660 7f5c973a57a0 20 read_log 193305'14034 (0'0) promote 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by osd.343.0:389 2014-08-11 18:43:45.818464 2014-08-13 18:06:06.806672 7f5c973a57a0 20 read_log 193305'14035 (193305'14034) modify 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.951337.0:19775 2014-08-11 18:43:45.810912 2014-08-13 18:06:06.806684 7f5c973a57a0 20 read_log 193305'14036 (0'0) promote 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by osd.343.0:439 2014-08-11 18:43:55.578680 2014-08-13 18:06:06.806696 7f5c973a57a0 20 read_log 193305'14037 (193305'14036) modify 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.951337.0:22476 2014-08-11 18:43:55.571408 2014-08-13 18:06:06.806709 7f5c973a57a0 20 read_log 193305'14038 (0'0) promote aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:488 2014-08-11 18:44:04.840689 2014-08-13 18:06:06.806722 7f5c973a57a0 20 read_log 193305'14039 (193305'14038) modify aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by client.951337.0:25048 2014-08-11 18:44:04.832045 2014-08-13 18:06:06.806734 7f5c973a57a0 20 read_log 193305'14040 (0'0) promote 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by osd.343.0:496 2014-08-11 18:44:06.153479 2014-08-13 18:06:06.806746 7f5c973a57a0 20 read_log 193305'14041 (193305'14040) modify 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.951337.0:25416 2014-08-11 18:44:06.143479 2014-08-13 18:06:06.806761 7f5c973a57a0 20 read_log 193305'14042 (0'0) promote 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:504 2014-08-11 18:44:07.642518 2014-08-13 18:06:06.806773 7f5c973a57a0 20 read_log 193305'14043 (193305'14042) modify 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by client.951337.0:25802 2014-08-11 18:44:07.632289 2014-08-13 18:06:06.806786 7f5c973a57a0 20 read_log 193305'14044 (0'0) promote d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:508 2014-08-11 18:44:09.479971 2014-08-13 18:06:06.806800 7f5c973a57a0 20 read_log 193305'14045 (193305'14044) modify d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by client.951337.0:26298 2014-08-11 18:44:09.472556 2014-08-13 18:06:06.806813 7f5c973a57a0 20 read_log 193305'14046 (0'0) promote 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by osd.343.0:514 2014-08-11 18:44:10.634805 2014-08-13 18:06:06.806828 7f5c973a57a0 20 read_log 193305'14047 (193305'14046) modify 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.951337.0:26622 2014-08-11 18:44:10.627683 2014-08-13 18:06:06.806841 7f5c973a57a0 20 read_log 193305'14048 (0'0) promote 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:528 2014-08-11 18:44:14.033202 2014-08-13 18:06:06.806863 7f5c973a57a0 20 read_log 193305'14049 (193305'14048) modify 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by client.951337.0:27542 2014-08-11 18:44:14.027692 2014-08-13 18:06:06.806878 7f5c973a57a0 20 read_log 193305'14050 (0'0) promote b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by osd.343.0:530 2014-08-11 18:44:14.096084 2014-08-13 18:06:06.806890 7f5c973a57a0 20 read_log 193305'14051 (193305'14050) modify b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.951337.0:27560 2014-08-11 18:44:14.089800 2014-08-13 18:06:06.806905 7f5c973a57a0 20 read_log 193305'14052 (0'0) promote 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by osd.343.0:536 2014-08-11 18:44:14.671586 2014-08-13 18:06:06.806917 7f5c973a57a0 20 read_log 193305'14053 (193305'14052) modify 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.951337.0:27724 2014-08-11 18:44:14.665610 2014-08-13 18:06:06.806932 7f5c973a57a0 20 read_log 193305'14054 (0'0) promote ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by osd.343.0:538 2014-08-11 18:44:15.335198 2014-08-13 18:06:06.806944 7f5c973a57a0 20 read_log 193305'14055 (193305'14054) modify ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.951337.0:27910 2014-08-11 18:44:15.326005 2014-08-13 18:06:06.806957 7f5c973a57a0 20 read_log 193305'14056 (0'0) promote 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by osd.343.0:540 2014-08-11 18:44:15.665202 2014-08-13 18:06:06.806969 7f5c973a57a0 20 read_log 193305'14057 (193305'14056) modify 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.951337.0:28003 2014-08-11 18:44:15.654383 2014-08-13 18:06:06.806982 7f5c973a57a0 20 read_log 193305'14058 (0'0) promote c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by osd.343.0:590 2014-08-11 18:44:27.306817 2014-08-13 18:06:06.806996 7f5c973a57a0 20 read_log 193305'14059 (193305'14058) modify c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.951337.0:31138 2014-08-11 18:44:27.299836 2014-08-13 18:06:06.807010 7f5c973a57a0 20 read_log 193305'14060 (0'0) promote faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by osd.343.0:608 2014-08-11 18:44:30.159446 2014-08-13 18:06:06.807023 7f5c973a57a0 20 read_log 193305'14061 (193305'14060) modify faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.951337.0:31966 2014-08-11 18:44:30.150806 2014-08-13 18:06:06.807037 7f5c973a57a0 20 read_log 193305'14062 (193305'13999) modify 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.951337.0:34008 2014-08-11 18:44:34.756550 2014-08-13 18:06:06.807049 7f5c973a57a0 20 read_log 193305'14063 (193305'14001) modify 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by client.951337.0:37018 2014-08-11 18:44:36.659981 2014-08-13 18:06:06.807064 7f5c973a57a0 20 read_log 193305'14064 (193305'14003) modify 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by client.951337.0:37847 2014-08-11 18:44:37.083272 2014-08-13 18:06:06.807076 7f5c973a57a0 20 read_log 193305'14065 (193305'14005) modify 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.951337.0:38382 2014-08-11 18:44:37.384776 2014-08-13 18:06:06.807090 7f5c973a57a0 20 read_log 193305'14066 (193305'14007) modify 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.951337.0:38458 2014-08-11 18:44:37.466369 2014-08-13 18:06:06.807103 7f5c973a57a0 20 read_log 193305'14067 (193305'14009) modify 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.951337.0:39103 2014-08-11 18:44:37.826849 2014-08-13 18:06:06.807118 7f5c973a57a0 20 read_log 193305'14068 (193305'14011) modify 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.951337.0:40246 2014-08-11 18:44:38.347817 2014-08-13 18:06:06.807140 7f5c973a57a0 20 read_log 193305'14069 (193305'14013) modify 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by client.951337.0:40678 2014-08-11 18:44:38.484971 2014-08-13 18:06:06.807161 7f5c973a57a0 20 read_log 193305'14070 (193305'14015) modify 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.951337.0:41446 2014-08-11 18:44:38.755030 2014-08-13 18:06:06.807174 7f5c973a57a0 20 read_log 193305'14071 (193305'14017) modify a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.951337.0:41903 2014-08-11 18:44:38.918103 2014-08-13 18:06:06.807187 7f5c973a57a0 20 read_log 193305'14072 (193305'14019) modify 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by client.951337.0:44030 2014-08-11 18:44:39.773934 2014-08-13 18:06:06.807199 7f5c973a57a0 20 read_log 193305'14073 (193305'14021) modify de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by client.951337.0:45507 2014-08-11 18:44:40.221818 2014-08-13 18:06:06.807212 7f5c973a57a0 20 read_log 193305'14074 (193305'14023) modify a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by client.951337.0:46075 2014-08-11 18:44:40.381612 2014-08-13 18:06:06.807224 7f5c973a57a0 20 read_log 193305'14075 (193305'14025) modify f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.951337.0:46372 2014-08-11 18:44:40.465456 2014-08-13 18:06:06.807236 7f5c973a57a0 20 read_log 193305'14076 (193305'14027) modify 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by client.951337.0:49006 2014-08-11 18:44:41.223786 2014-08-13 18:06:06.807251 7f5c973a57a0 20 read_log 193305'14077 (193305'14029) modify 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.951337.0:50243 2014-08-11 18:44:41.609803 2014-08-13 18:06:06.807264 7f5c973a57a0 20 read_log 193305'14078 (193305'14031) modify 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.951337.0:50461 2014-08-11 18:44:41.672337 2014-08-13 18:06:06.807276 7f5c973a57a0 20 read_log 193305'14079 (193305'14033) modify 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by client.951337.0:51624 2014-08-11 18:44:42.003577 2014-08-13 18:06:06.807289 7f5c973a57a0 20 read_log 193305'14080 (193305'14035) modify 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.951337.0:52486 2014-08-11 18:44:42.246514 2014-08-13 18:06:06.807301 7f5c973a57a0 20 read_log 193305'14081 (193305'14037) modify 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.951337.0:55187 2014-08-11 18:44:43.048592 2014-08-13 18:06:06.807317 7f5c973a57a0 20 read_log 193305'14082 (193305'14039) modify aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by client.951337.0:57759 2014-08-11 18:44:43.850887 2014-08-13 18:06:06.807332 7f5c973a57a0 20 read_log 193305'14083 (193305'14041) modify 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.951337.0:58127 2014-08-11 18:44:43.954246 2014-08-13 18:06:06.807344 7f5c973a57a0 20 read_log 193305'14084 (193305'14043) modify 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by client.951337.0:58513 2014-08-11 18:44:44.064899 2014-08-13 18:06:06.807360 7f5c973a57a0 20 read_log 193305'14085 (193305'14045) modify d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by client.951337.0:59009 2014-08-11 18:44:44.208661 2014-08-13 18:06:06.807381 7f5c973a57a0 20 read_log 193305'14086 (193305'14047) modify 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.951337.0:59333 2014-08-11 18:44:44.296418 2014-08-13 18:06:06.807398 7f5c973a57a0 20 read_log 193305'14087 (193305'14049) modify 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by client.951337.0:60253 2014-08-11 18:44:44.573408 2014-08-13 18:06:06.807414 7f5c973a57a0 20 read_log 193305'14088 (193305'14051) modify b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.951337.0:60271 2014-08-11 18:44:44.578152 2014-08-13 18:06:06.807445 7f5c973a57a0 20 read_log 193305'14089 (193305'14053) modify 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.951337.0:60435 2014-08-11 18:44:44.628605 2014-08-13 18:06:06.807460 7f5c973a57a0 20 read_log 193305'14090 (193305'14055) modify ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.951337.0:60621 2014-08-11 18:44:44.683170 2014-08-13 18:06:06.807481 7f5c973a57a0 20 read_log 193305'14091 (193305'14057) modify 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.951337.0:60714 2014-08-11 18:44:44.708688 2014-08-13 18:06:06.807496 7f5c973a57a0 20 read_log 193305'14092 (193305'14059) modify c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.951337.0:63849 2014-08-11 18:44:45.627999 2014-08-13 18:06:06.807509 7f5c973a57a0 20 read_log 193305'14093 (193305'14061) modify faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.951337.0:64677 2014-08-11 18:44:45.886776 2014-08-13 18:06:06.807528 7f5c973a57a0 20 read_log 194209'14094 (192888'13985) clean 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.343.0:1011 2014-08-12 14:05:31.063175 2014-08-13 18:06:06.807542 7f5c973a57a0 20 read_log 194209'14095 (192888'13802) clean a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.343.0:1012 2014-08-12 14:05:31.063569 2014-08-13 18:06:06.807555 7f5c973a57a0 20 read_log 194209'14096 (192834'13031) clean abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.343.0:1013 2014-08-12 14:05:31.063911 2014-08-13 18:06:06.807569 7f5c973a57a0 20 read_log 194209'14097 (192834'13062) clean f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.343.0:1014 2014-08-12 14:05:31.064416 2014-08-13 18:06:06.807582 7f5c973a57a0 20 read_log 194209'14098 (192834'13572) clean 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.343.0:1015 2014-08-12 14:05:31.087363 2014-08-13 18:06:06.807598 7f5c973a57a0 20 read_log 194209'14099 (192888'13946) clean 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.343.0:1016 2014-08-12 14:05:31.090313 2014-08-13 18:06:06.807611 7f5c973a57a0 20 read_log 194209'14100 (192888'13784) clean 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.343.0:1017 2014-08-12 14:05:31.093431 2014-08-13 18:06:06.807625 7f5c973a57a0 20 read_log 194209'14101 (192888'13799) clean da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.343.0:1018 2014-08-12 14:05:31.096545 2014-08-13 18:06:06.807638 7f5c973a57a0 20 read_log 194209'14102 (192888'13987) clean 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.343.0:1019 2014-08-12 14:05:31.099930 2014-08-13 18:06:06.807650 7f5c973a57a0 20 read_log 194209'14103 (192888'13988) clean dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.343.0:1020 2014-08-12 14:05:31.103120 2014-08-13 18:06:06.807663 7f5c973a57a0 20 read_log 194209'14104 (192834'13606) clean ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.343.0:1021 2014-08-12 14:05:31.106080 2014-08-13 18:06:06.807675 7f5c973a57a0 20 read_log 194209'14105 (193305'14087) clean 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:1022 2014-08-12 14:05:31.112078 2014-08-13 18:06:06.807687 7f5c973a57a0 20 read_log 194209'14106 (193305'14082) clean aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:1023 2014-08-12 14:05:31.112678 2014-08-13 18:06:06.807699 7f5c973a57a0 20 read_log 194209'14107 (192834'12942) clean f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.343.0:1024 2014-08-12 14:05:31.115759 2014-08-13 18:06:06.807719 7f5c973a57a0 20 read_log 194209'14108 (192834'13571) clean dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.343.0:1025 2014-08-12 14:05:31.119246 2014-08-13 18:06:06.807735 7f5c973a57a0 20 read_log 194209'14109 (192834'13518) clean a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.343.0:1026 2014-08-12 14:05:31.125111 2014-08-13 18:06:06.807758 7f5c973a57a0 20 read_log 194209'14110 (193305'14079) clean 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:1027 2014-08-12 14:05:31.125382 2014-08-13 18:06:06.807771 7f5c973a57a0 20 read_log 194209'14111 (192834'12905) clean 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.343.0:1028 2014-08-12 14:05:31.128573 2014-08-13 18:06:06.807783 7f5c973a57a0 20 read_log 194209'14112 (192888'13752) clean d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.343.0:1029 2014-08-12 14:05:31.131639 2014-08-13 18:06:06.807796 7f5c973a57a0 20 read_log 194209'14113 (192834'12946) clean 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.343.0:1030 2014-08-12 14:05:31.134704 2014-08-13 18:06:06.807808 7f5c973a57a0 20 read_log 194209'14114 (193305'14072) clean 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:1031 2014-08-12 14:05:31.137802 2014-08-13 18:06:06.807820 7f5c973a57a0 20 read_log 194209'14115 (192834'13463) clean 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.343.0:1032 2014-08-12 14:05:31.141170 2014-08-13 18:06:06.807833 7f5c973a57a0 20 read_log 194209'14116 (192834'13563) clean bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.343.0:1033 2014-08-12 14:05:31.144423 2014-08-13 18:06:06.807865 7f5c973a57a0 20 read_log 194209'14117 (192888'13986) clean b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.343.0:1034 2014-08-12 14:05:31.152694 2014-08-13 18:06:06.807880 7f5c973a57a0 20 read_log 194209'14118 (193305'14076) clean 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:1035 2014-08-12 14:05:31.152973 2014-08-13 18:06:06.807892 7f5c973a57a0 20 read_log 194209'14119 (192888'13794) clean e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.343.0:1036 2014-08-12 14:05:31.153231 2014-08-13 18:06:06.807907 7f5c973a57a0 20 read_log 194209'14120 (192888'13786) clean 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.343.0:1037 2014-08-12 14:05:31.153673 2014-08-13 18:06:06.807922 7f5c973a57a0 20 read_log 194209'14121 (193305'14074) clean a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:1038 2014-08-12 14:05:31.161812 2014-08-13 18:06:06.807934 7f5c973a57a0 20 read_log 194209'14122 (193305'14069) clean 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:1039 2014-08-12 14:05:31.167025 2014-08-13 18:06:06.807947 7f5c973a57a0 20 read_log 194209'14123 (192888'13783) clean 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.343.0:1040 2014-08-12 14:05:31.167287 2014-08-13 18:06:06.807962 7f5c973a57a0 20 read_log 194209'14124 (192888'13948) clean 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.343.0:1041 2014-08-12 14:05:31.170837 2014-08-13 18:06:06.807975 7f5c973a57a0 20 read_log 194209'14125 (192888'13791) clean e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.343.0:1042 2014-08-12 14:05:31.174667 2014-08-13 18:06:06.807988 7f5c973a57a0 20 read_log 194209'14126 (192834'13604) clean ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.343.0:1043 2014-08-12 14:05:31.178324 2014-08-13 18:06:06.808002 7f5c973a57a0 20 read_log 194209'14127 (192888'13950) clean 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.343.0:1044 2014-08-12 14:05:31.181996 2014-08-13 18:06:06.808026 7f5c973a57a0 20 read_log 194209'14128 (192888'13972) clean 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.343.0:1045 2014-08-12 14:05:31.185496 2014-08-13 18:06:06.808041 7f5c973a57a0 20 read_log 194209'14129 (192834'12991) clean 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.343.0:1046 2014-08-12 14:05:31.189047 2014-08-13 18:06:06.808054 7f5c973a57a0 20 read_log 194209'14130 (193305'14064) clean 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:1047 2014-08-12 14:05:31.192558 2014-08-13 18:06:06.808068 7f5c973a57a0 20 read_log 194209'14131 (192834'12975) clean c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.343.0:1048 2014-08-12 14:05:31.195963 2014-08-13 18:06:06.808081 7f5c973a57a0 20 read_log 194209'14132 (192888'13939) clean 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.343.0:1049 2014-08-12 14:05:31.199588 2014-08-13 18:06:06.808096 7f5c973a57a0 20 read_log 194209'14133 (193305'14085) clean d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:1050 2014-08-12 14:05:31.203079 2014-08-13 18:06:06.808108 7f5c973a57a0 20 read_log 194209'14134 (192888'13951) clean 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.343.0:1051 2014-08-12 14:05:31.206597 2014-08-13 18:06:06.808120 7f5c973a57a0 20 read_log 194209'14135 (192888'13996) clean 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.343.0:1052 2014-08-12 14:05:31.210234 2014-08-13 18:06:06.808134 7f5c973a57a0 20 read_log 194209'14136 (193305'14084) clean 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:1053 2014-08-12 14:05:31.213661 2014-08-13 18:06:06.808154 7f5c973a57a0 20 read_log 194209'14137 (192888'13771) clean d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.343.0:1054 2014-08-12 14:05:31.217139 2014-08-13 18:06:06.808169 7f5c973a57a0 20 read_log 194209'14138 (193305'14073) clean de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:1055 2014-08-12 14:05:31.220704 2014-08-13 18:06:06.808181 7f5c973a57a0 20 read_log 194209'14139 (192888'13977) clean 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.343.0:1056 2014-08-12 14:05:31.224153 2014-08-13 18:06:06.808194 7f5c973a57a0 20 read_log 194209'14140 (192888'13957) clean ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.343.0:1057 2014-08-12 14:05:31.227740 2014-08-13 18:06:06.808206 7f5c973a57a0 20 read_log 194209'14141 (192834'13595) clean cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.343.0:1058 2014-08-12 14:05:31.231372 2014-08-13 18:06:06.808221 7f5c973a57a0 20 read_log 194209'14142 (192834'13602) clean 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.343.0:1059 2014-08-12 14:05:31.234858 2014-08-13 18:06:06.808234 7f5c973a57a0 20 read_log 194209'14143 (192834'13455) clean 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.343.0:1060 2014-08-12 14:05:31.238669 2014-08-13 18:06:06.808248 7f5c973a57a0 20 read_log 194209'14144 (192834'13011) clean 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.343.0:1061 2014-08-12 14:05:31.242180 2014-08-13 18:06:06.808265 7f5c973a57a0 20 read_log 194209'14145 (192888'13953) clean 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.343.0:1062 2014-08-12 14:05:31.245793 2014-08-13 18:06:06.808278 7f5c973a57a0 20 read_log 194209'14146 (192888'13963) clean 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.343.0:1063 2014-08-12 14:05:31.249375 2014-08-13 18:06:06.808290 7f5c973a57a0 20 read_log 194209'14147 (192888'13956) clean 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.343.0:1064 2014-08-12 14:05:31.252754 2014-08-13 18:06:06.808310 7f5c973a57a0 20 read_log 194209'14148 (194209'14094) delete 3bd62e06/benchmark_data_storage0111-ib.pocstor_2228_object20849/head//24 by osd.343.0:1065 2014-08-12 14:05:31.289310 2014-08-13 18:06:06.808346 7f5c973a57a0 20 read_log 194209'14149 (194209'14095) delete a4f53e06/benchmark_data_storage0111-ib.pocstor_41097_object25730/head//24 by osd.343.0:1066 2014-08-12 14:05:31.292673 2014-08-13 18:06:06.808377 7f5c973a57a0 20 read_log 194209'14150 (194209'14096) delete abb3e06/benchmark_data_storage0111-ib.pocstor_26659_object16380/head//24 by osd.343.0:1067 2014-08-12 14:05:31.295483 2014-08-13 18:06:06.808409 7f5c973a57a0 20 read_log 194209'14151 (194209'14097) delete f8b04e06/benchmark_data_storage0111-ib.pocstor_26659_object21340/head//24 by osd.343.0:1068 2014-08-12 14:05:31.298436 2014-08-13 18:06:06.808453 7f5c973a57a0 20 read_log 194209'14152 (194209'14098) delete 10634e06/benchmark_data_storage0111-ib.pocstor_41526_object16515/head//24 by osd.343.0:1069 2014-08-12 14:05:31.301882 2014-08-13 18:06:06.808502 7f5c973a57a0 20 read_log 194209'14153 (194209'14099) delete 88254e06/benchmark_data_storage0111-ib.pocstor_2228_object4771/head//24 by osd.343.0:1070 2014-08-12 14:05:31.304523 2014-08-13 18:06:06.808517 7f5c973a57a0 20 read_log 194209'14154 (194209'14100) delete 635b4e06/benchmark_data_storage0111-ib.pocstor_41097_object16917/head//24 by osd.343.0:1071 2014-08-12 14:05:31.307451 2014-08-13 18:06:06.808538 7f5c973a57a0 20 read_log 194209'14155 (194209'14101) delete da6b4e06/benchmark_data_storage0111-ib.pocstor_41097_object22806/head//24 by osd.343.0:1072 2014-08-12 14:05:31.310802 2014-08-13 18:06:06.808600 7f5c973a57a0 20 read_log 194209'14156 (194209'14102) delete 313e4e06/benchmark_data_storage0111-ib.pocstor_2228_object21894/head//24 by osd.343.0:1073 2014-08-12 14:05:31.313350 2014-08-13 18:06:06.808628 7f5c973a57a0 20 read_log 194209'14157 (194209'14103) delete dc696e06/benchmark_data_storage0111-ib.pocstor_2228_object22335/head//24 by osd.343.0:1074 2014-08-12 14:05:31.316731 2014-08-13 18:06:06.808651 7f5c973a57a0 20 read_log 194209'14158 (194209'14104) delete ff1b6e06/benchmark_data_storage0111-ib.pocstor_41526_object20748/head//24 by osd.343.0:1075 2014-08-12 14:05:31.319358 2014-08-13 18:06:06.808667 7f5c973a57a0 20 read_log 194209'14159 (194209'14105) delete 5d637e06/benchmark_data_storage0111-ib.pocstor_62621_object27541/head//24 by osd.343.0:1076 2014-08-12 14:05:31.322674 2014-08-13 18:06:06.808680 7f5c973a57a0 20 read_log 194209'14160 (193305'14063) clean 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:1077 2014-08-12 14:05:31.323125 2014-08-13 18:06:06.808695 7f5c973a57a0 20 read_log 194209'14161 (194209'14106) delete aef87e06/benchmark_data_storage0111-ib.pocstor_62621_object25047/head//24 by osd.343.0:1078 2014-08-12 14:05:31.326190 2014-08-13 18:06:06.808709 7f5c973a57a0 20 read_log 194209'14162 (194209'14107) delete f60e7e06/benchmark_data_storage0111-ib.pocstor_26659_object5687/head//24 by osd.343.0:1079 2014-08-12 14:05:31.328939 2014-08-13 18:06:06.808722 7f5c973a57a0 20 read_log 194209'14163 (194209'14108) delete dbcd8e06/benchmark_data_storage0111-ib.pocstor_41526_object16472/head//24 by osd.343.0:1080 2014-08-12 14:05:31.329334 2014-08-13 18:06:06.808734 7f5c973a57a0 20 read_log 194209'14164 (194209'14109) delete a1af8e06/benchmark_data_storage0111-ib.pocstor_41526_object9889/head//24 by osd.343.0:1081 2014-08-12 14:05:31.332871 2014-08-13 18:06:06.808747 7f5c973a57a0 20 read_log 194209'14165 (194209'14110) delete 89e39e06/benchmark_data_storage0111-ib.pocstor_62621_object18912/head//24 by osd.343.0:1082 2014-08-12 14:05:31.333277 2014-08-13 18:06:06.808759 7f5c973a57a0 20 read_log 194209'14166 (194209'14111) delete 62769e06/benchmark_data_storage0111-ib.pocstor_26659_object156/head//24 by osd.343.0:1083 2014-08-12 14:05:31.336571 2014-08-13 18:06:06.808771 7f5c973a57a0 20 read_log 194209'14167 (194209'14112) delete d7ec9e06/benchmark_data_storage0111-ib.pocstor_41097_object28/head//24 by osd.343.0:1084 2014-08-12 14:05:31.337104 2014-08-13 18:06:06.808809 7f5c973a57a0 20 read_log 194209'14168 (194209'14113) delete 3ff0ae06/benchmark_data_storage0111-ib.pocstor_26659_object6073/head//24 by osd.343.0:1085 2014-08-12 14:05:31.337623 2014-08-13 18:06:06.808825 7f5c973a57a0 20 read_log 194209'14169 (194209'14114) delete 6f5fae06/benchmark_data_storage0111-ib.pocstor_62621_object11318/head//24 by osd.343.0:1086 2014-08-12 14:05:31.340657 2014-08-13 18:06:06.808838 7f5c973a57a0 20 read_log 194209'14170 (194209'14115) delete 88b2ce06/benchmark_data_storage0111-ib.pocstor_41526_object3882/head//24 by osd.343.0:1087 2014-08-12 14:05:31.341104 2014-08-13 18:06:06.808850 7f5c973a57a0 20 read_log 194209'14171 (194209'14116) delete bd56ce06/benchmark_data_storage0111-ib.pocstor_41526_object15708/head//24 by osd.343.0:1088 2014-08-12 14:05:31.341684 2014-08-13 18:06:06.808863 7f5c973a57a0 20 read_log 194209'14172 (194209'14117) delete b251de06/benchmark_data_storage0111-ib.pocstor_2228_object21673/head//24 by osd.343.0:1089 2014-08-12 14:05:31.344579 2014-08-13 18:06:06.808875 7f5c973a57a0 20 read_log 194209'14173 (194209'14118) delete 1bc8de06/benchmark_data_storage0111-ib.pocstor_62621_object16294/head//24 by osd.343.0:1090 2014-08-12 14:05:31.345007 2014-08-13 18:06:06.808887 7f5c973a57a0 20 read_log 194209'14174 (194209'14119) delete e65dde06/benchmark_data_storage0111-ib.pocstor_41097_object20416/head//24 by osd.343.0:1091 2014-08-12 14:05:31.345504 2014-08-13 18:06:06.808900 7f5c973a57a0 20 read_log 194209'14175 (194209'14120) delete 78b1ee06/benchmark_data_storage0111-ib.pocstor_41097_object17219/head//24 by osd.343.0:1092 2014-08-12 14:05:31.348307 2014-08-13 18:06:06.808912 7f5c973a57a0 20 read_log 194209'14176 (194209'14121) delete a497ee06/benchmark_data_storage0111-ib.pocstor_62621_object13363/head//24 by osd.343.0:1093 2014-08-12 14:05:31.348606 2014-08-13 18:06:06.808924 7f5c973a57a0 20 read_log 194209'14177 (194209'14122) delete 9793fe06/benchmark_data_storage0111-ib.pocstor_62621_object7966/head//24 by osd.343.0:1094 2014-08-12 14:05:31.349386 2014-08-13 18:06:06.808937 7f5c973a57a0 20 read_log 194209'14178 (194209'14123) delete 723bfe06/benchmark_data_storage0111-ib.pocstor_41097_object16515/head//24 by osd.343.0:1095 2014-08-12 14:05:31.352211 2014-08-13 18:06:06.808951 7f5c973a57a0 20 read_log 194209'14179 (194209'14124) delete 954bfe06/benchmark_data_storage0111-ib.pocstor_2228_object6272/head//24 by osd.343.0:1096 2014-08-12 14:05:31.352548 2014-08-13 18:06:06.808963 7f5c973a57a0 20 read_log 194209'14180 (194209'14125) delete e5f30206/benchmark_data_storage0111-ib.pocstor_41097_object19350/head//24 by osd.343.0:1097 2014-08-12 14:05:31.355866 2014-08-13 18:06:06.808975 7f5c973a57a0 20 read_log 194209'14181 (194209'14126) delete ebf70206/benchmark_data_storage0111-ib.pocstor_41526_object20231/head//24 by osd.343.0:1098 2014-08-12 14:05:31.356181 2014-08-13 18:06:06.808987 7f5c973a57a0 20 read_log 194209'14182 (194209'14127) delete 63db0206/benchmark_data_storage0111-ib.pocstor_2228_object6554/head//24 by osd.343.0:1099 2014-08-12 14:05:31.356892 2014-08-13 18:06:06.809000 7f5c973a57a0 20 read_log 194209'14183 (194209'14128) delete 252e0206/benchmark_data_storage0111-ib.pocstor_2228_object15984/head//24 by osd.343.0:1100 2014-08-12 14:05:31.360111 2014-08-13 18:06:06.809012 7f5c973a57a0 20 read_log 194209'14184 (194209'14129) delete 3d01206/benchmark_data_storage0111-ib.pocstor_26659_object11340/head//24 by osd.343.0:1101 2014-08-12 14:05:31.360453 2014-08-13 18:06:06.809024 7f5c973a57a0 20 read_log 194209'14185 (194209'14130) delete 22331206/benchmark_data_storage0111-ib.pocstor_62621_object5135/head//24 by osd.343.0:1102 2014-08-12 14:05:31.361020 2014-08-13 18:06:06.809037 7f5c973a57a0 20 read_log 194209'14186 (194209'14131) delete c8642206/benchmark_data_storage0111-ib.pocstor_26659_object9657/head//24 by osd.343.0:1103 2014-08-12 14:05:31.364365 2014-08-13 18:06:06.809049 7f5c973a57a0 20 read_log 194209'14187 (194209'14132) delete 40292206/benchmark_data_storage0111-ib.pocstor_2228_object1496/head//24 by osd.343.0:1104 2014-08-12 14:05:31.364704 2014-08-13 18:06:06.809073 7f5c973a57a0 20 read_log 194209'14188 (194209'14133) delete d5213206/benchmark_data_storage0111-ib.pocstor_62621_object26297/head//24 by osd.343.0:1105 2014-08-12 14:05:31.365428 2014-08-13 18:06:06.809088 7f5c973a57a0 20 read_log 194209'14189 (194209'14134) delete 73753206/benchmark_data_storage0111-ib.pocstor_2228_object8493/head//24 by osd.343.0:1106 2014-08-12 14:05:31.368978 2014-08-13 18:06:06.809101 7f5c973a57a0 20 read_log 194209'14190 (194209'14135) delete 85a35206/benchmark_data_storage0111-ib.pocstor_2228_object28845/head//24 by osd.343.0:1107 2014-08-12 14:05:31.369323 2014-08-13 18:06:06.809115 7f5c973a57a0 20 read_log 194209'14191 (194209'14136) delete 10226206/benchmark_data_storage0111-ib.pocstor_62621_object25801/head//24 by osd.343.0:1108 2014-08-12 14:05:31.369914 2014-08-13 18:06:06.809128 7f5c973a57a0 20 read_log 194209'14192 (194209'14137) delete d2c96206/benchmark_data_storage0111-ib.pocstor_41097_object10672/head//24 by osd.343.0:1109 2014-08-12 14:05:31.372840 2014-08-13 18:06:06.809140 7f5c973a57a0 20 read_log 194209'14193 (194209'14138) delete de6b6206/benchmark_data_storage0111-ib.pocstor_62621_object12795/head//24 by osd.343.0:1110 2014-08-12 14:05:31.373155 2014-08-13 18:06:06.809160 7f5c973a57a0 20 read_log 194209'14194 (194209'14139) delete 5c37206/benchmark_data_storage0111-ib.pocstor_2228_object18401/head//24 by osd.343.0:1111 2014-08-12 14:05:31.373864 2014-08-13 18:06:06.809173 7f5c973a57a0 20 read_log 194209'14195 (194209'14140) delete ead57206/benchmark_data_storage0111-ib.pocstor_2228_object9848/head//24 by osd.343.0:1112 2014-08-12 14:05:31.376605 2014-08-13 18:06:06.809186 7f5c973a57a0 20 read_log 194209'14196 (194209'14141) delete cd677206/benchmark_data_storage0111-ib.pocstor_41526_object18968/head//24 by osd.343.0:1113 2014-08-12 14:05:31.376965 2014-08-13 18:06:06.809198 7f5c973a57a0 20 read_log 194209'14197 (194209'14142) delete 95b18206/benchmark_data_storage0111-ib.pocstor_41526_object20137/head//24 by osd.343.0:1114 2014-08-12 14:05:31.377625 2014-08-13 18:06:06.809212 7f5c973a57a0 20 read_log 194209'14198 (194209'14143) delete 99d88206/benchmark_data_storage0111-ib.pocstor_41526_object2707/head//24 by osd.343.0:1115 2014-08-12 14:05:31.380591 2014-08-13 18:06:06.809225 7f5c973a57a0 20 read_log 194209'14199 (194209'14144) delete 8dfb9206/benchmark_data_storage0111-ib.pocstor_26659_object13821/head//24 by osd.343.0:1116 2014-08-12 14:05:31.380932 2014-08-13 18:06:06.809240 7f5c973a57a0 20 read_log 194209'14200 (194209'14145) delete 8c1a206/benchmark_data_storage0111-ib.pocstor_2228_object8844/head//24 by osd.343.0:1117 2014-08-12 14:05:31.381477 2014-08-13 18:06:06.809252 7f5c973a57a0 20 read_log 194209'14201 (194209'14146) delete 5da5a206/benchmark_data_storage0111-ib.pocstor_2228_object10590/head//24 by osd.343.0:1118 2014-08-12 14:05:31.385079 2014-08-13 18:06:06.809267 7f5c973a57a0 20 read_log 194209'14202 (194209'14147) delete 7bb8a206/benchmark_data_storage0111-ib.pocstor_2228_object9656/head//24 by osd.343.0:1119 2014-08-12 14:05:31.385434 2014-08-13 18:06:06.809280 7f5c973a57a0 20 read_log 194209'14203 (194209'14160) delete 492ba206/benchmark_data_storage0111-ib.pocstor_62621_object4306/head//24 by osd.343.0:1120 2014-08-12 14:05:31.385655 2014-08-13 18:06:06.809294 7f5c973a57a0 20 read_log 194215'14204 (194215'14204) modify 206/hit_set_24.206_archive_2014-08-12 15:32:25.739433_2014-08-12 15:44:15.482903/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:15.483247 2014-08-13 18:06:06.809310 7f5c973a57a0 20 read_log 194215'14205 (192888'13810) clean df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by osd.343.0:2585 2014-08-12 15:44:15.487361 2014-08-13 18:06:06.809325 7f5c973a57a0 20 read_log 194215'14206 (194215'14205) delete df83c206/benchmark_data_storage0111-ib.pocstor_41097_object29549/head//24 by client.2174595.0:106897 0.000000 2014-08-13 18:06:06.809344 7f5c973a57a0 20 read_log 194215'14207 (192888'13944) clean 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by osd.343.0:2587 2014-08-12 15:44:15.495491 2014-08-13 18:06:06.809359 7f5c973a57a0 20 read_log 194215'14208 (194215'14207) delete 976d206/benchmark_data_storage0111-ib.pocstor_2228_object3388/head//24 by client.2174595.0:106899 0.000000 2014-08-13 18:06:06.809371 7f5c973a57a0 20 read_log 194215'14209 (192834'13566) clean f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by osd.343.0:2589 2014-08-12 15:44:15.503823 2014-08-13 18:06:06.809401 7f5c973a57a0 20 read_log 194215'14210 (194215'14209) delete f7e8d206/benchmark_data_storage0111-ib.pocstor_41526_object15911/head//24 by client.2174595.0:106901 0.000000 2014-08-13 18:06:06.809414 7f5c973a57a0 20 read_log 194215'14211 (192888'13796) clean 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by osd.343.0:2591 2014-08-12 15:44:15.511222 2014-08-13 18:06:06.809427 7f5c973a57a0 20 read_log 194215'14212 (194215'14211) delete 1bd9d206/benchmark_data_storage0111-ib.pocstor_41097_object21717/head//24 by client.2174595.0:106903 0.000000 2014-08-13 18:06:06.809464 7f5c973a57a0 20 read_log 194215'14213 (192834'12958) clean ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by osd.343.0:2593 2014-08-12 15:44:15.518952 2014-08-13 18:06:06.809486 7f5c973a57a0 20 read_log 194215'14214 (194215'14213) delete ba3ad206/benchmark_data_storage0111-ib.pocstor_26659_object7265/head//24 by client.2174595.0:106905 0.000000 2014-08-13 18:06:06.809515 7f5c973a57a0 20 read_log 194215'14215 (192834'13554) clean 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by osd.343.0:2595 2014-08-12 15:44:15.525956 2014-08-13 18:06:06.809529 7f5c973a57a0 20 read_log 194215'14216 (194215'14215) delete 40ae206/benchmark_data_storage0111-ib.pocstor_41526_object14664/head//24 by client.2174595.0:106907 0.000000 2014-08-13 18:06:06.809540 7f5c973a57a0 20 read_log 194215'14217 (193305'14066) clean 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by osd.343.0:2597 2014-08-12 15:44:15.533120 2014-08-13 18:06:06.809561 7f5c973a57a0 20 read_log 194215'14218 (194215'14217) delete 8c9de206/benchmark_data_storage0111-ib.pocstor_62621_object5746/head//24 by client.2174595.0:106909 0.000000 2014-08-13 18:06:06.809575 7f5c973a57a0 20 read_log 194215'14219 (192834'12952) clean ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by osd.343.0:2599 2014-08-12 15:44:15.540172 2014-08-13 18:06:06.809588 7f5c973a57a0 20 read_log 194215'14220 (194215'14219) delete ee21f206/benchmark_data_storage0111-ib.pocstor_26659_object6778/head//24 by client.2174595.0:106911 0.000000 2014-08-13 18:06:06.809599 7f5c973a57a0 20 read_log 194215'14221 (192834'13445) clean 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by osd.343.0:2601 2014-08-12 15:44:15.547485 2014-08-13 18:06:06.809612 7f5c973a57a0 20 read_log 194215'14222 (194215'14221) delete 85a20606/benchmark_data_storage0111-ib.pocstor_41526_object785/head//24 by client.2174595.0:106913 0.000000 2014-08-13 18:06:06.809630 7f5c973a57a0 20 read_log 194215'14223 (192888'13990) clean 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by osd.343.0:2603 2014-08-12 15:44:15.554528 2014-08-13 18:06:06.809646 7f5c973a57a0 20 read_log 194215'14224 (194215'14223) delete 6aa50606/benchmark_data_storage0111-ib.pocstor_2228_object25053/head//24 by client.2174595.0:106915 0.000000 2014-08-13 18:06:06.809661 7f5c973a57a0 20 read_log 194215'14225 (192888'13954) clean 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by osd.343.0:2605 2014-08-12 15:44:15.561308 2014-08-13 18:06:06.809674 7f5c973a57a0 20 read_log 194215'14226 (194215'14225) delete 76fd0606/benchmark_data_storage0111-ib.pocstor_2228_object9222/head//24 by client.2174595.0:106917 0.000000 2014-08-13 18:06:06.809692 7f5c973a57a0 20 read_log 194215'14227 (192834'12929) clean 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by osd.343.0:2607 2014-08-12 15:44:15.568994 2014-08-13 18:06:06.809707 7f5c973a57a0 20 read_log 194215'14228 (194215'14227) delete 7ef1606/benchmark_data_storage0111-ib.pocstor_26659_object4344/head//24 by client.2174595.0:106919 0.000000 2014-08-13 18:06:06.809718 7f5c973a57a0 20 read_log 194215'14229 (192888'13943) clean 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by osd.343.0:2609 2014-08-12 15:44:15.576142 2014-08-13 18:06:06.809731 7f5c973a57a0 20 read_log 194215'14230 (194215'14229) delete 27a02606/benchmark_data_storage0111-ib.pocstor_2228_object2938/head//24 by client.2174595.0:106921 0.000000 2014-08-13 18:06:06.809741 7f5c973a57a0 20 read_log 194215'14231 (193305'14062) clean 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by osd.343.0:2611 2014-08-12 15:44:15.583654 2014-08-13 18:06:06.809754 7f5c973a57a0 20 read_log 194215'14232 (194215'14231) delete 3a382606/benchmark_data_storage0111-ib.pocstor_62621_object1296/head//24 by client.2174595.0:106923 0.000000 2014-08-13 18:06:06.809765 7f5c973a57a0 20 read_log 194215'14233 (192888'13945) clean d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by osd.343.0:2613 2014-08-12 15:44:15.590597 2014-08-13 18:06:06.809785 7f5c973a57a0 20 read_log 194215'14234 (194215'14233) delete d44f2606/benchmark_data_storage0111-ib.pocstor_2228_object4674/head//24 by client.2174595.0:106925 0.000000 2014-08-13 18:06:06.809797 7f5c973a57a0 20 read_log 194215'14235 (193305'14078) clean 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by osd.343.0:2615 2014-08-12 15:44:15.597818 2014-08-13 18:06:06.809810 7f5c973a57a0 20 read_log 194215'14236 (194215'14235) delete 4f43606/benchmark_data_storage0111-ib.pocstor_62621_object17749/head//24 by client.2174595.0:106927 0.000000 2014-08-13 18:06:06.809821 7f5c973a57a0 20 read_log 194215'14237 (192888'13782) clean 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by osd.343.0:2617 2014-08-12 15:44:15.604059 2014-08-13 18:06:06.809833 7f5c973a57a0 20 read_log 194215'14238 (194215'14237) delete 70b53606/benchmark_data_storage0111-ib.pocstor_41097_object16105/head//24 by client.2174595.0:106929 0.000000 2014-08-13 18:06:06.809844 7f5c973a57a0 20 read_log 194215'14239 (192834'12994) clean 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by osd.343.0:2619 2014-08-12 15:44:15.610345 2014-08-13 18:06:06.809857 7f5c973a57a0 20 read_log 194215'14240 (194215'14239) delete 710c3606/benchmark_data_storage0111-ib.pocstor_26659_object11763/head//24 by client.2174595.0:106931 0.000000 2014-08-13 18:06:06.809868 7f5c973a57a0 20 read_log 194215'14241 (193305'14092) clean c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by osd.343.0:2621 2014-08-12 15:44:15.616755 2014-08-13 18:06:06.809881 7f5c973a57a0 20 read_log 194215'14242 (194215'14241) delete c16d3606/benchmark_data_storage0111-ib.pocstor_62621_object31137/head//24 by client.2174595.0:106933 0.000000 2014-08-13 18:06:06.809892 7f5c973a57a0 20 read_log 194215'14243 (192888'13975) clean d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by osd.343.0:2623 2014-08-12 15:44:15.624018 2014-08-13 18:06:06.809904 7f5c973a57a0 20 read_log 194215'14244 (194215'14243) delete d41c4606/benchmark_data_storage0111-ib.pocstor_2228_object17849/head//24 by client.2174595.0:106935 0.000000 2014-08-13 18:06:06.809918 7f5c973a57a0 20 read_log 194215'14245 (192888'13759) clean 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by osd.343.0:2625 2014-08-12 15:44:15.630848 2014-08-13 18:06:06.809931 7f5c973a57a0 20 read_log 194215'14246 (194215'14245) delete 2b316606/benchmark_data_storage0111-ib.pocstor_41097_object4224/head//24 by client.2174595.0:106937 0.000000 2014-08-13 18:06:06.809952 7f5c973a57a0 20 read_log 194215'14247 (192888'13781) clean dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by osd.343.0:2627 2014-08-12 15:44:15.637682 2014-08-13 18:06:06.809965 7f5c973a57a0 20 read_log 194215'14248 (194215'14247) delete dea16606/benchmark_data_storage0111-ib.pocstor_41097_object14568/head//24 by client.2174595.0:106939 0.000000 2014-08-13 18:06:06.809976 7f5c973a57a0 20 read_log 194215'14249 (192888'13958) clean 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by osd.343.0:2629 2014-08-12 15:44:15.644360 2014-08-13 18:06:06.809988 7f5c973a57a0 20 read_log 194215'14250 (194215'14249) delete 298f6606/benchmark_data_storage0111-ib.pocstor_2228_object9862/head//24 by client.2174595.0:106941 0.000000 2014-08-13 18:06:06.810003 7f5c973a57a0 20 read_log 194215'14251 (193305'14065) clean 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by osd.343.0:2631 2014-08-12 15:44:15.651474 2014-08-13 18:06:06.810019 7f5c973a57a0 20 read_log 194215'14252 (194215'14251) delete 1c217606/benchmark_data_storage0111-ib.pocstor_62621_object5670/head//24 by client.2174595.0:106943 0.000000 2014-08-13 18:06:06.810039 7f5c973a57a0 20 read_log 194215'14253 (193305'14088) clean b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by osd.343.0:2633 2014-08-12 15:44:15.658319 2014-08-13 18:06:06.810055 7f5c973a57a0 20 read_log 194215'14254 (194215'14253) delete b2667606/benchmark_data_storage0111-ib.pocstor_62621_object27559/head//24 by client.2174595.0:106945 0.000000 2014-08-13 18:06:06.810066 7f5c973a57a0 20 read_log 194215'14255 (193305'14070) clean 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by osd.343.0:2635 2014-08-12 15:44:15.665789 2014-08-13 18:06:06.810088 7f5c973a57a0 20 read_log 194215'14256 (194215'14255) delete 25ff7606/benchmark_data_storage0111-ib.pocstor_62621_object8734/head//24 by client.2174595.0:106947 0.000000 2014-08-13 18:06:06.810099 7f5c973a57a0 20 read_log 194215'14257 (192888'13980) clean 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by osd.343.0:2637 2014-08-12 15:44:15.672323 2014-08-13 18:06:06.810112 7f5c973a57a0 20 read_log 194215'14258 (194215'14257) delete 61508606/benchmark_data_storage0111-ib.pocstor_2228_object19255/head//24 by client.2174595.0:106949 0.000000 2014-08-13 18:06:06.810130 7f5c973a57a0 20 read_log 194215'14259 (193305'14075) clean f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by osd.343.0:2639 2014-08-12 15:44:15.679127 2014-08-13 18:06:06.810151 7f5c973a57a0 20 read_log 194215'14260 (194215'14259) delete f8578606/benchmark_data_storage0111-ib.pocstor_62621_object13660/head//24 by client.2174595.0:106951 0.000000 2014-08-13 18:06:06.810165 7f5c973a57a0 20 read_log 194215'14261 (192888'13959) clean 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by osd.343.0:2641 2014-08-12 15:44:15.685731 2014-08-13 18:06:06.810188 7f5c973a57a0 20 read_log 194215'14262 (194215'14261) delete 4c8e8606/benchmark_data_storage0111-ib.pocstor_2228_object9922/head//24 by client.2174595.0:106953 0.000000 2014-08-13 18:06:06.810200 7f5c973a57a0 20 read_log 194215'14263 (192834'13536) clean f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by osd.343.0:2643 2014-08-12 15:44:15.692525 2014-08-13 18:06:06.810213 7f5c973a57a0 20 read_log 194215'14264 (194215'14263) delete f1f19606/benchmark_data_storage0111-ib.pocstor_41526_object12232/head//24 by client.2174595.0:106955 0.000000 2014-08-13 18:06:06.810224 7f5c973a57a0 20 read_log 194215'14265 (192888'13772) clean 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by osd.343.0:2645 2014-08-12 15:44:15.699187 2014-08-13 18:06:06.810236 7f5c973a57a0 20 read_log 194215'14266 (194215'14265) delete 5cc79606/benchmark_data_storage0111-ib.pocstor_41097_object11236/head//24 by client.2174595.0:106957 0.000000 2014-08-13 18:06:06.810254 7f5c973a57a0 20 read_log 194215'14267 (192834'13056) clean 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by osd.343.0:2647 2014-08-12 15:44:15.706179 2014-08-13 18:06:06.810274 7f5c973a57a0 20 read_log 194215'14268 (194215'14267) delete 70af9606/benchmark_data_storage0111-ib.pocstor_26659_object20337/head//24 by client.2174595.0:106959 0.000000 2014-08-13 18:06:06.810286 7f5c973a57a0 20 read_log 194215'14269 (192834'13029) clean 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by osd.343.0:2649 2014-08-12 15:44:15.712788 2014-08-13 18:06:06.810298 7f5c973a57a0 20 read_log 194215'14270 (194215'14269) delete 17d0a606/benchmark_data_storage0111-ib.pocstor_26659_object16040/head//24 by client.2174595.0:106961 0.000000 2014-08-13 18:06:06.810309 7f5c973a57a0 20 read_log 194215'14271 (192888'13793) clean dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by osd.343.0:2651 2014-08-12 15:44:15.720060 2014-08-13 18:06:06.810322 7f5c973a57a0 20 read_log 194215'14272 (194215'14271) delete dc17a606/benchmark_data_storage0111-ib.pocstor_41097_object20097/head//24 by client.2174595.0:106963 0.000000 2014-08-13 18:06:06.810332 7f5c973a57a0 20 read_log 194215'14273 (192888'13789) clean 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by osd.343.0:2653 2014-08-12 15:44:15.727003 2014-08-13 18:06:06.810345 7f5c973a57a0 20 read_log 194215'14274 (194215'14273) delete 3d29a606/benchmark_data_storage0111-ib.pocstor_41097_object18386/head//24 by client.2174595.0:106965 0.000000 2014-08-13 18:06:06.810360 7f5c973a57a0 20 read_log 194215'14275 (192888'13805) clean b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by osd.343.0:2655 2014-08-12 15:44:15.733966 2014-08-13 18:06:06.810373 7f5c973a57a0 20 read_log 194215'14276 (194215'14275) delete b77b606/benchmark_data_storage0111-ib.pocstor_41097_object27793/head//24 by client.2174595.0:106967 0.000000 2014-08-13 18:06:06.810383 7f5c973a57a0 20 read_log 194215'14277 (192834'12989) clean 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by osd.343.0:2657 2014-08-12 15:44:15.741255 2014-08-13 18:06:06.810407 7f5c973a57a0 20 read_log 194215'14278 (194215'14277) delete 5e18b606/benchmark_data_storage0111-ib.pocstor_26659_object11026/head//24 by client.2174595.0:106969 0.000000 2014-08-13 18:06:06.810421 7f5c973a57a0 20 read_log 194215'14279 (192834'13068) clean 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by osd.343.0:2659 2014-08-12 15:44:15.748625 2014-08-13 18:06:06.810434 7f5c973a57a0 20 read_log 194215'14280 (194215'14279) delete 64abb606/benchmark_data_storage0111-ib.pocstor_26659_object21858/head//24 by client.2174595.0:106971 0.000000 2014-08-13 18:06:06.810459 7f5c973a57a0 20 read_log 194215'14281 (192888'13806) clean f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by osd.343.0:2661 2014-08-12 15:44:15.755637 2014-08-13 18:06:06.810475 7f5c973a57a0 20 read_log 194215'14282 (194215'14281) delete f4b1c606/benchmark_data_storage0111-ib.pocstor_41097_object27821/head//24 by client.2174595.0:106973 0.000000 2014-08-13 18:06:06.810486 7f5c973a57a0 20 read_log 194215'14283 (192834'13042) clean 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by osd.343.0:2663 2014-08-12 15:44:15.762418 2014-08-13 18:06:06.810498 7f5c973a57a0 20 read_log 194215'14284 (194215'14283) delete 10b6c606/benchmark_data_storage0111-ib.pocstor_26659_object18051/head//24 by client.2174595.0:106975 0.000000 2014-08-13 18:06:06.810521 7f5c973a57a0 20 read_log 194215'14285 (192888'13755) clean 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by osd.343.0:2665 2014-08-12 15:44:15.769184 2014-08-13 18:06:06.810536 7f5c973a57a0 20 read_log 194215'14286 (194215'14285) delete 7220d606/benchmark_data_storage0111-ib.pocstor_41097_object665/head//24 by client.2174595.0:106977 0.000000 2014-08-13 18:06:06.810555 7f5c973a57a0 20 read_log 194215'14287 (192888'13807) clean d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by osd.343.0:2667 2014-08-12 15:44:15.776061 2014-08-13 18:06:06.810570 7f5c973a57a0 20 read_log 194215'14288 (194215'14287) delete d67ad606/benchmark_data_storage0111-ib.pocstor_41097_object27878/head//24 by client.2174595.0:106979 0.000000 2014-08-13 18:06:06.810582 7f5c973a57a0 20 read_log 194215'14289 (192834'13055) clean f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by osd.343.0:2669 2014-08-12 15:44:15.783231 2014-08-13 18:06:06.810594 7f5c973a57a0 20 read_log 194215'14290 (194215'14289) delete f57bd606/benchmark_data_storage0111-ib.pocstor_26659_object20089/head//24 by client.2174595.0:106981 0.000000 2014-08-13 18:06:06.810605 7f5c973a57a0 20 read_log 194215'14291 (192888'13967) clean 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by osd.343.0:2671 2014-08-12 15:44:15.790603 2014-08-13 18:06:06.810620 7f5c973a57a0 20 read_log 194215'14292 (194215'14291) delete 398dd606/benchmark_data_storage0111-ib.pocstor_2228_object11973/head//24 by client.2174595.0:106983 0.000000 2014-08-13 18:06:06.810637 7f5c973a57a0 20 read_log 194215'14293 (193305'14071) clean a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by osd.343.0:2673 2014-08-12 15:44:15.797755 2014-08-13 18:06:06.810650 7f5c973a57a0 20 read_log 194215'14294 (194215'14293) delete a9b7e606/benchmark_data_storage0111-ib.pocstor_62621_object9191/head//24 by client.2174595.0:106985 0.000000 2014-08-13 18:06:06.810664 7f5c973a57a0 20 read_log 194215'14295 (192834'13047) clean b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by osd.343.0:2675 2014-08-12 15:44:15.804737 2014-08-13 18:06:06.810677 7f5c973a57a0 20 read_log 194215'14296 (194215'14295) delete b48ae606/benchmark_data_storage0111-ib.pocstor_26659_object19615/head//24 by client.2174595.0:106987 0.000000 2014-08-13 18:06:06.810688 7f5c973a57a0 20 read_log 194215'14297 (193305'14093) clean faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by osd.343.0:2677 2014-08-12 15:44:15.811871 2014-08-13 18:06:06.810711 7f5c973a57a0 20 read_log 194215'14298 (194215'14297) delete faaae606/benchmark_data_storage0111-ib.pocstor_62621_object31965/head//24 by client.2174595.0:106989 0.000000 2014-08-13 18:06:06.810726 7f5c973a57a0 20 read_log 194215'14299 (192834'13482) clean a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by osd.343.0:2679 2014-08-12 15:44:15.819454 2014-08-13 18:06:06.810738 7f5c973a57a0 20 read_log 194215'14300 (194215'14299) delete a9bae606/benchmark_data_storage0111-ib.pocstor_41526_object6544/head//24 by client.2174595.0:106991 0.000000 2014-08-13 18:06:06.810749 7f5c973a57a0 20 read_log 194215'14301 (192888'13769) clean 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by osd.343.0:2681 2014-08-12 15:44:15.826871 2014-08-13 18:06:06.810764 7f5c973a57a0 20 read_log 194215'14302 (194215'14301) delete 84ebe606/benchmark_data_storage0111-ib.pocstor_41097_object10229/head//24 by client.2174595.0:106993 0.000000 2014-08-13 18:06:06.810793 7f5c973a57a0 20 read_log 194215'14303 (192888'13792) clean 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by osd.343.0:2683 2014-08-12 15:44:15.834403 2014-08-13 18:06:06.810809 7f5c973a57a0 20 read_log 194215'14304 (194215'14303) delete 911ce606/benchmark_data_storage0111-ib.pocstor_41097_object19536/head//24 by client.2174595.0:106995 0.000000 2014-08-13 18:06:06.810827 7f5c973a57a0 20 read_log 194215'14305 (192888'13938) clean 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by osd.343.0:2685 2014-08-12 15:44:15.841939 2014-08-13 18:06:06.810843 7f5c973a57a0 20 read_log 194215'14306 (194215'14305) delete 3e5fe606/benchmark_data_storage0111-ib.pocstor_2228_object656/head//24 by client.2174595.0:106997 0.000000 2014-08-13 18:06:06.810883 7f5c973a57a0 20 read_log 194215'14307 (193305'14081) clean 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by osd.343.0:2687 2014-08-12 15:44:15.848939 2014-08-13 18:06:06.810899 7f5c973a57a0 20 read_log 194215'14308 (194215'14307) delete 26b9f606/benchmark_data_storage0111-ib.pocstor_62621_object22475/head//24 by client.2174595.0:106999 0.000000 2014-08-13 18:06:06.810910 7f5c973a57a0 20 read_log 194215'14309 (192834'12972) clean b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by osd.343.0:2689 2014-08-12 15:44:15.855848 2014-08-13 18:06:06.810923 7f5c973a57a0 20 read_log 194215'14310 (194215'14309) delete b7d20a06/benchmark_data_storage0111-ib.pocstor_26659_object9173/head//24 by client.2174595.0:107001 0.000000 2014-08-13 18:06:06.810934 7f5c973a57a0 20 read_log 194215'14311 (192888'13797) clean c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by osd.343.0:2691 2014-08-12 15:44:15.863081 2014-08-13 18:06:06.810957 7f5c973a57a0 20 read_log 194215'14312 (194215'14311) delete c5250a06/benchmark_data_storage0111-ib.pocstor_41097_object22133/head//24 by client.2174595.0:107003 0.000000 2014-08-13 18:06:06.811009 7f5c973a57a0 20 read_log 194215'14313 (192888'13753) clean 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by osd.343.0:2693 2014-08-12 15:44:15.870449 2014-08-13 18:06:06.811022 7f5c973a57a0 20 read_log 194215'14314 (194215'14313) delete 754c0a06/benchmark_data_storage0111-ib.pocstor_41097_object629/head//24 by client.2174595.0:107005 0.000000 2014-08-13 18:06:06.811033 7f5c973a57a0 20 read_log 194215'14315 (192834'13580) clean 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by osd.343.0:2695 2014-08-12 15:44:15.877601 2014-08-13 18:06:06.811045 7f5c973a57a0 20 read_log 194215'14316 (194215'14315) delete 505f0a06/benchmark_data_storage0111-ib.pocstor_41526_object17282/head//24 by client.2174595.0:107007 0.000000 2014-08-13 18:06:06.811057 7f5c973a57a0 20 read_log 194215'14317 (192834'13608) clean c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by osd.343.0:2697 2014-08-12 15:44:15.885776 2014-08-13 18:06:06.811071 7f5c973a57a0 20 read_log 194215'14318 (194215'14317) delete c1e51a06/benchmark_data_storage0111-ib.pocstor_41526_object21086/head//24 by client.2174595.0:107009 0.000000 2014-08-13 18:06:06.811082 7f5c973a57a0 20 read_log 194215'14319 (192834'12909) clean 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by osd.343.0:2699 2014-08-12 15:44:15.893010 2014-08-13 18:06:06.811095 7f5c973a57a0 20 read_log 194215'14320 (194215'14319) delete 5d8d1a06/benchmark_data_storage0111-ib.pocstor_26659_object866/head//24 by client.2174595.0:107011 0.000000 2014-08-13 18:06:06.811106 7f5c973a57a0 20 read_log 194215'14321 (192888'13995) clean 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by osd.343.0:2701 2014-08-12 15:44:15.899983 2014-08-13 18:06:06.811118 7f5c973a57a0 20 read_log 194215'14322 (194215'14321) delete 6f0f1a06/benchmark_data_storage0111-ib.pocstor_2228_object28463/head//24 by client.2174595.0:107013 0.000000 2014-08-13 18:06:06.811129 7f5c973a57a0 20 read_log 194215'14323 (192888'13811) clean 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by osd.343.0:2703 2014-08-12 15:44:15.907128 2014-08-13 18:06:06.811141 7f5c973a57a0 20 read_log 194215'14324 (194215'14323) delete 1f142a06/benchmark_data_storage0111-ib.pocstor_41097_object29724/head//24 by client.2174595.0:107015 0.000000 2014-08-13 18:06:06.811158 7f5c973a57a0 20 read_log 194215'14325 (192888'13780) clean 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by osd.343.0:2705 2014-08-12 15:44:15.914858 2014-08-13 18:06:06.811171 7f5c973a57a0 20 read_log 194215'14326 (194215'14325) delete 94e03a06/benchmark_data_storage0111-ib.pocstor_41097_object14517/head//24 by client.2174595.0:107017 0.000000 2014-08-13 18:06:06.811189 7f5c973a57a0 20 read_log 194215'14327 (192888'13760) clean c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by osd.343.0:2707 2014-08-12 15:44:15.921502 2014-08-13 18:06:06.811204 7f5c973a57a0 20 read_log 194215'14328 (194215'14327) delete c973a06/benchmark_data_storage0111-ib.pocstor_41097_object5054/head//24 by client.2174595.0:107019 0.000000 2014-08-13 18:06:06.811217 7f5c973a57a0 20 read_log 194215'14329 (192834'13511) clean ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by osd.343.0:2709 2014-08-12 15:44:15.928017 2014-08-13 18:06:06.811230 7f5c973a57a0 20 read_log 194215'14330 (194215'14329) delete ac964a06/benchmark_data_storage0111-ib.pocstor_41526_object9260/head//24 by client.2174595.0:107021 0.000000 2014-08-13 18:06:06.811241 7f5c973a57a0 20 read_log 194215'14331 (192834'13620) clean eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by osd.343.0:2711 2014-08-12 15:44:15.935218 2014-08-13 18:06:06.811253 7f5c973a57a0 20 read_log 194215'14332 (194215'14331) delete eca94a06/benchmark_data_storage0111-ib.pocstor_41526_object22854/head//24 by client.2174595.0:107023 0.000000 2014-08-13 18:06:06.811264 7f5c973a57a0 20 read_log 194215'14333 (192888'13997) clean d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by osd.343.0:2713 2014-08-12 15:44:15.942705 2014-08-13 18:06:06.811276 7f5c973a57a0 20 read_log 194215'14334 (194215'14333) delete d3ce4a06/benchmark_data_storage0111-ib.pocstor_2228_object30948/head//24 by client.2174595.0:107025 0.000000 2014-08-13 18:06:06.811287 7f5c973a57a0 20 read_log 194215'14335 (193305'14086) clean 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by osd.343.0:2715 2014-08-12 15:44:15.949305 2014-08-13 18:06:06.811302 7f5c973a57a0 20 read_log 194215'14336 (194215'14335) delete 113a5a06/benchmark_data_storage0111-ib.pocstor_62621_object26621/head//24 by client.2174595.0:107027 0.000000 2014-08-13 18:06:06.811313 7f5c973a57a0 20 read_log 194215'14337 (192834'13474) clean deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by osd.343.0:2717 2014-08-12 15:44:15.956674 2014-08-13 18:06:06.811325 7f5c973a57a0 20 read_log 194215'14338 (194215'14337) delete deaa5a06/benchmark_data_storage0111-ib.pocstor_41526_object5700/head//24 by client.2174595.0:107029 0.000000 2014-08-13 18:06:06.811336 7f5c973a57a0 20 read_log 194215'14339 (193305'14091) clean 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by osd.343.0:2719 2014-08-12 15:44:15.964166 2014-08-13 18:06:06.811352 7f5c973a57a0 20 read_log 194215'14340 (194215'14339) delete 6c2c6a06/benchmark_data_storage0111-ib.pocstor_62621_object28002/head//24 by client.2174595.0:107031 0.000000 2014-08-13 18:06:06.811363 7f5c973a57a0 20 read_log 194215'14341 (192834'13037) clean 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by osd.343.0:2721 2014-08-12 15:44:15.971439 2014-08-13 18:06:06.811377 7f5c973a57a0 20 read_log 194215'14342 (194215'14341) delete 3a619a06/benchmark_data_storage0111-ib.pocstor_26659_object17656/head//24 by client.2174595.0:107033 0.000000 2014-08-13 18:06:06.811388 7f5c973a57a0 20 read_log 194215'14343 (192888'13983) clean f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by osd.343.0:2723 2014-08-12 15:44:15.979108 2014-08-13 18:06:06.811403 7f5c973a57a0 20 read_log 194215'14344 (194215'14343) delete f0b19a06/benchmark_data_storage0111-ib.pocstor_2228_object20723/head//24 by client.2174595.0:107035 0.000000 2014-08-13 18:06:06.811414 7f5c973a57a0 20 read_log 194215'14345 (192888'13978) clean caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by osd.343.0:2725 2014-08-12 15:44:15.986811 2014-08-13 18:06:06.811436 7f5c973a57a0 20 read_log 194215'14346 (194215'14345) delete caf1aa06/benchmark_data_storage0111-ib.pocstor_2228_object18649/head//24 by client.2174595.0:107037 0.000000 2014-08-13 18:06:06.811461 7f5c973a57a0 20 read_log 194215'14347 (192834'13458) clean fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by osd.343.0:2727 2014-08-12 15:44:15.993904 2014-08-13 18:06:06.811484 7f5c973a57a0 20 read_log 194215'14348 (194215'14347) delete fe52aa06/benchmark_data_storage0111-ib.pocstor_41526_object3342/head//24 by client.2174595.0:107039 0.000000 2014-08-13 18:06:06.811498 7f5c973a57a0 20 read_log 194215'14349 (193305'14067) clean 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by osd.343.0:2729 2014-08-12 15:44:16.001260 2014-08-13 18:06:06.811511 7f5c973a57a0 20 read_log 194215'14350 (194215'14349) delete 241aaa06/benchmark_data_storage0111-ib.pocstor_62621_object6391/head//24 by client.2174595.0:107041 0.000000 2014-08-13 18:06:06.811529 7f5c973a57a0 20 read_log 194215'14351 (192888'13778) clean 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by osd.343.0:2731 2014-08-12 15:44:16.009241 2014-08-13 18:06:06.811545 7f5c973a57a0 20 read_log 194215'14352 (194215'14351) delete 691baa06/benchmark_data_storage0111-ib.pocstor_41097_object13802/head//24 by client.2174595.0:107043 0.000000 2014-08-13 18:06:06.811556 7f5c973a57a0 20 read_log 194215'14353 (193305'14083) clean 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by osd.343.0:2733 2014-08-12 15:44:16.016562 2014-08-13 18:06:06.811569 7f5c973a57a0 20 read_log 194215'14354 (194215'14353) delete 841eaa06/benchmark_data_storage0111-ib.pocstor_62621_object25415/head//24 by client.2174595.0:107045 0.000000 2014-08-13 18:06:06.811587 7f5c973a57a0 20 read_log 194215'14355 (193305'14077) clean 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by osd.343.0:2735 2014-08-12 15:44:16.023514 2014-08-13 18:06:06.811601 7f5c973a57a0 20 read_log 194215'14356 (194215'14355) delete 44d1ba06/benchmark_data_storage0111-ib.pocstor_62621_object17531/head//24 by client.2174595.0:107047 0.000000 2014-08-13 18:06:06.811612 7f5c973a57a0 20 read_log 194215'14357 (192834'13614) clean d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by osd.343.0:2737 2014-08-12 15:44:16.029799 2014-08-13 18:06:06.811624 7f5c973a57a0 20 read_log 194215'14358 (194215'14357) delete d78cba06/benchmark_data_storage0111-ib.pocstor_41526_object21687/head//24 by client.2174595.0:107049 0.000000 2014-08-13 18:06:06.811644 7f5c973a57a0 20 read_log 194215'14359 (192834'12948) clean ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by osd.343.0:2739 2014-08-12 15:44:16.036941 2014-08-13 18:06:06.811665 7f5c973a57a0 20 read_log 194215'14360 (194215'14359) delete ba3dba06/benchmark_data_storage0111-ib.pocstor_26659_object6349/head//24 by client.2174595.0:107051 0.000000 2014-08-13 18:06:06.811680 7f5c973a57a0 20 read_log 194215'14361 (193305'14080) clean 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by osd.343.0:2741 2014-08-12 15:44:16.043568 2014-08-13 18:06:06.811693 7f5c973a57a0 20 read_log 194215'14362 (194215'14361) delete 9dddba06/benchmark_data_storage0111-ib.pocstor_62621_object19774/head//24 by client.2174595.0:107053 0.000000 2014-08-13 18:06:06.811707 7f5c973a57a0 20 read_log 194215'14363 (192834'13514) clean 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by osd.343.0:2743 2014-08-12 15:44:16.050630 2014-08-13 18:06:06.811723 7f5c973a57a0 20 read_log 194215'14364 (194215'14363) delete 849eba06/benchmark_data_storage0111-ib.pocstor_41526_object9771/head//24 by client.2174595.0:107055 0.000000 2014-08-13 18:06:06.811734 7f5c973a57a0 20 read_log 194215'14365 (192888'13949) clean 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by osd.343.0:2745 2014-08-12 15:44:16.058976 2014-08-13 18:06:06.811747 7f5c973a57a0 20 read_log 194215'14366 (194215'14365) delete 4a63ca06/benchmark_data_storage0111-ib.pocstor_2228_object6424/head//24 by client.2174595.0:107057 0.000000 2014-08-13 18:06:06.811771 7f5c973a57a0 20 read_log 194215'14367 (192834'12973) clean f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by osd.343.0:2747 2014-08-12 15:44:16.066787 2014-08-13 18:06:06.811823 7f5c973a57a0 20 read_log 194215'14368 (194215'14367) delete f80fca06/benchmark_data_storage0111-ib.pocstor_26659_object9281/head//24 by client.2174595.0:107059 0.000000 2014-08-13 18:06:06.811835 7f5c973a57a0 20 read_log 194215'14369 (192888'13808) clean bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by osd.343.0:2749 2014-08-12 15:44:16.074276 2014-08-13 18:06:06.811847 7f5c973a57a0 20 read_log 194215'14370 (194215'14369) delete bebbda06/benchmark_data_storage0111-ib.pocstor_41097_object28155/head//24 by client.2174595.0:107061 0.000000 2014-08-13 18:06:06.811861 7f5c973a57a0 20 read_log 194215'14371 (192834'13611) clean 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by osd.343.0:2751 2014-08-12 15:44:16.081782 2014-08-13 18:06:06.811885 7f5c973a57a0 20 read_log 194215'14372 (194215'14371) delete 5a13ea06/benchmark_data_storage0111-ib.pocstor_41526_object21453/head//24 by client.2174595.0:107063 0.000000 2014-08-13 18:06:06.811897 7f5c973a57a0 20 read_log 194215'14373 (192888'13776) clean ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by osd.343.0:2753 2014-08-12 15:44:16.088887 2014-08-13 18:06:06.811917 7f5c973a57a0 20 read_log 194215'14374 (194215'14373) delete ff73ea06/benchmark_data_storage0111-ib.pocstor_41097_object13124/head//24 by client.2174595.0:107065 0.000000 2014-08-13 18:06:06.811932 7f5c973a57a0 20 read_log 194215'14375 (193305'14090) clean ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by osd.343.0:2755 2014-08-12 15:44:16.096186 2014-08-13 18:06:06.811945 7f5c973a57a0 20 read_log 194215'14376 (194215'14375) delete ee7bea06/benchmark_data_storage0111-ib.pocstor_62621_object27909/head//24 by client.2174595.0:107067 0.000000 2014-08-13 18:06:06.811956 7f5c973a57a0 20 read_log 194215'14377 (192888'13974) clean c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by osd.343.0:2757 2014-08-12 15:44:16.104169 2014-08-13 18:06:06.811977 7f5c973a57a0 20 read_log 194215'14378 (194215'14377) delete c56fea06/benchmark_data_storage0111-ib.pocstor_2228_object17193/head//24 by client.2174595.0:107069 0.000000 2014-08-13 18:06:06.811989 7f5c973a57a0 20 read_log 194215'14379 (192888'13804) clean 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by osd.343.0:2759 2014-08-12 15:44:16.111274 2014-08-13 18:06:06.812001 7f5c973a57a0 20 read_log 194215'14380 (194215'14379) delete 223dfa06/benchmark_data_storage0111-ib.pocstor_41097_object27791/head//24 by client.2174595.0:107071 0.000000 2014-08-13 18:06:06.812023 7f5c973a57a0 20 read_log 194215'14381 (193305'14068) clean 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by osd.343.0:2761 2014-08-12 15:44:16.118314 2014-08-13 18:06:06.812051 7f5c973a57a0 20 read_log 194215'14382 (194215'14381) delete 3b950e06/benchmark_data_storage0111-ib.pocstor_62621_object7534/head//24 by client.2174595.0:107073 0.000000 2014-08-13 18:06:06.812065 7f5c973a57a0 20 read_log 194215'14383 (192888'13994) clean c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by osd.343.0:2763 2014-08-12 15:44:16.125542 2014-08-13 18:06:06.812077 7f5c973a57a0 20 read_log 194215'14384 (194215'14383) delete c2c50e06/benchmark_data_storage0111-ib.pocstor_2228_object26656/head//24 by client.2174595.0:107075 0.000000 2014-08-13 18:06:06.812092 7f5c973a57a0 20 read_log 194215'14385 (193305'14089) clean 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by osd.343.0:2765 2014-08-12 15:44:16.133203 2014-08-13 18:06:06.812107 7f5c973a57a0 20 read_log 194215'14386 (194215'14385) delete 74960e06/benchmark_data_storage0111-ib.pocstor_62621_object27723/head//24 by client.2174595.0:107077 0.000000 2014-08-13 18:06:06.812127 7f5c973a57a0 20 read_log 194215'14387 (192888'13758) clean a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by osd.343.0:2767 2014-08-12 15:44:16.140443 2014-08-13 18:06:06.812140 7f5c973a57a0 20 read_log 194215'14388 (194215'14387) delete a16f0e06/benchmark_data_storage0111-ib.pocstor_41097_object2917/head//24 by client.2174595.0:107079 0.000000 2014-08-13 18:06:06.812157 7f5c973a57a0 20 read_log 194215'14389 (192834'12910) clean fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by osd.343.0:2769 2014-08-12 15:44:16.147307 2014-08-13 18:06:06.812170 7f5c973a57a0 20 read_log 194215'14390 (194215'14389) delete fc231e06/benchmark_data_storage0111-ib.pocstor_26659_object880/head//24 by client.2174595.0:107081 0.000000 2014-08-13 18:06:06.812188 7f5c973a57a0 20 read_log 194215'14391 (194215'14391) modify 206/hit_set_24.206_archive_2014-08-12 15:44:15.483446_2014-08-12 17:24:44.732489/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.733026 2014-08-13 18:06:06.812206 7f5c973a57a0 20 read_log 194215'14392 (194215'14204) delete 206/hit_set_24.206_archive_2014-08-12 15:32:25.739433_2014-08-12 15:44:15.482903/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:44.733026 2014-08-13 18:06:06.812219 7f5c973a57a0 20 read_log 194215'14393 (0'0) promote cdac7606/benchmark_data_storage0111-ib.pocstor_34628_object1236/head//24 by osd.343.0:3904 2014-08-12 17:24:44.737953 2014-08-13 18:06:06.812235 7f5c973a57a0 20 read_log 194215'14394 (194215'14393) modify cdac7606/benchmark_data_storage0111-ib.pocstor_34628_object1236/head//24 by client.2193759.0:1237 2014-08-12 17:24:44.725615 2014-08-13 18:06:06.812258 7f5c973a57a0 20 read_log 194370'14395 (0'0) promote 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by osd.343.0:3933 2014-08-12 20:30:15.905568 2014-08-13 18:06:06.812302 7f5c973a57a0 20 read_log 194370'14396 (194370'14395) modify 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by client.1508531.0:402 2014-08-12 20:30:15.898415 2014-08-13 18:06:06.812319 7f5c973a57a0 20 read_log 194370'14397 (0'0) promote 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by osd.343.0:3941 2014-08-12 20:30:18.294433 2014-08-13 18:06:06.812331 7f5c973a57a0 20 read_log 194370'14398 (194370'14397) modify 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by client.1508531.0:1047 2014-08-12 20:30:18.286872 2014-08-13 18:06:06.812346 7f5c973a57a0 20 read_log 194370'14399 (0'0) promote 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by osd.343.0:3947 2014-08-12 20:30:20.242315 2014-08-13 18:06:06.812359 7f5c973a57a0 20 read_log 194370'14400 (194370'14399) modify 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by client.1508531.0:1561 2014-08-12 20:30:20.233949 2014-08-13 18:06:06.812371 7f5c973a57a0 20 read_log 194370'14401 (0'0) promote 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by osd.343.0:3957 2014-08-12 20:30:24.174933 2014-08-13 18:06:06.812395 7f5c973a57a0 20 read_log 194370'14402 (194370'14401) modify 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by client.1508531.0:2622 2014-08-12 20:30:24.166720 2014-08-13 18:06:06.812408 7f5c973a57a0 20 read_log 194370'14403 (0'0) promote d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by osd.343.0:3995 2014-08-12 20:30:28.634648 2014-08-13 18:06:06.812420 7f5c973a57a0 20 read_log 194370'14404 (194370'14403) modify d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by client.1508531.0:3819 2014-08-12 20:30:28.626869 2014-08-13 18:06:06.812443 7f5c973a57a0 20 read_log 194370'14405 (0'0) promote eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by osd.343.0:4001 2014-08-12 20:30:30.242898 2014-08-13 18:06:06.812463 7f5c973a57a0 20 read_log 194370'14406 (194370'14405) modify eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by client.1508531.0:4240 2014-08-12 20:30:30.235018 2014-08-13 18:06:06.812478 7f5c973a57a0 20 read_log 194370'14407 (0'0) promote a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by osd.343.0:4047 2014-08-12 20:30:38.825583 2014-08-13 18:06:06.812491 7f5c973a57a0 20 read_log 194370'14408 (194370'14407) modify a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by client.1508531.0:6241 2014-08-12 20:30:38.818632 2014-08-13 18:06:06.812503 7f5c973a57a0 20 read_log 194370'14409 (0'0) promote 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by osd.343.0:4055 2014-08-12 20:30:40.033704 2014-08-13 18:06:06.812515 7f5c973a57a0 20 read_log 194370'14410 (194370'14409) modify 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by client.1508531.0:6512 2014-08-12 20:30:40.024973 2014-08-13 18:06:06.812528 7f5c973a57a0 20 read_log 194370'14411 (0'0) promote 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by osd.343.0:4057 2014-08-12 20:30:40.352287 2014-08-13 18:06:06.812540 7f5c973a57a0 20 read_log 194370'14412 (194370'14411) modify 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by client.1508531.0:6579 2014-08-12 20:30:40.344182 2014-08-13 18:06:06.812552 7f5c973a57a0 20 read_log 194370'14413 (0'0) promote 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by osd.343.0:4073 2014-08-12 20:30:45.078425 2014-08-13 18:06:06.812564 7f5c973a57a0 20 read_log 194370'14414 (194370'14413) modify 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by client.1508531.0:7536 2014-08-12 20:30:45.071702 2014-08-13 18:06:06.812577 7f5c973a57a0 20 read_log 194370'14415 (0'0) promote b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by osd.343.0:4097 2014-08-12 20:30:52.051275 2014-08-13 18:06:06.812589 7f5c973a57a0 20 read_log 194370'14416 (194370'14415) modify b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by client.1508531.0:8800 2014-08-12 20:30:52.042021 2014-08-13 18:06:06.812601 7f5c973a57a0 20 read_log 194370'14417 (0'0) promote 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by osd.343.0:4101 2014-08-12 20:30:53.112165 2014-08-13 18:06:06.812613 7f5c973a57a0 20 read_log 194370'14418 (194370'14417) modify 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by client.1508531.0:8967 2014-08-12 20:30:53.102927 2014-08-13 18:06:06.812626 7f5c973a57a0 20 read_log 194370'14419 (0'0) promote 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by osd.343.0:4107 2014-08-12 20:30:53.696432 2014-08-13 18:06:06.812638 7f5c973a57a0 20 read_log 194370'14420 (194370'14419) modify 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by client.1508531.0:9061 2014-08-12 20:30:53.686310 2014-08-13 18:06:06.812650 7f5c973a57a0 20 read_log 194370'14421 (0'0) promote fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by osd.343.0:4121 2014-08-12 20:30:59.171425 2014-08-13 18:06:06.812662 7f5c973a57a0 20 read_log 194370'14422 (194370'14421) modify fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by client.1508531.0:9984 2014-08-12 20:30:59.165164 2014-08-13 18:06:06.812674 7f5c973a57a0 20 read_log 194370'14423 (0'0) promote e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by osd.343.0:4189 2014-08-12 20:31:22.729143 2014-08-13 18:06:06.812686 7f5c973a57a0 20 read_log 194370'14424 (194370'14423) modify e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by client.1508531.0:13351 2014-08-12 20:31:22.720976 2014-08-13 18:06:06.812699 7f5c973a57a0 20 read_log 194377'14425 (0'0) promote eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by osd.343.0:4231 2014-08-12 20:33:04.930562 2014-08-13 18:06:06.812718 7f5c973a57a0 20 read_log 194377'14426 (194377'14425) modify eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by client.2231190.0:1935 2014-08-12 20:33:04.923832 2014-08-13 18:06:06.812731 7f5c973a57a0 20 read_log 194377'14427 (0'0) promote 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by osd.343.0:4247 2014-08-12 20:33:07.301305 2014-08-13 18:06:06.812743 7f5c973a57a0 20 read_log 194377'14428 (194377'14427) modify 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by client.2231190.0:2514 2014-08-12 20:33:07.293079 2014-08-13 18:06:06.812758 7f5c973a57a0 20 read_log 194377'14429 (0'0) promote 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by osd.343.0:4279 2014-08-12 20:33:16.750358 2014-08-13 18:06:06.812770 7f5c973a57a0 20 read_log 194377'14430 (194377'14429) modify 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by client.2231190.0:4444 2014-08-12 20:33:16.740558 2014-08-13 18:06:06.812785 7f5c973a57a0 20 read_log 194382'14431 (0'0) promote 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by osd.343.0:4305 2014-08-12 20:33:59.332216 2014-08-13 18:06:06.812797 7f5c973a57a0 20 read_log 194382'14432 (194382'14431) modify 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by client.2231190.0:5645 2014-08-12 20:33:59.307896 2014-08-13 18:06:06.812810 7f5c973a57a0 20 read_log 194393'14433 (0'0) promote 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by osd.343.0:4340 2014-08-12 20:47:26.474881 2014-08-13 18:06:06.812822 7f5c973a57a0 20 read_log 194393'14434 (194393'14433) modify 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by client.1064095.0:124 2014-08-12 20:47:26.463639 2014-08-13 18:06:06.812834 7f5c973a57a0 20 read_log 194397'14435 (0'0) promote 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by osd.343.0:4401 2014-08-12 20:53:04.649981 2014-08-13 18:06:06.812849 7f5c973a57a0 20 read_log 194397'14436 (194397'14435) modify 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by client.2235069.0:875 2014-08-12 20:53:04.641065 2014-08-13 18:06:06.812861 7f5c973a57a0 20 read_log 194416'14437 (0'0) promote f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by osd.343.0:4571 2014-08-13 15:28:32.574102 2014-08-13 18:06:06.812876 7f5c973a57a0 20 read_log 194416'14438 (194416'14437) modify f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by client.2441958.0:530 2014-08-13 15:28:32.566858 2014-08-13 18:06:06.812889 7f5c973a57a0 20 read_log 194416'14439 (0'0) promote f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by osd.343.0:4605 2014-08-13 15:28:38.062746 2014-08-13 18:06:06.812903 7f5c973a57a0 20 read_log 194416'14440 (194416'14439) modify f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by client.2441958.0:1939 2014-08-13 15:28:38.053390 2014-08-13 18:06:06.812916 7f5c973a57a0 20 read_log 194416'14441 (0'0) promote ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by osd.343.0:4611 2014-08-13 15:28:38.398844 2014-08-13 18:06:06.812930 7f5c973a57a0 20 read_log 194416'14442 (194416'14441) modify ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by client.2441958.0:2029 2014-08-13 15:28:38.389494 2014-08-13 18:06:06.812942 7f5c973a57a0 20 read_log 194416'14443 (0'0) promote f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by osd.343.0:4655 2014-08-13 15:28:43.547333 2014-08-13 18:06:06.812957 7f5c973a57a0 20 read_log 194416'14444 (194416'14443) modify f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by client.2441958.0:3388 2014-08-13 15:28:43.533618 2014-08-13 18:06:06.812998 7f5c973a57a0 20 read_log 194416'14445 (0'0) promote 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by osd.343.0:4671 2014-08-13 15:28:46.626885 2014-08-13 18:06:06.813014 7f5c973a57a0 20 read_log 194416'14446 (194416'14445) modify 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by client.2441958.0:4170 2014-08-13 15:28:46.617969 2014-08-13 18:06:06.813027 7f5c973a57a0 20 read_log 194416'14447 (0'0) promote 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by osd.343.0:4681 2014-08-13 15:28:47.244598 2014-08-13 18:06:06.813039 7f5c973a57a0 20 read_log 194416'14448 (194416'14447) modify 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by client.2441958.0:4333 2014-08-13 15:28:47.236323 2014-08-13 18:06:06.813054 7f5c973a57a0 20 read_log 194416'14449 (0'0) promote f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by osd.343.0:4703 2014-08-13 15:28:51.259914 2014-08-13 18:06:06.813066 7f5c973a57a0 20 read_log 194416'14450 (194416'14449) modify f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by client.2441958.0:5357 2014-08-13 15:28:51.250657 2014-08-13 18:06:06.813081 7f5c973a57a0 20 read_log 194416'14451 (0'0) promote 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by osd.343.0:4781 2014-08-13 15:29:03.612007 2014-08-13 18:06:06.813093 7f5c973a57a0 20 read_log 194416'14452 (194416'14451) modify 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by client.2441958.0:8551 2014-08-13 15:29:03.602974 2014-08-13 18:06:06.813107 7f5c973a57a0 20 read_log 194416'14453 (0'0) promote 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by osd.343.0:4803 2014-08-13 15:29:09.818220 2014-08-13 18:06:06.813119 7f5c973a57a0 20 read_log 194416'14454 (194416'14453) modify 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by client.2441958.0:10101 2014-08-13 15:29:09.810333 2014-08-13 18:06:06.813131 7f5c973a57a0 20 read_log 194416'14455 (0'0) promote 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by osd.343.0:4833 2014-08-13 15:29:16.404377 2014-08-13 18:06:06.813143 7f5c973a57a0 20 read_log 194416'14456 (194416'14455) modify 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by client.2441958.0:11719 2014-08-13 15:29:16.395680 2014-08-13 18:06:06.813164 7f5c973a57a0 20 read_log 194416'14457 (0'0) promote 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by osd.343.0:4835 2014-08-13 15:29:16.454061 2014-08-13 18:06:06.813176 7f5c973a57a0 20 read_log 194416'14458 (194416'14457) modify 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by client.2441958.0:11733 2014-08-13 15:29:16.446957 2014-08-13 18:06:06.813193 7f5c973a57a0 20 read_log 194416'14459 (0'0) promote 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by osd.343.0:4883 2014-08-13 15:29:25.957169 2014-08-13 18:06:06.813208 7f5c973a57a0 20 read_log 194416'14460 (194416'14459) modify 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by client.2441958.0:13932 2014-08-13 15:29:25.945823 2014-08-13 18:06:06.813221 7f5c973a57a0 20 read_log 194416'14461 (0'0) promote d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by osd.343.0:4905 2014-08-13 15:29:29.240942 2014-08-13 18:06:06.813235 7f5c973a57a0 20 read_log 194416'14462 (194416'14461) modify d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by client.2441958.0:14761 2014-08-13 15:29:29.232650 2014-08-13 18:06:06.813249 7f5c973a57a0 20 read_log 194416'14463 (0'0) promote a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by osd.343.0:4913 2014-08-13 15:29:30.045767 2014-08-13 18:06:06.813264 7f5c973a57a0 20 read_log 194416'14464 (194416'14463) modify a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by client.2441958.0:14945 2014-08-13 15:29:30.039259 2014-08-13 18:06:06.813278 7f5c973a57a0 20 read_log 194416'14465 (0'0) promote 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by osd.343.0:4949 2014-08-13 15:29:44.726877 2014-08-13 18:06:06.813296 7f5c973a57a0 20 read_log 194416'14466 (194416'14465) modify 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by client.2441958.0:16818 2014-08-13 15:29:44.717637 2014-08-13 18:06:06.813310 7f5c973a57a0 20 read_log 194416'14467 (0'0) promote 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by osd.343.0:4961 2014-08-13 15:29:53.472737 2014-08-13 18:06:06.813323 7f5c973a57a0 20 read_log 194416'14468 (194416'14467) modify 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by client.2441958.0:17432 2014-08-13 15:29:53.463866 2014-08-13 18:06:06.813338 7f5c973a57a0 20 read_log 194416'14469 (0'0) promote 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by osd.343.0:4965 2014-08-13 15:29:54.523057 2014-08-13 18:06:06.813371 7f5c973a57a0 20 read_log 194416'14470 (194416'14469) modify 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by client.2441958.0:17505 2014-08-13 15:29:54.513049 2014-08-13 18:06:06.813388 7f5c973a57a0 20 read_log 194422'14471 (0'0) promote c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by osd.343.0:5001 2014-08-13 15:32:18.184016 2014-08-13 18:06:06.813403 7f5c973a57a0 20 read_log 194422'14472 (194422'14471) modify c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by client.1148203.0:707 2014-08-13 15:32:18.176089 2014-08-13 18:06:06.813416 7f5c973a57a0 20 read_log 194422'14473 (0'0) promote d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by osd.343.0:5035 2014-08-13 15:32:24.791144 2014-08-13 18:06:06.813444 7f5c973a57a0 20 read_log 194422'14474 (194422'14473) modify d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by client.1148203.0:2407 2014-08-13 15:32:24.783012 2014-08-13 18:06:06.813460 7f5c973a57a0 20 read_log 194422'14475 (0'0) promote ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by osd.343.0:5073 2014-08-13 15:32:29.872702 2014-08-13 18:06:06.813483 7f5c973a57a0 20 read_log 194422'14476 (194422'14475) modify ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by client.1148203.0:3701 2014-08-13 15:32:29.864799 2014-08-13 18:06:06.813503 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.813965 7f5c973a57a0 10 read_log checking for missing items over interval (194370'14395,194422'14476] 2014-08-13 18:06:06.814013 7f5c973a57a0 15 read_log missing 194422'14476 (194422'14475) modify ef487606/benchmark_data_storage0111-ib.pocstor_41231_object3700/head//24 by client.1148203.0:3701 2014-08-13 15:32:29.864799 2014-08-13 18:06:06.814053 7f5c973a57a0 15 read_log missing 194422'14474 (194422'14473) modify d6963206/benchmark_data_storage0111-ib.pocstor_41231_object2406/head//24 by client.1148203.0:2407 2014-08-13 15:32:24.783012 2014-08-13 18:06:06.814090 7f5c973a57a0 15 read_log missing 194422'14472 (194422'14471) modify c5c2206/benchmark_data_storage0111-ib.pocstor_41231_object706/head//24 by client.1148203.0:707 2014-08-13 15:32:18.176089 2014-08-13 18:06:06.814126 7f5c973a57a0 15 read_log missing 194416'14470 (194416'14469) modify 94e9f206/benchmark_data_storage0111-ib.pocstor_19103_object17504/head//24 by client.2441958.0:17505 2014-08-13 15:29:54.513049 2014-08-13 18:06:06.814168 7f5c973a57a0 15 read_log missing 194416'14468 (194416'14467) modify 2c3be206/benchmark_data_storage0111-ib.pocstor_19103_object17431/head//24 by client.2441958.0:17432 2014-08-13 15:29:53.463866 2014-08-13 18:06:06.814209 7f5c973a57a0 15 read_log missing 194416'14466 (194416'14465) modify 6246ea06/benchmark_data_storage0111-ib.pocstor_19103_object16817/head//24 by client.2441958.0:16818 2014-08-13 15:29:44.717637 2014-08-13 18:06:06.814246 7f5c973a57a0 15 read_log missing 194416'14464 (194416'14463) modify a323b206/benchmark_data_storage0111-ib.pocstor_19103_object14944/head//24 by client.2441958.0:14945 2014-08-13 15:29:30.039259 2014-08-13 18:06:06.814286 7f5c973a57a0 15 read_log missing 194416'14462 (194416'14461) modify d9a9ba06/benchmark_data_storage0111-ib.pocstor_19103_object14760/head//24 by client.2441958.0:14761 2014-08-13 15:29:29.232650 2014-08-13 18:06:06.814323 7f5c973a57a0 15 read_log missing 194416'14460 (194416'14459) modify 1311ae06/benchmark_data_storage0111-ib.pocstor_19103_object13931/head//24 by client.2441958.0:13932 2014-08-13 15:29:25.945823 2014-08-13 18:06:06.814359 7f5c973a57a0 15 read_log missing 194416'14458 (194416'14457) modify 709e7606/benchmark_data_storage0111-ib.pocstor_19103_object11732/head//24 by client.2441958.0:11733 2014-08-13 15:29:16.446957 2014-08-13 18:06:06.814395 7f5c973a57a0 15 read_log missing 194416'14456 (194416'14455) modify 7e7e5206/benchmark_data_storage0111-ib.pocstor_19103_object11718/head//24 by client.2441958.0:11719 2014-08-13 15:29:16.395680 2014-08-13 18:06:06.814430 7f5c973a57a0 15 read_log missing 194416'14454 (194416'14453) modify 31b5c606/benchmark_data_storage0111-ib.pocstor_19103_object10100/head//24 by client.2441958.0:10101 2014-08-13 15:29:09.810333 2014-08-13 18:06:06.814466 7f5c973a57a0 15 read_log missing 194416'14452 (194416'14451) modify 8404a606/benchmark_data_storage0111-ib.pocstor_19103_object8550/head//24 by client.2441958.0:8551 2014-08-13 15:29:03.602974 2014-08-13 18:06:06.814501 7f5c973a57a0 15 read_log missing 194416'14450 (194416'14449) modify f6647e06/benchmark_data_storage0111-ib.pocstor_19103_object5356/head//24 by client.2441958.0:5357 2014-08-13 15:28:51.250657 2014-08-13 18:06:06.814537 7f5c973a57a0 15 read_log missing 194416'14448 (194416'14447) modify 8d98a06/benchmark_data_storage0111-ib.pocstor_19103_object4332/head//24 by client.2441958.0:4333 2014-08-13 15:28:47.236323 2014-08-13 18:06:06.814573 7f5c973a57a0 15 read_log missing 194416'14446 (194416'14445) modify 9b65a06/benchmark_data_storage0111-ib.pocstor_19103_object4169/head//24 by client.2441958.0:4170 2014-08-13 15:28:46.617969 2014-08-13 18:06:06.814608 7f5c973a57a0 15 read_log missing 194416'14444 (194416'14443) modify f27f606/benchmark_data_storage0111-ib.pocstor_19103_object3387/head//24 by client.2441958.0:3388 2014-08-13 15:28:43.533618 2014-08-13 18:06:06.814644 7f5c973a57a0 15 read_log missing 194416'14442 (194416'14441) modify ac310a06/benchmark_data_storage0111-ib.pocstor_19103_object2028/head//24 by client.2441958.0:2029 2014-08-13 15:28:38.389494 2014-08-13 18:06:06.814680 7f5c973a57a0 15 read_log missing 194416'14440 (194416'14439) modify f3495a06/benchmark_data_storage0111-ib.pocstor_19103_object1938/head//24 by client.2441958.0:1939 2014-08-13 15:28:38.053390 2014-08-13 18:06:06.814716 7f5c973a57a0 15 read_log missing 194416'14438 (194416'14437) modify f3b0f206/benchmark_data_storage0111-ib.pocstor_19103_object529/head//24 by client.2441958.0:530 2014-08-13 15:28:32.566858 2014-08-13 18:06:06.814752 7f5c973a57a0 15 read_log missing 194397'14436 (194397'14435) modify 8d0c1206/benchmark_data_storage0101-ib.pocstor_6819_object874/head//24 by client.2235069.0:875 2014-08-12 20:53:04.641065 2014-08-13 18:06:06.814787 7f5c973a57a0 15 read_log missing 194393'14434 (194393'14433) modify 55344a06/benchmark_data_storage0111-ib.pocstor_24170_object123/head//24 by client.1064095.0:124 2014-08-12 20:47:26.463639 2014-08-13 18:06:06.814823 7f5c973a57a0 15 read_log missing 194382'14432 (194382'14431) modify 88a81e06/benchmark_data_storage0111-ib.pocstor_25065_object5644/head//24 by client.2231190.0:5645 2014-08-12 20:33:59.307896 2014-08-13 18:06:06.814859 7f5c973a57a0 15 read_log missing 194377'14430 (194377'14429) modify 9fd77606/benchmark_data_storage0111-ib.pocstor_25065_object4443/head//24 by client.2231190.0:4444 2014-08-12 20:33:16.740558 2014-08-13 18:06:06.814894 7f5c973a57a0 15 read_log missing 194377'14428 (194377'14427) modify 21cfca06/benchmark_data_storage0111-ib.pocstor_25065_object2513/head//24 by client.2231190.0:2514 2014-08-12 20:33:07.293079 2014-08-13 18:06:06.814930 7f5c973a57a0 15 read_log missing 194377'14426 (194377'14425) modify eb780606/benchmark_data_storage0111-ib.pocstor_25065_object1934/head//24 by client.2231190.0:1935 2014-08-12 20:33:04.923832 2014-08-13 18:06:06.814965 7f5c973a57a0 15 read_log missing 194370'14424 (194370'14423) modify e09fda06/benchmark_data_storage0111-ib.pocstor_12495_object13350/head//24 by client.1508531.0:13351 2014-08-12 20:31:22.720976 2014-08-13 18:06:06.815001 7f5c973a57a0 15 read_log missing 194370'14422 (194370'14421) modify fb630a06/benchmark_data_storage0111-ib.pocstor_12495_object9983/head//24 by client.1508531.0:9984 2014-08-12 20:30:59.165164 2014-08-13 18:06:06.815036 7f5c973a57a0 15 read_log missing 194370'14420 (194370'14419) modify 97d56206/benchmark_data_storage0111-ib.pocstor_12495_object9060/head//24 by client.1508531.0:9061 2014-08-12 20:30:53.686310 2014-08-13 18:06:06.815072 7f5c973a57a0 15 read_log missing 194370'14418 (194370'14417) modify 64fbc206/benchmark_data_storage0111-ib.pocstor_12495_object8966/head//24 by client.1508531.0:8967 2014-08-12 20:30:53.102927 2014-08-13 18:06:06.815108 7f5c973a57a0 15 read_log missing 194370'14416 (194370'14415) modify b180b606/benchmark_data_storage0111-ib.pocstor_12495_object8799/head//24 by client.1508531.0:8800 2014-08-12 20:30:52.042021 2014-08-13 18:06:06.815143 7f5c973a57a0 15 read_log missing 194370'14414 (194370'14413) modify 1ae7be06/benchmark_data_storage0111-ib.pocstor_12495_object7535/head//24 by client.1508531.0:7536 2014-08-12 20:30:45.071702 2014-08-13 18:06:06.815185 7f5c973a57a0 15 read_log missing 194370'14412 (194370'14411) modify 6198a206/benchmark_data_storage0111-ib.pocstor_12495_object6578/head//24 by client.1508531.0:6579 2014-08-12 20:30:40.344182 2014-08-13 18:06:06.815259 7f5c973a57a0 15 read_log missing 194370'14410 (194370'14409) modify 7a544a06/benchmark_data_storage0111-ib.pocstor_12495_object6511/head//24 by client.1508531.0:6512 2014-08-12 20:30:40.024973 2014-08-13 18:06:06.815301 7f5c973a57a0 15 read_log missing 194370'14408 (194370'14407) modify a9df0206/benchmark_data_storage0111-ib.pocstor_12495_object6240/head//24 by client.1508531.0:6241 2014-08-12 20:30:38.818632 2014-08-13 18:06:06.815338 7f5c973a57a0 15 read_log missing 194370'14406 (194370'14405) modify eb41f206/benchmark_data_storage0111-ib.pocstor_12495_object4239/head//24 by client.1508531.0:4240 2014-08-12 20:30:30.235018 2014-08-13 18:06:06.815390 7f5c973a57a0 15 read_log missing 194370'14404 (194370'14403) modify d6c89206/benchmark_data_storage0111-ib.pocstor_12495_object3818/head//24 by client.1508531.0:3819 2014-08-12 20:30:28.626869 2014-08-13 18:06:06.815441 7f5c973a57a0 15 read_log missing 194370'14402 (194370'14401) modify 9cd90206/benchmark_data_storage0111-ib.pocstor_12495_object2621/head//24 by client.1508531.0:2622 2014-08-12 20:30:24.166720 2014-08-13 18:06:06.815487 7f5c973a57a0 15 read_log missing 194370'14400 (194370'14399) modify 7c356e06/benchmark_data_storage0111-ib.pocstor_12495_object1560/head//24 by client.1508531.0:1561 2014-08-12 20:30:20.233949 2014-08-13 18:06:06.815526 7f5c973a57a0 15 read_log missing 194370'14398 (194370'14397) modify 7d981206/benchmark_data_storage0111-ib.pocstor_12495_object1046/head//24 by client.1508531.0:1047 2014-08-12 20:30:18.286872 2014-08-13 18:06:06.815562 7f5c973a57a0 15 read_log missing 194370'14396 (194370'14395) modify 2302a206/benchmark_data_storage0111-ib.pocstor_12495_object401/head//24 by client.1508531.0:402 2014-08-12 20:30:15.898415 2014-08-13 18:06:06.815573 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.815599 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=0 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive m=41] handle_loaded 2014-08-13 18:06:06.815617 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=0 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Initial 0.015157 0 0.000000 2014-08-13 18:06:06.815629 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=0 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Reset 2014-08-13 18:06:06.815638 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=0 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194487 2014-08-13 18:06:06.815648 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log((192718'11573,194422'14476], crt=194422'14476) 2014-08-13 18:06:06.815658 7f5c973a57a0 10 osd.4 194487 pgid 24.20e coll 24.20e_head 2014-08-13 18:06:06.815793 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.20e 2014-08-13 18:06:06.815806 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.815839 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.20e(unlocked)] enter Initial 2014-08-13 18:06:06.815851 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.20e(unlocked)] enter NotTrimming 2014-08-13 18:06:06.816033 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.816176 7f5c973a57a0 20 read_log 192718'8902 (0'0) promote 1a453a0e/core.19866+13+2/head//24 by osd.367.0:40779 2014-08-06 15:33:39.061719 2014-08-13 18:06:06.816198 7f5c973a57a0 20 read_log 192718'8903 (192718'8902) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:1 2014-08-06 15:33:27.695325 2014-08-13 18:06:06.816212 7f5c973a57a0 20 read_log 192718'8904 (192718'8903) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:1 2014-08-06 15:33:27.688105 2014-08-13 18:06:06.816226 7f5c973a57a0 20 read_log 192718'8905 (192718'8904) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:2 2014-08-06 15:33:40.521781 2014-08-13 18:06:06.816239 7f5c973a57a0 20 read_log 192718'8906 (192718'8905) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:2 2014-08-06 15:33:40.522254 2014-08-13 18:06:06.816257 7f5c973a57a0 20 read_log 192718'8907 (192718'8906) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:3 2014-08-06 15:33:40.691386 2014-08-13 18:06:06.816270 7f5c973a57a0 20 read_log 192718'8908 (192718'8907) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:3 2014-08-06 15:33:40.693838 2014-08-13 18:06:06.816286 7f5c973a57a0 20 read_log 192718'8909 (192718'8908) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:4 2014-08-06 15:33:40.741564 2014-08-13 18:06:06.816301 7f5c973a57a0 20 read_log 192718'8910 (192718'8909) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:4 2014-08-06 15:33:40.773085 2014-08-13 18:06:06.816314 7f5c973a57a0 20 read_log 192718'8911 (192718'8910) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:5 2014-08-06 15:33:40.813135 2014-08-13 18:06:06.816326 7f5c973a57a0 20 read_log 192718'8912 (192718'8911) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:5 2014-08-06 15:33:40.843554 2014-08-13 18:06:06.816338 7f5c973a57a0 20 read_log 192718'8913 (192718'8912) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:6 2014-08-06 15:33:40.882714 2014-08-13 18:06:06.816351 7f5c973a57a0 20 read_log 192718'8914 (192718'8913) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:6 2014-08-06 15:33:40.913481 2014-08-13 18:06:06.816363 7f5c973a57a0 20 read_log 192718'8915 (192718'8914) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:7 2014-08-06 15:33:40.951275 2014-08-13 18:06:06.816375 7f5c973a57a0 20 read_log 192718'8916 (192718'8915) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:7 2014-08-06 15:33:40.982908 2014-08-13 18:06:06.816388 7f5c973a57a0 20 read_log 192718'8917 (192718'8916) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:8 2014-08-06 15:33:41.020753 2014-08-13 18:06:06.816400 7f5c973a57a0 20 read_log 192718'8918 (192718'8917) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:8 2014-08-06 15:33:41.052491 2014-08-13 18:06:06.816440 7f5c973a57a0 20 read_log 192718'8919 (192718'8918) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:9 2014-08-06 15:33:41.090937 2014-08-13 18:06:06.816456 7f5c973a57a0 20 read_log 192718'8920 (192718'8919) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:9 2014-08-06 15:33:42.178893 2014-08-13 18:06:06.816469 7f5c973a57a0 20 read_log 192718'8921 (192718'8920) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:10 2014-08-06 15:33:42.183573 2014-08-13 18:06:06.816481 7f5c973a57a0 20 read_log 192718'8922 (192718'8921) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:10 2014-08-06 15:33:42.210413 2014-08-13 18:06:06.816493 7f5c973a57a0 20 read_log 192718'8923 (192718'8922) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:11 2014-08-06 15:33:42.215797 2014-08-13 18:06:06.816506 7f5c973a57a0 20 read_log 192718'8924 (192718'8923) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:11 2014-08-06 15:33:42.237654 2014-08-13 18:06:06.816518 7f5c973a57a0 20 read_log 192718'8925 (192718'8924) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:12 2014-08-06 15:33:42.242515 2014-08-13 18:06:06.816530 7f5c973a57a0 20 read_log 192718'8926 (192718'8925) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:12 2014-08-06 15:33:42.266075 2014-08-13 18:06:06.816542 7f5c973a57a0 20 read_log 192718'8927 (192718'8926) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:13 2014-08-06 15:33:42.270236 2014-08-13 18:06:06.816556 7f5c973a57a0 20 read_log 192718'8928 (192718'8927) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:13 2014-08-06 15:33:42.288280 2014-08-13 18:06:06.816568 7f5c973a57a0 20 read_log 192718'8929 (192718'8928) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:14 2014-08-06 15:33:42.295655 2014-08-13 18:06:06.816580 7f5c973a57a0 20 read_log 192718'8930 (192718'8929) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:14 2014-08-06 15:33:42.315139 2014-08-13 18:06:06.816592 7f5c973a57a0 20 read_log 192718'8931 (192718'8930) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:15 2014-08-06 15:33:42.321287 2014-08-13 18:06:06.816604 7f5c973a57a0 20 read_log 192718'8932 (192718'8931) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:15 2014-08-06 15:33:42.340571 2014-08-13 18:06:06.816617 7f5c973a57a0 20 read_log 192718'8933 (192718'8932) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:16 2014-08-06 15:33:42.348789 2014-08-13 18:06:06.816632 7f5c973a57a0 20 read_log 192718'8934 (192718'8933) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:17 2014-08-06 15:33:42.375567 2014-08-13 18:06:06.816645 7f5c973a57a0 20 read_log 192718'8935 (192718'8934) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:16 2014-08-06 15:33:42.385460 2014-08-13 18:06:06.816657 7f5c973a57a0 20 read_log 192718'8936 (192718'8935) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:18 2014-08-06 15:33:42.400075 2014-08-13 18:06:06.816669 7f5c973a57a0 20 read_log 192718'8937 (192718'8936) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:17 2014-08-06 15:33:42.408147 2014-08-13 18:06:06.816685 7f5c973a57a0 20 read_log 192718'8938 (192718'8937) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:19 2014-08-06 15:33:42.425381 2014-08-13 18:06:06.816697 7f5c973a57a0 20 read_log 192718'8939 (192718'8938) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:18 2014-08-06 15:33:42.430404 2014-08-13 18:06:06.816712 7f5c973a57a0 20 read_log 192718'8940 (192718'8939) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:20 2014-08-06 15:33:42.454455 2014-08-13 18:06:06.816724 7f5c973a57a0 20 read_log 192718'8941 (192718'8940) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:19 2014-08-06 15:33:42.486319 2014-08-13 18:06:06.816738 7f5c973a57a0 20 read_log 192718'8942 (192718'8941) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:21 2014-08-06 15:33:42.523677 2014-08-13 18:06:06.816757 7f5c973a57a0 20 read_log 192718'8943 (192718'8942) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:20 2014-08-06 15:33:42.556224 2014-08-13 18:06:06.816772 7f5c973a57a0 20 read_log 192718'8944 (192718'8943) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:22 2014-08-06 15:33:42.593423 2014-08-13 18:06:06.816784 7f5c973a57a0 20 read_log 192718'8945 (192718'8944) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:21 2014-08-06 15:33:42.625831 2014-08-13 18:06:06.816799 7f5c973a57a0 20 read_log 192718'8946 (192718'8945) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:23 2014-08-06 15:33:42.870238 2014-08-13 18:06:06.816812 7f5c973a57a0 20 read_log 192718'8947 (192718'8946) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:22 2014-08-06 15:33:42.908163 2014-08-13 18:06:06.816826 7f5c973a57a0 20 read_log 192718'8948 (192718'8947) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:24 2014-08-06 15:33:42.944674 2014-08-13 18:06:06.816838 7f5c973a57a0 20 read_log 192718'8949 (192718'8948) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:23 2014-08-06 15:33:42.977335 2014-08-13 18:06:06.816852 7f5c973a57a0 20 read_log 192718'8950 (192718'8949) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:25 2014-08-06 15:33:43.013856 2014-08-13 18:06:06.816865 7f5c973a57a0 20 read_log 192718'8951 (192718'8950) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:24 2014-08-06 15:33:43.046956 2014-08-13 18:06:06.816877 7f5c973a57a0 20 read_log 192718'8952 (192718'8951) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:26 2014-08-06 15:33:43.083178 2014-08-13 18:06:06.816892 7f5c973a57a0 20 read_log 192718'8953 (192718'8952) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:25 2014-08-06 15:33:43.116748 2014-08-13 18:06:06.816906 7f5c973a57a0 20 read_log 192718'8954 (192718'8953) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:27 2014-08-06 15:33:43.152991 2014-08-13 18:06:06.816919 7f5c973a57a0 20 read_log 192718'8955 (192718'8954) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:26 2014-08-06 15:33:43.187284 2014-08-13 18:06:06.816933 7f5c973a57a0 20 read_log 192718'8956 (192718'8955) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:28 2014-08-06 15:33:43.401362 2014-08-13 18:06:06.816948 7f5c973a57a0 20 read_log 192718'8957 (192718'8956) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:27 2014-08-06 15:33:43.434874 2014-08-13 18:06:06.816960 7f5c973a57a0 20 read_log 192718'8958 (192718'8957) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:29 2014-08-06 15:33:43.468718 2014-08-13 18:06:06.816974 7f5c973a57a0 20 read_log 192718'8959 (192718'8958) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:28 2014-08-06 15:33:43.501726 2014-08-13 18:06:06.816987 7f5c973a57a0 20 read_log 192718'8960 (192718'8959) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:30 2014-08-06 15:33:43.534999 2014-08-13 18:06:06.816999 7f5c973a57a0 20 read_log 192718'8961 (192718'8960) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:29 2014-08-06 15:33:43.567675 2014-08-13 18:06:06.817013 7f5c973a57a0 20 read_log 192718'8962 (192718'8961) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:31 2014-08-06 15:33:43.602488 2014-08-13 18:06:06.817025 7f5c973a57a0 20 read_log 192718'8963 (192718'8962) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:30 2014-08-06 15:33:43.634610 2014-08-13 18:06:06.817038 7f5c973a57a0 20 read_log 192718'8964 (192718'8963) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:32 2014-08-06 15:33:43.669976 2014-08-13 18:06:06.817053 7f5c973a57a0 20 read_log 192718'8965 (192718'8964) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:31 2014-08-06 15:33:43.701629 2014-08-13 18:06:06.817065 7f5c973a57a0 20 read_log 192718'8966 (192718'8965) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:33 2014-08-06 15:33:43.902819 2014-08-13 18:06:06.817084 7f5c973a57a0 20 read_log 192718'8967 (192718'8966) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:32 2014-08-06 15:33:43.940740 2014-08-13 18:06:06.817098 7f5c973a57a0 20 read_log 192718'8968 (192718'8967) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:34 2014-08-06 15:33:43.973932 2014-08-13 18:06:06.817110 7f5c973a57a0 20 read_log 192718'8969 (192718'8968) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:33 2014-08-06 15:33:44.007865 2014-08-13 18:06:06.817125 7f5c973a57a0 20 read_log 192718'8970 (192718'8969) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:35 2014-08-06 15:33:44.043451 2014-08-13 18:06:06.817139 7f5c973a57a0 20 read_log 192718'8971 (192718'8970) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:34 2014-08-06 15:33:44.076726 2014-08-13 18:06:06.817164 7f5c973a57a0 20 read_log 192718'8972 (192718'8971) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:36 2014-08-06 15:33:44.109684 2014-08-13 18:06:06.817177 7f5c973a57a0 20 read_log 192718'8973 (192718'8972) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:35 2014-08-06 15:33:44.142168 2014-08-13 18:06:06.817189 7f5c973a57a0 20 read_log 192718'8974 (192718'8973) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:37 2014-08-06 15:33:44.178451 2014-08-13 18:06:06.817202 7f5c973a57a0 20 read_log 192718'8975 (192718'8974) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:36 2014-08-06 15:33:44.208912 2014-08-13 18:06:06.817214 7f5c973a57a0 20 read_log 192718'8976 (192718'8975) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:38 2014-08-06 15:33:44.413554 2014-08-13 18:06:06.817226 7f5c973a57a0 20 read_log 192718'8977 (192718'8976) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:37 2014-08-06 15:33:44.448297 2014-08-13 18:06:06.817238 7f5c973a57a0 20 read_log 192718'8978 (192718'8977) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:39 2014-08-06 15:33:44.485195 2014-08-13 18:06:06.817250 7f5c973a57a0 20 read_log 192718'8979 (192718'8978) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:38 2014-08-06 15:33:44.512953 2014-08-13 18:06:06.817262 7f5c973a57a0 20 read_log 192718'8980 (192718'8979) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:40 2014-08-06 15:33:44.551863 2014-08-13 18:06:06.817274 7f5c973a57a0 20 read_log 192718'8981 (192718'8980) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:39 2014-08-06 15:33:44.582111 2014-08-13 18:06:06.817288 7f5c973a57a0 20 read_log 192718'8982 (192718'8981) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:41 2014-08-06 15:33:44.629915 2014-08-13 18:06:06.817301 7f5c973a57a0 20 read_log 192718'8983 (192718'8982) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:40 2014-08-06 15:33:44.648730 2014-08-13 18:06:06.817318 7f5c973a57a0 20 read_log 192718'8984 (192718'8983) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:42 2014-08-06 15:33:44.685656 2014-08-13 18:06:06.817350 7f5c973a57a0 20 read_log 192718'8985 (192718'8984) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:41 2014-08-06 15:33:44.716541 2014-08-13 18:06:06.817370 7f5c973a57a0 20 read_log 192718'8986 (192718'8985) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:43 2014-08-06 15:33:44.921162 2014-08-13 18:06:06.817382 7f5c973a57a0 20 read_log 192718'8987 (192718'8986) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:42 2014-08-06 15:33:44.951653 2014-08-13 18:06:06.817394 7f5c973a57a0 20 read_log 192718'8988 (192718'8987) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:44 2014-08-06 15:33:44.989866 2014-08-13 18:06:06.817407 7f5c973a57a0 20 read_log 192718'8989 (192718'8988) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:43 2014-08-06 15:33:45.020128 2014-08-13 18:06:06.817419 7f5c973a57a0 20 read_log 192718'8990 (192718'8989) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:45 2014-08-06 15:33:45.054932 2014-08-13 18:06:06.817438 7f5c973a57a0 20 read_log 192718'8991 (192718'8990) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:44 2014-08-06 15:33:45.090444 2014-08-13 18:06:06.817450 7f5c973a57a0 20 read_log 192718'8992 (192718'8991) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:46 2014-08-06 15:33:45.132712 2014-08-13 18:06:06.817462 7f5c973a57a0 20 read_log 192718'8993 (192718'8992) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:45 2014-08-06 15:33:45.165385 2014-08-13 18:06:06.817474 7f5c973a57a0 20 read_log 192718'8994 (192718'8993) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:47 2014-08-06 15:33:45.205637 2014-08-13 18:06:06.817486 7f5c973a57a0 20 read_log 192718'8995 (192718'8994) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:46 2014-08-06 15:33:45.240480 2014-08-13 18:06:06.817499 7f5c973a57a0 20 read_log 192718'8996 (192718'8995) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:48 2014-08-06 15:33:45.468574 2014-08-13 18:06:06.817511 7f5c973a57a0 20 read_log 192718'8997 (192718'8996) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:47 2014-08-06 15:33:45.509095 2014-08-13 18:06:06.817544 7f5c973a57a0 20 read_log 192718'8998 (192718'8997) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:49 2014-08-06 15:33:45.548182 2014-08-13 18:06:06.817556 7f5c973a57a0 20 read_log 192718'8999 (192718'8998) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:48 2014-08-06 15:33:45.583813 2014-08-13 18:06:06.817584 7f5c973a57a0 20 read_log 192718'9000 (192718'8999) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:50 2014-08-06 15:33:45.627773 2014-08-13 18:06:06.817599 7f5c973a57a0 20 read_log 192718'9001 (192718'9000) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:49 2014-08-06 15:33:45.662404 2014-08-13 18:06:06.817618 7f5c973a57a0 20 read_log 192718'9002 (192718'9001) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:51 2014-08-06 15:33:45.694376 2014-08-13 18:06:06.817634 7f5c973a57a0 20 read_log 192718'9003 (192718'9002) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:50 2014-08-06 15:33:45.719720 2014-08-13 18:06:06.817647 7f5c973a57a0 20 read_log 192718'9004 (192718'9003) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:52 2014-08-06 15:33:45.788765 2014-08-13 18:06:06.817659 7f5c973a57a0 20 read_log 192718'9005 (192718'9004) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:51 2014-08-06 15:33:45.859312 2014-08-13 18:06:06.817682 7f5c973a57a0 20 read_log 192718'9006 (192718'9005) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:53 2014-08-06 15:33:45.908071 2014-08-13 18:06:06.817700 7f5c973a57a0 20 read_log 192718'9007 (192718'9006) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:52 2014-08-06 15:33:45.960220 2014-08-13 18:06:06.817723 7f5c973a57a0 20 read_log 192718'9008 (192718'9007) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:54 2014-08-06 15:33:46.014389 2014-08-13 18:06:06.817740 7f5c973a57a0 20 read_log 192718'9009 (192718'9008) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:53 2014-08-06 15:33:46.042290 2014-08-13 18:06:06.817752 7f5c973a57a0 20 read_log 192718'9010 (192718'9009) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:55 2014-08-06 15:33:46.074016 2014-08-13 18:06:06.817765 7f5c973a57a0 20 read_log 192718'9011 (192718'9010) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:54 2014-08-06 15:33:46.098642 2014-08-13 18:06:06.817796 7f5c973a57a0 20 read_log 192718'9012 (192718'9011) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:56 2014-08-06 15:33:46.127015 2014-08-13 18:06:06.817812 7f5c973a57a0 20 read_log 192718'9013 (192718'9012) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:55 2014-08-06 15:33:46.152137 2014-08-13 18:06:06.817824 7f5c973a57a0 20 read_log 192718'9014 (192718'9013) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:57 2014-08-06 15:33:46.180304 2014-08-13 18:06:06.817846 7f5c973a57a0 20 read_log 192718'9015 (192718'9014) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:56 2014-08-06 15:33:46.205526 2014-08-13 18:06:06.817860 7f5c973a57a0 20 read_log 192718'9016 (192718'9015) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:58 2014-08-06 15:33:46.233485 2014-08-13 18:06:06.817879 7f5c973a57a0 20 read_log 192718'9017 (192718'9016) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:57 2014-08-06 15:33:46.256496 2014-08-13 18:06:06.817904 7f5c973a57a0 20 read_log 192718'9018 (192718'9017) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:59 2014-08-06 15:33:46.288865 2014-08-13 18:06:06.817919 7f5c973a57a0 20 read_log 192718'9019 (192718'9018) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:58 2014-08-06 15:33:46.312524 2014-08-13 18:06:06.817932 7f5c973a57a0 20 read_log 192718'9020 (192718'9019) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:60 2014-08-06 15:33:46.341421 2014-08-13 18:06:06.817944 7f5c973a57a0 20 read_log 192718'9021 (192718'9020) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:59 2014-08-06 15:33:46.534739 2014-08-13 18:06:06.817957 7f5c973a57a0 20 read_log 192718'9022 (192718'9021) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:61 2014-08-06 15:33:46.563086 2014-08-13 18:06:06.817969 7f5c973a57a0 20 read_log 192718'9023 (192718'9022) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:60 2014-08-06 15:33:46.589751 2014-08-13 18:06:06.817981 7f5c973a57a0 20 read_log 192718'9024 (192718'9023) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:62 2014-08-06 15:33:46.616991 2014-08-13 18:06:06.817993 7f5c973a57a0 20 read_log 192718'9025 (192718'9024) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:61 2014-08-06 15:33:46.644054 2014-08-13 18:06:06.818006 7f5c973a57a0 20 read_log 192718'9026 (192718'9025) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:63 2014-08-06 15:33:46.670383 2014-08-13 18:06:06.818018 7f5c973a57a0 20 read_log 192718'9027 (192718'9026) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:62 2014-08-06 15:33:46.696713 2014-08-13 18:06:06.818030 7f5c973a57a0 20 read_log 192718'9028 (192718'9027) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:64 2014-08-06 15:33:46.723494 2014-08-13 18:06:06.818042 7f5c973a57a0 20 read_log 192718'9029 (192718'9028) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:63 2014-08-06 15:33:46.746932 2014-08-13 18:06:06.818055 7f5c973a57a0 20 read_log 192718'9030 (192718'9029) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:65 2014-08-06 15:33:46.775157 2014-08-13 18:06:06.818067 7f5c973a57a0 20 read_log 192718'9031 (192718'9030) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:64 2014-08-06 15:33:46.938135 2014-08-13 18:06:06.818079 7f5c973a57a0 20 read_log 192718'9032 (192718'9031) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:66 2014-08-06 15:33:46.967041 2014-08-13 18:06:06.818091 7f5c973a57a0 20 read_log 192718'9033 (192718'9032) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:65 2014-08-06 15:33:46.993449 2014-08-13 18:06:06.818104 7f5c973a57a0 20 read_log 192718'9034 (192718'9033) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:67 2014-08-06 15:33:47.018161 2014-08-13 18:06:06.818116 7f5c973a57a0 20 read_log 192718'9035 (192718'9034) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:66 2014-08-06 15:33:47.042690 2014-08-13 18:06:06.818130 7f5c973a57a0 20 read_log 192718'9036 (192718'9035) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:68 2014-08-06 15:33:47.069738 2014-08-13 18:06:06.818154 7f5c973a57a0 20 read_log 192718'9037 (192718'9036) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:67 2014-08-06 15:33:47.098292 2014-08-13 18:06:06.818172 7f5c973a57a0 20 read_log 192718'9038 (192718'9037) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:69 2014-08-06 15:33:47.144123 2014-08-13 18:06:06.818192 7f5c973a57a0 20 read_log 192718'9039 (192718'9038) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:68 2014-08-06 15:33:47.193446 2014-08-13 18:06:06.818206 7f5c973a57a0 20 read_log 192718'9040 (192718'9039) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:70 2014-08-06 15:33:47.220413 2014-08-13 18:06:06.818219 7f5c973a57a0 20 read_log 192718'9041 (192718'9040) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:69 2014-08-06 15:33:47.373971 2014-08-13 18:06:06.818231 7f5c973a57a0 20 read_log 192718'9042 (192718'9041) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:71 2014-08-06 15:33:47.403297 2014-08-13 18:06:06.818243 7f5c973a57a0 20 read_log 192718'9043 (192718'9042) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:70 2014-08-06 15:33:47.428720 2014-08-13 18:06:06.818258 7f5c973a57a0 20 read_log 192718'9044 (192718'9043) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:72 2014-08-06 15:33:47.453769 2014-08-13 18:06:06.818271 7f5c973a57a0 20 read_log 192718'9045 (192718'9044) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:71 2014-08-06 15:33:47.481730 2014-08-13 18:06:06.818283 7f5c973a57a0 20 read_log 192718'9046 (192718'9045) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:73 2014-08-06 15:33:47.505702 2014-08-13 18:06:06.818295 7f5c973a57a0 20 read_log 192718'9047 (192718'9046) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:72 2014-08-06 15:33:47.531510 2014-08-13 18:06:06.818311 7f5c973a57a0 20 read_log 192718'9048 (192718'9047) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:74 2014-08-06 15:33:47.556849 2014-08-13 18:06:06.818323 7f5c973a57a0 20 read_log 192718'9049 (192718'9048) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:73 2014-08-06 15:33:47.582648 2014-08-13 18:06:06.818335 7f5c973a57a0 20 read_log 192718'9050 (192718'9049) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:75 2014-08-06 15:33:47.608495 2014-08-13 18:06:06.818347 7f5c973a57a0 20 read_log 192718'9051 (192718'9050) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:74 2014-08-06 15:33:47.758962 2014-08-13 18:06:06.818363 7f5c973a57a0 20 read_log 192718'9052 (192718'9051) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:76 2014-08-06 15:33:47.784831 2014-08-13 18:06:06.818375 7f5c973a57a0 20 read_log 192718'9053 (192718'9052) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:75 2014-08-06 15:33:47.813206 2014-08-13 18:06:06.818387 7f5c973a57a0 20 read_log 192718'9054 (192718'9053) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:77 2014-08-06 15:33:47.838667 2014-08-13 18:06:06.818402 7f5c973a57a0 20 read_log 192718'9055 (192718'9054) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:76 2014-08-06 15:33:47.865093 2014-08-13 18:06:06.818417 7f5c973a57a0 20 read_log 192718'9056 (192718'9055) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:78 2014-08-06 15:33:47.890555 2014-08-13 18:06:06.818430 7f5c973a57a0 20 read_log 192718'9057 (192718'9056) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:77 2014-08-06 15:33:47.916152 2014-08-13 18:06:06.818450 7f5c973a57a0 20 read_log 192718'9058 (192718'9057) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:79 2014-08-06 15:33:47.941991 2014-08-13 18:06:06.818463 7f5c973a57a0 20 read_log 192718'9059 (192718'9058) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:78 2014-08-06 15:33:47.967509 2014-08-13 18:06:06.818475 7f5c973a57a0 20 read_log 192718'9060 (192718'9059) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:80 2014-08-06 15:33:47.992933 2014-08-13 18:06:06.818496 7f5c973a57a0 20 read_log 192718'9061 (192718'9060) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:79 2014-08-06 15:33:48.146960 2014-08-13 18:06:06.818511 7f5c973a57a0 20 read_log 192718'9062 (192718'9061) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:81 2014-08-06 15:33:48.175445 2014-08-13 18:06:06.818588 7f5c973a57a0 20 read_log 192718'9063 (192718'9062) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:80 2014-08-06 15:33:48.201339 2014-08-13 18:06:06.818605 7f5c973a57a0 20 read_log 192718'9064 (192718'9063) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:82 2014-08-06 15:33:48.226824 2014-08-13 18:06:06.818618 7f5c973a57a0 20 read_log 192718'9065 (192718'9064) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:81 2014-08-06 15:33:48.249480 2014-08-13 18:06:06.818630 7f5c973a57a0 20 read_log 192718'9066 (192718'9065) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:83 2014-08-06 15:33:48.276078 2014-08-13 18:06:06.818651 7f5c973a57a0 20 read_log 192718'9067 (192718'9066) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:82 2014-08-06 15:33:48.301039 2014-08-13 18:06:06.818667 7f5c973a57a0 20 read_log 192718'9068 (192718'9067) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:84 2014-08-06 15:33:48.327004 2014-08-13 18:06:06.818679 7f5c973a57a0 20 read_log 192718'9069 (192718'9068) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:83 2014-08-06 15:33:48.354868 2014-08-13 18:06:06.818692 7f5c973a57a0 20 read_log 192718'9070 (192718'9069) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:85 2014-08-06 15:33:48.381248 2014-08-13 18:06:06.818704 7f5c973a57a0 20 read_log 192718'9071 (192718'9070) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:84 2014-08-06 15:33:48.543411 2014-08-13 18:06:06.818716 7f5c973a57a0 20 read_log 192718'9072 (192718'9071) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:86 2014-08-06 15:33:48.563574 2014-08-13 18:06:06.818728 7f5c973a57a0 20 read_log 192718'9073 (192718'9072) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:85 2014-08-06 15:33:48.589747 2014-08-13 18:06:06.818740 7f5c973a57a0 20 read_log 192718'9074 (192718'9073) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:87 2014-08-06 15:33:48.618150 2014-08-13 18:06:06.818754 7f5c973a57a0 20 read_log 192718'9075 (192718'9074) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:86 2014-08-06 15:33:48.643471 2014-08-13 18:06:06.818766 7f5c973a57a0 20 read_log 192718'9076 (192718'9075) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:88 2014-08-06 15:33:48.669740 2014-08-13 18:06:06.818778 7f5c973a57a0 20 read_log 192718'9077 (192718'9076) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:87 2014-08-06 15:33:48.696564 2014-08-13 18:06:06.818808 7f5c973a57a0 20 read_log 192718'9078 (192718'9077) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:89 2014-08-06 15:33:48.722646 2014-08-13 18:06:06.818831 7f5c973a57a0 20 read_log 192718'9079 (192718'9078) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:88 2014-08-06 15:33:48.745807 2014-08-13 18:06:06.818848 7f5c973a57a0 20 read_log 192718'9080 (192718'9079) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:90 2014-08-06 15:33:48.775073 2014-08-13 18:06:06.818860 7f5c973a57a0 20 read_log 192718'9081 (192718'9080) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:89 2014-08-06 15:33:48.934134 2014-08-13 18:06:06.818872 7f5c973a57a0 20 read_log 192718'9082 (192718'9081) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:91 2014-08-06 15:33:48.960896 2014-08-13 18:06:06.818885 7f5c973a57a0 20 read_log 192718'9083 (192718'9082) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:90 2014-08-06 15:33:48.986938 2014-08-13 18:06:06.818897 7f5c973a57a0 20 read_log 192718'9084 (192718'9083) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:92 2014-08-06 15:33:49.013036 2014-08-13 18:06:06.818909 7f5c973a57a0 20 read_log 192718'9085 (192718'9084) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:91 2014-08-06 15:33:49.039747 2014-08-13 18:06:06.818924 7f5c973a57a0 20 read_log 192718'9086 (192718'9085) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:92 2014-08-06 15:33:49.094268 2014-08-13 18:06:06.818946 7f5c973a57a0 20 read_log 192718'9087 (192718'9086) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:93 2014-08-06 15:33:49.098281 2014-08-13 18:06:06.818962 7f5c973a57a0 20 read_log 192718'9088 (192718'9087) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:94 2014-08-06 15:33:49.146534 2014-08-13 18:06:06.818974 7f5c973a57a0 20 read_log 192718'9089 (192718'9088) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:93 2014-08-06 15:33:49.146130 2014-08-13 18:06:06.818986 7f5c973a57a0 20 read_log 192718'9090 (192718'9089) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:95 2014-08-06 15:33:49.330730 2014-08-13 18:06:06.818999 7f5c973a57a0 20 read_log 192718'9091 (192718'9090) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:94 2014-08-06 15:33:49.330871 2014-08-13 18:06:06.819011 7f5c973a57a0 20 read_log 192718'9092 (192718'9091) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:96 2014-08-06 15:33:49.387500 2014-08-13 18:06:06.819023 7f5c973a57a0 20 read_log 192718'9093 (192718'9092) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:95 2014-08-06 15:33:49.386184 2014-08-13 18:06:06.819035 7f5c973a57a0 20 read_log 192718'9094 (192718'9093) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:97 2014-08-06 15:33:49.419018 2014-08-13 18:06:06.819047 7f5c973a57a0 20 read_log 192718'9095 (192718'9094) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:96 2014-08-06 15:33:49.437329 2014-08-13 18:06:06.819059 7f5c973a57a0 20 read_log 192718'9096 (192718'9095) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:98 2014-08-06 15:33:49.466762 2014-08-13 18:06:06.819071 7f5c973a57a0 20 read_log 192718'9097 (192718'9096) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:97 2014-08-06 15:33:49.490454 2014-08-13 18:06:06.819083 7f5c973a57a0 20 read_log 192718'9098 (192718'9097) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:99 2014-08-06 15:33:49.519624 2014-08-13 18:06:06.819098 7f5c973a57a0 20 read_log 192718'9099 (192718'9098) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:98 2014-08-06 15:33:49.543130 2014-08-13 18:06:06.819110 7f5c973a57a0 20 read_log 192718'9100 (192718'9099) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:100 2014-08-06 15:33:49.575618 2014-08-13 18:06:06.819124 7f5c973a57a0 20 read_log 192718'9101 (192718'9100) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:99 2014-08-06 15:33:49.727968 2014-08-13 18:06:06.819137 7f5c973a57a0 20 read_log 192718'9102 (192718'9101) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:101 2014-08-06 15:33:49.756554 2014-08-13 18:06:06.819157 7f5c973a57a0 20 read_log 192718'9103 (192718'9102) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:100 2014-08-06 15:33:49.782048 2014-08-13 18:06:06.819172 7f5c973a57a0 20 read_log 192718'9104 (192718'9103) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:102 2014-08-06 15:33:49.811017 2014-08-13 18:06:06.819184 7f5c973a57a0 20 read_log 192718'9105 (192718'9104) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:101 2014-08-06 15:33:49.837888 2014-08-13 18:06:06.819199 7f5c973a57a0 20 read_log 192718'9106 (192718'9105) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:103 2014-08-06 15:33:49.865239 2014-08-13 18:06:06.819212 7f5c973a57a0 20 read_log 192718'9107 (192718'9106) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:102 2014-08-06 15:33:49.890008 2014-08-13 18:06:06.819226 7f5c973a57a0 20 read_log 192718'9108 (192718'9107) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:104 2014-08-06 15:33:49.919250 2014-08-13 18:06:06.819238 7f5c973a57a0 20 read_log 192718'9109 (192718'9108) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:103 2014-08-06 15:33:49.942082 2014-08-13 18:06:06.819253 7f5c973a57a0 20 read_log 192718'9110 (192718'9109) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:105 2014-08-06 15:33:49.972422 2014-08-13 18:06:06.819272 7f5c973a57a0 20 read_log 192718'9111 (192718'9110) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:104 2014-08-06 15:33:50.127074 2014-08-13 18:06:06.819289 7f5c973a57a0 20 read_log 192718'9112 (192718'9111) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:106 2014-08-06 15:33:50.156511 2014-08-13 18:06:06.819302 7f5c973a57a0 20 read_log 192718'9113 (192718'9112) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:105 2014-08-06 15:33:50.241121 2014-08-13 18:06:06.819314 7f5c973a57a0 20 read_log 192718'9114 (192718'9113) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:107 2014-08-06 15:33:50.298756 2014-08-13 18:06:06.819327 7f5c973a57a0 20 read_log 192718'9115 (192718'9114) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:106 2014-08-06 15:33:50.330806 2014-08-13 18:06:06.819341 7f5c973a57a0 20 read_log 192718'9116 (192718'9115) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:108 2014-08-06 15:33:50.364831 2014-08-13 18:06:06.819354 7f5c973a57a0 20 read_log 192718'9117 (192718'9116) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:107 2014-08-06 15:33:50.399341 2014-08-13 18:06:06.819366 7f5c973a57a0 20 read_log 192718'9118 (192718'9117) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:109 2014-08-06 15:33:50.435105 2014-08-13 18:06:06.819378 7f5c973a57a0 20 read_log 192718'9119 (192718'9118) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:108 2014-08-06 15:33:50.466830 2014-08-13 18:06:06.819394 7f5c973a57a0 20 read_log 192718'9120 (192718'9119) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:110 2014-08-06 15:33:50.502574 2014-08-13 18:06:06.819406 7f5c973a57a0 20 read_log 192718'9121 (192718'9120) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:109 2014-08-06 15:33:50.757938 2014-08-13 18:06:06.819418 7f5c973a57a0 20 read_log 192718'9122 (192718'9121) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:111 2014-08-06 15:33:50.793010 2014-08-13 18:06:06.819431 7f5c973a57a0 20 read_log 192718'9123 (192718'9122) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:110 2014-08-06 15:33:50.825902 2014-08-13 18:06:06.819446 7f5c973a57a0 20 read_log 192718'9124 (192718'9123) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:112 2014-08-06 15:33:50.860935 2014-08-13 18:06:06.819458 7f5c973a57a0 20 read_log 192718'9125 (192718'9124) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:111 2014-08-06 15:33:50.897174 2014-08-13 18:06:06.819470 7f5c973a57a0 20 read_log 192718'9126 (192718'9125) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:113 2014-08-06 15:33:50.939688 2014-08-13 18:06:06.819482 7f5c973a57a0 20 read_log 192718'9127 (192718'9126) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:112 2014-08-06 15:33:50.972702 2014-08-13 18:06:06.819494 7f5c973a57a0 20 read_log 192718'9128 (192718'9127) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:114 2014-08-06 15:33:51.008366 2014-08-13 18:06:06.819516 7f5c973a57a0 20 read_log 192718'9129 (192718'9128) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:113 2014-08-06 15:33:51.040998 2014-08-13 18:06:06.819532 7f5c973a57a0 20 read_log 192718'9130 (192718'9129) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:115 2014-08-06 15:33:51.085477 2014-08-13 18:06:06.819560 7f5c973a57a0 20 read_log 192718'9131 (192718'9130) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:114 2014-08-06 15:33:51.164684 2014-08-13 18:06:06.819577 7f5c973a57a0 20 read_log 192718'9132 (192718'9131) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:116 2014-08-06 15:33:51.324742 2014-08-13 18:06:06.819589 7f5c973a57a0 20 read_log 192718'9133 (192718'9132) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:115 2014-08-06 15:33:51.361592 2014-08-13 18:06:06.819609 7f5c973a57a0 20 read_log 192718'9134 (192718'9133) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:117 2014-08-06 15:33:51.484211 2014-08-13 18:06:06.819636 7f5c973a57a0 20 read_log 192718'9135 (192718'9134) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:116 2014-08-06 15:33:51.519890 2014-08-13 18:06:06.819659 7f5c973a57a0 20 read_log 192718'9136 (192718'9135) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:118 2014-08-06 15:33:51.562528 2014-08-13 18:06:06.819676 7f5c973a57a0 20 read_log 192718'9137 (192718'9136) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:117 2014-08-06 15:33:51.599588 2014-08-13 18:06:06.819688 7f5c973a57a0 20 read_log 192718'9138 (192718'9137) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:119 2014-08-06 15:33:51.638092 2014-08-13 18:06:06.819700 7f5c973a57a0 20 read_log 192718'9139 (192718'9138) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:118 2014-08-06 15:33:51.674076 2014-08-13 18:06:06.819713 7f5c973a57a0 20 read_log 192718'9140 (192718'9139) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:120 2014-08-06 15:33:51.712797 2014-08-13 18:06:06.819725 7f5c973a57a0 20 read_log 192718'9141 (192718'9140) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:119 2014-08-06 15:33:51.746234 2014-08-13 18:06:06.819737 7f5c973a57a0 20 read_log 192718'9142 (192718'9141) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:121 2014-08-06 15:33:51.784349 2014-08-13 18:06:06.819749 7f5c973a57a0 20 read_log 192718'9143 (192718'9142) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:120 2014-08-06 15:33:51.824167 2014-08-13 18:06:06.819761 7f5c973a57a0 20 read_log 192718'9144 (192718'9143) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:122 2014-08-06 15:33:51.867246 2014-08-13 18:06:06.819774 7f5c973a57a0 20 read_log 192718'9145 (192718'9144) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:121 2014-08-06 15:33:51.903714 2014-08-13 18:06:06.819787 7f5c973a57a0 20 read_log 192718'9146 (192718'9145) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:123 2014-08-06 15:33:51.942229 2014-08-13 18:06:06.819799 7f5c973a57a0 20 read_log 192718'9147 (192718'9146) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:122 2014-08-06 15:33:52.203799 2014-08-13 18:06:06.819819 7f5c973a57a0 20 read_log 192718'9148 (192718'9147) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:124 2014-08-06 15:33:52.241584 2014-08-13 18:06:06.819832 7f5c973a57a0 20 read_log 192718'9149 (192718'9148) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:123 2014-08-06 15:33:52.277068 2014-08-13 18:06:06.819844 7f5c973a57a0 20 read_log 192718'9150 (192718'9149) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:125 2014-08-06 15:33:52.317953 2014-08-13 18:06:06.819858 7f5c973a57a0 20 read_log 192718'9151 (192718'9150) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:124 2014-08-06 15:33:52.353796 2014-08-13 18:06:06.819870 7f5c973a57a0 20 read_log 192718'9152 (192718'9151) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:126 2014-08-06 15:33:52.393481 2014-08-13 18:06:06.819882 7f5c973a57a0 20 read_log 192718'9153 (192718'9152) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:125 2014-08-06 15:33:52.429297 2014-08-13 18:06:06.819894 7f5c973a57a0 20 read_log 192718'9154 (192718'9153) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:127 2014-08-06 15:33:52.477491 2014-08-13 18:06:06.819927 7f5c973a57a0 20 read_log 192718'9155 (192718'9154) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:126 2014-08-06 15:33:52.504186 2014-08-13 18:06:06.819940 7f5c973a57a0 20 read_log 192718'9156 (192718'9155) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:128 2014-08-06 15:33:52.543157 2014-08-13 18:06:06.819952 7f5c973a57a0 20 read_log 192718'9157 (192718'9156) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:127 2014-08-06 15:33:52.763883 2014-08-13 18:06:06.819964 7f5c973a57a0 20 read_log 192718'9158 (192718'9157) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:129 2014-08-06 15:33:52.810650 2014-08-13 18:06:06.819986 7f5c973a57a0 20 read_log 192718'9159 (192718'9158) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:128 2014-08-06 15:33:52.846641 2014-08-13 18:06:06.820010 7f5c973a57a0 20 read_log 192718'9160 (192718'9159) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:130 2014-08-06 15:33:52.885142 2014-08-13 18:06:06.820026 7f5c973a57a0 20 read_log 192718'9161 (192718'9160) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:129 2014-08-06 15:33:52.921558 2014-08-13 18:06:06.820038 7f5c973a57a0 20 read_log 192718'9162 (192718'9161) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:131 2014-08-06 15:33:52.959276 2014-08-13 18:06:06.820051 7f5c973a57a0 20 read_log 192718'9163 (192718'9162) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:130 2014-08-06 15:33:52.991247 2014-08-13 18:06:06.820065 7f5c973a57a0 20 read_log 192718'9164 (192718'9163) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:132 2014-08-06 15:33:53.026127 2014-08-13 18:06:06.820078 7f5c973a57a0 20 read_log 192718'9165 (192718'9164) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:131 2014-08-06 15:33:53.058356 2014-08-13 18:06:06.820090 7f5c973a57a0 20 read_log 192718'9166 (192718'9165) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:133 2014-08-06 15:33:53.093606 2014-08-13 18:06:06.820105 7f5c973a57a0 20 read_log 192718'9167 (192718'9166) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:132 2014-08-06 15:33:53.291865 2014-08-13 18:06:06.820117 7f5c973a57a0 20 read_log 192718'9168 (192718'9167) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:134 2014-08-06 15:33:53.324328 2014-08-13 18:06:06.820129 7f5c973a57a0 20 read_log 192718'9169 (192718'9168) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:133 2014-08-06 15:33:53.359442 2014-08-13 18:06:06.820141 7f5c973a57a0 20 read_log 192718'9170 (192718'9169) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:135 2014-08-06 15:33:53.396204 2014-08-13 18:06:06.820177 7f5c973a57a0 20 read_log 192718'9171 (192718'9170) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:134 2014-08-06 15:33:53.427731 2014-08-13 18:06:06.820190 7f5c973a57a0 20 read_log 192718'9172 (192718'9171) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:136 2014-08-06 15:33:53.462780 2014-08-13 18:06:06.820208 7f5c973a57a0 20 read_log 192718'9173 (192718'9172) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:135 2014-08-06 15:33:53.494577 2014-08-13 18:06:06.820220 7f5c973a57a0 20 read_log 192718'9174 (192718'9173) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:137 2014-08-06 15:33:53.530318 2014-08-13 18:06:06.820243 7f5c973a57a0 20 read_log 192718'9175 (192718'9174) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:136 2014-08-06 15:33:53.561819 2014-08-13 18:06:06.820256 7f5c973a57a0 20 read_log 192718'9176 (192718'9175) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:138 2014-08-06 15:33:53.600514 2014-08-13 18:06:06.820268 7f5c973a57a0 20 read_log 192718'9177 (192718'9176) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:137 2014-08-06 15:33:53.797131 2014-08-13 18:06:06.820280 7f5c973a57a0 20 read_log 192718'9178 (192718'9177) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:139 2014-08-06 15:33:53.835838 2014-08-13 18:06:06.820292 7f5c973a57a0 20 read_log 192718'9179 (192718'9178) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:138 2014-08-06 15:33:53.876048 2014-08-13 18:06:06.820304 7f5c973a57a0 20 read_log 192718'9180 (192718'9179) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:140 2014-08-06 15:33:53.910714 2014-08-13 18:06:06.820316 7f5c973a57a0 20 read_log 192718'9181 (192718'9180) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:139 2014-08-06 15:33:53.943270 2014-08-13 18:06:06.820341 7f5c973a57a0 20 read_log 192718'9182 (192718'9181) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:141 2014-08-06 15:33:53.978042 2014-08-13 18:06:06.820361 7f5c973a57a0 20 read_log 192718'9183 (192718'9182) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:140 2014-08-06 15:33:54.010280 2014-08-13 18:06:06.820384 7f5c973a57a0 20 read_log 192718'9184 (192718'9183) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:142 2014-08-06 15:33:54.045371 2014-08-13 18:06:06.820399 7f5c973a57a0 20 read_log 192718'9185 (192718'9184) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:141 2014-08-06 15:33:54.077409 2014-08-13 18:06:06.820412 7f5c973a57a0 20 read_log 192718'9186 (192718'9185) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:143 2014-08-06 15:33:54.112113 2014-08-13 18:06:06.820424 7f5c973a57a0 20 read_log 192718'9187 (192718'9186) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:142 2014-08-06 15:33:54.310712 2014-08-13 18:06:06.820443 7f5c973a57a0 20 read_log 192718'9188 (192718'9187) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:144 2014-08-06 15:33:54.351214 2014-08-13 18:06:06.820459 7f5c973a57a0 20 read_log 192718'9189 (192718'9188) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:143 2014-08-06 15:33:54.383298 2014-08-13 18:06:06.820489 7f5c973a57a0 20 read_log 192718'9190 (192718'9189) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:145 2014-08-06 15:33:54.418203 2014-08-13 18:06:06.820504 7f5c973a57a0 20 read_log 192718'9191 (192718'9190) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:144 2014-08-06 15:33:54.464300 2014-08-13 18:06:06.820516 7f5c973a57a0 20 read_log 192718'9192 (192718'9191) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:146 2014-08-06 15:33:54.491854 2014-08-13 18:06:06.820529 7f5c973a57a0 20 read_log 192718'9193 (192718'9192) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:145 2014-08-06 15:33:54.518304 2014-08-13 18:06:06.820541 7f5c973a57a0 20 read_log 192718'9194 (192718'9193) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:147 2014-08-06 15:33:54.545591 2014-08-13 18:06:06.820553 7f5c973a57a0 20 read_log 192718'9195 (192718'9194) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:146 2014-08-06 15:33:54.572640 2014-08-13 18:06:06.820565 7f5c973a57a0 20 read_log 192718'9196 (192718'9195) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:148 2014-08-06 15:33:54.601259 2014-08-13 18:06:06.820577 7f5c973a57a0 20 read_log 192718'9197 (192718'9196) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:147 2014-08-06 15:33:54.759197 2014-08-13 18:06:06.820589 7f5c973a57a0 20 read_log 192718'9198 (192718'9197) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:149 2014-08-06 15:33:54.788460 2014-08-13 18:06:06.820601 7f5c973a57a0 20 read_log 192718'9199 (192718'9198) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:148 2014-08-06 15:33:54.817174 2014-08-13 18:06:06.820613 7f5c973a57a0 20 read_log 192718'9200 (192718'9199) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:150 2014-08-06 15:33:54.855905 2014-08-13 18:06:06.820625 7f5c973a57a0 20 read_log 192718'9201 (192718'9200) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:149 2014-08-06 15:33:54.871692 2014-08-13 18:06:06.820637 7f5c973a57a0 20 read_log 192718'9202 (192718'9201) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:151 2014-08-06 15:33:54.921069 2014-08-13 18:06:06.820649 7f5c973a57a0 20 read_log 192718'9203 (192718'9202) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:150 2014-08-06 15:33:54.925942 2014-08-13 18:06:06.820661 7f5c973a57a0 20 read_log 192718'9204 (192718'9203) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:151 2014-08-06 15:33:54.981492 2014-08-13 18:06:06.820673 7f5c973a57a0 20 read_log 192718'9205 (192718'9204) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:152 2014-08-06 15:33:55.004717 2014-08-13 18:06:06.820688 7f5c973a57a0 20 read_log 192718'9206 (192718'9205) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:152 2014-08-06 15:33:55.143600 2014-08-13 18:06:06.820707 7f5c973a57a0 20 read_log 192718'9207 (192718'9206) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:153 2014-08-06 15:33:55.202419 2014-08-13 18:06:06.820721 7f5c973a57a0 20 read_log 192718'9208 (192718'9207) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:153 2014-08-06 15:33:55.214821 2014-08-13 18:06:06.820734 7f5c973a57a0 20 read_log 192718'9209 (192718'9208) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:154 2014-08-06 15:33:55.255712 2014-08-13 18:06:06.820748 7f5c973a57a0 20 read_log 192718'9210 (192718'9209) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:154 2014-08-06 15:33:55.255496 2014-08-13 18:06:06.820760 7f5c973a57a0 20 read_log 192718'9211 (192718'9210) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:155 2014-08-06 15:33:55.336803 2014-08-13 18:06:06.820774 7f5c973a57a0 20 read_log 192718'9212 (192718'9211) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:155 2014-08-06 15:33:55.336510 2014-08-13 18:06:06.820810 7f5c973a57a0 20 read_log 192718'9214 (192718'9212) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:156 2014-08-06 15:33:57.971693 2014-08-13 18:06:06.820823 7f5c973a57a0 20 read_log 192718'9215 (192718'9214) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:156 2014-08-06 15:33:57.971595 2014-08-13 18:06:06.820835 7f5c973a57a0 20 read_log 192718'9216 (192718'9215) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:157 2014-08-06 15:33:58.026236 2014-08-13 18:06:06.820847 7f5c973a57a0 20 read_log 192718'9217 (192718'9216) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:157 2014-08-06 15:33:58.077564 2014-08-13 18:06:06.820862 7f5c973a57a0 20 read_log 192718'9218 (192718'9217) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:158 2014-08-06 15:33:58.113666 2014-08-13 18:06:06.820874 7f5c973a57a0 20 read_log 192718'9219 (192718'9218) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:158 2014-08-06 15:33:58.113617 2014-08-13 18:06:06.820888 7f5c973a57a0 20 read_log 192718'9220 (192718'9219) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:159 2014-08-06 15:33:58.240974 2014-08-13 18:06:06.820901 7f5c973a57a0 20 read_log 192718'9221 (192718'9220) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:159 2014-08-06 15:33:58.241693 2014-08-13 18:06:06.820914 7f5c973a57a0 20 read_log 192718'9222 (192718'9221) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:160 2014-08-06 15:33:58.304257 2014-08-13 18:06:06.820928 7f5c973a57a0 20 read_log 192718'9223 (192718'9222) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:160 2014-08-06 15:33:58.304325 2014-08-13 18:06:06.820942 7f5c973a57a0 20 read_log 192718'9224 (192718'9223) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:161 2014-08-06 15:33:58.492123 2014-08-13 18:06:06.820955 7f5c973a57a0 20 read_log 192718'9225 (192718'9224) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:161 2014-08-06 15:33:58.492161 2014-08-13 18:06:06.820969 7f5c973a57a0 20 read_log 192718'9226 (192718'9225) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:162 2014-08-06 15:33:58.545954 2014-08-13 18:06:06.820983 7f5c973a57a0 20 read_log 192718'9227 (192718'9226) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:162 2014-08-06 15:33:58.545810 2014-08-13 18:06:06.820998 7f5c973a57a0 20 read_log 192718'9228 (192718'9227) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:163 2014-08-06 15:33:58.608016 2014-08-13 18:06:06.821010 7f5c973a57a0 20 read_log 192718'9229 (192718'9228) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:163 2014-08-06 15:33:58.608485 2014-08-13 18:06:06.821024 7f5c973a57a0 20 read_log 192718'9230 (192718'9229) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:164 2014-08-06 15:33:58.737578 2014-08-13 18:06:06.821036 7f5c973a57a0 20 read_log 192718'9231 (192718'9230) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:164 2014-08-06 15:33:58.737576 2014-08-13 18:06:06.821071 7f5c973a57a0 20 read_log 192718'9232 (192718'9231) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:165 2014-08-06 15:33:58.891433 2014-08-13 18:06:06.821087 7f5c973a57a0 20 read_log 192718'9233 (192718'9232) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:165 2014-08-06 15:33:58.891276 2014-08-13 18:06:06.821099 7f5c973a57a0 20 read_log 192718'9234 (192718'9233) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:166 2014-08-06 15:33:58.953346 2014-08-13 18:06:06.821114 7f5c973a57a0 20 read_log 192718'9235 (192718'9234) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:166 2014-08-06 15:33:58.953558 2014-08-13 18:06:06.821128 7f5c973a57a0 20 read_log 192718'9236 (192718'9235) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:167 2014-08-06 15:33:59.451856 2014-08-13 18:06:06.821141 7f5c973a57a0 20 read_log 192718'9237 (192718'9236) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:167 2014-08-06 15:33:59.451713 2014-08-13 18:06:06.821177 7f5c973a57a0 20 read_log 192718'9238 (192718'9237) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:168 2014-08-06 15:33:59.485057 2014-08-13 18:06:06.821192 7f5c973a57a0 20 read_log 192718'9239 (192718'9238) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:168 2014-08-06 15:33:59.483377 2014-08-13 18:06:06.821205 7f5c973a57a0 20 read_log 192718'9240 (192718'9239) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:169 2014-08-06 15:33:59.514184 2014-08-13 18:06:06.821217 7f5c973a57a0 20 read_log 192718'9241 (192718'9240) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:169 2014-08-06 15:33:59.592692 2014-08-13 18:06:06.821233 7f5c973a57a0 20 read_log 192718'9242 (192718'9241) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:170 2014-08-06 15:33:59.635541 2014-08-13 18:06:06.821247 7f5c973a57a0 20 read_log 192718'9243 (192718'9242) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:170 2014-08-06 15:33:59.635613 2014-08-13 18:06:06.821262 7f5c973a57a0 20 read_log 192718'9244 (192718'9243) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:171 2014-08-06 15:33:59.779157 2014-08-13 18:06:06.821276 7f5c973a57a0 20 read_log 192718'9245 (192718'9244) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:171 2014-08-06 15:33:59.779241 2014-08-13 18:06:06.821289 7f5c973a57a0 20 read_log 192718'9246 (192718'9245) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:172 2014-08-06 15:33:59.833125 2014-08-13 18:06:06.821301 7f5c973a57a0 20 read_log 192718'9247 (192718'9246) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:172 2014-08-06 15:33:59.833053 2014-08-13 18:06:06.821313 7f5c973a57a0 20 read_log 192718'9248 (192718'9247) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:173 2014-08-06 15:33:59.899740 2014-08-13 18:06:06.821325 7f5c973a57a0 20 read_log 192718'9249 (192718'9248) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:173 2014-08-06 15:33:59.900223 2014-08-13 18:06:06.821337 7f5c973a57a0 20 read_log 192718'9250 (192718'9249) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:174 2014-08-06 15:34:00.033148 2014-08-13 18:06:06.821352 7f5c973a57a0 20 read_log 192718'9251 (192718'9250) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:174 2014-08-06 15:34:00.035125 2014-08-13 18:06:06.821367 7f5c973a57a0 20 read_log 192718'9252 (192718'9251) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:175 2014-08-06 15:34:00.147843 2014-08-13 18:06:06.821380 7f5c973a57a0 20 read_log 192718'9253 (192718'9252) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:175 2014-08-06 15:34:00.147811 2014-08-13 18:06:06.821404 7f5c973a57a0 20 read_log 192718'9254 (192718'9253) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:176 2014-08-06 15:34:00.203091 2014-08-13 18:06:06.821420 7f5c973a57a0 20 read_log 192718'9255 (192718'9254) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:176 2014-08-06 15:34:00.203010 2014-08-13 18:06:06.821453 7f5c973a57a0 20 read_log 192718'9256 (192718'9255) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:177 2014-08-06 15:34:00.279904 2014-08-13 18:06:06.821470 7f5c973a57a0 20 read_log 192718'9257 (192718'9256) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:177 2014-08-06 15:34:00.279959 2014-08-13 18:06:06.821482 7f5c973a57a0 20 read_log 192718'9258 (192718'9257) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:178 2014-08-06 15:34:00.412395 2014-08-13 18:06:06.821505 7f5c973a57a0 20 read_log 192718'9259 (192718'9258) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:178 2014-08-06 15:34:00.412327 2014-08-13 18:06:06.821520 7f5c973a57a0 20 read_log 192718'9260 (192718'9259) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:179 2014-08-06 15:34:00.448703 2014-08-13 18:06:06.821533 7f5c973a57a0 20 read_log 192718'9261 (192718'9260) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:179 2014-08-06 15:34:00.448684 2014-08-13 18:06:06.821547 7f5c973a57a0 20 read_log 192718'9262 (192718'9261) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:180 2014-08-06 15:34:00.524352 2014-08-13 18:06:06.821559 7f5c973a57a0 20 read_log 192718'9263 (192718'9262) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:180 2014-08-06 15:34:00.525559 2014-08-13 18:06:06.821574 7f5c973a57a0 20 read_log 192718'9264 (192718'9263) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:181 2014-08-06 15:34:00.555553 2014-08-13 18:06:06.821587 7f5c973a57a0 20 read_log 192718'9265 (192718'9264) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:181 2014-08-06 15:34:00.558518 2014-08-13 18:06:06.821599 7f5c973a57a0 20 read_log 192718'9266 (192718'9265) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:182 2014-08-06 15:34:00.586530 2014-08-13 18:06:06.821611 7f5c973a57a0 20 read_log 192718'9267 (192718'9266) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:182 2014-08-06 15:34:00.709623 2014-08-13 18:06:06.821630 7f5c973a57a0 20 read_log 192718'9268 (192718'9267) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:183 2014-08-06 15:34:00.724023 2014-08-13 18:06:06.821646 7f5c973a57a0 20 read_log 192718'9269 (192718'9268) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:183 2014-08-06 15:34:00.738272 2014-08-13 18:06:06.821659 7f5c973a57a0 20 read_log 192718'9270 (192718'9269) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:184 2014-08-06 15:34:00.760095 2014-08-13 18:06:06.821671 7f5c973a57a0 20 read_log 192718'9271 (192718'9270) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:184 2014-08-06 15:34:00.759493 2014-08-13 18:06:06.821683 7f5c973a57a0 20 read_log 192718'9272 (192718'9271) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:185 2014-08-06 15:34:00.877716 2014-08-13 18:06:06.821696 7f5c973a57a0 20 read_log 192718'9273 (192718'9272) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:185 2014-08-06 15:34:00.881886 2014-08-13 18:06:06.821708 7f5c973a57a0 20 read_log 192718'9274 (192718'9273) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:186 2014-08-06 15:34:00.908512 2014-08-13 18:06:06.821720 7f5c973a57a0 20 read_log 192718'9275 (192718'9274) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:186 2014-08-06 15:34:00.911689 2014-08-13 18:06:06.821732 7f5c973a57a0 20 read_log 192718'9276 (192718'9275) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:187 2014-08-06 15:34:00.938439 2014-08-13 18:06:06.821744 7f5c973a57a0 20 read_log 192718'9277 (192718'9276) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:187 2014-08-06 15:34:00.942208 2014-08-13 18:06:06.821756 7f5c973a57a0 20 read_log 192718'9278 (192718'9277) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:188 2014-08-06 15:34:00.966178 2014-08-13 18:06:06.821768 7f5c973a57a0 20 read_log 192718'9279 (192718'9278) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:188 2014-08-06 15:34:00.969434 2014-08-13 18:06:06.821787 7f5c973a57a0 20 read_log 192718'9280 (192718'9279) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:189 2014-08-06 15:34:01.064579 2014-08-13 18:06:06.821802 7f5c973a57a0 20 read_log 192718'9281 (192718'9280) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:189 2014-08-06 15:34:01.065508 2014-08-13 18:06:06.821815 7f5c973a57a0 20 read_log 192718'9282 (192718'9281) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:190 2014-08-06 15:34:01.094522 2014-08-13 18:06:06.821827 7f5c973a57a0 20 read_log 192718'9283 (192718'9282) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:190 2014-08-06 15:34:01.106804 2014-08-13 18:06:06.821857 7f5c973a57a0 20 read_log 192718'9284 (192718'9283) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:191 2014-08-06 15:34:01.122505 2014-08-13 18:06:06.821872 7f5c973a57a0 20 read_log 192718'9285 (192718'9284) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:191 2014-08-06 15:34:01.132211 2014-08-13 18:06:06.821884 7f5c973a57a0 20 read_log 192718'9286 (192718'9285) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:192 2014-08-06 15:34:01.188180 2014-08-13 18:06:06.821897 7f5c973a57a0 20 read_log 192718'9287 (192718'9286) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:192 2014-08-06 15:34:01.242561 2014-08-13 18:06:06.821909 7f5c973a57a0 20 read_log 192718'9288 (192718'9287) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:193 2014-08-06 15:34:01.334920 2014-08-13 18:06:06.821921 7f5c973a57a0 20 read_log 192718'9289 (192718'9288) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:193 2014-08-06 15:34:01.486341 2014-08-13 18:06:06.821933 7f5c973a57a0 20 read_log 192718'9290 (192718'9289) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:194 2014-08-06 15:34:01.655882 2014-08-13 18:06:06.821945 7f5c973a57a0 20 read_log 192718'9291 (192718'9290) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:194 2014-08-06 15:34:01.747673 2014-08-13 18:06:06.821957 7f5c973a57a0 20 read_log 192718'9292 (192718'9291) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:195 2014-08-06 15:34:02.139908 2014-08-13 18:06:06.821971 7f5c973a57a0 20 read_log 192718'9293 (192718'9292) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:195 2014-08-06 15:34:02.141621 2014-08-13 18:06:06.821986 7f5c973a57a0 20 read_log 192718'9294 (192718'9293) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:196 2014-08-06 15:34:03.056050 2014-08-13 18:06:06.821998 7f5c973a57a0 20 read_log 192718'9295 (192718'9294) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:196 2014-08-06 15:34:03.056369 2014-08-13 18:06:06.822013 7f5c973a57a0 20 read_log 192718'9296 (192718'9295) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:197 2014-08-06 15:34:03.089419 2014-08-13 18:06:06.822025 7f5c973a57a0 20 read_log 192718'9297 (192718'9296) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:197 2014-08-06 15:34:03.089741 2014-08-13 18:06:06.822037 7f5c973a57a0 20 read_log 192718'9298 (192718'9297) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:198 2014-08-06 15:34:03.229921 2014-08-13 18:06:06.822051 7f5c973a57a0 20 read_log 192718'9299 (192718'9298) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:198 2014-08-06 15:34:03.304814 2014-08-13 18:06:06.822064 7f5c973a57a0 20 read_log 192718'9300 (192718'9299) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:199 2014-08-06 15:34:03.461725 2014-08-13 18:06:06.822078 7f5c973a57a0 20 read_log 192718'9301 (192718'9300) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:199 2014-08-06 15:34:03.679729 2014-08-13 18:06:06.822092 7f5c973a57a0 20 read_log 192718'9302 (192718'9301) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:200 2014-08-06 15:34:03.721100 2014-08-13 18:06:06.822104 7f5c973a57a0 20 read_log 192718'9303 (192718'9302) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:200 2014-08-06 15:34:03.906722 2014-08-13 18:06:06.822123 7f5c973a57a0 20 read_log 192718'9304 (192718'9303) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:201 2014-08-06 15:34:04.077007 2014-08-13 18:06:06.822138 7f5c973a57a0 20 read_log 192718'9305 (192718'9304) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:201 2014-08-06 15:34:04.157280 2014-08-13 18:06:06.822159 7f5c973a57a0 20 read_log 192718'9306 (192718'9305) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:202 2014-08-06 15:34:04.425285 2014-08-13 18:06:06.822174 7f5c973a57a0 20 read_log 192718'9307 (192718'9306) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:202 2014-08-06 15:34:04.549345 2014-08-13 18:06:06.822187 7f5c973a57a0 20 read_log 192718'9308 (192718'9307) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:203 2014-08-06 15:34:04.552950 2014-08-13 18:06:06.822199 7f5c973a57a0 20 read_log 192718'9309 (192718'9308) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:203 2014-08-06 15:34:04.941910 2014-08-13 18:06:06.822271 7f5c973a57a0 20 read_log 192718'9310 (192718'9309) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:204 2014-08-06 15:34:04.964242 2014-08-13 18:06:06.822291 7f5c973a57a0 20 read_log 192718'9311 (192718'9310) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:204 2014-08-06 15:34:05.100719 2014-08-13 18:06:06.822303 7f5c973a57a0 20 read_log 192718'9312 (192718'9311) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:205 2014-08-06 15:34:05.161560 2014-08-13 18:06:06.822336 7f5c973a57a0 20 read_log 192718'9313 (192718'9312) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:205 2014-08-06 15:34:05.329177 2014-08-13 18:06:06.822349 7f5c973a57a0 20 read_log 192718'9314 (192718'9313) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:206 2014-08-06 15:34:05.359101 2014-08-13 18:06:06.822361 7f5c973a57a0 20 read_log 192718'9315 (192718'9314) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:206 2014-08-06 15:34:05.784527 2014-08-13 18:06:06.822373 7f5c973a57a0 20 read_log 192718'9316 (192718'9315) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:207 2014-08-06 15:34:05.829646 2014-08-13 18:06:06.822385 7f5c973a57a0 20 read_log 192718'9317 (192718'9316) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:207 2014-08-06 15:34:06.162798 2014-08-13 18:06:06.822397 7f5c973a57a0 20 read_log 192718'9318 (192718'9317) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:208 2014-08-06 15:34:06.838592 2014-08-13 18:06:06.822418 7f5c973a57a0 20 read_log 192718'9319 (192718'9318) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:208 2014-08-06 15:34:06.836982 2014-08-13 18:06:06.822441 7f5c973a57a0 20 read_log 192718'9320 (192718'9319) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:209 2014-08-06 15:34:07.098794 2014-08-13 18:06:06.822462 7f5c973a57a0 20 read_log 192718'9321 (192718'9320) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:209 2014-08-06 15:34:07.293837 2014-08-13 18:06:06.822475 7f5c973a57a0 20 read_log 192718'9322 (192718'9321) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:210 2014-08-06 15:34:07.297198 2014-08-13 18:06:06.822488 7f5c973a57a0 20 read_log 192718'9323 (192718'9322) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:211 2014-08-06 15:34:08.854318 2014-08-13 18:06:06.822501 7f5c973a57a0 20 read_log 192718'9324 (192718'9323) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:210 2014-08-06 15:34:08.851889 2014-08-13 18:06:06.822514 7f5c973a57a0 20 read_log 192718'9325 (192718'9324) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:212 2014-08-06 15:34:09.210699 2014-08-13 18:06:06.822527 7f5c973a57a0 20 read_log 192718'9326 (192718'9325) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:211 2014-08-06 15:34:09.368644 2014-08-13 18:06:06.822539 7f5c973a57a0 20 read_log 192718'9327 (192718'9326) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:213 2014-08-06 15:34:09.370781 2014-08-13 18:06:06.822561 7f5c973a57a0 20 read_log 192718'9328 (192718'9327) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:212 2014-08-06 15:34:09.869788 2014-08-13 18:06:06.822574 7f5c973a57a0 20 read_log 192718'9329 (192718'9328) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:214 2014-08-06 15:34:09.938053 2014-08-13 18:06:06.822595 7f5c973a57a0 20 read_log 192718'9330 (192718'9329) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:213 2014-08-06 15:34:10.066025 2014-08-13 18:06:06.822611 7f5c973a57a0 20 read_log 192718'9331 (192718'9330) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:215 2014-08-06 15:34:10.085322 2014-08-13 18:06:06.822624 7f5c973a57a0 20 read_log 192718'9332 (192718'9331) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:214 2014-08-06 15:34:10.605639 2014-08-13 18:06:06.822636 7f5c973a57a0 20 read_log 192718'9333 (192718'9332) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:216 2014-08-06 15:34:10.606044 2014-08-13 18:06:06.822648 7f5c973a57a0 20 read_log 192718'9334 (192718'9333) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:215 2014-08-06 15:34:10.859213 2014-08-13 18:06:06.822660 7f5c973a57a0 20 read_log 192718'9335 (192718'9334) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:217 2014-08-06 15:34:10.859199 2014-08-13 18:06:06.822672 7f5c973a57a0 20 read_log 192718'9336 (192718'9335) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:216 2014-08-06 15:34:11.438948 2014-08-13 18:06:06.822684 7f5c973a57a0 20 read_log 192718'9337 (192718'9336) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:218 2014-08-06 15:34:11.458663 2014-08-13 18:06:06.822696 7f5c973a57a0 20 read_log 192718'9338 (192718'9337) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:217 2014-08-06 15:34:11.723917 2014-08-13 18:06:06.822709 7f5c973a57a0 20 read_log 192718'9339 (192718'9338) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:219 2014-08-06 15:34:11.860683 2014-08-13 18:06:06.822721 7f5c973a57a0 20 read_log 192718'9340 (192718'9339) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:218 2014-08-06 15:34:12.103531 2014-08-13 18:06:06.822733 7f5c973a57a0 20 read_log 192718'9341 (192718'9340) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:219 2014-08-06 15:34:12.336299 2014-08-13 18:06:06.822747 7f5c973a57a0 20 read_log 192718'9342 (192718'9341) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:220 2014-08-06 15:34:12.338248 2014-08-13 18:06:06.822759 7f5c973a57a0 20 read_log 192718'9343 (192718'9342) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:220 2014-08-06 15:34:12.705435 2014-08-13 18:06:06.822771 7f5c973a57a0 20 read_log 192718'9344 (192718'9343) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:221 2014-08-06 15:34:12.730209 2014-08-13 18:06:06.822783 7f5c973a57a0 20 read_log 192718'9345 (192718'9344) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:221 2014-08-06 15:34:12.803448 2014-08-13 18:06:06.822795 7f5c973a57a0 20 read_log 192718'9346 (192718'9345) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:222 2014-08-06 15:34:13.355028 2014-08-13 18:06:06.822807 7f5c973a57a0 20 read_log 192718'9347 (192718'9346) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:222 2014-08-06 15:34:13.375514 2014-08-13 18:06:06.822819 7f5c973a57a0 20 read_log 192718'9348 (192718'9347) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:223 2014-08-06 15:34:13.559349 2014-08-13 18:06:06.822831 7f5c973a57a0 20 read_log 192718'9349 (192718'9348) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:223 2014-08-06 15:34:14.537320 2014-08-13 18:06:06.822843 7f5c973a57a0 20 read_log 192718'9350 (192718'9349) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:224 2014-08-06 15:34:15.022606 2014-08-13 18:06:06.822856 7f5c973a57a0 20 read_log 192718'9351 (192718'9350) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:224 2014-08-06 15:34:15.022523 2014-08-13 18:06:06.822876 7f5c973a57a0 20 read_log 192718'9352 (192718'9351) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:225 2014-08-06 15:34:15.167395 2014-08-13 18:06:06.822891 7f5c973a57a0 20 read_log 192718'9353 (192718'9352) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:225 2014-08-06 15:34:15.167332 2014-08-13 18:06:06.822903 7f5c973a57a0 20 read_log 192718'9354 (192718'9353) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:226 2014-08-06 15:34:15.411682 2014-08-13 18:06:06.822915 7f5c973a57a0 20 read_log 192718'9355 (192718'9354) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:226 2014-08-06 15:34:15.563129 2014-08-13 18:06:06.822929 7f5c973a57a0 20 read_log 192718'9356 (192718'9355) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:227 2014-08-06 15:34:15.981223 2014-08-13 18:06:06.822941 7f5c973a57a0 20 read_log 192718'9357 (192718'9356) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:227 2014-08-06 15:34:16.035424 2014-08-13 18:06:06.822956 7f5c973a57a0 20 read_log 192718'9358 (192718'9357) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:228 2014-08-06 15:34:16.330017 2014-08-13 18:06:06.822968 7f5c973a57a0 20 read_log 192718'9359 (192718'9358) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:228 2014-08-06 15:34:16.329990 2014-08-13 18:06:06.822980 7f5c973a57a0 20 read_log 192718'9360 (192718'9359) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:229 2014-08-06 15:34:16.812061 2014-08-13 18:06:06.822992 7f5c973a57a0 20 read_log 192718'9361 (192718'9360) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:229 2014-08-06 15:34:16.812154 2014-08-13 18:06:06.823006 7f5c973a57a0 20 read_log 192718'9362 (192718'9361) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:230 2014-08-06 15:34:17.483789 2014-08-13 18:06:06.823019 7f5c973a57a0 20 read_log 192718'9363 (192718'9362) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:230 2014-08-06 15:34:17.667190 2014-08-13 18:06:06.823033 7f5c973a57a0 20 read_log 192718'9364 (192718'9363) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:231 2014-08-06 15:34:19.824887 2014-08-13 18:06:06.823045 7f5c973a57a0 20 read_log 192718'9365 (192718'9364) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:231 2014-08-06 15:34:20.308524 2014-08-13 18:06:06.823059 7f5c973a57a0 20 read_log 192718'9366 (192718'9365) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:232 2014-08-06 15:34:20.320008 2014-08-13 18:06:06.823071 7f5c973a57a0 20 read_log 192718'9367 (192718'9366) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:232 2014-08-06 15:34:20.348612 2014-08-13 18:06:06.823085 7f5c973a57a0 20 read_log 192718'9368 (192718'9367) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:233 2014-08-06 15:34:20.669785 2014-08-13 18:06:06.823097 7f5c973a57a0 20 read_log 192718'9369 (192718'9368) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:233 2014-08-06 15:34:20.671873 2014-08-13 18:06:06.823109 7f5c973a57a0 20 read_log 192718'9370 (192718'9369) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:234 2014-08-06 15:34:21.194975 2014-08-13 18:06:06.823123 7f5c973a57a0 20 read_log 192718'9371 (192718'9370) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:234 2014-08-06 15:34:21.440500 2014-08-13 18:06:06.823135 7f5c973a57a0 20 read_log 192718'9372 (192718'9371) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:235 2014-08-06 15:34:21.709092 2014-08-13 18:06:06.823154 7f5c973a57a0 20 read_log 192718'9373 (192718'9372) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:235 2014-08-06 15:34:21.707768 2014-08-13 18:06:06.823169 7f5c973a57a0 20 read_log 192718'9374 (192718'9373) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:236 2014-08-06 15:34:22.049806 2014-08-13 18:06:06.823182 7f5c973a57a0 20 read_log 192718'9375 (192718'9374) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:236 2014-08-06 15:34:22.432285 2014-08-13 18:06:06.823202 7f5c973a57a0 20 read_log 192718'9376 (192718'9375) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:237 2014-08-06 15:34:22.434508 2014-08-13 18:06:06.823236 7f5c973a57a0 20 read_log 192718'9377 (192718'9376) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:237 2014-08-06 15:34:22.678458 2014-08-13 18:06:06.823251 7f5c973a57a0 20 read_log 192718'9378 (192718'9377) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:238 2014-08-06 15:34:22.692091 2014-08-13 18:06:06.823263 7f5c973a57a0 20 read_log 192718'9379 (192718'9378) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:238 2014-08-06 15:34:24.864361 2014-08-13 18:06:06.823281 7f5c973a57a0 20 read_log 192718'9380 (192718'9379) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:239 2014-08-06 15:34:25.389579 2014-08-13 18:06:06.823293 7f5c973a57a0 20 read_log 192718'9381 (192718'9380) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:239 2014-08-06 15:34:25.402166 2014-08-13 18:06:06.823305 7f5c973a57a0 20 read_log 192718'9382 (192718'9381) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:240 2014-08-06 15:34:26.168635 2014-08-13 18:06:06.823320 7f5c973a57a0 20 read_log 192718'9383 (192718'9382) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:240 2014-08-06 15:34:26.168581 2014-08-13 18:06:06.823333 7f5c973a57a0 20 read_log 192718'9384 (192718'9383) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:241 2014-08-06 15:34:26.347523 2014-08-13 18:06:06.823345 7f5c973a57a0 20 read_log 192718'9385 (192718'9384) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:241 2014-08-06 15:34:26.347411 2014-08-13 18:06:06.823357 7f5c973a57a0 20 read_log 192718'9386 (192718'9385) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:242 2014-08-06 15:34:26.598632 2014-08-13 18:06:06.823369 7f5c973a57a0 20 read_log 192718'9388 (192718'9386) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:242 2014-08-06 15:34:26.804029 2014-08-13 18:06:06.823381 7f5c973a57a0 20 read_log 192718'9389 (192718'9388) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:243 2014-08-06 15:34:26.817738 2014-08-13 18:06:06.823402 7f5c973a57a0 20 read_log 192718'9390 (192718'9389) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:243 2014-08-06 15:34:26.845248 2014-08-13 18:06:06.823423 7f5c973a57a0 20 read_log 192718'9391 (192718'9390) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:244 2014-08-06 15:34:27.104873 2014-08-13 18:06:06.823435 7f5c973a57a0 20 read_log 192718'9392 (192718'9391) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:244 2014-08-06 15:34:27.104927 2014-08-13 18:06:06.823455 7f5c973a57a0 20 read_log 192718'9393 (192718'9392) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:245 2014-08-06 15:34:27.304309 2014-08-13 18:06:06.823470 7f5c973a57a0 20 read_log 192718'9394 (192718'9393) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:245 2014-08-06 15:34:27.304330 2014-08-13 18:06:06.823483 7f5c973a57a0 20 read_log 192718'9395 (192718'9394) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:246 2014-08-06 15:34:27.344086 2014-08-13 18:06:06.823506 7f5c973a57a0 20 read_log 192718'9396 (192718'9395) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:246 2014-08-06 15:34:27.383166 2014-08-13 18:06:06.823522 7f5c973a57a0 20 read_log 192718'9397 (192718'9396) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:247 2014-08-06 15:34:27.406323 2014-08-13 18:06:06.823534 7f5c973a57a0 20 read_log 192718'9398 (192718'9397) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:247 2014-08-06 15:34:27.787451 2014-08-13 18:06:06.823556 7f5c973a57a0 20 read_log 192718'9399 (192718'9398) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:248 2014-08-06 15:34:27.797634 2014-08-13 18:06:06.823569 7f5c973a57a0 20 read_log 192718'9400 (192718'9399) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:249 2014-08-06 15:34:28.050031 2014-08-13 18:06:06.823607 7f5c973a57a0 20 read_log 192718'9401 (192718'9400) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:248 2014-08-06 15:34:28.048855 2014-08-13 18:06:06.823623 7f5c973a57a0 20 read_log 192718'9402 (192718'9401) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:250 2014-08-06 15:34:28.159440 2014-08-13 18:06:06.823645 7f5c973a57a0 20 read_log 192718'9403 (192718'9402) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:249 2014-08-06 15:34:28.337805 2014-08-13 18:06:06.823661 7f5c973a57a0 20 read_log 192718'9404 (192718'9403) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:251 2014-08-06 15:34:28.339697 2014-08-13 18:06:06.823673 7f5c973a57a0 20 read_log 192718'9405 (192718'9404) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:250 2014-08-06 15:34:28.549405 2014-08-13 18:06:06.823692 7f5c973a57a0 20 read_log 192718'9406 (192718'9405) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:252 2014-08-06 15:34:28.550392 2014-08-13 18:06:06.823707 7f5c973a57a0 20 read_log 192718'9407 (192718'9406) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:251 2014-08-06 15:34:29.042621 2014-08-13 18:06:06.823719 7f5c973a57a0 20 read_log 192718'9408 (192718'9407) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:253 2014-08-06 15:34:29.045759 2014-08-13 18:06:06.823731 7f5c973a57a0 20 read_log 192718'9409 (192718'9408) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:252 2014-08-06 15:34:29.071606 2014-08-13 18:06:06.823744 7f5c973a57a0 20 read_log 192718'9410 (192718'9409) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:254 2014-08-06 15:34:29.098234 2014-08-13 18:06:06.823756 7f5c973a57a0 20 read_log 192718'9411 (192718'9410) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:253 2014-08-06 15:34:29.155804 2014-08-13 18:06:06.823768 7f5c973a57a0 20 read_log 192718'9412 (192718'9411) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:254 2014-08-06 15:34:29.321339 2014-08-13 18:06:06.823780 7f5c973a57a0 20 read_log 192718'9413 (192718'9412) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:255 2014-08-06 15:34:29.329220 2014-08-13 18:06:06.823792 7f5c973a57a0 20 read_log 192718'9414 (192718'9413) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:255 2014-08-06 15:34:29.526970 2014-08-13 18:06:06.823804 7f5c973a57a0 20 read_log 192718'9415 (192718'9414) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:256 2014-08-06 15:34:29.528647 2014-08-13 18:06:06.823816 7f5c973a57a0 20 read_log 192718'9416 (192718'9415) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:256 2014-08-06 15:34:30.017798 2014-08-13 18:06:06.823829 7f5c973a57a0 20 read_log 192718'9417 (192718'9416) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:257 2014-08-06 15:34:30.019104 2014-08-13 18:06:06.823852 7f5c973a57a0 20 read_log 192718'9418 (192718'9417) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:257 2014-08-06 15:34:30.060530 2014-08-13 18:06:06.823865 7f5c973a57a0 20 read_log 192718'9419 (192718'9418) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:258 2014-08-06 15:34:30.103520 2014-08-13 18:06:06.823878 7f5c973a57a0 20 read_log 192718'9420 (192718'9419) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:258 2014-08-06 15:34:30.132825 2014-08-13 18:06:06.823890 7f5c973a57a0 20 read_log 192718'9421 (192718'9420) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:259 2014-08-06 15:34:30.298001 2014-08-13 18:06:06.823902 7f5c973a57a0 20 read_log 192718'9422 (192718'9421) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:259 2014-08-06 15:34:30.299139 2014-08-13 18:06:06.823915 7f5c973a57a0 20 read_log 192718'9423 (192718'9422) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:260 2014-08-06 15:34:30.516436 2014-08-13 18:06:06.823927 7f5c973a57a0 20 read_log 192718'9424 (192718'9423) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:260 2014-08-06 15:34:30.517083 2014-08-13 18:06:06.823954 7f5c973a57a0 20 read_log 192718'9425 (192718'9424) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:261 2014-08-06 15:34:30.850372 2014-08-13 18:06:06.823970 7f5c973a57a0 20 read_log 192718'9426 (192718'9425) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:261 2014-08-06 15:34:30.850057 2014-08-13 18:06:06.823982 7f5c973a57a0 20 read_log 192718'9427 (192718'9426) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:262 2014-08-06 15:34:31.045738 2014-08-13 18:06:06.823994 7f5c973a57a0 20 read_log 192718'9428 (192718'9427) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:262 2014-08-06 15:34:31.090955 2014-08-13 18:06:06.824006 7f5c973a57a0 20 read_log 192718'9429 (192718'9428) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:263 2014-08-06 15:34:31.280619 2014-08-13 18:06:06.824018 7f5c973a57a0 20 read_log 192718'9430 (192718'9429) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:263 2014-08-06 15:34:31.280380 2014-08-13 18:06:06.824030 7f5c973a57a0 20 read_log 192718'9431 (192718'9430) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:264 2014-08-06 15:34:31.650949 2014-08-13 18:06:06.824042 7f5c973a57a0 20 read_log 192718'9432 (192718'9431) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:264 2014-08-06 15:34:31.653118 2014-08-13 18:06:06.824054 7f5c973a57a0 20 read_log 192718'9433 (192718'9432) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:265 2014-08-06 15:34:31.976817 2014-08-13 18:06:06.824066 7f5c973a57a0 20 read_log 192718'9434 (192718'9433) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:265 2014-08-06 15:34:31.976930 2014-08-13 18:06:06.824079 7f5c973a57a0 20 read_log 192718'9435 (192718'9434) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:266 2014-08-06 15:34:32.005079 2014-08-13 18:06:06.824373 7f5c973a57a0 20 read_log 192718'9436 (192718'9435) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:266 2014-08-06 15:34:32.007062 2014-08-13 18:06:06.824392 7f5c973a57a0 20 read_log 192718'9437 (192718'9436) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:267 2014-08-06 15:34:32.032314 2014-08-13 18:06:06.824405 7f5c973a57a0 20 read_log 192718'9438 (192718'9437) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:267 2014-08-06 15:34:32.033584 2014-08-13 18:06:06.824419 7f5c973a57a0 20 read_log 192718'9439 (192718'9438) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:268 2014-08-06 15:34:32.074160 2014-08-13 18:06:06.824444 7f5c973a57a0 20 read_log 192718'9440 (192718'9439) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:268 2014-08-06 15:34:32.074144 2014-08-13 18:06:06.824457 7f5c973a57a0 20 read_log 192718'9441 (192718'9440) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:269 2014-08-06 15:34:32.099869 2014-08-13 18:06:06.824471 7f5c973a57a0 20 read_log 192718'9442 (192718'9441) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:269 2014-08-06 15:34:32.101380 2014-08-13 18:06:06.824484 7f5c973a57a0 20 read_log 192718'9443 (192718'9442) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:270 2014-08-06 15:34:32.138885 2014-08-13 18:06:06.824502 7f5c973a57a0 20 read_log 192718'9444 (192718'9443) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:270 2014-08-06 15:34:32.315720 2014-08-13 18:06:06.824514 7f5c973a57a0 20 read_log 192718'9445 (192718'9444) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:271 2014-08-06 15:34:32.360034 2014-08-13 18:06:06.824526 7f5c973a57a0 20 read_log 192718'9446 (192718'9445) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:271 2014-08-06 15:34:32.430779 2014-08-13 18:06:06.824542 7f5c973a57a0 20 read_log 192718'9447 (192718'9446) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:272 2014-08-06 15:34:32.565501 2014-08-13 18:06:06.824557 7f5c973a57a0 20 read_log 192718'9448 (192718'9447) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:272 2014-08-06 15:34:32.616932 2014-08-13 18:06:06.824593 7f5c973a57a0 20 read_log 192718'9449 (192718'9448) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:273 2014-08-06 15:34:32.779821 2014-08-13 18:06:06.824609 7f5c973a57a0 20 read_log 192718'9450 (192718'9449) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:273 2014-08-06 15:34:32.810195 2014-08-13 18:06:06.824629 7f5c973a57a0 20 read_log 192718'9451 (192718'9450) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:274 2014-08-06 15:34:33.181793 2014-08-13 18:06:06.824645 7f5c973a57a0 20 read_log 192718'9452 (192718'9451) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:274 2014-08-06 15:34:33.203156 2014-08-13 18:06:06.824657 7f5c973a57a0 20 read_log 192718'9453 (192718'9452) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:275 2014-08-06 15:34:33.236098 2014-08-13 18:06:06.824669 7f5c973a57a0 20 read_log 192718'9454 (192718'9453) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:275 2014-08-06 15:34:33.243098 2014-08-13 18:06:06.824682 7f5c973a57a0 20 read_log 192718'9455 (192718'9454) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:276 2014-08-06 15:34:33.314795 2014-08-13 18:06:06.824694 7f5c973a57a0 20 read_log 192718'9456 (192718'9455) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:276 2014-08-06 15:34:33.450532 2014-08-13 18:06:06.824715 7f5c973a57a0 20 read_log 192718'9457 (192718'9456) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:277 2014-08-06 15:34:33.452455 2014-08-13 18:06:06.824728 7f5c973a57a0 20 read_log 192718'9458 (192718'9457) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:277 2014-08-06 15:34:33.669689 2014-08-13 18:06:06.824740 7f5c973a57a0 20 read_log 192718'9459 (192718'9458) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:278 2014-08-06 15:34:33.683699 2014-08-13 18:06:06.824752 7f5c973a57a0 20 read_log 192718'9460 (192718'9459) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:278 2014-08-06 15:34:33.697681 2014-08-13 18:06:06.824770 7f5c973a57a0 20 read_log 192718'9461 (192718'9460) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:279 2014-08-06 15:34:33.716216 2014-08-13 18:06:06.824783 7f5c973a57a0 20 read_log 192718'9462 (192718'9461) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:279 2014-08-06 15:34:33.721828 2014-08-13 18:06:06.824795 7f5c973a57a0 20 read_log 192718'9463 (192718'9462) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:280 2014-08-06 15:34:33.754244 2014-08-13 18:06:06.824807 7f5c973a57a0 20 read_log 192718'9464 (192718'9463) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:280 2014-08-06 15:34:33.754951 2014-08-13 18:06:06.824819 7f5c973a57a0 20 read_log 192718'9465 (192718'9464) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:281 2014-08-06 15:34:33.895576 2014-08-13 18:06:06.824831 7f5c973a57a0 20 read_log 192718'9466 (192718'9465) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:281 2014-08-06 15:34:34.134558 2014-08-13 18:06:06.824843 7f5c973a57a0 20 read_log 192718'9467 (192718'9466) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:282 2014-08-06 15:34:34.136568 2014-08-13 18:06:06.824855 7f5c973a57a0 20 read_log 192718'9468 (192718'9467) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:282 2014-08-06 15:34:34.432043 2014-08-13 18:06:06.824879 7f5c973a57a0 20 read_log 192718'9469 (192718'9468) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:283 2014-08-06 15:34:34.485561 2014-08-13 18:06:06.824891 7f5c973a57a0 20 read_log 192718'9470 (192718'9469) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:283 2014-08-06 15:34:34.635547 2014-08-13 18:06:06.824953 7f5c973a57a0 20 read_log 192718'9471 (192718'9470) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:284 2014-08-06 15:34:34.970708 2014-08-13 18:06:06.824968 7f5c973a57a0 20 read_log 192718'9472 (192718'9471) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:284 2014-08-06 15:34:35.127758 2014-08-13 18:06:06.824990 7f5c973a57a0 20 read_log 192718'9473 (192718'9472) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:285 2014-08-06 15:34:35.413390 2014-08-13 18:06:06.825005 7f5c973a57a0 20 read_log 192718'9474 (192718'9473) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:285 2014-08-06 15:34:35.413442 2014-08-13 18:06:06.825017 7f5c973a57a0 20 read_log 192718'9475 (192718'9474) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:286 2014-08-06 15:34:35.484512 2014-08-13 18:06:06.825033 7f5c973a57a0 20 read_log 192718'9476 (192718'9475) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:286 2014-08-06 15:34:35.484663 2014-08-13 18:06:06.825046 7f5c973a57a0 20 read_log 192718'9477 (192718'9476) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:287 2014-08-06 15:34:35.533701 2014-08-13 18:06:06.825058 7f5c973a57a0 20 read_log 192718'9478 (192718'9477) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:287 2014-08-06 15:34:35.703432 2014-08-13 18:06:06.825070 7f5c973a57a0 20 read_log 192718'9479 (192718'9478) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:288 2014-08-06 15:34:35.956280 2014-08-13 18:06:06.825082 7f5c973a57a0 20 read_log 192718'9480 (192718'9479) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:288 2014-08-06 15:34:36.162178 2014-08-13 18:06:06.825106 7f5c973a57a0 20 read_log 192718'9481 (192718'9480) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:289 2014-08-06 15:34:36.189673 2014-08-13 18:06:06.825119 7f5c973a57a0 20 read_log 192718'9482 (192718'9481) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:289 2014-08-06 15:34:36.201506 2014-08-13 18:06:06.825131 7f5c973a57a0 20 read_log 192718'9483 (192718'9482) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:290 2014-08-06 15:34:36.219218 2014-08-13 18:06:06.825162 7f5c973a57a0 20 read_log 192718'9484 (192718'9483) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:290 2014-08-06 15:34:36.262544 2014-08-13 18:06:06.825177 7f5c973a57a0 20 read_log 192718'9485 (192718'9484) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:291 2014-08-06 15:34:36.469422 2014-08-13 18:06:06.825190 7f5c973a57a0 20 read_log 192718'9486 (192718'9485) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:291 2014-08-06 15:34:36.468418 2014-08-13 18:06:06.825202 7f5c973a57a0 20 read_log 192718'9487 (192718'9486) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:292 2014-08-06 15:34:36.654039 2014-08-13 18:06:06.825215 7f5c973a57a0 20 read_log 192718'9488 (192718'9487) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:292 2014-08-06 15:34:36.653427 2014-08-13 18:06:06.825232 7f5c973a57a0 20 read_log 192718'9489 (192718'9488) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:293 2014-08-06 15:34:36.683891 2014-08-13 18:06:06.825246 7f5c973a57a0 20 read_log 192718'9490 (192718'9489) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:293 2014-08-06 15:34:36.687012 2014-08-13 18:06:06.825259 7f5c973a57a0 20 read_log 192718'9491 (192718'9490) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:294 2014-08-06 15:34:37.261459 2014-08-13 18:06:06.825271 7f5c973a57a0 20 read_log 192718'9492 (192718'9491) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:294 2014-08-06 15:34:37.262969 2014-08-13 18:06:06.825286 7f5c973a57a0 20 read_log 192718'9493 (192718'9492) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:295 2014-08-06 15:34:37.293606 2014-08-13 18:06:06.825300 7f5c973a57a0 20 read_log 192718'9494 (192718'9493) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:295 2014-08-06 15:34:37.296923 2014-08-13 18:06:06.825312 7f5c973a57a0 20 read_log 192718'9495 (192718'9494) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:296 2014-08-06 15:34:37.418449 2014-08-13 18:06:06.825324 7f5c973a57a0 20 read_log 192718'9496 (192718'9495) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:296 2014-08-06 15:34:37.444352 2014-08-13 18:06:06.825358 7f5c973a57a0 20 read_log 192718'9497 (192718'9496) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:297 2014-08-06 15:34:37.475696 2014-08-13 18:06:06.825374 7f5c973a57a0 20 read_log 192718'9498 (192718'9497) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:297 2014-08-06 15:34:37.493942 2014-08-13 18:06:06.825386 7f5c973a57a0 20 read_log 192718'9499 (192718'9498) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:298 2014-08-06 15:34:37.662732 2014-08-13 18:06:06.825399 7f5c973a57a0 20 read_log 192718'9500 (192718'9499) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:298 2014-08-06 15:34:37.898344 2014-08-13 18:06:06.825411 7f5c973a57a0 20 read_log 192718'9501 (192718'9500) modify 1a453a0e/core.19866+13+2/head//24 by client.594099.0:299 2014-08-06 15:34:38.032839 2014-08-13 18:06:06.825423 7f5c973a57a0 20 read_log 192718'9502 (192718'9501) modify 1a453a0e/core.19866+13+2/head//24 by client.417577.0:299 2014-08-06 15:34:38.033089 2014-08-13 18:06:06.825435 7f5c973a57a0 20 read_log 192788'9506 (192718'9502) clean 1a453a0e/core.19866+13+2/head//24 by osd.367.0:43043 2014-08-11 10:23:49.050751 2014-08-13 18:06:06.825447 7f5c973a57a0 20 read_log 192798'9507 (192788'9506) delete 1a453a0e/core.19866+13+2/head//24 by osd.367.0:43044 2014-08-11 10:24:43.632401 2014-08-13 18:06:06.825460 7f5c973a57a0 20 read_log 192834'9835 (0'0) promote b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by osd.367.0:43700 2014-08-11 15:23:41.427154 2014-08-13 18:06:06.825472 7f5c973a57a0 20 read_log 192834'9836 (192834'9835) modify b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.933448.0:1474 2014-08-11 15:23:41.369678 2014-08-13 18:06:06.825485 7f5c973a57a0 20 read_log 192834'9913 (0'0) promote 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.367.0:43994 2014-08-11 15:24:12.512128 2014-08-13 18:06:06.825497 7f5c973a57a0 20 read_log 192834'9914 (192834'9913) modify 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by client.933448.0:8051 2014-08-11 15:24:12.503712 2014-08-13 18:06:06.825510 7f5c973a57a0 20 read_log 192834'9915 (0'0) promote b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by osd.367.0:44002 2014-08-11 15:24:13.338244 2014-08-13 18:06:06.825522 7f5c973a57a0 20 read_log 192834'9916 (192834'9915) modify b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.933448.0:8261 2014-08-11 15:24:13.330648 2014-08-13 18:06:06.825534 7f5c973a57a0 20 read_log 192834'9945 (0'0) promote 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by osd.367.0:44110 2014-08-11 15:24:23.319699 2014-08-13 18:06:06.825546 7f5c973a57a0 20 read_log 192834'9946 (192834'9945) modify 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.933448.0:10457 2014-08-11 15:24:23.311353 2014-08-13 18:06:06.825559 7f5c973a57a0 20 read_log 192834'9951 (0'0) promote eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by osd.367.0:44128 2014-08-11 15:24:26.987126 2014-08-13 18:06:06.825572 7f5c973a57a0 20 read_log 192834'9952 (192834'9951) modify eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.933448.0:11245 2014-08-11 15:24:26.980403 2014-08-13 18:06:06.825584 7f5c973a57a0 20 read_log 192834'9959 (0'0) promote 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by osd.367.0:44146 2014-08-11 15:24:28.453759 2014-08-13 18:06:06.825596 7f5c973a57a0 20 read_log 192834'9960 (192834'9959) modify 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.933448.0:11590 2014-08-11 15:24:28.444246 2014-08-13 18:06:06.825609 7f5c973a57a0 20 read_log 192834'9969 (0'0) promote a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.367.0:44178 2014-08-11 15:24:31.639462 2014-08-13 18:06:06.825628 7f5c973a57a0 20 read_log 192834'9970 (192834'9969) modify a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by client.933448.0:12246 2014-08-11 15:24:31.632594 2014-08-13 18:06:06.825643 7f5c973a57a0 20 read_log 192834'9973 (0'0) promote 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.367.0:44186 2014-08-11 15:24:32.581542 2014-08-13 18:06:06.825655 7f5c973a57a0 20 read_log 192834'9974 (192834'9973) modify 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by client.933448.0:12497 2014-08-11 15:24:32.572392 2014-08-13 18:06:06.825668 7f5c973a57a0 20 read_log 192834'10007 (0'0) promote 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by osd.367.0:44275 2014-08-11 15:24:41.807564 2014-08-13 18:06:06.825682 7f5c973a57a0 20 read_log 192834'10008 (0'0) promote 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by osd.367.0:44276 2014-08-11 15:24:41.809934 2014-08-13 18:06:06.825695 7f5c973a57a0 20 read_log 192834'10009 (192834'10008) modify 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.933448.0:14137 2014-08-11 15:24:41.553448 2014-08-13 18:06:06.825710 7f5c973a57a0 20 read_log 192834'10010 (192834'10007) modify 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.933448.0:14074 2014-08-11 15:24:41.040219 2014-08-13 18:06:06.825722 7f5c973a57a0 20 read_log 192834'10031 (0'0) promote 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by osd.367.0:44417 2014-08-11 15:24:51.989709 2014-08-13 18:06:06.825734 7f5c973a57a0 20 read_log 192834'10032 (192834'10031) modify 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.933448.0:15949 2014-08-11 15:24:51.981475 2014-08-13 18:06:06.825749 7f5c973a57a0 20 read_log 192834'10041 (0'0) promote b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by osd.367.0:44451 2014-08-11 15:24:56.754712 2014-08-13 18:06:06.825763 7f5c973a57a0 20 read_log 192834'10042 (192834'10041) modify b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.933448.0:16700 2014-08-11 15:24:56.747312 2014-08-13 18:06:06.825775 7f5c973a57a0 20 read_log 192834'10049 (0'0) promote a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by osd.367.0:44475 2014-08-11 15:24:59.246609 2014-08-13 18:06:06.825789 7f5c973a57a0 20 read_log 192834'10050 (192834'10049) modify a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.933448.0:17317 2014-08-11 15:24:59.239737 2014-08-13 18:06:06.825802 7f5c973a57a0 20 read_log 192834'10053 (0'0) promote ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by osd.367.0:44504 2014-08-11 15:25:00.769563 2014-08-13 18:06:06.825816 7f5c973a57a0 20 read_log 192834'10054 (192834'10053) modify ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.933448.0:17619 2014-08-11 15:25:00.759783 2014-08-13 18:06:06.825829 7f5c973a57a0 20 read_log 192834'10061 (0'0) promote 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by osd.367.0:44520 2014-08-11 15:25:03.090170 2014-08-13 18:06:06.825841 7f5c973a57a0 20 read_log 192834'10062 (192834'10061) modify 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.933448.0:18021 2014-08-11 15:25:03.082182 2014-08-13 18:06:06.825855 7f5c973a57a0 20 read_log 192834'10073 (0'0) promote ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by osd.367.0:44548 2014-08-11 15:25:05.509993 2014-08-13 18:06:06.825868 7f5c973a57a0 20 read_log 192834'10074 (192834'10073) modify ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.933448.0:18490 2014-08-11 15:25:05.503239 2014-08-13 18:06:06.825880 7f5c973a57a0 20 read_log 192834'10087 (0'0) promote c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.367.0:44597 2014-08-11 15:25:09.187315 2014-08-13 18:06:06.825901 7f5c973a57a0 20 read_log 192834'10088 (192834'10087) modify c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by client.933448.0:19226 2014-08-11 15:25:09.179086 2014-08-13 18:06:06.825916 7f5c973a57a0 20 read_log 192834'10107 (0'0) promote 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by osd.367.0:44684 2014-08-11 15:25:20.308380 2014-08-13 18:06:06.825929 7f5c973a57a0 20 read_log 192834'10108 (192834'10107) modify 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.933448.0:20323 2014-08-11 15:25:20.299110 2014-08-13 18:06:06.825943 7f5c973a57a0 20 read_log 192834'10109 (0'0) promote 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.367.0:44686 2014-08-11 15:25:20.685775 2014-08-13 18:06:06.825956 7f5c973a57a0 20 read_log 192834'10110 (192834'10109) modify 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by client.933448.0:20376 2014-08-11 15:25:20.678929 2014-08-13 18:06:06.825970 7f5c973a57a0 20 read_log 192834'10127 (0'0) promote 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.367.0:44747 2014-08-11 15:25:25.904854 2014-08-13 18:06:06.825983 7f5c973a57a0 20 read_log 192834'10128 (192834'10127) modify 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by client.933448.0:21044 2014-08-11 15:25:25.896007 2014-08-13 18:06:06.825995 7f5c973a57a0 20 read_log 192834'10131 (0'0) promote b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by osd.367.0:44784 2014-08-11 15:25:31.102596 2014-08-13 18:06:06.826008 7f5c973a57a0 20 read_log 192834'10132 (192834'10131) modify b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.933448.0:21580 2014-08-11 15:25:31.095215 2014-08-13 18:06:06.826023 7f5c973a57a0 20 read_log 192834'10153 (192834'9836) modify b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.933448.0:23765 2014-08-11 15:25:39.807801 2014-08-13 18:06:06.826035 7f5c973a57a0 20 read_log 192834'10191 (192834'9914) modify 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by client.933448.0:30342 2014-08-11 15:25:48.760307 2014-08-13 18:06:06.826050 7f5c973a57a0 20 read_log 192834'10192 (192834'9916) modify b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.933448.0:30552 2014-08-11 15:25:48.980703 2014-08-13 18:06:06.826062 7f5c973a57a0 20 read_log 192834'10207 (192834'9946) modify 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.933448.0:32748 2014-08-11 15:25:52.333271 2014-08-13 18:06:06.826077 7f5c973a57a0 20 read_log 192834'10210 (192834'9952) modify eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.933448.0:33536 2014-08-11 15:25:53.645114 2014-08-13 18:06:06.826091 7f5c973a57a0 20 read_log 192834'10214 (192834'9960) modify 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.933448.0:33881 2014-08-11 15:25:54.329367 2014-08-13 18:06:06.826104 7f5c973a57a0 20 read_log 192834'10219 (192834'9970) modify a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by client.933448.0:34537 2014-08-11 15:25:55.141588 2014-08-13 18:06:06.826118 7f5c973a57a0 20 read_log 192834'10221 (192834'9974) modify 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by client.933448.0:34788 2014-08-11 15:25:55.429979 2014-08-13 18:06:06.826130 7f5c973a57a0 20 read_log 192834'10238 (192834'10010) modify 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.933448.0:36365 2014-08-11 15:25:57.612388 2014-08-13 18:06:06.826151 7f5c973a57a0 20 read_log 192834'10239 (192834'10009) modify 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.933448.0:36428 2014-08-11 15:25:57.721027 2014-08-13 18:06:06.826166 7f5c973a57a0 20 read_log 192834'10250 (192834'10032) modify 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.933448.0:38240 2014-08-11 15:26:00.688589 2014-08-13 18:06:06.826204 7f5c973a57a0 20 read_log 192834'10255 (192834'10042) modify b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.933448.0:38991 2014-08-11 15:26:01.456353 2014-08-13 18:06:06.826220 7f5c973a57a0 20 read_log 192834'10259 (192834'10050) modify a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.933448.0:39608 2014-08-11 15:26:01.993612 2014-08-13 18:06:06.826232 7f5c973a57a0 20 read_log 192834'10261 (192834'10054) modify ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.933448.0:39910 2014-08-11 15:26:02.164784 2014-08-13 18:06:06.826247 7f5c973a57a0 20 read_log 192834'10265 (192834'10062) modify 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.933448.0:40312 2014-08-11 15:26:02.564529 2014-08-13 18:06:06.826259 7f5c973a57a0 20 read_log 192834'10271 (192834'10074) modify ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.933448.0:40781 2014-08-11 15:26:02.729403 2014-08-13 18:06:06.826277 7f5c973a57a0 20 read_log 192834'10278 (192834'10088) modify c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by client.933448.0:41517 2014-08-11 15:26:02.968215 2014-08-13 18:06:06.826306 7f5c973a57a0 20 read_log 192834'10288 (192834'10108) modify 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.933448.0:42614 2014-08-11 15:26:03.332409 2014-08-13 18:06:06.826321 7f5c973a57a0 20 read_log 192834'10289 (192834'10110) modify 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by client.933448.0:42667 2014-08-11 15:26:03.347371 2014-08-13 18:06:06.826333 7f5c973a57a0 20 read_log 192834'10298 (192834'10128) modify 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by client.933448.0:43335 2014-08-11 15:26:03.581678 2014-08-13 18:06:06.826352 7f5c973a57a0 20 read_log 192834'10300 (192834'10132) modify b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.933448.0:43871 2014-08-11 15:26:03.749194 2014-08-13 18:06:06.826365 7f5c973a57a0 20 read_log 192834'10324 (0'0) promote 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by osd.367.0:45670 2014-08-11 15:26:18.230299 2014-08-13 18:06:06.826377 7f5c973a57a0 20 read_log 192834'10325 (192834'10324) modify 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.1883634.0:1028 2014-08-11 15:26:18.222005 2014-08-13 18:06:06.826398 7f5c973a57a0 20 read_log 192834'10338 (0'0) promote 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.367.0:45704 2014-08-11 15:26:21.772016 2014-08-13 18:06:06.826415 7f5c973a57a0 20 read_log 192834'10339 (192834'10338) modify 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by client.1883634.0:1976 2014-08-11 15:26:21.763612 2014-08-13 18:06:06.826431 7f5c973a57a0 20 read_log 192834'10342 (0'0) promote 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.367.0:45722 2014-08-11 15:26:22.671660 2014-08-13 18:06:06.826454 7f5c973a57a0 20 read_log 192834'10343 (192834'10342) modify 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by client.1883634.0:2218 2014-08-11 15:26:22.664375 2014-08-13 18:06:06.826467 7f5c973a57a0 20 read_log 192834'10350 (0'0) promote 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by osd.367.0:45749 2014-08-11 15:26:25.003570 2014-08-13 18:06:06.826479 7f5c973a57a0 20 read_log 192834'10351 (192834'10350) modify 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.1883634.0:2869 2014-08-11 15:26:24.986652 2014-08-13 18:06:06.826501 7f5c973a57a0 20 read_log 192834'10364 (0'0) promote fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.367.0:45812 2014-08-11 15:26:30.620693 2014-08-13 18:06:06.826520 7f5c973a57a0 20 read_log 192834'10365 (192834'10364) modify fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by client.1883634.0:4458 2014-08-11 15:26:30.613197 2014-08-13 18:06:06.826550 7f5c973a57a0 20 read_log 192834'10390 (0'0) promote 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by osd.367.0:45886 2014-08-11 15:26:36.868098 2014-08-13 18:06:06.826567 7f5c973a57a0 20 read_log 192834'10391 (192834'10390) modify 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.1883634.0:6128 2014-08-11 15:26:36.859905 2014-08-13 18:06:06.826579 7f5c973a57a0 20 read_log 192834'10410 (0'0) promote 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by osd.367.0:45981 2014-08-11 15:26:44.861499 2014-08-13 18:06:06.826592 7f5c973a57a0 20 read_log 192834'10411 (192834'10410) modify 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.1883634.0:8072 2014-08-11 15:26:44.854070 2014-08-13 18:06:06.826604 7f5c973a57a0 20 read_log 192834'10412 (0'0) promote 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by osd.367.0:45985 2014-08-11 15:26:45.140993 2014-08-13 18:06:06.826616 7f5c973a57a0 20 read_log 192834'10413 (192834'10412) modify 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.1883634.0:8145 2014-08-11 15:26:45.133317 2014-08-13 18:06:06.826637 7f5c973a57a0 20 read_log 192834'10442 (0'0) promote c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by osd.367.0:46189 2014-08-11 15:26:53.006762 2014-08-13 18:06:06.826650 7f5c973a57a0 20 read_log 192834'10443 (192834'10442) modify c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.1883634.0:9935 2014-08-11 15:26:52.998361 2014-08-13 18:06:06.826662 7f5c973a57a0 20 read_log 192834'10464 (0'0) promote c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.367.0:46331 2014-08-11 15:27:01.171999 2014-08-13 18:06:06.826675 7f5c973a57a0 20 read_log 192834'10465 (192834'10464) modify c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by client.1883634.0:11751 2014-08-11 15:27:01.163584 2014-08-13 18:06:06.826687 7f5c973a57a0 20 read_log 192834'10466 (0'0) promote 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.367.0:46333 2014-08-11 15:27:01.278361 2014-08-13 18:06:06.826703 7f5c973a57a0 20 read_log 192834'10467 (192834'10466) modify 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by client.1883634.0:11787 2014-08-11 15:27:01.271808 2014-08-13 18:06:06.826718 7f5c973a57a0 20 read_log 192834'10478 (0'0) promote d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.367.0:46434 2014-08-11 15:27:03.766805 2014-08-13 18:06:06.826730 7f5c973a57a0 20 read_log 192834'10479 (192834'10478) modify d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by client.1883634.0:12381 2014-08-11 15:27:03.759006 2014-08-13 18:06:06.826742 7f5c973a57a0 20 read_log 192834'10512 (0'0) promote 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by osd.367.0:46564 2014-08-11 15:27:09.966075 2014-08-13 18:06:06.826755 7f5c973a57a0 20 read_log 192834'10513 (192834'10512) modify 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.1883634.0:13522 2014-08-11 15:27:09.958549 2014-08-13 18:06:06.826767 7f5c973a57a0 20 read_log 192834'10532 (0'0) promote c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by osd.367.0:46616 2014-08-11 15:27:15.831369 2014-08-13 18:06:06.826780 7f5c973a57a0 20 read_log 192834'10533 (192834'10532) modify c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.1883634.0:14880 2014-08-11 15:27:15.822132 2014-08-13 18:06:06.826792 7f5c973a57a0 20 read_log 192834'10546 (0'0) promote a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by osd.367.0:46636 2014-08-11 15:27:17.403151 2014-08-13 18:06:06.826804 7f5c973a57a0 20 read_log 192834'10547 (192834'10546) modify a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.1883634.0:15165 2014-08-11 15:27:17.394485 2014-08-13 18:06:06.826824 7f5c973a57a0 20 read_log 192834'10554 (0'0) promote abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by osd.367.0:46696 2014-08-11 15:27:21.299461 2014-08-13 18:06:06.826839 7f5c973a57a0 20 read_log 192834'10555 (192834'10554) modify abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.1883634.0:15708 2014-08-11 15:27:21.292173 2014-08-13 18:06:06.826862 7f5c973a57a0 20 read_log 192834'10556 (0'0) promote 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by osd.367.0:46706 2014-08-11 15:27:24.871224 2014-08-13 18:06:06.826875 7f5c973a57a0 20 read_log 192834'10557 (192834'10556) modify 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.1883634.0:15959 2014-08-11 15:27:24.861453 2014-08-13 18:06:06.826888 7f5c973a57a0 20 read_log 192834'10578 (0'0) promote 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.367.0:46797 2014-08-11 15:27:34.317427 2014-08-13 18:06:06.826900 7f5c973a57a0 20 read_log 192834'10579 (192834'10578) modify 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by client.1883634.0:17330 2014-08-11 15:27:34.308390 2014-08-13 18:06:06.826912 7f5c973a57a0 20 read_log 192834'10594 (0'0) promote 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by osd.367.0:46861 2014-08-11 15:27:40.508954 2014-08-13 18:06:06.826924 7f5c973a57a0 20 read_log 192834'10595 (192834'10594) modify 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.1883634.0:18039 2014-08-11 15:27:40.500597 2014-08-13 18:06:06.826936 7f5c973a57a0 20 read_log 192834'10618 (0'0) promote 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.367.0:46928 2014-08-11 15:27:47.976828 2014-08-13 18:06:06.826948 7f5c973a57a0 20 read_log 192834'10619 (192834'10618) modify 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by client.1883634.0:18877 2014-08-11 15:27:47.969599 2014-08-13 18:06:06.826961 7f5c973a57a0 20 read_log 192834'10674 (0'0) promote 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.367.0:47146 2014-08-11 15:28:04.960476 2014-08-13 18:06:06.826973 7f5c973a57a0 20 read_log 192834'10675 (192834'10674) modify 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by client.1883634.0:21686 2014-08-11 15:28:04.953091 2014-08-13 18:06:06.826985 7f5c973a57a0 20 read_log 192834'10676 (0'0) promote 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by osd.367.0:47176 2014-08-11 15:28:06.660812 2014-08-13 18:06:06.826997 7f5c973a57a0 20 read_log 192834'10677 (192834'10676) modify 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.1883634.0:22092 2014-08-11 15:28:06.651135 2014-08-13 18:06:06.827009 7f5c973a57a0 20 read_log 192834'10741 (192834'10325) modify 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.1883634.0:24352 2014-08-11 15:28:18.931915 2014-08-13 18:06:06.827022 7f5c973a57a0 20 read_log 192834'10749 (192834'10339) modify 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by client.1883634.0:25300 2014-08-11 15:28:20.474491 2014-08-13 18:06:06.827034 7f5c973a57a0 20 read_log 192834'10751 (192834'10343) modify 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by client.1883634.0:25542 2014-08-11 15:28:21.518397 2014-08-13 18:06:06.827046 7f5c973a57a0 20 read_log 192834'10755 (192834'10351) modify 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.1883634.0:26193 2014-08-11 15:28:22.748557 2014-08-13 18:06:06.827061 7f5c973a57a0 20 read_log 192834'10763 (192834'10365) modify fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by client.1883634.0:27782 2014-08-11 15:28:24.980375 2014-08-13 18:06:06.827073 7f5c973a57a0 20 read_log 192834'10776 (192834'10391) modify 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.1883634.0:29452 2014-08-11 15:28:27.290531 2014-08-13 18:06:06.827097 7f5c973a57a0 20 read_log 192834'10786 (192834'10411) modify 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.1883634.0:31396 2014-08-11 15:28:30.421943 2014-08-13 18:06:06.827113 7f5c973a57a0 20 read_log 192834'10787 (192834'10413) modify 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.1883634.0:31469 2014-08-11 15:28:30.573803 2014-08-13 18:06:06.827125 7f5c973a57a0 20 read_log 192834'10804 (192834'10443) modify c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.1883634.0:33259 2014-08-11 15:28:32.949970 2014-08-13 18:06:06.827140 7f5c973a57a0 20 read_log 192834'10818 (192834'10465) modify c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by client.1883634.0:35075 2014-08-11 15:28:35.996335 2014-08-13 18:06:06.827158 7f5c973a57a0 20 read_log 192834'10819 (192834'10467) modify 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by client.1883634.0:35111 2014-08-11 15:28:36.041264 2014-08-13 18:06:06.827170 7f5c973a57a0 20 read_log 192834'10825 (192834'10479) modify d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by client.1883634.0:35705 2014-08-11 15:28:37.026666 2014-08-13 18:06:06.827183 7f5c973a57a0 20 read_log 192834'10844 (192834'10513) modify 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.1883634.0:36846 2014-08-11 15:28:38.483690 2014-08-13 18:06:06.827196 7f5c973a57a0 20 read_log 192834'10854 (192834'10533) modify c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.1883634.0:38204 2014-08-11 15:28:40.360246 2014-08-13 18:06:06.827210 7f5c973a57a0 20 read_log 192834'10861 (192834'10547) modify a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.1883634.0:38489 2014-08-11 15:28:40.699539 2014-08-13 18:06:06.827249 7f5c973a57a0 20 read_log 192834'10865 (192834'10555) modify abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.1883634.0:39032 2014-08-11 15:28:41.501660 2014-08-13 18:06:06.827262 7f5c973a57a0 20 read_log 192834'10866 (192834'10557) modify 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.1883634.0:39283 2014-08-11 15:28:41.964185 2014-08-13 18:06:06.827274 7f5c973a57a0 20 read_log 192834'10877 (192834'10579) modify 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by client.1883634.0:40654 2014-08-11 15:28:44.085797 2014-08-13 18:06:06.827290 7f5c973a57a0 20 read_log 192834'10885 (192834'10595) modify 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.1883634.0:41363 2014-08-11 15:28:44.965739 2014-08-13 18:06:06.827305 7f5c973a57a0 20 read_log 192834'10897 (192834'10619) modify 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by client.1883634.0:42201 2014-08-11 15:28:46.167720 2014-08-13 18:06:06.827317 7f5c973a57a0 20 read_log 192834'10924 (192834'10675) modify 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by client.1883634.0:45010 2014-08-11 15:28:47.121262 2014-08-13 18:06:06.827330 7f5c973a57a0 20 read_log 192834'10925 (192834'10677) modify 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.1883634.0:45416 2014-08-11 15:28:47.270369 2014-08-13 18:06:06.827348 7f5c973a57a0 20 read_log 192888'10944 (0'0) promote ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by osd.367.0:51521 2014-08-11 16:45:04.731686 2014-08-13 18:06:06.827361 7f5c973a57a0 20 read_log 192888'10945 (192888'10944) modify ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.1315259.0:2504 2014-08-11 16:45:04.724932 2014-08-13 18:06:06.827388 7f5c973a57a0 20 read_log 192888'10946 (0'0) promote 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by osd.367.0:51523 2014-08-11 16:45:05.067198 2014-08-13 18:06:06.827405 7f5c973a57a0 20 read_log 192888'10947 (192888'10946) modify 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.1315259.0:2593 2014-08-11 16:45:05.060076 2014-08-13 18:06:06.827435 7f5c973a57a0 20 read_log 192888'10950 (0'0) promote a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.367.0:51531 2014-08-11 16:45:07.185859 2014-08-13 18:06:06.827452 7f5c973a57a0 20 read_log 192888'10951 (192888'10950) modify a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by client.1315259.0:3145 2014-08-11 16:45:07.176934 2014-08-13 18:06:06.827464 7f5c973a57a0 20 read_log 192888'10954 (0'0) promote 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by osd.367.0:51577 2014-08-11 16:45:13.116289 2014-08-13 18:06:06.827479 7f5c973a57a0 20 read_log 192888'10955 (192888'10954) modify 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.1315259.0:4731 2014-08-11 16:45:13.108903 2014-08-13 18:06:06.827491 7f5c973a57a0 20 read_log 192888'10962 (0'0) promote bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by osd.367.0:51625 2014-08-11 16:45:19.634169 2014-08-13 18:06:06.827504 7f5c973a57a0 20 read_log 192888'10963 (192888'10962) modify bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.1315259.0:6512 2014-08-11 16:45:19.625808 2014-08-13 18:06:06.827516 7f5c973a57a0 20 read_log 192888'10976 (0'0) promote 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by osd.367.0:51751 2014-08-11 16:45:34.155147 2014-08-13 18:06:06.827540 7f5c973a57a0 20 read_log 192888'10977 (192888'10976) modify 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.1315259.0:10386 2014-08-11 16:45:34.148712 2014-08-13 18:06:06.827553 7f5c973a57a0 20 read_log 192888'10982 (0'0) promote a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by osd.367.0:51833 2014-08-11 16:45:47.265985 2014-08-13 18:06:06.827569 7f5c973a57a0 20 read_log 192888'10983 (192888'10982) modify a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.1315259.0:13598 2014-08-11 16:45:47.258506 2014-08-13 18:06:06.827584 7f5c973a57a0 20 read_log 192888'10984 (0'0) promote cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by osd.367.0:51851 2014-08-11 16:45:50.173287 2014-08-13 18:06:06.827617 7f5c973a57a0 20 read_log 192888'10985 (192888'10984) modify cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.1315259.0:14358 2014-08-11 16:45:50.165361 2014-08-13 18:06:06.827633 7f5c973a57a0 20 read_log 192888'10986 (0'0) promote 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by osd.367.0:51861 2014-08-11 16:45:51.629138 2014-08-13 18:06:06.827646 7f5c973a57a0 20 read_log 192888'10987 (192888'10986) modify 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.1315259.0:14757 2014-08-11 16:45:51.622514 2014-08-13 18:06:06.827658 7f5c973a57a0 20 read_log 192888'10988 (0'0) promote 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by osd.367.0:51863 2014-08-11 16:45:51.803029 2014-08-13 18:06:06.827670 7f5c973a57a0 20 read_log 192888'10989 (192888'10988) modify 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.1315259.0:14803 2014-08-11 16:45:51.794589 2014-08-13 18:06:06.827683 7f5c973a57a0 20 read_log 192888'10992 (0'0) promote 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.367.0:51891 2014-08-11 16:45:56.753371 2014-08-13 18:06:06.827695 7f5c973a57a0 20 read_log 192888'10993 (192888'10992) modify 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by client.1315259.0:16086 2014-08-11 16:45:56.747312 2014-08-13 18:06:06.827707 7f5c973a57a0 20 read_log 192888'10994 (0'0) promote 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by osd.367.0:51893 2014-08-11 16:45:57.273489 2014-08-13 18:06:06.827734 7f5c973a57a0 20 read_log 192888'10995 (192888'10994) modify 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.1315259.0:16224 2014-08-11 16:45:57.266341 2014-08-13 18:06:06.827756 7f5c973a57a0 20 read_log 192888'10996 (0'0) promote 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by osd.367.0:51907 2014-08-11 16:45:59.318397 2014-08-13 18:06:06.827771 7f5c973a57a0 20 read_log 192888'10997 (192888'10996) modify 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.1315259.0:16741 2014-08-11 16:45:59.311077 2014-08-13 18:06:06.827784 7f5c973a57a0 20 read_log 192888'11000 (0'0) promote 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by osd.367.0:51923 2014-08-11 16:46:01.895782 2014-08-13 18:06:06.827796 7f5c973a57a0 20 read_log 192888'11001 (192888'11000) modify 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.1315259.0:17392 2014-08-11 16:46:01.887096 2014-08-13 18:06:06.827809 7f5c973a57a0 20 read_log 192888'11006 (0'0) promote d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by osd.367.0:51969 2014-08-11 16:46:06.327682 2014-08-13 18:06:06.827821 7f5c973a57a0 20 read_log 192888'11007 (192888'11006) modify d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.1315259.0:18512 2014-08-11 16:46:06.321872 2014-08-13 18:06:06.827833 7f5c973a57a0 20 read_log 192888'11010 (0'0) promote 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by osd.367.0:51975 2014-08-11 16:46:06.554185 2014-08-13 18:06:06.827845 7f5c973a57a0 20 read_log 192888'11011 (192888'11010) modify 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.1315259.0:18570 2014-08-11 16:46:06.545898 2014-08-13 18:06:06.827857 7f5c973a57a0 20 read_log 192888'11014 (0'0) promote c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by osd.367.0:52013 2014-08-11 16:46:11.080518 2014-08-13 18:06:06.827869 7f5c973a57a0 20 read_log 192888'11015 (192888'11014) modify c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.1315259.0:19716 2014-08-11 16:46:11.073196 2014-08-13 18:06:06.827882 7f5c973a57a0 20 read_log 192888'11016 (0'0) promote e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by osd.367.0:52045 2014-08-11 16:46:15.053276 2014-08-13 18:06:06.827894 7f5c973a57a0 20 read_log 192888'11017 (192888'11016) modify e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.1315259.0:20776 2014-08-11 16:46:15.045952 2014-08-13 18:06:06.827906 7f5c973a57a0 20 read_log 192888'11018 (0'0) promote 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by osd.367.0:52119 2014-08-11 16:46:22.968218 2014-08-13 18:06:06.827918 7f5c973a57a0 20 read_log 192888'11019 (192888'11018) modify 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.1315259.0:22847 2014-08-11 16:46:22.960256 2014-08-13 18:06:06.827931 7f5c973a57a0 20 read_log 192888'11022 (0'0) promote 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by osd.367.0:52163 2014-08-11 16:46:30.118991 2014-08-13 18:06:06.827945 7f5c973a57a0 20 read_log 192888'11023 (192888'11022) modify 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.1315259.0:24651 2014-08-11 16:46:30.111221 2014-08-13 18:06:06.827958 7f5c973a57a0 20 read_log 192888'11024 (0'0) promote 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.367.0:52167 2014-08-11 16:46:30.438201 2014-08-13 18:06:06.827972 7f5c973a57a0 20 read_log 192888'11025 (192888'11024) modify 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by client.1315259.0:24717 2014-08-11 16:46:30.430340 2014-08-13 18:06:06.827984 7f5c973a57a0 20 read_log 192888'11026 (0'0) promote c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by osd.367.0:52177 2014-08-11 16:46:32.623601 2014-08-13 18:06:06.827998 7f5c973a57a0 20 read_log 192888'11027 (192888'11026) modify c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.1315259.0:25272 2014-08-11 16:46:32.612928 2014-08-13 18:06:06.828020 7f5c973a57a0 20 read_log 192888'11028 (0'0) promote 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.367.0:52189 2014-08-11 16:46:33.560470 2014-08-13 18:06:06.828035 7f5c973a57a0 20 read_log 192888'11029 (192888'11028) modify 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by client.1315259.0:25508 2014-08-11 16:46:33.551832 2014-08-13 18:06:06.828047 7f5c973a57a0 20 read_log 192888'11032 (0'0) promote bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by osd.367.0:52213 2014-08-11 16:46:36.286931 2014-08-13 18:06:06.828059 7f5c973a57a0 20 read_log 192888'11033 (192888'11032) modify bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.1315259.0:26129 2014-08-11 16:46:36.277747 2014-08-13 18:06:06.828072 7f5c973a57a0 20 read_log 192888'11034 (0'0) promote 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.367.0:52245 2014-08-11 16:46:40.911730 2014-08-13 18:06:06.828086 7f5c973a57a0 20 read_log 192888'11035 (192888'11034) modify 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by client.1315259.0:27216 2014-08-11 16:46:40.903077 2014-08-13 18:06:06.828099 7f5c973a57a0 20 read_log 192888'11036 (0'0) promote ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.367.0:52267 2014-08-11 16:46:42.482224 2014-08-13 18:06:06.828113 7f5c973a57a0 20 read_log 192888'11037 (192888'11036) modify ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by client.1315259.0:27621 2014-08-11 16:46:42.473539 2014-08-13 18:06:06.828125 7f5c973a57a0 20 read_log 192888'11038 (0'0) promote 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by osd.367.0:52273 2014-08-11 16:46:42.793603 2014-08-13 18:06:06.828139 7f5c973a57a0 20 read_log 192888'11039 (192888'11038) modify 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.1315259.0:27700 2014-08-11 16:46:42.785176 2014-08-13 18:06:06.828161 7f5c973a57a0 20 read_log 192888'11054 (192888'10945) modify ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.1315259.0:33231 2014-08-11 16:46:56.584382 2014-08-13 18:06:06.828174 7f5c973a57a0 20 read_log 192888'11055 (192888'10947) modify 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.1315259.0:33320 2014-08-11 16:46:56.607744 2014-08-13 18:06:06.828186 7f5c973a57a0 20 read_log 192888'11057 (192888'10951) modify a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by client.1315259.0:33872 2014-08-11 16:46:56.746650 2014-08-13 18:06:06.828201 7f5c973a57a0 20 read_log 192888'11059 (192888'10955) modify 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.1315259.0:35458 2014-08-11 16:46:57.168188 2014-08-13 18:06:06.828215 7f5c973a57a0 20 read_log 192888'11063 (192888'10963) modify bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.1315259.0:37239 2014-08-11 16:46:57.630888 2014-08-13 18:06:06.828229 7f5c973a57a0 20 read_log 192888'11070 (192888'10977) modify 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.1315259.0:41113 2014-08-11 16:46:58.661659 2014-08-13 18:06:06.828246 7f5c973a57a0 20 read_log 192888'11073 (192888'10983) modify a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.1315259.0:44325 2014-08-11 16:46:59.546653 2014-08-13 18:06:06.828259 7f5c973a57a0 20 read_log 192888'11074 (192888'10985) modify cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.1315259.0:45085 2014-08-11 16:46:59.765559 2014-08-13 18:06:06.828271 7f5c973a57a0 20 read_log 192888'11075 (192888'10987) modify 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.1315259.0:45484 2014-08-11 16:46:59.883476 2014-08-13 18:06:06.828287 7f5c973a57a0 20 read_log 192888'11076 (192888'10989) modify 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.1315259.0:45530 2014-08-11 16:46:59.895663 2014-08-13 18:06:06.828325 7f5c973a57a0 20 read_log 192888'11078 (192888'10993) modify 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by client.1315259.0:46813 2014-08-11 16:47:00.265180 2014-08-13 18:06:06.828346 7f5c973a57a0 20 read_log 192888'11079 (192888'10995) modify 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.1315259.0:46951 2014-08-11 16:47:00.305221 2014-08-13 18:06:06.828358 7f5c973a57a0 20 read_log 192888'11080 (192888'10997) modify 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.1315259.0:47468 2014-08-11 16:47:00.466660 2014-08-13 18:06:06.828380 7f5c973a57a0 20 read_log 192888'11082 (192888'11001) modify 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.1315259.0:48119 2014-08-11 16:47:00.675237 2014-08-13 18:06:06.828398 7f5c973a57a0 20 read_log 192888'11085 (192888'11007) modify d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.1315259.0:49239 2014-08-11 16:47:00.999326 2014-08-13 18:06:06.828411 7f5c973a57a0 20 read_log 192888'11087 (192888'11011) modify 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.1315259.0:49297 2014-08-11 16:47:01.018475 2014-08-13 18:06:06.828431 7f5c973a57a0 20 read_log 192888'11089 (192888'11015) modify c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.1315259.0:50443 2014-08-11 16:47:01.371381 2014-08-13 18:06:06.828447 7f5c973a57a0 20 read_log 192888'11090 (192888'11017) modify e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.1315259.0:51503 2014-08-11 16:47:01.698788 2014-08-13 18:06:06.828459 7f5c973a57a0 20 read_log 192888'11091 (192888'11019) modify 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.1315259.0:53574 2014-08-11 16:47:02.294503 2014-08-13 18:06:06.828478 7f5c973a57a0 20 read_log 192888'11093 (192888'11023) modify 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.1315259.0:55378 2014-08-11 16:47:02.894661 2014-08-13 18:06:06.828494 7f5c973a57a0 20 read_log 192888'11094 (192888'11025) modify 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by client.1315259.0:55444 2014-08-11 16:47:02.914255 2014-08-13 18:06:06.828507 7f5c973a57a0 20 read_log 192888'11095 (192888'11027) modify c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.1315259.0:55999 2014-08-11 16:47:03.076849 2014-08-13 18:06:06.828519 7f5c973a57a0 20 read_log 192888'11096 (192888'11029) modify 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by client.1315259.0:56235 2014-08-11 16:47:03.143353 2014-08-13 18:06:06.828531 7f5c973a57a0 20 read_log 192888'11098 (192888'11033) modify bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.1315259.0:56856 2014-08-11 16:47:03.334345 2014-08-13 18:06:06.828543 7f5c973a57a0 20 read_log 192888'11099 (192888'11035) modify 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by client.1315259.0:57943 2014-08-11 16:47:03.657181 2014-08-13 18:06:06.828556 7f5c973a57a0 20 read_log 192888'11100 (192888'11037) modify ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by client.1315259.0:58348 2014-08-11 16:47:03.775170 2014-08-13 18:06:06.828568 7f5c973a57a0 20 read_log 192888'11101 (192888'11039) modify 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.1315259.0:58427 2014-08-11 16:47:03.799058 2014-08-13 18:06:06.828580 7f5c973a57a0 20 read_log 192888'11109 (0'0) promote e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.367.0:52824 2014-08-11 17:05:16.231403 2014-08-13 18:06:06.828592 7f5c973a57a0 20 read_log 192888'11110 (192888'11109) modify e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by client.942868.0:302 2014-08-11 17:05:16.221594 2014-08-13 18:06:06.828605 7f5c973a57a0 20 read_log 192888'11111 (0'0) promote 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.367.0:52837 2014-08-11 17:05:17.299470 2014-08-13 18:06:06.828624 7f5c973a57a0 20 read_log 192888'11112 (192888'11111) modify 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by client.942868.0:555 2014-08-11 17:05:17.291487 2014-08-13 18:06:06.828640 7f5c973a57a0 20 read_log 192888'11113 (0'0) promote 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by osd.367.0:52844 2014-08-11 17:05:18.652499 2014-08-13 18:06:06.828652 7f5c973a57a0 20 read_log 192888'11114 (192888'11113) modify 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.942868.0:887 2014-08-11 17:05:18.645175 2014-08-13 18:06:06.828665 7f5c973a57a0 20 read_log 192888'11127 (0'0) promote fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.367.0:52981 2014-08-11 17:05:33.694708 2014-08-13 18:06:06.828677 7f5c973a57a0 20 read_log 192888'11128 (192888'11127) modify fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by client.942868.0:4784 2014-08-11 17:05:33.687569 2014-08-13 18:06:06.828690 7f5c973a57a0 20 read_log 192888'11131 (0'0) promote afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.367.0:52985 2014-08-11 17:05:33.944768 2014-08-13 18:06:06.828702 7f5c973a57a0 20 read_log 192888'11132 (192888'11131) modify afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by client.942868.0:4845 2014-08-11 17:05:33.937089 2014-08-13 18:06:06.828714 7f5c973a57a0 20 read_log 192888'11135 (0'0) promote 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by osd.367.0:53015 2014-08-11 17:05:36.841503 2014-08-13 18:06:06.828727 7f5c973a57a0 20 read_log 192888'11136 (192888'11135) modify 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.942868.0:5594 2014-08-11 17:05:36.834546 2014-08-13 18:06:06.828739 7f5c973a57a0 20 read_log 192888'11137 (0'0) promote 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.367.0:53035 2014-08-11 17:05:40.002577 2014-08-13 18:06:06.828751 7f5c973a57a0 20 read_log 192888'11138 (192888'11137) modify 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by client.942868.0:6467 2014-08-11 17:05:39.995654 2014-08-13 18:06:06.828764 7f5c973a57a0 20 read_log 192888'11139 (0'0) promote b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.367.0:53051 2014-08-11 17:05:41.085115 2014-08-13 18:06:06.828776 7f5c973a57a0 20 read_log 192888'11140 (192888'11139) modify b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by client.942868.0:6768 2014-08-11 17:05:41.078382 2014-08-13 18:06:06.828789 7f5c973a57a0 20 read_log 192888'11141 (0'0) promote 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by osd.367.0:53055 2014-08-11 17:05:42.131591 2014-08-13 18:06:06.828801 7f5c973a57a0 20 read_log 192888'11142 (192888'11141) modify 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.942868.0:7026 2014-08-11 17:05:42.122543 2014-08-13 18:06:06.828825 7f5c973a57a0 20 read_log 192888'11143 (0'0) promote 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by osd.367.0:53079 2014-08-11 17:05:46.272201 2014-08-13 18:06:06.828842 7f5c973a57a0 20 read_log 192888'11144 (192888'11143) modify 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.942868.0:8124 2014-08-11 17:05:46.265733 2014-08-13 18:06:06.828854 7f5c973a57a0 20 read_log 192888'11145 (0'0) promote 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by osd.367.0:53095 2014-08-11 17:05:48.222282 2014-08-13 18:06:06.828869 7f5c973a57a0 20 read_log 192888'11146 (192888'11145) modify 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.942868.0:8663 2014-08-11 17:05:48.214940 2014-08-13 18:06:06.828882 7f5c973a57a0 20 read_log 192888'11149 (0'0) promote 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.367.0:53135 2014-08-11 17:05:52.729959 2014-08-13 18:06:06.828904 7f5c973a57a0 20 read_log 192888'11150 (192888'11149) modify 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by client.942868.0:9868 2014-08-11 17:05:52.723736 2014-08-13 18:06:06.828923 7f5c973a57a0 20 read_log 192888'11153 (0'0) promote 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.367.0:53167 2014-08-11 17:05:57.277656 2014-08-13 18:06:06.828935 7f5c973a57a0 20 read_log 192888'11154 (192888'11153) modify 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by client.942868.0:11127 2014-08-11 17:05:57.270410 2014-08-13 18:06:06.828948 7f5c973a57a0 20 read_log 192888'11159 (0'0) promote 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.367.0:53241 2014-08-11 17:06:07.286353 2014-08-13 18:06:06.828960 7f5c973a57a0 20 read_log 192888'11160 (192888'11159) modify 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by client.942868.0:13780 2014-08-11 17:06:07.278093 2014-08-13 18:06:06.828973 7f5c973a57a0 20 read_log 192888'11161 (0'0) promote 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by osd.367.0:53253 2014-08-11 17:06:08.970062 2014-08-13 18:06:06.828985 7f5c973a57a0 20 read_log 192888'11162 (192888'11161) modify 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.942868.0:14180 2014-08-11 17:06:08.957099 2014-08-13 18:06:06.828998 7f5c973a57a0 20 read_log 192888'11175 (0'0) promote f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by osd.367.0:53383 2014-08-11 17:06:29.098136 2014-08-13 18:06:06.829010 7f5c973a57a0 20 read_log 192888'11176 (192888'11175) modify f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.942868.0:19399 2014-08-11 17:06:29.090335 2014-08-13 18:06:06.829022 7f5c973a57a0 20 read_log 192888'11177 (0'0) promote b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by osd.367.0:53395 2014-08-11 17:06:30.044232 2014-08-13 18:06:06.829041 7f5c973a57a0 20 read_log 192888'11178 (192888'11177) modify b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.942868.0:19652 2014-08-11 17:06:30.037503 2014-08-13 18:06:06.829056 7f5c973a57a0 20 read_log 192888'11179 (0'0) promote 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by osd.367.0:53411 2014-08-11 17:06:33.351894 2014-08-13 18:06:06.829099 7f5c973a57a0 20 read_log 192888'11180 (192888'11179) modify 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.942868.0:20513 2014-08-11 17:06:33.345372 2014-08-13 18:06:06.829116 7f5c973a57a0 20 read_log 192888'11183 (0'0) promote b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by osd.367.0:53439 2014-08-11 17:06:36.530468 2014-08-13 18:06:06.829128 7f5c973a57a0 20 read_log 192888'11184 (192888'11183) modify b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.942868.0:21343 2014-08-11 17:06:36.522598 2014-08-13 18:06:06.829141 7f5c973a57a0 20 read_log 192888'11185 (0'0) promote 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by osd.367.0:53449 2014-08-11 17:06:37.908131 2014-08-13 18:06:06.829176 7f5c973a57a0 20 read_log 192888'11186 (192888'11185) modify 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.942868.0:21715 2014-08-11 17:06:37.901376 2014-08-13 18:06:06.829192 7f5c973a57a0 20 read_log 192888'11191 (0'0) promote e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by osd.367.0:53463 2014-08-11 17:06:38.767926 2014-08-13 18:06:06.829205 7f5c973a57a0 20 read_log 192888'11192 (192888'11191) modify e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.942868.0:21944 2014-08-11 17:06:38.761155 2014-08-13 18:06:06.829217 7f5c973a57a0 20 read_log 192888'11197 (0'0) promote 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by osd.367.0:53499 2014-08-11 17:06:44.300325 2014-08-13 18:06:06.829241 7f5c973a57a0 20 read_log 192888'11198 (192888'11197) modify 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.942868.0:23345 2014-08-11 17:06:44.293222 2014-08-13 18:06:06.829256 7f5c973a57a0 20 read_log 192888'11199 (0'0) promote ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by osd.367.0:53521 2014-08-11 17:06:46.922675 2014-08-13 18:06:06.829268 7f5c973a57a0 20 read_log 192888'11200 (192888'11199) modify ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.942868.0:24057 2014-08-11 17:06:46.914526 2014-08-13 18:06:06.829293 7f5c973a57a0 20 read_log 192888'11201 (0'0) promote a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by osd.367.0:53527 2014-08-11 17:06:47.407458 2014-08-13 18:06:06.829306 7f5c973a57a0 20 read_log 192888'11202 (192888'11201) modify a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.942868.0:24188 2014-08-11 17:06:47.399811 2014-08-13 18:06:06.829345 7f5c973a57a0 20 read_log 192888'11205 (0'0) promote da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.367.0:53569 2014-08-11 17:06:51.614893 2014-08-13 18:06:06.829358 7f5c973a57a0 20 read_log 192888'11206 (192888'11205) modify da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by client.942868.0:25316 2014-08-11 17:06:51.607465 2014-08-13 18:06:06.829377 7f5c973a57a0 20 read_log 192888'11207 (0'0) promote 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by osd.367.0:53573 2014-08-11 17:06:51.824881 2014-08-13 18:06:06.829397 7f5c973a57a0 20 read_log 192888'11208 (192888'11207) modify 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.942868.0:25372 2014-08-11 17:06:51.817380 2014-08-13 18:06:06.829414 7f5c973a57a0 20 read_log 192888'11209 (0'0) promote d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by osd.367.0:53595 2014-08-11 17:06:53.832892 2014-08-13 18:06:06.829426 7f5c973a57a0 20 read_log 192888'11210 (192888'11209) modify d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.942868.0:25853 2014-08-11 17:06:53.825462 2014-08-13 18:06:06.829439 7f5c973a57a0 20 read_log 192888'11211 (0'0) promote 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.367.0:53613 2014-08-11 17:06:55.829649 2014-08-13 18:06:06.829451 7f5c973a57a0 20 read_log 192888'11212 (192888'11211) modify 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by client.942868.0:26369 2014-08-11 17:06:55.822495 2014-08-13 18:06:06.829463 7f5c973a57a0 20 read_log 192888'11213 (0'0) promote d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by osd.367.0:53633 2014-08-11 17:06:58.615402 2014-08-13 18:06:06.829475 7f5c973a57a0 20 read_log 192888'11214 (192888'11213) modify d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.942868.0:27106 2014-08-11 17:06:58.608483 2014-08-13 18:06:06.829487 7f5c973a57a0 20 read_log 192888'11219 (0'0) promote 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.367.0:53671 2014-08-11 17:07:02.579471 2014-08-13 18:06:06.829499 7f5c973a57a0 20 read_log 192888'11220 (192888'11219) modify 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by client.942868.0:28159 2014-08-11 17:07:02.569063 2014-08-13 18:06:06.829512 7f5c973a57a0 20 read_log 192888'11223 (0'0) promote fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.367.0:53683 2014-08-11 17:07:04.075975 2014-08-13 18:06:06.829524 7f5c973a57a0 20 read_log 192888'11224 (192888'11223) modify fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by client.942868.0:28565 2014-08-11 17:07:04.068114 2014-08-13 18:06:06.829536 7f5c973a57a0 20 read_log 192888'11225 (0'0) promote dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by osd.367.0:53711 2014-08-11 17:07:06.773078 2014-08-13 18:06:06.829557 7f5c973a57a0 20 read_log 192888'11226 (192888'11225) modify dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.942868.0:29252 2014-08-11 17:07:06.765594 2014-08-13 18:06:06.829572 7f5c973a57a0 20 read_log 192888'11227 (192888'11110) modify e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by client.942868.0:31568 2014-08-11 17:07:15.975062 2014-08-13 18:06:06.829584 7f5c973a57a0 20 read_log 192888'11228 (192888'11112) modify 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by client.942868.0:31821 2014-08-11 17:07:16.067687 2014-08-13 18:06:06.829597 7f5c973a57a0 20 read_log 192888'11229 (192888'11114) modify 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.942868.0:32153 2014-08-11 17:07:16.168980 2014-08-13 18:06:06.829609 7f5c973a57a0 20 read_log 192888'11236 (192888'11128) modify fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by client.942868.0:36050 2014-08-11 17:07:17.229791 2014-08-13 18:06:06.829621 7f5c973a57a0 20 read_log 192888'11238 (192888'11132) modify afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by client.942868.0:36111 2014-08-11 17:07:17.245184 2014-08-13 18:06:06.829655 7f5c973a57a0 20 read_log 192888'11240 (192888'11136) modify 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.942868.0:36860 2014-08-11 17:07:17.452605 2014-08-13 18:06:06.829668 7f5c973a57a0 20 read_log 192888'11241 (192888'11138) modify 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by client.942868.0:37733 2014-08-11 17:07:17.693147 2014-08-13 18:06:06.829680 7f5c973a57a0 20 read_log 192888'11242 (192888'11140) modify b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by client.942868.0:38034 2014-08-11 17:07:17.774082 2014-08-13 18:06:06.829705 7f5c973a57a0 20 read_log 192888'11243 (192888'11142) modify 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.942868.0:38292 2014-08-11 17:07:17.844766 2014-08-13 18:06:06.829718 7f5c973a57a0 20 read_log 192888'11244 (192888'11144) modify 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.942868.0:39390 2014-08-11 17:07:18.168110 2014-08-13 18:06:06.829731 7f5c973a57a0 20 read_log 192888'11245 (192888'11146) modify 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.942868.0:39929 2014-08-11 17:07:18.320540 2014-08-13 18:06:06.829743 7f5c973a57a0 20 read_log 192888'11247 (192888'11150) modify 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by client.942868.0:41134 2014-08-11 17:07:18.651489 2014-08-13 18:06:06.829756 7f5c973a57a0 20 read_log 192888'11249 (192888'11154) modify 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by client.942868.0:42393 2014-08-11 17:07:18.985753 2014-08-13 18:06:06.829768 7f5c973a57a0 20 read_log 192888'11252 (192888'11160) modify 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by client.942868.0:45046 2014-08-11 17:07:19.732475 2014-08-13 18:06:06.829780 7f5c973a57a0 20 read_log 192888'11253 (192888'11162) modify 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.942868.0:45446 2014-08-11 17:07:19.840005 2014-08-13 18:06:06.829793 7f5c973a57a0 20 read_log 192888'11260 (192888'11176) modify f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.942868.0:50665 2014-08-11 17:07:21.394326 2014-08-13 18:06:06.829807 7f5c973a57a0 20 read_log 192888'11261 (192888'11178) modify b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.942868.0:50918 2014-08-11 17:07:21.472939 2014-08-13 18:06:06.829822 7f5c973a57a0 20 read_log 192888'11262 (192888'11180) modify 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.942868.0:51779 2014-08-11 17:07:21.750058 2014-08-13 18:06:06.829837 7f5c973a57a0 20 read_log 192888'11264 (192888'11184) modify b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.942868.0:52609 2014-08-11 17:07:21.994603 2014-08-13 18:06:06.829859 7f5c973a57a0 20 read_log 192888'11265 (192888'11186) modify 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.942868.0:52981 2014-08-11 17:07:22.112001 2014-08-13 18:06:06.829874 7f5c973a57a0 20 read_log 192888'11268 (192888'11192) modify e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.942868.0:53210 2014-08-11 17:07:22.184497 2014-08-13 18:06:06.829886 7f5c973a57a0 20 read_log 192888'11271 (192888'11198) modify 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.942868.0:54611 2014-08-11 17:07:22.608337 2014-08-13 18:06:06.829900 7f5c973a57a0 20 read_log 192888'11272 (192888'11200) modify ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.942868.0:55323 2014-08-11 17:07:22.812649 2014-08-13 18:06:06.829913 7f5c973a57a0 20 read_log 192888'11273 (192888'11202) modify a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.942868.0:55454 2014-08-11 17:07:22.851943 2014-08-13 18:06:06.829927 7f5c973a57a0 20 read_log 192888'11275 (192888'11206) modify da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by client.942868.0:56582 2014-08-11 17:07:23.198619 2014-08-13 18:06:06.829941 7f5c973a57a0 20 read_log 192888'11276 (192888'11208) modify 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.942868.0:56638 2014-08-11 17:07:23.215162 2014-08-13 18:06:06.829956 7f5c973a57a0 20 read_log 192888'11277 (192888'11210) modify d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.942868.0:57119 2014-08-11 17:07:23.362588 2014-08-13 18:06:06.829968 7f5c973a57a0 20 read_log 192888'11278 (192888'11212) modify 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by client.942868.0:57635 2014-08-11 17:07:23.512804 2014-08-13 18:06:06.829982 7f5c973a57a0 20 read_log 192888'11279 (192888'11214) modify d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.942868.0:58372 2014-08-11 17:07:23.724071 2014-08-13 18:06:06.829995 7f5c973a57a0 20 read_log 192888'11282 (192888'11220) modify 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by client.942868.0:59425 2014-08-11 17:07:24.042484 2014-08-13 18:06:06.830009 7f5c973a57a0 20 read_log 192888'11284 (192888'11224) modify fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by client.942868.0:59831 2014-08-11 17:07:24.163865 2014-08-13 18:06:06.830021 7f5c973a57a0 20 read_log 192888'11285 (192888'11226) modify dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.942868.0:60518 2014-08-11 17:07:24.362534 2014-08-13 18:06:06.830035 7f5c973a57a0 20 read_log 193305'11286 (0'0) promote 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by osd.15.0:46 2014-08-11 18:42:48.054685 2014-08-13 18:06:06.830048 7f5c973a57a0 20 read_log 193305'11287 (193305'11286) modify 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.951337.0:4008 2014-08-11 18:42:48.047800 2014-08-13 18:06:06.830062 7f5c973a57a0 20 read_log 193305'11288 (0'0) promote ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by osd.15.0:50 2014-08-11 18:42:49.702963 2014-08-13 18:06:06.830075 7f5c973a57a0 20 read_log 193305'11289 (193305'11288) modify ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.951337.0:4450 2014-08-11 18:42:49.695709 2014-08-13 18:06:06.830089 7f5c973a57a0 20 read_log 193305'11290 (0'0) promote 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by osd.15.0:52 2014-08-11 18:42:50.765268 2014-08-13 18:06:06.830101 7f5c973a57a0 20 read_log 193305'11291 (193305'11290) modify 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.951337.0:4741 2014-08-11 18:42:50.758489 2014-08-13 18:06:06.830115 7f5c973a57a0 20 read_log 193305'11292 (0'0) promote 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by osd.15.0:80 2014-08-11 18:43:05.103704 2014-08-13 18:06:06.830134 7f5c973a57a0 20 read_log 193305'11293 (193305'11292) modify 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.951337.0:8606 2014-08-11 18:43:05.096233 2014-08-13 18:06:06.830154 7f5c973a57a0 20 read_log 193305'11294 (0'0) promote 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by osd.15.0:86 2014-08-11 18:43:07.446727 2014-08-13 18:06:06.830169 7f5c973a57a0 20 read_log 193305'11295 (193305'11294) modify 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.951337.0:9225 2014-08-11 18:43:07.439410 2014-08-13 18:06:06.830182 7f5c973a57a0 20 read_log 193305'11296 (0'0) promote 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:102 2014-08-11 18:43:13.926684 2014-08-13 18:06:06.830196 7f5c973a57a0 20 read_log 193305'11297 (193305'11296) modify 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by client.951337.0:10978 2014-08-11 18:43:13.918710 2014-08-13 18:06:06.830209 7f5c973a57a0 20 read_log 193305'11298 (0'0) promote e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by osd.15.0:108 2014-08-11 18:43:16.197733 2014-08-13 18:06:06.830223 7f5c973a57a0 20 read_log 193305'11299 (193305'11298) modify e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.951337.0:11540 2014-08-11 18:43:16.189296 2014-08-13 18:06:06.830236 7f5c973a57a0 20 read_log 193305'11300 (0'0) promote 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by osd.15.0:118 2014-08-11 18:43:21.372339 2014-08-13 18:06:06.830253 7f5c973a57a0 20 read_log 193305'11301 (193305'11300) modify 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.951337.0:12928 2014-08-11 18:43:21.365399 2014-08-13 18:06:06.830267 7f5c973a57a0 20 read_log 193305'11302 (0'0) promote 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by osd.15.0:120 2014-08-11 18:43:21.484409 2014-08-13 18:06:06.830280 7f5c973a57a0 20 read_log 193305'11303 (193305'11302) modify 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.951337.0:12959 2014-08-11 18:43:21.477648 2014-08-13 18:06:06.830292 7f5c973a57a0 20 read_log 193305'11304 (0'0) promote d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by osd.15.0:124 2014-08-11 18:43:21.959908 2014-08-13 18:06:06.830307 7f5c973a57a0 20 read_log 193305'11305 (193305'11304) modify d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.951337.0:13090 2014-08-11 18:43:21.952537 2014-08-13 18:06:06.830322 7f5c973a57a0 20 read_log 193305'11306 (0'0) promote d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by osd.15.0:128 2014-08-11 18:43:22.963397 2014-08-13 18:06:06.830335 7f5c973a57a0 20 read_log 193305'11307 (193305'11306) modify d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.951337.0:13358 2014-08-11 18:43:22.956467 2014-08-13 18:06:06.830350 7f5c973a57a0 20 read_log 193305'11308 (0'0) promote 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:130 2014-08-11 18:43:23.082095 2014-08-13 18:06:06.830365 7f5c973a57a0 20 read_log 193305'11309 (193305'11308) modify 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by client.951337.0:13388 2014-08-11 18:43:23.075749 2014-08-13 18:06:06.830386 7f5c973a57a0 20 read_log 193305'11310 (0'0) promote 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:134 2014-08-11 18:43:23.587497 2014-08-13 18:06:06.830406 7f5c973a57a0 20 read_log 193305'11311 (193305'11310) modify 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by client.951337.0:13507 2014-08-11 18:43:23.581360 2014-08-13 18:06:06.830428 7f5c973a57a0 20 read_log 193305'11312 (0'0) promote 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by osd.15.0:138 2014-08-11 18:43:23.786903 2014-08-13 18:06:06.830464 7f5c973a57a0 20 read_log 193305'11313 (193305'11312) modify 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.951337.0:13561 2014-08-11 18:43:23.780761 2014-08-13 18:06:06.830488 7f5c973a57a0 20 read_log 193305'11314 (0'0) promote 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by osd.15.0:142 2014-08-11 18:43:29.273991 2014-08-13 18:06:06.830501 7f5c973a57a0 20 read_log 193305'11315 (193305'11314) modify 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.951337.0:15088 2014-08-11 18:43:29.262485 2014-08-13 18:06:06.830514 7f5c973a57a0 20 read_log 193305'11316 (0'0) promote 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by osd.15.0:156 2014-08-11 18:43:31.888423 2014-08-13 18:06:06.830536 7f5c973a57a0 20 read_log 193305'11317 (193305'11316) modify 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.951337.0:15860 2014-08-11 18:43:31.880732 2014-08-13 18:06:06.830648 7f5c973a57a0 20 read_log 193305'11318 (0'0) promote 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:158 2014-08-11 18:43:32.071092 2014-08-13 18:06:06.830667 7f5c973a57a0 20 read_log 193305'11319 (193305'11318) modify 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by client.951337.0:15914 2014-08-11 18:43:32.062261 2014-08-13 18:06:06.830679 7f5c973a57a0 20 read_log 193305'11320 (0'0) promote 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by osd.15.0:164 2014-08-11 18:43:33.065977 2014-08-13 18:06:06.830692 7f5c973a57a0 20 read_log 193305'11321 (193305'11320) modify 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.951337.0:16190 2014-08-11 18:43:33.058240 2014-08-13 18:06:06.830704 7f5c973a57a0 20 read_log 193305'11322 (0'0) promote 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by osd.15.0:170 2014-08-11 18:43:34.565791 2014-08-13 18:06:06.830716 7f5c973a57a0 20 read_log 193305'11323 (193305'11322) modify 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.951337.0:16613 2014-08-11 18:43:34.557413 2014-08-13 18:06:06.830729 7f5c973a57a0 20 read_log 193305'11324 (0'0) promote 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:186 2014-08-11 18:43:41.249832 2014-08-13 18:06:06.830741 7f5c973a57a0 20 read_log 193305'11325 (193305'11324) modify 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by client.951337.0:18535 2014-08-11 18:43:41.242579 2014-08-13 18:06:06.830753 7f5c973a57a0 20 read_log 193305'11326 (0'0) promote c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:188 2014-08-11 18:43:41.517078 2014-08-13 18:06:06.830765 7f5c973a57a0 20 read_log 193305'11327 (193305'11326) modify c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by client.951337.0:18614 2014-08-11 18:43:41.509413 2014-08-13 18:06:06.830778 7f5c973a57a0 20 read_log 193305'11328 (0'0) promote b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by osd.15.0:192 2014-08-11 18:43:42.544271 2014-08-13 18:06:06.830790 7f5c973a57a0 20 read_log 193305'11329 (193305'11328) modify b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.951337.0:18893 2014-08-11 18:43:42.535377 2014-08-13 18:06:06.830802 7f5c973a57a0 20 read_log 193305'11330 (0'0) promote bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by osd.15.0:198 2014-08-11 18:43:44.018122 2014-08-13 18:06:06.830815 7f5c973a57a0 20 read_log 193305'11331 (193305'11330) modify bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.951337.0:19270 2014-08-11 18:43:44.009958 2014-08-13 18:06:06.830828 7f5c973a57a0 20 read_log 193305'11332 (0'0) promote 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:202 2014-08-11 18:43:44.214961 2014-08-13 18:06:06.830850 7f5c973a57a0 20 read_log 193305'11333 (193305'11332) modify 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by client.951337.0:19317 2014-08-11 18:43:44.209037 2014-08-13 18:06:06.830863 7f5c973a57a0 20 read_log 193305'11334 (0'0) promote ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by osd.15.0:222 2014-08-11 18:43:50.875402 2014-08-13 18:06:06.830876 7f5c973a57a0 20 read_log 193305'11335 (193305'11334) modify ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.951337.0:21183 2014-08-11 18:43:50.866318 2014-08-13 18:06:06.830888 7f5c973a57a0 20 read_log 193305'11336 (0'0) promote f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:228 2014-08-11 18:43:53.999903 2014-08-13 18:06:06.830900 7f5c973a57a0 20 read_log 193305'11337 (193305'11336) modify f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by client.951337.0:22056 2014-08-11 18:43:53.993100 2014-08-13 18:06:06.830913 7f5c973a57a0 20 read_log 193305'11338 (0'0) promote 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:230 2014-08-11 18:43:55.168467 2014-08-13 18:06:06.830925 7f5c973a57a0 20 read_log 193305'11339 (193305'11338) modify 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by client.951337.0:22369 2014-08-11 18:43:55.160101 2014-08-13 18:06:06.830937 7f5c973a57a0 20 read_log 193305'11340 (0'0) promote 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by osd.15.0:236 2014-08-11 18:43:57.041541 2014-08-13 18:06:06.830949 7f5c973a57a0 20 read_log 193305'11341 (193305'11340) modify 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.951337.0:22884 2014-08-11 18:43:57.032819 2014-08-13 18:06:06.830968 7f5c973a57a0 20 read_log 193305'11342 (0'0) promote a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:246 2014-08-11 18:43:58.953589 2014-08-13 18:06:06.830980 7f5c973a57a0 20 read_log 193305'11343 (193305'11342) modify a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by client.951337.0:23432 2014-08-11 18:43:58.944233 2014-08-13 18:06:06.830993 7f5c973a57a0 20 read_log 193305'11344 (0'0) promote 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by osd.15.0:248 2014-08-11 18:43:59.211798 2014-08-13 18:06:06.831005 7f5c973a57a0 20 read_log 193305'11345 (193305'11344) modify 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.951337.0:23500 2014-08-11 18:43:59.203263 2014-08-13 18:06:06.831017 7f5c973a57a0 20 read_log 193305'11346 (0'0) promote c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:256 2014-08-11 18:44:02.863498 2014-08-13 18:06:06.831029 7f5c973a57a0 20 read_log 193305'11347 (193305'11346) modify c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by client.951337.0:24511 2014-08-11 18:44:02.854883 2014-08-13 18:06:06.831042 7f5c973a57a0 20 read_log 193305'11348 (0'0) promote 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by osd.15.0:266 2014-08-11 18:44:05.935682 2014-08-13 18:06:06.831054 7f5c973a57a0 20 read_log 193305'11349 (193305'11348) modify 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.951337.0:25355 2014-08-11 18:44:05.925906 2014-08-13 18:06:06.831066 7f5c973a57a0 20 read_log 193305'11350 (0'0) promote ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by osd.15.0:296 2014-08-11 18:44:17.230935 2014-08-13 18:06:06.831078 7f5c973a57a0 20 read_log 193305'11351 (193305'11350) modify ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.951337.0:28434 2014-08-11 18:44:17.223909 2014-08-13 18:06:06.831090 7f5c973a57a0 20 read_log 193305'11352 (0'0) promote ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by osd.15.0:310 2014-08-11 18:44:19.448066 2014-08-13 18:06:06.831110 7f5c973a57a0 20 read_log 193305'11353 (193305'11352) modify ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.951337.0:29029 2014-08-11 18:44:19.440514 2014-08-13 18:06:06.831123 7f5c973a57a0 20 read_log 193305'11354 (0'0) promote 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:324 2014-08-11 18:44:23.592672 2014-08-13 18:06:06.831135 7f5c973a57a0 20 read_log 193305'11355 (193305'11354) modify 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by client.951337.0:30140 2014-08-11 18:44:23.584266 2014-08-13 18:06:06.831153 7f5c973a57a0 20 read_log 193305'11356 (0'0) promote 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by osd.15.0:328 2014-08-11 18:44:24.557511 2014-08-13 18:06:06.831166 7f5c973a57a0 20 read_log 193305'11357 (193305'11356) modify 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.951337.0:30386 2014-08-11 18:44:24.550381 2014-08-13 18:06:06.831178 7f5c973a57a0 20 read_log 193305'11358 (0'0) promote c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:336 2014-08-11 18:44:26.288653 2014-08-13 18:06:06.831191 7f5c973a57a0 20 read_log 193305'11359 (193305'11358) modify c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by client.951337.0:30848 2014-08-11 18:44:26.279327 2014-08-13 18:06:06.831203 7f5c973a57a0 20 read_log 193305'11360 (0'0) promote e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by osd.15.0:342 2014-08-11 18:44:26.975458 2014-08-13 18:06:06.831215 7f5c973a57a0 20 read_log 193305'11361 (193305'11360) modify e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.951337.0:31043 2014-08-11 18:44:26.966134 2014-08-13 18:06:06.831228 7f5c973a57a0 20 read_log 193305'11362 (193305'11287) modify 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.951337.0:36719 2014-08-11 18:44:36.489446 2014-08-13 18:06:06.831240 7f5c973a57a0 20 read_log 193305'11363 (193305'11289) modify ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.951337.0:37161 2014-08-11 18:44:36.742046 2014-08-13 18:06:06.831252 7f5c973a57a0 20 read_log 193305'11364 (193305'11291) modify 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.951337.0:37452 2014-08-11 18:44:36.919572 2014-08-13 18:06:06.831264 7f5c973a57a0 20 read_log 193305'11365 (193305'11293) modify 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.951337.0:41317 2014-08-11 18:44:38.716308 2014-08-13 18:06:06.831276 7f5c973a57a0 20 read_log 193305'11366 (193305'11295) modify 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.951337.0:41936 2014-08-11 18:44:38.932609 2014-08-13 18:06:06.831288 7f5c973a57a0 20 read_log 193305'11367 (193305'11297) modify 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by client.951337.0:43689 2014-08-11 18:44:39.666394 2014-08-13 18:06:06.831300 7f5c973a57a0 20 read_log 193305'11368 (193305'11299) modify e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.951337.0:44251 2014-08-11 18:44:39.832782 2014-08-13 18:06:06.831316 7f5c973a57a0 20 read_log 193305'11369 (193305'11301) modify 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.951337.0:45639 2014-08-11 18:44:40.258866 2014-08-13 18:06:06.831331 7f5c973a57a0 20 read_log 193305'11370 (193305'11303) modify 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.951337.0:45670 2014-08-11 18:44:40.267087 2014-08-13 18:06:06.831343 7f5c973a57a0 20 read_log 193305'11371 (193305'11305) modify d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.951337.0:45801 2014-08-11 18:44:40.307624 2014-08-13 18:06:06.831356 7f5c973a57a0 20 read_log 193305'11372 (193305'11307) modify d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.951337.0:46069 2014-08-11 18:44:40.380477 2014-08-13 18:06:06.831374 7f5c973a57a0 20 read_log 193305'11373 (193305'11309) modify 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by client.951337.0:46099 2014-08-11 18:44:40.388204 2014-08-13 18:06:06.831387 7f5c973a57a0 20 read_log 193305'11374 (193305'11311) modify 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by client.951337.0:46218 2014-08-11 18:44:40.422552 2014-08-13 18:06:06.831399 7f5c973a57a0 20 read_log 193305'11375 (193305'11313) modify 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.951337.0:46272 2014-08-11 18:44:40.437100 2014-08-13 18:06:06.831411 7f5c973a57a0 20 read_log 193305'11376 (193305'11315) modify 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.951337.0:47799 2014-08-11 18:44:40.869127 2014-08-13 18:06:06.831433 7f5c973a57a0 20 read_log 193305'11377 (193305'11317) modify 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.951337.0:48571 2014-08-11 18:44:41.091429 2014-08-13 18:06:06.831447 7f5c973a57a0 20 read_log 193305'11378 (193305'11378) modify 20e/hit_set_24.20e_archive_2014-08-11 18:39:56.767178_2014-08-11 18:44:41.106881/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:41.107140 2014-08-13 18:06:06.831466 7f5c973a57a0 20 read_log 193305'11379 (193305'11319) modify 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by client.951337.0:48625 2014-08-11 18:44:41.106326 2014-08-13 18:06:06.831483 7f5c973a57a0 20 read_log 193305'11380 (193305'11321) modify 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.951337.0:48901 2014-08-11 18:44:41.190339 2014-08-13 18:06:06.831497 7f5c973a57a0 20 read_log 193305'11381 (193305'11323) modify 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.951337.0:49324 2014-08-11 18:44:41.330594 2014-08-13 18:06:06.831512 7f5c973a57a0 20 read_log 193305'11382 (193305'11325) modify 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by client.951337.0:51246 2014-08-11 18:44:41.894517 2014-08-13 18:06:06.831535 7f5c973a57a0 20 read_log 193305'11383 (193305'11327) modify c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by client.951337.0:51325 2014-08-11 18:44:41.916226 2014-08-13 18:06:06.831550 7f5c973a57a0 20 read_log 193305'11384 (193305'11329) modify b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.951337.0:51604 2014-08-11 18:44:41.999347 2014-08-13 18:06:06.831568 7f5c973a57a0 20 read_log 193305'11385 (193305'11331) modify bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.951337.0:51981 2014-08-11 18:44:42.102363 2014-08-13 18:06:06.831591 7f5c973a57a0 20 read_log 193305'11386 (193305'11333) modify 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by client.951337.0:52028 2014-08-11 18:44:42.114223 2014-08-13 18:06:06.831604 7f5c973a57a0 20 read_log 193305'11387 (193305'11335) modify ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.951337.0:53894 2014-08-11 18:44:42.667192 2014-08-13 18:06:06.831617 7f5c973a57a0 20 read_log 193305'11388 (193305'11337) modify f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by client.951337.0:54767 2014-08-11 18:44:42.933118 2014-08-13 18:06:06.831629 7f5c973a57a0 20 read_log 193305'11389 (193305'11339) modify 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by client.951337.0:55080 2014-08-11 18:44:43.018625 2014-08-13 18:06:06.831642 7f5c973a57a0 20 read_log 193305'11390 (193305'11341) modify 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.951337.0:55595 2014-08-11 18:44:43.172902 2014-08-13 18:06:06.831654 7f5c973a57a0 20 read_log 193305'11391 (193305'11343) modify a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by client.951337.0:56143 2014-08-11 18:44:43.333511 2014-08-13 18:06:06.831684 7f5c973a57a0 20 read_log 193305'11392 (193305'11345) modify 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.951337.0:56211 2014-08-11 18:44:43.354579 2014-08-13 18:06:06.831700 7f5c973a57a0 20 read_log 193305'11393 (193305'11347) modify c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by client.951337.0:57222 2014-08-11 18:44:43.676753 2014-08-13 18:06:06.831716 7f5c973a57a0 20 read_log 193305'11394 (193305'11349) modify 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.951337.0:58066 2014-08-11 18:44:43.937092 2014-08-13 18:06:06.831729 7f5c973a57a0 20 read_log 193305'11395 (193305'11351) modify ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.951337.0:61145 2014-08-11 18:44:44.835223 2014-08-13 18:06:06.831741 7f5c973a57a0 20 read_log 193305'11396 (193305'11353) modify ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.951337.0:61740 2014-08-11 18:44:45.015364 2014-08-13 18:06:06.831753 7f5c973a57a0 20 read_log 193305'11397 (193305'11355) modify 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by client.951337.0:62851 2014-08-11 18:44:45.343961 2014-08-13 18:06:06.831765 7f5c973a57a0 20 read_log 193305'11398 (193305'11357) modify 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.951337.0:63097 2014-08-11 18:44:45.408425 2014-08-13 18:06:06.831778 7f5c973a57a0 20 read_log 193305'11399 (193305'11359) modify c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by client.951337.0:63559 2014-08-11 18:44:45.543292 2014-08-13 18:06:06.831790 7f5c973a57a0 20 read_log 193305'11400 (193305'11361) modify e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.951337.0:63754 2014-08-11 18:44:45.599149 2014-08-13 18:06:06.831802 7f5c973a57a0 20 read_log 194209'11401 (192888'11236) clean fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.15.0:775 2014-08-12 14:07:13.508568 2014-08-13 18:06:06.831814 7f5c973a57a0 20 read_log 194209'11402 (193305'11388) clean f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:776 2014-08-12 14:07:13.509140 2014-08-13 18:06:06.831827 7f5c973a57a0 20 read_log 194209'11403 (193305'11382) clean 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:777 2014-08-12 14:07:13.509583 2014-08-13 18:06:06.831839 7f5c973a57a0 20 read_log 194209'11404 (192834'10278) clean c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.15.0:778 2014-08-12 14:07:13.510067 2014-08-13 18:06:06.831851 7f5c973a57a0 20 read_log 194209'11405 (192834'10825) clean d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.15.0:779 2014-08-12 14:07:13.520780 2014-08-13 18:06:06.831863 7f5c973a57a0 20 read_log 194209'11406 (192888'11099) clean 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.15.0:780 2014-08-12 14:07:13.521054 2014-08-13 18:06:06.831885 7f5c973a57a0 20 read_log 194209'11407 (193305'11399) clean c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:781 2014-08-12 14:07:13.523929 2014-08-13 18:06:06.831901 7f5c973a57a0 20 read_log 194209'11408 (192888'11241) clean 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.15.0:782 2014-08-12 14:07:13.527062 2014-08-13 18:06:06.831913 7f5c973a57a0 20 read_log 194209'11409 (193305'11373) clean 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:783 2014-08-12 14:07:13.530489 2014-08-13 18:06:06.831925 7f5c973a57a0 20 read_log 194209'11410 (192834'10818) clean c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.15.0:784 2014-08-12 14:07:13.533540 2014-08-13 18:06:06.831965 7f5c973a57a0 20 read_log 194209'11411 (193305'11397) clean 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:785 2014-08-12 14:07:13.536595 2014-08-13 18:06:06.831981 7f5c973a57a0 20 read_log 194209'11412 (192888'11238) clean afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.15.0:786 2014-08-12 14:07:13.539602 2014-08-13 18:06:06.831993 7f5c973a57a0 20 read_log 194209'11413 (192834'10877) clean 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.15.0:787 2014-08-12 14:07:13.542717 2014-08-13 18:06:06.832005 7f5c973a57a0 20 read_log 194209'11414 (192834'10289) clean 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.15.0:788 2014-08-12 14:07:13.545965 2014-08-13 18:06:06.832018 7f5c973a57a0 20 read_log 194209'11415 (192888'11078) clean 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.15.0:789 2014-08-12 14:07:13.549035 2014-08-13 18:06:06.832051 7f5c973a57a0 20 read_log 194209'11416 (192834'10749) clean 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.15.0:790 2014-08-12 14:07:13.552077 2014-08-13 18:06:06.832064 7f5c973a57a0 20 read_log 194209'11417 (192888'11284) clean fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.15.0:791 2014-08-12 14:07:13.555170 2014-08-13 18:06:06.832077 7f5c973a57a0 20 read_log 194209'11418 (192888'11282) clean 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.15.0:792 2014-08-12 14:07:13.558286 2014-08-13 18:06:06.832091 7f5c973a57a0 20 read_log 194209'11419 (192888'11247) clean 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.15.0:793 2014-08-12 14:07:13.561518 2014-08-13 18:06:06.832103 7f5c973a57a0 20 read_log 194209'11420 (192834'10219) clean a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.15.0:794 2014-08-12 14:07:13.564502 2014-08-13 18:06:06.832116 7f5c973a57a0 20 read_log 194209'11421 (192888'11278) clean 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.15.0:795 2014-08-12 14:07:13.567278 2014-08-13 18:06:06.832131 7f5c973a57a0 20 read_log 194209'11422 (192888'11227) clean e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.15.0:796 2014-08-12 14:07:13.570324 2014-08-13 18:06:06.832143 7f5c973a57a0 20 read_log 194209'11423 (192834'10763) clean fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.15.0:797 2014-08-12 14:07:13.573380 2014-08-13 18:06:06.832163 7f5c973a57a0 20 read_log 194209'11424 (192888'11275) clean da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.15.0:798 2014-08-12 14:07:13.576481 2014-08-13 18:06:06.832176 7f5c973a57a0 20 read_log 194209'11425 (192834'10221) clean 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.15.0:799 2014-08-12 14:07:13.579897 2014-08-13 18:06:06.832188 7f5c973a57a0 20 read_log 194209'11426 (192834'10751) clean 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.15.0:800 2014-08-12 14:07:13.582798 2014-08-13 18:06:06.832201 7f5c973a57a0 20 read_log 194209'11427 (192888'11228) clean 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.15.0:801 2014-08-12 14:07:13.585929 2014-08-13 18:06:06.832215 7f5c973a57a0 20 read_log 194209'11428 (192834'10897) clean 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.15.0:802 2014-08-12 14:07:13.589011 2014-08-13 18:06:06.832227 7f5c973a57a0 20 read_log 194209'11429 (193305'11386) clean 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:803 2014-08-12 14:07:13.592169 2014-08-13 18:06:06.832242 7f5c973a57a0 20 read_log 194209'11430 (192834'10819) clean 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.15.0:804 2014-08-12 14:07:13.595319 2014-08-13 18:06:06.832262 7f5c973a57a0 20 read_log 194209'11431 (193305'11379) clean 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:805 2014-08-12 14:07:13.598700 2014-08-13 18:06:06.832276 7f5c973a57a0 20 read_log 194209'11432 (193305'11367) clean 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:806 2014-08-12 14:07:13.601399 2014-08-13 18:06:06.832289 7f5c973a57a0 20 read_log 194209'11433 (192888'11094) clean 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.15.0:807 2014-08-12 14:07:13.604360 2014-08-13 18:06:06.832304 7f5c973a57a0 20 read_log 194209'11434 (193305'11393) clean c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:808 2014-08-12 14:07:13.608100 2014-08-13 18:06:06.832317 7f5c973a57a0 20 read_log 194209'11435 (193305'11383) clean c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:809 2014-08-12 14:07:13.611578 2014-08-13 18:06:06.832329 7f5c973a57a0 20 read_log 194209'11436 (193305'11389) clean 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:810 2014-08-12 14:07:13.615255 2014-08-13 18:06:06.832341 7f5c973a57a0 20 read_log 194209'11437 (192888'11057) clean a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.15.0:811 2014-08-12 14:07:13.618592 2014-08-13 18:06:06.832361 7f5c973a57a0 20 read_log 194209'11438 (192888'11242) clean b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.15.0:812 2014-08-12 14:07:13.622045 2014-08-13 18:06:06.832374 7f5c973a57a0 20 read_log 194209'11439 (193305'11391) clean a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:813 2014-08-12 14:07:13.625416 2014-08-13 18:06:06.832387 7f5c973a57a0 20 read_log 194209'11440 (192834'10924) clean 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.15.0:814 2014-08-12 14:07:13.628971 2014-08-13 18:06:06.832402 7f5c973a57a0 20 read_log 194209'11441 (192888'11249) clean 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.15.0:815 2014-08-12 14:07:13.632462 2014-08-13 18:06:06.832418 7f5c973a57a0 20 read_log 194209'11442 (192834'10191) clean 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.15.0:816 2014-08-12 14:07:13.635702 2014-08-13 18:06:06.832441 7f5c973a57a0 20 read_log 194209'11443 (192834'10298) clean 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.15.0:817 2014-08-12 14:07:13.639361 2014-08-13 18:06:06.832461 7f5c973a57a0 20 read_log 194209'11444 (193305'11374) clean 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:818 2014-08-12 14:07:13.642685 2014-08-13 18:06:06.832474 7f5c973a57a0 20 read_log 194209'11445 (192888'11252) clean 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.15.0:819 2014-08-12 14:07:13.646067 2014-08-13 18:06:06.832496 7f5c973a57a0 20 read_log 194209'11446 (192888'11100) clean ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.15.0:820 2014-08-12 14:07:13.649838 2014-08-13 18:06:06.832509 7f5c973a57a0 20 read_log 194209'11447 (194209'11401) delete fbdf2e0e/benchmark_data_storage0111-ib.pocstor_2228_object4783/head//24 by osd.15.0:821 2014-08-12 14:07:13.686184 2014-08-13 18:06:06.832522 7f5c973a57a0 20 read_log 194209'11448 (194209'11402) delete f9cd3e0e/benchmark_data_storage0111-ib.pocstor_62621_object22055/head//24 by osd.15.0:822 2014-08-12 14:07:13.689564 2014-08-13 18:06:06.832542 7f5c973a57a0 20 read_log 194209'11449 (194209'11403) delete 56f04e0e/benchmark_data_storage0111-ib.pocstor_62621_object18534/head//24 by osd.15.0:823 2014-08-12 14:07:13.692683 2014-08-13 18:06:06.832555 7f5c973a57a0 20 read_log 194209'11450 (194209'11407) delete c9e05e0e/benchmark_data_storage0111-ib.pocstor_62621_object30847/head//24 by osd.15.0:824 2014-08-12 14:07:13.696172 2014-08-13 18:06:06.832602 7f5c973a57a0 20 read_log 194209'11451 (194209'11411) delete 15027e0e/benchmark_data_storage0111-ib.pocstor_62621_object30139/head//24 by osd.15.0:825 2014-08-12 14:07:13.699730 2014-08-13 18:06:06.832619 7f5c973a57a0 20 read_log 194209'11452 (194209'11429) delete 8a37ee0e/benchmark_data_storage0111-ib.pocstor_62621_object19316/head//24 by osd.15.0:826 2014-08-12 14:07:13.707082 2014-08-13 18:06:06.832638 7f5c973a57a0 20 read_log 194209'11453 (194209'11431) delete 4e9dee0e/benchmark_data_storage0111-ib.pocstor_62621_object15913/head//24 by osd.15.0:827 2014-08-12 14:07:13.710323 2014-08-13 18:06:06.832653 7f5c973a57a0 20 read_log 194209'11454 (194209'11434) delete c757fe0e/benchmark_data_storage0111-ib.pocstor_62621_object24510/head//24 by osd.15.0:828 2014-08-12 14:07:13.713637 2014-08-13 18:06:06.832665 7f5c973a57a0 20 read_log 194209'11455 (194209'11435) delete c932020e/benchmark_data_storage0111-ib.pocstor_62621_object18613/head//24 by osd.15.0:829 2014-08-12 14:07:13.716569 2014-08-13 18:06:06.832678 7f5c973a57a0 20 read_log 194209'11456 (194209'11436) delete 49bb020e/benchmark_data_storage0111-ib.pocstor_62621_object22368/head//24 by osd.15.0:830 2014-08-12 14:07:13.719984 2014-08-13 18:06:06.832702 7f5c973a57a0 20 read_log 194209'11457 (194209'11439) delete a546120e/benchmark_data_storage0111-ib.pocstor_62621_object23431/head//24 by osd.15.0:831 2014-08-12 14:07:13.723451 2014-08-13 18:06:06.832715 7f5c973a57a0 20 read_log 194209'11458 (192888'11096) clean 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.15.0:832 2014-08-12 14:07:13.796523 2014-08-13 18:06:06.832728 7f5c973a57a0 20 read_log 194209'11459 (194209'11425) delete 4a9fbe0e/benchmark_data_storage0111-ib.pocstor_26659_object12496/head//24 by osd.15.0:833 2014-08-12 14:07:18.799293 2014-08-13 18:06:06.832740 7f5c973a57a0 20 read_log 194209'11460 (194209'11426) delete 3e50de0e/benchmark_data_storage0111-ib.pocstor_41526_object2217/head//24 by osd.15.0:834 2014-08-12 14:07:18.799731 2014-08-13 18:06:06.832753 7f5c973a57a0 20 read_log 194209'11461 (194209'11427) delete 15f6de0e/benchmark_data_storage0111-ib.pocstor_2228_object554/head//24 by osd.15.0:835 2014-08-12 14:07:18.800157 2014-08-13 18:06:06.832765 7f5c973a57a0 20 read_log 194209'11462 (194209'11428) delete 991cde0e/benchmark_data_storage0111-ib.pocstor_41526_object18876/head//24 by osd.15.0:836 2014-08-12 14:07:18.800534 2014-08-13 18:06:06.832777 7f5c973a57a0 20 read_log 194209'11463 (194209'11430) delete 40faee0e/benchmark_data_storage0111-ib.pocstor_41526_object11786/head//24 by osd.15.0:837 2014-08-12 14:07:18.803704 2014-08-13 18:06:06.832809 7f5c973a57a0 20 read_log 194209'11464 (194209'11432) delete 3faeee0e/benchmark_data_storage0111-ib.pocstor_62621_object10977/head//24 by osd.15.0:838 2014-08-12 14:07:18.804135 2014-08-13 18:06:06.832822 7f5c973a57a0 20 read_log 194209'11465 (194209'11433) delete 59ffee0e/benchmark_data_storage0111-ib.pocstor_41097_object24716/head//24 by osd.15.0:839 2014-08-12 14:07:18.805226 2014-08-13 18:06:06.832835 7f5c973a57a0 20 read_log 194209'11466 (194209'11437) delete a692120e/benchmark_data_storage0111-ib.pocstor_41097_object3144/head//24 by osd.15.0:840 2014-08-12 14:07:18.808615 2014-08-13 18:06:06.832847 7f5c973a57a0 20 read_log 194209'11467 (194209'11438) delete b7e5120e/benchmark_data_storage0111-ib.pocstor_2228_object6767/head//24 by osd.15.0:841 2014-08-12 14:07:18.809008 2014-08-13 18:06:06.832859 7f5c973a57a0 20 read_log 194209'11468 (194209'11440) delete 3338120e/benchmark_data_storage0111-ib.pocstor_41526_object21685/head//24 by osd.15.0:842 2014-08-12 14:07:18.809931 2014-08-13 18:06:06.832872 7f5c973a57a0 20 read_log 194209'11469 (194209'11441) delete 58a6320e/benchmark_data_storage0111-ib.pocstor_2228_object11126/head//24 by osd.15.0:843 2014-08-12 14:07:18.810309 2014-08-13 18:06:06.832884 7f5c973a57a0 20 read_log 194209'11470 (194209'11442) delete 9f99320e/benchmark_data_storage0111-ib.pocstor_26659_object8050/head//24 by osd.15.0:844 2014-08-12 14:07:18.813682 2014-08-13 18:06:06.832903 7f5c973a57a0 20 read_log 194209'11471 (194209'11443) delete 141a320e/benchmark_data_storage0111-ib.pocstor_26659_object21043/head//24 by osd.15.0:845 2014-08-12 14:07:18.814103 2014-08-13 18:06:06.832916 7f5c973a57a0 20 read_log 194209'11472 (194209'11444) delete 47e320e/benchmark_data_storage0111-ib.pocstor_62621_object13506/head//24 by osd.15.0:846 2014-08-12 14:07:18.814852 2014-08-13 18:06:06.832928 7f5c973a57a0 20 read_log 194209'11473 (194209'11445) delete 73d1420e/benchmark_data_storage0111-ib.pocstor_2228_object13779/head//24 by osd.15.0:847 2014-08-12 14:07:18.815208 2014-08-13 18:06:06.832951 7f5c973a57a0 20 read_log 194209'11474 (194209'11446) delete ad98520e/benchmark_data_storage0111-ib.pocstor_41097_object27620/head//24 by osd.15.0:848 2014-08-12 14:07:18.819495 2014-08-13 18:06:06.832967 7f5c973a57a0 20 read_log 194209'11475 (194209'11458) delete 7de620e/benchmark_data_storage0111-ib.pocstor_41097_object25507/head//24 by osd.15.0:849 2014-08-12 14:07:18.819902 2014-08-13 18:06:06.832980 7f5c973a57a0 20 read_log 194209'11476 (194209'11404) delete c8944e0e/benchmark_data_storage0111-ib.pocstor_26659_object19225/head//24 by osd.15.0:850 2014-08-12 14:07:18.854504 2014-08-13 18:06:06.832992 7f5c973a57a0 20 read_log 194209'11477 (194209'11405) delete d3974e0e/benchmark_data_storage0111-ib.pocstor_41526_object12380/head//24 by osd.15.0:851 2014-08-12 14:07:18.855325 2014-08-13 18:06:06.833004 7f5c973a57a0 20 read_log 194209'11478 (194209'11406) delete 274a4e0e/benchmark_data_storage0111-ib.pocstor_41097_object27215/head//24 by osd.15.0:852 2014-08-12 14:07:18.855690 2014-08-13 18:06:06.833017 7f5c973a57a0 20 read_log 194209'11479 (194209'11408) delete 984a5e0e/benchmark_data_storage0111-ib.pocstor_2228_object6466/head//24 by osd.15.0:853 2014-08-12 14:07:18.856054 2014-08-13 18:06:06.833029 7f5c973a57a0 20 read_log 194209'11480 (194209'11409) delete 69c86e0e/benchmark_data_storage0111-ib.pocstor_62621_object13387/head//24 by osd.15.0:854 2014-08-12 14:07:18.859349 2014-08-13 18:06:06.833041 7f5c973a57a0 20 read_log 194209'11481 (194209'11410) delete c107e0e/benchmark_data_storage0111-ib.pocstor_41526_object11750/head//24 by osd.15.0:855 2014-08-12 14:07:18.859724 2014-08-13 18:06:06.833054 7f5c973a57a0 20 read_log 194209'11482 (194209'11412) delete afcf7e0e/benchmark_data_storage0111-ib.pocstor_2228_object4844/head//24 by osd.15.0:856 2014-08-12 14:07:18.860528 2014-08-13 18:06:06.833066 7f5c973a57a0 20 read_log 194209'11483 (194209'11413) delete 49b18e0e/benchmark_data_storage0111-ib.pocstor_41526_object17329/head//24 by osd.15.0:857 2014-08-12 14:07:18.860939 2014-08-13 18:06:06.833081 7f5c973a57a0 20 read_log 194209'11484 (194209'11414) delete 98cf8e0e/benchmark_data_storage0111-ib.pocstor_26659_object20375/head//24 by osd.15.0:858 2014-08-12 14:07:18.863104 2014-08-13 18:06:06.833094 7f5c973a57a0 20 read_log 194209'11485 (194209'11415) delete 9bd29e0e/benchmark_data_storage0111-ib.pocstor_41097_object16085/head//24 by osd.15.0:859 2014-08-12 14:07:18.863843 2014-08-13 18:06:06.833106 7f5c973a57a0 20 read_log 194209'11486 (194209'11416) delete 3c249e0e/benchmark_data_storage0111-ib.pocstor_41526_object1975/head//24 by osd.15.0:860 2014-08-12 14:07:18.864219 2014-08-13 18:06:06.833118 7f5c973a57a0 20 read_log 194209'11487 (194209'11417) delete fde0ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28564/head//24 by osd.15.0:861 2014-08-12 14:07:18.864563 2014-08-13 18:06:06.833137 7f5c973a57a0 20 read_log 194209'11488 (194209'11418) delete 7af2ae0e/benchmark_data_storage0111-ib.pocstor_2228_object28158/head//24 by osd.15.0:862 2014-08-12 14:07:18.868470 2014-08-13 18:06:06.833163 7f5c973a57a0 20 read_log 194209'11489 (194209'11419) delete 74b5ae0e/benchmark_data_storage0111-ib.pocstor_2228_object9867/head//24 by osd.15.0:863 2014-08-12 14:07:18.868930 2014-08-13 18:06:06.833183 7f5c973a57a0 20 read_log 194209'11490 (194209'11420) delete a817ae0e/benchmark_data_storage0111-ib.pocstor_26659_object12245/head//24 by osd.15.0:864 2014-08-12 14:07:18.869732 2014-08-13 18:06:06.833214 7f5c973a57a0 20 read_log 194209'11491 (194209'11421) delete 7cf8ae0e/benchmark_data_storage0111-ib.pocstor_2228_object26368/head//24 by osd.15.0:865 2014-08-12 14:07:18.870116 2014-08-13 18:06:06.833248 7f5c973a57a0 20 read_log 194209'11492 (194209'11422) delete e15eae0e/benchmark_data_storage0111-ib.pocstor_2228_object301/head//24 by osd.15.0:866 2014-08-12 14:07:18.872639 2014-08-13 18:06:06.833265 7f5c973a57a0 20 read_log 194209'11493 (194209'11423) delete fb1be0e/benchmark_data_storage0111-ib.pocstor_41526_object4457/head//24 by osd.15.0:867 2014-08-12 14:07:18.872927 2014-08-13 18:06:06.833277 7f5c973a57a0 20 read_log 194209'11494 (194209'11424) delete da3bbe0e/benchmark_data_storage0111-ib.pocstor_2228_object25315/head//24 by osd.15.0:868 2014-08-12 14:07:18.873733 2014-08-13 18:06:06.833297 7f5c973a57a0 20 read_log 194215'11495 (194215'11495) modify 20e/hit_set_24.20e_archive_2014-08-12 15:32:25.779939_2014-08-12 15:44:23.690307/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:23.690778 2014-08-13 18:06:06.833314 7f5c973a57a0 20 read_log 194215'11496 (193305'11378) delete 20e/hit_set_24.20e_archive_2014-08-11 18:39:56.767178_2014-08-11 18:44:41.106881/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:44:23.690778 2014-08-13 18:06:06.833327 7f5c973a57a0 20 read_log 194215'11497 (193305'11365) clean 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by osd.15.0:1653 2014-08-12 15:44:23.695696 2014-08-13 18:06:06.833340 7f5c973a57a0 20 read_log 194215'11498 (194215'11497) delete 5219720e/benchmark_data_storage0111-ib.pocstor_62621_object8605/head//24 by client.2174595.0:108597 0.000000 2014-08-13 18:06:06.833351 7f5c973a57a0 20 read_log 194215'11499 (193305'11380) clean 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by osd.15.0:1655 2014-08-12 15:44:23.703442 2014-08-13 18:06:06.833363 7f5c973a57a0 20 read_log 194215'11500 (194215'11499) delete 15aa820e/benchmark_data_storage0111-ib.pocstor_62621_object16189/head//24 by client.2174595.0:108599 0.000000 2014-08-13 18:06:06.833374 7f5c973a57a0 20 read_log 194215'11501 (193305'11394) clean 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by osd.15.0:1657 2014-08-12 15:44:23.711235 2014-08-13 18:06:06.833387 7f5c973a57a0 20 read_log 194215'11502 (194215'11501) delete 88a5920e/benchmark_data_storage0111-ib.pocstor_62621_object25354/head//24 by client.2174595.0:108601 0.000000 2014-08-13 18:06:06.833398 7f5c973a57a0 20 read_log 194215'11503 (192834'10854) clean c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by osd.15.0:1659 2014-08-12 15:44:23.718791 2014-08-13 18:06:06.833428 7f5c973a57a0 20 read_log 194215'11504 (194215'11503) delete c32e920e/benchmark_data_storage0111-ib.pocstor_41526_object14879/head//24 by client.2174595.0:108603 0.000000 2014-08-13 18:06:06.833441 7f5c973a57a0 20 read_log 194215'11505 (192888'11080) clean 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by osd.15.0:1661 2014-08-12 15:44:23.727698 2014-08-13 18:06:06.833454 7f5c973a57a0 20 read_log 194215'11506 (194215'11505) delete 2c80b20e/benchmark_data_storage0111-ib.pocstor_41097_object16740/head//24 by client.2174595.0:108605 0.000000 2014-08-13 18:06:06.833465 7f5c973a57a0 20 read_log 194215'11507 (192888'11273) clean a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by osd.15.0:1663 2014-08-12 15:44:23.735669 2014-08-13 18:06:06.833477 7f5c973a57a0 20 read_log 194215'11508 (194215'11507) delete a2e0b20e/benchmark_data_storage0111-ib.pocstor_2228_object24187/head//24 by client.2174595.0:108607 0.000000 2014-08-13 18:06:06.833488 7f5c973a57a0 20 read_log 194215'11509 (192888'11243) clean 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by osd.15.0:1665 2014-08-12 15:44:23.743510 2014-08-13 18:06:06.833510 7f5c973a57a0 20 read_log 194215'11510 (194215'11509) delete 6771b20e/benchmark_data_storage0111-ib.pocstor_2228_object7025/head//24 by client.2174595.0:108609 0.000000 2014-08-13 18:06:06.833522 7f5c973a57a0 20 read_log 194215'11511 (192888'11089) clean c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by osd.15.0:1667 2014-08-12 15:44:23.751776 2014-08-13 18:06:06.833538 7f5c973a57a0 20 read_log 194215'11512 (194215'11511) delete c5a2b20e/benchmark_data_storage0111-ib.pocstor_41097_object19715/head//24 by client.2174595.0:108611 0.000000 2014-08-13 18:06:06.833549 7f5c973a57a0 20 read_log 194215'11513 (193305'11384) clean b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by osd.15.0:1669 2014-08-12 15:44:23.759007 2014-08-13 18:06:06.833561 7f5c973a57a0 20 read_log 194215'11514 (194215'11513) delete b765d20e/benchmark_data_storage0111-ib.pocstor_62621_object18892/head//24 by client.2174595.0:108613 0.000000 2014-08-13 18:06:06.833572 7f5c973a57a0 20 read_log 194215'11515 (192888'11075) clean 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by osd.15.0:1671 2014-08-12 15:44:23.766640 2014-08-13 18:06:06.833584 7f5c973a57a0 20 read_log 194215'11516 (194215'11515) delete 7726d20e/benchmark_data_storage0111-ib.pocstor_41097_object14756/head//24 by client.2174595.0:108615 0.000000 2014-08-13 18:06:06.833596 7f5c973a57a0 20 read_log 194215'11517 (192888'11229) clean 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by osd.15.0:1673 2014-08-12 15:44:23.774186 2014-08-13 18:06:06.833608 7f5c973a57a0 20 read_log 194215'11518 (194215'11517) delete 4b2e20e/benchmark_data_storage0111-ib.pocstor_2228_object886/head//24 by client.2174595.0:108617 0.000000 2014-08-13 18:06:06.833618 7f5c973a57a0 20 read_log 194215'11519 (193305'11369) clean 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by osd.15.0:1675 2014-08-12 15:44:23.781559 2014-08-13 18:06:06.833631 7f5c973a57a0 20 read_log 194215'11520 (194215'11519) delete 9d06f20e/benchmark_data_storage0111-ib.pocstor_62621_object12927/head//24 by client.2174595.0:108619 0.000000 2014-08-13 18:06:06.833642 7f5c973a57a0 20 read_log 194215'11521 (193305'11390) clean 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by osd.15.0:1677 2014-08-12 15:44:23.789860 2014-08-13 18:06:06.833654 7f5c973a57a0 20 read_log 194215'11522 (194215'11521) delete 72e6f20e/benchmark_data_storage0111-ib.pocstor_62621_object22883/head//24 by client.2174595.0:108621 0.000000 2014-08-13 18:06:06.833665 7f5c973a57a0 20 read_log 194215'11523 (192888'11059) clean 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by osd.15.0:1679 2014-08-12 15:44:23.797328 2014-08-13 18:06:06.833677 7f5c973a57a0 20 read_log 194215'11524 (194215'11523) delete 71cdf20e/benchmark_data_storage0111-ib.pocstor_41097_object4730/head//24 by client.2174595.0:108623 0.000000 2014-08-13 18:06:06.833691 7f5c973a57a0 20 read_log 194215'11525 (192834'10786) clean 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by osd.15.0:1681 2014-08-12 15:44:23.806196 2014-08-13 18:06:06.833704 7f5c973a57a0 20 read_log 194215'11526 (194215'11525) delete 33edf20e/benchmark_data_storage0111-ib.pocstor_41526_object8071/head//24 by client.2174595.0:108625 0.000000 2014-08-13 18:06:06.833717 7f5c973a57a0 20 read_log 194215'11527 (192834'10271) clean ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by osd.15.0:1683 2014-08-12 15:44:23.813212 2014-08-13 18:06:06.833729 7f5c973a57a0 20 read_log 194215'11528 (194215'11527) delete ea5ef20e/benchmark_data_storage0111-ib.pocstor_26659_object18489/head//24 by client.2174595.0:108627 0.000000 2014-08-13 18:06:06.833742 7f5c973a57a0 20 read_log 194215'11529 (193305'11396) clean ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by osd.15.0:1685 2014-08-12 15:44:23.820878 2014-08-13 18:06:06.833761 7f5c973a57a0 20 read_log 194215'11530 (194215'11529) delete ed3b060e/benchmark_data_storage0111-ib.pocstor_62621_object29028/head//24 by client.2174595.0:108629 0.000000 2014-08-13 18:06:06.833774 7f5c973a57a0 20 read_log 194215'11531 (192888'11090) clean e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by osd.15.0:1687 2014-08-12 15:44:23.828834 2014-08-13 18:06:06.833787 7f5c973a57a0 20 read_log 194215'11532 (194215'11531) delete e8e1160e/benchmark_data_storage0111-ib.pocstor_41097_object20775/head//24 by client.2174595.0:108631 0.000000 2014-08-13 18:06:06.833798 7f5c973a57a0 20 read_log 194215'11533 (192834'10192) clean b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by osd.15.0:1689 2014-08-12 15:44:23.836042 2014-08-13 18:06:06.833810 7f5c973a57a0 20 read_log 194215'11534 (194215'11533) delete b6a6160e/benchmark_data_storage0111-ib.pocstor_26659_object8260/head//24 by client.2174595.0:108633 0.000000 2014-08-13 18:06:06.833821 7f5c973a57a0 20 read_log 194215'11535 (192888'11261) clean b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by osd.15.0:1691 2014-08-12 15:44:23.844244 2014-08-13 18:06:06.833856 7f5c973a57a0 20 read_log 194215'11536 (194215'11535) delete b8d9160e/benchmark_data_storage0111-ib.pocstor_2228_object19651/head//24 by client.2174595.0:108635 0.000000 2014-08-13 18:06:06.833870 7f5c973a57a0 20 read_log 194215'11537 (192888'11268) clean e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by osd.15.0:1693 2014-08-12 15:44:23.852194 2014-08-13 18:06:06.833883 7f5c973a57a0 20 read_log 194215'11538 (194215'11537) delete e76c160e/benchmark_data_storage0111-ib.pocstor_2228_object21943/head//24 by client.2174595.0:108637 0.000000 2014-08-13 18:06:06.833896 7f5c973a57a0 20 read_log 194215'11539 (192888'11054) clean ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by osd.15.0:1695 2014-08-12 15:44:23.859365 2014-08-13 18:06:06.833910 7f5c973a57a0 20 read_log 194215'11540 (194215'11539) delete ecce160e/benchmark_data_storage0111-ib.pocstor_41097_object2503/head//24 by client.2174595.0:108639 0.000000 2014-08-13 18:06:06.833921 7f5c973a57a0 20 read_log 194215'11541 (193305'11392) clean 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by osd.15.0:1697 2014-08-12 15:44:23.867263 2014-08-13 18:06:06.833935 7f5c973a57a0 20 read_log 194215'11542 (194215'11541) delete 56b5260e/benchmark_data_storage0111-ib.pocstor_62621_object23499/head//24 by client.2174595.0:108641 0.000000 2014-08-13 18:06:06.833946 7f5c973a57a0 20 read_log 194215'11543 (192888'11277) clean d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by osd.15.0:1699 2014-08-12 15:44:23.874398 2014-08-13 18:06:06.833958 7f5c973a57a0 20 read_log 194215'11544 (194215'11543) delete d3c5260e/benchmark_data_storage0111-ib.pocstor_2228_object25852/head//24 by client.2174595.0:108643 0.000000 2014-08-13 18:06:06.833971 7f5c973a57a0 20 read_log 194215'11545 (192834'10787) clean 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by osd.15.0:1701 2014-08-12 15:44:23.881815 2014-08-13 18:06:06.833986 7f5c973a57a0 20 read_log 194215'11546 (194215'11545) delete 599b260e/benchmark_data_storage0111-ib.pocstor_41526_object8144/head//24 by client.2174595.0:108645 0.000000 2014-08-13 18:06:06.833997 7f5c973a57a0 20 read_log 194215'11547 (192888'11260) clean f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by osd.15.0:1703 2014-08-12 15:44:23.890787 2014-08-13 18:06:06.834010 7f5c973a57a0 20 read_log 194215'11548 (194215'11547) delete f85c260e/benchmark_data_storage0111-ib.pocstor_2228_object19398/head//24 by client.2174595.0:108647 0.000000 2014-08-13 18:06:06.834023 7f5c973a57a0 20 read_log 194215'11549 (193305'11362) clean 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by osd.15.0:1705 2014-08-12 15:44:23.898094 2014-08-13 18:06:06.834042 7f5c973a57a0 20 read_log 194215'11550 (194215'11549) delete 89b8360e/benchmark_data_storage0111-ib.pocstor_62621_object4007/head//24 by client.2174595.0:108649 0.000000 2014-08-13 18:06:06.834056 7f5c973a57a0 20 read_log 194215'11551 (193305'11368) clean e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by osd.15.0:1707 2014-08-12 15:44:23.905932 2014-08-13 18:06:06.834068 7f5c973a57a0 20 read_log 194215'11552 (194215'11551) delete e7b9360e/benchmark_data_storage0111-ib.pocstor_62621_object11539/head//24 by client.2174595.0:108651 0.000000 2014-08-13 18:06:06.834079 7f5c973a57a0 20 read_log 194215'11553 (192888'11276) clean 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by osd.15.0:1709 2014-08-12 15:44:23.913994 2014-08-13 18:06:06.834093 7f5c973a57a0 20 read_log 194215'11554 (194215'11553) delete 1138460e/benchmark_data_storage0111-ib.pocstor_2228_object25371/head//24 by client.2174595.0:108653 0.000000 2014-08-13 18:06:06.834104 7f5c973a57a0 20 read_log 194215'11555 (192888'11079) clean 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by osd.15.0:1711 2014-08-12 15:44:23.921971 2014-08-13 18:06:06.834117 7f5c973a57a0 20 read_log 194215'11556 (194215'11555) delete 83ea460e/benchmark_data_storage0111-ib.pocstor_41097_object16223/head//24 by client.2174595.0:108655 0.000000 2014-08-13 18:06:06.834130 7f5c973a57a0 20 read_log 194215'11557 (192834'10865) clean abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by osd.15.0:1713 2014-08-12 15:44:23.929997 2014-08-13 18:06:06.834142 7f5c973a57a0 20 read_log 194215'11558 (194215'11557) delete abeb460e/benchmark_data_storage0111-ib.pocstor_41526_object15707/head//24 by client.2174595.0:108657 0.000000 2014-08-13 18:06:06.834161 7f5c973a57a0 20 read_log 194215'11559 (192834'10265) clean 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by osd.15.0:1715 2014-08-12 15:44:23.937568 2014-08-13 18:06:06.834174 7f5c973a57a0 20 read_log 194215'11560 (194215'11559) delete 48e1560e/benchmark_data_storage0111-ib.pocstor_26659_object18020/head//24 by client.2174595.0:108659 0.000000 2014-08-13 18:06:06.834185 7f5c973a57a0 20 read_log 194215'11561 (192834'10866) clean 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by osd.15.0:1717 2014-08-12 15:44:23.945625 2014-08-13 18:06:06.834197 7f5c973a57a0 20 read_log 194215'11562 (194215'11561) delete 4d73560e/benchmark_data_storage0111-ib.pocstor_41526_object15958/head//24 by client.2174595.0:108661 0.000000 2014-08-13 18:06:06.834208 7f5c973a57a0 20 read_log 194215'11563 (192888'11271) clean 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by osd.15.0:1719 2014-08-12 15:44:23.952559 2014-08-13 18:06:06.834223 7f5c973a57a0 20 read_log 194215'11564 (194215'11563) delete 6087560e/benchmark_data_storage0111-ib.pocstor_2228_object23344/head//24 by client.2174595.0:108663 0.000000 2014-08-13 18:06:06.834234 7f5c973a57a0 20 read_log 194215'11565 (192888'11063) clean bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by osd.15.0:1721 2014-08-12 15:44:23.959569 2014-08-13 18:06:06.834246 7f5c973a57a0 20 read_log 194215'11566 (194215'11565) delete bb03660e/benchmark_data_storage0111-ib.pocstor_41097_object6511/head//24 by client.2174595.0:108665 0.000000 2014-08-13 18:06:06.834259 7f5c973a57a0 20 read_log 194215'11567 (192888'11082) clean 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by osd.15.0:1723 2014-08-12 15:44:23.966555 2014-08-13 18:06:06.834274 7f5c973a57a0 20 read_log 194215'11568 (194215'11567) delete 239d660e/benchmark_data_storage0111-ib.pocstor_41097_object17391/head//24 by client.2174595.0:108667 0.000000 2014-08-13 18:06:06.834287 7f5c973a57a0 20 read_log 194215'11569 (193305'11363) clean ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by osd.15.0:1725 2014-08-12 15:44:23.974109 2014-08-13 18:06:06.834308 7f5c973a57a0 20 read_log 194215'11570 (194215'11569) delete ff00760e/benchmark_data_storage0111-ib.pocstor_62621_object4449/head//24 by client.2174595.0:108669 0.000000 2014-08-13 18:06:06.834328 7f5c973a57a0 20 read_log 194215'11571 (192834'10207) clean 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by osd.15.0:1727 2014-08-12 15:44:23.981880 2014-08-13 18:06:06.834341 7f5c973a57a0 20 read_log 194215'11572 (194215'11571) delete 6b02760e/benchmark_data_storage0111-ib.pocstor_26659_object10456/head//24 by client.2174595.0:108671 0.000000 2014-08-13 18:06:06.834376 7f5c973a57a0 20 read_log 194215'11573 (192888'11265) clean 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by osd.15.0:1729 2014-08-12 15:44:23.989563 2014-08-13 18:06:06.834399 7f5c973a57a0 20 read_log 194215'11574 (194215'11573) delete 3f9a760e/benchmark_data_storage0111-ib.pocstor_2228_object21714/head//24 by client.2174595.0:108673 0.000000 2014-08-13 18:06:06.834416 7f5c973a57a0 20 read_log 194215'11575 (192834'10776) clean 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by osd.15.0:1731 2014-08-12 15:44:23.996440 2014-08-13 18:06:06.834431 7f5c973a57a0 20 read_log 194215'11576 (194215'11575) delete 553c760e/benchmark_data_storage0111-ib.pocstor_41526_object6127/head//24 by client.2174595.0:108675 0.000000 2014-08-13 18:06:06.834442 7f5c973a57a0 20 read_log 194215'11577 (192888'11240) clean 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by osd.15.0:1733 2014-08-12 15:44:24.003833 2014-08-13 18:06:06.834465 7f5c973a57a0 20 read_log 194215'11578 (194215'11577) delete 9be760e/benchmark_data_storage0111-ib.pocstor_2228_object5593/head//24 by client.2174595.0:108677 0.000000 2014-08-13 18:06:06.834477 7f5c973a57a0 20 read_log 194215'11579 (192834'10288) clean 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by osd.15.0:1735 2014-08-12 15:44:24.011536 2014-08-13 18:06:06.834490 7f5c973a57a0 20 read_log 194215'11580 (194215'11579) delete 9013860e/benchmark_data_storage0111-ib.pocstor_26659_object20322/head//24 by client.2174595.0:108679 0.000000 2014-08-13 18:06:06.834509 7f5c973a57a0 20 read_log 194215'11581 (192888'11272) clean ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by osd.15.0:1737 2014-08-12 15:44:24.018670 2014-08-13 18:06:06.834524 7f5c973a57a0 20 read_log 194215'11582 (194215'11581) delete ae53860e/benchmark_data_storage0111-ib.pocstor_2228_object24056/head//24 by client.2174595.0:108681 0.000000 2014-08-13 18:06:06.834535 7f5c973a57a0 20 read_log 194215'11583 (193305'11376) clean 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by osd.15.0:1739 2014-08-12 15:44:24.026864 2014-08-13 18:06:06.834557 7f5c973a57a0 20 read_log 194215'11584 (194215'11583) delete 34b5860e/benchmark_data_storage0111-ib.pocstor_62621_object15087/head//24 by client.2174595.0:108683 0.000000 2014-08-13 18:06:06.834572 7f5c973a57a0 20 read_log 194215'11585 (193305'11370) clean 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by osd.15.0:1741 2014-08-12 15:44:24.034673 2014-08-13 18:06:06.834585 7f5c973a57a0 20 read_log 194215'11586 (194215'11585) delete 9038860e/benchmark_data_storage0111-ib.pocstor_62621_object12958/head//24 by client.2174595.0:108685 0.000000 2014-08-13 18:06:06.834605 7f5c973a57a0 20 read_log 194215'11587 (193305'11375) clean 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by osd.15.0:1743 2014-08-12 15:44:24.042017 2014-08-13 18:06:06.834626 7f5c973a57a0 20 read_log 194215'11588 (194215'11587) delete 4923960e/benchmark_data_storage0111-ib.pocstor_62621_object13560/head//24 by client.2174595.0:108687 0.000000 2014-08-13 18:06:06.834637 7f5c973a57a0 20 read_log 194215'11589 (193305'11385) clean bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by osd.15.0:1745 2014-08-12 15:44:24.050215 2014-08-13 18:06:06.834674 7f5c973a57a0 20 read_log 194215'11590 (194215'11589) delete bf01a60e/benchmark_data_storage0111-ib.pocstor_62621_object19269/head//24 by client.2174595.0:108689 0.000000 2014-08-13 18:06:06.834689 7f5c973a57a0 20 read_log 194215'11591 (192834'10804) clean c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by osd.15.0:1747 2014-08-12 15:44:24.057930 2014-08-13 18:06:06.834702 7f5c973a57a0 20 read_log 194215'11592 (194215'11591) delete c5b4a60e/benchmark_data_storage0111-ib.pocstor_41526_object9934/head//24 by client.2174595.0:108691 0.000000 2014-08-13 18:06:06.834713 7f5c973a57a0 20 read_log 194215'11593 (192888'11245) clean 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by osd.15.0:1749 2014-08-12 15:44:24.066067 2014-08-13 18:06:06.834737 7f5c973a57a0 20 read_log 194215'11594 (194215'11593) delete 9dcca60e/benchmark_data_storage0111-ib.pocstor_2228_object8662/head//24 by client.2174595.0:108693 0.000000 2014-08-13 18:06:06.834749 7f5c973a57a0 20 read_log 194215'11595 (192834'10210) clean eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by osd.15.0:1751 2014-08-12 15:44:24.073248 2014-08-13 18:06:06.834762 7f5c973a57a0 20 read_log 194215'11596 (194215'11595) delete eca2b60e/benchmark_data_storage0111-ib.pocstor_26659_object11244/head//24 by client.2174595.0:108695 0.000000 2014-08-13 18:06:06.834797 7f5c973a57a0 20 read_log 194215'11597 (192888'11087) clean 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by osd.15.0:1753 2014-08-12 15:44:24.080719 2014-08-13 18:06:06.834813 7f5c973a57a0 20 read_log 194215'11598 (194215'11597) delete 2018b60e/benchmark_data_storage0111-ib.pocstor_41097_object18569/head//24 by client.2174595.0:108697 0.000000 2014-08-13 18:06:06.834824 7f5c973a57a0 20 read_log 194215'11599 (193305'11387) clean ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by osd.15.0:1755 2014-08-12 15:44:24.088721 2014-08-13 18:06:06.834837 7f5c973a57a0 20 read_log 194215'11600 (194215'11599) delete ed23c60e/benchmark_data_storage0111-ib.pocstor_62621_object21182/head//24 by client.2174595.0:108699 0.000000 2014-08-13 18:06:06.834847 7f5c973a57a0 20 read_log 194215'11601 (192834'10259) clean a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by osd.15.0:1757 2014-08-12 15:44:24.095933 2014-08-13 18:06:06.834860 7f5c973a57a0 20 read_log 194215'11602 (194215'11601) delete a489d60e/benchmark_data_storage0111-ib.pocstor_26659_object17316/head//24 by client.2174595.0:108701 0.000000 2014-08-13 18:06:06.834871 7f5c973a57a0 20 read_log 194215'11603 (193305'11366) clean 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by osd.15.0:1759 2014-08-12 15:44:24.104288 2014-08-13 18:06:06.834883 7f5c973a57a0 20 read_log 194215'11604 (194215'11603) delete 206cf60e/benchmark_data_storage0111-ib.pocstor_62621_object9224/head//24 by client.2174595.0:108703 0.000000 2014-08-13 18:06:06.834894 7f5c973a57a0 20 read_log 194215'11605 (192834'10153) clean b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by osd.15.0:1761 2014-08-12 15:44:24.112055 2014-08-13 18:06:06.834906 7f5c973a57a0 20 read_log 194215'11606 (194215'11605) delete b8390a0e/benchmark_data_storage0111-ib.pocstor_26659_object1473/head//24 by client.2174595.0:108705 0.000000 2014-08-13 18:06:06.834917 7f5c973a57a0 20 read_log 194215'11607 (192834'10238) clean 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by osd.15.0:1763 2014-08-12 15:44:24.119164 2014-08-13 18:06:06.834929 7f5c973a57a0 20 read_log 194215'11608 (194215'11607) delete 6b491a0e/benchmark_data_storage0111-ib.pocstor_26659_object14073/head//24 by client.2174595.0:108707 0.000000 2014-08-13 18:06:06.834940 7f5c973a57a0 20 read_log 194215'11609 (192888'11285) clean dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by osd.15.0:1765 2014-08-12 15:44:24.127231 2014-08-13 18:06:06.834959 7f5c973a57a0 20 read_log 194215'11610 (194215'11609) delete dc962a0e/benchmark_data_storage0111-ib.pocstor_2228_object29251/head//24 by client.2174595.0:108709 0.000000 2014-08-13 18:06:06.834970 7f5c973a57a0 20 read_log 194215'11611 (192888'11253) clean 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by osd.15.0:1767 2014-08-12 15:44:24.134463 2014-08-13 18:06:06.834983 7f5c973a57a0 20 read_log 194215'11612 (194215'11611) delete 2ed72a0e/benchmark_data_storage0111-ib.pocstor_2228_object14179/head//24 by client.2174595.0:108711 0.000000 2014-08-13 18:06:06.834993 7f5c973a57a0 20 read_log 194215'11613 (192888'11076) clean 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by osd.15.0:1769 2014-08-12 15:44:24.144339 2014-08-13 18:06:06.835006 7f5c973a57a0 20 read_log 194215'11614 (194215'11613) delete 9f82a0e/benchmark_data_storage0111-ib.pocstor_41097_object14802/head//24 by client.2174595.0:108713 0.000000 2014-08-13 18:06:06.835026 7f5c973a57a0 20 read_log 194215'11615 (193305'11364) clean 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by osd.15.0:1771 2014-08-12 15:44:24.152555 2014-08-13 18:06:06.835039 7f5c973a57a0 20 read_log 194215'11616 (194215'11615) delete 51e53a0e/benchmark_data_storage0111-ib.pocstor_62621_object4740/head//24 by client.2174595.0:108715 0.000000 2014-08-13 18:06:06.835050 7f5c973a57a0 20 read_log 194215'11617 (192888'11091) clean 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by osd.15.0:1773 2014-08-12 15:44:24.160405 2014-08-13 18:06:06.835062 7f5c973a57a0 20 read_log 194215'11618 (194215'11617) delete 84e54a0e/benchmark_data_storage0111-ib.pocstor_41097_object22846/head//24 by client.2174595.0:108717 0.000000 2014-08-13 18:06:06.835073 7f5c973a57a0 20 read_log 194215'11619 (192834'10250) clean 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by osd.15.0:1775 2014-08-12 15:44:24.167468 2014-08-13 18:06:06.835086 7f5c973a57a0 20 read_log 194215'11620 (194215'11619) delete 5db64a0e/benchmark_data_storage0111-ib.pocstor_26659_object15948/head//24 by client.2174595.0:108719 0.000000 2014-08-13 18:06:06.835097 7f5c973a57a0 20 read_log 194215'11621 (192888'11101) clean 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by osd.15.0:1777 2014-08-12 15:44:24.175134 2014-08-13 18:06:06.835109 7f5c973a57a0 20 read_log 194215'11622 (194215'11621) delete 1e995a0e/benchmark_data_storage0111-ib.pocstor_41097_object27699/head//24 by client.2174595.0:108721 0.000000 2014-08-13 18:06:06.835120 7f5c973a57a0 20 read_log 194215'11623 (192888'11279) clean d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by osd.15.0:1779 2014-08-12 15:44:24.182625 2014-08-13 18:06:06.835132 7f5c973a57a0 20 read_log 194215'11624 (194215'11623) delete d9ea5a0e/benchmark_data_storage0111-ib.pocstor_2228_object27105/head//24 by client.2174595.0:108723 0.000000 2014-08-13 18:06:06.835143 7f5c973a57a0 20 read_log 194215'11625 (192888'11070) clean 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by osd.15.0:1781 2014-08-12 15:44:24.189427 2014-08-13 18:06:06.835164 7f5c973a57a0 20 read_log 194215'11626 (194215'11625) delete 90ab5a0e/benchmark_data_storage0111-ib.pocstor_41097_object10385/head//24 by client.2174595.0:108725 0.000000 2014-08-13 18:06:06.835175 7f5c973a57a0 20 read_log 194215'11627 (192834'10741) clean 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by osd.15.0:1783 2014-08-12 15:44:24.197253 2014-08-13 18:06:06.835188 7f5c973a57a0 20 read_log 194215'11628 (194215'11627) delete 9a826a0e/benchmark_data_storage0111-ib.pocstor_41526_object1027/head//24 by client.2174595.0:108727 0.000000 2014-08-13 18:06:06.835198 7f5c973a57a0 20 read_log 194215'11629 (192888'11074) clean cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by osd.15.0:1785 2014-08-12 15:44:24.204418 2014-08-13 18:06:06.835223 7f5c973a57a0 20 read_log 194215'11630 (194215'11629) delete cf886a0e/benchmark_data_storage0111-ib.pocstor_41097_object14357/head//24 by client.2174595.0:108729 0.000000 2014-08-13 18:06:06.835236 7f5c973a57a0 20 read_log 194215'11631 (192888'11085) clean d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by osd.15.0:1787 2014-08-12 15:44:24.212449 2014-08-13 18:06:06.835249 7f5c973a57a0 20 read_log 194215'11632 (194215'11631) delete d8837a0e/benchmark_data_storage0111-ib.pocstor_41097_object18511/head//24 by client.2174595.0:108731 0.000000 2014-08-13 18:06:06.835264 7f5c973a57a0 20 read_log 194215'11633 (192834'10925) clean 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by osd.15.0:1789 2014-08-12 15:44:24.220157 2014-08-13 18:06:06.835277 7f5c973a57a0 20 read_log 194215'11634 (194215'11633) delete 49357a0e/benchmark_data_storage0111-ib.pocstor_41526_object22091/head//24 by client.2174595.0:108733 0.000000 2014-08-13 18:06:06.835288 7f5c973a57a0 20 read_log 194215'11635 (192834'10300) clean b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by osd.15.0:1791 2014-08-12 15:44:24.227125 2014-08-13 18:06:06.835300 7f5c973a57a0 20 read_log 194215'11636 (194215'11635) delete b0e7a0e/benchmark_data_storage0111-ib.pocstor_26659_object21579/head//24 by client.2174595.0:108735 0.000000 2014-08-13 18:06:06.835315 7f5c973a57a0 20 read_log 194215'11637 (192834'10214) clean 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by osd.15.0:1793 2014-08-12 15:44:24.232087 2014-08-13 18:06:06.835328 7f5c973a57a0 20 read_log 194215'11638 (194215'11637) delete 76808a0e/benchmark_data_storage0111-ib.pocstor_26659_object11589/head//24 by client.2174595.0:108737 0.000000 2014-08-13 18:06:06.835339 7f5c973a57a0 20 read_log 194215'11639 (192888'11098) clean bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by osd.15.0:1795 2014-08-12 15:44:24.238256 2014-08-13 18:06:06.835351 7f5c973a57a0 20 read_log 194215'11640 (194215'11639) delete bbb28a0e/benchmark_data_storage0111-ib.pocstor_41097_object26128/head//24 by client.2174595.0:108739 0.000000 2014-08-13 18:06:06.835365 7f5c973a57a0 20 read_log 194215'11641 (192834'10261) clean ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by osd.15.0:1797 2014-08-12 15:44:24.244038 2014-08-13 18:06:06.835377 7f5c973a57a0 20 read_log 194215'11642 (194215'11641) delete ac798a0e/benchmark_data_storage0111-ib.pocstor_26659_object17618/head//24 by client.2174595.0:108741 0.000000 2014-08-13 18:06:06.835388 7f5c973a57a0 20 read_log 194215'11643 (193305'11372) clean d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by osd.15.0:1799 2014-08-12 15:44:24.250228 2014-08-13 18:06:06.835415 7f5c973a57a0 20 read_log 194215'11644 (194215'11643) delete d249a0e/benchmark_data_storage0111-ib.pocstor_62621_object13357/head//24 by client.2174595.0:108743 0.000000 2014-08-13 18:06:06.835427 7f5c973a57a0 20 read_log 194215'11645 (193305'11371) clean d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by osd.15.0:1801 2014-08-12 15:44:24.256859 2014-08-13 18:06:06.835439 7f5c973a57a0 20 read_log 194215'11646 (194215'11645) delete d1f69a0e/benchmark_data_storage0111-ib.pocstor_62621_object13089/head//24 by client.2174595.0:108745 0.000000 2014-08-13 18:06:06.835463 7f5c973a57a0 20 read_log 194215'11647 (192834'10255) clean b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by osd.15.0:1803 2014-08-12 15:44:24.262535 2014-08-13 18:06:06.835476 7f5c973a57a0 20 read_log 194215'11648 (194215'11647) delete b8d79a0e/benchmark_data_storage0111-ib.pocstor_26659_object16699/head//24 by client.2174595.0:108747 0.000000 2014-08-13 18:06:06.835487 7f5c973a57a0 20 read_log 194215'11649 (192888'11055) clean 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by osd.15.0:1805 2014-08-12 15:44:24.268366 2014-08-13 18:06:06.835516 7f5c973a57a0 20 read_log 194215'11650 (194215'11649) delete 7fcd9a0e/benchmark_data_storage0111-ib.pocstor_41097_object2592/head//24 by client.2174595.0:108749 0.000000 2014-08-13 18:06:06.835530 7f5c973a57a0 20 read_log 194215'11651 (192888'11095) clean c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by osd.15.0:1807 2014-08-12 15:44:24.274211 2014-08-13 18:06:06.835543 7f5c973a57a0 20 read_log 194215'11652 (194215'11651) delete c4b4aa0e/benchmark_data_storage0111-ib.pocstor_41097_object25271/head//24 by client.2174595.0:108751 0.000000 2014-08-13 18:06:06.835554 7f5c973a57a0 20 read_log 194215'11653 (192834'10885) clean 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by osd.15.0:1809 2014-08-12 15:44:24.280454 2014-08-13 18:06:06.835568 7f5c973a57a0 20 read_log 194215'11654 (194215'11653) delete 5365ca0e/benchmark_data_storage0111-ib.pocstor_41526_object18038/head//24 by client.2174595.0:108753 0.000000 2014-08-13 18:06:06.835579 7f5c973a57a0 20 read_log 194215'11655 (193305'11395) clean ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by osd.15.0:1811 2014-08-12 15:44:24.286026 2014-08-13 18:06:06.835599 7f5c973a57a0 20 read_log 194215'11656 (194215'11655) delete ee2aca0e/benchmark_data_storage0111-ib.pocstor_62621_object28433/head//24 by client.2174595.0:108755 0.000000 2014-08-13 18:06:06.835623 7f5c973a57a0 20 read_log 194215'11657 (193305'11398) clean 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by osd.15.0:1813 2014-08-12 15:44:24.291561 2014-08-13 18:06:06.835639 7f5c973a57a0 20 read_log 194215'11658 (194215'11657) delete 36cdca0e/benchmark_data_storage0111-ib.pocstor_62621_object30385/head//24 by client.2174595.0:108757 0.000000 2014-08-13 18:06:06.835650 7f5c973a57a0 20 read_log 194215'11659 (192888'11262) clean 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by osd.15.0:1815 2014-08-12 15:44:24.297457 2014-08-13 18:06:06.835663 7f5c973a57a0 20 read_log 194215'11660 (194215'11659) delete 600ea0e/benchmark_data_storage0111-ib.pocstor_2228_object20512/head//24 by client.2174595.0:108759 0.000000 2014-08-13 18:06:06.835674 7f5c973a57a0 20 read_log 194215'11661 (193305'11381) clean 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by osd.15.0:1817 2014-08-12 15:44:24.303972 2014-08-13 18:06:06.835689 7f5c973a57a0 20 read_log 194215'11662 (194215'11661) delete 621bea0e/benchmark_data_storage0111-ib.pocstor_62621_object16612/head//24 by client.2174595.0:108761 0.000000 2014-08-13 18:06:06.835703 7f5c973a57a0 20 read_log 194215'11663 (192888'11264) clean b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by osd.15.0:1819 2014-08-12 15:44:24.309941 2014-08-13 18:06:06.835715 7f5c973a57a0 20 read_log 194215'11664 (194215'11663) delete b66cea0e/benchmark_data_storage0111-ib.pocstor_2228_object21342/head//24 by client.2174595.0:108763 0.000000 2014-08-13 18:06:06.835726 7f5c973a57a0 20 read_log 194215'11665 (192834'10239) clean 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by osd.15.0:1821 2014-08-12 15:44:24.315737 2014-08-13 18:06:06.835741 7f5c973a57a0 20 read_log 194215'11666 (194215'11665) delete 416fea0e/benchmark_data_storage0111-ib.pocstor_26659_object14136/head//24 by client.2174595.0:108765 0.000000 2014-08-13 18:06:06.835763 7f5c973a57a0 20 read_log 194215'11667 (193305'11400) clean e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by osd.15.0:1823 2014-08-12 15:44:24.322302 2014-08-13 18:06:06.835776 7f5c973a57a0 20 read_log 194215'11668 (194215'11667) delete e4f1fa0e/benchmark_data_storage0111-ib.pocstor_62621_object31042/head//24 by client.2174595.0:108767 0.000000 2014-08-13 18:06:06.835796 7f5c973a57a0 20 read_log 194215'11669 (192834'10755) clean 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by osd.15.0:1825 2014-08-12 15:44:24.328074 2014-08-13 18:06:06.835820 7f5c973a57a0 20 read_log 194215'11670 (194215'11669) delete 1a9bfa0e/benchmark_data_storage0111-ib.pocstor_41526_object2868/head//24 by client.2174595.0:108769 0.000000 2014-08-13 18:06:06.835837 7f5c973a57a0 20 read_log 194215'11671 (192834'10861) clean a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by osd.15.0:1827 2014-08-12 15:44:24.334123 2014-08-13 18:06:06.835861 7f5c973a57a0 20 read_log 194215'11672 (194215'11671) delete a80efa0e/benchmark_data_storage0111-ib.pocstor_41526_object15164/head//24 by client.2174595.0:108771 0.000000 2014-08-13 18:06:06.835873 7f5c973a57a0 20 read_log 194215'11673 (192888'11073) clean a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by osd.15.0:1829 2014-08-12 15:44:24.341270 2014-08-13 18:06:06.835889 7f5c973a57a0 20 read_log 194215'11674 (194215'11673) delete a91a0e0e/benchmark_data_storage0111-ib.pocstor_41097_object13597/head//24 by client.2174595.0:108773 0.000000 2014-08-13 18:06:06.835900 7f5c973a57a0 20 read_log 194215'11675 (192834'10844) clean 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by osd.15.0:1831 2014-08-12 15:44:24.349447 2014-08-13 18:06:06.835912 7f5c973a57a0 20 read_log 194215'11676 (194215'11675) delete 147b1e0e/benchmark_data_storage0111-ib.pocstor_41526_object13521/head//24 by client.2174595.0:108775 0.000000 2014-08-13 18:06:06.835923 7f5c973a57a0 20 read_log 194215'11677 (192888'11244) clean 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by osd.15.0:1833 2014-08-12 15:44:24.358034 2014-08-13 18:06:06.835948 7f5c973a57a0 20 read_log 194215'11678 (194215'11677) delete 497e1e0e/benchmark_data_storage0111-ib.pocstor_2228_object8123/head//24 by client.2174595.0:108777 0.000000 2014-08-13 18:06:06.835963 7f5c973a57a0 20 read_log 194215'11679 (192888'11093) clean 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by osd.15.0:1835 2014-08-12 15:44:24.366154 2014-08-13 18:06:06.835976 7f5c973a57a0 20 read_log 194215'11680 (194215'11679) delete 58e1e0e/benchmark_data_storage0111-ib.pocstor_41097_object24650/head//24 by client.2174595.0:108779 0.000000 2014-08-13 18:06:06.835993 7f5c973a57a0 20 read_log 194215'11681 (193305'11377) clean 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by osd.15.0:1837 2014-08-12 15:44:24.373913 2014-08-13 18:06:06.836006 7f5c973a57a0 20 read_log 194215'11682 (194215'11681) delete 8a252e0e/benchmark_data_storage0111-ib.pocstor_62621_object15859/head//24 by client.2174595.0:108781 0.000000 2014-08-13 18:06:06.836017 7f5c973a57a0 20 read_log 194215'11683 (194215'11683) modify 20e/hit_set_24.20e_archive_2014-08-12 15:44:23.690803_2014-08-12 17:24:40.657618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.657832 2014-08-13 18:06:06.836032 7f5c973a57a0 20 read_log 194215'11684 (194215'11495) delete 20e/hit_set_24.20e_archive_2014-08-12 15:32:25.779939_2014-08-12 15:44:23.690307/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.657832 2014-08-13 18:06:06.836055 7f5c973a57a0 20 read_log 194215'11685 (0'0) promote 152960e/benchmark_data_storage0111-ib.pocstor_34628_object194/head//24 by osd.15.0:2353 2014-08-12 17:24:40.660810 2014-08-13 18:06:06.836071 7f5c973a57a0 20 read_log 194215'11686 (194215'11685) modify 152960e/benchmark_data_storage0111-ib.pocstor_34628_object194/head//24 by client.2193759.0:195 2014-08-12 17:24:40.652290 2014-08-13 18:06:06.836090 7f5c973a57a0 20 read_log 194215'11687 (0'0) promote 7c285a0e/benchmark_data_storage0111-ib.pocstor_34628_object1052/head//24 by osd.15.0:2366 2014-08-12 17:24:43.988272 2014-08-13 18:06:06.836102 7f5c973a57a0 20 read_log 194215'11688 (194215'11687) modify 7c285a0e/benchmark_data_storage0111-ib.pocstor_34628_object1052/head//24 by client.2193759.0:1053 2014-08-12 17:24:43.978956 2014-08-13 18:06:06.836127 7f5c973a57a0 20 read_log 194215'11689 (0'0) promote 37487a0e/benchmark_data_storage0111-ib.pocstor_34628_object1082/head//24 by osd.15.0:2368 2014-08-12 17:24:44.101624 2014-08-13 18:06:06.836175 7f5c973a57a0 20 read_log 194215'11690 (194215'11689) modify 37487a0e/benchmark_data_storage0111-ib.pocstor_34628_object1082/head//24 by client.2193759.0:1083 2014-08-12 17:24:44.092775 2014-08-13 18:06:06.836193 7f5c973a57a0 20 read_log 194370'11691 (0'0) promote 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by osd.15.0:2396 2014-08-12 20:30:18.557266 2014-08-13 18:06:06.836205 7f5c973a57a0 20 read_log 194370'11692 (194370'11691) modify 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by client.1508531.0:1119 2014-08-12 20:30:18.549021 2014-08-13 18:06:06.836221 7f5c973a57a0 20 read_log 194370'11693 (0'0) promote 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by osd.15.0:2398 2014-08-12 20:30:19.244249 2014-08-13 18:06:06.836233 7f5c973a57a0 20 read_log 194370'11694 (194370'11693) modify 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by client.1508531.0:1294 2014-08-12 20:30:19.237527 2014-08-13 18:06:06.836245 7f5c973a57a0 20 read_log 194370'11695 (0'0) promote 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by osd.15.0:2404 2014-08-12 20:30:20.404641 2014-08-13 18:06:06.836260 7f5c973a57a0 20 read_log 194370'11696 (194370'11695) modify 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by client.1508531.0:1607 2014-08-12 20:30:20.397467 2014-08-13 18:06:06.836272 7f5c973a57a0 20 read_log 194370'11697 (0'0) promote fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by osd.15.0:2412 2014-08-12 20:30:24.187761 2014-08-13 18:06:06.836291 7f5c973a57a0 20 read_log 194370'11698 (194370'11697) modify fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by client.1508531.0:2626 2014-08-12 20:30:24.180964 2014-08-13 18:06:06.836305 7f5c973a57a0 20 read_log 194370'11699 (0'0) promote e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by osd.15.0:2414 2014-08-12 20:30:24.596818 2014-08-13 18:06:06.836318 7f5c973a57a0 20 read_log 194370'11700 (194370'11699) modify e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by client.1508531.0:2739 2014-08-12 20:30:24.590302 2014-08-13 18:06:06.836330 7f5c973a57a0 20 read_log 194370'11701 (0'0) promote 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by osd.15.0:2458 2014-08-12 20:30:43.168972 2014-08-13 18:06:06.836342 7f5c973a57a0 20 read_log 194370'11702 (194370'11701) modify 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by client.1508531.0:7161 2014-08-12 20:30:43.162041 2014-08-13 18:06:06.836355 7f5c973a57a0 20 read_log 194370'11703 (0'0) promote c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by osd.15.0:2464 2014-08-12 20:30:44.418682 2014-08-13 18:06:06.836367 7f5c973a57a0 20 read_log 194370'11704 (194370'11703) modify c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by client.1508531.0:7411 2014-08-12 20:30:44.411439 2014-08-13 18:06:06.836379 7f5c973a57a0 20 read_log 194370'11705 (0'0) promote 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by osd.15.0:2468 2014-08-12 20:30:44.737234 2014-08-13 18:06:06.836392 7f5c973a57a0 20 read_log 194370'11706 (194370'11705) modify 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by client.1508531.0:7475 2014-08-12 20:30:44.729692 2014-08-13 18:06:06.836404 7f5c973a57a0 20 read_log 194370'11707 (0'0) promote f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by osd.15.0:2474 2014-08-12 20:30:48.231402 2014-08-13 18:06:06.836416 7f5c973a57a0 20 read_log 194370'11708 (194370'11707) modify f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by client.1508531.0:8121 2014-08-12 20:30:48.221225 2014-08-13 18:06:06.836436 7f5c973a57a0 20 read_log 194370'11709 (0'0) promote 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by osd.15.0:2494 2014-08-12 20:30:54.937975 2014-08-13 18:06:06.836448 7f5c973a57a0 20 read_log 194370'11710 (194370'11709) modify 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by client.1508531.0:9267 2014-08-12 20:30:54.930205 2014-08-13 18:06:06.836461 7f5c973a57a0 20 read_log 194370'11711 (0'0) promote d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by osd.15.0:2496 2014-08-12 20:30:55.070463 2014-08-13 18:06:06.836473 7f5c973a57a0 20 read_log 194370'11712 (194370'11711) modify d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by client.1508531.0:9292 2014-08-12 20:30:55.064089 2014-08-13 18:06:06.836485 7f5c973a57a0 20 read_log 194370'11713 (0'0) promote 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by osd.15.0:2498 2014-08-12 20:30:56.513009 2014-08-13 18:06:06.836499 7f5c973a57a0 20 read_log 194370'11714 (194370'11713) modify 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by client.1508531.0:9530 2014-08-12 20:30:56.505775 2014-08-13 18:06:06.836512 7f5c973a57a0 20 read_log 194370'11715 (0'0) promote 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by osd.15.0:2500 2014-08-12 20:30:57.665176 2014-08-13 18:06:06.836524 7f5c973a57a0 20 read_log 194370'11716 (194370'11715) modify 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by client.1508531.0:9723 2014-08-12 20:30:57.657537 2014-08-13 18:06:06.836536 7f5c973a57a0 20 read_log 194370'11717 (0'0) promote 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by osd.15.0:2536 2014-08-12 20:31:20.012680 2014-08-13 18:06:06.836549 7f5c973a57a0 20 read_log 194370'11718 (194370'11717) modify 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by client.1508531.0:13070 2014-08-12 20:31:20.004477 2014-08-13 18:06:06.836561 7f5c973a57a0 20 read_log 194377'11719 (0'0) promote 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by osd.15.0:2566 2014-08-12 20:33:09.516224 2014-08-13 18:06:06.836573 7f5c973a57a0 20 read_log 194377'11720 (194377'11719) modify 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by client.2231190.0:3047 2014-08-12 20:33:09.507056 2014-08-13 18:06:06.836586 7f5c973a57a0 20 read_log 194377'11721 (0'0) promote 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by osd.15.0:2570 2014-08-12 20:33:11.875729 2014-08-13 18:06:06.836598 7f5c973a57a0 20 read_log 194377'11722 (194377'11721) modify 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by client.2231190.0:3601 2014-08-12 20:33:11.866129 2014-08-13 18:06:06.836610 7f5c973a57a0 20 read_log 194377'11723 (0'0) promote c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by osd.15.0:2572 2014-08-12 20:33:13.289171 2014-08-13 18:06:06.836622 7f5c973a57a0 20 read_log 194377'11724 (194377'11723) modify c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by client.2231190.0:3886 2014-08-12 20:33:13.281188 2014-08-13 18:06:06.836670 7f5c973a57a0 20 read_log 194377'11725 (0'0) promote 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by osd.15.0:2578 2014-08-12 20:33:16.143638 2014-08-13 18:06:06.836683 7f5c973a57a0 20 read_log 194377'11726 (194377'11725) modify 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by client.2231190.0:4362 2014-08-12 20:33:16.133686 2014-08-13 18:06:06.836695 7f5c973a57a0 20 read_log 194380'11727 (0'0) promote eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by osd.15.0:2590 2014-08-12 20:33:24.395718 2014-08-13 18:06:06.836735 7f5c973a57a0 20 read_log 194380'11728 (194380'11727) modify eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by client.2231190.0:5098 2014-08-12 20:33:24.385329 2014-08-13 18:06:06.836751 7f5c973a57a0 20 read_log 194383'11729 (0'0) promote bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by osd.15.0:2606 2014-08-12 20:34:03.197149 2014-08-13 18:06:06.836783 7f5c973a57a0 20 read_log 194383'11730 (194383'11729) modify bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by client.2231190.0:6330 2014-08-12 20:34:03.189069 2014-08-13 18:06:06.836796 7f5c973a57a0 20 read_log 194383'11731 (0'0) promote 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by osd.15.0:2610 2014-08-12 20:34:07.051698 2014-08-13 18:06:06.836809 7f5c973a57a0 20 read_log 194383'11732 (194383'11731) modify 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by client.2231190.0:6699 2014-08-12 20:34:07.042697 2014-08-13 18:06:06.836821 7f5c973a57a0 20 read_log 194383'11733 (0'0) promote 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by osd.15.0:2616 2014-08-12 20:34:35.631328 2014-08-13 18:06:06.836833 7f5c973a57a0 20 read_log 194383'11734 (194383'11733) modify 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by client.2231190.0:7747 2014-08-12 20:34:35.621285 2014-08-13 18:06:06.836848 7f5c973a57a0 20 read_log 194393'11735 (194393'11735) modify 20e/hit_set_24.20e_archive_2014-08-12 20:37:08.036878_2014-08-12 20:47:30.630815/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:30.631587 2014-08-13 18:06:06.836860 7f5c973a57a0 20 read_log 194393'11736 (194215'11683) delete 20e/hit_set_24.20e_archive_2014-08-12 15:44:23.690803_2014-08-12 17:24:40.657618/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:30.631587 2014-08-13 18:06:06.836876 7f5c973a57a0 20 read_log 194393'11737 (0'0) promote 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by osd.15.0:2633 2014-08-12 20:47:30.636666 2014-08-13 18:06:06.836888 7f5c973a57a0 20 read_log 194393'11738 (194393'11737) modify 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by client.1064095.0:735 2014-08-12 20:47:30.624403 2014-08-13 18:06:06.836902 7f5c973a57a0 20 read_log 194393'11739 (0'0) promote 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by osd.15.0:2637 2014-08-12 20:49:25.445822 2014-08-13 18:06:06.836915 7f5c973a57a0 20 read_log 194393'11740 (194393'11739) modify 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by client.1064182.0:12 2014-08-12 20:49:25.421095 2014-08-13 18:06:06.836930 7f5c973a57a0 20 read_log 194393'11741 (0'0) promote 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by osd.15.0:2645 2014-08-12 20:49:28.267993 2014-08-13 18:06:06.836942 7f5c973a57a0 20 read_log 194393'11742 (194393'11741) modify 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by client.1064182.0:481 2014-08-12 20:49:28.260541 2014-08-13 18:06:06.836957 7f5c973a57a0 20 read_log 194397'11743 (194397'11743) modify 20e/hit_set_24.20e_archive_2014-08-12 20:47:30.631832_2014-08-12 20:53:02.890619/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:02.891332 2014-08-13 18:06:06.836969 7f5c973a57a0 20 read_log 194397'11744 (194393'11735) delete 20e/hit_set_24.20e_archive_2014-08-12 20:37:08.036878_2014-08-12 20:47:30.630815/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:02.891332 2014-08-13 18:06:06.836984 7f5c973a57a0 20 read_log 194397'11745 (0'0) promote f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by osd.15.0:2668 2014-08-12 20:53:02.896821 2014-08-13 18:06:06.836997 7f5c973a57a0 20 read_log 194397'11746 (194397'11745) modify f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by client.2235069.0:705 2014-08-12 20:53:02.883290 2014-08-13 18:06:06.837019 7f5c973a57a0 20 read_log 194397'11747 (0'0) promote 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by osd.15.0:2670 2014-08-12 20:53:03.282378 2014-08-13 18:06:06.837033 7f5c973a57a0 20 read_log 194397'11748 (194397'11747) modify 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by client.2235069.0:755 2014-08-12 20:53:03.272166 2014-08-13 18:06:06.837046 7f5c973a57a0 20 read_log 194416'11749 (194416'11749) modify 20e/hit_set_24.20e_archive_2014-08-12 20:53:02.891413_2014-08-13 15:28:37.371929/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 18:06:06.837061 7f5c973a57a0 20 read_log 194416'11750 (194397'11743) delete 20e/hit_set_24.20e_archive_2014-08-12 20:47:30.631832_2014-08-12 20:53:02.890619/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 18:06:06.837073 7f5c973a57a0 20 read_log 194416'11751 (0'0) promote ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by osd.15.0:2819 2014-08-13 15:28:37.377849 2014-08-13 18:06:06.837085 7f5c973a57a0 20 read_log 194416'11752 (194416'11751) modify ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by client.2441958.0:1757 2014-08-13 15:28:37.366408 2014-08-13 18:06:06.837098 7f5c973a57a0 20 read_log 194416'11753 (0'0) promote eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by osd.15.0:2835 2014-08-13 15:28:42.472115 2014-08-13 18:06:06.837112 7f5c973a57a0 20 read_log 194416'11754 (194416'11753) modify eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by client.2441958.0:3105 2014-08-13 15:28:42.462454 2014-08-13 18:06:06.837124 7f5c973a57a0 20 read_log 194416'11755 (0'0) promote e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by osd.15.0:2841 2014-08-13 15:28:45.845193 2014-08-13 18:06:06.837139 7f5c973a57a0 20 read_log 194416'11756 (194416'11755) modify e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by client.2441958.0:3977 2014-08-13 15:28:45.835441 2014-08-13 18:06:06.837156 7f5c973a57a0 20 read_log 194416'11757 (0'0) promote dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by osd.15.0:2845 2014-08-13 15:28:46.740340 2014-08-13 18:06:06.837171 7f5c973a57a0 20 read_log 194416'11758 (194416'11757) modify dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by client.2441958.0:4198 2014-08-13 15:28:46.733616 2014-08-13 18:06:06.837183 7f5c973a57a0 20 read_log 194416'11759 (0'0) promote 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by osd.15.0:2847 2014-08-13 15:28:47.338938 2014-08-13 18:06:06.837199 7f5c973a57a0 20 read_log 194416'11760 (194416'11759) modify 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by client.2441958.0:4358 2014-08-13 15:28:47.332439 2014-08-13 18:06:06.837213 7f5c973a57a0 20 read_log 194416'11761 (0'0) promote cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by osd.15.0:2857 2014-08-13 15:28:52.868690 2014-08-13 18:06:06.837226 7f5c973a57a0 20 read_log 194416'11762 (194416'11761) modify cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by client.2441958.0:5773 2014-08-13 15:28:52.861760 2014-08-13 18:06:06.837240 7f5c973a57a0 20 read_log 194416'11763 (0'0) promote 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by osd.15.0:2869 2014-08-13 15:28:55.644734 2014-08-13 18:06:06.837252 7f5c973a57a0 20 read_log 194416'11764 (194416'11763) modify 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by client.2441958.0:6439 2014-08-13 15:28:55.636855 2014-08-13 18:06:06.837270 7f5c973a57a0 20 read_log 194416'11765 (0'0) promote 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by osd.15.0:2871 2014-08-13 15:28:56.314454 2014-08-13 18:06:06.837292 7f5c973a57a0 20 read_log 194416'11766 (194416'11765) modify 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by client.2441958.0:6607 2014-08-13 15:28:56.306485 2014-08-13 18:06:06.837306 7f5c973a57a0 20 read_log 194416'11767 (0'0) promote 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by osd.15.0:2897 2014-08-13 15:29:07.683717 2014-08-13 18:06:06.837322 7f5c973a57a0 20 read_log 194416'11768 (194416'11767) modify 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by client.2441958.0:9578 2014-08-13 15:29:07.675105 2014-08-13 18:06:06.837337 7f5c973a57a0 20 read_log 194416'11769 (0'0) promote a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by osd.15.0:2901 2014-08-13 15:29:09.251756 2014-08-13 18:06:06.837349 7f5c973a57a0 20 read_log 194416'11770 (194416'11769) modify a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by client.2441958.0:9956 2014-08-13 15:29:09.242751 2014-08-13 18:06:06.837362 7f5c973a57a0 20 read_log 194416'11771 (0'0) promote d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by osd.15.0:2905 2014-08-13 15:29:11.495677 2014-08-13 18:06:06.837374 7f5c973a57a0 20 read_log 194416'11772 (194416'11771) modify d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by client.2441958.0:10515 2014-08-13 15:29:11.486380 2014-08-13 18:06:06.837386 7f5c973a57a0 20 read_log 194416'11773 (0'0) promote 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by osd.15.0:2911 2014-08-13 15:29:13.891865 2014-08-13 18:06:06.837399 7f5c973a57a0 20 read_log 194416'11774 (194416'11773) modify 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by client.2441958.0:11134 2014-08-13 15:29:13.881622 2014-08-13 18:06:06.837423 7f5c973a57a0 20 read_log 194416'11775 (0'0) promote dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by osd.15.0:2927 2014-08-13 15:29:24.940835 2014-08-13 18:06:06.837440 7f5c973a57a0 20 read_log 194416'11776 (194416'11775) modify dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by client.2441958.0:13681 2014-08-13 15:29:24.932178 2014-08-13 18:06:06.837453 7f5c973a57a0 20 read_log 194416'11777 (0'0) promote e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by osd.15.0:2933 2014-08-13 15:29:25.656553 2014-08-13 18:06:06.837471 7f5c973a57a0 20 read_log 194416'11778 (194416'11777) modify e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by client.2441958.0:13858 2014-08-13 15:29:25.648625 2014-08-13 18:06:06.837510 7f5c973a57a0 20 read_log 194416'11779 (0'0) promote 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by osd.15.0:2943 2014-08-13 15:29:27.566695 2014-08-13 18:06:06.837534 7f5c973a57a0 20 read_log 194416'11780 (194416'11779) modify 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by client.2441958.0:14352 2014-08-13 15:29:27.556476 2014-08-13 18:06:06.837547 7f5c973a57a0 20 read_log 194416'11781 (0'0) promote 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by osd.15.0:2961 2014-08-13 15:29:31.860218 2014-08-13 18:06:06.837559 7f5c973a57a0 20 read_log 194416'11782 (194416'11781) modify 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by client.2441958.0:15353 2014-08-13 15:29:31.852271 2014-08-13 18:06:06.837601 7f5c973a57a0 20 read_log 194416'11783 (0'0) promote f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by osd.15.0:2975 2014-08-13 15:29:43.772509 2014-08-13 18:06:06.837616 7f5c973a57a0 20 read_log 194416'11784 (194416'11783) modify f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by client.2441958.0:16752 2014-08-13 15:29:43.762592 2014-08-13 18:06:06.837629 7f5c973a57a0 20 read_log 194416'11785 (0'0) promote b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by osd.15.0:2983 2014-08-13 15:29:54.148756 2014-08-13 18:06:06.837649 7f5c973a57a0 20 read_log 194416'11786 (194416'11785) modify b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by client.2441958.0:17480 2014-08-13 15:29:54.140182 2014-08-13 18:06:06.837664 7f5c973a57a0 20 read_log 194416'11787 (0'0) promote 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by osd.15.0:2985 2014-08-13 15:29:54.433501 2014-08-13 18:06:06.837676 7f5c973a57a0 20 read_log 194416'11788 (194416'11787) modify 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by client.2441958.0:17500 2014-08-13 15:29:54.424428 2014-08-13 18:06:06.837692 7f5c973a57a0 20 read_log 194422'11789 (0'0) promote 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by osd.15.0:3013 2014-08-13 15:32:23.894340 2014-08-13 18:06:06.837704 7f5c973a57a0 20 read_log 194422'11790 (194422'11789) modify 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by client.1148203.0:2173 2014-08-13 15:32:23.886263 2014-08-13 18:06:06.837717 7f5c973a57a0 20 read_log 194422'11791 (0'0) promote 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by osd.15.0:3021 2014-08-13 15:32:26.581047 2014-08-13 18:06:06.837731 7f5c973a57a0 20 read_log 194422'11792 (194422'11791) modify 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by client.1148203.0:2865 2014-08-13 15:32:26.572355 2014-08-13 18:06:06.837744 7f5c973a57a0 20 read_log 194422'11793 (0'0) promote 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by osd.15.0:3025 2014-08-13 15:32:27.970042 2014-08-13 18:06:06.837756 7f5c973a57a0 20 read_log 194422'11794 (194422'11793) modify 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by client.1148203.0:3230 2014-08-13 15:32:27.962899 2014-08-13 18:06:06.837768 7f5c973a57a0 20 read_log 194422'11795 (0'0) promote bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by osd.15.0:3045 2014-08-13 15:32:41.048428 2014-08-13 18:06:06.837780 7f5c973a57a0 20 read_log 194422'11796 (194422'11795) modify bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by client.1148203.0:6167 2014-08-13 15:32:41.037193 2014-08-13 18:06:06.837793 7f5c973a57a0 20 read_log 194422'11797 (0'0) promote dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by osd.15.0:3067 2014-08-13 15:32:51.368290 2014-08-13 18:06:06.837805 7f5c973a57a0 20 read_log 194422'11798 (194422'11797) modify dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by client.1148203.0:7946 2014-08-13 15:32:51.359365 2014-08-13 18:06:06.837817 7f5c973a57a0 20 read_log 194422'11799 (0'0) promote bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by osd.15.0:3071 2014-08-13 15:32:53.078937 2014-08-13 18:06:06.837829 7f5c973a57a0 20 read_log 194422'11800 (194422'11799) modify bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by client.1148203.0:8216 2014-08-13 15:32:53.069000 2014-08-13 18:06:06.837841 7f5c973a57a0 20 read_log 194422'11801 (0'0) promote 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by osd.15.0:3075 2014-08-13 15:32:58.155342 2014-08-13 18:06:06.837866 7f5c973a57a0 20 read_log 194422'11802 (194422'11801) modify 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by client.1148203.0:8855 2014-08-13 15:32:58.146666 2014-08-13 18:06:06.837879 7f5c973a57a0 20 read_log 194422'11803 (0'0) promote 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by osd.15.0:3081 2014-08-13 15:33:00.679424 2014-08-13 18:06:06.837891 7f5c973a57a0 20 read_log 194422'11804 (194422'11803) modify 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by client.1148203.0:9113 2014-08-13 15:33:00.671918 2014-08-13 18:06:06.837914 7f5c973a57a0 20 read_log 194422'11805 (0'0) promote 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by osd.15.0:3083 2014-08-13 15:33:00.906158 2014-08-13 18:06:06.837929 7f5c973a57a0 20 read_log 194422'11806 (194422'11805) modify 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by client.1148203.0:9137 2014-08-13 15:33:00.896712 2014-08-13 18:06:06.837945 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.838601 7f5c973a57a0 10 read_log checking for missing items over interval (194370'11691,194422'11806] 2014-08-13 18:06:06.838650 7f5c973a57a0 15 read_log missing 194422'11806 (194422'11805) modify 143bae0e/benchmark_data_storage0111-ib.pocstor_41231_object9136/head//24 by client.1148203.0:9137 2014-08-13 15:33:00.896712 2014-08-13 18:06:06.838691 7f5c973a57a0 15 read_log missing 194422'11804 (194422'11803) modify 1aefee0e/benchmark_data_storage0111-ib.pocstor_41231_object9112/head//24 by client.1148203.0:9113 2014-08-13 15:33:00.671918 2014-08-13 18:06:06.838728 7f5c973a57a0 15 read_log missing 194422'11802 (194422'11801) modify 97c0e60e/benchmark_data_storage0111-ib.pocstor_41231_object8854/head//24 by client.1148203.0:8855 2014-08-13 15:32:58.146666 2014-08-13 18:06:06.838764 7f5c973a57a0 15 read_log missing 194422'11800 (194422'11799) modify bb0bb20e/benchmark_data_storage0111-ib.pocstor_41231_object8215/head//24 by client.1148203.0:8216 2014-08-13 15:32:53.069000 2014-08-13 18:06:06.838800 7f5c973a57a0 15 read_log missing 194422'11798 (194422'11797) modify dcbc60e/benchmark_data_storage0111-ib.pocstor_41231_object7945/head//24 by client.1148203.0:7946 2014-08-13 15:32:51.359365 2014-08-13 18:06:06.838836 7f5c973a57a0 15 read_log missing 194422'11796 (194422'11795) modify bba2720e/benchmark_data_storage0111-ib.pocstor_41231_object6166/head//24 by client.1148203.0:6167 2014-08-13 15:32:41.037193 2014-08-13 18:06:06.838871 7f5c973a57a0 15 read_log missing 194422'11794 (194422'11793) modify 4a995a0e/benchmark_data_storage0111-ib.pocstor_41231_object3229/head//24 by client.1148203.0:3230 2014-08-13 15:32:27.962899 2014-08-13 18:06:06.838907 7f5c973a57a0 15 read_log missing 194422'11792 (194422'11791) modify 9ff63a0e/benchmark_data_storage0111-ib.pocstor_41231_object2864/head//24 by client.1148203.0:2865 2014-08-13 15:32:26.572355 2014-08-13 18:06:06.838943 7f5c973a57a0 15 read_log missing 194422'11790 (194422'11789) modify 3486f60e/benchmark_data_storage0111-ib.pocstor_41231_object2172/head//24 by client.1148203.0:2173 2014-08-13 15:32:23.886263 2014-08-13 18:06:06.838978 7f5c973a57a0 15 read_log missing 194416'11788 (194416'11787) modify 3df30e0e/benchmark_data_storage0111-ib.pocstor_19103_object17499/head//24 by client.2441958.0:17500 2014-08-13 15:29:54.424428 2014-08-13 18:06:06.839014 7f5c973a57a0 15 read_log missing 194416'11786 (194416'11785) modify b051620e/benchmark_data_storage0111-ib.pocstor_19103_object17479/head//24 by client.2441958.0:17480 2014-08-13 15:29:54.140182 2014-08-13 18:06:06.839049 7f5c973a57a0 15 read_log missing 194416'11784 (194416'11783) modify f5e1960e/benchmark_data_storage0111-ib.pocstor_19103_object16751/head//24 by client.2441958.0:16752 2014-08-13 15:29:43.762592 2014-08-13 18:06:06.839085 7f5c973a57a0 15 read_log missing 194416'11782 (194416'11781) modify 3de4420e/benchmark_data_storage0111-ib.pocstor_19103_object15352/head//24 by client.2441958.0:15353 2014-08-13 15:29:31.852271 2014-08-13 18:06:06.839121 7f5c973a57a0 15 read_log missing 194416'11780 (194416'11779) modify 368b760e/benchmark_data_storage0111-ib.pocstor_19103_object14351/head//24 by client.2441958.0:14352 2014-08-13 15:29:27.556476 2014-08-13 18:06:06.839162 7f5c973a57a0 15 read_log missing 194416'11778 (194416'11777) modify e25b1a0e/benchmark_data_storage0111-ib.pocstor_19103_object13857/head//24 by client.2441958.0:13858 2014-08-13 15:29:25.648625 2014-08-13 18:06:06.839199 7f5c973a57a0 15 read_log missing 194416'11776 (194416'11775) modify dd3ff20e/benchmark_data_storage0111-ib.pocstor_19103_object13680/head//24 by client.2441958.0:13681 2014-08-13 15:29:24.932178 2014-08-13 18:06:06.839239 7f5c973a57a0 15 read_log missing 194416'11774 (194416'11773) modify 8b8d4e0e/benchmark_data_storage0111-ib.pocstor_19103_object11133/head//24 by client.2441958.0:11134 2014-08-13 15:29:13.881622 2014-08-13 18:06:06.839280 7f5c973a57a0 15 read_log missing 194416'11772 (194416'11771) modify d97a1e0e/benchmark_data_storage0111-ib.pocstor_19103_object10514/head//24 by client.2441958.0:10515 2014-08-13 15:29:11.486380 2014-08-13 18:06:06.839316 7f5c973a57a0 15 read_log missing 194416'11770 (194416'11769) modify a433820e/benchmark_data_storage0111-ib.pocstor_19103_object9955/head//24 by client.2441958.0:9956 2014-08-13 15:29:09.242751 2014-08-13 18:06:06.839365 7f5c973a57a0 15 read_log missing 194416'11768 (194416'11767) modify 1240120e/benchmark_data_storage0111-ib.pocstor_19103_object9577/head//24 by client.2441958.0:9578 2014-08-13 15:29:07.675105 2014-08-13 18:06:06.839412 7f5c973a57a0 15 read_log missing 194416'11766 (194416'11765) modify 9006420e/benchmark_data_storage0111-ib.pocstor_19103_object6606/head//24 by client.2441958.0:6607 2014-08-13 15:28:56.306485 2014-08-13 18:06:06.839450 7f5c973a57a0 15 read_log missing 194416'11764 (194416'11763) modify 115e7e0e/benchmark_data_storage0111-ib.pocstor_19103_object6438/head//24 by client.2441958.0:6439 2014-08-13 15:28:55.636855 2014-08-13 18:06:06.839486 7f5c973a57a0 15 read_log missing 194416'11762 (194416'11761) modify cfaf360e/benchmark_data_storage0111-ib.pocstor_19103_object5772/head//24 by client.2441958.0:5773 2014-08-13 15:28:52.861760 2014-08-13 18:06:06.839522 7f5c973a57a0 15 read_log missing 194416'11760 (194416'11759) modify 5d2c320e/benchmark_data_storage0111-ib.pocstor_19103_object4357/head//24 by client.2441958.0:4358 2014-08-13 15:28:47.332439 2014-08-13 18:06:06.839559 7f5c973a57a0 15 read_log missing 194416'11758 (194416'11757) modify dc7dd60e/benchmark_data_storage0111-ib.pocstor_19103_object4197/head//24 by client.2441958.0:4198 2014-08-13 15:28:46.733616 2014-08-13 18:06:06.839595 7f5c973a57a0 15 read_log missing 194416'11756 (194416'11755) modify e1b5920e/benchmark_data_storage0111-ib.pocstor_19103_object3976/head//24 by client.2441958.0:3977 2014-08-13 15:28:45.835441 2014-08-13 18:06:06.839631 7f5c973a57a0 15 read_log missing 194416'11754 (194416'11753) modify eb30320e/benchmark_data_storage0111-ib.pocstor_19103_object3104/head//24 by client.2441958.0:3105 2014-08-13 15:28:42.462454 2014-08-13 18:06:06.839667 7f5c973a57a0 15 read_log missing 194416'11752 (194416'11751) modify ced6860e/benchmark_data_storage0111-ib.pocstor_19103_object1756/head//24 by client.2441958.0:1757 2014-08-13 15:28:37.366408 2014-08-13 18:06:06.839707 7f5c973a57a0 15 read_log missing 194416'11749 (194416'11749) modify 20e/hit_set_24.20e_archive_2014-08-12 20:53:02.891413_2014-08-13 15:28:37.371929/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.372177 2014-08-13 18:06:06.839743 7f5c973a57a0 15 read_log missing 194397'11748 (194397'11747) modify 5b5d60e/benchmark_data_storage0101-ib.pocstor_6819_object754/head//24 by client.2235069.0:755 2014-08-12 20:53:03.272166 2014-08-13 18:06:06.839781 7f5c973a57a0 15 read_log missing 194397'11746 (194397'11745) modify f2216e0e/benchmark_data_storage0101-ib.pocstor_6819_object704/head//24 by client.2235069.0:705 2014-08-12 20:53:02.883290 2014-08-13 18:06:06.839822 7f5c973a57a0 15 read_log missing 194393'11742 (194393'11741) modify 7bbcf60e/benchmark_data_storage0111-ib.pocstor_32482_object480/head//24 by client.1064182.0:481 2014-08-12 20:49:28.260541 2014-08-13 18:06:06.839859 7f5c973a57a0 15 read_log missing 194393'11740 (194393'11739) modify 1c5dba0e/benchmark_data_storage0111-ib.pocstor_32482_object11/head//24 by client.1064182.0:12 2014-08-12 20:49:25.421095 2014-08-13 18:06:06.839895 7f5c973a57a0 15 read_log missing 194393'11738 (194393'11737) modify 5475ce0e/benchmark_data_storage0111-ib.pocstor_24170_object734/head//24 by client.1064095.0:735 2014-08-12 20:47:30.624403 2014-08-13 18:06:06.839936 7f5c973a57a0 15 read_log missing 194383'11734 (194383'11733) modify 26e3520e/benchmark_data_storage0111-ib.pocstor_25065_object7746/head//24 by client.2231190.0:7747 2014-08-12 20:34:35.621285 2014-08-13 18:06:06.839978 7f5c973a57a0 15 read_log missing 194383'11732 (194383'11731) modify 76209a0e/benchmark_data_storage0111-ib.pocstor_25065_object6698/head//24 by client.2231190.0:6699 2014-08-12 20:34:07.042697 2014-08-13 18:06:06.840015 7f5c973a57a0 15 read_log missing 194383'11730 (194383'11729) modify bdbad60e/benchmark_data_storage0111-ib.pocstor_25065_object6329/head//24 by client.2231190.0:6330 2014-08-12 20:34:03.189069 2014-08-13 18:06:06.840051 7f5c973a57a0 15 read_log missing 194380'11728 (194380'11727) modify eeff060e/benchmark_data_storage0111-ib.pocstor_25065_object5097/head//24 by client.2231190.0:5098 2014-08-12 20:33:24.385329 2014-08-13 18:06:06.840087 7f5c973a57a0 15 read_log missing 194377'11726 (194377'11725) modify 28fa6e0e/benchmark_data_storage0111-ib.pocstor_25065_object4361/head//24 by client.2231190.0:4362 2014-08-12 20:33:16.133686 2014-08-13 18:06:06.840124 7f5c973a57a0 15 read_log missing 194377'11724 (194377'11723) modify c6f29a0e/benchmark_data_storage0111-ib.pocstor_25065_object3885/head//24 by client.2231190.0:3886 2014-08-12 20:33:13.281188 2014-08-13 18:06:06.840166 7f5c973a57a0 15 read_log missing 194377'11722 (194377'11721) modify 84f1560e/benchmark_data_storage0111-ib.pocstor_25065_object3600/head//24 by client.2231190.0:3601 2014-08-12 20:33:11.866129 2014-08-13 18:06:06.840203 7f5c973a57a0 15 read_log missing 194377'11720 (194377'11719) modify 6b1fae0e/benchmark_data_storage0111-ib.pocstor_25065_object3046/head//24 by client.2231190.0:3047 2014-08-12 20:33:09.507056 2014-08-13 18:06:06.840239 7f5c973a57a0 15 read_log missing 194370'11718 (194370'11717) modify 36e1e60e/benchmark_data_storage0111-ib.pocstor_12495_object13069/head//24 by client.1508531.0:13070 2014-08-12 20:31:20.004477 2014-08-13 18:06:06.840279 7f5c973a57a0 15 read_log missing 194370'11716 (194370'11715) modify 1a5d820e/benchmark_data_storage0111-ib.pocstor_12495_object9722/head//24 by client.1508531.0:9723 2014-08-12 20:30:57.657537 2014-08-13 18:06:06.840316 7f5c973a57a0 15 read_log missing 194370'11714 (194370'11713) modify 92144a0e/benchmark_data_storage0111-ib.pocstor_12495_object9529/head//24 by client.1508531.0:9530 2014-08-12 20:30:56.505775 2014-08-13 18:06:06.840357 7f5c973a57a0 15 read_log missing 194370'11712 (194370'11711) modify d0c2e0e/benchmark_data_storage0111-ib.pocstor_12495_object9291/head//24 by client.1508531.0:9292 2014-08-12 20:30:55.064089 2014-08-13 18:06:06.840395 7f5c973a57a0 15 read_log missing 194370'11710 (194370'11709) modify 36be360e/benchmark_data_storage0111-ib.pocstor_12495_object9266/head//24 by client.1508531.0:9267 2014-08-12 20:30:54.930205 2014-08-13 18:06:06.840442 7f5c973a57a0 15 read_log missing 194370'11708 (194370'11707) modify f32b420e/benchmark_data_storage0111-ib.pocstor_12495_object8120/head//24 by client.1508531.0:8121 2014-08-12 20:30:48.221225 2014-08-13 18:06:06.840494 7f5c973a57a0 15 read_log missing 194370'11706 (194370'11705) modify 1a942e0e/benchmark_data_storage0111-ib.pocstor_12495_object7474/head//24 by client.1508531.0:7475 2014-08-12 20:30:44.729692 2014-08-13 18:06:06.840532 7f5c973a57a0 15 read_log missing 194370'11704 (194370'11703) modify c0df4a0e/benchmark_data_storage0111-ib.pocstor_12495_object7410/head//24 by client.1508531.0:7411 2014-08-12 20:30:44.411439 2014-08-13 18:06:06.840569 7f5c973a57a0 15 read_log missing 194370'11702 (194370'11701) modify 5cebe60e/benchmark_data_storage0111-ib.pocstor_12495_object7160/head//24 by client.1508531.0:7161 2014-08-12 20:30:43.162041 2014-08-13 18:06:06.840606 7f5c973a57a0 15 read_log missing 194370'11700 (194370'11699) modify e2ef7a0e/benchmark_data_storage0111-ib.pocstor_12495_object2738/head//24 by client.1508531.0:2739 2014-08-12 20:30:24.590302 2014-08-13 18:06:06.840643 7f5c973a57a0 15 read_log missing 194370'11698 (194370'11697) modify fecb3e0e/benchmark_data_storage0111-ib.pocstor_12495_object2625/head//24 by client.1508531.0:2626 2014-08-12 20:30:24.180964 2014-08-13 18:06:06.840679 7f5c973a57a0 15 read_log missing 194370'11696 (194370'11695) modify 2f09aa0e/benchmark_data_storage0111-ib.pocstor_12495_object1606/head//24 by client.1508531.0:1607 2014-08-12 20:30:20.397467 2014-08-13 18:06:06.840715 7f5c973a57a0 15 read_log missing 194370'11694 (194370'11693) modify 91cd320e/benchmark_data_storage0111-ib.pocstor_12495_object1293/head//24 by client.1508531.0:1294 2014-08-12 20:30:19.237527 2014-08-13 18:06:06.840751 7f5c973a57a0 15 read_log missing 194370'11692 (194370'11691) modify 35ab620e/benchmark_data_storage0111-ib.pocstor_12495_object1118/head//24 by client.1508531.0:1119 2014-08-12 20:30:18.549021 2014-08-13 18:06:06.840762 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.840798 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 18:06:06.840816 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.024977 0 0.000000 2014-08-13 18:06:06.840829 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 18:06:06.840839 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=0 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194487 2014-08-13 18:06:06.840849 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] log((192718'8900,194422'11806], crt=194422'11806) 2014-08-13 18:06:06.840860 7f5c973a57a0 10 osd.4 194487 pgid 24.299 coll 24.299_head 2014-08-13 18:06:06.840982 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.299 2014-08-13 18:06:06.840996 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.841029 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.299(unlocked)] enter Initial 2014-08-13 18:06:06.841040 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[24.299(unlocked)] enter NotTrimming 2014-08-13 18:06:06.841186 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.841323 7f5c973a57a0 20 read_log 192834'13598 (0'0) promote e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by osd.372.0:29784 2014-08-11 15:23:49.578363 2014-08-13 18:06:06.841345 7f5c973a57a0 20 read_log 192834'13599 (192834'13598) modify e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.933448.0:2966 2014-08-11 15:23:49.570293 2014-08-13 18:06:06.841358 7f5c973a57a0 20 read_log 192834'13604 (0'0) promote c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by osd.372.0:29792 2014-08-11 15:23:50.163965 2014-08-13 18:06:06.841377 7f5c973a57a0 20 read_log 192834'13605 (192834'13604) modify c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.933448.0:3065 2014-08-11 15:23:49.995911 2014-08-13 18:06:06.841390 7f5c973a57a0 20 read_log 192834'13610 (0'0) promote e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by osd.372.0:29800 2014-08-11 15:23:51.490438 2014-08-13 18:06:06.841403 7f5c973a57a0 20 read_log 192834'13611 (192834'13610) modify e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.933448.0:3377 2014-08-11 15:23:51.482370 2014-08-13 18:06:06.841415 7f5c973a57a0 20 read_log 192834'13665 (0'0) promote 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by osd.372.0:29895 2014-08-11 15:24:04.488609 2014-08-13 18:06:06.841439 7f5c973a57a0 20 read_log 192834'13666 (192834'13665) modify 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.933448.0:6196 2014-08-11 15:24:04.481779 2014-08-13 18:06:06.841456 7f5c973a57a0 20 read_log 192834'13697 (0'0) promote d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by osd.372.0:29971 2014-08-11 15:24:13.448301 2014-08-13 18:06:06.841469 7f5c973a57a0 20 read_log 192834'13698 (192834'13697) modify d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.933448.0:8290 2014-08-11 15:24:13.441461 2014-08-13 18:06:06.841508 7f5c973a57a0 20 read_log 192834'13699 (0'0) promote 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by osd.372.0:29977 2014-08-11 15:24:17.369035 2014-08-13 18:06:06.841521 7f5c973a57a0 20 read_log 192834'13700 (192834'13699) modify 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.933448.0:9052 2014-08-11 15:24:17.362409 2014-08-13 18:06:06.841533 7f5c973a57a0 20 read_log 192834'13709 (0'0) promote 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by osd.372.0:29989 2014-08-11 15:24:18.560489 2014-08-13 18:06:06.841546 7f5c973a57a0 20 read_log 192834'13710 (192834'13709) modify 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.933448.0:9364 2014-08-11 15:24:18.553459 2014-08-13 18:06:06.841583 7f5c973a57a0 20 read_log 192834'13713 (0'0) promote 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by osd.372.0:30011 2014-08-11 15:24:21.016297 2014-08-13 18:06:06.841600 7f5c973a57a0 20 read_log 192834'13714 (192834'13713) modify 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.933448.0:9960 2014-08-11 15:24:21.009005 2014-08-13 18:06:06.841613 7f5c973a57a0 20 read_log 192834'13723 (0'0) promote a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by osd.372.0:30029 2014-08-11 15:24:22.799925 2014-08-13 18:06:06.841625 7f5c973a57a0 20 read_log 192834'13724 (192834'13723) modify a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.933448.0:10348 2014-08-11 15:24:22.791544 2014-08-13 18:06:06.841638 7f5c973a57a0 20 read_log 192834'13727 (0'0) promote c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by osd.372.0:30035 2014-08-11 15:24:23.757948 2014-08-13 18:06:06.841650 7f5c973a57a0 20 read_log 192834'13728 (192834'13727) modify c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.933448.0:10565 2014-08-11 15:24:23.751870 2014-08-13 18:06:06.841662 7f5c973a57a0 20 read_log 192834'13765 (0'0) promote c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by osd.372.0:30121 2014-08-11 15:24:34.974574 2014-08-13 18:06:06.841674 7f5c973a57a0 20 read_log 192834'13766 (192834'13765) modify c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.933448.0:13014 2014-08-11 15:24:34.966502 2014-08-13 18:06:06.841687 7f5c973a57a0 20 read_log 192834'13795 (0'0) promote 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by osd.372.0:30217 2014-08-11 15:24:51.917895 2014-08-13 18:06:06.841699 7f5c973a57a0 20 read_log 192834'13796 (192834'13795) modify 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.933448.0:15937 2014-08-11 15:24:51.910959 2014-08-13 18:06:06.841711 7f5c973a57a0 20 read_log 192834'13801 (0'0) promote 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by osd.372.0:30227 2014-08-11 15:24:55.950079 2014-08-13 18:06:06.841723 7f5c973a57a0 20 read_log 192834'13802 (192834'13801) modify 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.933448.0:16489 2014-08-11 15:24:55.942940 2014-08-13 18:06:06.841735 7f5c973a57a0 20 read_log 192834'13809 (0'0) promote 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by osd.372.0:30243 2014-08-11 15:24:58.428986 2014-08-13 18:06:06.841748 7f5c973a57a0 20 read_log 192834'13810 (192834'13809) modify 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.933448.0:17096 2014-08-11 15:24:58.421764 2014-08-13 18:06:06.841760 7f5c973a57a0 20 read_log 192834'13813 (0'0) promote 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by osd.372.0:30259 2014-08-11 15:24:59.796670 2014-08-13 18:06:06.841772 7f5c973a57a0 20 read_log 192834'13814 (192834'13813) modify 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.933448.0:17463 2014-08-11 15:24:59.790107 2014-08-13 18:06:06.841784 7f5c973a57a0 20 read_log 192834'13817 (0'0) promote 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by osd.372.0:30265 2014-08-11 15:25:01.417570 2014-08-13 18:06:06.841796 7f5c973a57a0 20 read_log 192834'13818 (192834'13817) modify 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.933448.0:17760 2014-08-11 15:25:01.408718 2014-08-13 18:06:06.841809 7f5c973a57a0 20 read_log 192834'13837 (0'0) promote ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by osd.372.0:30297 2014-08-11 15:25:06.296427 2014-08-13 18:06:06.841821 7f5c973a57a0 20 read_log 192834'13838 (192834'13837) modify ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.933448.0:18675 2014-08-11 15:25:06.289483 2014-08-13 18:06:06.841841 7f5c973a57a0 20 read_log 192834'13843 (0'0) promote bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by osd.372.0:30311 2014-08-11 15:25:07.269677 2014-08-13 18:06:06.841856 7f5c973a57a0 20 read_log 192834'13844 (192834'13843) modify bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.933448.0:18869 2014-08-11 15:25:07.262374 2014-08-13 18:06:06.841869 7f5c973a57a0 20 read_log 192834'13865 (0'0) promote d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by osd.372.0:30369 2014-08-11 15:25:22.122486 2014-08-13 18:06:06.841884 7f5c973a57a0 20 read_log 192834'13866 (192834'13865) modify d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.933448.0:20604 2014-08-11 15:25:22.114568 2014-08-13 18:06:06.841896 7f5c973a57a0 20 read_log 192834'13875 (0'0) promote 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by osd.372.0:30381 2014-08-11 15:25:26.072839 2014-08-13 18:06:06.841910 7f5c973a57a0 20 read_log 192834'13876 (192834'13875) modify 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.933448.0:21078 2014-08-11 15:25:26.064541 2014-08-13 18:06:06.841923 7f5c973a57a0 20 read_log 192834'13879 (0'0) promote 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by osd.372.0:30389 2014-08-11 15:25:27.643403 2014-08-13 18:06:06.841937 7f5c973a57a0 20 read_log 192834'13880 (192834'13879) modify 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.933448.0:21166 2014-08-11 15:25:27.634566 2014-08-13 18:06:06.841950 7f5c973a57a0 20 read_log 192834'13898 (192834'13565) modify fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by client.933448.0:23518 2014-08-11 15:25:39.433726 2014-08-13 18:06:06.841964 7f5c973a57a0 20 read_log 192834'13902 (192834'13573) modify f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by client.933448.0:24030 2014-08-11 15:25:40.203989 2014-08-13 18:06:06.841977 7f5c973a57a0 20 read_log 192834'13915 (192834'13599) modify e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.933448.0:25257 2014-08-11 15:25:41.916979 2014-08-13 18:06:06.841991 7f5c973a57a0 20 read_log 192834'13918 (192834'13605) modify c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.933448.0:25356 2014-08-11 15:25:42.026094 2014-08-13 18:06:06.842004 7f5c973a57a0 20 read_log 192834'13921 (192834'13611) modify e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.933448.0:25668 2014-08-11 15:25:42.329133 2014-08-13 18:06:06.842018 7f5c973a57a0 20 read_log 192834'13948 (192834'13666) modify 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.933448.0:28487 2014-08-11 15:25:46.309038 2014-08-13 18:06:06.842030 7f5c973a57a0 20 read_log 192834'13964 (192834'13698) modify d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.933448.0:30581 2014-08-11 15:25:49.016987 2014-08-13 18:06:06.842042 7f5c973a57a0 20 read_log 192834'13965 (192834'13700) modify 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.933448.0:31343 2014-08-11 15:25:49.949046 2014-08-13 18:06:06.842054 7f5c973a57a0 20 read_log 192834'13970 (192834'13710) modify 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.933448.0:31655 2014-08-11 15:25:50.325417 2014-08-13 18:06:06.842067 7f5c973a57a0 20 read_log 192834'13972 (192834'13714) modify 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.933448.0:32251 2014-08-11 15:25:51.333815 2014-08-13 18:06:06.842081 7f5c973a57a0 20 read_log 192834'13977 (192834'13724) modify a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.933448.0:32639 2014-08-11 15:25:52.145391 2014-08-13 18:06:06.842093 7f5c973a57a0 20 read_log 192834'13979 (192834'13728) modify c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.933448.0:32856 2014-08-11 15:25:52.441126 2014-08-13 18:06:06.842118 7f5c973a57a0 20 read_log 192834'13998 (192834'13766) modify c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.933448.0:35305 2014-08-11 15:25:56.076314 2014-08-13 18:06:06.842133 7f5c973a57a0 20 read_log 192834'14013 (192834'13796) modify 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.933448.0:38228 2014-08-11 15:26:00.680245 2014-08-13 18:06:06.842155 7f5c973a57a0 20 read_log 192834'14016 (192834'13802) modify 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.933448.0:38780 2014-08-11 15:26:01.261824 2014-08-13 18:06:06.842171 7f5c973a57a0 20 read_log 192834'14020 (192834'13810) modify 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.933448.0:39387 2014-08-11 15:26:01.839792 2014-08-13 18:06:06.842184 7f5c973a57a0 20 read_log 192834'14022 (192834'13814) modify 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.933448.0:39754 2014-08-11 15:26:02.060829 2014-08-13 18:06:06.842199 7f5c973a57a0 20 read_log 192834'14024 (192834'13818) modify 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.933448.0:40051 2014-08-11 15:26:02.246605 2014-08-13 18:06:06.842211 7f5c973a57a0 20 read_log 192834'14034 (192834'13838) modify ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.933448.0:40966 2014-08-11 15:26:02.801891 2014-08-13 18:06:06.842226 7f5c973a57a0 20 read_log 192834'14037 (192834'13844) modify bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.933448.0:41160 2014-08-11 15:26:02.855920 2014-08-13 18:06:06.842240 7f5c973a57a0 20 read_log 192834'14048 (192834'13866) modify d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.933448.0:42895 2014-08-11 15:26:03.425754 2014-08-13 18:06:06.842255 7f5c973a57a0 20 read_log 192834'14053 (192834'13876) modify 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.933448.0:43369 2014-08-11 15:26:03.591326 2014-08-13 18:06:06.842283 7f5c973a57a0 20 read_log 192834'14055 (192834'13880) modify 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.933448.0:43457 2014-08-11 15:26:03.622668 2014-08-13 18:06:06.842298 7f5c973a57a0 20 read_log 192834'14074 (0'0) promote 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by osd.372.0:30844 2014-08-11 15:26:17.061546 2014-08-13 18:06:06.842314 7f5c973a57a0 20 read_log 192834'14075 (192834'14074) modify 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.1883634.0:706 2014-08-11 15:26:17.054534 2014-08-13 18:06:06.842329 7f5c973a57a0 20 read_log 192834'14098 (0'0) promote bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by osd.372.0:30884 2014-08-11 15:26:21.739061 2014-08-13 18:06:06.842341 7f5c973a57a0 20 read_log 192834'14099 (192834'14098) modify bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.1883634.0:1967 2014-08-11 15:26:21.732715 2014-08-13 18:06:06.842354 7f5c973a57a0 20 read_log 192834'14108 (0'0) promote 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by osd.372.0:30906 2014-08-11 15:26:23.537702 2014-08-13 18:06:06.842366 7f5c973a57a0 20 read_log 192834'14109 (192834'14108) modify 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.1883634.0:2456 2014-08-11 15:26:23.529686 2014-08-13 18:06:06.842379 7f5c973a57a0 20 read_log 192834'14130 (0'0) promote 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by osd.372.0:30958 2014-08-11 15:26:31.942985 2014-08-13 18:06:06.842391 7f5c973a57a0 20 read_log 192834'14131 (192834'14130) modify 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.1883634.0:4792 2014-08-11 15:26:31.935135 2014-08-13 18:06:06.842407 7f5c973a57a0 20 read_log 192834'14154 (0'0) promote ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by osd.372.0:31014 2014-08-11 15:26:39.832357 2014-08-13 18:06:06.842430 7f5c973a57a0 20 read_log 192834'14155 (192834'14154) modify ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.1883634.0:6882 2014-08-11 15:26:39.826591 2014-08-13 18:06:06.842457 7f5c973a57a0 20 read_log 192834'14164 (0'0) promote d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by osd.372.0:31024 2014-08-11 15:26:40.691780 2014-08-13 18:06:06.842470 7f5c973a57a0 20 read_log 192834'14165 (192834'14164) modify d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.1883634.0:7072 2014-08-11 15:26:40.685654 2014-08-13 18:06:06.842484 7f5c973a57a0 20 read_log 192834'14174 (0'0) promote a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by osd.372.0:31046 2014-08-11 15:26:43.679546 2014-08-13 18:06:06.842506 7f5c973a57a0 20 read_log 192834'14175 (192834'14174) modify a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.1883634.0:7795 2014-08-11 15:26:43.671750 2014-08-13 18:06:06.842519 7f5c973a57a0 20 read_log 192834'14178 (0'0) promote eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by osd.372.0:31050 2014-08-11 15:26:44.136714 2014-08-13 18:06:06.842532 7f5c973a57a0 20 read_log 192834'14179 (192834'14178) modify eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.1883634.0:7868 2014-08-11 15:26:44.127881 2014-08-13 18:06:06.842554 7f5c973a57a0 20 read_log 192834'14184 (0'0) promote 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by osd.372.0:31056 2014-08-11 15:26:44.704057 2014-08-13 18:06:06.842567 7f5c973a57a0 20 read_log 192834'14185 (192834'14184) modify 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.1883634.0:8026 2014-08-11 15:26:44.694437 2014-08-13 18:06:06.842593 7f5c973a57a0 20 read_log 192834'14200 (0'0) promote 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by osd.372.0:31102 2014-08-11 15:26:51.073804 2014-08-13 18:06:06.842609 7f5c973a57a0 20 read_log 192834'14201 (192834'14200) modify 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.1883634.0:9498 2014-08-11 15:26:51.066901 2014-08-13 18:06:06.842622 7f5c973a57a0 20 read_log 192834'14210 (0'0) promote b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by osd.372.0:31130 2014-08-11 15:26:54.944265 2014-08-13 18:06:06.842634 7f5c973a57a0 20 read_log 192834'14211 (192834'14210) modify b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.1883634.0:10405 2014-08-11 15:26:54.935611 2014-08-13 18:06:06.842646 7f5c973a57a0 20 read_log 192834'14236 (0'0) promote 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by osd.372.0:31172 2014-08-11 15:27:01.407681 2014-08-13 18:06:06.842659 7f5c973a57a0 20 read_log 192834'14237 (192834'14236) modify 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.1883634.0:11829 2014-08-11 15:27:01.399626 2014-08-13 18:06:06.842671 7f5c973a57a0 20 read_log 192834'14242 (0'0) promote 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by osd.372.0:31186 2014-08-11 15:27:02.956335 2014-08-13 18:06:06.842683 7f5c973a57a0 20 read_log 192834'14243 (192834'14242) modify 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.1883634.0:12196 2014-08-11 15:27:02.947506 2014-08-13 18:06:06.842695 7f5c973a57a0 20 read_log 192834'14248 (0'0) promote cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by osd.372.0:31200 2014-08-11 15:27:04.345674 2014-08-13 18:06:06.842707 7f5c973a57a0 20 read_log 192834'14249 (192834'14248) modify cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.1883634.0:12525 2014-08-11 15:27:04.335830 2014-08-13 18:06:06.842720 7f5c973a57a0 20 read_log 192834'14254 (0'0) promote 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by osd.372.0:31208 2014-08-11 15:27:06.213175 2014-08-13 18:06:06.842742 7f5c973a57a0 20 read_log 192834'14255 (192834'14254) modify 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.1883634.0:12818 2014-08-11 15:27:06.206725 2014-08-13 18:06:06.842755 7f5c973a57a0 20 read_log 192834'14258 (0'0) promote 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by osd.372.0:31216 2014-08-11 15:27:07.933355 2014-08-13 18:06:06.842769 7f5c973a57a0 20 read_log 192834'14259 (192834'14258) modify 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.1883634.0:13162 2014-08-11 15:27:07.924025 2014-08-13 18:06:06.842782 7f5c973a57a0 20 read_log 192834'14276 (0'0) promote 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by osd.372.0:31250 2014-08-11 15:27:11.305149 2014-08-13 18:06:06.842794 7f5c973a57a0 20 read_log 192834'14277 (192834'14276) modify 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.1883634.0:13944 2014-08-11 15:27:11.297051 2014-08-13 18:06:06.842806 7f5c973a57a0 20 read_log 192834'14304 (0'0) promote ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by osd.372.0:31298 2014-08-11 15:27:19.392236 2014-08-13 18:06:06.842818 7f5c973a57a0 20 read_log 192834'14305 (192834'14304) modify ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.1883634.0:15406 2014-08-11 15:27:19.385383 2014-08-13 18:06:06.842831 7f5c973a57a0 20 read_log 192834'14306 (0'0) promote 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by osd.372.0:31300 2014-08-11 15:27:19.906962 2014-08-13 18:06:06.842851 7f5c973a57a0 20 read_log 192834'14307 (192834'14306) modify 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.1883634.0:15518 2014-08-11 15:27:19.898786 2014-08-13 18:06:06.842868 7f5c973a57a0 20 read_log 192834'14336 (0'0) promote 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by osd.372.0:31356 2014-08-11 15:27:40.059711 2014-08-13 18:06:06.842880 7f5c973a57a0 20 read_log 192834'14337 (192834'14336) modify 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.1883634.0:17928 2014-08-11 15:27:40.049894 2014-08-13 18:06:06.842903 7f5c973a57a0 20 read_log 192834'14344 (0'0) promote 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by osd.372.0:31370 2014-08-11 15:27:47.115181 2014-08-13 18:06:06.842916 7f5c973a57a0 20 read_log 192834'14345 (192834'14344) modify 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.1883634.0:18693 2014-08-11 15:27:47.106754 2014-08-13 18:06:06.842928 7f5c973a57a0 20 read_log 192834'14352 (0'0) promote 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by osd.372.0:31390 2014-08-11 15:27:49.034253 2014-08-13 18:06:06.842941 7f5c973a57a0 20 read_log 192834'14353 (192834'14352) modify 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.1883634.0:19038 2014-08-11 15:27:49.026447 2014-08-13 18:06:06.842953 7f5c973a57a0 20 read_log 192834'14354 (0'0) promote 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by osd.372.0:31400 2014-08-11 15:27:51.310863 2014-08-13 18:06:06.842965 7f5c973a57a0 20 read_log 192834'14355 (192834'14354) modify 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.1883634.0:19329 2014-08-11 15:27:51.302086 2014-08-13 18:06:06.842978 7f5c973a57a0 20 read_log 192834'14360 (0'0) promote be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by osd.372.0:31416 2014-08-11 15:27:54.694449 2014-08-13 18:06:06.842991 7f5c973a57a0 20 read_log 192834'14361 (192834'14360) modify be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.1883634.0:19934 2014-08-11 15:27:54.685261 2014-08-13 18:06:06.843004 7f5c973a57a0 20 read_log 192834'14374 (0'0) promote e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by osd.372.0:31440 2014-08-11 15:28:01.671420 2014-08-13 18:06:06.843026 7f5c973a57a0 20 read_log 192834'14375 (192834'14374) modify e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.1883634.0:20821 2014-08-11 15:28:01.662663 2014-08-13 18:06:06.843042 7f5c973a57a0 20 read_log 192834'14376 (0'0) promote eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by osd.372.0:31442 2014-08-11 15:28:01.899250 2014-08-13 18:06:06.843054 7f5c973a57a0 20 read_log 192834'14377 (192834'14376) modify eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.1883634.0:20883 2014-08-11 15:28:01.890913 2014-08-13 18:06:06.843067 7f5c973a57a0 20 read_log 192834'14392 (0'0) promote a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by osd.372.0:31500 2014-08-11 15:28:12.155667 2014-08-13 18:06:06.843079 7f5c973a57a0 20 read_log 192834'14393 (192834'14392) modify a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.1883634.0:23029 2014-08-11 15:28:12.147537 2014-08-13 18:06:06.843091 7f5c973a57a0 20 read_log 192834'14404 (192834'14075) modify 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.1883634.0:24030 2014-08-11 15:28:18.262266 2014-08-13 18:06:06.843103 7f5c973a57a0 20 read_log 192834'14416 (192834'14099) modify bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.1883634.0:25291 2014-08-11 15:28:20.465682 2014-08-13 18:06:06.843116 7f5c973a57a0 20 read_log 192834'14421 (192834'14109) modify 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.1883634.0:25780 2014-08-11 15:28:21.861010 2014-08-13 18:06:06.843128 7f5c973a57a0 20 read_log 192834'14432 (192834'14131) modify 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.1883634.0:28116 2014-08-11 15:28:25.396671 2014-08-13 18:06:06.843141 7f5c973a57a0 20 read_log 192834'14444 (192834'14155) modify ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.1883634.0:30206 2014-08-11 15:28:28.228450 2014-08-13 18:06:06.843168 7f5c973a57a0 20 read_log 192834'14449 (192834'14165) modify d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.1883634.0:30396 2014-08-11 15:28:28.592812 2014-08-13 18:06:06.843185 7f5c973a57a0 20 read_log 192834'14454 (192834'14175) modify a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.1883634.0:31119 2014-08-11 15:28:29.932695 2014-08-13 18:06:06.843197 7f5c973a57a0 20 read_log 192834'14456 (192834'14179) modify eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.1883634.0:31192 2014-08-11 15:28:30.075352 2014-08-13 18:06:06.843225 7f5c973a57a0 20 read_log 192834'14459 (192834'14185) modify 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.1883634.0:31350 2014-08-11 15:28:30.363848 2014-08-13 18:06:06.843238 7f5c973a57a0 20 read_log 192834'14467 (192834'14201) modify 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.1883634.0:32822 2014-08-11 15:28:32.371063 2014-08-13 18:06:06.843251 7f5c973a57a0 20 read_log 192834'14472 (192834'14211) modify b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.1883634.0:33729 2014-08-11 15:28:33.737774 2014-08-13 18:06:06.843271 7f5c973a57a0 20 read_log 192834'14485 (192834'14237) modify 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.1883634.0:35153 2014-08-11 15:28:36.103936 2014-08-13 18:06:06.843290 7f5c973a57a0 20 read_log 192834'14488 (192834'14243) modify 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.1883634.0:35520 2014-08-11 15:28:36.773981 2014-08-13 18:06:06.843305 7f5c973a57a0 20 read_log 192834'14493 (192834'14249) modify cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.1883634.0:35849 2014-08-11 15:28:37.223281 2014-08-13 18:06:06.843329 7f5c973a57a0 20 read_log 192834'14496 (192834'14255) modify 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.1883634.0:36142 2014-08-11 15:28:37.621637 2014-08-13 18:06:06.843349 7f5c973a57a0 20 read_log 192834'14498 (192834'14259) modify 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.1883634.0:36486 2014-08-11 15:28:38.075389 2014-08-13 18:06:06.843364 7f5c973a57a0 20 read_log 192834'14507 (192834'14277) modify 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.1883634.0:37268 2014-08-11 15:28:39.095287 2014-08-13 18:06:06.843376 7f5c973a57a0 20 read_log 192834'14521 (192834'14305) modify ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.1883634.0:38730 2014-08-11 15:28:41.074582 2014-08-13 18:06:06.843389 7f5c973a57a0 20 read_log 192834'14522 (192834'14307) modify 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.1883634.0:38842 2014-08-11 15:28:41.199634 2014-08-13 18:06:06.843401 7f5c973a57a0 20 read_log 192834'14537 (192834'14337) modify 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.1883634.0:41252 2014-08-11 15:28:44.814170 2014-08-13 18:06:06.843413 7f5c973a57a0 20 read_log 192834'14541 (192834'14345) modify 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.1883634.0:42017 2014-08-11 15:28:45.638669 2014-08-13 18:06:06.843426 7f5c973a57a0 20 read_log 192834'14545 (192834'14353) modify 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.1883634.0:42362 2014-08-11 15:28:46.219619 2014-08-13 18:06:06.843438 7f5c973a57a0 20 read_log 192834'14546 (192834'14355) modify 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.1883634.0:42653 2014-08-11 15:28:46.313273 2014-08-13 18:06:06.843450 7f5c973a57a0 20 read_log 192834'14549 (192834'14361) modify be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.1883634.0:43258 2014-08-11 15:28:46.501937 2014-08-13 18:06:06.843484 7f5c973a57a0 20 read_log 192834'14556 (192834'14375) modify e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.1883634.0:44145 2014-08-11 15:28:46.774131 2014-08-13 18:06:06.843501 7f5c973a57a0 20 read_log 192834'14557 (192834'14377) modify eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.1883634.0:44207 2014-08-11 15:28:46.795364 2014-08-13 18:06:06.843514 7f5c973a57a0 20 read_log 192834'14565 (192834'14393) modify a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.1883634.0:46353 2014-08-11 15:28:47.566734 2014-08-13 18:06:06.843526 7f5c973a57a0 20 read_log 192888'14569 (0'0) promote f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by osd.296.0:2325 2014-08-11 16:45:00.353250 2014-08-13 18:06:06.843538 7f5c973a57a0 20 read_log 192888'14570 (192888'14569) modify f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.1315259.0:1334 2014-08-11 16:45:00.336957 2014-08-13 18:06:06.843551 7f5c973a57a0 20 read_log 192888'14575 (0'0) promote 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by osd.296.0:2331 2014-08-11 16:45:09.240695 2014-08-13 18:06:06.843563 7f5c973a57a0 20 read_log 192888'14576 (192888'14575) modify 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.1315259.0:3670 2014-08-11 16:45:09.233471 2014-08-13 18:06:06.843597 7f5c973a57a0 20 read_log 192888'14579 (0'0) promote 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by osd.296.0:2335 2014-08-11 16:45:13.293513 2014-08-13 18:06:06.843610 7f5c973a57a0 20 read_log 192888'14580 (192888'14579) modify 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.1315259.0:4780 2014-08-11 16:45:13.285590 2014-08-13 18:06:06.843623 7f5c973a57a0 20 read_log 192888'14583 (0'0) promote e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by osd.296.0:2339 2014-08-11 16:45:15.678497 2014-08-13 18:06:06.843660 7f5c973a57a0 20 read_log 192888'14584 (192888'14583) modify e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.1315259.0:5419 2014-08-11 16:45:15.671719 2014-08-13 18:06:06.843677 7f5c973a57a0 20 read_log 192888'14585 (0'0) promote 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by osd.296.0:2341 2014-08-11 16:45:16.764595 2014-08-13 18:06:06.843689 7f5c973a57a0 20 read_log 192888'14586 (192888'14585) modify 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.1315259.0:5718 2014-08-11 16:45:16.757542 2014-08-13 18:06:06.843702 7f5c973a57a0 20 read_log 192888'14587 (0'0) promote 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by osd.296.0:2343 2014-08-11 16:45:20.922722 2014-08-13 18:06:06.843729 7f5c973a57a0 20 read_log 192888'14588 (192888'14587) modify 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.1315259.0:6869 2014-08-11 16:45:20.915355 2014-08-13 18:06:06.843744 7f5c973a57a0 20 read_log 192888'14589 (0'0) promote e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by osd.296.0:2345 2014-08-11 16:45:30.233888 2014-08-13 18:06:06.843756 7f5c973a57a0 20 read_log 192888'14590 (192888'14589) modify e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.1315259.0:9379 2014-08-11 16:45:30.225308 2014-08-13 18:06:06.843769 7f5c973a57a0 20 read_log 192888'14595 (0'0) promote a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by osd.296.0:2351 2014-08-11 16:45:34.040290 2014-08-13 18:06:06.843783 7f5c973a57a0 20 read_log 192888'14596 (192888'14595) modify a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.1315259.0:10363 2014-08-11 16:45:34.033118 2014-08-13 18:06:06.843796 7f5c973a57a0 20 read_log 192888'14597 (0'0) promote d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by osd.296.0:2353 2014-08-11 16:45:35.658200 2014-08-13 18:06:06.843810 7f5c973a57a0 20 read_log 192888'14598 (192888'14597) modify d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.1315259.0:10760 2014-08-11 16:45:35.649339 2014-08-13 18:06:06.843823 7f5c973a57a0 20 read_log 192888'14607 (0'0) promote 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by osd.296.0:2363 2014-08-11 16:45:49.965880 2014-08-13 18:06:06.843836 7f5c973a57a0 20 read_log 192888'14608 (192888'14607) modify 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.1315259.0:14301 2014-08-11 16:45:49.957333 2014-08-13 18:06:06.843850 7f5c973a57a0 20 read_log 192888'14609 (0'0) promote 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by osd.296.0:2365 2014-08-11 16:45:50.653718 2014-08-13 18:06:06.843883 7f5c973a57a0 20 read_log 192888'14610 (192888'14609) modify 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.1315259.0:14489 2014-08-11 16:45:50.647225 2014-08-13 18:06:06.843896 7f5c973a57a0 20 read_log 192888'14617 (0'0) promote 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by osd.296.0:2373 2014-08-11 16:46:07.923625 2014-08-13 18:06:06.843910 7f5c973a57a0 20 read_log 192888'14618 (192888'14617) modify 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.1315259.0:18935 2014-08-11 16:46:07.917642 2014-08-13 18:06:06.843923 7f5c973a57a0 20 read_log 192888'14619 (0'0) promote 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by osd.296.0:2375 2014-08-11 16:46:08.299473 2014-08-13 18:06:06.843937 7f5c973a57a0 20 read_log 192888'14620 (192888'14619) modify 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.1315259.0:19014 2014-08-11 16:46:08.292465 2014-08-13 18:06:06.843950 7f5c973a57a0 20 read_log 192888'14627 (0'0) promote a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by osd.296.0:2383 2014-08-11 16:46:19.879634 2014-08-13 18:06:06.843971 7f5c973a57a0 20 read_log 192888'14628 (192888'14627) modify a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.1315259.0:22059 2014-08-11 16:46:19.871353 2014-08-13 18:06:06.843986 7f5c973a57a0 20 read_log 192888'14629 (0'0) promote 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by osd.296.0:2385 2014-08-11 16:46:20.263146 2014-08-13 18:06:06.843999 7f5c973a57a0 20 read_log 192888'14630 (192888'14629) modify 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.1315259.0:22142 2014-08-11 16:46:20.256768 2014-08-13 18:06:06.844011 7f5c973a57a0 20 read_log 192888'14637 (0'0) promote e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by osd.296.0:2393 2014-08-11 16:46:26.322618 2014-08-13 18:06:06.844025 7f5c973a57a0 20 read_log 192888'14638 (192888'14637) modify e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.1315259.0:23649 2014-08-11 16:46:26.315367 2014-08-13 18:06:06.844038 7f5c973a57a0 20 read_log 192888'14641 (0'0) promote 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by osd.296.0:2397 2014-08-11 16:46:27.773121 2014-08-13 18:06:06.844052 7f5c973a57a0 20 read_log 192888'14642 (192888'14641) modify 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.1315259.0:24035 2014-08-11 16:46:27.766588 2014-08-13 18:06:06.844066 7f5c973a57a0 20 read_log 192888'14647 (0'0) promote 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by osd.296.0:2403 2014-08-11 16:46:40.488543 2014-08-13 18:06:06.844079 7f5c973a57a0 20 read_log 192888'14648 (192888'14647) modify 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.1315259.0:27112 2014-08-11 16:46:40.479513 2014-08-13 18:06:06.844091 7f5c973a57a0 20 read_log 192888'14649 (0'0) promote 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by osd.296.0:2405 2014-08-11 16:46:42.764523 2014-08-13 18:06:06.844105 7f5c973a57a0 20 read_log 192888'14650 (192888'14649) modify 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.1315259.0:27693 2014-08-11 16:46:42.757427 2014-08-13 18:06:06.844118 7f5c973a57a0 20 read_log 192888'14651 (0'0) promote 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by osd.296.0:2407 2014-08-11 16:46:42.799372 2014-08-13 18:06:06.844134 7f5c973a57a0 20 read_log 192888'14652 (192888'14651) modify 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.1315259.0:27702 2014-08-11 16:46:42.792916 2014-08-13 18:06:06.844154 7f5c973a57a0 20 read_log 192888'14657 (0'0) promote 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by osd.296.0:2413 2014-08-11 16:46:49.693938 2014-08-13 18:06:06.844170 7f5c973a57a0 20 read_log 192888'14658 (192888'14657) modify 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.1315259.0:29412 2014-08-11 16:46:49.685707 2014-08-13 18:06:06.844182 7f5c973a57a0 20 read_log 192888'14663 (0'0) promote 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by osd.296.0:2419 2014-08-11 16:46:54.351232 2014-08-13 18:06:06.844197 7f5c973a57a0 20 read_log 192888'14664 (192888'14663) modify 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.1315259.0:30593 2014-08-11 16:46:54.343627 2014-08-13 18:06:06.844209 7f5c973a57a0 20 read_log 192888'14665 (192888'14570) modify f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.1315259.0:32061 2014-08-11 16:46:56.280204 2014-08-13 18:06:06.844224 7f5c973a57a0 20 read_log 192888'14668 (192888'14576) modify 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.1315259.0:34397 2014-08-11 16:46:56.882272 2014-08-13 18:06:06.844236 7f5c973a57a0 20 read_log 192888'14670 (192888'14580) modify 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.1315259.0:35507 2014-08-11 16:46:57.178795 2014-08-13 18:06:06.844260 7f5c973a57a0 20 read_log 192888'14672 (192888'14584) modify e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.1315259.0:36146 2014-08-11 16:46:57.348196 2014-08-13 18:06:06.844276 7f5c973a57a0 20 read_log 192888'14673 (192888'14586) modify 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.1315259.0:36445 2014-08-11 16:46:57.424701 2014-08-13 18:06:06.844289 7f5c973a57a0 20 read_log 192888'14674 (192888'14588) modify 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.1315259.0:37596 2014-08-11 16:46:57.725370 2014-08-13 18:06:06.844301 7f5c973a57a0 20 read_log 192888'14675 (192888'14590) modify e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.1315259.0:40106 2014-08-11 16:46:58.390803 2014-08-13 18:06:06.844313 7f5c973a57a0 20 read_log 192888'14678 (192888'14596) modify a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.1315259.0:41090 2014-08-11 16:46:58.656180 2014-08-13 18:06:06.844325 7f5c973a57a0 20 read_log 192888'14679 (192888'14598) modify d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.1315259.0:41487 2014-08-11 16:46:58.758401 2014-08-13 18:06:06.844341 7f5c973a57a0 20 read_log 192888'14684 (192888'14608) modify 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.1315259.0:45028 2014-08-11 16:46:59.748910 2014-08-13 18:06:06.844356 7f5c973a57a0 20 read_log 192888'14685 (192888'14610) modify 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.1315259.0:45216 2014-08-11 16:46:59.803345 2014-08-13 18:06:06.844369 7f5c973a57a0 20 read_log 192888'14689 (192888'14618) modify 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.1315259.0:49662 2014-08-11 16:47:01.135983 2014-08-13 18:06:06.844392 7f5c973a57a0 20 read_log 192888'14690 (192888'14620) modify 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.1315259.0:49741 2014-08-11 16:47:01.158818 2014-08-13 18:06:06.844409 7f5c973a57a0 20 read_log 192888'14694 (192888'14628) modify a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.1315259.0:52786 2014-08-11 16:47:02.068769 2014-08-13 18:06:06.844422 7f5c973a57a0 20 read_log 192888'14695 (192888'14630) modify 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.1315259.0:52869 2014-08-11 16:47:02.092484 2014-08-13 18:06:06.844445 7f5c973a57a0 20 read_log 192888'14699 (192888'14638) modify e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.1315259.0:54376 2014-08-11 16:47:02.549643 2014-08-13 18:06:06.844461 7f5c973a57a0 20 read_log 192888'14701 (192888'14642) modify 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.1315259.0:54762 2014-08-11 16:47:02.686222 2014-08-13 18:06:06.844473 7f5c973a57a0 20 read_log 192888'14704 (192888'14648) modify 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.1315259.0:57839 2014-08-11 16:47:03.628121 2014-08-13 18:06:06.844493 7f5c973a57a0 20 read_log 192888'14705 (192888'14650) modify 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.1315259.0:58420 2014-08-11 16:47:03.796443 2014-08-13 18:06:06.844508 7f5c973a57a0 20 read_log 192888'14706 (192888'14652) modify 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.1315259.0:58429 2014-08-11 16:47:03.799517 2014-08-13 18:06:06.844521 7f5c973a57a0 20 read_log 192888'14709 (192888'14658) modify 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.1315259.0:60139 2014-08-11 16:47:04.355840 2014-08-13 18:06:06.844534 7f5c973a57a0 20 read_log 192888'14712 (192888'14664) modify 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.1315259.0:61320 2014-08-11 16:47:04.703546 2014-08-13 18:06:06.844546 7f5c973a57a0 20 read_log 192888'14715 (0'0) promote aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by osd.296.0:2470 2014-08-11 17:05:15.027834 2014-08-13 18:06:06.844568 7f5c973a57a0 20 read_log 192888'14716 (192888'14715) modify aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.942868.0:50 2014-08-11 17:05:15.017412 2014-08-13 18:06:06.844584 7f5c973a57a0 20 read_log 192888'14721 (0'0) promote 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by osd.296.0:2476 2014-08-11 17:05:21.587792 2014-08-13 18:06:06.844607 7f5c973a57a0 20 read_log 192888'14722 (192888'14721) modify 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.942868.0:1590 2014-08-11 17:05:21.579251 2014-08-13 18:06:06.844628 7f5c973a57a0 20 read_log 192888'14723 (0'0) promote 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by osd.296.0:2478 2014-08-11 17:05:23.420991 2014-08-13 18:06:06.844649 7f5c973a57a0 20 read_log 192888'14724 (192888'14723) modify 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.942868.0:2059 2014-08-11 17:05:23.413425 2014-08-13 18:06:06.844664 7f5c973a57a0 20 read_log 192888'14725 (0'0) promote 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by osd.296.0:2480 2014-08-11 17:05:27.975928 2014-08-13 18:06:06.844677 7f5c973a57a0 20 read_log 192888'14726 (192888'14725) modify 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.942868.0:3252 2014-08-11 17:05:27.966253 2014-08-13 18:06:06.844698 7f5c973a57a0 20 read_log 192888'14729 (0'0) promote 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by osd.296.0:2484 2014-08-11 17:05:28.717614 2014-08-13 18:06:06.844713 7f5c973a57a0 20 read_log 192888'14730 (192888'14729) modify 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.942868.0:3454 2014-08-11 17:05:28.709618 2014-08-13 18:06:06.844726 7f5c973a57a0 20 read_log 192888'14733 (0'0) promote 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by osd.296.0:2488 2014-08-11 17:05:34.020762 2014-08-13 18:06:06.844738 7f5c973a57a0 20 read_log 192888'14734 (192888'14733) modify 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.942868.0:4860 2014-08-11 17:05:34.013231 2014-08-13 18:06:06.844750 7f5c973a57a0 20 read_log 192888'14743 (0'0) promote 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by osd.296.0:2498 2014-08-11 17:05:39.534276 2014-08-13 18:06:06.844763 7f5c973a57a0 20 read_log 192888'14744 (192888'14743) modify 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.942868.0:6337 2014-08-11 17:05:39.526134 2014-08-13 18:06:06.844775 7f5c973a57a0 20 read_log 192888'14745 (0'0) promote f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by osd.296.0:2500 2014-08-11 17:05:39.673797 2014-08-13 18:06:06.844787 7f5c973a57a0 20 read_log 192888'14746 (192888'14745) modify f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.942868.0:6376 2014-08-11 17:05:39.668171 2014-08-13 18:06:06.844799 7f5c973a57a0 20 read_log 192888'14749 (0'0) promote 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by osd.296.0:2504 2014-08-11 17:05:44.449483 2014-08-13 18:06:06.844812 7f5c973a57a0 20 read_log 192888'14750 (192888'14749) modify 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.942868.0:7618 2014-08-11 17:05:44.441850 2014-08-13 18:06:06.844824 7f5c973a57a0 20 read_log 192888'14751 (0'0) promote dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by osd.296.0:2506 2014-08-11 17:05:45.032638 2014-08-13 18:06:06.844836 7f5c973a57a0 20 read_log 192888'14752 (192888'14751) modify dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.942868.0:7780 2014-08-11 17:05:45.025885 2014-08-13 18:06:06.844849 7f5c973a57a0 20 read_log 192888'14753 (0'0) promote cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by osd.296.0:2508 2014-08-11 17:05:45.829366 2014-08-13 18:06:06.844868 7f5c973a57a0 20 read_log 192888'14754 (192888'14753) modify cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.942868.0:8001 2014-08-11 17:05:45.822674 2014-08-13 18:06:06.844881 7f5c973a57a0 20 read_log 192888'14757 (0'0) promote ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by osd.296.0:2512 2014-08-11 17:05:48.191038 2014-08-13 18:06:06.844893 7f5c973a57a0 20 read_log 192888'14758 (192888'14757) modify ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.942868.0:8654 2014-08-11 17:05:48.182545 2014-08-13 18:06:06.844905 7f5c973a57a0 20 read_log 192888'14761 (0'0) promote c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by osd.296.0:2516 2014-08-11 17:05:56.170686 2014-08-13 18:06:06.844918 7f5c973a57a0 20 read_log 192888'14762 (192888'14761) modify c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.942868.0:10825 2014-08-11 17:05:56.163465 2014-08-13 18:06:06.844930 7f5c973a57a0 20 read_log 192888'14765 (0'0) promote aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by osd.296.0:2520 2014-08-11 17:05:59.380916 2014-08-13 18:06:06.844942 7f5c973a57a0 20 read_log 192888'14766 (192888'14765) modify aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.942868.0:11699 2014-08-11 17:05:59.374644 2014-08-13 18:06:06.844957 7f5c973a57a0 20 read_log 192888'14769 (0'0) promote 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by osd.296.0:2524 2014-08-11 17:06:02.272971 2014-08-13 18:06:06.844969 7f5c973a57a0 20 read_log 192888'14770 (192888'14769) modify 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.942868.0:12494 2014-08-11 17:06:02.265943 2014-08-13 18:06:06.844981 7f5c973a57a0 20 read_log 192888'14773 (0'0) promote 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by osd.296.0:2528 2014-08-11 17:06:08.508392 2014-08-13 18:06:06.844994 7f5c973a57a0 20 read_log 192888'14774 (192888'14773) modify 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.942868.0:14068 2014-08-11 17:06:08.501394 2014-08-13 18:06:06.845006 7f5c973a57a0 20 read_log 192888'14775 (0'0) promote 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by osd.296.0:2530 2014-08-11 17:06:08.850292 2014-08-13 18:06:06.845018 7f5c973a57a0 20 read_log 192888'14776 (192888'14775) modify 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.942868.0:14152 2014-08-11 17:06:08.843598 2014-08-13 18:06:06.845030 7f5c973a57a0 20 read_log 192888'14781 (0'0) promote 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by osd.296.0:2536 2014-08-11 17:06:15.348129 2014-08-13 18:06:06.845042 7f5c973a57a0 20 read_log 192888'14782 (192888'14781) modify 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.942868.0:15803 2014-08-11 17:06:15.340211 2014-08-13 18:06:06.845058 7f5c973a57a0 20 read_log 192888'14783 (0'0) promote e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by osd.296.0:2538 2014-08-11 17:06:16.699901 2014-08-13 18:06:06.845070 7f5c973a57a0 20 read_log 192888'14784 (192888'14783) modify e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.942868.0:16154 2014-08-11 17:06:16.691337 2014-08-13 18:06:06.845105 7f5c973a57a0 20 read_log 192888'14787 (0'0) promote d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by osd.296.0:2542 2014-08-11 17:06:19.730195 2014-08-13 18:06:06.845121 7f5c973a57a0 20 read_log 192888'14788 (192888'14787) modify d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.942868.0:16911 2014-08-11 17:06:19.719683 2014-08-13 18:06:06.845133 7f5c973a57a0 20 read_log 192888'14795 (0'0) promote 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by osd.296.0:2550 2014-08-11 17:06:23.909781 2014-08-13 18:06:06.845159 7f5c973a57a0 20 read_log 192888'14796 (192888'14795) modify 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.942868.0:18021 2014-08-11 17:06:23.903005 2014-08-13 18:06:06.845175 7f5c973a57a0 20 read_log 192888'14797 (0'0) promote 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by osd.296.0:2552 2014-08-11 17:06:24.144016 2014-08-13 18:06:06.845188 7f5c973a57a0 20 read_log 192888'14798 (192888'14797) modify 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.942868.0:18083 2014-08-11 17:06:24.137346 2014-08-13 18:06:06.845202 7f5c973a57a0 20 read_log 192888'14805 (0'0) promote e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by osd.296.0:2560 2014-08-11 17:06:36.053544 2014-08-13 18:06:06.845215 7f5c973a57a0 20 read_log 192888'14806 (192888'14805) modify e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.942868.0:21215 2014-08-11 17:06:36.047096 2014-08-13 18:06:06.845229 7f5c973a57a0 20 read_log 192888'14807 (0'0) promote 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by osd.296.0:2562 2014-08-11 17:06:36.312146 2014-08-13 18:06:06.845241 7f5c973a57a0 20 read_log 192888'14808 (192888'14807) modify 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.942868.0:21285 2014-08-11 17:06:36.305555 2014-08-13 18:06:06.845254 7f5c973a57a0 20 read_log 192888'14809 (0'0) promote 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by osd.296.0:2564 2014-08-11 17:06:37.218916 2014-08-13 18:06:06.845271 7f5c973a57a0 20 read_log 192888'14810 (192888'14809) modify 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.942868.0:21529 2014-08-11 17:06:37.212501 2014-08-13 18:06:06.845284 7f5c973a57a0 20 read_log 192888'14811 (0'0) promote aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by osd.296.0:2566 2014-08-11 17:06:37.390625 2014-08-13 18:06:06.845296 7f5c973a57a0 20 read_log 192888'14812 (192888'14811) modify aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.942868.0:21575 2014-08-11 17:06:37.383476 2014-08-13 18:06:06.845308 7f5c973a57a0 20 read_log 192888'14813 (0'0) promote 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by osd.296.0:2568 2014-08-11 17:06:41.624724 2014-08-13 18:06:06.845320 7f5c973a57a0 20 read_log 192888'14814 (192888'14813) modify 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.942868.0:22665 2014-08-11 17:06:41.617733 2014-08-13 18:06:06.845337 7f5c973a57a0 20 read_log 192888'14817 (0'0) promote 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by osd.296.0:2572 2014-08-11 17:06:44.632273 2014-08-13 18:06:06.845349 7f5c973a57a0 20 read_log 192888'14818 (192888'14817) modify 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.942868.0:23436 2014-08-11 17:06:44.625170 2014-08-13 18:06:06.845361 7f5c973a57a0 20 read_log 192888'14819 (0'0) promote 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by osd.296.0:2574 2014-08-11 17:06:46.265673 2014-08-13 18:06:06.845374 7f5c973a57a0 20 read_log 192888'14820 (192888'14819) modify 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.942868.0:23879 2014-08-11 17:06:46.259401 2014-08-13 18:06:06.845405 7f5c973a57a0 20 read_log 192888'14823 (0'0) promote 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by osd.296.0:2578 2014-08-11 17:06:49.847754 2014-08-13 18:06:06.845421 7f5c973a57a0 20 read_log 192888'14824 (192888'14823) modify 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.942868.0:24847 2014-08-11 17:06:49.838674 2014-08-13 18:06:06.845449 7f5c973a57a0 20 read_log 192888'14825 (0'0) promote 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by osd.296.0:2580 2014-08-11 17:06:50.866167 2014-08-13 18:06:06.845475 7f5c973a57a0 20 read_log 192888'14826 (192888'14825) modify 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.942868.0:25122 2014-08-11 17:06:50.858902 2014-08-13 18:06:06.845491 7f5c973a57a0 20 read_log 192888'14829 (0'0) promote 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by osd.296.0:2584 2014-08-11 17:06:54.429010 2014-08-13 18:06:06.845512 7f5c973a57a0 20 read_log 192888'14830 (192888'14829) modify 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.942868.0:26008 2014-08-11 17:06:54.422492 2014-08-13 18:06:06.845528 7f5c973a57a0 20 read_log 192888'14833 (0'0) promote 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by osd.296.0:2588 2014-08-11 17:06:57.883108 2014-08-13 18:06:06.845540 7f5c973a57a0 20 read_log 192888'14834 (192888'14833) modify 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.942868.0:26906 2014-08-11 17:06:57.873918 2014-08-13 18:06:06.845553 7f5c973a57a0 20 read_log 192888'14835 (0'0) promote 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by osd.296.0:2590 2014-08-11 17:07:01.593576 2014-08-13 18:06:06.845565 7f5c973a57a0 20 read_log 192888'14836 (192888'14835) modify 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.942868.0:27891 2014-08-11 17:07:01.586538 2014-08-13 18:06:06.845577 7f5c973a57a0 20 read_log 192888'14839 (0'0) promote 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by osd.296.0:2594 2014-08-11 17:07:01.967636 2014-08-13 18:06:06.845589 7f5c973a57a0 20 read_log 192888'14840 (192888'14839) modify 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.942868.0:27992 2014-08-11 17:07:01.958056 2014-08-13 18:06:06.845602 7f5c973a57a0 20 read_log 192888'14841 (0'0) promote d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by osd.296.0:2596 2014-08-11 17:07:08.242621 2014-08-13 18:06:06.845618 7f5c973a57a0 20 read_log 192888'14842 (192888'14841) modify d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.942868.0:29606 2014-08-11 17:07:08.234868 2014-08-13 18:06:06.845637 7f5c973a57a0 20 read_log 192888'14843 (0'0) promote d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by osd.296.0:2598 2014-08-11 17:07:08.394110 2014-08-13 18:06:06.845654 7f5c973a57a0 20 read_log 192888'14844 (192888'14843) modify d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.942868.0:29645 2014-08-11 17:07:08.387361 2014-08-13 18:06:06.845666 7f5c973a57a0 20 read_log 192888'14845 (0'0) promote d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by osd.296.0:2600 2014-08-11 17:07:09.905259 2014-08-13 18:06:06.845679 7f5c973a57a0 20 read_log 192888'14846 (192888'14845) modify d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.942868.0:30024 2014-08-11 17:07:09.897719 2014-08-13 18:06:06.845691 7f5c973a57a0 20 read_log 192888'14849 (192888'14716) modify aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.942868.0:31316 2014-08-11 17:07:15.868564 2014-08-13 18:06:06.845704 7f5c973a57a0 20 read_log 192888'14852 (192888'14722) modify 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.942868.0:32856 2014-08-11 17:07:16.362010 2014-08-13 18:06:06.845716 7f5c973a57a0 20 read_log 192888'14853 (192888'14724) modify 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.942868.0:33325 2014-08-11 17:07:16.486404 2014-08-13 18:06:06.845729 7f5c973a57a0 20 read_log 192888'14854 (192888'14726) modify 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.942868.0:34518 2014-08-11 17:07:16.811412 2014-08-13 18:06:06.845741 7f5c973a57a0 20 read_log 192888'14856 (192888'14730) modify 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.942868.0:34720 2014-08-11 17:07:16.873952 2014-08-13 18:06:06.845780 7f5c973a57a0 20 read_log 192888'14858 (192888'14734) modify 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.942868.0:36126 2014-08-11 17:07:17.248739 2014-08-13 18:06:06.845796 7f5c973a57a0 20 read_log 192888'14863 (192888'14744) modify 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.942868.0:37603 2014-08-11 17:07:17.656265 2014-08-13 18:06:06.845808 7f5c973a57a0 20 read_log 192888'14864 (192888'14746) modify f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.942868.0:37642 2014-08-11 17:07:17.667265 2014-08-13 18:06:06.845821 7f5c973a57a0 20 read_log 192888'14866 (192888'14750) modify 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.942868.0:38884 2014-08-11 17:07:18.019146 2014-08-13 18:06:06.845833 7f5c973a57a0 20 read_log 192888'14867 (192888'14752) modify dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.942868.0:39046 2014-08-11 17:07:18.064668 2014-08-13 18:06:06.845845 7f5c973a57a0 20 read_log 192888'14868 (192888'14754) modify cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.942868.0:39267 2014-08-11 17:07:18.134463 2014-08-13 18:06:06.845858 7f5c973a57a0 20 read_log 192888'14870 (192888'14758) modify ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.942868.0:39920 2014-08-11 17:07:18.319867 2014-08-13 18:06:06.845870 7f5c973a57a0 20 read_log 192888'14872 (192888'14762) modify c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.942868.0:42091 2014-08-11 17:07:18.902102 2014-08-13 18:06:06.845882 7f5c973a57a0 20 read_log 192888'14874 (192888'14766) modify aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.942868.0:42965 2014-08-11 17:07:19.140046 2014-08-13 18:06:06.845895 7f5c973a57a0 20 read_log 192888'14876 (192888'14770) modify 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.942868.0:43760 2014-08-11 17:07:19.366300 2014-08-13 18:06:06.845907 7f5c973a57a0 20 read_log 192888'14878 (192888'14774) modify 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.942868.0:45334 2014-08-11 17:07:19.809199 2014-08-13 18:06:06.845919 7f5c973a57a0 20 read_log 192888'14879 (192888'14776) modify 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.942868.0:45418 2014-08-11 17:07:19.832702 2014-08-13 18:06:06.845931 7f5c973a57a0 20 read_log 192888'14882 (192888'14782) modify 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.942868.0:47069 2014-08-11 17:07:20.299038 2014-08-13 18:06:06.845943 7f5c973a57a0 20 read_log 192888'14883 (192888'14784) modify e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.942868.0:47420 2014-08-11 17:07:20.400153 2014-08-13 18:06:06.845955 7f5c973a57a0 20 read_log 192888'14885 (192888'14788) modify d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.942868.0:48177 2014-08-11 17:07:20.646214 2014-08-13 18:06:06.845969 7f5c973a57a0 20 read_log 192888'14889 (192888'14796) modify 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.942868.0:49287 2014-08-11 17:07:20.965957 2014-08-13 18:06:06.845984 7f5c973a57a0 20 read_log 192888'14890 (192888'14798) modify 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.942868.0:49349 2014-08-11 17:07:20.986954 2014-08-13 18:06:06.845997 7f5c973a57a0 20 read_log 192888'14894 (192888'14806) modify e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.942868.0:52481 2014-08-11 17:07:21.955242 2014-08-13 18:06:06.846011 7f5c973a57a0 20 read_log 192888'14895 (192888'14808) modify 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.942868.0:52551 2014-08-11 17:07:21.976572 2014-08-13 18:06:06.846024 7f5c973a57a0 20 read_log 192888'14896 (192888'14810) modify 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.942868.0:52795 2014-08-11 17:07:22.053566 2014-08-13 18:06:06.846045 7f5c973a57a0 20 read_log 192888'14897 (192888'14812) modify aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.942868.0:52841 2014-08-11 17:07:22.066314 2014-08-13 18:06:06.846060 7f5c973a57a0 20 read_log 192888'14898 (192888'14814) modify 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.942868.0:53931 2014-08-11 17:07:22.419265 2014-08-13 18:06:06.846072 7f5c973a57a0 20 read_log 192888'14900 (192888'14818) modify 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.942868.0:54702 2014-08-11 17:07:22.633185 2014-08-13 18:06:06.846087 7f5c973a57a0 20 read_log 192888'14901 (192888'14820) modify 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.942868.0:55145 2014-08-11 17:07:22.756321 2014-08-13 18:06:06.846099 7f5c973a57a0 20 read_log 192888'14903 (192888'14824) modify 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.942868.0:56113 2014-08-11 17:07:23.056590 2014-08-13 18:06:06.846114 7f5c973a57a0 20 read_log 192888'14904 (192888'14826) modify 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.942868.0:56388 2014-08-11 17:07:23.134804 2014-08-13 18:06:06.846126 7f5c973a57a0 20 read_log 192888'14906 (192888'14830) modify 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.942868.0:57274 2014-08-11 17:07:23.409198 2014-08-13 18:06:06.846140 7f5c973a57a0 20 read_log 192888'14908 (192888'14834) modify 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.942868.0:58172 2014-08-11 17:07:23.663235 2014-08-13 18:06:06.846163 7f5c973a57a0 20 read_log 192888'14909 (192888'14836) modify 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.942868.0:59157 2014-08-11 17:07:23.961610 2014-08-13 18:06:06.846176 7f5c973a57a0 20 read_log 192888'14911 (192888'14840) modify 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.942868.0:59258 2014-08-11 17:07:23.992276 2014-08-13 18:06:06.846188 7f5c973a57a0 20 read_log 192888'14912 (192888'14842) modify d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.942868.0:60872 2014-08-11 17:07:24.471393 2014-08-13 18:06:06.846222 7f5c973a57a0 20 read_log 192888'14913 (192888'14844) modify d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.942868.0:60911 2014-08-11 17:07:24.481538 2014-08-13 18:06:06.846234 7f5c973a57a0 20 read_log 192888'14914 (192888'14846) modify d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.942868.0:61290 2014-08-11 17:07:24.596675 2014-08-13 18:06:06.846249 7f5c973a57a0 20 read_log 193305'14916 (0'0) promote 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by osd.396.0:4 2014-08-11 18:42:34.185500 2014-08-13 18:06:06.846263 7f5c973a57a0 20 read_log 193305'14917 (193305'14916) modify 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.951337.0:321 2014-08-11 18:42:34.177622 2014-08-13 18:06:06.846278 7f5c973a57a0 20 read_log 193305'14918 (0'0) promote 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by osd.396.0:15 2014-08-11 18:42:40.889235 2014-08-13 18:06:06.846291 7f5c973a57a0 20 read_log 193305'14919 (193305'14918) modify 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.951337.0:2035 2014-08-11 18:42:40.880420 2014-08-13 18:06:06.846304 7f5c973a57a0 20 read_log 193305'14920 (0'0) promote 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by osd.396.0:23 2014-08-11 18:42:46.919282 2014-08-13 18:06:06.846322 7f5c973a57a0 20 read_log 193305'14921 (193305'14920) modify 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.951337.0:3697 2014-08-11 18:42:46.910275 2014-08-13 18:06:06.846334 7f5c973a57a0 20 read_log 193305'14922 (0'0) promote 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by osd.396.0:31 2014-08-11 18:42:52.450498 2014-08-13 18:06:06.846359 7f5c973a57a0 20 read_log 193305'14923 (193305'14922) modify 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.951337.0:5180 2014-08-11 18:42:52.442427 2014-08-13 18:06:06.846375 7f5c973a57a0 20 read_log 193305'14924 (0'0) promote 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by osd.396.0:33 2014-08-11 18:42:52.911166 2014-08-13 18:06:06.846388 7f5c973a57a0 20 read_log 193305'14925 (193305'14924) modify 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.951337.0:5307 2014-08-11 18:42:52.903810 2014-08-13 18:06:06.846400 7f5c973a57a0 20 read_log 193305'14926 (0'0) promote e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by osd.396.0:35 2014-08-11 18:42:55.381751 2014-08-13 18:06:06.846416 7f5c973a57a0 20 read_log 193305'14927 (193305'14926) modify e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.951337.0:5988 2014-08-11 18:42:55.375708 2014-08-13 18:06:06.846428 7f5c973a57a0 20 read_log 193305'14928 (0'0) promote b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by osd.396.0:39 2014-08-11 18:42:56.683179 2014-08-13 18:06:06.846450 7f5c973a57a0 20 read_log 193305'14929 (193305'14928) modify b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.951337.0:6346 2014-08-11 18:42:56.676584 2014-08-13 18:06:06.846469 7f5c973a57a0 20 read_log 193305'14930 (0'0) promote 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by osd.396.0:57 2014-08-11 18:43:07.393194 2014-08-13 18:06:06.846482 7f5c973a57a0 20 read_log 193305'14931 (193305'14930) modify 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.951337.0:9210 2014-08-11 18:43:07.386043 2014-08-13 18:06:06.846505 7f5c973a57a0 20 read_log 193305'14932 (0'0) promote 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by osd.396.0:73 2014-08-11 18:43:18.088415 2014-08-13 18:06:06.846542 7f5c973a57a0 20 read_log 193305'14933 (193305'14932) modify 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.951337.0:12054 2014-08-11 18:43:18.080851 2014-08-13 18:06:06.846560 7f5c973a57a0 20 read_log 193305'14934 (0'0) promote 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by osd.396.0:77 2014-08-11 18:43:20.182722 2014-08-13 18:06:06.846572 7f5c973a57a0 20 read_log 193305'14935 (193305'14934) modify 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.951337.0:12622 2014-08-11 18:43:20.171564 2014-08-13 18:06:06.846587 7f5c973a57a0 20 read_log 193305'14936 (0'0) promote e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by osd.396.0:93 2014-08-11 18:43:28.947327 2014-08-13 18:06:06.846599 7f5c973a57a0 20 read_log 193305'14937 (193305'14936) modify e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.951337.0:14989 2014-08-11 18:43:28.939175 2014-08-13 18:06:06.846614 7f5c973a57a0 20 read_log 193305'14938 (0'0) promote e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by osd.396.0:95 2014-08-11 18:43:29.215693 2014-08-13 18:06:06.846627 7f5c973a57a0 20 read_log 193305'14939 (193305'14938) modify e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.951337.0:15071 2014-08-11 18:43:29.206782 2014-08-13 18:06:06.846643 7f5c973a57a0 20 read_log 193305'14940 (0'0) promote 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by osd.396.0:97 2014-08-11 18:43:30.936131 2014-08-13 18:06:06.846658 7f5c973a57a0 20 read_log 193305'14941 (193305'14940) modify 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.951337.0:15584 2014-08-11 18:43:30.927084 2014-08-13 18:06:06.846671 7f5c973a57a0 20 read_log 193305'14942 (0'0) promote fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by osd.396.0:105 2014-08-11 18:43:32.913926 2014-08-13 18:06:06.846690 7f5c973a57a0 20 read_log 193305'14943 (193305'14942) modify fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.951337.0:16149 2014-08-11 18:43:32.906088 2014-08-13 18:06:06.846714 7f5c973a57a0 20 read_log 193305'14944 (0'0) promote c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by osd.396.0:111 2014-08-11 18:43:35.373431 2014-08-13 18:06:06.846727 7f5c973a57a0 20 read_log 193305'14945 (193305'14944) modify c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.951337.0:16852 2014-08-11 18:43:35.365248 2014-08-13 18:06:06.846739 7f5c973a57a0 20 read_log 193305'14946 (0'0) promote d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by osd.396.0:125 2014-08-11 18:43:40.113971 2014-08-13 18:06:06.846752 7f5c973a57a0 20 read_log 193305'14947 (193305'14946) modify d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.951337.0:18218 2014-08-11 18:43:40.105175 2014-08-13 18:06:06.846764 7f5c973a57a0 20 read_log 193305'14948 (0'0) promote 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by osd.396.0:129 2014-08-11 18:43:41.044716 2014-08-13 18:06:06.846776 7f5c973a57a0 20 read_log 193305'14949 (193305'14948) modify 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.951337.0:18475 2014-08-11 18:43:41.034915 2014-08-13 18:06:06.846789 7f5c973a57a0 20 read_log 193305'14950 (0'0) promote 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by osd.396.0:135 2014-08-11 18:43:45.027307 2014-08-13 18:06:06.846801 7f5c973a57a0 20 read_log 193305'14951 (193305'14950) modify 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.951337.0:19547 2014-08-11 18:43:45.018810 2014-08-13 18:06:06.846813 7f5c973a57a0 20 read_log 193305'14952 (0'0) promote d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by osd.396.0:137 2014-08-11 18:43:45.252035 2014-08-13 18:06:06.846825 7f5c973a57a0 20 read_log 193305'14953 (193305'14952) modify d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.951337.0:19612 2014-08-11 18:43:45.243841 2014-08-13 18:06:06.846838 7f5c973a57a0 20 read_log 193305'14954 (0'0) promote de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by osd.396.0:139 2014-08-11 18:43:49.321032 2014-08-13 18:06:06.846850 7f5c973a57a0 20 read_log 193305'14955 (193305'14954) modify de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.951337.0:20753 2014-08-11 18:43:49.313548 2014-08-13 18:06:06.846862 7f5c973a57a0 20 read_log 193305'14956 (0'0) promote 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by osd.396.0:141 2014-08-11 18:43:50.417737 2014-08-13 18:06:06.846874 7f5c973a57a0 20 read_log 193305'14957 (193305'14956) modify 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.951337.0:21054 2014-08-11 18:43:50.408740 2014-08-13 18:06:06.846886 7f5c973a57a0 20 read_log 193305'14958 (0'0) promote 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by osd.396.0:145 2014-08-11 18:43:51.359499 2014-08-13 18:06:06.846907 7f5c973a57a0 20 read_log 193305'14959 (193305'14958) modify 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.951337.0:21322 2014-08-11 18:43:51.350728 2014-08-13 18:06:06.846923 7f5c973a57a0 20 read_log 193305'14960 (193305'14960) modify 299/hit_set_24.299_archive_2014-08-11 18:38:51.580559_2014-08-11 18:43:55.287544/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:55.287671 2014-08-13 18:06:06.846936 7f5c973a57a0 20 read_log 193305'14961 (0'0) promote 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by osd.396.0:152 2014-08-11 18:43:55.290463 2014-08-13 18:06:06.846974 7f5c973a57a0 20 read_log 193305'14962 (193305'14961) modify 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.951337.0:22402 2014-08-11 18:43:55.279754 2014-08-13 18:06:06.846990 7f5c973a57a0 20 read_log 193305'14963 (0'0) promote 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by osd.396.0:154 2014-08-11 18:43:55.428034 2014-08-13 18:06:06.847006 7f5c973a57a0 20 read_log 193305'14964 (193305'14963) modify 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.951337.0:22435 2014-08-11 18:43:55.419081 2014-08-13 18:06:06.847018 7f5c973a57a0 20 read_log 193305'14965 (0'0) promote 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by osd.396.0:156 2014-08-11 18:43:56.579278 2014-08-13 18:06:06.847030 7f5c973a57a0 20 read_log 193305'14966 (193305'14965) modify 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.951337.0:22753 2014-08-11 18:43:56.570426 2014-08-13 18:06:06.847044 7f5c973a57a0 20 read_log 193305'14967 (0'0) promote a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by osd.396.0:158 2014-08-11 18:43:57.346635 2014-08-13 18:06:06.847057 7f5c973a57a0 20 read_log 193305'14968 (193305'14967) modify a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.951337.0:22972 2014-08-11 18:43:57.337063 2014-08-13 18:06:06.847069 7f5c973a57a0 20 read_log 193305'14969 (0'0) promote 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by osd.396.0:162 2014-08-11 18:43:58.310029 2014-08-13 18:06:06.847081 7f5c973a57a0 20 read_log 193305'14970 (193305'14969) modify 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.951337.0:23249 2014-08-11 18:43:58.299364 2014-08-13 18:06:06.847098 7f5c973a57a0 20 read_log 193305'14971 (0'0) promote b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by osd.396.0:164 2014-08-11 18:43:59.499929 2014-08-13 18:06:06.847113 7f5c973a57a0 20 read_log 193305'14972 (193305'14971) modify b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.951337.0:23575 2014-08-11 18:43:59.492172 2014-08-13 18:06:06.847126 7f5c973a57a0 20 read_log 193305'14973 (0'0) promote 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by osd.396.0:176 2014-08-11 18:44:08.828000 2014-08-13 18:06:06.847158 7f5c973a57a0 20 read_log 193305'14974 (193305'14973) modify 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.951337.0:26115 2014-08-11 18:44:08.819791 2014-08-13 18:06:06.847174 7f5c973a57a0 20 read_log 193305'14975 (0'0) promote 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by osd.396.0:180 2014-08-11 18:44:15.086166 2014-08-13 18:06:06.847198 7f5c973a57a0 20 read_log 193305'14976 (193305'14975) modify 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.951337.0:27840 2014-08-11 18:44:15.078785 2014-08-13 18:06:06.847211 7f5c973a57a0 20 read_log 193305'14977 (0'0) promote bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by osd.396.0:184 2014-08-11 18:44:18.307104 2014-08-13 18:06:06.847223 7f5c973a57a0 20 read_log 193305'14978 (193305'14977) modify bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.951337.0:28729 2014-08-11 18:44:18.298539 2014-08-13 18:06:06.847247 7f5c973a57a0 20 read_log 193305'14979 (0'0) promote b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by osd.396.0:186 2014-08-11 18:44:20.570028 2014-08-13 18:06:06.847260 7f5c973a57a0 20 read_log 193305'14980 (193305'14979) modify b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.951337.0:29336 2014-08-11 18:44:20.560423 2014-08-13 18:06:06.847279 7f5c973a57a0 20 read_log 193305'14981 (0'0) promote e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by osd.396.0:194 2014-08-11 18:44:28.996890 2014-08-13 18:06:06.847302 7f5c973a57a0 20 read_log 193305'14982 (193305'14981) modify e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.951337.0:31628 2014-08-11 18:44:28.989158 2014-08-13 18:06:06.847321 7f5c973a57a0 20 read_log 193305'14983 (193305'14917) modify 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.951337.0:33032 2014-08-11 18:44:34.080221 2014-08-13 18:06:06.847348 7f5c973a57a0 20 read_log 193305'14984 (193305'14919) modify 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.951337.0:34746 2014-08-11 18:44:35.166597 2014-08-13 18:06:06.847368 7f5c973a57a0 20 read_log 193305'14985 (193305'14921) modify 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.951337.0:36408 2014-08-11 18:44:36.179538 2014-08-13 18:06:06.847381 7f5c973a57a0 20 read_log 193305'14986 (193305'14923) modify 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.951337.0:37891 2014-08-11 18:44:37.103587 2014-08-13 18:06:06.847395 7f5c973a57a0 20 read_log 193305'14987 (193305'14925) modify 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.951337.0:38018 2014-08-11 18:44:37.200150 2014-08-13 18:06:06.847408 7f5c973a57a0 20 read_log 193305'14988 (193305'14927) modify e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.951337.0:38699 2014-08-11 18:44:37.604140 2014-08-13 18:06:06.847420 7f5c973a57a0 20 read_log 193305'14989 (193305'14929) modify b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.951337.0:39057 2014-08-11 18:44:37.803510 2014-08-13 18:06:06.847432 7f5c973a57a0 20 read_log 193305'14990 (193305'14931) modify 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.951337.0:41921 2014-08-11 18:44:38.925468 2014-08-13 18:06:06.847445 7f5c973a57a0 20 read_log 193305'14991 (193305'14933) modify 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.951337.0:44765 2014-08-11 18:44:39.983018 2014-08-13 18:06:06.847457 7f5c973a57a0 20 read_log 193305'14992 (193305'14935) modify 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.951337.0:45333 2014-08-11 18:44:40.165915 2014-08-13 18:06:06.847470 7f5c973a57a0 20 read_log 193305'14993 (193305'14937) modify e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.951337.0:47700 2014-08-11 18:44:40.841457 2014-08-13 18:06:06.847482 7f5c973a57a0 20 read_log 193305'14994 (193305'14939) modify e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.951337.0:47782 2014-08-11 18:44:40.864950 2014-08-13 18:06:06.847494 7f5c973a57a0 20 read_log 193305'14995 (193305'14941) modify 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.951337.0:48295 2014-08-11 18:44:41.011473 2014-08-13 18:06:06.847507 7f5c973a57a0 20 read_log 193305'14996 (193305'14943) modify fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.951337.0:48860 2014-08-11 18:44:41.177457 2014-08-13 18:06:06.847527 7f5c973a57a0 20 read_log 193305'14997 (193305'14945) modify c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.951337.0:49563 2014-08-11 18:44:41.404203 2014-08-13 18:06:06.847544 7f5c973a57a0 20 read_log 193305'14998 (193305'14947) modify d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.951337.0:50929 2014-08-11 18:44:41.803811 2014-08-13 18:06:06.847557 7f5c973a57a0 20 read_log 193305'14999 (193305'14949) modify 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.951337.0:51186 2014-08-11 18:44:41.879217 2014-08-13 18:06:06.847570 7f5c973a57a0 20 read_log 193305'15000 (193305'14951) modify 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.951337.0:52258 2014-08-11 18:44:42.177949 2014-08-13 18:06:06.847605 7f5c973a57a0 20 read_log 193305'15001 (193305'14953) modify d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.951337.0:52323 2014-08-11 18:44:42.202491 2014-08-13 18:06:06.847626 7f5c973a57a0 20 read_log 193305'15002 (193305'14955) modify de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.951337.0:53464 2014-08-11 18:44:42.539077 2014-08-13 18:06:06.847641 7f5c973a57a0 20 read_log 193305'15003 (193305'14957) modify 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.951337.0:53765 2014-08-11 18:44:42.626615 2014-08-13 18:06:06.847653 7f5c973a57a0 20 read_log 193305'15004 (193305'14959) modify 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.951337.0:54033 2014-08-11 18:44:42.707869 2014-08-13 18:06:06.847666 7f5c973a57a0 20 read_log 193305'15005 (193305'14962) modify 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.951337.0:55113 2014-08-11 18:44:43.028898 2014-08-13 18:06:06.847678 7f5c973a57a0 20 read_log 193305'15006 (193305'14964) modify 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.951337.0:55146 2014-08-11 18:44:43.038430 2014-08-13 18:06:06.847690 7f5c973a57a0 20 read_log 193305'15007 (193305'14966) modify 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.951337.0:55464 2014-08-11 18:44:43.131160 2014-08-13 18:06:06.847703 7f5c973a57a0 20 read_log 193305'15008 (193305'14968) modify a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.951337.0:55683 2014-08-11 18:44:43.200385 2014-08-13 18:06:06.847715 7f5c973a57a0 20 read_log 193305'15009 (193305'14970) modify 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.951337.0:55960 2014-08-11 18:44:43.279765 2014-08-13 18:06:06.847727 7f5c973a57a0 20 read_log 193305'15010 (193305'14972) modify b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.951337.0:56286 2014-08-11 18:44:43.374272 2014-08-13 18:06:06.847739 7f5c973a57a0 20 read_log 193305'15011 (193305'14974) modify 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.951337.0:58826 2014-08-11 18:44:44.152280 2014-08-13 18:06:06.847751 7f5c973a57a0 20 read_log 193305'15012 (193305'14976) modify 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.951337.0:60551 2014-08-11 18:44:44.662406 2014-08-13 18:06:06.847764 7f5c973a57a0 20 read_log 193305'15013 (193305'14978) modify bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.951337.0:61440 2014-08-11 18:44:44.921632 2014-08-13 18:06:06.847776 7f5c973a57a0 20 read_log 193305'15014 (193305'14980) modify b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.951337.0:62047 2014-08-11 18:44:45.110721 2014-08-13 18:06:06.847788 7f5c973a57a0 20 read_log 193305'15015 (193305'14982) modify e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.951337.0:64339 2014-08-11 18:44:45.772918 2014-08-13 18:06:06.847801 7f5c973a57a0 20 read_log 194215'15016 (194215'15016) modify 299/hit_set_24.299_archive_2014-08-12 15:32:25.825077_2014-08-12 15:46:22.535741/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:22.535888 2014-08-13 18:06:06.847813 7f5c973a57a0 20 read_log 194215'15017 (193305'14960) delete 299/hit_set_24.299_archive_2014-08-11 18:38:51.580559_2014-08-11 18:43:55.287544/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:22.535888 2014-08-13 18:06:06.847826 7f5c973a57a0 20 read_log 194215'15018 (192834'13948) clean 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by osd.396.0:529 2014-08-12 15:46:22.540873 2014-08-13 18:06:06.847838 7f5c973a57a0 20 read_log 194215'15019 (194215'15018) delete 56680299/benchmark_data_storage0111-ib.pocstor_26659_object6195/head//24 by client.2174595.0:136808 0.000000 2014-08-13 18:06:06.847850 7f5c973a57a0 20 read_log 194215'15020 (192834'14048) clean d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by osd.396.0:531 2014-08-12 15:46:22.546796 2014-08-13 18:06:06.847889 7f5c973a57a0 20 read_log 194215'15021 (194215'15020) delete d7801299/benchmark_data_storage0111-ib.pocstor_26659_object20603/head//24 by client.2174595.0:136810 0.000000 2014-08-13 18:06:06.847904 7f5c973a57a0 20 read_log 194215'15022 (192888'14685) clean 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by osd.396.0:533 2014-08-12 15:46:22.553182 2014-08-13 18:06:06.847916 7f5c973a57a0 20 read_log 194215'15023 (194215'15022) delete 3be31299/benchmark_data_storage0111-ib.pocstor_41097_object14488/head//24 by client.2174595.0:136812 0.000000 2014-08-13 18:06:06.847927 7f5c973a57a0 20 read_log 194215'15024 (192888'14690) clean 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by osd.396.0:535 2014-08-12 15:46:22.560242 2014-08-13 18:06:06.847941 7f5c973a57a0 20 read_log 194215'15025 (194215'15024) delete 7f041299/benchmark_data_storage0111-ib.pocstor_41097_object19013/head//24 by client.2174595.0:136814 0.000000 2014-08-13 18:06:06.847988 7f5c973a57a0 20 read_log 194215'15026 (193305'14993) clean e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by osd.396.0:537 2014-08-12 15:46:22.567610 2014-08-13 18:06:06.848005 7f5c973a57a0 20 read_log 194215'15027 (194215'15026) delete e9aa1299/benchmark_data_storage0111-ib.pocstor_62621_object14988/head//24 by client.2174595.0:136816 0.000000 2014-08-13 18:06:06.848016 7f5c973a57a0 20 read_log 194215'15028 (192834'13998) clean c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by osd.396.0:539 2014-08-12 15:46:22.574022 2014-08-13 18:06:06.848028 7f5c973a57a0 20 read_log 194215'15029 (194215'15028) delete c6412299/benchmark_data_storage0111-ib.pocstor_26659_object13013/head//24 by client.2174595.0:136818 0.000000 2014-08-13 18:06:06.848041 7f5c973a57a0 20 read_log 194215'15030 (192888'14668) clean 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by osd.396.0:541 2014-08-12 15:46:22.581071 2014-08-13 18:06:06.848053 7f5c973a57a0 20 read_log 194215'15031 (194215'15030) delete 994e4299/benchmark_data_storage0111-ib.pocstor_41097_object3669/head//24 by client.2174595.0:136820 0.000000 2014-08-13 18:06:06.848064 7f5c973a57a0 20 read_log 194215'15032 (192888'14882) clean 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by osd.396.0:543 2014-08-12 15:46:22.587475 2014-08-13 18:06:06.848076 7f5c973a57a0 20 read_log 194215'15033 (194215'15032) delete 22f4299/benchmark_data_storage0111-ib.pocstor_2228_object15802/head//24 by client.2174595.0:136822 0.000000 2014-08-13 18:06:06.848087 7f5c973a57a0 20 read_log 194215'15034 (192888'14874) clean aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by osd.396.0:545 2014-08-12 15:46:22.593445 2014-08-13 18:06:06.848102 7f5c973a57a0 20 read_log 194215'15035 (194215'15034) delete aeb75299/benchmark_data_storage0111-ib.pocstor_2228_object11698/head//24 by client.2174595.0:136824 0.000000 2014-08-13 18:06:06.848113 7f5c973a57a0 20 read_log 194215'15036 (192834'14053) clean 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by osd.396.0:547 2014-08-12 15:46:22.599411 2014-08-13 18:06:06.848128 7f5c973a57a0 20 read_log 194215'15037 (194215'15036) delete 4cba6299/benchmark_data_storage0111-ib.pocstor_26659_object21077/head//24 by client.2174595.0:136826 0.000000 2014-08-13 18:06:06.848139 7f5c973a57a0 20 read_log 194215'15038 (193305'14989) clean b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by osd.396.0:549 2014-08-12 15:46:22.604929 2014-08-13 18:06:06.848161 7f5c973a57a0 20 read_log 194215'15039 (194215'15038) delete b2c6299/benchmark_data_storage0111-ib.pocstor_62621_object6345/head//24 by client.2174595.0:136828 0.000000 2014-08-13 18:06:06.848172 7f5c973a57a0 20 read_log 194215'15040 (192888'14674) clean 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by osd.396.0:551 2014-08-12 15:46:22.610616 2014-08-13 18:06:06.848195 7f5c973a57a0 20 read_log 194215'15041 (194215'15040) delete 61cd6299/benchmark_data_storage0111-ib.pocstor_41097_object6868/head//24 by client.2174595.0:136830 0.000000 2014-08-13 18:06:06.848209 7f5c973a57a0 20 read_log 194215'15042 (192888'14897) clean aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by osd.396.0:553 2014-08-12 15:46:22.616129 2014-08-13 18:06:06.848222 7f5c973a57a0 20 read_log 194215'15043 (194215'15042) delete aa37299/benchmark_data_storage0111-ib.pocstor_2228_object21574/head//24 by client.2174595.0:136832 0.000000 2014-08-13 18:06:06.848233 7f5c973a57a0 20 read_log 194215'15044 (193305'15010) clean b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by osd.396.0:555 2014-08-12 15:46:22.621602 2014-08-13 18:06:06.848247 7f5c973a57a0 20 read_log 194215'15045 (194215'15044) delete b3228299/benchmark_data_storage0111-ib.pocstor_62621_object23574/head//24 by client.2174595.0:136834 0.000000 2014-08-13 18:06:06.848258 7f5c973a57a0 20 read_log 194215'15046 (192888'14868) clean cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by osd.396.0:557 2014-08-12 15:46:22.626887 2014-08-13 18:06:06.848271 7f5c973a57a0 20 read_log 194215'15047 (194215'15046) delete cd138299/benchmark_data_storage0111-ib.pocstor_2228_object8000/head//24 by client.2174595.0:136836 0.000000 2014-08-13 18:06:06.848282 7f5c973a57a0 20 read_log 194215'15048 (192888'14704) clean 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by osd.396.0:559 2014-08-12 15:46:22.633046 2014-08-13 18:06:06.848296 7f5c973a57a0 20 read_log 194215'15049 (194215'15048) delete 5f699299/benchmark_data_storage0111-ib.pocstor_41097_object27111/head//24 by client.2174595.0:136838 0.000000 2014-08-13 18:06:06.848310 7f5c973a57a0 20 read_log 194215'15050 (193305'14996) clean fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by osd.396.0:561 2014-08-12 15:46:22.638666 2014-08-13 18:06:06.848324 7f5c973a57a0 20 read_log 194215'15051 (194215'15050) delete fc9b9299/benchmark_data_storage0111-ib.pocstor_62621_object16148/head//24 by client.2174595.0:136840 0.000000 2014-08-13 18:06:06.848335 7f5c973a57a0 20 read_log 194215'15052 (193305'15011) clean 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by osd.396.0:563 2014-08-12 15:46:22.643616 2014-08-13 18:06:06.848348 7f5c973a57a0 20 read_log 194215'15053 (194215'15052) delete 6c60a299/benchmark_data_storage0111-ib.pocstor_62621_object26114/head//24 by client.2174595.0:136842 0.000000 2014-08-13 18:06:06.848362 7f5c973a57a0 20 read_log 194215'15054 (193305'14984) clean 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by osd.396.0:565 2014-08-12 15:46:22.648837 2014-08-13 18:06:06.848374 7f5c973a57a0 20 read_log 194215'15055 (194215'15054) delete 76f4a299/benchmark_data_storage0111-ib.pocstor_62621_object2034/head//24 by client.2174595.0:136844 0.000000 2014-08-13 18:06:06.848385 7f5c973a57a0 20 read_log 194215'15056 (192888'14854) clean 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by osd.396.0:567 2014-08-12 15:46:22.654264 2014-08-13 18:06:06.848397 7f5c973a57a0 20 read_log 194215'15057 (194215'15056) delete 8d0ba299/benchmark_data_storage0111-ib.pocstor_2228_object3251/head//24 by client.2174595.0:136846 0.000000 2014-08-13 18:06:06.848411 7f5c973a57a0 20 read_log 194215'15058 (192834'14522) clean 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by osd.396.0:569 2014-08-12 15:46:22.659083 2014-08-13 18:06:06.848423 7f5c973a57a0 20 read_log 194215'15059 (194215'15058) delete 8bf7b299/benchmark_data_storage0111-ib.pocstor_41526_object15517/head//24 by client.2174595.0:136848 0.000000 2014-08-13 18:06:06.848440 7f5c973a57a0 20 read_log 194215'15060 (192888'14679) clean d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by osd.396.0:571 2014-08-12 15:46:22.664235 2014-08-13 18:06:06.848464 7f5c973a57a0 20 read_log 194215'15061 (194215'15060) delete d0fcb299/benchmark_data_storage0111-ib.pocstor_41097_object10759/head//24 by client.2174595.0:136850 0.000000 2014-08-13 18:06:06.848478 7f5c973a57a0 20 read_log 194215'15062 (193305'15013) clean bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by osd.396.0:573 2014-08-12 15:46:22.669762 2014-08-13 18:06:06.848501 7f5c973a57a0 20 read_log 194215'15063 (194215'15062) delete bb5ac299/benchmark_data_storage0111-ib.pocstor_62621_object28728/head//24 by client.2174595.0:136852 0.000000 2014-08-13 18:06:06.848512 7f5c973a57a0 20 read_log 194215'15064 (192888'14689) clean 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by osd.396.0:575 2014-08-12 15:46:22.675693 2014-08-13 18:06:06.848525 7f5c973a57a0 20 read_log 194215'15065 (194215'15064) delete 8fedc299/benchmark_data_storage0111-ib.pocstor_41097_object18934/head//24 by client.2174595.0:136854 0.000000 2014-08-13 18:06:06.848544 7f5c973a57a0 20 read_log 194215'15066 (193305'14983) clean 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by osd.396.0:577 2014-08-12 15:46:22.682100 2014-08-13 18:06:06.848560 7f5c973a57a0 20 read_log 194215'15067 (194215'15066) delete 2a7bd299/benchmark_data_storage0111-ib.pocstor_62621_object320/head//24 by client.2174595.0:136856 0.000000 2014-08-13 18:06:06.848571 7f5c973a57a0 20 read_log 194215'15068 (192834'13979) clean c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by osd.396.0:579 2014-08-12 15:46:22.687954 2014-08-13 18:06:06.848591 7f5c973a57a0 20 read_log 194215'15069 (194215'15068) delete c6d0e299/benchmark_data_storage0111-ib.pocstor_26659_object10564/head//24 by client.2174595.0:136858 0.000000 2014-08-13 18:06:06.848605 7f5c973a57a0 20 read_log 194215'15070 (193305'14995) clean 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by osd.396.0:581 2014-08-12 15:46:22.693529 2014-08-13 18:06:06.848651 7f5c973a57a0 20 read_log 194215'15071 (194215'15070) delete 9a6ce299/benchmark_data_storage0111-ib.pocstor_62621_object15583/head//24 by client.2174595.0:136860 0.000000 2014-08-13 18:06:06.848663 7f5c973a57a0 20 read_log 194215'15072 (192888'14885) clean d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by osd.396.0:583 2014-08-12 15:46:22.699228 2014-08-13 18:06:06.848680 7f5c973a57a0 20 read_log 194215'15073 (194215'15072) delete d403f299/benchmark_data_storage0111-ib.pocstor_2228_object16910/head//24 by client.2174595.0:136862 0.000000 2014-08-13 18:06:06.848691 7f5c973a57a0 20 read_log 194215'15074 (192888'14684) clean 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by osd.396.0:585 2014-08-12 15:46:22.704955 2014-08-13 18:06:06.848703 7f5c973a57a0 20 read_log 194215'15075 (194215'15074) delete 7a83f299/benchmark_data_storage0111-ib.pocstor_41097_object14300/head//24 by client.2174595.0:136864 0.000000 2014-08-13 18:06:06.848723 7f5c973a57a0 20 read_log 194215'15076 (192834'13902) clean f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by osd.396.0:587 2014-08-12 15:46:22.711108 2014-08-13 18:06:06.848739 7f5c973a57a0 20 read_log 194215'15077 (194215'15076) delete f3d4f299/benchmark_data_storage0111-ib.pocstor_26659_object1738/head//24 by client.2174595.0:136866 0.000000 2014-08-13 18:06:06.848751 7f5c973a57a0 20 read_log 194215'15078 (192834'14545) clean 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by osd.396.0:589 2014-08-12 15:46:22.716722 2014-08-13 18:06:06.848763 7f5c973a57a0 20 read_log 194215'15079 (194215'15078) delete 2157f299/benchmark_data_storage0111-ib.pocstor_41526_object19037/head//24 by client.2174595.0:136868 0.000000 2014-08-13 18:06:06.848774 7f5c973a57a0 20 read_log 194215'15080 (192834'14020) clean 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by osd.396.0:591 2014-08-12 15:46:22.721929 2014-08-13 18:06:06.848794 7f5c973a57a0 20 read_log 194215'15081 (194215'15080) delete 774d0699/benchmark_data_storage0111-ib.pocstor_26659_object17095/head//24 by client.2174595.0:136870 0.000000 2014-08-13 18:06:06.848808 7f5c973a57a0 20 read_log 194215'15082 (192888'14694) clean a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by osd.396.0:593 2014-08-12 15:46:22.727303 2014-08-13 18:06:06.848824 7f5c973a57a0 20 read_log 194215'15083 (194215'15082) delete a42f0699/benchmark_data_storage0111-ib.pocstor_41097_object22058/head//24 by client.2174595.0:136872 0.000000 2014-08-13 18:06:06.848835 7f5c973a57a0 20 read_log 194215'15084 (192834'14016) clean 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by osd.396.0:595 2014-08-12 15:46:22.732694 2014-08-13 18:06:06.848847 7f5c973a57a0 20 read_log 194215'15085 (194215'15084) delete 2acf0699/benchmark_data_storage0111-ib.pocstor_26659_object16488/head//24 by client.2174595.0:136874 0.000000 2014-08-13 18:06:06.848858 7f5c973a57a0 20 read_log 194215'15086 (192888'14858) clean 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by osd.396.0:597 2014-08-12 15:46:22.738595 2014-08-13 18:06:06.848871 7f5c973a57a0 20 read_log 194215'15087 (194215'15086) delete 73e31699/benchmark_data_storage0111-ib.pocstor_2228_object4859/head//24 by client.2174595.0:136876 0.000000 2014-08-13 18:06:06.848881 7f5c973a57a0 20 read_log 194215'15088 (192888'14673) clean 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by osd.396.0:599 2014-08-12 15:46:22.744188 2014-08-13 18:06:06.848894 7f5c973a57a0 20 read_log 194215'15089 (194215'15088) delete 1ba81699/benchmark_data_storage0111-ib.pocstor_41097_object5717/head//24 by client.2174595.0:136878 0.000000 2014-08-13 18:06:06.848905 7f5c973a57a0 20 read_log 194215'15090 (192888'14890) clean 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by osd.396.0:601 2014-08-12 15:46:22.750069 2014-08-13 18:06:06.848928 7f5c973a57a0 20 read_log 194215'15091 (194215'15090) delete 36d33699/benchmark_data_storage0111-ib.pocstor_2228_object18082/head//24 by client.2174595.0:136880 0.000000 2014-08-13 18:06:06.848940 7f5c973a57a0 20 read_log 194215'15092 (193305'15006) clean 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by osd.396.0:603 2014-08-12 15:46:22.756028 2014-08-13 18:06:06.848953 7f5c973a57a0 20 read_log 194215'15093 (194215'15092) delete 4c983699/benchmark_data_storage0111-ib.pocstor_62621_object22434/head//24 by client.2174595.0:136882 0.000000 2014-08-13 18:06:06.848964 7f5c973a57a0 20 read_log 194215'15094 (192888'14912) clean d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by osd.396.0:605 2014-08-12 15:46:22.761985 2014-08-13 18:06:06.848977 7f5c973a57a0 20 read_log 194215'15095 (194215'15094) delete d4fd3699/benchmark_data_storage0111-ib.pocstor_2228_object29605/head//24 by client.2174595.0:136884 0.000000 2014-08-13 18:06:06.848988 7f5c973a57a0 20 read_log 194215'15096 (193305'14990) clean 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by osd.396.0:607 2014-08-12 15:46:22.767957 2014-08-13 18:06:06.849001 7f5c973a57a0 20 read_log 194215'15097 (194215'15096) delete 11c84699/benchmark_data_storage0111-ib.pocstor_62621_object9209/head//24 by client.2174595.0:136886 0.000000 2014-08-13 18:06:06.849012 7f5c973a57a0 20 read_log 194215'15098 (193305'14986) clean 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by osd.396.0:609 2014-08-12 15:46:22.773441 2014-08-13 18:06:06.849025 7f5c973a57a0 20 read_log 194215'15099 (194215'15098) delete 304a4699/benchmark_data_storage0111-ib.pocstor_62621_object5179/head//24 by client.2174595.0:136888 0.000000 2014-08-13 18:06:06.849037 7f5c973a57a0 20 read_log 194215'15100 (192888'14908) clean 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by osd.396.0:611 2014-08-12 15:46:22.779199 2014-08-13 18:06:06.849060 7f5c973a57a0 20 read_log 194215'15101 (194215'15100) delete 472c4699/benchmark_data_storage0111-ib.pocstor_2228_object26905/head//24 by client.2174595.0:136890 0.000000 2014-08-13 18:06:06.849071 7f5c973a57a0 20 read_log 194215'15102 (193305'14987) clean 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by osd.396.0:613 2014-08-12 15:46:22.784800 2014-08-13 18:06:06.849084 7f5c973a57a0 20 read_log 194215'15103 (194215'15102) delete 3dc25699/benchmark_data_storage0111-ib.pocstor_62621_object5306/head//24 by client.2174595.0:136892 0.000000 2014-08-13 18:06:06.849098 7f5c973a57a0 20 read_log 194215'15104 (192834'14432) clean 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by osd.396.0:615 2014-08-12 15:46:22.790648 2014-08-13 18:06:06.849110 7f5c973a57a0 20 read_log 194215'15105 (194215'15104) delete 45b55699/benchmark_data_storage0111-ib.pocstor_41526_object4791/head//24 by client.2174595.0:136894 0.000000 2014-08-13 18:06:06.849123 7f5c973a57a0 20 read_log 194215'15106 (193305'15007) clean 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by osd.396.0:617 2014-08-12 15:46:22.796592 2014-08-13 18:06:06.849136 7f5c973a57a0 20 read_log 194215'15107 (194215'15106) delete 31595699/benchmark_data_storage0111-ib.pocstor_62621_object22752/head//24 by client.2174595.0:136896 0.000000 2014-08-13 18:06:06.849154 7f5c973a57a0 20 read_log 194215'15108 (192888'14900) clean 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by osd.396.0:619 2014-08-12 15:46:22.802239 2014-08-13 18:06:06.849169 7f5c973a57a0 20 read_log 194215'15109 (194215'15108) delete 3fbc5699/benchmark_data_storage0111-ib.pocstor_2228_object23435/head//24 by client.2174595.0:136898 0.000000 2014-08-13 18:06:06.849180 7f5c973a57a0 20 read_log 194215'15110 (192834'13965) clean 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by osd.396.0:621 2014-08-12 15:46:22.808743 2014-08-13 18:06:06.849193 7f5c973a57a0 20 read_log 194215'15111 (194215'15110) delete 82ef5699/benchmark_data_storage0111-ib.pocstor_26659_object9051/head//24 by client.2174595.0:136900 0.000000 2014-08-13 18:06:06.849203 7f5c973a57a0 20 read_log 194215'15112 (192834'14416) clean bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by osd.396.0:623 2014-08-12 15:46:22.814521 2014-08-13 18:06:06.849216 7f5c973a57a0 20 read_log 194215'15113 (194215'15112) delete bed56699/benchmark_data_storage0111-ib.pocstor_41526_object1966/head//24 by client.2174595.0:136902 0.000000 2014-08-13 18:06:06.849226 7f5c973a57a0 20 read_log 194215'15114 (192834'14546) clean 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by osd.396.0:625 2014-08-12 15:46:22.820612 2014-08-13 18:06:06.849241 7f5c973a57a0 20 read_log 194215'15115 (194215'15114) delete 30a07699/benchmark_data_storage0111-ib.pocstor_41526_object19328/head//24 by client.2174595.0:136904 0.000000 2014-08-13 18:06:06.849252 7f5c973a57a0 20 read_log 194215'15116 (192834'14055) clean 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by osd.396.0:627 2014-08-12 15:46:22.826383 2014-08-13 18:06:06.849264 7f5c973a57a0 20 read_log 194215'15117 (194215'15116) delete 830b7699/benchmark_data_storage0111-ib.pocstor_26659_object21165/head//24 by client.2174595.0:136906 0.000000 2014-08-13 18:06:06.849277 7f5c973a57a0 20 read_log 194215'15118 (192888'14849) clean aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by osd.396.0:629 2014-08-12 15:46:22.832451 2014-08-13 18:06:06.849316 7f5c973a57a0 20 read_log 194215'15119 (194215'15118) delete aef7699/benchmark_data_storage0111-ib.pocstor_2228_object49/head//24 by client.2174595.0:136908 0.000000 2014-08-13 18:06:06.849330 7f5c973a57a0 20 read_log 194215'15120 (192888'14911) clean 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by osd.396.0:631 2014-08-12 15:46:22.838801 2014-08-13 18:06:06.849353 7f5c973a57a0 20 read_log 194215'15121 (194215'15120) delete 43788699/benchmark_data_storage0111-ib.pocstor_2228_object27991/head//24 by client.2174595.0:136910 0.000000 2014-08-13 18:06:06.849368 7f5c973a57a0 20 read_log 194215'15122 (192834'13921) clean e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by osd.396.0:633 2014-08-12 15:46:22.844230 2014-08-13 18:06:06.849380 7f5c973a57a0 20 read_log 194215'15123 (194215'15122) delete e96c8699/benchmark_data_storage0111-ib.pocstor_26659_object3376/head//24 by client.2174595.0:136912 0.000000 2014-08-13 18:06:06.849391 7f5c973a57a0 20 read_log 194215'15124 (192888'14863) clean 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by osd.396.0:635 2014-08-12 15:46:22.849449 2014-08-13 18:06:06.849407 7f5c973a57a0 20 read_log 194215'15125 (194215'15124) delete 71ef8699/benchmark_data_storage0111-ib.pocstor_2228_object6336/head//24 by client.2174595.0:136914 0.000000 2014-08-13 18:06:06.849419 7f5c973a57a0 20 read_log 194215'15126 (193305'14994) clean e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by osd.396.0:637 2014-08-12 15:46:22.854783 2014-08-13 18:06:06.849431 7f5c973a57a0 20 read_log 194215'15127 (194215'15126) delete e2809699/benchmark_data_storage0111-ib.pocstor_62621_object15070/head//24 by client.2174595.0:136916 0.000000 2014-08-13 18:06:06.849457 7f5c973a57a0 20 read_log 194215'15128 (193305'14997) clean c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by osd.396.0:639 2014-08-12 15:46:22.860442 2014-08-13 18:06:06.849473 7f5c973a57a0 20 read_log 194215'15129 (194215'15128) delete c4189699/benchmark_data_storage0111-ib.pocstor_62621_object16851/head//24 by client.2174595.0:136918 0.000000 2014-08-13 18:06:06.849484 7f5c973a57a0 20 read_log 194215'15130 (192834'14507) clean 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by osd.396.0:641 2014-08-12 15:46:22.866012 2014-08-13 18:06:06.849510 7f5c973a57a0 20 read_log 194215'15131 (194215'15130) delete 11bc9699/benchmark_data_storage0111-ib.pocstor_41526_object13943/head//24 by client.2174595.0:136920 0.000000 2014-08-13 18:06:06.849521 7f5c973a57a0 20 read_log 194215'15132 (192834'14454) clean a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by osd.396.0:643 2014-08-12 15:46:22.870916 2014-08-13 18:06:06.849534 7f5c973a57a0 20 read_log 194215'15133 (194215'15132) delete a2a3a699/benchmark_data_storage0111-ib.pocstor_41526_object7794/head//24 by client.2174595.0:136922 0.000000 2014-08-13 18:06:06.849544 7f5c973a57a0 20 read_log 194215'15134 (192888'14712) clean 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by osd.396.0:645 2014-08-12 15:46:22.875849 2014-08-13 18:06:06.849557 7f5c973a57a0 20 read_log 194215'15135 (194215'15134) delete 6107b699/benchmark_data_storage0111-ib.pocstor_41097_object30592/head//24 by client.2174595.0:136924 0.000000 2014-08-13 18:06:06.849577 7f5c973a57a0 20 read_log 194215'15136 (192834'13964) clean d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by osd.396.0:647 2014-08-12 15:46:22.880554 2014-08-13 18:06:06.849590 7f5c973a57a0 20 read_log 194215'15137 (194215'15136) delete d152c699/benchmark_data_storage0111-ib.pocstor_26659_object8289/head//24 by client.2174595.0:136926 0.000000 2014-08-13 18:06:06.849601 7f5c973a57a0 20 read_log 194215'15138 (192888'14906) clean 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by osd.396.0:649 2014-08-12 15:46:22.886022 2014-08-13 18:06:06.849614 7f5c973a57a0 20 read_log 194215'15139 (194215'15138) delete 5946c699/benchmark_data_storage0111-ib.pocstor_2228_object26007/head//24 by client.2174595.0:136928 0.000000 2014-08-13 18:06:06.849625 7f5c973a57a0 20 read_log 194215'15140 (192834'14421) clean 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by osd.396.0:651 2014-08-12 15:46:22.891379 2014-08-13 18:06:06.849657 7f5c973a57a0 20 read_log 194215'15141 (194215'15140) delete 53f6c699/benchmark_data_storage0111-ib.pocstor_41526_object2455/head//24 by client.2174595.0:136930 0.000000 2014-08-13 18:06:06.849672 7f5c973a57a0 20 read_log 194215'15142 (192834'14521) clean ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by osd.396.0:653 2014-08-12 15:46:22.897006 2014-08-13 18:06:06.849685 7f5c973a57a0 20 read_log 194215'15143 (194215'15142) delete ae7fc699/benchmark_data_storage0111-ib.pocstor_41526_object15405/head//24 by client.2174595.0:136932 0.000000 2014-08-13 18:06:06.849696 7f5c973a57a0 20 read_log 194215'15144 (192834'14493) clean cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by osd.396.0:655 2014-08-12 15:46:22.902608 2014-08-13 18:06:06.849731 7f5c973a57a0 20 read_log 194215'15145 (194215'15144) delete cc64d699/benchmark_data_storage0111-ib.pocstor_41526_object12524/head//24 by client.2174595.0:136934 0.000000 2014-08-13 18:06:06.849755 7f5c973a57a0 20 read_log 194215'15146 (192888'14901) clean 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by osd.396.0:657 2014-08-12 15:46:22.908070 2014-08-13 18:06:06.849771 7f5c973a57a0 20 read_log 194215'15147 (194215'15146) delete 31ed699/benchmark_data_storage0111-ib.pocstor_2228_object23878/head//24 by client.2174595.0:136936 0.000000 2014-08-13 18:06:06.849782 7f5c973a57a0 20 read_log 194215'15148 (192888'14914) clean d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by osd.396.0:659 2014-08-12 15:46:22.913677 2014-08-13 18:06:06.849795 7f5c973a57a0 20 read_log 194215'15149 (194215'15148) delete d47ed699/benchmark_data_storage0111-ib.pocstor_2228_object30023/head//24 by client.2174595.0:136938 0.000000 2014-08-13 18:06:06.849806 7f5c973a57a0 20 read_log 194215'15150 (192888'14706) clean 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by osd.396.0:661 2014-08-12 15:46:22.918972 2014-08-13 18:06:06.849818 7f5c973a57a0 20 read_log 194215'15151 (194215'15150) delete 6d62e699/benchmark_data_storage0111-ib.pocstor_41097_object27701/head//24 by client.2174595.0:136940 0.000000 2014-08-13 18:06:06.849829 7f5c973a57a0 20 read_log 194215'15152 (192834'14541) clean 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by osd.396.0:663 2014-08-12 15:46:22.924525 2014-08-13 18:06:06.849841 7f5c973a57a0 20 read_log 194215'15153 (194215'15152) delete 3258e699/benchmark_data_storage0111-ib.pocstor_41526_object18692/head//24 by client.2174595.0:136942 0.000000 2014-08-13 18:06:06.849852 7f5c973a57a0 20 read_log 194215'15154 (192888'14701) clean 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by osd.396.0:665 2014-08-12 15:46:22.930799 2014-08-13 18:06:06.849864 7f5c973a57a0 20 read_log 194215'15155 (194215'15154) delete 3d88e699/benchmark_data_storage0111-ib.pocstor_41097_object24034/head//24 by client.2174595.0:136944 0.000000 2014-08-13 18:06:06.849875 7f5c973a57a0 20 read_log 194215'15156 (192834'13972) clean 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by osd.396.0:667 2014-08-12 15:46:22.937451 2014-08-13 18:06:06.849887 7f5c973a57a0 20 read_log 194215'15157 (194215'15156) delete 68d8e699/benchmark_data_storage0111-ib.pocstor_26659_object9959/head//24 by client.2174595.0:136946 0.000000 2014-08-13 18:06:06.849898 7f5c973a57a0 20 read_log 194215'15158 (192888'14672) clean e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by osd.396.0:669 2014-08-12 15:46:22.945406 2014-08-13 18:06:06.849910 7f5c973a57a0 20 read_log 194215'15159 (194215'15158) delete e4a20a99/benchmark_data_storage0111-ib.pocstor_41097_object5418/head//24 by client.2174595.0:136948 0.000000 2014-08-13 18:06:06.849921 7f5c973a57a0 20 read_log 194215'15160 (192834'14449) clean d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by osd.396.0:671 2014-08-12 15:46:22.951819 2014-08-13 18:06:06.849957 7f5c973a57a0 20 read_log 194215'15161 (194215'15160) delete d7f80a99/benchmark_data_storage0111-ib.pocstor_41526_object7071/head//24 by client.2174595.0:136950 0.000000 2014-08-13 18:06:06.849972 7f5c973a57a0 20 read_log 194215'15162 (192834'14496) clean 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by osd.396.0:673 2014-08-12 15:46:22.957587 2014-08-13 18:06:06.849985 7f5c973a57a0 20 read_log 194215'15163 (194215'15162) delete 97f0a99/benchmark_data_storage0111-ib.pocstor_41526_object12817/head//24 by client.2174595.0:136952 0.000000 2014-08-13 18:06:06.849996 7f5c973a57a0 20 read_log 194215'15164 (192888'14876) clean 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by osd.396.0:675 2014-08-12 15:46:22.963596 2014-08-13 18:06:06.850008 7f5c973a57a0 20 read_log 194215'15165 (194215'15164) delete 5dd01a99/benchmark_data_storage0111-ib.pocstor_2228_object12493/head//24 by client.2174595.0:136954 0.000000 2014-08-13 18:06:06.850019 7f5c973a57a0 20 read_log 194215'15166 (192888'14856) clean 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by osd.396.0:677 2014-08-12 15:46:22.969334 2014-08-13 18:06:06.850031 7f5c973a57a0 20 read_log 194215'15167 (194215'15166) delete 50161a99/benchmark_data_storage0111-ib.pocstor_2228_object3453/head//24 by client.2174595.0:136956 0.000000 2014-08-13 18:06:06.850042 7f5c973a57a0 20 read_log 194215'15168 (192834'13898) clean fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by osd.396.0:679 2014-08-12 15:46:22.974878 2014-08-13 18:06:06.850054 7f5c973a57a0 20 read_log 194215'15169 (194215'15168) delete fd971a99/benchmark_data_storage0111-ib.pocstor_26659_object1226/head//24 by client.2174595.0:136958 0.000000 2014-08-13 18:06:06.850065 7f5c973a57a0 20 read_log 194215'15170 (192888'14896) clean 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by osd.396.0:681 2014-08-12 15:46:22.980752 2014-08-13 18:06:06.850077 7f5c973a57a0 20 read_log 194215'15171 (194215'15170) delete 6b8d1a99/benchmark_data_storage0111-ib.pocstor_2228_object21528/head//24 by client.2174595.0:136960 0.000000 2014-08-13 18:06:06.850088 7f5c973a57a0 20 read_log 194215'15172 (192834'14467) clean 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by osd.396.0:683 2014-08-12 15:46:22.987141 2014-08-13 18:06:06.850100 7f5c973a57a0 20 read_log 194215'15173 (194215'15172) delete 42ef1a99/benchmark_data_storage0111-ib.pocstor_41526_object9497/head//24 by client.2174595.0:136962 0.000000 2014-08-13 18:06:06.850111 7f5c973a57a0 20 read_log 194215'15174 (193305'15012) clean 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by osd.396.0:685 2014-08-12 15:46:22.993084 2014-08-13 18:06:06.850123 7f5c973a57a0 20 read_log 194215'15175 (194215'15174) delete 6db02a99/benchmark_data_storage0111-ib.pocstor_62621_object27839/head//24 by client.2174595.0:136964 0.000000 2014-08-13 18:06:06.850134 7f5c973a57a0 20 read_log 194215'15176 (192888'14864) clean f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by osd.396.0:687 2014-08-12 15:46:22.999297 2014-08-13 18:06:06.850159 7f5c973a57a0 20 read_log 194215'15177 (194215'15176) delete f5f12a99/benchmark_data_storage0111-ib.pocstor_2228_object6375/head//24 by client.2174595.0:136966 0.000000 2014-08-13 18:06:06.850172 7f5c973a57a0 20 read_log 194215'15178 (192888'14866) clean 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by osd.396.0:689 2014-08-12 15:46:23.004785 2014-08-13 18:06:06.850187 7f5c973a57a0 20 read_log 194215'15179 (194215'15178) delete 57273a99/benchmark_data_storage0111-ib.pocstor_2228_object7617/head//24 by client.2174595.0:136968 0.000000 2014-08-13 18:06:06.850199 7f5c973a57a0 20 read_log 194215'15180 (192888'14709) clean 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by osd.396.0:691 2014-08-12 15:46:23.010335 2014-08-13 18:06:06.850220 7f5c973a57a0 20 read_log 194215'15181 (194215'15180) delete 730d3a99/benchmark_data_storage0111-ib.pocstor_41097_object29411/head//24 by client.2174595.0:136970 0.000000 2014-08-13 18:06:06.850234 7f5c973a57a0 20 read_log 194215'15182 (192888'14904) clean 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by osd.396.0:693 2014-08-12 15:46:23.016249 2014-08-13 18:06:06.850247 7f5c973a57a0 20 read_log 194215'15183 (194215'15182) delete 33d64a99/benchmark_data_storage0111-ib.pocstor_2228_object25121/head//24 by client.2174595.0:136972 0.000000 2014-08-13 18:06:06.850258 7f5c973a57a0 20 read_log 194215'15184 (193305'15002) clean de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by osd.396.0:695 2014-08-12 15:46:23.021906 2014-08-13 18:06:06.850273 7f5c973a57a0 20 read_log 194215'15185 (194215'15184) delete de994a99/benchmark_data_storage0111-ib.pocstor_62621_object20752/head//24 by client.2174595.0:136974 0.000000 2014-08-13 18:06:06.850283 7f5c973a57a0 20 read_log 194215'15186 (192834'14022) clean 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by osd.396.0:697 2014-08-12 15:46:23.027691 2014-08-13 18:06:06.850296 7f5c973a57a0 20 read_log 194215'15187 (194215'15186) delete 7fbe4a99/benchmark_data_storage0111-ib.pocstor_26659_object17462/head//24 by client.2174595.0:136976 0.000000 2014-08-13 18:06:06.850310 7f5c973a57a0 20 read_log 194215'15188 (192888'14879) clean 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by osd.396.0:699 2014-08-12 15:46:23.033225 2014-08-13 18:06:06.850323 7f5c973a57a0 20 read_log 194215'15189 (194215'15188) delete 7ccb5a99/benchmark_data_storage0111-ib.pocstor_2228_object14151/head//24 by client.2174595.0:136978 0.000000 2014-08-13 18:06:06.850334 7f5c973a57a0 20 read_log 194215'15190 (193305'15009) clean 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by osd.396.0:701 2014-08-12 15:46:23.038927 2014-08-13 18:06:06.850346 7f5c973a57a0 20 read_log 194215'15191 (194215'15190) delete 43406a99/benchmark_data_storage0111-ib.pocstor_62621_object23248/head//24 by client.2174595.0:136980 0.000000 2014-08-13 18:06:06.850360 7f5c973a57a0 20 read_log 194215'15192 (192888'14670) clean 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by osd.396.0:703 2014-08-12 15:46:23.044625 2014-08-13 18:06:06.850373 7f5c973a57a0 20 read_log 194215'15193 (194215'15192) delete 9c338a99/benchmark_data_storage0111-ib.pocstor_41097_object4779/head//24 by client.2174595.0:136982 0.000000 2014-08-13 18:06:06.850384 7f5c973a57a0 20 read_log 194215'15194 (193305'15015) clean e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by osd.396.0:705 2014-08-12 15:46:23.050635 2014-08-13 18:06:06.850396 7f5c973a57a0 20 read_log 194215'15195 (194215'15194) delete e01d8a99/benchmark_data_storage0111-ib.pocstor_62621_object31627/head//24 by client.2174595.0:136984 0.000000 2014-08-13 18:06:06.850411 7f5c973a57a0 20 read_log 194215'15196 (193305'15004) clean 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by osd.396.0:707 2014-08-12 15:46:23.057547 2014-08-13 18:06:06.850424 7f5c973a57a0 20 read_log 194215'15197 (194215'15196) delete 784f8a99/benchmark_data_storage0111-ib.pocstor_62621_object21321/head//24 by client.2174595.0:136986 0.000000 2014-08-13 18:06:06.850435 7f5c973a57a0 20 read_log 194215'15198 (192888'14909) clean 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by osd.396.0:709 2014-08-12 15:46:23.063747 2014-08-13 18:06:06.850458 7f5c973a57a0 20 read_log 194215'15199 (194215'15198) delete 98069a99/benchmark_data_storage0111-ib.pocstor_2228_object27890/head//24 by client.2174595.0:136988 0.000000 2014-08-13 18:06:06.850472 7f5c973a57a0 20 read_log 194215'15200 (192888'14699) clean e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by osd.396.0:711 2014-08-12 15:46:23.070050 2014-08-13 18:06:06.850494 7f5c973a57a0 20 read_log 194215'15201 (194215'15200) delete e6169a99/benchmark_data_storage0111-ib.pocstor_41097_object23648/head//24 by client.2174595.0:136990 0.000000 2014-08-13 18:06:06.850520 7f5c973a57a0 20 read_log 194215'15202 (192834'14557) clean eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by osd.396.0:713 2014-08-12 15:46:23.076399 2014-08-13 18:06:06.850533 7f5c973a57a0 20 read_log 194215'15203 (194215'15202) delete eec89a99/benchmark_data_storage0111-ib.pocstor_41526_object20882/head//24 by client.2174595.0:136992 0.000000 2014-08-13 18:06:06.850544 7f5c973a57a0 20 read_log 194215'15204 (193305'15003) clean 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by osd.396.0:715 2014-08-12 15:46:23.082885 2014-08-13 18:06:06.850557 7f5c973a57a0 20 read_log 194215'15205 (194215'15204) delete 9ae0aa99/benchmark_data_storage0111-ib.pocstor_62621_object21053/head//24 by client.2174595.0:136994 0.000000 2014-08-13 18:06:06.850568 7f5c973a57a0 20 read_log 194215'15206 (193305'15000) clean 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by osd.396.0:717 2014-08-12 15:46:23.088612 2014-08-13 18:06:06.850580 7f5c973a57a0 20 read_log 194215'15207 (194215'15206) delete 14d2aa99/benchmark_data_storage0111-ib.pocstor_62621_object19546/head//24 by client.2174595.0:136996 0.000000 2014-08-13 18:06:06.850591 7f5c973a57a0 20 read_log 194215'15208 (192888'14852) clean 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by osd.396.0:719 2014-08-12 15:46:23.094394 2014-08-13 18:06:06.850613 7f5c973a57a0 20 read_log 194215'15209 (194215'15208) delete 8ca4aa99/benchmark_data_storage0111-ib.pocstor_2228_object1589/head//24 by client.2174595.0:136998 0.000000 2014-08-13 18:06:06.850625 7f5c973a57a0 20 read_log 194215'15210 (193305'14998) clean d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by osd.396.0:721 2014-08-12 15:46:23.100438 2014-08-13 18:06:06.850644 7f5c973a57a0 20 read_log 194215'15211 (194215'15210) delete d442ba99/benchmark_data_storage0111-ib.pocstor_62621_object18217/head//24 by client.2174595.0:137000 0.000000 2014-08-13 18:06:06.850672 7f5c973a57a0 20 read_log 194215'15212 (192888'14898) clean 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by osd.396.0:723 2014-08-12 15:46:23.106130 2014-08-13 18:06:06.850687 7f5c973a57a0 20 read_log 194215'15213 (194215'15212) delete 4b0ca99/benchmark_data_storage0111-ib.pocstor_2228_object22664/head//24 by client.2174595.0:137002 0.000000 2014-08-13 18:06:06.850706 7f5c973a57a0 20 read_log 194215'15214 (192834'14013) clean 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by osd.396.0:725 2014-08-12 15:46:23.111152 2014-08-13 18:06:06.850719 7f5c973a57a0 20 read_log 194215'15215 (194215'15214) delete 11e6ca99/benchmark_data_storage0111-ib.pocstor_26659_object15936/head//24 by client.2174595.0:137004 0.000000 2014-08-13 18:06:06.850731 7f5c973a57a0 20 read_log 194215'15216 (192834'14037) clean bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by osd.396.0:727 2014-08-12 15:46:23.116174 2014-08-13 18:06:06.850743 7f5c973a57a0 20 read_log 194215'15217 (194215'15216) delete bd87ca99/benchmark_data_storage0111-ib.pocstor_26659_object18868/head//24 by client.2174595.0:137006 0.000000 2014-08-13 18:06:06.850754 7f5c973a57a0 20 read_log 194215'15218 (192834'14485) clean 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by osd.396.0:729 2014-08-12 15:46:23.121523 2014-08-13 18:06:06.850767 7f5c973a57a0 20 read_log 194215'15219 (194215'15218) delete 2f68ca99/benchmark_data_storage0111-ib.pocstor_41526_object11828/head//24 by client.2174595.0:137008 0.000000 2014-08-13 18:06:06.850778 7f5c973a57a0 20 read_log 194215'15220 (192834'14034) clean ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by osd.396.0:731 2014-08-12 15:46:23.127300 2014-08-13 18:06:06.850798 7f5c973a57a0 20 read_log 194215'15221 (194215'15220) delete ebbaca99/benchmark_data_storage0111-ib.pocstor_26659_object18674/head//24 by client.2174595.0:137010 0.000000 2014-08-13 18:06:06.850812 7f5c973a57a0 20 read_log 194215'15222 (192834'14456) clean eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by osd.396.0:733 2014-08-12 15:46:23.133100 2014-08-13 18:06:06.850825 7f5c973a57a0 20 read_log 194215'15223 (194215'15222) delete eb61da99/benchmark_data_storage0111-ib.pocstor_41526_object7867/head//24 by client.2174595.0:137012 0.000000 2014-08-13 18:06:06.850836 7f5c973a57a0 20 read_log 194215'15224 (192888'14894) clean e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by osd.396.0:735 2014-08-12 15:46:23.138956 2014-08-13 18:06:06.850848 7f5c973a57a0 20 read_log 194215'15225 (194215'15224) delete e4fda99/benchmark_data_storage0111-ib.pocstor_2228_object21214/head//24 by client.2174595.0:137014 0.000000 2014-08-13 18:06:06.850859 7f5c973a57a0 20 read_log 194215'15226 (192888'14665) clean f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by osd.396.0:737 2014-08-12 15:46:23.144657 2014-08-13 18:06:06.850882 7f5c973a57a0 20 read_log 194215'15227 (194215'15226) delete f761ea99/benchmark_data_storage0111-ib.pocstor_41097_object1333/head//24 by client.2174595.0:137016 0.000000 2014-08-13 18:06:06.850915 7f5c973a57a0 20 read_log 194215'15228 (192888'14878) clean 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by osd.396.0:739 2014-08-12 15:46:23.150380 2014-08-13 18:06:06.850928 7f5c973a57a0 20 read_log 194215'15229 (194215'15228) delete 6647ea99/benchmark_data_storage0111-ib.pocstor_2228_object14067/head//24 by client.2174595.0:137018 0.000000 2014-08-13 18:06:06.850939 7f5c973a57a0 20 read_log 194215'15230 (192834'14549) clean be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by osd.396.0:741 2014-08-12 15:46:23.156271 2014-08-13 18:06:06.850952 7f5c973a57a0 20 read_log 194215'15231 (194215'15230) delete be88ea99/benchmark_data_storage0111-ib.pocstor_41526_object19933/head//24 by client.2174595.0:137020 0.000000 2014-08-13 18:06:06.850966 7f5c973a57a0 20 read_log 194215'15232 (192834'14404) clean 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by osd.396.0:743 2014-08-12 15:46:23.162486 2014-08-13 18:06:06.850978 7f5c973a57a0 20 read_log 194215'15233 (194215'15232) delete 7fd0fa99/benchmark_data_storage0111-ib.pocstor_41526_object705/head//24 by client.2174595.0:137022 0.000000 2014-08-13 18:06:06.850989 7f5c973a57a0 20 read_log 194215'15234 (192834'13977) clean a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by osd.396.0:745 2014-08-12 15:46:23.167716 2014-08-13 18:06:06.851002 7f5c973a57a0 20 read_log 194215'15235 (194215'15234) delete a433fa99/benchmark_data_storage0111-ib.pocstor_26659_object10347/head//24 by client.2174595.0:137024 0.000000 2014-08-13 18:06:06.851016 7f5c973a57a0 20 read_log 194215'15236 (192888'14883) clean e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by osd.396.0:747 2014-08-12 15:46:23.174057 2014-08-13 18:06:06.851029 7f5c973a57a0 20 read_log 194215'15237 (194215'15236) delete e829fa99/benchmark_data_storage0111-ib.pocstor_2228_object16153/head//24 by client.2174595.0:137026 0.000000 2014-08-13 18:06:06.851040 7f5c973a57a0 20 read_log 194215'15238 (193305'14988) clean e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by osd.396.0:749 2014-08-12 15:46:23.180767 2014-08-13 18:06:06.851052 7f5c973a57a0 20 read_log 194215'15239 (194215'15238) delete e02afa99/benchmark_data_storage0111-ib.pocstor_62621_object5987/head//24 by client.2174595.0:137028 0.000000 2014-08-13 18:06:06.851067 7f5c973a57a0 20 read_log 194215'15240 (193305'14992) clean 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by osd.396.0:751 2014-08-12 15:46:23.187338 2014-08-13 18:06:06.851098 7f5c973a57a0 20 read_log 194215'15241 (194215'15240) delete 955afa99/benchmark_data_storage0111-ib.pocstor_62621_object12621/head//24 by client.2174595.0:137030 0.000000 2014-08-13 18:06:06.851117 7f5c973a57a0 20 read_log 194215'15242 (192834'13918) clean c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by osd.396.0:753 2014-08-12 15:46:23.194150 2014-08-13 18:06:06.851137 7f5c973a57a0 20 read_log 194215'15243 (194215'15242) delete c14efa99/benchmark_data_storage0111-ib.pocstor_26659_object3064/head//24 by client.2174595.0:137032 0.000000 2014-08-13 18:06:06.851157 7f5c973a57a0 20 read_log 194215'15244 (192834'14565) clean a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by osd.396.0:755 2014-08-12 15:46:23.199891 2014-08-13 18:06:06.851171 7f5c973a57a0 20 read_log 194215'15245 (194215'15244) delete a3890e99/benchmark_data_storage0111-ib.pocstor_41526_object23028/head//24 by client.2174595.0:137034 0.000000 2014-08-13 18:06:06.851190 7f5c973a57a0 20 read_log 194215'15246 (192888'14872) clean c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by osd.396.0:757 2014-08-12 15:46:23.205659 2014-08-13 18:06:06.851206 7f5c973a57a0 20 read_log 194215'15247 (194215'15246) delete c12c0e99/benchmark_data_storage0111-ib.pocstor_2228_object10824/head//24 by client.2174595.0:137036 0.000000 2014-08-13 18:06:06.851217 7f5c973a57a0 20 read_log 194215'15248 (192834'14444) clean ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by osd.396.0:759 2014-08-12 15:46:23.211354 2014-08-13 18:06:06.851229 7f5c973a57a0 20 read_log 194215'15249 (194215'15248) delete ceed0e99/benchmark_data_storage0111-ib.pocstor_41526_object6881/head//24 by client.2174595.0:137038 0.000000 2014-08-13 18:06:06.851240 7f5c973a57a0 20 read_log 194215'15250 (192834'13915) clean e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by osd.396.0:761 2014-08-12 15:46:23.217129 2014-08-13 18:06:06.851253 7f5c973a57a0 20 read_log 194215'15251 (194215'15250) delete e1f81e99/benchmark_data_storage0111-ib.pocstor_26659_object2965/head//24 by client.2174595.0:137040 0.000000 2014-08-13 18:06:06.851268 7f5c973a57a0 20 read_log 194215'15252 (193305'15008) clean a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by osd.396.0:763 2014-08-12 15:46:23.222865 2014-08-13 18:06:06.851286 7f5c973a57a0 20 read_log 194215'15253 (194215'15252) delete a1b82e99/benchmark_data_storage0111-ib.pocstor_62621_object22971/head//24 by client.2174595.0:137042 0.000000 2014-08-13 18:06:06.851298 7f5c973a57a0 20 read_log 194215'15254 (192888'14705) clean 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by osd.396.0:765 2014-08-12 15:46:23.228534 2014-08-13 18:06:06.851313 7f5c973a57a0 20 read_log 194215'15255 (194215'15254) delete 83f63e99/benchmark_data_storage0111-ib.pocstor_41097_object27692/head//24 by client.2174595.0:137044 0.000000 2014-08-13 18:06:06.851325 7f5c973a57a0 20 read_log 194215'15256 (192888'14678) clean a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by osd.396.0:767 2014-08-12 15:46:23.233671 2014-08-13 18:06:06.851339 7f5c973a57a0 20 read_log 194215'15257 (194215'15256) delete a2df4e99/benchmark_data_storage0111-ib.pocstor_41097_object10362/head//24 by client.2174595.0:137046 0.000000 2014-08-13 18:06:06.851350 7f5c973a57a0 20 read_log 194215'15258 (192888'14675) clean e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by osd.396.0:769 2014-08-12 15:46:23.239022 2014-08-13 18:06:06.851365 7f5c973a57a0 20 read_log 194215'15259 (194215'15258) delete e2655e99/benchmark_data_storage0111-ib.pocstor_41097_object9378/head//24 by client.2174595.0:137048 0.000000 2014-08-13 18:06:06.851377 7f5c973a57a0 20 read_log 194215'15260 (192888'14853) clean 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by osd.396.0:771 2014-08-12 15:46:23.244571 2014-08-13 18:06:06.851409 7f5c973a57a0 20 read_log 194215'15261 (194215'15260) delete 94855e99/benchmark_data_storage0111-ib.pocstor_2228_object2058/head//24 by client.2174595.0:137050 0.000000 2014-08-13 18:06:06.851424 7f5c973a57a0 20 read_log 194215'15262 (193305'14985) clean 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by osd.396.0:773 2014-08-12 15:46:23.250581 2014-08-13 18:06:06.851437 7f5c973a57a0 20 read_log 194215'15263 (194215'15262) delete 61d06e99/benchmark_data_storage0111-ib.pocstor_62621_object3696/head//24 by client.2174595.0:137052 0.000000 2014-08-13 18:06:06.851448 7f5c973a57a0 20 read_log 194215'15264 (192888'14889) clean 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by osd.396.0:775 2014-08-12 15:46:23.255906 2014-08-13 18:06:06.851464 7f5c973a57a0 20 read_log 194215'15265 (194215'15264) delete 492e6e99/benchmark_data_storage0111-ib.pocstor_2228_object18020/head//24 by client.2174595.0:137054 0.000000 2014-08-13 18:06:06.851475 7f5c973a57a0 20 read_log 194215'15266 (192834'14556) clean e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by osd.396.0:777 2014-08-12 15:46:23.261388 2014-08-13 18:06:06.851487 7f5c973a57a0 20 read_log 194215'15267 (194215'15266) delete e59e6e99/benchmark_data_storage0111-ib.pocstor_41526_object20820/head//24 by client.2174595.0:137056 0.000000 2014-08-13 18:06:06.851498 7f5c973a57a0 20 read_log 194215'15268 (193305'14999) clean 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by osd.396.0:779 2014-08-12 15:46:23.267332 2014-08-13 18:06:06.851521 7f5c973a57a0 20 read_log 194215'15269 (194215'15268) delete 9a257e99/benchmark_data_storage0111-ib.pocstor_62621_object18474/head//24 by client.2174595.0:137058 0.000000 2014-08-13 18:06:06.851533 7f5c973a57a0 20 read_log 194215'15270 (192888'14870) clean ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by osd.396.0:781 2014-08-12 15:46:23.273057 2014-08-13 18:06:06.851546 7f5c973a57a0 20 read_log 194215'15271 (194215'15270) delete ac887e99/benchmark_data_storage0111-ib.pocstor_2228_object8653/head//24 by client.2174595.0:137060 0.000000 2014-08-13 18:06:06.851564 7f5c973a57a0 20 read_log 194215'15272 (193305'15001) clean d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by osd.396.0:783 2014-08-12 15:46:23.278679 2014-08-13 18:06:06.851580 7f5c973a57a0 20 read_log 194215'15273 (194215'15272) delete d2af7e99/benchmark_data_storage0111-ib.pocstor_62621_object19611/head//24 by client.2174595.0:137062 0.000000 2014-08-13 18:06:06.851591 7f5c973a57a0 20 read_log 194215'15274 (192888'14913) clean d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by osd.396.0:785 2014-08-12 15:46:23.283678 2014-08-13 18:06:06.851604 7f5c973a57a0 20 read_log 194215'15275 (194215'15274) delete d6258e99/benchmark_data_storage0111-ib.pocstor_2228_object29644/head//24 by client.2174595.0:137064 0.000000 2014-08-13 18:06:06.851615 7f5c973a57a0 20 read_log 194215'15276 (192834'14498) clean 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by osd.396.0:787 2014-08-12 15:46:23.289466 2014-08-13 18:06:06.851627 7f5c973a57a0 20 read_log 194215'15277 (194215'15276) delete 861b8e99/benchmark_data_storage0111-ib.pocstor_41526_object13161/head//24 by client.2174595.0:137066 0.000000 2014-08-13 18:06:06.851638 7f5c973a57a0 20 read_log 194215'15278 (192834'14488) clean 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by osd.396.0:789 2014-08-12 15:46:23.295103 2014-08-13 18:06:06.851650 7f5c973a57a0 20 read_log 194215'15279 (194215'15278) delete 74a2ae99/benchmark_data_storage0111-ib.pocstor_41526_object12195/head//24 by client.2174595.0:137068 0.000000 2014-08-13 18:06:06.851661 7f5c973a57a0 20 read_log 194215'15280 (192888'14895) clean 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by osd.396.0:791 2014-08-12 15:46:23.300957 2014-08-13 18:06:06.851681 7f5c973a57a0 20 read_log 194215'15281 (194215'15280) delete 8d33ae99/benchmark_data_storage0111-ib.pocstor_2228_object21284/head//24 by client.2174595.0:137070 0.000000 2014-08-13 18:06:06.851692 7f5c973a57a0 20 read_log 194215'15282 (192834'14472) clean b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by osd.396.0:793 2014-08-12 15:46:23.307475 2014-08-13 18:06:06.851705 7f5c973a57a0 20 read_log 194215'15283 (194215'15282) delete b24cae99/benchmark_data_storage0111-ib.pocstor_41526_object10404/head//24 by client.2174595.0:137072 0.000000 2014-08-13 18:06:06.851717 7f5c973a57a0 20 read_log 194215'15284 (192888'14903) clean 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by osd.396.0:795 2014-08-12 15:46:23.313700 2014-08-13 18:06:06.851730 7f5c973a57a0 20 read_log 194215'15285 (194215'15284) delete 7dc2be99/benchmark_data_storage0111-ib.pocstor_2228_object24846/head//24 by client.2174595.0:137074 0.000000 2014-08-13 18:06:06.851741 7f5c973a57a0 20 read_log 194215'15286 (192834'13970) clean 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by osd.396.0:797 2014-08-12 15:46:23.319927 2014-08-13 18:06:06.851753 7f5c973a57a0 20 read_log 194215'15287 (194215'15286) delete 48f2be99/benchmark_data_storage0111-ib.pocstor_26659_object9363/head//24 by client.2174595.0:137076 0.000000 2014-08-13 18:06:06.851764 7f5c973a57a0 20 read_log 194215'15288 (192834'14537) clean 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by osd.396.0:799 2014-08-12 15:46:23.325965 2014-08-13 18:06:06.851776 7f5c973a57a0 20 read_log 194215'15289 (194215'15288) delete 2be3be99/benchmark_data_storage0111-ib.pocstor_41526_object17927/head//24 by client.2174595.0:137078 0.000000 2014-08-13 18:06:06.851786 7f5c973a57a0 20 read_log 194215'15290 (192888'14867) clean dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by osd.396.0:801 2014-08-12 15:46:23.331452 2014-08-13 18:06:06.851799 7f5c973a57a0 20 read_log 194215'15291 (194215'15290) delete dd7abe99/benchmark_data_storage0111-ib.pocstor_2228_object7779/head//24 by client.2174595.0:137080 0.000000 2014-08-13 18:06:06.851809 7f5c973a57a0 20 read_log 194215'15292 (192888'14695) clean 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by osd.396.0:803 2014-08-12 15:46:23.336701 2014-08-13 18:06:06.851821 7f5c973a57a0 20 read_log 194215'15293 (194215'15292) delete 7ddbe99/benchmark_data_storage0111-ib.pocstor_41097_object22141/head//24 by client.2174595.0:137082 0.000000 2014-08-13 18:06:06.851832 7f5c973a57a0 20 read_log 194215'15294 (192834'14024) clean 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by osd.396.0:805 2014-08-12 15:46:23.342473 2014-08-13 18:06:06.851844 7f5c973a57a0 20 read_log 194215'15295 (194215'15294) delete 2c6fbe99/benchmark_data_storage0111-ib.pocstor_26659_object17759/head//24 by client.2174595.0:137084 0.000000 2014-08-13 18:06:06.851855 7f5c973a57a0 20 read_log 194215'15296 (193305'15005) clean 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by osd.396.0:807 2014-08-12 15:46:23.348316 2014-08-13 18:06:06.851867 7f5c973a57a0 20 read_log 194215'15297 (194215'15296) delete 2310ce99/benchmark_data_storage0111-ib.pocstor_62621_object22401/head//24 by client.2174595.0:137086 0.000000 2014-08-13 18:06:06.851878 7f5c973a57a0 20 read_log 194215'15298 (193305'15014) clean b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by osd.396.0:809 2014-08-12 15:46:23.353740 2014-08-13 18:06:06.851890 7f5c973a57a0 20 read_log 194215'15299 (194215'15298) delete b194ce99/benchmark_data_storage0111-ib.pocstor_62621_object29335/head//24 by client.2174595.0:137088 0.000000 2014-08-13 18:06:06.851901 7f5c973a57a0 20 read_log 194215'15300 (193305'14991) clean 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by osd.396.0:811 2014-08-12 15:46:23.359191 2014-08-13 18:06:06.851943 7f5c973a57a0 20 read_log 194215'15301 (194215'15300) delete 7546ee99/benchmark_data_storage0111-ib.pocstor_62621_object12053/head//24 by client.2174595.0:137090 0.000000 2014-08-13 18:06:06.851958 7f5c973a57a0 20 read_log 194215'15302 (192834'14459) clean 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by osd.396.0:813 2014-08-12 15:46:23.364238 2014-08-13 18:06:06.851971 7f5c973a57a0 20 read_log 194215'15303 (194215'15302) delete 53aeee99/benchmark_data_storage0111-ib.pocstor_41526_object8025/head//24 by client.2174595.0:137092 0.000000 2014-08-13 18:06:06.851982 7f5c973a57a0 20 read_log 194215'15304 (194215'15304) modify 299/hit_set_24.299_archive_2014-08-12 15:46:22.535891_2014-08-12 17:24:42.988027/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.988140 2014-08-13 18:06:06.852012 7f5c973a57a0 20 read_log 194215'15305 (194215'15016) delete 299/hit_set_24.299_archive_2014-08-12 15:32:25.825077_2014-08-12 15:46:22.535741/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:42.988140 2014-08-13 18:06:06.852027 7f5c973a57a0 20 read_log 194215'15306 (0'0) promote 1b056e99/benchmark_data_storage0111-ib.pocstor_34628_object790/head//24 by osd.396.0:1210 2014-08-12 17:24:42.990773 2014-08-13 18:06:06.852039 7f5c973a57a0 20 read_log 194215'15307 (194215'15306) modify 1b056e99/benchmark_data_storage0111-ib.pocstor_34628_object790/head//24 by client.2193759.0:791 2014-08-12 17:24:42.981361 2014-08-13 18:06:06.852054 7f5c973a57a0 20 read_log 194215'15308 (0'0) promote 43f8e99/benchmark_data_storage0111-ib.pocstor_34628_object881/head//24 by osd.396.0:1212 2014-08-12 17:24:43.333341 2014-08-13 18:06:06.852067 7f5c973a57a0 20 read_log 194215'15309 (194215'15308) modify 43f8e99/benchmark_data_storage0111-ib.pocstor_34628_object881/head//24 by client.2193759.0:882 2014-08-12 17:24:43.321742 2014-08-13 18:06:06.852079 7f5c973a57a0 20 read_log 194215'15310 (0'0) promote 3b59da99/benchmark_data_storage0111-ib.pocstor_34628_object1059/head//24 by osd.396.0:1214 2014-08-12 17:24:44.013517 2014-08-13 18:06:06.852091 7f5c973a57a0 20 read_log 194215'15311 (194215'15310) modify 3b59da99/benchmark_data_storage0111-ib.pocstor_34628_object1059/head//24 by client.2193759.0:1060 2014-08-12 17:24:44.005241 2014-08-13 18:06:06.852104 7f5c973a57a0 20 read_log 194370'15312 (0'0) promote 9cae0699/benchmark_data_storage0111-ib.pocstor_12495_object139/head//24 by osd.396.0:1223 2014-08-12 20:30:14.679773 2014-08-13 18:06:06.852116 7f5c973a57a0 20 read_log 194370'15313 (194370'15312) modify 9cae0699/benchmark_data_storage0111-ib.pocstor_12495_object139/head//24 by client.1508531.0:140 2014-08-12 20:30:14.670910 2014-08-13 18:06:06.852128 7f5c973a57a0 20 read_log 194370'15314 (0'0) promote 8aa7ee99/benchmark_data_storage0111-ib.pocstor_12495_object320/head//24 by osd.396.0:1229 2014-08-12 20:30:15.509343 2014-08-13 18:06:06.852140 7f5c973a57a0 20 read_log 194370'15315 (194370'15314) modify 8aa7ee99/benchmark_data_storage0111-ib.pocstor_12495_object320/head//24 by client.1508531.0:321 2014-08-12 20:30:15.501868 2014-08-13 18:06:06.852158 7f5c973a57a0 20 read_log 194370'15316 (0'0) promote acc91a99/benchmark_data_storage0111-ib.pocstor_12495_object987/head//24 by osd.396.0:1237 2014-08-12 20:30:18.080041 2014-08-13 18:06:06.852171 7f5c973a57a0 20 read_log 194370'15317 (194370'15316) modify acc91a99/benchmark_data_storage0111-ib.pocstor_12495_object987/head//24 by client.1508531.0:988 2014-08-12 20:30:18.072641 2014-08-13 18:06:06.852183 7f5c973a57a0 20 read_log 194370'15318 (0'0) promote 96764699/benchmark_data_storage0111-ib.pocstor_12495_object2180/head//24 by osd.396.0:1247 2014-08-12 20:30:22.575736 2014-08-13 18:06:06.852198 7f5c973a57a0 20 read_log 194370'15319 (194370'15318) modify 96764699/benchmark_data_storage0111-ib.pocstor_12495_object2180/head//24 by client.1508531.0:2181 2014-08-12 20:30:22.567206 2014-08-13 18:06:06.852218 7f5c973a57a0 20 read_log 194370'15320 (0'0) promote bff57e99/benchmark_data_storage0111-ib.pocstor_12495_object2493/head//24 by osd.396.0:1249 2014-08-12 20:30:23.715490 2014-08-13 18:06:06.852233 7f5c973a57a0 20 read_log 194370'15321 (194370'15320) modify bff57e99/benchmark_data_storage0111-ib.pocstor_12495_object2493/head//24 by client.1508531.0:2494 2014-08-12 20:30:23.704600 2014-08-13 18:06:06.852245 7f5c973a57a0 20 read_log 194370'15322 (0'0) promote 8181d299/benchmark_data_storage0111-ib.pocstor_12495_object4343/head//24 by osd.396.0:1261 2014-08-12 20:30:30.617042 2014-08-13 18:06:06.852262 7f5c973a57a0 20 read_log 194370'15323 (194370'15322) modify 8181d299/benchmark_data_storage0111-ib.pocstor_12495_object4343/head//24 by client.1508531.0:4344 2014-08-12 20:30:30.608998 2014-08-13 18:06:06.852297 7f5c973a57a0 20 read_log 194370'15324 (0'0) promote 247f6e99/benchmark_data_storage0111-ib.pocstor_12495_object5228/head//24 by osd.396.0:1269 2014-08-12 20:30:34.191157 2014-08-13 18:06:06.852313 7f5c973a57a0 20 read_log 194370'15325 (194370'15324) modify 247f6e99/benchmark_data_storage0111-ib.pocstor_12495_object5228/head//24 by client.1508531.0:5229 2014-08-12 20:30:34.181438 2014-08-13 18:06:06.852327 7f5c973a57a0 20 read_log 194370'15326 (0'0) promote 220a9a99/benchmark_data_storage0111-ib.pocstor_12495_object5651/head//24 by osd.396.0:1275 2014-08-12 20:30:36.123836 2014-08-13 18:06:06.852340 7f5c973a57a0 20 read_log 194370'15327 (194370'15326) modify 220a9a99/benchmark_data_storage0111-ib.pocstor_12495_object5651/head//24 by client.1508531.0:5652 2014-08-12 20:30:36.113731 2014-08-13 18:06:06.852352 7f5c973a57a0 20 read_log 194370'15328 (0'0) promote 3ae7fa99/benchmark_data_storage0111-ib.pocstor_12495_object6649/head//24 by osd.396.0:1279 2014-08-12 20:30:40.652283 2014-08-13 18:06:06.852370 7f5c973a57a0 20 read_log 194370'15329 (194370'15328) modify 3ae7fa99/benchmark_data_storage0111-ib.pocstor_12495_object6649/head//24 by client.1508531.0:6650 2014-08-12 20:30:40.645058 2014-08-13 18:06:06.852383 7f5c973a57a0 20 read_log 194370'15330 (0'0) promote 1f91a699/benchmark_data_storage0111-ib.pocstor_12495_object6708/head//24 by osd.396.0:1281 2014-08-12 20:30:40.914268 2014-08-13 18:06:06.852404 7f5c973a57a0 20 read_log 194370'15331 (194370'15330) modify 1f91a699/benchmark_data_storage0111-ib.pocstor_12495_object6708/head//24 by client.1508531.0:6709 2014-08-12 20:30:40.908250 2014-08-13 18:06:06.852421 7f5c973a57a0 20 read_log 194370'15332 (0'0) promote d24ae99/benchmark_data_storage0111-ib.pocstor_12495_object7741/head//24 by osd.396.0:1287 2014-08-12 20:30:46.194226 2014-08-13 18:06:06.852441 7f5c973a57a0 20 read_log 194370'15333 (194370'15332) modify d24ae99/benchmark_data_storage0111-ib.pocstor_12495_object7741/head//24 by client.1508531.0:7742 2014-08-12 20:30:46.186782 2014-08-13 18:06:06.852455 7f5c973a57a0 20 read_log 194370'15334 (0'0) promote 8759299/benchmark_data_storage0111-ib.pocstor_12495_object8285/head//24 by osd.396.0:1291 2014-08-12 20:30:49.087710 2014-08-13 18:06:06.852471 7f5c973a57a0 20 read_log 194370'15335 (194370'15334) modify 8759299/benchmark_data_storage0111-ib.pocstor_12495_object8285/head//24 by client.1508531.0:8286 2014-08-12 20:30:49.079026 2014-08-13 18:06:06.852489 7f5c973a57a0 20 read_log 194370'15336 (0'0) promote 24ed1a99/benchmark_data_storage0111-ib.pocstor_12495_object9282/head//24 by osd.396.0:1297 2014-08-12 20:30:55.021664 2014-08-13 18:06:06.852505 7f5c973a57a0 20 read_log 194370'15337 (194370'15336) modify 24ed1a99/benchmark_data_storage0111-ib.pocstor_12495_object9282/head//24 by client.1508531.0:9283 2014-08-12 20:30:55.012508 2014-08-13 18:06:06.852518 7f5c973a57a0 20 read_log 194370'15338 (0'0) promote d64e6299/benchmark_data_storage0111-ib.pocstor_12495_object10206/head//24 by osd.396.0:1301 2014-08-12 20:31:00.472447 2014-08-13 18:06:06.852531 7f5c973a57a0 20 read_log 194370'15339 (194370'15338) modify d64e6299/benchmark_data_storage0111-ib.pocstor_12495_object10206/head//24 by client.1508531.0:10207 2014-08-12 20:31:00.462507 2014-08-13 18:06:06.852564 7f5c973a57a0 20 read_log 194370'15340 (0'0) promote 38862299/benchmark_data_storage0111-ib.pocstor_12495_object10505/head//24 by osd.396.0:1305 2014-08-12 20:31:02.257545 2014-08-13 18:06:06.852578 7f5c973a57a0 20 read_log 194370'15341 (194370'15340) modify 38862299/benchmark_data_storage0111-ib.pocstor_12495_object10505/head//24 by client.1508531.0:10506 2014-08-12 20:31:02.249616 2014-08-13 18:06:06.852600 7f5c973a57a0 20 read_log 194370'15342 (0'0) promote ea103699/benchmark_data_storage0111-ib.pocstor_12495_object10949/head//24 by osd.396.0:1311 2014-08-12 20:31:04.977730 2014-08-13 18:06:06.852613 7f5c973a57a0 20 read_log 194370'15343 (194370'15342) modify ea103699/benchmark_data_storage0111-ib.pocstor_12495_object10949/head//24 by client.1508531.0:10950 2014-08-12 20:31:04.967557 2014-08-13 18:06:06.852625 7f5c973a57a0 20 read_log 194370'15344 (0'0) promote ae709e99/benchmark_data_storage0111-ib.pocstor_12495_object11463/head//24 by osd.396.0:1317 2014-08-12 20:31:08.088522 2014-08-13 18:06:06.852645 7f5c973a57a0 20 read_log 194370'15345 (194370'15344) modify ae709e99/benchmark_data_storage0111-ib.pocstor_12495_object11463/head//24 by client.1508531.0:11464 2014-08-12 20:31:08.079977 2014-08-13 18:06:06.852660 7f5c973a57a0 20 read_log 194370'15346 (0'0) promote 6d3fe699/benchmark_data_storage0111-ib.pocstor_12495_object11918/head//24 by osd.396.0:1319 2014-08-12 20:31:10.778867 2014-08-13 18:06:06.852672 7f5c973a57a0 20 read_log 194370'15347 (194370'15346) modify 6d3fe699/benchmark_data_storage0111-ib.pocstor_12495_object11918/head//24 by client.1508531.0:11919 2014-08-12 20:31:10.770035 2014-08-13 18:06:06.852684 7f5c973a57a0 20 read_log 194370'15348 (0'0) promote c7c61e99/benchmark_data_storage0111-ib.pocstor_12495_object12175/head//24 by osd.396.0:1321 2014-08-12 20:31:12.666304 2014-08-13 18:06:06.852697 7f5c973a57a0 20 read_log 194370'15349 (194370'15348) modify c7c61e99/benchmark_data_storage0111-ib.pocstor_12495_object12175/head//24 by client.1508531.0:12176 2014-08-12 20:31:12.657844 2014-08-13 18:06:06.852709 7f5c973a57a0 20 read_log 194370'15350 (0'0) promote 1ff19e99/benchmark_data_storage0111-ib.pocstor_12495_object12984/head//24 by osd.396.0:1329 2014-08-12 20:31:19.217025 2014-08-13 18:06:06.852721 7f5c973a57a0 20 read_log 194370'15351 (194370'15350) modify 1ff19e99/benchmark_data_storage0111-ib.pocstor_12495_object12984/head//24 by client.1508531.0:12985 2014-08-12 20:31:19.208131 2014-08-13 18:06:06.852733 7f5c973a57a0 20 read_log 194370'15352 (0'0) promote 5020e99/benchmark_data_storage0111-ib.pocstor_12495_object13208/head//24 by osd.396.0:1333 2014-08-12 20:31:21.390019 2014-08-13 18:06:06.852746 7f5c973a57a0 20 read_log 194370'15353 (194370'15352) modify 5020e99/benchmark_data_storage0111-ib.pocstor_12495_object13208/head//24 by client.1508531.0:13209 2014-08-12 20:31:21.380233 2014-08-13 18:06:06.852758 7f5c973a57a0 20 read_log 194377'15354 (0'0) promote 8212c299/benchmark_data_storage0111-ib.pocstor_25065_object972/head//24 by osd.396.0:1339 2014-08-12 20:33:01.202646 2014-08-13 18:06:06.852770 7f5c973a57a0 20 read_log 194377'15355 (194377'15354) modify 8212c299/benchmark_data_storage0111-ib.pocstor_25065_object972/head//24 by client.2231190.0:973 2014-08-12 20:33:01.194403 2014-08-13 18:06:06.852783 7f5c973a57a0 20 read_log 194377'15356 (0'0) promote b83d699/benchmark_data_storage0111-ib.pocstor_25065_object1317/head//24 by osd.396.0:1343 2014-08-12 20:33:02.516360 2014-08-13 18:06:06.852795 7f5c973a57a0 20 read_log 194377'15357 (194377'15356) modify b83d699/benchmark_data_storage0111-ib.pocstor_25065_object1317/head//24 by client.2231190.0:1318 2014-08-12 20:33:02.508595 2014-08-13 18:06:06.852807 7f5c973a57a0 20 read_log 194377'15358 (0'0) promote 9fa81699/benchmark_data_storage0111-ib.pocstor_25065_object1483/head//24 by osd.396.0:1345 2014-08-12 20:33:03.137526 2014-08-13 18:06:06.852827 7f5c973a57a0 20 read_log 194377'15359 (194377'15358) modify 9fa81699/benchmark_data_storage0111-ib.pocstor_25065_object1483/head//24 by client.2231190.0:1484 2014-08-12 20:33:03.131429 2014-08-13 18:06:06.852840 7f5c973a57a0 20 read_log 194377'15360 (0'0) promote 742d9a99/benchmark_data_storage0111-ib.pocstor_25065_object3219/head//24 by osd.396.0:1353 2014-08-12 20:33:10.154487 2014-08-13 18:06:06.852852 7f5c973a57a0 20 read_log 194377'15361 (194377'15360) modify 742d9a99/benchmark_data_storage0111-ib.pocstor_25065_object3219/head//24 by client.2231190.0:3220 2014-08-12 20:33:10.146006 2014-08-13 18:06:06.852864 7f5c973a57a0 20 read_log 194377'15362 (0'0) promote 72c32a99/benchmark_data_storage0111-ib.pocstor_25065_object3233/head//24 by osd.396.0:1355 2014-08-12 20:33:10.205763 2014-08-13 18:06:06.852876 7f5c973a57a0 20 read_log 194377'15363 (194377'15362) modify 72c32a99/benchmark_data_storage0111-ib.pocstor_25065_object3233/head//24 by client.2231190.0:3234 2014-08-12 20:33:10.197907 2014-08-13 18:06:06.852889 7f5c973a57a0 20 read_log 194383'15364 (0'0) promote 38c90299/benchmark_data_storage0111-ib.pocstor_25065_object6446/head//24 by osd.396.0:1369 2014-08-12 20:34:03.906698 2014-08-13 18:06:06.852900 7f5c973a57a0 20 read_log 194383'15365 (194383'15364) modify 38c90299/benchmark_data_storage0111-ib.pocstor_25065_object6446/head//24 by client.2231190.0:6447 2014-08-12 20:34:03.897689 2014-08-13 18:06:06.852913 7f5c973a57a0 20 read_log 194383'15366 (0'0) promote d1656299/benchmark_data_storage0111-ib.pocstor_25065_object6747/head//24 by osd.396.0:1371 2014-08-12 20:34:08.294014 2014-08-13 18:06:06.852925 7f5c973a57a0 20 read_log 194383'15367 (194383'15366) modify d1656299/benchmark_data_storage0111-ib.pocstor_25065_object6747/head//24 by client.2231190.0:6748 2014-08-12 20:34:08.284675 2014-08-13 18:06:06.852937 7f5c973a57a0 20 read_log 194383'15368 (0'0) promote ba77ea99/benchmark_data_storage0111-ib.pocstor_25065_object7383/head//24 by osd.396.0:1377 2014-08-12 20:34:25.601299 2014-08-13 18:06:06.852949 7f5c973a57a0 20 read_log 194383'15369 (194383'15368) modify ba77ea99/benchmark_data_storage0111-ib.pocstor_25065_object7383/head//24 by client.2231190.0:7384 2014-08-12 20:34:25.592195 2014-08-13 18:06:06.852961 7f5c973a57a0 20 read_log 194383'15370 (0'0) promote 6f6aaa99/benchmark_data_storage0111-ib.pocstor_25065_object7457/head//24 by osd.396.0:1379 2014-08-12 20:34:27.606043 2014-08-13 18:06:06.852973 7f5c973a57a0 20 read_log 194383'15371 (194383'15370) modify 6f6aaa99/benchmark_data_storage0111-ib.pocstor_25065_object7457/head//24 by client.2231190.0:7458 2014-08-12 20:34:27.596612 2014-08-13 18:06:06.852986 7f5c973a57a0 20 read_log 194416'15372 (0'0) promote 3c19699/benchmark_data_storage0111-ib.pocstor_19103_object1477/head//24 by osd.396.0:1453 2014-08-13 15:28:36.326543 2014-08-13 18:06:06.852998 7f5c973a57a0 20 read_log 194416'15373 (194416'15372) modify 3c19699/benchmark_data_storage0111-ib.pocstor_19103_object1477/head//24 by client.2441958.0:1478 2014-08-13 15:28:36.317322 2014-08-13 18:06:06.853009 7f5c973a57a0 20 read_log 194416'15374 (0'0) promote 74d0d699/benchmark_data_storage0111-ib.pocstor_19103_object1698/head//24 by osd.396.0:1457 2014-08-13 15:28:37.159528 2014-08-13 18:06:06.853021 7f5c973a57a0 20 read_log 194416'15375 (194416'15374) modify 74d0d699/benchmark_data_storage0111-ib.pocstor_19103_object1698/head//24 by client.2441958.0:1699 2014-08-13 15:28:37.149912 2014-08-13 18:06:06.853033 7f5c973a57a0 20 read_log 194416'15376 (0'0) promote 8bf9f699/benchmark_data_storage0111-ib.pocstor_19103_object3913/head//24 by osd.396.0:1465 2014-08-13 15:28:45.608453 2014-08-13 18:06:06.853045 7f5c973a57a0 20 read_log 194416'15377 (194416'15376) modify 8bf9f699/benchmark_data_storage0111-ib.pocstor_19103_object3913/head//24 by client.2441958.0:3914 2014-08-13 15:28:45.598890 2014-08-13 18:06:06.853064 7f5c973a57a0 20 read_log 194416'15378 (0'0) promote c1e70e99/benchmark_data_storage0111-ib.pocstor_19103_object4064/head//24 by osd.396.0:1467 2014-08-13 15:28:46.180461 2014-08-13 18:06:06.853077 7f5c973a57a0 20 read_log 194416'15379 (194416'15378) modify c1e70e99/benchmark_data_storage0111-ib.pocstor_19103_object4064/head//24 by client.2441958.0:4065 2014-08-13 15:28:46.171096 2014-08-13 18:06:06.853089 7f5c973a57a0 20 read_log 194416'15380 (0'0) promote c1ac6e99/benchmark_data_storage0111-ib.pocstor_19103_object5160/head//24 by osd.396.0:1471 2014-08-13 15:28:50.513119 2014-08-13 18:06:06.853101 7f5c973a57a0 20 read_log 194416'15381 (194416'15380) modify c1ac6e99/benchmark_data_storage0111-ib.pocstor_19103_object5160/head//24 by client.2441958.0:5161 2014-08-13 15:28:50.503745 2014-08-13 18:06:06.853114 7f5c973a57a0 20 read_log 194416'15382 (0'0) promote 8b29a99/benchmark_data_storage0111-ib.pocstor_19103_object8488/head//24 by osd.396.0:1489 2014-08-13 15:29:03.383821 2014-08-13 18:06:06.853126 7f5c973a57a0 20 read_log 194416'15383 (194416'15382) modify 8b29a99/benchmark_data_storage0111-ib.pocstor_19103_object8488/head//24 by client.2441958.0:8489 2014-08-13 15:29:03.372840 2014-08-13 18:06:06.853138 7f5c973a57a0 20 read_log 194416'15384 (0'0) promote bd899699/benchmark_data_storage0111-ib.pocstor_19103_object9467/head//24 by osd.396.0:1493 2014-08-13 15:29:07.254506 2014-08-13 18:06:06.853169 7f5c973a57a0 20 read_log 194416'15385 (194416'15384) modify bd899699/benchmark_data_storage0111-ib.pocstor_19103_object9467/head//24 by client.2441958.0:9468 2014-08-13 15:29:07.245297 2014-08-13 18:06:06.853182 7f5c973a57a0 20 read_log 194416'15386 (0'0) promote f81cda99/benchmark_data_storage0111-ib.pocstor_19103_object10320/head//24 by osd.396.0:1505 2014-08-13 15:29:10.767106 2014-08-13 18:06:06.853194 7f5c973a57a0 20 read_log 194416'15387 (194416'15386) modify f81cda99/benchmark_data_storage0111-ib.pocstor_19103_object10320/head//24 by client.2441958.0:10321 2014-08-13 15:29:10.760119 2014-08-13 18:06:06.853206 7f5c973a57a0 20 read_log 194416'15388 (0'0) promote cd422299/benchmark_data_storage0111-ib.pocstor_19103_object13546/head//24 by osd.396.0:1517 2014-08-13 15:29:24.325197 2014-08-13 18:06:06.853218 7f5c973a57a0 20 read_log 194416'15389 (194416'15388) modify cd422299/benchmark_data_storage0111-ib.pocstor_19103_object13546/head//24 by client.2441958.0:13547 2014-08-13 15:29:24.314471 2014-08-13 18:06:06.853231 7f5c973a57a0 20 read_log 194416'15390 (0'0) promote e86a3299/benchmark_data_storage0111-ib.pocstor_19103_object15375/head//24 by osd.396.0:1529 2014-08-13 15:29:31.995192 2014-08-13 18:06:06.853243 7f5c973a57a0 20 read_log 194416'15391 (194416'15390) modify e86a3299/benchmark_data_storage0111-ib.pocstor_19103_object15375/head//24 by client.2441958.0:15376 2014-08-13 15:29:31.986210 2014-08-13 18:06:06.853261 7f5c973a57a0 20 read_log 194416'15392 (0'0) promote b4fee299/benchmark_data_storage0111-ib.pocstor_19103_object18117/head//24 by osd.396.0:1537 2014-08-13 15:30:03.118564 2014-08-13 18:06:06.853276 7f5c973a57a0 20 read_log 194416'15393 (194416'15392) modify b4fee299/benchmark_data_storage0111-ib.pocstor_19103_object18117/head//24 by client.2441958.0:18118 2014-08-13 15:30:03.107935 2014-08-13 18:06:06.853289 7f5c973a57a0 20 read_log 194422'15394 (0'0) promote 9c655e99/benchmark_data_storage0111-ib.pocstor_41231_object391/head//24 by osd.396.0:1543 2014-08-13 15:32:16.803859 2014-08-13 18:06:06.853301 7f5c973a57a0 20 read_log 194422'15395 (194422'15394) modify 9c655e99/benchmark_data_storage0111-ib.pocstor_41231_object391/head//24 by client.1148203.0:392 2014-08-13 15:32:16.794636 2014-08-13 18:06:06.853313 7f5c973a57a0 20 read_log 194422'15396 (0'0) promote 2d89ae99/benchmark_data_storage0111-ib.pocstor_41231_object1748/head//24 by osd.396.0:1553 2014-08-13 15:32:22.258762 2014-08-13 18:06:06.853325 7f5c973a57a0 20 read_log 194422'15397 (194422'15396) modify 2d89ae99/benchmark_data_storage0111-ib.pocstor_41231_object1748/head//24 by client.1148203.0:1749 2014-08-13 15:32:22.245855 2014-08-13 18:06:06.853365 7f5c973a57a0 20 read_log 194422'15398 (0'0) promote 8f21d299/benchmark_data_storage0111-ib.pocstor_41231_object2063/head//24 by osd.396.0:1555 2014-08-13 15:32:23.478200 2014-08-13 18:06:06.853387 7f5c973a57a0 20 read_log 194422'15399 (194422'15398) modify 8f21d299/benchmark_data_storage0111-ib.pocstor_41231_object2063/head//24 by client.1148203.0:2064 2014-08-13 15:32:23.469696 2014-08-13 18:06:06.853414 7f5c973a57a0 20 read_log 194422'15400 (0'0) promote a4adc699/benchmark_data_storage0111-ib.pocstor_41231_object4764/head//24 by osd.396.0:1559 2014-08-13 15:32:33.947276 2014-08-13 18:06:06.853429 7f5c973a57a0 20 read_log 194422'15401 (194422'15400) modify a4adc699/benchmark_data_storage0111-ib.pocstor_41231_object4764/head//24 by client.1148203.0:4765 2014-08-13 15:32:33.937347 2014-08-13 18:06:06.853454 7f5c973a57a0 20 read_log 194422'15402 (0'0) promote 9e665299/benchmark_data_storage0111-ib.pocstor_41231_object5122/head//24 by osd.396.0:1565 2014-08-13 15:32:35.439211 2014-08-13 18:06:06.853469 7f5c973a57a0 20 read_log 194422'15403 (194422'15402) modify 9e665299/benchmark_data_storage0111-ib.pocstor_41231_object5122/head//24 by client.1148203.0:5123 2014-08-13 15:32:35.430767 2014-08-13 18:06:06.853482 7f5c973a57a0 20 read_log 194422'15404 (0'0) promote 3ed83a99/benchmark_data_storage0111-ib.pocstor_41231_object7336/head//24 by osd.396.0:1573 2014-08-13 15:32:47.594955 2014-08-13 18:06:06.853494 7f5c973a57a0 20 read_log 194422'15405 (194422'15404) modify 3ed83a99/benchmark_data_storage0111-ib.pocstor_41231_object7336/head//24 by client.1148203.0:7337 2014-08-13 15:32:47.584290 2014-08-13 18:06:06.853507 7f5c973a57a0 20 read_log 194422'15406 (0'0) promote 24483299/benchmark_data_storage0111-ib.pocstor_41231_object8416/head//24 by osd.396.0:1579 2014-08-13 15:32:54.484376 2014-08-13 18:06:06.853519 7f5c973a57a0 20 read_log 194422'15407 (194422'15406) modify 24483299/benchmark_data_storage0111-ib.pocstor_41231_object8416/head//24 by client.1148203.0:8417 2014-08-13 15:32:54.475561 2014-08-13 18:06:06.853532 7f5c973a57a0 20 read_log 194422'15408 (0'0) promote aaac3299/benchmark_data_storage0111-ib.pocstor_41231_object8504/head//24 by osd.396.0:1581 2014-08-13 15:32:55.161577 2014-08-13 18:06:06.853544 7f5c973a57a0 20 read_log 194422'15409 (194422'15408) modify aaac3299/benchmark_data_storage0111-ib.pocstor_41231_object8504/head//24 by client.1148203.0:8505 2014-08-13 15:32:55.152028 2014-08-13 18:06:06.853556 7f5c973a57a0 20 read_log 194422'15410 (0'0) promote 1a22be99/benchmark_data_storage0111-ib.pocstor_41231_object8712/head//24 by osd.396.0:1585 2014-08-13 15:32:56.796657 2014-08-13 18:06:06.853568 7f5c973a57a0 20 read_log 194422'15411 (194422'15410) modify 1a22be99/benchmark_data_storage0111-ib.pocstor_41231_object8712/head//24 by client.1148203.0:8713 2014-08-13 15:32:56.788368 2014-08-13 18:06:06.853593 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.854006 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.854038 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.854057 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.013028 0 0.000000 2014-08-13 18:06:06.854070 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.854079 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.854089 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] log((192834'13573,194422'15411], crt=0'0) 2014-08-13 18:06:06.854100 7f5c973a57a0 10 osd.4 194487 pgid 24.2a1 coll 24.2a1_head 2014-08-13 18:06:06.854230 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.2a1 2014-08-13 18:06:06.854242 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.854288 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2a1(unlocked)] enter Initial 2014-08-13 18:06:06.854300 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.2a1(unlocked)] enter NotTrimming 2014-08-13 18:06:06.854476 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.854610 7f5c973a57a0 20 read_log 192834'12414 (0'0) promote 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.331.0:39406 2014-08-11 15:23:52.017585 2014-08-13 18:06:06.854630 7f5c973a57a0 20 read_log 192834'12415 (192834'12414) modify 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by client.933448.0:3488 2014-08-11 15:23:52.010687 2014-08-13 18:06:06.854643 7f5c973a57a0 20 read_log 192834'12446 (0'0) promote 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by osd.331.0:39468 2014-08-11 15:23:59.082691 2014-08-13 18:06:06.854656 7f5c973a57a0 20 read_log 192834'12447 (192834'12446) modify 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.933448.0:4859 2014-08-11 15:23:59.072933 2014-08-13 18:06:06.854669 7f5c973a57a0 20 read_log 192834'12454 (0'0) promote 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.331.0:39488 2014-08-11 15:24:00.677158 2014-08-13 18:06:06.854681 7f5c973a57a0 20 read_log 192834'12455 (192834'12454) modify 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by client.933448.0:5257 2014-08-11 15:24:00.669125 2014-08-13 18:06:06.854694 7f5c973a57a0 20 read_log 192834'12470 (0'0) promote 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by osd.331.0:39538 2014-08-11 15:24:04.449920 2014-08-13 18:06:06.854710 7f5c973a57a0 20 read_log 192834'12471 (192834'12470) modify 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.933448.0:6185 2014-08-11 15:24:04.440367 2014-08-13 18:06:06.854722 7f5c973a57a0 20 read_log 192834'12486 (0'0) promote 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by osd.331.0:39606 2014-08-11 15:24:10.641221 2014-08-13 18:06:06.854749 7f5c973a57a0 20 read_log 192834'12487 (192834'12486) modify 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.933448.0:7586 2014-08-11 15:24:10.634236 2014-08-13 18:06:06.854764 7f5c973a57a0 20 read_log 192834'12500 (0'0) promote 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by osd.331.0:39642 2014-08-11 15:24:14.214385 2014-08-13 18:06:06.854777 7f5c973a57a0 20 read_log 192834'12501 (192834'12500) modify 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.933448.0:8468 2014-08-11 15:24:14.206252 2014-08-13 18:06:06.854789 7f5c973a57a0 20 read_log 192834'12516 (0'0) promote 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by osd.331.0:39678 2014-08-11 15:24:18.394517 2014-08-13 18:06:06.854802 7f5c973a57a0 20 read_log 192834'12517 (192834'12516) modify 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.933448.0:9318 2014-08-11 15:24:18.386610 2014-08-13 18:06:06.854814 7f5c973a57a0 20 read_log 192834'12520 (0'0) promote ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by osd.331.0:39690 2014-08-11 15:24:19.031831 2014-08-13 18:06:06.854827 7f5c973a57a0 20 read_log 192834'12521 (192834'12520) modify ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.933448.0:9494 2014-08-11 15:24:19.023932 2014-08-13 18:06:06.854839 7f5c973a57a0 20 read_log 192834'12526 (0'0) promote 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by osd.331.0:39720 2014-08-11 15:24:22.703587 2014-08-13 18:06:06.854851 7f5c973a57a0 20 read_log 192834'12527 (192834'12526) modify 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.933448.0:10334 2014-08-11 15:24:22.694393 2014-08-13 18:06:06.854864 7f5c973a57a0 20 read_log 192834'12550 (0'0) promote 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by osd.331.0:39780 2014-08-11 15:24:27.073502 2014-08-13 18:06:06.854876 7f5c973a57a0 20 read_log 192834'12551 (192834'12550) modify 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.933448.0:11265 2014-08-11 15:24:27.065584 2014-08-13 18:06:06.854889 7f5c973a57a0 20 read_log 192834'12576 (0'0) promote c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by osd.331.0:39860 2014-08-11 15:24:34.127069 2014-08-13 18:06:06.854901 7f5c973a57a0 20 read_log 192834'12577 (192834'12576) modify c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.933448.0:12840 2014-08-11 15:24:34.118518 2014-08-13 18:06:06.854913 7f5c973a57a0 20 read_log 192834'12578 (0'0) promote 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by osd.331.0:39862 2014-08-11 15:24:34.211178 2014-08-13 18:06:06.854925 7f5c973a57a0 20 read_log 192834'12579 (192834'12578) modify 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.933448.0:12860 2014-08-11 15:24:34.203522 2014-08-13 18:06:06.854938 7f5c973a57a0 20 read_log 192834'12584 (0'0) promote 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.331.0:39874 2014-08-11 15:24:35.714231 2014-08-13 18:06:06.854950 7f5c973a57a0 20 read_log 192834'12585 (192834'12584) modify 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by client.933448.0:13170 2014-08-11 15:24:35.705231 2014-08-13 18:06:06.854963 7f5c973a57a0 20 read_log 192834'12586 (0'0) promote d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by osd.331.0:39876 2014-08-11 15:24:35.810678 2014-08-13 18:06:06.854975 7f5c973a57a0 20 read_log 192834'12587 (192834'12586) modify d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.933448.0:13195 2014-08-11 15:24:35.801548 2014-08-13 18:06:06.854987 7f5c973a57a0 20 read_log 192834'12606 (0'0) promote b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by osd.331.0:39931 2014-08-11 15:24:40.869111 2014-08-13 18:06:06.855007 7f5c973a57a0 20 read_log 192834'12607 (192834'12606) modify b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.933448.0:14030 2014-08-11 15:24:40.860659 2014-08-13 18:06:06.855021 7f5c973a57a0 20 read_log 192834'12610 (0'0) promote bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.331.0:39947 2014-08-11 15:24:41.958964 2014-08-13 18:06:06.855034 7f5c973a57a0 20 read_log 192834'12611 (192834'12610) modify bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by client.933448.0:14186 2014-08-11 15:24:41.950259 2014-08-13 18:06:06.855046 7f5c973a57a0 20 read_log 192834'12620 (0'0) promote f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by osd.331.0:40032 2014-08-11 15:24:49.011368 2014-08-13 18:06:06.855058 7f5c973a57a0 20 read_log 192834'12621 (192834'12620) modify f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.933448.0:15265 2014-08-11 15:24:49.003253 2014-08-13 18:06:06.855070 7f5c973a57a0 20 read_log 192834'12634 (0'0) promote d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by osd.331.0:40087 2014-08-11 15:24:56.177592 2014-08-13 18:06:06.855082 7f5c973a57a0 20 read_log 192834'12635 (192834'12634) modify d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.933448.0:16547 2014-08-11 15:24:56.169692 2014-08-13 18:06:06.855097 7f5c973a57a0 20 read_log 192834'12638 (0'0) promote 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by osd.331.0:40095 2014-08-11 15:24:56.517409 2014-08-13 18:06:06.855110 7f5c973a57a0 20 read_log 192834'12639 (192834'12638) modify 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.933448.0:16637 2014-08-11 15:24:56.508864 2014-08-13 18:06:06.855124 7f5c973a57a0 20 read_log 192834'12676 (0'0) promote eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by osd.331.0:40325 2014-08-11 15:25:16.765463 2014-08-13 18:06:06.855137 7f5c973a57a0 20 read_log 192834'12677 (192834'12676) modify eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.933448.0:20135 2014-08-11 15:25:16.756929 2014-08-13 18:06:06.855174 7f5c973a57a0 20 read_log 192834'12690 (0'0) promote 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by osd.331.0:40375 2014-08-11 15:25:21.222348 2014-08-13 18:06:06.855191 7f5c973a57a0 20 read_log 192834'12691 (192834'12690) modify 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.933448.0:20506 2014-08-11 15:25:21.213155 2014-08-13 18:06:06.855204 7f5c973a57a0 20 read_log 192834'12694 (0'0) promote c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.331.0:40412 2014-08-11 15:25:28.117708 2014-08-13 18:06:06.855218 7f5c973a57a0 20 read_log 192834'12695 (192834'12694) modify c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by client.933448.0:21246 2014-08-11 15:25:28.107590 2014-08-13 18:06:06.855238 7f5c973a57a0 20 read_log 192834'12704 (0'0) promote 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by osd.331.0:40468 2014-08-11 15:25:31.261997 2014-08-13 18:06:06.855254 7f5c973a57a0 20 read_log 192834'12705 (192834'12704) modify 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.933448.0:21622 2014-08-11 15:25:31.253902 2014-08-13 18:06:06.855267 7f5c973a57a0 20 read_log 192834'12750 (192834'12415) modify 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by client.933448.0:25779 2014-08-11 15:25:42.436494 2014-08-13 18:06:06.855282 7f5c973a57a0 20 read_log 192834'12766 (192834'12447) modify 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.933448.0:27150 2014-08-11 15:25:44.156238 2014-08-13 18:06:06.855295 7f5c973a57a0 20 read_log 192834'12770 (192834'12455) modify 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by client.933448.0:27548 2014-08-11 15:25:44.622989 2014-08-13 18:06:06.855342 7f5c973a57a0 20 read_log 192834'12778 (192834'12471) modify 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.933448.0:28476 2014-08-11 15:25:46.272787 2014-08-13 18:06:06.855359 7f5c973a57a0 20 read_log 192834'12786 (192834'12487) modify 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.933448.0:29877 2014-08-11 15:25:48.062553 2014-08-13 18:06:06.855372 7f5c973a57a0 20 read_log 192834'12793 (192834'12501) modify 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.933448.0:30759 2014-08-11 15:25:49.204087 2014-08-13 18:06:06.855389 7f5c973a57a0 20 read_log 192834'12801 (192834'12517) modify 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.933448.0:31609 2014-08-11 15:25:50.269438 2014-08-13 18:06:06.855402 7f5c973a57a0 20 read_log 192834'12803 (192834'12521) modify ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.933448.0:31785 2014-08-11 15:25:50.601978 2014-08-13 18:06:06.855414 7f5c973a57a0 20 read_log 192834'12806 (192834'12527) modify 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.933448.0:32625 2014-08-11 15:25:52.120864 2014-08-13 18:06:06.855426 7f5c973a57a0 20 read_log 192834'12818 (192834'12551) modify 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.933448.0:33556 2014-08-11 15:25:53.671213 2014-08-13 18:06:06.855439 7f5c973a57a0 20 read_log 192834'12831 (192834'12577) modify c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.933448.0:35131 2014-08-11 15:25:55.848803 2014-08-13 18:06:06.855451 7f5c973a57a0 20 read_log 192834'12832 (192834'12579) modify 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.933448.0:35151 2014-08-11 15:25:55.881915 2014-08-13 18:06:06.855466 7f5c973a57a0 20 read_log 192834'12835 (192834'12585) modify 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by client.933448.0:35461 2014-08-11 15:25:56.313517 2014-08-13 18:06:06.855492 7f5c973a57a0 20 read_log 192834'12836 (192834'12587) modify d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.933448.0:35486 2014-08-11 15:25:56.368485 2014-08-13 18:06:06.855505 7f5c973a57a0 20 read_log 192834'12846 (192834'12607) modify b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.933448.0:36321 2014-08-11 15:25:57.525740 2014-08-13 18:06:06.855524 7f5c973a57a0 20 read_log 192834'12848 (192834'12611) modify bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by client.933448.0:36477 2014-08-11 15:25:57.789503 2014-08-13 18:06:06.855540 7f5c973a57a0 20 read_log 192834'12853 (192834'12621) modify f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.933448.0:37556 2014-08-11 15:25:59.675508 2014-08-13 18:06:06.855553 7f5c973a57a0 20 read_log 192834'12860 (192834'12635) modify d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.933448.0:38838 2014-08-11 15:26:01.321465 2014-08-13 18:06:06.855572 7f5c973a57a0 20 read_log 192834'12862 (192834'12639) modify 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.933448.0:38928 2014-08-11 15:26:01.407197 2014-08-13 18:06:06.855589 7f5c973a57a0 20 read_log 192834'12881 (192834'12677) modify eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.933448.0:42426 2014-08-11 15:26:03.276578 2014-08-13 18:06:06.855602 7f5c973a57a0 20 read_log 192834'12888 (192834'12691) modify 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.933448.0:42797 2014-08-11 15:26:03.390498 2014-08-13 18:06:06.855614 7f5c973a57a0 20 read_log 192834'12890 (192834'12695) modify c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by client.933448.0:43537 2014-08-11 15:26:03.652556 2014-08-13 18:06:06.855627 7f5c973a57a0 20 read_log 192834'12895 (192834'12705) modify 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.933448.0:43913 2014-08-11 15:26:03.760571 2014-08-13 18:06:06.855646 7f5c973a57a0 20 read_log 192834'12915 (0'0) promote 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by osd.331.0:41302 2014-08-11 15:26:16.640917 2014-08-13 18:06:06.855662 7f5c973a57a0 20 read_log 192834'12916 (192834'12915) modify 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.1883634.0:589 2014-08-11 15:26:16.633316 2014-08-13 18:06:06.855674 7f5c973a57a0 20 read_log 192834'12919 (0'0) promote 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by osd.331.0:41334 2014-08-11 15:26:18.920549 2014-08-13 18:06:06.855687 7f5c973a57a0 20 read_log 192834'12920 (192834'12919) modify 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.1883634.0:1219 2014-08-11 15:26:18.912827 2014-08-13 18:06:06.855699 7f5c973a57a0 20 read_log 192834'12923 (0'0) promote 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by osd.331.0:41340 2014-08-11 15:26:19.356170 2014-08-13 18:06:06.855737 7f5c973a57a0 20 read_log 192834'12924 (192834'12923) modify 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.1883634.0:1339 2014-08-11 15:26:19.348176 2014-08-13 18:06:06.855753 7f5c973a57a0 20 read_log 192834'12939 (0'0) promote 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.331.0:41382 2014-08-11 15:26:22.077844 2014-08-13 18:06:06.855766 7f5c973a57a0 20 read_log 192834'12940 (192834'12939) modify 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by client.1883634.0:2058 2014-08-11 15:26:22.068822 2014-08-13 18:06:06.855779 7f5c973a57a0 20 read_log 192834'12975 (0'0) promote 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by osd.331.0:41518 2014-08-11 15:26:32.163631 2014-08-13 18:06:06.855791 7f5c973a57a0 20 read_log 192834'12976 (192834'12975) modify 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.1883634.0:4838 2014-08-11 15:26:32.154695 2014-08-13 18:06:06.855803 7f5c973a57a0 20 read_log 192834'12985 (0'0) promote fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.331.0:41534 2014-08-11 15:26:33.858737 2014-08-13 18:06:06.855815 7f5c973a57a0 20 read_log 192834'12986 (192834'12985) modify fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by client.1883634.0:5283 2014-08-11 15:26:33.850962 2014-08-13 18:06:06.855828 7f5c973a57a0 20 read_log 192834'12993 (0'0) promote 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by osd.331.0:41550 2014-08-11 15:26:34.483720 2014-08-13 18:06:06.855848 7f5c973a57a0 20 read_log 192834'12994 (192834'12993) modify 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.1883634.0:5461 2014-08-11 15:26:34.477407 2014-08-13 18:06:06.855864 7f5c973a57a0 20 read_log 192834'13001 (0'0) promote 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by osd.331.0:41566 2014-08-11 15:26:36.244507 2014-08-13 18:06:06.855877 7f5c973a57a0 20 read_log 192834'13002 (192834'13001) modify 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.1883634.0:5953 2014-08-11 15:26:36.235454 2014-08-13 18:06:06.855890 7f5c973a57a0 20 read_log 192834'13005 (0'0) promote 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by osd.331.0:41578 2014-08-11 15:26:36.910690 2014-08-13 18:06:06.855902 7f5c973a57a0 20 read_log 192834'13006 (192834'13005) modify 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.1883634.0:6140 2014-08-11 15:26:36.902510 2014-08-13 18:06:06.855914 7f5c973a57a0 20 read_log 192834'13013 (0'0) promote 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.331.0:41615 2014-08-11 15:26:39.365989 2014-08-13 18:06:06.855932 7f5c973a57a0 20 read_log 192834'13014 (192834'13013) modify 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by client.1883634.0:6746 2014-08-11 15:26:39.357196 2014-08-13 18:06:06.855952 7f5c973a57a0 20 read_log 192834'13015 (0'0) promote baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by osd.331.0:41621 2014-08-11 15:26:39.510893 2014-08-13 18:06:06.855967 7f5c973a57a0 20 read_log 192834'13016 (192834'13015) modify baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.1883634.0:6788 2014-08-11 15:26:39.501640 2014-08-13 18:06:06.855980 7f5c973a57a0 20 read_log 192834'13023 (0'0) promote 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.331.0:41647 2014-08-11 15:26:41.572610 2014-08-13 18:06:06.855992 7f5c973a57a0 20 read_log 192834'13024 (192834'13023) modify 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by client.1883634.0:7300 2014-08-11 15:26:41.564159 2014-08-13 18:06:06.856004 7f5c973a57a0 20 read_log 192834'13049 (0'0) promote 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by osd.331.0:41790 2014-08-11 15:26:47.159627 2014-08-13 18:06:06.856016 7f5c973a57a0 20 read_log 192834'13050 (192834'13049) modify 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.1883634.0:8627 2014-08-11 15:26:47.152674 2014-08-13 18:06:06.856028 7f5c973a57a0 20 read_log 192834'13063 (0'0) promote 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by osd.331.0:41863 2014-08-11 15:26:51.091448 2014-08-13 18:06:06.856041 7f5c973a57a0 20 read_log 192834'13064 (192834'13063) modify 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.1883634.0:9503 2014-08-11 15:26:51.083619 2014-08-13 18:06:06.856053 7f5c973a57a0 20 read_log 192834'13067 (0'0) promote e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by osd.331.0:41877 2014-08-11 15:26:52.275236 2014-08-13 18:06:06.856065 7f5c973a57a0 20 read_log 192834'13068 (192834'13067) modify e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.1883634.0:9760 2014-08-11 15:26:52.264486 2014-08-13 18:06:06.856077 7f5c973a57a0 20 read_log 192834'13079 (0'0) promote 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.331.0:41912 2014-08-11 15:26:54.859496 2014-08-13 18:06:06.856089 7f5c973a57a0 20 read_log 192834'13080 (192834'13079) modify 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by client.1883634.0:10380 2014-08-11 15:26:54.847812 2014-08-13 18:06:06.856101 7f5c973a57a0 20 read_log 192834'13083 (0'0) promote a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.331.0:41959 2014-08-11 15:26:58.104026 2014-08-13 18:06:06.856122 7f5c973a57a0 20 read_log 192834'13084 (192834'13083) modify a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by client.1883634.0:11048 2014-08-11 15:26:58.095225 2014-08-13 18:06:06.856135 7f5c973a57a0 20 read_log 192834'13105 (0'0) promote 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.331.0:42038 2014-08-11 15:27:01.711486 2014-08-13 18:06:06.856167 7f5c973a57a0 20 read_log 192834'13106 (192834'13105) modify 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by client.1883634.0:11916 2014-08-11 15:27:01.701462 2014-08-13 18:06:06.856181 7f5c973a57a0 20 read_log 192834'13111 (0'0) promote 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.331.0:42050 2014-08-11 15:27:02.757782 2014-08-13 18:06:06.856203 7f5c973a57a0 20 read_log 192834'13112 (192834'13111) modify 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by client.1883634.0:12157 2014-08-11 15:27:02.747501 2014-08-13 18:06:06.856220 7f5c973a57a0 20 read_log 192834'13119 (0'0) promote 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by osd.331.0:42068 2014-08-11 15:27:04.163667 2014-08-13 18:06:06.856241 7f5c973a57a0 20 read_log 192834'13120 (192834'13119) modify 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.1883634.0:12471 2014-08-11 15:27:04.154621 2014-08-13 18:06:06.856265 7f5c973a57a0 20 read_log 192834'13125 (0'0) promote eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.331.0:42269 2014-08-11 15:27:07.271244 2014-08-13 18:06:06.856281 7f5c973a57a0 20 read_log 192834'13126 (192834'13125) modify eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by client.1883634.0:13026 2014-08-11 15:27:07.262935 2014-08-13 18:06:06.856310 7f5c973a57a0 20 read_log 192834'13129 (0'0) promote 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by osd.331.0:42283 2014-08-11 15:27:08.401680 2014-08-13 18:06:06.856323 7f5c973a57a0 20 read_log 192834'13130 (192834'13129) modify 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.1883634.0:13283 2014-08-11 15:27:08.393268 2014-08-13 18:06:06.856336 7f5c973a57a0 20 read_log 192834'13133 (0'0) promote a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.331.0:42287 2014-08-11 15:27:09.817956 2014-08-13 18:06:06.856351 7f5c973a57a0 20 read_log 192834'13134 (192834'13133) modify a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by client.1883634.0:13476 2014-08-11 15:27:09.809735 2014-08-13 18:06:06.856375 7f5c973a57a0 20 read_log 192834'13157 (0'0) promote 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by osd.331.0:42369 2014-08-11 15:27:15.570848 2014-08-13 18:06:06.856388 7f5c973a57a0 20 read_log 192834'13158 (192834'13157) modify 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.1883634.0:14830 2014-08-11 15:27:15.561042 2014-08-13 18:06:06.856401 7f5c973a57a0 20 read_log 192834'13189 (0'0) promote c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.331.0:42546 2014-08-11 15:27:33.108829 2014-08-13 18:06:06.856413 7f5c973a57a0 20 read_log 192834'13190 (192834'13189) modify c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by client.1883634.0:17201 2014-08-11 15:27:33.098580 2014-08-13 18:06:06.856426 7f5c973a57a0 20 read_log 192834'13201 (0'0) promote 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by osd.331.0:42627 2014-08-11 15:27:41.581485 2014-08-13 18:06:06.856438 7f5c973a57a0 20 read_log 192834'13202 (192834'13201) modify 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.1883634.0:18142 2014-08-11 15:27:41.570677 2014-08-13 18:06:06.856450 7f5c973a57a0 20 read_log 192834'13205 (0'0) promote cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by osd.331.0:42666 2014-08-11 15:27:44.506514 2014-08-13 18:06:06.856463 7f5c973a57a0 20 read_log 192834'13206 (192834'13205) modify cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.1883634.0:18441 2014-08-11 15:27:44.484629 2014-08-13 18:06:06.856475 7f5c973a57a0 20 read_log 192834'13221 (0'0) promote c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.331.0:42748 2014-08-11 15:27:53.848544 2014-08-13 18:06:06.856487 7f5c973a57a0 20 read_log 192834'13222 (192834'13221) modify c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by client.1883634.0:19696 2014-08-11 15:27:53.838623 2014-08-13 18:06:06.856500 7f5c973a57a0 20 read_log 192834'13235 (0'0) promote 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.331.0:42840 2014-08-11 15:28:03.305542 2014-08-13 18:06:06.856512 7f5c973a57a0 20 read_log 192834'13236 (192834'13235) modify 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by client.1883634.0:21194 2014-08-11 15:28:03.295126 2014-08-13 18:06:06.856524 7f5c973a57a0 20 read_log 192834'13247 (0'0) promote cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by osd.331.0:42913 2014-08-11 15:28:06.166524 2014-08-13 18:06:06.856537 7f5c973a57a0 20 read_log 192834'13248 (192834'13247) modify cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.1883634.0:22001 2014-08-11 15:28:06.156066 2014-08-13 18:06:06.856564 7f5c973a57a0 20 read_log 192834'13278 (192834'12916) modify 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.1883634.0:23913 2014-08-11 15:28:18.075392 2014-08-13 18:06:06.856582 7f5c973a57a0 20 read_log 192834'13280 (192834'12920) modify 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.1883634.0:24543 2014-08-11 15:28:19.216910 2014-08-13 18:06:06.856595 7f5c973a57a0 20 read_log 192834'13282 (192834'12924) modify 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.1883634.0:24663 2014-08-11 15:28:19.338944 2014-08-13 18:06:06.856607 7f5c973a57a0 20 read_log 192834'13290 (192834'12940) modify 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by client.1883634.0:25382 2014-08-11 15:28:21.026750 2014-08-13 18:06:06.856620 7f5c973a57a0 20 read_log 192834'13308 (192834'12976) modify 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.1883634.0:28162 2014-08-11 15:28:25.456696 2014-08-13 18:06:06.856632 7f5c973a57a0 20 read_log 192834'13313 (192834'12986) modify fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by client.1883634.0:28607 2014-08-11 15:28:26.029388 2014-08-13 18:06:06.856645 7f5c973a57a0 20 read_log 192834'13317 (192834'12994) modify 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.1883634.0:28785 2014-08-11 15:28:26.287810 2014-08-13 18:06:06.856657 7f5c973a57a0 20 read_log 192834'13321 (192834'13002) modify 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.1883634.0:29277 2014-08-11 15:28:26.922283 2014-08-13 18:06:06.856670 7f5c973a57a0 20 read_log 192834'13323 (192834'13006) modify 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.1883634.0:29464 2014-08-11 15:28:27.316442 2014-08-13 18:06:06.856682 7f5c973a57a0 20 read_log 192834'13327 (192834'13014) modify 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by client.1883634.0:30070 2014-08-11 15:28:28.055881 2014-08-13 18:06:06.856694 7f5c973a57a0 20 read_log 192834'13328 (192834'13016) modify baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.1883634.0:30112 2014-08-11 15:28:28.118618 2014-08-13 18:06:06.856707 7f5c973a57a0 20 read_log 192834'13332 (192834'13024) modify 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by client.1883634.0:30624 2014-08-11 15:28:28.995063 2014-08-13 18:06:06.856738 7f5c973a57a0 20 read_log 192834'13345 (192834'13050) modify 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.1883634.0:31951 2014-08-11 15:28:31.162872 2014-08-13 18:06:06.856755 7f5c973a57a0 20 read_log 192834'13352 (192834'13064) modify 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.1883634.0:32827 2014-08-11 15:28:32.375594 2014-08-13 18:06:06.856768 7f5c973a57a0 20 read_log 192834'13354 (192834'13068) modify e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.1883634.0:33084 2014-08-11 15:28:32.760614 2014-08-13 18:06:06.856782 7f5c973a57a0 20 read_log 192834'13360 (192834'13080) modify 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by client.1883634.0:33704 2014-08-11 15:28:33.685374 2014-08-13 18:06:06.856795 7f5c973a57a0 20 read_log 192834'13364 (192834'13084) modify a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by client.1883634.0:34372 2014-08-11 15:28:34.603383 2014-08-13 18:06:06.856810 7f5c973a57a0 20 read_log 192834'13375 (192834'13106) modify 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by client.1883634.0:35240 2014-08-11 15:28:36.316157 2014-08-13 18:06:06.856822 7f5c973a57a0 20 read_log 192834'13378 (192834'13112) modify 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by client.1883634.0:35481 2014-08-11 15:28:36.709793 2014-08-13 18:06:06.856835 7f5c973a57a0 20 read_log 192834'13382 (192834'13120) modify 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.1883634.0:35795 2014-08-11 15:28:37.134655 2014-08-13 18:06:06.856855 7f5c973a57a0 20 read_log 192834'13385 (192834'13126) modify eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by client.1883634.0:36350 2014-08-11 15:28:37.916584 2014-08-13 18:06:06.856870 7f5c973a57a0 20 read_log 192834'13387 (192834'13130) modify 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.1883634.0:36607 2014-08-11 15:28:38.210715 2014-08-13 18:06:06.856882 7f5c973a57a0 20 read_log 192834'13389 (192834'13134) modify a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by client.1883634.0:36800 2014-08-11 15:28:38.423530 2014-08-13 18:06:06.856897 7f5c973a57a0 20 read_log 192834'13401 (192834'13158) modify 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.1883634.0:38154 2014-08-11 15:28:40.292568 2014-08-13 18:06:06.856909 7f5c973a57a0 20 read_log 192834'13417 (192834'13190) modify c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by client.1883634.0:40525 2014-08-11 15:28:43.874140 2014-08-13 18:06:06.856923 7f5c973a57a0 20 read_log 192834'13423 (192834'13202) modify 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.1883634.0:41466 2014-08-11 15:28:45.155021 2014-08-13 18:06:06.856935 7f5c973a57a0 20 read_log 192834'13425 (192834'13206) modify cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.1883634.0:41765 2014-08-11 15:28:45.474134 2014-08-13 18:06:06.856950 7f5c973a57a0 20 read_log 192834'13433 (192834'13222) modify c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by client.1883634.0:43020 2014-08-11 15:28:46.433949 2014-08-13 18:06:06.856965 7f5c973a57a0 20 read_log 192834'13440 (192834'13236) modify 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by client.1883634.0:44518 2014-08-11 15:28:46.879826 2014-08-13 18:06:06.856980 7f5c973a57a0 20 read_log 192834'13446 (192834'13248) modify cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.1883634.0:45325 2014-08-11 15:28:47.241326 2014-08-13 18:06:06.856992 7f5c973a57a0 20 read_log 192888'13460 (0'0) promote 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.287.0:16787 2014-08-11 16:44:55.255982 2014-08-13 18:06:06.857007 7f5c973a57a0 20 read_log 192888'13461 (192888'13460) modify 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by client.1315259.0:79 2014-08-11 16:44:55.243805 2014-08-13 18:06:06.857020 7f5c973a57a0 20 read_log 192888'13462 (0'0) promote bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.287.0:16792 2014-08-11 16:44:55.605389 2014-08-13 18:06:06.857034 7f5c973a57a0 20 read_log 192888'13463 (192888'13462) modify bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by client.1315259.0:169 2014-08-11 16:44:55.599610 2014-08-13 18:06:06.857047 7f5c973a57a0 20 read_log 192888'13470 (0'0) promote 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.287.0:16820 2014-08-11 16:45:06.117232 2014-08-13 18:06:06.857061 7f5c973a57a0 20 read_log 192888'13471 (192888'13470) modify 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by client.1315259.0:2874 2014-08-11 16:45:06.110086 2014-08-13 18:06:06.857074 7f5c973a57a0 20 read_log 192888'13474 (0'0) promote 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by osd.287.0:16844 2014-08-11 16:45:12.132645 2014-08-13 18:06:06.857088 7f5c973a57a0 20 read_log 192888'13475 (192888'13474) modify 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.1315259.0:4460 2014-08-11 16:45:12.125046 2014-08-13 18:06:06.857120 7f5c973a57a0 20 read_log 192888'13484 (0'0) promote 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.287.0:16874 2014-08-11 16:45:19.100200 2014-08-13 18:06:06.857149 7f5c973a57a0 20 read_log 192888'13485 (192888'13484) modify 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by client.1315259.0:6365 2014-08-11 16:45:19.093714 2014-08-13 18:06:06.857166 7f5c973a57a0 20 read_log 192888'13486 (0'0) promote e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by osd.287.0:16876 2014-08-11 16:45:19.213470 2014-08-13 18:06:06.857179 7f5c973a57a0 20 read_log 192888'13487 (192888'13486) modify e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.1315259.0:6396 2014-08-11 16:45:19.206766 2014-08-13 18:06:06.857191 7f5c973a57a0 20 read_log 192888'13490 (0'0) promote 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by osd.287.0:16898 2014-08-11 16:45:24.233967 2014-08-13 18:06:06.857203 7f5c973a57a0 20 read_log 192888'13491 (192888'13490) modify 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.1315259.0:7779 2014-08-11 16:45:24.225363 2014-08-13 18:06:06.857216 7f5c973a57a0 20 read_log 192888'13498 (0'0) promote 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.287.0:16914 2014-08-11 16:45:26.987971 2014-08-13 18:06:06.857230 7f5c973a57a0 20 read_log 192888'13499 (192888'13498) modify 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by client.1315259.0:8508 2014-08-11 16:45:26.981030 2014-08-13 18:06:06.857243 7f5c973a57a0 20 read_log 192888'13500 (0'0) promote 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by osd.287.0:16918 2014-08-11 16:45:27.720332 2014-08-13 18:06:06.857260 7f5c973a57a0 20 read_log 192888'13501 (192888'13500) modify 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.1315259.0:8707 2014-08-11 16:45:27.710262 2014-08-13 18:06:06.857274 7f5c973a57a0 20 read_log 192888'13502 (0'0) promote 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by osd.287.0:16920 2014-08-11 16:45:28.702369 2014-08-13 18:06:06.857287 7f5c973a57a0 20 read_log 192888'13503 (192888'13502) modify 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.1315259.0:8954 2014-08-11 16:45:28.695832 2014-08-13 18:06:06.857299 7f5c973a57a0 20 read_log 192888'13504 (0'0) promote e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by osd.287.0:16928 2014-08-11 16:45:29.995690 2014-08-13 18:06:06.857315 7f5c973a57a0 20 read_log 192888'13505 (192888'13504) modify e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.1315259.0:9315 2014-08-11 16:45:29.988327 2014-08-13 18:06:06.857327 7f5c973a57a0 20 read_log 192888'13506 (0'0) promote 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by osd.287.0:16946 2014-08-11 16:45:32.911130 2014-08-13 18:06:06.857340 7f5c973a57a0 20 read_log 192888'13507 (192888'13506) modify 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.1315259.0:10070 2014-08-11 16:45:32.903677 2014-08-13 18:06:06.857352 7f5c973a57a0 20 read_log 192888'13512 (0'0) promote 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by osd.287.0:16960 2014-08-11 16:45:36.591542 2014-08-13 18:06:06.857369 7f5c973a57a0 20 read_log 192888'13513 (192888'13512) modify 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.1315259.0:10994 2014-08-11 16:45:36.584923 2014-08-13 18:06:06.857382 7f5c973a57a0 20 read_log 192888'13516 (0'0) promote d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.287.0:16970 2014-08-11 16:45:41.043616 2014-08-13 18:06:06.857394 7f5c973a57a0 20 read_log 192888'13517 (192888'13516) modify d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by client.1315259.0:12092 2014-08-11 16:45:41.036955 2014-08-13 18:06:06.857410 7f5c973a57a0 20 read_log 192888'13518 (0'0) promote 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.287.0:16976 2014-08-11 16:45:42.579874 2014-08-13 18:06:06.857445 7f5c973a57a0 20 read_log 192888'13519 (192888'13518) modify 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by client.1315259.0:12452 2014-08-11 16:45:42.573518 2014-08-13 18:06:06.857461 7f5c973a57a0 20 read_log 192888'13522 (0'0) promote dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by osd.287.0:16988 2014-08-11 16:45:44.314036 2014-08-13 18:06:06.857490 7f5c973a57a0 20 read_log 192888'13523 (192888'13522) modify dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.1315259.0:12873 2014-08-11 16:45:44.307661 2014-08-13 18:06:06.857503 7f5c973a57a0 20 read_log 192888'13532 (0'0) promote 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by osd.287.0:17018 2014-08-11 16:45:56.409708 2014-08-13 18:06:06.857522 7f5c973a57a0 20 read_log 192888'13533 (192888'13532) modify 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.1315259.0:15995 2014-08-11 16:45:56.402496 2014-08-13 18:06:06.857538 7f5c973a57a0 20 read_log 192888'13538 (0'0) promote 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by osd.287.0:17028 2014-08-11 16:45:57.217483 2014-08-13 18:06:06.857551 7f5c973a57a0 20 read_log 192888'13539 (192888'13538) modify 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.1315259.0:16209 2014-08-11 16:45:57.209951 2014-08-13 18:06:06.857572 7f5c973a57a0 20 read_log 192888'13544 (0'0) promote 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by osd.287.0:17042 2014-08-11 16:45:59.211054 2014-08-13 18:06:06.857592 7f5c973a57a0 20 read_log 192888'13545 (192888'13544) modify 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.1315259.0:16711 2014-08-11 16:45:59.201578 2014-08-13 18:06:06.857608 7f5c973a57a0 20 read_log 192888'13546 (0'0) promote 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.287.0:17044 2014-08-11 16:45:59.825739 2014-08-13 18:06:06.857630 7f5c973a57a0 20 read_log 192888'13547 (192888'13546) modify 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by client.1315259.0:16876 2014-08-11 16:45:59.818541 2014-08-13 18:06:06.857666 7f5c973a57a0 20 read_log 192888'13548 (0'0) promote 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.287.0:17048 2014-08-11 16:46:00.911426 2014-08-13 18:06:06.857679 7f5c973a57a0 20 read_log 192888'13549 (192888'13548) modify 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by client.1315259.0:17143 2014-08-11 16:46:00.904447 2014-08-13 18:06:06.857691 7f5c973a57a0 20 read_log 192888'13550 (0'0) promote e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.287.0:17052 2014-08-11 16:46:01.833871 2014-08-13 18:06:06.857704 7f5c973a57a0 20 read_log 192888'13551 (192888'13550) modify e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by client.1315259.0:17376 2014-08-11 16:46:01.826683 2014-08-13 18:06:06.857716 7f5c973a57a0 20 read_log 192888'13552 (0'0) promote 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by osd.287.0:17072 2014-08-11 16:46:06.224722 2014-08-13 18:06:06.857728 7f5c973a57a0 20 read_log 192888'13553 (192888'13552) modify 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.1315259.0:18484 2014-08-11 16:46:06.217348 2014-08-13 18:06:06.857741 7f5c973a57a0 20 read_log 192888'13554 (0'0) promote a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by osd.287.0:17076 2014-08-11 16:46:07.397436 2014-08-13 18:06:06.857753 7f5c973a57a0 20 read_log 192888'13555 (192888'13554) modify a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.1315259.0:18796 2014-08-11 16:46:07.390571 2014-08-13 18:06:06.857765 7f5c973a57a0 20 read_log 192888'13556 (0'0) promote 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by osd.287.0:17078 2014-08-11 16:46:07.413033 2014-08-13 18:06:06.857785 7f5c973a57a0 20 read_log 192888'13557 (192888'13556) modify 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.1315259.0:18800 2014-08-11 16:46:07.405515 2014-08-13 18:06:06.857801 7f5c973a57a0 20 read_log 192888'13560 (0'0) promote 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by osd.287.0:17116 2014-08-11 16:46:19.997056 2014-08-13 18:06:06.857813 7f5c973a57a0 20 read_log 192888'13561 (192888'13560) modify 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.1315259.0:22081 2014-08-11 16:46:19.988290 2014-08-13 18:06:06.857826 7f5c973a57a0 20 read_log 192888'13566 (0'0) promote 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by osd.287.0:17128 2014-08-11 16:46:22.877814 2014-08-13 18:06:06.857838 7f5c973a57a0 20 read_log 192888'13567 (192888'13566) modify 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.1315259.0:22822 2014-08-11 16:46:22.869879 2014-08-13 18:06:06.857850 7f5c973a57a0 20 read_log 192888'13570 (0'0) promote 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by osd.287.0:17132 2014-08-11 16:46:23.310758 2014-08-13 18:06:06.857862 7f5c973a57a0 20 read_log 192888'13571 (192888'13570) modify 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.1315259.0:22922 2014-08-11 16:46:23.301617 2014-08-13 18:06:06.857874 7f5c973a57a0 20 read_log 192888'13572 (0'0) promote b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by osd.287.0:17136 2014-08-11 16:46:24.961749 2014-08-13 18:06:06.857886 7f5c973a57a0 20 read_log 192888'13573 (192888'13572) modify b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.1315259.0:23332 2014-08-11 16:46:24.954930 2014-08-13 18:06:06.857898 7f5c973a57a0 20 read_log 192888'13578 (0'0) promote 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.287.0:17172 2014-08-11 16:46:31.831699 2014-08-13 18:06:06.857910 7f5c973a57a0 20 read_log 192888'13579 (192888'13578) modify 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by client.1315259.0:25087 2014-08-11 16:46:31.824579 2014-08-13 18:06:06.857923 7f5c973a57a0 20 read_log 192888'13590 (0'0) promote 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.287.0:17194 2014-08-11 16:46:36.944757 2014-08-13 18:06:06.857935 7f5c973a57a0 20 read_log 192888'13591 (192888'13590) modify 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by client.1315259.0:26287 2014-08-11 16:46:36.938580 2014-08-13 18:06:06.857947 7f5c973a57a0 20 read_log 192888'13592 (0'0) promote 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by osd.287.0:17206 2014-08-11 16:46:37.939135 2014-08-13 18:06:06.857962 7f5c973a57a0 20 read_log 192888'13593 (192888'13592) modify 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.1315259.0:26512 2014-08-11 16:46:37.931763 2014-08-13 18:06:06.857974 7f5c973a57a0 20 read_log 192888'13594 (0'0) promote 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by osd.287.0:17232 2014-08-11 16:46:46.958687 2014-08-13 18:06:06.857988 7f5c973a57a0 20 read_log 192888'13595 (192888'13594) modify 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.1315259.0:28721 2014-08-11 16:46:46.951706 2014-08-13 18:06:06.858001 7f5c973a57a0 20 read_log 192888'13598 (0'0) promote 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by osd.287.0:17238 2014-08-11 16:46:48.291207 2014-08-13 18:06:06.858015 7f5c973a57a0 20 read_log 192888'13599 (192888'13598) modify 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.1315259.0:29071 2014-08-11 16:46:48.284221 2014-08-13 18:06:06.858028 7f5c973a57a0 20 read_log 192888'13600 (192888'13461) modify 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by client.1315259.0:30806 2014-08-11 16:46:55.896294 2014-08-13 18:06:06.858050 7f5c973a57a0 20 read_log 192888'13601 (192888'13463) modify bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by client.1315259.0:30896 2014-08-11 16:46:55.929479 2014-08-13 18:06:06.858065 7f5c973a57a0 20 read_log 192888'13605 (192888'13471) modify 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by client.1315259.0:33601 2014-08-11 16:46:56.677711 2014-08-13 18:06:06.858078 7f5c973a57a0 20 read_log 192888'13607 (192888'13475) modify 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.1315259.0:35187 2014-08-11 16:46:57.095456 2014-08-13 18:06:06.858093 7f5c973a57a0 20 read_log 192888'13612 (192888'13485) modify 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by client.1315259.0:37092 2014-08-11 16:46:57.591969 2014-08-13 18:06:06.858106 7f5c973a57a0 20 read_log 192888'13613 (192888'13487) modify e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.1315259.0:37123 2014-08-11 16:46:57.601056 2014-08-13 18:06:06.858121 7f5c973a57a0 20 read_log 192888'13615 (192888'13491) modify 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.1315259.0:38506 2014-08-11 16:46:57.968956 2014-08-13 18:06:06.858133 7f5c973a57a0 20 read_log 192888'13619 (192888'13499) modify 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by client.1315259.0:39235 2014-08-11 16:46:58.159067 2014-08-13 18:06:06.858163 7f5c973a57a0 20 read_log 192888'13620 (192888'13501) modify 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.1315259.0:39434 2014-08-11 16:46:58.210918 2014-08-13 18:06:06.858180 7f5c973a57a0 20 read_log 192888'13621 (192888'13503) modify 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.1315259.0:39681 2014-08-11 16:46:58.276248 2014-08-13 18:06:06.858193 7f5c973a57a0 20 read_log 192888'13622 (192888'13505) modify e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.1315259.0:40042 2014-08-11 16:46:58.372316 2014-08-13 18:06:06.858207 7f5c973a57a0 20 read_log 192888'13623 (192888'13507) modify 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.1315259.0:40797 2014-08-11 16:46:58.583295 2014-08-13 18:06:06.858227 7f5c973a57a0 20 read_log 192888'13626 (192888'13513) modify 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.1315259.0:41721 2014-08-11 16:46:58.819623 2014-08-13 18:06:06.858243 7f5c973a57a0 20 read_log 192888'13628 (192888'13517) modify d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by client.1315259.0:42819 2014-08-11 16:46:59.114997 2014-08-13 18:06:06.858255 7f5c973a57a0 20 read_log 192888'13629 (192888'13519) modify 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by client.1315259.0:43179 2014-08-11 16:46:59.215917 2014-08-13 18:06:06.858270 7f5c973a57a0 20 read_log 192888'13631 (192888'13523) modify dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.1315259.0:43600 2014-08-11 16:46:59.336123 2014-08-13 18:06:06.858282 7f5c973a57a0 20 read_log 192888'13636 (192888'13533) modify 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.1315259.0:46722 2014-08-11 16:47:00.239881 2014-08-13 18:06:06.858297 7f5c973a57a0 20 read_log 192888'13639 (192888'13539) modify 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.1315259.0:46936 2014-08-11 16:47:00.300911 2014-08-13 18:06:06.858314 7f5c973a57a0 20 read_log 192888'13642 (192888'13545) modify 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.1315259.0:47438 2014-08-11 16:47:00.455528 2014-08-13 18:06:06.858327 7f5c973a57a0 20 read_log 192888'13643 (192888'13547) modify 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by client.1315259.0:47603 2014-08-11 16:47:00.506762 2014-08-13 18:06:06.858339 7f5c973a57a0 20 read_log 192888'13644 (192888'13549) modify 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by client.1315259.0:47870 2014-08-11 16:47:00.593206 2014-08-13 18:06:06.858359 7f5c973a57a0 20 read_log 192888'13645 (192888'13551) modify e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by client.1315259.0:48103 2014-08-11 16:47:00.671352 2014-08-13 18:06:06.858372 7f5c973a57a0 20 read_log 192888'13646 (192888'13553) modify 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.1315259.0:49211 2014-08-11 16:47:00.991196 2014-08-13 18:06:06.858384 7f5c973a57a0 20 read_log 192888'13647 (192888'13555) modify a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.1315259.0:49523 2014-08-11 16:47:01.088888 2014-08-13 18:06:06.858399 7f5c973a57a0 20 read_log 192888'13648 (192888'13557) modify 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.1315259.0:49527 2014-08-11 16:47:01.091513 2014-08-13 18:06:06.858413 7f5c973a57a0 20 read_log 192888'13650 (192888'13561) modify 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.1315259.0:52808 2014-08-11 16:47:02.075189 2014-08-13 18:06:06.858427 7f5c973a57a0 20 read_log 192888'13653 (192888'13567) modify 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.1315259.0:53549 2014-08-11 16:47:02.288464 2014-08-13 18:06:06.858440 7f5c973a57a0 20 read_log 192888'13655 (192888'13571) modify 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.1315259.0:53649 2014-08-11 16:47:02.317430 2014-08-13 18:06:06.858452 7f5c973a57a0 20 read_log 192888'13656 (192888'13573) modify b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.1315259.0:54059 2014-08-11 16:47:02.447962 2014-08-13 18:06:06.858464 7f5c973a57a0 20 read_log 192888'13659 (192888'13579) modify 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by client.1315259.0:55814 2014-08-11 16:47:03.023592 2014-08-13 18:06:06.858479 7f5c973a57a0 20 read_log 192888'13665 (192888'13591) modify 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by client.1315259.0:57014 2014-08-11 16:47:03.385497 2014-08-13 18:06:06.858497 7f5c973a57a0 20 read_log 192888'13666 (192888'13593) modify 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.1315259.0:57239 2014-08-11 16:47:03.452406 2014-08-13 18:06:06.858518 7f5c973a57a0 20 read_log 192888'13667 (192888'13595) modify 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.1315259.0:59448 2014-08-11 16:47:04.122497 2014-08-13 18:06:06.858555 7f5c973a57a0 20 read_log 192888'13669 (192888'13599) modify 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.1315259.0:59798 2014-08-11 16:47:04.243853 2014-08-13 18:06:06.858579 7f5c973a57a0 20 read_log 192888'13674 (0'0) promote b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by osd.287.0:17548 2014-08-11 17:05:22.568616 2014-08-13 18:06:06.858598 7f5c973a57a0 20 read_log 192888'13675 (192888'13674) modify b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.942868.0:1836 2014-08-11 17:05:22.561802 2014-08-13 18:06:06.858621 7f5c973a57a0 20 read_log 192888'13676 (0'0) promote 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by osd.287.0:17554 2014-08-11 17:05:24.278921 2014-08-13 18:06:06.858637 7f5c973a57a0 20 read_log 192888'13677 (192888'13676) modify 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.942868.0:2254 2014-08-11 17:05:24.270738 2014-08-13 18:06:06.858650 7f5c973a57a0 20 read_log 192888'13680 (0'0) promote cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.287.0:17564 2014-08-11 17:05:25.637772 2014-08-13 18:06:06.858671 7f5c973a57a0 20 read_log 192888'13681 (192888'13680) modify cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by client.942868.0:2620 2014-08-11 17:05:25.629596 2014-08-13 18:06:06.858684 7f5c973a57a0 20 read_log 192888'13682 (0'0) promote af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by osd.287.0:17578 2014-08-11 17:05:29.567305 2014-08-13 18:06:06.858704 7f5c973a57a0 20 read_log 192888'13683 (192888'13682) modify af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.942868.0:3683 2014-08-11 17:05:29.559541 2014-08-13 18:06:06.858719 7f5c973a57a0 20 read_log 192888'13692 (0'0) promote b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by osd.287.0:17608 2014-08-11 17:05:35.909609 2014-08-13 18:06:06.858732 7f5c973a57a0 20 read_log 192888'13693 (192888'13692) modify b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.942868.0:5344 2014-08-11 17:05:35.902685 2014-08-13 18:06:06.858745 7f5c973a57a0 20 read_log 192888'13694 (0'0) promote 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.287.0:17620 2014-08-11 17:05:38.016853 2014-08-13 18:06:06.858757 7f5c973a57a0 20 read_log 192888'13695 (192888'13694) modify 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by client.942868.0:5917 2014-08-11 17:05:38.010395 2014-08-13 18:06:06.858770 7f5c973a57a0 20 read_log 192888'13700 (0'0) promote 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by osd.287.0:17672 2014-08-11 17:05:53.836993 2014-08-13 18:06:06.858782 7f5c973a57a0 20 read_log 192888'13701 (192888'13700) modify 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.942868.0:10175 2014-08-11 17:05:53.830808 2014-08-13 18:06:06.858807 7f5c973a57a0 20 read_log 192888'13704 (0'0) promote a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by osd.287.0:17680 2014-08-11 17:05:55.329593 2014-08-13 18:06:06.858831 7f5c973a57a0 20 read_log 192888'13705 (192888'13704) modify a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.942868.0:10590 2014-08-11 17:05:55.321917 2014-08-13 18:06:06.858844 7f5c973a57a0 20 read_log 192888'13706 (0'0) promote 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by osd.287.0:17692 2014-08-11 17:05:57.076833 2014-08-13 18:06:06.858857 7f5c973a57a0 20 read_log 192888'13707 (192888'13706) modify 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.942868.0:11078 2014-08-11 17:05:57.069576 2014-08-13 18:06:06.858869 7f5c973a57a0 20 read_log 192888'13714 (0'0) promote 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.287.0:17728 2014-08-11 17:06:07.697543 2014-08-13 18:06:06.858882 7f5c973a57a0 20 read_log 192888'13715 (192888'13714) modify 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by client.942868.0:13884 2014-08-11 17:06:07.691044 2014-08-13 18:06:06.858894 7f5c973a57a0 20 read_log 192888'13718 (0'0) promote 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by osd.287.0:17746 2014-08-11 17:06:11.508877 2014-08-13 18:06:06.858906 7f5c973a57a0 20 read_log 192888'13719 (192888'13718) modify 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.942868.0:14817 2014-08-11 17:06:11.502057 2014-08-13 18:06:06.858919 7f5c973a57a0 20 read_log 192888'13724 (0'0) promote 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.287.0:17784 2014-08-11 17:06:20.092309 2014-08-13 18:06:06.858931 7f5c973a57a0 20 read_log 192888'13725 (192888'13724) modify 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by client.942868.0:17008 2014-08-11 17:06:20.084301 2014-08-13 18:06:06.858943 7f5c973a57a0 20 read_log 192888'13726 (0'0) promote 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by osd.287.0:17796 2014-08-11 17:06:24.277319 2014-08-13 18:06:06.858955 7f5c973a57a0 20 read_log 192888'13727 (192888'13726) modify 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.942868.0:18118 2014-08-11 17:06:24.270459 2014-08-13 18:06:06.858968 7f5c973a57a0 20 read_log 192888'13730 (0'0) promote 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by osd.287.0:17806 2014-08-11 17:06:26.635647 2014-08-13 18:06:06.858990 7f5c973a57a0 20 read_log 192888'13731 (192888'13730) modify 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.942868.0:18744 2014-08-11 17:06:26.629014 2014-08-13 18:06:06.859005 7f5c973a57a0 20 read_log 192888'13738 (0'0) promote 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by osd.287.0:17832 2014-08-11 17:06:31.485260 2014-08-13 18:06:06.859018 7f5c973a57a0 20 read_log 192888'13739 (192888'13738) modify 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.942868.0:20032 2014-08-11 17:06:31.476726 2014-08-13 18:06:06.859038 7f5c973a57a0 20 read_log 192888'13740 (0'0) promote df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.287.0:17840 2014-08-11 17:06:32.217468 2014-08-13 18:06:06.859055 7f5c973a57a0 20 read_log 192888'13741 (192888'13740) modify df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by client.942868.0:20229 2014-08-11 17:06:32.208004 2014-08-13 18:06:06.859068 7f5c973a57a0 20 read_log 192888'13746 (0'0) promote c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.287.0:17870 2014-08-11 17:06:38.336781 2014-08-13 18:06:06.859080 7f5c973a57a0 20 read_log 192888'13747 (192888'13746) modify c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by client.942868.0:21828 2014-08-11 17:06:38.329830 2014-08-13 18:06:06.859092 7f5c973a57a0 20 read_log 192888'13750 (0'0) promote 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by osd.287.0:17882 2014-08-11 17:06:40.660047 2014-08-13 18:06:06.859104 7f5c973a57a0 20 read_log 192888'13751 (192888'13750) modify 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.942868.0:22423 2014-08-11 17:06:40.653422 2014-08-13 18:06:06.859119 7f5c973a57a0 20 read_log 192888'13752 (0'0) promote 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.287.0:17886 2014-08-11 17:06:45.222870 2014-08-13 18:06:06.859131 7f5c973a57a0 20 read_log 192888'13753 (192888'13752) modify 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by client.942868.0:23597 2014-08-11 17:06:45.215818 2014-08-13 18:06:06.859143 7f5c973a57a0 20 read_log 192888'13756 (0'0) promote 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by osd.287.0:17894 2014-08-11 17:06:46.430345 2014-08-13 18:06:06.859172 7f5c973a57a0 20 read_log 192888'13757 (192888'13756) modify 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.942868.0:23924 2014-08-11 17:06:46.422996 2014-08-13 18:06:06.859185 7f5c973a57a0 20 read_log 192888'13758 (0'0) promote d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by osd.287.0:17902 2014-08-11 17:06:48.492236 2014-08-13 18:06:06.859198 7f5c973a57a0 20 read_log 192888'13759 (192888'13758) modify d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.942868.0:24480 2014-08-11 17:06:48.485263 2014-08-13 18:06:06.859213 7f5c973a57a0 20 read_log 192888'13760 (0'0) promote 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.287.0:17906 2014-08-11 17:06:49.812696 2014-08-13 18:06:06.859229 7f5c973a57a0 20 read_log 192888'13761 (192888'13760) modify 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by client.942868.0:24839 2014-08-11 17:06:49.806992 2014-08-13 18:06:06.859242 7f5c973a57a0 20 read_log 192888'13766 (0'0) promote 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by osd.287.0:17936 2014-08-11 17:06:56.588130 2014-08-13 18:06:06.859254 7f5c973a57a0 20 read_log 192888'13767 (192888'13766) modify 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.942868.0:26572 2014-08-11 17:06:56.581988 2014-08-13 18:06:06.859267 7f5c973a57a0 20 read_log 192888'13768 (0'0) promote 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by osd.287.0:17942 2014-08-11 17:06:56.811537 2014-08-13 18:06:06.859286 7f5c973a57a0 20 read_log 192888'13769 (192888'13768) modify 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.942868.0:26633 2014-08-11 17:06:56.805013 2014-08-13 18:06:06.859301 7f5c973a57a0 20 read_log 192888'13770 (0'0) promote cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.287.0:17962 2014-08-11 17:07:00.866695 2014-08-13 18:06:06.859323 7f5c973a57a0 20 read_log 192888'13771 (192888'13770) modify cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by client.942868.0:27696 2014-08-11 17:07:00.858713 2014-08-13 18:06:06.859340 7f5c973a57a0 20 read_log 192888'13772 (0'0) promote a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.287.0:17970 2014-08-11 17:07:04.594589 2014-08-13 18:06:06.859352 7f5c973a57a0 20 read_log 192888'13773 (192888'13772) modify a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by client.942868.0:28708 2014-08-11 17:07:04.587972 2014-08-13 18:06:06.859372 7f5c973a57a0 20 read_log 192888'13774 (0'0) promote 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by osd.287.0:17988 2014-08-11 17:07:08.837774 2014-08-13 18:06:06.859385 7f5c973a57a0 20 read_log 192888'13775 (192888'13774) modify 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.942868.0:29756 2014-08-11 17:07:08.830178 2014-08-13 18:06:06.859398 7f5c973a57a0 20 read_log 192888'13776 (0'0) promote 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.287.0:17994 2014-08-11 17:07:09.529258 2014-08-13 18:06:06.859410 7f5c973a57a0 20 read_log 192888'13777 (192888'13776) modify 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by client.942868.0:29932 2014-08-11 17:07:09.521058 2014-08-13 18:06:06.859423 7f5c973a57a0 20 read_log 192888'13778 (0'0) promote fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by osd.287.0:18000 2014-08-11 17:07:10.127710 2014-08-13 18:06:06.859435 7f5c973a57a0 20 read_log 192888'13779 (192888'13778) modify fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.942868.0:30081 2014-08-11 17:07:10.121072 2014-08-13 18:06:06.859448 7f5c973a57a0 20 read_log 192888'13780 (0'0) promote 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by osd.287.0:18022 2014-08-11 17:07:12.782187 2014-08-13 18:06:06.859460 7f5c973a57a0 20 read_log 192888'13781 (192888'13780) modify 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.942868.0:30740 2014-08-11 17:07:12.776189 2014-08-13 18:06:06.859472 7f5c973a57a0 20 read_log 192888'13782 (0'0) promote f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.287.0:18024 2014-08-11 17:07:12.854944 2014-08-13 18:06:06.859484 7f5c973a57a0 20 read_log 192888'13783 (192888'13782) modify f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by client.942868.0:30755 2014-08-11 17:07:12.847785 2014-08-13 18:06:06.859496 7f5c973a57a0 20 read_log 192888'13789 (192888'13675) modify b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.942868.0:33102 2014-08-11 17:07:16.426581 2014-08-13 18:06:06.859509 7f5c973a57a0 20 read_log 192888'13790 (192888'13677) modify 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.942868.0:33520 2014-08-11 17:07:16.538403 2014-08-13 18:06:06.859521 7f5c973a57a0 20 read_log 192888'13792 (192888'13681) modify cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by client.942868.0:33886 2014-08-11 17:07:16.638109 2014-08-13 18:06:06.859533 7f5c973a57a0 20 read_log 192888'13793 (192888'13683) modify af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.942868.0:34949 2014-08-11 17:07:16.940638 2014-08-13 18:06:06.859546 7f5c973a57a0 20 read_log 192888'13798 (192888'13693) modify b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.942868.0:36610 2014-08-11 17:07:17.388544 2014-08-13 18:06:06.859590 7f5c973a57a0 20 read_log 192888'13799 (192888'13695) modify 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by client.942868.0:37183 2014-08-11 17:07:17.537203 2014-08-13 18:06:06.859607 7f5c973a57a0 20 read_log 192888'13802 (192888'13701) modify 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.942868.0:41441 2014-08-11 17:07:18.729336 2014-08-13 18:06:06.859620 7f5c973a57a0 20 read_log 192888'13804 (192888'13705) modify a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.942868.0:41856 2014-08-11 17:07:18.841185 2014-08-13 18:06:06.859632 7f5c973a57a0 20 read_log 192888'13805 (192888'13707) modify 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.942868.0:42344 2014-08-11 17:07:18.974047 2014-08-13 18:06:06.859644 7f5c973a57a0 20 read_log 192888'13809 (192888'13715) modify 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by client.942868.0:45150 2014-08-11 17:07:19.759466 2014-08-13 18:06:06.859659 7f5c973a57a0 20 read_log 192888'13811 (192888'13719) modify 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.942868.0:46083 2014-08-11 17:07:20.015929 2014-08-13 18:06:06.859672 7f5c973a57a0 20 read_log 192888'13814 (192888'13725) modify 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by client.942868.0:48274 2014-08-11 17:07:20.672979 2014-08-13 18:06:06.859686 7f5c973a57a0 20 read_log 192888'13815 (192888'13727) modify 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.942868.0:49384 2014-08-11 17:07:20.996007 2014-08-13 18:06:06.859698 7f5c973a57a0 20 read_log 192888'13817 (192888'13731) modify 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.942868.0:50010 2014-08-11 17:07:21.188612 2014-08-13 18:06:06.859713 7f5c973a57a0 20 read_log 192888'13821 (192888'13739) modify 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.942868.0:51298 2014-08-11 17:07:21.600410 2014-08-13 18:06:06.859725 7f5c973a57a0 20 read_log 192888'13822 (192888'13741) modify df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by client.942868.0:51495 2014-08-11 17:07:21.663258 2014-08-13 18:06:06.859739 7f5c973a57a0 20 read_log 192888'13825 (192888'13747) modify c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by client.942868.0:53094 2014-08-11 17:07:22.145522 2014-08-13 18:06:06.859752 7f5c973a57a0 20 read_log 192888'13827 (192888'13751) modify 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.942868.0:53689 2014-08-11 17:07:22.343385 2014-08-13 18:06:06.859764 7f5c973a57a0 20 read_log 192888'13828 (192888'13753) modify 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by client.942868.0:54863 2014-08-11 17:07:22.677572 2014-08-13 18:06:06.859778 7f5c973a57a0 20 read_log 192888'13830 (192888'13757) modify 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.942868.0:55190 2014-08-11 17:07:22.769185 2014-08-13 18:06:06.859791 7f5c973a57a0 20 read_log 192888'13831 (192888'13759) modify d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.942868.0:55746 2014-08-11 17:07:22.941318 2014-08-13 18:06:06.859803 7f5c973a57a0 20 read_log 192888'13832 (192888'13761) modify 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by client.942868.0:56105 2014-08-11 17:07:23.053698 2014-08-13 18:06:06.859816 7f5c973a57a0 20 read_log 192888'13835 (192888'13767) modify 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.942868.0:57838 2014-08-11 17:07:23.570648 2014-08-13 18:06:06.859831 7f5c973a57a0 20 read_log 192888'13836 (192888'13769) modify 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.942868.0:57899 2014-08-11 17:07:23.586438 2014-08-13 18:06:06.859843 7f5c973a57a0 20 read_log 192888'13837 (192888'13771) modify cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by client.942868.0:58962 2014-08-11 17:07:23.904060 2014-08-13 18:06:06.859868 7f5c973a57a0 20 read_log 192888'13838 (192888'13773) modify a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by client.942868.0:59974 2014-08-11 17:07:24.206669 2014-08-13 18:06:06.859883 7f5c973a57a0 20 read_log 192888'13839 (192888'13775) modify 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.942868.0:61022 2014-08-11 17:07:24.516104 2014-08-13 18:06:06.859896 7f5c973a57a0 20 read_log 192888'13840 (192888'13777) modify 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by client.942868.0:61198 2014-08-11 17:07:24.568633 2014-08-13 18:06:06.859910 7f5c973a57a0 20 read_log 192888'13841 (192888'13779) modify fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.942868.0:61347 2014-08-11 17:07:24.612898 2014-08-13 18:06:06.859922 7f5c973a57a0 20 read_log 192888'13842 (192888'13781) modify 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.942868.0:62006 2014-08-11 17:07:24.819529 2014-08-13 18:06:06.859952 7f5c973a57a0 20 read_log 192888'13843 (192888'13783) modify f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by client.942868.0:62021 2014-08-11 17:07:24.823999 2014-08-13 18:06:06.859968 7f5c973a57a0 20 read_log 193305'13846 (0'0) promote b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1 2014-08-11 18:42:32.814693 2014-08-13 18:06:06.859981 7f5c973a57a0 20 read_log 193305'13847 (193305'13846) modify b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by client.951337.0:2 2014-08-11 18:42:32.802553 2014-08-13 18:06:06.859996 7f5c973a57a0 20 read_log 193305'13848 (0'0) promote 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by osd.17.0:8 2014-08-11 18:42:34.934816 2014-08-13 18:06:06.860008 7f5c973a57a0 20 read_log 193305'13849 (193305'13848) modify 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.951337.0:474 2014-08-11 18:42:34.927729 2014-08-13 18:06:06.860023 7f5c973a57a0 20 read_log 193305'13850 (0'0) promote 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by osd.17.0:10 2014-08-11 18:42:35.268824 2014-08-13 18:06:06.860035 7f5c973a57a0 20 read_log 193305'13851 (193305'13850) modify 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.951337.0:548 2014-08-11 18:42:35.261381 2014-08-13 18:06:06.860049 7f5c973a57a0 20 read_log 193305'13852 (0'0) promote 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:23 2014-08-11 18:42:36.794168 2014-08-13 18:06:06.860064 7f5c973a57a0 20 read_log 193305'13853 (193305'13852) modify 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by client.951337.0:932 2014-08-11 18:42:36.720429 2014-08-13 18:06:06.860077 7f5c973a57a0 20 read_log 193305'13854 (0'0) promote 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:85 2014-08-11 18:42:51.408499 2014-08-13 18:06:06.860089 7f5c973a57a0 20 read_log 193305'13855 (193305'13854) modify 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by client.951337.0:4911 2014-08-11 18:42:51.400279 2014-08-13 18:06:06.860101 7f5c973a57a0 20 read_log 193305'13856 (0'0) promote 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:95 2014-08-11 18:42:52.899346 2014-08-13 18:06:06.860115 7f5c973a57a0 20 read_log 193305'13857 (193305'13856) modify 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by client.951337.0:5303 2014-08-11 18:42:52.889184 2014-08-13 18:06:06.860129 7f5c973a57a0 20 read_log 193305'13858 (0'0) promote 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by osd.17.0:97 2014-08-11 18:42:53.570702 2014-08-13 18:06:06.860142 7f5c973a57a0 20 read_log 193305'13859 (193305'13858) modify 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.951337.0:5490 2014-08-11 18:42:53.563157 2014-08-13 18:06:06.860169 7f5c973a57a0 20 read_log 193305'13860 (0'0) promote 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:109 2014-08-11 18:42:56.007818 2014-08-13 18:06:06.860184 7f5c973a57a0 20 read_log 193305'13861 (193305'13860) modify 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by client.951337.0:6160 2014-08-11 18:42:56.000458 2014-08-13 18:06:06.860197 7f5c973a57a0 20 read_log 193305'13862 (0'0) promote dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by osd.17.0:131 2014-08-11 18:43:00.544443 2014-08-13 18:06:06.860212 7f5c973a57a0 20 read_log 193305'13863 (193305'13862) modify dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.951337.0:7416 2014-08-11 18:43:00.536917 2014-08-13 18:06:06.860227 7f5c973a57a0 20 read_log 193305'13864 (0'0) promote 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by osd.17.0:143 2014-08-11 18:43:03.622986 2014-08-13 18:06:06.860244 7f5c973a57a0 20 read_log 193305'13865 (193305'13864) modify 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.951337.0:8242 2014-08-11 18:43:03.615856 2014-08-13 18:06:06.860258 7f5c973a57a0 20 read_log 193305'13866 (0'0) promote 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by osd.17.0:159 2014-08-11 18:43:08.597263 2014-08-13 18:06:06.860271 7f5c973a57a0 20 read_log 193305'13867 (193305'13866) modify 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.951337.0:9535 2014-08-11 18:43:08.587522 2014-08-13 18:06:06.860285 7f5c973a57a0 20 read_log 193305'13868 (0'0) promote 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by osd.17.0:161 2014-08-11 18:43:09.357897 2014-08-13 18:06:06.860298 7f5c973a57a0 20 read_log 193305'13869 (193305'13868) modify 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.951337.0:9742 2014-08-11 18:43:09.350113 2014-08-13 18:06:06.860315 7f5c973a57a0 20 read_log 193305'13870 (0'0) promote 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by osd.17.0:167 2014-08-11 18:43:10.000332 2014-08-13 18:06:06.860328 7f5c973a57a0 20 read_log 193305'13871 (193305'13870) modify 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.951337.0:9918 2014-08-11 18:43:09.990208 2014-08-13 18:06:06.860343 7f5c973a57a0 20 read_log 193305'13872 (0'0) promote ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by osd.17.0:173 2014-08-11 18:43:11.439680 2014-08-13 18:06:06.860355 7f5c973a57a0 20 read_log 193305'13873 (193305'13872) modify ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.951337.0:10317 2014-08-11 18:43:11.431564 2014-08-13 18:06:06.860369 7f5c973a57a0 20 read_log 193305'13874 (0'0) promote 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:201 2014-08-11 18:43:18.362657 2014-08-13 18:06:06.860382 7f5c973a57a0 20 read_log 193305'13875 (193305'13874) modify 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by client.951337.0:12129 2014-08-11 18:43:18.356094 2014-08-13 18:06:06.860394 7f5c973a57a0 20 read_log 193305'13876 (0'0) promote ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:214 2014-08-11 18:43:21.353237 2014-08-13 18:06:06.860407 7f5c973a57a0 20 read_log 193305'13877 (193305'13876) modify ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by client.951337.0:12923 2014-08-11 18:43:21.346442 2014-08-13 18:06:06.860419 7f5c973a57a0 20 read_log 193305'13878 (0'0) promote 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:218 2014-08-11 18:43:22.052631 2014-08-13 18:06:06.860431 7f5c973a57a0 20 read_log 193305'13879 (193305'13878) modify 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by client.951337.0:13116 2014-08-11 18:43:22.046456 2014-08-13 18:06:06.860451 7f5c973a57a0 20 read_log 193305'13880 (0'0) promote b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by osd.17.0:222 2014-08-11 18:43:22.822538 2014-08-13 18:06:06.860477 7f5c973a57a0 20 read_log 193305'13881 (193305'13880) modify b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.951337.0:13318 2014-08-11 18:43:22.813181 2014-08-13 18:06:06.860494 7f5c973a57a0 20 read_log 193305'13882 (0'0) promote 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:243 2014-08-11 18:43:29.216490 2014-08-13 18:06:06.860506 7f5c973a57a0 20 read_log 193305'13883 (193305'13882) modify 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by client.951337.0:15072 2014-08-11 18:43:29.210284 2014-08-13 18:06:06.860532 7f5c973a57a0 20 read_log 193305'13884 (0'0) promote 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by osd.17.0:251 2014-08-11 18:43:30.648736 2014-08-13 18:06:06.860545 7f5c973a57a0 20 read_log 193305'13885 (193305'13884) modify 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.951337.0:15497 2014-08-11 18:43:30.641401 2014-08-13 18:06:06.860564 7f5c973a57a0 20 read_log 193305'13886 (0'0) promote 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by osd.17.0:257 2014-08-11 18:43:31.778700 2014-08-13 18:06:06.860586 7f5c973a57a0 20 read_log 193305'13887 (193305'13886) modify 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.951337.0:15827 2014-08-11 18:43:31.771381 2014-08-13 18:06:06.860601 7f5c973a57a0 20 read_log 193305'13888 (193305'13888) modify 2a1/hit_set_24.2a1_archive_2014-08-11 18:38:33.126704_2014-08-11 18:43:37.373511/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:37.373771 2014-08-13 18:06:06.860615 7f5c973a57a0 20 read_log 193305'13889 (0'0) promote 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by osd.17.0:268 2014-08-11 18:43:37.377105 2014-08-13 18:06:06.860627 7f5c973a57a0 20 read_log 193305'13890 (193305'13889) modify 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.951337.0:17420 2014-08-11 18:43:37.366298 2014-08-13 18:06:06.860650 7f5c973a57a0 20 read_log 193305'13891 (0'0) promote 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by osd.17.0:278 2014-08-11 18:43:39.078991 2014-08-13 18:06:06.860663 7f5c973a57a0 20 read_log 193305'13892 (193305'13891) modify 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.951337.0:17912 2014-08-11 18:43:39.070327 2014-08-13 18:06:06.860675 7f5c973a57a0 20 read_log 193305'13893 (0'0) promote f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:312 2014-08-11 18:43:46.408850 2014-08-13 18:06:06.860702 7f5c973a57a0 20 read_log 193305'13894 (193305'13893) modify f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by client.951337.0:19946 2014-08-11 18:43:46.401796 2014-08-13 18:06:06.860715 7f5c973a57a0 20 read_log 193305'13895 (0'0) promote 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:356 2014-08-11 18:43:57.215262 2014-08-13 18:06:06.860727 7f5c973a57a0 20 read_log 193305'13896 (193305'13895) modify 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by client.951337.0:22935 2014-08-11 18:43:57.208525 2014-08-13 18:06:06.860739 7f5c973a57a0 20 read_log 193305'13897 (0'0) promote 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by osd.17.0:383 2014-08-11 18:44:01.985860 2014-08-13 18:06:06.860755 7f5c973a57a0 20 read_log 193305'13898 (193305'13897) modify 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.951337.0:24261 2014-08-11 18:44:01.976622 2014-08-13 18:06:06.860778 7f5c973a57a0 20 read_log 193305'13899 (0'0) promote aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by osd.17.0:397 2014-08-11 18:44:04.659279 2014-08-13 18:06:06.860794 7f5c973a57a0 20 read_log 193305'13900 (193305'13899) modify aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.951337.0:24996 2014-08-11 18:44:04.651110 2014-08-13 18:06:06.860806 7f5c973a57a0 20 read_log 193305'13901 (0'0) promote 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by osd.17.0:409 2014-08-11 18:44:07.987238 2014-08-13 18:06:06.860819 7f5c973a57a0 20 read_log 193305'13902 (193305'13901) modify 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.951337.0:25901 2014-08-11 18:44:07.980629 2014-08-13 18:06:06.860831 7f5c973a57a0 20 read_log 193305'13903 (0'0) promote 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:427 2014-08-11 18:44:13.330964 2014-08-13 18:06:06.860843 7f5c973a57a0 20 read_log 193305'13904 (193305'13903) modify 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by client.951337.0:27341 2014-08-11 18:44:13.322852 2014-08-13 18:06:06.860855 7f5c973a57a0 20 read_log 193305'13905 (0'0) promote 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:451 2014-08-11 18:44:20.909281 2014-08-13 18:06:06.860867 7f5c973a57a0 20 read_log 193305'13906 (193305'13905) modify 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by client.951337.0:29430 2014-08-11 18:44:20.900747 2014-08-13 18:06:06.860880 7f5c973a57a0 20 read_log 193305'13907 (0'0) promote 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by osd.17.0:463 2014-08-11 18:44:23.934684 2014-08-13 18:06:06.860892 7f5c973a57a0 20 read_log 193305'13908 (193305'13907) modify 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.951337.0:30233 2014-08-11 18:44:23.924983 2014-08-13 18:06:06.860904 7f5c973a57a0 20 read_log 193305'13909 (0'0) promote e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:477 2014-08-11 18:44:26.464272 2014-08-13 18:06:06.860916 7f5c973a57a0 20 read_log 193305'13910 (193305'13909) modify e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by client.951337.0:30895 2014-08-11 18:44:26.455945 2014-08-13 18:06:06.860928 7f5c973a57a0 20 read_log 193305'13911 (0'0) promote 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by osd.17.0:501 2014-08-11 18:44:32.083570 2014-08-13 18:06:06.860941 7f5c973a57a0 20 read_log 193305'13912 (193305'13911) modify 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.951337.0:32500 2014-08-11 18:44:32.075785 2014-08-13 18:06:06.860953 7f5c973a57a0 20 read_log 193305'13913 (193305'13847) modify b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by client.951337.0:32713 2014-08-11 18:44:33.823691 2014-08-13 18:06:06.860965 7f5c973a57a0 20 read_log 193305'13914 (193305'13849) modify 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.951337.0:33185 2014-08-11 18:44:34.223969 2014-08-13 18:06:06.860977 7f5c973a57a0 20 read_log 193305'13915 (193305'13851) modify 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.951337.0:33259 2014-08-11 18:44:34.281589 2014-08-13 18:06:06.860989 7f5c973a57a0 20 read_log 193305'13916 (193305'13853) modify 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by client.951337.0:33643 2014-08-11 18:44:34.552503 2014-08-13 18:06:06.861002 7f5c973a57a0 20 read_log 193305'13917 (193305'13855) modify 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by client.951337.0:37622 2014-08-11 18:44:36.995436 2014-08-13 18:06:06.861014 7f5c973a57a0 20 read_log 193305'13918 (193305'13857) modify 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by client.951337.0:38014 2014-08-11 18:44:37.197751 2014-08-13 18:06:06.861033 7f5c973a57a0 20 read_log 193305'13919 (193305'13859) modify 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.951337.0:38201 2014-08-11 18:44:37.285099 2014-08-13 18:06:06.861049 7f5c973a57a0 20 read_log 193305'13920 (193305'13861) modify 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by client.951337.0:38871 2014-08-11 18:44:37.703329 2014-08-13 18:06:06.861061 7f5c973a57a0 20 read_log 193305'13921 (193305'13863) modify dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.951337.0:40127 2014-08-11 18:44:38.298514 2014-08-13 18:06:06.861076 7f5c973a57a0 20 read_log 193305'13922 (193305'13865) modify 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.951337.0:40953 2014-08-11 18:44:38.586488 2014-08-13 18:06:06.861088 7f5c973a57a0 20 read_log 193305'13923 (193305'13867) modify 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.951337.0:42246 2014-08-11 18:44:39.162709 2014-08-13 18:06:06.861102 7f5c973a57a0 20 read_log 193305'13924 (193305'13869) modify 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.951337.0:42453 2014-08-11 18:44:39.237610 2014-08-13 18:06:06.861115 7f5c973a57a0 20 read_log 193305'13925 (193305'13871) modify 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.951337.0:42629 2014-08-11 18:44:39.320870 2014-08-13 18:06:06.861129 7f5c973a57a0 20 read_log 193305'13926 (193305'13873) modify ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.951337.0:43028 2014-08-11 18:44:39.461329 2014-08-13 18:06:06.861141 7f5c973a57a0 20 read_log 193305'13927 (193305'13875) modify 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by client.951337.0:44840 2014-08-11 18:44:40.008527 2014-08-13 18:06:06.861162 7f5c973a57a0 20 read_log 193305'13928 (193305'13877) modify ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by client.951337.0:45634 2014-08-11 18:44:40.257328 2014-08-13 18:06:06.861174 7f5c973a57a0 20 read_log 193305'13929 (193305'13879) modify 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by client.951337.0:45827 2014-08-11 18:44:40.313407 2014-08-13 18:06:06.861187 7f5c973a57a0 20 read_log 193305'13930 (193305'13881) modify b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.951337.0:46029 2014-08-11 18:44:40.370533 2014-08-13 18:06:06.861199 7f5c973a57a0 20 read_log 193305'13931 (193305'13883) modify 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by client.951337.0:47783 2014-08-11 18:44:40.865418 2014-08-13 18:06:06.861213 7f5c973a57a0 20 read_log 193305'13932 (193305'13885) modify 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.951337.0:48208 2014-08-11 18:44:40.988037 2014-08-13 18:06:06.861226 7f5c973a57a0 20 read_log 193305'13933 (193305'13887) modify 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.951337.0:48538 2014-08-11 18:44:41.082418 2014-08-13 18:06:06.861238 7f5c973a57a0 20 read_log 193305'13934 (193305'13890) modify 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.951337.0:50131 2014-08-11 18:44:41.580663 2014-08-13 18:06:06.861251 7f5c973a57a0 20 read_log 193305'13935 (193305'13892) modify 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.951337.0:50623 2014-08-11 18:44:41.719228 2014-08-13 18:06:06.861263 7f5c973a57a0 20 read_log 193305'13936 (193305'13894) modify f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by client.951337.0:52657 2014-08-11 18:44:42.297074 2014-08-13 18:06:06.861277 7f5c973a57a0 20 read_log 193305'13937 (193305'13896) modify 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by client.951337.0:55646 2014-08-11 18:44:43.188624 2014-08-13 18:06:06.861315 7f5c973a57a0 20 read_log 193305'13938 (193305'13898) modify 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.951337.0:56972 2014-08-11 18:44:43.589799 2014-08-13 18:06:06.861361 7f5c973a57a0 20 read_log 193305'13939 (193305'13900) modify aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.951337.0:57707 2014-08-11 18:44:43.836769 2014-08-13 18:06:06.861381 7f5c973a57a0 20 read_log 193305'13940 (193305'13902) modify 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.951337.0:58612 2014-08-11 18:44:44.091813 2014-08-13 18:06:06.861396 7f5c973a57a0 20 read_log 193305'13941 (193305'13904) modify 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by client.951337.0:60052 2014-08-11 18:44:44.506405 2014-08-13 18:06:06.861418 7f5c973a57a0 20 read_log 193305'13942 (193305'13906) modify 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by client.951337.0:62141 2014-08-11 18:44:45.137774 2014-08-13 18:06:06.861434 7f5c973a57a0 20 read_log 193305'13943 (193305'13908) modify 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.951337.0:62944 2014-08-11 18:44:45.368493 2014-08-13 18:06:06.861447 7f5c973a57a0 20 read_log 193305'13944 (193305'13910) modify e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by client.951337.0:63606 2014-08-11 18:44:45.557740 2014-08-13 18:06:06.861467 7f5c973a57a0 20 read_log 193305'13945 (193305'13912) modify 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.951337.0:65211 2014-08-11 18:44:46.051100 2014-08-13 18:06:06.861483 7f5c973a57a0 20 read_log 194209'13946 (192888'13628) clean d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.17.0:1312 2014-08-12 14:36:39.840975 2014-08-13 18:06:06.861495 7f5c973a57a0 20 read_log 194209'13947 (192834'12750) clean 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.17.0:1313 2014-08-12 14:36:39.841511 2014-08-13 18:06:06.861518 7f5c973a57a0 20 read_log 194209'13948 (192888'13665) clean 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.17.0:1314 2014-08-12 14:36:39.841939 2014-08-13 18:06:06.861534 7f5c973a57a0 20 read_log 194209'13949 (192834'13290) clean 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.17.0:1315 2014-08-12 14:36:39.842440 2014-08-13 18:06:06.861546 7f5c973a57a0 20 read_log 194209'13950 (192888'13659) clean 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.17.0:1316 2014-08-12 14:36:39.853153 2014-08-13 18:06:06.861559 7f5c973a57a0 20 read_log 194209'13951 (192834'13364) clean a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.17.0:1317 2014-08-12 14:36:39.853441 2014-08-13 18:06:06.861571 7f5c973a57a0 20 read_log 194209'13952 (192834'12835) clean 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.17.0:1318 2014-08-12 14:36:39.856558 2014-08-13 18:06:06.861583 7f5c973a57a0 20 read_log 194209'13953 (192888'13832) clean 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.17.0:1319 2014-08-12 14:36:39.859767 2014-08-13 18:06:06.861596 7f5c973a57a0 20 read_log 194209'13954 (192834'13389) clean a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.17.0:1320 2014-08-12 14:36:39.863056 2014-08-13 18:06:06.861608 7f5c973a57a0 20 read_log 194209'13955 (192888'13612) clean 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.17.0:1321 2014-08-12 14:36:39.866254 2014-08-13 18:06:06.861620 7f5c973a57a0 20 read_log 194209'13956 (192888'13600) clean 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.17.0:1322 2014-08-12 14:36:39.869358 2014-08-13 18:06:06.861632 7f5c973a57a0 20 read_log 194209'13957 (192834'13360) clean 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.17.0:1323 2014-08-12 14:36:39.872782 2014-08-13 18:06:06.861671 7f5c973a57a0 20 read_log 194209'13958 (192888'13605) clean 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.17.0:1324 2014-08-12 14:36:39.875720 2014-08-13 18:06:06.861687 7f5c973a57a0 20 read_log 194209'13959 (193305'13913) clean b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1325 2014-08-12 14:36:39.878882 2014-08-13 18:06:06.861699 7f5c973a57a0 20 read_log 194209'13960 (193305'13942) clean 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:1326 2014-08-12 14:36:39.882152 2014-08-13 18:06:06.861712 7f5c973a57a0 20 read_log 194209'13961 (193305'13936) clean f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:1327 2014-08-12 14:36:39.885392 2014-08-13 18:06:06.861724 7f5c973a57a0 20 read_log 194209'13962 (192888'13828) clean 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.17.0:1328 2014-08-12 14:36:39.888879 2014-08-13 18:06:06.861736 7f5c973a57a0 20 read_log 194209'13963 (192834'13313) clean fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.17.0:1329 2014-08-12 14:36:39.891830 2014-08-13 18:06:06.861749 7f5c973a57a0 20 read_log 194209'13964 (192888'13792) clean cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.17.0:1330 2014-08-12 14:36:39.897889 2014-08-13 18:06:06.861761 7f5c973a57a0 20 read_log 194209'13965 (192888'13840) clean 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.17.0:1331 2014-08-12 14:36:39.898167 2014-08-13 18:06:06.861773 7f5c973a57a0 20 read_log 194209'13966 (193305'13928) clean ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:1332 2014-08-12 14:36:39.901406 2014-08-13 18:06:06.861785 7f5c973a57a0 20 read_log 194209'13967 (192834'13378) clean 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.17.0:1333 2014-08-12 14:36:39.904842 2014-08-13 18:06:06.861799 7f5c973a57a0 20 read_log 194209'13968 (192888'13825) clean c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.17.0:1334 2014-08-12 14:36:39.908161 2014-08-13 18:06:06.861811 7f5c973a57a0 20 read_log 194209'13969 (193305'13944) clean e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:1335 2014-08-12 14:36:39.911222 2014-08-13 18:06:06.861823 7f5c973a57a0 20 read_log 194209'13970 (192888'13643) clean 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.17.0:1336 2014-08-12 14:36:39.914346 2014-08-13 18:06:06.861838 7f5c973a57a0 20 read_log 194209'13971 (192834'13440) clean 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.17.0:1337 2014-08-12 14:36:39.917712 2014-08-13 18:06:06.861853 7f5c973a57a0 20 read_log 194209'13972 (192834'13375) clean 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.17.0:1338 2014-08-12 14:36:39.921471 2014-08-13 18:06:06.861866 7f5c973a57a0 20 read_log 194209'13973 (192888'13644) clean 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.17.0:1339 2014-08-12 14:36:39.924330 2014-08-13 18:06:06.861878 7f5c973a57a0 20 read_log 194209'13974 (193305'13917) clean 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:1340 2014-08-12 14:36:39.927406 2014-08-13 18:06:06.861890 7f5c973a57a0 20 read_log 194209'13975 (192888'13814) clean 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.17.0:1341 2014-08-12 14:36:39.930786 2014-08-13 18:06:06.861905 7f5c973a57a0 20 read_log 194209'13976 (192888'13843) clean f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.17.0:1342 2014-08-12 14:36:39.934541 2014-08-13 18:06:06.861919 7f5c973a57a0 20 read_log 194209'13977 (193305'13937) clean 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:1343 2014-08-12 14:36:39.937405 2014-08-13 18:06:06.861939 7f5c973a57a0 20 read_log 194209'13978 (192888'13645) clean e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.17.0:1344 2014-08-12 14:36:39.940426 2014-08-13 18:06:06.861954 7f5c973a57a0 20 read_log 194209'13979 (193305'13927) clean 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:1345 2014-08-12 14:36:39.943623 2014-08-13 18:06:06.861966 7f5c973a57a0 20 read_log 194209'13980 (192888'13799) clean 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.17.0:1346 2014-08-12 14:36:39.946743 2014-08-13 18:06:06.861980 7f5c973a57a0 20 read_log 194209'13981 (193305'13916) clean 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:1347 2014-08-12 14:36:39.950002 2014-08-13 18:06:06.861993 7f5c973a57a0 20 read_log 194209'13982 (192834'13385) clean eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.17.0:1348 2014-08-12 14:36:39.953360 2014-08-13 18:06:06.862007 7f5c973a57a0 20 read_log 194209'13983 (192888'13619) clean 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.17.0:1349 2014-08-12 14:36:39.956540 2014-08-13 18:06:06.862019 7f5c973a57a0 20 read_log 194209'13984 (192834'12890) clean c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.17.0:1350 2014-08-12 14:36:39.959665 2014-08-13 18:06:06.862034 7f5c973a57a0 20 read_log 194209'13985 (192888'13837) clean cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.17.0:1351 2014-08-12 14:36:39.963405 2014-08-13 18:06:06.862046 7f5c973a57a0 20 read_log 194209'13986 (192888'13656) clean b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by osd.17.0:1352 2014-08-12 14:36:39.967013 2014-08-13 18:06:06.862060 7f5c973a57a0 20 read_log 194209'13987 (192888'13793) clean af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by osd.17.0:1353 2014-08-12 14:36:39.970637 2014-08-13 18:06:06.862073 7f5c973a57a0 20 read_log 194209'13988 (193305'13941) clean 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:1354 2014-08-12 14:36:39.974087 2014-08-13 18:06:06.862087 7f5c973a57a0 20 read_log 194209'13989 (193305'13918) clean 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:1355 2014-08-12 14:36:39.977730 2014-08-13 18:06:06.862100 7f5c973a57a0 20 read_log 194209'13990 (193305'13920) clean 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:1356 2014-08-12 14:36:39.981344 2014-08-13 18:06:06.862114 7f5c973a57a0 20 read_log 194209'13991 (192834'13327) clean 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.17.0:1357 2014-08-12 14:36:39.984830 2014-08-13 18:06:06.862126 7f5c973a57a0 20 read_log 194209'13992 (192834'13433) clean c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.17.0:1358 2014-08-12 14:36:39.988708 2014-08-13 18:06:06.862138 7f5c973a57a0 20 read_log 194209'13993 (192888'13629) clean 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.17.0:1359 2014-08-12 14:36:39.992137 2014-08-13 18:06:06.862183 7f5c973a57a0 20 read_log 194209'13994 (192888'13838) clean a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.17.0:1360 2014-08-12 14:36:39.995692 2014-08-13 18:06:06.862200 7f5c973a57a0 20 read_log 194209'13995 (192834'12848) clean bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.17.0:1361 2014-08-12 14:36:39.999090 2014-08-13 18:06:06.862213 7f5c973a57a0 20 read_log 194209'13996 (192888'13809) clean 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.17.0:1362 2014-08-12 14:36:40.002686 2014-08-13 18:06:06.862228 7f5c973a57a0 20 read_log 194209'13997 (193305'13931) clean 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:1363 2014-08-12 14:36:40.005797 2014-08-13 18:06:06.862257 7f5c973a57a0 20 read_log 194209'13998 (192834'13417) clean c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.17.0:1364 2014-08-12 14:36:40.009246 2014-08-13 18:06:06.862273 7f5c973a57a0 20 read_log 194209'13999 (192888'13822) clean df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.17.0:1365 2014-08-12 14:36:40.012776 2014-08-13 18:06:06.862286 7f5c973a57a0 20 read_log 194209'14000 (193305'13929) clean 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:1366 2014-08-12 14:36:40.016334 2014-08-13 18:06:06.862300 7f5c973a57a0 20 read_log 194209'14001 (192888'13601) clean bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.17.0:1367 2014-08-12 14:36:40.019766 2014-08-13 18:06:06.862318 7f5c973a57a0 20 read_log 194209'14002 (192834'12770) clean 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.17.0:1368 2014-08-12 14:36:40.023538 2014-08-13 18:06:06.862331 7f5c973a57a0 20 read_log 194209'14003 (194209'13946) delete d162faa1/benchmark_data_storage0111-ib.pocstor_41097_object12091/head//24 by osd.17.0:1369 2014-08-12 14:36:40.060792 2014-08-13 18:06:06.862343 7f5c973a57a0 20 read_log 194209'14004 (194209'13947) delete 9d06faa1/benchmark_data_storage0111-ib.pocstor_26659_object3487/head//24 by osd.17.0:1370 2014-08-12 14:36:40.063594 2014-08-13 18:06:06.862356 7f5c973a57a0 20 read_log 194209'14005 (192834'13332) clean 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.17.0:1371 2014-08-12 14:36:40.064040 2014-08-13 18:06:06.862368 7f5c973a57a0 20 read_log 194209'14006 (194209'13948) delete 1a88faa1/benchmark_data_storage0111-ib.pocstor_41097_object26286/head//24 by osd.17.0:1372 2014-08-12 14:36:40.083667 2014-08-13 18:06:06.862380 7f5c973a57a0 20 read_log 194209'14007 (194209'13949) delete 5c5cfaa1/benchmark_data_storage0111-ib.pocstor_41526_object2057/head//24 by osd.17.0:1373 2014-08-12 14:36:40.086818 2014-08-13 18:06:06.862393 7f5c973a57a0 20 read_log 194209'14008 (194209'13950) delete 9e610ea1/benchmark_data_storage0111-ib.pocstor_41097_object25086/head//24 by osd.17.0:1374 2014-08-12 14:36:40.087209 2014-08-13 18:06:06.862405 7f5c973a57a0 20 read_log 194209'14009 (194209'13951) delete a2040ea1/benchmark_data_storage0111-ib.pocstor_41526_object11047/head//24 by osd.17.0:1375 2014-08-12 14:36:40.087609 2014-08-13 18:06:06.862417 7f5c973a57a0 20 read_log 194209'14010 (194209'13952) delete 2b050ea1/benchmark_data_storage0111-ib.pocstor_26659_object13169/head//24 by osd.17.0:1376 2014-08-12 14:36:40.091059 2014-08-13 18:06:06.862439 7f5c973a57a0 20 read_log 194209'14011 (194209'13953) delete 18160ea1/benchmark_data_storage0111-ib.pocstor_2228_object24838/head//24 by osd.17.0:1377 2014-08-12 14:36:40.091418 2014-08-13 18:06:06.862455 7f5c973a57a0 20 read_log 194209'14012 (194209'13954) delete a2d60ea1/benchmark_data_storage0111-ib.pocstor_41526_object13475/head//24 by osd.17.0:1378 2014-08-12 14:36:40.091766 2014-08-13 18:06:06.862468 7f5c973a57a0 20 read_log 194209'14013 (194209'13955) delete 2cf60ea1/benchmark_data_storage0111-ib.pocstor_41097_object6364/head//24 by osd.17.0:1379 2014-08-12 14:36:40.095118 2014-08-13 18:06:06.862480 7f5c973a57a0 20 read_log 194209'14014 (194209'13956) delete 5022ea1/benchmark_data_storage0111-ib.pocstor_41097_object78/head//24 by osd.17.0:1380 2014-08-12 14:36:40.095474 2014-08-13 18:06:06.862508 7f5c973a57a0 20 read_log 194209'14015 (194209'13957) delete 5bbc2ea1/benchmark_data_storage0111-ib.pocstor_41526_object10379/head//24 by osd.17.0:1381 2014-08-12 14:36:40.096353 2014-08-13 18:06:06.862522 7f5c973a57a0 20 read_log 194209'14016 (194209'13958) delete 92033ea1/benchmark_data_storage0111-ib.pocstor_41097_object2873/head//24 by osd.17.0:1382 2014-08-12 14:36:40.099416 2014-08-13 18:06:06.862536 7f5c973a57a0 20 read_log 194209'14017 (194209'13959) delete b2954ea1/benchmark_data_storage0111-ib.pocstor_62621_object1/head//24 by osd.17.0:1383 2014-08-12 14:36:40.099790 2014-08-13 18:06:06.862569 7f5c973a57a0 20 read_log 194209'14018 (194209'13960) delete 64f54ea1/benchmark_data_storage0111-ib.pocstor_62621_object29429/head//24 by osd.17.0:1384 2014-08-12 14:36:40.100593 2014-08-13 18:06:06.862584 7f5c973a57a0 20 read_log 194209'14019 (194209'13961) delete f7594ea1/benchmark_data_storage0111-ib.pocstor_62621_object19945/head//24 by osd.17.0:1385 2014-08-12 14:36:40.103651 2014-08-13 18:06:06.862597 7f5c973a57a0 20 read_log 194209'14020 (194209'13966) delete ea1b5ea1/benchmark_data_storage0111-ib.pocstor_62621_object12922/head//24 by osd.17.0:1386 2014-08-12 14:36:40.104947 2014-08-13 18:06:06.862609 7f5c973a57a0 20 read_log 194209'14021 (194209'13969) delete e6bc6ea1/benchmark_data_storage0111-ib.pocstor_62621_object30894/head//24 by osd.17.0:1387 2014-08-12 14:36:40.105976 2014-08-13 18:06:06.862621 7f5c973a57a0 20 read_log 194209'14022 (194209'13974) delete 10519ea1/benchmark_data_storage0111-ib.pocstor_62621_object4910/head//24 by osd.17.0:1388 2014-08-12 14:36:40.109593 2014-08-13 18:06:06.862634 7f5c973a57a0 20 read_log 194209'14023 (194209'13977) delete 4d12aea1/benchmark_data_storage0111-ib.pocstor_62621_object22934/head//24 by osd.17.0:1389 2014-08-12 14:36:40.110396 2014-08-13 18:06:06.862646 7f5c973a57a0 20 read_log 194209'14024 (194209'13979) delete 6694aea1/benchmark_data_storage0111-ib.pocstor_62621_object12128/head//24 by osd.17.0:1390 2014-08-12 14:36:40.111164 2014-08-13 18:06:06.862658 7f5c973a57a0 20 read_log 194209'14025 (194209'13981) delete 1b72bea1/benchmark_data_storage0111-ib.pocstor_62621_object931/head//24 by osd.17.0:1391 2014-08-12 14:36:40.114126 2014-08-13 18:06:06.862671 7f5c973a57a0 20 read_log 194209'14026 (194209'13988) delete 8ef712a1/benchmark_data_storage0111-ib.pocstor_62621_object27340/head//24 by osd.17.0:1392 2014-08-12 14:36:40.118331 2014-08-13 18:06:06.862683 7f5c973a57a0 20 read_log 194209'14027 (194209'13989) delete 9a3912a1/benchmark_data_storage0111-ib.pocstor_62621_object5302/head//24 by osd.17.0:1393 2014-08-12 14:36:40.118692 2014-08-13 18:06:06.862695 7f5c973a57a0 20 read_log 194209'14028 (194209'13990) delete 11c822a1/benchmark_data_storage0111-ib.pocstor_62621_object6159/head//24 by osd.17.0:1394 2014-08-12 14:36:40.119202 2014-08-13 18:06:06.862707 7f5c973a57a0 20 read_log 194209'14029 (194209'13997) delete 1a3f42a1/benchmark_data_storage0111-ib.pocstor_62621_object15071/head//24 by osd.17.0:1395 2014-08-12 14:36:40.123630 2014-08-13 18:06:06.862719 7f5c973a57a0 20 read_log 194209'14030 (194209'14000) delete 7a8462a1/benchmark_data_storage0111-ib.pocstor_62621_object13115/head//24 by osd.17.0:1396 2014-08-12 14:36:40.124602 2014-08-13 18:06:06.862759 7f5c973a57a0 20 read_log 194209'14031 (194209'13991) delete 6f2f22a1/benchmark_data_storage0111-ib.pocstor_41526_object6745/head//24 by osd.17.0:1397 2014-08-12 14:36:50.230403 2014-08-13 18:06:06.862776 7f5c973a57a0 20 read_log 194209'14032 (194209'13992) delete c3c532a1/benchmark_data_storage0111-ib.pocstor_41526_object19695/head//24 by osd.17.0:1398 2014-08-12 14:36:50.230902 2014-08-13 18:06:06.862789 7f5c973a57a0 20 read_log 194209'14033 (194209'13993) delete 180a32a1/benchmark_data_storage0111-ib.pocstor_41097_object12451/head//24 by osd.17.0:1399 2014-08-12 14:36:50.231235 2014-08-13 18:06:06.862803 7f5c973a57a0 20 read_log 194209'14034 (194209'13994) delete a42e32a1/benchmark_data_storage0111-ib.pocstor_2228_object28707/head//24 by osd.17.0:1400 2014-08-12 14:36:50.231504 2014-08-13 18:06:06.862816 7f5c973a57a0 20 read_log 194209'14035 (194209'13995) delete bd9042a1/benchmark_data_storage0111-ib.pocstor_26659_object14185/head//24 by osd.17.0:1401 2014-08-12 14:36:50.234662 2014-08-13 18:06:06.862828 7f5c973a57a0 20 read_log 194209'14036 (194209'13996) delete 936942a1/benchmark_data_storage0111-ib.pocstor_2228_object13883/head//24 by osd.17.0:1402 2014-08-12 14:36:50.235023 2014-08-13 18:06:06.862840 7f5c973a57a0 20 read_log 194209'14037 (194209'13998) delete c21a52a1/benchmark_data_storage0111-ib.pocstor_41526_object17200/head//24 by osd.17.0:1403 2014-08-12 14:36:50.235703 2014-08-13 18:06:06.862860 7f5c973a57a0 20 read_log 194209'14038 (194209'13999) delete df3e52a1/benchmark_data_storage0111-ib.pocstor_2228_object20228/head//24 by osd.17.0:1404 2014-08-12 14:36:50.235987 2014-08-13 18:06:06.862873 7f5c973a57a0 20 read_log 194209'14039 (194209'14001) delete bc8462a1/benchmark_data_storage0111-ib.pocstor_41097_object168/head//24 by osd.17.0:1405 2014-08-12 14:36:50.239037 2014-08-13 18:06:06.862885 7f5c973a57a0 20 read_log 194209'14040 (194209'14002) delete 9da562a1/benchmark_data_storage0111-ib.pocstor_26659_object5256/head//24 by osd.17.0:1406 2014-08-12 14:36:50.239395 2014-08-13 18:06:06.862898 7f5c973a57a0 20 read_log 194209'14041 (194209'14005) delete 662962a1/benchmark_data_storage0111-ib.pocstor_41526_object7299/head//24 by osd.17.0:1407 2014-08-12 14:36:50.239691 2014-08-13 18:06:06.862910 7f5c973a57a0 20 read_log 194209'14042 (194209'13962) delete 43b4ea1/benchmark_data_storage0111-ib.pocstor_2228_object23596/head//24 by osd.17.0:1408 2014-08-12 14:36:50.267483 2014-08-13 18:06:06.862949 7f5c973a57a0 20 read_log 194209'14043 (194209'13963) delete fcf15ea1/benchmark_data_storage0111-ib.pocstor_41526_object5282/head//24 by osd.17.0:1409 2014-08-12 14:36:50.267805 2014-08-13 18:06:06.862967 7f5c973a57a0 20 read_log 194209'14044 (194209'13964) delete cc35ea1/benchmark_data_storage0111-ib.pocstor_2228_object2619/head//24 by osd.17.0:1410 2014-08-12 14:36:50.268127 2014-08-13 18:06:06.862979 7f5c973a57a0 20 read_log 194209'14045 (194209'13965) delete 986a5ea1/benchmark_data_storage0111-ib.pocstor_2228_object29931/head//24 by osd.17.0:1411 2014-08-12 14:36:50.268438 2014-08-13 18:06:06.862994 7f5c973a57a0 20 read_log 194209'14046 (194209'13967) delete 44fb5ea1/benchmark_data_storage0111-ib.pocstor_41526_object12156/head//24 by osd.17.0:1412 2014-08-12 14:36:50.270990 2014-08-13 18:06:06.863007 7f5c973a57a0 20 read_log 194209'14047 (194209'13968) delete c036ea1/benchmark_data_storage0111-ib.pocstor_2228_object21827/head//24 by osd.17.0:1413 2014-08-12 14:36:50.271296 2014-08-13 18:06:06.863019 7f5c973a57a0 20 read_log 194209'14048 (194209'13970) delete 7fd97ea1/benchmark_data_storage0111-ib.pocstor_41097_object16875/head//24 by osd.17.0:1414 2014-08-12 14:36:50.271596 2014-08-13 18:06:06.863031 7f5c973a57a0 20 read_log 194209'14049 (194209'13971) delete 476c7ea1/benchmark_data_storage0111-ib.pocstor_41526_object21193/head//24 by osd.17.0:1415 2014-08-12 14:36:50.272140 2014-08-13 18:06:06.863044 7f5c973a57a0 20 read_log 194209'14050 (194209'13972) delete 42e98ea1/benchmark_data_storage0111-ib.pocstor_41526_object11915/head//24 by osd.17.0:1416 2014-08-12 14:36:50.274663 2014-08-13 18:06:06.863056 7f5c973a57a0 20 read_log 194209'14051 (194209'13973) delete 4cdd8ea1/benchmark_data_storage0111-ib.pocstor_41097_object17142/head//24 by osd.17.0:1417 2014-08-12 14:36:50.274946 2014-08-13 18:06:06.863071 7f5c973a57a0 20 read_log 194209'14052 (194209'13975) delete 6ff19ea1/benchmark_data_storage0111-ib.pocstor_2228_object17007/head//24 by osd.17.0:1418 2014-08-12 14:36:50.275257 2014-08-13 18:06:06.863084 7f5c973a57a0 20 read_log 194209'14053 (194209'13976) delete f8c0aea1/benchmark_data_storage0111-ib.pocstor_2228_object30754/head//24 by osd.17.0:1419 2014-08-12 14:36:50.275791 2014-08-13 18:06:06.863096 7f5c973a57a0 20 read_log 194209'14054 (194209'13978) delete e554aea1/benchmark_data_storage0111-ib.pocstor_41097_object17375/head//24 by osd.17.0:1420 2014-08-12 14:36:50.278244 2014-08-13 18:06:06.863108 7f5c973a57a0 20 read_log 194209'14055 (194209'13980) delete 2d02bea1/benchmark_data_storage0111-ib.pocstor_2228_object5916/head//24 by osd.17.0:1421 2014-08-12 14:36:50.278528 2014-08-13 18:06:06.863121 7f5c973a57a0 20 read_log 194209'14056 (194209'13982) delete eefcbea1/benchmark_data_storage0111-ib.pocstor_41526_object13025/head//24 by osd.17.0:1422 2014-08-12 14:36:50.278803 2014-08-13 18:06:06.863133 7f5c973a57a0 20 read_log 194209'14057 (194209'13983) delete 1660dea1/benchmark_data_storage0111-ib.pocstor_41097_object8507/head//24 by osd.17.0:1423 2014-08-12 14:36:50.279330 2014-08-13 18:06:06.863162 7f5c973a57a0 20 read_log 194209'14058 (194209'13984) delete c572fea1/benchmark_data_storage0111-ib.pocstor_26659_object21245/head//24 by osd.17.0:1424 2014-08-12 14:36:50.281811 2014-08-13 18:06:06.863178 7f5c973a57a0 20 read_log 194209'14059 (194209'13985) delete cd5efea1/benchmark_data_storage0111-ib.pocstor_2228_object27695/head//24 by osd.17.0:1425 2014-08-12 14:36:50.282098 2014-08-13 18:06:06.863198 7f5c973a57a0 20 read_log 194215'14060 (194215'14060) modify 2a1/hit_set_24.2a1_archive_2014-08-12 15:32:25.911312_2014-08-12 15:46:28.833325/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:28.833985 2014-08-13 18:06:06.863216 7f5c973a57a0 20 read_log 194215'14061 (193305'13888) delete 2a1/hit_set_24.2a1_archive_2014-08-11 18:38:33.126704_2014-08-11 18:43:37.373511/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:46:28.833985 2014-08-13 18:06:06.863230 7f5c973a57a0 20 read_log 194215'14062 (194209'13986) delete b1dd02a1/benchmark_data_storage0111-ib.pocstor_41097_object23331/head//24 by client.2174595.0:138494 0.000000 2014-08-13 18:06:06.863253 7f5c973a57a0 20 read_log 194215'14063 (194209'13987) delete af6112a1/benchmark_data_storage0111-ib.pocstor_2228_object3682/head//24 by client.2174595.0:138496 0.000000 2014-08-13 18:06:06.863268 7f5c973a57a0 20 read_log 194215'14064 (192888'13631) clean dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by osd.17.0:2349 2014-08-12 15:46:28.850046 2014-08-13 18:06:06.863281 7f5c973a57a0 20 read_log 194215'14065 (194215'14064) delete dbfc62a1/benchmark_data_storage0111-ib.pocstor_41097_object12872/head//24 by client.2174595.0:138498 0.000000 2014-08-13 18:06:06.863300 7f5c973a57a0 20 read_log 194215'14066 (192888'13811) clean 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by osd.17.0:2351 2014-08-12 15:46:28.858047 2014-08-13 18:06:06.863313 7f5c973a57a0 20 read_log 194215'14067 (194215'14066) delete 677072a1/benchmark_data_storage0111-ib.pocstor_2228_object14816/head//24 by client.2174595.0:138500 0.000000 2014-08-13 18:06:06.863325 7f5c973a57a0 20 read_log 194215'14068 (192834'13354) clean e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by osd.17.0:2353 2014-08-12 15:46:28.866034 2014-08-13 18:06:06.863337 7f5c973a57a0 20 read_log 194215'14069 (194215'14068) delete e80572a1/benchmark_data_storage0111-ib.pocstor_41526_object9759/head//24 by client.2174595.0:138502 0.000000 2014-08-13 18:06:06.863348 7f5c973a57a0 20 read_log 194215'14070 (192834'12895) clean 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by osd.17.0:2355 2014-08-12 15:46:28.874075 2014-08-13 18:06:06.863361 7f5c973a57a0 20 read_log 194215'14071 (194215'14070) delete 713f72a1/benchmark_data_storage0111-ib.pocstor_26659_object21621/head//24 by client.2174595.0:138504 0.000000 2014-08-13 18:06:06.863372 7f5c973a57a0 20 read_log 194215'14072 (193305'13926) clean ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by osd.17.0:2357 2014-08-12 15:46:28.882296 2014-08-13 18:06:06.863384 7f5c973a57a0 20 read_log 194215'14073 (194215'14072) delete ad5482a1/benchmark_data_storage0111-ib.pocstor_62621_object10316/head//24 by client.2174595.0:138506 0.000000 2014-08-13 18:06:06.863395 7f5c973a57a0 20 read_log 194215'14074 (193305'13921) clean dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by osd.17.0:2359 2014-08-12 15:46:28.890285 2014-08-13 18:06:06.863407 7f5c973a57a0 20 read_log 194215'14075 (194215'14074) delete dbab82a1/benchmark_data_storage0111-ib.pocstor_62621_object7415/head//24 by client.2174595.0:138508 0.000000 2014-08-13 18:06:06.863418 7f5c973a57a0 20 read_log 194215'14076 (192888'13667) clean 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by osd.17.0:2361 2014-08-12 15:46:28.897968 2014-08-13 18:06:06.863438 7f5c973a57a0 20 read_log 194215'14077 (194215'14076) delete 817e82a1/benchmark_data_storage0111-ib.pocstor_41097_object28720/head//24 by client.2174595.0:138510 0.000000 2014-08-13 18:06:06.863450 7f5c973a57a0 20 read_log 194215'14078 (192834'13382) clean 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by osd.17.0:2363 2014-08-12 15:46:28.905241 2014-08-13 18:06:06.863462 7f5c973a57a0 20 read_log 194215'14079 (194215'14078) delete 721492a1/benchmark_data_storage0111-ib.pocstor_41526_object12470/head//24 by client.2174595.0:138512 0.000000 2014-08-13 18:06:06.863473 7f5c973a57a0 20 read_log 194215'14080 (192834'13308) clean 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by osd.17.0:2365 2014-08-12 15:46:28.913600 2014-08-13 18:06:06.863486 7f5c973a57a0 20 read_log 194215'14081 (194215'14080) delete 99c592a1/benchmark_data_storage0111-ib.pocstor_41526_object4837/head//24 by client.2174595.0:138514 0.000000 2014-08-13 18:06:06.863508 7f5c973a57a0 20 read_log 194215'14082 (192888'13789) clean b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by osd.17.0:2367 2014-08-12 15:46:28.921566 2014-08-13 18:06:06.863520 7f5c973a57a0 20 read_log 194215'14083 (194215'14082) delete b45892a1/benchmark_data_storage0111-ib.pocstor_2228_object1835/head//24 by client.2174595.0:138516 0.000000 2014-08-13 18:06:06.863531 7f5c973a57a0 20 read_log 194215'14084 (192834'12862) clean 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by osd.17.0:2369 2014-08-12 15:46:28.929405 2014-08-13 18:06:06.863543 7f5c973a57a0 20 read_log 194215'14085 (194215'14084) delete 4509a2a1/benchmark_data_storage0111-ib.pocstor_26659_object16636/head//24 by client.2174595.0:138518 0.000000 2014-08-13 18:06:06.863554 7f5c973a57a0 20 read_log 194215'14086 (193305'13933) clean 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by osd.17.0:2371 2014-08-12 15:46:28.937425 2014-08-13 18:06:06.863566 7f5c973a57a0 20 read_log 194215'14087 (194215'14086) delete 333b2a1/benchmark_data_storage0111-ib.pocstor_62621_object15826/head//24 by client.2174595.0:138520 0.000000 2014-08-13 18:06:06.863577 7f5c973a57a0 20 read_log 194215'14088 (192888'13650) clean 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by osd.17.0:2373 2014-08-12 15:46:28.946083 2014-08-13 18:06:06.863589 7f5c973a57a0 20 read_log 194215'14089 (194215'14088) delete 8488b2a1/benchmark_data_storage0111-ib.pocstor_41097_object22080/head//24 by client.2174595.0:138522 0.000000 2014-08-13 18:06:06.863600 7f5c973a57a0 20 read_log 194215'14090 (192834'12846) clean b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by osd.17.0:2375 2014-08-12 15:46:28.953599 2014-08-13 18:06:06.863613 7f5c973a57a0 20 read_log 194215'14091 (194215'14090) delete b122c2a1/benchmark_data_storage0111-ib.pocstor_26659_object14029/head//24 by client.2174595.0:138524 0.000000 2014-08-13 18:06:06.863624 7f5c973a57a0 20 read_log 194215'14092 (192888'13666) clean 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by osd.17.0:2377 2014-08-12 15:46:28.961737 2014-08-13 18:06:06.863636 7f5c973a57a0 20 read_log 194215'14093 (194215'14092) delete 6325c2a1/benchmark_data_storage0111-ib.pocstor_41097_object26511/head//24 by client.2174595.0:138526 0.000000 2014-08-13 18:06:06.863647 7f5c973a57a0 20 read_log 194215'14094 (192834'12803) clean ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by osd.17.0:2379 2014-08-12 15:46:28.970540 2014-08-13 18:06:06.863659 7f5c973a57a0 20 read_log 194215'14095 (194215'14094) delete ddeac2a1/benchmark_data_storage0111-ib.pocstor_26659_object9493/head//24 by client.2174595.0:138528 0.000000 2014-08-13 18:06:06.863670 7f5c973a57a0 20 read_log 194215'14096 (192888'13622) clean e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by osd.17.0:2381 2014-08-12 15:46:28.978584 2014-08-13 18:06:06.863709 7f5c973a57a0 20 read_log 194215'14097 (194215'14096) delete e221d2a1/benchmark_data_storage0111-ib.pocstor_41097_object9314/head//24 by client.2174595.0:138530 0.000000 2014-08-13 18:06:06.863724 7f5c973a57a0 20 read_log 194215'14098 (192888'13827) clean 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by osd.17.0:2383 2014-08-12 15:46:28.986409 2014-08-13 18:06:06.863737 7f5c973a57a0 20 read_log 194215'14099 (194215'14098) delete 5d88d2a1/benchmark_data_storage0111-ib.pocstor_2228_object22422/head//24 by client.2174595.0:138532 0.000000 2014-08-13 18:06:06.863748 7f5c973a57a0 20 read_log 194215'14100 (192834'13401) clean 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by osd.17.0:2385 2014-08-12 15:46:28.994554 2014-08-13 18:06:06.863762 7f5c973a57a0 20 read_log 194215'14101 (194215'14100) delete 4e2e2a1/benchmark_data_storage0111-ib.pocstor_41526_object14829/head//24 by client.2174595.0:138534 0.000000 2014-08-13 18:06:06.863773 7f5c973a57a0 20 read_log 194215'14102 (192888'13655) clean 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by osd.17.0:2387 2014-08-12 15:46:29.001994 2014-08-13 18:06:06.863788 7f5c973a57a0 20 read_log 194215'14103 (194215'14102) delete 6059e2a1/benchmark_data_storage0111-ib.pocstor_41097_object22921/head//24 by client.2174595.0:138536 0.000000 2014-08-13 18:06:06.863799 7f5c973a57a0 20 read_log 194215'14104 (193305'13923) clean 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by osd.17.0:2389 2014-08-12 15:46:29.010150 2014-08-13 18:06:06.863813 7f5c973a57a0 20 read_log 194215'14105 (194215'14104) delete 3dabe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9534/head//24 by client.2174595.0:138538 0.000000 2014-08-13 18:06:06.863824 7f5c973a57a0 20 read_log 194215'14106 (193305'13924) clean 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by osd.17.0:2391 2014-08-12 15:46:29.018287 2014-08-13 18:06:06.863838 7f5c973a57a0 20 read_log 194215'14107 (194215'14106) delete 39bbe2a1/benchmark_data_storage0111-ib.pocstor_62621_object9741/head//24 by client.2174595.0:138540 0.000000 2014-08-13 18:06:06.863849 7f5c973a57a0 20 read_log 194215'14108 (192834'13328) clean baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by osd.17.0:2393 2014-08-12 15:46:29.026601 2014-08-13 18:06:06.863863 7f5c973a57a0 20 read_log 194215'14109 (194215'14108) delete baeee2a1/benchmark_data_storage0111-ib.pocstor_41526_object6787/head//24 by client.2174595.0:138542 0.000000 2014-08-13 18:06:06.863874 7f5c973a57a0 20 read_log 194215'14110 (193305'13922) clean 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by osd.17.0:2395 2014-08-12 15:46:29.034236 2014-08-13 18:06:06.863887 7f5c973a57a0 20 read_log 194215'14111 (194215'14110) delete 4c2106a1/benchmark_data_storage0111-ib.pocstor_62621_object8241/head//24 by client.2174595.0:138544 0.000000 2014-08-13 18:06:06.863922 7f5c973a57a0 20 read_log 194215'14112 (192888'13653) clean 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by osd.17.0:2397 2014-08-12 15:46:29.042138 2014-08-13 18:06:06.863938 7f5c973a57a0 20 read_log 194215'14113 (194215'14112) delete 466e06a1/benchmark_data_storage0111-ib.pocstor_41097_object22821/head//24 by client.2174595.0:138546 0.000000 2014-08-13 18:06:06.863950 7f5c973a57a0 20 read_log 194215'14114 (192888'13842) clean 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by osd.17.0:2399 2014-08-12 15:46:29.049938 2014-08-13 18:06:06.863963 7f5c973a57a0 20 read_log 194215'14115 (194215'14114) delete 112216a1/benchmark_data_storage0111-ib.pocstor_2228_object30739/head//24 by client.2174595.0:138548 0.000000 2014-08-13 18:06:06.863976 7f5c973a57a0 20 read_log 194215'14116 (192888'13615) clean 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by osd.17.0:2401 2014-08-12 15:46:29.057974 2014-08-13 18:06:06.863999 7f5c973a57a0 20 read_log 194215'14117 (194215'14116) delete 3f9316a1/benchmark_data_storage0111-ib.pocstor_41097_object7778/head//24 by client.2174595.0:138550 0.000000 2014-08-13 18:06:06.864013 7f5c973a57a0 20 read_log 194215'14118 (192888'13647) clean a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by osd.17.0:2403 2014-08-12 15:46:29.065809 2014-08-13 18:06:06.864025 7f5c973a57a0 20 read_log 194215'14119 (194215'14118) delete a4e516a1/benchmark_data_storage0111-ib.pocstor_41097_object18795/head//24 by client.2174595.0:138552 0.000000 2014-08-13 18:06:06.864036 7f5c973a57a0 20 read_log 194215'14120 (192888'13639) clean 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by osd.17.0:2405 2014-08-12 15:46:29.073334 2014-08-13 18:06:06.864051 7f5c973a57a0 20 read_log 194215'14121 (194215'14120) delete 93bd16a1/benchmark_data_storage0111-ib.pocstor_41097_object16208/head//24 by client.2174595.0:138554 0.000000 2014-08-13 18:06:06.864062 7f5c973a57a0 20 read_log 194215'14122 (192888'13839) clean 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by osd.17.0:2407 2014-08-12 15:46:29.080624 2014-08-13 18:06:06.864074 7f5c973a57a0 20 read_log 194215'14123 (194215'14122) delete 30cf16a1/benchmark_data_storage0111-ib.pocstor_2228_object29755/head//24 by client.2174595.0:138556 0.000000 2014-08-13 18:06:06.864104 7f5c973a57a0 20 read_log 194215'14124 (192834'12793) clean 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by osd.17.0:2409 2014-08-12 15:46:29.088641 2014-08-13 18:06:06.864119 7f5c973a57a0 20 read_log 194215'14125 (194215'14124) delete 20ef16a1/benchmark_data_storage0111-ib.pocstor_26659_object8467/head//24 by client.2174595.0:138558 0.000000 2014-08-13 18:06:06.864130 7f5c973a57a0 20 read_log 194215'14126 (192888'13835) clean 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by osd.17.0:2411 2014-08-12 15:46:29.095567 2014-08-13 18:06:06.864143 7f5c973a57a0 20 read_log 194215'14127 (194215'14126) delete 2b6726a1/benchmark_data_storage0111-ib.pocstor_2228_object26571/head//24 by client.2174595.0:138560 0.000000 2014-08-13 18:06:06.864161 7f5c973a57a0 20 read_log 194215'14128 (192834'12786) clean 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by osd.17.0:2413 2014-08-12 15:46:29.103327 2014-08-13 18:06:06.864174 7f5c973a57a0 20 read_log 194215'14129 (194215'14128) delete 394a26a1/benchmark_data_storage0111-ib.pocstor_26659_object7585/head//24 by client.2174595.0:138562 0.000000 2014-08-13 18:06:06.864185 7f5c973a57a0 20 read_log 194215'14130 (192834'13278) clean 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by osd.17.0:2415 2014-08-12 15:46:29.109627 2014-08-13 18:06:06.864198 7f5c973a57a0 20 read_log 194215'14131 (194215'14130) delete 1be26a1/benchmark_data_storage0111-ib.pocstor_41526_object588/head//24 by client.2174595.0:138564 0.000000 2014-08-13 18:06:06.864208 7f5c973a57a0 20 read_log 194215'14132 (192834'12881) clean eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by osd.17.0:2417 2014-08-12 15:46:29.116724 2014-08-13 18:06:06.864221 7f5c973a57a0 20 read_log 194215'14133 (194215'14132) delete eeff26a1/benchmark_data_storage0111-ib.pocstor_26659_object20134/head//24 by client.2174595.0:138566 0.000000 2014-08-13 18:06:06.864232 7f5c973a57a0 20 read_log 194215'14134 (192834'12831) clean c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by osd.17.0:2419 2014-08-12 15:46:29.124190 2014-08-13 18:06:06.864247 7f5c973a57a0 20 read_log 194215'14135 (194215'14134) delete c48336a1/benchmark_data_storage0111-ib.pocstor_26659_object12839/head//24 by client.2174595.0:138568 0.000000 2014-08-13 18:06:06.864258 7f5c973a57a0 20 read_log 194215'14136 (193305'13935) clean 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by osd.17.0:2421 2014-08-12 15:46:29.130871 2014-08-13 18:06:06.864283 7f5c973a57a0 20 read_log 194215'14137 (194215'14136) delete 472836a1/benchmark_data_storage0111-ib.pocstor_62621_object17911/head//24 by client.2174595.0:138570 0.000000 2014-08-13 18:06:06.864296 7f5c973a57a0 20 read_log 194215'14138 (192834'12853) clean f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by osd.17.0:2423 2014-08-12 15:46:29.138518 2014-08-13 18:06:06.864308 7f5c973a57a0 20 read_log 194215'14139 (194215'14138) delete f21a36a1/benchmark_data_storage0111-ib.pocstor_26659_object15264/head//24 by client.2174595.0:138572 0.000000 2014-08-13 18:06:06.864319 7f5c973a57a0 20 read_log 194215'14140 (193305'13915) clean 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by osd.17.0:2425 2014-08-12 15:46:29.144874 2014-08-13 18:06:06.864334 7f5c973a57a0 20 read_log 194215'14141 (194215'14140) delete 403146a1/benchmark_data_storage0111-ib.pocstor_62621_object547/head//24 by client.2174595.0:138574 0.000000 2014-08-13 18:06:06.864345 7f5c973a57a0 20 read_log 194215'14142 (192834'13317) clean 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by osd.17.0:2427 2014-08-12 15:46:29.153010 2014-08-13 18:06:06.864357 7f5c973a57a0 20 read_log 194215'14143 (194215'14142) delete 426146a1/benchmark_data_storage0111-ib.pocstor_41526_object5460/head//24 by client.2174595.0:138576 0.000000 2014-08-13 18:06:06.864368 7f5c973a57a0 20 read_log 194215'14144 (192834'13323) clean 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by osd.17.0:2429 2014-08-12 15:46:29.159837 2014-08-13 18:06:06.864380 7f5c973a57a0 20 read_log 194215'14145 (194215'14144) delete 39d446a1/benchmark_data_storage0111-ib.pocstor_41526_object6139/head//24 by client.2174595.0:138578 0.000000 2014-08-13 18:06:06.864404 7f5c973a57a0 20 read_log 194215'14146 (192834'12888) clean 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by osd.17.0:2431 2014-08-12 15:46:29.167925 2014-08-13 18:06:06.864417 7f5c973a57a0 20 read_log 194215'14147 (194215'14146) delete 75b746a1/benchmark_data_storage0111-ib.pocstor_26659_object20505/head//24 by client.2174595.0:138580 0.000000 2014-08-13 18:06:06.864443 7f5c973a57a0 20 read_log 194215'14148 (193305'13945) clean 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by osd.17.0:2433 2014-08-12 15:46:29.174867 2014-08-13 18:06:06.864459 7f5c973a57a0 20 read_log 194215'14149 (194215'14148) delete 683356a1/benchmark_data_storage0111-ib.pocstor_62621_object32499/head//24 by client.2174595.0:138582 0.000000 2014-08-13 18:06:06.864470 7f5c973a57a0 20 read_log 194215'14150 (192834'12836) clean d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by osd.17.0:2435 2014-08-12 15:46:29.182603 2014-08-13 18:06:06.864492 7f5c973a57a0 20 read_log 194215'14151 (194215'14150) delete d87556a1/benchmark_data_storage0111-ib.pocstor_26659_object13194/head//24 by client.2174595.0:138584 0.000000 2014-08-13 18:06:06.864507 7f5c973a57a0 20 read_log 194215'14152 (193305'13943) clean 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by osd.17.0:2437 2014-08-12 15:46:29.191884 2014-08-13 18:06:06.864524 7f5c973a57a0 20 read_log 194215'14153 (194215'14152) delete 7e2656a1/benchmark_data_storage0111-ib.pocstor_62621_object30232/head//24 by client.2174595.0:138586 0.000000 2014-08-13 18:06:06.864535 7f5c973a57a0 20 read_log 194215'14154 (192834'12778) clean 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by osd.17.0:2439 2014-08-12 15:46:29.200003 2014-08-13 18:06:06.864548 7f5c973a57a0 20 read_log 194215'14155 (194215'14154) delete 131756a1/benchmark_data_storage0111-ib.pocstor_26659_object6184/head//24 by client.2174595.0:138588 0.000000 2014-08-13 18:06:06.864559 7f5c973a57a0 20 read_log 194215'14156 (192888'13817) clean 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by osd.17.0:2441 2014-08-12 15:46:29.207379 2014-08-13 18:06:06.864578 7f5c973a57a0 20 read_log 194215'14157 (194215'14156) delete 7bac56a1/benchmark_data_storage0111-ib.pocstor_2228_object18743/head//24 by client.2174595.0:138590 0.000000 2014-08-13 18:06:06.864603 7f5c973a57a0 20 read_log 194215'14158 (193305'13919) clean 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by osd.17.0:2443 2014-08-12 15:46:29.214546 2014-08-13 18:06:06.864619 7f5c973a57a0 20 read_log 194215'14159 (194215'14158) delete 9f3366a1/benchmark_data_storage0111-ib.pocstor_62621_object5489/head//24 by client.2174595.0:138592 0.000000 2014-08-13 18:06:06.864631 7f5c973a57a0 20 read_log 194215'14160 (192834'12818) clean 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by osd.17.0:2445 2014-08-12 15:46:29.222557 2014-08-13 18:06:06.864653 7f5c973a57a0 20 read_log 194215'14161 (194215'14160) delete 8dcc66a1/benchmark_data_storage0111-ib.pocstor_26659_object11264/head//24 by client.2174595.0:138594 0.000000 2014-08-13 18:06:06.864665 7f5c973a57a0 20 read_log 194215'14162 (192888'13613) clean e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by osd.17.0:2447 2014-08-12 15:46:29.229873 2014-08-13 18:06:06.864677 7f5c973a57a0 20 read_log 194215'14163 (194215'14162) delete e9ed66a1/benchmark_data_storage0111-ib.pocstor_41097_object6395/head//24 by client.2174595.0:138596 0.000000 2014-08-13 18:06:06.864688 7f5c973a57a0 20 read_log 194215'14164 (192888'13623) clean 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by osd.17.0:2449 2014-08-12 15:46:29.237567 2014-08-13 18:06:06.864701 7f5c973a57a0 20 read_log 194215'14165 (194215'14164) delete 89de66a1/benchmark_data_storage0111-ib.pocstor_41097_object10069/head//24 by client.2174595.0:138598 0.000000 2014-08-13 18:06:06.864711 7f5c973a57a0 20 read_log 194215'14166 (192834'13352) clean 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by osd.17.0:2451 2014-08-12 15:46:29.246260 2014-08-13 18:06:06.864724 7f5c973a57a0 20 read_log 194215'14167 (194215'14166) delete 96686a1/benchmark_data_storage0111-ib.pocstor_41526_object9502/head//24 by client.2174595.0:138600 0.000000 2014-08-13 18:06:06.864734 7f5c973a57a0 20 read_log 194215'14168 (192888'13836) clean 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by osd.17.0:2453 2014-08-12 15:46:29.254727 2014-08-13 18:06:06.864747 7f5c973a57a0 20 read_log 194215'14169 (194215'14168) delete 5b25a6a1/benchmark_data_storage0111-ib.pocstor_2228_object26632/head//24 by client.2174595.0:138602 0.000000 2014-08-13 18:06:06.864758 7f5c973a57a0 20 read_log 194215'14170 (192888'13636) clean 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by osd.17.0:2455 2014-08-12 15:46:29.262525 2014-08-13 18:06:06.864794 7f5c973a57a0 20 read_log 194215'14171 (194215'14170) delete 18afa6a1/benchmark_data_storage0111-ib.pocstor_41097_object15994/head//24 by client.2174595.0:138604 0.000000 2014-08-13 18:06:06.864806 7f5c973a57a0 20 read_log 194215'14172 (192888'13642) clean 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by osd.17.0:2457 2014-08-12 15:46:29.270944 2014-08-13 18:06:06.864818 7f5c973a57a0 20 read_log 194215'14173 (194215'14172) delete 3182b6a1/benchmark_data_storage0111-ib.pocstor_41097_object16710/head//24 by client.2174595.0:138606 0.000000 2014-08-13 18:06:06.864829 7f5c973a57a0 20 read_log 194215'14174 (192834'13282) clean 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by osd.17.0:2459 2014-08-12 15:46:29.278928 2014-08-13 18:06:06.864842 7f5c973a57a0 20 read_log 194215'14175 (194215'14174) delete 8180c6a1/benchmark_data_storage0111-ib.pocstor_41526_object1338/head//24 by client.2174595.0:138608 0.000000 2014-08-13 18:06:06.864853 7f5c973a57a0 20 read_log 194215'14176 (192888'13646) clean 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by osd.17.0:2461 2014-08-12 15:46:29.286468 2014-08-13 18:06:06.864885 7f5c973a57a0 20 read_log 194215'14177 (194215'14176) delete 3ad0c6a1/benchmark_data_storage0111-ib.pocstor_41097_object18483/head//24 by client.2174595.0:138610 0.000000 2014-08-13 18:06:06.864920 7f5c973a57a0 20 read_log 194215'14178 (192834'13387) clean 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by osd.17.0:2463 2014-08-12 15:46:29.294533 2014-08-13 18:06:06.864940 7f5c973a57a0 20 read_log 194215'14179 (194215'14178) delete 54e3c6a1/benchmark_data_storage0111-ib.pocstor_41526_object13282/head//24 by client.2174595.0:138612 0.000000 2014-08-13 18:06:06.864952 7f5c973a57a0 20 read_log 194215'14180 (192834'12801) clean 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by osd.17.0:2465 2014-08-12 15:46:29.303555 2014-08-13 18:06:06.864964 7f5c973a57a0 20 read_log 194215'14181 (194215'14180) delete 3aac6a1/benchmark_data_storage0111-ib.pocstor_26659_object9317/head//24 by client.2174595.0:138614 0.000000 2014-08-13 18:06:06.864975 7f5c973a57a0 20 read_log 194215'14182 (192834'12832) clean 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by osd.17.0:2467 2014-08-12 15:46:29.311521 2014-08-13 18:06:06.864987 7f5c973a57a0 20 read_log 194215'14183 (194215'14182) delete 48d1d6a1/benchmark_data_storage0111-ib.pocstor_26659_object12859/head//24 by client.2174595.0:138616 0.000000 2014-08-13 18:06:06.864998 7f5c973a57a0 20 read_log 194215'14184 (192888'13626) clean 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by osd.17.0:2469 2014-08-12 15:46:29.319596 2014-08-13 18:06:06.865011 7f5c973a57a0 20 read_log 194215'14185 (194215'14184) delete 18bbd6a1/benchmark_data_storage0111-ib.pocstor_41097_object10993/head//24 by client.2174595.0:138618 0.000000 2014-08-13 18:06:06.865022 7f5c973a57a0 20 read_log 194215'14186 (192834'13423) clean 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by osd.17.0:2471 2014-08-12 15:46:29.327665 2014-08-13 18:06:06.865034 7f5c973a57a0 20 read_log 194215'14187 (194215'14186) delete 7a53e6a1/benchmark_data_storage0111-ib.pocstor_41526_object18141/head//24 by client.2174595.0:138620 0.000000 2014-08-13 18:06:06.865045 7f5c973a57a0 20 read_log 194215'14188 (192834'13345) clean 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by osd.17.0:2473 2014-08-12 15:46:29.335451 2014-08-13 18:06:06.865059 7f5c973a57a0 20 read_log 194215'14189 (194215'14188) delete 4e2ee6a1/benchmark_data_storage0111-ib.pocstor_41526_object8626/head//24 by client.2174595.0:138622 0.000000 2014-08-13 18:06:06.865070 7f5c973a57a0 20 read_log 194215'14190 (192888'13607) clean 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by osd.17.0:2475 2014-08-12 15:46:29.343472 2014-08-13 18:06:06.865083 7f5c973a57a0 20 read_log 194215'14191 (194215'14190) delete 3281f6a1/benchmark_data_storage0111-ib.pocstor_41097_object4459/head//24 by client.2174595.0:138624 0.000000 2014-08-13 18:06:06.865094 7f5c973a57a0 20 read_log 194215'14192 (192888'13648) clean 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by osd.17.0:2477 2014-08-12 15:46:29.350500 2014-08-13 18:06:06.865108 7f5c973a57a0 20 read_log 194215'14193 (194215'14192) delete 9d44f6a1/benchmark_data_storage0111-ib.pocstor_41097_object18799/head//24 by client.2174595.0:138626 0.000000 2014-08-13 18:06:06.865119 7f5c973a57a0 20 read_log 194215'14194 (192834'13425) clean cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by osd.17.0:2479 2014-08-12 15:46:29.357780 2014-08-13 18:06:06.865132 7f5c973a57a0 20 read_log 194215'14195 (194215'14194) delete cb89f6a1/benchmark_data_storage0111-ib.pocstor_41526_object18440/head//24 by client.2174595.0:138628 0.000000 2014-08-13 18:06:06.865143 7f5c973a57a0 20 read_log 194215'14196 (192888'13790) clean 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by osd.17.0:2481 2014-08-12 15:46:29.365905 2014-08-13 18:06:06.865171 7f5c973a57a0 20 read_log 194215'14197 (194215'14196) delete 545cf6a1/benchmark_data_storage0111-ib.pocstor_2228_object2253/head//24 by client.2174595.0:138630 0.000000 2014-08-13 18:06:06.865184 7f5c973a57a0 20 read_log 194215'14198 (192888'13831) clean d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by osd.17.0:2483 2014-08-12 15:46:29.373032 2014-08-13 18:06:06.865197 7f5c973a57a0 20 read_log 194215'14199 (194215'14198) delete d70ef6a1/benchmark_data_storage0111-ib.pocstor_2228_object24479/head//24 by client.2174595.0:138632 0.000000 2014-08-13 18:06:06.865208 7f5c973a57a0 20 read_log 194215'14200 (192834'13446) clean cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by osd.17.0:2485 2014-08-12 15:46:29.380974 2014-08-13 18:06:06.865220 7f5c973a57a0 20 read_log 194215'14201 (194215'14200) delete cf0ef6a1/benchmark_data_storage0111-ib.pocstor_41526_object22000/head//24 by client.2174595.0:138634 0.000000 2014-08-13 18:06:06.865232 7f5c973a57a0 20 read_log 194215'14202 (193305'13930) clean b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by osd.17.0:2487 2014-08-12 15:46:29.388285 2014-08-13 18:06:06.865244 7f5c973a57a0 20 read_log 194215'14203 (194215'14202) delete b6531aa1/benchmark_data_storage0111-ib.pocstor_62621_object13317/head//24 by client.2174595.0:138636 0.000000 2014-08-13 18:06:06.865261 7f5c973a57a0 20 read_log 194215'14204 (192888'13805) clean 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by osd.17.0:2489 2014-08-12 15:46:29.396769 2014-08-13 18:06:06.865274 7f5c973a57a0 20 read_log 194215'14205 (194215'14204) delete 86d23aa1/benchmark_data_storage0111-ib.pocstor_2228_object11077/head//24 by client.2174595.0:138638 0.000000 2014-08-13 18:06:06.865285 7f5c973a57a0 20 read_log 194215'14206 (192888'13821) clean 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by osd.17.0:2491 2014-08-12 15:46:29.404665 2014-08-13 18:06:06.865297 7f5c973a57a0 20 read_log 194215'14207 (194215'14206) delete 92173aa1/benchmark_data_storage0111-ib.pocstor_2228_object20031/head//24 by client.2174595.0:138640 0.000000 2014-08-13 18:06:06.865311 7f5c973a57a0 20 read_log 194215'14208 (192834'12860) clean d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by osd.17.0:2493 2014-08-12 15:46:29.412478 2014-08-13 18:06:06.865326 7f5c973a57a0 20 read_log 194215'14209 (194215'14208) delete d4b83aa1/benchmark_data_storage0111-ib.pocstor_26659_object16546/head//24 by client.2174595.0:138642 0.000000 2014-08-13 18:06:06.865337 7f5c973a57a0 20 read_log 194215'14210 (192834'13280) clean 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by osd.17.0:2495 2014-08-12 15:46:29.419639 2014-08-13 18:06:06.865350 7f5c973a57a0 20 read_log 194215'14211 (194215'14210) delete 19693aa1/benchmark_data_storage0111-ib.pocstor_41526_object1218/head//24 by client.2174595.0:138644 0.000000 2014-08-13 18:06:06.865365 7f5c973a57a0 20 read_log 194215'14212 (192888'13841) clean fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by osd.17.0:2497 2014-08-12 15:46:29.427988 2014-08-13 18:06:06.865378 7f5c973a57a0 20 read_log 194215'14213 (194215'14212) delete fb194aa1/benchmark_data_storage0111-ib.pocstor_2228_object30080/head//24 by client.2174595.0:138646 0.000000 2014-08-13 18:06:06.865395 7f5c973a57a0 20 read_log 194215'14214 (193305'13938) clean 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by osd.17.0:2499 2014-08-12 15:46:29.437909 2014-08-13 18:06:06.865411 7f5c973a57a0 20 read_log 194215'14215 (194215'14214) delete 96ae4aa1/benchmark_data_storage0111-ib.pocstor_62621_object24260/head//24 by client.2174595.0:138648 0.000000 2014-08-13 18:06:06.865455 7f5c973a57a0 20 read_log 194215'14216 (192888'13830) clean 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by osd.17.0:2501 2014-08-12 15:46:29.446323 2014-08-13 18:06:06.865512 7f5c973a57a0 20 read_log 194215'14217 (194215'14216) delete 9cfe5aa1/benchmark_data_storage0111-ib.pocstor_2228_object23923/head//24 by client.2174595.0:138650 0.000000 2014-08-13 18:06:06.865546 7f5c973a57a0 20 read_log 194215'14218 (192888'13621) clean 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by osd.17.0:2503 2014-08-12 15:46:29.454128 2014-08-13 18:06:06.865564 7f5c973a57a0 20 read_log 194215'14219 (194215'14218) delete 58af5aa1/benchmark_data_storage0111-ib.pocstor_41097_object8953/head//24 by client.2174595.0:138652 0.000000 2014-08-13 18:06:06.865576 7f5c973a57a0 20 read_log 194215'14220 (192834'13321) clean 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by osd.17.0:2505 2014-08-12 15:46:29.461270 2014-08-13 18:06:06.865603 7f5c973a57a0 20 read_log 194215'14221 (194215'14220) delete 3ace6aa1/benchmark_data_storage0111-ib.pocstor_41526_object5952/head//24 by client.2174595.0:138654 0.000000 2014-08-13 18:06:06.865642 7f5c973a57a0 20 read_log 194215'14222 (193305'13934) clean 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by osd.17.0:2507 2014-08-12 15:46:29.470272 2014-08-13 18:06:06.865656 7f5c973a57a0 20 read_log 194215'14223 (194215'14222) delete 388e7aa1/benchmark_data_storage0111-ib.pocstor_62621_object17419/head//24 by client.2174595.0:138656 0.000000 2014-08-13 18:06:06.865675 7f5c973a57a0 20 read_log 194215'14224 (192888'13804) clean a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by osd.17.0:2509 2014-08-12 15:46:29.476973 2014-08-13 18:06:06.865690 7f5c973a57a0 20 read_log 194215'14225 (194215'14224) delete a5b88aa1/benchmark_data_storage0111-ib.pocstor_2228_object10589/head//24 by client.2174595.0:138658 0.000000 2014-08-13 18:06:06.865702 7f5c973a57a0 20 read_log 194215'14226 (192834'12806) clean 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by osd.17.0:2511 2014-08-12 15:46:29.483962 2014-08-13 18:06:06.865725 7f5c973a57a0 20 read_log 194215'14227 (194215'14226) delete 9dea8aa1/benchmark_data_storage0111-ib.pocstor_26659_object10333/head//24 by client.2174595.0:138660 0.000000 2014-08-13 18:06:06.865740 7f5c973a57a0 20 read_log 194215'14228 (193305'13932) clean 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by osd.17.0:2513 2014-08-12 15:46:29.491554 2014-08-13 18:06:06.865752 7f5c973a57a0 20 read_log 194215'14229 (194215'14228) delete 8c4e8aa1/benchmark_data_storage0111-ib.pocstor_62621_object15496/head//24 by client.2174595.0:138662 0.000000 2014-08-13 18:06:06.865773 7f5c973a57a0 20 read_log 194215'14230 (192888'13802) clean 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by osd.17.0:2515 2014-08-12 15:46:29.498537 2014-08-13 18:06:06.865789 7f5c973a57a0 20 read_log 194215'14231 (194215'14230) delete 171f8aa1/benchmark_data_storage0111-ib.pocstor_2228_object10174/head//24 by client.2174595.0:138664 0.000000 2014-08-13 18:06:06.865800 7f5c973a57a0 20 read_log 194215'14232 (192888'13798) clean b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by osd.17.0:2517 2014-08-12 15:46:29.506305 2014-08-13 18:06:06.865813 7f5c973a57a0 20 read_log 194215'14233 (194215'14232) delete b2509aa1/benchmark_data_storage0111-ib.pocstor_2228_object5343/head//24 by client.2174595.0:138666 0.000000 2014-08-13 18:06:06.865828 7f5c973a57a0 20 read_log 194215'14234 (193305'13925) clean 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by osd.17.0:2519 2014-08-12 15:46:29.514597 2014-08-13 18:06:06.865841 7f5c973a57a0 20 read_log 194215'14235 (194215'14234) delete 40999aa1/benchmark_data_storage0111-ib.pocstor_62621_object9917/head//24 by client.2174595.0:138668 0.000000 2014-08-13 18:06:06.865852 7f5c973a57a0 20 read_log 194215'14236 (192834'12766) clean 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by osd.17.0:2521 2014-08-12 15:46:29.521884 2014-08-13 18:06:06.865890 7f5c973a57a0 20 read_log 194215'14237 (194215'14236) delete 2c2baaa1/benchmark_data_storage0111-ib.pocstor_26659_object4858/head//24 by client.2174595.0:138670 0.000000 2014-08-13 18:06:06.865917 7f5c973a57a0 20 read_log 194215'14238 (193305'13914) clean 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by osd.17.0:2523 2014-08-12 15:46:29.529893 2014-08-13 18:06:06.865949 7f5c973a57a0 20 read_log 194215'14239 (194215'14238) delete 2837baa1/benchmark_data_storage0111-ib.pocstor_62621_object473/head//24 by client.2174595.0:138672 0.000000 2014-08-13 18:06:06.865973 7f5c973a57a0 20 read_log 194215'14240 (192888'13669) clean 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by osd.17.0:2525 2014-08-12 15:46:29.537921 2014-08-13 18:06:06.865989 7f5c973a57a0 20 read_log 194215'14241 (194215'14240) delete 4911caa1/benchmark_data_storage0111-ib.pocstor_41097_object29070/head//24 by client.2174595.0:138674 0.000000 2014-08-13 18:06:06.866004 7f5c973a57a0 20 read_log 194215'14242 (193305'13940) clean 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by osd.17.0:2527 2014-08-12 15:46:29.544912 2014-08-13 18:06:06.866017 7f5c973a57a0 20 read_log 194215'14243 (194215'14242) delete 8021caa1/benchmark_data_storage0111-ib.pocstor_62621_object25900/head//24 by client.2174595.0:138676 0.000000 2014-08-13 18:06:06.866028 7f5c973a57a0 20 read_log 194215'14244 (192888'13815) clean 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by osd.17.0:2529 2014-08-12 15:46:29.551574 2014-08-13 18:06:06.866044 7f5c973a57a0 20 read_log 194215'14245 (194215'14244) delete 262caa1/benchmark_data_storage0111-ib.pocstor_2228_object18117/head//24 by client.2174595.0:138678 0.000000 2014-08-13 18:06:06.866055 7f5c973a57a0 20 read_log 194215'14246 (193305'13939) clean aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by osd.17.0:2531 2014-08-12 15:46:29.559693 2014-08-13 18:06:06.866067 7f5c973a57a0 20 read_log 194215'14247 (194215'14246) delete aef2caa1/benchmark_data_storage0111-ib.pocstor_62621_object24995/head//24 by client.2174595.0:138680 0.000000 2014-08-13 18:06:06.866078 7f5c973a57a0 20 read_log 194215'14248 (192888'13620) clean 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by osd.17.0:2533 2014-08-12 15:46:29.565482 2014-08-13 18:06:06.866090 7f5c973a57a0 20 read_log 194215'14249 (194215'14248) delete 29b4caa1/benchmark_data_storage0111-ib.pocstor_41097_object8706/head//24 by client.2174595.0:138682 0.000000 2014-08-13 18:06:06.866101 7f5c973a57a0 20 read_log 194215'14250 (194215'14250) modify 2a1/hit_set_24.2a1_archive_2014-08-12 15:46:28.834066_2014-08-12 17:24:40.103729/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.104530 2014-08-13 18:06:06.866123 7f5c973a57a0 20 read_log 194215'14251 (194215'14060) delete 2a1/hit_set_24.2a1_archive_2014-08-12 15:32:25.911312_2014-08-12 15:46:28.833325/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:40.104530 2014-08-13 18:06:06.866151 7f5c973a57a0 20 read_log 194215'14252 (0'0) promote 2829e2a1/benchmark_data_storage0111-ib.pocstor_34628_object82/head//24 by osd.17.0:5118 2014-08-12 17:24:40.109282 2014-08-13 18:06:06.866172 7f5c973a57a0 20 read_log 194215'14253 (194215'14252) modify 2829e2a1/benchmark_data_storage0111-ib.pocstor_34628_object82/head//24 by client.2193759.0:83 2014-08-12 17:24:40.096720 2014-08-13 18:06:06.866189 7f5c973a57a0 20 read_log 194215'14254 (0'0) promote a67eaea1/benchmark_data_storage0111-ib.pocstor_34628_object148/head//24 by osd.17.0:5120 2014-08-12 17:24:40.423123 2014-08-13 18:06:06.866201 7f5c973a57a0 20 read_log 194215'14255 (194215'14254) modify a67eaea1/benchmark_data_storage0111-ib.pocstor_34628_object148/head//24 by client.2193759.0:149 2014-08-12 17:24:40.415934 2014-08-13 18:06:06.866223 7f5c973a57a0 20 read_log 194370'14256 (0'0) promote 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by osd.17.0:5164 2014-08-12 20:30:14.431722 2014-08-13 18:06:06.866238 7f5c973a57a0 20 read_log 194370'14257 (194370'14256) modify 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by client.1508531.0:90 2014-08-12 20:30:14.423052 2014-08-13 18:06:06.866251 7f5c973a57a0 20 read_log 194370'14258 (0'0) promote e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by osd.17.0:5190 2014-08-12 20:30:19.248604 2014-08-13 18:06:06.866263 7f5c973a57a0 20 read_log 194370'14259 (194370'14258) modify e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by client.1508531.0:1295 2014-08-12 20:30:19.241079 2014-08-13 18:06:06.866275 7f5c973a57a0 20 read_log 194370'14260 (0'0) promote c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by osd.17.0:5202 2014-08-12 20:30:23.347203 2014-08-13 18:06:06.866290 7f5c973a57a0 20 read_log 194370'14261 (194370'14260) modify c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by client.1508531.0:2394 2014-08-12 20:30:23.338742 2014-08-13 18:06:06.866303 7f5c973a57a0 20 read_log 194370'14262 (0'0) promote 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by osd.17.0:5206 2014-08-12 20:30:23.625447 2014-08-13 18:06:06.866315 7f5c973a57a0 20 read_log 194370'14263 (194370'14262) modify 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by client.1508531.0:2471 2014-08-12 20:30:23.619503 2014-08-13 18:06:06.866330 7f5c973a57a0 20 read_log 194370'14264 (0'0) promote f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by osd.17.0:5212 2014-08-12 20:30:23.743929 2014-08-13 18:06:06.866345 7f5c973a57a0 20 read_log 194370'14265 (194370'14264) modify f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by client.1508531.0:2503 2014-08-12 20:30:23.736973 2014-08-13 18:06:06.866357 7f5c973a57a0 20 read_log 194370'14266 (0'0) promote 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by osd.17.0:5236 2014-08-12 20:30:27.039928 2014-08-13 18:06:06.866369 7f5c973a57a0 20 read_log 194370'14267 (194370'14266) modify 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by client.1508531.0:3407 2014-08-12 20:30:27.030616 2014-08-13 18:06:06.866385 7f5c973a57a0 20 read_log 194370'14268 (0'0) promote 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by osd.17.0:5238 2014-08-12 20:30:27.191047 2014-08-13 18:06:06.866400 7f5c973a57a0 20 read_log 194370'14269 (194370'14268) modify 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by client.1508531.0:3446 2014-08-12 20:30:27.183554 2014-08-13 18:06:06.866421 7f5c973a57a0 20 read_log 194370'14270 (0'0) promote 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by osd.17.0:5250 2014-08-12 20:30:29.308269 2014-08-13 18:06:06.866440 7f5c973a57a0 20 read_log 194370'14271 (194370'14270) modify 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by client.1508531.0:4003 2014-08-12 20:30:29.299649 2014-08-13 18:06:06.866453 7f5c973a57a0 20 read_log 194370'14272 (0'0) promote 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by osd.17.0:5252 2014-08-12 20:30:29.835072 2014-08-13 18:06:06.866475 7f5c973a57a0 20 read_log 194370'14273 (194370'14272) modify 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by client.1508531.0:4144 2014-08-12 20:30:29.826139 2014-08-13 18:06:06.866491 7f5c973a57a0 20 read_log 194370'14274 (0'0) promote c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by osd.17.0:5256 2014-08-12 20:30:30.112833 2014-08-13 18:06:06.866503 7f5c973a57a0 20 read_log 194370'14275 (194370'14274) modify c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by client.1508531.0:4204 2014-08-12 20:30:30.106220 2014-08-13 18:06:06.866523 7f5c973a57a0 20 read_log 194370'14276 (0'0) promote cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by osd.17.0:5308 2014-08-12 20:30:44.431443 2014-08-13 18:06:06.866547 7f5c973a57a0 20 read_log 194370'14277 (194370'14276) modify cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by client.1508531.0:7413 2014-08-12 20:30:44.421622 2014-08-13 18:06:06.866560 7f5c973a57a0 20 read_log 194370'14278 (0'0) promote f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by osd.17.0:5310 2014-08-12 20:30:46.538344 2014-08-13 18:06:06.866572 7f5c973a57a0 20 read_log 194370'14279 (194370'14278) modify f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by client.1508531.0:7803 2014-08-12 20:30:46.527237 2014-08-13 18:06:06.866585 7f5c973a57a0 20 read_log 194370'14280 (0'0) promote 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by osd.17.0:5338 2014-08-12 20:30:56.142438 2014-08-13 18:06:06.866597 7f5c973a57a0 20 read_log 194370'14281 (194370'14280) modify 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by client.1508531.0:9470 2014-08-12 20:30:56.134566 2014-08-13 18:06:06.866615 7f5c973a57a0 20 read_log 194370'14282 (0'0) promote d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by osd.17.0:5342 2014-08-12 20:30:56.610566 2014-08-13 18:06:06.866635 7f5c973a57a0 20 read_log 194370'14283 (194370'14282) modify d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by client.1508531.0:9549 2014-08-12 20:30:56.603705 2014-08-13 18:06:06.866651 7f5c973a57a0 20 read_log 194370'14284 (0'0) promote f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by osd.17.0:5374 2014-08-12 20:31:05.874503 2014-08-13 18:06:06.866664 7f5c973a57a0 20 read_log 194370'14285 (194370'14284) modify f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by client.1508531.0:11098 2014-08-12 20:31:05.865726 2014-08-13 18:06:06.866676 7f5c973a57a0 20 read_log 194370'14286 (0'0) promote f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by osd.17.0:5380 2014-08-12 20:31:07.367077 2014-08-13 18:06:06.866689 7f5c973a57a0 20 read_log 194370'14287 (194370'14286) modify f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by client.1508531.0:11343 2014-08-12 20:31:07.357266 2014-08-13 18:06:06.866701 7f5c973a57a0 20 read_log 194377'14288 (0'0) promote f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by osd.17.0:5410 2014-08-12 20:32:59.631872 2014-08-13 18:06:06.866714 7f5c973a57a0 20 read_log 194377'14289 (194377'14288) modify f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by client.2231190.0:562 2014-08-12 20:32:59.614726 2014-08-13 18:06:06.866726 7f5c973a57a0 20 read_log 194377'14290 (0'0) promote 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by osd.17.0:5434 2014-08-12 20:33:04.294938 2014-08-13 18:06:06.866738 7f5c973a57a0 20 read_log 194377'14291 (194377'14290) modify 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by client.2231190.0:1767 2014-08-12 20:33:04.287040 2014-08-13 18:06:06.866750 7f5c973a57a0 20 read_log 194377'14292 (0'0) promote 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by osd.17.0:5456 2014-08-12 20:33:08.917726 2014-08-13 18:06:06.866762 7f5c973a57a0 20 read_log 194377'14293 (194377'14292) modify 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by client.2231190.0:2909 2014-08-12 20:33:08.909638 2014-08-13 18:06:06.866774 7f5c973a57a0 20 read_log 194383'14294 (0'0) promote 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by osd.17.0:5536 2014-08-12 20:34:02.111147 2014-08-13 18:06:06.866797 7f5c973a57a0 20 read_log 194383'14295 (194383'14294) modify 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by client.2231190.0:6136 2014-08-12 20:34:02.103724 2014-08-13 18:06:06.866810 7f5c973a57a0 20 read_log 194393'14296 (0'0) promote 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by osd.17.0:5592 2014-08-12 20:47:27.899691 2014-08-13 18:06:06.866822 7f5c973a57a0 20 read_log 194393'14297 (194393'14296) modify 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by client.1064095.0:463 2014-08-12 20:47:27.892264 2014-08-13 18:06:06.866834 7f5c973a57a0 20 read_log 194393'14298 (0'0) promote 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by osd.17.0:5594 2014-08-12 20:49:26.683858 2014-08-13 18:06:06.866846 7f5c973a57a0 20 read_log 194393'14299 (194393'14298) modify 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by client.1064182.0:257 2014-08-12 20:49:26.673351 2014-08-13 18:06:06.866859 7f5c973a57a0 20 read_log 194393'14300 (0'0) promote 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by osd.17.0:5605 2014-08-12 20:52:33.500232 2014-08-13 18:06:06.866883 7f5c973a57a0 20 read_log 194393'14301 (194393'14300) modify 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by client.1064422.0:239 2014-08-12 20:52:33.493329 2014-08-13 18:06:06.866896 7f5c973a57a0 20 read_log 194397'14302 (0'0) promote 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by osd.17.0:5615 2014-08-12 20:53:03.782624 2014-08-13 18:06:06.866911 7f5c973a57a0 20 read_log 194397'14303 (194397'14302) modify 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by client.2235069.0:816 2014-08-12 20:53:03.772632 2014-08-13 18:06:06.866926 7f5c973a57a0 20 read_log 194416'14304 (0'0) promote 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by osd.17.0:5757 2014-08-13 15:28:31.743571 2014-08-13 18:06:06.866941 7f5c973a57a0 20 read_log 194416'14305 (194416'14304) modify 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by client.2441958.0:363 2014-08-13 15:28:31.734296 2014-08-13 18:06:06.866954 7f5c973a57a0 20 read_log 194416'14306 (0'0) promote 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by osd.17.0:5759 2014-08-13 15:28:32.066502 2014-08-13 18:06:06.866966 7f5c973a57a0 20 read_log 194416'14307 (194416'14306) modify 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by client.2441958.0:429 2014-08-13 15:28:32.058547 2014-08-13 18:06:06.866981 7f5c973a57a0 20 read_log 194416'14308 (0'0) promote bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by osd.17.0:5761 2014-08-13 15:28:32.728977 2014-08-13 18:06:06.866998 7f5c973a57a0 20 read_log 194416'14309 (194416'14308) modify bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by client.2441958.0:562 2014-08-13 15:28:32.721495 2014-08-13 18:06:06.867055 7f5c973a57a0 20 read_log 194416'14310 (0'0) promote 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by osd.17.0:5778 2014-08-13 15:28:34.932032 2014-08-13 18:06:06.867071 7f5c973a57a0 20 read_log 194416'14311 (194416'14310) modify 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by client.2441958.0:1110 2014-08-13 15:28:34.924839 2014-08-13 18:06:06.867099 7f5c973a57a0 20 read_log 194416'14312 (0'0) promote 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by osd.17.0:5791 2014-08-13 15:28:39.169035 2014-08-13 18:06:06.867112 7f5c973a57a0 20 read_log 194416'14313 (194416'14312) modify 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by client.2441958.0:2236 2014-08-13 15:28:39.160468 2014-08-13 18:06:06.867140 7f5c973a57a0 20 read_log 194416'14314 (0'0) promote b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by osd.17.0:5797 2014-08-13 15:28:40.247706 2014-08-13 18:06:06.867162 7f5c973a57a0 20 read_log 194416'14315 (194416'14314) modify b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by client.2441958.0:2517 2014-08-13 15:28:40.238988 2014-08-13 18:06:06.867184 7f5c973a57a0 20 read_log 194416'14316 (0'0) promote afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by osd.17.0:5799 2014-08-13 15:28:41.627499 2014-08-13 18:06:06.867200 7f5c973a57a0 20 read_log 194416'14317 (194416'14316) modify afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by client.2441958.0:2880 2014-08-13 15:28:41.620853 2014-08-13 18:06:06.867213 7f5c973a57a0 20 read_log 194416'14318 (0'0) promote 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by osd.17.0:5863 2014-08-13 15:29:06.699626 2014-08-13 18:06:06.867227 7f5c973a57a0 20 read_log 194416'14319 (194416'14318) modify 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by client.2441958.0:9320 2014-08-13 15:29:06.690738 2014-08-13 18:06:06.867242 7f5c973a57a0 20 read_log 194416'14320 (0'0) promote 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by osd.17.0:5885 2014-08-13 15:29:11.212498 2014-08-13 18:06:06.867255 7f5c973a57a0 20 read_log 194416'14321 (194416'14320) modify 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by client.2441958.0:10440 2014-08-13 15:29:11.205144 2014-08-13 18:06:06.867271 7f5c973a57a0 20 read_log 194416'14322 (0'0) promote b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by osd.17.0:5905 2014-08-13 15:29:16.096567 2014-08-13 18:06:06.867286 7f5c973a57a0 20 read_log 194416'14323 (194416'14322) modify b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by client.2441958.0:11646 2014-08-13 15:29:16.086396 2014-08-13 18:06:06.867298 7f5c973a57a0 20 read_log 194416'14324 (0'0) promote 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by osd.17.0:5933 2014-08-13 15:29:23.189638 2014-08-13 18:06:06.867310 7f5c973a57a0 20 read_log 194416'14325 (194416'14324) modify 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by client.2441958.0:13280 2014-08-13 15:29:23.179367 2014-08-13 18:06:06.867329 7f5c973a57a0 20 read_log 194416'14326 (0'0) promote d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by osd.17.0:5935 2014-08-13 15:29:24.233138 2014-08-13 18:06:06.867348 7f5c973a57a0 20 read_log 194416'14327 (194416'14326) modify d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by client.2441958.0:13528 2014-08-13 15:29:24.226232 2014-08-13 18:06:06.867366 7f5c973a57a0 20 read_log 194416'14328 (0'0) promote ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by osd.17.0:5947 2014-08-13 15:29:25.275428 2014-08-13 18:06:06.867381 7f5c973a57a0 20 read_log 194416'14329 (194416'14328) modify ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by client.2441958.0:13765 2014-08-13 15:29:25.265313 2014-08-13 18:06:06.867394 7f5c973a57a0 20 read_log 194416'14330 (0'0) promote e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by osd.17.0:5963 2014-08-13 15:29:28.306818 2014-08-13 18:06:06.867406 7f5c973a57a0 20 read_log 194416'14331 (194416'14330) modify e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by client.2441958.0:14523 2014-08-13 15:29:28.297506 2014-08-13 18:06:06.867419 7f5c973a57a0 20 read_log 194416'14332 (0'0) promote 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by osd.17.0:5981 2014-08-13 15:29:31.158489 2014-08-13 18:06:06.867436 7f5c973a57a0 20 read_log 194416'14333 (194416'14332) modify 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by client.2441958.0:15185 2014-08-13 15:29:31.148722 2014-08-13 18:06:06.867465 7f5c973a57a0 20 read_log 194416'14334 (0'0) promote 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by osd.17.0:6025 2014-08-13 15:30:12.366117 2014-08-13 18:06:06.867484 7f5c973a57a0 20 read_log 194416'14335 (194416'14334) modify 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by client.2441958.0:18473 2014-08-13 15:30:12.358498 2014-08-13 18:06:06.867496 7f5c973a57a0 20 read_log 194422'14336 (0'0) promote ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by osd.17.0:6047 2014-08-13 15:32:19.796684 2014-08-13 18:06:06.867509 7f5c973a57a0 20 read_log 194422'14337 (194422'14336) modify ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by client.1148203.0:1132 2014-08-13 15:32:19.788514 2014-08-13 18:06:06.867524 7f5c973a57a0 20 read_log 194422'14338 (0'0) promote baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by osd.17.0:6051 2014-08-13 15:32:20.401675 2014-08-13 18:06:06.867546 7f5c973a57a0 20 read_log 194422'14339 (194422'14338) modify baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by client.1148203.0:1278 2014-08-13 15:32:20.393588 2014-08-13 18:06:06.867563 7f5c973a57a0 20 read_log 194422'14340 (0'0) promote 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by osd.17.0:6079 2014-08-13 15:32:29.365901 2014-08-13 18:06:06.867581 7f5c973a57a0 20 read_log 194422'14341 (194422'14340) modify 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by client.1148203.0:3569 2014-08-13 15:32:29.356032 2014-08-13 18:06:06.867593 7f5c973a57a0 20 read_log 194422'14342 (0'0) promote b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by osd.17.0:6091 2014-08-13 15:32:29.903126 2014-08-13 18:06:06.867606 7f5c973a57a0 20 read_log 194422'14343 (194422'14342) modify b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by client.1148203.0:3708 2014-08-13 15:32:29.891182 2014-08-13 18:06:06.867632 7f5c973a57a0 20 read_log 194422'14344 (0'0) promote 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by osd.17.0:6115 2014-08-13 15:32:34.165788 2014-08-13 18:06:06.867645 7f5c973a57a0 20 read_log 194422'14345 (194422'14344) modify 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by client.1148203.0:4823 2014-08-13 15:32:34.155051 2014-08-13 18:06:06.867657 7f5c973a57a0 20 read_log 194422'14346 (0'0) promote 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by osd.17.0:6139 2014-08-13 15:32:40.804449 2014-08-13 18:06:06.867673 7f5c973a57a0 20 read_log 194422'14347 (194422'14346) modify 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by client.1148203.0:6124 2014-08-13 15:32:40.795858 2014-08-13 18:06:06.867687 7f5c973a57a0 20 read_log 194422'14348 (0'0) promote 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by osd.17.0:6195 2014-08-13 15:33:08.620257 2014-08-13 18:06:06.867700 7f5c973a57a0 20 read_log 194422'14349 (194422'14348) modify 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by client.1148203.0:9853 2014-08-13 15:33:08.610416 2014-08-13 18:06:06.867712 7f5c973a57a0 20 read_log 194422'14350 (0'0) promote 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by osd.17.0:6197 2014-08-13 15:33:10.060667 2014-08-13 18:06:06.867736 7f5c973a57a0 20 read_log 194422'14351 (194422'14350) modify 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by client.1148203.0:9957 2014-08-13 15:33:10.052128 2014-08-13 18:06:06.867749 7f5c973a57a0 20 read_log 194422'14352 (0'0) promote a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by osd.17.0:6209 2014-08-13 15:33:25.680868 2014-08-13 18:06:06.867762 7f5c973a57a0 20 read_log 194422'14353 (194422'14352) modify a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by client.1148203.0:10562 2014-08-13 15:33:25.671849 2014-08-13 18:06:06.867783 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.868268 7f5c973a57a0 10 read_log checking for missing items over interval (194370'14256,194422'14353] 2014-08-13 18:06:06.868317 7f5c973a57a0 15 read_log missing 194422'14353 (194422'14352) modify a17fd2a1/benchmark_data_storage0111-ib.pocstor_41231_object10561/head//24 by client.1148203.0:10562 2014-08-13 15:33:25.671849 2014-08-13 18:06:06.868358 7f5c973a57a0 15 read_log missing 194422'14351 (194422'14350) modify 64f7daa1/benchmark_data_storage0111-ib.pocstor_41231_object9956/head//24 by client.1148203.0:9957 2014-08-13 15:33:10.052128 2014-08-13 18:06:06.868392 7f5c973a57a0 15 read_log missing 194422'14349 (194422'14348) modify 27d066a1/benchmark_data_storage0111-ib.pocstor_41231_object9852/head//24 by client.1148203.0:9853 2014-08-13 15:33:08.610416 2014-08-13 18:06:06.868428 7f5c973a57a0 15 read_log missing 194422'14347 (194422'14346) modify 52facea1/benchmark_data_storage0111-ib.pocstor_41231_object6123/head//24 by client.1148203.0:6124 2014-08-13 15:32:40.795858 2014-08-13 18:06:06.868464 7f5c973a57a0 15 read_log missing 194422'14345 (194422'14344) modify 6596bea1/benchmark_data_storage0111-ib.pocstor_41231_object4822/head//24 by client.1148203.0:4823 2014-08-13 15:32:34.155051 2014-08-13 18:06:06.868499 7f5c973a57a0 15 read_log missing 194422'14343 (194422'14342) modify b2835aa1/benchmark_data_storage0111-ib.pocstor_41231_object3707/head//24 by client.1148203.0:3708 2014-08-13 15:32:29.891182 2014-08-13 18:06:06.868534 7f5c973a57a0 15 read_log missing 194422'14341 (194422'14340) modify 4a0bd6a1/benchmark_data_storage0111-ib.pocstor_41231_object3568/head//24 by client.1148203.0:3569 2014-08-13 15:32:29.356032 2014-08-13 18:06:06.868569 7f5c973a57a0 15 read_log missing 194422'14339 (194422'14338) modify baafc2a1/benchmark_data_storage0111-ib.pocstor_41231_object1277/head//24 by client.1148203.0:1278 2014-08-13 15:32:20.393588 2014-08-13 18:06:06.868604 7f5c973a57a0 15 read_log missing 194422'14337 (194422'14336) modify ad4362a1/benchmark_data_storage0111-ib.pocstor_41231_object1131/head//24 by client.1148203.0:1132 2014-08-13 15:32:19.788514 2014-08-13 18:06:06.868640 7f5c973a57a0 15 read_log missing 194416'14335 (194416'14334) modify 1933b6a1/benchmark_data_storage0111-ib.pocstor_19103_object18472/head//24 by client.2441958.0:18473 2014-08-13 15:30:12.358498 2014-08-13 18:06:06.868675 7f5c973a57a0 15 read_log missing 194416'14333 (194416'14332) modify 5af55aa1/benchmark_data_storage0111-ib.pocstor_19103_object15184/head//24 by client.2441958.0:15185 2014-08-13 15:29:31.148722 2014-08-13 18:06:06.868710 7f5c973a57a0 15 read_log missing 194416'14331 (194416'14330) modify e06dcea1/benchmark_data_storage0111-ib.pocstor_19103_object14522/head//24 by client.2441958.0:14523 2014-08-13 15:29:28.297506 2014-08-13 18:06:06.868745 7f5c973a57a0 15 read_log missing 194416'14329 (194416'14328) modify ed12dea1/benchmark_data_storage0111-ib.pocstor_19103_object13764/head//24 by client.2441958.0:13765 2014-08-13 15:29:25.265313 2014-08-13 18:06:06.868781 7f5c973a57a0 15 read_log missing 194416'14327 (194416'14326) modify d196daa1/benchmark_data_storage0111-ib.pocstor_19103_object13527/head//24 by client.2441958.0:13528 2014-08-13 15:29:24.226232 2014-08-13 18:06:06.868816 7f5c973a57a0 15 read_log missing 194416'14325 (194416'14324) modify 8811c6a1/benchmark_data_storage0111-ib.pocstor_19103_object13279/head//24 by client.2441958.0:13280 2014-08-13 15:29:23.179367 2014-08-13 18:06:06.868852 7f5c973a57a0 15 read_log missing 194416'14323 (194416'14322) modify b2441aa1/benchmark_data_storage0111-ib.pocstor_19103_object11645/head//24 by client.2441958.0:11646 2014-08-13 15:29:16.086396 2014-08-13 18:06:06.868887 7f5c973a57a0 15 read_log missing 194416'14321 (194416'14320) modify 696716a1/benchmark_data_storage0111-ib.pocstor_19103_object10439/head//24 by client.2441958.0:10440 2014-08-13 15:29:11.205144 2014-08-13 18:06:06.868923 7f5c973a57a0 15 read_log missing 194416'14319 (194416'14318) modify 3ad262a1/benchmark_data_storage0111-ib.pocstor_19103_object9319/head//24 by client.2441958.0:9320 2014-08-13 15:29:06.690738 2014-08-13 18:06:06.868958 7f5c973a57a0 15 read_log missing 194416'14317 (194416'14316) modify afeb3aa1/benchmark_data_storage0111-ib.pocstor_19103_object2879/head//24 by client.2441958.0:2880 2014-08-13 15:28:41.620853 2014-08-13 18:06:06.868994 7f5c973a57a0 15 read_log missing 194416'14315 (194416'14314) modify b06c5ea1/benchmark_data_storage0111-ib.pocstor_19103_object2516/head//24 by client.2441958.0:2517 2014-08-13 15:28:40.238988 2014-08-13 18:06:06.869029 7f5c973a57a0 15 read_log missing 194416'14313 (194416'14312) modify 4d5baaa1/benchmark_data_storage0111-ib.pocstor_19103_object2235/head//24 by client.2441958.0:2236 2014-08-13 15:28:39.160468 2014-08-13 18:06:06.869064 7f5c973a57a0 15 read_log missing 194416'14311 (194416'14310) modify 1ca73ea1/benchmark_data_storage0111-ib.pocstor_19103_object1109/head//24 by client.2441958.0:1110 2014-08-13 15:28:34.924839 2014-08-13 18:06:06.869101 7f5c973a57a0 15 read_log missing 194416'14309 (194416'14308) modify bef332a1/benchmark_data_storage0111-ib.pocstor_19103_object561/head//24 by client.2441958.0:562 2014-08-13 15:28:32.721495 2014-08-13 18:06:06.869137 7f5c973a57a0 15 read_log missing 194416'14307 (194416'14306) modify 798842a1/benchmark_data_storage0111-ib.pocstor_19103_object428/head//24 by client.2441958.0:429 2014-08-13 15:28:32.058547 2014-08-13 18:06:06.869178 7f5c973a57a0 15 read_log missing 194416'14305 (194416'14304) modify 72f0a6a1/benchmark_data_storage0111-ib.pocstor_19103_object362/head//24 by client.2441958.0:363 2014-08-13 15:28:31.734296 2014-08-13 18:06:06.869215 7f5c973a57a0 15 read_log missing 194397'14303 (194397'14302) modify 41ce3aa1/benchmark_data_storage0101-ib.pocstor_6819_object815/head//24 by client.2235069.0:816 2014-08-12 20:53:03.772632 2014-08-13 18:06:06.869251 7f5c973a57a0 15 read_log missing 194393'14301 (194393'14300) modify 5bd3f2a1/benchmark_data_storage0111-ib.pocstor_48304_object238/head//24 by client.1064422.0:239 2014-08-12 20:52:33.493329 2014-08-13 18:06:06.869287 7f5c973a57a0 15 read_log missing 194393'14299 (194393'14298) modify 639ea1/benchmark_data_storage0111-ib.pocstor_32482_object256/head//24 by client.1064182.0:257 2014-08-12 20:49:26.673351 2014-08-13 18:06:06.869327 7f5c973a57a0 15 read_log missing 194393'14297 (194393'14296) modify 820c82a1/benchmark_data_storage0111-ib.pocstor_24170_object462/head//24 by client.1064095.0:463 2014-08-12 20:47:27.892264 2014-08-13 18:06:06.869367 7f5c973a57a0 15 read_log missing 194383'14295 (194383'14294) modify 4a4402a1/benchmark_data_storage0111-ib.pocstor_25065_object6135/head//24 by client.2231190.0:6136 2014-08-12 20:34:02.103724 2014-08-13 18:06:06.869418 7f5c973a57a0 15 read_log missing 194377'14293 (194377'14292) modify 3a9e6ea1/benchmark_data_storage0111-ib.pocstor_25065_object2908/head//24 by client.2231190.0:2909 2014-08-12 20:33:08.909638 2014-08-13 18:06:06.869467 7f5c973a57a0 15 read_log missing 194377'14291 (194377'14290) modify 35bebaa1/benchmark_data_storage0111-ib.pocstor_25065_object1766/head//24 by client.2231190.0:1767 2014-08-12 20:33:04.287040 2014-08-13 18:06:06.869514 7f5c973a57a0 15 read_log missing 194377'14289 (194377'14288) modify f21d8ea1/benchmark_data_storage0111-ib.pocstor_25065_object561/head//24 by client.2231190.0:562 2014-08-12 20:32:59.614726 2014-08-13 18:06:06.869552 7f5c973a57a0 15 read_log missing 194370'14287 (194370'14286) modify f00172a1/benchmark_data_storage0111-ib.pocstor_12495_object11342/head//24 by client.1508531.0:11343 2014-08-12 20:31:07.357266 2014-08-13 18:06:06.869588 7f5c973a57a0 15 read_log missing 194370'14285 (194370'14284) modify f7d65ea1/benchmark_data_storage0111-ib.pocstor_12495_object11097/head//24 by client.1508531.0:11098 2014-08-12 20:31:05.865726 2014-08-13 18:06:06.869624 7f5c973a57a0 15 read_log missing 194370'14283 (194370'14282) modify d5c45aa1/benchmark_data_storage0111-ib.pocstor_12495_object9548/head//24 by client.1508531.0:9549 2014-08-12 20:30:56.603705 2014-08-13 18:06:06.869659 7f5c973a57a0 15 read_log missing 194370'14281 (194370'14280) modify 773abaa1/benchmark_data_storage0111-ib.pocstor_12495_object9469/head//24 by client.1508531.0:9470 2014-08-12 20:30:56.134566 2014-08-13 18:06:06.869694 7f5c973a57a0 15 read_log missing 194370'14279 (194370'14278) modify f8a84aa1/benchmark_data_storage0111-ib.pocstor_12495_object7802/head//24 by client.1508531.0:7803 2014-08-12 20:30:46.527237 2014-08-13 18:06:06.869730 7f5c973a57a0 15 read_log missing 194370'14277 (194370'14276) modify cc7972a1/benchmark_data_storage0111-ib.pocstor_12495_object7412/head//24 by client.1508531.0:7413 2014-08-12 20:30:44.421622 2014-08-13 18:06:06.869765 7f5c973a57a0 15 read_log missing 194370'14275 (194370'14274) modify c8edc2a1/benchmark_data_storage0111-ib.pocstor_12495_object4203/head//24 by client.1508531.0:4204 2014-08-12 20:30:30.106220 2014-08-13 18:06:06.869801 7f5c973a57a0 15 read_log missing 194370'14273 (194370'14272) modify 608472a1/benchmark_data_storage0111-ib.pocstor_12495_object4143/head//24 by client.1508531.0:4144 2014-08-12 20:30:29.826139 2014-08-13 18:06:06.869842 7f5c973a57a0 15 read_log missing 194370'14271 (194370'14270) modify 7421eea1/benchmark_data_storage0111-ib.pocstor_12495_object4002/head//24 by client.1508531.0:4003 2014-08-12 20:30:29.299649 2014-08-13 18:06:06.869879 7f5c973a57a0 15 read_log missing 194370'14269 (194370'14268) modify 5a5696a1/benchmark_data_storage0111-ib.pocstor_12495_object3445/head//24 by client.1508531.0:3446 2014-08-12 20:30:27.183554 2014-08-13 18:06:06.869916 7f5c973a57a0 15 read_log missing 194370'14267 (194370'14266) modify 1bd3e6a1/benchmark_data_storage0111-ib.pocstor_12495_object3406/head//24 by client.1508531.0:3407 2014-08-12 20:30:27.030616 2014-08-13 18:06:06.869952 7f5c973a57a0 15 read_log missing 194370'14265 (194370'14264) modify f303a2a1/benchmark_data_storage0111-ib.pocstor_12495_object2502/head//24 by client.1508531.0:2503 2014-08-12 20:30:23.736973 2014-08-13 18:06:06.869988 7f5c973a57a0 15 read_log missing 194370'14263 (194370'14262) modify 87cba2a1/benchmark_data_storage0111-ib.pocstor_12495_object2470/head//24 by client.1508531.0:2471 2014-08-12 20:30:23.619503 2014-08-13 18:06:06.870023 7f5c973a57a0 15 read_log missing 194370'14261 (194370'14260) modify c9d922a1/benchmark_data_storage0111-ib.pocstor_12495_object2393/head//24 by client.1508531.0:2394 2014-08-12 20:30:23.338742 2014-08-13 18:06:06.870059 7f5c973a57a0 15 read_log missing 194370'14259 (194370'14258) modify e9a5aea1/benchmark_data_storage0111-ib.pocstor_12495_object1294/head//24 by client.1508531.0:1295 2014-08-12 20:30:19.241079 2014-08-13 18:06:06.870094 7f5c973a57a0 15 read_log missing 194370'14257 (194370'14256) modify 669b72a1/benchmark_data_storage0111-ib.pocstor_12495_object89/head//24 by client.1508531.0:90 2014-08-12 20:30:14.423052 2014-08-13 18:06:06.870105 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.870131 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=0 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive m=49] handle_loaded 2014-08-13 18:06:06.870174 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=0 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.015887 0 0.000000 2014-08-13 18:06:06.870189 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=0 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Reset 2014-08-13 18:06:06.870198 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=0 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194487 2014-08-13 18:06:06.870208 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log((192718'11272,194422'14353], crt=194422'14353) 2014-08-13 18:06:06.870218 7f5c973a57a0 10 osd.4 194487 pgid 24.2ec coll 24.2ec_head 2014-08-13 18:06:06.870374 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.2ec 2014-08-13 18:06:06.870387 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.870419 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2ec(unlocked)] enter Initial 2014-08-13 18:06:06.870435 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.2ec(unlocked)] enter NotTrimming 2014-08-13 18:06:06.870610 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.870733 7f5c973a57a0 20 read_log 192834'14496 (192834'14495) modify 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.933448.0:649 2014-08-11 15:23:38.123120 2014-08-13 18:06:06.870754 7f5c973a57a0 20 read_log 192834'14501 (0'0) promote fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by osd.361.0:31390 2014-08-11 15:23:41.382364 2014-08-13 18:06:06.870767 7f5c973a57a0 20 read_log 192834'14502 (192834'14501) modify fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.933448.0:1475 2014-08-11 15:23:41.373562 2014-08-13 18:06:06.870780 7f5c973a57a0 20 read_log 192834'14509 (0'0) promote 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by osd.361.0:31412 2014-08-11 15:23:43.535180 2014-08-13 18:06:06.870793 7f5c973a57a0 20 read_log 192834'14510 (192834'14509) modify 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.933448.0:1884 2014-08-11 15:23:43.527854 2014-08-13 18:06:06.870805 7f5c973a57a0 20 read_log 192834'14517 (0'0) promote b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.361.0:31426 2014-08-11 15:23:46.032690 2014-08-13 18:06:06.870829 7f5c973a57a0 20 read_log 192834'14518 (192834'14517) modify b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by client.933448.0:2280 2014-08-11 15:23:46.022163 2014-08-13 18:06:06.870845 7f5c973a57a0 20 read_log 192834'14519 (0'0) promote 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.361.0:31428 2014-08-11 15:23:46.269969 2014-08-13 18:06:06.870858 7f5c973a57a0 20 read_log 192834'14520 (192834'14519) modify 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by client.933448.0:2331 2014-08-11 15:23:46.262640 2014-08-13 18:06:06.870870 7f5c973a57a0 20 read_log 192834'14529 (0'0) promote 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by osd.361.0:31448 2014-08-11 15:23:48.986050 2014-08-13 18:06:06.870883 7f5c973a57a0 20 read_log 192834'14530 (192834'14529) modify 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.933448.0:2871 2014-08-11 15:23:48.976320 2014-08-13 18:06:06.870895 7f5c973a57a0 20 read_log 192834'14539 (0'0) promote 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.361.0:31462 2014-08-11 15:23:51.508348 2014-08-13 18:06:06.870907 7f5c973a57a0 20 read_log 192834'14540 (192834'14539) modify 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by client.933448.0:3380 2014-08-11 15:23:51.500196 2014-08-13 18:06:06.870920 7f5c973a57a0 20 read_log 192834'14569 (0'0) promote b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.361.0:31526 2014-08-11 15:24:02.361534 2014-08-13 18:06:06.870932 7f5c973a57a0 20 read_log 192834'14570 (192834'14569) modify b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by client.933448.0:5649 2014-08-11 15:24:02.354625 2014-08-13 18:06:06.870944 7f5c973a57a0 20 read_log 192834'14577 (0'0) promote ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by osd.361.0:31544 2014-08-11 15:24:03.765210 2014-08-13 18:06:06.870959 7f5c973a57a0 20 read_log 192834'14578 (192834'14577) modify ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.933448.0:6010 2014-08-11 15:24:03.757923 2014-08-13 18:06:06.870973 7f5c973a57a0 20 read_log 192834'14589 (0'0) promote dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by osd.361.0:31572 2014-08-11 15:24:06.342494 2014-08-13 18:06:06.870985 7f5c973a57a0 20 read_log 192834'14590 (192834'14589) modify dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.933448.0:6679 2014-08-11 15:24:06.335993 2014-08-13 18:06:06.870997 7f5c973a57a0 20 read_log 192834'14597 (0'0) promote 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.361.0:31586 2014-08-11 15:24:09.028306 2014-08-13 18:06:06.871009 7f5c973a57a0 20 read_log 192834'14598 (192834'14597) modify 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by client.933448.0:7173 2014-08-11 15:24:09.021258 2014-08-13 18:06:06.871021 7f5c973a57a0 20 read_log 192834'14617 (0'0) promote 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.361.0:31624 2014-08-11 15:24:12.599377 2014-08-13 18:06:06.871033 7f5c973a57a0 20 read_log 192834'14618 (192834'14617) modify 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by client.933448.0:8068 2014-08-11 15:24:12.589061 2014-08-13 18:06:06.871046 7f5c973a57a0 20 read_log 192834'14629 (0'0) promote b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by osd.361.0:31640 2014-08-11 15:24:14.585593 2014-08-13 18:06:06.871058 7f5c973a57a0 20 read_log 192834'14630 (192834'14629) modify b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.933448.0:8567 2014-08-11 15:24:14.578329 2014-08-13 18:06:06.871070 7f5c973a57a0 20 read_log 192834'14635 (0'0) promote 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.361.0:31652 2014-08-11 15:24:17.345073 2014-08-13 18:06:06.871089 7f5c973a57a0 20 read_log 192834'14636 (192834'14635) modify 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by client.933448.0:9046 2014-08-11 15:24:17.336967 2014-08-13 18:06:06.871102 7f5c973a57a0 20 read_log 192834'14683 (0'0) promote 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.361.0:31760 2014-08-11 15:24:31.141414 2014-08-13 18:06:06.871115 7f5c973a57a0 20 read_log 192834'14684 (192834'14683) modify 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by client.933448.0:12114 2014-08-11 15:24:31.134774 2014-08-13 18:06:06.871127 7f5c973a57a0 20 read_log 192834'14685 (0'0) promote 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by osd.361.0:31764 2014-08-11 15:24:31.321595 2014-08-13 18:06:06.871139 7f5c973a57a0 20 read_log 192834'14686 (192834'14685) modify 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.933448.0:12162 2014-08-11 15:24:31.314481 2014-08-13 18:06:06.871158 7f5c973a57a0 20 read_log 192834'14687 (0'0) promote 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.361.0:31768 2014-08-11 15:24:32.013926 2014-08-13 18:06:06.871174 7f5c973a57a0 20 read_log 192834'14688 (192834'14687) modify 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by client.933448.0:12347 2014-08-11 15:24:32.006741 2014-08-13 18:06:06.871187 7f5c973a57a0 20 read_log 192834'14709 (0'0) promote bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.361.0:31822 2014-08-11 15:24:40.428318 2014-08-13 18:06:06.871201 7f5c973a57a0 20 read_log 192834'14710 (192834'14709) modify bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by client.933448.0:13916 2014-08-11 15:24:40.420218 2014-08-13 18:06:06.871213 7f5c973a57a0 20 read_log 192834'14753 (0'0) promote 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by osd.361.0:31927 2014-08-11 15:24:53.389670 2014-08-13 18:06:06.871226 7f5c973a57a0 20 read_log 192834'14754 (192834'14753) modify 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.933448.0:16171 2014-08-11 15:24:53.381794 2014-08-13 18:06:06.871238 7f5c973a57a0 20 read_log 192834'14757 (0'0) promote 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by osd.361.0:31953 2014-08-11 15:24:55.957766 2014-08-13 18:06:06.871250 7f5c973a57a0 20 read_log 192834'14758 (192834'14757) modify 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.933448.0:16491 2014-08-11 15:24:55.950399 2014-08-13 18:06:06.871262 7f5c973a57a0 20 read_log 192834'14779 (0'0) promote 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.361.0:32001 2014-08-11 15:25:02.081447 2014-08-13 18:06:06.871279 7f5c973a57a0 20 read_log 192834'14780 (192834'14779) modify 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by client.933448.0:17860 2014-08-11 15:25:02.034769 2014-08-13 18:06:06.871291 7f5c973a57a0 20 read_log 192834'14793 (0'0) promote 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by osd.361.0:32040 2014-08-11 15:25:07.204393 2014-08-13 18:06:06.871303 7f5c973a57a0 20 read_log 192834'14794 (192834'14793) modify 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.933448.0:18854 2014-08-11 15:25:07.196859 2014-08-13 18:06:06.871319 7f5c973a57a0 20 read_log 192834'14797 (0'0) promote 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by osd.361.0:32074 2014-08-11 15:25:08.477011 2014-08-13 18:06:06.871357 7f5c973a57a0 20 read_log 192834'14798 (192834'14797) modify 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.933448.0:19065 2014-08-11 15:25:08.371525 2014-08-13 18:06:06.871373 7f5c973a57a0 20 read_log 192834'14801 (0'0) promote 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.361.0:32086 2014-08-11 15:25:11.524047 2014-08-13 18:06:06.871404 7f5c973a57a0 20 read_log 192834'14802 (192834'14801) modify 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by client.933448.0:19337 2014-08-11 15:25:11.516347 2014-08-13 18:06:06.871421 7f5c973a57a0 20 read_log 192834'14829 (0'0) promote 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by osd.361.0:32167 2014-08-11 15:25:20.070087 2014-08-13 18:06:06.871434 7f5c973a57a0 20 read_log 192834'14830 (192834'14829) modify 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.933448.0:20278 2014-08-11 15:25:20.063264 2014-08-13 18:06:06.871459 7f5c973a57a0 20 read_log 192834'14857 (0'0) promote 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by osd.361.0:32240 2014-08-11 15:25:30.822549 2014-08-13 18:06:06.871472 7f5c973a57a0 20 read_log 192834'14858 (192834'14857) modify 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.933448.0:21519 2014-08-11 15:25:30.814592 2014-08-13 18:06:06.871485 7f5c973a57a0 20 read_log 192834'14863 (0'0) promote 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by osd.361.0:32279 2014-08-11 15:25:32.850718 2014-08-13 18:06:06.871504 7f5c973a57a0 20 read_log 192834'14864 (192834'14863) modify 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.933448.0:21930 2014-08-11 15:25:32.839880 2014-08-13 18:06:06.871517 7f5c973a57a0 20 read_log 192834'14895 (192834'14496) modify 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.933448.0:22940 2014-08-11 15:25:38.830050 2014-08-13 18:06:06.871529 7f5c973a57a0 20 read_log 192834'14898 (192834'14502) modify fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.933448.0:23766 2014-08-11 15:25:39.810271 2014-08-13 18:06:06.871563 7f5c973a57a0 20 read_log 192834'14902 (192834'14510) modify 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.933448.0:24175 2014-08-11 15:25:40.363049 2014-08-13 18:06:06.871576 7f5c973a57a0 20 read_log 192834'14907 (192834'14518) modify b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by client.933448.0:24571 2014-08-11 15:25:41.035827 2014-08-13 18:06:06.871589 7f5c973a57a0 20 read_log 192834'14908 (192834'14520) modify 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by client.933448.0:24622 2014-08-11 15:25:41.111385 2014-08-13 18:06:06.871601 7f5c973a57a0 20 read_log 192834'14913 (192834'14530) modify 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.933448.0:25162 2014-08-11 15:25:41.837740 2014-08-13 18:06:06.871613 7f5c973a57a0 20 read_log 192834'14919 (192834'14540) modify 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by client.933448.0:25671 2014-08-11 15:25:42.330900 2014-08-13 18:06:06.871625 7f5c973a57a0 20 read_log 192834'14934 (192834'14570) modify b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by client.933448.0:27940 2014-08-11 15:25:45.288257 2014-08-13 18:06:06.871638 7f5c973a57a0 20 read_log 192834'14939 (192834'14578) modify ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.933448.0:28301 2014-08-11 15:25:45.891774 2014-08-13 18:06:06.871650 7f5c973a57a0 20 read_log 192834'14946 (192834'14590) modify dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.933448.0:28970 2014-08-11 15:25:46.895716 2014-08-13 18:06:06.871662 7f5c973a57a0 20 read_log 192834'14950 (192834'14598) modify 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by client.933448.0:29464 2014-08-11 15:25:47.369107 2014-08-13 18:06:06.871674 7f5c973a57a0 20 read_log 192834'14960 (192834'14618) modify 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by client.933448.0:30359 2014-08-11 15:25:48.776178 2014-08-13 18:06:06.871687 7f5c973a57a0 20 read_log 192834'14968 (192834'14630) modify b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.933448.0:30858 2014-08-11 15:25:49.318543 2014-08-13 18:06:06.871710 7f5c973a57a0 20 read_log 192834'14971 (192834'14636) modify 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by client.933448.0:31337 2014-08-11 15:25:49.941353 2014-08-13 18:06:06.871726 7f5c973a57a0 20 read_log 192834'14998 (192834'14684) modify 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by client.933448.0:34405 2014-08-11 15:25:54.972525 2014-08-13 18:06:06.871738 7f5c973a57a0 20 read_log 192834'14999 (192834'14686) modify 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.933448.0:34453 2014-08-11 15:25:55.024071 2014-08-13 18:06:06.871751 7f5c973a57a0 20 read_log 192834'15000 (192834'14688) modify 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by client.933448.0:34638 2014-08-11 15:25:55.308966 2014-08-13 18:06:06.871763 7f5c973a57a0 20 read_log 192834'15011 (192834'14710) modify bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by client.933448.0:36207 2014-08-11 15:25:57.346835 2014-08-13 18:06:06.871776 7f5c973a57a0 20 read_log 192834'15033 (192834'14754) modify 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.933448.0:38462 2014-08-11 15:26:00.893555 2014-08-13 18:06:06.871788 7f5c973a57a0 20 read_log 192834'15035 (192834'14758) modify 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.933448.0:38782 2014-08-11 15:26:01.263012 2014-08-13 18:06:06.871800 7f5c973a57a0 20 read_log 192834'15046 (192834'14780) modify 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by client.933448.0:40151 2014-08-11 15:26:02.351595 2014-08-13 18:06:06.871813 7f5c973a57a0 20 read_log 192834'15053 (192834'14794) modify 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.933448.0:41145 2014-08-11 15:26:02.851758 2014-08-13 18:06:06.871825 7f5c973a57a0 20 read_log 192834'15055 (192834'14798) modify 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.933448.0:41356 2014-08-11 15:26:02.919060 2014-08-13 18:06:06.871837 7f5c973a57a0 20 read_log 192834'15057 (192834'14802) modify 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by client.933448.0:41628 2014-08-11 15:26:03.004141 2014-08-13 18:06:06.871849 7f5c973a57a0 20 read_log 192834'15071 (192834'14830) modify 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.933448.0:42569 2014-08-11 15:26:03.318293 2014-08-13 18:06:06.871862 7f5c973a57a0 20 read_log 192834'15085 (192834'14858) modify 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.933448.0:43810 2014-08-11 15:26:03.730996 2014-08-13 18:06:06.871874 7f5c973a57a0 20 read_log 192834'15088 (192834'14864) modify 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.933448.0:44221 2014-08-11 15:26:03.853987 2014-08-13 18:06:06.871886 7f5c973a57a0 20 read_log 192834'15104 (0'0) promote d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.361.0:32911 2014-08-11 15:26:16.993682 2014-08-13 18:06:06.871898 7f5c973a57a0 20 read_log 192834'15105 (192834'15104) modify d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by client.1883634.0:687 2014-08-11 15:26:16.986686 2014-08-13 18:06:06.871911 7f5c973a57a0 20 read_log 192834'15122 (0'0) promote 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.361.0:32949 2014-08-11 15:26:22.144254 2014-08-13 18:06:06.871923 7f5c973a57a0 20 read_log 192834'15123 (192834'15122) modify 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by client.1883634.0:2077 2014-08-11 15:26:22.136952 2014-08-13 18:06:06.871935 7f5c973a57a0 20 read_log 192834'15134 (0'0) promote bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by osd.361.0:32965 2014-08-11 15:26:23.596406 2014-08-13 18:06:06.871947 7f5c973a57a0 20 read_log 192834'15135 (192834'15134) modify bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.1883634.0:2473 2014-08-11 15:26:23.588806 2014-08-13 18:06:06.871968 7f5c973a57a0 20 read_log 192834'15138 (0'0) promote 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by osd.361.0:32969 2014-08-11 15:26:23.611803 2014-08-13 18:06:06.871984 7f5c973a57a0 20 read_log 192834'15139 (192834'15138) modify 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.1883634.0:2477 2014-08-11 15:26:23.601830 2014-08-13 18:06:06.872003 7f5c973a57a0 20 read_log 192834'15144 (0'0) promote 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by osd.361.0:32975 2014-08-11 15:26:25.701871 2014-08-13 18:06:06.872016 7f5c973a57a0 20 read_log 192834'15145 (192834'15144) modify 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.1883634.0:3068 2014-08-11 15:26:25.694604 2014-08-13 18:06:06.872031 7f5c973a57a0 20 read_log 192834'15148 (0'0) promote c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by osd.361.0:32979 2014-08-11 15:26:26.183683 2014-08-13 18:06:06.872043 7f5c973a57a0 20 read_log 192834'15149 (192834'15148) modify c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.1883634.0:3204 2014-08-11 15:26:26.176264 2014-08-13 18:06:06.872058 7f5c973a57a0 20 read_log 192834'15154 (0'0) promote 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.361.0:32987 2014-08-11 15:26:27.074559 2014-08-13 18:06:06.872070 7f5c973a57a0 20 read_log 192834'15155 (192834'15154) modify 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by client.1883634.0:3456 2014-08-11 15:26:27.067324 2014-08-13 18:06:06.872082 7f5c973a57a0 20 read_log 192834'15162 (0'0) promote 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by osd.361.0:33003 2014-08-11 15:26:30.212288 2014-08-13 18:06:06.872094 7f5c973a57a0 20 read_log 192834'15163 (192834'15162) modify 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.1883634.0:4344 2014-08-11 15:26:30.204700 2014-08-13 18:06:06.872107 7f5c973a57a0 20 read_log 192834'15164 (0'0) promote 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by osd.361.0:33011 2014-08-11 15:26:30.924146 2014-08-13 18:06:06.872121 7f5c973a57a0 20 read_log 192834'15165 (192834'15164) modify 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.1883634.0:4526 2014-08-11 15:26:30.916858 2014-08-13 18:06:06.872133 7f5c973a57a0 20 read_log 192834'15174 (0'0) promote 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.361.0:33027 2014-08-11 15:26:33.383695 2014-08-13 18:06:06.872173 7f5c973a57a0 20 read_log 192834'15175 (192834'15174) modify 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by client.1883634.0:5148 2014-08-11 15:26:33.376010 2014-08-13 18:06:06.872191 7f5c973a57a0 20 read_log 192834'15216 (0'0) promote d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by osd.361.0:33198 2014-08-11 15:26:47.322218 2014-08-13 18:06:06.872204 7f5c973a57a0 20 read_log 192834'15217 (192834'15216) modify d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.1883634.0:8669 2014-08-11 15:26:47.313356 2014-08-13 18:06:06.872218 7f5c973a57a0 20 read_log 192834'15220 (0'0) promote 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by osd.361.0:33204 2014-08-11 15:26:48.147012 2014-08-13 18:06:06.872231 7f5c973a57a0 20 read_log 192834'15221 (192834'15220) modify 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.1883634.0:8844 2014-08-11 15:26:48.138119 2014-08-13 18:06:06.872248 7f5c973a57a0 20 read_log 192834'15238 (0'0) promote a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by osd.361.0:33317 2014-08-11 15:26:53.636578 2014-08-13 18:06:06.872263 7f5c973a57a0 20 read_log 192834'15239 (192834'15238) modify a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.1883634.0:10085 2014-08-11 15:26:53.628859 2014-08-13 18:06:06.872283 7f5c973a57a0 20 read_log 192834'15246 (0'0) promote a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by osd.361.0:33402 2014-08-11 15:26:57.356376 2014-08-13 18:06:06.872298 7f5c973a57a0 20 read_log 192834'15247 (192834'15246) modify a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.1883634.0:10874 2014-08-11 15:26:57.348666 2014-08-13 18:06:06.872311 7f5c973a57a0 20 read_log 192834'15264 (0'0) promote 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.361.0:33508 2014-08-11 15:27:03.999643 2014-08-13 18:06:06.872323 7f5c973a57a0 20 read_log 192834'15265 (192834'15264) modify 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by client.1883634.0:12434 2014-08-11 15:27:03.991917 2014-08-13 18:06:06.872338 7f5c973a57a0 20 read_log 192834'15274 (0'0) promote 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by osd.361.0:33593 2014-08-11 15:27:07.236032 2014-08-13 18:06:06.872350 7f5c973a57a0 20 read_log 192834'15275 (192834'15274) modify 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.1883634.0:13018 2014-08-11 15:27:07.227646 2014-08-13 18:06:06.872364 7f5c973a57a0 20 read_log 192834'15294 (0'0) promote 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.361.0:33633 2014-08-11 15:27:14.048790 2014-08-13 18:06:06.872390 7f5c973a57a0 20 read_log 192834'15295 (192834'15294) modify 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by client.1883634.0:14619 2014-08-11 15:27:14.037923 2014-08-13 18:06:06.872403 7f5c973a57a0 20 read_log 192834'15308 (0'0) promote 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by osd.361.0:33653 2014-08-11 15:27:18.987179 2014-08-13 18:06:06.872415 7f5c973a57a0 20 read_log 192834'15309 (192834'15308) modify 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.1883634.0:15297 2014-08-11 15:27:18.977724 2014-08-13 18:06:06.872442 7f5c973a57a0 20 read_log 192834'15318 (0'0) promote a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by osd.361.0:33690 2014-08-11 15:27:24.914490 2014-08-13 18:06:06.872455 7f5c973a57a0 20 read_log 192834'15319 (192834'15318) modify a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.1883634.0:15966 2014-08-11 15:27:24.905293 2014-08-13 18:06:06.872468 7f5c973a57a0 20 read_log 192834'15336 (0'0) promote 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by osd.361.0:33745 2014-08-11 15:27:33.917876 2014-08-13 18:06:06.872490 7f5c973a57a0 20 read_log 192834'15337 (192834'15336) modify 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.1883634.0:17255 2014-08-11 15:27:33.910079 2014-08-13 18:06:06.872503 7f5c973a57a0 20 read_log 192834'15344 (0'0) promote 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by osd.361.0:33814 2014-08-11 15:27:47.295439 2014-08-13 18:06:06.872515 7f5c973a57a0 20 read_log 192834'15345 (192834'15344) modify 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.1883634.0:18736 2014-08-11 15:27:47.287721 2014-08-13 18:06:06.872527 7f5c973a57a0 20 read_log 192834'15348 (0'0) promote 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by osd.361.0:33820 2014-08-11 15:27:48.995204 2014-08-13 18:06:06.872540 7f5c973a57a0 20 read_log 192834'15349 (192834'15348) modify 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.1883634.0:19032 2014-08-11 15:27:48.986754 2014-08-13 18:06:06.872552 7f5c973a57a0 20 read_log 192834'15354 (0'0) promote ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.361.0:33853 2014-08-11 15:27:51.139924 2014-08-13 18:06:06.872564 7f5c973a57a0 20 read_log 192834'15355 (192834'15354) modify ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by client.1883634.0:19307 2014-08-11 15:27:51.131704 2014-08-13 18:06:06.872586 7f5c973a57a0 20 read_log 192834'15378 (0'0) promote 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by osd.361.0:33935 2014-08-11 15:28:02.399447 2014-08-13 18:06:06.872601 7f5c973a57a0 20 read_log 192834'15379 (192834'15378) modify 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.1883634.0:21023 2014-08-11 15:28:02.390910 2014-08-13 18:06:06.872614 7f5c973a57a0 20 read_log 192834'15382 (0'0) promote 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by osd.361.0:33941 2014-08-11 15:28:02.759781 2014-08-13 18:06:06.872626 7f5c973a57a0 20 read_log 192834'15383 (192834'15382) modify 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.1883634.0:21092 2014-08-11 15:28:02.751789 2014-08-13 18:06:06.872639 7f5c973a57a0 20 read_log 192834'15388 (0'0) promote d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.361.0:33947 2014-08-11 15:28:03.258525 2014-08-13 18:06:06.872651 7f5c973a57a0 20 read_log 192834'15389 (192834'15388) modify d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by client.1883634.0:21187 2014-08-11 15:28:03.248729 2014-08-13 18:06:06.872663 7f5c973a57a0 20 read_log 192834'15410 (0'0) promote 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by osd.361.0:34013 2014-08-11 15:28:10.432366 2014-08-13 18:06:06.872675 7f5c973a57a0 20 read_log 192834'15411 (192834'15410) modify 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.1883634.0:22666 2014-08-11 15:28:10.422456 2014-08-13 18:06:06.872688 7f5c973a57a0 20 read_log 192834'15432 (192834'15105) modify d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by client.1883634.0:24011 2014-08-11 15:28:18.232309 2014-08-13 18:06:06.872700 7f5c973a57a0 20 read_log 192834'15441 (192834'15123) modify 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by client.1883634.0:25401 2014-08-11 15:28:21.153341 2014-08-13 18:06:06.872727 7f5c973a57a0 20 read_log 192834'15447 (192834'15135) modify bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.1883634.0:25797 2014-08-11 15:28:21.888982 2014-08-13 18:06:06.872761 7f5c973a57a0 20 read_log 192834'15449 (192834'15139) modify 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.1883634.0:25801 2014-08-11 15:28:21.898156 2014-08-13 18:06:06.872792 7f5c973a57a0 20 read_log 192834'15452 (192834'15145) modify 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.1883634.0:26392 2014-08-11 15:28:23.076613 2014-08-13 18:06:06.872805 7f5c973a57a0 20 read_log 192834'15454 (192834'15149) modify c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.1883634.0:26528 2014-08-11 15:28:23.221132 2014-08-13 18:06:06.872820 7f5c973a57a0 20 read_log 192834'15457 (192834'15155) modify 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by client.1883634.0:26780 2014-08-11 15:28:23.574459 2014-08-13 18:06:06.872835 7f5c973a57a0 20 read_log 192834'15461 (192834'15163) modify 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.1883634.0:27668 2014-08-11 15:28:24.775709 2014-08-13 18:06:06.872848 7f5c973a57a0 20 read_log 192834'15462 (192834'15165) modify 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.1883634.0:27850 2014-08-11 15:28:25.042698 2014-08-13 18:06:06.872860 7f5c973a57a0 20 read_log 192834'15467 (192834'15175) modify 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by client.1883634.0:28472 2014-08-11 15:28:25.856133 2014-08-13 18:06:06.872876 7f5c973a57a0 20 read_log 192834'15488 (192834'15217) modify d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.1883634.0:31993 2014-08-11 15:28:31.206074 2014-08-13 18:06:06.872891 7f5c973a57a0 20 read_log 192834'15490 (192834'15221) modify 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.1883634.0:32168 2014-08-11 15:28:31.449180 2014-08-13 18:06:06.872927 7f5c973a57a0 20 read_log 192834'15501 (192834'15239) modify a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.1883634.0:33409 2014-08-11 15:28:33.134892 2014-08-13 18:06:06.872943 7f5c973a57a0 20 read_log 192834'15505 (192834'15247) modify a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.1883634.0:34198 2014-08-11 15:28:34.356260 2014-08-13 18:06:06.872968 7f5c973a57a0 20 read_log 192834'15514 (192834'15265) modify 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by client.1883634.0:35758 2014-08-11 15:28:37.094485 2014-08-13 18:06:06.872981 7f5c973a57a0 20 read_log 192834'15519 (192834'15275) modify 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.1883634.0:36342 2014-08-11 15:28:37.910318 2014-08-13 18:06:06.873000 7f5c973a57a0 20 read_log 192834'15529 (192834'15295) modify 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by client.1883634.0:37943 2014-08-11 15:28:39.994679 2014-08-13 18:06:06.873016 7f5c973a57a0 20 read_log 192834'15536 (192834'15309) modify 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.1883634.0:38621 2014-08-11 15:28:40.941859 2014-08-13 18:06:06.873029 7f5c973a57a0 20 read_log 192834'15541 (192834'15319) modify a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.1883634.0:39290 2014-08-11 15:28:41.968621 2014-08-13 18:06:06.873041 7f5c973a57a0 20 read_log 192834'15550 (192834'15337) modify 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.1883634.0:40579 2014-08-11 15:28:44.002991 2014-08-13 18:06:06.873067 7f5c973a57a0 20 read_log 192834'15554 (192834'15345) modify 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.1883634.0:42060 2014-08-11 15:28:46.119209 2014-08-13 18:06:06.873080 7f5c973a57a0 20 read_log 192834'15556 (192834'15349) modify 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.1883634.0:42356 2014-08-11 15:28:46.217478 2014-08-13 18:06:06.873093 7f5c973a57a0 20 read_log 192834'15559 (192834'15355) modify ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by client.1883634.0:42631 2014-08-11 15:28:46.304569 2014-08-13 18:06:06.873105 7f5c973a57a0 20 read_log 192834'15571 (192834'15379) modify 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.1883634.0:44347 2014-08-11 15:28:46.834311 2014-08-13 18:06:06.873117 7f5c973a57a0 20 read_log 192834'15573 (192834'15383) modify 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.1883634.0:44416 2014-08-11 15:28:46.853378 2014-08-13 18:06:06.873129 7f5c973a57a0 20 read_log 192834'15576 (192834'15389) modify d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by client.1883634.0:44511 2014-08-11 15:28:46.878083 2014-08-13 18:06:06.873142 7f5c973a57a0 20 read_log 192834'15587 (192834'15411) modify 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.1883634.0:45990 2014-08-11 15:28:47.458413 2014-08-13 18:06:06.873162 7f5c973a57a0 20 read_log 192888'15598 (0'0) promote 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by osd.345.0:1369 2014-08-11 16:45:01.722244 2014-08-13 18:06:06.873175 7f5c973a57a0 20 read_log 192888'15599 (192888'15598) modify 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.1315259.0:1689 2014-08-11 16:45:01.715110 2014-08-13 18:06:06.873198 7f5c973a57a0 20 read_log 192888'15600 (0'0) promote 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by osd.345.0:1377 2014-08-11 16:45:05.102385 2014-08-13 18:06:06.873212 7f5c973a57a0 20 read_log 192888'15601 (192888'15600) modify 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.1315259.0:2602 2014-08-11 16:45:05.093700 2014-08-13 18:06:06.873237 7f5c973a57a0 20 read_log 192888'15602 (0'0) promote daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.345.0:1381 2014-08-11 16:45:05.784027 2014-08-13 18:06:06.873253 7f5c973a57a0 20 read_log 192888'15603 (192888'15602) modify daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by client.1315259.0:2783 2014-08-11 16:45:05.776749 2014-08-13 18:06:06.873265 7f5c973a57a0 20 read_log 192888'15604 (0'0) promote 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by osd.345.0:1383 2014-08-11 16:45:06.129703 2014-08-13 18:06:06.873278 7f5c973a57a0 20 read_log 192888'15605 (192888'15604) modify 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.1315259.0:2877 2014-08-11 16:45:06.121452 2014-08-13 18:06:06.873290 7f5c973a57a0 20 read_log 192888'15606 (0'0) promote e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by osd.345.0:1387 2014-08-11 16:45:07.651994 2014-08-13 18:06:06.873302 7f5c973a57a0 20 read_log 192888'15607 (192888'15606) modify e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.1315259.0:3252 2014-08-11 16:45:07.643104 2014-08-13 18:06:06.873323 7f5c973a57a0 20 read_log 192888'15608 (0'0) promote 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by osd.345.0:1405 2014-08-11 16:45:11.045442 2014-08-13 18:06:06.873337 7f5c973a57a0 20 read_log 192888'15609 (192888'15608) modify 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.1315259.0:4160 2014-08-11 16:45:11.037083 2014-08-13 18:06:06.873351 7f5c973a57a0 20 read_log 192888'15612 (0'0) promote 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by osd.345.0:1409 2014-08-11 16:45:11.859177 2014-08-13 18:06:06.873363 7f5c973a57a0 20 read_log 192888'15613 (192888'15612) modify 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.1315259.0:4384 2014-08-11 16:45:11.850082 2014-08-13 18:06:06.873381 7f5c973a57a0 20 read_log 192888'15614 (0'0) promote eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.345.0:1411 2014-08-11 16:45:12.787675 2014-08-13 18:06:06.873401 7f5c973a57a0 20 read_log 192888'15615 (192888'15614) modify eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by client.1315259.0:4639 2014-08-11 16:45:12.776300 2014-08-13 18:06:06.873416 7f5c973a57a0 20 read_log 192888'15622 (0'0) promote 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by osd.345.0:1433 2014-08-11 16:45:18.366298 2014-08-13 18:06:06.873428 7f5c973a57a0 20 read_log 192888'15623 (192888'15622) modify 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.1315259.0:6161 2014-08-11 16:45:18.359020 2014-08-13 18:06:06.873441 7f5c973a57a0 20 read_log 192888'15624 (0'0) promote 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.345.0:1435 2014-08-11 16:45:18.447636 2014-08-13 18:06:06.873453 7f5c973a57a0 20 read_log 192888'15625 (192888'15624) modify 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by client.1315259.0:6184 2014-08-11 16:45:18.439851 2014-08-13 18:06:06.873465 7f5c973a57a0 20 read_log 192888'15632 (0'0) promote 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by osd.345.0:1473 2014-08-11 16:45:25.610487 2014-08-13 18:06:06.873477 7f5c973a57a0 20 read_log 192888'15633 (192888'15632) modify 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.1315259.0:8158 2014-08-11 16:45:25.602967 2014-08-13 18:06:06.873490 7f5c973a57a0 20 read_log 192888'15634 (0'0) promote 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by osd.345.0:1491 2014-08-11 16:45:30.223522 2014-08-13 18:06:06.873502 7f5c973a57a0 20 read_log 192888'15635 (192888'15634) modify 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.1315259.0:9377 2014-08-11 16:45:30.213806 2014-08-13 18:06:06.873524 7f5c973a57a0 20 read_log 192888'15638 (0'0) promote cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by osd.345.0:1501 2014-08-11 16:45:32.799403 2014-08-13 18:06:06.873539 7f5c973a57a0 20 read_log 192888'15639 (192888'15638) modify cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.1315259.0:10038 2014-08-11 16:45:32.790035 2014-08-13 18:06:06.873552 7f5c973a57a0 20 read_log 192888'15640 (0'0) promote 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.345.0:1511 2014-08-11 16:45:39.345930 2014-08-13 18:06:06.873564 7f5c973a57a0 20 read_log 192888'15641 (192888'15640) modify 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by client.1315259.0:11678 2014-08-11 16:45:39.282834 2014-08-13 18:06:06.873576 7f5c973a57a0 20 read_log 192888'15642 (0'0) promote 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by osd.345.0:1515 2014-08-11 16:45:39.512309 2014-08-13 18:06:06.873588 7f5c973a57a0 20 read_log 192888'15643 (192888'15642) modify 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.1315259.0:11735 2014-08-11 16:45:39.503932 2014-08-13 18:06:06.873601 7f5c973a57a0 20 read_log 192888'15648 (0'0) promote 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by osd.345.0:1529 2014-08-11 16:45:42.293815 2014-08-13 18:06:06.873613 7f5c973a57a0 20 read_log 192888'15649 (192888'15648) modify 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.1315259.0:12379 2014-08-11 16:45:42.285804 2014-08-13 18:06:06.873626 7f5c973a57a0 20 read_log 192888'15650 (0'0) promote 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by osd.345.0:1535 2014-08-11 16:45:45.058479 2014-08-13 18:06:06.873638 7f5c973a57a0 20 read_log 192888'15651 (192888'15650) modify 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.1315259.0:13048 2014-08-11 16:45:45.049508 2014-08-13 18:06:06.873651 7f5c973a57a0 20 read_log 192888'15656 (0'0) promote b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.345.0:1549 2014-08-11 16:45:47.743477 2014-08-13 18:06:06.873663 7f5c973a57a0 20 read_log 192888'15657 (192888'15656) modify b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by client.1315259.0:13723 2014-08-11 16:45:47.736478 2014-08-13 18:06:06.873675 7f5c973a57a0 20 read_log 192888'15660 (0'0) promote d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.345.0:1565 2014-08-11 16:45:50.578659 2014-08-13 18:06:06.873687 7f5c973a57a0 20 read_log 192888'15661 (192888'15660) modify d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by client.1315259.0:14468 2014-08-11 16:45:50.570485 2014-08-13 18:06:06.873700 7f5c973a57a0 20 read_log 192888'15662 (0'0) promote 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.345.0:1569 2014-08-11 16:45:51.244444 2014-08-13 18:06:06.873712 7f5c973a57a0 20 read_log 192888'15663 (192888'15662) modify 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by client.1315259.0:14651 2014-08-11 16:45:51.236894 2014-08-13 18:06:06.873726 7f5c973a57a0 20 read_log 192888'15664 (0'0) promote 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by osd.345.0:1583 2014-08-11 16:45:55.745531 2014-08-13 18:06:06.873741 7f5c973a57a0 20 read_log 192888'15665 (192888'15664) modify 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.1315259.0:15819 2014-08-11 16:45:55.737419 2014-08-13 18:06:06.873753 7f5c973a57a0 20 read_log 192888'15666 (0'0) promote 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by osd.345.0:1585 2014-08-11 16:45:56.528214 2014-08-13 18:06:06.873767 7f5c973a57a0 20 read_log 192888'15667 (192888'15666) modify 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.1315259.0:16026 2014-08-11 16:45:56.519763 2014-08-13 18:06:06.873791 7f5c973a57a0 20 read_log 192888'15670 (0'0) promote b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by osd.345.0:1595 2014-08-11 16:45:59.246201 2014-08-13 18:06:06.873806 7f5c973a57a0 20 read_log 192888'15671 (192888'15670) modify b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.1315259.0:16721 2014-08-11 16:45:59.238007 2014-08-13 18:06:06.873818 7f5c973a57a0 20 read_log 192888'15674 (0'0) promote ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.345.0:1619 2014-08-11 16:46:04.679796 2014-08-13 18:06:06.873833 7f5c973a57a0 20 read_log 192888'15675 (192888'15674) modify ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by client.1315259.0:18070 2014-08-11 16:46:04.671303 2014-08-13 18:06:06.873845 7f5c973a57a0 20 read_log 192888'15678 (0'0) promote ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by osd.345.0:1625 2014-08-11 16:46:05.689748 2014-08-13 18:06:06.873859 7f5c973a57a0 20 read_log 192888'15679 (192888'15678) modify ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.1315259.0:18340 2014-08-11 16:46:05.682037 2014-08-13 18:06:06.873872 7f5c973a57a0 20 read_log 192888'15684 (0'0) promote 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by osd.345.0:1639 2014-08-11 16:46:07.263137 2014-08-13 18:06:06.873884 7f5c973a57a0 20 read_log 192888'15685 (192888'15684) modify 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.1315259.0:18760 2014-08-11 16:46:07.254130 2014-08-13 18:06:06.873896 7f5c973a57a0 20 read_log 192888'15690 (0'0) promote b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by osd.345.0:1675 2014-08-11 16:46:18.054131 2014-08-13 18:06:06.873908 7f5c973a57a0 20 read_log 192888'15691 (192888'15690) modify b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.1315259.0:21569 2014-08-11 16:46:18.046153 2014-08-13 18:06:06.873922 7f5c973a57a0 20 read_log 192888'15696 (0'0) promote afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.345.0:1713 2014-08-11 16:46:25.766605 2014-08-13 18:06:06.873935 7f5c973a57a0 20 read_log 192888'15697 (192888'15696) modify afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by client.1315259.0:23523 2014-08-11 16:46:25.756267 2014-08-13 18:06:06.873947 7f5c973a57a0 20 read_log 192888'15704 (0'0) promote 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by osd.345.0:1743 2014-08-11 16:46:34.412758 2014-08-13 18:06:06.873961 7f5c973a57a0 20 read_log 192888'15705 (192888'15704) modify 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.1315259.0:25706 2014-08-11 16:46:34.404412 2014-08-13 18:06:06.873974 7f5c973a57a0 20 read_log 192888'15706 (0'0) promote 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.345.0:1745 2014-08-11 16:46:34.625375 2014-08-13 18:06:06.873987 7f5c973a57a0 20 read_log 192888'15707 (192888'15706) modify 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by client.1315259.0:25755 2014-08-11 16:46:34.616758 2014-08-13 18:06:06.874002 7f5c973a57a0 20 read_log 192888'15714 (0'0) promote e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.345.0:1761 2014-08-11 16:46:42.022461 2014-08-13 18:06:06.874014 7f5c973a57a0 20 read_log 192888'15715 (192888'15714) modify e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by client.1315259.0:27503 2014-08-11 16:46:42.015164 2014-08-13 18:06:06.874031 7f5c973a57a0 20 read_log 192888'15716 (0'0) promote d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by osd.345.0:1775 2014-08-11 16:46:49.285156 2014-08-13 18:06:06.874045 7f5c973a57a0 20 read_log 192888'15717 (192888'15716) modify d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.1315259.0:29313 2014-08-11 16:46:49.277108 2014-08-13 18:06:06.874073 7f5c973a57a0 20 read_log 192888'15720 (0'0) promote dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by osd.345.0:1787 2014-08-11 16:46:52.870561 2014-08-13 18:06:06.874089 7f5c973a57a0 20 read_log 192888'15721 (192888'15720) modify dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.1315259.0:30204 2014-08-11 16:46:52.862993 2014-08-13 18:06:06.874101 7f5c973a57a0 20 read_log 192888'15722 (0'0) promote 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by osd.345.0:1789 2014-08-11 16:46:53.513921 2014-08-13 18:06:06.874115 7f5c973a57a0 20 read_log 192888'15723 (192888'15722) modify 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.1315259.0:30376 2014-08-11 16:46:53.506265 2014-08-13 18:06:06.874129 7f5c973a57a0 20 read_log 192888'15725 (192888'15599) modify 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.1315259.0:32416 2014-08-11 16:46:56.371678 2014-08-13 18:06:06.874164 7f5c973a57a0 20 read_log 192888'15726 (192888'15601) modify 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.1315259.0:33329 2014-08-11 16:46:56.609405 2014-08-13 18:06:06.874181 7f5c973a57a0 20 read_log 192888'15727 (192888'15603) modify daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by client.1315259.0:33510 2014-08-11 16:46:56.653872 2014-08-13 18:06:06.874193 7f5c973a57a0 20 read_log 192888'15728 (192888'15605) modify 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.1315259.0:33604 2014-08-11 16:46:56.678363 2014-08-13 18:06:06.874208 7f5c973a57a0 20 read_log 192888'15729 (192888'15607) modify e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.1315259.0:33979 2014-08-11 16:46:56.774026 2014-08-13 18:06:06.874220 7f5c973a57a0 20 read_log 192888'15730 (192888'15609) modify 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.1315259.0:34887 2014-08-11 16:46:57.011843 2014-08-13 18:06:06.874234 7f5c973a57a0 20 read_log 192888'15732 (192888'15613) modify 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.1315259.0:35111 2014-08-11 16:46:57.075854 2014-08-13 18:06:06.874247 7f5c973a57a0 20 read_log 192888'15733 (192888'15615) modify eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by client.1315259.0:35366 2014-08-11 16:46:57.146043 2014-08-13 18:06:06.874264 7f5c973a57a0 20 read_log 192888'15737 (192888'15623) modify 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.1315259.0:36888 2014-08-11 16:46:57.538679 2014-08-13 18:06:06.874277 7f5c973a57a0 20 read_log 192888'15738 (192888'15625) modify 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by client.1315259.0:36911 2014-08-11 16:46:57.543176 2014-08-13 18:06:06.874291 7f5c973a57a0 20 read_log 192888'15742 (192888'15633) modify 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.1315259.0:38885 2014-08-11 16:46:58.069406 2014-08-13 18:06:06.874303 7f5c973a57a0 20 read_log 192888'15743 (192888'15635) modify 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.1315259.0:40104 2014-08-11 16:46:58.390650 2014-08-13 18:06:06.874316 7f5c973a57a0 20 read_log 192888'15745 (192888'15639) modify cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.1315259.0:40765 2014-08-11 16:46:58.575784 2014-08-13 18:06:06.874328 7f5c973a57a0 20 read_log 192888'15746 (192888'15641) modify 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by client.1315259.0:42405 2014-08-11 16:46:59.002131 2014-08-13 18:06:06.874341 7f5c973a57a0 20 read_log 192888'15747 (192888'15643) modify 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.1315259.0:42462 2014-08-11 16:46:59.018106 2014-08-13 18:06:06.874359 7f5c973a57a0 20 read_log 192888'15750 (192888'15649) modify 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.1315259.0:43106 2014-08-11 16:46:59.194357 2014-08-13 18:06:06.874381 7f5c973a57a0 20 read_log 192888'15751 (192888'15651) modify 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.1315259.0:43775 2014-08-11 16:46:59.385060 2014-08-13 18:06:06.874411 7f5c973a57a0 20 read_log 192888'15754 (192888'15657) modify b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by client.1315259.0:44450 2014-08-11 16:46:59.581784 2014-08-13 18:06:06.874424 7f5c973a57a0 20 read_log 192888'15756 (192888'15661) modify d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by client.1315259.0:45195 2014-08-11 16:46:59.796362 2014-08-13 18:06:06.874446 7f5c973a57a0 20 read_log 192888'15757 (192888'15663) modify 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by client.1315259.0:45378 2014-08-11 16:46:59.851603 2014-08-13 18:06:06.874461 7f5c973a57a0 20 read_log 192888'15758 (192888'15665) modify 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.1315259.0:46546 2014-08-11 16:47:00.189603 2014-08-13 18:06:06.874474 7f5c973a57a0 20 read_log 192888'15759 (192888'15667) modify 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.1315259.0:46753 2014-08-11 16:47:00.248818 2014-08-13 18:06:06.874493 7f5c973a57a0 20 read_log 192888'15761 (192888'15671) modify b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.1315259.0:47448 2014-08-11 16:47:00.458454 2014-08-13 18:06:06.874509 7f5c973a57a0 20 read_log 192888'15763 (192888'15675) modify ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by client.1315259.0:48797 2014-08-11 16:47:00.869749 2014-08-13 18:06:06.874522 7f5c973a57a0 20 read_log 192888'15765 (192888'15679) modify ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.1315259.0:49067 2014-08-11 16:47:00.949050 2014-08-13 18:06:06.874541 7f5c973a57a0 20 read_log 192888'15768 (192888'15685) modify 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.1315259.0:49487 2014-08-11 16:47:01.079008 2014-08-13 18:06:06.874557 7f5c973a57a0 20 read_log 192888'15771 (192888'15691) modify b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.1315259.0:52296 2014-08-11 16:47:01.934692 2014-08-13 18:06:06.874570 7f5c973a57a0 20 read_log 192888'15774 (192888'15697) modify afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by client.1315259.0:54250 2014-08-11 16:47:02.510596 2014-08-13 18:06:06.874582 7f5c973a57a0 20 read_log 192888'15778 (192888'15705) modify 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.1315259.0:56433 2014-08-11 16:47:03.203281 2014-08-13 18:06:06.874594 7f5c973a57a0 20 read_log 192888'15779 (192888'15707) modify 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by client.1315259.0:56482 2014-08-11 16:47:03.218742 2014-08-13 18:06:06.874607 7f5c973a57a0 20 read_log 192888'15783 (192888'15715) modify e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by client.1315259.0:58230 2014-08-11 16:47:03.742243 2014-08-13 18:06:06.874619 7f5c973a57a0 20 read_log 192888'15784 (192888'15717) modify d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.1315259.0:60040 2014-08-11 16:47:04.322586 2014-08-13 18:06:06.874631 7f5c973a57a0 20 read_log 192888'15786 (192888'15721) modify dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.1315259.0:60931 2014-08-11 16:47:04.584399 2014-08-13 18:06:06.874644 7f5c973a57a0 20 read_log 192888'15787 (192888'15723) modify 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.1315259.0:61103 2014-08-11 16:47:04.634313 2014-08-13 18:06:06.874656 7f5c973a57a0 20 read_log 192888'15790 (0'0) promote 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.345.0:2021 2014-08-11 17:05:14.950939 2014-08-13 18:06:06.874669 7f5c973a57a0 20 read_log 192888'15791 (192888'15790) modify 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by client.942868.0:32 2014-08-11 17:05:14.939868 2014-08-13 18:06:06.874689 7f5c973a57a0 20 read_log 192888'15798 (0'0) promote a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by osd.345.0:2072 2014-08-11 17:05:27.415611 2014-08-13 18:06:06.874702 7f5c973a57a0 20 read_log 192888'15799 (192888'15798) modify a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.942868.0:3100 2014-08-11 17:05:27.405656 2014-08-13 18:06:06.874714 7f5c973a57a0 20 read_log 192888'15800 (0'0) promote 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.345.0:2084 2014-08-11 17:05:29.265988 2014-08-13 18:06:06.874726 7f5c973a57a0 20 read_log 192888'15801 (192888'15800) modify 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by client.942868.0:3601 2014-08-11 17:05:29.257204 2014-08-13 18:06:06.874739 7f5c973a57a0 20 read_log 192888'15802 (0'0) promote ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by osd.345.0:2086 2014-08-11 17:05:30.122575 2014-08-13 18:06:06.874751 7f5c973a57a0 20 read_log 192888'15803 (192888'15802) modify ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.942868.0:3833 2014-08-11 17:05:30.113920 2014-08-13 18:06:06.874764 7f5c973a57a0 20 read_log 192888'15806 (0'0) promote 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by osd.345.0:2092 2014-08-11 17:05:33.245421 2014-08-13 18:06:06.874776 7f5c973a57a0 20 read_log 192888'15807 (192888'15806) modify 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.942868.0:4672 2014-08-11 17:05:33.237251 2014-08-13 18:06:06.874788 7f5c973a57a0 20 read_log 192888'15808 (0'0) promote 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by osd.345.0:2094 2014-08-11 17:05:33.898574 2014-08-13 18:06:06.874800 7f5c973a57a0 20 read_log 192888'15809 (192888'15808) modify 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.942868.0:4836 2014-08-11 17:05:33.890276 2014-08-13 18:06:06.874812 7f5c973a57a0 20 read_log 192888'15814 (0'0) promote a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.345.0:2122 2014-08-11 17:05:42.381575 2014-08-13 18:06:06.874824 7f5c973a57a0 20 read_log 192888'15815 (192888'15814) modify a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by client.942868.0:7095 2014-08-11 17:05:42.373354 2014-08-13 18:06:06.874844 7f5c973a57a0 20 read_log 192888'15816 (0'0) promote ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by osd.345.0:2124 2014-08-11 17:05:42.801571 2014-08-13 18:06:06.874858 7f5c973a57a0 20 read_log 192888'15817 (192888'15816) modify ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.942868.0:7209 2014-08-11 17:05:42.794205 2014-08-13 18:06:06.874871 7f5c973a57a0 20 read_log 192888'15818 (0'0) promote 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by osd.345.0:2132 2014-08-11 17:05:43.980378 2014-08-13 18:06:06.874883 7f5c973a57a0 20 read_log 192888'15819 (192888'15818) modify 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.942868.0:7494 2014-08-11 17:05:43.970544 2014-08-13 18:06:06.874895 7f5c973a57a0 20 read_log 192888'15820 (0'0) promote 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by osd.345.0:2148 2014-08-11 17:05:46.187165 2014-08-13 18:06:06.874907 7f5c973a57a0 20 read_log 192888'15821 (192888'15820) modify 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.942868.0:8099 2014-08-11 17:05:46.177209 2014-08-13 18:06:06.874920 7f5c973a57a0 20 read_log 192888'15822 (0'0) promote 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by osd.345.0:2162 2014-08-11 17:05:48.890430 2014-08-13 18:06:06.874932 7f5c973a57a0 20 read_log 192888'15823 (192888'15822) modify 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.942868.0:8825 2014-08-11 17:05:48.882468 2014-08-13 18:06:06.874972 7f5c973a57a0 20 read_log 192888'15824 (0'0) promote 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.345.0:2164 2014-08-11 17:05:49.068551 2014-08-13 18:06:06.874985 7f5c973a57a0 20 read_log 192888'15825 (192888'15824) modify 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by client.942868.0:8874 2014-08-11 17:05:49.060330 2014-08-13 18:06:06.874998 7f5c973a57a0 20 read_log 192888'15828 (0'0) promote cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.345.0:2178 2014-08-11 17:05:51.606176 2014-08-13 18:06:06.875010 7f5c973a57a0 20 read_log 192888'15829 (192888'15828) modify cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by client.942868.0:9558 2014-08-11 17:05:51.597437 2014-08-13 18:06:06.875022 7f5c973a57a0 20 read_log 192888'15832 (0'0) promote 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.345.0:2186 2014-08-11 17:05:53.846327 2014-08-13 18:06:06.875035 7f5c973a57a0 20 read_log 192888'15833 (192888'15832) modify 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by client.942868.0:10177 2014-08-11 17:05:53.837724 2014-08-13 18:06:06.875047 7f5c973a57a0 20 read_log 192888'15834 (0'0) promote f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by osd.345.0:2188 2014-08-11 17:05:53.909740 2014-08-13 18:06:06.875059 7f5c973a57a0 20 read_log 192888'15835 (192888'15834) modify f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.942868.0:10195 2014-08-11 17:05:53.902240 2014-08-13 18:06:06.875071 7f5c973a57a0 20 read_log 192888'15836 (0'0) promote d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.345.0:2200 2014-08-11 17:05:56.661238 2014-08-13 18:06:06.875084 7f5c973a57a0 20 read_log 192888'15837 (192888'15836) modify d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by client.942868.0:10962 2014-08-11 17:05:56.651483 2014-08-13 18:06:06.875106 7f5c973a57a0 20 read_log 192888'15838 (0'0) promote f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by osd.345.0:2204 2014-08-11 17:05:58.645975 2014-08-13 18:06:06.875118 7f5c973a57a0 20 read_log 192888'15839 (192888'15838) modify f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.942868.0:11493 2014-08-11 17:05:58.638379 2014-08-13 18:06:06.875130 7f5c973a57a0 20 read_log 192888'15840 (0'0) promote d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.345.0:2212 2014-08-11 17:05:59.299350 2014-08-13 18:06:06.875142 7f5c973a57a0 20 read_log 192888'15841 (192888'15840) modify d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by client.942868.0:11675 2014-08-11 17:05:59.290480 2014-08-13 18:06:06.875162 7f5c973a57a0 20 read_log 192888'15850 (0'0) promote 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by osd.345.0:2240 2014-08-11 17:06:07.703040 2014-08-13 18:06:06.875175 7f5c973a57a0 20 read_log 192888'15851 (192888'15850) modify 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.942868.0:13885 2014-08-11 17:06:07.694867 2014-08-13 18:06:06.875188 7f5c973a57a0 20 read_log 192888'15856 (0'0) promote 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by osd.345.0:2260 2014-08-11 17:06:11.442130 2014-08-13 18:06:06.875200 7f5c973a57a0 20 read_log 192888'15857 (192888'15856) modify 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.942868.0:14800 2014-08-11 17:06:11.434797 2014-08-13 18:06:06.875213 7f5c973a57a0 20 read_log 192888'15872 (0'0) promote 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by osd.345.0:2384 2014-08-11 17:06:47.954521 2014-08-13 18:06:06.875225 7f5c973a57a0 20 read_log 192888'15873 (192888'15872) modify 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.942868.0:24338 2014-08-11 17:06:47.946456 2014-08-13 18:06:06.875253 7f5c973a57a0 20 read_log 192888'15874 (0'0) promote 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by osd.345.0:2390 2014-08-11 17:06:49.090638 2014-08-13 18:06:06.875270 7f5c973a57a0 20 read_log 192888'15875 (192888'15874) modify 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.942868.0:24642 2014-08-11 17:06:49.081543 2014-08-13 18:06:06.875282 7f5c973a57a0 20 read_log 192888'15876 (0'0) promote f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by osd.345.0:2394 2014-08-11 17:06:49.906735 2014-08-13 18:06:06.875294 7f5c973a57a0 20 read_log 192888'15877 (192888'15876) modify f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.942868.0:24863 2014-08-11 17:06:49.896991 2014-08-13 18:06:06.875328 7f5c973a57a0 20 read_log 192888'15878 (0'0) promote c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.345.0:2408 2014-08-11 17:06:54.416271 2014-08-13 18:06:06.875341 7f5c973a57a0 20 read_log 192888'15879 (192888'15878) modify c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by client.942868.0:26004 2014-08-11 17:06:54.408208 2014-08-13 18:06:06.875353 7f5c973a57a0 20 read_log 192888'15890 (0'0) promote 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by osd.345.0:2428 2014-08-11 17:06:57.901890 2014-08-13 18:06:06.875366 7f5c973a57a0 20 read_log 192888'15891 (192888'15890) modify 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.942868.0:26911 2014-08-11 17:06:57.892216 2014-08-13 18:06:06.875388 7f5c973a57a0 20 read_log 192888'15898 (0'0) promote e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by osd.345.0:2446 2014-08-11 17:07:02.715518 2014-08-13 18:06:06.875401 7f5c973a57a0 20 read_log 192888'15899 (192888'15898) modify e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.942868.0:28197 2014-08-11 17:07:02.707758 2014-08-13 18:06:06.875414 7f5c973a57a0 20 read_log 192888'15900 (0'0) promote b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by osd.345.0:2448 2014-08-11 17:07:03.020422 2014-08-13 18:06:06.875443 7f5c973a57a0 20 read_log 192888'15901 (192888'15900) modify b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.942868.0:28281 2014-08-11 17:07:03.012510 2014-08-13 18:06:06.875459 7f5c973a57a0 20 read_log 192888'15906 (192888'15791) modify 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by client.942868.0:31298 2014-08-11 17:07:15.860715 2014-08-13 18:06:06.875484 7f5c973a57a0 20 read_log 192888'15910 (192888'15799) modify a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.942868.0:34366 2014-08-11 17:07:16.767947 2014-08-13 18:06:06.875500 7f5c973a57a0 20 read_log 192888'15911 (192888'15801) modify 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by client.942868.0:34867 2014-08-11 17:07:16.918834 2014-08-13 18:06:06.875513 7f5c973a57a0 20 read_log 192888'15912 (192888'15803) modify ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.942868.0:35099 2014-08-11 17:07:16.984186 2014-08-13 18:06:06.875531 7f5c973a57a0 20 read_log 192888'15914 (192888'15807) modify 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.942868.0:35938 2014-08-11 17:07:17.202147 2014-08-13 18:06:06.875554 7f5c973a57a0 20 read_log 192888'15915 (192888'15809) modify 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.942868.0:36102 2014-08-11 17:07:17.242031 2014-08-13 18:06:06.875585 7f5c973a57a0 20 read_log 192888'15918 (192888'15815) modify a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by client.942868.0:38361 2014-08-11 17:07:17.864781 2014-08-13 18:06:06.875601 7f5c973a57a0 20 read_log 192888'15919 (192888'15817) modify ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.942868.0:38475 2014-08-11 17:07:17.897641 2014-08-13 18:06:06.875626 7f5c973a57a0 20 read_log 192888'15920 (192888'15819) modify 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.942868.0:38760 2014-08-11 17:07:17.980934 2014-08-13 18:06:06.875641 7f5c973a57a0 20 read_log 192888'15921 (192888'15821) modify 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.942868.0:39365 2014-08-11 17:07:18.162413 2014-08-13 18:06:06.875663 7f5c973a57a0 20 read_log 192888'15922 (192888'15823) modify 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.942868.0:40091 2014-08-11 17:07:18.362436 2014-08-13 18:06:06.875679 7f5c973a57a0 20 read_log 192888'15923 (192888'15825) modify 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by client.942868.0:40140 2014-08-11 17:07:18.374403 2014-08-13 18:06:06.875692 7f5c973a57a0 20 read_log 192888'15925 (192888'15829) modify cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by client.942868.0:40824 2014-08-11 17:07:18.557510 2014-08-13 18:06:06.875704 7f5c973a57a0 20 read_log 192888'15927 (192888'15833) modify 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by client.942868.0:41443 2014-08-11 17:07:18.729836 2014-08-13 18:06:06.875716 7f5c973a57a0 20 read_log 192888'15928 (192888'15835) modify f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.942868.0:41461 2014-08-11 17:07:18.734076 2014-08-13 18:06:06.875729 7f5c973a57a0 20 read_log 192888'15929 (192888'15837) modify d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by client.942868.0:42228 2014-08-11 17:07:18.942368 2014-08-13 18:06:06.875741 7f5c973a57a0 20 read_log 192888'15930 (192888'15839) modify f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.942868.0:42759 2014-08-11 17:07:19.084647 2014-08-13 18:06:06.875753 7f5c973a57a0 20 read_log 192888'15931 (192888'15841) modify d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by client.942868.0:42941 2014-08-11 17:07:19.134618 2014-08-13 18:06:06.875766 7f5c973a57a0 20 read_log 192888'15936 (192888'15851) modify 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.942868.0:45151 2014-08-11 17:07:19.759652 2014-08-13 18:06:06.875778 7f5c973a57a0 20 read_log 192888'15939 (192888'15857) modify 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.942868.0:46066 2014-08-11 17:07:20.011743 2014-08-13 18:06:06.875804 7f5c973a57a0 20 read_log 192888'15947 (192888'15873) modify 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.942868.0:55604 2014-08-11 17:07:22.899045 2014-08-13 18:06:06.875817 7f5c973a57a0 20 read_log 192888'15948 (192888'15875) modify 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.942868.0:55908 2014-08-11 17:07:22.992938 2014-08-13 18:06:06.875829 7f5c973a57a0 20 read_log 192888'15949 (192888'15877) modify f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.942868.0:56129 2014-08-11 17:07:23.059950 2014-08-13 18:06:06.875841 7f5c973a57a0 20 read_log 192888'15950 (192888'15879) modify c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by client.942868.0:57270 2014-08-11 17:07:23.408376 2014-08-13 18:06:06.875854 7f5c973a57a0 20 read_log 192888'15956 (192888'15891) modify 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.942868.0:58177 2014-08-11 17:07:23.664174 2014-08-13 18:06:06.875866 7f5c973a57a0 20 read_log 192888'15960 (192888'15899) modify e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.942868.0:59463 2014-08-11 17:07:24.053975 2014-08-13 18:06:06.875878 7f5c973a57a0 20 read_log 192888'15961 (192888'15901) modify b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.942868.0:59547 2014-08-11 17:07:24.077643 2014-08-13 18:06:06.875891 7f5c973a57a0 20 read_log 193305'15964 (0'0) promote 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:2 2014-08-11 18:42:33.589303 2014-08-13 18:06:06.875911 7f5c973a57a0 20 read_log 193305'15965 (193305'15964) modify 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by client.951337.0:162 2014-08-11 18:42:33.568773 2014-08-13 18:06:06.875926 7f5c973a57a0 20 read_log 193305'15966 (0'0) promote 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by osd.374.0:18 2014-08-11 18:42:36.025992 2014-08-13 18:06:06.875938 7f5c973a57a0 20 read_log 193305'15967 (193305'15966) modify 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.951337.0:737 2014-08-11 18:42:36.017516 2014-08-13 18:06:06.875951 7f5c973a57a0 20 read_log 193305'15968 (0'0) promote b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:61 2014-08-11 18:42:41.852589 2014-08-13 18:06:06.875963 7f5c973a57a0 20 read_log 193305'15969 (193305'15968) modify b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by client.951337.0:2301 2014-08-11 18:42:41.844652 2014-08-13 18:06:06.875976 7f5c973a57a0 20 read_log 193305'15970 (0'0) promote dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:69 2014-08-11 18:42:42.959895 2014-08-13 18:06:06.875988 7f5c973a57a0 20 read_log 193305'15971 (193305'15970) modify dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by client.951337.0:2606 2014-08-11 18:42:42.951618 2014-08-13 18:06:06.876000 7f5c973a57a0 20 read_log 193305'15972 (0'0) promote bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by osd.374.0:75 2014-08-11 18:42:43.913121 2014-08-13 18:06:06.876012 7f5c973a57a0 20 read_log 193305'15973 (193305'15972) modify bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.951337.0:2868 2014-08-11 18:42:43.904850 2014-08-13 18:06:06.876024 7f5c973a57a0 20 read_log 193305'15974 (0'0) promote e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by osd.374.0:169 2014-08-11 18:42:58.719861 2014-08-13 18:06:06.876036 7f5c973a57a0 20 read_log 193305'15975 (193305'15974) modify e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.951337.0:6910 2014-08-11 18:42:58.712353 2014-08-13 18:06:06.876049 7f5c973a57a0 20 read_log 193305'15976 (0'0) promote fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by osd.374.0:177 2014-08-11 18:42:59.933916 2014-08-13 18:06:06.876063 7f5c973a57a0 20 read_log 193305'15977 (193305'15976) modify fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.951337.0:7247 2014-08-11 18:42:59.926062 2014-08-13 18:06:06.876076 7f5c973a57a0 20 read_log 193305'15978 (0'0) promote c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:229 2014-08-11 18:43:08.668625 2014-08-13 18:06:06.876088 7f5c973a57a0 20 read_log 193305'15979 (193305'15978) modify c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by client.951337.0:9555 2014-08-11 18:43:08.660899 2014-08-13 18:06:06.876100 7f5c973a57a0 20 read_log 193305'15980 (0'0) promote ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by osd.374.0:233 2014-08-11 18:43:09.907456 2014-08-13 18:06:06.876113 7f5c973a57a0 20 read_log 193305'15981 (193305'15980) modify ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.951337.0:9893 2014-08-11 18:43:09.899852 2014-08-13 18:06:06.876125 7f5c973a57a0 20 read_log 193305'15982 (0'0) promote 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by osd.374.0:243 2014-08-11 18:43:12.413963 2014-08-13 18:06:06.876137 7f5c973a57a0 20 read_log 193305'15983 (193305'15982) modify 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.951337.0:10571 2014-08-11 18:43:12.406865 2014-08-13 18:06:06.876177 7f5c973a57a0 20 read_log 193305'15984 (0'0) promote 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by osd.374.0:259 2014-08-11 18:43:15.444564 2014-08-13 18:06:06.876205 7f5c973a57a0 20 read_log 193305'15985 (193305'15984) modify 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.951337.0:11339 2014-08-11 18:43:15.436638 2014-08-13 18:06:06.876220 7f5c973a57a0 20 read_log 193305'15986 (0'0) promote aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by osd.374.0:261 2014-08-11 18:43:15.971798 2014-08-13 18:06:06.876233 7f5c973a57a0 20 read_log 193305'15987 (193305'15986) modify aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.951337.0:11479 2014-08-11 18:43:15.963451 2014-08-13 18:06:06.876245 7f5c973a57a0 20 read_log 193305'15988 (0'0) promote ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:314 2014-08-11 18:43:23.728779 2014-08-13 18:06:06.876261 7f5c973a57a0 20 read_log 193305'15989 (193305'15988) modify ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by client.951337.0:13544 2014-08-11 18:43:23.721100 2014-08-13 18:06:06.876273 7f5c973a57a0 20 read_log 193305'15990 (0'0) promote fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:326 2014-08-11 18:43:25.133702 2014-08-13 18:06:06.876285 7f5c973a57a0 20 read_log 193305'15991 (193305'15990) modify fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by client.951337.0:13930 2014-08-11 18:43:25.125897 2014-08-13 18:06:06.876301 7f5c973a57a0 20 read_log 193305'15992 (0'0) promote 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by osd.374.0:418 2014-08-11 18:43:38.436804 2014-08-13 18:06:06.876314 7f5c973a57a0 20 read_log 193305'15993 (193305'15992) modify 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.951337.0:17746 2014-08-11 18:43:38.429380 2014-08-13 18:06:06.876326 7f5c973a57a0 20 read_log 193305'15994 (0'0) promote accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by osd.374.0:436 2014-08-11 18:43:40.385470 2014-08-13 18:06:06.876339 7f5c973a57a0 20 read_log 193305'15995 (193305'15994) modify accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.951337.0:18297 2014-08-11 18:43:40.378237 2014-08-13 18:06:06.876354 7f5c973a57a0 20 read_log 193305'15996 (0'0) promote c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by osd.374.0:450 2014-08-11 18:43:42.849426 2014-08-13 18:06:06.876367 7f5c973a57a0 20 read_log 193305'15997 (193305'15996) modify c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.951337.0:18973 2014-08-11 18:43:42.838944 2014-08-13 18:06:06.876379 7f5c973a57a0 20 read_log 193305'15998 (0'0) promote 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:452 2014-08-11 18:43:43.665472 2014-08-13 18:06:06.876403 7f5c973a57a0 20 read_log 193305'15999 (193305'15998) modify 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by client.951337.0:19183 2014-08-11 18:43:43.657263 2014-08-13 18:06:06.876421 7f5c973a57a0 20 read_log 193305'16000 (0'0) promote f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by osd.374.0:468 2014-08-11 18:43:46.535369 2014-08-13 18:06:06.876433 7f5c973a57a0 20 read_log 193305'16001 (193305'16000) modify f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.951337.0:19978 2014-08-11 18:43:46.526511 2014-08-13 18:06:06.876458 7f5c973a57a0 20 read_log 193305'16002 (0'0) promote 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by osd.374.0:496 2014-08-11 18:43:50.893444 2014-08-13 18:06:06.876471 7f5c973a57a0 20 read_log 193305'16003 (193305'16002) modify 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.951337.0:21188 2014-08-11 18:43:50.883961 2014-08-13 18:06:06.876484 7f5c973a57a0 20 read_log 193305'16004 (0'0) promote 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:518 2014-08-11 18:43:54.164507 2014-08-13 18:06:06.876514 7f5c973a57a0 20 read_log 193305'16005 (193305'16004) modify 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by client.951337.0:22094 2014-08-11 18:43:54.156202 2014-08-13 18:06:06.876530 7f5c973a57a0 20 read_log 193305'16006 (0'0) promote afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by osd.374.0:552 2014-08-11 18:43:59.695747 2014-08-13 18:06:06.876542 7f5c973a57a0 20 read_log 193305'16007 (193305'16006) modify afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.951337.0:23632 2014-08-11 18:43:59.688466 2014-08-13 18:06:06.876555 7f5c973a57a0 20 read_log 193305'16008 (0'0) promote fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by osd.374.0:558 2014-08-11 18:44:00.115830 2014-08-13 18:06:06.876568 7f5c973a57a0 20 read_log 193305'16009 (193305'16008) modify fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.951337.0:23751 2014-08-11 18:44:00.108332 2014-08-13 18:06:06.876586 7f5c973a57a0 20 read_log 193305'16010 (0'0) promote bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by osd.374.0:567 2014-08-11 18:44:00.914886 2014-08-13 18:06:06.876599 7f5c973a57a0 20 read_log 193305'16011 (193305'16010) modify bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.951337.0:23971 2014-08-11 18:44:00.908506 2014-08-13 18:06:06.876623 7f5c973a57a0 20 read_log 193305'16012 (0'0) promote 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by osd.374.0:573 2014-08-11 18:44:01.679936 2014-08-13 18:06:06.876636 7f5c973a57a0 20 read_log 193305'16013 (193305'16012) modify 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.951337.0:24174 2014-08-11 18:44:01.670748 2014-08-13 18:06:06.876649 7f5c973a57a0 20 read_log 193305'16014 (0'0) promote 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by osd.374.0:589 2014-08-11 18:44:03.699386 2014-08-13 18:06:06.876661 7f5c973a57a0 20 read_log 193305'16015 (193305'16014) modify 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.951337.0:24744 2014-08-11 18:44:03.690280 2014-08-13 18:06:06.876673 7f5c973a57a0 20 read_log 193305'16016 (0'0) promote f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by osd.374.0:663 2014-08-11 18:44:15.034704 2014-08-13 18:06:06.876686 7f5c973a57a0 20 read_log 193305'16017 (193305'16016) modify f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.951337.0:27825 2014-08-11 18:44:15.027941 2014-08-13 18:06:06.876705 7f5c973a57a0 20 read_log 193305'16018 (193305'16018) modify 2ec/hit_set_24.2ec_archive_2014-08-11 18:39:19.769124_2014-08-11 18:44:23.450177/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:23.450471 2014-08-13 18:06:06.876722 7f5c973a57a0 20 read_log 193305'16019 (0'0) promote d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:718 2014-08-11 18:44:23.454196 2014-08-13 18:06:06.876734 7f5c973a57a0 20 read_log 193305'16020 (193305'16019) modify d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by client.951337.0:30108 2014-08-11 18:44:23.441590 2014-08-13 18:06:06.876747 7f5c973a57a0 20 read_log 193305'16021 (0'0) promote ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by osd.374.0:724 2014-08-11 18:44:24.495194 2014-08-13 18:06:06.876759 7f5c973a57a0 20 read_log 193305'16022 (193305'16021) modify ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.951337.0:30370 2014-08-11 18:44:24.486716 2014-08-13 18:06:06.876772 7f5c973a57a0 20 read_log 193305'16023 (193305'15965) modify 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by client.951337.0:32873 2014-08-11 18:44:33.922295 2014-08-13 18:06:06.876791 7f5c973a57a0 20 read_log 193305'16024 (193305'15967) modify 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.951337.0:33448 2014-08-11 18:44:34.432454 2014-08-13 18:06:06.876806 7f5c973a57a0 20 read_log 193305'16025 (193305'15969) modify b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by client.951337.0:35012 2014-08-11 18:44:35.328807 2014-08-13 18:06:06.876819 7f5c973a57a0 20 read_log 193305'16026 (193305'15971) modify dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by client.951337.0:35317 2014-08-11 18:44:35.506860 2014-08-13 18:06:06.876831 7f5c973a57a0 20 read_log 193305'16027 (193305'15973) modify bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.951337.0:35579 2014-08-11 18:44:35.666618 2014-08-13 18:06:06.876844 7f5c973a57a0 20 read_log 193305'16028 (193305'15975) modify e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.951337.0:39621 2014-08-11 18:44:38.114615 2014-08-13 18:06:06.876856 7f5c973a57a0 20 read_log 193305'16029 (193305'15977) modify fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.951337.0:39958 2014-08-11 18:44:38.237807 2014-08-13 18:06:06.876878 7f5c973a57a0 20 read_log 193305'16030 (193305'15979) modify c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by client.951337.0:42266 2014-08-11 18:44:39.169567 2014-08-13 18:06:06.876891 7f5c973a57a0 20 read_log 193305'16031 (193305'15981) modify ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.951337.0:42604 2014-08-11 18:44:39.311978 2014-08-13 18:06:06.876904 7f5c973a57a0 20 read_log 193305'16032 (193305'15983) modify 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.951337.0:43282 2014-08-11 18:44:39.542699 2014-08-13 18:06:06.876916 7f5c973a57a0 20 read_log 193305'16033 (193305'15985) modify 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.951337.0:44050 2014-08-11 18:44:39.780677 2014-08-13 18:06:06.876929 7f5c973a57a0 20 read_log 193305'16034 (193305'15987) modify aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.951337.0:44190 2014-08-11 18:44:39.817235 2014-08-13 18:06:06.876943 7f5c973a57a0 20 read_log 193305'16035 (193305'15989) modify ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by client.951337.0:46255 2014-08-11 18:44:40.432902 2014-08-13 18:06:06.876978 7f5c973a57a0 20 read_log 193305'16036 (193305'15991) modify fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by client.951337.0:46641 2014-08-11 18:44:40.544893 2014-08-13 18:06:06.876998 7f5c973a57a0 20 read_log 193305'16037 (193305'15993) modify 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.951337.0:50457 2014-08-11 18:44:41.671977 2014-08-13 18:06:06.877011 7f5c973a57a0 20 read_log 193305'16038 (193305'15995) modify accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.951337.0:51008 2014-08-11 18:44:41.826160 2014-08-13 18:06:06.877023 7f5c973a57a0 20 read_log 193305'16039 (193305'15997) modify c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.951337.0:51684 2014-08-11 18:44:42.023248 2014-08-13 18:06:06.877036 7f5c973a57a0 20 read_log 193305'16040 (193305'15999) modify 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by client.951337.0:51894 2014-08-11 18:44:42.080729 2014-08-13 18:06:06.877052 7f5c973a57a0 20 read_log 193305'16041 (193305'16001) modify f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.951337.0:52689 2014-08-11 18:44:42.309714 2014-08-13 18:06:06.877072 7f5c973a57a0 20 read_log 193305'16042 (193305'16003) modify 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.951337.0:53899 2014-08-11 18:44:42.668428 2014-08-13 18:06:06.877086 7f5c973a57a0 20 read_log 193305'16043 (193305'16005) modify 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by client.951337.0:54805 2014-08-11 18:44:42.944183 2014-08-13 18:06:06.877131 7f5c973a57a0 20 read_log 193305'16044 (193305'16007) modify afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.951337.0:56343 2014-08-11 18:44:43.391063 2014-08-13 18:06:06.877153 7f5c973a57a0 20 read_log 193305'16045 (193305'16009) modify fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.951337.0:56462 2014-08-11 18:44:43.425998 2014-08-13 18:06:06.877176 7f5c973a57a0 20 read_log 193305'16046 (193305'16011) modify bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.951337.0:56682 2014-08-11 18:44:43.497290 2014-08-13 18:06:06.877189 7f5c973a57a0 20 read_log 193305'16047 (193305'16013) modify 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.951337.0:56885 2014-08-11 18:44:43.563611 2014-08-13 18:06:06.877201 7f5c973a57a0 20 read_log 193305'16048 (193305'16015) modify 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.951337.0:57455 2014-08-11 18:44:43.759270 2014-08-13 18:06:06.877214 7f5c973a57a0 20 read_log 193305'16049 (193305'16017) modify f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.951337.0:60536 2014-08-11 18:44:44.658970 2014-08-13 18:06:06.877230 7f5c973a57a0 20 read_log 193305'16050 (193305'16020) modify d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by client.951337.0:62819 2014-08-11 18:44:45.336165 2014-08-13 18:06:06.877242 7f5c973a57a0 20 read_log 193305'16051 (193305'16022) modify ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.951337.0:63081 2014-08-11 18:44:45.404051 2014-08-13 18:06:06.877255 7f5c973a57a0 20 read_log 194209'16052 (192888'15733) clean eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.374.0:1947 2014-08-12 14:07:58.520520 2014-08-13 18:06:06.877267 7f5c973a57a0 20 read_log 194209'16053 (192888'15756) clean d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.374.0:1948 2014-08-12 14:07:58.520950 2014-08-13 18:06:06.877280 7f5c973a57a0 20 read_log 194209'16054 (192888'15727) clean daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.374.0:1949 2014-08-12 14:07:58.521359 2014-08-13 18:06:06.877292 7f5c973a57a0 20 read_log 194209'16055 (192888'15906) clean 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.374.0:1950 2014-08-12 14:07:58.522593 2014-08-13 18:06:06.877305 7f5c973a57a0 20 read_log 194209'16056 (192888'15779) clean 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.374.0:1951 2014-08-12 14:07:58.533857 2014-08-13 18:06:06.877317 7f5c973a57a0 20 read_log 194209'16057 (192888'15763) clean ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.374.0:1952 2014-08-12 14:07:58.534304 2014-08-13 18:06:06.877329 7f5c973a57a0 20 read_log 194209'16058 (193305'16043) clean 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:1953 2014-08-12 14:07:58.538127 2014-08-13 18:06:06.877351 7f5c973a57a0 20 read_log 194209'16059 (192834'15587) clean 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by osd.374.0:1954 2014-08-12 14:07:58.541366 2014-08-13 18:06:06.877364 7f5c973a57a0 20 read_log 194209'16060 (193305'16040) clean 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:1955 2014-08-12 14:07:58.544816 2014-08-13 18:06:06.877390 7f5c973a57a0 20 read_log 194209'16061 (192834'15088) clean 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by osd.374.0:1956 2014-08-12 14:07:58.548033 2014-08-13 18:06:06.877406 7f5c973a57a0 20 read_log 194209'16062 (192888'15936) clean 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by osd.374.0:1957 2014-08-12 14:07:58.551329 2014-08-13 18:06:06.877437 7f5c973a57a0 20 read_log 194209'16063 (193305'16050) clean d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:1958 2014-08-12 14:07:58.557403 2014-08-13 18:06:06.877454 7f5c973a57a0 20 read_log 194209'16064 (192834'15057) clean 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.374.0:1959 2014-08-12 14:07:58.560464 2014-08-13 18:06:06.877467 7f5c973a57a0 20 read_log 194209'16065 (192834'15046) clean 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.374.0:1960 2014-08-12 14:07:58.560756 2014-08-13 18:06:06.877482 7f5c973a57a0 20 read_log 194209'16066 (192888'15774) clean afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.374.0:1961 2014-08-12 14:07:58.563943 2014-08-13 18:06:06.877495 7f5c973a57a0 20 read_log 194209'16067 (192834'15576) clean d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.374.0:1962 2014-08-12 14:07:58.567603 2014-08-13 18:06:06.877515 7f5c973a57a0 20 read_log 194209'16068 (192888'15911) clean 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.374.0:1963 2014-08-12 14:07:58.570907 2014-08-13 18:06:06.877529 7f5c973a57a0 20 read_log 194209'16069 (192888'15925) clean cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.374.0:1964 2014-08-12 14:07:58.574167 2014-08-13 18:06:06.877542 7f5c973a57a0 20 read_log 194209'16070 (193305'16035) clean ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:1965 2014-08-12 14:07:58.577522 2014-08-13 18:06:06.877554 7f5c973a57a0 20 read_log 194209'16071 (192888'15923) clean 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.374.0:1966 2014-08-12 14:07:58.580933 2014-08-13 18:06:06.877567 7f5c973a57a0 20 read_log 194209'16072 (193305'16023) clean 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:1967 2014-08-12 14:07:58.586957 2014-08-13 18:06:06.877579 7f5c973a57a0 20 read_log 194209'16073 (192834'14934) clean b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.374.0:1968 2014-08-12 14:07:58.587255 2014-08-13 18:06:06.877591 7f5c973a57a0 20 read_log 194209'16074 (192888'15950) clean c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.374.0:1969 2014-08-12 14:07:58.590499 2014-08-13 18:06:06.877604 7f5c973a57a0 20 read_log 194209'16075 (192888'15783) clean e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.374.0:1970 2014-08-12 14:07:58.596901 2014-08-13 18:06:06.877616 7f5c973a57a0 20 read_log 194209'16076 (192888'15918) clean a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.374.0:1971 2014-08-12 14:07:58.597266 2014-08-13 18:06:06.877636 7f5c973a57a0 20 read_log 194209'16077 (192834'14971) clean 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.374.0:1972 2014-08-12 14:07:58.600442 2014-08-13 18:06:06.877649 7f5c973a57a0 20 read_log 194209'16078 (193305'16036) clean fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:1973 2014-08-12 14:07:58.604024 2014-08-13 18:06:06.877661 7f5c973a57a0 20 read_log 194209'16079 (192888'15927) clean 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.374.0:1974 2014-08-12 14:07:58.607266 2014-08-13 18:06:06.877674 7f5c973a57a0 20 read_log 194209'16080 (193305'16030) clean c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:1975 2014-08-12 14:07:58.610585 2014-08-13 18:06:06.877686 7f5c973a57a0 20 read_log 194209'16081 (192834'15529) clean 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.374.0:1976 2014-08-12 14:07:58.613838 2014-08-13 18:06:06.877698 7f5c973a57a0 20 read_log 194209'16082 (192834'15011) clean bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.374.0:1977 2014-08-12 14:07:58.616941 2014-08-13 18:06:06.877718 7f5c973a57a0 20 read_log 194209'16083 (192834'15559) clean ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.374.0:1978 2014-08-12 14:07:58.620115 2014-08-13 18:06:06.877733 7f5c973a57a0 20 read_log 194209'16084 (192834'14950) clean 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.374.0:1979 2014-08-12 14:07:58.623165 2014-08-13 18:06:06.877746 7f5c973a57a0 20 read_log 194209'16085 (192834'15457) clean 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.374.0:1980 2014-08-12 14:07:58.626226 2014-08-13 18:06:06.877758 7f5c973a57a0 20 read_log 194209'16086 (192888'15738) clean 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.374.0:1981 2014-08-12 14:07:58.631939 2014-08-13 18:06:06.877771 7f5c973a57a0 20 read_log 194209'16087 (192834'15514) clean 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.374.0:1982 2014-08-12 14:07:58.635180 2014-08-13 18:06:06.877783 7f5c973a57a0 20 read_log 194209'16088 (192834'15000) clean 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.374.0:1983 2014-08-12 14:07:58.635506 2014-08-13 18:06:06.877795 7f5c973a57a0 20 read_log 194209'16089 (193305'16026) clean dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:1984 2014-08-12 14:07:58.638645 2014-08-13 18:06:06.877807 7f5c973a57a0 20 read_log 194209'16090 (192834'15432) clean d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.374.0:1985 2014-08-12 14:07:58.641903 2014-08-13 18:06:06.877820 7f5c973a57a0 20 read_log 194209'16091 (192834'15441) clean 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.374.0:1986 2014-08-12 14:07:58.645097 2014-08-13 18:06:06.877832 7f5c973a57a0 20 read_log 194209'16092 (192834'15467) clean 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.374.0:1987 2014-08-12 14:07:58.648203 2014-08-13 18:06:06.877844 7f5c973a57a0 20 read_log 194209'16093 (192888'15754) clean b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.374.0:1988 2014-08-12 14:07:58.652250 2014-08-13 18:06:06.877857 7f5c973a57a0 20 read_log 194209'16094 (192834'14960) clean 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.374.0:1989 2014-08-12 14:07:58.655939 2014-08-13 18:06:06.877869 7f5c973a57a0 20 read_log 194209'16095 (192888'15931) clean d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.374.0:1990 2014-08-12 14:07:58.659401 2014-08-13 18:06:06.877881 7f5c973a57a0 20 read_log 194209'16096 (193305'16025) clean b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:1991 2014-08-12 14:07:58.659662 2014-08-13 18:06:06.877894 7f5c973a57a0 20 read_log 194209'16097 (192834'14919) clean 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.374.0:1992 2014-08-12 14:07:58.663193 2014-08-13 18:06:06.877907 7f5c973a57a0 20 read_log 194209'16098 (192888'15746) clean 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.374.0:1993 2014-08-12 14:07:58.667229 2014-08-13 18:06:06.877919 7f5c973a57a0 20 read_log 194209'16099 (192888'15757) clean 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.374.0:1994 2014-08-12 14:07:58.670357 2014-08-13 18:06:06.877931 7f5c973a57a0 20 read_log 194209'16100 (192888'15929) clean d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.374.0:1995 2014-08-12 14:07:58.676080 2014-08-13 18:06:06.877944 7f5c973a57a0 20 read_log 194209'16101 (192834'14998) clean 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.374.0:1996 2014-08-12 14:07:58.676637 2014-08-13 18:06:06.877956 7f5c973a57a0 20 read_log 194209'16102 (192834'14907) clean b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.374.0:1997 2014-08-12 14:07:58.679578 2014-08-13 18:06:06.877978 7f5c973a57a0 20 read_log 194209'16103 (194209'16052) delete eba9f2ec/benchmark_data_storage0111-ib.pocstor_41097_object4638/head//24 by osd.374.0:1998 2014-08-12 14:07:58.715623 2014-08-13 18:06:06.877991 7f5c973a57a0 20 read_log 194209'16104 (194209'16053) delete d32af2ec/benchmark_data_storage0111-ib.pocstor_41097_object14467/head//24 by osd.374.0:1999 2014-08-12 14:07:58.719320 2014-08-13 18:06:06.878003 7f5c973a57a0 20 read_log 194209'16105 (194209'16054) delete daeb06ec/benchmark_data_storage0111-ib.pocstor_41097_object2782/head//24 by osd.374.0:2000 2014-08-12 14:07:58.722239 2014-08-13 18:06:06.878015 7f5c973a57a0 20 read_log 194209'16106 (194209'16055) delete 324c06ec/benchmark_data_storage0111-ib.pocstor_2228_object31/head//24 by osd.374.0:2001 2014-08-12 14:07:58.725393 2014-08-13 18:06:06.878028 7f5c973a57a0 20 read_log 194209'16107 (194209'16056) delete 4e9116ec/benchmark_data_storage0111-ib.pocstor_41097_object25754/head//24 by osd.374.0:2002 2014-08-12 14:07:58.729182 2014-08-13 18:06:06.878040 7f5c973a57a0 20 read_log 194209'16108 (194209'16057) delete ae6426ec/benchmark_data_storage0111-ib.pocstor_41097_object18069/head//24 by osd.374.0:2003 2014-08-12 14:07:58.732796 2014-08-13 18:06:06.878052 7f5c973a57a0 20 read_log 194209'16109 (194209'16058) delete 973526ec/benchmark_data_storage0111-ib.pocstor_62621_object22093/head//24 by osd.374.0:2004 2014-08-12 14:07:58.736764 2014-08-13 18:06:06.878064 7f5c973a57a0 20 read_log 194209'16110 (194209'16060) delete 657546ec/benchmark_data_storage0111-ib.pocstor_62621_object19182/head//24 by osd.374.0:2005 2014-08-12 14:07:58.740501 2014-08-13 18:06:06.878077 7f5c973a57a0 20 read_log 194209'16111 (194209'16063) delete d6ff66ec/benchmark_data_storage0111-ib.pocstor_62621_object30107/head//24 by osd.374.0:2006 2014-08-12 14:07:58.744034 2014-08-13 18:06:06.878092 7f5c973a57a0 20 read_log 194209'16112 (194209'16070) delete ab0da6ec/benchmark_data_storage0111-ib.pocstor_62621_object13543/head//24 by osd.374.0:2007 2014-08-12 14:07:58.745651 2014-08-13 18:06:06.878104 7f5c973a57a0 20 read_log 194209'16113 (194209'16072) delete 3059b6ec/benchmark_data_storage0111-ib.pocstor_62621_object161/head//24 by osd.374.0:2008 2014-08-12 14:07:58.749014 2014-08-13 18:06:06.878118 7f5c973a57a0 20 read_log 194209'16114 (194209'16078) delete fb47d6ec/benchmark_data_storage0111-ib.pocstor_62621_object13929/head//24 by osd.374.0:2009 2014-08-12 14:07:58.750747 2014-08-13 18:06:06.878131 7f5c973a57a0 20 read_log 194209'16115 (194209'16080) delete c7a6f6ec/benchmark_data_storage0111-ib.pocstor_62621_object9554/head//24 by osd.374.0:2010 2014-08-12 14:07:58.754096 2014-08-13 18:06:06.878150 7f5c973a57a0 20 read_log 194209'16116 (194209'16089) delete dbea4aec/benchmark_data_storage0111-ib.pocstor_62621_object2605/head//24 by osd.374.0:2011 2014-08-12 14:07:58.758915 2014-08-13 18:06:06.878165 7f5c973a57a0 20 read_log 194209'16117 (194209'16096) delete b7617aec/benchmark_data_storage0111-ib.pocstor_62621_object2300/head//24 by osd.374.0:2012 2014-08-12 14:07:58.760546 2014-08-13 18:06:06.878178 7f5c973a57a0 20 read_log 194209'16118 (192834'14908) clean 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.374.0:2013 2014-08-12 14:07:58.828989 2014-08-13 18:06:06.878192 7f5c973a57a0 20 read_log 194209'16119 (194209'16064) delete 7fd476ec/benchmark_data_storage0111-ib.pocstor_26659_object19336/head//24 by osd.374.0:2014 2014-08-12 14:08:03.831205 2014-08-13 18:06:06.878205 7f5c973a57a0 20 read_log 194209'16120 (194209'16065) delete 8a8386ec/benchmark_data_storage0111-ib.pocstor_26659_object17859/head//24 by osd.374.0:2015 2014-08-12 14:08:03.831662 2014-08-13 18:06:06.878219 7f5c973a57a0 20 read_log 194209'16121 (194209'16066) delete afd786ec/benchmark_data_storage0111-ib.pocstor_41097_object23522/head//24 by osd.374.0:2016 2014-08-12 14:08:03.832042 2014-08-13 18:06:06.878232 7f5c973a57a0 20 read_log 194209'16122 (194209'16067) delete d26986ec/benchmark_data_storage0111-ib.pocstor_41526_object21186/head//24 by osd.374.0:2017 2014-08-12 14:08:03.832431 2014-08-13 18:06:06.878255 7f5c973a57a0 20 read_log 194209'16123 (194209'16068) delete 2ae6a6ec/benchmark_data_storage0111-ib.pocstor_2228_object3600/head//24 by osd.374.0:2018 2014-08-12 14:08:03.835696 2014-08-13 18:06:06.878271 7f5c973a57a0 20 read_log 194209'16124 (194209'16069) delete cf78a6ec/benchmark_data_storage0111-ib.pocstor_2228_object9557/head//24 by osd.374.0:2019 2014-08-12 14:08:03.836546 2014-08-13 18:06:06.878283 7f5c973a57a0 20 read_log 194209'16125 (194209'16071) delete 60c2b6ec/benchmark_data_storage0111-ib.pocstor_2228_object8873/head//24 by osd.374.0:2020 2014-08-12 14:08:03.836924 2014-08-13 18:06:06.878296 7f5c973a57a0 20 read_log 194209'16126 (194209'16073) delete b92cb6ec/benchmark_data_storage0111-ib.pocstor_26659_object5648/head//24 by osd.374.0:2021 2014-08-12 14:08:03.837861 2014-08-13 18:06:06.878312 7f5c973a57a0 20 read_log 194209'16127 (194209'16074) delete c0c4c6ec/benchmark_data_storage0111-ib.pocstor_2228_object26003/head//24 by osd.374.0:2022 2014-08-12 14:08:03.840579 2014-08-13 18:06:06.878324 7f5c973a57a0 20 read_log 194209'16128 (194209'16075) delete e48ec6ec/benchmark_data_storage0111-ib.pocstor_41097_object27502/head//24 by osd.374.0:2023 2014-08-12 14:08:03.841327 2014-08-13 18:06:06.878381 7f5c973a57a0 20 read_log 194209'16129 (194209'16076) delete a270d6ec/benchmark_data_storage0111-ib.pocstor_2228_object7094/head//24 by osd.374.0:2024 2014-08-12 14:08:03.841685 2014-08-13 18:06:06.878402 7f5c973a57a0 20 read_log 194209'16130 (194209'16077) delete 3eb6d6ec/benchmark_data_storage0111-ib.pocstor_26659_object9045/head//24 by osd.374.0:2025 2014-08-12 14:08:03.842455 2014-08-13 18:06:06.878417 7f5c973a57a0 20 read_log 194209'16131 (194209'16079) delete 2de0f6ec/benchmark_data_storage0111-ib.pocstor_2228_object10176/head//24 by osd.374.0:2026 2014-08-12 14:08:03.846380 2014-08-13 18:06:06.878440 7f5c973a57a0 20 read_log 194209'16132 (194209'16081) delete 6a190aec/benchmark_data_storage0111-ib.pocstor_41526_object14618/head//24 by osd.374.0:2027 2014-08-12 14:08:03.846757 2014-08-13 18:06:06.878454 7f5c973a57a0 20 read_log 194209'16133 (194209'16082) delete bbfb0aec/benchmark_data_storage0111-ib.pocstor_26659_object13915/head//24 by osd.374.0:2028 2014-08-12 14:08:03.847631 2014-08-13 18:06:06.878504 7f5c973a57a0 20 read_log 194209'16134 (194209'16083) delete ba621aec/benchmark_data_storage0111-ib.pocstor_41526_object19306/head//24 by osd.374.0:2029 2014-08-12 14:08:03.847998 2014-08-13 18:06:06.878518 7f5c973a57a0 20 read_log 194209'16135 (194209'16084) delete 55362aec/benchmark_data_storage0111-ib.pocstor_26659_object7172/head//24 by osd.374.0:2030 2014-08-12 14:08:03.851317 2014-08-13 18:06:06.878535 7f5c973a57a0 20 read_log 194209'16136 (194209'16085) delete 1a5b2aec/benchmark_data_storage0111-ib.pocstor_41526_object3455/head//24 by osd.374.0:2031 2014-08-12 14:08:03.851686 2014-08-13 18:06:06.878548 7f5c973a57a0 20 read_log 194209'16137 (194209'16086) delete 76693aec/benchmark_data_storage0111-ib.pocstor_41097_object6183/head//24 by osd.374.0:2032 2014-08-12 14:08:03.852801 2014-08-13 18:06:06.878561 7f5c973a57a0 20 read_log 194209'16138 (194209'16087) delete 46514aec/benchmark_data_storage0111-ib.pocstor_41526_object12433/head//24 by osd.374.0:2033 2014-08-12 14:08:03.853221 2014-08-13 18:06:06.878573 7f5c973a57a0 20 read_log 194209'16139 (194209'16088) delete 9b854aec/benchmark_data_storage0111-ib.pocstor_26659_object12346/head//24 by osd.374.0:2034 2014-08-12 14:08:03.856616 2014-08-13 18:06:06.878585 7f5c973a57a0 20 read_log 194209'16140 (194209'16090) delete d1b06aec/benchmark_data_storage0111-ib.pocstor_41526_object686/head//24 by osd.374.0:2035 2014-08-12 14:08:03.857036 2014-08-13 18:06:06.878598 7f5c973a57a0 20 read_log 194209'16141 (194209'16091) delete 7cd46aec/benchmark_data_storage0111-ib.pocstor_41526_object2076/head//24 by osd.374.0:2036 2014-08-12 14:08:03.857696 2014-08-13 18:06:06.878610 7f5c973a57a0 20 read_log 194209'16142 (194209'16092) delete 38256aec/benchmark_data_storage0111-ib.pocstor_41526_object5147/head//24 by osd.374.0:2037 2014-08-12 14:08:03.858056 2014-08-13 18:06:06.878630 7f5c973a57a0 20 read_log 194209'16143 (194209'16093) delete b9f86aec/benchmark_data_storage0111-ib.pocstor_41097_object13722/head//24 by osd.374.0:2038 2014-08-12 14:08:03.861330 2014-08-13 18:06:06.878646 7f5c973a57a0 20 read_log 194209'16144 (194209'16094) delete 6b7a6aec/benchmark_data_storage0111-ib.pocstor_26659_object8067/head//24 by osd.374.0:2039 2014-08-12 14:08:03.862198 2014-08-13 18:06:06.878658 7f5c973a57a0 20 read_log 194209'16145 (194209'16095) delete d83c6aec/benchmark_data_storage0111-ib.pocstor_2228_object11674/head//24 by osd.374.0:2040 2014-08-12 14:08:03.862603 2014-08-13 18:06:06.878671 7f5c973a57a0 20 read_log 194209'16146 (194209'16097) delete 24787aec/benchmark_data_storage0111-ib.pocstor_26659_object3379/head//24 by osd.374.0:2041 2014-08-12 14:08:03.863440 2014-08-13 18:06:06.878683 7f5c973a57a0 20 read_log 194209'16147 (194209'16098) delete 314d7aec/benchmark_data_storage0111-ib.pocstor_41097_object11677/head//24 by osd.374.0:2042 2014-08-12 14:08:03.866342 2014-08-13 18:06:06.878695 7f5c973a57a0 20 read_log 194209'16148 (194209'16099) delete 869d7aec/benchmark_data_storage0111-ib.pocstor_41097_object14650/head//24 by osd.374.0:2043 2014-08-12 14:08:03.867225 2014-08-13 18:06:06.878707 7f5c973a57a0 20 read_log 194209'16149 (194209'16100) delete d7a49aec/benchmark_data_storage0111-ib.pocstor_2228_object10961/head//24 by osd.374.0:2044 2014-08-12 14:08:03.867596 2014-08-13 18:06:06.878720 7f5c973a57a0 20 read_log 194209'16150 (194209'16101) delete 1a26aaec/benchmark_data_storage0111-ib.pocstor_26659_object12113/head//24 by osd.374.0:2045 2014-08-12 14:08:03.868457 2014-08-13 18:06:06.878732 7f5c973a57a0 20 read_log 194209'16151 (194209'16102) delete b1b1baec/benchmark_data_storage0111-ib.pocstor_26659_object2279/head//24 by osd.374.0:2046 2014-08-12 14:08:03.871328 2014-08-13 18:06:06.878744 7f5c973a57a0 20 read_log 194209'16152 (194209'16118) delete 799abaec/benchmark_data_storage0111-ib.pocstor_26659_object2330/head//24 by osd.374.0:2047 2014-08-12 14:08:03.872198 2014-08-13 18:06:06.878757 7f5c973a57a0 20 read_log 194215'16153 (194215'16153) modify 2ec/hit_set_24.2ec_archive_2014-08-12 15:32:26.012127_2014-08-12 15:47:44.557466/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:47:44.557704 2014-08-13 18:06:06.878769 7f5c973a57a0 20 read_log 194215'16154 (193305'16018) delete 2ec/hit_set_24.2ec_archive_2014-08-11 18:39:19.769124_2014-08-11 18:44:23.450177/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:47:44.557704 2014-08-13 18:06:06.878782 7f5c973a57a0 20 read_log 194215'16155 (192888'15745) clean cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by osd.374.0:3981 2014-08-12 15:47:44.560989 2014-08-13 18:06:06.878794 7f5c973a57a0 20 read_log 194215'16156 (194215'16155) delete cb3202ec/benchmark_data_storage0111-ib.pocstor_41097_object10037/head//24 by client.2174595.0:154013 0.000000 2014-08-13 18:06:06.878805 7f5c973a57a0 20 read_log 194215'16157 (192888'15930) clean f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by osd.374.0:3983 2014-08-12 15:47:44.568991 2014-08-13 18:06:06.878818 7f5c973a57a0 20 read_log 194215'16158 (194215'16157) delete f3ba02ec/benchmark_data_storage0111-ib.pocstor_2228_object11492/head//24 by client.2174595.0:154015 0.000000 2014-08-13 18:06:06.878829 7f5c973a57a0 20 read_log 194215'16159 (192888'15728) clean 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by osd.374.0:3985 2014-08-12 15:47:44.576581 2014-08-13 18:06:06.878841 7f5c973a57a0 20 read_log 194215'16160 (194215'16159) delete 5abd02ec/benchmark_data_storage0111-ib.pocstor_41097_object2876/head//24 by client.2174595.0:154017 0.000000 2014-08-13 18:06:06.878852 7f5c973a57a0 20 read_log 194215'16161 (193305'16049) clean f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by osd.374.0:3987 2014-08-12 15:47:44.584405 2014-08-13 18:06:06.878874 7f5c973a57a0 20 read_log 194215'16162 (194215'16161) delete f4c12ec/benchmark_data_storage0111-ib.pocstor_62621_object27824/head//24 by client.2174595.0:154019 0.000000 2014-08-13 18:06:06.878888 7f5c973a57a0 20 read_log 194215'16163 (192834'15536) clean 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by osd.374.0:3989 2014-08-12 15:47:44.592385 2014-08-13 18:06:06.878901 7f5c973a57a0 20 read_log 194215'16164 (194215'16163) delete 69c322ec/benchmark_data_storage0111-ib.pocstor_41526_object15296/head//24 by client.2174595.0:154021 0.000000 2014-08-13 18:06:06.878912 7f5c973a57a0 20 read_log 194215'16165 (192888'15960) clean e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by osd.374.0:3991 2014-08-12 15:47:44.600770 2014-08-13 18:06:06.878924 7f5c973a57a0 20 read_log 194215'16166 (194215'16165) delete e56032ec/benchmark_data_storage0111-ib.pocstor_2228_object28196/head//24 by client.2174595.0:154023 0.000000 2014-08-13 18:06:06.878935 7f5c973a57a0 20 read_log 194215'16167 (193305'16048) clean 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by osd.374.0:3993 2014-08-12 15:47:44.607174 2014-08-13 18:06:06.878949 7f5c973a57a0 20 read_log 194215'16168 (194215'16167) delete 6ff432ec/benchmark_data_storage0111-ib.pocstor_62621_object24743/head//24 by client.2174595.0:154025 0.000000 2014-08-13 18:06:06.878962 7f5c973a57a0 20 read_log 194215'16169 (192888'15726) clean 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by osd.374.0:3995 2014-08-12 15:47:44.614105 2014-08-13 18:06:06.878974 7f5c973a57a0 20 read_log 194215'16170 (194215'16169) delete 7e1042ec/benchmark_data_storage0111-ib.pocstor_41097_object2601/head//24 by client.2174595.0:154027 0.000000 2014-08-13 18:06:06.878987 7f5c973a57a0 20 read_log 194215'16171 (192834'15449) clean 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by osd.374.0:3997 2014-08-12 15:47:44.622503 2014-08-13 18:06:06.879000 7f5c973a57a0 20 read_log 194215'16172 (194215'16171) delete 7a1142ec/benchmark_data_storage0111-ib.pocstor_41526_object2476/head//24 by client.2174595.0:154029 0.000000 2014-08-13 18:06:06.879012 7f5c973a57a0 20 read_log 194215'16173 (193305'16029) clean fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by osd.374.0:3999 2014-08-12 15:47:44.630005 2014-08-13 18:06:06.879024 7f5c973a57a0 20 read_log 194215'16174 (194215'16173) delete fb742ec/benchmark_data_storage0111-ib.pocstor_62621_object7246/head//24 by client.2174595.0:154031 0.000000 2014-08-13 18:06:06.879038 7f5c973a57a0 20 read_log 194215'16175 (193305'16044) clean afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by osd.374.0:4001 2014-08-12 15:47:44.636915 2014-08-13 18:06:06.879051 7f5c973a57a0 20 read_log 194215'16176 (194215'16175) delete afeb42ec/benchmark_data_storage0111-ib.pocstor_62621_object23631/head//24 by client.2174595.0:154033 0.000000 2014-08-13 18:06:06.879064 7f5c973a57a0 20 read_log 194215'16177 (192888'15742) clean 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by osd.374.0:4003 2014-08-12 15:47:44.644755 2014-08-13 18:06:06.879077 7f5c973a57a0 20 read_log 194215'16178 (194215'16177) delete 4fe052ec/benchmark_data_storage0111-ib.pocstor_41097_object8157/head//24 by client.2174595.0:154035 0.000000 2014-08-13 18:06:06.879089 7f5c973a57a0 20 read_log 194215'16179 (192888'15730) clean 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by osd.374.0:4005 2014-08-12 15:47:44.652045 2014-08-13 18:06:06.879102 7f5c973a57a0 20 read_log 194215'16180 (194215'16179) delete 38352ec/benchmark_data_storage0111-ib.pocstor_41097_object4159/head//24 by client.2174595.0:154037 0.000000 2014-08-13 18:06:06.879115 7f5c973a57a0 20 read_log 194215'16181 (192834'15053) clean 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by osd.374.0:4007 2014-08-12 15:47:44.659616 2014-08-13 18:06:06.879158 7f5c973a57a0 20 read_log 194215'16182 (194215'16181) delete 2c5952ec/benchmark_data_storage0111-ib.pocstor_26659_object18853/head//24 by client.2174595.0:154039 0.000000 2014-08-13 18:06:06.879173 7f5c973a57a0 20 read_log 194215'16183 (192834'15571) clean 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by osd.374.0:4009 2014-08-12 15:47:44.667248 2014-08-13 18:06:06.879186 7f5c973a57a0 20 read_log 194215'16184 (194215'16183) delete 518952ec/benchmark_data_storage0111-ib.pocstor_41526_object21022/head//24 by client.2174595.0:154041 0.000000 2014-08-13 18:06:06.879197 7f5c973a57a0 20 read_log 194215'16185 (192834'15519) clean 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by osd.374.0:4011 2014-08-12 15:47:44.676092 2014-08-13 18:06:06.879209 7f5c973a57a0 20 read_log 194215'16186 (194215'16185) delete 870d62ec/benchmark_data_storage0111-ib.pocstor_41526_object13017/head//24 by client.2174595.0:154043 0.000000 2014-08-13 18:06:06.879220 7f5c973a57a0 20 read_log 194215'16187 (192888'15786) clean dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by osd.374.0:4013 2014-08-12 15:47:44.683494 2014-08-13 18:06:06.879235 7f5c973a57a0 20 read_log 194215'16188 (194215'16187) delete dfdf62ec/benchmark_data_storage0111-ib.pocstor_41097_object30203/head//24 by client.2174595.0:154045 0.000000 2014-08-13 18:06:06.879246 7f5c973a57a0 20 read_log 194215'16189 (193305'16033) clean 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by osd.374.0:4015 2014-08-12 15:47:44.690566 2014-08-13 18:06:06.879263 7f5c973a57a0 20 read_log 194215'16190 (194215'16189) delete 95a572ec/benchmark_data_storage0111-ib.pocstor_62621_object11338/head//24 by client.2174595.0:154047 0.000000 2014-08-13 18:06:06.879275 7f5c973a57a0 20 read_log 194215'16191 (192834'15550) clean 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by osd.374.0:4017 2014-08-12 15:47:44.698085 2014-08-13 18:06:06.879287 7f5c973a57a0 20 read_log 194215'16192 (194215'16191) delete 5dda72ec/benchmark_data_storage0111-ib.pocstor_41526_object17254/head//24 by client.2174595.0:154049 0.000000 2014-08-13 18:06:06.879298 7f5c973a57a0 20 read_log 194215'16193 (192888'15961) clean b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by osd.374.0:4019 2014-08-12 15:47:44.705139 2014-08-13 18:06:06.879310 7f5c973a57a0 20 read_log 194215'16194 (194215'16193) delete b44c72ec/benchmark_data_storage0111-ib.pocstor_2228_object28280/head//24 by client.2174595.0:154051 0.000000 2014-08-13 18:06:06.879321 7f5c973a57a0 20 read_log 194215'16195 (192834'15462) clean 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by osd.374.0:4021 2014-08-12 15:47:44.712973 2014-08-13 18:06:06.879333 7f5c973a57a0 20 read_log 194215'16196 (194215'16195) delete 8c6282ec/benchmark_data_storage0111-ib.pocstor_41526_object4525/head//24 by client.2174595.0:154053 0.000000 2014-08-13 18:06:06.879344 7f5c973a57a0 20 read_log 194215'16197 (192834'15461) clean 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by osd.374.0:4023 2014-08-12 15:47:44.720387 2014-08-13 18:06:06.879356 7f5c973a57a0 20 read_log 194215'16198 (194215'16197) delete 872782ec/benchmark_data_storage0111-ib.pocstor_41526_object4343/head//24 by client.2174595.0:154055 0.000000 2014-08-13 18:06:06.879367 7f5c973a57a0 20 read_log 194215'16199 (193305'16045) clean fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by osd.374.0:4025 2014-08-12 15:47:44.728176 2014-08-13 18:06:06.879379 7f5c973a57a0 20 read_log 194215'16200 (194215'16199) delete fcd292ec/benchmark_data_storage0111-ib.pocstor_62621_object23750/head//24 by client.2174595.0:154057 0.000000 2014-08-13 18:06:06.879399 7f5c973a57a0 20 read_log 194215'16201 (192888'15729) clean e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by osd.374.0:4027 2014-08-12 15:47:44.736506 2014-08-13 18:06:06.879422 7f5c973a57a0 20 read_log 194215'16202 (194215'16201) delete e00392ec/benchmark_data_storage0111-ib.pocstor_41097_object3251/head//24 by client.2174595.0:154059 0.000000 2014-08-13 18:06:06.879449 7f5c973a57a0 20 read_log 194215'16203 (192888'15948) clean 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by osd.374.0:4029 2014-08-12 15:47:44.744133 2014-08-13 18:06:06.879465 7f5c973a57a0 20 read_log 194215'16204 (194215'16203) delete 84e592ec/benchmark_data_storage0111-ib.pocstor_2228_object24641/head//24 by client.2174595.0:154061 0.000000 2014-08-13 18:06:06.879476 7f5c973a57a0 20 read_log 194215'16205 (192834'15035) clean 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by osd.374.0:4031 2014-08-12 15:47:44.751819 2014-08-13 18:06:06.879498 7f5c973a57a0 20 read_log 194215'16206 (194215'16205) delete 796792ec/benchmark_data_storage0111-ib.pocstor_26659_object16490/head//24 by client.2174595.0:154063 0.000000 2014-08-13 18:06:06.879516 7f5c973a57a0 20 read_log 194215'16207 (192888'15949) clean f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by osd.374.0:4033 2014-08-12 15:47:44.760453 2014-08-13 18:06:06.879528 7f5c973a57a0 20 read_log 194215'16208 (194215'16207) delete f7a8a2ec/benchmark_data_storage0111-ib.pocstor_2228_object24862/head//24 by client.2174595.0:154065 0.000000 2014-08-13 18:06:06.879541 7f5c973a57a0 20 read_log 194215'16209 (192834'15452) clean 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by osd.374.0:4035 2014-08-12 15:47:44.768942 2014-08-13 18:06:06.879554 7f5c973a57a0 20 read_log 194215'16210 (194215'16209) delete 7907b2ec/benchmark_data_storage0111-ib.pocstor_41526_object3067/head//24 by client.2174595.0:154067 0.000000 2014-08-13 18:06:06.879565 7f5c973a57a0 20 read_log 194215'16211 (193305'16038) clean accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by osd.374.0:4037 2014-08-12 15:47:44.777786 2014-08-13 18:06:06.879577 7f5c973a57a0 20 read_log 194215'16212 (194215'16211) delete accb2ec/benchmark_data_storage0111-ib.pocstor_62621_object18296/head//24 by client.2174595.0:154069 0.000000 2014-08-13 18:06:06.879588 7f5c973a57a0 20 read_log 194215'16213 (192834'15488) clean d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by osd.374.0:4039 2014-08-12 15:47:44.786412 2014-08-13 18:06:06.879600 7f5c973a57a0 20 read_log 194215'16214 (194215'16213) delete d571c2ec/benchmark_data_storage0111-ib.pocstor_41526_object8668/head//24 by client.2174595.0:154071 0.000000 2014-08-13 18:06:06.879623 7f5c973a57a0 20 read_log 194215'16215 (192834'14946) clean dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by osd.374.0:4041 2014-08-12 15:47:44.794135 2014-08-13 18:06:06.879643 7f5c973a57a0 20 read_log 194215'16216 (194215'16215) delete dbd3c2ec/benchmark_data_storage0111-ib.pocstor_26659_object6678/head//24 by client.2174595.0:154073 0.000000 2014-08-13 18:06:06.879658 7f5c973a57a0 20 read_log 194215'16217 (192888'15743) clean 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by osd.374.0:4043 2014-08-12 15:47:44.801826 2014-08-13 18:06:06.879671 7f5c973a57a0 20 read_log 194215'16218 (194215'16217) delete 5175c2ec/benchmark_data_storage0111-ib.pocstor_41097_object9376/head//24 by client.2174595.0:154075 0.000000 2014-08-13 18:06:06.879682 7f5c973a57a0 20 read_log 194215'16219 (192888'15750) clean 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by osd.374.0:4045 2014-08-12 15:47:44.809112 2014-08-13 18:06:06.879695 7f5c973a57a0 20 read_log 194215'16220 (194215'16219) delete 3257c2ec/benchmark_data_storage0111-ib.pocstor_41097_object12378/head//24 by client.2174595.0:154077 0.000000 2014-08-13 18:06:06.879706 7f5c973a57a0 20 read_log 194215'16221 (193305'16047) clean 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by osd.374.0:4047 2014-08-12 15:47:44.817026 2014-08-13 18:06:06.879744 7f5c973a57a0 20 read_log 194215'16222 (194215'16221) delete 6eecc2ec/benchmark_data_storage0111-ib.pocstor_62621_object24173/head//24 by client.2174595.0:154079 0.000000 2014-08-13 18:06:06.879759 7f5c973a57a0 20 read_log 194215'16223 (192834'15071) clean 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by osd.374.0:4049 2014-08-12 15:47:44.823982 2014-08-13 18:06:06.879771 7f5c973a57a0 20 read_log 194215'16224 (194215'16223) delete 6670d2ec/benchmark_data_storage0111-ib.pocstor_26659_object20277/head//24 by client.2174595.0:154081 0.000000 2014-08-13 18:06:06.879782 7f5c973a57a0 20 read_log 194215'16225 (192834'15554) clean 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by osd.374.0:4051 2014-08-12 15:47:44.832340 2014-08-13 18:06:06.879795 7f5c973a57a0 20 read_log 194215'16226 (194215'16225) delete 2744d2ec/benchmark_data_storage0111-ib.pocstor_41526_object18735/head//24 by client.2174595.0:154083 0.000000 2014-08-13 18:06:06.879805 7f5c973a57a0 20 read_log 194215'16227 (192888'15747) clean 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by osd.374.0:4053 2014-08-12 15:47:44.839497 2014-08-13 18:06:06.879817 7f5c973a57a0 20 read_log 194215'16228 (194215'16227) delete 1d3cd2ec/benchmark_data_storage0111-ib.pocstor_41097_object11734/head//24 by client.2174595.0:154085 0.000000 2014-08-13 18:06:06.879836 7f5c973a57a0 20 read_log 194215'16229 (194209'16059) delete 6e7936ec/benchmark_data_storage0111-ib.pocstor_41526_object22665/head//24 by client.2174595.0:154087 0.000000 2014-08-13 18:06:06.879849 7f5c973a57a0 20 read_log 194215'16230 (194209'16061) delete 1bec46ec/benchmark_data_storage0111-ib.pocstor_26659_object21929/head//24 by client.2174595.0:154089 0.000000 2014-08-13 18:06:06.879860 7f5c973a57a0 20 read_log 194215'16231 (194209'16062) delete 3a9066ec/benchmark_data_storage0111-ib.pocstor_2228_object13884/head//24 by client.2174595.0:154091 0.000000 2014-08-13 18:06:06.879871 7f5c973a57a0 20 read_log 194215'16232 (192888'15922) clean 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by osd.374.0:4058 2014-08-12 15:47:44.861845 2014-08-13 18:06:06.879884 7f5c973a57a0 20 read_log 194215'16233 (194215'16232) delete 52ebbaec/benchmark_data_storage0111-ib.pocstor_2228_object8824/head//24 by client.2174595.0:154093 0.000000 2014-08-13 18:06:06.879895 7f5c973a57a0 20 read_log 194215'16234 (193305'16046) clean bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by osd.374.0:4060 2014-08-12 15:47:44.869274 2014-08-13 18:06:06.879909 7f5c973a57a0 20 read_log 194215'16235 (194215'16234) delete bb6fbaec/benchmark_data_storage0111-ib.pocstor_62621_object23970/head//24 by client.2174595.0:154095 0.000000 2014-08-13 18:06:06.879920 7f5c973a57a0 20 read_log 194215'16236 (192888'15778) clean 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by osd.374.0:4062 2014-08-12 15:47:44.876060 2014-08-13 18:06:06.879933 7f5c973a57a0 20 read_log 194215'16237 (194215'16236) delete 7c85caec/benchmark_data_storage0111-ib.pocstor_41097_object25705/head//24 by client.2174595.0:154097 0.000000 2014-08-13 18:06:06.879944 7f5c973a57a0 20 read_log 194215'16238 (192888'15920) clean 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by osd.374.0:4064 2014-08-12 15:47:44.883846 2014-08-13 18:06:06.879956 7f5c973a57a0 20 read_log 194215'16239 (194215'16238) delete 9aebcaec/benchmark_data_storage0111-ib.pocstor_2228_object7493/head//24 by client.2174595.0:154099 0.000000 2014-08-13 18:06:06.879967 7f5c973a57a0 20 read_log 194215'16240 (192888'15915) clean 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by osd.374.0:4066 2014-08-12 15:47:44.892082 2014-08-13 18:06:06.879980 7f5c973a57a0 20 read_log 194215'16241 (194215'16240) delete 2baccaec/benchmark_data_storage0111-ib.pocstor_2228_object4835/head//24 by client.2174595.0:154101 0.000000 2014-08-13 18:06:06.879998 7f5c973a57a0 20 read_log 194215'16242 (193305'16032) clean 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by osd.374.0:4068 2014-08-12 15:47:44.900084 2014-08-13 18:06:06.880013 7f5c973a57a0 20 read_log 194215'16243 (194215'16242) delete 81edcaec/benchmark_data_storage0111-ib.pocstor_62621_object10570/head//24 by client.2174595.0:154103 0.000000 2014-08-13 18:06:06.880024 7f5c973a57a0 20 read_log 194215'16244 (192888'15787) clean 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by osd.374.0:4070 2014-08-12 15:47:44.907409 2014-08-13 18:06:06.880044 7f5c973a57a0 20 read_log 194215'16245 (194215'16244) delete 8400eaec/benchmark_data_storage0111-ib.pocstor_41097_object30375/head//24 by client.2174595.0:154105 0.000000 2014-08-13 18:06:06.880056 7f5c973a57a0 20 read_log 194215'16246 (192888'15737) clean 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by osd.374.0:4072 2014-08-12 15:47:44.915670 2014-08-13 18:06:06.880078 7f5c973a57a0 20 read_log 194215'16247 (194215'16246) delete 2063eaec/benchmark_data_storage0111-ib.pocstor_41097_object6160/head//24 by client.2174595.0:154107 0.000000 2014-08-13 18:06:06.880096 7f5c973a57a0 20 read_log 194215'16248 (192888'15771) clean b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by osd.374.0:4074 2014-08-12 15:47:44.923582 2014-08-13 18:06:06.880109 7f5c973a57a0 20 read_log 194215'16249 (194215'16248) delete b88ceaec/benchmark_data_storage0111-ib.pocstor_41097_object21568/head//24 by client.2174595.0:154109 0.000000 2014-08-13 18:06:06.880125 7f5c973a57a0 20 read_log 194215'16250 (192834'15447) clean bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by osd.374.0:4076 2014-08-12 15:47:44.931369 2014-08-13 18:06:06.880155 7f5c973a57a0 20 read_log 194215'16251 (194215'16250) delete bd0cfaec/benchmark_data_storage0111-ib.pocstor_41526_object2472/head//24 by client.2174595.0:154111 0.000000 2014-08-13 18:06:06.880173 7f5c973a57a0 20 read_log 194215'16252 (193305'16034) clean aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by osd.374.0:4078 2014-08-12 15:47:44.938726 2014-08-13 18:06:06.880186 7f5c973a57a0 20 read_log 194215'16253 (194215'16252) delete aa2ffaec/benchmark_data_storage0111-ib.pocstor_62621_object11478/head//24 by client.2174595.0:154113 0.000000 2014-08-13 18:06:06.880209 7f5c973a57a0 20 read_log 194215'16254 (192888'15765) clean ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by osd.374.0:4080 2014-08-12 15:47:44.946521 2014-08-13 18:06:06.880222 7f5c973a57a0 20 read_log 194215'16255 (194215'16254) delete ece20eec/benchmark_data_storage0111-ib.pocstor_41097_object18339/head//24 by client.2174595.0:154115 0.000000 2014-08-13 18:06:06.880233 7f5c973a57a0 20 read_log 194215'16256 (193305'16041) clean f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by osd.374.0:4082 2014-08-12 15:47:44.954000 2014-08-13 18:06:06.880251 7f5c973a57a0 20 read_log 194215'16257 (194215'16256) delete f6080eec/benchmark_data_storage0111-ib.pocstor_62621_object19977/head//24 by client.2174595.0:154117 0.000000 2014-08-13 18:06:06.880262 7f5c973a57a0 20 read_log 194215'16258 (192834'14968) clean b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by osd.374.0:4084 2014-08-12 15:47:44.961953 2014-08-13 18:06:06.880275 7f5c973a57a0 20 read_log 194215'16259 (194215'16258) delete b2f11eec/benchmark_data_storage0111-ib.pocstor_26659_object8566/head//24 by client.2174595.0:154119 0.000000 2014-08-13 18:06:06.880286 7f5c973a57a0 20 read_log 194215'16260 (192888'15759) clean 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by osd.374.0:4086 2014-08-12 15:47:44.969198 2014-08-13 18:06:06.880298 7f5c973a57a0 20 read_log 194215'16261 (194215'16260) delete 67841eec/benchmark_data_storage0111-ib.pocstor_41097_object16025/head//24 by client.2174595.0:154121 0.000000 2014-08-13 18:06:06.880316 7f5c973a57a0 20 read_log 194215'16262 (192834'14902) clean 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by osd.374.0:4088 2014-08-12 15:47:44.977289 2014-08-13 18:06:06.880331 7f5c973a57a0 20 read_log 194215'16263 (194215'16262) delete 4ec61eec/benchmark_data_storage0111-ib.pocstor_26659_object1883/head//24 by client.2174595.0:154123 0.000000 2014-08-13 18:06:06.880342 7f5c973a57a0 20 read_log 194215'16264 (192888'15919) clean ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by osd.374.0:4090 2014-08-12 15:47:44.984600 2014-08-13 18:06:06.880355 7f5c973a57a0 20 read_log 194215'16265 (194215'16264) delete ba42eec/benchmark_data_storage0111-ib.pocstor_2228_object7208/head//24 by client.2174595.0:154125 0.000000 2014-08-13 18:06:06.880373 7f5c973a57a0 20 read_log 194215'16266 (193305'16028) clean e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by osd.374.0:4092 2014-08-12 15:47:44.991663 2014-08-13 18:06:06.880388 7f5c973a57a0 20 read_log 194215'16267 (194215'16266) delete e75e2eec/benchmark_data_storage0111-ib.pocstor_62621_object6909/head//24 by client.2174595.0:154127 0.000000 2014-08-13 18:06:06.880399 7f5c973a57a0 20 read_log 194215'16268 (193305'16031) clean ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by osd.374.0:4094 2014-08-12 15:47:45.000050 2014-08-13 18:06:06.880412 7f5c973a57a0 20 read_log 194215'16269 (194215'16268) delete ccef2eec/benchmark_data_storage0111-ib.pocstor_62621_object9892/head//24 by client.2174595.0:154129 0.000000 2014-08-13 18:06:06.880423 7f5c973a57a0 20 read_log 194215'16270 (192888'15768) clean 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by osd.374.0:4096 2014-08-12 15:47:45.007162 2014-08-13 18:06:06.880436 7f5c973a57a0 20 read_log 194215'16271 (194215'16270) delete 97803eec/benchmark_data_storage0111-ib.pocstor_41097_object18759/head//24 by client.2174595.0:154131 0.000000 2014-08-13 18:06:06.880447 7f5c973a57a0 20 read_log 194215'16272 (193305'16039) clean c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by osd.374.0:4098 2014-08-12 15:47:45.014396 2014-08-13 18:06:06.880459 7f5c973a57a0 20 read_log 194215'16273 (194215'16272) delete c4373eec/benchmark_data_storage0111-ib.pocstor_62621_object18972/head//24 by client.2174595.0:154133 0.000000 2014-08-13 18:06:06.880470 7f5c973a57a0 20 read_log 194215'16274 (192834'15490) clean 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by osd.374.0:4100 2014-08-12 15:47:45.021998 2014-08-13 18:06:06.880484 7f5c973a57a0 20 read_log 194215'16275 (194215'16274) delete 94e34eec/benchmark_data_storage0111-ib.pocstor_41526_object8843/head//24 by client.2174595.0:154135 0.000000 2014-08-13 18:06:06.880495 7f5c973a57a0 20 read_log 194215'16276 (192888'15751) clean 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by osd.374.0:4102 2014-08-12 15:47:45.030493 2014-08-13 18:06:06.880507 7f5c973a57a0 20 read_log 194215'16277 (194215'16276) delete 66e54eec/benchmark_data_storage0111-ib.pocstor_41097_object13047/head//24 by client.2174595.0:154137 0.000000 2014-08-13 18:06:06.880519 7f5c973a57a0 20 read_log 194215'16278 (192834'15501) clean a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by osd.374.0:4104 2014-08-12 15:47:45.038941 2014-08-13 18:06:06.880531 7f5c973a57a0 20 read_log 194215'16279 (194215'16278) delete a3e64eec/benchmark_data_storage0111-ib.pocstor_41526_object10084/head//24 by client.2174595.0:154139 0.000000 2014-08-13 18:06:06.880542 7f5c973a57a0 20 read_log 194215'16280 (192888'15956) clean 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by osd.374.0:4106 2014-08-12 15:47:45.046303 2014-08-13 18:06:06.880554 7f5c973a57a0 20 read_log 194215'16281 (194215'16280) delete 831f4eec/benchmark_data_storage0111-ib.pocstor_2228_object26910/head//24 by client.2174595.0:154141 0.000000 2014-08-13 18:06:06.880574 7f5c973a57a0 20 read_log 194215'16282 (193305'16042) clean 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by osd.374.0:4108 2014-08-12 15:47:45.054280 2014-08-13 18:06:06.880587 7f5c973a57a0 20 read_log 194215'16283 (194215'16282) delete 7f015eec/benchmark_data_storage0111-ib.pocstor_62621_object21187/head//24 by client.2174595.0:154143 0.000000 2014-08-13 18:06:06.880598 7f5c973a57a0 20 read_log 194215'16284 (193305'16027) clean bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by osd.374.0:4110 2014-08-12 15:47:45.061839 2014-08-13 18:06:06.880611 7f5c973a57a0 20 read_log 194215'16285 (194215'16284) delete bd4c5eec/benchmark_data_storage0111-ib.pocstor_62621_object2867/head//24 by client.2174595.0:154145 0.000000 2014-08-13 18:06:06.880621 7f5c973a57a0 20 read_log 194215'16286 (192888'15725) clean 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by osd.374.0:4112 2014-08-12 15:47:45.068938 2014-08-13 18:06:06.880634 7f5c973a57a0 20 read_log 194215'16287 (194215'16286) delete 985f5eec/benchmark_data_storage0111-ib.pocstor_41097_object1688/head//24 by client.2174595.0:154147 0.000000 2014-08-13 18:06:06.880645 7f5c973a57a0 20 read_log 194215'16288 (192888'15921) clean 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by osd.374.0:4114 2014-08-12 15:47:45.077721 2014-08-13 18:06:06.880657 7f5c973a57a0 20 read_log 194215'16289 (194215'16288) delete 26936eec/benchmark_data_storage0111-ib.pocstor_2228_object8098/head//24 by client.2174595.0:154149 0.000000 2014-08-13 18:06:06.880668 7f5c973a57a0 20 read_log 194215'16290 (192888'15939) clean 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by osd.374.0:4116 2014-08-12 15:47:45.085365 2014-08-13 18:06:06.880684 7f5c973a57a0 20 read_log 194215'16291 (194215'16290) delete 4e397eec/benchmark_data_storage0111-ib.pocstor_2228_object14799/head//24 by client.2174595.0:154151 0.000000 2014-08-13 18:06:06.880695 7f5c973a57a0 20 read_log 194215'16292 (192834'15541) clean a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by osd.374.0:4118 2014-08-12 15:47:45.092808 2014-08-13 18:06:06.880707 7f5c973a57a0 20 read_log 194215'16293 (194215'16292) delete a84e7eec/benchmark_data_storage0111-ib.pocstor_41526_object15965/head//24 by client.2174595.0:154153 0.000000 2014-08-13 18:06:06.880719 7f5c973a57a0 20 read_log 194215'16294 (192834'14895) clean 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by osd.374.0:4120 2014-08-12 15:47:45.106304 2014-08-13 18:06:06.880733 7f5c973a57a0 20 read_log 194215'16295 (194215'16294) delete 485e7eec/benchmark_data_storage0111-ib.pocstor_26659_object648/head//24 by client.2174595.0:154155 0.000000 2014-08-13 18:06:06.880744 7f5c973a57a0 20 read_log 194215'16296 (192834'15085) clean 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by osd.374.0:4122 2014-08-12 15:47:45.114558 2014-08-13 18:06:06.880757 7f5c973a57a0 20 read_log 194215'16297 (194215'16296) delete 5f588eec/benchmark_data_storage0111-ib.pocstor_26659_object21518/head//24 by client.2174595.0:154157 0.000000 2014-08-13 18:06:06.880768 7f5c973a57a0 20 read_log 194215'16298 (193305'16051) clean ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by osd.374.0:4124 2014-08-12 15:47:45.122223 2014-08-13 18:06:06.880782 7f5c973a57a0 20 read_log 194215'16299 (194215'16298) delete ae598eec/benchmark_data_storage0111-ib.pocstor_62621_object30369/head//24 by client.2174595.0:154159 0.000000 2014-08-13 18:06:06.880793 7f5c973a57a0 20 read_log 194215'16300 (193305'16024) clean 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by osd.374.0:4126 2014-08-12 15:47:45.129485 2014-08-13 18:06:06.880806 7f5c973a57a0 20 read_log 194215'16301 (194215'16300) delete 47599eec/benchmark_data_storage0111-ib.pocstor_62621_object736/head//24 by client.2174595.0:154161 0.000000 2014-08-13 18:06:06.880823 7f5c973a57a0 20 read_log 194215'16302 (192834'15033) clean 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by osd.374.0:4128 2014-08-12 15:47:45.137364 2014-08-13 18:06:06.880838 7f5c973a57a0 20 read_log 194215'16303 (194215'16302) delete 6cf4aeec/benchmark_data_storage0111-ib.pocstor_26659_object16170/head//24 by client.2174595.0:154163 0.000000 2014-08-13 18:06:06.880849 7f5c973a57a0 20 read_log 194215'16304 (193305'16037) clean 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by osd.374.0:4130 2014-08-12 15:47:45.144840 2014-08-13 18:06:06.880861 7f5c973a57a0 20 read_log 194215'16305 (194215'16304) delete 5de8aeec/benchmark_data_storage0111-ib.pocstor_62621_object17745/head//24 by client.2174595.0:154165 0.000000 2014-08-13 18:06:06.880872 7f5c973a57a0 20 read_log 194215'16306 (192834'15556) clean 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by osd.374.0:4132 2014-08-12 15:47:45.153037 2014-08-13 18:06:06.880884 7f5c973a57a0 20 read_log 194215'16307 (194215'16306) delete 9070beec/benchmark_data_storage0111-ib.pocstor_41526_object19031/head//24 by client.2174595.0:154167 0.000000 2014-08-13 18:06:06.880895 7f5c973a57a0 20 read_log 194215'16308 (192888'15732) clean 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by osd.374.0:4134 2014-08-12 15:47:45.161904 2014-08-13 18:06:06.880908 7f5c973a57a0 20 read_log 194215'16309 (194215'16308) delete 5001beec/benchmark_data_storage0111-ib.pocstor_41097_object4383/head//24 by client.2174595.0:154169 0.000000 2014-08-13 18:06:06.880919 7f5c973a57a0 20 read_log 194215'16310 (192888'15761) clean b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by osd.374.0:4136 2014-08-12 15:47:45.170476 2014-08-13 18:06:06.880933 7f5c973a57a0 20 read_log 194215'16311 (194215'16310) delete b59beec/benchmark_data_storage0111-ib.pocstor_41097_object16720/head//24 by client.2174595.0:154171 0.000000 2014-08-13 18:06:06.880944 7f5c973a57a0 20 read_log 194215'16312 (192888'15910) clean a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by osd.374.0:4138 2014-08-12 15:47:45.178545 2014-08-13 18:06:06.880959 7f5c973a57a0 20 read_log 194215'16313 (194215'16312) delete a7ebeec/benchmark_data_storage0111-ib.pocstor_2228_object3099/head//24 by client.2174595.0:154173 0.000000 2014-08-13 18:06:06.880970 7f5c973a57a0 20 read_log 194215'16314 (192834'15573) clean 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by osd.374.0:4140 2014-08-12 15:47:45.185904 2014-08-13 18:06:06.881003 7f5c973a57a0 20 read_log 194215'16315 (194215'16314) delete 778ebeec/benchmark_data_storage0111-ib.pocstor_41526_object21091/head//24 by client.2174595.0:154175 0.000000 2014-08-13 18:06:06.881016 7f5c973a57a0 20 read_log 194215'16316 (192888'15928) clean f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by osd.374.0:4142 2014-08-12 15:47:45.194138 2014-08-13 18:06:06.881029 7f5c973a57a0 20 read_log 194215'16317 (194215'16316) delete f9f0ceec/benchmark_data_storage0111-ib.pocstor_2228_object10194/head//24 by client.2174595.0:154177 0.000000 2014-08-13 18:06:06.881040 7f5c973a57a0 20 read_log 194215'16318 (192834'14898) clean fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by osd.374.0:4144 2014-08-12 15:47:45.201567 2014-08-13 18:06:06.881055 7f5c973a57a0 20 read_log 194215'16319 (194215'16318) delete fe21ceec/benchmark_data_storage0111-ib.pocstor_26659_object1474/head//24 by client.2174595.0:154179 0.000000 2014-08-13 18:06:06.881066 7f5c973a57a0 20 read_log 194215'16320 (192834'14999) clean 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by osd.374.0:4146 2014-08-12 15:47:45.209378 2014-08-13 18:06:06.881078 7f5c973a57a0 20 read_log 194215'16321 (194215'16320) delete 4ea5ceec/benchmark_data_storage0111-ib.pocstor_26659_object12161/head//24 by client.2174595.0:154181 0.000000 2014-08-13 18:06:06.881112 7f5c973a57a0 20 read_log 194215'16322 (192834'15055) clean 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by osd.374.0:4148 2014-08-12 15:47:45.216765 2014-08-13 18:06:06.881127 7f5c973a57a0 20 read_log 194215'16323 (194215'16322) delete 6c5fceec/benchmark_data_storage0111-ib.pocstor_26659_object19064/head//24 by client.2174595.0:154183 0.000000 2014-08-13 18:06:06.881138 7f5c973a57a0 20 read_log 194215'16324 (192888'15912) clean ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by osd.374.0:4150 2014-08-12 15:47:45.225554 2014-08-13 18:06:06.881156 7f5c973a57a0 20 read_log 194215'16325 (194215'16324) delete ec32deec/benchmark_data_storage0111-ib.pocstor_2228_object3832/head//24 by client.2174595.0:154185 0.000000 2014-08-13 18:06:06.881167 7f5c973a57a0 20 read_log 194215'16326 (192888'15758) clean 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by osd.374.0:4152 2014-08-12 15:47:45.234244 2014-08-13 18:06:06.881180 7f5c973a57a0 20 read_log 194215'16327 (194215'16326) delete 1f74deec/benchmark_data_storage0111-ib.pocstor_41097_object15818/head//24 by client.2174595.0:154187 0.000000 2014-08-13 18:06:06.881193 7f5c973a57a0 20 read_log 194215'16328 (192834'15454) clean c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by osd.374.0:4154 2014-08-12 15:47:45.242362 2014-08-13 18:06:06.881205 7f5c973a57a0 20 read_log 194215'16329 (194215'16328) delete c9fdeec/benchmark_data_storage0111-ib.pocstor_41526_object3203/head//24 by client.2174595.0:154189 0.000000 2014-08-13 18:06:06.881217 7f5c973a57a0 20 read_log 194215'16330 (192834'14913) clean 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by osd.374.0:4156 2014-08-12 15:47:45.250111 2014-08-13 18:06:06.881236 7f5c973a57a0 20 read_log 194215'16331 (194215'16330) delete 40f0eeec/benchmark_data_storage0111-ib.pocstor_26659_object2870/head//24 by client.2174595.0:154191 0.000000 2014-08-13 18:06:06.881248 7f5c973a57a0 20 read_log 194215'16332 (192888'15947) clean 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by osd.374.0:4158 2014-08-12 15:47:45.257553 2014-08-13 18:06:06.881260 7f5c973a57a0 20 read_log 194215'16333 (194215'16332) delete 1c83eeec/benchmark_data_storage0111-ib.pocstor_2228_object24337/head//24 by client.2174595.0:154193 0.000000 2014-08-13 18:06:06.881273 7f5c973a57a0 20 read_log 194215'16334 (192834'14939) clean ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by osd.374.0:4160 2014-08-12 15:47:45.265618 2014-08-13 18:06:06.881286 7f5c973a57a0 20 read_log 194215'16335 (194215'16334) delete ee04eeec/benchmark_data_storage0111-ib.pocstor_26659_object6009/head//24 by client.2174595.0:154195 0.000000 2014-08-13 18:06:06.881298 7f5c973a57a0 20 read_log 194215'16336 (192888'15784) clean d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by osd.374.0:4162 2014-08-12 15:47:45.273186 2014-08-13 18:06:06.881315 7f5c973a57a0 20 read_log 194215'16337 (194215'16336) delete d891feec/benchmark_data_storage0111-ib.pocstor_41097_object29312/head//24 by client.2174595.0:154197 0.000000 2014-08-13 18:06:06.881328 7f5c973a57a0 20 read_log 194215'16338 (192888'15914) clean 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by osd.374.0:4164 2014-08-12 15:47:45.281601 2014-08-13 18:06:06.881341 7f5c973a57a0 20 read_log 194215'16339 (194215'16338) delete 233feec/benchmark_data_storage0111-ib.pocstor_2228_object4671/head//24 by client.2174595.0:154199 0.000000 2014-08-13 18:06:06.881352 7f5c973a57a0 20 read_log 194215'16340 (192834'15505) clean a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by osd.374.0:4166 2014-08-12 15:47:45.289962 2014-08-13 18:06:06.881369 7f5c973a57a0 20 read_log 194215'16341 (194215'16340) delete a20bfeec/benchmark_data_storage0111-ib.pocstor_41526_object10873/head//24 by client.2174595.0:154201 0.000000 2014-08-13 18:06:06.881391 7f5c973a57a0 20 read_log 194370'16342 (0'0) promote 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by osd.374.0:4900 2014-08-12 20:30:24.527969 2014-08-13 18:06:06.881406 7f5c973a57a0 20 read_log 194370'16343 (194370'16342) modify 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by client.1508531.0:2720 2014-08-12 20:30:24.520471 2014-08-13 18:06:06.881424 7f5c973a57a0 20 read_log 194370'16344 (0'0) promote 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by osd.374.0:4916 2014-08-12 20:30:26.904986 2014-08-13 18:06:06.881436 7f5c973a57a0 20 read_log 194370'16345 (194370'16344) modify 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by client.1508531.0:3371 2014-08-12 20:30:26.898087 2014-08-13 18:06:06.881462 7f5c973a57a0 20 read_log 194370'16346 (0'0) promote 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by osd.374.0:4940 2014-08-12 20:30:30.854778 2014-08-13 18:06:06.881478 7f5c973a57a0 20 read_log 194370'16347 (194370'16346) modify 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by client.1508531.0:4410 2014-08-12 20:30:30.848680 2014-08-13 18:06:06.881497 7f5c973a57a0 20 read_log 194370'16348 (0'0) promote 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by osd.374.0:4982 2014-08-12 20:30:38.552347 2014-08-13 18:06:06.881513 7f5c973a57a0 20 read_log 194370'16349 (194370'16348) modify 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by client.1508531.0:6178 2014-08-12 20:30:38.545133 2014-08-13 18:06:06.881526 7f5c973a57a0 20 read_log 194370'16350 (0'0) promote 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by osd.374.0:5010 2014-08-12 20:30:46.624293 2014-08-13 18:06:06.881547 7f5c973a57a0 20 read_log 194370'16351 (194370'16350) modify 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by client.1508531.0:7818 2014-08-12 20:30:46.615726 2014-08-13 18:06:06.881563 7f5c973a57a0 20 read_log 194370'16352 (0'0) promote 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by osd.374.0:5078 2014-08-12 20:31:07.839294 2014-08-13 18:06:06.881576 7f5c973a57a0 20 read_log 194370'16353 (194370'16352) modify 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by client.1508531.0:11419 2014-08-12 20:31:07.830173 2014-08-13 18:06:06.881596 7f5c973a57a0 20 read_log 194370'16354 (0'0) promote 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by osd.374.0:5106 2014-08-12 20:31:14.279038 2014-08-13 18:06:06.881612 7f5c973a57a0 20 read_log 194370'16355 (194370'16354) modify 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by client.1508531.0:12400 2014-08-12 20:31:14.270603 2014-08-13 18:06:06.881625 7f5c973a57a0 20 read_log 194370'16356 (0'0) promote 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by osd.374.0:5136 2014-08-12 20:31:25.690707 2014-08-13 18:06:06.881641 7f5c973a57a0 20 read_log 194370'16357 (194370'16356) modify 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by client.1508531.0:13662 2014-08-12 20:31:25.680926 2014-08-13 18:06:06.881656 7f5c973a57a0 20 read_log 194377'16358 (0'0) promote 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by osd.374.0:5172 2014-08-12 20:33:02.110149 2014-08-13 18:06:06.881669 7f5c973a57a0 20 read_log 194377'16359 (194377'16358) modify 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by client.2231190.0:1211 2014-08-12 20:33:02.102879 2014-08-13 18:06:06.881681 7f5c973a57a0 20 read_log 194377'16360 (0'0) promote 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by osd.374.0:5218 2014-08-12 20:33:07.613644 2014-08-13 18:06:06.881693 7f5c973a57a0 20 read_log 194377'16361 (194377'16360) modify 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by client.2231190.0:2577 2014-08-12 20:33:07.605310 2014-08-13 18:06:06.881714 7f5c973a57a0 20 read_log 194377'16362 (0'0) promote 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by osd.374.0:5236 2014-08-12 20:33:09.593729 2014-08-13 18:06:06.881729 7f5c973a57a0 20 read_log 194377'16363 (194377'16362) modify 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by client.2231190.0:3068 2014-08-12 20:33:09.586317 2014-08-13 18:06:06.881742 7f5c973a57a0 20 read_log 194377'16364 (0'0) promote 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by osd.374.0:5244 2014-08-12 20:33:11.004822 2014-08-13 18:06:06.881754 7f5c973a57a0 20 read_log 194377'16365 (194377'16364) modify 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by client.2231190.0:3430 2014-08-12 20:33:10.995220 2014-08-13 18:06:06.881767 7f5c973a57a0 20 read_log 194380'16366 (0'0) promote c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by osd.374.0:5284 2014-08-12 20:33:26.188953 2014-08-13 18:06:06.881779 7f5c973a57a0 20 read_log 194380'16367 (194380'16366) modify c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by client.2231190.0:5229 2014-08-12 20:33:26.179871 2014-08-13 18:06:06.881791 7f5c973a57a0 20 read_log 194383'16368 (0'0) promote 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by osd.374.0:5298 2014-08-12 20:34:00.306372 2014-08-13 18:06:06.881803 7f5c973a57a0 20 read_log 194383'16369 (194383'16368) modify 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by client.2231190.0:5795 2014-08-12 20:34:00.296784 2014-08-13 18:06:06.881815 7f5c973a57a0 20 read_log 194383'16370 (0'0) promote 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by osd.374.0:5336 2014-08-12 20:34:26.543149 2014-08-13 18:06:06.881828 7f5c973a57a0 20 read_log 194383'16371 (194383'16370) modify 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by client.2231190.0:7419 2014-08-12 20:34:26.535065 2014-08-13 18:06:06.881840 7f5c973a57a0 20 read_log 194393'16372 (194393'16372) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:37:09.049317_2014-08-12 20:47:26.077872/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.078448 2014-08-13 18:06:06.881853 7f5c973a57a0 20 read_log 194393'16373 (194215'16153) delete 2ec/hit_set_24.2ec_archive_2014-08-12 15:32:26.012127_2014-08-12 15:47:44.557466/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:26.078448 2014-08-13 18:06:06.881865 7f5c973a57a0 20 read_log 194393'16374 (0'0) promote fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by osd.374.0:5343 2014-08-12 20:47:26.084436 2014-08-13 18:06:06.881878 7f5c973a57a0 20 read_log 194393'16375 (194393'16374) modify fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by client.1064095.0:38 2014-08-12 20:47:26.071882 2014-08-13 18:06:06.881890 7f5c973a57a0 20 read_log 194393'16376 (0'0) promote a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by osd.374.0:5372 2014-08-12 20:49:26.301315 2014-08-13 18:06:06.881903 7f5c973a57a0 20 read_log 194393'16377 (194393'16376) modify a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by client.1064182.0:174 2014-08-12 20:49:26.293716 2014-08-13 18:06:06.881915 7f5c973a57a0 20 read_log 194393'16378 (0'0) promote 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by osd.374.0:5378 2014-08-12 20:49:27.260261 2014-08-13 18:06:06.881927 7f5c973a57a0 20 read_log 194393'16379 (194393'16378) modify 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by client.1064182.0:381 2014-08-12 20:49:27.253733 2014-08-13 18:06:06.881940 7f5c973a57a0 20 read_log 194396'16380 (194396'16380) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:47:26.078497_2014-08-12 20:53:00.018899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.019463 2014-08-13 18:06:06.881960 7f5c973a57a0 20 read_log 194396'16381 (194393'16372) delete 2ec/hit_set_24.2ec_archive_2014-08-12 20:37:09.049317_2014-08-12 20:47:26.077872/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:00.019463 2014-08-13 18:06:06.881975 7f5c973a57a0 20 read_log 194396'16382 (0'0) promote dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by osd.374.0:5391 2014-08-12 20:53:00.025127 2014-08-13 18:06:06.881987 7f5c973a57a0 20 read_log 194396'16383 (194396'16382) modify dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by client.2235069.0:279 2014-08-12 20:53:00.011801 2014-08-13 18:06:06.882002 7f5c973a57a0 20 read_log 194416'16384 (194416'16384) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:53:00.019540_2014-08-13 15:28:37.194823/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 18:06:06.882015 7f5c973a57a0 20 read_log 194416'16385 (194396'16380) delete 2ec/hit_set_24.2ec_archive_2014-08-12 20:47:26.078497_2014-08-12 20:53:00.018899/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 18:06:06.882028 7f5c973a57a0 20 read_log 194416'16386 (0'0) promote efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by osd.374.0:5597 2014-08-13 15:28:37.198503 2014-08-13 18:06:06.882042 7f5c973a57a0 20 read_log 194416'16387 (194416'16386) modify efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by client.2441958.0:1710 2014-08-13 15:28:37.189859 2014-08-13 18:06:06.882055 7f5c973a57a0 20 read_log 194416'16388 (0'0) promote 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by osd.374.0:5623 2014-08-13 15:28:41.156396 2014-08-13 18:06:06.882069 7f5c973a57a0 20 read_log 194416'16389 (194416'16388) modify 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by client.2441958.0:2754 2014-08-13 15:28:41.147516 2014-08-13 18:06:06.882082 7f5c973a57a0 20 read_log 194416'16390 (0'0) promote c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by osd.374.0:5637 2014-08-13 15:28:43.094680 2014-08-13 18:06:06.882096 7f5c973a57a0 20 read_log 194416'16391 (194416'16390) modify c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by client.2441958.0:3268 2014-08-13 15:28:43.084947 2014-08-13 18:06:06.882109 7f5c973a57a0 20 read_log 194416'16392 (0'0) promote 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by osd.374.0:5641 2014-08-13 15:28:43.410479 2014-08-13 18:06:06.882123 7f5c973a57a0 20 read_log 194416'16393 (194416'16392) modify 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by client.2441958.0:3353 2014-08-13 15:28:43.399752 2014-08-13 18:06:06.882136 7f5c973a57a0 20 read_log 194416'16394 (0'0) promote 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by osd.374.0:5683 2014-08-13 15:28:50.940871 2014-08-13 18:06:06.882154 7f5c973a57a0 20 read_log 194416'16395 (194416'16394) modify 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by client.2441958.0:5273 2014-08-13 15:28:50.932603 2014-08-13 18:06:06.882170 7f5c973a57a0 20 read_log 194416'16396 (0'0) promote 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by osd.374.0:5709 2014-08-13 15:28:55.983030 2014-08-13 18:06:06.882185 7f5c973a57a0 20 read_log 194416'16397 (194416'16396) modify 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by client.2441958.0:6531 2014-08-13 15:28:55.974403 2014-08-13 18:06:06.882200 7f5c973a57a0 20 read_log 194416'16398 (0'0) promote 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by osd.374.0:5717 2014-08-13 15:28:56.921705 2014-08-13 18:06:06.882212 7f5c973a57a0 20 read_log 194416'16399 (194416'16398) modify 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by client.2441958.0:6770 2014-08-13 15:28:56.914226 2014-08-13 18:06:06.882232 7f5c973a57a0 20 read_log 194416'16400 (0'0) promote d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by osd.374.0:5739 2014-08-13 15:28:59.057271 2014-08-13 18:06:06.882247 7f5c973a57a0 20 read_log 194416'16401 (194416'16400) modify d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by client.2441958.0:7342 2014-08-13 15:28:59.046808 2014-08-13 18:06:06.882259 7f5c973a57a0 20 read_log 194416'16402 (0'0) promote 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by osd.374.0:5787 2014-08-13 15:29:06.946845 2014-08-13 18:06:06.882273 7f5c973a57a0 20 read_log 194416'16403 (194416'16402) modify 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by client.2441958.0:9387 2014-08-13 15:29:06.939232 2014-08-13 18:06:06.882286 7f5c973a57a0 20 read_log 194416'16404 (0'0) promote 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by osd.374.0:5803 2014-08-13 15:29:10.697215 2014-08-13 18:06:06.882298 7f5c973a57a0 20 read_log 194416'16405 (194416'16404) modify 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by client.2441958.0:10303 2014-08-13 15:29:10.688257 2014-08-13 18:06:06.882314 7f5c973a57a0 20 read_log 194416'16406 (0'0) promote b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by osd.374.0:5825 2014-08-13 15:29:15.514651 2014-08-13 18:06:06.882328 7f5c973a57a0 20 read_log 194416'16407 (194416'16406) modify b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by client.2441958.0:11505 2014-08-13 15:29:15.506072 2014-08-13 18:06:06.882362 7f5c973a57a0 20 read_log 194416'16408 (0'0) promote d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by osd.374.0:5829 2014-08-13 15:29:16.569822 2014-08-13 18:06:06.882377 7f5c973a57a0 20 read_log 194416'16409 (194416'16408) modify d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by client.2441958.0:11763 2014-08-13 15:29:16.560433 2014-08-13 18:06:06.882389 7f5c973a57a0 20 read_log 194416'16410 (0'0) promote e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by osd.374.0:5879 2014-08-13 15:29:22.903298 2014-08-13 18:06:06.882402 7f5c973a57a0 20 read_log 194416'16411 (194416'16410) modify e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by client.2441958.0:13223 2014-08-13 15:29:22.895758 2014-08-13 18:06:06.882414 7f5c973a57a0 20 read_log 194416'16412 (0'0) promote 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by osd.374.0:5897 2014-08-13 15:29:26.290554 2014-08-13 18:06:06.882426 7f5c973a57a0 20 read_log 194416'16413 (194416'16412) modify 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by client.2441958.0:14020 2014-08-13 15:29:26.280813 2014-08-13 18:06:06.882448 7f5c973a57a0 20 read_log 194416'16414 (0'0) promote f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by osd.374.0:5899 2014-08-13 15:29:26.335225 2014-08-13 18:06:06.882464 7f5c973a57a0 20 read_log 194416'16415 (194416'16414) modify f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by client.2441958.0:14032 2014-08-13 15:29:26.325083 2014-08-13 18:06:06.882477 7f5c973a57a0 20 read_log 194416'16416 (0'0) promote 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by osd.374.0:5943 2014-08-13 15:29:41.821557 2014-08-13 18:06:06.882504 7f5c973a57a0 20 read_log 194416'16417 (194416'16416) modify 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by client.2441958.0:16612 2014-08-13 15:29:41.812045 2014-08-13 18:06:06.882520 7f5c973a57a0 20 read_log 194416'16418 (0'0) promote db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by osd.374.0:5947 2014-08-13 15:29:42.875150 2014-08-13 18:06:06.882542 7f5c973a57a0 20 read_log 194416'16419 (194416'16418) modify db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by client.2441958.0:16688 2014-08-13 15:29:42.865793 2014-08-13 18:06:06.882566 7f5c973a57a0 20 read_log 194416'16420 (0'0) promote 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by osd.374.0:5969 2014-08-13 15:29:59.184442 2014-08-13 18:06:06.882581 7f5c973a57a0 20 read_log 194416'16421 (194416'16420) modify 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by client.2441958.0:17840 2014-08-13 15:29:59.173753 2014-08-13 18:06:06.882594 7f5c973a57a0 20 read_log 194416'16422 (0'0) promote bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by osd.374.0:5973 2014-08-13 15:30:03.607587 2014-08-13 18:06:06.882606 7f5c973a57a0 20 read_log 194416'16423 (194416'16422) modify bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by client.2441958.0:18154 2014-08-13 15:30:03.598192 2014-08-13 18:06:06.882618 7f5c973a57a0 20 read_log 194422'16424 (0'0) promote 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by osd.374.0:5995 2014-08-13 15:32:17.391424 2014-08-13 18:06:06.882630 7f5c973a57a0 20 read_log 194422'16425 (194422'16424) modify 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by client.1148203.0:516 2014-08-13 15:32:17.381974 2014-08-13 18:06:06.882643 7f5c973a57a0 20 read_log 194422'16426 (0'0) promote f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by osd.374.0:6031 2014-08-13 15:32:21.457303 2014-08-13 18:06:06.882663 7f5c973a57a0 20 read_log 194422'16427 (194422'16426) modify f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by client.1148203.0:1558 2014-08-13 15:32:21.449286 2014-08-13 18:06:06.882679 7f5c973a57a0 20 read_log 194422'16428 (0'0) promote f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by osd.374.0:6055 2014-08-13 15:32:26.793871 2014-08-13 18:06:06.882691 7f5c973a57a0 20 read_log 194422'16429 (194422'16428) modify f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by client.1148203.0:2918 2014-08-13 15:32:26.785090 2014-08-13 18:06:06.882704 7f5c973a57a0 20 read_log 194422'16430 (0'0) promote ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by osd.374.0:6067 2014-08-13 15:32:28.357437 2014-08-13 18:06:06.882716 7f5c973a57a0 20 read_log 194422'16431 (194422'16430) modify ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by client.1148203.0:3316 2014-08-13 15:32:28.348962 2014-08-13 18:06:06.882728 7f5c973a57a0 20 read_log 194422'16432 (0'0) promote 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by osd.374.0:6079 2014-08-13 15:32:31.313768 2014-08-13 18:06:06.882740 7f5c973a57a0 20 read_log 194422'16433 (194422'16432) modify 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by client.1148203.0:4081 2014-08-13 15:32:31.305194 2014-08-13 18:06:06.882752 7f5c973a57a0 20 read_log 194422'16434 (0'0) promote 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by osd.374.0:6089 2014-08-13 15:32:33.213894 2014-08-13 18:06:06.882765 7f5c973a57a0 20 read_log 194422'16435 (194422'16434) modify 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by client.1148203.0:4579 2014-08-13 15:32:33.206430 2014-08-13 18:06:06.882777 7f5c973a57a0 20 read_log 194422'16436 (0'0) promote e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by osd.374.0:6099 2014-08-13 15:32:34.659633 2014-08-13 18:06:06.882789 7f5c973a57a0 20 read_log 194422'16437 (194422'16436) modify e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by client.1148203.0:4953 2014-08-13 15:32:34.652060 2014-08-13 18:06:06.882801 7f5c973a57a0 20 read_log 194422'16438 (0'0) promote d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by osd.374.0:6101 2014-08-13 15:32:34.729571 2014-08-13 18:06:06.882813 7f5c973a57a0 20 read_log 194422'16439 (194422'16438) modify d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by client.1148203.0:4972 2014-08-13 15:32:34.722380 2014-08-13 18:06:06.882828 7f5c973a57a0 20 read_log 194422'16440 (0'0) promote 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by osd.374.0:6111 2014-08-13 15:32:35.576832 2014-08-13 18:06:06.882844 7f5c973a57a0 20 read_log 194422'16441 (194422'16440) modify 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by client.1148203.0:5149 2014-08-13 15:32:35.566917 2014-08-13 18:06:06.882857 7f5c973a57a0 20 read_log 194422'16442 (0'0) promote ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by osd.374.0:6143 2014-08-13 15:32:42.690107 2014-08-13 18:06:06.882869 7f5c973a57a0 20 read_log 194422'16443 (194422'16442) modify ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by client.1148203.0:6469 2014-08-13 15:32:42.681075 2014-08-13 18:06:06.882881 7f5c973a57a0 20 read_log 194422'16444 (0'0) promote 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by osd.374.0:6157 2014-08-13 15:32:46.664082 2014-08-13 18:06:06.882894 7f5c973a57a0 20 read_log 194422'16445 (194422'16444) modify 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by client.1148203.0:7183 2014-08-13 15:32:46.654917 2014-08-13 18:06:06.882906 7f5c973a57a0 20 read_log 194422'16446 (0'0) promote 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by osd.374.0:6163 2014-08-13 15:32:47.391190 2014-08-13 18:06:06.882918 7f5c973a57a0 20 read_log 194422'16447 (194422'16446) modify 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by client.1148203.0:7305 2014-08-13 15:32:47.382373 2014-08-13 18:06:06.882930 7f5c973a57a0 20 read_log 194422'16448 (0'0) promote 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by osd.374.0:6219 2014-08-13 15:33:02.415725 2014-08-13 18:06:06.882943 7f5c973a57a0 20 read_log 194422'16449 (194422'16448) modify 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by client.1148203.0:9290 2014-08-13 15:33:02.407003 2014-08-13 18:06:06.882982 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.883478 7f5c973a57a0 10 read_log checking for missing items over interval (194370'16342,194422'16449] 2014-08-13 18:06:06.883525 7f5c973a57a0 15 read_log missing 194422'16449 (194422'16448) modify 8a4d6eec/benchmark_data_storage0111-ib.pocstor_41231_object9289/head//24 by client.1148203.0:9290 2014-08-13 15:33:02.407003 2014-08-13 18:06:06.883565 7f5c973a57a0 15 read_log missing 194422'16447 (194422'16446) modify 9847b6ec/benchmark_data_storage0111-ib.pocstor_41231_object7304/head//24 by client.1148203.0:7305 2014-08-13 15:32:47.382373 2014-08-13 18:06:06.883602 7f5c973a57a0 15 read_log missing 194422'16445 (194422'16444) modify 7b7caeec/benchmark_data_storage0111-ib.pocstor_41231_object7182/head//24 by client.1148203.0:7183 2014-08-13 15:32:46.654917 2014-08-13 18:06:06.883650 7f5c973a57a0 15 read_log missing 194422'16443 (194422'16442) modify ab57caec/benchmark_data_storage0111-ib.pocstor_41231_object6468/head//24 by client.1148203.0:6469 2014-08-13 15:32:42.681075 2014-08-13 18:06:06.883688 7f5c973a57a0 15 read_log missing 194422'16441 (194422'16440) modify 3df016ec/benchmark_data_storage0111-ib.pocstor_41231_object5148/head//24 by client.1148203.0:5149 2014-08-13 15:32:35.566917 2014-08-13 18:06:06.883724 7f5c973a57a0 15 read_log missing 194422'16439 (194422'16438) modify d46abaec/benchmark_data_storage0111-ib.pocstor_41231_object4971/head//24 by client.1148203.0:4972 2014-08-13 15:32:34.722380 2014-08-13 18:06:06.883759 7f5c973a57a0 15 read_log missing 194422'16437 (194422'16436) modify e093eeec/benchmark_data_storage0111-ib.pocstor_41231_object4952/head//24 by client.1148203.0:4953 2014-08-13 15:32:34.652060 2014-08-13 18:06:06.883798 7f5c973a57a0 15 read_log missing 194422'16435 (194422'16434) modify 64316ec/benchmark_data_storage0111-ib.pocstor_41231_object4578/head//24 by client.1148203.0:4579 2014-08-13 15:32:33.206430 2014-08-13 18:06:06.883834 7f5c973a57a0 15 read_log missing 194422'16433 (194422'16432) modify 9b100aec/benchmark_data_storage0111-ib.pocstor_41231_object4080/head//24 by client.1148203.0:4081 2014-08-13 15:32:31.305194 2014-08-13 18:06:06.883869 7f5c973a57a0 15 read_log missing 194422'16431 (194422'16430) modify ec1082ec/benchmark_data_storage0111-ib.pocstor_41231_object3315/head//24 by client.1148203.0:3316 2014-08-13 15:32:28.348962 2014-08-13 18:06:06.883905 7f5c973a57a0 15 read_log missing 194422'16429 (194422'16428) modify f8c75eec/benchmark_data_storage0111-ib.pocstor_41231_object2917/head//24 by client.1148203.0:2918 2014-08-13 15:32:26.785090 2014-08-13 18:06:06.883940 7f5c973a57a0 15 read_log missing 194422'16427 (194422'16426) modify f6d02aec/benchmark_data_storage0111-ib.pocstor_41231_object1557/head//24 by client.1148203.0:1558 2014-08-13 15:32:21.449286 2014-08-13 18:06:06.883975 7f5c973a57a0 15 read_log missing 194422'16425 (194422'16424) modify 5f921eec/benchmark_data_storage0111-ib.pocstor_41231_object515/head//24 by client.1148203.0:516 2014-08-13 15:32:17.381974 2014-08-13 18:06:06.884011 7f5c973a57a0 15 read_log missing 194416'16423 (194416'16422) modify bc5d92ec/benchmark_data_storage0111-ib.pocstor_19103_object18153/head//24 by client.2441958.0:18154 2014-08-13 15:30:03.598192 2014-08-13 18:06:06.884046 7f5c973a57a0 15 read_log missing 194416'16421 (194416'16420) modify 580bc2ec/benchmark_data_storage0111-ib.pocstor_19103_object17839/head//24 by client.2441958.0:17840 2014-08-13 15:29:59.173753 2014-08-13 18:06:06.884082 7f5c973a57a0 15 read_log missing 194416'16419 (194416'16418) modify db2ebeec/benchmark_data_storage0111-ib.pocstor_19103_object16687/head//24 by client.2441958.0:16688 2014-08-13 15:29:42.865793 2014-08-13 18:06:06.884118 7f5c973a57a0 15 read_log missing 194416'16417 (194416'16416) modify 993a6ec/benchmark_data_storage0111-ib.pocstor_19103_object16611/head//24 by client.2441958.0:16612 2014-08-13 15:29:41.812045 2014-08-13 18:06:06.884159 7f5c973a57a0 15 read_log missing 194416'16415 (194416'16414) modify f57c0eec/benchmark_data_storage0111-ib.pocstor_19103_object14031/head//24 by client.2441958.0:14032 2014-08-13 15:29:26.325083 2014-08-13 18:06:06.884196 7f5c973a57a0 15 read_log missing 194416'16413 (194416'16412) modify 5b5aaaec/benchmark_data_storage0111-ib.pocstor_19103_object14019/head//24 by client.2441958.0:14020 2014-08-13 15:29:26.280813 2014-08-13 18:06:06.884235 7f5c973a57a0 15 read_log missing 194416'16411 (194416'16410) modify e04e4eec/benchmark_data_storage0111-ib.pocstor_19103_object13222/head//24 by client.2441958.0:13223 2014-08-13 15:29:22.895758 2014-08-13 18:06:06.884276 7f5c973a57a0 15 read_log missing 194416'16409 (194416'16408) modify d32d22ec/benchmark_data_storage0111-ib.pocstor_19103_object11762/head//24 by client.2441958.0:11763 2014-08-13 15:29:16.560433 2014-08-13 18:06:06.884312 7f5c973a57a0 15 read_log missing 194416'16407 (194416'16406) modify b7618aec/benchmark_data_storage0111-ib.pocstor_19103_object11504/head//24 by client.2441958.0:11505 2014-08-13 15:29:15.506072 2014-08-13 18:06:06.884361 7f5c973a57a0 15 read_log missing 194416'16405 (194416'16404) modify 27505eec/benchmark_data_storage0111-ib.pocstor_19103_object10302/head//24 by client.2441958.0:10303 2014-08-13 15:29:10.688257 2014-08-13 18:06:06.884407 7f5c973a57a0 15 read_log missing 194416'16403 (194416'16402) modify 4565feec/benchmark_data_storage0111-ib.pocstor_19103_object9386/head//24 by client.2441958.0:9387 2014-08-13 15:29:06.939232 2014-08-13 18:06:06.884445 7f5c973a57a0 15 read_log missing 194416'16401 (194416'16400) modify d03c4eec/benchmark_data_storage0111-ib.pocstor_19103_object7341/head//24 by client.2441958.0:7342 2014-08-13 15:28:59.046808 2014-08-13 18:06:06.884481 7f5c973a57a0 15 read_log missing 194416'16399 (194416'16398) modify 38e152ec/benchmark_data_storage0111-ib.pocstor_19103_object6769/head//24 by client.2441958.0:6770 2014-08-13 15:28:56.914226 2014-08-13 18:06:06.884516 7f5c973a57a0 15 read_log missing 194416'16397 (194416'16396) modify 7f7faec/benchmark_data_storage0111-ib.pocstor_19103_object6530/head//24 by client.2441958.0:6531 2014-08-13 15:28:55.974403 2014-08-13 18:06:06.884552 7f5c973a57a0 15 read_log missing 194416'16395 (194416'16394) modify 5898eec/benchmark_data_storage0111-ib.pocstor_19103_object5272/head//24 by client.2441958.0:5273 2014-08-13 15:28:50.932603 2014-08-13 18:06:06.884588 7f5c973a57a0 15 read_log missing 194416'16393 (194416'16392) modify 956ae6ec/benchmark_data_storage0111-ib.pocstor_19103_object3352/head//24 by client.2441958.0:3353 2014-08-13 15:28:43.399752 2014-08-13 18:06:06.884624 7f5c973a57a0 15 read_log missing 194416'16391 (194416'16390) modify c494feec/benchmark_data_storage0111-ib.pocstor_19103_object3267/head//24 by client.2441958.0:3268 2014-08-13 15:28:43.084947 2014-08-13 18:06:06.884659 7f5c973a57a0 15 read_log missing 194416'16389 (194416'16388) modify 5802b6ec/benchmark_data_storage0111-ib.pocstor_19103_object2753/head//24 by client.2441958.0:2754 2014-08-13 15:28:41.147516 2014-08-13 18:06:06.884694 7f5c973a57a0 15 read_log missing 194416'16387 (194416'16386) modify efacfeec/benchmark_data_storage0111-ib.pocstor_19103_object1709/head//24 by client.2441958.0:1710 2014-08-13 15:28:37.189859 2014-08-13 18:06:06.884734 7f5c973a57a0 15 read_log missing 194416'16384 (194416'16384) modify 2ec/hit_set_24.2ec_archive_2014-08-12 20:53:00.019540_2014-08-13 15:28:37.194823/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:37.195058 2014-08-13 18:06:06.884769 7f5c973a57a0 15 read_log missing 194396'16383 (194396'16382) modify dd3b4aec/benchmark_data_storage0101-ib.pocstor_6819_object278/head//24 by client.2235069.0:279 2014-08-12 20:53:00.011801 2014-08-13 18:06:06.884809 7f5c973a57a0 15 read_log missing 194393'16379 (194393'16378) modify 60d5c6ec/benchmark_data_storage0111-ib.pocstor_32482_object380/head//24 by client.1064182.0:381 2014-08-12 20:49:27.253733 2014-08-13 18:06:06.884844 7f5c973a57a0 15 read_log missing 194393'16377 (194393'16376) modify a6df2aec/benchmark_data_storage0111-ib.pocstor_32482_object173/head//24 by client.1064182.0:174 2014-08-12 20:49:26.293716 2014-08-13 18:06:06.884880 7f5c973a57a0 15 read_log missing 194393'16375 (194393'16374) modify fa3c8eec/benchmark_data_storage0111-ib.pocstor_24170_object37/head//24 by client.1064095.0:38 2014-08-12 20:47:26.071882 2014-08-13 18:06:06.884919 7f5c973a57a0 15 read_log missing 194383'16371 (194383'16370) modify 4c766ec/benchmark_data_storage0111-ib.pocstor_25065_object7418/head//24 by client.2231190.0:7419 2014-08-12 20:34:26.535065 2014-08-13 18:06:06.884955 7f5c973a57a0 15 read_log missing 194383'16369 (194383'16368) modify 8a196ec/benchmark_data_storage0111-ib.pocstor_25065_object5794/head//24 by client.2231190.0:5795 2014-08-12 20:34:00.296784 2014-08-13 18:06:06.884991 7f5c973a57a0 15 read_log missing 194380'16367 (194380'16366) modify c8d7d6ec/benchmark_data_storage0111-ib.pocstor_25065_object5228/head//24 by client.2231190.0:5229 2014-08-12 20:33:26.179871 2014-08-13 18:06:06.885026 7f5c973a57a0 15 read_log missing 194377'16365 (194377'16364) modify 5ed3d2ec/benchmark_data_storage0111-ib.pocstor_25065_object3429/head//24 by client.2231190.0:3430 2014-08-12 20:33:10.995220 2014-08-13 18:06:06.885061 7f5c973a57a0 15 read_log missing 194377'16363 (194377'16362) modify 6f8c2ec/benchmark_data_storage0111-ib.pocstor_25065_object3067/head//24 by client.2231190.0:3068 2014-08-12 20:33:09.586317 2014-08-13 18:06:06.885097 7f5c973a57a0 15 read_log missing 194377'16361 (194377'16360) modify 8a1936ec/benchmark_data_storage0111-ib.pocstor_25065_object2576/head//24 by client.2231190.0:2577 2014-08-12 20:33:07.605310 2014-08-13 18:06:06.885133 7f5c973a57a0 15 read_log missing 194377'16359 (194377'16358) modify 457932ec/benchmark_data_storage0111-ib.pocstor_25065_object1210/head//24 by client.2231190.0:1211 2014-08-12 20:33:02.102879 2014-08-13 18:06:06.885175 7f5c973a57a0 15 read_log missing 194370'16357 (194370'16356) modify 6244b6ec/benchmark_data_storage0111-ib.pocstor_12495_object13661/head//24 by client.1508531.0:13662 2014-08-12 20:31:25.680926 2014-08-13 18:06:06.885211 7f5c973a57a0 15 read_log missing 194370'16355 (194370'16354) modify 8f07d6ec/benchmark_data_storage0111-ib.pocstor_12495_object12399/head//24 by client.1508531.0:12400 2014-08-12 20:31:14.270603 2014-08-13 18:06:06.885276 7f5c973a57a0 15 read_log missing 194370'16353 (194370'16352) modify 325c32ec/benchmark_data_storage0111-ib.pocstor_12495_object11418/head//24 by client.1508531.0:11419 2014-08-12 20:31:07.830173 2014-08-13 18:06:06.885318 7f5c973a57a0 15 read_log missing 194370'16351 (194370'16350) modify 3fdfcaec/benchmark_data_storage0111-ib.pocstor_12495_object7817/head//24 by client.1508531.0:7818 2014-08-12 20:30:46.615726 2014-08-13 18:06:06.885355 7f5c973a57a0 15 read_log missing 194370'16349 (194370'16348) modify 782172ec/benchmark_data_storage0111-ib.pocstor_12495_object6177/head//24 by client.1508531.0:6178 2014-08-12 20:30:38.545133 2014-08-13 18:06:06.885401 7f5c973a57a0 15 read_log missing 194370'16347 (194370'16346) modify 6043baec/benchmark_data_storage0111-ib.pocstor_12495_object4409/head//24 by client.1508531.0:4410 2014-08-12 20:30:30.848680 2014-08-13 18:06:06.885448 7f5c973a57a0 15 read_log missing 194370'16345 (194370'16344) modify 7021d2ec/benchmark_data_storage0111-ib.pocstor_12495_object3370/head//24 by client.1508531.0:3371 2014-08-12 20:30:26.898087 2014-08-13 18:06:06.885488 7f5c973a57a0 15 read_log missing 194370'16343 (194370'16342) modify 6ab942ec/benchmark_data_storage0111-ib.pocstor_12495_object2719/head//24 by client.1508531.0:2720 2014-08-12 20:30:24.520471 2014-08-13 18:06:06.885500 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.885527 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive m=52] handle_loaded 2014-08-13 18:06:06.885544 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Initial 0.015125 0 0.000000 2014-08-13 18:06:06.885557 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset 2014-08-13 18:06:06.885567 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=0 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194487 2014-08-13 18:06:06.885577 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] log((192834'14495,194422'16449], crt=194422'16449) 2014-08-13 18:06:06.885588 7f5c973a57a0 10 osd.4 194487 pgid 24.30b coll 24.30b_head 2014-08-13 18:06:06.885738 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.30b 2014-08-13 18:06:06.885751 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.885783 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.30b(unlocked)] enter Initial 2014-08-13 18:06:06.885794 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.30b(unlocked)] enter NotTrimming 2014-08-13 18:06:06.885951 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.886066 7f5c973a57a0 20 read_log 192718'15574 (192718'15573) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:1 2014-08-06 15:32:19.580604 2014-08-13 18:06:06.886088 7f5c973a57a0 20 read_log 192718'15575 (192718'15574) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:2 2014-08-06 15:32:19.692600 2014-08-13 18:06:06.886101 7f5c973a57a0 20 read_log 192718'15576 (192718'15575) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:3 2014-08-06 15:32:19.743542 2014-08-13 18:06:06.886114 7f5c973a57a0 20 read_log 192718'15577 (192718'15576) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:4 2014-08-06 15:32:19.800245 2014-08-13 18:06:06.886126 7f5c973a57a0 20 read_log 192718'15578 (192718'15577) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:5 2014-08-06 15:32:19.853298 2014-08-13 18:06:06.886139 7f5c973a57a0 20 read_log 192718'15579 (192718'15578) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:6 2014-08-06 15:32:19.889022 2014-08-13 18:06:06.886158 7f5c973a57a0 20 read_log 192718'15580 (192718'15579) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:7 2014-08-06 15:32:19.913613 2014-08-13 18:06:06.886171 7f5c973a57a0 20 read_log 192718'15581 (192718'15580) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:8 2014-08-06 15:32:20.053339 2014-08-13 18:06:06.886183 7f5c973a57a0 20 read_log 192718'15582 (192718'15581) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:9 2014-08-06 15:32:20.111084 2014-08-13 18:06:06.886196 7f5c973a57a0 20 read_log 192718'15583 (192718'15582) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:10 2014-08-06 15:32:20.143403 2014-08-13 18:06:06.886208 7f5c973a57a0 20 read_log 192718'15584 (192718'15583) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:11 2014-08-06 15:32:20.177363 2014-08-13 18:06:06.886220 7f5c973a57a0 20 read_log 192718'15585 (192718'15584) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:12 2014-08-06 15:32:20.221803 2014-08-13 18:06:06.886242 7f5c973a57a0 20 read_log 192718'15586 (192718'15585) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:13 2014-08-06 15:32:20.424468 2014-08-13 18:06:06.886257 7f5c973a57a0 20 read_log 192718'15587 (192718'15586) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:14 2014-08-06 15:32:20.456094 2014-08-13 18:06:06.886269 7f5c973a57a0 20 read_log 192718'15588 (192718'15587) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:15 2014-08-06 15:32:20.486172 2014-08-13 18:06:06.886281 7f5c973a57a0 20 read_log 192718'15589 (192718'15588) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:16 2014-08-06 15:32:20.516832 2014-08-13 18:06:06.886297 7f5c973a57a0 20 read_log 192718'15590 (192718'15589) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:17 2014-08-06 15:32:21.014821 2014-08-13 18:06:06.886310 7f5c973a57a0 20 read_log 192718'15591 (192718'15590) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:18 2014-08-06 15:32:21.046317 2014-08-13 18:06:06.886322 7f5c973a57a0 20 read_log 192718'15592 (192718'15591) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:19 2014-08-06 15:32:21.498016 2014-08-13 18:06:06.886334 7f5c973a57a0 20 read_log 192718'15593 (192718'15592) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:20 2014-08-06 15:32:21.530318 2014-08-13 18:06:06.886346 7f5c973a57a0 20 read_log 192718'15594 (192718'15593) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:21 2014-08-06 15:32:21.557135 2014-08-13 18:06:06.886358 7f5c973a57a0 20 read_log 192718'15595 (192718'15594) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:22 2014-08-06 15:32:21.650377 2014-08-13 18:06:06.886370 7f5c973a57a0 20 read_log 192718'15596 (192718'15595) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:23 2014-08-06 15:32:22.089062 2014-08-13 18:06:06.886383 7f5c973a57a0 20 read_log 192718'15597 (192718'15596) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:24 2014-08-06 15:32:22.117791 2014-08-13 18:06:06.886395 7f5c973a57a0 20 read_log 192718'15598 (192718'15597) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:25 2014-08-06 15:32:22.144425 2014-08-13 18:06:06.886407 7f5c973a57a0 20 read_log 192718'15599 (192718'15598) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:26 2014-08-06 15:32:22.170562 2014-08-13 18:06:06.886419 7f5c973a57a0 20 read_log 192718'15600 (192718'15599) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:27 2014-08-06 15:32:22.343725 2014-08-13 18:06:06.886432 7f5c973a57a0 20 read_log 192718'15601 (192718'15600) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:28 2014-08-06 15:32:22.388523 2014-08-13 18:06:06.886444 7f5c973a57a0 20 read_log 192718'15602 (192718'15601) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:29 2014-08-06 15:32:22.422259 2014-08-13 18:06:06.886456 7f5c973a57a0 20 read_log 192718'15603 (192718'15602) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:30 2014-08-06 15:32:22.451117 2014-08-13 18:06:06.886500 7f5c973a57a0 20 read_log 192718'15604 (192718'15603) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:31 2014-08-06 15:32:22.912549 2014-08-13 18:06:06.886513 7f5c973a57a0 20 read_log 192718'15605 (192718'15604) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:32 2014-08-06 15:32:22.938720 2014-08-13 18:06:06.886536 7f5c973a57a0 20 read_log 192718'15606 (192718'15605) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:33 2014-08-06 15:32:22.967018 2014-08-13 18:06:06.886549 7f5c973a57a0 20 read_log 192718'15607 (192718'15606) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:34 2014-08-06 15:32:22.997888 2014-08-13 18:06:06.886574 7f5c973a57a0 20 read_log 192718'15608 (192718'15607) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:35 2014-08-06 15:32:23.025778 2014-08-13 18:06:06.886590 7f5c973a57a0 20 read_log 192718'15609 (192718'15608) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:36 2014-08-06 15:32:23.057045 2014-08-13 18:06:06.886642 7f5c973a57a0 20 read_log 192718'15610 (192718'15609) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:37 2014-08-06 15:32:23.082529 2014-08-13 18:06:06.886660 7f5c973a57a0 20 read_log 192718'15611 (192718'15610) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:38 2014-08-06 15:32:23.113545 2014-08-13 18:06:06.886672 7f5c973a57a0 20 read_log 192718'15612 (192718'15611) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:39 2014-08-06 15:32:23.140084 2014-08-13 18:06:06.886684 7f5c973a57a0 20 read_log 192718'15613 (192718'15612) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:40 2014-08-06 15:32:23.169759 2014-08-13 18:06:06.886697 7f5c973a57a0 20 read_log 192718'15614 (192718'15613) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:41 2014-08-06 15:32:23.195278 2014-08-13 18:06:06.886709 7f5c973a57a0 20 read_log 192718'15615 (192718'15614) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:42 2014-08-06 15:32:23.290058 2014-08-13 18:06:06.886721 7f5c973a57a0 20 read_log 192718'15616 (192718'15615) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:43 2014-08-06 15:32:23.488138 2014-08-13 18:06:06.886733 7f5c973a57a0 20 read_log 192718'15617 (192718'15616) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:44 2014-08-06 15:32:23.641047 2014-08-13 18:06:06.886745 7f5c973a57a0 20 read_log 192718'15618 (192718'15617) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:45 2014-08-06 15:32:23.693469 2014-08-13 18:06:06.886756 7f5c973a57a0 20 read_log 192718'15619 (192718'15618) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:46 2014-08-06 15:32:23.721590 2014-08-13 18:06:06.886786 7f5c973a57a0 20 read_log 192718'15620 (192718'15619) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:47 2014-08-06 15:32:24.400743 2014-08-13 18:06:06.886801 7f5c973a57a0 20 read_log 192718'15621 (192718'15620) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:48 2014-08-06 15:32:24.443279 2014-08-13 18:06:06.886814 7f5c973a57a0 20 read_log 192718'15622 (192718'15621) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:49 2014-08-06 15:32:24.487274 2014-08-13 18:06:06.886826 7f5c973a57a0 20 read_log 192718'15623 (192718'15622) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:50 2014-08-06 15:32:24.526395 2014-08-13 18:06:06.886838 7f5c973a57a0 20 read_log 192718'15624 (192718'15623) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:51 2014-08-06 15:32:24.552731 2014-08-13 18:06:06.886850 7f5c973a57a0 20 read_log 192718'15625 (192718'15624) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:52 2014-08-06 15:32:24.582598 2014-08-13 18:06:06.886862 7f5c973a57a0 20 read_log 192718'15626 (192718'15625) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:53 2014-08-06 15:32:24.613349 2014-08-13 18:06:06.886874 7f5c973a57a0 20 read_log 192718'15627 (192718'15626) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:54 2014-08-06 15:32:24.647504 2014-08-13 18:06:06.886886 7f5c973a57a0 20 read_log 192718'15628 (192718'15627) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:55 2014-08-06 15:32:24.849001 2014-08-13 18:06:06.886899 7f5c973a57a0 20 read_log 192718'15629 (192718'15628) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:56 2014-08-06 15:32:24.879150 2014-08-13 18:06:06.886911 7f5c973a57a0 20 read_log 192718'15630 (192718'15629) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:57 2014-08-06 15:32:25.056064 2014-08-13 18:06:06.886925 7f5c973a57a0 20 read_log 192718'15631 (192718'15630) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:58 2014-08-06 15:32:25.081502 2014-08-13 18:06:06.886937 7f5c973a57a0 20 read_log 192718'15632 (192718'15631) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:59 2014-08-06 15:32:25.108178 2014-08-13 18:06:06.886949 7f5c973a57a0 20 read_log 192718'15633 (192718'15632) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:60 2014-08-06 15:32:25.133724 2014-08-13 18:06:06.886968 7f5c973a57a0 20 read_log 192718'15634 (192718'15633) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:61 2014-08-06 15:32:25.159726 2014-08-13 18:06:06.886981 7f5c973a57a0 20 read_log 192718'15635 (192718'15634) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:62 2014-08-06 15:32:25.187642 2014-08-13 18:06:06.886993 7f5c973a57a0 20 read_log 192718'15636 (192718'15635) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:63 2014-08-06 15:32:25.213435 2014-08-13 18:06:06.887005 7f5c973a57a0 20 read_log 192718'15637 (192718'15636) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:64 2014-08-06 15:32:25.246718 2014-08-13 18:06:06.887017 7f5c973a57a0 20 read_log 192718'15638 (192718'15637) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:65 2014-08-06 15:32:25.315430 2014-08-13 18:06:06.887032 7f5c973a57a0 20 read_log 192718'15639 (192718'15638) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:66 2014-08-06 15:32:25.351597 2014-08-13 18:06:06.887044 7f5c973a57a0 20 read_log 192718'15640 (192718'15639) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:67 2014-08-06 15:32:25.454422 2014-08-13 18:06:06.887058 7f5c973a57a0 20 read_log 192718'15641 (192718'15640) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:68 2014-08-06 15:32:25.571455 2014-08-13 18:06:06.887077 7f5c973a57a0 20 read_log 192718'15642 (192718'15641) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:69 2014-08-06 15:32:25.677280 2014-08-13 18:06:06.887090 7f5c973a57a0 20 read_log 192718'15643 (192718'15642) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:70 2014-08-06 15:32:25.702622 2014-08-13 18:06:06.887104 7f5c973a57a0 20 read_log 192718'15644 (192718'15643) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:71 2014-08-06 15:32:25.727944 2014-08-13 18:06:06.887116 7f5c973a57a0 20 read_log 192718'15645 (192718'15644) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:72 2014-08-06 15:32:25.752849 2014-08-13 18:06:06.887130 7f5c973a57a0 20 read_log 192718'15646 (192718'15645) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:73 2014-08-06 15:32:25.778536 2014-08-13 18:06:06.887143 7f5c973a57a0 20 read_log 192718'15647 (192718'15646) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:74 2014-08-06 15:32:25.805235 2014-08-13 18:06:06.887166 7f5c973a57a0 20 read_log 192718'15648 (192718'15647) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:75 2014-08-06 15:32:25.844938 2014-08-13 18:06:06.887179 7f5c973a57a0 20 read_log 192718'15649 (192718'15648) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:76 2014-08-06 15:32:25.870417 2014-08-13 18:06:06.887192 7f5c973a57a0 20 read_log 192718'15650 (192718'15649) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:77 2014-08-06 15:32:25.896148 2014-08-13 18:06:06.887204 7f5c973a57a0 20 read_log 192718'15651 (192718'15650) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:78 2014-08-06 15:32:25.921919 2014-08-13 18:06:06.887218 7f5c973a57a0 20 read_log 192718'15652 (192718'15651) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:79 2014-08-06 15:32:26.358745 2014-08-13 18:06:06.887230 7f5c973a57a0 20 read_log 192718'15653 (192718'15652) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:80 2014-08-06 15:32:26.385430 2014-08-13 18:06:06.887246 7f5c973a57a0 20 read_log 192718'15654 (192718'15653) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:81 2014-08-06 15:32:26.409391 2014-08-13 18:06:06.887258 7f5c973a57a0 20 read_log 192718'15655 (192718'15654) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:82 2014-08-06 15:32:26.433938 2014-08-13 18:06:06.887270 7f5c973a57a0 20 read_log 192718'15656 (192718'15655) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:83 2014-08-06 15:32:26.457708 2014-08-13 18:06:06.887283 7f5c973a57a0 20 read_log 192718'15657 (192718'15656) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:84 2014-08-06 15:32:26.487117 2014-08-13 18:06:06.887302 7f5c973a57a0 20 read_log 192718'15658 (192718'15657) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:85 2014-08-06 15:32:26.514888 2014-08-13 18:06:06.887322 7f5c973a57a0 20 read_log 192718'15659 (192718'15658) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:86 2014-08-06 15:32:26.544492 2014-08-13 18:06:06.887334 7f5c973a57a0 20 read_log 192718'15660 (192718'15659) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:87 2014-08-06 15:32:26.573226 2014-08-13 18:06:06.887346 7f5c973a57a0 20 read_log 192718'15661 (192718'15660) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:88 2014-08-06 15:32:26.599445 2014-08-13 18:06:06.887362 7f5c973a57a0 20 read_log 192718'15662 (192718'15661) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:89 2014-08-06 15:32:26.626167 2014-08-13 18:06:06.887374 7f5c973a57a0 20 read_log 192718'15663 (192718'15662) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:90 2014-08-06 15:32:26.652695 2014-08-13 18:06:06.887389 7f5c973a57a0 20 read_log 192718'15664 (192718'15663) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:91 2014-08-06 15:32:26.678707 2014-08-13 18:06:06.887401 7f5c973a57a0 20 read_log 192718'15665 (192718'15664) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:92 2014-08-06 15:32:26.705214 2014-08-13 18:06:06.887419 7f5c973a57a0 20 read_log 192718'15666 (192718'15665) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:93 2014-08-06 15:32:26.730781 2014-08-13 18:06:06.887431 7f5c973a57a0 20 read_log 192718'15667 (192718'15666) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:94 2014-08-06 15:32:26.756977 2014-08-13 18:06:06.887453 7f5c973a57a0 20 read_log 192718'15668 (192718'15667) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:95 2014-08-06 15:32:26.785347 2014-08-13 18:06:06.887468 7f5c973a57a0 20 read_log 192718'15669 (192718'15668) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:96 2014-08-06 15:32:26.813690 2014-08-13 18:06:06.887481 7f5c973a57a0 20 read_log 192718'15670 (192718'15669) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:97 2014-08-06 15:32:26.842333 2014-08-13 18:06:06.887505 7f5c973a57a0 20 read_log 192718'15671 (192718'15670) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:98 2014-08-06 15:32:26.871918 2014-08-13 18:06:06.887520 7f5c973a57a0 20 read_log 192718'15672 (192718'15671) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:99 2014-08-06 15:32:26.900684 2014-08-13 18:06:06.887533 7f5c973a57a0 20 read_log 192718'15673 (192718'15672) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:100 2014-08-06 15:32:26.929639 2014-08-13 18:06:06.887556 7f5c973a57a0 20 read_log 192718'15674 (192718'15673) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:101 2014-08-06 15:32:26.955041 2014-08-13 18:06:06.887569 7f5c973a57a0 20 read_log 192718'15675 (192718'15674) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:102 2014-08-06 15:32:26.981658 2014-08-13 18:06:06.887598 7f5c973a57a0 20 read_log 192718'15676 (192718'15675) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:103 2014-08-06 15:32:27.007156 2014-08-13 18:06:06.887612 7f5c973a57a0 20 read_log 192718'15677 (192718'15676) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:104 2014-08-06 15:32:27.061791 2014-08-13 18:06:06.887627 7f5c973a57a0 20 read_log 192718'15678 (192718'15677) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:105 2014-08-06 15:32:27.157448 2014-08-13 18:06:06.887642 7f5c973a57a0 20 read_log 192718'15679 (192718'15678) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:106 2014-08-06 15:32:27.182423 2014-08-13 18:06:06.887655 7f5c973a57a0 20 read_log 192718'15680 (192718'15679) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:107 2014-08-06 15:32:27.206050 2014-08-13 18:06:06.887667 7f5c973a57a0 20 read_log 192718'15681 (192718'15680) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:108 2014-08-06 15:32:27.235056 2014-08-13 18:06:06.887690 7f5c973a57a0 20 read_log 192718'15682 (192718'15681) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:109 2014-08-06 15:32:27.259678 2014-08-13 18:06:06.887714 7f5c973a57a0 20 read_log 192718'15683 (192718'15682) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:110 2014-08-06 15:32:27.286879 2014-08-13 18:06:06.887733 7f5c973a57a0 20 read_log 192718'15684 (192718'15683) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:111 2014-08-06 15:32:27.312605 2014-08-13 18:06:06.887748 7f5c973a57a0 20 read_log 192718'15685 (192718'15684) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:112 2014-08-06 15:32:27.337221 2014-08-13 18:06:06.887760 7f5c973a57a0 20 read_log 192718'15686 (192718'15685) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:113 2014-08-06 15:32:27.361964 2014-08-13 18:06:06.887776 7f5c973a57a0 20 read_log 192718'15687 (192718'15686) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:114 2014-08-06 15:32:27.463402 2014-08-13 18:06:06.887791 7f5c973a57a0 20 read_log 192718'15688 (192718'15687) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:115 2014-08-06 15:32:27.531049 2014-08-13 18:06:06.887803 7f5c973a57a0 20 read_log 192718'15689 (192718'15688) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:116 2014-08-06 15:32:27.606088 2014-08-13 18:06:06.887824 7f5c973a57a0 20 read_log 192718'15690 (192718'15689) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:117 2014-08-06 15:32:27.649802 2014-08-13 18:06:06.887840 7f5c973a57a0 20 read_log 192718'15691 (192718'15690) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:118 2014-08-06 15:32:27.709757 2014-08-13 18:06:06.887852 7f5c973a57a0 20 read_log 192718'15692 (192718'15691) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:119 2014-08-06 15:32:27.736318 2014-08-13 18:06:06.887865 7f5c973a57a0 20 read_log 192718'15693 (192718'15692) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:120 2014-08-06 15:32:27.762588 2014-08-13 18:06:06.887877 7f5c973a57a0 20 read_log 192718'15694 (192718'15693) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:121 2014-08-06 15:32:27.787708 2014-08-13 18:06:06.887889 7f5c973a57a0 20 read_log 192718'15695 (192718'15694) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:122 2014-08-06 15:32:27.812620 2014-08-13 18:06:06.887901 7f5c973a57a0 20 read_log 192718'15696 (192718'15695) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:123 2014-08-06 15:32:27.839238 2014-08-13 18:06:06.887913 7f5c973a57a0 20 read_log 192718'15697 (192718'15696) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:124 2014-08-06 15:32:27.865306 2014-08-13 18:06:06.887925 7f5c973a57a0 20 read_log 192718'15698 (192718'15697) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:125 2014-08-06 15:32:27.937336 2014-08-13 18:06:06.887937 7f5c973a57a0 20 read_log 192718'15699 (192718'15698) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:126 2014-08-06 15:32:27.966533 2014-08-13 18:06:06.887950 7f5c973a57a0 20 read_log 192718'15700 (192718'15699) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:127 2014-08-06 15:32:27.993186 2014-08-13 18:06:06.887962 7f5c973a57a0 20 read_log 192718'15701 (192718'15700) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:128 2014-08-06 15:32:28.023756 2014-08-13 18:06:06.887974 7f5c973a57a0 20 read_log 192718'15702 (192718'15701) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:129 2014-08-06 15:32:28.050464 2014-08-13 18:06:06.887986 7f5c973a57a0 20 read_log 192718'15703 (192718'15702) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:130 2014-08-06 15:32:28.101485 2014-08-13 18:06:06.887998 7f5c973a57a0 20 read_log 192718'15704 (192718'15703) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:131 2014-08-06 15:32:28.207307 2014-08-13 18:06:06.888010 7f5c973a57a0 20 read_log 192718'15705 (192718'15704) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:132 2014-08-06 15:32:28.242223 2014-08-13 18:06:06.888031 7f5c973a57a0 20 read_log 192718'15706 (192718'15705) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:133 2014-08-06 15:32:28.268995 2014-08-13 18:06:06.888044 7f5c973a57a0 20 read_log 192718'15707 (192718'15706) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:134 2014-08-06 15:32:28.294469 2014-08-13 18:06:06.888057 7f5c973a57a0 20 read_log 192718'15708 (192718'15707) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:135 2014-08-06 15:32:28.413560 2014-08-13 18:06:06.888069 7f5c973a57a0 20 read_log 192718'15709 (192718'15708) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:136 2014-08-06 15:32:28.460080 2014-08-13 18:06:06.888081 7f5c973a57a0 20 read_log 192718'15710 (192718'15709) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:137 2014-08-06 15:32:28.615753 2014-08-13 18:06:06.888093 7f5c973a57a0 20 read_log 192718'15711 (192718'15710) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:138 2014-08-06 15:32:28.643496 2014-08-13 18:06:06.888105 7f5c973a57a0 20 read_log 192718'15712 (192718'15711) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:139 2014-08-06 15:32:28.684501 2014-08-13 18:06:06.888138 7f5c973a57a0 20 read_log 192718'15713 (192718'15712) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:140 2014-08-06 15:32:28.713122 2014-08-13 18:06:06.888161 7f5c973a57a0 20 read_log 192718'15714 (192718'15713) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:141 2014-08-06 15:32:28.741290 2014-08-13 18:06:06.888174 7f5c973a57a0 20 read_log 192718'15715 (192718'15714) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:142 2014-08-06 15:32:28.768859 2014-08-13 18:06:06.888186 7f5c973a57a0 20 read_log 192718'15716 (192718'15715) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:143 2014-08-06 15:32:29.285443 2014-08-13 18:06:06.888199 7f5c973a57a0 20 read_log 192718'15717 (192718'15716) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:144 2014-08-06 15:32:29.310911 2014-08-13 18:06:06.888211 7f5c973a57a0 20 read_log 192718'15718 (192718'15717) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:145 2014-08-06 15:32:29.336192 2014-08-13 18:06:06.888223 7f5c973a57a0 20 read_log 192718'15719 (192718'15718) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:146 2014-08-06 15:32:29.361498 2014-08-13 18:06:06.888238 7f5c973a57a0 20 read_log 192718'15720 (192718'15719) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:147 2014-08-06 15:32:29.389212 2014-08-13 18:06:06.888251 7f5c973a57a0 20 read_log 192718'15721 (192718'15720) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:148 2014-08-06 15:32:29.417043 2014-08-13 18:06:06.888263 7f5c973a57a0 20 read_log 192718'15722 (192718'15721) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:149 2014-08-06 15:32:29.443457 2014-08-13 18:06:06.888275 7f5c973a57a0 20 read_log 192718'15723 (192718'15722) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:150 2014-08-06 15:32:29.469226 2014-08-13 18:06:06.888291 7f5c973a57a0 20 read_log 192718'15724 (192718'15723) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:151 2014-08-06 15:32:29.503349 2014-08-13 18:06:06.888304 7f5c973a57a0 20 read_log 192718'15725 (192718'15724) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:152 2014-08-06 15:32:29.532822 2014-08-13 18:06:06.888316 7f5c973a57a0 20 read_log 192718'15726 (192718'15725) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:153 2014-08-06 15:32:29.559896 2014-08-13 18:06:06.888328 7f5c973a57a0 20 read_log 192718'15727 (192718'15726) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:154 2014-08-06 15:32:29.589702 2014-08-13 18:06:06.888344 7f5c973a57a0 20 read_log 192718'15728 (192718'15727) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:155 2014-08-06 15:32:29.616076 2014-08-13 18:06:06.888356 7f5c973a57a0 20 read_log 192718'15729 (192718'15728) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:156 2014-08-06 15:32:29.643624 2014-08-13 18:06:06.888375 7f5c973a57a0 20 read_log 192718'15730 (192718'15729) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:157 2014-08-06 15:32:29.671726 2014-08-13 18:06:06.888394 7f5c973a57a0 20 read_log 192718'15731 (192718'15730) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:158 2014-08-06 15:32:29.701999 2014-08-13 18:06:06.888406 7f5c973a57a0 20 read_log 192718'15732 (192718'15731) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:159 2014-08-06 15:32:29.729118 2014-08-13 18:06:06.888419 7f5c973a57a0 20 read_log 192718'15733 (192718'15429) delete 89fe6f0b/core.23865+28+2/head//24 by osd.329.0:16798 2014-08-06 15:32:29.755294 2014-08-13 18:06:06.888431 7f5c973a57a0 20 read_log 192718'15735 (192718'15732) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:160 2014-08-06 15:32:29.760138 2014-08-13 18:06:06.888457 7f5c973a57a0 20 read_log 192718'15736 (192718'15735) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:161 2014-08-06 15:32:29.788898 2014-08-13 18:06:06.888470 7f5c973a57a0 20 read_log 192718'15737 (192718'15736) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:162 2014-08-06 15:32:29.815989 2014-08-13 18:06:06.888489 7f5c973a57a0 20 read_log 192718'15738 (192718'15737) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:163 2014-08-06 15:32:29.844163 2014-08-13 18:06:06.888505 7f5c973a57a0 20 read_log 192718'15739 (192718'15738) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:164 2014-08-06 15:32:29.872463 2014-08-13 18:06:06.888517 7f5c973a57a0 20 read_log 192718'15740 (192718'15739) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:165 2014-08-06 15:32:29.900225 2014-08-13 18:06:06.888530 7f5c973a57a0 20 read_log 192718'15741 (192718'15740) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:166 2014-08-06 15:32:29.930765 2014-08-13 18:06:06.888556 7f5c973a57a0 20 read_log 192718'15742 (192718'15741) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:167 2014-08-06 15:32:29.957888 2014-08-13 18:06:06.888569 7f5c973a57a0 20 read_log 192718'15743 (192718'15742) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:168 2014-08-06 15:32:29.988954 2014-08-13 18:06:06.888589 7f5c973a57a0 20 read_log 192718'15744 (192718'15743) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:169 2014-08-06 15:32:30.021263 2014-08-13 18:06:06.888605 7f5c973a57a0 20 read_log 192718'15745 (192718'15744) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:170 2014-08-06 15:32:30.047725 2014-08-13 18:06:06.888618 7f5c973a57a0 20 read_log 192718'15746 (192718'15745) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:171 2014-08-06 15:32:30.075145 2014-08-13 18:06:06.888630 7f5c973a57a0 20 read_log 192718'15747 (192718'15746) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:172 2014-08-06 15:32:30.104842 2014-08-13 18:06:06.888642 7f5c973a57a0 20 read_log 192718'15748 (192718'15747) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:173 2014-08-06 15:32:30.133522 2014-08-13 18:06:06.888655 7f5c973a57a0 20 read_log 192718'15749 (192718'15748) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:174 2014-08-06 15:32:30.163963 2014-08-13 18:06:06.888667 7f5c973a57a0 20 read_log 192718'15750 (192718'15749) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:175 2014-08-06 15:32:30.192470 2014-08-13 18:06:06.888679 7f5c973a57a0 20 read_log 192718'15751 (192718'15750) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:176 2014-08-06 15:32:30.218896 2014-08-13 18:06:06.888691 7f5c973a57a0 20 read_log 192718'15752 (192718'15751) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:177 2014-08-06 15:32:30.245890 2014-08-13 18:06:06.888703 7f5c973a57a0 20 read_log 192718'15753 (192718'15752) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:178 2014-08-06 15:32:30.277306 2014-08-13 18:06:06.888716 7f5c973a57a0 20 read_log 192718'15754 (192718'15753) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:179 2014-08-06 15:32:30.309189 2014-08-13 18:06:06.888735 7f5c973a57a0 20 read_log 192718'15755 (192718'15754) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:180 2014-08-06 15:32:30.334983 2014-08-13 18:06:06.888748 7f5c973a57a0 20 read_log 192718'15756 (192718'15755) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:181 2014-08-06 15:32:30.365307 2014-08-13 18:06:06.888760 7f5c973a57a0 20 read_log 192718'15757 (192718'15756) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:182 2014-08-06 15:32:30.396031 2014-08-13 18:06:06.888772 7f5c973a57a0 20 read_log 192718'15758 (192718'15757) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:183 2014-08-06 15:32:30.425216 2014-08-13 18:06:06.888784 7f5c973a57a0 20 read_log 192718'15759 (192718'15758) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:184 2014-08-06 15:32:30.455305 2014-08-13 18:06:06.888796 7f5c973a57a0 20 read_log 192718'15760 (192718'15759) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:185 2014-08-06 15:32:30.488465 2014-08-13 18:06:06.888808 7f5c973a57a0 20 read_log 192718'15761 (192718'15760) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:186 2014-08-06 15:32:30.520055 2014-08-13 18:06:06.888820 7f5c973a57a0 20 read_log 192718'15762 (192718'15761) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:187 2014-08-06 15:32:30.761882 2014-08-13 18:06:06.888832 7f5c973a57a0 20 read_log 192718'15763 (192718'15762) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:188 2014-08-06 15:32:30.795007 2014-08-13 18:06:06.888844 7f5c973a57a0 20 read_log 192718'15764 (192718'15763) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:189 2014-08-06 15:32:30.825046 2014-08-13 18:06:06.888856 7f5c973a57a0 20 read_log 192718'15765 (192718'15764) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:190 2014-08-06 15:32:30.857840 2014-08-13 18:06:06.888868 7f5c973a57a0 20 read_log 192718'15766 (192718'15765) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:191 2014-08-06 15:32:30.888511 2014-08-13 18:06:06.888880 7f5c973a57a0 20 read_log 192718'15767 (192718'15766) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:192 2014-08-06 15:32:30.917396 2014-08-13 18:06:06.888892 7f5c973a57a0 20 read_log 192718'15768 (192718'15767) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:193 2014-08-06 15:32:30.946535 2014-08-13 18:06:06.888907 7f5c973a57a0 20 read_log 192718'15769 (192718'15768) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:194 2014-08-06 15:32:30.975519 2014-08-13 18:06:06.888919 7f5c973a57a0 20 read_log 192718'15770 (192718'15769) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:195 2014-08-06 15:32:31.006792 2014-08-13 18:06:06.888934 7f5c973a57a0 20 read_log 192718'15771 (192718'15770) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:196 2014-08-06 15:32:31.039697 2014-08-13 18:06:06.888946 7f5c973a57a0 20 read_log 192718'15772 (192718'15771) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:197 2014-08-06 15:32:31.070106 2014-08-13 18:06:06.888961 7f5c973a57a0 20 read_log 192718'15773 (192718'15772) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:198 2014-08-06 15:32:31.101274 2014-08-13 18:06:06.888973 7f5c973a57a0 20 read_log 192718'15774 (192718'15773) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:199 2014-08-06 15:32:31.130545 2014-08-13 18:06:06.888985 7f5c973a57a0 20 read_log 192718'15775 (192718'15774) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:200 2014-08-06 15:32:31.158673 2014-08-13 18:06:06.889000 7f5c973a57a0 20 read_log 192718'15776 (192718'15775) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:201 2014-08-06 15:32:31.188293 2014-08-13 18:06:06.889013 7f5c973a57a0 20 read_log 192718'15777 (192718'15776) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:202 2014-08-06 15:32:31.218281 2014-08-13 18:06:06.889027 7f5c973a57a0 20 read_log 192718'15778 (192718'15777) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:203 2014-08-06 15:32:31.249829 2014-08-13 18:06:06.889064 7f5c973a57a0 20 read_log 192718'15779 (192718'15778) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:204 2014-08-06 15:32:31.280650 2014-08-13 18:06:06.889080 7f5c973a57a0 20 read_log 192718'15780 (192718'15779) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:205 2014-08-06 15:32:31.310511 2014-08-13 18:06:06.889092 7f5c973a57a0 20 read_log 192718'15781 (192718'15780) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:206 2014-08-06 15:32:31.342265 2014-08-13 18:06:06.889107 7f5c973a57a0 20 read_log 192718'15782 (192718'15781) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:207 2014-08-06 15:32:31.808262 2014-08-13 18:06:06.889119 7f5c973a57a0 20 read_log 192718'15783 (192718'15782) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:208 2014-08-06 15:32:31.839470 2014-08-13 18:06:06.889133 7f5c973a57a0 20 read_log 192718'15784 (192718'15783) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:209 2014-08-06 15:32:31.865758 2014-08-13 18:06:06.889152 7f5c973a57a0 20 read_log 192718'15785 (192718'15784) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:210 2014-08-06 15:32:31.921105 2014-08-13 18:06:06.889167 7f5c973a57a0 20 read_log 192718'15786 (192718'15785) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:211 2014-08-06 15:32:31.953348 2014-08-13 18:06:06.889180 7f5c973a57a0 20 read_log 192718'15787 (192718'15786) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:212 2014-08-06 15:32:32.043388 2014-08-13 18:06:06.889192 7f5c973a57a0 20 read_log 192718'15788 (192718'15787) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:213 2014-08-06 15:32:32.157295 2014-08-13 18:06:06.889204 7f5c973a57a0 20 read_log 192718'15789 (192718'15788) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:214 2014-08-06 15:32:32.198584 2014-08-13 18:06:06.889219 7f5c973a57a0 20 read_log 192718'15790 (192718'15789) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:215 2014-08-06 15:32:32.229528 2014-08-13 18:06:06.889231 7f5c973a57a0 20 read_log 192718'15791 (192718'15790) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:216 2014-08-06 15:32:32.258098 2014-08-13 18:06:06.889245 7f5c973a57a0 20 read_log 192718'15792 (192718'15791) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:217 2014-08-06 15:32:32.287448 2014-08-13 18:06:06.889257 7f5c973a57a0 20 read_log 192718'15793 (192718'15792) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:218 2014-08-06 15:32:32.319038 2014-08-13 18:06:06.889275 7f5c973a57a0 20 read_log 192718'15794 (192718'15793) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:219 2014-08-06 15:32:32.351863 2014-08-13 18:06:06.889287 7f5c973a57a0 20 read_log 192718'15795 (192718'15794) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:220 2014-08-06 15:32:32.503901 2014-08-13 18:06:06.889299 7f5c973a57a0 20 read_log 192718'15796 (192718'15795) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:221 2014-08-06 15:32:32.678899 2014-08-13 18:06:06.889315 7f5c973a57a0 20 read_log 192718'15797 (192718'15796) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:222 2014-08-06 15:32:32.717315 2014-08-13 18:06:06.889327 7f5c973a57a0 20 read_log 192718'15798 (192718'15797) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:223 2014-08-06 15:32:32.746581 2014-08-13 18:06:06.889339 7f5c973a57a0 20 read_log 192718'15799 (192718'15798) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:224 2014-08-06 15:32:32.776221 2014-08-13 18:06:06.889351 7f5c973a57a0 20 read_log 192718'15800 (192718'15799) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:225 2014-08-06 15:32:32.804897 2014-08-13 18:06:06.889367 7f5c973a57a0 20 read_log 192718'15801 (192718'15800) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:226 2014-08-06 15:32:32.832389 2014-08-13 18:06:06.889379 7f5c973a57a0 20 read_log 192718'15802 (192718'15801) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:227 2014-08-06 15:32:32.862095 2014-08-13 18:06:06.889398 7f5c973a57a0 20 read_log 192718'15803 (192718'15802) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:228 2014-08-06 15:32:32.891437 2014-08-13 18:06:06.889416 7f5c973a57a0 20 read_log 192718'15804 (192718'15803) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:229 2014-08-06 15:32:32.920910 2014-08-13 18:06:06.889429 7f5c973a57a0 20 read_log 192718'15805 (192718'15804) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:230 2014-08-06 15:32:32.950345 2014-08-13 18:06:06.889441 7f5c973a57a0 20 read_log 192718'15806 (192718'15805) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:231 2014-08-06 15:32:33.060401 2014-08-13 18:06:06.889479 7f5c973a57a0 20 read_log 192718'15807 (192718'15806) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:232 2014-08-06 15:32:33.093213 2014-08-13 18:06:06.889497 7f5c973a57a0 20 read_log 192718'15808 (192718'15807) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:233 2014-08-06 15:32:33.122825 2014-08-13 18:06:06.889510 7f5c973a57a0 20 read_log 192718'15809 (192718'15808) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:234 2014-08-06 15:32:33.153468 2014-08-13 18:06:06.889532 7f5c973a57a0 20 read_log 192718'15810 (192718'15809) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:235 2014-08-06 15:32:33.184769 2014-08-13 18:06:06.889548 7f5c973a57a0 20 read_log 192718'15811 (192718'15810) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:236 2014-08-06 15:32:33.300763 2014-08-13 18:06:06.889560 7f5c973a57a0 20 read_log 192718'15812 (192718'15811) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:237 2014-08-06 15:32:33.331022 2014-08-13 18:06:06.889581 7f5c973a57a0 20 read_log 192718'15813 (192718'15812) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:238 2014-08-06 15:32:33.363149 2014-08-13 18:06:06.889601 7f5c973a57a0 20 read_log 192718'15814 (192718'15813) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:239 2014-08-06 15:32:33.391820 2014-08-13 18:06:06.889616 7f5c973a57a0 20 read_log 192718'15815 (192718'15814) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:240 2014-08-06 15:32:33.423518 2014-08-13 18:06:06.889630 7f5c973a57a0 20 read_log 192718'15816 (192718'15815) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:241 2014-08-06 15:32:33.582206 2014-08-13 18:06:06.889643 7f5c973a57a0 20 read_log 192718'15817 (192718'15816) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:242 2014-08-06 15:32:33.611622 2014-08-13 18:06:06.889655 7f5c973a57a0 20 read_log 192718'15818 (192718'15817) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:243 2014-08-06 15:32:33.641234 2014-08-13 18:06:06.889667 7f5c973a57a0 20 read_log 192718'15819 (192718'15818) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:244 2014-08-06 15:32:33.676046 2014-08-13 18:06:06.889679 7f5c973a57a0 20 read_log 192718'15820 (192718'15819) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:245 2014-08-06 15:32:33.877051 2014-08-13 18:06:06.889691 7f5c973a57a0 20 read_log 192718'15821 (192718'15820) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:246 2014-08-06 15:32:34.206285 2014-08-13 18:06:06.889704 7f5c973a57a0 20 read_log 192718'15822 (192718'15821) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:247 2014-08-06 15:32:34.301523 2014-08-13 18:06:06.889716 7f5c973a57a0 20 read_log 192718'15823 (192718'15822) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:248 2014-08-06 15:32:34.332088 2014-08-13 18:06:06.889728 7f5c973a57a0 20 read_log 192718'15824 (192718'15823) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:249 2014-08-06 15:32:34.368011 2014-08-13 18:06:06.889740 7f5c973a57a0 20 read_log 192718'15825 (192718'15824) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:250 2014-08-06 15:32:34.419966 2014-08-13 18:06:06.889762 7f5c973a57a0 20 read_log 192718'15826 (192718'15825) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:251 2014-08-06 15:32:34.605399 2014-08-13 18:06:06.889783 7f5c973a57a0 20 read_log 192718'15827 (192718'15826) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:252 2014-08-06 15:32:34.639394 2014-08-13 18:06:06.889798 7f5c973a57a0 20 read_log 192718'15828 (192718'15827) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:253 2014-08-06 15:32:34.669433 2014-08-13 18:06:06.889811 7f5c973a57a0 20 read_log 192718'15829 (192718'15828) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:254 2014-08-06 15:32:34.702937 2014-08-13 18:06:06.889823 7f5c973a57a0 20 read_log 192718'15830 (192718'15829) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:255 2014-08-06 15:32:34.803300 2014-08-13 18:06:06.889835 7f5c973a57a0 20 read_log 192718'15831 (192718'15830) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:256 2014-08-06 15:32:35.255714 2014-08-13 18:06:06.889847 7f5c973a57a0 20 read_log 192718'15832 (192718'15831) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:257 2014-08-06 15:32:35.308846 2014-08-13 18:06:06.889859 7f5c973a57a0 20 read_log 192718'15833 (192718'15832) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:258 2014-08-06 15:32:35.366357 2014-08-13 18:06:06.889871 7f5c973a57a0 20 read_log 192718'15834 (192718'15833) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:259 2014-08-06 15:32:35.415661 2014-08-13 18:06:06.889883 7f5c973a57a0 20 read_log 192718'15835 (192718'15834) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:260 2014-08-06 15:32:35.464969 2014-08-13 18:06:06.889895 7f5c973a57a0 20 read_log 192718'15836 (192718'15835) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:261 2014-08-06 15:32:35.514187 2014-08-13 18:06:06.889907 7f5c973a57a0 20 read_log 192718'15837 (192718'15836) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:262 2014-08-06 15:32:35.568987 2014-08-13 18:06:06.889919 7f5c973a57a0 20 read_log 192718'15838 (192718'15837) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:263 2014-08-06 15:32:35.601145 2014-08-13 18:06:06.889931 7f5c973a57a0 20 read_log 192718'15839 (192718'15838) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:264 2014-08-06 15:32:35.668833 2014-08-13 18:06:06.889944 7f5c973a57a0 20 read_log 192718'15840 (192718'15839) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:265 2014-08-06 15:32:35.699950 2014-08-13 18:06:06.889973 7f5c973a57a0 20 read_log 192718'15841 (192718'15840) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:266 2014-08-06 15:32:35.969883 2014-08-13 18:06:06.889989 7f5c973a57a0 20 read_log 192718'15842 (192718'15841) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:267 2014-08-06 15:32:36.023444 2014-08-13 18:06:06.890001 7f5c973a57a0 20 read_log 192718'15843 (192718'15842) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:268 2014-08-06 15:32:36.080317 2014-08-13 18:06:06.890015 7f5c973a57a0 20 read_log 192718'15844 (192718'15843) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:269 2014-08-06 15:32:36.757327 2014-08-13 18:06:06.890029 7f5c973a57a0 20 read_log 192718'15845 (192718'15844) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:270 2014-08-06 15:32:36.785950 2014-08-13 18:06:06.890041 7f5c973a57a0 20 read_log 192718'15846 (192718'15845) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:271 2014-08-06 15:32:36.937618 2014-08-13 18:06:06.890054 7f5c973a57a0 20 read_log 192718'15847 (192718'15846) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:272 2014-08-06 15:32:36.985462 2014-08-13 18:06:06.890068 7f5c973a57a0 20 read_log 192718'15848 (192718'15847) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:273 2014-08-06 15:32:37.018827 2014-08-13 18:06:06.890080 7f5c973a57a0 20 read_log 192718'15849 (192718'15848) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:274 2014-08-06 15:32:37.049528 2014-08-13 18:06:06.890094 7f5c973a57a0 20 read_log 192718'15850 (192718'15849) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:275 2014-08-06 15:32:37.083157 2014-08-13 18:06:06.890115 7f5c973a57a0 20 read_log 192718'15851 (192718'15850) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:276 2014-08-06 15:32:37.258908 2014-08-13 18:06:06.890131 7f5c973a57a0 20 read_log 192718'15852 (192718'15851) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:277 2014-08-06 15:32:37.287544 2014-08-13 18:06:06.890143 7f5c973a57a0 20 read_log 192718'15853 (192718'15852) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:278 2014-08-06 15:32:37.316662 2014-08-13 18:06:06.890164 7f5c973a57a0 20 read_log 192718'15854 (192718'15853) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:279 2014-08-06 15:32:37.345721 2014-08-13 18:06:06.890176 7f5c973a57a0 20 read_log 192718'15855 (192718'15854) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:280 2014-08-06 15:32:37.373466 2014-08-13 18:06:06.890189 7f5c973a57a0 20 read_log 192718'15856 (192718'15855) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:281 2014-08-06 15:32:37.403193 2014-08-13 18:06:06.890203 7f5c973a57a0 20 read_log 192718'15857 (192718'15856) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:282 2014-08-06 15:32:37.432064 2014-08-13 18:06:06.890217 7f5c973a57a0 20 read_log 192718'15858 (192718'15857) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:283 2014-08-06 15:32:37.591303 2014-08-13 18:06:06.890230 7f5c973a57a0 20 read_log 192718'15859 (192718'15858) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:284 2014-08-06 15:32:37.621113 2014-08-13 18:06:06.890244 7f5c973a57a0 20 read_log 192718'15860 (192718'15859) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:285 2014-08-06 15:32:37.650187 2014-08-13 18:06:06.890256 7f5c973a57a0 20 read_log 192718'15861 (192718'15860) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:286 2014-08-06 15:32:37.677911 2014-08-13 18:06:06.890276 7f5c973a57a0 20 read_log 192718'15862 (192718'15861) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:287 2014-08-06 15:32:37.706028 2014-08-13 18:06:06.890297 7f5c973a57a0 20 read_log 192718'15863 (192718'15862) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:288 2014-08-06 15:32:37.735929 2014-08-13 18:06:06.890316 7f5c973a57a0 20 read_log 192718'15864 (192718'15863) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:289 2014-08-06 15:32:37.764668 2014-08-13 18:06:06.890332 7f5c973a57a0 20 read_log 192718'15865 (192718'15864) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:290 2014-08-06 15:32:37.795652 2014-08-13 18:06:06.890344 7f5c973a57a0 20 read_log 192718'15866 (192718'15865) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:291 2014-08-06 15:32:37.823011 2014-08-13 18:06:06.890356 7f5c973a57a0 20 read_log 192718'15867 (192718'15866) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:292 2014-08-06 15:32:37.854278 2014-08-13 18:06:06.890375 7f5c973a57a0 20 read_log 192718'15868 (192718'15867) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:293 2014-08-06 15:32:37.951764 2014-08-13 18:06:06.890387 7f5c973a57a0 20 read_log 192718'15869 (192718'15868) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:294 2014-08-06 15:32:37.986952 2014-08-13 18:06:06.890399 7f5c973a57a0 20 read_log 192718'15870 (192718'15869) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:295 2014-08-06 15:32:38.018236 2014-08-13 18:06:06.890416 7f5c973a57a0 20 read_log 192718'15871 (192718'15870) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:296 2014-08-06 15:32:38.067222 2014-08-13 18:06:06.890428 7f5c973a57a0 20 read_log 192718'15872 (192718'15871) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:297 2014-08-06 15:32:38.099743 2014-08-13 18:06:06.890441 7f5c973a57a0 20 read_log 192718'15873 (192718'15872) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:298 2014-08-06 15:32:38.151085 2014-08-13 18:06:06.890461 7f5c973a57a0 20 read_log 192718'15874 (192718'15873) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:299 2014-08-06 15:32:38.346945 2014-08-13 18:06:06.890483 7f5c973a57a0 20 read_log 192718'15875 (192718'15874) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:300 2014-08-06 15:32:38.378868 2014-08-13 18:06:06.890505 7f5c973a57a0 20 read_log 192718'15876 (192718'15875) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:301 2014-08-06 15:32:38.412130 2014-08-13 18:06:06.890521 7f5c973a57a0 20 read_log 192718'15877 (192718'15876) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:302 2014-08-06 15:32:38.444119 2014-08-13 18:06:06.890533 7f5c973a57a0 20 read_log 192718'15878 (192718'15877) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:303 2014-08-06 15:32:38.680306 2014-08-13 18:06:06.890552 7f5c973a57a0 20 read_log 192718'15879 (192718'15878) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:304 2014-08-06 15:32:38.717013 2014-08-13 18:06:06.890568 7f5c973a57a0 20 read_log 192718'15880 (192718'15879) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:305 2014-08-06 15:32:38.745250 2014-08-13 18:06:06.890581 7f5c973a57a0 20 read_log 192718'15881 (192718'15880) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:306 2014-08-06 15:32:38.778823 2014-08-13 18:06:06.890605 7f5c973a57a0 20 read_log 192718'15882 (192718'15881) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:307 2014-08-06 15:32:38.807525 2014-08-13 18:06:06.890621 7f5c973a57a0 20 read_log 192718'15883 (192718'15882) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:308 2014-08-06 15:32:38.985846 2014-08-13 18:06:06.890634 7f5c973a57a0 20 read_log 192718'15884 (192718'15883) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:309 2014-08-06 15:32:39.029032 2014-08-13 18:06:06.890646 7f5c973a57a0 20 read_log 192718'15885 (192718'15884) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:310 2014-08-06 15:32:39.058417 2014-08-13 18:06:06.890658 7f5c973a57a0 20 read_log 192718'15886 (192718'15885) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:311 2014-08-06 15:32:39.090498 2014-08-13 18:06:06.890670 7f5c973a57a0 20 read_log 192718'15887 (192718'15886) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:312 2014-08-06 15:32:39.127763 2014-08-13 18:06:06.890682 7f5c973a57a0 20 read_log 192718'15888 (192718'15887) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:313 2014-08-06 15:32:39.295909 2014-08-13 18:06:06.890694 7f5c973a57a0 20 read_log 192718'15889 (192718'15888) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:314 2014-08-06 15:32:39.325375 2014-08-13 18:06:06.890709 7f5c973a57a0 20 read_log 192718'15890 (192718'15889) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:315 2014-08-06 15:32:39.352790 2014-08-13 18:06:06.890721 7f5c973a57a0 20 read_log 192718'15891 (192718'15890) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:316 2014-08-06 15:32:39.380442 2014-08-13 18:06:06.890734 7f5c973a57a0 20 read_log 192718'15892 (192718'15891) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:317 2014-08-06 15:32:39.410129 2014-08-13 18:06:06.890754 7f5c973a57a0 20 read_log 192718'15893 (192718'15892) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:318 2014-08-06 15:32:39.574470 2014-08-13 18:06:06.890770 7f5c973a57a0 20 read_log 192718'15894 (192718'15893) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:319 2014-08-06 15:32:39.604596 2014-08-13 18:06:06.890783 7f5c973a57a0 20 read_log 192718'15895 (192718'15894) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:320 2014-08-06 15:32:39.643024 2014-08-13 18:06:06.890795 7f5c973a57a0 20 read_log 192718'15896 (192718'15895) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:321 2014-08-06 15:32:39.671328 2014-08-13 18:06:06.890807 7f5c973a57a0 20 read_log 192718'15897 (192718'15896) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:322 2014-08-06 15:32:39.700551 2014-08-13 18:06:06.890819 7f5c973a57a0 20 read_log 192718'15898 (192718'15897) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:323 2014-08-06 15:32:39.854787 2014-08-13 18:06:06.890838 7f5c973a57a0 20 read_log 192718'15899 (192718'15898) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:324 2014-08-06 15:32:39.883585 2014-08-13 18:06:06.890896 7f5c973a57a0 20 read_log 192718'15900 (192718'15899) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:325 2014-08-06 15:32:39.915875 2014-08-13 18:06:06.890912 7f5c973a57a0 20 read_log 192718'15901 (192718'15900) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:326 2014-08-06 15:32:40.081389 2014-08-13 18:06:06.890925 7f5c973a57a0 20 read_log 192718'15902 (192718'15901) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:327 2014-08-06 15:32:40.111398 2014-08-13 18:06:06.890937 7f5c973a57a0 20 read_log 192718'15903 (192718'15902) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:328 2014-08-06 15:32:40.146788 2014-08-13 18:06:06.890950 7f5c973a57a0 20 read_log 192718'15904 (192718'15903) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:329 2014-08-06 15:32:40.174377 2014-08-13 18:06:06.890962 7f5c973a57a0 20 read_log 192718'15905 (192718'15904) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:330 2014-08-06 15:32:40.201925 2014-08-13 18:06:06.890974 7f5c973a57a0 20 read_log 192718'15906 (192718'15905) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:331 2014-08-06 15:32:40.229458 2014-08-13 18:06:06.890986 7f5c973a57a0 20 read_log 192718'15907 (192718'15906) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:332 2014-08-06 15:32:40.684087 2014-08-13 18:06:06.891005 7f5c973a57a0 20 read_log 192718'15908 (192718'15907) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:333 2014-08-06 15:32:40.712206 2014-08-13 18:06:06.891017 7f5c973a57a0 20 read_log 192718'15909 (192718'15908) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:334 2014-08-06 15:32:40.740754 2014-08-13 18:06:06.891029 7f5c973a57a0 20 read_log 192718'15910 (192718'15909) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:335 2014-08-06 15:32:40.767189 2014-08-13 18:06:06.891044 7f5c973a57a0 20 read_log 192718'15911 (192718'15910) modify 168b030b/usr.tar2+63+2/head//24 by client.468086.0:336 2014-08-06 15:32:40.789228 2014-08-13 18:06:06.891059 7f5c973a57a0 20 read_log 192803'15912 (192718'15911) clean 168b030b/usr.tar2+63+2/head//24 by osd.329.0:17126 2014-08-11 10:25:18.791671 2014-08-13 18:06:06.891071 7f5c973a57a0 20 read_log 192803'15914 (192803'15912) delete 168b030b/usr.tar2+63+2/head//24 by osd.329.0:17128 2014-08-11 10:25:23.011486 2014-08-13 18:06:06.891086 7f5c973a57a0 20 read_log 192803'15920 (0'0) promote 38370f0b/core.31758+10+2/head//24 by osd.329.0:17308 2014-08-11 12:41:26.692787 2014-08-13 18:06:06.891099 7f5c973a57a0 20 read_log 192803'15921 (192803'15920) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:1 2014-08-11 12:41:14.584333 2014-08-13 18:06:06.891113 7f5c973a57a0 20 read_log 192803'15922 (192803'15921) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:2 2014-08-11 12:41:29.908617 2014-08-13 18:06:06.891125 7f5c973a57a0 20 read_log 192803'15923 (192803'15922) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:3 2014-08-11 12:41:30.206228 2014-08-13 18:06:06.891137 7f5c973a57a0 20 read_log 192803'15924 (192803'15923) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:4 2014-08-11 12:41:30.234759 2014-08-13 18:06:06.891155 7f5c973a57a0 20 read_log 192803'15925 (192803'15924) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:5 2014-08-11 12:41:30.275220 2014-08-13 18:06:06.891170 7f5c973a57a0 20 read_log 192803'15926 (192803'15925) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:6 2014-08-11 12:41:30.301367 2014-08-13 18:06:06.891182 7f5c973a57a0 20 read_log 192803'15927 (192803'15926) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:7 2014-08-11 12:41:30.332589 2014-08-13 18:06:06.891196 7f5c973a57a0 20 read_log 192803'15928 (192803'15927) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:8 2014-08-11 12:41:30.372722 2014-08-13 18:06:06.891215 7f5c973a57a0 20 read_log 192803'15929 (192803'15928) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:9 2014-08-11 12:41:30.446094 2014-08-13 18:06:06.891228 7f5c973a57a0 20 read_log 192803'15930 (192803'15929) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:10 2014-08-11 12:41:30.484689 2014-08-13 18:06:06.891240 7f5c973a57a0 20 read_log 192803'15931 (192803'15930) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:11 2014-08-11 12:41:30.524245 2014-08-13 18:06:06.891252 7f5c973a57a0 20 read_log 192803'15932 (192803'15931) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:12 2014-08-11 12:41:30.563772 2014-08-13 18:06:06.891268 7f5c973a57a0 20 read_log 192803'15933 (192803'15932) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:13 2014-08-11 12:41:30.602676 2014-08-13 18:06:06.891281 7f5c973a57a0 20 read_log 192803'15934 (192803'15933) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:14 2014-08-11 12:41:31.031106 2014-08-13 18:06:06.891293 7f5c973a57a0 20 read_log 192803'15935 (192803'15934) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:15 2014-08-11 12:41:31.069165 2014-08-13 18:06:06.891305 7f5c973a57a0 20 read_log 192803'15936 (192803'15935) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:16 2014-08-11 12:41:31.113055 2014-08-13 18:06:06.891326 7f5c973a57a0 20 read_log 192803'15937 (192803'15936) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:17 2014-08-11 12:41:31.158425 2014-08-13 18:06:06.891338 7f5c973a57a0 20 read_log 192803'15938 (192803'15937) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:18 2014-08-11 12:41:31.190003 2014-08-13 18:06:06.891350 7f5c973a57a0 20 read_log 192803'15939 (192803'15938) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:19 2014-08-11 12:41:31.228294 2014-08-13 18:06:06.891369 7f5c973a57a0 20 read_log 192803'15940 (192803'15939) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:20 2014-08-11 12:41:31.267155 2014-08-13 18:06:06.891382 7f5c973a57a0 20 read_log 192803'15941 (192803'15940) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:21 2014-08-11 12:41:31.305578 2014-08-13 18:06:06.891394 7f5c973a57a0 20 read_log 192803'15942 (192803'15941) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:22 2014-08-11 12:41:31.731234 2014-08-13 18:06:06.891425 7f5c973a57a0 20 read_log 192803'15943 (192803'15942) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:23 2014-08-11 12:41:31.769503 2014-08-13 18:06:06.891438 7f5c973a57a0 20 read_log 192803'15944 (192803'15943) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:24 2014-08-11 12:41:31.806968 2014-08-13 18:06:06.891457 7f5c973a57a0 20 read_log 192803'15945 (192803'15944) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:25 2014-08-11 12:41:31.843653 2014-08-13 18:06:06.891471 7f5c973a57a0 20 read_log 192803'15946 (192803'15945) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:26 2014-08-11 12:41:31.888164 2014-08-13 18:06:06.891483 7f5c973a57a0 20 read_log 192803'15947 (192803'15946) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:27 2014-08-11 12:41:31.924983 2014-08-13 18:06:06.891501 7f5c973a57a0 20 read_log 192803'15948 (192803'15947) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:28 2014-08-11 12:41:31.965521 2014-08-13 18:06:06.891515 7f5c973a57a0 20 read_log 192803'15949 (192803'15948) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:29 2014-08-11 12:41:32.003940 2014-08-13 18:06:06.891530 7f5c973a57a0 20 read_log 192803'15950 (192803'15949) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:30 2014-08-11 12:41:32.043532 2014-08-13 18:06:06.891542 7f5c973a57a0 20 read_log 192803'15951 (192803'15950) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:31 2014-08-11 12:41:32.087114 2014-08-13 18:06:06.891554 7f5c973a57a0 20 read_log 192803'15952 (192803'15951) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:32 2014-08-11 12:41:32.519446 2014-08-13 18:06:06.891597 7f5c973a57a0 20 read_log 192803'15953 (192803'15952) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:33 2014-08-11 12:41:32.603633 2014-08-13 18:06:06.891611 7f5c973a57a0 20 read_log 192803'15954 (192803'15953) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:34 2014-08-11 12:41:32.652710 2014-08-13 18:06:06.891624 7f5c973a57a0 20 read_log 192803'15955 (192803'15954) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:35 2014-08-11 12:41:32.688000 2014-08-13 18:06:06.891636 7f5c973a57a0 20 read_log 192803'15956 (192803'15955) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:36 2014-08-11 12:41:32.727064 2014-08-13 18:06:06.891658 7f5c973a57a0 20 read_log 192803'15957 (192803'15956) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:37 2014-08-11 12:41:32.767510 2014-08-13 18:06:06.891672 7f5c973a57a0 20 read_log 192803'15958 (192803'15957) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:38 2014-08-11 12:41:32.807719 2014-08-13 18:06:06.891684 7f5c973a57a0 20 read_log 192803'15959 (192803'15958) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:39 2014-08-11 12:41:32.849469 2014-08-13 18:06:06.891696 7f5c973a57a0 20 read_log 192803'15960 (192803'15959) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:40 2014-08-11 12:41:32.888526 2014-08-13 18:06:06.891709 7f5c973a57a0 20 read_log 192803'15961 (192803'15960) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:41 2014-08-11 12:41:32.928690 2014-08-13 18:06:06.891721 7f5c973a57a0 20 read_log 192803'15962 (192803'15961) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:42 2014-08-11 12:41:32.967777 2014-08-13 18:06:06.891733 7f5c973a57a0 20 read_log 192803'15963 (192803'15962) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:43 2014-08-11 12:41:33.006951 2014-08-13 18:06:06.891745 7f5c973a57a0 20 read_log 192803'15964 (192803'15963) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:44 2014-08-11 12:41:33.479763 2014-08-13 18:06:06.891757 7f5c973a57a0 20 read_log 192803'15965 (192803'15964) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:45 2014-08-11 12:41:33.523917 2014-08-13 18:06:06.891769 7f5c973a57a0 20 read_log 192803'15966 (192803'15965) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:46 2014-08-11 12:41:33.564443 2014-08-13 18:06:06.891781 7f5c973a57a0 20 read_log 192803'15967 (192803'15966) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:47 2014-08-11 12:41:33.603004 2014-08-13 18:06:06.891793 7f5c973a57a0 20 read_log 192803'15968 (192803'15967) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:48 2014-08-11 12:41:33.642330 2014-08-13 18:06:06.891805 7f5c973a57a0 20 read_log 192803'15969 (192803'15968) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:49 2014-08-11 12:41:33.680993 2014-08-13 18:06:06.891817 7f5c973a57a0 20 read_log 192803'15970 (192803'15969) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:50 2014-08-11 12:41:33.718934 2014-08-13 18:06:06.891829 7f5c973a57a0 20 read_log 192803'15971 (192803'15970) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:51 2014-08-11 12:41:33.760369 2014-08-13 18:06:06.891841 7f5c973a57a0 20 read_log 192803'15972 (192803'15971) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:52 2014-08-11 12:41:33.800733 2014-08-13 18:06:06.891854 7f5c973a57a0 20 read_log 192803'15973 (192803'15972) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:53 2014-08-11 12:41:33.839582 2014-08-13 18:06:06.891867 7f5c973a57a0 20 read_log 192803'15974 (192803'15973) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:54 2014-08-11 12:41:34.280119 2014-08-13 18:06:06.891879 7f5c973a57a0 20 read_log 192803'15975 (192803'15974) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:55 2014-08-11 12:41:34.330195 2014-08-13 18:06:06.891891 7f5c973a57a0 20 read_log 192803'15976 (192803'15975) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:56 2014-08-11 12:41:34.372370 2014-08-13 18:06:06.891910 7f5c973a57a0 20 read_log 192803'15977 (192803'15976) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:57 2014-08-11 12:41:34.412854 2014-08-13 18:06:06.891925 7f5c973a57a0 20 read_log 192803'15978 (192803'15977) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:58 2014-08-11 12:41:34.454883 2014-08-13 18:06:06.891938 7f5c973a57a0 20 read_log 192803'15979 (192803'15978) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:59 2014-08-11 12:41:34.496574 2014-08-13 18:06:06.891952 7f5c973a57a0 20 read_log 192803'15980 (192803'15979) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:60 2014-08-11 12:41:34.538729 2014-08-13 18:06:06.891964 7f5c973a57a0 20 read_log 192803'15981 (192803'15980) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:61 2014-08-11 12:41:34.581158 2014-08-13 18:06:06.891978 7f5c973a57a0 20 read_log 192803'15982 (192803'15981) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:62 2014-08-11 12:41:34.622644 2014-08-13 18:06:06.891991 7f5c973a57a0 20 read_log 192803'15983 (192803'15982) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:63 2014-08-11 12:41:34.663544 2014-08-13 18:06:06.892005 7f5c973a57a0 20 read_log 192803'15984 (192803'15983) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:64 2014-08-11 12:41:35.123383 2014-08-13 18:06:06.892017 7f5c973a57a0 20 read_log 192803'15985 (192803'15984) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:65 2014-08-11 12:41:35.164421 2014-08-13 18:06:06.892031 7f5c973a57a0 20 read_log 192803'15986 (192803'15985) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:66 2014-08-11 12:41:35.209219 2014-08-13 18:06:06.892043 7f5c973a57a0 20 read_log 192803'15987 (192803'15986) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:67 2014-08-11 12:41:35.237741 2014-08-13 18:06:06.892057 7f5c973a57a0 20 read_log 192803'15988 (192803'15987) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:68 2014-08-11 12:41:35.263654 2014-08-13 18:06:06.892069 7f5c973a57a0 20 read_log 192803'15989 (192803'15988) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:69 2014-08-11 12:41:35.291033 2014-08-13 18:06:06.892083 7f5c973a57a0 20 read_log 192803'15990 (192803'15989) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:70 2014-08-11 12:41:35.335372 2014-08-13 18:06:06.892095 7f5c973a57a0 20 read_log 192803'15991 (192803'15990) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:71 2014-08-11 12:41:35.362239 2014-08-13 18:06:06.892109 7f5c973a57a0 20 read_log 192803'15992 (192803'15991) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:72 2014-08-11 12:41:35.387605 2014-08-13 18:06:06.892122 7f5c973a57a0 20 read_log 192803'15993 (192803'15992) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:73 2014-08-11 12:41:35.412993 2014-08-13 18:06:06.892136 7f5c973a57a0 20 read_log 192803'15994 (192803'15993) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:74 2014-08-11 12:41:35.702376 2014-08-13 18:06:06.892154 7f5c973a57a0 20 read_log 192803'15995 (192803'15994) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:75 2014-08-11 12:41:35.734449 2014-08-13 18:06:06.892166 7f5c973a57a0 20 read_log 192803'15996 (192803'15995) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:76 2014-08-11 12:41:35.761356 2014-08-13 18:06:06.892179 7f5c973a57a0 20 read_log 192803'15997 (192803'15996) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:77 2014-08-11 12:41:35.788815 2014-08-13 18:06:06.892191 7f5c973a57a0 20 read_log 192803'15998 (192803'15997) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:78 2014-08-11 12:41:35.816034 2014-08-13 18:06:06.892206 7f5c973a57a0 20 read_log 192803'15999 (192803'15998) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:79 2014-08-11 12:41:35.843437 2014-08-13 18:06:06.892220 7f5c973a57a0 20 read_log 192803'16000 (192803'15999) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:80 2014-08-11 12:41:35.870187 2014-08-13 18:06:06.892239 7f5c973a57a0 20 read_log 192803'16001 (192803'16000) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:81 2014-08-11 12:41:35.898619 2014-08-13 18:06:06.892253 7f5c973a57a0 20 read_log 192803'16002 (192803'16001) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:82 2014-08-11 12:41:35.926316 2014-08-13 18:06:06.892268 7f5c973a57a0 20 read_log 192803'16003 (192803'16002) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:83 2014-08-11 12:41:35.953374 2014-08-13 18:06:06.892282 7f5c973a57a0 20 read_log 192803'16004 (192803'16003) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:84 2014-08-11 12:41:36.255486 2014-08-13 18:06:06.892294 7f5c973a57a0 20 read_log 192803'16005 (192803'16004) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:85 2014-08-11 12:41:36.286134 2014-08-13 18:06:06.892306 7f5c973a57a0 20 read_log 192803'16006 (192803'16005) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:86 2014-08-11 12:41:36.314678 2014-08-13 18:06:06.892324 7f5c973a57a0 20 read_log 192803'16007 (192803'16006) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:87 2014-08-11 12:41:36.341156 2014-08-13 18:06:06.892336 7f5c973a57a0 20 read_log 192803'16008 (192803'16007) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:88 2014-08-11 12:41:36.368535 2014-08-13 18:06:06.892348 7f5c973a57a0 20 read_log 192803'16009 (192803'16008) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:89 2014-08-11 12:41:36.395850 2014-08-13 18:06:06.892364 7f5c973a57a0 20 read_log 192803'16010 (192803'16009) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:90 2014-08-11 12:41:36.423477 2014-08-13 18:06:06.892378 7f5c973a57a0 20 read_log 192803'16011 (192803'16010) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:91 2014-08-11 12:41:36.450384 2014-08-13 18:06:06.892398 7f5c973a57a0 20 read_log 192803'16012 (192803'16011) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:92 2014-08-11 12:41:36.477995 2014-08-13 18:06:06.892420 7f5c973a57a0 20 read_log 192803'16013 (192803'16012) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:93 2014-08-11 12:41:36.506294 2014-08-13 18:06:06.892432 7f5c973a57a0 20 read_log 192803'16014 (192803'16013) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:94 2014-08-11 12:41:36.808809 2014-08-13 18:06:06.892445 7f5c973a57a0 20 read_log 192803'16015 (192803'16014) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:95 2014-08-11 12:41:36.835716 2014-08-13 18:06:06.892464 7f5c973a57a0 20 read_log 192803'16016 (192803'16015) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:96 2014-08-11 12:41:36.862845 2014-08-13 18:06:06.892480 7f5c973a57a0 20 read_log 192803'16017 (192803'16016) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:97 2014-08-11 12:41:36.891487 2014-08-13 18:06:06.892493 7f5c973a57a0 20 read_log 192803'16018 (192803'16017) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:98 2014-08-11 12:41:36.918330 2014-08-13 18:06:06.892532 7f5c973a57a0 20 read_log 192803'16019 (192803'16018) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:99 2014-08-11 12:41:36.942066 2014-08-13 18:06:06.892548 7f5c973a57a0 20 read_log 192803'16020 (192803'16019) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:100 2014-08-11 12:41:36.969858 2014-08-13 18:06:06.892570 7f5c973a57a0 20 read_log 192803'16021 (192803'16020) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:101 2014-08-11 12:41:36.997621 2014-08-13 18:06:06.892583 7f5c973a57a0 20 read_log 192803'16022 (192803'16021) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:102 2014-08-11 12:41:37.027554 2014-08-13 18:06:06.892602 7f5c973a57a0 20 read_log 192803'16023 (192803'16022) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:103 2014-08-11 12:41:37.054344 2014-08-13 18:06:06.892615 7f5c973a57a0 20 read_log 192803'16024 (192803'16023) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:104 2014-08-11 12:41:37.353887 2014-08-13 18:06:06.892636 7f5c973a57a0 20 read_log 192803'16025 (192803'16024) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:105 2014-08-11 12:41:37.384777 2014-08-13 18:06:06.892651 7f5c973a57a0 20 read_log 192803'16026 (192803'16025) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:106 2014-08-11 12:41:37.422262 2014-08-13 18:06:06.892663 7f5c973a57a0 20 read_log 192803'16027 (192803'16026) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:107 2014-08-11 12:41:37.474103 2014-08-13 18:06:06.892676 7f5c973a57a0 20 read_log 192803'16028 (192803'16027) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:108 2014-08-11 12:41:37.525652 2014-08-13 18:06:06.892688 7f5c973a57a0 20 read_log 192803'16029 (192803'16028) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:109 2014-08-11 12:41:37.569239 2014-08-13 18:06:06.892700 7f5c973a57a0 20 read_log 192803'16030 (192803'16029) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:110 2014-08-11 12:41:37.621145 2014-08-13 18:06:06.892712 7f5c973a57a0 20 read_log 192803'16031 (192803'16030) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:111 2014-08-11 12:41:37.747796 2014-08-13 18:06:06.892724 7f5c973a57a0 20 read_log 192803'16032 (192803'16031) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:112 2014-08-11 12:41:37.816216 2014-08-13 18:06:06.892736 7f5c973a57a0 20 read_log 192803'16033 (192803'16032) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:113 2014-08-11 12:41:37.859471 2014-08-13 18:06:06.892747 7f5c973a57a0 20 read_log 192803'16034 (192803'16033) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:114 2014-08-11 12:41:37.903006 2014-08-13 18:06:06.892760 7f5c973a57a0 20 read_log 192803'16035 (192803'16034) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:115 2014-08-11 12:41:37.946699 2014-08-13 18:06:06.892772 7f5c973a57a0 20 read_log 192803'16036 (192803'16035) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:116 2014-08-11 12:41:37.990123 2014-08-13 18:06:06.892784 7f5c973a57a0 20 read_log 192803'16037 (192803'16036) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:117 2014-08-11 12:41:38.033317 2014-08-13 18:06:06.892796 7f5c973a57a0 20 read_log 192803'16038 (192803'16037) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:118 2014-08-11 12:41:38.077117 2014-08-13 18:06:06.892808 7f5c973a57a0 20 read_log 192803'16039 (192803'16038) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:119 2014-08-11 12:41:38.128929 2014-08-13 18:06:06.892820 7f5c973a57a0 20 read_log 192803'16040 (192803'16039) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:120 2014-08-11 12:41:38.172172 2014-08-13 18:06:06.892832 7f5c973a57a0 20 read_log 192803'16041 (192803'16040) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:121 2014-08-11 12:41:38.214622 2014-08-13 18:06:06.892844 7f5c973a57a0 20 read_log 192803'16042 (192803'16041) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:122 2014-08-11 12:41:38.367462 2014-08-13 18:06:06.892856 7f5c973a57a0 20 read_log 192803'16043 (192803'16042) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:123 2014-08-11 12:41:38.410656 2014-08-13 18:06:06.892867 7f5c973a57a0 20 read_log 192803'16044 (192803'16043) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:124 2014-08-11 12:41:38.454365 2014-08-13 18:06:06.892879 7f5c973a57a0 20 read_log 192803'16045 (192803'16044) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:125 2014-08-11 12:41:38.497818 2014-08-13 18:06:06.892891 7f5c973a57a0 20 read_log 192803'16046 (192803'16045) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:126 2014-08-11 12:41:38.541902 2014-08-13 18:06:06.892903 7f5c973a57a0 20 read_log 192803'16047 (192803'16046) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:127 2014-08-11 12:41:38.585148 2014-08-13 18:06:06.892915 7f5c973a57a0 20 read_log 192803'16048 (192803'16047) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:128 2014-08-11 12:41:38.631007 2014-08-13 18:06:06.892933 7f5c973a57a0 20 read_log 192803'16049 (192803'16048) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:129 2014-08-11 12:41:38.674525 2014-08-13 18:06:06.892948 7f5c973a57a0 20 read_log 192803'16050 (192803'16049) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:130 2014-08-11 12:41:38.716831 2014-08-13 18:06:06.892960 7f5c973a57a0 20 read_log 192803'16051 (192803'16050) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:131 2014-08-11 12:41:38.753143 2014-08-13 18:06:06.892972 7f5c973a57a0 20 read_log 192803'16052 (192803'16051) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:132 2014-08-11 12:41:38.904926 2014-08-13 18:06:06.892986 7f5c973a57a0 20 read_log 192803'16053 (192803'16052) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:133 2014-08-11 12:41:38.948182 2014-08-13 18:06:06.892999 7f5c973a57a0 20 read_log 192803'16054 (192803'16053) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:134 2014-08-11 12:41:38.991799 2014-08-13 18:06:06.893013 7f5c973a57a0 20 read_log 192803'16055 (192803'16054) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:135 2014-08-11 12:41:39.035322 2014-08-13 18:06:06.893030 7f5c973a57a0 20 read_log 192803'16056 (192803'16055) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:136 2014-08-11 12:41:39.078760 2014-08-13 18:06:06.893044 7f5c973a57a0 20 read_log 192803'16057 (192803'16056) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:137 2014-08-11 12:41:39.122100 2014-08-13 18:06:06.893057 7f5c973a57a0 20 read_log 192803'16058 (192803'16057) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:138 2014-08-11 12:41:39.165436 2014-08-13 18:06:06.893071 7f5c973a57a0 20 read_log 192803'16059 (192803'16058) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:139 2014-08-11 12:41:39.206373 2014-08-13 18:06:06.893083 7f5c973a57a0 20 read_log 192803'16060 (192803'16059) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:140 2014-08-11 12:41:39.250283 2014-08-13 18:06:06.893097 7f5c973a57a0 20 read_log 192803'16061 (192803'16060) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:141 2014-08-11 12:41:39.293431 2014-08-13 18:06:06.893109 7f5c973a57a0 20 read_log 192803'16062 (192803'16061) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:142 2014-08-11 12:41:39.436755 2014-08-13 18:06:06.893121 7f5c973a57a0 20 read_log 192803'16063 (192803'16062) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:143 2014-08-11 12:41:39.482931 2014-08-13 18:06:06.893135 7f5c973a57a0 20 read_log 192803'16064 (192803'16063) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:144 2014-08-11 12:41:39.534794 2014-08-13 18:06:06.893156 7f5c973a57a0 20 read_log 192803'16065 (192803'16064) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:145 2014-08-11 12:41:39.578946 2014-08-13 18:06:06.893172 7f5c973a57a0 20 read_log 192803'16066 (192803'16065) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:146 2014-08-11 12:41:39.630158 2014-08-13 18:06:06.893184 7f5c973a57a0 20 read_log 192803'16067 (192803'16066) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:147 2014-08-11 12:41:39.673657 2014-08-13 18:06:06.893198 7f5c973a57a0 20 read_log 192803'16068 (192803'16067) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:148 2014-08-11 12:41:39.715631 2014-08-13 18:06:06.893210 7f5c973a57a0 20 read_log 192803'16069 (192803'16068) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:149 2014-08-11 12:41:39.758283 2014-08-13 18:06:06.893224 7f5c973a57a0 20 read_log 192803'16070 (192803'16069) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:150 2014-08-11 12:41:39.801822 2014-08-13 18:06:06.893237 7f5c973a57a0 20 read_log 192803'16071 (192803'16070) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:151 2014-08-11 12:41:39.845393 2014-08-13 18:06:06.893250 7f5c973a57a0 20 read_log 192803'16072 (192803'16071) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:152 2014-08-11 12:41:39.957475 2014-08-13 18:06:06.893272 7f5c973a57a0 20 read_log 192803'16073 (192803'16072) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:153 2014-08-11 12:41:40.000556 2014-08-13 18:06:06.893287 7f5c973a57a0 20 read_log 192803'16074 (192803'16073) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:154 2014-08-11 12:41:40.052829 2014-08-13 18:06:06.893300 7f5c973a57a0 20 read_log 192803'16075 (192803'16074) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:155 2014-08-11 12:41:40.095123 2014-08-13 18:06:06.893315 7f5c973a57a0 20 read_log 192803'16076 (192803'16075) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:156 2014-08-11 12:41:40.139972 2014-08-13 18:06:06.893328 7f5c973a57a0 20 read_log 192803'16077 (192803'16076) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:157 2014-08-11 12:41:40.182159 2014-08-13 18:06:06.893340 7f5c973a57a0 20 read_log 192803'16078 (192803'16077) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:158 2014-08-11 12:41:40.227043 2014-08-13 18:06:06.893352 7f5c973a57a0 20 read_log 192803'16079 (192803'16078) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:159 2014-08-11 12:41:40.272321 2014-08-13 18:06:06.893367 7f5c973a57a0 20 read_log 192803'16080 (192803'16079) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:160 2014-08-11 12:41:40.315953 2014-08-13 18:06:06.893382 7f5c973a57a0 20 read_log 192803'16081 (192803'16080) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:161 2014-08-11 12:41:40.359539 2014-08-13 18:06:06.893394 7f5c973a57a0 20 read_log 192803'16082 (192803'16081) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:162 2014-08-11 12:41:40.486508 2014-08-13 18:06:06.893422 7f5c973a57a0 20 read_log 192803'16083 (192803'16082) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:163 2014-08-11 12:41:40.530085 2014-08-13 18:06:06.893467 7f5c973a57a0 20 read_log 192803'16084 (192803'16083) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:164 2014-08-11 12:41:40.571413 2014-08-13 18:06:06.893480 7f5c973a57a0 20 read_log 192803'16085 (192803'16084) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:165 2014-08-11 12:41:40.618113 2014-08-13 18:06:06.893494 7f5c973a57a0 20 read_log 192803'16086 (192803'16085) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:166 2014-08-11 12:41:40.662109 2014-08-13 18:06:06.893509 7f5c973a57a0 20 read_log 192803'16087 (192803'16086) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:167 2014-08-11 12:41:40.713882 2014-08-13 18:06:06.893522 7f5c973a57a0 20 read_log 192803'16088 (192803'16087) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:168 2014-08-11 12:41:40.757398 2014-08-13 18:06:06.893534 7f5c973a57a0 20 read_log 192803'16089 (192803'16088) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:169 2014-08-11 12:41:40.801054 2014-08-13 18:06:06.893555 7f5c973a57a0 20 read_log 192803'16090 (192803'16089) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:170 2014-08-11 12:41:40.844869 2014-08-13 18:06:06.893568 7f5c973a57a0 20 read_log 192803'16091 (192803'16090) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:171 2014-08-11 12:41:40.905155 2014-08-13 18:06:06.893580 7f5c973a57a0 20 read_log 192803'16092 (192803'16091) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:172 2014-08-11 12:41:41.207003 2014-08-13 18:06:06.893593 7f5c973a57a0 20 read_log 192803'16093 (192803'16092) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:173 2014-08-11 12:41:41.259143 2014-08-13 18:06:06.893611 7f5c973a57a0 20 read_log 192803'16094 (192803'16093) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:174 2014-08-11 12:41:41.311301 2014-08-13 18:06:06.893623 7f5c973a57a0 20 read_log 192803'16095 (192803'16094) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:175 2014-08-11 12:41:41.356954 2014-08-13 18:06:06.893636 7f5c973a57a0 20 read_log 192803'16096 (192803'16095) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:176 2014-08-11 12:41:41.382747 2014-08-13 18:06:06.893657 7f5c973a57a0 20 read_log 192803'16097 (192803'16096) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:177 2014-08-11 12:41:41.409218 2014-08-13 18:06:06.893672 7f5c973a57a0 20 read_log 192803'16098 (192803'16097) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:178 2014-08-11 12:41:41.460532 2014-08-13 18:06:06.893684 7f5c973a57a0 20 read_log 192803'16099 (192803'16098) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:179 2014-08-11 12:41:41.504376 2014-08-13 18:06:06.893697 7f5c973a57a0 20 read_log 192803'16100 (192803'16099) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:180 2014-08-11 12:41:41.565073 2014-08-13 18:06:06.893709 7f5c973a57a0 20 read_log 192803'16101 (192803'16100) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:181 2014-08-11 12:41:41.608844 2014-08-13 18:06:06.893721 7f5c973a57a0 20 read_log 192803'16102 (192803'16101) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:182 2014-08-11 12:41:42.035600 2014-08-13 18:06:06.893733 7f5c973a57a0 20 read_log 192803'16103 (192803'16102) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:183 2014-08-11 12:41:42.083272 2014-08-13 18:06:06.893745 7f5c973a57a0 20 read_log 192803'16104 (192803'16103) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:184 2014-08-11 12:41:42.135249 2014-08-13 18:06:06.893757 7f5c973a57a0 20 read_log 192803'16105 (192803'16104) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:185 2014-08-11 12:41:42.187248 2014-08-13 18:06:06.893769 7f5c973a57a0 20 read_log 192803'16106 (192803'16105) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:186 2014-08-11 12:41:42.239144 2014-08-13 18:06:06.893780 7f5c973a57a0 20 read_log 192803'16107 (192803'16106) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:187 2014-08-11 12:41:42.291123 2014-08-13 18:06:06.893792 7f5c973a57a0 20 read_log 192803'16108 (192803'16107) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:188 2014-08-11 12:41:42.343167 2014-08-13 18:06:06.893804 7f5c973a57a0 20 read_log 192803'16109 (192803'16108) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:189 2014-08-11 12:41:42.395154 2014-08-13 18:06:06.893817 7f5c973a57a0 20 read_log 192803'16110 (192803'16109) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:190 2014-08-11 12:41:42.447125 2014-08-13 18:06:06.893829 7f5c973a57a0 20 read_log 192803'16111 (192803'16110) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:191 2014-08-11 12:41:42.499126 2014-08-13 18:06:06.893857 7f5c973a57a0 20 read_log 192803'16112 (192803'16111) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:192 2014-08-11 12:41:42.892760 2014-08-13 18:06:06.893875 7f5c973a57a0 20 read_log 192803'16113 (192803'16112) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:193 2014-08-11 12:41:43.020052 2014-08-13 18:06:06.893887 7f5c973a57a0 20 read_log 192803'16114 (192803'16113) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:194 2014-08-11 12:41:43.069794 2014-08-13 18:06:06.893900 7f5c973a57a0 20 read_log 192803'16115 (192803'16114) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:195 2014-08-11 12:41:43.107350 2014-08-13 18:06:06.893912 7f5c973a57a0 20 read_log 192803'16116 (192803'16115) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:196 2014-08-11 12:41:43.153469 2014-08-13 18:06:06.893926 7f5c973a57a0 20 read_log 192803'16117 (192803'16116) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:197 2014-08-11 12:41:43.197216 2014-08-13 18:06:06.893938 7f5c973a57a0 20 read_log 192803'16118 (192803'16117) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:198 2014-08-11 12:41:43.234679 2014-08-13 18:06:06.893952 7f5c973a57a0 20 read_log 192803'16119 (192803'16118) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:199 2014-08-11 12:41:43.260504 2014-08-13 18:06:06.893965 7f5c973a57a0 20 read_log 192803'16120 (192803'16119) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:200 2014-08-11 12:41:43.292357 2014-08-13 18:06:06.894004 7f5c973a57a0 20 read_log 192803'16121 (192803'16120) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:201 2014-08-11 12:41:43.333026 2014-08-13 18:06:06.894020 7f5c973a57a0 20 read_log 192803'16122 (192803'16121) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:202 2014-08-11 12:41:43.373258 2014-08-13 18:06:06.894033 7f5c973a57a0 20 read_log 192803'16123 (192803'16122) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:203 2014-08-11 12:41:43.416124 2014-08-13 18:06:06.894047 7f5c973a57a0 20 read_log 192803'16124 (192803'16123) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:204 2014-08-11 12:41:43.906477 2014-08-13 18:06:06.894059 7f5c973a57a0 20 read_log 192803'16125 (192803'16124) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:205 2014-08-11 12:41:43.947390 2014-08-13 18:06:06.894073 7f5c973a57a0 20 read_log 192803'16126 (192803'16125) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:206 2014-08-11 12:41:43.988949 2014-08-13 18:06:06.894086 7f5c973a57a0 20 read_log 192803'16127 (192803'16126) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:207 2014-08-11 12:41:44.030022 2014-08-13 18:06:06.894100 7f5c973a57a0 20 read_log 192803'16128 (192803'16127) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:208 2014-08-11 12:41:44.075665 2014-08-13 18:06:06.894112 7f5c973a57a0 20 read_log 192803'16129 (192803'16128) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:209 2014-08-11 12:41:44.111043 2014-08-13 18:06:06.894126 7f5c973a57a0 20 read_log 192803'16130 (192803'16129) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:210 2014-08-11 12:41:44.152130 2014-08-13 18:06:06.894139 7f5c973a57a0 20 read_log 192803'16131 (192803'16130) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:211 2014-08-11 12:41:44.192980 2014-08-13 18:06:06.894163 7f5c973a57a0 20 read_log 192803'16132 (192803'16131) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:212 2014-08-11 12:41:44.234677 2014-08-13 18:06:06.894178 7f5c973a57a0 20 read_log 192803'16133 (192803'16132) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:213 2014-08-11 12:41:44.281226 2014-08-13 18:06:06.894190 7f5c973a57a0 20 read_log 192803'16134 (192803'16133) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:214 2014-08-11 12:41:44.736089 2014-08-13 18:06:06.894205 7f5c973a57a0 20 read_log 192803'16135 (192803'16134) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:215 2014-08-11 12:41:44.782843 2014-08-13 18:06:06.894217 7f5c973a57a0 20 read_log 192803'16136 (192803'16135) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:216 2014-08-11 12:41:44.822866 2014-08-13 18:06:06.894229 7f5c973a57a0 20 read_log 192803'16137 (192803'16136) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:217 2014-08-11 12:41:44.863865 2014-08-13 18:06:06.894241 7f5c973a57a0 20 read_log 192803'16138 (192803'16137) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:218 2014-08-11 12:41:44.907758 2014-08-13 18:06:06.894253 7f5c973a57a0 20 read_log 192803'16139 (192803'16138) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:219 2014-08-11 12:41:44.946272 2014-08-13 18:06:06.894271 7f5c973a57a0 20 read_log 192803'16140 (192803'16139) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:220 2014-08-11 12:41:44.987150 2014-08-13 18:06:06.894283 7f5c973a57a0 20 read_log 192803'16141 (192803'16140) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:221 2014-08-11 12:41:45.028292 2014-08-13 18:06:06.894295 7f5c973a57a0 20 read_log 192803'16142 (192803'16141) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:222 2014-08-11 12:41:45.064875 2014-08-13 18:06:06.894310 7f5c973a57a0 20 read_log 192803'16143 (192803'16142) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:223 2014-08-11 12:41:45.111755 2014-08-13 18:06:06.894322 7f5c973a57a0 20 read_log 192803'16144 (192803'16143) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:224 2014-08-11 12:41:45.565107 2014-08-13 18:06:06.894341 7f5c973a57a0 20 read_log 192803'16145 (192803'16144) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:225 2014-08-11 12:41:45.614779 2014-08-13 18:06:06.894360 7f5c973a57a0 20 read_log 192803'16146 (192803'16145) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:226 2014-08-11 12:41:45.656016 2014-08-13 18:06:06.894377 7f5c973a57a0 20 read_log 192803'16147 (192803'16146) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:227 2014-08-11 12:41:45.698152 2014-08-13 18:06:06.894390 7f5c973a57a0 20 read_log 192803'16148 (192803'16147) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:228 2014-08-11 12:41:45.746862 2014-08-13 18:06:06.894402 7f5c973a57a0 20 read_log 192803'16149 (192803'16148) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:229 2014-08-11 12:41:45.781446 2014-08-13 18:06:06.894420 7f5c973a57a0 20 read_log 192803'16150 (192803'16149) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:230 2014-08-11 12:41:45.836212 2014-08-13 18:06:06.894433 7f5c973a57a0 20 read_log 192803'16151 (192803'16150) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:231 2014-08-11 12:41:45.865019 2014-08-13 18:06:06.894445 7f5c973a57a0 20 read_log 192803'16152 (192803'16151) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:232 2014-08-11 12:41:45.906990 2014-08-13 18:06:06.894473 7f5c973a57a0 20 read_log 192803'16153 (192803'16152) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:233 2014-08-11 12:41:45.956489 2014-08-13 18:06:06.894486 7f5c973a57a0 20 read_log 192803'16154 (192803'16153) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:234 2014-08-11 12:41:46.408153 2014-08-13 18:06:06.894506 7f5c973a57a0 20 read_log 192803'16155 (192803'16154) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:235 2014-08-11 12:41:46.448559 2014-08-13 18:06:06.894522 7f5c973a57a0 20 read_log 192803'16156 (192803'16155) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:236 2014-08-11 12:41:46.490895 2014-08-13 18:06:06.894534 7f5c973a57a0 20 read_log 192803'16157 (192803'16156) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:237 2014-08-11 12:41:46.532343 2014-08-13 18:06:06.894555 7f5c973a57a0 20 read_log 192803'16158 (192803'16157) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:238 2014-08-11 12:41:46.573066 2014-08-13 18:06:06.894571 7f5c973a57a0 20 read_log 192803'16159 (192803'16158) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:239 2014-08-11 12:41:46.614521 2014-08-13 18:06:06.894583 7f5c973a57a0 20 read_log 192803'16160 (192803'16159) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:240 2014-08-11 12:41:46.657857 2014-08-13 18:06:06.894610 7f5c973a57a0 20 read_log 192803'16161 (192803'16160) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:241 2014-08-11 12:41:46.699566 2014-08-13 18:06:06.894623 7f5c973a57a0 20 read_log 192803'16162 (192803'16161) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:242 2014-08-11 12:41:46.741318 2014-08-13 18:06:06.894635 7f5c973a57a0 20 read_log 192803'16163 (192803'16162) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:243 2014-08-11 12:41:46.785888 2014-08-13 18:06:06.894647 7f5c973a57a0 20 read_log 192803'16164 (192803'16163) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:244 2014-08-11 12:41:47.235111 2014-08-13 18:06:06.894659 7f5c973a57a0 20 read_log 192803'16165 (192803'16164) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:245 2014-08-11 12:41:47.278508 2014-08-13 18:06:06.894672 7f5c973a57a0 20 read_log 192803'16166 (192803'16165) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:246 2014-08-11 12:41:47.344741 2014-08-13 18:06:06.894684 7f5c973a57a0 20 read_log 192803'16167 (192803'16166) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:247 2014-08-11 12:41:47.371933 2014-08-13 18:06:06.894696 7f5c973a57a0 20 read_log 192803'16168 (192803'16167) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:248 2014-08-11 12:41:47.414230 2014-08-13 18:06:06.894715 7f5c973a57a0 20 read_log 192803'16169 (192803'16168) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:249 2014-08-11 12:41:47.444168 2014-08-13 18:06:06.894730 7f5c973a57a0 20 read_log 192803'16170 (192803'16169) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:250 2014-08-11 12:41:47.483294 2014-08-13 18:06:06.894742 7f5c973a57a0 20 read_log 192803'16171 (192803'16170) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:251 2014-08-11 12:41:47.528251 2014-08-13 18:06:06.894755 7f5c973a57a0 20 read_log 192803'16172 (192803'16171) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:252 2014-08-11 12:41:47.559492 2014-08-13 18:06:06.894767 7f5c973a57a0 20 read_log 192803'16173 (192803'16172) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:253 2014-08-11 12:41:47.607692 2014-08-13 18:06:06.894779 7f5c973a57a0 20 read_log 192803'16174 (192803'16173) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:254 2014-08-11 12:41:47.874637 2014-08-13 18:06:06.894791 7f5c973a57a0 20 read_log 192803'16175 (192803'16174) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:255 2014-08-11 12:41:47.902557 2014-08-13 18:06:06.894803 7f5c973a57a0 20 read_log 192803'16176 (192803'16175) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:256 2014-08-11 12:41:47.931902 2014-08-13 18:06:06.894822 7f5c973a57a0 20 read_log 192803'16177 (192803'16176) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:257 2014-08-11 12:41:47.958528 2014-08-13 18:06:06.894838 7f5c973a57a0 20 read_log 192803'16178 (192803'16177) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:258 2014-08-11 12:41:47.993759 2014-08-13 18:06:06.894851 7f5c973a57a0 20 read_log 192803'16179 (192803'16178) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:259 2014-08-11 12:41:48.085934 2014-08-13 18:06:06.894863 7f5c973a57a0 20 read_log 192803'16180 (192803'16179) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:260 2014-08-11 12:41:48.151118 2014-08-13 18:06:06.894875 7f5c973a57a0 20 read_log 192803'16181 (192803'16180) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:261 2014-08-11 12:41:48.209173 2014-08-13 18:06:06.894887 7f5c973a57a0 20 read_log 192803'16182 (192803'16181) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:262 2014-08-11 12:41:48.236125 2014-08-13 18:06:06.894899 7f5c973a57a0 20 read_log 192803'16183 (192803'16182) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:263 2014-08-11 12:41:48.271880 2014-08-13 18:06:06.894911 7f5c973a57a0 20 read_log 192803'16184 (192803'16183) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:264 2014-08-11 12:41:48.298021 2014-08-13 18:06:06.894923 7f5c973a57a0 20 read_log 192803'16185 (192803'16184) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:265 2014-08-11 12:41:48.324061 2014-08-13 18:06:06.894935 7f5c973a57a0 20 read_log 192803'16186 (192803'16185) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:266 2014-08-11 12:41:48.362339 2014-08-13 18:06:06.894947 7f5c973a57a0 20 read_log 192803'16187 (192803'16186) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:267 2014-08-11 12:41:48.387486 2014-08-13 18:06:06.894959 7f5c973a57a0 20 read_log 192803'16188 (192803'16187) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:268 2014-08-11 12:41:48.415772 2014-08-13 18:06:06.894980 7f5c973a57a0 20 read_log 192803'16189 (192803'16188) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:269 2014-08-11 12:41:48.440966 2014-08-13 18:06:06.894995 7f5c973a57a0 20 read_log 192803'16190 (192803'16189) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:270 2014-08-11 12:41:48.465818 2014-08-13 18:06:06.895008 7f5c973a57a0 20 read_log 192803'16191 (192803'16190) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:271 2014-08-11 12:41:48.739188 2014-08-13 18:06:06.895020 7f5c973a57a0 20 read_log 192803'16192 (192803'16191) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:272 2014-08-11 12:41:48.765576 2014-08-13 18:06:06.895041 7f5c973a57a0 20 read_log 192803'16193 (192803'16192) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:273 2014-08-11 12:41:48.801130 2014-08-13 18:06:06.895056 7f5c973a57a0 20 read_log 192803'16194 (192803'16193) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:274 2014-08-11 12:41:48.827409 2014-08-13 18:06:06.895071 7f5c973a57a0 20 read_log 192803'16195 (192803'16194) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:275 2014-08-11 12:41:48.853715 2014-08-13 18:06:06.895083 7f5c973a57a0 20 read_log 192803'16196 (192803'16195) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:276 2014-08-11 12:41:48.883573 2014-08-13 18:06:06.895095 7f5c973a57a0 20 read_log 192803'16197 (192803'16196) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:277 2014-08-11 12:41:48.909748 2014-08-13 18:06:06.895107 7f5c973a57a0 20 read_log 192803'16198 (192803'16197) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:278 2014-08-11 12:41:48.937869 2014-08-13 18:06:06.895123 7f5c973a57a0 20 read_log 192803'16199 (192803'16198) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:279 2014-08-11 12:41:48.964587 2014-08-13 18:06:06.895138 7f5c973a57a0 20 read_log 192803'16200 (192803'16199) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:280 2014-08-11 12:41:48.989247 2014-08-13 18:06:06.895155 7f5c973a57a0 20 read_log 192803'16201 (192803'16200) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:281 2014-08-11 12:41:49.252365 2014-08-13 18:06:06.895172 7f5c973a57a0 20 read_log 192803'16202 (192803'16201) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:282 2014-08-11 12:41:49.278484 2014-08-13 18:06:06.895184 7f5c973a57a0 20 read_log 192803'16203 (192803'16202) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:283 2014-08-11 12:41:49.311409 2014-08-13 18:06:06.895196 7f5c973a57a0 20 read_log 192803'16204 (192803'16203) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:284 2014-08-11 12:41:49.333092 2014-08-13 18:06:06.895230 7f5c973a57a0 20 read_log 192803'16205 (192803'16204) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:285 2014-08-11 12:41:49.354291 2014-08-13 18:06:06.895243 7f5c973a57a0 20 read_log 192803'16206 (192803'16205) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:286 2014-08-11 12:41:49.386862 2014-08-13 18:06:06.895255 7f5c973a57a0 20 read_log 192803'16207 (192803'16206) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:287 2014-08-11 12:41:49.412651 2014-08-13 18:06:06.895268 7f5c973a57a0 20 read_log 192803'16208 (192803'16207) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:288 2014-08-11 12:41:49.443169 2014-08-13 18:06:06.895280 7f5c973a57a0 20 read_log 192803'16209 (192803'16208) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:289 2014-08-11 12:41:49.469952 2014-08-13 18:06:06.895299 7f5c973a57a0 20 read_log 192803'16210 (192803'16209) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:290 2014-08-11 12:41:49.495540 2014-08-13 18:06:06.895314 7f5c973a57a0 20 read_log 192803'16211 (192803'16210) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:291 2014-08-11 12:41:49.765672 2014-08-13 18:06:06.895326 7f5c973a57a0 20 read_log 192803'16212 (192803'16211) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:292 2014-08-11 12:41:49.791810 2014-08-13 18:06:06.895338 7f5c973a57a0 20 read_log 192803'16213 (192803'16212) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:293 2014-08-11 12:41:49.819891 2014-08-13 18:06:06.895357 7f5c973a57a0 20 read_log 192803'16214 (192803'16213) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:294 2014-08-11 12:41:49.846513 2014-08-13 18:06:06.895370 7f5c973a57a0 20 read_log 192803'16215 (192803'16214) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:295 2014-08-11 12:41:49.874414 2014-08-13 18:06:06.895383 7f5c973a57a0 20 read_log 192803'16216 (192803'16215) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:296 2014-08-11 12:41:49.902413 2014-08-13 18:06:06.895411 7f5c973a57a0 20 read_log 192803'16217 (192803'16216) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:297 2014-08-11 12:41:49.929492 2014-08-13 18:06:06.895424 7f5c973a57a0 20 read_log 192803'16218 (192803'16217) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:298 2014-08-11 12:41:49.964434 2014-08-13 18:06:06.895437 7f5c973a57a0 20 read_log 192803'16219 (192803'16218) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:299 2014-08-11 12:41:49.990034 2014-08-13 18:06:06.895449 7f5c973a57a0 20 read_log 192803'16220 (192803'16219) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:300 2014-08-11 12:41:50.015121 2014-08-13 18:06:06.895461 7f5c973a57a0 20 read_log 192803'16221 (192803'16220) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:301 2014-08-11 12:41:50.287899 2014-08-13 18:06:06.895473 7f5c973a57a0 20 read_log 192803'16222 (192803'16221) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:302 2014-08-11 12:41:50.310825 2014-08-13 18:06:06.895485 7f5c973a57a0 20 read_log 192803'16223 (192803'16222) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:303 2014-08-11 12:41:50.341822 2014-08-13 18:06:06.895497 7f5c973a57a0 20 read_log 192803'16224 (192803'16223) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:304 2014-08-11 12:41:50.369068 2014-08-13 18:06:06.895509 7f5c973a57a0 20 read_log 192803'16225 (192803'16224) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:305 2014-08-11 12:41:50.395939 2014-08-13 18:06:06.895521 7f5c973a57a0 20 read_log 192803'16226 (192803'16225) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:306 2014-08-11 12:41:50.424776 2014-08-13 18:06:06.895533 7f5c973a57a0 20 read_log 192803'16227 (192803'16226) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:307 2014-08-11 12:41:50.451800 2014-08-13 18:06:06.895544 7f5c973a57a0 20 read_log 192803'16228 (192803'16227) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:308 2014-08-11 12:41:50.486703 2014-08-13 18:06:06.895577 7f5c973a57a0 20 read_log 192803'16229 (192803'16228) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:309 2014-08-11 12:41:50.514690 2014-08-13 18:06:06.895589 7f5c973a57a0 20 read_log 192803'16230 (192803'16229) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:310 2014-08-11 12:41:50.540233 2014-08-13 18:06:06.895601 7f5c973a57a0 20 read_log 192803'16231 (192803'16230) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:311 2014-08-11 12:41:50.819383 2014-08-13 18:06:06.895613 7f5c973a57a0 20 read_log 192803'16232 (192803'16231) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:312 2014-08-11 12:41:50.845963 2014-08-13 18:06:06.895625 7f5c973a57a0 20 read_log 192803'16233 (192803'16232) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:313 2014-08-11 12:41:50.893780 2014-08-13 18:06:06.895637 7f5c973a57a0 20 read_log 192803'16234 (192803'16233) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:314 2014-08-11 12:41:50.920124 2014-08-13 18:06:06.895649 7f5c973a57a0 20 read_log 192803'16235 (192803'16234) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:315 2014-08-11 12:41:50.947099 2014-08-13 18:06:06.895660 7f5c973a57a0 20 read_log 192803'16236 (192803'16235) modify 38370f0b/core.31758+10+2/head//24 by client.917983.0:316 2014-08-11 12:41:50.983102 2014-08-13 18:06:06.895672 7f5c973a57a0 20 read_log 192803'16321 (192803'16236) clean 38370f0b/core.31758+10+2/head//24 by osd.329.0:17951 2014-08-11 12:44:51.390607 2014-08-13 18:06:06.895684 7f5c973a57a0 20 read_log 192803'16322 (192803'16321) delete 38370f0b/core.31758+10+2/head//24 by osd.329.0:17952 2014-08-11 12:44:51.429624 2014-08-13 18:06:06.895696 7f5c973a57a0 20 read_log 192834'16738 (0'0) promote a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by osd.329.0:18377 2014-08-11 15:23:36.854654 2014-08-13 18:06:06.895719 7f5c973a57a0 20 read_log 192834'16739 (192834'16738) modify a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.933448.0:310 2014-08-11 15:23:36.845920 2014-08-13 18:06:06.895740 7f5c973a57a0 20 read_log 192834'16750 (0'0) promote e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by osd.329.0:18389 2014-08-11 15:23:40.006879 2014-08-13 18:06:06.895753 7f5c973a57a0 20 read_log 192834'16751 (192834'16750) modify e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.933448.0:1120 2014-08-11 15:23:39.999653 2014-08-13 18:06:06.895766 7f5c973a57a0 20 read_log 192834'16774 (0'0) promote 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by osd.329.0:18413 2014-08-11 15:23:47.193970 2014-08-13 18:06:06.895779 7f5c973a57a0 20 read_log 192834'16775 (192834'16774) modify 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.933448.0:2528 2014-08-11 15:23:47.183318 2014-08-13 18:06:06.895791 7f5c973a57a0 20 read_log 192834'16810 (0'0) promote a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18449 2014-08-11 15:24:02.922788 2014-08-13 18:06:06.895803 7f5c973a57a0 20 read_log 192834'16811 (192834'16810) modify a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.933448.0:5797 2014-08-11 15:24:02.914240 2014-08-13 18:06:06.895815 7f5c973a57a0 20 read_log 192834'16830 (0'0) promote a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by osd.329.0:18469 2014-08-11 15:24:06.485195 2014-08-13 18:06:06.895827 7f5c973a57a0 20 read_log 192834'16831 (192834'16830) modify a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.933448.0:6716 2014-08-11 15:24:06.477268 2014-08-13 18:06:06.895839 7f5c973a57a0 20 read_log 192834'16834 (0'0) promote 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by osd.329.0:18473 2014-08-11 15:24:07.574214 2014-08-13 18:06:06.895851 7f5c973a57a0 20 read_log 192834'16835 (192834'16834) modify 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.933448.0:6974 2014-08-11 15:24:07.565870 2014-08-13 18:06:06.895863 7f5c973a57a0 20 read_log 192834'16848 (0'0) promote e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by osd.329.0:18487 2014-08-11 15:24:11.032499 2014-08-13 18:06:06.895875 7f5c973a57a0 20 read_log 192834'16849 (192834'16848) modify e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.933448.0:7688 2014-08-11 15:24:11.024496 2014-08-13 18:06:06.895887 7f5c973a57a0 20 read_log 192834'16852 (0'0) promote e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by osd.329.0:18491 2014-08-11 15:24:11.193931 2014-08-13 18:06:06.895899 7f5c973a57a0 20 read_log 192834'16853 (192834'16852) modify e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.933448.0:7731 2014-08-11 15:24:11.185239 2014-08-13 18:06:06.895913 7f5c973a57a0 20 read_log 192834'16864 (0'0) promote 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by osd.329.0:18503 2014-08-11 15:24:14.751126 2014-08-13 18:06:06.895925 7f5c973a57a0 20 read_log 192834'16865 (192834'16864) modify 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.933448.0:8610 2014-08-11 15:24:14.742118 2014-08-13 18:06:06.895937 7f5c973a57a0 20 read_log 192834'16872 (0'0) promote 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by osd.329.0:18511 2014-08-11 15:24:15.247920 2014-08-13 18:06:06.895949 7f5c973a57a0 20 read_log 192834'16873 (192834'16872) modify 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.933448.0:8729 2014-08-11 15:24:15.191577 2014-08-13 18:06:06.895961 7f5c973a57a0 20 read_log 192834'16886 (0'0) promote 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by osd.329.0:18525 2014-08-11 15:24:19.256400 2014-08-13 18:06:06.895976 7f5c973a57a0 20 read_log 192834'16887 (192834'16886) modify 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.933448.0:9544 2014-08-11 15:24:19.247988 2014-08-13 18:06:06.895999 7f5c973a57a0 20 read_log 192834'16888 (0'0) promote 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by osd.329.0:18527 2014-08-11 15:24:19.357454 2014-08-13 18:06:06.896014 7f5c973a57a0 20 read_log 192834'16889 (192834'16888) modify 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.933448.0:9565 2014-08-11 15:24:19.348446 2014-08-13 18:06:06.896026 7f5c973a57a0 20 read_log 192834'16898 (0'0) promote e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by osd.329.0:18537 2014-08-11 15:24:23.503560 2014-08-13 18:06:06.896041 7f5c973a57a0 20 read_log 192834'16899 (192834'16898) modify e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.933448.0:10499 2014-08-11 15:24:23.496516 2014-08-13 18:06:06.896054 7f5c973a57a0 20 read_log 192834'16910 (0'0) promote c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by osd.329.0:18549 2014-08-11 15:24:26.019099 2014-08-13 18:06:06.896068 7f5c973a57a0 20 read_log 192834'16911 (192834'16910) modify c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.933448.0:10989 2014-08-11 15:24:26.012079 2014-08-13 18:06:06.896080 7f5c973a57a0 20 read_log 192834'16912 (0'0) promote cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by osd.329.0:18551 2014-08-11 15:24:26.971074 2014-08-13 18:06:06.896094 7f5c973a57a0 20 read_log 192834'16913 (192834'16912) modify cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.933448.0:11240 2014-08-11 15:24:26.962305 2014-08-13 18:06:06.896107 7f5c973a57a0 20 read_log 192834'16914 (0'0) promote ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by osd.329.0:18553 2014-08-11 15:24:27.378318 2014-08-13 18:06:06.896122 7f5c973a57a0 20 read_log 192834'16915 (192834'16914) modify ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.933448.0:11347 2014-08-11 15:24:27.370099 2014-08-13 18:06:06.896136 7f5c973a57a0 20 read_log 192834'16916 (0'0) promote e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by osd.329.0:18555 2014-08-11 15:24:27.468983 2014-08-13 18:06:06.896154 7f5c973a57a0 20 read_log 192834'16917 (192834'16916) modify e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.933448.0:11371 2014-08-11 15:24:27.459612 2014-08-13 18:06:06.896167 7f5c973a57a0 20 read_log 192834'16934 (0'0) promote 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by osd.329.0:18573 2014-08-11 15:24:31.289578 2014-08-13 18:06:06.896182 7f5c973a57a0 20 read_log 192834'16935 (192834'16934) modify 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.933448.0:12153 2014-08-11 15:24:31.281162 2014-08-13 18:06:06.896197 7f5c973a57a0 20 read_log 192834'16950 (0'0) promote 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by osd.329.0:18589 2014-08-11 15:24:36.211364 2014-08-13 18:06:06.896209 7f5c973a57a0 20 read_log 192834'16951 (192834'16950) modify 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.933448.0:13292 2014-08-11 15:24:36.204105 2014-08-13 18:06:06.896224 7f5c973a57a0 20 read_log 192834'16968 (0'0) promote 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by osd.329.0:18607 2014-08-11 15:24:46.266676 2014-08-13 18:06:06.896237 7f5c973a57a0 20 read_log 192834'16969 (192834'16968) modify 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.933448.0:14584 2014-08-11 15:24:46.257476 2014-08-13 18:06:06.896251 7f5c973a57a0 20 read_log 192834'17010 (0'0) promote 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by osd.329.0:18649 2014-08-11 15:24:57.166185 2014-08-13 18:06:06.896266 7f5c973a57a0 20 read_log 192834'17011 (192834'17010) modify 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.933448.0:16805 2014-08-11 15:24:57.158687 2014-08-13 18:06:06.896303 7f5c973a57a0 20 read_log 192834'17030 (0'0) promote f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by osd.329.0:18669 2014-08-11 15:25:01.699471 2014-08-13 18:06:06.896323 7f5c973a57a0 20 read_log 192834'17031 (192834'17030) modify f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.933448.0:17799 2014-08-11 15:25:01.690378 2014-08-13 18:06:06.896336 7f5c973a57a0 20 read_log 192834'17050 (0'0) promote 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by osd.329.0:18689 2014-08-11 15:25:08.435372 2014-08-13 18:06:06.896348 7f5c973a57a0 20 read_log 192834'17051 (192834'17050) modify 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.933448.0:19074 2014-08-11 15:25:08.426033 2014-08-13 18:06:06.896364 7f5c973a57a0 20 read_log 192834'17056 (0'0) promote 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by osd.329.0:18695 2014-08-11 15:25:09.173771 2014-08-13 18:06:06.896378 7f5c973a57a0 20 read_log 192834'17057 (192834'17056) modify 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.933448.0:19223 2014-08-11 15:25:09.161454 2014-08-13 18:06:06.896390 7f5c973a57a0 20 read_log 192834'17062 (0'0) promote c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by osd.329.0:18701 2014-08-11 15:25:11.704334 2014-08-13 18:06:06.896402 7f5c973a57a0 20 read_log 192834'17063 (192834'17062) modify c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.933448.0:19379 2014-08-11 15:25:11.696372 2014-08-13 18:06:06.896420 7f5c973a57a0 20 read_log 192834'17078 (0'0) promote 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by osd.329.0:18717 2014-08-11 15:25:19.671033 2014-08-13 18:06:06.896433 7f5c973a57a0 20 read_log 192834'17079 (192834'17078) modify 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.933448.0:20205 2014-08-11 15:25:19.661958 2014-08-13 18:06:06.896445 7f5c973a57a0 20 read_log 192834'17092 (0'0) promote a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by osd.329.0:18731 2014-08-11 15:25:23.135050 2014-08-13 18:06:06.896457 7f5c973a57a0 20 read_log 192834'17093 (192834'17092) modify a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.933448.0:20733 2014-08-11 15:25:23.125218 2014-08-13 18:06:06.896481 7f5c973a57a0 20 read_log 192834'17098 (0'0) promote 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by osd.329.0:18737 2014-08-11 15:25:25.682346 2014-08-13 18:06:06.896496 7f5c973a57a0 20 read_log 192834'17099 (192834'17098) modify 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.933448.0:21000 2014-08-11 15:25:25.674366 2014-08-13 18:06:06.896508 7f5c973a57a0 20 read_log 192834'17114 (0'0) promote 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by osd.329.0:18753 2014-08-11 15:25:32.149006 2014-08-13 18:06:06.896530 7f5c973a57a0 20 read_log 192834'17115 (192834'17114) modify 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.933448.0:21826 2014-08-11 15:25:32.137728 2014-08-13 18:06:06.896543 7f5c973a57a0 20 read_log 192834'17118 (0'0) promote d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by osd.329.0:18757 2014-08-11 15:25:32.659802 2014-08-13 18:06:06.896584 7f5c973a57a0 20 read_log 192834'17119 (192834'17118) modify d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.933448.0:21897 2014-08-11 15:25:32.600742 2014-08-13 18:06:06.896607 7f5c973a57a0 20 read_log 192834'17125 (0'0) promote d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by osd.329.0:18764 2014-08-11 15:25:32.850533 2014-08-13 18:06:06.896630 7f5c973a57a0 20 read_log 192834'17126 (192834'17125) modify d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.933448.0:21925 2014-08-11 15:25:32.809119 2014-08-13 18:06:06.896651 7f5c973a57a0 20 read_log 192834'17143 (192834'16811) clean a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18782 2014-08-11 15:25:33.176475 2014-08-13 18:06:06.896678 7f5c973a57a0 20 read_log 192834'17145 (192834'17143) delete a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18784 2014-08-11 15:25:33.238799 2014-08-13 18:06:06.896692 7f5c973a57a0 20 read_log 192834'17151 (192834'16739) modify a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.933448.0:22601 2014-08-11 15:25:38.274879 2014-08-13 18:06:06.896708 7f5c973a57a0 20 read_log 192834'17158 (192834'16751) modify e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.933448.0:23411 2014-08-11 15:25:39.323646 2014-08-13 18:06:06.896720 7f5c973a57a0 20 read_log 192834'17171 (192834'16775) modify 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.933448.0:24819 2014-08-11 15:25:41.379371 2014-08-13 18:06:06.896732 7f5c973a57a0 20 read_log 192834'17191 (0'0) promote a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.329.0:18830 2014-08-11 15:25:45.494774 2014-08-13 18:06:06.896745 7f5c973a57a0 20 read_log 192834'17192 (192834'17191) modify a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.933448.0:28088 2014-08-11 15:25:45.470255 2014-08-13 18:06:06.896757 7f5c973a57a0 20 read_log 192834'17203 (192834'16831) modify a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.933448.0:29007 2014-08-11 15:25:46.928210 2014-08-13 18:06:06.896772 7f5c973a57a0 20 read_log 192834'17207 (192834'16835) modify 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.933448.0:29265 2014-08-11 15:25:47.166002 2014-08-13 18:06:06.896785 7f5c973a57a0 20 read_log 192834'17214 (192834'16849) modify e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.933448.0:29979 2014-08-11 15:25:48.213189 2014-08-13 18:06:06.896797 7f5c973a57a0 20 read_log 192834'17216 (192834'16853) modify e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.933448.0:30022 2014-08-11 15:25:48.265666 2014-08-13 18:06:06.896810 7f5c973a57a0 20 read_log 192834'17222 (192834'16865) modify 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.933448.0:30901 2014-08-11 15:25:49.372673 2014-08-13 18:06:06.896822 7f5c973a57a0 20 read_log 192834'17225 (192834'16873) modify 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.933448.0:31020 2014-08-11 15:25:49.499061 2014-08-13 18:06:06.896834 7f5c973a57a0 20 read_log 192834'17235 (192834'16887) modify 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.933448.0:31835 2014-08-11 15:25:50.678866 2014-08-13 18:06:06.896846 7f5c973a57a0 20 read_log 192834'17236 (192834'16889) modify 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.933448.0:31856 2014-08-11 15:25:50.697834 2014-08-13 18:06:06.896858 7f5c973a57a0 20 read_log 192834'17242 (192834'16899) modify e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.933448.0:32790 2014-08-11 15:25:52.372204 2014-08-13 18:06:06.896871 7f5c973a57a0 20 read_log 192834'17248 (192834'16911) modify c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.933448.0:33280 2014-08-11 15:25:53.190748 2014-08-13 18:06:06.896883 7f5c973a57a0 20 read_log 192834'17249 (192834'16913) modify cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.933448.0:33531 2014-08-11 15:25:53.640410 2014-08-13 18:06:06.896895 7f5c973a57a0 20 read_log 192834'17250 (192834'16915) modify ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.933448.0:33638 2014-08-11 15:25:53.864963 2014-08-13 18:06:06.896908 7f5c973a57a0 20 read_log 192834'17251 (192834'16917) modify e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.933448.0:33662 2014-08-11 15:25:53.927264 2014-08-13 18:06:06.896930 7f5c973a57a0 20 read_log 192834'17260 (192834'16935) modify 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.933448.0:34444 2014-08-11 15:25:55.019232 2014-08-13 18:06:06.896945 7f5c973a57a0 20 read_log 192834'17269 (192834'16951) modify 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.933448.0:35583 2014-08-11 15:25:56.555513 2014-08-13 18:06:06.896958 7f5c973a57a0 20 read_log 192834'17278 (192834'16969) modify 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.933448.0:36875 2014-08-11 15:25:58.424869 2014-08-13 18:06:06.896970 7f5c973a57a0 20 read_log 192834'17299 (192834'17011) modify 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.933448.0:39096 2014-08-11 15:26:01.539015 2014-08-13 18:06:06.896982 7f5c973a57a0 20 read_log 192834'17309 (192834'17031) modify f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.933448.0:40090 2014-08-11 15:26:02.309017 2014-08-13 18:06:06.896995 7f5c973a57a0 20 read_log 192834'17319 (192834'17051) modify 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.933448.0:41365 2014-08-11 15:26:02.921360 2014-08-13 18:06:06.897007 7f5c973a57a0 20 read_log 192834'17322 (192834'17057) modify 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.933448.0:41514 2014-08-11 15:26:02.967857 2014-08-13 18:06:06.897019 7f5c973a57a0 20 read_log 192834'17325 (192834'17063) modify c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.933448.0:41670 2014-08-11 15:26:03.016925 2014-08-13 18:06:06.897031 7f5c973a57a0 20 read_log 192834'17333 (192834'17079) modify 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.933448.0:42496 2014-08-11 15:26:03.298270 2014-08-13 18:06:06.897044 7f5c973a57a0 20 read_log 192834'17340 (192834'17093) modify a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.933448.0:43024 2014-08-11 15:26:03.476830 2014-08-13 18:06:06.897056 7f5c973a57a0 20 read_log 192834'17343 (192834'17099) modify 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.933448.0:43291 2014-08-11 15:26:03.563103 2014-08-13 18:06:06.897090 7f5c973a57a0 20 read_log 192834'17351 (192834'17115) modify 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.933448.0:44117 2014-08-11 15:26:03.821873 2014-08-13 18:06:06.897107 7f5c973a57a0 20 read_log 192834'17352 (192834'17119) modify d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.933448.0:44188 2014-08-11 15:26:03.843961 2014-08-13 18:06:06.897120 7f5c973a57a0 20 read_log 192834'17353 (192834'17126) modify d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.933448.0:44216 2014-08-11 15:26:03.853138 2014-08-13 18:06:06.897135 7f5c973a57a0 20 read_log 192834'17420 (0'0) promote 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by osd.329.0:19059 2014-08-11 15:26:34.808059 2014-08-13 18:06:06.897152 7f5c973a57a0 20 read_log 192834'17421 (192834'17420) modify 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.1883634.0:5551 2014-08-11 15:26:34.798486 2014-08-13 18:06:06.897168 7f5c973a57a0 20 read_log 192834'17432 (0'0) promote 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by osd.329.0:19071 2014-08-11 15:26:38.601634 2014-08-13 18:06:06.897180 7f5c973a57a0 20 read_log 192834'17433 (192834'17432) modify 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.1883634.0:6526 2014-08-11 15:26:38.593968 2014-08-13 18:06:06.897193 7f5c973a57a0 20 read_log 192834'17442 (0'0) promote 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by osd.329.0:19081 2014-08-11 15:26:40.853410 2014-08-13 18:06:06.897206 7f5c973a57a0 20 read_log 192834'17443 (192834'17442) modify 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.1883634.0:7111 2014-08-11 15:26:40.846711 2014-08-13 18:06:06.897225 7f5c973a57a0 20 read_log 192834'17478 (0'0) promote 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by osd.329.0:19117 2014-08-11 15:26:51.616483 2014-08-13 18:06:06.897240 7f5c973a57a0 20 read_log 192834'17479 (192834'17478) modify 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.1883634.0:9626 2014-08-11 15:26:51.608460 2014-08-13 18:06:06.897252 7f5c973a57a0 20 read_log 192834'17486 (0'0) promote b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by osd.329.0:19125 2014-08-11 15:26:53.788811 2014-08-13 18:06:06.897264 7f5c973a57a0 20 read_log 192834'17487 (192834'17486) modify b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.1883634.0:10127 2014-08-11 15:26:53.781163 2014-08-13 18:06:06.897277 7f5c973a57a0 20 read_log 192834'17500 (0'0) promote 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by osd.329.0:19139 2014-08-11 15:26:56.158768 2014-08-13 18:06:06.897295 7f5c973a57a0 20 read_log 192834'17501 (192834'17500) modify 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.1883634.0:10677 2014-08-11 15:26:56.149613 2014-08-13 18:06:06.897308 7f5c973a57a0 20 read_log 192834'17518 (0'0) promote 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by osd.329.0:19157 2014-08-11 15:27:01.241826 2014-08-13 18:06:06.897320 7f5c973a57a0 20 read_log 192834'17519 (192834'17518) modify 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.1883634.0:11774 2014-08-11 15:27:01.232511 2014-08-13 18:06:06.897335 7f5c973a57a0 20 read_log 192834'17520 (0'0) promote 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by osd.329.0:19159 2014-08-11 15:27:01.965895 2014-08-13 18:06:06.897350 7f5c973a57a0 20 read_log 192834'17521 (192834'17520) modify 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.1883634.0:11980 2014-08-11 15:27:01.955482 2014-08-13 18:06:06.897363 7f5c973a57a0 20 read_log 192834'17522 (0'0) promote 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by osd.329.0:19161 2014-08-11 15:27:02.052446 2014-08-13 18:06:06.897375 7f5c973a57a0 20 read_log 192834'17523 (192834'17522) modify 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.1883634.0:12007 2014-08-11 15:27:02.042572 2014-08-13 18:06:06.897392 7f5c973a57a0 20 read_log 192834'17528 (0'0) promote 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by osd.329.0:19167 2014-08-11 15:27:03.554839 2014-08-13 18:06:06.897404 7f5c973a57a0 20 read_log 192834'17529 (192834'17528) modify 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.1883634.0:12326 2014-08-11 15:27:03.545441 2014-08-13 18:06:06.897417 7f5c973a57a0 20 read_log 192834'17556 (0'0) promote 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by osd.329.0:19195 2014-08-11 15:27:11.260717 2014-08-13 18:06:06.897432 7f5c973a57a0 20 read_log 192834'17557 (192834'17556) modify 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.1883634.0:13930 2014-08-11 15:27:11.251856 2014-08-13 18:06:06.897445 7f5c973a57a0 20 read_log 192834'17558 (0'0) promote d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by osd.329.0:19197 2014-08-11 15:27:11.592111 2014-08-13 18:06:06.897457 7f5c973a57a0 20 read_log 192834'17559 (192834'17558) modify d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.1883634.0:14021 2014-08-11 15:27:11.583622 2014-08-13 18:06:06.897469 7f5c973a57a0 20 read_log 192834'17582 (0'0) promote c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by osd.329.0:19221 2014-08-11 15:27:29.694185 2014-08-13 18:06:06.897488 7f5c973a57a0 20 read_log 192834'17583 (192834'17582) modify c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.1883634.0:16861 2014-08-11 15:27:29.684112 2014-08-13 18:06:06.897511 7f5c973a57a0 20 read_log 192834'17618 (0'0) promote ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by osd.329.0:19257 2014-08-11 15:27:52.382622 2014-08-13 18:06:06.897526 7f5c973a57a0 20 read_log 192834'17619 (192834'17618) modify ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.1883634.0:19421 2014-08-11 15:27:52.351160 2014-08-13 18:06:06.897555 7f5c973a57a0 20 read_log 192834'17630 (0'0) promote 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by osd.329.0:19269 2014-08-11 15:27:55.163671 2014-08-13 18:06:06.897568 7f5c973a57a0 20 read_log 192834'17631 (192834'17630) modify 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.1883634.0:20059 2014-08-11 15:27:55.077912 2014-08-13 18:06:06.897588 7f5c973a57a0 20 read_log 192834'17646 (0'0) promote da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by osd.329.0:19285 2014-08-11 15:28:02.987115 2014-08-13 18:06:06.897604 7f5c973a57a0 20 read_log 192834'17647 (192834'17646) modify da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.1883634.0:21137 2014-08-11 15:28:02.979341 2014-08-13 18:06:06.897624 7f5c973a57a0 20 read_log 192834'17678 (0'0) promote f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by osd.329.0:19317 2014-08-11 15:28:11.026872 2014-08-13 18:06:06.897643 7f5c973a57a0 20 read_log 192834'17679 (192834'17678) modify f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.1883634.0:22778 2014-08-11 15:28:11.017984 2014-08-13 18:06:06.897659 7f5c973a57a0 20 read_log 192834'17716 (192834'17421) modify 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.1883634.0:28875 2014-08-11 15:28:26.429111 2014-08-13 18:06:06.897672 7f5c973a57a0 20 read_log 192834'17722 (192834'17433) modify 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.1883634.0:29850 2014-08-11 15:28:27.759373 2014-08-13 18:06:06.897684 7f5c973a57a0 20 read_log 192834'17727 (192834'17443) modify 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.1883634.0:30435 2014-08-11 15:28:28.634154 2014-08-13 18:06:06.897697 7f5c973a57a0 20 read_log 192834'17747 (192834'17479) modify 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.1883634.0:32950 2014-08-11 15:28:32.571817 2014-08-13 18:06:06.897709 7f5c973a57a0 20 read_log 192834'17751 (192834'17487) modify b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.1883634.0:33451 2014-08-11 15:28:33.225636 2014-08-13 18:06:06.897721 7f5c973a57a0 20 read_log 192834'17758 (192834'17501) modify 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.1883634.0:34001 2014-08-11 15:28:34.110871 2014-08-13 18:06:06.897734 7f5c973a57a0 20 read_log 192834'17767 (192834'17519) modify 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.1883634.0:35098 2014-08-11 15:28:36.021059 2014-08-13 18:06:06.897747 7f5c973a57a0 20 read_log 192834'17768 (192834'17521) modify 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.1883634.0:35304 2014-08-11 15:28:36.413465 2014-08-13 18:06:06.897760 7f5c973a57a0 20 read_log 192834'17769 (192834'17523) modify 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.1883634.0:35331 2014-08-11 15:28:36.468214 2014-08-13 18:06:06.897797 7f5c973a57a0 20 read_log 192834'17772 (192834'17529) modify 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.1883634.0:35650 2014-08-11 15:28:36.956826 2014-08-13 18:06:06.897815 7f5c973a57a0 20 read_log 192834'17786 (192834'17557) modify 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.1883634.0:37254 2014-08-11 15:28:39.067259 2014-08-13 18:06:06.897828 7f5c973a57a0 20 read_log 192834'17787 (192834'17559) modify d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.1883634.0:37345 2014-08-11 15:28:39.199626 2014-08-13 18:06:06.897851 7f5c973a57a0 20 read_log 192834'17799 (192834'17583) modify c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.1883634.0:40185 2014-08-11 15:28:43.320185 2014-08-13 18:06:06.897866 7f5c973a57a0 20 read_log 192834'17817 (192834'17619) modify ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.1883634.0:42745 2014-08-11 15:28:46.341762 2014-08-13 18:06:06.897879 7f5c973a57a0 20 read_log 192834'17823 (192834'17631) modify 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.1883634.0:43383 2014-08-11 15:28:46.536921 2014-08-13 18:06:06.897891 7f5c973a57a0 20 read_log 192834'17831 (192834'17647) modify da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.1883634.0:44461 2014-08-11 15:28:46.864706 2014-08-13 18:06:06.897903 7f5c973a57a0 20 read_log 192834'17847 (192834'17679) modify f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.1883634.0:46102 2014-08-11 15:28:47.491579 2014-08-13 18:06:06.897916 7f5c973a57a0 20 read_log 192888'17851 (0'0) promote aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by osd.320.0:1678 2014-08-11 16:44:55.449708 2014-08-13 18:06:06.897928 7f5c973a57a0 20 read_log 192888'17852 (192888'17851) modify aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.1315259.0:124 2014-08-11 16:44:55.423845 2014-08-13 18:06:06.897941 7f5c973a57a0 20 read_log 192888'17855 (0'0) promote 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by osd.320.0:1682 2014-08-11 16:45:01.181701 2014-08-13 18:06:06.897953 7f5c973a57a0 20 read_log 192888'17856 (192888'17855) modify 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.1315259.0:1542 2014-08-11 16:45:01.174411 2014-08-13 18:06:06.897965 7f5c973a57a0 20 read_log 192888'17859 (0'0) promote 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by osd.320.0:1686 2014-08-11 16:45:09.102840 2014-08-13 18:06:06.897996 7f5c973a57a0 20 read_log 192888'17860 (192888'17859) modify 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.1315259.0:3633 2014-08-11 16:45:09.095033 2014-08-13 18:06:06.898012 7f5c973a57a0 20 read_log 192888'17861 (0'0) promote a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by osd.320.0:1688 2014-08-11 16:45:09.221953 2014-08-13 18:06:06.898024 7f5c973a57a0 20 read_log 192888'17862 (192888'17861) modify a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.1315259.0:3665 2014-08-11 16:45:09.215188 2014-08-13 18:06:06.898037 7f5c973a57a0 20 read_log 192888'17867 (0'0) promote 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by osd.320.0:1694 2014-08-11 16:45:29.478499 2014-08-13 18:06:06.898049 7f5c973a57a0 20 read_log 192888'17868 (192888'17867) modify 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.1315259.0:9170 2014-08-11 16:45:29.471167 2014-08-13 18:06:06.898061 7f5c973a57a0 20 read_log 192888'17869 (0'0) promote 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by osd.320.0:1696 2014-08-11 16:45:30.303956 2014-08-13 18:06:06.898073 7f5c973a57a0 20 read_log 192888'17870 (192888'17869) modify 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.1315259.0:9392 2014-08-11 16:45:30.295664 2014-08-13 18:06:06.898085 7f5c973a57a0 20 read_log 192888'17873 (0'0) promote 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by osd.320.0:1700 2014-08-11 16:45:37.094397 2014-08-13 18:06:06.898098 7f5c973a57a0 20 read_log 192888'17874 (192888'17873) modify 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.1315259.0:11118 2014-08-11 16:45:37.087169 2014-08-13 18:06:06.898119 7f5c973a57a0 20 read_log 192888'17875 (0'0) promote 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by osd.320.0:1702 2014-08-11 16:45:41.126604 2014-08-13 18:06:06.898134 7f5c973a57a0 20 read_log 192888'17876 (192888'17875) modify 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.1315259.0:12113 2014-08-11 16:45:41.119217 2014-08-13 18:06:06.898153 7f5c973a57a0 20 read_log 192888'17879 (0'0) promote e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by osd.320.0:1706 2014-08-11 16:45:43.565827 2014-08-13 18:06:06.898168 7f5c973a57a0 20 read_log 192888'17880 (192888'17879) modify e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.1315259.0:12704 2014-08-11 16:45:43.558567 2014-08-13 18:06:06.898180 7f5c973a57a0 20 read_log 192888'17881 (0'0) promote 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by osd.320.0:1708 2014-08-11 16:45:48.949857 2014-08-13 18:06:06.898193 7f5c973a57a0 20 read_log 192888'17882 (192888'17881) modify 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.1315259.0:14050 2014-08-11 16:45:48.941918 2014-08-13 18:06:06.898218 7f5c973a57a0 20 read_log 192888'17891 (0'0) promote 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by osd.320.0:1718 2014-08-11 16:45:53.579130 2014-08-13 18:06:06.898243 7f5c973a57a0 20 read_log 192888'17892 (192888'17891) modify 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.1315259.0:15265 2014-08-11 16:45:53.572460 2014-08-13 18:06:06.898258 7f5c973a57a0 20 read_log 192888'17893 (0'0) promote 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by osd.320.0:1720 2014-08-11 16:45:56.433203 2014-08-13 18:06:06.898271 7f5c973a57a0 20 read_log 192888'17894 (192888'17893) modify 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.1315259.0:16001 2014-08-11 16:45:56.425966 2014-08-13 18:06:06.898284 7f5c973a57a0 20 read_log 192888'17899 (0'0) promote d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by osd.320.0:1726 2014-08-11 16:46:08.843061 2014-08-13 18:06:06.898296 7f5c973a57a0 20 read_log 192888'17900 (192888'17899) modify d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.1315259.0:19131 2014-08-11 16:46:08.835789 2014-08-13 18:06:06.898312 7f5c973a57a0 20 read_log 192888'17905 (0'0) promote 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by osd.320.0:1732 2014-08-11 16:46:15.016123 2014-08-13 18:06:06.898324 7f5c973a57a0 20 read_log 192888'17906 (192888'17905) modify 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.1315259.0:20766 2014-08-11 16:46:15.009616 2014-08-13 18:06:06.898337 7f5c973a57a0 20 read_log 192888'17913 (0'0) promote a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by osd.320.0:1740 2014-08-11 16:46:21.689391 2014-08-13 18:06:06.898349 7f5c973a57a0 20 read_log 192888'17914 (192888'17913) modify a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.1315259.0:22505 2014-08-11 16:46:21.683175 2014-08-13 18:06:06.898365 7f5c973a57a0 20 read_log 192888'17917 (0'0) promote 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by osd.320.0:1744 2014-08-11 16:46:28.339556 2014-08-13 18:06:06.898377 7f5c973a57a0 20 read_log 192888'17918 (192888'17917) modify 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.1315259.0:24186 2014-08-11 16:46:28.332139 2014-08-13 18:06:06.898390 7f5c973a57a0 20 read_log 192888'17919 (0'0) promote 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by osd.320.0:1746 2014-08-11 16:46:29.046989 2014-08-13 18:06:06.898402 7f5c973a57a0 20 read_log 192888'17920 (192888'17919) modify 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.1315259.0:24376 2014-08-11 16:46:29.040191 2014-08-13 18:06:06.898433 7f5c973a57a0 20 read_log 192888'17923 (0'0) promote be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by osd.320.0:1750 2014-08-11 16:46:30.616541 2014-08-13 18:06:06.898448 7f5c973a57a0 20 read_log 192888'17924 (192888'17923) modify be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.1315259.0:24760 2014-08-11 16:46:30.608690 2014-08-13 18:06:06.898462 7f5c973a57a0 20 read_log 192888'17927 (0'0) promote 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by osd.320.0:1754 2014-08-11 16:46:32.907449 2014-08-13 18:06:06.898474 7f5c973a57a0 20 read_log 192888'17928 (192888'17927) modify 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.1315259.0:25345 2014-08-11 16:46:32.899952 2014-08-13 18:06:06.898500 7f5c973a57a0 20 read_log 192888'17935 (0'0) promote 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by osd.320.0:1762 2014-08-11 16:46:48.354518 2014-08-13 18:06:06.898513 7f5c973a57a0 20 read_log 192888'17936 (192888'17935) modify 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.1315259.0:29088 2014-08-11 16:46:48.348348 2014-08-13 18:06:06.898556 7f5c973a57a0 20 read_log 192888'17937 (0'0) promote 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by osd.320.0:1764 2014-08-11 16:46:51.216313 2014-08-13 18:06:06.898569 7f5c973a57a0 20 read_log 192888'17938 (192888'17937) modify 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.1315259.0:29763 2014-08-11 16:46:51.209232 2014-08-13 18:06:06.898581 7f5c973a57a0 20 read_log 192888'17941 (192888'17852) modify aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.1315259.0:30851 2014-08-11 16:46:55.912452 2014-08-13 18:06:06.898600 7f5c973a57a0 20 read_log 192888'17943 (192888'17856) modify 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.1315259.0:32269 2014-08-11 16:46:56.332539 2014-08-13 18:06:06.898613 7f5c973a57a0 20 read_log 192888'17945 (192888'17860) modify 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.1315259.0:34360 2014-08-11 16:46:56.873094 2014-08-13 18:06:06.898625 7f5c973a57a0 20 read_log 192888'17946 (192888'17862) modify a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.1315259.0:34392 2014-08-11 16:46:56.881271 2014-08-13 18:06:06.898646 7f5c973a57a0 20 read_log 192888'17949 (192888'17868) modify 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.1315259.0:39897 2014-08-11 16:46:58.332416 2014-08-13 18:06:06.898660 7f5c973a57a0 20 read_log 192888'17950 (192888'17870) modify 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.1315259.0:40119 2014-08-11 16:46:58.394569 2014-08-13 18:06:06.898672 7f5c973a57a0 20 read_log 192888'17952 (192888'17874) modify 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.1315259.0:41845 2014-08-11 16:46:58.853254 2014-08-13 18:06:06.898684 7f5c973a57a0 20 read_log 192888'17953 (192888'17876) modify 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.1315259.0:42840 2014-08-11 16:46:59.119755 2014-08-13 18:06:06.898697 7f5c973a57a0 20 read_log 192888'17955 (192888'17880) modify e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.1315259.0:43431 2014-08-11 16:46:59.289030 2014-08-13 18:06:06.898709 7f5c973a57a0 20 read_log 192888'17956 (192888'17882) modify 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.1315259.0:44777 2014-08-11 16:46:59.673721 2014-08-13 18:06:06.898721 7f5c973a57a0 20 read_log 192888'17961 (192888'17892) modify 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.1315259.0:45992 2014-08-11 16:47:00.029303 2014-08-13 18:06:06.898734 7f5c973a57a0 20 read_log 192888'17962 (192888'17894) modify 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.1315259.0:46728 2014-08-11 16:47:00.242494 2014-08-13 18:06:06.898779 7f5c973a57a0 20 read_log 192888'17965 (192888'17900) modify d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.1315259.0:49858 2014-08-11 16:47:01.194041 2014-08-13 18:06:06.898793 7f5c973a57a0 20 read_log 192888'17968 (192888'17906) modify 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.1315259.0:51493 2014-08-11 16:47:01.697000 2014-08-13 18:06:06.898806 7f5c973a57a0 20 read_log 192888'17972 (192888'17914) modify a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.1315259.0:53232 2014-08-11 16:47:02.193519 2014-08-13 18:06:06.898818 7f5c973a57a0 20 read_log 192888'17974 (192888'17918) modify 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.1315259.0:54913 2014-08-11 16:47:02.754549 2014-08-13 18:06:06.898830 7f5c973a57a0 20 read_log 192888'17975 (192888'17920) modify 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.1315259.0:55103 2014-08-11 16:47:02.815369 2014-08-13 18:06:06.898843 7f5c973a57a0 20 read_log 192888'17977 (192888'17924) modify be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.1315259.0:55487 2014-08-11 16:47:02.926134 2014-08-13 18:06:06.898855 7f5c973a57a0 20 read_log 192888'17979 (192888'17928) modify 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.1315259.0:56072 2014-08-11 16:47:03.095984 2014-08-13 18:06:06.898868 7f5c973a57a0 20 read_log 192888'17983 (192888'17936) modify 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.1315259.0:59815 2014-08-11 16:47:04.248647 2014-08-13 18:06:06.898880 7f5c973a57a0 20 read_log 192888'17984 (192888'17938) modify 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.1315259.0:60490 2014-08-11 16:47:04.458596 2014-08-13 18:06:06.898892 7f5c973a57a0 20 read_log 192888'17990 (0'0) promote c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by osd.320.0:1816 2014-08-11 17:05:20.723171 2014-08-13 18:06:06.898905 7f5c973a57a0 20 read_log 192888'17991 (192888'17990) modify c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.942868.0:1395 2014-08-11 17:05:20.716813 2014-08-13 18:06:06.898927 7f5c973a57a0 20 read_log 192888'17992 (0'0) promote 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by osd.320.0:1818 2014-08-11 17:05:21.935641 2014-08-13 18:06:06.898943 7f5c973a57a0 20 read_log 192888'17993 (192888'17992) modify 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.942868.0:1674 2014-08-11 17:05:21.928062 2014-08-13 18:06:06.898956 7f5c973a57a0 20 read_log 192888'17994 (0'0) promote e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by osd.320.0:1820 2014-08-11 17:05:22.927420 2014-08-13 18:06:06.898968 7f5c973a57a0 20 read_log 192888'17995 (192888'17994) modify e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.942868.0:1929 2014-08-11 17:05:22.920726 2014-08-13 18:06:06.898981 7f5c973a57a0 20 read_log 192888'18000 (0'0) promote 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by osd.320.0:1826 2014-08-11 17:05:30.773990 2014-08-13 18:06:06.898995 7f5c973a57a0 20 read_log 192888'18001 (192888'18000) modify 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.942868.0:4010 2014-08-11 17:05:30.766403 2014-08-13 18:06:06.899010 7f5c973a57a0 20 read_log 192888'18004 (0'0) promote 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by osd.320.0:1830 2014-08-11 17:05:32.997773 2014-08-13 18:06:06.899022 7f5c973a57a0 20 read_log 192888'18005 (192888'18004) modify 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.942868.0:4613 2014-08-11 17:05:32.992543 2014-08-13 18:06:06.899035 7f5c973a57a0 20 read_log 192888'18006 (0'0) promote b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by osd.320.0:1832 2014-08-11 17:05:33.708532 2014-08-13 18:06:06.899060 7f5c973a57a0 20 read_log 192888'18007 (192888'18006) modify b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.942868.0:4788 2014-08-11 17:05:33.702051 2014-08-13 18:06:06.899076 7f5c973a57a0 20 read_log 192888'18010 (0'0) promote c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by osd.320.0:1836 2014-08-11 17:05:37.842812 2014-08-13 18:06:06.899092 7f5c973a57a0 20 read_log 192888'18011 (192888'18010) modify c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.942868.0:5868 2014-08-11 17:05:37.834852 2014-08-13 18:06:06.899105 7f5c973a57a0 20 read_log 192888'18020 (0'0) promote 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by osd.320.0:1846 2014-08-11 17:05:42.674396 2014-08-13 18:06:06.899117 7f5c973a57a0 20 read_log 192888'18021 (192888'18020) modify 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.942868.0:7175 2014-08-11 17:05:42.667282 2014-08-13 18:06:06.899130 7f5c973a57a0 20 read_log 192888'18024 (0'0) promote 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by osd.320.0:1850 2014-08-11 17:05:46.348391 2014-08-13 18:06:06.899150 7f5c973a57a0 20 read_log 192888'18025 (192888'18024) modify 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.942868.0:8145 2014-08-11 17:05:46.341757 2014-08-13 18:06:06.899172 7f5c973a57a0 20 read_log 192888'18026 (0'0) promote 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by osd.320.0:1852 2014-08-11 17:05:46.447805 2014-08-13 18:06:06.899195 7f5c973a57a0 20 read_log 192888'18027 (192888'18026) modify 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.942868.0:8173 2014-08-11 17:05:46.442253 2014-08-13 18:06:06.899216 7f5c973a57a0 20 read_log 192888'18028 (0'0) promote 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by osd.320.0:1854 2014-08-11 17:05:48.596413 2014-08-13 18:06:06.899231 7f5c973a57a0 20 read_log 192888'18029 (192888'18028) modify 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.942868.0:8747 2014-08-11 17:05:48.588740 2014-08-13 18:06:06.899244 7f5c973a57a0 20 read_log 192888'18034 (0'0) promote 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by osd.320.0:1860 2014-08-11 17:06:01.605270 2014-08-13 18:06:06.899263 7f5c973a57a0 20 read_log 192888'18035 (192888'18034) modify 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.942868.0:12307 2014-08-11 17:06:01.596439 2014-08-13 18:06:06.899277 7f5c973a57a0 20 read_log 192888'18038 (0'0) promote d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by osd.320.0:1864 2014-08-11 17:06:03.748518 2014-08-13 18:06:06.899290 7f5c973a57a0 20 read_log 192888'18039 (192888'18038) modify d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.942868.0:12878 2014-08-11 17:06:03.741729 2014-08-13 18:06:06.899306 7f5c973a57a0 20 read_log 192888'18040 (0'0) promote e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by osd.320.0:1866 2014-08-11 17:06:05.796056 2014-08-13 18:06:06.899321 7f5c973a57a0 20 read_log 192888'18041 (192888'18040) modify e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.942868.0:13406 2014-08-11 17:06:05.786856 2014-08-13 18:06:06.899334 7f5c973a57a0 20 read_log 192888'18042 (0'0) promote f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by osd.320.0:1868 2014-08-11 17:06:05.933077 2014-08-13 18:06:06.899346 7f5c973a57a0 20 read_log 192888'18043 (192888'18042) modify f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.942868.0:13441 2014-08-11 17:06:05.924705 2014-08-13 18:06:06.899358 7f5c973a57a0 20 read_log 192888'18044 (0'0) promote 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by osd.320.0:1870 2014-08-11 17:06:07.730122 2014-08-13 18:06:06.899409 7f5c973a57a0 20 read_log 192888'18045 (192888'18044) modify 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.942868.0:13892 2014-08-11 17:06:07.722188 2014-08-13 18:06:06.899427 7f5c973a57a0 20 read_log 192888'18048 (0'0) promote 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by osd.320.0:1874 2014-08-11 17:06:11.109526 2014-08-13 18:06:06.899449 7f5c973a57a0 20 read_log 192888'18049 (192888'18048) modify 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.942868.0:14714 2014-08-11 17:06:11.102039 2014-08-13 18:06:06.899476 7f5c973a57a0 20 read_log 192888'18056 (0'0) promote d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by osd.320.0:1882 2014-08-11 17:06:19.059635 2014-08-13 18:06:06.899560 7f5c973a57a0 20 read_log 192888'18057 (192888'18056) modify d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.942868.0:16748 2014-08-11 17:06:19.053461 2014-08-13 18:06:06.899588 7f5c973a57a0 20 read_log 192888'18058 (0'0) promote 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by osd.320.0:1884 2014-08-11 17:06:24.908608 2014-08-13 18:06:06.899611 7f5c973a57a0 20 read_log 192888'18059 (192888'18058) modify 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.942868.0:18284 2014-08-11 17:06:24.902377 2014-08-13 18:06:06.899634 7f5c973a57a0 20 read_log 192888'18060 (0'0) promote 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by osd.320.0:1886 2014-08-11 17:06:25.024103 2014-08-13 18:06:06.899658 7f5c973a57a0 20 read_log 192888'18061 (192888'18060) modify 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.942868.0:18316 2014-08-11 17:06:25.018265 2014-08-13 18:06:06.899686 7f5c973a57a0 20 read_log 192888'18066 (0'0) promote 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by osd.320.0:1892 2014-08-11 17:06:32.424671 2014-08-13 18:06:06.899709 7f5c973a57a0 20 read_log 192888'18067 (192888'18066) modify 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.942868.0:20285 2014-08-11 17:06:32.417397 2014-08-13 18:06:06.899728 7f5c973a57a0 20 read_log 192888'18068 (0'0) promote 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by osd.320.0:1894 2014-08-11 17:06:34.902789 2014-08-13 18:06:06.899749 7f5c973a57a0 20 read_log 192888'18069 (192888'18068) modify 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.942868.0:20923 2014-08-11 17:06:34.895805 2014-08-13 18:06:06.899762 7f5c973a57a0 20 read_log 192888'18074 (0'0) promote e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by osd.320.0:1900 2014-08-11 17:06:35.540149 2014-08-13 18:06:06.899775 7f5c973a57a0 20 read_log 192888'18075 (192888'18074) modify e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.942868.0:21092 2014-08-11 17:06:35.533818 2014-08-13 18:06:06.899787 7f5c973a57a0 20 read_log 192888'18076 (0'0) promote 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by osd.320.0:1902 2014-08-11 17:06:36.417365 2014-08-13 18:06:06.899806 7f5c973a57a0 20 read_log 192888'18077 (192888'18076) modify 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.942868.0:21312 2014-08-11 17:06:36.407055 2014-08-13 18:06:06.899818 7f5c973a57a0 20 read_log 192888'18078 (0'0) promote 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by osd.320.0:1904 2014-08-11 17:06:36.553513 2014-08-13 18:06:06.899831 7f5c973a57a0 20 read_log 192888'18079 (192888'18078) modify 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.942868.0:21349 2014-08-11 17:06:36.545151 2014-08-13 18:06:06.899843 7f5c973a57a0 20 read_log 192888'18086 (0'0) promote 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by osd.320.0:1912 2014-08-11 17:06:40.972929 2014-08-13 18:06:06.899868 7f5c973a57a0 20 read_log 192888'18087 (192888'18086) modify 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.942868.0:22494 2014-08-11 17:06:40.966218 2014-08-13 18:06:06.899884 7f5c973a57a0 20 read_log 192888'18090 (0'0) promote cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by osd.320.0:1916 2014-08-11 17:06:47.746107 2014-08-13 18:06:06.899896 7f5c973a57a0 20 read_log 192888'18091 (192888'18090) modify cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.942868.0:24281 2014-08-11 17:06:47.739203 2014-08-13 18:06:06.899910 7f5c973a57a0 20 read_log 192888'18100 (0'0) promote d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by osd.320.0:1926 2014-08-11 17:07:06.656943 2014-08-13 18:06:06.899924 7f5c973a57a0 20 read_log 192888'18101 (192888'18100) modify d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.942868.0:29221 2014-08-11 17:07:06.649734 2014-08-13 18:06:06.899939 7f5c973a57a0 20 read_log 192888'18102 (0'0) promote 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by osd.320.0:1928 2014-08-11 17:07:06.685316 2014-08-13 18:06:06.899951 7f5c973a57a0 20 read_log 192888'18103 (192888'18102) modify 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.942868.0:29229 2014-08-11 17:07:06.679430 2014-08-13 18:06:06.899967 7f5c973a57a0 20 read_log 192888'18109 (192888'17991) modify c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.942868.0:32661 2014-08-11 17:07:16.313140 2014-08-13 18:06:06.899980 7f5c973a57a0 20 read_log 192888'18110 (192888'17993) modify 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.942868.0:32940 2014-08-11 17:07:16.382794 2014-08-13 18:06:06.899992 7f5c973a57a0 20 read_log 192888'18111 (192888'17995) modify e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.942868.0:33195 2014-08-11 17:07:16.451339 2014-08-13 18:06:06.900006 7f5c973a57a0 20 read_log 192888'18114 (192888'18001) modify 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.942868.0:35276 2014-08-11 17:07:17.032357 2014-08-13 18:06:06.900020 7f5c973a57a0 20 read_log 192888'18116 (192888'18005) modify 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.942868.0:35879 2014-08-11 17:07:17.187537 2014-08-13 18:06:06.900033 7f5c973a57a0 20 read_log 192888'18117 (192888'18007) modify b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.942868.0:36054 2014-08-11 17:07:17.230212 2014-08-13 18:06:06.900047 7f5c973a57a0 20 read_log 192888'18119 (192888'18011) modify c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.942868.0:37134 2014-08-11 17:07:17.525020 2014-08-13 18:06:06.900060 7f5c973a57a0 20 read_log 192888'18124 (192888'18021) modify 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.942868.0:38441 2014-08-11 17:07:17.887755 2014-08-13 18:06:06.900075 7f5c973a57a0 20 read_log 192888'18126 (192888'18025) modify 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.942868.0:39411 2014-08-11 17:07:18.175535 2014-08-13 18:06:06.900087 7f5c973a57a0 20 read_log 192888'18127 (192888'18027) modify 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.942868.0:39439 2014-08-11 17:07:18.181667 2014-08-13 18:06:06.900101 7f5c973a57a0 20 read_log 192888'18128 (192888'18029) modify 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.942868.0:40013 2014-08-11 17:07:18.342138 2014-08-13 18:06:06.900114 7f5c973a57a0 20 read_log 192888'18131 (192888'18035) modify 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.942868.0:43573 2014-08-11 17:07:19.310715 2014-08-13 18:06:06.900126 7f5c973a57a0 20 read_log 192888'18133 (192888'18039) modify d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.942868.0:44144 2014-08-11 17:07:19.483064 2014-08-13 18:06:06.900153 7f5c973a57a0 20 read_log 192888'18134 (192888'18041) modify e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.942868.0:44672 2014-08-11 17:07:19.632009 2014-08-13 18:06:06.900169 7f5c973a57a0 20 read_log 192888'18135 (192888'18043) modify f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.942868.0:44707 2014-08-11 17:07:19.642635 2014-08-13 18:06:06.900182 7f5c973a57a0 20 read_log 192888'18136 (192888'18045) modify 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.942868.0:45158 2014-08-11 17:07:19.761472 2014-08-13 18:06:06.900194 7f5c973a57a0 20 read_log 192888'18138 (192888'18049) modify 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.942868.0:45980 2014-08-11 17:07:19.987064 2014-08-13 18:06:06.900206 7f5c973a57a0 20 read_log 192888'18142 (192888'18057) modify d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.942868.0:48014 2014-08-11 17:07:20.595215 2014-08-13 18:06:06.900221 7f5c973a57a0 20 read_log 192888'18143 (192888'18059) modify 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.942868.0:49550 2014-08-11 17:07:21.047019 2014-08-13 18:06:06.900233 7f5c973a57a0 20 read_log 192888'18144 (192888'18061) modify 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.942868.0:49582 2014-08-11 17:07:21.056415 2014-08-13 18:06:06.900246 7f5c973a57a0 20 read_log 192888'18147 (192888'18067) modify 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.942868.0:51551 2014-08-11 17:07:21.678656 2014-08-13 18:06:06.900258 7f5c973a57a0 20 read_log 192888'18148 (192888'18069) modify 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.942868.0:52189 2014-08-11 17:07:21.865138 2014-08-13 18:06:06.900275 7f5c973a57a0 20 read_log 192888'18151 (192888'18075) modify e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.942868.0:52358 2014-08-11 17:07:21.913564 2014-08-13 18:06:06.900290 7f5c973a57a0 20 read_log 192888'18152 (192888'18077) modify 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.942868.0:52578 2014-08-11 17:07:21.985962 2014-08-13 18:06:06.900302 7f5c973a57a0 20 read_log 192888'18153 (192888'18079) modify 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.942868.0:52615 2014-08-11 17:07:21.998310 2014-08-13 18:06:06.900315 7f5c973a57a0 20 read_log 192888'18157 (192888'18087) modify 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.942868.0:53760 2014-08-11 17:07:22.366063 2014-08-13 18:06:06.900330 7f5c973a57a0 20 read_log 192888'18159 (192888'18091) modify cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.942868.0:55547 2014-08-11 17:07:22.881690 2014-08-13 18:06:06.900342 7f5c973a57a0 20 read_log 192888'18164 (192888'18101) modify d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.942868.0:60487 2014-08-11 17:07:24.355004 2014-08-13 18:06:06.900355 7f5c973a57a0 20 read_log 192888'18165 (192888'18103) modify 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.942868.0:60495 2014-08-11 17:07:24.356795 2014-08-13 18:06:06.900372 7f5c973a57a0 20 read_log 193305'18168 (193305'18168) modify 30b/hit_set_24.30b_archive_2014-08-11 18:38:11.461328_2014-08-11 18:42:40.507251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:40.507471 2014-08-13 18:06:06.900388 7f5c973a57a0 20 read_log 193305'18169 (0'0) promote ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by osd.4.0:26 2014-08-11 18:42:40.511218 2014-08-13 18:06:06.900401 7f5c973a57a0 20 read_log 193305'18170 (193305'18169) modify ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.951337.0:1930 2014-08-11 18:42:40.501871 2014-08-13 18:06:06.900434 7f5c973a57a0 20 read_log 193305'18171 (0'0) promote df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by osd.4.0:131 2014-08-11 18:43:02.858078 2014-08-13 18:06:06.900451 7f5c973a57a0 20 read_log 193305'18172 (193305'18171) modify df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.951337.0:8051 2014-08-11 18:43:02.850377 2014-08-13 18:06:06.900474 7f5c973a57a0 20 read_log 193305'18173 (0'0) promote d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by osd.4.0:133 2014-08-11 18:43:03.282446 2014-08-13 18:06:06.900487 7f5c973a57a0 20 read_log 193305'18174 (193305'18173) modify d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.951337.0:8167 2014-08-11 18:43:03.274902 2014-08-13 18:06:06.900499 7f5c973a57a0 20 read_log 193305'18175 (0'0) promote 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by osd.4.0:135 2014-08-11 18:43:04.196778 2014-08-13 18:06:06.900519 7f5c973a57a0 20 read_log 193305'18176 (193305'18175) modify 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.951337.0:8376 2014-08-11 18:43:04.189559 2014-08-13 18:06:06.900534 7f5c973a57a0 20 read_log 193305'18177 (0'0) promote a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by osd.4.0:153 2014-08-11 18:43:07.654184 2014-08-13 18:06:06.900568 7f5c973a57a0 20 read_log 193305'18178 (193305'18177) modify a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.951337.0:9282 2014-08-11 18:43:07.646399 2014-08-13 18:06:06.900582 7f5c973a57a0 20 read_log 193305'18179 (0'0) promote 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by osd.4.0:159 2014-08-11 18:43:08.745876 2014-08-13 18:06:06.900595 7f5c973a57a0 20 read_log 193305'18180 (193305'18179) modify 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.951337.0:9576 2014-08-11 18:43:08.738356 2014-08-13 18:06:06.900620 7f5c973a57a0 20 read_log 193305'18181 (0'0) promote 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by osd.4.0:161 2014-08-11 18:43:10.184413 2014-08-13 18:06:06.900636 7f5c973a57a0 20 read_log 193305'18182 (193305'18181) modify 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.951337.0:9970 2014-08-11 18:43:10.176036 2014-08-13 18:06:06.900649 7f5c973a57a0 20 read_log 193305'18183 (0'0) promote 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by osd.4.0:199 2014-08-11 18:43:17.937465 2014-08-13 18:06:06.900661 7f5c973a57a0 20 read_log 193305'18184 (193305'18183) modify 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.951337.0:12012 2014-08-11 18:43:17.929806 2014-08-13 18:06:06.900674 7f5c973a57a0 20 read_log 193305'18185 (0'0) promote 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by osd.4.0:201 2014-08-11 18:43:18.746955 2014-08-13 18:06:06.900686 7f5c973a57a0 20 read_log 193305'18186 (193305'18185) modify 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.951337.0:12234 2014-08-11 18:43:18.738991 2014-08-13 18:06:06.900699 7f5c973a57a0 20 read_log 193305'18187 (0'0) promote 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by osd.4.0:203 2014-08-11 18:43:19.101383 2014-08-13 18:06:06.900711 7f5c973a57a0 20 read_log 193305'18188 (193305'18187) modify 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.951337.0:12331 2014-08-11 18:43:19.093497 2014-08-13 18:06:06.900723 7f5c973a57a0 20 read_log 193305'18189 (0'0) promote 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by osd.4.0:205 2014-08-11 18:43:19.201922 2014-08-13 18:06:06.900735 7f5c973a57a0 20 read_log 193305'18190 (193305'18189) modify 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.951337.0:12359 2014-08-11 18:43:19.194880 2014-08-13 18:06:06.900758 7f5c973a57a0 20 read_log 193305'18191 (0'0) promote a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by osd.4.0:213 2014-08-11 18:43:21.980066 2014-08-13 18:06:06.900773 7f5c973a57a0 20 read_log 193305'18192 (193305'18191) modify a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.951337.0:13096 2014-08-11 18:43:21.972908 2014-08-13 18:06:06.900786 7f5c973a57a0 20 read_log 193305'18193 (0'0) promote 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by osd.4.0:221 2014-08-11 18:43:23.969774 2014-08-13 18:06:06.900798 7f5c973a57a0 20 read_log 193305'18194 (193305'18193) modify 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.951337.0:13608 2014-08-11 18:43:23.962729 2014-08-13 18:06:06.900810 7f5c973a57a0 20 read_log 193305'18195 (0'0) promote a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by osd.4.0:227 2014-08-11 18:43:25.589217 2014-08-13 18:06:06.900823 7f5c973a57a0 20 read_log 193305'18196 (193305'18195) modify a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.951337.0:14062 2014-08-11 18:43:25.581935 2014-08-13 18:06:06.900835 7f5c973a57a0 20 read_log 193305'18197 (0'0) promote 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by osd.4.0:241 2014-08-11 18:43:29.468077 2014-08-13 18:06:06.900847 7f5c973a57a0 20 read_log 193305'18198 (193305'18197) modify 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.951337.0:15148 2014-08-11 18:43:29.458935 2014-08-13 18:06:06.900859 7f5c973a57a0 20 read_log 193305'18199 (0'0) promote 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by osd.4.0:301 2014-08-11 18:43:39.164371 2014-08-13 18:06:06.900881 7f5c973a57a0 20 read_log 193305'18200 (193305'18199) modify 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.951337.0:17933 2014-08-11 18:43:39.156527 2014-08-13 18:06:06.900894 7f5c973a57a0 20 read_log 193305'18201 (0'0) promote 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by osd.4.0:303 2014-08-11 18:43:39.396421 2014-08-13 18:06:06.900906 7f5c973a57a0 20 read_log 193305'18202 (193305'18201) modify 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.951337.0:18005 2014-08-11 18:43:39.388848 2014-08-13 18:06:06.900918 7f5c973a57a0 20 read_log 193305'18203 (0'0) promote 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by osd.4.0:311 2014-08-11 18:43:41.376248 2014-08-13 18:06:06.900930 7f5c973a57a0 20 read_log 193305'18204 (193305'18203) modify 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.951337.0:18572 2014-08-11 18:43:41.366986 2014-08-13 18:06:06.900943 7f5c973a57a0 20 read_log 193305'18205 (0'0) promote e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by osd.4.0:319 2014-08-11 18:43:43.114866 2014-08-13 18:06:06.900955 7f5c973a57a0 20 read_log 193305'18206 (193305'18205) modify e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.951337.0:19040 2014-08-11 18:43:43.104278 2014-08-13 18:06:06.900967 7f5c973a57a0 20 read_log 193305'18207 (0'0) promote c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by osd.4.0:339 2014-08-11 18:43:47.517692 2014-08-13 18:06:06.900980 7f5c973a57a0 20 read_log 193305'18208 (193305'18207) modify c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.951337.0:20231 2014-08-11 18:43:47.506883 2014-08-13 18:06:06.901012 7f5c973a57a0 20 read_log 193305'18209 (0'0) promote bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by osd.4.0:343 2014-08-11 18:43:48.572450 2014-08-13 18:06:06.901028 7f5c973a57a0 20 read_log 193305'18210 (193305'18209) modify bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.951337.0:20536 2014-08-11 18:43:48.563743 2014-08-13 18:06:06.901064 7f5c973a57a0 20 read_log 193305'18211 (0'0) promote 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by osd.4.0:379 2014-08-11 18:43:55.932728 2014-08-13 18:06:06.901079 7f5c973a57a0 20 read_log 193305'18212 (193305'18211) modify 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.951337.0:22577 2014-08-11 18:43:55.921927 2014-08-13 18:06:06.901092 7f5c973a57a0 20 read_log 193305'18213 (0'0) promote 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by osd.4.0:383 2014-08-11 18:43:56.434302 2014-08-13 18:06:06.901106 7f5c973a57a0 20 read_log 193305'18214 (193305'18213) modify 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.951337.0:22715 2014-08-11 18:43:56.427330 2014-08-13 18:06:06.901121 7f5c973a57a0 20 read_log 193305'18215 (0'0) promote 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by osd.4.0:397 2014-08-11 18:44:01.323805 2014-08-13 18:06:06.901134 7f5c973a57a0 20 read_log 193305'18216 (193305'18215) modify 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.951337.0:24076 2014-08-11 18:44:01.314588 2014-08-13 18:06:06.901153 7f5c973a57a0 20 read_log 193305'18217 (0'0) promote 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by osd.4.0:399 2014-08-11 18:44:01.399462 2014-08-13 18:06:06.901168 7f5c973a57a0 20 read_log 193305'18218 (193305'18217) modify 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.951337.0:24096 2014-08-11 18:44:01.387652 2014-08-13 18:06:06.901180 7f5c973a57a0 20 read_log 193305'18219 (0'0) promote 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by osd.4.0:419 2014-08-11 18:44:05.926555 2014-08-13 18:06:06.901193 7f5c973a57a0 20 read_log 193305'18220 (193305'18219) modify 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.951337.0:25353 2014-08-11 18:44:05.918574 2014-08-13 18:06:06.901205 7f5c973a57a0 20 read_log 193305'18221 (0'0) promote 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by osd.4.0:429 2014-08-11 18:44:08.825183 2014-08-13 18:06:06.901217 7f5c973a57a0 20 read_log 193305'18222 (193305'18221) modify 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.951337.0:26114 2014-08-11 18:44:08.817000 2014-08-13 18:06:06.901230 7f5c973a57a0 20 read_log 193305'18223 (0'0) promote 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by osd.4.0:453 2014-08-11 18:44:12.581712 2014-08-13 18:06:06.901244 7f5c973a57a0 20 read_log 193305'18224 (193305'18223) modify 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.951337.0:27142 2014-08-11 18:44:12.573845 2014-08-13 18:06:06.901259 7f5c973a57a0 20 read_log 193305'18225 (0'0) promote 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by osd.4.0:455 2014-08-11 18:44:12.781967 2014-08-13 18:06:06.901272 7f5c973a57a0 20 read_log 193305'18226 (193305'18225) modify 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.951337.0:27196 2014-08-11 18:44:12.773927 2014-08-13 18:06:06.901287 7f5c973a57a0 20 read_log 193305'18227 (0'0) promote 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by osd.4.0:469 2014-08-11 18:44:17.277740 2014-08-13 18:06:06.901302 7f5c973a57a0 20 read_log 193305'18228 (193305'18227) modify 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.951337.0:28445 2014-08-11 18:44:17.270412 2014-08-13 18:06:06.901314 7f5c973a57a0 20 read_log 193305'18229 (0'0) promote 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by osd.4.0:473 2014-08-11 18:44:18.198134 2014-08-13 18:06:06.901326 7f5c973a57a0 20 read_log 193305'18230 (193305'18229) modify 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.951337.0:28702 2014-08-11 18:44:18.190143 2014-08-13 18:06:06.901346 7f5c973a57a0 20 read_log 193305'18231 (0'0) promote bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by osd.4.0:495 2014-08-11 18:44:23.232654 2014-08-13 18:06:06.901366 7f5c973a57a0 20 read_log 193305'18232 (193305'18231) modify bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.951337.0:30065 2014-08-11 18:44:23.223808 2014-08-13 18:06:06.901381 7f5c973a57a0 20 read_log 193305'18233 (0'0) promote d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by osd.4.0:511 2014-08-11 18:44:28.704864 2014-08-13 18:06:06.901408 7f5c973a57a0 20 read_log 193305'18234 (193305'18233) modify d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.951337.0:31541 2014-08-11 18:44:28.697550 2014-08-13 18:06:06.901423 7f5c973a57a0 20 read_log 193305'18235 (0'0) promote d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by osd.4.0:529 2014-08-11 18:44:30.561501 2014-08-13 18:06:06.901442 7f5c973a57a0 20 read_log 193305'18236 (193305'18235) modify d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.951337.0:32085 2014-08-11 18:44:30.551483 2014-08-13 18:06:06.901460 7f5c973a57a0 20 read_log 193305'18237 (0'0) promote b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by osd.4.0:541 2014-08-11 18:44:31.441198 2014-08-13 18:06:06.901473 7f5c973a57a0 20 read_log 193305'18238 (193305'18237) modify b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.951337.0:32316 2014-08-11 18:44:31.431448 2014-08-13 18:06:06.901485 7f5c973a57a0 20 read_log 193305'18239 (0'0) promote 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by osd.4.0:543 2014-08-11 18:44:31.594003 2014-08-13 18:06:06.901506 7f5c973a57a0 20 read_log 193305'18240 (193305'18239) modify 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.951337.0:32357 2014-08-11 18:44:31.584382 2014-08-13 18:06:06.901522 7f5c973a57a0 20 read_log 193305'18241 (193305'18170) modify ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.951337.0:34641 2014-08-11 18:44:35.109648 2014-08-13 18:06:06.901535 7f5c973a57a0 20 read_log 193305'18242 (193305'18172) modify df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.951337.0:40762 2014-08-11 18:44:38.519031 2014-08-13 18:06:06.901547 7f5c973a57a0 20 read_log 193305'18243 (193305'18174) modify d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.951337.0:40878 2014-08-11 18:44:38.564422 2014-08-13 18:06:06.901559 7f5c973a57a0 20 read_log 193305'18244 (193305'18176) modify 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.951337.0:41087 2014-08-11 18:44:38.630794 2014-08-13 18:06:06.901572 7f5c973a57a0 20 read_log 193305'18245 (193305'18178) modify a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.951337.0:41993 2014-08-11 18:44:38.973406 2014-08-13 18:06:06.901584 7f5c973a57a0 20 read_log 193305'18246 (193305'18180) modify 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.951337.0:42287 2014-08-11 18:44:39.175560 2014-08-13 18:06:06.901596 7f5c973a57a0 20 read_log 193305'18247 (193305'18182) modify 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.951337.0:42681 2014-08-11 18:44:39.345429 2014-08-13 18:06:06.901608 7f5c973a57a0 20 read_log 193305'18248 (193305'18184) modify 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.951337.0:44723 2014-08-11 18:44:39.970216 2014-08-13 18:06:06.901624 7f5c973a57a0 20 read_log 193305'18249 (193305'18186) modify 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.951337.0:44945 2014-08-11 18:44:40.044497 2014-08-13 18:06:06.901646 7f5c973a57a0 20 read_log 193305'18250 (193305'18188) modify 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.951337.0:45042 2014-08-11 18:44:40.075987 2014-08-13 18:06:06.901669 7f5c973a57a0 20 read_log 193305'18251 (193305'18190) modify 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.951337.0:45070 2014-08-11 18:44:40.082308 2014-08-13 18:06:06.901684 7f5c973a57a0 20 read_log 193305'18252 (193305'18192) modify a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.951337.0:45807 2014-08-11 18:44:40.308289 2014-08-13 18:06:06.901697 7f5c973a57a0 20 read_log 193305'18253 (193305'18194) modify 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.951337.0:46319 2014-08-11 18:44:40.451735 2014-08-13 18:06:06.901709 7f5c973a57a0 20 read_log 193305'18254 (193305'18196) modify a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.951337.0:46773 2014-08-11 18:44:40.584809 2014-08-13 18:06:06.901721 7f5c973a57a0 20 read_log 193305'18255 (193305'18198) modify 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.951337.0:47859 2014-08-11 18:44:40.886571 2014-08-13 18:06:06.901733 7f5c973a57a0 20 read_log 193305'18256 (193305'18200) modify 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.951337.0:50644 2014-08-11 18:44:41.724179 2014-08-13 18:06:06.901745 7f5c973a57a0 20 read_log 193305'18257 (193305'18202) modify 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.951337.0:50716 2014-08-11 18:44:41.743185 2014-08-13 18:06:06.901757 7f5c973a57a0 20 read_log 193305'18258 (193305'18204) modify 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.951337.0:51283 2014-08-11 18:44:41.905944 2014-08-13 18:06:06.901770 7f5c973a57a0 20 read_log 193305'18259 (193305'18206) modify e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.951337.0:51751 2014-08-11 18:44:42.040053 2014-08-13 18:06:06.901783 7f5c973a57a0 20 read_log 193305'18260 (193305'18208) modify c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.951337.0:52942 2014-08-11 18:44:42.386029 2014-08-13 18:06:06.901795 7f5c973a57a0 20 read_log 193305'18261 (193305'18210) modify bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.951337.0:53247 2014-08-11 18:44:42.473956 2014-08-13 18:06:06.901807 7f5c973a57a0 20 read_log 193305'18262 (193305'18212) modify 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.951337.0:55288 2014-08-11 18:44:43.079223 2014-08-13 18:06:06.901819 7f5c973a57a0 20 read_log 193305'18263 (193305'18214) modify 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.951337.0:55426 2014-08-11 18:44:43.119821 2014-08-13 18:06:06.901832 7f5c973a57a0 20 read_log 193305'18264 (193305'18216) modify 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.951337.0:56787 2014-08-11 18:44:43.529643 2014-08-13 18:06:06.901845 7f5c973a57a0 20 read_log 193305'18265 (193305'18218) modify 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.951337.0:56807 2014-08-11 18:44:43.536332 2014-08-13 18:06:06.901857 7f5c973a57a0 20 read_log 193305'18266 (193305'18220) modify 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.951337.0:58064 2014-08-11 18:44:43.936225 2014-08-13 18:06:06.901869 7f5c973a57a0 20 read_log 193305'18267 (193305'18222) modify 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.951337.0:58825 2014-08-11 18:44:44.152212 2014-08-13 18:06:06.901881 7f5c973a57a0 20 read_log 193305'18268 (193305'18224) modify 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.951337.0:59853 2014-08-11 18:44:44.446654 2014-08-13 18:06:06.901893 7f5c973a57a0 20 read_log 193305'18269 (193305'18226) modify 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.951337.0:59907 2014-08-11 18:44:44.464127 2014-08-13 18:06:06.901905 7f5c973a57a0 20 read_log 193305'18270 (193305'18228) modify 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.951337.0:61156 2014-08-11 18:44:44.838716 2014-08-13 18:06:06.901925 7f5c973a57a0 20 read_log 193305'18271 (193305'18230) modify 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.951337.0:61413 2014-08-11 18:44:44.914604 2014-08-13 18:06:06.901940 7f5c973a57a0 20 read_log 193305'18272 (193305'18232) modify bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.951337.0:62776 2014-08-11 18:44:45.324090 2014-08-13 18:06:06.901953 7f5c973a57a0 20 read_log 193305'18273 (193305'18234) modify d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.951337.0:64252 2014-08-11 18:44:45.747289 2014-08-13 18:06:06.901967 7f5c973a57a0 20 read_log 193305'18274 (193305'18236) modify d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.951337.0:64796 2014-08-11 18:44:45.919881 2014-08-13 18:06:06.901981 7f5c973a57a0 20 read_log 193305'18275 (193305'18238) modify b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.951337.0:65027 2014-08-11 18:44:45.991668 2014-08-13 18:06:06.901994 7f5c973a57a0 20 read_log 193305'18276 (193305'18240) modify 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.951337.0:65068 2014-08-11 18:44:46.007445 2014-08-13 18:06:06.902006 7f5c973a57a0 20 read_log 194215'18277 (194215'18277) modify 30b/hit_set_24.30b_archive_2014-08-12 15:32:25.863655_2014-08-12 15:48:15.603804/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:15.604108 2014-08-13 18:06:06.902019 7f5c973a57a0 20 read_log 194215'18278 (193305'18168) delete 30b/hit_set_24.30b_archive_2014-08-11 18:38:11.461328_2014-08-11 18:42:40.507251/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:15.604108 2014-08-13 18:06:06.902033 7f5c973a57a0 20 read_log 194215'18279 (192834'17831) clean da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by osd.4.0:2499 2014-08-12 15:48:15.614058 2014-08-13 18:06:06.902046 7f5c973a57a0 20 read_log 194215'18280 (194215'18279) delete da96030b/benchmark_data_storage0111-ib.pocstor_41526_object21136/head//24 by client.2174595.0:160490 0.000000 2014-08-13 18:06:06.902057 7f5c973a57a0 20 read_log 194215'18281 (192888'17962) clean 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by osd.4.0:2501 2014-08-12 15:48:15.622292 2014-08-13 18:06:06.902072 7f5c973a57a0 20 read_log 194215'18282 (194215'18281) delete 7ea9030b/benchmark_data_storage0111-ib.pocstor_41097_object16000/head//24 by client.2174595.0:160492 0.000000 2014-08-13 18:06:06.902083 7f5c973a57a0 20 read_log 194215'18283 (193305'18250) clean 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by osd.4.0:2503 2014-08-12 15:48:15.675695 2014-08-13 18:06:06.902095 7f5c973a57a0 20 read_log 194215'18284 (194215'18283) delete 71ac030b/benchmark_data_storage0111-ib.pocstor_62621_object12330/head//24 by client.2174595.0:160494 0.000000 2014-08-13 18:06:06.902106 7f5c973a57a0 20 read_log 194215'18285 (193305'18258) clean 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by osd.4.0:2505 2014-08-12 15:48:15.692726 2014-08-13 18:06:06.902120 7f5c973a57a0 20 read_log 194215'18286 (194215'18285) delete 7fbd030b/benchmark_data_storage0111-ib.pocstor_62621_object18571/head//24 by client.2174595.0:160496 0.000000 2014-08-13 18:06:06.902131 7f5c973a57a0 20 read_log 194215'18287 (193305'18256) clean 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by osd.4.0:2507 2014-08-12 15:48:15.700465 2014-08-13 18:06:06.902143 7f5c973a57a0 20 read_log 194215'18288 (194215'18287) delete 8b76130b/benchmark_data_storage0111-ib.pocstor_62621_object17932/head//24 by client.2174595.0:160498 0.000000 2014-08-13 18:06:06.902162 7f5c973a57a0 20 read_log 194215'18289 (193305'18274) clean d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by osd.4.0:2509 2014-08-12 15:48:15.708847 2014-08-13 18:06:06.902182 7f5c973a57a0 20 read_log 194215'18290 (194215'18289) delete d10c130b/benchmark_data_storage0111-ib.pocstor_62621_object32084/head//24 by client.2174595.0:160500 0.000000 2014-08-13 18:06:06.902196 7f5c973a57a0 20 read_log 194215'18291 (192888'18136) clean 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by osd.4.0:2511 2014-08-12 15:48:15.719121 2014-08-13 18:06:06.902208 7f5c973a57a0 20 read_log 194215'18292 (194215'18291) delete 3a13430b/benchmark_data_storage0111-ib.pocstor_2228_object13891/head//24 by client.2174595.0:160502 0.000000 2014-08-13 18:06:06.902220 7f5c973a57a0 20 read_log 194215'18293 (193305'18241) clean ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by osd.4.0:2513 2014-08-12 15:48:15.728509 2014-08-13 18:06:06.902233 7f5c973a57a0 20 read_log 194215'18294 (194215'18293) delete ab5b430b/benchmark_data_storage0111-ib.pocstor_62621_object1929/head//24 by client.2174595.0:160504 0.000000 2014-08-13 18:06:06.902244 7f5c973a57a0 20 read_log 194215'18295 (192888'18128) clean 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by osd.4.0:2515 2014-08-12 15:48:15.738295 2014-08-13 18:06:06.902261 7f5c973a57a0 20 read_log 194215'18296 (194215'18295) delete 8c46530b/benchmark_data_storage0111-ib.pocstor_2228_object8746/head//24 by client.2174595.0:160506 0.000000 2014-08-13 18:06:06.902273 7f5c973a57a0 20 read_log 194215'18297 (192888'18127) clean 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by osd.4.0:2517 2014-08-12 15:48:15.748184 2014-08-13 18:06:06.902285 7f5c973a57a0 20 read_log 194215'18298 (194215'18297) delete 2a09530b/benchmark_data_storage0111-ib.pocstor_2228_object8172/head//24 by client.2174595.0:160508 0.000000 2014-08-13 18:06:06.902296 7f5c973a57a0 20 read_log 194215'18299 (192834'17772) clean 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by osd.4.0:2519 2014-08-12 15:48:15.758269 2014-08-13 18:06:06.902311 7f5c973a57a0 20 read_log 194215'18300 (194215'18299) delete 9183630b/benchmark_data_storage0111-ib.pocstor_41526_object12325/head//24 by client.2174595.0:160510 0.000000 2014-08-13 18:06:06.902322 7f5c973a57a0 20 read_log 194215'18301 (192888'18153) clean 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by osd.4.0:2521 2014-08-12 15:48:15.767103 2014-08-13 18:06:06.902356 7f5c973a57a0 20 read_log 194215'18302 (194215'18301) delete 61c4630b/benchmark_data_storage0111-ib.pocstor_2228_object21348/head//24 by client.2174595.0:160512 0.000000 2014-08-13 18:06:06.902371 7f5c973a57a0 20 read_log 194215'18303 (192834'17192) clean a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by osd.4.0:2523 2014-08-12 15:48:15.779603 2014-08-13 18:06:06.902384 7f5c973a57a0 20 read_log 194215'18304 (194215'18303) delete a37b630b/benchmark_data_storage0111-ib.pocstor_26659_object5796/head//24 by client.2174595.0:160514 0.000000 2014-08-13 18:06:06.902406 7f5c973a57a0 20 read_log 194215'18305 (192834'17251) clean e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by osd.4.0:2525 2014-08-12 15:48:15.787502 2014-08-13 18:06:06.902423 7f5c973a57a0 20 read_log 194215'18306 (194215'18305) delete e0ac630b/benchmark_data_storage0111-ib.pocstor_26659_object11370/head//24 by client.2174595.0:160516 0.000000 2014-08-13 18:06:06.902434 7f5c973a57a0 20 read_log 194215'18307 (192888'17952) clean 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by osd.4.0:2527 2014-08-12 15:48:15.795148 2014-08-13 18:06:06.902454 7f5c973a57a0 20 read_log 194215'18308 (194215'18307) delete 7c31730b/benchmark_data_storage0111-ib.pocstor_41097_object11117/head//24 by client.2174595.0:160518 0.000000 2014-08-13 18:06:06.902468 7f5c973a57a0 20 read_log 194215'18309 (193305'18270) clean 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by osd.4.0:2529 2014-08-12 15:48:15.805318 2014-08-13 18:06:06.902500 7f5c973a57a0 20 read_log 194215'18310 (194215'18309) delete 9e2b730b/benchmark_data_storage0111-ib.pocstor_62621_object28444/head//24 by client.2174595.0:160520 0.000000 2014-08-13 18:06:06.902516 7f5c973a57a0 20 read_log 194215'18311 (192888'17945) clean 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by osd.4.0:2531 2014-08-12 15:48:15.814257 2014-08-13 18:06:06.902550 7f5c973a57a0 20 read_log 194215'18312 (194215'18311) delete 151c730b/benchmark_data_storage0111-ib.pocstor_41097_object3632/head//24 by client.2174595.0:160522 0.000000 2014-08-13 18:06:06.902562 7f5c973a57a0 20 read_log 194215'18313 (192834'17236) clean 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by osd.4.0:2533 2014-08-12 15:48:15.823922 2014-08-13 18:06:06.902575 7f5c973a57a0 20 read_log 194215'18314 (194215'18313) delete 1fe3930b/benchmark_data_storage0111-ib.pocstor_26659_object9564/head//24 by client.2174595.0:160524 0.000000 2014-08-13 18:06:06.902599 7f5c973a57a0 20 read_log 194215'18315 (192888'18134) clean e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by osd.4.0:2535 2014-08-12 15:48:15.833671 2014-08-13 18:06:06.902616 7f5c973a57a0 20 read_log 194215'18316 (194215'18315) delete e2eb930b/benchmark_data_storage0111-ib.pocstor_2228_object13405/head//24 by client.2174595.0:160526 0.000000 2014-08-13 18:06:06.902627 7f5c973a57a0 20 read_log 194215'18317 (192834'17171) clean 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by osd.4.0:2537 2014-08-12 15:48:15.842926 2014-08-13 18:06:06.902640 7f5c973a57a0 20 read_log 194215'18318 (194215'18317) delete 85c9a30b/benchmark_data_storage0111-ib.pocstor_26659_object2527/head//24 by client.2174595.0:160528 0.000000 2014-08-13 18:06:06.902651 7f5c973a57a0 20 read_log 194215'18319 (192888'17946) clean a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by osd.4.0:2539 2014-08-12 15:48:15.852239 2014-08-13 18:06:06.902664 7f5c973a57a0 20 read_log 194215'18320 (194215'18319) delete a7bda30b/benchmark_data_storage0111-ib.pocstor_41097_object3664/head//24 by client.2174595.0:160530 0.000000 2014-08-13 18:06:06.902675 7f5c973a57a0 20 read_log 194215'18321 (192834'17151) clean a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by osd.4.0:2541 2014-08-12 15:48:15.860577 2014-08-13 18:06:06.902687 7f5c973a57a0 20 read_log 194215'18322 (194215'18321) delete a272b30b/benchmark_data_storage0111-ib.pocstor_26659_object309/head//24 by client.2174595.0:160532 0.000000 2014-08-13 18:06:06.902698 7f5c973a57a0 20 read_log 194215'18323 (192834'17716) clean 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by osd.4.0:2543 2014-08-12 15:48:15.868447 2014-08-13 18:06:06.902719 7f5c973a57a0 20 read_log 194215'18324 (194215'18323) delete 7756b30b/benchmark_data_storage0111-ib.pocstor_41526_object5550/head//24 by client.2174595.0:160534 0.000000 2014-08-13 18:06:06.902730 7f5c973a57a0 20 read_log 194215'18325 (192888'17941) clean aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by osd.4.0:2545 2014-08-12 15:48:15.877531 2014-08-13 18:06:06.902743 7f5c973a57a0 20 read_log 194215'18326 (194215'18325) delete aa99b30b/benchmark_data_storage0111-ib.pocstor_41097_object123/head//24 by client.2174595.0:160536 0.000000 2014-08-13 18:06:06.902753 7f5c973a57a0 20 read_log 194215'18327 (192888'18159) clean cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by osd.4.0:2547 2014-08-12 15:48:15.885897 2014-08-13 18:06:06.902766 7f5c973a57a0 20 read_log 194215'18328 (194215'18327) delete cd84c30b/benchmark_data_storage0111-ib.pocstor_2228_object24280/head//24 by client.2174595.0:160538 0.000000 2014-08-13 18:06:06.902777 7f5c973a57a0 20 read_log 194215'18329 (193305'18261) clean bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by osd.4.0:2549 2014-08-12 15:48:15.897343 2014-08-13 18:06:06.902797 7f5c973a57a0 20 read_log 194215'18330 (194215'18329) delete bb68c30b/benchmark_data_storage0111-ib.pocstor_62621_object20535/head//24 by client.2174595.0:160540 0.000000 2014-08-13 18:06:06.902809 7f5c973a57a0 20 read_log 194215'18331 (193305'18245) clean a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by osd.4.0:2551 2014-08-12 15:48:15.907064 2014-08-13 18:06:06.902821 7f5c973a57a0 20 read_log 194215'18332 (194215'18331) delete a5fec30b/benchmark_data_storage0111-ib.pocstor_62621_object9281/head//24 by client.2174595.0:160542 0.000000 2014-08-13 18:06:06.902832 7f5c973a57a0 20 read_log 194215'18333 (193305'18260) clean c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by osd.4.0:2553 2014-08-12 15:48:15.916382 2014-08-13 18:06:06.902844 7f5c973a57a0 20 read_log 194215'18334 (194215'18333) delete c028e30b/benchmark_data_storage0111-ib.pocstor_62621_object20230/head//24 by client.2174595.0:160544 0.000000 2014-08-13 18:06:06.902855 7f5c973a57a0 20 read_log 194215'18335 (193305'18271) clean 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by osd.4.0:2555 2014-08-12 15:48:15.924967 2014-08-13 18:06:06.902867 7f5c973a57a0 20 read_log 194215'18336 (194215'18335) delete 9a90f30b/benchmark_data_storage0111-ib.pocstor_62621_object28701/head//24 by client.2174595.0:160546 0.000000 2014-08-13 18:06:06.902878 7f5c973a57a0 20 read_log 194215'18337 (192888'17955) clean e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by osd.4.0:2557 2014-08-12 15:48:15.933508 2014-08-13 18:06:06.902899 7f5c973a57a0 20 read_log 194215'18338 (194215'18337) delete e614f30b/benchmark_data_storage0111-ib.pocstor_41097_object12703/head//24 by client.2174595.0:160548 0.000000 2014-08-13 18:06:06.902911 7f5c973a57a0 20 read_log 194215'18339 (192834'17787) clean d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by osd.4.0:2559 2014-08-12 15:48:15.943347 2014-08-13 18:06:06.902923 7f5c973a57a0 20 read_log 194215'18340 (194215'18339) delete d944f30b/benchmark_data_storage0111-ib.pocstor_41526_object14020/head//24 by client.2174595.0:160550 0.000000 2014-08-13 18:06:06.902934 7f5c973a57a0 20 read_log 194215'18341 (192834'17319) clean 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by osd.4.0:2561 2014-08-12 15:48:15.952390 2014-08-13 18:06:06.902946 7f5c973a57a0 20 read_log 194215'18342 (194215'18341) delete 775ef30b/benchmark_data_storage0111-ib.pocstor_26659_object19073/head//24 by client.2174595.0:160552 0.000000 2014-08-13 18:06:06.902957 7f5c973a57a0 20 read_log 194215'18343 (193305'18253) clean 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by osd.4.0:2563 2014-08-12 15:48:16.006517 2014-08-13 18:06:06.902972 7f5c973a57a0 20 read_log 194215'18344 (194215'18343) delete 6564070b/benchmark_data_storage0111-ib.pocstor_62621_object13607/head//24 by client.2174595.0:160554 0.000000 2014-08-13 18:06:06.902983 7f5c973a57a0 20 read_log 194215'18345 (192888'17983) clean 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by osd.4.0:2565 2014-08-12 15:48:16.014932 2014-08-13 18:06:06.902995 7f5c973a57a0 20 read_log 194215'18346 (194215'18345) delete 4b1f070b/benchmark_data_storage0111-ib.pocstor_41097_object29087/head//24 by client.2174595.0:160556 0.000000 2014-08-13 18:06:06.903006 7f5c973a57a0 20 read_log 194215'18347 (192834'17769) clean 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by osd.4.0:2567 2014-08-12 15:48:16.023008 2014-08-13 18:06:06.903019 7f5c973a57a0 20 read_log 194215'18348 (194215'18347) delete 84170b/benchmark_data_storage0111-ib.pocstor_41526_object12006/head//24 by client.2174595.0:160558 0.000000 2014-08-13 18:06:06.903029 7f5c973a57a0 20 read_log 194215'18349 (192888'18111) clean e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by osd.4.0:2569 2014-08-12 15:48:16.031143 2014-08-13 18:06:06.903073 7f5c973a57a0 20 read_log 194215'18350 (194215'18349) delete e60a170b/benchmark_data_storage0111-ib.pocstor_2228_object1928/head//24 by client.2174595.0:160560 0.000000 2014-08-13 18:06:06.903101 7f5c973a57a0 20 read_log 194215'18351 (192834'17767) clean 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by osd.4.0:2571 2014-08-12 15:48:16.039399 2014-08-13 18:06:06.903119 7f5c973a57a0 20 read_log 194215'18352 (194215'18351) delete 656c270b/benchmark_data_storage0111-ib.pocstor_41526_object11773/head//24 by client.2174595.0:160562 0.000000 2014-08-13 18:06:06.903131 7f5c973a57a0 20 read_log 194215'18353 (193305'18268) clean 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by osd.4.0:2573 2014-08-12 15:48:16.048353 2014-08-13 18:06:06.903161 7f5c973a57a0 20 read_log 194215'18354 (194215'18353) delete 8660370b/benchmark_data_storage0111-ib.pocstor_62621_object27141/head//24 by client.2174595.0:160564 0.000000 2014-08-13 18:06:06.903176 7f5c973a57a0 20 read_log 194215'18355 (192834'17340) clean a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by osd.4.0:2575 2014-08-12 15:48:16.056757 2014-08-13 18:06:06.903200 7f5c973a57a0 20 read_log 194215'18356 (194215'18355) delete a190370b/benchmark_data_storage0111-ib.pocstor_26659_object20732/head//24 by client.2174595.0:160566 0.000000 2014-08-13 18:06:06.903215 7f5c973a57a0 20 read_log 194215'18357 (192834'17242) clean e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by osd.4.0:2577 2014-08-12 15:48:16.065638 2014-08-13 18:06:06.903233 7f5c973a57a0 20 read_log 194215'18358 (194215'18357) delete e058470b/benchmark_data_storage0111-ib.pocstor_26659_object10498/head//24 by client.2174595.0:160568 0.000000 2014-08-13 18:06:06.903245 7f5c973a57a0 20 read_log 194215'18359 (192834'17847) clean f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by osd.4.0:2579 2014-08-12 15:48:16.074161 2014-08-13 18:06:06.903258 7f5c973a57a0 20 read_log 194215'18360 (194215'18359) delete f7d5570b/benchmark_data_storage0111-ib.pocstor_41526_object22777/head//24 by client.2174595.0:160570 0.000000 2014-08-13 18:06:06.903269 7f5c973a57a0 20 read_log 194215'18361 (192834'17823) clean 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by osd.4.0:2581 2014-08-12 15:48:16.082421 2014-08-13 18:06:06.903283 7f5c973a57a0 20 read_log 194215'18362 (194215'18361) delete 6a23670b/benchmark_data_storage0111-ib.pocstor_41526_object20058/head//24 by client.2174595.0:160572 0.000000 2014-08-13 18:06:06.903295 7f5c973a57a0 20 read_log 194215'18363 (192888'17943) clean 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by osd.4.0:2583 2014-08-12 15:48:16.090435 2014-08-13 18:06:06.903307 7f5c973a57a0 20 read_log 194215'18364 (194215'18363) delete 95e4770b/benchmark_data_storage0111-ib.pocstor_41097_object1541/head//24 by client.2174595.0:160574 0.000000 2014-08-13 18:06:06.903318 7f5c973a57a0 20 read_log 194215'18365 (192834'17352) clean d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by osd.4.0:2585 2014-08-12 15:48:16.098066 2014-08-13 18:06:06.903331 7f5c973a57a0 20 read_log 194215'18366 (194215'18365) delete d07770b/benchmark_data_storage0111-ib.pocstor_26659_object21896/head//24 by client.2174595.0:160576 0.000000 2014-08-13 18:06:06.903349 7f5c973a57a0 20 read_log 194215'18367 (192888'17949) clean 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by osd.4.0:2587 2014-08-12 15:48:16.105486 2014-08-13 18:06:06.903366 7f5c973a57a0 20 read_log 194215'18368 (194215'18367) delete 1abd970b/benchmark_data_storage0111-ib.pocstor_41097_object9169/head//24 by client.2174595.0:160578 0.000000 2014-08-13 18:06:06.903377 7f5c973a57a0 20 read_log 194215'18369 (192834'17299) clean 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by osd.4.0:2589 2014-08-12 15:48:16.112601 2014-08-13 18:06:06.903400 7f5c973a57a0 20 read_log 194215'18370 (194215'18369) delete 6c5f970b/benchmark_data_storage0111-ib.pocstor_26659_object16804/head//24 by client.2174595.0:160580 0.000000 2014-08-13 18:06:06.903414 7f5c973a57a0 20 read_log 194215'18371 (192834'17817) clean ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by osd.4.0:2591 2014-08-12 15:48:16.120558 2014-08-13 18:06:06.903427 7f5c973a57a0 20 read_log 194215'18372 (194215'18371) delete ce10a70b/benchmark_data_storage0111-ib.pocstor_41526_object19420/head//24 by client.2174595.0:160582 0.000000 2014-08-13 18:06:06.903438 7f5c973a57a0 20 read_log 194215'18373 (192834'17322) clean 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by osd.4.0:2593 2014-08-12 15:48:16.129488 2014-08-13 18:06:06.903450 7f5c973a57a0 20 read_log 194215'18374 (194215'18373) delete 9424a70b/benchmark_data_storage0111-ib.pocstor_26659_object19222/head//24 by client.2174595.0:160584 0.000000 2014-08-13 18:06:06.903461 7f5c973a57a0 20 read_log 194215'18375 (192834'17250) clean ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by osd.4.0:2595 2014-08-12 15:48:16.138879 2014-08-13 18:06:06.903493 7f5c973a57a0 20 read_log 194215'18376 (194215'18375) delete ed09a70b/benchmark_data_storage0111-ib.pocstor_26659_object11346/head//24 by client.2174595.0:160586 0.000000 2014-08-13 18:06:06.903508 7f5c973a57a0 20 read_log 194215'18377 (192834'17158) clean e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by osd.4.0:2597 2014-08-12 15:48:16.149562 2014-08-13 18:06:06.903521 7f5c973a57a0 20 read_log 194215'18378 (194215'18377) delete e10ca70b/benchmark_data_storage0111-ib.pocstor_26659_object1119/head//24 by client.2174595.0:160588 0.000000 2014-08-13 18:06:06.903532 7f5c973a57a0 20 read_log 194215'18379 (192888'17972) clean a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by osd.4.0:2599 2014-08-12 15:48:16.158418 2014-08-13 18:06:06.903545 7f5c973a57a0 20 read_log 194215'18380 (194215'18379) delete a269b70b/benchmark_data_storage0111-ib.pocstor_41097_object22504/head//24 by client.2174595.0:160590 0.000000 2014-08-13 18:06:06.903556 7f5c973a57a0 20 read_log 194215'18381 (192888'18114) clean 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by osd.4.0:2601 2014-08-12 15:48:16.167277 2014-08-13 18:06:06.903568 7f5c973a57a0 20 read_log 194215'18382 (194215'18381) delete 327cb70b/benchmark_data_storage0111-ib.pocstor_2228_object4009/head//24 by client.2174595.0:160592 0.000000 2014-08-13 18:06:06.903579 7f5c973a57a0 20 read_log 194215'18383 (192888'18109) clean c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by osd.4.0:2603 2014-08-12 15:48:16.177775 2014-08-13 18:06:06.903591 7f5c973a57a0 20 read_log 194215'18384 (194215'18383) delete c620c70b/benchmark_data_storage0111-ib.pocstor_2228_object1394/head//24 by client.2174595.0:160594 0.000000 2014-08-13 18:06:06.903602 7f5c973a57a0 20 read_log 194215'18385 (192888'18147) clean 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by osd.4.0:2605 2014-08-12 15:48:16.186554 2014-08-13 18:06:06.903614 7f5c973a57a0 20 read_log 194215'18386 (194215'18385) delete 4079d70b/benchmark_data_storage0111-ib.pocstor_2228_object20284/head//24 by client.2174595.0:160596 0.000000 2014-08-13 18:06:06.903625 7f5c973a57a0 20 read_log 194215'18387 (192834'17248) clean c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by osd.4.0:2607 2014-08-12 15:48:16.196497 2014-08-13 18:06:06.903637 7f5c973a57a0 20 read_log 194215'18388 (194215'18387) delete c59ad70b/benchmark_data_storage0111-ib.pocstor_26659_object10988/head//24 by client.2174595.0:160598 0.000000 2014-08-13 18:06:06.903648 7f5c973a57a0 20 read_log 194215'18389 (192834'17269) clean 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by osd.4.0:2609 2014-08-12 15:48:16.205487 2014-08-13 18:06:06.903668 7f5c973a57a0 20 read_log 194215'18390 (194215'18389) delete 921dd70b/benchmark_data_storage0111-ib.pocstor_26659_object13291/head//24 by client.2174595.0:160600 0.000000 2014-08-13 18:06:06.903679 7f5c973a57a0 20 read_log 194215'18391 (192888'18151) clean e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by osd.4.0:2611 2014-08-12 15:48:16.213286 2014-08-13 18:06:06.903694 7f5c973a57a0 20 read_log 194215'18392 (194215'18391) delete e9fdd70b/benchmark_data_storage0111-ib.pocstor_2228_object21091/head//24 by client.2174595.0:160602 0.000000 2014-08-13 18:06:06.903705 7f5c973a57a0 20 read_log 194215'18393 (192888'18157) clean 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by osd.4.0:2613 2014-08-12 15:48:16.221772 2014-08-13 18:06:06.903718 7f5c973a57a0 20 read_log 194215'18394 (194215'18393) delete 85dfd70b/benchmark_data_storage0111-ib.pocstor_2228_object22493/head//24 by client.2174595.0:160604 0.000000 2014-08-13 18:06:06.903748 7f5c973a57a0 20 read_log 194215'18395 (192834'17786) clean 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by osd.4.0:2615 2014-08-12 15:48:16.230497 2014-08-13 18:06:06.903763 7f5c973a57a0 20 read_log 194215'18396 (194215'18395) delete 9b39e70b/benchmark_data_storage0111-ib.pocstor_41526_object13929/head//24 by client.2174595.0:160606 0.000000 2014-08-13 18:06:06.903774 7f5c973a57a0 20 read_log 194215'18397 (193305'18275) clean b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by osd.4.0:2617 2014-08-12 15:48:16.240639 2014-08-13 18:06:06.903786 7f5c973a57a0 20 read_log 194215'18398 (194215'18397) delete b112f70b/benchmark_data_storage0111-ib.pocstor_62621_object32315/head//24 by client.2174595.0:160608 0.000000 2014-08-13 18:06:06.903797 7f5c973a57a0 20 read_log 194215'18399 (192834'17799) clean c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by osd.4.0:2619 2014-08-12 15:48:16.250388 2014-08-13 18:06:06.903810 7f5c973a57a0 20 read_log 194215'18400 (194215'18399) delete c674f70b/benchmark_data_storage0111-ib.pocstor_41526_object16860/head//24 by client.2174595.0:160610 0.000000 2014-08-13 18:06:06.903820 7f5c973a57a0 20 read_log 194215'18401 (193305'18276) clean 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by osd.4.0:2621 2014-08-12 15:48:16.258931 2014-08-13 18:06:06.903833 7f5c973a57a0 20 read_log 194215'18402 (194215'18401) delete 3f000b0b/benchmark_data_storage0111-ib.pocstor_62621_object32356/head//24 by client.2174595.0:160612 0.000000 2014-08-13 18:06:06.903844 7f5c973a57a0 20 read_log 194215'18403 (193305'18272) clean bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by osd.4.0:2623 2014-08-12 15:48:16.267409 2014-08-13 18:06:06.903858 7f5c973a57a0 20 read_log 194215'18404 (194215'18403) delete bd130b0b/benchmark_data_storage0111-ib.pocstor_62621_object30064/head//24 by client.2174595.0:160614 0.000000 2014-08-13 18:06:06.903869 7f5c973a57a0 20 read_log 194215'18405 (193305'18248) clean 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by osd.4.0:2625 2014-08-12 15:48:16.275936 2014-08-13 18:06:06.903883 7f5c973a57a0 20 read_log 194215'18406 (194215'18405) delete 2d101b0b/benchmark_data_storage0111-ib.pocstor_62621_object12011/head//24 by client.2174595.0:160616 0.000000 2014-08-13 18:06:06.903896 7f5c973a57a0 20 read_log 194215'18407 (193305'18252) clean a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by osd.4.0:2627 2014-08-12 15:48:16.286200 2014-08-13 18:06:06.903908 7f5c973a57a0 20 read_log 194215'18408 (194215'18407) delete a3241b0b/benchmark_data_storage0111-ib.pocstor_62621_object13095/head//24 by client.2174595.0:160618 0.000000 2014-08-13 18:06:06.903919 7f5c973a57a0 20 read_log 194215'18409 (192888'18164) clean d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by osd.4.0:2629 2014-08-12 15:48:16.294691 2014-08-13 18:06:06.903940 7f5c973a57a0 20 read_log 194215'18410 (194215'18409) delete d4f81b0b/benchmark_data_storage0111-ib.pocstor_2228_object29220/head//24 by client.2174595.0:160620 0.000000 2014-08-13 18:06:06.903954 7f5c973a57a0 20 read_log 194215'18411 (192888'18138) clean 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by osd.4.0:2631 2014-08-12 15:48:16.302788 2014-08-13 18:06:06.903973 7f5c973a57a0 20 read_log 194215'18412 (194215'18411) delete 9d772b0b/benchmark_data_storage0111-ib.pocstor_2228_object14713/head//24 by client.2174595.0:160622 0.000000 2014-08-13 18:06:06.903984 7f5c973a57a0 20 read_log 194215'18413 (192834'17222) clean 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by osd.4.0:2633 2014-08-12 15:48:16.311194 2014-08-13 18:06:06.903998 7f5c973a57a0 20 read_log 194215'18414 (194215'18413) delete 2ad82b0b/benchmark_data_storage0111-ib.pocstor_26659_object8609/head//24 by client.2174595.0:160624 0.000000 2014-08-13 18:06:06.904013 7f5c973a57a0 20 read_log 194215'18415 (193305'18244) clean 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by osd.4.0:2635 2014-08-12 15:48:16.319498 2014-08-13 18:06:06.904028 7f5c973a57a0 20 read_log 194215'18416 (194215'18415) delete 7f6f2b0b/benchmark_data_storage0111-ib.pocstor_62621_object8375/head//24 by client.2174595.0:160626 0.000000 2014-08-13 18:06:06.904039 7f5c973a57a0 20 read_log 194215'18417 (193305'18247) clean 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by osd.4.0:2637 2014-08-12 15:48:16.328340 2014-08-13 18:06:06.904053 7f5c973a57a0 20 read_log 194215'18418 (194215'18417) delete 62493b0b/benchmark_data_storage0111-ib.pocstor_62621_object9969/head//24 by client.2174595.0:160628 0.000000 2014-08-13 18:06:06.904064 7f5c973a57a0 20 read_log 194215'18419 (192834'17343) clean 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by osd.4.0:2639 2014-08-12 15:48:16.337192 2014-08-13 18:06:06.904077 7f5c973a57a0 20 read_log 194215'18420 (194215'18419) delete 29394b0b/benchmark_data_storage0111-ib.pocstor_26659_object20999/head//24 by client.2174595.0:160630 0.000000 2014-08-13 18:06:06.904088 7f5c973a57a0 20 read_log 194215'18421 (192834'17325) clean c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by osd.4.0:2641 2014-08-12 15:48:16.346718 2014-08-13 18:06:06.904100 7f5c973a57a0 20 read_log 194215'18422 (194215'18421) delete c83e4b0b/benchmark_data_storage0111-ib.pocstor_26659_object19378/head//24 by client.2174595.0:160632 0.000000 2014-08-13 18:06:06.904112 7f5c973a57a0 20 read_log 194215'18423 (192888'17950) clean 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by osd.4.0:2643 2014-08-12 15:48:16.355336 2014-08-13 18:06:06.904125 7f5c973a57a0 20 read_log 194215'18424 (194215'18423) delete 496a5b0b/benchmark_data_storage0111-ib.pocstor_41097_object9391/head//24 by client.2174595.0:160634 0.000000 2014-08-13 18:06:06.904138 7f5c973a57a0 20 read_log 194215'18425 (192888'18135) clean f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by osd.4.0:2645 2014-08-12 15:48:16.365805 2014-08-13 18:06:06.904161 7f5c973a57a0 20 read_log 194215'18426 (194215'18425) delete f4b5b0b/benchmark_data_storage0111-ib.pocstor_2228_object13440/head//24 by client.2174595.0:160636 0.000000 2014-08-13 18:06:06.904175 7f5c973a57a0 20 read_log 194215'18427 (192834'17351) clean 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by osd.4.0:2647 2014-08-12 15:48:16.374695 2014-08-13 18:06:06.904188 7f5c973a57a0 20 read_log 194215'18428 (194215'18427) delete 77257b0b/benchmark_data_storage0111-ib.pocstor_26659_object21825/head//24 by client.2174595.0:160638 0.000000 2014-08-13 18:06:06.904199 7f5c973a57a0 20 read_log 194215'18429 (192834'17278) clean 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by osd.4.0:2649 2014-08-12 15:48:16.383421 2014-08-13 18:06:06.904228 7f5c973a57a0 20 read_log 194215'18430 (194215'18429) delete 992d7b0b/benchmark_data_storage0111-ib.pocstor_26659_object14583/head//24 by client.2174595.0:160640 0.000000 2014-08-13 18:06:06.904243 7f5c973a57a0 20 read_log 194215'18431 (192888'18131) clean 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by osd.4.0:2651 2014-08-12 15:48:16.393296 2014-08-13 18:06:06.904255 7f5c973a57a0 20 read_log 194215'18432 (194215'18431) delete 8bed7b0b/benchmark_data_storage0111-ib.pocstor_2228_object12306/head//24 by client.2174595.0:160642 0.000000 2014-08-13 18:06:06.904271 7f5c973a57a0 20 read_log 194215'18433 (192834'17353) clean d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by osd.4.0:2653 2014-08-12 15:48:16.401817 2014-08-13 18:06:06.904284 7f5c973a57a0 20 read_log 194215'18434 (194215'18433) delete d2418b0b/benchmark_data_storage0111-ib.pocstor_26659_object21924/head//24 by client.2174595.0:160644 0.000000 2014-08-13 18:06:06.904295 7f5c973a57a0 20 read_log 194215'18435 (192834'17309) clean f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by osd.4.0:2655 2014-08-12 15:48:16.411407 2014-08-13 18:06:06.904309 7f5c973a57a0 20 read_log 194215'18436 (194215'18435) delete f5918b0b/benchmark_data_storage0111-ib.pocstor_26659_object17798/head//24 by client.2174595.0:160646 0.000000 2014-08-13 18:06:06.904320 7f5c973a57a0 20 read_log 194215'18437 (192888'17984) clean 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by osd.4.0:2657 2014-08-12 15:48:16.421521 2014-08-13 18:06:06.904333 7f5c973a57a0 20 read_log 194215'18438 (194215'18437) delete 956f8b0b/benchmark_data_storage0111-ib.pocstor_41097_object29762/head//24 by client.2174595.0:160648 0.000000 2014-08-13 18:06:06.904348 7f5c973a57a0 20 read_log 194215'18439 (192834'17260) clean 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by osd.4.0:2659 2014-08-12 15:48:16.448559 2014-08-13 18:06:06.904360 7f5c973a57a0 20 read_log 194215'18440 (194215'18439) delete 8e29b0b/benchmark_data_storage0111-ib.pocstor_26659_object12152/head//24 by client.2174595.0:160650 0.000000 2014-08-13 18:06:06.904371 7f5c973a57a0 20 read_log 194215'18441 (193305'18259) clean e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by osd.4.0:2661 2014-08-12 15:48:16.458774 2014-08-13 18:06:06.904383 7f5c973a57a0 20 read_log 194215'18442 (194215'18441) delete e3869b0b/benchmark_data_storage0111-ib.pocstor_62621_object19039/head//24 by client.2174595.0:160652 0.000000 2014-08-13 18:06:06.904394 7f5c973a57a0 20 read_log 194215'18443 (192888'17956) clean 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by osd.4.0:2663 2014-08-12 15:48:16.468416 2014-08-13 18:06:06.904406 7f5c973a57a0 20 read_log 194215'18444 (194215'18443) delete 2f7a9b0b/benchmark_data_storage0111-ib.pocstor_41097_object14049/head//24 by client.2174595.0:160654 0.000000 2014-08-13 18:06:06.904421 7f5c973a57a0 20 read_log 194215'18445 (192888'17968) clean 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by osd.4.0:2665 2014-08-12 15:48:16.476818 2014-08-13 18:06:06.904434 7f5c973a57a0 20 read_log 194215'18446 (194215'18445) delete 2cb8ab0b/benchmark_data_storage0111-ib.pocstor_41097_object20765/head//24 by client.2174595.0:160656 0.000000 2014-08-13 18:06:06.904455 7f5c973a57a0 20 read_log 194215'18447 (193305'18251) clean 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by osd.4.0:2667 2014-08-12 15:48:16.484892 2014-08-13 18:06:06.904471 7f5c973a57a0 20 read_log 194215'18448 (194215'18447) delete 4869ab0b/benchmark_data_storage0111-ib.pocstor_62621_object12358/head//24 by client.2174595.0:160658 0.000000 2014-08-13 18:06:06.904482 7f5c973a57a0 20 read_log 194215'18449 (192888'18148) clean 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by osd.4.0:2669 2014-08-12 15:48:16.492943 2014-08-13 18:06:06.904515 7f5c973a57a0 20 read_log 194215'18450 (194215'18449) delete 7ffaab0b/benchmark_data_storage0111-ib.pocstor_2228_object20922/head//24 by client.2174595.0:160660 0.000000 2014-08-13 18:06:06.904529 7f5c973a57a0 20 read_log 194215'18451 (193305'18257) clean 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by osd.4.0:2671 2014-08-12 15:48:16.501209 2014-08-13 18:06:06.904552 7f5c973a57a0 20 read_log 194215'18452 (194215'18451) delete 8bb2bb0b/benchmark_data_storage0111-ib.pocstor_62621_object18004/head//24 by client.2174595.0:160662 0.000000 2014-08-13 18:06:06.904567 7f5c973a57a0 20 read_log 194215'18453 (192834'17207) clean 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by osd.4.0:2673 2014-08-12 15:48:16.510799 2014-08-13 18:06:06.904579 7f5c973a57a0 20 read_log 194215'18454 (194215'18453) delete 70a7bb0b/benchmark_data_storage0111-ib.pocstor_26659_object6973/head//24 by client.2174595.0:160664 0.000000 2014-08-13 18:06:06.904598 7f5c973a57a0 20 read_log 194215'18455 (192834'17203) clean a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by osd.4.0:2675 2014-08-12 15:48:16.520531 2014-08-13 18:06:06.904613 7f5c973a57a0 20 read_log 194215'18456 (194215'18455) delete a63cbb0b/benchmark_data_storage0111-ib.pocstor_26659_object6715/head//24 by client.2174595.0:160666 0.000000 2014-08-13 18:06:06.904624 7f5c973a57a0 20 read_log 194215'18457 (192834'17214) clean e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by osd.4.0:2677 2014-08-12 15:48:16.529447 2014-08-13 18:06:06.904637 7f5c973a57a0 20 read_log 194215'18458 (194215'18457) delete e570cb0b/benchmark_data_storage0111-ib.pocstor_26659_object7687/head//24 by client.2174595.0:160668 0.000000 2014-08-13 18:06:06.904648 7f5c973a57a0 20 read_log 194215'18459 (192888'17953) clean 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by osd.4.0:2679 2014-08-12 15:48:16.538132 2014-08-13 18:06:06.904660 7f5c973a57a0 20 read_log 194215'18460 (194215'18459) delete 2f92cb0b/benchmark_data_storage0111-ib.pocstor_41097_object12112/head//24 by client.2174595.0:160670 0.000000 2014-08-13 18:06:06.904671 7f5c973a57a0 20 read_log 194215'18461 (192888'18142) clean d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by osd.4.0:2681 2014-08-12 15:48:16.555928 2014-08-13 18:06:06.904683 7f5c973a57a0 20 read_log 194215'18462 (194215'18461) delete d2a6eb0b/benchmark_data_storage0111-ib.pocstor_2228_object16747/head//24 by client.2174595.0:160672 0.000000 2014-08-13 18:06:06.904694 7f5c973a57a0 20 read_log 194215'18463 (193305'18263) clean 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by osd.4.0:2683 2014-08-12 15:48:16.566026 2014-08-13 18:06:06.904707 7f5c973a57a0 20 read_log 194215'18464 (194215'18463) delete 9eb0fb0b/benchmark_data_storage0111-ib.pocstor_62621_object22714/head//24 by client.2174595.0:160674 0.000000 2014-08-13 18:06:06.904718 7f5c973a57a0 20 read_log 194215'18465 (192888'18124) clean 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by osd.4.0:2685 2014-08-12 15:48:16.575026 2014-08-13 18:06:06.904730 7f5c973a57a0 20 read_log 194215'18466 (194215'18465) delete 49a2fb0b/benchmark_data_storage0111-ib.pocstor_2228_object7174/head//24 by client.2174595.0:160676 0.000000 2014-08-13 18:06:06.904741 7f5c973a57a0 20 read_log 194215'18467 (193305'18266) clean 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by osd.4.0:2687 2014-08-12 15:48:16.585135 2014-08-13 18:06:06.904753 7f5c973a57a0 20 read_log 194215'18468 (194215'18467) delete 7f0ffb0b/benchmark_data_storage0111-ib.pocstor_62621_object25352/head//24 by client.2174595.0:160678 0.000000 2014-08-13 18:06:06.904764 7f5c973a57a0 20 read_log 194215'18469 (192888'18110) clean 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by osd.4.0:2689 2014-08-12 15:48:16.594372 2014-08-13 18:06:06.904783 7f5c973a57a0 20 read_log 194215'18470 (194215'18469) delete 8e560f0b/benchmark_data_storage0111-ib.pocstor_2228_object1673/head//24 by client.2174595.0:160680 0.000000 2014-08-13 18:06:06.904794 7f5c973a57a0 20 read_log 194215'18471 (192834'17747) clean 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by osd.4.0:2691 2014-08-12 15:48:16.602842 2014-08-13 18:06:06.904807 7f5c973a57a0 20 read_log 194215'18472 (194215'18471) delete 51980f0b/benchmark_data_storage0111-ib.pocstor_41526_object9625/head//24 by client.2174595.0:160682 0.000000 2014-08-13 18:06:06.904818 7f5c973a57a0 20 read_log 194215'18473 (192888'18116) clean 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by osd.4.0:2693 2014-08-12 15:48:16.611061 2014-08-13 18:06:06.904830 7f5c973a57a0 20 read_log 194215'18474 (194215'18473) delete 28b0f0b/benchmark_data_storage0111-ib.pocstor_2228_object4612/head//24 by client.2174595.0:160684 0.000000 2014-08-13 18:06:06.904841 7f5c973a57a0 20 read_log 194215'18475 (192834'17249) clean cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by osd.4.0:2695 2014-08-12 15:48:16.618466 2014-08-13 18:06:06.904853 7f5c973a57a0 20 read_log 194215'18476 (194215'18475) delete cf461f0b/benchmark_data_storage0111-ib.pocstor_26659_object11239/head//24 by client.2174595.0:160686 0.000000 2014-08-13 18:06:06.904864 7f5c973a57a0 20 read_log 194215'18477 (192888'17961) clean 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by osd.4.0:2697 2014-08-12 15:48:16.626523 2014-08-13 18:06:06.904876 7f5c973a57a0 20 read_log 194215'18478 (194215'18477) delete 82861f0b/benchmark_data_storage0111-ib.pocstor_41097_object15264/head//24 by client.2174595.0:160688 0.000000 2014-08-13 18:06:06.904887 7f5c973a57a0 20 read_log 194215'18479 (192888'17979) clean 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by osd.4.0:2699 2014-08-12 15:48:16.635419 2014-08-13 18:06:06.904907 7f5c973a57a0 20 read_log 194215'18480 (194215'18479) delete 755a1f0b/benchmark_data_storage0111-ib.pocstor_41097_object25344/head//24 by client.2174595.0:160690 0.000000 2014-08-13 18:06:06.904919 7f5c973a57a0 20 read_log 194215'18481 (192888'18119) clean c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by osd.4.0:2701 2014-08-12 15:48:16.643505 2014-08-13 18:06:06.904931 7f5c973a57a0 20 read_log 194215'18482 (194215'18481) delete c0572f0b/benchmark_data_storage0111-ib.pocstor_2228_object5867/head//24 by client.2174595.0:160692 0.000000 2014-08-13 18:06:06.904942 7f5c973a57a0 20 read_log 194215'18483 (193305'18246) clean 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by osd.4.0:2703 2014-08-12 15:48:16.651625 2014-08-13 18:06:06.904955 7f5c973a57a0 20 read_log 194215'18484 (194215'18483) delete 55082f0b/benchmark_data_storage0111-ib.pocstor_62621_object9575/head//24 by client.2174595.0:160694 0.000000 2014-08-13 18:06:06.904967 7f5c973a57a0 20 read_log 194215'18485 (192888'17974) clean 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by osd.4.0:2705 2014-08-12 15:48:16.660964 2014-08-13 18:06:06.904979 7f5c973a57a0 20 read_log 194215'18486 (194215'18485) delete 869c2f0b/benchmark_data_storage0111-ib.pocstor_41097_object24185/head//24 by client.2174595.0:160696 0.000000 2014-08-13 18:06:06.904991 7f5c973a57a0 20 read_log 194215'18487 (192834'17225) clean 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by osd.4.0:2707 2014-08-12 15:48:16.669714 2014-08-13 18:06:06.905019 7f5c973a57a0 20 read_log 194215'18488 (194215'18487) delete 1d4d2f0b/benchmark_data_storage0111-ib.pocstor_26659_object8728/head//24 by client.2174595.0:160698 0.000000 2014-08-13 18:06:06.905034 7f5c973a57a0 20 read_log 194215'18489 (192834'17722) clean 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by osd.4.0:2709 2014-08-12 15:48:16.678155 2014-08-13 18:06:06.905071 7f5c973a57a0 20 read_log 194215'18490 (194215'18489) delete 87903f0b/benchmark_data_storage0111-ib.pocstor_41526_object6525/head//24 by client.2174595.0:160700 0.000000 2014-08-13 18:06:06.905086 7f5c973a57a0 20 read_log 194215'18491 (192834'17758) clean 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by osd.4.0:2711 2014-08-12 15:48:16.686889 2014-08-13 18:06:06.905099 7f5c973a57a0 20 read_log 194215'18492 (194215'18491) delete 14214f0b/benchmark_data_storage0111-ib.pocstor_41526_object10676/head//24 by client.2174595.0:160702 0.000000 2014-08-13 18:06:06.905110 7f5c973a57a0 20 read_log 194215'18493 (192888'18133) clean d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by osd.4.0:2713 2014-08-12 15:48:16.694833 2014-08-13 18:06:06.905124 7f5c973a57a0 20 read_log 194215'18494 (194215'18493) delete d74e4f0b/benchmark_data_storage0111-ib.pocstor_2228_object12877/head//24 by client.2174595.0:160704 0.000000 2014-08-13 18:06:06.905135 7f5c973a57a0 20 read_log 194215'18495 (193305'18254) clean a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by osd.4.0:2715 2014-08-12 15:48:16.701823 2014-08-13 18:06:06.905164 7f5c973a57a0 20 read_log 194215'18496 (194215'18495) delete a0615f0b/benchmark_data_storage0111-ib.pocstor_62621_object14061/head//24 by client.2174595.0:160706 0.000000 2014-08-13 18:06:06.905179 7f5c973a57a0 20 read_log 194215'18497 (193305'18243) clean d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by osd.4.0:2717 2014-08-12 15:48:16.711152 2014-08-13 18:06:06.905192 7f5c973a57a0 20 read_log 194215'18498 (194215'18497) delete d4615f0b/benchmark_data_storage0111-ib.pocstor_62621_object8166/head//24 by client.2174595.0:160708 0.000000 2014-08-13 18:06:06.905203 7f5c973a57a0 20 read_log 194215'18499 (192834'17333) clean 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by osd.4.0:2719 2014-08-12 15:48:16.719616 2014-08-13 18:06:06.905216 7f5c973a57a0 20 read_log 194215'18500 (194215'18499) delete 65e45f0b/benchmark_data_storage0111-ib.pocstor_26659_object20204/head//24 by client.2174595.0:160710 0.000000 2014-08-13 18:06:06.905227 7f5c973a57a0 20 read_log 194215'18501 (192834'17235) clean 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by osd.4.0:2721 2014-08-12 15:48:16.728109 2014-08-13 18:06:06.905253 7f5c973a57a0 20 read_log 194215'18502 (194215'18501) delete 68b65f0b/benchmark_data_storage0111-ib.pocstor_26659_object9543/head//24 by client.2174595.0:160712 0.000000 2014-08-13 18:06:06.905267 7f5c973a57a0 20 read_log 194215'18503 (192888'17975) clean 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by osd.4.0:2723 2014-08-12 15:48:16.735976 2014-08-13 18:06:06.905282 7f5c973a57a0 20 read_log 194215'18504 (194215'18503) delete 131b5f0b/benchmark_data_storage0111-ib.pocstor_41097_object24375/head//24 by client.2174595.0:160714 0.000000 2014-08-13 18:06:06.905293 7f5c973a57a0 20 read_log 194215'18505 (192888'18143) clean 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by osd.4.0:2725 2014-08-12 15:48:16.743633 2014-08-13 18:06:06.905306 7f5c973a57a0 20 read_log 194215'18506 (194215'18505) delete 29ab7f0b/benchmark_data_storage0111-ib.pocstor_2228_object18283/head//24 by client.2174595.0:160716 0.000000 2014-08-13 18:06:06.905317 7f5c973a57a0 20 read_log 194215'18507 (192834'17751) clean b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by osd.4.0:2727 2014-08-12 15:48:16.752139 2014-08-13 18:06:06.905329 7f5c973a57a0 20 read_log 194215'18508 (194215'18507) delete b0ac7f0b/benchmark_data_storage0111-ib.pocstor_41526_object10126/head//24 by client.2174595.0:160718 0.000000 2014-08-13 18:06:06.905340 7f5c973a57a0 20 read_log 194215'18509 (193305'18264) clean 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by osd.4.0:2729 2014-08-12 15:48:16.760354 2014-08-13 18:06:06.905360 7f5c973a57a0 20 read_log 194215'18510 (194215'18509) delete 6e318f0b/benchmark_data_storage0111-ib.pocstor_62621_object24075/head//24 by client.2174595.0:160720 0.000000 2014-08-13 18:06:06.905373 7f5c973a57a0 20 read_log 194215'18511 (192888'18152) clean 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by osd.4.0:2731 2014-08-12 15:48:16.769309 2014-08-13 18:06:06.905386 7f5c973a57a0 20 read_log 194215'18512 (194215'18511) delete 7cb18f0b/benchmark_data_storage0111-ib.pocstor_2228_object21311/head//24 by client.2174595.0:160722 0.000000 2014-08-13 18:06:06.905397 7f5c973a57a0 20 read_log 194215'18513 (193305'18267) clean 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by osd.4.0:2733 2014-08-12 15:48:16.778579 2014-08-13 18:06:06.905409 7f5c973a57a0 20 read_log 194215'18514 (194215'18513) delete 90048f0b/benchmark_data_storage0111-ib.pocstor_62621_object26113/head//24 by client.2174595.0:160724 0.000000 2014-08-13 18:06:06.905420 7f5c973a57a0 20 read_log 194215'18515 (192834'17216) clean e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by osd.4.0:2735 2014-08-12 15:48:16.788307 2014-08-13 18:06:06.905432 7f5c973a57a0 20 read_log 194215'18516 (194215'18515) delete e0368f0b/benchmark_data_storage0111-ib.pocstor_26659_object7730/head//24 by client.2174595.0:160726 0.000000 2014-08-13 18:06:06.905449 7f5c973a57a0 20 read_log 194215'18517 (193305'18249) clean 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by osd.4.0:2737 2014-08-12 15:48:16.796235 2014-08-13 18:06:06.905474 7f5c973a57a0 20 read_log 194215'18518 (194215'18517) delete 2abd8f0b/benchmark_data_storage0111-ib.pocstor_62621_object12233/head//24 by client.2174595.0:160728 0.000000 2014-08-13 18:06:06.905489 7f5c973a57a0 20 read_log 194215'18519 (192888'18165) clean 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by osd.4.0:2739 2014-08-12 15:48:16.804794 2014-08-13 18:06:06.905506 7f5c973a57a0 20 read_log 194215'18520 (194215'18519) delete 411e8f0b/benchmark_data_storage0111-ib.pocstor_2228_object29228/head//24 by client.2174595.0:160730 0.000000 2014-08-13 18:06:06.905527 7f5c973a57a0 20 read_log 194215'18521 (192834'17727) clean 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by osd.4.0:2741 2014-08-12 15:48:16.812858 2014-08-13 18:06:06.905543 7f5c973a57a0 20 read_log 194215'18522 (194215'18521) delete 5f319f0b/benchmark_data_storage0111-ib.pocstor_41526_object7110/head//24 by client.2174595.0:160732 0.000000 2014-08-13 18:06:06.905555 7f5c973a57a0 20 read_log 194215'18523 (193305'18273) clean d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by osd.4.0:2743 2014-08-12 15:48:16.823024 2014-08-13 18:06:06.905578 7f5c973a57a0 20 read_log 194215'18524 (194215'18523) delete d5bb9f0b/benchmark_data_storage0111-ib.pocstor_62621_object31540/head//24 by client.2174595.0:160734 0.000000 2014-08-13 18:06:06.905592 7f5c973a57a0 20 read_log 194215'18525 (192834'17768) clean 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by osd.4.0:2745 2014-08-12 15:48:16.832344 2014-08-13 18:06:06.905605 7f5c973a57a0 20 read_log 194215'18526 (194215'18525) delete 8f16af0b/benchmark_data_storage0111-ib.pocstor_41526_object11979/head//24 by client.2174595.0:160736 0.000000 2014-08-13 18:06:06.905624 7f5c973a57a0 20 read_log 194215'18527 (192888'17965) clean d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by osd.4.0:2747 2014-08-12 15:48:16.840478 2014-08-13 18:06:06.905640 7f5c973a57a0 20 read_log 194215'18528 (194215'18527) delete d00aaf0b/benchmark_data_storage0111-ib.pocstor_41097_object19130/head//24 by client.2174595.0:160738 0.000000 2014-08-13 18:06:06.905651 7f5c973a57a0 20 read_log 194215'18529 (192888'18126) clean 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by osd.4.0:2749 2014-08-12 15:48:16.849979 2014-08-13 18:06:06.905671 7f5c973a57a0 20 read_log 194215'18530 (194215'18529) delete 2b41bf0b/benchmark_data_storage0111-ib.pocstor_2228_object8144/head//24 by client.2174595.0:160740 0.000000 2014-08-13 18:06:06.905684 7f5c973a57a0 20 read_log 194215'18531 (192888'18144) clean 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by osd.4.0:2751 2014-08-12 15:48:16.857428 2014-08-13 18:06:06.905697 7f5c973a57a0 20 read_log 194215'18532 (194215'18531) delete 1135bf0b/benchmark_data_storage0111-ib.pocstor_2228_object18315/head//24 by client.2174595.0:160742 0.000000 2014-08-13 18:06:06.905708 7f5c973a57a0 20 read_log 194215'18533 (193305'18242) clean df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by osd.4.0:2753 2014-08-12 15:48:16.866341 2014-08-13 18:06:06.905720 7f5c973a57a0 20 read_log 194215'18534 (194215'18533) delete df1dbf0b/benchmark_data_storage0111-ib.pocstor_62621_object8050/head//24 by client.2174595.0:160744 0.000000 2014-08-13 18:06:06.905731 7f5c973a57a0 20 read_log 194215'18535 (193305'18269) clean 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by osd.4.0:2755 2014-08-12 15:48:16.875215 2014-08-13 18:06:06.905744 7f5c973a57a0 20 read_log 194215'18536 (194215'18535) delete 9093cf0b/benchmark_data_storage0111-ib.pocstor_62621_object27195/head//24 by client.2174595.0:160746 0.000000 2014-08-13 18:06:06.905755 7f5c973a57a0 20 read_log 194215'18537 (192888'18117) clean b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by osd.4.0:2757 2014-08-12 15:48:16.883755 2014-08-13 18:06:06.905767 7f5c973a57a0 20 read_log 194215'18538 (194215'18537) delete b018cf0b/benchmark_data_storage0111-ib.pocstor_2228_object4787/head//24 by client.2174595.0:160748 0.000000 2014-08-13 18:06:06.905786 7f5c973a57a0 20 read_log 194215'18539 (193305'18265) clean 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by osd.4.0:2759 2014-08-12 15:48:16.892363 2014-08-13 18:06:06.905799 7f5c973a57a0 20 read_log 194215'18540 (194215'18539) delete 1dfccf0b/benchmark_data_storage0111-ib.pocstor_62621_object24095/head//24 by client.2174595.0:160750 0.000000 2014-08-13 18:06:06.905811 7f5c973a57a0 20 read_log 194215'18541 (192888'17977) clean be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by osd.4.0:2761 2014-08-12 15:48:16.900886 2014-08-13 18:06:06.905823 7f5c973a57a0 20 read_log 194215'18542 (194215'18541) delete be9dcf0b/benchmark_data_storage0111-ib.pocstor_41097_object24759/head//24 by client.2174595.0:160752 0.000000 2014-08-13 18:06:06.905834 7f5c973a57a0 20 read_log 194215'18543 (193305'18262) clean 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by osd.4.0:2763 2014-08-12 15:48:16.909571 2014-08-13 18:06:06.905846 7f5c973a57a0 20 read_log 194215'18544 (194215'18543) delete 8c0fcf0b/benchmark_data_storage0111-ib.pocstor_62621_object22576/head//24 by client.2174595.0:160754 0.000000 2014-08-13 18:06:06.905857 7f5c973a57a0 20 read_log 194215'18545 (193305'18255) clean 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by osd.4.0:2765 2014-08-12 15:48:16.917487 2014-08-13 18:06:06.905870 7f5c973a57a0 20 read_log 194215'18546 (194215'18545) delete 6249ff0b/benchmark_data_storage0111-ib.pocstor_62621_object15147/head//24 by client.2174595.0:160756 0.000000 2014-08-13 18:06:06.905881 7f5c973a57a0 20 read_log 194370'18547 (0'0) promote d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by osd.301.0:3202 2014-08-12 20:30:15.984800 2014-08-13 18:06:06.905894 7f5c973a57a0 20 read_log 194370'18548 (194370'18547) modify d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by client.1508531.0:423 2014-08-12 20:30:15.977446 2014-08-13 18:06:06.905906 7f5c973a57a0 20 read_log 194370'18549 (0'0) promote e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by osd.301.0:3222 2014-08-12 20:30:20.297306 2014-08-13 18:06:06.905929 7f5c973a57a0 20 read_log 194370'18550 (194370'18549) modify e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by client.1508531.0:1577 2014-08-12 20:30:20.290518 2014-08-13 18:06:06.905945 7f5c973a57a0 20 read_log 194370'18551 (0'0) promote a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by osd.301.0:3240 2014-08-12 20:30:22.812103 2014-08-13 18:06:06.905957 7f5c973a57a0 20 read_log 194370'18552 (194370'18551) modify a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by client.1508531.0:2246 2014-08-12 20:30:22.801067 2014-08-13 18:06:06.905970 7f5c973a57a0 20 read_log 194370'18553 (0'0) promote 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by osd.301.0:3294 2014-08-12 20:30:32.686355 2014-08-13 18:06:06.905982 7f5c973a57a0 20 read_log 194370'18554 (194370'18553) modify 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by client.1508531.0:4881 2014-08-12 20:30:32.677788 2014-08-13 18:06:06.906005 7f5c973a57a0 20 read_log 194370'18555 (0'0) promote d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by osd.301.0:3298 2014-08-12 20:30:33.102369 2014-08-13 18:06:06.906021 7f5c973a57a0 20 read_log 194370'18556 (194370'18555) modify d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by client.1508531.0:4972 2014-08-12 20:30:33.095955 2014-08-13 18:06:06.906034 7f5c973a57a0 20 read_log 194370'18557 (0'0) promote 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by osd.301.0:3314 2014-08-12 20:30:36.226278 2014-08-13 18:06:06.906048 7f5c973a57a0 20 read_log 194370'18558 (194370'18557) modify 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by client.1508531.0:5674 2014-08-12 20:30:36.218820 2014-08-13 18:06:06.906061 7f5c973a57a0 20 read_log 194370'18559 (0'0) promote 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by osd.301.0:3322 2014-08-12 20:30:37.835165 2014-08-13 18:06:06.906073 7f5c973a57a0 20 read_log 194370'18560 (194370'18559) modify 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by client.1508531.0:6024 2014-08-12 20:30:37.828080 2014-08-13 18:06:06.906088 7f5c973a57a0 20 read_log 194370'18561 (0'0) promote 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by osd.301.0:3328 2014-08-12 20:30:39.150319 2014-08-13 18:06:06.906100 7f5c973a57a0 20 read_log 194370'18562 (194370'18561) modify 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by client.1508531.0:6327 2014-08-12 20:30:39.141385 2014-08-13 18:06:06.906112 7f5c973a57a0 20 read_log 194370'18563 (0'0) promote 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by osd.301.0:3336 2014-08-12 20:30:40.679066 2014-08-13 18:06:06.906128 7f5c973a57a0 20 read_log 194370'18564 (194370'18563) modify 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by client.1508531.0:6655 2014-08-12 20:30:40.670736 2014-08-13 18:06:06.906140 7f5c973a57a0 20 read_log 194370'18565 (0'0) promote eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by osd.301.0:3342 2014-08-12 20:30:42.561288 2014-08-13 18:06:06.906160 7f5c973a57a0 20 read_log 194370'18566 (194370'18565) modify eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by client.1508531.0:7024 2014-08-12 20:30:42.552204 2014-08-13 18:06:06.906176 7f5c973a57a0 20 read_log 194370'18567 (0'0) promote a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by osd.301.0:3348 2014-08-12 20:30:43.648976 2014-08-13 18:06:06.906189 7f5c973a57a0 20 read_log 194370'18568 (194370'18567) modify a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by client.1508531.0:7259 2014-08-12 20:30:43.640862 2014-08-13 18:06:06.906201 7f5c973a57a0 20 read_log 194370'18569 (0'0) promote d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by osd.301.0:3376 2014-08-12 20:30:49.062684 2014-08-13 18:06:06.906224 7f5c973a57a0 20 read_log 194370'18570 (194370'18569) modify d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by client.1508531.0:8281 2014-08-12 20:30:49.053145 2014-08-13 18:06:06.906240 7f5c973a57a0 20 read_log 194370'18571 (0'0) promote 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by osd.301.0:3438 2014-08-12 20:31:06.635907 2014-08-13 18:06:06.906253 7f5c973a57a0 20 read_log 194370'18572 (194370'18571) modify 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by client.1508531.0:11225 2014-08-12 20:31:06.626292 2014-08-13 18:06:06.906265 7f5c973a57a0 20 read_log 194370'18573 (0'0) promote 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by osd.301.0:3456 2014-08-12 20:31:09.504784 2014-08-13 18:06:06.906286 7f5c973a57a0 20 read_log 194370'18574 (194370'18573) modify 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by client.1508531.0:11697 2014-08-12 20:31:09.495710 2014-08-13 18:06:06.906302 7f5c973a57a0 20 read_log 194370'18575 (0'0) promote aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by osd.301.0:3472 2014-08-12 20:31:12.334532 2014-08-13 18:06:06.906314 7f5c973a57a0 20 read_log 194370'18576 (194370'18575) modify aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by client.1508531.0:12132 2014-08-12 20:31:12.323754 2014-08-13 18:06:06.906336 7f5c973a57a0 20 read_log 194377'18577 (0'0) promote d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by osd.10.0:6686 2014-08-12 20:32:59.420399 2014-08-13 18:06:06.906349 7f5c973a57a0 20 read_log 194377'18578 (194377'18577) modify d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by client.2231190.0:510 2014-08-12 20:32:59.412977 2014-08-13 18:06:06.906362 7f5c973a57a0 20 read_log 194377'18579 (0'0) promote eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by osd.10.0:6690 2014-08-12 20:33:01.015336 2014-08-13 18:06:06.906382 7f5c973a57a0 20 read_log 194377'18580 (194377'18579) modify eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by client.2231190.0:923 2014-08-12 20:33:01.007967 2014-08-13 18:06:06.906422 7f5c973a57a0 20 read_log 194377'18581 (0'0) promote f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by osd.10.0:6692 2014-08-12 20:33:02.446757 2014-08-13 18:06:06.906437 7f5c973a57a0 20 read_log 194377'18582 (194377'18581) modify f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by client.2231190.0:1301 2014-08-12 20:33:02.440688 2014-08-13 18:06:06.906450 7f5c973a57a0 20 read_log 194377'18583 (0'0) promote a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by osd.10.0:6704 2014-08-12 20:33:05.327851 2014-08-13 18:06:06.906462 7f5c973a57a0 20 read_log 194377'18584 (194377'18583) modify a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by client.2231190.0:2017 2014-08-12 20:33:05.320173 2014-08-13 18:06:06.906475 7f5c973a57a0 20 read_log 194377'18585 (0'0) promote 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by osd.10.0:6710 2014-08-12 20:33:05.997041 2014-08-13 18:06:06.906487 7f5c973a57a0 20 read_log 194377'18586 (194377'18585) modify 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by client.2231190.0:2183 2014-08-12 20:33:05.991812 2014-08-13 18:06:06.906499 7f5c973a57a0 20 read_log 194377'18587 (0'0) promote 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by osd.10.0:6716 2014-08-12 20:33:08.985499 2014-08-13 18:06:06.906512 7f5c973a57a0 20 read_log 194377'18588 (194377'18587) modify 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by client.2231190.0:2921 2014-08-12 20:33:08.976531 2014-08-13 18:06:06.906524 7f5c973a57a0 20 read_log 194377'18589 (0'0) promote e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by osd.10.0:6724 2014-08-12 20:33:09.953748 2014-08-13 18:06:06.906544 7f5c973a57a0 20 read_log 194377'18590 (194377'18589) modify e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by client.2231190.0:3165 2014-08-12 20:33:09.942727 2014-08-13 18:06:06.906559 7f5c973a57a0 20 read_log 194378'18591 (0'0) promote 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by osd.10.0:6738 2014-08-12 20:33:20.419690 2014-08-13 18:06:06.906572 7f5c973a57a0 20 read_log 194378'18592 (194378'18591) modify 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by client.2231190.0:4862 2014-08-12 20:33:20.412102 2014-08-13 18:06:06.906593 7f5c973a57a0 20 read_log 194380'18593 (0'0) promote 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by osd.10.0:6744 2014-08-12 20:33:29.996469 2014-08-13 18:06:06.906609 7f5c973a57a0 20 read_log 194380'18594 (194380'18593) modify 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by client.2231190.0:5507 2014-08-12 20:33:29.989100 2014-08-13 18:06:06.906622 7f5c973a57a0 20 read_log 194393'18595 (194393'18595) modify 30b/hit_set_24.30b_archive_2014-08-12 20:37:09.053178_2014-08-12 20:49:27.261750/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.261985 2014-08-13 18:06:06.906635 7f5c973a57a0 20 read_log 194393'18596 (194215'18277) delete 30b/hit_set_24.30b_archive_2014-08-12 15:32:25.863655_2014-08-12 15:48:15.603804/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:49:27.261985 2014-08-13 18:06:06.906647 7f5c973a57a0 20 read_log 194393'18597 (0'0) promote ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by osd.301.0:3685 2014-08-12 20:49:27.265309 2014-08-13 18:06:06.906659 7f5c973a57a0 20 read_log 194393'18598 (194393'18597) modify ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by client.1064182.0:382 2014-08-12 20:49:27.257464 2014-08-13 18:06:06.906672 7f5c973a57a0 20 read_log 194396'18599 (0'0) promote d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by osd.301.0:3715 2014-08-12 20:53:00.628865 2014-08-13 18:06:06.906684 7f5c973a57a0 20 read_log 194396'18600 (194396'18599) modify d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by client.2235069.0:364 2014-08-12 20:53:00.620088 2014-08-13 18:06:06.906697 7f5c973a57a0 20 read_log 194416'18601 (194416'18601) modify 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 18:06:06.906709 7f5c973a57a0 20 read_log 194416'18602 (194393'18595) delete 30b/hit_set_24.30b_archive_2014-08-12 20:37:09.053178_2014-08-12 20:49:27.261750/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 18:06:06.906722 7f5c973a57a0 20 read_log 194416'18603 (0'0) promote c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by osd.301.0:3867 2014-08-13 15:28:34.596370 2014-08-13 18:06:06.906734 7f5c973a57a0 20 read_log 194416'18604 (194416'18603) modify c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by client.2441958.0:1020 2014-08-13 15:28:34.585941 2014-08-13 18:06:06.906746 7f5c973a57a0 20 read_log 194416'18605 (0'0) promote 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by osd.301.0:3897 2014-08-13 15:28:38.704066 2014-08-13 18:06:06.906761 7f5c973a57a0 20 read_log 194416'18606 (194416'18605) modify 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by client.2441958.0:2112 2014-08-13 15:28:38.696406 2014-08-13 18:06:06.906774 7f5c973a57a0 20 read_log 194416'18607 (0'0) promote ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by osd.301.0:3905 2014-08-13 15:28:40.400240 2014-08-13 18:06:06.906796 7f5c973a57a0 20 read_log 194416'18608 (194416'18607) modify ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by client.2441958.0:2555 2014-08-13 15:28:40.388566 2014-08-13 18:06:06.906811 7f5c973a57a0 20 read_log 194416'18609 (0'0) promote a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by osd.301.0:3949 2014-08-13 15:28:48.479236 2014-08-13 18:06:06.906823 7f5c973a57a0 20 read_log 194416'18610 (194416'18609) modify a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by client.2441958.0:4625 2014-08-13 15:28:48.470757 2014-08-13 18:06:06.906838 7f5c973a57a0 20 read_log 194416'18611 (0'0) promote 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by osd.301.0:3951 2014-08-13 15:28:48.606666 2014-08-13 18:06:06.906850 7f5c973a57a0 20 read_log 194416'18612 (194416'18611) modify 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by client.2441958.0:4659 2014-08-13 15:28:48.598232 2014-08-13 18:06:06.906864 7f5c973a57a0 20 read_log 194416'18613 (0'0) promote 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by osd.301.0:3991 2014-08-13 15:28:55.397809 2014-08-13 18:06:06.906877 7f5c973a57a0 20 read_log 194416'18614 (194416'18613) modify 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by client.2441958.0:6372 2014-08-13 15:28:55.388244 2014-08-13 18:06:06.906892 7f5c973a57a0 20 read_log 194416'18615 (0'0) promote 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by osd.301.0:4055 2014-08-13 15:29:07.624257 2014-08-13 18:06:06.906904 7f5c973a57a0 20 read_log 194416'18616 (194416'18615) modify 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by client.2441958.0:9562 2014-08-13 15:29:07.616896 2014-08-13 18:06:06.906919 7f5c973a57a0 20 read_log 194416'18617 (0'0) promote f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by osd.301.0:4067 2014-08-13 15:29:11.323561 2014-08-13 18:06:06.906931 7f5c973a57a0 20 read_log 194416'18618 (194416'18617) modify f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by client.2441958.0:10469 2014-08-13 15:29:11.314182 2014-08-13 18:06:06.906945 7f5c973a57a0 20 read_log 194416'18619 (0'0) promote 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by osd.301.0:4071 2014-08-13 15:29:11.821492 2014-08-13 18:06:06.906957 7f5c973a57a0 20 read_log 194416'18620 (194416'18619) modify 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by client.2441958.0:10603 2014-08-13 15:29:11.812825 2014-08-13 18:06:06.906970 7f5c973a57a0 20 read_log 194416'18621 (0'0) promote 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by osd.301.0:4093 2014-08-13 15:29:14.872061 2014-08-13 18:06:06.906982 7f5c973a57a0 20 read_log 194416'18622 (194416'18621) modify 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by client.2441958.0:11350 2014-08-13 15:29:14.864431 2014-08-13 18:06:06.906996 7f5c973a57a0 20 read_log 194416'18623 (0'0) promote d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by osd.301.0:4147 2014-08-13 15:29:29.751060 2014-08-13 18:06:06.907008 7f5c973a57a0 20 read_log 194416'18624 (194416'18623) modify d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by client.2441958.0:14878 2014-08-13 15:29:29.742414 2014-08-13 18:06:06.907023 7f5c973a57a0 20 read_log 194416'18625 (0'0) promote 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by osd.301.0:4159 2014-08-13 15:29:30.717320 2014-08-13 18:06:06.907038 7f5c973a57a0 20 read_log 194416'18626 (194416'18625) modify 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by client.2441958.0:15097 2014-08-13 15:29:30.708981 2014-08-13 18:06:06.907052 7f5c973a57a0 20 read_log 194416'18627 (0'0) promote ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by osd.301.0:4173 2014-08-13 15:29:33.685595 2014-08-13 18:06:06.907087 7f5c973a57a0 20 read_log 194416'18628 (194416'18627) modify ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by client.2441958.0:15685 2014-08-13 15:29:33.676427 2014-08-13 18:06:06.907102 7f5c973a57a0 20 read_log 194416'18629 (0'0) promote 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by osd.301.0:4189 2014-08-13 15:29:45.860462 2014-08-13 18:06:06.907115 7f5c973a57a0 20 read_log 194416'18630 (194416'18629) modify 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by client.2441958.0:16896 2014-08-13 15:29:45.851032 2014-08-13 18:06:06.907129 7f5c973a57a0 20 read_log 194416'18631 (0'0) promote 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by osd.301.0:4203 2014-08-13 15:29:55.046533 2014-08-13 18:06:06.907142 7f5c973a57a0 20 read_log 194416'18632 (194416'18631) modify 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by client.2441958.0:17543 2014-08-13 15:29:55.037173 2014-08-13 18:06:06.907162 7f5c973a57a0 20 read_log 194416'18633 (0'0) promote 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by osd.301.0:4209 2014-08-13 15:30:09.294520 2014-08-13 18:06:06.907175 7f5c973a57a0 20 read_log 194416'18634 (194416'18633) modify 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by client.2441958.0:18363 2014-08-13 15:30:09.286116 2014-08-13 18:06:06.907187 7f5c973a57a0 20 read_log 194416'18635 (0'0) promote 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by osd.301.0:4217 2014-08-13 15:30:15.202330 2014-08-13 18:06:06.907202 7f5c973a57a0 20 read_log 194416'18636 (194416'18635) modify 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by client.2441958.0:18577 2014-08-13 15:30:15.193629 2014-08-13 18:06:06.907215 7f5c973a57a0 20 read_log 194422'18637 (0'0) promote eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by osd.301.0:4233 2014-08-13 15:32:16.926044 2014-08-13 18:06:06.907230 7f5c973a57a0 20 read_log 194422'18638 (194422'18637) modify eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by client.1148203.0:420 2014-08-13 15:32:16.918548 2014-08-13 18:06:06.907242 7f5c973a57a0 20 read_log 194422'18639 (0'0) promote b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by osd.301.0:4257 2014-08-13 15:32:23.686723 2014-08-13 18:06:06.907255 7f5c973a57a0 20 read_log 194422'18640 (194422'18639) modify b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by client.1148203.0:2118 2014-08-13 15:32:23.675230 2014-08-13 18:06:06.907267 7f5c973a57a0 20 read_log 194422'18641 (0'0) promote bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by osd.301.0:4281 2014-08-13 15:32:29.929305 2014-08-13 18:06:06.907285 7f5c973a57a0 20 read_log 194422'18642 (194422'18641) modify bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by client.1148203.0:3716 2014-08-13 15:32:29.921559 2014-08-13 18:06:06.907297 7f5c973a57a0 20 read_log 194422'18643 (0'0) promote 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by osd.301.0:4315 2014-08-13 15:32:35.203109 2014-08-13 18:06:06.907313 7f5c973a57a0 20 read_log 194422'18644 (194422'18643) modify 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by client.1148203.0:5079 2014-08-13 15:32:35.194308 2014-08-13 18:06:06.907332 7f5c973a57a0 20 read_log 194422'18645 (0'0) promote 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by osd.301.0:4345 2014-08-13 15:32:44.256364 2014-08-13 18:06:06.907344 7f5c973a57a0 20 read_log 194422'18646 (194422'18645) modify 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by client.1148203.0:6766 2014-08-13 15:32:44.248958 2014-08-13 18:06:06.907357 7f5c973a57a0 20 read_log 194422'18647 (0'0) promote 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by osd.301.0:4347 2014-08-13 15:32:45.107587 2014-08-13 18:06:06.907374 7f5c973a57a0 20 read_log 194422'18648 (194422'18647) modify 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by client.1148203.0:6938 2014-08-13 15:32:45.099903 2014-08-13 18:06:06.907389 7f5c973a57a0 20 read_log 194422'18649 (0'0) promote 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by osd.301.0:4363 2014-08-13 15:32:47.484346 2014-08-13 18:06:06.907402 7f5c973a57a0 20 read_log 194422'18650 (194422'18649) modify 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by client.1148203.0:7320 2014-08-13 15:32:47.475508 2014-08-13 18:06:06.907432 7f5c973a57a0 20 read_log 194422'18651 (0'0) promote eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by osd.301.0:4383 2014-08-13 15:32:53.769159 2014-08-13 18:06:06.907446 7f5c973a57a0 20 read_log 194422'18652 (194422'18651) modify eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by client.1148203.0:8324 2014-08-13 15:32:53.760637 2014-08-13 18:06:06.907466 7f5c973a57a0 20 read_log 194422'18653 (0'0) promote 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by osd.301.0:4389 2014-08-13 15:32:54.756341 2014-08-13 18:06:06.907481 7f5c973a57a0 20 read_log 194422'18654 (194422'18653) modify 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by client.1148203.0:8452 2014-08-13 15:32:54.747584 2014-08-13 18:06:06.907494 7f5c973a57a0 20 read_log 194422'18655 (0'0) promote e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by osd.301.0:4401 2014-08-13 15:33:04.760428 2014-08-13 18:06:06.907514 7f5c973a57a0 20 read_log 194422'18656 (194422'18655) modify e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by client.1148203.0:9529 2014-08-13 15:33:04.752691 2014-08-13 18:06:06.907530 7f5c973a57a0 20 read_log 194422'18657 (0'0) promote c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by osd.301.0:4405 2014-08-13 15:33:06.256675 2014-08-13 18:06:06.907542 7f5c973a57a0 20 read_log 194422'18658 (194422'18657) modify c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by client.1148203.0:9685 2014-08-13 15:33:06.246976 2014-08-13 18:06:06.907561 7f5c973a57a0 20 read_log 194422'18659 (0'0) promote c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by osd.301.0:4413 2014-08-13 15:33:07.054088 2014-08-13 18:06:06.907575 7f5c973a57a0 20 read_log 194422'18660 (194422'18659) modify c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by client.1148203.0:9743 2014-08-13 15:33:07.044878 2014-08-13 18:06:06.907592 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.908204 7f5c973a57a0 10 read_log checking for missing items over interval (194370'18547,194422'18660] 2014-08-13 18:06:06.908252 7f5c973a57a0 15 read_log missing 194422'18660 (194422'18659) modify c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 by client.1148203.0:9743 2014-08-13 15:33:07.044878 2014-08-13 18:06:06.908297 7f5c973a57a0 15 read_log missing 194422'18658 (194422'18657) modify c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 by client.1148203.0:9685 2014-08-13 15:33:06.246976 2014-08-13 18:06:06.908334 7f5c973a57a0 15 read_log missing 194422'18656 (194422'18655) modify e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 by client.1148203.0:9529 2014-08-13 15:33:04.752691 2014-08-13 18:06:06.908370 7f5c973a57a0 15 read_log missing 194422'18654 (194422'18653) modify 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 by client.1148203.0:8452 2014-08-13 15:32:54.747584 2014-08-13 18:06:06.908405 7f5c973a57a0 15 read_log missing 194422'18652 (194422'18651) modify eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 by client.1148203.0:8324 2014-08-13 15:32:53.760637 2014-08-13 18:06:06.908445 7f5c973a57a0 15 read_log missing 194422'18650 (194422'18649) modify 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 by client.1148203.0:7320 2014-08-13 15:32:47.475508 2014-08-13 18:06:06.908492 7f5c973a57a0 15 read_log missing 194422'18648 (194422'18647) modify 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 by client.1148203.0:6938 2014-08-13 15:32:45.099903 2014-08-13 18:06:06.908563 7f5c973a57a0 15 read_log missing 194422'18646 (194422'18645) modify 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 by client.1148203.0:6766 2014-08-13 15:32:44.248958 2014-08-13 18:06:06.908602 7f5c973a57a0 15 read_log missing 194422'18644 (194422'18643) modify 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 by client.1148203.0:5079 2014-08-13 15:32:35.194308 2014-08-13 18:06:06.908639 7f5c973a57a0 15 read_log missing 194422'18642 (194422'18641) modify bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 by client.1148203.0:3716 2014-08-13 15:32:29.921559 2014-08-13 18:06:06.908675 7f5c973a57a0 15 read_log missing 194422'18640 (194422'18639) modify b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 by client.1148203.0:2118 2014-08-13 15:32:23.675230 2014-08-13 18:06:06.908710 7f5c973a57a0 15 read_log missing 194422'18638 (194422'18637) modify eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 by client.1148203.0:420 2014-08-13 15:32:16.918548 2014-08-13 18:06:06.908746 7f5c973a57a0 15 read_log missing 194416'18636 (194416'18635) modify 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 by client.2441958.0:18577 2014-08-13 15:30:15.193629 2014-08-13 18:06:06.908782 7f5c973a57a0 15 read_log missing 194416'18634 (194416'18633) modify 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 by client.2441958.0:18363 2014-08-13 15:30:09.286116 2014-08-13 18:06:06.908818 7f5c973a57a0 15 read_log missing 194416'18632 (194416'18631) modify 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 by client.2441958.0:17543 2014-08-13 15:29:55.037173 2014-08-13 18:06:06.908854 7f5c973a57a0 15 read_log missing 194416'18630 (194416'18629) modify 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 by client.2441958.0:16896 2014-08-13 15:29:45.851032 2014-08-13 18:06:06.908889 7f5c973a57a0 15 read_log missing 194416'18628 (194416'18627) modify ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 by client.2441958.0:15685 2014-08-13 15:29:33.676427 2014-08-13 18:06:06.908925 7f5c973a57a0 15 read_log missing 194416'18626 (194416'18625) modify 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 by client.2441958.0:15097 2014-08-13 15:29:30.708981 2014-08-13 18:06:06.908961 7f5c973a57a0 15 read_log missing 194416'18624 (194416'18623) modify d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 by client.2441958.0:14878 2014-08-13 15:29:29.742414 2014-08-13 18:06:06.908997 7f5c973a57a0 15 read_log missing 194416'18622 (194416'18621) modify 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 by client.2441958.0:11350 2014-08-13 15:29:14.864431 2014-08-13 18:06:06.909032 7f5c973a57a0 15 read_log missing 194416'18620 (194416'18619) modify 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 by client.2441958.0:10603 2014-08-13 15:29:11.812825 2014-08-13 18:06:06.909068 7f5c973a57a0 15 read_log missing 194416'18618 (194416'18617) modify f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 by client.2441958.0:10469 2014-08-13 15:29:11.314182 2014-08-13 18:06:06.909104 7f5c973a57a0 15 read_log missing 194416'18616 (194416'18615) modify 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 by client.2441958.0:9562 2014-08-13 15:29:07.616896 2014-08-13 18:06:06.909140 7f5c973a57a0 15 read_log missing 194416'18614 (194416'18613) modify 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 by client.2441958.0:6372 2014-08-13 15:28:55.388244 2014-08-13 18:06:06.909182 7f5c973a57a0 15 read_log missing 194416'18612 (194416'18611) modify 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 by client.2441958.0:4659 2014-08-13 15:28:48.598232 2014-08-13 18:06:06.909219 7f5c973a57a0 15 read_log missing 194416'18610 (194416'18609) modify a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 by client.2441958.0:4625 2014-08-13 15:28:48.470757 2014-08-13 18:06:06.909255 7f5c973a57a0 15 read_log missing 194416'18608 (194416'18607) modify ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 by client.2441958.0:2555 2014-08-13 15:28:40.388566 2014-08-13 18:06:06.909290 7f5c973a57a0 15 read_log missing 194416'18606 (194416'18605) modify 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 by client.2441958.0:2112 2014-08-13 15:28:38.696406 2014-08-13 18:06:06.909330 7f5c973a57a0 15 read_log missing 194416'18604 (194416'18603) modify c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 by client.2441958.0:1020 2014-08-13 15:28:34.585941 2014-08-13 18:06:06.909374 7f5c973a57a0 15 read_log missing 194416'18601 (194416'18601) modify 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:34.591922 2014-08-13 18:06:06.909411 7f5c973a57a0 15 read_log missing 194396'18600 (194396'18599) modify d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 by client.2235069.0:364 2014-08-12 20:53:00.620088 2014-08-13 18:06:06.909447 7f5c973a57a0 15 read_log missing 194393'18598 (194393'18597) modify ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 by client.1064182.0:382 2014-08-12 20:49:27.257464 2014-08-13 18:06:06.909509 7f5c973a57a0 15 read_log missing 194380'18594 (194380'18593) modify 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 by client.2231190.0:5507 2014-08-12 20:33:29.989100 2014-08-13 18:06:06.909550 7f5c973a57a0 15 read_log missing 194378'18592 (194378'18591) modify 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 by client.2231190.0:4862 2014-08-12 20:33:20.412102 2014-08-13 18:06:06.909587 7f5c973a57a0 15 read_log missing 194377'18590 (194377'18589) modify e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 by client.2231190.0:3165 2014-08-12 20:33:09.942727 2014-08-13 18:06:06.909622 7f5c973a57a0 15 read_log missing 194377'18588 (194377'18587) modify 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 by client.2231190.0:2921 2014-08-12 20:33:08.976531 2014-08-13 18:06:06.909659 7f5c973a57a0 15 read_log missing 194377'18586 (194377'18585) modify 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 by client.2231190.0:2183 2014-08-12 20:33:05.991812 2014-08-13 18:06:06.909694 7f5c973a57a0 15 read_log missing 194377'18584 (194377'18583) modify a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 by client.2231190.0:2017 2014-08-12 20:33:05.320173 2014-08-13 18:06:06.909730 7f5c973a57a0 15 read_log missing 194377'18582 (194377'18581) modify f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 by client.2231190.0:1301 2014-08-12 20:33:02.440688 2014-08-13 18:06:06.909766 7f5c973a57a0 15 read_log missing 194377'18580 (194377'18579) modify eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 by client.2231190.0:923 2014-08-12 20:33:01.007967 2014-08-13 18:06:06.909802 7f5c973a57a0 15 read_log missing 194377'18578 (194377'18577) modify d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 by client.2231190.0:510 2014-08-12 20:32:59.412977 2014-08-13 18:06:06.909838 7f5c973a57a0 15 read_log missing 194370'18576 (194370'18575) modify aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 by client.1508531.0:12132 2014-08-12 20:31:12.323754 2014-08-13 18:06:06.909875 7f5c973a57a0 15 read_log missing 194370'18574 (194370'18573) modify 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 by client.1508531.0:11697 2014-08-12 20:31:09.495710 2014-08-13 18:06:06.909910 7f5c973a57a0 15 read_log missing 194370'18572 (194370'18571) modify 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 by client.1508531.0:11225 2014-08-12 20:31:06.626292 2014-08-13 18:06:06.909946 7f5c973a57a0 15 read_log missing 194370'18570 (194370'18569) modify d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 by client.1508531.0:8281 2014-08-12 20:30:49.053145 2014-08-13 18:06:06.909982 7f5c973a57a0 15 read_log missing 194370'18568 (194370'18567) modify a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 by client.1508531.0:7259 2014-08-12 20:30:43.640862 2014-08-13 18:06:06.910018 7f5c973a57a0 15 read_log missing 194370'18566 (194370'18565) modify eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 by client.1508531.0:7024 2014-08-12 20:30:42.552204 2014-08-13 18:06:06.910054 7f5c973a57a0 15 read_log missing 194370'18564 (194370'18563) modify 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 by client.1508531.0:6655 2014-08-12 20:30:40.670736 2014-08-13 18:06:06.910090 7f5c973a57a0 15 read_log missing 194370'18562 (194370'18561) modify 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 by client.1508531.0:6327 2014-08-12 20:30:39.141385 2014-08-13 18:06:06.910126 7f5c973a57a0 15 read_log missing 194370'18560 (194370'18559) modify 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 by client.1508531.0:6024 2014-08-12 20:30:37.828080 2014-08-13 18:06:06.910167 7f5c973a57a0 15 read_log missing 194370'18558 (194370'18557) modify 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 by client.1508531.0:5674 2014-08-12 20:30:36.218820 2014-08-13 18:06:06.910204 7f5c973a57a0 15 read_log missing 194370'18556 (194370'18555) modify d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 by client.1508531.0:4972 2014-08-12 20:30:33.095955 2014-08-13 18:06:06.910241 7f5c973a57a0 15 read_log missing 194370'18554 (194370'18553) modify 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 by client.1508531.0:4881 2014-08-12 20:30:32.677788 2014-08-13 18:06:06.910280 7f5c973a57a0 15 read_log missing 194370'18552 (194370'18551) modify a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 by client.1508531.0:2246 2014-08-12 20:30:22.801067 2014-08-13 18:06:06.910317 7f5c973a57a0 15 read_log missing 194370'18550 (194370'18549) modify e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 by client.1508531.0:1577 2014-08-12 20:30:20.290518 2014-08-13 18:06:06.910358 7f5c973a57a0 15 read_log missing 194370'18548 (194370'18547) modify d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 by client.1508531.0:423 2014-08-12 20:30:15.977446 2014-08-13 18:06:06.910369 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.910407 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_loaded 2014-08-13 18:06:06.910435 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Initial 0.024653 0 0.000000 2014-08-13 18:06:06.910448 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset 2014-08-13 18:06:06.910458 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=0 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194487 2014-08-13 18:06:06.910468 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] log((192718'15573,194422'18660], crt=194422'18660) 2014-08-13 18:06:06.910492 7f5c973a57a0 10 osd.4 194487 pgid 24.33a coll 24.33a_head 2014-08-13 18:06:06.910636 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.33a 2014-08-13 18:06:06.910649 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.910682 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.33a(unlocked)] enter Initial 2014-08-13 18:06:06.910694 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.33a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.910837 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.910938 7f5c973a57a0 20 read_log 192834'14162 (192834'14161) modify 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.933448.0:1124 2014-08-11 15:23:40.014770 2014-08-13 18:06:06.910959 7f5c973a57a0 20 read_log 192834'14187 (0'0) promote 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by osd.285.0:15089 2014-08-11 15:23:45.523176 2014-08-13 18:06:06.910972 7f5c973a57a0 20 read_log 192834'14188 (192834'14187) modify 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.933448.0:2197 2014-08-11 15:23:45.514326 2014-08-13 18:06:06.910985 7f5c973a57a0 20 read_log 192834'14189 (0'0) promote fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by osd.285.0:15091 2014-08-11 15:23:45.601818 2014-08-13 18:06:06.910997 7f5c973a57a0 20 read_log 192834'14190 (192834'14189) modify fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.933448.0:2211 2014-08-11 15:23:45.592825 2014-08-13 18:06:06.911010 7f5c973a57a0 20 read_log 192834'14222 (0'0) promote 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by osd.285.0:15124 2014-08-11 15:23:59.605777 2014-08-13 18:06:06.911022 7f5c973a57a0 20 read_log 192834'14225 (192834'14222) modify 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.933448.0:4945 2014-08-11 15:23:59.429020 2014-08-13 18:06:06.911040 7f5c973a57a0 20 read_log 192834'14233 (0'0) promote 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by osd.285.0:15135 2014-08-11 15:24:02.302393 2014-08-13 18:06:06.911073 7f5c973a57a0 20 read_log 192834'14234 (192834'14233) modify 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.933448.0:5633 2014-08-11 15:24:02.294951 2014-08-13 18:06:06.911088 7f5c973a57a0 20 read_log 192834'14235 (0'0) promote f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by osd.285.0:15137 2014-08-11 15:24:03.045645 2014-08-13 18:06:06.911101 7f5c973a57a0 20 read_log 192834'14236 (192834'14235) modify f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.933448.0:5830 2014-08-11 15:24:03.037968 2014-08-13 18:06:06.911124 7f5c973a57a0 20 read_log 192834'14267 (0'0) promote 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by osd.285.0:15169 2014-08-11 15:24:12.234007 2014-08-13 18:06:06.911139 7f5c973a57a0 20 read_log 192834'14268 (192834'14267) modify 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.933448.0:8004 2014-08-11 15:24:12.226134 2014-08-13 18:06:06.911158 7f5c973a57a0 20 read_log 192834'14271 (0'0) promote 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by osd.285.0:15173 2014-08-11 15:24:13.386656 2014-08-13 18:06:06.911173 7f5c973a57a0 20 read_log 192834'14272 (192834'14271) modify 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.933448.0:8274 2014-08-11 15:24:13.380030 2014-08-13 18:06:06.911185 7f5c973a57a0 20 read_log 192834'14293 (0'0) promote 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by osd.285.0:15195 2014-08-11 15:24:18.366663 2014-08-13 18:06:06.911197 7f5c973a57a0 20 read_log 192834'14294 (192834'14293) modify 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.933448.0:9310 2014-08-11 15:24:18.359061 2014-08-13 18:06:06.911210 7f5c973a57a0 20 read_log 192834'14297 (0'0) promote f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by osd.285.0:15199 2014-08-11 15:24:19.581398 2014-08-13 18:06:06.911222 7f5c973a57a0 20 read_log 192834'14298 (192834'14297) modify f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.933448.0:9623 2014-08-11 15:24:19.573092 2014-08-13 18:06:06.911235 7f5c973a57a0 20 read_log 192834'14301 (0'0) promote 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by osd.285.0:15203 2014-08-11 15:24:21.738710 2014-08-13 18:06:06.911247 7f5c973a57a0 20 read_log 192834'14302 (192834'14301) modify 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.933448.0:10111 2014-08-11 15:24:21.730563 2014-08-13 18:06:06.911263 7f5c973a57a0 20 read_log 192834'14307 (0'0) promote d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by osd.285.0:15209 2014-08-11 15:24:24.854277 2014-08-13 18:06:06.911278 7f5c973a57a0 20 read_log 192834'14308 (192834'14307) modify d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.933448.0:10689 2014-08-11 15:24:24.846797 2014-08-13 18:06:06.911291 7f5c973a57a0 20 read_log 192834'14315 (0'0) promote f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by osd.285.0:15217 2014-08-11 15:24:25.363711 2014-08-13 18:06:06.911303 7f5c973a57a0 20 read_log 192834'14316 (192834'14315) modify f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.933448.0:10820 2014-08-11 15:24:25.356541 2014-08-13 18:06:06.911316 7f5c973a57a0 20 read_log 192834'14355 (0'0) promote 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by osd.285.0:15257 2014-08-11 15:24:35.458767 2014-08-13 18:06:06.911328 7f5c973a57a0 20 read_log 192834'14356 (192834'14355) modify 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.933448.0:13118 2014-08-11 15:24:35.449703 2014-08-13 18:06:06.911341 7f5c973a57a0 20 read_log 192834'14367 (0'0) promote c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by osd.285.0:15269 2014-08-11 15:24:46.127342 2014-08-13 18:06:06.911357 7f5c973a57a0 20 read_log 192834'14368 (192834'14367) modify c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.933448.0:14551 2014-08-11 15:24:46.119345 2014-08-13 18:06:06.911372 7f5c973a57a0 20 read_log 192834'14373 (0'0) promote 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by osd.285.0:15275 2014-08-11 15:24:46.298819 2014-08-13 18:06:06.911385 7f5c973a57a0 20 read_log 192834'14374 (192834'14373) modify 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.933448.0:14592 2014-08-11 15:24:46.288790 2014-08-13 18:06:06.911409 7f5c973a57a0 20 read_log 192834'14425 (0'0) promote 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by osd.285.0:15327 2014-08-11 15:24:58.358542 2014-08-13 18:06:06.911424 7f5c973a57a0 20 read_log 192834'14426 (192834'14425) modify 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.933448.0:17077 2014-08-11 15:24:58.350293 2014-08-13 18:06:06.911441 7f5c973a57a0 20 read_log 192834'14451 (0'0) promote b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by osd.285.0:15353 2014-08-11 15:25:06.581086 2014-08-13 18:06:06.911456 7f5c973a57a0 20 read_log 192834'14452 (192834'14451) modify b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.933448.0:18734 2014-08-11 15:25:06.570785 2014-08-13 18:06:06.911483 7f5c973a57a0 20 read_log 192834'14487 (0'0) promote 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by osd.285.0:15389 2014-08-11 15:25:21.338689 2014-08-13 18:06:06.911496 7f5c973a57a0 20 read_log 192834'14488 (192834'14487) modify 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.933448.0:20524 2014-08-11 15:25:21.329408 2014-08-13 18:06:06.911516 7f5c973a57a0 20 read_log 192834'14491 (0'0) promote 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by osd.285.0:15393 2014-08-11 15:25:22.162973 2014-08-13 18:06:06.911534 7f5c973a57a0 20 read_log 192834'14492 (192834'14491) modify 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.933448.0:20611 2014-08-11 15:25:22.152890 2014-08-13 18:06:06.911547 7f5c973a57a0 20 read_log 192834'14518 (192834'14160) modify e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by client.933448.0:23383 2014-08-11 15:25:39.294057 2014-08-13 18:06:06.911566 7f5c973a57a0 20 read_log 192834'14519 (192834'14162) modify 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.933448.0:23415 2014-08-11 15:25:39.329813 2014-08-13 18:06:06.911582 7f5c973a57a0 20 read_log 192834'14532 (192834'14188) modify 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.933448.0:24488 2014-08-11 15:25:40.937228 2014-08-13 18:06:06.911595 7f5c973a57a0 20 read_log 192834'14533 (192834'14190) modify fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.933448.0:24502 2014-08-11 15:25:40.962331 2014-08-13 18:06:06.911607 7f5c973a57a0 20 read_log 192834'14550 (192834'14225) modify 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.933448.0:27236 2014-08-11 15:25:44.261666 2014-08-13 18:06:06.911620 7f5c973a57a0 20 read_log 192834'14555 (192834'14234) modify 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.933448.0:27924 2014-08-11 15:25:45.267745 2014-08-13 18:06:06.911632 7f5c973a57a0 20 read_log 192834'14556 (192834'14236) modify f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.933448.0:28121 2014-08-11 15:25:45.527785 2014-08-13 18:06:06.911644 7f5c973a57a0 20 read_log 192834'14572 (192834'14268) modify 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.933448.0:30295 2014-08-11 15:25:48.696166 2014-08-13 18:06:06.911656 7f5c973a57a0 20 read_log 192834'14574 (192834'14272) modify 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.933448.0:30565 2014-08-11 15:25:49.000389 2014-08-13 18:06:06.911669 7f5c973a57a0 20 read_log 192834'14585 (192834'14294) modify 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.933448.0:31601 2014-08-11 15:25:50.259247 2014-08-13 18:06:06.911681 7f5c973a57a0 20 read_log 192834'14587 (192834'14298) modify f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.933448.0:31914 2014-08-11 15:25:50.792946 2014-08-13 18:06:06.911694 7f5c973a57a0 20 read_log 192834'14589 (192834'14302) modify 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.933448.0:32402 2014-08-11 15:25:51.734472 2014-08-13 18:06:06.911715 7f5c973a57a0 20 read_log 192834'14592 (192834'14308) modify d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.933448.0:32980 2014-08-11 15:25:52.710888 2014-08-13 18:06:06.911731 7f5c973a57a0 20 read_log 192834'14596 (192834'14316) modify f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.933448.0:33111 2014-08-11 15:25:52.855817 2014-08-13 18:06:06.911743 7f5c973a57a0 20 read_log 192834'14616 (192834'14356) modify 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.933448.0:35409 2014-08-11 15:25:56.219556 2014-08-13 18:06:06.911763 7f5c973a57a0 20 read_log 192834'14622 (192834'14368) modify c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.933448.0:36842 2014-08-11 15:25:58.362770 2014-08-13 18:06:06.911780 7f5c973a57a0 20 read_log 192834'14625 (192834'14374) modify 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.933448.0:36883 2014-08-11 15:25:58.435189 2014-08-13 18:06:06.911792 7f5c973a57a0 20 read_log 192834'14651 (192834'14426) modify 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.933448.0:39368 2014-08-11 15:26:01.827325 2014-08-13 18:06:06.911805 7f5c973a57a0 20 read_log 192834'14664 (192834'14452) modify b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.933448.0:41025 2014-08-11 15:26:02.818611 2014-08-13 18:06:06.911817 7f5c973a57a0 20 read_log 192834'14682 (192834'14488) modify 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.933448.0:42815 2014-08-11 15:26:03.396706 2014-08-13 18:06:06.911830 7f5c973a57a0 20 read_log 192834'14684 (192834'14492) modify 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.933448.0:42902 2014-08-11 15:26:03.428755 2014-08-13 18:06:06.911842 7f5c973a57a0 20 read_log 192834'14707 (0'0) promote 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by osd.285.0:15609 2014-08-11 15:26:16.709175 2014-08-13 18:06:06.911862 7f5c973a57a0 20 read_log 192834'14708 (192834'14707) modify 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.1883634.0:608 2014-08-11 15:26:16.702233 2014-08-13 18:06:06.911878 7f5c973a57a0 20 read_log 192834'14709 (0'0) promote 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by osd.285.0:15611 2014-08-11 15:26:17.058838 2014-08-13 18:06:06.911891 7f5c973a57a0 20 read_log 192834'14710 (192834'14709) modify 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.1883634.0:705 2014-08-11 15:26:17.050502 2014-08-13 18:06:06.911903 7f5c973a57a0 20 read_log 192834'14717 (0'0) promote 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by osd.285.0:15619 2014-08-11 15:26:20.496379 2014-08-13 18:06:06.911916 7f5c973a57a0 20 read_log 192834'14718 (192834'14717) modify 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.1883634.0:1655 2014-08-11 15:26:20.488970 2014-08-13 18:06:06.911928 7f5c973a57a0 20 read_log 192834'14719 (0'0) promote b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by osd.285.0:15621 2014-08-11 15:26:20.517988 2014-08-13 18:06:06.911942 7f5c973a57a0 20 read_log 192834'14720 (192834'14719) modify b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.1883634.0:1661 2014-08-11 15:26:20.510842 2014-08-13 18:06:06.911954 7f5c973a57a0 20 read_log 192834'14721 (0'0) promote 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by osd.285.0:15623 2014-08-11 15:26:20.634423 2014-08-13 18:06:06.911967 7f5c973a57a0 20 read_log 192834'14722 (192834'14721) modify 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.1883634.0:1693 2014-08-11 15:26:20.626192 2014-08-13 18:06:06.911979 7f5c973a57a0 20 read_log 192834'14753 (0'0) promote f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by osd.285.0:15655 2014-08-11 15:26:28.217223 2014-08-13 18:06:06.912002 7f5c973a57a0 20 read_log 192834'14754 (192834'14753) modify f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.1883634.0:3780 2014-08-11 15:26:28.209413 2014-08-13 18:06:06.912018 7f5c973a57a0 20 read_log 192834'14757 (0'0) promote 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by osd.285.0:15659 2014-08-11 15:26:29.202101 2014-08-13 18:06:06.912030 7f5c973a57a0 20 read_log 192834'14758 (192834'14757) modify 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.1883634.0:4058 2014-08-11 15:26:29.193426 2014-08-13 18:06:06.912046 7f5c973a57a0 20 read_log 192834'14763 (0'0) promote 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by osd.285.0:15665 2014-08-11 15:26:33.201305 2014-08-13 18:06:06.912061 7f5c973a57a0 20 read_log 192834'14764 (192834'14763) modify 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.1883634.0:5095 2014-08-11 15:26:33.193287 2014-08-13 18:06:06.912074 7f5c973a57a0 20 read_log 192834'14765 (0'0) promote f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by osd.285.0:15667 2014-08-11 15:26:33.733870 2014-08-13 18:06:06.912089 7f5c973a57a0 20 read_log 192834'14766 (192834'14765) modify f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.1883634.0:5247 2014-08-11 15:26:33.725826 2014-08-13 18:06:06.912104 7f5c973a57a0 20 read_log 192834'14777 (0'0) promote f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by osd.285.0:15679 2014-08-11 15:26:36.161859 2014-08-13 18:06:06.912116 7f5c973a57a0 20 read_log 192834'14778 (192834'14777) modify f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.1883634.0:5931 2014-08-11 15:26:36.153295 2014-08-13 18:06:06.912153 7f5c973a57a0 20 read_log 192834'14785 (0'0) promote 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by osd.285.0:15687 2014-08-11 15:26:39.275795 2014-08-13 18:06:06.912179 7f5c973a57a0 20 read_log 192834'14786 (192834'14785) modify 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.1883634.0:6720 2014-08-11 15:26:39.267074 2014-08-13 18:06:06.912196 7f5c973a57a0 20 read_log 192834'14791 (0'0) promote a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by osd.285.0:15693 2014-08-11 15:26:39.645606 2014-08-13 18:06:06.912208 7f5c973a57a0 20 read_log 192834'14792 (192834'14791) modify a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.1883634.0:6827 2014-08-11 15:26:39.637399 2014-08-13 18:06:06.912231 7f5c973a57a0 20 read_log 192834'14803 (0'0) promote 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by osd.285.0:15705 2014-08-11 15:26:43.581545 2014-08-13 18:06:06.912247 7f5c973a57a0 20 read_log 192834'14804 (192834'14803) modify 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.1883634.0:7770 2014-08-11 15:26:43.572710 2014-08-13 18:06:06.912260 7f5c973a57a0 20 read_log 192834'14833 (0'0) promote 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by osd.285.0:15735 2014-08-11 15:26:50.087951 2014-08-13 18:06:06.912283 7f5c973a57a0 20 read_log 192834'14834 (192834'14833) modify 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.1883634.0:9252 2014-08-11 15:26:50.079805 2014-08-13 18:06:06.912296 7f5c973a57a0 20 read_log 192834'14841 (0'0) promote 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by osd.285.0:15743 2014-08-11 15:26:51.999674 2014-08-13 18:06:06.912308 7f5c973a57a0 20 read_log 192834'14842 (192834'14841) modify 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.1883634.0:9702 2014-08-11 15:26:51.989350 2014-08-13 18:06:06.912326 7f5c973a57a0 20 read_log 192834'14849 (0'0) promote 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by osd.285.0:15751 2014-08-11 15:26:53.728506 2014-08-13 18:06:06.912429 7f5c973a57a0 20 read_log 192834'14850 (192834'14849) modify 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.1883634.0:10110 2014-08-11 15:26:53.721002 2014-08-13 18:06:06.912449 7f5c973a57a0 20 read_log 192834'14851 (0'0) promote c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by osd.285.0:15753 2014-08-11 15:26:54.205151 2014-08-13 18:06:06.912462 7f5c973a57a0 20 read_log 192834'14852 (192834'14851) modify c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.1883634.0:10239 2014-08-11 15:26:54.197679 2014-08-13 18:06:06.912474 7f5c973a57a0 20 read_log 192834'14873 (0'0) promote fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by osd.285.0:15775 2014-08-11 15:27:01.208552 2014-08-13 18:06:06.912487 7f5c973a57a0 20 read_log 192834'14874 (192834'14873) modify fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.1883634.0:11762 2014-08-11 15:27:01.197866 2014-08-13 18:06:06.912499 7f5c973a57a0 20 read_log 192834'14879 (0'0) promote 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by osd.285.0:15781 2014-08-11 15:27:02.665953 2014-08-13 18:06:06.912511 7f5c973a57a0 20 read_log 192834'14880 (192834'14879) modify 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.1883634.0:12138 2014-08-11 15:27:02.655752 2014-08-13 18:06:06.912523 7f5c973a57a0 20 read_log 192834'14885 (0'0) promote ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by osd.285.0:15787 2014-08-11 15:27:04.358375 2014-08-13 18:06:06.912536 7f5c973a57a0 20 read_log 192834'14886 (192834'14885) modify ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.1883634.0:12529 2014-08-11 15:27:04.349833 2014-08-13 18:06:06.912548 7f5c973a57a0 20 read_log 192834'14893 (0'0) promote 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by osd.285.0:15795 2014-08-11 15:27:10.125759 2014-08-13 18:06:06.912577 7f5c973a57a0 20 read_log 192834'14894 (192834'14893) modify 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.1883634.0:13572 2014-08-11 15:27:10.118053 2014-08-13 18:06:06.912593 7f5c973a57a0 20 read_log 192834'14901 (0'0) promote ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by osd.285.0:15803 2014-08-11 15:27:12.647227 2014-08-13 18:06:06.912605 7f5c973a57a0 20 read_log 192834'14902 (192834'14901) modify ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.1883634.0:14303 2014-08-11 15:27:12.638463 2014-08-13 18:06:06.912618 7f5c973a57a0 20 read_log 192834'14917 (0'0) promote 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by osd.285.0:15819 2014-08-11 15:27:15.934264 2014-08-13 18:06:06.912630 7f5c973a57a0 20 read_log 192834'14918 (192834'14917) modify 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.1883634.0:14895 2014-08-11 15:27:15.925402 2014-08-13 18:06:06.912642 7f5c973a57a0 20 read_log 192834'14975 (0'0) promote b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by osd.285.0:15877 2014-08-11 15:27:56.370505 2014-08-13 18:06:06.912655 7f5c973a57a0 20 read_log 192834'14976 (192834'14975) modify b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.1883634.0:20358 2014-08-11 15:27:56.362062 2014-08-13 18:06:06.912668 7f5c973a57a0 20 read_log 192834'14987 (0'0) promote 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by osd.285.0:15889 2014-08-11 15:28:03.035761 2014-08-13 18:06:06.912680 7f5c973a57a0 20 read_log 192834'14988 (192834'14987) modify 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.1883634.0:21149 2014-08-11 15:28:03.026835 2014-08-13 18:06:06.912692 7f5c973a57a0 20 read_log 192834'14999 (0'0) promote f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by osd.285.0:15901 2014-08-11 15:28:04.935339 2014-08-13 18:06:06.912720 7f5c973a57a0 20 read_log 192834'15000 (192834'14999) modify f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.1883634.0:21677 2014-08-11 15:28:04.926949 2014-08-13 18:06:06.912736 7f5c973a57a0 20 read_log 192834'15009 (0'0) promote 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by osd.285.0:15911 2014-08-11 15:28:06.737540 2014-08-13 18:06:06.912748 7f5c973a57a0 20 read_log 192834'15010 (192834'15009) modify 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.1883634.0:22107 2014-08-11 15:28:06.728351 2014-08-13 18:06:06.912762 7f5c973a57a0 20 read_log 192834'15011 (0'0) promote d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by osd.285.0:15913 2014-08-11 15:28:07.306261 2014-08-13 18:06:06.912777 7f5c973a57a0 20 read_log 192834'15012 (192834'15011) modify d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.1883634.0:22209 2014-08-11 15:28:07.296450 2014-08-13 18:06:06.912789 7f5c973a57a0 20 read_log 192834'15015 (0'0) promote 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by osd.285.0:15917 2014-08-11 15:28:08.528707 2014-08-13 18:06:06.912803 7f5c973a57a0 20 read_log 192834'15016 (192834'15015) modify 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.1883634.0:22417 2014-08-11 15:28:08.518458 2014-08-13 18:06:06.912817 7f5c973a57a0 20 read_log 192834'15037 (192834'14708) modify 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.1883634.0:23932 2014-08-11 15:28:18.102226 2014-08-13 18:06:06.912831 7f5c973a57a0 20 read_log 192834'15038 (192834'14710) modify 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.1883634.0:24029 2014-08-11 15:28:18.258434 2014-08-13 18:06:06.912844 7f5c973a57a0 20 read_log 192834'15042 (192834'14718) modify 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.1883634.0:24979 2014-08-11 15:28:19.853757 2014-08-13 18:06:06.912858 7f5c973a57a0 20 read_log 192834'15043 (192834'14720) modify b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.1883634.0:24985 2014-08-11 15:28:19.862282 2014-08-13 18:06:06.912870 7f5c973a57a0 20 read_log 192834'15044 (192834'14722) modify 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.1883634.0:25017 2014-08-11 15:28:19.944148 2014-08-13 18:06:06.912883 7f5c973a57a0 20 read_log 192834'15060 (192834'14754) modify f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.1883634.0:27104 2014-08-11 15:28:24.038126 2014-08-13 18:06:06.912895 7f5c973a57a0 20 read_log 192834'15062 (192834'14758) modify 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.1883634.0:27382 2014-08-11 15:28:24.429378 2014-08-13 18:06:06.912929 7f5c973a57a0 20 read_log 192834'15065 (192834'14764) modify 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.1883634.0:28419 2014-08-11 15:28:25.787274 2014-08-13 18:06:06.912945 7f5c973a57a0 20 read_log 192834'15066 (192834'14766) modify f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.1883634.0:28571 2014-08-11 15:28:25.997599 2014-08-13 18:06:06.912958 7f5c973a57a0 20 read_log 192834'15072 (192834'14778) modify f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.1883634.0:29255 2014-08-11 15:28:26.895782 2014-08-13 18:06:06.912972 7f5c973a57a0 20 read_log 192834'15076 (192834'14786) modify 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.1883634.0:30044 2014-08-11 15:28:27.999688 2014-08-13 18:06:06.912984 7f5c973a57a0 20 read_log 192834'15079 (192834'14792) modify a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.1883634.0:30151 2014-08-11 15:28:28.168122 2014-08-13 18:06:06.912997 7f5c973a57a0 20 read_log 192834'15085 (192834'14804) modify 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.1883634.0:31094 2014-08-11 15:28:29.878010 2014-08-13 18:06:06.913020 7f5c973a57a0 20 read_log 192834'15100 (192834'14834) modify 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.1883634.0:32576 2014-08-11 15:28:32.041413 2014-08-13 18:06:06.913035 7f5c973a57a0 20 read_log 192834'15104 (192834'14842) modify 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.1883634.0:33026 2014-08-11 15:28:32.686551 2014-08-13 18:06:06.913047 7f5c973a57a0 20 read_log 192834'15108 (192834'14850) modify 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.1883634.0:33434 2014-08-11 15:28:33.192983 2014-08-13 18:06:06.913062 7f5c973a57a0 20 read_log 192834'15109 (192834'14852) modify c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.1883634.0:33563 2014-08-11 15:28:33.464327 2014-08-13 18:06:06.913075 7f5c973a57a0 20 read_log 192834'15122 (192834'14874) modify fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.1883634.0:35086 2014-08-11 15:28:36.003340 2014-08-13 18:06:06.913089 7f5c973a57a0 20 read_log 192834'15125 (192834'14880) modify 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.1883634.0:35462 2014-08-11 15:28:36.667851 2014-08-13 18:06:06.913101 7f5c973a57a0 20 read_log 192834'15128 (192834'14886) modify ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.1883634.0:35853 2014-08-11 15:28:37.227144 2014-08-13 18:06:06.913118 7f5c973a57a0 20 read_log 192834'15132 (192834'14894) modify 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.1883634.0:36896 2014-08-11 15:28:38.556572 2014-08-13 18:06:06.913133 7f5c973a57a0 20 read_log 192834'15136 (192834'14902) modify ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.1883634.0:37627 2014-08-11 15:28:39.585103 2014-08-13 18:06:06.913151 7f5c973a57a0 20 read_log 192834'15144 (192834'14918) modify 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.1883634.0:38219 2014-08-11 15:28:40.372122 2014-08-13 18:06:06.913166 7f5c973a57a0 20 read_log 192834'15173 (192834'14976) modify b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.1883634.0:43682 2014-08-11 15:28:46.631171 2014-08-13 18:06:06.913179 7f5c973a57a0 20 read_log 192834'15179 (192834'14988) modify 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.1883634.0:44473 2014-08-11 15:28:46.868141 2014-08-13 18:06:06.913191 7f5c973a57a0 20 read_log 192834'15185 (192834'15000) modify f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.1883634.0:45001 2014-08-11 15:28:47.119151 2014-08-13 18:06:06.913204 7f5c973a57a0 20 read_log 192834'15190 (192834'15010) modify 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.1883634.0:45431 2014-08-11 15:28:47.275251 2014-08-13 18:06:06.913216 7f5c973a57a0 20 read_log 192834'15191 (192834'15012) modify d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.1883634.0:45533 2014-08-11 15:28:47.309162 2014-08-13 18:06:06.913228 7f5c973a57a0 20 read_log 192834'15193 (192834'15016) modify 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.1883634.0:45741 2014-08-11 15:28:47.371580 2014-08-13 18:06:06.913243 7f5c973a57a0 20 read_log 192888'15202 (0'0) promote 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by osd.368.0:33045 2014-08-11 16:44:55.898755 2014-08-13 18:06:06.913255 7f5c973a57a0 20 read_log 192888'15203 (192888'15202) modify 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.1315259.0:245 2014-08-11 16:44:55.889706 2014-08-13 18:06:06.913270 7f5c973a57a0 20 read_log 192888'15208 (0'0) promote be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by osd.368.0:33093 2014-08-11 16:45:00.853385 2014-08-13 18:06:06.913285 7f5c973a57a0 20 read_log 192888'15209 (192888'15208) modify be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.1315259.0:1454 2014-08-11 16:45:00.846015 2014-08-13 18:06:06.913310 7f5c973a57a0 20 read_log 192888'15212 (0'0) promote 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by osd.368.0:33111 2014-08-11 16:45:03.308596 2014-08-13 18:06:06.913325 7f5c973a57a0 20 read_log 192888'15213 (192888'15212) modify 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.1315259.0:2118 2014-08-11 16:45:03.301151 2014-08-13 18:06:06.913341 7f5c973a57a0 20 read_log 192888'15220 (0'0) promote eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by osd.368.0:33167 2014-08-11 16:45:12.788806 2014-08-13 18:06:06.913353 7f5c973a57a0 20 read_log 192888'15221 (192888'15220) modify eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.1315259.0:4641 2014-08-11 16:45:12.783258 2014-08-13 18:06:06.913366 7f5c973a57a0 20 read_log 192888'15224 (0'0) promote 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by osd.368.0:33189 2014-08-11 16:45:14.648375 2014-08-13 18:06:06.913378 7f5c973a57a0 20 read_log 192888'15225 (192888'15224) modify 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.1315259.0:5135 2014-08-11 16:45:14.641313 2014-08-13 18:06:06.913395 7f5c973a57a0 20 read_log 192888'15230 (0'0) promote a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by osd.368.0:33221 2014-08-11 16:45:20.269191 2014-08-13 18:06:06.913408 7f5c973a57a0 20 read_log 192888'15231 (192888'15230) modify a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.1315259.0:6688 2014-08-11 16:45:20.258889 2014-08-13 18:06:06.913420 7f5c973a57a0 20 read_log 192888'15232 (0'0) promote bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by osd.368.0:33223 2014-08-11 16:45:20.469987 2014-08-13 18:06:06.913449 7f5c973a57a0 20 read_log 192888'15233 (192888'15232) modify bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.1315259.0:6744 2014-08-11 16:45:20.461612 2014-08-13 18:06:06.913462 7f5c973a57a0 20 read_log 192888'15234 (0'0) promote b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by osd.368.0:33243 2014-08-11 16:45:21.413626 2014-08-13 18:06:06.913481 7f5c973a57a0 20 read_log 192888'15235 (192888'15234) modify b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.1315259.0:7004 2014-08-11 16:45:21.406100 2014-08-13 18:06:06.913496 7f5c973a57a0 20 read_log 192888'15236 (0'0) promote 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by osd.368.0:33251 2014-08-11 16:45:22.795822 2014-08-13 18:06:06.913509 7f5c973a57a0 20 read_log 192888'15237 (192888'15236) modify 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.1315259.0:7386 2014-08-11 16:45:22.785937 2014-08-13 18:06:06.913528 7f5c973a57a0 20 read_log 192888'15238 (0'0) promote a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by osd.368.0:33253 2014-08-11 16:45:22.878540 2014-08-13 18:06:06.913544 7f5c973a57a0 20 read_log 192888'15239 (192888'15238) modify a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.1315259.0:7409 2014-08-11 16:45:22.869413 2014-08-13 18:06:06.913556 7f5c973a57a0 20 read_log 192888'15240 (0'0) promote 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by osd.368.0:33285 2014-08-11 16:45:26.226445 2014-08-13 18:06:06.913569 7f5c973a57a0 20 read_log 192888'15241 (192888'15240) modify 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.1315259.0:8328 2014-08-11 16:45:26.219207 2014-08-13 18:06:06.913594 7f5c973a57a0 20 read_log 192888'15246 (0'0) promote de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by osd.368.0:33343 2014-08-11 16:45:33.543662 2014-08-13 18:06:06.913607 7f5c973a57a0 20 read_log 192888'15247 (192888'15246) modify de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.1315259.0:10239 2014-08-11 16:45:33.536133 2014-08-13 18:06:06.913642 7f5c973a57a0 20 read_log 192888'15250 (0'0) promote 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by osd.368.0:33351 2014-08-11 16:45:34.314300 2014-08-13 18:06:06.913658 7f5c973a57a0 20 read_log 192888'15251 (192888'15250) modify 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.1315259.0:10426 2014-08-11 16:45:34.306249 2014-08-13 18:06:06.913671 7f5c973a57a0 20 read_log 192888'15258 (0'0) promote 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by osd.368.0:33381 2014-08-11 16:45:40.951726 2014-08-13 18:06:06.913692 7f5c973a57a0 20 read_log 192888'15259 (192888'15258) modify 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.1315259.0:12068 2014-08-11 16:45:40.943960 2014-08-13 18:06:06.913708 7f5c973a57a0 20 read_log 192888'15266 (0'0) promote be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by osd.368.0:33397 2014-08-11 16:45:44.126038 2014-08-13 18:06:06.913720 7f5c973a57a0 20 read_log 192888'15267 (192888'15266) modify be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.1315259.0:12823 2014-08-11 16:45:44.117411 2014-08-13 18:06:06.913733 7f5c973a57a0 20 read_log 192888'15268 (0'0) promote 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by osd.368.0:33465 2014-08-11 16:45:52.568381 2014-08-13 18:06:06.913745 7f5c973a57a0 20 read_log 192888'15269 (192888'15268) modify 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.1315259.0:15003 2014-08-11 16:45:52.559759 2014-08-13 18:06:06.913757 7f5c973a57a0 20 read_log 192888'15270 (0'0) promote a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by osd.368.0:33467 2014-08-11 16:45:52.850646 2014-08-13 18:06:06.913769 7f5c973a57a0 20 read_log 192888'15271 (192888'15270) modify a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.1315259.0:15074 2014-08-11 16:45:52.842828 2014-08-13 18:06:06.913782 7f5c973a57a0 20 read_log 192888'15272 (0'0) promote 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by osd.368.0:33479 2014-08-11 16:45:54.838099 2014-08-13 18:06:06.913794 7f5c973a57a0 20 read_log 192888'15273 (192888'15272) modify 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.1315259.0:15578 2014-08-11 16:45:54.831057 2014-08-13 18:06:06.913806 7f5c973a57a0 20 read_log 192888'15274 (0'0) promote 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by osd.368.0:33507 2014-08-11 16:45:58.693949 2014-08-13 18:06:06.913818 7f5c973a57a0 20 read_log 192888'15275 (192888'15274) modify 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.1315259.0:16574 2014-08-11 16:45:58.686505 2014-08-13 18:06:06.913831 7f5c973a57a0 20 read_log 192888'15278 (0'0) promote c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by osd.368.0:33523 2014-08-11 16:46:02.163265 2014-08-13 18:06:06.913844 7f5c973a57a0 20 read_log 192888'15279 (192888'15278) modify c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.1315259.0:17464 2014-08-11 16:46:02.154428 2014-08-13 18:06:06.913857 7f5c973a57a0 20 read_log 192888'15284 (0'0) promote 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by osd.368.0:33597 2014-08-11 16:46:13.701128 2014-08-13 18:06:06.913869 7f5c973a57a0 20 read_log 192888'15285 (192888'15284) modify 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.1315259.0:20430 2014-08-11 16:46:13.693922 2014-08-13 18:06:06.913881 7f5c973a57a0 20 read_log 192888'15294 (0'0) promote 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by osd.368.0:33721 2014-08-11 16:46:29.003707 2014-08-13 18:06:06.913894 7f5c973a57a0 20 read_log 192888'15295 (192888'15294) modify 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.1315259.0:24364 2014-08-11 16:46:28.996545 2014-08-13 18:06:06.913936 7f5c973a57a0 20 read_log 192888'15296 (0'0) promote b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by osd.368.0:33723 2014-08-11 16:46:29.398063 2014-08-13 18:06:06.913952 7f5c973a57a0 20 read_log 192888'15297 (192888'15296) modify b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.1315259.0:24461 2014-08-11 16:46:29.390049 2014-08-13 18:06:06.913965 7f5c973a57a0 20 read_log 192888'15298 (0'0) promote 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by osd.368.0:33735 2014-08-11 16:46:30.818931 2014-08-13 18:06:06.913995 7f5c973a57a0 20 read_log 192888'15299 (192888'15298) modify 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.1315259.0:24813 2014-08-11 16:46:30.811518 2014-08-13 18:06:06.914010 7f5c973a57a0 20 read_log 192888'15302 (0'0) promote 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by osd.368.0:33739 2014-08-11 16:46:31.877404 2014-08-13 18:06:06.914023 7f5c973a57a0 20 read_log 192888'15303 (192888'15302) modify 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.1315259.0:25099 2014-08-11 16:46:31.868799 2014-08-13 18:06:06.914035 7f5c973a57a0 20 read_log 192888'15310 (0'0) promote ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by osd.368.0:33791 2014-08-11 16:46:39.631796 2014-08-13 18:06:06.914047 7f5c973a57a0 20 read_log 192888'15311 (192888'15310) modify ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.1315259.0:26942 2014-08-11 16:46:39.625045 2014-08-13 18:06:06.914060 7f5c973a57a0 20 read_log 192888'15312 (0'0) promote 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by osd.368.0:33799 2014-08-11 16:46:42.870797 2014-08-13 18:06:06.914072 7f5c973a57a0 20 read_log 192888'15313 (192888'15312) modify 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.1315259.0:27720 2014-08-11 16:46:42.863053 2014-08-13 18:06:06.914084 7f5c973a57a0 20 read_log 192888'15316 (0'0) promote d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by osd.368.0:33807 2014-08-11 16:46:44.468653 2014-08-13 18:06:06.914098 7f5c973a57a0 20 read_log 192888'15317 (192888'15316) modify d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.1315259.0:28130 2014-08-11 16:46:44.460055 2014-08-13 18:06:06.914110 7f5c973a57a0 20 read_log 192888'15318 (0'0) promote 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by osd.368.0:33817 2014-08-11 16:46:45.645608 2014-08-13 18:06:06.914122 7f5c973a57a0 20 read_log 192888'15319 (192888'15318) modify 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.1315259.0:28407 2014-08-11 16:46:45.638001 2014-08-13 18:06:06.914135 7f5c973a57a0 20 read_log 192888'15324 (192888'15203) modify 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.1315259.0:30972 2014-08-11 16:46:55.954528 2014-08-13 18:06:06.914157 7f5c973a57a0 20 read_log 192888'15327 (192888'15209) modify be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.1315259.0:32181 2014-08-11 16:46:56.308882 2014-08-13 18:06:06.914173 7f5c973a57a0 20 read_log 192888'15329 (192888'15213) modify 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.1315259.0:32845 2014-08-11 16:46:56.484376 2014-08-13 18:06:06.914185 7f5c973a57a0 20 read_log 192888'15333 (192888'15221) modify eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.1315259.0:35368 2014-08-11 16:46:57.146865 2014-08-13 18:06:06.914198 7f5c973a57a0 20 read_log 192888'15335 (192888'15225) modify 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.1315259.0:35862 2014-08-11 16:46:57.265215 2014-08-13 18:06:06.914210 7f5c973a57a0 20 read_log 192888'15338 (192888'15231) modify a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.1315259.0:37415 2014-08-11 16:46:57.677054 2014-08-13 18:06:06.914237 7f5c973a57a0 20 read_log 192888'15339 (192888'15233) modify bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.1315259.0:37471 2014-08-11 16:46:57.692255 2014-08-13 18:06:06.914253 7f5c973a57a0 20 read_log 192888'15340 (192888'15235) modify b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.1315259.0:37731 2014-08-11 16:46:57.759153 2014-08-13 18:06:06.914265 7f5c973a57a0 20 read_log 192888'15341 (192888'15237) modify 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.1315259.0:38113 2014-08-11 16:46:57.860451 2014-08-13 18:06:06.914278 7f5c973a57a0 20 read_log 192888'15342 (192888'15239) modify a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.1315259.0:38136 2014-08-11 16:46:57.865822 2014-08-13 18:06:06.914290 7f5c973a57a0 20 read_log 192888'15343 (192888'15241) modify 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.1315259.0:39055 2014-08-11 16:46:58.114634 2014-08-13 18:06:06.914303 7f5c973a57a0 20 read_log 192888'15346 (192888'15247) modify de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.1315259.0:40966 2014-08-11 16:46:58.627342 2014-08-13 18:06:06.914315 7f5c973a57a0 20 read_log 192888'15348 (192888'15251) modify 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.1315259.0:41153 2014-08-11 16:46:58.672822 2014-08-13 18:06:06.914327 7f5c973a57a0 20 read_log 192888'15352 (192888'15259) modify 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.1315259.0:42795 2014-08-11 16:46:59.108049 2014-08-13 18:06:06.914340 7f5c973a57a0 20 read_log 192888'15356 (192888'15267) modify be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.1315259.0:43550 2014-08-11 16:46:59.322360 2014-08-13 18:06:06.914352 7f5c973a57a0 20 read_log 192888'15357 (192888'15269) modify 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.1315259.0:45730 2014-08-11 16:46:59.952823 2014-08-13 18:06:06.914364 7f5c973a57a0 20 read_log 192888'15358 (192888'15271) modify a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.1315259.0:45801 2014-08-11 16:46:59.974189 2014-08-13 18:06:06.914387 7f5c973a57a0 20 read_log 192888'15359 (192888'15273) modify 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.1315259.0:46305 2014-08-11 16:47:00.120898 2014-08-13 18:06:06.914408 7f5c973a57a0 20 read_log 192888'15360 (192888'15275) modify 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.1315259.0:47301 2014-08-11 16:47:00.414315 2014-08-13 18:06:06.914434 7f5c973a57a0 20 read_log 192888'15362 (192888'15279) modify c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.1315259.0:48191 2014-08-11 16:47:00.693678 2014-08-13 18:06:06.914450 7f5c973a57a0 20 read_log 192888'15365 (192888'15285) modify 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.1315259.0:51157 2014-08-11 16:47:01.596434 2014-08-13 18:06:06.914465 7f5c973a57a0 20 read_log 192888'15370 (192888'15295) modify 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.1315259.0:55091 2014-08-11 16:47:02.810051 2014-08-13 18:06:06.914478 7f5c973a57a0 20 read_log 192888'15371 (192888'15297) modify b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.1315259.0:55188 2014-08-11 16:47:02.841108 2014-08-13 18:06:06.914490 7f5c973a57a0 20 read_log 192888'15372 (192888'15299) modify 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.1315259.0:55540 2014-08-11 16:47:02.942272 2014-08-13 18:06:06.914503 7f5c973a57a0 20 read_log 192888'15374 (192888'15303) modify 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.1315259.0:55826 2014-08-11 16:47:03.028204 2014-08-13 18:06:06.914526 7f5c973a57a0 20 read_log 192888'15378 (192888'15311) modify ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.1315259.0:57669 2014-08-11 16:47:03.578571 2014-08-13 18:06:06.914565 7f5c973a57a0 20 read_log 192888'15379 (192888'15313) modify 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.1315259.0:58447 2014-08-11 16:47:03.804436 2014-08-13 18:06:06.914581 7f5c973a57a0 20 read_log 192888'15381 (192888'15317) modify d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.1315259.0:58857 2014-08-11 16:47:03.928922 2014-08-13 18:06:06.914593 7f5c973a57a0 20 read_log 192888'15382 (192888'15319) modify 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.1315259.0:59134 2014-08-11 16:47:04.013559 2014-08-13 18:06:06.914606 7f5c973a57a0 20 read_log 192888'15395 (0'0) promote 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by osd.368.0:34361 2014-08-11 17:05:21.369147 2014-08-13 18:06:06.914618 7f5c973a57a0 20 read_log 192888'15396 (192888'15395) modify 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.942868.0:1552 2014-08-11 17:05:21.361364 2014-08-13 18:06:06.914631 7f5c973a57a0 20 read_log 192888'15397 (0'0) promote f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by osd.368.0:34373 2014-08-11 17:05:22.107220 2014-08-13 18:06:06.914643 7f5c973a57a0 20 read_log 192888'15398 (192888'15397) modify f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.942868.0:1717 2014-08-11 17:05:22.098848 2014-08-13 18:06:06.914655 7f5c973a57a0 20 read_log 192888'15399 (0'0) promote 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by osd.368.0:34399 2014-08-11 17:05:25.608119 2014-08-13 18:06:06.914668 7f5c973a57a0 20 read_log 192888'15400 (192888'15399) modify 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.942868.0:2612 2014-08-11 17:05:25.600995 2014-08-13 18:06:06.914680 7f5c973a57a0 20 read_log 192888'15405 (0'0) promote 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by osd.368.0:34427 2014-08-11 17:05:30.632236 2014-08-13 18:06:06.914692 7f5c973a57a0 20 read_log 192888'15406 (192888'15405) modify 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.942868.0:3971 2014-08-11 17:05:30.622700 2014-08-13 18:06:06.914705 7f5c973a57a0 20 read_log 192888'15407 (0'0) promote 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by osd.368.0:34431 2014-08-11 17:05:31.377944 2014-08-13 18:06:06.914717 7f5c973a57a0 20 read_log 192888'15408 (192888'15407) modify 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.942868.0:4174 2014-08-11 17:05:31.369321 2014-08-13 18:06:06.914730 7f5c973a57a0 20 read_log 192888'15409 (0'0) promote 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by osd.368.0:34455 2014-08-11 17:05:35.728450 2014-08-13 18:06:06.914742 7f5c973a57a0 20 read_log 192888'15410 (192888'15409) modify 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.942868.0:5295 2014-08-11 17:05:35.719816 2014-08-13 18:06:06.914754 7f5c973a57a0 20 read_log 192888'15413 (0'0) promote 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by osd.368.0:34463 2014-08-11 17:05:37.044725 2014-08-13 18:06:06.914767 7f5c973a57a0 20 read_log 192888'15414 (192888'15413) modify 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.942868.0:5649 2014-08-11 17:05:37.037132 2014-08-13 18:06:06.914779 7f5c973a57a0 20 read_log 192888'15427 (0'0) promote 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by osd.368.0:34583 2014-08-11 17:05:52.433754 2014-08-13 18:06:06.914792 7f5c973a57a0 20 read_log 192888'15428 (192888'15427) modify 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.942868.0:9787 2014-08-11 17:05:52.425731 2014-08-13 18:06:06.914804 7f5c973a57a0 20 read_log 192888'15433 (0'0) promote 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by osd.368.0:34649 2014-08-11 17:06:00.661890 2014-08-13 18:06:06.914823 7f5c973a57a0 20 read_log 192888'15434 (192888'15433) modify 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.942868.0:12043 2014-08-11 17:06:00.655161 2014-08-13 18:06:06.914836 7f5c973a57a0 20 read_log 192888'15443 (0'0) promote d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by osd.368.0:34693 2014-08-11 17:06:05.809441 2014-08-13 18:06:06.914848 7f5c973a57a0 20 read_log 192888'15444 (192888'15443) modify d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.942868.0:13410 2014-08-11 17:06:05.802176 2014-08-13 18:06:06.914861 7f5c973a57a0 20 read_log 192888'15445 (0'0) promote b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by osd.368.0:34733 2014-08-11 17:06:13.119785 2014-08-13 18:06:06.914873 7f5c973a57a0 20 read_log 192888'15446 (192888'15445) modify b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.942868.0:15230 2014-08-11 17:06:13.111811 2014-08-13 18:06:06.914888 7f5c973a57a0 20 read_log 192888'15451 (0'0) promote af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by osd.368.0:34755 2014-08-11 17:06:15.744963 2014-08-13 18:06:06.914900 7f5c973a57a0 20 read_log 192888'15452 (192888'15451) modify af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.942868.0:15905 2014-08-11 17:06:15.734822 2014-08-13 18:06:06.914915 7f5c973a57a0 20 read_log 192888'15455 (0'0) promote 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by osd.368.0:34779 2014-08-11 17:06:19.319683 2014-08-13 18:06:06.914927 7f5c973a57a0 20 read_log 192888'15456 (192888'15455) modify 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.942868.0:16816 2014-08-11 17:06:19.312728 2014-08-13 18:06:06.914940 7f5c973a57a0 20 read_log 192888'15457 (0'0) promote 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by osd.368.0:34785 2014-08-11 17:06:20.006342 2014-08-13 18:06:06.914952 7f5c973a57a0 20 read_log 192888'15458 (192888'15457) modify 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.942868.0:16985 2014-08-11 17:06:19.997695 2014-08-13 18:06:06.914970 7f5c973a57a0 20 read_log 192888'15461 (0'0) promote 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by osd.368.0:34803 2014-08-11 17:06:23.063876 2014-08-13 18:06:06.914984 7f5c973a57a0 20 read_log 192888'15462 (192888'15461) modify 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.942868.0:17795 2014-08-11 17:06:23.056628 2014-08-13 18:06:06.914998 7f5c973a57a0 20 read_log 192888'15467 (0'0) promote 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by osd.368.0:34863 2014-08-11 17:06:34.676219 2014-08-13 18:06:06.915012 7f5c973a57a0 20 read_log 192888'15468 (192888'15467) modify 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.942868.0:20862 2014-08-11 17:06:34.668394 2014-08-13 18:06:06.915025 7f5c973a57a0 20 read_log 192888'15473 (0'0) promote 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by osd.368.0:34893 2014-08-11 17:06:41.201593 2014-08-13 18:06:06.915039 7f5c973a57a0 20 read_log 192888'15474 (192888'15473) modify 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.942868.0:22552 2014-08-11 17:06:41.194045 2014-08-13 18:06:06.915053 7f5c973a57a0 20 read_log 192888'15475 (0'0) promote 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by osd.368.0:34947 2014-08-11 17:06:49.148182 2014-08-13 18:06:06.915067 7f5c973a57a0 20 read_log 192888'15476 (192888'15475) modify 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.942868.0:24658 2014-08-11 17:06:49.139180 2014-08-13 18:06:06.915081 7f5c973a57a0 20 read_log 192888'15477 (0'0) promote a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by osd.368.0:34961 2014-08-11 17:06:50.337174 2014-08-13 18:06:06.915103 7f5c973a57a0 20 read_log 192888'15478 (192888'15477) modify a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.942868.0:24980 2014-08-11 17:06:50.329903 2014-08-13 18:06:06.915118 7f5c973a57a0 20 read_log 192888'15481 (0'0) promote f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by osd.368.0:34983 2014-08-11 17:06:52.578208 2014-08-13 18:06:06.915131 7f5c973a57a0 20 read_log 192888'15482 (192888'15481) modify f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.942868.0:25573 2014-08-11 17:06:52.570959 2014-08-13 18:06:06.915150 7f5c973a57a0 20 read_log 192888'15485 (0'0) promote f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by osd.368.0:35067 2014-08-11 17:07:04.428393 2014-08-13 18:06:06.915166 7f5c973a57a0 20 read_log 192888'15486 (192888'15485) modify f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.942868.0:28662 2014-08-11 17:07:04.421003 2014-08-13 18:06:06.915178 7f5c973a57a0 20 read_log 192888'15487 (0'0) promote c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by osd.368.0:35073 2014-08-11 17:07:05.501492 2014-08-13 18:06:06.915191 7f5c973a57a0 20 read_log 192888'15488 (192888'15487) modify c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.942868.0:28955 2014-08-11 17:07:05.494665 2014-08-13 18:06:06.915204 7f5c973a57a0 20 read_log 192888'15489 (0'0) promote 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by osd.368.0:35077 2014-08-11 17:07:06.166724 2014-08-13 18:06:06.915217 7f5c973a57a0 20 read_log 192888'15490 (192888'15489) modify 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.942868.0:29117 2014-08-11 17:07:06.159242 2014-08-13 18:06:06.915229 7f5c973a57a0 20 read_log 192888'15491 (0'0) promote 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by osd.368.0:35081 2014-08-11 17:07:07.140676 2014-08-13 18:06:06.915246 7f5c973a57a0 20 read_log 192888'15492 (192888'15491) modify 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.942868.0:29345 2014-08-11 17:07:07.132927 2014-08-13 18:06:06.915262 7f5c973a57a0 20 read_log 192888'15499 (192888'15396) modify 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.942868.0:32818 2014-08-11 17:07:16.353307 2014-08-13 18:06:06.915275 7f5c973a57a0 20 read_log 192888'15500 (192888'15398) modify f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.942868.0:32983 2014-08-11 17:07:16.395800 2014-08-13 18:06:06.915287 7f5c973a57a0 20 read_log 192888'15501 (192888'15400) modify 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.942868.0:33878 2014-08-11 17:07:16.636734 2014-08-13 18:06:06.915299 7f5c973a57a0 20 read_log 192888'15504 (192888'15406) modify 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.942868.0:35237 2014-08-11 17:07:17.023609 2014-08-13 18:06:06.915312 7f5c973a57a0 20 read_log 192888'15505 (192888'15408) modify 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.942868.0:35440 2014-08-11 17:07:17.075124 2014-08-13 18:06:06.915346 7f5c973a57a0 20 read_log 192888'15506 (192888'15410) modify 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.942868.0:36561 2014-08-11 17:07:17.375086 2014-08-13 18:06:06.915362 7f5c973a57a0 20 read_log 192888'15508 (192888'15414) modify 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.942868.0:36915 2014-08-11 17:07:17.467253 2014-08-13 18:06:06.915378 7f5c973a57a0 20 read_log 192888'15515 (192888'15428) modify 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.942868.0:41053 2014-08-11 17:07:18.622289 2014-08-13 18:06:06.915391 7f5c973a57a0 20 read_log 192888'15518 (192888'15434) modify 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.942868.0:43309 2014-08-11 17:07:19.236761 2014-08-13 18:06:06.915411 7f5c973a57a0 20 read_log 192888'15523 (192888'15444) modify d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.942868.0:44676 2014-08-11 17:07:19.633664 2014-08-13 18:06:06.915426 7f5c973a57a0 20 read_log 192888'15524 (192888'15446) modify b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.942868.0:46496 2014-08-11 17:07:20.139540 2014-08-13 18:06:06.915452 7f5c973a57a0 20 read_log 192888'15527 (192888'15452) modify af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.942868.0:47171 2014-08-11 17:07:20.329171 2014-08-13 18:06:06.915465 7f5c973a57a0 20 read_log 192888'15529 (192888'15456) modify 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.942868.0:48082 2014-08-11 17:07:20.616123 2014-08-13 18:06:06.915478 7f5c973a57a0 20 read_log 192888'15530 (192888'15458) modify 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.942868.0:48251 2014-08-11 17:07:20.667435 2014-08-13 18:06:06.915503 7f5c973a57a0 20 read_log 192888'15532 (192888'15462) modify 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.942868.0:49061 2014-08-11 17:07:20.897075 2014-08-13 18:06:06.915516 7f5c973a57a0 20 read_log 192888'15535 (192888'15468) modify 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.942868.0:52128 2014-08-11 17:07:21.848096 2014-08-13 18:06:06.915528 7f5c973a57a0 20 read_log 192888'15538 (192888'15474) modify 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.942868.0:53818 2014-08-11 17:07:22.383746 2014-08-13 18:06:06.915542 7f5c973a57a0 20 read_log 192888'15539 (192888'15476) modify 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.942868.0:55924 2014-08-11 17:07:22.997564 2014-08-13 18:06:06.915555 7f5c973a57a0 20 read_log 192888'15540 (192888'15478) modify a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.942868.0:56246 2014-08-11 17:07:23.092675 2014-08-13 18:06:06.915568 7f5c973a57a0 20 read_log 192888'15542 (192888'15482) modify f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.942868.0:56839 2014-08-11 17:07:23.274220 2014-08-13 18:06:06.915580 7f5c973a57a0 20 read_log 192888'15544 (192888'15486) modify f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.942868.0:59928 2014-08-11 17:07:24.189898 2014-08-13 18:06:06.915593 7f5c973a57a0 20 read_log 192888'15545 (192888'15488) modify c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.942868.0:60221 2014-08-11 17:07:24.275013 2014-08-13 18:06:06.915605 7f5c973a57a0 20 read_log 192888'15546 (192888'15490) modify 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.942868.0:60383 2014-08-11 17:07:24.324648 2014-08-13 18:06:06.915618 7f5c973a57a0 20 read_log 192888'15547 (192888'15492) modify 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.942868.0:60611 2014-08-11 17:07:24.390088 2014-08-13 18:06:06.915638 7f5c973a57a0 20 read_log 193305'15549 (0'0) promote 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by osd.304.0:1 2014-08-11 18:42:33.705921 2014-08-13 18:06:06.915653 7f5c973a57a0 20 read_log 193305'15550 (193305'15549) modify 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.951337.0:194 2014-08-11 18:42:33.695470 2014-08-13 18:06:06.915665 7f5c973a57a0 20 read_log 193305'15551 (0'0) promote 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by osd.304.0:40 2014-08-11 18:42:50.521334 2014-08-13 18:06:06.915678 7f5c973a57a0 20 read_log 193305'15552 (193305'15551) modify 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.951337.0:4673 2014-08-11 18:42:50.513623 2014-08-13 18:06:06.915690 7f5c973a57a0 20 read_log 193305'15553 (0'0) promote 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by osd.304.0:44 2014-08-11 18:42:51.635201 2014-08-13 18:06:06.915710 7f5c973a57a0 20 read_log 193305'15554 (193305'15553) modify 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.951337.0:4974 2014-08-11 18:42:51.627466 2014-08-13 18:06:06.915726 7f5c973a57a0 20 read_log 193305'15555 (0'0) promote b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by osd.304.0:56 2014-08-11 18:42:58.036339 2014-08-13 18:06:06.915738 7f5c973a57a0 20 read_log 193305'15556 (193305'15555) modify b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.951337.0:6720 2014-08-11 18:42:58.025265 2014-08-13 18:06:06.915751 7f5c973a57a0 20 read_log 193305'15557 (0'0) promote fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by osd.304.0:58 2014-08-11 18:42:58.300554 2014-08-13 18:06:06.915763 7f5c973a57a0 20 read_log 193305'15558 (193305'15557) modify fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.951337.0:6793 2014-08-11 18:42:58.290875 2014-08-13 18:06:06.915775 7f5c973a57a0 20 read_log 193305'15559 (0'0) promote 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by osd.304.0:62 2014-08-11 18:43:01.945118 2014-08-13 18:06:06.915795 7f5c973a57a0 20 read_log 193305'15560 (193305'15559) modify 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.951337.0:7801 2014-08-11 18:43:01.935038 2014-08-13 18:06:06.915810 7f5c973a57a0 20 read_log 193305'15561 (0'0) promote 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by osd.304.0:64 2014-08-11 18:43:02.839760 2014-08-13 18:06:06.915822 7f5c973a57a0 20 read_log 193305'15562 (193305'15561) modify 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.951337.0:8046 2014-08-11 18:43:02.832315 2014-08-13 18:06:06.915837 7f5c973a57a0 20 read_log 193305'15563 (0'0) promote 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by osd.304.0:76 2014-08-11 18:43:07.406977 2014-08-13 18:06:06.915849 7f5c973a57a0 20 read_log 193305'15564 (193305'15563) modify 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.951337.0:9214 2014-08-11 18:43:07.399991 2014-08-13 18:06:06.915861 7f5c973a57a0 20 read_log 193305'15565 (0'0) promote 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by osd.304.0:78 2014-08-11 18:43:08.195284 2014-08-13 18:06:06.915876 7f5c973a57a0 20 read_log 193305'15566 (193305'15565) modify 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.951337.0:9427 2014-08-11 18:43:08.188814 2014-08-13 18:06:06.915891 7f5c973a57a0 20 read_log 193305'15567 (0'0) promote cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by osd.304.0:92 2014-08-11 18:43:10.634154 2014-08-13 18:06:06.915903 7f5c973a57a0 20 read_log 193305'15568 (193305'15567) modify cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.951337.0:10094 2014-08-11 18:43:10.626155 2014-08-13 18:06:06.915915 7f5c973a57a0 20 read_log 193305'15569 (0'0) promote deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by osd.304.0:94 2014-08-11 18:43:11.506550 2014-08-13 18:06:06.915928 7f5c973a57a0 20 read_log 193305'15570 (193305'15569) modify deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.951337.0:10332 2014-08-11 18:43:11.498277 2014-08-13 18:06:06.915940 7f5c973a57a0 20 read_log 193305'15571 (0'0) promote 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by osd.304.0:98 2014-08-11 18:43:12.320363 2014-08-13 18:06:06.915952 7f5c973a57a0 20 read_log 193305'15572 (193305'15571) modify 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.951337.0:10546 2014-08-11 18:43:12.311864 2014-08-13 18:06:06.915965 7f5c973a57a0 20 read_log 193305'15573 (0'0) promote e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by osd.304.0:113 2014-08-11 18:43:19.226572 2014-08-13 18:06:06.915986 7f5c973a57a0 20 read_log 193305'15574 (193305'15573) modify e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.951337.0:12366 2014-08-11 18:43:19.219127 2014-08-13 18:06:06.916002 7f5c973a57a0 20 read_log 193305'15575 (0'0) promote 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by osd.304.0:127 2014-08-11 18:43:21.897577 2014-08-13 18:06:06.916021 7f5c973a57a0 20 read_log 193305'15576 (193305'15575) modify 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.951337.0:13073 2014-08-11 18:43:21.890722 2014-08-13 18:06:06.916038 7f5c973a57a0 20 read_log 193305'15577 (0'0) promote 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by osd.304.0:133 2014-08-11 18:43:24.328436 2014-08-13 18:06:06.916057 7f5c973a57a0 20 read_log 193305'15578 (193305'15577) modify 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.951337.0:13702 2014-08-11 18:43:24.320530 2014-08-13 18:06:06.916079 7f5c973a57a0 20 read_log 193305'15579 (0'0) promote 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by osd.304.0:153 2014-08-11 18:43:32.767967 2014-08-13 18:06:06.916093 7f5c973a57a0 20 read_log 193305'15580 (193305'15579) modify 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.951337.0:16104 2014-08-11 18:43:32.758534 2014-08-13 18:06:06.916105 7f5c973a57a0 20 read_log 193305'15581 (0'0) promote 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by osd.304.0:157 2014-08-11 18:43:35.798347 2014-08-13 18:06:06.916129 7f5c973a57a0 20 read_log 193305'15582 (193305'15581) modify 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.951337.0:16952 2014-08-11 18:43:35.789792 2014-08-13 18:06:06.916150 7f5c973a57a0 20 read_log 193305'15583 (0'0) promote 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by osd.304.0:165 2014-08-11 18:43:36.775821 2014-08-13 18:06:06.916165 7f5c973a57a0 20 read_log 193305'15584 (193305'15583) modify 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.951337.0:17238 2014-08-11 18:43:36.765850 2014-08-13 18:06:06.916182 7f5c973a57a0 20 read_log 193305'15585 (0'0) promote c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by osd.304.0:171 2014-08-11 18:43:37.733100 2014-08-13 18:06:06.916194 7f5c973a57a0 20 read_log 193305'15586 (193305'15585) modify c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.951337.0:17530 2014-08-11 18:43:37.725478 2014-08-13 18:06:06.916219 7f5c973a57a0 20 read_log 193305'15587 (0'0) promote 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by osd.304.0:175 2014-08-11 18:43:38.573705 2014-08-13 18:06:06.916231 7f5c973a57a0 20 read_log 193305'15588 (193305'15587) modify 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.951337.0:17789 2014-08-11 18:43:38.565859 2014-08-13 18:06:06.916244 7f5c973a57a0 20 read_log 193305'15589 (0'0) promote 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by osd.304.0:191 2014-08-11 18:43:43.019978 2014-08-13 18:06:06.916271 7f5c973a57a0 20 read_log 193305'15590 (193305'15589) modify 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.951337.0:19019 2014-08-11 18:43:43.011850 2014-08-13 18:06:06.916287 7f5c973a57a0 20 read_log 193305'15591 (0'0) promote 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by osd.304.0:195 2014-08-11 18:43:44.023448 2014-08-13 18:06:06.916302 7f5c973a57a0 20 read_log 193305'15592 (193305'15591) modify 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.951337.0:19271 2014-08-11 18:43:44.014951 2014-08-13 18:06:06.916317 7f5c973a57a0 20 read_log 193305'15593 (0'0) promote 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by osd.304.0:205 2014-08-11 18:43:45.837641 2014-08-13 18:06:06.916338 7f5c973a57a0 20 read_log 193305'15594 (193305'15593) modify 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.951337.0:19780 2014-08-11 18:43:45.830070 2014-08-13 18:06:06.916357 7f5c973a57a0 20 read_log 193305'15595 (0'0) promote 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by osd.304.0:213 2014-08-11 18:43:50.719769 2014-08-13 18:06:06.916379 7f5c973a57a0 20 read_log 193305'15596 (193305'15595) modify 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.951337.0:21138 2014-08-11 18:43:50.711486 2014-08-13 18:06:06.916398 7f5c973a57a0 20 read_log 193305'15597 (0'0) promote c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by osd.304.0:227 2014-08-11 18:43:55.975253 2014-08-13 18:06:06.916412 7f5c973a57a0 20 read_log 193305'15598 (193305'15597) modify c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.951337.0:22588 2014-08-11 18:43:55.961892 2014-08-13 18:06:06.916424 7f5c973a57a0 20 read_log 193305'15599 (0'0) promote f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by osd.304.0:275 2014-08-11 18:44:10.048400 2014-08-13 18:06:06.916436 7f5c973a57a0 20 read_log 193305'15600 (193305'15599) modify f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.951337.0:26460 2014-08-11 18:44:10.041187 2014-08-13 18:06:06.916452 7f5c973a57a0 20 read_log 193305'15601 (0'0) promote 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by osd.304.0:283 2014-08-11 18:44:14.699398 2014-08-13 18:06:06.916465 7f5c973a57a0 20 read_log 193305'15602 (193305'15601) modify 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.951337.0:27732 2014-08-11 18:44:14.693192 2014-08-13 18:06:06.916477 7f5c973a57a0 20 read_log 193305'15603 (0'0) promote 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by osd.304.0:287 2014-08-11 18:44:19.494239 2014-08-13 18:06:06.916489 7f5c973a57a0 20 read_log 193305'15604 (193305'15603) modify 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.951337.0:29042 2014-08-11 18:44:19.487048 2014-08-13 18:06:06.916507 7f5c973a57a0 20 read_log 193305'15605 (0'0) promote 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by osd.304.0:293 2014-08-11 18:44:21.173698 2014-08-13 18:06:06.916520 7f5c973a57a0 20 read_log 193305'15606 (193305'15605) modify 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.951337.0:29502 2014-08-11 18:44:21.166549 2014-08-13 18:06:06.916544 7f5c973a57a0 20 read_log 193305'15607 (0'0) promote 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by osd.304.0:303 2014-08-11 18:44:24.162345 2014-08-13 18:06:06.916560 7f5c973a57a0 20 read_log 193305'15608 (193305'15607) modify 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.951337.0:30289 2014-08-11 18:44:24.155129 2014-08-13 18:06:06.916572 7f5c973a57a0 20 read_log 193305'15609 (0'0) promote dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by osd.304.0:309 2014-08-11 18:44:26.439910 2014-08-13 18:06:06.916585 7f5c973a57a0 20 read_log 193305'15610 (193305'15609) modify dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.951337.0:30889 2014-08-11 18:44:26.431959 2014-08-13 18:06:06.916600 7f5c973a57a0 20 read_log 193305'15611 (193305'15550) modify 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.951337.0:32905 2014-08-11 18:44:33.949112 2014-08-13 18:06:06.916613 7f5c973a57a0 20 read_log 193305'15612 (193305'15552) modify 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.951337.0:37384 2014-08-11 18:44:36.878940 2014-08-13 18:06:06.916625 7f5c973a57a0 20 read_log 193305'15613 (193305'15554) modify 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.951337.0:37685 2014-08-11 18:44:37.023487 2014-08-13 18:06:06.916672 7f5c973a57a0 20 read_log 193305'15614 (193305'15556) modify b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.951337.0:39431 2014-08-11 18:44:38.003227 2014-08-13 18:06:06.916689 7f5c973a57a0 20 read_log 193305'15615 (193305'15558) modify fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.951337.0:39504 2014-08-11 18:44:38.047885 2014-08-13 18:06:06.916702 7f5c973a57a0 20 read_log 193305'15616 (193305'15560) modify 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.951337.0:40512 2014-08-11 18:44:38.435040 2014-08-13 18:06:06.916714 7f5c973a57a0 20 read_log 193305'15617 (193305'15562) modify 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.951337.0:40757 2014-08-11 18:44:38.517026 2014-08-13 18:06:06.916726 7f5c973a57a0 20 read_log 193305'15618 (193305'15564) modify 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.951337.0:41925 2014-08-11 18:44:38.927749 2014-08-13 18:06:06.916739 7f5c973a57a0 20 read_log 193305'15619 (193305'15566) modify 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.951337.0:42138 2014-08-11 18:44:39.121265 2014-08-13 18:06:06.916751 7f5c973a57a0 20 read_log 193305'15620 (193305'15568) modify cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.951337.0:42805 2014-08-11 18:44:39.387165 2014-08-13 18:06:06.916763 7f5c973a57a0 20 read_log 193305'15621 (193305'15570) modify deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.951337.0:43043 2014-08-11 18:44:39.465777 2014-08-13 18:06:06.916775 7f5c973a57a0 20 read_log 193305'15622 (193305'15572) modify 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.951337.0:43257 2014-08-11 18:44:39.534353 2014-08-13 18:06:06.916807 7f5c973a57a0 20 read_log 193305'15623 (193305'15623) modify 33a/hit_set_24.33a_archive_2014-08-11 18:39:39.883446_2014-08-11 18:44:40.084433/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:40.084663 2014-08-13 18:06:06.916823 7f5c973a57a0 20 read_log 193305'15624 (193305'15574) modify e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.951337.0:45077 2014-08-11 18:44:40.083366 2014-08-13 18:06:06.916835 7f5c973a57a0 20 read_log 193305'15625 (193305'15576) modify 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.951337.0:45784 2014-08-11 18:44:40.303394 2014-08-13 18:06:06.916848 7f5c973a57a0 20 read_log 193305'15626 (193305'15578) modify 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.951337.0:46413 2014-08-11 18:44:40.475890 2014-08-13 18:06:06.916860 7f5c973a57a0 20 read_log 193305'15627 (193305'15580) modify 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.951337.0:48815 2014-08-11 18:44:41.163819 2014-08-13 18:06:06.916872 7f5c973a57a0 20 read_log 193305'15628 (193305'15582) modify 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.951337.0:49663 2014-08-11 18:44:41.434142 2014-08-13 18:06:06.916884 7f5c973a57a0 20 read_log 193305'15629 (193305'15584) modify 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.951337.0:49949 2014-08-11 18:44:41.524260 2014-08-13 18:06:06.916896 7f5c973a57a0 20 read_log 193305'15630 (193305'15586) modify c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.951337.0:50241 2014-08-11 18:44:41.609318 2014-08-13 18:06:06.916909 7f5c973a57a0 20 read_log 193305'15631 (193305'15588) modify 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.951337.0:50500 2014-08-11 18:44:41.682508 2014-08-13 18:06:06.916921 7f5c973a57a0 20 read_log 193305'15632 (193305'15590) modify 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.951337.0:51730 2014-08-11 18:44:42.034688 2014-08-13 18:06:06.916969 7f5c973a57a0 20 read_log 193305'15633 (193305'15592) modify 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.951337.0:51982 2014-08-11 18:44:42.102559 2014-08-13 18:06:06.916985 7f5c973a57a0 20 read_log 193305'15634 (193305'15594) modify 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.951337.0:52491 2014-08-11 18:44:42.248927 2014-08-13 18:06:06.916999 7f5c973a57a0 20 read_log 193305'15635 (193305'15596) modify 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.951337.0:53849 2014-08-11 18:44:42.656593 2014-08-13 18:06:06.917012 7f5c973a57a0 20 read_log 193305'15636 (193305'15598) modify c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.951337.0:55299 2014-08-11 18:44:43.082503 2014-08-13 18:06:06.917024 7f5c973a57a0 20 read_log 193305'15637 (193305'15600) modify f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.951337.0:59171 2014-08-11 18:44:44.252832 2014-08-13 18:06:06.917036 7f5c973a57a0 20 read_log 193305'15638 (193305'15602) modify 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.951337.0:60443 2014-08-11 18:44:44.630331 2014-08-13 18:06:06.917049 7f5c973a57a0 20 read_log 193305'15639 (193305'15604) modify 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.951337.0:61753 2014-08-11 18:44:45.019353 2014-08-13 18:06:06.917061 7f5c973a57a0 20 read_log 193305'15640 (193305'15606) modify 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.951337.0:62213 2014-08-11 18:44:45.158826 2014-08-13 18:06:06.917073 7f5c973a57a0 20 read_log 193305'15641 (193305'15608) modify 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.951337.0:63000 2014-08-11 18:44:45.383347 2014-08-13 18:06:06.917085 7f5c973a57a0 20 read_log 193305'15642 (193305'15610) modify dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.951337.0:63600 2014-08-11 18:44:45.555656 2014-08-13 18:06:06.917098 7f5c973a57a0 20 read_log 194215'15643 (194215'15643) modify 33a/hit_set_24.33a_archive_2014-08-12 15:32:25.735124_2014-08-12 15:48:57.821063/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:57.821307 2014-08-13 18:06:06.917113 7f5c973a57a0 20 read_log 194215'15644 (193305'15623) delete 33a/hit_set_24.33a_archive_2014-08-11 18:39:39.883446_2014-08-11 18:44:40.084433/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:48:57.821307 2014-08-13 18:06:06.917125 7f5c973a57a0 20 read_log 194215'15645 (192834'14574) clean 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by osd.304.0:1242 2014-08-12 15:48:57.828539 2014-08-13 18:06:06.917140 7f5c973a57a0 20 read_log 194215'15646 (194215'15645) delete 8153033a/benchmark_data_storage0111-ib.pocstor_26659_object8273/head//24 by client.2174595.0:170253 0.000000 2014-08-13 18:06:06.917157 7f5c973a57a0 20 read_log 194215'15647 (193305'15624) clean e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by osd.304.0:1244 2014-08-12 15:48:57.841510 2014-08-13 18:06:06.917172 7f5c973a57a0 20 read_log 194215'15648 (194215'15647) delete e0f9033a/benchmark_data_storage0111-ib.pocstor_62621_object12365/head//24 by client.2174595.0:170255 0.000000 2014-08-13 18:06:06.917183 7f5c973a57a0 20 read_log 194215'15649 (193305'15639) clean 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by osd.304.0:1246 2014-08-12 15:48:57.849884 2014-08-13 18:06:06.917196 7f5c973a57a0 20 read_log 194215'15650 (194215'15649) delete 607c133a/benchmark_data_storage0111-ib.pocstor_62621_object29041/head//24 by client.2174595.0:170257 0.000000 2014-08-13 18:06:06.917207 7f5c973a57a0 20 read_log 194215'15651 (192888'15518) clean 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by osd.304.0:1248 2014-08-12 15:48:57.856886 2014-08-13 18:06:06.917227 7f5c973a57a0 20 read_log 194215'15652 (194215'15651) delete 62f4233a/benchmark_data_storage0111-ib.pocstor_2228_object12042/head//24 by client.2174595.0:170259 0.000000 2014-08-13 18:06:06.917243 7f5c973a57a0 20 read_log 194215'15653 (192834'15038) clean 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by osd.304.0:1250 2014-08-12 15:48:57.863772 2014-08-13 18:06:06.917256 7f5c973a57a0 20 read_log 194215'15654 (194215'15653) delete 8188233a/benchmark_data_storage0111-ib.pocstor_41526_object704/head//24 by client.2174595.0:170261 0.000000 2014-08-13 18:06:06.917267 7f5c973a57a0 20 read_log 194215'15655 (193305'15612) clean 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by osd.304.0:1252 2014-08-12 15:48:57.870645 2014-08-13 18:06:06.917279 7f5c973a57a0 20 read_log 194215'15656 (194215'15655) delete 14ab233a/benchmark_data_storage0111-ib.pocstor_62621_object4672/head//24 by client.2174595.0:170263 0.000000 2014-08-13 18:06:06.917296 7f5c973a57a0 20 read_log 194215'15657 (193305'15634) clean 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by osd.304.0:1254 2014-08-12 15:48:57.877426 2014-08-13 18:06:06.917309 7f5c973a57a0 20 read_log 194215'15658 (194215'15657) delete 3de0333a/benchmark_data_storage0111-ib.pocstor_62621_object19779/head//24 by client.2174595.0:170265 0.000000 2014-08-13 18:06:06.917320 7f5c973a57a0 20 read_log 194215'15659 (192888'15523) clean d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by osd.304.0:1256 2014-08-12 15:48:57.884407 2014-08-13 18:06:06.917337 7f5c973a57a0 20 read_log 194215'15660 (194215'15659) delete d5f7333a/benchmark_data_storage0111-ib.pocstor_2228_object13409/head//24 by client.2174595.0:170267 0.000000 2014-08-13 18:06:06.917348 7f5c973a57a0 20 read_log 194215'15661 (192834'14518) clean e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by osd.304.0:1258 2014-08-12 15:48:57.907387 2014-08-13 18:06:06.917360 7f5c973a57a0 20 read_log 194215'15662 (194215'15661) delete e1de333a/benchmark_data_storage0111-ib.pocstor_26659_object1091/head//24 by client.2174595.0:170269 0.000000 2014-08-13 18:06:06.917371 7f5c973a57a0 20 read_log 194215'15663 (192834'14664) clean b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by osd.304.0:1260 2014-08-12 15:48:57.915419 2014-08-13 18:06:06.917387 7f5c973a57a0 20 read_log 194215'15664 (194215'15663) delete b274433a/benchmark_data_storage0111-ib.pocstor_26659_object18733/head//24 by client.2174595.0:170271 0.000000 2014-08-13 18:06:06.917398 7f5c973a57a0 20 read_log 194215'15665 (192888'15379) clean 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by osd.304.0:1262 2014-08-12 15:48:57.922581 2014-08-13 18:06:06.917410 7f5c973a57a0 20 read_log 194215'15666 (194215'15665) delete 6f6a433a/benchmark_data_storage0111-ib.pocstor_41097_object27719/head//24 by client.2174595.0:170273 0.000000 2014-08-13 18:06:06.917430 7f5c973a57a0 20 read_log 194215'15667 (193305'15621) clean deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by osd.304.0:1264 2014-08-12 15:48:57.929923 2014-08-13 18:06:06.917446 7f5c973a57a0 20 read_log 194215'15668 (194215'15667) delete deba433a/benchmark_data_storage0111-ib.pocstor_62621_object10331/head//24 by client.2174595.0:170275 0.000000 2014-08-13 18:06:06.917457 7f5c973a57a0 20 read_log 194215'15669 (192834'15190) clean 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by osd.304.0:1266 2014-08-12 15:48:57.937398 2014-08-13 18:06:06.917479 7f5c973a57a0 20 read_log 194215'15670 (194215'15669) delete 75df433a/benchmark_data_storage0111-ib.pocstor_41526_object22106/head//24 by client.2174595.0:170277 0.000000 2014-08-13 18:06:06.917494 7f5c973a57a0 20 read_log 194215'15671 (193305'15614) clean b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by osd.304.0:1268 2014-08-12 15:48:57.943909 2014-08-13 18:06:06.917521 7f5c973a57a0 20 read_log 194215'15672 (194215'15671) delete b850533a/benchmark_data_storage0111-ib.pocstor_62621_object6719/head//24 by client.2174595.0:170279 0.000000 2014-08-13 18:06:06.917536 7f5c973a57a0 20 read_log 194215'15673 (192888'15346) clean de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by osd.304.0:1270 2014-08-12 15:48:57.959140 2014-08-13 18:06:06.917549 7f5c973a57a0 20 read_log 194215'15674 (194215'15673) delete de52533a/benchmark_data_storage0111-ib.pocstor_41097_object10238/head//24 by client.2174595.0:170281 0.000000 2014-08-13 18:06:06.917560 7f5c973a57a0 20 read_log 194215'15675 (192888'15535) clean 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by osd.304.0:1272 2014-08-12 15:48:57.965839 2014-08-13 18:06:06.917588 7f5c973a57a0 20 read_log 194215'15676 (194215'15675) delete 8ceb533a/benchmark_data_storage0111-ib.pocstor_2228_object20861/head//24 by client.2174595.0:170283 0.000000 2014-08-13 18:06:06.917599 7f5c973a57a0 20 read_log 194215'15677 (193305'15615) clean fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by osd.304.0:1274 2014-08-12 15:48:57.972358 2014-08-13 18:06:06.917612 7f5c973a57a0 20 read_log 194215'15678 (194215'15677) delete fe9f633a/benchmark_data_storage0111-ib.pocstor_62621_object6792/head//24 by client.2174595.0:170285 0.000000 2014-08-13 18:06:06.917623 7f5c973a57a0 20 read_log 194215'15679 (192888'15348) clean 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by osd.304.0:1276 2014-08-12 15:48:57.979538 2014-08-13 18:06:06.917637 7f5c973a57a0 20 read_log 194215'15680 (194215'15679) delete 9950733a/benchmark_data_storage0111-ib.pocstor_41097_object10425/head//24 by client.2174595.0:170287 0.000000 2014-08-13 18:06:06.917649 7f5c973a57a0 20 read_log 194215'15681 (192834'14625) clean 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by osd.304.0:1278 2014-08-12 15:48:57.986032 2014-08-13 18:06:06.917661 7f5c973a57a0 20 read_log 194215'15682 (194215'15681) delete 9382733a/benchmark_data_storage0111-ib.pocstor_26659_object14591/head//24 by client.2174595.0:170289 0.000000 2014-08-13 18:06:06.917672 7f5c973a57a0 20 read_log 194215'15683 (192888'15539) clean 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by osd.304.0:1280 2014-08-12 15:48:57.991517 2014-08-13 18:06:06.917684 7f5c973a57a0 20 read_log 194215'15684 (194215'15683) delete 4139733a/benchmark_data_storage0111-ib.pocstor_2228_object24657/head//24 by client.2174595.0:170291 0.000000 2014-08-13 18:06:06.917695 7f5c973a57a0 20 read_log 194215'15685 (192834'15122) clean fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by osd.304.0:1282 2014-08-12 15:48:57.997465 2014-08-13 18:06:06.917737 7f5c973a57a0 20 read_log 194215'15686 (194215'15685) delete fe6c733a/benchmark_data_storage0111-ib.pocstor_41526_object11761/head//24 by client.2174595.0:170293 0.000000 2014-08-13 18:06:06.917749 7f5c973a57a0 20 read_log 194215'15687 (192888'15360) clean 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by osd.304.0:1284 2014-08-12 15:48:58.017425 2014-08-13 18:06:06.917762 7f5c973a57a0 20 read_log 194215'15688 (194215'15687) delete 5a10833a/benchmark_data_storage0111-ib.pocstor_41097_object16573/head//24 by client.2174595.0:170295 0.000000 2014-08-13 18:06:06.917773 7f5c973a57a0 20 read_log 194215'15689 (193305'15617) clean 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by osd.304.0:1286 2014-08-12 15:48:58.023333 2014-08-13 18:06:06.917785 7f5c973a57a0 20 read_log 194215'15690 (194215'15689) delete 9165933a/benchmark_data_storage0111-ib.pocstor_62621_object8045/head//24 by client.2174595.0:170297 0.000000 2014-08-13 18:06:06.917796 7f5c973a57a0 20 read_log 194215'15691 (192834'14589) clean 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by osd.304.0:1288 2014-08-12 15:48:58.028802 2014-08-13 18:06:06.917819 7f5c973a57a0 20 read_log 194215'15692 (194215'15691) delete 617b933a/benchmark_data_storage0111-ib.pocstor_26659_object10110/head//24 by client.2174595.0:170299 0.000000 2014-08-13 18:06:06.917833 7f5c973a57a0 20 read_log 194215'15693 (192888'15499) clean 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by osd.304.0:1290 2014-08-12 15:48:58.035068 2014-08-13 18:06:06.917846 7f5c973a57a0 20 read_log 194215'15694 (194215'15693) delete 35d933a/benchmark_data_storage0111-ib.pocstor_2228_object1551/head//24 by client.2174595.0:170301 0.000000 2014-08-13 18:06:06.917857 7f5c973a57a0 20 read_log 194215'15695 (192888'15372) clean 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by osd.304.0:1292 2014-08-12 15:48:58.040580 2014-08-13 18:06:06.917869 7f5c973a57a0 20 read_log 194215'15696 (194215'15695) delete 144a33a/benchmark_data_storage0111-ib.pocstor_41097_object24812/head//24 by client.2174595.0:170303 0.000000 2014-08-13 18:06:06.917880 7f5c973a57a0 20 read_log 194215'15697 (192834'15144) clean 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by osd.304.0:1294 2014-08-12 15:48:58.045868 2014-08-13 18:06:06.917892 7f5c973a57a0 20 read_log 194215'15698 (194215'15697) delete 18a7a33a/benchmark_data_storage0111-ib.pocstor_41526_object14894/head//24 by client.2174595.0:170305 0.000000 2014-08-13 18:06:06.917903 7f5c973a57a0 20 read_log 194215'15699 (192888'15501) clean 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by osd.304.0:1296 2014-08-12 15:48:58.051167 2014-08-13 18:06:06.917915 7f5c973a57a0 20 read_log 194215'15700 (194215'15699) delete 8451c33a/benchmark_data_storage0111-ib.pocstor_2228_object2611/head//24 by client.2174595.0:170307 0.000000 2014-08-13 18:06:06.917926 7f5c973a57a0 20 read_log 194215'15701 (192888'15506) clean 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by osd.304.0:1298 2014-08-12 15:48:58.056637 2014-08-13 18:06:06.917939 7f5c973a57a0 20 read_log 194215'15702 (194215'15701) delete 40ec33a/benchmark_data_storage0111-ib.pocstor_2228_object5294/head//24 by client.2174595.0:170309 0.000000 2014-08-13 18:06:06.917950 7f5c973a57a0 20 read_log 194215'15703 (192888'15352) clean 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by osd.304.0:1300 2014-08-12 15:48:58.062432 2014-08-13 18:06:06.917962 7f5c973a57a0 20 read_log 194215'15704 (194215'15703) delete 1a89d33a/benchmark_data_storage0111-ib.pocstor_41097_object12067/head//24 by client.2174595.0:170311 0.000000 2014-08-13 18:06:06.917973 7f5c973a57a0 20 read_log 194215'15705 (192888'15508) clean 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by osd.304.0:1302 2014-08-12 15:48:58.068584 2014-08-13 18:06:06.917986 7f5c973a57a0 20 read_log 194215'15706 (194215'15705) delete 1e0cd33a/benchmark_data_storage0111-ib.pocstor_2228_object5648/head//24 by client.2174595.0:170313 0.000000 2014-08-13 18:06:06.917997 7f5c973a57a0 20 read_log 194215'15707 (192834'15062) clean 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by osd.304.0:1304 2014-08-12 15:48:58.074133 2014-08-13 18:06:06.918010 7f5c973a57a0 20 read_log 194215'15708 (194215'15707) delete 9e47e33a/benchmark_data_storage0111-ib.pocstor_41526_object4057/head//24 by client.2174595.0:170315 0.000000 2014-08-13 18:06:06.918021 7f5c973a57a0 20 read_log 194215'15709 (192834'15043) clean b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by osd.304.0:1306 2014-08-12 15:48:58.106862 2014-08-13 18:06:06.918034 7f5c973a57a0 20 read_log 194215'15710 (194215'15709) delete b070f33a/benchmark_data_storage0111-ib.pocstor_41526_object1660/head//24 by client.2174595.0:170317 0.000000 2014-08-13 18:06:06.918045 7f5c973a57a0 20 read_log 194215'15711 (192834'15109) clean c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by osd.304.0:1308 2014-08-12 15:48:58.121314 2014-08-13 18:06:06.918066 7f5c973a57a0 20 read_log 194215'15712 (194215'15711) delete c957f33a/benchmark_data_storage0111-ib.pocstor_41526_object10238/head//24 by client.2174595.0:170319 0.000000 2014-08-13 18:06:06.918080 7f5c973a57a0 20 read_log 194215'15713 (192888'15544) clean f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by osd.304.0:1310 2014-08-12 15:48:58.129344 2014-08-13 18:06:06.918093 7f5c973a57a0 20 read_log 194215'15714 (194215'15713) delete f29ff33a/benchmark_data_storage0111-ib.pocstor_2228_object28661/head//24 by client.2174595.0:170321 0.000000 2014-08-13 18:06:06.918104 7f5c973a57a0 20 read_log 194215'15715 (192888'15339) clean bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by osd.304.0:1312 2014-08-12 15:48:58.146135 2014-08-13 18:06:06.918136 7f5c973a57a0 20 read_log 194215'15716 (194215'15715) delete bf32073a/benchmark_data_storage0111-ib.pocstor_41097_object6743/head//24 by client.2174595.0:170323 0.000000 2014-08-13 18:06:06.918162 7f5c973a57a0 20 read_log 194215'15717 (192834'14556) clean f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by osd.304.0:1314 2014-08-12 15:48:58.153056 2014-08-13 18:06:06.918178 7f5c973a57a0 20 read_log 194215'15718 (194215'15717) delete f4ea073a/benchmark_data_storage0111-ib.pocstor_26659_object5829/head//24 by client.2174595.0:170325 0.000000 2014-08-13 18:06:06.918189 7f5c973a57a0 20 read_log 194215'15719 (192888'15530) clean 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by osd.304.0:1316 2014-08-12 15:48:58.160084 2014-08-13 18:06:06.918202 7f5c973a57a0 20 read_log 194215'15720 (194215'15719) delete 44bb173a/benchmark_data_storage0111-ib.pocstor_2228_object16984/head//24 by client.2174595.0:170327 0.000000 2014-08-13 18:06:06.918212 7f5c973a57a0 20 read_log 194215'15721 (192834'15136) clean ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by osd.304.0:1318 2014-08-12 15:48:58.167256 2014-08-13 18:06:06.918225 7f5c973a57a0 20 read_log 194215'15722 (194215'15721) delete ad82273a/benchmark_data_storage0111-ib.pocstor_41526_object14302/head//24 by client.2174595.0:170329 0.000000 2014-08-13 18:06:06.918238 7f5c973a57a0 20 read_log 194215'15723 (193305'15613) clean 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by osd.304.0:1320 2014-08-12 15:48:58.174195 2014-08-13 18:06:06.918251 7f5c973a57a0 20 read_log 194215'15724 (194215'15723) delete 8a17273a/benchmark_data_storage0111-ib.pocstor_62621_object4973/head//24 by client.2174595.0:170331 0.000000 2014-08-13 18:06:06.918262 7f5c973a57a0 20 read_log 194215'15725 (193305'15627) clean 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by osd.304.0:1322 2014-08-12 15:48:58.180789 2014-08-13 18:06:06.918274 7f5c973a57a0 20 read_log 194215'15726 (194215'15725) delete 1a87273a/benchmark_data_storage0111-ib.pocstor_62621_object16103/head//24 by client.2174595.0:170333 0.000000 2014-08-13 18:06:06.918288 7f5c973a57a0 20 read_log 194215'15727 (193305'15628) clean 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by osd.304.0:1324 2014-08-12 15:48:58.187466 2014-08-13 18:06:06.918304 7f5c973a57a0 20 read_log 194215'15728 (194215'15727) delete 9f02373a/benchmark_data_storage0111-ib.pocstor_62621_object16951/head//24 by client.2174595.0:170335 0.000000 2014-08-13 18:06:06.918315 7f5c973a57a0 20 read_log 194215'15729 (192888'15538) clean 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by osd.304.0:1326 2014-08-12 15:48:58.194919 2014-08-13 18:06:06.918327 7f5c973a57a0 20 read_log 194215'15730 (194215'15729) delete 5c80473a/benchmark_data_storage0111-ib.pocstor_2228_object22551/head//24 by client.2174595.0:170337 0.000000 2014-08-13 18:06:06.918342 7f5c973a57a0 20 read_log 194215'15731 (192834'15191) clean d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by osd.304.0:1328 2014-08-12 15:48:58.201320 2014-08-13 18:06:06.918362 7f5c973a57a0 20 read_log 194215'15732 (194215'15731) delete d5f0473a/benchmark_data_storage0111-ib.pocstor_41526_object22208/head//24 by client.2174595.0:170339 0.000000 2014-08-13 18:06:06.918376 7f5c973a57a0 20 read_log 194215'15733 (192888'15504) clean 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by osd.304.0:1330 2014-08-12 15:48:58.209941 2014-08-13 18:06:06.918393 7f5c973a57a0 20 read_log 194215'15734 (194215'15733) delete 7bd5473a/benchmark_data_storage0111-ib.pocstor_2228_object3970/head//24 by client.2174595.0:170341 0.000000 2014-08-13 18:06:06.918416 7f5c973a57a0 20 read_log 194215'15735 (193305'15620) clean cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by osd.304.0:1332 2014-08-12 15:48:58.217339 2014-08-13 18:06:06.918429 7f5c973a57a0 20 read_log 194215'15736 (194215'15735) delete cd76573a/benchmark_data_storage0111-ib.pocstor_62621_object10093/head//24 by client.2174595.0:170343 0.000000 2014-08-13 18:06:06.918440 7f5c973a57a0 20 read_log 194215'15737 (192834'15100) clean 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by osd.304.0:1334 2014-08-12 15:48:58.224422 2014-08-13 18:06:06.918465 7f5c973a57a0 20 read_log 194215'15738 (194215'15737) delete 4330673a/benchmark_data_storage0111-ib.pocstor_41526_object9251/head//24 by client.2174595.0:170345 0.000000 2014-08-13 18:06:06.918477 7f5c973a57a0 20 read_log 194215'15739 (192888'15359) clean 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by osd.304.0:1336 2014-08-12 15:48:58.230922 2014-08-13 18:06:06.918489 7f5c973a57a0 20 read_log 194215'15740 (194215'15739) delete 7ffc673a/benchmark_data_storage0111-ib.pocstor_41097_object15577/head//24 by client.2174595.0:170347 0.000000 2014-08-13 18:06:06.918508 7f5c973a57a0 20 read_log 194215'15741 (192888'15343) clean 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by osd.304.0:1338 2014-08-12 15:48:58.239914 2014-08-13 18:06:06.918522 7f5c973a57a0 20 read_log 194215'15742 (194215'15741) delete 64c0773a/benchmark_data_storage0111-ib.pocstor_41097_object8327/head//24 by client.2174595.0:170349 0.000000 2014-08-13 18:06:06.918533 7f5c973a57a0 20 read_log 194215'15743 (193305'15630) clean c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by osd.304.0:1340 2014-08-12 15:48:58.246803 2014-08-13 18:06:06.918545 7f5c973a57a0 20 read_log 194215'15744 (194215'15743) delete c1d0773a/benchmark_data_storage0111-ib.pocstor_62621_object17529/head//24 by client.2174595.0:170351 0.000000 2014-08-13 18:06:06.918556 7f5c973a57a0 20 read_log 194215'15745 (192888'15357) clean 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by osd.304.0:1342 2014-08-12 15:48:58.280139 2014-08-13 18:06:06.918573 7f5c973a57a0 20 read_log 194215'15746 (194215'15745) delete 7e96873a/benchmark_data_storage0111-ib.pocstor_41097_object15002/head//24 by client.2174595.0:170353 0.000000 2014-08-13 18:06:06.918584 7f5c973a57a0 20 read_log 194215'15747 (193305'15633) clean 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by osd.304.0:1344 2014-08-12 15:48:58.288215 2014-08-13 18:06:06.918596 7f5c973a57a0 20 read_log 194215'15748 (194215'15747) delete 42ad873a/benchmark_data_storage0111-ib.pocstor_62621_object19270/head//24 by client.2174595.0:170355 0.000000 2014-08-13 18:06:06.918607 7f5c973a57a0 20 read_log 194215'15749 (192834'14519) clean 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by osd.304.0:1346 2014-08-12 15:48:58.295819 2014-08-13 18:06:06.918620 7f5c973a57a0 20 read_log 194215'15750 (194215'15749) delete 8d0f873a/benchmark_data_storage0111-ib.pocstor_26659_object1123/head//24 by client.2174595.0:170357 0.000000 2014-08-13 18:06:06.918631 7f5c973a57a0 20 read_log 194215'15751 (192888'15342) clean a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by osd.304.0:1348 2014-08-12 15:48:58.303355 2014-08-13 18:06:06.918680 7f5c973a57a0 20 read_log 194215'15752 (194215'15751) delete a89f873a/benchmark_data_storage0111-ib.pocstor_41097_object7408/head//24 by client.2174595.0:170359 0.000000 2014-08-13 18:06:06.918696 7f5c973a57a0 20 read_log 194215'15753 (192834'14596) clean f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by osd.304.0:1350 2014-08-12 15:48:58.311909 2014-08-13 18:06:06.918709 7f5c973a57a0 20 read_log 194215'15754 (194215'15753) delete f4fa973a/benchmark_data_storage0111-ib.pocstor_26659_object10819/head//24 by client.2174595.0:170361 0.000000 2014-08-13 18:06:06.918720 7f5c973a57a0 20 read_log 194215'15755 (192834'15179) clean 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by osd.304.0:1352 2014-08-12 15:48:58.320308 2014-08-13 18:06:06.918732 7f5c973a57a0 20 read_log 194215'15756 (194215'15755) delete 65f3a73a/benchmark_data_storage0111-ib.pocstor_41526_object21148/head//24 by client.2174595.0:170363 0.000000 2014-08-13 18:06:06.918743 7f5c973a57a0 20 read_log 194215'15757 (192888'15540) clean a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by osd.304.0:1354 2014-08-12 15:48:58.329425 2014-08-13 18:06:06.918756 7f5c973a57a0 20 read_log 194215'15758 (194215'15757) delete a421c73a/benchmark_data_storage0111-ib.pocstor_2228_object24979/head//24 by client.2174595.0:170365 0.000000 2014-08-13 18:06:06.918766 7f5c973a57a0 20 read_log 194215'15759 (193305'15625) clean 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by osd.304.0:1356 2014-08-12 15:48:58.336883 2014-08-13 18:06:06.918779 7f5c973a57a0 20 read_log 194215'15760 (194215'15759) delete 4131c73a/benchmark_data_storage0111-ib.pocstor_62621_object13072/head//24 by client.2174595.0:170367 0.000000 2014-08-13 18:06:06.918790 7f5c973a57a0 20 read_log 194215'15761 (192888'15527) clean af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by osd.304.0:1358 2014-08-12 15:48:58.344307 2014-08-13 18:06:06.918802 7f5c973a57a0 20 read_log 194215'15762 (194215'15761) delete af74c73a/benchmark_data_storage0111-ib.pocstor_2228_object15904/head//24 by client.2174595.0:170369 0.000000 2014-08-13 18:06:06.918813 7f5c973a57a0 20 read_log 194215'15763 (192888'15532) clean 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by osd.304.0:1360 2014-08-12 15:48:58.351826 2014-08-13 18:06:06.918825 7f5c973a57a0 20 read_log 194215'15764 (194215'15763) delete 7c4ac73a/benchmark_data_storage0111-ib.pocstor_2228_object17794/head//24 by client.2174595.0:170371 0.000000 2014-08-13 18:06:06.918836 7f5c973a57a0 20 read_log 194215'15765 (192834'15072) clean f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by osd.304.0:1362 2014-08-12 15:48:58.359632 2014-08-13 18:06:06.918848 7f5c973a57a0 20 read_log 194215'15766 (194215'15765) delete f11bc73a/benchmark_data_storage0111-ib.pocstor_41526_object5930/head//24 by client.2174595.0:170373 0.000000 2014-08-13 18:06:06.918859 7f5c973a57a0 20 read_log 194215'15767 (193305'15622) clean 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by osd.304.0:1364 2014-08-12 15:48:58.366981 2014-08-13 18:06:06.918871 7f5c973a57a0 20 read_log 194215'15768 (194215'15767) delete 307cc73a/benchmark_data_storage0111-ib.pocstor_62621_object10545/head//24 by client.2174595.0:170375 0.000000 2014-08-13 18:06:06.918882 7f5c973a57a0 20 read_log 194215'15769 (192834'14532) clean 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by osd.304.0:1366 2014-08-12 15:48:58.373560 2014-08-13 18:06:06.918894 7f5c973a57a0 20 read_log 194215'15770 (194215'15769) delete 4bcfc73a/benchmark_data_storage0111-ib.pocstor_26659_object2196/head//24 by client.2174595.0:170377 0.000000 2014-08-13 18:06:06.918905 7f5c973a57a0 20 read_log 194215'15771 (193305'15616) clean 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by osd.304.0:1368 2014-08-12 15:48:58.380613 2014-08-13 18:06:06.918924 7f5c973a57a0 20 read_log 194215'15772 (194215'15771) delete 60550b3a/benchmark_data_storage0111-ib.pocstor_62621_object7800/head//24 by client.2174595.0:170379 0.000000 2014-08-13 18:06:06.918935 7f5c973a57a0 20 read_log 194215'15773 (192834'15076) clean 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by osd.304.0:1370 2014-08-12 15:48:58.387695 2014-08-13 18:06:06.918947 7f5c973a57a0 20 read_log 194215'15774 (194215'15773) delete 5da32b3a/benchmark_data_storage0111-ib.pocstor_41526_object6719/head//24 by client.2174595.0:170381 0.000000 2014-08-13 18:06:06.918958 7f5c973a57a0 20 read_log 194215'15775 (192888'15333) clean eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by osd.304.0:1372 2014-08-12 15:48:58.395329 2014-08-13 18:06:06.918970 7f5c973a57a0 20 read_log 194215'15776 (194215'15775) delete eeaf2b3a/benchmark_data_storage0111-ib.pocstor_41097_object4640/head//24 by client.2174595.0:170383 0.000000 2014-08-13 18:06:06.918984 7f5c973a57a0 20 read_log 194215'15777 (192834'14533) clean fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by osd.304.0:1374 2014-08-12 15:48:58.403242 2014-08-13 18:06:06.918997 7f5c973a57a0 20 read_log 194215'15778 (194215'15777) delete fb283b3a/benchmark_data_storage0111-ib.pocstor_26659_object2210/head//24 by client.2174595.0:170385 0.000000 2014-08-13 18:06:06.919010 7f5c973a57a0 20 read_log 194215'15779 (193305'15635) clean 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by osd.304.0:1376 2014-08-12 15:48:58.410808 2014-08-13 18:06:06.919042 7f5c973a57a0 20 read_log 194215'15780 (194215'15779) delete 34524b3a/benchmark_data_storage0111-ib.pocstor_62621_object21137/head//24 by client.2174595.0:170387 0.000000 2014-08-13 18:06:06.919057 7f5c973a57a0 20 read_log 194215'15781 (192888'15362) clean c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by osd.304.0:1378 2014-08-12 15:48:58.418577 2014-08-13 18:06:06.919069 7f5c973a57a0 20 read_log 194215'15782 (194215'15781) delete c7bb4b3a/benchmark_data_storage0111-ib.pocstor_41097_object17463/head//24 by client.2174595.0:170389 0.000000 2014-08-13 18:06:06.919083 7f5c973a57a0 20 read_log 194215'15783 (192888'15329) clean 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by osd.304.0:1380 2014-08-12 15:48:58.427827 2014-08-13 18:06:06.919095 7f5c973a57a0 20 read_log 194215'15784 (194215'15783) delete 93d4b3a/benchmark_data_storage0111-ib.pocstor_41097_object2117/head//24 by client.2174595.0:170391 0.000000 2014-08-13 18:06:06.919108 7f5c973a57a0 20 read_log 194215'15785 (192834'15108) clean 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by osd.304.0:1382 2014-08-12 15:48:58.434875 2014-08-13 18:06:06.919121 7f5c973a57a0 20 read_log 194215'15786 (194215'15785) delete 188f5b3a/benchmark_data_storage0111-ib.pocstor_41526_object10109/head//24 by client.2174595.0:170393 0.000000 2014-08-13 18:06:06.919134 7f5c973a57a0 20 read_log 194215'15787 (192834'15085) clean 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by osd.304.0:1384 2014-08-12 15:48:58.441839 2014-08-13 18:06:06.919152 7f5c973a57a0 20 read_log 194215'15788 (194215'15787) delete 93117b3a/benchmark_data_storage0111-ib.pocstor_41526_object7769/head//24 by client.2174595.0:170395 0.000000 2014-08-13 18:06:06.919165 7f5c973a57a0 20 read_log 194215'15789 (192834'14572) clean 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by osd.304.0:1386 2014-08-12 15:48:58.448941 2014-08-13 18:06:06.919178 7f5c973a57a0 20 read_log 194215'15790 (194215'15789) delete 47437b3a/benchmark_data_storage0111-ib.pocstor_26659_object8003/head//24 by client.2174595.0:170397 0.000000 2014-08-13 18:06:06.919189 7f5c973a57a0 20 read_log 194215'15791 (193305'15626) clean 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by osd.304.0:1388 2014-08-12 15:48:58.456424 2014-08-13 18:06:06.919209 7f5c973a57a0 20 read_log 194215'15792 (194215'15791) delete 99d37b3a/benchmark_data_storage0111-ib.pocstor_62621_object13701/head//24 by client.2174595.0:170399 0.000000 2014-08-13 18:06:06.919223 7f5c973a57a0 20 read_log 194215'15793 (193305'15611) clean 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by osd.304.0:1390 2014-08-12 15:48:58.463909 2014-08-13 18:06:06.919238 7f5c973a57a0 20 read_log 194215'15794 (194215'15793) delete 99f47b3a/benchmark_data_storage0111-ib.pocstor_62621_object193/head//24 by client.2174595.0:170401 0.000000 2014-08-13 18:06:06.919251 7f5c973a57a0 20 read_log 194215'15795 (192834'15125) clean 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by osd.304.0:1392 2014-08-12 15:48:58.471543 2014-08-13 18:06:06.919264 7f5c973a57a0 20 read_log 194215'15796 (194215'15795) delete 5b0b7b3a/benchmark_data_storage0111-ib.pocstor_41526_object12137/head//24 by client.2174595.0:170403 0.000000 2014-08-13 18:06:06.919275 7f5c973a57a0 20 read_log 194215'15797 (192888'15324) clean 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by osd.304.0:1394 2014-08-12 15:48:58.479124 2014-08-13 18:06:06.919290 7f5c973a57a0 20 read_log 194215'15798 (194215'15797) delete 51248b3a/benchmark_data_storage0111-ib.pocstor_41097_object244/head//24 by client.2174595.0:170405 0.000000 2014-08-13 18:06:06.919302 7f5c973a57a0 20 read_log 194215'15799 (192888'15500) clean f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by osd.304.0:1396 2014-08-12 15:48:58.486422 2014-08-13 18:06:06.919314 7f5c973a57a0 20 read_log 194215'15800 (194215'15799) delete f1e59b3a/benchmark_data_storage0111-ib.pocstor_2228_object1716/head//24 by client.2174595.0:170407 0.000000 2014-08-13 18:06:06.919325 7f5c973a57a0 20 read_log 194215'15801 (192834'14651) clean 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by osd.304.0:1398 2014-08-12 15:48:58.493415 2014-08-13 18:06:06.919341 7f5c973a57a0 20 read_log 194215'15802 (194215'15801) delete 4f12ab3a/benchmark_data_storage0111-ib.pocstor_26659_object17076/head//24 by client.2174595.0:170409 0.000000 2014-08-13 18:06:06.919352 7f5c973a57a0 20 read_log 194215'15803 (192888'15529) clean 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by osd.304.0:1400 2014-08-12 15:48:58.500261 2014-08-13 18:06:06.919364 7f5c973a57a0 20 read_log 194215'15804 (194215'15803) delete 553cab3a/benchmark_data_storage0111-ib.pocstor_2228_object16815/head//24 by client.2174595.0:170411 0.000000 2014-08-13 18:06:06.919375 7f5c973a57a0 20 read_log 194215'15805 (192834'15060) clean f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by osd.304.0:1402 2014-08-12 15:48:58.507343 2014-08-13 18:06:06.919391 7f5c973a57a0 20 read_log 194215'15806 (194215'15805) delete f98cab3a/benchmark_data_storage0111-ib.pocstor_41526_object3779/head//24 by client.2174595.0:170413 0.000000 2014-08-13 18:06:06.919414 7f5c973a57a0 20 read_log 194215'15807 (193305'15640) clean 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by osd.304.0:1404 2014-08-12 15:48:58.514514 2014-08-13 18:06:06.919431 7f5c973a57a0 20 read_log 194215'15808 (194215'15807) delete 89efab3a/benchmark_data_storage0111-ib.pocstor_62621_object29501/head//24 by client.2174595.0:170415 0.000000 2014-08-13 18:06:06.919472 7f5c973a57a0 20 read_log 194215'15809 (192888'15370) clean 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by osd.304.0:1406 2014-08-12 15:48:58.522291 2014-08-13 18:06:06.919488 7f5c973a57a0 20 read_log 194215'15810 (194215'15809) delete 5505bb3a/benchmark_data_storage0111-ib.pocstor_41097_object24363/head//24 by client.2174595.0:170417 0.000000 2014-08-13 18:06:06.919507 7f5c973a57a0 20 read_log 194215'15811 (192888'15335) clean 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by osd.304.0:1408 2014-08-12 15:48:58.529519 2014-08-13 18:06:06.919533 7f5c973a57a0 20 read_log 194215'15812 (194215'15811) delete 115ebb3a/benchmark_data_storage0111-ib.pocstor_41097_object5134/head//24 by client.2174595.0:170419 0.000000 2014-08-13 18:06:06.919547 7f5c973a57a0 20 read_log 194215'15813 (192834'15079) clean a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by osd.304.0:1410 2014-08-12 15:48:58.536928 2014-08-13 18:06:06.919559 7f5c973a57a0 20 read_log 194215'15814 (194215'15813) delete a81fbb3a/benchmark_data_storage0111-ib.pocstor_41526_object6826/head//24 by client.2174595.0:170421 0.000000 2014-08-13 18:06:06.919575 7f5c973a57a0 20 read_log 194215'15815 (192834'15104) clean 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by osd.304.0:1412 2014-08-12 15:48:58.543570 2014-08-13 18:06:06.919588 7f5c973a57a0 20 read_log 194215'15816 (194215'15815) delete 4952cb3a/benchmark_data_storage0111-ib.pocstor_41526_object9701/head//24 by client.2174595.0:170423 0.000000 2014-08-13 18:06:06.919599 7f5c973a57a0 20 read_log 194215'15817 (192834'15037) clean 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by osd.304.0:1414 2014-08-12 15:48:58.550359 2014-08-13 18:06:06.919611 7f5c973a57a0 20 read_log 194215'15818 (194215'15817) delete 1153db3a/benchmark_data_storage0111-ib.pocstor_41526_object607/head//24 by client.2174595.0:170425 0.000000 2014-08-13 18:06:06.919622 7f5c973a57a0 20 read_log 194215'15819 (192834'14550) clean 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by osd.304.0:1416 2014-08-12 15:48:58.557762 2014-08-13 18:06:06.919634 7f5c973a57a0 20 read_log 194215'15820 (194215'15819) delete 7966db3a/benchmark_data_storage0111-ib.pocstor_26659_object4944/head//24 by client.2174595.0:170427 0.000000 2014-08-13 18:06:06.919645 7f5c973a57a0 20 read_log 194215'15821 (193305'15619) clean 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by osd.304.0:1418 2014-08-12 15:48:58.564935 2014-08-13 18:06:06.919666 7f5c973a57a0 20 read_log 194215'15822 (194215'15821) delete 6c0ddb3a/benchmark_data_storage0111-ib.pocstor_62621_object9426/head//24 by client.2174595.0:170429 0.000000 2014-08-13 18:06:06.919678 7f5c973a57a0 20 read_log 194215'15823 (193305'15618) clean 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by osd.304.0:1420 2014-08-12 15:48:58.572018 2014-08-13 18:06:06.919691 7f5c973a57a0 20 read_log 194215'15824 (194215'15823) delete 6730eb3a/benchmark_data_storage0111-ib.pocstor_62621_object9213/head//24 by client.2174595.0:170431 0.000000 2014-08-13 18:06:06.919702 7f5c973a57a0 20 read_log 194215'15825 (192834'14616) clean 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by osd.304.0:1422 2014-08-12 15:48:58.579378 2014-08-13 18:06:06.919714 7f5c973a57a0 20 read_log 194215'15826 (194215'15825) delete 3622eb3a/benchmark_data_storage0111-ib.pocstor_26659_object13117/head//24 by client.2174595.0:170433 0.000000 2014-08-13 18:06:06.919725 7f5c973a57a0 20 read_log 194215'15827 (192888'15327) clean be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by osd.304.0:1424 2014-08-12 15:48:58.586255 2014-08-13 18:06:06.919738 7f5c973a57a0 20 read_log 194215'15828 (194215'15827) delete be43eb3a/benchmark_data_storage0111-ib.pocstor_41097_object1453/head//24 by client.2174595.0:170435 0.000000 2014-08-13 18:06:06.919748 7f5c973a57a0 20 read_log 194215'15829 (192834'14587) clean f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by osd.304.0:1426 2014-08-12 15:48:58.594131 2014-08-13 18:06:06.919761 7f5c973a57a0 20 read_log 194215'15830 (194215'15829) delete f1b0fb3a/benchmark_data_storage0111-ib.pocstor_26659_object9622/head//24 by client.2174595.0:170437 0.000000 2014-08-13 18:06:06.919772 7f5c973a57a0 20 read_log 194215'15831 (192888'15371) clean b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by osd.304.0:1428 2014-08-12 15:48:58.601172 2014-08-13 18:06:06.919794 7f5c973a57a0 20 read_log 194215'15832 (194215'15831) delete b65cfb3a/benchmark_data_storage0111-ib.pocstor_41097_object24460/head//24 by client.2174595.0:170439 0.000000 2014-08-13 18:06:06.919808 7f5c973a57a0 20 read_log 194215'15833 (192888'15505) clean 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by osd.304.0:1430 2014-08-12 15:48:58.609991 2014-08-13 18:06:06.919821 7f5c973a57a0 20 read_log 194215'15834 (194215'15833) delete 396ffb3a/benchmark_data_storage0111-ib.pocstor_2228_object4173/head//24 by client.2174595.0:170441 0.000000 2014-08-13 18:06:06.919832 7f5c973a57a0 20 read_log 194215'15835 (193305'15638) clean 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by osd.304.0:1432 2014-08-12 15:48:58.617416 2014-08-13 18:06:06.919844 7f5c973a57a0 20 read_log 194215'15836 (194215'15835) delete 32ad0f3a/benchmark_data_storage0111-ib.pocstor_62621_object27731/head//24 by client.2174595.0:170443 0.000000 2014-08-13 18:06:06.919855 7f5c973a57a0 20 read_log 194215'15837 (193305'15637) clean f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by osd.304.0:1434 2014-08-12 15:48:58.624562 2014-08-13 18:06:06.919877 7f5c973a57a0 20 read_log 194215'15838 (194215'15837) delete f93e1f3a/benchmark_data_storage0111-ib.pocstor_62621_object26459/head//24 by client.2174595.0:170445 0.000000 2014-08-13 18:06:06.919888 7f5c973a57a0 20 read_log 194215'15839 (192888'15374) clean 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by osd.304.0:1436 2014-08-12 15:48:58.646026 2014-08-13 18:06:06.919901 7f5c973a57a0 20 read_log 194215'15840 (194215'15839) delete 77ae1f3a/benchmark_data_storage0111-ib.pocstor_41097_object25098/head//24 by client.2174595.0:170447 0.000000 2014-08-13 18:06:06.919912 7f5c973a57a0 20 read_log 194215'15841 (193305'15631) clean 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by osd.304.0:1438 2014-08-12 15:48:58.652808 2014-08-13 18:06:06.919924 7f5c973a57a0 20 read_log 194215'15842 (194215'15841) delete 59af1f3a/benchmark_data_storage0111-ib.pocstor_62621_object17788/head//24 by client.2174595.0:170449 0.000000 2014-08-13 18:06:06.919935 7f5c973a57a0 20 read_log 194215'15843 (193305'15641) clean 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by osd.304.0:1440 2014-08-12 15:48:58.659971 2014-08-13 18:06:06.919947 7f5c973a57a0 20 read_log 194215'15844 (194215'15843) delete 1cf32f3a/benchmark_data_storage0111-ib.pocstor_62621_object30288/head//24 by client.2174595.0:170451 0.000000 2014-08-13 18:06:06.919958 7f5c973a57a0 20 read_log 194215'15845 (192888'15546) clean 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by osd.304.0:1442 2014-08-12 15:48:58.667391 2014-08-13 18:06:06.919970 7f5c973a57a0 20 read_log 194215'15846 (194215'15845) delete 20f33f3a/benchmark_data_storage0111-ib.pocstor_2228_object29116/head//24 by client.2174595.0:170453 0.000000 2014-08-13 18:06:06.919981 7f5c973a57a0 20 read_log 194215'15847 (192888'15340) clean b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by osd.304.0:1444 2014-08-12 15:48:58.674258 2014-08-13 18:06:06.919993 7f5c973a57a0 20 read_log 194215'15848 (194215'15847) delete b1643f3a/benchmark_data_storage0111-ib.pocstor_41097_object7003/head//24 by client.2174595.0:170455 0.000000 2014-08-13 18:06:06.920006 7f5c973a57a0 20 read_log 194215'15849 (192834'15065) clean 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by osd.304.0:1446 2014-08-12 15:48:58.682531 2014-08-13 18:06:06.920018 7f5c973a57a0 20 read_log 194215'15850 (194215'15849) delete 80384f3a/benchmark_data_storage0111-ib.pocstor_41526_object5094/head//24 by client.2174595.0:170457 0.000000 2014-08-13 18:06:06.920029 7f5c973a57a0 20 read_log 194215'15851 (192888'15356) clean be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by osd.304.0:1448 2014-08-12 15:48:58.690000 2014-08-13 18:06:06.920052 7f5c973a57a0 20 read_log 194215'15852 (194215'15851) delete be784f3a/benchmark_data_storage0111-ib.pocstor_41097_object12822/head//24 by client.2174595.0:170459 0.000000 2014-08-13 18:06:06.920064 7f5c973a57a0 20 read_log 194215'15853 (192888'15524) clean b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by osd.304.0:1450 2014-08-12 15:48:58.697855 2014-08-13 18:06:06.920076 7f5c973a57a0 20 read_log 194215'15854 (194215'15853) delete b1c84f3a/benchmark_data_storage0111-ib.pocstor_2228_object15229/head//24 by client.2174595.0:170461 0.000000 2014-08-13 18:06:06.920087 7f5c973a57a0 20 read_log 194215'15855 (192834'15044) clean 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by osd.304.0:1452 2014-08-12 15:48:58.705384 2014-08-13 18:06:06.920102 7f5c973a57a0 20 read_log 194215'15856 (194215'15855) delete 46a4f3a/benchmark_data_storage0111-ib.pocstor_41526_object1692/head//24 by client.2174595.0:170463 0.000000 2014-08-13 18:06:06.920114 7f5c973a57a0 20 read_log 194215'15857 (192834'14622) clean c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by osd.304.0:1454 2014-08-12 15:48:58.714083 2014-08-13 18:06:06.920126 7f5c973a57a0 20 read_log 194215'15858 (194215'15857) delete c75b4f3a/benchmark_data_storage0111-ib.pocstor_26659_object14550/head//24 by client.2174595.0:170465 0.000000 2014-08-13 18:06:06.920154 7f5c973a57a0 20 read_log 194215'15859 (192834'14592) clean d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by osd.304.0:1456 2014-08-12 15:48:58.721567 2014-08-13 18:06:06.920167 7f5c973a57a0 20 read_log 194215'15860 (194215'15859) delete d5415f3a/benchmark_data_storage0111-ib.pocstor_26659_object10688/head//24 by client.2174595.0:170467 0.000000 2014-08-13 18:06:06.920188 7f5c973a57a0 20 read_log 194215'15861 (193305'15642) clean dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by osd.304.0:1458 2014-08-12 15:48:58.729123 2014-08-13 18:06:06.920203 7f5c973a57a0 20 read_log 194215'15862 (194215'15861) delete dad96f3a/benchmark_data_storage0111-ib.pocstor_62621_object30888/head//24 by client.2174595.0:170469 0.000000 2014-08-13 18:06:06.920214 7f5c973a57a0 20 read_log 194215'15863 (192888'15378) clean ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by osd.304.0:1460 2014-08-12 15:48:58.737016 2014-08-13 18:06:06.920228 7f5c973a57a0 20 read_log 194215'15864 (194215'15863) delete ab8d6f3a/benchmark_data_storage0111-ib.pocstor_41097_object26941/head//24 by client.2174595.0:170471 0.000000 2014-08-13 18:06:06.920239 7f5c973a57a0 20 read_log 194215'15865 (192834'15128) clean ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by osd.304.0:1462 2014-08-12 15:48:58.744069 2014-08-13 18:06:06.920252 7f5c973a57a0 20 read_log 194215'15866 (194215'15865) delete ee67f3a/benchmark_data_storage0111-ib.pocstor_41526_object12528/head//24 by client.2174595.0:170473 0.000000 2014-08-13 18:06:06.920263 7f5c973a57a0 20 read_log 194215'15867 (192834'15066) clean f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by osd.304.0:1464 2014-08-12 15:48:58.751891 2014-08-13 18:06:06.920281 7f5c973a57a0 20 read_log 194215'15868 (194215'15867) delete f5e48f3a/benchmark_data_storage0111-ib.pocstor_41526_object5246/head//24 by client.2174595.0:170475 0.000000 2014-08-13 18:06:06.920294 7f5c973a57a0 20 read_log 194215'15869 (192888'15338) clean a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by osd.304.0:1466 2014-08-12 15:48:58.758510 2014-08-13 18:06:06.920307 7f5c973a57a0 20 read_log 194215'15870 (194215'15869) delete a4ea8f3a/benchmark_data_storage0111-ib.pocstor_41097_object6687/head//24 by client.2174595.0:170477 0.000000 2014-08-13 18:06:06.920318 7f5c973a57a0 20 read_log 194215'15871 (192834'14555) clean 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by osd.304.0:1468 2014-08-12 15:48:58.765770 2014-08-13 18:06:06.920349 7f5c973a57a0 20 read_log 194215'15872 (194215'15871) delete 6bee8f3a/benchmark_data_storage0111-ib.pocstor_26659_object5632/head//24 by client.2174595.0:170479 0.000000 2014-08-13 18:06:06.920372 7f5c973a57a0 20 read_log 194215'15873 (193305'15632) clean 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by osd.304.0:1470 2014-08-12 15:48:58.773086 2014-08-13 18:06:06.920391 7f5c973a57a0 20 read_log 194215'15874 (194215'15873) delete 32cf8f3a/benchmark_data_storage0111-ib.pocstor_62621_object19018/head//24 by client.2174595.0:170481 0.000000 2014-08-13 18:06:06.920402 7f5c973a57a0 20 read_log 194215'15875 (192888'15547) clean 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by osd.304.0:1472 2014-08-12 15:48:58.781181 2014-08-13 18:06:06.920415 7f5c973a57a0 20 read_log 194215'15876 (194215'15875) delete 49229f3a/benchmark_data_storage0111-ib.pocstor_2228_object29344/head//24 by client.2174595.0:170483 0.000000 2014-08-13 18:06:06.920428 7f5c973a57a0 20 read_log 194215'15877 (192888'15381) clean d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by osd.304.0:1474 2014-08-12 15:48:58.789086 2014-08-13 18:06:06.920441 7f5c973a57a0 20 read_log 194215'15878 (194215'15877) delete d6159f3a/benchmark_data_storage0111-ib.pocstor_41097_object28129/head//24 by client.2174595.0:170485 0.000000 2014-08-13 18:06:06.920459 7f5c973a57a0 20 read_log 194215'15879 (192834'15042) clean 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by osd.304.0:1476 2014-08-12 15:48:58.796702 2014-08-13 18:06:06.920475 7f5c973a57a0 20 read_log 194215'15880 (194215'15879) delete 59979f3a/benchmark_data_storage0111-ib.pocstor_41526_object1654/head//24 by client.2174595.0:170487 0.000000 2014-08-13 18:06:06.920486 7f5c973a57a0 20 read_log 194215'15881 (192888'15358) clean a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by osd.304.0:1478 2014-08-12 15:48:58.804620 2014-08-13 18:06:06.920501 7f5c973a57a0 20 read_log 194215'15882 (194215'15881) delete a1a89f3a/benchmark_data_storage0111-ib.pocstor_41097_object15073/head//24 by client.2174595.0:170489 0.000000 2014-08-13 18:06:06.920512 7f5c973a57a0 20 read_log 194215'15883 (192834'15185) clean f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by osd.304.0:1480 2014-08-12 15:48:58.812597 2014-08-13 18:06:06.920524 7f5c973a57a0 20 read_log 194215'15884 (194215'15883) delete f38b9f3a/benchmark_data_storage0111-ib.pocstor_41526_object21676/head//24 by client.2174595.0:170491 0.000000 2014-08-13 18:06:06.920535 7f5c973a57a0 20 read_log 194215'15885 (192834'15173) clean b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by osd.304.0:1482 2014-08-12 15:48:58.820753 2014-08-13 18:06:06.920547 7f5c973a57a0 20 read_log 194215'15886 (194215'15885) delete b1a2af3a/benchmark_data_storage0111-ib.pocstor_41526_object20357/head//24 by client.2174595.0:170493 0.000000 2014-08-13 18:06:06.920558 7f5c973a57a0 20 read_log 194215'15887 (192888'15341) clean 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by osd.304.0:1484 2014-08-12 15:48:58.828071 2014-08-13 18:06:06.920571 7f5c973a57a0 20 read_log 194215'15888 (194215'15887) delete 6830bf3a/benchmark_data_storage0111-ib.pocstor_41097_object7385/head//24 by client.2174595.0:170495 0.000000 2014-08-13 18:06:06.920582 7f5c973a57a0 20 read_log 194215'15889 (192834'14585) clean 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by osd.304.0:1486 2014-08-12 15:48:58.835089 2014-08-13 18:06:06.920594 7f5c973a57a0 20 read_log 194215'15890 (194215'15889) delete 26adbf3a/benchmark_data_storage0111-ib.pocstor_26659_object9309/head//24 by client.2174595.0:170497 0.000000 2014-08-13 18:06:06.920605 7f5c973a57a0 20 read_log 194215'15891 (193305'15636) clean c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by osd.304.0:1488 2014-08-12 15:48:58.842880 2014-08-13 18:06:06.920642 7f5c973a57a0 20 read_log 194215'15892 (194215'15891) delete c3fbf3a/benchmark_data_storage0111-ib.pocstor_62621_object22587/head//24 by client.2174595.0:170499 0.000000 2014-08-13 18:06:06.920656 7f5c973a57a0 20 read_log 194215'15893 (192888'15365) clean 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by osd.304.0:1490 2014-08-12 15:48:58.850448 2014-08-13 18:06:06.920669 7f5c973a57a0 20 read_log 194215'15894 (194215'15893) delete 59ffcf3a/benchmark_data_storage0111-ib.pocstor_41097_object20429/head//24 by client.2174595.0:170501 0.000000 2014-08-13 18:06:06.920680 7f5c973a57a0 20 read_log 194215'15895 (192888'15382) clean 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by osd.304.0:1492 2014-08-12 15:48:58.857113 2014-08-13 18:06:06.920692 7f5c973a57a0 20 read_log 194215'15896 (194215'15895) delete 5d4fdf3a/benchmark_data_storage0111-ib.pocstor_41097_object28406/head//24 by client.2174595.0:170503 0.000000 2014-08-13 18:06:06.920703 7f5c973a57a0 20 read_log 194215'15897 (193305'15629) clean 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by osd.304.0:1494 2014-08-12 15:48:58.864247 2014-08-13 18:06:06.920715 7f5c973a57a0 20 read_log 194215'15898 (194215'15897) delete 7710ef3a/benchmark_data_storage0111-ib.pocstor_62621_object17237/head//24 by client.2174595.0:170505 0.000000 2014-08-13 18:06:06.920726 7f5c973a57a0 20 read_log 194215'15899 (192834'15193) clean 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by osd.304.0:1496 2014-08-12 15:48:58.871743 2014-08-13 18:06:06.920739 7f5c973a57a0 20 read_log 194215'15900 (194215'15899) delete 9c3ef3a/benchmark_data_storage0111-ib.pocstor_41526_object22416/head//24 by client.2174595.0:170507 0.000000 2014-08-13 18:06:06.920750 7f5c973a57a0 20 read_log 194215'15901 (192834'15132) clean 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by osd.304.0:1498 2014-08-12 15:48:58.879492 2014-08-13 18:06:06.920779 7f5c973a57a0 20 read_log 194215'15902 (194215'15901) delete 997ef3a/benchmark_data_storage0111-ib.pocstor_41526_object13571/head//24 by client.2174595.0:170509 0.000000 2014-08-13 18:06:06.920793 7f5c973a57a0 20 read_log 194215'15903 (192834'14684) clean 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by osd.304.0:1500 2014-08-12 15:48:58.886606 2014-08-13 18:06:06.920805 7f5c973a57a0 20 read_log 194215'15904 (194215'15903) delete 5878ef3a/benchmark_data_storage0111-ib.pocstor_26659_object20610/head//24 by client.2174595.0:170511 0.000000 2014-08-13 18:06:06.920816 7f5c973a57a0 20 read_log 194215'15905 (192888'15545) clean c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by osd.304.0:1502 2014-08-12 15:48:58.894488 2014-08-13 18:06:06.920829 7f5c973a57a0 20 read_log 194215'15906 (194215'15905) delete c4dbef3a/benchmark_data_storage0111-ib.pocstor_2228_object28954/head//24 by client.2174595.0:170513 0.000000 2014-08-13 18:06:06.920840 7f5c973a57a0 20 read_log 194215'15907 (192888'15515) clean 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by osd.304.0:1504 2014-08-12 15:48:58.902234 2014-08-13 18:06:06.920854 7f5c973a57a0 20 read_log 194215'15908 (194215'15907) delete 277def3a/benchmark_data_storage0111-ib.pocstor_2228_object9786/head//24 by client.2174595.0:170515 0.000000 2014-08-13 18:06:06.920865 7f5c973a57a0 20 read_log 194215'15909 (192834'14682) clean 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by osd.304.0:1506 2014-08-12 15:48:58.910205 2014-08-13 18:06:06.920880 7f5c973a57a0 20 read_log 194215'15910 (194215'15909) delete 5ebff3a/benchmark_data_storage0111-ib.pocstor_26659_object20523/head//24 by client.2174595.0:170517 0.000000 2014-08-13 18:06:06.920891 7f5c973a57a0 20 read_log 194215'15911 (192888'15542) clean f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by osd.304.0:1508 2014-08-12 15:48:58.917821 2014-08-13 18:06:06.920913 7f5c973a57a0 20 read_log 194215'15912 (194215'15911) delete f4ccff3a/benchmark_data_storage0111-ib.pocstor_2228_object25572/head//24 by client.2174595.0:170519 0.000000 2014-08-13 18:06:06.920927 7f5c973a57a0 20 read_log 194217'15913 (194217'15913) modify 33a/hit_set_24.33a_archive_2014-08-12 15:48:57.821312_2014-08-12 17:25:04.496076/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 18:06:06.920940 7f5c973a57a0 20 read_log 194217'15914 (194215'15643) delete 33a/hit_set_24.33a_archive_2014-08-12 15:32:25.735124_2014-08-12 15:48:57.821063/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 18:06:06.920955 7f5c973a57a0 20 read_log 194217'15915 (0'0) promote d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by osd.304.0:2262 2014-08-12 17:25:04.499120 2014-08-13 18:06:06.920969 7f5c973a57a0 20 read_log 194217'15916 (194217'15915) modify d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by client.1487927.0:307 2014-08-12 17:25:04.489405 2014-08-13 18:06:06.920984 7f5c973a57a0 20 read_log 194217'15917 (0'0) promote 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by osd.304.0:2266 2014-08-12 17:25:05.614118 2014-08-13 18:06:06.920998 7f5c973a57a0 20 read_log 194217'15918 (194217'15917) modify 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by client.1487927.0:434 2014-08-12 17:25:05.605946 2014-08-13 18:06:06.921013 7f5c973a57a0 20 read_log 194370'15919 (0'0) promote 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by osd.304.0:2272 2014-08-12 20:30:15.868992 2014-08-13 18:06:06.921026 7f5c973a57a0 20 read_log 194370'15920 (194370'15919) modify 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by client.1508531.0:392 2014-08-12 20:30:15.862085 2014-08-13 18:06:06.921038 7f5c973a57a0 20 read_log 194370'15921 (0'0) promote 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by osd.304.0:2290 2014-08-12 20:30:20.189642 2014-08-13 18:06:06.921050 7f5c973a57a0 20 read_log 194370'15922 (194370'15921) modify 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by client.1508531.0:1547 2014-08-12 20:30:20.183075 2014-08-13 18:06:06.921063 7f5c973a57a0 20 read_log 194370'15923 (0'0) promote 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by osd.304.0:2318 2014-08-12 20:30:29.623516 2014-08-13 18:06:06.921077 7f5c973a57a0 20 read_log 194370'15924 (194370'15923) modify 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by client.1508531.0:4089 2014-08-12 20:30:29.617119 2014-08-13 18:06:06.921090 7f5c973a57a0 20 read_log 194370'15925 (0'0) promote 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by osd.304.0:2346 2014-08-12 20:30:36.419403 2014-08-13 18:06:06.921102 7f5c973a57a0 20 read_log 194370'15926 (194370'15925) modify 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by client.1508531.0:5714 2014-08-12 20:30:36.411349 2014-08-13 18:06:06.921116 7f5c973a57a0 20 read_log 194370'15927 (0'0) promote 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by osd.304.0:2356 2014-08-12 20:30:39.333156 2014-08-13 18:06:06.921128 7f5c973a57a0 20 read_log 194370'15928 (194370'15927) modify 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by client.1508531.0:6368 2014-08-12 20:30:39.326068 2014-08-13 18:06:06.921141 7f5c973a57a0 20 read_log 194370'15929 (0'0) promote 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by osd.304.0:2358 2014-08-12 20:30:39.903464 2014-08-13 18:06:06.921161 7f5c973a57a0 20 read_log 194370'15930 (194370'15929) modify 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by client.1508531.0:6487 2014-08-12 20:30:39.894743 2014-08-13 18:06:06.921184 7f5c973a57a0 20 read_log 194370'15931 (0'0) promote fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by osd.304.0:2374 2014-08-12 20:30:49.259262 2014-08-13 18:06:06.921199 7f5c973a57a0 20 read_log 194370'15932 (194370'15931) modify fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by client.1508531.0:8317 2014-08-12 20:30:49.252693 2014-08-13 18:06:06.921212 7f5c973a57a0 20 read_log 194370'15933 (0'0) promote 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by osd.304.0:2400 2014-08-12 20:31:07.746055 2014-08-13 18:06:06.921227 7f5c973a57a0 20 read_log 194370'15934 (194370'15933) modify 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by client.1508531.0:11403 2014-08-12 20:31:07.740071 2014-08-13 18:06:06.921242 7f5c973a57a0 20 read_log 194370'15935 (0'0) promote a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by osd.304.0:2402 2014-08-12 20:31:07.896567 2014-08-13 18:06:06.921259 7f5c973a57a0 20 read_log 194370'15936 (194370'15935) modify a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by client.1508531.0:11430 2014-08-12 20:31:07.890091 2014-08-13 18:06:06.921274 7f5c973a57a0 20 read_log 194370'15937 (0'0) promote 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by osd.304.0:2408 2014-08-12 20:31:12.899146 2014-08-13 18:06:06.921287 7f5c973a57a0 20 read_log 194370'15938 (194370'15937) modify 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by client.1508531.0:12208 2014-08-12 20:31:12.891369 2014-08-13 18:06:06.921299 7f5c973a57a0 20 read_log 194370'15939 (0'0) promote 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by osd.304.0:2414 2014-08-12 20:31:14.757246 2014-08-13 18:06:06.921316 7f5c973a57a0 20 read_log 194370'15940 (194370'15939) modify 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by client.1508531.0:12466 2014-08-12 20:31:14.749585 2014-08-13 18:06:06.921329 7f5c973a57a0 20 read_log 194377'15941 (0'0) promote 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by osd.304.0:2440 2014-08-12 20:33:04.737896 2014-08-13 18:06:06.921341 7f5c973a57a0 20 read_log 194377'15942 (194377'15941) modify 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by client.2231190.0:1883 2014-08-12 20:33:04.731658 2014-08-13 18:06:06.921357 7f5c973a57a0 20 read_log 194377'15943 (0'0) promote 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by osd.304.0:2462 2014-08-12 20:33:12.309625 2014-08-13 18:06:06.921395 7f5c973a57a0 20 read_log 194377'15944 (194377'15943) modify 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by client.2231190.0:3695 2014-08-12 20:33:12.304340 2014-08-13 18:06:06.921415 7f5c973a57a0 20 read_log 194377'15945 (0'0) promote e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by osd.304.0:2470 2014-08-12 20:33:14.997340 2014-08-13 18:06:06.921434 7f5c973a57a0 20 read_log 194377'15946 (194377'15945) modify e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by client.2231190.0:4182 2014-08-12 20:33:14.989295 2014-08-13 18:06:06.921451 7f5c973a57a0 20 read_log 194383'15947 (0'0) promote b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by osd.304.0:2488 2014-08-12 20:34:10.323439 2014-08-13 18:06:06.921463 7f5c973a57a0 20 read_log 194383'15948 (194383'15947) modify b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by client.2231190.0:6822 2014-08-12 20:34:10.315611 2014-08-13 18:06:06.921484 7f5c973a57a0 20 read_log 194383'15949 (0'0) promote 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by osd.304.0:2490 2014-08-12 20:34:10.846295 2014-08-13 18:06:06.921501 7f5c973a57a0 20 read_log 194383'15950 (194383'15949) modify 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by client.2231190.0:6841 2014-08-12 20:34:10.839540 2014-08-13 18:06:06.921521 7f5c973a57a0 20 read_log 194383'15951 (0'0) promote 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by osd.304.0:2508 2014-08-12 20:34:36.974881 2014-08-13 18:06:06.921536 7f5c973a57a0 20 read_log 194383'15952 (194383'15951) modify 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by client.2231190.0:7792 2014-08-12 20:34:36.968217 2014-08-13 18:06:06.921548 7f5c973a57a0 20 read_log 194393'15953 (0'0) promote 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by osd.304.0:2510 2014-08-12 20:47:25.958135 2014-08-13 18:06:06.921560 7f5c973a57a0 20 read_log 194393'15954 (194393'15953) modify 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by client.1064095.0:18 2014-08-12 20:47:25.949889 2014-08-13 18:06:06.921573 7f5c973a57a0 20 read_log 194397'15955 (0'0) promote 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by osd.304.0:2542 2014-08-12 20:53:06.101508 2014-08-13 18:06:06.921588 7f5c973a57a0 20 read_log 194397'15956 (194397'15955) modify 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by client.2235069.0:926 2014-08-12 20:53:06.093281 2014-08-13 18:06:06.921601 7f5c973a57a0 20 read_log 194416'15957 (0'0) promote fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by osd.304.0:2636 2014-08-13 15:28:30.857506 2014-08-13 18:06:06.921625 7f5c973a57a0 20 read_log 194416'15958 (194416'15957) modify fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by client.2441958.0:175 2014-08-13 15:28:30.846495 2014-08-13 18:06:06.921638 7f5c973a57a0 20 read_log 194416'15959 (0'0) promote bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by osd.304.0:2641 2014-08-13 15:28:34.025376 2014-08-13 18:06:06.921650 7f5c973a57a0 20 read_log 194416'15960 (194416'15959) modify bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by client.2441958.0:886 2014-08-13 15:28:34.018559 2014-08-13 18:06:06.921663 7f5c973a57a0 20 read_log 194416'15961 (0'0) promote 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by osd.304.0:2648 2014-08-13 15:28:36.112854 2014-08-13 18:06:06.921675 7f5c973a57a0 20 read_log 194416'15962 (194416'15961) modify 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by client.2441958.0:1421 2014-08-13 15:28:36.105385 2014-08-13 18:06:06.921687 7f5c973a57a0 20 read_log 194416'15963 (0'0) promote 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by osd.304.0:2650 2014-08-13 15:28:36.397928 2014-08-13 18:06:06.921700 7f5c973a57a0 20 read_log 194416'15964 (194416'15963) modify 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by client.2441958.0:1498 2014-08-13 15:28:36.390188 2014-08-13 18:06:06.921712 7f5c973a57a0 20 read_log 194416'15965 (0'0) promote e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by osd.304.0:2657 2014-08-13 15:28:40.177505 2014-08-13 18:06:06.921724 7f5c973a57a0 20 read_log 194416'15966 (194416'15965) modify e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by client.2441958.0:2498 2014-08-13 15:28:40.168526 2014-08-13 18:06:06.921736 7f5c973a57a0 20 read_log 194416'15967 (0'0) promote d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by osd.304.0:2661 2014-08-13 15:28:40.475143 2014-08-13 18:06:06.921748 7f5c973a57a0 20 read_log 194416'15968 (194416'15967) modify d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by client.2441958.0:2576 2014-08-13 15:28:40.467172 2014-08-13 18:06:06.921761 7f5c973a57a0 20 read_log 194416'15969 (0'0) promote da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by osd.304.0:2670 2014-08-13 15:28:45.984364 2014-08-13 18:06:06.921780 7f5c973a57a0 20 read_log 194416'15970 (194416'15969) modify da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by client.2441958.0:4014 2014-08-13 15:28:45.977701 2014-08-13 18:06:06.921793 7f5c973a57a0 20 read_log 194416'15971 (0'0) promote eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by osd.304.0:2676 2014-08-13 15:28:47.667845 2014-08-13 18:06:06.921807 7f5c973a57a0 20 read_log 194416'15972 (194416'15971) modify eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by client.2441958.0:4446 2014-08-13 15:28:47.660091 2014-08-13 18:06:06.921819 7f5c973a57a0 20 read_log 194416'15973 (0'0) promote 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by osd.304.0:2680 2014-08-13 15:28:48.119425 2014-08-13 18:06:06.921831 7f5c973a57a0 20 read_log 194416'15974 (194416'15973) modify 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by client.2441958.0:4542 2014-08-13 15:28:48.111350 2014-08-13 18:06:06.921843 7f5c973a57a0 20 read_log 194416'15975 (0'0) promote cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by osd.304.0:2692 2014-08-13 15:28:51.544599 2014-08-13 18:06:06.921856 7f5c973a57a0 20 read_log 194416'15976 (194416'15975) modify cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by client.2441958.0:5434 2014-08-13 15:28:51.536921 2014-08-13 18:06:06.921870 7f5c973a57a0 20 read_log 194416'15977 (0'0) promote 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by osd.304.0:2710 2014-08-13 15:28:56.459404 2014-08-13 18:06:06.921882 7f5c973a57a0 20 read_log 194416'15978 (194416'15977) modify 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by client.2441958.0:6645 2014-08-13 15:28:56.451593 2014-08-13 18:06:06.921897 7f5c973a57a0 20 read_log 194416'15979 (0'0) promote f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by osd.304.0:2720 2014-08-13 15:28:58.842643 2014-08-13 18:06:06.921909 7f5c973a57a0 20 read_log 194416'15980 (194416'15979) modify f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by client.2441958.0:7286 2014-08-13 15:28:58.835798 2014-08-13 18:06:06.921921 7f5c973a57a0 20 read_log 194416'15981 (0'0) promote b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by osd.304.0:2740 2014-08-13 15:29:05.612289 2014-08-13 18:06:06.921933 7f5c973a57a0 20 read_log 194416'15982 (194416'15981) modify b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by client.2441958.0:9035 2014-08-13 15:29:05.605167 2014-08-13 18:06:06.921946 7f5c973a57a0 20 read_log 194416'15983 (0'0) promote 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by osd.304.0:2744 2014-08-13 15:29:07.123515 2014-08-13 18:06:06.921960 7f5c973a57a0 20 read_log 194416'15984 (194416'15983) modify 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by client.2441958.0:9432 2014-08-13 15:29:07.115784 2014-08-13 18:06:06.921975 7f5c973a57a0 20 read_log 194416'15985 (0'0) promote fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by osd.304.0:2750 2014-08-13 15:29:07.440706 2014-08-13 18:06:06.921987 7f5c973a57a0 20 read_log 194416'15986 (194416'15985) modify fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by client.2441958.0:9514 2014-08-13 15:29:07.431964 2014-08-13 18:06:06.922001 7f5c973a57a0 20 read_log 194416'15987 (0'0) promote 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by osd.304.0:2756 2014-08-13 15:29:10.518230 2014-08-13 18:06:06.922014 7f5c973a57a0 20 read_log 194416'15988 (194416'15987) modify 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by client.2441958.0:10263 2014-08-13 15:29:10.507788 2014-08-13 18:06:06.922028 7f5c973a57a0 20 read_log 194416'15989 (0'0) promote d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by osd.304.0:2776 2014-08-13 15:29:19.234295 2014-08-13 18:06:06.922047 7f5c973a57a0 20 read_log 194416'15990 (194416'15989) modify d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by client.2441958.0:12379 2014-08-13 15:29:19.224710 2014-08-13 18:06:06.922062 7f5c973a57a0 20 read_log 194416'15991 (0'0) promote 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by osd.304.0:2780 2014-08-13 15:29:19.789551 2014-08-13 18:06:06.922075 7f5c973a57a0 20 read_log 194416'15992 (194416'15991) modify 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by client.2441958.0:12524 2014-08-13 15:29:19.782078 2014-08-13 18:06:06.922089 7f5c973a57a0 20 read_log 194416'15993 (0'0) promote b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by osd.304.0:2790 2014-08-13 15:29:22.498882 2014-08-13 18:06:06.922102 7f5c973a57a0 20 read_log 194416'15994 (194416'15993) modify b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by client.2441958.0:13120 2014-08-13 15:29:22.491211 2014-08-13 18:06:06.922133 7f5c973a57a0 20 read_log 194416'15995 (0'0) promote ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by osd.304.0:2796 2014-08-13 15:29:26.225346 2014-08-13 18:06:06.922154 7f5c973a57a0 20 read_log 194416'15996 (194416'15995) modify ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by client.2441958.0:14003 2014-08-13 15:29:26.217514 2014-08-13 18:06:06.922169 7f5c973a57a0 20 read_log 194416'15997 (0'0) promote eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by osd.304.0:2802 2014-08-13 15:29:28.953608 2014-08-13 18:06:06.922182 7f5c973a57a0 20 read_log 194416'15998 (194416'15997) modify eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by client.2441958.0:14695 2014-08-13 15:29:28.945683 2014-08-13 18:06:06.922194 7f5c973a57a0 20 read_log 194416'15999 (0'0) promote 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by osd.304.0:2806 2014-08-13 15:29:33.724536 2014-08-13 18:06:06.922206 7f5c973a57a0 20 read_log 194416'16000 (194416'15999) modify 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by client.2441958.0:15691 2014-08-13 15:29:33.715254 2014-08-13 18:06:06.922219 7f5c973a57a0 20 read_log 194416'16001 (0'0) promote 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by osd.304.0:2814 2014-08-13 15:29:37.910468 2014-08-13 18:06:06.922233 7f5c973a57a0 20 read_log 194416'16002 (194416'16001) modify 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by client.2441958.0:16332 2014-08-13 15:29:37.901575 2014-08-13 18:06:06.922251 7f5c973a57a0 20 read_log 194416'16003 (0'0) promote 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by osd.304.0:2818 2014-08-13 15:29:41.340684 2014-08-13 18:06:06.922265 7f5c973a57a0 20 read_log 194416'16004 (194416'16003) modify 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by client.2441958.0:16577 2014-08-13 15:29:41.332043 2014-08-13 18:06:06.922278 7f5c973a57a0 20 read_log 194416'16005 (0'0) promote eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by osd.304.0:2834 2014-08-13 15:29:54.664006 2014-08-13 18:06:06.922290 7f5c973a57a0 20 read_log 194416'16006 (194416'16005) modify eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by client.2441958.0:17516 2014-08-13 15:29:54.655401 2014-08-13 18:06:06.922303 7f5c973a57a0 20 read_log 194416'16007 (0'0) promote ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by osd.304.0:2836 2014-08-13 15:29:56.390028 2014-08-13 18:06:06.922315 7f5c973a57a0 20 read_log 194416'16008 (194416'16007) modify ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by client.2441958.0:17638 2014-08-13 15:29:56.381296 2014-08-13 18:06:06.922327 7f5c973a57a0 20 read_log 194416'16009 (0'0) promote 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by osd.304.0:2842 2014-08-13 15:30:00.877414 2014-08-13 18:06:06.922351 7f5c973a57a0 20 read_log 194416'16010 (194416'16009) modify 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by client.2441958.0:17958 2014-08-13 15:30:00.867823 2014-08-13 18:06:06.922367 7f5c973a57a0 20 read_log 194416'16011 (0'0) promote 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by osd.304.0:2848 2014-08-13 15:30:09.642168 2014-08-13 18:06:06.922379 7f5c973a57a0 20 read_log 194416'16012 (194416'16011) modify 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by client.2441958.0:18376 2014-08-13 15:30:09.633913 2014-08-13 18:06:06.922404 7f5c973a57a0 20 read_log 194422'16013 (0'0) promote 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by osd.304.0:2852 2014-08-13 15:32:16.737428 2014-08-13 18:06:06.922419 7f5c973a57a0 20 read_log 194422'16014 (194422'16013) modify 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by client.1148203.0:376 2014-08-13 15:32:16.730155 2014-08-13 18:06:06.922432 7f5c973a57a0 20 read_log 194422'16015 (0'0) promote d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by osd.304.0:2870 2014-08-13 15:32:23.702897 2014-08-13 18:06:06.922457 7f5c973a57a0 20 read_log 194422'16016 (194422'16015) modify d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by client.1148203.0:2123 2014-08-13 15:32:23.694114 2014-08-13 18:06:06.922470 7f5c973a57a0 20 read_log 194422'16017 (0'0) promote c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by osd.304.0:2888 2014-08-13 15:32:28.814325 2014-08-13 18:06:06.922482 7f5c973a57a0 20 read_log 194422'16018 (194422'16017) modify c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by client.1148203.0:3431 2014-08-13 15:32:28.807064 2014-08-13 18:06:06.922506 7f5c973a57a0 20 read_log 194422'16019 (0'0) promote 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by osd.304.0:2898 2014-08-13 15:32:34.015941 2014-08-13 18:06:06.922519 7f5c973a57a0 20 read_log 194422'16020 (194422'16019) modify 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by client.1148203.0:4784 2014-08-13 15:32:34.007596 2014-08-13 18:06:06.922532 7f5c973a57a0 20 read_log 194422'16021 (0'0) promote f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by osd.304.0:2902 2014-08-13 15:32:35.236035 2014-08-13 18:06:06.922544 7f5c973a57a0 20 read_log 194422'16022 (194422'16021) modify f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by client.1148203.0:5085 2014-08-13 15:32:35.228439 2014-08-13 18:06:06.922556 7f5c973a57a0 20 read_log 194422'16023 (0'0) promote a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by osd.304.0:2914 2014-08-13 15:32:39.906919 2014-08-13 18:06:06.922568 7f5c973a57a0 20 read_log 194422'16024 (194422'16023) modify a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by client.1148203.0:5960 2014-08-13 15:32:39.898895 2014-08-13 18:06:06.922627 7f5c973a57a0 20 read_log 194422'16025 (0'0) promote 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by osd.304.0:2922 2014-08-13 15:32:46.849289 2014-08-13 18:06:06.922643 7f5c973a57a0 20 read_log 194422'16026 (194422'16025) modify 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by client.1148203.0:7217 2014-08-13 15:32:46.840221 2014-08-13 18:06:06.922656 7f5c973a57a0 20 read_log 194422'16027 (0'0) promote 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by osd.304.0:2926 2014-08-13 15:32:47.864308 2014-08-13 18:06:06.922668 7f5c973a57a0 20 read_log 194422'16028 (194422'16027) modify 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by client.1148203.0:7382 2014-08-13 15:32:47.852956 2014-08-13 18:06:06.922681 7f5c973a57a0 20 read_log 194422'16029 (0'0) promote 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by osd.304.0:2940 2014-08-13 15:32:52.655792 2014-08-13 18:06:06.922695 7f5c973a57a0 20 read_log 194422'16030 (194422'16029) modify 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by client.1148203.0:8149 2014-08-13 15:32:52.647182 2014-08-13 18:06:06.922708 7f5c973a57a0 20 read_log 194422'16031 (0'0) promote 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by osd.304.0:2952 2014-08-13 15:33:07.870709 2014-08-13 18:06:06.922720 7f5c973a57a0 20 read_log 194422'16032 (194422'16031) modify 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by client.1148203.0:9801 2014-08-13 15:33:07.862907 2014-08-13 18:06:06.922735 7f5c973a57a0 20 read_log 194422'16033 (0'0) promote 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by osd.304.0:2958 2014-08-13 15:33:18.363531 2014-08-13 18:06:06.922747 7f5c973a57a0 20 read_log 194422'16034 (194422'16033) modify 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by client.1148203.0:10297 2014-08-13 15:33:18.354570 2014-08-13 18:06:06.922764 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.923155 7f5c973a57a0 10 read_log checking for missing items over interval (194215'15912,194422'16034] 2014-08-13 18:06:06.923202 7f5c973a57a0 15 read_log missing 194422'16034 (194422'16033) modify 837abf3a/benchmark_data_storage0111-ib.pocstor_41231_object10296/head//24 by client.1148203.0:10297 2014-08-13 15:33:18.354570 2014-08-13 18:06:06.923246 7f5c973a57a0 15 read_log missing 194422'16032 (194422'16031) modify 71622f3a/benchmark_data_storage0111-ib.pocstor_41231_object9800/head//24 by client.1148203.0:9801 2014-08-13 15:33:07.862907 2014-08-13 18:06:06.923287 7f5c973a57a0 15 read_log missing 194422'16030 (194422'16029) modify 893c333a/benchmark_data_storage0111-ib.pocstor_41231_object8148/head//24 by client.1148203.0:8149 2014-08-13 15:32:52.647182 2014-08-13 18:06:06.923324 7f5c973a57a0 15 read_log missing 194422'16028 (194422'16027) modify 5c7e3f3a/benchmark_data_storage0111-ib.pocstor_41231_object7381/head//24 by client.1148203.0:7382 2014-08-13 15:32:47.852956 2014-08-13 18:06:06.923359 7f5c973a57a0 15 read_log missing 194422'16026 (194422'16025) modify 505e6f3a/benchmark_data_storage0111-ib.pocstor_41231_object7216/head//24 by client.1148203.0:7217 2014-08-13 15:32:46.840221 2014-08-13 18:06:06.923407 7f5c973a57a0 15 read_log missing 194422'16024 (194422'16023) modify a838673a/benchmark_data_storage0111-ib.pocstor_41231_object5959/head//24 by client.1148203.0:5960 2014-08-13 15:32:39.898895 2014-08-13 18:06:06.923444 7f5c973a57a0 15 read_log missing 194422'16022 (194422'16021) modify f82d773a/benchmark_data_storage0111-ib.pocstor_41231_object5084/head//24 by client.1148203.0:5085 2014-08-13 15:32:35.228439 2014-08-13 18:06:06.923479 7f5c973a57a0 15 read_log missing 194422'16020 (194422'16019) modify 1c56f73a/benchmark_data_storage0111-ib.pocstor_41231_object4783/head//24 by client.1148203.0:4784 2014-08-13 15:32:34.007596 2014-08-13 18:06:06.923515 7f5c973a57a0 15 read_log missing 194422'16018 (194422'16017) modify c1796b3a/benchmark_data_storage0111-ib.pocstor_41231_object3430/head//24 by client.1148203.0:3431 2014-08-13 15:32:28.807064 2014-08-13 18:06:06.923551 7f5c973a57a0 15 read_log missing 194422'16016 (194422'16015) modify d8051b3a/benchmark_data_storage0111-ib.pocstor_41231_object2122/head//24 by client.1148203.0:2123 2014-08-13 15:32:23.694114 2014-08-13 18:06:06.923586 7f5c973a57a0 15 read_log missing 194422'16014 (194422'16013) modify 7d1a473a/benchmark_data_storage0111-ib.pocstor_41231_object375/head//24 by client.1148203.0:376 2014-08-13 15:32:16.730155 2014-08-13 18:06:06.923623 7f5c973a57a0 15 read_log missing 194416'16012 (194416'16011) modify 7f863f3a/benchmark_data_storage0111-ib.pocstor_19103_object18375/head//24 by client.2441958.0:18376 2014-08-13 15:30:09.633913 2014-08-13 18:06:06.923658 7f5c973a57a0 15 read_log missing 194416'16010 (194416'16009) modify 3a145b3a/benchmark_data_storage0111-ib.pocstor_19103_object17957/head//24 by client.2441958.0:17958 2014-08-13 15:30:00.867823 2014-08-13 18:06:06.923694 7f5c973a57a0 15 read_log missing 194416'16008 (194416'16007) modify ba08873a/benchmark_data_storage0111-ib.pocstor_19103_object17637/head//24 by client.2441958.0:17638 2014-08-13 15:29:56.381296 2014-08-13 18:06:06.923729 7f5c973a57a0 15 read_log missing 194416'16006 (194416'16005) modify eff3bf3a/benchmark_data_storage0111-ib.pocstor_19103_object17515/head//24 by client.2441958.0:17516 2014-08-13 15:29:54.655401 2014-08-13 18:06:06.923765 7f5c973a57a0 15 read_log missing 194416'16004 (194416'16003) modify 5d4fdb3a/benchmark_data_storage0111-ib.pocstor_19103_object16576/head//24 by client.2441958.0:16577 2014-08-13 15:29:41.332043 2014-08-13 18:06:06.923801 7f5c973a57a0 15 read_log missing 194416'16002 (194416'16001) modify 50949f3a/benchmark_data_storage0111-ib.pocstor_19103_object16331/head//24 by client.2441958.0:16332 2014-08-13 15:29:37.901575 2014-08-13 18:06:06.923836 7f5c973a57a0 15 read_log missing 194416'16000 (194416'15999) modify 44eaa33a/benchmark_data_storage0111-ib.pocstor_19103_object15690/head//24 by client.2441958.0:15691 2014-08-13 15:29:33.715254 2014-08-13 18:06:06.923872 7f5c973a57a0 15 read_log missing 194416'15998 (194416'15997) modify eb8a8f3a/benchmark_data_storage0111-ib.pocstor_19103_object14694/head//24 by client.2441958.0:14695 2014-08-13 15:29:28.945683 2014-08-13 18:06:06.923908 7f5c973a57a0 15 read_log missing 194416'15996 (194416'15995) modify ad7a833a/benchmark_data_storage0111-ib.pocstor_19103_object14002/head//24 by client.2441958.0:14003 2014-08-13 15:29:26.217514 2014-08-13 18:06:06.923943 7f5c973a57a0 15 read_log missing 194416'15994 (194416'15993) modify b2993b3a/benchmark_data_storage0111-ib.pocstor_19103_object13119/head//24 by client.2441958.0:13120 2014-08-13 15:29:22.491211 2014-08-13 18:06:06.923979 7f5c973a57a0 15 read_log missing 194416'15992 (194416'15991) modify 7b97373a/benchmark_data_storage0111-ib.pocstor_19103_object12523/head//24 by client.2441958.0:12524 2014-08-13 15:29:19.782078 2014-08-13 18:06:06.924017 7f5c973a57a0 15 read_log missing 194416'15990 (194416'15989) modify d811fb3a/benchmark_data_storage0111-ib.pocstor_19103_object12378/head//24 by client.2441958.0:12379 2014-08-13 15:29:19.224710 2014-08-13 18:06:06.924053 7f5c973a57a0 15 read_log missing 194416'15988 (194416'15987) modify 53a9df3a/benchmark_data_storage0111-ib.pocstor_19103_object10262/head//24 by client.2441958.0:10263 2014-08-13 15:29:10.507788 2014-08-13 18:06:06.924088 7f5c973a57a0 15 read_log missing 194416'15986 (194416'15985) modify fb14573a/benchmark_data_storage0111-ib.pocstor_19103_object9513/head//24 by client.2441958.0:9514 2014-08-13 15:29:07.431964 2014-08-13 18:06:06.924123 7f5c973a57a0 15 read_log missing 194416'15984 (194416'15983) modify 49d773a/benchmark_data_storage0111-ib.pocstor_19103_object9431/head//24 by client.2441958.0:9432 2014-08-13 15:29:07.115784 2014-08-13 18:06:06.924165 7f5c973a57a0 15 read_log missing 194416'15982 (194416'15981) modify b4fe8f3a/benchmark_data_storage0111-ib.pocstor_19103_object9034/head//24 by client.2441958.0:9035 2014-08-13 15:29:05.605167 2014-08-13 18:06:06.924201 7f5c973a57a0 15 read_log missing 194416'15980 (194416'15979) modify f7cb233a/benchmark_data_storage0111-ib.pocstor_19103_object7285/head//24 by client.2441958.0:7286 2014-08-13 15:28:58.835798 2014-08-13 18:06:06.924240 7f5c973a57a0 15 read_log missing 194416'15978 (194416'15977) modify 80e5af3a/benchmark_data_storage0111-ib.pocstor_19103_object6644/head//24 by client.2441958.0:6645 2014-08-13 15:28:56.451593 2014-08-13 18:06:06.924282 7f5c973a57a0 15 read_log missing 194416'15976 (194416'15975) modify cd9c333a/benchmark_data_storage0111-ib.pocstor_19103_object5433/head//24 by client.2441958.0:5434 2014-08-13 15:28:51.536921 2014-08-13 18:06:06.924319 7f5c973a57a0 15 read_log missing 194416'15974 (194416'15973) modify 7374ef3a/benchmark_data_storage0111-ib.pocstor_19103_object4541/head//24 by client.2441958.0:4542 2014-08-13 15:28:48.111350 2014-08-13 18:06:06.924365 7f5c973a57a0 15 read_log missing 194416'15972 (194416'15971) modify eee90b3a/benchmark_data_storage0111-ib.pocstor_19103_object4445/head//24 by client.2441958.0:4446 2014-08-13 15:28:47.660091 2014-08-13 18:06:06.924412 7f5c973a57a0 15 read_log missing 194416'15970 (194416'15969) modify da14bb3a/benchmark_data_storage0111-ib.pocstor_19103_object4013/head//24 by client.2441958.0:4014 2014-08-13 15:28:45.977701 2014-08-13 18:06:06.924451 7f5c973a57a0 15 read_log missing 194416'15968 (194416'15967) modify d3ddcb3a/benchmark_data_storage0111-ib.pocstor_19103_object2575/head//24 by client.2441958.0:2576 2014-08-13 15:28:40.467172 2014-08-13 18:06:06.924487 7f5c973a57a0 15 read_log missing 194416'15966 (194416'15965) modify e959b33a/benchmark_data_storage0111-ib.pocstor_19103_object2497/head//24 by client.2441958.0:2498 2014-08-13 15:28:40.168526 2014-08-13 18:06:06.924523 7f5c973a57a0 15 read_log missing 194416'15964 (194416'15963) modify 82c5533a/benchmark_data_storage0111-ib.pocstor_19103_object1497/head//24 by client.2441958.0:1498 2014-08-13 15:28:36.390188 2014-08-13 18:06:06.924559 7f5c973a57a0 15 read_log missing 194416'15962 (194416'15961) modify 13206f3a/benchmark_data_storage0111-ib.pocstor_19103_object1420/head//24 by client.2441958.0:1421 2014-08-13 15:28:36.105385 2014-08-13 18:06:06.924595 7f5c973a57a0 15 read_log missing 194416'15960 (194416'15959) modify bcd3c33a/benchmark_data_storage0111-ib.pocstor_19103_object885/head//24 by client.2441958.0:886 2014-08-13 15:28:34.018559 2014-08-13 18:06:06.924631 7f5c973a57a0 15 read_log missing 194416'15958 (194416'15957) modify fb75333a/benchmark_data_storage0111-ib.pocstor_19103_object174/head//24 by client.2441958.0:175 2014-08-13 15:28:30.846495 2014-08-13 18:06:06.924666 7f5c973a57a0 15 read_log missing 194397'15956 (194397'15955) modify 78f7ff3a/benchmark_data_storage0101-ib.pocstor_6819_object925/head//24 by client.2235069.0:926 2014-08-12 20:53:06.093281 2014-08-13 18:06:06.924702 7f5c973a57a0 15 read_log missing 194393'15954 (194393'15953) modify 17255f3a/benchmark_data_storage0111-ib.pocstor_24170_object17/head//24 by client.1064095.0:18 2014-08-12 20:47:25.949889 2014-08-13 18:06:06.924737 7f5c973a57a0 15 read_log missing 194383'15952 (194383'15951) modify 2454433a/benchmark_data_storage0111-ib.pocstor_25065_object7791/head//24 by client.2231190.0:7792 2014-08-12 20:34:36.968217 2014-08-13 18:06:06.924785 7f5c973a57a0 15 read_log missing 194383'15950 (194383'15949) modify 2804773a/benchmark_data_storage0111-ib.pocstor_25065_object6840/head//24 by client.2231190.0:6841 2014-08-12 20:34:10.839540 2014-08-13 18:06:06.924826 7f5c973a57a0 15 read_log missing 194383'15948 (194383'15947) modify b9e4ff3a/benchmark_data_storage0111-ib.pocstor_25065_object6821/head//24 by client.2231190.0:6822 2014-08-12 20:34:10.315611 2014-08-13 18:06:06.924863 7f5c973a57a0 15 read_log missing 194377'15946 (194377'15945) modify e12e433a/benchmark_data_storage0111-ib.pocstor_25065_object4181/head//24 by client.2231190.0:4182 2014-08-12 20:33:14.989295 2014-08-13 18:06:06.924903 7f5c973a57a0 15 read_log missing 194377'15944 (194377'15943) modify 108e33a/benchmark_data_storage0111-ib.pocstor_25065_object3694/head//24 by client.2231190.0:3695 2014-08-12 20:33:12.304340 2014-08-13 18:06:06.924940 7f5c973a57a0 15 read_log missing 194377'15942 (194377'15941) modify 3898df3a/benchmark_data_storage0111-ib.pocstor_25065_object1882/head//24 by client.2231190.0:1883 2014-08-12 20:33:04.731658 2014-08-13 18:06:06.924977 7f5c973a57a0 15 read_log missing 194370'15940 (194370'15939) modify 22c6cb3a/benchmark_data_storage0111-ib.pocstor_12495_object12465/head//24 by client.1508531.0:12466 2014-08-12 20:31:14.749585 2014-08-13 18:06:06.925023 7f5c973a57a0 15 read_log missing 194370'15938 (194370'15937) modify 6669233a/benchmark_data_storage0111-ib.pocstor_12495_object12207/head//24 by client.1508531.0:12208 2014-08-12 20:31:12.891369 2014-08-13 18:06:06.925061 7f5c973a57a0 15 read_log missing 194370'15936 (194370'15935) modify a4a17b3a/benchmark_data_storage0111-ib.pocstor_12495_object11429/head//24 by client.1508531.0:11430 2014-08-12 20:31:07.890091 2014-08-13 18:06:06.925097 7f5c973a57a0 15 read_log missing 194370'15934 (194370'15933) modify 7578ef3a/benchmark_data_storage0111-ib.pocstor_12495_object11402/head//24 by client.1508531.0:11403 2014-08-12 20:31:07.740071 2014-08-13 18:06:06.925133 7f5c973a57a0 15 read_log missing 194370'15932 (194370'15931) modify fc33073a/benchmark_data_storage0111-ib.pocstor_12495_object8316/head//24 by client.1508531.0:8317 2014-08-12 20:30:49.252693 2014-08-13 18:06:06.925183 7f5c973a57a0 15 read_log missing 194370'15930 (194370'15929) modify 749b3b3a/benchmark_data_storage0111-ib.pocstor_12495_object6486/head//24 by client.1508531.0:6487 2014-08-12 20:30:39.894743 2014-08-13 18:06:06.925239 7f5c973a57a0 15 read_log missing 194370'15928 (194370'15927) modify 914ca33a/benchmark_data_storage0111-ib.pocstor_12495_object6367/head//24 by client.1508531.0:6368 2014-08-12 20:30:39.326068 2014-08-13 18:06:06.925277 7f5c973a57a0 15 read_log missing 194370'15926 (194370'15925) modify 399dcf3a/benchmark_data_storage0111-ib.pocstor_12495_object5713/head//24 by client.1508531.0:5714 2014-08-12 20:30:36.411349 2014-08-13 18:06:06.925318 7f5c973a57a0 15 read_log missing 194370'15924 (194370'15923) modify 734af33a/benchmark_data_storage0111-ib.pocstor_12495_object4088/head//24 by client.1508531.0:4089 2014-08-12 20:30:29.617119 2014-08-13 18:06:06.925369 7f5c973a57a0 15 read_log missing 194370'15922 (194370'15921) modify 3913133a/benchmark_data_storage0111-ib.pocstor_12495_object1546/head//24 by client.1508531.0:1547 2014-08-12 20:30:20.183075 2014-08-13 18:06:06.925406 7f5c973a57a0 15 read_log missing 194370'15920 (194370'15919) modify 5d2d873a/benchmark_data_storage0111-ib.pocstor_12495_object391/head//24 by client.1508531.0:392 2014-08-12 20:30:15.862085 2014-08-13 18:06:06.925442 7f5c973a57a0 15 read_log missing 194217'15918 (194217'15917) modify 4db6f3a/benchmark_data_storage0111-ib.pocstor_39693_object433/head//24 by client.1487927.0:434 2014-08-12 17:25:05.605946 2014-08-13 18:06:06.925478 7f5c973a57a0 15 read_log missing 194217'15916 (194217'15915) modify d383b73a/benchmark_data_storage0111-ib.pocstor_39693_object306/head//24 by client.1487927.0:307 2014-08-12 17:25:04.489405 2014-08-13 18:06:06.925518 7f5c973a57a0 15 read_log missing 194217'15913 (194217'15913) modify 33a/hit_set_24.33a_archive_2014-08-12 15:48:57.821312_2014-08-12 17:25:04.496076/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:04.496306 2014-08-13 18:06:06.925530 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.925556 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=0 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive m=61] handle_loaded 2014-08-13 18:06:06.925573 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=0 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Initial 0.014891 0 0.000000 2014-08-13 18:06:06.925585 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=0 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Reset 2014-08-13 18:06:06.925594 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=0 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194487 2014-08-13 18:06:06.925604 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log((192834'14161,194422'16034], crt=194422'16034) 2014-08-13 18:06:06.925615 7f5c973a57a0 10 osd.4 194487 pgid 24.350 coll 24.350_head 2014-08-13 18:06:06.925767 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.350 2014-08-13 18:06:06.925781 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.925823 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.350(unlocked)] enter Initial 2014-08-13 18:06:06.925835 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.350(unlocked)] enter NotTrimming 2014-08-13 18:06:06.925977 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.926107 7f5c973a57a0 20 read_log 192834'15731 (192834'15722) delete a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17280 2014-08-11 15:27:29.003777 2014-08-13 18:06:06.926129 7f5c973a57a0 20 read_log 192834'15732 (192834'15201) clean bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17281 2014-08-11 15:27:29.162877 2014-08-13 18:06:06.926142 7f5c973a57a0 20 read_log 192834'15740 (192834'15732) delete bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17289 2014-08-11 15:27:29.439631 2014-08-13 18:06:06.926164 7f5c973a57a0 20 read_log 192834'15748 (192834'15209) clean edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17297 2014-08-11 15:27:29.539874 2014-08-13 18:06:06.926177 7f5c973a57a0 20 read_log 192834'15752 (0'0) promote 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17301 2014-08-11 15:27:29.798676 2014-08-13 18:06:06.926190 7f5c973a57a0 20 read_log 192834'15753 (192834'15752) modify 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by client.1883634.0:16886 2014-08-11 15:27:29.788807 2014-08-13 18:06:06.926203 7f5c973a57a0 20 read_log 192834'15764 (192834'15748) delete edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17313 2014-08-11 15:27:34.507816 2014-08-13 18:06:06.926215 7f5c973a57a0 20 read_log 192834'15766 (192834'15227) clean 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17315 2014-08-11 15:27:34.606858 2014-08-13 18:06:06.926227 7f5c973a57a0 20 read_log 192834'15767 (192834'15223) clean cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17316 2014-08-11 15:27:34.611485 2014-08-13 18:06:06.926240 7f5c973a57a0 20 read_log 192834'15768 (192834'15213) clean 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17317 2014-08-11 15:27:34.611810 2014-08-13 18:06:06.926256 7f5c973a57a0 20 read_log 192834'15774 (192834'15766) delete 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17323 2014-08-11 15:27:34.734901 2014-08-13 18:06:06.926269 7f5c973a57a0 20 read_log 192834'15778 (192834'15767) delete cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17327 2014-08-11 15:27:34.833712 2014-08-13 18:06:06.926281 7f5c973a57a0 20 read_log 192834'15779 (192834'15768) delete 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17328 2014-08-11 15:27:34.835683 2014-08-13 18:06:06.926294 7f5c973a57a0 20 read_log 192834'15806 (192834'15249) clean e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17354 2014-08-11 15:27:40.022394 2014-08-13 18:06:06.926306 7f5c973a57a0 20 read_log 192834'15815 (192834'15806) delete e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17363 2014-08-11 15:27:40.244569 2014-08-13 18:06:06.926318 7f5c973a57a0 20 read_log 192834'15823 (0'0) promote f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17371 2014-08-11 15:27:40.439075 2014-08-13 18:06:06.926344 7f5c973a57a0 20 read_log 192834'15824 (192834'15823) modify f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by client.1883634.0:18019 2014-08-11 15:27:40.430814 2014-08-13 18:06:06.926360 7f5c973a57a0 20 read_log 192834'15830 (192834'15346) clean b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17378 2014-08-11 15:27:45.395855 2014-08-13 18:06:06.926373 7f5c973a57a0 20 read_log 192834'15831 (192834'15294) clean 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17379 2014-08-11 15:27:45.396452 2014-08-13 18:06:06.926385 7f5c973a57a0 20 read_log 192834'15835 (192834'15830) delete b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17383 2014-08-11 15:27:45.421435 2014-08-13 18:06:06.926398 7f5c973a57a0 20 read_log 192834'15836 (192834'15831) delete 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17384 2014-08-11 15:27:45.425752 2014-08-13 18:06:06.926410 7f5c973a57a0 20 read_log 192834'15843 (192834'15348) clean 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17391 2014-08-11 15:27:45.721258 2014-08-13 18:06:06.926439 7f5c973a57a0 20 read_log 192834'15852 (0'0) promote dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:17400 2014-08-11 15:27:48.967829 2014-08-13 18:06:06.926453 7f5c973a57a0 20 read_log 192834'15853 (192834'15852) modify dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by client.1883634.0:19027 2014-08-11 15:27:48.958878 2014-08-13 18:06:06.926472 7f5c973a57a0 20 read_log 192834'15858 (192834'15843) delete 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17406 2014-08-11 15:27:50.523494 2014-08-13 18:06:06.926488 7f5c973a57a0 20 read_log 192834'15887 (192834'15436) clean 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17435 2014-08-11 15:27:55.795501 2014-08-13 18:06:06.926501 7f5c973a57a0 20 read_log 192834'15890 (192834'15438) clean 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17438 2014-08-11 15:27:55.878505 2014-08-13 18:06:06.926520 7f5c973a57a0 20 read_log 192834'15896 (192834'15887) delete 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17444 2014-08-11 15:27:55.900172 2014-08-13 18:06:06.926533 7f5c973a57a0 20 read_log 192834'15898 (192834'15890) delete 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17446 2014-08-11 15:27:55.912818 2014-08-13 18:06:06.926548 7f5c973a57a0 20 read_log 192834'15921 (0'0) promote 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:17469 2014-08-11 15:28:02.069080 2014-08-13 18:06:06.926561 7f5c973a57a0 20 read_log 192834'15922 (192834'15921) modify 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by client.1883634.0:20943 2014-08-11 15:28:02.060855 2014-08-13 18:06:06.926573 7f5c973a57a0 20 read_log 192834'15931 (0'0) promote d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:17479 2014-08-11 15:28:04.789308 2014-08-13 18:06:06.926586 7f5c973a57a0 20 read_log 192834'15932 (192834'15931) modify d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by client.1883634.0:21627 2014-08-11 15:28:04.780152 2014-08-13 18:06:06.926598 7f5c973a57a0 20 read_log 192834'15945 (192834'15549) clean 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17493 2014-08-11 15:28:06.398159 2014-08-13 18:06:06.926611 7f5c973a57a0 20 read_log 192834'15954 (192834'15945) delete 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17502 2014-08-11 15:28:06.440828 2014-08-13 18:06:06.926623 7f5c973a57a0 20 read_log 192834'15975 (192834'15563) clean 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17523 2014-08-11 15:28:11.579495 2014-08-13 18:06:06.926644 7f5c973a57a0 20 read_log 192834'15988 (192834'15665) clean 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17536 2014-08-11 15:28:16.628059 2014-08-13 18:06:06.926660 7f5c973a57a0 20 read_log 192834'15991 (192834'15975) delete 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17539 2014-08-11 15:28:16.634404 2014-08-13 18:06:06.926672 7f5c973a57a0 20 read_log 192834'15995 (192834'15988) delete 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17543 2014-08-11 15:28:16.735101 2014-08-13 18:06:06.926685 7f5c973a57a0 20 read_log 192834'16013 (0'0) promote b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17561 2014-08-11 15:28:18.564085 2014-08-13 18:06:06.926697 7f5c973a57a0 20 read_log 192834'16014 (192834'16013) modify b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by client.1883634.0:24214 2014-08-11 15:28:18.537958 2014-08-13 18:06:06.926709 7f5c973a57a0 20 read_log 192834'16019 (0'0) promote 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17567 2014-08-11 15:28:19.123685 2014-08-13 18:06:06.926721 7f5c973a57a0 20 read_log 192834'16020 (192834'16019) modify 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by client.1883634.0:24466 2014-08-11 15:28:19.087449 2014-08-13 18:06:06.926734 7f5c973a57a0 20 read_log 192834'16023 (0'0) promote 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17571 2014-08-11 15:28:19.396264 2014-08-13 18:06:06.926746 7f5c973a57a0 20 read_log 192834'16024 (192834'16023) modify 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by client.1883634.0:24650 2014-08-11 15:28:19.327856 2014-08-13 18:06:06.926758 7f5c973a57a0 20 read_log 192834'16031 (0'0) promote 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17579 2014-08-11 15:28:20.276662 2014-08-13 18:06:06.926770 7f5c973a57a0 20 read_log 192834'16032 (192834'16031) modify 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by client.1883634.0:25138 2014-08-11 15:28:20.252347 2014-08-13 18:06:06.926782 7f5c973a57a0 20 read_log 192834'16037 (0'0) promote a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17585 2014-08-11 15:28:21.539584 2014-08-13 18:06:06.926812 7f5c973a57a0 20 read_log 192834'16038 (192834'16037) modify a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by client.1883634.0:25531 2014-08-11 15:28:21.507809 2014-08-13 18:06:06.926828 7f5c973a57a0 20 read_log 192834'16039 (0'0) promote d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17587 2014-08-11 15:28:21.692880 2014-08-13 18:06:06.926840 7f5c973a57a0 20 read_log 192834'16040 (192834'16039) modify d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by client.1883634.0:25640 2014-08-11 15:28:21.664684 2014-08-13 18:06:06.926853 7f5c973a57a0 20 read_log 192834'16051 (192834'15675) clean bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17599 2014-08-11 15:28:22.512931 2014-08-13 18:06:06.926867 7f5c973a57a0 20 read_log 192834'16059 (0'0) promote bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17607 2014-08-11 15:28:23.444160 2014-08-13 18:06:06.926880 7f5c973a57a0 20 read_log 192834'16060 (192834'16059) modify bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by client.1883634.0:26684 2014-08-11 15:28:23.415652 2014-08-13 18:06:06.926894 7f5c973a57a0 20 read_log 192834'16067 (0'0) promote edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17615 2014-08-11 15:28:24.510669 2014-08-13 18:06:06.926907 7f5c973a57a0 20 read_log 192834'16068 (192834'16067) modify edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by client.1883634.0:27440 2014-08-11 15:28:24.471386 2014-08-13 18:06:06.926931 7f5c973a57a0 20 read_log 192834'16071 (0'0) promote 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17619 2014-08-11 15:28:24.775525 2014-08-13 18:06:06.926947 7f5c973a57a0 20 read_log 192834'16072 (192834'16071) modify 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by client.1883634.0:27648 2014-08-11 15:28:24.742592 2014-08-13 18:06:06.926959 7f5c973a57a0 20 read_log 192834'16081 (0'0) promote cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17629 2014-08-11 15:28:25.665640 2014-08-13 18:06:06.926974 7f5c973a57a0 20 read_log 192834'16082 (192834'16081) modify cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by client.1883634.0:28318 2014-08-11 15:28:25.640691 2014-08-13 18:06:06.926989 7f5c973a57a0 20 read_log 192834'16085 (0'0) promote 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17633 2014-08-11 15:28:25.877379 2014-08-13 18:06:06.927001 7f5c973a57a0 20 read_log 192834'16086 (192834'16085) modify 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by client.1883634.0:28465 2014-08-11 15:28:25.846626 2014-08-13 18:06:06.927016 7f5c973a57a0 20 read_log 192834'16100 (192834'16051) delete bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17648 2014-08-11 15:28:26.850528 2014-08-13 18:06:06.927028 7f5c973a57a0 20 read_log 192834'16111 (0'0) promote e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17659 2014-08-11 15:28:27.453644 2014-08-13 18:06:06.927042 7f5c973a57a0 20 read_log 192834'16112 (192834'16111) modify e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by client.1883634.0:29562 2014-08-11 15:28:27.428185 2014-08-13 18:06:06.927056 7f5c973a57a0 20 read_log 192834'16131 (0'0) promote 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:17679 2014-08-11 15:28:30.578850 2014-08-13 18:06:06.927068 7f5c973a57a0 20 read_log 192834'16132 (192834'16131) modify 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by client.1883634.0:31281 2014-08-11 15:28:30.210083 2014-08-13 18:06:06.927081 7f5c973a57a0 20 read_log 192834'16133 (0'0) promote b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17681 2014-08-11 15:28:30.608408 2014-08-13 18:06:06.927095 7f5c973a57a0 20 read_log 192834'16134 (192834'16133) modify b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by client.1883634.0:31465 2014-08-11 15:28:30.568941 2014-08-13 18:06:06.927107 7f5c973a57a0 20 read_log 192834'16135 (0'0) promote 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:17683 2014-08-11 15:28:30.689935 2014-08-13 18:06:06.927122 7f5c973a57a0 20 read_log 192834'16136 (192834'16135) modify 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by client.1883634.0:31524 2014-08-11 15:28:30.660602 2014-08-13 18:06:06.927134 7f5c973a57a0 20 read_log 192834'16155 (192834'15753) clean 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17703 2014-08-11 15:28:32.015113 2014-08-13 18:06:06.927157 7f5c973a57a0 20 read_log 192834'16157 (192834'16155) delete 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17705 2014-08-11 15:28:32.056882 2014-08-13 18:06:06.927173 7f5c973a57a0 20 read_log 192834'16173 (0'0) promote 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:17721 2014-08-11 15:28:33.106466 2014-08-13 18:06:06.927185 7f5c973a57a0 20 read_log 192834'16174 (192834'16173) modify 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by client.1883634.0:33375 2014-08-11 15:28:33.079063 2014-08-13 18:06:06.927198 7f5c973a57a0 20 read_log 192834'16175 (0'0) promote 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:17723 2014-08-11 15:28:33.157308 2014-08-13 18:06:06.927218 7f5c973a57a0 20 read_log 192834'16176 (192834'16175) modify 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by client.1883634.0:33407 2014-08-11 15:28:33.129969 2014-08-13 18:06:06.927233 7f5c973a57a0 20 read_log 192834'16195 (0'0) promote 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:17743 2014-08-11 15:28:36.714381 2014-08-13 18:06:06.927248 7f5c973a57a0 20 read_log 192834'16196 (192834'16195) modify 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by client.1883634.0:35471 2014-08-11 15:28:36.685448 2014-08-13 18:06:06.927263 7f5c973a57a0 20 read_log 192834'16222 (0'0) promote 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:17770 2014-08-11 15:28:37.777287 2014-08-13 18:06:06.927275 7f5c973a57a0 20 read_log 192834'16223 (192834'16222) modify 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by client.1883634.0:36204 2014-08-11 15:28:37.730189 2014-08-13 18:06:06.927288 7f5c973a57a0 20 read_log 192834'16236 (0'0) promote 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:17784 2014-08-11 15:28:39.712284 2014-08-13 18:06:06.927300 7f5c973a57a0 20 read_log 192834'16237 (192834'16236) modify 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by client.1883634.0:37732 2014-08-11 15:28:39.686029 2014-08-13 18:06:06.927312 7f5c973a57a0 20 read_log 192834'16244 (0'0) promote bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:17792 2014-08-11 15:28:40.679352 2014-08-13 18:06:06.927325 7f5c973a57a0 20 read_log 192834'16245 (192834'16244) modify bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by client.1883634.0:38455 2014-08-11 15:28:40.655188 2014-08-13 18:06:06.927343 7f5c973a57a0 20 read_log 192834'16260 (192834'15824) clean f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17808 2014-08-11 15:28:42.902141 2014-08-13 18:06:06.927356 7f5c973a57a0 20 read_log 192834'16263 (192834'16260) delete f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17811 2014-08-11 15:28:42.928481 2014-08-13 18:06:06.927369 7f5c973a57a0 20 read_log 192834'16269 (0'0) promote 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:17817 2014-08-11 15:28:43.395187 2014-08-13 18:06:06.927381 7f5c973a57a0 20 read_log 192834'16270 (192834'16269) modify 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by client.1883634.0:40210 2014-08-11 15:28:43.367956 2014-08-13 18:06:06.927396 7f5c973a57a0 20 read_log 192834'16291 (0'0) promote f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:17839 2014-08-11 15:28:44.953948 2014-08-13 18:06:06.927418 7f5c973a57a0 20 read_log 192834'16292 (192834'16291) modify f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by client.1883634.0:41343 2014-08-11 15:28:44.926388 2014-08-13 18:06:06.927435 7f5c973a57a0 20 read_log 192834'16299 (192834'15853) modify dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by client.1883634.0:42351 2014-08-11 15:28:46.216382 2014-08-13 18:06:06.927454 7f5c973a57a0 20 read_log 192834'16312 (192834'15922) modify 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by client.1883634.0:44267 2014-08-11 15:28:46.810083 2014-08-13 18:06:06.927475 7f5c973a57a0 20 read_log 192834'16317 (192834'15932) modify d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by client.1883634.0:44951 2014-08-11 15:28:47.100579 2014-08-13 18:06:06.927488 7f5c973a57a0 20 read_log 192834'16347 (192834'16014) clean b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17895 2014-08-11 15:29:18.702713 2014-08-13 18:06:06.927501 7f5c973a57a0 20 read_log 192834'16350 (192834'16038) clean a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17898 2014-08-11 15:29:23.722519 2014-08-13 18:06:06.927545 7f5c973a57a0 20 read_log 192834'16351 (192834'16032) clean 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17899 2014-08-11 15:29:23.722816 2014-08-13 18:06:06.927571 7f5c973a57a0 20 read_log 192834'16354 (192834'16060) clean bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17902 2014-08-11 15:29:23.744743 2014-08-13 18:06:06.927587 7f5c973a57a0 20 read_log 192834'16355 (192834'16020) clean 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17903 2014-08-11 15:29:23.752500 2014-08-13 18:06:06.927599 7f5c973a57a0 20 read_log 192834'16356 (192834'16024) clean 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17904 2014-08-11 15:29:23.756378 2014-08-13 18:06:06.927612 7f5c973a57a0 20 read_log 192834'16357 (192834'16040) clean d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17905 2014-08-11 15:29:23.760451 2014-08-13 18:06:06.927629 7f5c973a57a0 20 read_log 192834'16359 (192834'16347) delete b5007f50/benchmark_data_storage0111-ib.pocstor_41526_object889/head//24 by osd.335.0:17907 2014-08-11 15:29:23.768252 2014-08-13 18:06:06.927652 7f5c973a57a0 20 read_log 192834'16371 (192834'16350) delete a28a3350/benchmark_data_storage0111-ib.pocstor_41526_object2206/head//24 by osd.335.0:17919 2014-08-11 15:29:23.825422 2014-08-13 18:06:06.927668 7f5c973a57a0 20 read_log 192834'16372 (192834'16351) delete 2d046350/benchmark_data_storage0111-ib.pocstor_41526_object1813/head//24 by osd.335.0:17920 2014-08-11 15:29:23.826554 2014-08-13 18:06:06.927681 7f5c973a57a0 20 read_log 192834'16375 (192834'16354) delete bdb2750/benchmark_data_storage0111-ib.pocstor_41526_object3359/head//24 by osd.335.0:17923 2014-08-11 15:29:23.837205 2014-08-13 18:06:06.927693 7f5c973a57a0 20 read_log 192834'16376 (192834'16355) delete 13900b50/benchmark_data_storage0111-ib.pocstor_41526_object1141/head//24 by osd.335.0:17924 2014-08-11 15:29:23.849995 2014-08-13 18:06:06.927705 7f5c973a57a0 20 read_log 192834'16377 (192834'16356) delete 8c423b50/benchmark_data_storage0111-ib.pocstor_41526_object1325/head//24 by osd.335.0:17925 2014-08-11 15:29:23.850494 2014-08-13 18:06:06.927718 7f5c973a57a0 20 read_log 192834'16378 (192834'16357) delete d2d2cb50/benchmark_data_storage0111-ib.pocstor_41526_object2315/head//24 by osd.335.0:17926 2014-08-11 15:29:23.854583 2014-08-13 18:06:06.927730 7f5c973a57a0 20 read_log 192834'16392 (192834'16086) clean 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17940 2014-08-11 15:29:29.013570 2014-08-13 18:06:06.927742 7f5c973a57a0 20 read_log 192834'16396 (192834'16082) clean cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17944 2014-08-11 15:29:29.032013 2014-08-13 18:06:06.927755 7f5c973a57a0 20 read_log 192834'16397 (192834'16068) clean edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17945 2014-08-11 15:29:29.036095 2014-08-13 18:06:06.927767 7f5c973a57a0 20 read_log 192834'16399 (192834'16072) clean 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17947 2014-08-11 15:29:29.046468 2014-08-13 18:06:06.927780 7f5c973a57a0 20 read_log 192834'16405 (192834'16112) clean e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17953 2014-08-11 15:29:29.064153 2014-08-13 18:06:06.927792 7f5c973a57a0 20 read_log 192834'16421 (192834'16392) delete 8d452350/benchmark_data_storage0111-ib.pocstor_41526_object5140/head//24 by osd.335.0:17969 2014-08-11 15:29:29.129482 2014-08-13 18:06:06.927804 7f5c973a57a0 20 read_log 192834'16426 (192834'16396) delete cd205750/benchmark_data_storage0111-ib.pocstor_41526_object4993/head//24 by osd.335.0:17974 2014-08-11 15:29:29.139237 2014-08-13 18:06:06.927816 7f5c973a57a0 20 read_log 192834'16427 (192834'16397) delete edbd8750/benchmark_data_storage0111-ib.pocstor_41526_object4115/head//24 by osd.335.0:17975 2014-08-11 15:29:29.140315 2014-08-13 18:06:06.927840 7f5c973a57a0 20 read_log 192834'16428 (192834'16399) delete 21f6a750/benchmark_data_storage0111-ib.pocstor_41526_object4323/head//24 by osd.335.0:17976 2014-08-11 15:29:29.143984 2014-08-13 18:06:06.927856 7f5c973a57a0 20 read_log 192834'16434 (192834'16405) delete e3a4fb50/benchmark_data_storage0111-ib.pocstor_41526_object6237/head//24 by osd.335.0:17982 2014-08-11 15:29:29.156572 2014-08-13 18:06:06.927868 7f5c973a57a0 20 read_log 192834'16450 (192834'16134) clean b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:17998 2014-08-11 15:29:34.233221 2014-08-13 18:06:06.927881 7f5c973a57a0 20 read_log 192834'16453 (192834'16132) clean 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:18001 2014-08-11 15:29:34.243419 2014-08-13 18:06:06.927893 7f5c973a57a0 20 read_log 192834'16454 (192834'16174) clean 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:18002 2014-08-11 15:29:34.244568 2014-08-13 18:06:06.927905 7f5c973a57a0 20 read_log 192834'16457 (192834'16136) clean 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:18005 2014-08-11 15:29:34.254779 2014-08-13 18:06:06.927917 7f5c973a57a0 20 read_log 192834'16459 (192834'16176) clean 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:18007 2014-08-11 15:29:34.262108 2014-08-13 18:06:06.927930 7f5c973a57a0 20 read_log 192834'16472 (192834'16450) delete b5b28350/benchmark_data_storage0111-ib.pocstor_41526_object8140/head//24 by osd.335.0:18020 2014-08-11 15:29:34.304977 2014-08-13 18:06:06.927942 7f5c973a57a0 20 read_log 192834'16475 (192834'16453) delete 60e24750/benchmark_data_storage0111-ib.pocstor_41526_object7956/head//24 by osd.335.0:18023 2014-08-11 15:29:34.314487 2014-08-13 18:06:06.927954 7f5c973a57a0 20 read_log 192834'16476 (192834'16454) delete 4495e750/benchmark_data_storage0111-ib.pocstor_41526_object10050/head//24 by osd.335.0:18024 2014-08-11 15:29:34.317293 2014-08-13 18:06:06.927966 7f5c973a57a0 20 read_log 192834'16481 (192834'16457) delete 614b5b50/benchmark_data_storage0111-ib.pocstor_41526_object8199/head//24 by osd.335.0:18029 2014-08-11 15:29:34.326493 2014-08-13 18:06:06.927979 7f5c973a57a0 20 read_log 192834'16483 (192834'16459) delete 3c974f50/benchmark_data_storage0111-ib.pocstor_41526_object10082/head//24 by osd.335.0:18031 2014-08-11 15:29:34.332875 2014-08-13 18:06:06.927991 7f5c973a57a0 20 read_log 192834'16500 (192834'16196) clean 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:18048 2014-08-11 15:29:39.417180 2014-08-13 18:06:06.928004 7f5c973a57a0 20 read_log 192834'16503 (192834'16223) clean 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:18051 2014-08-11 15:29:39.427162 2014-08-13 18:06:06.928016 7f5c973a57a0 20 read_log 192834'16514 (192834'16500) delete 79592750/benchmark_data_storage0111-ib.pocstor_41526_object12146/head//24 by osd.335.0:18062 2014-08-11 15:29:39.459588 2014-08-13 18:06:06.928028 7f5c973a57a0 20 read_log 192834'16517 (192834'16503) delete 81f2f50/benchmark_data_storage0111-ib.pocstor_41526_object12879/head//24 by osd.335.0:18065 2014-08-11 15:29:39.469998 2014-08-13 18:06:06.928063 7f5c973a57a0 20 read_log 192834'16534 (192834'16237) clean 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:18082 2014-08-11 15:29:44.531384 2014-08-13 18:06:06.928077 7f5c973a57a0 20 read_log 192834'16539 (192834'16270) clean 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:18087 2014-08-11 15:29:44.548303 2014-08-13 18:06:06.928091 7f5c973a57a0 20 read_log 192834'16540 (192834'16245) clean bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:18088 2014-08-11 15:29:44.549171 2014-08-13 18:06:06.928103 7f5c973a57a0 20 read_log 192834'16555 (192834'16534) delete 63a8a350/benchmark_data_storage0111-ib.pocstor_41526_object14407/head//24 by osd.335.0:18103 2014-08-11 15:29:44.594759 2014-08-13 18:06:06.928124 7f5c973a57a0 20 read_log 192834'16560 (192834'16539) delete 81913b50/benchmark_data_storage0111-ib.pocstor_41526_object16885/head//24 by osd.335.0:18108 2014-08-11 15:29:44.606006 2014-08-13 18:06:06.928139 7f5c973a57a0 20 read_log 192834'16561 (192834'16540) delete bbed9b50/benchmark_data_storage0111-ib.pocstor_41526_object15130/head//24 by osd.335.0:18109 2014-08-11 15:29:44.607937 2014-08-13 18:06:06.928159 7f5c973a57a0 20 read_log 192834'16584 (192834'16317) clean d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:18132 2014-08-11 15:29:49.663337 2014-08-13 18:06:06.928175 7f5c973a57a0 20 read_log 192834'16588 (192834'16312) clean 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:18136 2014-08-11 15:29:49.670715 2014-08-13 18:06:06.928187 7f5c973a57a0 20 read_log 192834'16590 (192834'16292) clean f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:18138 2014-08-11 15:29:49.675658 2014-08-13 18:06:06.928217 7f5c973a57a0 20 read_log 192834'16602 (192834'16584) delete d39a350/benchmark_data_storage0111-ib.pocstor_41526_object21626/head//24 by osd.335.0:18150 2014-08-11 15:29:49.697567 2014-08-13 18:06:06.928232 7f5c973a57a0 20 read_log 192834'16606 (192834'16588) delete 8316d750/benchmark_data_storage0111-ib.pocstor_41526_object20942/head//24 by osd.335.0:18154 2014-08-11 15:29:49.703364 2014-08-13 18:06:06.928247 7f5c973a57a0 20 read_log 192834'16607 (192834'16590) delete f1f3e750/benchmark_data_storage0111-ib.pocstor_41526_object18018/head//24 by osd.335.0:18155 2014-08-11 15:29:49.704323 2014-08-13 18:06:06.928260 7f5c973a57a0 20 read_log 192834'16609 (192834'16299) clean dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:18157 2014-08-11 15:29:49.706653 2014-08-13 18:06:06.928273 7f5c973a57a0 20 read_log 192834'16614 (192834'16609) delete dc57bb50/benchmark_data_storage0111-ib.pocstor_41526_object19026/head//24 by osd.335.0:18162 2014-08-11 15:29:49.711594 2014-08-13 18:06:06.928286 7f5c973a57a0 20 read_log 192888'16666 (0'0) promote b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by osd.51.0:1972 2014-08-11 16:45:08.002090 2014-08-13 18:06:06.928302 7f5c973a57a0 20 read_log 192888'16667 (192888'16666) modify b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.1315259.0:3340 2014-08-11 16:45:07.995057 2014-08-13 18:06:06.928315 7f5c973a57a0 20 read_log 192888'16672 (0'0) promote 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by osd.51.0:1990 2014-08-11 16:45:12.396800 2014-08-13 18:06:06.928328 7f5c973a57a0 20 read_log 192888'16673 (192888'16672) modify 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.1315259.0:4532 2014-08-11 16:45:12.388186 2014-08-13 18:06:06.928340 7f5c973a57a0 20 read_log 192888'16682 (0'0) promote c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by osd.51.0:2014 2014-08-11 16:45:21.240689 2014-08-13 18:06:06.928357 7f5c973a57a0 20 read_log 192888'16683 (192888'16682) modify c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.1315259.0:6956 2014-08-11 16:45:21.233950 2014-08-13 18:06:06.928369 7f5c973a57a0 20 read_log 192888'16684 (0'0) promote 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by osd.51.0:2022 2014-08-11 16:45:22.099679 2014-08-13 18:06:06.928402 7f5c973a57a0 20 read_log 192888'16685 (192888'16684) modify 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.1315259.0:7194 2014-08-11 16:45:22.093008 2014-08-13 18:06:06.928428 7f5c973a57a0 20 read_log 192888'16686 (0'0) promote 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by osd.51.0:2032 2014-08-11 16:45:24.280000 2014-08-13 18:06:06.928453 7f5c973a57a0 20 read_log 192888'16687 (192888'16686) modify 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.1315259.0:7789 2014-08-11 16:45:24.271155 2014-08-13 18:06:06.928479 7f5c973a57a0 20 read_log 192888'16690 (0'0) promote 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by osd.51.0:2036 2014-08-11 16:45:25.035265 2014-08-13 18:06:06.928495 7f5c973a57a0 20 read_log 192888'16691 (192888'16690) modify 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.1315259.0:7998 2014-08-11 16:45:25.026310 2014-08-13 18:06:06.928508 7f5c973a57a0 20 read_log 192888'16694 (0'0) promote 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by osd.51.0:2048 2014-08-11 16:45:28.962813 2014-08-13 18:06:06.928530 7f5c973a57a0 20 read_log 192888'16695 (192888'16694) modify 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.1315259.0:9026 2014-08-11 16:45:28.955203 2014-08-13 18:06:06.928543 7f5c973a57a0 20 read_log 192888'16696 (0'0) promote 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by osd.51.0:2056 2014-08-11 16:45:30.343818 2014-08-13 18:06:06.928555 7f5c973a57a0 20 read_log 192888'16697 (192888'16696) modify 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.1315259.0:9399 2014-08-11 16:45:30.334562 2014-08-13 18:06:06.928568 7f5c973a57a0 20 read_log 192888'16698 (0'0) promote 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by osd.51.0:2064 2014-08-11 16:45:31.963079 2014-08-13 18:06:06.928584 7f5c973a57a0 20 read_log 192888'16699 (192888'16698) modify 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.1315259.0:9804 2014-08-11 16:45:31.955732 2014-08-13 18:06:06.928611 7f5c973a57a0 20 read_log 192888'16702 (0'0) promote 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by osd.51.0:2072 2014-08-11 16:45:35.017465 2014-08-13 18:06:06.928624 7f5c973a57a0 20 read_log 192888'16703 (192888'16702) modify 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.1315259.0:10601 2014-08-11 16:45:35.010589 2014-08-13 18:06:06.928636 7f5c973a57a0 20 read_log 192888'16712 (0'0) promote 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by osd.51.0:2096 2014-08-11 16:45:41.156376 2014-08-13 18:06:06.928657 7f5c973a57a0 20 read_log 192888'16713 (192888'16712) modify 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.1315259.0:12120 2014-08-11 16:45:41.146355 2014-08-13 18:06:06.928670 7f5c973a57a0 20 read_log 192888'16718 (0'0) promote e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by osd.51.0:2140 2014-08-11 16:45:54.498907 2014-08-13 18:06:06.928682 7f5c973a57a0 20 read_log 192888'16719 (192888'16718) modify e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.1315259.0:15488 2014-08-11 16:45:54.490574 2014-08-13 18:06:06.928694 7f5c973a57a0 20 read_log 192888'16720 (0'0) promote 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by osd.51.0:2144 2014-08-11 16:45:56.369899 2014-08-13 18:06:06.928707 7f5c973a57a0 20 read_log 192888'16721 (192888'16720) modify 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.1315259.0:15985 2014-08-11 16:45:56.363154 2014-08-13 18:06:06.928721 7f5c973a57a0 20 read_log 192888'16724 (0'0) promote 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by osd.51.0:2158 2014-08-11 16:45:58.139895 2014-08-13 18:06:06.928733 7f5c973a57a0 20 read_log 192888'16725 (192888'16724) modify 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.1315259.0:16452 2014-08-11 16:45:58.131248 2014-08-13 18:06:06.928745 7f5c973a57a0 20 read_log 192888'16726 (0'0) promote 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by osd.51.0:2164 2014-08-11 16:46:02.279688 2014-08-13 18:06:06.928768 7f5c973a57a0 20 read_log 192888'16727 (192888'16726) modify 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.1315259.0:17492 2014-08-11 16:46:02.271124 2014-08-13 18:06:06.928784 7f5c973a57a0 20 read_log 192888'16734 (0'0) promote 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by osd.51.0:2202 2014-08-11 16:46:15.129255 2014-08-13 18:06:06.928797 7f5c973a57a0 20 read_log 192888'16735 (192888'16734) modify 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.1315259.0:20797 2014-08-11 16:46:15.121829 2014-08-13 18:06:06.928809 7f5c973a57a0 20 read_log 192888'16736 (0'0) promote 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by osd.51.0:2204 2014-08-11 16:46:16.425481 2014-08-13 18:06:06.928821 7f5c973a57a0 20 read_log 192888'16737 (192888'16736) modify 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.1315259.0:21150 2014-08-11 16:46:16.418800 2014-08-13 18:06:06.928834 7f5c973a57a0 20 read_log 192888'16750 (0'0) promote a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by osd.51.0:2242 2014-08-11 16:46:30.127470 2014-08-13 18:06:06.928846 7f5c973a57a0 20 read_log 192888'16751 (192888'16750) modify a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.1315259.0:24653 2014-08-11 16:46:30.119728 2014-08-13 18:06:06.928858 7f5c973a57a0 20 read_log 192888'16754 (0'0) promote f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by osd.51.0:2246 2014-08-11 16:46:31.223415 2014-08-13 18:06:06.928870 7f5c973a57a0 20 read_log 192888'16755 (192888'16754) modify f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.1315259.0:24921 2014-08-11 16:46:31.215193 2014-08-13 18:06:06.928882 7f5c973a57a0 20 read_log 192888'16758 (0'0) promote c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by osd.51.0:2258 2014-08-11 16:46:37.907598 2014-08-13 18:06:06.928896 7f5c973a57a0 20 read_log 192888'16759 (192888'16758) modify c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.1315259.0:26504 2014-08-11 16:46:37.900177 2014-08-13 18:06:06.928908 7f5c973a57a0 20 read_log 192888'16760 (0'0) promote 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by osd.51.0:2260 2014-08-11 16:46:38.714094 2014-08-13 18:06:06.928920 7f5c973a57a0 20 read_log 192888'16761 (192888'16760) modify 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.1315259.0:26710 2014-08-11 16:46:38.706678 2014-08-13 18:06:06.928932 7f5c973a57a0 20 read_log 192888'16762 (0'0) promote b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by osd.51.0:2262 2014-08-11 16:46:40.730616 2014-08-13 18:06:06.928944 7f5c973a57a0 20 read_log 192888'16763 (192888'16762) modify b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.1315259.0:27170 2014-08-11 16:46:40.722748 2014-08-13 18:06:06.928956 7f5c973a57a0 20 read_log 192888'16764 (0'0) promote 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by osd.51.0:2266 2014-08-11 16:46:43.143069 2014-08-13 18:06:06.928968 7f5c973a57a0 20 read_log 192888'16765 (192888'16764) modify 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.1315259.0:27790 2014-08-11 16:46:43.135649 2014-08-13 18:06:06.928983 7f5c973a57a0 20 read_log 192888'16766 (0'0) promote 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by osd.51.0:2284 2014-08-11 16:46:51.604745 2014-08-13 18:06:06.928995 7f5c973a57a0 20 read_log 192888'16767 (192888'16766) modify 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.1315259.0:29863 2014-08-11 16:46:51.597373 2014-08-13 18:06:06.929010 7f5c973a57a0 20 read_log 192888'16768 (0'0) promote 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by osd.51.0:2286 2014-08-11 16:46:51.793641 2014-08-13 18:06:06.929030 7f5c973a57a0 20 read_log 192888'16769 (192888'16768) modify 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.1315259.0:29913 2014-08-11 16:46:51.785909 2014-08-13 18:06:06.929044 7f5c973a57a0 20 read_log 192888'16770 (0'0) promote f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by osd.51.0:2290 2014-08-11 16:46:52.936303 2014-08-13 18:06:06.929057 7f5c973a57a0 20 read_log 192888'16771 (192888'16770) modify f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.1315259.0:30222 2014-08-11 16:46:52.927785 2014-08-13 18:06:06.929071 7f5c973a57a0 20 read_log 192888'16775 (192888'16667) modify b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.1315259.0:34067 2014-08-11 16:46:56.796437 2014-08-13 18:06:06.929083 7f5c973a57a0 20 read_log 192888'16778 (192888'16673) modify 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.1315259.0:35259 2014-08-11 16:46:57.116479 2014-08-13 18:06:06.929098 7f5c973a57a0 20 read_log 192888'16783 (192888'16683) modify c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.1315259.0:37683 2014-08-11 16:46:57.747663 2014-08-13 18:06:06.929110 7f5c973a57a0 20 read_log 192888'16784 (192888'16685) modify 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.1315259.0:37921 2014-08-11 16:46:57.810745 2014-08-13 18:06:06.929124 7f5c973a57a0 20 read_log 192888'16785 (192888'16687) modify 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.1315259.0:38516 2014-08-11 16:46:57.970062 2014-08-13 18:06:06.929137 7f5c973a57a0 20 read_log 192888'16787 (192888'16691) modify 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.1315259.0:38725 2014-08-11 16:46:58.029597 2014-08-13 18:06:06.929155 7f5c973a57a0 20 read_log 192888'16789 (192888'16695) modify 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.1315259.0:39753 2014-08-11 16:46:58.294443 2014-08-13 18:06:06.929170 7f5c973a57a0 20 read_log 192888'16790 (192888'16697) modify 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.1315259.0:40126 2014-08-11 16:46:58.395446 2014-08-13 18:06:06.929182 7f5c973a57a0 20 read_log 192888'16791 (192888'16699) modify 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.1315259.0:40531 2014-08-11 16:46:58.515144 2014-08-13 18:06:06.929195 7f5c973a57a0 20 read_log 192888'16793 (192888'16703) modify 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.1315259.0:41328 2014-08-11 16:46:58.718115 2014-08-13 18:06:06.929207 7f5c973a57a0 20 read_log 192888'16798 (192888'16713) modify 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.1315259.0:42847 2014-08-11 16:46:59.121365 2014-08-13 18:06:06.929221 7f5c973a57a0 20 read_log 192888'16801 (192888'16719) modify e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.1315259.0:46215 2014-08-11 16:47:00.094092 2014-08-13 18:06:06.929235 7f5c973a57a0 20 read_log 192888'16802 (192888'16721) modify 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.1315259.0:46712 2014-08-11 16:47:00.237145 2014-08-13 18:06:06.929249 7f5c973a57a0 20 read_log 192888'16804 (192888'16725) modify 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.1315259.0:47179 2014-08-11 16:47:00.374867 2014-08-13 18:06:06.929261 7f5c973a57a0 20 read_log 192888'16805 (192888'16727) modify 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.1315259.0:48219 2014-08-11 16:47:00.701502 2014-08-13 18:06:06.929274 7f5c973a57a0 20 read_log 192888'16809 (192888'16735) modify 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.1315259.0:51524 2014-08-11 16:47:01.705344 2014-08-13 18:06:06.929289 7f5c973a57a0 20 read_log 192888'16810 (192888'16737) modify 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.1315259.0:51877 2014-08-11 16:47:01.809213 2014-08-13 18:06:06.929311 7f5c973a57a0 20 read_log 192888'16817 (192888'16751) modify a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.1315259.0:55380 2014-08-11 16:47:02.895538 2014-08-13 18:06:06.929327 7f5c973a57a0 20 read_log 192888'16819 (192888'16755) modify f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.1315259.0:55648 2014-08-11 16:47:02.973488 2014-08-13 18:06:06.929339 7f5c973a57a0 20 read_log 192888'16821 (192888'16759) modify c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.1315259.0:57231 2014-08-11 16:47:03.448915 2014-08-13 18:06:06.929355 7f5c973a57a0 20 read_log 192888'16822 (192888'16761) modify 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.1315259.0:57437 2014-08-11 16:47:03.512625 2014-08-13 18:06:06.929368 7f5c973a57a0 20 read_log 192888'16823 (192888'16763) modify b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.1315259.0:57897 2014-08-11 16:47:03.643324 2014-08-13 18:06:06.929381 7f5c973a57a0 20 read_log 192888'16824 (192888'16765) modify 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.1315259.0:58517 2014-08-11 16:47:03.825459 2014-08-13 18:06:06.929393 7f5c973a57a0 20 read_log 192888'16825 (192888'16767) modify 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.1315259.0:60590 2014-08-11 16:47:04.484847 2014-08-13 18:06:06.929410 7f5c973a57a0 20 read_log 192888'16826 (192888'16769) modify 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.1315259.0:60640 2014-08-11 16:47:04.499732 2014-08-13 18:06:06.929423 7f5c973a57a0 20 read_log 192888'16827 (192888'16771) modify f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.1315259.0:60949 2014-08-11 16:47:04.589670 2014-08-13 18:06:06.929436 7f5c973a57a0 20 read_log 192888'16837 (0'0) promote 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by osd.51.0:2490 2014-08-11 17:05:17.443839 2014-08-13 18:06:06.929452 7f5c973a57a0 20 read_log 192888'16838 (192888'16837) modify 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.942868.0:593 2014-08-11 17:05:17.436652 2014-08-13 18:06:06.929473 7f5c973a57a0 20 read_log 192888'16843 (0'0) promote 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by osd.51.0:2496 2014-08-11 17:05:18.461911 2014-08-13 18:06:06.929489 7f5c973a57a0 20 read_log 192888'16844 (192888'16843) modify 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.942868.0:836 2014-08-11 17:05:18.453230 2014-08-13 18:06:06.929502 7f5c973a57a0 20 read_log 192888'16845 (0'0) promote 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by osd.51.0:2498 2014-08-11 17:05:19.805430 2014-08-13 18:06:06.929525 7f5c973a57a0 20 read_log 192888'16846 (192888'16845) modify 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.942868.0:1159 2014-08-11 17:05:19.797346 2014-08-13 18:06:06.929541 7f5c973a57a0 20 read_log 192888'16847 (0'0) promote 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by osd.51.0:2500 2014-08-11 17:05:19.981263 2014-08-13 18:06:06.929553 7f5c973a57a0 20 read_log 192888'16848 (192888'16847) modify 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.942868.0:1204 2014-08-11 17:05:19.972770 2014-08-13 18:06:06.929575 7f5c973a57a0 20 read_log 192888'16851 (0'0) promote e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by osd.51.0:2521 2014-08-11 17:05:26.268415 2014-08-13 18:06:06.929588 7f5c973a57a0 20 read_log 192888'16852 (192888'16851) modify e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.942868.0:2790 2014-08-11 17:05:26.261159 2014-08-13 18:06:06.929634 7f5c973a57a0 20 read_log 192888'16853 (0'0) promote 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by osd.51.0:2525 2014-08-11 17:05:27.140074 2014-08-13 18:06:06.929679 7f5c973a57a0 20 read_log 192888'16854 (192888'16853) modify 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.942868.0:3026 2014-08-11 17:05:27.132189 2014-08-13 18:06:06.929696 7f5c973a57a0 20 read_log 192888'16855 (0'0) promote 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by osd.51.0:2527 2014-08-11 17:05:30.581622 2014-08-13 18:06:06.929718 7f5c973a57a0 20 read_log 192888'16856 (192888'16855) modify 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.942868.0:3958 2014-08-11 17:05:30.574356 2014-08-13 18:06:06.929734 7f5c973a57a0 20 read_log 192888'16861 (0'0) promote a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by osd.51.0:2547 2014-08-11 17:05:35.802422 2014-08-13 18:06:06.929746 7f5c973a57a0 20 read_log 192888'16862 (192888'16861) modify a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.942868.0:5315 2014-08-11 17:05:35.794790 2014-08-13 18:06:06.929758 7f5c973a57a0 20 read_log 192888'16863 (0'0) promote 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by osd.51.0:2551 2014-08-11 17:05:36.790258 2014-08-13 18:06:06.929771 7f5c973a57a0 20 read_log 192888'16864 (192888'16863) modify 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.942868.0:5580 2014-08-11 17:05:36.782699 2014-08-13 18:06:06.929783 7f5c973a57a0 20 read_log 192888'16865 (0'0) promote 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by osd.51.0:2585 2014-08-11 17:05:46.010203 2014-08-13 18:06:06.929795 7f5c973a57a0 20 read_log 192888'16866 (192888'16865) modify 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.942868.0:8051 2014-08-11 17:05:46.002342 2014-08-13 18:06:06.929810 7f5c973a57a0 20 read_log 192888'16875 (0'0) promote c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by osd.51.0:2603 2014-08-11 17:05:51.030821 2014-08-13 18:06:06.929823 7f5c973a57a0 20 read_log 192888'16876 (192888'16875) modify c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.942868.0:9399 2014-08-11 17:05:51.021561 2014-08-13 18:06:06.929835 7f5c973a57a0 20 read_log 192888'16881 (0'0) promote db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by osd.51.0:2621 2014-08-11 17:05:56.921405 2014-08-13 18:06:06.929848 7f5c973a57a0 20 read_log 192888'16882 (192888'16881) modify db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.942868.0:11035 2014-08-11 17:05:56.914126 2014-08-13 18:06:06.929860 7f5c973a57a0 20 read_log 192888'16887 (0'0) promote 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by osd.51.0:2629 2014-08-11 17:06:00.300999 2014-08-13 18:06:06.929872 7f5c973a57a0 20 read_log 192888'16888 (192888'16887) modify 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.942868.0:11942 2014-08-11 17:06:00.295175 2014-08-13 18:06:06.929885 7f5c973a57a0 20 read_log 192888'16893 (0'0) promote f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by osd.51.0:2649 2014-08-11 17:06:04.214835 2014-08-13 18:06:06.929897 7f5c973a57a0 20 read_log 192888'16894 (192888'16893) modify f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.942868.0:13007 2014-08-11 17:06:04.207175 2014-08-13 18:06:06.929909 7f5c973a57a0 20 read_log 192888'16895 (0'0) promote d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by osd.51.0:2651 2014-08-11 17:06:04.270043 2014-08-13 18:06:06.929921 7f5c973a57a0 20 read_log 192888'16896 (192888'16895) modify d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.942868.0:13022 2014-08-11 17:06:04.261899 2014-08-13 18:06:06.929934 7f5c973a57a0 20 read_log 192888'16899 (0'0) promote 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by osd.51.0:2667 2014-08-11 17:06:07.552641 2014-08-13 18:06:06.929953 7f5c973a57a0 20 read_log 192888'16900 (192888'16899) modify 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.942868.0:13847 2014-08-11 17:06:07.542820 2014-08-13 18:06:06.929970 7f5c973a57a0 20 read_log 192888'16903 (0'0) promote deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by osd.51.0:2675 2014-08-11 17:06:09.386954 2014-08-13 18:06:06.929982 7f5c973a57a0 20 read_log 192888'16904 (192888'16903) modify deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.942868.0:14285 2014-08-11 17:06:09.379028 2014-08-13 18:06:06.930005 7f5c973a57a0 20 read_log 192888'16905 (0'0) promote ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by osd.51.0:2683 2014-08-11 17:06:14.783958 2014-08-13 18:06:06.930021 7f5c973a57a0 20 read_log 192888'16906 (192888'16905) modify ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.942868.0:15658 2014-08-11 17:06:14.776593 2014-08-13 18:06:06.930034 7f5c973a57a0 20 read_log 192888'16907 (0'0) promote 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by osd.51.0:2703 2014-08-11 17:06:18.736788 2014-08-13 18:06:06.930046 7f5c973a57a0 20 read_log 192888'16908 (192888'16907) modify 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.942868.0:16662 2014-08-11 17:06:18.729252 2014-08-13 18:06:06.930059 7f5c973a57a0 20 read_log 192888'16909 (0'0) promote 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by osd.51.0:2713 2014-08-11 17:06:21.629647 2014-08-13 18:06:06.930071 7f5c973a57a0 20 read_log 192888'16910 (192888'16909) modify 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.942868.0:17415 2014-08-11 17:06:21.622734 2014-08-13 18:06:06.930085 7f5c973a57a0 20 read_log 192888'16913 (0'0) promote 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by osd.51.0:2719 2014-08-11 17:06:23.500127 2014-08-13 18:06:06.930100 7f5c973a57a0 20 read_log 192888'16914 (192888'16913) modify 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.942868.0:17912 2014-08-11 17:06:23.492270 2014-08-13 18:06:06.930113 7f5c973a57a0 20 read_log 192888'16915 (0'0) promote e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by osd.51.0:2727 2014-08-11 17:06:27.449820 2014-08-13 18:06:06.930125 7f5c973a57a0 20 read_log 192888'16916 (192888'16915) modify e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.942868.0:18962 2014-08-11 17:06:27.442272 2014-08-13 18:06:06.930137 7f5c973a57a0 20 read_log 192888'16917 (0'0) promote d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by osd.51.0:2735 2014-08-11 17:06:32.070302 2014-08-13 18:06:06.930158 7f5c973a57a0 20 read_log 192888'16918 (192888'16917) modify d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.942868.0:20189 2014-08-11 17:06:32.062416 2014-08-13 18:06:06.930174 7f5c973a57a0 20 read_log 192888'16919 (0'0) promote a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by osd.51.0:2739 2014-08-11 17:06:32.521282 2014-08-13 18:06:06.930186 7f5c973a57a0 20 read_log 192888'16920 (192888'16919) modify a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.942868.0:20311 2014-08-11 17:06:32.513382 2014-08-13 18:06:06.930202 7f5c973a57a0 20 read_log 192888'16921 (0'0) promote 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by osd.51.0:2743 2014-08-11 17:06:32.904790 2014-08-13 18:06:06.930227 7f5c973a57a0 20 read_log 192888'16922 (192888'16921) modify 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.942868.0:20397 2014-08-11 17:06:32.897078 2014-08-13 18:06:06.930247 7f5c973a57a0 20 read_log 192888'16927 (0'0) promote b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by osd.51.0:2755 2014-08-11 17:06:35.891790 2014-08-13 18:06:06.930276 7f5c973a57a0 20 read_log 192888'16928 (192888'16927) modify b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.942868.0:21174 2014-08-11 17:06:35.884458 2014-08-13 18:06:06.930293 7f5c973a57a0 20 read_log 192888'16929 (0'0) promote 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by osd.51.0:2757 2014-08-11 17:06:36.576736 2014-08-13 18:06:06.930306 7f5c973a57a0 20 read_log 192888'16930 (192888'16929) modify 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.942868.0:21355 2014-08-11 17:06:36.567013 2014-08-13 18:06:06.930326 7f5c973a57a0 20 read_log 192888'16933 (0'0) promote 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by osd.51.0:2765 2014-08-11 17:06:38.370879 2014-08-13 18:06:06.930341 7f5c973a57a0 20 read_log 192888'16934 (192888'16933) modify 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.942868.0:21837 2014-08-11 17:06:38.363160 2014-08-13 18:06:06.930354 7f5c973a57a0 20 read_log 192888'16939 (0'0) promote a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by osd.51.0:2781 2014-08-11 17:06:43.811619 2014-08-13 18:06:06.930366 7f5c973a57a0 20 read_log 192888'16940 (192888'16939) modify a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.942868.0:23212 2014-08-11 17:06:43.803959 2014-08-13 18:06:06.930378 7f5c973a57a0 20 read_log 192888'16945 (0'0) promote 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by osd.51.0:2803 2014-08-11 17:06:48.830144 2014-08-13 18:06:06.930390 7f5c973a57a0 20 read_log 192888'16946 (192888'16945) modify 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.942868.0:24571 2014-08-11 17:06:48.822047 2014-08-13 18:06:06.930403 7f5c973a57a0 20 read_log 192888'16951 (0'0) promote 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by osd.51.0:2825 2014-08-11 17:06:54.470316 2014-08-13 18:06:06.930415 7f5c973a57a0 20 read_log 192888'16952 (192888'16951) modify 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.942868.0:26019 2014-08-11 17:06:54.462128 2014-08-13 18:06:06.930427 7f5c973a57a0 20 read_log 192888'16953 (0'0) promote 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by osd.51.0:2829 2014-08-11 17:06:55.202313 2014-08-13 18:06:06.930439 7f5c973a57a0 20 read_log 192888'16954 (192888'16953) modify 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.942868.0:26198 2014-08-11 17:06:55.194148 2014-08-13 18:06:06.930451 7f5c973a57a0 20 read_log 192888'16955 (0'0) promote 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by osd.51.0:2835 2014-08-11 17:06:56.290335 2014-08-13 18:06:06.930464 7f5c973a57a0 20 read_log 192888'16956 (192888'16955) modify 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.942868.0:26493 2014-08-11 17:06:56.282823 2014-08-13 18:06:06.930476 7f5c973a57a0 20 read_log 192888'16957 (0'0) promote eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by osd.51.0:2841 2014-08-11 17:06:58.313749 2014-08-13 18:06:06.930488 7f5c973a57a0 20 read_log 192888'16958 (192888'16957) modify eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.942868.0:27024 2014-08-11 17:06:58.305684 2014-08-13 18:06:06.930502 7f5c973a57a0 20 read_log 192888'16961 (0'0) promote f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by osd.51.0:2849 2014-08-11 17:07:02.643901 2014-08-13 18:06:06.930515 7f5c973a57a0 20 read_log 192888'16962 (192888'16961) modify f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.942868.0:28178 2014-08-11 17:07:02.636718 2014-08-13 18:06:06.930527 7f5c973a57a0 20 read_log 192888'16967 (0'0) promote dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by osd.51.0:2863 2014-08-11 17:07:07.735958 2014-08-13 18:06:06.930550 7f5c973a57a0 20 read_log 192888'16968 (192888'16967) modify dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.942868.0:29481 2014-08-11 17:07:07.727174 2014-08-13 18:06:06.930565 7f5c973a57a0 20 read_log 192888'16969 (0'0) promote e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by osd.51.0:2871 2014-08-11 17:07:12.944188 2014-08-13 18:06:06.930578 7f5c973a57a0 20 read_log 192888'16970 (192888'16969) modify e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.942868.0:30778 2014-08-11 17:07:12.936368 2014-08-13 18:06:06.930590 7f5c973a57a0 20 read_log 192888'16973 (0'0) promote 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by osd.51.0:2877 2014-08-11 17:07:14.536729 2014-08-13 18:06:06.930602 7f5c973a57a0 20 read_log 192888'16974 (192888'16973) modify 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.942868.0:31190 2014-08-11 17:07:14.526514 2014-08-13 18:06:06.930614 7f5c973a57a0 20 read_log 192888'16978 (192888'16838) modify 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.942868.0:31859 2014-08-11 17:07:16.079265 2014-08-13 18:06:06.930627 7f5c973a57a0 20 read_log 192888'16981 (192888'16844) modify 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.942868.0:32102 2014-08-11 17:07:16.155528 2014-08-13 18:06:06.930639 7f5c973a57a0 20 read_log 192888'16982 (192888'16846) modify 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.942868.0:32425 2014-08-11 17:07:16.250078 2014-08-13 18:06:06.930651 7f5c973a57a0 20 read_log 192888'16983 (192888'16848) modify 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.942868.0:32470 2014-08-11 17:07:16.263268 2014-08-13 18:06:06.930664 7f5c973a57a0 20 read_log 192888'16985 (192888'16852) modify e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.942868.0:34056 2014-08-11 17:07:16.686599 2014-08-13 18:06:06.930676 7f5c973a57a0 20 read_log 192888'16986 (192888'16854) modify 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.942868.0:34292 2014-08-11 17:07:16.748452 2014-08-13 18:06:06.930690 7f5c973a57a0 20 read_log 192888'16987 (192888'16856) modify 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.942868.0:35224 2014-08-11 17:07:17.020184 2014-08-13 18:06:06.930703 7f5c973a57a0 20 read_log 192888'16990 (192888'16862) modify a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.942868.0:36581 2014-08-11 17:07:17.379751 2014-08-13 18:06:06.930717 7f5c973a57a0 20 read_log 192888'16991 (192888'16864) modify 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.942868.0:36846 2014-08-11 17:07:17.449171 2014-08-13 18:06:06.930730 7f5c973a57a0 20 read_log 192888'16992 (192888'16866) modify 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.942868.0:39317 2014-08-11 17:07:18.147515 2014-08-13 18:06:06.930742 7f5c973a57a0 20 read_log 192888'16997 (192888'16876) modify c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.942868.0:40665 2014-08-11 17:07:18.514436 2014-08-13 18:06:06.930756 7f5c973a57a0 20 read_log 192888'17000 (192888'16882) modify db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.942868.0:42301 2014-08-11 17:07:18.962047 2014-08-13 18:06:06.930769 7f5c973a57a0 20 read_log 192888'17003 (192888'16888) modify 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.942868.0:43208 2014-08-11 17:07:19.207821 2014-08-13 18:06:06.930783 7f5c973a57a0 20 read_log 192888'17006 (192888'16894) modify f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.942868.0:44273 2014-08-11 17:07:19.519128 2014-08-13 18:06:06.930797 7f5c973a57a0 20 read_log 192888'17007 (192888'16896) modify d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.942868.0:44288 2014-08-11 17:07:19.524137 2014-08-13 18:06:06.930817 7f5c973a57a0 20 read_log 192888'17009 (192888'16900) modify 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.942868.0:45113 2014-08-11 17:07:19.750103 2014-08-13 18:06:06.930832 7f5c973a57a0 20 read_log 192888'17011 (192888'16904) modify deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.942868.0:45551 2014-08-11 17:07:19.869505 2014-08-13 18:06:06.930844 7f5c973a57a0 20 read_log 192888'17012 (192888'16906) modify ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.942868.0:46924 2014-08-11 17:07:20.256382 2014-08-13 18:06:06.930859 7f5c973a57a0 20 read_log 192888'17013 (192888'16908) modify 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.942868.0:47928 2014-08-11 17:07:20.568465 2014-08-13 18:06:06.930872 7f5c973a57a0 20 read_log 192888'17014 (192888'16910) modify 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.942868.0:48681 2014-08-11 17:07:20.785727 2014-08-13 18:06:06.930886 7f5c973a57a0 20 read_log 192888'17016 (192888'16914) modify 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.942868.0:49178 2014-08-11 17:07:20.931894 2014-08-13 18:06:06.930898 7f5c973a57a0 20 read_log 192888'17017 (192888'16916) modify e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.942868.0:50228 2014-08-11 17:07:21.253477 2014-08-13 18:06:06.930910 7f5c973a57a0 20 read_log 192888'17018 (192888'16918) modify d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.942868.0:51455 2014-08-11 17:07:21.653708 2014-08-13 18:06:06.930923 7f5c973a57a0 20 read_log 192888'17019 (192888'16920) modify a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.942868.0:51577 2014-08-11 17:07:21.687141 2014-08-13 18:06:06.930937 7f5c973a57a0 20 read_log 192888'17020 (192888'16922) modify 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.942868.0:51663 2014-08-11 17:07:21.713923 2014-08-13 18:06:06.930951 7f5c973a57a0 20 read_log 192888'17023 (192888'16928) modify b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.942868.0:52440 2014-08-11 17:07:21.940939 2014-08-13 18:06:06.930964 7f5c973a57a0 20 read_log 192888'17024 (192888'16930) modify 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.942868.0:52621 2014-08-11 17:07:21.999472 2014-08-13 18:06:06.930998 7f5c973a57a0 20 read_log 192888'17026 (192888'16934) modify 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.942868.0:53103 2014-08-11 17:07:22.148402 2014-08-13 18:06:06.931013 7f5c973a57a0 20 read_log 192888'17029 (192888'16940) modify a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.942868.0:54478 2014-08-11 17:07:22.568773 2014-08-13 18:06:06.931025 7f5c973a57a0 20 read_log 192888'17032 (192888'16946) modify 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.942868.0:55837 2014-08-11 17:07:22.972350 2014-08-13 18:06:06.931039 7f5c973a57a0 20 read_log 192888'17035 (192888'16952) modify 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.942868.0:57285 2014-08-11 17:07:23.412466 2014-08-13 18:06:06.931052 7f5c973a57a0 20 read_log 192888'17036 (192888'16954) modify 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.942868.0:57464 2014-08-11 17:07:23.463170 2014-08-13 18:06:06.931066 7f5c973a57a0 20 read_log 192888'17037 (192888'16956) modify 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.942868.0:57759 2014-08-11 17:07:23.547536 2014-08-13 18:06:06.931079 7f5c973a57a0 20 read_log 192888'17038 (192888'16958) modify eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.942868.0:58290 2014-08-11 17:07:23.697059 2014-08-13 18:06:06.931093 7f5c973a57a0 20 read_log 192888'17040 (192888'16962) modify f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.942868.0:59444 2014-08-11 17:07:24.049065 2014-08-13 18:06:06.931116 7f5c973a57a0 20 read_log 192888'17043 (192888'16968) modify dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.942868.0:60747 2014-08-11 17:07:24.431819 2014-08-13 18:06:06.931131 7f5c973a57a0 20 read_log 192888'17044 (192888'16970) modify e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.942868.0:62044 2014-08-11 17:07:24.832056 2014-08-13 18:06:06.931143 7f5c973a57a0 20 read_log 192888'17046 (192888'16974) modify 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.942868.0:62456 2014-08-11 17:07:24.951367 2014-08-13 18:06:06.931163 7f5c973a57a0 20 read_log 193305'17047 (0'0) promote 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by osd.356.0:1 2014-08-11 18:42:33.802159 2014-08-13 18:06:06.931176 7f5c973a57a0 20 read_log 193305'17048 (193305'17047) modify 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.951337.0:217 2014-08-11 18:42:33.789721 2014-08-13 18:06:06.931188 7f5c973a57a0 20 read_log 193305'17049 (0'0) promote 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by osd.356.0:3 2014-08-11 18:42:35.868439 2014-08-13 18:06:06.931200 7f5c973a57a0 20 read_log 193305'17050 (193305'17049) modify 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.951337.0:694 2014-08-11 18:42:35.861295 2014-08-13 18:06:06.931213 7f5c973a57a0 20 read_log 193305'17051 (0'0) promote d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by osd.356.0:5 2014-08-11 18:42:36.054869 2014-08-13 18:06:06.931225 7f5c973a57a0 20 read_log 193305'17052 (193305'17051) modify d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.951337.0:746 2014-08-11 18:42:36.049345 2014-08-13 18:06:06.931239 7f5c973a57a0 20 read_log 193305'17053 (193305'17053) modify 350/hit_set_24.350_archive_2014-08-11 18:37:36.154490_2014-08-11 18:42:41.505389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:41.505523 2014-08-13 18:06:06.931252 7f5c973a57a0 20 read_log 193305'17054 (0'0) promote fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by osd.356.0:21 2014-08-11 18:42:41.507717 2014-08-13 18:06:06.931270 7f5c973a57a0 20 read_log 193305'17055 (193305'17054) modify fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.951337.0:2206 2014-08-11 18:42:41.501187 2014-08-13 18:06:06.931284 7f5c973a57a0 20 read_log 193305'17056 (0'0) promote f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by osd.356.0:43 2014-08-11 18:42:51.738517 2014-08-13 18:06:06.931297 7f5c973a57a0 20 read_log 193305'17057 (193305'17056) modify f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.951337.0:5003 2014-08-11 18:42:51.730540 2014-08-13 18:06:06.931311 7f5c973a57a0 20 read_log 193305'17058 (0'0) promote efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by osd.356.0:45 2014-08-11 18:42:54.217394 2014-08-13 18:06:06.931327 7f5c973a57a0 20 read_log 193305'17059 (193305'17058) modify efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.951337.0:5666 2014-08-11 18:42:54.206683 2014-08-13 18:06:06.931342 7f5c973a57a0 20 read_log 193305'17060 (0'0) promote 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by osd.356.0:51 2014-08-11 18:42:56.206075 2014-08-13 18:06:06.931354 7f5c973a57a0 20 read_log 193305'17061 (193305'17060) modify 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.951337.0:6214 2014-08-11 18:42:56.198000 2014-08-13 18:06:06.931369 7f5c973a57a0 20 read_log 193305'17062 (0'0) promote 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by osd.356.0:69 2014-08-11 18:43:05.953857 2014-08-13 18:06:06.931408 7f5c973a57a0 20 read_log 193305'17063 (193305'17062) modify 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.951337.0:8826 2014-08-11 18:43:05.946396 2014-08-13 18:06:06.931428 7f5c973a57a0 20 read_log 193305'17064 (0'0) promote d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by osd.356.0:79 2014-08-11 18:43:10.265706 2014-08-13 18:06:06.931451 7f5c973a57a0 20 read_log 193305'17065 (193305'17064) modify d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.951337.0:9992 2014-08-11 18:43:10.258690 2014-08-13 18:06:06.931464 7f5c973a57a0 20 read_log 193305'17066 (0'0) promote b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by osd.356.0:81 2014-08-11 18:43:11.278661 2014-08-13 18:06:06.931479 7f5c973a57a0 20 read_log 193305'17067 (193305'17066) modify b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.951337.0:10273 2014-08-11 18:43:11.270940 2014-08-13 18:06:06.931501 7f5c973a57a0 20 read_log 193305'17068 (0'0) promote d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by osd.356.0:89 2014-08-11 18:43:13.014672 2014-08-13 18:06:06.931514 7f5c973a57a0 20 read_log 193305'17069 (193305'17068) modify d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.951337.0:10734 2014-08-11 18:43:13.007355 2014-08-13 18:06:06.931527 7f5c973a57a0 20 read_log 193305'17070 (0'0) promote 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by osd.356.0:95 2014-08-11 18:43:15.378357 2014-08-13 18:06:06.931551 7f5c973a57a0 20 read_log 193305'17071 (193305'17070) modify 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.951337.0:11321 2014-08-11 18:43:15.370364 2014-08-13 18:06:06.931565 7f5c973a57a0 20 read_log 193305'17072 (0'0) promote 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by osd.356.0:99 2014-08-11 18:43:16.521862 2014-08-13 18:06:06.931577 7f5c973a57a0 20 read_log 193305'17073 (193305'17072) modify 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.951337.0:11627 2014-08-11 18:43:16.514615 2014-08-13 18:06:06.931591 7f5c973a57a0 20 read_log 193305'17074 (0'0) promote f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by osd.356.0:111 2014-08-11 18:43:22.863547 2014-08-13 18:06:06.931608 7f5c973a57a0 20 read_log 193305'17075 (193305'17074) modify f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.951337.0:13330 2014-08-11 18:43:22.855741 2014-08-13 18:06:06.931623 7f5c973a57a0 20 read_log 193305'17076 (0'0) promote d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by osd.356.0:117 2014-08-11 18:43:24.419895 2014-08-13 18:06:06.931635 7f5c973a57a0 20 read_log 193305'17077 (193305'17076) modify d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.951337.0:13729 2014-08-11 18:43:24.413797 2014-08-13 18:06:06.931648 7f5c973a57a0 20 read_log 193305'17078 (0'0) promote 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by osd.356.0:121 2014-08-11 18:43:25.529223 2014-08-13 18:06:06.931660 7f5c973a57a0 20 read_log 193305'17079 (193305'17078) modify 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.951337.0:14044 2014-08-11 18:43:25.521270 2014-08-13 18:06:06.931672 7f5c973a57a0 20 read_log 193305'17080 (0'0) promote 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by osd.356.0:125 2014-08-11 18:43:26.193569 2014-08-13 18:06:06.931684 7f5c973a57a0 20 read_log 193305'17081 (193305'17080) modify 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.951337.0:14186 2014-08-11 18:43:26.185086 2014-08-13 18:06:06.931697 7f5c973a57a0 20 read_log 193305'17082 (0'0) promote cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by osd.356.0:145 2014-08-11 18:43:30.984457 2014-08-13 18:06:06.931736 7f5c973a57a0 20 read_log 193305'17083 (193305'17082) modify cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.951337.0:15595 2014-08-11 18:43:30.974035 2014-08-13 18:06:06.931752 7f5c973a57a0 20 read_log 193305'17084 (0'0) promote 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by osd.356.0:149 2014-08-11 18:43:34.749255 2014-08-13 18:06:06.931764 7f5c973a57a0 20 read_log 193305'17085 (193305'17084) modify 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.951337.0:16670 2014-08-11 18:43:34.742308 2014-08-13 18:06:06.931776 7f5c973a57a0 20 read_log 193305'17086 (0'0) promote 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by osd.356.0:159 2014-08-11 18:43:37.081002 2014-08-13 18:06:06.931788 7f5c973a57a0 20 read_log 193305'17087 (193305'17086) modify 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.951337.0:17332 2014-08-11 18:43:37.072943 2014-08-13 18:06:06.931801 7f5c973a57a0 20 read_log 193305'17088 (0'0) promote f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by osd.356.0:171 2014-08-11 18:43:42.713712 2014-08-13 18:06:06.931813 7f5c973a57a0 20 read_log 193305'17089 (193305'17088) modify f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.951337.0:18935 2014-08-11 18:43:42.706790 2014-08-13 18:06:06.931825 7f5c973a57a0 20 read_log 193305'17090 (0'0) promote 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by osd.356.0:177 2014-08-11 18:43:45.883199 2014-08-13 18:06:06.931846 7f5c973a57a0 20 read_log 193305'17091 (193305'17090) modify 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.951337.0:19794 2014-08-11 18:43:45.876639 2014-08-13 18:06:06.931861 7f5c973a57a0 20 read_log 193305'17092 (0'0) promote 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by osd.356.0:185 2014-08-11 18:43:49.452087 2014-08-13 18:06:06.931873 7f5c973a57a0 20 read_log 193305'17093 (193305'17092) modify 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.951337.0:20791 2014-08-11 18:43:49.444640 2014-08-13 18:06:06.931887 7f5c973a57a0 20 read_log 193305'17094 (0'0) promote a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by osd.356.0:191 2014-08-11 18:43:52.027819 2014-08-13 18:06:06.931899 7f5c973a57a0 20 read_log 193305'17095 (193305'17094) modify a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.951337.0:21507 2014-08-11 18:43:52.019578 2014-08-13 18:06:06.931912 7f5c973a57a0 20 read_log 193305'17096 (0'0) promote f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by osd.356.0:201 2014-08-11 18:43:56.648858 2014-08-13 18:06:06.931926 7f5c973a57a0 20 read_log 193305'17097 (193305'17096) modify f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.951337.0:22773 2014-08-11 18:43:56.641559 2014-08-13 18:06:06.931939 7f5c973a57a0 20 read_log 193305'17098 (0'0) promote c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by osd.356.0:209 2014-08-11 18:43:58.678251 2014-08-13 18:06:06.931953 7f5c973a57a0 20 read_log 193305'17099 (193305'17098) modify c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.951337.0:23354 2014-08-11 18:43:58.671098 2014-08-13 18:06:06.931965 7f5c973a57a0 20 read_log 193305'17100 (0'0) promote f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by osd.356.0:213 2014-08-11 18:44:00.458243 2014-08-13 18:06:06.931979 7f5c973a57a0 20 read_log 193305'17101 (193305'17100) modify f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.951337.0:23845 2014-08-11 18:44:00.449613 2014-08-13 18:06:06.932011 7f5c973a57a0 20 read_log 193305'17102 (0'0) promote 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by osd.356.0:217 2014-08-11 18:44:04.865607 2014-08-13 18:06:06.932032 7f5c973a57a0 20 read_log 193305'17103 (193305'17102) modify 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.951337.0:25056 2014-08-11 18:44:04.859683 2014-08-13 18:06:06.932047 7f5c973a57a0 20 read_log 193305'17104 (0'0) promote ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by osd.356.0:243 2014-08-11 18:44:17.193329 2014-08-13 18:06:06.932059 7f5c973a57a0 20 read_log 193305'17105 (193305'17104) modify ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.951337.0:28425 2014-08-11 18:44:17.184770 2014-08-13 18:06:06.932073 7f5c973a57a0 20 read_log 193305'17106 (0'0) promote c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by osd.356.0:253 2014-08-11 18:44:21.863605 2014-08-13 18:06:06.932086 7f5c973a57a0 20 read_log 193305'17107 (193305'17106) modify c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.951337.0:29695 2014-08-11 18:44:21.855461 2014-08-13 18:06:06.932100 7f5c973a57a0 20 read_log 193305'17108 (193305'17048) modify 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.951337.0:32928 2014-08-11 18:44:33.962268 2014-08-13 18:06:06.932112 7f5c973a57a0 20 read_log 193305'17109 (193305'17050) modify 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.951337.0:33405 2014-08-11 18:44:34.388894 2014-08-13 18:06:06.932125 7f5c973a57a0 20 read_log 193305'17110 (193305'17052) modify d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.951337.0:33457 2014-08-11 18:44:34.437696 2014-08-13 18:06:06.932137 7f5c973a57a0 20 read_log 193305'17111 (193305'17055) modify fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.951337.0:34917 2014-08-11 18:44:35.258149 2014-08-13 18:06:06.932165 7f5c973a57a0 20 read_log 193305'17112 (193305'17057) modify f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.951337.0:37714 2014-08-11 18:44:37.034529 2014-08-13 18:06:06.932181 7f5c973a57a0 20 read_log 193305'17113 (193305'17059) modify efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.951337.0:38377 2014-08-11 18:44:37.381250 2014-08-13 18:06:06.932194 7f5c973a57a0 20 read_log 193305'17114 (193305'17061) modify 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.951337.0:38925 2014-08-11 18:44:37.737430 2014-08-13 18:06:06.932208 7f5c973a57a0 20 read_log 193305'17115 (193305'17063) modify 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.951337.0:41537 2014-08-11 18:44:38.785552 2014-08-13 18:06:06.932220 7f5c973a57a0 20 read_log 193305'17116 (193305'17065) modify d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.951337.0:42703 2014-08-11 18:44:39.356617 2014-08-13 18:06:06.932234 7f5c973a57a0 20 read_log 193305'17117 (193305'17067) modify b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.951337.0:42984 2014-08-11 18:44:39.444342 2014-08-13 18:06:06.932247 7f5c973a57a0 20 read_log 193305'17118 (193305'17069) modify d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.951337.0:43445 2014-08-11 18:44:39.592492 2014-08-13 18:06:06.932264 7f5c973a57a0 20 read_log 193305'17119 (193305'17071) modify 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.951337.0:44032 2014-08-11 18:44:39.774287 2014-08-13 18:06:06.932279 7f5c973a57a0 20 read_log 193305'17120 (193305'17073) modify 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.951337.0:44338 2014-08-11 18:44:39.856607 2014-08-13 18:06:06.932291 7f5c973a57a0 20 read_log 193305'17121 (193305'17075) modify f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.951337.0:46041 2014-08-11 18:44:40.373310 2014-08-13 18:06:06.932304 7f5c973a57a0 20 read_log 193305'17122 (193305'17077) modify d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.951337.0:46440 2014-08-11 18:44:40.482582 2014-08-13 18:06:06.932327 7f5c973a57a0 20 read_log 193305'17123 (193305'17079) modify 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.951337.0:46755 2014-08-11 18:44:40.579895 2014-08-13 18:06:06.932342 7f5c973a57a0 20 read_log 193305'17124 (193305'17081) modify 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.951337.0:46897 2014-08-11 18:44:40.620662 2014-08-13 18:06:06.932355 7f5c973a57a0 20 read_log 193305'17125 (193305'17083) modify cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.951337.0:48306 2014-08-11 18:44:41.015955 2014-08-13 18:06:06.932367 7f5c973a57a0 20 read_log 193305'17126 (193305'17085) modify 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.951337.0:49381 2014-08-11 18:44:41.349136 2014-08-13 18:06:06.932381 7f5c973a57a0 20 read_log 193305'17127 (193305'17087) modify 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.951337.0:50043 2014-08-11 18:44:41.547409 2014-08-13 18:06:06.932405 7f5c973a57a0 20 read_log 193305'17128 (193305'17089) modify f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.951337.0:51646 2014-08-11 18:44:42.009807 2014-08-13 18:06:06.932448 7f5c973a57a0 20 read_log 193305'17129 (193305'17091) modify 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.951337.0:52505 2014-08-11 18:44:42.251727 2014-08-13 18:06:06.932465 7f5c973a57a0 20 read_log 193305'17130 (193305'17093) modify 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.951337.0:53502 2014-08-11 18:44:42.548126 2014-08-13 18:06:06.932477 7f5c973a57a0 20 read_log 193305'17131 (193305'17095) modify a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.951337.0:54218 2014-08-11 18:44:42.764693 2014-08-13 18:06:06.932497 7f5c973a57a0 20 read_log 193305'17132 (193305'17097) modify f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.951337.0:55484 2014-08-11 18:44:43.140563 2014-08-13 18:06:06.932515 7f5c973a57a0 20 read_log 193305'17133 (193305'17099) modify c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.951337.0:56065 2014-08-11 18:44:43.311153 2014-08-13 18:06:06.932527 7f5c973a57a0 20 read_log 193305'17134 (193305'17101) modify f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.951337.0:56556 2014-08-11 18:44:43.453315 2014-08-13 18:06:06.932540 7f5c973a57a0 20 read_log 193305'17135 (193305'17103) modify 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.951337.0:57767 2014-08-11 18:44:43.854226 2014-08-13 18:06:06.932552 7f5c973a57a0 20 read_log 193305'17136 (193305'17105) modify ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.951337.0:61136 2014-08-11 18:44:44.834288 2014-08-13 18:06:06.932564 7f5c973a57a0 20 read_log 193305'17137 (193305'17107) modify c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.951337.0:62406 2014-08-11 18:44:45.218117 2014-08-13 18:06:06.932577 7f5c973a57a0 20 read_log 194215'17138 (194215'17138) modify 350/hit_set_24.350_archive_2014-08-12 15:32:25.978048_2014-08-12 15:49:16.440311/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:16.440550 2014-08-13 18:06:06.932589 7f5c973a57a0 20 read_log 194215'17139 (193305'17053) delete 350/hit_set_24.350_archive_2014-08-11 18:37:36.154490_2014-08-11 18:42:41.505389/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:16.440550 2014-08-13 18:06:06.932601 7f5c973a57a0 20 read_log 194215'17140 (192888'17016) clean 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by osd.356.0:1047 2014-08-12 15:49:16.444986 2014-08-13 18:06:06.932614 7f5c973a57a0 20 read_log 194215'17141 (194215'17140) delete 68370350/benchmark_data_storage0111-ib.pocstor_2228_object17911/head//24 by client.2174595.0:174649 0.000000 2014-08-13 18:06:06.932633 7f5c973a57a0 20 read_log 194215'17142 (192888'17000) clean db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by osd.356.0:1049 2014-08-12 15:49:16.453124 2014-08-13 18:06:06.932649 7f5c973a57a0 20 read_log 194215'17143 (194215'17142) delete db90350/benchmark_data_storage0111-ib.pocstor_2228_object11034/head//24 by client.2174595.0:174651 0.000000 2014-08-13 18:06:06.932660 7f5c973a57a0 20 read_log 194215'17144 (193305'17135) clean 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by osd.356.0:1051 2014-08-12 15:49:16.460690 2014-08-13 18:06:06.932672 7f5c973a57a0 20 read_log 194215'17145 (194215'17144) delete 535a0350/benchmark_data_storage0111-ib.pocstor_62621_object25055/head//24 by client.2174595.0:174653 0.000000 2014-08-13 18:06:06.932683 7f5c973a57a0 20 read_log 194215'17146 (192888'16981) clean 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by osd.356.0:1053 2014-08-12 15:49:16.467683 2014-08-13 18:06:06.932695 7f5c973a57a0 20 read_log 194215'17147 (194215'17146) delete 8f741350/benchmark_data_storage0111-ib.pocstor_2228_object835/head//24 by client.2174595.0:174655 0.000000 2014-08-13 18:06:06.932706 7f5c973a57a0 20 read_log 194215'17148 (192888'16987) clean 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by osd.356.0:1055 2014-08-12 15:49:16.474986 2014-08-13 18:06:06.932718 7f5c973a57a0 20 read_log 194215'17149 (194215'17148) delete 242a1350/benchmark_data_storage0111-ib.pocstor_2228_object3957/head//24 by client.2174595.0:174657 0.000000 2014-08-13 18:06:06.932729 7f5c973a57a0 20 read_log 194215'17150 (193305'17127) clean 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by osd.356.0:1057 2014-08-12 15:49:16.482525 2014-08-13 18:06:06.932742 7f5c973a57a0 20 read_log 194215'17151 (194215'17150) delete 24ce1350/benchmark_data_storage0111-ib.pocstor_62621_object17331/head//24 by client.2174595.0:174659 0.000000 2014-08-13 18:06:06.932752 7f5c973a57a0 20 read_log 194215'17152 (192888'16784) clean 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by osd.356.0:1059 2014-08-12 15:49:16.490115 2014-08-13 18:06:06.932765 7f5c973a57a0 20 read_log 194215'17153 (194215'17152) delete 9d852350/benchmark_data_storage0111-ib.pocstor_41097_object7193/head//24 by client.2174595.0:174661 0.000000 2014-08-13 18:06:06.932775 7f5c973a57a0 20 read_log 194215'17154 (192888'16783) clean c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by osd.356.0:1061 2014-08-12 15:49:16.497736 2014-08-13 18:06:06.932788 7f5c973a57a0 20 read_log 194215'17155 (194215'17154) delete c8263350/benchmark_data_storage0111-ib.pocstor_41097_object6955/head//24 by client.2174595.0:174663 0.000000 2014-08-13 18:06:06.932798 7f5c973a57a0 20 read_log 194215'17156 (192888'17023) clean b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by osd.356.0:1063 2014-08-12 15:49:16.505080 2014-08-13 18:06:06.932811 7f5c973a57a0 20 read_log 194215'17157 (194215'17156) delete b4f63350/benchmark_data_storage0111-ib.pocstor_2228_object21173/head//24 by client.2174595.0:174665 0.000000 2014-08-13 18:06:06.932822 7f5c973a57a0 20 read_log 194215'17158 (192888'16824) clean 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by osd.356.0:1065 2014-08-12 15:49:16.512192 2014-08-13 18:06:06.932834 7f5c973a57a0 20 read_log 194215'17159 (194215'17158) delete 7cf64350/benchmark_data_storage0111-ib.pocstor_41097_object27789/head//24 by client.2174595.0:174667 0.000000 2014-08-13 18:06:06.932845 7f5c973a57a0 20 read_log 194215'17160 (193305'17120) clean 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by osd.356.0:1067 2014-08-12 15:49:16.519349 2014-08-13 18:06:06.932857 7f5c973a57a0 20 read_log 194215'17161 (194215'17160) delete 651a4350/benchmark_data_storage0111-ib.pocstor_62621_object11626/head//24 by client.2174595.0:174669 0.000000 2014-08-13 18:06:06.932875 7f5c973a57a0 20 read_log 194215'17162 (193305'17134) clean f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by osd.356.0:1069 2014-08-12 15:49:16.526266 2014-08-13 18:06:06.932890 7f5c973a57a0 20 read_log 194215'17163 (194215'17162) delete f7d35350/benchmark_data_storage0111-ib.pocstor_62621_object23844/head//24 by client.2174595.0:174671 0.000000 2014-08-13 18:06:06.932901 7f5c973a57a0 20 read_log 194215'17164 (192888'16985) clean e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by osd.356.0:1071 2014-08-12 15:49:16.534061 2014-08-13 18:06:06.932913 7f5c973a57a0 20 read_log 194215'17165 (194215'17164) delete e2765350/benchmark_data_storage0111-ib.pocstor_2228_object2789/head//24 by client.2174595.0:174673 0.000000 2014-08-13 18:06:06.932924 7f5c973a57a0 20 read_log 194215'17166 (193305'17124) clean 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by osd.356.0:1073 2014-08-12 15:49:16.541323 2014-08-13 18:06:06.932939 7f5c973a57a0 20 read_log 194215'17167 (194215'17166) delete 1865350/benchmark_data_storage0111-ib.pocstor_62621_object14185/head//24 by client.2174595.0:174675 0.000000 2014-08-13 18:06:06.932950 7f5c973a57a0 20 read_log 194215'17168 (193305'17108) clean 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by osd.356.0:1075 2014-08-12 15:49:16.548306 2014-08-13 18:06:06.932962 7f5c973a57a0 20 read_log 194215'17169 (194215'17168) delete 18467350/benchmark_data_storage0111-ib.pocstor_62621_object216/head//24 by client.2174595.0:174677 0.000000 2014-08-13 18:06:06.932973 7f5c973a57a0 20 read_log 194215'17170 (193305'17129) clean 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by osd.356.0:1077 2014-08-12 15:49:16.556725 2014-08-13 18:06:06.932985 7f5c973a57a0 20 read_log 194215'17171 (194215'17170) delete 97667350/benchmark_data_storage0111-ib.pocstor_62621_object19793/head//24 by client.2174595.0:174679 0.000000 2014-08-13 18:06:06.932996 7f5c973a57a0 20 read_log 194215'17172 (192888'16785) clean 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by osd.356.0:1079 2014-08-12 15:49:16.562791 2014-08-13 18:06:06.933009 7f5c973a57a0 20 read_log 194215'17173 (194215'17172) delete 5ba08350/benchmark_data_storage0111-ib.pocstor_41097_object7788/head//24 by client.2174595.0:174681 0.000000 2014-08-13 18:06:06.933020 7f5c973a57a0 20 read_log 194215'17174 (192888'16821) clean c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by osd.356.0:1081 2014-08-12 15:49:16.569706 2014-08-13 18:06:06.933033 7f5c973a57a0 20 read_log 194215'17175 (194215'17174) delete c3c48350/benchmark_data_storage0111-ib.pocstor_41097_object26503/head//24 by client.2174595.0:174683 0.000000 2014-08-13 18:06:06.933043 7f5c973a57a0 20 read_log 194215'17176 (193305'17121) clean f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by osd.356.0:1083 2014-08-12 15:49:16.576250 2014-08-13 18:06:06.933056 7f5c973a57a0 20 read_log 194215'17177 (194215'17176) delete f4298350/benchmark_data_storage0111-ib.pocstor_62621_object13329/head//24 by client.2174595.0:174685 0.000000 2014-08-13 18:06:06.933066 7f5c973a57a0 20 read_log 194215'17178 (192888'16810) clean 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by osd.356.0:1085 2014-08-12 15:49:16.584084 2014-08-13 18:06:06.933081 7f5c973a57a0 20 read_log 194215'17179 (194215'17178) delete 59b99350/benchmark_data_storage0111-ib.pocstor_41097_object21149/head//24 by client.2174595.0:174687 0.000000 2014-08-13 18:06:06.933092 7f5c973a57a0 20 read_log 194215'17180 (193305'17131) clean a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by osd.356.0:1087 2014-08-12 15:49:16.591897 2014-08-13 18:06:06.933104 7f5c973a57a0 20 read_log 194215'17181 (194215'17180) delete a121a350/benchmark_data_storage0111-ib.pocstor_62621_object21506/head//24 by client.2174595.0:174689 0.000000 2014-08-13 18:06:06.933126 7f5c973a57a0 20 read_log 194215'17182 (192888'16802) clean 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by osd.356.0:1089 2014-08-12 15:49:16.599071 2014-08-13 18:06:06.933141 7f5c973a57a0 20 read_log 194215'17183 (194215'17182) delete 47e3a350/benchmark_data_storage0111-ib.pocstor_41097_object15984/head//24 by client.2174595.0:174691 0.000000 2014-08-13 18:06:06.933160 7f5c973a57a0 20 read_log 194215'17184 (193305'17132) clean f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by osd.356.0:1091 2014-08-12 15:49:16.605344 2014-08-13 18:06:06.933173 7f5c973a57a0 20 read_log 194215'17185 (194215'17184) delete f9e2b350/benchmark_data_storage0111-ib.pocstor_62621_object22772/head//24 by client.2174595.0:174693 0.000000 2014-08-13 18:06:06.933183 7f5c973a57a0 20 read_log 194215'17186 (192888'17018) clean d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by osd.356.0:1093 2014-08-12 15:49:16.611612 2014-08-13 18:06:06.933197 7f5c973a57a0 20 read_log 194215'17187 (194215'17186) delete d45c350/benchmark_data_storage0111-ib.pocstor_2228_object20188/head//24 by client.2174595.0:174695 0.000000 2014-08-13 18:06:06.933210 7f5c973a57a0 20 read_log 194215'17188 (192888'17006) clean f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by osd.356.0:1095 2014-08-12 15:49:16.617911 2014-08-13 18:06:06.933222 7f5c973a57a0 20 read_log 194215'17189 (194215'17188) delete f9d4d350/benchmark_data_storage0111-ib.pocstor_2228_object13006/head//24 by client.2174595.0:174697 0.000000 2014-08-13 18:06:06.933233 7f5c973a57a0 20 read_log 194215'17190 (193305'17118) clean d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by osd.356.0:1097 2014-08-12 15:49:16.624032 2014-08-13 18:06:06.933245 7f5c973a57a0 20 read_log 194215'17191 (194215'17190) delete d033e350/benchmark_data_storage0111-ib.pocstor_62621_object10733/head//24 by client.2174595.0:174699 0.000000 2014-08-13 18:06:06.933259 7f5c973a57a0 20 read_log 194215'17192 (192888'17040) clean f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by osd.356.0:1099 2014-08-12 15:49:16.630768 2014-08-13 18:06:06.933276 7f5c973a57a0 20 read_log 194215'17193 (194215'17192) delete f62ce350/benchmark_data_storage0111-ib.pocstor_2228_object28177/head//24 by client.2174595.0:174701 0.000000 2014-08-13 18:06:06.933288 7f5c973a57a0 20 read_log 194215'17194 (192888'17017) clean e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by osd.356.0:1101 2014-08-12 15:49:16.637659 2014-08-13 18:06:06.933300 7f5c973a57a0 20 read_log 194215'17195 (194215'17194) delete e53fe350/benchmark_data_storage0111-ib.pocstor_2228_object18961/head//24 by client.2174595.0:174703 0.000000 2014-08-13 18:06:06.933311 7f5c973a57a0 20 read_log 194215'17196 (192888'16804) clean 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by osd.356.0:1103 2014-08-12 15:49:16.643993 2014-08-13 18:06:06.933327 7f5c973a57a0 20 read_log 194215'17197 (194215'17196) delete 10f1f350/benchmark_data_storage0111-ib.pocstor_41097_object16451/head//24 by client.2174595.0:174705 0.000000 2014-08-13 18:06:06.933341 7f5c973a57a0 20 read_log 194215'17198 (192888'16791) clean 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by osd.356.0:1105 2014-08-12 15:49:16.650325 2014-08-13 18:06:06.933353 7f5c973a57a0 20 read_log 194215'17199 (194215'17198) delete 8803f350/benchmark_data_storage0111-ib.pocstor_41097_object9803/head//24 by client.2174595.0:174707 0.000000 2014-08-13 18:06:06.933364 7f5c973a57a0 20 read_log 194215'17200 (192888'16822) clean 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by osd.356.0:1107 2014-08-12 15:49:16.657736 2014-08-13 18:06:06.933376 7f5c973a57a0 20 read_log 194215'17201 (194215'17200) delete 6399f350/benchmark_data_storage0111-ib.pocstor_41097_object26709/head//24 by client.2174595.0:174709 0.000000 2014-08-13 18:06:06.933405 7f5c973a57a0 20 read_log 194215'17202 (192888'16801) clean e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by osd.356.0:1109 2014-08-12 15:49:16.664471 2014-08-13 18:06:06.933422 7f5c973a57a0 20 read_log 194215'17203 (194215'17202) delete e5e21750/benchmark_data_storage0111-ib.pocstor_41097_object15487/head//24 by client.2174595.0:174711 0.000000 2014-08-13 18:06:06.933440 7f5c973a57a0 20 read_log 194215'17204 (192888'16825) clean 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by osd.356.0:1111 2014-08-12 15:49:16.671531 2014-08-13 18:06:06.933456 7f5c973a57a0 20 read_log 194215'17205 (194215'17204) delete 2d631750/benchmark_data_storage0111-ib.pocstor_41097_object29862/head//24 by client.2174595.0:174713 0.000000 2014-08-13 18:06:06.933468 7f5c973a57a0 20 read_log 194215'17206 (193305'17130) clean 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by osd.356.0:1113 2014-08-12 15:49:16.678484 2014-08-13 18:06:06.933481 7f5c973a57a0 20 read_log 194215'17207 (194215'17206) delete 5a0f1750/benchmark_data_storage0111-ib.pocstor_62621_object20790/head//24 by client.2174595.0:174715 0.000000 2014-08-13 18:06:06.933506 7f5c973a57a0 20 read_log 194215'17208 (192888'17009) clean 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by osd.356.0:1115 2014-08-12 15:49:16.685368 2014-08-13 18:06:06.933519 7f5c973a57a0 20 read_log 194215'17209 (194215'17208) delete 6b912750/benchmark_data_storage0111-ib.pocstor_2228_object13846/head//24 by client.2174595.0:174717 0.000000 2014-08-13 18:06:06.933536 7f5c973a57a0 20 read_log 194215'17210 (192888'16991) clean 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by osd.356.0:1117 2014-08-12 15:49:16.692352 2014-08-13 18:06:06.933551 7f5c973a57a0 20 read_log 194215'17211 (194215'17210) delete 9c45750/benchmark_data_storage0111-ib.pocstor_2228_object5579/head//24 by client.2174595.0:174719 0.000000 2014-08-13 18:06:06.933562 7f5c973a57a0 20 read_log 194215'17212 (193305'17126) clean 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by osd.356.0:1119 2014-08-12 15:49:16.699620 2014-08-13 18:06:06.933576 7f5c973a57a0 20 read_log 194215'17213 (194215'17212) delete 75d65750/benchmark_data_storage0111-ib.pocstor_62621_object16669/head//24 by client.2174595.0:174721 0.000000 2014-08-13 18:06:06.933587 7f5c973a57a0 20 read_log 194215'17214 (192888'16805) clean 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by osd.356.0:1121 2014-08-12 15:49:16.705123 2014-08-13 18:06:06.933600 7f5c973a57a0 20 read_log 194215'17215 (194215'17214) delete 53375750/benchmark_data_storage0111-ib.pocstor_41097_object17491/head//24 by client.2174595.0:174723 0.000000 2014-08-13 18:06:06.933610 7f5c973a57a0 20 read_log 194215'17216 (192888'17011) clean deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by osd.356.0:1123 2014-08-12 15:49:16.710426 2014-08-13 18:06:06.933623 7f5c973a57a0 20 read_log 194215'17217 (194215'17216) delete deab5750/benchmark_data_storage0111-ib.pocstor_2228_object14284/head//24 by client.2174595.0:174725 0.000000 2014-08-13 18:06:06.933634 7f5c973a57a0 20 read_log 194215'17218 (192888'17046) clean 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by osd.356.0:1125 2014-08-12 15:49:16.715928 2014-08-13 18:06:06.933646 7f5c973a57a0 20 read_log 194215'17219 (194215'17218) delete 5f106750/benchmark_data_storage0111-ib.pocstor_2228_object31189/head//24 by client.2174595.0:174727 0.000000 2014-08-13 18:06:06.933657 7f5c973a57a0 20 read_log 194215'17220 (192888'17026) clean 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by osd.356.0:1127 2014-08-12 15:49:16.721060 2014-08-13 18:06:06.933669 7f5c973a57a0 20 read_log 194215'17221 (194215'17220) delete 3bd56750/benchmark_data_storage0111-ib.pocstor_2228_object21836/head//24 by client.2174595.0:174729 0.000000 2014-08-13 18:06:06.933740 7f5c973a57a0 20 read_log 194215'17222 (192888'17014) clean 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by osd.356.0:1129 2014-08-12 15:49:16.726399 2014-08-13 18:06:06.933758 7f5c973a57a0 20 read_log 194215'17223 (194215'17222) delete 241f6750/benchmark_data_storage0111-ib.pocstor_2228_object17414/head//24 by client.2174595.0:174731 0.000000 2014-08-13 18:06:06.933769 7f5c973a57a0 20 read_log 194215'17224 (193305'17113) clean efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by osd.356.0:1131 2014-08-12 15:49:16.731880 2014-08-13 18:06:06.933782 7f5c973a57a0 20 read_log 194215'17225 (194215'17224) delete efc58750/benchmark_data_storage0111-ib.pocstor_62621_object5665/head//24 by client.2174595.0:174733 0.000000 2014-08-13 18:06:06.933793 7f5c973a57a0 20 read_log 194215'17226 (192888'16798) clean 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by osd.356.0:1133 2014-08-12 15:49:16.737276 2014-08-13 18:06:06.933805 7f5c973a57a0 20 read_log 194215'17227 (194215'17226) delete 9db98750/benchmark_data_storage0111-ib.pocstor_41097_object12119/head//24 by client.2174595.0:174735 0.000000 2014-08-13 18:06:06.933816 7f5c973a57a0 20 read_log 194215'17228 (192888'17013) clean 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by osd.356.0:1135 2014-08-12 15:49:16.742388 2014-08-13 18:06:06.933828 7f5c973a57a0 20 read_log 194215'17229 (194215'17228) delete 22a8a750/benchmark_data_storage0111-ib.pocstor_2228_object16661/head//24 by client.2174595.0:174737 0.000000 2014-08-13 18:06:06.933839 7f5c973a57a0 20 read_log 194215'17230 (192888'16819) clean f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by osd.356.0:1137 2014-08-12 15:49:16.747780 2014-08-13 18:06:06.933851 7f5c973a57a0 20 read_log 194215'17231 (194215'17230) delete f356b750/benchmark_data_storage0111-ib.pocstor_41097_object24920/head//24 by client.2174595.0:174739 0.000000 2014-08-13 18:06:06.933871 7f5c973a57a0 20 read_log 194215'17232 (192888'16778) clean 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by osd.356.0:1139 2014-08-12 15:49:16.752999 2014-08-13 18:06:06.933885 7f5c973a57a0 20 read_log 194215'17233 (194215'17232) delete 44cc750/benchmark_data_storage0111-ib.pocstor_41097_object4531/head//24 by client.2174595.0:174741 0.000000 2014-08-13 18:06:06.933896 7f5c973a57a0 20 read_log 194215'17234 (193305'17109) clean 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by osd.356.0:1141 2014-08-12 15:49:16.758453 2014-08-13 18:06:06.933915 7f5c973a57a0 20 read_log 194215'17235 (194215'17234) delete 1693e750/benchmark_data_storage0111-ib.pocstor_62621_object693/head//24 by client.2174595.0:174743 0.000000 2014-08-13 18:06:06.933930 7f5c973a57a0 20 read_log 194215'17236 (193305'17117) clean b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by osd.356.0:1143 2014-08-12 15:49:16.764157 2014-08-13 18:06:06.933943 7f5c973a57a0 20 read_log 194215'17237 (194215'17236) delete b3aaf750/benchmark_data_storage0111-ib.pocstor_62621_object10272/head//24 by client.2174595.0:174745 0.000000 2014-08-13 18:06:06.933954 7f5c973a57a0 20 read_log 194215'17238 (192888'17037) clean 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by osd.356.0:1145 2014-08-12 15:49:16.769482 2014-08-13 18:06:06.933967 7f5c973a57a0 20 read_log 194215'17239 (194215'17238) delete 66a30b50/benchmark_data_storage0111-ib.pocstor_2228_object26492/head//24 by client.2174595.0:174747 0.000000 2014-08-13 18:06:06.933978 7f5c973a57a0 20 read_log 194215'17240 (192888'16809) clean 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by osd.356.0:1147 2014-08-12 15:49:16.775023 2014-08-13 18:06:06.934000 7f5c973a57a0 20 read_log 194215'17241 (194215'17240) delete 78340b50/benchmark_data_storage0111-ib.pocstor_41097_object20796/head//24 by client.2174595.0:174749 0.000000 2014-08-13 18:06:06.934021 7f5c973a57a0 20 read_log 194215'17242 (192888'16775) clean b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by osd.356.0:1149 2014-08-12 15:49:16.780471 2014-08-13 18:06:06.934037 7f5c973a57a0 20 read_log 194215'17243 (194215'17242) delete b1850b50/benchmark_data_storage0111-ib.pocstor_41097_object3339/head//24 by client.2174595.0:174751 0.000000 2014-08-13 18:06:06.934048 7f5c973a57a0 20 read_log 194215'17244 (192888'17020) clean 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by osd.356.0:1151 2014-08-12 15:49:16.786218 2014-08-13 18:06:06.934061 7f5c973a57a0 20 read_log 194215'17245 (194215'17244) delete 7b771b50/benchmark_data_storage0111-ib.pocstor_2228_object20396/head//24 by client.2174595.0:174753 0.000000 2014-08-13 18:06:06.934072 7f5c973a57a0 20 read_log 194215'17246 (192888'17019) clean a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by osd.356.0:1153 2014-08-12 15:49:16.792047 2014-08-13 18:06:06.934084 7f5c973a57a0 20 read_log 194215'17247 (194215'17246) delete a552b50/benchmark_data_storage0111-ib.pocstor_2228_object20310/head//24 by client.2174595.0:174755 0.000000 2014-08-13 18:06:06.934095 7f5c973a57a0 20 read_log 194215'17248 (192888'17012) clean ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by osd.356.0:1155 2014-08-12 15:49:16.797673 2014-08-13 18:06:06.934108 7f5c973a57a0 20 read_log 194215'17249 (194215'17248) delete ce082b50/benchmark_data_storage0111-ib.pocstor_2228_object15657/head//24 by client.2174595.0:174757 0.000000 2014-08-13 18:06:06.934118 7f5c973a57a0 20 read_log 194215'17250 (192888'16982) clean 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by osd.356.0:1157 2014-08-12 15:49:16.803673 2014-08-13 18:06:06.934131 7f5c973a57a0 20 read_log 194215'17251 (194215'17250) delete 239a2b50/benchmark_data_storage0111-ib.pocstor_2228_object1158/head//24 by client.2174595.0:174759 0.000000 2014-08-13 18:06:06.934142 7f5c973a57a0 20 read_log 194215'17252 (193305'17115) clean 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by osd.356.0:1159 2014-08-12 15:49:16.810051 2014-08-13 18:06:06.934170 7f5c973a57a0 20 read_log 194215'17253 (194215'17252) delete 79274b50/benchmark_data_storage0111-ib.pocstor_62621_object8825/head//24 by client.2174595.0:174761 0.000000 2014-08-13 18:06:06.934196 7f5c973a57a0 20 read_log 194215'17254 (192888'17032) clean 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by osd.356.0:1161 2014-08-12 15:49:16.816705 2014-08-13 18:06:06.934209 7f5c973a57a0 20 read_log 194215'17255 (194215'17254) delete 26194b50/benchmark_data_storage0111-ib.pocstor_2228_object24570/head//24 by client.2174595.0:174763 0.000000 2014-08-13 18:06:06.934231 7f5c973a57a0 20 read_log 194215'17256 (193305'17136) clean ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by osd.356.0:1163 2014-08-12 15:49:16.823746 2014-08-13 18:06:06.934244 7f5c973a57a0 20 read_log 194215'17257 (194215'17256) delete ae2b4b50/benchmark_data_storage0111-ib.pocstor_62621_object28424/head//24 by client.2174595.0:174765 0.000000 2014-08-13 18:06:06.934255 7f5c973a57a0 20 read_log 194215'17258 (193305'17114) clean 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by osd.356.0:1165 2014-08-12 15:49:16.830392 2014-08-13 18:06:06.934278 7f5c973a57a0 20 read_log 194215'17259 (194215'17258) delete 12bd4b50/benchmark_data_storage0111-ib.pocstor_62621_object6213/head//24 by client.2174595.0:174767 0.000000 2014-08-13 18:06:06.934292 7f5c973a57a0 20 read_log 194215'17260 (193305'17110) clean d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by osd.356.0:1167 2014-08-12 15:49:16.837126 2014-08-13 18:06:06.934305 7f5c973a57a0 20 read_log 194215'17261 (194215'17260) delete d5e15b50/benchmark_data_storage0111-ib.pocstor_62621_object745/head//24 by client.2174595.0:174769 0.000000 2014-08-13 18:06:06.934332 7f5c973a57a0 20 read_log 194215'17262 (192888'16990) clean a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by osd.356.0:1169 2014-08-12 15:49:16.843927 2014-08-13 18:06:06.934348 7f5c973a57a0 20 read_log 194215'17263 (194215'17262) delete a7925b50/benchmark_data_storage0111-ib.pocstor_2228_object5314/head//24 by client.2174595.0:174771 0.000000 2014-08-13 18:06:06.934364 7f5c973a57a0 20 read_log 194215'17264 (192888'17044) clean e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by osd.356.0:1171 2014-08-12 15:49:16.850556 2014-08-13 18:06:06.934379 7f5c973a57a0 20 read_log 194215'17265 (194215'17264) delete e1246b50/benchmark_data_storage0111-ib.pocstor_2228_object30777/head//24 by client.2174595.0:174773 0.000000 2014-08-13 18:06:06.934390 7f5c973a57a0 20 read_log 194215'17266 (192888'17036) clean 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by osd.356.0:1173 2014-08-12 15:49:16.857886 2014-08-13 18:06:06.934402 7f5c973a57a0 20 read_log 194215'17267 (194215'17266) delete 1b8a7b50/benchmark_data_storage0111-ib.pocstor_2228_object26197/head//24 by client.2174595.0:174775 0.000000 2014-08-13 18:06:06.934413 7f5c973a57a0 20 read_log 194215'17268 (192888'16790) clean 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by osd.356.0:1175 2014-08-12 15:49:16.864360 2014-08-13 18:06:06.934425 7f5c973a57a0 20 read_log 194215'17269 (194215'17268) delete 878dab50/benchmark_data_storage0111-ib.pocstor_41097_object9398/head//24 by client.2174595.0:174777 0.000000 2014-08-13 18:06:06.934449 7f5c973a57a0 20 read_log 194215'17270 (192888'16787) clean 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by osd.356.0:1177 2014-08-12 15:49:16.871107 2014-08-13 18:06:06.934464 7f5c973a57a0 20 read_log 194215'17271 (194215'17270) delete 97cdbb50/benchmark_data_storage0111-ib.pocstor_41097_object7997/head//24 by client.2174595.0:174779 0.000000 2014-08-13 18:06:06.934475 7f5c973a57a0 20 read_log 194215'17272 (192888'16789) clean 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by osd.356.0:1179 2014-08-12 15:49:16.877830 2014-08-13 18:06:06.934492 7f5c973a57a0 20 read_log 194215'17273 (194215'17272) delete 6b34cb50/benchmark_data_storage0111-ib.pocstor_41097_object9025/head//24 by client.2174595.0:174781 0.000000 2014-08-13 18:06:06.934523 7f5c973a57a0 20 read_log 194215'17274 (193305'17123) clean 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by osd.356.0:1181 2014-08-12 15:49:16.884360 2014-08-13 18:06:06.934543 7f5c973a57a0 20 read_log 194215'17275 (194215'17274) delete 8cb4cb50/benchmark_data_storage0111-ib.pocstor_62621_object14043/head//24 by client.2174595.0:174783 0.000000 2014-08-13 18:06:06.934563 7f5c973a57a0 20 read_log 194215'17276 (192888'16986) clean 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by osd.356.0:1183 2014-08-12 15:49:16.890480 2014-08-13 18:06:06.934583 7f5c973a57a0 20 read_log 194215'17277 (194215'17276) delete 7a68cb50/benchmark_data_storage0111-ib.pocstor_2228_object3025/head//24 by client.2174595.0:174785 0.000000 2014-08-13 18:06:06.934594 7f5c973a57a0 20 read_log 194215'17278 (193305'17119) clean 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by osd.356.0:1185 2014-08-12 15:49:16.897175 2014-08-13 18:06:06.934607 7f5c973a57a0 20 read_log 194215'17279 (194215'17278) delete 37f9cb50/benchmark_data_storage0111-ib.pocstor_62621_object11320/head//24 by client.2174595.0:174787 0.000000 2014-08-13 18:06:06.934618 7f5c973a57a0 20 read_log 194215'17280 (192888'16826) clean 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by osd.356.0:1187 2014-08-12 15:49:16.903365 2014-08-13 18:06:06.934630 7f5c973a57a0 20 read_log 194215'17281 (194215'17280) delete 58e1db50/benchmark_data_storage0111-ib.pocstor_41097_object29912/head//24 by client.2174595.0:174789 0.000000 2014-08-13 18:06:06.934648 7f5c973a57a0 20 read_log 194215'17282 (193305'17133) clean c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by osd.356.0:1189 2014-08-12 15:49:16.909796 2014-08-13 18:06:06.934663 7f5c973a57a0 20 read_log 194215'17283 (194215'17282) delete c4ccdb50/benchmark_data_storage0111-ib.pocstor_62621_object23353/head//24 by client.2174595.0:174791 0.000000 2014-08-13 18:06:06.934674 7f5c973a57a0 20 read_log 194215'17284 (192888'16983) clean 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by osd.356.0:1191 2014-08-12 15:49:16.916672 2014-08-13 18:06:06.934698 7f5c973a57a0 20 read_log 194215'17285 (194215'17284) delete 7ce1eb50/benchmark_data_storage0111-ib.pocstor_2228_object1203/head//24 by client.2174595.0:174793 0.000000 2014-08-13 18:06:06.934710 7f5c973a57a0 20 read_log 194215'17286 (192888'17007) clean d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by osd.356.0:1193 2014-08-12 15:49:16.923542 2014-08-13 18:06:06.934722 7f5c973a57a0 20 read_log 194215'17287 (194215'17286) delete d6ed1f50/benchmark_data_storage0111-ib.pocstor_2228_object13021/head//24 by client.2174595.0:174795 0.000000 2014-08-13 18:06:06.934733 7f5c973a57a0 20 read_log 194215'17288 (193305'17128) clean f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by osd.356.0:1195 2014-08-12 15:49:16.930574 2014-08-13 18:06:06.934746 7f5c973a57a0 20 read_log 194215'17289 (194215'17288) delete f07c2f50/benchmark_data_storage0111-ib.pocstor_62621_object18934/head//24 by client.2174595.0:174797 0.000000 2014-08-13 18:06:06.934757 7f5c973a57a0 20 read_log 194215'17290 (192888'17038) clean eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by osd.356.0:1197 2014-08-12 15:49:16.937618 2014-08-13 18:06:06.934770 7f5c973a57a0 20 read_log 194215'17291 (194215'17290) delete eaee2f50/benchmark_data_storage0111-ib.pocstor_2228_object27023/head//24 by client.2174595.0:174799 0.000000 2014-08-13 18:06:06.934781 7f5c973a57a0 20 read_log 194215'17292 (193305'17125) clean cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by osd.356.0:1199 2014-08-12 15:49:16.944471 2014-08-13 18:06:06.934793 7f5c973a57a0 20 read_log 194215'17293 (194215'17292) delete cd653f50/benchmark_data_storage0111-ib.pocstor_62621_object15594/head//24 by client.2174595.0:174801 0.000000 2014-08-13 18:06:06.934804 7f5c973a57a0 20 read_log 194215'17294 (192888'17029) clean a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by osd.356.0:1201 2014-08-12 15:49:16.951230 2014-08-13 18:06:06.934817 7f5c973a57a0 20 read_log 194215'17295 (194215'17294) delete a5183f50/benchmark_data_storage0111-ib.pocstor_2228_object23211/head//24 by client.2174595.0:174803 0.000000 2014-08-13 18:06:06.934827 7f5c973a57a0 20 read_log 194215'17296 (192888'17024) clean 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by osd.356.0:1203 2014-08-12 15:49:16.957915 2014-08-13 18:06:06.934840 7f5c973a57a0 20 read_log 194215'17297 (194215'17296) delete 811b4f50/benchmark_data_storage0111-ib.pocstor_2228_object21354/head//24 by client.2174595.0:174805 0.000000 2014-08-13 18:06:06.934850 7f5c973a57a0 20 read_log 194215'17298 (193305'17112) clean f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by osd.356.0:1205 2014-08-12 15:49:16.965083 2014-08-13 18:06:06.934862 7f5c973a57a0 20 read_log 194215'17299 (194215'17298) delete f96d4f50/benchmark_data_storage0111-ib.pocstor_62621_object5002/head//24 by client.2174595.0:174807 0.000000 2014-08-13 18:06:06.934873 7f5c973a57a0 20 read_log 194215'17300 (193305'17122) clean d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by osd.356.0:1207 2014-08-12 15:49:16.972195 2014-08-13 18:06:06.934886 7f5c973a57a0 20 read_log 194215'17301 (194215'17300) delete d1ad4f50/benchmark_data_storage0111-ib.pocstor_62621_object13728/head//24 by client.2174595.0:174809 0.000000 2014-08-13 18:06:06.934906 7f5c973a57a0 20 read_log 194215'17302 (192888'16817) clean a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by osd.356.0:1209 2014-08-12 15:49:16.979265 2014-08-13 18:06:06.934919 7f5c973a57a0 20 read_log 194215'17303 (194215'17302) delete a855f50/benchmark_data_storage0111-ib.pocstor_41097_object24652/head//24 by client.2174595.0:174811 0.000000 2014-08-13 18:06:06.934930 7f5c973a57a0 20 read_log 194215'17304 (192888'16793) clean 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by osd.356.0:1211 2014-08-12 15:49:16.986441 2014-08-13 18:06:06.934942 7f5c973a57a0 20 read_log 194215'17305 (194215'17304) delete 24656f50/benchmark_data_storage0111-ib.pocstor_41097_object10600/head//24 by client.2174595.0:174813 0.000000 2014-08-13 18:06:06.934953 7f5c973a57a0 20 read_log 194215'17306 (192888'17043) clean dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by osd.356.0:1213 2014-08-12 15:49:16.993442 2014-08-13 18:06:06.934966 7f5c973a57a0 20 read_log 194215'17307 (194215'17306) delete dce87f50/benchmark_data_storage0111-ib.pocstor_2228_object29480/head//24 by client.2174595.0:174815 0.000000 2014-08-13 18:06:06.934977 7f5c973a57a0 20 read_log 194215'17308 (193305'17111) clean fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by osd.356.0:1215 2014-08-12 15:49:17.000131 2014-08-13 18:06:06.934989 7f5c973a57a0 20 read_log 194215'17309 (194215'17308) delete fc258f50/benchmark_data_storage0111-ib.pocstor_62621_object2205/head//24 by client.2174595.0:174817 0.000000 2014-08-13 18:06:06.935000 7f5c973a57a0 20 read_log 194215'17310 (192888'17035) clean 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by osd.356.0:1217 2014-08-12 15:49:17.007034 2014-08-13 18:06:06.935012 7f5c973a57a0 20 read_log 194215'17311 (194215'17310) delete 3a878f50/benchmark_data_storage0111-ib.pocstor_2228_object26018/head//24 by client.2174595.0:174819 0.000000 2014-08-13 18:06:06.935023 7f5c973a57a0 20 read_log 194215'17312 (193305'17116) clean d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by osd.356.0:1219 2014-08-12 15:49:17.013750 2014-08-13 18:06:06.935036 7f5c973a57a0 20 read_log 194215'17313 (194215'17312) delete d8aa8f50/benchmark_data_storage0111-ib.pocstor_62621_object9991/head//24 by client.2174595.0:174821 0.000000 2014-08-13 18:06:06.935047 7f5c973a57a0 20 read_log 194215'17314 (192888'16992) clean 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by osd.356.0:1221 2014-08-12 15:49:17.020804 2014-08-13 18:06:06.935079 7f5c973a57a0 20 read_log 194215'17315 (194215'17314) delete 3f149f50/benchmark_data_storage0111-ib.pocstor_2228_object8050/head//24 by client.2174595.0:174823 0.000000 2014-08-13 18:06:06.935093 7f5c973a57a0 20 read_log 194215'17316 (192888'16997) clean c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by osd.356.0:1223 2014-08-12 15:49:17.027592 2014-08-13 18:06:06.935106 7f5c973a57a0 20 read_log 194215'17317 (194215'17316) delete c431af50/benchmark_data_storage0111-ib.pocstor_2228_object9398/head//24 by client.2174595.0:174825 0.000000 2014-08-13 18:06:06.935117 7f5c973a57a0 20 read_log 194215'17318 (192888'17003) clean 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by osd.356.0:1225 2014-08-12 15:49:17.034180 2014-08-13 18:06:06.935131 7f5c973a57a0 20 read_log 194215'17319 (194215'17318) delete 5663bf50/benchmark_data_storage0111-ib.pocstor_2228_object11941/head//24 by client.2174595.0:174827 0.000000 2014-08-13 18:06:06.935143 7f5c973a57a0 20 read_log 194215'17320 (193305'17137) clean c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by osd.356.0:1227 2014-08-12 15:49:17.041118 2014-08-13 18:06:06.935163 7f5c973a57a0 20 read_log 194215'17321 (194215'17320) delete c08df50/benchmark_data_storage0111-ib.pocstor_62621_object29694/head//24 by client.2174595.0:174829 0.000000 2014-08-13 18:06:06.935182 7f5c973a57a0 20 read_log 194215'17322 (192888'16978) clean 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by osd.356.0:1229 2014-08-12 15:49:17.047858 2014-08-13 18:06:06.935197 7f5c973a57a0 20 read_log 194215'17323 (194215'17322) delete 48a5ef50/benchmark_data_storage0111-ib.pocstor_2228_object592/head//24 by client.2174595.0:174831 0.000000 2014-08-13 18:06:06.935208 7f5c973a57a0 20 read_log 194215'17324 (192888'16823) clean b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by osd.356.0:1231 2014-08-12 15:49:17.054922 2014-08-13 18:06:06.935221 7f5c973a57a0 20 read_log 194215'17325 (194215'17324) delete b9bfef50/benchmark_data_storage0111-ib.pocstor_41097_object27169/head//24 by client.2174595.0:174833 0.000000 2014-08-13 18:06:06.935235 7f5c973a57a0 20 read_log 194215'17326 (192888'16827) clean f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by osd.356.0:1233 2014-08-12 15:49:17.062074 2014-08-13 18:06:06.935247 7f5c973a57a0 20 read_log 194215'17327 (194215'17326) delete f1ecff50/benchmark_data_storage0111-ib.pocstor_41097_object30221/head//24 by client.2174595.0:174835 0.000000 2014-08-13 18:06:06.935263 7f5c973a57a0 20 read_log 194216'17328 (194216'17328) modify 350/hit_set_24.350_archive_2014-08-12 15:49:16.440554_2014-08-12 17:25:03.186720/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.186960 2014-08-13 18:06:06.935278 7f5c973a57a0 20 read_log 194216'17329 (194215'17138) delete 350/hit_set_24.350_archive_2014-08-12 15:32:25.978048_2014-08-12 15:49:16.440311/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.186960 2014-08-13 18:06:06.935291 7f5c973a57a0 20 read_log 194216'17330 (0'0) promote 8eaf2750/benchmark_data_storage0111-ib.pocstor_39693_object14/head//24 by osd.356.0:1772 2014-08-12 17:25:03.190251 2014-08-13 18:06:06.935307 7f5c973a57a0 20 read_log 194216'17331 (194216'17330) modify 8eaf2750/benchmark_data_storage0111-ib.pocstor_39693_object14/head//24 by client.1487927.0:15 2014-08-12 17:25:03.177457 2014-08-13 18:06:06.935320 7f5c973a57a0 20 read_log 194370'17332 (0'0) promote 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by osd.356.0:1779 2014-08-12 20:30:16.257189 2014-08-13 18:06:06.935335 7f5c973a57a0 20 read_log 194370'17333 (194370'17332) modify 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by client.1508531.0:490 2014-08-12 20:30:16.248882 2014-08-13 18:06:06.935348 7f5c973a57a0 20 read_log 194370'17334 (0'0) promote 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by osd.356.0:1787 2014-08-12 20:30:21.179789 2014-08-13 18:06:06.935363 7f5c973a57a0 20 read_log 194370'17335 (194370'17334) modify 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by client.1508531.0:1821 2014-08-12 20:30:21.172164 2014-08-13 18:06:06.935386 7f5c973a57a0 20 read_log 194370'17336 (0'0) promote a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by osd.356.0:1819 2014-08-12 20:30:45.312449 2014-08-13 18:06:06.935406 7f5c973a57a0 20 read_log 194370'17337 (194370'17336) modify a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by client.1508531.0:7578 2014-08-12 20:30:45.303588 2014-08-13 18:06:06.935421 7f5c973a57a0 20 read_log 194370'17338 (0'0) promote 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by osd.356.0:1821 2014-08-12 20:30:46.293447 2014-08-13 18:06:06.935444 7f5c973a57a0 20 read_log 194370'17339 (194370'17338) modify 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by client.1508531.0:7761 2014-08-12 20:30:46.285561 2014-08-13 18:06:06.935457 7f5c973a57a0 20 read_log 194370'17340 (0'0) promote b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by osd.356.0:1823 2014-08-12 20:30:47.384244 2014-08-13 18:06:06.935487 7f5c973a57a0 20 read_log 194370'17341 (194370'17340) modify b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by client.1508531.0:7959 2014-08-12 20:30:47.377375 2014-08-13 18:06:06.935504 7f5c973a57a0 20 read_log 194370'17342 (0'0) promote 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by osd.356.0:1829 2014-08-12 20:30:51.479978 2014-08-13 18:06:06.935517 7f5c973a57a0 20 read_log 194370'17343 (194370'17342) modify 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by client.1508531.0:8704 2014-08-12 20:30:51.472682 2014-08-13 18:06:06.935537 7f5c973a57a0 20 read_log 194370'17344 (0'0) promote b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by osd.356.0:1831 2014-08-12 20:30:52.055736 2014-08-13 18:06:06.935553 7f5c973a57a0 20 read_log 194370'17345 (194370'17344) modify b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by client.1508531.0:8801 2014-08-12 20:30:52.048105 2014-08-13 18:06:06.935565 7f5c973a57a0 20 read_log 194370'17346 (0'0) promote 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by osd.356.0:1863 2014-08-12 20:31:27.913261 2014-08-13 18:06:06.935578 7f5c973a57a0 20 read_log 194370'17347 (194370'17346) modify 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by client.1508531.0:13844 2014-08-12 20:31:27.906128 2014-08-13 18:06:06.935595 7f5c973a57a0 20 read_log 194370'17348 (0'0) promote 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by osd.356.0:1869 2014-08-12 20:31:30.741976 2014-08-13 18:06:06.935610 7f5c973a57a0 20 read_log 194370'17349 (194370'17348) modify 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by client.1508531.0:13946 2014-08-12 20:31:30.733784 2014-08-13 18:06:06.935622 7f5c973a57a0 20 read_log 194370'17350 (0'0) promote 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by osd.356.0:1871 2014-08-12 20:31:32.307468 2014-08-13 18:06:06.935635 7f5c973a57a0 20 read_log 194370'17351 (194370'17350) modify 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by client.1508531.0:14004 2014-08-12 20:31:32.299744 2014-08-13 18:06:06.935647 7f5c973a57a0 20 read_log 194377'17352 (0'0) promote 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by osd.356.0:1873 2014-08-12 20:32:58.005669 2014-08-13 18:06:06.935660 7f5c973a57a0 20 read_log 194377'17353 (194377'17352) modify 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by client.2231190.0:202 2014-08-12 20:32:57.997217 2014-08-13 18:06:06.935672 7f5c973a57a0 20 read_log 194377'17354 (0'0) promote e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by osd.356.0:1885 2014-08-12 20:33:02.781514 2014-08-13 18:06:06.935694 7f5c973a57a0 20 read_log 194377'17355 (194377'17354) modify e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by client.2231190.0:1389 2014-08-12 20:33:02.774558 2014-08-13 18:06:06.935707 7f5c973a57a0 20 read_log 194377'17356 (0'0) promote 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by osd.356.0:1889 2014-08-12 20:33:03.709956 2014-08-13 18:06:06.935720 7f5c973a57a0 20 read_log 194377'17357 (194377'17356) modify 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by client.2231190.0:1638 2014-08-12 20:33:03.702259 2014-08-13 18:06:06.935735 7f5c973a57a0 20 read_log 194377'17358 (0'0) promote 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by osd.356.0:1903 2014-08-12 20:33:12.347777 2014-08-13 18:06:06.935747 7f5c973a57a0 20 read_log 194377'17359 (194377'17358) modify 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by client.2231190.0:3703 2014-08-12 20:33:12.341623 2014-08-13 18:06:06.935760 7f5c973a57a0 20 read_log 194377'17360 (0'0) promote 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by osd.356.0:1907 2014-08-12 20:33:14.272285 2014-08-13 18:06:06.935802 7f5c973a57a0 20 read_log 194377'17361 (194377'17360) modify 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by client.2231190.0:4060 2014-08-12 20:33:14.263601 2014-08-13 18:06:06.935819 7f5c973a57a0 20 read_log 194377'17362 (0'0) promote 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by osd.356.0:1909 2014-08-12 20:33:15.606095 2014-08-13 18:06:06.935831 7f5c973a57a0 20 read_log 194377'17363 (194377'17362) modify 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by client.2231190.0:4283 2014-08-12 20:33:15.597519 2014-08-13 18:06:06.935844 7f5c973a57a0 20 read_log 194379'17364 (0'0) promote 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by osd.356.0:1915 2014-08-12 20:33:20.693640 2014-08-13 18:06:06.935856 7f5c973a57a0 20 read_log 194379'17365 (194379'17364) modify 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by client.2231190.0:4878 2014-08-12 20:33:20.668730 2014-08-13 18:06:06.935869 7f5c973a57a0 20 read_log 194416'17366 (0'0) promote ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by osd.356.0:2097 2014-08-13 15:28:41.588026 2014-08-13 18:06:06.935881 7f5c973a57a0 20 read_log 194416'17367 (194416'17366) modify ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by client.2441958.0:2869 2014-08-13 15:28:41.580397 2014-08-13 18:06:06.935893 7f5c973a57a0 20 read_log 194416'17368 (0'0) promote c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by osd.356.0:2107 2014-08-13 15:28:44.431377 2014-08-13 18:06:06.935906 7f5c973a57a0 20 read_log 194416'17369 (194416'17368) modify c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by client.2441958.0:3617 2014-08-13 15:28:44.423650 2014-08-13 18:06:06.935918 7f5c973a57a0 20 read_log 194416'17370 (0'0) promote 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by osd.356.0:2109 2014-08-13 15:28:45.821212 2014-08-13 18:06:06.935930 7f5c973a57a0 20 read_log 194416'17371 (194416'17370) modify 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by client.2441958.0:3971 2014-08-13 15:28:45.813008 2014-08-13 18:06:06.935942 7f5c973a57a0 20 read_log 194416'17372 (0'0) promote ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by osd.356.0:2115 2014-08-13 15:28:49.749259 2014-08-13 18:06:06.935954 7f5c973a57a0 20 read_log 194416'17373 (194416'17372) modify ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by client.2441958.0:4962 2014-08-13 15:28:49.741277 2014-08-13 18:06:06.935966 7f5c973a57a0 20 read_log 194416'17374 (0'0) promote 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by osd.356.0:2119 2014-08-13 15:28:50.287985 2014-08-13 18:06:06.935978 7f5c973a57a0 20 read_log 194416'17375 (194416'17374) modify 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by client.2441958.0:5104 2014-08-13 15:28:50.281361 2014-08-13 18:06:06.935991 7f5c973a57a0 20 read_log 194416'17376 (0'0) promote f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by osd.356.0:2121 2014-08-13 15:28:50.649857 2014-08-13 18:06:06.936006 7f5c973a57a0 20 read_log 194416'17377 (194416'17376) modify f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by client.2441958.0:5198 2014-08-13 15:28:50.642625 2014-08-13 18:06:06.936019 7f5c973a57a0 20 read_log 194416'17378 (0'0) promote 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by osd.356.0:2139 2014-08-13 15:28:56.509710 2014-08-13 18:06:06.936033 7f5c973a57a0 20 read_log 194416'17379 (194416'17378) modify 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by client.2441958.0:6659 2014-08-13 15:28:56.502292 2014-08-13 18:06:06.936053 7f5c973a57a0 20 read_log 194416'17380 (0'0) promote d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by osd.356.0:2147 2014-08-13 15:29:00.199723 2014-08-13 18:06:06.936068 7f5c973a57a0 20 read_log 194416'17381 (194416'17380) modify d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by client.2441958.0:7646 2014-08-13 15:29:00.192240 2014-08-13 18:06:06.936080 7f5c973a57a0 20 read_log 194416'17382 (0'0) promote a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by osd.356.0:2163 2014-08-13 15:29:05.338299 2014-08-13 18:06:06.936095 7f5c973a57a0 20 read_log 194416'17383 (194416'17382) modify a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by client.2441958.0:8966 2014-08-13 15:29:05.330008 2014-08-13 18:06:06.936110 7f5c973a57a0 20 read_log 194416'17384 (0'0) promote 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by osd.356.0:2179 2014-08-13 15:29:11.139605 2014-08-13 18:06:06.936125 7f5c973a57a0 20 read_log 194416'17385 (194416'17384) modify 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by client.2441958.0:10420 2014-08-13 15:29:11.132288 2014-08-13 18:06:06.936137 7f5c973a57a0 20 read_log 194416'17386 (0'0) promote dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by osd.356.0:2187 2014-08-13 15:29:15.129261 2014-08-13 18:06:06.936165 7f5c973a57a0 20 read_log 194416'17387 (194416'17386) modify dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by client.2441958.0:11410 2014-08-13 15:29:15.119476 2014-08-13 18:06:06.936178 7f5c973a57a0 20 read_log 194416'17388 (0'0) promote 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by osd.356.0:2191 2014-08-13 15:29:19.812140 2014-08-13 18:06:06.936190 7f5c973a57a0 20 read_log 194416'17389 (194416'17388) modify 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by client.2441958.0:12530 2014-08-13 15:29:19.803948 2014-08-13 18:06:06.936203 7f5c973a57a0 20 read_log 194416'17390 (0'0) promote 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by osd.356.0:2195 2014-08-13 15:29:20.587732 2014-08-13 18:06:06.936215 7f5c973a57a0 20 read_log 194416'17391 (194416'17390) modify 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by client.2441958.0:12708 2014-08-13 15:29:20.579787 2014-08-13 18:06:06.936230 7f5c973a57a0 20 read_log 194416'17392 (0'0) promote dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by osd.356.0:2199 2014-08-13 15:29:21.236488 2014-08-13 18:06:06.936242 7f5c973a57a0 20 read_log 194416'17393 (194416'17392) modify dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by client.2441958.0:12847 2014-08-13 15:29:21.228530 2014-08-13 18:06:06.936257 7f5c973a57a0 20 read_log 194416'17394 (0'0) promote ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by osd.356.0:2203 2014-08-13 15:29:22.935524 2014-08-13 18:06:06.936269 7f5c973a57a0 20 read_log 194416'17395 (194416'17394) modify ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by client.2441958.0:13230 2014-08-13 15:29:22.928752 2014-08-13 18:06:06.936283 7f5c973a57a0 20 read_log 194416'17396 (0'0) promote 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by osd.356.0:2207 2014-08-13 15:29:23.938036 2014-08-13 18:06:06.936296 7f5c973a57a0 20 read_log 194416'17397 (194416'17396) modify 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by client.2441958.0:13466 2014-08-13 15:29:23.930792 2014-08-13 18:06:06.936313 7f5c973a57a0 20 read_log 194416'17398 (0'0) promote 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by osd.356.0:2209 2014-08-13 15:29:24.008495 2014-08-13 18:06:06.936326 7f5c973a57a0 20 read_log 194416'17399 (194416'17398) modify 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by client.2441958.0:13481 2014-08-13 15:29:24.001910 2014-08-13 18:06:06.936345 7f5c973a57a0 20 read_log 194416'17400 (0'0) promote 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by osd.356.0:2213 2014-08-13 15:29:26.682067 2014-08-13 18:06:06.936364 7f5c973a57a0 20 read_log 194416'17401 (194416'17400) modify 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by client.2441958.0:14123 2014-08-13 15:29:26.670759 2014-08-13 18:06:06.936377 7f5c973a57a0 20 read_log 194416'17402 (0'0) promote aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by osd.356.0:2225 2014-08-13 15:29:48.419803 2014-08-13 18:06:06.936389 7f5c973a57a0 20 read_log 194416'17403 (194416'17402) modify aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by client.2441958.0:17075 2014-08-13 15:29:48.410568 2014-08-13 18:06:06.936405 7f5c973a57a0 20 read_log 194416'17404 (0'0) promote 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by osd.356.0:2229 2014-08-13 15:29:50.066265 2014-08-13 18:06:06.936420 7f5c973a57a0 20 read_log 194416'17405 (194416'17404) modify 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by client.2441958.0:17192 2014-08-13 15:29:50.058856 2014-08-13 18:06:06.936442 7f5c973a57a0 20 read_log 194416'17406 (0'0) promote ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by osd.356.0:2237 2014-08-13 15:30:00.290538 2014-08-13 18:06:06.936458 7f5c973a57a0 20 read_log 194416'17407 (194416'17406) modify ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by client.2441958.0:17916 2014-08-13 15:30:00.280228 2014-08-13 18:06:06.936516 7f5c973a57a0 20 read_log 194416'17408 (0'0) promote 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by osd.356.0:2239 2014-08-13 15:30:16.335107 2014-08-13 18:06:06.936539 7f5c973a57a0 20 read_log 194416'17409 (194416'17408) modify 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by client.2441958.0:18617 2014-08-13 15:30:16.323541 2014-08-13 18:06:06.936554 7f5c973a57a0 20 read_log 194422'17410 (0'0) promote 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by osd.356.0:2257 2014-08-13 15:32:26.787872 2014-08-13 18:06:06.936567 7f5c973a57a0 20 read_log 194422'17411 (194422'17410) modify 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by client.1148203.0:2916 2014-08-13 15:32:26.777366 2014-08-13 18:06:06.936587 7f5c973a57a0 20 read_log 194422'17412 (0'0) promote 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by osd.356.0:2265 2014-08-13 15:32:27.995798 2014-08-13 18:06:06.936603 7f5c973a57a0 20 read_log 194422'17413 (194422'17412) modify 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by client.1148203.0:3236 2014-08-13 15:32:27.984825 2014-08-13 18:06:06.936616 7f5c973a57a0 20 read_log 194422'17414 (0'0) promote e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by osd.356.0:2267 2014-08-13 15:32:30.898203 2014-08-13 18:06:06.936631 7f5c973a57a0 20 read_log 194422'17415 (194422'17414) modify e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by client.1148203.0:3975 2014-08-13 15:32:30.888992 2014-08-13 18:06:06.936644 7f5c973a57a0 20 read_log 194422'17416 (0'0) promote 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by osd.356.0:2269 2014-08-13 15:32:32.797439 2014-08-13 18:06:06.936656 7f5c973a57a0 20 read_log 194422'17417 (194422'17416) modify 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by client.1148203.0:4468 2014-08-13 15:32:32.789466 2014-08-13 18:06:06.936669 7f5c973a57a0 20 read_log 194422'17418 (0'0) promote 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by osd.356.0:2275 2014-08-13 15:32:35.994587 2014-08-13 18:06:06.936681 7f5c973a57a0 20 read_log 194422'17419 (194422'17418) modify 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by client.1148203.0:5228 2014-08-13 15:32:35.986693 2014-08-13 18:06:06.936704 7f5c973a57a0 20 read_log 194422'17420 (0'0) promote d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by osd.356.0:2277 2014-08-13 15:32:37.054422 2014-08-13 18:06:06.936719 7f5c973a57a0 20 read_log 194422'17421 (194422'17420) modify d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by client.1148203.0:5444 2014-08-13 15:32:37.046145 2014-08-13 18:06:06.936732 7f5c973a57a0 20 read_log 194422'17422 (0'0) promote cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by osd.356.0:2281 2014-08-13 15:32:37.712762 2014-08-13 18:06:06.936744 7f5c973a57a0 20 read_log 194422'17423 (194422'17422) modify cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by client.1148203.0:5556 2014-08-13 15:32:37.704474 2014-08-13 18:06:06.936764 7f5c973a57a0 20 read_log 194422'17424 (0'0) promote 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by osd.356.0:2291 2014-08-13 15:32:44.209205 2014-08-13 18:06:06.936780 7f5c973a57a0 20 read_log 194422'17425 (194422'17424) modify 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by client.1148203.0:6757 2014-08-13 15:32:44.201499 2014-08-13 18:06:06.936793 7f5c973a57a0 20 read_log 194422'17426 (0'0) promote eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by osd.356.0:2293 2014-08-13 15:32:44.973381 2014-08-13 18:06:06.936805 7f5c973a57a0 20 read_log 194422'17427 (194422'17426) modify eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by client.1148203.0:6911 2014-08-13 15:32:44.966068 2014-08-13 18:06:06.936817 7f5c973a57a0 20 read_log 194422'17428 (0'0) promote 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by osd.356.0:2299 2014-08-13 15:32:46.578731 2014-08-13 18:06:06.936829 7f5c973a57a0 20 read_log 194422'17429 (194422'17428) modify 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by client.1148203.0:7169 2014-08-13 15:32:46.567134 2014-08-13 18:06:06.936842 7f5c973a57a0 20 read_log 194422'17430 (0'0) promote 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by osd.356.0:2307 2014-08-13 15:32:54.300993 2014-08-13 18:06:06.936854 7f5c973a57a0 20 read_log 194422'17431 (194422'17430) modify 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by client.1148203.0:8394 2014-08-13 15:32:54.290972 2014-08-13 18:06:06.936866 7f5c973a57a0 20 read_log 194422'17432 (0'0) promote ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by osd.356.0:2309 2014-08-13 15:32:57.088447 2014-08-13 18:06:06.936879 7f5c973a57a0 20 read_log 194422'17433 (194422'17432) modify ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by client.1148203.0:8745 2014-08-13 15:32:57.079855 2014-08-13 18:06:06.936895 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.937272 7f5c973a57a0 10 read_log checking for missing items over interval (194370'17332,194422'17433] 2014-08-13 18:06:06.937321 7f5c973a57a0 15 read_log missing 194422'17433 (194422'17432) modify ff183f50/benchmark_data_storage0111-ib.pocstor_41231_object8744/head//24 by client.1148203.0:8745 2014-08-13 15:32:57.079855 2014-08-13 18:06:06.937366 7f5c973a57a0 15 read_log missing 194422'17431 (194422'17430) modify 61832b50/benchmark_data_storage0111-ib.pocstor_41231_object8393/head//24 by client.1148203.0:8394 2014-08-13 15:32:54.290972 2014-08-13 18:06:06.937403 7f5c973a57a0 15 read_log missing 194422'17429 (194422'17428) modify 157faf50/benchmark_data_storage0111-ib.pocstor_41231_object7168/head//24 by client.1148203.0:7169 2014-08-13 15:32:46.567134 2014-08-13 18:06:06.937452 7f5c973a57a0 15 read_log missing 194422'17427 (194422'17426) modify eee43750/benchmark_data_storage0111-ib.pocstor_41231_object6910/head//24 by client.1148203.0:6911 2014-08-13 15:32:44.966068 2014-08-13 18:06:06.937499 7f5c973a57a0 15 read_log missing 194422'17425 (194422'17424) modify 908ecf50/benchmark_data_storage0111-ib.pocstor_41231_object6756/head//24 by client.1148203.0:6757 2014-08-13 15:32:44.201499 2014-08-13 18:06:06.937536 7f5c973a57a0 15 read_log missing 194422'17423 (194422'17422) modify cbf17f50/benchmark_data_storage0111-ib.pocstor_41231_object5555/head//24 by client.1148203.0:5556 2014-08-13 15:32:37.704474 2014-08-13 18:06:06.937571 7f5c973a57a0 15 read_log missing 194422'17421 (194422'17420) modify d2cf1f50/benchmark_data_storage0111-ib.pocstor_41231_object5443/head//24 by client.1148203.0:5444 2014-08-13 15:32:37.046145 2014-08-13 18:06:06.937607 7f5c973a57a0 15 read_log missing 194422'17419 (194422'17418) modify 90e73b50/benchmark_data_storage0111-ib.pocstor_41231_object5227/head//24 by client.1148203.0:5228 2014-08-13 15:32:35.986693 2014-08-13 18:06:06.937643 7f5c973a57a0 15 read_log missing 194422'17417 (194422'17416) modify 8fa9fb50/benchmark_data_storage0111-ib.pocstor_41231_object4467/head//24 by client.1148203.0:4468 2014-08-13 15:32:32.789466 2014-08-13 18:06:06.937679 7f5c973a57a0 15 read_log missing 194422'17415 (194422'17414) modify e29c5350/benchmark_data_storage0111-ib.pocstor_41231_object3974/head//24 by client.1148203.0:3975 2014-08-13 15:32:30.888992 2014-08-13 18:06:06.937714 7f5c973a57a0 15 read_log missing 194422'17413 (194422'17412) modify 7a0ecb50/benchmark_data_storage0111-ib.pocstor_41231_object3235/head//24 by client.1148203.0:3236 2014-08-13 15:32:27.984825 2014-08-13 18:06:06.937750 7f5c973a57a0 15 read_log missing 194422'17411 (194422'17410) modify 9681ef50/benchmark_data_storage0111-ib.pocstor_41231_object2915/head//24 by client.1148203.0:2916 2014-08-13 15:32:26.777366 2014-08-13 18:06:06.937786 7f5c973a57a0 15 read_log missing 194416'17409 (194416'17408) modify 30291b50/benchmark_data_storage0111-ib.pocstor_19103_object18616/head//24 by client.2441958.0:18617 2014-08-13 15:30:16.323541 2014-08-13 18:06:06.937821 7f5c973a57a0 15 read_log missing 194416'17407 (194416'17406) modify ded5e350/benchmark_data_storage0111-ib.pocstor_19103_object17915/head//24 by client.2441958.0:17916 2014-08-13 15:30:00.280228 2014-08-13 18:06:06.937857 7f5c973a57a0 15 read_log missing 194416'17405 (194416'17404) modify 546d1350/benchmark_data_storage0111-ib.pocstor_19103_object17191/head//24 by client.2441958.0:17192 2014-08-13 15:29:50.058856 2014-08-13 18:06:06.937892 7f5c973a57a0 15 read_log missing 194416'17403 (194416'17402) modify aae5b750/benchmark_data_storage0111-ib.pocstor_19103_object17074/head//24 by client.2441958.0:17075 2014-08-13 15:29:48.410568 2014-08-13 18:06:06.937928 7f5c973a57a0 15 read_log missing 194416'17401 (194416'17400) modify 7f12d750/benchmark_data_storage0111-ib.pocstor_19103_object14122/head//24 by client.2441958.0:14123 2014-08-13 15:29:26.670759 2014-08-13 18:06:06.937964 7f5c973a57a0 15 read_log missing 194416'17399 (194416'17398) modify 4591bb50/benchmark_data_storage0111-ib.pocstor_19103_object13480/head//24 by client.2441958.0:13481 2014-08-13 15:29:24.001910 2014-08-13 18:06:06.938000 7f5c973a57a0 15 read_log missing 194416'17397 (194416'17396) modify 46def50/benchmark_data_storage0111-ib.pocstor_19103_object13465/head//24 by client.2441958.0:13466 2014-08-13 15:29:23.930792 2014-08-13 18:06:06.938036 7f5c973a57a0 15 read_log missing 194416'17395 (194416'17394) modify ed313b50/benchmark_data_storage0111-ib.pocstor_19103_object13229/head//24 by client.2441958.0:13230 2014-08-13 15:29:22.928752 2014-08-13 18:06:06.938071 7f5c973a57a0 15 read_log missing 194416'17393 (194416'17392) modify dcae5750/benchmark_data_storage0111-ib.pocstor_19103_object12846/head//24 by client.2441958.0:12847 2014-08-13 15:29:21.228530 2014-08-13 18:06:06.938107 7f5c973a57a0 15 read_log missing 194416'17391 (194416'17390) modify 17cfd750/benchmark_data_storage0111-ib.pocstor_19103_object12707/head//24 by client.2441958.0:12708 2014-08-13 15:29:20.579787 2014-08-13 18:06:06.938143 7f5c973a57a0 15 read_log missing 194416'17389 (194416'17388) modify 54db3750/benchmark_data_storage0111-ib.pocstor_19103_object12529/head//24 by client.2441958.0:12530 2014-08-13 15:29:19.803948 2014-08-13 18:06:06.938192 7f5c973a57a0 15 read_log missing 194416'17387 (194416'17386) modify dd730350/benchmark_data_storage0111-ib.pocstor_19103_object11409/head//24 by client.2441958.0:11410 2014-08-13 15:29:15.119476 2014-08-13 18:06:06.938236 7f5c973a57a0 15 read_log missing 194416'17385 (194416'17384) modify 7d4b2b50/benchmark_data_storage0111-ib.pocstor_19103_object10419/head//24 by client.2441958.0:10420 2014-08-13 15:29:11.132288 2014-08-13 18:06:06.938273 7f5c973a57a0 15 read_log missing 194416'17383 (194416'17382) modify a5999750/benchmark_data_storage0111-ib.pocstor_19103_object8965/head//24 by client.2441958.0:8966 2014-08-13 15:29:05.330008 2014-08-13 18:06:06.938312 7f5c973a57a0 15 read_log missing 194416'17381 (194416'17380) modify d8a1f750/benchmark_data_storage0111-ib.pocstor_19103_object7645/head//24 by client.2441958.0:7646 2014-08-13 15:29:00.192240 2014-08-13 18:06:06.938348 7f5c973a57a0 15 read_log missing 194416'17379 (194416'17378) modify 708a8350/benchmark_data_storage0111-ib.pocstor_19103_object6658/head//24 by client.2441958.0:6659 2014-08-13 15:28:56.502292 2014-08-13 18:06:06.938389 7f5c973a57a0 15 read_log missing 194416'17377 (194416'17376) modify f9ccaf50/benchmark_data_storage0111-ib.pocstor_19103_object5197/head//24 by client.2441958.0:5198 2014-08-13 15:28:50.642625 2014-08-13 18:06:06.938436 7f5c973a57a0 15 read_log missing 194416'17375 (194416'17374) modify 8cb2e350/benchmark_data_storage0111-ib.pocstor_19103_object5103/head//24 by client.2441958.0:5104 2014-08-13 15:28:50.281361 2014-08-13 18:06:06.938485 7f5c973a57a0 15 read_log missing 194416'17373 (194416'17372) modify ff21cb50/benchmark_data_storage0111-ib.pocstor_19103_object4961/head//24 by client.2441958.0:4962 2014-08-13 15:28:49.741277 2014-08-13 18:06:06.938532 7f5c973a57a0 15 read_log missing 194416'17371 (194416'17370) modify 81878f50/benchmark_data_storage0111-ib.pocstor_19103_object3970/head//24 by client.2441958.0:3971 2014-08-13 15:28:45.813008 2014-08-13 18:06:06.938571 7f5c973a57a0 15 read_log missing 194416'17369 (194416'17368) modify c79cdb50/benchmark_data_storage0111-ib.pocstor_19103_object3616/head//24 by client.2441958.0:3617 2014-08-13 15:28:44.423650 2014-08-13 18:06:06.938607 7f5c973a57a0 15 read_log missing 194416'17367 (194416'17366) modify ad74ef50/benchmark_data_storage0111-ib.pocstor_19103_object2868/head//24 by client.2441958.0:2869 2014-08-13 15:28:41.580397 2014-08-13 18:06:06.938643 7f5c973a57a0 15 read_log missing 194379'17365 (194379'17364) modify 65464350/benchmark_data_storage0111-ib.pocstor_25065_object4877/head//24 by client.2231190.0:4878 2014-08-12 20:33:20.668730 2014-08-13 18:06:06.938689 7f5c973a57a0 15 read_log missing 194377'17363 (194377'17362) modify 566d6350/benchmark_data_storage0111-ib.pocstor_25065_object4282/head//24 by client.2231190.0:4283 2014-08-12 20:33:15.597519 2014-08-13 18:06:06.938726 7f5c973a57a0 15 read_log missing 194377'17361 (194377'17360) modify 90449350/benchmark_data_storage0111-ib.pocstor_25065_object4059/head//24 by client.2231190.0:4060 2014-08-12 20:33:14.263601 2014-08-13 18:06:06.938768 7f5c973a57a0 15 read_log missing 194377'17359 (194377'17358) modify 8125d750/benchmark_data_storage0111-ib.pocstor_25065_object3702/head//24 by client.2231190.0:3703 2014-08-12 20:33:12.341623 2014-08-13 18:06:06.938805 7f5c973a57a0 15 read_log missing 194377'17357 (194377'17356) modify 94141b50/benchmark_data_storage0111-ib.pocstor_25065_object1637/head//24 by client.2231190.0:1638 2014-08-12 20:33:03.702259 2014-08-13 18:06:06.938854 7f5c973a57a0 15 read_log missing 194377'17355 (194377'17354) modify e409eb50/benchmark_data_storage0111-ib.pocstor_25065_object1388/head//24 by client.2231190.0:1389 2014-08-12 20:33:02.774558 2014-08-13 18:06:06.938892 7f5c973a57a0 15 read_log missing 194377'17353 (194377'17352) modify 829bd350/benchmark_data_storage0111-ib.pocstor_25065_object201/head//24 by client.2231190.0:202 2014-08-12 20:32:57.997217 2014-08-13 18:06:06.938929 7f5c973a57a0 15 read_log missing 194370'17351 (194370'17350) modify 30426750/benchmark_data_storage0111-ib.pocstor_12495_object14003/head//24 by client.1508531.0:14004 2014-08-12 20:31:32.299744 2014-08-13 18:06:06.938966 7f5c973a57a0 15 read_log missing 194370'17349 (194370'17348) modify 2d4bf750/benchmark_data_storage0111-ib.pocstor_12495_object13945/head//24 by client.1508531.0:13946 2014-08-12 20:31:30.733784 2014-08-13 18:06:06.939002 7f5c973a57a0 15 read_log missing 194370'17347 (194370'17346) modify 3b4c5350/benchmark_data_storage0111-ib.pocstor_12495_object13843/head//24 by client.1508531.0:13844 2014-08-12 20:31:27.906128 2014-08-13 18:06:06.939038 7f5c973a57a0 15 read_log missing 194370'17345 (194370'17344) modify b8105f50/benchmark_data_storage0111-ib.pocstor_12495_object8800/head//24 by client.1508531.0:8801 2014-08-12 20:30:52.048105 2014-08-13 18:06:06.939075 7f5c973a57a0 15 read_log missing 194370'17343 (194370'17342) modify 5d522750/benchmark_data_storage0111-ib.pocstor_12495_object8703/head//24 by client.1508531.0:8704 2014-08-12 20:30:51.472682 2014-08-13 18:06:06.939111 7f5c973a57a0 15 read_log missing 194370'17341 (194370'17340) modify b38ecf50/benchmark_data_storage0111-ib.pocstor_12495_object7958/head//24 by client.1508531.0:7959 2014-08-12 20:30:47.377375 2014-08-13 18:06:06.939152 7f5c973a57a0 15 read_log missing 194370'17339 (194370'17338) modify 22d21750/benchmark_data_storage0111-ib.pocstor_12495_object7760/head//24 by client.1508531.0:7761 2014-08-12 20:30:46.285561 2014-08-13 18:06:06.939190 7f5c973a57a0 15 read_log missing 194370'17337 (194370'17336) modify a751c350/benchmark_data_storage0111-ib.pocstor_12495_object7577/head//24 by client.1508531.0:7578 2014-08-12 20:30:45.303588 2014-08-13 18:06:06.939226 7f5c973a57a0 15 read_log missing 194370'17335 (194370'17334) modify 6cbfd350/benchmark_data_storage0111-ib.pocstor_12495_object1820/head//24 by client.1508531.0:1821 2014-08-12 20:30:21.172164 2014-08-13 18:06:06.939262 7f5c973a57a0 15 read_log missing 194370'17333 (194370'17332) modify 71e6750/benchmark_data_storage0111-ib.pocstor_12495_object489/head//24 by client.1508531.0:490 2014-08-12 20:30:16.248882 2014-08-13 18:06:06.939273 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.939301 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=0 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive m=51] handle_loaded 2014-08-13 18:06:06.939319 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=0 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Initial 0.013496 0 0.000000 2014-08-13 18:06:06.939331 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=0 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Reset 2014-08-13 18:06:06.939344 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=0 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194487 2014-08-13 18:06:06.939354 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log((192834'15725,194422'17433], crt=194422'17433) 2014-08-13 18:06:06.939364 7f5c973a57a0 10 osd.4 194487 pgid 24.369 coll 24.369_head 2014-08-13 18:06:06.939541 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.369 2014-08-13 18:06:06.939554 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.939588 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.369(unlocked)] enter Initial 2014-08-13 18:06:06.939600 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.369(unlocked)] enter NotTrimming 2014-08-13 18:06:06.939765 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.939887 7f5c973a57a0 20 read_log 192834'11429 (192834'11428) modify 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.933448.0:2663 2014-08-11 15:23:47.808008 2014-08-13 18:06:06.939908 7f5c973a57a0 20 read_log 192834'11442 (0'0) promote 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.338.0:12956 2014-08-11 15:23:57.605666 2014-08-13 18:06:06.939922 7f5c973a57a0 20 read_log 192834'11443 (192834'11442) modify 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by client.933448.0:4560 2014-08-11 15:23:57.594074 2014-08-13 18:06:06.939935 7f5c973a57a0 20 read_log 192834'11450 (0'0) promote 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:12964 2014-08-11 15:23:59.286759 2014-08-13 18:06:06.939948 7f5c973a57a0 20 read_log 192834'11451 (192834'11450) modify 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by client.933448.0:4905 2014-08-11 15:23:59.277462 2014-08-13 18:06:06.939963 7f5c973a57a0 20 read_log 192834'11470 (0'0) promote 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by osd.338.0:12984 2014-08-11 15:24:03.061694 2014-08-13 18:06:06.939976 7f5c973a57a0 20 read_log 192834'11471 (192834'11470) modify 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.933448.0:5834 2014-08-11 15:24:03.051794 2014-08-13 18:06:06.939989 7f5c973a57a0 20 read_log 192834'11498 (0'0) promote 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by osd.338.0:13012 2014-08-11 15:24:07.808801 2014-08-13 18:06:06.940001 7f5c973a57a0 20 read_log 192834'11499 (192834'11498) modify 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.933448.0:7029 2014-08-11 15:24:07.800422 2014-08-13 18:06:06.940014 7f5c973a57a0 20 read_log 192834'11508 (0'0) promote 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by osd.338.0:13022 2014-08-11 15:24:10.567263 2014-08-13 18:06:06.940026 7f5c973a57a0 20 read_log 192834'11509 (192834'11508) modify 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.933448.0:7566 2014-08-11 15:24:10.558757 2014-08-13 18:06:06.940039 7f5c973a57a0 20 read_log 192834'11516 (0'0) promote e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13030 2014-08-11 15:24:12.516818 2014-08-13 18:06:06.940051 7f5c973a57a0 20 read_log 192834'11517 (192834'11516) modify e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by client.933448.0:8052 2014-08-11 15:24:12.508898 2014-08-13 18:06:06.940064 7f5c973a57a0 20 read_log 192834'11578 (0'0) promote 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:13092 2014-08-11 15:24:25.398774 2014-08-13 18:06:06.940076 7f5c973a57a0 20 read_log 192834'11579 (192834'11578) modify 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by client.933448.0:10829 2014-08-11 15:24:25.389836 2014-08-13 18:06:06.940088 7f5c973a57a0 20 read_log 192834'11580 (0'0) promote 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:13094 2014-08-11 15:24:25.877238 2014-08-13 18:06:06.940101 7f5c973a57a0 20 read_log 192834'11581 (192834'11580) modify 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by client.933448.0:10953 2014-08-11 15:24:25.868009 2014-08-13 18:06:06.940113 7f5c973a57a0 20 read_log 192834'11584 (0'0) promote a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13098 2014-08-11 15:24:26.974271 2014-08-13 18:06:06.940135 7f5c973a57a0 20 read_log 192834'11585 (192834'11584) modify a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by client.933448.0:11241 2014-08-11 15:24:26.965943 2014-08-13 18:06:06.940159 7f5c973a57a0 20 read_log 192834'11590 (0'0) promote 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13104 2014-08-11 15:24:31.241056 2014-08-13 18:06:06.940174 7f5c973a57a0 20 read_log 192834'11591 (192834'11590) modify 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by client.933448.0:12140 2014-08-11 15:24:31.232396 2014-08-13 18:06:06.940187 7f5c973a57a0 20 read_log 192834'11604 (0'0) promote 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by osd.338.0:13118 2014-08-11 15:24:39.601167 2014-08-13 18:06:06.940200 7f5c973a57a0 20 read_log 192834'11605 (192834'11604) modify 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.933448.0:13689 2014-08-11 15:24:39.594556 2014-08-13 18:06:06.940213 7f5c973a57a0 20 read_log 192834'11624 (0'0) promote 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13138 2014-08-11 15:24:47.654738 2014-08-13 18:06:06.940225 7f5c973a57a0 20 read_log 192834'11625 (192834'11624) modify 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by client.933448.0:14907 2014-08-11 15:24:47.646644 2014-08-13 18:06:06.940238 7f5c973a57a0 20 read_log 192834'11638 (0'0) promote 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by osd.338.0:13152 2014-08-11 15:24:49.377263 2014-08-13 18:06:06.940250 7f5c973a57a0 20 read_log 192834'11639 (192834'11638) modify 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.933448.0:15363 2014-08-11 15:24:49.369581 2014-08-13 18:06:06.940263 7f5c973a57a0 20 read_log 192834'11672 (0'0) promote f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13186 2014-08-11 15:24:58.657771 2014-08-13 18:06:06.940275 7f5c973a57a0 20 read_log 192834'11673 (192834'11672) modify f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by client.933448.0:17157 2014-08-11 15:24:58.649479 2014-08-13 18:06:06.940291 7f5c973a57a0 20 read_log 192834'11684 (0'0) promote 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13198 2014-08-11 15:25:05.515490 2014-08-13 18:06:06.940304 7f5c973a57a0 20 read_log 192834'11685 (192834'11684) modify 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by client.933448.0:18491 2014-08-11 15:25:05.506978 2014-08-13 18:06:06.940316 7f5c973a57a0 20 read_log 192834'11692 (0'0) promote 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by osd.338.0:13206 2014-08-11 15:25:07.965562 2014-08-13 18:06:06.940329 7f5c973a57a0 20 read_log 192834'11693 (192834'11692) modify 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.933448.0:18992 2014-08-11 15:25:07.957170 2014-08-13 18:06:06.940345 7f5c973a57a0 20 read_log 192834'11694 (0'0) promote 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by osd.338.0:13208 2014-08-11 15:25:08.697584 2014-08-13 18:06:06.940358 7f5c973a57a0 20 read_log 192834'11695 (192834'11694) modify 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.933448.0:19120 2014-08-11 15:25:08.688746 2014-08-13 18:06:06.940371 7f5c973a57a0 20 read_log 192834'11698 (0'0) promote bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:13212 2014-08-11 15:25:11.378619 2014-08-13 18:06:06.940386 7f5c973a57a0 20 read_log 192834'11699 (192834'11698) modify bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by client.933448.0:19298 2014-08-11 15:25:11.370658 2014-08-13 18:06:06.940399 7f5c973a57a0 20 read_log 192834'11704 (0'0) promote f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by osd.338.0:13218 2014-08-11 15:25:12.090591 2014-08-13 18:06:06.940464 7f5c973a57a0 20 read_log 192834'11705 (192834'11704) modify f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.933448.0:19449 2014-08-11 15:25:12.081508 2014-08-13 18:06:06.940484 7f5c973a57a0 20 read_log 192834'11720 (0'0) promote 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by osd.338.0:13234 2014-08-11 15:25:22.875983 2014-08-13 18:06:06.940497 7f5c973a57a0 20 read_log 192834'11721 (192834'11720) modify 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.933448.0:20678 2014-08-11 15:25:22.866990 2014-08-13 18:06:06.940512 7f5c973a57a0 20 read_log 192834'11749 (192834'11429) modify 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.933448.0:24954 2014-08-11 15:25:41.571332 2014-08-13 18:06:06.940525 7f5c973a57a0 20 read_log 192834'11756 (192834'11443) modify 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by client.933448.0:26851 2014-08-11 15:25:43.816649 2014-08-13 18:06:06.940538 7f5c973a57a0 20 read_log 192834'11760 (192834'11451) modify 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by client.933448.0:27196 2014-08-11 15:25:44.207179 2014-08-13 18:06:06.940550 7f5c973a57a0 20 read_log 192834'11770 (192834'11471) modify 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.933448.0:28125 2014-08-11 15:25:45.530060 2014-08-13 18:06:06.940563 7f5c973a57a0 20 read_log 192834'11784 (192834'11499) modify 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.933448.0:29320 2014-08-11 15:25:47.216754 2014-08-13 18:06:06.940578 7f5c973a57a0 20 read_log 192834'11789 (192834'11509) modify 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.933448.0:29857 2014-08-11 15:25:48.014366 2014-08-13 18:06:06.940593 7f5c973a57a0 20 read_log 192834'11793 (192834'11517) modify e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by client.933448.0:30343 2014-08-11 15:25:48.760690 2014-08-13 18:06:06.940606 7f5c973a57a0 20 read_log 192834'11824 (192834'11579) modify 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by client.933448.0:33120 2014-08-11 15:25:52.867364 2014-08-13 18:06:06.940619 7f5c973a57a0 20 read_log 192834'11825 (192834'11581) modify 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by client.933448.0:33244 2014-08-11 15:25:53.112138 2014-08-13 18:06:06.940631 7f5c973a57a0 20 read_log 192834'11827 (192834'11585) modify a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by client.933448.0:33532 2014-08-11 15:25:53.640855 2014-08-13 18:06:06.940644 7f5c973a57a0 20 read_log 192834'11830 (192834'11591) modify 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by client.933448.0:34431 2014-08-11 15:25:55.004758 2014-08-13 18:06:06.940664 7f5c973a57a0 20 read_log 192834'11837 (192834'11605) modify 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.933448.0:35980 2014-08-11 15:25:57.099456 2014-08-13 18:06:06.940680 7f5c973a57a0 20 read_log 192834'11847 (192834'11625) modify 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by client.933448.0:37198 2014-08-11 15:25:59.107351 2014-08-13 18:06:06.940693 7f5c973a57a0 20 read_log 192834'11854 (192834'11639) modify 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.933448.0:37654 2014-08-11 15:25:59.811219 2014-08-13 18:06:06.940706 7f5c973a57a0 20 read_log 192834'11871 (192834'11673) modify f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by client.933448.0:39448 2014-08-11 15:26:01.882970 2014-08-13 18:06:06.940718 7f5c973a57a0 20 read_log 192834'11877 (192834'11685) modify 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by client.933448.0:40782 2014-08-11 15:26:02.729495 2014-08-13 18:06:06.940731 7f5c973a57a0 20 read_log 192834'11881 (192834'11693) modify 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.933448.0:41283 2014-08-11 15:26:02.896172 2014-08-13 18:06:06.940754 7f5c973a57a0 20 read_log 192834'11882 (192834'11695) modify 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.933448.0:41411 2014-08-11 15:26:02.935495 2014-08-13 18:06:06.940770 7f5c973a57a0 20 read_log 192834'11884 (192834'11699) modify bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by client.933448.0:41589 2014-08-11 15:26:02.991009 2014-08-13 18:06:06.940783 7f5c973a57a0 20 read_log 192834'11887 (192834'11705) modify f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.933448.0:41740 2014-08-11 15:26:03.037888 2014-08-13 18:06:06.940795 7f5c973a57a0 20 read_log 192834'11895 (192834'11721) modify 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.933448.0:42969 2014-08-11 15:26:03.458620 2014-08-13 18:06:06.940829 7f5c973a57a0 20 read_log 192834'11916 (0'0) promote eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13430 2014-08-11 15:26:18.477909 2014-08-13 18:06:06.940844 7f5c973a57a0 20 read_log 192834'11917 (192834'11916) modify eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by client.1883634.0:1097 2014-08-11 15:26:18.469033 2014-08-13 18:06:06.940857 7f5c973a57a0 20 read_log 192834'11924 (0'0) promote d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.338.0:13438 2014-08-11 15:26:19.788157 2014-08-13 18:06:06.940870 7f5c973a57a0 20 read_log 192834'11925 (192834'11924) modify d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by client.1883634.0:1459 2014-08-11 15:26:19.778352 2014-08-13 18:06:06.940882 7f5c973a57a0 20 read_log 192834'11940 (0'0) promote 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:13454 2014-08-11 15:26:22.943425 2014-08-13 18:06:06.940895 7f5c973a57a0 20 read_log 192834'11941 (192834'11940) modify 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.1883634.0:2290 2014-08-11 15:26:22.932366 2014-08-13 18:06:06.940907 7f5c973a57a0 20 read_log 192834'11946 (0'0) promote 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:13460 2014-08-11 15:26:23.008433 2014-08-13 18:06:06.940919 7f5c973a57a0 20 read_log 192834'11947 (192834'11946) modify 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.1883634.0:2308 2014-08-11 15:26:22.999247 2014-08-13 18:06:06.940932 7f5c973a57a0 20 read_log 192834'11978 (0'0) promote 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by osd.338.0:13492 2014-08-11 15:26:31.461640 2014-08-13 18:06:06.940944 7f5c973a57a0 20 read_log 192834'11979 (192834'11978) modify 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.1883634.0:4671 2014-08-11 15:26:31.451471 2014-08-13 18:06:06.940956 7f5c973a57a0 20 read_log 192834'12002 (0'0) promote 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by osd.338.0:13516 2014-08-11 15:26:36.724030 2014-08-13 18:06:06.940968 7f5c973a57a0 20 read_log 192834'12003 (192834'12002) modify 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.1883634.0:6087 2014-08-11 15:26:36.714638 2014-08-13 18:06:06.940983 7f5c973a57a0 20 read_log 192834'12020 (0'0) promote 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by osd.338.0:13534 2014-08-11 15:26:39.451719 2014-08-13 18:06:06.940996 7f5c973a57a0 20 read_log 192834'12021 (192834'12020) modify 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.1883634.0:6771 2014-08-11 15:26:39.443036 2014-08-13 18:06:06.941010 7f5c973a57a0 20 read_log 192834'12022 (0'0) promote fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13536 2014-08-11 15:26:39.526174 2014-08-13 18:06:06.941023 7f5c973a57a0 20 read_log 192834'12023 (192834'12022) modify fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by client.1883634.0:6793 2014-08-11 15:26:39.518675 2014-08-13 18:06:06.941045 7f5c973a57a0 20 read_log 192834'12028 (0'0) promote 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:13542 2014-08-11 15:26:40.281637 2014-08-13 18:06:06.941060 7f5c973a57a0 20 read_log 192834'12029 (192834'12028) modify 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.1883634.0:6979 2014-08-11 15:26:40.272345 2014-08-13 18:06:06.941073 7f5c973a57a0 20 read_log 192834'12038 (0'0) promote 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by osd.338.0:13552 2014-08-11 15:26:40.865155 2014-08-13 18:06:06.941088 7f5c973a57a0 20 read_log 192834'12039 (192834'12038) modify 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.1883634.0:7114 2014-08-11 15:26:40.857517 2014-08-13 18:06:06.941102 7f5c973a57a0 20 read_log 192834'12042 (0'0) promote d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by osd.338.0:13556 2014-08-11 15:26:44.309377 2014-08-13 18:06:06.941115 7f5c973a57a0 20 read_log 192834'12043 (192834'12042) modify d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.1883634.0:7914 2014-08-11 15:26:44.300640 2014-08-13 18:06:06.941129 7f5c973a57a0 20 read_log 192834'12044 (0'0) promote d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:13558 2014-08-11 15:26:45.357941 2014-08-13 18:06:06.941141 7f5c973a57a0 20 read_log 192834'12045 (192834'12044) modify d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.1883634.0:8194 2014-08-11 15:26:45.347158 2014-08-13 18:06:06.941162 7f5c973a57a0 20 read_log 192834'12060 (0'0) promote 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by osd.338.0:13574 2014-08-11 15:26:49.331038 2014-08-13 18:06:06.941175 7f5c973a57a0 20 read_log 192834'12061 (192834'12060) modify 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.1883634.0:9084 2014-08-11 15:26:49.320720 2014-08-13 18:06:06.941188 7f5c973a57a0 20 read_log 192834'12094 (0'0) promote ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:13608 2014-08-11 15:26:55.528921 2014-08-13 18:06:06.941200 7f5c973a57a0 20 read_log 192834'12095 (192834'12094) modify ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.1883634.0:10536 2014-08-11 15:26:55.519729 2014-08-13 18:06:06.941214 7f5c973a57a0 20 read_log 192834'12104 (0'0) promote f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by osd.338.0:13618 2014-08-11 15:26:59.046657 2014-08-13 18:06:06.941227 7f5c973a57a0 20 read_log 192834'12105 (192834'12104) modify f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.1883634.0:11257 2014-08-11 15:26:59.037343 2014-08-13 18:06:06.941242 7f5c973a57a0 20 read_log 192834'12126 (0'0) promote b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by osd.338.0:13640 2014-08-11 15:27:05.105671 2014-08-13 18:06:06.941257 7f5c973a57a0 20 read_log 192834'12127 (192834'12126) modify b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.1883634.0:12646 2014-08-11 15:27:05.095901 2014-08-13 18:06:06.941271 7f5c973a57a0 20 read_log 192834'12156 (0'0) promote dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13670 2014-08-11 15:27:11.126437 2014-08-13 18:06:06.941284 7f5c973a57a0 20 read_log 192834'12157 (192834'12156) modify dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by client.1883634.0:13888 2014-08-11 15:27:11.117446 2014-08-13 18:06:06.941298 7f5c973a57a0 20 read_log 192834'12174 (0'0) promote c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.338.0:13688 2014-08-11 15:27:17.618008 2014-08-13 18:06:06.941314 7f5c973a57a0 20 read_log 192834'12175 (192834'12174) modify c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by client.1883634.0:15197 2014-08-11 15:27:17.607226 2014-08-13 18:06:06.941336 7f5c973a57a0 20 read_log 192834'12196 (0'0) promote c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by osd.338.0:13710 2014-08-11 15:27:28.400497 2014-08-13 18:06:06.941351 7f5c973a57a0 20 read_log 192834'12197 (192834'12196) modify c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.1883634.0:16596 2014-08-11 15:27:28.391074 2014-08-13 18:06:06.941368 7f5c973a57a0 20 read_log 192834'12198 (0'0) promote 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.338.0:13712 2014-08-11 15:27:28.591588 2014-08-13 18:06:06.941380 7f5c973a57a0 20 read_log 192834'12199 (192834'12198) modify 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by client.1883634.0:16637 2014-08-11 15:27:28.581804 2014-08-13 18:06:06.941393 7f5c973a57a0 20 read_log 192834'12216 (0'0) promote 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.338.0:13730 2014-08-11 15:27:38.818154 2014-08-13 18:06:06.941408 7f5c973a57a0 20 read_log 192834'12217 (192834'12216) modify 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by client.1883634.0:17742 2014-08-11 15:27:38.808875 2014-08-13 18:06:06.941423 7f5c973a57a0 20 read_log 192834'12256 (0'0) promote bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.338.0:13770 2014-08-11 15:27:50.136394 2014-08-13 18:06:06.941445 7f5c973a57a0 20 read_log 192834'12257 (192834'12256) modify bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by client.1883634.0:19259 2014-08-11 15:27:50.127232 2014-08-13 18:06:06.941462 7f5c973a57a0 20 read_log 192834'12272 (0'0) promote 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by osd.338.0:13786 2014-08-11 15:27:55.567259 2014-08-13 18:06:06.941477 7f5c973a57a0 20 read_log 192834'12273 (192834'12272) modify 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.1883634.0:20183 2014-08-11 15:27:55.557801 2014-08-13 18:06:06.941500 7f5c973a57a0 20 read_log 192834'12332 (0'0) promote f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.338.0:13846 2014-08-11 15:28:03.717752 2014-08-13 18:06:06.941513 7f5c973a57a0 20 read_log 192834'12333 (192834'12332) modify f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by client.1883634.0:21298 2014-08-11 15:28:03.708896 2014-08-13 18:06:06.941526 7f5c973a57a0 20 read_log 192834'12342 (0'0) promote 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.338.0:13856 2014-08-11 15:28:06.723358 2014-08-13 18:06:06.941554 7f5c973a57a0 20 read_log 192834'12343 (192834'12342) modify 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by client.1883634.0:22104 2014-08-11 15:28:06.712110 2014-08-13 18:06:06.941570 7f5c973a57a0 20 read_log 192834'12364 (0'0) promote 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by osd.338.0:13878 2014-08-11 15:28:08.856462 2014-08-13 18:06:06.941585 7f5c973a57a0 20 read_log 192834'12365 (192834'12364) modify 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.1883634.0:22473 2014-08-11 15:28:08.847311 2014-08-13 18:06:06.941600 7f5c973a57a0 20 read_log 192834'12367 (192834'11760) clean 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:13881 2014-08-11 15:28:08.882193 2014-08-13 18:06:06.941612 7f5c973a57a0 20 read_log 192834'12369 (192834'11793) clean e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13883 2014-08-11 15:28:09.467476 2014-08-13 18:06:06.941625 7f5c973a57a0 20 read_log 192834'12371 (192834'11877) clean 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13885 2014-08-11 15:28:09.814756 2014-08-13 18:06:06.941651 7f5c973a57a0 20 read_log 192834'12372 (192834'12023) clean fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13886 2014-08-11 15:28:09.815099 2014-08-13 18:06:06.941679 7f5c973a57a0 20 read_log 192834'12374 (192834'11827) clean a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13888 2014-08-11 15:28:09.832537 2014-08-13 18:06:06.941694 7f5c973a57a0 20 read_log 192834'12375 (192834'11830) clean 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13889 2014-08-11 15:28:09.832832 2014-08-13 18:06:06.941707 7f5c973a57a0 20 read_log 192834'12376 (192834'11847) clean 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13890 2014-08-11 15:28:09.851877 2014-08-13 18:06:06.941727 7f5c973a57a0 20 read_log 192834'12380 (192834'11917) clean eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13894 2014-08-11 15:28:09.867380 2014-08-13 18:06:06.941743 7f5c973a57a0 20 read_log 192834'12400 (192834'12367) delete 25583369/benchmark_data_storage0111-ib.pocstor_26659_object4904/head//24 by osd.338.0:13914 2014-08-11 15:28:10.054995 2014-08-13 18:06:06.941756 7f5c973a57a0 20 read_log 192834'12401 (192834'12372) delete fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:13915 2014-08-11 15:28:10.055489 2014-08-13 18:06:06.941769 7f5c973a57a0 20 read_log 192834'12402 (192834'12369) delete e5437369/benchmark_data_storage0111-ib.pocstor_26659_object8051/head//24 by osd.338.0:13916 2014-08-11 15:28:10.055780 2014-08-13 18:06:06.941781 7f5c973a57a0 20 read_log 192834'12403 (192834'12380) delete eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:13917 2014-08-11 15:28:10.059713 2014-08-13 18:06:06.941794 7f5c973a57a0 20 read_log 192834'12404 (192834'12371) delete 4dcad369/benchmark_data_storage0111-ib.pocstor_26659_object18490/head//24 by osd.338.0:13918 2014-08-11 15:28:10.061758 2014-08-13 18:06:06.941806 7f5c973a57a0 20 read_log 192834'12405 (192834'12374) delete a500e369/benchmark_data_storage0111-ib.pocstor_26659_object11240/head//24 by osd.338.0:13919 2014-08-11 15:28:10.062470 2014-08-13 18:06:06.941821 7f5c973a57a0 20 read_log 192834'12406 (192834'12375) delete 8f2be369/benchmark_data_storage0111-ib.pocstor_26659_object12139/head//24 by osd.338.0:13920 2014-08-11 15:28:10.062825 2014-08-13 18:06:06.941834 7f5c973a57a0 20 read_log 192834'12407 (192834'12376) delete 569af369/benchmark_data_storage0111-ib.pocstor_26659_object14906/head//24 by osd.338.0:13921 2014-08-11 15:28:10.067817 2014-08-13 18:06:06.941847 7f5c973a57a0 20 read_log 192834'12428 (0'0) promote b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by osd.338.0:13942 2014-08-11 15:28:12.300144 2014-08-13 18:06:06.941859 7f5c973a57a0 20 read_log 192834'12429 (192834'12428) modify b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.1883634.0:23055 2014-08-11 15:28:12.289580 2014-08-13 18:06:06.941872 7f5c973a57a0 20 read_log 192834'12432 (0'0) promote 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by osd.338.0:13946 2014-08-11 15:28:13.602545 2014-08-13 18:06:06.941884 7f5c973a57a0 20 read_log 192834'12433 (192834'12432) modify 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.1883634.0:23243 2014-08-11 15:28:13.593326 2014-08-13 18:06:06.941899 7f5c973a57a0 20 read_log 192834'12439 (192834'12157) clean dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13953 2014-08-11 15:28:15.576065 2014-08-13 18:06:06.941911 7f5c973a57a0 20 read_log 192834'12454 (192834'11871) clean f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13968 2014-08-11 15:28:16.102118 2014-08-13 18:06:06.941924 7f5c973a57a0 20 read_log 192834'12455 (192834'11884) clean bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:13969 2014-08-11 15:28:16.110089 2014-08-13 18:06:06.941936 7f5c973a57a0 20 read_log 192834'12457 (192834'12095) clean ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:13971 2014-08-11 15:28:16.312186 2014-08-13 18:06:06.941965 7f5c973a57a0 20 read_log 192834'12458 (192834'12045) clean d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:13972 2014-08-11 15:28:16.312513 2014-08-13 18:06:06.941982 7f5c973a57a0 20 read_log 192834'12460 (192834'11941) clean 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:13974 2014-08-11 15:28:16.440614 2014-08-13 18:06:06.941994 7f5c973a57a0 20 read_log 192834'12461 (192834'11825) clean 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:13975 2014-08-11 15:28:16.554188 2014-08-13 18:06:06.942007 7f5c973a57a0 20 read_log 192834'12462 (192834'11824) clean 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:13976 2014-08-11 15:28:16.658741 2014-08-13 18:06:06.942020 7f5c973a57a0 20 read_log 192834'12463 (192834'12029) clean 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:13977 2014-08-11 15:28:16.659077 2014-08-13 18:06:06.942032 7f5c973a57a0 20 read_log 192834'12465 (192834'12439) delete dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:13979 2014-08-11 15:28:16.798178 2014-08-13 18:06:06.942044 7f5c973a57a0 20 read_log 192834'12472 (192834'11947) clean 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:13986 2014-08-11 15:28:16.824111 2014-08-13 18:06:06.942057 7f5c973a57a0 20 read_log 192834'12485 (192834'12454) delete f5f01769/benchmark_data_storage0111-ib.pocstor_26659_object17156/head//24 by osd.338.0:13999 2014-08-11 15:28:16.873434 2014-08-13 18:06:06.942069 7f5c973a57a0 20 read_log 192834'12486 (192834'12455) delete bc142769/benchmark_data_storage0111-ib.pocstor_26659_object19297/head//24 by osd.338.0:14000 2014-08-11 15:28:16.876617 2014-08-13 18:06:06.942082 7f5c973a57a0 20 read_log 192834'12487 (192834'12457) delete ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:14001 2014-08-11 15:28:16.879811 2014-08-13 18:06:06.942094 7f5c973a57a0 20 read_log 192834'12488 (192834'12458) delete d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:14002 2014-08-11 15:28:16.882491 2014-08-13 18:06:06.942107 7f5c973a57a0 20 read_log 192834'12489 (192834'12460) delete 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:14003 2014-08-11 15:28:16.887606 2014-08-13 18:06:06.942119 7f5c973a57a0 20 read_log 192834'12490 (192834'12463) delete 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:14004 2014-08-11 15:28:16.893249 2014-08-13 18:06:06.942132 7f5c973a57a0 20 read_log 192834'12491 (192834'12461) delete 82799769/benchmark_data_storage0111-ib.pocstor_26659_object10952/head//24 by osd.338.0:14005 2014-08-11 15:28:16.893702 2014-08-13 18:06:06.942144 7f5c973a57a0 20 read_log 192834'12492 (192834'12472) delete 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:14006 2014-08-11 15:28:16.896122 2014-08-13 18:06:06.942165 7f5c973a57a0 20 read_log 192834'12493 (192834'12462) delete 35b2e769/benchmark_data_storage0111-ib.pocstor_26659_object10828/head//24 by osd.338.0:14007 2014-08-11 15:28:16.897830 2014-08-13 18:06:06.942198 7f5c973a57a0 20 read_log 192834'12503 (0'0) promote eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.338.0:14017 2014-08-11 15:28:19.058274 2014-08-13 18:06:06.942228 7f5c973a57a0 20 read_log 192834'12504 (192834'12503) modify eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by client.1883634.0:24421 2014-08-11 15:28:19.031741 2014-08-13 18:06:06.942243 7f5c973a57a0 20 read_log 192834'12509 (192834'11925) modify d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by client.1883634.0:24783 2014-08-11 15:28:19.519578 2014-08-13 18:06:06.942256 7f5c973a57a0 20 read_log 192834'12519 (0'0) promote 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.338.0:14033 2014-08-11 15:28:21.639334 2014-08-13 18:06:06.942279 7f5c973a57a0 20 read_log 192834'12520 (192834'12519) modify 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.1883634.0:25614 2014-08-11 15:28:21.613290 2014-08-13 18:06:06.942294 7f5c973a57a0 20 read_log 192834'12522 (0'0) promote 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.338.0:14036 2014-08-11 15:28:21.687946 2014-08-13 18:06:06.942309 7f5c973a57a0 20 read_log 192834'12523 (192834'12522) modify 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.1883634.0:25632 2014-08-11 15:28:21.658847 2014-08-13 18:06:06.942324 7f5c973a57a0 20 read_log 192834'12544 (192834'11979) modify 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.1883634.0:27995 2014-08-11 15:28:25.246055 2014-08-13 18:06:06.942337 7f5c973a57a0 20 read_log 192834'12559 (192834'12003) modify 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.1883634.0:29411 2014-08-11 15:28:27.188875 2014-08-13 18:06:06.942349 7f5c973a57a0 20 read_log 192834'12569 (192834'12021) modify 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.1883634.0:30095 2014-08-11 15:28:28.082149 2014-08-13 18:06:06.942368 7f5c973a57a0 20 read_log 192834'12570 (0'0) promote fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.338.0:14083 2014-08-11 15:28:28.154256 2014-08-13 18:06:06.942381 7f5c973a57a0 20 read_log 192834'12571 (192834'12570) modify fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by client.1883634.0:30117 2014-08-11 15:28:28.128162 2014-08-13 18:06:06.942393 7f5c973a57a0 20 read_log 192834'12574 (0'0) promote 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.338.0:14087 2014-08-11 15:28:28.460223 2014-08-13 18:06:06.942409 7f5c973a57a0 20 read_log 192834'12575 (192834'12574) modify 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.1883634.0:30303 2014-08-11 15:28:28.434262 2014-08-13 18:06:06.942422 7f5c973a57a0 20 read_log 192834'12582 (192834'12039) modify 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.1883634.0:30438 2014-08-11 15:28:28.635573 2014-08-13 18:06:06.942444 7f5c973a57a0 20 read_log 192834'12584 (192834'12043) modify d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.1883634.0:31238 2014-08-11 15:28:30.145163 2014-08-13 18:06:06.942463 7f5c973a57a0 20 read_log 192834'12586 (0'0) promote d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.338.0:14099 2014-08-11 15:28:30.681202 2014-08-13 18:06:06.942475 7f5c973a57a0 20 read_log 192834'12587 (192834'12586) modify d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.1883634.0:31518 2014-08-11 15:28:30.652881 2014-08-13 18:06:06.942500 7f5c973a57a0 20 read_log 192834'12597 (192834'12061) modify 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.1883634.0:32408 2014-08-11 15:28:31.812262 2014-08-13 18:06:06.942512 7f5c973a57a0 20 read_log 192834'12616 (0'0) promote ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.338.0:14129 2014-08-11 15:28:33.949505 2014-08-13 18:06:06.942525 7f5c973a57a0 20 read_log 192834'12617 (192834'12616) modify ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.1883634.0:33860 2014-08-11 15:28:33.916286 2014-08-13 18:06:06.942538 7f5c973a57a0 20 read_log 192834'12623 (192834'12105) modify f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.1883634.0:34581 2014-08-11 15:28:35.031808 2014-08-13 18:06:06.942550 7f5c973a57a0 20 read_log 192834'12635 (192834'12127) modify b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.1883634.0:35970 2014-08-11 15:28:37.414489 2014-08-13 18:06:06.942563 7f5c973a57a0 20 read_log 192834'12652 (0'0) promote dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.338.0:14165 2014-08-11 15:28:39.031281 2014-08-13 18:06:06.942610 7f5c973a57a0 20 read_log 192834'12653 (192834'12652) modify dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by client.1883634.0:37212 2014-08-11 15:28:39.001580 2014-08-13 18:06:06.942628 7f5c973a57a0 20 read_log 192834'12663 (192834'12175) modify c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by client.1883634.0:38521 2014-08-11 15:28:40.757255 2014-08-13 18:06:06.942649 7f5c973a57a0 20 read_log 192834'12674 (192834'12197) modify c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.1883634.0:39920 2014-08-11 15:28:42.780660 2014-08-13 18:06:06.942662 7f5c973a57a0 20 read_log 192834'12675 (192834'12199) modify 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by client.1883634.0:39961 2014-08-11 15:28:42.940909 2014-08-13 18:06:06.942675 7f5c973a57a0 20 read_log 192834'12684 (192834'12217) modify 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by client.1883634.0:41066 2014-08-11 15:28:44.577069 2014-08-13 18:06:06.942687 7f5c973a57a0 20 read_log 192834'12704 (192834'12257) modify bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by client.1883634.0:42583 2014-08-11 15:28:46.288337 2014-08-13 18:06:06.942709 7f5c973a57a0 20 read_log 192834'12712 (192834'12273) modify 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.1883634.0:43507 2014-08-11 15:28:46.575864 2014-08-13 18:06:06.942725 7f5c973a57a0 20 read_log 192834'12721 (192834'12333) modify f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by client.1883634.0:44622 2014-08-11 15:28:46.908095 2014-08-13 18:06:06.942743 7f5c973a57a0 20 read_log 192834'12726 (192834'12343) modify 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by client.1883634.0:45428 2014-08-11 15:28:47.274040 2014-08-13 18:06:06.942759 7f5c973a57a0 20 read_log 192834'12732 (192834'12365) modify 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.1883634.0:45797 2014-08-11 15:28:47.392947 2014-08-13 18:06:06.942772 7f5c973a57a0 20 read_log 192834'12739 (192834'12429) modify b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.1883634.0:46379 2014-08-11 15:28:47.573669 2014-08-13 18:06:06.942785 7f5c973a57a0 20 read_log 192834'12741 (192834'12433) modify 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.1883634.0:46567 2014-08-11 15:28:47.633148 2014-08-13 18:06:06.942797 7f5c973a57a0 20 read_log 192888'12746 (0'0) promote 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by osd.305.0:27402 2014-08-11 16:44:58.481828 2014-08-13 18:06:06.942810 7f5c973a57a0 20 read_log 192888'12747 (192888'12746) modify 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.1315259.0:843 2014-08-11 16:44:58.473411 2014-08-13 18:06:06.942822 7f5c973a57a0 20 read_log 192888'12748 (0'0) promote f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.305.0:27406 2014-08-11 16:44:58.664888 2014-08-13 18:06:06.942835 7f5c973a57a0 20 read_log 192888'12749 (192888'12748) modify f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by client.1315259.0:892 2014-08-11 16:44:58.655515 2014-08-13 18:06:06.942847 7f5c973a57a0 20 read_log 192888'12750 (0'0) promote e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by osd.305.0:27410 2014-08-11 16:44:59.005411 2014-08-13 18:06:06.942863 7f5c973a57a0 20 read_log 192888'12751 (192888'12750) modify e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.1315259.0:984 2014-08-11 16:44:58.997019 2014-08-13 18:06:06.942876 7f5c973a57a0 20 read_log 192888'12754 (0'0) promote 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by osd.305.0:27418 2014-08-11 16:45:01.546408 2014-08-13 18:06:06.942895 7f5c973a57a0 20 read_log 192888'12755 (192888'12754) modify 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.1315259.0:1641 2014-08-11 16:45:01.538449 2014-08-13 18:06:06.942910 7f5c973a57a0 20 read_log 192888'12756 (0'0) promote ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by osd.305.0:27422 2014-08-11 16:45:02.120529 2014-08-13 18:06:06.942923 7f5c973a57a0 20 read_log 192888'12757 (192888'12756) modify ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.1315259.0:1796 2014-08-11 16:45:02.112815 2014-08-13 18:06:06.942936 7f5c973a57a0 20 read_log 192888'12758 (0'0) promote 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.305.0:27426 2014-08-11 16:45:02.903642 2014-08-13 18:06:06.942948 7f5c973a57a0 20 read_log 192888'12759 (192888'12758) modify 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by client.1315259.0:2008 2014-08-11 16:45:02.895752 2014-08-13 18:06:06.942961 7f5c973a57a0 20 read_log 192888'12766 (0'0) promote e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by osd.305.0:27455 2014-08-11 16:45:06.866469 2014-08-13 18:06:06.942973 7f5c973a57a0 20 read_log 192888'12767 (192888'12766) modify e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.1315259.0:3068 2014-08-11 16:45:06.858142 2014-08-13 18:06:06.942985 7f5c973a57a0 20 read_log 192888'12768 (0'0) promote 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by osd.305.0:27471 2014-08-11 16:45:09.639102 2014-08-13 18:06:06.942998 7f5c973a57a0 20 read_log 192888'12769 (192888'12768) modify 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.1315259.0:3776 2014-08-11 16:45:09.631060 2014-08-13 18:06:06.943010 7f5c973a57a0 20 read_log 192888'12770 (0'0) promote 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by osd.305.0:27475 2014-08-11 16:45:10.072817 2014-08-13 18:06:06.943022 7f5c973a57a0 20 read_log 192888'12771 (192888'12770) modify 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.1315259.0:3894 2014-08-11 16:45:10.065369 2014-08-13 18:06:06.943035 7f5c973a57a0 20 read_log 192888'12772 (0'0) promote c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by osd.305.0:27483 2014-08-11 16:45:10.684042 2014-08-13 18:06:06.943047 7f5c973a57a0 20 read_log 192888'12773 (192888'12772) modify c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.1315259.0:4061 2014-08-11 16:45:10.676331 2014-08-13 18:06:06.943059 7f5c973a57a0 20 read_log 192888'12774 (0'0) promote d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.305.0:27501 2014-08-11 16:45:13.041530 2014-08-13 18:06:06.943071 7f5c973a57a0 20 read_log 192888'12775 (192888'12774) modify d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by client.1315259.0:4710 2014-08-11 16:45:13.032955 2014-08-13 18:06:06.943084 7f5c973a57a0 20 read_log 192888'12780 (0'0) promote afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by osd.305.0:27509 2014-08-11 16:45:14.287842 2014-08-13 18:06:06.943096 7f5c973a57a0 20 read_log 192888'12781 (192888'12780) modify afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.1315259.0:5044 2014-08-11 16:45:14.279057 2014-08-13 18:06:06.943111 7f5c973a57a0 20 read_log 192888'12784 (0'0) promote db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by osd.305.0:27539 2014-08-11 16:45:18.541294 2014-08-13 18:06:06.943124 7f5c973a57a0 20 read_log 192888'12785 (192888'12784) modify db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.1315259.0:6210 2014-08-11 16:45:18.533172 2014-08-13 18:06:06.943138 7f5c973a57a0 20 read_log 192888'12786 (0'0) promote 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by osd.305.0:27541 2014-08-11 16:45:18.781674 2014-08-13 18:06:06.943163 7f5c973a57a0 20 read_log 192888'12787 (192888'12786) modify 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.1315259.0:6276 2014-08-11 16:45:18.771615 2014-08-13 18:06:06.943178 7f5c973a57a0 20 read_log 192888'12788 (0'0) promote 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.305.0:27547 2014-08-11 16:45:19.485946 2014-08-13 18:06:06.943191 7f5c973a57a0 20 read_log 192888'12789 (192888'12788) modify 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by client.1315259.0:6471 2014-08-11 16:45:19.477888 2014-08-13 18:06:06.943205 7f5c973a57a0 20 read_log 192888'12790 (0'0) promote 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by osd.305.0:27549 2014-08-11 16:45:20.103082 2014-08-13 18:06:06.943218 7f5c973a57a0 20 read_log 192888'12791 (192888'12790) modify 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.1315259.0:6643 2014-08-11 16:45:20.095501 2014-08-13 18:06:06.943233 7f5c973a57a0 20 read_log 192888'12792 (0'0) promote c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.305.0:27557 2014-08-11 16:45:22.253642 2014-08-13 18:06:06.943245 7f5c973a57a0 20 read_log 192888'12793 (192888'12792) modify c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by client.1315259.0:7236 2014-08-11 16:45:22.244724 2014-08-13 18:06:06.943258 7f5c973a57a0 20 read_log 192888'12794 (0'0) promote 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.305.0:27567 2014-08-11 16:45:23.899001 2014-08-13 18:06:06.943272 7f5c973a57a0 20 read_log 192888'12795 (192888'12794) modify 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by client.1315259.0:7694 2014-08-11 16:45:23.891896 2014-08-13 18:06:06.943287 7f5c973a57a0 20 read_log 192888'12798 (0'0) promote f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.305.0:27607 2014-08-11 16:45:27.514750 2014-08-13 18:06:06.943299 7f5c973a57a0 20 read_log 192888'12799 (192888'12798) modify f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by client.1315259.0:8650 2014-08-11 16:45:27.506696 2014-08-13 18:06:06.943318 7f5c973a57a0 20 read_log 192888'12802 (0'0) promote 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.305.0:27651 2014-08-11 16:45:34.733452 2014-08-13 18:06:06.943330 7f5c973a57a0 20 read_log 192888'12803 (192888'12802) modify 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by client.1315259.0:10530 2014-08-11 16:45:34.725705 2014-08-13 18:06:06.943342 7f5c973a57a0 20 read_log 192888'12804 (0'0) promote f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.305.0:27665 2014-08-11 16:45:37.339602 2014-08-13 18:06:06.943358 7f5c973a57a0 20 read_log 192888'12805 (192888'12804) modify f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by client.1315259.0:11180 2014-08-11 16:45:37.331965 2014-08-13 18:06:06.943373 7f5c973a57a0 20 read_log 192888'12806 (0'0) promote d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.305.0:27679 2014-08-11 16:45:38.042929 2014-08-13 18:06:06.943385 7f5c973a57a0 20 read_log 192888'12807 (192888'12806) modify d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by client.1315259.0:11358 2014-08-11 16:45:38.035641 2014-08-13 18:06:06.943401 7f5c973a57a0 20 read_log 192888'12808 (0'0) promote 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by osd.305.0:27687 2014-08-11 16:45:39.500694 2014-08-13 18:06:06.943417 7f5c973a57a0 20 read_log 192888'12809 (192888'12808) modify 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.1315259.0:11732 2014-08-11 16:45:39.492511 2014-08-13 18:06:06.943438 7f5c973a57a0 20 read_log 192888'12810 (0'0) promote 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.305.0:27703 2014-08-11 16:45:42.624694 2014-08-13 18:06:06.943466 7f5c973a57a0 20 read_log 192888'12811 (192888'12810) modify 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by client.1315259.0:12463 2014-08-11 16:45:42.616822 2014-08-13 18:06:06.943489 7f5c973a57a0 20 read_log 192888'12818 (0'0) promote 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by osd.305.0:27745 2014-08-11 16:45:51.620835 2014-08-13 18:06:06.943504 7f5c973a57a0 20 read_log 192888'12819 (192888'12818) modify 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.1315259.0:14754 2014-08-11 16:45:51.611943 2014-08-13 18:06:06.943517 7f5c973a57a0 20 read_log 192888'12820 (0'0) promote 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by osd.305.0:27777 2014-08-11 16:45:57.492638 2014-08-13 18:06:06.943529 7f5c973a57a0 20 read_log 192888'12821 (192888'12820) modify 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.1315259.0:16283 2014-08-11 16:45:57.484663 2014-08-13 18:06:06.943542 7f5c973a57a0 20 read_log 192888'12822 (0'0) promote f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.305.0:27801 2014-08-11 16:46:00.261347 2014-08-13 18:06:06.943554 7f5c973a57a0 20 read_log 192888'12823 (192888'12822) modify f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by client.1315259.0:16981 2014-08-11 16:46:00.252542 2014-08-13 18:06:06.943566 7f5c973a57a0 20 read_log 192888'12832 (0'0) promote 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by osd.305.0:27831 2014-08-11 16:46:05.527123 2014-08-13 18:06:06.943578 7f5c973a57a0 20 read_log 192888'12833 (192888'12832) modify 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.1315259.0:18296 2014-08-11 16:46:05.517121 2014-08-13 18:06:06.943592 7f5c973a57a0 20 read_log 192888'12834 (0'0) promote c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by osd.305.0:27835 2014-08-11 16:46:06.936200 2014-08-13 18:06:06.943604 7f5c973a57a0 20 read_log 192888'12835 (192888'12834) modify c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.1315259.0:18673 2014-08-11 16:46:06.927103 2014-08-13 18:06:06.943672 7f5c973a57a0 20 read_log 192888'12836 (0'0) promote 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.305.0:27839 2014-08-11 16:46:07.412455 2014-08-13 18:06:06.943691 7f5c973a57a0 20 read_log 192888'12837 (192888'12836) modify 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by client.1315259.0:18799 2014-08-11 16:46:07.401681 2014-08-13 18:06:06.943704 7f5c973a57a0 20 read_log 192888'12838 (0'0) promote d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by osd.305.0:27843 2014-08-11 16:46:09.228380 2014-08-13 18:06:06.943716 7f5c973a57a0 20 read_log 192888'12839 (192888'12838) modify d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.1315259.0:19234 2014-08-11 16:46:09.219655 2014-08-13 18:06:06.943729 7f5c973a57a0 20 read_log 192888'12840 (0'0) promote 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by osd.305.0:27871 2014-08-11 16:46:13.465331 2014-08-13 18:06:06.943741 7f5c973a57a0 20 read_log 192888'12841 (192888'12840) modify 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.1315259.0:20365 2014-08-11 16:46:13.456177 2014-08-13 18:06:06.943754 7f5c973a57a0 20 read_log 192888'12842 (0'0) promote 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by osd.305.0:27877 2014-08-11 16:46:16.146016 2014-08-13 18:06:06.943766 7f5c973a57a0 20 read_log 192888'12843 (192888'12842) modify 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.1315259.0:21073 2014-08-11 16:46:16.136835 2014-08-13 18:06:06.943779 7f5c973a57a0 20 read_log 192888'12844 (0'0) promote a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.305.0:27895 2014-08-11 16:46:18.285906 2014-08-13 18:06:06.943798 7f5c973a57a0 20 read_log 192888'12845 (192888'12844) modify a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by client.1315259.0:21630 2014-08-11 16:46:18.276682 2014-08-13 18:06:06.943813 7f5c973a57a0 20 read_log 192888'12858 (0'0) promote 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by osd.305.0:27997 2014-08-11 16:46:34.940573 2014-08-13 18:06:06.943826 7f5c973a57a0 20 read_log 192888'12859 (192888'12858) modify 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.1315259.0:25833 2014-08-11 16:46:34.930191 2014-08-13 18:06:06.943838 7f5c973a57a0 20 read_log 192888'12862 (0'0) promote 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by osd.305.0:28045 2014-08-11 16:46:43.287565 2014-08-13 18:06:06.943851 7f5c973a57a0 20 read_log 192888'12863 (192888'12862) modify 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.1315259.0:27827 2014-08-11 16:46:43.279264 2014-08-13 18:06:06.943863 7f5c973a57a0 20 read_log 192888'12864 (0'0) promote f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by osd.305.0:28053 2014-08-11 16:46:45.115545 2014-08-13 18:06:06.943875 7f5c973a57a0 20 read_log 192888'12865 (192888'12864) modify f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.1315259.0:28274 2014-08-11 16:46:45.107281 2014-08-13 18:06:06.943898 7f5c973a57a0 20 read_log 192888'12868 (0'0) promote 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by osd.305.0:28077 2014-08-11 16:46:48.908918 2014-08-13 18:06:06.943915 7f5c973a57a0 20 read_log 192888'12869 (192888'12868) modify 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.1315259.0:29216 2014-08-11 16:46:48.899651 2014-08-13 18:06:06.943927 7f5c973a57a0 20 read_log 192888'12870 (0'0) promote e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by osd.305.0:28117 2014-08-11 16:46:54.279204 2014-08-13 18:06:06.943940 7f5c973a57a0 20 read_log 192888'12871 (192888'12870) modify e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.1315259.0:30574 2014-08-11 16:46:54.270345 2014-08-13 18:06:06.943952 7f5c973a57a0 20 read_log 192888'12873 (192888'12747) modify 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.1315259.0:31570 2014-08-11 16:46:56.132979 2014-08-13 18:06:06.943965 7f5c973a57a0 20 read_log 192888'12874 (192888'12749) modify f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by client.1315259.0:31619 2014-08-11 16:46:56.148294 2014-08-13 18:06:06.943977 7f5c973a57a0 20 read_log 192888'12875 (192888'12751) modify e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.1315259.0:31711 2014-08-11 16:46:56.178349 2014-08-13 18:06:06.943989 7f5c973a57a0 20 read_log 192888'12877 (192888'12755) modify 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.1315259.0:32368 2014-08-11 16:46:56.359187 2014-08-13 18:06:06.944005 7f5c973a57a0 20 read_log 192888'12878 (192888'12757) modify ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.1315259.0:32523 2014-08-11 16:46:56.401165 2014-08-13 18:06:06.944017 7f5c973a57a0 20 read_log 192888'12879 (192888'12759) modify 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by client.1315259.0:32735 2014-08-11 16:46:56.456087 2014-08-13 18:06:06.944030 7f5c973a57a0 20 read_log 192888'12883 (192888'12767) modify e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.1315259.0:33795 2014-08-11 16:46:56.727196 2014-08-13 18:06:06.944046 7f5c973a57a0 20 read_log 192888'12884 (192888'12769) modify 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.1315259.0:34503 2014-08-11 16:46:56.911069 2014-08-13 18:06:06.944061 7f5c973a57a0 20 read_log 192888'12885 (192888'12771) modify 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.1315259.0:34621 2014-08-11 16:46:56.943153 2014-08-13 18:06:06.944081 7f5c973a57a0 20 read_log 192888'12886 (192888'12773) modify c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.1315259.0:34788 2014-08-11 16:46:56.986354 2014-08-13 18:06:06.944099 7f5c973a57a0 20 read_log 192888'12887 (192888'12775) modify d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by client.1315259.0:35437 2014-08-11 16:46:57.164075 2014-08-13 18:06:06.944112 7f5c973a57a0 20 read_log 192888'12890 (192888'12781) modify afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.1315259.0:35771 2014-08-11 16:46:57.241531 2014-08-13 18:06:06.944124 7f5c973a57a0 20 read_log 192888'12892 (192888'12785) modify db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.1315259.0:36937 2014-08-11 16:46:57.550493 2014-08-13 18:06:06.944158 7f5c973a57a0 20 read_log 192888'12893 (192888'12787) modify 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.1315259.0:37003 2014-08-11 16:46:57.569100 2014-08-13 18:06:06.944175 7f5c973a57a0 20 read_log 192888'12894 (192888'12789) modify 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by client.1315259.0:37198 2014-08-11 16:46:57.620096 2014-08-13 18:06:06.944195 7f5c973a57a0 20 read_log 192888'12895 (192888'12791) modify 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.1315259.0:37370 2014-08-11 16:46:57.664670 2014-08-13 18:06:06.944212 7f5c973a57a0 20 read_log 192888'12896 (192888'12793) modify c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by client.1315259.0:37963 2014-08-11 16:46:57.821958 2014-08-13 18:06:06.944224 7f5c973a57a0 20 read_log 192888'12897 (192888'12795) modify 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by client.1315259.0:38421 2014-08-11 16:46:57.943109 2014-08-13 18:06:06.944245 7f5c973a57a0 20 read_log 192888'12899 (192888'12799) modify f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by client.1315259.0:39377 2014-08-11 16:46:58.195773 2014-08-13 18:06:06.944261 7f5c973a57a0 20 read_log 192888'12901 (192888'12803) modify 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by client.1315259.0:41257 2014-08-11 16:46:58.700899 2014-08-13 18:06:06.944273 7f5c973a57a0 20 read_log 192888'12902 (192888'12805) modify f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by client.1315259.0:41907 2014-08-11 16:46:58.868370 2014-08-13 18:06:06.944286 7f5c973a57a0 20 read_log 192888'12903 (192888'12807) modify d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by client.1315259.0:42085 2014-08-11 16:46:58.915331 2014-08-13 18:06:06.944298 7f5c973a57a0 20 read_log 192888'12904 (192888'12809) modify 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.1315259.0:42459 2014-08-11 16:46:59.016260 2014-08-13 18:06:06.944311 7f5c973a57a0 20 read_log 192888'12905 (192888'12811) modify 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by client.1315259.0:43190 2014-08-11 16:46:59.218752 2014-08-13 18:06:06.944327 7f5c973a57a0 20 read_log 192888'12909 (192888'12819) modify 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.1315259.0:45481 2014-08-11 16:46:59.882019 2014-08-13 18:06:06.944342 7f5c973a57a0 20 read_log 192888'12910 (192888'12821) modify 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.1315259.0:47010 2014-08-11 16:47:00.321359 2014-08-13 18:06:06.944355 7f5c973a57a0 20 read_log 192888'12911 (192888'12823) modify f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by client.1315259.0:47708 2014-08-11 16:47:00.542200 2014-08-13 18:06:06.944367 7f5c973a57a0 20 read_log 192888'12916 (192888'12833) modify 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.1315259.0:49023 2014-08-11 16:47:00.936109 2014-08-13 18:06:06.944379 7f5c973a57a0 20 read_log 192888'12917 (192888'12835) modify c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.1315259.0:49400 2014-08-11 16:47:01.048651 2014-08-13 18:06:06.944409 7f5c973a57a0 20 read_log 192888'12918 (192888'12837) modify 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by client.1315259.0:49526 2014-08-11 16:47:01.091294 2014-08-13 18:06:06.944427 7f5c973a57a0 20 read_log 192888'12919 (192888'12839) modify d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.1315259.0:49961 2014-08-11 16:47:01.222499 2014-08-13 18:06:06.944440 7f5c973a57a0 20 read_log 192888'12920 (192888'12841) modify 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.1315259.0:51092 2014-08-11 16:47:01.579019 2014-08-13 18:06:06.944453 7f5c973a57a0 20 read_log 192888'12921 (192888'12843) modify 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.1315259.0:51800 2014-08-11 16:47:01.787232 2014-08-13 18:06:06.944465 7f5c973a57a0 20 read_log 192888'12922 (192888'12845) modify a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by client.1315259.0:52357 2014-08-11 16:47:01.952121 2014-08-13 18:06:06.944477 7f5c973a57a0 20 read_log 192888'12929 (192888'12859) modify 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.1315259.0:56560 2014-08-11 16:47:03.244641 2014-08-13 18:06:06.944490 7f5c973a57a0 20 read_log 192888'12931 (192888'12863) modify 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.1315259.0:58554 2014-08-11 16:47:03.835799 2014-08-13 18:06:06.944502 7f5c973a57a0 20 read_log 192888'12932 (192888'12865) modify f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.1315259.0:59001 2014-08-11 16:47:03.971162 2014-08-13 18:06:06.944515 7f5c973a57a0 20 read_log 192888'12934 (192888'12869) modify 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.1315259.0:59943 2014-08-11 16:47:04.292852 2014-08-13 18:06:06.944527 7f5c973a57a0 20 read_log 192888'12935 (192888'12871) modify e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.1315259.0:61301 2014-08-11 16:47:04.697605 2014-08-13 18:06:06.944539 7f5c973a57a0 20 read_log 192888'12950 (0'0) promote 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by osd.305.0:28536 2014-08-11 17:05:25.641419 2014-08-13 18:06:06.944552 7f5c973a57a0 20 read_log 192888'12951 (192888'12950) modify 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.942868.0:2621 2014-08-11 17:05:25.633232 2014-08-13 18:06:06.944564 7f5c973a57a0 20 read_log 192888'12952 (0'0) promote 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.305.0:28544 2014-08-11 17:05:27.579611 2014-08-13 18:06:06.944577 7f5c973a57a0 20 read_log 192888'12953 (192888'12952) modify 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by client.942868.0:3145 2014-08-11 17:05:27.571718 2014-08-13 18:06:06.944589 7f5c973a57a0 20 read_log 192888'12954 (0'0) promote de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by osd.305.0:28548 2014-08-11 17:05:27.867341 2014-08-13 18:06:06.944601 7f5c973a57a0 20 read_log 192888'12955 (192888'12954) modify de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.942868.0:3223 2014-08-11 17:05:27.859526 2014-08-13 18:06:06.944614 7f5c973a57a0 20 read_log 192888'12956 (0'0) promote 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.305.0:28572 2014-08-11 17:05:31.644777 2014-08-13 18:06:06.944626 7f5c973a57a0 20 read_log 192888'12957 (192888'12956) modify 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by client.942868.0:4247 2014-08-11 17:05:31.638035 2014-08-13 18:06:06.944639 7f5c973a57a0 20 read_log 192888'12960 (0'0) promote ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.305.0:28610 2014-08-11 17:05:36.452783 2014-08-13 18:06:06.944651 7f5c973a57a0 20 read_log 192888'12961 (192888'12960) modify ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by client.942868.0:5489 2014-08-11 17:05:36.444042 2014-08-13 18:06:06.944686 7f5c973a57a0 20 read_log 192888'12964 (0'0) promote 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by osd.305.0:28630 2014-08-11 17:05:39.848129 2014-08-13 18:06:06.944703 7f5c973a57a0 20 read_log 192888'12965 (192888'12964) modify 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.942868.0:6423 2014-08-11 17:05:39.837896 2014-08-13 18:06:06.944715 7f5c973a57a0 20 read_log 192888'12968 (0'0) promote 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.305.0:28648 2014-08-11 17:05:41.501503 2014-08-13 18:06:06.944730 7f5c973a57a0 20 read_log 192888'12969 (192888'12968) modify 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by client.942868.0:6883 2014-08-11 17:05:41.494179 2014-08-13 18:06:06.944743 7f5c973a57a0 20 read_log 192888'12978 (0'0) promote 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by osd.305.0:28708 2014-08-11 17:05:49.071536 2014-08-13 18:06:06.944755 7f5c973a57a0 20 read_log 192888'12979 (192888'12978) modify 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.942868.0:8875 2014-08-11 17:05:49.063922 2014-08-13 18:06:06.944771 7f5c973a57a0 20 read_log 192888'12980 (0'0) promote 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by osd.305.0:28728 2014-08-11 17:05:52.369211 2014-08-13 18:06:06.944784 7f5c973a57a0 20 read_log 192888'12981 (192888'12980) modify 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.942868.0:9769 2014-08-11 17:05:52.361275 2014-08-13 18:06:06.944798 7f5c973a57a0 20 read_log 192888'12982 (0'0) promote 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by osd.305.0:28758 2014-08-11 17:05:55.808279 2014-08-13 18:06:06.944811 7f5c973a57a0 20 read_log 192888'12983 (192888'12982) modify 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.942868.0:10724 2014-08-11 17:05:55.800446 2014-08-13 18:06:06.944825 7f5c973a57a0 20 read_log 192888'12986 (0'0) promote a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.305.0:28794 2014-08-11 17:06:02.564670 2014-08-13 18:06:06.944838 7f5c973a57a0 20 read_log 192888'12987 (192888'12986) modify a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by client.942868.0:12575 2014-08-11 17:06:02.556515 2014-08-13 18:06:06.944850 7f5c973a57a0 20 read_log 192888'12990 (0'0) promote ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.305.0:28844 2014-08-11 17:06:07.336797 2014-08-13 18:06:06.944863 7f5c973a57a0 20 read_log 192888'12991 (192888'12990) modify ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by client.942868.0:13793 2014-08-11 17:06:07.329064 2014-08-13 18:06:06.944877 7f5c973a57a0 20 read_log 192888'12992 (0'0) promote 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by osd.305.0:28858 2014-08-11 17:06:08.995967 2014-08-13 18:06:06.944889 7f5c973a57a0 20 read_log 192888'12993 (192888'12992) modify 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.942868.0:14188 2014-08-11 17:06:08.988265 2014-08-13 18:06:06.944904 7f5c973a57a0 20 read_log 192888'12994 (0'0) promote c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by osd.305.0:28860 2014-08-11 17:06:09.812658 2014-08-13 18:06:06.944916 7f5c973a57a0 20 read_log 192888'12995 (192888'12994) modify c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.942868.0:14392 2014-08-11 17:06:09.804228 2014-08-13 18:06:06.944931 7f5c973a57a0 20 read_log 192888'12998 (0'0) promote fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by osd.305.0:28878 2014-08-11 17:06:12.501838 2014-08-13 18:06:06.944943 7f5c973a57a0 20 read_log 192888'12999 (192888'12998) modify fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.942868.0:15072 2014-08-11 17:06:12.493407 2014-08-13 18:06:06.944964 7f5c973a57a0 20 read_log 192888'13004 (0'0) promote 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by osd.305.0:28954 2014-08-11 17:06:28.152689 2014-08-13 18:06:06.944979 7f5c973a57a0 20 read_log 192888'13005 (192888'13004) modify 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.942868.0:19146 2014-08-11 17:06:28.145059 2014-08-13 18:06:06.944992 7f5c973a57a0 20 read_log 192888'13006 (0'0) promote 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by osd.305.0:28956 2014-08-11 17:06:28.815603 2014-08-13 18:06:06.945006 7f5c973a57a0 20 read_log 192888'13007 (192888'13006) modify 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.942868.0:19323 2014-08-11 17:06:28.807139 2014-08-13 18:06:06.945037 7f5c973a57a0 20 read_log 192888'13008 (0'0) promote 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by osd.305.0:28962 2014-08-11 17:06:29.431846 2014-08-13 18:06:06.945052 7f5c973a57a0 20 read_log 192888'13009 (192888'13008) modify 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.942868.0:19489 2014-08-11 17:06:29.424256 2014-08-13 18:06:06.945065 7f5c973a57a0 20 read_log 192888'13010 (0'0) promote e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.305.0:28966 2014-08-11 17:06:30.010752 2014-08-13 18:06:06.945079 7f5c973a57a0 20 read_log 192888'13011 (192888'13010) modify e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by client.942868.0:19643 2014-08-11 17:06:30.002215 2014-08-13 18:06:06.945093 7f5c973a57a0 20 read_log 192888'13012 (0'0) promote abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.305.0:28978 2014-08-11 17:06:31.909292 2014-08-13 18:06:06.945107 7f5c973a57a0 20 read_log 192888'13013 (192888'13012) modify abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by client.942868.0:20146 2014-08-11 17:06:31.900532 2014-08-13 18:06:06.945122 7f5c973a57a0 20 read_log 192888'13014 (0'0) promote 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by osd.305.0:28986 2014-08-11 17:06:32.844293 2014-08-13 18:06:06.945135 7f5c973a57a0 20 read_log 192888'13015 (192888'13014) modify 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.942868.0:20383 2014-08-11 17:06:32.834976 2014-08-13 18:06:06.945154 7f5c973a57a0 20 read_log 192888'13020 (0'0) promote 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by osd.305.0:29024 2014-08-11 17:06:38.200266 2014-08-13 18:06:06.945169 7f5c973a57a0 20 read_log 192888'13021 (192888'13020) modify 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.942868.0:21793 2014-08-11 17:06:38.193885 2014-08-13 18:06:06.945182 7f5c973a57a0 20 read_log 192888'13022 (0'0) promote ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by osd.305.0:29052 2014-08-11 17:06:43.449258 2014-08-13 18:06:06.945194 7f5c973a57a0 20 read_log 192888'13023 (192888'13022) modify ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.942868.0:23132 2014-08-11 17:06:43.441067 2014-08-13 18:06:06.945207 7f5c973a57a0 20 read_log 192888'13026 (0'0) promote 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by osd.305.0:29062 2014-08-11 17:06:47.049327 2014-08-13 18:06:06.945221 7f5c973a57a0 20 read_log 192888'13027 (192888'13026) modify 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.942868.0:24091 2014-08-11 17:06:47.039087 2014-08-13 18:06:06.945234 7f5c973a57a0 20 read_log 192888'13032 (0'0) promote 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by osd.305.0:29102 2014-08-11 17:06:51.734317 2014-08-13 18:06:06.945248 7f5c973a57a0 20 read_log 192888'13033 (192888'13032) modify 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.942868.0:25346 2014-08-11 17:06:51.721568 2014-08-13 18:06:06.945273 7f5c973a57a0 20 read_log 192888'13036 (0'0) promote df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by osd.305.0:29134 2014-08-11 17:06:58.324199 2014-08-13 18:06:06.945289 7f5c973a57a0 20 read_log 192888'13037 (192888'13036) modify df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.942868.0:27027 2014-08-11 17:06:58.316743 2014-08-13 18:06:06.945326 7f5c973a57a0 20 read_log 192888'13038 (0'0) promote 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by osd.305.0:29142 2014-08-11 17:06:59.884021 2014-08-13 18:06:06.945339 7f5c973a57a0 20 read_log 192888'13039 (192888'13038) modify 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.942868.0:27453 2014-08-11 17:06:59.876660 2014-08-13 18:06:06.945352 7f5c973a57a0 20 read_log 192888'13042 (0'0) promote cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by osd.305.0:29170 2014-08-11 17:07:05.443233 2014-08-13 18:06:06.945368 7f5c973a57a0 20 read_log 192888'13043 (192888'13042) modify cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.942868.0:28938 2014-08-11 17:07:05.434496 2014-08-13 18:06:06.945381 7f5c973a57a0 20 read_log 192888'13044 (0'0) promote 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by osd.305.0:29174 2014-08-11 17:07:05.477243 2014-08-13 18:06:06.945400 7f5c973a57a0 20 read_log 192888'13045 (192888'13044) modify 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.942868.0:28948 2014-08-11 17:07:05.470175 2014-08-13 18:06:06.945419 7f5c973a57a0 20 read_log 192888'13048 (0'0) promote 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by osd.305.0:29194 2014-08-11 17:07:09.144107 2014-08-13 18:06:06.945432 7f5c973a57a0 20 read_log 192888'13049 (192888'13048) modify 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.942868.0:29835 2014-08-11 17:07:09.134805 2014-08-13 18:06:06.945452 7f5c973a57a0 20 read_log 192888'13050 (0'0) promote 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by osd.305.0:29198 2014-08-11 17:07:09.194766 2014-08-13 18:06:06.945465 7f5c973a57a0 20 read_log 192888'13051 (192888'13050) modify 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.942868.0:29848 2014-08-11 17:07:09.186436 2014-08-13 18:06:06.945478 7f5c973a57a0 20 read_log 192888'13052 (0'0) promote 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.305.0:29202 2014-08-11 17:07:10.206619 2014-08-13 18:06:06.945490 7f5c973a57a0 20 read_log 192888'13053 (192888'13052) modify 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by client.942868.0:30096 2014-08-11 17:07:10.197905 2014-08-13 18:06:06.945510 7f5c973a57a0 20 read_log 192888'13068 (192888'12951) modify 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.942868.0:33887 2014-08-11 17:07:16.639184 2014-08-13 18:06:06.945526 7f5c973a57a0 20 read_log 192888'13069 (192888'12953) modify 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by client.942868.0:34411 2014-08-11 17:07:16.780699 2014-08-13 18:06:06.945539 7f5c973a57a0 20 read_log 192888'13070 (192888'12955) modify de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.942868.0:34489 2014-08-11 17:07:16.803582 2014-08-13 18:06:06.945561 7f5c973a57a0 20 read_log 192888'13071 (192888'12957) modify 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by client.942868.0:35513 2014-08-11 17:07:17.092353 2014-08-13 18:06:06.945577 7f5c973a57a0 20 read_log 192888'13073 (192888'12961) modify ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by client.942868.0:36755 2014-08-11 17:07:17.425832 2014-08-13 18:06:06.945597 7f5c973a57a0 20 read_log 192888'13075 (192888'12965) modify 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.942868.0:37689 2014-08-11 17:07:17.680174 2014-08-13 18:06:06.945622 7f5c973a57a0 20 read_log 192888'13077 (192888'12969) modify 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by client.942868.0:38149 2014-08-11 17:07:17.807532 2014-08-13 18:06:06.945637 7f5c973a57a0 20 read_log 192888'13082 (192888'12979) modify 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.942868.0:40141 2014-08-11 17:07:18.374477 2014-08-13 18:06:06.945660 7f5c973a57a0 20 read_log 192888'13083 (192888'12981) modify 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.942868.0:41035 2014-08-11 17:07:18.616035 2014-08-13 18:06:06.945676 7f5c973a57a0 20 read_log 192888'13084 (192888'12983) modify 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.942868.0:41990 2014-08-11 17:07:18.875637 2014-08-13 18:06:06.945688 7f5c973a57a0 20 read_log 192888'13086 (192888'12987) modify a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by client.942868.0:43841 2014-08-11 17:07:19.391794 2014-08-13 18:06:06.945701 7f5c973a57a0 20 read_log 192888'13088 (192888'12991) modify ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by client.942868.0:45059 2014-08-11 17:07:19.735426 2014-08-13 18:06:06.945714 7f5c973a57a0 20 read_log 192888'13089 (192888'12993) modify 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.942868.0:45454 2014-08-11 17:07:19.841867 2014-08-13 18:06:06.945726 7f5c973a57a0 20 read_log 192888'13090 (192888'12995) modify c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.942868.0:45658 2014-08-11 17:07:19.898216 2014-08-13 18:06:06.945739 7f5c973a57a0 20 read_log 192888'13092 (192888'12999) modify fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.942868.0:46338 2014-08-11 17:07:20.093354 2014-08-13 18:06:06.945751 7f5c973a57a0 20 read_log 192888'13095 (192888'13005) modify 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.942868.0:50412 2014-08-11 17:07:21.312190 2014-08-13 18:06:06.945764 7f5c973a57a0 20 read_log 192888'13096 (192888'13007) modify 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.942868.0:50589 2014-08-11 17:07:21.371077 2014-08-13 18:06:06.945779 7f5c973a57a0 20 read_log 192888'13097 (192888'13009) modify 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.942868.0:50755 2014-08-11 17:07:21.421740 2014-08-13 18:06:06.945793 7f5c973a57a0 20 read_log 192888'13098 (192888'13011) modify e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by client.942868.0:50909 2014-08-11 17:07:21.469579 2014-08-13 18:06:06.945807 7f5c973a57a0 20 read_log 192888'13099 (192888'13013) modify abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by client.942868.0:51412 2014-08-11 17:07:21.641750 2014-08-13 18:06:06.945830 7f5c973a57a0 20 read_log 192888'13100 (192888'13015) modify 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.942868.0:51649 2014-08-11 17:07:21.709709 2014-08-13 18:06:06.945843 7f5c973a57a0 20 read_log 192888'13103 (192888'13021) modify 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.942868.0:53059 2014-08-11 17:07:22.136174 2014-08-13 18:06:06.945855 7f5c973a57a0 20 read_log 192888'13104 (192888'13023) modify ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.942868.0:54398 2014-08-11 17:07:22.546559 2014-08-13 18:06:06.945868 7f5c973a57a0 20 read_log 192888'13106 (192888'13027) modify 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.942868.0:55357 2014-08-11 17:07:22.823264 2014-08-13 18:06:06.945880 7f5c973a57a0 20 read_log 192888'13109 (192888'13033) modify 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.942868.0:56612 2014-08-11 17:07:23.206976 2014-08-13 18:06:06.945892 7f5c973a57a0 20 read_log 192888'13111 (192888'13037) modify df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.942868.0:58293 2014-08-11 17:07:23.697664 2014-08-13 18:06:06.945912 7f5c973a57a0 20 read_log 192888'13112 (192888'13039) modify 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.942868.0:58719 2014-08-11 17:07:23.831699 2014-08-13 18:06:06.945927 7f5c973a57a0 20 read_log 192888'13114 (192888'13043) modify cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.942868.0:60204 2014-08-11 17:07:24.269977 2014-08-13 18:06:06.945940 7f5c973a57a0 20 read_log 192888'13115 (192888'13045) modify 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.942868.0:60214 2014-08-11 17:07:24.273250 2014-08-13 18:06:06.945952 7f5c973a57a0 20 read_log 192888'13117 (192888'13049) modify 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.942868.0:61101 2014-08-11 17:07:24.541956 2014-08-13 18:06:06.945964 7f5c973a57a0 20 read_log 192888'13118 (192888'13051) modify 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.942868.0:61114 2014-08-11 17:07:24.545268 2014-08-13 18:06:06.945977 7f5c973a57a0 20 read_log 192888'13119 (192888'13053) modify 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by client.942868.0:61362 2014-08-11 17:07:24.618109 2014-08-13 18:06:06.946013 7f5c973a57a0 20 read_log 193305'13124 (0'0) promote ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by osd.45.0:29 2014-08-11 18:42:38.303434 2014-08-13 18:06:06.946027 7f5c973a57a0 20 read_log 193305'13125 (193305'13124) modify ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.951337.0:1366 2014-08-11 18:42:38.293149 2014-08-13 18:06:06.946039 7f5c973a57a0 20 read_log 193305'13126 (0'0) promote b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by osd.45.0:31 2014-08-11 18:42:38.310384 2014-08-13 18:06:06.946052 7f5c973a57a0 20 read_log 193305'13127 (193305'13126) modify b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.951337.0:1368 2014-08-11 18:42:38.300363 2014-08-13 18:06:06.946064 7f5c973a57a0 20 read_log 193305'13128 (0'0) promote fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:39 2014-08-11 18:42:40.089141 2014-08-13 18:06:06.946078 7f5c973a57a0 20 read_log 193305'13129 (193305'13128) modify fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by client.951337.0:1820 2014-08-11 18:42:40.081185 2014-08-13 18:06:06.946091 7f5c973a57a0 20 read_log 193305'13130 (0'0) promote a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by osd.45.0:88 2014-08-11 18:42:50.487939 2014-08-13 18:06:06.946105 7f5c973a57a0 20 read_log 193305'13131 (193305'13130) modify a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.951337.0:4664 2014-08-11 18:42:50.480683 2014-08-13 18:06:06.946118 7f5c973a57a0 20 read_log 193305'13132 (0'0) promote c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:92 2014-08-11 18:42:51.290705 2014-08-13 18:06:06.946133 7f5c973a57a0 20 read_log 193305'13133 (193305'13132) modify c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by client.951337.0:4879 2014-08-11 18:42:51.283775 2014-08-13 18:06:06.946150 7f5c973a57a0 20 read_log 193305'13134 (0'0) promote f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by osd.45.0:132 2014-08-11 18:42:58.446992 2014-08-13 18:06:06.946165 7f5c973a57a0 20 read_log 193305'13135 (193305'13134) modify f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.951337.0:6834 2014-08-11 18:42:58.437393 2014-08-13 18:06:06.946178 7f5c973a57a0 20 read_log 193305'13136 (0'0) promote db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by osd.45.0:162 2014-08-11 18:43:07.189900 2014-08-13 18:06:06.946190 7f5c973a57a0 20 read_log 193305'13137 (193305'13136) modify db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.951337.0:9153 2014-08-11 18:43:07.179205 2014-08-13 18:06:06.946213 7f5c973a57a0 20 read_log 193305'13138 (0'0) promote e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:192 2014-08-11 18:43:16.461963 2014-08-13 18:06:06.946228 7f5c973a57a0 20 read_log 193305'13139 (193305'13138) modify e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by client.951337.0:11610 2014-08-11 18:43:16.454333 2014-08-13 18:06:06.946241 7f5c973a57a0 20 read_log 193305'13140 (0'0) promote c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by osd.45.0:196 2014-08-11 18:43:17.236684 2014-08-13 18:06:06.946253 7f5c973a57a0 20 read_log 193305'13141 (193305'13140) modify c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.951337.0:11818 2014-08-11 18:43:17.228946 2014-08-13 18:06:06.946266 7f5c973a57a0 20 read_log 193305'13142 (0'0) promote de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by osd.45.0:200 2014-08-11 18:43:20.733203 2014-08-13 18:06:06.946281 7f5c973a57a0 20 read_log 193305'13143 (193305'13142) modify de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.951337.0:12758 2014-08-11 18:43:20.726013 2014-08-13 18:06:06.946294 7f5c973a57a0 20 read_log 193305'13144 (193305'13144) modify 369/hit_set_24.369_archive_2014-08-11 18:38:30.736751_2014-08-11 18:43:33.856980/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:33.857214 2014-08-13 18:06:06.946310 7f5c973a57a0 20 read_log 193305'13145 (0'0) promote e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by osd.45.0:257 2014-08-11 18:43:33.860077 2014-08-13 18:06:06.946325 7f5c973a57a0 20 read_log 193305'13146 (193305'13145) modify e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.951337.0:16402 2014-08-11 18:43:33.849577 2014-08-13 18:06:06.946338 7f5c973a57a0 20 read_log 193305'13147 (0'0) promote 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by osd.45.0:259 2014-08-11 18:43:34.233886 2014-08-13 18:06:06.946350 7f5c973a57a0 20 read_log 193305'13148 (193305'13147) modify 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.951337.0:16512 2014-08-11 18:43:34.225802 2014-08-13 18:06:06.946368 7f5c973a57a0 20 read_log 193305'13149 (0'0) promote b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by osd.45.0:279 2014-08-11 18:43:37.018257 2014-08-13 18:06:06.946381 7f5c973a57a0 20 read_log 193305'13150 (193305'13149) modify b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.951337.0:17312 2014-08-11 18:43:37.008815 2014-08-13 18:06:06.946394 7f5c973a57a0 20 read_log 193305'13151 (0'0) promote bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by osd.45.0:311 2014-08-11 18:43:44.335616 2014-08-13 18:06:06.946410 7f5c973a57a0 20 read_log 193305'13152 (193305'13151) modify bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.951337.0:19349 2014-08-11 18:43:44.327489 2014-08-13 18:06:06.946425 7f5c973a57a0 20 read_log 193305'13153 (0'0) promote 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by osd.45.0:319 2014-08-11 18:43:46.882141 2014-08-13 18:06:06.946437 7f5c973a57a0 20 read_log 193305'13154 (193305'13153) modify 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.951337.0:19908 2014-08-11 18:43:46.270297 2014-08-13 18:06:06.946481 7f5c973a57a0 20 read_log 193305'13155 (0'0) promote abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:385 2014-08-11 18:44:00.112093 2014-08-13 18:06:06.946502 7f5c973a57a0 20 read_log 193305'13156 (193305'13155) modify abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by client.951337.0:23750 2014-08-11 18:44:00.104709 2014-08-13 18:06:06.946525 7f5c973a57a0 20 read_log 193305'13157 (0'0) promote cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by osd.45.0:391 2014-08-11 18:44:00.682529 2014-08-13 18:06:06.946550 7f5c973a57a0 20 read_log 193305'13158 (193305'13157) modify cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.951337.0:23906 2014-08-11 18:44:00.673963 2014-08-13 18:06:06.946567 7f5c973a57a0 20 read_log 193305'13159 (0'0) promote c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by osd.45.0:431 2014-08-11 18:44:10.643085 2014-08-13 18:06:06.946580 7f5c973a57a0 20 read_log 193305'13160 (193305'13159) modify c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.951337.0:26624 2014-08-11 18:44:10.634300 2014-08-13 18:06:06.946595 7f5c973a57a0 20 read_log 193305'13161 (0'0) promote 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:433 2014-08-11 18:44:10.921470 2014-08-13 18:06:06.946607 7f5c973a57a0 20 read_log 193305'13162 (193305'13161) modify 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by client.951337.0:26702 2014-08-11 18:44:10.910431 2014-08-13 18:06:06.946620 7f5c973a57a0 20 read_log 193305'13163 (0'0) promote d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by osd.45.0:460 2014-08-11 18:44:17.930311 2014-08-13 18:06:06.946635 7f5c973a57a0 20 read_log 193305'13164 (193305'13163) modify d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.951337.0:28625 2014-08-11 18:44:17.921716 2014-08-13 18:06:06.946648 7f5c973a57a0 20 read_log 193305'13165 (0'0) promote 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by osd.45.0:470 2014-08-11 18:44:19.412934 2014-08-13 18:06:06.946661 7f5c973a57a0 20 read_log 193305'13166 (193305'13165) modify 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.951337.0:29019 2014-08-11 18:44:19.404723 2014-08-13 18:06:06.946673 7f5c973a57a0 20 read_log 193305'13167 (0'0) promote c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by osd.45.0:474 2014-08-11 18:44:20.057795 2014-08-13 18:06:06.946692 7f5c973a57a0 20 read_log 193305'13168 (193305'13167) modify c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.951337.0:29197 2014-08-11 18:44:20.050376 2014-08-13 18:06:06.946709 7f5c973a57a0 20 read_log 193305'13169 (0'0) promote a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:520 2014-08-11 18:44:29.234075 2014-08-13 18:06:06.946721 7f5c973a57a0 20 read_log 193305'13170 (193305'13169) modify a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by client.951337.0:31693 2014-08-11 18:44:29.223561 2014-08-13 18:06:06.946734 7f5c973a57a0 20 read_log 193305'13171 (0'0) promote 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:532 2014-08-11 18:44:30.504922 2014-08-13 18:06:06.946746 7f5c973a57a0 20 read_log 193305'13172 (193305'13171) modify 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by client.951337.0:32068 2014-08-11 18:44:30.495713 2014-08-13 18:06:06.946759 7f5c973a57a0 20 read_log 193305'13173 (0'0) promote 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by osd.45.0:536 2014-08-11 18:44:30.819469 2014-08-13 18:06:06.946771 7f5c973a57a0 20 read_log 193305'13174 (193305'13173) modify 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.951337.0:32160 2014-08-11 18:44:30.808668 2014-08-13 18:06:06.946784 7f5c973a57a0 20 read_log 193305'13175 (0'0) promote 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by osd.45.0:544 2014-08-11 18:44:32.542139 2014-08-13 18:06:06.946796 7f5c973a57a0 20 read_log 193305'13176 (193305'13175) modify 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.951337.0:32631 2014-08-11 18:44:32.534292 2014-08-13 18:06:06.946832 7f5c973a57a0 20 read_log 193305'13177 (193305'13125) modify ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.951337.0:34077 2014-08-11 18:44:34.785894 2014-08-13 18:06:06.946849 7f5c973a57a0 20 read_log 193305'13178 (193305'13127) modify b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.951337.0:34079 2014-08-11 18:44:34.786537 2014-08-13 18:06:06.946861 7f5c973a57a0 20 read_log 193305'13179 (193305'13129) modify fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by client.951337.0:34531 2014-08-11 18:44:35.056791 2014-08-13 18:06:06.946874 7f5c973a57a0 20 read_log 193305'13180 (193305'13131) modify a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.951337.0:37375 2014-08-11 18:44:36.873993 2014-08-13 18:06:06.946886 7f5c973a57a0 20 read_log 193305'13181 (193305'13133) modify c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by client.951337.0:37590 2014-08-11 18:44:36.985643 2014-08-13 18:06:06.946899 7f5c973a57a0 20 read_log 193305'13182 (193305'13135) modify f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.951337.0:39545 2014-08-11 18:44:38.067003 2014-08-13 18:06:06.946911 7f5c973a57a0 20 read_log 193305'13183 (193305'13137) modify db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.951337.0:41864 2014-08-11 18:44:38.900327 2014-08-13 18:06:06.946923 7f5c973a57a0 20 read_log 193305'13184 (193305'13139) modify e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by client.951337.0:44321 2014-08-11 18:44:39.852111 2014-08-13 18:06:06.946936 7f5c973a57a0 20 read_log 193305'13185 (193305'13141) modify c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.951337.0:44529 2014-08-11 18:44:39.904970 2014-08-13 18:06:06.946949 7f5c973a57a0 20 read_log 193305'13186 (193305'13143) modify de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.951337.0:45469 2014-08-11 18:44:40.209935 2014-08-13 18:06:06.946963 7f5c973a57a0 20 read_log 193305'13187 (193305'13146) modify e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.951337.0:49113 2014-08-11 18:44:41.257210 2014-08-13 18:06:06.946976 7f5c973a57a0 20 read_log 193305'13188 (193305'13148) modify 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.951337.0:49223 2014-08-11 18:44:41.298646 2014-08-13 18:06:06.946988 7f5c973a57a0 20 read_log 193305'13189 (193305'13150) modify b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.951337.0:50023 2014-08-11 18:44:41.543018 2014-08-13 18:06:06.947000 7f5c973a57a0 20 read_log 193305'13190 (193305'13152) modify bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.951337.0:52060 2014-08-11 18:44:42.123016 2014-08-13 18:06:06.947012 7f5c973a57a0 20 read_log 193305'13191 (193305'13154) modify 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.951337.0:52619 2014-08-11 18:44:42.285745 2014-08-13 18:06:06.947024 7f5c973a57a0 20 read_log 193305'13192 (193305'13156) modify abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by client.951337.0:56461 2014-08-11 18:44:43.425941 2014-08-13 18:06:06.947037 7f5c973a57a0 20 read_log 193305'13193 (193305'13158) modify cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.951337.0:56617 2014-08-11 18:44:43.471577 2014-08-13 18:06:06.947049 7f5c973a57a0 20 read_log 193305'13194 (193305'13160) modify c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.951337.0:59335 2014-08-11 18:44:44.298773 2014-08-13 18:06:06.947061 7f5c973a57a0 20 read_log 193305'13195 (193305'13162) modify 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by client.951337.0:59413 2014-08-11 18:44:44.320332 2014-08-13 18:06:06.947074 7f5c973a57a0 20 read_log 193305'13196 (193305'13164) modify d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.951337.0:61336 2014-08-11 18:44:44.893215 2014-08-13 18:06:06.947096 7f5c973a57a0 20 read_log 193305'13197 (193305'13166) modify 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.951337.0:61730 2014-08-11 18:44:45.012543 2014-08-13 18:06:06.947109 7f5c973a57a0 20 read_log 193305'13198 (193305'13168) modify c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.951337.0:61908 2014-08-11 18:44:45.066822 2014-08-13 18:06:06.947123 7f5c973a57a0 20 read_log 193305'13199 (193305'13170) modify a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by client.951337.0:64404 2014-08-11 18:44:45.796744 2014-08-13 18:06:06.947135 7f5c973a57a0 20 read_log 193305'13200 (193305'13172) modify 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by client.951337.0:64779 2014-08-11 18:44:45.915396 2014-08-13 18:06:06.947159 7f5c973a57a0 20 read_log 193305'13201 (193305'13174) modify 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.951337.0:64871 2014-08-11 18:44:45.944862 2014-08-13 18:06:06.947173 7f5c973a57a0 20 read_log 193305'13202 (193305'13176) modify 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.951337.0:65342 2014-08-11 18:44:46.095252 2014-08-13 18:06:06.947185 7f5c973a57a0 20 read_log 194209'13203 (192888'12902) clean f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.45.0:1101 2014-08-12 14:06:05.766946 2014-08-13 18:06:06.947198 7f5c973a57a0 20 read_log 194209'13204 (192834'12509) clean d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.45.0:1102 2014-08-12 14:06:05.767344 2014-08-13 18:06:06.947210 7f5c973a57a0 20 read_log 194209'13205 (192888'12918) clean 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.45.0:1103 2014-08-12 14:06:05.767711 2014-08-13 18:06:06.947223 7f5c973a57a0 20 read_log 194209'13206 (192834'11756) clean 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.45.0:1104 2014-08-12 14:06:05.768422 2014-08-13 18:06:06.947236 7f5c973a57a0 20 read_log 194209'13207 (193305'13179) clean fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:1105 2014-08-12 14:06:05.817748 2014-08-13 18:06:06.947248 7f5c973a57a0 20 read_log 194209'13208 (192888'13073) clean ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.45.0:1106 2014-08-12 14:06:05.821198 2014-08-13 18:06:06.947260 7f5c973a57a0 20 read_log 194209'13209 (192888'13071) clean 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.45.0:1107 2014-08-12 14:06:05.824612 2014-08-13 18:06:06.947273 7f5c973a57a0 20 read_log 194209'13210 (192834'12675) clean 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.45.0:1108 2014-08-12 14:06:05.828241 2014-08-13 18:06:06.947288 7f5c973a57a0 20 read_log 194209'13211 (192834'12726) clean 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.45.0:1109 2014-08-12 14:06:05.831600 2014-08-13 18:06:06.947301 7f5c973a57a0 20 read_log 194209'13212 (192888'12887) clean d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.45.0:1110 2014-08-12 14:06:05.834856 2014-08-13 18:06:06.947317 7f5c973a57a0 20 read_log 194209'13213 (193305'13199) clean a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:1111 2014-08-12 14:06:05.841121 2014-08-13 18:06:06.947329 7f5c973a57a0 20 read_log 194209'13214 (192834'12653) clean dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.45.0:1112 2014-08-12 14:06:05.841414 2014-08-13 18:06:06.947342 7f5c973a57a0 20 read_log 194209'13215 (192888'12897) clean 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.45.0:1113 2014-08-12 14:06:05.844630 2014-08-13 18:06:06.947365 7f5c973a57a0 20 read_log 194209'13216 (192888'12894) clean 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.45.0:1114 2014-08-12 14:06:05.850834 2014-08-13 18:06:06.947379 7f5c973a57a0 20 read_log 194209'13217 (192834'12571) clean fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.45.0:1115 2014-08-12 14:06:05.851145 2014-08-13 18:06:06.947392 7f5c973a57a0 20 read_log 194209'13218 (192888'13077) clean 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.45.0:1116 2014-08-12 14:06:05.854494 2014-08-13 18:06:06.947408 7f5c973a57a0 20 read_log 194209'13219 (192888'12905) clean 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.45.0:1117 2014-08-12 14:06:05.858425 2014-08-13 18:06:06.947421 7f5c973a57a0 20 read_log 194209'13220 (192834'12704) clean bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.45.0:1118 2014-08-12 14:06:05.861672 2014-08-13 18:06:06.947433 7f5c973a57a0 20 read_log 194209'13221 (193305'13200) clean 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:1119 2014-08-12 14:06:05.865020 2014-08-13 18:06:06.947445 7f5c973a57a0 20 read_log 194209'13222 (192888'12874) clean f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.45.0:1120 2014-08-12 14:06:05.868389 2014-08-13 18:06:06.947461 7f5c973a57a0 20 read_log 194209'13223 (192888'13088) clean ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.45.0:1121 2014-08-12 14:06:05.871609 2014-08-13 18:06:06.947484 7f5c973a57a0 20 read_log 194209'13224 (192888'12911) clean f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.45.0:1122 2014-08-12 14:06:05.874907 2014-08-13 18:06:06.947497 7f5c973a57a0 20 read_log 194209'13225 (192834'12684) clean 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.45.0:1123 2014-08-12 14:06:05.878215 2014-08-13 18:06:06.947509 7f5c973a57a0 20 read_log 194209'13226 (192888'12901) clean 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.45.0:1124 2014-08-12 14:06:05.881476 2014-08-13 18:06:06.947531 7f5c973a57a0 20 read_log 194209'13227 (192888'13086) clean a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.45.0:1125 2014-08-12 14:06:05.884726 2014-08-13 18:06:06.947545 7f5c973a57a0 20 read_log 194209'13228 (193305'13184) clean e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:1126 2014-08-12 14:06:05.887956 2014-08-13 18:06:06.947557 7f5c973a57a0 20 read_log 194209'13229 (192888'13119) clean 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.45.0:1127 2014-08-12 14:06:05.891093 2014-08-13 18:06:06.947577 7f5c973a57a0 20 read_log 194209'13230 (192888'13099) clean abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.45.0:1128 2014-08-12 14:06:05.894347 2014-08-13 18:06:06.947590 7f5c973a57a0 20 read_log 194209'13231 (193305'13195) clean 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:1129 2014-08-12 14:06:05.900345 2014-08-13 18:06:06.947603 7f5c973a57a0 20 read_log 194209'13232 (192834'12504) clean eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.45.0:1130 2014-08-12 14:06:05.903592 2014-08-13 18:06:06.947615 7f5c973a57a0 20 read_log 194209'13233 (192888'12922) clean a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.45.0:1131 2014-08-12 14:06:05.903878 2014-08-13 18:06:06.947628 7f5c973a57a0 20 read_log 194209'13234 (192888'12899) clean f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.45.0:1132 2014-08-12 14:06:05.907079 2014-08-13 18:06:06.947658 7f5c973a57a0 20 read_log 194209'13235 (193305'13192) clean abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:1133 2014-08-12 14:06:05.910529 2014-08-13 18:06:06.947682 7f5c973a57a0 20 read_log 194209'13236 (192834'12663) clean c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.45.0:1134 2014-08-12 14:06:05.913839 2014-08-13 18:06:06.947697 7f5c973a57a0 20 read_log 194209'13237 (192888'12896) clean c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.45.0:1135 2014-08-12 14:06:05.917246 2014-08-13 18:06:06.947710 7f5c973a57a0 20 read_log 194209'13238 (192834'12721) clean f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.45.0:1136 2014-08-12 14:06:05.920334 2014-08-13 18:06:06.947723 7f5c973a57a0 20 read_log 194209'13239 (193305'13181) clean c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:1137 2014-08-12 14:06:05.923692 2014-08-13 18:06:06.947743 7f5c973a57a0 20 read_log 194209'13240 (192888'12879) clean 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.45.0:1138 2014-08-12 14:06:05.927071 2014-08-13 18:06:06.947759 7f5c973a57a0 20 read_log 194209'13241 (192888'12903) clean d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.45.0:1139 2014-08-12 14:06:05.930619 2014-08-13 18:06:06.947772 7f5c973a57a0 20 read_log 194209'13242 (192888'13100) clean 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by osd.45.0:1140 2014-08-12 14:06:05.933809 2014-08-13 18:06:06.947785 7f5c973a57a0 20 read_log 194209'13243 (192888'13098) clean e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.45.0:1141 2014-08-12 14:06:05.937278 2014-08-13 18:06:06.947802 7f5c973a57a0 20 read_log 194209'13244 (192888'13069) clean 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.45.0:1142 2014-08-12 14:06:05.940690 2014-08-13 18:06:06.947821 7f5c973a57a0 20 read_log 194209'13245 (194209'13203) delete f9d8ef69/benchmark_data_storage0111-ib.pocstor_41097_object11179/head//24 by osd.45.0:1143 2014-08-12 14:06:05.974803 2014-08-13 18:06:06.947834 7f5c973a57a0 20 read_log 194209'13246 (194209'13204) delete d98aef69/benchmark_data_storage0111-ib.pocstor_41526_object1458/head//24 by osd.45.0:1144 2014-08-12 14:06:05.977478 2014-08-13 18:06:06.947846 7f5c973a57a0 20 read_log 194209'13247 (194209'13205) delete 18e3ff69/benchmark_data_storage0111-ib.pocstor_41097_object18798/head//24 by osd.45.0:1145 2014-08-12 14:06:05.980352 2014-08-13 18:06:06.947882 7f5c973a57a0 20 read_log 194209'13248 (194209'13206) delete 1dbaff69/benchmark_data_storage0111-ib.pocstor_26659_object4559/head//24 by osd.45.0:1146 2014-08-12 14:06:05.984148 2014-08-13 18:06:06.947897 7f5c973a57a0 20 read_log 194209'13249 (194209'13207) delete fcf40369/benchmark_data_storage0111-ib.pocstor_62621_object1819/head//24 by osd.45.0:1147 2014-08-12 14:06:05.987807 2014-08-13 18:06:06.947909 7f5c973a57a0 20 read_log 194209'13250 (194209'13214) delete dfdd2369/benchmark_data_storage0111-ib.pocstor_41526_object13887/head//24 by osd.45.0:1148 2014-08-12 14:06:05.992042 2014-08-13 18:06:06.947922 7f5c973a57a0 20 read_log 194209'13251 (194209'13213) delete a32e3369/benchmark_data_storage0111-ib.pocstor_62621_object31692/head//24 by osd.45.0:1149 2014-08-12 14:06:05.995483 2014-08-13 18:06:06.947934 7f5c973a57a0 20 read_log 194209'13252 (194209'13221) delete 4a5a7369/benchmark_data_storage0111-ib.pocstor_62621_object32067/head//24 by osd.45.0:1150 2014-08-12 14:06:05.999430 2014-08-13 18:06:06.947947 7f5c973a57a0 20 read_log 194209'13253 (194209'13228) delete e6a59369/benchmark_data_storage0111-ib.pocstor_62621_object11609/head//24 by osd.45.0:1151 2014-08-12 14:06:06.003103 2014-08-13 18:06:06.947959 7f5c973a57a0 20 read_log 194209'13254 (194209'13231) delete 8fb2c369/benchmark_data_storage0111-ib.pocstor_62621_object26701/head//24 by osd.45.0:1152 2014-08-12 14:06:06.006492 2014-08-13 18:06:06.947972 7f5c973a57a0 20 read_log 194209'13255 (194209'13235) delete abcac369/benchmark_data_storage0111-ib.pocstor_62621_object23749/head//24 by osd.45.0:1153 2014-08-12 14:06:06.009732 2014-08-13 18:06:06.947994 7f5c973a57a0 20 read_log 194209'13256 (194209'13239) delete c28f369/benchmark_data_storage0111-ib.pocstor_62621_object4878/head//24 by osd.45.0:1154 2014-08-12 14:06:06.012940 2014-08-13 18:06:06.948010 7f5c973a57a0 20 read_log 194209'13257 (194209'13243) delete e62b0769/benchmark_data_storage0111-ib.pocstor_2228_object19642/head//24 by osd.45.0:1155 2014-08-12 14:06:06.048493 2014-08-13 18:06:06.948022 7f5c973a57a0 20 read_log 194209'13258 (194209'13244) delete 342769/benchmark_data_storage0111-ib.pocstor_2228_object3144/head//24 by osd.45.0:1156 2014-08-12 14:06:06.051242 2014-08-13 18:06:06.948045 7f5c973a57a0 20 read_log 194209'13259 (194209'13208) delete ea1b0369/benchmark_data_storage0111-ib.pocstor_2228_object5488/head//24 by osd.45.0:1157 2014-08-12 14:06:06.082493 2014-08-13 18:06:06.948061 7f5c973a57a0 20 read_log 194209'13260 (194209'13209) delete 9f2b0369/benchmark_data_storage0111-ib.pocstor_2228_object4246/head//24 by osd.45.0:1158 2014-08-12 14:06:06.085637 2014-08-13 18:06:06.948074 7f5c973a57a0 20 read_log 194209'13261 (194209'13210) delete 80641369/benchmark_data_storage0111-ib.pocstor_41526_object16636/head//24 by osd.45.0:1159 2014-08-12 14:06:06.088717 2014-08-13 18:06:06.948086 7f5c973a57a0 20 read_log 194209'13262 (194209'13211) delete 4bb42369/benchmark_data_storage0111-ib.pocstor_41526_object22103/head//24 by osd.45.0:1160 2014-08-12 14:06:06.091891 2014-08-13 18:06:06.948099 7f5c973a57a0 20 read_log 194209'13263 (194209'13212) delete d2a82369/benchmark_data_storage0111-ib.pocstor_41097_object4709/head//24 by osd.45.0:1161 2014-08-12 14:06:06.095090 2014-08-13 18:06:06.948116 7f5c973a57a0 20 read_log 194209'13264 (194209'13215) delete 63fe3369/benchmark_data_storage0111-ib.pocstor_41097_object7693/head//24 by osd.45.0:1162 2014-08-12 14:06:06.098340 2014-08-13 18:06:06.948129 7f5c973a57a0 20 read_log 194209'13265 (194209'13217) delete fa304369/benchmark_data_storage0111-ib.pocstor_41526_object6792/head//24 by osd.45.0:1163 2014-08-12 14:06:06.101657 2014-08-13 18:06:06.948142 7f5c973a57a0 20 read_log 194209'13266 (194209'13216) delete 2ba54369/benchmark_data_storage0111-ib.pocstor_41097_object6470/head//24 by osd.45.0:1164 2014-08-12 14:06:06.104816 2014-08-13 18:06:06.948166 7f5c973a57a0 20 read_log 194209'13267 (194209'13218) delete 14895369/benchmark_data_storage0111-ib.pocstor_2228_object6882/head//24 by osd.45.0:1165 2014-08-12 14:06:06.108292 2014-08-13 18:06:06.948179 7f5c973a57a0 20 read_log 194209'13268 (194209'13219) delete 965c6369/benchmark_data_storage0111-ib.pocstor_41097_object12462/head//24 by osd.45.0:1166 2014-08-12 14:06:06.111452 2014-08-13 18:06:06.948192 7f5c973a57a0 20 read_log 194209'13269 (194209'13220) delete bb087369/benchmark_data_storage0111-ib.pocstor_41526_object19258/head//24 by osd.45.0:1167 2014-08-12 14:06:06.114655 2014-08-13 18:06:06.948208 7f5c973a57a0 20 read_log 194209'13270 (194209'13222) delete f2fd7369/benchmark_data_storage0111-ib.pocstor_41097_object891/head//24 by osd.45.0:1168 2014-08-12 14:06:06.117788 2014-08-13 18:06:06.948220 7f5c973a57a0 20 read_log 194209'13271 (194209'13223) delete ad158369/benchmark_data_storage0111-ib.pocstor_2228_object13792/head//24 by osd.45.0:1169 2014-08-12 14:06:06.120823 2014-08-13 18:06:06.948233 7f5c973a57a0 20 read_log 194209'13272 (194209'13224) delete f52c8369/benchmark_data_storage0111-ib.pocstor_41097_object16980/head//24 by osd.45.0:1170 2014-08-12 14:06:06.123789 2014-08-13 18:06:06.948264 7f5c973a57a0 20 read_log 194209'13273 (194209'13225) delete 25ae8369/benchmark_data_storage0111-ib.pocstor_41526_object17741/head//24 by osd.45.0:1171 2014-08-12 14:06:06.126737 2014-08-13 18:06:06.948277 7f5c973a57a0 20 read_log 194209'13274 (194209'13226) delete 18629369/benchmark_data_storage0111-ib.pocstor_41097_object10529/head//24 by osd.45.0:1172 2014-08-12 14:06:06.129717 2014-08-13 18:06:06.948300 7f5c973a57a0 20 read_log 194209'13275 (194209'13227) delete a8c39369/benchmark_data_storage0111-ib.pocstor_2228_object12574/head//24 by osd.45.0:1173 2014-08-12 14:06:06.132708 2014-08-13 18:06:06.948322 7f5c973a57a0 20 read_log 194209'13276 (194209'13229) delete 42aa9369/benchmark_data_storage0111-ib.pocstor_2228_object30095/head//24 by osd.45.0:1174 2014-08-12 14:06:06.135732 2014-08-13 18:06:06.948348 7f5c973a57a0 20 read_log 194209'13277 (194209'13230) delete abaa9369/benchmark_data_storage0111-ib.pocstor_2228_object20145/head//24 by osd.45.0:1175 2014-08-12 14:06:06.138775 2014-08-13 18:06:06.948364 7f5c973a57a0 20 read_log 194209'13278 (194209'13232) delete eaaba369/benchmark_data_storage0111-ib.pocstor_41526_object1096/head//24 by osd.45.0:1176 2014-08-12 14:06:06.141686 2014-08-13 18:06:06.948376 7f5c973a57a0 20 read_log 194209'13279 (194209'13233) delete a64c369/benchmark_data_storage0111-ib.pocstor_41097_object21629/head//24 by osd.45.0:1177 2014-08-12 14:06:06.144572 2014-08-13 18:06:06.948389 7f5c973a57a0 20 read_log 194209'13280 (194209'13234) delete f309c369/benchmark_data_storage0111-ib.pocstor_41097_object8649/head//24 by osd.45.0:1178 2014-08-12 14:06:06.147522 2014-08-13 18:06:06.948401 7f5c973a57a0 20 read_log 194209'13281 (194209'13236) delete c821d369/benchmark_data_storage0111-ib.pocstor_41526_object15196/head//24 by osd.45.0:1179 2014-08-12 14:06:06.150493 2014-08-13 18:06:06.948414 7f5c973a57a0 20 read_log 194209'13282 (194209'13237) delete c86ad369/benchmark_data_storage0111-ib.pocstor_41097_object7235/head//24 by osd.45.0:1180 2014-08-12 14:06:06.153463 2014-08-13 18:06:06.948426 7f5c973a57a0 20 read_log 194209'13283 (194209'13238) delete f423f369/benchmark_data_storage0111-ib.pocstor_41526_object21297/head//24 by osd.45.0:1181 2014-08-12 14:06:06.156586 2014-08-13 18:06:06.948439 7f5c973a57a0 20 read_log 194209'13284 (194209'13240) delete 86930769/benchmark_data_storage0111-ib.pocstor_41097_object2007/head//24 by osd.45.0:1182 2014-08-12 14:06:06.159413 2014-08-13 18:06:06.948451 7f5c973a57a0 20 read_log 194209'13285 (194209'13241) delete d6740769/benchmark_data_storage0111-ib.pocstor_41097_object11357/head//24 by osd.45.0:1183 2014-08-12 14:06:06.162395 2014-08-13 18:06:06.948464 7f5c973a57a0 20 read_log 194209'13286 (192888'13084) clean 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by osd.45.0:1184 2014-08-12 14:06:06.162739 2014-08-13 18:06:06.948476 7f5c973a57a0 20 read_log 194215'13287 (194215'13287) modify 369/hit_set_24.369_archive_2014-08-12 15:32:25.738576_2014-08-12 15:49:40.152882/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:40.153160 2014-08-13 18:06:06.948489 7f5c973a57a0 20 read_log 194215'13288 (193305'13144) delete 369/hit_set_24.369_archive_2014-08-11 18:38:30.736751_2014-08-11 18:43:33.856980/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:49:40.153160 2014-08-13 18:06:06.948509 7f5c973a57a0 20 read_log 194215'13289 (194209'13242) delete 47690769/benchmark_data_storage0111-ib.pocstor_2228_object20382/head//24 by client.2174595.0:180166 0.000000 2014-08-13 18:06:06.948525 7f5c973a57a0 20 read_log 194215'13290 (194209'13286) delete 55ea2769/benchmark_data_storage0111-ib.pocstor_2228_object10723/head//24 by client.2174595.0:180168 0.000000 2014-08-13 18:06:06.948536 7f5c973a57a0 20 read_log 194215'13291 (193305'13183) clean db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by osd.45.0:2955 2014-08-12 15:49:40.170168 2014-08-13 18:06:06.948549 7f5c973a57a0 20 read_log 194215'13292 (194215'13291) delete db5b2769/benchmark_data_storage0111-ib.pocstor_62621_object9152/head//24 by client.2174595.0:180170 0.000000 2014-08-13 18:06:06.948560 7f5c973a57a0 20 read_log 194215'13293 (192888'13097) clean 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by osd.45.0:2957 2014-08-12 15:49:40.178164 2014-08-13 18:06:06.948573 7f5c973a57a0 20 read_log 194215'13294 (194215'13293) delete 46944769/benchmark_data_storage0111-ib.pocstor_2228_object19488/head//24 by client.2174595.0:180172 0.000000 2014-08-13 18:06:06.948591 7f5c973a57a0 20 read_log 194215'13295 (192888'12917) clean c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by osd.45.0:2959 2014-08-12 15:49:40.186438 2014-08-13 18:06:06.948606 7f5c973a57a0 20 read_log 194215'13296 (194215'13295) delete c6705769/benchmark_data_storage0111-ib.pocstor_41097_object18672/head//24 by client.2174595.0:180174 0.000000 2014-08-13 18:06:06.948618 7f5c973a57a0 20 read_log 194215'13297 (193305'13180) clean a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by osd.45.0:2961 2014-08-12 15:49:40.194663 2014-08-13 18:06:06.948630 7f5c973a57a0 20 read_log 194215'13298 (194215'13297) delete a31a5769/benchmark_data_storage0111-ib.pocstor_62621_object4663/head//24 by client.2174595.0:180176 0.000000 2014-08-13 18:06:06.948641 7f5c973a57a0 20 read_log 194215'13299 (192834'12617) clean ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by osd.45.0:2963 2014-08-12 15:49:40.206138 2014-08-13 18:06:06.948653 7f5c973a57a0 20 read_log 194215'13300 (194215'13299) delete ca856769/benchmark_data_storage0111-ib.pocstor_41526_object10535/head//24 by client.2174595.0:180178 0.000000 2014-08-13 18:06:06.948664 7f5c973a57a0 20 read_log 194215'13301 (192834'12587) clean d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by osd.45.0:2965 2014-08-12 15:49:40.217078 2014-08-13 18:06:06.948677 7f5c973a57a0 20 read_log 194215'13302 (194215'13301) delete d68e6769/benchmark_data_storage0111-ib.pocstor_41526_object8193/head//24 by client.2174595.0:180180 0.000000 2014-08-13 18:06:06.948687 7f5c973a57a0 20 read_log 194215'13303 (192834'12741) clean 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by osd.45.0:2967 2014-08-12 15:49:40.225443 2014-08-13 18:06:06.948700 7f5c973a57a0 20 read_log 194215'13304 (194215'13303) delete 7d07769/benchmark_data_storage0111-ib.pocstor_41526_object23242/head//24 by client.2174595.0:180182 0.000000 2014-08-13 18:06:06.948711 7f5c973a57a0 20 read_log 194215'13305 (192888'13089) clean 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by osd.45.0:2969 2014-08-12 15:49:40.233165 2014-08-13 18:06:06.948723 7f5c973a57a0 20 read_log 194215'13306 (194215'13305) delete 2a667769/benchmark_data_storage0111-ib.pocstor_2228_object14187/head//24 by client.2174595.0:180184 0.000000 2014-08-13 18:06:06.948734 7f5c973a57a0 20 read_log 194215'13307 (192888'13112) clean 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by osd.45.0:2971 2014-08-12 15:49:40.240812 2014-08-13 18:06:06.948746 7f5c973a57a0 20 read_log 194215'13308 (194215'13307) delete 2068769/benchmark_data_storage0111-ib.pocstor_2228_object27452/head//24 by client.2174595.0:180186 0.000000 2014-08-13 18:06:06.948757 7f5c973a57a0 20 read_log 194215'13309 (192888'13118) clean 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by osd.45.0:2973 2014-08-12 15:49:40.248699 2014-08-13 18:06:06.948769 7f5c973a57a0 20 read_log 194215'13310 (194215'13309) delete 99b09769/benchmark_data_storage0111-ib.pocstor_2228_object29847/head//24 by client.2174595.0:180188 0.000000 2014-08-13 18:06:06.948780 7f5c973a57a0 20 read_log 194215'13311 (192888'13070) clean de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by osd.45.0:2975 2014-08-12 15:49:40.256105 2014-08-13 18:06:06.948793 7f5c973a57a0 20 read_log 194215'13312 (194215'13311) delete de729769/benchmark_data_storage0111-ib.pocstor_2228_object3222/head//24 by client.2174595.0:180190 0.000000 2014-08-13 18:06:06.948804 7f5c973a57a0 20 read_log 194215'13313 (192834'12520) clean 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by osd.45.0:2977 2014-08-12 15:49:40.270494 2014-08-13 18:06:06.948819 7f5c973a57a0 20 read_log 194215'13314 (194215'13313) delete 69439769/benchmark_data_storage0111-ib.pocstor_41526_object2289/head//24 by client.2174595.0:180192 0.000000 2014-08-13 18:06:06.948855 7f5c973a57a0 20 read_log 194215'13315 (193305'13185) clean c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by osd.45.0:2979 2014-08-12 15:49:40.278069 2014-08-13 18:06:06.948871 7f5c973a57a0 20 read_log 194215'13316 (194215'13315) delete c1669769/benchmark_data_storage0111-ib.pocstor_62621_object11817/head//24 by client.2174595.0:180194 0.000000 2014-08-13 18:06:06.948883 7f5c973a57a0 20 read_log 194215'13317 (192834'12575) clean 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by osd.45.0:2981 2014-08-12 15:49:40.290321 2014-08-13 18:06:06.948895 7f5c973a57a0 20 read_log 194215'13318 (194215'13317) delete 59479769/benchmark_data_storage0111-ib.pocstor_41526_object6978/head//24 by client.2174595.0:180196 0.000000 2014-08-13 18:06:06.948906 7f5c973a57a0 20 read_log 194215'13319 (192888'12935) clean e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by osd.45.0:2983 2014-08-12 15:49:40.298751 2014-08-13 18:06:06.948919 7f5c973a57a0 20 read_log 194215'13320 (194215'13319) delete e509a769/benchmark_data_storage0111-ib.pocstor_41097_object30573/head//24 by client.2174595.0:180198 0.000000 2014-08-13 18:06:06.948932 7f5c973a57a0 20 read_log 194215'13321 (192888'12921) clean 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by osd.45.0:2985 2014-08-12 15:49:40.306858 2014-08-13 18:06:06.948945 7f5c973a57a0 20 read_log 194215'13322 (194215'13321) delete 564ea769/benchmark_data_storage0111-ib.pocstor_41097_object21072/head//24 by client.2174595.0:180200 0.000000 2014-08-13 18:06:06.948958 7f5c973a57a0 20 read_log 194215'13323 (192888'12934) clean 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by osd.45.0:2987 2014-08-12 15:49:40.314194 2014-08-13 18:06:06.948970 7f5c973a57a0 20 read_log 194215'13324 (194215'13323) delete 142ab769/benchmark_data_storage0111-ib.pocstor_41097_object29215/head//24 by client.2174595.0:180202 0.000000 2014-08-13 18:06:06.948981 7f5c973a57a0 20 read_log 194215'13325 (193305'13177) clean ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by osd.45.0:2989 2014-08-12 15:49:40.322145 2014-08-13 18:06:06.948996 7f5c973a57a0 20 read_log 194215'13326 (194215'13325) delete ff88c769/benchmark_data_storage0111-ib.pocstor_62621_object1365/head//24 by client.2174595.0:180204 0.000000 2014-08-13 18:06:06.949007 7f5c973a57a0 20 read_log 194215'13327 (192888'12873) clean 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by osd.45.0:2991 2014-08-12 15:49:40.329597 2014-08-13 18:06:06.949024 7f5c973a57a0 20 read_log 194215'13328 (194215'13327) delete 57e8c769/benchmark_data_storage0111-ib.pocstor_41097_object842/head//24 by client.2174595.0:180206 0.000000 2014-08-13 18:06:06.949038 7f5c973a57a0 20 read_log 194215'13329 (193305'13189) clean b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by osd.45.0:2993 2014-08-12 15:49:40.337847 2014-08-13 18:06:06.949053 7f5c973a57a0 20 read_log 194215'13330 (194215'13329) delete b3f7d769/benchmark_data_storage0111-ib.pocstor_62621_object17311/head//24 by client.2174595.0:180208 0.000000 2014-08-13 18:06:06.949064 7f5c973a57a0 20 read_log 194215'13331 (193305'13188) clean 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by osd.45.0:2995 2014-08-12 15:49:40.345259 2014-08-13 18:06:06.949078 7f5c973a57a0 20 read_log 194215'13332 (194215'13331) delete 152ad769/benchmark_data_storage0111-ib.pocstor_62621_object16511/head//24 by client.2174595.0:180210 0.000000 2014-08-13 18:06:06.949351 7f5c973a57a0 20 read_log 194215'13333 (192834'12523) clean 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by osd.45.0:2997 2014-08-12 15:49:40.356720 2014-08-13 18:06:06.949369 7f5c973a57a0 20 read_log 194215'13334 (194215'13333) delete 3e0ed769/benchmark_data_storage0111-ib.pocstor_41526_object2307/head//24 by client.2174595.0:180212 0.000000 2014-08-13 18:06:06.949388 7f5c973a57a0 20 read_log 194215'13335 (192888'12904) clean 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by osd.45.0:2999 2014-08-12 15:49:40.365404 2014-08-13 18:06:06.949403 7f5c973a57a0 20 read_log 194215'13336 (194215'13335) delete 8843f769/benchmark_data_storage0111-ib.pocstor_41097_object11731/head//24 by client.2174595.0:180214 0.000000 2014-08-13 18:06:06.949415 7f5c973a57a0 20 read_log 194215'13337 (192888'13095) clean 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by osd.45.0:3001 2014-08-12 15:49:40.373167 2014-08-13 18:06:06.949427 7f5c973a57a0 20 read_log 194215'13338 (194215'13337) delete 4e330b69/benchmark_data_storage0111-ib.pocstor_2228_object19145/head//24 by client.2174595.0:180216 0.000000 2014-08-13 18:06:06.949438 7f5c973a57a0 20 read_log 194215'13339 (192834'11882) clean 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by osd.45.0:3003 2014-08-12 15:49:40.380550 2014-08-13 18:06:06.949451 7f5c973a57a0 20 read_log 194215'13340 (194215'13339) delete 86930b69/benchmark_data_storage0111-ib.pocstor_26659_object19119/head//24 by client.2174595.0:180218 0.000000 2014-08-13 18:06:06.949485 7f5c973a57a0 20 read_log 194215'13341 (192834'11837) clean 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by osd.45.0:3005 2014-08-12 15:49:40.387983 2014-08-13 18:06:06.949501 7f5c973a57a0 20 read_log 194215'13342 (194215'13341) delete 63370b69/benchmark_data_storage0111-ib.pocstor_26659_object13688/head//24 by client.2174595.0:180220 0.000000 2014-08-13 18:06:06.949515 7f5c973a57a0 20 read_log 194215'13343 (192834'11895) clean 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by osd.45.0:3007 2014-08-12 15:49:40.396164 2014-08-13 18:06:06.949528 7f5c973a57a0 20 read_log 194215'13344 (194215'13343) delete 21d01b69/benchmark_data_storage0111-ib.pocstor_26659_object20677/head//24 by client.2174595.0:180222 0.000000 2014-08-13 18:06:06.949539 7f5c973a57a0 20 read_log 194215'13345 (192834'12623) clean f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by osd.45.0:3009 2014-08-12 15:49:40.404584 2014-08-13 18:06:06.949554 7f5c973a57a0 20 read_log 194215'13346 (194215'13345) delete f9251b69/benchmark_data_storage0111-ib.pocstor_41526_object11256/head//24 by client.2174595.0:180224 0.000000 2014-08-13 18:06:06.949566 7f5c973a57a0 20 read_log 194215'13347 (192834'11784) clean 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by osd.45.0:3011 2014-08-12 15:49:40.412273 2014-08-13 18:06:06.949592 7f5c973a57a0 20 read_log 194215'13348 (194215'13347) delete 83e61b69/benchmark_data_storage0111-ib.pocstor_26659_object7028/head//24 by client.2174595.0:180226 0.000000 2014-08-13 18:06:06.949610 7f5c973a57a0 20 read_log 194215'13349 (192834'12544) clean 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by osd.45.0:3013 2014-08-12 15:49:40.419554 2014-08-13 18:06:06.949625 7f5c973a57a0 20 read_log 194215'13350 (194215'13349) delete 7d5e1b69/benchmark_data_storage0111-ib.pocstor_41526_object4670/head//24 by client.2174595.0:180228 0.000000 2014-08-13 18:06:06.949641 7f5c973a57a0 20 read_log 194215'13351 (192888'12883) clean e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by osd.45.0:3015 2014-08-12 15:49:40.427267 2014-08-13 18:06:06.949656 7f5c973a57a0 20 read_log 194215'13352 (194215'13351) delete e912b69/benchmark_data_storage0111-ib.pocstor_41097_object3067/head//24 by client.2174595.0:180230 0.000000 2014-08-13 18:06:06.949668 7f5c973a57a0 20 read_log 194215'13353 (192834'11770) clean 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by osd.45.0:3017 2014-08-12 15:49:40.435123 2014-08-13 18:06:06.949689 7f5c973a57a0 20 read_log 194215'13354 (194215'13353) delete 4d892b69/benchmark_data_storage0111-ib.pocstor_26659_object5833/head//24 by client.2174595.0:180232 0.000000 2014-08-13 18:06:06.949711 7f5c973a57a0 20 read_log 194215'13355 (192834'11887) clean f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by osd.45.0:3019 2014-08-12 15:49:40.443201 2014-08-13 18:06:06.949753 7f5c973a57a0 20 read_log 194215'13356 (194215'13355) delete f1be2b69/benchmark_data_storage0111-ib.pocstor_26659_object19448/head//24 by client.2174595.0:180234 0.000000 2014-08-13 18:06:06.949766 7f5c973a57a0 20 read_log 194215'13357 (193305'13191) clean 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by osd.45.0:3021 2014-08-12 15:49:40.451547 2014-08-13 18:06:06.949779 7f5c973a57a0 20 read_log 194215'13358 (194215'13357) delete 6d313b69/benchmark_data_storage0111-ib.pocstor_62621_object19907/head//24 by client.2174595.0:180236 0.000000 2014-08-13 18:06:06.949804 7f5c973a57a0 20 read_log 194215'13359 (192834'12584) clean d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by osd.45.0:3023 2014-08-12 15:49:40.459929 2014-08-13 18:06:06.949817 7f5c973a57a0 20 read_log 194215'13360 (194215'13359) delete d1cc3b69/benchmark_data_storage0111-ib.pocstor_41526_object7913/head//24 by client.2174595.0:180238 0.000000 2014-08-13 18:06:06.949828 7f5c973a57a0 20 read_log 194215'13361 (193305'13186) clean de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by osd.45.0:3025 2014-08-12 15:49:40.467173 2014-08-13 18:06:06.949841 7f5c973a57a0 20 read_log 194215'13362 (194215'13361) delete de204b69/benchmark_data_storage0111-ib.pocstor_62621_object12757/head//24 by client.2174595.0:180240 0.000000 2014-08-13 18:06:06.949852 7f5c973a57a0 20 read_log 194215'13363 (192888'13111) clean df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by osd.45.0:3027 2014-08-12 15:49:40.475017 2014-08-13 18:06:06.949864 7f5c973a57a0 20 read_log 194215'13364 (194215'13363) delete df6a4b69/benchmark_data_storage0111-ib.pocstor_2228_object27026/head//24 by client.2174595.0:180242 0.000000 2014-08-13 18:06:06.949875 7f5c973a57a0 20 read_log 194215'13365 (192888'12892) clean db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by osd.45.0:3029 2014-08-12 15:49:40.482990 2014-08-13 18:06:06.949888 7f5c973a57a0 20 read_log 194215'13366 (194215'13365) delete db015b69/benchmark_data_storage0111-ib.pocstor_41097_object6209/head//24 by client.2174595.0:180244 0.000000 2014-08-13 18:06:06.949899 7f5c973a57a0 20 read_log 194215'13367 (192834'12597) clean 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by osd.45.0:3031 2014-08-12 15:49:40.491182 2014-08-13 18:06:06.949911 7f5c973a57a0 20 read_log 194215'13368 (194215'13367) delete 63fb5b69/benchmark_data_storage0111-ib.pocstor_41526_object9083/head//24 by client.2174595.0:180246 0.000000 2014-08-13 18:06:06.949929 7f5c973a57a0 20 read_log 194215'13369 (192888'13114) clean cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by osd.45.0:3033 2014-08-12 15:49:40.498963 2014-08-13 18:06:06.949946 7f5c973a57a0 20 read_log 194215'13370 (194215'13369) delete cac5b69/benchmark_data_storage0111-ib.pocstor_2228_object28937/head//24 by client.2174595.0:180248 0.000000 2014-08-13 18:06:06.949958 7f5c973a57a0 20 read_log 194215'13371 (192888'13115) clean 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by osd.45.0:3035 2014-08-12 15:49:40.507492 2014-08-13 18:06:06.949970 7f5c973a57a0 20 read_log 194215'13372 (194215'13371) delete 60b06b69/benchmark_data_storage0111-ib.pocstor_2228_object28947/head//24 by client.2174595.0:180250 0.000000 2014-08-13 18:06:06.949981 7f5c973a57a0 20 read_log 194215'13373 (192888'13082) clean 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by osd.45.0:3037 2014-08-12 15:49:40.515221 2014-08-13 18:06:06.949994 7f5c973a57a0 20 read_log 194215'13374 (194215'13373) delete 63f66b69/benchmark_data_storage0111-ib.pocstor_2228_object8874/head//24 by client.2174595.0:180252 0.000000 2014-08-13 18:06:06.950023 7f5c973a57a0 20 read_log 194215'13375 (192888'12893) clean 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by osd.45.0:3039 2014-08-12 15:49:40.523105 2014-08-13 18:06:06.950039 7f5c973a57a0 20 read_log 194215'13376 (194215'13375) delete 99886b69/benchmark_data_storage0111-ib.pocstor_41097_object6275/head//24 by client.2174595.0:180254 0.000000 2014-08-13 18:06:06.950050 7f5c973a57a0 20 read_log 194215'13377 (192888'13068) clean 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by osd.45.0:3041 2014-08-12 15:49:40.530246 2014-08-13 18:06:06.950063 7f5c973a57a0 20 read_log 194215'13378 (194215'13377) delete 65c86b69/benchmark_data_storage0111-ib.pocstor_2228_object2620/head//24 by client.2174595.0:180256 0.000000 2014-08-13 18:06:06.950074 7f5c973a57a0 20 read_log 194215'13379 (192888'12909) clean 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by osd.45.0:3043 2014-08-12 15:49:40.538634 2014-08-13 18:06:06.950086 7f5c973a57a0 20 read_log 194215'13380 (194215'13379) delete 6d4c6b69/benchmark_data_storage0111-ib.pocstor_41097_object14753/head//24 by client.2174595.0:180258 0.000000 2014-08-13 18:06:06.950117 7f5c973a57a0 20 read_log 194215'13381 (193305'13197) clean 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by osd.45.0:3045 2014-08-12 15:49:40.546188 2014-08-13 18:06:06.950133 7f5c973a57a0 20 read_log 194215'13382 (194215'13381) delete 574e6b69/benchmark_data_storage0111-ib.pocstor_62621_object29018/head//24 by client.2174595.0:180260 0.000000 2014-08-13 18:06:06.950145 7f5c973a57a0 20 read_log 194215'13383 (192888'12886) clean c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by osd.45.0:3047 2014-08-12 15:49:40.553772 2014-08-13 18:06:06.950165 7f5c973a57a0 20 read_log 194215'13384 (194215'13383) delete c8637b69/benchmark_data_storage0111-ib.pocstor_41097_object4060/head//24 by client.2174595.0:180262 0.000000 2014-08-13 18:06:06.950176 7f5c973a57a0 20 read_log 194215'13385 (193305'13194) clean c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by osd.45.0:3049 2014-08-12 15:49:40.562187 2014-08-13 18:06:06.950189 7f5c973a57a0 20 read_log 194215'13386 (194215'13385) delete c79d7b69/benchmark_data_storage0111-ib.pocstor_62621_object26623/head//24 by client.2174595.0:180264 0.000000 2014-08-13 18:06:06.950200 7f5c973a57a0 20 read_log 194215'13387 (192834'12712) clean 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by osd.45.0:3051 2014-08-12 15:49:40.570787 2014-08-13 18:06:06.950213 7f5c973a57a0 20 read_log 194215'13388 (194215'13387) delete 62008b69/benchmark_data_storage0111-ib.pocstor_41526_object20182/head//24 by client.2174595.0:180266 0.000000 2014-08-13 18:06:06.950226 7f5c973a57a0 20 read_log 194215'13389 (192888'12895) clean 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by osd.45.0:3053 2014-08-12 15:49:40.578826 2014-08-13 18:06:06.950238 7f5c973a57a0 20 read_log 194215'13390 (194215'13389) delete 57778b69/benchmark_data_storage0111-ib.pocstor_41097_object6642/head//24 by client.2174595.0:180268 0.000000 2014-08-13 18:06:06.950249 7f5c973a57a0 20 read_log 194215'13391 (192888'12877) clean 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by osd.45.0:3055 2014-08-12 15:49:40.587240 2014-08-13 18:06:06.950262 7f5c973a57a0 20 read_log 194215'13392 (194215'13391) delete 247b9b69/benchmark_data_storage0111-ib.pocstor_41097_object1640/head//24 by client.2174595.0:180270 0.000000 2014-08-13 18:06:06.950273 7f5c973a57a0 20 read_log 194215'13393 (193305'13193) clean cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by osd.45.0:3057 2014-08-12 15:49:40.595118 2014-08-13 18:06:06.950285 7f5c973a57a0 20 read_log 194215'13394 (194215'13393) delete cd22ab69/benchmark_data_storage0111-ib.pocstor_62621_object23905/head//24 by client.2174595.0:180272 0.000000 2014-08-13 18:06:06.950307 7f5c973a57a0 20 read_log 194215'13395 (192834'12635) clean b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by osd.45.0:3059 2014-08-12 15:49:40.603338 2014-08-13 18:06:06.950322 7f5c973a57a0 20 read_log 194215'13396 (194215'13395) delete b9cab69/benchmark_data_storage0111-ib.pocstor_41526_object12645/head//24 by client.2174595.0:180274 0.000000 2014-08-13 18:06:06.950333 7f5c973a57a0 20 read_log 194215'13397 (192888'12885) clean 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by osd.45.0:3061 2014-08-12 15:49:40.610674 2014-08-13 18:06:06.950346 7f5c973a57a0 20 read_log 194215'13398 (194215'13397) delete 80d1bb69/benchmark_data_storage0111-ib.pocstor_41097_object3893/head//24 by client.2174595.0:180276 0.000000 2014-08-13 18:06:06.950357 7f5c973a57a0 20 read_log 194215'13399 (192888'12929) clean 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by osd.45.0:3063 2014-08-12 15:49:40.618226 2014-08-13 18:06:06.950369 7f5c973a57a0 20 read_log 194215'13400 (194215'13399) delete 5bb4cb69/benchmark_data_storage0111-ib.pocstor_41097_object25832/head//24 by client.2174595.0:180278 0.000000 2014-08-13 18:06:06.950380 7f5c973a57a0 20 read_log 194215'13401 (192888'12910) clean 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by osd.45.0:3065 2014-08-12 15:49:40.626164 2014-08-13 18:06:06.950392 7f5c973a57a0 20 read_log 194215'13402 (194215'13401) delete 925acb69/benchmark_data_storage0111-ib.pocstor_41097_object16282/head//24 by client.2174595.0:180280 0.000000 2014-08-13 18:06:06.950403 7f5c973a57a0 20 read_log 194215'13403 (192834'11749) clean 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by osd.45.0:3067 2014-08-12 15:49:40.633952 2014-08-13 18:06:06.950416 7f5c973a57a0 20 read_log 194215'13404 (194215'13403) delete 7b32db69/benchmark_data_storage0111-ib.pocstor_26659_object2662/head//24 by client.2174595.0:180282 0.000000 2014-08-13 18:06:06.950434 7f5c973a57a0 20 read_log 194215'13405 (192888'12890) clean afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by osd.45.0:3069 2014-08-12 15:49:40.642162 2014-08-13 18:06:06.950451 7f5c973a57a0 20 read_log 194215'13406 (194215'13405) delete afd8db69/benchmark_data_storage0111-ib.pocstor_41097_object5043/head//24 by client.2174595.0:180284 0.000000 2014-08-13 18:06:06.950467 7f5c973a57a0 20 read_log 194215'13407 (193305'13190) clean bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by osd.45.0:3071 2014-08-12 15:49:40.649819 2014-08-13 18:06:06.950489 7f5c973a57a0 20 read_log 194215'13408 (194215'13407) delete bc4adb69/benchmark_data_storage0111-ib.pocstor_62621_object19348/head//24 by client.2174595.0:180286 0.000000 2014-08-13 18:06:06.950503 7f5c973a57a0 20 read_log 194215'13409 (192888'13096) clean 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by osd.45.0:3073 2014-08-12 15:49:40.657008 2014-08-13 18:06:06.950516 7f5c973a57a0 20 read_log 194215'13410 (194215'13409) delete 9e5edb69/benchmark_data_storage0111-ib.pocstor_2228_object19322/head//24 by client.2174595.0:180288 0.000000 2014-08-13 18:06:06.950527 7f5c973a57a0 20 read_log 194215'13411 (193305'13182) clean f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by osd.45.0:3075 2014-08-12 15:49:40.665307 2014-08-13 18:06:06.950540 7f5c973a57a0 20 read_log 194215'13412 (194215'13411) delete f7baeb69/benchmark_data_storage0111-ib.pocstor_62621_object6833/head//24 by client.2174595.0:180290 0.000000 2014-08-13 18:06:06.950551 7f5c973a57a0 20 read_log 194215'13413 (192834'11854) clean 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by osd.45.0:3077 2014-08-12 15:49:40.673007 2014-08-13 18:06:06.950563 7f5c973a57a0 20 read_log 194215'13414 (194215'13413) delete 59aceb69/benchmark_data_storage0111-ib.pocstor_26659_object15362/head//24 by client.2174595.0:180292 0.000000 2014-08-13 18:06:06.950581 7f5c973a57a0 20 read_log 194215'13415 (192888'13092) clean fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by osd.45.0:3079 2014-08-12 15:49:40.680756 2014-08-13 18:06:06.950596 7f5c973a57a0 20 read_log 194215'13416 (194215'13415) delete fa0deb69/benchmark_data_storage0111-ib.pocstor_2228_object15071/head//24 by client.2174595.0:180294 0.000000 2014-08-13 18:06:06.950607 7f5c973a57a0 20 read_log 194215'13417 (192834'12732) clean 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by osd.45.0:3081 2014-08-12 15:49:40.688893 2014-08-13 18:06:06.950620 7f5c973a57a0 20 read_log 194215'13418 (194215'13417) delete 4fc6fb69/benchmark_data_storage0111-ib.pocstor_41526_object22472/head//24 by client.2174595.0:180296 0.000000 2014-08-13 18:06:06.950631 7f5c973a57a0 20 read_log 194215'13419 (192888'12931) clean 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by osd.45.0:3083 2014-08-12 15:49:40.697032 2014-08-13 18:06:06.950643 7f5c973a57a0 20 read_log 194215'13420 (194215'13419) delete 3029fb69/benchmark_data_storage0111-ib.pocstor_41097_object27826/head//24 by client.2174595.0:180298 0.000000 2014-08-13 18:06:06.950655 7f5c973a57a0 20 read_log 194215'13421 (192834'12569) clean 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by osd.45.0:3085 2014-08-12 15:49:40.704900 2014-08-13 18:06:06.950667 7f5c973a57a0 20 read_log 194215'13422 (194215'13421) delete 4f060f69/benchmark_data_storage0111-ib.pocstor_41526_object6770/head//24 by client.2174595.0:180300 0.000000 2014-08-13 18:06:06.950678 7f5c973a57a0 20 read_log 194215'13423 (192888'12920) clean 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by osd.45.0:3087 2014-08-12 15:49:40.713079 2014-08-13 18:06:06.950691 7f5c973a57a0 20 read_log 194215'13424 (194215'13423) delete 3e290f69/benchmark_data_storage0111-ib.pocstor_41097_object20364/head//24 by client.2174595.0:180302 0.000000 2014-08-13 18:06:06.950702 7f5c973a57a0 20 read_log 194215'13425 (192888'13103) clean 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by osd.45.0:3089 2014-08-12 15:49:40.722446 2014-08-13 18:06:06.950714 7f5c973a57a0 20 read_log 194215'13426 (194215'13425) delete 6f7e0f69/benchmark_data_storage0111-ib.pocstor_2228_object21792/head//24 by client.2174595.0:180304 0.000000 2014-08-13 18:06:06.950725 7f5c973a57a0 20 read_log 194215'13427 (192888'13109) clean 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by osd.45.0:3091 2014-08-12 15:49:40.729801 2014-08-13 18:06:06.950738 7f5c973a57a0 20 read_log 194215'13428 (194215'13427) delete 4a721f69/benchmark_data_storage0111-ib.pocstor_2228_object25345/head//24 by client.2174595.0:180306 0.000000 2014-08-13 18:06:06.950749 7f5c973a57a0 20 read_log 194215'13429 (192888'12875) clean e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by osd.45.0:3093 2014-08-12 15:49:40.737477 2014-08-13 18:06:06.950761 7f5c973a57a0 20 read_log 194215'13430 (194215'13429) delete e80d1f69/benchmark_data_storage0111-ib.pocstor_41097_object983/head//24 by client.2174595.0:180308 0.000000 2014-08-13 18:06:06.950773 7f5c973a57a0 20 read_log 194215'13431 (193305'13201) clean 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by osd.45.0:3095 2014-08-12 15:49:40.744824 2014-08-13 18:06:06.950785 7f5c973a57a0 20 read_log 194215'13432 (194215'13431) delete 443f1f69/benchmark_data_storage0111-ib.pocstor_62621_object32159/head//24 by client.2174595.0:180310 0.000000 2014-08-13 18:06:06.950798 7f5c973a57a0 20 read_log 194215'13433 (193305'13202) clean 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by osd.45.0:3097 2014-08-12 15:49:40.752303 2014-08-13 18:06:06.950830 7f5c973a57a0 20 read_log 194215'13434 (194215'13433) delete 3f713f69/benchmark_data_storage0111-ib.pocstor_62621_object32630/head//24 by client.2174595.0:180312 0.000000 2014-08-13 18:06:06.950851 7f5c973a57a0 20 read_log 194215'13435 (192888'13090) clean c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by osd.45.0:3099 2014-08-12 15:49:40.760095 2014-08-13 18:06:06.950866 7f5c973a57a0 20 read_log 194215'13436 (194215'13435) delete c6543f69/benchmark_data_storage0111-ib.pocstor_2228_object14391/head//24 by client.2174595.0:180314 0.000000 2014-08-13 18:06:06.950877 7f5c973a57a0 20 read_log 194215'13437 (192888'12932) clean f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by osd.45.0:3101 2014-08-12 15:49:40.767826 2014-08-13 18:06:06.950890 7f5c973a57a0 20 read_log 194215'13438 (194215'13437) delete f4393f69/benchmark_data_storage0111-ib.pocstor_41097_object28273/head//24 by client.2174595.0:180316 0.000000 2014-08-13 18:06:06.950901 7f5c973a57a0 20 read_log 194215'13439 (192888'13106) clean 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by osd.45.0:3103 2014-08-12 15:49:40.776314 2014-08-13 18:06:06.950913 7f5c973a57a0 20 read_log 194215'13440 (194215'13439) delete 32e34f69/benchmark_data_storage0111-ib.pocstor_2228_object24090/head//24 by client.2174595.0:180318 0.000000 2014-08-13 18:06:06.950925 7f5c973a57a0 20 read_log 194215'13441 (192888'12884) clean 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by osd.45.0:3105 2014-08-12 15:49:40.784405 2014-08-13 18:06:06.950939 7f5c973a57a0 20 read_log 194215'13442 (194215'13441) delete 41635f69/benchmark_data_storage0111-ib.pocstor_41097_object3775/head//24 by client.2174595.0:180320 0.000000 2014-08-13 18:06:06.950950 7f5c973a57a0 20 read_log 194215'13443 (192888'13117) clean 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by osd.45.0:3107 2014-08-12 15:49:40.792240 2014-08-13 18:06:06.950963 7f5c973a57a0 20 read_log 194215'13444 (194215'13443) delete 1c355f69/benchmark_data_storage0111-ib.pocstor_2228_object29834/head//24 by client.2174595.0:180322 0.000000 2014-08-13 18:06:06.950975 7f5c973a57a0 20 read_log 194215'13445 (192834'12739) clean b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by osd.45.0:3109 2014-08-12 15:49:40.801432 2014-08-13 18:06:06.950990 7f5c973a57a0 20 read_log 194215'13446 (194215'13445) delete b7706f69/benchmark_data_storage0111-ib.pocstor_41526_object23054/head//24 by client.2174595.0:180324 0.000000 2014-08-13 18:06:06.951001 7f5c973a57a0 20 read_log 194215'13447 (192888'12916) clean 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by osd.45.0:3111 2014-08-12 15:49:40.808291 2014-08-13 18:06:06.951016 7f5c973a57a0 20 read_log 194215'13448 (194215'13447) delete 5f907f69/benchmark_data_storage0111-ib.pocstor_41097_object18295/head//24 by client.2174595.0:180326 0.000000 2014-08-13 18:06:06.951027 7f5c973a57a0 20 read_log 194215'13449 (192888'13083) clean 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by osd.45.0:3113 2014-08-12 15:49:40.815359 2014-08-13 18:06:06.951041 7f5c973a57a0 20 read_log 194215'13450 (194215'13449) delete 6fc87f69/benchmark_data_storage0111-ib.pocstor_2228_object9768/head//24 by client.2174595.0:180328 0.000000 2014-08-13 18:06:06.951052 7f5c973a57a0 20 read_log 194215'13451 (193305'13196) clean d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by osd.45.0:3115 2014-08-12 15:49:40.822707 2014-08-13 18:06:06.951067 7f5c973a57a0 20 read_log 194215'13452 (194215'13451) delete d33a7f69/benchmark_data_storage0111-ib.pocstor_62621_object28624/head//24 by client.2174595.0:180330 0.000000 2014-08-13 18:06:06.951078 7f5c973a57a0 20 read_log 194215'13453 (193305'13198) clean c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by osd.45.0:3117 2014-08-12 15:49:40.830345 2014-08-13 18:06:06.951092 7f5c973a57a0 20 read_log 194215'13454 (194215'13453) delete c8258f69/benchmark_data_storage0111-ib.pocstor_62621_object29196/head//24 by client.2174595.0:180332 0.000000 2014-08-13 18:06:06.951129 7f5c973a57a0 20 read_log 194215'13455 (193305'13187) clean e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by osd.45.0:3119 2014-08-12 15:49:40.838265 2014-08-13 18:06:06.951153 7f5c973a57a0 20 read_log 194215'13456 (194215'13455) delete e9c09f69/benchmark_data_storage0111-ib.pocstor_62621_object16401/head//24 by client.2174595.0:180334 0.000000 2014-08-13 18:06:06.951168 7f5c973a57a0 20 read_log 194215'13457 (192888'13075) clean 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by osd.45.0:3121 2014-08-12 15:49:40.846159 2014-08-13 18:06:06.951181 7f5c973a57a0 20 read_log 194215'13458 (194215'13457) delete 18d69f69/benchmark_data_storage0111-ib.pocstor_2228_object6422/head//24 by client.2174595.0:180336 0.000000 2014-08-13 18:06:06.951192 7f5c973a57a0 20 read_log 194215'13459 (193305'13178) clean b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by osd.45.0:3123 2014-08-12 15:49:40.853841 2014-08-13 18:06:06.951204 7f5c973a57a0 20 read_log 194215'13460 (194215'13459) delete b70d9f69/benchmark_data_storage0111-ib.pocstor_62621_object1367/head//24 by client.2174595.0:180338 0.000000 2014-08-13 18:06:06.951215 7f5c973a57a0 20 read_log 194215'13461 (192834'12559) clean 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by osd.45.0:3125 2014-08-12 15:49:40.861122 2014-08-13 18:06:06.951227 7f5c973a57a0 20 read_log 194215'13462 (194215'13461) delete 3047af69/benchmark_data_storage0111-ib.pocstor_41526_object6086/head//24 by client.2174595.0:180340 0.000000 2014-08-13 18:06:06.951238 7f5c973a57a0 20 read_log 194215'13463 (192834'11789) clean 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by osd.45.0:3127 2014-08-12 15:49:40.868792 2014-08-13 18:06:06.951253 7f5c973a57a0 20 read_log 194215'13464 (194215'13463) delete 9745bf69/benchmark_data_storage0111-ib.pocstor_26659_object7565/head//24 by client.2174595.0:180342 0.000000 2014-08-13 18:06:06.951264 7f5c973a57a0 20 read_log 194215'13465 (192834'12582) clean 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by osd.45.0:3129 2014-08-12 15:49:40.875934 2014-08-13 18:06:06.951279 7f5c973a57a0 20 read_log 194215'13466 (194215'13465) delete 78f2cf69/benchmark_data_storage0111-ib.pocstor_41526_object7113/head//24 by client.2174595.0:180344 0.000000 2014-08-13 18:06:06.951290 7f5c973a57a0 20 read_log 194215'13467 (192888'13104) clean ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by osd.45.0:3131 2014-08-12 15:49:40.883057 2014-08-13 18:06:06.951307 7f5c973a57a0 20 read_log 194215'13468 (194215'13467) delete ce8fcf69/benchmark_data_storage0111-ib.pocstor_2228_object23131/head//24 by client.2174595.0:180346 0.000000 2014-08-13 18:06:06.951319 7f5c973a57a0 20 read_log 194215'13469 (192888'12919) clean d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by osd.45.0:3133 2014-08-12 15:49:40.890159 2014-08-13 18:06:06.951331 7f5c973a57a0 20 read_log 194215'13470 (194215'13469) delete d119df69/benchmark_data_storage0111-ib.pocstor_41097_object19233/head//24 by client.2174595.0:180348 0.000000 2014-08-13 18:06:06.951348 7f5c973a57a0 20 read_log 194215'13471 (192834'12674) clean c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by osd.45.0:3135 2014-08-12 15:49:40.897524 2014-08-13 18:06:06.951363 7f5c973a57a0 20 read_log 194215'13472 (194215'13471) delete c0bdf69/benchmark_data_storage0111-ib.pocstor_41526_object16595/head//24 by client.2174595.0:180350 0.000000 2014-08-13 18:06:06.951374 7f5c973a57a0 20 read_log 194215'13473 (192834'11881) clean 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by osd.45.0:3137 2014-08-12 15:49:40.904791 2014-08-13 18:06:06.951387 7f5c973a57a0 20 read_log 194215'13474 (194215'13473) delete 893ddf69/benchmark_data_storage0111-ib.pocstor_26659_object18991/head//24 by client.2174595.0:180352 0.000000 2014-08-13 18:06:06.951408 7f5c973a57a0 20 read_log 194215'13475 (192888'12878) clean ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by osd.45.0:3139 2014-08-12 15:49:40.912294 2014-08-13 18:06:06.951422 7f5c973a57a0 20 read_log 194215'13476 (194215'13475) delete ae22ef69/benchmark_data_storage0111-ib.pocstor_41097_object1795/head//24 by client.2174595.0:180354 0.000000 2014-08-13 18:06:06.951447 7f5c973a57a0 20 read_log 194370'13477 (0'0) promote ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by osd.45.0:3399 2014-08-12 20:30:22.021328 2014-08-13 18:06:06.951460 7f5c973a57a0 20 read_log 194370'13478 (194370'13477) modify ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by client.1508531.0:2028 2014-08-12 20:30:22.014253 2014-08-13 18:06:06.951481 7f5c973a57a0 20 read_log 194370'13479 (0'0) promote fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by osd.45.0:3405 2014-08-12 20:30:22.454132 2014-08-13 18:06:06.951494 7f5c973a57a0 20 read_log 194370'13480 (194370'13479) modify fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by client.1508531.0:2148 2014-08-12 20:30:22.447561 2014-08-13 18:06:06.951515 7f5c973a57a0 20 read_log 194370'13481 (0'0) promote 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by osd.45.0:3437 2014-08-12 20:30:28.913119 2014-08-13 18:06:06.951540 7f5c973a57a0 20 read_log 194370'13482 (194370'13481) modify 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by client.1508531.0:3896 2014-08-12 20:30:28.906404 2014-08-13 18:06:06.951557 7f5c973a57a0 20 read_log 194370'13483 (0'0) promote 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by osd.45.0:3441 2014-08-12 20:30:29.602747 2014-08-13 18:06:06.951569 7f5c973a57a0 20 read_log 194370'13484 (194370'13483) modify 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by client.1508531.0:4083 2014-08-12 20:30:29.596244 2014-08-13 18:06:06.951592 7f5c973a57a0 20 read_log 194370'13485 (0'0) promote 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by osd.45.0:3503 2014-08-12 20:30:41.005350 2014-08-13 18:06:06.951605 7f5c973a57a0 20 read_log 194370'13486 (194370'13485) modify 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by client.1508531.0:6729 2014-08-12 20:30:40.996084 2014-08-13 18:06:06.951618 7f5c973a57a0 20 read_log 194370'13487 (0'0) promote e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by osd.45.0:3523 2014-08-12 20:30:45.669858 2014-08-13 18:06:06.951630 7f5c973a57a0 20 read_log 194370'13488 (194370'13487) modify e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by client.1508531.0:7646 2014-08-12 20:30:45.662745 2014-08-13 18:06:06.951643 7f5c973a57a0 20 read_log 194370'13489 (0'0) promote 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by osd.45.0:3533 2014-08-12 20:30:48.035291 2014-08-13 18:06:06.951655 7f5c973a57a0 20 read_log 194370'13490 (194370'13489) modify 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by client.1508531.0:8084 2014-08-12 20:30:48.028390 2014-08-13 18:06:06.951667 7f5c973a57a0 20 read_log 194370'13491 (0'0) promote d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by osd.45.0:3539 2014-08-12 20:30:48.160452 2014-08-13 18:06:06.951679 7f5c973a57a0 20 read_log 194370'13492 (194370'13491) modify d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by client.1508531.0:8107 2014-08-12 20:30:48.149665 2014-08-13 18:06:06.951692 7f5c973a57a0 20 read_log 194370'13493 (0'0) promote 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by osd.45.0:3569 2014-08-12 20:31:00.741134 2014-08-13 18:06:06.951704 7f5c973a57a0 20 read_log 194370'13494 (194370'13493) modify 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by client.1508531.0:10253 2014-08-12 20:31:00.733201 2014-08-13 18:06:06.951725 7f5c973a57a0 20 read_log 194370'13495 (0'0) promote 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by osd.45.0:3575 2014-08-12 20:31:02.107572 2014-08-13 18:06:06.951740 7f5c973a57a0 20 read_log 194370'13496 (194370'13495) modify 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by client.1508531.0:10481 2014-08-12 20:31:02.100588 2014-08-13 18:06:06.951753 7f5c973a57a0 20 read_log 194370'13497 (0'0) promote d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by osd.45.0:3591 2014-08-12 20:31:06.337415 2014-08-13 18:06:06.951765 7f5c973a57a0 20 read_log 194370'13498 (194370'13497) modify d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by client.1508531.0:11177 2014-08-12 20:31:06.329571 2014-08-13 18:06:06.951778 7f5c973a57a0 20 read_log 194370'13499 (0'0) promote e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by osd.45.0:3593 2014-08-12 20:31:06.860950 2014-08-13 18:06:06.951790 7f5c973a57a0 20 read_log 194370'13500 (194370'13499) modify e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by client.1508531.0:11261 2014-08-12 20:31:06.852597 2014-08-13 18:06:06.951802 7f5c973a57a0 20 read_log 194370'13501 (0'0) promote 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by osd.45.0:3601 2014-08-12 20:31:08.926182 2014-08-13 18:06:06.951814 7f5c973a57a0 20 read_log 194370'13502 (194370'13501) modify 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by client.1508531.0:11600 2014-08-12 20:31:08.918385 2014-08-13 18:06:06.951827 7f5c973a57a0 20 read_log 194370'13503 (0'0) promote 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by osd.45.0:3605 2014-08-12 20:31:11.137458 2014-08-13 18:06:06.951848 7f5c973a57a0 20 read_log 194370'13504 (194370'13503) modify 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by client.1508531.0:11969 2014-08-12 20:31:11.129136 2014-08-13 18:06:06.951864 7f5c973a57a0 20 read_log 194370'13505 (0'0) promote 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by osd.45.0:3613 2014-08-12 20:31:23.482958 2014-08-13 18:06:06.951877 7f5c973a57a0 20 read_log 194370'13506 (194370'13505) modify 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by client.1508531.0:13431 2014-08-12 20:31:23.475320 2014-08-13 18:06:06.951900 7f5c973a57a0 20 read_log 194370'13507 (0'0) promote 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by osd.45.0:3615 2014-08-12 20:31:23.587754 2014-08-13 18:06:06.951913 7f5c973a57a0 20 read_log 194370'13508 (194370'13507) modify 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by client.1508531.0:13442 2014-08-12 20:31:23.580280 2014-08-13 18:06:06.951925 7f5c973a57a0 20 read_log 194370'13509 (0'0) promote fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by osd.45.0:3629 2014-08-12 20:31:28.499813 2014-08-13 18:06:06.951938 7f5c973a57a0 20 read_log 194370'13510 (194370'13509) modify fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by client.1508531.0:13865 2014-08-12 20:31:28.491484 2014-08-13 18:06:06.951950 7f5c973a57a0 20 read_log 194377'13511 (0'0) promote b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by osd.45.0:3645 2014-08-12 20:33:01.112480 2014-08-13 18:06:06.951963 7f5c973a57a0 20 read_log 194377'13512 (194377'13511) modify b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by client.2231190.0:950 2014-08-12 20:33:01.106449 2014-08-13 18:06:06.951975 7f5c973a57a0 20 read_log 194377'13513 (0'0) promote fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by osd.45.0:3659 2014-08-12 20:33:05.700290 2014-08-13 18:06:06.951988 7f5c973a57a0 20 read_log 194377'13514 (194377'13513) modify fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by client.2231190.0:2105 2014-08-12 20:33:05.692627 2014-08-13 18:06:06.952008 7f5c973a57a0 20 read_log 194377'13515 (0'0) promote 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by osd.45.0:3661 2014-08-12 20:33:05.714862 2014-08-13 18:06:06.952026 7f5c973a57a0 20 read_log 194377'13516 (194377'13515) modify 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by client.2231190.0:2109 2014-08-12 20:33:05.709251 2014-08-13 18:06:06.952039 7f5c973a57a0 20 read_log 194377'13517 (0'0) promote 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by osd.45.0:3707 2014-08-12 20:33:15.694535 2014-08-13 18:06:06.952052 7f5c973a57a0 20 read_log 194377'13518 (194377'13517) modify 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by client.2231190.0:4296 2014-08-12 20:33:15.688131 2014-08-13 18:06:06.952064 7f5c973a57a0 20 read_log 194382'13519 (0'0) promote 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by osd.45.0:3731 2014-08-12 20:33:59.599982 2014-08-13 18:06:06.952077 7f5c973a57a0 20 read_log 194382'13520 (194382'13519) modify 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by client.2231190.0:5693 2014-08-12 20:33:59.591804 2014-08-13 18:06:06.952089 7f5c973a57a0 20 read_log 194382'13521 (0'0) promote 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by osd.45.0:3733 2014-08-12 20:33:59.975295 2014-08-13 18:06:06.952101 7f5c973a57a0 20 read_log 194382'13522 (194382'13521) modify 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by client.2231190.0:5763 2014-08-12 20:33:59.966938 2014-08-13 18:06:06.952114 7f5c973a57a0 20 read_log 194383'13523 (0'0) promote 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by osd.45.0:3745 2014-08-12 20:34:02.795334 2014-08-13 18:06:06.952126 7f5c973a57a0 20 read_log 194383'13524 (194383'13523) modify 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by client.2231190.0:6262 2014-08-12 20:34:02.788571 2014-08-13 18:06:06.952139 7f5c973a57a0 20 read_log 194383'13525 (0'0) promote 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by osd.45.0:3751 2014-08-12 20:34:07.121273 2014-08-13 18:06:06.952165 7f5c973a57a0 20 read_log 194383'13526 (194383'13525) modify 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by client.2231190.0:6704 2014-08-12 20:34:07.114278 2014-08-13 18:06:06.952183 7f5c973a57a0 20 read_log 194393'13527 (0'0) promote 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by osd.45.0:3772 2014-08-12 20:47:25.921551 2014-08-13 18:06:06.952210 7f5c973a57a0 20 read_log 194393'13528 (194393'13527) modify 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by client.1064095.0:3 2014-08-12 20:47:25.910100 2014-08-13 18:06:06.952226 7f5c973a57a0 20 read_log 194393'13529 (0'0) promote 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by osd.45.0:3799 2014-08-12 20:52:33.410142 2014-08-13 18:06:06.952239 7f5c973a57a0 20 read_log 194393'13530 (194393'13529) modify 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by client.1064422.0:218 2014-08-12 20:52:33.403536 2014-08-13 18:06:06.952260 7f5c973a57a0 20 read_log 194397'13531 (0'0) promote 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by osd.45.0:3820 2014-08-12 20:53:03.600982 2014-08-13 18:06:06.952276 7f5c973a57a0 20 read_log 194397'13532 (194397'13531) modify 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by client.2235069.0:797 2014-08-12 20:53:03.593214 2014-08-13 18:06:06.952289 7f5c973a57a0 20 read_log 194416'13533 (0'0) promote 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by osd.45.0:3934 2014-08-13 15:28:32.170447 2014-08-13 18:06:06.952325 7f5c973a57a0 20 read_log 194416'13534 (194416'13533) modify 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by client.2441958.0:448 2014-08-13 15:28:32.162246 2014-08-13 18:06:06.952342 7f5c973a57a0 20 read_log 194416'13535 (0'0) promote b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by osd.45.0:3962 2014-08-13 15:28:39.954509 2014-08-13 18:06:06.952354 7f5c973a57a0 20 read_log 194416'13536 (194416'13535) modify b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by client.2441958.0:2439 2014-08-13 15:28:39.945728 2014-08-13 18:06:06.952367 7f5c973a57a0 20 read_log 194416'13537 (0'0) promote 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by osd.45.0:3980 2014-08-13 15:28:45.583488 2014-08-13 18:06:06.952379 7f5c973a57a0 20 read_log 194416'13538 (194416'13537) modify 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by client.2441958.0:3908 2014-08-13 15:28:45.577009 2014-08-13 18:06:06.952396 7f5c973a57a0 20 read_log 194416'13539 (0'0) promote ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by osd.45.0:3982 2014-08-13 15:28:45.959559 2014-08-13 18:06:06.952409 7f5c973a57a0 20 read_log 194416'13540 (194416'13539) modify ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by client.2441958.0:4007 2014-08-13 15:28:45.952340 2014-08-13 18:06:06.952430 7f5c973a57a0 20 read_log 194416'13541 (0'0) promote bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by osd.45.0:4037 2014-08-13 15:28:56.428969 2014-08-13 18:06:06.952446 7f5c973a57a0 20 read_log 194416'13542 (194416'13541) modify bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by client.2441958.0:6636 2014-08-13 15:28:56.420019 2014-08-13 18:06:06.952459 7f5c973a57a0 20 read_log 194416'13543 (0'0) promote 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by osd.45.0:4051 2014-08-13 15:28:57.822315 2014-08-13 18:06:06.952471 7f5c973a57a0 20 read_log 194416'13544 (194416'13543) modify 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by client.2441958.0:7012 2014-08-13 15:28:57.812126 2014-08-13 18:06:06.952484 7f5c973a57a0 20 read_log 194416'13545 (0'0) promote b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by osd.45.0:4053 2014-08-13 15:28:58.067874 2014-08-13 18:06:06.952496 7f5c973a57a0 20 read_log 194416'13546 (194416'13545) modify b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by client.2441958.0:7077 2014-08-13 15:28:58.055264 2014-08-13 18:06:06.952509 7f5c973a57a0 20 read_log 194416'13547 (0'0) promote 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by osd.45.0:4101 2014-08-13 15:29:05.158874 2014-08-13 18:06:06.952542 7f5c973a57a0 20 read_log 194416'13548 (194416'13547) modify 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by client.2441958.0:8931 2014-08-13 15:29:05.149794 2014-08-13 18:06:06.952557 7f5c973a57a0 20 read_log 194416'13549 (0'0) promote 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by osd.45.0:4157 2014-08-13 15:29:16.079732 2014-08-13 18:06:06.952570 7f5c973a57a0 20 read_log 194416'13550 (194416'13549) modify 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by client.2441958.0:11642 2014-08-13 15:29:16.070252 2014-08-13 18:06:06.952582 7f5c973a57a0 20 read_log 194416'13551 (0'0) promote f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by osd.45.0:4185 2014-08-13 15:29:24.195078 2014-08-13 18:06:06.952595 7f5c973a57a0 20 read_log 194416'13552 (194416'13551) modify f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by client.2441958.0:13520 2014-08-13 15:29:24.186479 2014-08-13 18:06:06.952608 7f5c973a57a0 20 read_log 194416'13553 (0'0) promote 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by osd.45.0:4207 2014-08-13 15:29:31.601741 2014-08-13 18:06:06.952628 7f5c973a57a0 20 read_log 194416'13554 (194416'13553) modify 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by client.2441958.0:15295 2014-08-13 15:29:31.592735 2014-08-13 18:06:06.952643 7f5c973a57a0 20 read_log 194422'13555 (0'0) promote 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by osd.45.0:4267 2014-08-13 15:32:16.666305 2014-08-13 18:06:06.952655 7f5c973a57a0 20 read_log 194422'13556 (194422'13555) modify 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by client.1148203.0:357 2014-08-13 15:32:16.659486 2014-08-13 18:06:06.952668 7f5c973a57a0 20 read_log 194422'13557 (0'0) promote b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by osd.45.0:4271 2014-08-13 15:32:18.007212 2014-08-13 18:06:06.952680 7f5c973a57a0 20 read_log 194422'13558 (194422'13557) modify b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by client.1148203.0:660 2014-08-13 15:32:17.997715 2014-08-13 18:06:06.952692 7f5c973a57a0 20 read_log 194422'13559 (0'0) promote ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by osd.45.0:4315 2014-08-13 15:32:27.587934 2014-08-13 18:06:06.952705 7f5c973a57a0 20 read_log 194422'13560 (194422'13559) modify ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by client.1148203.0:3129 2014-08-13 15:32:27.579892 2014-08-13 18:06:06.952717 7f5c973a57a0 20 read_log 194422'13561 (0'0) promote 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by osd.45.0:4317 2014-08-13 15:32:27.917519 2014-08-13 18:06:06.952729 7f5c973a57a0 20 read_log 194422'13562 (194422'13561) modify 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by client.1148203.0:3216 2014-08-13 15:32:27.910541 2014-08-13 18:06:06.952742 7f5c973a57a0 20 read_log 194422'13563 (0'0) promote 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by osd.45.0:4319 2014-08-13 15:32:28.298173 2014-08-13 18:06:06.952754 7f5c973a57a0 20 read_log 194422'13564 (194422'13563) modify 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by client.1148203.0:3301 2014-08-13 15:32:28.289735 2014-08-13 18:06:06.952766 7f5c973a57a0 20 read_log 194422'13565 (0'0) promote 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by osd.45.0:4359 2014-08-13 15:32:41.147567 2014-08-13 18:06:06.952778 7f5c973a57a0 20 read_log 194422'13566 (194422'13565) modify 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by client.1148203.0:6186 2014-08-13 15:32:41.139420 2014-08-13 18:06:06.952791 7f5c973a57a0 20 read_log 194422'13567 (0'0) promote 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by osd.45.0:4365 2014-08-13 15:32:42.381606 2014-08-13 18:06:06.952803 7f5c973a57a0 20 read_log 194422'13568 (194422'13567) modify 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by client.1148203.0:6412 2014-08-13 15:32:42.373457 2014-08-13 18:06:06.952815 7f5c973a57a0 20 read_log 194422'13569 (0'0) promote cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by osd.45.0:4425 2014-08-13 15:33:00.878658 2014-08-13 18:06:06.952828 7f5c973a57a0 20 read_log 194422'13570 (194422'13569) modify cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by client.1148203.0:9134 2014-08-13 15:33:00.869629 2014-08-13 18:06:06.952843 7f5c973a57a0 20 read_log 194422'13571 (0'0) promote 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by osd.45.0:4427 2014-08-13 15:33:01.333839 2014-08-13 18:06:06.952855 7f5c973a57a0 20 read_log 194422'13572 (194422'13571) modify 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by client.1148203.0:9180 2014-08-13 15:33:01.324967 2014-08-13 18:06:06.952870 7f5c973a57a0 20 read_log 194422'13573 (0'0) promote 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by osd.45.0:4441 2014-08-13 15:33:11.111036 2014-08-13 18:06:06.952884 7f5c973a57a0 20 read_log 194422'13574 (194422'13573) modify 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by client.1148203.0:10028 2014-08-13 15:33:11.102604 2014-08-13 18:06:06.952899 7f5c973a57a0 20 read_log 194422'13575 (0'0) promote a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by osd.45.0:4445 2014-08-13 15:33:25.819849 2014-08-13 18:06:06.952912 7f5c973a57a0 20 read_log 194422'13576 (194422'13575) modify a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by client.1148203.0:10567 2014-08-13 15:33:25.810541 2014-08-13 18:06:06.952930 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.953473 7f5c973a57a0 10 read_log checking for missing items over interval (194370'13477,194422'13576] 2014-08-13 18:06:06.953583 7f5c973a57a0 15 read_log missing 194422'13576 (194422'13575) modify a5e76b69/benchmark_data_storage0111-ib.pocstor_41231_object10566/head//24 by client.1148203.0:10567 2014-08-13 15:33:25.810541 2014-08-13 18:06:06.953645 7f5c973a57a0 15 read_log missing 194422'13574 (194422'13573) modify 3c254769/benchmark_data_storage0111-ib.pocstor_41231_object10027/head//24 by client.1148203.0:10028 2014-08-13 15:33:11.102604 2014-08-13 18:06:06.953697 7f5c973a57a0 15 read_log missing 194422'13572 (194422'13571) modify 760d1769/benchmark_data_storage0111-ib.pocstor_41231_object9179/head//24 by client.1148203.0:9180 2014-08-13 15:33:01.324967 2014-08-13 18:06:06.953743 7f5c973a57a0 15 read_log missing 194422'13570 (194422'13569) modify cac4ff69/benchmark_data_storage0111-ib.pocstor_41231_object9133/head//24 by client.1148203.0:9134 2014-08-13 15:33:00.869629 2014-08-13 18:06:06.953780 7f5c973a57a0 15 read_log missing 194422'13568 (194422'13567) modify 2600bb69/benchmark_data_storage0111-ib.pocstor_41231_object6411/head//24 by client.1148203.0:6412 2014-08-13 15:32:42.373457 2014-08-13 18:06:06.953816 7f5c973a57a0 15 read_log missing 194422'13566 (194422'13565) modify 2858b769/benchmark_data_storage0111-ib.pocstor_41231_object6185/head//24 by client.1148203.0:6186 2014-08-13 15:32:41.139420 2014-08-13 18:06:06.953851 7f5c973a57a0 15 read_log missing 194422'13564 (194422'13563) modify 234b1369/benchmark_data_storage0111-ib.pocstor_41231_object3300/head//24 by client.1148203.0:3301 2014-08-13 15:32:28.289735 2014-08-13 18:06:06.953886 7f5c973a57a0 15 read_log missing 194422'13562 (194422'13561) modify 7e389369/benchmark_data_storage0111-ib.pocstor_41231_object3215/head//24 by client.1148203.0:3216 2014-08-13 15:32:27.910541 2014-08-13 18:06:06.953922 7f5c973a57a0 15 read_log missing 194422'13560 (194422'13559) modify ebd20b69/benchmark_data_storage0111-ib.pocstor_41231_object3128/head//24 by client.1148203.0:3129 2014-08-13 15:32:27.579892 2014-08-13 18:06:06.953958 7f5c973a57a0 15 read_log missing 194422'13558 (194422'13557) modify b3653b69/benchmark_data_storage0111-ib.pocstor_41231_object659/head//24 by client.1148203.0:660 2014-08-13 15:32:17.997715 2014-08-13 18:06:06.953994 7f5c973a57a0 15 read_log missing 194422'13556 (194422'13555) modify 6af25f69/benchmark_data_storage0111-ib.pocstor_41231_object356/head//24 by client.1148203.0:357 2014-08-13 15:32:16.659486 2014-08-13 18:06:06.954030 7f5c973a57a0 15 read_log missing 194416'13554 (194416'13553) modify 21acff69/benchmark_data_storage0111-ib.pocstor_19103_object15294/head//24 by client.2441958.0:15295 2014-08-13 15:29:31.592735 2014-08-13 18:06:06.954065 7f5c973a57a0 15 read_log missing 194416'13552 (194416'13551) modify f1da8f69/benchmark_data_storage0111-ib.pocstor_19103_object13519/head//24 by client.2441958.0:13520 2014-08-13 15:29:24.186479 2014-08-13 18:06:06.954100 7f5c973a57a0 15 read_log missing 194416'13550 (194416'13549) modify 32285b69/benchmark_data_storage0111-ib.pocstor_19103_object11641/head//24 by client.2441958.0:11642 2014-08-13 15:29:16.070252 2014-08-13 18:06:06.954136 7f5c973a57a0 15 read_log missing 194416'13548 (194416'13547) modify 105ecb69/benchmark_data_storage0111-ib.pocstor_19103_object8930/head//24 by client.2441958.0:8931 2014-08-13 15:29:05.149794 2014-08-13 18:06:06.954179 7f5c973a57a0 15 read_log missing 194416'13546 (194416'13545) modify b74a0769/benchmark_data_storage0111-ib.pocstor_19103_object7076/head//24 by client.2441958.0:7077 2014-08-13 15:28:58.055264 2014-08-13 18:06:06.954215 7f5c973a57a0 15 read_log missing 194416'13544 (194416'13543) modify 1471369/benchmark_data_storage0111-ib.pocstor_19103_object7011/head//24 by client.2441958.0:7012 2014-08-13 15:28:57.812126 2014-08-13 18:06:06.954255 7f5c973a57a0 15 read_log missing 194416'13542 (194416'13541) modify bcb44b69/benchmark_data_storage0111-ib.pocstor_19103_object6635/head//24 by client.2441958.0:6636 2014-08-13 15:28:56.420019 2014-08-13 18:06:06.954291 7f5c973a57a0 15 read_log missing 194416'13540 (194416'13539) modify ac8a2769/benchmark_data_storage0111-ib.pocstor_19103_object4006/head//24 by client.2441958.0:4007 2014-08-13 15:28:45.952340 2014-08-13 18:06:06.954327 7f5c973a57a0 15 read_log missing 194416'13538 (194416'13537) modify 6f1e1369/benchmark_data_storage0111-ib.pocstor_19103_object3907/head//24 by client.2441958.0:3908 2014-08-13 15:28:45.577009 2014-08-13 18:06:06.954362 7f5c973a57a0 15 read_log missing 194416'13536 (194416'13535) modify b3734369/benchmark_data_storage0111-ib.pocstor_19103_object2438/head//24 by client.2441958.0:2439 2014-08-13 15:28:39.945728 2014-08-13 18:06:06.954414 7f5c973a57a0 15 read_log missing 194416'13534 (194416'13533) modify 7c132b69/benchmark_data_storage0111-ib.pocstor_19103_object447/head//24 by client.2441958.0:448 2014-08-13 15:28:32.162246 2014-08-13 18:06:06.954461 7f5c973a57a0 15 read_log missing 194397'13532 (194397'13531) modify 9d4b6369/benchmark_data_storage0101-ib.pocstor_6819_object796/head//24 by client.2235069.0:797 2014-08-12 20:53:03.593214 2014-08-13 18:06:06.954499 7f5c973a57a0 15 read_log missing 194393'13530 (194393'13529) modify 36786369/benchmark_data_storage0111-ib.pocstor_48304_object217/head//24 by client.1064422.0:218 2014-08-12 20:52:33.403536 2014-08-13 18:06:06.954535 7f5c973a57a0 15 read_log missing 194393'13528 (194393'13527) modify 6e18a369/benchmark_data_storage0111-ib.pocstor_24170_object2/head//24 by client.1064095.0:3 2014-08-12 20:47:25.910100 2014-08-13 18:06:06.954571 7f5c973a57a0 15 read_log missing 194383'13526 (194383'13525) modify 4d5d6f69/benchmark_data_storage0111-ib.pocstor_25065_object6703/head//24 by client.2231190.0:6704 2014-08-12 20:34:07.114278 2014-08-13 18:06:06.954607 7f5c973a57a0 15 read_log missing 194383'13524 (194383'13523) modify 879d2769/benchmark_data_storage0111-ib.pocstor_25065_object6261/head//24 by client.2231190.0:6262 2014-08-12 20:34:02.788571 2014-08-13 18:06:06.954661 7f5c973a57a0 15 read_log missing 194382'13522 (194382'13521) modify 533b8769/benchmark_data_storage0111-ib.pocstor_25065_object5762/head//24 by client.2231190.0:5763 2014-08-12 20:33:59.966938 2014-08-13 18:06:06.954713 7f5c973a57a0 15 read_log missing 194382'13520 (194382'13519) modify 8de22f69/benchmark_data_storage0111-ib.pocstor_25065_object5692/head//24 by client.2231190.0:5693 2014-08-12 20:33:59.591804 2014-08-13 18:06:06.954767 7f5c973a57a0 15 read_log missing 194377'13518 (194377'13517) modify 6e40fb69/benchmark_data_storage0111-ib.pocstor_25065_object4295/head//24 by client.2231190.0:4296 2014-08-12 20:33:15.688131 2014-08-13 18:06:06.954821 7f5c973a57a0 15 read_log missing 194377'13516 (194377'13515) modify 95c8369/benchmark_data_storage0111-ib.pocstor_25065_object2108/head//24 by client.2231190.0:2109 2014-08-12 20:33:05.709251 2014-08-13 18:06:06.954876 7f5c973a57a0 15 read_log missing 194377'13514 (194377'13513) modify fbb47f69/benchmark_data_storage0111-ib.pocstor_25065_object2104/head//24 by client.2231190.0:2105 2014-08-12 20:33:05.692627 2014-08-13 18:06:06.954915 7f5c973a57a0 15 read_log missing 194377'13512 (194377'13511) modify b94b7b69/benchmark_data_storage0111-ib.pocstor_25065_object949/head//24 by client.2231190.0:950 2014-08-12 20:33:01.106449 2014-08-13 18:06:06.954951 7f5c973a57a0 15 read_log missing 194370'13510 (194370'13509) modify fa75db69/benchmark_data_storage0111-ib.pocstor_12495_object13864/head//24 by client.1508531.0:13865 2014-08-12 20:31:28.491484 2014-08-13 18:06:06.955002 7f5c973a57a0 15 read_log missing 194370'13508 (194370'13507) modify 681a9b69/benchmark_data_storage0111-ib.pocstor_12495_object13441/head//24 by client.1508531.0:13442 2014-08-12 20:31:23.580280 2014-08-13 18:06:06.955040 7f5c973a57a0 15 read_log missing 194370'13506 (194370'13505) modify 96eae369/benchmark_data_storage0111-ib.pocstor_12495_object13430/head//24 by client.1508531.0:13431 2014-08-12 20:31:23.475320 2014-08-13 18:06:06.955082 7f5c973a57a0 15 read_log missing 194370'13504 (194370'13503) modify 624bb369/benchmark_data_storage0111-ib.pocstor_12495_object11968/head//24 by client.1508531.0:11969 2014-08-12 20:31:11.129136 2014-08-13 18:06:06.955127 7f5c973a57a0 15 read_log missing 194370'13502 (194370'13501) modify 45bdb769/benchmark_data_storage0111-ib.pocstor_12495_object11599/head//24 by client.1508531.0:11600 2014-08-12 20:31:08.918385 2014-08-13 18:06:06.955180 7f5c973a57a0 15 read_log missing 194370'13500 (194370'13499) modify e9498769/benchmark_data_storage0111-ib.pocstor_12495_object11260/head//24 by client.1508531.0:11261 2014-08-12 20:31:06.852597 2014-08-13 18:06:06.955220 7f5c973a57a0 15 read_log missing 194370'13498 (194370'13497) modify d5d4a369/benchmark_data_storage0111-ib.pocstor_12495_object11176/head//24 by client.1508531.0:11177 2014-08-12 20:31:06.329571 2014-08-13 18:06:06.955257 7f5c973a57a0 15 read_log missing 194370'13496 (194370'13495) modify 13fdf369/benchmark_data_storage0111-ib.pocstor_12495_object10480/head//24 by client.1508531.0:10481 2014-08-12 20:31:02.100588 2014-08-13 18:06:06.955291 7f5c973a57a0 15 read_log missing 194370'13494 (194370'13493) modify 8ce07369/benchmark_data_storage0111-ib.pocstor_12495_object10252/head//24 by client.1508531.0:10253 2014-08-12 20:31:00.733201 2014-08-13 18:06:06.955329 7f5c973a57a0 15 read_log missing 194370'13492 (194370'13491) modify d90c7f69/benchmark_data_storage0111-ib.pocstor_12495_object8106/head//24 by client.1508531.0:8107 2014-08-12 20:30:48.149665 2014-08-13 18:06:06.955367 7f5c973a57a0 15 read_log missing 194370'13490 (194370'13489) modify 60a8f369/benchmark_data_storage0111-ib.pocstor_12495_object8083/head//24 by client.1508531.0:8084 2014-08-12 20:30:48.028390 2014-08-13 18:06:06.955403 7f5c973a57a0 15 read_log missing 194370'13488 (194370'13487) modify e2ceab69/benchmark_data_storage0111-ib.pocstor_12495_object7645/head//24 by client.1508531.0:7646 2014-08-12 20:30:45.662745 2014-08-13 18:06:06.955452 7f5c973a57a0 15 read_log missing 194370'13486 (194370'13485) modify 1bc17b69/benchmark_data_storage0111-ib.pocstor_12495_object6728/head//24 by client.1508531.0:6729 2014-08-12 20:30:40.996084 2014-08-13 18:06:06.955489 7f5c973a57a0 15 read_log missing 194370'13484 (194370'13483) modify 31c95369/benchmark_data_storage0111-ib.pocstor_12495_object4082/head//24 by client.1508531.0:4083 2014-08-12 20:30:29.596244 2014-08-13 18:06:06.955526 7f5c973a57a0 15 read_log missing 194370'13482 (194370'13481) modify 7514c769/benchmark_data_storage0111-ib.pocstor_12495_object3895/head//24 by client.1508531.0:3896 2014-08-12 20:30:28.906404 2014-08-13 18:06:06.955562 7f5c973a57a0 15 read_log missing 194370'13480 (194370'13479) modify fb895769/benchmark_data_storage0111-ib.pocstor_12495_object2147/head//24 by client.1508531.0:2148 2014-08-12 20:30:22.447561 2014-08-13 18:06:06.955598 7f5c973a57a0 15 read_log missing 194370'13478 (194370'13477) modify ca841b69/benchmark_data_storage0111-ib.pocstor_12495_object2027/head//24 by client.1508531.0:2028 2014-08-12 20:30:22.014253 2014-08-13 18:06:06.955609 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.955637 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=0 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive m=50] handle_loaded 2014-08-13 18:06:06.955654 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=0 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Initial 0.016065 0 0.000000 2014-08-13 18:06:06.955666 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=0 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Reset 2014-08-13 18:06:06.955675 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=0 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194487 2014-08-13 18:06:06.955684 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log((192834'11428,194422'13576], crt=194422'13576) 2014-08-13 18:06:06.955695 7f5c973a57a0 10 osd.4 194487 pgid 24.39a coll 24.39a_head 2014-08-13 18:06:06.955864 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.39a 2014-08-13 18:06:06.955878 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.955910 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] enter Initial 2014-08-13 18:06:06.955922 7f5c973a57a0 20 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] enter NotTrimming 2014-08-13 18:06:06.956109 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.956222 7f5c973a57a0 20 read_log 192834'11375 (0'0) promote 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by osd.346.0:12983 2014-08-11 15:23:40.711310 2014-08-13 18:06:06.956249 7f5c973a57a0 20 read_log 192834'11376 (192834'11375) modify 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.933448.0:1300 2014-08-11 15:23:40.698804 2014-08-13 18:06:06.956264 7f5c973a57a0 20 read_log 192834'11425 (0'0) promote df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by osd.346.0:13033 2014-08-11 15:23:53.464058 2014-08-13 18:06:06.956277 7f5c973a57a0 20 read_log 192834'11426 (192834'11425) modify df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.933448.0:3754 2014-08-11 15:23:53.456241 2014-08-13 18:06:06.956289 7f5c973a57a0 20 read_log 192834'11427 (0'0) promote 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by osd.346.0:13035 2014-08-11 15:23:55.456867 2014-08-13 18:06:06.956302 7f5c973a57a0 20 read_log 192834'11428 (192834'11427) modify 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.933448.0:4098 2014-08-11 15:23:55.449029 2014-08-13 18:06:06.956319 7f5c973a57a0 20 read_log 192834'11453 (0'0) promote 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by osd.346.0:13061 2014-08-11 15:24:03.729715 2014-08-13 18:06:06.956334 7f5c973a57a0 20 read_log 192834'11454 (192834'11453) modify 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.933448.0:6000 2014-08-11 15:24:03.719699 2014-08-13 18:06:06.956360 7f5c973a57a0 20 read_log 192834'11495 (0'0) promote 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by osd.346.0:13103 2014-08-11 15:24:17.183348 2014-08-13 18:06:06.956375 7f5c973a57a0 20 read_log 192834'11496 (192834'11495) modify 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.933448.0:9009 2014-08-11 15:24:17.173764 2014-08-13 18:06:06.956388 7f5c973a57a0 20 read_log 192834'11505 (0'0) promote 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by osd.346.0:13113 2014-08-11 15:24:19.664646 2014-08-13 18:06:06.956400 7f5c973a57a0 20 read_log 192834'11506 (192834'11505) modify 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.933448.0:9646 2014-08-11 15:24:19.655719 2014-08-13 18:06:06.956413 7f5c973a57a0 20 read_log 192834'11525 (0'0) promote 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by osd.346.0:13133 2014-08-11 15:24:25.529612 2014-08-13 18:06:06.956444 7f5c973a57a0 20 read_log 192834'11526 (192834'11525) modify 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.933448.0:10863 2014-08-11 15:24:25.519540 2014-08-13 18:06:06.956458 7f5c973a57a0 20 read_log 192834'11527 (0'0) promote d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by osd.346.0:13135 2014-08-11 15:24:25.589303 2014-08-13 18:06:06.956470 7f5c973a57a0 20 read_log 192834'11528 (192834'11527) modify d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.933448.0:10880 2014-08-11 15:24:25.581648 2014-08-13 18:06:06.956482 7f5c973a57a0 20 read_log 192834'11529 (0'0) promote 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by osd.346.0:13137 2014-08-11 15:24:25.722823 2014-08-13 18:06:06.956494 7f5c973a57a0 20 read_log 192834'11530 (192834'11529) modify 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.933448.0:10915 2014-08-11 15:24:25.713221 2014-08-13 18:06:06.956507 7f5c973a57a0 20 read_log 192834'11533 (0'0) promote 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by osd.346.0:13141 2014-08-11 15:24:26.729678 2014-08-13 18:06:06.956519 7f5c973a57a0 20 read_log 192834'11534 (192834'11533) modify 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.933448.0:11177 2014-08-11 15:24:26.718603 2014-08-13 18:06:06.956584 7f5c973a57a0 20 read_log 192834'11543 (0'0) promote b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by osd.346.0:13151 2014-08-11 15:24:29.652401 2014-08-13 18:06:06.956598 7f5c973a57a0 20 read_log 192834'11544 (192834'11543) modify b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.933448.0:11857 2014-08-11 15:24:29.643421 2014-08-13 18:06:06.956610 7f5c973a57a0 20 read_log 192834'11565 (0'0) promote 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by osd.346.0:13173 2014-08-11 15:24:35.801867 2014-08-13 18:06:06.956623 7f5c973a57a0 20 read_log 192834'11566 (192834'11565) modify 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.933448.0:13193 2014-08-11 15:24:35.794114 2014-08-13 18:06:06.956635 7f5c973a57a0 20 read_log 192834'11567 (0'0) promote 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by osd.346.0:13175 2014-08-11 15:24:35.830566 2014-08-13 18:06:06.956647 7f5c973a57a0 20 read_log 192834'11568 (192834'11567) modify 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.933448.0:13201 2014-08-11 15:24:35.823288 2014-08-13 18:06:06.956659 7f5c973a57a0 20 read_log 192834'11575 (0'0) promote 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by osd.346.0:13183 2014-08-11 15:24:39.708983 2014-08-13 18:06:06.956671 7f5c973a57a0 20 read_log 192834'11576 (192834'11575) modify 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.933448.0:13719 2014-08-11 15:24:39.702123 2014-08-13 18:06:06.956712 7f5c973a57a0 20 read_log 192834'11581 (0'0) promote 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by osd.346.0:13189 2014-08-11 15:24:43.614896 2014-08-13 18:06:06.956728 7f5c973a57a0 20 read_log 192834'11582 (192834'11581) modify 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.933448.0:14284 2014-08-11 15:24:43.605540 2014-08-13 18:06:06.956740 7f5c973a57a0 20 read_log 192834'11595 (0'0) promote 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by osd.346.0:13203 2014-08-11 15:24:49.189960 2014-08-13 18:06:06.956752 7f5c973a57a0 20 read_log 192834'11596 (192834'11595) modify 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.933448.0:15313 2014-08-11 15:24:49.181594 2014-08-13 18:06:06.956765 7f5c973a57a0 20 read_log 192834'11599 (0'0) promote 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by osd.346.0:13207 2014-08-11 15:24:49.815373 2014-08-13 18:06:06.956777 7f5c973a57a0 20 read_log 192834'11600 (192834'11599) modify 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.933448.0:15481 2014-08-11 15:24:49.807741 2014-08-13 18:06:06.956791 7f5c973a57a0 20 read_log 192834'11603 (0'0) promote dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by osd.346.0:13211 2014-08-11 15:24:50.291943 2014-08-13 18:06:06.956805 7f5c973a57a0 20 read_log 192834'11604 (192834'11603) modify dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.933448.0:15594 2014-08-11 15:24:50.283454 2014-08-13 18:06:06.956818 7f5c973a57a0 20 read_log 192834'11647 (0'0) promote c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by osd.346.0:13255 2014-08-11 15:25:01.424029 2014-08-13 18:06:06.956830 7f5c973a57a0 20 read_log 192834'11648 (192834'11647) modify c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.933448.0:17761 2014-08-11 15:25:01.414991 2014-08-13 18:06:06.956842 7f5c973a57a0 20 read_log 192834'11659 (0'0) promote b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by osd.346.0:13267 2014-08-11 15:25:06.248281 2014-08-13 18:06:06.956854 7f5c973a57a0 20 read_log 192834'11660 (192834'11659) modify b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.933448.0:18664 2014-08-11 15:25:06.239065 2014-08-13 18:06:06.956869 7f5c973a57a0 20 read_log 192834'11699 (0'0) promote b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by osd.346.0:13307 2014-08-11 15:25:20.645598 2014-08-13 18:06:06.956881 7f5c973a57a0 20 read_log 192834'11700 (192834'11699) modify b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.933448.0:20367 2014-08-11 15:25:20.636700 2014-08-13 18:06:06.956896 7f5c973a57a0 20 read_log 192834'11703 (0'0) promote 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by osd.346.0:13311 2014-08-11 15:25:20.841904 2014-08-13 18:06:06.956908 7f5c973a57a0 20 read_log 192834'11704 (192834'11703) modify 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.933448.0:20417 2014-08-11 15:25:20.833713 2014-08-13 18:06:06.956925 7f5c973a57a0 20 read_log 192834'11717 (0'0) promote 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by osd.346.0:13325 2014-08-11 15:25:34.846995 2014-08-13 18:06:06.956938 7f5c973a57a0 20 read_log 192834'11718 (192834'11717) modify 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.933448.0:22168 2014-08-11 15:25:34.837381 2014-08-13 18:06:06.956952 7f5c973a57a0 20 read_log 192834'11731 (192834'11376) modify 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.933448.0:23591 2014-08-11 15:25:39.529856 2014-08-13 18:06:06.956965 7f5c973a57a0 20 read_log 192834'11756 (192834'11426) modify df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.933448.0:26045 2014-08-11 15:25:42.876839 2014-08-13 18:06:06.956986 7f5c973a57a0 20 read_log 192834'11757 (192834'11428) modify 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.933448.0:26389 2014-08-11 15:25:43.340436 2014-08-13 18:06:06.957001 7f5c973a57a0 20 read_log 192834'11770 (192834'11454) modify 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.933448.0:28291 2014-08-11 15:25:45.876388 2014-08-13 18:06:06.957013 7f5c973a57a0 20 read_log 192834'11791 (192834'11496) modify 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.933448.0:31300 2014-08-11 15:25:49.887739 2014-08-13 18:06:06.957027 7f5c973a57a0 20 read_log 192834'11796 (192834'11506) modify 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.933448.0:31937 2014-08-11 15:25:50.834765 2014-08-13 18:06:06.957040 7f5c973a57a0 20 read_log 192834'11806 (192834'11526) modify 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.933448.0:33154 2014-08-11 15:25:52.919630 2014-08-13 18:06:06.957054 7f5c973a57a0 20 read_log 192834'11807 (192834'11528) modify d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.933448.0:33171 2014-08-11 15:25:52.937715 2014-08-13 18:06:06.957066 7f5c973a57a0 20 read_log 192834'11808 (192834'11530) modify 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.933448.0:33206 2014-08-11 15:25:53.000575 2014-08-13 18:06:06.957080 7f5c973a57a0 20 read_log 192834'11810 (192834'11534) modify 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.933448.0:33468 2014-08-11 15:25:53.539928 2014-08-13 18:06:06.957092 7f5c973a57a0 20 read_log 192834'11815 (192834'11544) modify b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.933448.0:34148 2014-08-11 15:25:54.638271 2014-08-13 18:06:06.957104 7f5c973a57a0 20 read_log 192834'11826 (192834'11566) modify 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.933448.0:35484 2014-08-11 15:25:56.364536 2014-08-13 18:06:06.957118 7f5c973a57a0 20 read_log 192834'11827 (192834'11568) modify 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.933448.0:35492 2014-08-11 15:25:56.376410 2014-08-13 18:06:06.957153 7f5c973a57a0 20 read_log 192834'11831 (192834'11576) modify 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.933448.0:36010 2014-08-11 15:25:57.129164 2014-08-13 18:06:06.957169 7f5c973a57a0 20 read_log 192834'11834 (192834'11582) modify 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.933448.0:36575 2014-08-11 15:25:57.993090 2014-08-13 18:06:06.957181 7f5c973a57a0 20 read_log 192834'11841 (192834'11596) modify 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.933448.0:37604 2014-08-11 15:25:59.740883 2014-08-13 18:06:06.957195 7f5c973a57a0 20 read_log 192834'11843 (192834'11600) modify 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.933448.0:37772 2014-08-11 15:26:00.008805 2014-08-13 18:06:06.957210 7f5c973a57a0 20 read_log 192834'11845 (192834'11604) modify dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.933448.0:37885 2014-08-11 15:26:00.238894 2014-08-13 18:06:06.957222 7f5c973a57a0 20 read_log 192834'11867 (192834'11648) modify c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.933448.0:40052 2014-08-11 15:26:02.246842 2014-08-13 18:06:06.957235 7f5c973a57a0 20 read_log 192834'11873 (192834'11660) modify b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.933448.0:40955 2014-08-11 15:26:02.797810 2014-08-13 18:06:06.957249 7f5c973a57a0 20 read_log 192834'11893 (192834'11700) modify b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.933448.0:42658 2014-08-11 15:26:03.345963 2014-08-13 18:06:06.957261 7f5c973a57a0 20 read_log 192834'11895 (192834'11704) modify 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.933448.0:42708 2014-08-11 15:26:03.363717 2014-08-13 18:06:06.957290 7f5c973a57a0 20 read_log 192834'11902 (192834'11718) modify 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.933448.0:44459 2014-08-11 15:26:03.947531 2014-08-13 18:06:06.957305 7f5c973a57a0 20 read_log 192834'11909 (0'0) promote dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by osd.346.0:13517 2014-08-11 15:26:17.744773 2014-08-13 18:06:06.957318 7f5c973a57a0 20 read_log 192834'11910 (192834'11909) modify dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.1883634.0:893 2014-08-11 15:26:17.733910 2014-08-13 18:06:06.957330 7f5c973a57a0 20 read_log 192834'11915 (0'0) promote 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by osd.346.0:13523 2014-08-11 15:26:18.977754 2014-08-13 18:06:06.957343 7f5c973a57a0 20 read_log 192834'11916 (192834'11915) modify 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.1883634.0:1235 2014-08-11 15:26:18.970204 2014-08-13 18:06:06.957355 7f5c973a57a0 20 read_log 192834'11921 (0'0) promote fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by osd.346.0:13529 2014-08-11 15:26:19.771328 2014-08-13 18:06:06.957367 7f5c973a57a0 20 read_log 192834'11922 (192834'11921) modify fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.1883634.0:1455 2014-08-11 15:26:19.763655 2014-08-13 18:06:06.957379 7f5c973a57a0 20 read_log 192834'11923 (0'0) promote 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by osd.346.0:13531 2014-08-11 15:26:20.401680 2014-08-13 18:06:06.957391 7f5c973a57a0 20 read_log 192834'11924 (192834'11923) modify 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.1883634.0:1629 2014-08-11 15:26:20.393853 2014-08-13 18:06:06.957413 7f5c973a57a0 20 read_log 192834'11929 (0'0) promote 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by osd.346.0:13537 2014-08-11 15:26:22.737201 2014-08-13 18:06:06.957433 7f5c973a57a0 20 read_log 192834'11930 (192834'11929) modify 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.1883634.0:2235 2014-08-11 15:26:22.729464 2014-08-13 18:06:06.957449 7f5c973a57a0 20 read_log 192834'11935 (0'0) promote 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by osd.346.0:13543 2014-08-11 15:26:23.530659 2014-08-13 18:06:06.957470 7f5c973a57a0 20 read_log 192834'11936 (192834'11935) modify 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.1883634.0:2454 2014-08-11 15:26:23.522974 2014-08-13 18:06:06.957485 7f5c973a57a0 20 read_log 192834'11945 (0'0) promote 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by osd.346.0:13553 2014-08-11 15:26:25.618147 2014-08-13 18:06:06.957498 7f5c973a57a0 20 read_log 192834'11946 (192834'11945) modify 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.1883634.0:3043 2014-08-11 15:26:25.606464 2014-08-13 18:06:06.957518 7f5c973a57a0 20 read_log 192834'11955 (0'0) promote 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by osd.346.0:13563 2014-08-11 15:26:29.821144 2014-08-13 18:06:06.957534 7f5c973a57a0 20 read_log 192834'11956 (192834'11955) modify 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.1883634.0:4232 2014-08-11 15:26:29.812822 2014-08-13 18:06:06.957546 7f5c973a57a0 20 read_log 192834'11965 (0'0) promote 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by osd.346.0:13573 2014-08-11 15:26:33.056567 2014-08-13 18:06:06.957568 7f5c973a57a0 20 read_log 192834'11966 (192834'11965) modify 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.1883634.0:5054 2014-08-11 15:26:33.048315 2014-08-13 18:06:06.957584 7f5c973a57a0 20 read_log 192834'11971 (0'0) promote 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by osd.346.0:13579 2014-08-11 15:26:34.290114 2014-08-13 18:06:06.957610 7f5c973a57a0 20 read_log 192834'11972 (192834'11971) modify 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.1883634.0:5405 2014-08-11 15:26:34.279282 2014-08-13 18:06:06.957625 7f5c973a57a0 20 read_log 192834'11973 (0'0) promote 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by osd.346.0:13581 2014-08-11 15:26:35.263766 2014-08-13 18:06:06.957638 7f5c973a57a0 20 read_log 192834'11974 (192834'11973) modify 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.1883634.0:5679 2014-08-11 15:26:35.255206 2014-08-13 18:06:06.957650 7f5c973a57a0 20 read_log 192834'11985 (0'0) promote f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by osd.346.0:13593 2014-08-11 15:26:37.452258 2014-08-13 18:06:06.957662 7f5c973a57a0 20 read_log 192834'11986 (192834'11985) modify f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.1883634.0:6279 2014-08-11 15:26:37.442771 2014-08-13 18:06:06.957675 7f5c973a57a0 20 read_log 192834'11987 (0'0) promote a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by osd.346.0:13595 2014-08-11 15:26:40.775524 2014-08-13 18:06:06.957687 7f5c973a57a0 20 read_log 192834'11988 (192834'11987) modify a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.1883634.0:7089 2014-08-11 15:26:40.765805 2014-08-13 18:06:06.957699 7f5c973a57a0 20 read_log 192834'12005 (0'0) promote 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by osd.346.0:13613 2014-08-11 15:26:46.521197 2014-08-13 18:06:06.957711 7f5c973a57a0 20 read_log 192834'12006 (192834'12005) modify 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.1883634.0:8478 2014-08-11 15:26:46.512465 2014-08-13 18:06:06.957723 7f5c973a57a0 20 read_log 192834'12019 (0'0) promote f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by osd.346.0:13627 2014-08-11 15:26:50.866751 2014-08-13 18:06:06.957735 7f5c973a57a0 20 read_log 192834'12020 (192834'12019) modify f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.1883634.0:9438 2014-08-11 15:26:50.857121 2014-08-13 18:06:06.957748 7f5c973a57a0 20 read_log 192834'12023 (0'0) promote 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by osd.346.0:13631 2014-08-11 15:26:51.961872 2014-08-13 18:06:06.957759 7f5c973a57a0 20 read_log 192834'12024 (192834'12023) modify 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.1883634.0:9694 2014-08-11 15:26:51.953248 2014-08-13 18:06:06.957773 7f5c973a57a0 20 read_log 192834'12035 (0'0) promote 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by osd.346.0:13643 2014-08-11 15:26:55.599977 2014-08-13 18:06:06.957785 7f5c973a57a0 20 read_log 192834'12036 (192834'12035) modify 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.1883634.0:10552 2014-08-11 15:26:55.591405 2014-08-13 18:06:06.957797 7f5c973a57a0 20 read_log 192834'12037 (0'0) promote 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by osd.346.0:13645 2014-08-11 15:26:55.649085 2014-08-13 18:06:06.957810 7f5c973a57a0 20 read_log 192834'12038 (192834'12037) modify 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.1883634.0:10563 2014-08-11 15:26:55.641203 2014-08-13 18:06:06.957822 7f5c973a57a0 20 read_log 192834'12049 (0'0) promote 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by osd.346.0:13657 2014-08-11 15:27:00.230796 2014-08-13 18:06:06.957834 7f5c973a57a0 20 read_log 192834'12050 (192834'12049) modify 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.1883634.0:11527 2014-08-11 15:27:00.223244 2014-08-13 18:06:06.957846 7f5c973a57a0 20 read_log 192834'12063 (0'0) promote 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by osd.346.0:13671 2014-08-11 15:27:02.456859 2014-08-13 18:06:06.957866 7f5c973a57a0 20 read_log 192834'12064 (192834'12063) modify 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.1883634.0:12100 2014-08-11 15:27:02.449026 2014-08-13 18:06:06.957878 7f5c973a57a0 20 read_log 192834'12111 (0'0) promote 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by osd.346.0:13719 2014-08-11 15:27:16.840562 2014-08-13 18:06:06.957891 7f5c973a57a0 20 read_log 192834'12112 (192834'12111) modify 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.1883634.0:15051 2014-08-11 15:27:16.831873 2014-08-13 18:06:06.957904 7f5c973a57a0 20 read_log 192834'12147 (0'0) promote fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by osd.346.0:13755 2014-08-11 15:27:42.564008 2014-08-13 18:06:06.957916 7f5c973a57a0 20 read_log 192834'12148 (192834'12147) modify fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.1883634.0:18250 2014-08-11 15:27:42.553386 2014-08-13 18:06:06.957942 7f5c973a57a0 20 read_log 192834'12157 (0'0) promote 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by osd.346.0:13765 2014-08-11 15:27:51.504538 2014-08-13 18:06:06.957954 7f5c973a57a0 20 read_log 192834'12158 (192834'12157) modify 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.1883634.0:19355 2014-08-11 15:27:51.494524 2014-08-13 18:06:06.957967 7f5c973a57a0 20 read_log 192834'12175 (0'0) promote bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by osd.346.0:13783 2014-08-11 15:28:01.363956 2014-08-13 18:06:06.957979 7f5c973a57a0 20 read_log 192834'12176 (192834'12175) modify bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.1883634.0:20748 2014-08-11 15:28:01.354775 2014-08-13 18:06:06.957991 7f5c973a57a0 20 read_log 192834'12179 (0'0) promote af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by osd.346.0:13787 2014-08-11 15:28:01.680887 2014-08-13 18:06:06.958003 7f5c973a57a0 20 read_log 192834'12180 (192834'12179) modify af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.1883634.0:20823 2014-08-11 15:28:01.671719 2014-08-13 18:06:06.958015 7f5c973a57a0 20 read_log 192834'12205 (0'0) promote f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by osd.346.0:13813 2014-08-11 15:28:13.726265 2014-08-13 18:06:06.958027 7f5c973a57a0 20 read_log 192834'12206 (192834'12205) modify f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.1883634.0:23263 2014-08-11 15:28:13.717103 2014-08-13 18:06:06.958042 7f5c973a57a0 20 read_log 192834'12207 (0'0) promote 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by osd.346.0:13815 2014-08-11 15:28:13.765260 2014-08-13 18:06:06.958055 7f5c973a57a0 20 read_log 192834'12208 (192834'12207) modify 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.1883634.0:23270 2014-08-11 15:28:13.757325 2014-08-13 18:06:06.958067 7f5c973a57a0 20 read_log 192834'12214 (192834'11910) modify dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.1883634.0:24217 2014-08-11 15:28:18.554572 2014-08-13 18:06:06.958079 7f5c973a57a0 20 read_log 192834'12217 (192834'11916) modify 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.1883634.0:24559 2014-08-11 15:28:19.228978 2014-08-13 18:06:06.958091 7f5c973a57a0 20 read_log 192834'12220 (192834'11922) modify fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.1883634.0:24779 2014-08-11 15:28:19.509252 2014-08-13 18:06:06.958104 7f5c973a57a0 20 read_log 192834'12221 (192834'11924) modify 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.1883634.0:24953 2014-08-11 15:28:19.812831 2014-08-13 18:06:06.958116 7f5c973a57a0 20 read_log 192834'12224 (192834'11930) modify 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.1883634.0:25559 2014-08-11 15:28:21.541239 2014-08-13 18:06:06.958138 7f5c973a57a0 20 read_log 192834'12227 (192834'11936) modify 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.1883634.0:25778 2014-08-11 15:28:21.852665 2014-08-13 18:06:06.958163 7f5c973a57a0 20 read_log 192834'12232 (192834'11946) modify 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.1883634.0:26367 2014-08-11 15:28:23.036551 2014-08-13 18:06:06.958184 7f5c973a57a0 20 read_log 192834'12237 (192834'11956) modify 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.1883634.0:27556 2014-08-11 15:28:24.610524 2014-08-13 18:06:06.958198 7f5c973a57a0 20 read_log 192834'12242 (192834'11966) modify 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.1883634.0:28378 2014-08-11 15:28:25.738413 2014-08-13 18:06:06.958217 7f5c973a57a0 20 read_log 192834'12245 (192834'11972) modify 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.1883634.0:28729 2014-08-11 15:28:26.196393 2014-08-13 18:06:06.958237 7f5c973a57a0 20 read_log 192834'12246 (192834'11974) modify 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.1883634.0:29003 2014-08-11 15:28:26.604851 2014-08-13 18:06:06.958250 7f5c973a57a0 20 read_log 192834'12252 (192834'11986) modify f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.1883634.0:29603 2014-08-11 15:28:27.487015 2014-08-13 18:06:06.958262 7f5c973a57a0 20 read_log 192834'12253 (192834'11988) modify a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.1883634.0:30413 2014-08-11 15:28:28.609156 2014-08-13 18:06:06.958281 7f5c973a57a0 20 read_log 192834'12262 (192834'12006) modify 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.1883634.0:31802 2014-08-11 15:28:30.985331 2014-08-13 18:06:06.958298 7f5c973a57a0 20 read_log 192834'12269 (192834'12020) modify f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.1883634.0:32762 2014-08-11 15:28:32.279375 2014-08-13 18:06:06.958310 7f5c973a57a0 20 read_log 192834'12271 (192834'12024) modify 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.1883634.0:33018 2014-08-11 15:28:32.672533 2014-08-13 18:06:06.958323 7f5c973a57a0 20 read_log 192834'12277 (192834'12036) modify 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.1883634.0:33876 2014-08-11 15:28:33.932542 2014-08-13 18:06:06.958335 7f5c973a57a0 20 read_log 192834'12278 (192834'12038) modify 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.1883634.0:33887 2014-08-11 15:28:33.950175 2014-08-13 18:06:06.958347 7f5c973a57a0 20 read_log 192834'12284 (192834'12050) modify 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.1883634.0:34851 2014-08-11 15:28:35.401774 2014-08-13 18:06:06.958360 7f5c973a57a0 20 read_log 192834'12293 (192834'12064) modify 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.1883634.0:35424 2014-08-11 15:28:36.613371 2014-08-13 18:06:06.958372 7f5c973a57a0 20 read_log 192834'12317 (192834'12112) modify 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.1883634.0:38375 2014-08-11 15:28:40.557369 2014-08-13 18:06:06.958384 7f5c973a57a0 20 read_log 192834'12335 (192834'12148) modify fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.1883634.0:41574 2014-08-11 15:28:45.346539 2014-08-13 18:06:06.958396 7f5c973a57a0 20 read_log 192834'12340 (192834'12158) modify 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.1883634.0:42679 2014-08-11 15:28:46.320064 2014-08-13 18:06:06.958415 7f5c973a57a0 20 read_log 192834'12349 (192834'12176) modify bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.1883634.0:44072 2014-08-11 15:28:46.751928 2014-08-13 18:06:06.958432 7f5c973a57a0 20 read_log 192834'12351 (192834'12180) modify af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.1883634.0:44147 2014-08-11 15:28:46.774794 2014-08-13 18:06:06.958452 7f5c973a57a0 20 read_log 192834'12364 (192834'12206) modify f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.1883634.0:46587 2014-08-11 15:28:47.639470 2014-08-13 18:06:06.958467 7f5c973a57a0 20 read_log 192834'12365 (192834'12208) modify 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.1883634.0:46594 2014-08-11 15:28:47.642020 2014-08-13 18:06:06.958480 7f5c973a57a0 20 read_log 192888'12368 (0'0) promote 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by osd.53.0:1865 2014-08-11 16:44:55.383268 2014-08-13 18:06:06.958492 7f5c973a57a0 20 read_log 192888'12369 (192888'12368) modify 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.1315259.0:111 2014-08-11 16:44:55.374000 2014-08-13 18:06:06.958523 7f5c973a57a0 20 read_log 192888'12370 (0'0) promote a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by osd.53.0:1867 2014-08-11 16:44:55.400506 2014-08-13 18:06:06.958538 7f5c973a57a0 20 read_log 192888'12371 (192888'12370) modify a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.1315259.0:116 2014-08-11 16:44:55.393410 2014-08-13 18:06:06.958551 7f5c973a57a0 20 read_log 192888'12372 (0'0) promote d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by osd.53.0:1878 2014-08-11 16:44:57.833198 2014-08-13 18:06:06.958563 7f5c973a57a0 20 read_log 192888'12373 (192888'12372) modify d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.1315259.0:671 2014-08-11 16:44:57.824793 2014-08-13 18:06:06.958575 7f5c973a57a0 20 read_log 192888'12374 (0'0) promote 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by osd.53.0:1884 2014-08-11 16:44:59.015272 2014-08-13 18:06:06.958587 7f5c973a57a0 20 read_log 192888'12375 (192888'12374) modify 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.1315259.0:987 2014-08-11 16:44:59.007888 2014-08-13 18:06:06.958600 7f5c973a57a0 20 read_log 192888'12378 (0'0) promote f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by osd.53.0:1888 2014-08-11 16:44:59.342570 2014-08-13 18:06:06.958612 7f5c973a57a0 20 read_log 192888'12379 (192888'12378) modify f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.1315259.0:1075 2014-08-11 16:44:59.332433 2014-08-13 18:06:06.958624 7f5c973a57a0 20 read_log 192888'12386 (0'0) promote 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by osd.53.0:1915 2014-08-11 16:45:05.160933 2014-08-13 18:06:06.958636 7f5c973a57a0 20 read_log 192888'12387 (192888'12386) modify 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.1315259.0:2618 2014-08-11 16:45:05.153710 2014-08-13 18:06:06.958648 7f5c973a57a0 20 read_log 192888'12388 (0'0) promote d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by osd.53.0:1931 2014-08-11 16:45:10.552799 2014-08-13 18:06:06.958663 7f5c973a57a0 20 read_log 192888'12389 (192888'12388) modify d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.1315259.0:4024 2014-08-11 16:45:10.544237 2014-08-13 18:06:06.958675 7f5c973a57a0 20 read_log 192888'12392 (0'0) promote 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by osd.53.0:1949 2014-08-11 16:45:16.717728 2014-08-13 18:06:06.958690 7f5c973a57a0 20 read_log 192888'12393 (192888'12392) modify 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.1315259.0:5705 2014-08-11 16:45:16.710711 2014-08-13 18:06:06.958702 7f5c973a57a0 20 read_log 192888'12400 (0'0) promote c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by osd.53.0:1993 2014-08-11 16:45:25.889662 2014-08-13 18:06:06.958715 7f5c973a57a0 20 read_log 192888'12401 (192888'12400) modify c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.1315259.0:8235 2014-08-11 16:45:25.883037 2014-08-13 18:06:06.958756 7f5c973a57a0 20 read_log 192888'12402 (0'0) promote 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by osd.53.0:1999 2014-08-11 16:45:26.936857 2014-08-13 18:06:06.958772 7f5c973a57a0 20 read_log 192888'12403 (192888'12402) modify 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.1315259.0:8498 2014-08-11 16:45:26.928186 2014-08-13 18:06:06.958784 7f5c973a57a0 20 read_log 192888'12410 (0'0) promote 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by osd.53.0:2037 2014-08-11 16:45:34.052460 2014-08-13 18:06:06.958799 7f5c973a57a0 20 read_log 192888'12411 (192888'12410) modify 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.1315259.0:10365 2014-08-11 16:45:34.043356 2014-08-13 18:06:06.958816 7f5c973a57a0 20 read_log 192888'12416 (0'0) promote 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by osd.53.0:2059 2014-08-11 16:45:39.223933 2014-08-13 18:06:06.958828 7f5c973a57a0 20 read_log 192888'12417 (192888'12416) modify 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.1315259.0:11661 2014-08-11 16:45:39.217190 2014-08-13 18:06:06.958841 7f5c973a57a0 20 read_log 192888'12418 (0'0) promote eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by osd.53.0:2071 2014-08-11 16:45:41.471408 2014-08-13 18:06:06.958853 7f5c973a57a0 20 read_log 192888'12419 (192888'12418) modify eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.1315259.0:12201 2014-08-11 16:45:41.463546 2014-08-13 18:06:06.958867 7f5c973a57a0 20 read_log 192888'12420 (0'0) promote 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by osd.53.0:2075 2014-08-11 16:45:42.788268 2014-08-13 18:06:06.958879 7f5c973a57a0 20 read_log 192888'12421 (192888'12420) modify 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.1315259.0:12505 2014-08-11 16:45:42.781002 2014-08-13 18:06:06.958891 7f5c973a57a0 20 read_log 192888'12442 (0'0) promote 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by osd.53.0:2143 2014-08-11 16:45:58.763281 2014-08-13 18:06:06.958903 7f5c973a57a0 20 read_log 192888'12443 (192888'12442) modify 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.1315259.0:16592 2014-08-11 16:45:58.755273 2014-08-13 18:06:06.958916 7f5c973a57a0 20 read_log 192888'12444 (0'0) promote 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by osd.53.0:2147 2014-08-11 16:45:59.915992 2014-08-13 18:06:06.958930 7f5c973a57a0 20 read_log 192888'12445 (192888'12444) modify 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.1315259.0:16898 2014-08-11 16:45:59.908643 2014-08-13 18:06:06.958942 7f5c973a57a0 20 read_log 192888'12446 (0'0) promote f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by osd.53.0:2157 2014-08-11 16:46:01.580487 2014-08-13 18:06:06.958957 7f5c973a57a0 20 read_log 192888'12447 (192888'12446) modify f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.1315259.0:17320 2014-08-11 16:46:01.571507 2014-08-13 18:06:06.958969 7f5c973a57a0 20 read_log 192888'12448 (0'0) promote dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by osd.53.0:2161 2014-08-11 16:46:03.063227 2014-08-13 18:06:06.958983 7f5c973a57a0 20 read_log 192888'12449 (192888'12448) modify dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.1315259.0:17671 2014-08-11 16:46:03.055781 2014-08-13 18:06:06.958996 7f5c973a57a0 20 read_log 192888'12452 (0'0) promote e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by osd.53.0:2171 2014-08-11 16:46:03.993420 2014-08-13 18:06:06.959010 7f5c973a57a0 20 read_log 192888'12453 (192888'12452) modify e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.1315259.0:17891 2014-08-11 16:46:03.957026 2014-08-13 18:06:06.959033 7f5c973a57a0 20 read_log 192888'12454 (0'0) promote 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by osd.53.0:2173 2014-08-11 16:46:04.877779 2014-08-13 18:06:06.959048 7f5c973a57a0 20 read_log 192888'12455 (192888'12454) modify 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.1315259.0:18122 2014-08-11 16:46:04.870614 2014-08-13 18:06:06.959060 7f5c973a57a0 20 read_log 192888'12456 (0'0) promote 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by osd.53.0:2175 2014-08-11 16:46:04.903738 2014-08-13 18:06:06.959075 7f5c973a57a0 20 read_log 192888'12457 (192888'12456) modify 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.1315259.0:18129 2014-08-11 16:46:04.895979 2014-08-13 18:06:06.959088 7f5c973a57a0 20 read_log 192888'12460 (0'0) promote e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by osd.53.0:2211 2014-08-11 16:46:12.524070 2014-08-13 18:06:06.959100 7f5c973a57a0 20 read_log 192888'12461 (192888'12460) modify e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.1315259.0:20109 2014-08-11 16:46:12.512960 2014-08-13 18:06:06.959113 7f5c973a57a0 20 read_log 192888'12462 (0'0) promote 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by osd.53.0:2215 2014-08-11 16:46:13.572111 2014-08-13 18:06:06.959127 7f5c973a57a0 20 read_log 192888'12463 (192888'12462) modify 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.1315259.0:20395 2014-08-11 16:46:13.563930 2014-08-13 18:06:06.959140 7f5c973a57a0 20 read_log 192888'12466 (0'0) promote 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by osd.53.0:2225 2014-08-11 16:46:14.593998 2014-08-13 18:06:06.959166 7f5c973a57a0 20 read_log 192888'12467 (192888'12466) modify 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.1315259.0:20650 2014-08-11 16:46:14.585665 2014-08-13 18:06:06.959179 7f5c973a57a0 20 read_log 192888'12470 (0'0) promote d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by osd.53.0:2237 2014-08-11 16:46:17.978491 2014-08-13 18:06:06.959191 7f5c973a57a0 20 read_log 192888'12471 (192888'12470) modify d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.1315259.0:21548 2014-08-11 16:46:17.970388 2014-08-13 18:06:06.959203 7f5c973a57a0 20 read_log 192888'12474 (0'0) promote c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by osd.53.0:2253 2014-08-11 16:46:22.788946 2014-08-13 18:06:06.959218 7f5c973a57a0 20 read_log 192888'12475 (192888'12474) modify c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.1315259.0:22798 2014-08-11 16:46:22.781797 2014-08-13 18:06:06.959230 7f5c973a57a0 20 read_log 192888'12478 (0'0) promote b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by osd.53.0:2263 2014-08-11 16:46:25.000529 2014-08-13 18:06:06.959247 7f5c973a57a0 20 read_log 192888'12479 (192888'12478) modify b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.1315259.0:23342 2014-08-11 16:46:24.991539 2014-08-13 18:06:06.959262 7f5c973a57a0 20 read_log 192888'12482 (0'0) promote 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by osd.53.0:2279 2014-08-11 16:46:28.177916 2014-08-13 18:06:06.959274 7f5c973a57a0 20 read_log 192888'12483 (192888'12482) modify 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.1315259.0:24142 2014-08-11 16:46:28.170330 2014-08-13 18:06:06.959289 7f5c973a57a0 20 read_log 192888'12486 (0'0) promote 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by osd.53.0:2287 2014-08-11 16:46:30.158464 2014-08-13 18:06:06.959307 7f5c973a57a0 20 read_log 192888'12487 (192888'12486) modify 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.1315259.0:24660 2014-08-11 16:46:30.149948 2014-08-13 18:06:06.959327 7f5c973a57a0 20 read_log 192888'12494 (0'0) promote 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by osd.53.0:2323 2014-08-11 16:46:43.036669 2014-08-13 18:06:06.959342 7f5c973a57a0 20 read_log 192888'12495 (192888'12494) modify 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.1315259.0:27762 2014-08-11 16:46:43.027373 2014-08-13 18:06:06.959387 7f5c973a57a0 20 read_log 192888'12498 (0'0) promote 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by osd.53.0:2333 2014-08-11 16:46:46.836034 2014-08-13 18:06:06.959406 7f5c973a57a0 20 read_log 192888'12499 (192888'12498) modify 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.1315259.0:28689 2014-08-11 16:46:46.828603 2014-08-13 18:06:06.959427 7f5c973a57a0 20 read_log 192888'12502 (0'0) promote 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by osd.53.0:2359 2014-08-11 16:46:54.119376 2014-08-13 18:06:06.959442 7f5c973a57a0 20 read_log 192888'12503 (192888'12502) modify 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.1315259.0:30534 2014-08-11 16:46:54.111958 2014-08-13 18:06:06.959455 7f5c973a57a0 20 read_log 192888'12504 (192888'12369) modify 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.1315259.0:30838 2014-08-11 16:46:55.906571 2014-08-13 18:06:06.959471 7f5c973a57a0 20 read_log 192888'12505 (192888'12371) modify a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.1315259.0:30843 2014-08-11 16:46:55.908805 2014-08-13 18:06:06.959493 7f5c973a57a0 20 read_log 192888'12506 (192888'12373) modify d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.1315259.0:31398 2014-08-11 16:46:56.081015 2014-08-13 18:06:06.959506 7f5c973a57a0 20 read_log 192888'12507 (192888'12375) modify 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.1315259.0:31714 2014-08-11 16:46:56.178587 2014-08-13 18:06:06.959518 7f5c973a57a0 20 read_log 192888'12509 (192888'12379) modify f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.1315259.0:31802 2014-08-11 16:46:56.204657 2014-08-13 18:06:06.959541 7f5c973a57a0 20 read_log 192888'12513 (192888'12387) modify 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.1315259.0:33345 2014-08-11 16:46:56.613530 2014-08-13 18:06:06.959553 7f5c973a57a0 20 read_log 192888'12514 (192888'12389) modify d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.1315259.0:34751 2014-08-11 16:46:56.976364 2014-08-13 18:06:06.959566 7f5c973a57a0 20 read_log 192888'12516 (192888'12393) modify 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.1315259.0:36432 2014-08-11 16:46:57.420603 2014-08-13 18:06:06.959578 7f5c973a57a0 20 read_log 192888'12520 (192888'12401) modify c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.1315259.0:38962 2014-08-11 16:46:58.091769 2014-08-13 18:06:06.959590 7f5c973a57a0 20 read_log 192888'12521 (192888'12403) modify 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.1315259.0:39225 2014-08-11 16:46:58.156709 2014-08-13 18:06:06.959603 7f5c973a57a0 20 read_log 192888'12525 (192888'12411) modify 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.1315259.0:41092 2014-08-11 16:46:58.656324 2014-08-13 18:06:06.959615 7f5c973a57a0 20 read_log 192888'12528 (192888'12417) modify 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.1315259.0:42388 2014-08-11 16:46:58.997673 2014-08-13 18:06:06.959627 7f5c973a57a0 20 read_log 192888'12529 (192888'12419) modify eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.1315259.0:42928 2014-08-11 16:46:59.143980 2014-08-13 18:06:06.959639 7f5c973a57a0 20 read_log 192888'12530 (192888'12421) modify 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.1315259.0:43232 2014-08-11 16:46:59.229077 2014-08-13 18:06:06.959659 7f5c973a57a0 20 read_log 192888'12541 (192888'12443) modify 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.1315259.0:47319 2014-08-11 16:47:00.418691 2014-08-13 18:06:06.959674 7f5c973a57a0 20 read_log 192888'12542 (192888'12445) modify 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.1315259.0:47625 2014-08-11 16:47:00.516835 2014-08-13 18:06:06.959687 7f5c973a57a0 20 read_log 192888'12543 (192888'12447) modify f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.1315259.0:48047 2014-08-11 16:47:00.653680 2014-08-13 18:06:06.959699 7f5c973a57a0 20 read_log 192888'12544 (192888'12449) modify dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.1315259.0:48398 2014-08-11 16:47:00.753281 2014-08-13 18:06:06.959711 7f5c973a57a0 20 read_log 192888'12546 (192888'12453) modify e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.1315259.0:48618 2014-08-11 16:47:00.816175 2014-08-13 18:06:06.959724 7f5c973a57a0 20 read_log 192888'12547 (192888'12455) modify 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.1315259.0:48849 2014-08-11 16:47:00.885176 2014-08-13 18:06:06.959737 7f5c973a57a0 20 read_log 192888'12548 (192888'12457) modify 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.1315259.0:48856 2014-08-11 16:47:00.886137 2014-08-13 18:06:06.959750 7f5c973a57a0 20 read_log 192888'12550 (192888'12461) modify e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.1315259.0:50836 2014-08-11 16:47:01.493274 2014-08-13 18:06:06.959762 7f5c973a57a0 20 read_log 192888'12551 (192888'12463) modify 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.1315259.0:51122 2014-08-11 16:47:01.586388 2014-08-13 18:06:06.959775 7f5c973a57a0 20 read_log 192888'12553 (192888'12467) modify 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.1315259.0:51377 2014-08-11 16:47:01.662153 2014-08-13 18:06:06.959787 7f5c973a57a0 20 read_log 192888'12555 (192888'12471) modify d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.1315259.0:52275 2014-08-11 16:47:01.927022 2014-08-13 18:06:06.959799 7f5c973a57a0 20 read_log 192888'12557 (192888'12475) modify c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.1315259.0:53525 2014-08-11 16:47:02.277973 2014-08-13 18:06:06.959811 7f5c973a57a0 20 read_log 192888'12559 (192888'12479) modify b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.1315259.0:54069 2014-08-11 16:47:02.452238 2014-08-13 18:06:06.959823 7f5c973a57a0 20 read_log 192888'12561 (192888'12483) modify 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.1315259.0:54869 2014-08-11 16:47:02.741844 2014-08-13 18:06:06.959836 7f5c973a57a0 20 read_log 192888'12563 (192888'12487) modify 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.1315259.0:55387 2014-08-11 16:47:02.897703 2014-08-13 18:06:06.959848 7f5c973a57a0 20 read_log 192888'12567 (192888'12495) modify 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.1315259.0:58489 2014-08-11 16:47:03.817197 2014-08-13 18:06:06.959860 7f5c973a57a0 20 read_log 192888'12569 (192888'12499) modify 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.1315259.0:59416 2014-08-11 16:47:04.108973 2014-08-13 18:06:06.959890 7f5c973a57a0 20 read_log 192888'12571 (192888'12503) modify 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.1315259.0:61261 2014-08-11 16:47:04.687107 2014-08-13 18:06:06.959906 7f5c973a57a0 20 read_log 192888'12576 (0'0) promote 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by osd.53.0:2629 2014-08-11 17:05:20.442140 2014-08-13 18:06:06.959918 7f5c973a57a0 20 read_log 192888'12577 (192888'12576) modify 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.942868.0:1322 2014-08-11 17:05:20.434400 2014-08-13 18:06:06.959941 7f5c973a57a0 20 read_log 192888'12582 (0'0) promote 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by osd.53.0:2652 2014-08-11 17:05:27.824069 2014-08-13 18:06:06.959956 7f5c973a57a0 20 read_log 192888'12583 (192888'12582) modify 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.942868.0:3211 2014-08-11 17:05:27.815726 2014-08-13 18:06:06.959969 7f5c973a57a0 20 read_log 192888'12590 (0'0) promote d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by osd.53.0:2666 2014-08-11 17:05:33.055738 2014-08-13 18:06:06.959981 7f5c973a57a0 20 read_log 192888'12591 (192888'12590) modify d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.942868.0:4628 2014-08-11 17:05:33.049194 2014-08-13 18:06:06.959993 7f5c973a57a0 20 read_log 192888'12592 (0'0) promote 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by osd.53.0:2676 2014-08-11 17:05:34.826544 2014-08-13 18:06:06.960007 7f5c973a57a0 20 read_log 192888'12593 (192888'12592) modify 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.942868.0:5053 2014-08-11 17:05:34.819231 2014-08-13 18:06:06.960022 7f5c973a57a0 20 read_log 192888'12602 (0'0) promote 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by osd.53.0:2722 2014-08-11 17:05:42.594013 2014-08-13 18:06:06.960034 7f5c973a57a0 20 read_log 192888'12603 (192888'12602) modify 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.942868.0:7153 2014-08-11 17:05:42.585500 2014-08-13 18:06:06.960046 7f5c973a57a0 20 read_log 192888'12604 (0'0) promote ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by osd.53.0:2728 2014-08-11 17:05:44.379369 2014-08-13 18:06:06.960060 7f5c973a57a0 20 read_log 192888'12605 (192888'12604) modify ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.942868.0:7599 2014-08-11 17:05:44.372173 2014-08-13 18:06:06.960073 7f5c973a57a0 20 read_log 192888'12606 (0'0) promote 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by osd.53.0:2742 2014-08-11 17:05:48.884789 2014-08-13 18:06:06.960087 7f5c973a57a0 20 read_log 192888'12607 (192888'12606) modify 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.942868.0:8823 2014-08-11 17:05:48.875327 2014-08-13 18:06:06.960100 7f5c973a57a0 20 read_log 192888'12608 (0'0) promote 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by osd.53.0:2762 2014-08-11 17:05:52.292885 2014-08-13 18:06:06.960114 7f5c973a57a0 20 read_log 192888'12609 (192888'12608) modify 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.942868.0:9748 2014-08-11 17:05:52.284982 2014-08-13 18:06:06.960127 7f5c973a57a0 20 read_log 192888'12610 (0'0) promote 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by osd.53.0:2774 2014-08-11 17:05:54.794589 2014-08-13 18:06:06.960141 7f5c973a57a0 20 read_log 192888'12611 (192888'12610) modify 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.942868.0:10441 2014-08-11 17:05:54.787112 2014-08-13 18:06:06.960161 7f5c973a57a0 20 read_log 192888'12614 (0'0) promote d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by osd.53.0:2804 2014-08-11 17:06:01.913170 2014-08-13 18:06:06.960174 7f5c973a57a0 20 read_log 192888'12615 (192888'12614) modify d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.942868.0:12393 2014-08-11 17:06:01.904756 2014-08-13 18:06:06.960186 7f5c973a57a0 20 read_log 192888'12616 (0'0) promote 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by osd.53.0:2806 2014-08-11 17:06:02.871602 2014-08-13 18:06:06.960199 7f5c973a57a0 20 read_log 192888'12617 (192888'12616) modify 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.942868.0:12661 2014-08-11 17:06:02.862826 2014-08-13 18:06:06.960220 7f5c973a57a0 20 read_log 192888'12620 (0'0) promote 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by osd.53.0:2822 2014-08-11 17:06:06.222188 2014-08-13 18:06:06.960235 7f5c973a57a0 20 read_log 192888'12621 (192888'12620) modify 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.942868.0:13514 2014-08-11 17:06:06.213957 2014-08-13 18:06:06.960250 7f5c973a57a0 20 read_log 192888'12626 (0'0) promote 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by osd.53.0:2854 2014-08-11 17:06:12.924197 2014-08-13 18:06:06.960263 7f5c973a57a0 20 read_log 192888'12627 (192888'12626) modify 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.942868.0:15180 2014-08-11 17:06:12.916057 2014-08-13 18:06:06.960275 7f5c973a57a0 20 read_log 192888'12628 (0'0) promote e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by osd.53.0:2866 2014-08-11 17:06:14.561760 2014-08-13 18:06:06.960287 7f5c973a57a0 20 read_log 192888'12629 (192888'12628) modify e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.942868.0:15601 2014-08-11 17:06:14.555047 2014-08-13 18:06:06.960300 7f5c973a57a0 20 read_log 192888'12632 (0'0) promote 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by osd.53.0:2876 2014-08-11 17:06:16.480713 2014-08-13 18:06:06.960312 7f5c973a57a0 20 read_log 192888'12633 (192888'12632) modify 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.942868.0:16097 2014-08-11 17:06:16.473295 2014-08-13 18:06:06.960324 7f5c973a57a0 20 read_log 192888'12638 (0'0) promote ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by osd.53.0:2938 2014-08-11 17:06:31.295201 2014-08-13 18:06:06.960339 7f5c973a57a0 20 read_log 192888'12639 (192888'12638) modify ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.942868.0:19981 2014-08-11 17:06:31.286061 2014-08-13 18:06:06.960380 7f5c973a57a0 20 read_log 192888'12642 (0'0) promote 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by osd.53.0:2960 2014-08-11 17:06:35.019717 2014-08-13 18:06:06.960397 7f5c973a57a0 20 read_log 192888'12643 (192888'12642) modify 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.942868.0:20954 2014-08-11 17:06:35.010610 2014-08-13 18:06:06.960409 7f5c973a57a0 20 read_log 192888'12646 (0'0) promote 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by osd.53.0:2974 2014-08-11 17:06:39.333142 2014-08-13 18:06:06.960434 7f5c973a57a0 20 read_log 192888'12647 (192888'12646) modify 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.942868.0:22097 2014-08-11 17:06:39.325462 2014-08-13 18:06:06.960451 7f5c973a57a0 20 read_log 192888'12648 (0'0) promote 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by osd.53.0:2990 2014-08-11 17:06:44.015003 2014-08-13 18:06:06.960463 7f5c973a57a0 20 read_log 192888'12649 (192888'12648) modify 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.942868.0:23267 2014-08-11 17:06:44.006553 2014-08-13 18:06:06.960488 7f5c973a57a0 20 read_log 192888'12650 (0'0) promote fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by osd.53.0:3004 2014-08-11 17:06:45.912874 2014-08-13 18:06:06.960502 7f5c973a57a0 20 read_log 192888'12651 (192888'12650) modify fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.942868.0:23782 2014-08-11 17:06:45.904366 2014-08-13 18:06:06.960514 7f5c973a57a0 20 read_log 192888'12652 (0'0) promote 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by osd.53.0:3006 2014-08-11 17:06:46.120649 2014-08-13 18:06:06.960527 7f5c973a57a0 20 read_log 192888'12653 (192888'12652) modify 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.942868.0:23838 2014-08-11 17:06:46.112325 2014-08-13 18:06:06.960546 7f5c973a57a0 20 read_log 192888'12656 (0'0) promote c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by osd.53.0:3034 2014-08-11 17:06:52.462498 2014-08-13 18:06:06.960561 7f5c973a57a0 20 read_log 192888'12657 (192888'12656) modify c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.942868.0:25542 2014-08-11 17:06:52.454655 2014-08-13 18:06:06.960574 7f5c973a57a0 20 read_log 192888'12658 (0'0) promote c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by osd.53.0:3036 2014-08-11 17:06:52.477178 2014-08-13 18:06:06.960586 7f5c973a57a0 20 read_log 192888'12659 (192888'12658) modify c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.942868.0:25546 2014-08-11 17:06:52.469096 2014-08-13 18:06:06.960599 7f5c973a57a0 20 read_log 192888'12664 (0'0) promote 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by osd.53.0:3064 2014-08-11 17:07:01.564991 2014-08-13 18:06:06.960611 7f5c973a57a0 20 read_log 192888'12665 (192888'12664) modify 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.942868.0:27883 2014-08-11 17:07:01.556873 2014-08-13 18:06:06.960624 7f5c973a57a0 20 read_log 192888'12673 (192888'12577) modify 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.942868.0:32588 2014-08-11 17:07:16.294958 2014-08-13 18:06:06.960636 7f5c973a57a0 20 read_log 192888'12676 (192888'12583) modify 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.942868.0:34477 2014-08-11 17:07:16.801474 2014-08-13 18:06:06.960657 7f5c973a57a0 20 read_log 192888'12680 (192888'12591) modify d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.942868.0:35894 2014-08-11 17:07:17.191056 2014-08-13 18:06:06.960670 7f5c973a57a0 20 read_log 192888'12681 (192888'12593) modify 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.942868.0:36319 2014-08-11 17:07:17.296341 2014-08-13 18:06:06.960684 7f5c973a57a0 20 read_log 192888'12686 (192888'12603) modify 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.942868.0:38419 2014-08-11 17:07:17.882285 2014-08-13 18:06:06.960696 7f5c973a57a0 20 read_log 192888'12687 (192888'12605) modify ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.942868.0:38865 2014-08-11 17:07:18.012738 2014-08-13 18:06:06.960709 7f5c973a57a0 20 read_log 192888'12688 (192888'12607) modify 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.942868.0:40089 2014-08-11 17:07:18.361616 2014-08-13 18:06:06.960721 7f5c973a57a0 20 read_log 192888'12689 (192888'12609) modify 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.942868.0:41014 2014-08-11 17:07:18.610084 2014-08-13 18:06:06.960733 7f5c973a57a0 20 read_log 192888'12690 (192888'12611) modify 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.942868.0:41707 2014-08-11 17:07:18.802927 2014-08-13 18:06:06.960745 7f5c973a57a0 20 read_log 192888'12692 (192888'12615) modify d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.942868.0:43659 2014-08-11 17:07:19.335679 2014-08-13 18:06:06.960758 7f5c973a57a0 20 read_log 192888'12693 (192888'12617) modify 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.942868.0:43927 2014-08-11 17:07:19.416546 2014-08-13 18:06:06.960770 7f5c973a57a0 20 read_log 192888'12695 (192888'12621) modify 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.942868.0:44780 2014-08-11 17:07:19.660594 2014-08-13 18:06:06.960785 7f5c973a57a0 20 read_log 192888'12698 (192888'12627) modify 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.942868.0:46446 2014-08-11 17:07:20.123284 2014-08-13 18:06:06.960798 7f5c973a57a0 20 read_log 192888'12699 (192888'12629) modify e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.942868.0:46867 2014-08-11 17:07:20.239795 2014-08-13 18:06:06.960840 7f5c973a57a0 20 read_log 192888'12701 (192888'12633) modify 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.942868.0:47363 2014-08-11 17:07:20.382427 2014-08-13 18:06:06.960856 7f5c973a57a0 20 read_log 192888'12704 (192888'12639) modify ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.942868.0:51247 2014-08-11 17:07:21.584368 2014-08-13 18:06:06.960868 7f5c973a57a0 20 read_log 192888'12706 (192888'12643) modify 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.942868.0:52220 2014-08-11 17:07:21.874732 2014-08-13 18:06:06.960881 7f5c973a57a0 20 read_log 192888'12708 (192888'12647) modify 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.942868.0:53363 2014-08-11 17:07:22.239381 2014-08-13 18:06:06.960893 7f5c973a57a0 20 read_log 192888'12709 (192888'12649) modify 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.942868.0:54533 2014-08-11 17:07:22.584666 2014-08-13 18:06:06.960905 7f5c973a57a0 20 read_log 192888'12710 (192888'12651) modify fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.942868.0:55048 2014-08-11 17:07:22.726760 2014-08-13 18:06:06.960917 7f5c973a57a0 20 read_log 192888'12711 (192888'12653) modify 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.942868.0:55104 2014-08-11 17:07:22.744878 2014-08-13 18:06:06.960932 7f5c973a57a0 20 read_log 192888'12713 (192888'12657) modify c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.942868.0:56808 2014-08-11 17:07:23.264460 2014-08-13 18:06:06.960944 7f5c973a57a0 20 read_log 192888'12714 (192888'12659) modify c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.942868.0:56812 2014-08-11 17:07:23.266562 2014-08-13 18:06:06.960958 7f5c973a57a0 20 read_log 192888'12717 (192888'12665) modify 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.942868.0:59149 2014-08-11 17:07:23.959845 2014-08-13 18:06:06.960971 7f5c973a57a0 20 read_log 193305'12721 (0'0) promote 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by osd.325.0:19 2014-08-11 18:42:36.835277 2014-08-13 18:06:06.960985 7f5c973a57a0 20 read_log 193305'12722 (193305'12721) modify 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.951337.0:961 2014-08-11 18:42:36.826632 2014-08-13 18:06:06.960997 7f5c973a57a0 20 read_log 193305'12723 (0'0) promote a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by osd.325.0:21 2014-08-11 18:42:37.735078 2014-08-13 18:06:06.961012 7f5c973a57a0 20 read_log 193305'12724 (193305'12723) modify a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.951337.0:1210 2014-08-11 18:42:37.727210 2014-08-13 18:06:06.961027 7f5c973a57a0 20 read_log 193305'12725 (0'0) promote fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by osd.325.0:52 2014-08-11 18:42:44.294427 2014-08-13 18:06:06.961039 7f5c973a57a0 20 read_log 193305'12726 (193305'12725) modify fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.951337.0:2974 2014-08-11 18:42:44.286836 2014-08-13 18:06:06.961054 7f5c973a57a0 20 read_log 193305'12727 (0'0) promote 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by osd.325.0:64 2014-08-11 18:42:47.862278 2014-08-13 18:06:06.961066 7f5c973a57a0 20 read_log 193305'12728 (193305'12727) modify 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.951337.0:3954 2014-08-11 18:42:47.852246 2014-08-13 18:06:06.961080 7f5c973a57a0 20 read_log 193305'12729 (0'0) promote 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by osd.325.0:68 2014-08-11 18:42:48.124204 2014-08-13 18:06:06.961092 7f5c973a57a0 20 read_log 193305'12730 (193305'12729) modify 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.951337.0:4027 2014-08-11 18:42:48.115775 2014-08-13 18:06:06.961112 7f5c973a57a0 20 read_log 193305'12731 (0'0) promote 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by osd.325.0:88 2014-08-11 18:42:53.761103 2014-08-13 18:06:06.961127 7f5c973a57a0 20 read_log 193305'12732 (193305'12731) modify 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.951337.0:5542 2014-08-11 18:42:53.750412 2014-08-13 18:06:06.961139 7f5c973a57a0 20 read_log 193305'12733 (0'0) promote 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by osd.325.0:92 2014-08-11 18:42:54.152778 2014-08-13 18:06:06.961156 7f5c973a57a0 20 read_log 193305'12734 (193305'12733) modify 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.951337.0:5649 2014-08-11 18:42:54.145526 2014-08-13 18:06:06.961171 7f5c973a57a0 20 read_log 193305'12735 (0'0) promote 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by osd.325.0:108 2014-08-11 18:42:57.926559 2014-08-13 18:06:06.961184 7f5c973a57a0 20 read_log 193305'12736 (193305'12735) modify 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.951337.0:6690 2014-08-11 18:42:57.917772 2014-08-13 18:06:06.961196 7f5c973a57a0 20 read_log 193305'12737 (0'0) promote 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by osd.325.0:148 2014-08-11 18:43:07.498801 2014-08-13 18:06:06.961208 7f5c973a57a0 20 read_log 193305'12738 (193305'12737) modify 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.951337.0:9238 2014-08-11 18:43:07.486230 2014-08-13 18:06:06.961223 7f5c973a57a0 20 read_log 193305'12739 (0'0) promote 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by osd.325.0:190 2014-08-11 18:43:17.492263 2014-08-13 18:06:06.961235 7f5c973a57a0 20 read_log 193305'12740 (193305'12739) modify 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.951337.0:11889 2014-08-11 18:43:17.484481 2014-08-13 18:06:06.961269 7f5c973a57a0 20 read_log 193305'12741 (0'0) promote c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by osd.325.0:196 2014-08-11 18:43:18.906139 2014-08-13 18:06:06.961284 7f5c973a57a0 20 read_log 193305'12742 (193305'12741) modify c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.951337.0:12277 2014-08-11 18:43:18.897854 2014-08-13 18:06:06.961297 7f5c973a57a0 20 read_log 193305'12743 (0'0) promote 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by osd.325.0:198 2014-08-11 18:43:19.401694 2014-08-13 18:06:06.961325 7f5c973a57a0 20 read_log 193305'12744 (193305'12743) modify 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.951337.0:12413 2014-08-11 18:43:19.394021 2014-08-13 18:06:06.961374 7f5c973a57a0 20 read_log 193305'12745 (0'0) promote 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by osd.325.0:200 2014-08-11 18:43:20.149984 2014-08-13 18:06:06.961396 7f5c973a57a0 20 read_log 193305'12746 (193305'12745) modify 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.951337.0:12614 2014-08-11 18:43:20.142676 2014-08-13 18:06:06.961437 7f5c973a57a0 20 read_log 193305'12747 (0'0) promote f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by osd.325.0:217 2014-08-11 18:43:23.957693 2014-08-13 18:06:06.961450 7f5c973a57a0 20 read_log 193305'12748 (193305'12747) modify f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.951337.0:13604 2014-08-11 18:43:23.949435 2014-08-13 18:06:06.961478 7f5c973a57a0 20 read_log 193305'12749 (0'0) promote cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by osd.325.0:227 2014-08-11 18:43:25.855570 2014-08-13 18:06:06.961492 7f5c973a57a0 20 read_log 193305'12750 (193305'12749) modify cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.951337.0:14127 2014-08-11 18:43:25.846069 2014-08-13 18:06:06.961536 7f5c973a57a0 20 read_log 193305'12751 (0'0) promote b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by osd.325.0:362 2014-08-11 18:43:55.502763 2014-08-13 18:06:06.961556 7f5c973a57a0 20 read_log 193305'12752 (193305'12751) modify b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.951337.0:22454 2014-08-11 18:43:55.493161 2014-08-13 18:06:06.961569 7f5c973a57a0 20 read_log 193305'12753 (0'0) promote b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by osd.325.0:366 2014-08-11 18:43:56.487628 2014-08-13 18:06:06.961582 7f5c973a57a0 20 read_log 193305'12754 (193305'12753) modify b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.951337.0:22727 2014-08-11 18:43:56.479265 2014-08-13 18:06:06.961597 7f5c973a57a0 20 read_log 193305'12755 (0'0) promote 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by osd.325.0:370 2014-08-11 18:43:57.637324 2014-08-13 18:06:06.961610 7f5c973a57a0 20 read_log 193305'12756 (193305'12755) modify 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.951337.0:23056 2014-08-11 18:43:57.628972 2014-08-13 18:06:06.961622 7f5c973a57a0 20 read_log 193305'12757 (0'0) promote feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by osd.325.0:382 2014-08-11 18:43:59.095918 2014-08-13 18:06:06.961635 7f5c973a57a0 20 read_log 193305'12758 (193305'12757) modify feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.951337.0:23472 2014-08-11 18:43:59.086354 2014-08-13 18:06:06.961647 7f5c973a57a0 20 read_log 193305'12759 (0'0) promote f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by osd.325.0:420 2014-08-11 18:44:05.435034 2014-08-13 18:06:06.961659 7f5c973a57a0 20 read_log 193305'12760 (193305'12759) modify f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.951337.0:25217 2014-08-11 18:44:05.425805 2014-08-13 18:06:06.961671 7f5c973a57a0 20 read_log 193305'12761 (0'0) promote e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by osd.325.0:444 2014-08-11 18:44:15.175379 2014-08-13 18:06:06.961684 7f5c973a57a0 20 read_log 193305'12762 (193305'12761) modify e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.951337.0:27865 2014-08-11 18:44:15.166186 2014-08-13 18:06:06.961696 7f5c973a57a0 20 read_log 193305'12763 (0'0) promote 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by osd.325.0:450 2014-08-11 18:44:17.757383 2014-08-13 18:06:06.961720 7f5c973a57a0 20 read_log 193305'12764 (193305'12763) modify 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.951337.0:28576 2014-08-11 18:44:17.749603 2014-08-13 18:06:06.961733 7f5c973a57a0 20 read_log 193305'12765 (0'0) promote 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by osd.325.0:480 2014-08-11 18:44:21.215262 2014-08-13 18:06:06.961769 7f5c973a57a0 20 read_log 193305'12766 (193305'12765) modify 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.951337.0:29513 2014-08-11 18:44:21.205954 2014-08-13 18:06:06.961786 7f5c973a57a0 20 read_log 193305'12767 (0'0) promote 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by osd.325.0:490 2014-08-11 18:44:26.079118 2014-08-13 18:06:06.961799 7f5c973a57a0 20 read_log 193305'12768 (193305'12767) modify 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.951337.0:30790 2014-08-11 18:44:26.066081 2014-08-13 18:06:06.961814 7f5c973a57a0 20 read_log 193305'12769 (193305'12722) modify 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.951337.0:33672 2014-08-11 18:44:34.572021 2014-08-13 18:06:06.961830 7f5c973a57a0 20 read_log 193305'12770 (193305'12724) modify a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.951337.0:33921 2014-08-11 18:44:34.717676 2014-08-13 18:06:06.961853 7f5c973a57a0 20 read_log 193305'12771 (193305'12726) modify fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.951337.0:35685 2014-08-11 18:44:35.702476 2014-08-13 18:06:06.961871 7f5c973a57a0 20 read_log 193305'12772 (193305'12728) modify 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.951337.0:36665 2014-08-11 18:44:36.463487 2014-08-13 18:06:06.961884 7f5c973a57a0 20 read_log 193305'12773 (193305'12730) modify 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.951337.0:36738 2014-08-11 18:44:36.500962 2014-08-13 18:06:06.961920 7f5c973a57a0 20 read_log 193305'12774 (193305'12732) modify 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.951337.0:38253 2014-08-11 18:44:37.305858 2014-08-13 18:06:06.961941 7f5c973a57a0 20 read_log 193305'12775 (193305'12734) modify 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.951337.0:38360 2014-08-11 18:44:37.366464 2014-08-13 18:06:06.961957 7f5c973a57a0 20 read_log 193305'12776 (193305'12736) modify 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.951337.0:39401 2014-08-11 18:44:37.988147 2014-08-13 18:06:06.961972 7f5c973a57a0 20 read_log 193305'12777 (193305'12738) modify 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.951337.0:41949 2014-08-11 18:44:38.940231 2014-08-13 18:06:06.961987 7f5c973a57a0 20 read_log 193305'12778 (193305'12740) modify 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.951337.0:44600 2014-08-11 18:44:39.928990 2014-08-13 18:06:06.962000 7f5c973a57a0 20 read_log 193305'12779 (193305'12742) modify c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.951337.0:44988 2014-08-11 18:44:40.059738 2014-08-13 18:06:06.962013 7f5c973a57a0 20 read_log 193305'12780 (193305'12744) modify 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.951337.0:45124 2014-08-11 18:44:40.096802 2014-08-13 18:06:06.962025 7f5c973a57a0 20 read_log 193305'12781 (193305'12746) modify 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.951337.0:45325 2014-08-11 18:44:40.163551 2014-08-13 18:06:06.962037 7f5c973a57a0 20 read_log 193305'12782 (193305'12748) modify f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.951337.0:46315 2014-08-11 18:44:40.448619 2014-08-13 18:06:06.962050 7f5c973a57a0 20 read_log 193305'12783 (193305'12750) modify cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.951337.0:46838 2014-08-11 18:44:40.602189 2014-08-13 18:06:06.962062 7f5c973a57a0 20 read_log 193305'12784 (193305'12752) modify b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.951337.0:55165 2014-08-11 18:44:43.042363 2014-08-13 18:06:06.962074 7f5c973a57a0 20 read_log 193305'12785 (193305'12754) modify b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.951337.0:55438 2014-08-11 18:44:43.123289 2014-08-13 18:06:06.962086 7f5c973a57a0 20 read_log 193305'12786 (193305'12756) modify 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.951337.0:55767 2014-08-11 18:44:43.225264 2014-08-13 18:06:06.962099 7f5c973a57a0 20 read_log 193305'12787 (193305'12758) modify feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.951337.0:56183 2014-08-11 18:44:43.347657 2014-08-13 18:06:06.962111 7f5c973a57a0 20 read_log 193305'12788 (193305'12760) modify f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.951337.0:57928 2014-08-11 18:44:43.897261 2014-08-13 18:06:06.962123 7f5c973a57a0 20 read_log 193305'12789 (193305'12762) modify e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.951337.0:60576 2014-08-11 18:44:44.669363 2014-08-13 18:06:06.962136 7f5c973a57a0 20 read_log 193305'12790 (193305'12764) modify 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.951337.0:61287 2014-08-11 18:44:44.879334 2014-08-13 18:06:06.962163 7f5c973a57a0 20 read_log 193305'12791 (193305'12766) modify 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.951337.0:62224 2014-08-11 18:44:45.162413 2014-08-13 18:06:06.962179 7f5c973a57a0 20 read_log 193305'12792 (193305'12768) modify 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.951337.0:63501 2014-08-11 18:44:45.526454 2014-08-13 18:06:06.962192 7f5c973a57a0 20 read_log 194215'12793 (194215'12793) modify 39a/hit_set_24.39a_archive_2014-08-12 15:32:25.737468_2014-08-12 15:50:26.492943/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:26.493191 2014-08-13 18:06:06.962205 7f5c973a57a0 20 read_log 194215'12794 (192834'11807) clean d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by osd.325.0:2318 2014-08-12 15:50:26.498736 2014-08-13 18:06:06.962217 7f5c973a57a0 20 read_log 194215'12795 (194215'12794) delete d113039a/benchmark_data_storage0111-ib.pocstor_26659_object10879/head//24 by client.2174595.0:190441 0.000000 2014-08-13 18:06:06.962228 7f5c973a57a0 20 read_log 194215'12796 (192834'12293) clean 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by osd.325.0:2320 2014-08-12 15:50:26.530490 2014-08-13 18:06:06.962243 7f5c973a57a0 20 read_log 194215'12797 (194215'12796) delete 7147039a/benchmark_data_storage0111-ib.pocstor_41526_object12099/head//24 by client.2174595.0:190443 0.000000 2014-08-13 18:06:06.962254 7f5c973a57a0 20 read_log 194215'12798 (193305'12770) clean a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by osd.325.0:2322 2014-08-12 15:50:26.560218 2014-08-13 18:06:06.962267 7f5c973a57a0 20 read_log 194215'12799 (194215'12798) delete a8d7039a/benchmark_data_storage0111-ib.pocstor_62621_object1209/head//24 by client.2174595.0:190445 0.000000 2014-08-13 18:06:06.962277 7f5c973a57a0 20 read_log 194215'12800 (192888'12688) clean 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by osd.325.0:2324 2014-08-12 15:50:26.573778 2014-08-13 18:06:06.962290 7f5c973a57a0 20 read_log 194215'12801 (194215'12800) delete 74aa039a/benchmark_data_storage0111-ib.pocstor_2228_object8822/head//24 by client.2174595.0:190447 0.000000 2014-08-13 18:06:06.962301 7f5c973a57a0 20 read_log 194215'12802 (192888'12553) clean 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by osd.325.0:2326 2014-08-12 15:50:26.587278 2014-08-13 18:06:06.962313 7f5c973a57a0 20 read_log 194215'12803 (194215'12802) delete 918e039a/benchmark_data_storage0111-ib.pocstor_41097_object20649/head//24 by client.2174595.0:190449 0.000000 2014-08-13 18:06:06.962324 7f5c973a57a0 20 read_log 194215'12804 (192834'12284) clean 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by osd.325.0:2328 2014-08-12 15:50:26.602366 2014-08-13 18:06:06.962337 7f5c973a57a0 20 read_log 194215'12805 (194215'12804) delete 8056139a/benchmark_data_storage0111-ib.pocstor_41526_object11526/head//24 by client.2174595.0:190451 0.000000 2014-08-13 18:06:06.962348 7f5c973a57a0 20 read_log 194215'12806 (193305'12776) clean 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by osd.325.0:2330 2014-08-12 15:50:26.616354 2014-08-13 18:06:06.962370 7f5c973a57a0 20 read_log 194215'12807 (194215'12806) delete 1558139a/benchmark_data_storage0111-ib.pocstor_62621_object6689/head//24 by client.2174595.0:190453 0.000000 2014-08-13 18:06:06.962382 7f5c973a57a0 20 read_log 194215'12808 (192888'12689) clean 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by osd.325.0:2332 2014-08-12 15:50:26.630632 2014-08-13 18:06:06.962395 7f5c973a57a0 20 read_log 194215'12809 (194215'12808) delete 5a9c139a/benchmark_data_storage0111-ib.pocstor_2228_object9747/head//24 by client.2174595.0:190455 0.000000 2014-08-13 18:06:06.962426 7f5c973a57a0 20 read_log 194215'12810 (192888'12507) clean 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by osd.325.0:2334 2014-08-12 15:50:26.646704 2014-08-13 18:06:06.962443 7f5c973a57a0 20 read_log 194215'12811 (194215'12810) delete 5860239a/benchmark_data_storage0111-ib.pocstor_41097_object986/head//24 by client.2174595.0:190457 0.000000 2014-08-13 18:06:06.962454 7f5c973a57a0 20 read_log 194215'12812 (192834'12227) clean 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by osd.325.0:2336 2014-08-12 15:50:26.659809 2014-08-13 18:06:06.962477 7f5c973a57a0 20 read_log 194215'12813 (194215'12812) delete 9c13239a/benchmark_data_storage0111-ib.pocstor_41526_object2453/head//24 by client.2174595.0:190459 0.000000 2014-08-13 18:06:06.962509 7f5c973a57a0 20 read_log 194215'12814 (192834'11902) clean 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by osd.325.0:2338 2014-08-12 15:50:26.673679 2014-08-13 18:06:06.962531 7f5c973a57a0 20 read_log 194215'12815 (194215'12814) delete 2f16239a/benchmark_data_storage0111-ib.pocstor_26659_object22167/head//24 by client.2174595.0:190461 0.000000 2014-08-13 18:06:06.962553 7f5c973a57a0 20 read_log 194215'12816 (192834'12220) clean fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by osd.325.0:2340 2014-08-12 15:50:26.688533 2014-08-13 18:06:06.962581 7f5c973a57a0 20 read_log 194215'12817 (194215'12816) delete fe19239a/benchmark_data_storage0111-ib.pocstor_41526_object1454/head//24 by client.2174595.0:190463 0.000000 2014-08-13 18:06:06.962621 7f5c973a57a0 20 read_log 194215'12818 (192834'12245) clean 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by osd.325.0:2342 2014-08-12 15:50:26.703366 2014-08-13 18:06:06.962639 7f5c973a57a0 20 read_log 194215'12819 (194215'12818) delete 957c239a/benchmark_data_storage0111-ib.pocstor_41526_object5404/head//24 by client.2174595.0:190465 0.000000 2014-08-13 18:06:06.962654 7f5c973a57a0 20 read_log 194215'12820 (193305'12769) clean 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by osd.325.0:2344 2014-08-12 15:50:26.718586 2014-08-13 18:06:06.962672 7f5c973a57a0 20 read_log 194215'12821 (194215'12820) delete 9a15339a/benchmark_data_storage0111-ib.pocstor_62621_object960/head//24 by client.2174595.0:190467 0.000000 2014-08-13 18:06:06.962683 7f5c973a57a0 20 read_log 194215'12822 (192834'11810) clean 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by osd.325.0:2346 2014-08-12 15:50:26.731141 2014-08-13 18:06:06.962709 7f5c973a57a0 20 read_log 194215'12823 (194215'12822) delete 9f61439a/benchmark_data_storage0111-ib.pocstor_26659_object11176/head//24 by client.2174595.0:190469 0.000000 2014-08-13 18:06:06.962727 7f5c973a57a0 20 read_log 194215'12824 (193305'12773) clean 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by osd.325.0:2348 2014-08-12 15:50:26.743517 2014-08-13 18:06:06.962744 7f5c973a57a0 20 read_log 194215'12825 (194215'12824) delete 2b26439a/benchmark_data_storage0111-ib.pocstor_62621_object4026/head//24 by client.2174595.0:190471 0.000000 2014-08-13 18:06:06.962761 7f5c973a57a0 20 read_log 194215'12826 (192834'12262) clean 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by osd.325.0:2350 2014-08-12 15:50:26.757645 2014-08-13 18:06:06.962774 7f5c973a57a0 20 read_log 194215'12827 (194215'12826) delete 8368439a/benchmark_data_storage0111-ib.pocstor_41526_object8477/head//24 by client.2174595.0:190473 0.000000 2014-08-13 18:06:06.962785 7f5c973a57a0 20 read_log 194215'12828 (192888'12557) clean c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by osd.325.0:2352 2014-08-12 15:50:26.773343 2014-08-13 18:06:06.962798 7f5c973a57a0 20 read_log 194215'12829 (194215'12828) delete c9fc439a/benchmark_data_storage0111-ib.pocstor_41097_object22797/head//24 by client.2174595.0:190475 0.000000 2014-08-13 18:06:06.962819 7f5c973a57a0 20 read_log 194215'12830 (193305'12783) clean cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by osd.325.0:2354 2014-08-12 15:50:26.788736 2014-08-13 18:06:06.962835 7f5c973a57a0 20 read_log 194215'12831 (194215'12830) delete cc3e439a/benchmark_data_storage0111-ib.pocstor_62621_object14126/head//24 by client.2174595.0:190477 0.000000 2014-08-13 18:06:06.962846 7f5c973a57a0 20 read_log 194215'12832 (192888'12714) clean c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by osd.325.0:2356 2014-08-12 15:50:26.801139 2014-08-13 18:06:06.962858 7f5c973a57a0 20 read_log 194215'12833 (194215'12832) delete c601539a/benchmark_data_storage0111-ib.pocstor_2228_object25545/head//24 by client.2174595.0:190479 0.000000 2014-08-13 18:06:06.962890 7f5c973a57a0 20 read_log 194215'12834 (192834'12269) clean f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by osd.325.0:2358 2014-08-12 15:50:26.815053 2014-08-13 18:06:06.962905 7f5c973a57a0 20 read_log 194215'12835 (194215'12834) delete f5bb539a/benchmark_data_storage0111-ib.pocstor_41526_object9437/head//24 by client.2174595.0:190481 0.000000 2014-08-13 18:06:06.962917 7f5c973a57a0 20 read_log 194215'12836 (192834'11756) clean df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by osd.325.0:2360 2014-08-12 15:50:26.829789 2014-08-13 18:06:06.962930 7f5c973a57a0 20 read_log 194215'12837 (194215'12836) delete df43639a/benchmark_data_storage0111-ib.pocstor_26659_object3753/head//24 by client.2174595.0:190483 0.000000 2014-08-13 18:06:06.962940 7f5c973a57a0 20 read_log 194215'12838 (193305'12780) clean 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by osd.325.0:2362 2014-08-12 15:50:26.844025 2014-08-13 18:06:06.962953 7f5c973a57a0 20 read_log 194215'12839 (194215'12838) delete 5c69639a/benchmark_data_storage0111-ib.pocstor_62621_object12412/head//24 by client.2174595.0:190485 0.000000 2014-08-13 18:06:06.962963 7f5c973a57a0 20 read_log 194215'12840 (192888'12528) clean 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by osd.325.0:2364 2014-08-12 15:50:26.889538 2014-08-13 18:06:06.962976 7f5c973a57a0 20 read_log 194215'12841 (194215'12840) delete 59aa639a/benchmark_data_storage0111-ib.pocstor_41097_object11660/head//24 by client.2174595.0:190487 0.000000 2014-08-13 18:06:06.962986 7f5c973a57a0 20 read_log 194215'12842 (192834'12252) clean f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by osd.325.0:2366 2014-08-12 15:50:26.904158 2014-08-13 18:06:06.962998 7f5c973a57a0 20 read_log 194215'12843 (194215'12842) delete f858739a/benchmark_data_storage0111-ib.pocstor_41526_object6278/head//24 by client.2174595.0:190489 0.000000 2014-08-13 18:06:06.963009 7f5c973a57a0 20 read_log 194215'12844 (192888'12514) clean d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by osd.325.0:2368 2014-08-12 15:50:26.919116 2014-08-13 18:06:06.963021 7f5c973a57a0 20 read_log 194215'12845 (194215'12844) delete d6c1939a/benchmark_data_storage0111-ib.pocstor_41097_object4023/head//24 by client.2174595.0:190491 0.000000 2014-08-13 18:06:06.963032 7f5c973a57a0 20 read_log 194215'12846 (193305'12774) clean 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by osd.325.0:2370 2014-08-12 15:50:26.932619 2014-08-13 18:06:06.963044 7f5c973a57a0 20 read_log 194215'12847 (194215'12846) delete 894a939a/benchmark_data_storage0111-ib.pocstor_62621_object5541/head//24 by client.2174595.0:190493 0.000000 2014-08-13 18:06:06.963058 7f5c973a57a0 20 read_log 194215'12848 (192834'11831) clean 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by osd.325.0:2372 2014-08-12 15:50:26.945728 2014-08-13 18:06:06.963072 7f5c973a57a0 20 read_log 194215'12849 (194215'12848) delete 419d939a/benchmark_data_storage0111-ib.pocstor_26659_object13718/head//24 by client.2174595.0:190495 0.000000 2014-08-13 18:06:06.963104 7f5c973a57a0 20 read_log 194215'12850 (192834'11867) clean c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by osd.325.0:2374 2014-08-12 15:50:26.960969 2014-08-13 18:06:06.963119 7f5c973a57a0 20 read_log 194215'12851 (194215'12850) delete c8dd939a/benchmark_data_storage0111-ib.pocstor_26659_object17760/head//24 by client.2174595.0:190497 0.000000 2014-08-13 18:06:06.963130 7f5c973a57a0 20 read_log 194215'12852 (192834'11815) clean b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by osd.325.0:2376 2014-08-12 15:50:26.974412 2014-08-13 18:06:06.963143 7f5c973a57a0 20 read_log 194215'12853 (194215'12852) delete b8d5a39a/benchmark_data_storage0111-ib.pocstor_26659_object11856/head//24 by client.2174595.0:190499 0.000000 2014-08-13 18:06:06.963163 7f5c973a57a0 20 read_log 194215'12854 (193305'12791) clean 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by osd.325.0:2378 2014-08-12 15:50:26.988834 2014-08-13 18:06:06.963175 7f5c973a57a0 20 read_log 194215'12855 (194215'12854) delete 18b0b39a/benchmark_data_storage0111-ib.pocstor_62621_object29512/head//24 by client.2174595.0:190501 0.000000 2014-08-13 18:06:06.963186 7f5c973a57a0 20 read_log 194215'12856 (192888'12513) clean 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by osd.325.0:2380 2014-08-12 15:50:27.001361 2014-08-13 18:06:06.963198 7f5c973a57a0 20 read_log 194215'12857 (194215'12856) delete 23a1b39a/benchmark_data_storage0111-ib.pocstor_41097_object2617/head//24 by client.2174595.0:190503 0.000000 2014-08-13 18:06:06.963209 7f5c973a57a0 20 read_log 194215'12858 (193305'12790) clean 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by osd.325.0:2382 2014-08-12 15:50:27.015826 2014-08-13 18:06:06.963222 7f5c973a57a0 20 read_log 194215'12859 (194215'12858) delete 79dbc39a/benchmark_data_storage0111-ib.pocstor_62621_object28575/head//24 by client.2174595.0:190505 0.000000 2014-08-13 18:06:06.963233 7f5c973a57a0 20 read_log 194215'12860 (192834'11893) clean b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by osd.325.0:2384 2014-08-12 15:50:27.030439 2014-08-13 18:06:06.963245 7f5c973a57a0 20 read_log 194215'12861 (194215'12860) delete b93d39a/benchmark_data_storage0111-ib.pocstor_26659_object20366/head//24 by client.2174595.0:190507 0.000000 2014-08-13 18:06:06.963256 7f5c973a57a0 20 read_log 194215'12862 (192834'12224) clean 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by osd.325.0:2386 2014-08-12 15:50:27.043700 2014-08-13 18:06:06.963271 7f5c973a57a0 20 read_log 194215'12863 (194215'12862) delete 6f27d39a/benchmark_data_storage0111-ib.pocstor_41526_object2234/head//24 by client.2174595.0:190509 0.000000 2014-08-13 18:06:06.963282 7f5c973a57a0 20 read_log 194215'12864 (192888'12516) clean 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by osd.325.0:2388 2014-08-12 15:50:27.058018 2014-08-13 18:06:06.963295 7f5c973a57a0 20 read_log 194215'12865 (194215'12864) delete 79e9d39a/benchmark_data_storage0111-ib.pocstor_41097_object5704/head//24 by client.2174595.0:190511 0.000000 2014-08-13 18:06:06.963306 7f5c973a57a0 20 read_log 194215'12866 (192888'12692) clean d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by osd.325.0:2390 2014-08-12 15:50:27.071713 2014-08-13 18:06:06.963323 7f5c973a57a0 20 read_log 194215'12867 (194215'12866) delete d857e39a/benchmark_data_storage0111-ib.pocstor_2228_object12392/head//24 by client.2174595.0:190513 0.000000 2014-08-13 18:06:06.963334 7f5c973a57a0 20 read_log 194215'12868 (192834'11845) clean dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by osd.325.0:2392 2014-08-12 15:50:27.087665 2014-08-13 18:06:06.963346 7f5c973a57a0 20 read_log 194215'12869 (194215'12868) delete dc38f39a/benchmark_data_storage0111-ib.pocstor_26659_object15593/head//24 by client.2174595.0:190515 0.000000 2014-08-13 18:06:06.963367 7f5c973a57a0 20 read_log 194215'12870 (192888'12717) clean 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by osd.325.0:2394 2014-08-12 15:50:27.100203 2014-08-13 18:06:06.963380 7f5c973a57a0 20 read_log 194215'12871 (194215'12870) delete 75b2179a/benchmark_data_storage0111-ib.pocstor_2228_object27882/head//24 by client.2174595.0:190517 0.000000 2014-08-13 18:06:06.963402 7f5c973a57a0 20 read_log 194215'12872 (192834'12214) clean dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by osd.325.0:2396 2014-08-12 15:50:27.114111 2014-08-13 18:06:06.963418 7f5c973a57a0 20 read_log 194215'12873 (194215'12872) delete dae1279a/benchmark_data_storage0111-ib.pocstor_41526_object892/head//24 by client.2174595.0:190519 0.000000 2014-08-13 18:06:06.963429 7f5c973a57a0 20 read_log 194215'12874 (192888'12673) clean 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by osd.325.0:2398 2014-08-12 15:50:27.128655 2014-08-13 18:06:06.963452 7f5c973a57a0 20 read_log 194215'12875 (194215'12874) delete 49fc279a/benchmark_data_storage0111-ib.pocstor_2228_object1321/head//24 by client.2174595.0:190521 0.000000 2014-08-13 18:06:06.963463 7f5c973a57a0 20 read_log 194215'12876 (192834'12237) clean 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by osd.325.0:2400 2014-08-12 15:50:27.143921 2014-08-13 18:06:06.963476 7f5c973a57a0 20 read_log 194215'12877 (194215'12876) delete 5997379a/benchmark_data_storage0111-ib.pocstor_41526_object4231/head//24 by client.2174595.0:190523 0.000000 2014-08-13 18:06:06.963487 7f5c973a57a0 20 read_log 194215'12878 (192834'11895) clean 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by osd.325.0:2402 2014-08-12 15:50:27.156919 2014-08-13 18:06:06.963505 7f5c973a57a0 20 read_log 194215'12879 (194215'12878) delete 9d6a379a/benchmark_data_storage0111-ib.pocstor_26659_object20416/head//24 by client.2174595.0:190525 0.000000 2014-08-13 18:06:06.963516 7f5c973a57a0 20 read_log 194215'12880 (192888'12555) clean d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by osd.325.0:2404 2014-08-12 15:50:27.171481 2014-08-13 18:06:06.963528 7f5c973a57a0 20 read_log 194215'12881 (194215'12880) delete d42c379a/benchmark_data_storage0111-ib.pocstor_41097_object21547/head//24 by client.2174595.0:190527 0.000000 2014-08-13 18:06:06.963539 7f5c973a57a0 20 read_log 194215'12882 (192888'12529) clean eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by osd.325.0:2406 2014-08-12 15:50:27.186559 2014-08-13 18:06:06.963551 7f5c973a57a0 20 read_log 194215'12883 (194215'12882) delete eb9f379a/benchmark_data_storage0111-ib.pocstor_41097_object12200/head//24 by client.2174595.0:190529 0.000000 2014-08-13 18:06:06.963562 7f5c973a57a0 20 read_log 194215'12884 (193305'12781) clean 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by osd.325.0:2408 2014-08-12 15:50:27.201278 2014-08-13 18:06:06.963574 7f5c973a57a0 20 read_log 194215'12885 (194215'12884) delete 7862479a/benchmark_data_storage0111-ib.pocstor_62621_object12613/head//24 by client.2174595.0:190531 0.000000 2014-08-13 18:06:06.963585 7f5c973a57a0 20 read_log 194215'12886 (193305'12792) clean 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by osd.325.0:2410 2014-08-12 15:50:27.214916 2014-08-13 18:06:06.963605 7f5c973a57a0 20 read_log 194215'12887 (194215'12886) delete 500c479a/benchmark_data_storage0111-ib.pocstor_62621_object30789/head//24 by client.2174595.0:190533 0.000000 2014-08-13 18:06:06.963621 7f5c973a57a0 20 read_log 194215'12888 (192888'12521) clean 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by osd.325.0:2412 2014-08-12 15:50:27.228685 2014-08-13 18:06:06.963634 7f5c973a57a0 20 read_log 194215'12889 (194215'12888) delete 97c3679a/benchmark_data_storage0111-ib.pocstor_41097_object8497/head//24 by client.2174595.0:190535 0.000000 2014-08-13 18:06:06.963656 7f5c973a57a0 20 read_log 194215'12890 (193305'12778) clean 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by osd.325.0:2414 2014-08-12 15:50:27.244199 2014-08-13 18:06:06.963683 7f5c973a57a0 20 read_log 194215'12891 (194215'12890) delete 7ac3679a/benchmark_data_storage0111-ib.pocstor_62621_object11888/head//24 by client.2174595.0:190537 0.000000 2014-08-13 18:06:06.963697 7f5c973a57a0 20 read_log 194215'12892 (192888'12506) clean d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by osd.325.0:2416 2014-08-12 15:50:27.259251 2014-08-13 18:06:06.963713 7f5c973a57a0 20 read_log 194215'12893 (194215'12892) delete d564679a/benchmark_data_storage0111-ib.pocstor_41097_object670/head//24 by client.2174595.0:190539 0.000000 2014-08-13 18:06:06.963738 7f5c973a57a0 20 read_log 194215'12894 (192834'12246) clean 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by osd.325.0:2418 2014-08-12 15:50:27.271750 2014-08-13 18:06:06.963751 7f5c973a57a0 20 read_log 194215'12895 (194215'12894) delete 572b779a/benchmark_data_storage0111-ib.pocstor_41526_object5678/head//24 by client.2174595.0:190541 0.000000 2014-08-13 18:06:06.963762 7f5c973a57a0 20 read_log 194215'12896 (192888'12561) clean 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by osd.325.0:2420 2014-08-12 15:50:27.284514 2014-08-13 18:06:06.963785 7f5c973a57a0 20 read_log 194215'12897 (194215'12896) delete 2fd1879a/benchmark_data_storage0111-ib.pocstor_41097_object24141/head//24 by client.2174595.0:190543 0.000000 2014-08-13 18:06:06.963796 7f5c973a57a0 20 read_log 194215'12898 (192834'11827) clean 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by osd.325.0:2422 2014-08-12 15:50:27.300500 2014-08-13 18:06:06.963808 7f5c973a57a0 20 read_log 194215'12899 (194215'12898) delete 83d3879a/benchmark_data_storage0111-ib.pocstor_26659_object13200/head//24 by client.2174595.0:190545 0.000000 2014-08-13 18:06:06.963819 7f5c973a57a0 20 read_log 194215'12900 (192888'12547) clean 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by osd.325.0:2424 2014-08-12 15:50:27.327363 2014-08-13 18:06:06.963832 7f5c973a57a0 20 read_log 194215'12901 (194215'12900) delete 6e04879a/benchmark_data_storage0111-ib.pocstor_41097_object18121/head//24 by client.2174595.0:190547 0.000000 2014-08-13 18:06:06.963842 7f5c973a57a0 20 read_log 194215'12902 (192888'12709) clean 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by osd.325.0:2426 2014-08-12 15:50:27.342003 2014-08-13 18:06:06.963855 7f5c973a57a0 20 read_log 194215'12903 (194215'12902) delete 4b75879a/benchmark_data_storage0111-ib.pocstor_2228_object23266/head//24 by client.2174595.0:190549 0.000000 2014-08-13 18:06:06.963866 7f5c973a57a0 20 read_log 194215'12904 (192834'11826) clean 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by osd.325.0:2428 2014-08-12 15:50:27.357282 2014-08-13 18:06:06.963878 7f5c973a57a0 20 read_log 194215'12905 (194215'12904) delete 4226979a/benchmark_data_storage0111-ib.pocstor_26659_object13192/head//24 by client.2174595.0:190551 0.000000 2014-08-13 18:06:06.963889 7f5c973a57a0 20 read_log 194215'12906 (193305'12789) clean e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by osd.325.0:2430 2014-08-12 15:50:27.373123 2014-08-13 18:06:06.963903 7f5c973a57a0 20 read_log 194215'12907 (194215'12906) delete e1af979a/benchmark_data_storage0111-ib.pocstor_62621_object27864/head//24 by client.2174595.0:190553 0.000000 2014-08-13 18:06:06.963914 7f5c973a57a0 20 read_log 194215'12908 (192888'12530) clean 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by osd.325.0:2432 2014-08-12 15:50:27.387897 2014-08-13 18:06:06.963929 7f5c973a57a0 20 read_log 194215'12909 (194215'12908) delete 7f01b79a/benchmark_data_storage0111-ib.pocstor_41097_object12504/head//24 by client.2174595.0:190555 0.000000 2014-08-13 18:06:06.963947 7f5c973a57a0 20 read_log 194215'12910 (192834'11791) clean 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by osd.325.0:2434 2014-08-12 15:50:27.401528 2014-08-13 18:06:06.963959 7f5c973a57a0 20 read_log 194215'12911 (194215'12910) delete 4502c79a/benchmark_data_storage0111-ib.pocstor_26659_object9008/head//24 by client.2174595.0:190557 0.000000 2014-08-13 18:06:06.963970 7f5c973a57a0 20 read_log 194215'12912 (192888'12543) clean f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by osd.325.0:2436 2014-08-12 15:50:27.416233 2014-08-13 18:06:06.963982 7f5c973a57a0 20 read_log 194215'12913 (194215'12912) delete f24ac79a/benchmark_data_storage0111-ib.pocstor_41097_object17319/head//24 by client.2174595.0:190559 0.000000 2014-08-13 18:06:06.963993 7f5c973a57a0 20 read_log 194215'12914 (192888'12698) clean 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by osd.325.0:2438 2014-08-12 15:50:27.428697 2014-08-13 18:06:06.964005 7f5c973a57a0 20 read_log 194215'12915 (194215'12914) delete 20c3e79a/benchmark_data_storage0111-ib.pocstor_2228_object15179/head//24 by client.2174595.0:190561 0.000000 2014-08-13 18:06:06.964016 7f5c973a57a0 20 read_log 194215'12916 (192888'12551) clean 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by osd.325.0:2440 2014-08-12 15:50:27.442500 2014-08-13 18:06:06.964028 7f5c973a57a0 20 read_log 194215'12917 (194215'12916) delete 9086e79a/benchmark_data_storage0111-ib.pocstor_41097_object20394/head//24 by client.2174595.0:190563 0.000000 2014-08-13 18:06:06.964039 7f5c973a57a0 20 read_log 194215'12918 (192834'12221) clean 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by osd.325.0:2442 2014-08-12 15:50:27.470283 2014-08-13 18:06:06.964051 7f5c973a57a0 20 read_log 194215'12919 (194215'12918) delete 3970f79a/benchmark_data_storage0111-ib.pocstor_41526_object1628/head//24 by client.2174595.0:190565 0.000000 2014-08-13 18:06:06.964062 7f5c973a57a0 20 read_log 194215'12920 (192834'12217) clean 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by osd.325.0:2444 2014-08-12 15:50:27.485493 2014-08-13 18:06:06.964074 7f5c973a57a0 20 read_log 194215'12921 (194215'12920) delete 2ffef79a/benchmark_data_storage0111-ib.pocstor_41526_object1234/head//24 by client.2174595.0:190567 0.000000 2014-08-13 18:06:06.964085 7f5c973a57a0 20 read_log 194215'12922 (192888'12509) clean f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by osd.325.0:2446 2014-08-12 15:50:27.498601 2014-08-13 18:06:06.964097 7f5c973a57a0 20 read_log 194215'12923 (194215'12922) delete f60b9a/benchmark_data_storage0111-ib.pocstor_41097_object1074/head//24 by client.2174595.0:190569 0.000000 2014-08-13 18:06:06.964108 7f5c973a57a0 20 read_log 194215'12924 (192834'12242) clean 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by osd.325.0:2448 2014-08-12 15:50:27.512695 2014-08-13 18:06:06.964120 7f5c973a57a0 20 read_log 194215'12925 (194215'12924) delete 915a0b9a/benchmark_data_storage0111-ib.pocstor_41526_object5053/head//24 by client.2174595.0:190571 0.000000 2014-08-13 18:06:06.964131 7f5c973a57a0 20 read_log 194215'12926 (192834'11843) clean 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by osd.325.0:2450 2014-08-12 15:50:27.526648 2014-08-13 18:06:06.964167 7f5c973a57a0 20 read_log 194215'12927 (194215'12926) delete 3ddc0b9a/benchmark_data_storage0111-ib.pocstor_26659_object15480/head//24 by client.2174595.0:190573 0.000000 2014-08-13 18:06:06.964182 7f5c973a57a0 20 read_log 194215'12928 (193305'12779) clean c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by osd.325.0:2452 2014-08-12 15:50:27.540862 2014-08-13 18:06:06.964195 7f5c973a57a0 20 read_log 194215'12929 (194215'12928) delete c701b9a/benchmark_data_storage0111-ib.pocstor_62621_object12276/head//24 by client.2174595.0:190575 0.000000 2014-08-13 18:06:06.964213 7f5c973a57a0 20 read_log 194215'12930 (192888'12548) clean 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by osd.325.0:2454 2014-08-12 15:50:27.555235 2014-08-13 18:06:06.964229 7f5c973a57a0 20 read_log 194215'12931 (194215'12930) delete 80e01b9a/benchmark_data_storage0111-ib.pocstor_41097_object18128/head//24 by client.2174595.0:190577 0.000000 2014-08-13 18:06:06.964240 7f5c973a57a0 20 read_log 194215'12932 (192888'12546) clean e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by osd.325.0:2456 2014-08-12 15:50:27.569053 2014-08-13 18:06:06.964252 7f5c973a57a0 20 read_log 194215'12933 (194215'12932) delete e2c11b9a/benchmark_data_storage0111-ib.pocstor_41097_object17890/head//24 by client.2174595.0:190579 0.000000 2014-08-13 18:06:06.964266 7f5c973a57a0 20 read_log 194215'12934 (192888'12681) clean 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by osd.325.0:2458 2014-08-12 15:50:27.582850 2014-08-13 18:06:06.964279 7f5c973a57a0 20 read_log 194215'12935 (194215'12934) delete 9581b9a/benchmark_data_storage0111-ib.pocstor_2228_object5052/head//24 by client.2174595.0:190581 0.000000 2014-08-13 18:06:06.964290 7f5c973a57a0 20 read_log 194215'12936 (192888'12687) clean ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by osd.325.0:2460 2014-08-12 15:50:27.606056 2014-08-13 18:06:06.964302 7f5c973a57a0 20 read_log 194215'12937 (194215'12936) delete ef912b9a/benchmark_data_storage0111-ib.pocstor_2228_object7598/head//24 by client.2174595.0:190583 0.000000 2014-08-13 18:06:06.964315 7f5c973a57a0 20 read_log 194215'12938 (192888'12701) clean 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by osd.325.0:2462 2014-08-12 15:50:27.619264 2014-08-13 18:06:06.964328 7f5c973a57a0 20 read_log 194215'12939 (194215'12938) delete 10422b9a/benchmark_data_storage0111-ib.pocstor_2228_object16096/head//24 by client.2174595.0:190585 0.000000 2014-08-13 18:06:06.964339 7f5c973a57a0 20 read_log 194215'12940 (192888'12505) clean a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by osd.325.0:2464 2014-08-12 15:50:27.633354 2014-08-13 18:06:06.964352 7f5c973a57a0 20 read_log 194215'12941 (194215'12940) delete a1343b9a/benchmark_data_storage0111-ib.pocstor_41097_object115/head//24 by client.2174595.0:190587 0.000000 2014-08-13 18:06:06.964366 7f5c973a57a0 20 read_log 194215'12942 (192888'12690) clean 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by osd.325.0:2466 2014-08-12 15:50:27.649008 2014-08-13 18:06:06.964379 7f5c973a57a0 20 read_log 194215'12943 (194215'12942) delete 30ed3b9a/benchmark_data_storage0111-ib.pocstor_2228_object10440/head//24 by client.2174595.0:190589 0.000000 2014-08-13 18:06:06.964390 7f5c973a57a0 20 read_log 194215'12944 (192834'11808) clean 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by osd.325.0:2468 2014-08-12 15:50:27.665935 2014-08-13 18:06:06.964402 7f5c973a57a0 20 read_log 194215'12945 (194215'12944) delete 6524b9a/benchmark_data_storage0111-ib.pocstor_26659_object10914/head//24 by client.2174595.0:190591 0.000000 2014-08-13 18:06:06.964416 7f5c973a57a0 20 read_log 194215'12946 (192888'12567) clean 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by osd.325.0:2470 2014-08-12 15:50:27.681452 2014-08-13 18:06:06.964431 7f5c973a57a0 20 read_log 194215'12947 (194215'12946) delete 601a4b9a/benchmark_data_storage0111-ib.pocstor_41097_object27761/head//24 by client.2174595.0:190593 0.000000 2014-08-13 18:06:06.964454 7f5c973a57a0 20 read_log 194215'12948 (192834'12349) clean bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by osd.325.0:2472 2014-08-12 15:50:27.694175 2014-08-13 18:06:06.964467 7f5c973a57a0 20 read_log 194215'12949 (194215'12948) delete bd105b9a/benchmark_data_storage0111-ib.pocstor_41526_object20747/head//24 by client.2174595.0:190595 0.000000 2014-08-13 18:06:06.964498 7f5c973a57a0 20 read_log 194215'12950 (192834'11841) clean 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by osd.325.0:2474 2014-08-12 15:50:27.708527 2014-08-13 18:06:06.964515 7f5c973a57a0 20 read_log 194215'12951 (194215'12950) delete 12725b9a/benchmark_data_storage0111-ib.pocstor_26659_object15312/head//24 by client.2174595.0:190597 0.000000 2014-08-13 18:06:06.964526 7f5c973a57a0 20 read_log 194215'12952 (192888'12550) clean e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by osd.325.0:2476 2014-08-12 15:50:27.722426 2014-08-13 18:06:06.964548 7f5c973a57a0 20 read_log 194215'12953 (194215'12952) delete e0065b9a/benchmark_data_storage0111-ib.pocstor_41097_object20108/head//24 by client.2174595.0:190599 0.000000 2014-08-13 18:06:06.964559 7f5c973a57a0 20 read_log 194215'12954 (192888'12559) clean b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by osd.325.0:2478 2014-08-12 15:50:27.737703 2014-08-13 18:06:06.964572 7f5c973a57a0 20 read_log 194215'12955 (194215'12954) delete b3c96b9a/benchmark_data_storage0111-ib.pocstor_41097_object23341/head//24 by client.2174595.0:190601 0.000000 2014-08-13 18:06:06.964583 7f5c973a57a0 20 read_log 194215'12956 (192834'11873) clean b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by osd.325.0:2480 2014-08-12 15:50:27.752253 2014-08-13 18:06:06.964601 7f5c973a57a0 20 read_log 194215'12957 (194215'12956) delete b03e6b9a/benchmark_data_storage0111-ib.pocstor_26659_object18663/head//24 by client.2174595.0:190603 0.000000 2014-08-13 18:06:06.964615 7f5c973a57a0 20 read_log 194215'12958 (192888'12544) clean dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by osd.325.0:2482 2014-08-12 15:50:27.766988 2014-08-13 18:06:06.964652 7f5c973a57a0 20 read_log 194215'12959 (194215'12958) delete dd48b9a/benchmark_data_storage0111-ib.pocstor_41097_object17670/head//24 by client.2174595.0:190605 0.000000 2014-08-13 18:06:06.964683 7f5c973a57a0 20 read_log 194215'12960 (193305'12782) clean f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by osd.325.0:2484 2014-08-12 15:50:27.779954 2014-08-13 18:06:06.964716 7f5c973a57a0 20 read_log 194215'12961 (194215'12960) delete f34e9b9a/benchmark_data_storage0111-ib.pocstor_62621_object13603/head//24 by client.2174595.0:190607 0.000000 2014-08-13 18:06:06.964761 7f5c973a57a0 20 read_log 194215'12962 (192834'11731) clean 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by osd.325.0:2486 2014-08-12 15:50:27.793538 2014-08-13 18:06:06.964779 7f5c973a57a0 20 read_log 194215'12963 (194215'12962) delete 12ef9b9a/benchmark_data_storage0111-ib.pocstor_26659_object1299/head//24 by client.2174595.0:190609 0.000000 2014-08-13 18:06:06.964863 7f5c973a57a0 20 read_log 194215'12964 (192834'12271) clean 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by osd.325.0:2488 2014-08-12 15:50:27.808252 2014-08-13 18:06:06.964918 7f5c973a57a0 20 read_log 194215'12965 (194215'12964) delete 58a2bb9a/benchmark_data_storage0111-ib.pocstor_41526_object9693/head//24 by client.2174595.0:190611 0.000000 2014-08-13 18:06:06.965060 7f5c973a57a0 20 read_log 194215'12966 (193305'12771) clean fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by osd.325.0:2490 2014-08-12 15:50:27.822572 2014-08-13 18:06:06.965435 7f5c973a57a0 20 read_log 194215'12967 (194215'12966) delete fa39bb9a/benchmark_data_storage0111-ib.pocstor_62621_object2973/head//24 by client.2174595.0:190613 0.000000 2014-08-13 18:06:06.965473 7f5c973a57a0 20 read_log 194215'12968 (193305'12787) clean feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by osd.325.0:2492 2014-08-12 15:50:27.835093 2014-08-13 18:06:06.965496 7f5c973a57a0 20 read_log 194215'12969 (194215'12968) delete feeabb9a/benchmark_data_storage0111-ib.pocstor_62621_object23471/head//24 by client.2174595.0:190615 0.000000 2014-08-13 18:06:06.965547 7f5c973a57a0 20 read_log 194215'12970 (192834'12232) clean 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by osd.325.0:2494 2014-08-12 15:50:27.848752 2014-08-13 18:06:06.965601 7f5c973a57a0 20 read_log 194215'12971 (194215'12970) delete 29dcbb9a/benchmark_data_storage0111-ib.pocstor_41526_object3042/head//24 by client.2174595.0:190617 0.000000 2014-08-13 18:06:06.965640 7f5c973a57a0 20 read_log 194215'12972 (192834'12340) clean 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by osd.325.0:2496 2014-08-12 15:50:27.862019 2014-08-13 18:06:06.965656 7f5c973a57a0 20 read_log 194215'12973 (194215'12972) delete 34d8cb9a/benchmark_data_storage0111-ib.pocstor_41526_object19354/head//24 by client.2174595.0:190619 0.000000 2014-08-13 18:06:06.965667 7f5c973a57a0 20 read_log 194215'12974 (193305'12775) clean 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by osd.325.0:2498 2014-08-12 15:50:27.875423 2014-08-13 18:06:06.965680 7f5c973a57a0 20 read_log 194215'12975 (194215'12974) delete 53bddb9a/benchmark_data_storage0111-ib.pocstor_62621_object5648/head//24 by client.2174595.0:190621 0.000000 2014-08-13 18:06:06.965697 7f5c973a57a0 20 read_log 194215'12976 (192888'12542) clean 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by osd.325.0:2500 2014-08-12 15:50:27.888418 2014-08-13 18:06:06.965724 7f5c973a57a0 20 read_log 194215'12977 (194215'12976) delete 52d0eb9a/benchmark_data_storage0111-ib.pocstor_41097_object16897/head//24 by client.2174595.0:190623 0.000000 2014-08-13 18:06:06.965748 7f5c973a57a0 20 read_log 194215'12978 (192834'12335) clean fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by osd.325.0:2502 2014-08-12 15:50:27.904153 2014-08-13 18:06:06.965764 7f5c973a57a0 20 read_log 194215'12979 (194215'12978) delete fbe0fb9a/benchmark_data_storage0111-ib.pocstor_41526_object18249/head//24 by client.2174595.0:190625 0.000000 2014-08-13 18:06:06.965776 7f5c973a57a0 20 read_log 194215'12980 (192834'12277) clean 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by osd.325.0:2504 2014-08-12 15:50:27.918318 2014-08-13 18:06:06.965788 7f5c973a57a0 20 read_log 194215'12981 (194215'12980) delete 9c21fb9a/benchmark_data_storage0111-ib.pocstor_41526_object10551/head//24 by client.2174595.0:190627 0.000000 2014-08-13 18:06:06.965808 7f5c973a57a0 20 read_log 194215'12982 (193305'12788) clean f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by osd.325.0:2506 2014-08-12 15:50:27.933199 2014-08-13 18:06:06.965825 7f5c973a57a0 20 read_log 194215'12983 (194215'12982) delete f672fb9a/benchmark_data_storage0111-ib.pocstor_62621_object25216/head//24 by client.2174595.0:190629 0.000000 2014-08-13 18:06:06.965837 7f5c973a57a0 20 read_log 194215'12984 (192834'11834) clean 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by osd.325.0:2508 2014-08-12 15:50:27.945894 2014-08-13 18:06:06.965849 7f5c973a57a0 20 read_log 194215'12985 (194215'12984) delete 5ad3fb9a/benchmark_data_storage0111-ib.pocstor_26659_object14283/head//24 by client.2174595.0:190631 0.000000 2014-08-13 18:06:06.965881 7f5c973a57a0 20 read_log 194215'12986 (192888'12520) clean c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by osd.325.0:2510 2014-08-12 15:50:27.959341 2014-08-13 18:06:06.965897 7f5c973a57a0 20 read_log 194215'12987 (194215'12986) delete c9c9fb9a/benchmark_data_storage0111-ib.pocstor_41097_object8234/head//24 by client.2174595.0:190633 0.000000 2014-08-13 18:06:06.965908 7f5c973a57a0 20 read_log 194215'12988 (192834'12365) clean 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by osd.325.0:2512 2014-08-12 15:50:27.973469 2014-08-13 18:06:06.965921 7f5c973a57a0 20 read_log 194215'12989 (194215'12988) delete 164bfb9a/benchmark_data_storage0111-ib.pocstor_41526_object23269/head//24 by client.2174595.0:190635 0.000000 2014-08-13 18:06:06.965960 7f5c973a57a0 20 read_log 194215'12990 (192888'12711) clean 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by osd.325.0:2514 2014-08-12 15:50:27.986035 2014-08-13 18:06:06.965977 7f5c973a57a0 20 read_log 194215'12991 (194215'12990) delete 93ca0f9a/benchmark_data_storage0111-ib.pocstor_2228_object23837/head//24 by client.2174595.0:190637 0.000000 2014-08-13 18:06:06.965988 7f5c973a57a0 20 read_log 194215'12992 (192888'12571) clean 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by osd.325.0:2516 2014-08-12 15:50:28.000475 2014-08-13 18:06:06.966000 7f5c973a57a0 20 read_log 194215'12993 (194215'12992) delete 1d2c0f9a/benchmark_data_storage0111-ib.pocstor_41097_object30533/head//24 by client.2174595.0:190639 0.000000 2014-08-13 18:06:06.966011 7f5c973a57a0 20 read_log 194215'12994 (192834'12278) clean 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by osd.325.0:2518 2014-08-12 15:50:28.016226 2014-08-13 18:06:06.966023 7f5c973a57a0 20 read_log 194215'12995 (194215'12994) delete 23021f9a/benchmark_data_storage0111-ib.pocstor_41526_object10562/head//24 by client.2174595.0:190641 0.000000 2014-08-13 18:06:06.966034 7f5c973a57a0 20 read_log 194215'12996 (192834'12351) clean af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by osd.325.0:2520 2014-08-12 15:50:28.023934 2014-08-13 18:06:06.966047 7f5c973a57a0 20 read_log 194215'12997 (194215'12996) delete af321f9a/benchmark_data_storage0111-ib.pocstor_41526_object20822/head//24 by client.2174595.0:190643 0.000000 2014-08-13 18:06:06.966058 7f5c973a57a0 20 read_log 194215'12998 (192888'12680) clean d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by osd.325.0:2522 2014-08-12 15:50:28.032742 2014-08-13 18:06:06.966070 7f5c973a57a0 20 read_log 194215'12999 (194215'12998) delete d0b61f9a/benchmark_data_storage0111-ib.pocstor_2228_object4627/head//24 by client.2174595.0:190645 0.000000 2014-08-13 18:06:06.966081 7f5c973a57a0 20 read_log 194215'13000 (193305'12786) clean 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by osd.325.0:2524 2014-08-12 15:50:28.042228 2014-08-13 18:06:06.966094 7f5c973a57a0 20 read_log 194215'13001 (194215'13000) delete 71d91f9a/benchmark_data_storage0111-ib.pocstor_62621_object23055/head//24 by client.2174595.0:190647 0.000000 2014-08-13 18:06:06.966104 7f5c973a57a0 20 read_log 194215'13002 (193305'12784) clean b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by osd.325.0:2526 2014-08-12 15:50:28.050922 2014-08-13 18:06:06.966117 7f5c973a57a0 20 read_log 194215'13003 (194215'13002) delete b4a32f9a/benchmark_data_storage0111-ib.pocstor_62621_object22453/head//24 by client.2174595.0:190649 0.000000 2014-08-13 18:06:06.966128 7f5c973a57a0 20 read_log 194215'13004 (192834'12317) clean 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by osd.325.0:2528 2014-08-12 15:50:28.058746 2014-08-13 18:06:06.966140 7f5c973a57a0 20 read_log 194215'13005 (194215'13004) delete 7e742f9a/benchmark_data_storage0111-ib.pocstor_41526_object15050/head//24 by client.2174595.0:190651 0.000000 2014-08-13 18:06:06.966156 7f5c973a57a0 20 read_log 194215'13006 (192888'12504) clean 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by osd.325.0:2530 2014-08-12 15:50:28.066870 2014-08-13 18:06:06.966172 7f5c973a57a0 20 read_log 194215'13007 (194215'13006) delete 57292f9a/benchmark_data_storage0111-ib.pocstor_41097_object110/head//24 by client.2174595.0:190653 0.000000 2014-08-13 18:06:06.966183 7f5c973a57a0 20 read_log 194215'13008 (192888'12569) clean 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by osd.325.0:2532 2014-08-12 15:50:28.075322 2014-08-13 18:06:06.966196 7f5c973a57a0 20 read_log 194215'13009 (194215'13008) delete 9c5a2f9a/benchmark_data_storage0111-ib.pocstor_41097_object28688/head//24 by client.2174595.0:190655 0.000000 2014-08-13 18:06:06.966217 7f5c973a57a0 20 read_log 194215'13010 (193305'12772) clean 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by osd.325.0:2534 2014-08-12 15:50:28.084124 2014-08-13 18:06:06.966232 7f5c973a57a0 20 read_log 194215'13011 (194215'13010) delete 5ec73f9a/benchmark_data_storage0111-ib.pocstor_62621_object3953/head//24 by client.2174595.0:190657 0.000000 2014-08-13 18:06:06.966244 7f5c973a57a0 20 read_log 194215'13012 (192888'12708) clean 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by osd.325.0:2536 2014-08-12 15:50:28.093704 2014-08-13 18:06:06.966256 7f5c973a57a0 20 read_log 194215'13013 (194215'13012) delete 84f9a/benchmark_data_storage0111-ib.pocstor_2228_object22096/head//24 by client.2174595.0:190659 0.000000 2014-08-13 18:06:06.966270 7f5c973a57a0 20 read_log 194215'13014 (192834'12364) clean f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by osd.325.0:2538 2014-08-12 15:50:28.105822 2014-08-13 18:06:06.966282 7f5c973a57a0 20 read_log 194215'13015 (194215'13014) delete f3c35f9a/benchmark_data_storage0111-ib.pocstor_41526_object23262/head//24 by client.2174595.0:190661 0.000000 2014-08-13 18:06:06.966293 7f5c973a57a0 20 read_log 194215'13016 (192834'11806) clean 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by osd.325.0:2540 2014-08-12 15:50:28.117598 2014-08-13 18:06:06.966306 7f5c973a57a0 20 read_log 194215'13017 (194215'13016) delete 42ea5f9a/benchmark_data_storage0111-ib.pocstor_26659_object10862/head//24 by client.2174595.0:190663 0.000000 2014-08-13 18:06:06.966317 7f5c973a57a0 20 read_log 194215'13018 (192888'12693) clean 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by osd.325.0:2542 2014-08-12 15:50:28.130743 2014-08-13 18:06:06.966329 7f5c973a57a0 20 read_log 194215'13019 (194215'13018) delete 27936f9a/benchmark_data_storage0111-ib.pocstor_2228_object12660/head//24 by client.2174595.0:190665 0.000000 2014-08-13 18:06:06.966362 7f5c973a57a0 20 read_log 194215'13020 (192834'11796) clean 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by osd.325.0:2544 2014-08-12 15:50:28.142807 2014-08-13 18:06:06.966379 7f5c973a57a0 20 read_log 194215'13021 (194215'13020) delete 4eb6f9a/benchmark_data_storage0111-ib.pocstor_26659_object9645/head//24 by client.2174595.0:190667 0.000000 2014-08-13 18:06:06.966399 7f5c973a57a0 20 read_log 194215'13022 (192888'12541) clean 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by osd.325.0:2546 2014-08-12 15:50:28.157510 2014-08-13 18:06:06.966433 7f5c973a57a0 20 read_log 194215'13023 (194215'13022) delete 59cc6f9a/benchmark_data_storage0111-ib.pocstor_41097_object16591/head//24 by client.2174595.0:190669 0.000000 2014-08-13 18:06:06.966465 7f5c973a57a0 20 read_log 194215'13024 (192888'12676) clean 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by osd.325.0:2548 2014-08-12 15:50:28.171217 2014-08-13 18:06:06.966479 7f5c973a57a0 20 read_log 194215'13025 (194215'13024) delete 46e87f9a/benchmark_data_storage0111-ib.pocstor_2228_object3210/head//24 by client.2174595.0:190671 0.000000 2014-08-13 18:06:06.966498 7f5c973a57a0 20 read_log 194215'13026 (192834'12253) clean a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by osd.325.0:2550 2014-08-12 15:50:28.186252 2014-08-13 18:06:06.966514 7f5c973a57a0 20 read_log 194215'13027 (194215'13026) delete a76b7f9a/benchmark_data_storage0111-ib.pocstor_41526_object7088/head//24 by client.2174595.0:190673 0.000000 2014-08-13 18:06:06.966525 7f5c973a57a0 20 read_log 194215'13028 (192834'11757) clean 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by osd.325.0:2552 2014-08-12 15:50:28.199716 2014-08-13 18:06:06.966538 7f5c973a57a0 20 read_log 194215'13029 (194215'13028) delete 9ca38f9a/benchmark_data_storage0111-ib.pocstor_26659_object4097/head//24 by client.2174595.0:190675 0.000000 2014-08-13 18:06:06.966559 7f5c973a57a0 20 read_log 194215'13030 (192888'12695) clean 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by osd.325.0:2554 2014-08-12 15:50:28.212691 2014-08-13 18:06:06.966574 7f5c973a57a0 20 read_log 194215'13031 (194215'13030) delete 9bb78f9a/benchmark_data_storage0111-ib.pocstor_2228_object13513/head//24 by client.2174595.0:190677 0.000000 2014-08-13 18:06:06.966585 7f5c973a57a0 20 read_log 194215'13032 (193305'12785) clean b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by osd.325.0:2556 2014-08-12 15:50:28.227014 2014-08-13 18:06:06.966597 7f5c973a57a0 20 read_log 194215'13033 (194215'13032) delete b3e49f9a/benchmark_data_storage0111-ib.pocstor_62621_object22726/head//24 by client.2174595.0:190679 0.000000 2014-08-13 18:06:06.966608 7f5c973a57a0 20 read_log 194215'13034 (192888'12525) clean 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by osd.325.0:2558 2014-08-12 15:50:28.240289 2014-08-13 18:06:06.966621 7f5c973a57a0 20 read_log 194215'13035 (194215'13034) delete 3d799f9a/benchmark_data_storage0111-ib.pocstor_41097_object10364/head//24 by client.2174595.0:190681 0.000000 2014-08-13 18:06:06.966631 7f5c973a57a0 20 read_log 194215'13036 (192888'12699) clean e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by osd.325.0:2560 2014-08-12 15:50:28.253618 2014-08-13 18:06:06.966652 7f5c973a57a0 20 read_log 194215'13037 (194215'13036) delete e30f9f9a/benchmark_data_storage0111-ib.pocstor_2228_object15600/head//24 by client.2174595.0:190683 0.000000 2014-08-13 18:06:06.966663 7f5c973a57a0 20 read_log 194215'13038 (192888'12713) clean c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by osd.325.0:2562 2014-08-12 15:50:28.269951 2014-08-13 18:06:06.966676 7f5c973a57a0 20 read_log 194215'13039 (194215'13038) delete c197af9a/benchmark_data_storage0111-ib.pocstor_2228_object25541/head//24 by client.2174595.0:190685 0.000000 2014-08-13 18:06:06.966687 7f5c973a57a0 20 read_log 194215'13040 (192888'12563) clean 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by osd.325.0:2564 2014-08-12 15:50:28.282727 2014-08-13 18:06:06.966699 7f5c973a57a0 20 read_log 194215'13041 (194215'13040) delete 782baf9a/benchmark_data_storage0111-ib.pocstor_41097_object24659/head//24 by client.2174595.0:190687 0.000000 2014-08-13 18:06:06.966710 7f5c973a57a0 20 read_log 194215'13042 (192888'12686) clean 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by osd.325.0:2566 2014-08-12 15:50:28.296106 2014-08-13 18:06:06.966723 7f5c973a57a0 20 read_log 194215'13043 (194215'13042) delete 85c1cf9a/benchmark_data_storage0111-ib.pocstor_2228_object7152/head//24 by client.2174595.0:190689 0.000000 2014-08-13 18:06:06.966734 7f5c973a57a0 20 read_log 194215'13044 (192888'12704) clean ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by osd.325.0:2568 2014-08-12 15:50:28.308436 2014-08-13 18:06:06.966746 7f5c973a57a0 20 read_log 194215'13045 (194215'13044) delete ad8adf9a/benchmark_data_storage0111-ib.pocstor_2228_object19980/head//24 by client.2174595.0:190691 0.000000 2014-08-13 18:06:06.966757 7f5c973a57a0 20 read_log 194215'13046 (193305'12777) clean 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by osd.325.0:2570 2014-08-12 15:50:28.322080 2014-08-13 18:06:06.966769 7f5c973a57a0 20 read_log 194215'13047 (194215'13046) delete 2b9cdf9a/benchmark_data_storage0111-ib.pocstor_62621_object9237/head//24 by client.2174595.0:190693 0.000000 2014-08-13 18:06:06.966789 7f5c973a57a0 20 read_log 194215'13048 (192888'12706) clean 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by osd.325.0:2572 2014-08-12 15:50:28.336072 2014-08-13 18:06:06.966802 7f5c973a57a0 20 read_log 194215'13049 (194215'13048) delete 6278ff9a/benchmark_data_storage0111-ib.pocstor_2228_object20953/head//24 by client.2174595.0:190695 0.000000 2014-08-13 18:06:06.966834 7f5c973a57a0 20 read_log 194215'13050 (192834'11770) clean 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by osd.325.0:2574 2014-08-12 15:50:28.349354 2014-08-13 18:06:06.966847 7f5c973a57a0 20 read_log 194215'13051 (194215'13050) delete 8b6cff9a/benchmark_data_storage0111-ib.pocstor_26659_object5999/head//24 by client.2174595.0:190697 0.000000 2014-08-13 18:06:06.966861 7f5c973a57a0 20 read_log 194215'13052 (192888'12710) clean fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by osd.325.0:2576 2014-08-12 15:50:28.363662 2014-08-13 18:06:06.966874 7f5c973a57a0 20 read_log 194215'13053 (194215'13052) delete fd8fff9a/benchmark_data_storage0111-ib.pocstor_2228_object23781/head//24 by client.2174595.0:190699 0.000000 2014-08-13 18:06:06.966885 7f5c973a57a0 20 read_log 194370'13054 (0'0) promote 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by osd.324.0:4372 2014-08-12 20:30:17.653172 2014-08-13 18:06:06.966897 7f5c973a57a0 20 read_log 194370'13055 (194370'13054) modify 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by client.1508531.0:869 2014-08-12 20:30:17.645600 2014-08-13 18:06:06.966913 7f5c973a57a0 20 read_log 194370'13056 (0'0) promote fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by osd.324.0:4416 2014-08-12 20:30:25.651001 2014-08-13 18:06:06.966925 7f5c973a57a0 20 read_log 194370'13057 (194370'13056) modify fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by client.1508531.0:3028 2014-08-12 20:30:25.643346 2014-08-13 18:06:06.966937 7f5c973a57a0 20 read_log 194370'13058 (0'0) promote e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by osd.324.0:4440 2014-08-12 20:30:29.416303 2014-08-13 18:06:06.966950 7f5c973a57a0 20 read_log 194370'13059 (194370'13058) modify e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by client.1508531.0:4031 2014-08-12 20:30:29.406321 2014-08-13 18:06:06.966962 7f5c973a57a0 20 read_log 194370'13060 (0'0) promote b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by osd.324.0:4450 2014-08-12 20:30:30.210042 2014-08-13 18:06:06.966974 7f5c973a57a0 20 read_log 194370'13061 (194370'13060) modify b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by client.1508531.0:4231 2014-08-12 20:30:30.202760 2014-08-13 18:06:06.966987 7f5c973a57a0 20 read_log 194370'13062 (0'0) promote 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by osd.324.0:4460 2014-08-12 20:30:32.802646 2014-08-13 18:06:06.967007 7f5c973a57a0 20 read_log 194370'13063 (194370'13062) modify 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by client.1508531.0:4908 2014-08-12 20:30:32.794881 2014-08-13 18:06:06.967021 7f5c973a57a0 20 read_log 194370'13064 (0'0) promote 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by osd.324.0:4488 2014-08-12 20:30:38.782573 2014-08-13 18:06:06.967033 7f5c973a57a0 20 read_log 194370'13065 (194370'13064) modify 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by client.1508531.0:6231 2014-08-12 20:30:38.774709 2014-08-13 18:06:06.967045 7f5c973a57a0 20 read_log 194370'13066 (0'0) promote fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by osd.324.0:4498 2014-08-12 20:30:39.800147 2014-08-13 18:06:06.967058 7f5c973a57a0 20 read_log 194370'13067 (194370'13066) modify fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by client.1508531.0:6465 2014-08-12 20:30:39.791322 2014-08-13 18:06:06.967072 7f5c973a57a0 20 read_log 194370'13068 (0'0) promote 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by osd.324.0:4502 2014-08-12 20:30:40.839066 2014-08-13 18:06:06.967084 7f5c973a57a0 20 read_log 194370'13069 (194370'13068) modify 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by client.1508531.0:6688 2014-08-12 20:30:40.829729 2014-08-13 18:06:06.967106 7f5c973a57a0 20 read_log 194370'13070 (0'0) promote dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by osd.324.0:4536 2014-08-12 20:30:48.419090 2014-08-13 18:06:06.967119 7f5c973a57a0 20 read_log 194370'13071 (194370'13070) modify dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by client.1508531.0:8156 2014-08-12 20:30:48.410034 2014-08-13 18:06:06.967133 7f5c973a57a0 20 read_log 194370'13072 (0'0) promote c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by osd.324.0:4562 2014-08-12 20:30:57.167267 2014-08-13 18:06:06.967145 7f5c973a57a0 20 read_log 194370'13073 (194370'13072) modify c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by client.1508531.0:9643 2014-08-12 20:30:57.159265 2014-08-13 18:06:06.967166 7f5c973a57a0 20 read_log 194370'13074 (0'0) promote 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by osd.324.0:4654 2014-08-12 20:31:19.085945 2014-08-13 18:06:06.967179 7f5c973a57a0 20 read_log 194370'13075 (194370'13074) modify 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by client.1508531.0:12970 2014-08-12 20:31:19.077634 2014-08-13 18:06:06.967191 7f5c973a57a0 20 read_log 194370'13076 (0'0) promote e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by osd.324.0:4670 2014-08-12 20:31:26.769589 2014-08-13 18:06:06.967203 7f5c973a57a0 20 read_log 194370'13077 (194370'13076) modify e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by client.1508531.0:13763 2014-08-12 20:31:26.761274 2014-08-13 18:06:06.967216 7f5c973a57a0 20 read_log 194377'13078 (0'0) promote d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by osd.13.0:2811 2014-08-12 20:33:03.110943 2014-08-13 18:06:06.967228 7f5c973a57a0 20 read_log 194377'13079 (194377'13078) modify d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by client.2231190.0:1476 2014-08-12 20:33:03.102034 2014-08-13 18:06:06.967243 7f5c973a57a0 20 read_log 194377'13080 (0'0) promote 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by osd.13.0:2823 2014-08-12 20:33:06.459017 2014-08-13 18:06:06.967256 7f5c973a57a0 20 read_log 194377'13081 (194377'13080) modify 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by client.2231190.0:2299 2014-08-12 20:33:06.453284 2014-08-13 18:06:06.967268 7f5c973a57a0 20 read_log 194377'13082 (0'0) promote 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by osd.13.0:2825 2014-08-12 20:33:06.747346 2014-08-13 18:06:06.967280 7f5c973a57a0 20 read_log 194377'13083 (194377'13082) modify 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by client.2231190.0:2377 2014-08-12 20:33:06.741091 2014-08-13 18:06:06.967298 7f5c973a57a0 20 read_log 194377'13084 (0'0) promote b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by osd.13.0:2855 2014-08-12 20:33:17.041923 2014-08-13 18:06:06.967313 7f5c973a57a0 20 read_log 194377'13085 (194377'13084) modify b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by client.2231190.0:4484 2014-08-12 20:33:17.034718 2014-08-13 18:06:06.967326 7f5c973a57a0 20 read_log 194380'13086 (0'0) promote 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by osd.13.0:2861 2014-08-12 20:33:25.607982 2014-08-13 18:06:06.967338 7f5c973a57a0 20 read_log 194380'13087 (194380'13086) modify 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by client.2231190.0:5187 2014-08-12 20:33:25.601065 2014-08-13 18:06:06.967350 7f5c973a57a0 20 read_log 194383'13088 (0'0) promote d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by osd.325.0:7 2014-08-12 20:34:04.303075 2014-08-13 18:06:06.967377 7f5c973a57a0 20 read_log 194383'13089 (194383'13088) modify d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by client.2231190.0:6498 2014-08-12 20:34:04.278600 2014-08-13 18:06:06.967393 7f5c973a57a0 20 read_log 194383'13090 (0'0) promote fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by osd.325.0:19 2014-08-12 20:34:19.084519 2014-08-13 18:06:06.967406 7f5c973a57a0 20 read_log 194383'13091 (194383'13090) modify fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by client.2231190.0:7146 2014-08-12 20:34:19.077512 2014-08-13 18:06:06.967429 7f5c973a57a0 20 read_log 194383'13092 (0'0) promote 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by osd.325.0:21 2014-08-12 20:34:23.787632 2014-08-13 18:06:06.967442 7f5c973a57a0 20 read_log 194383'13093 (194383'13092) modify 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by client.2231190.0:7317 2014-08-12 20:34:23.779474 2014-08-13 18:06:06.967455 7f5c973a57a0 20 read_log 194416'13094 (0'0) promote 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by osd.395.0:4258 2014-08-13 15:28:44.079719 2014-08-13 18:06:06.967469 7f5c973a57a0 20 read_log 194416'13095 (194416'13094) modify 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by client.2441958.0:3528 2014-08-13 15:28:44.070814 2014-08-13 18:06:06.967491 7f5c973a57a0 20 read_log 194416'13096 (0'0) promote 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by osd.395.0:4278 2014-08-13 15:28:51.121397 2014-08-13 18:06:06.967504 7f5c973a57a0 20 read_log 194416'13097 (194416'13096) modify 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by client.2441958.0:5321 2014-08-13 15:28:51.114390 2014-08-13 18:06:06.967516 7f5c973a57a0 20 read_log 194416'13098 (0'0) promote 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by osd.395.0:4294 2014-08-13 15:28:56.070304 2014-08-13 18:06:06.967537 7f5c973a57a0 20 read_log 194416'13099 (194416'13098) modify 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by client.2441958.0:6554 2014-08-13 15:28:56.060052 2014-08-13 18:06:06.967553 7f5c973a57a0 20 read_log 194416'13100 (0'0) promote 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by osd.395.0:4310 2014-08-13 15:28:58.542579 2014-08-13 18:06:06.967566 7f5c973a57a0 20 read_log 194416'13101 (194416'13100) modify 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by client.2441958.0:7204 2014-08-13 15:28:58.533818 2014-08-13 18:06:06.967579 7f5c973a57a0 20 read_log 194416'13102 (0'0) promote c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by osd.395.0:4400 2014-08-13 15:29:19.354688 2014-08-13 18:06:06.967591 7f5c973a57a0 20 read_log 194416'13103 (194416'13102) modify c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by client.2441958.0:12410 2014-08-13 15:29:19.345491 2014-08-13 18:06:06.967603 7f5c973a57a0 20 read_log 194416'13104 (0'0) promote 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by osd.395.0:4434 2014-08-13 15:29:28.500047 2014-08-13 18:06:06.967616 7f5c973a57a0 20 read_log 194416'13105 (194416'13104) modify 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by client.2441958.0:14574 2014-08-13 15:29:28.490784 2014-08-13 18:06:06.967630 7f5c973a57a0 20 read_log 194416'13106 (0'0) promote b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by osd.395.0:4436 2014-08-13 15:29:29.693888 2014-08-13 18:06:06.967649 7f5c973a57a0 20 read_log 194416'13107 (194416'13106) modify b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by client.2441958.0:14864 2014-08-13 15:29:29.685386 2014-08-13 18:06:06.967671 7f5c973a57a0 20 read_log 194416'13108 (0'0) promote 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by osd.395.0:4444 2014-08-13 15:29:33.200498 2014-08-13 18:06:06.967692 7f5c973a57a0 20 read_log 194416'13109 (194416'13108) modify 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by client.2441958.0:15592 2014-08-13 15:29:33.188173 2014-08-13 18:06:06.967707 7f5c973a57a0 20 read_log 194416'13110 (0'0) promote f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by osd.395.0:4450 2014-08-13 15:29:48.002028 2014-08-13 18:06:06.967720 7f5c973a57a0 20 read_log 194416'13111 (194416'13110) modify f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by client.2441958.0:17046 2014-08-13 15:29:47.991603 2014-08-13 18:06:06.967732 7f5c973a57a0 20 read_log 194416'13112 (0'0) promote 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by osd.395.0:4468 2014-08-13 15:30:11.258275 2014-08-13 18:06:06.967776 7f5c973a57a0 20 read_log 194416'13113 (194416'13112) modify 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by client.2441958.0:18433 2014-08-13 15:30:11.249320 2014-08-13 18:06:06.967793 7f5c973a57a0 20 read_log 194416'13114 (0'0) promote 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by osd.395.0:4470 2014-08-13 15:30:12.135566 2014-08-13 18:06:06.967805 7f5c973a57a0 20 read_log 194416'13115 (194416'13114) modify 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by client.2441958.0:18464 2014-08-13 15:30:12.126663 2014-08-13 18:06:06.967821 7f5c973a57a0 20 read_log 194422'13116 (0'0) promote cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by osd.395.0:4480 2014-08-13 15:32:16.353064 2014-08-13 18:06:06.967837 7f5c973a57a0 20 read_log 194422'13117 (194422'13116) modify cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by client.1148203.0:295 2014-08-13 15:32:16.344464 2014-08-13 18:06:06.967850 7f5c973a57a0 20 read_log 194422'13118 (0'0) promote 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by osd.395.0:4484 2014-08-13 15:32:18.824551 2014-08-13 18:06:06.967862 7f5c973a57a0 20 read_log 194422'13119 (194422'13118) modify 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by client.1148203.0:874 2014-08-13 15:32:18.813722 2014-08-13 18:06:06.967875 7f5c973a57a0 20 read_log 194422'13120 (0'0) promote c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by osd.395.0:4498 2014-08-13 15:32:22.257929 2014-08-13 18:06:06.967887 7f5c973a57a0 20 read_log 194422'13121 (194422'13120) modify c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by client.1148203.0:1750 2014-08-13 15:32:22.249403 2014-08-13 18:06:06.967903 7f5c973a57a0 20 read_log 194422'13122 (0'0) promote de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by osd.395.0:4514 2014-08-13 15:32:25.466808 2014-08-13 18:06:06.967915 7f5c973a57a0 20 read_log 194422'13123 (194422'13122) modify de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by client.1148203.0:2585 2014-08-13 15:32:25.460245 2014-08-13 18:06:06.967928 7f5c973a57a0 20 read_log 194422'13124 (0'0) promote 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by osd.395.0:4516 2014-08-13 15:32:25.932710 2014-08-13 18:06:06.967940 7f5c973a57a0 20 read_log 194422'13125 (194422'13124) modify 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by client.1148203.0:2710 2014-08-13 15:32:25.923689 2014-08-13 18:06:06.967953 7f5c973a57a0 20 read_log 194422'13126 (0'0) promote 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by osd.395.0:4524 2014-08-13 15:32:27.674331 2014-08-13 18:06:06.967965 7f5c973a57a0 20 read_log 194422'13127 (194422'13126) modify 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by client.1148203.0:3151 2014-08-13 15:32:27.666329 2014-08-13 18:06:06.967977 7f5c973a57a0 20 read_log 194422'13128 (0'0) promote 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by osd.395.0:4540 2014-08-13 15:32:31.055137 2014-08-13 18:06:06.968012 7f5c973a57a0 20 read_log 194422'13129 (194422'13128) modify 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by client.1148203.0:4016 2014-08-13 15:32:31.046978 2014-08-13 18:06:06.968028 7f5c973a57a0 20 read_log 194422'13130 (0'0) promote ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by osd.395.0:4554 2014-08-13 15:32:35.380206 2014-08-13 18:06:06.968041 7f5c973a57a0 20 read_log 194422'13131 (194422'13130) modify ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by client.1148203.0:5110 2014-08-13 15:32:35.372868 2014-08-13 18:06:06.968053 7f5c973a57a0 20 read_log 194422'13132 (0'0) promote 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by osd.395.0:4556 2014-08-13 15:32:36.140636 2014-08-13 18:06:06.968065 7f5c973a57a0 20 read_log 194422'13133 (194422'13132) modify 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by client.1148203.0:5259 2014-08-13 15:32:36.130829 2014-08-13 18:06:06.968078 7f5c973a57a0 20 read_log 194422'13134 (0'0) promote a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by osd.395.0:4558 2014-08-13 15:32:37.003814 2014-08-13 18:06:06.968090 7f5c973a57a0 20 read_log 194422'13135 (194422'13134) modify a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by client.1148203.0:5434 2014-08-13 15:32:36.992775 2014-08-13 18:06:06.968103 7f5c973a57a0 20 read_log 194422'13136 (0'0) promote bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by osd.395.0:4568 2014-08-13 15:32:38.746016 2014-08-13 18:06:06.968115 7f5c973a57a0 20 read_log 194422'13137 (194422'13136) modify bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by client.1148203.0:5737 2014-08-13 15:32:38.736377 2014-08-13 18:06:06.968127 7f5c973a57a0 20 read_log 194422'13138 (0'0) promote bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by osd.395.0:4574 2014-08-13 15:32:39.973211 2014-08-13 18:06:06.968140 7f5c973a57a0 20 read_log 194422'13139 (194422'13138) modify bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by client.1148203.0:5971 2014-08-13 15:32:39.963483 2014-08-13 18:06:06.968158 7f5c973a57a0 20 read_log 194422'13140 (0'0) promote 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by osd.395.0:4588 2014-08-13 15:32:47.138728 2014-08-13 18:06:06.968174 7f5c973a57a0 20 read_log 194422'13141 (194422'13140) modify 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by client.1148203.0:7265 2014-08-13 15:32:47.129206 2014-08-13 18:06:06.968186 7f5c973a57a0 20 read_log 194422'13142 (0'0) promote d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by osd.395.0:4602 2014-08-13 15:32:50.373573 2014-08-13 18:06:06.968200 7f5c973a57a0 20 read_log 194422'13143 (194422'13142) modify d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by client.1148203.0:7782 2014-08-13 15:32:50.364448 2014-08-13 18:06:06.968213 7f5c973a57a0 20 read_log 194422'13144 (0'0) promote 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by osd.395.0:4612 2014-08-13 15:32:54.349970 2014-08-13 18:06:06.968225 7f5c973a57a0 20 read_log 194422'13145 (194422'13144) modify 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by client.1148203.0:8400 2014-08-13 15:32:54.340862 2014-08-13 18:06:06.968240 7f5c973a57a0 20 read_log 194422'13146 (0'0) promote b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by osd.395.0:4618 2014-08-13 15:32:55.512494 2014-08-13 18:06:06.968253 7f5c973a57a0 20 read_log 194422'13147 (194422'13146) modify b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by client.1148203.0:8556 2014-08-13 15:32:55.498245 2014-08-13 18:06:06.968265 7f5c973a57a0 20 read_log 194422'13148 (0'0) promote c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by osd.395.0:4626 2014-08-13 15:32:56.934578 2014-08-13 18:06:06.968284 7f5c973a57a0 20 read_log 194422'13149 (194422'13148) modify c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by client.1148203.0:8728 2014-08-13 15:32:56.925677 2014-08-13 18:06:06.968300 7f5c973a57a0 20 read_log 194422'13150 (0'0) promote 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by osd.395.0:4632 2014-08-13 15:32:57.924797 2014-08-13 18:06:06.968313 7f5c973a57a0 20 read_log 194422'13151 (194422'13150) modify 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by client.1148203.0:8832 2014-08-13 15:32:57.916210 2014-08-13 18:06:06.968329 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.968732 7f5c973a57a0 10 read_log checking for missing items over interval (194370'13054,194422'13151] 2014-08-13 18:06:06.968804 7f5c973a57a0 15 read_log missing 194422'13151 (194422'13150) modify 1e32c39a/benchmark_data_storage0111-ib.pocstor_41231_object8831/head//24 by client.1148203.0:8832 2014-08-13 15:32:57.916210 2014-08-13 18:06:06.968846 7f5c973a57a0 15 read_log missing 194422'13149 (194422'13148) modify c4fec79a/benchmark_data_storage0111-ib.pocstor_41231_object8727/head//24 by client.1148203.0:8728 2014-08-13 15:32:56.925677 2014-08-13 18:06:06.968882 7f5c973a57a0 15 read_log missing 194422'13147 (194422'13146) modify b1569f9a/benchmark_data_storage0111-ib.pocstor_41231_object8555/head//24 by client.1148203.0:8556 2014-08-13 15:32:55.498245 2014-08-13 18:06:06.968918 7f5c973a57a0 15 read_log missing 194422'13145 (194422'13144) modify 595a939a/benchmark_data_storage0111-ib.pocstor_41231_object8399/head//24 by client.1148203.0:8400 2014-08-13 15:32:54.340862 2014-08-13 18:06:06.968953 7f5c973a57a0 15 read_log missing 194422'13143 (194422'13142) modify d80c739a/benchmark_data_storage0111-ib.pocstor_41231_object7781/head//24 by client.1148203.0:7782 2014-08-13 15:32:50.364448 2014-08-13 18:06:06.968989 7f5c973a57a0 15 read_log missing 194422'13141 (194422'13140) modify 344bdf9a/benchmark_data_storage0111-ib.pocstor_41231_object7264/head//24 by client.1148203.0:7265 2014-08-13 15:32:47.129206 2014-08-13 18:06:06.969023 7f5c973a57a0 15 read_log missing 194422'13139 (194422'13138) modify bc9b679a/benchmark_data_storage0111-ib.pocstor_41231_object5970/head//24 by client.1148203.0:5971 2014-08-13 15:32:39.963483 2014-08-13 18:06:06.969059 7f5c973a57a0 15 read_log missing 194422'13137 (194422'13136) modify bebd379a/benchmark_data_storage0111-ib.pocstor_41231_object5736/head//24 by client.1148203.0:5737 2014-08-13 15:32:38.736377 2014-08-13 18:06:06.969094 7f5c973a57a0 15 read_log missing 194422'13135 (194422'13134) modify a217a39a/benchmark_data_storage0111-ib.pocstor_41231_object5433/head//24 by client.1148203.0:5434 2014-08-13 15:32:36.992775 2014-08-13 18:06:06.969129 7f5c973a57a0 15 read_log missing 194422'13133 (194422'13132) modify 1e4d939a/benchmark_data_storage0111-ib.pocstor_41231_object5258/head//24 by client.1148203.0:5259 2014-08-13 15:32:36.130829 2014-08-13 18:06:06.969171 7f5c973a57a0 15 read_log missing 194422'13131 (194422'13130) modify ecf979a/benchmark_data_storage0111-ib.pocstor_41231_object5109/head//24 by client.1148203.0:5110 2014-08-13 15:32:35.372868 2014-08-13 18:06:06.969208 7f5c973a57a0 15 read_log missing 194422'13129 (194422'13128) modify 63ec4b9a/benchmark_data_storage0111-ib.pocstor_41231_object4015/head//24 by client.1148203.0:4016 2014-08-13 15:32:31.046978 2014-08-13 18:06:06.969248 7f5c973a57a0 15 read_log missing 194422'13127 (194422'13126) modify 871ebb9a/benchmark_data_storage0111-ib.pocstor_41231_object3150/head//24 by client.1148203.0:3151 2014-08-13 15:32:27.666329 2014-08-13 18:06:06.969288 7f5c973a57a0 15 read_log missing 194422'13125 (194422'13124) modify 6de48b9a/benchmark_data_storage0111-ib.pocstor_41231_object2709/head//24 by client.1148203.0:2710 2014-08-13 15:32:25.923689 2014-08-13 18:06:06.969325 7f5c973a57a0 15 read_log missing 194422'13123 (194422'13122) modify de45ab9a/benchmark_data_storage0111-ib.pocstor_41231_object2584/head//24 by client.1148203.0:2585 2014-08-13 15:32:25.460245 2014-08-13 18:06:06.969372 7f5c973a57a0 15 read_log missing 194422'13121 (194422'13120) modify c26ddb9a/benchmark_data_storage0111-ib.pocstor_41231_object1749/head//24 by client.1148203.0:1750 2014-08-13 15:32:22.249403 2014-08-13 18:06:06.969421 7f5c973a57a0 15 read_log missing 194422'13119 (194422'13118) modify 409e779a/benchmark_data_storage0111-ib.pocstor_41231_object873/head//24 by client.1148203.0:874 2014-08-13 15:32:18.813722 2014-08-13 18:06:06.969460 7f5c973a57a0 15 read_log missing 194422'13117 (194422'13116) modify cc3d3b9a/benchmark_data_storage0111-ib.pocstor_41231_object294/head//24 by client.1148203.0:295 2014-08-13 15:32:16.344464 2014-08-13 18:06:06.969496 7f5c973a57a0 15 read_log missing 194416'13115 (194416'13114) modify 124b4b9a/benchmark_data_storage0111-ib.pocstor_19103_object18463/head//24 by client.2441958.0:18464 2014-08-13 15:30:12.126663 2014-08-13 18:06:06.969533 7f5c973a57a0 15 read_log missing 194416'13113 (194416'13112) modify 740d5f9a/benchmark_data_storage0111-ib.pocstor_19103_object18432/head//24 by client.2441958.0:18433 2014-08-13 15:30:11.249320 2014-08-13 18:06:06.969568 7f5c973a57a0 15 read_log missing 194416'13111 (194416'13110) modify f6b2db9a/benchmark_data_storage0111-ib.pocstor_19103_object17045/head//24 by client.2441958.0:17046 2014-08-13 15:29:47.991603 2014-08-13 18:06:06.969604 7f5c973a57a0 15 read_log missing 194416'13109 (194416'13108) modify 7a91879a/benchmark_data_storage0111-ib.pocstor_19103_object15591/head//24 by client.2441958.0:15592 2014-08-13 15:29:33.188173 2014-08-13 18:06:06.969639 7f5c973a57a0 15 read_log missing 194416'13107 (194416'13106) modify b16e739a/benchmark_data_storage0111-ib.pocstor_19103_object14863/head//24 by client.2441958.0:14864 2014-08-13 15:29:29.685386 2014-08-13 18:06:06.969674 7f5c973a57a0 15 read_log missing 194416'13105 (194416'13104) modify 7285d39a/benchmark_data_storage0111-ib.pocstor_19103_object14573/head//24 by client.2441958.0:14574 2014-08-13 15:29:28.490784 2014-08-13 18:06:06.969710 7f5c973a57a0 15 read_log missing 194416'13103 (194416'13102) modify c840cb9a/benchmark_data_storage0111-ib.pocstor_19103_object12409/head//24 by client.2441958.0:12410 2014-08-13 15:29:19.345491 2014-08-13 18:06:06.969746 7f5c973a57a0 15 read_log missing 194416'13101 (194416'13100) modify 3ab6db9a/benchmark_data_storage0111-ib.pocstor_19103_object7203/head//24 by client.2441958.0:7204 2014-08-13 15:28:58.533818 2014-08-13 18:06:06.969782 7f5c973a57a0 15 read_log missing 194416'13099 (194416'13098) modify 619739a/benchmark_data_storage0111-ib.pocstor_19103_object6553/head//24 by client.2441958.0:6554 2014-08-13 15:28:56.060052 2014-08-13 18:06:06.969818 7f5c973a57a0 15 read_log missing 194416'13097 (194416'13096) modify 3efdcf9a/benchmark_data_storage0111-ib.pocstor_19103_object5320/head//24 by client.2441958.0:5321 2014-08-13 15:28:51.114390 2014-08-13 18:06:06.969876 7f5c973a57a0 15 read_log missing 194416'13095 (194416'13094) modify 42a75f9a/benchmark_data_storage0111-ib.pocstor_19103_object3527/head//24 by client.2441958.0:3528 2014-08-13 15:28:44.070814 2014-08-13 18:06:06.969914 7f5c973a57a0 15 read_log missing 194383'13093 (194383'13092) modify 63a09b9a/benchmark_data_storage0111-ib.pocstor_25065_object7316/head//24 by client.2231190.0:7317 2014-08-12 20:34:23.779474 2014-08-13 18:06:06.969950 7f5c973a57a0 15 read_log missing 194383'13091 (194383'13090) modify fa47739a/benchmark_data_storage0111-ib.pocstor_25065_object7145/head//24 by client.2231190.0:7146 2014-08-12 20:34:19.077512 2014-08-13 18:06:06.969986 7f5c973a57a0 15 read_log missing 194383'13089 (194383'13088) modify d917279a/benchmark_data_storage0111-ib.pocstor_25065_object6497/head//24 by client.2231190.0:6498 2014-08-12 20:34:04.278600 2014-08-13 18:06:06.970022 7f5c973a57a0 15 read_log missing 194380'13087 (194380'13086) modify 5ab3af9a/benchmark_data_storage0111-ib.pocstor_25065_object5186/head//24 by client.2231190.0:5187 2014-08-12 20:33:25.601065 2014-08-13 18:06:06.970057 7f5c973a57a0 15 read_log missing 194377'13085 (194377'13084) modify b1d85f9a/benchmark_data_storage0111-ib.pocstor_25065_object4483/head//24 by client.2231190.0:4484 2014-08-12 20:33:17.034718 2014-08-13 18:06:06.970093 7f5c973a57a0 15 read_log missing 194377'13083 (194377'13082) modify 3cb8679a/benchmark_data_storage0111-ib.pocstor_25065_object2376/head//24 by client.2231190.0:2377 2014-08-12 20:33:06.741091 2014-08-13 18:06:06.970129 7f5c973a57a0 15 read_log missing 194377'13081 (194377'13080) modify 2947df9a/benchmark_data_storage0111-ib.pocstor_25065_object2298/head//24 by client.2231190.0:2299 2014-08-12 20:33:06.453284 2014-08-13 18:06:06.970171 7f5c973a57a0 15 read_log missing 194377'13079 (194377'13078) modify d0e11b9a/benchmark_data_storage0111-ib.pocstor_25065_object1475/head//24 by client.2231190.0:1476 2014-08-12 20:33:03.102034 2014-08-13 18:06:06.970207 7f5c973a57a0 15 read_log missing 194370'13077 (194370'13076) modify e1afe79a/benchmark_data_storage0111-ib.pocstor_12495_object13762/head//24 by client.1508531.0:13763 2014-08-12 20:31:26.761274 2014-08-13 18:06:06.970247 7f5c973a57a0 15 read_log missing 194370'13075 (194370'13074) modify 91cc539a/benchmark_data_storage0111-ib.pocstor_12495_object12969/head//24 by client.1508531.0:12970 2014-08-12 20:31:19.077634 2014-08-13 18:06:06.970288 7f5c973a57a0 15 read_log missing 194370'13073 (194370'13072) modify c13bcb9a/benchmark_data_storage0111-ib.pocstor_12495_object9642/head//24 by client.1508531.0:9643 2014-08-12 20:30:57.159265 2014-08-13 18:06:06.970325 7f5c973a57a0 15 read_log missing 194370'13071 (194370'13070) modify dd21f39a/benchmark_data_storage0111-ib.pocstor_12495_object8155/head//24 by client.1508531.0:8156 2014-08-12 20:30:48.410034 2014-08-13 18:06:06.970371 7f5c973a57a0 15 read_log missing 194370'13069 (194370'13068) modify 587f079a/benchmark_data_storage0111-ib.pocstor_12495_object6687/head//24 by client.1508531.0:6688 2014-08-12 20:30:40.829729 2014-08-13 18:06:06.970418 7f5c973a57a0 15 read_log missing 194370'13067 (194370'13066) modify fcc9579a/benchmark_data_storage0111-ib.pocstor_12495_object6464/head//24 by client.1508531.0:6465 2014-08-12 20:30:39.791322 2014-08-13 18:06:06.970457 7f5c973a57a0 15 read_log missing 194370'13065 (194370'13064) modify 38f43f9a/benchmark_data_storage0111-ib.pocstor_12495_object6230/head//24 by client.1508531.0:6231 2014-08-12 20:30:38.774709 2014-08-13 18:06:06.970493 7f5c973a57a0 15 read_log missing 194370'13063 (194370'13062) modify 2313ab9a/benchmark_data_storage0111-ib.pocstor_12495_object4907/head//24 by client.1508531.0:4908 2014-08-12 20:30:32.794881 2014-08-13 18:06:06.970529 7f5c973a57a0 15 read_log missing 194370'13061 (194370'13060) modify b45e7b9a/benchmark_data_storage0111-ib.pocstor_12495_object4230/head//24 by client.1508531.0:4231 2014-08-12 20:30:30.202760 2014-08-13 18:06:06.970565 7f5c973a57a0 15 read_log missing 194370'13059 (194370'13058) modify e0f7379a/benchmark_data_storage0111-ib.pocstor_12495_object4030/head//24 by client.1508531.0:4031 2014-08-12 20:30:29.406321 2014-08-13 18:06:06.970602 7f5c973a57a0 15 read_log missing 194370'13057 (194370'13056) modify fe6f2b9a/benchmark_data_storage0111-ib.pocstor_12495_object3027/head//24 by client.1508531.0:3028 2014-08-12 20:30:25.643346 2014-08-13 18:06:06.970637 7f5c973a57a0 15 read_log missing 194370'13055 (194370'13054) modify 45b5d79a/benchmark_data_storage0111-ib.pocstor_12495_object868/head//24 by client.1508531.0:869 2014-08-12 20:30:17.645600 2014-08-13 18:06:06.970648 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.970678 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive m=49] handle_loaded 2014-08-13 18:06:06.970696 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Initial 0.014786 0 0.000000 2014-08-13 18:06:06.970708 7f5c973a57a0 5 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Reset 2014-08-13 18:06:06.970717 7f5c973a57a0 20 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=0 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194470 2014-08-13 18:06:06.970727 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] log((192718'10533,194422'13151], crt=194422'13151) 2014-08-13 18:06:06.970738 7f5c973a57a0 10 osd.4 194487 pgid 24.3b7 coll 24.3b7_head 2014-08-13 18:06:06.970873 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.3b7 2014-08-13 18:06:06.970886 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.970919 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3b7(unlocked)] enter Initial 2014-08-13 18:06:06.970931 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3b7(unlocked)] enter NotTrimming 2014-08-13 18:06:06.971076 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.971217 7f5c973a57a0 20 read_log 192718'15727 (192718'15726) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:280 2014-08-06 15:23:12.592898 2014-08-13 18:06:06.971242 7f5c973a57a0 20 read_log 192718'15728 (192718'15727) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:280 2014-08-06 15:23:12.620061 2014-08-13 18:06:06.971256 7f5c973a57a0 20 read_log 192718'15729 (192718'15728) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:281 2014-08-06 15:23:12.680961 2014-08-13 18:06:06.971270 7f5c973a57a0 20 read_log 192718'15730 (192718'15729) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:281 2014-08-06 15:23:12.706703 2014-08-13 18:06:06.971282 7f5c973a57a0 20 read_log 192718'15731 (192718'15730) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:282 2014-08-06 15:23:12.748671 2014-08-13 18:06:06.971299 7f5c973a57a0 20 read_log 192718'15732 (192718'15731) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:282 2014-08-06 15:23:12.766485 2014-08-13 18:06:06.971322 7f5c973a57a0 20 read_log 192718'15733 (192718'15732) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:283 2014-08-06 15:23:12.797400 2014-08-13 18:06:06.971337 7f5c973a57a0 20 read_log 192718'15734 (192718'15733) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:283 2014-08-06 15:23:12.823267 2014-08-13 18:06:06.971361 7f5c973a57a0 20 read_log 192718'15735 (192718'15734) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:284 2014-08-06 15:23:12.852654 2014-08-13 18:06:06.971378 7f5c973a57a0 20 read_log 192718'15736 (192718'15735) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:284 2014-08-06 15:23:12.879426 2014-08-13 18:06:06.971391 7f5c973a57a0 20 read_log 192718'15737 (192718'15736) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:285 2014-08-06 15:23:12.926570 2014-08-13 18:06:06.971403 7f5c973a57a0 20 read_log 192718'15738 (192718'15737) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:285 2014-08-06 15:23:12.934988 2014-08-13 18:06:06.971415 7f5c973a57a0 20 read_log 192718'15739 (192718'15738) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:286 2014-08-06 15:23:12.964785 2014-08-13 18:06:06.971427 7f5c973a57a0 20 read_log 192718'15740 (192718'15739) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:286 2014-08-06 15:23:12.990135 2014-08-13 18:06:06.971439 7f5c973a57a0 20 read_log 192718'15741 (192718'15740) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:287 2014-08-06 15:23:13.023338 2014-08-13 18:06:06.971460 7f5c973a57a0 20 read_log 192718'15742 (192718'15741) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:287 2014-08-06 15:23:13.218211 2014-08-13 18:06:06.971472 7f5c973a57a0 20 read_log 192718'15743 (192718'15742) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:288 2014-08-06 15:23:13.248739 2014-08-13 18:06:06.971485 7f5c973a57a0 20 read_log 192718'15744 (192718'15743) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:288 2014-08-06 15:23:13.277080 2014-08-13 18:06:06.971497 7f5c973a57a0 20 read_log 192718'15745 (192718'15744) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:289 2014-08-06 15:23:13.308129 2014-08-13 18:06:06.971509 7f5c973a57a0 20 read_log 192718'15746 (192718'15745) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:289 2014-08-06 15:23:13.334989 2014-08-13 18:06:06.971521 7f5c973a57a0 20 read_log 192718'15747 (192718'15746) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:290 2014-08-06 15:23:13.367015 2014-08-13 18:06:06.971533 7f5c973a57a0 20 read_log 192718'15748 (192718'15747) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:290 2014-08-06 15:23:13.385246 2014-08-13 18:06:06.971545 7f5c973a57a0 20 read_log 192718'15749 (192718'15748) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:291 2014-08-06 15:23:13.416462 2014-08-13 18:06:06.971558 7f5c973a57a0 20 read_log 192718'15750 (192718'15749) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:291 2014-08-06 15:23:13.438750 2014-08-13 18:06:06.971570 7f5c973a57a0 20 read_log 192718'15751 (192718'15750) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:292 2014-08-06 15:23:13.478822 2014-08-13 18:06:06.971583 7f5c973a57a0 20 read_log 192718'15752 (192718'15751) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:292 2014-08-06 15:23:13.632401 2014-08-13 18:06:06.971595 7f5c973a57a0 20 read_log 192718'15753 (192718'15752) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:293 2014-08-06 15:23:13.661490 2014-08-13 18:06:06.971607 7f5c973a57a0 20 read_log 192718'15754 (192718'15753) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:293 2014-08-06 15:23:13.686672 2014-08-13 18:06:06.971619 7f5c973a57a0 20 read_log 192718'15755 (192718'15754) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:294 2014-08-06 15:23:13.715513 2014-08-13 18:06:06.971631 7f5c973a57a0 20 read_log 192718'15756 (192718'15755) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:294 2014-08-06 15:23:13.741236 2014-08-13 18:06:06.971652 7f5c973a57a0 20 read_log 192718'15757 (192718'15756) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:295 2014-08-06 15:23:13.778692 2014-08-13 18:06:06.971668 7f5c973a57a0 20 read_log 192718'15758 (192718'15757) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:295 2014-08-06 15:23:13.795316 2014-08-13 18:06:06.971680 7f5c973a57a0 20 read_log 192718'15759 (192718'15758) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:296 2014-08-06 15:23:13.827873 2014-08-13 18:06:06.971692 7f5c973a57a0 20 read_log 192718'15760 (192718'15759) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:296 2014-08-06 15:23:13.853232 2014-08-13 18:06:06.971706 7f5c973a57a0 20 read_log 192718'15761 (192718'15760) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:297 2014-08-06 15:23:13.895526 2014-08-13 18:06:06.971718 7f5c973a57a0 20 read_log 192718'15762 (192718'15761) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:297 2014-08-06 15:23:14.039796 2014-08-13 18:06:06.971732 7f5c973a57a0 20 read_log 192718'15763 (192718'15762) modify 9148e3b7/core.50654+12+2/head//24 by client.592113.0:298 2014-08-06 15:23:14.063721 2014-08-13 18:06:06.971745 7f5c973a57a0 20 read_log 192718'15764 (192718'15763) modify 9148e3b7/core.50654+12+2/head//24 by client.416131.0:298 2014-08-06 15:23:14.089812 2014-08-13 18:06:06.971758 7f5c973a57a0 20 read_log 192718'15865 (192718'15558) delete a59f37b7/core.10872+97+2/head//24 by osd.317.0:17237 2014-08-06 15:31:07.383440 2014-08-13 18:06:06.971772 7f5c973a57a0 20 read_log 192718'16433 (192718'15764) clean 9148e3b7/core.50654+12+2/head//24 by osd.317.0:17806 2014-08-06 15:48:41.860477 2014-08-13 18:06:06.971784 7f5c973a57a0 20 read_log 192798'16436 (192718'16433) delete 9148e3b7/core.50654+12+2/head//24 by osd.317.0:18127 2014-08-11 10:24:43.624903 2014-08-13 18:06:06.971799 7f5c973a57a0 20 read_log 192803'16440 (192718'14158) delete 339f5bb7/usr.tar3+38+2/head//24 by osd.317.0:18131 2014-08-11 10:30:28.665138 2014-08-13 18:06:06.971812 7f5c973a57a0 20 read_log 192834'17023 (0'0) promote f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.317.0:19065 2014-08-11 15:23:40.530492 2014-08-13 18:06:06.971825 7f5c973a57a0 20 read_log 192834'17024 (192834'17023) modify f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by client.933448.0:1255 2014-08-11 15:23:40.522528 2014-08-13 18:06:06.971840 7f5c973a57a0 20 read_log 192834'17039 (0'0) promote 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.317.0:19081 2014-08-11 15:23:48.797531 2014-08-13 18:06:06.971852 7f5c973a57a0 20 read_log 192834'17040 (192834'17039) modify 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by client.933448.0:2834 2014-08-11 15:23:48.788658 2014-08-13 18:06:06.971867 7f5c973a57a0 20 read_log 192834'17055 (0'0) promote f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by osd.317.0:19097 2014-08-11 15:23:55.217781 2014-08-13 18:06:06.971879 7f5c973a57a0 20 read_log 192834'17056 (192834'17055) modify f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.933448.0:4043 2014-08-11 15:23:55.208817 2014-08-13 18:06:06.971893 7f5c973a57a0 20 read_log 192834'17097 (0'0) promote 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by osd.317.0:19139 2014-08-11 15:24:09.574408 2014-08-13 18:06:06.971923 7f5c973a57a0 20 read_log 192834'17098 (192834'17097) modify 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.933448.0:7310 2014-08-11 15:24:09.566590 2014-08-13 18:06:06.971939 7f5c973a57a0 20 read_log 192834'17155 (0'0) promote 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.317.0:19197 2014-08-11 15:24:26.675065 2014-08-13 18:06:06.971952 7f5c973a57a0 20 read_log 192834'17156 (192834'17155) modify 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by client.933448.0:11163 2014-08-11 15:24:26.667210 2014-08-13 18:06:06.971977 7f5c973a57a0 20 read_log 192834'17165 (0'0) promote b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by osd.317.0:19207 2014-08-11 15:24:32.317027 2014-08-13 18:06:06.971992 7f5c973a57a0 20 read_log 192834'17166 (192834'17165) modify b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.933448.0:12429 2014-08-11 15:24:32.310013 2014-08-13 18:06:06.972005 7f5c973a57a0 20 read_log 192834'17181 (0'0) promote 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by osd.317.0:19223 2014-08-11 15:24:34.594620 2014-08-13 18:06:06.972019 7f5c973a57a0 20 read_log 192834'17182 (192834'17181) modify 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.933448.0:12957 2014-08-11 15:24:34.583709 2014-08-13 18:06:06.972032 7f5c973a57a0 20 read_log 192834'17185 (0'0) promote 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by osd.317.0:19227 2014-08-11 15:24:35.733848 2014-08-13 18:06:06.972046 7f5c973a57a0 20 read_log 192834'17186 (192834'17185) modify 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.933448.0:13175 2014-08-11 15:24:35.724085 2014-08-13 18:06:06.972059 7f5c973a57a0 20 read_log 192834'17195 (0'0) promote 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by osd.317.0:19237 2014-08-11 15:24:40.101207 2014-08-13 18:06:06.972073 7f5c973a57a0 20 read_log 192834'17196 (192834'17195) modify 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.933448.0:13826 2014-08-11 15:24:40.092251 2014-08-13 18:06:06.972088 7f5c973a57a0 20 read_log 192834'17211 (0'0) promote abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.317.0:19253 2014-08-11 15:24:45.960382 2014-08-13 18:06:06.972100 7f5c973a57a0 20 read_log 192834'17212 (192834'17211) modify abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by client.933448.0:14515 2014-08-11 15:24:45.951967 2014-08-13 18:06:06.972112 7f5c973a57a0 20 read_log 192834'17219 (0'0) promote 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by osd.317.0:19261 2014-08-11 15:24:49.736097 2014-08-13 18:06:06.972125 7f5c973a57a0 20 read_log 192834'17220 (192834'17219) modify 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.933448.0:15458 2014-08-11 15:24:49.722546 2014-08-13 18:06:06.972139 7f5c973a57a0 20 read_log 192834'17231 (0'0) promote 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.317.0:19273 2014-08-11 15:24:53.670689 2014-08-13 18:06:06.972160 7f5c973a57a0 20 read_log 192834'17232 (192834'17231) modify 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by client.933448.0:16239 2014-08-11 15:24:53.662692 2014-08-13 18:06:06.972175 7f5c973a57a0 20 read_log 192834'17241 (0'0) promote 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by osd.317.0:19283 2014-08-11 15:24:56.539699 2014-08-13 18:06:06.972188 7f5c973a57a0 20 read_log 192834'17242 (192834'17241) modify 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.933448.0:16643 2014-08-11 15:24:56.531915 2014-08-13 18:06:06.972200 7f5c973a57a0 20 read_log 192834'17255 (0'0) promote 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.317.0:19297 2014-08-11 15:24:59.371890 2014-08-13 18:06:06.972212 7f5c973a57a0 20 read_log 192834'17256 (192834'17255) modify 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by client.933448.0:17350 2014-08-11 15:24:59.364431 2014-08-13 18:06:06.972227 7f5c973a57a0 20 read_log 192834'17259 (0'0) promote 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.317.0:19301 2014-08-11 15:25:01.082938 2014-08-13 18:06:06.972242 7f5c973a57a0 20 read_log 192834'17260 (192834'17259) modify 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by client.933448.0:17698 2014-08-11 15:25:01.074830 2014-08-13 18:06:06.972265 7f5c973a57a0 20 read_log 192834'17275 (0'0) promote f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by osd.317.0:19317 2014-08-11 15:25:05.949622 2014-08-13 18:06:06.972280 7f5c973a57a0 20 read_log 192834'17276 (192834'17275) modify f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.933448.0:18605 2014-08-11 15:25:05.940910 2014-08-13 18:06:06.972293 7f5c973a57a0 20 read_log 192834'17317 (0'0) promote 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by osd.317.0:19359 2014-08-11 15:25:30.668946 2014-08-13 18:06:06.972305 7f5c973a57a0 20 read_log 192834'17318 (192834'17317) modify 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.933448.0:21493 2014-08-11 15:25:30.658927 2014-08-13 18:06:06.972317 7f5c973a57a0 20 read_log 192834'17330 (192834'17024) modify f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by client.933448.0:23546 2014-08-11 15:25:39.464984 2014-08-13 18:06:06.972334 7f5c973a57a0 20 read_log 192834'17338 (192834'17040) modify 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by client.933448.0:25125 2014-08-11 15:25:41.807109 2014-08-13 18:06:06.972349 7f5c973a57a0 20 read_log 192834'17346 (192834'17056) modify f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.933448.0:26334 2014-08-11 15:25:43.279758 2014-08-13 18:06:06.972361 7f5c973a57a0 20 read_log 192834'17367 (192834'17098) modify 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.933448.0:29601 2014-08-11 15:25:47.512335 2014-08-13 18:06:06.972387 7f5c973a57a0 20 read_log 192834'17396 (192834'17156) modify 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by client.933448.0:33454 2014-08-11 15:25:53.524996 2014-08-13 18:06:06.972400 7f5c973a57a0 20 read_log 192834'17401 (192834'17166) modify b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.933448.0:34720 2014-08-11 15:25:55.390367 2014-08-13 18:06:06.972413 7f5c973a57a0 20 read_log 192834'17409 (192834'17182) modify 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.933448.0:35248 2014-08-11 15:25:56.004388 2014-08-13 18:06:06.972436 7f5c973a57a0 20 read_log 192834'17411 (192834'17186) modify 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.933448.0:35466 2014-08-11 15:25:56.320554 2014-08-13 18:06:06.972449 7f5c973a57a0 20 read_log 192834'17416 (192834'17196) modify 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.933448.0:36117 2014-08-11 15:25:57.257164 2014-08-13 18:06:06.972470 7f5c973a57a0 20 read_log 192834'17424 (192834'17212) modify abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by client.933448.0:36806 2014-08-11 15:25:58.330423 2014-08-13 18:06:06.972493 7f5c973a57a0 20 read_log 192834'17428 (192834'17220) modify 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.933448.0:37749 2014-08-11 15:25:59.949825 2014-08-13 18:06:06.972508 7f5c973a57a0 20 read_log 192834'17434 (192834'17232) modify 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by client.933448.0:38530 2014-08-11 15:26:00.971272 2014-08-13 18:06:06.972521 7f5c973a57a0 20 read_log 192834'17439 (192834'17242) modify 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.933448.0:38934 2014-08-11 15:26:01.412493 2014-08-13 18:06:06.972533 7f5c973a57a0 20 read_log 192834'17446 (192834'17256) modify 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by client.933448.0:39641 2014-08-11 15:26:02.007815 2014-08-13 18:06:06.972546 7f5c973a57a0 20 read_log 192834'17448 (192834'17260) modify 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by client.933448.0:39989 2014-08-11 15:26:02.205566 2014-08-13 18:06:06.972581 7f5c973a57a0 20 read_log 192834'17456 (192834'17276) modify f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.933448.0:40896 2014-08-11 15:26:02.773871 2014-08-13 18:06:06.972625 7f5c973a57a0 20 read_log 192834'17477 (192834'17318) modify 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.933448.0:43784 2014-08-11 15:26:03.723468 2014-08-13 18:06:06.972642 7f5c973a57a0 20 read_log 192834'17490 (0'0) promote 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by osd.317.0:19532 2014-08-11 15:26:18.147259 2014-08-13 18:06:06.972654 7f5c973a57a0 20 read_log 192834'17491 (192834'17490) modify 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.1883634.0:1005 2014-08-11 15:26:18.139396 2014-08-13 18:06:06.972667 7f5c973a57a0 20 read_log 192834'17542 (0'0) promote a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by osd.317.0:19584 2014-08-11 15:26:27.736745 2014-08-13 18:06:06.972679 7f5c973a57a0 20 read_log 192834'17543 (192834'17542) modify a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.1883634.0:3643 2014-08-11 15:26:27.727335 2014-08-13 18:06:06.972692 7f5c973a57a0 20 read_log 192834'17544 (0'0) promote 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by osd.317.0:19586 2014-08-11 15:26:28.512710 2014-08-13 18:06:06.972704 7f5c973a57a0 20 read_log 192834'17545 (192834'17544) modify 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.1883634.0:3863 2014-08-11 15:26:28.504815 2014-08-13 18:06:06.972716 7f5c973a57a0 20 read_log 192834'17564 (0'0) promote acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by osd.317.0:19606 2014-08-11 15:26:34.148592 2014-08-13 18:06:06.972728 7f5c973a57a0 20 read_log 192834'17565 (192834'17564) modify acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.1883634.0:5366 2014-08-11 15:26:34.141462 2014-08-13 18:06:06.972741 7f5c973a57a0 20 read_log 192834'17570 (0'0) promote adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.317.0:19612 2014-08-11 15:26:36.318268 2014-08-13 18:06:06.972753 7f5c973a57a0 20 read_log 192834'17571 (192834'17570) modify adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by client.1883634.0:5974 2014-08-11 15:26:36.310277 2014-08-13 18:06:06.972765 7f5c973a57a0 20 read_log 192834'17586 (0'0) promote d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by osd.317.0:19628 2014-08-11 15:26:40.195846 2014-08-13 18:06:06.972786 7f5c973a57a0 20 read_log 192834'17587 (192834'17586) modify d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.1883634.0:6962 2014-08-11 15:26:40.188026 2014-08-13 18:06:06.972802 7f5c973a57a0 20 read_log 192834'17596 (0'0) promote 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by osd.317.0:19638 2014-08-11 15:26:45.160281 2014-08-13 18:06:06.972814 7f5c973a57a0 20 read_log 192834'17597 (192834'17596) modify 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.1883634.0:8148 2014-08-11 15:26:45.150794 2014-08-13 18:06:06.972827 7f5c973a57a0 20 read_log 192834'17598 (0'0) promote eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by osd.317.0:19640 2014-08-11 15:26:46.077398 2014-08-13 18:06:06.972839 7f5c973a57a0 20 read_log 192834'17599 (192834'17598) modify eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.1883634.0:8391 2014-08-11 15:26:46.068076 2014-08-13 18:06:06.972852 7f5c973a57a0 20 read_log 192834'17608 (0'0) promote 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by osd.317.0:19650 2014-08-11 15:26:49.682007 2014-08-13 18:06:06.972864 7f5c973a57a0 20 read_log 192834'17609 (192834'17608) modify 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.1883634.0:9155 2014-08-11 15:26:49.670694 2014-08-13 18:06:06.972876 7f5c973a57a0 20 read_log 192834'17628 (0'0) promote 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by osd.317.0:19670 2014-08-11 15:26:53.726406 2014-08-13 18:06:06.972899 7f5c973a57a0 20 read_log 192834'17629 (192834'17628) modify 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.1883634.0:10109 2014-08-11 15:26:53.717496 2014-08-13 18:06:06.972917 7f5c973a57a0 20 read_log 192834'17644 (0'0) promote f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by osd.317.0:19686 2014-08-11 15:26:58.566572 2014-08-13 18:06:06.972930 7f5c973a57a0 20 read_log 192834'17645 (192834'17644) modify f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.1883634.0:11155 2014-08-11 15:26:58.557201 2014-08-13 18:06:06.972943 7f5c973a57a0 20 read_log 192834'17654 (0'0) promote db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by osd.317.0:19696 2014-08-11 15:27:02.761501 2014-08-13 18:06:06.972957 7f5c973a57a0 20 read_log 192834'17655 (192834'17654) modify db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.1883634.0:12158 2014-08-11 15:27:02.752094 2014-08-13 18:06:06.972970 7f5c973a57a0 20 read_log 192834'17710 (0'0) promote eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by osd.317.0:19752 2014-08-11 15:27:29.574624 2014-08-13 18:06:06.972982 7f5c973a57a0 20 read_log 192834'17711 (192834'17710) modify eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.1883634.0:16838 2014-08-11 15:27:29.564982 2014-08-13 18:06:06.972994 7f5c973a57a0 20 read_log 192834'17714 (0'0) promote 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by osd.317.0:19756 2014-08-11 15:27:34.012370 2014-08-13 18:06:06.973007 7f5c973a57a0 20 read_log 192834'17715 (192834'17714) modify 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.1883634.0:17274 2014-08-11 15:27:34.002352 2014-08-13 18:06:06.973019 7f5c973a57a0 20 read_log 192834'17728 (0'0) promote b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by osd.317.0:19770 2014-08-11 15:27:47.104103 2014-08-13 18:06:06.973031 7f5c973a57a0 20 read_log 192834'17729 (192834'17728) modify b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.1883634.0:18691 2014-08-11 15:27:47.093957 2014-08-13 18:06:06.973044 7f5c973a57a0 20 read_log 192834'17732 (0'0) promote 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by osd.317.0:19774 2014-08-11 15:27:47.773474 2014-08-13 18:06:06.973067 7f5c973a57a0 20 read_log 192834'17733 (192834'17732) modify 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.1883634.0:18834 2014-08-11 15:27:47.763955 2014-08-13 18:06:06.973081 7f5c973a57a0 20 read_log 192834'17736 (0'0) promote f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.317.0:19778 2014-08-11 15:27:48.597862 2014-08-13 18:06:06.973093 7f5c973a57a0 20 read_log 192834'17737 (192834'17736) modify f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by client.1883634.0:18978 2014-08-11 15:27:48.589213 2014-08-13 18:06:06.973124 7f5c973a57a0 20 read_log 192834'17744 (0'0) promote 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by osd.317.0:19786 2014-08-11 15:27:50.233709 2014-08-13 18:06:06.973137 7f5c973a57a0 20 read_log 192834'17745 (192834'17744) modify 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.1883634.0:19280 2014-08-11 15:27:50.225030 2014-08-13 18:06:06.973157 7f5c973a57a0 20 read_log 192834'17760 (0'0) promote e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.317.0:19802 2014-08-11 15:27:56.565945 2014-08-13 18:06:06.973172 7f5c973a57a0 20 read_log 192834'17761 (192834'17760) modify e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by client.1883634.0:20390 2014-08-11 15:27:56.555620 2014-08-13 18:06:06.973189 7f5c973a57a0 20 read_log 192834'17807 (192834'17491) modify 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.1883634.0:24329 2014-08-11 15:28:18.915962 2014-08-13 18:06:06.973209 7f5c973a57a0 20 read_log 192834'17833 (192834'17543) modify a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.1883634.0:26967 2014-08-11 15:28:23.845686 2014-08-13 18:06:06.973233 7f5c973a57a0 20 read_log 192834'17834 (192834'17545) modify 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.1883634.0:27187 2014-08-11 15:28:24.149979 2014-08-13 18:06:06.973247 7f5c973a57a0 20 read_log 192834'17844 (192834'17565) modify acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.1883634.0:28690 2014-08-11 15:28:26.144243 2014-08-13 18:06:06.973260 7f5c973a57a0 20 read_log 192834'17847 (192834'17571) modify adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by client.1883634.0:29298 2014-08-11 15:28:26.947557 2014-08-13 18:06:06.973276 7f5c973a57a0 20 read_log 192834'17855 (192834'17587) modify d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.1883634.0:30286 2014-08-11 15:28:28.401415 2014-08-13 18:06:06.973291 7f5c973a57a0 20 read_log 192834'17860 (192834'17597) modify 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.1883634.0:31472 2014-08-11 15:28:30.577931 2014-08-13 18:06:06.973304 7f5c973a57a0 20 read_log 192834'17861 (192834'17599) modify eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.1883634.0:31715 2014-08-11 15:28:30.878325 2014-08-13 18:06:06.973328 7f5c973a57a0 20 read_log 192834'17866 (192834'17609) modify 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.1883634.0:32479 2014-08-11 15:28:31.912051 2014-08-13 18:06:06.973340 7f5c973a57a0 20 read_log 192834'17876 (192834'17629) modify 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.1883634.0:33433 2014-08-11 15:28:33.190837 2014-08-13 18:06:06.973353 7f5c973a57a0 20 read_log 192834'17884 (192834'17645) modify f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.1883634.0:34479 2014-08-11 15:28:34.875792 2014-08-13 18:06:06.973386 7f5c973a57a0 20 read_log 192834'17891 (192834'17655) modify db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.1883634.0:35482 2014-08-11 15:28:36.711352 2014-08-13 18:06:06.973402 7f5c973a57a0 20 read_log 192834'17919 (192834'17711) modify eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.1883634.0:40162 2014-08-11 15:28:43.279722 2014-08-13 18:06:06.973438 7f5c973a57a0 20 read_log 192834'17921 (192834'17715) modify 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.1883634.0:40598 2014-08-11 15:28:44.041766 2014-08-13 18:06:06.973451 7f5c973a57a0 20 read_log 192834'17928 (192834'17729) modify b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.1883634.0:42015 2014-08-11 15:28:45.636176 2014-08-13 18:06:06.973464 7f5c973a57a0 20 read_log 192834'17930 (192834'17733) modify 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.1883634.0:42158 2014-08-11 15:28:46.153680 2014-08-13 18:06:06.973476 7f5c973a57a0 20 read_log 192834'17932 (192834'17737) modify f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by client.1883634.0:42302 2014-08-11 15:28:46.200853 2014-08-13 18:06:06.973489 7f5c973a57a0 20 read_log 192834'17936 (192834'17745) modify 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.1883634.0:42604 2014-08-11 15:28:46.295428 2014-08-13 18:06:06.973501 7f5c973a57a0 20 read_log 192834'17944 (192834'17761) modify e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by client.1883634.0:43714 2014-08-11 15:28:46.640688 2014-08-13 18:06:06.973513 7f5c973a57a0 20 read_log 192888'17974 (0'0) promote 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:17514 2014-08-11 16:45:12.002365 2014-08-13 18:06:06.973534 7f5c973a57a0 20 read_log 192888'17975 (192888'17974) modify 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by client.1315259.0:4424 2014-08-11 16:45:11.994368 2014-08-13 18:06:06.973549 7f5c973a57a0 20 read_log 192888'17980 (0'0) promote 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by osd.356.0:17530 2014-08-11 16:45:19.410219 2014-08-13 18:06:06.973561 7f5c973a57a0 20 read_log 192888'17981 (192888'17980) modify 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.1315259.0:6450 2014-08-11 16:45:19.401855 2014-08-13 18:06:06.973574 7f5c973a57a0 20 read_log 192888'17982 (0'0) promote 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by osd.356.0:17536 2014-08-11 16:45:20.283910 2014-08-13 18:06:06.973586 7f5c973a57a0 20 read_log 192888'17983 (192888'17982) modify 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.1315259.0:6693 2014-08-11 16:45:20.276653 2014-08-13 18:06:06.973598 7f5c973a57a0 20 read_log 192888'17992 (0'0) promote 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:17550 2014-08-11 16:45:28.324798 2014-08-13 18:06:06.973610 7f5c973a57a0 20 read_log 192888'17993 (192888'17992) modify 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by client.1315259.0:8850 2014-08-11 16:45:28.317281 2014-08-13 18:06:06.973622 7f5c973a57a0 20 read_log 192888'17994 (0'0) promote f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by osd.356.0:17554 2014-08-11 16:45:30.386087 2014-08-13 18:06:06.973634 7f5c973a57a0 20 read_log 192888'17995 (192888'17994) modify f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.1315259.0:9407 2014-08-11 16:45:30.377349 2014-08-13 18:06:06.973646 7f5c973a57a0 20 read_log 192888'17996 (0'0) promote 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by osd.356.0:17558 2014-08-11 16:45:30.937502 2014-08-13 18:06:06.973659 7f5c973a57a0 20 read_log 192888'17997 (192888'17996) modify 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.1315259.0:9548 2014-08-11 16:45:30.930144 2014-08-13 18:06:06.973671 7f5c973a57a0 20 read_log 192888'17998 (0'0) promote 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:17560 2014-08-11 16:45:32.184642 2014-08-13 18:06:06.973683 7f5c973a57a0 20 read_log 192888'17999 (192888'17998) modify 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by client.1315259.0:9866 2014-08-11 16:45:32.175982 2014-08-13 18:06:06.973695 7f5c973a57a0 20 read_log 192888'18002 (0'0) promote 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by osd.356.0:17566 2014-08-11 16:45:34.219372 2014-08-13 18:06:06.973707 7f5c973a57a0 20 read_log 192888'18003 (192888'18002) modify 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.1315259.0:10402 2014-08-11 16:45:34.211276 2014-08-13 18:06:06.973719 7f5c973a57a0 20 read_log 192888'18004 (0'0) promote b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by osd.356.0:17568 2014-08-11 16:45:36.560867 2014-08-13 18:06:06.973731 7f5c973a57a0 20 read_log 192888'18005 (192888'18004) modify b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.1315259.0:10986 2014-08-11 16:45:36.553773 2014-08-13 18:06:06.973744 7f5c973a57a0 20 read_log 192888'18006 (0'0) promote 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:17570 2014-08-11 16:45:36.568784 2014-08-13 18:06:06.973756 7f5c973a57a0 20 read_log 192888'18007 (192888'18006) modify 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by client.1315259.0:10988 2014-08-11 16:45:36.561494 2014-08-13 18:06:06.973771 7f5c973a57a0 20 read_log 192888'18008 (0'0) promote f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by osd.356.0:17574 2014-08-11 16:45:40.497053 2014-08-13 18:06:06.973793 7f5c973a57a0 20 read_log 192888'18009 (192888'18008) modify f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.1315259.0:11953 2014-08-11 16:45:40.489805 2014-08-13 18:06:06.973809 7f5c973a57a0 20 read_log 192888'18010 (0'0) promote 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by osd.356.0:17578 2014-08-11 16:45:41.704342 2014-08-13 18:06:06.973821 7f5c973a57a0 20 read_log 192888'18011 (192888'18010) modify 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.1315259.0:12261 2014-08-11 16:45:41.696943 2014-08-13 18:06:06.973836 7f5c973a57a0 20 read_log 192888'18012 (0'0) promote c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by osd.356.0:17582 2014-08-11 16:45:43.192582 2014-08-13 18:06:06.973848 7f5c973a57a0 20 read_log 192888'18013 (192888'18012) modify c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.1315259.0:12608 2014-08-11 16:45:43.185428 2014-08-13 18:06:06.973862 7f5c973a57a0 20 read_log 192888'18016 (0'0) promote 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by osd.356.0:17588 2014-08-11 16:45:46.666613 2014-08-13 18:06:06.973875 7f5c973a57a0 20 read_log 192888'18017 (192888'18016) modify 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.1315259.0:13448 2014-08-11 16:45:46.658781 2014-08-13 18:06:06.973889 7f5c973a57a0 20 read_log 192888'18018 (0'0) promote 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by osd.356.0:17592 2014-08-11 16:45:47.503893 2014-08-13 18:06:06.973901 7f5c973a57a0 20 read_log 192888'18019 (192888'18018) modify 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.1315259.0:13660 2014-08-11 16:45:47.492376 2014-08-13 18:06:06.973914 7f5c973a57a0 20 read_log 192888'18022 (0'0) promote 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by osd.356.0:17600 2014-08-11 16:45:48.487877 2014-08-13 18:06:06.973926 7f5c973a57a0 20 read_log 192888'18023 (192888'18022) modify 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.1315259.0:13924 2014-08-11 16:45:48.480282 2014-08-13 18:06:06.973938 7f5c973a57a0 20 read_log 192888'18024 (0'0) promote 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by osd.356.0:17604 2014-08-11 16:45:49.866046 2014-08-13 18:06:06.973952 7f5c973a57a0 20 read_log 192888'18025 (192888'18024) modify 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.1315259.0:14274 2014-08-11 16:45:49.857673 2014-08-13 18:06:06.973964 7f5c973a57a0 20 read_log 192888'18026 (0'0) promote f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:17606 2014-08-11 16:45:50.481070 2014-08-13 18:06:06.973978 7f5c973a57a0 20 read_log 192888'18027 (192888'18026) modify f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by client.1315259.0:14442 2014-08-11 16:45:50.475072 2014-08-13 18:06:06.973991 7f5c973a57a0 20 read_log 192888'18028 (0'0) promote 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by osd.356.0:17614 2014-08-11 16:45:52.806472 2014-08-13 18:06:06.974005 7f5c973a57a0 20 read_log 192888'18029 (192888'18028) modify 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.1315259.0:15063 2014-08-11 16:45:52.799840 2014-08-13 18:06:06.974017 7f5c973a57a0 20 read_log 192888'18030 (0'0) promote 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by osd.356.0:17618 2014-08-11 16:45:57.700734 2014-08-13 18:06:06.974031 7f5c973a57a0 20 read_log 192888'18031 (192888'18030) modify 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.1315259.0:16339 2014-08-11 16:45:57.693288 2014-08-13 18:06:06.974046 7f5c973a57a0 20 read_log 192888'18032 (0'0) promote eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by osd.356.0:17620 2014-08-11 16:45:57.824688 2014-08-13 18:06:06.974074 7f5c973a57a0 20 read_log 192888'18033 (192888'18032) modify eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.1315259.0:16372 2014-08-11 16:45:57.817071 2014-08-13 18:06:06.974090 7f5c973a57a0 20 read_log 192888'18036 (0'0) promote bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by osd.356.0:17628 2014-08-11 16:46:00.083738 2014-08-13 18:06:06.974102 7f5c973a57a0 20 read_log 192888'18037 (192888'18036) modify bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.1315259.0:16944 2014-08-11 16:46:00.076273 2014-08-13 18:06:06.974117 7f5c973a57a0 20 read_log 192888'18042 (0'0) promote c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:17636 2014-08-11 16:46:02.979924 2014-08-13 18:06:06.974129 7f5c973a57a0 20 read_log 192888'18043 (192888'18042) modify c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by client.1315259.0:17654 2014-08-11 16:46:02.972957 2014-08-13 18:06:06.974144 7f5c973a57a0 20 read_log 192888'18050 (0'0) promote 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:17660 2014-08-11 16:46:19.189029 2014-08-13 18:06:06.974164 7f5c973a57a0 20 read_log 192888'18051 (192888'18050) modify 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by client.1315259.0:21875 2014-08-11 16:46:19.181546 2014-08-13 18:06:06.974177 7f5c973a57a0 20 read_log 192888'18052 (0'0) promote 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:17670 2014-08-11 16:46:22.259285 2014-08-13 18:06:06.974189 7f5c973a57a0 20 read_log 192888'18053 (192888'18052) modify 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by client.1315259.0:22656 2014-08-11 16:46:22.252615 2014-08-13 18:06:06.974202 7f5c973a57a0 20 read_log 192888'18056 (0'0) promote 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:17682 2014-08-11 16:46:26.690193 2014-08-13 18:06:06.974214 7f5c973a57a0 20 read_log 192888'18057 (192888'18056) modify 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by client.1315259.0:23749 2014-08-11 16:46:26.682917 2014-08-13 18:06:06.974226 7f5c973a57a0 20 read_log 192888'18062 (0'0) promote 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by osd.356.0:17692 2014-08-11 16:46:31.248453 2014-08-13 18:06:06.974240 7f5c973a57a0 20 read_log 192888'18063 (192888'18062) modify 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.1315259.0:24928 2014-08-11 16:46:31.240772 2014-08-13 18:06:06.974258 7f5c973a57a0 20 read_log 192888'18066 (0'0) promote c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by osd.356.0:17698 2014-08-11 16:46:36.491539 2014-08-13 18:06:06.974271 7f5c973a57a0 20 read_log 192888'18067 (192888'18066) modify c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.1315259.0:26173 2014-08-11 16:46:36.483805 2014-08-13 18:06:06.974285 7f5c973a57a0 20 read_log 192888'18072 (0'0) promote d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by osd.356.0:17708 2014-08-11 16:46:41.218457 2014-08-13 18:06:06.974301 7f5c973a57a0 20 read_log 192888'18073 (192888'18072) modify d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.1315259.0:27296 2014-08-11 16:46:41.211637 2014-08-13 18:06:06.974315 7f5c973a57a0 20 read_log 192888'18080 (0'0) promote 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by osd.356.0:17720 2014-08-11 16:46:46.375284 2014-08-13 18:06:06.974328 7f5c973a57a0 20 read_log 192888'18081 (192888'18080) modify 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.1315259.0:28568 2014-08-11 16:46:46.368158 2014-08-13 18:06:06.974340 7f5c973a57a0 20 read_log 192888'18082 (0'0) promote e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by osd.356.0:17722 2014-08-11 16:46:48.138146 2014-08-13 18:06:06.974363 7f5c973a57a0 20 read_log 192888'18083 (192888'18082) modify e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.1315259.0:29031 2014-08-11 16:46:48.132040 2014-08-13 18:06:06.974379 7f5c973a57a0 20 read_log 192888'18084 (0'0) promote 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by osd.356.0:17724 2014-08-11 16:46:48.584329 2014-08-13 18:06:06.974422 7f5c973a57a0 20 read_log 192888'18085 (192888'18084) modify 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.1315259.0:29147 2014-08-11 16:46:48.576764 2014-08-13 18:06:06.974456 7f5c973a57a0 20 read_log 192888'18086 (0'0) promote cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by osd.356.0:17726 2014-08-11 16:46:48.983730 2014-08-13 18:06:06.974482 7f5c973a57a0 20 read_log 192888'18087 (192888'18086) modify cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.1315259.0:29236 2014-08-11 16:46:48.976894 2014-08-13 18:06:06.974495 7f5c973a57a0 20 read_log 192888'18090 (0'0) promote 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by osd.356.0:17732 2014-08-11 16:46:52.139828 2014-08-13 18:06:06.974514 7f5c973a57a0 20 read_log 192888'18091 (192888'18090) modify 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.1315259.0:30007 2014-08-11 16:46:52.132185 2014-08-13 18:06:06.974530 7f5c973a57a0 20 read_log 192888'18098 (192888'17975) modify 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by client.1315259.0:35151 2014-08-11 16:46:57.085177 2014-08-13 18:06:06.974543 7f5c973a57a0 20 read_log 192888'18101 (192888'17981) modify 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.1315259.0:37177 2014-08-11 16:46:57.614629 2014-08-13 18:06:06.974556 7f5c973a57a0 20 read_log 192888'18102 (192888'17983) modify 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.1315259.0:37420 2014-08-11 16:46:57.678696 2014-08-13 18:06:06.974568 7f5c973a57a0 20 read_log 192888'18107 (192888'17993) modify 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by client.1315259.0:39577 2014-08-11 16:46:58.248935 2014-08-13 18:06:06.974584 7f5c973a57a0 20 read_log 192888'18108 (192888'17995) modify f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.1315259.0:40134 2014-08-11 16:46:58.398546 2014-08-13 18:06:06.974597 7f5c973a57a0 20 read_log 192888'18109 (192888'17997) modify 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.1315259.0:40275 2014-08-11 16:46:58.439164 2014-08-13 18:06:06.974609 7f5c973a57a0 20 read_log 192888'18110 (192888'17999) modify 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by client.1315259.0:40593 2014-08-11 16:46:58.532314 2014-08-13 18:06:06.974621 7f5c973a57a0 20 read_log 192888'18112 (192888'18003) modify 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.1315259.0:41129 2014-08-11 16:46:58.666241 2014-08-13 18:06:06.974634 7f5c973a57a0 20 read_log 192888'18113 (192888'18005) modify b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.1315259.0:41713 2014-08-11 16:46:58.816973 2014-08-13 18:06:06.974646 7f5c973a57a0 20 read_log 192888'18114 (192888'18007) modify 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by client.1315259.0:41715 2014-08-11 16:46:58.817841 2014-08-13 18:06:06.974658 7f5c973a57a0 20 read_log 192888'18115 (192888'18009) modify f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.1315259.0:42680 2014-08-11 16:46:59.075527 2014-08-13 18:06:06.974671 7f5c973a57a0 20 read_log 192888'18116 (192888'18011) modify 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.1315259.0:42988 2014-08-11 16:46:59.159910 2014-08-13 18:06:06.974683 7f5c973a57a0 20 read_log 192888'18117 (192888'18013) modify c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.1315259.0:43335 2014-08-11 16:46:59.262756 2014-08-13 18:06:06.974724 7f5c973a57a0 20 read_log 192888'18119 (192888'18017) modify 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.1315259.0:44175 2014-08-11 16:46:59.503100 2014-08-13 18:06:06.974741 7f5c973a57a0 20 read_log 192888'18120 (192888'18019) modify 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.1315259.0:44387 2014-08-11 16:46:59.564089 2014-08-13 18:06:06.974753 7f5c973a57a0 20 read_log 192888'18122 (192888'18023) modify 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.1315259.0:44651 2014-08-11 16:46:59.639652 2014-08-13 18:06:06.974778 7f5c973a57a0 20 read_log 192888'18123 (192888'18025) modify 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.1315259.0:45001 2014-08-11 16:46:59.742512 2014-08-13 18:06:06.974806 7f5c973a57a0 20 read_log 192888'18124 (192888'18027) modify f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by client.1315259.0:45169 2014-08-11 16:46:59.790114 2014-08-13 18:06:06.974821 7f5c973a57a0 20 read_log 192888'18125 (192888'18029) modify 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.1315259.0:45790 2014-08-11 16:46:59.970401 2014-08-13 18:06:06.974834 7f5c973a57a0 20 read_log 192888'18126 (192888'18031) modify 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.1315259.0:47066 2014-08-11 16:47:00.336937 2014-08-13 18:06:06.974846 7f5c973a57a0 20 read_log 192888'18127 (192888'18033) modify eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.1315259.0:47099 2014-08-11 16:47:00.346308 2014-08-13 18:06:06.974859 7f5c973a57a0 20 read_log 192888'18129 (192888'18037) modify bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.1315259.0:47671 2014-08-11 16:47:00.529269 2014-08-13 18:06:06.974872 7f5c973a57a0 20 read_log 192888'18132 (192888'18043) modify c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by client.1315259.0:48381 2014-08-11 16:47:00.748918 2014-08-13 18:06:06.974884 7f5c973a57a0 20 read_log 192888'18136 (192888'18051) modify 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by client.1315259.0:52602 2014-08-11 16:47:02.019561 2014-08-13 18:06:06.974896 7f5c973a57a0 20 read_log 192888'18137 (192888'18053) modify 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by client.1315259.0:53383 2014-08-11 16:47:02.236567 2014-08-13 18:06:06.974909 7f5c973a57a0 20 read_log 192888'18139 (192888'18057) modify 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by client.1315259.0:54476 2014-08-11 16:47:02.576988 2014-08-13 18:06:06.974921 7f5c973a57a0 20 read_log 192888'18142 (192888'18063) modify 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.1315259.0:55655 2014-08-11 16:47:02.974342 2014-08-13 18:06:06.974933 7f5c973a57a0 20 read_log 192888'18144 (192888'18067) modify c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.1315259.0:56900 2014-08-11 16:47:03.348419 2014-08-13 18:06:06.974946 7f5c973a57a0 20 read_log 192888'18147 (192888'18073) modify d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.1315259.0:58023 2014-08-11 16:47:03.679815 2014-08-13 18:06:06.974958 7f5c973a57a0 20 read_log 192888'18151 (192888'18081) modify 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.1315259.0:59295 2014-08-11 16:47:04.067924 2014-08-13 18:06:06.974970 7f5c973a57a0 20 read_log 192888'18152 (192888'18083) modify e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.1315259.0:59758 2014-08-11 16:47:04.232717 2014-08-13 18:06:06.974984 7f5c973a57a0 20 read_log 192888'18153 (192888'18085) modify 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.1315259.0:59874 2014-08-11 16:47:04.267420 2014-08-13 18:06:06.974999 7f5c973a57a0 20 read_log 192888'18154 (192888'18087) modify cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.1315259.0:59963 2014-08-11 16:47:04.297850 2014-08-13 18:06:06.975018 7f5c973a57a0 20 read_log 192888'18156 (192888'18091) modify 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.1315259.0:60734 2014-08-11 16:47:04.524504 2014-08-13 18:06:06.975033 7f5c973a57a0 20 read_log 192888'18162 (0'0) promote e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by osd.356.0:17868 2014-08-11 17:05:27.050421 2014-08-13 18:06:06.975046 7f5c973a57a0 20 read_log 192888'18163 (192888'18162) modify e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.942868.0:3002 2014-08-11 17:05:27.043401 2014-08-13 18:06:06.975060 7f5c973a57a0 20 read_log 192888'18164 (0'0) promote 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:17880 2014-08-11 17:05:31.133597 2014-08-13 18:06:06.975074 7f5c973a57a0 20 read_log 192888'18165 (192888'18164) modify 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by client.942868.0:4108 2014-08-11 17:05:31.126492 2014-08-13 18:06:06.975088 7f5c973a57a0 20 read_log 192888'18168 (0'0) promote ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by osd.356.0:17888 2014-08-11 17:05:34.547744 2014-08-13 18:06:06.975100 7f5c973a57a0 20 read_log 192888'18169 (192888'18168) modify ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.942868.0:4984 2014-08-11 17:05:34.539875 2014-08-13 18:06:06.975113 7f5c973a57a0 20 read_log 192888'18174 (0'0) promote 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by osd.356.0:17896 2014-08-11 17:05:39.743511 2014-08-13 18:06:06.975125 7f5c973a57a0 20 read_log 192888'18175 (192888'18174) modify 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.942868.0:6395 2014-08-11 17:05:39.736743 2014-08-13 18:06:06.975139 7f5c973a57a0 20 read_log 192888'18176 (0'0) promote e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:17898 2014-08-11 17:05:40.059169 2014-08-13 18:06:06.975157 7f5c973a57a0 20 read_log 192888'18177 (192888'18176) modify e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by client.942868.0:6483 2014-08-11 17:05:40.051694 2014-08-13 18:06:06.975172 7f5c973a57a0 20 read_log 192888'18178 (0'0) promote 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by osd.356.0:17900 2014-08-11 17:05:40.401566 2014-08-13 18:06:06.975184 7f5c973a57a0 20 read_log 192888'18179 (192888'18178) modify 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.942868.0:6578 2014-08-11 17:05:40.393251 2014-08-13 18:06:06.975199 7f5c973a57a0 20 read_log 192888'18180 (0'0) promote b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by osd.356.0:17904 2014-08-11 17:05:42.777995 2014-08-13 18:06:06.975211 7f5c973a57a0 20 read_log 192888'18181 (192888'18180) modify b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.942868.0:7202 2014-08-11 17:05:42.769006 2014-08-13 18:06:06.975223 7f5c973a57a0 20 read_log 192888'18182 (0'0) promote 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by osd.356.0:17906 2014-08-11 17:05:44.974864 2014-08-13 18:06:06.975241 7f5c973a57a0 20 read_log 192888'18183 (192888'18182) modify 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.942868.0:7764 2014-08-11 17:05:44.966952 2014-08-13 18:06:06.975255 7f5c973a57a0 20 read_log 192888'18186 (0'0) promote 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by osd.356.0:17910 2014-08-11 17:05:46.111310 2014-08-13 18:06:06.975268 7f5c973a57a0 20 read_log 192888'18187 (192888'18186) modify 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.942868.0:8079 2014-08-11 17:05:46.104011 2014-08-13 18:06:06.975280 7f5c973a57a0 20 read_log 192888'18188 (0'0) promote a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by osd.356.0:17914 2014-08-11 17:05:48.718843 2014-08-13 18:06:06.975300 7f5c973a57a0 20 read_log 192888'18189 (192888'18188) modify a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.942868.0:8777 2014-08-11 17:05:48.708387 2014-08-13 18:06:06.975314 7f5c973a57a0 20 read_log 192888'18194 (0'0) promote d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by osd.356.0:17922 2014-08-11 17:05:51.465210 2014-08-13 18:06:06.975327 7f5c973a57a0 20 read_log 192888'18195 (192888'18194) modify d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.942868.0:9519 2014-08-11 17:05:51.457463 2014-08-13 18:06:06.975339 7f5c973a57a0 20 read_log 192888'18200 (0'0) promote 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:17932 2014-08-11 17:06:00.398521 2014-08-13 18:06:06.975352 7f5c973a57a0 20 read_log 192888'18201 (192888'18200) modify 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by client.942868.0:11969 2014-08-11 17:06:00.391456 2014-08-13 18:06:06.975364 7f5c973a57a0 20 read_log 192888'18208 (0'0) promote 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by osd.356.0:17944 2014-08-11 17:06:07.019455 2014-08-13 18:06:06.975376 7f5c973a57a0 20 read_log 192888'18209 (192888'18208) modify 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.942868.0:13713 2014-08-11 17:06:07.011627 2014-08-13 18:06:06.975389 7f5c973a57a0 20 read_log 192888'18212 (0'0) promote b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:17948 2014-08-11 17:06:08.408545 2014-08-13 18:06:06.975418 7f5c973a57a0 20 read_log 192888'18213 (192888'18212) modify b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by client.942868.0:14043 2014-08-11 17:06:08.401363 2014-08-13 18:06:06.975431 7f5c973a57a0 20 read_log 192888'18214 (0'0) promote ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by osd.356.0:17952 2014-08-11 17:06:14.143721 2014-08-13 18:06:06.975456 7f5c973a57a0 20 read_log 192888'18215 (192888'18214) modify ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.942868.0:15492 2014-08-11 17:06:14.135201 2014-08-13 18:06:06.975472 7f5c973a57a0 20 read_log 192888'18216 (0'0) promote c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by osd.356.0:17956 2014-08-11 17:06:14.717093 2014-08-13 18:06:06.975485 7f5c973a57a0 20 read_log 192888'18217 (192888'18216) modify c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.942868.0:15641 2014-08-11 17:06:14.709466 2014-08-13 18:06:06.975513 7f5c973a57a0 20 read_log 192888'18220 (0'0) promote a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:17964 2014-08-11 17:06:16.375005 2014-08-13 18:06:06.975526 7f5c973a57a0 20 read_log 192888'18221 (192888'18220) modify a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by client.942868.0:16069 2014-08-11 17:06:16.368059 2014-08-13 18:06:06.975539 7f5c973a57a0 20 read_log 192888'18226 (0'0) promote e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by osd.356.0:17976 2014-08-11 17:06:19.734637 2014-08-13 18:06:06.975551 7f5c973a57a0 20 read_log 192888'18227 (192888'18226) modify e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.942868.0:16913 2014-08-11 17:06:19.727190 2014-08-13 18:06:06.975564 7f5c973a57a0 20 read_log 192888'18228 (0'0) promote eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by osd.356.0:17980 2014-08-11 17:06:22.635337 2014-08-13 18:06:06.975576 7f5c973a57a0 20 read_log 192888'18229 (192888'18228) modify eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.942868.0:17681 2014-08-11 17:06:22.628455 2014-08-13 18:06:06.975589 7f5c973a57a0 20 read_log 192888'18230 (0'0) promote bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by osd.356.0:17984 2014-08-11 17:06:23.858678 2014-08-13 18:06:06.975611 7f5c973a57a0 20 read_log 192888'18231 (192888'18230) modify bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.942868.0:18007 2014-08-11 17:06:23.850755 2014-08-13 18:06:06.975627 7f5c973a57a0 20 read_log 192888'18232 (0'0) promote 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by osd.356.0:17986 2014-08-11 17:06:26.085348 2014-08-13 18:06:06.975639 7f5c973a57a0 20 read_log 192888'18233 (192888'18232) modify 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.942868.0:18597 2014-08-11 17:06:26.077379 2014-08-13 18:06:06.975664 7f5c973a57a0 20 read_log 192888'18238 (0'0) promote 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:17992 2014-08-11 17:06:28.915000 2014-08-13 18:06:06.975677 7f5c973a57a0 20 read_log 192888'18239 (192888'18238) modify 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by client.942868.0:19350 2014-08-11 17:06:28.907553 2014-08-13 18:06:06.975689 7f5c973a57a0 20 read_log 192888'18240 (0'0) promote ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:17994 2014-08-11 17:06:29.278181 2014-08-13 18:06:06.975701 7f5c973a57a0 20 read_log 192888'18241 (192888'18240) modify ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by client.942868.0:19448 2014-08-11 17:06:29.271423 2014-08-13 18:06:06.975713 7f5c973a57a0 20 read_log 192888'18252 (0'0) promote 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by osd.356.0:18020 2014-08-11 17:06:44.437087 2014-08-13 18:06:06.975726 7f5c973a57a0 20 read_log 192888'18253 (192888'18252) modify 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.942868.0:23382 2014-08-11 17:06:44.430200 2014-08-13 18:06:06.975738 7f5c973a57a0 20 read_log 192888'18256 (0'0) promote e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:18026 2014-08-11 17:06:45.447660 2014-08-13 18:06:06.975750 7f5c973a57a0 20 read_log 192888'18257 (192888'18256) modify e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by client.942868.0:23658 2014-08-11 17:06:45.440344 2014-08-13 18:06:06.975762 7f5c973a57a0 20 read_log 192888'18258 (0'0) promote b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:18028 2014-08-11 17:06:46.541032 2014-08-13 18:06:06.975774 7f5c973a57a0 20 read_log 192888'18259 (192888'18258) modify b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by client.942868.0:23954 2014-08-11 17:06:46.533672 2014-08-13 18:06:06.975787 7f5c973a57a0 20 read_log 192888'18262 (0'0) promote 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:18032 2014-08-11 17:06:48.962049 2014-08-13 18:06:06.975799 7f5c973a57a0 20 read_log 192888'18263 (192888'18262) modify 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by client.942868.0:24607 2014-08-11 17:06:48.954257 2014-08-13 18:06:06.975811 7f5c973a57a0 20 read_log 192888'18266 (0'0) promote e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:18036 2014-08-11 17:06:51.331039 2014-08-13 18:06:06.975824 7f5c973a57a0 20 read_log 192888'18267 (192888'18266) modify e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by client.942868.0:25247 2014-08-11 17:06:51.324057 2014-08-13 18:06:06.975845 7f5c973a57a0 20 read_log 192888'18268 (0'0) promote c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:18038 2014-08-11 17:06:53.243571 2014-08-13 18:06:06.975860 7f5c973a57a0 20 read_log 192888'18269 (192888'18268) modify c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by client.942868.0:25726 2014-08-11 17:06:53.236076 2014-08-13 18:06:06.975873 7f5c973a57a0 20 read_log 192888'18280 (0'0) promote 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by osd.356.0:18058 2014-08-11 17:06:57.866249 2014-08-13 18:06:06.975892 7f5c973a57a0 20 read_log 192888'18281 (192888'18280) modify 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.942868.0:26902 2014-08-11 17:06:57.859512 2014-08-13 18:06:06.975910 7f5c973a57a0 20 read_log 192888'18282 (0'0) promote c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by osd.356.0:18064 2014-08-11 17:07:02.018786 2014-08-13 18:06:06.975923 7f5c973a57a0 20 read_log 192888'18283 (192888'18282) modify c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.942868.0:28006 2014-08-11 17:07:02.011695 2014-08-13 18:06:06.975935 7f5c973a57a0 20 read_log 192888'18290 (0'0) promote ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by osd.356.0:18084 2014-08-11 17:07:11.683933 2014-08-13 18:06:06.975947 7f5c973a57a0 20 read_log 192888'18291 (192888'18290) modify ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.942868.0:30464 2014-08-11 17:07:11.676867 2014-08-13 18:06:06.975963 7f5c973a57a0 20 read_log 192888'18295 (192888'18163) modify e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.942868.0:34268 2014-08-11 17:07:16.742708 2014-08-13 18:06:06.975976 7f5c973a57a0 20 read_log 192888'18296 (192888'18165) modify 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by client.942868.0:35374 2014-08-11 17:07:17.058108 2014-08-13 18:06:06.975989 7f5c973a57a0 20 read_log 192888'18298 (192888'18169) modify ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.942868.0:36250 2014-08-11 17:07:17.280170 2014-08-13 18:06:06.976024 7f5c973a57a0 20 read_log 192888'18301 (192888'18175) modify 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.942868.0:37661 2014-08-11 17:07:17.672214 2014-08-13 18:06:06.976041 7f5c973a57a0 20 read_log 192888'18302 (192888'18177) modify e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by client.942868.0:37749 2014-08-11 17:07:17.696350 2014-08-13 18:06:06.976065 7f5c973a57a0 20 read_log 192888'18303 (192888'18179) modify 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.942868.0:37844 2014-08-11 17:07:17.722356 2014-08-13 18:06:06.976081 7f5c973a57a0 20 read_log 192888'18304 (192888'18181) modify b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.942868.0:38468 2014-08-11 17:07:17.895439 2014-08-13 18:06:06.976094 7f5c973a57a0 20 read_log 192888'18305 (192888'18183) modify 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.942868.0:39030 2014-08-11 17:07:18.061065 2014-08-13 18:06:06.976116 7f5c973a57a0 20 read_log 192888'18307 (192888'18187) modify 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.942868.0:39345 2014-08-11 17:07:18.155657 2014-08-13 18:06:06.976129 7f5c973a57a0 20 read_log 192888'18308 (192888'18189) modify a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.942868.0:40043 2014-08-11 17:07:18.350524 2014-08-13 18:06:06.976141 7f5c973a57a0 20 read_log 192888'18311 (192888'18195) modify d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.942868.0:40785 2014-08-11 17:07:18.547543 2014-08-13 18:06:06.976177 7f5c973a57a0 20 read_log 192888'18314 (192888'18201) modify 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by client.942868.0:43235 2014-08-11 17:07:19.215432 2014-08-13 18:06:06.976220 7f5c973a57a0 20 read_log 192888'18318 (192888'18209) modify 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.942868.0:44979 2014-08-11 17:07:19.714582 2014-08-13 18:06:06.976237 7f5c973a57a0 20 read_log 192888'18320 (192888'18213) modify b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by client.942868.0:45309 2014-08-11 17:07:19.802505 2014-08-13 18:06:06.976250 7f5c973a57a0 20 read_log 192888'18321 (192888'18215) modify ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.942868.0:46758 2014-08-11 17:07:20.211654 2014-08-13 18:06:06.976270 7f5c973a57a0 20 read_log 192888'18322 (192888'18217) modify c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.942868.0:46907 2014-08-11 17:07:20.251961 2014-08-13 18:06:06.976284 7f5c973a57a0 20 read_log 192888'18324 (192888'18221) modify a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by client.942868.0:47335 2014-08-11 17:07:20.375032 2014-08-13 18:06:06.976297 7f5c973a57a0 20 read_log 192888'18327 (192888'18227) modify e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.942868.0:48179 2014-08-11 17:07:20.648985 2014-08-13 18:06:06.976309 7f5c973a57a0 20 read_log 192888'18328 (192888'18229) modify eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.942868.0:48947 2014-08-11 17:07:20.864714 2014-08-13 18:06:06.976322 7f5c973a57a0 20 read_log 192888'18329 (192888'18231) modify bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.942868.0:49273 2014-08-11 17:07:20.962030 2014-08-13 18:06:06.976334 7f5c973a57a0 20 read_log 192888'18330 (192888'18233) modify 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.942868.0:49863 2014-08-11 17:07:21.147035 2014-08-13 18:06:06.976346 7f5c973a57a0 20 read_log 192888'18333 (192888'18239) modify 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by client.942868.0:50616 2014-08-11 17:07:21.377797 2014-08-13 18:06:06.976358 7f5c973a57a0 20 read_log 192888'18334 (192888'18241) modify ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by client.942868.0:50714 2014-08-11 17:07:21.408598 2014-08-13 18:06:06.976378 7f5c973a57a0 20 read_log 192888'18340 (192888'18253) modify 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.942868.0:54648 2014-08-11 17:07:22.617915 2014-08-13 18:06:06.976392 7f5c973a57a0 20 read_log 192888'18342 (192888'18257) modify e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by client.942868.0:54924 2014-08-11 17:07:22.695823 2014-08-13 18:06:06.976409 7f5c973a57a0 20 read_log 192888'18343 (192888'18259) modify b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by client.942868.0:55220 2014-08-11 17:07:22.777171 2014-08-13 18:06:06.976421 7f5c973a57a0 20 read_log 192888'18345 (192888'18263) modify 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by client.942868.0:55873 2014-08-11 17:07:22.982825 2014-08-13 18:06:06.976434 7f5c973a57a0 20 read_log 192888'18347 (192888'18267) modify e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by client.942868.0:56513 2014-08-11 17:07:23.174997 2014-08-13 18:06:06.976446 7f5c973a57a0 20 read_log 192888'18348 (192888'18269) modify c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by client.942868.0:56992 2014-08-11 17:07:23.324201 2014-08-13 18:06:06.976458 7f5c973a57a0 20 read_log 192888'18354 (192888'18281) modify 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.942868.0:58168 2014-08-11 17:07:23.662669 2014-08-13 18:06:06.976495 7f5c973a57a0 20 read_log 192888'18355 (192888'18283) modify c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.942868.0:59272 2014-08-11 17:07:23.996372 2014-08-13 18:06:06.976508 7f5c973a57a0 20 read_log 192888'18359 (192888'18291) modify ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.942868.0:61730 2014-08-11 17:07:24.729264 2014-08-13 18:06:06.976521 7f5c973a57a0 20 read_log 193305'18361 (193305'18361) modify 3b7/hit_set_24.3b7_archive_2014-08-11 18:37:35.561357_2014-08-11 18:42:37.204550/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:42:37.204792 2014-08-13 18:06:06.976533 7f5c973a57a0 20 read_log 193305'18362 (0'0) promote 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:11 2014-08-11 18:42:37.208362 2014-08-13 18:06:06.976555 7f5c973a57a0 20 read_log 193305'18363 (193305'18362) modify 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by client.951337.0:1065 2014-08-11 18:42:37.199835 2014-08-13 18:06:06.976570 7f5c973a57a0 20 read_log 193305'18364 (0'0) promote 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by osd.356.0:16 2014-08-11 18:42:38.680835 2014-08-13 18:06:06.976583 7f5c973a57a0 20 read_log 193305'18365 (193305'18364) modify 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.951337.0:1447 2014-08-11 18:42:38.672964 2014-08-13 18:06:06.976595 7f5c973a57a0 20 read_log 193305'18366 (0'0) promote 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by osd.356.0:23 2014-08-11 18:42:42.178816 2014-08-13 18:06:06.976607 7f5c973a57a0 20 read_log 193305'18367 (193305'18366) modify 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.951337.0:2391 2014-08-11 18:42:42.171349 2014-08-13 18:06:06.976619 7f5c973a57a0 20 read_log 193305'18368 (0'0) promote d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by osd.356.0:27 2014-08-11 18:42:45.126919 2014-08-13 18:06:06.976631 7f5c973a57a0 20 read_log 193305'18369 (193305'18368) modify d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.951337.0:3200 2014-08-11 18:42:45.119639 2014-08-13 18:06:06.976644 7f5c973a57a0 20 read_log 193305'18370 (0'0) promote f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by osd.356.0:29 2014-08-11 18:42:45.388065 2014-08-13 18:06:06.976656 7f5c973a57a0 20 read_log 193305'18371 (193305'18370) modify f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.951337.0:3273 2014-08-11 18:42:45.381633 2014-08-13 18:06:06.976668 7f5c973a57a0 20 read_log 193305'18372 (0'0) promote 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by osd.356.0:57 2014-08-11 18:42:58.743410 2014-08-13 18:06:06.976680 7f5c973a57a0 20 read_log 193305'18373 (193305'18372) modify 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.951337.0:6916 2014-08-11 18:42:58.734468 2014-08-13 18:06:06.976693 7f5c973a57a0 20 read_log 193305'18374 (0'0) promote 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:59 2014-08-11 18:42:59.748810 2014-08-13 18:06:06.976705 7f5c973a57a0 20 read_log 193305'18375 (193305'18374) modify 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by client.951337.0:7195 2014-08-11 18:42:59.741637 2014-08-13 18:06:06.976719 7f5c973a57a0 20 read_log 193305'18376 (0'0) promote 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:61 2014-08-11 18:43:00.690202 2014-08-13 18:06:06.976732 7f5c973a57a0 20 read_log 193305'18377 (193305'18376) modify 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by client.951337.0:7456 2014-08-11 18:43:00.682863 2014-08-13 18:06:06.976746 7f5c973a57a0 20 read_log 193305'18378 (0'0) promote 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by osd.356.0:67 2014-08-11 18:43:05.656056 2014-08-13 18:06:06.976758 7f5c973a57a0 20 read_log 193305'18379 (193305'18378) modify 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.951337.0:8747 2014-08-11 18:43:05.646812 2014-08-13 18:06:06.976771 7f5c973a57a0 20 read_log 193305'18380 (0'0) promote a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by osd.356.0:77 2014-08-11 18:43:08.134102 2014-08-13 18:06:06.976785 7f5c973a57a0 20 read_log 193305'18381 (193305'18380) modify a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.951337.0:9410 2014-08-11 18:43:08.126059 2014-08-13 18:06:06.976797 7f5c973a57a0 20 read_log 193305'18382 (0'0) promote f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:97 2014-08-11 18:43:16.158212 2014-08-13 18:06:06.976833 7f5c973a57a0 20 read_log 193305'18383 (193305'18382) modify f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by client.951337.0:11530 2014-08-11 18:43:16.151070 2014-08-13 18:06:06.976849 7f5c973a57a0 20 read_log 193305'18384 (0'0) promote 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by osd.356.0:101 2014-08-11 18:43:17.031646 2014-08-13 18:06:06.976861 7f5c973a57a0 20 read_log 193305'18385 (193305'18384) modify 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.951337.0:11763 2014-08-11 18:43:17.024298 2014-08-13 18:06:06.976876 7f5c973a57a0 20 read_log 193305'18386 (0'0) promote 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:127 2014-08-11 18:43:26.578134 2014-08-13 18:06:06.976889 7f5c973a57a0 20 read_log 193305'18387 (193305'18386) modify 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by client.951337.0:14304 2014-08-11 18:43:26.569915 2014-08-13 18:06:06.976901 7f5c973a57a0 20 read_log 193305'18388 (0'0) promote 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by osd.356.0:131 2014-08-11 18:43:26.985726 2014-08-13 18:06:06.976913 7f5c973a57a0 20 read_log 193305'18389 (193305'18388) modify 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.951337.0:14428 2014-08-11 18:43:26.977152 2014-08-13 18:06:06.976928 7f5c973a57a0 20 read_log 193305'18390 (0'0) promote 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:133 2014-08-11 18:43:27.240278 2014-08-13 18:06:06.976941 7f5c973a57a0 20 read_log 193305'18391 (193305'18390) modify 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by client.951337.0:14505 2014-08-11 18:43:27.232734 2014-08-13 18:06:06.976955 7f5c973a57a0 20 read_log 193305'18392 (0'0) promote eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by osd.356.0:151 2014-08-11 18:43:34.971042 2014-08-13 18:06:06.976967 7f5c973a57a0 20 read_log 193305'18393 (193305'18392) modify eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.951337.0:16734 2014-08-11 18:43:34.963481 2014-08-13 18:06:06.976982 7f5c973a57a0 20 read_log 193305'18394 (0'0) promote 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by osd.356.0:155 2014-08-11 18:43:36.091909 2014-08-13 18:06:06.976995 7f5c973a57a0 20 read_log 193305'18395 (193305'18394) modify 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.951337.0:17039 2014-08-11 18:43:36.082545 2014-08-13 18:06:06.977010 7f5c973a57a0 20 read_log 193305'18396 (0'0) promote 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:163 2014-08-11 18:43:38.270245 2014-08-13 18:06:06.977022 7f5c973a57a0 20 read_log 193305'18397 (193305'18396) modify 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by client.951337.0:17694 2014-08-11 18:43:38.261756 2014-08-13 18:06:06.977037 7f5c973a57a0 20 read_log 193305'18398 (0'0) promote a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by osd.356.0:165 2014-08-11 18:43:38.790420 2014-08-13 18:06:06.977049 7f5c973a57a0 20 read_log 193305'18399 (193305'18398) modify a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.951337.0:17849 2014-08-11 18:43:38.782952 2014-08-13 18:06:06.977064 7f5c973a57a0 20 read_log 193305'18400 (0'0) promote a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by osd.356.0:169 2014-08-11 18:43:39.496343 2014-08-13 18:06:06.977076 7f5c973a57a0 20 read_log 193305'18401 (193305'18400) modify a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.951337.0:18035 2014-08-11 18:43:39.487984 2014-08-13 18:06:06.977090 7f5c973a57a0 20 read_log 193305'18402 (0'0) promote 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:179 2014-08-11 18:43:46.368230 2014-08-13 18:06:06.977109 7f5c973a57a0 20 read_log 193305'18403 (193305'18402) modify 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by client.951337.0:19935 2014-08-11 18:43:46.361211 2014-08-13 18:06:06.977124 7f5c973a57a0 20 read_log 193305'18404 (0'0) promote e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by osd.356.0:183 2014-08-11 18:43:48.768594 2014-08-13 18:06:06.977136 7f5c973a57a0 20 read_log 193305'18405 (193305'18404) modify e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.951337.0:20593 2014-08-11 18:43:48.761186 2014-08-13 18:06:06.977157 7f5c973a57a0 20 read_log 193305'18406 (0'0) promote e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by osd.356.0:205 2014-08-11 18:43:57.557515 2014-08-13 18:06:06.977171 7f5c973a57a0 20 read_log 193305'18407 (193305'18406) modify e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.951337.0:23034 2014-08-11 18:43:57.550580 2014-08-13 18:06:06.977184 7f5c973a57a0 20 read_log 193305'18408 (0'0) promote 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:211 2014-08-11 18:43:58.913423 2014-08-13 18:06:06.977196 7f5c973a57a0 20 read_log 193305'18409 (193305'18408) modify 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by client.951337.0:23421 2014-08-11 18:43:58.906515 2014-08-13 18:06:06.977209 7f5c973a57a0 20 read_log 193305'18410 (0'0) promote ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by osd.356.0:215 2014-08-11 18:44:04.046591 2014-08-13 18:06:06.977221 7f5c973a57a0 20 read_log 193305'18411 (193305'18410) modify ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.951337.0:24842 2014-08-11 18:44:04.038708 2014-08-13 18:06:06.977238 7f5c973a57a0 20 read_log 193305'18412 (0'0) promote 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by osd.356.0:221 2014-08-11 18:44:06.368856 2014-08-13 18:06:06.977253 7f5c973a57a0 20 read_log 193305'18413 (193305'18412) modify 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.951337.0:25471 2014-08-11 18:44:06.360640 2014-08-13 18:06:06.977265 7f5c973a57a0 20 read_log 193305'18414 (0'0) promote 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by osd.356.0:225 2014-08-11 18:44:10.944441 2014-08-13 18:06:06.977280 7f5c973a57a0 20 read_log 193305'18415 (193305'18414) modify 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.951337.0:26710 2014-08-11 18:44:10.936117 2014-08-13 18:06:06.977299 7f5c973a57a0 20 read_log 193305'18416 (0'0) promote 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by osd.356.0:245 2014-08-11 18:44:18.716670 2014-08-13 18:06:06.977311 7f5c973a57a0 20 read_log 193305'18417 (193305'18416) modify 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.951337.0:28825 2014-08-11 18:44:18.709267 2014-08-13 18:06:06.977326 7f5c973a57a0 20 read_log 193305'18418 (0'0) promote 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:257 2014-08-11 18:44:25.943866 2014-08-13 18:06:06.977341 7f5c973a57a0 20 read_log 193305'18419 (193305'18418) modify 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by client.951337.0:30753 2014-08-11 18:44:25.935534 2014-08-13 18:06:06.977356 7f5c973a57a0 20 read_log 193305'18420 (0'0) promote 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by osd.356.0:259 2014-08-11 18:44:26.934616 2014-08-13 18:06:06.977369 7f5c973a57a0 20 read_log 193305'18421 (193305'18420) modify 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.951337.0:31032 2014-08-11 18:44:26.927766 2014-08-13 18:06:06.977381 7f5c973a57a0 20 read_log 193305'18422 (0'0) promote 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by osd.356.0:261 2014-08-11 18:44:27.422222 2014-08-13 18:06:06.977418 7f5c973a57a0 20 read_log 193305'18423 (193305'18422) modify 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.951337.0:31171 2014-08-11 18:44:27.414085 2014-08-13 18:06:06.977435 7f5c973a57a0 20 read_log 193305'18424 (0'0) promote 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by osd.356.0:271 2014-08-11 18:44:32.093220 2014-08-13 18:06:06.977459 7f5c973a57a0 20 read_log 193305'18425 (193305'18424) modify 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.951337.0:32503 2014-08-11 18:44:32.086306 2014-08-13 18:06:06.977475 7f5c973a57a0 20 read_log 193305'18426 (193305'18363) modify 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by client.951337.0:33776 2014-08-11 18:44:34.640597 2014-08-13 18:06:06.977488 7f5c973a57a0 20 read_log 193305'18427 (193305'18365) modify 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.951337.0:34158 2014-08-11 18:44:34.826402 2014-08-13 18:06:06.977513 7f5c973a57a0 20 read_log 193305'18428 (193305'18367) modify 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.951337.0:35102 2014-08-11 18:44:35.387709 2014-08-13 18:06:06.977527 7f5c973a57a0 20 read_log 193305'18429 (193305'18369) modify d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.951337.0:35911 2014-08-11 18:44:35.809106 2014-08-13 18:06:06.977539 7f5c973a57a0 20 read_log 193305'18430 (193305'18371) modify f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.951337.0:35984 2014-08-11 18:44:35.850584 2014-08-13 18:06:06.977561 7f5c973a57a0 20 read_log 193305'18431 (193305'18373) modify 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.951337.0:39627 2014-08-11 18:44:38.117119 2014-08-13 18:06:06.977581 7f5c973a57a0 20 read_log 193305'18432 (193305'18375) modify 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by client.951337.0:39906 2014-08-11 18:44:38.223650 2014-08-13 18:06:06.977593 7f5c973a57a0 20 read_log 193305'18433 (193305'18377) modify 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by client.951337.0:40167 2014-08-11 18:44:38.317545 2014-08-13 18:06:06.977626 7f5c973a57a0 20 read_log 193305'18434 (193305'18379) modify 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.951337.0:41458 2014-08-11 18:44:38.758743 2014-08-13 18:06:06.977641 7f5c973a57a0 20 read_log 193305'18435 (193305'18381) modify a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.951337.0:42121 2014-08-11 18:44:39.114149 2014-08-13 18:06:06.977653 7f5c973a57a0 20 read_log 193305'18436 (193305'18383) modify f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by client.951337.0:44241 2014-08-11 18:44:39.830075 2014-08-13 18:06:06.977666 7f5c973a57a0 20 read_log 193305'18437 (193305'18385) modify 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.951337.0:44474 2014-08-11 18:44:39.890399 2014-08-13 18:06:06.977678 7f5c973a57a0 20 read_log 193305'18438 (193305'18387) modify 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by client.951337.0:47015 2014-08-11 18:44:40.651442 2014-08-13 18:06:06.977690 7f5c973a57a0 20 read_log 193305'18439 (193305'18389) modify 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.951337.0:47139 2014-08-11 18:44:40.683615 2014-08-13 18:06:06.977703 7f5c973a57a0 20 read_log 193305'18440 (193305'18391) modify 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by client.951337.0:47216 2014-08-11 18:44:40.704675 2014-08-13 18:06:06.977715 7f5c973a57a0 20 read_log 193305'18441 (193305'18393) modify eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.951337.0:49445 2014-08-11 18:44:41.367277 2014-08-13 18:06:06.977727 7f5c973a57a0 20 read_log 193305'18442 (193305'18395) modify 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.951337.0:49750 2014-08-11 18:44:41.461083 2014-08-13 18:06:06.977746 7f5c973a57a0 20 read_log 193305'18443 (193305'18397) modify 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by client.951337.0:50405 2014-08-11 18:44:41.655479 2014-08-13 18:06:06.977762 7f5c973a57a0 20 read_log 193305'18444 (193305'18399) modify a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.951337.0:50560 2014-08-11 18:44:41.698389 2014-08-13 18:06:06.977774 7f5c973a57a0 20 read_log 193305'18445 (193305'18401) modify a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.951337.0:50746 2014-08-11 18:44:41.750475 2014-08-13 18:06:06.977786 7f5c973a57a0 20 read_log 193305'18446 (193305'18403) modify 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by client.951337.0:52646 2014-08-11 18:44:42.293977 2014-08-13 18:06:06.977798 7f5c973a57a0 20 read_log 193305'18447 (193305'18405) modify e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.951337.0:53304 2014-08-11 18:44:42.488173 2014-08-13 18:06:06.977811 7f5c973a57a0 20 read_log 193305'18448 (193305'18407) modify e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.951337.0:55745 2014-08-11 18:44:43.219896 2014-08-13 18:06:06.977823 7f5c973a57a0 20 read_log 193305'18449 (193305'18409) modify 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by client.951337.0:56132 2014-08-11 18:44:43.330009 2014-08-13 18:06:06.977838 7f5c973a57a0 20 read_log 193305'18450 (193305'18411) modify ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.951337.0:57553 2014-08-11 18:44:43.792525 2014-08-13 18:06:06.977851 7f5c973a57a0 20 read_log 193305'18451 (193305'18413) modify 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.951337.0:58182 2014-08-11 18:44:43.969106 2014-08-13 18:06:06.977864 7f5c973a57a0 20 read_log 193305'18452 (193305'18415) modify 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.951337.0:59421 2014-08-11 18:44:44.321946 2014-08-13 18:06:06.977876 7f5c973a57a0 20 read_log 193305'18453 (193305'18417) modify 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.951337.0:61536 2014-08-11 18:44:44.950318 2014-08-13 18:06:06.977888 7f5c973a57a0 20 read_log 193305'18454 (193305'18419) modify 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by client.951337.0:63464 2014-08-11 18:44:45.515408 2014-08-13 18:06:06.977900 7f5c973a57a0 20 read_log 193305'18455 (193305'18421) modify 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.951337.0:63743 2014-08-11 18:44:45.596876 2014-08-13 18:06:06.977912 7f5c973a57a0 20 read_log 193305'18456 (193305'18423) modify 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.951337.0:63882 2014-08-11 18:44:45.636167 2014-08-13 18:06:06.977925 7f5c973a57a0 20 read_log 193305'18457 (193305'18425) modify 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.951337.0:65214 2014-08-11 18:44:46.052662 2014-08-13 18:06:06.977938 7f5c973a57a0 20 read_log 194209'18458 (192888'18324) clean a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:583 2014-08-12 14:07:41.714253 2014-08-13 18:06:06.977950 7f5c973a57a0 20 read_log 194209'18459 (192834'17424) clean abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.356.0:584 2014-08-12 14:07:41.714670 2014-08-13 18:06:06.977966 7f5c973a57a0 20 read_log 194209'18460 (192888'18314) clean 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:585 2014-08-12 14:07:41.715336 2014-08-13 18:06:06.977980 7f5c973a57a0 20 read_log 194209'18461 (192888'18124) clean f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:586 2014-08-12 14:07:41.715873 2014-08-13 18:06:06.978002 7f5c973a57a0 20 read_log 194209'18462 (193305'18436) clean f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:587 2014-08-12 14:07:41.724594 2014-08-13 18:06:06.978016 7f5c973a57a0 20 read_log 194209'18463 (192888'18320) clean b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:588 2014-08-12 14:07:41.727547 2014-08-13 18:06:06.978029 7f5c973a57a0 20 read_log 194209'18464 (192888'18342) clean e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:589 2014-08-12 14:07:41.733314 2014-08-13 18:06:06.978043 7f5c973a57a0 20 read_log 194209'18465 (193305'18454) clean 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:590 2014-08-12 14:07:41.733574 2014-08-13 18:06:06.978056 7f5c973a57a0 20 read_log 194209'18466 (193305'18446) clean 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:591 2014-08-12 14:07:41.736625 2014-08-13 18:06:06.978068 7f5c973a57a0 20 read_log 194209'18467 (193305'18443) clean 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:592 2014-08-12 14:07:41.739768 2014-08-13 18:06:06.978081 7f5c973a57a0 20 read_log 194209'18468 (192888'18132) clean c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:593 2014-08-12 14:07:41.741207 2014-08-13 18:06:06.978095 7f5c973a57a0 20 read_log 194209'18469 (192834'17448) clean 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.356.0:594 2014-08-12 14:07:41.747904 2014-08-13 18:06:06.978108 7f5c973a57a0 20 read_log 194209'18470 (193305'18440) clean 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:595 2014-08-12 14:07:41.750818 2014-08-13 18:06:06.978122 7f5c973a57a0 20 read_log 194209'18471 (192888'18334) clean ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:596 2014-08-12 14:07:41.751071 2014-08-13 18:06:06.978134 7f5c973a57a0 20 read_log 194209'18472 (192834'17330) clean f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.356.0:597 2014-08-12 14:07:41.754198 2014-08-13 18:06:06.978152 7f5c973a57a0 20 read_log 194209'18473 (192888'18139) clean 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:598 2014-08-12 14:07:41.757353 2014-08-13 18:06:06.978167 7f5c973a57a0 20 read_log 194209'18474 (193305'18432) clean 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:599 2014-08-12 14:07:41.760462 2014-08-13 18:06:06.978180 7f5c973a57a0 20 read_log 194209'18475 (192888'18107) clean 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:600 2014-08-12 14:07:41.763596 2014-08-13 18:06:06.978194 7f5c973a57a0 20 read_log 194209'18476 (192888'18296) clean 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:601 2014-08-12 14:07:41.766743 2014-08-13 18:06:06.978207 7f5c973a57a0 20 read_log 194209'18477 (192888'18098) clean 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:602 2014-08-12 14:07:41.769846 2014-08-13 18:06:06.978219 7f5c973a57a0 20 read_log 194209'18478 (193305'18433) clean 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:603 2014-08-12 14:07:41.775752 2014-08-13 18:06:06.978236 7f5c973a57a0 20 read_log 194209'18479 (192834'17847) clean adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.356.0:604 2014-08-12 14:07:41.776037 2014-08-13 18:06:06.978249 7f5c973a57a0 20 read_log 194209'18480 (192888'18347) clean e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:605 2014-08-12 14:07:41.778932 2014-08-13 18:06:06.978264 7f5c973a57a0 20 read_log 194209'18481 (193305'18426) clean 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:606 2014-08-12 14:07:41.782081 2014-08-13 18:06:06.978286 7f5c973a57a0 20 read_log 194209'18482 (192834'17434) clean 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.356.0:607 2014-08-12 14:07:41.785240 2014-08-13 18:06:06.978301 7f5c973a57a0 20 read_log 194209'18483 (192888'18302) clean e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:608 2014-08-12 14:07:41.788208 2014-08-13 18:06:06.978314 7f5c973a57a0 20 read_log 194209'18484 (192888'18343) clean b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:609 2014-08-12 14:07:41.791427 2014-08-13 18:06:06.978326 7f5c973a57a0 20 read_log 194209'18485 (193305'18449) clean 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:610 2014-08-12 14:07:41.794403 2014-08-13 18:06:06.978339 7f5c973a57a0 20 read_log 194209'18486 (192888'18348) clean c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:611 2014-08-12 14:07:41.797556 2014-08-13 18:06:06.978351 7f5c973a57a0 20 read_log 194209'18487 (192888'18110) clean 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:612 2014-08-12 14:07:41.800829 2014-08-13 18:06:06.978372 7f5c973a57a0 20 read_log 194209'18488 (192834'17944) clean e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.356.0:613 2014-08-12 14:07:41.803885 2014-08-13 18:06:06.978388 7f5c973a57a0 20 read_log 194209'18489 (192888'18136) clean 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:614 2014-08-12 14:07:41.807036 2014-08-13 18:06:06.978401 7f5c973a57a0 20 read_log 194209'18490 (192888'18333) clean 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:615 2014-08-12 14:07:41.809919 2014-08-13 18:06:06.978427 7f5c973a57a0 20 read_log 194209'18491 (192834'17396) clean 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.356.0:616 2014-08-12 14:07:41.813425 2014-08-13 18:06:06.978443 7f5c973a57a0 20 read_log 194209'18492 (192888'18345) clean 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:617 2014-08-12 14:07:41.817160 2014-08-13 18:06:06.978455 7f5c973a57a0 20 read_log 194209'18493 (192888'18137) clean 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:618 2014-08-12 14:07:41.820235 2014-08-13 18:06:06.978483 7f5c973a57a0 20 read_log 194209'18494 (192834'17338) clean 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.356.0:619 2014-08-12 14:07:41.823315 2014-08-13 18:06:06.978496 7f5c973a57a0 20 read_log 194209'18495 (192834'17932) clean f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.356.0:620 2014-08-12 14:07:41.826370 2014-08-13 18:06:06.978519 7f5c973a57a0 20 read_log 194209'18496 (192888'18114) clean 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:621 2014-08-12 14:07:41.829675 2014-08-13 18:06:06.978535 7f5c973a57a0 20 read_log 194209'18497 (193305'18438) clean 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:622 2014-08-12 14:07:41.832740 2014-08-13 18:06:06.978548 7f5c973a57a0 20 read_log 194209'18498 (192834'17446) clean 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.356.0:623 2014-08-12 14:07:41.855840 2014-08-13 18:06:06.978561 7f5c973a57a0 20 read_log 194209'18499 (194209'18458) delete a4c2c7b7/benchmark_data_storage0111-ib.pocstor_2228_object16068/head//24 by osd.356.0:624 2014-08-12 14:07:41.867000 2014-08-13 18:06:06.978573 7f5c973a57a0 20 read_log 194209'18500 (194209'18459) delete abe9c7b7/benchmark_data_storage0111-ib.pocstor_26659_object14514/head//24 by osd.356.0:625 2014-08-12 14:07:41.867382 2014-08-13 18:06:06.978586 7f5c973a57a0 20 read_log 194209'18501 (194209'18460) delete 5628d7b7/benchmark_data_storage0111-ib.pocstor_2228_object11968/head//24 by osd.356.0:626 2014-08-12 14:07:41.870105 2014-08-13 18:06:06.978611 7f5c973a57a0 20 read_log 194209'18502 (194209'18461) delete f969d7b7/benchmark_data_storage0111-ib.pocstor_41097_object14441/head//24 by osd.356.0:627 2014-08-12 14:07:41.870718 2014-08-13 18:06:06.978636 7f5c973a57a0 20 read_log 194209'18503 (194209'18462) delete f8ad7b7/benchmark_data_storage0111-ib.pocstor_62621_object11529/head//24 by osd.356.0:628 2014-08-12 14:07:41.871078 2014-08-13 18:06:06.978649 7f5c973a57a0 20 read_log 194209'18504 (194209'18465) delete 8863e7b7/benchmark_data_storage0111-ib.pocstor_62621_object30752/head//24 by osd.356.0:629 2014-08-12 14:07:41.874977 2014-08-13 18:06:06.978662 7f5c973a57a0 20 read_log 194209'18505 (194209'18466) delete 1973e7b7/benchmark_data_storage0111-ib.pocstor_62621_object19934/head//24 by osd.356.0:630 2014-08-12 14:07:41.875363 2014-08-13 18:06:06.978674 7f5c973a57a0 20 read_log 194209'18506 (194209'18467) delete 2ab4e7b7/benchmark_data_storage0111-ib.pocstor_62621_object17693/head//24 by osd.356.0:631 2014-08-12 14:07:41.875961 2014-08-13 18:06:06.978686 7f5c973a57a0 20 read_log 194209'18507 (194209'18470) delete 54630bb7/benchmark_data_storage0111-ib.pocstor_62621_object14504/head//24 by osd.356.0:632 2014-08-12 14:07:41.879498 2014-08-13 18:06:06.978698 7f5c973a57a0 20 read_log 194209'18508 (194209'18474) delete 3c583bb7/benchmark_data_storage0111-ib.pocstor_62621_object7194/head//24 by osd.356.0:633 2014-08-12 14:07:41.880567 2014-08-13 18:06:06.978711 7f5c973a57a0 20 read_log 194209'18509 (194209'18478) delete 53316bb7/benchmark_data_storage0111-ib.pocstor_62621_object7455/head//24 by osd.356.0:634 2014-08-12 14:07:41.884031 2014-08-13 18:06:06.978723 7f5c973a57a0 20 read_log 194209'18510 (194209'18481) delete 70097bb7/benchmark_data_storage0111-ib.pocstor_62621_object1064/head//24 by osd.356.0:635 2014-08-12 14:07:41.884480 2014-08-13 18:06:06.978737 7f5c973a57a0 20 read_log 194209'18511 (194209'18485) delete 298d9bb7/benchmark_data_storage0111-ib.pocstor_62621_object23420/head//24 by osd.356.0:636 2014-08-12 14:07:41.885428 2014-08-13 18:06:06.978749 7f5c973a57a0 20 read_log 194209'18512 (194209'18497) delete 1433dbb7/benchmark_data_storage0111-ib.pocstor_62621_object14303/head//24 by osd.356.0:637 2014-08-12 14:07:41.889031 2014-08-13 18:06:06.978762 7f5c973a57a0 20 read_log 194209'18513 (194209'18472) delete f8ba2bb7/benchmark_data_storage0111-ib.pocstor_26659_object1254/head//24 by osd.356.0:638 2014-08-12 14:07:51.998618 2014-08-13 18:06:06.978774 7f5c973a57a0 20 read_log 194209'18514 (194209'18473) delete 72103bb7/benchmark_data_storage0111-ib.pocstor_41097_object23748/head//24 by osd.356.0:639 2014-08-12 14:07:51.999204 2014-08-13 18:06:06.978786 7f5c973a57a0 20 read_log 194209'18515 (194209'18475) delete 74a83bb7/benchmark_data_storage0111-ib.pocstor_41097_object8849/head//24 by osd.356.0:640 2014-08-12 14:07:51.999657 2014-08-13 18:06:06.978799 7f5c973a57a0 20 read_log 194209'18516 (194209'18476) delete 925b3bb7/benchmark_data_storage0111-ib.pocstor_2228_object4107/head//24 by osd.356.0:641 2014-08-12 14:07:52.000091 2014-08-13 18:06:06.978811 7f5c973a57a0 20 read_log 194209'18517 (194209'18477) delete 91e44bb7/benchmark_data_storage0111-ib.pocstor_41097_object4423/head//24 by osd.356.0:642 2014-08-12 14:07:52.004933 2014-08-13 18:06:06.978824 7f5c973a57a0 20 read_log 194209'18518 (194209'18479) delete adc37bb7/benchmark_data_storage0111-ib.pocstor_41526_object5973/head//24 by osd.356.0:643 2014-08-12 14:07:52.005357 2014-08-13 18:06:06.978836 7f5c973a57a0 20 read_log 194209'18519 (194209'18480) delete e7957bb7/benchmark_data_storage0111-ib.pocstor_2228_object25246/head//24 by osd.356.0:644 2014-08-12 14:07:52.005735 2014-08-13 18:06:06.978849 7f5c973a57a0 20 read_log 194209'18520 (194209'18482) delete 874a8bb7/benchmark_data_storage0111-ib.pocstor_26659_object16238/head//24 by osd.356.0:645 2014-08-12 14:07:52.006337 2014-08-13 18:06:06.978861 7f5c973a57a0 20 read_log 194209'18521 (194209'18483) delete e74d8bb7/benchmark_data_storage0111-ib.pocstor_2228_object6482/head//24 by osd.356.0:646 2014-08-12 14:07:52.009860 2014-08-13 18:06:06.978922 7f5c973a57a0 20 read_log 194209'18522 (194209'18484) delete b99b9bb7/benchmark_data_storage0111-ib.pocstor_2228_object23953/head//24 by osd.356.0:647 2014-08-12 14:07:52.010274 2014-08-13 18:06:06.978940 7f5c973a57a0 20 read_log 194209'18523 (194209'18486) delete c2dd9bb7/benchmark_data_storage0111-ib.pocstor_2228_object25725/head//24 by osd.356.0:648 2014-08-12 14:07:52.011253 2014-08-13 18:06:06.978953 7f5c973a57a0 20 read_log 194209'18524 (194209'18487) delete 7934abb7/benchmark_data_storage0111-ib.pocstor_41097_object9865/head//24 by osd.356.0:649 2014-08-12 14:07:52.011672 2014-08-13 18:06:06.978965 7f5c973a57a0 20 read_log 194209'18525 (194209'18488) delete e408abb7/benchmark_data_storage0111-ib.pocstor_41526_object20389/head//24 by osd.356.0:650 2014-08-12 14:07:52.014938 2014-08-13 18:06:06.978978 7f5c973a57a0 20 read_log 194209'18526 (194209'18489) delete 395babb7/benchmark_data_storage0111-ib.pocstor_41097_object21874/head//24 by osd.356.0:651 2014-08-12 14:07:52.015324 2014-08-13 18:06:06.979005 7f5c973a57a0 20 read_log 194209'18527 (194209'18490) delete 27feabb7/benchmark_data_storage0111-ib.pocstor_2228_object19349/head//24 by osd.356.0:652 2014-08-12 14:07:52.016173 2014-08-13 18:06:06.979020 7f5c973a57a0 20 read_log 194209'18528 (194209'18491) delete 81afabb7/benchmark_data_storage0111-ib.pocstor_26659_object11162/head//24 by osd.356.0:653 2014-08-12 14:07:52.016573 2014-08-13 18:06:06.979033 7f5c973a57a0 20 read_log 194209'18529 (194209'18492) delete 7409bbb7/benchmark_data_storage0111-ib.pocstor_2228_object24606/head//24 by osd.356.0:654 2014-08-12 14:07:52.020299 2014-08-13 18:06:06.979048 7f5c973a57a0 20 read_log 194209'18530 (194209'18493) delete 5c6abbb7/benchmark_data_storage0111-ib.pocstor_41097_object22655/head//24 by osd.356.0:655 2014-08-12 14:07:52.020707 2014-08-13 18:06:06.979062 7f5c973a57a0 20 read_log 194209'18531 (194209'18494) delete 1fb4cbb7/benchmark_data_storage0111-ib.pocstor_26659_object2833/head//24 by osd.356.0:656 2014-08-12 14:07:52.021098 2014-08-13 18:06:06.979077 7f5c973a57a0 20 read_log 194209'18532 (194209'18495) delete f30acbb7/benchmark_data_storage0111-ib.pocstor_41526_object18977/head//24 by osd.356.0:657 2014-08-12 14:07:52.021690 2014-08-13 18:06:06.979090 7f5c973a57a0 20 read_log 194209'18533 (194209'18496) delete 99c1dbb7/benchmark_data_storage0111-ib.pocstor_41097_object10987/head//24 by osd.356.0:658 2014-08-12 14:07:52.024843 2014-08-13 18:06:06.979105 7f5c973a57a0 20 read_log 194209'18534 (194209'18498) delete 1dd3dbb7/benchmark_data_storage0111-ib.pocstor_26659_object17349/head//24 by osd.356.0:659 2014-08-12 14:07:52.025263 2014-08-13 18:06:06.979117 7f5c973a57a0 20 read_log 194209'18535 (194209'18463) delete b6abd7b7/benchmark_data_storage0111-ib.pocstor_2228_object14042/head//24 by osd.356.0:660 2014-08-12 14:07:52.057844 2014-08-13 18:06:06.979132 7f5c973a57a0 20 read_log 194209'18536 (194209'18464) delete e800e7b7/benchmark_data_storage0111-ib.pocstor_2228_object23657/head//24 by osd.356.0:661 2014-08-12 14:07:52.058288 2014-08-13 18:06:06.979145 7f5c973a57a0 20 read_log 194209'18537 (194209'18468) delete c256e7b7/benchmark_data_storage0111-ib.pocstor_41097_object17653/head//24 by osd.356.0:662 2014-08-12 14:07:52.060694 2014-08-13 18:06:06.979165 7f5c973a57a0 20 read_log 194209'18538 (194209'18469) delete 1cc0f7b7/benchmark_data_storage0111-ib.pocstor_26659_object17697/head//24 by osd.356.0:663 2014-08-12 14:07:52.061031 2014-08-13 18:06:06.979178 7f5c973a57a0 20 read_log 194209'18539 (194209'18471) delete ef0e1bb7/benchmark_data_storage0111-ib.pocstor_2228_object19447/head//24 by osd.356.0:664 2014-08-12 14:07:52.061321 2014-08-13 18:06:06.979192 7f5c973a57a0 20 read_log 194215'18540 (194215'18540) modify 3b7/hit_set_24.3b7_archive_2014-08-12 15:32:25.978488_2014-08-12 15:50:56.911217/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:56.911482 2014-08-13 18:06:06.979216 7f5c973a57a0 20 read_log 194215'18541 (193305'18361) delete 3b7/hit_set_24.3b7_archive_2014-08-11 18:37:35.561357_2014-08-11 18:42:37.204550/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:50:56.911482 2014-08-13 18:06:06.979232 7f5c973a57a0 20 read_log 194215'18542 (193305'18427) clean 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by osd.356.0:1236 2014-08-12 15:50:56.914547 2014-08-13 18:06:06.979248 7f5c973a57a0 20 read_log 194215'18543 (194215'18542) delete 7c4c03b7/benchmark_data_storage0111-ib.pocstor_62621_object1446/head//24 by client.2174595.0:196574 0.000000 2014-08-13 18:06:06.979260 7f5c973a57a0 20 read_log 194215'18544 (192834'17855) clean d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by osd.356.0:1238 2014-08-12 15:50:56.923968 2014-08-13 18:06:06.979273 7f5c973a57a0 20 read_log 194215'18545 (194215'18544) delete d12d13b7/benchmark_data_storage0111-ib.pocstor_41526_object6961/head//24 by client.2174595.0:196576 0.000000 2014-08-13 18:06:06.979288 7f5c973a57a0 20 read_log 194215'18546 (192888'18151) clean 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by osd.356.0:1240 2014-08-12 15:50:56.931770 2014-08-13 18:06:06.979301 7f5c973a57a0 20 read_log 194215'18547 (194215'18546) delete 3c8323b7/benchmark_data_storage0111-ib.pocstor_41097_object28567/head//24 by client.2174595.0:196578 0.000000 2014-08-13 18:06:06.979311 7f5c973a57a0 20 read_log 194215'18548 (192834'17919) clean eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by osd.356.0:1242 2014-08-12 15:50:56.939662 2014-08-13 18:06:06.979324 7f5c973a57a0 20 read_log 194215'18549 (194215'18548) delete eed433b7/benchmark_data_storage0111-ib.pocstor_41526_object16837/head//24 by client.2174595.0:196580 0.000000 2014-08-13 18:06:06.979338 7f5c973a57a0 20 read_log 194215'18550 (193305'18451) clean 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by osd.356.0:1244 2014-08-12 15:50:56.947360 2014-08-13 18:06:06.979351 7f5c973a57a0 20 read_log 194215'18551 (194215'18550) delete 373933b7/benchmark_data_storage0111-ib.pocstor_62621_object25470/head//24 by client.2174595.0:196582 0.000000 2014-08-13 18:06:06.979371 7f5c973a57a0 20 read_log 194215'18552 (192888'18117) clean c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by osd.356.0:1246 2014-08-12 15:50:56.955114 2014-08-13 18:06:06.979390 7f5c973a57a0 20 read_log 194215'18553 (194215'18552) delete c83b33b7/benchmark_data_storage0111-ib.pocstor_41097_object12607/head//24 by client.2174595.0:196584 0.000000 2014-08-13 18:06:06.979401 7f5c973a57a0 20 read_log 194215'18554 (192888'18307) clean 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by osd.356.0:1248 2014-08-12 15:50:56.962192 2014-08-13 18:06:06.979424 7f5c973a57a0 20 read_log 194215'18555 (194215'18554) delete 903343b7/benchmark_data_storage0111-ib.pocstor_2228_object8078/head//24 by client.2174595.0:196586 0.000000 2014-08-13 18:06:06.979436 7f5c973a57a0 20 read_log 194215'18556 (192888'18147) clean d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by osd.356.0:1250 2014-08-12 15:50:56.969710 2014-08-13 18:06:06.979448 7f5c973a57a0 20 read_log 194215'18557 (194215'18556) delete d39343b7/benchmark_data_storage0111-ib.pocstor_41097_object27295/head//24 by client.2174595.0:196588 0.000000 2014-08-13 18:06:06.979467 7f5c973a57a0 20 read_log 194215'18558 (192888'18120) clean 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by osd.356.0:1252 2014-08-12 15:50:56.977487 2014-08-13 18:06:06.979483 7f5c973a57a0 20 read_log 194215'18559 (194215'18558) delete 360e43b7/benchmark_data_storage0111-ib.pocstor_41097_object13659/head//24 by client.2174595.0:196590 0.000000 2014-08-13 18:06:06.979501 7f5c973a57a0 20 read_log 194215'18560 (193305'18452) clean 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by osd.356.0:1254 2014-08-12 15:50:56.985314 2014-08-13 18:06:06.979525 7f5c973a57a0 20 read_log 194215'18561 (194215'18560) delete 442463b7/benchmark_data_storage0111-ib.pocstor_62621_object26709/head//24 by client.2174595.0:196592 0.000000 2014-08-13 18:06:06.979537 7f5c973a57a0 20 read_log 194215'18562 (192888'18123) clean 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by osd.356.0:1256 2014-08-12 15:50:56.992600 2014-08-13 18:06:06.979549 7f5c973a57a0 20 read_log 194215'18563 (194215'18562) delete 3ad273b7/benchmark_data_storage0111-ib.pocstor_41097_object14273/head//24 by client.2174595.0:196594 0.000000 2014-08-13 18:06:06.979560 7f5c973a57a0 20 read_log 194215'18564 (192888'18129) clean bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by osd.356.0:1258 2014-08-12 15:50:57.001002 2014-08-13 18:06:06.979588 7f5c973a57a0 20 read_log 194215'18565 (194215'18564) delete bd7583b7/benchmark_data_storage0111-ib.pocstor_41097_object16943/head//24 by client.2174595.0:196596 0.000000 2014-08-13 18:06:06.979600 7f5c973a57a0 20 read_log 194215'18566 (192834'17401) clean b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by osd.356.0:1260 2014-08-12 15:50:57.008322 2014-08-13 18:06:06.979613 7f5c973a57a0 20 read_log 194215'18567 (194215'18566) delete b18b93b7/benchmark_data_storage0111-ib.pocstor_26659_object12428/head//24 by client.2174595.0:196598 0.000000 2014-08-13 18:06:06.979624 7f5c973a57a0 20 read_log 194215'18568 (193305'18444) clean a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by osd.356.0:1262 2014-08-12 15:50:57.015506 2014-08-13 18:06:06.979642 7f5c973a57a0 20 read_log 194215'18569 (194215'18568) delete a5ce93b7/benchmark_data_storage0111-ib.pocstor_62621_object17848/head//24 by client.2174595.0:196600 0.000000 2014-08-13 18:06:06.979654 7f5c973a57a0 20 read_log 194215'18570 (193305'18434) clean 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by osd.356.0:1264 2014-08-12 15:50:57.022820 2014-08-13 18:06:06.979666 7f5c973a57a0 20 read_log 194215'18571 (194215'18570) delete 106f93b7/benchmark_data_storage0111-ib.pocstor_62621_object8746/head//24 by client.2174595.0:196602 0.000000 2014-08-13 18:06:06.979677 7f5c973a57a0 20 read_log 194215'18572 (192834'17367) clean 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by osd.356.0:1266 2014-08-12 15:50:57.029742 2014-08-13 18:06:06.979700 7f5c973a57a0 20 read_log 194215'18573 (194215'18572) delete 2caca3b7/benchmark_data_storage0111-ib.pocstor_26659_object7309/head//24 by client.2174595.0:196604 0.000000 2014-08-13 18:06:06.979715 7f5c973a57a0 20 read_log 194215'18574 (193305'18435) clean a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by osd.356.0:1268 2014-08-12 15:50:57.037354 2014-08-13 18:06:06.979731 7f5c973a57a0 20 read_log 194215'18575 (194215'18574) delete a711b3b7/benchmark_data_storage0111-ib.pocstor_62621_object9409/head//24 by client.2174595.0:196606 0.000000 2014-08-13 18:06:06.979742 7f5c973a57a0 20 read_log 194215'18576 (192834'17439) clean 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by osd.356.0:1270 2014-08-12 15:50:57.045101 2014-08-13 18:06:06.979756 7f5c973a57a0 20 read_log 194215'18577 (194215'18576) delete 5856c3b7/benchmark_data_storage0111-ib.pocstor_26659_object16642/head//24 by client.2174595.0:196608 0.000000 2014-08-13 18:06:06.979767 7f5c973a57a0 20 read_log 194215'18578 (192834'17807) clean 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by osd.356.0:1272 2014-08-12 15:50:57.052539 2014-08-13 18:06:06.979816 7f5c973a57a0 20 read_log 194215'18579 (194215'18578) delete 569ec3b7/benchmark_data_storage0111-ib.pocstor_41526_object1004/head//24 by client.2174595.0:196610 0.000000 2014-08-13 18:06:06.979840 7f5c973a57a0 20 read_log 194215'18580 (192888'18305) clean 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by osd.356.0:1274 2014-08-12 15:50:57.060019 2014-08-13 18:06:06.979855 7f5c973a57a0 20 read_log 194215'18581 (194215'18580) delete 1684d3b7/benchmark_data_storage0111-ib.pocstor_2228_object7763/head//24 by client.2174595.0:196612 0.000000 2014-08-13 18:06:06.979867 7f5c973a57a0 20 read_log 194215'18582 (192888'18113) clean b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by osd.356.0:1276 2014-08-12 15:50:57.067458 2014-08-13 18:06:06.979890 7f5c973a57a0 20 read_log 194215'18583 (194215'18582) delete b4e4d3b7/benchmark_data_storage0111-ib.pocstor_41097_object10985/head//24 by client.2174595.0:196614 0.000000 2014-08-13 18:06:06.979905 7f5c973a57a0 20 read_log 194215'18584 (192888'18156) clean 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by osd.356.0:1278 2014-08-12 15:50:57.075594 2014-08-13 18:06:06.979918 7f5c973a57a0 20 read_log 194215'18585 (194215'18584) delete 2f76d3b7/benchmark_data_storage0111-ib.pocstor_41097_object30006/head//24 by client.2174595.0:196616 0.000000 2014-08-13 18:06:06.979929 7f5c973a57a0 20 read_log 194215'18586 (192834'17844) clean acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by osd.356.0:1280 2014-08-12 15:50:57.081661 2014-08-13 18:06:06.979942 7f5c973a57a0 20 read_log 194215'18587 (194215'18586) delete acbed3b7/benchmark_data_storage0111-ib.pocstor_41526_object5365/head//24 by client.2174595.0:196618 0.000000 2014-08-13 18:06:06.979952 7f5c973a57a0 20 read_log 194215'18588 (192888'18329) clean bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by osd.356.0:1282 2014-08-12 15:50:57.087674 2014-08-13 18:06:06.979965 7f5c973a57a0 20 read_log 194215'18589 (194215'18588) delete bb5de3b7/benchmark_data_storage0111-ib.pocstor_2228_object18006/head//24 by client.2174595.0:196620 0.000000 2014-08-13 18:06:06.979976 7f5c973a57a0 20 read_log 194215'18590 (192834'17346) clean f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by osd.356.0:1284 2014-08-12 15:50:57.093834 2014-08-13 18:06:06.979988 7f5c973a57a0 20 read_log 194215'18591 (194215'18590) delete f73f3b7/benchmark_data_storage0111-ib.pocstor_26659_object4042/head//24 by client.2174595.0:196622 0.000000 2014-08-13 18:06:06.979999 7f5c973a57a0 20 read_log 194215'18592 (192888'18115) clean f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by osd.356.0:1286 2014-08-12 15:50:57.100230 2014-08-13 18:06:06.980011 7f5c973a57a0 20 read_log 194215'18593 (194215'18592) delete f304f3b7/benchmark_data_storage0111-ib.pocstor_41097_object11952/head//24 by client.2174595.0:196624 0.000000 2014-08-13 18:06:06.980022 7f5c973a57a0 20 read_log 194215'18594 (192888'18308) clean a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by osd.356.0:1288 2014-08-12 15:50:57.106344 2014-08-13 18:06:06.980035 7f5c973a57a0 20 read_log 194215'18595 (194215'18594) delete a006f3b7/benchmark_data_storage0111-ib.pocstor_2228_object8776/head//24 by client.2174595.0:196626 0.000000 2014-08-13 18:06:06.980046 7f5c973a57a0 20 read_log 194215'18596 (192888'18126) clean 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by osd.356.0:1290 2014-08-12 15:50:57.112373 2014-08-13 18:06:06.980058 7f5c973a57a0 20 read_log 194215'18597 (194215'18596) delete 60a507b7/benchmark_data_storage0111-ib.pocstor_41097_object16338/head//24 by client.2174595.0:196628 0.000000 2014-08-13 18:06:06.980069 7f5c973a57a0 20 read_log 194215'18598 (193305'18448) clean e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by osd.356.0:1292 2014-08-12 15:50:57.117962 2014-08-13 18:06:06.980081 7f5c973a57a0 20 read_log 194215'18599 (194215'18598) delete e38707b7/benchmark_data_storage0111-ib.pocstor_62621_object23033/head//24 by client.2174595.0:196630 0.000000 2014-08-13 18:06:06.980102 7f5c973a57a0 20 read_log 194215'18600 (192888'18321) clean ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by osd.356.0:1294 2014-08-12 15:50:57.124127 2014-08-13 18:06:06.980116 7f5c973a57a0 20 read_log 194215'18601 (194215'18600) delete ed0907b7/benchmark_data_storage0111-ib.pocstor_2228_object15491/head//24 by client.2174595.0:196632 0.000000 2014-08-13 18:06:06.980127 7f5c973a57a0 20 read_log 194215'18602 (192888'18102) clean 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by osd.356.0:1296 2014-08-12 15:50:57.130579 2014-08-13 18:06:06.980155 7f5c973a57a0 20 read_log 194215'18603 (194215'18602) delete 45e907b7/benchmark_data_storage0111-ib.pocstor_41097_object6692/head//24 by client.2174595.0:196634 0.000000 2014-08-13 18:06:06.980171 7f5c973a57a0 20 read_log 194215'18604 (192834'17456) clean f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by osd.356.0:1298 2014-08-12 15:50:57.136881 2014-08-13 18:06:06.980184 7f5c973a57a0 20 read_log 194215'18605 (194215'18604) delete f20b17b7/benchmark_data_storage0111-ib.pocstor_26659_object18604/head//24 by client.2174595.0:196636 0.000000 2014-08-13 18:06:06.980195 7f5c973a57a0 20 read_log 194215'18606 (192888'18301) clean 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by osd.356.0:1300 2014-08-12 15:50:57.143069 2014-08-13 18:06:06.980207 7f5c973a57a0 20 read_log 194215'18607 (194215'18606) delete 981c17b7/benchmark_data_storage0111-ib.pocstor_2228_object6394/head//24 by client.2174595.0:196638 0.000000 2014-08-13 18:06:06.980218 7f5c973a57a0 20 read_log 194215'18608 (192834'17876) clean 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by osd.356.0:1302 2014-08-12 15:50:57.149343 2014-08-13 18:06:06.980233 7f5c973a57a0 20 read_log 194215'18609 (194215'18608) delete 442527b7/benchmark_data_storage0111-ib.pocstor_41526_object10108/head//24 by client.2174595.0:196640 0.000000 2014-08-13 18:06:06.980244 7f5c973a57a0 20 read_log 194215'18610 (192834'17428) clean 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by osd.356.0:1304 2014-08-12 15:50:57.155240 2014-08-13 18:06:06.980256 7f5c973a57a0 20 read_log 194215'18611 (194215'18610) delete 323537b7/benchmark_data_storage0111-ib.pocstor_26659_object15457/head//24 by client.2174595.0:196642 0.000000 2014-08-13 18:06:06.980267 7f5c973a57a0 20 read_log 194215'18612 (192888'18119) clean 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by osd.356.0:1306 2014-08-12 15:50:57.161080 2014-08-13 18:06:06.980286 7f5c973a57a0 20 read_log 194215'18613 (194215'18612) delete 948837b7/benchmark_data_storage0111-ib.pocstor_41097_object13447/head//24 by client.2174595.0:196644 0.000000 2014-08-13 18:06:06.980296 7f5c973a57a0 20 read_log 194215'18614 (193305'18441) clean eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by osd.356.0:1308 2014-08-12 15:50:57.167313 2014-08-13 18:06:06.980309 7f5c973a57a0 20 read_log 194215'18615 (194215'18614) delete eb2937b7/benchmark_data_storage0111-ib.pocstor_62621_object16733/head//24 by client.2174595.0:196646 0.000000 2014-08-13 18:06:06.980322 7f5c973a57a0 20 read_log 194215'18616 (193305'18430) clean f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by osd.356.0:1310 2014-08-12 15:50:57.174337 2014-08-13 18:06:06.980337 7f5c973a57a0 20 read_log 194215'18617 (194215'18616) delete f9ec37b7/benchmark_data_storage0111-ib.pocstor_62621_object3272/head//24 by client.2174595.0:196648 0.000000 2014-08-13 18:06:06.980358 7f5c973a57a0 20 read_log 194215'18618 (193305'18455) clean 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by osd.356.0:1312 2014-08-12 15:50:57.181825 2014-08-13 18:06:06.980375 7f5c973a57a0 20 read_log 194215'18619 (194215'18618) delete 2fad37b7/benchmark_data_storage0111-ib.pocstor_62621_object31031/head//24 by client.2174595.0:196650 0.000000 2014-08-13 18:06:06.980427 7f5c973a57a0 20 read_log 194215'18620 (192834'17834) clean 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by osd.356.0:1314 2014-08-12 15:50:57.189768 2014-08-13 18:06:06.980454 7f5c973a57a0 20 read_log 194215'18621 (194215'18620) delete 16f647b7/benchmark_data_storage0111-ib.pocstor_41526_object3862/head//24 by client.2174595.0:196652 0.000000 2014-08-13 18:06:06.980471 7f5c973a57a0 20 read_log 194215'18622 (192834'17921) clean 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by osd.356.0:1316 2014-08-12 15:50:57.197224 2014-08-13 18:06:06.980484 7f5c973a57a0 20 read_log 194215'18623 (194215'18622) delete 13c847b7/benchmark_data_storage0111-ib.pocstor_41526_object17273/head//24 by client.2174595.0:196654 0.000000 2014-08-13 18:06:06.980495 7f5c973a57a0 20 read_log 194215'18624 (192888'18127) clean eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by osd.356.0:1318 2014-08-12 15:50:57.204537 2014-08-13 18:06:06.980507 7f5c973a57a0 20 read_log 194215'18625 (194215'18624) delete eaa167b7/benchmark_data_storage0111-ib.pocstor_41097_object16371/head//24 by client.2174595.0:196656 0.000000 2014-08-13 18:06:06.980526 7f5c973a57a0 20 read_log 194215'18626 (193305'18428) clean 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by osd.356.0:1320 2014-08-12 15:50:57.211494 2014-08-13 18:06:06.980539 7f5c973a57a0 20 read_log 194215'18627 (194215'18626) delete 44f167b7/benchmark_data_storage0111-ib.pocstor_62621_object2390/head//24 by client.2174595.0:196658 0.000000 2014-08-13 18:06:06.980550 7f5c973a57a0 20 read_log 194215'18628 (192834'17477) clean 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by osd.356.0:1322 2014-08-12 15:50:57.218445 2014-08-13 18:06:06.980562 7f5c973a57a0 20 read_log 194215'18629 (194215'18628) delete 895667b7/benchmark_data_storage0111-ib.pocstor_26659_object21492/head//24 by client.2174595.0:196660 0.000000 2014-08-13 18:06:06.980573 7f5c973a57a0 20 read_log 194215'18630 (192834'17860) clean 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by osd.356.0:1324 2014-08-12 15:50:57.225519 2014-08-13 18:06:06.980585 7f5c973a57a0 20 read_log 194215'18631 (194215'18630) delete 3ade67b7/benchmark_data_storage0111-ib.pocstor_41526_object8147/head//24 by client.2174595.0:196662 0.000000 2014-08-13 18:06:06.980596 7f5c973a57a0 20 read_log 194215'18632 (193305'18431) clean 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by osd.356.0:1326 2014-08-12 15:50:57.232637 2014-08-13 18:06:06.980608 7f5c973a57a0 20 read_log 194215'18633 (194215'18632) delete 2a0f67b7/benchmark_data_storage0111-ib.pocstor_62621_object6915/head//24 by client.2174595.0:196664 0.000000 2014-08-13 18:06:06.980619 7f5c973a57a0 20 read_log 194215'18634 (193305'18453) clean 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by osd.356.0:1328 2014-08-12 15:50:57.239435 2014-08-13 18:06:06.980631 7f5c973a57a0 20 read_log 194215'18635 (194215'18634) delete 3bfc77b7/benchmark_data_storage0111-ib.pocstor_62621_object28824/head//24 by client.2174595.0:196666 0.000000 2014-08-13 18:06:06.980642 7f5c973a57a0 20 read_log 194215'18636 (192888'18122) clean 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by osd.356.0:1330 2014-08-12 15:50:57.246442 2014-08-13 18:06:06.980661 7f5c973a57a0 20 read_log 194215'18637 (194215'18636) delete 16ef77b7/benchmark_data_storage0111-ib.pocstor_41097_object13923/head//24 by client.2174595.0:196668 0.000000 2014-08-13 18:06:06.980674 7f5c973a57a0 20 read_log 194215'18638 (192888'18303) clean 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by osd.356.0:1332 2014-08-12 15:50:57.253740 2014-08-13 18:06:06.980689 7f5c973a57a0 20 read_log 194215'18639 (194215'18638) delete 781987b7/benchmark_data_storage0111-ib.pocstor_2228_object6577/head//24 by client.2174595.0:196670 0.000000 2014-08-13 18:06:06.980707 7f5c973a57a0 20 read_log 194215'18640 (192888'18311) clean d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by osd.356.0:1334 2014-08-12 15:50:57.261632 2014-08-13 18:06:06.980722 7f5c973a57a0 20 read_log 194215'18641 (194215'18640) delete d71f87b7/benchmark_data_storage0111-ib.pocstor_2228_object9518/head//24 by client.2174595.0:196672 0.000000 2014-08-13 18:06:06.980733 7f5c973a57a0 20 read_log 194215'18642 (193305'18445) clean a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by osd.356.0:1336 2014-08-12 15:50:57.269743 2014-08-13 18:06:06.980746 7f5c973a57a0 20 read_log 194215'18643 (194215'18642) delete a83697b7/benchmark_data_storage0111-ib.pocstor_62621_object18034/head//24 by client.2174595.0:196674 0.000000 2014-08-13 18:06:06.980756 7f5c973a57a0 20 read_log 194215'18644 (192834'17936) clean 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by osd.356.0:1338 2014-08-12 15:50:57.277575 2014-08-13 18:06:06.980769 7f5c973a57a0 20 read_log 194215'18645 (194215'18644) delete 549e97b7/benchmark_data_storage0111-ib.pocstor_41526_object19279/head//24 by client.2174595.0:196676 0.000000 2014-08-13 18:06:06.980779 7f5c973a57a0 20 read_log 194215'18646 (192888'18152) clean e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by osd.356.0:1340 2014-08-12 15:50:57.284595 2014-08-13 18:06:06.980791 7f5c973a57a0 20 read_log 194215'18647 (194215'18646) delete e5e2a7b7/benchmark_data_storage0111-ib.pocstor_41097_object29030/head//24 by client.2174595.0:196678 0.000000 2014-08-13 18:06:06.980802 7f5c973a57a0 20 read_log 194215'18648 (192888'18327) clean e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by osd.356.0:1342 2014-08-12 15:50:57.291978 2014-08-13 18:06:06.980814 7f5c973a57a0 20 read_log 194215'18649 (194215'18648) delete e6c4a7b7/benchmark_data_storage0111-ib.pocstor_2228_object16912/head//24 by client.2174595.0:196680 0.000000 2014-08-13 18:06:06.980825 7f5c973a57a0 20 read_log 194215'18650 (192888'18340) clean 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by osd.356.0:1344 2014-08-12 15:50:57.299059 2014-08-13 18:06:06.980837 7f5c973a57a0 20 read_log 194215'18651 (194215'18650) delete 17a7b7/benchmark_data_storage0111-ib.pocstor_2228_object23381/head//24 by client.2174595.0:196682 0.000000 2014-08-13 18:06:06.980848 7f5c973a57a0 20 read_log 194215'18652 (192834'17411) clean 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by osd.356.0:1346 2014-08-12 15:50:57.306782 2014-08-13 18:06:06.980860 7f5c973a57a0 20 read_log 194215'18653 (194215'18652) delete 99eea7b7/benchmark_data_storage0111-ib.pocstor_26659_object13174/head//24 by client.2174595.0:196684 0.000000 2014-08-13 18:06:06.980871 7f5c973a57a0 20 read_log 194215'18654 (192888'18355) clean c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by osd.356.0:1348 2014-08-12 15:50:57.314761 2014-08-13 18:06:06.980883 7f5c973a57a0 20 read_log 194215'18655 (194215'18654) delete c981b7b7/benchmark_data_storage0111-ib.pocstor_2228_object28005/head//24 by client.2174595.0:196686 0.000000 2014-08-13 18:06:06.980893 7f5c973a57a0 20 read_log 194215'18656 (192888'18101) clean 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by osd.356.0:1350 2014-08-12 15:50:57.322144 2014-08-13 18:06:06.980906 7f5c973a57a0 20 read_log 194215'18657 (194215'18656) delete 67a9dbb7/benchmark_data_storage0111-ib.pocstor_41097_object6449/head//24 by client.2174595.0:196688 0.000000 2014-08-13 18:06:06.980917 7f5c973a57a0 20 read_log 194215'18658 (192888'18109) clean 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by osd.356.0:1352 2014-08-12 15:50:57.330181 2014-08-13 18:06:06.980929 7f5c973a57a0 20 read_log 194215'18659 (194215'18658) delete 3a4adbb7/benchmark_data_storage0111-ib.pocstor_41097_object9547/head//24 by client.2174595.0:196690 0.000000 2014-08-13 18:06:06.980967 7f5c973a57a0 20 read_log 194215'18660 (193305'18437) clean 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by osd.356.0:1354 2014-08-12 15:50:57.337730 2014-08-13 18:06:06.980984 7f5c973a57a0 20 read_log 194215'18661 (194215'18660) delete 2cbadbb7/benchmark_data_storage0111-ib.pocstor_62621_object11762/head//24 by client.2174595.0:196692 0.000000 2014-08-13 18:06:06.980995 7f5c973a57a0 20 read_log 194215'18662 (192834'17930) clean 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by osd.356.0:1356 2014-08-12 15:50:57.345169 2014-08-13 18:06:06.981008 7f5c973a57a0 20 read_log 194215'18663 (194215'18662) delete 7aecdbb7/benchmark_data_storage0111-ib.pocstor_41526_object18833/head//24 by client.2174595.0:196694 0.000000 2014-08-13 18:06:06.981020 7f5c973a57a0 20 read_log 194215'18664 (193305'18457) clean 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by osd.356.0:1358 2014-08-12 15:50:57.353138 2014-08-13 18:06:06.981033 7f5c973a57a0 20 read_log 194215'18665 (194215'18664) delete 919ebb7/benchmark_data_storage0111-ib.pocstor_62621_object32502/head//24 by client.2174595.0:196696 0.000000 2014-08-13 18:06:06.981046 7f5c973a57a0 20 read_log 194215'18666 (192888'18154) clean cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by osd.356.0:1360 2014-08-12 15:50:57.361130 2014-08-13 18:06:06.981059 7f5c973a57a0 20 read_log 194215'18667 (194215'18666) delete cd7eebb7/benchmark_data_storage0111-ib.pocstor_41097_object29235/head//24 by client.2174595.0:196698 0.000000 2014-08-13 18:06:06.981069 7f5c973a57a0 20 read_log 194215'18668 (192888'18112) clean 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by osd.356.0:1362 2014-08-12 15:50:57.369486 2014-08-13 18:06:06.981081 7f5c973a57a0 20 read_log 194215'18669 (194215'18668) delete 4e96fbb7/benchmark_data_storage0111-ib.pocstor_41097_object10401/head//24 by client.2174595.0:196700 0.000000 2014-08-13 18:06:06.981092 7f5c973a57a0 20 read_log 194215'18670 (192834'17891) clean db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by osd.356.0:1364 2014-08-12 15:50:57.377587 2014-08-13 18:06:06.981106 7f5c973a57a0 20 read_log 194215'18671 (194215'18670) delete db29fbb7/benchmark_data_storage0111-ib.pocstor_41526_object12157/head//24 by client.2174595.0:196702 0.000000 2014-08-13 18:06:06.981117 7f5c973a57a0 20 read_log 194215'18672 (193305'18439) clean 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by osd.356.0:1366 2014-08-12 15:50:57.385858 2014-08-13 18:06:06.981132 7f5c973a57a0 20 read_log 194215'18673 (194215'18672) delete 37690fb7/benchmark_data_storage0111-ib.pocstor_62621_object14427/head//24 by client.2174595.0:196704 0.000000 2014-08-13 18:06:06.981143 7f5c973a57a0 20 read_log 194215'18674 (192834'17416) clean 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by osd.356.0:1368 2014-08-12 15:50:57.393776 2014-08-13 18:06:06.981162 7f5c973a57a0 20 read_log 194215'18675 (194215'18674) delete 84592fb7/benchmark_data_storage0111-ib.pocstor_26659_object13825/head//24 by client.2174595.0:196706 0.000000 2014-08-13 18:06:06.981174 7f5c973a57a0 20 read_log 194215'18676 (193305'18456) clean 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by osd.356.0:1370 2014-08-12 15:50:57.401412 2014-08-13 18:06:06.981188 7f5c973a57a0 20 read_log 194215'18677 (194215'18676) delete 3fd2fb7/benchmark_data_storage0111-ib.pocstor_62621_object31170/head//24 by client.2174595.0:196708 0.000000 2014-08-13 18:06:06.981199 7f5c973a57a0 20 read_log 194215'18678 (192834'17866) clean 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by osd.356.0:1372 2014-08-12 15:50:57.409105 2014-08-13 18:06:06.981211 7f5c973a57a0 20 read_log 194215'18679 (194215'18678) delete 6aaf2fb7/benchmark_data_storage0111-ib.pocstor_41526_object9154/head//24 by client.2174595.0:196710 0.000000 2014-08-13 18:06:06.981229 7f5c973a57a0 20 read_log 194215'18680 (192888'18153) clean 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by osd.356.0:1374 2014-08-12 15:50:57.416926 2014-08-13 18:06:06.981244 7f5c973a57a0 20 read_log 194215'18681 (194215'18680) delete 18673fb7/benchmark_data_storage0111-ib.pocstor_41097_object29146/head//24 by client.2174595.0:196712 0.000000 2014-08-13 18:06:06.981255 7f5c973a57a0 20 read_log 194215'18682 (192834'17884) clean f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by osd.356.0:1376 2014-08-12 15:50:57.424114 2014-08-13 18:06:06.981267 7f5c973a57a0 20 read_log 194215'18683 (194215'18682) delete f8b83fb7/benchmark_data_storage0111-ib.pocstor_41526_object11154/head//24 by client.2174595.0:196714 0.000000 2014-08-13 18:06:06.981283 7f5c973a57a0 20 read_log 194215'18684 (192888'18359) clean ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by osd.356.0:1378 2014-08-12 15:50:57.431334 2014-08-13 18:06:06.981299 7f5c973a57a0 20 read_log 194215'18685 (194215'18684) delete ef884fb7/benchmark_data_storage0111-ib.pocstor_2228_object30463/head//24 by client.2174595.0:196716 0.000000 2014-08-13 18:06:06.981310 7f5c973a57a0 20 read_log 194215'18686 (192888'18304) clean b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by osd.356.0:1380 2014-08-12 15:50:57.439067 2014-08-13 18:06:06.981322 7f5c973a57a0 20 read_log 194215'18687 (194215'18686) delete b34d4fb7/benchmark_data_storage0111-ib.pocstor_2228_object7201/head//24 by client.2174595.0:196718 0.000000 2014-08-13 18:06:06.981336 7f5c973a57a0 20 read_log 194215'18688 (192834'17928) clean b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by osd.356.0:1382 2014-08-12 15:50:57.446565 2014-08-13 18:06:06.981349 7f5c973a57a0 20 read_log 194215'18689 (194215'18688) delete b73a5fb7/benchmark_data_storage0111-ib.pocstor_41526_object18690/head//24 by client.2174595.0:196720 0.000000 2014-08-13 18:06:06.981360 7f5c973a57a0 20 read_log 194215'18690 (192888'18295) clean e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by osd.356.0:1384 2014-08-12 15:50:57.455099 2014-08-13 18:06:06.981373 7f5c973a57a0 20 read_log 194215'18691 (194215'18690) delete e3bc6fb7/benchmark_data_storage0111-ib.pocstor_2228_object3001/head//24 by client.2174595.0:196722 0.000000 2014-08-13 18:06:06.981387 7f5c973a57a0 20 read_log 194215'18692 (192888'18322) clean c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by osd.356.0:1386 2014-08-12 15:50:57.462982 2014-08-13 18:06:06.981400 7f5c973a57a0 20 read_log 194215'18693 (194215'18692) delete c5407fb7/benchmark_data_storage0111-ib.pocstor_2228_object15640/head//24 by client.2174595.0:196724 0.000000 2014-08-13 18:06:06.981420 7f5c973a57a0 20 read_log 194215'18694 (192888'18354) clean 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by osd.356.0:1388 2014-08-12 15:50:57.471038 2014-08-13 18:06:06.981446 7f5c973a57a0 20 read_log 194215'18695 (194215'18694) delete 25457fb7/benchmark_data_storage0111-ib.pocstor_2228_object26901/head//24 by client.2174595.0:196726 0.000000 2014-08-13 18:06:06.981465 7f5c973a57a0 20 read_log 194215'18696 (192888'18318) clean 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by osd.356.0:1390 2014-08-12 15:50:57.478951 2014-08-13 18:06:06.981481 7f5c973a57a0 20 read_log 194215'18697 (194215'18696) delete 99a57fb7/benchmark_data_storage0111-ib.pocstor_2228_object13712/head//24 by client.2174595.0:196728 0.000000 2014-08-13 18:06:06.981493 7f5c973a57a0 20 read_log 194215'18698 (193305'18442) clean 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by osd.356.0:1392 2014-08-12 15:50:57.486856 2014-08-13 18:06:06.981515 7f5c973a57a0 20 read_log 194215'18699 (194215'18698) delete 9e848fb7/benchmark_data_storage0111-ib.pocstor_62621_object17038/head//24 by client.2174595.0:196730 0.000000 2014-08-13 18:06:06.981537 7f5c973a57a0 20 read_log 194215'18700 (193305'18450) clean ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by osd.356.0:1394 2014-08-12 15:50:57.494599 2014-08-13 18:06:06.981554 7f5c973a57a0 20 read_log 194215'18701 (194215'18700) delete ee778fb7/benchmark_data_storage0111-ib.pocstor_62621_object24841/head//24 by client.2174595.0:196732 0.000000 2014-08-13 18:06:06.981565 7f5c973a57a0 20 read_log 194215'18702 (193305'18429) clean d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by osd.356.0:1396 2014-08-12 15:50:57.501848 2014-08-13 18:06:06.981578 7f5c973a57a0 20 read_log 194215'18703 (194215'18702) delete d8409fb7/benchmark_data_storage0111-ib.pocstor_62621_object3199/head//24 by client.2174595.0:196734 0.000000 2014-08-13 18:06:06.981589 7f5c973a57a0 20 read_log 194215'18704 (192888'18144) clean c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by osd.356.0:1398 2014-08-12 15:50:57.510487 2014-08-13 18:06:06.981608 7f5c973a57a0 20 read_log 194215'18705 (194215'18704) delete c1819fb7/benchmark_data_storage0111-ib.pocstor_41097_object26172/head//24 by client.2174595.0:196736 0.000000 2014-08-13 18:06:06.981632 7f5c973a57a0 20 read_log 194215'18706 (192888'18108) clean f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by osd.356.0:1400 2014-08-12 15:50:57.517915 2014-08-13 18:06:06.981645 7f5c973a57a0 20 read_log 194215'18707 (194215'18706) delete f5069fb7/benchmark_data_storage0111-ib.pocstor_41097_object9406/head//24 by client.2174595.0:196738 0.000000 2014-08-13 18:06:06.981656 7f5c973a57a0 20 read_log 194215'18708 (192888'18298) clean ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by osd.356.0:1402 2014-08-12 15:50:57.525098 2014-08-13 18:06:06.981677 7f5c973a57a0 20 read_log 194215'18709 (194215'18708) delete ef01afb7/benchmark_data_storage0111-ib.pocstor_2228_object4983/head//24 by client.2174595.0:196740 0.000000 2014-08-13 18:06:06.981688 7f5c973a57a0 20 read_log 194215'18710 (192888'18142) clean 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by osd.356.0:1404 2014-08-12 15:50:57.532776 2014-08-13 18:06:06.981701 7f5c973a57a0 20 read_log 194215'18711 (194215'18710) delete 94e7cfb7/benchmark_data_storage0111-ib.pocstor_41097_object24927/head//24 by client.2174595.0:196742 0.000000 2014-08-13 18:06:06.981712 7f5c973a57a0 20 read_log 194215'18712 (192888'18125) clean 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by osd.356.0:1406 2014-08-12 15:50:57.540472 2014-08-13 18:06:06.981742 7f5c973a57a0 20 read_log 194215'18713 (194215'18712) delete 3a2ccfb7/benchmark_data_storage0111-ib.pocstor_41097_object15062/head//24 by client.2174595.0:196744 0.000000 2014-08-13 18:06:06.981756 7f5c973a57a0 20 read_log 194215'18714 (192888'18116) clean 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by osd.356.0:1408 2014-08-12 15:50:57.548312 2014-08-13 18:06:06.981769 7f5c973a57a0 20 read_log 194215'18715 (194215'18714) delete 7d51dfb7/benchmark_data_storage0111-ib.pocstor_41097_object12260/head//24 by client.2174595.0:196746 0.000000 2014-08-13 18:06:06.981780 7f5c973a57a0 20 read_log 194215'18716 (192834'17861) clean eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by osd.356.0:1410 2014-08-12 15:50:57.556333 2014-08-13 18:06:06.981795 7f5c973a57a0 20 read_log 194215'18717 (194215'18716) delete eb3adfb7/benchmark_data_storage0111-ib.pocstor_41526_object8390/head//24 by client.2174595.0:196748 0.000000 2014-08-13 18:06:06.981807 7f5c973a57a0 20 read_log 194215'18718 (192888'18328) clean eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by osd.356.0:1412 2014-08-12 15:50:57.564498 2014-08-13 18:06:06.981819 7f5c973a57a0 20 read_log 194215'18719 (194215'18718) delete eedcdfb7/benchmark_data_storage0111-ib.pocstor_2228_object17680/head//24 by client.2174595.0:196750 0.000000 2014-08-13 18:06:06.981840 7f5c973a57a0 20 read_log 194215'18720 (192834'17409) clean 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by osd.356.0:1414 2014-08-12 15:50:57.572345 2014-08-13 18:06:06.981856 7f5c973a57a0 20 read_log 194215'18721 (194215'18720) delete 74caefb7/benchmark_data_storage0111-ib.pocstor_26659_object12956/head//24 by client.2174595.0:196752 0.000000 2014-08-13 18:06:06.981867 7f5c973a57a0 20 read_log 194215'18722 (192834'17833) clean a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by osd.356.0:1416 2014-08-12 15:50:57.580396 2014-08-13 18:06:06.981879 7f5c973a57a0 20 read_log 194215'18723 (194215'18722) delete a7f3ffb7/benchmark_data_storage0111-ib.pocstor_41526_object3642/head//24 by client.2174595.0:196754 0.000000 2014-08-13 18:06:06.981890 7f5c973a57a0 20 read_log 194215'18724 (192888'18330) clean 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by osd.356.0:1418 2014-08-12 15:50:57.612878 2014-08-13 18:06:06.981903 7f5c973a57a0 20 read_log 194215'18725 (194215'18724) delete 6897ffb7/benchmark_data_storage0111-ib.pocstor_2228_object18596/head//24 by client.2174595.0:196756 0.000000 2014-08-13 18:06:06.981926 7f5c973a57a0 20 read_log 194215'18726 (193305'18447) clean e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by osd.356.0:1420 2014-08-12 15:50:57.620632 2014-08-13 18:06:06.981939 7f5c973a57a0 20 read_log 194215'18727 (194215'18726) delete e678ffb7/benchmark_data_storage0111-ib.pocstor_62621_object20592/head//24 by client.2174595.0:196758 0.000000 2014-08-13 18:06:06.981950 7f5c973a57a0 20 read_log 194217'18728 (194217'18728) modify 3b7/hit_set_24.3b7_archive_2014-08-12 15:50:56.911487_2014-08-12 17:25:03.810214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.810430 2014-08-13 18:06:06.981963 7f5c973a57a0 20 read_log 194217'18729 (194215'18540) delete 3b7/hit_set_24.3b7_archive_2014-08-12 15:32:25.978488_2014-08-12 15:50:56.911217/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:25:03.810430 2014-08-13 18:06:06.981975 7f5c973a57a0 20 read_log 194217'18730 (0'0) promote 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by osd.356.0:1775 2014-08-12 17:25:03.816059 2014-08-13 18:06:06.981988 7f5c973a57a0 20 read_log 194217'18731 (194217'18730) modify 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by client.1487927.0:129 2014-08-12 17:25:03.805212 2014-08-13 18:06:06.982000 7f5c973a57a0 20 read_log 194370'18732 (0'0) promote efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by osd.356.0:1789 2014-08-12 20:30:22.056991 2014-08-13 18:06:06.982013 7f5c973a57a0 20 read_log 194370'18733 (194370'18732) modify efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by client.1508531.0:2038 2014-08-12 20:30:22.049716 2014-08-13 18:06:06.982025 7f5c973a57a0 20 read_log 194370'18734 (0'0) promote b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by osd.356.0:1803 2014-08-12 20:30:30.164477 2014-08-13 18:06:06.982037 7f5c973a57a0 20 read_log 194370'18735 (194370'18734) modify b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by client.1508531.0:4218 2014-08-12 20:30:30.156160 2014-08-13 18:06:06.982050 7f5c973a57a0 20 read_log 194370'18736 (0'0) promote 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by osd.356.0:1811 2014-08-12 20:30:35.339641 2014-08-13 18:06:06.982062 7f5c973a57a0 20 read_log 194370'18737 (194370'18736) modify 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by client.1508531.0:5485 2014-08-12 20:30:35.334220 2014-08-13 18:06:06.982074 7f5c973a57a0 20 read_log 194370'18738 (0'0) promote c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by osd.356.0:1825 2014-08-12 20:30:48.946434 2014-08-13 18:06:06.982094 7f5c973a57a0 20 read_log 194370'18739 (194370'18738) modify c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by client.1508531.0:8257 2014-08-12 20:30:48.939251 2014-08-13 18:06:06.982109 7f5c973a57a0 20 read_log 194370'18740 (0'0) promote 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by osd.356.0:1839 2014-08-12 20:30:59.726485 2014-08-13 18:06:06.982121 7f5c973a57a0 20 read_log 194370'18741 (194370'18740) modify 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by client.1508531.0:10082 2014-08-12 20:30:59.719353 2014-08-13 18:06:06.982135 7f5c973a57a0 20 read_log 194370'18742 (0'0) promote 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by osd.356.0:1841 2014-08-12 20:31:00.857931 2014-08-13 18:06:06.982154 7f5c973a57a0 20 read_log 194370'18743 (194370'18742) modify 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by client.1508531.0:10272 2014-08-12 20:31:00.849645 2014-08-13 18:06:06.982170 7f5c973a57a0 20 read_log 194370'18744 (0'0) promote 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by osd.356.0:1843 2014-08-12 20:31:01.325389 2014-08-13 18:06:06.982183 7f5c973a57a0 20 read_log 194370'18745 (194370'18744) modify 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by client.1508531.0:10351 2014-08-12 20:31:01.318249 2014-08-13 18:06:06.982217 7f5c973a57a0 20 read_log 194370'18746 (0'0) promote a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by osd.356.0:1847 2014-08-12 20:31:03.723013 2014-08-13 18:06:06.982233 7f5c973a57a0 20 read_log 194370'18747 (194370'18746) modify a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by client.1508531.0:10735 2014-08-12 20:31:03.714078 2014-08-13 18:06:06.982246 7f5c973a57a0 20 read_log 194370'18748 (0'0) promote 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by osd.356.0:1849 2014-08-12 20:31:07.515806 2014-08-13 18:06:06.982258 7f5c973a57a0 20 read_log 194370'18749 (194370'18748) modify 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by client.1508531.0:11367 2014-08-12 20:31:07.508175 2014-08-13 18:06:06.982270 7f5c973a57a0 20 read_log 194370'18750 (0'0) promote e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by osd.356.0:1857 2014-08-12 20:31:16.612356 2014-08-13 18:06:06.982286 7f5c973a57a0 20 read_log 194370'18751 (194370'18750) modify e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by client.1508531.0:12709 2014-08-12 20:31:16.602273 2014-08-13 18:06:06.982300 7f5c973a57a0 20 read_log 194370'18752 (0'0) promote 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by osd.356.0:1859 2014-08-12 20:31:17.721544 2014-08-13 18:06:06.982313 7f5c973a57a0 20 read_log 194370'18753 (194370'18752) modify 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by client.1508531.0:12825 2014-08-12 20:31:17.712236 2014-08-13 18:06:06.982328 7f5c973a57a0 20 read_log 194377'18754 (0'0) promote 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by osd.356.0:1895 2014-08-12 20:33:08.237705 2014-08-13 18:06:06.982341 7f5c973a57a0 20 read_log 194377'18755 (194377'18754) modify 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by client.2231190.0:2731 2014-08-12 20:33:08.231506 2014-08-13 18:06:06.982355 7f5c973a57a0 20 read_log 194377'18756 (0'0) promote 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by osd.356.0:1901 2014-08-12 20:33:12.086791 2014-08-13 18:06:06.982368 7f5c973a57a0 20 read_log 194377'18757 (194377'18756) modify 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by client.2231190.0:3645 2014-08-12 20:33:12.080752 2014-08-13 18:06:06.982394 7f5c973a57a0 20 read_log 194377'18758 (0'0) promote 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by osd.356.0:1913 2014-08-12 20:33:17.900671 2014-08-13 18:06:06.982416 7f5c973a57a0 20 read_log 194377'18759 (194377'18758) modify 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by client.2231190.0:4599 2014-08-12 20:33:17.892578 2014-08-13 18:06:06.982437 7f5c973a57a0 20 read_log 194383'18760 (0'0) promote 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by osd.356.0:1923 2014-08-12 20:34:00.084002 2014-08-13 18:06:06.982450 7f5c973a57a0 20 read_log 194383'18761 (194383'18760) modify 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by client.2231190.0:5769 2014-08-12 20:34:00.019150 2014-08-13 18:06:06.982476 7f5c973a57a0 20 read_log 194383'18762 (0'0) promote b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by osd.356.0:1925 2014-08-12 20:34:07.590059 2014-08-13 18:06:06.982489 7f5c973a57a0 20 read_log 194383'18763 (194383'18762) modify b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by client.2231190.0:6721 2014-08-12 20:34:07.583644 2014-08-13 18:06:06.982502 7f5c973a57a0 20 read_log 194383'18764 (0'0) promote c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by osd.356.0:1935 2014-08-12 20:34:24.401887 2014-08-13 18:06:06.982514 7f5c973a57a0 20 read_log 194383'18765 (194383'18764) modify c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by client.2231190.0:7341 2014-08-12 20:34:24.392949 2014-08-13 18:06:06.982538 7f5c973a57a0 20 read_log 194383'18766 (0'0) promote 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by osd.356.0:1937 2014-08-12 20:34:30.701553 2014-08-13 18:06:06.982551 7f5c973a57a0 20 read_log 194383'18767 (194383'18766) modify 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by client.2231190.0:7571 2014-08-12 20:34:30.693282 2014-08-13 18:06:06.982563 7f5c973a57a0 20 read_log 194393'18768 (194393'18768) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:37:09.043051_2014-08-12 20:47:27.879383/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.879597 2014-08-13 18:06:06.982576 7f5c973a57a0 20 read_log 194393'18769 (194217'18728) delete 3b7/hit_set_24.3b7_archive_2014-08-12 15:50:56.911487_2014-08-12 17:25:03.810214/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:47:27.879597 2014-08-13 18:06:06.982588 7f5c973a57a0 20 read_log 194393'18770 (0'0) promote 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by osd.356.0:1943 2014-08-12 20:47:27.882962 2014-08-13 18:06:06.982601 7f5c973a57a0 20 read_log 194393'18771 (194393'18770) modify 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by client.1064095.0:459 2014-08-12 20:47:27.874894 2014-08-13 18:06:06.982618 7f5c973a57a0 20 read_log 194397'18772 (194397'18772) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:47:27.879602_2014-08-12 20:53:06.322101/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:06.322318 2014-08-13 18:06:06.982631 7f5c973a57a0 20 read_log 194397'18773 (194393'18768) delete 3b7/hit_set_24.3b7_archive_2014-08-12 20:37:09.043051_2014-08-12 20:47:27.879383/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:53:06.322318 2014-08-13 18:06:06.982643 7f5c973a57a0 20 read_log 194397'18774 (0'0) promote 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by osd.356.0:1962 2014-08-12 20:53:06.325818 2014-08-13 18:06:06.982655 7f5c973a57a0 20 read_log 194397'18775 (194397'18774) modify 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by client.2235069.0:933 2014-08-12 20:53:06.315038 2014-08-13 18:06:06.982668 7f5c973a57a0 20 read_log 194416'18776 (194416'18776) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:53:06.322323_2014-08-13 15:28:31.768122/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 18:06:06.982691 7f5c973a57a0 20 read_log 194416'18777 (194397'18772) delete 3b7/hit_set_24.3b7_archive_2014-08-12 20:47:27.879602_2014-08-12 20:53:06.322101/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 18:06:06.982707 7f5c973a57a0 20 read_log 194416'18778 (0'0) promote 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by osd.356.0:2077 2014-08-13 15:28:31.771636 2014-08-13 18:06:06.982719 7f5c973a57a0 20 read_log 194416'18779 (194416'18778) modify 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by client.2441958.0:369 2014-08-13 15:28:31.761856 2014-08-13 18:06:06.982732 7f5c973a57a0 20 read_log 194416'18780 (0'0) promote 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by osd.356.0:2079 2014-08-13 15:28:33.333612 2014-08-13 18:06:06.982744 7f5c973a57a0 20 read_log 194416'18781 (194416'18780) modify 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by client.2441958.0:719 2014-08-13 15:28:33.326094 2014-08-13 18:06:06.982757 7f5c973a57a0 20 read_log 194416'18782 (0'0) promote b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by osd.356.0:2085 2014-08-13 15:28:36.783438 2014-08-13 18:06:06.982769 7f5c973a57a0 20 read_log 194416'18783 (194416'18782) modify b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by client.2441958.0:1599 2014-08-13 15:28:36.774701 2014-08-13 18:06:06.982794 7f5c973a57a0 20 read_log 194416'18784 (0'0) promote 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by osd.356.0:2105 2014-08-13 15:28:43.236766 2014-08-13 18:06:06.982807 7f5c973a57a0 20 read_log 194416'18785 (194416'18784) modify 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by client.2441958.0:3307 2014-08-13 15:28:43.228885 2014-08-13 18:06:06.982819 7f5c973a57a0 20 read_log 194416'18786 (0'0) promote f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by osd.356.0:2117 2014-08-13 15:28:49.895930 2014-08-13 18:06:06.982831 7f5c973a57a0 20 read_log 194416'18787 (194416'18786) modify f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by client.2441958.0:5001 2014-08-13 15:28:49.888965 2014-08-13 18:06:06.982844 7f5c973a57a0 20 read_log 194416'18788 (0'0) promote 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by osd.356.0:2125 2014-08-13 15:28:52.067271 2014-08-13 18:06:06.982856 7f5c973a57a0 20 read_log 194416'18789 (194416'18788) modify 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by client.2441958.0:5572 2014-08-13 15:28:52.060924 2014-08-13 18:06:06.982868 7f5c973a57a0 20 read_log 194416'18790 (0'0) promote c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by osd.356.0:2135 2014-08-13 15:28:54.342025 2014-08-13 18:06:06.982880 7f5c973a57a0 20 read_log 194416'18791 (194416'18790) modify c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by client.2441958.0:6123 2014-08-13 15:28:54.334808 2014-08-13 18:06:06.982892 7f5c973a57a0 20 read_log 194416'18792 (0'0) promote 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by osd.356.0:2143 2014-08-13 15:28:56.819379 2014-08-13 18:06:06.982904 7f5c973a57a0 20 read_log 194416'18793 (194416'18792) modify 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by client.2441958.0:6743 2014-08-13 15:28:56.811884 2014-08-13 18:06:06.982916 7f5c973a57a0 20 read_log 194416'18794 (0'0) promote d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by osd.356.0:2145 2014-08-13 15:29:00.088480 2014-08-13 18:06:06.982929 7f5c973a57a0 20 read_log 194416'18795 (194416'18794) modify d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by client.2441958.0:7616 2014-08-13 15:29:00.080339 2014-08-13 18:06:06.982967 7f5c973a57a0 20 read_log 194416'18796 (0'0) promote 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by osd.356.0:2153 2014-08-13 15:29:00.974133 2014-08-13 18:06:06.982983 7f5c973a57a0 20 read_log 194416'18797 (194416'18796) modify 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by client.2441958.0:7852 2014-08-13 15:29:00.965404 2014-08-13 18:06:06.982995 7f5c973a57a0 20 read_log 194416'18798 (0'0) promote 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by osd.356.0:2155 2014-08-13 15:29:02.330139 2014-08-13 18:06:06.983008 7f5c973a57a0 20 read_log 194416'18799 (194416'18798) modify 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by client.2441958.0:8213 2014-08-13 15:29:02.324036 2014-08-13 18:06:06.983022 7f5c973a57a0 20 read_log 194416'18800 (0'0) promote fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by osd.356.0:2161 2014-08-13 15:29:05.284996 2014-08-13 18:06:06.983035 7f5c973a57a0 20 read_log 194416'18801 (194416'18800) modify fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by client.2441958.0:8955 2014-08-13 15:29:05.276987 2014-08-13 18:06:06.983047 7f5c973a57a0 20 read_log 194416'18802 (0'0) promote 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by osd.356.0:2171 2014-08-13 15:29:08.363714 2014-08-13 18:06:06.983061 7f5c973a57a0 20 read_log 194416'18803 (194416'18802) modify 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by client.2441958.0:9757 2014-08-13 15:29:08.356283 2014-08-13 18:06:06.983074 7f5c973a57a0 20 read_log 194416'18804 (0'0) promote 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by osd.356.0:2183 2014-08-13 15:29:13.558051 2014-08-13 18:06:06.983089 7f5c973a57a0 20 read_log 194416'18805 (194416'18804) modify 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by client.2441958.0:11063 2014-08-13 15:29:13.549874 2014-08-13 18:06:06.983117 7f5c973a57a0 20 read_log 194416'18806 (0'0) promote 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by osd.356.0:2189 2014-08-13 15:29:18.883994 2014-08-13 18:06:06.983132 7f5c973a57a0 20 read_log 194416'18807 (194416'18806) modify 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by client.2441958.0:12291 2014-08-13 15:29:18.874106 2014-08-13 18:06:06.983144 7f5c973a57a0 20 read_log 194416'18808 (0'0) promote e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by osd.356.0:2197 2014-08-13 15:29:20.731051 2014-08-13 18:06:06.983164 7f5c973a57a0 20 read_log 194416'18809 (194416'18808) modify e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by client.2441958.0:12738 2014-08-13 15:29:20.723333 2014-08-13 18:06:06.983177 7f5c973a57a0 20 read_log 194416'18810 (0'0) promote 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by osd.356.0:2201 2014-08-13 15:29:21.357739 2014-08-13 18:06:06.983191 7f5c973a57a0 20 read_log 194416'18811 (194416'18810) modify 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by client.2441958.0:12879 2014-08-13 15:29:21.350712 2014-08-13 18:06:06.983204 7f5c973a57a0 20 read_log 194416'18812 (0'0) promote edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by osd.356.0:2211 2014-08-13 15:29:26.439987 2014-08-13 18:06:06.983216 7f5c973a57a0 20 read_log 194416'18813 (194416'18812) modify edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by client.2441958.0:14059 2014-08-13 15:29:26.429216 2014-08-13 18:06:06.983229 7f5c973a57a0 20 read_log 194416'18814 (0'0) promote 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by osd.356.0:2215 2014-08-13 15:29:28.359569 2014-08-13 18:06:06.983243 7f5c973a57a0 20 read_log 194416'18815 (194416'18814) modify 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by client.2441958.0:14537 2014-08-13 15:29:28.349970 2014-08-13 18:06:06.983264 7f5c973a57a0 20 read_log 194416'18816 (0'0) promote 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by osd.356.0:2217 2014-08-13 15:29:32.179418 2014-08-13 18:06:06.983281 7f5c973a57a0 20 read_log 194416'18817 (194416'18816) modify 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by client.2441958.0:15404 2014-08-13 15:29:32.170808 2014-08-13 18:06:06.983295 7f5c973a57a0 20 read_log 194416'18818 (0'0) promote 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by osd.356.0:2227 2014-08-13 15:29:49.344089 2014-08-13 18:06:06.983307 7f5c973a57a0 20 read_log 194416'18819 (194416'18818) modify 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by client.2441958.0:17140 2014-08-13 15:29:49.335836 2014-08-13 18:06:06.983320 7f5c973a57a0 20 read_log 194416'18820 (0'0) promote 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by osd.356.0:2235 2014-08-13 15:30:00.051696 2014-08-13 18:06:06.983336 7f5c973a57a0 20 read_log 194416'18821 (194416'18820) modify 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by client.2441958.0:17900 2014-08-13 15:30:00.042666 2014-08-13 18:06:06.983351 7f5c973a57a0 20 read_log 194422'18822 (0'0) promote 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by osd.356.0:2247 2014-08-13 15:32:20.656769 2014-08-13 18:06:06.983363 7f5c973a57a0 20 read_log 194422'18823 (194422'18822) modify 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by client.1148203.0:1343 2014-08-13 15:32:20.648957 2014-08-13 18:06:06.983376 7f5c973a57a0 20 read_log 194422'18824 (0'0) promote a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by osd.356.0:2261 2014-08-13 15:32:27.177459 2014-08-13 18:06:06.983392 7f5c973a57a0 20 read_log 194422'18825 (194422'18824) modify a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by client.1148203.0:3021 2014-08-13 15:32:27.170065 2014-08-13 18:06:06.983413 7f5c973a57a0 20 read_log 194422'18826 (0'0) promote 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by osd.356.0:2263 2014-08-13 15:32:27.840527 2014-08-13 18:06:06.983433 7f5c973a57a0 20 read_log 194422'18827 (194422'18826) modify 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by client.1148203.0:3196 2014-08-13 15:32:27.833290 2014-08-13 18:06:06.983448 7f5c973a57a0 20 read_log 194422'18828 (0'0) promote 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by osd.356.0:2271 2014-08-13 15:32:34.011483 2014-08-13 18:06:06.983461 7f5c973a57a0 20 read_log 194422'18829 (194422'18828) modify 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by client.1148203.0:4783 2014-08-13 15:32:34.003860 2014-08-13 18:06:06.983481 7f5c973a57a0 20 read_log 194422'18830 (0'0) promote cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by osd.356.0:2273 2014-08-13 15:32:34.647899 2014-08-13 18:06:06.983496 7f5c973a57a0 20 read_log 194422'18831 (194422'18830) modify cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by client.1148203.0:4950 2014-08-13 15:32:34.641471 2014-08-13 18:06:06.983509 7f5c973a57a0 20 read_log 194422'18832 (0'0) promote e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by osd.356.0:2283 2014-08-13 15:32:38.043010 2014-08-13 18:06:06.983528 7f5c973a57a0 20 read_log 194422'18833 (194422'18832) modify e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by client.1148203.0:5620 2014-08-13 15:32:38.034770 2014-08-13 18:06:06.983544 7f5c973a57a0 20 read_log 194422'18834 (0'0) promote bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by osd.356.0:2285 2014-08-13 15:32:38.054487 2014-08-13 18:06:06.983556 7f5c973a57a0 20 read_log 194422'18835 (194422'18834) modify bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by client.1148203.0:5622 2014-08-13 15:32:38.045437 2014-08-13 18:06:06.983594 7f5c973a57a0 20 read_log 194422'18836 (0'0) promote 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by osd.356.0:2295 2014-08-13 15:32:46.093217 2014-08-13 18:06:06.983616 7f5c973a57a0 20 read_log 194422'18837 (194422'18836) modify 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by client.1148203.0:7093 2014-08-13 15:32:46.086223 2014-08-13 18:06:06.983639 7f5c973a57a0 20 read_log 194422'18838 (0'0) promote fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by osd.356.0:2297 2014-08-13 15:32:46.389557 2014-08-13 18:06:06.983655 7f5c973a57a0 20 read_log 194422'18839 (194422'18838) modify fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by client.1148203.0:7140 2014-08-13 15:32:46.380212 2014-08-13 18:06:06.983668 7f5c973a57a0 20 read_log 194422'18840 (0'0) promote e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by osd.356.0:2319 2014-08-13 15:33:06.361050 2014-08-13 18:06:06.983681 7f5c973a57a0 20 read_log 194422'18841 (194422'18840) modify e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by client.1148203.0:9693 2014-08-13 15:33:06.353096 2014-08-13 18:06:06.983693 7f5c973a57a0 20 read_log 194422'18842 (0'0) promote 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by osd.356.0:2323 2014-08-13 15:33:11.972044 2014-08-13 18:06:06.983706 7f5c973a57a0 20 read_log 194422'18843 (194422'18842) modify 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by client.1148203.0:10072 2014-08-13 15:33:11.963605 2014-08-13 18:06:06.983718 7f5c973a57a0 20 read_log 194422'18844 (0'0) promote aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by osd.356.0:2325 2014-08-13 15:33:17.566159 2014-08-13 18:06:06.983730 7f5c973a57a0 20 read_log 194422'18845 (194422'18844) modify aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by client.1148203.0:10269 2014-08-13 15:33:17.557449 2014-08-13 18:06:06.983747 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.984172 7f5c973a57a0 10 read_log checking for missing items over interval (194217'18730,194422'18845] 2014-08-13 18:06:06.984220 7f5c973a57a0 15 read_log missing 194422'18845 (194422'18844) modify aa6427b7/benchmark_data_storage0111-ib.pocstor_41231_object10268/head//24 by client.1148203.0:10269 2014-08-13 15:33:17.557449 2014-08-13 18:06:06.984260 7f5c973a57a0 15 read_log missing 194422'18843 (194422'18842) modify 2e37dbb7/benchmark_data_storage0111-ib.pocstor_41231_object10071/head//24 by client.1148203.0:10072 2014-08-13 15:33:11.963605 2014-08-13 18:06:06.984297 7f5c973a57a0 15 read_log missing 194422'18841 (194422'18840) modify e998d3b7/benchmark_data_storage0111-ib.pocstor_41231_object9692/head//24 by client.1148203.0:9693 2014-08-13 15:33:06.353096 2014-08-13 18:06:06.984333 7f5c973a57a0 15 read_log missing 194422'18839 (194422'18838) modify fa5e3b7/benchmark_data_storage0111-ib.pocstor_41231_object7139/head//24 by client.1148203.0:7140 2014-08-13 15:32:46.380212 2014-08-13 18:06:06.984369 7f5c973a57a0 15 read_log missing 194422'18837 (194422'18836) modify 2a428bb7/benchmark_data_storage0111-ib.pocstor_41231_object7092/head//24 by client.1148203.0:7093 2014-08-13 15:32:46.086223 2014-08-13 18:06:06.984405 7f5c973a57a0 15 read_log missing 194422'18835 (194422'18834) modify bc21a7b7/benchmark_data_storage0111-ib.pocstor_41231_object5621/head//24 by client.1148203.0:5622 2014-08-13 15:32:38.045437 2014-08-13 18:06:06.984441 7f5c973a57a0 15 read_log missing 194422'18833 (194422'18832) modify e2c0d3b7/benchmark_data_storage0111-ib.pocstor_41231_object5619/head//24 by client.1148203.0:5620 2014-08-13 15:32:38.034770 2014-08-13 18:06:06.984476 7f5c973a57a0 15 read_log missing 194422'18831 (194422'18830) modify cab4f3b7/benchmark_data_storage0111-ib.pocstor_41231_object4949/head//24 by client.1148203.0:4950 2014-08-13 15:32:34.641471 2014-08-13 18:06:06.984512 7f5c973a57a0 15 read_log missing 194422'18829 (194422'18828) modify 7aca7bb7/benchmark_data_storage0111-ib.pocstor_41231_object4782/head//24 by client.1148203.0:4783 2014-08-13 15:32:34.003860 2014-08-13 18:06:06.984547 7f5c973a57a0 15 read_log missing 194422'18827 (194422'18826) modify 419437b7/benchmark_data_storage0111-ib.pocstor_41231_object3195/head//24 by client.1148203.0:3196 2014-08-13 15:32:27.833290 2014-08-13 18:06:06.984583 7f5c973a57a0 15 read_log missing 194422'18825 (194422'18824) modify a3e32fb7/benchmark_data_storage0111-ib.pocstor_41231_object3020/head//24 by client.1148203.0:3021 2014-08-13 15:32:27.170065 2014-08-13 18:06:06.984619 7f5c973a57a0 15 read_log missing 194422'18823 (194422'18822) modify 6b6c77b7/benchmark_data_storage0111-ib.pocstor_41231_object1342/head//24 by client.1148203.0:1343 2014-08-13 15:32:20.648957 2014-08-13 18:06:06.984654 7f5c973a57a0 15 read_log missing 194416'18821 (194416'18820) modify 600a1bb7/benchmark_data_storage0111-ib.pocstor_19103_object17899/head//24 by client.2441958.0:17900 2014-08-13 15:30:00.042666 2014-08-13 18:06:06.984690 7f5c973a57a0 15 read_log missing 194416'18819 (194416'18818) modify 531cfbb7/benchmark_data_storage0111-ib.pocstor_19103_object17139/head//24 by client.2441958.0:17140 2014-08-13 15:29:49.335836 2014-08-13 18:06:06.984726 7f5c973a57a0 15 read_log missing 194416'18817 (194416'18816) modify 30ab23b7/benchmark_data_storage0111-ib.pocstor_19103_object15403/head//24 by client.2441958.0:15404 2014-08-13 15:29:32.170808 2014-08-13 18:06:06.984762 7f5c973a57a0 15 read_log missing 194416'18815 (194416'18814) modify 3fdd97b7/benchmark_data_storage0111-ib.pocstor_19103_object14536/head//24 by client.2441958.0:14537 2014-08-13 15:29:28.349970 2014-08-13 18:06:06.984798 7f5c973a57a0 15 read_log missing 194416'18813 (194416'18812) modify edd7b7/benchmark_data_storage0111-ib.pocstor_19103_object14058/head//24 by client.2441958.0:14059 2014-08-13 15:29:26.429216 2014-08-13 18:06:06.984834 7f5c973a57a0 15 read_log missing 194416'18811 (194416'18810) modify 3ce553b7/benchmark_data_storage0111-ib.pocstor_19103_object12878/head//24 by client.2441958.0:12879 2014-08-13 15:29:21.350712 2014-08-13 18:06:06.984870 7f5c973a57a0 15 read_log missing 194416'18809 (194416'18808) modify e4859fb7/benchmark_data_storage0111-ib.pocstor_19103_object12737/head//24 by client.2441958.0:12738 2014-08-13 15:29:20.723333 2014-08-13 18:06:06.984905 7f5c973a57a0 15 read_log missing 194416'18807 (194416'18806) modify 712a37b7/benchmark_data_storage0111-ib.pocstor_19103_object12290/head//24 by client.2441958.0:12291 2014-08-13 15:29:18.874106 2014-08-13 18:06:06.984941 7f5c973a57a0 15 read_log missing 194416'18805 (194416'18804) modify 2efaf3b7/benchmark_data_storage0111-ib.pocstor_19103_object11062/head//24 by client.2441958.0:11063 2014-08-13 15:29:13.549874 2014-08-13 18:06:06.984977 7f5c973a57a0 15 read_log missing 194416'18803 (194416'18802) modify 6f8f7bb7/benchmark_data_storage0111-ib.pocstor_19103_object9756/head//24 by client.2441958.0:9757 2014-08-13 15:29:08.356283 2014-08-13 18:06:06.985012 7f5c973a57a0 15 read_log missing 194416'18801 (194416'18800) modify fc7397b7/benchmark_data_storage0111-ib.pocstor_19103_object8954/head//24 by client.2441958.0:8955 2014-08-13 15:29:05.276987 2014-08-13 18:06:06.985048 7f5c973a57a0 15 read_log missing 194416'18799 (194416'18798) modify 5aac87b7/benchmark_data_storage0111-ib.pocstor_19103_object8212/head//24 by client.2441958.0:8213 2014-08-13 15:29:02.324036 2014-08-13 18:06:06.985084 7f5c973a57a0 15 read_log missing 194416'18797 (194416'18796) modify 2acbcbb7/benchmark_data_storage0111-ib.pocstor_19103_object7851/head//24 by client.2441958.0:7852 2014-08-13 15:29:00.965404 2014-08-13 18:06:06.985119 7f5c973a57a0 15 read_log missing 194416'18795 (194416'18794) modify d89af7b7/benchmark_data_storage0111-ib.pocstor_19103_object7615/head//24 by client.2441958.0:7616 2014-08-13 15:29:00.080339 2014-08-13 18:06:06.985159 7f5c973a57a0 15 read_log missing 194416'18793 (194416'18792) modify 2e56e7b7/benchmark_data_storage0111-ib.pocstor_19103_object6742/head//24 by client.2441958.0:6743 2014-08-13 15:28:56.811884 2014-08-13 18:06:06.985196 7f5c973a57a0 15 read_log missing 194416'18791 (194416'18790) modify c9dcbfb7/benchmark_data_storage0111-ib.pocstor_19103_object6122/head//24 by client.2441958.0:6123 2014-08-13 15:28:54.334808 2014-08-13 18:06:06.985232 7f5c973a57a0 15 read_log missing 194416'18789 (194416'18788) modify 24e9f3b7/benchmark_data_storage0111-ib.pocstor_19103_object5571/head//24 by client.2441958.0:5572 2014-08-13 15:28:52.060924 2014-08-13 18:06:06.985273 7f5c973a57a0 15 read_log missing 194416'18787 (194416'18786) modify f4efa7b7/benchmark_data_storage0111-ib.pocstor_19103_object5000/head//24 by client.2441958.0:5001 2014-08-13 15:28:49.888965 2014-08-13 18:06:06.985314 7f5c973a57a0 15 read_log missing 194416'18785 (194416'18784) modify 9a8b7fb7/benchmark_data_storage0111-ib.pocstor_19103_object3306/head//24 by client.2441958.0:3307 2014-08-13 15:28:43.228885 2014-08-13 18:06:06.985350 7f5c973a57a0 15 read_log missing 194416'18783 (194416'18782) modify b2e167b7/benchmark_data_storage0111-ib.pocstor_19103_object1598/head//24 by client.2441958.0:1599 2014-08-13 15:28:36.774701 2014-08-13 18:06:06.985402 7f5c973a57a0 15 read_log missing 194416'18781 (194416'18780) modify 2d2a17b7/benchmark_data_storage0111-ib.pocstor_19103_object718/head//24 by client.2441958.0:719 2014-08-13 15:28:33.326094 2014-08-13 18:06:06.985482 7f5c973a57a0 15 read_log missing 194416'18779 (194416'18778) modify 2d1d23b7/benchmark_data_storage0111-ib.pocstor_19103_object368/head//24 by client.2441958.0:369 2014-08-13 15:28:31.761856 2014-08-13 18:06:06.985525 7f5c973a57a0 15 read_log missing 194416'18776 (194416'18776) modify 3b7/hit_set_24.3b7_archive_2014-08-12 20:53:06.322323_2014-08-13 15:28:31.768122/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:31.768341 2014-08-13 18:06:06.985560 7f5c973a57a0 15 read_log missing 194397'18775 (194397'18774) modify 33bffb7/benchmark_data_storage0101-ib.pocstor_6819_object932/head//24 by client.2235069.0:933 2014-08-12 20:53:06.315038 2014-08-13 18:06:06.985600 7f5c973a57a0 15 read_log missing 194393'18771 (194393'18770) modify 2bdbcfb7/benchmark_data_storage0111-ib.pocstor_24170_object458/head//24 by client.1064095.0:459 2014-08-12 20:47:27.874894 2014-08-13 18:06:06.985639 7f5c973a57a0 15 read_log missing 194383'18767 (194383'18766) modify 6a335bb7/benchmark_data_storage0111-ib.pocstor_25065_object7570/head//24 by client.2231190.0:7571 2014-08-12 20:34:30.693282 2014-08-13 18:06:06.985675 7f5c973a57a0 15 read_log missing 194383'18765 (194383'18764) modify c46903b7/benchmark_data_storage0111-ib.pocstor_25065_object7340/head//24 by client.2231190.0:7341 2014-08-12 20:34:24.392949 2014-08-13 18:06:06.985710 7f5c973a57a0 15 read_log missing 194383'18763 (194383'18762) modify b76ef7b7/benchmark_data_storage0111-ib.pocstor_25065_object6720/head//24 by client.2231190.0:6721 2014-08-12 20:34:07.583644 2014-08-13 18:06:06.985746 7f5c973a57a0 15 read_log missing 194383'18761 (194383'18760) modify 754f3b7/benchmark_data_storage0111-ib.pocstor_25065_object5768/head//24 by client.2231190.0:5769 2014-08-12 20:34:00.019150 2014-08-13 18:06:06.985782 7f5c973a57a0 15 read_log missing 194377'18759 (194377'18758) modify 23d32fb7/benchmark_data_storage0111-ib.pocstor_25065_object4598/head//24 by client.2231190.0:4599 2014-08-12 20:33:17.892578 2014-08-13 18:06:06.985818 7f5c973a57a0 15 read_log missing 194377'18757 (194377'18756) modify 872aafb7/benchmark_data_storage0111-ib.pocstor_25065_object3644/head//24 by client.2231190.0:3645 2014-08-12 20:33:12.080752 2014-08-13 18:06:06.985856 7f5c973a57a0 15 read_log missing 194377'18755 (194377'18754) modify 994933b7/benchmark_data_storage0111-ib.pocstor_25065_object2730/head//24 by client.2231190.0:2731 2014-08-12 20:33:08.231506 2014-08-13 18:06:06.985893 7f5c973a57a0 15 read_log missing 194370'18753 (194370'18752) modify 2b131fb7/benchmark_data_storage0111-ib.pocstor_12495_object12824/head//24 by client.1508531.0:12825 2014-08-12 20:31:17.712236 2014-08-13 18:06:06.985929 7f5c973a57a0 15 read_log missing 194370'18751 (194370'18750) modify e8887fb7/benchmark_data_storage0111-ib.pocstor_12495_object12708/head//24 by client.1508531.0:12709 2014-08-12 20:31:16.602273 2014-08-13 18:06:06.985965 7f5c973a57a0 15 read_log missing 194370'18749 (194370'18748) modify 959c87b7/benchmark_data_storage0111-ib.pocstor_12495_object11366/head//24 by client.1508531.0:11367 2014-08-12 20:31:07.508175 2014-08-13 18:06:06.986001 7f5c973a57a0 15 read_log missing 194370'18747 (194370'18746) modify a91b97b7/benchmark_data_storage0111-ib.pocstor_12495_object10734/head//24 by client.1508531.0:10735 2014-08-12 20:31:03.714078 2014-08-13 18:06:06.986037 7f5c973a57a0 15 read_log missing 194370'18745 (194370'18744) modify 6304c7b7/benchmark_data_storage0111-ib.pocstor_12495_object10350/head//24 by client.1508531.0:10351 2014-08-12 20:31:01.318249 2014-08-13 18:06:06.986073 7f5c973a57a0 15 read_log missing 194370'18743 (194370'18742) modify 8f5ebb7/benchmark_data_storage0111-ib.pocstor_12495_object10271/head//24 by client.1508531.0:10272 2014-08-12 20:31:00.849645 2014-08-13 18:06:06.986109 7f5c973a57a0 15 read_log missing 194370'18741 (194370'18740) modify 66b1dfb7/benchmark_data_storage0111-ib.pocstor_12495_object10081/head//24 by client.1508531.0:10082 2014-08-12 20:30:59.719353 2014-08-13 18:06:06.986144 7f5c973a57a0 15 read_log missing 194370'18739 (194370'18738) modify c92527b7/benchmark_data_storage0111-ib.pocstor_12495_object8256/head//24 by client.1508531.0:8257 2014-08-12 20:30:48.939251 2014-08-13 18:06:06.986187 7f5c973a57a0 15 read_log missing 194370'18737 (194370'18736) modify 9efbbbb7/benchmark_data_storage0111-ib.pocstor_12495_object5484/head//24 by client.1508531.0:5485 2014-08-12 20:30:35.334220 2014-08-13 18:06:06.986224 7f5c973a57a0 15 read_log missing 194370'18735 (194370'18734) modify b6790fb7/benchmark_data_storage0111-ib.pocstor_12495_object4217/head//24 by client.1508531.0:4218 2014-08-12 20:30:30.156160 2014-08-13 18:06:06.986260 7f5c973a57a0 15 read_log missing 194370'18733 (194370'18732) modify efc1ebb7/benchmark_data_storage0111-ib.pocstor_12495_object2037/head//24 by client.1508531.0:2038 2014-08-12 20:30:22.049716 2014-08-13 18:06:06.986300 7f5c973a57a0 15 read_log missing 194217'18731 (194217'18730) modify 8867c3b7/benchmark_data_storage0111-ib.pocstor_39693_object128/head//24 by client.1487927.0:129 2014-08-12 17:25:03.805212 2014-08-13 18:06:06.986311 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.986341 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive m=56] handle_loaded 2014-08-13 18:06:06.986358 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Initial 0.015440 0 0.000000 2014-08-13 18:06:06.986378 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 18:06:06.986403 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=0 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194487 2014-08-13 18:06:06.986414 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] log((192718'15726,194422'18845], crt=194422'18845) 2014-08-13 18:06:06.986427 7f5c973a57a0 10 osd.4 194487 pgid 24.3bf coll 24.3bf_head 2014-08-13 18:06:06.986592 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.3bf 2014-08-13 18:06:06.986606 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.986636 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.3bf(unlocked)] enter Initial 2014-08-13 18:06:06.986648 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[24.3bf(unlocked)] enter NotTrimming 2014-08-13 18:06:06.986796 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.986936 7f5c973a57a0 20 read_log 192834'12053 (0'0) promote 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.318.0:12439 2014-08-11 15:23:50.920416 2014-08-13 18:06:06.986967 7f5c973a57a0 20 read_log 192834'12054 (192834'12053) modify 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by client.933448.0:3248 2014-08-11 15:23:50.912591 2014-08-13 18:06:06.986983 7f5c973a57a0 20 read_log 192834'12055 (0'0) promote acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by osd.318.0:12441 2014-08-11 15:23:51.644583 2014-08-13 18:06:06.986996 7f5c973a57a0 20 read_log 192834'12056 (192834'12055) modify acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.933448.0:3405 2014-08-11 15:23:51.635870 2014-08-13 18:06:06.987009 7f5c973a57a0 20 read_log 192834'12073 (0'0) promote e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by osd.318.0:12459 2014-08-11 15:23:58.266628 2014-08-13 18:06:06.987021 7f5c973a57a0 20 read_log 192834'12074 (192834'12073) modify e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.933448.0:4708 2014-08-11 15:23:58.259895 2014-08-13 18:06:06.987034 7f5c973a57a0 20 read_log 192834'12081 (0'0) promote 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by osd.318.0:12467 2014-08-11 15:24:00.277004 2014-08-13 18:06:06.987046 7f5c973a57a0 20 read_log 192834'12082 (192834'12081) modify 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.933448.0:5165 2014-08-11 15:24:00.259614 2014-08-13 18:06:06.987059 7f5c973a57a0 20 read_log 192834'12089 (0'0) promote 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by osd.318.0:12475 2014-08-11 15:24:06.292097 2014-08-13 18:06:06.987071 7f5c973a57a0 20 read_log 192834'12090 (192834'12089) modify 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.933448.0:6665 2014-08-11 15:24:06.283874 2014-08-13 18:06:06.987084 7f5c973a57a0 20 read_log 192834'12135 (0'0) promote 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by osd.318.0:12521 2014-08-11 15:24:17.688344 2014-08-13 18:06:06.987096 7f5c973a57a0 20 read_log 192834'12136 (192834'12135) modify 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.933448.0:9136 2014-08-11 15:24:17.680802 2014-08-13 18:06:06.987108 7f5c973a57a0 20 read_log 192834'12143 (0'0) promote d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by osd.318.0:12529 2014-08-11 15:24:19.375199 2014-08-13 18:06:06.987120 7f5c973a57a0 20 read_log 192834'12144 (192834'12143) modify d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.933448.0:9569 2014-08-11 15:24:19.367033 2014-08-13 18:06:06.987132 7f5c973a57a0 20 read_log 192834'12153 (0'0) promote 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by osd.318.0:12539 2014-08-11 15:24:21.944902 2014-08-13 18:06:06.987144 7f5c973a57a0 20 read_log 192834'12154 (192834'12153) modify 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.933448.0:10165 2014-08-11 15:24:21.936874 2014-08-13 18:06:06.987166 7f5c973a57a0 20 read_log 192834'12157 (0'0) promote b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.318.0:12543 2014-08-11 15:24:24.512579 2014-08-13 18:06:06.987179 7f5c973a57a0 20 read_log 192834'12158 (192834'12157) modify b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by client.933448.0:10625 2014-08-11 15:24:24.503866 2014-08-13 18:06:06.987192 7f5c973a57a0 20 read_log 192834'12161 (0'0) promote 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by osd.318.0:12547 2014-08-11 15:24:25.327545 2014-08-13 18:06:06.987204 7f5c973a57a0 20 read_log 192834'12162 (192834'12161) modify 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.933448.0:10809 2014-08-11 15:24:25.315651 2014-08-13 18:06:06.987216 7f5c973a57a0 20 read_log 192834'12193 (0'0) promote c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by osd.318.0:12579 2014-08-11 15:24:35.711707 2014-08-13 18:06:06.987255 7f5c973a57a0 20 read_log 192834'12194 (192834'12193) modify c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.933448.0:13169 2014-08-11 15:24:35.701430 2014-08-13 18:06:06.987271 7f5c973a57a0 20 read_log 192834'12199 (0'0) promote 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by osd.318.0:12585 2014-08-11 15:24:36.893864 2014-08-13 18:06:06.987287 7f5c973a57a0 20 read_log 192834'12200 (192834'12199) modify 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.933448.0:13395 2014-08-11 15:24:36.883668 2014-08-13 18:06:06.987303 7f5c973a57a0 20 read_log 192834'12203 (0'0) promote e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.318.0:12589 2014-08-11 15:24:39.199826 2014-08-13 18:06:06.987315 7f5c973a57a0 20 read_log 192834'12204 (192834'12203) modify e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by client.933448.0:13579 2014-08-11 15:24:39.190927 2014-08-13 18:06:06.987328 7f5c973a57a0 20 read_log 192834'12211 (0'0) promote cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by osd.318.0:12597 2014-08-11 15:24:47.290157 2014-08-13 18:06:06.987344 7f5c973a57a0 20 read_log 192834'12212 (192834'12211) modify cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.933448.0:14816 2014-08-11 15:24:47.280651 2014-08-13 18:06:06.987357 7f5c973a57a0 20 read_log 192834'12237 (0'0) promote 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.318.0:12623 2014-08-11 15:24:59.435234 2014-08-13 18:06:06.987369 7f5c973a57a0 20 read_log 192834'12238 (192834'12237) modify 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by client.933448.0:17367 2014-08-11 15:24:59.427278 2014-08-13 18:06:06.987382 7f5c973a57a0 20 read_log 192834'12239 (0'0) promote dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.318.0:12625 2014-08-11 15:24:59.815897 2014-08-13 18:06:06.987394 7f5c973a57a0 20 read_log 192834'12240 (192834'12239) modify dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by client.933448.0:17468 2014-08-11 15:24:59.808384 2014-08-13 18:06:06.987406 7f5c973a57a0 20 read_log 192834'12251 (0'0) promote d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.318.0:12637 2014-08-11 15:25:06.224619 2014-08-13 18:06:06.987432 7f5c973a57a0 20 read_log 192834'12252 (192834'12251) modify d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by client.933448.0:18660 2014-08-11 15:25:06.216566 2014-08-13 18:06:06.987450 7f5c973a57a0 20 read_log 192834'12257 (0'0) promote 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by osd.318.0:12643 2014-08-11 15:25:07.587764 2014-08-13 18:06:06.987462 7f5c973a57a0 20 read_log 192834'12258 (192834'12257) modify 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.933448.0:18939 2014-08-11 15:25:07.578830 2014-08-13 18:06:06.987486 7f5c973a57a0 20 read_log 192834'12261 (0'0) promote f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by osd.318.0:12647 2014-08-11 15:25:08.896232 2014-08-13 18:06:06.987499 7f5c973a57a0 20 read_log 192834'12262 (192834'12261) modify f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.933448.0:19172 2014-08-11 15:25:08.888470 2014-08-13 18:06:06.987514 7f5c973a57a0 20 read_log 192834'12277 (0'0) promote 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.318.0:12663 2014-08-11 15:25:16.416550 2014-08-13 18:06:06.987535 7f5c973a57a0 20 read_log 192834'12278 (192834'12277) modify 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by client.933448.0:20083 2014-08-11 15:25:16.408423 2014-08-13 18:06:06.987549 7f5c973a57a0 20 read_log 192834'12283 (0'0) promote 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.318.0:12669 2014-08-11 15:25:21.145753 2014-08-13 18:06:06.987568 7f5c973a57a0 20 read_log 192834'12284 (192834'12283) modify 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by client.933448.0:20489 2014-08-11 15:25:21.135926 2014-08-13 18:06:06.987583 7f5c973a57a0 20 read_log 192834'12325 (192834'12012) modify e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by client.933448.0:22806 2014-08-11 15:25:38.665654 2014-08-13 18:06:06.987631 7f5c973a57a0 20 read_log 192834'12326 (192834'12014) modify 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by client.933448.0:22864 2014-08-11 15:25:38.727873 2014-08-13 18:06:06.987649 7f5c973a57a0 20 read_log 192834'12346 (192834'12054) modify 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by client.933448.0:25539 2014-08-11 15:25:42.186818 2014-08-13 18:06:06.987669 7f5c973a57a0 20 read_log 192834'12347 (192834'12056) modify acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.933448.0:25696 2014-08-11 15:25:42.346335 2014-08-13 18:06:06.987685 7f5c973a57a0 20 read_log 192834'12356 (192834'12074) modify e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.933448.0:26999 2014-08-11 15:25:43.990485 2014-08-13 18:06:06.987698 7f5c973a57a0 20 read_log 192834'12360 (192834'12082) modify 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.933448.0:27456 2014-08-11 15:25:44.517265 2014-08-13 18:06:06.987729 7f5c973a57a0 20 read_log 192834'12364 (192834'12090) modify 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.933448.0:28956 2014-08-11 15:25:46.877125 2014-08-13 18:06:06.987744 7f5c973a57a0 20 read_log 192834'12387 (192834'12136) modify 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.933448.0:31427 2014-08-11 15:25:50.050318 2014-08-13 18:06:06.987757 7f5c973a57a0 20 read_log 192834'12391 (192834'12144) modify d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.933448.0:31860 2014-08-11 15:25:50.702295 2014-08-13 18:06:06.987769 7f5c973a57a0 20 read_log 192834'12396 (192834'12154) modify 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.933448.0:32456 2014-08-11 15:25:51.836699 2014-08-13 18:06:06.987782 7f5c973a57a0 20 read_log 192834'12398 (192834'12158) modify b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by client.933448.0:32916 2014-08-11 15:25:52.564423 2014-08-13 18:06:06.987794 7f5c973a57a0 20 read_log 192834'12400 (192834'12162) modify 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.933448.0:33100 2014-08-11 15:25:52.847362 2014-08-13 18:06:06.987806 7f5c973a57a0 20 read_log 192834'12416 (192834'12194) modify c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.933448.0:35460 2014-08-11 15:25:56.311597 2014-08-13 18:06:06.987818 7f5c973a57a0 20 read_log 192834'12419 (192834'12200) modify 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.933448.0:35686 2014-08-11 15:25:56.681245 2014-08-13 18:06:06.987831 7f5c973a57a0 20 read_log 192834'12421 (192834'12204) modify e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by client.933448.0:35870 2014-08-11 15:25:56.892525 2014-08-13 18:06:06.987843 7f5c973a57a0 20 read_log 192834'12425 (192834'12212) modify cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.933448.0:37107 2014-08-11 15:25:58.943170 2014-08-13 18:06:06.987855 7f5c973a57a0 20 read_log 192834'12438 (192834'12238) modify 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by client.933448.0:39658 2014-08-11 15:26:02.013690 2014-08-13 18:06:06.987867 7f5c973a57a0 20 read_log 192834'12439 (192834'12240) modify dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by client.933448.0:39759 2014-08-11 15:26:02.064344 2014-08-13 18:06:06.987880 7f5c973a57a0 20 read_log 192834'12445 (192834'12252) modify d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by client.933448.0:40951 2014-08-11 15:26:02.796460 2014-08-13 18:06:06.987901 7f5c973a57a0 20 read_log 192834'12448 (192834'12258) modify 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.933448.0:41230 2014-08-11 15:26:02.877142 2014-08-13 18:06:06.987916 7f5c973a57a0 20 read_log 192834'12450 (192834'12262) modify f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.933448.0:41463 2014-08-11 15:26:02.949849 2014-08-13 18:06:06.987929 7f5c973a57a0 20 read_log 192834'12458 (192834'12278) modify 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by client.933448.0:42374 2014-08-11 15:26:03.258454 2014-08-13 18:06:06.987941 7f5c973a57a0 20 read_log 192834'12461 (192834'12284) modify 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by client.933448.0:42780 2014-08-11 15:26:03.384766 2014-08-13 18:06:06.987953 7f5c973a57a0 20 read_log 192834'12485 (0'0) promote f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.318.0:12871 2014-08-11 15:26:15.405606 2014-08-13 18:06:06.987966 7f5c973a57a0 20 read_log 192834'12486 (192834'12485) modify f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by client.1883634.0:247 2014-08-11 15:26:15.397583 2014-08-13 18:06:06.987978 7f5c973a57a0 20 read_log 192834'12521 (0'0) promote f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by osd.318.0:12907 2014-08-11 15:26:27.880871 2014-08-13 18:06:06.987990 7f5c973a57a0 20 read_log 192834'12522 (192834'12521) modify f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.1883634.0:3684 2014-08-11 15:26:27.873348 2014-08-13 18:06:06.988005 7f5c973a57a0 20 read_log 192834'12523 (0'0) promote c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by osd.318.0:12909 2014-08-11 15:26:28.014507 2014-08-13 18:06:06.988018 7f5c973a57a0 20 read_log 192834'12524 (192834'12523) modify c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.1883634.0:3721 2014-08-11 15:26:28.006149 2014-08-13 18:06:06.988032 7f5c973a57a0 20 read_log 192834'12527 (0'0) promote f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.318.0:12913 2014-08-11 15:26:30.069205 2014-08-13 18:06:06.988044 7f5c973a57a0 20 read_log 192834'12528 (192834'12527) modify f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by client.1883634.0:4303 2014-08-11 15:26:30.061797 2014-08-13 18:06:06.988058 7f5c973a57a0 20 read_log 192834'12547 (0'0) promote d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by osd.318.0:12933 2014-08-11 15:26:34.676141 2014-08-13 18:06:06.988071 7f5c973a57a0 20 read_log 192834'12548 (192834'12547) modify d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.1883634.0:5514 2014-08-11 15:26:34.667882 2014-08-13 18:06:06.988085 7f5c973a57a0 20 read_log 192834'12585 (0'0) promote 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by osd.318.0:12971 2014-08-11 15:26:46.686219 2014-08-13 18:06:06.988097 7f5c973a57a0 20 read_log 192834'12586 (192834'12585) modify 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.1883634.0:8514 2014-08-11 15:26:46.676044 2014-08-13 18:06:06.988112 7f5c973a57a0 20 read_log 192834'12595 (0'0) promote d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by osd.318.0:12981 2014-08-11 15:26:48.358336 2014-08-13 18:06:06.988124 7f5c973a57a0 20 read_log 192834'12596 (192834'12595) modify d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.1883634.0:8893 2014-08-11 15:26:48.347449 2014-08-13 18:06:06.988138 7f5c973a57a0 20 read_log 192834'12615 (0'0) promote a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by osd.318.0:13001 2014-08-11 15:26:55.553568 2014-08-13 18:06:06.988161 7f5c973a57a0 20 read_log 192834'12616 (192834'12615) modify a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.1883634.0:10541 2014-08-11 15:26:55.544157 2014-08-13 18:06:06.988186 7f5c973a57a0 20 read_log 192834'12619 (0'0) promote 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by osd.318.0:13005 2014-08-11 15:26:55.729145 2014-08-13 18:06:06.988201 7f5c973a57a0 20 read_log 192834'12620 (192834'12619) modify 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.1883634.0:10585 2014-08-11 15:26:55.722330 2014-08-13 18:06:06.988214 7f5c973a57a0 20 read_log 192834'12629 (0'0) promote 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.318.0:13015 2014-08-11 15:26:59.141676 2014-08-13 18:06:06.988226 7f5c973a57a0 20 read_log 192834'12630 (192834'12629) modify 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by client.1883634.0:11277 2014-08-11 15:26:59.133717 2014-08-13 18:06:06.988238 7f5c973a57a0 20 read_log 192834'12645 (0'0) promote 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by osd.318.0:13031 2014-08-11 15:27:03.092882 2014-08-13 18:06:06.988253 7f5c973a57a0 20 read_log 192834'12646 (192834'12645) modify 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.1883634.0:12225 2014-08-11 15:27:03.081015 2014-08-13 18:06:06.988268 7f5c973a57a0 20 read_log 192834'12665 (0'0) promote c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by osd.318.0:13051 2014-08-11 15:27:09.362379 2014-08-13 18:06:06.988280 7f5c973a57a0 20 read_log 192834'12666 (192834'12665) modify c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.1883634.0:13398 2014-08-11 15:27:09.352997 2014-08-13 18:06:06.988292 7f5c973a57a0 20 read_log 192834'12667 (0'0) promote ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.318.0:13053 2014-08-11 15:27:09.983996 2014-08-13 18:06:06.988308 7f5c973a57a0 20 read_log 192834'12668 (192834'12667) modify ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by client.1883634.0:13528 2014-08-11 15:27:09.975506 2014-08-13 18:06:06.988322 7f5c973a57a0 20 read_log 192834'12671 (0'0) promote 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by osd.318.0:13057 2014-08-11 15:27:11.563098 2014-08-13 18:06:06.988334 7f5c973a57a0 20 read_log 192834'12672 (192834'12671) modify 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.1883634.0:14016 2014-08-11 15:27:11.555402 2014-08-13 18:06:06.988350 7f5c973a57a0 20 read_log 192834'12681 (0'0) promote 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by osd.318.0:13067 2014-08-11 15:27:15.543281 2014-08-13 18:06:06.988365 7f5c973a57a0 20 read_log 192834'12682 (192834'12681) modify 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.1883634.0:14825 2014-08-11 15:27:15.533733 2014-08-13 18:06:06.988387 7f5c973a57a0 20 read_log 192834'12689 (0'0) promote 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.318.0:13075 2014-08-11 15:27:17.770204 2014-08-13 18:06:06.988406 7f5c973a57a0 20 read_log 192834'12690 (192834'12689) modify 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by client.1883634.0:15227 2014-08-11 15:27:17.762603 2014-08-13 18:06:06.988419 7f5c973a57a0 20 read_log 192834'12721 (0'0) promote da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.318.0:13107 2014-08-11 15:27:28.369746 2014-08-13 18:06:06.988442 7f5c973a57a0 20 read_log 192834'12722 (192834'12721) modify da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by client.1883634.0:16588 2014-08-11 15:27:28.360738 2014-08-13 18:06:06.988455 7f5c973a57a0 20 read_log 192834'12725 (0'0) promote 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.318.0:13111 2014-08-11 15:27:28.780280 2014-08-13 18:06:06.988467 7f5c973a57a0 20 read_log 192834'12726 (192834'12725) modify 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by client.1883634.0:16680 2014-08-11 15:27:28.772338 2014-08-13 18:06:06.988497 7f5c973a57a0 20 read_log 192834'12731 (0'0) promote 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by osd.318.0:13117 2014-08-11 15:27:33.942922 2014-08-13 18:06:06.988513 7f5c973a57a0 20 read_log 192834'12732 (192834'12731) modify 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.1883634.0:17261 2014-08-11 15:27:33.933177 2014-08-13 18:06:06.988534 7f5c973a57a0 20 read_log 192834'12753 (0'0) promote b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by osd.318.0:13139 2014-08-11 15:27:46.480179 2014-08-13 18:06:06.988549 7f5c973a57a0 20 read_log 192834'12754 (192834'12753) modify b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.1883634.0:18563 2014-08-11 15:27:46.470567 2014-08-13 18:06:06.988561 7f5c973a57a0 20 read_log 192834'12769 (0'0) promote ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by osd.318.0:13155 2014-08-11 15:27:52.965688 2014-08-13 18:06:06.988574 7f5c973a57a0 20 read_log 192834'12770 (192834'12769) modify ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.1883634.0:19496 2014-08-11 15:27:52.951526 2014-08-13 18:06:06.988645 7f5c973a57a0 20 read_log 192834'12789 (0'0) promote fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by osd.318.0:13175 2014-08-11 15:28:02.955588 2014-08-13 18:06:06.988662 7f5c973a57a0 20 read_log 192834'12790 (192834'12789) modify fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.1883634.0:21127 2014-08-11 15:28:02.944261 2014-08-13 18:06:06.988675 7f5c973a57a0 20 read_log 192834'12791 (0'0) promote 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by osd.318.0:13177 2014-08-11 15:28:04.991407 2014-08-13 18:06:06.988687 7f5c973a57a0 20 read_log 192834'12792 (192834'12791) modify 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.1883634.0:21696 2014-08-11 15:28:04.983250 2014-08-13 18:06:06.988699 7f5c973a57a0 20 read_log 192834'12811 (0'0) promote 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by osd.318.0:13197 2014-08-11 15:28:10.492758 2014-08-13 18:06:06.988712 7f5c973a57a0 20 read_log 192834'12812 (192834'12811) modify 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.1883634.0:22677 2014-08-11 15:28:10.482821 2014-08-13 18:06:06.988724 7f5c973a57a0 20 read_log 192834'12813 (0'0) promote f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by osd.318.0:13199 2014-08-11 15:28:10.732156 2014-08-13 18:06:06.988736 7f5c973a57a0 20 read_log 192834'12814 (192834'12813) modify f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.1883634.0:22720 2014-08-11 15:28:10.723655 2014-08-13 18:06:06.988749 7f5c973a57a0 20 read_log 192834'12817 (0'0) promote 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.318.0:13203 2014-08-11 15:28:10.750175 2014-08-13 18:06:06.988761 7f5c973a57a0 20 read_log 192834'12818 (192834'12817) modify 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by client.1883634.0:22723 2014-08-11 15:28:10.741813 2014-08-13 18:06:06.988773 7f5c973a57a0 20 read_log 192834'12838 (192834'12486) modify f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by client.1883634.0:23571 2014-08-11 15:28:17.143427 2014-08-13 18:06:06.988785 7f5c973a57a0 20 read_log 192834'12856 (192834'12522) modify f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.1883634.0:27008 2014-08-11 15:28:23.899618 2014-08-13 18:06:06.988800 7f5c973a57a0 20 read_log 192834'12857 (192834'12524) modify c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.1883634.0:27045 2014-08-11 15:28:23.935217 2014-08-13 18:06:06.988813 7f5c973a57a0 20 read_log 192834'12859 (192834'12528) modify f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by client.1883634.0:27627 2014-08-11 15:28:24.717120 2014-08-13 18:06:06.988833 7f5c973a57a0 20 read_log 192834'12869 (192834'12548) modify d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.1883634.0:28838 2014-08-11 15:28:26.361619 2014-08-13 18:06:06.988848 7f5c973a57a0 20 read_log 192834'12888 (192834'12586) modify 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.1883634.0:31838 2014-08-11 15:28:31.027454 2014-08-13 18:06:06.988860 7f5c973a57a0 20 read_log 192834'12893 (192834'12596) modify d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.1883634.0:32217 2014-08-11 15:28:31.516789 2014-08-13 18:06:06.988873 7f5c973a57a0 20 read_log 192834'12903 (192834'12616) modify a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.1883634.0:33865 2014-08-11 15:28:33.922460 2014-08-13 18:06:06.988885 7f5c973a57a0 20 read_log 192834'12905 (192834'12620) modify 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.1883634.0:33909 2014-08-11 15:28:33.994305 2014-08-13 18:06:06.988897 7f5c973a57a0 20 read_log 192834'12910 (192834'12630) modify 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by client.1883634.0:34601 2014-08-11 15:28:35.056361 2014-08-13 18:06:06.988910 7f5c973a57a0 20 read_log 192834'12920 (192834'12646) modify 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.1883634.0:35549 2014-08-11 15:28:36.821845 2014-08-13 18:06:06.988922 7f5c973a57a0 20 read_log 192834'12930 (192834'12666) modify c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.1883634.0:36722 2014-08-11 15:28:38.333998 2014-08-13 18:06:06.988934 7f5c973a57a0 20 read_log 192834'12931 (192834'12668) modify ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by client.1883634.0:36852 2014-08-11 15:28:38.491244 2014-08-13 18:06:06.988946 7f5c973a57a0 20 read_log 192834'12933 (192834'12672) modify 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.1883634.0:37340 2014-08-11 15:28:39.189103 2014-08-13 18:06:06.988958 7f5c973a57a0 20 read_log 192834'12938 (192834'12682) modify 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.1883634.0:38149 2014-08-11 15:28:40.287679 2014-08-13 18:06:06.988971 7f5c973a57a0 20 read_log 192834'12942 (192834'12690) modify 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by client.1883634.0:38551 2014-08-11 15:28:40.832874 2014-08-13 18:06:06.988983 7f5c973a57a0 20 read_log 192834'12958 (192834'12722) modify da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by client.1883634.0:39912 2014-08-11 15:28:42.750468 2014-08-13 18:06:06.988995 7f5c973a57a0 20 read_log 192834'12960 (192834'12726) modify 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by client.1883634.0:40004 2014-08-11 15:28:43.020209 2014-08-13 18:06:06.989007 7f5c973a57a0 20 read_log 192834'12963 (192834'12732) modify 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.1883634.0:40585 2014-08-11 15:28:44.023379 2014-08-13 18:06:06.989019 7f5c973a57a0 20 read_log 192834'12974 (192834'12754) modify b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.1883634.0:41887 2014-08-11 15:28:45.563270 2014-08-13 18:06:06.989034 7f5c973a57a0 20 read_log 192834'12982 (192834'12770) modify ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.1883634.0:42820 2014-08-11 15:28:46.368857 2014-08-13 18:06:06.989054 7f5c973a57a0 20 read_log 192834'12992 (192834'12790) modify fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.1883634.0:44451 2014-08-11 15:28:46.860992 2014-08-13 18:06:06.989068 7f5c973a57a0 20 read_log 192834'12993 (192834'12792) modify 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.1883634.0:45020 2014-08-11 15:28:47.124501 2014-08-13 18:06:06.989080 7f5c973a57a0 20 read_log 192834'13003 (192834'12812) modify 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.1883634.0:46001 2014-08-11 15:28:47.460485 2014-08-13 18:06:06.989126 7f5c973a57a0 20 read_log 192834'13004 (192834'12814) modify f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.1883634.0:46044 2014-08-11 15:28:47.473068 2014-08-13 18:06:06.989142 7f5c973a57a0 20 read_log 192834'13006 (192834'12818) modify 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by client.1883634.0:46047 2014-08-11 15:28:47.473955 2014-08-13 18:06:06.989161 7f5c973a57a0 20 read_log 192888'13018 (0'0) promote 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by osd.63.0:2341 2014-08-11 16:44:56.514349 2014-08-13 18:06:06.989176 7f5c973a57a0 20 read_log 192888'13019 (192888'13018) modify 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.1315259.0:381 2014-08-11 16:44:56.506779 2014-08-13 18:06:06.989189 7f5c973a57a0 20 read_log 192888'13020 (0'0) promote 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by osd.63.0:2350 2014-08-11 16:44:58.239975 2014-08-13 18:06:06.989201 7f5c973a57a0 20 read_log 192888'13021 (192888'13020) modify 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.1315259.0:779 2014-08-11 16:44:58.234225 2014-08-13 18:06:06.989213 7f5c973a57a0 20 read_log 192888'13024 (0'0) promote 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by osd.63.0:2354 2014-08-11 16:44:58.512734 2014-08-13 18:06:06.989228 7f5c973a57a0 20 read_log 192888'13025 (192888'13024) modify 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.1315259.0:852 2014-08-11 16:44:58.507292 2014-08-13 18:06:06.989243 7f5c973a57a0 20 read_log 192888'13028 (0'0) promote b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by osd.63.0:2358 2014-08-11 16:44:59.297270 2014-08-13 18:06:06.989258 7f5c973a57a0 20 read_log 192888'13029 (192888'13028) modify b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.1315259.0:1064 2014-08-11 16:44:59.291081 2014-08-13 18:06:06.989270 7f5c973a57a0 20 read_log 192888'13032 (0'0) promote dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by osd.63.0:2372 2014-08-11 16:45:03.242153 2014-08-13 18:06:06.989282 7f5c973a57a0 20 read_log 192888'13033 (192888'13032) modify dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.1315259.0:2100 2014-08-11 16:45:03.236033 2014-08-13 18:06:06.989300 7f5c973a57a0 20 read_log 192888'13038 (0'0) promote ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by osd.63.0:2380 2014-08-11 16:45:04.263051 2014-08-13 18:06:06.989313 7f5c973a57a0 20 read_log 192888'13039 (192888'13038) modify ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.1315259.0:2377 2014-08-11 16:45:04.257977 2014-08-13 18:06:06.989325 7f5c973a57a0 20 read_log 192888'13044 (0'0) promote 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by osd.63.0:2412 2014-08-11 16:45:14.210476 2014-08-13 18:06:06.989341 7f5c973a57a0 20 read_log 192888'13045 (192888'13044) modify 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.1315259.0:5030 2014-08-11 16:45:14.202884 2014-08-13 18:06:06.989356 7f5c973a57a0 20 read_log 192888'13048 (0'0) promote 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by osd.63.0:2416 2014-08-11 16:45:15.950716 2014-08-13 18:06:06.989368 7f5c973a57a0 20 read_log 192888'13049 (192888'13048) modify 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.1315259.0:5493 2014-08-11 16:45:15.944932 2014-08-13 18:06:06.989396 7f5c973a57a0 20 read_log 192888'13050 (0'0) promote 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.63.0:2418 2014-08-11 16:45:16.133226 2014-08-13 18:06:06.989409 7f5c973a57a0 20 read_log 192888'13051 (192888'13050) modify 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by client.1315259.0:5544 2014-08-11 16:45:16.126949 2014-08-13 18:06:06.989461 7f5c973a57a0 20 read_log 192888'13058 (0'0) promote 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by osd.63.0:2436 2014-08-11 16:45:21.623701 2014-08-13 18:06:06.989489 7f5c973a57a0 20 read_log 192888'13059 (192888'13058) modify 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.1315259.0:7063 2014-08-11 16:45:21.617468 2014-08-13 18:06:06.989503 7f5c973a57a0 20 read_log 192888'13066 (0'0) promote f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.63.0:2470 2014-08-11 16:45:31.268004 2014-08-13 18:06:06.989515 7f5c973a57a0 20 read_log 192888'13067 (192888'13066) modify f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by client.1315259.0:9641 2014-08-11 16:45:31.262391 2014-08-13 18:06:06.989528 7f5c973a57a0 20 read_log 192888'13072 (0'0) promote 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.63.0:2480 2014-08-11 16:45:34.485517 2014-08-13 18:06:06.989540 7f5c973a57a0 20 read_log 192888'13073 (192888'13072) modify 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by client.1315259.0:10469 2014-08-11 16:45:34.479256 2014-08-13 18:06:06.989552 7f5c973a57a0 20 read_log 192888'13080 (0'0) promote 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by osd.63.0:2496 2014-08-11 16:45:40.703724 2014-08-13 18:06:06.989565 7f5c973a57a0 20 read_log 192888'13081 (192888'13080) modify 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.1315259.0:12006 2014-08-11 16:45:40.698252 2014-08-13 18:06:06.989582 7f5c973a57a0 20 read_log 192888'13086 (0'0) promote 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by osd.63.0:2548 2014-08-11 16:45:53.959373 2014-08-13 18:06:06.989595 7f5c973a57a0 20 read_log 192888'13087 (192888'13086) modify 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.1315259.0:15366 2014-08-11 16:45:53.954108 2014-08-13 18:06:06.989607 7f5c973a57a0 20 read_log 192888'13092 (0'0) promote 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by osd.63.0:2564 2014-08-11 16:46:02.201034 2014-08-13 18:06:06.989619 7f5c973a57a0 20 read_log 192888'13093 (192888'13092) modify 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.1315259.0:17475 2014-08-11 16:46:02.195207 2014-08-13 18:06:06.989640 7f5c973a57a0 20 read_log 192888'13096 (0'0) promote 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.63.0:2568 2014-08-11 16:46:03.727654 2014-08-13 18:06:06.989656 7f5c973a57a0 20 read_log 192888'13097 (192888'13096) modify 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by client.1315259.0:17844 2014-08-11 16:46:03.721406 2014-08-13 18:06:06.989669 7f5c973a57a0 20 read_log 192888'13100 (0'0) promote f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by osd.63.0:2574 2014-08-11 16:46:05.981573 2014-08-13 18:06:06.989682 7f5c973a57a0 20 read_log 192888'13101 (192888'13100) modify f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.1315259.0:18419 2014-08-11 16:46:05.975916 2014-08-13 18:06:06.989694 7f5c973a57a0 20 read_log 192888'13102 (0'0) promote 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by osd.63.0:2576 2014-08-11 16:46:06.869613 2014-08-13 18:06:06.989707 7f5c973a57a0 20 read_log 192888'13103 (192888'13102) modify 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.1315259.0:18656 2014-08-11 16:46:06.864396 2014-08-13 18:06:06.989719 7f5c973a57a0 20 read_log 192888'13104 (0'0) promote 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by osd.63.0:2578 2014-08-11 16:46:07.459680 2014-08-13 18:06:06.989731 7f5c973a57a0 20 read_log 192888'13105 (192888'13104) modify 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.1315259.0:18813 2014-08-11 16:46:07.454120 2014-08-13 18:06:06.989751 7f5c973a57a0 20 read_log 192888'13110 (0'0) promote 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.63.0:2602 2014-08-11 16:46:18.780546 2014-08-13 18:06:06.989763 7f5c973a57a0 20 read_log 192888'13111 (192888'13110) modify 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by client.1315259.0:21764 2014-08-11 16:46:18.774353 2014-08-13 18:06:06.989775 7f5c973a57a0 20 read_log 192888'13114 (0'0) promote 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by osd.63.0:2618 2014-08-11 16:46:23.058543 2014-08-13 18:06:06.989788 7f5c973a57a0 20 read_log 192888'13115 (192888'13114) modify 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.1315259.0:22872 2014-08-11 16:46:23.052571 2014-08-13 18:06:06.989800 7f5c973a57a0 20 read_log 192888'13118 (0'0) promote e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by osd.63.0:2636 2014-08-11 16:46:28.896787 2014-08-13 18:06:06.989825 7f5c973a57a0 20 read_log 192888'13119 (192888'13118) modify e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.1315259.0:24336 2014-08-11 16:46:28.890831 2014-08-13 18:06:06.989840 7f5c973a57a0 20 read_log 192888'13122 (0'0) promote 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by osd.63.0:2648 2014-08-11 16:46:32.245392 2014-08-13 18:06:06.989853 7f5c973a57a0 20 read_log 192888'13123 (192888'13122) modify 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.1315259.0:25185 2014-08-11 16:46:32.239510 2014-08-13 18:06:06.989865 7f5c973a57a0 20 read_log 192888'13124 (0'0) promote 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by osd.63.0:2660 2014-08-11 16:46:36.396936 2014-08-13 18:06:06.989877 7f5c973a57a0 20 read_log 192888'13125 (192888'13124) modify 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.1315259.0:26149 2014-08-11 16:46:36.389546 2014-08-13 18:06:06.989892 7f5c973a57a0 20 read_log 192888'13132 (0'0) promote c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by osd.63.0:2678 2014-08-11 16:46:39.864162 2014-08-13 18:06:06.989905 7f5c973a57a0 20 read_log 192888'13133 (192888'13132) modify c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.1315259.0:26996 2014-08-11 16:46:39.858450 2014-08-13 18:06:06.989917 7f5c973a57a0 20 read_log 192888'13136 (0'0) promote adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by osd.63.0:2694 2014-08-11 16:46:47.332561 2014-08-13 18:06:06.989929 7f5c973a57a0 20 read_log 192888'13137 (192888'13136) modify adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.1315259.0:28819 2014-08-11 16:46:47.326675 2014-08-13 18:06:06.989948 7f5c973a57a0 20 read_log 192888'13140 (0'0) promote d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by osd.63.0:2700 2014-08-11 16:46:48.850958 2014-08-13 18:06:06.989960 7f5c973a57a0 20 read_log 192888'13141 (192888'13140) modify d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.1315259.0:29204 2014-08-11 16:46:48.843197 2014-08-13 18:06:06.989973 7f5c973a57a0 20 read_log 192888'13144 (0'0) promote f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by osd.63.0:2704 2014-08-11 16:46:50.194159 2014-08-13 18:06:06.989988 7f5c973a57a0 20 read_log 192888'13145 (192888'13144) modify f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.1315259.0:29513 2014-08-11 16:46:50.188457 2014-08-13 18:06:06.990001 7f5c973a57a0 20 read_log 192888'13147 (192888'13019) modify 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.1315259.0:31108 2014-08-11 16:46:56.000250 2014-08-13 18:06:06.990026 7f5c973a57a0 20 read_log 192888'13148 (192888'13021) modify 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.1315259.0:31506 2014-08-11 16:46:56.114865 2014-08-13 18:06:06.990061 7f5c973a57a0 20 read_log 192888'13150 (192888'13025) modify 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.1315259.0:31579 2014-08-11 16:46:56.135246 2014-08-13 18:06:06.990078 7f5c973a57a0 20 read_log 192888'13152 (192888'13029) modify b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.1315259.0:31791 2014-08-11 16:46:56.199828 2014-08-13 18:06:06.990092 7f5c973a57a0 20 read_log 192888'13154 (192888'13033) modify dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.1315259.0:32827 2014-08-11 16:46:56.480045 2014-08-13 18:06:06.990112 7f5c973a57a0 20 read_log 192888'13157 (192888'13039) modify ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.1315259.0:33104 2014-08-11 16:46:56.554071 2014-08-13 18:06:06.990128 7f5c973a57a0 20 read_log 192888'13160 (192888'13045) modify 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.1315259.0:35757 2014-08-11 16:46:57.238752 2014-08-13 18:06:06.990141 7f5c973a57a0 20 read_log 192888'13162 (192888'13049) modify 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.1315259.0:36220 2014-08-11 16:46:57.366910 2014-08-13 18:06:06.990163 7f5c973a57a0 20 read_log 192888'13163 (192888'13051) modify 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by client.1315259.0:36271 2014-08-11 16:46:57.378361 2014-08-13 18:06:06.990176 7f5c973a57a0 20 read_log 192888'13167 (192888'13059) modify 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.1315259.0:37790 2014-08-11 16:46:57.774425 2014-08-13 18:06:06.990191 7f5c973a57a0 20 read_log 192888'13171 (192888'13067) modify f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by client.1315259.0:40368 2014-08-11 16:46:58.464763 2014-08-13 18:06:06.990206 7f5c973a57a0 20 read_log 192888'13174 (192888'13073) modify 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by client.1315259.0:41196 2014-08-11 16:46:58.683595 2014-08-13 18:06:06.990218 7f5c973a57a0 20 read_log 192888'13178 (192888'13081) modify 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.1315259.0:42733 2014-08-11 16:46:59.089662 2014-08-13 18:06:06.990230 7f5c973a57a0 20 read_log 192888'13181 (192888'13087) modify 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.1315259.0:46093 2014-08-11 16:47:00.058626 2014-08-13 18:06:06.990249 7f5c973a57a0 20 read_log 192888'13184 (192888'13093) modify 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.1315259.0:48202 2014-08-11 16:47:00.695736 2014-08-13 18:06:06.990270 7f5c973a57a0 20 read_log 192888'13186 (192888'13097) modify 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by client.1315259.0:48571 2014-08-11 16:47:00.801205 2014-08-13 18:06:06.990288 7f5c973a57a0 20 read_log 192888'13188 (192888'13101) modify f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.1315259.0:49146 2014-08-11 16:47:00.970794 2014-08-13 18:06:06.990303 7f5c973a57a0 20 read_log 192888'13189 (192888'13103) modify 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.1315259.0:49383 2014-08-11 16:47:01.044349 2014-08-13 18:06:06.990326 7f5c973a57a0 20 read_log 192888'13190 (192888'13105) modify 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.1315259.0:49540 2014-08-11 16:47:01.095165 2014-08-13 18:06:06.990344 7f5c973a57a0 20 read_log 192888'13193 (192888'13111) modify 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by client.1315259.0:52491 2014-08-11 16:47:01.989906 2014-08-13 18:06:06.990357 7f5c973a57a0 20 read_log 192888'13195 (192888'13115) modify 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.1315259.0:53599 2014-08-11 16:47:02.302146 2014-08-13 18:06:06.990379 7f5c973a57a0 20 read_log 192888'13197 (192888'13119) modify e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.1315259.0:55063 2014-08-11 16:47:02.799704 2014-08-13 18:06:06.990399 7f5c973a57a0 20 read_log 192888'13199 (192888'13123) modify 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.1315259.0:55912 2014-08-11 16:47:03.050423 2014-08-13 18:06:06.990435 7f5c973a57a0 20 read_log 192888'13200 (192888'13125) modify 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.1315259.0:56876 2014-08-11 16:47:03.339329 2014-08-13 18:06:06.990448 7f5c973a57a0 20 read_log 192888'13204 (192888'13133) modify c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.1315259.0:57723 2014-08-11 16:47:03.593676 2014-08-13 18:06:06.990460 7f5c973a57a0 20 read_log 192888'13206 (192888'13137) modify adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.1315259.0:59546 2014-08-11 16:47:04.154934 2014-08-13 18:06:06.990473 7f5c973a57a0 20 read_log 192888'13208 (192888'13141) modify d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.1315259.0:59931 2014-08-11 16:47:04.287965 2014-08-13 18:06:06.990485 7f5c973a57a0 20 read_log 192888'13210 (192888'13145) modify f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.1315259.0:60240 2014-08-11 16:47:04.389446 2014-08-13 18:06:06.990497 7f5c973a57a0 20 read_log 192888'13213 (0'0) promote 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by osd.63.0:2908 2014-08-11 17:05:16.026506 2014-08-13 18:06:06.990510 7f5c973a57a0 20 read_log 192888'13214 (192888'13213) modify 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.942868.0:251 2014-08-11 17:05:16.020273 2014-08-13 18:06:06.990522 7f5c973a57a0 20 read_log 192888'13215 (0'0) promote bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by osd.63.0:2914 2014-08-11 17:05:17.173859 2014-08-13 18:06:06.990535 7f5c973a57a0 20 read_log 192888'13216 (192888'13215) modify bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.942868.0:523 2014-08-11 17:05:17.168743 2014-08-13 18:06:06.990549 7f5c973a57a0 20 read_log 192888'13217 (0'0) promote d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.63.0:2916 2014-08-11 17:05:18.714615 2014-08-13 18:06:06.990561 7f5c973a57a0 20 read_log 192888'13218 (192888'13217) modify d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by client.942868.0:904 2014-08-11 17:05:18.708782 2014-08-13 18:06:06.990574 7f5c973a57a0 20 read_log 192888'13225 (0'0) promote e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by osd.63.0:2929 2014-08-11 17:05:21.997270 2014-08-13 18:06:06.990601 7f5c973a57a0 20 read_log 192888'13226 (192888'13225) modify e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.942868.0:1690 2014-08-11 17:05:21.991088 2014-08-13 18:06:06.990614 7f5c973a57a0 20 read_log 192888'13231 (0'0) promote bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by osd.63.0:2953 2014-08-11 17:05:27.591804 2014-08-13 18:06:06.990627 7f5c973a57a0 20 read_log 192888'13232 (192888'13231) modify bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.942868.0:3149 2014-08-11 17:05:27.586004 2014-08-13 18:06:06.990639 7f5c973a57a0 20 read_log 192888'13237 (0'0) promote 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by osd.63.0:2995 2014-08-11 17:05:40.841671 2014-08-13 18:06:06.990651 7f5c973a57a0 20 read_log 192888'13238 (192888'13237) modify 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.942868.0:6701 2014-08-11 17:05:40.835695 2014-08-13 18:06:06.990664 7f5c973a57a0 20 read_log 192888'13241 (0'0) promote 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.63.0:3017 2014-08-11 17:05:48.814057 2014-08-13 18:06:06.990676 7f5c973a57a0 20 read_log 192888'13242 (192888'13241) modify 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by client.942868.0:8805 2014-08-11 17:05:48.808790 2014-08-13 18:06:06.990695 7f5c973a57a0 20 read_log 192888'13243 (0'0) promote c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by osd.63.0:3023 2014-08-11 17:05:51.151986 2014-08-13 18:06:06.990708 7f5c973a57a0 20 read_log 192888'13244 (192888'13243) modify c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.942868.0:9433 2014-08-11 17:05:51.146275 2014-08-13 18:06:06.990720 7f5c973a57a0 20 read_log 192888'13247 (0'0) promote ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by osd.63.0:3039 2014-08-11 17:05:53.476772 2014-08-13 18:06:06.990732 7f5c973a57a0 20 read_log 192888'13248 (192888'13247) modify ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.942868.0:10075 2014-08-11 17:05:53.471438 2014-08-13 18:06:06.990745 7f5c973a57a0 20 read_log 192888'13249 (0'0) promote 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by osd.63.0:3041 2014-08-11 17:05:53.547422 2014-08-13 18:06:06.990757 7f5c973a57a0 20 read_log 192888'13250 (192888'13249) modify 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.942868.0:10095 2014-08-11 17:05:53.542405 2014-08-13 18:06:06.990769 7f5c973a57a0 20 read_log 192888'13253 (0'0) promote 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by osd.63.0:3049 2014-08-11 17:05:56.024802 2014-08-13 18:06:06.990781 7f5c973a57a0 20 read_log 192888'13254 (192888'13253) modify 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.942868.0:10785 2014-08-11 17:05:56.018522 2014-08-13 18:06:06.990793 7f5c973a57a0 20 read_log 192888'13261 (0'0) promote 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.63.0:3085 2014-08-11 17:06:04.546055 2014-08-13 18:06:06.990805 7f5c973a57a0 20 read_log 192888'13262 (192888'13261) modify 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by client.942868.0:13100 2014-08-11 17:06:04.541004 2014-08-13 18:06:06.990818 7f5c973a57a0 20 read_log 192888'13263 (0'0) promote e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by osd.63.0:3089 2014-08-11 17:06:06.461972 2014-08-13 18:06:06.990830 7f5c973a57a0 20 read_log 192888'13264 (192888'13263) modify e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.942868.0:13575 2014-08-11 17:06:06.456216 2014-08-13 18:06:06.990842 7f5c973a57a0 20 read_log 192888'13265 (0'0) promote fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by osd.63.0:3091 2014-08-11 17:06:07.361719 2014-08-13 18:06:06.990854 7f5c973a57a0 20 read_log 192888'13266 (192888'13265) modify fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.942868.0:13800 2014-08-11 17:06:07.356195 2014-08-13 18:06:06.990866 7f5c973a57a0 20 read_log 192888'13273 (0'0) promote 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by osd.63.0:3111 2014-08-11 17:06:20.131968 2014-08-13 18:06:06.990879 7f5c973a57a0 20 read_log 192888'13274 (192888'13273) modify 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.942868.0:17019 2014-08-11 17:06:20.126452 2014-08-13 18:06:06.990891 7f5c973a57a0 20 read_log 192888'13275 (0'0) promote 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.63.0:3115 2014-08-11 17:06:20.455417 2014-08-13 18:06:06.990903 7f5c973a57a0 20 read_log 192888'13276 (192888'13275) modify 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by client.942868.0:17105 2014-08-11 17:06:20.450218 2014-08-13 18:06:06.990915 7f5c973a57a0 20 read_log 192888'13277 (0'0) promote 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.63.0:3123 2014-08-11 17:06:23.470737 2014-08-13 18:06:06.990928 7f5c973a57a0 20 read_log 192888'13278 (192888'13277) modify 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by client.942868.0:17905 2014-08-11 17:06:23.465589 2014-08-13 18:06:06.990949 7f5c973a57a0 20 read_log 192888'13281 (0'0) promote 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by osd.63.0:3147 2014-08-11 17:06:35.391110 2014-08-13 18:06:06.990965 7f5c973a57a0 20 read_log 192888'13282 (192888'13281) modify 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.942868.0:21052 2014-08-11 17:06:35.385330 2014-08-13 18:06:06.990977 7f5c973a57a0 20 read_log 192888'13287 (0'0) promote 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by osd.63.0:3159 2014-08-11 17:06:38.432324 2014-08-13 18:06:06.990991 7f5c973a57a0 20 read_log 192888'13288 (192888'13287) modify 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.942868.0:21854 2014-08-11 17:06:38.426622 2014-08-13 18:06:06.991004 7f5c973a57a0 20 read_log 192888'13289 (0'0) promote 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by osd.63.0:3163 2014-08-11 17:06:38.894415 2014-08-13 18:06:06.991018 7f5c973a57a0 20 read_log 192888'13290 (192888'13289) modify 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.942868.0:21978 2014-08-11 17:06:38.888769 2014-08-13 18:06:06.991031 7f5c973a57a0 20 read_log 192888'13291 (0'0) promote 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by osd.63.0:3165 2014-08-11 17:06:39.569837 2014-08-13 18:06:06.991043 7f5c973a57a0 20 read_log 192888'13292 (192888'13291) modify 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.942868.0:22162 2014-08-11 17:06:39.564177 2014-08-13 18:06:06.991057 7f5c973a57a0 20 read_log 192888'13293 (0'0) promote 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by osd.63.0:3175 2014-08-11 17:06:42.516783 2014-08-13 18:06:06.991069 7f5c973a57a0 20 read_log 192888'13294 (192888'13293) modify 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.942868.0:22909 2014-08-11 17:06:42.510832 2014-08-13 18:06:06.991083 7f5c973a57a0 20 read_log 192888'13295 (0'0) promote cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by osd.63.0:3177 2014-08-11 17:06:44.928057 2014-08-13 18:06:06.991097 7f5c973a57a0 20 read_log 192888'13296 (192888'13295) modify cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.942868.0:23517 2014-08-11 17:06:44.922386 2014-08-13 18:06:06.991119 7f5c973a57a0 20 read_log 192888'13297 (0'0) promote a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by osd.63.0:3183 2014-08-11 17:06:46.837513 2014-08-13 18:06:06.991131 7f5c973a57a0 20 read_log 192888'13298 (192888'13297) modify a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.942868.0:24034 2014-08-11 17:06:46.831311 2014-08-13 18:06:06.991150 7f5c973a57a0 20 read_log 192888'13305 (0'0) promote 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by osd.63.0:3193 2014-08-11 17:06:49.967136 2014-08-13 18:06:06.991166 7f5c973a57a0 20 read_log 192888'13306 (192888'13305) modify 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.942868.0:24881 2014-08-11 17:06:49.961602 2014-08-13 18:06:06.991179 7f5c973a57a0 20 read_log 192888'13309 (0'0) promote 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by osd.63.0:3197 2014-08-11 17:06:50.766202 2014-08-13 18:06:06.991193 7f5c973a57a0 20 read_log 192888'13310 (192888'13309) modify 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.942868.0:25095 2014-08-11 17:06:50.760186 2014-08-13 18:06:06.991205 7f5c973a57a0 20 read_log 192888'13311 (0'0) promote e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by osd.63.0:3201 2014-08-11 17:06:51.567773 2014-08-13 18:06:06.991218 7f5c973a57a0 20 read_log 192888'13312 (192888'13311) modify e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.942868.0:25305 2014-08-11 17:06:51.561765 2014-08-13 18:06:06.991237 7f5c973a57a0 20 read_log 192888'13315 (0'0) promote 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.63.0:3211 2014-08-11 17:06:55.163074 2014-08-13 18:06:06.991255 7f5c973a57a0 20 read_log 192888'13316 (192888'13315) modify 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by client.942868.0:26188 2014-08-11 17:06:55.157641 2014-08-13 18:06:06.991270 7f5c973a57a0 20 read_log 192888'13317 (0'0) promote 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.63.0:3213 2014-08-11 17:06:55.991229 2014-08-13 18:06:06.991283 7f5c973a57a0 20 read_log 192888'13318 (192888'13317) modify 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by client.942868.0:26411 2014-08-11 17:06:55.985065 2014-08-13 18:06:06.991295 7f5c973a57a0 20 read_log 192888'13327 (0'0) promote a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by osd.63.0:3265 2014-08-11 17:07:08.595537 2014-08-13 18:06:06.991313 7f5c973a57a0 20 read_log 192888'13328 (192888'13327) modify a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.942868.0:29694 2014-08-11 17:07:08.589508 2014-08-13 18:06:06.991326 7f5c973a57a0 20 read_log 192888'13329 (0'0) promote afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by osd.63.0:3285 2014-08-11 17:07:14.606968 2014-08-13 18:06:06.991338 7f5c973a57a0 20 read_log 192888'13330 (192888'13329) modify afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.942868.0:31209 2014-08-11 17:07:14.600171 2014-08-13 18:06:06.991357 7f5c973a57a0 20 read_log 192888'13331 (192888'13214) modify 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.942868.0:31517 2014-08-11 17:07:15.949907 2014-08-13 18:06:06.991372 7f5c973a57a0 20 read_log 192888'13332 (192888'13216) modify bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.942868.0:31789 2014-08-11 17:07:16.058045 2014-08-13 18:06:06.991400 7f5c973a57a0 20 read_log 192888'13333 (192888'13218) modify d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by client.942868.0:32170 2014-08-11 17:07:16.172900 2014-08-13 18:06:06.991415 7f5c973a57a0 20 read_log 192888'13337 (192888'13226) modify e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.942868.0:32956 2014-08-11 17:07:16.386731 2014-08-13 18:06:06.991435 7f5c973a57a0 20 read_log 192888'13340 (192888'13232) modify bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.942868.0:34415 2014-08-11 17:07:16.781081 2014-08-13 18:06:06.991451 7f5c973a57a0 20 read_log 192888'13343 (192888'13238) modify 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.942868.0:37967 2014-08-11 17:07:17.756254 2014-08-13 18:06:06.991463 7f5c973a57a0 20 read_log 192888'13345 (192888'13242) modify 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by client.942868.0:40071 2014-08-11 17:07:18.357573 2014-08-13 18:06:06.991486 7f5c973a57a0 20 read_log 192888'13346 (192888'13244) modify c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.942868.0:40699 2014-08-11 17:07:18.524456 2014-08-13 18:06:06.991502 7f5c973a57a0 20 read_log 192888'13348 (192888'13248) modify ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.942868.0:41341 2014-08-11 17:07:18.703708 2014-08-13 18:06:06.991514 7f5c973a57a0 20 read_log 192888'13349 (192888'13250) modify 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.942868.0:41361 2014-08-11 17:07:18.708485 2014-08-13 18:06:06.991529 7f5c973a57a0 20 read_log 192888'13351 (192888'13254) modify 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.942868.0:42051 2014-08-11 17:07:18.892404 2014-08-13 18:06:06.991542 7f5c973a57a0 20 read_log 192888'13355 (192888'13262) modify 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by client.942868.0:44366 2014-08-11 17:07:19.546908 2014-08-13 18:06:06.991583 7f5c973a57a0 20 read_log 192888'13356 (192888'13264) modify e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.942868.0:44841 2014-08-11 17:07:19.676915 2014-08-13 18:06:06.991600 7f5c973a57a0 20 read_log 192888'13357 (192888'13266) modify fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.942868.0:45066 2014-08-11 17:07:19.737132 2014-08-13 18:06:06.991613 7f5c973a57a0 20 read_log 192888'13361 (192888'13274) modify 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.942868.0:48285 2014-08-11 17:07:20.675970 2014-08-13 18:06:06.991633 7f5c973a57a0 20 read_log 192888'13362 (192888'13276) modify 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by client.942868.0:48371 2014-08-11 17:07:20.699647 2014-08-13 18:06:06.991649 7f5c973a57a0 20 read_log 192888'13363 (192888'13278) modify 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by client.942868.0:49171 2014-08-11 17:07:20.930478 2014-08-13 18:06:06.991661 7f5c973a57a0 20 read_log 192888'13365 (192888'13282) modify 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.942868.0:52318 2014-08-11 17:07:21.902490 2014-08-13 18:06:06.991673 7f5c973a57a0 20 read_log 192888'13368 (192888'13288) modify 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.942868.0:53120 2014-08-11 17:07:22.154653 2014-08-13 18:06:06.991686 7f5c973a57a0 20 read_log 192888'13369 (192888'13290) modify 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.942868.0:53244 2014-08-11 17:07:22.197483 2014-08-13 18:06:06.991698 7f5c973a57a0 20 read_log 192888'13370 (192888'13292) modify 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.942868.0:53428 2014-08-11 17:07:22.259563 2014-08-13 18:06:06.991710 7f5c973a57a0 20 read_log 192888'13371 (192888'13294) modify 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.942868.0:54175 2014-08-11 17:07:22.483793 2014-08-13 18:06:06.991723 7f5c973a57a0 20 read_log 192888'13372 (192888'13296) modify cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.942868.0:54783 2014-08-11 17:07:22.655131 2014-08-13 18:06:06.991755 7f5c973a57a0 20 read_log 192888'13373 (192888'13298) modify a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.942868.0:55300 2014-08-11 17:07:22.804687 2014-08-13 18:06:06.991772 7f5c973a57a0 20 read_log 192888'13377 (192888'13306) modify 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.942868.0:56147 2014-08-11 17:07:23.065609 2014-08-13 18:06:06.991785 7f5c973a57a0 20 read_log 192888'13379 (192888'13310) modify 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.942868.0:56361 2014-08-11 17:07:23.126742 2014-08-13 18:06:06.991797 7f5c973a57a0 20 read_log 192888'13380 (192888'13312) modify e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.942868.0:56571 2014-08-11 17:07:23.194927 2014-08-13 18:06:06.991810 7f5c973a57a0 20 read_log 192888'13382 (192888'13316) modify 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by client.942868.0:57454 2014-08-11 17:07:23.459797 2014-08-13 18:06:06.991822 7f5c973a57a0 20 read_log 192888'13383 (192888'13318) modify 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by client.942868.0:57677 2014-08-11 17:07:23.524921 2014-08-13 18:06:06.991834 7f5c973a57a0 20 read_log 192888'13388 (192888'13328) modify a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.942868.0:60960 2014-08-11 17:07:24.495758 2014-08-13 18:06:06.991847 7f5c973a57a0 20 read_log 192888'13389 (192888'13330) modify afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.942868.0:62475 2014-08-11 17:07:24.959035 2014-08-13 18:06:06.991859 7f5c973a57a0 20 read_log 193305'13390 (0'0) promote d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by osd.27.0:14 2014-08-11 18:42:41.077521 2014-08-13 18:06:06.991879 7f5c973a57a0 20 read_log 193305'13391 (193305'13390) modify d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.951337.0:2087 2014-08-11 18:42:41.069328 2014-08-13 18:06:06.991894 7f5c973a57a0 20 read_log 193305'13392 (0'0) promote 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by osd.27.0:24 2014-08-11 18:42:45.983295 2014-08-13 18:06:06.991907 7f5c973a57a0 20 read_log 193305'13393 (193305'13392) modify 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.951337.0:3437 2014-08-11 18:42:45.975808 2014-08-13 18:06:06.991921 7f5c973a57a0 20 read_log 193305'13394 (0'0) promote 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by osd.27.0:36 2014-08-11 18:42:47.244125 2014-08-13 18:06:06.991933 7f5c973a57a0 20 read_log 193305'13395 (193305'13394) modify 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.951337.0:3787 2014-08-11 18:42:47.237528 2014-08-13 18:06:06.991948 7f5c973a57a0 20 read_log 193305'13396 (0'0) promote a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:40 2014-08-11 18:42:47.464134 2014-08-13 18:06:06.991977 7f5c973a57a0 20 read_log 193305'13397 (193305'13396) modify a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by client.951337.0:3846 2014-08-11 18:42:47.457381 2014-08-13 18:06:06.991992 7f5c973a57a0 20 read_log 193305'13398 (0'0) promote 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:58 2014-08-11 18:42:59.280930 2014-08-13 18:06:06.992005 7f5c973a57a0 20 read_log 193305'13399 (193305'13398) modify 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by client.951337.0:7065 2014-08-11 18:42:59.274133 2014-08-13 18:06:06.992019 7f5c973a57a0 20 read_log 193305'13400 (0'0) promote df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by osd.27.0:66 2014-08-11 18:43:02.062890 2014-08-13 18:06:06.992032 7f5c973a57a0 20 read_log 193305'13401 (193305'13400) modify df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.951337.0:7835 2014-08-11 18:43:02.056854 2014-08-13 18:06:06.992046 7f5c973a57a0 20 read_log 193305'13402 (0'0) promote 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by osd.27.0:88 2014-08-11 18:43:06.690295 2014-08-13 18:06:06.992059 7f5c973a57a0 20 read_log 193305'13403 (193305'13402) modify 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.951337.0:9015 2014-08-11 18:43:06.681386 2014-08-13 18:06:06.992074 7f5c973a57a0 20 read_log 193305'13404 (0'0) promote 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:104 2014-08-11 18:43:11.101945 2014-08-13 18:06:06.992087 7f5c973a57a0 20 read_log 193305'13405 (193305'13404) modify 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by client.951337.0:10224 2014-08-11 18:43:11.094459 2014-08-13 18:06:06.992101 7f5c973a57a0 20 read_log 193305'13406 (0'0) promote b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:112 2014-08-11 18:43:13.287270 2014-08-13 18:06:06.992163 7f5c973a57a0 20 read_log 193305'13407 (193305'13406) modify b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by client.951337.0:10807 2014-08-11 18:43:13.280438 2014-08-13 18:06:06.992179 7f5c973a57a0 20 read_log 193305'13408 (0'0) promote 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by osd.27.0:116 2014-08-11 18:43:13.663828 2014-08-13 18:06:06.992192 7f5c973a57a0 20 read_log 193305'13409 (193305'13408) modify 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.951337.0:10908 2014-08-11 18:43:13.658287 2014-08-13 18:06:06.992205 7f5c973a57a0 20 read_log 193305'13410 (193305'13410) modify 3bf/hit_set_24.3bf_archive_2014-08-11 18:38:15.792949_2014-08-11 18:43:20.626482/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:43:20.626706 2014-08-13 18:06:06.992225 7f5c973a57a0 20 read_log 193305'13411 (0'0) promote 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by osd.27.0:137 2014-08-11 18:43:20.629186 2014-08-13 18:06:06.992245 7f5c973a57a0 20 read_log 193305'13412 (193305'13411) modify 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.951337.0:12731 2014-08-11 18:43:20.620819 2014-08-13 18:06:06.992258 7f5c973a57a0 20 read_log 193305'13413 (0'0) promote 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by osd.27.0:139 2014-08-11 18:43:21.349622 2014-08-13 18:06:06.992272 7f5c973a57a0 20 read_log 193305'13414 (193305'13413) modify 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.951337.0:12922 2014-08-11 18:43:21.342777 2014-08-13 18:06:06.992289 7f5c973a57a0 20 read_log 193305'13415 (0'0) promote 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:149 2014-08-11 18:43:24.512279 2014-08-13 18:06:06.992302 7f5c973a57a0 20 read_log 193305'13416 (193305'13415) modify 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by client.951337.0:13756 2014-08-11 18:43:24.504899 2014-08-13 18:06:06.992314 7f5c973a57a0 20 read_log 193305'13417 (0'0) promote 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by osd.27.0:161 2014-08-11 18:43:29.451095 2014-08-13 18:06:06.992326 7f5c973a57a0 20 read_log 193305'13418 (193305'13417) modify 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.951337.0:15143 2014-08-11 18:43:29.443952 2014-08-13 18:06:06.992338 7f5c973a57a0 20 read_log 193305'13419 (0'0) promote ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by osd.27.0:163 2014-08-11 18:43:29.553355 2014-08-13 18:06:06.992352 7f5c973a57a0 20 read_log 193305'13420 (193305'13419) modify ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.951337.0:15174 2014-08-11 18:43:29.547223 2014-08-13 18:06:06.992376 7f5c973a57a0 20 read_log 193305'13421 (0'0) promote f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by osd.27.0:165 2014-08-11 18:43:30.009412 2014-08-13 18:06:06.992389 7f5c973a57a0 20 read_log 193305'13422 (193305'13421) modify f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.951337.0:15311 2014-08-11 18:43:30.002185 2014-08-13 18:06:06.992415 7f5c973a57a0 20 read_log 193305'13423 (0'0) promote ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by osd.27.0:176 2014-08-11 18:43:33.828064 2014-08-13 18:06:06.992431 7f5c973a57a0 20 read_log 193305'13424 (193305'13423) modify ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.951337.0:16393 2014-08-11 18:43:33.819440 2014-08-13 18:06:06.992443 7f5c973a57a0 20 read_log 193305'13425 (0'0) promote 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:180 2014-08-11 18:43:36.402518 2014-08-13 18:06:06.992466 7f5c973a57a0 20 read_log 193305'13426 (193305'13425) modify 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by client.951337.0:17126 2014-08-11 18:43:36.395320 2014-08-13 18:06:06.992482 7f5c973a57a0 20 read_log 193305'13427 (0'0) promote ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:198 2014-08-11 18:43:43.741580 2014-08-13 18:06:06.992494 7f5c973a57a0 20 read_log 193305'13428 (193305'13427) modify ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by client.951337.0:19205 2014-08-11 18:43:43.733785 2014-08-13 18:06:06.992517 7f5c973a57a0 20 read_log 193305'13429 (0'0) promote 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by osd.27.0:240 2014-08-11 18:43:59.514352 2014-08-13 18:06:06.992533 7f5c973a57a0 20 read_log 193305'13430 (193305'13429) modify 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.951337.0:23579 2014-08-11 18:43:59.506751 2014-08-13 18:06:06.992556 7f5c973a57a0 20 read_log 193305'13431 (0'0) promote 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by osd.27.0:242 2014-08-11 18:43:59.706124 2014-08-13 18:06:06.992571 7f5c973a57a0 20 read_log 193305'13432 (193305'13431) modify 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.951337.0:23635 2014-08-11 18:43:59.699107 2014-08-13 18:06:06.992584 7f5c973a57a0 20 read_log 193305'13433 (0'0) promote 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by osd.27.0:262 2014-08-11 18:44:06.890159 2014-08-13 18:06:06.992596 7f5c973a57a0 20 read_log 193305'13434 (193305'13433) modify 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.951337.0:25605 2014-08-11 18:44:06.882262 2014-08-13 18:06:06.992615 7f5c973a57a0 20 read_log 193305'13435 (0'0) promote b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:270 2014-08-11 18:44:09.366111 2014-08-13 18:06:06.992634 7f5c973a57a0 20 read_log 193305'13436 (193305'13435) modify b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by client.951337.0:26266 2014-08-11 18:44:09.359645 2014-08-13 18:06:06.992652 7f5c973a57a0 20 read_log 193305'13437 (193305'13391) modify d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.951337.0:34798 2014-08-11 18:44:35.195917 2014-08-13 18:06:06.992665 7f5c973a57a0 20 read_log 193305'13438 (193305'13393) modify 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.951337.0:36148 2014-08-11 18:44:35.957183 2014-08-13 18:06:06.992680 7f5c973a57a0 20 read_log 193305'13439 (193305'13395) modify 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.951337.0:36498 2014-08-11 18:44:36.266506 2014-08-13 18:06:06.992702 7f5c973a57a0 20 read_log 193305'13440 (193305'13397) modify a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by client.951337.0:36557 2014-08-11 18:44:36.325854 2014-08-13 18:06:06.992718 7f5c973a57a0 20 read_log 193305'13441 (193305'13399) modify 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by client.951337.0:39776 2014-08-11 18:44:38.181871 2014-08-13 18:06:06.992731 7f5c973a57a0 20 read_log 193305'13442 (193305'13401) modify df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.951337.0:40546 2014-08-11 18:44:38.444339 2014-08-13 18:06:06.992743 7f5c973a57a0 20 read_log 193305'13443 (193305'13403) modify 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.951337.0:41726 2014-08-11 18:44:38.847398 2014-08-13 18:06:06.992761 7f5c973a57a0 20 read_log 193305'13444 (193305'13405) modify 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by client.951337.0:42935 2014-08-11 18:44:39.427803 2014-08-13 18:06:06.992773 7f5c973a57a0 20 read_log 193305'13445 (193305'13407) modify b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by client.951337.0:43518 2014-08-11 18:44:39.615884 2014-08-13 18:06:06.992787 7f5c973a57a0 20 read_log 193305'13446 (193305'13409) modify 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.951337.0:43619 2014-08-11 18:44:39.647523 2014-08-13 18:06:06.992799 7f5c973a57a0 20 read_log 193305'13447 (193305'13412) modify 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.951337.0:45442 2014-08-11 18:44:40.202377 2014-08-13 18:06:06.992819 7f5c973a57a0 20 read_log 193305'13448 (193305'13414) modify 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.951337.0:45633 2014-08-11 18:44:40.257112 2014-08-13 18:06:06.992835 7f5c973a57a0 20 read_log 193305'13449 (193305'13416) modify 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by client.951337.0:46467 2014-08-11 18:44:40.489965 2014-08-13 18:06:06.992848 7f5c973a57a0 20 read_log 193305'13450 (193305'13418) modify 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.951337.0:47854 2014-08-11 18:44:40.884192 2014-08-13 18:06:06.992868 7f5c973a57a0 20 read_log 193305'13451 (193305'13420) modify ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.951337.0:47885 2014-08-11 18:44:40.892586 2014-08-13 18:06:06.992883 7f5c973a57a0 20 read_log 193305'13452 (193305'13422) modify f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.951337.0:48022 2014-08-11 18:44:40.933522 2014-08-13 18:06:06.992895 7f5c973a57a0 20 read_log 193305'13453 (193305'13424) modify ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.951337.0:49104 2014-08-11 18:44:41.255802 2014-08-13 18:06:06.992907 7f5c973a57a0 20 read_log 193305'13454 (193305'13426) modify 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by client.951337.0:49837 2014-08-11 18:44:41.486832 2014-08-13 18:06:06.992919 7f5c973a57a0 20 read_log 193305'13455 (193305'13428) modify ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by client.951337.0:51916 2014-08-11 18:44:42.085579 2014-08-13 18:06:06.992932 7f5c973a57a0 20 read_log 193305'13456 (193305'13430) modify 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.951337.0:56290 2014-08-11 18:44:43.375049 2014-08-13 18:06:06.992944 7f5c973a57a0 20 read_log 193305'13457 (193305'13432) modify 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.951337.0:56346 2014-08-11 18:44:43.392348 2014-08-13 18:06:06.992956 7f5c973a57a0 20 read_log 193305'13458 (193305'13434) modify 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.951337.0:58316 2014-08-11 18:44:44.005193 2014-08-13 18:06:06.992968 7f5c973a57a0 20 read_log 193305'13459 (193305'13436) modify b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by client.951337.0:58977 2014-08-11 18:44:44.199123 2014-08-13 18:06:06.992980 7f5c973a57a0 20 read_log 194209'13460 (192834'12398) clean b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.27.0:771 2014-08-12 14:05:54.246391 2014-08-13 18:06:06.992993 7f5c973a57a0 20 read_log 194209'13461 (192834'12838) clean f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.27.0:772 2014-08-12 14:05:54.246718 2014-08-13 18:06:06.993005 7f5c973a57a0 20 read_log 194209'13462 (192834'12445) clean d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.27.0:773 2014-08-12 14:05:54.247370 2014-08-13 18:06:06.993017 7f5c973a57a0 20 read_log 194209'13463 (192834'12439) clean dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.27.0:774 2014-08-12 14:05:54.247865 2014-08-13 18:06:06.993030 7f5c973a57a0 20 read_log 194209'13464 (192834'12421) clean e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.27.0:775 2014-08-12 14:05:54.295144 2014-08-13 18:06:06.993042 7f5c973a57a0 20 read_log 194209'13465 (192834'12960) clean 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.27.0:776 2014-08-12 14:05:54.298212 2014-08-13 18:06:06.993054 7f5c973a57a0 20 read_log 194209'13466 (192834'12346) clean 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.27.0:777 2014-08-12 14:05:54.301447 2014-08-13 18:06:06.993066 7f5c973a57a0 20 read_log 194209'13467 (192888'13383) clean 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.27.0:778 2014-08-12 14:05:54.304311 2014-08-13 18:06:06.993078 7f5c973a57a0 20 read_log 194209'13468 (192888'13174) clean 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.27.0:779 2014-08-12 14:05:54.310633 2014-08-13 18:06:06.993091 7f5c973a57a0 20 read_log 194209'13469 (192834'12958) clean da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.27.0:780 2014-08-12 14:05:54.310996 2014-08-13 18:06:06.993110 7f5c973a57a0 20 read_log 194209'13470 (193305'13459) clean b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:781 2014-08-12 14:05:54.313938 2014-08-13 18:06:06.993123 7f5c973a57a0 20 read_log 194209'13471 (192888'13355) clean 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.27.0:782 2014-08-12 14:05:54.317455 2014-08-13 18:06:06.993136 7f5c973a57a0 20 read_log 194209'13472 (192834'12910) clean 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.27.0:783 2014-08-12 14:05:54.320441 2014-08-13 18:06:06.993153 7f5c973a57a0 20 read_log 194209'13473 (192834'12438) clean 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.27.0:784 2014-08-12 14:05:54.323820 2014-08-13 18:06:06.993168 7f5c973a57a0 20 read_log 194209'13474 (192834'12461) clean 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.27.0:785 2014-08-12 14:05:54.327003 2014-08-13 18:06:06.993181 7f5c973a57a0 20 read_log 194209'13475 (193305'13444) clean 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:786 2014-08-12 14:05:54.330516 2014-08-13 18:06:06.993194 7f5c973a57a0 20 read_log 194209'13476 (193305'13440) clean a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:787 2014-08-12 14:05:54.333560 2014-08-13 18:06:06.993207 7f5c973a57a0 20 read_log 194209'13477 (192888'13362) clean 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.27.0:788 2014-08-12 14:05:54.336644 2014-08-13 18:06:06.993219 7f5c973a57a0 20 read_log 194209'13478 (192888'13163) clean 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.27.0:789 2014-08-12 14:05:54.339647 2014-08-13 18:06:06.993231 7f5c973a57a0 20 read_log 194209'13479 (192888'13382) clean 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.27.0:790 2014-08-12 14:05:54.342704 2014-08-13 18:06:06.993243 7f5c973a57a0 20 read_log 194209'13480 (192834'13006) clean 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.27.0:791 2014-08-12 14:05:54.345694 2014-08-13 18:06:06.993261 7f5c973a57a0 20 read_log 194209'13481 (193305'13445) clean b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:792 2014-08-12 14:05:54.348740 2014-08-13 18:06:06.993273 7f5c973a57a0 20 read_log 194209'13482 (192888'13345) clean 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.27.0:793 2014-08-12 14:05:54.351817 2014-08-13 18:06:06.993286 7f5c973a57a0 20 read_log 194209'13483 (192888'13186) clean 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.27.0:794 2014-08-12 14:05:54.354813 2014-08-13 18:06:06.993301 7f5c973a57a0 20 read_log 194209'13484 (192834'12942) clean 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.27.0:795 2014-08-12 14:05:54.358239 2014-08-13 18:06:06.993316 7f5c973a57a0 20 read_log 194209'13485 (192834'12931) clean ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.27.0:796 2014-08-12 14:05:54.361292 2014-08-13 18:06:06.993329 7f5c973a57a0 20 read_log 194209'13486 (192834'12856) clean f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by osd.27.0:797 2014-08-12 14:05:54.364396 2014-08-13 18:06:06.993341 7f5c973a57a0 20 read_log 194209'13487 (193305'13441) clean 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:798 2014-08-12 14:05:54.367613 2014-08-13 18:06:06.993357 7f5c973a57a0 20 read_log 194209'13488 (192834'12425) clean cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by osd.27.0:799 2014-08-12 14:05:54.370830 2014-08-13 18:06:06.993369 7f5c973a57a0 20 read_log 194209'13489 (192888'13193) clean 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.27.0:800 2014-08-12 14:05:54.373822 2014-08-13 18:06:06.993422 7f5c973a57a0 20 read_log 194209'13490 (192888'13363) clean 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.27.0:801 2014-08-12 14:05:54.377075 2014-08-13 18:06:06.993448 7f5c973a57a0 20 read_log 194209'13491 (192834'12859) clean f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.27.0:802 2014-08-12 14:05:54.379882 2014-08-13 18:06:06.993461 7f5c973a57a0 20 read_log 194209'13492 (192834'12458) clean 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.27.0:803 2014-08-12 14:05:54.382929 2014-08-13 18:06:06.993474 7f5c973a57a0 20 read_log 194209'13493 (193305'13455) clean ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:804 2014-08-12 14:05:54.386025 2014-08-13 18:06:06.993496 7f5c973a57a0 20 read_log 194209'13494 (192888'13171) clean f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.27.0:805 2014-08-12 14:05:54.389277 2014-08-13 18:06:06.993509 7f5c973a57a0 20 read_log 194209'13495 (193305'13449) clean 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:806 2014-08-12 14:05:54.392409 2014-08-13 18:06:06.993522 7f5c973a57a0 20 read_log 194209'13496 (192888'13333) clean d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.27.0:807 2014-08-12 14:05:54.395462 2014-08-13 18:06:06.993544 7f5c973a57a0 20 read_log 194209'13497 (194209'13460) delete b83a93bf/benchmark_data_storage0111-ib.pocstor_26659_object10624/head//24 by osd.27.0:809 2014-08-12 14:05:54.428558 2014-08-13 18:06:06.993557 7f5c973a57a0 20 read_log 194209'13498 (194209'13461) delete f00c93bf/benchmark_data_storage0111-ib.pocstor_41526_object246/head//24 by osd.27.0:810 2014-08-12 14:05:54.431831 2014-08-13 18:06:06.993570 7f5c973a57a0 20 read_log 194209'13499 (194209'13462) delete d9f0a3bf/benchmark_data_storage0111-ib.pocstor_26659_object18659/head//24 by osd.27.0:811 2014-08-12 14:05:54.434516 2014-08-13 18:06:06.993589 7f5c973a57a0 20 read_log 194209'13500 (194209'13463) delete dcd5a3bf/benchmark_data_storage0111-ib.pocstor_26659_object17467/head//24 by osd.27.0:812 2014-08-12 14:05:54.438225 2014-08-13 18:06:06.993601 7f5c973a57a0 20 read_log 194209'13501 (194209'13464) delete e33fa3bf/benchmark_data_storage0111-ib.pocstor_26659_object13578/head//24 by osd.27.0:813 2014-08-12 14:05:54.441560 2014-08-13 18:06:06.993614 7f5c973a57a0 20 read_log 194209'13502 (194209'13465) delete 14bab3bf/benchmark_data_storage0111-ib.pocstor_41526_object16679/head//24 by osd.27.0:814 2014-08-12 14:05:54.444282 2014-08-13 18:06:06.993626 7f5c973a57a0 20 read_log 194209'13503 (194209'13466) delete 7c0cb3bf/benchmark_data_storage0111-ib.pocstor_26659_object3247/head//24 by osd.27.0:815 2014-08-12 14:05:54.448091 2014-08-13 18:06:06.993638 7f5c973a57a0 20 read_log 194209'13504 (194209'13467) delete 588fb3bf/benchmark_data_storage0111-ib.pocstor_2228_object26410/head//24 by osd.27.0:816 2014-08-12 14:05:54.451410 2014-08-13 18:06:06.993650 7f5c973a57a0 20 read_log 194209'13505 (194209'13468) delete 1866c3bf/benchmark_data_storage0111-ib.pocstor_41097_object10468/head//24 by osd.27.0:817 2014-08-12 14:05:54.454645 2014-08-13 18:06:06.993662 7f5c973a57a0 20 read_log 194209'13506 (193305'13454) clean 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:818 2014-08-12 14:05:54.454880 2014-08-13 18:06:06.993674 7f5c973a57a0 20 read_log 194209'13507 (194209'13469) delete da85d3bf/benchmark_data_storage0111-ib.pocstor_41526_object16587/head//24 by osd.27.0:819 2014-08-12 14:05:54.458455 2014-08-13 18:06:06.993686 7f5c973a57a0 20 read_log 194209'13508 (194209'13470) delete b748e3bf/benchmark_data_storage0111-ib.pocstor_62621_object26265/head//24 by osd.27.0:820 2014-08-12 14:05:54.461127 2014-08-13 18:06:06.993699 7f5c973a57a0 20 read_log 194209'13509 (194209'13471) delete 17c0f3bf/benchmark_data_storage0111-ib.pocstor_2228_object13099/head//24 by osd.27.0:821 2014-08-12 14:05:54.461546 2014-08-13 18:06:06.993744 7f5c973a57a0 20 read_log 194209'13510 (194209'13472) delete 7c2107bf/benchmark_data_storage0111-ib.pocstor_41526_object11276/head//24 by osd.27.0:822 2014-08-12 14:05:54.461907 2014-08-13 18:06:06.993762 7f5c973a57a0 20 read_log 194209'13511 (194209'13473) delete 236c07bf/benchmark_data_storage0111-ib.pocstor_26659_object17366/head//24 by osd.27.0:823 2014-08-12 14:05:54.465263 2014-08-13 18:06:06.993774 7f5c973a57a0 20 read_log 194209'13512 (194209'13474) delete 43bf07bf/benchmark_data_storage0111-ib.pocstor_26659_object20488/head//24 by osd.27.0:824 2014-08-12 14:05:54.465660 2014-08-13 18:06:06.993787 7f5c973a57a0 20 read_log 194209'13513 (194209'13475) delete 952427bf/benchmark_data_storage0111-ib.pocstor_62621_object10223/head//24 by osd.27.0:825 2014-08-12 14:05:54.466510 2014-08-13 18:06:06.993799 7f5c973a57a0 20 read_log 194209'13514 (194209'13476) delete a99427bf/benchmark_data_storage0111-ib.pocstor_62621_object3845/head//24 by osd.27.0:826 2014-08-12 14:05:54.469696 2014-08-13 18:06:06.993811 7f5c973a57a0 20 read_log 194209'13515 (194209'13481) delete b03a57bf/benchmark_data_storage0111-ib.pocstor_62621_object10806/head//24 by osd.27.0:827 2014-08-12 14:05:54.471230 2014-08-13 18:06:06.993826 7f5c973a57a0 20 read_log 194209'13516 (194209'13487) delete 8497bf/benchmark_data_storage0111-ib.pocstor_62621_object7064/head//24 by osd.27.0:828 2014-08-12 14:05:54.475403 2014-08-13 18:06:06.993841 7f5c973a57a0 20 read_log 194209'13517 (194209'13493) delete ab16b7bf/benchmark_data_storage0111-ib.pocstor_62621_object19204/head//24 by osd.27.0:829 2014-08-12 14:05:54.476720 2014-08-13 18:06:06.993854 7f5c973a57a0 20 read_log 194209'13518 (194209'13495) delete 695eb7bf/benchmark_data_storage0111-ib.pocstor_62621_object13755/head//24 by osd.27.0:830 2014-08-12 14:05:54.477343 2014-08-13 18:06:06.993866 7f5c973a57a0 20 read_log 194209'13519 (194209'13506) delete 429c7bf/benchmark_data_storage0111-ib.pocstor_62621_object17125/head//24 by osd.27.0:831 2014-08-12 14:05:54.480370 2014-08-13 18:06:06.993882 7f5c973a57a0 20 read_log 194209'13520 (194209'13489) delete 2d3597bf/benchmark_data_storage0111-ib.pocstor_41097_object21763/head//24 by osd.27.0:842 2014-08-12 14:06:04.569929 2014-08-13 18:06:06.993897 7f5c973a57a0 20 read_log 194209'13521 (194209'13490) delete 6702a7bf/benchmark_data_storage0111-ib.pocstor_2228_object17904/head//24 by osd.27.0:843 2014-08-12 14:06:04.570353 2014-08-13 18:06:06.993910 7f5c973a57a0 20 read_log 194209'13522 (194209'13491) delete f89da7bf/benchmark_data_storage0111-ib.pocstor_41526_object4302/head//24 by osd.27.0:844 2014-08-12 14:06:04.570709 2014-08-13 18:06:06.993955 7f5c973a57a0 20 read_log 194209'13523 (194209'13492) delete 2cb5b7bf/benchmark_data_storage0111-ib.pocstor_26659_object20082/head//24 by osd.27.0:845 2014-08-12 14:06:04.571106 2014-08-13 18:06:06.993975 7f5c973a57a0 20 read_log 194209'13524 (194209'13494) delete f19b7bf/benchmark_data_storage0111-ib.pocstor_41097_object9640/head//24 by osd.27.0:846 2014-08-12 14:06:04.573437 2014-08-13 18:06:06.993989 7f5c973a57a0 20 read_log 194209'13525 (194209'13496) delete d2d3c7bf/benchmark_data_storage0111-ib.pocstor_2228_object903/head//24 by osd.27.0:847 2014-08-12 14:06:04.574209 2014-08-13 18:06:06.994012 7f5c973a57a0 20 read_log 194209'13526 (194209'13477) delete 393927bf/benchmark_data_storage0111-ib.pocstor_2228_object17104/head//24 by osd.27.0:848 2014-08-12 14:06:04.607287 2014-08-13 18:06:06.994025 7f5c973a57a0 20 read_log 194209'13527 (194209'13478) delete 689a27bf/benchmark_data_storage0111-ib.pocstor_41097_object5543/head//24 by osd.27.0:849 2014-08-12 14:06:04.607673 2014-08-13 18:06:06.994037 7f5c973a57a0 20 read_log 194209'13528 (194209'13479) delete 3b4447bf/benchmark_data_storage0111-ib.pocstor_2228_object26187/head//24 by osd.27.0:850 2014-08-12 14:06:04.608046 2014-08-13 18:06:06.994050 7f5c973a57a0 20 read_log 194209'13529 (194209'13480) delete 723f47bf/benchmark_data_storage0111-ib.pocstor_41526_object22722/head//24 by osd.27.0:851 2014-08-12 14:06:04.608441 2014-08-13 18:06:06.994082 7f5c973a57a0 20 read_log 194209'13530 (194209'13482) delete 683967bf/benchmark_data_storage0111-ib.pocstor_2228_object8804/head//24 by osd.27.0:852 2014-08-12 14:06:04.611784 2014-08-13 18:06:06.994098 7f5c973a57a0 20 read_log 194209'13531 (194209'13483) delete 574277bf/benchmark_data_storage0111-ib.pocstor_41097_object17843/head//24 by osd.27.0:853 2014-08-12 14:06:04.612187 2014-08-13 18:06:06.994111 7f5c973a57a0 20 read_log 194209'13532 (194209'13484) delete 646a77bf/benchmark_data_storage0111-ib.pocstor_41526_object15226/head//24 by osd.27.0:854 2014-08-12 14:06:04.612495 2014-08-13 18:06:06.994123 7f5c973a57a0 20 read_log 194209'13533 (194209'13485) delete ed9787bf/benchmark_data_storage0111-ib.pocstor_41526_object13527/head//24 by osd.27.0:855 2014-08-12 14:06:04.613235 2014-08-13 18:06:06.994136 7f5c973a57a0 20 read_log 194215'13534 (194215'13534) modify 3bf/hit_set_24.3bf_archive_2014-08-12 15:32:25.978491_2014-08-12 15:51:03.552127/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:03.552625 2014-08-13 18:06:06.994158 7f5c973a57a0 20 read_log 194215'13535 (193305'13410) delete 3bf/hit_set_24.3bf_archive_2014-08-11 18:38:15.792949_2014-08-11 18:43:20.626482/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:03.552625 2014-08-13 18:06:06.994174 7f5c973a57a0 20 read_log 194215'13536 (192888'13197) clean e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by osd.27.0:1826 2014-08-12 15:51:03.556557 2014-08-13 18:06:06.994186 7f5c973a57a0 20 read_log 194215'13537 (194215'13536) delete e80403bf/benchmark_data_storage0111-ib.pocstor_41097_object24335/head//24 by client.2174595.0:198138 0.000000 2014-08-13 18:06:06.994198 7f5c973a57a0 20 read_log 194215'13538 (193305'13451) clean ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by osd.27.0:1828 2014-08-12 15:51:03.564477 2014-08-13 18:06:06.994210 7f5c973a57a0 20 read_log 194215'13539 (194215'13538) delete ccb623bf/benchmark_data_storage0111-ib.pocstor_62621_object15173/head//24 by client.2174595.0:198140 0.000000 2014-08-13 18:06:06.994221 7f5c973a57a0 20 read_log 194215'13540 (192834'12364) clean 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by osd.27.0:1830 2014-08-12 15:51:03.572221 2014-08-13 18:06:06.994233 7f5c973a57a0 20 read_log 194215'13541 (194215'13540) delete 4fb133bf/benchmark_data_storage0111-ib.pocstor_26659_object6664/head//24 by client.2174595.0:198142 0.000000 2014-08-13 18:06:06.994244 7f5c973a57a0 20 read_log 194215'13542 (193305'13447) clean 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by osd.27.0:1832 2014-08-12 15:51:03.579839 2014-08-13 18:06:06.994256 7f5c973a57a0 20 read_log 194215'13543 (194215'13542) delete 1beb33bf/benchmark_data_storage0111-ib.pocstor_62621_object12730/head//24 by client.2174595.0:198144 0.000000 2014-08-13 18:06:06.994270 7f5c973a57a0 20 read_log 194215'13544 (193305'13446) clean 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by osd.27.0:1834 2014-08-12 15:51:03.587534 2014-08-13 18:06:06.994282 7f5c973a57a0 20 read_log 194215'13545 (194215'13544) delete 43d043bf/benchmark_data_storage0111-ib.pocstor_62621_object10907/head//24 by client.2174595.0:198146 0.000000 2014-08-13 18:06:06.994293 7f5c973a57a0 20 read_log 194215'13546 (192888'13343) clean 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by osd.27.0:1836 2014-08-12 15:51:03.594892 2014-08-13 18:06:06.994306 7f5c973a57a0 20 read_log 194215'13547 (194215'13546) delete 3bd143bf/benchmark_data_storage0111-ib.pocstor_2228_object6700/head//24 by client.2174595.0:198148 0.000000 2014-08-13 18:06:06.994320 7f5c973a57a0 20 read_log 194215'13548 (192888'13188) clean f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by osd.27.0:1838 2014-08-12 15:51:03.602046 2014-08-13 18:06:06.994343 7f5c973a57a0 20 read_log 194215'13549 (194215'13548) delete f9e343bf/benchmark_data_storage0111-ib.pocstor_41097_object18418/head//24 by client.2174595.0:198150 0.000000 2014-08-13 18:06:06.994354 7f5c973a57a0 20 read_log 194215'13550 (192834'12448) clean 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by osd.27.0:1840 2014-08-12 15:51:03.609361 2014-08-13 18:06:06.994373 7f5c973a57a0 20 read_log 194215'13551 (194215'13550) delete 182a43bf/benchmark_data_storage0111-ib.pocstor_26659_object18938/head//24 by client.2174595.0:198152 0.000000 2014-08-13 18:06:06.994394 7f5c973a57a0 20 read_log 194215'13552 (192888'13195) clean 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by osd.27.0:1842 2014-08-12 15:51:03.616578 2014-08-13 18:06:06.994413 7f5c973a57a0 20 read_log 194215'13553 (194215'13552) delete 2e5c43bf/benchmark_data_storage0111-ib.pocstor_41097_object22871/head//24 by client.2174595.0:198154 0.000000 2014-08-13 18:06:06.994424 7f5c973a57a0 20 read_log 194215'13554 (192888'13150) clean 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by osd.27.0:1844 2014-08-12 15:51:03.623710 2014-08-13 18:06:06.994444 7f5c973a57a0 20 read_log 194215'13555 (194215'13554) delete 128c43bf/benchmark_data_storage0111-ib.pocstor_41097_object851/head//24 by client.2174595.0:198156 0.000000 2014-08-13 18:06:06.994458 7f5c973a57a0 20 read_log 194215'13556 (192834'12391) clean d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by osd.27.0:1846 2014-08-12 15:51:03.630814 2014-08-13 18:06:06.994471 7f5c973a57a0 20 read_log 194215'13557 (194215'13556) delete d6a353bf/benchmark_data_storage0111-ib.pocstor_26659_object9568/head//24 by client.2174595.0:198158 0.000000 2014-08-13 18:06:06.994482 7f5c973a57a0 20 read_log 194215'13558 (192888'13189) clean 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by osd.27.0:1848 2014-08-12 15:51:03.638301 2014-08-13 18:06:06.994498 7f5c973a57a0 20 read_log 194215'13559 (194215'13558) delete 5aca53bf/benchmark_data_storage0111-ib.pocstor_41097_object18655/head//24 by client.2174595.0:198160 0.000000 2014-08-13 18:06:06.994517 7f5c973a57a0 20 read_log 194215'13560 (192888'13379) clean 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by osd.27.0:1850 2014-08-12 15:51:03.645409 2014-08-13 18:06:06.994533 7f5c973a57a0 20 read_log 194215'13561 (194215'13560) delete 577163bf/benchmark_data_storage0111-ib.pocstor_2228_object25094/head//24 by client.2174595.0:198162 0.000000 2014-08-13 18:06:06.994548 7f5c973a57a0 20 read_log 194215'13562 (194209'13486) delete f37f87bf/benchmark_data_storage0111-ib.pocstor_41526_object3683/head//24 by client.2174595.0:198164 0.000000 2014-08-13 18:06:06.994559 7f5c973a57a0 20 read_log 194215'13563 (194209'13488) delete cf8497bf/benchmark_data_storage0111-ib.pocstor_26659_object14815/head//24 by client.2174595.0:198166 0.000000 2014-08-13 18:06:06.994570 7f5c973a57a0 20 read_log 194215'13564 (193305'13437) clean d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by osd.27.0:1854 2014-08-12 15:51:03.662111 2014-08-13 18:06:06.994594 7f5c973a57a0 20 read_log 194215'13565 (194215'13564) delete d88cc7bf/benchmark_data_storage0111-ib.pocstor_62621_object2086/head//24 by client.2174595.0:198168 0.000000 2014-08-13 18:06:06.994607 7f5c973a57a0 20 read_log 194215'13566 (192834'12905) clean 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by osd.27.0:1856 2014-08-12 15:51:03.668634 2014-08-13 18:06:06.994619 7f5c973a57a0 20 read_log 194215'13567 (194215'13566) delete 427fc7bf/benchmark_data_storage0111-ib.pocstor_41526_object10584/head//24 by client.2174595.0:198170 0.000000 2014-08-13 18:06:06.994630 7f5c973a57a0 20 read_log 194215'13568 (192834'12982) clean ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by osd.27.0:1858 2014-08-12 15:51:03.675781 2014-08-13 18:06:06.994653 7f5c973a57a0 20 read_log 194215'13569 (194215'13568) delete ad04d7bf/benchmark_data_storage0111-ib.pocstor_41526_object19495/head//24 by client.2174595.0:198172 0.000000 2014-08-13 18:06:06.994667 7f5c973a57a0 20 read_log 194215'13570 (192834'12893) clean d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by osd.27.0:1860 2014-08-12 15:51:03.683061 2014-08-13 18:06:06.994680 7f5c973a57a0 20 read_log 194215'13571 (194215'13570) delete d2bfe7bf/benchmark_data_storage0111-ib.pocstor_41526_object8892/head//24 by client.2174595.0:198174 0.000000 2014-08-13 18:06:06.994694 7f5c973a57a0 20 read_log 194215'13572 (192888'13206) clean adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by osd.27.0:1862 2014-08-12 15:51:03.690399 2014-08-13 18:06:06.994707 7f5c973a57a0 20 read_log 194215'13573 (194215'13572) delete adfe7bf/benchmark_data_storage0111-ib.pocstor_41097_object28818/head//24 by client.2174595.0:198176 0.000000 2014-08-13 18:06:06.994726 7f5c973a57a0 20 read_log 194215'13574 (192888'13337) clean e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by osd.27.0:1864 2014-08-12 15:51:03.697243 2014-08-13 18:06:06.994743 7f5c973a57a0 20 read_log 194215'13575 (194215'13574) delete e2aaf7bf/benchmark_data_storage0111-ib.pocstor_2228_object1689/head//24 by client.2174595.0:198178 0.000000 2014-08-13 18:06:06.994756 7f5c973a57a0 20 read_log 194215'13576 (192888'13388) clean a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by osd.27.0:1866 2014-08-12 15:51:03.704200 2014-08-13 18:06:06.994777 7f5c973a57a0 20 read_log 194215'13577 (194215'13576) delete a3cbf7bf/benchmark_data_storage0111-ib.pocstor_2228_object29693/head//24 by client.2174595.0:198180 0.000000 2014-08-13 18:06:06.994794 7f5c973a57a0 20 read_log 194215'13578 (192888'13351) clean 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by osd.27.0:1868 2014-08-12 15:51:03.711403 2014-08-13 18:06:06.994807 7f5c973a57a0 20 read_log 194215'13579 (194215'13578) delete 7baef7bf/benchmark_data_storage0111-ib.pocstor_2228_object10784/head//24 by client.2174595.0:198182 0.000000 2014-08-13 18:06:06.994818 7f5c973a57a0 20 read_log 194215'13580 (192888'13348) clean ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by osd.27.0:1870 2014-08-12 15:51:03.718289 2014-08-13 18:06:06.994830 7f5c973a57a0 20 read_log 194215'13581 (194215'13580) delete ef450bbf/benchmark_data_storage0111-ib.pocstor_2228_object10074/head//24 by client.2174595.0:198184 0.000000 2014-08-13 18:06:06.994844 7f5c973a57a0 20 read_log 194215'13582 (192888'13370) clean 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by osd.27.0:1872 2014-08-12 15:51:03.725262 2014-08-13 18:06:06.994875 7f5c973a57a0 20 read_log 194215'13583 (194215'13582) delete 38e50bbf/benchmark_data_storage0111-ib.pocstor_2228_object22161/head//24 by client.2174595.0:198186 0.000000 2014-08-13 18:06:06.994902 7f5c973a57a0 20 read_log 194215'13584 (192888'13377) clean 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by osd.27.0:1874 2014-08-12 15:51:03.732332 2014-08-13 18:06:06.994915 7f5c973a57a0 20 read_log 194215'13585 (194215'13584) delete 32960bbf/benchmark_data_storage0111-ib.pocstor_2228_object24880/head//24 by client.2174595.0:198188 0.000000 2014-08-13 18:06:06.994926 7f5c973a57a0 20 read_log 194215'13586 (192834'12869) clean d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by osd.27.0:1876 2014-08-12 15:51:03.739291 2014-08-13 18:06:06.994941 7f5c973a57a0 20 read_log 194215'13587 (194215'13586) delete d78e0bbf/benchmark_data_storage0111-ib.pocstor_41526_object5513/head//24 by client.2174595.0:198190 0.000000 2014-08-13 18:06:06.994953 7f5c973a57a0 20 read_log 194215'13588 (193305'13456) clean 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by osd.27.0:1878 2014-08-12 15:51:03.746374 2014-08-13 18:06:06.994981 7f5c973a57a0 20 read_log 194215'13589 (194215'13588) delete 38d01bbf/benchmark_data_storage0111-ib.pocstor_62621_object23578/head//24 by client.2174595.0:198192 0.000000 2014-08-13 18:06:06.994996 7f5c973a57a0 20 read_log 194215'13590 (192888'13368) clean 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by osd.27.0:1880 2014-08-12 15:51:03.753037 2014-08-13 18:06:06.995009 7f5c973a57a0 20 read_log 194215'13591 (194215'13590) delete 92a11bbf/benchmark_data_storage0111-ib.pocstor_2228_object21853/head//24 by client.2174595.0:198194 0.000000 2014-08-13 18:06:06.995020 7f5c973a57a0 20 read_log 194215'13592 (192888'13332) clean bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by osd.27.0:1882 2014-08-12 15:51:03.759920 2014-08-13 18:06:06.995032 7f5c973a57a0 20 read_log 194215'13593 (194215'13592) delete bb531bbf/benchmark_data_storage0111-ib.pocstor_2228_object522/head//24 by client.2174595.0:198196 0.000000 2014-08-13 18:06:06.995043 7f5c973a57a0 20 read_log 194215'13594 (193305'13439) clean 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by osd.27.0:1884 2014-08-12 15:51:03.766340 2014-08-13 18:06:06.995055 7f5c973a57a0 20 read_log 194215'13595 (194215'13594) delete 9c5d1bbf/benchmark_data_storage0111-ib.pocstor_62621_object3786/head//24 by client.2174595.0:198198 0.000000 2014-08-13 18:06:06.995066 7f5c973a57a0 20 read_log 194215'13596 (192834'12857) clean c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by osd.27.0:1886 2014-08-12 15:51:03.773666 2014-08-13 18:06:06.995078 7f5c973a57a0 20 read_log 194215'13597 (194215'13596) delete c5a02bbf/benchmark_data_storage0111-ib.pocstor_41526_object3720/head//24 by client.2174595.0:198200 0.000000 2014-08-13 18:06:06.995089 7f5c973a57a0 20 read_log 194215'13598 (192834'12396) clean 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by osd.27.0:1888 2014-08-12 15:51:03.780778 2014-08-13 18:06:06.995102 7f5c973a57a0 20 read_log 194215'13599 (194215'13598) delete 1f12bbf/benchmark_data_storage0111-ib.pocstor_26659_object10164/head//24 by client.2174595.0:198202 0.000000 2014-08-13 18:06:06.995112 7f5c973a57a0 20 read_log 194215'13600 (192888'13148) clean 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by osd.27.0:1890 2014-08-12 15:51:03.788052 2014-08-13 18:06:06.995125 7f5c973a57a0 20 read_log 194215'13601 (194215'13600) delete 6e942bbf/benchmark_data_storage0111-ib.pocstor_41097_object778/head//24 by client.2174595.0:198204 0.000000 2014-08-13 18:06:06.995136 7f5c973a57a0 20 read_log 194215'13602 (193305'13450) clean 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by osd.27.0:1892 2014-08-12 15:51:03.795059 2014-08-13 18:06:06.995153 7f5c973a57a0 20 read_log 194215'13603 (194215'13602) delete 45c42bbf/benchmark_data_storage0111-ib.pocstor_62621_object15142/head//24 by client.2174595.0:198206 0.000000 2014-08-13 18:06:06.995165 7f5c973a57a0 20 read_log 194215'13604 (193305'13442) clean df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by osd.27.0:1894 2014-08-12 15:51:03.803358 2014-08-13 18:06:06.995177 7f5c973a57a0 20 read_log 194215'13605 (194215'13604) delete df963bbf/benchmark_data_storage0111-ib.pocstor_62621_object7834/head//24 by client.2174595.0:198208 0.000000 2014-08-13 18:06:06.995188 7f5c973a57a0 20 read_log 194215'13606 (192834'12325) clean e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by osd.27.0:1896 2014-08-12 15:51:03.810112 2014-08-13 18:06:06.995200 7f5c973a57a0 20 read_log 194215'13607 (194215'13606) delete e1573bbf/benchmark_data_storage0111-ib.pocstor_26659_object514/head//24 by client.2174595.0:198210 0.000000 2014-08-13 18:06:06.995211 7f5c973a57a0 20 read_log 194215'13608 (192888'13200) clean 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by osd.27.0:1898 2014-08-12 15:51:03.817288 2014-08-13 18:06:06.995233 7f5c973a57a0 20 read_log 194215'13609 (194215'13608) delete 56cb3bbf/benchmark_data_storage0111-ib.pocstor_41097_object26148/head//24 by client.2174595.0:198212 0.000000 2014-08-13 18:06:06.995245 7f5c973a57a0 20 read_log 194215'13610 (192834'12888) clean 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by osd.27.0:1900 2014-08-12 15:51:03.824884 2014-08-13 18:06:06.995258 7f5c973a57a0 20 read_log 194215'13611 (194215'13610) delete 3cc34bbf/benchmark_data_storage0111-ib.pocstor_41526_object8513/head//24 by client.2174595.0:198214 0.000000 2014-08-13 18:06:06.995269 7f5c973a57a0 20 read_log 194215'13612 (192834'12387) clean 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by osd.27.0:1902 2014-08-12 15:51:03.831667 2014-08-13 18:06:06.995284 7f5c973a57a0 20 read_log 194215'13613 (194215'13612) delete 55725bbf/benchmark_data_storage0111-ib.pocstor_26659_object9135/head//24 by client.2174595.0:198216 0.000000 2014-08-13 18:06:06.995295 7f5c973a57a0 20 read_log 194215'13614 (192834'12416) clean c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by osd.27.0:1904 2014-08-12 15:51:03.838089 2014-08-13 18:06:06.995308 7f5c973a57a0 20 read_log 194215'13615 (194215'13614) delete c1c5bbf/benchmark_data_storage0111-ib.pocstor_26659_object13168/head//24 by client.2174595.0:198218 0.000000 2014-08-13 18:06:06.995318 7f5c973a57a0 20 read_log 194215'13616 (192888'13373) clean a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by osd.27.0:1906 2014-08-12 15:51:03.845357 2014-08-13 18:06:06.995335 7f5c973a57a0 20 read_log 194215'13617 (194215'13616) delete a6836bbf/benchmark_data_storage0111-ib.pocstor_2228_object24033/head//24 by client.2174595.0:198220 0.000000 2014-08-13 18:06:06.995346 7f5c973a57a0 20 read_log 194215'13618 (192888'13152) clean b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by osd.27.0:1908 2014-08-12 15:51:03.852813 2014-08-13 18:06:06.995358 7f5c973a57a0 20 read_log 194215'13619 (194215'13618) delete b2bf6bbf/benchmark_data_storage0111-ib.pocstor_41097_object1063/head//24 by client.2174595.0:198222 0.000000 2014-08-13 18:06:06.995369 7f5c973a57a0 20 read_log 194215'13620 (192888'13184) clean 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by osd.27.0:1910 2014-08-12 15:51:03.860462 2014-08-13 18:06:06.995385 7f5c973a57a0 20 read_log 194215'13621 (194215'13620) delete 5cd07bbf/benchmark_data_storage0111-ib.pocstor_41097_object17474/head//24 by client.2174595.0:198224 0.000000 2014-08-13 18:06:06.995407 7f5c973a57a0 20 read_log 194215'13622 (192834'13003) clean 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by osd.27.0:1912 2014-08-12 15:51:03.867653 2014-08-13 18:06:06.995427 7f5c973a57a0 20 read_log 194215'13623 (194215'13622) delete 2a437bbf/benchmark_data_storage0111-ib.pocstor_41526_object22676/head//24 by client.2174595.0:198226 0.000000 2014-08-13 18:06:06.995438 7f5c973a57a0 20 read_log 194215'13624 (192888'13369) clean 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by osd.27.0:1914 2014-08-12 15:51:03.874841 2014-08-13 18:06:06.995461 7f5c973a57a0 20 read_log 194215'13625 (194215'13624) delete 39649bbf/benchmark_data_storage0111-ib.pocstor_2228_object21977/head//24 by client.2174595.0:198228 0.000000 2014-08-13 18:06:06.995472 7f5c973a57a0 20 read_log 194215'13626 (193305'13448) clean 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by osd.27.0:1916 2014-08-12 15:51:03.882260 2014-08-13 18:06:06.995485 7f5c973a57a0 20 read_log 194215'13627 (194215'13626) delete 6af99bbf/benchmark_data_storage0111-ib.pocstor_62621_object12921/head//24 by client.2174595.0:198230 0.000000 2014-08-13 18:06:06.995504 7f5c973a57a0 20 read_log 194215'13628 (192888'13371) clean 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by osd.27.0:1918 2014-08-12 15:51:03.889134 2014-08-13 18:06:06.995524 7f5c973a57a0 20 read_log 194215'13629 (194215'13628) delete 8482abbf/benchmark_data_storage0111-ib.pocstor_2228_object22908/head//24 by client.2174595.0:198232 0.000000 2014-08-13 18:06:06.995538 7f5c973a57a0 20 read_log 194215'13630 (192888'13154) clean dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by osd.27.0:1920 2014-08-12 15:51:03.895933 2014-08-13 18:06:06.995551 7f5c973a57a0 20 read_log 194215'13631 (194215'13630) delete dca1bbbf/benchmark_data_storage0111-ib.pocstor_41097_object2099/head//24 by client.2174595.0:198234 0.000000 2014-08-13 18:06:06.995575 7f5c973a57a0 20 read_log 194215'13632 (192888'13331) clean 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by osd.27.0:1922 2014-08-12 15:51:03.902609 2014-08-13 18:06:06.995625 7f5c973a57a0 20 read_log 194215'13633 (194215'13632) delete 3905bbbf/benchmark_data_storage0111-ib.pocstor_2228_object250/head//24 by client.2174595.0:198236 0.000000 2014-08-13 18:06:06.995646 7f5c973a57a0 20 read_log 194215'13634 (192834'12326) clean 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by osd.27.0:1924 2014-08-12 15:51:03.909926 2014-08-13 18:06:06.995666 7f5c973a57a0 20 read_log 194215'13635 (194215'13634) delete 6566bbbf/benchmark_data_storage0111-ib.pocstor_26659_object572/head//24 by client.2174595.0:198238 0.000000 2014-08-13 18:06:06.995680 7f5c973a57a0 20 read_log 194215'13636 (193305'13452) clean f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by osd.27.0:1926 2014-08-12 15:51:03.917399 2014-08-13 18:06:06.995693 7f5c973a57a0 20 read_log 194215'13637 (194215'13636) delete f52bbbbf/benchmark_data_storage0111-ib.pocstor_62621_object15310/head//24 by client.2174595.0:198240 0.000000 2014-08-13 18:06:06.995713 7f5c973a57a0 20 read_log 194215'13638 (192888'13204) clean c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by osd.27.0:1928 2014-08-12 15:51:03.924637 2014-08-13 18:06:06.995729 7f5c973a57a0 20 read_log 194215'13639 (194215'13638) delete c8c3cbbf/benchmark_data_storage0111-ib.pocstor_41097_object26995/head//24 by client.2174595.0:198242 0.000000 2014-08-13 18:06:06.995747 7f5c973a57a0 20 read_log 194215'13640 (192834'12993) clean 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by osd.27.0:1930 2014-08-12 15:51:03.932527 2014-08-13 18:06:06.995767 7f5c973a57a0 20 read_log 194215'13641 (194215'13640) delete 1069cbbf/benchmark_data_storage0111-ib.pocstor_41526_object21695/head//24 by client.2174595.0:198244 0.000000 2014-08-13 18:06:06.995795 7f5c973a57a0 20 read_log 194215'13642 (192834'12933) clean 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by osd.27.0:1932 2014-08-12 15:51:03.940050 2014-08-13 18:06:06.995822 7f5c973a57a0 20 read_log 194215'13643 (194215'13642) delete 3c60dbbf/benchmark_data_storage0111-ib.pocstor_41526_object14015/head//24 by client.2174595.0:198246 0.000000 2014-08-13 18:06:06.995840 7f5c973a57a0 20 read_log 194215'13644 (192888'13357) clean fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by osd.27.0:1934 2014-08-12 15:51:03.947391 2014-08-13 18:06:06.995855 7f5c973a57a0 20 read_log 194215'13645 (194215'13644) delete fec0dbbf/benchmark_data_storage0111-ib.pocstor_2228_object13799/head//24 by client.2174595.0:198248 0.000000 2014-08-13 18:06:06.995866 7f5c973a57a0 20 read_log 194215'13646 (192834'12963) clean 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by osd.27.0:1936 2014-08-12 15:51:03.955321 2014-08-13 18:06:06.995884 7f5c973a57a0 20 read_log 194215'13647 (194215'13646) delete 6752dbbf/benchmark_data_storage0111-ib.pocstor_41526_object17260/head//24 by client.2174595.0:198250 0.000000 2014-08-13 18:06:06.995897 7f5c973a57a0 20 read_log 194215'13648 (192888'13157) clean ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by osd.27.0:1938 2014-08-12 15:51:03.962503 2014-08-13 18:06:06.995957 7f5c973a57a0 20 read_log 194215'13649 (194215'13648) delete ecbdebbf/benchmark_data_storage0111-ib.pocstor_41097_object2376/head//24 by client.2174595.0:198252 0.000000 2014-08-13 18:06:06.995988 7f5c973a57a0 20 read_log 194215'13650 (192888'13181) clean 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by osd.27.0:1940 2014-08-12 15:51:03.969296 2014-08-13 18:06:06.996002 7f5c973a57a0 20 read_log 194215'13651 (194215'13650) delete 77efebbf/benchmark_data_storage0111-ib.pocstor_41097_object15365/head//24 by client.2174595.0:198254 0.000000 2014-08-13 18:06:06.996018 7f5c973a57a0 20 read_log 194215'13652 (192888'13167) clean 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by osd.27.0:1942 2014-08-12 15:51:03.976613 2014-08-13 18:06:06.996041 7f5c973a57a0 20 read_log 194215'13653 (194215'13652) delete 2f780fbf/benchmark_data_storage0111-ib.pocstor_41097_object7062/head//24 by client.2174595.0:198256 0.000000 2014-08-13 18:06:06.996056 7f5c973a57a0 20 read_log 194215'13654 (192834'12992) clean fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by osd.27.0:1944 2014-08-12 15:51:03.984186 2014-08-13 18:06:06.996092 7f5c973a57a0 20 read_log 194215'13655 (194215'13654) delete fb511fbf/benchmark_data_storage0111-ib.pocstor_41526_object21126/head//24 by client.2174595.0:198258 0.000000 2014-08-13 18:06:06.996117 7f5c973a57a0 20 read_log 194215'13656 (193305'13438) clean 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by osd.27.0:1946 2014-08-12 15:51:03.991423 2014-08-13 18:06:06.996133 7f5c973a57a0 20 read_log 194215'13657 (194215'13656) delete 1b721fbf/benchmark_data_storage0111-ib.pocstor_62621_object3436/head//24 by client.2174595.0:198260 0.000000 2014-08-13 18:06:06.996144 7f5c973a57a0 20 read_log 194215'13658 (192888'13372) clean cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by osd.27.0:1948 2014-08-12 15:51:03.999509 2014-08-13 18:06:06.996164 7f5c973a57a0 20 read_log 194215'13659 (194215'13658) delete cf8b1fbf/benchmark_data_storage0111-ib.pocstor_2228_object23516/head//24 by client.2174595.0:198262 0.000000 2014-08-13 18:06:06.996183 7f5c973a57a0 20 read_log 194215'13660 (192888'13190) clean 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by osd.27.0:1950 2014-08-12 15:51:04.007296 2014-08-13 18:06:06.996199 7f5c973a57a0 20 read_log 194215'13661 (194215'13660) delete 64ec1fbf/benchmark_data_storage0111-ib.pocstor_41097_object18812/head//24 by client.2174595.0:198264 0.000000 2014-08-13 18:06:06.996210 7f5c973a57a0 20 read_log 194215'13662 (192888'13380) clean e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by osd.27.0:1952 2014-08-12 15:51:04.014366 2014-08-13 18:06:06.996236 7f5c973a57a0 20 read_log 194215'13663 (194215'13662) delete e58f1fbf/benchmark_data_storage0111-ib.pocstor_2228_object25304/head//24 by client.2174595.0:198266 0.000000 2014-08-13 18:06:06.996273 7f5c973a57a0 20 read_log 194215'13664 (192834'12938) clean 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by osd.27.0:1954 2014-08-12 15:51:04.021270 2014-08-13 18:06:06.996293 7f5c973a57a0 20 read_log 194215'13665 (194215'13664) delete 1e092fbf/benchmark_data_storage0111-ib.pocstor_41526_object14824/head//24 by client.2174595.0:198268 0.000000 2014-08-13 18:06:06.996304 7f5c973a57a0 20 read_log 194215'13666 (192834'12903) clean a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by osd.27.0:1956 2014-08-12 15:51:04.028683 2014-08-13 18:06:06.996320 7f5c973a57a0 20 read_log 194215'13667 (194215'13666) delete a5103fbf/benchmark_data_storage0111-ib.pocstor_41526_object10540/head//24 by client.2174595.0:198270 0.000000 2014-08-13 18:06:06.996338 7f5c973a57a0 20 read_log 194215'13668 (193305'13457) clean 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by osd.27.0:1958 2014-08-12 15:51:04.036325 2014-08-13 18:06:06.996361 7f5c973a57a0 20 read_log 194215'13669 (194215'13668) delete 64054fbf/benchmark_data_storage0111-ib.pocstor_62621_object23634/head//24 by client.2174595.0:198272 0.000000 2014-08-13 18:06:06.996375 7f5c973a57a0 20 read_log 194215'13670 (192888'13210) clean f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by osd.27.0:1960 2014-08-12 15:51:04.043476 2014-08-13 18:06:06.996393 7f5c973a57a0 20 read_log 194215'13671 (194215'13670) delete f5045fbf/benchmark_data_storage0111-ib.pocstor_41097_object29512/head//24 by client.2174595.0:198274 0.000000 2014-08-13 18:06:06.996404 7f5c973a57a0 20 read_log 194215'13672 (192834'12360) clean 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by osd.27.0:1962 2014-08-12 15:51:04.050644 2014-08-13 18:06:06.996438 7f5c973a57a0 20 read_log 194215'13673 (194215'13672) delete 38a5fbf/benchmark_data_storage0111-ib.pocstor_26659_object5164/head//24 by client.2174595.0:198276 0.000000 2014-08-13 18:06:06.996467 7f5c973a57a0 20 read_log 194215'13674 (192888'13340) clean bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by osd.27.0:1964 2014-08-12 15:51:04.057338 2014-08-13 18:06:06.996487 7f5c973a57a0 20 read_log 194215'13675 (194215'13674) delete bdf47fbf/benchmark_data_storage0111-ib.pocstor_2228_object3148/head//24 by client.2174595.0:198278 0.000000 2014-08-13 18:06:06.996525 7f5c973a57a0 20 read_log 194215'13676 (192834'12930) clean c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by osd.27.0:1966 2014-08-12 15:51:04.065172 2014-08-13 18:06:06.996541 7f5c973a57a0 20 read_log 194215'13677 (194215'13676) delete c8567fbf/benchmark_data_storage0111-ib.pocstor_41526_object13397/head//24 by client.2174595.0:198280 0.000000 2014-08-13 18:06:06.996553 7f5c973a57a0 20 read_log 194215'13678 (192888'13365) clean 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by osd.27.0:1968 2014-08-12 15:51:04.072076 2014-08-13 18:06:06.996565 7f5c973a57a0 20 read_log 194215'13679 (194215'13678) delete 14c67fbf/benchmark_data_storage0111-ib.pocstor_2228_object21051/head//24 by client.2174595.0:198282 0.000000 2014-08-13 18:06:06.996576 7f5c973a57a0 20 read_log 194215'13680 (192834'13004) clean f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by osd.27.0:1970 2014-08-12 15:51:04.079229 2014-08-13 18:06:06.996589 7f5c973a57a0 20 read_log 194215'13681 (194215'13680) delete f2677fbf/benchmark_data_storage0111-ib.pocstor_41526_object22719/head//24 by client.2174595.0:198284 0.000000 2014-08-13 18:06:06.996604 7f5c973a57a0 20 read_log 194215'13682 (192834'12356) clean e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by osd.27.0:1972 2014-08-12 15:51:04.086664 2014-08-13 18:06:06.996616 7f5c973a57a0 20 read_log 194215'13683 (194215'13682) delete e4f58fbf/benchmark_data_storage0111-ib.pocstor_26659_object4707/head//24 by client.2174595.0:198286 0.000000 2014-08-13 18:06:06.996627 7f5c973a57a0 20 read_log 194215'13684 (193305'13443) clean 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by osd.27.0:1974 2014-08-12 15:51:04.093874 2014-08-13 18:06:06.996640 7f5c973a57a0 20 read_log 194215'13685 (194215'13684) delete 83d68fbf/benchmark_data_storage0111-ib.pocstor_62621_object9014/head//24 by client.2174595.0:198288 0.000000 2014-08-13 18:06:06.996651 7f5c973a57a0 20 read_log 194215'13686 (192834'12974) clean b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by osd.27.0:1976 2014-08-12 15:51:04.101702 2014-08-13 18:06:06.996663 7f5c973a57a0 20 read_log 194215'13687 (194215'13686) delete b4b19fbf/benchmark_data_storage0111-ib.pocstor_41526_object18562/head//24 by client.2174595.0:198290 0.000000 2014-08-13 18:06:06.996674 7f5c973a57a0 20 read_log 194215'13688 (192888'13147) clean 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by osd.27.0:1978 2014-08-12 15:51:04.108349 2014-08-13 18:06:06.996706 7f5c973a57a0 20 read_log 194215'13689 (194215'13688) delete 8dc39fbf/benchmark_data_storage0111-ib.pocstor_41097_object380/head//24 by client.2174595.0:198292 0.000000 2014-08-13 18:06:06.996721 7f5c973a57a0 20 read_log 194215'13690 (192834'12920) clean 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by osd.27.0:1980 2014-08-12 15:51:04.114882 2014-08-13 18:06:06.996734 7f5c973a57a0 20 read_log 194215'13691 (194215'13690) delete 2f949fbf/benchmark_data_storage0111-ib.pocstor_41526_object12224/head//24 by client.2174595.0:198294 0.000000 2014-08-13 18:06:06.996745 7f5c973a57a0 20 read_log 194215'13692 (192888'13162) clean 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by osd.27.0:1982 2014-08-12 15:51:04.121746 2014-08-13 18:06:06.996758 7f5c973a57a0 20 read_log 194215'13693 (194215'13692) delete 8c54afbf/benchmark_data_storage0111-ib.pocstor_41097_object5492/head//24 by client.2174595.0:198296 0.000000 2014-08-13 18:06:06.996769 7f5c973a57a0 20 read_log 194215'13694 (192834'12419) clean 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by osd.27.0:1984 2014-08-12 15:51:04.128895 2014-08-13 18:06:06.996781 7f5c973a57a0 20 read_log 194215'13695 (194215'13694) delete 9b8bafbf/benchmark_data_storage0111-ib.pocstor_26659_object13394/head//24 by client.2174595.0:198298 0.000000 2014-08-13 18:06:06.996792 7f5c973a57a0 20 read_log 194215'13696 (192888'13356) clean e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by osd.27.0:1986 2014-08-12 15:51:04.135952 2014-08-13 18:06:06.996804 7f5c973a57a0 20 read_log 194215'13697 (194215'13696) delete e9aabfbf/benchmark_data_storage0111-ib.pocstor_2228_object13574/head//24 by client.2174595.0:198300 0.000000 2014-08-13 18:06:06.996815 7f5c973a57a0 20 read_log 194215'13698 (192888'13349) clean 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by osd.27.0:1988 2014-08-12 15:51:04.142672 2014-08-13 18:06:06.996828 7f5c973a57a0 20 read_log 194215'13699 (194215'13698) delete 8f80cfbf/benchmark_data_storage0111-ib.pocstor_2228_object10094/head//24 by client.2174595.0:198302 0.000000 2014-08-13 18:06:06.996839 7f5c973a57a0 20 read_log 194215'13700 (192888'13208) clean d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by osd.27.0:1990 2014-08-12 15:51:04.149916 2014-08-13 18:06:06.996851 7f5c973a57a0 20 read_log 194215'13701 (194215'13700) delete d693cfbf/benchmark_data_storage0111-ib.pocstor_41097_object29203/head//24 by client.2174595.0:198304 0.000000 2014-08-13 18:06:06.996862 7f5c973a57a0 20 read_log 194215'13702 (192834'12347) clean acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by osd.27.0:1992 2014-08-12 15:51:04.157011 2014-08-13 18:06:06.996874 7f5c973a57a0 20 read_log 194215'13703 (194215'13702) delete acf2dfbf/benchmark_data_storage0111-ib.pocstor_26659_object3404/head//24 by client.2174595.0:198306 0.000000 2014-08-13 18:06:06.996885 7f5c973a57a0 20 read_log 194215'13704 (192888'13346) clean c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by osd.27.0:1994 2014-08-12 15:51:04.164787 2014-08-13 18:06:06.996898 7f5c973a57a0 20 read_log 194215'13705 (194215'13704) delete c255dfbf/benchmark_data_storage0111-ib.pocstor_2228_object9432/head//24 by client.2174595.0:198308 0.000000 2014-08-13 18:06:06.996909 7f5c973a57a0 20 read_log 194215'13706 (192888'13199) clean 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by osd.27.0:1996 2014-08-12 15:51:04.171223 2014-08-13 18:06:06.996921 7f5c973a57a0 20 read_log 194215'13707 (194215'13706) delete 244bdfbf/benchmark_data_storage0111-ib.pocstor_41097_object25184/head//24 by client.2174595.0:198310 0.000000 2014-08-13 18:06:06.996932 7f5c973a57a0 20 read_log 194215'13708 (192834'12400) clean 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by osd.27.0:1998 2014-08-12 15:51:04.178011 2014-08-13 18:06:06.996951 7f5c973a57a0 20 read_log 194215'13709 (194215'13708) delete 62b2efbf/benchmark_data_storage0111-ib.pocstor_26659_object10808/head//24 by client.2174595.0:198312 0.000000 2014-08-13 18:06:06.996962 7f5c973a57a0 20 read_log 194215'13710 (192888'13361) clean 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by osd.27.0:2000 2014-08-12 15:51:04.185073 2014-08-13 18:06:06.996977 7f5c973a57a0 20 read_log 194215'13711 (194215'13710) delete 615efbf/benchmark_data_storage0111-ib.pocstor_2228_object17018/head//24 by client.2174595.0:198314 0.000000 2014-08-13 18:06:06.996989 7f5c973a57a0 20 read_log 194215'13712 (192834'12450) clean f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by osd.27.0:2002 2014-08-12 15:51:04.191644 2014-08-13 18:06:06.997001 7f5c973a57a0 20 read_log 194215'13713 (194215'13712) delete f065efbf/benchmark_data_storage0111-ib.pocstor_26659_object19171/head//24 by client.2174595.0:198316 0.000000 2014-08-13 18:06:06.997014 7f5c973a57a0 20 read_log 194215'13714 (193305'13458) clean 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by osd.27.0:2004 2014-08-12 15:51:04.198377 2014-08-13 18:06:06.997026 7f5c973a57a0 20 read_log 194215'13715 (194215'13714) delete 37a6efbf/benchmark_data_storage0111-ib.pocstor_62621_object25604/head//24 by client.2174595.0:198318 0.000000 2014-08-13 18:06:06.997040 7f5c973a57a0 20 read_log 194215'13716 (192888'13178) clean 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by osd.27.0:2006 2014-08-12 15:51:04.206475 2014-08-13 18:06:06.997055 7f5c973a57a0 20 read_log 194215'13717 (194215'13716) delete 1ef9efbf/benchmark_data_storage0111-ib.pocstor_41097_object12005/head//24 by client.2174595.0:198320 0.000000 2014-08-13 18:06:06.997066 7f5c973a57a0 20 read_log 194215'13718 (192888'13389) clean afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by osd.27.0:2008 2014-08-12 15:51:04.213946 2014-08-13 18:06:06.997079 7f5c973a57a0 20 read_log 194215'13719 (194215'13718) delete afdcefbf/benchmark_data_storage0111-ib.pocstor_2228_object31208/head//24 by client.2174595.0:198322 0.000000 2014-08-13 18:06:06.997092 7f5c973a57a0 20 read_log 194215'13720 (192888'13160) clean 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by osd.27.0:2010 2014-08-12 15:51:04.221130 2014-08-13 18:06:06.997104 7f5c973a57a0 20 read_log 194215'13721 (194215'13720) delete 256defbf/benchmark_data_storage0111-ib.pocstor_41097_object5029/head//24 by client.2174595.0:198324 0.000000 2014-08-13 18:06:06.997117 7f5c973a57a0 20 read_log 194215'13722 (193305'13453) clean ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by osd.27.0:2012 2014-08-12 15:51:04.227484 2014-08-13 18:06:06.997131 7f5c973a57a0 20 read_log 194215'13723 (194215'13722) delete ec80ffbf/benchmark_data_storage0111-ib.pocstor_62621_object16392/head//24 by client.2174595.0:198326 0.000000 2014-08-13 18:06:06.997142 7f5c973a57a0 20 read_log 194215'13724 (194215'13724) modify 3bf/hit_set_24.3bf_archive_2014-08-12 15:51:03.552629_2014-08-12 17:24:41.785138/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.785371 2014-08-13 18:06:06.997163 7f5c973a57a0 20 read_log 194215'13725 (194215'13534) delete 3bf/hit_set_24.3bf_archive_2014-08-12 15:32:25.978491_2014-08-12 15:51:03.552127/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.785371 2014-08-13 18:06:06.997178 7f5c973a57a0 20 read_log 194215'13726 (0'0) promote e56603bf/benchmark_data_storage0111-ib.pocstor_34628_object485/head//24 by osd.27.0:2353 2014-08-12 17:24:41.788558 2014-08-13 18:06:06.997192 7f5c973a57a0 20 read_log 194215'13727 (194215'13726) modify e56603bf/benchmark_data_storage0111-ib.pocstor_34628_object485/head//24 by client.2193759.0:486 2014-08-12 17:24:41.780036 2014-08-13 18:06:06.997212 7f5c973a57a0 20 read_log 194370'13728 (0'0) promote ffb0abbf/benchmark_data_storage0111-ib.pocstor_12495_object1602/head//24 by osd.19.0:4650 2014-08-12 20:30:20.388956 2014-08-13 18:06:06.997227 7f5c973a57a0 20 read_log 194370'13729 (194370'13728) modify ffb0abbf/benchmark_data_storage0111-ib.pocstor_12495_object1602/head//24 by client.1508531.0:1603 2014-08-12 20:30:20.382979 2014-08-13 18:06:06.997243 7f5c973a57a0 20 read_log 194370'13730 (0'0) promote 2d4c3fbf/benchmark_data_storage0111-ib.pocstor_12495_object1903/head//24 by osd.19.0:4658 2014-08-12 20:30:21.481769 2014-08-13 18:06:06.997257 7f5c973a57a0 20 read_log 194370'13731 (194370'13730) modify 2d4c3fbf/benchmark_data_storage0111-ib.pocstor_12495_object1903/head//24 by client.1508531.0:1904 2014-08-12 20:30:21.475144 2014-08-13 18:06:06.997270 7f5c973a57a0 20 read_log 194370'13732 (0'0) promote e1c40fbf/benchmark_data_storage0111-ib.pocstor_12495_object2723/head//24 by osd.19.0:4664 2014-08-12 20:30:24.540766 2014-08-13 18:06:06.997282 7f5c973a57a0 20 read_log 194370'13733 (194370'13732) modify e1c40fbf/benchmark_data_storage0111-ib.pocstor_12495_object2723/head//24 by client.1508531.0:2724 2014-08-12 20:30:24.534664 2014-08-13 18:06:06.997297 7f5c973a57a0 20 read_log 194370'13734 (0'0) promote 7028d3bf/benchmark_data_storage0111-ib.pocstor_12495_object3559/head//24 by osd.19.0:4672 2014-08-12 20:30:27.633709 2014-08-13 18:06:06.997310 7f5c973a57a0 20 read_log 194370'13735 (194370'13734) modify 7028d3bf/benchmark_data_storage0111-ib.pocstor_12495_object3559/head//24 by client.1508531.0:3560 2014-08-12 20:30:27.625198 2014-08-13 18:06:06.997322 7f5c973a57a0 20 read_log 194370'13736 (0'0) promote 40de5fbf/benchmark_data_storage0111-ib.pocstor_12495_object5663/head//24 by osd.19.0:4694 2014-08-12 20:30:36.178414 2014-08-13 18:06:06.997340 7f5c973a57a0 20 read_log 194370'13737 (194370'13736) modify 40de5fbf/benchmark_data_storage0111-ib.pocstor_12495_object5663/head//24 by client.1508531.0:5664 2014-08-12 20:30:36.170257 2014-08-13 18:06:06.997355 7f5c973a57a0 20 read_log 194370'13738 (0'0) promote fd417bbf/benchmark_data_storage0111-ib.pocstor_12495_object6099/head//24 by osd.19.0:4700 2014-08-12 20:30:38.202367 2014-08-13 18:06:06.997367 7f5c973a57a0 20 read_log 194370'13739 (194370'13738) modify fd417bbf/benchmark_data_storage0111-ib.pocstor_12495_object6099/head//24 by client.1508531.0:6100 2014-08-12 20:30:38.196246 2014-08-13 18:06:06.997394 7f5c973a57a0 20 read_log 194370'13740 (0'0) promote 700c0fbf/benchmark_data_storage0111-ib.pocstor_12495_object6411/head//24 by osd.19.0:4706 2014-08-12 20:30:39.534530 2014-08-13 18:06:06.997410 7f5c973a57a0 20 read_log 194370'13741 (194370'13740) modify 700c0fbf/benchmark_data_storage0111-ib.pocstor_12495_object6411/head//24 by client.1508531.0:6412 2014-08-12 20:30:39.527742 2014-08-13 18:06:06.997422 7f5c973a57a0 20 read_log 194370'13742 (0'0) promote b2e17bbf/benchmark_data_storage0111-ib.pocstor_12495_object7303/head//24 by osd.19.0:4724 2014-08-12 20:30:43.915880 2014-08-13 18:06:06.997445 7f5c973a57a0 20 read_log 194370'13743 (194370'13742) modify b2e17bbf/benchmark_data_storage0111-ib.pocstor_12495_object7303/head//24 by client.1508531.0:7304 2014-08-12 20:30:43.904828 2014-08-13 18:06:06.997458 7f5c973a57a0 20 read_log 194370'13744 (0'0) promote cf8933bf/benchmark_data_storage0111-ib.pocstor_12495_object8707/head//24 by osd.19.0:4756 2014-08-12 20:30:51.499238 2014-08-13 18:06:06.997470 7f5c973a57a0 20 read_log 194370'13745 (194370'13744) modify cf8933bf/benchmark_data_storage0111-ib.pocstor_12495_object8707/head//24 by client.1508531.0:8708 2014-08-12 20:30:51.493886 2014-08-13 18:06:06.997495 7f5c973a57a0 20 read_log 194370'13746 (0'0) promote 865c7bf/benchmark_data_storage0111-ib.pocstor_12495_object9836/head//24 by osd.19.0:4770 2014-08-12 20:30:58.331667 2014-08-13 18:06:06.997508 7f5c973a57a0 20 read_log 194370'13747 (194370'13746) modify 865c7bf/benchmark_data_storage0111-ib.pocstor_12495_object9836/head//24 by client.1508531.0:9837 2014-08-12 20:30:58.324036 2014-08-13 18:06:06.997541 7f5c973a57a0 20 read_log 194370'13748 (0'0) promote 7a1333bf/benchmark_data_storage0111-ib.pocstor_12495_object11725/head//24 by osd.19.0:4786 2014-08-12 20:31:09.674196 2014-08-13 18:06:06.997558 7f5c973a57a0 20 read_log 194370'13749 (194370'13748) modify 7a1333bf/benchmark_data_storage0111-ib.pocstor_12495_object11725/head//24 by client.1508531.0:11726 2014-08-12 20:31:09.667264 2014-08-13 18:06:06.997578 7f5c973a57a0 20 read_log 194370'13750 (0'0) promote d757d3bf/benchmark_data_storage0111-ib.pocstor_12495_object13273/head//24 by osd.19.0:4798 2014-08-12 20:31:22.004315 2014-08-13 18:06:06.997593 7f5c973a57a0 20 read_log 194370'13751 (194370'13750) modify d757d3bf/benchmark_data_storage0111-ib.pocstor_12495_object13273/head//24 by client.1508531.0:13274 2014-08-12 20:31:21.997243 2014-08-13 18:06:06.997606 7f5c973a57a0 20 read_log 194377'13752 (0'0) promote 79755fbf/benchmark_data_storage0111-ib.pocstor_25065_object2615/head//24 by osd.19.0:4838 2014-08-12 20:33:07.804959 2014-08-13 18:06:06.997618 7f5c973a57a0 20 read_log 194377'13753 (194377'13752) modify 79755fbf/benchmark_data_storage0111-ib.pocstor_25065_object2615/head//24 by client.2231190.0:2616 2014-08-12 20:33:07.795754 2014-08-13 18:06:06.997630 7f5c973a57a0 20 read_log 194377'13754 (0'0) promote b7d97fbf/benchmark_data_storage0111-ib.pocstor_25065_object3948/head//24 by osd.19.0:4850 2014-08-12 20:33:13.628830 2014-08-13 18:06:06.997643 7f5c973a57a0 20 read_log 194377'13755 (194377'13754) modify b7d97fbf/benchmark_data_storage0111-ib.pocstor_25065_object3948/head//24 by client.2231190.0:3949 2014-08-12 20:33:13.618902 2014-08-13 18:06:06.997655 7f5c973a57a0 20 read_log 194380'13756 (0'0) promote c538cbbf/benchmark_data_storage0111-ib.pocstor_25065_object5297/head//24 by osd.19.0:4868 2014-08-12 20:33:27.113369 2014-08-13 18:06:06.997667 7f5c973a57a0 20 read_log 194380'13757 (194380'13756) modify c538cbbf/benchmark_data_storage0111-ib.pocstor_25065_object5297/head//24 by client.2231190.0:5298 2014-08-12 20:33:27.106661 2014-08-13 18:06:06.997680 7f5c973a57a0 20 read_log 194380'13758 (0'0) promote c07903bf/benchmark_data_storage0111-ib.pocstor_25065_object5425/head//24 by osd.19.0:4874 2014-08-12 20:33:28.891875 2014-08-13 18:06:06.997692 7f5c973a57a0 20 read_log 194380'13759 (194380'13758) modify c07903bf/benchmark_data_storage0111-ib.pocstor_25065_object5425/head//24 by client.2231190.0:5426 2014-08-12 20:33:28.884787 2014-08-13 18:06:06.997704 7f5c973a57a0 20 read_log 194383'13760 (0'0) promote 41f87fbf/benchmark_data_storage0111-ib.pocstor_25065_object5885/head//24 by osd.19.0:4888 2014-08-12 20:34:00.870766 2014-08-13 18:06:06.997716 7f5c973a57a0 20 read_log 194383'13761 (194383'13760) modify 41f87fbf/benchmark_data_storage0111-ib.pocstor_25065_object5885/head//24 by client.2231190.0:5886 2014-08-12 20:34:00.862447 2014-08-13 18:06:06.997728 7f5c973a57a0 20 read_log 194383'13762 (0'0) promote a6a70bbf/benchmark_data_storage0111-ib.pocstor_25065_object5950/head//24 by osd.19.0:4890 2014-08-12 20:34:01.167736 2014-08-13 18:06:06.997743 7f5c973a57a0 20 read_log 194383'13763 (194383'13762) modify a6a70bbf/benchmark_data_storage0111-ib.pocstor_25065_object5950/head//24 by client.2231190.0:5951 2014-08-12 20:34:01.157856 2014-08-13 18:06:06.997756 7f5c973a57a0 20 read_log 194393'13764 (0'0) promote f990f3bf/benchmark_data_storage0111-ib.pocstor_48304_object150/head//24 by osd.19.0:4943 2014-08-12 20:52:33.125398 2014-08-13 18:06:06.997768 7f5c973a57a0 20 read_log 194393'13765 (194393'13764) modify f990f3bf/benchmark_data_storage0111-ib.pocstor_48304_object150/head//24 by client.1064422.0:151 2014-08-12 20:52:33.115711 2014-08-13 18:06:06.997789 7f5c973a57a0 20 read_log 194416'13766 (0'0) promote 801dfbbf/benchmark_data_storage0111-ib.pocstor_19103_object538/head//24 by osd.19.0:5091 2014-08-13 15:28:32.618697 2014-08-13 18:06:06.997813 7f5c973a57a0 20 read_log 194416'13767 (194416'13766) modify 801dfbbf/benchmark_data_storage0111-ib.pocstor_19103_object538/head//24 by client.2441958.0:539 2014-08-13 15:28:32.610887 2014-08-13 18:06:06.997828 7f5c973a57a0 20 read_log 194416'13768 (0'0) promote 44ad77bf/benchmark_data_storage0111-ib.pocstor_19103_object976/head//24 by osd.19.0:5102 2014-08-13 15:28:34.424112 2014-08-13 18:06:06.997859 7f5c973a57a0 20 read_log 194416'13769 (194416'13768) modify 44ad77bf/benchmark_data_storage0111-ib.pocstor_19103_object976/head//24 by client.2441958.0:977 2014-08-13 15:28:34.417346 2014-08-13 18:06:06.997872 7f5c973a57a0 20 read_log 194416'13770 (0'0) promote 648653bf/benchmark_data_storage0111-ib.pocstor_19103_object2959/head//24 by osd.19.0:5133 2014-08-13 15:28:41.923799 2014-08-13 18:06:06.997884 7f5c973a57a0 20 read_log 194416'13771 (194416'13770) modify 648653bf/benchmark_data_storage0111-ib.pocstor_19103_object2959/head//24 by client.2441958.0:2960 2014-08-13 15:28:41.917209 2014-08-13 18:06:06.997909 7f5c973a57a0 20 read_log 194416'13772 (0'0) promote 1736f7bf/benchmark_data_storage0111-ib.pocstor_19103_object4678/head//24 by osd.19.0:5157 2014-08-13 15:28:48.677571 2014-08-13 18:06:06.997922 7f5c973a57a0 20 read_log 194416'13773 (194416'13772) modify 1736f7bf/benchmark_data_storage0111-ib.pocstor_19103_object4678/head//24 by client.2441958.0:4679 2014-08-13 15:28:48.670385 2014-08-13 18:06:06.997934 7f5c973a57a0 20 read_log 194416'13774 (0'0) promote 127e77bf/benchmark_data_storage0111-ib.pocstor_19103_object6287/head//24 by osd.19.0:5179 2014-08-13 15:28:55.091692 2014-08-13 18:06:06.997947 7f5c973a57a0 20 read_log 194416'13775 (194416'13774) modify 127e77bf/benchmark_data_storage0111-ib.pocstor_19103_object6287/head//24 by client.2441958.0:6288 2014-08-13 15:28:55.083137 2014-08-13 18:06:06.997959 7f5c973a57a0 20 read_log 194416'13776 (0'0) promote 9e8ca3bf/benchmark_data_storage0111-ib.pocstor_19103_object6377/head//24 by osd.19.0:5183 2014-08-13 15:28:55.417795 2014-08-13 18:06:06.997977 7f5c973a57a0 20 read_log 194416'13777 (194416'13776) modify 9e8ca3bf/benchmark_data_storage0111-ib.pocstor_19103_object6377/head//24 by client.2441958.0:6378 2014-08-13 15:28:55.410346 2014-08-13 18:06:06.997989 7f5c973a57a0 20 read_log 194416'13778 (0'0) promote 95aa8fbf/benchmark_data_storage0111-ib.pocstor_19103_object7273/head//24 by osd.19.0:5191 2014-08-13 15:28:58.796500 2014-08-13 18:06:06.998001 7f5c973a57a0 20 read_log 194416'13779 (194416'13778) modify 95aa8fbf/benchmark_data_storage0111-ib.pocstor_19103_object7273/head//24 by client.2441958.0:7274 2014-08-13 15:28:58.787603 2014-08-13 18:06:06.998018 7f5c973a57a0 20 read_log 194416'13780 (0'0) promote 411383bf/benchmark_data_storage0111-ib.pocstor_19103_object7620/head//24 by osd.19.0:5197 2014-08-13 15:29:00.105735 2014-08-13 18:06:06.998033 7f5c973a57a0 20 read_log 194416'13781 (194416'13780) modify 411383bf/benchmark_data_storage0111-ib.pocstor_19103_object7620/head//24 by client.2441958.0:7621 2014-08-13 15:29:00.099008 2014-08-13 18:06:06.998046 7f5c973a57a0 20 read_log 194416'13782 (0'0) promote 78bf7fbf/benchmark_data_storage0111-ib.pocstor_19103_object12517/head//24 by osd.19.0:5251 2014-08-13 15:29:19.766645 2014-08-13 18:06:06.998058 7f5c973a57a0 20 read_log 194416'13783 (194416'13782) modify 78bf7fbf/benchmark_data_storage0111-ib.pocstor_19103_object12517/head//24 by client.2441958.0:12518 2014-08-13 15:29:19.759662 2014-08-13 18:06:06.998076 7f5c973a57a0 20 read_log 194416'13784 (0'0) promote 4142d7bf/benchmark_data_storage0111-ib.pocstor_19103_object14735/head//24 by osd.19.0:5285 2014-08-13 15:29:29.143426 2014-08-13 18:06:06.998100 7f5c973a57a0 20 read_log 194416'13785 (194416'13784) modify 4142d7bf/benchmark_data_storage0111-ib.pocstor_19103_object14735/head//24 by client.2441958.0:14736 2014-08-13 15:29:29.134575 2014-08-13 18:06:06.998116 7f5c973a57a0 20 read_log 194416'13786 (0'0) promote 9f1f5fbf/benchmark_data_storage0111-ib.pocstor_19103_object15019/head//24 by osd.19.0:5291 2014-08-13 15:29:30.383809 2014-08-13 18:06:06.998176 7f5c973a57a0 20 read_log 194416'13787 (194416'13786) modify 9f1f5fbf/benchmark_data_storage0111-ib.pocstor_19103_object15019/head//24 by client.2441958.0:15020 2014-08-13 15:29:30.376513 2014-08-13 18:06:06.998205 7f5c973a57a0 20 read_log 194416'13788 (0'0) promote 5e660fbf/benchmark_data_storage0111-ib.pocstor_19103_object16368/head//24 by osd.19.0:5307 2014-08-13 15:29:38.391449 2014-08-13 18:06:06.998218 7f5c973a57a0 20 read_log 194416'13789 (194416'13788) modify 5e660fbf/benchmark_data_storage0111-ib.pocstor_19103_object16368/head//24 by client.2441958.0:16369 2014-08-13 15:29:38.382308 2014-08-13 18:06:06.998237 7f5c973a57a0 20 read_log 194416'13790 (0'0) promote 739373bf/benchmark_data_storage0111-ib.pocstor_19103_object16613/head//24 by osd.19.0:5311 2014-08-13 15:29:41.844800 2014-08-13 18:06:06.998250 7f5c973a57a0 20 read_log 194416'13791 (194416'13790) modify 739373bf/benchmark_data_storage0111-ib.pocstor_19103_object16613/head//24 by client.2441958.0:16614 2014-08-13 15:29:41.837470 2014-08-13 18:06:06.998265 7f5c973a57a0 20 read_log 194416'13792 (0'0) promote 171767bf/benchmark_data_storage0111-ib.pocstor_19103_object17318/head//24 by osd.19.0:5325 2014-08-13 15:29:51.891613 2014-08-13 18:06:06.998285 7f5c973a57a0 20 read_log 194416'13793 (194416'13792) modify 171767bf/benchmark_data_storage0111-ib.pocstor_19103_object17318/head//24 by client.2441958.0:17319 2014-08-13 15:29:51.882464 2014-08-13 18:06:06.998298 7f5c973a57a0 20 read_log 194416'13794 (0'0) promote c22513bf/benchmark_data_storage0111-ib.pocstor_19103_object18618/head//24 by osd.19.0:5337 2014-08-13 15:30:16.388470 2014-08-13 18:06:06.998310 7f5c973a57a0 20 read_log 194416'13795 (194416'13794) modify c22513bf/benchmark_data_storage0111-ib.pocstor_19103_object18618/head//24 by client.2441958.0:18619 2014-08-13 15:30:16.380078 2014-08-13 18:06:06.998344 7f5c973a57a0 20 read_log 194422'13796 (0'0) promote 946ec7bf/benchmark_data_storage0111-ib.pocstor_41231_object998/head//24 by osd.19.0:5357 2014-08-13 15:32:19.294942 2014-08-13 18:06:06.998360 7f5c973a57a0 20 read_log 194422'13797 (194422'13796) modify 946ec7bf/benchmark_data_storage0111-ib.pocstor_41231_object998/head//24 by client.1148203.0:999 2014-08-13 15:32:19.287995 2014-08-13 18:06:06.998373 7f5c973a57a0 20 read_log 194422'13798 (0'0) promote a36a9bbf/benchmark_data_storage0111-ib.pocstor_41231_object3736/head//24 by osd.19.0:5383 2014-08-13 15:32:30.005484 2014-08-13 18:06:06.998394 7f5c973a57a0 20 read_log 194422'13799 (194422'13798) modify a36a9bbf/benchmark_data_storage0111-ib.pocstor_41231_object3736/head//24 by client.1148203.0:3737 2014-08-13 15:32:29.998002 2014-08-13 18:06:06.998410 7f5c973a57a0 20 read_log 194422'13800 (0'0) promote 2d6a1fbf/benchmark_data_storage0111-ib.pocstor_41231_object4583/head//24 by osd.19.0:5391 2014-08-13 15:32:33.243740 2014-08-13 18:06:06.998423 7f5c973a57a0 20 read_log 194422'13801 (194422'13800) modify 2d6a1fbf/benchmark_data_storage0111-ib.pocstor_41231_object4583/head//24 by client.1148203.0:4584 2014-08-13 15:32:33.234264 2014-08-13 18:06:06.998435 7f5c973a57a0 20 read_log 194422'13802 (0'0) promote 98e2dfbf/benchmark_data_storage0111-ib.pocstor_41231_object6292/head//24 by osd.19.0:5413 2014-08-13 15:32:41.752848 2014-08-13 18:06:06.998448 7f5c973a57a0 20 read_log 194422'13803 (194422'13802) modify 98e2dfbf/benchmark_data_storage0111-ib.pocstor_41231_object6292/head//24 by client.1148203.0:6293 2014-08-13 15:32:41.744371 2014-08-13 18:06:06.998460 7f5c973a57a0 20 read_log 194422'13804 (0'0) promote b4ee43bf/benchmark_data_storage0111-ib.pocstor_41231_object7436/head//24 by osd.19.0:5431 2014-08-13 15:32:48.222667 2014-08-13 18:06:06.998473 7f5c973a57a0 20 read_log 194422'13805 (194422'13804) modify b4ee43bf/benchmark_data_storage0111-ib.pocstor_41231_object7436/head//24 by client.1148203.0:7437 2014-08-13 15:32:48.214318 2014-08-13 18:06:06.998485 7f5c973a57a0 20 read_log 194422'13806 (0'0) promote fdc83fbf/benchmark_data_storage0111-ib.pocstor_41231_object8127/head//24 by osd.19.0:5439 2014-08-13 15:32:52.530922 2014-08-13 18:06:06.998499 7f5c973a57a0 20 read_log 194422'13807 (194422'13806) modify fdc83fbf/benchmark_data_storage0111-ib.pocstor_41231_object8127/head//24 by client.1148203.0:8128 2014-08-13 15:32:52.521785 2014-08-13 18:06:06.998512 7f5c973a57a0 20 read_log 194422'13808 (0'0) promote 9c81cfbf/benchmark_data_storage0111-ib.pocstor_41231_object10089/head//24 by osd.19.0:5469 2014-08-13 15:33:12.596365 2014-08-13 18:06:06.998524 7f5c973a57a0 20 read_log 194422'13809 (194422'13808) modify 9c81cfbf/benchmark_data_storage0111-ib.pocstor_41231_object10089/head//24 by client.1148203.0:10090 2014-08-13 15:33:12.589305 2014-08-13 18:06:06.998537 7f5c973a57a0 20 read_log 194422'13810 (0'0) promote ce8a4fbf/benchmark_data_storage0111-ib.pocstor_41231_object10098/head//24 by osd.19.0:5471 2014-08-13 15:33:12.841098 2014-08-13 18:06:06.998549 7f5c973a57a0 20 read_log 194422'13811 (194422'13810) modify ce8a4fbf/benchmark_data_storage0111-ib.pocstor_41231_object10098/head//24 by client.1148203.0:10099 2014-08-13 15:33:12.831737 2014-08-13 18:06:06.998566 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:06.998953 7f5c973a57a0 10 read_log done 2014-08-13 18:06:06.998988 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive] handle_loaded 2014-08-13 18:06:06.999007 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Initial 0.012370 0 0.000000 2014-08-13 18:06:06.999020 7f5c973a57a0 5 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:06.999029 7f5c973a57a0 20 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=0 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194486 2014-08-13 18:06:06.999039 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] log((192834'12014,194422'13811], crt=0'0) 2014-08-13 18:06:06.999050 7f5c973a57a0 10 osd.4 194487 pgid 24.3f7 coll 24.3f7_head 2014-08-13 18:06:06.999179 7f5c973a57a0 10 osd.4 194487 _open_lock_pg 24.3f7 2014-08-13 18:06:06.999191 7f5c973a57a0 10 osd.4 194487 _get_pool 24 2014-08-13 18:06:06.999220 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3f7(unlocked)] enter Initial 2014-08-13 18:06:06.999235 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3f7(unlocked)] enter NotTrimming 2014-08-13 18:06:06.999401 7f5c973a57a0 10 read_log 2014-08-13 18:06:06.999550 7f5c973a57a0 20 read_log 192834'13627 (0'0) promote c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by osd.306.0:27880 2014-08-11 15:23:35.826126 2014-08-13 18:06:06.999571 7f5c973a57a0 20 read_log 192834'13628 (192834'13627) modify c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.933448.0:53 2014-08-11 15:23:35.813800 2014-08-13 18:06:06.999585 7f5c973a57a0 20 read_log 192834'13665 (0'0) promote 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by osd.306.0:27971 2014-08-11 15:23:49.115086 2014-08-13 18:06:06.999598 7f5c973a57a0 20 read_log 192834'13666 (192834'13665) modify 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.933448.0:2852 2014-08-11 15:23:48.870454 2014-08-13 18:06:06.999610 7f5c973a57a0 20 read_log 192834'13689 (0'0) promote 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by osd.306.0:28015 2014-08-11 15:23:59.256466 2014-08-13 18:06:06.999623 7f5c973a57a0 20 read_log 192834'13690 (192834'13689) modify 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.933448.0:4898 2014-08-11 15:23:59.249077 2014-08-13 18:06:06.999635 7f5c973a57a0 20 read_log 192834'13719 (0'0) promote 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by osd.306.0:28089 2014-08-11 15:24:06.760031 2014-08-13 18:06:06.999647 7f5c973a57a0 20 read_log 192834'13720 (192834'13719) modify 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.933448.0:6788 2014-08-11 15:24:06.752258 2014-08-13 18:06:06.999660 7f5c973a57a0 20 read_log 192834'13739 (0'0) promote 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.306.0:28123 2014-08-11 15:24:10.515439 2014-08-13 18:06:06.999681 7f5c973a57a0 20 read_log 192834'13740 (192834'13739) modify 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by client.933448.0:7553 2014-08-11 15:24:10.507486 2014-08-13 18:06:06.999696 7f5c973a57a0 20 read_log 192834'13743 (0'0) promote f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.306.0:28129 2014-08-11 15:24:11.792918 2014-08-13 18:06:06.999708 7f5c973a57a0 20 read_log 192834'13744 (192834'13743) modify f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by client.933448.0:7890 2014-08-11 15:24:11.784675 2014-08-13 18:06:06.999721 7f5c973a57a0 20 read_log 192834'13767 (0'0) promote dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.306.0:28169 2014-08-11 15:24:17.771300 2014-08-13 18:06:06.999733 7f5c973a57a0 20 read_log 192834'13768 (192834'13767) modify dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by client.933448.0:9159 2014-08-11 15:24:17.764345 2014-08-13 18:06:06.999745 7f5c973a57a0 20 read_log 192834'13775 (0'0) promote d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.306.0:28189 2014-08-11 15:24:19.118498 2014-08-13 18:06:06.999757 7f5c973a57a0 20 read_log 192834'13776 (192834'13775) modify d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by client.933448.0:9518 2014-08-11 15:24:19.110819 2014-08-13 18:06:06.999769 7f5c973a57a0 20 read_log 192834'13777 (0'0) promote 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.306.0:28193 2014-08-11 15:24:19.347210 2014-08-13 18:06:06.999781 7f5c973a57a0 20 read_log 192834'13778 (192834'13777) modify 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by client.933448.0:9563 2014-08-11 15:24:19.339217 2014-08-13 18:06:06.999793 7f5c973a57a0 20 read_log 192834'13801 (0'0) promote 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.306.0:28253 2014-08-11 15:24:29.386308 2014-08-13 18:06:06.999805 7f5c973a57a0 20 read_log 192834'13802 (192834'13801) modify 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by client.933448.0:11793 2014-08-11 15:24:29.378570 2014-08-13 18:06:06.999818 7f5c973a57a0 20 read_log 192834'13817 (0'0) promote 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.306.0:28281 2014-08-11 15:24:33.471041 2014-08-13 18:06:06.999834 7f5c973a57a0 20 read_log 192834'13818 (192834'13817) modify 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by client.933448.0:12690 2014-08-11 15:24:33.462814 2014-08-13 18:06:06.999846 7f5c973a57a0 20 read_log 192834'13841 (0'0) promote b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by osd.306.0:28323 2014-08-11 15:24:41.519608 2014-08-13 18:06:06.999858 7f5c973a57a0 20 read_log 192834'13842 (192834'13841) modify b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.933448.0:14130 2014-08-11 15:24:41.510311 2014-08-13 18:06:06.999871 7f5c973a57a0 20 read_log 192834'13847 (0'0) promote 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by osd.306.0:28335 2014-08-11 15:24:44.823697 2014-08-13 18:06:06.999883 7f5c973a57a0 20 read_log 192834'13848 (192834'13847) modify 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.933448.0:14342 2014-08-11 15:24:44.815877 2014-08-13 18:06:06.999895 7f5c973a57a0 20 read_log 192834'13857 (0'0) promote 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by osd.306.0:28347 2014-08-11 15:24:47.167811 2014-08-13 18:06:06.999907 7f5c973a57a0 20 read_log 192834'13858 (192834'13857) modify 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.933448.0:14793 2014-08-11 15:24:47.159175 2014-08-13 18:06:06.999920 7f5c973a57a0 20 read_log 192834'13879 (0'0) promote 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.306.0:28373 2014-08-11 15:24:50.074217 2014-08-13 18:06:06.999943 7f5c973a57a0 20 read_log 192834'13880 (192834'13879) modify 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by client.933448.0:15550 2014-08-11 15:24:50.066438 2014-08-13 18:06:06.999959 7f5c973a57a0 20 read_log 192834'13883 (0'0) promote daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by osd.306.0:28377 2014-08-11 15:24:50.454041 2014-08-13 18:06:06.999971 7f5c973a57a0 20 read_log 192834'13884 (192834'13883) modify daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.933448.0:15632 2014-08-11 15:24:50.446190 2014-08-13 18:06:06.999984 7f5c973a57a0 20 read_log 192834'13895 (0'0) promote b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by osd.306.0:28419 2014-08-11 15:24:59.405618 2014-08-13 18:06:06.999996 7f5c973a57a0 20 read_log 192834'13896 (192834'13895) modify b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.933448.0:17359 2014-08-11 15:24:59.397856 2014-08-13 18:06:07.000008 7f5c973a57a0 20 read_log 192834'13897 (0'0) promote ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by osd.306.0:28423 2014-08-11 15:24:59.646939 2014-08-13 18:06:07.000023 7f5c973a57a0 20 read_log 192834'13898 (192834'13897) modify ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.933448.0:17424 2014-08-11 15:24:59.640020 2014-08-13 18:06:07.000043 7f5c973a57a0 20 read_log 192834'13903 (0'0) promote d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.306.0:28439 2014-08-11 15:25:01.143866 2014-08-13 18:06:07.000056 7f5c973a57a0 20 read_log 192834'13904 (192834'13903) modify d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by client.933448.0:17709 2014-08-11 15:25:01.130089 2014-08-13 18:06:07.000071 7f5c973a57a0 20 read_log 192834'13907 (0'0) promote fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.306.0:28451 2014-08-11 15:25:03.267502 2014-08-13 18:06:07.000083 7f5c973a57a0 20 read_log 192834'13908 (192834'13907) modify fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by client.933448.0:18050 2014-08-11 15:25:03.256054 2014-08-13 18:06:07.000098 7f5c973a57a0 20 read_log 192834'13935 (0'0) promote fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.306.0:28489 2014-08-11 15:25:12.470654 2014-08-13 18:06:07.000110 7f5c973a57a0 20 read_log 192834'13936 (192834'13935) modify fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by client.933448.0:19539 2014-08-11 15:25:12.462660 2014-08-13 18:06:07.000124 7f5c973a57a0 20 read_log 192834'13947 (0'0) promote 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by osd.306.0:28509 2014-08-11 15:25:19.915863 2014-08-13 18:06:07.000137 7f5c973a57a0 20 read_log 192834'13948 (192834'13947) modify 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.933448.0:20245 2014-08-11 15:25:19.907818 2014-08-13 18:06:07.000157 7f5c973a57a0 20 read_log 192834'13949 (0'0) promote b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by osd.306.0:28517 2014-08-11 15:25:20.183559 2014-08-13 18:06:07.000190 7f5c973a57a0 20 read_log 192834'13950 (192834'13949) modify b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.933448.0:20307 2014-08-11 15:25:20.175279 2014-08-13 18:06:07.000205 7f5c973a57a0 20 read_log 192834'13953 (0'0) promote 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by osd.306.0:28525 2014-08-11 15:25:22.051013 2014-08-13 18:06:07.000218 7f5c973a57a0 20 read_log 192834'13954 (192834'13953) modify 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.933448.0:20591 2014-08-11 15:25:22.043775 2014-08-13 18:06:07.000241 7f5c973a57a0 20 read_log 192834'13963 (192834'13628) modify c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.933448.0:22344 2014-08-11 15:25:37.667547 2014-08-13 18:06:07.000265 7f5c973a57a0 20 read_log 192834'13982 (192834'13666) modify 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.933448.0:25143 2014-08-11 15:25:41.827334 2014-08-13 18:06:07.000280 7f5c973a57a0 20 read_log 192834'13993 (192834'13690) modify 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.933448.0:27189 2014-08-11 15:25:44.197911 2014-08-13 18:06:07.000293 7f5c973a57a0 20 read_log 192834'14008 (192834'13720) modify 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.933448.0:29079 2014-08-11 15:25:46.983836 2014-08-13 18:06:07.000305 7f5c973a57a0 20 read_log 192834'14018 (192834'13740) modify 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by client.933448.0:29844 2014-08-11 15:25:47.990705 2014-08-13 18:06:07.000318 7f5c973a57a0 20 read_log 192834'14020 (192834'13744) modify f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by client.933448.0:30181 2014-08-11 15:25:48.510464 2014-08-13 18:06:07.000335 7f5c973a57a0 20 read_log 192834'14032 (192834'13768) modify dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by client.933448.0:31450 2014-08-11 15:25:50.076029 2014-08-13 18:06:07.000349 7f5c973a57a0 20 read_log 192834'14036 (192834'13776) modify d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by client.933448.0:31809 2014-08-11 15:25:50.628314 2014-08-13 18:06:07.000372 7f5c973a57a0 20 read_log 192834'14037 (192834'13778) modify 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by client.933448.0:31854 2014-08-11 15:25:50.695053 2014-08-13 18:06:07.000392 7f5c973a57a0 20 read_log 192834'14049 (192834'13802) modify 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by client.933448.0:34084 2014-08-11 15:25:54.560523 2014-08-13 18:06:07.000405 7f5c973a57a0 20 read_log 192834'14057 (192834'13818) modify 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by client.933448.0:34981 2014-08-11 15:25:55.660605 2014-08-13 18:06:07.000431 7f5c973a57a0 20 read_log 192834'14069 (192834'13842) modify b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.933448.0:36421 2014-08-11 15:25:57.712095 2014-08-13 18:06:07.000446 7f5c973a57a0 20 read_log 192834'14072 (192834'13848) modify 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.933448.0:36633 2014-08-11 15:25:58.105001 2014-08-13 18:06:07.000466 7f5c973a57a0 20 read_log 192834'14077 (192834'13858) modify 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.933448.0:37084 2014-08-11 15:25:58.899196 2014-08-13 18:06:07.000482 7f5c973a57a0 20 read_log 192834'14088 (192834'13880) modify 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by client.933448.0:37841 2014-08-11 15:26:00.169333 2014-08-13 18:06:07.000495 7f5c973a57a0 20 read_log 192834'14090 (192834'13884) modify daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.933448.0:37923 2014-08-11 15:26:00.269717 2014-08-13 18:06:07.000516 7f5c973a57a0 20 read_log 192834'14096 (192834'13896) modify b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.933448.0:39650 2014-08-11 15:26:02.009965 2014-08-13 18:06:07.000532 7f5c973a57a0 20 read_log 192834'14097 (192834'13898) modify ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.933448.0:39715 2014-08-11 15:26:02.040124 2014-08-13 18:06:07.000545 7f5c973a57a0 20 read_log 192834'14100 (192834'13904) modify d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by client.933448.0:40000 2014-08-11 15:26:02.213408 2014-08-13 18:06:07.000558 7f5c973a57a0 20 read_log 192834'14102 (192834'13908) modify fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by client.933448.0:40341 2014-08-11 15:26:02.576847 2014-08-13 18:06:07.000570 7f5c973a57a0 20 read_log 192834'14116 (192834'13936) modify fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by client.933448.0:41830 2014-08-11 15:26:03.067054 2014-08-13 18:06:07.000591 7f5c973a57a0 20 read_log 192834'14122 (192834'13948) modify 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.933448.0:42536 2014-08-11 15:26:03.309522 2014-08-13 18:06:07.000612 7f5c973a57a0 20 read_log 192834'14123 (192834'13950) modify b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.933448.0:42598 2014-08-11 15:26:03.328524 2014-08-13 18:06:07.000625 7f5c973a57a0 20 read_log 192834'14125 (192834'13954) modify 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.933448.0:42882 2014-08-11 15:26:03.420815 2014-08-13 18:06:07.000637 7f5c973a57a0 20 read_log 192834'14150 (0'0) promote 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by osd.306.0:28938 2014-08-11 15:26:19.560172 2014-08-13 18:06:07.000664 7f5c973a57a0 20 read_log 192834'14151 (192834'14150) modify 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.1883634.0:1396 2014-08-11 15:26:19.551952 2014-08-13 18:06:07.000677 7f5c973a57a0 20 read_log 192834'14170 (0'0) promote 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by osd.306.0:28974 2014-08-11 15:26:24.879518 2014-08-13 18:06:07.000690 7f5c973a57a0 20 read_log 192834'14171 (192834'14170) modify 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.1883634.0:2837 2014-08-11 15:26:24.872144 2014-08-13 18:06:07.000702 7f5c973a57a0 20 read_log 192834'14196 (0'0) promote 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by osd.306.0:29020 2014-08-11 15:26:31.483633 2014-08-13 18:06:07.000714 7f5c973a57a0 20 read_log 192834'14197 (192834'14196) modify 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.1883634.0:4678 2014-08-11 15:26:31.474601 2014-08-13 18:06:07.000728 7f5c973a57a0 20 read_log 192834'14216 (0'0) promote cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by osd.306.0:29054 2014-08-11 15:26:35.806194 2014-08-13 18:06:07.000740 7f5c973a57a0 20 read_log 192834'14217 (192834'14216) modify cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.1883634.0:5832 2014-08-11 15:26:35.798089 2014-08-13 18:06:07.000752 7f5c973a57a0 20 read_log 192834'14244 (0'0) promote df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by osd.306.0:29120 2014-08-11 15:26:43.667852 2014-08-13 18:06:07.000764 7f5c973a57a0 20 read_log 192834'14245 (192834'14244) modify df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.1883634.0:7792 2014-08-11 15:26:43.658736 2014-08-13 18:06:07.000785 7f5c973a57a0 20 read_log 192834'14254 (0'0) promote 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by osd.306.0:29144 2014-08-11 15:26:46.303517 2014-08-13 18:06:07.000801 7f5c973a57a0 20 read_log 192834'14255 (192834'14254) modify 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.1883634.0:8437 2014-08-11 15:26:46.294310 2014-08-13 18:06:07.000814 7f5c973a57a0 20 read_log 192834'14310 (0'0) promote 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by osd.306.0:29236 2014-08-11 15:26:58.651994 2014-08-13 18:06:07.000826 7f5c973a57a0 20 read_log 192834'14311 (192834'14310) modify 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.1883634.0:11174 2014-08-11 15:26:58.643100 2014-08-13 18:06:07.000839 7f5c973a57a0 20 read_log 192834'14320 (0'0) promote b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by osd.306.0:29266 2014-08-11 15:27:01.110941 2014-08-13 18:06:07.000851 7f5c973a57a0 20 read_log 192834'14321 (192834'14320) modify b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.1883634.0:11734 2014-08-11 15:27:01.103851 2014-08-13 18:06:07.000864 7f5c973a57a0 20 read_log 192834'14336 (0'0) promote f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by osd.306.0:29316 2014-08-11 15:27:07.208182 2014-08-13 18:06:07.000886 7f5c973a57a0 20 read_log 192834'14337 (192834'14336) modify f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.1883634.0:13010 2014-08-11 15:27:07.198134 2014-08-13 18:06:07.000902 7f5c973a57a0 20 read_log 192834'14364 (0'0) promote cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by osd.306.0:29378 2014-08-11 15:27:18.892648 2014-08-13 18:06:07.000914 7f5c973a57a0 20 read_log 192834'14365 (192834'14364) modify cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.1883634.0:15290 2014-08-11 15:27:18.883444 2014-08-13 18:06:07.000927 7f5c973a57a0 20 read_log 192834'14384 (0'0) promote 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by osd.306.0:29422 2014-08-11 15:27:27.303616 2014-08-13 18:06:07.000939 7f5c973a57a0 20 read_log 192834'14385 (192834'14384) modify 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.1883634.0:16375 2014-08-11 15:27:27.295529 2014-08-13 18:06:07.000952 7f5c973a57a0 20 read_log 192834'14386 (0'0) promote 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by osd.306.0:29430 2014-08-11 15:27:28.699178 2014-08-13 18:06:07.000964 7f5c973a57a0 20 read_log 192834'14387 (192834'14386) modify 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.1883634.0:16659 2014-08-11 15:27:28.690698 2014-08-13 18:06:07.000976 7f5c973a57a0 20 read_log 192834'14408 (0'0) promote 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.306.0:29468 2014-08-11 15:27:42.831667 2014-08-13 18:06:07.000988 7f5c973a57a0 20 read_log 192834'14409 (192834'14408) modify 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by client.1883634.0:18315 2014-08-11 15:27:42.821838 2014-08-13 18:06:07.001000 7f5c973a57a0 20 read_log 192834'14420 (0'0) promote 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by osd.306.0:29480 2014-08-11 15:27:47.001068 2014-08-13 18:06:07.001012 7f5c973a57a0 20 read_log 192834'14421 (192834'14420) modify 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.1883634.0:18674 2014-08-11 15:27:46.993331 2014-08-13 18:06:07.001025 7f5c973a57a0 20 read_log 192834'14452 (0'0) promote 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.306.0:29552 2014-08-11 15:28:02.982966 2014-08-13 18:06:07.001037 7f5c973a57a0 20 read_log 192834'14453 (192834'14452) modify 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by client.1883634.0:21135 2014-08-11 15:28:02.974405 2014-08-13 18:06:07.001049 7f5c973a57a0 20 read_log 192834'14470 (0'0) promote 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by osd.306.0:29590 2014-08-11 15:28:09.736738 2014-08-13 18:06:07.001064 7f5c973a57a0 20 read_log 192834'14471 (192834'14470) modify 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.1883634.0:22592 2014-08-11 15:28:09.727955 2014-08-13 18:06:07.001077 7f5c973a57a0 20 read_log 192834'14488 (192834'14151) modify 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.1883634.0:24720 2014-08-11 15:28:19.424616 2014-08-13 18:06:07.001090 7f5c973a57a0 20 read_log 192834'14498 (192834'14171) modify 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.1883634.0:26161 2014-08-11 15:28:22.694898 2014-08-13 18:06:07.001102 7f5c973a57a0 20 read_log 192834'14511 (192834'14197) modify 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.1883634.0:28002 2014-08-11 15:28:25.259435 2014-08-13 18:06:07.001116 7f5c973a57a0 20 read_log 192834'14521 (192834'14217) modify cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.1883634.0:29156 2014-08-11 15:28:26.804373 2014-08-13 18:06:07.001129 7f5c973a57a0 20 read_log 192834'14535 (192834'14245) modify df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.1883634.0:31116 2014-08-11 15:28:29.931270 2014-08-13 18:06:07.001154 7f5c973a57a0 20 read_log 192834'14540 (192834'14255) modify 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.1883634.0:31761 2014-08-11 15:28:30.931289 2014-08-13 18:06:07.001170 7f5c973a57a0 20 read_log 192834'14568 (192834'14311) modify 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.1883634.0:34498 2014-08-11 15:28:34.923419 2014-08-13 18:06:07.001182 7f5c973a57a0 20 read_log 192834'14573 (192834'14321) modify b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.1883634.0:35058 2014-08-11 15:28:35.985753 2014-08-13 18:06:07.001195 7f5c973a57a0 20 read_log 192834'14581 (192834'14337) modify f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.1883634.0:36334 2014-08-11 15:28:37.903473 2014-08-13 18:06:07.001207 7f5c973a57a0 20 read_log 192834'14595 (192834'14365) modify cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.1883634.0:38614 2014-08-11 15:28:40.936446 2014-08-13 18:06:07.001219 7f5c973a57a0 20 read_log 192834'14605 (192834'14385) modify 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.1883634.0:39699 2014-08-11 15:28:42.443577 2014-08-13 18:06:07.001231 7f5c973a57a0 20 read_log 192834'14606 (192834'14387) modify 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.1883634.0:39983 2014-08-11 15:28:43.008034 2014-08-13 18:06:07.001252 7f5c973a57a0 20 read_log 192834'14617 (192834'14409) modify 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by client.1883634.0:41639 2014-08-11 15:28:45.388854 2014-08-13 18:06:07.001265 7f5c973a57a0 20 read_log 192834'14623 (192834'14421) modify 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.1883634.0:41998 2014-08-11 15:28:45.625325 2014-08-13 18:06:07.001277 7f5c973a57a0 20 read_log 192834'14639 (192834'14453) modify 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by client.1883634.0:44459 2014-08-11 15:28:46.864570 2014-08-13 18:06:07.001295 7f5c973a57a0 20 read_log 192834'14648 (192834'14471) modify 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.1883634.0:45916 2014-08-11 15:28:47.432494 2014-08-13 18:06:07.001307 7f5c973a57a0 20 read_log 192888'14654 (0'0) promote a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.4.0:21440 2014-08-11 16:45:02.937110 2014-08-13 18:06:07.001320 7f5c973a57a0 20 read_log 192888'14655 (192888'14654) modify a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by client.1315259.0:2017 2014-08-11 16:45:02.929036 2014-08-13 18:06:07.001336 7f5c973a57a0 20 read_log 192888'14658 (0'0) promote c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by osd.4.0:21456 2014-08-11 16:45:05.014076 2014-08-13 18:06:07.001351 7f5c973a57a0 20 read_log 192888'14659 (192888'14658) modify c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.1315259.0:2578 2014-08-11 16:45:05.003359 2014-08-13 18:06:07.001364 7f5c973a57a0 20 read_log 192888'14660 (0'0) promote d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by osd.4.0:21478 2014-08-11 16:45:07.732218 2014-08-13 18:06:07.001376 7f5c973a57a0 20 read_log 192888'14661 (192888'14660) modify d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.1315259.0:3274 2014-08-11 16:45:07.725306 2014-08-13 18:06:07.001404 7f5c973a57a0 20 read_log 192888'14664 (0'0) promote b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by osd.4.0:21486 2014-08-11 16:45:09.941730 2014-08-13 18:06:07.001417 7f5c973a57a0 20 read_log 192888'14665 (192888'14664) modify b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.1315259.0:3857 2014-08-11 16:45:09.931205 2014-08-13 18:06:07.001500 7f5c973a57a0 20 read_log 192888'14672 (0'0) promote 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by osd.4.0:21582 2014-08-11 16:45:20.649586 2014-08-13 18:06:07.001520 7f5c973a57a0 20 read_log 192888'14673 (192888'14672) modify 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.1315259.0:6794 2014-08-11 16:45:20.641480 2014-08-13 18:06:07.001533 7f5c973a57a0 20 read_log 192888'14676 (0'0) promote e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by osd.4.0:21602 2014-08-11 16:45:25.258901 2014-08-13 18:06:07.001546 7f5c973a57a0 20 read_log 192888'14677 (192888'14676) modify e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.1315259.0:8060 2014-08-11 16:45:25.249117 2014-08-13 18:06:07.001558 7f5c973a57a0 20 read_log 192888'14678 (0'0) promote 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by osd.4.0:21624 2014-08-11 16:45:27.444965 2014-08-13 18:06:07.001578 7f5c973a57a0 20 read_log 192888'14679 (192888'14678) modify 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.1315259.0:8630 2014-08-11 16:45:27.435698 2014-08-13 18:06:07.001591 7f5c973a57a0 20 read_log 192888'14680 (0'0) promote e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by osd.4.0:21628 2014-08-11 16:45:27.690926 2014-08-13 18:06:07.001603 7f5c973a57a0 20 read_log 192888'14681 (192888'14680) modify e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.1315259.0:8698 2014-08-11 16:45:27.678606 2014-08-13 18:06:07.001616 7f5c973a57a0 20 read_log 192888'14686 (0'0) promote 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by osd.4.0:21660 2014-08-11 16:45:33.991986 2014-08-13 18:06:07.001628 7f5c973a57a0 20 read_log 192888'14687 (192888'14686) modify 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.1315259.0:10351 2014-08-11 16:45:33.983200 2014-08-13 18:06:07.001674 7f5c973a57a0 20 read_log 192888'14690 (0'0) promote 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by osd.4.0:21672 2014-08-11 16:45:35.287364 2014-08-13 18:06:07.001691 7f5c973a57a0 20 read_log 192888'14691 (192888'14690) modify 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.1315259.0:10668 2014-08-11 16:45:35.278289 2014-08-13 18:06:07.001703 7f5c973a57a0 20 read_log 192888'14692 (0'0) promote 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by osd.4.0:21696 2014-08-11 16:45:38.355299 2014-08-13 18:06:07.001716 7f5c973a57a0 20 read_log 192888'14693 (192888'14692) modify 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.1315259.0:11438 2014-08-11 16:45:38.347476 2014-08-13 18:06:07.001728 7f5c973a57a0 20 read_log 192888'14694 (0'0) promote 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.4.0:21700 2014-08-11 16:45:39.198432 2014-08-13 18:06:07.001741 7f5c973a57a0 20 read_log 192888'14695 (192888'14694) modify 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by client.1315259.0:11654 2014-08-11 16:45:39.190244 2014-08-13 18:06:07.001753 7f5c973a57a0 20 read_log 192888'14696 (0'0) promote ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by osd.4.0:21724 2014-08-11 16:45:42.534854 2014-08-13 18:06:07.001765 7f5c973a57a0 20 read_log 192888'14697 (192888'14696) modify ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.1315259.0:12440 2014-08-11 16:45:42.527527 2014-08-13 18:06:07.001777 7f5c973a57a0 20 read_log 192888'14698 (0'0) promote da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by osd.4.0:21730 2014-08-11 16:45:43.090714 2014-08-13 18:06:07.001790 7f5c973a57a0 20 read_log 192888'14699 (192888'14698) modify da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.1315259.0:12582 2014-08-11 16:45:43.081732 2014-08-13 18:06:07.001812 7f5c973a57a0 20 read_log 192888'14702 (0'0) promote 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by osd.4.0:21824 2014-08-11 16:45:57.119876 2014-08-13 18:06:07.001828 7f5c973a57a0 20 read_log 192888'14703 (192888'14702) modify 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.1315259.0:16182 2014-08-11 16:45:57.112089 2014-08-13 18:06:07.001840 7f5c973a57a0 20 read_log 192888'14704 (0'0) promote 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by osd.4.0:21828 2014-08-11 16:45:57.482144 2014-08-13 18:06:07.001852 7f5c973a57a0 20 read_log 192888'14705 (192888'14704) modify 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.1315259.0:16280 2014-08-11 16:45:57.473494 2014-08-13 18:06:07.001865 7f5c973a57a0 20 read_log 192888'14708 (0'0) promote ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by osd.4.0:21836 2014-08-11 16:45:58.568385 2014-08-13 18:06:07.001877 7f5c973a57a0 20 read_log 192888'14709 (192888'14708) modify ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.1315259.0:16540 2014-08-11 16:45:58.560219 2014-08-13 18:06:07.001889 7f5c973a57a0 20 read_log 192888'14710 (0'0) promote 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by osd.4.0:21846 2014-08-11 16:46:00.946618 2014-08-13 18:06:07.001901 7f5c973a57a0 20 read_log 192888'14711 (192888'14710) modify 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.1315259.0:17152 2014-08-11 16:46:00.937813 2014-08-13 18:06:07.001913 7f5c973a57a0 20 read_log 192888'14712 (0'0) promote fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.4.0:21872 2014-08-11 16:46:04.155557 2014-08-13 18:06:07.001925 7f5c973a57a0 20 read_log 192888'14713 (192888'14712) modify fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by client.1315259.0:17929 2014-08-11 16:46:04.148939 2014-08-13 18:06:07.001938 7f5c973a57a0 20 read_log 192888'14714 (0'0) promote 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by osd.4.0:21876 2014-08-11 16:46:04.555483 2014-08-13 18:06:07.001950 7f5c973a57a0 20 read_log 192888'14715 (192888'14714) modify 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.1315259.0:18036 2014-08-11 16:46:04.546308 2014-08-13 18:06:07.001962 7f5c973a57a0 20 read_log 192888'14716 (0'0) promote 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.4.0:21882 2014-08-11 16:46:06.279008 2014-08-13 18:06:07.001974 7f5c973a57a0 20 read_log 192888'14717 (192888'14716) modify 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by client.1315259.0:18498 2014-08-11 16:46:06.270706 2014-08-13 18:06:07.001986 7f5c973a57a0 20 read_log 192888'14718 (0'0) promote f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.4.0:21896 2014-08-11 16:46:07.122915 2014-08-13 18:06:07.001998 7f5c973a57a0 20 read_log 192888'14719 (192888'14718) modify f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by client.1315259.0:18723 2014-08-11 16:46:07.114287 2014-08-13 18:06:07.002011 7f5c973a57a0 20 read_log 192888'14720 (0'0) promote 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by osd.4.0:21902 2014-08-11 16:46:07.947990 2014-08-13 18:06:07.002023 7f5c973a57a0 20 read_log 192888'14721 (192888'14720) modify 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.1315259.0:18941 2014-08-11 16:46:07.939990 2014-08-13 18:06:07.002037 7f5c973a57a0 20 read_log 192888'14724 (0'0) promote 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by osd.4.0:21952 2014-08-11 16:46:15.340099 2014-08-13 18:06:07.002050 7f5c973a57a0 20 read_log 192888'14725 (192888'14724) modify 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.1315259.0:20854 2014-08-11 16:46:15.331011 2014-08-13 18:06:07.002071 7f5c973a57a0 20 read_log 192888'14726 (0'0) promote 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by osd.4.0:21966 2014-08-11 16:46:16.745419 2014-08-13 18:06:07.002086 7f5c973a57a0 20 read_log 192888'14727 (192888'14726) modify 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.1315259.0:21235 2014-08-11 16:46:16.736661 2014-08-13 18:06:07.002098 7f5c973a57a0 20 read_log 192888'14728 (0'0) promote efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by osd.4.0:21988 2014-08-11 16:46:19.578778 2014-08-13 18:06:07.002113 7f5c973a57a0 20 read_log 192888'14729 (192888'14728) modify efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.1315259.0:21979 2014-08-11 16:46:19.571035 2014-08-13 18:06:07.002127 7f5c973a57a0 20 read_log 192888'14730 (0'0) promote 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by osd.4.0:22006 2014-08-11 16:46:21.895428 2014-08-13 18:06:07.002141 7f5c973a57a0 20 read_log 192888'14731 (192888'14730) modify 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.1315259.0:22557 2014-08-11 16:46:21.887387 2014-08-13 18:06:07.002164 7f5c973a57a0 20 read_log 192888'14732 (0'0) promote be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by osd.4.0:22016 2014-08-11 16:46:22.873091 2014-08-13 18:06:07.002177 7f5c973a57a0 20 read_log 192888'14733 (192888'14732) modify be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.1315259.0:22820 2014-08-11 16:46:22.862434 2014-08-13 18:06:07.002192 7f5c973a57a0 20 read_log 192888'14734 (0'0) promote d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by osd.4.0:22018 2014-08-11 16:46:23.339711 2014-08-13 18:06:07.002204 7f5c973a57a0 20 read_log 192888'14735 (192888'14734) modify d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.1315259.0:22930 2014-08-11 16:46:23.330369 2014-08-13 18:06:07.002216 7f5c973a57a0 20 read_log 192888'14736 (0'0) promote 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by osd.4.0:22048 2014-08-11 16:46:28.853784 2014-08-13 18:06:07.002229 7f5c973a57a0 20 read_log 192888'14737 (192888'14736) modify 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.1315259.0:24323 2014-08-11 16:46:28.842990 2014-08-13 18:06:07.002247 7f5c973a57a0 20 read_log 192888'14740 (0'0) promote 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by osd.4.0:22078 2014-08-11 16:46:31.709722 2014-08-13 18:06:07.002259 7f5c973a57a0 20 read_log 192888'14741 (192888'14740) modify 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.1315259.0:25054 2014-08-11 16:46:31.701223 2014-08-13 18:06:07.002272 7f5c973a57a0 20 read_log 192888'14742 (0'0) promote 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by osd.4.0:22100 2014-08-11 16:46:34.820522 2014-08-13 18:06:07.002288 7f5c973a57a0 20 read_log 192888'14743 (192888'14742) modify 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.1315259.0:25801 2014-08-11 16:46:34.809800 2014-08-13 18:06:07.002300 7f5c973a57a0 20 read_log 192888'14744 (0'0) promote 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by osd.4.0:22104 2014-08-11 16:46:36.730503 2014-08-13 18:06:07.002313 7f5c973a57a0 20 read_log 192888'14745 (192888'14744) modify 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.1315259.0:26231 2014-08-11 16:46:36.720809 2014-08-13 18:06:07.002325 7f5c973a57a0 20 read_log 192888'14750 (0'0) promote cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by osd.4.0:22166 2014-08-11 16:46:47.937414 2014-08-13 18:06:07.002341 7f5c973a57a0 20 read_log 192888'14751 (192888'14750) modify cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.1315259.0:28977 2014-08-11 16:46:47.928446 2014-08-13 18:06:07.002363 7f5c973a57a0 20 read_log 192888'14758 (0'0) promote 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by osd.4.0:22218 2014-08-11 16:46:54.300245 2014-08-13 18:06:07.002382 7f5c973a57a0 20 read_log 192888'14759 (192888'14758) modify 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.1315259.0:30579 2014-08-11 16:46:54.290044 2014-08-13 18:06:07.002395 7f5c973a57a0 20 read_log 192888'14761 (192888'14655) modify a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by client.1315259.0:32744 2014-08-11 16:46:56.458468 2014-08-13 18:06:07.002416 7f5c973a57a0 20 read_log 192888'14763 (192888'14659) modify c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.1315259.0:33305 2014-08-11 16:46:56.604203 2014-08-13 18:06:07.002432 7f5c973a57a0 20 read_log 192888'14764 (192888'14661) modify d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.1315259.0:34001 2014-08-11 16:46:56.779032 2014-08-13 18:06:07.002444 7f5c973a57a0 20 read_log 192888'14766 (192888'14665) modify b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.1315259.0:34584 2014-08-11 16:46:56.932302 2014-08-13 18:06:07.002467 7f5c973a57a0 20 read_log 192888'14770 (192888'14673) modify 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.1315259.0:37521 2014-08-11 16:46:57.705566 2014-08-13 18:06:07.002482 7f5c973a57a0 20 read_log 192888'14772 (192888'14677) modify e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.1315259.0:38787 2014-08-11 16:46:58.045702 2014-08-13 18:06:07.002495 7f5c973a57a0 20 read_log 192888'14773 (192888'14679) modify 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.1315259.0:39357 2014-08-11 16:46:58.190224 2014-08-13 18:06:07.002517 7f5c973a57a0 20 read_log 192888'14774 (192888'14681) modify e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.1315259.0:39425 2014-08-11 16:46:58.208281 2014-08-13 18:06:07.002530 7f5c973a57a0 20 read_log 192888'14777 (192888'14687) modify 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.1315259.0:41078 2014-08-11 16:46:58.653011 2014-08-13 18:06:07.002543 7f5c973a57a0 20 read_log 192888'14779 (192888'14691) modify 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.1315259.0:41395 2014-08-11 16:46:58.735115 2014-08-13 18:06:07.002555 7f5c973a57a0 20 read_log 192888'14780 (192888'14693) modify 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.1315259.0:42165 2014-08-11 16:46:58.936456 2014-08-13 18:06:07.002571 7f5c973a57a0 20 read_log 192888'14781 (192888'14695) modify 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by client.1315259.0:42381 2014-08-11 16:46:58.995334 2014-08-13 18:06:07.002585 7f5c973a57a0 20 read_log 192888'14782 (192888'14697) modify ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.1315259.0:43167 2014-08-11 16:46:59.211841 2014-08-13 18:06:07.002598 7f5c973a57a0 20 read_log 192888'14783 (192888'14699) modify da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.1315259.0:43309 2014-08-11 16:46:59.252901 2014-08-13 18:06:07.002610 7f5c973a57a0 20 read_log 192888'14785 (192888'14703) modify 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.1315259.0:46909 2014-08-11 16:47:00.292764 2014-08-13 18:06:07.002623 7f5c973a57a0 20 read_log 192888'14786 (192888'14705) modify 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.1315259.0:47007 2014-08-11 16:47:00.320338 2014-08-13 18:06:07.002635 7f5c973a57a0 20 read_log 192888'14788 (192888'14709) modify ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.1315259.0:47267 2014-08-11 16:47:00.400567 2014-08-13 18:06:07.002648 7f5c973a57a0 20 read_log 192888'14789 (192888'14711) modify 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.1315259.0:47879 2014-08-11 16:47:00.596698 2014-08-13 18:06:07.002670 7f5c973a57a0 20 read_log 192888'14790 (192888'14713) modify fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by client.1315259.0:48656 2014-08-11 16:47:00.827139 2014-08-13 18:06:07.002685 7f5c973a57a0 20 read_log 192888'14791 (192888'14715) modify 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.1315259.0:48763 2014-08-11 16:47:00.859603 2014-08-13 18:06:07.002698 7f5c973a57a0 20 read_log 192888'14792 (192888'14717) modify 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by client.1315259.0:49225 2014-08-11 16:47:00.995000 2014-08-13 18:06:07.002710 7f5c973a57a0 20 read_log 192888'14793 (192888'14719) modify f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by client.1315259.0:49450 2014-08-11 16:47:01.066247 2014-08-13 18:06:07.002722 7f5c973a57a0 20 read_log 192888'14794 (192888'14721) modify 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.1315259.0:49668 2014-08-11 16:47:01.137496 2014-08-13 18:06:07.002734 7f5c973a57a0 20 read_log 192888'14796 (192888'14725) modify 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.1315259.0:51581 2014-08-11 16:47:01.724332 2014-08-13 18:06:07.002747 7f5c973a57a0 20 read_log 192888'14797 (192888'14727) modify 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.1315259.0:51962 2014-08-11 16:47:01.832891 2014-08-13 18:06:07.002770 7f5c973a57a0 20 read_log 192888'14798 (192888'14729) modify efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.1315259.0:52706 2014-08-11 16:47:02.047650 2014-08-13 18:06:07.002785 7f5c973a57a0 20 read_log 192888'14799 (192888'14731) modify 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.1315259.0:53284 2014-08-11 16:47:02.208030 2014-08-13 18:06:07.002798 7f5c973a57a0 20 read_log 192888'14800 (192888'14733) modify be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.1315259.0:53547 2014-08-11 16:47:02.286959 2014-08-13 18:06:07.002810 7f5c973a57a0 20 read_log 192888'14801 (192888'14735) modify d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.1315259.0:53657 2014-08-11 16:47:02.319461 2014-08-13 18:06:07.002824 7f5c973a57a0 20 read_log 192888'14802 (192888'14737) modify 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.1315259.0:55050 2014-08-11 16:47:02.796236 2014-08-13 18:06:07.002837 7f5c973a57a0 20 read_log 192888'14804 (192888'14741) modify 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.1315259.0:55781 2014-08-11 16:47:03.010939 2014-08-13 18:06:07.002851 7f5c973a57a0 20 read_log 192888'14805 (192888'14743) modify 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.1315259.0:56528 2014-08-11 16:47:03.233773 2014-08-13 18:06:07.002866 7f5c973a57a0 20 read_log 192888'14806 (192888'14745) modify 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.1315259.0:56958 2014-08-11 16:47:03.366402 2014-08-13 18:06:07.002878 7f5c973a57a0 20 read_log 192888'14809 (192888'14751) modify cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.1315259.0:59704 2014-08-11 16:47:04.215685 2014-08-13 18:06:07.002890 7f5c973a57a0 20 read_log 192888'14813 (192888'14759) modify 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.1315259.0:61306 2014-08-11 16:47:04.699315 2014-08-13 18:06:07.002906 7f5c973a57a0 20 read_log 192888'14816 (0'0) promote 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.4.0:22648 2014-08-11 17:05:15.756733 2014-08-13 18:06:07.002921 7f5c973a57a0 20 read_log 192888'14817 (192888'14816) modify 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by client.942868.0:185 2014-08-11 17:05:15.744166 2014-08-13 18:06:07.002934 7f5c973a57a0 20 read_log 192888'14822 (0'0) promote 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.4.0:22685 2014-08-11 17:05:21.171385 2014-08-13 18:06:07.002957 7f5c973a57a0 20 read_log 192888'14823 (192888'14822) modify 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by client.942868.0:1506 2014-08-11 17:05:21.161585 2014-08-13 18:06:07.002973 7f5c973a57a0 20 read_log 192888'14824 (0'0) promote e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by osd.4.0:22722 2014-08-11 17:05:26.942292 2014-08-13 18:06:07.002985 7f5c973a57a0 20 read_log 192888'14825 (192888'14824) modify e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.942868.0:2973 2014-08-11 17:05:26.933706 2014-08-13 18:06:07.003029 7f5c973a57a0 20 read_log 192888'14830 (0'0) promote dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.4.0:22760 2014-08-11 17:05:31.438896 2014-08-13 18:06:07.003046 7f5c973a57a0 20 read_log 192888'14831 (192888'14830) modify dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by client.942868.0:4191 2014-08-11 17:05:31.431316 2014-08-13 18:06:07.003069 7f5c973a57a0 20 read_log 192888'14832 (0'0) promote 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.4.0:22764 2014-08-11 17:05:31.847811 2014-08-13 18:06:07.003082 7f5c973a57a0 20 read_log 192888'14833 (192888'14832) modify 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by client.942868.0:4302 2014-08-11 17:05:31.839700 2014-08-13 18:06:07.003094 7f5c973a57a0 20 read_log 192888'14834 (0'0) promote 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by osd.4.0:22782 2014-08-11 17:05:33.606301 2014-08-13 18:06:07.003121 7f5c973a57a0 20 read_log 192888'14835 (192888'14834) modify 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.942868.0:4760 2014-08-11 17:05:33.599597 2014-08-13 18:06:07.003134 7f5c973a57a0 20 read_log 192888'14836 (0'0) promote e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.4.0:22802 2014-08-11 17:05:36.309744 2014-08-13 18:06:07.003151 7f5c973a57a0 20 read_log 192888'14837 (192888'14836) modify e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by client.942868.0:5451 2014-08-11 17:05:36.302167 2014-08-13 18:06:07.003167 7f5c973a57a0 20 read_log 192888'14842 (0'0) promote 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by osd.4.0:22826 2014-08-11 17:05:39.441896 2014-08-13 18:06:07.003179 7f5c973a57a0 20 read_log 192888'14843 (192888'14842) modify 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.942868.0:6311 2014-08-11 17:05:39.432965 2014-08-13 18:06:07.003196 7f5c973a57a0 20 read_log 192888'14848 (0'0) promote fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by osd.4.0:22860 2014-08-11 17:05:44.116272 2014-08-13 18:06:07.003208 7f5c973a57a0 20 read_log 192888'14849 (192888'14848) modify fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.942868.0:7527 2014-08-11 17:05:44.108566 2014-08-13 18:06:07.003221 7f5c973a57a0 20 read_log 192888'14854 (0'0) promote 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.4.0:22880 2014-08-11 17:05:47.165759 2014-08-13 18:06:07.003236 7f5c973a57a0 20 read_log 192888'14855 (192888'14854) modify 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by client.942868.0:8371 2014-08-11 17:05:47.156438 2014-08-13 18:06:07.003251 7f5c973a57a0 20 read_log 192888'14856 (0'0) promote 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by osd.4.0:22888 2014-08-11 17:05:48.850351 2014-08-13 18:06:07.003264 7f5c973a57a0 20 read_log 192888'14857 (192888'14856) modify 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.942868.0:8813 2014-08-11 17:05:48.839328 2014-08-13 18:06:07.003276 7f5c973a57a0 20 read_log 192888'14858 (0'0) promote abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.4.0:22890 2014-08-11 17:05:49.651794 2014-08-13 18:06:07.003295 7f5c973a57a0 20 read_log 192888'14859 (192888'14858) modify abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by client.942868.0:9036 2014-08-11 17:05:49.643654 2014-08-13 18:06:07.003310 7f5c973a57a0 20 read_log 192888'14864 (0'0) promote 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by osd.4.0:23016 2014-08-11 17:06:09.762906 2014-08-13 18:06:07.003331 7f5c973a57a0 20 read_log 192888'14865 (192888'14864) modify 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.942868.0:14379 2014-08-11 17:06:09.752978 2014-08-13 18:06:07.003348 7f5c973a57a0 20 read_log 192888'14866 (0'0) promote 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by osd.4.0:23044 2014-08-11 17:06:14.106738 2014-08-13 18:06:07.003360 7f5c973a57a0 20 read_log 192888'14867 (192888'14866) modify 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.942868.0:15480 2014-08-11 17:06:14.089192 2014-08-13 18:06:07.003380 7f5c973a57a0 20 read_log 192888'14868 (0'0) promote 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by osd.4.0:23060 2014-08-11 17:06:16.008122 2014-08-13 18:06:07.003395 7f5c973a57a0 20 read_log 192888'14869 (192888'14868) modify 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.942868.0:15973 2014-08-11 17:06:15.997883 2014-08-13 18:06:07.003408 7f5c973a57a0 20 read_log 192888'14870 (0'0) promote f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by osd.4.0:23066 2014-08-11 17:06:18.390139 2014-08-13 18:06:07.003420 7f5c973a57a0 20 read_log 192888'14871 (192888'14870) modify f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.942868.0:16571 2014-08-11 17:06:18.381535 2014-08-13 18:06:07.003433 7f5c973a57a0 20 read_log 192888'14872 (0'0) promote e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by osd.4.0:23074 2014-08-11 17:06:19.360374 2014-08-13 18:06:07.003445 7f5c973a57a0 20 read_log 192888'14873 (192888'14872) modify e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.942868.0:16826 2014-08-11 17:06:19.351425 2014-08-13 18:06:07.003457 7f5c973a57a0 20 read_log 192888'14882 (0'0) promote dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.4.0:23134 2014-08-11 17:06:26.916121 2014-08-13 18:06:07.003470 7f5c973a57a0 20 read_log 192888'14883 (192888'14882) modify dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by client.942868.0:18818 2014-08-11 17:06:26.908081 2014-08-13 18:06:07.003482 7f5c973a57a0 20 read_log 192888'14890 (0'0) promote 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.4.0:23170 2014-08-11 17:06:31.231601 2014-08-13 18:06:07.003494 7f5c973a57a0 20 read_log 192888'14891 (192888'14890) modify 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by client.942868.0:19964 2014-08-11 17:06:31.223850 2014-08-13 18:06:07.003506 7f5c973a57a0 20 read_log 192888'14892 (0'0) promote bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by osd.4.0:23210 2014-08-11 17:06:38.024771 2014-08-13 18:06:07.003518 7f5c973a57a0 20 read_log 192888'14893 (192888'14892) modify bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.942868.0:21744 2014-08-11 17:06:38.016662 2014-08-13 18:06:07.003531 7f5c973a57a0 20 read_log 192888'14894 (0'0) promote 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by osd.4.0:23220 2014-08-11 17:06:38.888915 2014-08-13 18:06:07.003543 7f5c973a57a0 20 read_log 192888'14895 (192888'14894) modify 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.942868.0:21976 2014-08-11 17:06:38.879263 2014-08-13 18:06:07.003555 7f5c973a57a0 20 read_log 192888'14896 (0'0) promote 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by osd.4.0:23226 2014-08-11 17:06:39.777720 2014-08-13 18:06:07.003597 7f5c973a57a0 20 read_log 192888'14897 (192888'14896) modify 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.942868.0:22211 2014-08-11 17:06:39.767946 2014-08-13 18:06:07.003615 7f5c973a57a0 20 read_log 192888'14898 (0'0) promote a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.4.0:23238 2014-08-11 17:06:41.504203 2014-08-13 18:06:07.003628 7f5c973a57a0 20 read_log 192888'14899 (192888'14898) modify a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by client.942868.0:22632 2014-08-11 17:06:41.495727 2014-08-13 18:06:07.003642 7f5c973a57a0 20 read_log 192888'14906 (0'0) promote 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by osd.4.0:23274 2014-08-11 17:06:46.280268 2014-08-13 18:06:07.003657 7f5c973a57a0 20 read_log 192888'14907 (192888'14906) modify 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.942868.0:23882 2014-08-11 17:06:46.270240 2014-08-13 18:06:07.003672 7f5c973a57a0 20 read_log 192888'14910 (0'0) promote f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.4.0:23288 2014-08-11 17:06:47.235678 2014-08-13 18:06:07.003687 7f5c973a57a0 20 read_log 192888'14911 (192888'14910) modify f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by client.942868.0:24141 2014-08-11 17:06:47.227811 2014-08-13 18:06:07.003699 7f5c973a57a0 20 read_log 192888'14912 (0'0) promote a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.4.0:23306 2014-08-11 17:06:48.863495 2014-08-13 18:06:07.003713 7f5c973a57a0 20 read_log 192888'14913 (192888'14912) modify a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by client.942868.0:24580 2014-08-11 17:06:48.854857 2014-08-13 18:06:07.003728 7f5c973a57a0 20 read_log 192888'14922 (0'0) promote 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.4.0:23426 2014-08-11 17:07:07.315042 2014-08-13 18:06:07.003740 7f5c973a57a0 20 read_log 192888'14923 (192888'14922) modify 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by client.942868.0:29389 2014-08-11 17:07:07.305968 2014-08-13 18:06:07.003755 7f5c973a57a0 20 read_log 192888'14924 (0'0) promote 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.4.0:23444 2014-08-11 17:07:10.351501 2014-08-13 18:06:07.003767 7f5c973a57a0 20 read_log 192888'14925 (192888'14924) modify 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by client.942868.0:30125 2014-08-11 17:07:10.339840 2014-08-13 18:06:07.003780 7f5c973a57a0 20 read_log 192888'14934 (192888'14817) modify 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by client.942868.0:31451 2014-08-11 17:07:15.928922 2014-08-13 18:06:07.003792 7f5c973a57a0 20 read_log 192888'14937 (192888'14823) modify 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by client.942868.0:32772 2014-08-11 17:07:16.341604 2014-08-13 18:06:07.003805 7f5c973a57a0 20 read_log 192888'14938 (192888'14825) modify e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.942868.0:34239 2014-08-11 17:07:16.734995 2014-08-13 18:06:07.003818 7f5c973a57a0 20 read_log 192888'14941 (192888'14831) modify dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by client.942868.0:35457 2014-08-11 17:07:17.078911 2014-08-13 18:06:07.003833 7f5c973a57a0 20 read_log 192888'14942 (192888'14833) modify 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by client.942868.0:35568 2014-08-11 17:07:17.106818 2014-08-13 18:06:07.003850 7f5c973a57a0 20 read_log 192888'14943 (192888'14835) modify 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.942868.0:36026 2014-08-11 17:07:17.223235 2014-08-13 18:06:07.003864 7f5c973a57a0 20 read_log 192888'14944 (192888'14837) modify e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by client.942868.0:36717 2014-08-11 17:07:17.416323 2014-08-13 18:06:07.003884 7f5c973a57a0 20 read_log 192888'14947 (192888'14843) modify 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.942868.0:37577 2014-08-11 17:07:17.650146 2014-08-13 18:06:07.003899 7f5c973a57a0 20 read_log 192888'14950 (192888'14849) modify fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.942868.0:38793 2014-08-11 17:07:17.989391 2014-08-13 18:06:07.003911 7f5c973a57a0 20 read_log 192888'14953 (192888'14855) modify 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by client.942868.0:39637 2014-08-11 17:07:18.240296 2014-08-13 18:06:07.003926 7f5c973a57a0 20 read_log 192888'14954 (192888'14857) modify 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.942868.0:40079 2014-08-11 17:07:18.359447 2014-08-13 18:06:07.003940 7f5c973a57a0 20 read_log 192888'14955 (192888'14859) modify abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by client.942868.0:40302 2014-08-11 17:07:18.414780 2014-08-13 18:06:07.003954 7f5c973a57a0 20 read_log 192888'14958 (192888'14865) modify 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.942868.0:45645 2014-08-11 17:07:19.895067 2014-08-13 18:06:07.003966 7f5c973a57a0 20 read_log 192888'14959 (192888'14867) modify 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.942868.0:46746 2014-08-11 17:07:20.207543 2014-08-13 18:06:07.003981 7f5c973a57a0 20 read_log 192888'14960 (192888'14869) modify 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.942868.0:47239 2014-08-11 17:07:20.347923 2014-08-13 18:06:07.003995 7f5c973a57a0 20 read_log 192888'14961 (192888'14871) modify f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.942868.0:47837 2014-08-11 17:07:20.536038 2014-08-13 18:06:07.004008 7f5c973a57a0 20 read_log 192888'14962 (192888'14873) modify e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.942868.0:48092 2014-08-11 17:07:20.620130 2014-08-13 18:06:07.004022 7f5c973a57a0 20 read_log 192888'14967 (192888'14883) modify dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by client.942868.0:50084 2014-08-11 17:07:21.213156 2014-08-13 18:06:07.004035 7f5c973a57a0 20 read_log 192888'14971 (192888'14891) modify 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by client.942868.0:51230 2014-08-11 17:07:21.579113 2014-08-13 18:06:07.004049 7f5c973a57a0 20 read_log 192888'14972 (192888'14893) modify bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.942868.0:53010 2014-08-11 17:07:22.121635 2014-08-13 18:06:07.004061 7f5c973a57a0 20 read_log 192888'14973 (192888'14895) modify 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.942868.0:53242 2014-08-11 17:07:22.193677 2014-08-13 18:06:07.004074 7f5c973a57a0 20 read_log 192888'14974 (192888'14897) modify 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.942868.0:53477 2014-08-11 17:07:22.276795 2014-08-13 18:06:07.004086 7f5c973a57a0 20 read_log 192888'14975 (192888'14899) modify a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by client.942868.0:53898 2014-08-11 17:07:22.410118 2014-08-13 18:06:07.004101 7f5c973a57a0 20 read_log 192888'14979 (192888'14907) modify 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.942868.0:55148 2014-08-11 17:07:22.758011 2014-08-13 18:06:07.004113 7f5c973a57a0 20 read_log 192888'14981 (192888'14911) modify f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by client.942868.0:55407 2014-08-11 17:07:22.838290 2014-08-13 18:06:07.004128 7f5c973a57a0 20 read_log 192888'14982 (192888'14913) modify a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by client.942868.0:55846 2014-08-11 17:07:22.974069 2014-08-13 18:06:07.004141 7f5c973a57a0 20 read_log 192888'14987 (192888'14923) modify 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by client.942868.0:60655 2014-08-11 17:07:24.406260 2014-08-13 18:06:07.004168 7f5c973a57a0 20 read_log 192888'14988 (192888'14925) modify 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by client.942868.0:61391 2014-08-11 17:07:24.626766 2014-08-13 18:06:07.004184 7f5c973a57a0 20 read_log 193305'14993 (0'0) promote 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by osd.330.0:15 2014-08-11 18:42:35.340522 2014-08-13 18:06:07.004196 7f5c973a57a0 20 read_log 193305'14994 (193305'14993) modify 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.951337.0:563 2014-08-11 18:42:35.331304 2014-08-13 18:06:07.004209 7f5c973a57a0 20 read_log 193305'14995 (0'0) promote 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:21 2014-08-11 18:42:36.670029 2014-08-13 18:06:07.004221 7f5c973a57a0 20 read_log 193305'14996 (193305'14995) modify 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by client.951337.0:916 2014-08-11 18:42:36.663123 2014-08-13 18:06:07.004236 7f5c973a57a0 20 read_log 193305'14997 (0'0) promote 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:23 2014-08-11 18:42:36.693711 2014-08-13 18:06:07.004248 7f5c973a57a0 20 read_log 193305'14998 (193305'14997) modify 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by client.951337.0:923 2014-08-11 18:42:36.687744 2014-08-13 18:06:07.004268 7f5c973a57a0 20 read_log 193305'14999 (0'0) promote f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:25 2014-08-11 18:42:38.375226 2014-08-13 18:06:07.004281 7f5c973a57a0 20 read_log 193305'15000 (193305'14999) modify f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by client.951337.0:1387 2014-08-11 18:42:38.367811 2014-08-13 18:06:07.004296 7f5c973a57a0 20 read_log 193305'15001 (0'0) promote d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by osd.330.0:43 2014-08-11 18:42:41.192695 2014-08-13 18:06:07.004311 7f5c973a57a0 20 read_log 193305'15002 (193305'15001) modify d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.951337.0:2119 2014-08-11 18:42:41.185186 2014-08-13 18:06:07.004326 7f5c973a57a0 20 read_log 193305'15003 (0'0) promote 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:57 2014-08-11 18:42:45.914253 2014-08-13 18:06:07.004338 7f5c973a57a0 20 read_log 193305'15004 (193305'15003) modify 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by client.951337.0:3417 2014-08-11 18:42:45.902501 2014-08-13 18:06:07.004351 7f5c973a57a0 20 read_log 193305'15005 (0'0) promote d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:59 2014-08-11 18:42:45.993795 2014-08-13 18:06:07.004363 7f5c973a57a0 20 read_log 193305'15006 (193305'15005) modify d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by client.951337.0:3439 2014-08-11 18:42:45.983241 2014-08-13 18:06:07.004382 7f5c973a57a0 20 read_log 193305'15007 (0'0) promote f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by osd.330.0:65 2014-08-11 18:42:46.530813 2014-08-13 18:06:07.004413 7f5c973a57a0 20 read_log 193305'15008 (193305'15007) modify f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.951337.0:3590 2014-08-11 18:42:46.523695 2014-08-13 18:06:07.004442 7f5c973a57a0 20 read_log 193305'15009 (0'0) promote eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by osd.330.0:95 2014-08-11 18:42:53.606651 2014-08-13 18:06:07.004458 7f5c973a57a0 20 read_log 193305'15010 (193305'15009) modify eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.951337.0:5500 2014-08-11 18:42:53.599454 2014-08-13 18:06:07.004478 7f5c973a57a0 20 read_log 193305'15011 (0'0) promote 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by osd.330.0:101 2014-08-11 18:42:55.925446 2014-08-13 18:06:07.004504 7f5c973a57a0 20 read_log 193305'15012 (193305'15011) modify 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.951337.0:6137 2014-08-11 18:42:55.918404 2014-08-13 18:06:07.004529 7f5c973a57a0 20 read_log 193305'15013 (0'0) promote 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:109 2014-08-11 18:42:58.182279 2014-08-13 18:06:07.004545 7f5c973a57a0 20 read_log 193305'15014 (193305'15013) modify 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by client.951337.0:6761 2014-08-11 18:42:58.175495 2014-08-13 18:06:07.004558 7f5c973a57a0 20 read_log 193305'15015 (0'0) promote ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by osd.330.0:177 2014-08-11 18:43:10.535781 2014-08-13 18:06:07.004578 7f5c973a57a0 20 read_log 193305'15016 (193305'15015) modify ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.951337.0:10067 2014-08-11 18:43:10.528566 2014-08-13 18:06:07.004594 7f5c973a57a0 20 read_log 193305'15017 (0'0) promote a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by osd.330.0:248 2014-08-11 18:43:26.437496 2014-08-13 18:06:07.004606 7f5c973a57a0 20 read_log 193305'15018 (193305'15017) modify a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.951337.0:14260 2014-08-11 18:43:26.426670 2014-08-13 18:06:07.004619 7f5c973a57a0 20 read_log 193305'15019 (0'0) promote af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by osd.330.0:250 2014-08-11 18:43:26.827260 2014-08-13 18:06:07.004635 7f5c973a57a0 20 read_log 193305'15020 (193305'15019) modify af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.951337.0:14379 2014-08-11 18:43:26.818495 2014-08-13 18:06:07.004649 7f5c973a57a0 20 read_log 193305'15021 (0'0) promote 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:254 2014-08-11 18:43:27.247088 2014-08-13 18:06:07.004661 7f5c973a57a0 20 read_log 193305'15022 (193305'15021) modify 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by client.951337.0:14507 2014-08-11 18:43:27.240157 2014-08-13 18:06:07.004673 7f5c973a57a0 20 read_log 193305'15023 (0'0) promote 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:258 2014-08-11 18:43:27.394066 2014-08-13 18:06:07.004686 7f5c973a57a0 20 read_log 193305'15024 (193305'15023) modify 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by client.951337.0:14552 2014-08-11 18:43:27.387768 2014-08-13 18:06:07.004698 7f5c973a57a0 20 read_log 193305'15025 (0'0) promote 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by osd.330.0:280 2014-08-11 18:43:32.332630 2014-08-13 18:06:07.004710 7f5c973a57a0 20 read_log 193305'15026 (193305'15025) modify 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.951337.0:15984 2014-08-11 18:43:32.324039 2014-08-13 18:06:07.004722 7f5c973a57a0 20 read_log 193305'15027 (0'0) promote e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by osd.330.0:290 2014-08-11 18:43:33.646698 2014-08-13 18:06:07.004734 7f5c973a57a0 20 read_log 193305'15028 (193305'15027) modify e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.951337.0:16339 2014-08-11 18:43:33.637715 2014-08-13 18:06:07.004747 7f5c973a57a0 20 read_log 193305'15029 (0'0) promote 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:296 2014-08-11 18:43:34.896768 2014-08-13 18:06:07.004759 7f5c973a57a0 20 read_log 193305'15030 (193305'15029) modify 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by client.951337.0:16712 2014-08-11 18:43:34.890803 2014-08-13 18:06:07.004772 7f5c973a57a0 20 read_log 193305'15031 (0'0) promote 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:316 2014-08-11 18:43:39.256170 2014-08-13 18:06:07.004791 7f5c973a57a0 20 read_log 193305'15032 (193305'15031) modify 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by client.951337.0:17961 2014-08-11 18:43:39.248285 2014-08-13 18:06:07.004816 7f5c973a57a0 20 read_log 193305'15033 (0'0) promote c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:330 2014-08-11 18:43:41.319741 2014-08-13 18:06:07.004829 7f5c973a57a0 20 read_log 193305'15034 (193305'15033) modify c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by client.951337.0:18556 2014-08-11 18:43:41.313165 2014-08-13 18:06:07.004841 7f5c973a57a0 20 read_log 193305'15035 (0'0) promote f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by osd.330.0:346 2014-08-11 18:43:45.432300 2014-08-13 18:06:07.004853 7f5c973a57a0 20 read_log 193305'15036 (193305'15035) modify f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.951337.0:19664 2014-08-11 18:43:45.424264 2014-08-13 18:06:07.004872 7f5c973a57a0 20 read_log 193305'15037 (0'0) promote d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:348 2014-08-11 18:43:45.771971 2014-08-13 18:06:07.004884 7f5c973a57a0 20 read_log 193305'15038 (193305'15037) modify d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by client.951337.0:19761 2014-08-11 18:43:45.762916 2014-08-13 18:06:07.004897 7f5c973a57a0 20 read_log 193305'15039 (0'0) promote ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by osd.330.0:354 2014-08-11 18:43:48.032963 2014-08-13 18:06:07.004909 7f5c973a57a0 20 read_log 193305'15040 (193305'15039) modify ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.951337.0:20381 2014-08-11 18:43:48.025253 2014-08-13 18:06:07.004921 7f5c973a57a0 20 read_log 193305'15041 (0'0) promote cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:356 2014-08-11 18:43:48.660399 2014-08-13 18:06:07.004936 7f5c973a57a0 20 read_log 193305'15042 (193305'15041) modify cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by client.951337.0:20560 2014-08-11 18:43:48.648947 2014-08-13 18:06:07.004949 7f5c973a57a0 20 read_log 193305'15043 (0'0) promote 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by osd.330.0:380 2014-08-11 18:43:55.986416 2014-08-13 18:06:07.004961 7f5c973a57a0 20 read_log 193305'15044 (193305'15043) modify 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.951337.0:22593 2014-08-11 18:43:55.979183 2014-08-13 18:06:07.004973 7f5c973a57a0 20 read_log 193305'15045 (0'0) promote e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by osd.330.0:412 2014-08-11 18:44:02.715279 2014-08-13 18:06:07.004988 7f5c973a57a0 20 read_log 193305'15046 (193305'15045) modify e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.951337.0:24468 2014-08-11 18:44:02.705878 2014-08-13 18:06:07.005000 7f5c973a57a0 20 read_log 193305'15047 (0'0) promote e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by osd.330.0:414 2014-08-11 18:44:03.092933 2014-08-13 18:06:07.005014 7f5c973a57a0 20 read_log 193305'15048 (193305'15047) modify e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.951337.0:24570 2014-08-11 18:44:03.084880 2014-08-13 18:06:07.005027 7f5c973a57a0 20 read_log 193305'15049 (0'0) promote cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by osd.330.0:434 2014-08-11 18:44:07.569587 2014-08-13 18:06:07.005044 7f5c973a57a0 20 read_log 193305'15050 (193305'15049) modify cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.951337.0:25786 2014-08-11 18:44:07.562019 2014-08-13 18:06:07.005058 7f5c973a57a0 20 read_log 193305'15051 (0'0) promote 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by osd.330.0:436 2014-08-11 18:44:07.578968 2014-08-13 18:06:07.005078 7f5c973a57a0 20 read_log 193305'15052 (193305'15051) modify 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.951337.0:25787 2014-08-11 18:44:07.566360 2014-08-13 18:06:07.005093 7f5c973a57a0 20 read_log 193305'15053 (0'0) promote 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by osd.330.0:442 2014-08-11 18:44:08.698966 2014-08-13 18:06:07.005106 7f5c973a57a0 20 read_log 193305'15054 (193305'15053) modify 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.951337.0:26080 2014-08-11 18:44:08.690787 2014-08-13 18:06:07.005121 7f5c973a57a0 20 read_log 193305'15055 (0'0) promote 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by osd.330.0:458 2014-08-11 18:44:11.600308 2014-08-13 18:06:07.005133 7f5c973a57a0 20 read_log 193305'15056 (193305'15055) modify 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.951337.0:26893 2014-08-11 18:44:11.591770 2014-08-13 18:06:07.005152 7f5c973a57a0 20 read_log 193305'15057 (0'0) promote 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by osd.330.0:460 2014-08-11 18:44:11.778828 2014-08-13 18:06:07.005167 7f5c973a57a0 20 read_log 193305'15058 (193305'15057) modify 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.951337.0:26935 2014-08-11 18:44:11.770765 2014-08-13 18:06:07.005180 7f5c973a57a0 20 read_log 193305'15059 (0'0) promote ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:466 2014-08-11 18:44:12.685596 2014-08-13 18:06:07.005194 7f5c973a57a0 20 read_log 193305'15060 (193305'15059) modify ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by client.951337.0:27171 2014-08-11 18:44:12.679213 2014-08-13 18:06:07.005207 7f5c973a57a0 20 read_log 193305'15061 (193305'15061) modify 3f7/hit_set_24.3f7_archive_2014-08-11 18:39:14.563860_2014-08-11 18:44:18.014758/head/.ceph-internal/24 by unknown.0.0:0 2014-08-11 18:44:18.015046 2014-08-13 18:06:07.005220 7f5c973a57a0 20 read_log 193305'15062 (0'0) promote 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:477 2014-08-11 18:44:18.018055 2014-08-13 18:06:07.005232 7f5c973a57a0 20 read_log 193305'15063 (193305'15062) modify 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by client.951337.0:28650 2014-08-11 18:44:18.009271 2014-08-13 18:06:07.005245 7f5c973a57a0 20 read_log 193305'15064 (0'0) promote ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:481 2014-08-11 18:44:18.373399 2014-08-13 18:06:07.005260 7f5c973a57a0 20 read_log 193305'15065 (193305'15064) modify ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by client.951337.0:28745 2014-08-11 18:44:18.365716 2014-08-13 18:06:07.005272 7f5c973a57a0 20 read_log 193305'15066 (0'0) promote fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by osd.330.0:491 2014-08-11 18:44:19.770796 2014-08-13 18:06:07.005285 7f5c973a57a0 20 read_log 193305'15067 (193305'15066) modify fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.951337.0:29117 2014-08-11 18:44:19.763161 2014-08-13 18:06:07.005297 7f5c973a57a0 20 read_log 193305'15068 (0'0) promote dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by osd.330.0:501 2014-08-11 18:44:21.006648 2014-08-13 18:06:07.005309 7f5c973a57a0 20 read_log 193305'15069 (193305'15068) modify dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.951337.0:29455 2014-08-11 18:44:20.999706 2014-08-13 18:06:07.005322 7f5c973a57a0 20 read_log 193305'15070 (0'0) promote 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by osd.330.0:503 2014-08-11 18:44:22.424664 2014-08-13 18:06:07.005347 7f5c973a57a0 20 read_log 193305'15071 (193305'15070) modify 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.951337.0:29841 2014-08-11 18:44:22.413648 2014-08-13 18:06:07.005363 7f5c973a57a0 20 read_log 193305'15072 (0'0) promote 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by osd.330.0:543 2014-08-11 18:44:32.651066 2014-08-13 18:06:07.005376 7f5c973a57a0 20 read_log 193305'15073 (193305'15072) modify 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.951337.0:32663 2014-08-11 18:44:32.644122 2014-08-13 18:06:07.005405 7f5c973a57a0 20 read_log 193305'15074 (193305'14994) modify 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.951337.0:33274 2014-08-11 18:44:34.291001 2014-08-13 18:06:07.005437 7f5c973a57a0 20 read_log 193305'15075 (193305'14996) modify 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by client.951337.0:33627 2014-08-11 18:44:34.543969 2014-08-13 18:06:07.005463 7f5c973a57a0 20 read_log 193305'15076 (193305'14998) modify 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by client.951337.0:33634 2014-08-11 18:44:34.547736 2014-08-13 18:06:07.005477 7f5c973a57a0 20 read_log 193305'15077 (193305'15000) modify f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by client.951337.0:34098 2014-08-11 18:44:34.794341 2014-08-13 18:06:07.005489 7f5c973a57a0 20 read_log 193305'15078 (193305'15002) modify d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.951337.0:34830 2014-08-11 18:44:35.212586 2014-08-13 18:06:07.005514 7f5c973a57a0 20 read_log 193305'15079 (193305'15004) modify 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by client.951337.0:36128 2014-08-11 18:44:35.949615 2014-08-13 18:06:07.005528 7f5c973a57a0 20 read_log 193305'15080 (193305'15006) modify d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by client.951337.0:36150 2014-08-11 18:44:35.957419 2014-08-13 18:06:07.005540 7f5c973a57a0 20 read_log 193305'15081 (193305'15008) modify f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.951337.0:36301 2014-08-11 18:44:36.122344 2014-08-13 18:06:07.005553 7f5c973a57a0 20 read_log 193305'15082 (193305'15010) modify eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.951337.0:38211 2014-08-11 18:44:37.288830 2014-08-13 18:06:07.005565 7f5c973a57a0 20 read_log 193305'15083 (193305'15012) modify 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.951337.0:38848 2014-08-11 18:44:37.689692 2014-08-13 18:06:07.005577 7f5c973a57a0 20 read_log 193305'15084 (193305'15014) modify 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by client.951337.0:39472 2014-08-11 18:44:38.030317 2014-08-13 18:06:07.005589 7f5c973a57a0 20 read_log 193305'15085 (193305'15016) modify ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.951337.0:42778 2014-08-11 18:44:39.377246 2014-08-13 18:06:07.005601 7f5c973a57a0 20 read_log 193305'15086 (193305'15018) modify a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.951337.0:46971 2014-08-11 18:44:40.639963 2014-08-13 18:06:07.005614 7f5c973a57a0 20 read_log 193305'15087 (193305'15020) modify af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.951337.0:47090 2014-08-11 18:44:40.671435 2014-08-13 18:06:07.005626 7f5c973a57a0 20 read_log 193305'15088 (193305'15022) modify 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by client.951337.0:47218 2014-08-11 18:44:40.704907 2014-08-13 18:06:07.005638 7f5c973a57a0 20 read_log 193305'15089 (193305'15024) modify 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by client.951337.0:47263 2014-08-11 18:44:40.719186 2014-08-13 18:06:07.005650 7f5c973a57a0 20 read_log 193305'15090 (193305'15026) modify 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.951337.0:48695 2014-08-11 18:44:41.129240 2014-08-13 18:06:07.005686 7f5c973a57a0 20 read_log 193305'15091 (193305'15028) modify e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.951337.0:49050 2014-08-11 18:44:41.237018 2014-08-13 18:06:07.005699 7f5c973a57a0 20 read_log 193305'15092 (193305'15030) modify 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by client.951337.0:49423 2014-08-11 18:44:41.361921 2014-08-13 18:06:07.005712 7f5c973a57a0 20 read_log 193305'15093 (193305'15032) modify 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by client.951337.0:50672 2014-08-11 18:44:41.731056 2014-08-13 18:06:07.005724 7f5c973a57a0 20 read_log 193305'15094 (193305'15034) modify c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by client.951337.0:51267 2014-08-11 18:44:41.902985 2014-08-13 18:06:07.005738 7f5c973a57a0 20 read_log 193305'15095 (193305'15036) modify f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.951337.0:52375 2014-08-11 18:44:42.216205 2014-08-13 18:06:07.005760 7f5c973a57a0 20 read_log 193305'15096 (193305'15038) modify d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by client.951337.0:52472 2014-08-11 18:44:42.243173 2014-08-13 18:06:07.005773 7f5c973a57a0 20 read_log 193305'15097 (193305'15040) modify ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.951337.0:53092 2014-08-11 18:44:42.426736 2014-08-13 18:06:07.005785 7f5c973a57a0 20 read_log 193305'15098 (193305'15042) modify cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by client.951337.0:53271 2014-08-11 18:44:42.480082 2014-08-13 18:06:07.005798 7f5c973a57a0 20 read_log 193305'15099 (193305'15044) modify 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.951337.0:55304 2014-08-11 18:44:43.083333 2014-08-13 18:06:07.005810 7f5c973a57a0 20 read_log 193305'15100 (193305'15046) modify e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.951337.0:57179 2014-08-11 18:44:43.660596 2014-08-13 18:06:07.005845 7f5c973a57a0 20 read_log 193305'15101 (193305'15048) modify e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.951337.0:57281 2014-08-11 18:44:43.700911 2014-08-13 18:06:07.005858 7f5c973a57a0 20 read_log 193305'15102 (193305'15050) modify cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.951337.0:58497 2014-08-11 18:44:44.059674 2014-08-13 18:06:07.005871 7f5c973a57a0 20 read_log 193305'15103 (193305'15052) modify 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.951337.0:58498 2014-08-11 18:44:44.059768 2014-08-13 18:06:07.005883 7f5c973a57a0 20 read_log 193305'15104 (193305'15054) modify 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.951337.0:58791 2014-08-11 18:44:44.142842 2014-08-13 18:06:07.005896 7f5c973a57a0 20 read_log 193305'15105 (193305'15056) modify 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.951337.0:59604 2014-08-11 18:44:44.374900 2014-08-13 18:06:07.005908 7f5c973a57a0 20 read_log 193305'15106 (193305'15058) modify 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.951337.0:59646 2014-08-11 18:44:44.385389 2014-08-13 18:06:07.005920 7f5c973a57a0 20 read_log 193305'15107 (193305'15060) modify ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by client.951337.0:59882 2014-08-11 18:44:44.454166 2014-08-13 18:06:07.005933 7f5c973a57a0 20 read_log 193305'15108 (193305'15063) modify 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by client.951337.0:61361 2014-08-11 18:44:44.901608 2014-08-13 18:06:07.005945 7f5c973a57a0 20 read_log 193305'15109 (193305'15065) modify ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by client.951337.0:61456 2014-08-11 18:44:44.926806 2014-08-13 18:06:07.005966 7f5c973a57a0 20 read_log 193305'15110 (193305'15067) modify fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.951337.0:61828 2014-08-11 18:44:45.043875 2014-08-13 18:06:07.005988 7f5c973a57a0 20 read_log 193305'15111 (193305'15069) modify dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.951337.0:62166 2014-08-11 18:44:45.144826 2014-08-13 18:06:07.006023 7f5c973a57a0 20 read_log 193305'15112 (193305'15071) modify 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.951337.0:62552 2014-08-11 18:44:45.261257 2014-08-13 18:06:07.006037 7f5c973a57a0 20 read_log 193305'15113 (193305'15073) modify 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.951337.0:65374 2014-08-11 18:44:46.106070 2014-08-13 18:06:07.006057 7f5c973a57a0 20 read_log 194209'15114 (192888'14942) clean 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.330.0:971 2014-08-12 14:05:29.932182 2014-08-13 18:06:07.006073 7f5c973a57a0 20 read_log 194209'15115 (192888'14967) clean dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.330.0:972 2014-08-12 14:05:29.932576 2014-08-13 18:06:07.006086 7f5c973a57a0 20 read_log 194209'15116 (193305'15076) clean 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:973 2014-08-12 14:05:29.932961 2014-08-13 18:06:07.006111 7f5c973a57a0 20 read_log 194209'15117 (192888'14987) clean 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.330.0:974 2014-08-12 14:05:29.933417 2014-08-13 18:06:07.006125 7f5c973a57a0 20 read_log 194209'15118 (192888'14944) clean e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.330.0:975 2014-08-12 14:05:29.957206 2014-08-13 18:06:07.006137 7f5c973a57a0 20 read_log 194209'15119 (192834'14018) clean 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.330.0:976 2014-08-12 14:05:29.960577 2014-08-13 18:06:07.006158 7f5c973a57a0 20 read_log 194209'15120 (192834'14037) clean 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.330.0:977 2014-08-12 14:05:29.963636 2014-08-13 18:06:07.006174 7f5c973a57a0 20 read_log 194209'15121 (193305'15108) clean 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:978 2014-08-12 14:05:29.967008 2014-08-13 18:06:07.006187 7f5c973a57a0 20 read_log 194209'15122 (193305'15098) clean cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:979 2014-08-12 14:05:29.970238 2014-08-13 18:06:07.006199 7f5c973a57a0 20 read_log 194209'15123 (192888'14941) clean dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.330.0:980 2014-08-12 14:05:29.973565 2014-08-13 18:06:07.006211 7f5c973a57a0 20 read_log 194209'15124 (192888'14988) clean 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.330.0:981 2014-08-12 14:05:29.976777 2014-08-13 18:06:07.006226 7f5c973a57a0 20 read_log 194209'15125 (193305'15096) clean d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:982 2014-08-12 14:05:29.979864 2014-08-13 18:06:07.006239 7f5c973a57a0 20 read_log 194209'15126 (192888'14781) clean 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.330.0:983 2014-08-12 14:05:29.982946 2014-08-13 18:06:07.006251 7f5c973a57a0 20 read_log 194209'15127 (192888'14975) clean a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.330.0:984 2014-08-12 14:05:29.986129 2014-08-13 18:06:07.006264 7f5c973a57a0 20 read_log 194209'15128 (192888'14792) clean 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.330.0:985 2014-08-12 14:05:29.989173 2014-08-13 18:06:07.006276 7f5c973a57a0 20 read_log 194209'15129 (192888'14761) clean a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.330.0:986 2014-08-12 14:05:29.994944 2014-08-13 18:06:07.006293 7f5c973a57a0 20 read_log 194209'15130 (193305'15109) clean ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:987 2014-08-12 14:05:29.997864 2014-08-13 18:06:07.006338 7f5c973a57a0 20 read_log 194209'15131 (193305'15079) clean 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:988 2014-08-12 14:05:29.998164 2014-08-13 18:06:07.006355 7f5c973a57a0 20 read_log 194209'15132 (192834'14102) clean fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.330.0:989 2014-08-12 14:05:30.001608 2014-08-13 18:06:07.006368 7f5c973a57a0 20 read_log 194209'15133 (192888'14981) clean f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.330.0:990 2014-08-12 14:05:30.005109 2014-08-13 18:06:07.006384 7f5c973a57a0 20 read_log 194209'15134 (192834'14090) clean daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by osd.330.0:991 2014-08-12 14:05:30.010564 2014-08-13 18:06:07.006405 7f5c973a57a0 20 read_log 194209'15135 (192888'14802) clean 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by osd.330.0:992 2014-08-12 14:05:30.010855 2014-08-13 18:06:07.006421 7f5c973a57a0 20 read_log 194209'15136 (193305'15093) clean 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:993 2014-08-12 14:05:30.014085 2014-08-13 18:06:07.006434 7f5c973a57a0 20 read_log 194209'15137 (193305'15075) clean 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:994 2014-08-12 14:05:30.017291 2014-08-13 18:06:07.006447 7f5c973a57a0 20 read_log 194209'15138 (192834'14617) clean 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.330.0:995 2014-08-12 14:05:30.020499 2014-08-13 18:06:07.006459 7f5c973a57a0 20 read_log 194209'15139 (192834'14100) clean d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.330.0:996 2014-08-12 14:05:30.023614 2014-08-13 18:06:07.006472 7f5c973a57a0 20 read_log 194209'15140 (192888'14971) clean 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.330.0:997 2014-08-12 14:05:30.026735 2014-08-13 18:06:07.006484 7f5c973a57a0 20 read_log 194209'15141 (192834'14116) clean fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.330.0:998 2014-08-12 14:05:30.029882 2014-08-13 18:06:07.006497 7f5c973a57a0 20 read_log 194209'15142 (193305'15088) clean 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:999 2014-08-12 14:05:30.033404 2014-08-13 18:06:07.006509 7f5c973a57a0 20 read_log 194209'15143 (192888'14953) clean 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.330.0:1000 2014-08-12 14:05:30.036995 2014-08-13 18:06:07.006522 7f5c973a57a0 20 read_log 194209'15144 (192834'14032) clean dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.330.0:1001 2014-08-12 14:05:30.040476 2014-08-13 18:06:07.006534 7f5c973a57a0 20 read_log 194209'15145 (193305'15094) clean c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:1002 2014-08-12 14:05:30.044185 2014-08-13 18:06:07.006547 7f5c973a57a0 20 read_log 194209'15146 (192834'14020) clean f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.330.0:1003 2014-08-12 14:05:30.047754 2014-08-13 18:06:07.006570 7f5c973a57a0 20 read_log 194209'15147 (193305'15080) clean d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:1004 2014-08-12 14:05:30.051444 2014-08-13 18:06:07.006584 7f5c973a57a0 20 read_log 194209'15148 (192834'14036) clean d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.330.0:1005 2014-08-12 14:05:30.055229 2014-08-13 18:06:07.006596 7f5c973a57a0 20 read_log 194209'15149 (192834'14088) clean 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.330.0:1006 2014-08-12 14:05:30.058559 2014-08-13 18:06:07.006609 7f5c973a57a0 20 read_log 194209'15150 (193305'15077) clean f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:1007 2014-08-12 14:05:30.062091 2014-08-13 18:06:07.006630 7f5c973a57a0 20 read_log 194209'15151 (192834'14049) clean 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.330.0:1008 2014-08-12 14:05:30.065522 2014-08-13 18:06:07.006645 7f5c973a57a0 20 read_log 194209'15152 (192888'14955) clean abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.330.0:1009 2014-08-12 14:05:30.069056 2014-08-13 18:06:07.006658 7f5c973a57a0 20 read_log 194209'15153 (192834'14057) clean 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.330.0:1010 2014-08-12 14:05:30.072596 2014-08-13 18:06:07.006670 7f5c973a57a0 20 read_log 194209'15154 (193305'15089) clean 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:1011 2014-08-12 14:05:30.075934 2014-08-13 18:06:07.006683 7f5c973a57a0 20 read_log 194209'15155 (192888'14790) clean fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.330.0:1012 2014-08-12 14:05:30.079497 2014-08-13 18:06:07.006695 7f5c973a57a0 20 read_log 194209'15156 (192888'14793) clean f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.330.0:1013 2014-08-12 14:05:30.083011 2014-08-13 18:06:07.006707 7f5c973a57a0 20 read_log 194209'15157 (192888'14937) clean 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.330.0:1014 2014-08-12 14:05:30.086492 2014-08-13 18:06:07.006719 7f5c973a57a0 20 read_log 194209'15158 (193305'15092) clean 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:1015 2014-08-12 14:05:30.089987 2014-08-13 18:06:07.006732 7f5c973a57a0 20 read_log 194209'15159 (192834'14639) clean 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.330.0:1016 2014-08-12 14:05:30.093830 2014-08-13 18:06:07.006744 7f5c973a57a0 20 read_log 194209'15160 (193305'15107) clean ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:1017 2014-08-12 14:05:30.097320 2014-08-13 18:06:07.006756 7f5c973a57a0 20 read_log 194209'15161 (192888'14982) clean a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.330.0:1018 2014-08-12 14:05:30.100814 2014-08-13 18:06:07.006769 7f5c973a57a0 20 read_log 194209'15162 (192888'14934) clean 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.330.0:1019 2014-08-12 14:05:30.104306 2014-08-13 18:06:07.006781 7f5c973a57a0 20 read_log 194209'15163 (194209'15114) delete 5e4a2ff7/benchmark_data_storage0111-ib.pocstor_2228_object4301/head//24 by osd.330.0:1020 2014-08-12 14:05:30.141397 2014-08-13 18:06:07.006793 7f5c973a57a0 20 read_log 194209'15164 (194209'15115) delete dab23ff7/benchmark_data_storage0111-ib.pocstor_2228_object18817/head//24 by osd.330.0:1021 2014-08-12 14:05:30.144832 2014-08-13 18:06:07.006806 7f5c973a57a0 20 read_log 194209'15165 (194209'15116) delete 943f3ff7/benchmark_data_storage0111-ib.pocstor_62621_object922/head//24 by osd.330.0:1022 2014-08-12 14:05:30.148145 2014-08-13 18:06:07.006818 7f5c973a57a0 20 read_log 194209'15166 (194209'15121) delete 47b47ff7/benchmark_data_storage0111-ib.pocstor_62621_object28649/head//24 by osd.330.0:1023 2014-08-12 14:05:30.151833 2014-08-13 18:06:07.006830 7f5c973a57a0 20 read_log 194209'15167 (193305'15084) clean 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:1024 2014-08-12 14:05:30.152370 2014-08-13 18:06:07.006843 7f5c973a57a0 20 read_log 194209'15168 (194209'15122) delete cb47ff7/benchmark_data_storage0111-ib.pocstor_62621_object20559/head//24 by osd.330.0:1025 2014-08-12 14:05:30.155531 2014-08-13 18:06:07.006857 7f5c973a57a0 20 read_log 194209'15169 (194209'15125) delete d2a87ff7/benchmark_data_storage0111-ib.pocstor_62621_object19760/head//24 by osd.330.0:1026 2014-08-12 14:05:30.158555 2014-08-13 18:06:07.006870 7f5c973a57a0 20 read_log 194209'15170 (194209'15130) delete ba148ff7/benchmark_data_storage0111-ib.pocstor_62621_object28744/head//24 by osd.330.0:1027 2014-08-12 14:05:30.159815 2014-08-13 18:06:07.006891 7f5c973a57a0 20 read_log 194209'15171 (194209'15131) delete 4de58ff7/benchmark_data_storage0111-ib.pocstor_62621_object3416/head//24 by osd.330.0:1028 2014-08-12 14:05:30.162837 2014-08-13 18:06:07.006906 7f5c973a57a0 20 read_log 194209'15172 (194209'15136) delete 3a21aff7/benchmark_data_storage0111-ib.pocstor_62621_object17960/head//24 by osd.330.0:1029 2014-08-12 14:05:30.164120 2014-08-13 18:06:07.006919 7f5c973a57a0 20 read_log 194209'15173 (194209'15137) delete 8e2baff7/benchmark_data_storage0111-ib.pocstor_62621_object915/head//24 by osd.330.0:1030 2014-08-12 14:05:30.167091 2014-08-13 18:06:07.006933 7f5c973a57a0 20 read_log 194209'15174 (194209'15142) delete 7cb7fff7/benchmark_data_storage0111-ib.pocstor_62621_object14506/head//24 by osd.330.0:1031 2014-08-12 14:05:30.168356 2014-08-13 18:06:07.006946 7f5c973a57a0 20 read_log 194209'15175 (194209'15145) delete c5e713f7/benchmark_data_storage0111-ib.pocstor_62621_object18555/head//24 by osd.330.0:1032 2014-08-12 14:05:30.171999 2014-08-13 18:06:07.006960 7f5c973a57a0 20 read_log 194209'15176 (194209'15147) delete d21233f7/benchmark_data_storage0111-ib.pocstor_62621_object3438/head//24 by osd.330.0:1033 2014-08-12 14:05:30.172955 2014-08-13 18:06:07.006973 7f5c973a57a0 20 read_log 194209'15177 (194209'15150) delete f8c443f7/benchmark_data_storage0111-ib.pocstor_62621_object1386/head//24 by osd.330.0:1034 2014-08-12 14:05:30.177253 2014-08-13 18:06:07.006987 7f5c973a57a0 20 read_log 194209'15178 (194209'15154) delete 3af253f7/benchmark_data_storage0111-ib.pocstor_62621_object14551/head//24 by osd.330.0:1035 2014-08-12 14:05:30.178392 2014-08-13 18:06:07.007001 7f5c973a57a0 20 read_log 194209'15179 (194209'15158) delete 45bf63f7/benchmark_data_storage0111-ib.pocstor_62621_object16711/head//24 by osd.330.0:1036 2014-08-12 14:05:30.182239 2014-08-13 18:06:07.007016 7f5c973a57a0 20 read_log 194209'15180 (194209'15160) delete ca4273f7/benchmark_data_storage0111-ib.pocstor_62621_object27170/head//24 by osd.330.0:1037 2014-08-12 14:05:30.182669 2014-08-13 18:06:07.007028 7f5c973a57a0 20 read_log 194209'15181 (194209'15167) delete 6a2d73f7/benchmark_data_storage0111-ib.pocstor_62621_object6760/head//24 by osd.330.0:1038 2014-08-12 14:05:30.185713 2014-08-13 18:06:07.007041 7f5c973a57a0 20 read_log 194209'15182 (194209'15138) delete 212eaff7/benchmark_data_storage0111-ib.pocstor_41526_object18314/head//24 by osd.330.0:1195 2014-08-12 14:05:45.334872 2014-08-13 18:06:07.007053 7f5c973a57a0 20 read_log 194209'15183 (194209'15139) delete d2e5bff7/benchmark_data_storage0111-ib.pocstor_26659_object17708/head//24 by osd.330.0:1196 2014-08-12 14:05:45.335300 2014-08-13 18:06:07.007068 7f5c973a57a0 20 read_log 194209'15184 (194209'15140) delete 1760dff7/benchmark_data_storage0111-ib.pocstor_2228_object19963/head//24 by osd.330.0:1197 2014-08-12 14:05:45.335839 2014-08-13 18:06:07.007080 7f5c973a57a0 20 read_log 194209'15185 (194209'15141) delete fc8beff7/benchmark_data_storage0111-ib.pocstor_26659_object19538/head//24 by osd.330.0:1198 2014-08-12 14:05:45.336770 2014-08-13 18:06:07.007095 7f5c973a57a0 20 read_log 194209'15186 (194209'15143) delete 253f03f7/benchmark_data_storage0111-ib.pocstor_2228_object8370/head//24 by osd.330.0:1199 2014-08-12 14:05:45.340138 2014-08-13 18:06:07.007108 7f5c973a57a0 20 read_log 194209'15187 (194209'15144) delete dad413f7/benchmark_data_storage0111-ib.pocstor_26659_object9158/head//24 by osd.330.0:1200 2014-08-12 14:05:45.340503 2014-08-13 18:06:07.007122 7f5c973a57a0 20 read_log 194209'15188 (194209'15146) delete f0c133f7/benchmark_data_storage0111-ib.pocstor_26659_object7889/head//24 by osd.330.0:1201 2014-08-12 14:05:45.341330 2014-08-13 18:06:07.007135 7f5c973a57a0 20 read_log 194209'15189 (194209'15148) delete d25433f7/benchmark_data_storage0111-ib.pocstor_26659_object9517/head//24 by osd.330.0:1202 2014-08-12 14:05:45.341690 2014-08-13 18:06:07.007154 7f5c973a57a0 20 read_log 194209'15190 (194209'15149) delete 15a143f7/benchmark_data_storage0111-ib.pocstor_26659_object15549/head//24 by osd.330.0:1203 2014-08-12 14:05:45.345275 2014-08-13 18:06:07.007185 7f5c973a57a0 20 read_log 194209'15191 (194209'15151) delete 87e843f7/benchmark_data_storage0111-ib.pocstor_26659_object11792/head//24 by osd.330.0:1204 2014-08-12 14:05:45.345671 2014-08-13 18:06:07.007201 7f5c973a57a0 20 read_log 194209'15192 (194209'15152) delete abbe43f7/benchmark_data_storage0111-ib.pocstor_2228_object9035/head//24 by osd.330.0:1205 2014-08-12 14:05:45.346304 2014-08-13 18:06:07.007221 7f5c973a57a0 20 read_log 194209'15193 (194209'15153) delete 3ea053f7/benchmark_data_storage0111-ib.pocstor_26659_object12689/head//24 by osd.330.0:1206 2014-08-12 14:05:45.346656 2014-08-13 18:06:07.007254 7f5c973a57a0 20 read_log 194209'15194 (194209'15155) delete fab353f7/benchmark_data_storage0111-ib.pocstor_41097_object17928/head//24 by osd.330.0:1207 2014-08-12 14:05:45.350553 2014-08-13 18:06:07.007270 7f5c973a57a0 20 read_log 194209'15195 (194209'15156) delete f3aa53f7/benchmark_data_storage0111-ib.pocstor_41097_object18722/head//24 by osd.330.0:1208 2014-08-12 14:05:45.351033 2014-08-13 18:06:07.007283 7f5c973a57a0 20 read_log 194209'15196 (194209'15157) delete 88e63f7/benchmark_data_storage0111-ib.pocstor_2228_object1505/head//24 by osd.330.0:1209 2014-08-12 14:05:45.351968 2014-08-13 18:06:07.007295 7f5c973a57a0 20 read_log 194209'15197 (194209'15159) delete 1cd073f7/benchmark_data_storage0111-ib.pocstor_41526_object21134/head//24 by osd.330.0:1210 2014-08-12 14:05:45.352386 2014-08-13 18:06:07.007313 7f5c973a57a0 20 read_log 194209'15198 (194209'15161) delete a1b73f7/benchmark_data_storage0111-ib.pocstor_2228_object24579/head//24 by osd.330.0:1211 2014-08-12 14:05:45.355829 2014-08-13 18:06:07.007327 7f5c973a57a0 20 read_log 194209'15199 (194209'15162) delete 979c73f7/benchmark_data_storage0111-ib.pocstor_2228_object184/head//24 by osd.330.0:1212 2014-08-12 14:05:45.356243 2014-08-13 18:06:07.007342 7f5c973a57a0 20 read_log 194209'15200 (194209'15117) delete 4524ff7/benchmark_data_storage0111-ib.pocstor_2228_object29388/head//24 by osd.330.0:1213 2014-08-12 14:05:45.389163 2014-08-13 18:06:07.007359 7f5c973a57a0 20 read_log 194209'15201 (194209'15118) delete e89e4ff7/benchmark_data_storage0111-ib.pocstor_2228_object5450/head//24 by osd.330.0:1214 2014-08-12 14:05:45.389589 2014-08-13 18:06:07.007372 7f5c973a57a0 20 read_log 194209'15202 (194209'15119) delete 9626ff7/benchmark_data_storage0111-ib.pocstor_26659_object7552/head//24 by osd.330.0:1215 2014-08-12 14:05:45.389942 2014-08-13 18:06:07.007386 7f5c973a57a0 20 read_log 194209'15203 (194209'15120) delete 5d9d6ff7/benchmark_data_storage0111-ib.pocstor_26659_object9562/head//24 by osd.330.0:1216 2014-08-12 14:05:45.390286 2014-08-13 18:06:07.007414 7f5c973a57a0 20 read_log 194209'15204 (194209'15123) delete dd77ff7/benchmark_data_storage0111-ib.pocstor_2228_object4190/head//24 by osd.330.0:1217 2014-08-12 14:05:45.393282 2014-08-13 18:06:07.007427 7f5c973a57a0 20 read_log 194209'15205 (194209'15124) delete 73587ff7/benchmark_data_storage0111-ib.pocstor_2228_object30124/head//24 by osd.330.0:1218 2014-08-12 14:05:45.393710 2014-08-13 18:06:07.007448 7f5c973a57a0 20 read_log 194209'15206 (194209'15126) delete 7a097ff7/benchmark_data_storage0111-ib.pocstor_41097_object11653/head//24 by osd.330.0:1219 2014-08-12 14:05:45.394599 2014-08-13 18:06:07.007464 7f5c973a57a0 20 read_log 194209'15207 (194209'15127) delete a2be7ff7/benchmark_data_storage0111-ib.pocstor_2228_object22631/head//24 by osd.330.0:1220 2014-08-12 14:05:45.395000 2014-08-13 18:06:07.007477 7f5c973a57a0 20 read_log 194209'15208 (194209'15128) delete 2aaf7ff7/benchmark_data_storage0111-ib.pocstor_41097_object18497/head//24 by osd.330.0:1221 2014-08-12 14:05:45.398158 2014-08-13 18:06:07.007497 7f5c973a57a0 20 read_log 194209'15209 (194209'15129) delete a6ef7ff7/benchmark_data_storage0111-ib.pocstor_41097_object2016/head//24 by osd.330.0:1222 2014-08-12 14:05:45.398557 2014-08-13 18:06:07.007513 7f5c973a57a0 20 read_log 194209'15210 (194209'15132) delete fe0d8ff7/benchmark_data_storage0111-ib.pocstor_26659_object18049/head//24 by osd.330.0:1223 2014-08-12 14:05:45.399413 2014-08-13 18:06:07.007535 7f5c973a57a0 20 read_log 194209'15211 (194209'15133) delete f5799ff7/benchmark_data_storage0111-ib.pocstor_2228_object24140/head//24 by osd.330.0:1224 2014-08-12 14:05:45.399784 2014-08-13 18:06:07.007550 7f5c973a57a0 20 read_log 194215'15212 (194215'15212) modify 3f7/hit_set_24.3f7_archive_2014-08-12 15:32:25.737625_2014-08-12 15:51:57.155796/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:57.156146 2014-08-13 18:06:07.007563 7f5c973a57a0 20 read_log 194215'15213 (193305'15061) delete 3f7/hit_set_24.3f7_archive_2014-08-11 18:39:14.563860_2014-08-11 18:44:18.014758/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 15:51:57.156146 2014-08-13 18:06:07.007575 7f5c973a57a0 20 read_log 194215'15214 (192834'14069) clean b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by osd.330.0:3092 2014-08-12 15:51:57.163015 2014-08-13 18:06:07.007594 7f5c973a57a0 20 read_log 194215'15215 (194215'15214) delete b2a083f7/benchmark_data_storage0111-ib.pocstor_26659_object14129/head//24 by client.2174595.0:209404 0.000000 2014-08-13 18:06:07.007605 7f5c973a57a0 20 read_log 194215'15216 (192888'14780) clean 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by osd.330.0:3094 2014-08-12 15:51:57.173454 2014-08-13 18:06:07.007625 7f5c973a57a0 20 read_log 194215'15217 (194215'15216) delete 5b5483f7/benchmark_data_storage0111-ib.pocstor_41097_object11437/head//24 by client.2174595.0:209406 0.000000 2014-08-13 18:06:07.007640 7f5c973a57a0 20 read_log 194215'15218 (193305'15103) clean 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by osd.330.0:3096 2014-08-12 15:51:57.183348 2014-08-13 18:06:07.007652 7f5c973a57a0 20 read_log 194215'15219 (194215'15218) delete 889c83f7/benchmark_data_storage0111-ib.pocstor_62621_object25786/head//24 by client.2174595.0:209408 0.000000 2014-08-13 18:06:07.007663 7f5c973a57a0 20 read_log 194215'15220 (192834'14521) clean cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by osd.330.0:3098 2014-08-12 15:51:57.190832 2014-08-13 18:06:07.007676 7f5c973a57a0 20 read_log 194215'15221 (194215'15220) delete cabf83f7/benchmark_data_storage0111-ib.pocstor_41526_object5831/head//24 by client.2174595.0:209410 0.000000 2014-08-13 18:06:07.007687 7f5c973a57a0 20 read_log 194215'15222 (192834'14122) clean 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by osd.330.0:3100 2014-08-12 15:51:57.199665 2014-08-13 18:06:07.007699 7f5c973a57a0 20 read_log 194215'15223 (194215'15222) delete 5f79a3f7/benchmark_data_storage0111-ib.pocstor_26659_object20244/head//24 by client.2174595.0:209412 0.000000 2014-08-13 18:06:07.007712 7f5c973a57a0 20 read_log 194215'15224 (192888'14958) clean 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by osd.330.0:3102 2014-08-12 15:51:57.208976 2014-08-13 18:06:07.007724 7f5c973a57a0 20 read_log 194215'15225 (194215'15224) delete 2292b3f7/benchmark_data_storage0111-ib.pocstor_2228_object14378/head//24 by client.2174595.0:209414 0.000000 2014-08-13 18:06:07.007735 7f5c973a57a0 20 read_log 194215'15226 (192834'14511) clean 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by osd.330.0:3104 2014-08-12 15:51:57.217245 2014-08-13 18:06:07.007748 7f5c973a57a0 20 read_log 194215'15227 (194215'15226) delete 38b4b3f7/benchmark_data_storage0111-ib.pocstor_41526_object4677/head//24 by client.2174595.0:209416 0.000000 2014-08-13 18:06:07.007758 7f5c973a57a0 20 read_log 194215'15228 (192888'14786) clean 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by osd.330.0:3106 2014-08-12 15:51:57.226430 2014-08-13 18:06:07.007771 7f5c973a57a0 20 read_log 194215'15229 (194215'15228) delete 5abfb3f7/benchmark_data_storage0111-ib.pocstor_41097_object16279/head//24 by client.2174595.0:209418 0.000000 2014-08-13 18:06:07.007807 7f5c973a57a0 20 read_log 194215'15230 (193305'15085) clean ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by osd.330.0:3108 2014-08-12 15:51:57.235148 2014-08-13 18:06:07.007824 7f5c973a57a0 20 read_log 194215'15231 (194215'15230) delete ae69c3f7/benchmark_data_storage0111-ib.pocstor_62621_object10066/head//24 by client.2174595.0:209420 0.000000 2014-08-13 18:06:07.007835 7f5c973a57a0 20 read_log 194215'15232 (192888'14763) clean c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by osd.330.0:3110 2014-08-12 15:51:57.244290 2014-08-13 18:06:07.007848 7f5c973a57a0 20 read_log 194215'15233 (194215'15232) delete c74ac3f7/benchmark_data_storage0111-ib.pocstor_41097_object2577/head//24 by client.2174595.0:209422 0.000000 2014-08-13 18:06:07.007859 7f5c973a57a0 20 read_log 194215'15234 (192888'14777) clean 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by osd.330.0:3112 2014-08-12 15:51:57.254100 2014-08-13 18:06:07.007871 7f5c973a57a0 20 read_log 194215'15235 (194215'15234) delete 4d16d3f7/benchmark_data_storage0111-ib.pocstor_41097_object10350/head//24 by client.2174595.0:209424 0.000000 2014-08-13 18:06:07.007882 7f5c973a57a0 20 read_log 194215'15236 (193305'15095) clean f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by osd.330.0:3114 2014-08-12 15:51:57.262152 2014-08-13 18:06:07.007894 7f5c973a57a0 20 read_log 194215'15237 (194215'15236) delete f5a8d3f7/benchmark_data_storage0111-ib.pocstor_62621_object19663/head//24 by client.2174595.0:209426 0.000000 2014-08-13 18:06:07.007905 7f5c973a57a0 20 read_log 194215'15238 (193305'15087) clean af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by osd.330.0:3116 2014-08-12 15:51:57.270789 2014-08-13 18:06:07.007919 7f5c973a57a0 20 read_log 194215'15239 (194215'15238) delete af1bd3f7/benchmark_data_storage0111-ib.pocstor_62621_object14378/head//24 by client.2174595.0:209428 0.000000 2014-08-13 18:06:07.007932 7f5c973a57a0 20 read_log 194215'15240 (192888'14954) clean 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by osd.330.0:3118 2014-08-12 15:51:57.279301 2014-08-13 18:06:07.007945 7f5c973a57a0 20 read_log 194215'15241 (194215'15240) delete 159ce3f7/benchmark_data_storage0111-ib.pocstor_2228_object8812/head//24 by client.2174595.0:209430 0.000000 2014-08-13 18:06:07.007956 7f5c973a57a0 20 read_log 194215'15242 (192888'14788) clean ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by osd.330.0:3120 2014-08-12 15:51:57.287883 2014-08-13 18:06:07.007968 7f5c973a57a0 20 read_log 194215'15243 (194215'15242) delete ec26f3f7/benchmark_data_storage0111-ib.pocstor_41097_object16539/head//24 by client.2174595.0:209432 0.000000 2014-08-13 18:06:07.007979 7f5c973a57a0 20 read_log 194215'15244 (192888'14791) clean 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by osd.330.0:3122 2014-08-12 15:51:57.297959 2014-08-13 18:06:07.007994 7f5c973a57a0 20 read_log 194215'15245 (194215'15244) delete 368bf3f7/benchmark_data_storage0111-ib.pocstor_41097_object18035/head//24 by client.2174595.0:209434 0.000000 2014-08-13 18:06:07.008005 7f5c973a57a0 20 read_log 194215'15246 (192834'13993) clean 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by osd.330.0:3124 2014-08-12 15:51:57.306825 2014-08-13 18:06:07.008018 7f5c973a57a0 20 read_log 194215'15247 (194215'15246) delete 104707f7/benchmark_data_storage0111-ib.pocstor_26659_object4897/head//24 by client.2174595.0:209436 0.000000 2014-08-13 18:06:07.008032 7f5c973a57a0 20 read_log 194215'15248 (192888'14801) clean d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by osd.330.0:3126 2014-08-12 15:51:57.315275 2014-08-13 18:06:07.008045 7f5c973a57a0 20 read_log 194215'15249 (194215'15248) delete d2ba07f7/benchmark_data_storage0111-ib.pocstor_41097_object22929/head//24 by client.2174595.0:209438 0.000000 2014-08-13 18:06:07.008067 7f5c973a57a0 20 read_log 194215'15250 (192888'14794) clean 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by osd.330.0:3128 2014-08-12 15:51:57.325646 2014-08-13 18:06:07.008082 7f5c973a57a0 20 read_log 194215'15251 (194215'15250) delete 1f2b07f7/benchmark_data_storage0111-ib.pocstor_41097_object18940/head//24 by client.2174595.0:209440 0.000000 2014-08-13 18:06:07.008093 7f5c973a57a0 20 read_log 194215'15252 (193305'15097) clean ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by osd.330.0:3130 2014-08-12 15:51:57.333537 2014-08-13 18:06:07.008106 7f5c973a57a0 20 read_log 194215'15253 (194215'15252) delete ec6117f7/benchmark_data_storage0111-ib.pocstor_62621_object20380/head//24 by client.2174595.0:209442 0.000000 2014-08-13 18:06:07.008117 7f5c973a57a0 20 read_log 194215'15254 (192834'14581) clean f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by osd.330.0:3132 2014-08-12 15:51:57.342202 2014-08-13 18:06:07.008131 7f5c973a57a0 20 read_log 194215'15255 (194215'15254) delete f7a217f7/benchmark_data_storage0111-ib.pocstor_41526_object13009/head//24 by client.2174595.0:209444 0.000000 2014-08-13 18:06:07.008143 7f5c973a57a0 20 read_log 194215'15256 (193305'15082) clean eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by osd.330.0:3134 2014-08-12 15:51:57.350514 2014-08-13 18:06:07.008163 7f5c973a57a0 20 read_log 194215'15257 (194215'15256) delete eb9517f7/benchmark_data_storage0111-ib.pocstor_62621_object5499/head//24 by client.2174595.0:209446 0.000000 2014-08-13 18:06:07.008174 7f5c973a57a0 20 read_log 194215'15258 (193305'15111) clean dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by osd.330.0:3136 2014-08-12 15:51:57.358340 2014-08-13 18:06:07.008188 7f5c973a57a0 20 read_log 194215'15259 (194215'15258) delete dbb17f7/benchmark_data_storage0111-ib.pocstor_62621_object29454/head//24 by client.2174595.0:209448 0.000000 2014-08-13 18:06:07.008199 7f5c973a57a0 20 read_log 194215'15260 (192834'14498) clean 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by osd.330.0:3138 2014-08-12 15:51:57.365159 2014-08-13 18:06:07.008212 7f5c973a57a0 20 read_log 194215'15261 (194215'15260) delete 65ec17f7/benchmark_data_storage0111-ib.pocstor_41526_object2836/head//24 by client.2174595.0:209450 0.000000 2014-08-13 18:06:07.008223 7f5c973a57a0 20 read_log 194215'15262 (192888'14772) clean e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by osd.330.0:3140 2014-08-12 15:51:57.371557 2014-08-13 18:06:07.008235 7f5c973a57a0 20 read_log 194215'15263 (194215'15262) delete e56027f7/benchmark_data_storage0111-ib.pocstor_41097_object8059/head//24 by client.2174595.0:209452 0.000000 2014-08-13 18:06:07.008246 7f5c973a57a0 20 read_log 194215'15264 (193305'15086) clean a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by osd.330.0:3142 2014-08-12 15:51:57.377277 2014-08-13 18:06:07.008261 7f5c973a57a0 20 read_log 194215'15265 (194215'15264) delete a41727f7/benchmark_data_storage0111-ib.pocstor_62621_object14259/head//24 by client.2174595.0:209454 0.000000 2014-08-13 18:06:07.008275 7f5c973a57a0 20 read_log 194215'15266 (192888'14770) clean 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by osd.330.0:3144 2014-08-12 15:51:57.384205 2014-08-13 18:06:07.008287 7f5c973a57a0 20 read_log 194215'15267 (194215'15266) delete 688637f7/benchmark_data_storage0111-ib.pocstor_41097_object6793/head//24 by client.2174595.0:209456 0.000000 2014-08-13 18:06:07.008298 7f5c973a57a0 20 read_log 194215'15268 (192888'14962) clean e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by osd.330.0:3146 2014-08-12 15:51:57.391008 2014-08-13 18:06:07.008310 7f5c973a57a0 20 read_log 194215'15269 (194215'15268) delete e0b837f7/benchmark_data_storage0111-ib.pocstor_2228_object16825/head//24 by client.2174595.0:209458 0.000000 2014-08-13 18:06:07.008332 7f5c973a57a0 20 read_log 194215'15270 (192834'14605) clean 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by osd.330.0:3148 2014-08-12 15:51:57.397179 2014-08-13 18:06:07.008348 7f5c973a57a0 20 read_log 194215'15271 (194215'15270) delete 6c4c37f7/benchmark_data_storage0111-ib.pocstor_41526_object16374/head//24 by client.2174595.0:209460 0.000000 2014-08-13 18:06:07.008359 7f5c973a57a0 20 read_log 194215'15272 (192888'14960) clean 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by osd.330.0:3150 2014-08-12 15:51:57.403346 2014-08-13 18:06:07.008375 7f5c973a57a0 20 read_log 194215'15273 (194215'15272) delete 16ad37f7/benchmark_data_storage0111-ib.pocstor_2228_object15972/head//24 by client.2174595.0:209462 0.000000 2014-08-13 18:06:07.008398 7f5c973a57a0 20 read_log 194215'15274 (192888'14979) clean 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by osd.330.0:3152 2014-08-12 15:51:57.409477 2014-08-13 18:06:07.008411 7f5c973a57a0 20 read_log 194215'15275 (194215'15274) delete 5cc047f7/benchmark_data_storage0111-ib.pocstor_2228_object23881/head//24 by client.2174595.0:209464 0.000000 2014-08-13 18:06:07.008433 7f5c973a57a0 20 read_log 194215'15276 (192834'14123) clean b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by osd.330.0:3154 2014-08-12 15:51:57.416222 2014-08-13 18:06:07.008449 7f5c973a57a0 20 read_log 194215'15277 (194215'15276) delete b6f157f7/benchmark_data_storage0111-ib.pocstor_26659_object20306/head//24 by client.2174595.0:209466 0.000000 2014-08-13 18:06:07.008460 7f5c973a57a0 20 read_log 194215'15278 (192834'14606) clean 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by osd.330.0:3156 2014-08-12 15:51:57.422245 2014-08-13 18:06:07.008473 7f5c973a57a0 20 read_log 194215'15279 (194215'15278) delete 678357f7/benchmark_data_storage0111-ib.pocstor_41526_object16658/head//24 by client.2174595.0:209468 0.000000 2014-08-13 18:06:07.008484 7f5c973a57a0 20 read_log 194215'15280 (193305'15081) clean f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by osd.330.0:3158 2014-08-12 15:51:57.428740 2014-08-13 18:06:07.008501 7f5c973a57a0 20 read_log 194215'15281 (194215'15280) delete f8a657f7/benchmark_data_storage0111-ib.pocstor_62621_object3589/head//24 by client.2174595.0:209470 0.000000 2014-08-13 18:06:07.008513 7f5c973a57a0 20 read_log 194215'15282 (193305'15102) clean cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by osd.330.0:3160 2014-08-12 15:51:57.434806 2014-08-13 18:06:07.008526 7f5c973a57a0 20 read_log 194215'15283 (194215'15282) delete cfee57f7/benchmark_data_storage0111-ib.pocstor_62621_object25785/head//24 by client.2174595.0:209472 0.000000 2014-08-13 18:06:07.008537 7f5c973a57a0 20 read_log 194215'15284 (192888'14972) clean bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by osd.330.0:3162 2014-08-12 15:51:57.441555 2014-08-13 18:06:07.008555 7f5c973a57a0 20 read_log 194215'15285 (194215'15284) delete bb8c67f7/benchmark_data_storage0111-ib.pocstor_2228_object21743/head//24 by client.2174595.0:209474 0.000000 2014-08-13 18:06:07.008567 7f5c973a57a0 20 read_log 194215'15286 (192834'14077) clean 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by osd.330.0:3164 2014-08-12 15:51:57.447930 2014-08-13 18:06:07.008617 7f5c973a57a0 20 read_log 194215'15287 (194215'15286) delete 9a8077f7/benchmark_data_storage0111-ib.pocstor_26659_object14792/head//24 by client.2174595.0:209476 0.000000 2014-08-13 18:06:07.008633 7f5c973a57a0 20 read_log 194215'15288 (193305'15078) clean d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by osd.330.0:3166 2014-08-12 15:51:57.454510 2014-08-13 18:06:07.008645 7f5c973a57a0 20 read_log 194215'15289 (194215'15288) delete d7687f7/benchmark_data_storage0111-ib.pocstor_62621_object2118/head//24 by client.2174595.0:209478 0.000000 2014-08-13 18:06:07.008663 7f5c973a57a0 20 read_log 194215'15290 (192888'14974) clean 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by osd.330.0:3168 2014-08-12 15:51:57.460955 2014-08-13 18:06:07.008678 7f5c973a57a0 20 read_log 194215'15291 (194215'15290) delete 161a87f7/benchmark_data_storage0111-ib.pocstor_2228_object22210/head//24 by client.2174595.0:209480 0.000000 2014-08-13 18:06:07.008690 7f5c973a57a0 20 read_log 194215'15292 (192888'14779) clean 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by osd.330.0:3170 2014-08-12 15:51:57.467655 2014-08-13 18:06:07.008702 7f5c973a57a0 20 read_log 194215'15293 (194215'15292) delete 539a97f7/benchmark_data_storage0111-ib.pocstor_41097_object10667/head//24 by client.2174595.0:209482 0.000000 2014-08-13 18:06:07.008713 7f5c973a57a0 20 read_log 194215'15294 (192834'13963) clean c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by osd.330.0:3172 2014-08-12 15:51:57.473987 2014-08-13 18:06:07.008738 7f5c973a57a0 20 read_log 194215'15295 (194215'15294) delete c85e97f7/benchmark_data_storage0111-ib.pocstor_26659_object52/head//24 by client.2174595.0:209484 0.000000 2014-08-13 18:06:07.008749 7f5c973a57a0 20 read_log 194215'15296 (192888'14773) clean 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by osd.330.0:3174 2014-08-12 15:51:57.481246 2014-08-13 18:06:07.008762 7f5c973a57a0 20 read_log 194215'15297 (194215'15296) delete 23a7f7/benchmark_data_storage0111-ib.pocstor_41097_object8629/head//24 by client.2174595.0:209486 0.000000 2014-08-13 18:06:07.008773 7f5c973a57a0 20 read_log 194215'15298 (192888'14973) clean 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by osd.330.0:3176 2014-08-12 15:51:57.488747 2014-08-13 18:06:07.008791 7f5c973a57a0 20 read_log 194215'15299 (194215'15298) delete 6093a7f7/benchmark_data_storage0111-ib.pocstor_2228_object21975/head//24 by client.2174595.0:209488 0.000000 2014-08-13 18:06:07.008802 7f5c973a57a0 20 read_log 194215'15300 (193305'15110) clean fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by osd.330.0:3178 2014-08-12 15:51:57.497669 2014-08-13 18:06:07.008814 7f5c973a57a0 20 read_log 194215'15301 (194215'15300) delete fca3a7f7/benchmark_data_storage0111-ib.pocstor_62621_object29116/head//24 by client.2174595.0:209490 0.000000 2014-08-13 18:06:07.008826 7f5c973a57a0 20 read_log 194215'15302 (192834'14535) clean df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by osd.330.0:3180 2014-08-12 15:51:57.507016 2014-08-13 18:06:07.008838 7f5c973a57a0 20 read_log 194215'15303 (194215'15302) delete df78a7f7/benchmark_data_storage0111-ib.pocstor_41526_object7791/head//24 by client.2174595.0:209492 0.000000 2014-08-13 18:06:07.008849 7f5c973a57a0 20 read_log 194215'15304 (192888'14789) clean 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by osd.330.0:3182 2014-08-12 15:51:57.514363 2014-08-13 18:06:07.008861 7f5c973a57a0 20 read_log 194215'15305 (194215'15304) delete 1d98a7f7/benchmark_data_storage0111-ib.pocstor_41097_object17151/head//24 by client.2174595.0:209494 0.000000 2014-08-13 18:06:07.008872 7f5c973a57a0 20 read_log 194215'15306 (193305'15100) clean e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by osd.330.0:3184 2014-08-12 15:51:57.522708 2014-08-13 18:06:07.008884 7f5c973a57a0 20 read_log 194215'15307 (194215'15306) delete e75da7f7/benchmark_data_storage0111-ib.pocstor_62621_object24467/head//24 by client.2174595.0:209496 0.000000 2014-08-13 18:06:07.008895 7f5c973a57a0 20 read_log 194215'15308 (193305'15101) clean e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by osd.330.0:3186 2014-08-12 15:51:57.530423 2014-08-13 18:06:07.008907 7f5c973a57a0 20 read_log 194215'15309 (194215'15308) delete e303b7f7/benchmark_data_storage0111-ib.pocstor_62621_object24569/head//24 by client.2174595.0:209498 0.000000 2014-08-13 18:06:07.008928 7f5c973a57a0 20 read_log 194215'15310 (193305'15091) clean e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by osd.330.0:3188 2014-08-12 15:51:57.537765 2014-08-13 18:06:07.008952 7f5c973a57a0 20 read_log 194215'15311 (194215'15310) delete e014b7f7/benchmark_data_storage0111-ib.pocstor_62621_object16338/head//24 by client.2174595.0:209500 0.000000 2014-08-13 18:06:07.008964 7f5c973a57a0 20 read_log 194215'15312 (192834'14568) clean 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by osd.330.0:3190 2014-08-12 15:51:57.546360 2014-08-13 18:06:07.008976 7f5c973a57a0 20 read_log 194215'15313 (194215'15312) delete 974b7f7/benchmark_data_storage0111-ib.pocstor_41526_object11173/head//24 by client.2174595.0:209502 0.000000 2014-08-13 18:06:07.008987 7f5c973a57a0 20 read_log 194215'15314 (193305'15105) clean 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by osd.330.0:3192 2014-08-12 15:51:57.554766 2014-08-13 18:06:07.009000 7f5c973a57a0 20 read_log 194215'15315 (194215'15314) delete 825db7f7/benchmark_data_storage0111-ib.pocstor_62621_object26892/head//24 by client.2174595.0:209504 0.000000 2014-08-13 18:06:07.009010 7f5c973a57a0 20 read_log 194215'15316 (192888'14782) clean ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by osd.330.0:3194 2014-08-12 15:51:57.562109 2014-08-13 18:06:07.009023 7f5c973a57a0 20 read_log 194215'15317 (194215'15316) delete ebb0c7f7/benchmark_data_storage0111-ib.pocstor_41097_object12439/head//24 by client.2174595.0:209506 0.000000 2014-08-13 18:06:07.009033 7f5c973a57a0 20 read_log 194215'15318 (192888'14943) clean 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by osd.330.0:3196 2014-08-12 15:51:57.569806 2014-08-13 18:06:07.009046 7f5c973a57a0 20 read_log 194215'15319 (194215'15318) delete 651ac7f7/benchmark_data_storage0111-ib.pocstor_2228_object4759/head//24 by client.2174595.0:209508 0.000000 2014-08-13 18:06:07.009057 7f5c973a57a0 20 read_log 194215'15320 (192834'14573) clean b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by osd.330.0:3198 2014-08-12 15:51:57.581068 2014-08-13 18:06:07.009069 7f5c973a57a0 20 read_log 194215'15321 (194215'15320) delete b961d7f7/benchmark_data_storage0111-ib.pocstor_41526_object11733/head//24 by client.2174595.0:209510 0.000000 2014-08-13 18:06:07.009080 7f5c973a57a0 20 read_log 194215'15322 (193305'15106) clean 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by osd.330.0:3200 2014-08-12 15:51:57.589873 2014-08-13 18:06:07.009092 7f5c973a57a0 20 read_log 194215'15323 (194215'15322) delete 4c42d7f7/benchmark_data_storage0111-ib.pocstor_62621_object26934/head//24 by client.2174595.0:209512 0.000000 2014-08-13 18:06:07.009103 7f5c973a57a0 20 read_log 194215'15324 (192888'14796) clean 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by osd.330.0:3202 2014-08-12 15:51:57.598898 2014-08-13 18:06:07.009115 7f5c973a57a0 20 read_log 194215'15325 (194215'15324) delete 5353d7f7/benchmark_data_storage0111-ib.pocstor_41097_object20853/head//24 by client.2174595.0:209514 0.000000 2014-08-13 18:06:07.009131 7f5c973a57a0 20 read_log 194215'15326 (192834'14097) clean ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by osd.330.0:3204 2014-08-12 15:51:57.607086 2014-08-13 18:06:07.009151 7f5c973a57a0 20 read_log 194215'15327 (194215'15326) delete ab4cd7f7/benchmark_data_storage0111-ib.pocstor_26659_object17423/head//24 by client.2174595.0:209516 0.000000 2014-08-13 18:06:07.009165 7f5c973a57a0 20 read_log 194215'15328 (192834'14488) clean 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by osd.330.0:3206 2014-08-12 15:51:57.616334 2014-08-13 18:06:07.009178 7f5c973a57a0 20 read_log 194215'15329 (194215'15328) delete 896cd7f7/benchmark_data_storage0111-ib.pocstor_41526_object1395/head//24 by client.2174595.0:209518 0.000000 2014-08-13 18:06:07.009196 7f5c973a57a0 20 read_log 194215'15330 (192888'14804) clean 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by osd.330.0:3208 2014-08-12 15:51:57.625101 2014-08-13 18:06:07.009209 7f5c973a57a0 20 read_log 194215'15331 (194215'15330) delete 2fc3e7f7/benchmark_data_storage0111-ib.pocstor_41097_object25053/head//24 by client.2174595.0:209520 0.000000 2014-08-13 18:06:07.009219 7f5c973a57a0 20 read_log 194215'15332 (192888'14813) clean 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by osd.330.0:3210 2014-08-12 15:51:57.635725 2014-08-13 18:06:07.009232 7f5c973a57a0 20 read_log 194215'15333 (194215'15332) delete 8f2ae7f7/benchmark_data_storage0111-ib.pocstor_41097_object30578/head//24 by client.2174595.0:209522 0.000000 2014-08-13 18:06:07.009243 7f5c973a57a0 20 read_log 194215'15334 (192834'14648) clean 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by osd.330.0:3212 2014-08-12 15:51:57.645636 2014-08-13 18:06:07.009255 7f5c973a57a0 20 read_log 194215'15335 (194215'15334) delete 601be7f7/benchmark_data_storage0111-ib.pocstor_41526_object22591/head//24 by client.2174595.0:209524 0.000000 2014-08-13 18:06:07.009266 7f5c973a57a0 20 read_log 194215'15336 (192888'14809) clean cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by osd.330.0:3214 2014-08-12 15:51:57.654479 2014-08-13 18:06:07.009281 7f5c973a57a0 20 read_log 194215'15337 (194215'15336) delete cf9ce7f7/benchmark_data_storage0111-ib.pocstor_41097_object28976/head//24 by client.2174595.0:209526 0.000000 2014-08-13 18:06:07.009292 7f5c973a57a0 20 read_log 194215'15338 (192834'14096) clean b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by osd.330.0:3216 2014-08-12 15:51:57.662568 2014-08-13 18:06:07.009307 7f5c973a57a0 20 read_log 194215'15339 (194215'15338) delete b2df7f7/benchmark_data_storage0111-ib.pocstor_26659_object17358/head//24 by client.2174595.0:209528 0.000000 2014-08-13 18:06:07.009321 7f5c973a57a0 20 read_log 194215'15340 (192834'14072) clean 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by osd.330.0:3218 2014-08-12 15:51:57.672342 2014-08-13 18:06:07.009333 7f5c973a57a0 20 read_log 194215'15341 (194215'15340) delete 7ce30bf7/benchmark_data_storage0111-ib.pocstor_26659_object14341/head//24 by client.2174595.0:209530 0.000000 2014-08-13 18:06:07.009344 7f5c973a57a0 20 read_log 194215'15342 (193305'15113) clean 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by osd.330.0:3220 2014-08-12 15:51:57.680876 2014-08-13 18:06:07.009359 7f5c973a57a0 20 read_log 194215'15343 (194215'15342) delete 55bf1bf7/benchmark_data_storage0111-ib.pocstor_62621_object32662/head//24 by client.2174595.0:209532 0.000000 2014-08-13 18:06:07.009370 7f5c973a57a0 20 read_log 194215'15344 (192888'14950) clean fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by osd.330.0:3222 2014-08-12 15:51:57.689587 2014-08-13 18:06:07.009383 7f5c973a57a0 20 read_log 194215'15345 (194215'15344) delete fb193bf7/benchmark_data_storage0111-ib.pocstor_2228_object7526/head//24 by client.2174595.0:209534 0.000000 2014-08-13 18:06:07.009394 7f5c973a57a0 20 read_log 194215'15346 (192888'14766) clean b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by osd.330.0:3224 2014-08-12 15:51:57.698015 2014-08-13 18:06:07.009410 7f5c973a57a0 20 read_log 194215'15347 (194215'15346) delete b7134bf7/benchmark_data_storage0111-ib.pocstor_41097_object3856/head//24 by client.2174595.0:209536 0.000000 2014-08-13 18:06:07.009421 7f5c973a57a0 20 read_log 194215'15348 (192888'14959) clean 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by osd.330.0:3226 2014-08-12 15:51:57.708252 2014-08-13 18:06:07.009448 7f5c973a57a0 20 read_log 194215'15349 (194215'15348) delete 59c65bf7/benchmark_data_storage0111-ib.pocstor_2228_object15479/head//24 by client.2174595.0:209538 0.000000 2014-08-13 18:06:07.009482 7f5c973a57a0 20 read_log 194215'15350 (193305'15074) clean 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by osd.330.0:3228 2014-08-12 15:51:57.716005 2014-08-13 18:06:07.009500 7f5c973a57a0 20 read_log 194215'15351 (194215'15350) delete 552e5bf7/benchmark_data_storage0111-ib.pocstor_62621_object562/head//24 by client.2174595.0:209540 0.000000 2014-08-13 18:06:07.009511 7f5c973a57a0 20 read_log 194215'15352 (192834'14008) clean 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by osd.330.0:3230 2014-08-12 15:51:57.724370 2014-08-13 18:06:07.009528 7f5c973a57a0 20 read_log 194215'15353 (194215'15352) delete 11446bf7/benchmark_data_storage0111-ib.pocstor_26659_object6787/head//24 by client.2174595.0:209542 0.000000 2014-08-13 18:06:07.009539 7f5c973a57a0 20 read_log 194215'15354 (192834'13982) clean 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by osd.330.0:3232 2014-08-12 15:51:57.734127 2014-08-13 18:06:07.009554 7f5c973a57a0 20 read_log 194215'15355 (194215'15354) delete 67856bf7/benchmark_data_storage0111-ib.pocstor_26659_object2851/head//24 by client.2174595.0:209544 0.000000 2014-08-13 18:06:07.009565 7f5c973a57a0 20 read_log 194215'15356 (193305'15112) clean 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by osd.330.0:3234 2014-08-12 15:51:57.742749 2014-08-13 18:06:07.009588 7f5c973a57a0 20 read_log 194215'15357 (194215'15356) delete 6ee86bf7/benchmark_data_storage0111-ib.pocstor_62621_object29840/head//24 by client.2174595.0:209546 0.000000 2014-08-13 18:06:07.009600 7f5c973a57a0 20 read_log 194215'15358 (192888'14947) clean 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by osd.330.0:3236 2014-08-12 15:51:57.751812 2014-08-13 18:06:07.009612 7f5c973a57a0 20 read_log 194215'15359 (194215'15358) delete 480e6bf7/benchmark_data_storage0111-ib.pocstor_2228_object6310/head//24 by client.2174595.0:209548 0.000000 2014-08-13 18:06:07.009624 7f5c973a57a0 20 read_log 194215'15360 (193305'15090) clean 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by osd.330.0:3238 2014-08-12 15:51:57.761110 2014-08-13 18:06:07.009642 7f5c973a57a0 20 read_log 194215'15361 (194215'15360) delete 91037bf7/benchmark_data_storage0111-ib.pocstor_62621_object15983/head//24 by client.2174595.0:209550 0.000000 2014-08-13 18:06:07.009653 7f5c973a57a0 20 read_log 194215'15362 (192888'14938) clean e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by osd.330.0:3240 2014-08-12 15:51:57.771432 2014-08-13 18:06:07.009666 7f5c973a57a0 20 read_log 194215'15363 (194215'15362) delete e2c18bf7/benchmark_data_storage0111-ib.pocstor_2228_object2972/head//24 by client.2174595.0:209552 0.000000 2014-08-13 18:06:07.009677 7f5c973a57a0 20 read_log 194215'15364 (193305'15099) clean 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by osd.330.0:3242 2014-08-12 15:51:57.780434 2014-08-13 18:06:07.009689 7f5c973a57a0 20 read_log 194215'15365 (194215'15364) delete 2fe58bf7/benchmark_data_storage0111-ib.pocstor_62621_object22592/head//24 by client.2174595.0:209554 0.000000 2014-08-13 18:06:07.009700 7f5c973a57a0 20 read_log 194215'15366 (192888'14797) clean 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by osd.330.0:3244 2014-08-12 15:51:57.788552 2014-08-13 18:06:07.009714 7f5c973a57a0 20 read_log 194215'15367 (194215'15366) delete 16878bf7/benchmark_data_storage0111-ib.pocstor_41097_object21234/head//24 by client.2174595.0:209556 0.000000 2014-08-13 18:06:07.009726 7f5c973a57a0 20 read_log 194215'15368 (192834'14595) clean cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by osd.330.0:3246 2014-08-12 15:51:57.796469 2014-08-13 18:06:07.009746 7f5c973a57a0 20 read_log 194215'15369 (194215'15368) delete cc7e8bf7/benchmark_data_storage0111-ib.pocstor_41526_object15289/head//24 by client.2174595.0:209558 0.000000 2014-08-13 18:06:07.009782 7f5c973a57a0 20 read_log 194215'15370 (192834'14125) clean 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by osd.330.0:3248 2014-08-12 15:51:57.805533 2014-08-13 18:06:07.009799 7f5c973a57a0 20 read_log 194215'15371 (194215'15370) delete 4586bbf7/benchmark_data_storage0111-ib.pocstor_26659_object20590/head//24 by client.2174595.0:209560 0.000000 2014-08-13 18:06:07.009810 7f5c973a57a0 20 read_log 194215'15372 (192888'14764) clean d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by osd.330.0:3250 2014-08-12 15:51:57.813563 2014-08-13 18:06:07.009823 7f5c973a57a0 20 read_log 194215'15373 (194215'15372) delete d1cbbf7/benchmark_data_storage0111-ib.pocstor_41097_object3273/head//24 by client.2174595.0:209562 0.000000 2014-08-13 18:06:07.009833 7f5c973a57a0 20 read_log 194215'15374 (193305'15104) clean 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by osd.330.0:3252 2014-08-12 15:51:57.821066 2014-08-13 18:06:07.009846 7f5c973a57a0 20 read_log 194215'15375 (194215'15374) delete 36fdbbf7/benchmark_data_storage0111-ib.pocstor_62621_object26079/head//24 by client.2174595.0:209564 0.000000 2014-08-13 18:06:07.009857 7f5c973a57a0 20 read_log 194215'15376 (192888'14798) clean efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by osd.330.0:3254 2014-08-12 15:51:57.829427 2014-08-13 18:06:07.009869 7f5c973a57a0 20 read_log 194215'15377 (194215'15376) delete efe2cbf7/benchmark_data_storage0111-ib.pocstor_41097_object21978/head//24 by client.2174595.0:209566 0.000000 2014-08-13 18:06:07.009880 7f5c973a57a0 20 read_log 194215'15378 (192888'14783) clean da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by osd.330.0:3256 2014-08-12 15:51:57.839143 2014-08-13 18:06:07.009892 7f5c973a57a0 20 read_log 194215'15379 (194215'15378) delete da13dbf7/benchmark_data_storage0111-ib.pocstor_41097_object12581/head//24 by client.2174595.0:209568 0.000000 2014-08-13 18:06:07.009933 7f5c973a57a0 20 read_log 194215'15380 (192888'14799) clean 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by osd.330.0:3258 2014-08-12 15:51:57.848891 2014-08-13 18:06:07.009950 7f5c973a57a0 20 read_log 194215'15381 (194215'15380) delete 36e4dbf7/benchmark_data_storage0111-ib.pocstor_41097_object22556/head//24 by client.2174595.0:209570 0.000000 2014-08-13 18:06:07.009961 7f5c973a57a0 20 read_log 194215'15382 (193305'15083) clean 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by osd.330.0:3260 2014-08-12 15:51:57.856768 2014-08-13 18:06:07.009973 7f5c973a57a0 20 read_log 194215'15383 (194215'15382) delete 66e4dbf7/benchmark_data_storage0111-ib.pocstor_62621_object6136/head//24 by client.2174595.0:209572 0.000000 2014-08-13 18:06:07.009984 7f5c973a57a0 20 read_log 194215'15384 (192834'14540) clean 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by osd.330.0:3262 2014-08-12 15:51:57.864561 2014-08-13 18:06:07.009997 7f5c973a57a0 20 read_log 194215'15385 (194215'15384) delete 3677dbf7/benchmark_data_storage0111-ib.pocstor_41526_object8436/head//24 by client.2174595.0:209574 0.000000 2014-08-13 18:06:07.010008 7f5c973a57a0 20 read_log 194215'15386 (192888'14961) clean f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by osd.330.0:3264 2014-08-12 15:51:57.872584 2014-08-13 18:06:07.010020 7f5c973a57a0 20 read_log 194215'15387 (194215'15386) delete f633ebf7/benchmark_data_storage0111-ib.pocstor_2228_object16570/head//24 by client.2174595.0:209576 0.000000 2014-08-13 18:06:07.010031 7f5c973a57a0 20 read_log 194215'15388 (192888'14774) clean e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by osd.330.0:3266 2014-08-12 15:51:57.881014 2014-08-13 18:06:07.010044 7f5c973a57a0 20 read_log 194215'15389 (194215'15388) delete e1cdebf7/benchmark_data_storage0111-ib.pocstor_41097_object8697/head//24 by client.2174595.0:209578 0.000000 2014-08-13 18:06:07.010065 7f5c973a57a0 20 read_log 194215'15390 (192888'14785) clean 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by osd.330.0:3268 2014-08-12 15:51:57.889693 2014-08-13 18:06:07.010078 7f5c973a57a0 20 read_log 194215'15391 (194215'15390) delete 4903fbf7/benchmark_data_storage0111-ib.pocstor_41097_object16181/head//24 by client.2174595.0:209580 0.000000 2014-08-13 18:06:07.010089 7f5c973a57a0 20 read_log 194215'15392 (192888'14806) clean 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by osd.330.0:3270 2014-08-12 15:51:57.899196 2014-08-13 18:06:07.010102 7f5c973a57a0 20 read_log 194215'15393 (194215'15392) delete 4a020ff7/benchmark_data_storage0111-ib.pocstor_41097_object26230/head//24 by client.2174595.0:209582 0.000000 2014-08-13 18:06:07.010117 7f5c973a57a0 20 read_log 194215'15394 (192888'14805) clean 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by osd.330.0:3272 2014-08-12 15:51:57.907103 2014-08-13 18:06:07.010129 7f5c973a57a0 20 read_log 194215'15395 (194215'15394) delete 330b0ff7/benchmark_data_storage0111-ib.pocstor_41097_object25800/head//24 by client.2174595.0:209584 0.000000 2014-08-13 18:06:07.010140 7f5c973a57a0 20 read_log 194215'15396 (192888'14800) clean be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by osd.330.0:3274 2014-08-12 15:51:57.917426 2014-08-13 18:06:07.010161 7f5c973a57a0 20 read_log 194215'15397 (194215'15396) delete be351ff7/benchmark_data_storage0111-ib.pocstor_41097_object22819/head//24 by client.2174595.0:209586 0.000000 2014-08-13 18:06:07.010173 7f5c973a57a0 20 read_log 194215'15398 (192834'14623) clean 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by osd.330.0:3276 2014-08-12 15:51:57.925992 2014-08-13 18:06:07.010185 7f5c973a57a0 20 read_log 194215'15399 (194215'15398) delete 325b1ff7/benchmark_data_storage0111-ib.pocstor_41526_object18673/head//24 by client.2174595.0:209588 0.000000 2014-08-13 18:06:07.010215 7f5c973a57a0 20 read_log 194215'15400 (194209'15134) delete daab9ff7/benchmark_data_storage0111-ib.pocstor_26659_object15631/head//24 by client.2174595.0:209590 0.000000 2014-08-13 18:06:07.010227 7f5c973a57a0 20 read_log 194215'15401 (194209'15135) delete 72a0aff7/benchmark_data_storage0111-ib.pocstor_41097_object24322/head//24 by client.2174595.0:209592 0.000000 2014-08-13 18:06:07.010245 7f5c973a57a0 20 read_log 194215'15402 (194215'15402) modify 3f7/hit_set_24.3f7_archive_2014-08-12 15:51:57.156178_2014-08-12 17:24:41.391144/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.391356 2014-08-13 18:06:07.010259 7f5c973a57a0 20 read_log 194215'15403 (194215'15212) delete 3f7/hit_set_24.3f7_archive_2014-08-12 15:32:25.737625_2014-08-12 15:51:57.155796/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 17:24:41.391356 2014-08-13 18:06:07.010272 7f5c973a57a0 20 read_log 194215'15404 (0'0) promote 4e0adbf7/benchmark_data_storage0111-ib.pocstor_34628_object386/head//24 by osd.330.0:3998 2014-08-12 17:24:41.394531 2014-08-13 18:06:07.010285 7f5c973a57a0 20 read_log 194215'15405 (194215'15404) modify 4e0adbf7/benchmark_data_storage0111-ib.pocstor_34628_object386/head//24 by client.2193759.0:387 2014-08-12 17:24:41.386371 2014-08-13 18:06:07.010307 7f5c973a57a0 20 read_log 194215'15406 (0'0) promote 5e186bf7/benchmark_data_storage0111-ib.pocstor_34628_object787/head//24 by osd.330.0:4008 2014-08-12 17:24:42.980428 2014-08-13 18:06:07.010320 7f5c973a57a0 20 read_log 194215'15407 (194215'15406) modify 5e186bf7/benchmark_data_storage0111-ib.pocstor_34628_object787/head//24 by client.2193759.0:788 2014-08-12 17:24:42.969146 2014-08-13 18:06:07.010333 7f5c973a57a0 20 read_log 194370'15408 (0'0) promote 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by osd.330.0:4033 2014-08-12 20:30:14.940235 2014-08-13 18:06:07.010353 7f5c973a57a0 20 read_log 194370'15409 (194370'15408) modify 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by client.1508531.0:203 2014-08-12 20:30:14.929515 2014-08-13 18:06:07.010371 7f5c973a57a0 20 read_log 194370'15410 (0'0) promote 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by osd.330.0:4059 2014-08-12 20:30:21.882955 2014-08-13 18:06:07.010383 7f5c973a57a0 20 read_log 194370'15411 (194370'15410) modify 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by client.1508531.0:1995 2014-08-12 20:30:21.874345 2014-08-13 18:06:07.010396 7f5c973a57a0 20 read_log 194370'15412 (0'0) promote 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by osd.330.0:4069 2014-08-12 20:30:24.817667 2014-08-13 18:06:07.010408 7f5c973a57a0 20 read_log 194370'15413 (194370'15412) modify 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by client.1508531.0:2799 2014-08-12 20:30:24.806873 2014-08-13 18:06:07.010420 7f5c973a57a0 20 read_log 194370'15414 (0'0) promote 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by osd.330.0:4073 2014-08-12 20:30:25.008021 2014-08-13 18:06:07.010440 7f5c973a57a0 20 read_log 194370'15415 (194370'15414) modify 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by client.1508531.0:2852 2014-08-12 20:30:25.000236 2014-08-13 18:06:07.010457 7f5c973a57a0 20 read_log 194370'15416 (0'0) promote d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by osd.330.0:4097 2014-08-12 20:30:29.474109 2014-08-13 18:06:07.010469 7f5c973a57a0 20 read_log 194370'15417 (194370'15416) modify d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by client.1508531.0:4047 2014-08-12 20:30:29.463943 2014-08-13 18:06:07.010482 7f5c973a57a0 20 read_log 194370'15418 (0'0) promote f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by osd.330.0:4107 2014-08-12 20:30:32.487020 2014-08-13 18:06:07.010494 7f5c973a57a0 20 read_log 194370'15419 (194370'15418) modify f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by client.1508531.0:4826 2014-08-12 20:30:32.478351 2014-08-13 18:06:07.010507 7f5c973a57a0 20 read_log 194370'15420 (0'0) promote c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by osd.330.0:4117 2014-08-12 20:30:35.407349 2014-08-13 18:06:07.010519 7f5c973a57a0 20 read_log 194370'15421 (194370'15420) modify c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by client.1508531.0:5502 2014-08-12 20:30:35.399089 2014-08-13 18:06:07.010531 7f5c973a57a0 20 read_log 194370'15422 (0'0) promote 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by osd.330.0:4141 2014-08-12 20:30:40.861330 2014-08-13 18:06:07.010543 7f5c973a57a0 20 read_log 194370'15423 (194370'15422) modify 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by client.1508531.0:6694 2014-08-12 20:30:40.851596 2014-08-13 18:06:07.010556 7f5c973a57a0 20 read_log 194370'15424 (0'0) promote 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by osd.330.0:4143 2014-08-12 20:30:40.880586 2014-08-13 18:06:07.010568 7f5c973a57a0 20 read_log 194370'15425 (194370'15424) modify 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by client.1508531.0:6699 2014-08-12 20:30:40.872634 2014-08-13 18:06:07.010580 7f5c973a57a0 20 read_log 194370'15426 (0'0) promote 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by osd.330.0:4159 2014-08-12 20:30:44.680911 2014-08-13 18:06:07.010592 7f5c973a57a0 20 read_log 194370'15427 (194370'15426) modify 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by client.1508531.0:7463 2014-08-12 20:30:44.671825 2014-08-13 18:06:07.010605 7f5c973a57a0 20 read_log 194370'15428 (0'0) promote 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by osd.330.0:4163 2014-08-12 20:30:44.818938 2014-08-13 18:06:07.010627 7f5c973a57a0 20 read_log 194370'15429 (194370'15428) modify 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by client.1508531.0:7491 2014-08-12 20:30:44.809735 2014-08-13 18:06:07.010640 7f5c973a57a0 20 read_log 194370'15430 (0'0) promote de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by osd.330.0:4173 2014-08-12 20:30:45.899902 2014-08-13 18:06:07.010653 7f5c973a57a0 20 read_log 194370'15431 (194370'15430) modify de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by client.1508531.0:7688 2014-08-12 20:30:45.885740 2014-08-13 18:06:07.010668 7f5c973a57a0 20 read_log 194370'15432 (0'0) promote a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by osd.330.0:4175 2014-08-12 20:30:46.029510 2014-08-13 18:06:07.010680 7f5c973a57a0 20 read_log 194370'15433 (194370'15432) modify a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by client.1508531.0:7711 2014-08-12 20:30:46.018440 2014-08-13 18:06:07.010694 7f5c973a57a0 20 read_log 194370'15434 (0'0) promote ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by osd.330.0:4179 2014-08-12 20:30:50.377398 2014-08-13 18:06:07.010708 7f5c973a57a0 20 read_log 194370'15435 (194370'15434) modify ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by client.1508531.0:8518 2014-08-12 20:30:50.366822 2014-08-13 18:06:07.010721 7f5c973a57a0 20 read_log 194370'15436 (0'0) promote ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by osd.330.0:4181 2014-08-12 20:30:51.589360 2014-08-13 18:06:07.010738 7f5c973a57a0 20 read_log 194370'15437 (194370'15436) modify ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by client.1508531.0:8722 2014-08-12 20:30:51.577344 2014-08-13 18:06:07.010752 7f5c973a57a0 20 read_log 194370'15438 (0'0) promote 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by osd.330.0:4237 2014-08-12 20:31:15.627731 2014-08-13 18:06:07.010765 7f5c973a57a0 20 read_log 194370'15439 (194370'15438) modify 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by client.1508531.0:12589 2014-08-12 20:31:15.617565 2014-08-13 18:06:07.010784 7f5c973a57a0 20 read_log 194370'15440 (0'0) promote fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by osd.330.0:4245 2014-08-12 20:31:16.689293 2014-08-13 18:06:07.010796 7f5c973a57a0 20 read_log 194370'15441 (194370'15440) modify fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by client.1508531.0:12717 2014-08-12 20:31:16.677145 2014-08-13 18:06:07.010809 7f5c973a57a0 20 read_log 194377'15442 (0'0) promote d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by osd.330.0:4273 2014-08-12 20:32:57.945917 2014-08-13 18:06:07.010821 7f5c973a57a0 20 read_log 194377'15443 (194377'15442) modify d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by client.2231190.0:185 2014-08-12 20:32:57.934951 2014-08-13 18:06:07.010836 7f5c973a57a0 20 read_log 194377'15444 (0'0) promote 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by osd.330.0:4279 2014-08-12 20:33:00.510117 2014-08-13 18:06:07.010849 7f5c973a57a0 20 read_log 194377'15445 (194377'15444) modify 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by client.2231190.0:787 2014-08-12 20:33:00.502816 2014-08-13 18:06:07.010863 7f5c973a57a0 20 read_log 194377'15446 (0'0) promote 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by osd.330.0:4295 2014-08-12 20:33:04.940914 2014-08-13 18:06:07.010875 7f5c973a57a0 20 read_log 194377'15447 (194377'15446) modify 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by client.2231190.0:1938 2014-08-12 20:33:04.934317 2014-08-13 18:06:07.010890 7f5c973a57a0 20 read_log 194377'15448 (0'0) promote 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by osd.330.0:4299 2014-08-12 20:33:05.215181 2014-08-13 18:06:07.010909 7f5c973a57a0 20 read_log 194377'15449 (194377'15448) modify 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by client.2231190.0:1993 2014-08-12 20:33:05.205475 2014-08-13 18:06:07.010924 7f5c973a57a0 20 read_log 194377'15450 (0'0) promote a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by osd.330.0:4307 2014-08-12 20:33:07.204187 2014-08-13 18:06:07.010937 7f5c973a57a0 20 read_log 194377'15451 (194377'15450) modify a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by client.2231190.0:2495 2014-08-12 20:33:07.191237 2014-08-13 18:06:07.010951 7f5c973a57a0 20 read_log 194377'15452 (0'0) promote b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by osd.330.0:4327 2014-08-12 20:33:11.508975 2014-08-13 18:06:07.010964 7f5c973a57a0 20 read_log 194377'15453 (194377'15452) modify b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by client.2231190.0:3530 2014-08-12 20:33:11.499104 2014-08-13 18:06:07.010979 7f5c973a57a0 20 read_log 194377'15454 (0'0) promote afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by osd.330.0:4333 2014-08-12 20:33:14.031056 2014-08-13 18:06:07.010991 7f5c973a57a0 20 read_log 194377'15455 (194377'15454) modify afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by client.2231190.0:4019 2014-08-12 20:33:14.019939 2014-08-13 18:06:07.011004 7f5c973a57a0 20 read_log 194377'15456 (0'0) promote a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by osd.330.0:4339 2014-08-12 20:33:14.820058 2014-08-13 18:06:07.011018 7f5c973a57a0 20 read_log 194377'15457 (194377'15456) modify a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by client.2231190.0:4155 2014-08-12 20:33:14.812906 2014-08-13 18:06:07.011031 7f5c973a57a0 20 read_log 194377'15458 (0'0) promote a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by osd.330.0:4341 2014-08-12 20:33:14.838016 2014-08-13 18:06:07.011045 7f5c973a57a0 20 read_log 194377'15459 (194377'15458) modify a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by client.2231190.0:4158 2014-08-12 20:33:14.826432 2014-08-13 18:06:07.011057 7f5c973a57a0 20 read_log 194377'15460 (0'0) promote b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by osd.330.0:4349 2014-08-12 20:33:18.628081 2014-08-13 18:06:07.011071 7f5c973a57a0 20 read_log 194377'15461 (194377'15460) modify b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by client.2231190.0:4699 2014-08-12 20:33:18.617965 2014-08-13 18:06:07.011084 7f5c973a57a0 20 read_log 194377'15462 (0'0) promote ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by osd.330.0:4351 2014-08-12 20:33:19.538997 2014-08-13 18:06:07.011096 7f5c973a57a0 20 read_log 194377'15463 (194377'15462) modify ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by client.2231190.0:4808 2014-08-12 20:33:19.529097 2014-08-13 18:06:07.011108 7f5c973a57a0 20 read_log 194383'15464 (0'0) promote 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by osd.330.0:4377 2014-08-12 20:34:01.736789 2014-08-13 18:06:07.011122 7f5c973a57a0 20 read_log 194383'15465 (194383'15464) modify 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by client.2231190.0:6063 2014-08-12 20:34:01.727784 2014-08-13 18:06:07.011135 7f5c973a57a0 20 read_log 194383'15466 (0'0) promote 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by osd.330.0:4385 2014-08-12 20:34:03.010476 2014-08-13 18:06:07.011155 7f5c973a57a0 20 read_log 194383'15467 (194383'15466) modify 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by client.2231190.0:6297 2014-08-12 20:34:03.000017 2014-08-13 18:06:07.011170 7f5c973a57a0 20 read_log 194393'15468 (194393'15468) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:37:09.025600_2014-08-12 20:52:57.054304/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:57.055717 2014-08-13 18:06:07.011191 7f5c973a57a0 20 read_log 194393'15469 (194215'15402) delete 3f7/hit_set_24.3f7_archive_2014-08-12 15:51:57.156178_2014-08-12 17:24:41.391144/head/.ceph-internal/24 by unknown.0.0:0 2014-08-12 20:52:57.055717 2014-08-13 18:06:07.011206 7f5c973a57a0 20 read_log 194393'15470 (0'0) promote e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by osd.330.0:4447 2014-08-12 20:52:57.062199 2014-08-13 18:06:07.011218 7f5c973a57a0 20 read_log 194393'15471 (194393'15470) modify e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by client.2235069.0:164 2014-08-12 20:52:57.047153 2014-08-13 18:06:07.011233 7f5c973a57a0 20 read_log 194416'15472 (194416'15472) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:52:57.055774_2014-08-13 15:28:32.051663/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 18:06:07.011262 7f5c973a57a0 20 read_log 194416'15473 (194393'15468) delete 3f7/hit_set_24.3f7_archive_2014-08-12 20:37:09.025600_2014-08-12 20:52:57.054304/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 18:06:07.011280 7f5c973a57a0 20 read_log 194416'15474 (0'0) promote 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by osd.330.0:4667 2014-08-13 15:28:32.055676 2014-08-13 18:06:07.011295 7f5c973a57a0 20 read_log 194416'15475 (194416'15474) modify 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by client.2441958.0:426 2014-08-13 15:28:32.043849 2014-08-13 18:06:07.011308 7f5c973a57a0 20 read_log 194416'15476 (0'0) promote e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by osd.330.0:4688 2014-08-13 15:28:37.908509 2014-08-13 18:06:07.011322 7f5c973a57a0 20 read_log 194416'15477 (194416'15476) modify e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by client.2441958.0:1897 2014-08-13 15:28:37.900974 2014-08-13 18:06:07.011340 7f5c973a57a0 20 read_log 194416'15478 (0'0) promote 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by osd.330.0:4695 2014-08-13 15:28:38.677634 2014-08-13 18:06:07.011352 7f5c973a57a0 20 read_log 194416'15479 (194416'15478) modify 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by client.2441958.0:2104 2014-08-13 15:28:38.667281 2014-08-13 18:06:07.011365 7f5c973a57a0 20 read_log 194416'15480 (0'0) promote 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by osd.330.0:4711 2014-08-13 15:28:40.710198 2014-08-13 18:06:07.011380 7f5c973a57a0 20 read_log 194416'15481 (194416'15480) modify 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by client.2441958.0:2635 2014-08-13 15:28:40.701579 2014-08-13 18:06:07.011393 7f5c973a57a0 20 read_log 194416'15482 (0'0) promote facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by osd.330.0:4725 2014-08-13 15:28:42.995238 2014-08-13 18:06:07.011405 7f5c973a57a0 20 read_log 194416'15483 (194416'15482) modify facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by client.2441958.0:3243 2014-08-13 15:28:42.984910 2014-08-13 18:06:07.011429 7f5c973a57a0 20 read_log 194416'15484 (0'0) promote 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by osd.330.0:4741 2014-08-13 15:28:48.997905 2014-08-13 18:06:07.011445 7f5c973a57a0 20 read_log 194416'15485 (194416'15484) modify 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by client.2441958.0:4763 2014-08-13 15:28:48.989745 2014-08-13 18:06:07.011458 7f5c973a57a0 20 read_log 194416'15486 (0'0) promote 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by osd.330.0:4743 2014-08-13 15:28:49.017045 2014-08-13 18:06:07.011478 7f5c973a57a0 20 read_log 194416'15487 (194416'15486) modify 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by client.2441958.0:4768 2014-08-13 15:28:49.007748 2014-08-13 18:06:07.011501 7f5c973a57a0 20 read_log 194416'15488 (0'0) promote 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by osd.330.0:4761 2014-08-13 15:28:54.248645 2014-08-13 18:06:07.011523 7f5c973a57a0 20 read_log 194416'15489 (194416'15488) modify 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by client.2441958.0:6101 2014-08-13 15:28:54.235380 2014-08-13 18:06:07.011539 7f5c973a57a0 20 read_log 194416'15490 (0'0) promote 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by osd.330.0:4769 2014-08-13 15:28:57.994805 2014-08-13 18:06:07.011552 7f5c973a57a0 20 read_log 194416'15491 (194416'15490) modify 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by client.2441958.0:7058 2014-08-13 15:28:57.984296 2014-08-13 18:06:07.011594 7f5c973a57a0 20 read_log 194416'15492 (0'0) promote c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by osd.330.0:4787 2014-08-13 15:29:01.074347 2014-08-13 18:06:07.011623 7f5c973a57a0 20 read_log 194416'15493 (194416'15492) modify c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by client.2441958.0:7878 2014-08-13 15:29:01.065004 2014-08-13 18:06:07.011640 7f5c973a57a0 20 read_log 194416'15494 (0'0) promote f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by osd.330.0:4835 2014-08-13 15:29:19.424557 2014-08-13 18:06:07.011652 7f5c973a57a0 20 read_log 194416'15495 (194416'15494) modify f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by client.2441958.0:12429 2014-08-13 15:29:19.416060 2014-08-13 18:06:07.011665 7f5c973a57a0 20 read_log 194416'15496 (0'0) promote 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by osd.330.0:4841 2014-08-13 15:29:19.815860 2014-08-13 18:06:07.011677 7f5c973a57a0 20 read_log 194416'15497 (194416'15496) modify 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by client.2441958.0:12531 2014-08-13 15:29:19.807708 2014-08-13 18:06:07.011690 7f5c973a57a0 20 read_log 194416'15498 (0'0) promote ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by osd.330.0:4845 2014-08-13 15:29:20.313716 2014-08-13 18:06:07.011702 7f5c973a57a0 20 read_log 194416'15499 (194416'15498) modify ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by client.2441958.0:12639 2014-08-13 15:29:20.303506 2014-08-13 18:06:07.011715 7f5c973a57a0 20 read_log 194416'15500 (0'0) promote 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by osd.330.0:4857 2014-08-13 15:29:23.535708 2014-08-13 18:06:07.011727 7f5c973a57a0 20 read_log 194416'15501 (194416'15500) modify 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by client.2441958.0:13364 2014-08-13 15:29:23.524595 2014-08-13 18:06:07.011739 7f5c973a57a0 20 read_log 194416'15502 (0'0) promote eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by osd.330.0:4859 2014-08-13 15:29:23.583424 2014-08-13 18:06:07.011751 7f5c973a57a0 20 read_log 194416'15503 (194416'15502) modify eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by client.2441958.0:13376 2014-08-13 15:29:23.572377 2014-08-13 18:06:07.011765 7f5c973a57a0 20 read_log 194416'15504 (0'0) promote cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by osd.330.0:4917 2014-08-13 15:29:53.743202 2014-08-13 18:06:07.011777 7f5c973a57a0 20 read_log 194416'15505 (194416'15504) modify cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by client.2441958.0:17451 2014-08-13 15:29:53.732383 2014-08-13 18:06:07.011789 7f5c973a57a0 20 read_log 194416'15506 (0'0) promote e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by osd.330.0:4919 2014-08-13 15:29:56.941801 2014-08-13 18:06:07.011801 7f5c973a57a0 20 read_log 194416'15507 (194416'15506) modify e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by client.2441958.0:17679 2014-08-13 15:29:56.930965 2014-08-13 18:06:07.011853 7f5c973a57a0 20 read_log 194422'15508 (0'0) promote 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by osd.330.0:4937 2014-08-13 15:32:15.288210 2014-08-13 18:06:07.011869 7f5c973a57a0 20 read_log 194422'15509 (194422'15508) modify 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by client.1148203.0:86 2014-08-13 15:32:15.277660 2014-08-13 18:06:07.011882 7f5c973a57a0 20 read_log 194422'15510 (0'0) promote 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by osd.330.0:4941 2014-08-13 15:32:15.817647 2014-08-13 18:06:07.011895 7f5c973a57a0 20 read_log 194422'15511 (194422'15510) modify 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by client.1148203.0:181 2014-08-13 15:32:15.808131 2014-08-13 18:06:07.011907 7f5c973a57a0 20 read_log 194422'15512 (0'0) promote 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by osd.330.0:4955 2014-08-13 15:32:17.902294 2014-08-13 18:06:07.011919 7f5c973a57a0 20 read_log 194422'15513 (194422'15512) modify 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by client.1148203.0:632 2014-08-13 15:32:17.893667 2014-08-13 18:06:07.011932 7f5c973a57a0 20 read_log 194422'15514 (0'0) promote a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by osd.330.0:4957 2014-08-13 15:32:18.112116 2014-08-13 18:06:07.011944 7f5c973a57a0 20 read_log 194422'15515 (194422'15514) modify a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by client.1148203.0:688 2014-08-13 15:32:18.102195 2014-08-13 18:06:07.011956 7f5c973a57a0 20 read_log 194422'15516 (0'0) promote 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by osd.330.0:4973 2014-08-13 15:32:21.622013 2014-08-13 18:06:07.011968 7f5c973a57a0 20 read_log 194422'15517 (194422'15516) modify 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by client.1148203.0:1600 2014-08-13 15:32:21.614021 2014-08-13 18:06:07.011980 7f5c973a57a0 20 read_log 194422'15518 (0'0) promote 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by osd.330.0:4981 2014-08-13 15:32:22.830385 2014-08-13 18:06:07.011992 7f5c973a57a0 20 read_log 194422'15519 (194422'15518) modify 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by client.1148203.0:1897 2014-08-13 15:32:22.815971 2014-08-13 18:06:07.012004 7f5c973a57a0 20 read_log 194422'15520 (0'0) promote 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by osd.330.0:4993 2014-08-13 15:32:25.791724 2014-08-13 18:06:07.012016 7f5c973a57a0 20 read_log 194422'15521 (194422'15520) modify 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by client.1148203.0:2671 2014-08-13 15:32:25.781809 2014-08-13 18:06:07.012029 7f5c973a57a0 20 read_log 194422'15522 (0'0) promote 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by osd.330.0:5007 2014-08-13 15:32:28.016320 2014-08-13 18:06:07.012041 7f5c973a57a0 20 read_log 194422'15523 (194422'15522) modify 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by client.1148203.0:3242 2014-08-13 15:32:28.006489 2014-08-13 18:06:07.012053 7f5c973a57a0 20 read_log 194422'15524 (0'0) promote 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by osd.330.0:5045 2014-08-13 15:32:42.914970 2014-08-13 18:06:07.012067 7f5c973a57a0 20 read_log 194422'15525 (194422'15524) modify 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by client.1148203.0:6512 2014-08-13 15:32:42.904663 2014-08-13 18:06:07.012082 7f5c973a57a0 20 read_log 194422'15526 (0'0) promote 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by osd.330.0:5067 2014-08-13 15:32:51.560170 2014-08-13 18:06:07.012095 7f5c973a57a0 20 read_log 194422'15527 (194422'15526) modify 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by client.1148203.0:7977 2014-08-13 15:32:51.551272 2014-08-13 18:06:07.012151 7f5c973a57a0 20 read_log 194422'15528 (0'0) promote 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by osd.330.0:5079 2014-08-13 15:32:54.854619 2014-08-13 18:06:07.012170 7f5c973a57a0 20 read_log 194422'15529 (194422'15528) modify 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by client.1148203.0:8465 2014-08-13 15:32:54.843415 2014-08-13 18:06:07.012183 7f5c973a57a0 20 read_log 194422'15530 (0'0) promote 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by osd.330.0:5089 2014-08-13 15:32:57.984761 2014-08-13 18:06:07.012196 7f5c973a57a0 20 read_log 194422'15531 (194422'15530) modify 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by client.1148203.0:8838 2014-08-13 15:32:57.973713 2014-08-13 18:06:07.012210 7f5c973a57a0 20 read_log 194422'15532 (0'0) promote 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by osd.330.0:5107 2014-08-13 15:33:03.202134 2014-08-13 18:06:07.012223 7f5c973a57a0 20 read_log 194422'15533 (194422'15532) modify 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by client.1148203.0:9372 2014-08-13 15:33:03.192061 2014-08-13 18:06:07.012235 7f5c973a57a0 20 read_log 194422'15534 (0'0) promote e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by osd.330.0:5111 2014-08-13 15:33:04.009393 2014-08-13 18:06:07.012249 7f5c973a57a0 20 read_log 194422'15535 (194422'15534) modify e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by client.1148203.0:9455 2014-08-13 15:33:03.998920 2014-08-13 18:06:07.012262 7f5c973a57a0 20 read_log 194422'15536 (0'0) promote e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by osd.330.0:5131 2014-08-13 15:33:23.813065 2014-08-13 18:06:07.012274 7f5c973a57a0 20 read_log 194422'15537 (194422'15536) modify e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by client.1148203.0:10495 2014-08-13 15:33:23.805389 2014-08-13 18:06:07.012293 7f5c973a57a0 20 read_log 0 divergent_priors 2014-08-13 18:06:07.012775 7f5c973a57a0 10 read_log checking for missing items over interval (194370'15408,194422'15537] 2014-08-13 18:06:07.012824 7f5c973a57a0 15 read_log missing 194422'15537 (194422'15536) modify e53ec7f7/benchmark_data_storage0111-ib.pocstor_41231_object10494/head//24 by client.1148203.0:10495 2014-08-13 15:33:23.805389 2014-08-13 18:06:07.012865 7f5c973a57a0 15 read_log missing 194422'15535 (194422'15534) modify e623d7f7/benchmark_data_storage0111-ib.pocstor_41231_object9454/head//24 by client.1148203.0:9455 2014-08-13 15:33:03.998920 2014-08-13 18:06:07.012901 7f5c973a57a0 15 read_log missing 194422'15533 (194422'15532) modify 2730fff7/benchmark_data_storage0111-ib.pocstor_41231_object9371/head//24 by client.1148203.0:9372 2014-08-13 15:33:03.192061 2014-08-13 18:06:07.012937 7f5c973a57a0 15 read_log missing 194422'15531 (194422'15530) modify 5dfa47f7/benchmark_data_storage0111-ib.pocstor_41231_object8837/head//24 by client.1148203.0:8838 2014-08-13 15:32:57.973713 2014-08-13 18:06:07.012973 7f5c973a57a0 15 read_log missing 194422'15529 (194422'15528) modify 4045bbf7/benchmark_data_storage0111-ib.pocstor_41231_object8464/head//24 by client.1148203.0:8465 2014-08-13 15:32:54.843415 2014-08-13 18:06:07.013008 7f5c973a57a0 15 read_log missing 194422'15527 (194422'15526) modify 5d92bf7/benchmark_data_storage0111-ib.pocstor_41231_object7976/head//24 by client.1148203.0:7977 2014-08-13 15:32:51.551272 2014-08-13 18:06:07.013044 7f5c973a57a0 15 read_log missing 194422'15525 (194422'15524) modify 4d664bf7/benchmark_data_storage0111-ib.pocstor_41231_object6511/head//24 by client.1148203.0:6512 2014-08-13 15:32:42.904663 2014-08-13 18:06:07.013079 7f5c973a57a0 15 read_log missing 194422'15523 (194422'15522) modify 87a7cff7/benchmark_data_storage0111-ib.pocstor_41231_object3241/head//24 by client.1148203.0:3242 2014-08-13 15:32:28.006489 2014-08-13 18:06:07.013115 7f5c973a57a0 15 read_log missing 194422'15521 (194422'15520) modify 706357f7/benchmark_data_storage0111-ib.pocstor_41231_object2670/head//24 by client.1148203.0:2671 2014-08-13 15:32:25.781809 2014-08-13 18:06:07.013176 7f5c973a57a0 15 read_log missing 194422'15519 (194422'15518) modify 1cfbe3f7/benchmark_data_storage0111-ib.pocstor_41231_object1896/head//24 by client.1148203.0:1897 2014-08-13 15:32:22.815971 2014-08-13 18:06:07.013217 7f5c973a57a0 15 read_log missing 194422'15517 (194422'15516) modify 898caff7/benchmark_data_storage0111-ib.pocstor_41231_object1599/head//24 by client.1148203.0:1600 2014-08-13 15:32:21.614021 2014-08-13 18:06:07.013260 7f5c973a57a0 15 read_log missing 194422'15515 (194422'15514) modify a98387f7/benchmark_data_storage0111-ib.pocstor_41231_object687/head//24 by client.1148203.0:688 2014-08-13 15:32:18.102195 2014-08-13 18:06:07.013300 7f5c973a57a0 15 read_log missing 194422'15513 (194422'15512) modify 754c7f7/benchmark_data_storage0111-ib.pocstor_41231_object631/head//24 by client.1148203.0:632 2014-08-13 15:32:17.893667 2014-08-13 18:06:07.013341 7f5c973a57a0 15 read_log missing 194422'15511 (194422'15510) modify 32fb7bf7/benchmark_data_storage0111-ib.pocstor_41231_object180/head//24 by client.1148203.0:181 2014-08-13 15:32:15.808131 2014-08-13 18:06:07.013380 7f5c973a57a0 15 read_log missing 194422'15509 (194422'15508) modify 8ec017f7/benchmark_data_storage0111-ib.pocstor_41231_object85/head//24 by client.1148203.0:86 2014-08-13 15:32:15.277660 2014-08-13 18:06:07.013428 7f5c973a57a0 15 read_log missing 194416'15507 (194416'15506) modify e529d7f7/benchmark_data_storage0111-ib.pocstor_19103_object17678/head//24 by client.2441958.0:17679 2014-08-13 15:29:56.930965 2014-08-13 18:06:07.013475 7f5c973a57a0 15 read_log missing 194416'15505 (194416'15504) modify cf4b9bf7/benchmark_data_storage0111-ib.pocstor_19103_object17450/head//24 by client.2441958.0:17451 2014-08-13 15:29:53.732383 2014-08-13 18:06:07.013513 7f5c973a57a0 15 read_log missing 194416'15503 (194416'15502) modify eafd0bf7/benchmark_data_storage0111-ib.pocstor_19103_object13375/head//24 by client.2441958.0:13376 2014-08-13 15:29:23.572377 2014-08-13 18:06:07.013549 7f5c973a57a0 15 read_log missing 194416'15501 (194416'15500) modify 3aa37bf7/benchmark_data_storage0111-ib.pocstor_19103_object13363/head//24 by client.2441958.0:13364 2014-08-13 15:29:23.524595 2014-08-13 18:06:07.013585 7f5c973a57a0 15 read_log missing 194416'15499 (194416'15498) modify ab7f7ff7/benchmark_data_storage0111-ib.pocstor_19103_object12638/head//24 by client.2441958.0:12639 2014-08-13 15:29:20.303506 2014-08-13 18:06:07.013620 7f5c973a57a0 15 read_log missing 194416'15497 (194416'15496) modify 708dcff7/benchmark_data_storage0111-ib.pocstor_19103_object12530/head//24 by client.2441958.0:12531 2014-08-13 15:29:19.807708 2014-08-13 18:06:07.013656 7f5c973a57a0 15 read_log missing 194416'15495 (194416'15494) modify f5e34bf7/benchmark_data_storage0111-ib.pocstor_19103_object12428/head//24 by client.2441958.0:12429 2014-08-13 15:29:19.416060 2014-08-13 18:06:07.013692 7f5c973a57a0 15 read_log missing 194416'15493 (194416'15492) modify c26c6bf7/benchmark_data_storage0111-ib.pocstor_19103_object7877/head//24 by client.2441958.0:7878 2014-08-13 15:29:01.065004 2014-08-13 18:06:07.013728 7f5c973a57a0 15 read_log missing 194416'15491 (194416'15490) modify 169a57f7/benchmark_data_storage0111-ib.pocstor_19103_object7057/head//24 by client.2441958.0:7058 2014-08-13 15:28:57.984296 2014-08-13 18:06:07.013764 7f5c973a57a0 15 read_log missing 194416'15489 (194416'15488) modify 90d157f7/benchmark_data_storage0111-ib.pocstor_19103_object6100/head//24 by client.2441958.0:6101 2014-08-13 15:28:54.235380 2014-08-13 18:06:07.013800 7f5c973a57a0 15 read_log missing 194416'15487 (194416'15486) modify 3b59cbf7/benchmark_data_storage0111-ib.pocstor_19103_object4767/head//24 by client.2441958.0:4768 2014-08-13 15:28:49.007748 2014-08-13 18:06:07.013836 7f5c973a57a0 15 read_log missing 194416'15485 (194416'15484) modify 61e287f7/benchmark_data_storage0111-ib.pocstor_19103_object4762/head//24 by client.2441958.0:4763 2014-08-13 15:28:48.989745 2014-08-13 18:06:07.013871 7f5c973a57a0 15 read_log missing 194416'15483 (194416'15482) modify facf7f7/benchmark_data_storage0111-ib.pocstor_19103_object3242/head//24 by client.2441958.0:3243 2014-08-13 15:28:42.984910 2014-08-13 18:06:07.013907 7f5c973a57a0 15 read_log missing 194416'15481 (194416'15480) modify 673f97f7/benchmark_data_storage0111-ib.pocstor_19103_object2634/head//24 by client.2441958.0:2635 2014-08-13 15:28:40.701579 2014-08-13 18:06:07.013943 7f5c973a57a0 15 read_log missing 194416'15479 (194416'15478) modify 3fdfdff7/benchmark_data_storage0111-ib.pocstor_19103_object2103/head//24 by client.2441958.0:2104 2014-08-13 15:28:38.667281 2014-08-13 18:06:07.013978 7f5c973a57a0 15 read_log missing 194416'15477 (194416'15476) modify e5977bf7/benchmark_data_storage0111-ib.pocstor_19103_object1896/head//24 by client.2441958.0:1897 2014-08-13 15:28:37.900974 2014-08-13 18:06:07.014014 7f5c973a57a0 15 read_log missing 194416'15475 (194416'15474) modify 4314a7f7/benchmark_data_storage0111-ib.pocstor_19103_object425/head//24 by client.2441958.0:426 2014-08-13 15:28:32.043849 2014-08-13 18:06:07.014053 7f5c973a57a0 15 read_log missing 194416'15472 (194416'15472) modify 3f7/hit_set_24.3f7_archive_2014-08-12 20:52:57.055774_2014-08-13 15:28:32.051663/head/.ceph-internal/24 by unknown.0.0:0 2014-08-13 15:28:32.051921 2014-08-13 18:06:07.014088 7f5c973a57a0 15 read_log missing 194393'15471 (194393'15470) modify e8a8b3f7/benchmark_data_storage0101-ib.pocstor_6819_object163/head//24 by client.2235069.0:164 2014-08-12 20:52:57.047153 2014-08-13 18:06:07.014128 7f5c973a57a0 15 read_log missing 194383'15467 (194383'15466) modify 979337f7/benchmark_data_storage0111-ib.pocstor_25065_object6296/head//24 by client.2231190.0:6297 2014-08-12 20:34:03.000017 2014-08-13 18:06:07.014179 7f5c973a57a0 15 read_log missing 194383'15465 (194383'15464) modify 2db2dff7/benchmark_data_storage0111-ib.pocstor_25065_object6062/head//24 by client.2231190.0:6063 2014-08-12 20:34:01.727784 2014-08-13 18:06:07.014217 7f5c973a57a0 15 read_log missing 194377'15463 (194377'15462) modify ffc657f7/benchmark_data_storage0111-ib.pocstor_25065_object4807/head//24 by client.2231190.0:4808 2014-08-12 20:33:19.529097 2014-08-13 18:06:07.014264 7f5c973a57a0 15 read_log missing 194377'15461 (194377'15460) modify b1765bf7/benchmark_data_storage0111-ib.pocstor_25065_object4698/head//24 by client.2231190.0:4699 2014-08-12 20:33:18.617965 2014-08-13 18:06:07.014306 7f5c973a57a0 15 read_log missing 194377'15459 (194377'15458) modify a71aa7f7/benchmark_data_storage0111-ib.pocstor_25065_object4157/head//24 by client.2231190.0:4158 2014-08-12 20:33:14.826432 2014-08-13 18:06:07.014342 7f5c973a57a0 15 read_log missing 194377'15457 (194377'15456) modify a32457f7/benchmark_data_storage0111-ib.pocstor_25065_object4154/head//24 by client.2231190.0:4155 2014-08-12 20:33:14.812906 2014-08-13 18:06:07.014378 7f5c973a57a0 15 read_log missing 194377'15455 (194377'15454) modify afc72bf7/benchmark_data_storage0111-ib.pocstor_25065_object4018/head//24 by client.2231190.0:4019 2014-08-12 20:33:14.019939 2014-08-13 18:06:07.014424 7f5c973a57a0 15 read_log missing 194377'15453 (194377'15452) modify b06ebbf7/benchmark_data_storage0111-ib.pocstor_25065_object3529/head//24 by client.2231190.0:3530 2014-08-12 20:33:11.499104 2014-08-13 18:06:07.014478 7f5c973a57a0 15 read_log missing 194377'15451 (194377'15450) modify a455fbf7/benchmark_data_storage0111-ib.pocstor_25065_object2494/head//24 by client.2231190.0:2495 2014-08-12 20:33:07.191237 2014-08-13 18:06:07.014525 7f5c973a57a0 15 read_log missing 194377'15449 (194377'15448) modify 6aeb53f7/benchmark_data_storage0111-ib.pocstor_25065_object1992/head//24 by client.2231190.0:1993 2014-08-12 20:33:05.205475 2014-08-13 18:06:07.014563 7f5c973a57a0 15 read_log missing 194377'15447 (194377'15446) modify 128ef3f7/benchmark_data_storage0111-ib.pocstor_25065_object1937/head//24 by client.2231190.0:1938 2014-08-12 20:33:04.934317 2014-08-13 18:06:07.014599 7f5c973a57a0 15 read_log missing 194377'15445 (194377'15444) modify 13dcebf7/benchmark_data_storage0111-ib.pocstor_25065_object786/head//24 by client.2231190.0:787 2014-08-12 20:33:00.502816 2014-08-13 18:06:07.014635 7f5c973a57a0 15 read_log missing 194377'15443 (194377'15442) modify d70483f7/benchmark_data_storage0111-ib.pocstor_25065_object184/head//24 by client.2231190.0:185 2014-08-12 20:32:57.934951 2014-08-13 18:06:07.014671 7f5c973a57a0 15 read_log missing 194370'15441 (194370'15440) modify fa72fff7/benchmark_data_storage0111-ib.pocstor_12495_object12716/head//24 by client.1508531.0:12717 2014-08-12 20:31:16.677145 2014-08-13 18:06:07.014707 7f5c973a57a0 15 read_log missing 194370'15439 (194370'15438) modify 40be73f7/benchmark_data_storage0111-ib.pocstor_12495_object12588/head//24 by client.1508531.0:12589 2014-08-12 20:31:15.617565 2014-08-13 18:06:07.014743 7f5c973a57a0 15 read_log missing 194370'15437 (194370'15436) modify ec52c3f7/benchmark_data_storage0111-ib.pocstor_12495_object8721/head//24 by client.1508531.0:8722 2014-08-12 20:30:51.577344 2014-08-13 18:06:07.014778 7f5c973a57a0 15 read_log missing 194370'15435 (194370'15434) modify ea8fb7f7/benchmark_data_storage0111-ib.pocstor_12495_object8517/head//24 by client.1508531.0:8518 2014-08-12 20:30:50.366822 2014-08-13 18:06:07.014814 7f5c973a57a0 15 read_log missing 194370'15433 (194370'15432) modify a7a5b7f7/benchmark_data_storage0111-ib.pocstor_12495_object7710/head//24 by client.1508531.0:7711 2014-08-12 20:30:46.018440 2014-08-13 18:06:07.014850 7f5c973a57a0 15 read_log missing 194370'15431 (194370'15430) modify de22bf7/benchmark_data_storage0111-ib.pocstor_12495_object7687/head//24 by client.1508531.0:7688 2014-08-12 20:30:45.885740 2014-08-13 18:06:07.014886 7f5c973a57a0 15 read_log missing 194370'15429 (194370'15428) modify 43b21ff7/benchmark_data_storage0111-ib.pocstor_12495_object7490/head//24 by client.1508531.0:7491 2014-08-12 20:30:44.809735 2014-08-13 18:06:07.014922 7f5c973a57a0 15 read_log missing 194370'15427 (194370'15426) modify 4b5e6ff7/benchmark_data_storage0111-ib.pocstor_12495_object7462/head//24 by client.1508531.0:7463 2014-08-12 20:30:44.671825 2014-08-13 18:06:07.014958 7f5c973a57a0 15 read_log missing 194370'15425 (194370'15424) modify 8073bf7/benchmark_data_storage0111-ib.pocstor_12495_object6698/head//24 by client.1508531.0:6699 2014-08-12 20:30:40.872634 2014-08-13 18:06:07.014994 7f5c973a57a0 15 read_log missing 194370'15423 (194370'15422) modify 694573f7/benchmark_data_storage0111-ib.pocstor_12495_object6693/head//24 by client.1508531.0:6694 2014-08-12 20:30:40.851596 2014-08-13 18:06:07.015030 7f5c973a57a0 15 read_log missing 194370'15421 (194370'15420) modify c1e103f7/benchmark_data_storage0111-ib.pocstor_12495_object5501/head//24 by client.1508531.0:5502 2014-08-12 20:30:35.399089 2014-08-13 18:06:07.015065 7f5c973a57a0 15 read_log missing 194370'15419 (194370'15418) modify f46767f7/benchmark_data_storage0111-ib.pocstor_12495_object4825/head//24 by client.1508531.0:4826 2014-08-12 20:30:32.478351 2014-08-13 18:06:07.015101 7f5c973a57a0 15 read_log missing 194370'15417 (194370'15416) modify d44227f7/benchmark_data_storage0111-ib.pocstor_12495_object4046/head//24 by client.1508531.0:4047 2014-08-12 20:30:29.463943 2014-08-13 18:06:07.015137 7f5c973a57a0 15 read_log missing 194370'15415 (194370'15414) modify 190a47f7/benchmark_data_storage0111-ib.pocstor_12495_object2851/head//24 by client.1508531.0:2852 2014-08-12 20:30:25.000236 2014-08-13 18:06:07.015179 7f5c973a57a0 15 read_log missing 194370'15413 (194370'15412) modify 4678abf7/benchmark_data_storage0111-ib.pocstor_12495_object2798/head//24 by client.1508531.0:2799 2014-08-12 20:30:24.806873 2014-08-13 18:06:07.015215 7f5c973a57a0 15 read_log missing 194370'15411 (194370'15410) modify 9c8883f7/benchmark_data_storage0111-ib.pocstor_12495_object1994/head//24 by client.1508531.0:1995 2014-08-12 20:30:21.874345 2014-08-13 18:06:07.015251 7f5c973a57a0 15 read_log missing 194370'15409 (194370'15408) modify 5bea7f7/benchmark_data_storage0111-ib.pocstor_12495_object202/head//24 by client.1508531.0:203 2014-08-12 20:30:14.929515 2014-08-13 18:06:07.015263 7f5c973a57a0 10 read_log done 2014-08-13 18:06:07.015293 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive m=64] handle_loaded 2014-08-13 18:06:07.015311 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Initial 0.016092 0 0.000000 2014-08-13 18:06:07.015325 7f5c973a57a0 5 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset 2014-08-13 18:06:07.015334 7f5c973a57a0 20 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=0 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194487 2014-08-13 18:06:07.015344 7f5c973a57a0 10 osd.4 194487 load_pgs loaded pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] log((192718'12872,194422'15537], crt=194422'15537) 2014-08-13 18:06:07.015358 7f5c973a57a0 0 osd.4 194487 load_pgs opened 74 pgs 2014-08-13 18:06:07.015364 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.30b(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015372 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.220(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015376 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.362(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015381 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.28f(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015386 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.14a(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015390 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.22c(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015393 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.619(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015396 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.30f(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015399 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.9b(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015402 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.581(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015406 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.f3(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015409 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.116(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015412 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3d(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015415 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4aa(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015418 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1d2(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015422 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1de(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015427 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[2.2a(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015431 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.113(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015434 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.39a(unlocked)] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.015438 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.33a(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015441 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[1.2b(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015444 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.34(unlocked)] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.015447 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.99(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015450 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.84(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015463 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4df(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015467 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.39d(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015470 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2b9(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015473 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.2c(unlocked)] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.015476 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3f7(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015479 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.775(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015482 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.630(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015485 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.e2(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015488 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.69f(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015492 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3e4(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015494 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2a1(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015498 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.639(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015505 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.707(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015509 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.5e(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015513 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1f1(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015516 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2d1(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015519 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.b6(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015522 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.3bf(unlocked)] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.015525 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1fb(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015527 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.350(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015530 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1f7(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015533 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.136(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015537 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4fe(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015539 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.653(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015542 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3b7(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015545 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3b9(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015548 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.791(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015551 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.6bd(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015555 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.181(unlocked)] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.015558 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.20e(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015561 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e4(unlocked)] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.015564 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e6(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015567 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.12a(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015570 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3af(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015573 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.481(unlocked)] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.015576 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.48e(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015579 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3ad(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015582 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.206(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015586 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1bb(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015589 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.299(unlocked)] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.015592 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d5(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015595 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.537(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015598 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1b8(unlocked)] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.015602 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.29a(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015605 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.cd(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015608 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.226(unlocked)] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.015611 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d9(unlocked)] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.015613 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2ec(unlocked)] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.015616 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1a0(unlocked)] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.015619 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.369(unlocked)] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.015622 7f5c973a57a0 10 osd.4 194487 build_past_intervals_parallel nothing to build 2014-08-13 18:06:07.015648 7f5c973a57a0 2 osd.4 194487 superblock: i am osd.4 2014-08-13 18:06:07.015653 7f5c973a57a0 10 osd.4 194487 create_logger 2014-08-13 18:06:07.016466 7f5c795df700 20 osd.4 194487 update_osd_stat osd_stat(8088 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) 2014-08-13 18:06:07.016486 7f5c795df700 5 osd.4 194487 heartbeat: osd_stat(8088 MB used, 2783 GB avail, 2791 GB total, peers []/[] op hist []) 2014-08-13 18:06:07.016754 7f5c758d8700 10 osd.4 194487 agent_entry start 2014-08-13 18:06:07.016765 7f5c758d8700 20 osd.4 194487 agent_entry empty queue 2014-08-13 18:06:07.017092 7f5c76ddb700 10 osd.4 194487 OSD::ms_get_authorizer type=mon 2014-08-13 18:06:07.017302 7f5c849f1700 10 osd.4 194487 ms_handle_connect on mon 2014-08-13 18:06:07.017314 7f5c849f1700 10 osd.4 194487 send_alive up_thru currently 194486 want 0 2014-08-13 18:06:07.017320 7f5c849f1700 20 osd.4 194487 send_pg_stats 2014-08-13 18:06:07.023544 7f5c849f1700 10 osd.4 194487 do_waiters -- start 2014-08-13 18:06:07.023554 7f5c849f1700 10 osd.4 194487 do_waiters -- finish 2014-08-13 18:06:07.023559 7f5c849f1700 20 osd.4 194487 _dispatch 0x2000fc0 osd_map(194494..194494 src has 185139..194494) v3 2014-08-13 18:06:07.023570 7f5c849f1700 0 osd.4 194487 ignoring osdmap until we have initialized 2014-08-13 18:06:07.023573 7f5c849f1700 10 osd.4 194487 do_waiters -- start 2014-08-13 18:06:07.023576 7f5c849f1700 10 osd.4 194487 do_waiters -- finish 2014-08-13 18:06:07.024643 7f5c849f1700 10 osd.4 194487 do_waiters -- start 2014-08-13 18:06:07.024647 7f5c849f1700 10 osd.4 194487 do_waiters -- finish 2014-08-13 18:06:07.024649 7f5c849f1700 20 osd.4 194487 _dispatch 0x2002ac0 osd_map(194494..194494 src has 185139..194494) v3 2014-08-13 18:06:07.024667 7f5c849f1700 0 osd.4 194487 ignoring osdmap until we have initialized 2014-08-13 18:06:07.024670 7f5c849f1700 10 osd.4 194487 do_waiters -- start 2014-08-13 18:06:07.024672 7f5c849f1700 10 osd.4 194487 do_waiters -- finish 2014-08-13 18:06:07.024881 7f5c973a57a0 10 osd.4 194487 ensuring pgs have consumed prior maps 2014-08-13 18:06:07.024903 7f5c973a57a0 7 osd.4 194487 consume_map version 194487 2014-08-13 18:06:07.024940 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 18:06:07.024967 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.024983 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.024995 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.024984 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.025008 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025019 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.025031 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025044 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025056 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025052 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 18:06:07.025071 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:07.025083 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.025084 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025091 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.025102 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025104 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.025113 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025120 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.025124 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] null 2014-08-13 18:06:07.025135 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.025138 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025139 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 18:06:07.025160 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.025160 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025165 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.025172 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 18:06:07.025183 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.025186 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025194 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.025198 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025207 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.025210 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:07.025222 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 18:06:07.025251 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.025225 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.025263 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.025268 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025271 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.025277 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.565668 2 0.000073 2014-08-13 18:06:07.025281 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025288 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.552406 2 0.000077 2014-08-13 18:06:07.025292 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.025297 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025304 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.025308 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.025311 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025316 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.025318 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.025324 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025328 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.025337 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025338 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 2014-08-13 18:06:07.025339 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000023 0 0.000000 2014-08-13 18:06:07.025350 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025352 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.025353 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.025363 7f5c973a57a0 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.025370 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.025373 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.025375 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 18:06:07.025383 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.025384 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.025392 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025399 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.025399 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.025404 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025408 7f5c74dd6700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.025415 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 18:06:07.025429 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025444 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025455 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:07.025466 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025478 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025489 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 18:06:07.025495 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 18:06:07.025504 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025515 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.025517 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025519 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.025530 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.025529 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:07.025540 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.025550 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.025545 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 18:06:07.025565 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.025562 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,301,324 2014-08-13 18:06:07.025572 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.025569 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.025542 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025573 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.025579 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.025584 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025585 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.548370 2 0.000063 2014-08-13 18:06:07.025592 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.025598 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.025599 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 18:06:07.025605 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.025611 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.025613 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 18:06:07.025618 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,58,343 2014-08-13 18:06:07.025620 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.025629 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.025631 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.025631 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 18:06:07.025631 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.025643 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.535108 2 0.000077 2014-08-13 18:06:07.025652 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.025657 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.025648 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025665 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.025667 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.025675 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025678 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.025678 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.025690 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000023 0 0.000000 2014-08-13 18:06:07.025694 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.025689 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.025702 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.025711 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025713 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.025709 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 18:06:07.025723 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.025723 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025725 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 18:06:07.025733 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.025736 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025737 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,302,324,368 down blocked_by {} 2014-08-13 18:06:07.025746 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 18:06:07.025749 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.025750 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] null 2014-08-13 18:06:07.025761 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) 2014-08-13 18:06:07.025764 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.025769 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2830 2014-08-13 18:06:07.025772 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,343,349,362 down blocked_by {} 2014-08-13 18:06:07.025777 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 18:06:07.025783 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.301 2014-08-13 18:06:07.025784 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.025798 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.302 2014-08-13 18:06:07.025798 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2681 2014-08-13 18:06:07.025791 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:07.025809 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 18:06:07.025811 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 18:06:07.025820 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 18:06:07.025822 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025823 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.025834 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 18:06:07.025834 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.025835 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025848 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.025849 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 18:06:07.025853 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025864 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.025862 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 18:06:07.025872 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025874 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.025884 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.025884 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025902 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.025899 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 18:06:07.025910 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 18:06:07.025930 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.025939 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 18:06:07.025962 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.025921 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.025965 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.025942 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.025976 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.025986 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.025996 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.025991 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 18:06:07.026009 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.026007 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,62,65 2014-08-13 18:06:07.026017 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.026012 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.026004 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.026019 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.026023 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.026028 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.026030 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.503669 2 0.000062 2014-08-13 18:06:07.026034 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.026041 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.026044 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.026047 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.026054 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.026055 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 18:06:07.026063 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.026074 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.026079 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.026085 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.026091 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.026096 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.026098 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.026102 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.546481 2 0.000066 2014-08-13 18:06:07.026107 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.026115 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.026114 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.026118 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.026125 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.026120 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.026131 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 18:06:07.026136 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.026144 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 18:06:07.026147 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 2014-08-13 18:06:07.026155 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,48,62,65,314 down blocked_by {} 2014-08-13 18:06:07.026159 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.026172 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.026166 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.026153 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 18:06:07.026188 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2733 2014-08-13 18:06:07.026181 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026200 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026196 7f5c973a57a0 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.026199 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 18:06:07.026208 7f5c973a57a0 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:07.026211 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.48 2014-08-13 18:06:07.026220 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.62 2014-08-13 18:06:07.026233 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.65 2014-08-13 18:06:07.026243 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.314 2014-08-13 18:06:07.026253 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.026262 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026267 7f5c74cd5700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.026274 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026323 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 18:06:07.026340 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.026344 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.026353 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.026350 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 18:06:07.026368 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.026363 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.026372 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.026373 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.026381 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.026390 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.026400 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.026405 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.026415 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.026424 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.571629 2 0.000060 2014-08-13 18:06:07.026434 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.026436 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.026443 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.026446 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.026452 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.026456 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.537996 2 0.000061 2014-08-13 18:06:07.026466 7f5c748d1700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.026463 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.026469 7f5c749d2700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.026468 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.026476 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.026478 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.026487 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.026488 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.026498 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026499 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.026509 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026510 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.026521 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.026531 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026544 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026583 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026608 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026611 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026630 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026676 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 18:06:07.026693 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.026697 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.026706 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.026710 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 18:06:07.026719 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.026730 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.026730 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.026733 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.026742 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.026763 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.026752 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.026773 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.026783 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.515068 2 0.000066 2014-08-13 18:06:07.026784 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.026794 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.026803 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.026810 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.026820 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000 2014-08-13 18:06:07.026823 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.026830 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.026834 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.026842 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.026844 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.506659 2 0.000082 2014-08-13 18:06:07.026852 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.026856 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.026862 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.026865 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.026874 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.026884 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.026895 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.026991 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 18:06:07.027009 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.027012 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.027021 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.027031 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.027042 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.027052 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,61,349 2014-08-13 18:06:07.027061 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.027063 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.027063 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.027072 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.561362 2 0.000062 2014-08-13 18:06:07.027083 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.027092 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.027100 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.027110 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000017 0 0.000000 2014-08-13 18:06:07.027120 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.027133 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.027144 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.027153 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.027163 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 18:06:07.027176 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 18:06:07.027192 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,330,331,349 down blocked_by {} 2014-08-13 18:06:07.027203 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.027234 7f5c747d0700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027215 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:3020 2014-08-13 18:06:07.027234 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.027246 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 18:06:07.027256 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.027257 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.027264 7f5c743cc700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027268 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.027267 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.330 2014-08-13 18:06:07.027277 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.331 2014-08-13 18:06:07.027286 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 18:06:07.027296 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.027296 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.027310 7f5c745ce700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027306 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.027317 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.027167 7f5c746cf700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027448 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 18:06:07.027466 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.027469 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.027467 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.027479 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.027490 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.027500 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.027511 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,285,312 2014-08-13 18:06:07.027520 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.027518 7f5c744cd700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027522 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.027514 7f5c740c9700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027531 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.576772 2 0.000062 2014-08-13 18:06:07.027541 7f5c74bd4700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027541 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.027550 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.027558 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.027568 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.027579 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.027589 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.027597 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.027605 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.027615 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 18:06:07.027626 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 18:06:07.027636 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,311,312,369 down blocked_by {} 2014-08-13 18:06:07.027646 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.027657 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2860 2014-08-13 18:06:07.027666 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.285 2014-08-13 18:06:07.027676 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.311 2014-08-13 18:06:07.027686 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.312 2014-08-13 18:06:07.027690 7f5c742cb700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027695 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.027705 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.027714 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.027724 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.027845 7f5c73dc6700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.027927 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.027950 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.027987 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.028012 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.028107 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 18:06:07.028128 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.028132 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.028141 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.028151 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.028161 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.028195 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.028211 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.028220 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.497106 2 0.000062 2014-08-13 18:06:07.028232 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.028245 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.028253 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.028264 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.028272 7f5c73cc5700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.028275 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.028288 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.028297 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.028307 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.028487 7f5c738c1700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.028495 7f5c736bf700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.028533 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.028609 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:07.028630 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.028634 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.028646 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.028658 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.028668 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.028677 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340 2014-08-13 18:06:07.028687 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.028689 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.028697 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.589350 2 0.000086 2014-08-13 18:06:07.028708 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.028716 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.028711 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.028724 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.028753 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000037 0 0.000000 2014-08-13 18:06:07.028771 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.028785 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.028806 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.028816 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.028825 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340 down blocked_by {} 2014-08-13 18:06:07.028834 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.028844 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:176853 2014-08-13 18:06:07.028853 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.028864 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.028872 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.028882 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.029010 7f5c734bd700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.029077 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 18:06:07.029097 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.029101 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.029112 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.029122 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.029132 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.029124 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 18:06:07.029145 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.029143 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,313,318 2014-08-13 18:06:07.029154 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.029150 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.029156 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.029161 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.029166 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.523827 2 0.000066 2014-08-13 18:06:07.029172 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.029179 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.029189 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.029183 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.029201 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,45,339 2014-08-13 18:06:07.029211 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.029198 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.029213 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.029215 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000026 0 0.000000 2014-08-13 18:06:07.029223 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.576380 2 0.000065 2014-08-13 18:06:07.029227 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.029236 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.029237 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.029246 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.029246 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.029255 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.029257 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.029268 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.029268 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 18:06:07.029281 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.029282 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) 2014-08-13 18:06:07.029291 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.029292 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,303,313,318,342 down blocked_by {} 2014-08-13 18:06:07.029301 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.029303 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.029311 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.029316 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2705 2014-08-13 18:06:07.029323 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 18:06:07.029327 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 18:06:07.029337 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) 2014-08-13 18:06:07.029340 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.029348 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,24,45,55,339,342 down blocked_by {} 2014-08-13 18:06:07.029350 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.313 2014-08-13 18:06:07.029356 7f5c72fb8700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.029359 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.029361 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.318 2014-08-13 18:06:07.029371 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.029372 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2933 2014-08-13 18:06:07.029387 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.24 2014-08-13 18:06:07.029386 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.029403 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 18:06:07.029403 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.029402 7f5c72eb7700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.029419 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.029417 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.55 2014-08-13 18:06:07.029426 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 18:06:07.029435 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.029445 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.029454 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.029467 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.029527 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.029549 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.029570 7f5c7c7e4700 20 osd.4 194487 get_map 194471 - loading and decoding 0x2019a00 2014-08-13 18:06:07.029920 7f5c728b1700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.029946 7f5c72bb4700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.029970 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194471 268447 bytes 2014-08-13 18:06:07.029980 7f5c729b2700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.030539 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.032474 7f5c7c7e4700 10 osd.4 pg_epoch: 194470 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.032526 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.032530 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.032526 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 18:06:07.032541 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.032552 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.032553 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.032566 7f5c7c7e4700 20 osd.4 194487 get_map 194472 - loading and decoding 0x2e00200 2014-08-13 18:06:07.032556 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.032585 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.032597 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.032608 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.032644 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.032653 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.032662 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.506083 2 0.000084 2014-08-13 18:06:07.032674 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.032683 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.032691 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.032702 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.032715 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.032727 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.032737 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.032747 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.032856 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194472 268447 bytes 2014-08-13 18:06:07.033234 7f5c721aa700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.033277 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.035170 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.035194 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.035180 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 18:06:07.035205 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.035199 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.035209 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.035209 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.035219 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.035220 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.035232 7f5c7c7e4700 20 osd.4 194487 get_map 194473 - loading and decoding 0x2e00000 2014-08-13 18:06:07.035231 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.035242 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.035252 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,21,310 2014-08-13 18:06:07.035262 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.035263 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.035273 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.500205 2 0.000075 2014-08-13 18:06:07.035284 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.035294 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.035302 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.035312 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.035323 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.035332 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.035340 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.035350 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.035360 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 18:06:07.035372 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 18:06:07.035384 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,15,21,303,310,342 down blocked_by {} 2014-08-13 18:06:07.035394 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.035405 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2569 2014-08-13 18:06:07.035415 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.15 2014-08-13 18:06:07.035425 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.21 2014-08-13 18:06:07.035435 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.035444 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.310 2014-08-13 18:06:07.035448 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194473 268447 bytes 2014-08-13 18:06:07.035453 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.035462 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.035471 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.035482 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.035572 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.035591 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.035988 7f5c71fa8700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.037697 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.037720 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.037708 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 18:06:07.037729 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.037724 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.037733 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.037733 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.037744 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.037745 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.037758 7f5c7c7e4700 20 osd.4 194487 get_map 194474 - loading and decoding 0x2e00800 2014-08-13 18:06:07.037754 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.037765 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.037776 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,5,47 2014-08-13 18:06:07.037785 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.037786 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.037796 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.569801 2 0.000063 2014-08-13 18:06:07.037807 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.037817 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.037825 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.037835 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.037846 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.037855 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.037864 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.037872 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.037882 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 18:06:07.037893 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) 2014-08-13 18:06:07.037903 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,5,11,47,54,368 down blocked_by {} 2014-08-13 18:06:07.037913 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.037924 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:3006 2014-08-13 18:06:07.037934 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.5 2014-08-13 18:06:07.037945 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.11 2014-08-13 18:06:07.037954 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.47 2014-08-13 18:06:07.037966 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194474 268950 bytes 2014-08-13 18:06:07.037962 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.54 2014-08-13 18:06:07.037971 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 18:06:07.037981 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.037991 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.038001 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.038380 7f5c71da6700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.040416 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.040451 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.040439 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 18:06:07.040459 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.040455 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.040463 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.040465 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.040473 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.040475 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.040483 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.040489 7f5c7c7e4700 20 osd.4 194487 get_map 194475 - loading and decoding 0x2e00600 2014-08-13 18:06:07.040493 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.040528 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.040538 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.040547 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.554029 2 0.000066 2014-08-13 18:06:07.040558 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.040566 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.040575 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.040585 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.040595 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.040607 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.040616 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.040626 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.040704 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.040720 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194475 268950 bytes 2014-08-13 18:06:07.040724 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.041135 7f5c71ba4700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.041241 7f5c71aa3700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.041593 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.042999 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.043035 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.043028 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 18:06:07.043039 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.043052 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.043049 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.043059 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.043057 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.043072 7f5c7c7e4700 20 osd.4 194487 get_map 194476 - loading and decoding 0x2e00400 2014-08-13 18:06:07.043068 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.043079 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.043089 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.043115 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.043125 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.043134 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.044115 2 0.000067 2014-08-13 18:06:07.043152 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.043163 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.043172 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.043183 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.043195 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.043206 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.043216 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.043227 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.043299 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194476 268950 bytes 2014-08-13 18:06:07.043541 7f5c717a0700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.043712 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.045559 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.045597 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.045586 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 18:06:07.045607 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.045601 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.045611 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.045611 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.045621 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.045621 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.045635 7f5c7c7e4700 20 osd.4 194487 get_map 194477 - loading and decoding 0x2e01000 2014-08-13 18:06:07.045632 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.045642 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.045680 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.045690 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.045699 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.575137 2 0.000062 2014-08-13 18:06:07.045710 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.045718 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.045727 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.045737 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.045748 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.045761 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.045771 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.045782 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.045852 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.045871 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.045911 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194477 268447 bytes 2014-08-13 18:06:07.045932 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.045966 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.046079 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.046101 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.046264 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.046288 7f5c7159e700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.046370 7f5c7149d700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.046501 7f5c7139c700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.046581 7f5c7129b700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.047908 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.047931 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.047920 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 18:06:07.047941 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.047935 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.047945 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.047945 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.047955 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.047955 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.047969 7f5c7c7e4700 20 osd.4 194487 get_map 194478 - loading and decoding 0x2e00e00 2014-08-13 18:06:07.047966 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.047977 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.048010 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.048019 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.048028 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.532008 2 0.000063 2014-08-13 18:06:07.048039 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.048048 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.048056 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.048066 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.048076 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.048089 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.048099 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.048109 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.048200 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194478 268447 bytes 2014-08-13 18:06:07.048704 7f5c70d96700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.048738 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.050717 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.050754 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.050740 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 18:06:07.050762 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.050758 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.050766 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.050768 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.050777 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.050779 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.050791 7f5c7c7e4700 20 osd.4 194487 get_map 194479 - loading and decoding 0x2e00c00 2014-08-13 18:06:07.050787 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.050797 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.050834 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.050843 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.050852 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.522262 2 0.000065 2014-08-13 18:06:07.050863 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.050872 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.050879 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.050889 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000 2014-08-13 18:06:07.050919 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.050931 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.050940 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.050950 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.051023 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194479 268447 bytes 2014-08-13 18:06:07.051017 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.051035 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.051379 7f5c70a93700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.051438 7f5c70b94700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.051474 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.053448 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.053494 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.053498 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.053493 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 18:06:07.053508 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.053518 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.053518 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.053531 7f5c7c7e4700 20 osd.4 194487 get_map 194480 - loading and decoding 0x2e00a00 2014-08-13 18:06:07.053522 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.053540 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.053551 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.053564 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.053597 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.053606 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.053616 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.554262 2 0.000072 2014-08-13 18:06:07.053628 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.053637 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.053645 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.053655 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.053666 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.053677 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.053686 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.053697 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.053767 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194480 268851 bytes 2014-08-13 18:06:07.054301 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.054334 7f5c70790700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.056081 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.056104 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.056095 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 18:06:07.056108 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.056120 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.056120 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.056125 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.056132 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.056135 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.056145 7f5c7c7e4700 20 osd.4 194487 get_map 194481 - loading and decoding 0x2e01c00 2014-08-13 18:06:07.056151 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.056162 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.056198 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.056207 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.056216 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.517207 2 0.000084 2014-08-13 18:06:07.056228 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.056236 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.056244 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.056255 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.056265 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.056277 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.056286 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.056297 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.056491 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194481 268851 bytes 2014-08-13 18:06:07.056867 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.056922 7f5c7058e700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.058200 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.058225 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.058238 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.058233 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 18:06:07.058251 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.058260 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.058262 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.058264 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.058275 7f5c7c7e4700 20 osd.4 194487 get_map 194482 - loading and decoding 0x2e01a00 2014-08-13 18:06:07.058276 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.058286 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.058296 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.058307 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,50,289 2014-08-13 18:06:07.058316 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.058317 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.058326 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.525182 2 0.000067 2014-08-13 18:06:07.058337 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.058345 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.058354 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.058364 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.058374 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.058383 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.058391 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.058404 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.058414 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,50,289 down blocked_by {} 2014-08-13 18:06:07.058424 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.058435 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2706 2014-08-13 18:06:07.058445 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.50 2014-08-13 18:06:07.058455 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 18:06:07.058464 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.058474 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.058485 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.058516 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194482 268851 bytes 2014-08-13 18:06:07.058516 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.058532 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.058597 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.058616 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.058689 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.058709 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.058780 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.058801 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.059019 7f5c7018a700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.059039 7f5c7038c700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.059084 7f5c7028b700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.059406 7f5c70089700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.060444 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.060469 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.060460 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 18:06:07.060480 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.060473 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.060484 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.060484 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.060494 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.060495 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.060507 7f5c7c7e4700 20 osd.4 194487 get_map 194483 - loading and decoding 0x2e01800 2014-08-13 18:06:07.060505 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.060515 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.060547 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.060556 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.060564 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.603372 2 0.000061 2014-08-13 18:06:07.060577 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.060586 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.060594 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.060605 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.060615 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.060631 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.060641 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.060651 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.060770 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194483 268258 bytes 2014-08-13 18:06:07.060982 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.061042 7f5c6fb84700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.062391 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.062416 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.062420 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.062430 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.062423 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 18:06:07.062440 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.062448 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.062452 7f5c7c7e4700 20 osd.4 194487 get_map 194484 - loading and decoding 0x2e01600 2014-08-13 18:06:07.062453 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.062462 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.062472 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.062483 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.062493 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,44,362 2014-08-13 18:06:07.062505 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.062506 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.062516 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.565599 2 0.000062 2014-08-13 18:06:07.062527 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.062537 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.062545 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.062555 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.062566 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.062576 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.062584 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.062593 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.062604 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 18:06:07.062616 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 18:06:07.062625 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,61,339,362,369 down blocked_by {} 2014-08-13 18:06:07.062635 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.062647 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2683 2014-08-13 18:06:07.062657 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 18:06:07.062667 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194484 268258 bytes 2014-08-13 18:06:07.062667 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.062677 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 18:06:07.062686 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 18:06:07.062697 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.062706 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.062715 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.062727 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.063248 7f5c6f982700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.064542 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.064566 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.064554 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 18:06:07.064577 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.064571 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.064581 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.064581 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.064592 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.064592 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.064606 7f5c7c7e4700 20 osd.4 194487 get_map 194485 - loading and decoding 0x2e01400 2014-08-13 18:06:07.064602 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.064612 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.064623 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,287,309 2014-08-13 18:06:07.064634 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.064636 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.064645 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.557317 2 0.000066 2014-08-13 18:06:07.064657 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.064666 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.064674 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.064684 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.064695 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.064704 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.064713 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.064722 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.064732 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.064746 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 18:06:07.064756 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 18:06:07.064766 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.064778 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2682 2014-08-13 18:06:07.064788 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.287 2014-08-13 18:06:07.064798 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.064807 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.306 2014-08-13 18:06:07.064816 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.309 2014-08-13 18:06:07.064820 7f5c7c7e4700 10 osd.4 194487 add_map_bl 194485 268258 bytes 2014-08-13 18:06:07.064825 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.381 2014-08-13 18:06:07.064834 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.064845 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.064856 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.065208 7f5c6f780700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.066473 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.066496 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.066490 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 18:06:07.066500 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.066511 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.066510 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.066515 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.066521 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.066525 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.066535 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.066545 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.066546 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.066560 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.066555 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,289,341 2014-08-13 18:06:07.066564 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.066563 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.066567 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.066572 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.066577 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.556914 2 0.000060 2014-08-13 18:06:07.066582 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.066589 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.066598 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.066603 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.066607 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.066617 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.066618 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.066619 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.066629 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.066630 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194351 2014-08-13 18:06:07.066638 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.066640 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.066647 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.066650 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:07.066656 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.066667 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 18:06:07.066678 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) 2014-08-13 18:06:07.066688 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,289,327,341,376 down blocked_by {} 2014-08-13 18:06:07.066697 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.066706 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 18:06:07.066708 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2688 2014-08-13 18:06:07.066717 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:07.066718 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 18:06:07.066729 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.327 2014-08-13 18:06:07.066727 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.096019 18 0.000568 2014-08-13 18:06:07.066740 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.341 2014-08-13 18:06:07.066743 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started 2014-08-13 18:06:07.066750 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 18:06:07.066753 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Start 2014-08-13 18:06:07.066761 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.066762 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray 2014-08-13 18:06:07.066770 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.066774 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.066784 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.066786 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray 2014-08-13 18:06:07.066799 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 2014-08-13 18:06:07.066809 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.066820 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.066954 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 18:06:07.066973 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.066976 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.066985 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.066994 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.067004 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.066977 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067018 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067039 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.067048 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.067056 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.572432 2 0.000079 2014-08-13 18:06:07.067070 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.067080 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.067088 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.067099 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.067109 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.067126 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.067104 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:07.067140 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.067136 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067147 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067143 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.067186 7f5c6f47d700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.067177 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.067203 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.067229 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067248 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067248 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.067260 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340 2014-08-13 18:06:07.067270 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.067272 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.067284 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067299 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067384 7f5c6f57e700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.067422 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 18:06:07.067447 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.067451 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.067461 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.067472 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.067484 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.067506 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.067516 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.067527 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.213458 2 0.000066 2014-08-13 18:06:07.067538 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.067548 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.067557 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.067558 7f5c6f37c700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.067568 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.067579 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.067592 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.067601 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067612 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067281 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.656499 2 0.000178 2014-08-13 18:06:07.067655 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.067664 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.067672 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.067691 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000027 0 0.000000 2014-08-13 18:06:07.067716 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.067732 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.067741 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.067752 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.067769 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340 down blocked_by {} 2014-08-13 18:06:07.067815 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.067827 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:177957 2014-08-13 18:06:07.067837 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.067872 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.067881 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.067892 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.067855 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] flushed 2014-08-13 18:06:07.067997 7f5c6f27b700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.067995 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.068183 7f5c6f17a700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.068176 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.068188 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.068217 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.068220 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.068243 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.068264 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 18:06:07.068280 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.068275 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 18:06:07.068285 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.068293 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.068297 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.068300 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.068307 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.068312 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.068318 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.068324 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.068329 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.068336 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,286,343 2014-08-13 18:06:07.068347 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.068342 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,31,374 2014-08-13 18:06:07.068353 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.068349 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.068355 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.068359 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.567476 2 0.000067 2014-08-13 18:06:07.068365 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.621411 2 0.000087 2014-08-13 18:06:07.068372 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.068378 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.068382 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.068388 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.068390 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.068397 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.068401 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.068408 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.068412 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.068419 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.068422 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.068428 7f5c6f079700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.068433 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.068435 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.068443 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.068447 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.068453 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.068459 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,286,343 down blocked_by {} 2014-08-13 18:06:07.068466 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 18:06:07.068469 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.068498 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2652 2014-08-13 18:06:07.068495 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) 2014-08-13 18:06:07.068516 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.286 2014-08-13 18:06:07.068517 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,20,31,337,363,374 down blocked_by {} 2014-08-13 18:06:07.068535 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 18:06:07.068549 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.068559 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.068561 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2871 2014-08-13 18:06:07.068568 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.068573 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.20 2014-08-13 18:06:07.068580 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.068583 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.31 2014-08-13 18:06:07.068593 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.337 2014-08-13 18:06:07.068602 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 18:06:07.068614 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.374 2014-08-13 18:06:07.068625 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.068634 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.068645 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.068963 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 18:06:07.068988 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.068992 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.069003 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.069013 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069024 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.069016 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 18:06:07.069036 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.069043 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.069054 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.069060 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.069063 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069072 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.069075 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.069082 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.551031 2 0.000064 2014-08-13 18:06:07.069085 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,28,304 2014-08-13 18:06:07.069094 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.069094 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.069096 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.069104 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.069105 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.625843 2 0.000083 2014-08-13 18:06:07.069113 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.069117 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.069126 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.069124 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.069135 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.069138 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.069146 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.069155 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.069157 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.069165 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.069166 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.069176 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.069176 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.069188 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.069205 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,28,304 down blocked_by {} 2014-08-13 18:06:07.069214 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.069227 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2905 2014-08-13 18:06:07.069237 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.28 2014-08-13 18:06:07.069248 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.304 2014-08-13 18:06:07.069259 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.069267 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.069277 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.069275 7f5c6eb74700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.069282 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 18:06:07.069302 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.069306 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.069310 7f5c6e972700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.069309 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 18:06:07.069317 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.069325 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.069327 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069329 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.069338 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.069338 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.069349 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,340,347 2014-08-13 18:06:07.069349 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069359 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.069362 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.069361 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.069357 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.069372 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.605673 2 0.000061 2014-08-13 18:06:07.069373 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,356,357 2014-08-13 18:06:07.069383 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.069383 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.069384 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.069393 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.069394 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.555526 2 0.000065 2014-08-13 18:06:07.069401 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.069407 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.069412 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.069417 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.069423 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.069427 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.069433 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.069437 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.069443 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.069449 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.069453 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.069471 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.069480 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,340,347 down blocked_by {} 2014-08-13 18:06:07.069488 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.069498 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.069506 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.069519 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2999 2014-08-13 18:06:07.069536 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 18:06:07.069543 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.069549 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 18:06:07.069553 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.347 2014-08-13 18:06:07.069561 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,352,356,357,371,380 down blocked_by {} 2014-08-13 18:06:07.069563 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.069558 7f5c6e770700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.069571 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.069583 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2658 2014-08-13 18:06:07.069573 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.069610 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.352 2014-08-13 18:06:07.069629 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.069628 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.356 2014-08-13 18:06:07.069649 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.357 2014-08-13 18:06:07.069669 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.371 2014-08-13 18:06:07.069692 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.380 2014-08-13 18:06:07.069704 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.069714 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.069725 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.069746 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 18:06:07.069764 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.069768 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.069777 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.069786 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069800 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.069841 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.069851 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.069849 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 18:06:07.069868 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.069862 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.588209 2 0.000062 2014-08-13 18:06:07.069872 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.069874 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.069881 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.069883 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.069891 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.069891 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.069902 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.069904 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.069913 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,344,373 2014-08-13 18:06:07.069915 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.069922 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.069924 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.069932 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.069934 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.577513 2 0.000077 2014-08-13 18:06:07.069942 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.069946 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.069953 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.069956 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.069965 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.069975 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.069988 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.069997 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.070005 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.070017 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.070026 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 18:06:07.070022 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.070041 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.070038 7f5c6e66f700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070044 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 18:06:07.070058 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,320,344,363,366,373 down blocked_by {} 2014-08-13 18:06:07.070067 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.070082 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2613 2014-08-13 18:06:07.070092 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.320 2014-08-13 18:06:07.070104 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.344 2014-08-13 18:06:07.070114 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 18:06:07.070123 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.366 2014-08-13 18:06:07.070131 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 18:06:07.070141 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.070149 7f5c6e46d700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070150 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.070140 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 18:06:07.070163 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.070163 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070170 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.070182 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.070200 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070213 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.070240 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 18:06:07.070258 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070250 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.070260 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.070261 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.070272 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.070271 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.595255 2 0.000073 2014-08-13 18:06:07.070282 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070284 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.070292 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.070293 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.070302 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.070312 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.070323 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.070322 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.070330 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.070341 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.070344 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.070352 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.586392 2 0.000066 2014-08-13 18:06:07.070355 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.070366 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.070364 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.070372 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.070381 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.070392 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.070405 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.070415 7f5c6e26b700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070416 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.070426 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.070437 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.070467 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.070484 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.070495 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.070563 7f5c7c7e4700 10 osd.4 pg_epoch: 194470 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070591 7f5c6e069700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070594 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070599 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070609 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070621 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 18:06:07.070636 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070645 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070650 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.070659 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.070661 7f5c7c7e4700 10 osd.4 pg_epoch: 194471 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070676 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070672 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070679 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070683 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.070688 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070694 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,338,376 2014-08-13 18:06:07.070704 7f5c7bde3700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.070698 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070705 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.070715 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.567672 2 0.000064 2014-08-13 18:06:07.070721 7f5c7c7e4700 10 osd.4 pg_epoch: 194472 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070727 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.070735 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070739 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.070739 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070753 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.070753 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070763 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070764 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000026 0 0.000000 2014-08-13 18:06:07.070776 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.070785 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.070786 7f5c7c7e4700 10 osd.4 pg_epoch: 194473 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070800 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070794 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.070803 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070803 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.070812 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070814 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 18:06:07.070821 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070827 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) 2014-08-13 18:06:07.070837 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,338,373,376,396 down blocked_by {} 2014-08-13 18:06:07.070842 7f5c6dc65700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070844 7f5c7c7e4700 10 osd.4 pg_epoch: 194474 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070848 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.070857 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070861 7f5c6de67700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.070859 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070862 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2594 2014-08-13 18:06:07.070869 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070873 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.338 2014-08-13 18:06:07.070878 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070884 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 18:06:07.070895 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 18:06:07.070901 7f5c7c7e4700 10 osd.4 pg_epoch: 194475 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070904 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.396 2014-08-13 18:06:07.070914 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070914 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.070917 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070924 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.070926 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070938 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.070935 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.070958 7f5c7c7e4700 10 osd.4 pg_epoch: 194476 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.070971 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.070974 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.070986 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.070995 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071017 7f5c7c7e4700 10 osd.4 pg_epoch: 194477 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071031 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071019 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.071033 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071042 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.071043 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071049 7f5c6df68700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.071053 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071075 7f5c7c7e4700 10 osd.4 pg_epoch: 194478 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071091 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071094 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071102 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071112 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071131 7f5c7bde3700 10 osd.4 pg_epoch: 194486 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 18:06:07.071152 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071143 7f5c7c7e4700 10 osd.4 pg_epoch: 194479 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071159 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071156 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.071161 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071166 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.071171 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071177 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071188 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071195 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.071217 7f5c7c7e4700 10 osd.4 pg_epoch: 194480 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071230 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071232 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.071232 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071243 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.071243 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071257 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071257 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.609508 2 0.000074 2014-08-13 18:06:07.071271 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.071281 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.071281 7f5c7c7e4700 10 osd.4 pg_epoch: 194481 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071295 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071289 7f5c7bde3700 1 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.071298 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071302 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.071307 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071314 7f5c7bde3700 5 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.071316 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071330 7f5c7bde3700 20 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.071340 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.071339 7f5c7c7e4700 10 osd.4 pg_epoch: 194482 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071334 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.071352 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.071353 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071355 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071365 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071374 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071398 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.071402 7f5c7c7e4700 10 osd.4 pg_epoch: 194483 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071416 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071416 7f5c7bde3700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.071419 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071431 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071441 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071450 7f5c6d962700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.071465 7f5c7c7e4700 10 osd.4 pg_epoch: 194484 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071481 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071484 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071493 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071487 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071503 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071506 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071517 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071522 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071525 7f5c7c7e4700 10 osd.4 pg_epoch: 194485 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071538 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071534 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071545 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071540 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071550 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071559 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071592 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071605 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.071628 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.071622 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071649 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071644 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.071654 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194442 2014-08-13 18:06:07.071663 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.071672 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:07.071716 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 18:06:07.071728 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:07.071721 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071738 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.514243 18 0.000589 2014-08-13 18:06:07.071755 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071775 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071767 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 18:06:07.071784 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 18:06:07.071799 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 18:06:07.071827 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000043 0 0.000000 2014-08-13 18:06:07.071839 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 18:06:07.071834 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071851 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071850 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194487 2014-08-13 18:06:07.071862 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071862 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.071873 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.071901 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071917 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071927 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071920 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.071932 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.071943 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.071945 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.071955 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.071985 7f5c6d25b700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.071987 7f5c6d059700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.071991 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072006 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072018 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072027 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.072079 7f5c6d35c700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072106 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072127 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072138 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072143 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072153 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072165 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072171 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072175 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 18:06:07.072187 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072197 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072196 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.072200 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.072210 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.072222 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.072233 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.072235 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072252 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072262 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072245 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 4,56,353 2014-08-13 18:06:07.072264 7f5c7c7e4700 20 osd.4 194487 need_heartbeat_peer_update 2014-08-13 18:06:07.072266 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.072266 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072277 7f5c6ce57700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072277 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072276 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.535278 2 0.000063 2014-08-13 18:06:07.072287 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072289 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.072298 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.072306 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.072319 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.072330 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.072329 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072341 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.072345 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072356 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072360 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072372 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072382 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072350 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.072390 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.072397 7f5c6cd56700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072336 7f5c6d760700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072401 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 18:06:07.072438 7f5c6cf58700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072448 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072463 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072474 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072437 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 18:06:07.072516 7f5c6ca53700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.072491 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,4,56,63,353,369 down blocked_by {} 2014-08-13 18:06:07.072541 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] up_thru 194486 >= same_since 194485, all is well 2014-08-13 18:06:07.072554 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194487:2721 2014-08-13 18:06:07.072549 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072565 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-13 18:06:07.072566 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072578 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072583 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.072592 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.072600 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.072576 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.56 2014-08-13 18:06:07.072601 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.072610 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.63 2014-08-13 18:06:07.072622 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.353 2014-08-13 18:06:07.072630 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.072640 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.072649 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.072661 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.072843 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.073071 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.073090 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.073100 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.073154 7f5c6c952700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.073220 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 18:06:07.073369 7f5c6c64f700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.073552 7f5c6c750700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.073657 7f5c6c851700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.073646 7f5c7c7e4700 10 osd.4 pg_epoch: 194486 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 18:06:07.073667 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.073672 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.073681 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.073692 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.073702 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.073750 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.073762 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.073772 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.549289 2 0.000061 2014-08-13 18:06:07.073798 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.073820 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.073840 7f5c7c7e4700 1 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.073854 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000033 0 0.000000 2014-08-13 18:06:07.073865 7f5c7c7e4700 5 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.073879 7f5c7c7e4700 20 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194487 2014-08-13 18:06:07.073888 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 NullEvt 2014-08-13 18:06:07.073898 7f5c7c7e4700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-13 18:06:07.074079 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.074354 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.074366 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.074373 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.074382 7f5c6cb54700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.074385 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.074395 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.074806 7f5c88628700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.074842 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.074863 7f5c7bde3700 15 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.074877 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.074944 7f5c6be47700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.074945 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.074965 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.074978 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.075015 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.075033 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.075047 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.075052 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194487 epoch_requested: 194487 FlushedEvt 2014-08-13 18:06:07.075065 7f5c7c7e4700 15 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.075079 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.075098 7f5c973a57a0 0 osd.4 194487 done with init, starting boot process 2014-08-13 18:06:07.075105 7f5c973a57a0 10 osd.4 194487 start_boot - have maps 185139..194487 2014-08-13 18:06:07.075272 7f5c6c54e700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.075287 7f5c6b740700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.075353 7f5c6b841700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.075547 7f5c7d1e5700 10 osd.4 194487 _maybe_boot mon has osdmaps 185139..194494 2014-08-13 18:06:07.076052 7f5c6c44d700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.076098 7f5c7d1e5700 10 osd.4 194487 _send_boot 2014-08-13 18:06:07.076106 7f5c7d1e5700 10 osd.4 194487 client_addr 192.168.100.115:6805/37708, cluster_addr 192.168.1.115:6813/37708, hb_back_addr 192.168.1.115:6817/37708, hb_front_addr 192.168.100.115:6806/37708 2014-08-13 18:06:07.076237 7f5c6db64700 10 osd.4 194487 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.076989 7f5c7d1e5700 10 osd.4 194487 _collect_metadata {arch=x86_64,back_addr=192.168.1.115:6813/37708,ceph_version=ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6),cpu=Intel(R) Xeon(R) CPU E5-2450 0 @ 2.10GHz,front_addr=192.168.100.115:6805/37708,hb_back_addr=192.168.1.115:6817/37708,hb_front_addr=192.168.100.115:6806/37708,hostname=storage0115-ib.pocstor,kernel_description=#1 SMP Wed Mar 13 00:26:49 UTC 2013,kernel_version=2.6.32-358.2.1.el6.x86_64,mem_swap_kb=23446856,mem_total_kb=24568044,os=Linux,osd_data=/var/lib/ceph/osd/ceph-4,osd_journal=/var/lib/ceph/osd/ceph-4/journal} 2014-08-13 18:06:07.146113 7f5c835ef700 10 osd.4 194487 do_waiters -- start 2014-08-13 18:06:07.146117 7f5c835ef700 10 osd.4 194487 do_waiters -- finish 2014-08-13 18:06:07.146119 7f5c835ef700 20 osd.4 194487 _dispatch 0x47ff980 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.146189 7f5c835ef700 3 osd.4 194487 handle_osd_map epochs [194488,194495], i have 194487, src has [185139,194495] 2014-08-13 18:06:07.146198 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194488 2014-08-13 18:06:07.147575 7f5c6a335700 10 osd.4 194487 new session 0x20c8e00 con=0x4b2f4e0 addr=192.168.1.115:6810/59615 2014-08-13 18:06:07.147664 7f5c6a335700 10 osd.4 194487 session 0x20c8e00 osd.0 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.150011 7f5c6a133700 10 osd.4 194487 new session 0x20c8ee0 con=0x4b2f640 addr=192.168.1.103:6813/37929 2014-08-13 18:06:07.150105 7f5c6a133700 10 osd.4 194487 session 0x20c8ee0 osd.11 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.150563 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194489 2014-08-13 18:06:07.151032 7f5c6a032700 10 osd.4 194487 new session 0x20c8fc0 con=0x4b2fbc0 addr=192.168.1.108:6826/4273 2014-08-13 18:06:07.151141 7f5c6a032700 10 osd.4 194487 session 0x20c8fc0 osd.352 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.152740 7f5c69c2e700 10 osd.4 194487 new session 0x20c90a0 con=0x4c58160 addr=192.168.1.103:6800/36920 2014-08-13 18:06:07.152819 7f5c69c2e700 10 osd.4 194487 session 0x20c90a0 osd.25 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.153218 7f5c69628700 10 osd.4 194487 new session 0x20c9180 con=0x4c58f20 addr=192.168.1.115:6838/62713 2014-08-13 18:06:07.153305 7f5c69628700 10 osd.4 194487 session 0x20c9180 osd.7 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.153448 7f5c6992b700 10 osd.4 194487 new session 0x20c9260 con=0x4c594a0 addr=192.168.1.110:6818/45268 2014-08-13 18:06:07.153533 7f5c6992b700 10 osd.4 194487 session 0x20c9260 osd.339 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.154694 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194490 2014-08-13 18:06:07.155674 7f5c69123700 10 osd.4 194487 new session 0x20c9340 con=0x4c59ce0 addr=192.168.1.101:6814/23703 2014-08-13 18:06:07.155711 7f5c69123700 10 osd.4 194487 session 0x20c9340 osd.309 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.155710 7f5c68c1e700 10 osd.4 194487 new session 0x20c9420 con=0x4c59760 addr=192.168.1.109:6816/33855 2014-08-13 18:06:07.155741 7f5c68c1e700 10 osd.4 194487 session 0x20c9420 osd.293 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.157030 7f5c6881a700 10 osd.4 194487 new session 0x20c9500 con=0x4c5b440 addr=192.168.1.110:6802/41438 2014-08-13 18:06:07.157110 7f5c6881a700 10 osd.4 194487 session 0x20c9500 osd.342 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.157659 7f5c69729700 10 osd.4 194487 new session 0x20c95e0 con=0x4c591e0 addr=192.168.1.114:6842/56512 2014-08-13 18:06:07.157696 7f5c69729700 10 osd.4 194487 session 0x20c95e0 osd.370 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.161578 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194491 2014-08-13 18:06:07.164509 7f5c67b0d700 10 osd.4 194487 new session 0x20c96c0 con=0x4c5c0a0 addr=192.168.1.114:6822/51844 2014-08-13 18:06:07.164542 7f5c67b0d700 10 osd.4 194487 session 0x20c96c0 osd.380 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.164825 7f5c66f01700 10 osd.4 194487 new session 0x20c97a0 con=0x4c5eb40 addr=192.168.1.102:6822/15390 2014-08-13 18:06:07.164855 7f5c66f01700 10 osd.4 194487 session 0x20c97a0 osd.381 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.165038 7f5c68719700 10 osd.4 194487 new session 0x20c9880 con=0x4c5b180 addr=192.168.1.101:6802/22303 2014-08-13 18:06:07.165075 7f5c68719700 10 osd.4 194487 session 0x20c9880 osd.314 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.165193 7f5c6780a700 10 osd.4 194487 new session 0x20c9960 con=0x4c5bde0 addr=192.168.1.102:6826/15861 2014-08-13 18:06:07.165227 7f5c6780a700 10 osd.4 194487 session 0x20c9960 osd.368 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.165328 7f5c666f9700 10 osd.4 194487 new session 0x20c9a40 con=0x4c5e880 addr=192.168.1.108:6824/4037 2014-08-13 18:06:07.165361 7f5c666f9700 10 osd.4 194487 session 0x20c9a40 osd.369 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.165369 7f5c663f6700 10 osd.4 194487 new session 0x20c9b20 con=0x4c5e5c0 addr=192.168.1.114:6830/52879 2014-08-13 18:06:07.165399 7f5c663f6700 10 osd.4 194487 session 0x20c9b20 osd.375 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.165491 7f5c68416700 10 osd.4 194487 new session 0x20c9c00 con=0x4c5c360 addr=192.168.1.110:6810/42728 2014-08-13 18:06:07.165526 7f5c68416700 10 osd.4 194487 session 0x20c9c00 osd.329 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.166238 7f5c68517700 10 osd.4 194487 new session 0x20c9ce0 con=0x4c5c620 addr=192.168.1.112:6838/8300 2014-08-13 18:06:07.166265 7f5c68618700 10 osd.4 194487 new session 0x20c9dc0 con=0x4c5aec0 addr=192.168.1.113:6816/17521 2014-08-13 18:06:07.166271 7f5c68517700 10 osd.4 194487 session 0x20c9ce0 osd.48 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.166315 7f5c68618700 10 osd.4 194487 session 0x20c9dc0 osd.312 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.166652 7f5c67507700 10 osd.4 194487 new session 0x20c9ea0 con=0x4c5ee00 addr=192.168.1.114:6832/53185 2014-08-13 18:06:07.166685 7f5c67507700 10 osd.4 194487 session 0x20c9ea0 osd.350 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.167137 7f5c661f4700 10 osd.4 194487 new session 0x20c9f80 con=0x4c5e300 addr=192.168.1.109:6818/34549 2014-08-13 18:06:07.167538 7f5c65cef700 10 osd.4 194487 new session 0x20ca060 con=0x4f218c0 addr=192.168.1.113:6810/15731 2014-08-13 18:06:07.167569 7f5c65cef700 10 osd.4 194487 session 0x20ca060 osd.302 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.168849 7f5c661f4700 10 osd.4 194487 session 0x20c9f80 osd.284 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.168897 7f5c69e30700 10 osd.4 194487 new session 0x20ca140 con=0x4c58420 addr=192.168.1.106:6804/5112 2014-08-13 18:06:07.168934 7f5c69e30700 10 osd.4 194487 session 0x20ca140 osd.327 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.169181 7f5c68d1f700 10 osd.4 194487 new session 0x20ca220 con=0x4c59a20 addr=192.168.1.101:6816/24270 2014-08-13 18:06:07.169218 7f5c68d1f700 10 osd.4 194487 session 0x20ca220 osd.311 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170159 7f5c641d4700 10 osd.4 194487 new session 0x20ca300 con=0x4f20dc0 addr=192.168.1.106:6802/4876 2014-08-13 18:06:07.170201 7f5c641d4700 10 osd.4 194487 session 0x20ca300 osd.330 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170852 7f5c634c7700 10 osd.4 194487 new session 0x20ca4c0 con=0x4fa1fa0 addr=192.168.1.109:6812/32413 2014-08-13 18:06:07.170851 7f5c649dc700 10 osd.4 194487 new session 0x20ca3e0 con=0x4f21080 addr=192.168.1.107:6815/53679 2014-08-13 18:06:07.170856 7f5c655e8700 10 osd.4 194487 new session 0x20ca5a0 con=0x4f21340 addr=192.168.1.114:6836/54427 2014-08-13 18:06:07.170883 7f5c634c7700 10 osd.4 194487 session 0x20ca4c0 osd.310 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170888 7f5c649dc700 10 osd.4 194487 session 0x20ca3e0 osd.2 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170890 7f5c655e8700 10 osd.4 194487 session 0x20ca5a0 osd.355 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170932 7f5c65aed700 10 osd.4 194487 new session 0x20ca680 con=0x4f21600 addr=192.168.1.104:6820/2308 2014-08-13 18:06:07.170934 7f5c635c8700 10 osd.4 194487 new session 0x20ca760 con=0x4fa2260 addr=192.168.1.102:6824/15623 2014-08-13 18:06:07.170938 7f5c632c5700 10 osd.4 194487 new session 0x20ca840 con=0x4fa1ce0 addr=192.168.1.102:6840/19195 2014-08-13 18:06:07.170965 7f5c65aed700 10 osd.4 194487 session 0x20ca680 osd.346 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170967 7f5c635c8700 10 osd.4 194487 session 0x20ca760 osd.365 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.170971 7f5c632c5700 10 osd.4 194487 session 0x20ca840 osd.378 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.171094 7f5c63bce700 10 osd.4 194487 new session 0x20ca920 con=0x4fa2520 addr=192.168.1.112:6830/6302 2014-08-13 18:06:07.171125 7f5c63bce700 10 osd.4 194487 session 0x20ca920 osd.60 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.171457 7f5c630c3700 10 osd.4 194487 new session 0x20caa00 con=0x4fa1a20 addr=192.168.1.108:6828/4534 2014-08-13 18:06:07.171516 7f5c630c3700 10 osd.4 194487 session 0x20caa00 osd.366 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.173776 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194492 2014-08-13 18:06:07.173995 7f5c629bc700 10 osd.4 194487 new session 0x20caae0 con=0x4fa5ac0 addr=192.168.1.111:6832/13543 2014-08-13 18:06:07.174243 7f5c629bc700 10 osd.4 194487 session 0x20caae0 osd.52 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.174284 7f5c623b6700 10 osd.4 194487 new session 0x20cabc0 con=0x4fa5800 addr=192.168.1.101:6806/6812 2014-08-13 18:06:07.174323 7f5c623b6700 10 osd.4 194487 session 0x20cabc0 osd.306 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.176958 7f5c618ab700 10 osd.4 194487 new session 0x20caca0 con=0x4fa5540 addr=192.168.1.104:6810/63264 2014-08-13 18:06:07.176966 7f5c617aa700 10 osd.4 194487 new session 0x20cad80 con=0x4fa5280 addr=192.168.1.111:6830/12680 2014-08-13 18:06:07.176989 7f5c618ab700 10 osd.4 194487 session 0x20caca0 osd.322 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.176997 7f5c617aa700 10 osd.4 194487 session 0x20cad80 osd.63 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.178116 7f5c614a7700 10 osd.4 194487 new session 0x20cae60 con=0x4fa4fc0 addr=192.168.1.104:6800/61726 2014-08-13 18:06:07.178139 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194493 2014-08-13 18:06:07.178147 7f5c614a7700 10 osd.4 194487 session 0x20cae60 osd.349 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.181607 7f5c612a5700 10 osd.4 194487 new session 0x20caf40 con=0x4fa4d00 addr=192.168.1.108:6800/30527 2014-08-13 18:06:07.181644 7f5c612a5700 10 osd.4 194487 session 0x20caf40 osd.382 has caps osdcap[grant(*)] 'allow *' 2014-08-13 18:06:07.182756 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194494 2014-08-13 18:06:07.187517 7f5c835ef700 10 osd.4 194487 handle_osd_map got inc map for epoch 194495 2014-08-13 18:06:07.192409 7f5c835ef700 10 osd.4 194487 advance to epoch 194488 (<= newest 194495) 2014-08-13 18:06:07.192510 7f5c835ef700 7 osd.4 194488 advance_map epoch 194488 74 pgs 2014-08-13 18:06:07.192538 7f5c835ef700 10 osd.4 194488 advance to epoch 194489 (<= newest 194495) 2014-08-13 18:06:07.192618 7f5c835ef700 7 osd.4 194489 advance_map epoch 194489 74 pgs 2014-08-13 18:06:07.192646 7f5c835ef700 10 osd.4 194489 advance to epoch 194490 (<= newest 194495) 2014-08-13 18:06:07.192721 7f5c835ef700 7 osd.4 194490 advance_map epoch 194490 74 pgs 2014-08-13 18:06:07.192746 7f5c835ef700 10 osd.4 194490 advance to epoch 194491 (<= newest 194495) 2014-08-13 18:06:07.192821 7f5c835ef700 7 osd.4 194491 advance_map epoch 194491 74 pgs 2014-08-13 18:06:07.192846 7f5c835ef700 10 osd.4 194491 advance to epoch 194492 (<= newest 194495) 2014-08-13 18:06:07.192918 7f5c835ef700 7 osd.4 194492 advance_map epoch 194492 74 pgs 2014-08-13 18:06:07.192943 7f5c835ef700 10 osd.4 194492 advance to epoch 194493 (<= newest 194495) 2014-08-13 18:06:07.193013 7f5c835ef700 7 osd.4 194493 advance_map epoch 194493 74 pgs 2014-08-13 18:06:07.193039 7f5c835ef700 10 osd.4 194493 advance to epoch 194494 (<= newest 194495) 2014-08-13 18:06:07.193109 7f5c835ef700 7 osd.4 194494 advance_map epoch 194494 74 pgs 2014-08-13 18:06:07.193135 7f5c835ef700 10 osd.4 194494 advance to epoch 194495 (<= newest 194495) 2014-08-13 18:06:07.193212 7f5c835ef700 7 osd.4 194495 advance_map epoch 194495 74 pgs 2014-08-13 18:06:07.193220 7f5c835ef700 10 osd.4 194495 up_epoch is 194495 2014-08-13 18:06:07.193222 7f5c835ef700 10 osd.4 194495 boot_epoch is 194495 2014-08-13 18:06:07.193246 7f5c835ef700 1 osd.4 194495 state: booting -> active 2014-08-13 18:06:07.193255 7f5c835ef700 10 osd.4 194495 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194495 [185139,194495] lci=[194495,194495]) 2014-08-13 18:06:07.193453 7f5c835ef700 0 osd.4 194495 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 18:06:07.193468 7f5c835ef700 7 osd.4 194495 consume_map version 194495 2014-08-13 18:06:07.193537 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 18:06:07.193567 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193606 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193627 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193654 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193645 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_advance_map [286,375]/[375,10] -- 286/375 2014-08-13 18:06:07.193684 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.193687 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193691 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Reset advmap 2014-08-13 18:06:07.193705 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193706 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.193700 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,18]/[351,18] -- 351/351 2014-08-13 18:06:07.193720 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [286,375] newacting [375,10] 2014-08-13 18:06:07.193722 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193732 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.193733 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.193737 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:07.193743 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.193749 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [4,286,375]/[4,375,10] r=0 lpr=194487 pi=193640-194485/36 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194488 2014-08-13 18:06:07.193754 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193757 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,18] newacting [351,18] 2014-08-13 18:06:07.193769 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193770 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.193783 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193785 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.168432 3 0.000243 2014-08-13 18:06:07.193785 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,286,375](4) acting [4,375,10](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.193798 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] null 2014-08-13 18:06:07.193803 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.168499 0 0.000000 2014-08-13 18:06:07.193805 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194485/194486/194485) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] noting past interval(194486-194487 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) 2014-08-13 18:06:07.193814 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.193818 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.193821 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] up [4,286,375] -> [286,375], acting [4,375,10] -> [375,10], acting_primary 4 -> 375, up_primary 4 -> 286, role 0 -> -1 2014-08-13 18:06:07.193828 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193832 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194486 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.193842 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] clear_primary_state 2014-08-13 18:06:07.193846 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194488 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.193844 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 18:06:07.193860 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194488 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.193859 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] agent_stop 2014-08-13 18:06:07.193865 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193873 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194488 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,18] newacting [351,18] 2014-08-13 18:06:07.193879 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] on_change 2014-08-13 18:06:07.193881 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193885 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194488 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.193894 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.193897 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,4,18] r=1 lpr=194488 pi=192970-194484/31 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.193897 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:07.193907 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:07.193914 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 18:06:07.193921 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [351,4,18](351) acting [351,4,18](351)): not rw, up_thru 194485 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [351,4,18](351) acting [351,4,18](351) maybe_went_rw) : primary up 192967-194485 includes interval 2014-08-13 18:06:07.193925 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] cancel_copy_ops 2014-08-13 18:06:07.193930 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193940 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194485/194485/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [351,4,18](351) acting [351,4,18](351) maybe_went_rw) 2014-08-13 18:06:07.193943 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] cancel_flush_ops 2014-08-13 18:06:07.193945 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] null 2014-08-13 18:06:07.193954 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,4,18] -> [351,18], acting [351,4,18] -> [351,18], acting_primary 351 -> 351, up_primary 351 -> 351, role 1 -> -1 2014-08-13 18:06:07.193960 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193959 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] on_change_cleanup 2014-08-13 18:06:07.193969 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.193976 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193977 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] on_change 2014-08-13 18:06:07.193982 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.193991 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.193995 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.194001 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] exit NotTrimming 2014-08-13 18:06:07.194008 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.194006 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194018 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped m=56] enter NotTrimming 2014-08-13 18:06:07.194020 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.194025 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194033 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.194033 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_stats 2014-08-13 18:06:07.194041 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.194046 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.194048 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.194059 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 18:06:07.194061 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.194062 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 18:06:07.194074 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.194074 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194074 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.194087 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.194092 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194091 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.194098 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.194104 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.194109 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.194107 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 18:06:07.194119 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.194124 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194123 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194132 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194139 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194153 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:07.194166 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194176 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [286,375]/[375,10,301] -- 286/375 2014-08-13 18:06:07.194186 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194199 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194201 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 18:06:07.194203 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.194201 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,18]/[351,18,21] -- 351/351 2014-08-13 18:06:07.194223 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194216 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194219 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194228 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.194236 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [286,375] newacting [375,10,301] 2014-08-13 18:06:07.194238 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194241 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194249 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.194251 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,18] newacting [351,18,21] 2014-08-13 18:06:07.194255 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:07.194263 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.194263 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10] r=-1 lpr=194488 pi=193640-194487/37 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194489 2014-08-13 18:06:07.194276 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18] r=-1 lpr=194488 pi=192970-194487/32 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.194274 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194290 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194291 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [286,375](286) acting [375,10](375)): not rw, up_thru 194482 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [286,375](286) acting [375,10](375)) : primary up 192969-194482 does not include interval 2014-08-13 18:06:07.194297 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [351,18](351) acting [351,18](351)): not rw, up_thru 194485 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [351,18](351) acting [351,18](351)) : primary up 192967-194485 does not include interval 2014-08-13 18:06:07.194303 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 18:06:07.194305 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194488/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] noting past interval(194488-194488 up [286,375](286) acting [375,10](375)) 2014-08-13 18:06:07.194312 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194488/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [351,18](351) acting [351,18](351)) 2014-08-13 18:06:07.194319 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 18:06:07.194320 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] up [286,375] -> [286,375], acting [375,10] -> [375,10,301], acting_primary 375 -> 375, up_primary 286 -> 286, role -1 -> -1 2014-08-13 18:06:07.194327 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] up [351,18] -> [351,18], acting [351,18] -> [351,18,21], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 18:06:07.194337 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.194338 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 18:06:07.194342 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.194349 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.194354 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.194355 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194363 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.194367 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.194371 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194377 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.194380 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.194385 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.194390 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.194395 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.194402 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194403 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.194408 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.194417 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.194418 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194420 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.194430 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.194433 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.194433 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194443 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.194446 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.194450 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] null 2014-08-13 18:06:07.194456 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.194458 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.194468 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.194466 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:07.194470 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.194482 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194483 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194484 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 18:06:07.194500 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:07.194513 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194528 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194532 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [286,375]/[375,10,301] -- 286/375 2014-08-13 18:06:07.194542 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194560 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194555 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194564 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.194568 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194576 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194582 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 18:06:07.194589 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194581 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,18]/[351,18,21] -- 351/351 2014-08-13 18:06:07.194597 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 18:06:07.194612 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194613 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194617 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.194619 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [301,375,10]/[375,10,301] -- 301/375 2014-08-13 18:06:07.194627 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194637 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194630 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194640 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.194643 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194643 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194653 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194658 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194671 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194674 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,21,52]/[351,18,21] -- 351/351 2014-08-13 18:06:07.194689 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194665 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [301,375,10] newacting [375,10,301] 2014-08-13 18:06:07.194687 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 18:06:07.194692 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.194703 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194704 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.194695 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.194715 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] acting_up_affected newup [351,21,52] newacting [351,18,21] 2014-08-13 18:06:07.194716 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:07.194721 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [286,375]/[375,10,301] r=-1 lpr=194489 pi=193640-194488/38 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194491 2014-08-13 18:06:07.194728 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.194732 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 18:06:07.194744 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,18]/[351,18,21] r=-1 lpr=194489 pi=192970-194488/33 crt=192888'481 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.194748 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:07.194746 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [286,375](286) acting [375,10,301](375)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [286,375](286) acting [375,10,301](375) maybe_went_rw) : primary up 192969-194489 includes interval 2014-08-13 18:06:07.194764 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:07.194763 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194488/194489/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] noting past interval(194489-194490 up [286,375](286) acting [375,10,301](375) maybe_went_rw) 2014-08-13 18:06:07.194769 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [351,18](351) acting [351,18,21](351)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [351,18](351) acting [351,18,21](351) maybe_went_rw) : primary up 192967-194489 includes interval 2014-08-13 18:06:07.194783 7f5c835ef700 10 osd.4 194495 maybe_update_heartbeat_peers updating 2014-08-13 18:06:07.194788 7f5c835ef700 20 osd.4 194495 24.30b heartbeat_peers 2014-08-13 18:06:07.194790 7f5c835ef700 20 osd.4 194495 22.220 heartbeat_peers 2014-08-13 18:06:07.194792 7f5c835ef700 20 osd.4 194495 22.362 heartbeat_peers 4,58,343 2014-08-13 18:06:07.194783 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194488/194489/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [351,18](351) acting [351,18,21](351) maybe_went_rw) 2014-08-13 18:06:07.194777 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] up [286,375] -> [301,375,10], acting [375,10,301] -> [375,10,301], acting_primary 375 -> 375, up_primary 286 -> 301, role -1 -> -1 2014-08-13 18:06:07.194797 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] up [351,18] -> [351,21,52], acting [351,18,21] -> [351,18,21], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 18:06:07.194800 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.194812 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.194812 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.194827 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.194828 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.194837 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.194841 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.194850 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.194863 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.194874 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.194853 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.194872 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.58 192.168.1.111:6835/14316 192.168.100.111:6829/14316 2014-08-13 18:06:07.194884 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.194887 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.194896 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.194899 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.194908 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.194912 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.194923 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.194927 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.194933 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.194940 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.194946 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.194970 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.343 192.168.1.104:6803/61960 192.168.100.104:6810/61960 2014-08-13 18:06:07.194977 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] handle_advance_map [351,21,52]/[351,21,52] -- 351/351 2014-08-13 18:06:07.194991 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.194994 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.194951 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.195010 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195014 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195021 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] acting_up_affected newup [351,21,52] newacting [351,21,52] 2014-08-13 18:06:07.195032 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.195042 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52]/[351,18,21] r=-1 lpr=194491 pi=192970-194490/34 crt=192888'481 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.195062 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [351,21,52](351) acting [351,18,21](351)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [351,21,52](351) acting [351,18,21](351)) : primary up 192967-194489 does not include interval 2014-08-13 18:06:07.195073 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194491/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [351,21,52](351) acting [351,18,21](351)) 2014-08-13 18:06:07.195050 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [301,375,10]/[301,375,10] -- 301/301 2014-08-13 18:06:07.195085 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,21,52] -> [351,21,52], acting [351,18,21] -> [351,21,52], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> -1 2014-08-13 18:06:07.195101 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195098 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.195107 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.195105 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.195117 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.195118 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195128 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.195138 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.195133 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [301,375,10] newacting [301,375,10] 2014-08-13 18:06:07.195147 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.195157 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.195156 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.195168 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.195188 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.195170 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10]/[375,10,301] r=-1 lpr=194491 pi=193640-194490/39 crt=194422'18660 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194492 2014-08-13 18:06:07.195198 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.195208 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.195219 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195218 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)) : primary up 192969-194489 does not include interval 2014-08-13 18:06:07.195246 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,21,52]/[351,21,52] -- 351/351 2014-08-13 18:06:07.195259 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195262 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.195275 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195250 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194491/194488) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] noting past interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 18:06:07.195285 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195290 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] up [301,375,10] -> [301,375,10], acting [375,10,301] -> [301,375,10], acting_primary 375 -> 301, up_primary 301 -> 301, role -1 -> -1 2014-08-13 18:06:07.195308 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,21,52]/[351,21,52] -- 351/351 2014-08-13 18:06:07.195321 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195306 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.195323 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.195331 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.195335 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195349 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195358 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.195370 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 18:06:07.195372 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.195383 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195383 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.195386 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.195396 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195405 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [351,4,18] newacting [351,4,18] 2014-08-13 18:06:07.195394 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.195416 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.195425 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,21,52] r=-1 lpr=194492 pi=192970-194491/35 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.195444 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [351,21,52](351) acting [351,21,52](351)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [351,21,52](351) acting [351,21,52](351) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.195423 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.195454 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194491/194492/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [351,21,52](351) acting [351,21,52](351) maybe_went_rw) 2014-08-13 18:06:07.195456 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.195466 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] up [351,21,52] -> [351,4,18], acting [351,21,52] -> [351,4,18], acting_primary 351 -> 351, up_primary 351 -> 351, role -1 -> 1 2014-08-13 18:06:07.195470 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.195481 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.195491 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.195500 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.195510 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.195487 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] [375,10,301] -> [301,375,10], acting primary 375 -> 301 2014-08-13 18:06:07.195519 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.195524 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.195531 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.195545 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.195536 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.195556 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.195570 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.195566 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.61 192.168.1.111:6841/16566 192.168.100.111:6839/16566 2014-08-13 18:06:07.195581 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.195568 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195591 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.195599 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.195608 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.195618 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195629 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.195612 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [301,375,10]/[301,375,10] -- 301/301 2014-08-13 18:06:07.195655 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195661 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.195674 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195680 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.195692 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195694 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.195705 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.001888 8 0.001538 2014-08-13 18:06:07.195721 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.195733 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.195727 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.349 192.168.1.104:6801/61726 192.168.100.104:6803/61726 2014-08-13 18:06:07.195742 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.195721 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [301,375,10]/[301,375,10] -- 301/301 2014-08-13 18:06:07.195756 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000022 0 0.000000 2014-08-13 18:06:07.195768 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.195776 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.195783 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.195835 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.195832 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.195875 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.195949 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.195951 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.362 192.168.1.108:6843/8170 192.168.100.108:6853/8170 2014-08-13 18:06:07.195968 7f5c835ef700 20 osd.4 194495 22.28f heartbeat_peers 2014-08-13 18:06:07.195971 7f5c835ef700 20 osd.4 194495 22.14a heartbeat_peers 2014-08-13 18:06:07.195929 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.195973 7f5c835ef700 20 osd.4 194495 22.22c heartbeat_peers 4,301,324 2014-08-13 18:06:07.195990 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_advance_map [58,343]/[58,343] -- 58/58 2014-08-13 18:06:07.196013 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.196019 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.196031 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.196041 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.196054 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.170331 2 0.000260 2014-08-13 18:06:07.196084 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [4,286,375]/[4,286,375] -- 4/4 2014-08-13 18:06:07.196103 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.196107 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.196117 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.196128 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [4,286,375] newacting [4,286,375] 2014-08-13 18:06:07.196139 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.196149 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [301,375,10] r=-1 lpr=194492 pi=193640-194491/40 crt=194422'18660 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194495 2014-08-13 18:06:07.196171 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [301,375,10](301) acting [301,375,10](301)): not rw, up_thru 194492 up_from 192972 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) : primary up 192972-194492 includes interval 2014-08-13 18:06:07.196259 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.196292 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194491/194492/194492) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] noting past interval(194492-194494 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 18:06:07.196308 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] up [301,375,10] -> [4,286,375], acting [301,375,10] -> [4,286,375], acting_primary 301 -> 4, up_primary 301 -> 4, role -1 -> 0 2014-08-13 18:06:07.196321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] clear_primary_state 2014-08-13 18:06:07.196336 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] agent_stop 2014-08-13 18:06:07.196295 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.170583 0 0.000000 2014-08-13 18:06:07.196347 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.196360 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd 194495:30297 2014-08-13 18:06:07.196372 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.196360 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.170658 0 0.000000 2014-08-13 18:06:07.196384 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.196400 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.196411 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.196397 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Started 0.170739 0 0.000000 2014-08-13 18:06:07.196421 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.196426 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.196432 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.196437 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194486 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.196443 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.196454 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.196450 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194488 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.196466 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.196476 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194488 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.196478 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.196486 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194488 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [58,343] newacting [58,343] 2014-08-13 18:06:07.196489 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] clear_stats 2014-08-13 18:06:07.196497 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194488 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.196500 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] requeue_ops 2014-08-13 18:06:07.196508 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,58,343] r=0 lpr=194488 pi=193306-194484/32 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.196513 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] on_role_change 2014-08-13 18:06:07.196523 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] requeue_ops 2014-08-13 18:06:07.196535 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] cancel_recovery 2014-08-13 18:06:07.196533 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,58,343](4) acting [4,58,343](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.196549 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] clear_recovery_state 2014-08-13 18:06:07.196550 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] noting past interval(194485-194487 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) 2014-08-13 18:06:07.196562 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.196563 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] up [4,58,343] -> [58,343], acting [4,58,343] -> [58,343], acting_primary 4 -> 58, up_primary 4 -> 58, role 0 -> -1 2014-08-13 18:06:07.196577 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_activate_map 2014-08-13 18:06:07.196577 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.196591 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] update_heartbeat_peers -> 4,286,375 2014-08-13 18:06:07.196597 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.196630 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.196654 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.196660 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.301 192.168.1.113:6815/16727 192.168.100.113:6840/16727 2014-08-13 18:06:07.196664 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.196687 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.196728 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.196742 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.196751 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.196765 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.196791 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.196848 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.324 192.168.1.104:6815/65209 192.168.100.104:6840/65209 2014-08-13 18:06:07.196827 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.196930 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.302 192.168.1.113:6811/15731 192.168.100.113:6834/15731 2014-08-13 18:06:07.196896 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.196965 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.196976 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.197047 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.197116 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.368 192.168.1.102:6827/15861 192.168.100.102:6811/15861 2014-08-13 18:06:07.197078 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.197133 7f5c835ef700 20 osd.4 194495 22.619 heartbeat_peers 4,62,65 2014-08-13 18:06:07.197258 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.197294 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [58,343]/[58,343] -- 58/58 2014-08-13 18:06:07.197344 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.197359 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.197376 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.197398 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.197425 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [58,343]/[58,343] -- 58/58 2014-08-13 18:06:07.197458 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.197463 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.197474 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.197485 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.197555 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 18:06:07.197585 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.197606 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.197618 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.197627 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] acting_up_affected newup [61,349,362] newacting [61,349,362] 2014-08-13 18:06:07.197638 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.197650 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [58,343] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'415 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.197676 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [58,343](58) acting [58,343](58)): not rw, up_thru 194488 up_from 192935 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [58,343](58) acting [58,343](58) maybe_went_rw) : primary up 192935-194488 includes interval 2014-08-13 18:06:07.197771 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [58,343](58) acting [58,343](58) maybe_went_rw) 2014-08-13 18:06:07.197792 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] up [58,343] -> [61,349,362], acting [58,343] -> [61,349,362], acting_primary 58 -> 61, up_primary 58 -> 61, role -1 -> -1 2014-08-13 18:06:07.197806 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.197816 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.197826 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.197837 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.197846 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.197856 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.197866 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.197877 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.197890 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.197899 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] [58,343] -> [61,349,362], acting primary 58 -> 61 2014-08-13 18:06:07.197916 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.197925 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.197936 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.197970 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 18:06:07.197989 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.197993 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.198005 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.198016 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.198022 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.62 192.168.1.112:6825/5505 192.168.100.112:6810/5505 2014-08-13 18:06:07.198044 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 18:06:07.198059 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.198064 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.198073 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.198083 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.198101 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.65 192.168.1.111:6843/17345 192.168.100.111:6842/17345 2014-08-13 18:06:07.198115 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [61,349,362]/[61,349,362] -- 61/61 2014-08-13 18:06:07.198129 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.198132 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.198142 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.198154 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.198182 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 18:06:07.198197 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.198198 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.44 192.168.1.111:6823/10782 192.168.100.111:6802/10782 2014-08-13 18:06:07.198200 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.198210 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.198219 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,58,343] newacting [4,58,343] 2014-08-13 18:06:07.198232 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.198242 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [61,349,362] r=-1 lpr=194491 pi=193306-194490/34 crt=191727'415 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.198261 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.48 192.168.1.112:6839/8300 192.168.100.112:6842/8300 2014-08-13 18:06:07.198266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [61,349,362](61) acting [61,349,362](61)): not rw, up_thru 194492 up_from 192938 last_epoch_clean 194484 generate_past_intervals interval(194491-194494 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) : primary up 192938-194492 includes interval 2014-08-13 18:06:07.198278 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194491) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 18:06:07.198292 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [61,349,362] -> [4,58,343], acting [61,349,362] -> [4,58,343], acting_primary 61 -> 4, up_primary 61 -> 4, role -1 -> 0 2014-08-13 18:06:07.198305 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.198318 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.314 192.168.1.101:6803/22303 192.168.100.101:6803/22303 2014-08-13 18:06:07.198319 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.198328 7f5c835ef700 20 osd.4 194495 22.30f heartbeat_peers 2014-08-13 18:06:07.198332 7f5c835ef700 20 osd.4 194495 24.9b heartbeat_peers 2014-08-13 18:06:07.198334 7f5c835ef700 20 osd.4 194495 22.581 heartbeat_peers 2014-08-13 18:06:07.198329 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.198337 7f5c835ef700 20 osd.4 194495 22.f3 heartbeat_peers 4,285,312 2014-08-13 18:06:07.198341 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2682 2014-08-13 18:06:07.198352 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.198366 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.198375 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.198387 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.198397 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.198399 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.285 192.168.1.101:6823/26295 192.168.100.101:6845/26295 2014-08-13 18:06:07.198406 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.198416 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.198428 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.198438 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.198454 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.198459 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.312 192.168.1.113:6817/17521 192.168.100.113:6843/17521 2014-08-13 18:06:07.198464 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.198474 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.198486 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.198495 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.198504 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.198518 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.198523 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.311 192.168.1.101:6817/24270 192.168.100.101:6831/24270 2014-08-13 18:06:07.198529 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.198541 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.198557 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,58,343 unchanged 2014-08-13 18:06:07.198568 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.198582 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.369 192.168.1.108:6825/4037 192.168.100.108:6814/4037 2014-08-13 18:06:07.198581 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002154 8 0.002154 2014-08-13 18:06:07.198592 7f5c835ef700 20 osd.4 194495 22.116 heartbeat_peers 2014-08-13 18:06:07.198595 7f5c835ef700 20 osd.4 194495 24.3d heartbeat_peers 2014-08-13 18:06:07.198597 7f5c835ef700 20 osd.4 194495 22.4aa heartbeat_peers 2014-08-13 18:06:07.198599 7f5c835ef700 20 osd.4 194495 22.1d2 heartbeat_peers 4,61,349 2014-08-13 18:06:07.198594 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.198605 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.198616 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.198630 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000025 0 0.000000 2014-08-13 18:06:07.198643 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.198653 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.198664 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.198674 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.198686 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 18:06:07.198703 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [58,343](58) acting [58,343](58) maybe_went_rw) 2014-08-13 18:06:07.198718 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,58,343](4) acting [4,58,343](4) maybe_went_rw) 2014-08-13 18:06:07.198730 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 18:06:07.198742 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,349,362](61) acting [58,343,61](58)) 2014-08-13 18:06:07.198753 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,343,349,362 down blocked_by {} 2014-08-13 18:06:07.198765 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.198777 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2683 2014-08-13 18:06:07.198789 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 18:06:07.198801 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.198812 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 18:06:07.198823 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 18:06:07.198815 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.198834 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 18:06:07.198848 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.198861 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.198936 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.199019 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.330 192.168.1.106:6803/4876 192.168.100.106:6807/4876 2014-08-13 18:06:07.199407 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 18:06:07.199442 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.199449 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.199465 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292] newacting [9,292] 2014-08-13 18:06:07.199480 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.199502 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.173342 3 0.000129 2014-08-13 18:06:07.199522 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.173407 0 0.000000 2014-08-13 18:06:07.199543 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.199557 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194486 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.199573 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194488 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.199588 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194488 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.199602 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194488 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292] newacting [9,292] 2014-08-13 18:06:07.199616 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194488 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.199630 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292,4] r=2 lpr=194488 pi=192972-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.199660 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [9,292,4](9) acting [9,292,4](9)): not rw, up_thru 194485 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [9,292,4](9) acting [9,292,4](9) maybe_went_rw) : primary up 192972-194485 includes interval 2014-08-13 18:06:07.199676 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194485/194485/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [9,292,4](9) acting [9,292,4](9) maybe_went_rw) 2014-08-13 18:06:07.199693 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292,4] -> [9,292], acting [9,292,4] -> [9,292], acting_primary 9 -> 9, up_primary 9 -> 9, role 2 -> -1 2014-08-13 18:06:07.199712 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.199726 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.199740 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.199756 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.199771 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.199785 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.199801 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.199817 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.199832 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.199846 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.199859 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.199872 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.199884 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.199900 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.199935 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 18:06:07.199954 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.199959 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.199973 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.199988 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.200016 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292]/[9,292] -- 9/9 2014-08-13 18:06:07.200033 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.200037 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.200051 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.200066 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.200098 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 18:06:07.200116 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.200120 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.200133 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.200147 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292,362] newacting [9,292,362] 2014-08-13 18:06:07.200162 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.200176 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292] r=-1 lpr=194488 pi=192972-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.200221 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [9,292](9) acting [9,292](9)): not rw, up_thru 194488 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [9,292](9) acting [9,292](9) maybe_went_rw) : primary up 192972-194488 includes interval 2014-08-13 18:06:07.200348 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194488/194488/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [9,292](9) acting [9,292](9) maybe_went_rw) 2014-08-13 18:06:07.200384 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292] -> [9,292,362], acting [9,292] -> [9,292,362], acting_primary 9 -> 9, up_primary 9 -> 9, role -1 -> -1 2014-08-13 18:06:07.200404 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.200420 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.200434 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.200450 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.200464 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.200479 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.200493 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.200509 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.200525 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.200540 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.200553 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.200570 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.200615 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 18:06:07.200636 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.200641 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.200657 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.200674 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.201129 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.331 192.168.1.104:6817/837 192.168.100.104:6843/837 2014-08-13 18:06:07.201151 7f5c835ef700 20 osd.4 194495 24.1de heartbeat_peers 2014-08-13 18:06:07.201155 7f5c835ef700 20 osd.4 194495 2.2a heartbeat_peers 4,340 2014-08-13 18:06:07.200711 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 18:06:07.201654 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.201662 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.340 192.168.1.104:6807/62463 192.168.100.104:6822/62463 2014-08-13 18:06:07.201666 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.201687 7f5c835ef700 20 osd.4 194495 22.113 heartbeat_peers 4,45,339 2014-08-13 18:06:07.201691 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.201710 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.201755 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,362]/[9,292,362] -- 9/9 2014-08-13 18:06:07.201775 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.201779 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.201794 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.201810 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.201841 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 18:06:07.201859 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.201863 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.201878 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.201892 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [9,292,4] newacting [9,292,4] 2014-08-13 18:06:07.201907 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.201922 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,362] r=-1 lpr=194491 pi=192972-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.201953 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [9,292,362](9) acting [9,292,362](9)): not rw, up_thru 194491 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [9,292,362](9) acting [9,292,362](9) maybe_went_rw) : primary up 192972-194491 includes interval 2014-08-13 18:06:07.201970 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194491/194491/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [9,292,362](9) acting [9,292,362](9) maybe_went_rw) 2014-08-13 18:06:07.201987 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] up [9,292,362] -> [9,292,4], acting [9,292,362] -> [9,292,4], acting_primary 9 -> 9, up_primary 9 -> 9, role -1 -> 2 2014-08-13 18:06:07.202006 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.202020 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.202034 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.202050 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.202064 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.202077 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.202092 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.202108 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.202123 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.202140 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.202163 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.202177 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.202191 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.202207 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.202224 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.202286 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.202302 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.202318 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.002775 8 0.001540 2014-08-13 18:06:07.202338 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.202354 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.202367 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.202385 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000032 0 0.000000 2014-08-13 18:06:07.202404 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.202425 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.202440 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.202529 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.203378 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.45 192.168.1.112:6829/6040 192.168.100.112:6823/6040 2014-08-13 18:06:07.203418 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru now 194495 (was 0), currently 194486 2014-08-13 18:06:07.203432 7f5c7c7e4700 10 osd.4 194495 send_alive up_thru currently 194486 want 194495 2014-08-13 18:06:07.203435 7f5c7c7e4700 10 osd.4 194495 send_alive want 194495 2014-08-13 18:06:07.203493 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.339 192.168.1.110:6819/45268 192.168.100.110:6846/45268 2014-08-13 18:06:07.203695 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.24 192.168.1.107:6801/52669 192.168.100.107:6803/52669 2014-08-13 18:06:07.203802 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.55 192.168.1.112:6835/6966 192.168.100.112:6836/6966 2014-08-13 18:06:07.203927 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.342 192.168.1.110:6803/41438 192.168.100.110:6810/41438 2014-08-13 18:06:07.203942 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4423700 don't know epoch, doing nothing 2014-08-13 18:06:07.203959 7f5c835ef700 20 osd.4 194495 24.39a heartbeat_peers 2014-08-13 18:06:07.203960 7f5c7c7e4700 7 osd.4 194495 do_notify osd 9 on 1 PGs 2014-08-13 18:06:07.203965 7f5c835ef700 20 osd.4 194495 24.33a heartbeat_peers 2014-08-13 18:06:07.203968 7f5c835ef700 20 osd.4 194495 1.2b heartbeat_peers 4,340 2014-08-13 18:06:07.203973 7f5c835ef700 20 osd.4 194495 24.34 heartbeat_peers 2014-08-13 18:06:07.203976 7f5c835ef700 20 osd.4 194495 22.99 heartbeat_peers 4,31,374 2014-08-13 18:06:07.204071 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.31 192.168.1.103:6832/41157 192.168.100.103:6850/41157 2014-08-13 18:06:07.204081 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4af8160 don't know epoch, doing nothing 2014-08-13 18:06:07.204093 7f5c7c7e4700 7 osd.4 194495 do_notify osd 351 on 1 PGs 2014-08-13 18:06:07.204217 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:07.204225 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.58 on 1 PGs 2014-08-13 18:06:07.204210 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.374 192.168.1.102:6829/16129 192.168.100.102:6817/16129 2014-08-13 18:06:07.204796 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.20 192.168.1.103:6816/38269 192.168.100.103:6827/38269 2014-08-13 18:06:07.204805 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:07.204830 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.61 on 1 PGs 2014-08-13 18:06:07.205879 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:07.205873 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.337 192.168.1.104:6805/62197 192.168.100.104:6816/62197 2014-08-13 18:06:07.205899 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.343 on 1 PGs 2014-08-13 18:06:07.206889 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4fa4fc0 don't know epoch, doing nothing 2014-08-13 18:06:07.206882 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.363 192.168.1.102:6843/20017 192.168.100.102:6847/20017 2014-08-13 18:06:07.206904 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.349 on 1 PGs 2014-08-13 18:06:07.206907 7f5c835ef700 20 osd.4 194495 22.84 heartbeat_peers 4,28,304 2014-08-13 18:06:07.207004 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:07.207010 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.362 on 1 PGs 2014-08-13 18:06:07.206999 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.28 192.168.1.103:6808/37391 192.168.100.103:6813/37391 2014-08-13 18:06:07.207096 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.304 192.168.1.101:6809/22782 192.168.100.101:6812/22782 2014-08-13 18:06:07.207112 7f5c835ef700 20 osd.4 194495 22.4df heartbeat_peers 4,356,357 2014-08-13 18:06:07.207107 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 18:06:07.207139 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.207147 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.207176 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28] newacting [58,28] 2014-08-13 18:06:07.207194 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.207216 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.181863 3 0.000188 2014-08-13 18:06:07.207237 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started 0.181944 0 0.000000 2014-08-13 18:06:07.207260 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.207261 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.356 192.168.1.108:6833/5235 192.168.100.108:6838/5235 2014-08-13 18:06:07.207274 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.207291 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.207310 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.207324 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28] newacting [58,28] 2014-08-13 18:06:07.207339 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.207357 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.207393 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [58,28,4](58) acting [58,28,4](58)): not rw, up_thru 194485 up_from 192935 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [58,28,4](58) acting [58,28,4](58) maybe_went_rw) : primary up 192935-194485 includes interval 2014-08-13 18:06:07.207406 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.357 192.168.1.102:6835/17224 192.168.100.102:6835/17224 2014-08-13 18:06:07.207414 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [58,28,4](58) acting [58,28,4](58) maybe_went_rw) 2014-08-13 18:06:07.207431 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28,4] -> [58,28], acting [58,28,4] -> [58,28], acting_primary 58 -> 58, up_primary 58 -> 58, role 2 -> -1 2014-08-13 18:06:07.207450 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.207468 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.207482 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.207498 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.207514 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.207528 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.207546 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.207563 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.207578 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.207592 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.207605 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.207618 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.207634 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.207654 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.207687 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.352 192.168.1.108:6827/4273 192.168.100.108:6823/4273 2014-08-13 18:06:07.207699 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 18:06:07.207724 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.207729 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.207747 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.207763 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.207799 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28]/[58,28] -- 58/58 2014-08-13 18:06:07.207820 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.207825 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.207839 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.207857 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.207899 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 18:06:07.207899 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.371 192.168.1.102:6833/16753 192.168.100.102:6832/16753 2014-08-13 18:06:07.207920 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.207926 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.207944 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.207957 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28,284] newacting [58,28,284] 2014-08-13 18:06:07.207972 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.207986 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.208014 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [58,28](58) acting [58,28](58)): not rw, up_thru 194488 up_from 192935 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [58,28](58) acting [58,28](58) maybe_went_rw) : primary up 192935-194488 includes interval 2014-08-13 18:06:07.208029 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [58,28](58) acting [58,28](58) maybe_went_rw) 2014-08-13 18:06:07.208050 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28] -> [58,28,284], acting [58,28] -> [58,28,284], acting_primary 58 -> 58, up_primary 58 -> 58, role -1 -> -1 2014-08-13 18:06:07.208072 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.208086 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.208103 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.208118 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.208136 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.208146 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.380 192.168.1.114:6823/51844 192.168.100.114:6803/51844 2014-08-13 18:06:07.208150 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.208169 7f5c835ef700 20 osd.4 194495 22.39d heartbeat_peers 4,344,373 2014-08-13 18:06:07.208173 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.208193 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.208208 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.208222 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.208236 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.208251 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.208295 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 18:06:07.208314 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.208318 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.208333 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.208348 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.208387 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 18:06:07.208408 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.208411 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.208425 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.208440 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.208488 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,284]/[58,28,284] -- 58/58 2014-08-13 18:06:07.208508 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.208512 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.208528 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.208545 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.208583 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 18:06:07.208607 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.208611 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.208608 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.344 192.168.1.106:6813/6757 192.168.100.106:6837/6757 2014-08-13 18:06:07.208626 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.208640 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] acting_up_affected newup [58,28,4] newacting [58,28,4] 2014-08-13 18:06:07.208658 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.208673 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,284] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'481 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.208706 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [58,28,284](58) acting [58,28,284](58)): not rw, up_thru 194491 up_from 192935 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [58,28,284](58) acting [58,28,284](58) maybe_went_rw) : primary up 192935-194491 includes interval 2014-08-13 18:06:07.208738 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.373 192.168.1.108:6841/7435 192.168.100.108:6850/7435 2014-08-13 18:06:07.208740 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [58,28,284](58) acting [58,28,284](58) maybe_went_rw) 2014-08-13 18:06:07.208757 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] up [58,28,284] -> [58,28,4], acting [58,28,284] -> [58,28,4], acting_primary 58 -> 58, up_primary 58 -> 58, role -1 -> 2 2014-08-13 18:06:07.208778 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.208793 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.208806 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.208825 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.208829 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.320 192.168.1.106:6821/10050 192.168.100.106:6849/10050 2014-08-13 18:06:07.208838 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.208852 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.208866 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.208884 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.208899 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.208917 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.208930 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.208943 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.208959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.208975 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.208969 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.366 192.168.1.108:6829/4534 192.168.100.108:6832/4534 2014-08-13 18:06:07.208994 7f5c835ef700 20 osd.4 194495 22.2b9 heartbeat_peers 2014-08-13 18:06:07.208999 7f5c835ef700 20 osd.4 194495 24.2c heartbeat_peers 2014-08-13 18:06:07.209002 7f5c835ef700 20 osd.4 194495 24.3f7 heartbeat_peers 2014-08-13 18:06:07.208992 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.209006 7f5c835ef700 20 osd.4 194495 22.775 heartbeat_peers 4,56,353 2014-08-13 18:06:07.209054 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.209075 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.209091 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Reset 0.001832 8 0.001479 2014-08-13 18:06:07.209111 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.209128 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.209143 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.209170 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Start 0.000043 0 0.000000 2014-08-13 18:06:07.209190 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.209211 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.209230 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.209325 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.209539 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.56 192.168.1.111:6827/11446 192.168.100.111:6811/11446 2014-08-13 18:06:07.209809 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.353 192.168.1.108:6837/6227 192.168.100.108:6844/6227 2014-08-13 18:06:07.209870 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 18:06:07.209911 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.209923 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.209940 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.209961 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.209970 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.0 192.168.1.115:6811/59615 192.168.100.115:6825/59615 2014-08-13 18:06:07.209981 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.184303 2 0.000277 2014-08-13 18:06:07.210003 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.210018 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.184353 0 0.000000 2014-08-13 18:06:07.210039 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.184386 0 0.000000 2014-08-13 18:06:07.210061 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Started 0.184463 0 0.000000 2014-08-13 18:06:07.210097 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.210112 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194486 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.210133 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194488 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.210149 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194488 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.210182 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194488 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [301,324] newacting [301,324] 2014-08-13 18:06:07.210199 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194488 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.210214 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [4,301,324] r=0 lpr=194488 pi=193306-194484/26 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.210247 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,301,324](4) acting [4,301,324](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.210278 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194485/194485/194485) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] noting past interval(194485-194487 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 18:06:07.210285 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.63 192.168.1.111:6831/12680 192.168.100.111:6820/12680 2014-08-13 18:06:07.210307 7f5c835ef700 20 osd.4 194495 22.630 heartbeat_peers 2014-08-13 18:06:07.210296 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] up [4,301,324] -> [301,324], acting [4,301,324] -> [301,324], acting_primary 4 -> 301, up_primary 4 -> 301, role 0 -> -1 2014-08-13 18:06:07.210312 7f5c835ef700 20 osd.4 194495 24.e2 heartbeat_peers 2014-08-13 18:06:07.210318 7f5c835ef700 20 osd.4 194495 22.69f heartbeat_peers 2014-08-13 18:06:07.210321 7f5c835ef700 20 osd.4 194495 22.3e4 heartbeat_peers 4,313,318 2014-08-13 18:06:07.210315 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.210331 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.210346 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.210359 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.210374 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.210394 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.210408 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.210422 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.210435 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.210456 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.210471 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.210489 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.210503 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.210517 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.210529 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.210547 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.210562 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.210579 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.210582 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.313 192.168.1.109:6821/35445 192.168.100.109:6850/35445 2014-08-13 18:06:07.210622 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 18:06:07.210642 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.210651 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.210667 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.210686 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.210711 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.318 192.168.1.110:6815/43857 192.168.100.110:6840/43857 2014-08-13 18:06:07.210722 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [301,324]/[301,324] -- 301/301 2014-08-13 18:06:07.210739 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.210743 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.210761 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.210777 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.210824 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 18:06:07.210843 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.210847 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.303 192.168.1.109:6803/30768 192.168.100.109:6807/30768 2014-08-13 18:06:07.210848 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.210864 7f5c835ef700 20 osd.4 194495 24.2a1 heartbeat_peers 2014-08-13 18:06:07.210869 7f5c835ef700 20 osd.4 194495 22.639 heartbeat_peers 2014-08-13 18:06:07.210872 7f5c835ef700 20 osd.4 194495 22.707 heartbeat_peers 4,21,310 2014-08-13 18:06:07.210866 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.210880 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] acting_up_affected newup [302,324,368] newacting [302,324,368] 2014-08-13 18:06:07.210895 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.210917 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [301,324] r=-1 lpr=194488 pi=193306-194487/27 crt=191727'446 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.210948 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [301,324](301) acting [301,324](301)): not rw, up_thru 194488 up_from 192972 last_epoch_clean 194483 generate_past_intervals interval(194488-194490 up [301,324](301) acting [301,324](301) maybe_went_rw) : primary up 192972-194488 includes interval 2014-08-13 18:06:07.210964 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.21 192.168.1.107:6822/54786 192.168.100.107:6838/54786 2014-08-13 18:06:07.210966 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194488/194488/194488) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 18:06:07.210982 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] up [301,324] -> [302,324,368], acting [301,324] -> [302,324,368], acting_primary 301 -> 302, up_primary 301 -> 302, role -1 -> -1 2014-08-13 18:06:07.211003 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.211018 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.211032 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.211047 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.211061 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.211078 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.211092 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.211107 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.211126 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.211141 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] [301,324] -> [302,324,368], acting primary 301 -> 302 2014-08-13 18:06:07.211174 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.211158 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.310 192.168.1.109:6813/32413 192.168.100.109:6833/32413 2014-08-13 18:06:07.211188 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.211204 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.211250 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 18:06:07.211271 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.211275 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.211290 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.211305 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.211348 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 18:06:07.211367 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.211371 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.211385 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.211399 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.211436 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [302,324,368]/[302,324,368] -- 302/302 2014-08-13 18:06:07.211454 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.211461 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.211476 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.211491 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.211521 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 18:06:07.211538 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.211542 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.211555 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.211569 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,301,324] newacting [4,301,324] 2014-08-13 18:06:07.211584 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.211598 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [302,324,368] r=-1 lpr=194491 pi=193306-194490/28 crt=191727'446 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.211635 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [302,324,368](302) acting [302,324,368](302)): not rw, up_thru 194491 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194491-194494 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) : primary up 192970-194491 includes interval 2014-08-13 18:06:07.211655 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194491/194491/194491) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 18:06:07.211675 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [302,324,368] -> [4,301,324], acting [302,324,368] -> [4,301,324], acting_primary 302 -> 4, up_primary 302 -> 4, role -1 -> 0 2014-08-13 18:06:07.211780 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.15 192.168.1.107:6814/53408 192.168.100.107:6819/53408 2014-08-13 18:06:07.211803 7f5c835ef700 20 osd.4 194495 24.5e heartbeat_peers 2014-08-13 18:06:07.211807 7f5c835ef700 20 osd.4 194495 22.1f1 heartbeat_peers 4,5,47 2014-08-13 18:06:07.211695 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.212005 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.212037 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.212041 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.5 192.168.1.103:6830/40192 192.168.100.103:6847/40192 2014-08-13 18:06:07.212056 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2831 2014-08-13 18:06:07.212073 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.212090 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.212109 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.212123 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.212137 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.212159 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.212186 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.212201 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.212218 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.212233 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.212252 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.212268 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.212282 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.212295 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.212295 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.47 192.168.1.111:6829/12102 192.168.100.111:6815/12102 2014-08-13 18:06:07.212309 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.212323 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.212339 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.212362 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.212382 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,301,324 unchanged 2014-08-13 18:06:07.212401 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.212403 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.11 192.168.1.103:6814/37929 192.168.100.103:6821/37929 2014-08-13 18:06:07.212417 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002320 8 0.002109 2014-08-13 18:06:07.212439 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.212457 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.212471 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.212489 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 2014-08-13 18:06:07.212513 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.212529 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.212545 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.212560 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.212581 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 18:06:07.212603 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 18:06:07.212614 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.54 192.168.1.111:6837/14923 192.168.100.111:6833/14923 2014-08-13 18:06:07.212621 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,301,324](4) acting [4,301,324](4) maybe_went_rw) 2014-08-13 18:06:07.212641 7f5c835ef700 20 osd.4 194495 22.2d1 heartbeat_peers 2014-08-13 18:06:07.212646 7f5c835ef700 20 osd.4 194495 24.b6 heartbeat_peers 2014-08-13 18:06:07.212650 7f5c835ef700 20 osd.4 194495 24.3bf heartbeat_peers 2014-08-13 18:06:07.212655 7f5c835ef700 20 osd.4 194495 22.1fb heartbeat_peers 2014-08-13 18:06:07.212658 7f5c835ef700 20 osd.4 194495 24.350 heartbeat_peers 2014-08-13 18:06:07.212644 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 18:06:07.212661 7f5c835ef700 20 osd.4 194495 24.1f7 heartbeat_peers 2014-08-13 18:06:07.212664 7f5c835ef700 20 osd.4 194495 24.136 heartbeat_peers 2014-08-13 18:06:07.212667 7f5c835ef700 20 osd.4 194495 22.4fe heartbeat_peers 2014-08-13 18:06:07.212670 7f5c835ef700 20 osd.4 194495 22.653 heartbeat_peers 2014-08-13 18:06:07.212673 7f5c835ef700 20 osd.4 194495 24.3b7 heartbeat_peers 2014-08-13 18:06:07.212675 7f5c835ef700 20 osd.4 194495 22.3b9 heartbeat_peers 2014-08-13 18:06:07.212679 7f5c835ef700 20 osd.4 194495 22.791 heartbeat_peers 2014-08-13 18:06:07.212665 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 18:06:07.212681 7f5c835ef700 20 osd.4 194495 22.6bd heartbeat_peers 4,50,289 2014-08-13 18:06:07.212686 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,302,324,368 down blocked_by {} 2014-08-13 18:06:07.212705 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.212727 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2832 2014-08-13 18:06:07.212745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.301 2014-08-13 18:06:07.212763 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.302 2014-08-13 18:06:07.212787 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 18:06:07.212803 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 18:06:07.212820 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.212840 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.212833 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.50 192.168.1.112:6843/9607 192.168.100.112:6849/9607 2014-08-13 18:06:07.212943 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.212994 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.289 192.168.1.113:6809/15284 192.168.100.113:6828/15284 2014-08-13 18:06:07.213016 7f5c835ef700 20 osd.4 194495 24.181 heartbeat_peers 2014-08-13 18:06:07.213020 7f5c835ef700 20 osd.4 194495 24.20e heartbeat_peers 2014-08-13 18:06:07.213022 7f5c835ef700 20 osd.4 194495 24.1e4 heartbeat_peers 2014-08-13 18:06:07.213025 7f5c835ef700 20 osd.4 194495 24.1e6 heartbeat_peers 2014-08-13 18:06:07.213028 7f5c835ef700 20 osd.4 194495 22.12a heartbeat_peers 2014-08-13 18:06:07.213030 7f5c835ef700 20 osd.4 194495 22.3af heartbeat_peers 4,44,362 2014-08-13 18:06:07.213039 7f5c835ef700 20 osd.4 194495 22.481 heartbeat_peers 4,287,309 2014-08-13 18:06:07.213051 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_advance_map [65,62]/[65,62] -- 65/65 2014-08-13 18:06:07.213091 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.213099 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.213115 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.213134 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.213138 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.287 192.168.1.109:6815/33435 192.168.100.109:6839/33435 2014-08-13 18:06:07.213152 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.187046 2 0.000248 2014-08-13 18:06:07.213182 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.213202 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.187107 0 0.000000 2014-08-13 18:06:07.213223 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.187138 0 0.000000 2014-08-13 18:06:07.213245 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Started 0.187202 0 0.000000 2014-08-13 18:06:07.213267 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.213286 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194486 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.213306 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194488 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.213325 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194488 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.213339 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194488 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [65,62] newacting [65,62] 2014-08-13 18:06:07.213354 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194488 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.213369 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,65,62] r=0 lpr=194488 pi=193306-194484/32 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.213402 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,65,62](4) acting [4,65,62](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.213408 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.309 192.168.1.101:6815/23703 192.168.100.101:6825/23703 2014-08-13 18:06:07.213418 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] noting past interval(194485-194487 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) 2014-08-13 18:06:07.213435 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] up [4,65,62] -> [65,62], acting [4,65,62] -> [65,62], acting_primary 4 -> 65, up_primary 4 -> 65, role 0 -> -1 2014-08-13 18:06:07.213453 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.213473 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.213487 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.213504 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.213520 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.213540 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.213553 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.213551 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.306 192.168.1.101:6807/6812 192.168.100.101:6805/6812 2014-08-13 18:06:07.213567 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.213581 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.213598 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.213615 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.213628 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.213647 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.213660 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.213672 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.213689 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.213689 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.381 192.168.1.102:6823/15390 192.168.100.102:6802/15390 2014-08-13 18:06:07.213707 7f5c835ef700 20 osd.4 194495 22.48e heartbeat_peers 4,289,341 2014-08-13 18:06:07.213703 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.213719 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.213758 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [65,62]/[65,62,44] -- 65/65 2014-08-13 18:06:07.213779 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.213784 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.213799 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.213812 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] acting_up_affected newup [65,62] newacting [65,62,44] 2014-08-13 18:06:07.213827 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.213845 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'439 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.213873 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [65,62](65) acting [65,62](65)): not rw, up_thru 194482 up_from 192939 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [65,62](65) acting [65,62](65)) : primary up 192939-194482 does not include interval 2014-08-13 18:06:07.213895 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [65,62](65) acting [65,62](65)) 2014-08-13 18:06:07.213911 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] up [65,62] -> [65,62], acting [65,62] -> [65,62,44], acting_primary 65 -> 65, up_primary 65 -> 65, role -1 -> -1 2014-08-13 18:06:07.213933 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.213949 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.213963 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.213983 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.213997 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.213992 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.341 192.168.1.106:6819/9142 192.168.100.106:6846/9142 2014-08-13 18:06:07.214012 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.214026 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.214041 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.214056 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.214071 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.214085 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.214101 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.214133 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [65,62]/[65,62,44] -- 65/65 2014-08-13 18:06:07.214152 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.214157 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.214182 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.214198 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.214236 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [44,48,314]/[65,62,44] -- 44/65 2014-08-13 18:06:07.214256 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.214261 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.214280 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.214298 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] acting_up_affected newup [44,48,314] newacting [65,62,44] 2014-08-13 18:06:07.214318 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.214336 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [65,62]/[65,62,44] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'439 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.214365 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [65,62](65) acting [65,62,44](65)): not rw, up_thru 194489 up_from 192939 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [65,62](65) acting [65,62,44](65) maybe_went_rw) : primary up 192939-194489 includes interval 2014-08-13 18:06:07.214383 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194489/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [65,62](65) acting [65,62,44](65) maybe_went_rw) 2014-08-13 18:06:07.214602 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] up [65,62] -> [44,48,314], acting [65,62,44] -> [65,62,44], acting_primary 65 -> 65, up_primary 65 -> 44, role -1 -> -1 2014-08-13 18:06:07.214624 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.214641 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.214657 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.214674 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.214690 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.214705 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.214720 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.214738 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.214754 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.214770 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.214785 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.214802 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.214850 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] handle_advance_map [44,48,314]/[44,48,314] -- 44/44 2014-08-13 18:06:07.214872 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.214878 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.214894 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.214909 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] acting_up_affected newup [44,48,314] newacting [44,48,314] 2014-08-13 18:06:07.214925 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.214940 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314]/[65,62,44] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'439 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.214972 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [44,48,314](44) acting [65,62,44](65)): not rw, up_thru 194489 up_from 192939 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [44,48,314](44) acting [65,62,44](65)) : primary up 192939-194489 does not include interval 2014-08-13 18:06:07.214992 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194488) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 18:06:07.215009 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] up [44,48,314] -> [44,48,314], acting [65,62,44] -> [44,48,314], acting_primary 65 -> 44, up_primary 44 -> 44, role -1 -> -1 2014-08-13 18:06:07.215028 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.215041 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.215055 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.215070 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.215084 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.215098 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.215111 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.215126 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.215141 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.215155 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] [65,62,44] -> [44,48,314], acting primary 65 -> 44 2014-08-13 18:06:07.215179 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.215202 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.215219 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.215261 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [44,48,314]/[44,48,314] -- 44/44 2014-08-13 18:06:07.215280 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.215285 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.215300 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.215315 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.215351 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [44,48,314]/[44,48,314] -- 44/44 2014-08-13 18:06:07.215369 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.215373 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.215387 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.215402 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.215431 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 18:06:07.215450 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.215454 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.215468 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.215481 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,65,62] newacting [4,65,62] 2014-08-13 18:06:07.215497 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.215514 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [44,48,314] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'439 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.215544 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [44,48,314](44) acting [44,48,314](44)): not rw, up_thru 194492 up_from 192926 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) : primary up 192926-194492 includes interval 2014-08-13 18:06:07.215561 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194492/194492) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 18:06:07.215580 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [44,48,314] -> [4,65,62], acting [44,48,314] -> [4,65,62], acting_primary 44 -> 4, up_primary 44 -> 4, role -1 -> 0 2014-08-13 18:06:07.215599 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.215615 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.215630 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.215646 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2734 2014-08-13 18:06:07.215665 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.215684 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.215698 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.215712 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.215726 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.215739 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.215753 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.215767 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.215782 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.215797 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.215812 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.215826 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.215840 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.215854 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.215867 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.215880 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.215896 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.215912 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.215931 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,62,65 unchanged 2014-08-13 18:06:07.215947 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.215962 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002696 8 0.002490 2014-08-13 18:06:07.215982 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.215997 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.216011 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.216029 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 2014-08-13 18:06:07.216048 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.216064 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.216079 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.216094 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.216112 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 18:06:07.216131 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 18:06:07.216149 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [65,62](65) acting [65,62,44](65) maybe_went_rw) 2014-08-13 18:06:07.216166 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [65,62](65) acting [65,62](65)) 2014-08-13 18:06:07.216182 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,65,62](4) acting [4,65,62](4) maybe_went_rw) 2014-08-13 18:06:07.216210 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 18:06:07.216228 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 18:06:07.216245 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,48,62,65,314 down blocked_by {} 2014-08-13 18:06:07.216262 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.216281 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2735 2014-08-13 18:06:07.216297 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 18:06:07.216315 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.48 2014-08-13 18:06:07.216331 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.62 2014-08-13 18:06:07.216347 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.65 2014-08-13 18:06:07.216362 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.314 2014-08-13 18:06:07.216379 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.216395 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.216501 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.216779 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.327 192.168.1.106:6805/5112 192.168.100.106:6813/5112 2014-08-13 18:06:07.216837 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 18:06:07.216889 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.216898 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.216916 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331] newacting [54,331] 2014-08-13 18:06:07.216937 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.216958 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.190447 3 0.000126 2014-08-13 18:06:07.216979 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started 0.190511 0 0.000000 2014-08-13 18:06:07.217002 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.217017 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194486 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.217020 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.376 192.168.1.108:6839/6893 192.168.100.108:6847/6893 2014-08-13 18:06:07.217043 7f5c835ef700 20 osd.4 194495 22.3ad heartbeat_peers 2014-08-13 18:06:07.217033 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194488 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.217048 7f5c835ef700 20 osd.4 194495 24.206 heartbeat_peers 2014-08-13 18:06:07.217051 7f5c835ef700 20 osd.4 194495 24.1bb heartbeat_peers 2014-08-13 18:06:07.217054 7f5c835ef700 20 osd.4 194495 24.299 heartbeat_peers 2014-08-13 18:06:07.217057 7f5c835ef700 20 osd.4 194495 22.3d5 heartbeat_peers 4,286,343 2014-08-13 18:06:07.217049 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194488 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.217065 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194488 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331] newacting [54,331] 2014-08-13 18:06:07.217080 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194488 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.217098 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331,4] r=2 lpr=194488 pi=192980-194484/24 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.217132 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [54,331,4](54) acting [54,331,4](54)): not rw, up_thru 194485 up_from 192936 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [54,331,4](54) acting [54,331,4](54) maybe_went_rw) : primary up 192936-194485 includes interval 2014-08-13 18:06:07.217150 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194485/194485/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [54,331,4](54) acting [54,331,4](54) maybe_went_rw) 2014-08-13 18:06:07.217147 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.286 192.168.1.113:6821/19173 192.168.100.113:6849/19173 2014-08-13 18:06:07.217176 7f5c835ef700 20 osd.4 194495 22.537 heartbeat_peers 2014-08-13 18:06:07.217181 7f5c835ef700 20 osd.4 194495 22.1b8 heartbeat_peers 4,340,347 2014-08-13 18:06:07.217168 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331,4] -> [54,331], acting [54,331,4] -> [54,331], acting_primary 54 -> 54, up_primary 54 -> 54, role 2 -> -1 2014-08-13 18:06:07.217202 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.217218 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.217235 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.217252 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.217266 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.217283 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.217298 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.217304 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.347 192.168.1.106:6801/4640 192.168.100.106:6803/4640 2014-08-13 18:06:07.217315 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.217329 7f5c835ef700 20 osd.4 194495 22.29a heartbeat_peers 2014-08-13 18:06:07.217333 7f5c835ef700 20 osd.4 194495 24.cd heartbeat_peers 2014-08-13 18:06:07.217336 7f5c835ef700 20 osd.4 194495 22.226 heartbeat_peers 2014-08-13 18:06:07.217339 7f5c835ef700 20 osd.4 194495 22.3d9 heartbeat_peers 4,338,376 2014-08-13 18:06:07.217330 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.217346 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.217359 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.217376 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.217389 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.217407 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.217425 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.338 192.168.1.106:6815/25893 192.168.100.106:6802/25893 2014-08-13 18:06:07.217447 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 18:06:07.217466 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.217471 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.217490 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.217505 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.217538 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331]/[54,331] -- 54/54 2014-08-13 18:06:07.217558 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.217562 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.217580 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.217594 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.396 192.168.1.105:6815/41806 192.168.100.105:6817/41806 2014-08-13 18:06:07.217595 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.217638 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 18:06:07.217614 7f5c835ef700 20 osd.4 194495 24.2ec heartbeat_peers 2014-08-13 18:06:07.217657 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.217661 7f5c835ef700 20 osd.4 194495 22.1a0 heartbeat_peers 2014-08-13 18:06:07.217666 7f5c835ef700 20 osd.4 194495 24.369 heartbeat_peers 2014-08-13 18:06:07.217670 7f5c835ef700 10 osd.4 194495 adding neighbor peer osd.3 2014-08-13 18:06:07.217661 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.217677 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.217690 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331,310] newacting [54,331,310] 2014-08-13 18:06:07.217709 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.217724 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331] r=-1 lpr=194488 pi=192980-194487/25 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.217752 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [54,331](54) acting [54,331](54)): not rw, up_thru 194488 up_from 192936 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [54,331](54) acting [54,331](54) maybe_went_rw) : primary up 192936-194488 includes interval 2014-08-13 18:06:07.217771 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194488/194488/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [54,331](54) acting [54,331](54) maybe_went_rw) 2014-08-13 18:06:07.217796 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331] -> [54,331,310], acting [54,331] -> [54,331,310], acting_primary 54 -> 54, up_primary 54 -> 54, role -1 -> -1 2014-08-13 18:06:07.217814 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.217829 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.217842 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.217857 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.217871 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.217885 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.217898 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.217913 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.217928 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.217942 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.217956 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.217971 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.218007 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 18:06:07.218025 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.218030 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.218044 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.218059 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.218090 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 18:06:07.218112 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.218115 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.218129 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.218144 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.218184 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,310]/[54,331,310] -- 54/54 2014-08-13 18:06:07.218203 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.218207 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.218221 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.218235 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.218264 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 18:06:07.218282 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.218286 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.218300 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.218314 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] acting_up_affected newup [54,331,4] newacting [54,331,4] 2014-08-13 18:06:07.218329 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.218343 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,310] r=-1 lpr=194491 pi=192980-194490/26 crt=192888'431 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.218372 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [54,331,310](54) acting [54,331,310](54)): not rw, up_thru 194491 up_from 192936 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [54,331,310](54) acting [54,331,310](54) maybe_went_rw) : primary up 192936-194491 includes interval 2014-08-13 18:06:07.218387 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194491/194491/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [54,331,310](54) acting [54,331,310](54) maybe_went_rw) 2014-08-13 18:06:07.218404 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] up [54,331,310] -> [54,331,4], acting [54,331,310] -> [54,331,4], acting_primary 54 -> 54, up_primary 54 -> 54, role -1 -> 2 2014-08-13 18:06:07.218422 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.218436 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.218450 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.218464 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.218477 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.218491 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.218505 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.218520 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.218534 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.218548 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.218562 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.218574 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.218587 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.218602 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.218618 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.218685 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.218702 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.218717 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Reset 0.001715 8 0.001420 2014-08-13 18:06:07.218736 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.218751 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.218767 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.218785 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.218803 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.218823 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.218839 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.218929 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.219069 7f5c835ef700 10 osd.4 194495 _add_heartbeat_peer: new peer osd.3 192.168.1.107:6841/56287 192.168.100.107:6845/56287 2014-08-13 18:06:07.219092 7f5c835ef700 10 osd.4 194495 adding neighbor peer osd.5 2014-08-13 18:06:07.219102 7f5c835ef700 10 osd.4 194495 maybe_update_heartbeat_peers 70 peers, extras 3,5 2014-08-13 18:06:07.219119 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.219138 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] take_waiters 2014-08-13 18:06:07.219182 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Reset 0.308733 10 0.002358 2014-08-13 18:06:07.219206 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started 2014-08-13 18:06:07.219191 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 18:06:07.219236 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.219226 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Start 2014-08-13 18:06:07.219242 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: transitioning to Primary 2014-08-13 18:06:07.219242 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 18:06:07.219259 7f5c835ef700 7 osd.4 194495 activate_map version 194495 2014-08-13 18:06:07.219275 7f5c835ef700 1 osd.4 194495 pausing recovery (NORECOVER flag set) 2014-08-13 18:06:07.219262 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.219264 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Start 0.000039 0 0.000000 2014-08-13 18:06:07.219292 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219296 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219284 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [378,367] newacting [291,350] 2014-08-13 18:06:07.219291 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary 2014-08-13 18:06:07.219305 7f5c758d8700 20 osd.4 194495 agent_entry empty queue 2014-08-13 18:06:07.219303 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.219312 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219323 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219311 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering 2014-08-13 18:06:07.219327 7f5c849f1700 20 osd.4 194495 _dispatch 0x2002400 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.219321 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [4,378,367]/[4,291,350] r=0 lpr=194487 pi=193637-194485/31 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194488 2014-08-13 18:06:07.219331 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.219349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.219359 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,378,367](4) acting [4,291,350](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.219371 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194492-194494 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 18:06:07.219379 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194485/194486/194485) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] noting past interval(194486-194487 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 18:06:07.219395 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 18:06:07.219398 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] up [4,378,367] -> [378,367], acting [4,291,350] -> [291,350], acting_primary 4 -> 291, up_primary 4 -> 378, role 0 -> -1 2014-08-13 18:06:07.219417 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194489-194490 up [286,375](286) acting [375,10,301](375) maybe_went_rw) 2014-08-13 18:06:07.219421 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] clear_primary_state 2014-08-13 18:06:07.219445 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] agent_stop 2014-08-13 18:06:07.219444 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194488-194488 up [286,375](286) acting [375,10](375)) 2014-08-13 18:06:07.219463 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] on_change 2014-08-13 18:06:07.219464 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194486-194487 up [4,286,375](4) acting [4,375,10](4) maybe_went_rw) 2014-08-13 18:06:07.219480 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.219493 7f5c849f1700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.219502 7f5c849f1700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.219486 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194485-194485 up [4,286,375](4) acting [4,286,375](4)) 2014-08-13 18:06:07.219506 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219495 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:07.219510 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219506 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194483-194484 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 18:06:07.219518 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 18:06:07.219528 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219537 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219541 7f5c835ef700 20 osd.4 194495 _dispatch 0x476e580 pg_notify(24.3d(1) epoch 194495) v5 2014-08-13 18:06:07.219529 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194482-194482 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 18:06:07.219536 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 18:06:07.219551 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 18:06:07.219555 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 18:06:07.219571 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] on_change 2014-08-13 18:06:07.219571 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.219593 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 18:06:07.219606 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.20 2014-08-13 18:06:07.219596 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] publish_stats_to_osd 194495:30298 2014-08-13 18:06:07.219613 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x476e580 2014-08-13 18:06:07.219610 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 18:06:07.219617 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.10 2014-08-13 18:06:07.219627 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_stats 2014-08-13 18:06:07.219638 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.286 2014-08-13 18:06:07.219646 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.219658 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.301 2014-08-13 18:06:07.219675 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219663 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_role_change 2014-08-13 18:06:07.219679 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219677 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: querying info from osd.375 2014-08-13 18:06:07.219691 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.219680 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.219697 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.219700 7f5c849f1700 20 osd.4 194495 _dispatch 0x4cfaac0 osd_map(194495..194495 src has 185139..194495) v3 2014-08-13 18:06:07.219698 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.219700 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.219714 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.219721 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.219732 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.219771 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 18:06:07.219796 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.219801 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.219817 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.219838 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.219833 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.219864 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [378,367]/[291,350] -- 378/291 2014-08-13 18:06:07.219878 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.219885 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.219887 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:07.219892 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.10 on 1 PGs 2014-08-13 18:06:07.219890 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.219907 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.219926 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:07.219930 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.286 on 1 PGs 2014-08-13 18:06:07.219923 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.219950 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing 2014-08-13 18:06:07.219959 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.301 on 1 PGs 2014-08-13 18:06:07.219966 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [365,350,291]/[291,350] -- 365/291 2014-08-13 18:06:07.219981 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing 2014-08-13 18:06:07.219984 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.375 on 1 PGs 2014-08-13 18:06:07.219987 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.219994 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.220010 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.220025 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [365,350,291] newacting [291,350] 2014-08-13 18:06:07.220041 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.220057 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [378,367]/[291,350] r=-1 lpr=194488 pi=193637-194487/32 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194491 2014-08-13 18:06:07.220064 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_advance_map [356,357]/[356,357] -- 356/356 2014-08-13 18:06:07.220093 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.220085 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [378,367](378) acting [291,350](291)): not rw, up_thru 194488 up_from 192974 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [378,367](378) acting [291,350](291) maybe_went_rw) : primary up 192974-194488 includes interval 2014-08-13 18:06:07.220101 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.220103 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194488/194488/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] noting past interval(194488-194490 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 18:06:07.220121 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.220123 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] up [378,367] -> [365,350,291], acting [291,350] -> [291,350], acting_primary 291 -> 291, up_primary 378 -> 365, role -1 -> -1 2014-08-13 18:06:07.220137 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.220143 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.220176 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.150687 2 0.000334 2014-08-13 18:06:07.220160 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.220199 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.220201 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.220216 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.150745 0 0.000000 2014-08-13 18:06:07.220220 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.220238 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.150788 0 0.000000 2014-08-13 18:06:07.220237 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.220256 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.220256 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Started 0.150850 0 0.000000 2014-08-13 18:06:07.220273 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.220277 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.220291 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.220293 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194486 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.220312 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194488 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.220308 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.220328 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194488 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.220330 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.220344 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194488 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [356,357] newacting [356,357] 2014-08-13 18:06:07.220346 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.220360 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194488 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.220364 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.220376 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [4,356,357] r=0 lpr=194488 pi=193306-194484/28 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.220409 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,356,357](4) acting [4,356,357](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.220412 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [365,350,291]/[365,350,291] -- 365/365 2014-08-13 18:06:07.220439 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.220426 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194485/194485/194485) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] noting past interval(194485-194487 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 18:06:07.220444 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.220444 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] up [4,356,357] -> [356,357], acting [4,356,357] -> [356,357], acting_primary 4 -> 356, up_primary 4 -> 356, role 0 -> -1 2014-08-13 18:06:07.220463 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.220465 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.220480 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [365,350,291] newacting [365,350,291] 2014-08-13 18:06:07.220484 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.220497 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.220500 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.220513 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291]/[291,350] r=-1 lpr=194491 pi=193637-194490/33 crt=194422'17302 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194492 2014-08-13 18:06:07.220515 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.220530 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.220546 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.220544 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [365,350,291](365) acting [291,350](291)): not rw, up_thru 194488 up_from 192974 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [365,350,291](365) acting [291,350](291)) : primary up 192974-194488 does not include interval 2014-08-13 18:06:07.220561 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.220562 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194491/194488) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] noting past interval(194491-194491 up [365,350,291](365) acting [291,350](291)) 2014-08-13 18:06:07.220577 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.220581 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] up [365,350,291] -> [365,350,291], acting [291,350] -> [365,350,291], acting_primary 291 -> 365, up_primary 365 -> 365, role -1 -> -1 2014-08-13 18:06:07.220592 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.220602 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.220611 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.220618 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.220627 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.220634 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.220643 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.220651 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.220659 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.220667 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.220674 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.220683 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.220689 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.220698 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.220704 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.220715 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.220718 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.220732 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.220736 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.220747 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] [291,350] -> [365,350,291], acting primary 291 -> 365 2014-08-13 18:06:07.220773 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.220774 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [356,357]/[356,357,371] -- 356/356 2014-08-13 18:06:07.220795 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.220788 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.220803 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.220805 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.220818 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.220832 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] acting_up_affected newup [356,357] newacting [356,357,371] 2014-08-13 18:06:07.220847 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.220849 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [365,350,291]/[365,350,291] -- 365/365 2014-08-13 18:06:07.220862 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357] r=-1 lpr=194488 pi=193306-194487/29 crt=191727'433 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.220873 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.220882 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.220894 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [356,357](356) acting [356,357](356)): not rw, up_thru 194486 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [356,357](356) acting [356,357](356)) : primary up 192969-194486 does not include interval 2014-08-13 18:06:07.220899 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.220915 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.220913 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194488/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [356,357](356) acting [356,357](356)) 2014-08-13 18:06:07.220931 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] up [356,357] -> [356,357], acting [356,357] -> [356,357,371], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.220949 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.220956 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [365,350,291]/[365,350,291] -- 365/365 2014-08-13 18:06:07.220965 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.220979 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.220980 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.220983 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.220997 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.220999 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221012 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.221016 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.221028 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.221046 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.221046 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [4,378,367]/[4,378,367] -- 4/4 2014-08-13 18:06:07.221070 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.221064 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.221074 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.221081 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.221091 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221098 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.221107 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] acting_up_affected newup [4,378,367] newacting [4,378,367] 2014-08-13 18:06:07.221114 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.221124 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.221132 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.221140 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [365,350,291] r=-1 lpr=194492 pi=193637-194491/34 crt=194422'17302 lcod 0'0 inactive NOTIFY m=59] set_last_peering_reset 194495 2014-08-13 18:06:07.221156 7f5c849f1700 3 osd.4 194495 handle_osd_map epochs [194495,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221167 7f5c849f1700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221171 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221173 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221178 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [365,350,291](365) acting [365,350,291](365)): not rw, up_thru 194492 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) : primary up 192966-194492 includes interval 2014-08-13 18:06:07.221185 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [356,357]/[356,357,371] -- 356/356 2014-08-13 18:06:07.221208 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.221202 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221196 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194491/194492/194492) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] noting past interval(194492-194494 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 18:06:07.221214 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221217 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3180 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221214 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.221216 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] up [365,350,291] -> [4,378,367], acting [365,350,291] -> [4,378,367], acting_primary 365 -> 4, up_primary 365 -> 4, role -1 -> 0 2014-08-13 18:06:07.221236 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221244 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221233 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221247 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221250 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221239 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] clear_primary_state 2014-08-13 18:06:07.221251 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.221258 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] agent_stop 2014-08-13 18:06:07.221285 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221275 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.221288 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221293 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cffbc0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221301 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221305 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221308 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221294 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd 194495:25289 2014-08-13 18:06:07.221311 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221292 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [352,371,380]/[356,357,371] -- 352/356 2014-08-13 18:06:07.221315 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.221313 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.221319 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.221339 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221343 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221331 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.221346 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3840 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221338 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221353 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221357 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221360 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221362 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221348 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.221356 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] acting_up_affected newup [352,371,380] newacting [356,357,371] 2014-08-13 18:06:07.221367 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.221385 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221389 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221375 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.221391 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfdc40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221384 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.221397 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221403 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221405 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221393 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [356,357]/[356,357,371] r=-1 lpr=194489 pi=193306-194488/30 crt=191727'433 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.221407 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221401 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.221430 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221433 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221423 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.221436 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfe540 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221424 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [356,357](356) acting [356,357,371](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [356,357](356) acting [356,357,371](356) maybe_went_rw) : primary up 192969-194489 includes interval 2014-08-13 18:06:07.221442 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221446 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221449 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221451 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221440 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.221444 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194488/194489/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [356,357](356) acting [356,357,371](356) maybe_went_rw) 2014-08-13 18:06:07.221473 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221476 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221459 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.221478 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee0000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221463 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] up [356,357] -> [352,371,380], acting [356,357,371] -> [356,357,371], acting_primary 356 -> 356, up_primary 356 -> 352, role -1 -> -1 2014-08-13 18:06:07.221484 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221489 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221491 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221494 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221481 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.221484 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.221499 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] clear_stats 2014-08-13 18:06:07.221502 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.221519 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221522 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221524 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc4c80 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221516 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 18:06:07.221530 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221520 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.221535 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221538 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221542 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221533 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] on_role_change 2014-08-13 18:06:07.221538 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.221550 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 18:06:07.221555 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.221571 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221574 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221577 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee7080 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221567 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] cancel_recovery 2014-08-13 18:06:07.221583 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221588 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221573 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.221590 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221593 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221584 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] clear_recovery_state 2014-08-13 18:06:07.221591 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.221615 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221618 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221603 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.221621 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfb600 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221611 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.221627 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221632 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221634 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221636 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221624 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_activate_map 2014-08-13 18:06:07.221629 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.221645 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] update_heartbeat_peers -> 4,367,378 2014-08-13 18:06:07.221645 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.221661 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221663 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.221664 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221670 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cf8000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221663 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.221677 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221684 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221669 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] take_waiters 2014-08-13 18:06:07.221687 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221690 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221685 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.221689 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.589525 10 0.002084 2014-08-13 18:06:07.221713 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221720 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221722 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed57c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221711 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started 2014-08-13 18:06:07.221728 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221733 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221735 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221738 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221729 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Start 2014-08-13 18:06:07.221728 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] handle_advance_map [352,371,380]/[352,371,380] -- 352/352 2014-08-13 18:06:07.221751 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.221759 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221762 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221747 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary 2014-08-13 18:06:07.221764 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfb840 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221755 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.221771 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221776 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221778 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221781 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221768 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000039 0 0.000000 2014-08-13 18:06:07.221774 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221805 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221790 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 18:06:07.221808 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221791 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] acting_up_affected newup [352,371,380] newacting [352,371,380] 2014-08-13 18:06:07.221810 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee1440 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221817 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221821 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221823 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221809 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering 2014-08-13 18:06:07.221827 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221811 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.221830 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380]/[356,357,371] r=-1 lpr=194491 pi=193306-194490/31 crt=191727'433 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.221830 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.221848 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221851 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221855 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee21c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221860 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221849 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.221865 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221869 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221871 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221864 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [352,371,380](352) acting [356,357,371](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [352,371,380](352) acting [356,357,371](356)) : primary up 192969-194489 does not include interval 2014-08-13 18:06:07.221870 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194492-194494 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 18:06:07.221892 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221895 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221899 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc5340 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221884 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194491/194488) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 18:06:07.221905 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221910 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221893 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194491-194491 up [365,350,291](365) acting [291,350](291)) 2014-08-13 18:06:07.221913 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221915 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221904 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] up [352,371,380] -> [352,371,380], acting [356,357,371] -> [352,371,380], acting_primary 356 -> 352, up_primary 352 -> 352, role -1 -> -1 2014-08-13 18:06:07.221915 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194488-194490 up [378,367](378) acting [291,350](291) maybe_went_rw) 2014-08-13 18:06:07.221930 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221926 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.221942 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221946 7f5c849f1700 20 osd.4 194495 _dispatch 0x4cfa880 osd_map(194495..194495 src has 185139..194495) v3 2014-08-13 18:06:07.221937 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194486-194487 up [4,378,367](4) acting [4,291,350](4) maybe_went_rw) 2014-08-13 18:06:07.221943 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.221959 7f5c849f1700 3 osd.4 194495 handle_osd_map epochs [194495,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.221967 7f5c849f1700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.221970 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.221972 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.221960 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.221959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194485-194485 up [4,378,367](4) acting [4,378,367](4)) 2014-08-13 18:06:07.221979 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.221981 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 18:06:07.221999 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.222001 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [365,350,291](365) acting [291,350](291)) 2014-08-13 18:06:07.222014 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.222020 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 18:06:07.222030 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.222040 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.222048 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.222060 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] publish_stats_to_osd 194495:25290 2014-08-13 18:06:07.222065 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.222078 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.291 2014-08-13 18:06:07.222080 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] [356,357,371] -> [352,371,380], acting primary 356 -> 352 2014-08-13 18:06:07.222101 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.350 2014-08-13 18:06:07.222107 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.222120 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.365 2014-08-13 18:06:07.222122 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.222137 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.367 2014-08-13 18:06:07.222139 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.222154 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.378 2014-08-13 18:06:07.222182 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222199 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222187 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [352,371,380]/[352,371,380] -- 352/352 2014-08-13 18:06:07.222202 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfcec0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222191 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.222209 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.222212 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222219 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222221 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222224 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222214 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.222212 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.222231 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.222259 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222262 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222247 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.222265 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee1680 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222271 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222276 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222278 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222280 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222304 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222292 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [352,371,380]/[352,371,380] -- 352/352 2014-08-13 18:06:07.222307 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222311 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.222310 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee2400 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222317 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222321 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222323 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222324 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.222325 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222315 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.222331 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.222346 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222349 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222351 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfa400 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222346 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.222357 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222364 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222367 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222369 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222392 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222394 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222396 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee7740 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222402 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222407 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222385 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,30]/[320,30] -- 320/320 2014-08-13 18:06:07.222409 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222412 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222412 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.222418 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.222437 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222440 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222442 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc3a80 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222435 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,30] newacting [320,30] 2014-08-13 18:06:07.222451 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222457 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222460 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222462 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222455 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.222486 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222489 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222474 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.195579 3 0.000472 2014-08-13 18:06:07.222491 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cf9d40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222497 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222501 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222503 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222505 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222494 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started 0.195639 0 0.000000 2014-08-13 18:06:07.222514 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.222525 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222531 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222534 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc33c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222539 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222528 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194486 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.222544 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222546 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222549 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222546 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194488 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.222569 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222572 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222561 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194488 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.222575 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3a80 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222581 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222585 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222587 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222590 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222576 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194488 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,30] newacting [320,30] 2014-08-13 18:06:07.222593 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194488 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.222612 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222615 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222617 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc60c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222607 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,4,30] r=1 lpr=194488 pi=192979-194484/29 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.222623 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222627 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222629 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222631 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222638 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [320,4,30](320) acting [320,4,30](320)): not rw, up_thru 194485 up_from 192979 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [320,4,30](320) acting [320,4,30](320) maybe_went_rw) : primary up 192979-194485 includes interval 2014-08-13 18:06:07.222652 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222657 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222659 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc5a00 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222665 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222669 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222655 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194485/194485/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [320,4,30](320) acting [320,4,30](320) maybe_went_rw) 2014-08-13 18:06:07.222672 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222674 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222673 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,4,30] -> [320,30], acting [320,4,30] -> [320,30], acting_primary 320 -> 320, up_primary 320 -> 320, role 1 -> -1 2014-08-13 18:06:07.222697 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222700 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222702 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee2f40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222693 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.222709 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222714 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222716 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222718 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222711 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.222731 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.222757 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222746 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.222760 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222762 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc7500 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222768 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222772 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222760 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.222775 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222777 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222776 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.222797 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222800 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222802 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc69c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222793 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.222808 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222812 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222815 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222817 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222810 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.222826 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.222838 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222841 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222844 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed6300 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222850 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222841 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.222854 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222857 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222859 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222855 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.222880 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222870 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.222882 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222886 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee5580 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222892 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222883 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.222897 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222899 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222901 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222900 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.222924 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222927 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222928 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3f00 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222934 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222938 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222940 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222942 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222935 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,30]/[320,30,12] -- 320/320 2014-08-13 18:06:07.222956 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.222964 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222967 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222969 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed0000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.222960 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.222975 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.222979 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.222982 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.222984 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.222976 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.222990 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,30] newacting [320,30,12] 2014-08-13 18:06:07.223006 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223009 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223011 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cf9200 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223017 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223007 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.223021 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223023 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223026 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223023 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30] r=-1 lpr=194488 pi=192979-194487/30 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.223049 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223052 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223055 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee5340 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223061 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223065 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223051 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [320,30](320) acting [320,30](320)): not rw, up_thru 194485 up_from 192979 last_epoch_clean 194483 generate_past_intervals interval(194488-194488 up [320,30](320) acting [320,30](320)) : primary up 192979-194485 does not include interval 2014-08-13 18:06:07.223067 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223069 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223069 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194488/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [320,30](320) acting [320,30](320)) 2014-08-13 18:06:07.223093 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223095 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223097 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfe780 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223087 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] up [320,30] -> [320,30], acting [320,30] -> [320,30,12], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 18:06:07.223104 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223108 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223111 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223113 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223107 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.223133 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223136 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223123 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.223141 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cff2c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223147 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223151 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223154 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223142 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.223156 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223176 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223178 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223159 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.223180 7f5c835ef700 20 osd.4 194495 _dispatch 0x5155580 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223184 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.223194 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223203 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223206 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223209 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223201 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.223217 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.223231 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223233 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223236 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed4140 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223243 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223247 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223234 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.223250 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223252 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223252 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.223273 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223276 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223278 7f5c835ef700 20 osd.4 194495 _dispatch 0x5156300 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223268 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.223284 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223290 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223292 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223295 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223283 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.223301 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.223316 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223318 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223320 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cf86c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223326 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223329 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223331 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223333 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223334 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] handle_advance_map [320,30]/[320,30,12] -- 320/320 2014-08-13 18:06:07.223353 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223354 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.223356 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223358 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc1d40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223364 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223368 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223370 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223373 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223359 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.223376 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.223394 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223397 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223399 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed6e40 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223391 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.223405 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223409 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223411 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223413 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223435 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223438 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223440 7f5c835ef700 20 osd.4 194495 _dispatch 0x5151440 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223428 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] handle_advance_map [320,12,343]/[320,30,12] -- 320/320 2014-08-13 18:06:07.223446 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223447 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.223450 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223453 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223456 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223452 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.223477 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223480 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223468 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.223482 7f5c835ef700 20 osd.4 194495 _dispatch 0x5150d80 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223488 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223492 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223494 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223497 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223483 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] acting_up_affected newup [320,12,343] newacting [320,30,12] 2014-08-13 18:06:07.223500 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.223518 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223520 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223522 7f5c835ef700 20 osd.4 194495 _dispatch 0x5151b00 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223515 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,30]/[320,30,12] r=-1 lpr=194489 pi=192979-194488/31 crt=192888'497 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.223528 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223532 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223534 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223536 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223556 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223559 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223544 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [320,30](320) acting [320,30,12](320)): not rw, up_thru 194489 up_from 192979 last_epoch_clean 194483 generate_past_intervals interval(194489-194490 up [320,30](320) acting [320,30,12](320) maybe_went_rw) : primary up 192979-194489 includes interval 2014-08-13 18:06:07.223563 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc0000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223569 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223573 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.223576 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223578 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223565 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194488/194489/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [320,30](320) acting [320,30,12](320) maybe_went_rw) 2014-08-13 18:06:07.223599 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.223602 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223584 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] up [320,30] -> [320,12,343], acting [320,30,12] -> [320,30,12], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 18:06:07.223606 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfc800 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223613 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.223618 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226217 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226223 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226246 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226249 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226251 7f5c835ef700 20 osd.4 194495 _dispatch 0x5154140 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.223607 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.226257 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226262 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226264 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226266 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.223645 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 18:06:07.226289 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226282 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.226291 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226294 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cff980 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226300 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226304 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226307 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226309 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226273 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.226295 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.226315 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.226332 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226319 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.226335 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226338 7f5c835ef700 20 osd.4 194495 _dispatch 0x5154ec0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226345 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226352 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226335 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,356,357] newacting [4,356,357] 2014-08-13 18:06:07.226336 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.226354 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226357 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226355 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.226356 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.226373 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [352,371,380] r=-1 lpr=194492 pi=193306-194491/32 crt=191727'433 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.226374 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.226390 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226393 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226396 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc1680 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226402 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226392 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.226406 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226412 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226414 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226408 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [352,371,380](352) acting [352,371,380](352)): not rw, up_thru 194492 up_from 192964 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) : primary up 192964-194492 includes interval 2014-08-13 18:06:07.226413 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.226436 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226439 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226426 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194491/194492/194492) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 18:06:07.226443 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc0d80 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226432 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.226450 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226455 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226458 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226461 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226450 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.226446 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [352,371,380] -> [4,356,357], acting [352,371,380] -> [4,356,357], acting_primary 352 -> 4, up_primary 352 -> 4, role -1 -> 0 2014-08-13 18:06:07.226467 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.226482 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226469 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.226485 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226487 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc2f40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226495 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226499 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226502 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226505 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226488 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.226487 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.226513 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.226527 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226530 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226532 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc45c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226538 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226542 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226545 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226531 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2659 2014-08-13 18:06:07.226547 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226549 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.226543 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 18:06:07.226569 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226572 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226572 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.226575 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed3600 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226581 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226568 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.226586 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226589 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226592 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226579 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.226586 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.226614 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226599 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.226617 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226605 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.226620 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc72c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226626 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226631 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226634 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226617 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] acting_up_affected newup [320,12,343] newacting [320,12,343] 2014-08-13 18:06:07.226621 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.226637 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226639 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.226638 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.226657 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226663 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226665 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfe300 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226657 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.226671 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226658 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343]/[320,30,12] r=-1 lpr=194491 pi=192979-194490/32 crt=192888'497 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.226677 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226680 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226682 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226675 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.226701 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226704 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226706 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc7080 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226694 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.226696 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [320,12,343](320) acting [320,30,12](320)): not rw, up_thru 194489 up_from 192979 last_epoch_clean 194483 generate_past_intervals interval(194491-194491 up [320,12,343](320) acting [320,30,12](320)) : primary up 192979-194489 does not include interval 2014-08-13 18:06:07.226713 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226717 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226720 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226722 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226712 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.226716 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194491/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [320,12,343](320) acting [320,30,12](320)) 2014-08-13 18:06:07.226729 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.226743 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226749 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226735 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,12,343] -> [320,12,343], acting [320,30,12] -> [320,12,343], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 18:06:07.226752 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfad00 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226758 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226762 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226749 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.226765 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226768 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226755 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.226766 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.226772 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.226787 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.226783 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.226805 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226809 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226811 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed5100 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226817 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226804 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.226803 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.226824 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226827 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226830 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226826 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.226825 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.226851 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226854 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226843 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.226857 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfaf40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226846 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.226863 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226868 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226870 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226873 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226859 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.226867 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.226877 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.226895 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226900 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226902 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfd580 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226891 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,356,357 unchanged 2014-08-13 18:06:07.226893 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.226909 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226915 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226918 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226920 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226910 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.226909 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.226927 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.226940 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226944 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226930 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.006652 8 0.002522 2014-08-13 18:06:07.226947 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee33c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226953 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226958 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.226944 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.226960 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226962 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226951 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.226968 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.226986 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.226989 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.226991 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed0d80 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.226982 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.226997 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.226985 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 18:06:07.227002 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227008 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227008 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.227011 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227003 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000035 0 0.000000 2014-08-13 18:06:07.227013 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.227034 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227037 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227025 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.227039 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed06c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227046 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227034 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.227053 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227055 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227058 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227044 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.227054 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.227064 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.227078 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227081 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227084 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee4140 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227090 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227094 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227081 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.227097 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227100 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227092 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,12,343]/[320,12,343] -- 320/320 2014-08-13 18:06:07.227113 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.227102 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 18:06:07.227119 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227123 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227127 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee6540 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227117 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.227133 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227138 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227141 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227124 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 18:06:07.227143 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227133 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.227151 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.227166 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227169 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227171 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3600 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227144 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [356,357](356) acting [356,357,371](356) maybe_went_rw) 2014-08-13 18:06:07.227178 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227182 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227193 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227196 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227181 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [356,357](356) acting [356,357](356)) 2014-08-13 18:06:07.227194 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 18:06:07.227200 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,356,357](4) acting [4,356,357](4) maybe_went_rw) 2014-08-13 18:06:07.227216 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.227220 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227224 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227227 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee4800 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227233 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227220 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.227220 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 18:06:07.227238 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227245 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227248 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227239 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.227245 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 18:06:07.227255 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] acting_up_affected newup [320,4,30] newacting [320,4,30] 2014-08-13 18:06:07.227275 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227278 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227280 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee6780 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227266 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,352,356,357,371,380 down blocked_by {} 2014-08-13 18:06:07.227272 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.227287 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227292 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227295 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227297 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227286 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.227289 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,12,343] r=-1 lpr=194492 pi=192979-194491/33 crt=192888'497 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.227317 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227320 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227309 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2660 2014-08-13 18:06:07.227322 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee69c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227329 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227334 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227337 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227322 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [320,12,343](320) acting [320,12,343](320)): not rw, up_thru 194492 up_from 192979 last_epoch_clean 194483 generate_past_intervals interval(194492-194494 up [320,12,343](320) acting [320,12,343](320) maybe_went_rw) : primary up 192979-194492 includes interval 2014-08-13 18:06:07.227340 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227328 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.352 2014-08-13 18:06:07.227341 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194491/194492/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [320,12,343](320) acting [320,12,343](320) maybe_went_rw) 2014-08-13 18:06:07.227363 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227350 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.356 2014-08-13 18:06:07.227366 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227371 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed2f40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227360 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] up [320,12,343] -> [320,4,30], acting [320,12,343] -> [320,4,30], acting_primary 320 -> 320, up_primary 320 -> 320, role -1 -> 1 2014-08-13 18:06:07.227379 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227386 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227372 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.357 2014-08-13 18:06:07.227389 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227392 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227382 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.227391 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.371 2014-08-13 18:06:07.227399 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.227413 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227416 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227419 7f5c835ef700 20 osd.4 194495 _dispatch 0x5160000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227410 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.380 2014-08-13 18:06:07.227425 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227416 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.227430 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227433 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227436 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227433 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.227431 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.227458 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227461 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227463 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed7080 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227452 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.227453 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.227470 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227475 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227477 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227480 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227469 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.227486 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.227501 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227504 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227506 7f5c835ef700 20 osd.4 194495 _dispatch 0x5168000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227512 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227502 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.227517 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227521 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227525 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227521 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.227545 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227548 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227538 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.227550 7f5c835ef700 20 osd.4 194495 _dispatch 0x5150b40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227557 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227561 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.227561 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227564 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227553 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.227567 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227567 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.227587 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227590 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227581 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.227592 7f5c835ef700 20 osd.4 194495 _dispatch 0x5152400 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227598 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227603 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227605 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227607 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227597 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.227616 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.227629 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227632 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227613 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_advance_map [373,344]/[373,344] -- 373/373 2014-08-13 18:06:07.227635 7f5c835ef700 20 osd.4 194495 _dispatch 0x5153840 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227640 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.227641 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227645 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227647 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227649 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227648 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.227673 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227676 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227665 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.227678 7f5c835ef700 20 osd.4 194495 _dispatch 0x5154800 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227684 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227688 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227691 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227694 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227681 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.227689 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.227701 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.157695 2 0.000231 2014-08-13 18:06:07.227717 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227720 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227709 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.227723 7f5c835ef700 20 osd.4 194495 _dispatch 0x5155c40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227729 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227733 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227736 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227738 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227727 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Reset 0.005213 8 0.004856 2014-08-13 18:06:07.227759 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227762 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227751 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.227764 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc0fc0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227771 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227775 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227777 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227781 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227768 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.227784 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.227800 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227803 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227804 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc3180 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227810 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227814 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227816 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227818 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227723 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.227837 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227837 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.157839 0 0.000000 2014-08-13 18:06:07.227802 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.227840 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227892 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee6e40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227905 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227910 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227912 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227915 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227864 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.157876 0 0.000000 2014-08-13 18:06:07.227939 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227942 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227944 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc2880 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227949 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227953 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227958 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227960 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227979 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227981 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227983 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee3cc0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.227989 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.227993 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.227995 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.227997 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228021 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228024 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228026 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc06c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.228032 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.228006 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Started 0.158060 0 0.000000 2014-08-13 18:06:07.228051 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.228038 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.228068 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194486 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.228081 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228084 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.227992 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.228086 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194488 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.228109 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228112 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228115 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed2400 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.228106 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.228122 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.228127 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.228129 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228132 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228126 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.228268 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228275 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228277 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf0e00 pg_notify(22.99(30) epoch 194495) v5 2014-08-13 18:06:07.228234 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.228360 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.20 2014-08-13 18:06:07.228103 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194488 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.228364 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf0e00 2014-08-13 18:06:07.228372 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194488 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [373,344] newacting [373,344] 2014-08-13 18:06:07.228400 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194488 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.228415 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [4,373,344] r=0 lpr=194488 pi=193306-194484/30 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.228489 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,373,344](4) acting [4,373,344](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.228514 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194485/194485/194485) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] noting past interval(194485-194487 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) 2014-08-13 18:06:07.228531 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228536 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228532 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] up [4,373,344] -> [373,344], acting [4,373,344] -> [373,344], acting_primary 4 -> 373, up_primary 4 -> 373, role 0 -> -1 2014-08-13 18:06:07.228552 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.228619 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228622 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228624 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4280 pg_notify(24.9b(38) epoch 194495) v5 2014-08-13 18:06:07.228703 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.291 2014-08-13 18:06:07.228567 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.228707 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4280 2014-08-13 18:06:07.228704 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 18:06:07.228742 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.228710 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.228748 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.228764 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.228805 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.228820 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.228834 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.228853 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228857 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228849 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.228866 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.228880 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.228883 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.228752 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.228885 7f5c835ef700 20 osd.4 194495 _dispatch 0x21abc00 pg_query(22.4aa epoch 194495) v3 2014-08-13 18:06:07.228882 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.228892 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.228927 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.321 epoch 194495 2014-08-13 18:06:07.228931 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21abc00 2014-08-13 18:06:07.228918 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.228938 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.201340 2 0.000197 2014-08-13 18:06:07.228959 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.228899 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.228992 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.201404 0 0.000000 2014-08-13 18:06:07.228999 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.229015 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.229029 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.229056 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.229044 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 321 2014-08-13 18:06:07.229070 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.229100 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229088 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.229103 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229019 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.201440 0 0.000000 2014-08-13 18:06:07.229125 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229127 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229129 7f5c835ef700 20 osd.4 194495 _dispatch 0x21abde0 pg_query(22.791 epoch 194495) v3 2014-08-13 18:06:07.229132 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [373,344]/[373,344,320] -- 373/373 2014-08-13 18:06:07.229155 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.229131 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Started 0.201590 0 0.000000 2014-08-13 18:06:07.229209 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.30 epoch 194495 2014-08-13 18:06:07.229161 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.229213 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21abde0 2014-08-13 18:06:07.229202 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.229214 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.229234 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,344] newacting [373,344,320] 2014-08-13 18:06:07.229250 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.229227 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 30 2014-08-13 18:06:07.229265 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344] r=-1 lpr=194488 pi=193306-194487/31 crt=191727'416 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.229221 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194486 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.229296 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [373,344](373) acting [373,344](373)): not rw, up_thru 194485 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [373,344](373) acting [373,344](373)) : primary up 192978-194485 does not include interval 2014-08-13 18:06:07.229305 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194488 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.229325 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194488 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.229315 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194488/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [373,344](373) acting [373,344](373)) 2014-08-13 18:06:07.229340 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194488 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [312,285] newacting [312,285] 2014-08-13 18:06:07.229365 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229370 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229357 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194488 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.229354 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] up [373,344] -> [373,344], acting [373,344] -> [373,344,320], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> -1 2014-08-13 18:06:07.229373 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [4,312,285] r=0 lpr=194488 pi=193306-194484/24 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.229375 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.229393 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.229413 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229417 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229420 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aa940 pg_query(24.1e6 epoch 194495) v3 2014-08-13 18:06:07.229410 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.229410 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,312,285](4) acting [4,312,285](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.229430 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.229429 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194485/194485/194485) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] noting past interval(194485-194487 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 18:06:07.229448 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.229464 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.285 epoch 194495 2014-08-13 18:06:07.229448 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] up [4,312,285] -> [312,285], acting [4,312,285] -> [312,285], acting_primary 4 -> 312, up_primary 4 -> 312, role 0 -> -1 2014-08-13 18:06:07.229471 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aa940 2014-08-13 18:06:07.229470 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.229470 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.229489 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.229490 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.229487 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(info 0'0) from replica 285 2014-08-13 18:06:07.229507 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.229510 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.229533 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229536 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229522 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.229531 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.229539 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.229557 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229560 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229562 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0e00 pg_notify(22.3af(1) epoch 194495) v5 2014-08-13 18:06:07.229549 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.229557 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.229572 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.229570 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.229589 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.229591 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.229607 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.229611 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0e00 2014-08-13 18:06:07.229605 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.229623 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.229631 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [373,344]/[373,344,320] -- 373/373 2014-08-13 18:06:07.229639 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.229654 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229655 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.229658 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229654 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.229660 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.229679 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229682 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229671 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.229685 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ab2a0 pg_query(24.1de epoch 194495) v3 2014-08-13 18:06:07.229679 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.229687 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.229699 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.229703 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.229727 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.364 epoch 194495 2014-08-13 18:06:07.229731 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ab2a0 2014-08-13 18:06:07.229721 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.229735 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.229741 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [366,320,363]/[373,344,320] -- 366/373 2014-08-13 18:06:07.229762 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.229744 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_query query(info 0'0) from replica 364 2014-08-13 18:06:07.229766 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.229785 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229788 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229782 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.229807 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229810 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229797 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] acting_up_affected newup [366,320,363] newacting [373,344,320] 2014-08-13 18:06:07.229812 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1dc0 pg_notify(22.6bd(28) epoch 194495) v5 2014-08-13 18:06:07.229815 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.229830 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [373,344]/[373,344,320] r=-1 lpr=194489 pi=193306-194488/32 crt=191727'416 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.229851 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.289 2014-08-13 18:06:07.229854 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1dc0 2014-08-13 18:06:07.229861 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [373,344](373) acting [373,344,320](373)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [373,344](373) acting [373,344,320](373) maybe_went_rw) : primary up 192978-194489 includes interval 2014-08-13 18:06:07.229889 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229893 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229886 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194488/194489/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [373,344](373) acting [373,344,320](373) maybe_went_rw) 2014-08-13 18:06:07.229910 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229917 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229919 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df47c0 pg_notify(22.1b8(2) epoch 194495) v5 2014-08-13 18:06:07.229904 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] up [373,344] -> [366,320,363], acting [373,344,320] -> [373,344,320], acting_primary 373 -> 373, up_primary 373 -> 366, role -1 -> -1 2014-08-13 18:06:07.229924 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.229941 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.229956 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.340 2014-08-13 18:06:07.229959 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df47c0 2014-08-13 18:06:07.229957 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.229974 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.229994 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.229996 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.229990 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.230014 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230017 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230005 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.230019 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df01c0 pg_notify(22.3e4(36) epoch 194495) v5 2014-08-13 18:06:07.230021 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.230038 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.230056 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.45 2014-08-13 18:06:07.230060 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df01c0 2014-08-13 18:06:07.230055 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.230072 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.230097 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230100 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230088 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.230118 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230121 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230105 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230122 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a9c20 pg_query(24.2a1 epoch 194495) v3 2014-08-13 18:06:07.230159 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.17 epoch 194495 2014-08-13 18:06:07.230162 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a9c20 2014-08-13 18:06:07.230149 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] handle_advance_map [366,320,363]/[366,320,363] -- 366/366 2014-08-13 18:06:07.230169 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230173 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230174 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(info 0'0) from replica 17 2014-08-13 18:06:07.230189 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.230220 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230223 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230215 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] acting_up_affected newup [366,320,363] newacting [366,320,363] 2014-08-13 18:06:07.230242 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230245 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230233 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.230247 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5b00 pg_notify(22.619(1) epoch 194495) v5 2014-08-13 18:06:07.230248 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363]/[373,344,320] r=-1 lpr=194491 pi=193306-194490/33 crt=191727'416 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.229754 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230287 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.44 2014-08-13 18:06:07.230291 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5b00 2014-08-13 18:06:07.230279 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [366,320,363](366) acting [373,344,320](373)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [366,320,363](366) acting [373,344,320](373)) : primary up 192978-194489 does not include interval 2014-08-13 18:06:07.230297 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194491/194488) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 18:06:07.230326 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230330 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230315 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] up [366,320,363] -> [366,320,363], acting [373,344,320] -> [366,320,363], acting_primary 373 -> 366, up_primary 366 -> 366, role -1 -> -1 2014-08-13 18:06:07.230309 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 18:06:07.230337 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230335 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.230348 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230351 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230354 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aa3a0 pg_query(22.2d1 epoch 194495) v3 2014-08-13 18:06:07.230342 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230351 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.230359 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.230368 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.230377 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230396 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.358 epoch 194495 2014-08-13 18:06:07.230386 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.230400 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aa3a0 2014-08-13 18:06:07.230402 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.230409 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,285]/[312,285] -- 312/312 2014-08-13 18:06:07.230428 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230418 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.230412 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 358 2014-08-13 18:06:07.230432 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230434 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.230458 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230461 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230450 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.230453 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.230466 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230481 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230470 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.230484 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230490 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3f00 pg_notify(24.b6(43) epoch 194495) v5 2014-08-13 18:06:07.230489 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] [373,344,320] -> [366,320,363], acting primary 373 -> 366 2014-08-13 18:06:07.230504 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 18:06:07.230523 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230528 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.315 2014-08-13 18:06:07.230516 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.230532 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3f00 2014-08-13 18:06:07.230527 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230533 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.230544 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.230552 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230571 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230575 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230562 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] acting_up_affected newup [312,311,369] newacting [312,311,369] 2014-08-13 18:06:07.230579 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.230593 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230596 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230599 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ab480 pg_query(24.3bf epoch 194495) v3 2014-08-13 18:06:07.230594 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,285] r=-1 lpr=194488 pi=193306-194487/25 crt=191648'343 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.230594 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [366,320,363]/[366,320,363] -- 366/366 2014-08-13 18:06:07.230616 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230621 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230640 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.19 epoch 194495 2014-08-13 18:06:07.230626 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [312,285](312) acting [312,285](312)): not rw, up_thru 194488 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194488-194490 up [312,285](312) acting [312,285](312) maybe_went_rw) : primary up 192974-194488 includes interval 2014-08-13 18:06:07.230644 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ab480 2014-08-13 18:06:07.230638 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.230646 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194488/194488/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 18:06:07.230656 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230657 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 19 2014-08-13 18:06:07.230666 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] up [312,285] -> [312,311,369], acting [312,285] -> [312,311,369], acting_primary 312 -> 312, up_primary 312 -> 312, role -1 -> -1 2014-08-13 18:06:07.230686 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.230700 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230707 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230696 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [366,320,363]/[366,320,363] -- 366/366 2014-08-13 18:06:07.230718 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230702 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.230728 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230731 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230733 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4ec0 pg_notify(24.136(35) epoch 194495) v5 2014-08-13 18:06:07.230722 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230724 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.230738 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.230742 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.230755 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230758 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.230777 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.356 2014-08-13 18:06:07.230780 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4ec0 2014-08-13 18:06:07.230774 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.230787 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 18:06:07.230793 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.230807 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230819 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230823 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230811 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.230811 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230841 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230830 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.230845 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230830 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.230851 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0700 pg_notify(22.113(34) epoch 194495) v5 2014-08-13 18:06:07.230848 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.230852 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,373,344] newacting [4,373,344] 2014-08-13 18:06:07.230864 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.230875 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.230891 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.55 2014-08-13 18:06:07.230895 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0700 2014-08-13 18:06:07.230882 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230893 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [366,320,363] r=-1 lpr=194492 pi=193306-194491/34 crt=191727'416 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.230931 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230935 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230925 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 18:06:07.230925 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [366,320,363](366) acting [366,320,363](366)): not rw, up_thru 194492 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) : primary up 192968-194492 includes interval 2014-08-13 18:06:07.230947 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.230953 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.230956 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.230958 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df24c0 pg_notify(24.136(1) epoch 194495) v5 2014-08-13 18:06:07.230945 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194491/194492/194492) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 18:06:07.230951 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.230968 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.230965 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [366,320,363] -> [4,373,344], acting [366,320,363] -> [4,373,344], acting_primary 366 -> 4, up_primary 366 -> 4, role -1 -> 0 2014-08-13 18:06:07.230985 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.230987 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.231006 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.50 2014-08-13 18:06:07.231009 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df24c0 2014-08-13 18:06:07.231008 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.231025 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.231024 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 18:06:07.231045 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.231045 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.231050 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.231045 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2614 2014-08-13 18:06:07.231049 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.231068 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.231071 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.231074 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2a00 pg_notify(22.99(30) epoch 194495) v5 2014-08-13 18:06:07.231065 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.231064 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.231082 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.231083 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.231101 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.231121 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.337 2014-08-13 18:06:07.231125 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2a00 2014-08-13 18:06:07.231119 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.231122 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [312,311,369]/[312,311,369] -- 312/312 2014-08-13 18:06:07.231142 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.231136 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.231146 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.231163 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.231152 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.231168 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.231170 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.231163 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.231193 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.231196 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.231199 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aa1c0 pg_query(22.653 epoch 194495) v3 2014-08-13 18:06:07.231190 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.231188 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.231214 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.231221 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 18:06:07.231238 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.371 epoch 194495 2014-08-13 18:06:07.231241 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aa1c0 2014-08-13 18:06:07.231242 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.231232 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.231247 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.231249 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.231254 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 371 2014-08-13 18:06:07.231264 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.245783 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.231267 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.245752 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,312,285] newacting [4,312,285] 2014-08-13 18:06:07.245787 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.245802 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.231249 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.245822 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [312,311,369] r=-1 lpr=194491 pi=193306-194490/26 crt=191648'343 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.245836 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.245802 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.245841 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.245844 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4980 pg_notify(22.6bd(2) epoch 194495) v5 2014-08-13 18:06:07.245843 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.245862 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.245861 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [312,311,369](312) acting [312,311,369](312)): not rw, up_thru 194491 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194491-194494 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) : primary up 192974-194491 includes interval 2014-08-13 18:06:07.245881 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.245895 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.32 2014-08-13 18:06:07.245899 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4980 2014-08-13 18:06:07.245885 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194491/194491/194488) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 18:06:07.245901 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.245905 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [312,311,369] -> [4,312,285], acting [312,311,369] -> [4,312,285], acting_primary 312 -> 4, up_primary 312 -> 4, role -1 -> 0 2014-08-13 18:06:07.245924 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.245927 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.245950 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.245954 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.245944 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.245946 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,344,373 unchanged 2014-08-13 18:06:07.245961 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.245974 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.245964 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.245978 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.245980 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1a40 pg_notify(22.3d5(2) epoch 194495) v5 2014-08-13 18:06:07.245980 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2861 2014-08-13 18:06:07.245982 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.017932 8 0.017940 2014-08-13 18:06:07.245998 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.246005 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.246021 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.343 2014-08-13 18:06:07.246025 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1a40 2014-08-13 18:06:07.246018 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.246023 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.246036 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.246041 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.246053 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.246062 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 2014-08-13 18:06:07.246079 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246069 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.246083 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246082 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.246084 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.246107 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246110 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246099 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.246112 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0000 pg_notify(22.707(38) epoch 194495) v5 2014-08-13 18:06:07.246101 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.246118 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.246118 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.246139 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.246154 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.15 2014-08-13 18:06:07.246158 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0000 2014-08-13 18:06:07.246140 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.246160 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 18:06:07.246171 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.246190 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.246211 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246215 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246206 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.246184 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 18:06:07.246221 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.246234 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246237 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246240 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a8f00 pg_query(24.1e4 epoch 194495) v3 2014-08-13 18:06:07.246227 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [373,344](373) acting [373,344,320](373) maybe_went_rw) 2014-08-13 18:06:07.246236 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.246252 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.246250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [373,344](373) acting [373,344](373)) 2014-08-13 18:06:07.246268 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.246281 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.31 epoch 194495 2014-08-13 18:06:07.246285 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a8f00 2014-08-13 18:06:07.246270 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,373,344](4) acting [4,373,344](4) maybe_went_rw) 2014-08-13 18:06:07.246287 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.246291 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 18:06:07.246298 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(info 0'0) from replica 31 2014-08-13 18:06:07.246308 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.246311 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 18:06:07.246330 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,320,344,363,366,373 down blocked_by {} 2014-08-13 18:06:07.246333 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,285,312 unchanged 2014-08-13 18:06:07.246349 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246353 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.246350 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.246372 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246376 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246378 7f5c835ef700 20 osd.4 194495 _dispatch 0x21abfc0 pg_query(22.12a epoch 194495) v3 2014-08-13 18:06:07.246369 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.017168 8 0.017232 2014-08-13 18:06:07.246372 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2615 2014-08-13 18:06:07.246391 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.246392 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.320 2014-08-13 18:06:07.246416 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.323 epoch 194495 2014-08-13 18:06:07.246420 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21abfc0 2014-08-13 18:06:07.246410 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.246413 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.344 2014-08-13 18:06:07.246426 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.246432 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 18:06:07.246433 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 323 2014-08-13 18:06:07.246447 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000036 0 0.000000 2014-08-13 18:06:07.246450 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.366 2014-08-13 18:06:07.246470 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.246471 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 18:06:07.246498 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246502 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246495 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.246504 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.246522 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246525 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246513 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.246528 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf40c0 pg_notify(22.362(34) epoch 194495) v5 2014-08-13 18:06:07.246525 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.246530 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.246549 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 18:06:07.246571 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.362 2014-08-13 18:06:07.246575 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf40c0 2014-08-13 18:06:07.246571 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 18:06:07.246595 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,312,285](4) acting [4,312,285](4) maybe_went_rw) 2014-08-13 18:06:07.246612 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246616 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246615 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 18:06:07.246635 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246638 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246640 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.246640 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5080 pg_notify(22.481(49) epoch 194495) v5 2014-08-13 18:06:07.246634 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 18:06:07.246652 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,311,312,369 down blocked_by {} 2014-08-13 18:06:07.246680 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.287 2014-08-13 18:06:07.246670 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.246684 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5080 2014-08-13 18:06:07.246690 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2862 2014-08-13 18:06:07.246699 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 18:06:07.246709 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.285 2014-08-13 18:06:07.246725 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246728 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246727 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.246729 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.311 2014-08-13 18:06:07.246747 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246735 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.246751 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246753 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4ec0 pg_notify(22.84(28) epoch 194495) v5 2014-08-13 18:06:07.246746 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.312 2014-08-13 18:06:07.246753 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364] newacting [328,364] 2014-08-13 18:06:07.246766 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.246770 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.246793 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.304 2014-08-13 18:06:07.246797 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4ec0 2014-08-13 18:06:07.246785 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.246789 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.176384 3 0.000132 2014-08-13 18:06:07.246804 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.246810 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started 0.176447 0 0.000000 2014-08-13 18:06:07.246831 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.246844 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246848 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246846 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.246867 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246869 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246871 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a9a40 pg_query(22.3ad epoch 194495) v3 2014-08-13 18:06:07.246863 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.246879 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.246902 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.246894 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364] newacting [328,364] 2014-08-13 18:06:07.246914 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.292 epoch 194495 2014-08-13 18:06:07.246917 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a9a40 2014-08-13 18:06:07.246911 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.246927 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.246930 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 292 2014-08-13 18:06:07.246969 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246972 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.246959 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [328,364,4](328) acting [328,364,4](328)): not rw, up_thru 194485 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [328,364,4](328) acting [328,364,4](328) maybe_went_rw) : primary up 192949-194485 includes interval 2014-08-13 18:06:07.246961 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 18:06:07.246992 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.246990 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.246977 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [328,364,4](328) acting [328,364,4](328) maybe_went_rw) 2014-08-13 18:06:07.246994 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247000 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4280 pg_notify(24.1bb(1) epoch 194495) v5 2014-08-13 18:06:07.246998 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.246997 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364,4] -> [328,364], acting [328,364,4] -> [328,364], acting_primary 328 -> 328, up_primary 328 -> 328, role 2 -> -1 2014-08-13 18:06:07.247018 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247017 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,374] newacting [345,374] 2014-08-13 18:06:07.247041 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.332 2014-08-13 18:06:07.247044 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4280 2014-08-13 18:06:07.247036 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.247036 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.247058 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.247062 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.220586 3 0.000122 2014-08-13 18:06:07.247086 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247076 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.247089 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247083 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started 0.220649 0 0.000000 2014-08-13 18:06:07.247091 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.247108 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247112 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247114 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5e80 pg_notify(22.99(1) epoch 194495) v5 2014-08-13 18:06:07.247103 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.247107 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.247120 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194486 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.247124 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247155 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.363 2014-08-13 18:06:07.247145 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.247159 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5e80 2014-08-13 18:06:07.247138 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194488 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247164 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.247164 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194488 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.247184 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.247183 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194488 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,374] newacting [345,374] 2014-08-13 18:06:07.247214 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.247221 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194488 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.247240 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247230 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.247246 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247239 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,4,374] r=1 lpr=194488 pi=192981-194484/25 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.247246 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.247264 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247280 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247284 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247286 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4440 pg_notify(22.3d5(28) epoch 194495) v5 2014-08-13 18:06:07.247273 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [345,4,374](345) acting [345,4,374](345)): not rw, up_thru 194485 up_from 192981 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [345,4,374](345) acting [345,4,374](345) maybe_went_rw) : primary up 192981-194485 includes interval 2014-08-13 18:06:07.247291 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194485/194485/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [345,4,374](345) acting [345,4,374](345) maybe_went_rw) 2014-08-13 18:06:07.247302 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 18:06:07.247324 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247327 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.286 2014-08-13 18:06:07.247310 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,4,374] -> [345,374], acting [345,4,374] -> [345,374], acting_primary 345 -> 345, up_primary 345 -> 345, role 1 -> -1 2014-08-13 18:06:07.247330 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4440 2014-08-13 18:06:07.247331 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247329 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247350 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.247350 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.247373 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247377 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247367 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.247369 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247384 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.247398 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247402 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247405 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ab840 pg_query(24.2ec epoch 194495) v3 2014-08-13 18:06:07.247401 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.247399 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364]/[328,364] -- 328/328 2014-08-13 18:06:07.247421 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247419 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.247425 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247444 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.374 epoch 194495 2014-08-13 18:06:07.247448 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ab840 2014-08-13 18:06:07.247435 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247442 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.247455 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.247459 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247461 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_query query(info 0'0) from replica 374 2014-08-13 18:06:07.247472 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.247488 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.247494 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 18:06:07.247513 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247502 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.247515 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247522 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247517 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.247517 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247543 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247533 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.247547 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247535 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.247550 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ab0c0 pg_query(22.537 epoch 194495) v3 2014-08-13 18:06:07.247550 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247551 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364,329] newacting [328,364,329] 2014-08-13 18:06:07.247570 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.247589 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.51 epoch 194495 2014-08-13 18:06:07.247593 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ab0c0 2014-08-13 18:06:07.247585 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.247591 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 18:06:07.247613 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247606 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 51 2014-08-13 18:06:07.247616 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [328,364](328) acting [328,364](328)): not rw, up_thru 194488 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [328,364](328) acting [328,364](328) maybe_went_rw) : primary up 192949-194488 includes interval 2014-08-13 18:06:07.247617 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247636 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.247635 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [328,364](328) acting [328,364](328) maybe_went_rw) 2014-08-13 18:06:07.247660 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247664 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247654 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247656 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364] -> [328,364,329], acting [328,364] -> [328,364,329], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> -1 2014-08-13 18:06:07.247686 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247689 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247679 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247692 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5b00 pg_notify(22.1b8(28) epoch 194495) v5 2014-08-13 18:06:07.247685 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,374]/[345,374] -- 345/345 2014-08-13 18:06:07.247705 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247695 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.247710 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247711 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.247734 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.347 2014-08-13 18:06:07.247737 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5b00 2014-08-13 18:06:07.247728 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.247730 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.247744 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247745 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.247762 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.247777 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247781 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247777 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247802 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247783 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 18:06:07.247805 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247793 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.247808 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df40c0 pg_notify(22.113(34) epoch 194495) v5 2014-08-13 18:06:07.247807 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247814 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.247816 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247830 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.247833 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.247848 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.24 2014-08-13 18:06:07.247852 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df40c0 2014-08-13 18:06:07.247846 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.247849 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,378,304] newacting [345,378,304] 2014-08-13 18:06:07.247867 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.247865 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247893 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247897 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247886 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,374] r=-1 lpr=194488 pi=192981-194487/26 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.247916 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.247919 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.247922 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aad00 pg_query(24.cd epoch 194495) v3 2014-08-13 18:06:07.247907 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 18:06:07.247928 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.247916 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [345,374](345) acting [345,374](345)): not rw, up_thru 194488 up_from 192981 last_epoch_clean 194483 generate_past_intervals interval(194488-194490 up [345,374](345) acting [345,374](345) maybe_went_rw) : primary up 192981-194488 includes interval 2014-08-13 18:06:07.247933 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.247935 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194488/194488/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [345,374](345) acting [345,374](345) maybe_went_rw) 2014-08-13 18:06:07.247964 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.6 epoch 194495 2014-08-13 18:06:07.247950 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.247967 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aad00 2014-08-13 18:06:07.247957 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,374] -> [345,378,304], acting [345,374] -> [345,378,304], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> -1 2014-08-13 18:06:07.247971 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.247978 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.247983 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(info 0'0) from replica 6 2014-08-13 18:06:07.247993 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.248009 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.248005 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 18:06:07.248027 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248029 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248032 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248026 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.248031 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248052 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248055 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248042 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.248057 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0fc0 pg_notify(22.707(1) epoch 194495) v5 2014-08-13 18:06:07.248049 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.248059 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.248066 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248076 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.248098 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.248101 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0fc0 2014-08-13 18:06:07.248093 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.248101 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,329]/[328,364,329] -- 328/328 2014-08-13 18:06:07.248122 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248110 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.248126 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.248126 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248144 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.248155 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248159 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248143 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.248159 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248180 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248175 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248183 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248193 7f5c835ef700 20 osd.4 194495 _dispatch 0x5155a00 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.248203 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.248209 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.248211 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248190 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 18:06:07.248213 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248216 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248216 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 18:06:07.248221 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248237 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248241 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248245 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248247 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ac740 pg_query(22.220 epoch 194495) v3 2014-08-13 18:06:07.248236 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.248242 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248253 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,364,4] newacting [328,364,4] 2014-08-13 18:06:07.248261 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.248272 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.248288 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.351 epoch 194495 2014-08-13 18:06:07.248292 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ac740 2014-08-13 18:06:07.248279 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248290 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,329] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'472 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.248305 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 351 2014-08-13 18:06:07.248319 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 18:06:07.248321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [328,364,329](328) acting [328,364,329](328)): not rw, up_thru 194491 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [328,364,329](328) acting [328,364,329](328) maybe_went_rw) : primary up 192949-194491 includes interval 2014-08-13 18:06:07.248339 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248343 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248344 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [328,364,329](328) acting [328,364,329](328) maybe_went_rw) 2014-08-13 18:06:07.248367 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248358 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.248371 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248363 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] up [328,364,329] -> [328,364,4], acting [328,364,329] -> [328,364,4], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> 2 2014-08-13 18:06:07.248375 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248394 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248383 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.248397 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248400 7f5c835ef700 20 osd.4 194495 _dispatch 0x524c440 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.248400 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.248415 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.248412 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,378,304]/[345,378,304] -- 345/345 2014-08-13 18:06:07.248433 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248441 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.338 2014-08-13 18:06:07.248444 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524c440 2014-08-13 18:06:07.248431 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.248438 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248449 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.248456 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.248465 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.248473 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248489 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248492 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248482 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.248499 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.248504 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 18:06:07.248523 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.248527 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248515 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.248531 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248533 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf55c0 pg_notify(22.1d2(1) epoch 194495) v5 2014-08-13 18:06:07.248527 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.248531 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.248543 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.248548 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.248559 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,4,374] newacting [345,4,374] 2014-08-13 18:06:07.248576 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.331 2014-08-13 18:06:07.248565 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.248580 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf55c0 2014-08-13 18:06:07.248579 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.248581 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.248597 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,378,304] r=-1 lpr=194491 pi=192981-194490/27 crt=192888'464 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.248600 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248624 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248628 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248618 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.248626 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [345,378,304](345) acting [345,378,304](345)): not rw, up_thru 194491 up_from 192981 last_epoch_clean 194483 generate_past_intervals interval(194491-194494 up [345,378,304](345) acting [345,378,304](345) maybe_went_rw) : primary up 192981-194491 includes interval 2014-08-13 18:06:07.248646 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248649 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248651 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a90e0 pg_query(22.581 epoch 194495) v3 2014-08-13 18:06:07.248644 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194491/194491/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [345,378,304](345) acting [345,378,304](345) maybe_went_rw) 2014-08-13 18:06:07.248663 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] up [345,378,304] -> [345,4,374], acting [345,378,304] -> [345,4,374], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> 1 2014-08-13 18:06:07.248687 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.320 epoch 194495 2014-08-13 18:06:07.248691 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a90e0 2014-08-13 18:06:07.248682 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.248685 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.248697 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.248704 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.248704 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 320 2014-08-13 18:06:07.248714 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.248731 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.248724 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Reset 0.001893 8 0.001581 2014-08-13 18:06:07.248747 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248751 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248748 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.248752 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.248771 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248775 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248764 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.248777 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ac920 pg_query(22.69f epoch 194495) v3 2014-08-13 18:06:07.248770 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.248780 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.248788 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.248799 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.248816 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.334 epoch 194495 2014-08-13 18:06:07.248819 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ac920 2014-08-13 18:06:07.248808 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 2014-08-13 18:06:07.248817 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.248829 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.248833 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.248832 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 334 2014-08-13 18:06:07.248849 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.248851 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.248874 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248864 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.248878 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248880 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.248871 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.248899 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.248902 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.248904 7f5c835ef700 20 osd.4 194495 _dispatch 0x21acce0 pg_query(22.639 epoch 194495) v3 2014-08-13 18:06:07.248896 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.248915 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.248940 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.367 epoch 194495 2014-08-13 18:06:07.248943 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21acce0 2014-08-13 18:06:07.248973 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.248954 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 367 2014-08-13 18:06:07.248974 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.248990 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.249006 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Reset 0.001902 8 0.001594 2014-08-13 18:06:07.249029 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249033 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249025 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.249041 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.249056 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249059 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249062 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dea680 pg_notify(24.136(35) epoch 194495) v5 2014-08-13 18:06:07.249057 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.249048 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249089 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249077 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 2014-08-13 18:06:07.249104 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.65 2014-08-13 18:06:07.249107 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dea680 2014-08-13 18:06:07.249101 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.249099 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249121 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249123 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.249152 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249156 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249141 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.249163 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249186 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249183 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249197 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249200 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deaa00 pg_notify(24.3d(37) epoch 194495) v5 2014-08-13 18:06:07.249191 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249237 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.373 2014-08-13 18:06:07.249227 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249241 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deaa00 2014-08-13 18:06:07.249248 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.249269 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249287 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249290 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249293 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249298 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249322 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249297 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_advance_map [25,362]/[20,303] -- 25/20 2014-08-13 18:06:07.249326 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249315 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249328 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a440c0 pg_notify(22.3d9(21) epoch 194495) v5 2014-08-13 18:06:07.249336 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249342 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Reset advmap 2014-08-13 18:06:07.249366 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.397 2014-08-13 18:06:07.249369 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a440c0 2014-08-13 18:06:07.249354 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249361 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.249378 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249378 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [25,362] newacting [20,303] 2014-08-13 18:06:07.249383 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249397 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.249402 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249415 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249419 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249417 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [4,25,362]/[4,20,303] r=0 lpr=194487 pi=193635-194485/33 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194488 2014-08-13 18:06:07.249438 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249441 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249444 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0540 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.249440 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249463 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249455 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,25,362](4) acting [4,20,303](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.249483 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.396 2014-08-13 18:06:07.249468 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249487 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0540 2014-08-13 18:06:07.249474 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194485/194486/194485) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] noting past interval(194486-194487 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 18:06:07.249488 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249495 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] up [4,25,362] -> [25,362], acting [4,20,303] -> [20,303], acting_primary 4 -> 20, up_primary 4 -> 25, role 0 -> -1 2014-08-13 18:06:07.249516 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] clear_primary_state 2014-08-13 18:06:07.249537 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249540 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249527 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249535 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] agent_stop 2014-08-13 18:06:07.249551 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249560 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249563 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249552 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] on_change 2014-08-13 18:06:07.249565 7f5c835ef700 20 osd.4 194495 _dispatch 0x4def000 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.249556 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249569 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.249575 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249587 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:07.249606 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.249609 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4def000 2014-08-13 18:06:07.249606 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] cancel_copy_ops 2014-08-13 18:06:07.249614 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249623 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] cancel_flush_ops 2014-08-13 18:06:07.249637 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249646 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249649 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249641 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] on_change_cleanup 2014-08-13 18:06:07.249642 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249657 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] on_change 2014-08-13 18:06:07.249660 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249682 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249685 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249687 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aba20 pg_query(22.630 epoch 194495) v3 2014-08-13 18:06:07.249675 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] exit NotTrimming 2014-08-13 18:06:07.249697 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:07.249692 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped m=57] enter NotTrimming 2014-08-13 18:06:07.249722 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249728 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.294 epoch 194495 2014-08-13 18:06:07.249732 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aba20 2014-08-13 18:06:07.249724 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_stats 2014-08-13 18:06:07.249728 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:07.249746 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249746 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.249748 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 294 2014-08-13 18:06:07.249766 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_role_change 2014-08-13 18:06:07.249766 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:07.249783 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.249802 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249805 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249799 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 18:06:07.249814 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 18:06:07.249827 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249830 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249832 7f5c835ef700 20 osd.4 194495 _dispatch 0x5249f80 pg_notify(24.1bb(39) epoch 194495) v5 2014-08-13 18:06:07.249831 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.249850 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:07.249871 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.379 2014-08-13 18:06:07.249875 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5249f80 2014-08-13 18:06:07.249869 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [25,362]/[20,303,373] -- 25/20 2014-08-13 18:06:07.249871 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194495 2014-08-13 18:06:07.249896 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249890 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.249901 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.249917 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.249919 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249923 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249921 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.249938 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [25,362] newacting [20,303,373] 2014-08-13 18:06:07.249954 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.249957 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.249960 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1500 pg_notify(22.22c(30) epoch 194495) v5 2014-08-13 18:06:07.249946 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,288]/[330,57] -- 330/330 2014-08-13 18:06:07.249955 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.249979 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.249974 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303] r=-1 lpr=194488 pi=193635-194487/34 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194489 2014-08-13 18:06:07.250000 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.324 2014-08-13 18:06:07.249986 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.250003 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1500 2014-08-13 18:06:07.250005 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.250006 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [25,362](25) acting [20,303](20)): not rw, up_thru 194483 up_from 192971 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [25,362](25) acting [20,303](20)) : primary up 192971-194483 does not include interval 2014-08-13 18:06:07.250024 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,288] newacting [330,57] 2014-08-13 18:06:07.250027 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194488/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] noting past interval(194488-194488 up [25,362](25) acting [20,303](20)) 2014-08-13 18:06:07.250042 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.250055 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250060 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250046 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] up [25,362] -> [25,362], acting [20,303] -> [20,303,373], acting_primary 20 -> 20, up_primary 25 -> 25, role -1 -> -1 2014-08-13 18:06:07.250059 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,4,288]/[330,4,57] r=1 lpr=194487 pi=193227-194485/39 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194488 2014-08-13 18:06:07.250079 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250067 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 18:06:07.250082 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250085 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf16c0 pg_notify(22.775(36) epoch 194495) v5 2014-08-13 18:06:07.250084 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.250100 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.250096 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [330,4,288](330) acting [330,4,57](330)): not rw, up_thru 194486 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [330,4,288](330) acting [330,4,57](330) maybe_went_rw) : primary up 192965-194486 includes interval 2014-08-13 18:06:07.250129 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.0 2014-08-13 18:06:07.250118 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_copy_ops 2014-08-13 18:06:07.250132 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf16c0 2014-08-13 18:06:07.250119 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194485/194486/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194486-194487 up [330,4,288](330) acting [330,4,57](330) maybe_went_rw) 2014-08-13 18:06:07.250140 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,4,288] -> [330,288], acting [330,4,57] -> [330,57], acting_primary 330 -> 330, up_primary 330 -> 330, role 1 -> -1 2014-08-13 18:06:07.250137 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_flush_ops 2014-08-13 18:06:07.250162 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change_cleanup 2014-08-13 18:06:07.250161 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.250178 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 18:06:07.250196 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] exit NotTrimming 2014-08-13 18:06:07.250193 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.250217 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250224 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250213 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] enter NotTrimming 2014-08-13 18:06:07.250220 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.250231 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 18:06:07.250248 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250252 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250240 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:07.250255 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1c00 pg_notify(22.1f1(1) epoch 194495) v5 2014-08-13 18:06:07.250248 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 18:06:07.250258 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:07.250267 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.250276 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:07.250300 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.11 2014-08-13 18:06:07.250303 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1c00 2014-08-13 18:06:07.250293 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.250303 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [25,362]/[20,303,373] -- 25/20 2014-08-13 18:06:07.250314 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:07.250330 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.250332 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:07.250335 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.250350 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250354 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250349 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_role_change 2014-08-13 18:06:07.250353 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.250373 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250377 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250366 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.250379 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5780 pg_notify(22.4df(1) epoch 194495) v5 2014-08-13 18:06:07.250372 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.250383 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 18:06:07.250401 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:07.250418 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.352 2014-08-13 18:06:07.250421 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5780 2014-08-13 18:06:07.250413 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [20,373,303]/[20,303,373] -- 20/20 2014-08-13 18:06:07.250419 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.250438 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.250442 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.250466 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250469 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250459 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.250457 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,288]/[330,57,304] -- 330/330 2014-08-13 18:06:07.250485 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.250490 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.250476 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [20,373,303] newacting [20,303,373] 2014-08-13 18:06:07.250493 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.250495 7f5c835ef700 20 osd.4 194495 _dispatch 0x4de9dc0 pg_notify(24.b6(1) epoch 194495) v5 2014-08-13 18:06:07.250490 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.250494 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.250509 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.250511 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [25,362]/[20,303,373] r=-1 lpr=194489 pi=193635-194488/35 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194491 2014-08-13 18:06:07.261107 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,288] newacting [330,57,304] 2014-08-13 18:06:07.261147 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.261163 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.327 2014-08-13 18:06:07.261177 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4de9dc0 2014-08-13 18:06:07.261166 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57] r=-1 lpr=194488 pi=193227-194487/40 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194489 2014-08-13 18:06:07.261146 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [25,362](25) acting [20,303,373](20)): not rw, up_thru 194489 up_from 192971 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [25,362](25) acting [20,303,373](20) maybe_went_rw) : primary up 192971-194489 includes interval 2014-08-13 18:06:07.261185 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194488/194489/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] noting past interval(194489-194490 up [25,362](25) acting [20,303,373](20) maybe_went_rw) 2014-08-13 18:06:07.261206 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] up [25,362] -> [20,373,303], acting [20,303,373] -> [20,303,373], acting_primary 20 -> 20, up_primary 25 -> 20, role -1 -> -1 2014-08-13 18:06:07.261228 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 18:06:07.261225 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [330,288](330) acting [330,57](330)): not rw, up_thru 194486 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [330,288](330) acting [330,57](330)) : primary up 192965-194486 does not include interval 2014-08-13 18:06:07.261247 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.261261 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261247 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194488/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194488-194488 up [330,288](330) acting [330,57](330)) 2014-08-13 18:06:07.261265 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261265 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.261269 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,288] -> [330,288], acting [330,57] -> [330,57,304], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 18:06:07.261301 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261304 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261291 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.261285 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_copy_ops 2014-08-13 18:06:07.261307 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5cc0 pg_notify(22.3af(36) epoch 194495) v5 2014-08-13 18:06:07.261310 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_flush_ops 2014-08-13 18:06:07.261310 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.261334 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.261333 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change_cleanup 2014-08-13 18:06:07.261357 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.339 2014-08-13 18:06:07.261361 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5cc0 2014-08-13 18:06:07.261356 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] on_change 2014-08-13 18:06:07.261357 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:07.261376 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:07.261377 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] exit NotTrimming 2014-08-13 18:06:07.261406 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261395 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:07.261410 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261398 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] enter NotTrimming 2014-08-13 18:06:07.261412 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.261430 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261416 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] cancel_recovery 2014-08-13 18:06:07.261433 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261436 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3480 pg_notify(2.2a(2) epoch 194495) v5 2014-08-13 18:06:07.261431 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:07.261434 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] clear_recovery_state 2014-08-13 18:06:07.261452 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:07.261454 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.261479 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.370 2014-08-13 18:06:07.261482 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3480 2014-08-13 18:06:07.261470 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 18:06:07.261486 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:07.261506 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.261508 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] handle_advance_map [20,373,303]/[20,373,303] -- 20/20 2014-08-13 18:06:07.261528 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261532 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261540 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.261550 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261552 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261555 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4440 pg_notify(22.84(2) epoch 194495) v5 2014-08-13 18:06:07.261547 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.261550 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,288]/[330,57,304] -- 330/330 2014-08-13 18:06:07.261567 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.261583 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.261594 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.7 2014-08-13 18:06:07.261597 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4440 2014-08-13 18:06:07.261585 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] acting_up_affected newup [20,373,303] newacting [20,373,303] 2014-08-13 18:06:07.261591 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.261604 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.261610 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.261638 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261623 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303]/[20,303,373] r=-1 lpr=194491 pi=193635-194490/36 crt=194422'18731 lcod 0'0 remapped NOTIFY m=57] set_last_peering_reset 194492 2014-08-13 18:06:07.261643 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261630 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.261667 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261670 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261672 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0380 pg_notify(22.481(42) epoch 194495) v5 2014-08-13 18:06:07.261661 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)): not rw, up_thru 194489 up_from 192971 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)) : primary up 192971-194489 does not include interval 2014-08-13 18:06:07.261673 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,304,57]/[330,57,304] -- 330/330 2014-08-13 18:06:07.261680 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194491/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] noting past interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 18:06:07.261699 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.261712 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.309 2014-08-13 18:06:07.261715 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0380 2014-08-13 18:06:07.261700 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] up [20,373,303] -> [20,373,303], acting [20,303,373] -> [20,373,303], acting_primary 20 -> 20, up_primary 20 -> 20, role -1 -> -1 2014-08-13 18:06:07.261704 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.261722 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 18:06:07.261724 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.261740 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.261742 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,304,57] newacting [330,57,304] 2014-08-13 18:06:07.261769 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261755 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.261772 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261760 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.261775 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_copy_ops 2014-08-13 18:06:07.261777 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,288]/[330,57,304] r=-1 lpr=194489 pi=193227-194488/41 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194491 2014-08-13 18:06:07.261795 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261798 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261801 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dea300 pg_notify(22.f3(26) epoch 194495) v5 2014-08-13 18:06:07.261792 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_flush_ops 2014-08-13 18:06:07.261809 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change_cleanup 2014-08-13 18:06:07.261810 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [330,288](330) acting [330,57,304](330)): not rw, up_thru 194489 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [330,288](330) acting [330,57,304](330) maybe_went_rw) : primary up 192965-194489 includes interval 2014-08-13 18:06:07.261827 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 18:06:07.261843 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.311 2014-08-13 18:06:07.261846 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dea300 2014-08-13 18:06:07.261835 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194488/194489/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194489-194490 up [330,288](330) acting [330,57,304](330) maybe_went_rw) 2014-08-13 18:06:07.261845 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] exit NotTrimming 2014-08-13 18:06:07.261857 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,288] -> [330,304,57], acting [330,57,304] -> [330,57,304], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 18:06:07.261864 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] enter NotTrimming 2014-08-13 18:06:07.261890 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261879 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.261894 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261882 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] cancel_recovery 2014-08-13 18:06:07.261898 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] clear_recovery_state 2014-08-13 18:06:07.261897 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.261922 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.261925 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.261927 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0c40 pg_notify(22.3d9(54) epoch 194495) v5 2014-08-13 18:06:07.261915 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.261916 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.261938 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:07.261967 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.293 2014-08-13 18:06:07.261971 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0c40 2014-08-13 18:06:07.261958 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:07.261965 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] handle_advance_map [20,373,303]/[20,373,303] -- 20/20 2014-08-13 18:06:07.261977 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:07.261993 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.261995 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:07.261998 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.262017 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:07.262018 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262036 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262035 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:07.262057 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262064 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262054 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 18:06:07.262084 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262071 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:07.262090 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262076 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] handle_advance_map [20,373,303]/[20,373,303] -- 20/20 2014-08-13 18:06:07.262092 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1f80 pg_notify(22.113(1) epoch 194495) v5 2014-08-13 18:06:07.262100 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262093 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262104 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.262130 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.262134 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1f80 2014-08-13 18:06:07.262122 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262136 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,304,57]/[330,304,57] -- 330/330 2014-08-13 18:06:07.262138 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262162 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262167 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.262184 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262181 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] handle_advance_map [4,25,362]/[4,25,362] -- 4/4 2014-08-13 18:06:07.262198 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262203 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262205 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262224 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262209 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: Reset advmap 2014-08-13 18:06:07.262227 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262208 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] acting_up_affected newup [330,304,57] newacting [330,304,57] 2014-08-13 18:06:07.262229 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5080 pg_notify(22.619(36) epoch 194495) v5 2014-08-13 18:06:07.262228 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262230 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.262246 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] acting_up_affected newup [4,25,362] newacting [4,25,362] 2014-08-13 18:06:07.262249 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57]/[330,57,304] r=-1 lpr=194491 pi=193227-194490/42 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] set_last_peering_reset 194492 2014-08-13 18:06:07.262273 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.314 2014-08-13 18:06:07.262276 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5080 2014-08-13 18:06:07.262264 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.262282 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [20,373,303] r=-1 lpr=194492 pi=193635-194491/37 crt=194422'18731 lcod 0'0 inactive NOTIFY m=57] set_last_peering_reset 194495 2014-08-13 18:06:07.262283 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [330,304,57](330) acting [330,57,304](330)): not rw, up_thru 194489 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [330,304,57](330) acting [330,57,304](330)) : primary up 192965-194489 does not include interval 2014-08-13 18:06:07.262302 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194491/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] noting past interval(194491-194491 up [330,304,57](330) acting [330,57,304](330)) 2014-08-13 18:06:07.262317 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [20,373,303](20) acting [20,373,303](20)): not rw, up_thru 194492 up_from 192971 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) : primary up 192971-194492 includes interval 2014-08-13 18:06:07.262336 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262321 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] up [330,304,57] -> [330,304,57], acting [330,57,304] -> [330,304,57], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> -1 2014-08-13 18:06:07.262341 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262338 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194491/194492/194488) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] noting past interval(194492-194494 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 18:06:07.262342 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 18:06:07.262361 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262364 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262367 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5240 pg_notify(22.f3(1) epoch 194495) v5 2014-08-13 18:06:07.262359 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.262357 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] up [20,373,303] -> [4,25,362], acting [20,373,303] -> [4,25,362], acting_primary 20 -> 4, up_primary 20 -> 4, role -1 -> 0 2014-08-13 18:06:07.262377 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.262382 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] clear_primary_state 2014-08-13 18:06:07.262405 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.312 2014-08-13 18:06:07.262395 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:07.262409 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5240 2014-08-13 18:06:07.262400 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] agent_stop 2014-08-13 18:06:07.262414 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:07.262418 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 18:06:07.262430 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:07.262438 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] publish_stats_to_osd 194495:29706 2014-08-13 18:06:07.262457 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262447 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 18:06:07.262462 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262457 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.262465 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:07.262482 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262485 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262488 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5940 pg_notify(22.619(36) epoch 194495) v5 2014-08-13 18:06:07.262476 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] cancel_copy_ops 2014-08-13 18:06:07.262484 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:07.262494 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] cancel_flush_ops 2014-08-13 18:06:07.262502 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_recovery 2014-08-13 18:06:07.262511 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.262528 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.48 2014-08-13 18:06:07.262532 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5940 2014-08-13 18:06:07.262519 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:07.262529 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.262539 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262547 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] requeue_ops 2014-08-13 18:06:07.262573 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262563 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change_cleanup 2014-08-13 18:06:07.262577 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262579 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] on_change 2014-08-13 18:06:07.262579 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,304,57]/[330,304,57] -- 330/330 2014-08-13 18:06:07.262606 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262606 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262609 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262598 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] exit NotTrimming 2014-08-13 18:06:07.262612 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4b40 pg_notify(22.4df(32) epoch 194495) v5 2014-08-13 18:06:07.262612 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.262616 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive NOTIFY m=57] enter NotTrimming 2014-08-13 18:06:07.262633 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] clear_stats 2014-08-13 18:06:07.262631 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262662 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.380 2014-08-13 18:06:07.262651 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] requeue_ops 2014-08-13 18:06:07.262666 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4b40 2014-08-13 18:06:07.262654 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262667 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] on_role_change 2014-08-13 18:06:07.262685 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] requeue_ops 2014-08-13 18:06:07.262693 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,304,57]/[330,304,57] -- 330/330 2014-08-13 18:06:07.262711 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262715 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262701 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] cancel_recovery 2014-08-13 18:06:07.262717 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262721 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] clear_recovery_state 2014-08-13 18:06:07.262735 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262723 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.262738 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262741 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5cc0 pg_notify(22.22c(28) epoch 194495) v5 2014-08-13 18:06:07.262740 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262740 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262763 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.262764 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_activate_map 2014-08-13 18:06:07.262784 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.368 2014-08-13 18:06:07.262787 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5cc0 2014-08-13 18:06:07.262786 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] update_heartbeat_peers -> 4,25,362 2014-08-13 18:06:07.262806 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.262796 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,288] -- 330/330 2014-08-13 18:06:07.262822 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.262824 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262809 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] take_waiters 2014-08-13 18:06:07.262829 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262827 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:07.262832 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Reset 0.662821 10 0.013090 2014-08-13 18:06:07.262843 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.262863 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262867 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262856 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started 2014-08-13 18:06:07.262869 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6900 pg_notify(24.9b(38) epoch 194495) v5 2014-08-13 18:06:07.262860 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,288] 2014-08-13 18:06:07.262874 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Start 2014-08-13 18:06:07.262878 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.262891 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: transitioning to Primary 2014-08-13 18:06:07.262895 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,304,57] r=-1 lpr=194492 pi=193227-194491/43 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194495 2014-08-13 18:06:07.262910 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.350 2014-08-13 18:06:07.262913 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6900 2014-08-13 18:06:07.262912 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Start 0.000038 0 0.000000 2014-08-13 18:06:07.262927 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [330,304,57](330) acting [330,304,57](330)): not rw, up_thru 194492 up_from 192965 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [330,304,57](330) acting [330,304,57](330) maybe_went_rw) : primary up 192965-194492 includes interval 2014-08-13 18:06:07.262936 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary 2014-08-13 18:06:07.262955 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262959 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262947 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194491/194492/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] noting past interval(194492-194494 up [330,304,57](330) acting [330,304,57](330) maybe_went_rw) 2014-08-13 18:06:07.262954 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering 2014-08-13 18:06:07.262968 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] up [330,304,57] -> [330,4,288], acting [330,304,57] -> [330,4,288], acting_primary 330 -> 330, up_primary 330 -> 330, role -1 -> 1 2014-08-13 18:06:07.262974 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.262991 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.262995 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.262997 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5400 pg_notify(22.481(49) epoch 194495) v5 2014-08-13 18:06:07.262988 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 18:06:07.262991 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.263005 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.263012 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194492-194494 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 18:06:07.263023 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.263038 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.381 2014-08-13 18:06:07.263042 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5400 2014-08-13 18:06:07.263036 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 18:06:07.263041 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:07.263058 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:07.263057 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194489-194490 up [25,362](25) acting [20,303,373](20) maybe_went_rw) 2014-08-13 18:06:07.263082 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263086 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263076 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:07.263079 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194488-194488 up [25,362](25) acting [20,303](20)) 2014-08-13 18:06:07.263092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_change 2014-08-13 18:06:07.263106 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263109 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263111 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6740 pg_notify(22.3af(36) epoch 194495) v5 2014-08-13 18:06:07.263102 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194486-194487 up [4,25,362](4) acting [4,20,303](4) maybe_went_rw) 2014-08-13 18:06:07.263110 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:07.263127 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:07.263124 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194485-194485 up [4,25,362](4) acting [4,25,362](4)) 2014-08-13 18:06:07.263151 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.263154 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6740 2014-08-13 18:06:07.263144 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] on_role_change 2014-08-13 18:06:07.263145 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194483-194484 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 18:06:07.263161 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.263176 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194482-194482 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 18:06:07.263183 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] cancel_recovery 2014-08-13 18:06:07.263207 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263212 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263195 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 18:06:07.263199 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:07.263215 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.263241 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263226 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.263247 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263249 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4d00 pg_notify(24.30b(44) epoch 194495) v5 2014-08-13 18:06:07.263237 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] publish_stats_to_osd 194495:29707 2014-08-13 18:06:07.263250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_activate_map 2014-08-13 18:06:07.263258 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.20 2014-08-13 18:06:07.263289 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.375 2014-08-13 18:06:07.263292 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4d00 2014-08-13 18:06:07.263279 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.25 2014-08-13 18:06:07.263298 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.303 2014-08-13 18:06:07.263316 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.362 2014-08-13 18:06:07.263340 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263344 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263332 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: querying info from osd.373 2014-08-13 18:06:07.263337 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] update_heartbeat_peers unchanged 2014-08-13 18:06:07.263351 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.263367 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263356 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] take_waiters 2014-08-13 18:06:07.263373 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263376 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df55c0 pg_notify(22.22c(1) epoch 194495) v5 2014-08-13 18:06:07.263372 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.263375 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Reset 0.248050 10 0.012925 2014-08-13 18:06:07.263398 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started 2014-08-13 18:06:07.263415 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.302 2014-08-13 18:06:07.263418 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df55c0 2014-08-13 18:06:07.263418 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Start 2014-08-13 18:06:07.263435 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: transitioning to Stray 2014-08-13 18:06:07.263458 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263462 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263455 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Start 0.000037 0 0.000000 2014-08-13 18:06:07.263484 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263486 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263475 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Started/Stray 2014-08-13 18:06:07.263486 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.263492 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ad0a0 pg_query(24.e2 epoch 194495) v3 2014-08-13 18:06:07.263498 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.263527 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.346 epoch 194495 2014-08-13 18:06:07.263516 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.263532 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ad0a0 2014-08-13 18:06:07.263543 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 18:06:07.263544 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_query query(info 0'0) from replica 346 2014-08-13 18:06:07.263573 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.263579 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.263593 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263597 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263596 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379] newacting [321,379] 2014-08-13 18:06:07.263616 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263618 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263621 7f5c835ef700 20 osd.4 194495 _dispatch 0x21acb00 pg_query(24.3f7 epoch 194495) v3 2014-08-13 18:06:07.263612 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.263632 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.263631 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.236801 3 0.000115 2014-08-13 18:06:07.263657 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.330 epoch 194495 2014-08-13 18:06:07.263660 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21acb00 2014-08-13 18:06:07.263652 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started 0.236858 0 0.000000 2014-08-13 18:06:07.263672 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.263672 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_query query(info 0'0) from replica 330 2014-08-13 18:06:07.263688 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194486 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.263688 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_advance_map [353,56]/[353,56] -- 353/353 2014-08-13 18:06:07.263716 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263717 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.263720 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263706 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194488 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.263722 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194488 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.263725 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.263740 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263747 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263738 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194488 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379] newacting [321,379] 2014-08-13 18:06:07.263754 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dead80 pg_notify(24.9b(1) epoch 194495) v5 2014-08-13 18:06:07.263746 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.263755 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194488 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.263763 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.263772 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379,4] r=2 lpr=194488 pi=192971-194484/24 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.263794 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.365 2014-08-13 18:06:07.263797 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dead80 2014-08-13 18:06:07.263784 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.191433 2 0.000399 2014-08-13 18:06:07.263804 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [321,379,4](321) acting [321,379,4](321)): not rw, up_thru 194485 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [321,379,4](321) acting [321,379,4](321) maybe_went_rw) : primary up 192971-194485 includes interval 2014-08-13 18:06:07.263806 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.263835 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263838 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263824 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.191484 0 0.000000 2014-08-13 18:06:07.263822 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [321,379,4](321) acting [321,379,4](321) maybe_went_rw) 2014-08-13 18:06:07.263845 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379,4] -> [321,379], acting [321,379,4] -> [321,379], acting_primary 321 -> 321, up_primary 321 -> 321, role 2 -> -1 2014-08-13 18:06:07.263851 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.191521 0 0.000000 2014-08-13 18:06:07.263866 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263870 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263872 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deabc0 pg_notify(22.39d(1) epoch 194495) v5 2014-08-13 18:06:07.263867 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.263871 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Started 0.191582 0 0.000000 2014-08-13 18:06:07.263884 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.263895 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.263901 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.263916 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.366 2014-08-13 18:06:07.263920 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deabc0 2014-08-13 18:06:07.263915 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194486 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.263917 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.263933 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.263934 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194488 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.263961 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263951 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.263965 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263953 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194488 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.263967 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.263971 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194488 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [353,56] newacting [353,56] 2014-08-13 18:06:07.263991 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.263994 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.263984 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.263997 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deb2c0 pg_notify(22.481(47) epoch 194495) v5 2014-08-13 18:06:07.263989 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194488 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.264001 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.264006 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,353,56] r=0 lpr=194488 pi=193306-194484/32 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.264019 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.264037 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.306 2014-08-13 18:06:07.264040 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deb2c0 2014-08-13 18:06:07.264034 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.264042 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,353,56](4) acting [4,353,56](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.264050 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.264065 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.264062 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] noting past interval(194485-194487 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) 2014-08-13 18:06:07.264081 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.264085 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.264081 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264081 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] up [4,353,56] -> [353,56], acting [4,353,56] -> [353,56], acting_primary 4 -> 353, up_primary 4 -> 353, role 0 -> -1 2014-08-13 18:06:07.264102 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.264118 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.264122 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.264124 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deee40 pg_notify(22.775(36) epoch 194495) v5 2014-08-13 18:06:07.264117 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.264119 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 18:06:07.264140 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264134 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.264163 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.63 2014-08-13 18:06:07.264150 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.264166 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deee40 2014-08-13 18:06:07.264145 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264168 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.264180 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264185 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.264198 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264208 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.264224 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.264232 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379]/[321,379] -- 321/321 2014-08-13 18:06:07.264239 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.264252 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264256 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264256 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.264274 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264277 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.264290 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264293 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.264308 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.264321 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.264323 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 18:06:07.264341 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264334 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.264345 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264348 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.264360 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264362 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.264375 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379,360] newacting [321,379,360] 2014-08-13 18:06:07.264379 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264391 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.264406 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379] r=-1 lpr=194488 pi=192971-194487/25 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.264416 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [353,56]/[353,56,63] -- 353/353 2014-08-13 18:06:07.264436 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264433 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [321,379](321) acting [321,379](321)): not rw, up_thru 194488 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [321,379](321) acting [321,379](321) maybe_went_rw) : primary up 192971-194488 includes interval 2014-08-13 18:06:07.264441 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264449 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [321,379](321) acting [321,379](321) maybe_went_rw) 2014-08-13 18:06:07.264456 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.264468 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379] -> [321,379,360], acting [321,379] -> [321,379,360], acting_primary 321 -> 321, up_primary 321 -> 321, role -1 -> -1 2014-08-13 18:06:07.264471 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] acting_up_affected newup [353,56] newacting [353,56,63] 2014-08-13 18:06:07.264487 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.264487 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.264502 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.264505 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'453 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.264517 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.264533 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.264534 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [353,56](353) acting [353,56](353)): not rw, up_thru 194482 up_from 192973 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [353,56](353) acting [353,56](353)) : primary up 192973-194482 does not include interval 2014-08-13 18:06:07.264548 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.264551 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [353,56](353) acting [353,56](353)) 2014-08-13 18:06:07.264562 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.264570 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] up [353,56] -> [353,56], acting [353,56] -> [353,56,63], acting_primary 353 -> 353, up_primary 353 -> 353, role -1 -> -1 2014-08-13 18:06:07.264578 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.264589 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.264594 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.264605 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.264610 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.264621 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.264625 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.264638 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.264640 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.264653 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.264656 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264669 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.264683 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.264692 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 18:06:07.264699 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.264712 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264719 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.264720 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264735 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.264736 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264750 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.264753 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264767 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264786 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 18:06:07.264804 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264799 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [353,56]/[353,56,63] -- 353/353 2014-08-13 18:06:07.264819 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264808 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264823 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264823 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264839 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264841 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.264857 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264871 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,360]/[321,379,360] -- 321/321 2014-08-13 18:06:07.264889 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264893 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264895 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [369,63,0]/[353,56,63] -- 369/353 2014-08-13 18:06:07.264914 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264907 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.264918 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264923 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.264935 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.264950 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] acting_up_affected newup [369,63,0] newacting [353,56,63] 2014-08-13 18:06:07.264957 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 18:06:07.264976 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.264966 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.264980 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.264982 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [353,56]/[353,56,63] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'453 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.264995 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.265010 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] acting_up_affected newup [321,379,4] newacting [321,379,4] 2014-08-13 18:06:07.265011 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [353,56](353) acting [353,56,63](353)): not rw, up_thru 194489 up_from 192973 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [353,56](353) acting [353,56,63](353) maybe_went_rw) : primary up 192973-194489 includes interval 2014-08-13 18:06:07.265027 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.265029 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [353,56](353) acting [353,56,63](353) maybe_went_rw) 2014-08-13 18:06:07.265042 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,360] r=-1 lpr=194491 pi=192971-194490/26 crt=192888'451 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.265047 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] up [353,56] -> [369,63,0], acting [353,56,63] -> [353,56,63], acting_primary 353 -> 353, up_primary 353 -> 369, role -1 -> -1 2014-08-13 18:06:07.265067 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.265070 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [321,379,360](321) acting [321,379,360](321)): not rw, up_thru 194491 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [321,379,360](321) acting [321,379,360](321) maybe_went_rw) : primary up 192971-194491 includes interval 2014-08-13 18:06:07.265083 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.265087 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [321,379,360](321) acting [321,379,360](321) maybe_went_rw) 2014-08-13 18:06:07.265098 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.265105 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] up [321,379,360] -> [321,379,4], acting [321,379,360] -> [321,379,4], acting_primary 321 -> 321, up_primary 321 -> 321, role -1 -> 2 2014-08-13 18:06:07.265115 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.265125 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.265132 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.265141 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.265149 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.265163 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.265164 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.265179 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.265181 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.265193 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.265198 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.265208 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.265223 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.265222 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.265240 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.265240 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.265256 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.265259 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.265271 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.265285 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.265300 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] handle_advance_map [369,63,0]/[369,63,0] -- 369/369 2014-08-13 18:06:07.265320 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.265324 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.265339 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.265354 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] acting_up_affected newup [369,63,0] newacting [369,63,0] 2014-08-13 18:06:07.265370 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.265384 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0]/[353,56,63] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'453 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.265412 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [369,63,0](369) acting [353,56,63](353)): not rw, up_thru 194489 up_from 192973 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [369,63,0](369) acting [353,56,63](353)) : primary up 192973-194489 does not include interval 2014-08-13 18:06:07.265427 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 18:06:07.265298 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.265445 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] up [369,63,0] -> [369,63,0], acting [353,56,63] -> [369,63,0], acting_primary 353 -> 369, up_primary 369 -> 369, role -1 -> -1 2014-08-13 18:06:07.265451 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.265464 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.265472 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.265479 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.265491 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.265494 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.265509 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.265523 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.265538 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.265552 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.265558 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.265570 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.265575 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.265586 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.265593 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Reset 0.001922 8 0.001624 2014-08-13 18:06:07.265603 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] [353,56,63] -> [369,63,0], acting primary 353 -> 369 2014-08-13 18:06:07.265614 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.265630 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.265634 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.265644 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.265650 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.265662 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.265668 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.265687 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.265701 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [369,63,0]/[369,63,0] -- 369/369 2014-08-13 18:06:07.265719 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.265707 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.265724 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.265724 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.265740 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.265755 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.265790 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [369,63,0]/[369,63,0] -- 369/369 2014-08-13 18:06:07.265808 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.265812 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.265812 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.265827 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.265842 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.265870 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 18:06:07.265896 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.265885 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 18:06:07.265906 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.265903 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.265910 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.265920 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.265926 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.265937 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.265942 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,353,56] newacting [4,353,56] 2014-08-13 18:06:07.265958 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.265957 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.238812 2 0.000248 2014-08-13 18:06:07.265974 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [369,63,0] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'453 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.265979 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.265996 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.238864 0 0.000000 2014-08-13 18:06:07.266004 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [369,63,0](369) acting [369,63,0](369)): not rw, up_thru 194492 up_from 192964 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) : primary up 192964-194492 includes interval 2014-08-13 18:06:07.266017 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.238897 0 0.000000 2014-08-13 18:06:07.266021 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 18:06:07.266036 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Started 0.238953 0 0.000000 2014-08-13 18:06:07.266040 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [369,63,0] -> [4,353,56], acting [369,63,0] -> [4,353,56], acting_primary 369 -> 4, up_primary 369 -> 4, role -1 -> 0 2014-08-13 18:06:07.266056 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.266060 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.266071 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194486 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.266076 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.266088 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194488 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.266092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.266104 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194488 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.266110 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2722 2014-08-13 18:06:07.266119 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194488 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [349,61] newacting [349,61] 2014-08-13 18:06:07.266128 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266136 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194488 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.266152 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [4,349,61] r=0 lpr=194488 pi=193306-194484/26 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.266144 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.266177 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.266193 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,349,61](4) acting [4,349,61](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.266193 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266211 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194485/194485/194485) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] noting past interval(194485-194487 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 18:06:07.266218 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266228 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] up [4,349,61] -> [349,61], acting [4,349,61] -> [349,61], acting_primary 4 -> 349, up_primary 4 -> 349, role 0 -> -1 2014-08-13 18:06:07.266234 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266248 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.266249 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.266264 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.266265 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.266280 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.266283 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.266295 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.266300 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.266310 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.266317 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.266327 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.266334 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.266343 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.266350 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.266358 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.266365 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.266372 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.266380 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.266390 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.266396 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.266406 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.266414 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.266422 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.266433 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.266438 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266452 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.266452 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,56,353 unchanged 2014-08-13 18:06:07.266466 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266471 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.266480 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.266488 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002593 8 0.002390 2014-08-13 18:06:07.266495 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.266510 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.266512 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.266527 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.266542 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.266547 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 18:06:07.266566 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.266560 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000032 0 0.000000 2014-08-13 18:06:07.266571 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.266579 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.266587 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.266596 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.266604 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.266614 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.266631 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.266634 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [349,61]/[349,61] -- 349/349 2014-08-13 18:06:07.266652 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.266648 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 18:06:07.266656 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.266671 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.266669 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 18:06:07.266686 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.266692 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [353,56](353) acting [353,56,63](353) maybe_went_rw) 2014-08-13 18:06:07.266712 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [353,56](353) acting [353,56](353)) 2014-08-13 18:06:07.266729 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,353,56](4) acting [4,353,56](4) maybe_went_rw) 2014-08-13 18:06:07.266730 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 18:06:07.266749 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.266748 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 18:06:07.266753 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.266768 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.266767 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 18:06:07.266784 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] acting_up_affected newup [331,58,330] newacting [331,58,330] 2014-08-13 18:06:07.266787 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,4,56,63,353,369 down blocked_by {} 2014-08-13 18:06:07.266800 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.266806 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.266815 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [349,61] r=-1 lpr=194488 pi=193306-194487/27 crt=191648'372 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.266829 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2723 2014-08-13 18:06:07.266844 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [349,61](349) acting [349,61](349)): not rw, up_thru 194488 up_from 192942 last_epoch_clean 194483 generate_past_intervals interval(194488-194490 up [349,61](349) acting [349,61](349) maybe_went_rw) : primary up 192942-194488 includes interval 2014-08-13 18:06:07.266847 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-13 18:06:07.266860 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194488/194488/194488) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 18:06:07.266866 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.56 2014-08-13 18:06:07.266877 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] up [349,61] -> [331,58,330], acting [349,61] -> [331,58,330], acting_primary 349 -> 331, up_primary 349 -> 331, role -1 -> -1 2014-08-13 18:06:07.266884 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.63 2014-08-13 18:06:07.266896 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.266901 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.353 2014-08-13 18:06:07.266911 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.266919 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.266928 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.266939 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.266946 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.266956 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.266962 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.266979 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.266994 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.267009 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.267024 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.267038 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] [349,61] -> [331,58,330], acting primary 349 -> 331 2014-08-13 18:06:07.267065 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.267062 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.267076 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.267091 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.267133 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 18:06:07.267151 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:07.267161 7f5c7bde3700 7 osd.4 194495 do_notify osd 20 on 1 PGs 2014-08-13 18:06:07.267161 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.267167 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.267182 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.267195 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:07.267199 7f5c7bde3700 7 osd.4 194495 do_notify osd 328 on 1 PGs 2014-08-13 18:06:07.267198 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.267229 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:07.267232 7f5c7bde3700 7 osd.4 194495 do_notify osd 330 on 1 PGs 2014-08-13 18:06:07.267236 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 18:06:07.267254 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.267255 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2f4e0 don't know epoch, doing nothing 2014-08-13 18:06:07.267259 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.0 on 1 PGs 2014-08-13 18:06:07.267258 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.267281 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b17220 don't know epoch, doing nothing 2014-08-13 18:06:07.267284 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.56 on 1 PGs 2014-08-13 18:06:07.267273 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.267291 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.267304 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4fa5280 don't know epoch, doing nothing 2014-08-13 18:06:07.267307 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.63 on 1 PGs 2014-08-13 18:06:07.267324 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:07.267327 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.320 on 1 PGs 2014-08-13 18:06:07.267327 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [331,58,330]/[331,58,330] -- 331/331 2014-08-13 18:06:07.267346 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.267350 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:07.267354 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.344 on 1 PGs 2014-08-13 18:06:07.267349 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.267377 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2fbc0 don't know epoch, doing nothing 2014-08-13 18:06:07.267380 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.352 on 1 PGs 2014-08-13 18:06:07.267399 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x50e1600 don't know epoch, doing nothing 2014-08-13 18:06:07.267402 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.353 sending saperate messages on 1 PGs 2014-08-13 18:06:07.267417 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:07.267419 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.356 on 1 PGs 2014-08-13 18:06:07.267438 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4afcd00 don't know epoch, doing nothing 2014-08-13 18:06:07.267441 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.357 on 1 PGs 2014-08-13 18:06:07.267365 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.267502 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.267539 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 18:06:07.267559 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.267564 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.267580 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4ab7640 don't know epoch, doing nothing 2014-08-13 18:06:07.267588 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.363 on 1 PGs 2014-08-13 18:06:07.267578 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.267593 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,349,61] newacting [4,349,61] 2014-08-13 18:06:07.267614 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4fa1a20 don't know epoch, doing nothing 2014-08-13 18:06:07.267609 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.275079 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.366 on 1 PGs 2014-08-13 18:06:07.267667 7f5c4f387700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.275091 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [331,58,330] r=-1 lpr=194491 pi=193306-194490/28 crt=191648'372 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.275130 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5e880 don't know epoch, doing nothing 2014-08-13 18:06:07.275134 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.369 on 1 PGs 2014-08-13 18:06:07.275158 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4851e40 don't know epoch, doing nothing 2014-08-13 18:06:07.275166 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.371 on 1 PGs 2014-08-13 18:06:07.275148 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [331,58,330](331) acting [331,58,330](331)): not rw, up_thru 194491 up_from 192951 last_epoch_clean 194483 generate_past_intervals interval(194491-194494 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) : primary up 192951-194491 includes interval 2014-08-13 18:06:07.275191 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:07.275195 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.373 on 1 PGs 2014-08-13 18:06:07.275189 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194491/194491/194491) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 18:06:07.275209 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [331,58,330] -> [4,349,61], acting [331,58,330] -> [4,349,61], acting_primary 331 -> 4, up_primary 331 -> 4, role -1 -> 0 2014-08-13 18:06:07.275225 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5c0a0 don't know epoch, doing nothing 2014-08-13 18:06:07.275229 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.380 on 1 PGs 2014-08-13 18:06:07.275229 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.275246 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.275262 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.275279 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:3021 2014-08-13 18:06:07.275295 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.275312 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.275327 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.275341 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.275358 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275363 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275358 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.275373 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.275391 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275394 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275397 7f5c835ef700 20 osd.4 194495 _dispatch 0x4def1c0 pg_notify(22.362(34) epoch 194495) v5 2014-08-13 18:06:07.275392 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.275409 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.275432 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.275445 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.349 2014-08-13 18:06:07.275450 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4def1c0 2014-08-13 18:06:07.275449 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.275472 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.275490 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.275513 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275505 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.275517 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275521 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.275548 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275551 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275540 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.275553 7f5c835ef700 20 osd.4 194495 _dispatch 0x524a140 pg_notify(22.3d9(54) epoch 194495) v5 2014-08-13 18:06:07.275554 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.275572 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.275592 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.382 2014-08-13 18:06:07.275595 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524a140 2014-08-13 18:06:07.275593 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.275612 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,61,349 unchanged 2014-08-13 18:06:07.275644 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275634 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.275648 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275651 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.009596 8 0.009373 2014-08-13 18:06:07.275682 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275685 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275675 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.275687 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf0fc0 pg_notify(24.30b(40) epoch 194495) v5 2014-08-13 18:06:07.275692 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.275708 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.275730 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.10 2014-08-13 18:06:07.275733 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf0fc0 2014-08-13 18:06:07.275731 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000039 0 0.000000 2014-08-13 18:06:07.275752 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.275768 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.275786 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275790 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275784 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.275800 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.275817 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275820 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275823 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1880 pg_notify(22.84(2) epoch 194495) v5 2014-08-13 18:06:07.275818 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 18:06:07.275840 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 18:06:07.275860 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.28 2014-08-13 18:06:07.275864 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1880 2014-08-13 18:06:07.275859 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,349,61](4) acting [4,349,61](4) maybe_went_rw) 2014-08-13 18:06:07.275879 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 18:06:07.275898 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 18:06:07.275914 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275918 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275914 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,330,331,349 down blocked_by {} 2014-08-13 18:06:07.275937 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.275940 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.275942 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4600 pg_notify(24.30b(1) epoch 194495) v5 2014-08-13 18:06:07.275933 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.275953 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:3022 2014-08-13 18:06:07.275979 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.301 2014-08-13 18:06:07.275982 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4600 2014-08-13 18:06:07.275971 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.58 2014-08-13 18:06:07.275991 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.276021 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276009 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.330 2014-08-13 18:06:07.276024 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276026 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.331 2014-08-13 18:06:07.276044 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276047 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276050 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a8d20 pg_query(22.14a epoch 194495) v3 2014-08-13 18:06:07.276046 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.349 2014-08-13 18:06:07.276065 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.276090 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.58 epoch 194495 2014-08-13 18:06:07.276094 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a8d20 2014-08-13 18:06:07.276082 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.276106 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 58 2014-08-13 18:06:07.276150 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276153 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276171 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276174 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276175 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ac560 pg_query(22.28f epoch 194495) v3 2014-08-13 18:06:07.276193 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.276226 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.9 epoch 194495 2014-08-13 18:06:07.276230 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ac560 2014-08-13 18:06:07.276241 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 9 2014-08-13 18:06:07.276253 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 18:06:07.276281 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276284 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276288 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276304 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276308 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276296 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.276311 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a85a0 pg_query(22.29a epoch 194495) v3 2014-08-13 18:06:07.276315 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.276303 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 18:06:07.276332 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,360] newacting [364,301] 2014-08-13 18:06:07.276345 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276348 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.345 epoch 194495 2014-08-13 18:06:07.276352 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a85a0 2014-08-13 18:06:07.276350 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.276354 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.276364 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 345 2014-08-13 18:06:07.276369 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,4,360]/[364,4,301] r=1 lpr=194487 pi=193125-194485/34 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194488 2014-08-13 18:06:07.276373 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327] newacting [294,327] 2014-08-13 18:06:07.276417 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276392 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.276403 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [364,4,360](364) acting [364,4,301](364)): not rw, up_thru 194486 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [364,4,360](364) acting [364,4,301](364) maybe_went_rw) : primary up 192966-194486 includes interval 2014-08-13 18:06:07.276421 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276422 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194485/194486/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194486-194487 up [364,4,360](364) acting [364,4,301](364) maybe_went_rw) 2014-08-13 18:06:07.276440 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276425 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.202560 3 0.000182 2014-08-13 18:06:07.276443 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276446 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ac1a0 pg_query(22.30f epoch 194495) v3 2014-08-13 18:06:07.276442 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,4,360] -> [364,360], acting [364,4,301] -> [364,301], acting_primary 364 -> 364, up_primary 364 -> 364, role 1 -> -1 2014-08-13 18:06:07.276447 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started 0.202650 0 0.000000 2014-08-13 18:06:07.276468 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.276468 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.276489 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.54 epoch 194495 2014-08-13 18:06:07.276494 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ac1a0 2014-08-13 18:06:07.276486 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194486 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.276487 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.276507 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194488 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.276506 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.276505 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 54 2014-08-13 18:06:07.276528 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194488 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.276531 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.276556 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276545 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194488 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327] newacting [294,327] 2014-08-13 18:06:07.276560 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276548 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.276561 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194488 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.276581 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276566 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.276584 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276587 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df32c0 pg_notify(22.362(1) epoch 194495) v5 2014-08-13 18:06:07.276577 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327,4] r=2 lpr=194488 pi=192977-194484/24 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.276585 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.276606 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.276614 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [294,327,4](294) acting [294,327,4](294)): not rw, up_thru 194485 up_from 192968 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [294,327,4](294) acting [294,327,4](294) maybe_went_rw) : primary up 192968-194485 includes interval 2014-08-13 18:06:07.276630 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.276633 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df32c0 2014-08-13 18:06:07.276624 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.276633 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [294,327,4](294) acting [294,327,4](294) maybe_went_rw) 2014-08-13 18:06:07.276643 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_role_change 2014-08-13 18:06:07.276652 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327,4] -> [294,327], acting [294,327,4] -> [294,327], acting_primary 294 -> 294, up_primary 294 -> 294, role 2 -> -1 2014-08-13 18:06:07.276661 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.276673 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.276688 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276678 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.276698 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276689 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.276698 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.276717 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276706 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.276720 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276723 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a9860 pg_query(24.5e epoch 194495) v3 2014-08-13 18:06:07.276719 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.276723 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.276739 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.276762 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.289 epoch 194495 2014-08-13 18:06:07.276765 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a9860 2014-08-13 18:06:07.276756 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.276757 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 18:06:07.276784 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276773 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.276777 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_query query(info 0'0) from replica 289 2014-08-13 18:06:07.276790 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.276793 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.276809 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.276827 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276812 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.276833 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276826 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.276834 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.276849 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.276864 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276868 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276870 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aaee0 pg_query(24.369 epoch 194495) v3 2014-08-13 18:06:07.276858 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,360]/[364,301] -- 364/364 2014-08-13 18:06:07.276865 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.276883 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276880 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.276888 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.276899 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.276916 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.45 epoch 194495 2014-08-13 18:06:07.276920 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aaee0 2014-08-13 18:06:07.276907 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.276925 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.276938 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 18:06:07.276933 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_query query(info 0'0) from replica 45 2014-08-13 18:06:07.276959 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276964 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.276980 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.276984 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276967 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,368,301]/[364,301] -- 364/364 2014-08-13 18:06:07.276980 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.276993 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.276997 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277011 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277014 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.276998 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.277018 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5940 pg_notify(24.b6(43) epoch 194495) v5 2014-08-13 18:06:07.277019 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.277029 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327]/[294,327] -- 294/294 2014-08-13 18:06:07.277049 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277037 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [364,368,301] newacting [364,301] 2014-08-13 18:06:07.277058 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.17 2014-08-13 18:06:07.277061 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5940 2014-08-13 18:06:07.277054 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277056 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.277069 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277074 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,360]/[364,301] r=-1 lpr=194488 pi=193125-194487/35 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194491 2014-08-13 18:06:07.277086 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277115 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277119 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277106 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [364,360](364) acting [364,301](364)): not rw, up_thru 194488 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [364,360](364) acting [364,301](364) maybe_went_rw) : primary up 192966-194488 includes interval 2014-08-13 18:06:07.277121 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 18:06:07.277123 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194488/194488/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194488-194490 up [364,360](364) acting [364,301](364) maybe_went_rw) 2014-08-13 18:06:07.277139 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277144 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277147 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277149 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a9680 pg_query(24.350,24.3b7 epoch 194495) v3 2014-08-13 18:06:07.277144 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277143 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,360] -> [364,368,301], acting [364,301] -> [364,301], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> -1 2014-08-13 18:06:07.277170 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277177 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.277191 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.356 epoch 194495 2014-08-13 18:06:07.277187 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327,363] newacting [294,327,363] 2014-08-13 18:06:07.277199 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a9680 2014-08-13 18:06:07.277197 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.277205 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.277216 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.277223 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327] r=-1 lpr=194488 pi=192977-194487/25 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.277224 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(info 0'0) from replica 356 2014-08-13 18:06:07.277236 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.277253 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.277254 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [294,327](294) acting [294,327](294)): not rw, up_thru 194488 up_from 192968 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [294,327](294) acting [294,327](294) maybe_went_rw) : primary up 192968-194488 includes interval 2014-08-13 18:06:07.277253 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 356 2014-08-13 18:06:07.277273 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [294,327](294) acting [294,327](294) maybe_went_rw) 2014-08-13 18:06:07.277272 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.277300 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277304 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277294 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.277291 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327] -> [294,327,363], acting [294,327] -> [294,327,363], acting_primary 294 -> 294, up_primary 294 -> 294, role -1 -> -1 2014-08-13 18:06:07.277324 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277327 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277315 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.277314 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.277330 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1dc0 pg_notify(24.1f7(1) epoch 194495) v5 2014-08-13 18:06:07.277332 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.277334 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.277349 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.277352 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.277375 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.55 2014-08-13 18:06:07.277378 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1dc0 2014-08-13 18:06:07.277366 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.277370 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.277388 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.277391 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277404 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.277419 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277423 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277419 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.277441 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277444 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277446 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2840 pg_notify(22.6bd(2) epoch 194495) v5 2014-08-13 18:06:07.277435 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.277432 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,368,301]/[364,368,301] -- 364/364 2014-08-13 18:06:07.277459 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277453 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.277464 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.277468 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.277488 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.50 2014-08-13 18:06:07.277491 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2840 2014-08-13 18:06:07.277483 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.277484 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.277507 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277504 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [364,368,301] newacting [364,368,301] 2014-08-13 18:06:07.277524 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.277538 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277541 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277539 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301]/[364,301] r=-1 lpr=194491 pi=193125-194490/36 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194492 2014-08-13 18:06:07.277559 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277545 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 18:06:07.277563 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277565 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277565 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2d80 pg_notify(24.1bb(39) epoch 194495) v5 2014-08-13 18:06:07.277572 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277572 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [364,368,301](364) acting [364,301](364)): not rw, up_thru 194488 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [364,368,301](364) acting [364,301](364)) : primary up 192966-194488 does not include interval 2014-08-13 18:06:07.277589 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277593 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194491/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] noting past interval(194491-194491 up [364,368,301](364) acting [364,301](364)) 2014-08-13 18:06:07.277611 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.337 2014-08-13 18:06:07.277615 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2d80 2014-08-13 18:06:07.277606 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277613 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] up [364,368,301] -> [364,368,301], acting [364,301] -> [364,368,301], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> -1 2014-08-13 18:06:07.277633 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.277641 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 18:06:07.277660 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277650 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.277662 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277666 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277665 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277665 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.277687 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277690 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277692 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2f40 pg_notify(22.4df(32) epoch 194495) v5 2014-08-13 18:06:07.277682 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277684 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.277699 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277701 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.277731 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.371 2014-08-13 18:06:07.277734 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2f40 2014-08-13 18:06:07.277723 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.277732 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,363]/[294,327,363] -- 294/294 2014-08-13 18:06:07.277753 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277740 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.277757 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277758 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.277783 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277773 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277787 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277775 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.277789 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277792 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.277806 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277809 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277811 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aa580 pg_query(24.206 epoch 194495) v3 2014-08-13 18:06:07.277807 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.277819 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 18:06:07.277839 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277826 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277849 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.343 epoch 194495 2014-08-13 18:06:07.277852 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aa580 2014-08-13 18:06:07.277843 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.277858 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.277867 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,368,301]/[364,368,301] -- 364/364 2014-08-13 18:06:07.277874 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] acting_up_affected newup [294,327,4] newacting [294,327,4] 2014-08-13 18:06:07.277897 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.277865 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(info 0'0) from replica 343 2014-08-13 18:06:07.277897 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.277903 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.277914 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,363] r=-1 lpr=194491 pi=192977-194490/26 crt=192888'483 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.277926 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.277942 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277947 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277943 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.277945 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [294,327,363](294) acting [294,327,363](294)): not rw, up_thru 194491 up_from 192968 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [294,327,363](294) acting [294,327,363](294) maybe_went_rw) : primary up 192968-194491 includes interval 2014-08-13 18:06:07.277967 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.277970 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.277972 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aab20 pg_query(24.20e epoch 194495) v3 2014-08-13 18:06:07.277964 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [294,327,363](294) acting [294,327,363](294) maybe_went_rw) 2014-08-13 18:06:07.277980 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,368,301]/[364,368,301] -- 364/364 2014-08-13 18:06:07.277982 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] up [294,327,363] -> [294,327,4], acting [294,327,363] -> [294,327,4], acting_primary 294 -> 294, up_primary 294 -> 294, role -1 -> 2 2014-08-13 18:06:07.278005 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.278004 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.278017 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.15 epoch 194495 2014-08-13 18:06:07.278021 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aab20 2014-08-13 18:06:07.278010 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.278018 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.278028 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.278035 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.278034 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 15 2014-08-13 18:06:07.278046 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.278054 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.278069 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.278091 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278094 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278084 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.278079 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,360] -- 364/364 2014-08-13 18:06:07.278107 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.278102 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.278116 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278119 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278121 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f700 pg_notify(22.362(34) epoch 194495) v5 2014-08-13 18:06:07.278112 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.278119 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.278131 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.278138 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.278163 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.362 2014-08-13 18:06:07.278166 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f700 2014-08-13 18:06:07.278148 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,360] 2014-08-13 18:06:07.278155 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.278175 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.278180 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.278192 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,368,301] r=-1 lpr=194492 pi=193125-194491/37 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194495 2014-08-13 18:06:07.278195 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.278214 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.278229 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [364,368,301](364) acting [364,368,301](364)): not rw, up_thru 194492 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [364,368,301](364) acting [364,368,301](364) maybe_went_rw) : primary up 192966-194492 includes interval 2014-08-13 18:06:07.278232 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.278244 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278248 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.278246 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194491/194492/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] noting past interval(194492-194494 up [364,368,301](364) acting [364,368,301](364) maybe_went_rw) 2014-08-13 18:06:07.278282 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] up [364,368,301] -> [364,4,360], acting [364,368,301] -> [364,4,360], acting_primary 364 -> 364, up_primary 364 -> 364, role -1 -> 1 2014-08-13 18:06:07.278285 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278287 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ac380 pg_query(24.33a epoch 194495) v3 2014-08-13 18:06:07.278286 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.278302 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.278327 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.304 epoch 194495 2014-08-13 18:06:07.278331 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ac380 2014-08-13 18:06:07.278317 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.278316 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.278337 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.278337 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.278345 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(info 0'0) from replica 304 2014-08-13 18:06:07.278355 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.278356 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Reset 0.001888 8 0.001600 2014-08-13 18:06:07.278374 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.278379 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.278401 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278390 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.278405 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278395 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.278408 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.278411 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.278426 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278430 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278432 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6580 pg_notify(22.39d(34) epoch 194495) v5 2014-08-13 18:06:07.278425 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.278442 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] on_role_change 2014-08-13 18:06:07.278458 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.278474 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.363 2014-08-13 18:06:07.278478 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6580 2014-08-13 18:06:07.278473 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.278488 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.278516 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278504 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.278519 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278522 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map 2014-08-13 18:06:07.278542 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278545 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278547 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351880 pg_notify(24.30b epoch 194495) v5 2014-08-13 18:06:07.278580 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.286 2014-08-13 18:06:07.278583 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351880 2014-08-13 18:06:07.278616 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278604 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] update_heartbeat_peers unchanged 2014-08-13 18:06:07.278620 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278623 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] take_waiters 2014-08-13 18:06:07.278644 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278646 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278649 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed6e40 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.278640 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Reset 0.525646 10 0.001906 2014-08-13 18:06:07.278658 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.278663 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.278666 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278668 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278661 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started 2014-08-13 18:06:07.278679 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Start 2014-08-13 18:06:07.278690 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278694 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278696 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed57c0 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.278702 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.278706 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.278693 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: transitioning to Stray 2014-08-13 18:06:07.278708 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278711 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278713 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.278731 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278734 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278736 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1340 pg_notify(24.3d(37) epoch 194495) v5 2014-08-13 18:06:07.278732 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Started/Stray 2014-08-13 18:06:07.278753 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.278776 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.278779 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1340 2014-08-13 18:06:07.278770 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.278817 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278821 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278432 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Start 0.000037 0 0.000000 2014-08-13 18:06:07.278839 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278842 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278844 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ad280 pg_query(22.1a0 epoch 194495) v3 2014-08-13 18:06:07.278838 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.278871 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.278863 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.278880 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.328 epoch 194495 2014-08-13 18:06:07.278883 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ad280 2014-08-13 18:06:07.278883 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.278895 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 328 2014-08-13 18:06:07.278932 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278935 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278917 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.278946 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.278954 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.278957 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.278959 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5780 pg_notify(24.1f7(43) epoch 194495) v5 2014-08-13 18:06:07.278953 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.278973 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.278969 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.278983 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.278997 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.351 2014-08-13 18:06:07.279000 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5780 2014-08-13 18:06:07.279001 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.250195 2 0.000190 2014-08-13 18:06:07.279021 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.279045 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279049 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279035 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.250250 0 0.000000 2014-08-13 18:06:07.279034 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 18:06:07.279056 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.250285 0 0.000000 2014-08-13 18:06:07.279068 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279068 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279071 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279074 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a9fe0 pg_query(24.181 epoch 194495) v3 2014-08-13 18:06:07.279073 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.250365 0 0.000000 2014-08-13 18:06:07.279075 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.279091 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.279094 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.279112 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.320 epoch 194495 2014-08-13 18:06:07.279116 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a9fe0 2014-08-13 18:06:07.279107 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.279113 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,304] newacting [346,288] 2014-08-13 18:06:07.279124 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.279134 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.279129 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_query query(info 0'0) from replica 320 2014-08-13 18:06:07.279145 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.279154 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,4,304]/[346,4,288] r=1 lpr=194487 pi=193125-194485/33 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194488 2014-08-13 18:06:07.279178 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279182 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279176 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340] newacting [340] 2014-08-13 18:06:07.279191 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.279208 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279212 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279197 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [346,4,304](346) acting [346,4,288](346)): not rw, up_thru 194486 up_from 192955 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [346,4,304](346) acting [346,4,288](346) maybe_went_rw) : primary up 192955-194486 includes interval 2014-08-13 18:06:07.279214 7f5c835ef700 20 osd.4 194495 _dispatch 0x21acec0 pg_query(22.1fb epoch 194495) v3 2014-08-13 18:06:07.279206 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.279217 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194485/194486/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194486-194487 up [346,4,304](346) acting [346,4,288](346) maybe_went_rw) 2014-08-13 18:06:07.279237 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,340](4) acting [4,340](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.279252 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.373 epoch 194495 2014-08-13 18:06:07.279256 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21acec0 2014-08-13 18:06:07.279237 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,4,304] -> [346,304], acting [346,4,288] -> [346,288], acting_primary 346 -> 346, up_primary 346 -> 346, role 1 -> -1 2014-08-13 18:06:07.279256 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] noting past interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:07.279261 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:07.279270 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 373 2014-08-13 18:06:07.279274 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] up [4,340] -> [340], acting [4,340] -> [340], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 18:06:07.279281 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.279294 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.279309 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279298 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.279314 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279310 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.279320 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 18:06:07.279326 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.279337 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279344 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279347 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a8960 pg_query(22.3b9 epoch 194495) v3 2014-08-13 18:06:07.279338 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 18:06:07.279344 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.279356 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 18:06:07.279359 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.279386 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.397 epoch 194495 2014-08-13 18:06:07.279374 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:07.279389 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a8960 2014-08-13 18:06:07.279376 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.279394 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.279395 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 18:06:07.279402 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 397 2014-08-13 18:06:07.279412 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.279415 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 18:06:07.279426 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.279435 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_role_change 2014-08-13 18:06:07.279454 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279444 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.279459 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279451 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.279461 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.279478 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279482 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279485 7f5c835ef700 20 osd.4 194495 _dispatch 0x21aa760 pg_query(24.299 epoch 194495) v3 2014-08-13 18:06:07.279468 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 18:06:07.279479 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.279490 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 18:06:07.279497 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.279511 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.279522 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.396 epoch 194495 2014-08-13 18:06:07.279526 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21aa760 2014-08-13 18:06:07.279513 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279525 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.279541 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.279538 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 396 2014-08-13 18:06:07.279554 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.279553 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 18:06:07.279582 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279572 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279585 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279589 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279587 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.279609 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279612 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279614 7f5c835ef700 20 osd.4 194495 _dispatch 0x4def540 pg_notify(24.1f7(39) epoch 194495) v5 2014-08-13 18:06:07.279604 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.279605 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.279625 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279626 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279630 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.279653 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.279656 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4def540 2014-08-13 18:06:07.279645 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.279661 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279660 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,304]/[346,288] -- 346/346 2014-08-13 18:06:07.279686 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279698 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279702 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279689 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.279690 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.279708 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279709 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.279713 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.279731 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279734 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279736 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed06c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.279726 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279727 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.279744 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.279748 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.279751 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279753 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279746 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.279774 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.279778 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.279765 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,288,369]/[346,288] -- 346/346 2014-08-13 18:06:07.279780 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df16c0 pg_notify(2.2a(1),1.2b(1) epoch 194495) v5 2014-08-13 18:06:07.279788 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279782 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.279804 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.279793 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.279808 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.279811 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.279828 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.324 2014-08-13 18:06:07.279831 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df16c0 2014-08-13 18:06:07.279823 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.279830 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] acting_up_affected newup [346,288,369] newacting [346,288] 2014-08-13 18:06:07.279840 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [324,370] newacting [324,370] 2014-08-13 18:06:07.279849 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.279857 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.279867 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,304]/[346,288] r=-1 lpr=194488 pi=193125-194487/34 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] set_last_peering_reset 194491 2014-08-13 18:06:07.279873 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.279898 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [340](340) acting [340](340)): not rw, up_thru 194488 up_from 192944 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) : primary up 192944-194488 includes interval 2014-08-13 18:06:07.279896 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [346,304](346) acting [346,288](346)): not rw, up_thru 194488 up_from 192955 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [346,304](346) acting [346,288](346) maybe_went_rw) : primary up 192955-194488 includes interval 2014-08-13 18:06:07.279916 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.279918 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194488/194488/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194488-194490 up [346,304](346) acting [346,288](346) maybe_went_rw) 2014-08-13 18:06:07.279932 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] up [340] -> [324,370], acting [340] -> [324,370], acting_primary 340 -> 324, up_primary 340 -> 324, role -1 -> -1 2014-08-13 18:06:07.279938 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,304] -> [346,288,369], acting [346,288] -> [346,288], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> -1 2014-08-13 18:06:07.279951 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.279958 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:07.279965 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.279975 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.279981 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.279992 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.279995 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.280009 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.280009 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 18:06:07.280024 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.280027 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 18:06:07.280038 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.280044 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 18:06:07.280054 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.280061 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:07.280070 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.280079 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 18:06:07.280097 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 18:06:07.280086 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] [340] -> [324,370], acting primary 340 -> 324 2014-08-13 18:06:07.280114 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.280113 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 18:06:07.280129 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.280130 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 18:06:07.280145 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.280149 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.280199 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.280219 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.280223 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.280224 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,288,369]/[346,288,369] -- 346/346 2014-08-13 18:06:07.280238 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.280255 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.280254 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.280261 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.280278 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.280294 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] acting_up_affected newup [346,288,369] newacting [346,288,369] 2014-08-13 18:06:07.280289 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.280315 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.280311 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.280318 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.280327 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369]/[346,288] r=-1 lpr=194491 pi=193125-194490/35 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] set_last_peering_reset 194492 2014-08-13 18:06:07.280334 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.280349 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.280363 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [346,288,369](346) acting [346,288](346)): not rw, up_thru 194488 up_from 192955 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [346,288,369](346) acting [346,288](346)) : primary up 192955-194488 does not include interval 2014-08-13 18:06:07.280380 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194491/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] noting past interval(194491-194491 up [346,288,369](346) acting [346,288](346)) 2014-08-13 18:06:07.280384 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.280402 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.280398 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] up [346,288,369] -> [346,288,369], acting [346,288] -> [346,288,369], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> -1 2014-08-13 18:06:07.280406 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.280417 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 18:06:07.280420 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.280433 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.280435 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.280448 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.280461 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:07.291321 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.291313 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_copy_ops 2014-08-13 18:06:07.291337 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.291351 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_flush_ops 2014-08-13 18:06:07.291364 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.291368 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change_cleanup 2014-08-13 18:06:07.291381 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340] newacting [4,340] 2014-08-13 18:06:07.291384 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 18:06:07.291397 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.291403 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit NotTrimming 2014-08-13 18:06:07.291413 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.291420 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter NotTrimming 2014-08-13 18:06:07.291437 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_recovery 2014-08-13 18:06:07.291447 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [324,370](324) acting [324,370](324)): not rw, up_thru 194491 up_from 192951 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) : primary up 192951-194491 includes interval 2014-08-13 18:06:07.291452 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] clear_recovery_state 2014-08-13 18:06:07.291466 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.291471 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.291484 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [324,370] -> [4,340], acting [324,370] -> [4,340], acting_primary 324 -> 4, up_primary 324 -> 4, role -1 -> 0 2014-08-13 18:06:07.291502 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.291517 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.291532 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.291528 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,288,369]/[346,288,369] -- 346/346 2014-08-13 18:06:07.291559 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.291548 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:176854 2014-08-13 18:06:07.291567 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.291566 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.291584 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.291584 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.291600 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.291603 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.291615 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.291628 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.291641 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.291641 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,288,369]/[346,288,369] -- 346/346 2014-08-13 18:06:07.291666 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.291657 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.291671 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.291671 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.291689 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.291689 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.291707 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.291707 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.291723 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.291737 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.291740 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,304] -- 346/346 2014-08-13 18:06:07.291751 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.291767 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.291765 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.291771 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:07.291778 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.291788 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.291793 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.291804 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,304] 2014-08-13 18:06:07.291811 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.291822 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.291832 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.291841 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,288,369] r=-1 lpr=194492 pi=193125-194491/36 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194495 2014-08-13 18:06:07.291851 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340 unchanged 2014-08-13 18:06:07.291868 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.291876 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [346,288,369](346) acting [346,288,369](346)): not rw, up_thru 194492 up_from 192955 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [346,288,369](346) acting [346,288,369](346) maybe_went_rw) : primary up 192955-194492 includes interval 2014-08-13 18:06:07.291884 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.012793 8 0.001727 2014-08-13 18:06:07.291893 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194491/194492/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] noting past interval(194492-194494 up [346,288,369](346) acting [346,288,369](346) maybe_went_rw) 2014-08-13 18:06:07.291906 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.291913 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] up [346,288,369] -> [346,4,304], acting [346,288,369] -> [346,4,304], acting_primary 346 -> 346, up_primary 346 -> 346, role -1 -> 1 2014-08-13 18:06:07.291922 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.291934 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 18:06:07.291938 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.291950 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.291956 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.291965 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.291976 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.291983 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_copy_ops 2014-08-13 18:06:07.291993 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.291999 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_flush_ops 2014-08-13 18:06:07.292010 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.292015 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change_cleanup 2014-08-13 18:06:07.292027 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.292031 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_change 2014-08-13 18:06:07.292045 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.292048 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit NotTrimming 2014-08-13 18:06:07.292064 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter NotTrimming 2014-08-13 18:06:07.292065 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.292082 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] on_role_change 2014-08-13 18:06:07.292083 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:07.292097 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.292102 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 18:06:07.292112 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] cancel_recovery 2014-08-13 18:06:07.292120 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.292128 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] clear_recovery_state 2014-08-13 18:06:07.292145 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.292140 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:176855 2014-08-13 18:06:07.292198 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map 2014-08-13 18:06:07.292207 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 18:06:07.292227 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.292243 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.370 2014-08-13 18:06:07.292265 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.292268 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] update_heartbeat_peers unchanged 2014-08-13 18:06:07.292281 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.292286 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] take_waiters 2014-08-13 18:06:07.292304 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Reset 0.604171 10 0.012758 2014-08-13 18:06:07.292326 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started 2014-08-13 18:06:07.292350 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.292343 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Start 2014-08-13 18:06:07.292358 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: transitioning to Stray 2014-08-13 18:06:07.292377 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Start 0.000033 0 0.000000 2014-08-13 18:06:07.292398 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Started/Stray 2014-08-13 18:06:07.292419 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.292412 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_advance_map [339,45]/[339,45] -- 339/339 2014-08-13 18:06:07.292439 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.292436 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.292446 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.292472 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292476 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292466 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.292482 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.292503 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292510 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292513 7f5c835ef700 20 osd.4 194495 _dispatch 0x524c7c0 pg_notify(22.775(36) epoch 194495) v5 2014-08-13 18:06:07.292503 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.263202 2 0.000248 2014-08-13 18:06:07.292525 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.292542 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.292552 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.0 2014-08-13 18:06:07.292556 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524c7c0 2014-08-13 18:06:07.292541 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.263250 0 0.000000 2014-08-13 18:06:07.292563 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.263282 0 0.000000 2014-08-13 18:06:07.292592 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292595 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292583 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Started 0.263347 0 0.000000 2014-08-13 18:06:07.292602 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.292597 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 18:06:07.292622 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292623 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.292625 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292617 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194486 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.292631 7f5c835ef700 20 osd.4 194495 _dispatch 0x524cec0 pg_notify(22.4df(1) epoch 194495) v5 2014-08-13 18:06:07.292633 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.292635 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194488 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.292651 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289] newacting [334,289] 2014-08-13 18:06:07.292655 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194488 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.292671 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.352 2014-08-13 18:06:07.292675 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524cec0 2014-08-13 18:06:07.292669 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.292673 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194488 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [339,45] newacting [339,45] 2014-08-13 18:06:07.292689 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.264413 3 0.000151 2014-08-13 18:06:07.292691 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194488 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.292712 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292715 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292708 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [4,339,45] r=0 lpr=194488 pi=193306-194484/28 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.292711 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started 0.264478 0 0.000000 2014-08-13 18:06:07.292736 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292739 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292741 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dea140 pg_notify(22.48e(34) epoch 194495) v5 2014-08-13 18:06:07.292731 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.292742 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,339,45](4) acting [4,339,45](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.292747 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194486 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.292760 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194485/194485/194485) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] noting past interval(194485-194487 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 18:06:07.292767 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194488 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.292782 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.327 2014-08-13 18:06:07.292785 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dea140 2014-08-13 18:06:07.292783 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194488 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.292779 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] up [4,339,45] -> [339,45], acting [4,339,45] -> [339,45], acting_primary 4 -> 339, up_primary 4 -> 339, role 0 -> -1 2014-08-13 18:06:07.292801 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.292800 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194488 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289] newacting [334,289] 2014-08-13 18:06:07.292826 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292829 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292819 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194488 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.292819 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.292847 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292850 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292837 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289,4] r=2 lpr=194488 pi=192974-194484/24 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.292840 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.292852 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3800 pg_notify(1.2b(2) epoch 194495) v5 2014-08-13 18:06:07.292857 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.292872 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.292869 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [334,289,4](334) acting [334,289,4](334)): not rw, up_thru 194485 up_from 192946 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [334,289,4](334) acting [334,289,4](334) maybe_went_rw) : primary up 192946-194485 includes interval 2014-08-13 18:06:07.292891 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.370 2014-08-13 18:06:07.292895 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3800 2014-08-13 18:06:07.292888 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [334,289,4](334) acting [334,289,4](334) maybe_went_rw) 2014-08-13 18:06:07.292889 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.292908 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.292907 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289,4] -> [334,289], acting [334,289,4] -> [334,289], acting_primary 334 -> 334, up_primary 334 -> 334, role 2 -> -1 2014-08-13 18:06:07.292926 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.292930 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.292946 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292950 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292941 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.292946 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.292968 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.292971 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.292959 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.292962 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.292973 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5400 pg_notify(22.1b8(2) epoch 194495) v5 2014-08-13 18:06:07.292975 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.292980 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.292992 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.292996 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.293018 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.7 2014-08-13 18:06:07.293021 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5400 2014-08-13 18:06:07.293009 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.293012 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.293025 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.293028 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.293041 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.293046 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.293064 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293068 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293056 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.293064 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.293072 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.293088 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293091 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293094 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2300 pg_notify(22.3e4(36) epoch 194495) v5 2014-08-13 18:06:07.293084 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.293090 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293100 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.293114 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.293137 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.293140 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2300 2014-08-13 18:06:07.293128 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.293129 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [339,45]/[339,45,24] -- 339/339 2014-08-13 18:06:07.293150 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293150 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293181 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293187 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293177 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293195 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.293209 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] acting_up_affected newup [339,45] newacting [339,45,24] 2014-08-13 18:06:07.293208 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 18:06:07.293228 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293232 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293232 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293234 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5240 pg_notify(22.4df(32) epoch 194495) v5 2014-08-13 18:06:07.293224 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.293237 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293240 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'383 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.293253 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.293275 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.380 2014-08-13 18:06:07.293278 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5240 2014-08-13 18:06:07.293270 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293271 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [339,45](339) acting [339,45](339)): not rw, up_thru 194482 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [339,45](339) acting [339,45](339)) : primary up 192978-194482 does not include interval 2014-08-13 18:06:07.293290 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194488/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [339,45](339) acting [339,45](339)) 2014-08-13 18:06:07.293301 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289]/[334,289] -- 334/334 2014-08-13 18:06:07.293316 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293319 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293320 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293310 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] up [339,45] -> [339,45], acting [339,45] -> [339,45,24], acting_primary 339 -> 339, up_primary 339 -> 339, role -1 -> -1 2014-08-13 18:06:07.293324 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293331 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.293351 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293340 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.293354 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293357 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6200 pg_notify(22.1f1(34) epoch 194495) v5 2014-08-13 18:06:07.293348 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.293355 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293364 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.293381 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.293397 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.368 2014-08-13 18:06:07.293400 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6200 2014-08-13 18:06:07.293390 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 18:06:07.293411 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293398 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.293415 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293416 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.293434 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.293435 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.293449 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289,65] newacting [334,289,65] 2014-08-13 18:06:07.293452 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.293465 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.293469 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.293484 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293488 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293480 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289] r=-1 lpr=194488 pi=192974-194487/25 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.293487 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.293508 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293510 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293513 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6c80 pg_notify(22.f3(26) epoch 194495) v5 2014-08-13 18:06:07.293504 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.293511 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [334,289](334) acting [334,289](334)): not rw, up_thru 194488 up_from 192946 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [334,289](334) acting [334,289](334) maybe_went_rw) : primary up 192946-194488 includes interval 2014-08-13 18:06:07.293522 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293530 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [334,289](334) acting [334,289](334) maybe_went_rw) 2014-08-13 18:06:07.293552 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.293555 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6c80 2014-08-13 18:06:07.293548 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289] -> [334,289,65], acting [334,289] -> [334,289,65], acting_primary 334 -> 334, up_primary 334 -> 334, role -1 -> -1 2014-08-13 18:06:07.293559 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [339,45]/[339,45,24] -- 339/339 2014-08-13 18:06:07.293567 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.293580 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293583 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.293585 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293601 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293605 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293599 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.293602 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.293624 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293627 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293615 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.293629 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351f80 pg_notify(24.30b(44) epoch 194495) v5 2014-08-13 18:06:07.293619 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293630 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.293646 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.293669 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.375 2014-08-13 18:06:07.293672 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351f80 2014-08-13 18:06:07.293662 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.293660 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [342,55,24]/[339,45,24] -- 342/339 2014-08-13 18:06:07.293682 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293680 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.293687 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293710 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293697 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.293714 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293705 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.293714 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.293722 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] acting_up_affected newup [342,55,24] newacting [339,45,24] 2014-08-13 18:06:07.293730 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.293745 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293748 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293751 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dea4c0 pg_notify(22.1d2(30) epoch 194495) v5 2014-08-13 18:06:07.293739 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.293748 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293758 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [339,45]/[339,45,24] r=-1 lpr=194489 pi=193306-194488/30 crt=191648'383 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.293794 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.330 2014-08-13 18:06:07.293798 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dea4c0 2014-08-13 18:06:07.293787 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 18:06:07.293788 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [339,45](339) acting [339,45,24](339)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [339,45](339) acting [339,45,24](339) maybe_went_rw) : primary up 192978-194489 includes interval 2014-08-13 18:06:07.293807 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293807 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194488/194489/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [339,45](339) acting [339,45,24](339) maybe_went_rw) 2014-08-13 18:06:07.293813 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293836 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293839 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293829 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.293826 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] up [339,45] -> [342,55,24], acting [339,45,24] -> [339,45,24], acting_primary 339 -> 339, up_primary 339 -> 342, role -1 -> -1 2014-08-13 18:06:07.293845 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293858 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293848 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.293862 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293869 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deb100 pg_notify(22.481(42) epoch 194495) v5 2014-08-13 18:06:07.293867 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.293878 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 18:06:07.293884 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.293898 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293909 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.365 2014-08-13 18:06:07.293912 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deb100 2014-08-13 18:06:07.293902 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.293901 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.293922 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.293923 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.293951 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293940 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.293954 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293942 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.293958 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.293972 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,65]/[334,289,65] -- 334/334 2014-08-13 18:06:07.293974 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.293989 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.293994 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.293995 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.293998 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df40c0 pg_notify(22.39d(1) epoch 194495) v5 2014-08-13 18:06:07.293994 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.293998 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294012 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.294014 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.294039 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.366 2014-08-13 18:06:07.294029 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.294042 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df40c0 2014-08-13 18:06:07.294031 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294047 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294062 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 18:06:07.294078 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294082 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294081 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294085 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294100 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294104 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294088 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] handle_advance_map [342,55,24]/[342,55,24] -- 342/342 2014-08-13 18:06:07.294107 7f5c835ef700 20 osd.4 194495 _dispatch 0x524d240 pg_notify(22.775(36) epoch 194495) v5 2014-08-13 18:06:07.294110 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294100 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.294115 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294116 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] acting_up_affected newup [334,289,4] newacting [334,289,4] 2014-08-13 18:06:07.294145 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.63 2014-08-13 18:06:07.294134 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.294149 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524d240 2014-08-13 18:06:07.294134 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.294149 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,65] r=-1 lpr=194491 pi=192974-194490/26 crt=192888'488 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.294166 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] acting_up_affected newup [342,55,24] newacting [342,55,24] 2014-08-13 18:06:07.294185 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294190 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294185 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.294188 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [334,289,65](334) acting [334,289,65](334)): not rw, up_thru 194491 up_from 192946 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [334,289,65](334) acting [334,289,65](334) maybe_went_rw) : primary up 192946-194491 includes interval 2014-08-13 18:06:07.294205 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24]/[339,45,24] r=-1 lpr=194491 pi=193306-194490/31 crt=191648'383 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.294206 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [334,289,65](334) acting [334,289,65](334) maybe_went_rw) 2014-08-13 18:06:07.294223 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294227 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294230 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f380 pg_notify(22.362(34) epoch 194495) v5 2014-08-13 18:06:07.294226 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] up [334,289,65] -> [334,289,4], acting [334,289,65] -> [334,289,4], acting_primary 334 -> 334, up_primary 334 -> 334, role -1 -> 2 2014-08-13 18:06:07.294239 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [342,55,24](342) acting [339,45,24](339)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [342,55,24](342) acting [339,45,24](339)) : primary up 192978-194489 does not include interval 2014-08-13 18:06:07.294247 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.294270 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.349 2014-08-13 18:06:07.294256 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194491/194488) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [342,55,24](342) acting [339,45,24](339)) 2014-08-13 18:06:07.294274 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f380 2014-08-13 18:06:07.294263 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.294280 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.294276 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] up [342,55,24] -> [342,55,24], acting [339,45,24] -> [342,55,24], acting_primary 339 -> 342, up_primary 342 -> 342, role -1 -> -1 2014-08-13 18:06:07.294296 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.294296 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.294312 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294315 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294313 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.294313 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.294331 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.294345 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294333 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.294348 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294351 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc45c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.294346 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.294359 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.294348 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.294365 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.294368 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294372 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294363 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.294367 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.294379 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.294394 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294398 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294386 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.294401 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1340 pg_notify(22.3d5(2) epoch 194495) v5 2014-08-13 18:06:07.294396 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.294404 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.294414 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.294420 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.294444 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.10 2014-08-13 18:06:07.294432 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.294448 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1340 2014-08-13 18:06:07.294435 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.294458 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.294449 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] [339,45,24] -> [342,55,24], acting primary 339 -> 342 2014-08-13 18:06:07.294476 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294477 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.294498 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294502 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294493 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.294495 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.294509 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294521 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294524 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294526 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351c00 pg_notify(24.30b(1) epoch 194495) v5 2014-08-13 18:06:07.294562 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.301 2014-08-13 18:06:07.294549 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [342,55,24]/[342,55,24] -- 342/342 2014-08-13 18:06:07.294566 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351c00 2014-08-13 18:06:07.294568 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294557 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.294575 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.294573 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294603 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294606 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294594 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.294594 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Reset 0.001864 8 0.001562 2014-08-13 18:06:07.294614 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294618 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.294633 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294636 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294639 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df08c0 pg_notify(22.1d2(30) epoch 194495) v5 2014-08-13 18:06:07.294634 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.294649 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.294650 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [342,55,24]/[342,55,24] -- 342/342 2014-08-13 18:06:07.294671 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294678 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.58 2014-08-13 18:06:07.294682 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df08c0 2014-08-13 18:06:07.294669 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Start 0.000035 0 0.000000 2014-08-13 18:06:07.294676 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294690 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.294693 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.294719 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294709 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.294723 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294710 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.294729 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.294741 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294745 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294747 7f5c835ef700 20 osd.4 194495 _dispatch 0x21a94a0 pg_query(22.116 epoch 194495) v3 2014-08-13 18:06:07.294740 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 18:06:07.294761 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294765 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.294783 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.345 epoch 194495 2014-08-13 18:06:07.294787 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21a94a0 2014-08-13 18:06:07.294780 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.294795 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,339,45] newacting [4,339,45] 2014-08-13 18:06:07.294814 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.294799 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 345 2014-08-13 18:06:07.294812 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.294827 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [342,55,24] r=-1 lpr=194492 pi=193306-194491/32 crt=191648'383 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.294848 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294852 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294870 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294874 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294856 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [342,55,24](342) acting [342,55,24](342)): not rw, up_thru 194492 up_from 192964 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) : primary up 192964-194492 includes interval 2014-08-13 18:06:07.294877 7f5c835ef700 20 osd.4 194495 _dispatch 0x4dea840 pg_notify(22.1f1(34) epoch 194495) v5 2014-08-13 18:06:07.294870 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [313,318]/[313,318] -- 313/313 2014-08-13 18:06:07.294904 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.294881 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194491/194492/194492) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 18:06:07.294924 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.54 2014-08-13 18:06:07.294911 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.294928 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4dea840 2014-08-13 18:06:07.294915 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [342,55,24] -> [4,339,45], acting [342,55,24] -> [4,339,45], acting_primary 342 -> 4, up_primary 342 -> 4, role -1 -> 0 2014-08-13 18:06:07.294929 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.294940 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.294947 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.294966 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.294970 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294959 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.294968 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.265721 2 0.000247 2014-08-13 18:06:07.294977 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.294996 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295000 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.294989 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.295002 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3100 pg_notify(22.362(1) epoch 194495) v5 2014-08-13 18:06:07.294995 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2934 2014-08-13 18:06:07.295006 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.265769 0 0.000000 2014-08-13 18:06:07.295015 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295043 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.295033 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.295047 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3100 2014-08-13 18:06:07.295035 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.265807 0 0.000000 2014-08-13 18:06:07.295050 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.295056 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.265877 0 0.000000 2014-08-13 18:06:07.295066 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295085 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295088 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295076 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.295083 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295092 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194486 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.295110 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295114 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295099 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295119 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1a40 pg_notify(22.48e(1) epoch 194495) v5 2014-08-13 18:06:07.295109 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194488 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.295120 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.295127 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194488 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.295159 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.289 2014-08-13 18:06:07.295137 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.295162 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1a40 2014-08-13 18:06:07.295144 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194488 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [313,318] newacting [313,318] 2014-08-13 18:06:07.295166 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.295170 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194488 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.295185 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.295187 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [4,313,318] r=0 lpr=194488 pi=193306-194484/32 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.295200 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.295222 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295227 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295215 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.295221 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,313,318](4) acting [4,313,318](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.295231 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.295249 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295253 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295240 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194485/194485/194485) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] noting past interval(194485-194487 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) 2014-08-13 18:06:07.295256 7f5c835ef700 20 osd.4 194495 _dispatch 0x524cb40 pg_notify(22.4df epoch 194495) v5 2014-08-13 18:06:07.295250 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.295259 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] up [4,313,318] -> [313,318], acting [4,313,318] -> [313,318], acting_primary 4 -> 313, up_primary 4 -> 313, role 0 -> -1 2014-08-13 18:06:07.295267 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.295280 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.295284 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.295297 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.356 2014-08-13 18:06:07.295302 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524cb40 2014-08-13 18:06:07.295298 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.295302 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.295314 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.295323 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.295331 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.295346 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295350 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295343 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,45,339 unchanged 2014-08-13 18:06:07.295345 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.295369 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295360 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.295372 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295362 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.295376 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ab660 pg_query(22.4fe epoch 194495) v3 2014-08-13 18:06:07.295377 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.295377 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002776 8 0.002534 2014-08-13 18:06:07.295396 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.295400 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.295419 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.337 epoch 194495 2014-08-13 18:06:07.295423 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ab660 2014-08-13 18:06:07.295411 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.295419 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.295429 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.295436 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.295435 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 337 2014-08-13 18:06:07.295448 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.295457 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000038 0 0.000000 2014-08-13 18:06:07.295466 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.295491 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295480 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.295481 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295495 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295498 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.295499 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.295518 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295522 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295524 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5cc0 pg_notify(22.4df(32) epoch 194495) v5 2014-08-13 18:06:07.295515 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.295514 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.295532 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.295533 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.295548 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.295567 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.371 2014-08-13 18:06:07.295571 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5cc0 2014-08-13 18:06:07.295555 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 18:06:07.295565 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.295578 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [342,55,24](342) acting [339,45,24](339)) 2014-08-13 18:06:07.295610 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295596 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [339,45](339) acting [339,45,24](339) maybe_went_rw) 2014-08-13 18:06:07.295614 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295603 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [313,318]/[313,318,45] -- 313/313 2014-08-13 18:06:07.295628 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.295615 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [339,45](339) acting [339,45](339)) 2014-08-13 18:06:07.295642 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295645 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295633 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.295648 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f000 pg_notify(22.362 epoch 194495) v5 2014-08-13 18:06:07.295633 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,339,45](4) acting [4,339,45](4) maybe_went_rw) 2014-08-13 18:06:07.295651 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.295656 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 18:06:07.295668 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [313,318] newacting [313,318,45] 2014-08-13 18:06:07.295687 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.343 2014-08-13 18:06:07.295691 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f000 2014-08-13 18:06:07.295685 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.295677 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [342,55,24](342) acting [339,45,55](339)) 2014-08-13 18:06:07.295701 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.295703 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,24,45,55,339,342 down blocked_by {} 2014-08-13 18:06:07.295726 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295729 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295724 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.295732 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [313,318](313) acting [313,318](313)): not rw, up_thru 194482 up_from 192981 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [313,318](313) acting [313,318](313)) : primary up 192981-194482 does not include interval 2014-08-13 18:06:07.295748 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295751 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295754 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df63c0 pg_notify(22.39d(34) epoch 194495) v5 2014-08-13 18:06:07.295744 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2935 2014-08-13 18:06:07.295750 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194488/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [313,318](313) acting [313,318](313)) 2014-08-13 18:06:07.295763 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.24 2014-08-13 18:06:07.295770 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] up [313,318] -> [313,318], acting [313,318] -> [313,318,45], acting_primary 313 -> 313, up_primary 313 -> 313, role -1 -> -1 2014-08-13 18:06:07.295792 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.363 2014-08-13 18:06:07.295796 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df63c0 2014-08-13 18:06:07.295783 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 18:06:07.295791 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.295802 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.55 2014-08-13 18:06:07.295810 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.295821 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 18:06:07.295829 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.295841 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.295845 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.295858 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.295860 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.295874 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295878 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295877 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.295878 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.295901 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295905 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295907 7f5c835ef700 20 osd.4 194495 _dispatch 0x524d400 pg_notify(22.4df epoch 194495) v5 2014-08-13 18:06:07.295897 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.295921 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.295944 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.357 2014-08-13 18:06:07.295947 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524d400 2014-08-13 18:06:07.295939 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.295956 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.295982 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.295981 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.295971 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.295989 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.295989 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296007 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.296010 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.296018 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4980 pg_notify(22.39d epoch 194495) v5 2014-08-13 18:06:07.296025 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [313,318]/[313,318,45] -- 313/313 2014-08-13 18:06:07.296045 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296053 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.344 2014-08-13 18:06:07.296056 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4980 2014-08-13 18:06:07.296050 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.296045 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296067 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.296080 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296090 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.296094 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.296085 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296087 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296113 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.296116 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.296118 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1180 pg_notify(22.3e4(1) epoch 194495) v5 2014-08-13 18:06:07.296107 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296124 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [303,342,45]/[313,318,45] -- 303/313 2014-08-13 18:06:07.296143 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296157 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.296160 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1180 2014-08-13 18:06:07.296147 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.296147 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296184 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296173 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.296189 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296189 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,342,45] newacting [313,318,45] 2014-08-13 18:06:07.296208 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.296208 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296225 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [313,318]/[313,318,45] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'442 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.296247 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296253 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [313,318](313) acting [313,318,45](313)): not rw, up_thru 194489 up_from 192981 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [313,318](313) acting [313,318,45](313) maybe_went_rw) : primary up 192981-194489 includes interval 2014-08-13 18:06:07.296270 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296270 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194488/194489/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [313,318](313) acting [313,318,45](313) maybe_went_rw) 2014-08-13 18:06:07.296274 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296292 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296288 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] up [313,318] -> [303,342,45], acting [313,318,45] -> [313,318,45], acting_primary 313 -> 313, up_primary 313 -> 303, role -1 -> -1 2014-08-13 18:06:07.296308 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.296323 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.296328 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296338 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.296351 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296355 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.296356 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296371 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.296374 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296387 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.296402 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.296411 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296418 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.296434 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296436 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.296438 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296451 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.296456 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296467 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.296484 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296493 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296515 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296519 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296525 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] handle_advance_map [303,342,45]/[303,342,45] -- 303/303 2014-08-13 18:06:07.296545 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296536 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296550 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.296565 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.296572 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296580 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,342,45] newacting [303,342,45] 2014-08-13 18:06:07.296595 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296597 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.296600 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296612 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45]/[313,318,45] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'442 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.296616 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296641 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [303,342,45](303) acting [313,318,45](313)): not rw, up_thru 194489 up_from 192981 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [303,342,45](303) acting [313,318,45](313)) : primary up 192981-194489 does not include interval 2014-08-13 18:06:07.296652 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:07.296658 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194491/194488) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [303,342,45](303) acting [313,318,45](313)) 2014-08-13 18:06:07.296674 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.296675 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] up [303,342,45] -> [303,342,45], acting [313,318,45] -> [303,342,45], acting_primary 313 -> 303, up_primary 303 -> 303, role -1 -> -1 2014-08-13 18:06:07.296679 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:07.296694 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.296701 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.296708 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.296718 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:07.296724 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.296739 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.296753 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.296767 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.296817 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:07.299593 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306340 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.306353 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194495 2014-08-13 18:06:07.306380 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.306393 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.306399 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.306417 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.306396 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306417 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] [313,318,45] -> [303,342,45], acting primary 313 -> 303 2014-08-13 18:06:07.306440 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306447 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.306461 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.306473 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306479 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.306499 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306530 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [303,342,45]/[303,342,45] -- 303/303 2014-08-13 18:06:07.306551 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.306557 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.306573 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.306589 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.306871 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4425280 don't know epoch, doing nothing 2014-08-13 18:06:07.306882 7f5c7c7e4700 7 osd.4 194495 do_notify osd 54 on 1 PGs 2014-08-13 18:06:07.306913 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:07.306917 7f5c7c7e4700 7 osd.4 194495 do_notify osd 58 on 1 PGs 2014-08-13 18:06:07.306941 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:07.306911 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [303,342,45]/[303,342,45] -- 303/303 2014-08-13 18:06:07.306944 7f5c7c7e4700 7 osd.4 194495 do_notify osd 320 on 1 PGs 2014-08-13 18:06:07.306935 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] flushed 2014-08-13 18:06:07.306964 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44270c0 don't know epoch, doing nothing 2014-08-13 18:06:07.306967 7f5c7c7e4700 7 osd.4 194495 do_notify osd 321 on 1 PGs 2014-08-13 18:06:07.306948 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.306986 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 don't know epoch, doing nothing 2014-08-13 18:06:07.306989 7f5c7c7e4700 7 osd.4 194495 do_notify osd 324 on 1 PGs 2014-08-13 18:06:07.306975 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.306980 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.307010 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:07.307014 7f5c7c7e4700 7 osd.4 194495 do_notify osd 345 on 1 PGs 2014-08-13 18:06:07.307006 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] flushed 2014-08-13 18:06:07.307034 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.307001 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.307053 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.307059 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] flushed 2014-08-13 18:06:07.307092 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 18:06:07.307120 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.307109 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.307125 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.307141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.307202 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:07.307211 7f5c7c7e4700 7 osd.4 194495 do_notify osd 364 on 1 PGs 2014-08-13 18:06:07.307239 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:07.307205 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,313,318] newacting [4,313,318] 2014-08-13 18:06:07.307243 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.20 on 1 PGs 2014-08-13 18:06:07.307267 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4af9ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.307248 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.307273 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.24 on 1 PGs 2014-08-13 18:06:07.307275 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [303,342,45] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.307287 7f5c4e97d700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.307311 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [303,342,45](303) acting [303,342,45](303)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.307349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194491/194492/194492) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 18:06:07.307381 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,342,45] -> [4,313,318], acting [303,342,45] -> [4,313,318], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 18:06:07.307394 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing 2014-08-13 18:06:07.307406 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.25 on 1 PGs 2014-08-13 18:06:07.307404 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.307432 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x449a3c0 don't know epoch, doing nothing 2014-08-13 18:06:07.307435 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.44 on 1 PGs 2014-08-13 18:06:07.307422 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.307456 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:07.307459 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.45 on 1 PGs 2014-08-13 18:06:07.307478 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5c620 don't know epoch, doing nothing 2014-08-13 18:06:07.307481 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.48 on 1 PGs 2014-08-13 18:06:07.307447 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.307502 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:07.307506 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.55 on 1 PGs 2014-08-13 18:06:07.307523 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:07.307526 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.58 on 1 PGs 2014-08-13 18:06:07.307516 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2706 2014-08-13 18:06:07.307545 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:07.307549 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.61 on 1 PGs 2014-08-13 18:06:07.307536 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.307555 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.307569 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4424360 don't know epoch, doing nothing 2014-08-13 18:06:07.307572 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.62 on 1 PGs 2014-08-13 18:06:07.307571 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.307596 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:07.307599 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.65 on 1 PGs 2014-08-13 18:06:07.307592 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.307608 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.307621 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:07.307625 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.285 on 1 PGs 2014-08-13 18:06:07.307646 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing 2014-08-13 18:06:07.307649 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.291 on 1 PGs 2014-08-13 18:06:07.307681 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing 2014-08-13 18:06:07.307685 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.301 on 1 PGs 2014-08-13 18:06:07.307623 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.307706 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4f218c0 don't know epoch, doing nothing 2014-08-13 18:06:07.307710 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.302 on 1 PGs 2014-08-13 18:06:07.307695 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.307729 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:07.307733 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.303 on 1 PGs 2014-08-13 18:06:07.307730 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.307749 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.307779 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.307801 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.307816 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.307831 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.307843 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c59a20 don't know epoch, doing nothing 2014-08-13 18:06:07.307854 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.311 on 1 PGs 2014-08-13 18:06:07.307845 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.307883 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5aec0 don't know epoch, doing nothing 2014-08-13 18:06:07.307886 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.312 on 1 PGs 2014-08-13 18:06:07.307866 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.307904 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5b180 don't know epoch, doing nothing 2014-08-13 18:06:07.307907 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.314 on 1 PGs 2014-08-13 18:06:07.307897 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.307937 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 don't know epoch, doing nothing 2014-08-13 18:06:07.307925 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.307940 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.324 on 2 PGs 2014-08-13 18:06:07.307946 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.307960 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:07.307963 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.330 on 1 PGs 2014-08-13 18:06:07.307969 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,313,318 unchanged 2014-08-13 18:06:07.307984 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing 2014-08-13 18:06:07.307988 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.331 on 1 PGs 2014-08-13 18:06:07.308007 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c594a0 don't know epoch, doing nothing 2014-08-13 18:06:07.308011 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.339 on 1 PGs 2014-08-13 18:06:07.308007 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.308029 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:07.308033 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.340 on 1 PGs 2014-08-13 18:06:07.308029 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.012952 8 0.012360 2014-08-13 18:06:07.308052 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5b440 don't know epoch, doing nothing 2014-08-13 18:06:07.308056 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.342 on 1 PGs 2014-08-13 18:06:07.308051 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.308086 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4fa4fc0 don't know epoch, doing nothing 2014-08-13 18:06:07.308090 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.349 on 1 PGs 2014-08-13 18:06:07.308080 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.308109 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing 2014-08-13 18:06:07.308113 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.350 on 1 PGs 2014-08-13 18:06:07.308132 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:07.308135 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.362 on 1 PGs 2014-08-13 18:06:07.308191 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4fa2260 don't know epoch, doing nothing 2014-08-13 18:06:07.308200 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.365 on 1 PGs 2014-08-13 18:06:07.308221 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.308224 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.367 on 1 PGs 2014-08-13 18:06:07.308335 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5bde0 don't know epoch, doing nothing 2014-08-13 18:06:07.308343 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.368 on 1 PGs 2014-08-13 18:06:07.308520 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5e880 don't know epoch, doing nothing 2014-08-13 18:06:07.308529 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.369 on 1 PGs 2014-08-13 18:06:07.308554 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing 2014-08-13 18:06:07.308557 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.370 on 1 PGs 2014-08-13 18:06:07.308096 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.308720 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:07.308728 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.373 on 1 PGs 2014-08-13 18:06:07.308753 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.308756 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.378 on 1 PGs 2014-08-13 18:06:07.308808 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 18:06:07.308843 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.308850 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.308867 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.308904 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323] newacting [304,323] 2014-08-13 18:06:07.308920 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.308936 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323,4] r=2 lpr=194487 pi=193250-194485/31 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194488 2014-08-13 18:06:07.308969 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [304,323,4](304) acting [304,323,4](304)): not rw, up_thru 194486 up_from 192965 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [304,323,4](304) acting [304,323,4](304) maybe_went_rw) : primary up 192965-194486 includes interval 2014-08-13 18:06:07.308987 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194485/194486/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194486-194487 up [304,323,4](304) acting [304,323,4](304) maybe_went_rw) 2014-08-13 18:06:07.309004 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323,4] -> [304,323], acting [304,323,4] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role 2 -> -1 2014-08-13 18:06:07.309024 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:07.309039 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.309053 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.309070 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 18:06:07.309084 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 18:06:07.309100 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 18:06:07.309114 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:07.309132 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 18:06:07.309148 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 18:06:07.309163 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_role_change 2014-08-13 18:06:07.308602 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000523 0 0.000000 2014-08-13 18:06:07.309187 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.309203 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 18:06:07.309218 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 18:06:07.309224 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.309236 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.309246 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.309265 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.309277 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 18:06:07.309281 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.309302 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.309308 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.309306 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 18:06:07.309324 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.309341 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.309327 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [303,342,45](303) acting [313,318,45](313)) 2014-08-13 18:06:07.309366 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [313,318](313) acting [313,318,45](313) maybe_went_rw) 2014-08-13 18:06:07.309372 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323]/[304,323] -- 304/304 2014-08-13 18:06:07.309394 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.309387 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [313,318](313) acting [313,318](313)) 2014-08-13 18:06:07.309399 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.309405 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,313,318](4) acting [4,313,318](4) maybe_went_rw) 2014-08-13 18:06:07.309415 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.309431 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.309430 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 18:06:07.309452 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,342,45](303) acting [313,318,303](313)) 2014-08-13 18:06:07.309469 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,373]/[304,323,373] -- 304/304 2014-08-13 18:06:07.309469 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,303,313,318,342 down blocked_by {} 2014-08-13 18:06:07.309493 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.309497 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.309513 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.309528 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323,373] newacting [304,323,373] 2014-08-13 18:06:07.309544 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.309558 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323] r=-1 lpr=194488 pi=193250-194487/32 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194491 2014-08-13 18:06:07.309494 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.309587 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [304,323](304) acting [304,323](304)): not rw, up_thru 194488 up_from 192965 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [304,323](304) acting [304,323](304) maybe_went_rw) : primary up 192965-194488 includes interval 2014-08-13 18:06:07.309603 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194488/194488/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194488-194490 up [304,323](304) acting [304,323](304) maybe_went_rw) 2014-08-13 18:06:07.309641 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323] -> [304,323,373], acting [304,323] -> [304,323,373], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 18:06:07.309661 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:07.309682 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.309697 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.309713 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 18:06:07.309728 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 18:06:07.309743 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 18:06:07.309757 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:07.309773 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 18:06:07.309789 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 18:06:07.309804 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 18:06:07.309818 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 18:06:07.309834 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.309873 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,373]/[304,323] -- 304/304 2014-08-13 18:06:07.309898 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.309878 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2707 2014-08-13 18:06:07.309903 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.309908 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.45 2014-08-13 18:06:07.309919 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.309935 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] acting_up_affected newup [304,323,373] newacting [304,323] 2014-08-13 18:06:07.309947 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.309951 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.309970 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373] r=-1 lpr=194491 pi=193250-194490/33 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] set_last_peering_reset 194492 2014-08-13 18:06:07.309967 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.313 2014-08-13 18:06:07.309992 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.318 2014-08-13 18:06:07.310001 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [304,323,373](304) acting [304,323,373](304)): not rw, up_thru 194488 up_from 192965 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [304,323,373](304) acting [304,323,373](304)) : primary up 192965-194488 does not include interval 2014-08-13 18:06:07.310010 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.310019 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194491/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] noting past interval(194491-194491 up [304,323,373](304) acting [304,323,373](304)) 2014-08-13 18:06:07.310029 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.310039 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] up [304,323,373] -> [304,323,373], acting [304,323,373] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 18:06:07.310058 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.310061 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 18:06:07.310078 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.310093 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.310109 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_copy_ops 2014-08-13 18:06:07.310125 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_flush_ops 2014-08-13 18:06:07.310139 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change_cleanup 2014-08-13 18:06:07.310154 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 18:06:07.310178 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit NotTrimming 2014-08-13 18:06:07.310199 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.310196 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter NotTrimming 2014-08-13 18:06:07.310212 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_recovery 2014-08-13 18:06:07.310227 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] clear_recovery_state 2014-08-13 18:06:07.310243 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.310286 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,373]/[304,323] -- 304/304 2014-08-13 18:06:07.310305 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2d6a0 don't know epoch, doing nothing 2014-08-13 18:06:07.310313 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.310315 7f5c7bde3700 7 osd.4 194495 do_notify osd 294 on 1 PGs 2014-08-13 18:06:07.310319 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.310336 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.310353 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.310369 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b12100 don't know epoch, doing nothing 2014-08-13 18:06:07.310373 7f5c7bde3700 7 osd.4 194495 do_notify osd 334 on 1 PGs 2014-08-13 18:06:07.310388 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,373]/[304,323] -- 304/304 2014-08-13 18:06:07.310405 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4f21600 don't know epoch, doing nothing 2014-08-13 18:06:07.310409 7f5c7bde3700 7 osd.4 194495 do_notify osd 346 on 1 PGs 2014-08-13 18:06:07.310411 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.310415 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.310432 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:07.310432 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.310449 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.310437 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.45 on 1 PGs 2014-08-13 18:06:07.310479 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,4]/[304,323] -- 304/304 2014-08-13 18:06:07.310495 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:07.310499 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.303 on 1 PGs 2014-08-13 18:06:07.310501 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.310533 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4494360 don't know epoch, doing nothing 2014-08-13 18:06:07.310537 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.313 on 1 PGs 2014-08-13 18:06:07.310573 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3988dc0 don't know epoch, doing nothing 2014-08-13 18:06:07.310578 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.318 sending saperate messages on 1 PGs 2014-08-13 18:06:07.310599 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5b440 don't know epoch, doing nothing 2014-08-13 18:06:07.310602 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.342 on 1 PGs 2014-08-13 18:06:07.310505 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:07.310622 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.310646 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323] 2014-08-13 18:06:07.310663 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.310679 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,373]/[304,323] r=-1 lpr=194492 pi=193250-194491/34 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194495 2014-08-13 18:06:07.310709 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [304,323,373](304) acting [304,323](304)): not rw, up_thru 194492 up_from 192965 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [304,323,373](304) acting [304,323](304) maybe_went_rw) : primary up 192965-194492 includes interval 2014-08-13 18:06:07.310731 7f5c4e77b700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.310726 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194491/194492/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] noting past interval(194492-194494 up [304,323,373](304) acting [304,323](304) maybe_went_rw) 2014-08-13 18:06:07.310745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] up [304,323,373] -> [304,323,4], acting [304,323] -> [304,323], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> -1 2014-08-13 18:06:07.310776 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.310765 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 18:06:07.310782 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.310783 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.310798 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.310813 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.310816 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.310819 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ad640 pg_query(22.2b9 epoch 194495) v3 2014-08-13 18:06:07.310816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_copy_ops 2014-08-13 18:06:07.310832 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_flush_ops 2014-08-13 18:06:07.310860 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.328 epoch 194495 2014-08-13 18:06:07.310849 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change_cleanup 2014-08-13 18:06:07.310865 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ad640 2014-08-13 18:06:07.310866 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] on_change 2014-08-13 18:06:07.310764 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 18:06:07.310884 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit NotTrimming 2014-08-13 18:06:07.310880 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 328 2014-08-13 18:06:07.310901 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.310902 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter NotTrimming 2014-08-13 18:06:07.310909 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.310929 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.310933 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.310919 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] cancel_recovery 2014-08-13 18:06:07.310929 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.310937 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] clear_recovery_state 2014-08-13 18:06:07.310954 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.310958 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.310960 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5e80 pg_notify(22.1d2(1) epoch 194495) v5 2014-08-13 18:06:07.310949 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395] newacting [17,395] 2014-08-13 18:06:07.310959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.310974 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.310980 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_activate_map 2014-08-13 18:06:07.311003 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.331 2014-08-13 18:06:07.310991 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,4,395] r=1 lpr=194487 pi=193188-194485/37 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194488 2014-08-13 18:06:07.311007 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5e80 2014-08-13 18:06:07.311049 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311052 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311040 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [17,4,395](17) acting [17,4,395](17)): not rw, up_thru 194486 up_from 192970 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [17,4,395](17) acting [17,4,395](17) maybe_went_rw) : primary up 192970-194486 includes interval 2014-08-13 18:06:07.311060 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] update_heartbeat_peers unchanged 2014-08-13 18:06:07.311085 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311089 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311078 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] take_waiters 2014-08-13 18:06:07.311091 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0a80 pg_notify(22.39d(34) epoch 194495) v5 2014-08-13 18:06:07.311079 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194485/194486/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194486-194487 up [17,4,395](17) acting [17,4,395](17) maybe_went_rw) 2014-08-13 18:06:07.311097 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Reset 0.385511 10 0.001819 2014-08-13 18:06:07.311132 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.320 2014-08-13 18:06:07.311121 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started 2014-08-13 18:06:07.311106 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,4,395] -> [17,395], acting [17,4,395] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role 1 -> -1 2014-08-13 18:06:07.311136 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0a80 2014-08-13 18:06:07.311140 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Start 2014-08-13 18:06:07.311139 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:07.311158 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: transitioning to Stray 2014-08-13 18:06:07.311189 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311193 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311188 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Start 0.000049 0 0.000000 2014-08-13 18:06:07.311170 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.311210 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Started/Stray 2014-08-13 18:06:07.311223 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311212 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.311229 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311231 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5780 pg_notify(24.9b epoch 194495) v5 2014-08-13 18:06:07.311232 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.311232 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 18:06:07.311253 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.311270 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.367 2014-08-13 18:06:07.311274 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5780 2014-08-13 18:06:07.311255 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 18:06:07.311290 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 18:06:07.311322 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311326 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311313 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:07.311348 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311351 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311353 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5400 pg_notify(22.619 epoch 194495) v5 2014-08-13 18:06:07.311366 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.311332 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 18:06:07.311389 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.65 2014-08-13 18:06:07.311392 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5400 2014-08-13 18:06:07.311375 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 18:06:07.311424 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311428 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311404 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_role_change 2014-08-13 18:06:07.311415 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.311442 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.311447 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311435 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.311450 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311453 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deec80 pg_notify(22.3d9(47) epoch 194495) v5 2014-08-13 18:06:07.311449 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.311466 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.311490 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.373 2014-08-13 18:06:07.311481 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.311494 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deec80 2014-08-13 18:06:07.311499 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.243759 2 0.000616 2014-08-13 18:06:07.311519 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.311539 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311542 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311534 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.243802 0 0.000000 2014-08-13 18:06:07.311562 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311565 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311552 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.243836 0 0.000000 2014-08-13 18:06:07.311567 7f5c835ef700 20 osd.4 194495 _dispatch 0x4def380 pg_notify(22.48e(34) epoch 194495) v5 2014-08-13 18:06:07.311570 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Started 0.243916 0 0.000000 2014-08-13 18:06:07.311591 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.311609 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.311613 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4def380 2014-08-13 18:06:07.311605 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194486 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.311621 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.311650 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311639 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.311653 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311654 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340] newacting [340] 2014-08-13 18:06:07.311668 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.311686 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311689 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311681 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [4,340] r=0 lpr=194488 crt=0'0 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.311691 7f5c835ef700 20 osd.4 194495 _dispatch 0x5249f80 pg_notify(22.775 epoch 194495) v5 2014-08-13 18:06:07.311727 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.56 2014-08-13 18:06:07.311712 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,340](4) acting [4,340](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.311729 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5249f80 2014-08-13 18:06:07.311731 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194485/194485/194485) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] noting past interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:07.311748 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] up [4,340] -> [340], acting [4,340] -> [340], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 18:06:07.311767 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311771 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311766 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.311782 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.311800 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311803 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311805 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2f40 pg_notify(2.2a(1) epoch 194495) v5 2014-08-13 18:06:07.311798 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.311452 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 18:06:07.311817 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.311833 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.311845 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.324 2014-08-13 18:06:07.311828 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 18:06:07.311849 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2f40 2014-08-13 18:06:07.311850 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.311855 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.311864 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.311880 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.311897 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311901 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311893 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.311898 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 18:06:07.311909 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.311922 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311925 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311925 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.311928 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bc6780 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.311925 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.311940 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.311946 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.311932 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.311949 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311952 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311941 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.311951 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.311959 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.311980 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.311944 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] flushed 2014-08-13 18:06:07.311985 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.311975 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.311972 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.311990 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4440 pg_notify(22.113 epoch 194495) v5 2014-08-13 18:06:07.311990 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.312004 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.312028 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.339 2014-08-13 18:06:07.312018 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.312032 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4440 2014-08-13 18:06:07.312014 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395]/[17,395] -- 17/17 2014-08-13 18:06:07.312044 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312035 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312071 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312074 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312067 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.312088 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312097 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312100 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312051 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.312102 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3640 pg_notify(2.2a(2) epoch 194495) v5 2014-08-13 18:06:07.312092 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312108 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312105 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.312124 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312140 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.370 2014-08-13 18:06:07.312143 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3640 2014-08-13 18:06:07.312151 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [340]/[340] -- 340/340 2014-08-13 18:06:07.312135 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312184 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312189 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312204 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312223 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312218 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312233 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312216 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395,382]/[17,395,382] -- 17/17 2014-08-13 18:06:07.312246 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312256 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.312272 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312274 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312276 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312279 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4ec0 pg_notify(22.f3(26) epoch 194495) v5 2014-08-13 18:06:07.312267 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.312278 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312287 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.312295 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312304 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395,382] newacting [17,395,382] 2014-08-13 18:06:07.312311 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [324,370] newacting [324,370] 2014-08-13 18:06:07.312331 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.311 2014-08-13 18:06:07.312325 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.312341 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4ec0 2014-08-13 18:06:07.312331 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.312349 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [340] r=-1 lpr=194488 pi=194485-194487/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.312342 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395] r=-1 lpr=194488 pi=193188-194487/38 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194491 2014-08-13 18:06:07.312376 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [340](340) acting [340](340)): not rw, up_thru 194488 up_from 192944 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) : primary up 192944-194488 includes interval 2014-08-13 18:06:07.312394 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312398 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312393 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194488/194488/194488) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.312396 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [17,395](17) acting [17,395](17)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [17,395](17) acting [17,395](17) maybe_went_rw) : primary up 192970-194489 includes interval 2014-08-13 18:06:07.312410 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] up [340] -> [324,370], acting [340] -> [324,370], acting_primary 340 -> 324, up_primary 340 -> 324, role -1 -> -1 2014-08-13 18:06:07.312428 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312432 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312434 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2140 pg_notify(22.707(38) epoch 194495) v5 2014-08-13 18:06:07.312430 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.312420 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194488/194488/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194488-194490 up [17,395](17) acting [17,395](17) maybe_went_rw) 2014-08-13 18:06:07.312446 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.312451 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,395] -> [17,395,382], acting [17,395] -> [17,395,382], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 18:06:07.312462 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.312476 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.312480 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2140 2014-08-13 18:06:07.312474 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:07.312479 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.312494 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.312493 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.312511 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.312512 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.312533 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312537 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312526 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.312532 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 18:06:07.312542 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.312560 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312563 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312565 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351f80 pg_notify(22.619(36) epoch 194495) v5 2014-08-13 18:06:07.312561 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.312550 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 18:06:07.312579 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 18:06:07.312578 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] [340] -> [324,370], acting primary 340 -> 324 2014-08-13 18:06:07.312606 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.314 2014-08-13 18:06:07.312610 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351f80 2014-08-13 18:06:07.312605 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.312603 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:07.312621 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.312626 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 18:06:07.312637 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312655 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312659 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312646 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 18:06:07.312673 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 18:06:07.312688 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312675 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.312691 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312694 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312694 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf16c0 pg_notify(22.f3(1) epoch 194495) v5 2014-08-13 18:06:07.312690 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 18:06:07.312699 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312710 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312716 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312737 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.312 2014-08-13 18:06:07.312740 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf16c0 2014-08-13 18:06:07.312734 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312756 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,395,382]/[17,395] -- 17/17 2014-08-13 18:06:07.312781 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312785 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312770 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.312789 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312800 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312804 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312807 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312793 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312809 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5080 pg_notify(22.619(36) epoch 194495) v5 2014-08-13 18:06:07.312806 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.312812 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312824 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.312831 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312848 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.48 2014-08-13 18:06:07.312852 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5080 2014-08-13 18:06:07.312846 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] acting_up_affected newup [17,395,382] newacting [17,395] 2014-08-13 18:06:07.312863 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.312867 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [324,370]/[324,370] -- 324/324 2014-08-13 18:06:07.312886 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312890 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.312880 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382] r=-1 lpr=194491 pi=193188-194490/39 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] set_last_peering_reset 194492 2014-08-13 18:06:07.312904 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.312919 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312924 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312924 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.312933 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [17,395,382](17) acting [17,395,382](17)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [17,395,382](17) acting [17,395,382](17)) : primary up 192970-194489 does not include interval 2014-08-13 18:06:07.312956 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.312959 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.312961 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6040 pg_notify(22.22c(28) epoch 194495) v5 2014-08-13 18:06:07.312952 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:07.312971 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.312953 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194491/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] noting past interval(194491-194491 up [17,395,382](17) acting [17,395,382](17)) 2014-08-13 18:06:07.312976 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.313000 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.368 2014-08-13 18:06:07.312990 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.313003 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6040 2014-08-13 18:06:07.312981 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] up [17,395,382] -> [17,395,382], acting [17,395,382] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 18:06:07.313009 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340] newacting [4,340] 2014-08-13 18:06:07.313015 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 18:06:07.313025 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.313042 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [324,370] r=-1 lpr=194491 pi=194485-194490/2 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.313038 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.313061 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313066 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313059 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.313072 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [324,370](324) acting [324,370](324)): not rw, up_thru 194491 up_from 192951 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) : primary up 192951-194491 includes interval 2014-08-13 18:06:07.313098 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313100 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313103 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352680 pg_notify(24.9b(38) epoch 194495) v5 2014-08-13 18:06:07.313089 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194491/194491/194491) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.313088 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_copy_ops 2014-08-13 18:06:07.313109 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [324,370] -> [4,340], acting [324,370] -> [4,340], acting_primary 324 -> 4, up_primary 324 -> 4, role -1 -> 0 2014-08-13 18:06:07.313113 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_flush_ops 2014-08-13 18:06:07.313128 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.313147 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.350 2014-08-13 18:06:07.313150 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352680 2014-08-13 18:06:07.313131 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change_cleanup 2014-08-13 18:06:07.313145 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.313158 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 18:06:07.313161 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.313181 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit NotTrimming 2014-08-13 18:06:07.313186 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:177958 2014-08-13 18:06:07.313207 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313211 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313203 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.313200 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter NotTrimming 2014-08-13 18:06:07.313219 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.313236 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313240 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313242 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6ac0 pg_notify(22.775(1) epoch 194495) v5 2014-08-13 18:06:07.313234 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.313232 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_recovery 2014-08-13 18:06:07.313251 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.313254 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] clear_recovery_state 2014-08-13 18:06:07.313266 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.313286 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.313291 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6ac0 2014-08-13 18:06:07.313277 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.313285 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.313301 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.313331 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313320 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.313335 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313338 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.313321 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,395,382]/[17,395] -- 17/17 2014-08-13 18:06:07.313360 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313363 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313353 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.313365 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0c40 pg_notify(22.22c(1) epoch 194495) v5 2014-08-13 18:06:07.313362 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.313369 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.313372 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.313385 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.313405 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.302 2014-08-13 18:06:07.313393 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.313409 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0c40 2014-08-13 18:06:07.313401 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.313412 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.313418 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.313433 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.313449 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313452 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313446 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.313462 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.313476 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313480 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313483 7f5c835ef700 20 osd.4 194495 _dispatch 0x4de9dc0 pg_notify(22.1d2(30) epoch 194495) v5 2014-08-13 18:06:07.313467 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,395,382]/[17,395] -- 17/17 2014-08-13 18:06:07.313480 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.313495 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.313500 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.313503 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340 unchanged 2014-08-13 18:06:07.313526 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.330 2014-08-13 18:06:07.313529 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4de9dc0 2014-08-13 18:06:07.313520 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.313518 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.313538 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.313538 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001948 8 0.001731 2014-08-13 18:06:07.313569 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313560 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.313573 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313576 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.313572 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,4,395]/[17,395] -- 17/17 2014-08-13 18:06:07.313596 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.313590 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.313610 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313614 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313601 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:07.313617 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deaf40 pg_notify(24.9b(1) epoch 194495) v5 2014-08-13 18:06:07.313611 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000036 0 0.000000 2014-08-13 18:06:07.313621 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.313634 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.313639 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,395] 2014-08-13 18:06:07.313660 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.365 2014-08-13 18:06:07.313664 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deaf40 2014-08-13 18:06:07.313652 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.313658 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.313670 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.313678 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,395,382]/[17,395] r=-1 lpr=194492 pi=193188-194491/40 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194495 2014-08-13 18:06:07.313687 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.313708 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313711 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313705 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.313711 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [17,395,382](17) acting [17,395](17)): not rw, up_thru 194492 up_from 192970 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [17,395,382](17) acting [17,395](17) maybe_went_rw) : primary up 192970-194492 includes interval 2014-08-13 18:06:07.313732 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313735 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313738 7f5c835ef700 20 osd.4 194495 _dispatch 0x5170000 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.313726 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.313730 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194491/194492/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] noting past interval(194492-194494 up [17,395,382](17) acting [17,395](17) maybe_went_rw) 2014-08-13 18:06:07.313747 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.313753 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.313756 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313759 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:07.313751 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] up [17,395,382] -> [17,4,395], acting [17,395] -> [17,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> -1 2014-08-13 18:06:07.313764 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 18:06:07.313784 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313787 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313774 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 18:06:07.313789 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f1c0 pg_notify(22.1d2 epoch 194495) v5 2014-08-13 18:06:07.313783 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.313792 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.313804 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:177959 2014-08-13 18:06:07.313811 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.313828 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.349 2014-08-13 18:06:07.313831 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f1c0 2014-08-13 18:06:07.313823 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.324 2014-08-13 18:06:07.313832 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_copy_ops 2014-08-13 18:06:07.313843 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.313852 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_flush_ops 2014-08-13 18:06:07.313875 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313862 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.370 2014-08-13 18:06:07.313881 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313870 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change_cleanup 2014-08-13 18:06:07.313884 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.313887 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] on_change 2014-08-13 18:06:07.313901 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313906 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313909 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6740 pg_notify(22.775 epoch 194495) v5 2014-08-13 18:06:07.313902 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.313906 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit NotTrimming 2014-08-13 18:06:07.313925 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter NotTrimming 2014-08-13 18:06:07.313946 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.353 2014-08-13 18:06:07.313949 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6740 2014-08-13 18:06:07.313944 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] cancel_recovery 2014-08-13 18:06:07.313966 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.313961 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] clear_recovery_state 2014-08-13 18:06:07.313982 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.313985 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.313980 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.314006 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.314000 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_activate_map 2014-08-13 18:06:07.321792 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.321805 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee6540 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.321819 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.321825 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.321828 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.321830 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.314020 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.321870 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.321873 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.321863 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.321875 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1180 pg_notify(24.30b(40) epoch 194495) v5 2014-08-13 18:06:07.321879 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.321876 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 18:06:07.321921 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.10 2014-08-13 18:06:07.321925 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1180 2014-08-13 18:06:07.321911 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.321914 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] take_waiters 2014-08-13 18:06:07.321930 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [26,30,340] newacting [26,30,340] 2014-08-13 18:06:07.321935 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Reset 0.451745 10 0.002681 2014-08-13 18:06:07.321949 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.321959 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started 2014-08-13 18:06:07.321977 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.321980 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.321967 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340]/[4] r=0 lpr=194487 pi=193125-194485/52 crt=193305'14233 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.321978 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Start 2014-08-13 18:06:07.321997 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: transitioning to Stray 2014-08-13 18:06:07.322013 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322017 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322002 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [26,30,340](26) acting [4](4)) : acting set is too small 2014-08-13 18:06:07.322019 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351a40 pg_notify(22.22c epoch 194495) v5 2014-08-13 18:06:07.322020 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194486/194486) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] noting past interval(194486-194487 up [26,30,340](26) acting [4](4)) 2014-08-13 18:06:07.322019 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Start 0.000041 0 0.000000 2014-08-13 18:06:07.322040 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] up [26,30,340] -> [26,30,340], acting [4] -> [26,30,340], acting_primary 4 -> 26, up_primary 26 -> 26, role 0 -> -1 2014-08-13 18:06:07.322058 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.301 2014-08-13 18:06:07.322044 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Started/Stray 2014-08-13 18:06:07.322062 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351a40 2014-08-13 18:06:07.322061 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.322069 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.322079 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.322101 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322105 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322089 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.322095 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.322123 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322111 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.322126 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322129 7f5c835ef700 20 osd.4 194495 _dispatch 0x524ec80 pg_notify(22.362 epoch 194495) v5 2014-08-13 18:06:07.322127 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.322144 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.322181 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.58 2014-08-13 18:06:07.322184 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524ec80 2014-08-13 18:06:07.322160 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.322217 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322221 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322222 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.322211 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.322228 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.322240 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322243 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322245 7f5c835ef700 20 osd.4 194495 _dispatch 0x5154140 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.322252 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.322257 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.322246 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.322260 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322262 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322262 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.322282 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322285 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322287 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a45cc0 pg_notify(24.3d(1) epoch 194495) v5 2014-08-13 18:06:07.322278 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.322294 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.322293 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 18:06:07.322310 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.322324 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322331 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.20 2014-08-13 18:06:07.322335 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a45cc0 2014-08-13 18:06:07.322325 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.322330 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.322340 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.322349 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44] newacting [367,44] 2014-08-13 18:06:07.322357 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.322377 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322380 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322375 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322401 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322403 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322405 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf0e00 pg_notify(24.9b(38) epoch 194495) v5 2014-08-13 18:06:07.322368 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.322441 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.291 2014-08-13 18:06:07.322426 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322444 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf0e00 2014-08-13 18:06:07.322431 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.289717 3 0.000124 2014-08-13 18:06:07.322454 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322459 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322461 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.289787 0 0.000000 2014-08-13 18:06:07.322479 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322483 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322476 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.322482 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.322492 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322512 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322515 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322517 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df24c0 pg_notify(22.f3 epoch 194495) v5 2014-08-13 18:06:07.322499 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.322531 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322554 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.285 2014-08-13 18:06:07.322558 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df24c0 2014-08-13 18:06:07.322559 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322540 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322564 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322565 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.322581 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.322593 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322597 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322595 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44] newacting [367,44] 2014-08-13 18:06:07.322597 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322616 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322619 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322621 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4280 pg_notify(22.1d2 epoch 194495) v5 2014-08-13 18:06:07.322612 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.322628 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.322635 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322657 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.322657 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322661 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4280 2014-08-13 18:06:07.322662 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322662 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [367,44,4](367) acting [367,44,4](367)): not rw, up_thru 194485 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [367,44,4](367) acting [367,44,4](367) maybe_went_rw) : primary up 192976-194485 includes interval 2014-08-13 18:06:07.322679 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.322693 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322696 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322689 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [367,44,4](367) acting [367,44,4](367) maybe_went_rw) 2014-08-13 18:06:07.322695 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322714 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322717 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322719 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df01c0 pg_notify(2.2a epoch 194495) v5 2014-08-13 18:06:07.322710 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44,4] -> [367,44], acting [367,44,4] -> [367,44], acting_primary 367 -> 367, up_primary 367 -> 367, role 2 -> -1 2014-08-13 18:06:07.322733 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322731 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.322753 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.340 2014-08-13 18:06:07.322755 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322757 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df01c0 2014-08-13 18:06:07.322754 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.322760 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322771 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.322789 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322778 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.322795 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322790 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.322799 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322815 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322818 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322820 7f5c835ef700 20 osd.4 194495 _dispatch 0x51569c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.322814 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.322827 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.322832 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.322834 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322836 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322830 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.322838 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322855 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322858 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322859 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322860 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3f00 pg_notify(22.113 epoch 194495) v5 2014-08-13 18:06:07.322852 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.322865 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322871 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.322882 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.322899 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.45 2014-08-13 18:06:07.322902 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3f00 2014-08-13 18:06:07.322890 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.322900 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.322890 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] flushed 2014-08-13 18:06:07.322913 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.322941 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322944 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322938 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.322941 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.322959 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.322963 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.322967 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.322969 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4d00 pg_notify(22.619(1) epoch 194495) v5 2014-08-13 18:06:07.322961 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.322964 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.322979 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.322980 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.323008 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.44 2014-08-13 18:06:07.322998 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323011 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4d00 2014-08-13 18:06:07.322999 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323045 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323049 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323043 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[26,30,340] -- 26/26 2014-08-13 18:06:07.323067 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323068 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323070 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323074 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1c00 pg_notify(22.113(34) epoch 194495) v5 2014-08-13 18:06:07.323073 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323090 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:07.323114 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.55 2014-08-13 18:06:07.323117 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1c00 2014-08-13 18:06:07.323106 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323083 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 18:06:07.323125 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323124 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.323153 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323156 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323132 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323160 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323177 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323191 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323194 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323196 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f540 pg_notify(24.3d epoch 194495) v5 2014-08-13 18:06:07.323210 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44]/[367,44] -- 367/367 2014-08-13 18:06:07.323229 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323231 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.362 2014-08-13 18:06:07.323225 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.323245 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f540 2014-08-13 18:06:07.323245 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.323242 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323263 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Reset 0.747782 10 0.000908 2014-08-13 18:06:07.323281 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323284 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323273 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323286 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.323304 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323307 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323296 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323309 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3480 pg_notify(22.113(34) epoch 194495) v5 2014-08-13 18:06:07.323304 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.323321 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.323348 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.24 2014-08-13 18:06:07.323351 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3480 2014-08-13 18:06:07.323340 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Start 0.000037 0 0.000000 2014-08-13 18:06:07.323334 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 18:06:07.323365 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323361 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.323370 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323386 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323390 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323382 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.323385 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323400 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.323401 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44,342] newacting [367,44,342] 2014-08-13 18:06:07.323423 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323426 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323428 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3d40 pg_notify(22.481(42) epoch 194495) v5 2014-08-13 18:06:07.323418 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.323436 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.323469 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.323472 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3d40 2014-08-13 18:06:07.323465 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [367,44](367) acting [367,44](367)): not rw, up_thru 194488 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [367,44](367) acting [367,44](367) maybe_went_rw) : primary up 192976-194488 includes interval 2014-08-13 18:06:07.323483 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [367,44](367) acting [367,44](367) maybe_went_rw) 2014-08-13 18:06:07.323511 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323514 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323500 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44] -> [367,44,342], acting [367,44] -> [367,44,342], acting_primary 367 -> 367, up_primary 367 -> 367, role -1 -> -1 2014-08-13 18:06:07.323520 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.323519 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.323532 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323535 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323537 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4ec0 pg_notify(22.39d(34) epoch 194495) v5 2014-08-13 18:06:07.323535 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.323551 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.323582 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.320 2014-08-13 18:06:07.323585 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4ec0 2014-08-13 18:06:07.323574 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.323591 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.323606 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.323622 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323629 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323621 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.323630 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing 2014-08-13 18:06:07.323640 7f5c7c7e4700 7 osd.4 194495 do_notify osd 26 sending separate messages 2014-08-13 18:06:07.323638 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.323658 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323661 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323663 7f5c835ef700 20 osd.4 194495 _dispatch 0x4deeac0 pg_notify(22.39d epoch 194495) v5 2014-08-13 18:06:07.323654 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.323669 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.323694 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab69e0 don't know epoch, doing nothing 2014-08-13 18:06:07.323684 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.323698 7f5c7c7e4700 7 osd.4 194495 do_notify osd 304 on 1 PGs 2014-08-13 18:06:07.323704 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.373 2014-08-13 18:06:07.323707 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4deeac0 2014-08-13 18:06:07.323703 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323722 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 don't know epoch, doing nothing 2014-08-13 18:06:07.323726 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.324 on 1 PGs 2014-08-13 18:06:07.323740 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323743 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323748 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:07.323752 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.340 on 1 PGs 2014-08-13 18:06:07.323741 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 18:06:07.323761 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323762 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323766 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323768 7f5c835ef700 20 osd.4 194495 _dispatch 0x21ad820 pg_query(22.226 epoch 194495) v3 2014-08-13 18:06:07.323771 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing 2014-08-13 18:06:07.323775 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.370 on 1 PGs 2014-08-13 18:06:07.323766 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323784 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323805 7f5c835ef700 7 osd.4 194495 handle_pg_query from osd.376 epoch 194495 2014-08-13 18:06:07.323809 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x21ad820 2014-08-13 18:06:07.323801 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323821 7f5c835ef700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 376 2014-08-13 18:06:07.323834 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 18:06:07.323853 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323857 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323877 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323881 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323872 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323887 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.323900 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323903 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323905 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1500 pg_notify(22.22c(30) epoch 194495) v5 2014-08-13 18:06:07.323919 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,342]/[367,44,342] -- 367/367 2014-08-13 18:06:07.323942 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.324 2014-08-13 18:06:07.323949 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1500 2014-08-13 18:06:07.323949 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.323963 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.323986 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.323989 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.323984 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.323976 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_advance_map [374,31]/[374,31] -- 374/374 2014-08-13 18:06:07.323999 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.324011 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.324011 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324017 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324020 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2a00 pg_notify(24.3d epoch 194495) v5 2014-08-13 18:06:07.324019 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.324037 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.324057 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.25 2014-08-13 18:06:07.324060 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2a00 2014-08-13 18:06:07.324057 7f5c4e579700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.324053 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.324033 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 18:06:07.324083 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.324076 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.255633 2 0.000286 2014-08-13 18:06:07.324374 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324377 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324376 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.324389 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324391 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324392 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf40c0 pg_notify(22.113(1) epoch 194495) v5 2014-08-13 18:06:07.324387 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.255953 0 0.000000 2014-08-13 18:06:07.324399 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.255980 0 0.000000 2014-08-13 18:06:07.324416 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.324417 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf40c0 2014-08-13 18:06:07.324414 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Started 0.256036 0 0.000000 2014-08-13 18:06:07.324426 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.324440 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324442 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324436 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194486 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.324446 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194488 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.324453 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324456 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324458 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0540 pg_notify(22.775(1) epoch 194495) v5 2014-08-13 18:06:07.324456 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194488 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.324465 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194488 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [374,31] newacting [374,31] 2014-08-13 18:06:07.324088 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.324480 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.324482 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0540 2014-08-13 18:06:07.324475 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194488 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.324476 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.324485 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [4,374,31] r=0 lpr=194488 pi=193306-194484/28 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.324487 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [367,44,4] newacting [367,44,4] 2014-08-13 18:06:07.324504 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324498 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.324508 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324506 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,374,31](4) acting [4,374,31](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.324509 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,342] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.324519 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324521 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324522 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352d80 pg_notify(24.9b epoch 194495) v5 2014-08-13 18:06:07.324516 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194485/194485/194485) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] noting past interval(194485-194487 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) 2014-08-13 18:06:07.324528 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [367,44,342](367) acting [367,44,342](367)): not rw, up_thru 194491 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [367,44,342](367) acting [367,44,342](367) maybe_went_rw) : primary up 192976-194491 includes interval 2014-08-13 18:06:07.324527 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] up [4,374,31] -> [374,31], acting [4,374,31] -> [374,31], acting_primary 4 -> 374, up_primary 4 -> 374, role 0 -> -1 2014-08-13 18:06:07.324544 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.378 2014-08-13 18:06:07.324546 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352d80 2014-08-13 18:06:07.324541 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.324539 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [367,44,342](367) acting [367,44,342](367) maybe_went_rw) 2014-08-13 18:06:07.324551 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.324553 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] up [367,44,342] -> [367,44,4], acting [367,44,342] -> [367,44,4], acting_primary 367 -> 367, up_primary 367 -> 367, role -1 -> 2 2014-08-13 18:06:07.324566 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324568 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324561 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.324566 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.324571 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.324580 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324582 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324583 7f5c835ef700 20 osd.4 194495 _dispatch 0x5353100 pg_notify(22.3e4 epoch 194495) v5 2014-08-13 18:06:07.324577 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.324566 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.324581 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.324587 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.324592 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.324605 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.318 2014-08-13 18:06:07.324607 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5353100 2014-08-13 18:06:07.324603 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.324613 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.324625 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324627 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324621 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.324638 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324640 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324631 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.324641 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3100 pg_notify(22.1d2(30) epoch 194495) v5 2014-08-13 18:06:07.324643 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.324652 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.324665 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.58 2014-08-13 18:06:07.324670 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3100 2014-08-13 18:06:07.324664 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.324673 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.324681 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.324689 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.324695 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324697 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324696 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.324706 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.324604 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.324721 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324723 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324724 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1a40 pg_notify(22.619 epoch 194495) v5 2014-08-13 18:06:07.324718 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.324728 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [374,31]/[374,31] -- 374/374 2014-08-13 18:06:07.324740 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.324737 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.324746 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.62 2014-08-13 18:06:07.324748 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1a40 2014-08-13 18:06:07.324743 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.324748 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.324753 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.324759 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.324768 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324763 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.324770 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324769 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.324783 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324784 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324786 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0e00 pg_notify(1.2b epoch 194495) v5 2014-08-13 18:06:07.324779 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.324780 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [374,31]/[374,31] -- 374/374 2014-08-13 18:06:07.324791 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.324788 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.324793 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.324807 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.340 2014-08-13 18:06:07.324809 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0e00 2014-08-13 18:06:07.324802 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.324798 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.324812 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.324814 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.324827 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324829 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324825 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.324839 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324840 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324842 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352f40 pg_notify(22.3e4 epoch 194495) v5 2014-08-13 18:06:07.324834 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 18:06:07.324836 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.324845 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.324848 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.324856 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.324863 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.313 2014-08-13 18:06:07.324865 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352f40 2014-08-13 18:06:07.324865 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] acting_up_affected newup [363,20,337] newacting [363,20,337] 2014-08-13 18:06:07.324874 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.324884 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324886 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324883 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [374,31] r=-1 lpr=194488 pi=193306-194487/29 crt=191648'365 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.324885 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.324899 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324901 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324902 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df47c0 pg_notify(22.3e4(36) epoch 194495) v5 2014-08-13 18:06:07.324898 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.324900 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [374,31](374) acting [374,31](374)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [374,31](374) acting [374,31](374) maybe_went_rw) : primary up 192967-194489 includes interval 2014-08-13 18:06:07.324909 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.002428 8 0.002073 2014-08-13 18:06:07.324910 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194488/194488/194488) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [374,31](374) acting [374,31](374) maybe_went_rw) 2014-08-13 18:06:07.324926 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.45 2014-08-13 18:06:07.324928 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df47c0 2014-08-13 18:06:07.324922 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.324921 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] up [374,31] -> [363,20,337], acting [374,31] -> [363,20,337], acting_primary 374 -> 363, up_primary 374 -> 363, role -1 -> -1 2014-08-13 18:06:07.324933 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.324933 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.324949 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324942 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.324951 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324943 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.324952 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.324961 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.324967 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.324972 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.324963 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 2014-08-13 18:06:07.324974 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3b80 pg_notify(24.3d(37) epoch 194495) v5 2014-08-13 18:06:07.324969 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.324977 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.324979 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.324988 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.324996 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.324998 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3b80 2014-08-13 18:06:07.324991 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.324997 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.325007 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.325002 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.325020 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325022 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325017 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] [374,31] -> [363,20,337], acting primary 374 -> 363 2014-08-13 18:06:07.325038 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325031 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.325041 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325043 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352bc0 pg_notify(24.3d(37) epoch 194495) v5 2014-08-13 18:06:07.325042 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.325052 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325066 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.373 2014-08-13 18:06:07.325067 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352bc0 2014-08-13 18:06:07.325072 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.325074 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 18:06:07.325085 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325087 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325089 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325087 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325096 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325106 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325108 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325109 7f5c835ef700 20 osd.4 194495 _dispatch 0x524d400 pg_notify(1.2b(1) epoch 194495) v5 2014-08-13 18:06:07.325105 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325103 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_advance_map [310,21]/[310,21] -- 310/310 2014-08-13 18:06:07.325120 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325129 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.324 2014-08-13 18:06:07.325131 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524d400 2014-08-13 18:06:07.325127 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 18:06:07.325138 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325131 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.325140 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325153 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325155 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325150 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325158 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325170 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325172 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325173 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5780 pg_notify(1.2b(2) epoch 194495) v5 2014-08-13 18:06:07.325144 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.325176 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.325193 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.370 2014-08-13 18:06:07.325195 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5780 2014-08-13 18:06:07.325190 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [363,20,337]/[363,20,337] -- 363/363 2014-08-13 18:06:07.325192 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.289851 2 0.000214 2014-08-13 18:06:07.325204 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325207 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325205 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.325216 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325218 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325216 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325218 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.289887 0 0.000000 2014-08-13 18:06:07.325230 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325232 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325226 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325233 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4280 pg_notify(22.3e4(36) epoch 194495) v5 2014-08-13 18:06:07.325231 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.289909 0 0.000000 2014-08-13 18:06:07.325244 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 18:06:07.325255 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.325244 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Started 0.289960 0 0.000000 2014-08-13 18:06:07.325257 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325257 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4280 2014-08-13 18:06:07.325259 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.325269 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194486 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.325281 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325283 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325279 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194488 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.325297 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325290 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194488 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325298 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325300 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0000 pg_notify(22.f3(26) epoch 194495) v5 2014-08-13 18:06:07.325259 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325300 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194488 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [310,21] newacting [310,21] 2014-08-13 18:06:07.325304 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325310 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194488 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.325323 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.325314 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,374,31] newacting [4,374,31] 2014-08-13 18:06:07.325325 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0000 2014-08-13 18:06:07.325320 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [4,310,21] r=0 lpr=194488 pi=193306-194484/32 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.325325 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.325335 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [363,20,337] r=-1 lpr=194491 pi=193306-194490/30 crt=191648'365 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.325328 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.325346 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325348 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325342 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,310,21](4) acting [4,310,21](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.325363 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325355 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [363,20,337](363) acting [363,20,337](363)): not rw, up_thru 194491 up_from 192979 last_epoch_clean 194484 generate_past_intervals interval(194491-194494 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) : primary up 192979-194491 includes interval 2014-08-13 18:06:07.325365 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325357 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194485/194485/194485) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] noting past interval(194485-194487 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) 2014-08-13 18:06:07.325367 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df39c0 pg_notify(22.3e4(1) epoch 194495) v5 2014-08-13 18:06:07.325366 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194491/194491/194491) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 18:06:07.325368 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] up [4,310,21] -> [310,21], acting [4,310,21] -> [310,21], acting_primary 4 -> 310, up_primary 4 -> 310, role 0 -> -1 2014-08-13 18:06:07.325380 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.325377 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [363,20,337] -> [4,374,31], acting [363,20,337] -> [4,374,31], acting_primary 363 -> 4, up_primary 363 -> 4, role -1 -> 0 2014-08-13 18:06:07.325393 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.325395 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df39c0 2014-08-13 18:06:07.325389 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.325390 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.325400 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.325401 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.325416 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.325410 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.325418 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.325411 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.325419 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.325422 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2872 2014-08-13 18:06:07.325428 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.325431 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325437 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.325441 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.325448 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.325450 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.325457 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.325458 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325467 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325466 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.325476 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325477 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.325484 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.325485 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.325492 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.325494 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325502 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.325503 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.325511 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.325511 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.325521 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.325530 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.325521 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.325538 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.325543 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.325547 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.325554 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325555 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.325566 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.325575 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325578 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [310,21]/[310,21,15] -- 310/310 2014-08-13 18:06:07.325594 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325590 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.325598 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325601 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,31,374 unchanged 2014-08-13 18:06:07.325607 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325611 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.325616 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] acting_up_affected newup [310,21] newacting [310,21,15] 2014-08-13 18:06:07.325621 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001195 8 0.001382 2014-08-13 18:06:07.325626 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.325633 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.325635 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'417 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.325642 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.325650 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.325660 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000 2014-08-13 18:06:07.325661 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [310,21](310) acting [310,21](310)): not rw, up_thru 194482 up_from 192972 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [310,21](310) acting [310,21](310)) : primary up 192972-194482 does not include interval 2014-08-13 18:06:07.325671 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.325672 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194488/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [310,21](310) acting [310,21](310)) 2014-08-13 18:06:07.325681 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.325683 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] up [310,21] -> [310,21], acting [310,21] -> [310,21,15], acting_primary 310 -> 310, up_primary 310 -> 310, role -1 -> -1 2014-08-13 18:06:07.325691 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.325700 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325695 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.325710 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 18:06:07.325710 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.325722 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [374,31](374) acting [374,31](374) maybe_went_rw) 2014-08-13 18:06:07.325724 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.325734 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.325733 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,374,31](4) acting [4,374,31](4) maybe_went_rw) 2014-08-13 18:06:07.325743 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.325744 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 18:06:07.325752 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.325755 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [363,20,337](363) acting [374,31,20](374)) 2014-08-13 18:06:07.325761 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.325765 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,20,31,337,363,374 down blocked_by {} 2014-08-13 18:06:07.325776 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.325776 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.325787 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.325788 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2873 2014-08-13 18:06:07.325797 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.325798 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.20 2014-08-13 18:06:07.325806 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.325809 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.31 2014-08-13 18:06:07.325821 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.337 2014-08-13 18:06:07.325821 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325831 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.363 2014-08-13 18:06:07.325840 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.374 2014-08-13 18:06:07.325842 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [310,21]/[310,21,15] -- 310/310 2014-08-13 18:06:07.325854 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325851 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.325857 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325860 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.325867 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325877 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.325919 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.325911 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [303,15,342]/[310,21,15] -- 303/310 2014-08-13 18:06:07.325922 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325924 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.325942 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.325951 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,15,342] newacting [310,21,15] 2014-08-13 18:06:07.325952 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 18:06:07.325967 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.325960 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.325970 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [310,21]/[310,21,15] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'417 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.325971 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.325984 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.325992 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.326001 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.256825 2 0.000177 2014-08-13 18:06:07.326008 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [310,21](310) acting [310,21,15](310)): not rw, up_thru 194489 up_from 192972 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [310,21](310) acting [310,21,15](310) maybe_went_rw) : primary up 192972-194489 includes interval 2014-08-13 18:06:07.326012 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.326021 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.256856 0 0.000000 2014-08-13 18:06:07.326020 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194488/194489/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [310,21](310) acting [310,21,15](310) maybe_went_rw) 2014-08-13 18:06:07.326032 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.256875 0 0.000000 2014-08-13 18:06:07.326032 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] up [310,21] -> [303,15,342], acting [310,21,15] -> [310,21,15], acting_primary 310 -> 310, up_primary 310 -> 303, role -1 -> -1 2014-08-13 18:06:07.326043 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Started 0.256926 0 0.000000 2014-08-13 18:06:07.326044 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.326053 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.326061 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194486 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.326064 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.331543 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194488 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.331564 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194488 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.331565 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.331574 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194488 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [28,304] newacting [28,304] 2014-08-13 18:06:07.331583 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194488 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.331592 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [4,28,304] r=0 lpr=194488 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.331585 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.331611 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,28,304](4) acting [4,28,304](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.331603 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.331621 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/194485) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] noting past interval(194485-194487 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 18:06:07.331632 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.331631 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] up [4,28,304] -> [28,304], acting [4,28,304] -> [28,304], acting_primary 4 -> 28, up_primary 4 -> 28, role 0 -> -1 2014-08-13 18:06:07.331643 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.331642 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.331652 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.331655 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.331663 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.331666 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.331672 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.331679 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.331683 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.331692 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.331689 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.331700 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.331706 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.331712 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.331720 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.331729 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.331738 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.331746 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.331755 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.331762 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.331769 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.331776 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.331783 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.331792 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.331735 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] handle_advance_map [303,15,342]/[303,15,342] -- 303/303 2014-08-13 18:06:07.331819 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 18:06:07.331827 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.331830 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.331832 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.331834 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.331843 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.331843 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.331852 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.331853 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] acting_up_affected newup [303,15,342] newacting [303,15,342] 2014-08-13 18:06:07.331864 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.331873 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342]/[310,21,15] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'417 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.331873 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,304]/[28,304] -- 28/28 2014-08-13 18:06:07.331884 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.331886 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.331894 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.331902 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.331911 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [303,15,342](303) acting [310,21,15](310)): not rw, up_thru 194489 up_from 192972 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [303,15,342](303) acting [310,21,15](310)) : primary up 192972-194489 does not include interval 2014-08-13 18:06:07.331921 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194491/194488) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 18:06:07.331927 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 18:06:07.331938 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.331931 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] up [303,15,342] -> [303,15,342], acting [310,21,15] -> [303,15,342], acting_primary 310 -> 303, up_primary 303 -> 303, role -1 -> -1 2014-08-13 18:06:07.331940 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.331948 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.331956 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] acting_up_affected newup [28,7,304] newacting [28,7,304] 2014-08-13 18:06:07.331943 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.331965 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.331969 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.331973 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,304] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'337 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.331978 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.331989 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.331990 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [28,304](28) acting [28,304](28)): not rw, up_thru 194488 up_from 192966 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) : primary up 192966-194488 includes interval 2014-08-13 18:06:07.331997 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.331999 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 18:06:07.332006 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.332009 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] up [28,304] -> [28,7,304], acting [28,304] -> [28,7,304], acting_primary 28 -> 28, up_primary 28 -> 28, role -1 -> -1 2014-08-13 18:06:07.332015 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332020 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332029 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.332026 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.332040 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332043 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.332049 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.332058 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.332052 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] [310,21,15] -> [303,15,342], acting primary 310 -> 303 2014-08-13 18:06:07.332066 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.332074 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332073 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.332083 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.332092 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.332083 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.332100 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.332104 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332109 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.332118 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332143 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 18:06:07.332154 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332156 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332164 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.332183 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332209 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 18:06:07.332219 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332221 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332229 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.332177 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [303,15,342]/[303,15,342] -- 303/303 2014-08-13 18:06:07.332237 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332243 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332247 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332258 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.332262 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [28,7,304]/[28,7,304] -- 28/28 2014-08-13 18:06:07.332272 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332275 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332270 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332283 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.332291 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332302 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [303,15,342]/[303,15,342] -- 303/303 2014-08-13 18:06:07.332313 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332311 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 18:06:07.332321 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332316 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332324 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332332 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.332329 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.332340 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,28,304] newacting [4,28,304] 2014-08-13 18:06:07.332348 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332349 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.332358 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [28,7,304] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'337 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.332367 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 18:06:07.332378 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.332375 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [28,7,304](28) acting [28,7,304](28)): not rw, up_thru 194491 up_from 192966 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) : primary up 192966-194491 includes interval 2014-08-13 18:06:07.332384 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.332384 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/194488) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 18:06:07.332393 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.332395 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [28,7,304] -> [4,28,304], acting [28,7,304] -> [4,28,304], acting_primary 28 -> 4, up_primary 28 -> 4, role -1 -> 0 2014-08-13 18:06:07.332406 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.332415 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.332423 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332432 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2906 2014-08-13 18:06:07.332442 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332402 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,310,21] newacting [4,310,21] 2014-08-13 18:06:07.332450 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.332452 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.332459 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.332461 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [303,15,342] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'417 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.332467 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332477 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332485 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332485 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [303,15,342](303) acting [303,15,342](303)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.332494 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.332497 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194491/194492/194492) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 18:06:07.332502 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332512 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.332521 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.332529 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.332537 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.332545 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.332508 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,15,342] -> [4,310,21], acting [303,15,342] -> [4,310,21], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 18:06:07.332553 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.332561 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.332561 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.332570 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.332578 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332588 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.332594 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.332598 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,28,304 unchanged 2014-08-13 18:06:07.332608 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.332617 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.006563 8 0.006419 2014-08-13 18:06:07.332605 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332631 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.332637 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2570 2014-08-13 18:06:07.332640 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.332648 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.332648 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332660 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.332659 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.332671 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.332676 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.332681 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.332686 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332691 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.332698 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332700 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.332707 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.332711 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 18:06:07.332716 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.332723 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 18:06:07.332725 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.332734 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 18:06:07.332735 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.332744 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,28,304 down blocked_by {} 2014-08-13 18:06:07.332755 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.332763 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.332765 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2907 2014-08-13 18:06:07.332773 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.332775 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.7 2014-08-13 18:06:07.332785 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.28 2014-08-13 18:06:07.332783 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.332795 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.332795 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.304 2014-08-13 18:06:07.332804 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.332806 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.332813 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.332816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.332821 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.332832 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.332856 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.332857 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.332876 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.332870 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,21,310 unchanged 2014-08-13 18:06:07.332881 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing 2014-08-13 18:06:07.332884 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.7 on 1 PGs 2014-08-13 18:06:07.332880 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.332892 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.007633 8 0.007388 2014-08-13 18:06:07.332902 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:07.332904 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.20 on 1 PGs 2014-08-13 18:06:07.332904 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.332915 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:07.332917 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.28 on 1 PGs 2014-08-13 18:06:07.332914 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.332930 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4857bc0 don't know epoch, doing nothing 2014-08-13 18:06:07.332932 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.31 on 1 PGs 2014-08-13 18:06:07.332942 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab69e0 don't know epoch, doing nothing 2014-08-13 18:06:07.332944 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.304 on 1 PGs 2014-08-13 18:06:07.332955 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:07.332956 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.337 on 1 PGs 2014-08-13 18:06:07.332930 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.332968 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab7640 don't know epoch, doing nothing 2014-08-13 18:06:07.332970 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.363 on 1 PGs 2014-08-13 18:06:07.332962 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000048 0 0.000000 2014-08-13 18:06:07.332981 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:07.332983 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.374 on 1 PGs 2014-08-13 18:06:07.332985 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.332995 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.333008 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.333017 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.333034 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 18:06:07.333034 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_advance_map [59,351]/[351,55] -- 59/351 2014-08-13 18:06:07.333055 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333049 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 18:06:07.333059 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Reset advmap 2014-08-13 18:06:07.333059 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [310,21](310) acting [310,21,15](310) maybe_went_rw) 2014-08-13 18:06:07.333070 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.333079 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [59,351] newacting [351,55] 2014-08-13 18:06:07.333071 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [310,21](310) acting [310,21](310)) 2014-08-13 18:06:07.333089 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.333092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,310,21](4) acting [4,310,21](4) maybe_went_rw) 2014-08-13 18:06:07.333099 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [4,59,351]/[4,351,55] r=0 lpr=194487 pi=193678-194485/35 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194488 2014-08-13 18:06:07.333105 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 18:06:07.333120 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,59,351](4) acting [4,351,55](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.333131 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194485/194486/194485) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] noting past interval(194486-194487 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) 2014-08-13 18:06:07.333117 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 18:06:07.333141 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] up [4,59,351] -> [59,351], acting [4,351,55] -> [351,55], acting_primary 4 -> 351, up_primary 4 -> 59, role 0 -> -1 2014-08-13 18:06:07.333153 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] clear_primary_state 2014-08-13 18:06:07.333150 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,15,21,303,310,342 down blocked_by {} 2014-08-13 18:06:07.333163 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.333163 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] agent_stop 2014-08-13 18:06:07.333178 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] on_change 2014-08-13 18:06:07.333175 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2571 2014-08-13 18:06:07.333188 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.333197 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:07.333207 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] cancel_copy_ops 2014-08-13 18:06:07.333193 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.15 2014-08-13 18:06:07.333216 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] cancel_flush_ops 2014-08-13 18:06:07.333221 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.21 2014-08-13 18:06:07.333225 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] on_change_cleanup 2014-08-13 18:06:07.333231 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.333235 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] on_change 2014-08-13 18:06:07.333245 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] exit NotTrimming 2014-08-13 18:06:07.333247 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.310 2014-08-13 18:06:07.333255 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped m=62] enter NotTrimming 2014-08-13 18:06:07.333259 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.342 2014-08-13 18:06:07.333264 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_stats 2014-08-13 18:06:07.333270 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.333275 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.333281 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.333284 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_role_change 2014-08-13 18:06:07.333293 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.333302 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 18:06:07.333310 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 18:06:07.333320 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333344 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.333340 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [59,351]/[351,55,376] -- 59/351 2014-08-13 18:06:07.333355 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333358 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.333367 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.333375 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [59,351] newacting [351,55,376] 2014-08-13 18:06:07.333385 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.333379 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 18:06:07.333394 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55] r=-1 lpr=194488 pi=193678-194487/36 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194489 2014-08-13 18:06:07.333402 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333406 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.333411 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [59,351](59) acting [351,55](351)): not rw, up_thru 194485 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [59,351](59) acting [351,55](351)) : primary up 192967-194485 does not include interval 2014-08-13 18:06:07.333416 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.333421 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194488/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] noting past interval(194488-194488 up [59,351](59) acting [351,55](351)) 2014-08-13 18:06:07.333432 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] up [59,351] -> [59,351], acting [351,55] -> [351,55,376], acting_primary 351 -> 351, up_primary 59 -> 59, role -1 -> -1 2014-08-13 18:06:07.333443 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 18:06:07.333426 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,285] newacting [289,285] 2014-08-13 18:06:07.333456 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.333475 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,4,285] r=1 lpr=194487 pi=193125-194485/44 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194488 2014-08-13 18:06:07.333497 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [289,4,285](289) acting [289,4,285](289)): not rw, up_thru 194486 up_from 192968 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [289,4,285](289) acting [289,4,285](289) maybe_went_rw) : primary up 192968-194486 includes interval 2014-08-13 18:06:07.333507 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194485/194486/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194486-194487 up [289,4,285](289) acting [289,4,285](289) maybe_went_rw) 2014-08-13 18:06:07.333524 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,4,285] -> [289,285], acting [289,4,285] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role 1 -> -1 2014-08-13 18:06:07.333452 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.333548 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:07.333555 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.333558 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.333568 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_copy_ops 2014-08-13 18:06:07.333571 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.333577 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_flush_ops 2014-08-13 18:06:07.333583 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 18:06:07.333588 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change_cleanup 2014-08-13 18:06:07.333592 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 18:06:07.333600 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.333602 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.333597 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 18:06:07.333604 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5240 pg_notify(22.84(2) epoch 194495) v5 2014-08-13 18:06:07.333602 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 18:06:07.333608 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit NotTrimming 2014-08-13 18:06:07.333612 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:07.333619 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter NotTrimming 2014-08-13 18:06:07.333633 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.7 2014-08-13 18:06:07.333635 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5240 2014-08-13 18:06:07.333629 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 18:06:07.333623 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 18:06:07.333639 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 18:06:07.333650 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333666 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.333667 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.333641 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 18:06:07.333671 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_role_change 2014-08-13 18:06:07.333672 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [59,351]/[351,55,376] -- 59/351 2014-08-13 18:06:07.333687 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333680 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.333690 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.333696 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 18:06:07.333700 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.333705 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 18:06:07.333710 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333716 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333733 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [55,351,376]/[351,55,376] -- 55/351 2014-08-13 18:06:07.333746 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333741 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 18:06:07.333749 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.333759 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.333768 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.333770 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.333771 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a43480 pg_notify(22.99 epoch 194495) v5 2014-08-13 18:06:07.333773 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333768 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [55,351,376] newacting [351,55,376] 2014-08-13 18:06:07.333778 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.333777 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.333794 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.31 2014-08-13 18:06:07.333796 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a43480 2014-08-13 18:06:07.333789 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [59,351]/[351,55,376] r=-1 lpr=194489 pi=193678-194488/37 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194491 2014-08-13 18:06:07.333789 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.333807 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [59,351](59) acting [351,55,376](351)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [59,351](59) acting [351,55,376](351) maybe_went_rw) : primary up 192967-194489 includes interval 2014-08-13 18:06:07.333818 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.333821 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.333813 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333819 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194488/194489/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] noting past interval(194489-194490 up [59,351](59) acting [351,55,376](351) maybe_went_rw) 2014-08-13 18:06:07.333830 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] up [59,351] -> [55,351,376], acting [351,55,376] -> [351,55,376], acting_primary 351 -> 351, up_primary 59 -> 55, role -1 -> -1 2014-08-13 18:06:07.333841 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 18:06:07.333850 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.333859 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.333869 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_copy_ops 2014-08-13 18:06:07.333859 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,285]/[289,285] -- 289/289 2014-08-13 18:06:07.333878 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_flush_ops 2014-08-13 18:06:07.333887 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333887 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change_cleanup 2014-08-13 18:06:07.333891 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.333895 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] on_change 2014-08-13 18:06:07.333900 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.333906 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] exit NotTrimming 2014-08-13 18:06:07.333910 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333916 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] enter NotTrimming 2014-08-13 18:06:07.333928 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] cancel_recovery 2014-08-13 18:06:07.333937 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] clear_recovery_state 2014-08-13 18:06:07.333946 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.333949 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,306,364]/[289,306,364] -- 289/289 2014-08-13 18:06:07.333972 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333980 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.333971 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] handle_advance_map [55,351,376]/[55,351,376] -- 55/55 2014-08-13 18:06:07.333982 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.333983 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2bc0 pg_notify(22.99(30) epoch 194495) v5 2014-08-13 18:06:07.333986 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.333975 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.333989 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.333989 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.334000 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.334009 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.337 2014-08-13 18:06:07.334011 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2bc0 2014-08-13 18:06:07.333999 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,306,364] newacting [289,306,364] 2014-08-13 18:06:07.334010 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] acting_up_affected newup [55,351,376] newacting [55,351,376] 2014-08-13 18:06:07.334014 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.334022 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.334036 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334038 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334031 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376]/[351,55,376] r=-1 lpr=194491 pi=193678-194490/38 crt=194422'15456 lcod 0'0 remapped NOTIFY m=62] set_last_peering_reset 194492 2014-08-13 18:06:07.334024 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,285] r=-1 lpr=194488 pi=193125-194487/45 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194491 2014-08-13 18:06:07.334055 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334057 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334049 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)) : primary up 192967-194489 does not include interval 2014-08-13 18:06:07.334058 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df32c0 pg_notify(22.84(28) epoch 194495) v5 2014-08-13 18:06:07.334060 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194491/194488) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] noting past interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 18:06:07.334062 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [289,285](289) acting [289,285](289)): not rw, up_thru 194489 up_from 192968 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [289,285](289) acting [289,285](289) maybe_went_rw) : primary up 192968-194489 includes interval 2014-08-13 18:06:07.334071 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] up [55,351,376] -> [55,351,376], acting [351,55,376] -> [55,351,376], acting_primary 351 -> 55, up_primary 55 -> 55, role -1 -> -1 2014-08-13 18:06:07.334074 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194488/194488/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194488-194490 up [289,285](289) acting [289,285](289) maybe_went_rw) 2014-08-13 18:06:07.334084 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.304 2014-08-13 18:06:07.334086 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df32c0 2014-08-13 18:06:07.334083 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 18:06:07.334085 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,285] -> [289,306,364], acting [289,285] -> [289,306,364], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.334092 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.334108 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334100 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:07.334101 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.334110 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334111 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.334112 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_copy_ops 2014-08-13 18:06:07.334120 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.334129 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334123 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_flush_ops 2014-08-13 18:06:07.334131 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334132 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a42f40 pg_notify(22.99(1) epoch 194495) v5 2014-08-13 18:06:07.334132 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change_cleanup 2014-08-13 18:06:07.334142 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 18:06:07.334130 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 18:06:07.334155 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.363 2014-08-13 18:06:07.334157 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a42f40 2014-08-13 18:06:07.334151 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] exit NotTrimming 2014-08-13 18:06:07.334154 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 18:06:07.334161 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] enter NotTrimming 2014-08-13 18:06:07.334177 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334179 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334169 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 18:06:07.334190 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334192 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334177 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] [351,55,376] -> [55,351,376], acting primary 351 -> 55 2014-08-13 18:06:07.334182 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:07.334193 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a432c0 pg_notify(22.99 epoch 194495) v5 2014-08-13 18:06:07.334194 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] cancel_recovery 2014-08-13 18:06:07.334195 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 18:06:07.334203 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] clear_recovery_state 2014-08-13 18:06:07.334206 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 18:06:07.334213 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334223 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.374 2014-08-13 18:06:07.334225 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a432c0 2014-08-13 18:06:07.334217 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 18:06:07.334245 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334247 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334238 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] handle_advance_map [55,351,376]/[55,351,376] -- 55/55 2014-08-13 18:06:07.334243 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 18:06:07.334253 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334259 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334260 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334254 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334262 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f1c0 pg_notify(22.84(2) epoch 194495) v5 2014-08-13 18:06:07.334256 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.334266 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.334275 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334284 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.28 2014-08-13 18:06:07.334285 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f1c0 2014-08-13 18:06:07.334280 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,306,364]/[289,285] -- 289/289 2014-08-13 18:06:07.334304 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334305 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334306 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334297 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] handle_advance_map [55,351,376]/[55,351,376] -- 55/55 2014-08-13 18:06:07.334310 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334308 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.334313 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.334317 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.334326 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334328 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334329 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf0c40 pg_notify(22.99(30) epoch 194495) v5 2014-08-13 18:06:07.334322 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.334333 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334327 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] acting_up_affected newup [289,306,364] newacting [289,285] 2014-08-13 18:06:07.334351 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.20 2014-08-13 18:06:07.334353 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf0c40 2014-08-13 18:06:07.334345 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.334350 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] handle_advance_map [4,59,351]/[4,59,351] -- 4/4 2014-08-13 18:06:07.334356 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] set_last_peering_reset 194492 2014-08-13 18:06:07.334365 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334374 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.334368 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: Reset advmap 2014-08-13 18:06:07.334377 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.334374 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [289,306,364](289) acting [289,306,364](289)): not rw, up_thru 194489 up_from 192968 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [289,306,364](289) acting [289,306,364](289)) : primary up 192968-194489 does not include interval 2014-08-13 18:06:07.334378 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.334387 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] acting_up_affected newup [4,59,351] newacting [4,59,351] 2014-08-13 18:06:07.334386 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194491/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] noting past interval(194491-194491 up [289,306,364](289) acting [289,306,364](289)) 2014-08-13 18:06:07.334396 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.334405 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [55,351,376] r=-1 lpr=194492 pi=193678-194491/39 crt=194422'15456 lcod 0'0 inactive NOTIFY m=62] set_last_peering_reset 194495 2014-08-13 18:06:07.334399 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] up [289,306,364] -> [289,306,364], acting [289,306,364] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.334422 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [55,351,376](55) acting [55,351,376](55)): not rw, up_thru 194492 up_from 192973 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) : primary up 192973-194492 includes interval 2014-08-13 18:06:07.334423 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 18:06:07.334433 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.334432 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194491/194492/194492) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] noting past interval(194492-194494 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 18:06:07.334443 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.334444 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] up [55,351,376] -> [4,59,351], acting [55,351,376] -> [4,59,351], acting_primary 55 -> 4, up_primary 55 -> 4, role -1 -> 0 2014-08-13 18:06:07.334456 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] clear_primary_state 2014-08-13 18:06:07.334456 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_copy_ops 2014-08-13 18:06:07.334466 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] agent_stop 2014-08-13 18:06:07.334466 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_flush_ops 2014-08-13 18:06:07.334477 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 18:06:07.334478 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change_cleanup 2014-08-13 18:06:07.334487 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 18:06:07.334487 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] publish_stats_to_osd 194495:25638 2014-08-13 18:06:07.334499 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit NotTrimming 2014-08-13 18:06:07.334500 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.334510 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] cancel_copy_ops 2014-08-13 18:06:07.334519 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] cancel_flush_ops 2014-08-13 18:06:07.334509 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter NotTrimming 2014-08-13 18:06:07.334528 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.334530 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_recovery 2014-08-13 18:06:07.334537 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.334540 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] clear_recovery_state 2014-08-13 18:06:07.334546 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] requeue_ops 2014-08-13 18:06:07.334551 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334556 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change_cleanup 2014-08-13 18:06:07.334565 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] on_change 2014-08-13 18:06:07.334574 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] exit NotTrimming 2014-08-13 18:06:07.334583 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive NOTIFY m=62] enter NotTrimming 2014-08-13 18:06:07.334585 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,306,364]/[289,285] -- 289/289 2014-08-13 18:06:07.334592 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] clear_stats 2014-08-13 18:06:07.334604 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334601 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] requeue_ops 2014-08-13 18:06:07.334607 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.334610 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] on_role_change 2014-08-13 18:06:07.334619 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] requeue_ops 2014-08-13 18:06:07.334618 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.334627 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] cancel_recovery 2014-08-13 18:06:07.334631 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334636 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] clear_recovery_state 2014-08-13 18:06:07.334647 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334654 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,306,364]/[289,285] -- 289/289 2014-08-13 18:06:07.334657 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_activate_map 2014-08-13 18:06:07.334668 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334667 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] update_heartbeat_peers -> 4,59,351 2014-08-13 18:06:07.334678 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.334680 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] take_waiters 2014-08-13 18:06:07.334673 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.334689 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Reset 0.534499 10 0.001431 2014-08-13 18:06:07.334692 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.334701 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started 2014-08-13 18:06:07.334702 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334711 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Start 2014-08-13 18:06:07.334720 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: transitioning to Primary 2014-08-13 18:06:07.334726 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,4,285]/[289,285] -- 289/289 2014-08-13 18:06:07.334731 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.334740 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.334742 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary 2014-08-13 18:06:07.334743 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:07.334752 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering 2014-08-13 18:06:07.334762 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.334753 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.334771 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.334773 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,285] 2014-08-13 18:06:07.334781 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194492-194494 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 18:06:07.334784 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.334794 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,306,364]/[289,285] r=-1 lpr=194492 pi=193125-194491/47 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194495 2014-08-13 18:06:07.334793 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 18:06:07.334805 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194489-194490 up [59,351](59) acting [351,55,376](351) maybe_went_rw) 2014-08-13 18:06:07.334812 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [289,306,364](289) acting [289,285](289)): not rw, up_thru 194492 up_from 192968 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [289,306,364](289) acting [289,285](289) maybe_went_rw) : primary up 192968-194492 includes interval 2014-08-13 18:06:07.334816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194488-194488 up [59,351](59) acting [351,55](351)) 2014-08-13 18:06:07.334827 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194486-194487 up [4,59,351](4) acting [4,351,55](4) maybe_went_rw) 2014-08-13 18:06:07.334825 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194491/194492/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] noting past interval(194492-194494 up [289,306,364](289) acting [289,285](289) maybe_went_rw) 2014-08-13 18:06:07.334839 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194485-194485 up [4,59,351](4) acting [4,59,351](4)) 2014-08-13 18:06:07.334844 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] up [289,306,364] -> [289,4,285], acting [289,285] -> [289,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.334849 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194483-194484 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 18:06:07.334857 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 18:06:07.334861 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194482-194482 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 18:06:07.334870 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.334871 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 18:06:07.334880 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.334882 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.334890 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_copy_ops 2014-08-13 18:06:07.334893 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] publish_stats_to_osd 194495:25639 2014-08-13 18:06:07.334900 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_flush_ops 2014-08-13 18:06:07.334905 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.55 2014-08-13 18:06:07.334910 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change_cleanup 2014-08-13 18:06:07.334916 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.59 2014-08-13 18:06:07.334922 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] on_change 2014-08-13 18:06:07.334927 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.351 2014-08-13 18:06:07.334933 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit NotTrimming 2014-08-13 18:06:07.334937 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: querying info from osd.376 2014-08-13 18:06:07.334944 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter NotTrimming 2014-08-13 18:06:07.334948 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.334953 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] cancel_recovery 2014-08-13 18:06:07.334959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.334963 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] clear_recovery_state 2014-08-13 18:06:07.334976 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.334987 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_activate_map 2014-08-13 18:06:07.335022 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.335067 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:07.335073 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.55 on 1 PGs 2014-08-13 18:06:07.335058 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] update_heartbeat_peers unchanged 2014-08-13 18:06:07.335075 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] take_waiters 2014-08-13 18:06:07.335089 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3b00c60 don't know epoch, doing nothing 2014-08-13 18:06:07.335092 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.59 sending saperate messages on 1 PGs 2014-08-13 18:06:07.335086 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Reset 0.717249 10 0.001306 2014-08-13 18:06:07.335101 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4af8160 don't know epoch, doing nothing 2014-08-13 18:06:07.335103 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.351 on 1 PGs 2014-08-13 18:06:07.335114 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.335116 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.376 on 1 PGs 2014-08-13 18:06:07.335098 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started 2014-08-13 18:06:07.335130 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Start 2014-08-13 18:06:07.335145 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: transitioning to Stray 2014-08-13 18:06:07.335164 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Start 0.000034 0 0.000000 2014-08-13 18:06:07.335158 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_advance_map [55,61]/[50,65] -- 55/50 2014-08-13 18:06:07.335183 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335176 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Started/Stray 2014-08-13 18:06:07.335187 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Reset advmap 2014-08-13 18:06:07.335198 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.335208 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [55,61] newacting [50,65] 2014-08-13 18:06:07.335217 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.335194 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.335226 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [4,55,61]/[4,50,65] r=0 lpr=194487 pi=193747-194485/32 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194488 2014-08-13 18:06:07.335227 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.335245 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,55,61](4) acting [4,50,65](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.335255 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194485/194486/194485) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] noting past interval(194486-194487 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) 2014-08-13 18:06:07.335276 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] up [4,55,61] -> [55,61], acting [4,50,65] -> [50,65], acting_primary 4 -> 50, up_primary 4 -> 55, role 0 -> -1 2014-08-13 18:06:07.335287 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] clear_primary_state 2014-08-13 18:06:07.335297 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] agent_stop 2014-08-13 18:06:07.335306 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] on_change 2014-08-13 18:06:07.335315 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.335324 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:07.335333 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] cancel_copy_ops 2014-08-13 18:06:07.335343 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.335342 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] cancel_flush_ops 2014-08-13 18:06:07.335351 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] on_change_cleanup 2014-08-13 18:06:07.335359 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] on_change 2014-08-13 18:06:07.335369 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] exit NotTrimming 2014-08-13 18:06:07.335378 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped m=51] enter NotTrimming 2014-08-13 18:06:07.335387 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_stats 2014-08-13 18:06:07.335396 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.335391 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_advance_map [5,47]/[5,47] -- 5/5 2014-08-13 18:06:07.335409 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335413 7f5c4df73700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.335438 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.335450 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.335459 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.335476 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.297611 2 0.000209 2014-08-13 18:06:07.335488 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.335405 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_role_change 2014-08-13 18:06:07.335497 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.297642 0 0.000000 2014-08-13 18:06:07.335511 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.335523 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.335527 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.297681 0 0.000000 2014-08-13 18:06:07.335532 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.335540 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Started 0.297733 0 0.000000 2014-08-13 18:06:07.335543 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.335551 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.335563 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194486 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.335565 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [55,61]/[50,65,356] -- 55/50 2014-08-13 18:06:07.335573 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194488 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.335580 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335582 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194488 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.335583 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.335580 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] flushed 2014-08-13 18:06:07.335592 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194488 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [5,47] newacting [5,47] 2014-08-13 18:06:07.335594 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.335604 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [55,61] newacting [50,65,356] 2014-08-13 18:06:07.335602 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194488 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.335614 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.335622 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65] r=-1 lpr=194488 pi=193747-194487/33 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194489 2014-08-13 18:06:07.335621 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [4,5,47] r=0 lpr=194488 pi=193306-194484/30 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.335641 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [55,61](55) acting [50,65](50)): not rw, up_thru 194483 up_from 192979 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [55,61](55) acting [50,65](50)) : primary up 192979-194483 does not include interval 2014-08-13 18:06:07.335643 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,5,47](4) acting [4,5,47](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.335651 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194488/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] noting past interval(194488-194488 up [55,61](55) acting [50,65](50)) 2014-08-13 18:06:07.335653 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194485/194485/194485) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] noting past interval(194485-194487 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) 2014-08-13 18:06:07.335662 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] up [55,61] -> [55,61], acting [50,65] -> [50,65,356], acting_primary 50 -> 50, up_primary 55 -> 55, role -1 -> -1 2014-08-13 18:06:07.335667 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] up [4,5,47] -> [5,47], acting [4,5,47] -> [5,47], acting_primary 4 -> 5, up_primary 4 -> 5, role 0 -> -1 2014-08-13 18:06:07.335673 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.335678 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.335683 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.335688 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.335692 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.335697 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.335702 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.335706 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.335712 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.335722 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.335718 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.335731 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.335740 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.335740 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.335750 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.335758 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.335759 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.335767 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.335768 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.335779 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.335782 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.335793 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.335798 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [55,61]/[50,65,356] -- 55/50 2014-08-13 18:06:07.335802 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.335815 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335818 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.335818 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.335828 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.335837 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.335838 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.335847 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.335854 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.335862 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 18:06:07.335875 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335862 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.335878 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.335887 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.335888 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.335896 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [50,65,356] newacting [50,65,356] 2014-08-13 18:06:07.335902 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.335906 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.335915 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [55,61]/[50,65,356] r=-1 lpr=194489 pi=193747-194488/34 crt=194422'15470 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194491 2014-08-13 18:06:07.335927 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [5,47]/[5,47,11] -- 5/5 2014-08-13 18:06:07.335931 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [55,61](55) acting [50,65,356](50)): not rw, up_thru 194489 up_from 192979 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [55,61](55) acting [50,65,356](50) maybe_went_rw) : primary up 192979-194489 includes interval 2014-08-13 18:06:07.335942 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.335943 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194488/194489/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] noting past interval(194489-194490 up [55,61](55) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.335945 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.335958 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.335960 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.335962 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf16c0 pg_notify(24.1f7(1) epoch 194495) v5 2014-08-13 18:06:07.335953 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] up [55,61] -> [50,65,356], acting [50,65,356] -> [50,65,356], acting_primary 50 -> 50, up_primary 55 -> 50, role -1 -> -1 2014-08-13 18:06:07.335954 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.335966 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.335975 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.335974 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] acting_up_affected newup [5,47] newacting [5,47,11] 2014-08-13 18:06:07.335992 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.55 2014-08-13 18:06:07.335985 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.335985 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.335994 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf16c0 2014-08-13 18:06:07.335996 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.335996 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47] r=-1 lpr=194488 pi=193306-194487/31 crt=191648'346 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.336006 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.336021 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336015 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.336023 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336024 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.336025 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [5,47](5) acting [5,47](5)): not rw, up_thru 194479 up_from 192977 last_epoch_clean 194483 generate_past_intervals interval(194488-194488 up [5,47](5) acting [5,47](5)) : primary up 192977-194479 does not include interval 2014-08-13 18:06:07.336033 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.336042 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.336039 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194488/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [5,47](5) acting [5,47](5)) 2014-08-13 18:06:07.336051 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.336055 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] up [5,47] -> [5,47], acting [5,47] -> [5,47,11], acting_primary 5 -> 5, up_primary 5 -> 5, role -1 -> -1 2014-08-13 18:06:07.336059 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.336067 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.336069 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336077 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.336088 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.336092 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 18:06:07.336106 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336098 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.336109 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.336111 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.336118 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.336129 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336121 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.336131 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336132 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a43640 pg_notify(24.1f7(43) epoch 194495) v5 2014-08-13 18:06:07.336128 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336131 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.336155 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.351 2014-08-13 18:06:07.336156 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a43640 2014-08-13 18:06:07.336150 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 18:06:07.336163 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336157 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.336169 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.336178 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336166 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.336181 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.336179 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.336191 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336192 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.336202 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336180 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336212 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [50,65,356]/[50,65,356] -- 50/50 2014-08-13 18:06:07.336226 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336222 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] handle_advance_map [5,47]/[5,47,11] -- 5/5 2014-08-13 18:06:07.336228 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.336238 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336237 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.336247 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336264 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [4,55,61]/[4,55,61] -- 4/4 2014-08-13 18:06:07.336241 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.336276 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336279 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.336288 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.336277 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.336296 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [4,55,61] newacting [4,55,61] 2014-08-13 18:06:07.336303 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336305 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.336314 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [50,65,356] r=-1 lpr=194491 pi=193747-194490/35 crt=194422'15470 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194495 2014-08-13 18:06:07.336341 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] handle_advance_map [11,54,368]/[5,47,11] -- 11/5 2014-08-13 18:06:07.336354 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336330 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [50,65,356](50) acting [50,65,356](50)): not rw, up_thru 194491 up_from 192979 last_epoch_clean 194484 generate_past_intervals interval(194491-194494 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) : primary up 192979-194491 includes interval 2014-08-13 18:06:07.336357 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.336356 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194491/194491/194488) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] noting past interval(194491-194494 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.336369 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] up [50,65,356] -> [4,55,61], acting [50,65,356] -> [4,55,61], acting_primary 50 -> 4, up_primary 50 -> 4, role -1 -> 0 2014-08-13 18:06:07.336367 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.336382 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] clear_primary_state 2014-08-13 18:06:07.336385 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] acting_up_affected newup [11,54,368] newacting [5,47,11] 2014-08-13 18:06:07.336393 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] agent_stop 2014-08-13 18:06:07.336395 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.336408 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336402 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.336411 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336406 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [5,47]/[5,47,11] r=-1 lpr=194489 pi=193306-194488/32 crt=191648'346 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.336413 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2840 pg_notify(24.1f7(39) epoch 194495) v5 2014-08-13 18:06:07.336413 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd 194495:22527 2014-08-13 18:06:07.336424 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.336434 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.336441 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.336426 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [5,47](5) acting [5,47,11](5)): not rw, up_thru 194489 up_from 192977 last_epoch_clean 194483 generate_past_intervals interval(194489-194490 up [5,47](5) acting [5,47,11](5) maybe_went_rw) : primary up 192977-194489 includes interval 2014-08-13 18:06:07.336444 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2840 2014-08-13 18:06:07.336443 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.336446 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194488/194489/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [5,47](5) acting [5,47,11](5) maybe_went_rw) 2014-08-13 18:06:07.336453 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.336458 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] up [5,47] -> [11,54,368], acting [5,47,11] -> [5,47,11], acting_primary 5 -> 5, up_primary 5 -> 11, role -1 -> -1 2014-08-13 18:06:07.336463 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.336474 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336476 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336472 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.336471 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.336481 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.336490 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.336500 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.336489 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.336509 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.336509 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.336519 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] clear_stats 2014-08-13 18:06:07.336528 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] requeue_ops 2014-08-13 18:06:07.336521 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.336536 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] on_role_change 2014-08-13 18:06:07.336537 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.336545 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] requeue_ops 2014-08-13 18:06:07.336547 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.336554 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] cancel_recovery 2014-08-13 18:06:07.336557 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.336564 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] clear_recovery_state 2014-08-13 18:06:07.336574 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336584 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_activate_map 2014-08-13 18:06:07.336568 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.336594 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] update_heartbeat_peers -> 4,55,61 2014-08-13 18:06:07.336604 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.336597 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.336606 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] take_waiters 2014-08-13 18:06:07.336608 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.336623 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336625 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336616 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Reset 0.631669 10 0.001235 2014-08-13 18:06:07.336617 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.336627 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee5340 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.336633 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.336637 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.336630 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started 2014-08-13 18:06:07.336639 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336640 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336630 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.336641 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Start 2014-08-13 18:06:07.336650 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: transitioning to Primary 2014-08-13 18:06:07.336660 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.336659 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 18:06:07.336675 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336677 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336678 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df6740 pg_notify(24.1f7 epoch 194495) v5 2014-08-13 18:06:07.336673 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.336672 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary 2014-08-13 18:06:07.336683 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.336701 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.59 2014-08-13 18:06:07.336694 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.336703 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df6740 2014-08-13 18:06:07.336683 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering 2014-08-13 18:06:07.336704 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] acting_up_affected newup [11,54,368] newacting [11,54,368] 2014-08-13 18:06:07.336714 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.336726 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.336728 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.336714 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.336726 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.336730 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368]/[5,47,11] r=-1 lpr=194491 pi=193306-194490/33 crt=191648'346 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.336737 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194491-194494 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.336749 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194489-194490 up [55,61](55) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.336750 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [11,54,368](11) acting [5,47,11](5)): not rw, up_thru 194489 up_from 192977 last_epoch_clean 194483 generate_past_intervals interval(194491-194491 up [11,54,368](11) acting [5,47,11](5)) : primary up 192977-194489 does not include interval 2014-08-13 18:06:07.336761 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194488-194488 up [55,61](55) acting [50,65](50)) 2014-08-13 18:06:07.336761 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194491/194488) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [11,54,368](11) acting [5,47,11](5)) 2014-08-13 18:06:07.336772 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194486-194487 up [4,55,61](4) acting [4,50,65](4) maybe_went_rw) 2014-08-13 18:06:07.336773 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] up [11,54,368] -> [11,54,368], acting [5,47,11] -> [11,54,368], acting_primary 5 -> 11, up_primary 11 -> 11, role -1 -> -1 2014-08-13 18:06:07.336785 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.336785 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194485-194485 up [4,55,61](4) acting [4,55,61](4)) 2014-08-13 18:06:07.336794 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.336796 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194483-194484 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.336803 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.336807 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194482-194482 up [50,65,356](50) acting [50,65](50)) 2014-08-13 18:06:07.336812 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.336818 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 18:06:07.336822 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.339754 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.339759 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.339769 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] publish_stats_to_osd 194495:22528 2014-08-13 18:06:07.339773 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.339780 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.50 2014-08-13 18:06:07.339782 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.339791 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.55 2014-08-13 18:06:07.339792 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.339801 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.61 2014-08-13 18:06:07.339801 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] [5,47,11] -> [11,54,368], acting primary 5 -> 11 2014-08-13 18:06:07.339812 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.65 2014-08-13 18:06:07.339822 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: querying info from osd.356 2014-08-13 18:06:07.339818 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.339832 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.339833 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.339842 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.339843 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.339867 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 18:06:07.339878 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.339881 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.339889 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.339898 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.339905 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.339918 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [11,54,368]/[11,54,368] -- 11/11 2014-08-13 18:06:07.339928 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.339930 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.339933 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 18:06:07.339939 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.339948 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.339948 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.339953 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.339962 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20] newacting [337,20] 2014-08-13 18:06:07.339965 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 18:06:07.339977 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.339971 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.339979 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.339982 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.291906 3 0.000116 2014-08-13 18:06:07.339988 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.339997 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.291958 0 0.000000 2014-08-13 18:06:07.340009 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.339997 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,5,47] newacting [4,5,47] 2014-08-13 18:06:07.340017 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.340023 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.340026 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340034 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [11,54,368] r=-1 lpr=194492 pi=193306-194491/34 crt=191648'346 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.340035 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340044 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20] newacting [337,20] 2014-08-13 18:06:07.340052 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.340053 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [11,54,368](11) acting [11,54,368](11)): not rw, up_thru 194492 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194492-194494 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) : primary up 192970-194492 includes interval 2014-08-13 18:06:07.340061 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.340063 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194491/194492/194492) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 18:06:07.340073 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [11,54,368] -> [4,5,47], acting [11,54,368] -> [4,5,47], acting_primary 11 -> 4, up_primary 11 -> 4, role -1 -> 0 2014-08-13 18:06:07.340078 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [337,20,4](337) acting [337,20,4](337)): not rw, up_thru 194485 up_from 192943 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [337,20,4](337) acting [337,20,4](337) maybe_went_rw) : primary up 192943-194485 includes interval 2014-08-13 18:06:07.340085 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.340088 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [337,20,4](337) acting [337,20,4](337) maybe_went_rw) 2014-08-13 18:06:07.340095 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.340098 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20,4] -> [337,20], acting [337,20,4] -> [337,20], acting_primary 337 -> 337, up_primary 337 -> 337, role 2 -> -1 2014-08-13 18:06:07.340109 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340104 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340117 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.340125 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340124 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:3007 2014-08-13 18:06:07.340135 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.340136 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340143 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.340152 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.340151 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.340161 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.340160 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340170 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340176 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.340179 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340185 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.340188 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340195 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.340197 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.340203 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340207 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340213 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.340217 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.340221 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.340228 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.340231 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340237 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.340248 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.340251 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 18:06:07.340263 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340257 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.340265 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.340265 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340275 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340274 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.340284 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340288 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.340299 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340301 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20]/[337,20] -- 337/337 2014-08-13 18:06:07.340311 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340313 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340309 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.340322 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340324 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,5,47 unchanged 2014-08-13 18:06:07.340330 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340335 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.340344 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.004793 8 0.004679 2014-08-13 18:06:07.340350 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 18:06:07.340361 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340363 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340356 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.340371 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340377 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.340380 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20,375] newacting [337,20,375] 2014-08-13 18:06:07.340386 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.340388 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.340397 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.340397 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.340413 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [337,20](337) acting [337,20](337)): not rw, up_thru 194488 up_from 192943 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [337,20](337) acting [337,20](337) maybe_went_rw) : primary up 192943-194488 includes interval 2014-08-13 18:06:07.340409 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.340422 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [337,20](337) acting [337,20](337) maybe_went_rw) 2014-08-13 18:06:07.340429 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.340432 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20] -> [337,20,375], acting [337,20] -> [337,20,375], acting_primary 337 -> 337, up_primary 337 -> 337, role -1 -> -1 2014-08-13 18:06:07.340439 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.340442 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340451 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.340459 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340449 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.340467 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.340476 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.340473 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 18:06:07.340484 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.340486 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [11,54,368](11) acting [5,47,11](5)) 2014-08-13 18:06:07.340492 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340497 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [5,47](5) acting [5,47,11](5) maybe_went_rw) 2014-08-13 18:06:07.340502 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.340511 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.340509 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [5,47](5) acting [5,47](5)) 2014-08-13 18:06:07.340519 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.340519 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,5,47](4) acting [4,5,47](4) maybe_went_rw) 2014-08-13 18:06:07.340528 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.340531 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 18:06:07.340537 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340543 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [5,47](5) acting [5,47](5) maybe_went_rw) 2014-08-13 18:06:07.340553 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,5,11,47,54,368 down blocked_by {} 2014-08-13 18:06:07.340558 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 18:06:07.340570 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340564 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.340572 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340581 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340576 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:3008 2014-08-13 18:06:07.340590 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340591 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.5 2014-08-13 18:06:07.340607 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.11 2014-08-13 18:06:07.340608 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 18:06:07.340619 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340618 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.47 2014-08-13 18:06:07.340621 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340630 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340628 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.54 2014-08-13 18:06:07.340638 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340647 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.368 2014-08-13 18:06:07.340657 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.340656 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,375]/[337,20,375] -- 337/337 2014-08-13 18:06:07.340668 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340668 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.340670 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340679 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340687 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340703 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 18:06:07.340714 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.340716 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.340724 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.340732 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [337,20,4] newacting [337,20,4] 2014-08-13 18:06:07.340739 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.340740 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.340748 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,375] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.340763 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [337,20,375](337) acting [337,20,375](337)): not rw, up_thru 194491 up_from 192943 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [337,20,375](337) acting [337,20,375](337) maybe_went_rw) : primary up 192943-194491 includes interval 2014-08-13 18:06:07.340776 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [337,20,375](337) acting [337,20,375](337) maybe_went_rw) 2014-08-13 18:06:07.340785 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] up [337,20,375] -> [337,20,4], acting [337,20,375] -> [337,20,4], acting_primary 337 -> 337, up_primary 337 -> 337, role -1 -> 2 2014-08-13 18:06:07.340796 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340804 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.340812 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4493f40 don't know epoch, doing nothing 2014-08-13 18:06:07.340811 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340818 7f5c7bde3700 7 osd.4 194495 do_notify osd 17 on 1 PGs 2014-08-13 18:06:07.340820 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.340828 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.340836 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.340843 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:07.340846 7f5c7bde3700 7 osd.4 194495 do_notify osd 289 on 1 PGs 2014-08-13 18:06:07.340844 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.340853 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.340859 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.340861 7f5c7bde3700 7 osd.4 194495 do_notify osd 367 on 1 PGs 2014-08-13 18:06:07.340862 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.340873 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.340881 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.340888 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x449d280 don't know epoch, doing nothing 2014-08-13 18:06:07.340890 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.5 on 1 PGs 2014-08-13 18:06:07.340889 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.340903 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2f640 don't know epoch, doing nothing 2014-08-13 18:06:07.340905 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.11 on 1 PGs 2014-08-13 18:06:07.340897 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.340909 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.340918 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:07.340920 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.15 on 1 PGs 2014-08-13 18:06:07.341008 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.341038 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3a689a0 don't know epoch, doing nothing 2014-08-13 18:06:07.341056 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.21 on 1 PGs 2014-08-13 18:06:07.341053 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.341063 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.341072 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.001063 8 0.000892 2014-08-13 18:06:07.341081 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x49a5960 don't know epoch, doing nothing 2014-08-13 18:06:07.341084 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.47 sending saperate messages on 1 PGs 2014-08-13 18:06:07.341083 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.341097 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4425280 don't know epoch, doing nothing 2014-08-13 18:06:07.341099 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.54 on 1 PGs 2014-08-13 18:06:07.341093 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.341103 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.341112 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:07.341114 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.303 on 1 PGs 2014-08-13 18:06:07.341114 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.341125 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.341139 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4fa1fa0 don't know epoch, doing nothing 2014-08-13 18:06:07.341141 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.310 on 1 PGs 2014-08-13 18:06:07.341136 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.341145 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.341159 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5b440 don't know epoch, doing nothing 2014-08-13 18:06:07.341161 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.342 on 1 PGs 2014-08-13 18:06:07.341193 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c5bde0 don't know epoch, doing nothing 2014-08-13 18:06:07.341196 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.368 on 1 PGs 2014-08-13 18:06:07.341202 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.341263 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 18:06:07.341294 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341300 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.341339 7f5c4dd71700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.341333 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374] newacting [358,374] 2014-08-13 18:06:07.341359 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.341373 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.300778 3 0.000114 2014-08-13 18:06:07.341384 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started 0.300827 0 0.000000 2014-08-13 18:06:07.341395 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.341403 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194486 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.341416 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194488 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341427 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194488 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.341435 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194488 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374] newacting [358,374] 2014-08-13 18:06:07.341426 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,21]/[371,21] -- 371/371 2014-08-13 18:06:07.341451 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341445 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194488 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.341456 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.341466 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,21] newacting [371,21] 2014-08-13 18:06:07.341475 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.341465 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374,4] r=2 lpr=194488 pi=192969-194484/24 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.341485 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.290566 3 0.000138 2014-08-13 18:06:07.341497 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started 0.290634 0 0.000000 2014-08-13 18:06:07.341507 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.341516 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194486 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.341519 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [358,374,4](358) acting [358,374,4](358)): not rw, up_thru 194485 up_from 192966 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [358,374,4](358) acting [358,374,4](358) maybe_went_rw) : primary up 192966-194485 includes interval 2014-08-13 18:06:07.341525 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194488 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341531 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [358,374,4](358) acting [358,374,4](358) maybe_went_rw) 2014-08-13 18:06:07.341535 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194488 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.341543 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194488 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,21] newacting [371,21] 2014-08-13 18:06:07.341542 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374,4] -> [358,374], acting [358,374,4] -> [358,374], acting_primary 358 -> 358, up_primary 358 -> 358, role 2 -> -1 2014-08-13 18:06:07.341552 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194488 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.341556 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.341561 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,4,21] r=1 lpr=194488 pi=192977-194484/29 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.341566 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.341574 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.341581 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [371,4,21](371) acting [371,4,21](371)): not rw, up_thru 194485 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [371,4,21](371) acting [371,4,21](371) maybe_went_rw) : primary up 192970-194485 includes interval 2014-08-13 18:06:07.341583 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.341591 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194485/194485/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [371,4,21](371) acting [371,4,21](371) maybe_went_rw) 2014-08-13 18:06:07.341592 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.341601 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,4,21] -> [371,21], acting [371,4,21] -> [371,21], acting_primary 371 -> 371, up_primary 371 -> 371, role 1 -> -1 2014-08-13 18:06:07.341607 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.341612 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.341617 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.341608 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.341622 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.341628 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.341631 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.341641 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.341638 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.341648 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.341656 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.341657 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.341665 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.341666 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.341673 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.341676 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.341681 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.341686 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.341695 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.341692 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.341702 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.341710 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.341718 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.341723 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 18:06:07.341727 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.341735 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341738 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341748 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,21]/[371,21,24] -- 371/371 2014-08-13 18:06:07.341759 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341762 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341755 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.341771 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.341772 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.341779 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,21] newacting [371,21,24] 2014-08-13 18:06:07.341788 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.341796 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21] r=-1 lpr=194488 pi=192977-194487/30 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.341791 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374]/[358,374] -- 358/358 2014-08-13 18:06:07.341813 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341812 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [371,21](371) acting [371,21](371)): not rw, up_thru 194485 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194488-194488 up [371,21](371) acting [371,21](371)) : primary up 192970-194485 does not include interval 2014-08-13 18:06:07.341818 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341829 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194488/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [371,21](371) acting [371,21](371)) 2014-08-13 18:06:07.341840 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] up [371,21] -> [371,21], acting [371,21] -> [371,21,24], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> -1 2014-08-13 18:06:07.341829 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.341850 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.341852 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.341859 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.341868 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.341877 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.341879 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 18:06:07.341891 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341886 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.341894 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341895 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.341904 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.341913 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.341903 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.341922 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.341932 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.341943 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.341945 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.341925 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374,2] newacting [358,374,2] 2014-08-13 18:06:07.341940 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.341946 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3f00 pg_notify(22.707(1) epoch 194495) v5 2014-08-13 18:06:07.341947 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.341950 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.341958 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374] r=-1 lpr=194488 pi=192969-194487/25 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.341978 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.341971 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] handle_advance_map [371,21]/[371,21,24] -- 371/371 2014-08-13 18:06:07.341980 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3f00 2014-08-13 18:06:07.341983 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.341985 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.341994 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342007 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342009 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342004 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342025 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] handle_advance_map [371,24,7]/[371,21,24] -- 371/371 2014-08-13 18:06:07.342033 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342036 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342035 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342037 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1c00 pg_notify(22.707(38) epoch 194495) v5 2014-08-13 18:06:07.342038 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342047 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342060 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.15 2014-08-13 18:06:07.342062 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1c00 2014-08-13 18:06:07.342056 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] acting_up_affected newup [371,24,7] newacting [371,21,24] 2014-08-13 18:06:07.342066 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.342075 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,21]/[371,21,24] r=-1 lpr=194489 pi=192977-194488/31 crt=192888'459 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.342083 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342085 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342091 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [371,21](371) acting [371,21,24](371)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194489-194490 up [371,21](371) acting [371,21,24](371) maybe_went_rw) : primary up 192970-194489 includes interval 2014-08-13 18:06:07.342104 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342105 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342107 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee1680 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.342100 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194488/194489/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [371,21](371) acting [371,21,24](371) maybe_went_rw) 2014-08-13 18:06:07.342112 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.342116 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.342117 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342119 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342111 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] up [371,21] -> [371,24,7], acting [371,21,24] -> [371,21,24], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> -1 2014-08-13 18:06:07.342123 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.342131 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342132 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342134 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3480 pg_notify(22.707 epoch 194495) v5 2014-08-13 18:06:07.342132 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.342141 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.342155 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.21 2014-08-13 18:06:07.342157 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3480 2014-08-13 18:06:07.342151 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.342160 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.342177 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342179 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342176 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.342190 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342192 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342186 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.342193 7f5c835ef700 20 osd.4 194495 _dispatch 0x5158000 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.342197 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.342196 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.342205 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.342210 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342211 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342205 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.342214 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.342223 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.342232 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342234 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342235 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ee7080 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.342239 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.342233 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342241 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.342243 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342244 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342255 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342256 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342258 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df4ec0 pg_notify(22.707 epoch 194495) v5 2014-08-13 18:06:07.342256 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 18:06:07.342268 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342279 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.310 2014-08-13 18:06:07.342281 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df4ec0 2014-08-13 18:06:07.342270 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342270 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [358,374](358) acting [358,374](358)): not rw, up_thru 194488 up_from 192966 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [358,374](358) acting [358,374](358) maybe_went_rw) : primary up 192966-194488 includes interval 2014-08-13 18:06:07.342284 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342293 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] acting_up_affected newup [371,24,7] newacting [371,24,7] 2014-08-13 18:06:07.342301 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342306 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342287 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [358,374](358) acting [358,374](358) maybe_went_rw) 2014-08-13 18:06:07.342305 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.342317 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342319 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342320 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1500 pg_notify(22.1f1 epoch 194495) v5 2014-08-13 18:06:07.342314 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7]/[371,21,24] r=-1 lpr=194491 pi=192977-194490/32 crt=192888'459 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.342310 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374] -> [358,374,2], acting [358,374] -> [358,374,2], acting_primary 358 -> 358, up_primary 358 -> 358, role -1 -> -1 2014-08-13 18:06:07.342328 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342333 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [371,24,7](371) acting [371,21,24](371)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194491-194491 up [371,24,7](371) acting [371,21,24](371)) : primary up 192970-194489 does not include interval 2014-08-13 18:06:07.342343 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.5 2014-08-13 18:06:07.342345 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1500 2014-08-13 18:06:07.342338 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.342344 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194491/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [371,24,7](371) acting [371,21,24](371)) 2014-08-13 18:06:07.342349 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342365 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342356 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,24,7] -> [371,24,7], acting [371,21,24] -> [371,24,7], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> -1 2014-08-13 18:06:07.342367 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342363 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.342367 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342378 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342372 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.342380 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342376 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.342383 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5b00 pg_notify(22.707(38) epoch 194495) v5 2014-08-13 18:06:07.342383 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.342385 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342393 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342395 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.342407 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.342 2014-08-13 18:06:07.342408 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5b00 2014-08-13 18:06:07.342404 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.342403 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.342413 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.342418 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.342423 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342431 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342433 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342430 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.342432 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.342439 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.342442 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.342455 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342456 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342451 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.342458 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3100 pg_notify(22.1f1(34) epoch 194495) v5 2014-08-13 18:06:07.342449 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342459 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.342469 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342480 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.368 2014-08-13 18:06:07.342482 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3100 2014-08-13 18:06:07.342478 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 18:06:07.342501 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342501 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342494 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 18:06:07.342503 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342505 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342508 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342505 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342516 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342518 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.342525 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342527 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342550 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,24,7]/[371,24,7] -- 371/371 2014-08-13 18:06:07.342561 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342563 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342572 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342569 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 18:06:07.342583 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342580 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342586 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342597 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 18:06:07.342608 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342610 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342595 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.342619 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342622 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342623 7f5c835ef700 20 osd.4 194495 _dispatch 0x524f1c0 pg_notify(22.1f1(1) epoch 194495) v5 2014-08-13 18:06:07.342619 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.342621 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342628 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] acting_up_affected newup [371,4,21] newacting [371,4,21] 2014-08-13 18:06:07.342638 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.342649 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.11 2014-08-13 18:06:07.342651 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524f1c0 2014-08-13 18:06:07.342646 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,24,7] r=-1 lpr=194492 pi=192977-194491/33 crt=192888'459 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.342646 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,2]/[358,374,2] -- 358/358 2014-08-13 18:06:07.342659 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342664 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [371,24,7](371) acting [371,24,7](371)): not rw, up_thru 194492 up_from 192970 last_epoch_clean 194483 generate_past_intervals interval(194492-194494 up [371,24,7](371) acting [371,24,7](371) maybe_went_rw) : primary up 192970-194492 includes interval 2014-08-13 18:06:07.342680 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342682 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342661 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342674 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194491/194492/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [371,24,7](371) acting [371,24,7](371) maybe_went_rw) 2014-08-13 18:06:07.342684 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.342685 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] up [371,24,7] -> [371,4,21], acting [371,24,7] -> [371,4,21], acting_primary 371 -> 371, up_primary 371 -> 371, role -1 -> 1 2014-08-13 18:06:07.342695 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342697 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342699 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a42d80 pg_notify(22.1f1(34) epoch 194495) v5 2014-08-13 18:06:07.342697 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342696 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342707 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.342715 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342724 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.54 2014-08-13 18:06:07.342727 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a42d80 2014-08-13 18:06:07.342718 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 18:06:07.342730 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.342724 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.342734 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.342733 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.342743 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.342753 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342755 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342752 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342750 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.342761 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.342765 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] acting_up_affected newup [358,374,4] newacting [358,374,4] 2014-08-13 18:06:07.342770 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.342775 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.342779 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.342786 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342788 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342790 7f5c835ef700 20 osd.4 194495 _dispatch 0x4ed2400 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.342788 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342788 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,2] r=-1 lpr=194491 pi=192969-194490/26 crt=192888'500 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.342796 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.342799 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.342801 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342802 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342797 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.342806 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.342808 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [358,374,2](358) acting [358,374,2](358)): not rw, up_thru 194491 up_from 192966 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [358,374,2](358) acting [358,374,2](358) maybe_went_rw) : primary up 192966-194491 includes interval 2014-08-13 18:06:07.342815 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342817 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [358,374,2](358) acting [358,374,2](358) maybe_went_rw) 2014-08-13 18:06:07.342825 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.342837 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342828 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] up [358,374,2] -> [358,374,4], acting [358,374,2] -> [358,374,4], acting_primary 358 -> 358, up_primary 358 -> 358, role -1 -> 2 2014-08-13 18:06:07.342839 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342849 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a43480 pg_notify(22.1f1 epoch 194495) v5 2014-08-13 18:06:07.342845 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342855 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.342863 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342872 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.47 2014-08-13 18:06:07.342874 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a43480 2014-08-13 18:06:07.342871 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.342874 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.342883 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.342883 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.342900 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.342901 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.342895 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.342895 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Reset 0.001388 8 0.001190 2014-08-13 18:06:07.342905 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.342909 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.342919 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.342915 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.342928 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.342929 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.342938 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.342938 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.342946 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.342950 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.342955 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.342963 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.342963 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.342973 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.342974 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.342988 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.343030 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.343037 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.343051 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.343060 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Reset 0.001665 8 0.001450 2014-08-13 18:06:07.343062 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,334]/[356,32] -- 356/356 2014-08-13 18:06:07.343073 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.343081 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343082 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.343085 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.343095 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343104 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,334] newacting [356,32] 2014-08-13 18:06:07.343091 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.343114 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343123 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,4,334]/[356,4,32] r=1 lpr=194487 pi=193150-194485/39 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194488 2014-08-13 18:06:07.343123 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Start 0.000041 0 0.000000 2014-08-13 18:06:07.343136 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.343142 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [356,4,334](356) acting [356,4,32](356)): not rw, up_thru 194486 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [356,4,334](356) acting [356,4,32](356) maybe_went_rw) : primary up 192969-194486 includes interval 2014-08-13 18:06:07.343152 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194485/194486/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194486-194487 up [356,4,334](356) acting [356,4,32](356) maybe_went_rw) 2014-08-13 18:06:07.343154 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.343168 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.343163 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,4,334] -> [356,334], acting [356,4,32] -> [356,32], acting_primary 356 -> 356, up_primary 356 -> 356, role 1 -> -1 2014-08-13 18:06:07.343181 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343190 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.343199 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.343209 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.343219 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.343218 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.343227 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.343235 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343245 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.343254 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.343252 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [330,28]/[17,315] -- 330/17 2014-08-13 18:06:07.343264 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 18:06:07.343272 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343272 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.343282 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.343290 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.343277 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 18:06:07.343300 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343299 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.343303 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343317 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [330,28] newacting [17,315] 2014-08-13 18:06:07.343323 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,334]/[356,32,322] -- 356/356 2014-08-13 18:06:07.343328 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343339 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343340 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [4,330,28]/[4,17,315] r=0 lpr=194487 pi=193748-194485/35 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194488 2014-08-13 18:06:07.343342 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.343351 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343361 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,334] newacting [356,32,322] 2014-08-13 18:06:07.343361 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,330,28](4) acting [4,17,315](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.343371 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343372 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194485/194486/194485) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] noting past interval(194486-194487 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) 2014-08-13 18:06:07.343380 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32] r=-1 lpr=194488 pi=193150-194487/40 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194489 2014-08-13 18:06:07.343387 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] up [4,330,28] -> [330,28], acting [4,17,315] -> [17,315], acting_primary 4 -> 17, up_primary 4 -> 330, role 0 -> -1 2014-08-13 18:06:07.343400 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] clear_primary_state 2014-08-13 18:06:07.343409 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] agent_stop 2014-08-13 18:06:07.343398 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [356,334](356) acting [356,32](356)): not rw, up_thru 194486 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [356,334](356) acting [356,32](356)) : primary up 192969-194486 does not include interval 2014-08-13 18:06:07.343420 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] on_change 2014-08-13 18:06:07.343427 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194488/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194488-194488 up [356,334](356) acting [356,32](356)) 2014-08-13 18:06:07.343429 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.343438 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:07.343438 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,334] -> [356,334], acting [356,32] -> [356,32,322], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.343449 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 18:06:07.343450 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343457 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.343460 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.343460 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 18:06:07.343470 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.343474 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 18:06:07.343481 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.343484 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] on_change 2014-08-13 18:06:07.343490 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.343495 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 18:06:07.343500 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.343506 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 18:06:07.343510 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343520 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.343530 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.343516 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_stats 2014-08-13 18:06:07.343539 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.343547 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.343546 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.343558 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_role_change 2014-08-13 18:06:07.343558 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343568 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.343579 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.343581 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,334]/[356,32,322] -- 356/356 2014-08-13 18:06:07.343588 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.343596 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343599 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.343599 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343609 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343618 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343622 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [330,28]/[17,315,327] -- 330/17 2014-08-13 18:06:07.343640 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343641 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,322,32]/[356,32,322] -- 356/356 2014-08-13 18:06:07.343643 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.343654 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343653 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343657 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.343662 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [330,28] newacting [17,315,327] 2014-08-13 18:06:07.343666 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343672 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343676 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,322,32] newacting [356,32,322] 2014-08-13 18:06:07.343685 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315] r=-1 lpr=194488 pi=193748-194487/36 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194489 2014-08-13 18:06:07.343686 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343695 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,334]/[356,32,322] r=-1 lpr=194489 pi=193150-194488/41 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194491 2014-08-13 18:06:07.343703 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [330,28](330) acting [17,315](17)): not rw, up_thru 194486 up_from 192970 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [330,28](330) acting [17,315](17)) : primary up 192970-194486 does not include interval 2014-08-13 18:06:07.343713 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [356,334](356) acting [356,32,322](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [356,334](356) acting [356,32,322](356) maybe_went_rw) : primary up 192969-194489 includes interval 2014-08-13 18:06:07.343713 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194488/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] noting past interval(194488-194488 up [330,28](330) acting [17,315](17)) 2014-08-13 18:06:07.343724 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194488/194489/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194489-194490 up [356,334](356) acting [356,32,322](356) maybe_went_rw) 2014-08-13 18:06:07.343725 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] up [330,28] -> [330,28], acting [17,315] -> [17,315,327], acting_primary 17 -> 17, up_primary 330 -> 330, role -1 -> -1 2014-08-13 18:06:07.343735 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,334] -> [356,322,32], acting [356,32,322] -> [356,32,322], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.343740 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.343747 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343750 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.343757 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.343759 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.343766 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.343769 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.343776 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.343778 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.343786 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.343791 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.343795 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.343801 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.343805 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.343812 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.343818 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.343822 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.343828 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.343832 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.343839 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.343842 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.343848 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.343855 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343858 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343879 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [330,28]/[17,315,327] -- 330/17 2014-08-13 18:06:07.343894 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343895 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,322,32]/[356,322,32] -- 356/356 2014-08-13 18:06:07.343899 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.343912 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343909 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343915 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.343920 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.343925 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.343935 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [356,322,32] newacting [356,322,32] 2014-08-13 18:06:07.343944 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.343953 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32]/[356,32,322] r=-1 lpr=194491 pi=193150-194490/42 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194492 2014-08-13 18:06:07.343962 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [327,17,315]/[17,315,327] -- 327/17 2014-08-13 18:06:07.343977 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.343970 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [356,322,32](356) acting [356,32,322](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [356,322,32](356) acting [356,32,322](356)) : primary up 192969-194489 does not include interval 2014-08-13 18:06:07.343980 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194491/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] noting past interval(194491-194491 up [356,322,32](356) acting [356,32,322](356)) 2014-08-13 18:06:07.343980 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.343990 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] up [356,322,32] -> [356,322,32], acting [356,32,322] -> [356,322,32], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.344001 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.343999 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344010 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.344012 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [327,17,315] newacting [17,315,327] 2014-08-13 18:06:07.344019 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.344023 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.344029 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.344033 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [330,28]/[17,315,327] r=-1 lpr=194489 pi=193748-194488/37 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194491 2014-08-13 18:06:07.344038 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.344047 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.344053 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [330,28](330) acting [17,315,327](17)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [330,28](330) acting [17,315,327](17) maybe_went_rw) : primary up 192970-194489 includes interval 2014-08-13 18:06:07.344056 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.344065 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.344064 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194488/194489/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] noting past interval(194489-194490 up [330,28](330) acting [17,315,327](17) maybe_went_rw) 2014-08-13 18:06:07.344075 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.344075 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] up [330,28] -> [327,17,315], acting [17,315,327] -> [17,315,327], acting_primary 17 -> 17, up_primary 330 -> 327, role -1 -> -1 2014-08-13 18:06:07.344084 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.344091 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.344093 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.344101 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.344103 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344110 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344120 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.344127 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,322,32]/[356,322,32] -- 356/356 2014-08-13 18:06:07.344129 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.344141 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344144 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.344138 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.344153 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344160 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] on_change 2014-08-13 18:06:07.344162 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344171 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.344181 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.344190 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.344189 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,322,32]/[356,322,32] -- 356/356 2014-08-13 18:06:07.344203 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344199 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.344205 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.344213 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344214 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344224 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344241 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] handle_advance_map [327,17,315]/[327,17,315] -- 327/327 2014-08-13 18:06:07.344256 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344253 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,334] -- 356/356 2014-08-13 18:06:07.344268 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344259 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.344270 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.344273 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344280 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344283 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] acting_up_affected newup [327,17,315] newacting [327,17,315] 2014-08-13 18:06:07.344289 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,334] 2014-08-13 18:06:07.344294 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.344299 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.344304 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315]/[17,315,327] r=-1 lpr=194491 pi=193748-194490/38 crt=194422'15454 lcod 0'0 remapped NOTIFY m=59] set_last_peering_reset 194492 2014-08-13 18:06:07.344309 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,322,32] r=-1 lpr=194492 pi=193150-194491/43 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194495 2014-08-13 18:06:07.344326 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [356,322,32](356) acting [356,322,32](356)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [356,322,32](356) acting [356,322,32](356) maybe_went_rw) : primary up 192969-194492 includes interval 2014-08-13 18:06:07.344326 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)): not rw, up_thru 194489 up_from 192970 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)) : primary up 192970-194489 does not include interval 2014-08-13 18:06:07.344336 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194491/194492/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] noting past interval(194492-194494 up [356,322,32](356) acting [356,322,32](356) maybe_went_rw) 2014-08-13 18:06:07.344337 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194491/194488) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] noting past interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 18:06:07.344348 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] up [356,322,32] -> [356,4,334], acting [356,322,32] -> [356,4,334], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> 1 2014-08-13 18:06:07.344360 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.344348 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] up [327,17,315] -> [327,17,315], acting [17,315,327] -> [327,17,315], acting_primary 17 -> 327, up_primary 327 -> 327, role -1 -> -1 2014-08-13 18:06:07.344369 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.344378 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.344376 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.344387 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.344388 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.344396 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.344397 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344405 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.344407 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.344413 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.344417 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.344423 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.344426 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.344433 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.344438 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.344442 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] on_role_change 2014-08-13 18:06:07.344449 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.344451 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.344459 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.344460 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.344468 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.344468 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] [17,315,327] -> [327,17,315], acting primary 17 -> 327 2014-08-13 18:06:07.344479 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344487 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] cancel_recovery 2014-08-13 18:06:07.344496 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] clear_recovery_state 2014-08-13 18:06:07.344506 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344489 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:07.344530 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [327,17,315]/[327,17,315] -- 327/327 2014-08-13 18:06:07.344547 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344550 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.344559 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344562 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 18:06:07.344568 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344573 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:07.344583 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.358205 10 0.001228 2014-08-13 18:06:07.344595 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 18:06:07.344590 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [327,17,315]/[327,17,315] -- 327/327 2014-08-13 18:06:07.344607 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344604 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 18:06:07.344609 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.344613 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 18:06:07.344619 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344625 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.344629 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.344636 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 18:06:07.344648 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.344647 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] handle_advance_map [4,330,28]/[4,330,28] -- 4/4 2014-08-13 18:06:07.344664 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344658 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.344667 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: Reset advmap 2014-08-13 18:06:07.344677 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.344685 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] acting_up_affected newup [4,330,28] newacting [4,330,28] 2014-08-13 18:06:07.344694 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.344703 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [327,17,315] r=-1 lpr=194492 pi=193748-194491/39 crt=194422'15454 lcod 0'0 inactive NOTIFY m=59] set_last_peering_reset 194495 2014-08-13 18:06:07.344720 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.344723 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [327,17,315](327) acting [327,17,315](327)): not rw, up_thru 194492 up_from 192966 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) : primary up 192966-194492 includes interval 2014-08-13 18:06:07.344747 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194491/194492/194492) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] noting past interval(194492-194494 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 18:06:07.344747 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 18:06:07.344762 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.344760 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] up [327,17,315] -> [4,330,28], acting [327,17,315] -> [4,330,28], acting_primary 327 -> 4, up_primary 327 -> 4, role -1 -> 0 2014-08-13 18:06:07.344765 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.344775 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343] newacting [397,343] 2014-08-13 18:06:07.344785 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.344773 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] clear_primary_state 2014-08-13 18:06:07.344794 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.291129 3 0.000117 2014-08-13 18:06:07.344805 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.291177 0 0.000000 2014-08-13 18:06:07.344801 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] agent_stop 2014-08-13 18:06:07.344815 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.344815 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.344824 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194486 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.344827 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] publish_stats_to_osd 194495:22350 2014-08-13 18:06:07.344833 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194488 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.344837 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344842 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194488 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.344848 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_copy_ops 2014-08-13 18:06:07.344851 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194488 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343] newacting [397,343] 2014-08-13 18:06:07.344860 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194488 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.344858 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] cancel_flush_ops 2014-08-13 18:06:07.344868 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343,4] r=2 lpr=194488 pi=192965-194484/24 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.344870 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344878 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344887 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] requeue_ops 2014-08-13 18:06:07.344888 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [397,343,4](397) acting [397,343,4](397)): not rw, up_thru 194485 up_from 192958 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [397,343,4](397) acting [397,343,4](397) maybe_went_rw) : primary up 192958-194485 includes interval 2014-08-13 18:06:07.344897 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change_cleanup 2014-08-13 18:06:07.344898 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194485/194485/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [397,343,4](397) acting [397,343,4](397) maybe_went_rw) 2014-08-13 18:06:07.344906 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] on_change 2014-08-13 18:06:07.344908 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343,4] -> [397,343], acting [397,343,4] -> [397,343], acting_primary 397 -> 397, up_primary 397 -> 397, role 2 -> -1 2014-08-13 18:06:07.344919 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.344919 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] exit NotTrimming 2014-08-13 18:06:07.344928 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.344930 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive NOTIFY m=59] enter NotTrimming 2014-08-13 18:06:07.344936 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.344940 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] clear_stats 2014-08-13 18:06:07.344946 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.344950 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 18:06:07.344947 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 18:06:07.344956 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.344960 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] on_role_change 2014-08-13 18:06:07.344965 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.344970 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] requeue_ops 2014-08-13 18:06:07.344974 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.344979 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] cancel_recovery 2014-08-13 18:06:07.344984 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.344993 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.344992 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] clear_recovery_state 2014-08-13 18:06:07.345002 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.345004 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350119 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_activate_map 2014-08-13 18:06:07.345010 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.350138 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] update_heartbeat_peers -> 4,28,330 2014-08-13 18:06:07.350152 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.350154 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] take_waiters 2014-08-13 18:06:07.350164 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Reset 0.703501 10 0.006630 2014-08-13 18:06:07.350149 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.350176 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started 2014-08-13 18:06:07.350176 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.350187 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Start 2014-08-13 18:06:07.350189 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350208 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: transitioning to Primary 2014-08-13 18:06:07.350214 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 18:06:07.350226 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350224 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Start 0.000037 0 0.000000 2014-08-13 18:06:07.350230 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350237 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 18:06:07.350239 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350247 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering 2014-08-13 18:06:07.350249 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350257 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.350269 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.350266 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343]/[397,343] -- 397/397 2014-08-13 18:06:07.350280 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350283 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350281 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194492-194494 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 18:06:07.350291 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350293 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 18:06:07.350300 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350305 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194489-194490 up [330,28](330) acting [17,315,327](17) maybe_went_rw) 2014-08-13 18:06:07.350319 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 18:06:07.350316 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194488-194488 up [330,28](330) acting [17,315](17)) 2014-08-13 18:06:07.350330 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350333 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350330 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194486-194487 up [4,330,28](4) acting [4,17,315](4) maybe_went_rw) 2014-08-13 18:06:07.350341 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350342 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194485-194485 up [4,330,28](4) acting [4,330,28](4)) 2014-08-13 18:06:07.350349 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343,380] newacting [397,343,380] 2014-08-13 18:06:07.350353 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194483-194484 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 18:06:07.350359 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.350369 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343] r=-1 lpr=194488 pi=192965-194487/25 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.350365 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194482-194482 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 18:06:07.350379 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 18:06:07.350387 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [397,343](397) acting [397,343](397)): not rw, up_thru 194488 up_from 192958 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [397,343](397) acting [397,343](397) maybe_went_rw) : primary up 192958-194488 includes interval 2014-08-13 18:06:07.350393 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.350397 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194488/194488/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [397,343](397) acting [397,343](397) maybe_went_rw) 2014-08-13 18:06:07.350406 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] publish_stats_to_osd 194495:22351 2014-08-13 18:06:07.350407 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343] -> [397,343,380], acting [397,343] -> [397,343,380], acting_primary 397 -> 397, up_primary 397 -> 397, role -1 -> -1 2014-08-13 18:06:07.350419 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.350417 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.17 2014-08-13 18:06:07.350427 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.350428 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.28 2014-08-13 18:06:07.350435 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.350439 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.315 2014-08-13 18:06:07.350445 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.350454 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.350462 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.350470 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.350450 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.327 2014-08-13 18:06:07.350479 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.350482 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: querying info from osd.330 2014-08-13 18:06:07.350488 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.350498 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.350495 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.350506 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.350512 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.350515 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350535 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 18:06:07.350546 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350548 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350556 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350564 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350580 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.350582 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 18:06:07.350592 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350594 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350602 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350610 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350619 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 18:06:07.350628 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,380]/[397,343,380] -- 397/397 2014-08-13 18:06:07.350639 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350639 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350641 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350643 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.350650 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350657 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [47,378] newacting [19,65] 2014-08-13 18:06:07.350659 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350667 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.350677 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 18:06:07.350687 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.350690 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350698 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.350677 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.307483 3 0.000120 2014-08-13 18:06:07.350706 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [397,343,4] newacting [397,343,4] 2014-08-13 18:06:07.350715 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.350713 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.307561 0 0.000000 2014-08-13 18:06:07.350724 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,380] r=-1 lpr=194491 pi=192965-194490/26 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.350726 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.350736 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.350741 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [397,343,380](397) acting [397,343,380](397)): not rw, up_thru 194491 up_from 192958 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [397,343,380](397) acting [397,343,380](397) maybe_went_rw) : primary up 192958-194491 includes interval 2014-08-13 18:06:07.350747 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.350750 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194491/194491/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [397,343,380](397) acting [397,343,380](397) maybe_went_rw) 2014-08-13 18:06:07.350759 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.350761 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] up [397,343,380] -> [397,343,4], acting [397,343,380] -> [397,343,4], acting_primary 397 -> 397, up_primary 397 -> 397, role -1 -> 2 2014-08-13 18:06:07.350769 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [47,378] newacting [19,65] 2014-08-13 18:06:07.350776 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.350779 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.350784 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.350789 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [4,47,378]/[19,65] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.350793 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.350801 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.350809 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.350807 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,47,378](4) acting [19,65](19)): not rw, up_thru 194485 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194485-194487 up [4,47,378](4) acting [19,65](19) maybe_went_rw) : primary up 192976-194485 includes interval 2014-08-13 18:06:07.350817 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.350820 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194485-194487 up [4,47,378](4) acting [19,65](19) maybe_went_rw) 2014-08-13 18:06:07.350826 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.350836 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.350845 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.350832 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] up [4,47,378] -> [47,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 4 -> 47, role -1 -> -1 2014-08-13 18:06:07.350853 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.350861 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.350859 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.350869 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.350870 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.350877 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.350880 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.350886 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350894 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.350896 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.350904 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.350913 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.350922 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.350933 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.350934 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.350943 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.350944 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.350953 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.006138 8 0.005964 2014-08-13 18:06:07.350953 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.350966 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.350965 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.350975 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.350977 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.350983 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.350994 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.351005 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.351004 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351020 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351018 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.351023 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351028 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.351034 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351049 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351071 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351080 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.351084 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351086 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351098 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351107 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351115 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 18:06:07.351130 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351131 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351133 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.351144 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351143 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335] newacting [30,335] 2014-08-13 18:06:07.351151 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.351151 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351162 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.294897 3 0.000117 2014-08-13 18:06:07.351179 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.294952 0 0.000000 2014-08-13 18:06:07.351189 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.351197 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194486 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.351166 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351206 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351206 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [47,19,378] newacting [19,65] 2014-08-13 18:06:07.351215 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351221 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.351224 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335] newacting [30,335] 2014-08-13 18:06:07.351231 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,378]/[19,65] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.351233 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.351242 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335,4] r=2 lpr=194488 pi=192976-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.351249 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [47,378](47) acting [19,65](19)): not rw, up_thru 194488 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194488-194490 up [47,378](47) acting [19,65](19) maybe_went_rw) : primary up 192976-194488 includes interval 2014-08-13 18:06:07.351260 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [30,335,4](30) acting [30,335,4](30)): not rw, up_thru 194485 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [30,335,4](30) acting [30,335,4](30) maybe_went_rw) : primary up 192976-194485 includes interval 2014-08-13 18:06:07.351259 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194488-194490 up [47,378](47) acting [19,65](19) maybe_went_rw) 2014-08-13 18:06:07.351269 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194485/194485/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [30,335,4](30) acting [30,335,4](30) maybe_went_rw) 2014-08-13 18:06:07.351274 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] up [47,378] -> [47,19,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 47 -> 47, role -1 -> -1 2014-08-13 18:06:07.351280 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335,4] -> [30,335], acting [30,335,4] -> [30,335], acting_primary 30 -> 30, up_primary 30 -> 30, role 2 -> -1 2014-08-13 18:06:07.351287 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.351290 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.351296 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.351299 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.351308 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.351306 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.351316 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.351324 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.351320 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.351333 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.351333 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.351342 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.351343 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.351351 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.351352 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.351360 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.351363 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.351369 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.351377 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.351377 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.351385 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.351393 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.351397 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.351402 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351408 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.351419 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351424 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 18:06:07.351434 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351437 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351445 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351449 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351454 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351442 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.351466 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351469 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351474 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335]/[30,335] -- 30/30 2014-08-13 18:06:07.351484 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351479 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351486 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351492 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351495 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351503 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351536 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351555 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351557 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351567 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351561 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 18:06:07.351578 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351577 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351581 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351590 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351598 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335,25] newacting [30,335,25] 2014-08-13 18:06:07.351606 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.351605 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [47,19,378]/[19,65] -- 47/19 2014-08-13 18:06:07.351619 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351615 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335] r=-1 lpr=194488 pi=192976-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.351622 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351631 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [30,335](30) acting [30,335](30)): not rw, up_thru 194488 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [30,335](30) acting [30,335](30) maybe_went_rw) : primary up 192976-194488 includes interval 2014-08-13 18:06:07.351632 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351641 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194488/194488/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [30,335](30) acting [30,335](30) maybe_went_rw) 2014-08-13 18:06:07.351642 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351651 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335] -> [30,335,25], acting [30,335] -> [30,335,25], acting_primary 30 -> 30, up_primary 30 -> 30, role -1 -> -1 2014-08-13 18:06:07.351665 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.351668 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 18:06:07.351673 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.351683 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351681 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.351685 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351690 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.351696 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.351700 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.351706 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [4,47,378] newacting [19,65] 2014-08-13 18:06:07.351709 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.351717 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.351716 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.351727 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.351726 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [47,19,378]/[19,65] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.351736 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.351745 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.351747 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [47,19,378](47) acting [19,65](19)): not rw, up_thru 194491 up_from 192976 last_epoch_clean 194463 generate_past_intervals interval(194491-194494 up [47,19,378](47) acting [19,65](19) maybe_went_rw) : primary up 192976-194491 includes interval 2014-08-13 18:06:07.351753 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.351759 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194491-194494 up [47,19,378](47) acting [19,65](19) maybe_went_rw) 2014-08-13 18:06:07.351762 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351770 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] up [47,19,378] -> [4,47,378], acting [19,65] -> [19,65], acting_primary 19 -> 19, up_primary 47 -> 4, role -1 -> -1 2014-08-13 18:06:07.351784 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.351786 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 18:06:07.351797 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351796 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.351799 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351806 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.351808 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351817 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351816 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.351827 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.351838 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 18:06:07.351836 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.351850 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351852 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351851 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.351861 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351870 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351865 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.351882 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.351892 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.351890 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,25]/[30,335,25] -- 30/30 2014-08-13 18:06:07.351904 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351901 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.351906 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351914 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351923 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351915 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.351935 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 18:06:07.351943 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 18:06:07.351953 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.351955 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.351963 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.351962 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.351972 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [30,335,4] newacting [30,335,4] 2014-08-13 18:06:07.351976 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 18:06:07.351981 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.351987 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.001261 8 0.001033 2014-08-13 18:06:07.351990 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,25] r=-1 lpr=194491 pi=192976-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.352000 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started 2014-08-13 18:06:07.352007 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [30,335,25](30) acting [30,335,25](30)): not rw, up_thru 194491 up_from 192976 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [30,335,25](30) acting [30,335,25](30) maybe_went_rw) : primary up 192976-194491 includes interval 2014-08-13 18:06:07.352011 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start 2014-08-13 18:06:07.352017 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194491/194491/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [30,335,25](30) acting [30,335,25](30) maybe_went_rw) 2014-08-13 18:06:07.352021 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.352027 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] up [30,335,25] -> [30,335,4], acting [30,335,25] -> [30,335,4], acting_primary 30 -> 30, up_primary 30 -> 30, role -1 -> 2 2014-08-13 18:06:07.352038 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352036 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000025 0 0.000000 2014-08-13 18:06:07.352047 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.352048 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray 2014-08-13 18:06:07.352056 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352061 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.352065 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.352072 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.352074 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.352082 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.352091 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352099 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.352107 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.352117 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.352115 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.352123 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.352138 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.352146 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352156 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.352163 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 18:06:07.352180 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352199 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.352199 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.352209 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.352219 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.001030 8 0.000800 2014-08-13 18:06:07.352211 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360] newacting [373,360] 2014-08-13 18:06:07.352230 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.352238 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.352243 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.352246 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.352256 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.306508 3 0.000122 2014-08-13 18:06:07.352257 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.352268 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.352267 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started 0.306557 0 0.000000 2014-08-13 18:06:07.352279 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.352280 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.352290 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.352287 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.352301 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352310 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352319 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360] newacting [373,360] 2014-08-13 18:06:07.352327 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.352340 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.352336 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.352326 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] flushed 2014-08-13 18:06:07.352356 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [373,360,4](373) acting [373,360,4](373)): not rw, up_thru 194485 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [373,360,4](373) acting [373,360,4](373) maybe_went_rw) : primary up 192978-194485 includes interval 2014-08-13 18:06:07.352365 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194485/194485/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [373,360,4](373) acting [373,360,4](373) maybe_went_rw) 2014-08-13 18:06:07.352369 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 18:06:07.352375 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360,4] -> [373,360], acting [373,360,4] -> [373,360], acting_primary 373 -> 373, up_primary 373 -> 373, role 2 -> -1 2014-08-13 18:06:07.352385 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352386 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352389 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.352395 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.352399 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.352404 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352408 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.352432 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.352436 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.294045 2 0.000162 2014-08-13 18:06:07.352443 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.352450 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.352453 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.352459 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.294076 0 0.000000 2014-08-13 18:06:07.352465 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352471 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.294097 0 0.000000 2014-08-13 18:06:07.352476 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.352482 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Started 0.294145 0 0.000000 2014-08-13 18:06:07.352485 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.352492 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.352494 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.352501 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194486 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.352503 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352510 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194488 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.352514 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.352520 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194488 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.352522 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.352529 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194488 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [50,289] newacting [50,289] 2014-08-13 18:06:07.352532 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352538 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194488 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.352547 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [4,50,289] r=0 lpr=194488 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.352554 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 18:06:07.352569 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352564 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,50,289](4) acting [4,50,289](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.352571 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352573 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194485/194485/194485) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] noting past interval(194485-194487 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 18:06:07.352583 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] up [4,50,289] -> [50,289], acting [4,50,289] -> [50,289], acting_primary 4 -> 50, up_primary 4 -> 50, role 0 -> -1 2014-08-13 18:06:07.352593 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.352580 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352590 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.352602 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.352605 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352611 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.352620 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.352627 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.352627 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360]/[373,360] -- 373/373 2014-08-13 18:06:07.352638 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352637 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.352641 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352645 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.352654 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.352650 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352662 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.352664 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352671 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.352679 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.352684 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 18:06:07.352688 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.352695 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352697 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352698 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352705 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.352707 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352713 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352719 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360,355] newacting [373,360,355] 2014-08-13 18:06:07.352721 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.352729 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.352728 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.352738 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.352739 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352754 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [373,360](373) acting [373,360](373)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [373,360](373) acting [373,360](373) maybe_went_rw) : primary up 192978-194489 includes interval 2014-08-13 18:06:07.352761 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 18:06:07.352773 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352766 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194488/194488/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [373,360](373) acting [373,360](373) maybe_went_rw) 2014-08-13 18:06:07.352775 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352776 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360] -> [373,360,355], acting [373,360] -> [373,360,355], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> -1 2014-08-13 18:06:07.352784 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352788 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352793 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352797 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.352808 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352812 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [50,289]/[50,289] -- 50/50 2014-08-13 18:06:07.352823 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352817 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.352825 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352826 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.352834 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352835 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.352842 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352844 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352853 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.352863 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 18:06:07.352865 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.352874 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352874 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.352876 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352882 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.352886 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352894 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [32,50,289] newacting [32,50,289] 2014-08-13 18:06:07.352903 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.352893 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352910 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [50,289] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.352926 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [50,289](50) acting [50,289](50)): not rw, up_thru 194489 up_from 192979 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) : primary up 192979-194489 includes interval 2014-08-13 18:06:07.352931 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 18:06:07.352935 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194488/194488/194488) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 18:06:07.352943 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352945 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] up [50,289] -> [32,50,289], acting [50,289] -> [32,50,289], acting_primary 50 -> 32, up_primary 50 -> 32, role -1 -> -1 2014-08-13 18:06:07.352946 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.352956 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.352958 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.352964 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.352968 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.352973 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.352984 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.352987 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 18:06:07.352998 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.352993 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.353001 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.353001 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353010 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.353016 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353018 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.353025 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353027 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.353036 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] [50,289] -> [32,50,289], acting primary 50 -> 32 2014-08-13 18:06:07.353044 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,355]/[373,360,355] -- 373/373 2014-08-13 18:06:07.353056 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353050 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.353058 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.353058 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353068 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353071 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353080 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353091 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 18:06:07.353101 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353097 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 18:06:07.353104 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353109 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353113 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353114 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353121 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353123 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353131 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] acting_up_affected newup [373,360,4] newacting [373,360,4] 2014-08-13 18:06:07.353142 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 18:06:07.353152 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353140 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.353154 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353155 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,355] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'492 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.353163 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353177 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353176 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [373,360,355](373) acting [373,360,355](373)): not rw, up_thru 194491 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [373,360,355](373) acting [373,360,355](373) maybe_went_rw) : primary up 192978-194491 includes interval 2014-08-13 18:06:07.353187 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194491/194491/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [373,360,355](373) acting [373,360,355](373) maybe_went_rw) 2014-08-13 18:06:07.353197 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] up [373,360,355] -> [373,360,4], acting [373,360,355] -> [373,360,4], acting_primary 373 -> 373, up_primary 373 -> 373, role -1 -> 2 2014-08-13 18:06:07.353198 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [32,50,289]/[32,50,289] -- 32/32 2014-08-13 18:06:07.353211 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353208 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.353216 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.353214 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353225 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353224 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353234 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353237 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.353245 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.353253 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 18:06:07.353264 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.353253 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.353275 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353282 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.353283 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,50,289] newacting [4,50,289] 2014-08-13 18:06:07.353292 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.353292 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.353301 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [32,50,289] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'442 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.353307 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.353317 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.353320 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [32,50,289](32) acting [32,50,289](32)): not rw, up_thru 194491 up_from 192973 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) : primary up 192973-194491 includes interval 2014-08-13 18:06:07.353325 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353329 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194491/194491/194491) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 18:06:07.353334 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.353343 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.353340 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [32,50,289] -> [4,50,289], acting [32,50,289] -> [4,50,289], acting_primary 32 -> 4, up_primary 32 -> 4, role -1 -> 0 2014-08-13 18:06:07.353352 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.353353 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353361 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.353363 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.353369 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.353379 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2707 2014-08-13 18:06:07.353388 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353397 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.353405 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.353405 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.353414 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353416 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.353422 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353425 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Reset 0.001146 8 0.001005 2014-08-13 18:06:07.353431 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.353439 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.353440 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.353448 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.353450 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.353457 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.353459 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.353466 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.353470 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.353475 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.353481 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.353485 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.353493 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.353493 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.353502 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.353507 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.353510 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.353518 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.353527 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353537 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.353547 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,50,289 unchanged 2014-08-13 18:06:07.353557 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.353556 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.353565 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001073 8 0.000970 2014-08-13 18:06:07.353576 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.353585 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.353594 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.353604 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.353614 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.353606 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 18:06:07.353629 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353624 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.353633 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.353633 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.353642 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.353644 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.353652 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 18:06:07.353664 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 18:06:07.353653 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364] newacting [356,364] 2014-08-13 18:06:07.353674 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 18:06:07.353679 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.353685 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,32,50,289 down blocked_by {} 2014-08-13 18:06:07.353693 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364,4] r=2 lpr=194487 pi=193151-194485/31 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194488 2014-08-13 18:06:07.353696 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.353706 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2708 2014-08-13 18:06:07.353712 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [356,364,4](356) acting [356,364,4](356)): not rw, up_thru 194486 up_from 192969 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [356,364,4](356) acting [356,364,4](356) maybe_went_rw) : primary up 192969-194486 includes interval 2014-08-13 18:06:07.353716 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.32 2014-08-13 18:06:07.353722 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194486-194487 up [356,364,4](356) acting [356,364,4](356) maybe_went_rw) 2014-08-13 18:06:07.353727 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.50 2014-08-13 18:06:07.353737 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 18:06:07.353733 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364,4] -> [356,364], acting [356,364,4] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role 2 -> -1 2014-08-13 18:06:07.353747 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.353748 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.353757 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.353758 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.353768 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.353778 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.353793 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.353789 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.353798 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.353806 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.353816 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.353826 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.353821 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 18:06:07.353841 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353835 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_role_change 2014-08-13 18:06:07.353845 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: Reset advmap 2014-08-13 18:06:07.353844 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.353856 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.353855 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.353866 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.353867 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] acting_up_affected newup [344,296] newacting [296,320] 2014-08-13 18:06:07.353876 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353878 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.353887 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [4,344,296]/[4,296,320] r=0 lpr=194487 pi=193679-194485/37 crt=194422'19613 lcod 0'0 mlcod 0'0 inactive m=47] set_last_peering_reset 194488 2014-08-13 18:06:07.353906 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,344,296](4) acting [4,296,320](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194466 generate_past_intervals interval(194486-194487 up [4,344,296](4) acting [4,296,320](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.353901 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.353917 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194485/194486/194486) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] noting past interval(194486-194487 up [4,344,296](4) acting [4,296,320](4) maybe_went_rw) 2014-08-13 18:06:07.353928 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] up [4,344,296] -> [344,296], acting [4,296,320] -> [296,320], acting_primary 4 -> 296, up_primary 4 -> 344, role 0 -> -1 2014-08-13 18:06:07.353927 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 18:06:07.353946 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.353940 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] clear_primary_state 2014-08-13 18:06:07.353949 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.353950 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] agent_stop 2014-08-13 18:06:07.353959 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] on_change 2014-08-13 18:06:07.353959 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.353969 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.353973 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.353978 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] requeue_ops 2014-08-13 18:06:07.353988 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] cancel_copy_ops 2014-08-13 18:06:07.353994 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364]/[356,364] -- 356/356 2014-08-13 18:06:07.353997 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] cancel_flush_ops 2014-08-13 18:06:07.354006 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] on_change_cleanup 2014-08-13 18:06:07.354014 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354015 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] on_change 2014-08-13 18:06:07.354017 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354025 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] exit NotTrimming 2014-08-13 18:06:07.354027 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354034 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped m=47] enter NotTrimming 2014-08-13 18:06:07.354039 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354044 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_stats 2014-08-13 18:06:07.354056 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.354059 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,381]/[356,364,381] -- 356/356 2014-08-13 18:06:07.354065 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_role_change 2014-08-13 18:06:07.354072 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354074 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.354075 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354083 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 18:06:07.354085 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354092 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 18:06:07.354094 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364,381] newacting [356,364,381] 2014-08-13 18:06:07.354102 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354107 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354116 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364] r=-1 lpr=194488 pi=193151-194487/32 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194491 2014-08-13 18:06:07.354124 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 18:06:07.354138 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354132 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [356,364](356) acting [356,364](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [356,364](356) acting [356,364](356) maybe_went_rw) : primary up 192969-194489 includes interval 2014-08-13 18:06:07.354141 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354150 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354142 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194488-194490 up [356,364](356) acting [356,364](356) maybe_went_rw) 2014-08-13 18:06:07.354163 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364] -> [356,364,381], acting [356,364] -> [356,364,381], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.354161 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354174 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.354185 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [344,296]/[296,320] -- 344/296 2014-08-13 18:06:07.354198 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354200 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354183 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354210 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354212 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.354219 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354223 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.354232 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.354240 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.354244 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[296,320] -- 320/296 2014-08-13 18:06:07.354257 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354252 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:07.354259 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354265 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.354269 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354276 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.354278 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [320,307,55] newacting [296,320] 2014-08-13 18:06:07.354286 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.354288 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354297 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [344,296]/[296,320] r=-1 lpr=194488 pi=193679-194487/38 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194491 2014-08-13 18:06:07.354297 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.354314 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [344,296](344) acting [296,320](296)): not rw, up_thru 194488 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194488-194490 up [344,296](344) acting [296,320](296) maybe_went_rw) : primary up 192967-194488 includes interval 2014-08-13 18:06:07.354321 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354325 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194488/194488/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194488-194490 up [344,296](344) acting [296,320](296) maybe_went_rw) 2014-08-13 18:06:07.354335 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [344,296] -> [320,307,55], acting [296,320] -> [296,320], acting_primary 296 -> 296, up_primary 344 -> 320, role -1 -> -1 2014-08-13 18:06:07.354346 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.354347 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,381]/[356,364] -- 356/356 2014-08-13 18:06:07.354355 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354367 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354364 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.354370 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354374 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 18:06:07.354380 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354386 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 18:06:07.354390 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] acting_up_affected newup [356,364,381] newacting [356,364] 2014-08-13 18:06:07.354397 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 18:06:07.354403 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354406 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.354413 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381] r=-1 lpr=194491 pi=193151-194490/33 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] set_last_peering_reset 194492 2014-08-13 18:06:07.354416 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 18:06:07.354425 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 18:06:07.354430 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [356,364,381](356) acting [356,364,381](356)): not rw, up_thru 194489 up_from 192969 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [356,364,381](356) acting [356,364,381](356)) : primary up 192969-194489 does not include interval 2014-08-13 18:06:07.354435 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 18:06:07.354444 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 18:06:07.354441 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] noting past interval(194491-194491 up [356,364,381](356) acting [356,364,381](356)) 2014-08-13 18:06:07.354455 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354456 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] up [356,364,381] -> [356,364,381], acting [356,364,381] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.354468 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.354477 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354480 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[320,307,296] -- 320/320 2014-08-13 18:06:07.354486 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.354495 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354497 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354499 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.354507 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354508 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.354516 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [320,307,55] newacting [320,307,296] 2014-08-13 18:06:07.354518 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.354526 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354527 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.354536 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[296,320] r=-1 lpr=194491 pi=193679-194490/39 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194492 2014-08-13 18:06:07.354538 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.354551 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.354553 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [320,307,55](320) acting [296,320](296)): not rw, up_thru 194488 up_from 192967 last_epoch_clean 194466 generate_past_intervals interval(194491-194491 up [320,307,55](320) acting [296,320](296)) : primary up 192967-194488 does not include interval 2014-08-13 18:06:07.354564 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194491/194488) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194491-194491 up [320,307,55](320) acting [296,320](296)) 2014-08-13 18:06:07.354574 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [320,307,55] -> [320,307,55], acting [296,320] -> [320,307,296], acting_primary 296 -> 320, up_primary 320 -> 320, role -1 -> -1 2014-08-13 18:06:07.354561 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.354586 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.354595 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354593 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.354604 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.354608 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354614 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 18:06:07.354624 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 18:06:07.354632 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 18:06:07.354631 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,381]/[356,364] -- 356/356 2014-08-13 18:06:07.354642 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.354650 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354651 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 18:06:07.354653 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354661 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 18:06:07.354663 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354671 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] [296,320] -> [320,307,296], acting primary 296 -> 320 2014-08-13 18:06:07.354673 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354687 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 18:06:07.354696 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 18:06:07.354703 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,381]/[356,364] -- 356/356 2014-08-13 18:06:07.354706 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354717 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354720 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354729 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354731 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[320,307,296] -- 320/320 2014-08-13 18:06:07.354746 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354739 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354748 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354758 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354760 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,4]/[356,364] -- 356/356 2014-08-13 18:06:07.354774 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354767 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354777 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:07.354786 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.354792 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [320,307,55]/[320,307,296] -- 320/320 2014-08-13 18:06:07.354805 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354798 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364] 2014-08-13 18:06:07.354808 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354808 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354818 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,381]/[356,364] r=-1 lpr=194492 pi=193151-194491/34 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194495 2014-08-13 18:06:07.354819 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354828 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.354836 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [356,364,381](356) acting [356,364](356)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [356,364,381](356) acting [356,364](356) maybe_went_rw) : primary up 192969-194492 includes interval 2014-08-13 18:06:07.354847 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [4,344,296]/[320,307,296] -- 4/320 2014-08-13 18:06:07.354849 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] noting past interval(194492-194494 up [356,364,381](356) acting [356,364](356) maybe_went_rw) 2014-08-13 18:06:07.354861 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.354860 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] up [356,364,381] -> [356,364,4], acting [356,364] -> [356,364], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> -1 2014-08-13 18:06:07.354864 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:07.354872 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.354874 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:07.354883 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [320,307,296] 2014-08-13 18:06:07.354882 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354893 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.354894 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.354903 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [320,307,55]/[320,307,296] r=-1 lpr=194492 pi=193679-194491/40 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194495 2014-08-13 18:06:07.354905 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:07.354914 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:07.354920 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [320,307,55](320) acting [320,307,296](320)): not rw, up_thru 194492 up_from 192979 last_epoch_clean 194466 generate_past_intervals interval(194492-194494 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) : primary up 192979-194492 includes interval 2014-08-13 18:06:07.354923 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:07.354930 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194491/194492/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] noting past interval(194492-194494 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) 2014-08-13 18:06:07.354933 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] on_change 2014-08-13 18:06:07.354941 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] up [320,307,55] -> [4,344,296], acting [320,307,296] -> [320,307,296], acting_primary 320 -> 320, up_primary 320 -> 4, role -1 -> -1 2014-08-13 18:06:07.354947 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:07.354954 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.354957 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:07.354964 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.354967 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] cancel_recovery 2014-08-13 18:06:07.354973 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.354984 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 18:06:07.354977 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:07.354993 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 18:06:07.354996 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.355002 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 18:06:07.355008 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_activate_map 2014-08-13 18:06:07.355012 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:07.355022 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 18:06:07.355031 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 18:06:07.355040 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] cancel_recovery 2014-08-13 18:06:07.355048 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] clear_recovery_state 2014-08-13 18:06:07.355058 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.355057 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] update_heartbeat_peers unchanged 2014-08-13 18:06:07.355069 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] take_waiters 2014-08-13 18:06:07.355069 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_activate_map 2014-08-13 18:06:07.355083 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Reset 0.415752 10 0.001192 2014-08-13 18:06:07.355095 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started 2014-08-13 18:06:07.355108 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Start 2014-08-13 18:06:07.355117 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: transitioning to Stray 2014-08-13 18:06:07.355127 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.355128 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] update_heartbeat_peers unchanged 2014-08-13 18:06:07.355140 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] take_waiters 2014-08-13 18:06:07.355140 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Started/Stray 2014-08-13 18:06:07.355151 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Reset 0.635025 10 0.001057 2014-08-13 18:06:07.355160 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.355163 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started 2014-08-13 18:06:07.355171 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.355179 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Start 2014-08-13 18:06:07.355189 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: transitioning to Stray 2014-08-13 18:06:07.355199 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.355211 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Started/Stray 2014-08-13 18:06:07.355230 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.355222 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.355234 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.355246 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.355251 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c53f40 don't know epoch, doing nothing 2014-08-13 18:06:07.355253 7f5c7bde3700 7 osd.4 194495 do_notify osd 19 on 1 PGs 2014-08-13 18:06:07.355270 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:07.355272 7f5c7bde3700 7 osd.4 194495 do_notify osd 356 on 1 PGs 2014-08-13 18:06:07.355287 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c514a0 don't know epoch, doing nothing 2014-08-13 18:06:07.355289 7f5c7bde3700 7 osd.4 194495 do_notify osd 358 on 1 PGs 2014-08-13 18:06:07.355293 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.355343 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:07.355349 7f5c7c7e4700 7 osd.4 194495 do_notify osd 30 on 1 PGs 2014-08-13 18:06:07.355367 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:07.355369 7f5c7c7e4700 7 osd.4 194495 do_notify osd 320 on 1 PGs 2014-08-13 18:06:07.355383 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:07.355384 7f5c7c7e4700 7 osd.4 194495 do_notify osd 337 on 1 PGs 2014-08-13 18:06:07.355454 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:07.355460 7f5c7c7e4700 7 osd.4 194495 do_notify osd 356 on 1 PGs 2014-08-13 18:06:07.355494 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] flushed 2014-08-13 18:06:07.355582 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4851e40 don't know epoch, doing nothing 2014-08-13 18:06:07.355587 7f5c7c7e4700 7 osd.4 194495 do_notify osd 371 on 1 PGs 2014-08-13 18:06:07.355603 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b10b00 don't know epoch, doing nothing 2014-08-13 18:06:07.355605 7f5c7c7e4700 7 osd.4 194495 do_notify osd 397 on 1 PGs 2014-08-13 18:06:07.355621 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4851a20 don't know epoch, doing nothing 2014-08-13 18:06:07.355623 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.32 on 1 PGs 2014-08-13 18:06:07.355629 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:07.355635 7f5c7bde3700 7 osd.4 194495 do_notify osd 373 on 1 PGs 2014-08-13 18:06:07.355699 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:07.355703 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.50 on 2 PGs 2014-08-13 18:06:07.355709 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4493f40 don't know epoch, doing nothing 2014-08-13 18:06:07.355738 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.17 on 1 PGs 2014-08-13 18:06:07.355744 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:07.355746 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.55 on 1 PGs 2014-08-13 18:06:07.355758 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:07.355760 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.28 on 1 PGs 2014-08-13 18:06:07.355764 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:07.355766 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.61 on 1 PGs 2014-08-13 18:06:07.355781 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c51080 don't know epoch, doing nothing 2014-08-13 18:06:07.355783 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.315 on 1 PGs 2014-08-13 18:06:07.355849 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c58420 don't know epoch, doing nothing 2014-08-13 18:06:07.355853 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.327 on 1 PGs 2014-08-13 18:06:07.355857 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:07.355861 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.65 on 1 PGs 2014-08-13 18:06:07.355869 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:07.355871 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.330 on 1 PGs 2014-08-13 18:06:07.355878 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:07.355880 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.289 on 1 PGs 2014-08-13 18:06:07.355891 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:07.355893 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.356 on 1 PGs 2014-08-13 18:06:07.355977 7f5c4db6f700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.355956 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 18:06:07.355988 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.355985 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 18:06:07.355994 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.356009 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.356005 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.356014 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.356013 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.356025 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.356025 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.287590 2 0.000228 2014-08-13 18:06:07.356037 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,52] newacting [15,61] 2014-08-13 18:06:07.356040 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.356048 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.356050 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.287628 0 0.000000 2014-08-13 18:06:07.356058 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,4,52]/[15,4,61] r=1 lpr=194487 pi=193264-194485/35 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194488 2014-08-13 18:06:07.356062 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.287650 0 0.000000 2014-08-13 18:06:07.356072 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Started 0.287700 0 0.000000 2014-08-13 18:06:07.356078 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [15,4,52](15) acting [15,4,61](15)): not rw, up_thru 194486 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [15,4,52](15) acting [15,4,61](15) maybe_went_rw) : primary up 192969-194486 includes interval 2014-08-13 18:06:07.356083 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.356092 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194486 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.356089 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194485/194486/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194486-194487 up [15,4,52](15) acting [15,4,61](15) maybe_went_rw) 2014-08-13 18:06:07.356101 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194488 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.356101 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,4,52] -> [15,52], acting [15,4,61] -> [15,61], acting_primary 15 -> 15, up_primary 15 -> 15, role 1 -> -1 2014-08-13 18:06:07.356112 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194488 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.356116 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.356121 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194488 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [343,286] newacting [343,286] 2014-08-13 18:06:07.356126 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.356131 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194488 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.356136 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.356140 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [4,343,286] r=0 lpr=194488 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.356147 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.356159 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,343,286](4) acting [4,343,286](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.356177 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194485/194485/194485) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] noting past interval(194485-194487 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 18:06:07.356165 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.356187 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] up [4,343,286] -> [343,286], acting [4,343,286] -> [343,286], acting_primary 4 -> 343, up_primary 4 -> 343, role 0 -> -1 2014-08-13 18:06:07.356193 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.356198 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.356203 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.356207 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.356216 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.356215 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.356225 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.356227 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.356220 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] flushed 2014-08-13 18:06:07.356234 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.356237 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_role_change 2014-08-13 18:06:07.356245 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.356255 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.356248 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.356258 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.356260 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0e00 pg_notify(22.6bd(2) epoch 194495) v5 2014-08-13 18:06:07.356255 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.356259 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.356265 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.356270 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.356274 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.356288 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.32 2014-08-13 18:06:07.356281 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.356291 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0e00 2014-08-13 18:06:07.356285 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.356295 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.356304 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.356305 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 18:06:07.356313 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.356321 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.356321 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.356325 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.356321 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.356324 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.356330 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.356335 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.356338 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.356346 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.356346 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360211 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360220 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360203 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360222 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5b00 pg_notify(24.136 epoch 194495) v5 2014-08-13 18:06:07.360221 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,52]/[15,61] -- 15/15 2014-08-13 18:06:07.360244 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360250 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.360243 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 18:06:07.360253 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5b00 2014-08-13 18:06:07.360248 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360257 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360259 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360260 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360270 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360269 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360280 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360290 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360292 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360307 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360308 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360299 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,61,379]/[15,61] -- 15/15 2014-08-13 18:06:07.360310 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf4ec0 pg_notify(24.136 epoch 194495) v5 2014-08-13 18:06:07.360312 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360315 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360331 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.55 2014-08-13 18:06:07.360333 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf4ec0 2014-08-13 18:06:07.360320 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [343,286]/[343,286] -- 343/343 2014-08-13 18:06:07.360326 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360337 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360336 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [15,61,379] newacting [15,61] 2014-08-13 18:06:07.360340 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360346 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.360355 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360350 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360359 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360360 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360358 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,52]/[15,61] r=-1 lpr=194488 pi=193264-194487/36 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194491 2014-08-13 18:06:07.360371 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360373 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360374 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df2680 pg_notify(22.6bd(2),24.136(1) epoch 194495) v5 2014-08-13 18:06:07.360378 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [15,52](15) acting [15,61](15)): not rw, up_thru 194488 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [15,52](15) acting [15,61](15) maybe_went_rw) : primary up 192969-194488 includes interval 2014-08-13 18:06:07.360383 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 18:06:07.360394 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360388 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194488/194488/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194488-194490 up [15,52](15) acting [15,61](15) maybe_went_rw) 2014-08-13 18:06:07.360399 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.50 2014-08-13 18:06:07.360401 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df2680 2014-08-13 18:06:07.360397 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360401 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,52] -> [15,61,379], acting [15,61] -> [15,61], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> -1 2014-08-13 18:06:07.360406 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360414 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.360415 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] acting_up_affected newup [10,343,286] newacting [10,343,286] 2014-08-13 18:06:07.360432 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360424 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.360434 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360425 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.360436 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.360438 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [343,286] r=-1 lpr=194488 pi=194485-194487/1 crt=191727'426 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.360448 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360450 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360452 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df55c0 pg_notify(24.136(35) epoch 194495) v5 2014-08-13 18:06:07.360447 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.360457 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.360457 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [343,286](343) acting [343,286](343)): not rw, up_thru 194488 up_from 192942 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) : primary up 192942-194488 includes interval 2014-08-13 18:06:07.360473 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.65 2014-08-13 18:06:07.360475 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df55c0 2014-08-13 18:06:07.360467 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.360467 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194488/194488/194488) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 18:06:07.360478 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:07.360479 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] up [343,286] -> [10,343,286], acting [343,286] -> [10,343,286], acting_primary 343 -> 10, up_primary 343 -> 10, role -1 -> -1 2014-08-13 18:06:07.360491 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.360491 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.360502 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360505 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360501 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.360502 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.360509 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.360512 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.360521 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360523 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360525 7f5c835ef700 20 osd.4 194495 _dispatch 0x4de9f80 pg_notify(24.b6(1) epoch 194495) v5 2014-08-13 18:06:07.360519 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.360523 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.360528 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.360534 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360539 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.360547 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.327 2014-08-13 18:06:07.360549 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4de9f80 2014-08-13 18:06:07.360548 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.360557 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.360569 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360560 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,61,379]/[15,61,379] -- 15/15 2014-08-13 18:06:07.360571 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360567 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.360575 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360582 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360583 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360585 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df3b80 pg_notify(24.b6 epoch 194495) v5 2014-08-13 18:06:07.360578 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360576 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] [343,286] -> [10,343,286], acting primary 343 -> 10 2014-08-13 18:06:07.360588 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360592 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.360607 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.330 2014-08-13 18:06:07.360599 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] acting_up_affected newup [15,61,379] newacting [15,61,379] 2014-08-13 18:06:07.360601 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.360610 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df3b80 2014-08-13 18:06:07.360611 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360610 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.360622 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379]/[15,61] r=-1 lpr=194491 pi=193264-194490/37 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] set_last_peering_reset 194492 2014-08-13 18:06:07.360630 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360632 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360642 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360644 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360635 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 18:06:07.360645 7f5c835ef700 20 osd.4 194495 _dispatch 0x4a45cc0 pg_notify(24.b6 epoch 194495) v5 2014-08-13 18:06:07.360647 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360640 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [15,61,379](15) acting [15,61](15)): not rw, up_thru 194488 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [15,61,379](15) acting [15,61](15)) : primary up 192969-194488 does not include interval 2014-08-13 18:06:07.360649 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360651 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194491/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] noting past interval(194491-194491 up [15,61,379](15) acting [15,61](15)) 2014-08-13 18:06:07.360659 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360667 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.28 2014-08-13 18:06:07.360669 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4a45cc0 2014-08-13 18:06:07.360662 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] up [15,61,379] -> [15,61,379], acting [15,61] -> [15,61,379], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> -1 2014-08-13 18:06:07.360668 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360675 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.360688 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360690 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360685 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.360700 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360702 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360694 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.360703 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf0fc0 pg_notify(22.6bd(28) epoch 194495) v5 2014-08-13 18:06:07.360691 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 18:06:07.360707 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360705 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.360709 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360714 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.360726 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.289 2014-08-13 18:06:07.360728 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf0fc0 2014-08-13 18:06:07.360722 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360723 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.360732 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360733 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.360743 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.360757 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360759 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360752 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.360753 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [10,343,286]/[10,343,286] -- 10/10 2014-08-13 18:06:07.360764 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360761 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.360766 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360776 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360778 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360771 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.360779 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf40c0 pg_notify(24.b6(43) epoch 194495) v5 2014-08-13 18:06:07.360776 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360782 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360786 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360803 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.17 2014-08-13 18:06:07.360805 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf40c0 2014-08-13 18:06:07.360809 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,61,379]/[15,61,379] -- 15/15 2014-08-13 18:06:07.360824 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360820 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 18:06:07.360831 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360833 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360834 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360827 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360837 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.360837 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360851 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360853 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360846 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.360848 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360856 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df39c0 pg_notify(24.b6(43) epoch 194495) v5 2014-08-13 18:06:07.360857 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,343,286] newacting [4,343,286] 2014-08-13 18:06:07.360866 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.360878 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.315 2014-08-13 18:06:07.360870 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,61,379]/[15,61,379] -- 15/15 2014-08-13 18:06:07.360880 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df39c0 2014-08-13 18:06:07.360875 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [10,343,286] r=-1 lpr=194491 pi=194485-194490/2 crt=191727'426 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.360884 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360888 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360894 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [10,343,286](10) acting [10,343,286](10)): not rw, up_thru 194491 up_from 192975 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) : primary up 192975-194491 includes interval 2014-08-13 18:06:07.360898 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360907 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360909 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360904 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194491/194491/194491) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 18:06:07.360908 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.360915 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [10,343,286] -> [4,343,286], acting [10,343,286] -> [4,343,286], acting_primary 10 -> 4, up_primary 10 -> 4, role -1 -> 0 2014-08-13 18:06:07.360930 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.360932 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.360933 7f5c835ef700 20 osd.4 194495 _dispatch 0x524cec0 pg_notify(24.136(35) epoch 194495) v5 2014-08-13 18:06:07.360927 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.360927 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,52] -- 15/15 2014-08-13 18:06:07.360941 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.360938 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.360948 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.360948 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:07.360960 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.356 2014-08-13 18:06:07.360962 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524cec0 2014-08-13 18:06:07.360959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.360960 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2653 2014-08-13 18:06:07.360971 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.360970 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,52] 2014-08-13 18:06:07.360980 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.360980 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.360990 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.360989 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,61,379] r=-1 lpr=194492 pi=193264-194491/38 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194495 2014-08-13 18:06:07.361004 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.361006 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.361001 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361010 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361010 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [15,61,379](15) acting [15,61,379](15)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [15,61,379](15) acting [15,61,379](15) maybe_went_rw) : primary up 192969-194492 includes interval 2014-08-13 18:06:07.361019 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361021 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194491/194492/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] noting past interval(194492-194494 up [15,61,379](15) acting [15,61,379](15) maybe_went_rw) 2014-08-13 18:06:07.361027 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.361032 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] up [15,61,379] -> [15,4,52], acting [15,61,379] -> [15,4,52], acting_primary 15 -> 15, up_primary 15 -> 15, role -1 -> 1 2014-08-13 18:06:07.361037 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.361044 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.361046 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.361054 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.361056 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.361062 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.361065 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.361073 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:07.361074 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.361082 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:07.361082 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.361091 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.361091 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:07.361100 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.361101 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_change 2014-08-13 18:06:07.361109 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.361111 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:07.361119 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361121 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:07.361129 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.361130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] on_role_change 2014-08-13 18:06:07.361139 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.361140 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 18:06:07.361150 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.361147 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] cancel_recovery 2014-08-13 18:06:07.361163 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:07.361159 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.005075 8 0.004896 2014-08-13 18:06:07.361172 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361181 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.361183 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:07.361191 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.361199 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.361209 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.361220 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.361229 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.361229 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 18:06:07.361239 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.361240 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:07.361249 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.361251 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Reset 0.520422 10 0.004973 2014-08-13 18:06:07.361260 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 18:06:07.361264 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started 2014-08-13 18:06:07.361271 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 18:06:07.361274 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Start 2014-08-13 18:06:07.361282 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 18:06:07.361286 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: transitioning to Stray 2014-08-13 18:06:07.361293 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,10,286,343 down blocked_by {} 2014-08-13 18:06:07.361298 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Start 0.000023 0 0.000000 2014-08-13 18:06:07.361303 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.361310 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Started/Stray 2014-08-13 18:06:07.361315 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2654 2014-08-13 18:06:07.361323 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.361325 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.10 2014-08-13 18:06:07.361333 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.361335 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.286 2014-08-13 18:06:07.361345 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.343 2014-08-13 18:06:07.361356 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.361366 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.361394 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.361405 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.361422 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338]/[31,338] -- 31/31 2014-08-13 18:06:07.361440 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361433 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 18:06:07.361447 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361443 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.361451 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.361453 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.361460 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,308] newacting [51,308] 2014-08-13 18:06:07.361462 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] acting_up_affected newup [31,338] newacting [31,338] 2014-08-13 18:06:07.361469 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.361472 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.361480 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.292343 3 0.000133 2014-08-13 18:06:07.361481 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338,4] r=2 lpr=194487 pi=193125-194485/43 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194488 2014-08-13 18:06:07.361491 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.292397 0 0.000000 2014-08-13 18:06:07.361500 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [31,338,4](31) acting [31,338,4](31)): not rw, up_thru 194486 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194486-194487 up [31,338,4](31) acting [31,338,4](31) maybe_went_rw) : primary up 192978-194486 includes interval 2014-08-13 18:06:07.361502 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.361511 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194486 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.361509 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194485/194486/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] noting past interval(194486-194487 up [31,338,4](31) acting [31,338,4](31) maybe_went_rw) 2014-08-13 18:06:07.361521 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194488 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.361520 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] up [31,338,4] -> [31,338], acting [31,338,4] -> [31,338], acting_primary 31 -> 31, up_primary 31 -> 31, role 2 -> -1 2014-08-13 18:06:07.361530 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194488 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.361532 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 18:06:07.361539 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194488 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,308] newacting [51,308] 2014-08-13 18:06:07.361542 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.361548 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194488 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.361552 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.361558 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,4,308] r=1 lpr=194488 pi=192975-194484/25 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.361562 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_copy_ops 2014-08-13 18:06:07.361571 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_flush_ops 2014-08-13 18:06:07.361576 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [51,4,308](51) acting [51,4,308](51)): not rw, up_thru 194485 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [51,4,308](51) acting [51,4,308](51) maybe_went_rw) : primary up 192971-194485 includes interval 2014-08-13 18:06:07.361581 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change_cleanup 2014-08-13 18:06:07.361587 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194485/194485/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [51,4,308](51) acting [51,4,308](51) maybe_went_rw) 2014-08-13 18:06:07.361591 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 18:06:07.361597 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,4,308] -> [51,308], acting [51,4,308] -> [51,308], acting_primary 51 -> 51, up_primary 51 -> 51, role 1 -> -1 2014-08-13 18:06:07.361602 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit NotTrimming 2014-08-13 18:06:07.361609 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.361611 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter NotTrimming 2014-08-13 18:06:07.361617 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.361622 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_role_change 2014-08-13 18:06:07.361627 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361631 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.361637 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.361641 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_recovery 2014-08-13 18:06:07.361646 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.361650 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] clear_recovery_state 2014-08-13 18:06:07.361656 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.361659 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361665 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.361674 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.361679 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338]/[31,338,16] -- 31/31 2014-08-13 18:06:07.361683 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.361694 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361692 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.361697 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.361700 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361707 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.361713 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.361716 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] acting_up_affected newup [31,338] newacting [31,338,16] 2014-08-13 18:06:07.361721 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.361726 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.361731 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361735 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338] r=-1 lpr=194488 pi=193125-194487/44 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] set_last_peering_reset 194489 2014-08-13 18:06:07.361734 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] flushed 2014-08-13 18:06:07.361752 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 18:06:07.361752 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [31,338](31) acting [31,338](31)): not rw, up_thru 194486 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194488-194488 up [31,338](31) acting [31,338](31)) : primary up 192978-194486 does not include interval 2014-08-13 18:06:07.361765 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361764 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194488/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] noting past interval(194488-194488 up [31,338](31) acting [31,338](31)) 2014-08-13 18:06:07.361767 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.361776 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.361775 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] up [31,338] -> [31,338], acting [31,338] -> [31,338,16], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 18:06:07.361785 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361787 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.361797 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.361803 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,308]/[51,308] -- 51/51 2014-08-13 18:06:07.361806 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.361814 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361816 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.361816 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_copy_ops 2014-08-13 18:06:07.361825 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.361826 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_flush_ops 2014-08-13 18:06:07.361834 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361835 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change_cleanup 2014-08-13 18:06:07.361845 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.361854 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit NotTrimming 2014-08-13 18:06:07.361854 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 18:06:07.361867 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361866 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter NotTrimming 2014-08-13 18:06:07.361870 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.361876 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_recovery 2014-08-13 18:06:07.361879 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.361885 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] clear_recovery_state 2014-08-13 18:06:07.361887 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,19,308] newacting [51,19,308] 2014-08-13 18:06:07.361896 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.361895 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361905 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,308] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.361916 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338]/[31,338,16] -- 31/31 2014-08-13 18:06:07.361930 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361920 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [51,308](51) acting [51,308](51)): not rw, up_thru 194488 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [51,308](51) acting [51,308](51) maybe_went_rw) : primary up 192971-194488 includes interval 2014-08-13 18:06:07.361932 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194488/194488/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [51,308](51) acting [51,308](51) maybe_went_rw) 2014-08-13 18:06:07.361932 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.361945 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.361944 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,308] -> [51,19,308], acting [51,308] -> [51,19,308], acting_primary 51 -> 51, up_primary 51 -> 51, role -1 -> -1 2014-08-13 18:06:07.361955 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.361955 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.361964 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.361972 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.361976 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 18:06:07.361981 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.361990 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.361989 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.361993 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.361998 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.362003 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.362007 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.362013 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,374] newacting [31,338,16] 2014-08-13 18:06:07.362016 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.362023 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.362026 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.362033 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338]/[31,338,16] r=-1 lpr=194489 pi=193125-194488/45 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194491 2014-08-13 18:06:07.362034 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.362043 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.362050 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [31,338](31) acting [31,338,16](31)): not rw, up_thru 194489 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194489-194490 up [31,338](31) acting [31,338,16](31) maybe_went_rw) : primary up 192978-194489 includes interval 2014-08-13 18:06:07.362053 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362061 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194488/194489/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] noting past interval(194489-194490 up [31,338](31) acting [31,338,16](31) maybe_went_rw) 2014-08-13 18:06:07.362072 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] up [31,338] -> [31,338,374], acting [31,338,16] -> [31,338,16], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 18:06:07.362074 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 18:06:07.362085 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362084 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.362088 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.362093 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.362097 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.362103 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.362106 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362114 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_copy_ops 2014-08-13 18:06:07.362122 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_flush_ops 2014-08-13 18:06:07.362126 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 18:06:07.362136 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362132 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change_cleanup 2014-08-13 18:06:07.362139 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.362142 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.362148 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.362153 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit NotTrimming 2014-08-13 18:06:07.362156 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362169 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter NotTrimming 2014-08-13 18:06:07.362178 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_recovery 2014-08-13 18:06:07.362182 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,19,308]/[51,19,308] -- 51/51 2014-08-13 18:06:07.362193 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362187 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] clear_recovery_state 2014-08-13 18:06:07.362195 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.362201 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362204 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.362213 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362223 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 18:06:07.362230 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 18:06:07.362238 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362241 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362241 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.362243 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.362251 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.362251 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.362260 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [51,4,308] newacting [51,4,308] 2014-08-13 18:06:07.362261 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362269 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.362277 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,19,308] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.362282 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 18:06:07.362295 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362293 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [51,19,308](51) acting [51,19,308](51)): not rw, up_thru 194491 up_from 192971 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [51,19,308](51) acting [51,19,308](51) maybe_went_rw) : primary up 192971-194491 includes interval 2014-08-13 18:06:07.362298 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.362305 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194491/194491/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [51,19,308](51) acting [51,19,308](51) maybe_went_rw) 2014-08-13 18:06:07.362309 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.362315 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] up [51,19,308] -> [51,4,308], acting [51,19,308] -> [51,4,308], acting_primary 51 -> 51, up_primary 51 -> 51, role -1 -> 1 2014-08-13 18:06:07.362319 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362327 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.362335 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.362339 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,374]/[31,338,16] -- 31/31 2014-08-13 18:06:07.362343 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.362355 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362354 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.362358 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.362363 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.362368 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.362372 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.362379 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362380 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.362389 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.362398 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.362397 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,16] -- 31/31 2014-08-13 18:06:07.362410 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.362415 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.362413 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:07.362423 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.362423 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:07.362431 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.362433 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,16] 2014-08-13 18:06:07.362441 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362444 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.362451 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.362454 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,374]/[31,338,16] r=-1 lpr=194491 pi=193125-194490/46 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194495 2014-08-13 18:06:07.362471 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [31,338,374](31) acting [31,338,16](31)): not rw, up_thru 194491 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194491-194494 up [31,338,374](31) acting [31,338,16](31) maybe_went_rw) : primary up 192978-194491 includes interval 2014-08-13 18:06:07.362480 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194491/194491/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] noting past interval(194491-194494 up [31,338,374](31) acting [31,338,16](31) maybe_went_rw) 2014-08-13 18:06:07.362487 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.362491 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] up [31,338,374] -> [31,338,4], acting [31,338,16] -> [31,338,16], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> -1 2014-08-13 18:06:07.362497 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.362504 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.362507 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.001005 8 0.000828 2014-08-13 18:06:07.362514 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.362519 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.362523 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.362528 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.362534 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_copy_ops 2014-08-13 18:06:07.362537 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.362544 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_flush_ops 2014-08-13 18:06:07.362547 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.362553 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change_cleanup 2014-08-13 18:06:07.362558 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.362563 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] on_change 2014-08-13 18:06:07.362570 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.362573 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit NotTrimming 2014-08-13 18:06:07.362580 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.362586 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter NotTrimming 2014-08-13 18:06:07.362596 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] cancel_recovery 2014-08-13 18:06:07.362605 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] clear_recovery_state 2014-08-13 18:06:07.362615 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.362625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_activate_map 2014-08-13 18:06:07.362633 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.362665 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 18:06:07.362680 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362682 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] update_heartbeat_peers unchanged 2014-08-13 18:06:07.362684 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.362693 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.362693 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] take_waiters 2014-08-13 18:06:07.362702 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.362704 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Reset 0.595497 10 0.001025 2014-08-13 18:06:07.362713 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.293270 2 0.000219 2014-08-13 18:06:07.362717 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started 2014-08-13 18:06:07.362724 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.362727 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Start 2014-08-13 18:06:07.362735 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.293302 0 0.000000 2014-08-13 18:06:07.362737 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: transitioning to Stray 2014-08-13 18:06:07.362747 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.293324 0 0.000000 2014-08-13 18:06:07.362748 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.362757 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Started 0.293374 0 0.000000 2014-08-13 18:06:07.362760 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Started/Stray 2014-08-13 18:06:07.362767 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.362772 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.362776 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194486 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.362784 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194488 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.362783 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.362795 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194488 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range start epoch 194486 >= end epoch 194485, nothing to do 2014-08-13 18:06:07.362803 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194488 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [340,347] newacting [340,347] 2014-08-13 18:06:07.362812 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194488 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.362820 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [4,340,347] r=0 lpr=194488 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.362836 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,340,347](4) acting [4,340,347](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.362847 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.362845 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194485/194485/194485) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] noting past interval(194485-194487 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 18:06:07.362855 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] up [4,340,347] -> [340,347], acting [4,340,347] -> [340,347], acting_primary 4 -> 340, up_primary 4 -> 340, role 0 -> -1 2014-08-13 18:06:07.362865 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.362878 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.362886 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.362894 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.362901 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.362906 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:07.362911 7f5c7c7e4700 7 osd.4 194495 do_notify osd 15 on 1 PGs 2014-08-13 18:06:07.362910 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.362918 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.362929 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4857bc0 don't know epoch, doing nothing 2014-08-13 18:06:07.362931 7f5c7c7e4700 7 osd.4 194495 do_notify osd 31 on 1 PGs 2014-08-13 18:06:07.362926 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.362934 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.362943 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.362952 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.362959 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.362968 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.362976 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.362974 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 18:06:07.362984 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.362994 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.362992 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.362998 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363000 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.363008 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363009 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363017 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336] newacting [285,336] 2014-08-13 18:06:07.363027 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363033 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 18:06:07.363035 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194488 2014-08-13 18:06:07.363044 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363047 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363053 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [285,336,4](285) acting [285,336,4](285)): not rw, up_thru 194486 up_from 192975 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [285,336,4](285) acting [285,336,4](285) maybe_went_rw) : primary up 192975-194486 includes interval 2014-08-13 18:06:07.363055 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363064 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363063 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194485/194486/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194486-194487 up [285,336,4](285) acting [285,336,4](285) maybe_went_rw) 2014-08-13 18:06:07.363073 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336,4] -> [285,336], acting [285,336,4] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role 2 -> -1 2014-08-13 18:06:07.363083 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [340,347]/[340,347] -- 340/340 2014-08-13 18:06:07.363084 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.363094 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363076 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.363094 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.363096 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363103 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.363105 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363113 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363113 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.363123 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.363131 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.363136 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 18:06:07.363146 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363140 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.363148 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363157 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363150 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.363170 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] acting_up_affected newup [7,340,347] newacting [7,340,347] 2014-08-13 18:06:07.363172 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.363180 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363181 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 18:06:07.363188 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [340,347] r=-1 lpr=194488 pi=194485-194487/1 crt=191648'362 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.363191 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.363200 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.363204 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [340,347](340) acting [340,347](340)): not rw, up_thru 194488 up_from 192944 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) : primary up 192944-194488 includes interval 2014-08-13 18:06:07.363208 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.363213 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194488/194488/194488) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 18:06:07.363219 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363224 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] up [340,347] -> [7,340,347], acting [340,347] -> [7,340,347], acting_primary 340 -> 7, up_primary 340 -> 7, role -1 -> -1 2014-08-13 18:06:07.363235 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.363238 7f5c4d96d700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.363240 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 18:06:07.363243 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.363255 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363252 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.363258 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363260 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.363267 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363269 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.363259 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] flushed 2014-08-13 18:06:07.363277 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363277 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.363288 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.363296 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.363296 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336]/[285,336] -- 285/285 2014-08-13 18:06:07.363309 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363306 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.363312 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363314 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] [340,347] -> [7,340,347], acting primary 340 -> 7 2014-08-13 18:06:07.363321 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363329 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.363330 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363337 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.363346 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363350 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,365]/[285,336,365] -- 285/285 2014-08-13 18:06:07.363363 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363365 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363370 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 18:06:07.363381 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363374 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363383 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363383 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336,365] newacting [285,336,365] 2014-08-13 18:06:07.363394 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363393 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363404 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194491 2014-08-13 18:06:07.363405 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363420 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [285,336](285) acting [285,336](285)): not rw, up_thru 194488 up_from 192975 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [285,336](285) acting [285,336](285) maybe_went_rw) : primary up 192975-194488 includes interval 2014-08-13 18:06:07.363427 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 18:06:07.363429 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194488/194488/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194488-194490 up [285,336](285) acting [285,336](285) maybe_went_rw) 2014-08-13 18:06:07.363438 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363440 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363439 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336] -> [285,336,365], acting [285,336] -> [285,336,365], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 18:06:07.363449 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363450 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.363458 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363460 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.363469 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.363477 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.363479 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [7,340,347]/[7,340,347] -- 7/7 2014-08-13 18:06:07.363490 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363486 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.363492 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363495 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.363501 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363505 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.363510 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363518 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.363527 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.363530 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 18:06:07.363544 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363539 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.363546 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.363548 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.363555 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363558 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363563 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,340,347] newacting [4,340,347] 2014-08-13 18:06:07.363572 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363580 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [7,340,347] r=-1 lpr=194491 pi=194485-194490/2 crt=191648'362 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.363581 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,365]/[285,336] -- 285/285 2014-08-13 18:06:07.363595 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363598 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363597 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [7,340,347](7) acting [7,340,347](7)): not rw, up_thru 194491 up_from 192980 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) : primary up 192980-194491 includes interval 2014-08-13 18:06:07.363607 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363608 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194491/194491/194491) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 18:06:07.363617 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [285,336,365] newacting [285,336] 2014-08-13 18:06:07.363619 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [7,340,347] -> [4,340,347], acting [7,340,347] -> [4,340,347], acting_primary 7 -> 4, up_primary 7 -> 4, role -1 -> 0 2014-08-13 18:06:07.363626 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363630 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.363635 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194492 2014-08-13 18:06:07.363640 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.363648 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.363653 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [285,336,365](285) acting [285,336,365](285)): not rw, up_thru 194488 up_from 192975 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [285,336,365](285) acting [285,336,365](285)) : primary up 192975-194488 does not include interval 2014-08-13 18:06:07.363658 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:3000 2014-08-13 18:06:07.363664 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194491/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] noting past interval(194491-194491 up [285,336,365](285) acting [285,336,365](285)) 2014-08-13 18:06:07.363668 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.363676 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] up [285,336,365] -> [285,336,365], acting [285,336,365] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 18:06:07.363681 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.363690 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.363689 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.363699 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.363700 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.363707 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.363709 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.363716 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.363720 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.363724 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.363729 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.363733 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.363739 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.363743 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.363749 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.363753 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.363759 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.363762 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.363769 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.363771 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.363779 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.363779 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.363788 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.363789 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.363796 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.363799 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363804 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.363814 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363820 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,365]/[285,336] -- 285/285 2014-08-13 18:06:07.363823 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.363835 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363835 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 18:06:07.363837 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363845 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.363847 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363854 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001087 8 0.000956 2014-08-13 18:06:07.363857 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363866 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.363874 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.363877 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,365]/[285,336] -- 285/285 2014-08-13 18:06:07.363882 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.363890 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363892 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363893 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.363902 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363904 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.363912 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.363913 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.363922 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.363931 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.363930 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,4]/[285,336] -- 285/285 2014-08-13 18:06:07.363943 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.363941 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 18:06:07.363946 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.363953 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 18:06:07.363956 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.363964 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 18:06:07.363966 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336] 2014-08-13 18:06:07.363975 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,340,347 down blocked_by {} 2014-08-13 18:06:07.363976 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.363985 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.363985 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,365]/[285,336] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194495 2014-08-13 18:06:07.363996 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:3001 2014-08-13 18:06:07.364006 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.7 2014-08-13 18:06:07.364006 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [285,336,365](285) acting [285,336](285)): not rw, up_thru 194492 up_from 192975 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [285,336,365](285) acting [285,336](285) maybe_went_rw) : primary up 192975-194492 includes interval 2014-08-13 18:06:07.364016 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.340 2014-08-13 18:06:07.364016 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194491/194492/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] noting past interval(194492-194494 up [285,336,365](285) acting [285,336](285) maybe_went_rw) 2014-08-13 18:06:07.364027 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.347 2014-08-13 18:06:07.364028 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] up [285,336,365] -> [285,336,4], acting [285,336] -> [285,336], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> -1 2014-08-13 18:06:07.364037 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.364040 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.364046 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.364050 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.364059 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.364069 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.364078 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.364085 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.364086 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.364095 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.364104 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.364113 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.364113 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 18:06:07.364129 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364123 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.364132 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.364132 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.364142 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24] newacting [345,24] 2014-08-13 18:06:07.364143 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364152 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.364159 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_activate_map 2014-08-13 18:06:07.364166 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.294251 3 0.000130 2014-08-13 18:06:07.364178 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started 0.294304 0 0.000000 2014-08-13 18:06:07.364188 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.364196 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194486 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.364205 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194488 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364204 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 18:06:07.364213 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194488 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364215 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] take_waiters 2014-08-13 18:06:07.364222 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194488 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24] newacting [345,24] 2014-08-13 18:06:07.364226 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.582242 10 0.001009 2014-08-13 18:06:07.364231 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194488 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.364239 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24,4] r=2 lpr=194488 pi=192984-194484/24 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.364238 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started 2014-08-13 18:06:07.364249 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Start 2014-08-13 18:06:07.364258 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray 2014-08-13 18:06:07.364261 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [345,24,4](345) acting [345,24,4](345)): not rw, up_thru 194485 up_from 192981 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [345,24,4](345) acting [345,24,4](345) maybe_went_rw) : primary up 192981-194485 includes interval 2014-08-13 18:06:07.364270 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194485/194485/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [345,24,4](345) acting [345,24,4](345) maybe_went_rw) 2014-08-13 18:06:07.364270 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.364281 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24,4] -> [345,24], acting [345,24,4] -> [345,24], acting_primary 345 -> 345, up_primary 345 -> 345, role 2 -> -1 2014-08-13 18:06:07.364282 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray 2014-08-13 18:06:07.364291 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364295 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.364300 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.364305 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.364309 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.364318 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.364326 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.364334 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.364342 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364351 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.364362 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.364360 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.364368 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.364375 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.364383 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.364390 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.364390 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 18:06:07.364406 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364400 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364410 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.364419 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359] newacting [323,359] 2014-08-13 18:06:07.364420 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 18:06:07.364431 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364428 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.364434 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364438 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.303823 3 0.000120 2014-08-13 18:06:07.364443 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364450 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started 0.303873 0 0.000000 2014-08-13 18:06:07.364452 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364462 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.364471 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194486 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.364471 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24]/[345,24] -- 345/345 2014-08-13 18:06:07.364482 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364480 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364484 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364489 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364493 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364499 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359] newacting [323,359] 2014-08-13 18:06:07.364503 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364508 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.364517 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359,4] r=2 lpr=194488 pi=192978-194484/24 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.364522 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 18:06:07.364533 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364535 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364535 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [323,359,4](323) acting [323,359,4](323)): not rw, up_thru 194485 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [323,359,4](323) acting [323,359,4](323) maybe_went_rw) : primary up 192978-194485 includes interval 2014-08-13 18:06:07.364544 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364545 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194485/194485/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [323,359,4](323) acting [323,359,4](323) maybe_went_rw) 2014-08-13 18:06:07.364553 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24,60] newacting [345,24,60] 2014-08-13 18:06:07.364556 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359,4] -> [323,359], acting [323,359,4] -> [323,359], acting_primary 323 -> 323, up_primary 323 -> 323, role 2 -> -1 2014-08-13 18:06:07.364562 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.364567 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364577 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.364584 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.364593 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.364571 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24] r=-1 lpr=194488 pi=192984-194487/25 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.364601 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.364610 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.364615 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [345,24](345) acting [345,24](345)): not rw, up_thru 194488 up_from 192981 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [345,24](345) acting [345,24](345) maybe_went_rw) : primary up 192981-194488 includes interval 2014-08-13 18:06:07.364619 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364625 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194488/194488/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [345,24](345) acting [345,24](345) maybe_went_rw) 2014-08-13 18:06:07.364630 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.364636 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24] -> [345,24,60], acting [345,24] -> [345,24,60], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> -1 2014-08-13 18:06:07.364639 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.364635 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] flushed 2014-08-13 18:06:07.364647 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364648 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.364657 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.364657 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.364667 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.364667 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.364675 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.364677 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.364685 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364686 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.364694 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.364702 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.364704 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 18:06:07.364716 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364711 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.364719 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364720 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.364727 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364729 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.364736 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364737 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.364747 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364753 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359]/[323,359] -- 323/323 2014-08-13 18:06:07.364766 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364768 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364767 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 18:06:07.364779 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364777 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364781 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364786 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364790 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364799 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364806 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 18:06:07.364816 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364818 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364817 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 18:06:07.364829 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364827 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364831 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364835 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359,364] newacting [323,359,364] 2014-08-13 18:06:07.364840 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.364845 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.364849 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.364854 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359] r=-1 lpr=194488 pi=192978-194487/25 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.364868 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,60]/[345,24,60] -- 345/345 2014-08-13 18:06:07.364869 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [323,359](323) acting [323,359](323)): not rw, up_thru 194488 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [323,359](323) acting [323,359](323) maybe_went_rw) : primary up 192978-194488 includes interval 2014-08-13 18:06:07.364879 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.364879 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194488/194488/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [323,359](323) acting [323,359](323) maybe_went_rw) 2014-08-13 18:06:07.364881 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.364889 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359] -> [323,359,364], acting [323,359] -> [323,359,364], acting_primary 323 -> 323, up_primary 323 -> 323, role -1 -> -1 2014-08-13 18:06:07.370893 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.370898 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.370913 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.370917 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.370927 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.370940 7f5c817ec700 10 osd.4 194495 note_peer_epoch osd.51 now has 194495 2014-08-13 18:06:07.370936 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.370936 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 18:06:07.370946 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x3cf1600 already has epoch 194495 2014-08-13 18:06:07.370949 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.370946 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.370952 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.370956 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.370962 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.370965 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.370971 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] acting_up_affected newup [345,24,4] newacting [345,24,4] 2014-08-13 18:06:07.370975 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.370982 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x3cf1600 already has epoch 194495 2014-08-13 18:06:07.370981 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.370985 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.370990 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,60] r=-1 lpr=194491 pi=192984-194490/26 crt=192888'458 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.370995 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.371003 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.371008 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [345,24,60](345) acting [345,24,60](345)): not rw, up_thru 194491 up_from 192981 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [345,24,60](345) acting [345,24,60](345) maybe_went_rw) : primary up 192981-194491 includes interval 2014-08-13 18:06:07.371014 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371019 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194491/194491/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [345,24,60](345) acting [345,24,60](345) maybe_went_rw) 2014-08-13 18:06:07.371029 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] up [345,24,60] -> [345,24,4], acting [345,24,60] -> [345,24,4], acting_primary 345 -> 345, up_primary 345 -> 345, role -1 -> 2 2014-08-13 18:06:07.371039 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.371039 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 18:06:07.371052 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371049 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.371055 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.371057 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.371064 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.371066 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.371073 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371075 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.371084 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.371092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.371093 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 18:06:07.371104 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371101 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.371106 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.371110 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.371116 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.371119 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.371125 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371128 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.371136 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.371144 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.371145 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,364]/[323,359,364] -- 323/323 2014-08-13 18:06:07.371160 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371154 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371163 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.371168 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.371172 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.371181 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371198 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 18:06:07.371208 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371209 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.371210 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.371218 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.371219 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.371228 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [323,359,4] newacting [323,359,4] 2014-08-13 18:06:07.371228 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Reset 0.007041 8 0.006865 2014-08-13 18:06:07.371238 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.371242 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.371247 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,364] r=-1 lpr=194491 pi=192978-194490/26 crt=192888'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.371252 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.371260 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.371266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [323,359,364](323) acting [323,359,364](323)): not rw, up_thru 194491 up_from 192978 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [323,359,364](323) acting [323,359,364](323) maybe_went_rw) : primary up 192978-194491 includes interval 2014-08-13 18:06:07.371271 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.371276 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194491/194491/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [323,359,364](323) acting [323,359,364](323) maybe_went_rw) 2014-08-13 18:06:07.371282 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.371287 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] up [323,359,364] -> [323,359,4], acting [323,359,364] -> [323,359,4], acting_primary 323 -> 323, up_primary 323 -> 323, role -1 -> 2 2014-08-13 18:06:07.371295 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.371298 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.371304 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.371308 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.371316 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.371325 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.371333 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.371341 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.371349 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.371356 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.371357 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.371365 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.371373 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.371384 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.371392 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.371387 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 18:06:07.371406 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371401 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.371410 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371410 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.371420 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.371420 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.371430 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374] newacting [6,374] 2014-08-13 18:06:07.371439 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.371447 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374,4] r=2 lpr=194487 pi=193125-194485/31 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194488 2014-08-13 18:06:07.371463 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.371467 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [6,374,4](6) acting [6,374,4](6)): not rw, up_thru 194486 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [6,374,4](6) acting [6,374,4](6) maybe_went_rw) : primary up 192967-194486 includes interval 2014-08-13 18:06:07.371473 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.371477 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194485/194486/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194486-194487 up [6,374,4](6) acting [6,374,4](6) maybe_went_rw) 2014-08-13 18:06:07.371483 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Reset 0.007021 8 0.006835 2014-08-13 18:06:07.371488 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374,4] -> [6,374], acting [6,374,4] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role 2 -> -1 2014-08-13 18:06:07.371496 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.371499 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:07.371505 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.371509 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.371514 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.371518 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.371525 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.371528 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 18:06:07.371536 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.371536 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 18:06:07.371546 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 18:06:07.371549 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.371555 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:07.371559 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.371566 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 18:06:07.371575 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 18:06:07.371584 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_role_change 2014-08-13 18:06:07.371592 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.371600 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 18:06:07.371608 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 18:06:07.371618 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371625 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.371613 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.371640 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 18:06:07.371653 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371656 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.371658 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_advance_map [44,362]/[44,362] -- 44/44 2014-08-13 18:06:07.371665 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.371675 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371674 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371679 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.371689 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.371692 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374]/[6,374] -- 6/6 2014-08-13 18:06:07.371697 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.371708 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371708 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.309124 2 0.000217 2014-08-13 18:06:07.371711 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.371720 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.371720 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.371729 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371730 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.309155 0 0.000000 2014-08-13 18:06:07.371743 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.309177 0 0.000000 2014-08-13 18:06:07.371748 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,336]/[6,374,336] -- 6/6 2014-08-13 18:06:07.371754 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Started 0.309227 0 0.000000 2014-08-13 18:06:07.371761 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371764 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.371765 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.371773 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.371774 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194486 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.371782 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374,336] newacting [6,374,336] 2014-08-13 18:06:07.371784 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194488 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.371792 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.371793 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194488 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.371801 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374] r=-1 lpr=194488 pi=193125-194487/32 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194491 2014-08-13 18:06:07.371802 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194488 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [44,362] newacting [44,362] 2014-08-13 18:06:07.371812 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194488 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.371818 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [6,374](6) acting [6,374](6)): not rw, up_thru 194488 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [6,374](6) acting [6,374](6) maybe_went_rw) : primary up 192967-194488 includes interval 2014-08-13 18:06:07.371821 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [4,44,362] r=0 lpr=194488 pi=193306-194484/32 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.371827 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194488/194488/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194488-194490 up [6,374](6) acting [6,374](6) maybe_went_rw) 2014-08-13 18:06:07.371837 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374] -> [6,374,336], acting [6,374] -> [6,374,336], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 18:06:07.371840 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,44,362](4) acting [4,44,362](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.371841 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.371849 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:07.371850 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194485/194485/194485) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] noting past interval(194485-194487 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) 2014-08-13 18:06:07.371860 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.371862 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] up [4,44,362] -> [44,362], acting [4,44,362] -> [44,362], acting_primary 4 -> 44, up_primary 4 -> 44, role 0 -> -1 2014-08-13 18:06:07.371868 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.371873 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.371878 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 18:06:07.371882 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.371887 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 18:06:07.371891 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.371896 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 18:06:07.371900 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.371905 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:07.371909 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.371919 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.371915 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 18:06:07.371927 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.371928 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 18:06:07.371936 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.371937 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 18:06:07.371944 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.371946 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 18:06:07.371954 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.371956 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.371963 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.371971 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.371977 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,336]/[6,374] -- 6/6 2014-08-13 18:06:07.371979 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.371992 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.371990 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.371995 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.371998 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.372005 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.372008 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.372014 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] acting_up_affected newup [6,374,336] newacting [6,374] 2014-08-13 18:06:07.372017 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.372024 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372027 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372033 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336] r=-1 lpr=194491 pi=193125-194490/33 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] set_last_peering_reset 194492 2014-08-13 18:06:07.372048 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [44,362]/[44,362,61] -- 44/44 2014-08-13 18:06:07.372050 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [6,374,336](6) acting [6,374,336](6)): not rw, up_thru 194488 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [6,374,336](6) acting [6,374,336](6)) : primary up 192967-194488 does not include interval 2014-08-13 18:06:07.372059 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372062 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372060 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194491/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] noting past interval(194491-194491 up [6,374,336](6) acting [6,374,336](6)) 2014-08-13 18:06:07.372071 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372071 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] up [6,374,336] -> [6,374,336], acting [6,374,336] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 18:06:07.372080 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] acting_up_affected newup [44,362] newacting [44,362,61] 2014-08-13 18:06:07.372084 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 18:06:07.372089 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372093 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372098 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362] r=-1 lpr=194488 pi=193306-194487/33 crt=191727'457 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.372103 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.372113 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_copy_ops 2014-08-13 18:06:07.372116 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [44,362](44) acting [44,362](44)): not rw, up_thru 194483 up_from 192926 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [44,362](44) acting [44,362](44)) : primary up 192926-194483 does not include interval 2014-08-13 18:06:07.372122 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_flush_ops 2014-08-13 18:06:07.372127 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194488/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [44,362](44) acting [44,362](44)) 2014-08-13 18:06:07.372132 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change_cleanup 2014-08-13 18:06:07.372137 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] up [44,362] -> [44,362], acting [44,362] -> [44,362,61], acting_primary 44 -> 44, up_primary 44 -> 44, role -1 -> -1 2014-08-13 18:06:07.372141 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 18:06:07.372151 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit NotTrimming 2014-08-13 18:06:07.372149 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.372164 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372172 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.372160 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter NotTrimming 2014-08-13 18:06:07.372182 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.372182 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_recovery 2014-08-13 18:06:07.372191 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.372192 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] clear_recovery_state 2014-08-13 18:06:07.372201 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.372203 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372209 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.372219 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.372225 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,336]/[6,374] -- 6/6 2014-08-13 18:06:07.372228 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.372240 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372237 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.372242 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.372246 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.372253 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.372255 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372262 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372278 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [44,362]/[44,362,61] -- 44/44 2014-08-13 18:06:07.372290 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372285 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,336]/[6,374] -- 6/6 2014-08-13 18:06:07.372299 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372293 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372301 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.372302 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372311 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.372312 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372321 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372339 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [61,369,339]/[44,362,61] -- 61/44 2014-08-13 18:06:07.372350 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372347 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,4]/[6,374] -- 6/6 2014-08-13 18:06:07.372352 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372362 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372361 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372365 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:07.372371 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,369,339] newacting [44,362,61] 2014-08-13 18:06:07.372375 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.372380 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372385 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374] 2014-08-13 18:06:07.372390 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [44,362]/[44,362,61] r=-1 lpr=194489 pi=193306-194488/34 crt=191727'457 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.372395 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372405 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,336]/[6,374] r=-1 lpr=194492 pi=193125-194491/34 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194495 2014-08-13 18:06:07.372408 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [44,362](44) acting [44,362,61](44)): not rw, up_thru 194489 up_from 192926 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [44,362](44) acting [44,362,61](44) maybe_went_rw) : primary up 192926-194489 includes interval 2014-08-13 18:06:07.372418 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194488/194489/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [44,362](44) acting [44,362,61](44) maybe_went_rw) 2014-08-13 18:06:07.372422 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [6,374,336](6) acting [6,374](6)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [6,374,336](6) acting [6,374](6) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.372428 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] up [44,362] -> [61,369,339], acting [44,362,61] -> [44,362,61], acting_primary 44 -> 44, up_primary 44 -> 61, role -1 -> -1 2014-08-13 18:06:07.372433 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194491/194492/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] noting past interval(194492-194494 up [6,374,336](6) acting [6,374](6) maybe_went_rw) 2014-08-13 18:06:07.372441 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.372443 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] up [6,374,336] -> [6,374,4], acting [6,374] -> [6,374], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> -1 2014-08-13 18:06:07.372450 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372455 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 18:06:07.372459 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.372465 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372469 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.372474 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.372479 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.372484 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_copy_ops 2014-08-13 18:06:07.372488 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.372494 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_flush_ops 2014-08-13 18:06:07.372498 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.372504 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change_cleanup 2014-08-13 18:06:07.372508 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.372513 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] on_change 2014-08-13 18:06:07.372518 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.372523 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit NotTrimming 2014-08-13 18:06:07.372527 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.372534 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter NotTrimming 2014-08-13 18:06:07.372537 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.372543 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] cancel_recovery 2014-08-13 18:06:07.372547 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372553 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] clear_recovery_state 2014-08-13 18:06:07.372563 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372573 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_activate_map 2014-08-13 18:06:07.372572 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] handle_advance_map [61,369,339]/[61,369,339] -- 61/61 2014-08-13 18:06:07.372583 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372586 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372594 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372603 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] acting_up_affected newup [61,369,339] newacting [61,369,339] 2014-08-13 18:06:07.372612 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372612 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] update_heartbeat_peers unchanged 2014-08-13 18:06:07.372623 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] take_waiters 2014-08-13 18:06:07.372621 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339]/[44,362,61] r=-1 lpr=194491 pi=193306-194490/35 crt=191727'457 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.372633 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Reset 0.701667 10 0.000990 2014-08-13 18:06:07.372642 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [61,369,339](61) acting [44,362,61](44)): not rw, up_thru 194489 up_from 192926 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [61,369,339](61) acting [44,362,61](44)) : primary up 192926-194489 does not include interval 2014-08-13 18:06:07.372645 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started 2014-08-13 18:06:07.372651 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194491/194488) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 18:06:07.372656 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Start 2014-08-13 18:06:07.372662 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] up [61,369,339] -> [61,369,339], acting [44,362,61] -> [61,369,339], acting_primary 44 -> 61, up_primary 61 -> 61, role -1 -> -1 2014-08-13 18:06:07.372665 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: transitioning to Stray 2014-08-13 18:06:07.372674 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.372676 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.372682 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372689 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Started/Stray 2014-08-13 18:06:07.372692 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.372700 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.372701 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.372710 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.372712 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.372718 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.372727 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.372736 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.372745 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.372753 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] [44,362,61] -> [61,369,339], acting primary 44 -> 61 2014-08-13 18:06:07.372768 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.372767 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.372775 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.372783 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372798 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,303]/[376,303] -- 376/376 2014-08-13 18:06:07.372812 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372809 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [61,369,339]/[61,369,339] -- 61/61 2014-08-13 18:06:07.372820 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372816 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.372822 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372826 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,303] newacting [376,303] 2014-08-13 18:06:07.372832 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372835 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.372841 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372849 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.302527 3 0.000133 2014-08-13 18:06:07.372860 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started 0.302576 0 0.000000 2014-08-13 18:06:07.372863 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [61,369,339]/[61,369,339] -- 61/61 2014-08-13 18:06:07.372874 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372870 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.372876 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372880 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194486 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.372886 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372890 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194488 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372895 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.372899 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194488 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.372907 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194488 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,303] newacting [376,303] 2014-08-13 18:06:07.372912 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 18:06:07.372916 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194488 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372924 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.372925 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,4,303] r=1 lpr=194488 pi=192975-194484/29 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.372926 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.372935 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.372943 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [376,4,303](376) acting [376,4,303](376)): not rw, up_thru 194485 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194485-194487 up [376,4,303](376) acting [376,4,303](376) maybe_went_rw) : primary up 192974-194485 includes interval 2014-08-13 18:06:07.372943 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,44,362] newacting [4,44,362] 2014-08-13 18:06:07.372952 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194485/194485/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [376,4,303](376) acting [376,4,303](376) maybe_went_rw) 2014-08-13 18:06:07.372953 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.372965 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [61,369,339] r=-1 lpr=194492 pi=193306-194491/36 crt=191727'457 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.372963 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,4,303] -> [376,303], acting [376,4,303] -> [376,303], acting_primary 376 -> 376, up_primary 376 -> 376, role 1 -> -1 2014-08-13 18:06:07.372975 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.372982 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [61,369,339](61) acting [61,369,339](61)): not rw, up_thru 194492 up_from 192938 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) : primary up 192938-194492 includes interval 2014-08-13 18:06:07.372984 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.372993 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.372992 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194491/194492/194492) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 18:06:07.373003 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.373003 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [61,369,339] -> [4,44,362], acting [61,369,339] -> [4,44,362], acting_primary 61 -> 4, up_primary 61 -> 4, role -1 -> 0 2014-08-13 18:06:07.373011 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.373015 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.373020 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.373025 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.373029 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.373023 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] flushed 2014-08-13 18:06:07.373035 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.373039 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.373046 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2684 2014-08-13 18:06:07.373049 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.373056 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373057 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.373065 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373065 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.373074 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.373075 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.373082 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.373084 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373091 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373092 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373101 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373109 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.373111 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,303]/[376,303,313] -- 376/376 2014-08-13 18:06:07.373123 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373117 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.373126 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373127 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.373134 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373136 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.373142 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,303] newacting [376,303,313] 2014-08-13 18:06:07.373152 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.373145 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.373161 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.373160 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303] r=-1 lpr=194488 pi=192975-194487/30 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.373170 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.373179 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.373182 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [376,303](376) acting [376,303](376)): not rw, up_thru 194485 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194488-194488 up [376,303](376) acting [376,303](376)) : primary up 192974-194485 does not include interval 2014-08-13 18:06:07.373188 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.373192 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194488/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [376,303](376) acting [376,303](376)) 2014-08-13 18:06:07.373196 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.373203 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] up [376,303] -> [376,303], acting [376,303] -> [376,303,313], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> -1 2014-08-13 18:06:07.373206 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373217 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.373214 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.373226 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.373228 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,44,362 unchanged 2014-08-13 18:06:07.373235 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.373238 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.373244 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.373247 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001483 8 0.001350 2014-08-13 18:06:07.373253 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.373259 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.373263 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.373269 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.373272 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.373278 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.373282 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.373289 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.373291 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.373301 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.373300 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.373310 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.373311 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.373320 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373321 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.373330 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.373338 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] handle_advance_map [376,303]/[376,303,313] -- 376/376 2014-08-13 18:06:07.373340 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 18:06:07.373349 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373352 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373352 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 18:06:07.373362 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373364 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [44,362](44) acting [44,362,61](44) maybe_went_rw) 2014-08-13 18:06:07.373371 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373374 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [44,362](44) acting [44,362](44)) 2014-08-13 18:06:07.373384 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,44,362](4) acting [4,44,362](4) maybe_went_rw) 2014-08-13 18:06:07.373392 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] handle_advance_map [376,313,319]/[376,303,313] -- 376/376 2014-08-13 18:06:07.373403 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373395 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 18:06:07.373405 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373406 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 18:06:07.373414 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373417 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,61,339,362,369 down blocked_by {} 2014-08-13 18:06:07.373423 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] acting_up_affected newup [376,313,319] newacting [376,303,313] 2014-08-13 18:06:07.373428 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.373433 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.373439 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2685 2014-08-13 18:06:07.373442 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,303]/[376,303,313] r=-1 lpr=194489 pi=192975-194488/31 crt=192888'461 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.373450 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.44 2014-08-13 18:06:07.373458 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [376,303](376) acting [376,303,313](376)): not rw, up_thru 194489 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194489-194490 up [376,303](376) acting [376,303,313](376) maybe_went_rw) : primary up 192974-194489 includes interval 2014-08-13 18:06:07.373461 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.61 2014-08-13 18:06:07.373468 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194488/194489/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [376,303](376) acting [376,303,313](376) maybe_went_rw) 2014-08-13 18:06:07.373470 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.339 2014-08-13 18:06:07.373480 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.362 2014-08-13 18:06:07.373478 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] up [376,303] -> [376,313,319], acting [376,303,313] -> [376,303,313], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> -1 2014-08-13 18:06:07.373489 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.369 2014-08-13 18:06:07.373490 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.373501 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.373500 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.373511 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.373513 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.373520 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.373529 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.373538 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.373547 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.373556 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.373565 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.373573 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.373587 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.373581 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.373591 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373616 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 18:06:07.373628 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373630 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373627 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_advance_map [287,309]/[287,309] -- 287/287 2014-08-13 18:06:07.373643 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373639 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373647 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.373648 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] acting_up_affected newup [376,313,319] newacting [376,313,319] 2014-08-13 18:06:07.373657 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.373658 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.373667 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.373667 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319]/[376,303,313] r=-1 lpr=194491 pi=192975-194490/32 crt=192888'461 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.373678 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.308965 2 0.000215 2014-08-13 18:06:07.373686 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [376,313,319](376) acting [376,303,313](376)): not rw, up_thru 194489 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194491-194491 up [376,313,319](376) acting [376,303,313](376)) : primary up 192974-194489 does not include interval 2014-08-13 18:06:07.373696 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.373695 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194491/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [376,313,319](376) acting [376,303,313](376)) 2014-08-13 18:06:07.373707 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.309003 0 0.000000 2014-08-13 18:06:07.373708 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,313,319] -> [376,313,319], acting [376,303,313] -> [376,313,319], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> -1 2014-08-13 18:06:07.373719 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.373720 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.309025 0 0.000000 2014-08-13 18:06:07.373728 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.373731 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Started 0.309075 0 0.000000 2014-08-13 18:06:07.373736 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373742 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.373745 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.373751 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194486 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.373754 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.373761 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194488 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.373763 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.373770 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194488 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.373771 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.373779 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194488 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [287,309] newacting [287,309] 2014-08-13 18:06:07.373781 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.373789 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194488 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.373790 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.373798 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [4,287,309] r=0 lpr=194488 pi=193306-194484/47 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.373799 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.373807 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.373816 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373816 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,287,309](4) acting [4,287,309](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194304 generate_past_intervals interval(194485-194487 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.373826 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194485/194485/194485) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] noting past interval(194485-194487 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 18:06:07.373836 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] up [4,287,309] -> [287,309], acting [4,287,309] -> [287,309], acting_primary 4 -> 287, up_primary 4 -> 287, role 0 -> -1 2014-08-13 18:06:07.373838 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 18:06:07.373853 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373847 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.373856 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373857 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.373864 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373865 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.373873 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373874 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.373887 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.373893 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,313,319]/[376,313,319] -- 376/376 2014-08-13 18:06:07.373895 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.373904 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373904 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.373906 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373912 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.373915 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373921 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.373925 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.373931 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.373940 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.373941 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 18:06:07.373952 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.373948 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.373954 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.373958 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373963 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194483 2014-08-13 18:06:07.373966 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.373972 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] acting_up_affected newup [376,4,303] newacting [376,4,303] 2014-08-13 18:06:07.373975 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.373982 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.373983 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.373991 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,313,319] r=-1 lpr=194492 pi=192975-194491/33 crt=192888'461 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.373991 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.374001 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374007 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [376,313,319](376) acting [376,313,319](376)): not rw, up_thru 194492 up_from 192974 last_epoch_clean 194483 generate_past_intervals interval(194492-194494 up [376,313,319](376) acting [376,313,319](376) maybe_went_rw) : primary up 192974-194492 includes interval 2014-08-13 18:06:07.374016 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194491/194492/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [376,313,319](376) acting [376,313,319](376) maybe_went_rw) 2014-08-13 18:06:07.374021 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [287,309]/[287,309] -- 287/287 2014-08-13 18:06:07.374033 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374026 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] up [376,313,319] -> [376,4,303], acting [376,313,319] -> [376,4,303], acting_primary 376 -> 376, up_primary 376 -> 376, role -1 -> 1 2014-08-13 18:06:07.374036 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374037 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374044 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374045 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.374053 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374054 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374065 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.374071 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [287,309]/[287,309] -- 287/287 2014-08-13 18:06:07.374073 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.374082 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374081 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.374084 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374090 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374093 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374099 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.374102 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374109 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.374117 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.374125 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374123 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 18:06:07.374134 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374133 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.374136 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.374145 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374151 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374160 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [303,306,381] newacting [303,306,381] 2014-08-13 18:06:07.374169 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.374170 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.374177 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [287,309] r=-1 lpr=194488 pi=193306-194487/48 crt=191727'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.374192 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [287,309](287) acting [287,309](287)): not rw, up_thru 194488 up_from 192975 last_epoch_clean 194304 generate_past_intervals interval(194488-194490 up [287,309](287) acting [287,309](287) maybe_went_rw) : primary up 192975-194488 includes interval 2014-08-13 18:06:07.374201 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194488/194488/194488) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 18:06:07.374210 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] up [287,309] -> [303,306,381], acting [287,309] -> [303,306,381], acting_primary 287 -> 303, up_primary 287 -> 303, role -1 -> -1 2014-08-13 18:06:07.374213 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.374221 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374223 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.374230 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.374233 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Reset 0.001363 8 0.001168 2014-08-13 18:06:07.374238 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374245 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.374248 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.374254 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.374256 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.374263 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.374265 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.374273 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374273 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.374283 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.374285 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.374292 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.374298 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.374301 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] [287,309] -> [303,306,381], acting primary 287 -> 303 2014-08-13 18:06:07.374308 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.374316 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.374324 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.374332 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374362 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.374355 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 18:06:07.374366 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374368 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374376 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374385 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374389 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_advance_map [338,376]/[338,376] -- 338/338 2014-08-13 18:06:07.374404 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374406 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 18:06:07.374408 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.374417 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374418 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.374419 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374427 7f5c7bde3700 1 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.374428 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374438 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374438 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.303644 2 0.000224 2014-08-13 18:06:07.374451 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.374459 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.303674 0 0.000000 2014-08-13 18:06:07.374459 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [303,306,381]/[303,306,381] -- 303/303 2014-08-13 18:06:07.374471 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374473 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374475 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.303699 0 0.000000 2014-08-13 18:06:07.374481 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374488 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Started 0.303761 0 0.000000 2014-08-13 18:06:07.374490 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374499 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.374507 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194486 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.374507 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 18:06:07.374519 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374517 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194488 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.374521 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374526 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194488 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.374530 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374535 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194488 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [338,376] newacting [338,376] 2014-08-13 18:06:07.374539 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,287,309] newacting [4,287,309] 2014-08-13 18:06:07.374544 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194488 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.374551 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.374553 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [4,338,376] r=0 lpr=194488 pi=193306-194484/48 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.374560 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [303,306,381] r=-1 lpr=194491 pi=193306-194490/49 crt=191727'463 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.374572 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,338,376](4) acting [4,338,376](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194389 generate_past_intervals interval(194485-194487 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.374577 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [303,306,381](303) acting [303,306,381](303)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194304 generate_past_intervals interval(194491-194494 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.374581 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194485/194485/194485) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] noting past interval(194485-194487 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) 2014-08-13 18:06:07.374588 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194491/194491/194491) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.374591 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] up [4,338,376] -> [338,376], acting [4,338,376] -> [338,376], acting_primary 4 -> 338, up_primary 4 -> 338, role 0 -> -1 2014-08-13 18:06:07.374598 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [303,306,381] -> [4,287,309], acting [303,306,381] -> [4,287,309], acting_primary 303 -> 4, up_primary 303 -> 4, role -1 -> 0 2014-08-13 18:06:07.374602 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.374610 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.374611 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.374619 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.374619 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.374628 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.374629 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374637 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.374640 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2683 2014-08-13 18:06:07.374646 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.374649 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374654 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.374659 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.374663 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.374668 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.374672 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.374660 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.374677 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374682 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.374687 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374693 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.374696 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374703 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.374705 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.374712 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374713 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.374720 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.374723 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.374729 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.374733 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.374737 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.374742 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.374745 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.374752 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.374755 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374760 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.374769 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.374777 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.374775 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,376]/[338,376,293] -- 338/338 2014-08-13 18:06:07.374787 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.374785 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.374789 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.374795 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374798 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.374805 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.374807 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] acting_up_affected newup [338,376] newacting [338,376,293] 2014-08-13 18:06:07.374816 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.374816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,287,309 unchanged 2014-08-13 18:06:07.374825 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376] r=-1 lpr=194488 pi=193306-194487/49 crt=191727'441 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.374827 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.374836 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001094 8 0.000988 2014-08-13 18:06:07.374841 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [338,376](338) acting [338,376](338)): not rw, up_thru 194482 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194488-194488 up [338,376](338) acting [338,376](338)) : primary up 194414-194482 does not include interval 2014-08-13 18:06:07.374847 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.374851 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194488/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [338,376](338) acting [338,376](338)) 2014-08-13 18:06:07.374857 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.374862 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] up [338,376] -> [338,376], acting [338,376] -> [338,376,293], acting_primary 338 -> 338, up_primary 338 -> 338, role -1 -> -1 2014-08-13 18:06:07.374866 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.374873 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.374877 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.374882 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.374889 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.374891 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.374898 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.374900 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.374908 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.374909 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.374917 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:07.374918 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.374932 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.374930 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.374942 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.374942 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 18:06:07.374952 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.374953 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 18:06:07.374961 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.374964 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.374970 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.374976 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [303,306,381](303) acting [287,309,303](287)) 2014-08-13 18:06:07.374981 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.374986 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 18:06:07.374997 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.375000 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] handle_advance_map [338,376]/[338,376,293] -- 338/338 2014-08-13 18:06:07.375011 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375007 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2684 2014-08-13 18:06:07.375014 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375017 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.287 2014-08-13 18:06:07.375023 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375028 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.303 2014-08-13 18:06:07.375033 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375039 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.306 2014-08-13 18:06:07.375055 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.309 2014-08-13 18:06:07.375055 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] handle_advance_map [338,373,396]/[338,376,293] -- 338/338 2014-08-13 18:06:07.375066 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375065 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.381 2014-08-13 18:06:07.375069 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375075 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.375078 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375085 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.375087 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] acting_up_affected newup [338,373,396] newacting [338,376,293] 2014-08-13 18:06:07.375097 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375106 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,376]/[338,376,293] r=-1 lpr=194489 pi=193306-194488/50 crt=191727'441 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.375121 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [338,376](338) acting [338,376,293](338)): not rw, up_thru 194489 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194489-194490 up [338,376](338) acting [338,376,293](338) maybe_went_rw) : primary up 194414-194489 includes interval 2014-08-13 18:06:07.375130 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194488/194489/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [338,376](338) acting [338,376,293](338) maybe_went_rw) 2014-08-13 18:06:07.375140 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] up [338,376] -> [338,373,396], acting [338,376,293] -> [338,376,293], acting_primary 338 -> 338, up_primary 338 -> 338, role -1 -> -1 2014-08-13 18:06:07.375151 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.375159 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.375160 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.375174 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.375183 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.375192 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.375188 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_advance_map [289,341]/[289,341] -- 289/289 2014-08-13 18:06:07.375203 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375201 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.375207 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:07.375211 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.375217 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: affected_by_map osd.4 now down 2014-08-13 18:06:07.375221 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.375226 7f5c7c7e4700 1 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering, affected_by_map, going to Reset 2014-08-13 18:06:07.375231 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.375237 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.308590 2 0.000209 2014-08-13 18:06:07.375241 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.375249 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:07.375250 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.375259 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.308621 0 0.000000 2014-08-13 18:06:07.375260 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375277 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started/Primary 0.308648 0 0.000000 2014-08-13 18:06:07.375283 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 18:06:07.375295 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375288 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Started 0.308698 0 0.000000 2014-08-13 18:06:07.375299 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375301 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Reset 2014-08-13 18:06:07.375308 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375310 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194486 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.375318 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] acting_up_affected newup [338,373,396] newacting [338,373,396] 2014-08-13 18:06:07.375320 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194488 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-13 18:06:07.375327 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375329 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194488 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.375336 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396]/[338,376,293] r=-1 lpr=194491 pi=193306-194490/51 crt=191727'441 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.375338 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194488 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] acting_up_affected newup [289,341] newacting [289,341] 2014-08-13 18:06:07.375347 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194488 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375353 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [338,373,396](338) acting [338,376,293](338)): not rw, up_thru 194489 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194491-194491 up [338,373,396](338) acting [338,376,293](338)) : primary up 194414-194489 does not include interval 2014-08-13 18:06:07.375356 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [4,289,341] r=0 lpr=194488 pi=192978-194484/30 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] set_last_peering_reset 194488 2014-08-13 18:06:07.375363 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194491/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [338,373,396](338) acting [338,376,293](338)) 2014-08-13 18:06:07.375375 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [4,289,341](4) acting [4,289,341](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194485-194487 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.375374 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] up [338,373,396] -> [338,373,396], acting [338,376,293] -> [338,373,396], acting_primary 338 -> 338, up_primary 338 -> 338, role -1 -> -1 2014-08-13 18:06:07.375385 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.375385 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194485/194485/194485) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] noting past interval(194485-194487 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) 2014-08-13 18:06:07.375394 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.375396 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] up [4,289,341] -> [289,341], acting [4,289,341] -> [289,341], acting_primary 4 -> 289, up_primary 4 -> 289, role 0 -> -1 2014-08-13 18:06:07.375402 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375406 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] clear_primary_state 2014-08-13 18:06:07.375411 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.375416 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] agent_stop 2014-08-13 18:06:07.375420 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.375425 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.375428 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.375434 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.375437 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.375442 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.375447 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.375453 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] cancel_copy_ops 2014-08-13 18:06:07.375456 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.375461 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] cancel_flush_ops 2014-08-13 18:06:07.375465 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.375471 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] on_change_cleanup 2014-08-13 18:06:07.375474 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.375480 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] on_change 2014-08-13 18:06:07.375484 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375490 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] exit NotTrimming 2014-08-13 18:06:07.375499 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive] enter NotTrimming 2014-08-13 18:06:07.375507 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_stats 2014-08-13 18:06:07.375506 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 18:06:07.375520 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375519 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375523 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375527 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.375532 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375535 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375541 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375544 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.375552 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.375561 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375561 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [338,373,396]/[338,373,396] -- 338/338 2014-08-13 18:06:07.375573 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375575 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375583 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375581 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [289,341]/[289,341,327] -- 289/289 2014-08-13 18:06:07.375594 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375593 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375597 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375605 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.375610 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 18:06:07.375621 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375614 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] acting_up_affected newup [289,341] newacting [289,341,327] 2014-08-13 18:06:07.375623 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375623 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375632 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341] r=-1 lpr=194488 pi=192978-194487/31 crt=191727'473 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194489 2014-08-13 18:06:07.375633 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.375642 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,338,376] newacting [4,338,376] 2014-08-13 18:06:07.375651 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375650 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [289,341](289) acting [289,341](289)): not rw, up_thru 194486 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [289,341](289) acting [289,341](289)) : primary up 192968-194486 does not include interval 2014-08-13 18:06:07.375660 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [338,373,396] r=-1 lpr=194492 pi=193306-194491/52 crt=191727'441 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.375661 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194488/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] noting past interval(194488-194488 up [289,341](289) acting [289,341](289)) 2014-08-13 18:06:07.375672 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] up [289,341] -> [289,341], acting [289,341] -> [289,341,327], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.375677 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [338,373,396](338) acting [338,373,396](338)): not rw, up_thru 194492 up_from 194414 last_epoch_clean 194389 generate_past_intervals interval(194492-194494 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) : primary up 194414-194492 includes interval 2014-08-13 18:06:07.375683 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.375687 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194491/194492/194488) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 18:06:07.375692 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.375698 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [338,373,396] -> [4,338,376], acting [338,373,396] -> [4,338,376], acting_primary 338 -> 4, up_primary 338 -> 4, role -1 -> 0 2014-08-13 18:06:07.375702 7f5c7c7e4700 15 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.375710 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.375711 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.375719 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.375720 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.375729 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.375737 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.375746 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.375728 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.375755 7f5c7c7e4700 20 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.375758 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2595 2014-08-13 18:06:07.375764 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.375769 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375773 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.375779 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.375783 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375788 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.375797 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375805 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375803 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,341]/[289,341,327] -- 289/289 2014-08-13 18:06:07.375816 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375815 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.375819 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375824 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.375832 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.375842 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.375829 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.375850 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.375859 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.375858 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375868 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.375876 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.375884 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.375883 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,327,376]/[289,341,327] -- 289/289 2014-08-13 18:06:07.375895 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.375893 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.375897 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.375901 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.375907 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.375911 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.375917 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] acting_up_affected newup [289,327,376] newacting [289,341,327] 2014-08-13 18:06:07.375922 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.375927 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.375933 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,338,376 unchanged 2014-08-13 18:06:07.375936 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,341]/[289,341,327] r=-1 lpr=194489 pi=192978-194488/32 crt=191727'473 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.375943 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.375953 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [289,341](289) acting [289,341,327](289)): not rw, up_thru 194489 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [289,341](289) acting [289,341,327](289) maybe_went_rw) : primary up 192968-194489 includes interval 2014-08-13 18:06:07.375952 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.001453 8 0.001338 2014-08-13 18:06:07.375963 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194488/194489/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] noting past interval(194489-194490 up [289,341](289) acting [289,341,327](289) maybe_went_rw) 2014-08-13 18:06:07.375966 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.375975 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.375974 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] up [289,341] -> [289,327,376], acting [289,341,327] -> [289,341,327], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.375984 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.375987 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.375994 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.375996 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.376005 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.376005 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.376016 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.376017 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.376025 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.376027 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.376034 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:07.376036 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.376045 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.376044 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 18:06:07.376055 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.376060 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [338,373,396](338) acting [338,376,293](338)) 2014-08-13 18:06:07.376065 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.376071 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [338,376](338) acting [338,376,293](338) maybe_went_rw) 2014-08-13 18:06:07.376075 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.376082 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [338,376](338) acting [338,376](338)) 2014-08-13 18:06:07.376084 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.376092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,338,376](4) acting [4,338,376](4) maybe_went_rw) 2014-08-13 18:06:07.376094 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.376103 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194484 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 18:06:07.376113 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194479-194481 up [338,376](338) acting [338,376](338) maybe_went_rw) 2014-08-13 18:06:07.376117 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] handle_advance_map [289,327,376]/[289,327,376] -- 289/289 2014-08-13 18:06:07.376129 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.376123 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,293,338,373,376,396 down blocked_by {} 2014-08-13 18:06:07.376131 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.376134 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.376141 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.376145 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2596 2014-08-13 18:06:07.376150 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] acting_up_affected newup [289,327,376] newacting [289,327,376] 2014-08-13 18:06:07.376155 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.293 2014-08-13 18:06:07.376166 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.376171 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.338 2014-08-13 18:06:07.376176 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376]/[289,341,327] r=-1 lpr=194491 pi=192978-194490/33 crt=191727'473 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194492 2014-08-13 18:06:07.376182 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.373 2014-08-13 18:06:07.376192 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 18:06:07.376193 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [289,327,376](289) acting [289,341,327](289)): not rw, up_thru 194489 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [289,327,376](289) acting [289,341,327](289)) : primary up 192968-194489 does not include interval 2014-08-13 18:06:07.376201 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.396 2014-08-13 18:06:07.376203 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194491/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] noting past interval(194491-194491 up [289,327,376](289) acting [289,341,327](289)) 2014-08-13 18:06:07.376211 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.376220 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.376213 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] up [289,327,376] -> [289,327,376], acting [289,341,327] -> [289,327,376], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> -1 2014-08-13 18:06:07.376242 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.376250 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.376258 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.376267 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.376275 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.376282 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.376289 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.376292 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.376300 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.376309 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.376317 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.376325 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.376319 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,314]/[374,12] -- 374/374 2014-08-13 18:06:07.376338 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.376334 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.376342 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.376357 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [289,327,376]/[289,327,376] -- 289/289 2014-08-13 18:06:07.381112 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381106 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381119 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.381128 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,314] newacting [374,12] 2014-08-13 18:06:07.381131 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381139 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.381141 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381149 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,4,314]/[374,4,12] r=1 lpr=194487 pi=192688-194485/48 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194488 2014-08-13 18:06:07.381170 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [289,327,376]/[289,327,376] -- 289/289 2014-08-13 18:06:07.381182 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381176 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [374,4,314](374) acting [374,4,12](374)): not rw, up_thru 194486 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [374,4,314](374) acting [374,4,12](374) maybe_went_rw) : primary up 192967-194486 includes interval 2014-08-13 18:06:07.381185 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.381189 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194485/194486/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194486-194487 up [374,4,314](374) acting [374,4,12](374) maybe_went_rw) 2014-08-13 18:06:07.381194 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381203 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381201 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,4,314] -> [374,314], acting [374,4,12] -> [374,12], acting_primary 374 -> 374, up_primary 374 -> 374, role 1 -> -1 2014-08-13 18:06:07.381213 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381220 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 18:06:07.381223 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.381231 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381232 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.381233 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.381242 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381243 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:07.381251 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] acting_up_affected newup [4,289,341] newacting [4,289,341] 2014-08-13 18:06:07.381253 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:07.381260 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.381262 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:07.381269 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [289,327,376] r=-1 lpr=194492 pi=192978-194491/34 crt=191727'473 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.381272 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381283 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:07.381287 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [289,327,376](289) acting [289,327,376](289)): not rw, up_thru 194492 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) : primary up 192968-194492 includes interval 2014-08-13 18:06:07.381293 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:07.381298 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194491/194492/194488) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] noting past interval(194492-194494 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 18:06:07.381303 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_role_change 2014-08-13 18:06:07.381309 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] up [289,327,376] -> [4,289,341], acting [289,327,376] -> [4,289,341], acting_primary 289 -> 4, up_primary 289 -> 4, role -1 -> 0 2014-08-13 18:06:07.381313 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.381321 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state 2014-08-13 18:06:07.381322 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 18:06:07.381330 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop 2014-08-13 18:06:07.381331 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:07.381340 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.381342 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381350 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 194495:2689 2014-08-13 18:06:07.381359 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.381368 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.381367 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,314]/[374,12,294] -- 374/374 2014-08-13 18:06:07.381377 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.381385 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381385 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.381389 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.381394 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.381399 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381403 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.381409 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,314] newacting [374,12,294] 2014-08-13 18:06:07.381412 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.381419 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.381421 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.381429 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12] r=-1 lpr=194488 pi=192688-194487/49 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194489 2014-08-13 18:06:07.381431 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.381440 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.381449 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] clear_stats 2014-08-13 18:06:07.381447 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194488 up [374,314](374) acting [374,12](374)): not rw, up_thru 194486 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194488-194488 up [374,314](374) acting [374,12](374)) : primary up 192967-194486 does not include interval 2014-08-13 18:06:07.381458 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.381457 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194488/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194488-194488 up [374,314](374) acting [374,12](374)) 2014-08-13 18:06:07.381467 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] on_role_change 2014-08-13 18:06:07.381469 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,314] -> [374,314], acting [374,12] -> [374,12,294], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 18:06:07.381475 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:07.381481 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381484 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-13 18:06:07.381490 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.381493 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-13 18:06:07.381500 7f5c7bde3700 15 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.381503 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381510 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:07.381513 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-13 18:06:07.381519 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:07.381524 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 4,289,341 unchanged 2014-08-13 18:06:07.381529 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:07.381534 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-13 18:06:07.381538 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381545 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.006244 8 0.001368 2014-08-13 18:06:07.381549 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:07.381557 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-13 18:06:07.381559 7f5c7bde3700 20 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:07.381567 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-13 18:06:07.381568 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 18:06:07.381575 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-13 18:06:07.381577 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:07.381586 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.381587 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381597 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-13 18:06:07.381605 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-13 18:06:07.381607 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,314]/[374,12,294] -- 374/374 2014-08-13 18:06:07.381615 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.381622 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381624 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381626 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.381636 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381634 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 18:06:07.381646 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381647 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [289,327,376](289) acting [289,341,327](289)) 2014-08-13 18:06:07.381658 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194489-194490 up [289,341](289) acting [289,341,327](289) maybe_went_rw) 2014-08-13 18:06:07.381667 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,294,12]/[374,12,294] -- 374/374 2014-08-13 18:06:07.381668 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194488 up [289,341](289) acting [289,341](289)) 2014-08-13 18:06:07.381681 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381679 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,289,341](4) acting [4,289,341](4) maybe_went_rw) 2014-08-13 18:06:07.381683 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.381689 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194483-194484 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 18:06:07.381697 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381701 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194482-194482 up [289,327,376](289) acting [289,341,376](289)) 2014-08-13 18:06:07.381706 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,294,12] newacting [374,12,294] 2014-08-13 18:06:07.381714 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,289,327,341,376 down blocked_by {} 2014-08-13 18:06:07.381717 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.381725 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.381726 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,314]/[374,12,294] r=-1 lpr=194489 pi=192688-194488/50 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194491 2014-08-13 18:06:07.381741 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194495:2690 2014-08-13 18:06:07.381743 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194489-194490 up [374,314](374) acting [374,12,294](374)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194489-194490 up [374,314](374) acting [374,12,294](374) maybe_went_rw) : primary up 192967-194489 includes interval 2014-08-13 18:06:07.381751 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.289 2014-08-13 18:06:07.381754 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194488/194489/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194489-194490 up [374,314](374) acting [374,12,294](374) maybe_went_rw) 2014-08-13 18:06:07.381762 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.327 2014-08-13 18:06:07.381765 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,314] -> [374,294,12], acting [374,12,294] -> [374,12,294], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 18:06:07.381772 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.341 2014-08-13 18:06:07.381776 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381781 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.376 2014-08-13 18:06:07.381786 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.381792 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.381796 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.381802 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.381806 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:07.381816 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:07.381825 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:07.381833 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:07.381843 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:07.381852 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:07.381862 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.381861 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 18:06:07.381871 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:07.381880 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.381891 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 18:06:07.381905 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381902 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,294,12]/[374,294,12] -- 374/374 2014-08-13 18:06:07.381909 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.381917 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.381918 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345] newacting [292,345] 2014-08-13 18:06:07.381923 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.381927 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.381933 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.381938 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.314829 3 0.000126 2014-08-13 18:06:07.381942 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] acting_up_affected newup [374,294,12] newacting [374,294,12] 2014-08-13 18:06:07.381950 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started 0.314880 0 0.000000 2014-08-13 18:06:07.381952 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.381961 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.381962 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12]/[374,12,294] r=-1 lpr=194491 pi=192688-194490/51 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] set_last_peering_reset 194492 2014-08-13 18:06:07.381969 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194486 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.381979 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194488 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.381979 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [374,294,12](374) acting [374,12,294](374)): not rw, up_thru 194489 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [374,294,12](374) acting [374,12,294](374)) : primary up 192967-194489 does not include interval 2014-08-13 18:06:07.381988 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194488 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.381989 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194491/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] noting past interval(194491-194491 up [374,294,12](374) acting [374,12,294](374)) 2014-08-13 18:06:07.381996 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194488 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345] newacting [292,345] 2014-08-13 18:06:07.382000 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] up [374,294,12] -> [374,294,12], acting [374,12,294] -> [374,294,12], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> -1 2014-08-13 18:06:07.382006 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194488 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.382011 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 18:06:07.382015 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345,4] r=2 lpr=194488 pi=192975-194484/24 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.382021 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382030 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.382033 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [292,345,4](292) acting [292,345,4](292)): not rw, up_thru 194485 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [292,345,4](292) acting [292,345,4](292) maybe_went_rw) : primary up 192972-194485 includes interval 2014-08-13 18:06:07.382039 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:07.382043 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194485/194485/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [292,345,4](292) acting [292,345,4](292) maybe_went_rw) 2014-08-13 18:06:07.382048 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:07.382054 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345,4] -> [292,345], acting [292,345,4] -> [292,345], acting_primary 292 -> 292, up_primary 292 -> 292, role 2 -> -1 2014-08-13 18:06:07.382058 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:07.382065 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382066 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 18:06:07.382074 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382076 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:07.382083 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382086 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:07.382092 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.382095 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_recovery 2014-08-13 18:06:07.382101 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.382104 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:07.382110 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.382114 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382119 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382128 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.382137 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.382136 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,294,12]/[374,294,12] -- 374/374 2014-08-13 18:06:07.382150 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382146 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.382153 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.382159 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382167 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.382167 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.382176 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.382178 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382186 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382198 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,294,12]/[374,294,12] -- 374/374 2014-08-13 18:06:07.382211 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382205 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 18:06:07.382220 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382214 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.382223 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382223 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.382232 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382233 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382242 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382251 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,314] -- 374/374 2014-08-13 18:06:07.382263 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382258 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345]/[292,345] -- 292/292 2014-08-13 18:06:07.382268 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382266 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:07.382270 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382275 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.382279 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382284 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,314] 2014-08-13 18:06:07.382289 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382294 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.382303 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,294,12] r=-1 lpr=194492 pi=192688-194491/52 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194495 2014-08-13 18:06:07.382308 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 18:06:07.382319 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382320 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [374,294,12](374) acting [374,294,12](374)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [374,294,12](374) acting [374,294,12](374) maybe_went_rw) : primary up 192967-194492 includes interval 2014-08-13 18:06:07.382321 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382330 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382329 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194491/194492/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] noting past interval(194492-194494 up [374,294,12](374) acting [374,294,12](374) maybe_went_rw) 2014-08-13 18:06:07.382339 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345,349] newacting [292,345,349] 2014-08-13 18:06:07.382341 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] up [374,294,12] -> [374,4,314], acting [374,294,12] -> [374,4,314], acting_primary 374 -> 374, up_primary 374 -> 374, role -1 -> 1 2014-08-13 18:06:07.382348 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.382352 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 18:06:07.382357 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345] r=-1 lpr=194488 pi=192975-194487/25 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.382362 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382371 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.382373 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [292,345](292) acting [292,345](292)): not rw, up_thru 194488 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [292,345](292) acting [292,345](292) maybe_went_rw) : primary up 192972-194488 includes interval 2014-08-13 18:06:07.382380 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:07.382383 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194488/194488/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [292,345](292) acting [292,345](292) maybe_went_rw) 2014-08-13 18:06:07.382389 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:07.382393 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345] -> [292,345,349], acting [292,345] -> [292,345,349], acting_primary 292 -> 292, up_primary 292 -> 292, role -1 -> -1 2014-08-13 18:06:07.382399 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:07.382404 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382408 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_change 2014-08-13 18:06:07.382414 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382418 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:07.382423 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382428 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:07.382432 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.382437 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] on_role_change 2014-08-13 18:06:07.382441 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.382446 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.382450 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.382455 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] cancel_recovery 2014-08-13 18:06:07.382459 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382464 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:07.382468 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.382473 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382478 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.382484 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map 2014-08-13 18:06:07.382487 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.382496 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.382504 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382524 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 18:06:07.382534 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382536 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382545 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] update_heartbeat_peers unchanged 2014-08-13 18:06:07.382551 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382556 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] take_waiters 2014-08-13 18:06:07.382561 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382567 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Reset 0.497009 10 0.005973 2014-08-13 18:06:07.382579 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started 2014-08-13 18:06:07.382580 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 18:06:07.382591 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382589 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Start 2014-08-13 18:06:07.382594 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382598 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: transitioning to Stray 2014-08-13 18:06:07.382602 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382609 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.382611 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382621 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Started/Stray 2014-08-13 18:06:07.382630 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,349]/[292,345,349] -- 292/292 2014-08-13 18:06:07.382640 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382632 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.382642 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382642 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.382652 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382661 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382677 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 18:06:07.382687 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382689 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382697 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382712 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.382705 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] acting_up_affected newup [292,345,4] newacting [292,345,4] 2014-08-13 18:06:07.382717 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.382725 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,349] r=-1 lpr=194491 pi=192975-194490/26 crt=192888'477 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.382740 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [292,345,349](292) acting [292,345,349](292)): not rw, up_thru 194491 up_from 192972 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [292,345,349](292) acting [292,345,349](292) maybe_went_rw) : primary up 192972-194491 includes interval 2014-08-13 18:06:07.382749 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194491/194491/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [292,345,349](292) acting [292,345,349](292) maybe_went_rw) 2014-08-13 18:06:07.382745 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 18:06:07.382763 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.382760 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] up [292,345,349] -> [292,345,4], acting [292,345,349] -> [292,345,4], acting_primary 292 -> 292, up_primary 292 -> 292, role -1 -> 2 2014-08-13 18:06:07.382766 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.382771 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382776 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,20] newacting [328,20] 2014-08-13 18:06:07.382780 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382786 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.382789 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382797 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.311483 3 0.000138 2014-08-13 18:06:07.382798 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.382807 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.382808 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started 0.311538 0 0.000000 2014-08-13 18:06:07.382815 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.382819 7f5c7bde3700 5 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.382823 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382828 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194486 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.382832 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.382837 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194488 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.382842 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.382846 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194488 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.382851 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.382855 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194488 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,20] newacting [328,20] 2014-08-13 18:06:07.382859 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382864 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194488 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.382867 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.382874 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,4,20] r=1 lpr=194488 pi=192975-194484/25 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.382876 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.382885 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.382891 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [328,4,20](328) acting [328,4,20](328)): not rw, up_thru 194485 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194485-194487 up [328,4,20](328) acting [328,4,20](328) maybe_went_rw) : primary up 192949-194485 includes interval 2014-08-13 18:06:07.382895 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.382901 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194485/194485/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194485-194487 up [328,4,20](328) acting [328,4,20](328) maybe_went_rw) 2014-08-13 18:06:07.382911 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,4,20] -> [328,20], acting [328,4,20] -> [328,20], acting_primary 328 -> 328, up_primary 328 -> 328, role 1 -> -1 2014-08-13 18:06:07.382921 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382929 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.382930 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.382937 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.382940 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.382947 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.382950 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Reset 0.000989 8 0.000829 2014-08-13 18:06:07.382955 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.382962 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.382963 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.382971 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.382972 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.382979 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.382982 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.382991 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.382990 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Start 0.000019 0 0.000000 2014-08-13 18:06:07.383000 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.383001 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.383007 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.383012 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.383016 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.383023 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.383023 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.383035 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383058 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 18:06:07.383069 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383074 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.383072 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383065 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] flushed 2014-08-13 18:06:07.383080 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383089 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383101 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 18:06:07.383109 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,20]/[328,20] -- 328/328 2014-08-13 18:06:07.383120 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383120 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383122 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383124 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383131 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383134 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383140 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383143 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45] newacting [343,45] 2014-08-13 18:06:07.383158 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.383167 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45,4] r=2 lpr=194487 pi=193262-194485/31 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194488 2014-08-13 18:06:07.383160 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 18:06:07.383177 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383180 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383185 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [343,45,4](343) acting [343,45,4](343)): not rw, up_thru 194486 up_from 192942 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [343,45,4](343) acting [343,45,4](343) maybe_went_rw) : primary up 192942-194486 includes interval 2014-08-13 18:06:07.383188 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383197 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,10,20] newacting [328,10,20] 2014-08-13 18:06:07.383196 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194485/194486/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194486-194487 up [343,45,4](343) acting [343,45,4](343) maybe_went_rw) 2014-08-13 18:06:07.383206 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.383208 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45,4] -> [343,45], acting [343,45,4] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role 2 -> -1 2014-08-13 18:06:07.383215 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,20] r=-1 lpr=194488 pi=192975-194487/26 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.383219 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:07.383229 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.383232 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [328,20](328) acting [328,20](328)): not rw, up_thru 194488 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194488-194490 up [328,20](328) acting [328,20](328) maybe_went_rw) : primary up 192949-194488 includes interval 2014-08-13 18:06:07.383237 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.383241 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194488/194488/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194488-194490 up [328,20](328) acting [328,20](328) maybe_went_rw) 2014-08-13 18:06:07.383251 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 18:06:07.383252 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,20] -> [328,10,20], acting [328,20] -> [328,10,20], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> -1 2014-08-13 18:06:07.383262 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 18:06:07.383264 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.383271 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 18:06:07.383272 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.383281 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.383280 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:07.383290 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.383291 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 18:06:07.383299 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.383300 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 18:06:07.383307 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.383310 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_role_change 2014-08-13 18:06:07.383315 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.383319 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.383325 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.383314 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.383329 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 18:06:07.383336 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.383338 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 18:06:07.383345 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.383348 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383354 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.383363 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383369 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 18:06:07.383383 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383384 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 18:06:07.383386 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383394 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383395 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383397 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383404 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383406 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383414 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383422 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45]/[343,45] -- 343/343 2014-08-13 18:06:07.383434 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383434 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 18:06:07.383436 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383444 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383447 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383446 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383455 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383458 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383464 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383484 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,303]/[343,45,303] -- 343/343 2014-08-13 18:06:07.383497 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383496 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,10,20]/[328,10,20] -- 328/328 2014-08-13 18:06:07.383500 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383509 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383509 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383511 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383518 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45,303] newacting [343,45,303] 2014-08-13 18:06:07.383520 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383528 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.383529 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383536 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45] r=-1 lpr=194488 pi=193262-194487/32 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194491 2014-08-13 18:06:07.383547 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 18:06:07.383558 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383552 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [343,45](343) acting [343,45](343)): not rw, up_thru 194488 up_from 192942 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [343,45](343) acting [343,45](343) maybe_went_rw) : primary up 192942-194488 includes interval 2014-08-13 18:06:07.383561 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.383563 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194488/194488/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194488-194490 up [343,45](343) acting [343,45](343) maybe_went_rw) 2014-08-13 18:06:07.383569 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194486 2014-08-13 18:06:07.383574 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45] -> [343,45,303], acting [343,45] -> [343,45,303], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 18:06:07.383578 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] acting_up_affected newup [328,4,20] newacting [328,4,20] 2014-08-13 18:06:07.383585 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:07.383587 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.383594 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.383595 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,10,20] r=-1 lpr=194491 pi=192975-194490/27 crt=192888'478 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.383603 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.383612 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [328,10,20](328) acting [328,10,20](328)): not rw, up_thru 194491 up_from 192949 last_epoch_clean 194486 generate_past_intervals interval(194491-194494 up [328,10,20](328) acting [328,10,20](328) maybe_went_rw) : primary up 192949-194491 includes interval 2014-08-13 18:06:07.383612 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 18:06:07.383622 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 18:06:07.383621 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194491/194491/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] noting past interval(194491-194494 up [328,10,20](328) acting [328,10,20](328) maybe_went_rw) 2014-08-13 18:06:07.383631 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 18:06:07.383632 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] up [328,10,20] -> [328,4,20], acting [328,10,20] -> [328,4,20], acting_primary 328 -> 328, up_primary 328 -> 328, role -1 -> 1 2014-08-13 18:06:07.383641 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:07.383644 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.383651 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 18:06:07.383652 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.383660 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 18:06:07.383661 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.383671 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 18:06:07.383672 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.383680 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 18:06:07.383681 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.383689 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change_cleanup 2014-08-13 18:06:07.383690 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383698 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] on_change 2014-08-13 18:06:07.383707 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] exit NotTrimming 2014-08-13 18:06:07.383713 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,303]/[343,45] -- 343/343 2014-08-13 18:06:07.383716 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter NotTrimming 2014-08-13 18:06:07.383727 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383724 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] on_role_change 2014-08-13 18:06:07.383730 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383733 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.383739 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383741 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] cancel_recovery 2014-08-13 18:06:07.383749 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] clear_recovery_state 2014-08-13 18:06:07.383748 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] acting_up_affected newup [343,45,303] newacting [343,45] 2014-08-13 18:06:07.383759 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.383759 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383768 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303] r=-1 lpr=194491 pi=193262-194490/33 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] set_last_peering_reset 194492 2014-08-13 18:06:07.383769 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:07.383784 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [343,45,303](343) acting [343,45,303](343)): not rw, up_thru 194488 up_from 192942 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [343,45,303](343) acting [343,45,303](343)) : primary up 192942-194488 does not include interval 2014-08-13 18:06:07.383794 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194491/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] noting past interval(194491-194491 up [343,45,303](343) acting [343,45,303](343)) 2014-08-13 18:06:07.383805 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] up [343,45,303] -> [343,45,303], acting [343,45,303] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 18:06:07.383810 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.383816 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 18:06:07.383820 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:07.383827 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.383829 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Reset 0.001010 8 0.000816 2014-08-13 18:06:07.383836 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.383841 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started 2014-08-13 18:06:07.383846 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_copy_ops 2014-08-13 18:06:07.383851 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Start 2014-08-13 18:06:07.383856 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_flush_ops 2014-08-13 18:06:07.383861 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.383866 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change_cleanup 2014-08-13 18:06:07.383872 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.383876 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 18:06:07.383883 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/Stray 2014-08-13 18:06:07.383886 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit NotTrimming 2014-08-13 18:06:07.383895 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter NotTrimming 2014-08-13 18:06:07.383898 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.383904 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_recovery 2014-08-13 18:06:07.383908 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.383914 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] clear_recovery_state 2014-08-13 18:06:07.383925 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383947 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,303]/[343,45] -- 343/343 2014-08-13 18:06:07.383960 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.383961 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.383963 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.383972 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.383981 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.383990 7f5c7bde3700 10 osd.4 pg_epoch: 194487 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 18:06:07.384008 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384001 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,303]/[343,45] -- 343/343 2014-08-13 18:06:07.384014 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384012 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384016 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.384021 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384026 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384031 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319] newacting [45,319] 2014-08-13 18:06:07.384036 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384041 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.384050 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319,4] r=2 lpr=194487 pi=193186-194485/31 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194488 2014-08-13 18:06:07.384055 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,4]/[343,45] -- 343/343 2014-08-13 18:06:07.384068 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384068 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [45,319,4](45) acting [45,319,4](45)): not rw, up_thru 194486 up_from 192974 last_epoch_clean 194480 generate_past_intervals interval(194486-194487 up [45,319,4](45) acting [45,319,4](45) maybe_went_rw) : primary up 192974-194486 includes interval 2014-08-13 18:06:07.384071 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:07.384077 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194485/194486/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194486-194487 up [45,319,4](45) acting [45,319,4](45) maybe_went_rw) 2014-08-13 18:06:07.384081 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384088 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319,4] -> [45,319], acting [45,319,4] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role 2 -> -1 2014-08-13 18:06:07.384090 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45] 2014-08-13 18:06:07.384099 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.384100 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.384108 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.384110 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,303]/[343,45] r=-1 lpr=194492 pi=193262-194491/34 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194495 2014-08-13 18:06:07.384117 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.384127 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.384127 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [343,45,303](343) acting [343,45](343)): not rw, up_thru 194492 up_from 192942 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [343,45,303](343) acting [343,45](343) maybe_went_rw) : primary up 192942-194492 includes interval 2014-08-13 18:06:07.384136 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.384138 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194491/194492/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] noting past interval(194492-194494 up [343,45,303](343) acting [343,45](343) maybe_went_rw) 2014-08-13 18:06:07.384145 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.384149 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] up [343,45,303] -> [343,45,4], acting [343,45] -> [343,45], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> -1 2014-08-13 18:06:07.384154 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.384168 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 18:06:07.384172 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.384178 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.384183 7f5c7bde3700 20 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.384188 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.384192 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 18:06:07.384199 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_copy_ops 2014-08-13 18:06:07.384201 7f5c7bde3700 15 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.384209 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_flush_ops 2014-08-13 18:06:07.384210 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.384218 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.384218 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change_cleanup 2014-08-13 18:06:07.384230 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384229 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] on_change 2014-08-13 18:06:07.384242 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit NotTrimming 2014-08-13 18:06:07.384252 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter NotTrimming 2014-08-13 18:06:07.384253 7f5c7bde3700 10 osd.4 pg_epoch: 194488 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 18:06:07.384261 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] cancel_recovery 2014-08-13 18:06:07.384270 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384270 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] clear_recovery_state 2014-08-13 18:06:07.384274 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384281 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384283 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384292 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384292 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_activate_map 2014-08-13 18:06:07.384292 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] flushed 2014-08-13 18:06:07.384313 7f5c7bde3700 10 osd.4 pg_epoch: 194489 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319]/[45,319] -- 45/45 2014-08-13 18:06:07.384326 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384328 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384337 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384340 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] update_heartbeat_peers unchanged 2014-08-13 18:06:07.384346 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384351 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] take_waiters 2014-08-13 18:06:07.384361 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Reset 0.568732 10 0.001014 2014-08-13 18:06:07.384367 7f5c7bde3700 10 osd.4 pg_epoch: 194490 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,26]/[45,319,26] -- 45/45 2014-08-13 18:06:07.384380 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384373 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started 2014-08-13 18:06:07.384383 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384384 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Start 2014-08-13 18:06:07.384393 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: transitioning to Stray 2014-08-13 18:06:07.384392 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384422 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Start 0.000038 0 0.000000 2014-08-13 18:06:07.384425 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319,26] newacting [45,319,26] 2014-08-13 18:06:07.384436 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.384435 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Started/Stray 2014-08-13 18:06:07.384446 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319] r=-1 lpr=194488 pi=193186-194487/32 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194491 2014-08-13 18:06:07.384450 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.384461 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.384463 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [45,319](45) acting [45,319](45)): not rw, up_thru 194488 up_from 192974 last_epoch_clean 194480 generate_past_intervals interval(194488-194490 up [45,319](45) acting [45,319](45) maybe_went_rw) : primary up 192974-194488 includes interval 2014-08-13 18:06:07.384489 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194488/194488/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194488-194490 up [45,319](45) acting [45,319](45) maybe_went_rw) 2014-08-13 18:06:07.384499 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319] -> [45,319,26], acting [45,319] -> [45,319,26], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 18:06:07.384516 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.384510 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.384518 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.384527 7f5c7bde3700 15 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.384536 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.384544 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.384544 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 18:06:07.384553 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.384563 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384562 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:07.384567 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Reset advmap 2014-08-13 18:06:07.384571 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.384578 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.384582 7f5c7bde3700 20 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.384588 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [348,331] newacting [332,337] 2014-08-13 18:06:07.384591 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.384599 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.384599 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.384609 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [4,348,331]/[4,332,337] r=0 lpr=194487 pi=193644-194485/32 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194488 2014-08-13 18:06:07.384610 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384628 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194486-194487 up [4,348,331](4) acting [4,332,337](4)): not rw, up_thru 194486 up_from 194485 last_epoch_clean 194484 generate_past_intervals interval(194486-194487 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) : primary up 194485-194486 includes interval 2014-08-13 18:06:07.384633 7f5c7bde3700 10 osd.4 pg_epoch: 194491 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,26]/[45,319] -- 45/45 2014-08-13 18:06:07.384648 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384639 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194485/194486/194485) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] noting past interval(194486-194487 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) 2014-08-13 18:06:07.384651 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384650 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] up [4,348,331] -> [348,331], acting [4,332,337] -> [332,337], acting_primary 4 -> 332, up_primary 4 -> 348, role 0 -> -1 2014-08-13 18:06:07.384661 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384663 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] clear_primary_state 2014-08-13 18:06:07.384670 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] acting_up_affected newup [45,319,26] newacting [45,319] 2014-08-13 18:06:07.384674 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] agent_stop 2014-08-13 18:06:07.384679 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.384684 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] on_change 2014-08-13 18:06:07.384688 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26] r=-1 lpr=194491 pi=193186-194490/33 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] set_last_peering_reset 194492 2014-08-13 18:06:07.384693 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.384703 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:07.384706 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [45,319,26](45) acting [45,319,26](45)): not rw, up_thru 194488 up_from 192974 last_epoch_clean 194480 generate_past_intervals interval(194491-194491 up [45,319,26](45) acting [45,319,26](45)) : primary up 192974-194488 does not include interval 2014-08-13 18:06:07.384714 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] cancel_copy_ops 2014-08-13 18:06:07.384717 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194491/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] noting past interval(194491-194491 up [45,319,26](45) acting [45,319,26](45)) 2014-08-13 18:06:07.384723 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] cancel_flush_ops 2014-08-13 18:06:07.384728 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] up [45,319,26] -> [45,319,26], acting [45,319,26] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 18:06:07.384733 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] on_change_cleanup 2014-08-13 18:06:07.384740 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.384743 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] on_change 2014-08-13 18:06:07.384749 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.384753 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] exit NotTrimming 2014-08-13 18:06:07.384758 7f5c7bde3700 15 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.384763 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped m=48] enter NotTrimming 2014-08-13 18:06:07.384768 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.384773 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_stats 2014-08-13 18:06:07.384778 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.384783 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.384787 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.384793 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_role_change 2014-08-13 18:06:07.384797 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.384802 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.384807 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.384812 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.384807 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] flushed 2014-08-13 18:06:07.384817 7f5c7bde3700 20 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.384822 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.384827 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.384836 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.384836 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384849 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384859 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 18:06:07.384873 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384870 7f5c7bde3700 10 osd.4 pg_epoch: 194492 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,26]/[45,319] -- 45/45 2014-08-13 18:06:07.384876 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.384884 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384885 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.384887 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384895 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384897 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384906 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384912 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [348,331]/[332,337] -- 348/332 2014-08-13 18:06:07.384925 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384926 7f5c7bde3700 10 osd.4 pg_epoch: 194493 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,26]/[45,319] -- 45/45 2014-08-13 18:06:07.384928 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.384939 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384937 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.384942 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.384947 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384952 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.384961 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.384969 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [332,337,379]/[332,337] -- 332/332 2014-08-13 18:06:07.384982 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384979 7f5c7bde3700 10 osd.4 pg_epoch: 194494 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,4]/[45,319] -- 45/45 2014-08-13 18:06:07.384985 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.384992 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.384994 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.384995 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:07.385004 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [332,337,379] newacting [332,337] 2014-08-13 18:06:07.385005 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194480 2014-08-13 18:06:07.385014 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.385014 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319] 2014-08-13 18:06:07.385023 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [348,331]/[332,337] r=-1 lpr=194488 pi=193644-194487/33 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194491 2014-08-13 18:06:07.385025 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.385034 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,26]/[45,319] r=-1 lpr=194492 pi=193186-194491/34 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194495 2014-08-13 18:06:07.385044 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [348,331](348) acting [332,337](332)): not rw, up_thru 194488 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194488-194490 up [348,331](348) acting [332,337](332) maybe_went_rw) : primary up 192968-194488 includes interval 2014-08-13 18:06:07.385051 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [45,319,26](45) acting [45,319](45)): not rw, up_thru 194492 up_from 192974 last_epoch_clean 194480 generate_past_intervals interval(194492-194494 up [45,319,26](45) acting [45,319](45) maybe_went_rw) : primary up 192974-194492 includes interval 2014-08-13 18:06:07.385055 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194488/194488/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] noting past interval(194488-194490 up [348,331](348) acting [332,337](332) maybe_went_rw) 2014-08-13 18:06:07.385061 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194491/194492/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] noting past interval(194492-194494 up [45,319,26](45) acting [45,319](45) maybe_went_rw) 2014-08-13 18:06:07.385066 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] up [348,331] -> [332,337,379], acting [332,337] -> [332,337], acting_primary 332 -> 332, up_primary 348 -> 332, role -1 -> -1 2014-08-13 18:06:07.385072 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] up [45,319,26] -> [45,319,4], acting [45,319] -> [45,319], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> -1 2014-08-13 18:06:07.385078 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.385084 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.385088 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.385094 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.385098 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385103 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.385108 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.385113 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:07.385118 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.385123 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:07.385128 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.385132 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:07.385137 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:07.385141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] on_change 2014-08-13 18:06:07.385152 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:07.385148 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.385163 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.385165 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:07.385173 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.385175 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] cancel_recovery 2014-08-13 18:06:07.385182 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.385185 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:07.385193 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385195 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385205 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_activate_map 2014-08-13 18:06:07.385217 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [332,337,379]/[332,337,379] -- 332/332 2014-08-13 18:06:07.385231 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.385234 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.385243 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.385250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 18:06:07.385252 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] acting_up_affected newup [332,337,379] newacting [332,337,379] 2014-08-13 18:06:07.385260 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] take_waiters 2014-08-13 18:06:07.385262 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.385270 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Reset 0.429604 10 0.001033 2014-08-13 18:06:07.385272 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379]/[332,337] r=-1 lpr=194491 pi=193644-194490/34 crt=194422'12517 lcod 0'0 remapped NOTIFY m=48] set_last_peering_reset 194492 2014-08-13 18:06:07.385284 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started 2014-08-13 18:06:07.385292 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194491 up [332,337,379](332) acting [332,337](332)): not rw, up_thru 194488 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194491-194491 up [332,337,379](332) acting [332,337](332)) : primary up 192968-194488 does not include interval 2014-08-13 18:06:07.385294 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Start 2014-08-13 18:06:07.385302 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194491/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] noting past interval(194491-194491 up [332,337,379](332) acting [332,337](332)) 2014-08-13 18:06:07.385304 7f5c7bde3700 1 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: transitioning to Stray 2014-08-13 18:06:07.385312 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] up [332,337,379] -> [332,337,379], acting [332,337] -> [332,337,379], acting_primary 332 -> 332, up_primary 332 -> 332, role -1 -> -1 2014-08-13 18:06:07.385315 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.385324 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.385326 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Started/Stray 2014-08-13 18:06:07.385332 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.385339 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.385342 7f5c7c7e4700 15 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.385352 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.385361 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.385370 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.385378 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.385387 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.385396 7f5c7c7e4700 20 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.385407 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.385404 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] cancel_recovery 2014-08-13 18:06:07.385413 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:07.385422 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385425 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.385440 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.385450 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.385444 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [332,337,379]/[332,337,379] -- 332/332 2014-08-13 18:06:07.385459 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.385455 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 20 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.385461 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.385471 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.385474 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194484/194484 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.385480 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385494 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.20 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.385501 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [332,337,379]/[332,337,379] -- 332/332 2014-08-13 18:06:07.385514 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.385507 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,25,362 -> 4,20,25,362 2014-08-13 18:06:07.385519 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.385517 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.385521 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.385527 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.385536 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385534 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194492-194494 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 18:06:07.385547 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 18:06:07.385554 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [4,348,331]/[4,348,331] -- 4/4 2014-08-13 18:06:07.385566 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.385569 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:07.385557 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 18:06:07.385578 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.385586 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.385587 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [4,348,331] newacting [4,348,331] 2014-08-13 18:06:07.385597 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.385599 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: have osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.385606 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [332,337,379] r=-1 lpr=194492 pi=193644-194491/35 crt=194422'12517 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194495 2014-08-13 18:06:07.385612 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.25 2014-08-13 18:06:07.385623 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.303 2014-08-13 18:06:07.385632 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.362 2014-08-13 18:06:07.385640 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: already requested info from osd.373 2014-08-13 18:06:07.385625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194492-194494 up [332,337,379](332) acting [332,337,379](332)): not rw, up_thru 194492 up_from 192968 last_epoch_clean 194484 generate_past_intervals interval(194492-194494 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) : primary up 192968-194492 includes interval 2014-08-13 18:06:07.385658 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194491/194492/194488) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] noting past interval(194492-194494 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 18:06:07.385668 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.385668 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] up [332,337,379] -> [4,348,331], acting [332,337,379] -> [4,348,331], acting_primary 332 -> 4, up_primary 332 -> 4, role -1 -> 0 2014-08-13 18:06:07.385679 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] clear_primary_state 2014-08-13 18:06:07.385684 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 20 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.385689 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] agent_stop 2014-08-13 18:06:07.385699 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.385698 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.20 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.385710 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] publish_stats_to_osd 194495:22151 2014-08-13 18:06:07.385721 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385721 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.20 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.385731 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:07.385735 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,31,374 -> 4,20,31,374 2014-08-13 18:06:07.385637 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] flushed 2014-08-13 18:06:07.385745 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.385741 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:07.385747 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.385751 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385760 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385759 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [363,20,337](363) acting [363,20,337](363) maybe_went_rw) 2014-08-13 18:06:07.385769 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.385776 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [374,31](374) acting [374,31](374) maybe_went_rw) 2014-08-13 18:06:07.385778 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:07.385787 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] on_change 2014-08-13 18:06:07.385787 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,20,31,337,363,374 down blocked_by {} 2014-08-13 18:06:07.385798 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.385797 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:07.385811 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:07.385810 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: have osd.20 info 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.385821 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] clear_stats 2014-08-13 18:06:07.385823 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.31 2014-08-13 18:06:07.385832 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.337 2014-08-13 18:06:07.385831 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] requeue_ops 2014-08-13 18:06:07.385841 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.363 2014-08-13 18:06:07.385841 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] on_role_change 2014-08-13 18:06:07.385850 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.374 2014-08-13 18:06:07.385851 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] requeue_ops 2014-08-13 18:06:07.385860 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] cancel_recovery 2014-08-13 18:06:07.385869 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] clear_recovery_state 2014-08-13 18:06:07.385877 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.385879 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.385889 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_activate_map 2014-08-13 18:06:07.385891 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 291 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.385899 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] update_heartbeat_peers -> 4,331,348 2014-08-13 18:06:07.385910 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.385907 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194484/194484 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.385911 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] take_waiters 2014-08-13 18:06:07.385922 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Reset 0.647701 10 0.001192 2014-08-13 18:06:07.385923 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.291 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.385933 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started 2014-08-13 18:06:07.385936 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,367,378 -> 4,291,367,378 2014-08-13 18:06:07.385946 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.385943 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Start 2014-08-13 18:06:07.385948 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.385952 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: transitioning to Primary 2014-08-13 18:06:07.385959 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194492-194494 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 18:06:07.385964 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Start 0.000021 0 0.000000 2014-08-13 18:06:07.385971 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194491-194491 up [365,350,291](365) acting [291,350](291)) 2014-08-13 18:06:07.385975 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary 2014-08-13 18:06:07.385982 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 18:06:07.385986 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering 2014-08-13 18:06:07.385993 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.385996 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:07.386006 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] _calc_past_interval_range: already have past intervals back to 194484 2014-08-13 18:06:07.386004 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: have osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386016 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.350 2014-08-13 18:06:07.386017 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194492-194494 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 18:06:07.386027 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.365 2014-08-13 18:06:07.386031 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194491-194491 up [332,337,379](332) acting [332,337](332)) 2014-08-13 18:06:07.386036 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.367 2014-08-13 18:06:07.386042 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194488-194490 up [348,331](348) acting [332,337](332) maybe_went_rw) 2014-08-13 18:06:07.386046 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.378 2014-08-13 18:06:07.386053 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194486-194487 up [4,348,331](4) acting [4,332,337](4) maybe_went_rw) 2014-08-13 18:06:07.386072 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386065 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194485-194485 up [4,348,331](4) acting [4,348,331](4)) 2014-08-13 18:06:07.386075 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194483-194484 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 18:06:07.386086 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194482-194482 up [332,337,379](332) acting [332,337](332)) 2014-08-13 18:06:07.386093 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 321 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.386096 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 18:06:07.386108 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.386108 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.386121 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] publish_stats_to_osd 194495:22152 2014-08-13 18:06:07.386130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.331 2014-08-13 18:06:07.386145 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386141 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.332 2014-08-13 18:06:07.386151 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.337 2014-08-13 18:06:07.386152 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194487/194484 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 364 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.386168 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.348 2014-08-13 18:06:07.386171 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] sending info 2014-08-13 18:06:07.386178 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: querying info from osd.379 2014-08-13 18:06:07.386189 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_activate_map: Not dirtying info: last_persisted is 194487 while current is 194495 2014-08-13 18:06:07.386198 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.386220 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386225 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 340 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.386238 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.386256 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.386254 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 18:06:07.386263 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.386273 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 18:06:07.386284 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 18:06:07.386293 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,340,347 down blocked_by {} 2014-08-13 18:06:07.386303 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.386312 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.7 2014-08-13 18:06:07.386321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: have osd.340 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.386332 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.347 2014-08-13 18:06:07.386361 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386525 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:07.386531 7f5c7c7e4700 7 osd.4 194495 do_notify osd 285 on 1 PGs 2014-08-13 18:06:07.386561 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab32e0 don't know epoch, doing nothing 2014-08-13 18:06:07.386563 7f5c7c7e4700 7 osd.4 194495 do_notify osd 292 on 1 PGs 2014-08-13 18:06:07.386577 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a2520 don't know epoch, doing nothing 2014-08-13 18:06:07.386578 7f5c7c7e4700 7 osd.4 194495 do_notify osd 323 on 1 PGs 2014-08-13 18:06:07.386590 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:07.386592 7f5c7c7e4700 7 osd.4 194495 do_notify osd 343 on 1 PGs 2014-08-13 18:06:07.386578 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 45 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.386608 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x449a3c0 don't know epoch, doing nothing 2014-08-13 18:06:07.386610 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.44 on 1 PGs 2014-08-13 18:06:07.386603 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.45 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386623 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:07.386625 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.61 on 1 PGs 2014-08-13 18:06:07.386621 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.45 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386636 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a7bc0 don't know epoch, doing nothing 2014-08-13 18:06:07.386638 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.287 on 1 PGs 2014-08-13 18:06:07.386635 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,313,318 -> 4,45,313,318 2014-08-13 18:06:07.386646 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.386648 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.386658 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [303,342,45](303) acting [303,342,45](303) maybe_went_rw) 2014-08-13 18:06:07.386670 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [303,342,45](303) acting [313,318,45](313)) 2014-08-13 18:06:07.386680 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,45,303,313,318,342 down blocked_by {} 2014-08-13 18:06:07.386690 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.386701 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: have osd.45 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386712 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 18:06:07.386716 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:07.386723 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.289 on 1 PGs 2014-08-13 18:06:07.386722 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.313 2014-08-13 18:06:07.386731 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.318 2014-08-13 18:06:07.386739 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:07.386741 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.303 on 1 PGs 2014-08-13 18:06:07.386739 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 18:06:07.386755 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4fa5800 don't know epoch, doing nothing 2014-08-13 18:06:07.386757 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.306 on 1 PGs 2014-08-13 18:06:07.386768 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386769 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.386771 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.309 on 1 PGs 2014-08-13 18:06:07.386782 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c58420 don't know epoch, doing nothing 2014-08-13 18:06:07.386784 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.327 on 1 PGs 2014-08-13 18:06:07.386797 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing 2014-08-13 18:06:07.386800 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.331 on 1 PGs 2014-08-13 18:06:07.386788 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194487/194480 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 17 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.386804 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] sending info 2014-08-13 18:06:07.386812 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing 2014-08-13 18:06:07.386814 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.332 on 1 PGs 2014-08-13 18:06:07.386850 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:07.386852 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.337 on 1 PGs 2014-08-13 18:06:07.386855 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.386864 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c594a0 don't know epoch, doing nothing 2014-08-13 18:06:07.386866 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.339 on 1 PGs 2014-08-13 18:06:07.386879 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab4620 don't know epoch, doing nothing 2014-08-13 18:06:07.386881 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.341 on 1 PGs 2014-08-13 18:06:07.386893 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x571b700 don't know epoch, doing nothing 2014-08-13 18:06:07.386895 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.348 sending saperate messages on 1 PGs 2014-08-13 18:06:07.386903 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:07.386905 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.362 on 1 PGs 2014-08-13 18:06:07.386916 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5e880 don't know epoch, doing nothing 2014-08-13 18:06:07.386918 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.369 on 1 PGs 2014-08-13 18:06:07.386942 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.386944 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.376 on 1 PGs 2014-08-13 18:06:07.386860 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 44 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.386956 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2d120 don't know epoch, doing nothing 2014-08-13 18:06:07.386958 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.379 on 1 PGs 2014-08-13 18:06:07.386950 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194484/194484 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.44 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386969 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c5eb40 don't know epoch, doing nothing 2014-08-13 18:06:07.386971 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.381 on 1 PGs 2014-08-13 18:06:07.386967 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.44 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.386980 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,62,65 -> 4,44,62,65 2014-08-13 18:06:07.386990 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.392336 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.387012 7f5c7c7e4700 10 osd.4 pg_epoch: 194487 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 18:06:07.392359 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [44,48,314](44) acting [44,48,314](44) maybe_went_rw) 2014-08-13 18:06:07.392372 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [44,48,314](44) acting [65,62,44](65)) 2014-08-13 18:06:07.392378 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392384 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392380 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.392386 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0700 pg_notify(24.1bb epoch 194495) v5 2014-08-13 18:06:07.387634 7f5c4d76b700 10 osd.4 194495 OSD::ms_get_authorizer type=osd 2014-08-13 18:06:07.392383 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,48,62,65,314 down blocked_by {} 2014-08-13 18:06:07.392394 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.392390 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:07.392414 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.331 2014-08-13 18:06:07.392407 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [396,368] newacting [396,363] 2014-08-13 18:06:07.392417 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0700 2014-08-13 18:06:07.392407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: have osd.44 info 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.392419 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:07.392422 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.48 2014-08-13 18:06:07.392432 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.62 2014-08-13 18:06:07.392430 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.324851 3 0.000108 2014-08-13 18:06:07.392441 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.65 2014-08-13 18:06:07.392443 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.324905 0 0.000000 2014-08-13 18:06:07.392450 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.314 2014-08-13 18:06:07.392456 7f5c7c7e4700 5 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:07.392472 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392466 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194486 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.392475 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392482 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.392477 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:07.392486 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.392496 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392497 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392499 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df1880 pg_notify(22.48e(1) epoch 194495) v5 2014-08-13 18:06:07.392501 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.392495 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] acting_up_affected newup [396,368] newacting [396,363] 2014-08-13 18:06:07.392506 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4af98c0 don't know epoch, doing nothing 2014-08-13 18:06:07.392508 7f5c7bde3700 7 osd.4 194495 do_notify osd 6 on 1 PGs 2014-08-13 18:06:07.392505 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.392514 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368,4]/[396,363] r=-1 lpr=194488 pi=194462-194484/8 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194488 2014-08-13 18:06:07.392526 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4493f40 don't know epoch, doing nothing 2014-08-13 18:06:07.392527 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.289 2014-08-13 18:06:07.392529 7f5c7bde3700 7 osd.4 194495 do_notify osd 17 on 1 PGs 2014-08-13 18:06:07.392529 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df1880 2014-08-13 18:06:07.392541 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:07.392543 7f5c7bde3700 7 osd.4 194495 do_notify osd 45 on 1 PGs 2014-08-13 18:06:07.392534 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194485-194487 up [396,368,4](396) acting [396,363](396)): not rw, up_thru 194485 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194485-194487 up [396,368,4](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194485 includes interval 2014-08-13 18:06:07.392545 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194485/194485/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194485-194487 up [396,368,4](396) acting [396,363](396) maybe_went_rw) 2014-08-13 18:06:07.392556 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392556 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf1600 already has epoch 194495 2014-08-13 18:06:07.392558 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392559 7f5c7bde3700 7 osd.4 194495 do_notify osd 51 on 1 PGs 2014-08-13 18:06:07.392557 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368,4] -> [396,368], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 18:06:07.392572 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x44270c0 don't know epoch, doing nothing 2014-08-13 18:06:07.392569 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.392579 7f5c7bde3700 7 osd.4 194495 do_notify osd 321 on 1 PGs 2014-08-13 18:06:07.392587 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392589 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392581 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.392593 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:07.392592 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df5080 pg_notify(22.3af epoch 194495) v5 2014-08-13 18:06:07.392595 7f5c7bde3700 7 osd.4 194495 do_notify osd 328 on 1 PGs 2014-08-13 18:06:07.392592 7f5c7c7e4700 15 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.392607 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:07.392609 7f5c7bde3700 7 osd.4 194495 do_notify osd 345 on 1 PGs 2014-08-13 18:06:07.392604 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.392616 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.44 2014-08-13 18:06:07.392630 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:07.392630 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df5080 2014-08-13 18:06:07.392632 7f5c7bde3700 7 osd.4 194495 do_notify osd 364 on 1 PGs 2014-08-13 18:06:07.392658 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392659 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392684 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392685 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392687 7f5c835ef700 20 osd.4 194495 _dispatch 0x524d240 pg_notify(24.1bb(39) epoch 194495) v5 2014-08-13 18:06:07.392714 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.337 2014-08-13 18:06:07.392716 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x524d240 2014-08-13 18:06:07.392614 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.392745 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:07.392736 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.392751 7f5c7bde3700 7 osd.4 194495 do_notify osd 374 on 1 PGs 2014-08-13 18:06:07.392752 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.392767 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.392770 7f5c7bde3700 7 osd.4 194495 do_notify osd 376 on 1 PGs 2014-08-13 18:06:07.392764 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.392781 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing 2014-08-13 18:06:07.392774 7f5c7c7e4700 20 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.392783 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.7 on 1 PGs 2014-08-13 18:06:07.392785 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.392797 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:07.392799 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.10 on 1 PGs 2014-08-13 18:06:07.392798 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392801 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392794 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.392812 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:07.392806 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.392814 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.286 on 1 PGs 2014-08-13 18:06:07.392826 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4c59760 don't know epoch, doing nothing 2014-08-13 18:06:07.392828 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.293 on 1 PGs 2014-08-13 18:06:07.392837 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392839 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392840 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b1b5a0 don't know epoch, doing nothing 2014-08-13 18:06:07.392841 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352680 pg_notify(22.3af epoch 194495) v5 2014-08-13 18:06:07.392842 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.338 on 1 PGs 2014-08-13 18:06:07.392834 7f5c7c7e4700 10 osd.4 pg_epoch: 194488 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 18:06:07.392850 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.392854 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.392863 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.362 2014-08-13 18:06:07.392865 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352680 2014-08-13 18:06:07.392863 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.392873 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.392888 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392890 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392896 7f5c7c7e4700 10 osd.4 pg_epoch: 194489 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368]/[396,363] -- 396/396 2014-08-13 18:06:07.392909 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.392909 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392908 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:07.392913 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.340 on 1 PGs 2014-08-13 18:06:07.392916 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392918 7f5c835ef700 20 osd.4 194495 _dispatch 0x5353f00 pg_notify(22.481(49) epoch 194495) v5 2014-08-13 18:06:07.392912 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.392922 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.392928 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:07.392932 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.343 on 1 PGs 2014-08-13 18:06:07.392932 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.392945 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing 2014-08-13 18:06:07.392947 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.347 on 1 PGs 2014-08-13 18:06:07.392960 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:07.392961 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.373 on 1 PGs 2014-08-13 18:06:07.392956 7f5c7c7e4700 10 osd.4 pg_epoch: 194490 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 18:06:07.392967 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.287 2014-08-13 18:06:07.392969 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.392970 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5353f00 2014-08-13 18:06:07.392972 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.392974 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.376 on 1 PGs 2014-08-13 18:06:07.392971 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.392982 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.392990 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing 2014-08-13 18:06:07.392992 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.396 on 1 PGs 2014-08-13 18:06:07.392996 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.392998 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.392991 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [396,368,284] newacting [396,363] 2014-08-13 18:06:07.393001 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.393014 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368]/[396,363] r=-1 lpr=194488 pi=194462-194487/9 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194491 2014-08-13 18:06:07.393022 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393031 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194490 up [396,368](396) acting [396,363](396)): not rw, up_thru 194488 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194488-194490 up [396,368](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194488 includes interval 2014-08-13 18:06:07.393030 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 358 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.393032 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393042 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194488/194488/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194488-194490 up [396,368](396) acting [396,363](396) maybe_went_rw) 2014-08-13 18:06:07.393047 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.393057 7f5c835ef700 20 osd.4 194495 _dispatch 0x51569c0 osd_map(194487..194495 src has 185139..194495) v3 2014-08-13 18:06:07.393053 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368] -> [396,368,284], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 18:06:07.393065 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194487,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.393068 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.393069 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393071 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393065 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.393075 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.393084 7f5c7c7e4700 15 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.393093 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.393097 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393099 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393093 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.393100 7f5c835ef700 20 osd.4 194495 _dispatch 0x5354600 pg_notify(24.1bb(1) epoch 194495) v5 2014-08-13 18:06:07.393098 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 315 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.393103 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.393116 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.393117 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194484/194484 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393126 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.393137 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.393141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.315 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393147 7f5c7c7e4700 20 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.393163 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.332 2014-08-13 18:06:07.393166 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5354600 2014-08-13 18:06:07.393156 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,28,330 -> 4,28,315,330 2014-08-13 18:06:07.393165 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.393173 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.393174 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.393175 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.393191 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393193 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393186 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.393188 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194492-194494 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 18:06:07.393209 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393201 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 18:06:07.393211 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393214 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df01c0 pg_notify(22.481(42) epoch 194495) v5 2014-08-13 18:06:07.393212 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 18:06:07.393211 7f5c7c7e4700 10 osd.4 pg_epoch: 194491 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 18:06:07.393228 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.393224 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.393231 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.393237 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.17 2014-08-13 18:06:07.393245 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.303 2014-08-13 18:06:07.393248 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df01c0 2014-08-13 18:06:07.393241 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.393247 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.28 2014-08-13 18:06:07.393252 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.393257 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: have osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393275 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393270 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.327 2014-08-13 18:06:07.393281 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393276 7f5c7c7e4700 10 osd.4 pg_epoch: 194492 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 18:06:07.393282 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: already requested info from osd.330 2014-08-13 18:06:07.393290 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.393293 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.393303 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393305 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393306 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf1880 pg_notify(22.48e(34) epoch 194495) v5 2014-08-13 18:06:07.393302 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.393312 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.393312 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.393328 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.393330 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf1880 2014-08-13 18:06:07.393332 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 19 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.393335 7f5c7c7e4700 10 osd.4 pg_epoch: 194493 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,284]/[396,363] -- 396/396 2014-08-13 18:06:07.393348 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.393347 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] sending info 2014-08-13 18:06:07.393351 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.393361 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.393372 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.393369 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.393383 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393385 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393377 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 55 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.393389 7f5c7c7e4700 10 osd.4 pg_epoch: 194494 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 18:06:07.393391 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194484/194484 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.55 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393403 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:07.393406 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: Reset advmap 2014-08-13 18:06:07.393406 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.55 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393416 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] _calc_past_interval_range: already have past intervals back to 194463 2014-08-13 18:06:07.393425 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393419 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,339 -> 4,45,55,339 2014-08-13 18:06:07.393431 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.393427 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393426 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] acting_up_affected newup [396,368,4] newacting [396,363] 2014-08-13 18:06:07.393433 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.393434 7f5c835ef700 20 osd.4 194495 _dispatch 0x5353480 pg_notify(24.1bb(39) epoch 194495) v5 2014-08-13 18:06:07.393437 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:07.393446 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,284]/[396,363] r=-1 lpr=194491 pi=194462-194490/10 crt=0'0 lcod 0'0 remapped NOTIFY] set_last_peering_reset 194495 2014-08-13 18:06:07.393469 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194491-194494 up [396,368,284](396) acting [396,363](396)): not rw, up_thru 194491 up_from 192960 last_epoch_clean 194463 generate_past_intervals interval(194491-194494 up [396,368,284](396) acting [396,363](396) maybe_went_rw) : primary up 192960-194491 includes interval 2014-08-13 18:06:07.393478 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194491/194491/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] noting past interval(194491-194494 up [396,368,284](396) acting [396,363](396) maybe_went_rw) 2014-08-13 18:06:07.393444 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [342,55,24](342) acting [342,55,24](342) maybe_went_rw) 2014-08-13 18:06:07.393489 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] up [396,368,284] -> [396,368,4], acting [396,363] -> [396,363], acting_primary 396 -> 396, up_primary 396 -> 396, role -1 -> -1 2014-08-13 18:06:07.393500 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.393495 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [342,55,24](342) acting [339,45,24](339)) 2014-08-13 18:06:07.393510 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] publish_stats_to_osd -- not primary 2014-08-13 18:06:07.393510 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,24,45,55,339,342 down blocked_by {} 2014-08-13 18:06:07.393521 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.393522 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.393533 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:07.393534 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.24 2014-08-13 18:06:07.393548 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.379 2014-08-13 18:06:07.393544 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.45 2014-08-13 18:06:07.393542 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:07.393550 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5353480 2014-08-13 18:06:07.393552 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: have osd.55 info 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.393556 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:07.393566 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:07.393564 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.339 2014-08-13 18:06:07.393576 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 18:06:07.393576 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:07.393587 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:07.393596 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] cancel_recovery 2014-08-13 18:06:07.393605 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] clear_recovery_state 2014-08-13 18:06:07.393614 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:07.393627 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.393625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 18:06:07.393649 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] update_heartbeat_peers unchanged 2014-08-13 18:06:07.393659 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 18:06:07.393667 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.393669 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Reset 0.001213 8 0.001025 2014-08-13 18:06:07.393681 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started 2014-08-13 18:06:07.393691 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Start 2014-08-13 18:06:07.393700 7f5c7c7e4700 1 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: transitioning to Stray 2014-08-13 18:06:07.393711 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Start 0.000020 0 0.000000 2014-08-13 18:06:07.393722 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/Stray 2014-08-13 18:06:07.393605 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.393734 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194486 while current is 194495 2014-08-13 18:06:07.393744 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 NullEvt 2014-08-13 18:06:07.393784 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.393845 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.393854 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing 2014-08-13 18:06:07.393856 7f5c7c7e4700 7 osd.4 194495 do_notify osd 396 on 1 PGs 2014-08-13 18:06:07.393892 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194486/194486 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 30 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.393908 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.393941 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.393946 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194487/194480 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 285 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.393959 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] sending info 2014-08-13 18:06:07.394003 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394007 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.394021 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194484/194484 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.61 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.394036 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.61 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.394048 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,362 -> 4,44,61,362 2014-08-13 18:06:07.394058 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.394060 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.394070 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [61,369,339](61) acting [61,369,339](61) maybe_went_rw) 2014-08-13 18:06:07.394081 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [61,369,339](61) acting [44,362,61](44)) 2014-08-13 18:06:07.394090 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,44,61,339,362,369 down blocked_by {} 2014-08-13 18:06:07.394100 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.394111 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.44 2014-08-13 18:06:07.394119 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: have osd.61 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.394130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.339 2014-08-13 18:06:07.394138 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.362 2014-08-13 18:06:07.394146 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 18:06:07.394180 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394197 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 289 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.394211 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.394234 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,50,289 unchanged 2014-08-13 18:06:07.394244 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.394254 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 18:06:07.394265 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 18:06:07.394275 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,32,50,289 down blocked_by {} 2014-08-13 18:06:07.394248 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394284 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.394292 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394293 7f5c835ef700 20 osd.4 194495 _dispatch 0x5351a40 pg_notify(22.48e(34) epoch 194495) v5 2014-08-13 18:06:07.394294 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.32 2014-08-13 18:06:07.394303 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.50 2014-08-13 18:06:07.394318 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.327 2014-08-13 18:06:07.394320 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5351a40 2014-08-13 18:06:07.394312 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: have osd.289 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.394341 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394353 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 356 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.394367 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194484/194484 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394378 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394381 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394382 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.356 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394394 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,55,61 -> 4,55,61,356 2014-08-13 18:06:07.394405 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.394408 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394410 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394411 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf5cc0 pg_notify(22.3af(36) epoch 194495) v5 2014-08-13 18:06:07.394407 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.394417 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194491-194494 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.394437 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.339 2014-08-13 18:06:07.394429 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior interval(194489-194490 up [55,61](55) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:07.394439 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf5cc0 2014-08-13 18:06:07.394439 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 18:06:07.394451 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.394462 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.50 2014-08-13 18:06:07.394470 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.55 2014-08-13 18:06:07.394482 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394485 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394479 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.61 2014-08-13 18:06:07.394488 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: already requested info from osd.65 2014-08-13 18:06:07.394502 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394504 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394505 7f5c835ef700 20 osd.4 194495 _dispatch 0x5355b00 pg_notify(22.1b8(2) epoch 194495) v5 2014-08-13 18:06:07.394497 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: have osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394526 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394530 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.7 2014-08-13 18:06:07.394537 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 50 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.394547 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5355b00 2014-08-13 18:06:07.394549 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394564 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.50 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394577 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,55,61,356 -> 4,50,55,61,356 2014-08-13 18:06:07.394587 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.394589 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394593 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194486/194483 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 371 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.394609 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394611 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394605 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.394636 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394638 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394639 7f5c835ef700 20 osd.4 194495 _dispatch 0x5353800 pg_notify(22.481(42) epoch 194495) v5 2014-08-13 18:06:07.394646 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394651 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 32 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.394662 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.32 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.394676 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.32 has stray content: 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.394689 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,50,289 -> 4,32,50,289 2014-08-13 18:06:07.394687 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.309 2014-08-13 18:06:07.394698 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.394699 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5353800 2014-08-13 18:06:07.394701 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394688 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] flushed 2014-08-13 18:06:07.394705 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194487/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 31 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.394717 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] sending info 2014-08-13 18:06:07.394728 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394730 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394749 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394751 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394752 7f5c835ef700 20 osd.4 194495 _dispatch 0x5356ac0 pg_notify(22.3d9(54) epoch 194495) v5 2014-08-13 18:06:07.394768 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394764 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 337 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.394773 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194486/194486 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 323 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.394786 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.293 2014-08-13 18:06:07.394788 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5356ac0 2014-08-13 18:06:07.394786 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.394786 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.337 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394810 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394812 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394804 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.337 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.394819 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.394818 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,374 -> 4,20,31,337,374 2014-08-13 18:06:07.394829 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.394834 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.394824 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 287 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:07.394836 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194484/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.394839 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.394851 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309 unchanged 2014-08-13 18:06:07.394860 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.394853 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.394862 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.394864 7f5c835ef700 20 osd.4 194495 _dispatch 0x5354280 pg_notify(22.481(49) epoch 194495) v5 2014-08-13 18:06:07.394866 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.394862 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.394870 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 343 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.394872 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.394883 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194486/194486 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.394885 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 18:06:07.394901 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.381 2014-08-13 18:06:07.394903 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5354280 2014-08-13 18:06:07.394895 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194485-194487 up [4,287,309](4) acting [4,287,309](4) maybe_went_rw) 2014-08-13 18:06:07.394898 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 18:06:07.394909 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.394920 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 18:06:07.394906 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 18:06:07.394932 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 18:06:07.394939 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.394942 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,10,286,343 down blocked_by {} 2014-08-13 18:06:07.394950 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.394953 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.394962 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 18:06:07.394964 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.10 2014-08-13 18:06:07.394971 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.306 2014-08-13 18:06:07.394974 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.286 2014-08-13 18:06:07.394979 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.309 2014-08-13 18:06:07.394988 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.381 2014-08-13 18:06:07.394986 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: have osd.343 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.395014 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395017 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395036 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 292 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395040 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 15 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.395049 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395059 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395062 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395056 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194484/194484 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.15 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395085 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395088 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395090 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395081 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.15 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395091 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0fc0 pg_notify(22.3af(36) epoch 194495) v5 2014-08-13 18:06:07.395090 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 332 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.395094 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,21,310 -> 4,15,21,310 2014-08-13 18:06:07.395105 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395103 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194484/194484 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395107 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.395119 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.369 2014-08-13 18:06:07.395122 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0fc0 2014-08-13 18:06:07.395121 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [303,15,342](303) acting [303,15,342](303) maybe_went_rw) 2014-08-13 18:06:07.395121 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.332 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395136 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,348 -> 4,331,332,348 2014-08-13 18:06:07.395135 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [303,15,342](303) acting [310,21,15](310)) 2014-08-13 18:06:07.395149 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395150 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395152 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395151 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.395149 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,15,21,303,310,342 down blocked_by {} 2014-08-13 18:06:07.395167 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395175 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395176 7f5c835ef700 20 osd.4 194495 _dispatch 0x5353b80 pg_notify(22.481(47) epoch 194495) v5 2014-08-13 18:06:07.395172 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.395180 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194492-194494 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 18:06:07.395184 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: have osd.15 info 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395199 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.306 2014-08-13 18:06:07.395202 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5353b80 2014-08-13 18:06:07.395196 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.21 2014-08-13 18:06:07.395195 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior interval(194491-194491 up [332,337,379](332) acting [332,337](332)) 2014-08-13 18:06:07.395205 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 18:06:07.395211 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 18:06:07.395215 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.310 2014-08-13 18:06:07.395224 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395226 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395223 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.395224 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.342 2014-08-13 18:06:07.395235 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.331 2014-08-13 18:06:07.395249 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395251 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395253 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395252 7f5c835ef700 20 osd.4 194495 _dispatch 0x4cfaf40 osd_map(194488..194495 src has 185139..194495) v3 2014-08-13 18:06:07.395244 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: have osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395259 7f5c835ef700 3 osd.4 194495 handle_osd_map epochs [194488,194495], i have 194495, src has [185139,194495] 2014-08-13 18:06:07.395262 7f5c835ef700 10 osd.4 194495 no new maps here, dropping 2014-08-13 18:06:07.395264 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395257 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.337 2014-08-13 18:06:07.395265 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.348 2014-08-13 18:06:07.395279 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395281 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395275 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: already requested info from osd.379 2014-08-13 18:06:07.395283 7f5c835ef700 20 osd.4 194495 _dispatch 0x5357380 pg_notify(22.3d5(2) epoch 194495) v5 2014-08-13 18:06:07.395304 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395305 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.10 2014-08-13 18:06:07.395308 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5357380 2014-08-13 18:06:07.395317 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 331 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.395331 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395333 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395334 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194483/194483 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.331 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395281 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 362 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.395350 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395353 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395354 7f5c835ef700 20 osd.4 194495 _dispatch 0x4df0380 pg_notify(22.3af(1) epoch 194495) v5 2014-08-13 18:06:07.395349 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.331 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.362 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395363 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,61,349 -> 4,61,331,349 2014-08-13 18:06:07.395374 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395378 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.61 2014-08-13 18:06:07.395368 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.362 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395380 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4df0380 2014-08-13 18:06:07.395376 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.395382 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343 -> 4,58,343,362 2014-08-13 18:06:07.395394 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395388 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [331,58,330](331) acting [331,58,330](331) maybe_went_rw) 2014-08-13 18:06:07.395396 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.395405 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395407 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395401 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [349,61](349) acting [349,61](349) maybe_went_rw) 2014-08-13 18:06:07.395407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [61,349,362](61) acting [61,349,362](61) maybe_went_rw) 2014-08-13 18:06:07.395421 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395412 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,330,331,349 down blocked_by {} 2014-08-13 18:06:07.395422 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395424 7f5c835ef700 20 osd.4 194495 _dispatch 0x5356040 pg_notify(22.1b8(2) epoch 194495) v5 2014-08-13 18:06:07.395420 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [58,343](58) acting [58,343](58) maybe_went_rw) 2014-08-13 18:06:07.395424 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.395435 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.58 2014-08-13 18:06:07.395446 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.340 2014-08-13 18:06:07.395448 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5356040 2014-08-13 18:06:07.395444 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.61 2014-08-13 18:06:07.395453 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.330 2014-08-13 18:06:07.395470 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395462 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: have osd.331 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395472 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395473 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.349 2014-08-13 18:06:07.395431 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,58,61,343,349,362 down blocked_by {} 2014-08-13 18:06:07.395485 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395487 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395489 7f5c835ef700 20 osd.4 194495 _dispatch 0x4de9dc0 pg_notify(22.3d5(2) epoch 194495) v5 2014-08-13 18:06:07.395488 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.395500 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.58 2014-08-13 18:06:07.395510 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.343 2014-08-13 18:06:07.395512 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4de9dc0 2014-08-13 18:06:07.395516 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395509 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.61 2014-08-13 18:06:07.395519 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.343 2014-08-13 18:06:07.395528 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.349 2014-08-13 18:06:07.395538 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395540 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395533 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194486/194483 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 320 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395537 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: have osd.362 info 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395553 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395555 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395548 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395557 7f5c835ef700 20 osd.4 194495 _dispatch 0x5352d80 pg_notify(22.48e epoch 194495) v5 2014-08-13 18:06:07.395569 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395576 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.341 2014-08-13 18:06:07.395578 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5352d80 2014-08-13 18:06:07.395594 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395599 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395601 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395590 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 304 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.395600 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194486/194486 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 334 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395612 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395614 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395605 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.395615 7f5c835ef700 20 osd.4 194495 _dispatch 0x53516c0 pg_notify(22.3d5(28) epoch 194495) v5 2014-08-13 18:06:07.395612 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395622 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,28,304 unchanged 2014-08-13 18:06:07.395638 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.286 2014-08-13 18:06:07.395632 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.395640 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x53516c0 2014-08-13 18:06:07.395647 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395643 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 18:06:07.395651 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 367 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395655 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 18:06:07.395666 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395669 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395664 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395666 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,7,28,304 down blocked_by {} 2014-08-13 18:06:07.395677 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.395687 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395689 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395690 7f5c835ef700 20 osd.4 194495 _dispatch 0x5356e40 pg_notify(22.1b8(28) epoch 194495) v5 2014-08-13 18:06:07.395687 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.7 2014-08-13 18:06:07.395697 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.28 2014-08-13 18:06:07.395708 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395712 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.347 2014-08-13 18:06:07.395714 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5356e40 2014-08-13 18:06:07.395706 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: have osd.304 info 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.395713 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 65 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.395736 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395727 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395740 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395742 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395743 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] osd.65 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395755 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395757 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395758 7f5c835ef700 20 osd.4 194495 _dispatch 0x53563c0 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.395749 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 363 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.395756 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,356 -> 4,50,55,61,65,356 2014-08-13 18:06:07.395769 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395772 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395764 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.363 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395782 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.338 2014-08-13 18:06:07.395784 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x53563c0 2014-08-13 18:06:07.395776 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 373 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.395780 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] osd.363 has stray content: 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.395788 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395794 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,374 -> 4,20,31,337,363,374 2014-08-13 18:06:07.395805 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395805 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395808 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395811 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395803 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.373 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.395815 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 286 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.395816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,362 -> 4,20,25,362,373 2014-08-13 18:06:07.395827 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.395828 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395830 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395830 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395832 7f5c835ef700 20 osd.4 194495 _dispatch 0x4bf16c0 pg_notify(22.3d9(47) epoch 194495) v5 2014-08-13 18:06:07.395826 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.395835 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 294 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395842 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 unchanged 2014-08-13 18:06:07.395856 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395856 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.373 2014-08-13 18:06:07.395849 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395858 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4bf16c0 2014-08-13 18:06:07.395860 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194487/194484 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 374 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395878 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395880 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395873 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] sending info 2014-08-13 18:06:07.395883 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.395888 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.395891 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:07.395893 7f5c7c7e4700 7 osd.4 194495 do_notify osd 30 on 1 PGs 2014-08-13 18:06:07.395901 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395903 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395904 7f5c835ef700 20 osd.4 194495 _dispatch 0x5356740 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.395910 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4857bc0 don't know epoch, doing nothing 2014-08-13 18:06:07.395912 7f5c7c7e4700 7 osd.4 194495 do_notify osd 31 on 1 PGs 2014-08-13 18:06:07.395925 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:07.395925 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.396 2014-08-13 18:06:07.395927 7f5c7c7e4700 7 osd.4 194495 do_notify osd 285 on 1 PGs 2014-08-13 18:06:07.395927 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5356740 2014-08-13 18:06:07.395939 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab32e0 don't know epoch, doing nothing 2014-08-13 18:06:07.395941 7f5c7c7e4700 7 osd.4 194495 do_notify osd 292 on 1 PGs 2014-08-13 18:06:07.395943 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.395947 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395949 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395954 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2d6a0 don't know epoch, doing nothing 2014-08-13 18:06:07.395956 7f5c7c7e4700 7 osd.4 194495 do_notify osd 294 on 1 PGs 2014-08-13 18:06:07.395949 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194486/194486 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 51 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.395962 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.395974 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:07.395976 7f5c7c7e4700 7 osd.4 194495 do_notify osd 320 on 1 PGs 2014-08-13 18:06:07.395977 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.395979 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.395980 7f5c835ef700 20 osd.4 194495 _dispatch 0x5354d00 pg_notify(22.3d9(50) epoch 194495) v5 2014-08-13 18:06:07.395987 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x49a2520 don't know epoch, doing nothing 2014-08-13 18:06:07.395989 7f5c7c7e4700 7 osd.4 194495 do_notify osd 323 on 1 PGs 2014-08-13 18:06:07.395991 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396001 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12100 don't know epoch, doing nothing 2014-08-13 18:06:07.396003 7f5c7c7e4700 7 osd.4 194495 do_notify osd 334 on 1 PGs 2014-08-13 18:06:07.396004 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.376 2014-08-13 18:06:07.395996 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 347 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.396006 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x5354d00 2014-08-13 18:06:07.396016 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.396018 7f5c7c7e4700 7 osd.4 194495 do_notify osd 367 on 1 PGs 2014-08-13 18:06:07.396009 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.396031 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4851e40 don't know epoch, doing nothing 2014-08-13 18:06:07.396032 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.396033 7f5c7c7e4700 7 osd.4 194495 do_notify osd 371 on 1 PGs 2014-08-13 18:06:07.396034 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.396034 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 unchanged 2014-08-13 18:06:07.396045 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396049 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 24 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.396058 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.396060 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.396062 7f5c835ef700 20 osd.4 194495 _dispatch 0x4de9f80 pg_notify(24.1bb epoch 194495) v5 2014-08-13 18:06:07.396061 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.24 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396064 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 379 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.396083 7f5c835ef700 7 osd.4 194495 handle_pg_notify from osd.348 2014-08-13 18:06:07.396085 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x4de9f80 2014-08-13 18:06:07.396075 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.24 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396080 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396088 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,55,339 -> 4,24,45,55,339 2014-08-13 18:06:07.396099 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.396102 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396097 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396106 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 6 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.396110 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,348 -> 4,331,332,348,379 2014-08-13 18:06:07.396120 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.396123 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.396118 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] sending info 2014-08-13 18:06:07.396132 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.396144 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194483/194483 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.396158 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,324 unchanged 2014-08-13 18:06:07.396179 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396174 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.396184 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [302,324,368](302) acting [302,324,368](302) maybe_went_rw) 2014-08-13 18:06:07.396184 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.396196 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [301,324](301) acting [301,324](301) maybe_went_rw) 2014-08-13 18:06:07.396198 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.303 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396210 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.396206 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,301,302,324,368 down blocked_by {} 2014-08-13 18:06:07.396212 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.396213 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396217 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.396226 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,310 -> 4,15,21,303,310 2014-08-13 18:06:07.396228 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.301 2014-08-13 18:06:07.396236 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.396239 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396237 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.302 2014-08-13 18:06:07.396243 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194486/194484 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 351 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.396246 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: have osd.324 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.396254 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.396257 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.368 2014-08-13 18:06:07.396284 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.396298 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.396303 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 338 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.396315 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194483/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.396300 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 0 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.396329 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376 unchanged 2014-08-13 18:06:07.396333 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194484/194484 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.0 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.396339 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.396349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [338,373,396](338) acting [338,373,396](338) maybe_went_rw) 2014-08-13 18:06:07.396349 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.0 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404549 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,56,353 -> 0,4,56,353 2014-08-13 18:06:07.404571 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.396362 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [338,373,396](338) acting [338,376,293](338)) 2014-08-13 18:06:07.404573 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.404586 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [369,63,0](369) acting [369,63,0](369) maybe_went_rw) 2014-08-13 18:06:07.404577 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,338,373,376,396 down blocked_by {} 2014-08-13 18:06:07.404598 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [369,63,0](369) acting [353,56,63](353)) 2014-08-13 18:06:07.404602 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.404608 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,4,56,63,353,369 down blocked_by {} 2014-08-13 18:06:07.404616 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: dropping osd.293 from info_requested, no longer in probe set 2014-08-13 18:06:07.404620 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.404628 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: have osd.338 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.404632 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: have osd.0 info 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404640 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.373 2014-08-13 18:06:07.404644 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.56 2014-08-13 18:06:07.404649 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.376 2014-08-13 18:06:07.404653 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.63 2014-08-13 18:06:07.404659 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.396 2014-08-13 18:06:07.404662 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.353 2014-08-13 18:06:07.404671 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 18:06:07.404693 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.404704 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.404715 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.404720 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4af98c0 don't know epoch, doing nothing 2014-08-13 18:06:07.404722 7f5c7bde3700 7 osd.4 194495 do_notify osd 6 on 1 PGs 2014-08-13 18:06:07.404722 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 11 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.404740 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c53f40 don't know epoch, doing nothing 2014-08-13 18:06:07.404743 7f5c7bde3700 7 osd.4 194495 do_notify osd 19 on 1 PGs 2014-08-13 18:06:07.404738 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194483/194483 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.11 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404756 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf1600 already has epoch 194495 2014-08-13 18:06:07.404758 7f5c7bde3700 7 osd.4 194495 do_notify osd 51 on 1 PGs 2014-08-13 18:06:07.404756 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.11 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404775 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4af8160 don't know epoch, doing nothing 2014-08-13 18:06:07.404768 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,47 -> 4,5,11,47 2014-08-13 18:06:07.404777 7f5c7bde3700 7 osd.4 194495 do_notify osd 351 on 1 PGs 2014-08-13 18:06:07.404778 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.404780 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.404788 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3c514a0 don't know epoch, doing nothing 2014-08-13 18:06:07.404790 7f5c7bde3700 7 osd.4 194495 do_notify osd 358 on 1 PGs 2014-08-13 18:06:07.404790 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [11,54,368](11) acting [11,54,368](11) maybe_went_rw) 2014-08-13 18:06:07.404806 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [11,54,368](11) acting [5,47,11](5)) 2014-08-13 18:06:07.404820 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:07.404823 7f5c7bde3700 7 osd.4 194495 do_notify osd 374 on 1 PGs 2014-08-13 18:06:07.404816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,5,11,47,54,368 down blocked_by {} 2014-08-13 18:06:07.404826 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.404837 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.5 2014-08-13 18:06:07.404846 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: have osd.11 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404856 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.47 2014-08-13 18:06:07.404865 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.54 2014-08-13 18:06:07.404860 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 397 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:07.404873 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.368 2014-08-13 18:06:07.404876 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.397 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.404898 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.404899 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.397 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.404913 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 352 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.404926 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376 -> 4,338,376,397 2014-08-13 18:06:07.404937 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.404928 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194484/194484 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.352 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404940 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.404943 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.352 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.404955 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,356,357 -> 4,352,356,357 2014-08-13 18:06:07.404964 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.404966 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.404975 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [352,371,380](352) acting [352,371,380](352) maybe_went_rw) 2014-08-13 18:06:07.404945 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 396 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.404986 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [352,371,380](352) acting [356,357,371](356)) 2014-08-13 18:06:07.404997 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,352,356,357,371,380 down blocked_by {} 2014-08-13 18:06:07.404993 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.396 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.405007 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.405017 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: have osd.352 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405028 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.356 2014-08-13 18:06:07.405018 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.396 has stray content: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.405037 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.357 2014-08-13 18:06:07.405046 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.371 2014-08-13 18:06:07.405042 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376,397 -> 4,338,376,396,397 2014-08-13 18:06:07.405057 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405060 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405054 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.380 2014-08-13 18:06:07.405065 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:07.405081 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405077 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.405093 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376,396,397 unchanged 2014-08-13 18:06:07.405092 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 339 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405105 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405106 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.339 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405108 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 327 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405122 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405121 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.339 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405137 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,362 -> 4,44,61,339,362 2014-08-13 18:06:07.405148 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405151 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405137 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.327 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405158 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,28,315,330 -> 4,28,315,327,330 2014-08-13 18:06:07.405169 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405155 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 370 notify: (query_epoch:194495, epoch_sent:194495, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.405173 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405174 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.370 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405177 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 7 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.405188 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.370 has stray content: 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405200 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340 -> 4,340,370 2014-08-13 18:06:07.405208 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405189 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.7 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405210 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.405219 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.405220 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] osd.7 has stray content: 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405230 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.405233 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,28,304 -> 4,7,28,304 2014-08-13 18:06:07.405243 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405246 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405240 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 18:06:07.405251 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.405262 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.324 2014-08-13 18:06:07.405250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 293 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:07.405270 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.340 2014-08-13 18:06:07.405275 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.293 22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.405279 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: have osd.370 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405290 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.293 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.405307 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405303 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,338,376,396,397 -> 4,293,338,376,396,397 2014-08-13 18:06:07.405313 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405317 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405322 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 309 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:07.405332 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.342 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405336 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.405346 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405352 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309 unchanged 2014-08-13 18:06:07.405364 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405368 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 311 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.405359 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339 -> 4,24,45,55,339,342 2014-08-13 18:06:07.405382 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405385 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405380 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194483/194483 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.311 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405389 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194487/194484 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 330 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.405394 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] osd.311 has stray content: 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405403 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] sending info 2014-08-13 18:06:07.405407 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,312 -> 4,285,311,312 2014-08-13 18:06:07.405416 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405418 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.405427 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [312,311,369](312) acting [312,311,369](312) maybe_went_rw) 2014-08-13 18:06:07.405438 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [312,285](312) acting [312,285](312) maybe_went_rw) 2014-08-13 18:06:07.405448 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,285,311,312,369 down blocked_by {} 2014-08-13 18:06:07.405462 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405458 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.405468 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.285 2014-08-13 18:06:07.405468 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 366 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405477 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: have osd.311 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405482 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194484/194484 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.366 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405489 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.312 2014-08-13 18:06:07.405497 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.369 2014-08-13 18:06:07.405497 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.366 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405511 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,373 -> 4,344,366,373 2014-08-13 18:06:07.405524 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405524 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405527 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.405539 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [366,320,363](366) acting [366,320,363](366) maybe_went_rw) 2014-08-13 18:06:07.405538 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 314 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405552 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [366,320,363](366) acting [373,344,320](373)) 2014-08-13 18:06:07.405553 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.314 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405569 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.314 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405564 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,320,344,363,366,373 down blocked_by {} 2014-08-13 18:06:07.405582 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,62,65 -> 4,44,62,65,314 2014-08-13 18:06:07.405591 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405595 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405589 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.405598 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 312 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.405605 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.320 2014-08-13 18:06:07.405609 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405615 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.344 2014-08-13 18:06:07.405624 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.363 2014-08-13 18:06:07.405625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,311,312 unchanged 2014-08-13 18:06:07.405636 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405634 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: have osd.366 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405639 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 48 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405648 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.373 2014-08-13 18:06:07.405651 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.48 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405664 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] osd.48 has stray content: 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405680 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405677 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,62,65,314 -> 4,44,48,62,65,314 2014-08-13 18:06:07.405687 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405690 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405693 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 380 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405697 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 349 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.405704 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.380 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405711 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.349 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405718 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.380 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405730 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357 -> 4,352,356,357,380 2014-08-13 18:06:07.405739 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405741 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405727 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.349 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 368 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.405755 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343,362 -> 4,58,343,349,362 2014-08-13 18:06:07.405755 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.368 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405766 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405769 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] osd.368 has stray content: 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.405781 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405781 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,324 -> 4,301,324,368 2014-08-13 18:06:07.405791 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405794 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405786 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 382 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:07.405798 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 350 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.405799 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.382 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.405811 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405815 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.382 has stray content: 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.405826 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.350 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.405829 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376,396,397 -> 4,293,338,376,382,396,397 2014-08-13 18:06:07.405840 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405842 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.405839 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,367,378 -> 4,291,350,367,378 2014-08-13 18:06:07.405850 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405852 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.405846 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 28 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.405856 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 381 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.405857 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405867 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.381 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.405872 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,7,28,304 unchanged 2014-08-13 18:06:07.405881 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.381 has stray content: 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.405885 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.073194 4 0.000384 2014-08-13 18:06:07.405893 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309 -> 4,287,309,381 2014-08-13 18:06:07.405902 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.405903 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.405905 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.405913 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [303,306,381](303) acting [303,306,381](303) maybe_went_rw) 2014-08-13 18:06:07.405922 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405925 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [287,309](287) acting [287,309](287) maybe_went_rw) 2014-08-13 18:06:07.405935 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,287,303,306,309,381 down blocked_by {} 2014-08-13 18:06:07.405934 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.7 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405946 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.405946 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405957 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.405958 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405968 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.303 2014-08-13 18:06:07.405977 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.306 2014-08-13 18:06:07.405986 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.309 info 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.405996 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) osd.28 (up) accepted 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) osd.304 (up) accepted 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.405998 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: have osd.381 info 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.406009 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,28,304 2014-08-13 18:06:07.406026 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406019 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,28,304] (== acting) backfill_targets 2014-08-13 18:06:07.406032 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.406040 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406045 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000140 0 0.000000 2014-08-13 18:06:07.406052 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.369 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406060 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.406067 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406070 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: osd.28 has no missing, identical log 2014-08-13 18:06:07.406080 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362 -> 4,44,61,339,362,369 2014-08-13 18:06:07.406090 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406093 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406081 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: osd.304 has no missing, identical log 2014-08-13 18:06:07.406097 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 375 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406107 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.406111 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194484/194484 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406119 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000059 0 0.000000 2014-08-13 18:06:07.406130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,286,375 unchanged 2014-08-13 18:06:07.406132 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.406144 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406140 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.406150 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194492-194494 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 18:06:07.406166 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 18:06:07.406166 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194486/194486 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 58 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406178 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 18:06:07.406182 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.406189 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.406201 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.10 2014-08-13 18:06:07.406210 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.286 2014-08-13 18:06:07.406218 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: already requested info from osd.301 2014-08-13 18:06:07.406227 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: have osd.375 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406239 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406245 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194486/194486 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 9 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406257 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406257 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.406276 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 302 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.406292 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406289 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.302 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.406296 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194486/194486 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 345 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406304 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] osd.302 has stray content: 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.406308 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.406317 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,324,368 -> 4,301,302,324,368 2014-08-13 18:06:07.406327 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406330 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406334 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194487/194484 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 346 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406329 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.406347 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] sending info 2014-08-13 18:06:07.406359 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406365 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194486/194486 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 54 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406378 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.406393 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406398 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.406401 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4f21600 don't know epoch, doing nothing 2014-08-13 18:06:07.406403 7f5c7c7e4700 7 osd.4 194495 do_notify osd 346 on 1 PGs 2014-08-13 18:06:07.406406 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406411 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.406423 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.61 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.406437 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] osd.61 has stray content: 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.406441 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 365 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406450 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,343,349,362 -> 4,58,61,343,349,362 2014-08-13 18:06:07.406461 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406463 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406457 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406467 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194487/194480 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 289 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406473 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] osd.365 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406480 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] sending info 2014-08-13 18:06:07.406486 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,350,367,378 -> 4,291,350,365,367,378 2014-08-13 18:06:07.406497 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406499 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406503 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 306 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.406514 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.306 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.406527 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.306 has stray content: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.406538 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406539 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,309,381 -> 4,287,306,309,381 2014-08-13 18:06:07.406549 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406552 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406543 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194487/194480 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 45 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406556 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] sending info 2014-08-13 18:06:07.406556 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 63 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406568 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.63 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406580 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.63 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406592 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,353 -> 0,4,56,63,353 2014-08-13 18:06:07.406601 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406602 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406604 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406630 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 10 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406629 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 17 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406642 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406645 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406656 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] osd.10 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406661 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] osd.17 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406669 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,286,375 -> 4,10,286,375 2014-08-13 18:06:07.406680 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406682 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406675 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,28,315,327,330 -> 4,17,28,315,327,330 2014-08-13 18:06:07.406686 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406690 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406685 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 301 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406698 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.406701 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4423700 don't know epoch, doing nothing 2014-08-13 18:06:07.406703 7f5c7bde3700 7 osd.4 194495 do_notify osd 9 on 1 PGs 2014-08-13 18:06:07.406697 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406719 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:07.406721 7f5c7bde3700 7 osd.4 194495 do_notify osd 45 on 1 PGs 2014-08-13 18:06:07.406712 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] osd.301 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406731 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4425280 don't know epoch, doing nothing 2014-08-13 18:06:07.406724 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,10,286,375 -> 4,10,286,301,375 2014-08-13 18:06:07.406733 7f5c7bde3700 7 osd.4 194495 do_notify osd 54 on 1 PGs 2014-08-13 18:06:07.406734 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406737 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406744 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:07.406745 7f5c7bde3700 7 osd.4 194495 do_notify osd 58 on 1 PGs 2014-08-13 18:06:07.406741 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194487/194480 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 356 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406757 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:07.406758 7f5c7bde3700 7 osd.4 194495 do_notify osd 289 on 1 PGs 2014-08-13 18:06:07.406753 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] sending info 2014-08-13 18:06:07.406770 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:07.406772 7f5c7bde3700 7 osd.4 194495 do_notify osd 330 on 1 PGs 2014-08-13 18:06:07.406786 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:07.406787 7f5c7bde3700 7 osd.4 194495 do_notify osd 345 on 1 PGs 2014-08-13 18:06:07.406806 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406811 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194487/194484 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 356 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406817 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 362 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.406823 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] sending info 2014-08-13 18:06:07.406841 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.362 info 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.406859 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406868 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 363 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406879 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.406884 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 55 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.406887 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.363 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406899 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194484/194484 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406914 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] osd.55 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406903 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.363 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.406927 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,59,351 -> 4,55,59,351 2014-08-13 18:06:07.406937 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406932 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,366,373 -> 4,344,363,366,373 2014-08-13 18:06:07.406945 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.406948 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.406939 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.406952 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194492-194494 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 18:06:07.406955 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 397 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.406964 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 18:06:07.406968 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.406975 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 18:06:07.406986 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.406996 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: have osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407011 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407008 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.59 2014-08-13 18:06:07.407016 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.351 2014-08-13 18:06:07.407016 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194486/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 396 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.407026 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: already requested info from osd.376 2014-08-13 18:06:07.407032 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] sending info 2014-08-13 18:06:07.407052 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407055 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407057 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.407068 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.324 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407070 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 50 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.407082 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.324 has stray content: 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407085 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407094 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,370 -> 4,324,340,370 2014-08-13 18:06:07.407103 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407107 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407101 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,32,50,289 unchanged 2014-08-13 18:06:07.407113 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.053480 4 0.000357 2014-08-13 18:06:07.407126 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.407126 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.407138 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194486/194486 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.324 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407149 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.32 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407162 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.324 has stray content: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407160 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407174 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340 -> 4,324,340 2014-08-13 18:06:07.407184 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407177 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407185 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.407197 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:07.407208 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:07.407219 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,324,340,370 down blocked_by {} 2014-08-13 18:06:07.407229 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.407205 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) osd.50 (up) accepted 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) osd.289 (up) accepted 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407239 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: have osd.324 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407248 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,50,289 2014-08-13 18:06:07.407250 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.340 2014-08-13 18:06:07.407258 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,50,289] (== acting) backfill_targets 2014-08-13 18:06:07.407260 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.370 2014-08-13 18:06:07.407270 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.407289 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407283 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000158 0 0.000000 2014-08-13 18:06:07.407294 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.407304 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: osd.50 has no missing, identical log 2014-08-13 18:06:07.407304 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 327 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407314 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: osd.289 has no missing, identical log 2014-08-13 18:06:07.407318 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194484/194484 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.327 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407323 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.407334 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] osd.327 has stray content: 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407335 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000041 0 0.000000 2014-08-13 18:06:07.407349 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.407359 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407346 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,341 -> 4,289,327,341 2014-08-13 18:06:07.407376 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407374 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 337 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407378 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:07.407389 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194492-194494 up [289,327,376](289) acting [289,327,376](289) maybe_went_rw) 2014-08-13 18:06:07.407388 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407407 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] osd.337 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407403 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(194491-194491 up [289,327,376](289) acting [289,341,327](289)) 2014-08-13 18:06:07.407421 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,348,379 -> 4,331,332,337,348,379 2014-08-13 18:06:07.407432 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407435 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407427 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 4,289,327,341,376 down blocked_by {} 2014-08-13 18:06:07.407438 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] up_thru 194486 < same_since 194495, must notify monitor 2014-08-13 18:06:07.407439 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 371 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407449 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.289 2014-08-13 18:06:07.407450 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.371 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407458 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: have osd.327 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407465 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] osd.371 has stray content: 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407457 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.407471 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.341 2014-08-13 18:06:07.407478 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357,380 -> 4,352,356,357,371,380 2014-08-13 18:06:07.407481 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: already requested info from osd.376 2014-08-13 18:06:07.407489 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407493 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407497 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194487/194480 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 343 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.407508 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407509 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] sending info 2014-08-13 18:06:07.407523 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 370 notify: (query_epoch:194495, epoch_sent:194495, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.407536 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.370 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407547 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407552 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194487/194484 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 15 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.407564 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] sending info 2014-08-13 18:06:07.407551 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] osd.370 has stray content: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.407576 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,324,340 -> 4,324,340,370 2014-08-13 18:06:07.407587 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407589 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407593 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 7 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.407609 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407604 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got osd.7 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407614 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194487/194480 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 304 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.407618 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] osd.7 has stray content: 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407628 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] sending info 2014-08-13 18:06:07.407631 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,340,347 -> 4,7,340,347 2014-08-13 18:06:07.407641 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407644 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.043722 4 0.000345 2014-08-13 18:06:07.407656 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.407667 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407668 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407672 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 286 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.407680 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.7 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407684 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] got osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.407691 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407699 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 18:06:07.407704 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407712 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetInfo 0.188381 5 0.023323 2014-08-13 18:06:07.407725 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.407731 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) osd.340 (up) accepted 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) osd.347 (up) accepted 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.407738 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.4 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407748 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340,347 2014-08-13 18:06:07.407755 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407758 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340,347] (== acting) backfill_targets 2014-08-13 18:06:07.407766 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.407770 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.407777 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407783 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000126 0 0.000000 2014-08-13 18:06:07.407789 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407796 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.407806 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: osd.340 has no missing, identical log 2014-08-13 18:06:07.407816 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: osd.347 has no missing, identical log 2014-08-13 18:06:07.407820 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] calc_acting newest update on osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) shard 286 (up) backfill 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.375 (up) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) shard 10 (stray) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407826 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.407836 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] choose_acting want [4,375,10] != acting [4,286,375], requesting pg_temp change 2014-08-13 18:06:07.407838 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000043 0 0.000000 2014-08-13 18:06:07.407854 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering/GetLog 0.000129 0 0.000000 2014-08-13 18:06:07.407866 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] state: Leaving Peering 2014-08-13 18:06:07.407850 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.407883 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407875 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 peering m=56] exit Started/Primary/Peering 0.188564 0 0.000000 2014-08-13 18:06:07.407887 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.188597 0 0.000000 2014-08-13 18:06:07.407898 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary 2014-08-13 18:06:07.407899 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407909 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.407920 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407913 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.342 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407924 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407936 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407928 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407951 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] osd.303 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407955 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,313,318 -> 4,45,313,318,342 2014-08-13 18:06:07.407967 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407970 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.407965 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,362,373 -> 4,20,25,303,362,373 2014-08-13 18:06:07.407976 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.407979 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.407970 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.407974 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 368 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.407983 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194486/194486 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 328 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.407987 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.368 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.407995 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.408001 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.368 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408013 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47 -> 4,5,11,47,368 2014-08-13 18:06:07.408023 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408025 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408029 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408029 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.408033 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 351 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408040 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.369 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408046 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408055 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408063 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 18:06:07.408074 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408067 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,311,312 -> 4,285,311,312,369 2014-08-13 18:06:07.408077 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408079 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408078 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194484/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 320 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.408091 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] sending info 2014-08-13 18:06:07.408083 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 330 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.408110 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.330 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408132 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408124 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.330 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408137 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194486/194486 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 373 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.408136 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,61,331,349 -> 4,61,330,331,349 2014-08-13 18:06:07.408152 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408156 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408149 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.408161 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 366 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408172 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.366 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408182 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408183 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408189 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.408193 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:07.408195 7f5c7c7e4700 7 osd.4 194495 do_notify osd 15 on 1 PGs 2014-08-13 18:06:07.408189 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 349 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.408200 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.349 info 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408212 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408212 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4ab69e0 don't know epoch, doing nothing 2014-08-13 18:06:07.408215 7f5c7c7e4700 7 osd.4 194495 do_notify osd 304 on 1 PGs 2014-08-13 18:06:07.408216 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 10 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.408226 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:07.408228 7f5c7c7e4700 7 osd.4 194495 do_notify osd 320 on 1 PGs 2014-08-13 18:06:07.408227 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got osd.10 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408239 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:07.408241 7f5c7c7e4700 7 osd.4 194495 do_notify osd 328 on 1 PGs 2014-08-13 18:06:07.408241 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] osd.10 has stray content: 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408252 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:07.408254 7f5c7c7e4700 7 osd.4 194495 do_notify osd 343 on 1 PGs 2014-08-13 18:06:07.408254 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,286,343 -> 4,10,286,343 2014-08-13 18:06:07.408264 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408266 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:07.408268 7f5c7c7e4700 7 osd.4 194495 do_notify osd 356 on 2 PGs 2014-08-13 18:06:07.408267 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.047027 4 0.000366 2014-08-13 18:06:07.408279 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.408291 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408302 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.10 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408313 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408323 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408368 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) osd.343 (up) accepted 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) osd.286 (up) accepted 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:07.408380 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,286,343 2014-08-13 18:06:07.408392 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,343,286] (== acting) backfill_targets 2014-08-13 18:06:07.408403 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.408420 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:07.408425 7f5c7c7e4700 7 osd.4 194495 do_notify osd 373 on 1 PGs 2014-08-13 18:06:07.408415 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000136 0 0.000000 2014-08-13 18:06:07.408440 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 18:06:07.408440 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.408450 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: osd.286 has no missing, identical log 2014-08-13 18:06:07.408463 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: osd.343 has no missing, identical log 2014-08-13 18:06:07.408472 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.408481 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408485 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000045 0 0.000000 2014-08-13 18:06:07.408497 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.408496 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408509 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408512 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408525 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 2014-08-13 18:06:07.408535 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408538 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408530 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 58 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.408542 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 0 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408544 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.58 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408554 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.0 info 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408566 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408570 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 352 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408559 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] osd.58 has stray content: 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.408580 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.352 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408593 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408588 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,61,330,331,349 -> 4,58,61,330,331,349 2014-08-13 18:06:07.408599 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408602 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408597 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 380 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408602 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.408607 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194486/194483 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 345 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.408609 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.380 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408622 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408623 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.408625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 375 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408637 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.408648 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408654 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408651 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 365 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495)) 2014-08-13 18:06:07.408659 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 54 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408662 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.365 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495) 2014-08-13 18:06:07.408671 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.54 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408676 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.365 has stray content: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495) 2014-08-13 18:06:07.408686 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] osd.54 has stray content: 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408689 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,306,309,381 -> 4,287,306,309,365,381 2014-08-13 18:06:07.408700 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408702 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408699 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,368 -> 4,5,11,47,54,368 2014-08-13 18:06:07.408709 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.408712 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408706 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 63 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408716 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.408717 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.63 info 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408730 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408727 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got dup osd.61 info 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408739 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408733 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 301 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408742 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.408746 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:07.408748 7f5c7bde3700 7 osd.4 194495 do_notify osd 345 on 1 PGs 2014-08-13 18:06:07.408745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.408755 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408763 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing 2014-08-13 18:06:07.408765 7f5c7bde3700 7 osd.4 194495 do_notify osd 396 on 1 PGs 2014-08-13 18:06:07.408759 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 356 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.408776 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b10b00 don't know epoch, doing nothing 2014-08-13 18:06:07.408778 7f5c7bde3700 7 osd.4 194495 do_notify osd 397 on 1 PGs 2014-08-13 18:06:07.408770 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.408791 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 18:06:07.408784 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357,371,380 unchanged 2014-08-13 18:06:07.408796 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408800 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194486/194486 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 337 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.408812 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.408827 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 289 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408840 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408851 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408856 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 371 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408855 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341 unchanged 2014-08-13 18:06:07.408872 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408867 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got dup osd.371 info 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408879 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.408876 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 343 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.408882 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 357 notify: (query_epoch:194495, epoch_sent:194495, info:22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.408893 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] got osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.408888 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.408907 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,352,356,357,371,380 unchanged 2014-08-13 18:06:07.408919 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.181855 9 0.000828 2014-08-13 18:06:07.408921 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,343,349,362 unchanged 2014-08-13 18:06:07.408933 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408931 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.408944 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408955 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.352 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408938 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 363 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.408965 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.408971 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.363 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.408975 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.408985 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.408985 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.371 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.408993 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 344 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.408996 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting osd.380 22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409004 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409020 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) osd.356 (up) accepted 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.357 (up) accepted 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409032 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,356,357 2014-08-13 18:06:07.409031 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,363,366,373 unchanged 2014-08-13 18:06:07.409044 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409042 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,356,357] (== acting) backfill_targets 2014-08-13 18:06:07.409048 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409053 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.409063 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409073 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409066 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000135 0 0.000000 2014-08-13 18:06:07.409077 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.409077 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409088 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.409092 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] requeue_ops 2014-08-13 18:06:07.409103 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409098 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.409106 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409110 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.409120 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409117 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409126 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409130 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409135 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409149 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409140 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.303 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409159 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409161 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409168 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409180 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409189 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409173 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,313,318,342 -> 4,45,303,313,318,342 2014-08-13 18:06:07.409199 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.409192 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409202 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409203 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409212 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409207 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 331 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409218 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.331 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409219 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409232 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409234 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409243 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409236 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 320 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409247 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409247 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.320 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409246 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.409259 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] requeue_ops 2014-08-13 18:06:07.409271 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409262 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] osd.320 has stray content: 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409275 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409276 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,344,363,366,373 -> 4,320,344,363,366,373 2014-08-13 18:06:07.409286 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.409290 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409286 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409295 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409294 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 367 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409299 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409306 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409311 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] requeue_ops 2014-08-13 18:06:07.409321 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:07.409324 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409333 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409334 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.409343 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409346 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.409350 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:07.409337 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 65 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409352 7f5c7c7e4700 7 osd.4 194495 do_notify osd 337 on 1 PGs 2014-08-13 18:06:07.409353 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409368 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 18:06:07.409368 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,48,62,65,314 unchanged 2014-08-13 18:06:07.409379 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409383 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 373 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.409393 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got osd.373 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409403 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194486/194486 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 328 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.409407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] osd.373 has stray content: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409415 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.409420 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,293,338,376,382,396,397 -> 4,293,338,373,376,382,396,397 2014-08-13 18:06:07.409430 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.409433 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.033408 8 0.008882 2014-08-13 18:06:07.409452 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409445 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.409457 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 56 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409469 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409484 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353 unchanged 2014-08-13 18:06:07.409479 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409496 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409502 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 368 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409498 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.293 22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.409513 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409513 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.368 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409527 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409524 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.373 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409531 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409535 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.409542 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.369 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409546 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.382 22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.409556 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] osd.369 has stray content: 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409557 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.396 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:07.409568 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353 -> 0,4,56,63,353,369 2014-08-13 18:06:07.409569 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting osd.397 22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495) 2014-08-13 18:06:07.409580 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.409582 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409586 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 302 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409596 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.302 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409610 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409613 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 353 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409597 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) osd.338 (up) accepted 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) osd.376 (up) accepted 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:07.409624 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409632 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,338,376 2014-08-13 18:06:07.409638 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,4,56,63,353,369 unchanged 2014-08-13 18:06:07.409642 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,338,376] (== acting) backfill_targets 2014-08-13 18:06:07.409650 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.143036 8 0.008970 2014-08-13 18:06:07.409654 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.409663 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.409667 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000221 0 0.000000 2014-08-13 18:06:07.409676 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409679 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.409689 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: osd.338 has no missing, identical log 2014-08-13 18:06:07.409687 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409698 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409699 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: requesting log+missing since 194490'0 from osd.376 2014-08-13 18:06:07.409712 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409708 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.63 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409718 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409728 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409741 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.376 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409727 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409760 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.4 because it is current primary 2014-08-13 18:06:07.409755 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] osd.376 has stray content: 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.409783 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) osd.353 (up) accepted 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.56 (up) accepted 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409783 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341 -> 4,289,327,341,376 2014-08-13 18:06:07.409795 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.409798 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409794 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,56,353 2014-08-13 18:06:07.409803 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,353,56] (== acting) backfill_targets 2014-08-13 18:06:07.409802 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409816 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.409815 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409826 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409828 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000166 0 0.000000 2014-08-13 18:06:07.409829 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.409840 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.409842 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] requeue_ops 2014-08-13 18:06:07.409854 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409850 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.409838 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.409857 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 339 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409861 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.409869 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409873 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.409883 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409884 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339,342 unchanged 2014-08-13 18:06:07.409900 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409898 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 10 notify: (query_epoch:194495, epoch_sent:194495, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409904 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 370 notify: (query_epoch:194495, epoch_sent:194495, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409912 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.409922 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.409915 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.370 info 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409926 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409926 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 301 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.409938 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.409952 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,301,302,324,368 unchanged 2014-08-13 18:06:07.409929 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 311 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.409964 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.197419 7 0.000674 2014-08-13 18:06:07.409963 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.409970 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.311 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.409975 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.409984 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.409988 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.409989 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410001 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410000 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.342 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.410011 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.302 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410015 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] osd.342 has stray content: 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.410022 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410030 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310 -> 4,15,21,303,310,342 2014-08-13 18:06:07.410041 7f5c7bde3700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.410033 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting osd.368 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410045 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410049 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.410053 7f5c7bde3700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.410055 7f5c7bde3700 7 osd.4 194495 do_queries querying osd.376 on 1 PGs 2014-08-13 18:06:07.410057 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) osd.301 (up) accepted 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.324 (up) accepted 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410073 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.30b=[4,375,10]} 2014-08-13 18:06:07.410068 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,301,324 2014-08-13 18:06:07.410077 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,301,324] (== acting) backfill_targets 2014-08-13 18:06:07.410088 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.410099 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000123 0 0.000000 2014-08-13 18:06:07.410109 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.410119 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: osd.324 has no missing, identical log 2014-08-13 18:06:07.410106 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 314 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.410128 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.410139 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000030 0 0.000000 2014-08-13 18:06:07.410149 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.410159 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.410136 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.314 info 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410166 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410175 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 312 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.410178 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 20 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.410189 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.312 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410201 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410192 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410205 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.410205 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 48 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.410209 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.410216 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.48 info 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410228 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410222 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] requeue_ops 2014-08-13 18:06:07.410233 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.410248 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 350 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.410262 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410275 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410280 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 330 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.410272 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 362 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.410290 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.330 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410302 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410294 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410306 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 365 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.410295 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.410314 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 18:06:07.410327 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.410316 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.410331 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410331 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.410334 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 349 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.410343 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got osd.303 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410345 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410358 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] osd.303 has stray content: 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410364 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,330,331,349 unchanged 2014-08-13 18:06:07.410375 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.410372 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,287,306,309,365,381 -> 4,287,303,306,309,365,381 2014-08-13 18:06:07.410382 7f5c7c7e4700 20 osd.4 194495 need_heartbeat_peer_update 2014-08-13 18:06:07.410381 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 58 notify: (query_epoch:194495, epoch_sent:194495, info:22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.410387 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.035479 7 0.000674 2014-08-13 18:06:07.410393 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] got osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410400 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.410407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,343,349,362 unchanged 2014-08-13 18:06:07.410414 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410420 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.211756 9 0.000693 2014-08-13 18:06:07.410426 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.410439 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.410437 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410448 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.306 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410451 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410459 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.410463 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410470 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.365 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495) 2014-08-13 18:06:07.410473 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410481 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting osd.381 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) 2014-08-13 18:06:07.410484 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410494 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.349 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410505 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting osd.362 22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.410530 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) osd.58 (up) accepted 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.343 (up) accepted 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.410510 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) osd.287 (up) accepted 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) osd.309 (up) accepted 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:07.410543 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,58,343 2014-08-13 18:06:07.410543 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,287,309 2014-08-13 18:06:07.410553 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,58,343] (== acting) backfill_targets 2014-08-13 18:06:07.410553 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,287,309] (== acting) backfill_targets 2014-08-13 18:06:07.410566 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.410566 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.410579 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000140 0 0.000000 2014-08-13 18:06:07.410579 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000179 0 0.000000 2014-08-13 18:06:07.410592 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.410592 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.410602 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.410603 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: osd.287 has no missing, identical log 2014-08-13 18:06:07.410613 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000022 0 0.000000 2014-08-13 18:06:07.410614 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: requesting log+missing since 194489'0 from osd.309 2014-08-13 18:06:07.410624 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.411770 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.411773 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.411774 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.411781 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.411783 7f5c835ef700 20 osd.4 194495 _dispatch 0x207e600 pg_log(22.3d9 epoch 194495 log log((194215'511,194215'511], crt=194215'511) query_epoch 194495) v4 2014-08-13 18:06:07.411793 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194486/194483 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MQuery from 376 query_epoch 194495 query: query(info 0'0) 2014-08-13 18:06:07.411810 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] sending info 2014-08-13 18:06:07.411806 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 291 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.411825 7f5c835ef700 7 osd.4 194495 handle_pg_log pg_log(22.3d9 epoch 194495 log log((194215'511,194215'511], crt=194215'511) query_epoch 194495) v4 from osd.376 2014-08-13 18:06:07.411830 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x207e600 2014-08-13 18:06:07.411829 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.411845 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.411855 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.411857 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.411859 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.411851 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 285 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.411861 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.411863 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.411875 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.324 2014-08-13 18:06:07.411883 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,285,311,312,369 unchanged 2014-08-13 18:06:07.411884 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.411904 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.411898 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.165386 7 0.000679 2014-08-13 18:06:07.411898 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.411908 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 25 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.411912 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.411925 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.411922 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.411937 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.411944 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 18:06:07.411944 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.411947 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.311 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.411958 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.411958 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetInfo 0.148985 7 0.000886 2014-08-13 18:06:07.411970 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.411974 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.411988 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.411998 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) osd.312 (up) accepted 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) osd.285 (up) accepted 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412001 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412009 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,285,312 2014-08-13 18:06:07.412013 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412019 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,312,285] (== acting) backfill_targets 2014-08-13 18:06:07.412024 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412031 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412035 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412044 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000132 0 0.000000 2014-08-13 18:06:07.412046 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412056 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412066 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: osd.312 has no missing, identical log 2014-08-13 18:06:07.412076 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412079 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412087 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000031 0 0.000000 2014-08-13 18:06:07.412090 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] choose_acting want [4,20,303] != acting [4,25,362], requesting pg_temp change 2014-08-13 18:06:07.412098 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412108 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412105 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering/GetLog 0.000131 0 0.000000 2014-08-13 18:06:07.412117 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] state: Leaving Peering 2014-08-13 18:06:07.412126 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 peering m=57] exit Started/Primary/Peering 0.149172 0 0.000000 2014-08-13 18:06:07.412125 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412138 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.149203 0 0.000000 2014-08-13 18:06:07.412139 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412156 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,58,61,330,331,349 unchanged 2014-08-13 18:06:07.412151 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary 2014-08-13 18:06:07.412173 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.412185 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412168 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.136384 8 0.000771 2014-08-13 18:06:07.412188 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.412190 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412195 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.412203 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.369 2014-08-13 18:06:07.412202 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412214 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.58 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412214 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.412234 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412226 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412236 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.330 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412238 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.412247 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.331 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412251 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.412262 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412257 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412266 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 340 notify: (query_epoch:194495, epoch_sent:194495, info:1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412278 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412281 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) osd.349 (up) accepted 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.61 (up) accepted 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412291 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,61,349 2014-08-13 18:06:07.412294 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,324,340,370 unchanged 2014-08-13 18:06:07.412300 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,349,61] (== acting) backfill_targets 2014-08-13 18:06:07.412305 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.098635 4 0.000570 2014-08-13 18:06:07.412313 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412317 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.412325 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000137 0 0.000000 2014-08-13 18:06:07.412329 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412336 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412341 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412346 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412352 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412357 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.412363 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.370 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412369 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412379 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412384 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) osd.340 (up) accepted 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412394 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340 2014-08-13 18:06:07.412395 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 340 notify: (query_epoch:194495, epoch_sent:194495, info:2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412402 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340] (== acting) backfill_targets 2014-08-13 18:06:07.412408 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412414 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412425 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,324,340,370 unchanged 2014-08-13 18:06:07.412426 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000109 0 0.000000 2014-08-13 18:06:07.412436 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.120426 6 0.000598 2014-08-13 18:06:07.412438 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412447 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.412447 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412460 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.412461 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412471 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412481 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412472 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.324 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412483 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412492 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.370 2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.412496 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412505 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.412509 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.412521 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412513 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) osd.340 (up) accepted 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412524 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,340 2014-08-13 18:06:07.412525 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 373 notify: (query_epoch:194495, epoch_sent:194495, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412533 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,340] (== acting) backfill_targets 2014-08-13 18:06:07.412536 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.412546 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412549 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.412543 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412553 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:07.412555 7f5c7c7e4700 7 osd.4 194495 do_notify osd 328 on 1 PGs 2014-08-13 18:06:07.412555 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000107 0 0.000000 2014-08-13 18:06:07.412571 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:07.412566 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412573 7f5c7c7e4700 7 osd.4 194495 do_notify osd 376 on 1 PGs 2014-08-13 18:06:07.412575 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412584 7f5c7c7e4700 20 osd.4 194495 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing 2014-08-13 18:06:07.412586 7f5c7c7e4700 7 osd.4 194495 do_queries querying osd.309 on 1 PGs 2014-08-13 18:06:07.412585 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000019 0 0.000000 2014-08-13 18:06:07.412598 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.30b=[4,375,10]} 2014-08-13 18:06:07.412596 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412605 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412619 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 45 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412631 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412633 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 318 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412646 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,24,45,55,339,342 unchanged 2014-08-13 18:06:07.412647 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412658 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.117144 6 0.000844 2014-08-13 18:06:07.412664 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,303,313,318,342 unchanged 2014-08-13 18:06:07.412676 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412670 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.412680 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 313 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.412682 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412664 7f5c87c27700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.412692 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412695 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.24 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412706 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412707 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,45,303,313,318,342 unchanged 2014-08-13 18:06:07.412717 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.55 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412719 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.103455 6 0.002369 2014-08-13 18:06:07.412719 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.412728 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412732 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.412739 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412745 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412762 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) osd.339 (up) accepted 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.45 (up) accepted 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412760 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.45 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412772 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,45,339 2014-08-13 18:06:07.412771 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412781 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,339,45] (== acting) backfill_targets 2014-08-13 18:06:07.412782 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412792 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412793 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412804 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.412805 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000134 0 0.000000 2014-08-13 18:06:07.412818 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412827 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412828 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) osd.313 (up) accepted 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.318 (up) accepted 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.412837 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 18:06:07.412839 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,313,318 2014-08-13 18:06:07.412848 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412848 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,313,318] (== acting) backfill_targets 2014-08-13 18:06:07.412859 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412861 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.412872 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000140 0 0.000000 2014-08-13 18:06:07.412872 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 44 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412884 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.412885 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got dup osd.44 info 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.412899 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412893 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.412904 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 18:06:07.412904 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 55 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412916 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.412917 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.55 2014-08-13 18:06:07.412927 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412927 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.55 info 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.412944 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412942 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 45 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412950 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 24 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.412955 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.45 2014-08-13 18:06:07.412961 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.24 2014-08-13 18:06:07.412965 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.45 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.412986 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.412971 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.24 info 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.412989 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.412990 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 324 notify: (query_epoch:194495, epoch_sent:194495, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.412996 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.412995 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 320 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413001 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.324 2014-08-13 18:06:07.413009 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got dup osd.320 info 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413021 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413013 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.413011 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.324 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413030 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413025 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 373 notify: (query_epoch:194495, epoch_sent:194495, info:22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413033 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 370 notify: (query_epoch:194495, epoch_sent:194495, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413036 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] got osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413043 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.370 2014-08-13 18:06:07.413055 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,320,344,363,366,373 unchanged 2014-08-13 18:06:07.413052 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.370 info 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413069 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413066 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.166948 9 0.000930 2014-08-13 18:06:07.413071 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413079 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.413082 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.342 2014-08-13 18:06:07.413092 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413091 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413109 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413103 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.320 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413113 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.413114 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413125 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.413125 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.363 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413136 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413136 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.366 22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413140 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413146 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413151 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.303 2014-08-13 18:06:07.413165 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413173 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) osd.373 (up) accepted 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.344 (up) accepted 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413183 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413184 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,344,373 2014-08-13 18:06:07.413186 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 7 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.413193 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,373,344] (== acting) backfill_targets 2014-08-13 18:06:07.413198 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.7 2014-08-13 18:06:07.413205 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.413207 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.7 info 22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413217 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000138 0 0.000000 2014-08-13 18:06:07.413227 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413229 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.413231 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 31 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413238 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.413243 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413249 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 18:06:07.413260 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.413258 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,363,374 unchanged 2014-08-13 18:06:07.413270 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413270 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413276 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 337 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413286 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.337 info 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413298 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413287 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413302 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.342 2014-08-13 18:06:07.413303 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 304 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.413314 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.304 2014-08-13 18:06:07.413313 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413331 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413323 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.304 info 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413340 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413335 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 378 notify: (query_epoch:194495, epoch_sent:194495, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413344 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 363 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413347 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] got osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413354 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.363 info 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413366 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413363 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:07.413370 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 374 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413376 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.191546 9 0.000856 2014-08-13 18:06:07.413381 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413389 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.413399 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,20,31,337,363,374 unchanged 2014-08-13 18:06:07.413401 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413410 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.087719 8 0.000561 2014-08-13 18:06:07.413414 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413423 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.413425 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413435 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.413437 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413430 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.413447 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.20 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.413449 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413458 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413459 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413469 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.337 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.413480 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.363 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.413490 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413491 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413502 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] choose_acting want [4,291,350] != acting [4,378,367], requesting pg_temp change 2014-08-13 18:06:07.413514 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) osd.374 (up) accepted 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.31 (up) accepted 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413516 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000128 0 0.000000 2014-08-13 18:06:07.413525 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,31,374 2014-08-13 18:06:07.413528 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] state: Leaving Peering 2014-08-13 18:06:07.413534 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,374,31] (== acting) backfill_targets 2014-08-13 18:06:07.413538 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.191729 0 0.000000 2014-08-13 18:06:07.413546 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.413551 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.191761 0 0.000000 2014-08-13 18:06:07.413558 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000135 0 0.000000 2014-08-13 18:06:07.413563 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 18:06:07.413569 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.413573 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.413583 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413579 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.413587 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.413591 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.30b=[4,375,10]} 2014-08-13 18:06:07.413590 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.413601 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.413610 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413625 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 28 notify: (query_epoch:194495, epoch_sent:194495, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.413637 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.28 2014-08-13 18:06:07.413638 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 58 notify: (query_epoch:194495, epoch_sent:194495, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413652 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.58 2014-08-13 18:06:07.413647 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] got dup osd.28 info 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413667 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413662 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] got dup osd.58 info 22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413670 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 20 notify: (query_epoch:194495, epoch_sent:194495, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413681 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413681 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.20 2014-08-13 18:06:07.413685 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 62 notify: (query_epoch:194495, epoch_sent:194495, info:22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413690 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] got dup osd.20 info 22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413697 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] got osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413708 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413711 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.413713 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,48,62,65,314 unchanged 2014-08-13 18:06:07.413724 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] requeue_ops 2014-08-13 18:06:07.413725 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.197646 9 0.006130 2014-08-13 18:06:07.413734 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413738 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.413738 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 55 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413751 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413751 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413765 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413747 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.413764 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.44 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413769 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 351 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413781 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413794 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413797 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.413776 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.48 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413807 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.413809 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413821 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.413824 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.413819 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413828 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.30b=[4,375,10]} 2014-08-13 18:06:07.413830 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting osd.314 22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413853 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) osd.65 (up) accepted 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.62 (up) accepted 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413862 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,62,65 2014-08-13 18:06:07.413865 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 59 notify: (query_epoch:194495, epoch_sent:194495, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.413871 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,65,62] (== acting) backfill_targets 2014-08-13 18:06:07.413879 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413883 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.413895 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000157 0 0.000000 2014-08-13 18:06:07.413896 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged 2014-08-13 18:06:07.413907 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.413908 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetInfo 0.079146 8 0.000533 2014-08-13 18:06:07.413916 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.413921 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.413927 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.413935 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413939 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.413953 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.413950 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413961 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.413968 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.413971 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413981 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.369 2014-08-13 18:06:07.413982 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.413990 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414009 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414012 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414012 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414023 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] choose_acting want [4,351,55] != acting [4,59,351], requesting pg_temp change 2014-08-13 18:06:07.414025 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414035 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414037 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering/GetLog 0.000116 0 0.000000 2014-08-13 18:06:07.414039 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414049 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] state: Leaving Peering 2014-08-13 18:06:07.414050 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414063 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414059 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 peering m=62] exit Started/Primary/Peering 0.079307 0 0.000000 2014-08-13 18:06:07.414071 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.414074 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.414066 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 15 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414076 7f5c835ef700 20 osd.4 194495 _dispatch 0x207ec00 pg_log(22.481 epoch 194495 log log((194215'519,194215'519], crt=194215'519) query_epoch 194495) v4 2014-08-13 18:06:07.414072 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.079330 0 0.000000 2014-08-13 18:06:07.414069 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.414078 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.15 info 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414092 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414086 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary 2014-08-13 18:06:07.414096 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 21 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414097 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.414108 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414114 7f5c835ef700 7 osd.4 194495 handle_pg_log pg_log(22.481 epoch 194495 log log((194215'519,194215'519], crt=194215'519) query_epoch 194495) v4 from osd.309 2014-08-13 18:06:07.414107 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414119 7f5c835ef700 15 osd.4 194495 require_same_or_newer_map 194495 (i am 194495) 0x207ec00 2014-08-13 18:06:07.414113 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414125 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414125 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310,342 unchanged 2014-08-13 18:06:07.414136 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414137 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414145 7f5c835ef700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:07.414147 7f5c835ef700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:07.414140 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414143 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 310 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414154 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414152 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] requeue_ops 2014-08-13 18:06:07.414167 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,15,21,303,310,342 unchanged 2014-08-13 18:06:07.414183 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414184 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.081176 8 0.000727 2014-08-13 18:06:07.414188 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414196 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.414200 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:07.414210 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414209 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414215 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 32 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.414220 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.15 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414227 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.32 2014-08-13 18:06:07.414231 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414241 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.303 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414236 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.32 info 22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414255 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414252 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414259 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414262 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting osd.342 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414271 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414285 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) osd.310 (up) accepted 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.21 (up) accepted 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414286 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:07.414298 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414295 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,21,310 2014-08-13 18:06:07.414304 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,310,21] (== acting) backfill_targets 2014-08-13 18:06:07.414314 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.414302 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 55 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414326 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000129 0 0.000000 2014-08-13 18:06:07.414330 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414337 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.414347 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.414346 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:07.414358 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000021 0 0.000000 2014-08-13 18:06:07.414359 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetInfo 0.077645 6 0.003500 2014-08-13 18:06:07.414369 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.414379 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414372 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.414386 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.414393 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 5 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414398 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.414406 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414409 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414420 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414421 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,54,368 unchanged 2014-08-13 18:06:07.414433 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414430 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.414436 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 342 notify: (query_epoch:194495, epoch_sent:194495, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414441 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.414448 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.342 2014-08-13 18:06:07.414458 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] got dup osd.342 info 22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414475 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414473 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:07.414478 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 368 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414484 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] choose_acting want [4,50,65] != acting [4,55,61], requesting pg_temp change 2014-08-13 18:06:07.414490 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.368 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414507 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414494 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.414500 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering/GetLog 0.000129 0 0.000000 2014-08-13 18:06:07.414510 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 11 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414512 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] state: Leaving Peering 2014-08-13 18:06:07.414522 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 peering m=51] exit Started/Primary/Peering 0.077839 0 0.000000 2014-08-13 18:06:07.414522 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.11 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414534 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414534 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.077862 0 0.000000 2014-08-13 18:06:07.414537 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 54 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414546 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary 2014-08-13 18:06:07.414548 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got dup osd.54 info 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414560 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414555 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.414566 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414563 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 47 notify: (query_epoch:194495, epoch_sent:194495, info:22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414570 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 50 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.414575 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] got osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414582 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.50 2014-08-13 18:06:07.414588 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,5,11,47,54,368 unchanged 2014-08-13 18:06:07.414591 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.50 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414609 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414600 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.074161 9 0.000631 2014-08-13 18:06:07.414613 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.414612 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 50 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.414624 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.414635 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414625 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414637 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414638 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 65 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.414646 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.11 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414650 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.414660 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414657 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414663 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 327 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.414667 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.54 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414676 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.414678 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting osd.368 22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414689 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414693 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 330 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414701 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) osd.5 (up) accepted 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.47 (up) accepted 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414704 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414711 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,5,47 2014-08-13 18:06:07.414720 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,5,47] (== acting) backfill_targets 2014-08-13 18:06:07.414720 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 18:06:07.414733 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414732 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.414736 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 28 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.414743 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000131 0 0.000000 2014-08-13 18:06:07.414747 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414755 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.414761 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 18:06:07.414765 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.414774 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetInfo 0.064517 7 0.000678 2014-08-13 18:06:07.414775 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 18:06:07.414786 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.414786 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.414798 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414800 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414812 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414813 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414823 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414826 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414836 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414834 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414840 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414845 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414851 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414860 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414857 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.414864 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414875 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.414885 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414887 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.414888 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414897 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] choose_acting want [4,17,315] != acting [4,330,28], requesting pg_temp change 2014-08-13 18:06:07.414899 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414910 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414911 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering/GetLog 0.000125 0 0.000000 2014-08-13 18:06:07.414914 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414923 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] state: Leaving Peering 2014-08-13 18:06:07.414926 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.414937 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414925 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.414933 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 peering m=59] exit Started/Primary/Peering 0.064687 0 0.000000 2014-08-13 18:06:07.414940 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.414951 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.414949 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.064713 0 0.000000 2014-08-13 18:06:07.414960 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.414963 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.414960 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary 2014-08-13 18:06:07.414967 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.414970 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.414980 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.414984 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 289 notify: (query_epoch:194495, epoch_sent:194495, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.414995 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.289 2014-08-13 18:06:07.415004 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] got dup osd.289 info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415021 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415015 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415025 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 17 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415030 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.415041 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415036 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.415046 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415045 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415050 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 315 notify: (query_epoch:194495, epoch_sent:194495, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415056 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415065 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415061 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.415072 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415069 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415075 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 356 notify: (query_epoch:194495, epoch_sent:194495, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:07.415080 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415089 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415088 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: In WaitActingChange, ignoring MNotifyRec 2014-08-13 18:06:07.415098 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415093 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415102 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415104 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] requeue_ops 2014-08-13 18:06:07.415114 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415114 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] requeue_ops 2014-08-13 18:06:07.415124 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415118 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415128 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415128 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415137 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415139 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415148 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415141 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415152 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] requeue_ops 2014-08-13 18:06:07.415162 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415152 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415170 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] requeue_ops 2014-08-13 18:06:07.415165 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415180 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415184 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.415181 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415190 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415188 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.415194 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415204 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] requeue_ops 2014-08-13 18:06:07.415213 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415216 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415227 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] requeue_ops 2014-08-13 18:06:07.415237 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415231 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 7 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415241 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415245 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.7 2014-08-13 18:06:07.415252 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] requeue_ops 2014-08-13 18:06:07.415262 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415255 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.7 info 22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415275 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415265 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 331 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.415278 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415279 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415292 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:07.415302 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415293 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:07.415305 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415306 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 293 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:07.415309 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 289 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415316 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.293 2014-08-13 18:06:07.415320 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.289 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415333 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415326 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.293 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415344 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415337 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 44 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.415348 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 10 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415349 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415358 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.10 2014-08-13 18:06:07.415363 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362,369 unchanged 2014-08-13 18:06:07.415374 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415377 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 337 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415388 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415402 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415367 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.10 info 22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415406 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415405 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 362 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.415410 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 340 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415415 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415421 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.340 2014-08-13 18:06:07.415429 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,44,61,339,362,369 unchanged 2014-08-13 18:06:07.415430 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.340 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415448 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415441 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.042121 6 0.000524 2014-08-13 18:06:07.415451 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 343 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415453 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.415461 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.343 2014-08-13 18:06:07.415467 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.415471 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.343 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415488 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415491 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 286 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415479 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415500 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.286 2014-08-13 18:06:07.415505 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.61 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.415517 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.339 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.415527 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415509 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] got dup osd.286 info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415542 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415536 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting osd.369 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.415545 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 347 notify: (query_epoch:194495, epoch_sent:194495, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:07.415555 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.347 2014-08-13 18:06:07.415560 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) osd.44 (up) accepted 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.362 (up) accepted 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.415570 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,44,362 2014-08-13 18:06:07.415564 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] got dup osd.347 info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415582 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415580 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,44,362] (== acting) backfill_targets 2014-08-13 18:06:07.415585 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 338 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.415592 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.415596 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.338 2014-08-13 18:06:07.415604 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000151 0 0.000000 2014-08-13 18:06:07.415606 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.338 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415622 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415615 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.415625 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.415626 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 373 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.415636 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.373 2014-08-13 18:06:07.415636 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000020 0 0.000000 2014-08-13 18:06:07.415648 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.415658 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415646 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.373 info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415662 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415666 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 396 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:07.415672 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 287 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:07.415675 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.396 2014-08-13 18:06:07.415685 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.287 2014-08-13 18:06:07.415686 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.396 info 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415705 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415696 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.287 info 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415708 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:07.415715 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415719 7f5c7c7e4700 7 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.376 2014-08-13 18:06:07.415722 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 332 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415728 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] got dup osd.376 info 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415745 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415735 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415747 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415748 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415752 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 303 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.415759 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415768 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415763 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.303 2014-08-13 18:06:07.415773 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415772 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.303 info 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415790 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415783 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415793 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415793 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 376 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415797 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415804 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.376 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415808 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415817 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415817 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415815 7f5c87c27700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.415827 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415833 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 379 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415842 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415852 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415846 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415860 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415855 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415863 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.415867 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.415866 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415875 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415879 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415889 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415898 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415901 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415911 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415920 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415914 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 327 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415924 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.415927 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got dup osd.327 info 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415940 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415934 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.415944 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.415947 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.415944 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 339 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.415950 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.415955 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.339 2014-08-13 18:06:07.415964 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.339 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.415982 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.415985 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 309 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:07.415988 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MLogRec from 376 2014-08-13 18:06:07.415996 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.309 2014-08-13 18:06:07.416000 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] proc_replica_log for osd.376: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) log((194215'511,194215'511], crt=194215'511) missing(28) 2014-08-13 18:06:07.416005 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.309 info 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495), identical to ours 2014-08-13 18:06:07.416023 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416018 7f5c7c7e4700 10 proc_replica_log for osd.376: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) log((194215'511,194215'511], crt=194215'511) missing(28) 2014-08-13 18:06:07.416026 7f5c7c7e4700 20 before missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 need 191727'373 have 0'0 2014-08-13 18:06:07.416029 7f5c7c7e4700 20 before missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 need 191648'48 have 0'0 2014-08-13 18:06:07.416031 7f5c7c7e4700 20 before missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 need 191727'415 have 0'0 2014-08-13 18:06:07.416033 7f5c7c7e4700 20 before missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 need 191648'126 have 0'0 2014-08-13 18:06:07.416035 7f5c7c7e4700 20 before missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 need 191727'441 have 0'0 2014-08-13 18:06:07.416037 7f5c7c7e4700 20 before missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 need 191727'370 have 0'0 2014-08-13 18:06:07.416028 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 381 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.416039 7f5c7c7e4700 20 before missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 need 191727'404 have 0'0 2014-08-13 18:06:07.416041 7f5c7c7e4700 20 before missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 need 191648'258 have 0'0 2014-08-13 18:06:07.416043 7f5c7c7e4700 20 before missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 need 191727'422 have 0'0 2014-08-13 18:06:07.416045 7f5c7c7e4700 20 before missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 need 191648'8 have 0'0 2014-08-13 18:06:07.416040 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.381 2014-08-13 18:06:07.416047 7f5c7c7e4700 20 before missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 need 191727'397 have 0'0 2014-08-13 18:06:07.416050 7f5c7c7e4700 20 before missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 need 191648'337 have 0'0 2014-08-13 18:06:07.416053 7f5c7c7e4700 20 before missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 need 191727'392 have 0'0 2014-08-13 18:06:07.416054 7f5c7c7e4700 20 before missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 need 191648'153 have 0'0 2014-08-13 18:06:07.416056 7f5c7c7e4700 20 before missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 need 191648'320 have 0'0 2014-08-13 18:06:07.416058 7f5c7c7e4700 20 before missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 need 191727'384 have 0'0 2014-08-13 18:06:07.416060 7f5c7c7e4700 20 before missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 need 191727'371 have 0'0 2014-08-13 18:06:07.416062 7f5c7c7e4700 20 before missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 need 191727'425 have 0'0 2014-08-13 18:06:07.416064 7f5c7c7e4700 20 before missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 need 191727'412 have 0'0 2014-08-13 18:06:07.416050 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.381 info 22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495), identical to ours 2014-08-13 18:06:07.416065 7f5c7c7e4700 20 before missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 need 191727'375 have 0'0 2014-08-13 18:06:07.416068 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416068 7f5c7c7e4700 20 before missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 need 191648'189 have 0'0 2014-08-13 18:06:07.416073 7f5c7c7e4700 20 before missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 need 191727'424 have 0'0 2014-08-13 18:06:07.416075 7f5c7c7e4700 20 before missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 need 191648'304 have 0'0 2014-08-13 18:06:07.416077 7f5c7c7e4700 20 before missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 need 191727'411 have 0'0 2014-08-13 18:06:07.416079 7f5c7c7e4700 20 before missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 need 191648'58 have 0'0 2014-08-13 18:06:07.416072 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 369 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.416081 7f5c7c7e4700 20 before missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 need 191727'405 have 0'0 2014-08-13 18:06:07.416083 7f5c7c7e4700 20 before missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 need 191648'198 have 0'0 2014-08-13 18:06:07.416085 7f5c7c7e4700 20 before missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 need 191727'390 have 0'0 2014-08-13 18:06:07.416082 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.369 2014-08-13 18:06:07.416088 7f5c7c7e4700 20 merge_log cut point (usually last shared) is 194215'511 (194215'496) delete 350933d9/benchmark_data_storage0111-ib.pocstor_64498_object30299/head//22 by client.1049635.0:64502 2014-08-12 17:19:28.143258 2014-08-13 18:06:07.416091 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.369 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.416109 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416101 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] peer osd.376 now 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) missing(28) 2014-08-13 18:06:07.416112 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 306 notify: (query_epoch:194495, epoch_sent:194495, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:07.416115 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 need 191727'373 have 0'0 2014-08-13 18:06:07.416123 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.306 2014-08-13 18:06:07.416125 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 need 191648'48 have 0'0 2014-08-13 18:06:07.416135 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 need 191727'415 have 0'0 2014-08-13 18:06:07.416132 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] got dup osd.306 info 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495), identical to ours 2014-08-13 18:06:07.416151 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416145 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 need 191648'126 have 0'0 2014-08-13 18:06:07.416154 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 61 notify: (query_epoch:194495, epoch_sent:194495, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:07.416160 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 need 191727'441 have 0'0 2014-08-13 18:06:07.416165 7f5c7bde3700 7 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.61 2014-08-13 18:06:07.416170 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 need 191727'370 have 0'0 2014-08-13 18:06:07.416180 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 need 191727'404 have 0'0 2014-08-13 18:06:07.416179 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] got dup osd.61 info 22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495), identical to ours 2014-08-13 18:06:07.416197 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416190 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 need 191648'258 have 0'0 2014-08-13 18:06:07.416199 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 need 191727'422 have 0'0 2014-08-13 18:06:07.416200 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 341 notify: (query_epoch:194495, epoch_sent:194495, info:22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.416209 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 need 191648'8 have 0'0 2014-08-13 18:06:07.416211 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] got osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416218 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 need 191727'397 have 0'0 2014-08-13 18:06:07.416226 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 4,289,327,341,376 unchanged 2014-08-13 18:06:07.416228 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 need 191648'337 have 0'0 2014-08-13 18:06:07.416237 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 need 191727'392 have 0'0 2014-08-13 18:06:07.416238 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.034623 8 0.000573 2014-08-13 18:06:07.416247 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 need 191648'153 have 0'0 2014-08-13 18:06:07.416252 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.416257 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 need 191648'320 have 0'0 2014-08-13 18:06:07.416266 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 need 191727'384 have 0'0 2014-08-13 18:06:07.416265 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.4 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416275 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 need 191727'371 have 0'0 2014-08-13 18:06:07.416277 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416285 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 need 191727'425 have 0'0 2014-08-13 18:06:07.416288 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.327 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416295 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 need 191727'412 have 0'0 2014-08-13 18:06:07.416298 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416304 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 need 191727'375 have 0'0 2014-08-13 18:06:07.416309 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting osd.376 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416314 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 need 191648'189 have 0'0 2014-08-13 18:06:07.416325 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 need 191727'424 have 0'0 2014-08-13 18:06:07.416334 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 need 191648'304 have 0'0 2014-08-13 18:06:07.416343 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 need 191727'411 have 0'0 2014-08-13 18:06:07.416353 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 need 191648'58 have 0'0 2014-08-13 18:06:07.416334 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.4 with 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) osd.289 (up) accepted 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) osd.341 (up) accepted 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416362 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 need 191727'405 have 0'0 2014-08-13 18:06:07.416367 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] actingbackfill is 4,289,341 2014-08-13 18:06:07.416371 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 need 191648'198 have 0'0 2014-08-13 18:06:07.416377 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] choose_acting want [4,289,341] (== acting) backfill_targets 2014-08-13 18:06:07.416381 7f5c7c7e4700 20 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] after missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 need 191727'390 have 0'0 2014-08-13 18:06:07.416389 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-13 18:06:07.416392 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.416401 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000150 0 0.000000 2014-08-13 18:06:07.416405 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.006727 7 0.000464 2014-08-13 18:06:07.416413 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:07.416418 7f5c7c7e4700 5 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.416430 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.416422 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: osd.289 has no missing, identical log 2014-08-13 18:06:07.416433 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.416434 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416443 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000031 0 0.000000 2014-08-13 18:06:07.416446 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416456 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.416454 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.416464 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416459 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416470 7f5c7c7e4700 15 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416479 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:07.416482 7f5c7c7e4700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.416479 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MNotifyRec from 348 notify: (query_epoch:194495, epoch_sent:194495, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:07.416485 7f5c7c7e4700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.416492 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416508 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:07.416520 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetInfo 0.030524 9 0.000632 2014-08-13 18:06:07.416532 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:07.416544 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416556 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416565 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416576 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416587 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:07.416596 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416607 7f5c88628700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-13 18:06:07.416626 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:07.416637 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] choose_acting want [4,332,337] != acting [4,348,331], requesting pg_temp change 2014-08-13 18:06:07.416651 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering/GetLog 0.000120 0 0.000000 2014-08-13 18:06:07.416662 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] state: Leaving Peering 2014-08-13 18:06:07.416671 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 peering m=48] exit Started/Primary/Peering 0.030685 0 0.000000 2014-08-13 18:06:07.416683 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.030708 0 0.000000 2014-08-13 18:06:07.416694 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary 2014-08-13 18:06:07.416703 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Started/Primary/Peering/WaitActingChange 2014-08-13 18:06:07.416713 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416717 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416728 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416738 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416741 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416751 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416760 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416764 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416774 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416783 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416787 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416797 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416805 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416809 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416819 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416827 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416830 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416840 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416848 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416851 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416861 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416869 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416873 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416882 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416891 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416895 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416904 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416913 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416917 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416927 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416935 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416938 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.416948 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.416957 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.416964 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.416967 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.417010 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 MLogRec from 309 2014-08-13 18:06:07.417023 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] proc_replica_log for osd.309: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) log((194215'519,194215'519], crt=194215'519) missing(32) 2014-08-13 18:06:07.417036 7f5c7bde3700 10 proc_replica_log for osd.309: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) log((194215'519,194215'519], crt=194215'519) missing(32) 2014-08-13 18:06:07.417041 7f5c7bde3700 20 before missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 need 191727'405 have 0'0 2014-08-13 18:06:07.417043 7f5c7bde3700 20 before missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 need 191648'214 have 0'0 2014-08-13 18:06:07.417045 7f5c7bde3700 20 before missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 need 191727'410 have 0'0 2014-08-13 18:06:07.417047 7f5c7bde3700 20 before missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 need 191648'366 have 0'0 2014-08-13 18:06:07.417048 7f5c7bde3700 20 before missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 need 191648'245 have 0'0 2014-08-13 18:06:07.417050 7f5c7bde3700 20 before missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 need 191727'415 have 0'0 2014-08-13 18:06:07.417052 7f5c7bde3700 20 before missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 need 191727'445 have 0'0 2014-08-13 18:06:07.417053 7f5c7bde3700 20 before missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 need 191648'179 have 0'0 2014-08-13 18:06:07.417055 7f5c7bde3700 20 before missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 need 191648'128 have 0'0 2014-08-13 18:06:07.417057 7f5c7bde3700 20 before missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 need 191727'447 have 0'0 2014-08-13 18:06:07.417059 7f5c7bde3700 20 before missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 need 191648'355 have 0'0 2014-08-13 18:06:07.417061 7f5c7bde3700 20 before missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 need 191648'39 have 0'0 2014-08-13 18:06:07.417063 7f5c7bde3700 20 before missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 need 191727'462 have 0'0 2014-08-13 18:06:07.417065 7f5c7bde3700 20 before missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 need 191727'429 have 0'0 2014-08-13 18:06:07.417066 7f5c7bde3700 20 before missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 need 191727'428 have 0'0 2014-08-13 18:06:07.417068 7f5c7bde3700 20 before missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 need 191727'394 have 0'0 2014-08-13 18:06:07.417070 7f5c7bde3700 20 before missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 need 191727'398 have 0'0 2014-08-13 18:06:07.417072 7f5c7bde3700 20 before missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 need 191648'160 have 0'0 2014-08-13 18:06:07.417073 7f5c7bde3700 20 before missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 need 191727'403 have 0'0 2014-08-13 18:06:07.417075 7f5c7bde3700 20 before missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 need 191727'455 have 0'0 2014-08-13 18:06:07.417077 7f5c7bde3700 20 before missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 need 191727'401 have 0'0 2014-08-13 18:06:07.417079 7f5c7bde3700 20 before missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 need 191727'425 have 0'0 2014-08-13 18:06:07.417080 7f5c7bde3700 20 before missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 need 191648'385 have 0'0 2014-08-13 18:06:07.417082 7f5c7bde3700 20 before missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 need 191648'32 have 0'0 2014-08-13 18:06:07.417084 7f5c7bde3700 20 before missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 need 191727'448 have 0'0 2014-08-13 18:06:07.417085 7f5c7bde3700 20 before missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 need 191727'457 have 0'0 2014-08-13 18:06:07.417087 7f5c7bde3700 20 before missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 need 191727'411 have 0'0 2014-08-13 18:06:07.417089 7f5c7bde3700 20 before missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 need 191727'433 have 0'0 2014-08-13 18:06:07.417091 7f5c7bde3700 20 before missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 need 191727'437 have 0'0 2014-08-13 18:06:07.417092 7f5c7bde3700 20 before missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 need 191727'439 have 0'0 2014-08-13 18:06:07.417094 7f5c7bde3700 20 before missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 need 191727'430 have 0'0 2014-08-13 18:06:07.417096 7f5c7bde3700 20 before missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 need 191648'168 have 0'0 2014-08-13 18:06:07.417098 7f5c7bde3700 20 merge_log cut point (usually last shared) is 194215'519 (194215'503) delete 96594481/benchmark_data_storage0111-ib.pocstor_64498_object30135/head//22 by client.1049635.0:64338 2014-08-12 17:19:28.088153 2014-08-13 18:06:07.417110 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] peer osd.309 now 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) missing(32) 2014-08-13 18:06:07.417123 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 need 191727'405 have 0'0 2014-08-13 18:06:07.417132 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 need 191648'214 have 0'0 2014-08-13 18:06:07.421422 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 need 191727'410 have 0'0 2014-08-13 18:06:07.421445 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 need 191648'366 have 0'0 2014-08-13 18:06:07.421455 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 need 191648'245 have 0'0 2014-08-13 18:06:07.421465 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 need 191727'415 have 0'0 2014-08-13 18:06:07.421474 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 need 191727'445 have 0'0 2014-08-13 18:06:07.421483 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 need 191648'179 have 0'0 2014-08-13 18:06:07.421529 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 need 191648'128 have 0'0 2014-08-13 18:06:07.421539 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 need 191727'447 have 0'0 2014-08-13 18:06:07.421559 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 need 191648'355 have 0'0 2014-08-13 18:06:07.421604 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 need 191648'39 have 0'0 2014-08-13 18:06:07.421614 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 need 191727'462 have 0'0 2014-08-13 18:06:07.421623 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 need 191727'429 have 0'0 2014-08-13 18:06:07.421647 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 need 191727'428 have 0'0 2014-08-13 18:06:07.421657 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 need 191727'394 have 0'0 2014-08-13 18:06:07.421666 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 need 191727'398 have 0'0 2014-08-13 18:06:07.421675 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 need 191648'160 have 0'0 2014-08-13 18:06:07.421684 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 need 191727'403 have 0'0 2014-08-13 18:06:07.421693 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 need 191727'455 have 0'0 2014-08-13 18:06:07.421702 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 need 191727'401 have 0'0 2014-08-13 18:06:07.421711 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 need 191727'425 have 0'0 2014-08-13 18:06:07.421719 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 need 191648'385 have 0'0 2014-08-13 18:06:07.421733 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 need 191648'32 have 0'0 2014-08-13 18:06:07.421742 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 need 191727'448 have 0'0 2014-08-13 18:06:07.421751 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 need 191727'457 have 0'0 2014-08-13 18:06:07.421760 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 need 191727'411 have 0'0 2014-08-13 18:06:07.421769 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 need 191727'433 have 0'0 2014-08-13 18:06:07.421777 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 need 191727'437 have 0'0 2014-08-13 18:06:07.421786 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 need 191727'439 have 0'0 2014-08-13 18:06:07.421795 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 need 191727'430 have 0'0 2014-08-13 18:06:07.421804 7f5c7bde3700 20 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] after missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 need 191648'168 have 0'0 2014-08-13 18:06:07.421814 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-13 18:06:07.421826 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.011234 7 0.001574 2014-08-13 18:06:07.421841 7f5c7bde3700 5 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:07.421856 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.421862 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.421875 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.421885 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.421889 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 194495 epoch_requested: 194495 FlushedEvt 2014-08-13 18:06:07.421899 7f5c7bde3700 15 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-13 18:06:07.421908 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:07.421912 7f5c7bde3700 10 osd.4 194495 queue_want_up_thru want 194495 <= queued 194495, currently 194486 2014-08-13 18:06:07.421915 7f5c7bde3700 10 osd.4 194495 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:07.545068 7f5c817ec700 10 osd.4 194495 note_peer_epoch osd.334 now has 194495 2014-08-13 18:06:07.545077 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4b12100 already has epoch 194495 2014-08-13 18:06:07.545259 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4b12100 already has epoch 194495 2014-08-13 18:06:07.599843 7f5c817ec700 10 osd.4 194495 note_peer_epoch osd.324 now has 194495 2014-08-13 18:06:07.599854 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 already has epoch 194495 2014-08-13 18:06:07.599924 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 already has epoch 194495 2014-08-13 18:06:07.671464 7f5c817ec700 10 osd.4 194495 note_peer_epoch osd.346 now has 194495 2014-08-13 18:06:07.671473 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4f21600 already has epoch 194495 2014-08-13 18:06:07.671572 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4f21600 already has epoch 194495 2014-08-13 18:06:07.851398 7f5c817ec700 10 osd.4 194495 note_peer_epoch osd.294 now has 194495 2014-08-13 18:06:07.851412 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4b2d6a0 already has epoch 194495 2014-08-13 18:06:07.851504 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4b2d6a0 already has epoch 194495 2014-08-13 18:06:07.899908 7f5c803ea700 10 osd.4 194495 note_peer_epoch osd.17 now has 194495 2014-08-13 18:06:07.899918 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4493f40 already has epoch 194495 2014-08-13 18:06:07.899977 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4493f40 already has epoch 194495 2014-08-13 18:06:07.968799 7f5c803ea700 10 osd.4 194495 note_peer_epoch osd.31 now has 194495 2014-08-13 18:06:07.968814 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4857bc0 already has epoch 194495 2014-08-13 18:06:07.968871 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4857bc0 already has epoch 194495 2014-08-13 18:06:08.101883 7f5c803ea700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 already has epoch 194495 2014-08-13 18:06:08.101988 7f5c817ec700 20 osd.4 194495 _share_map_outgoing 0x4b2cba0 already has epoch 194495 2014-08-13 18:06:08.150139 7f5c849f1700 10 osd.4 194495 do_waiters -- start 2014-08-13 18:06:08.150145 7f5c849f1700 10 osd.4 194495 do_waiters -- finish 2014-08-13 18:06:08.150147 7f5c849f1700 20 osd.4 194495 _dispatch 0x4cfa640 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.150250 7f5c849f1700 3 osd.4 194495 handle_osd_map epochs [194496,194496], i have 194495, src has [185139,194496] 2014-08-13 18:06:08.150260 7f5c849f1700 10 osd.4 194495 handle_osd_map got inc map for epoch 194496 2014-08-13 18:06:08.155545 7f5c849f1700 10 osd.4 194495 advance to epoch 194496 (<= newest 194496) 2014-08-13 18:06:08.155643 7f5c849f1700 7 osd.4 194496 advance_map epoch 194496 74 pgs 2014-08-13 18:06:08.155680 7f5c849f1700 10 osd.4 194496 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194496 [185139,194496] lci=[194495,194496]) 2014-08-13 18:06:08.156311 7f5c849f1700 0 osd.4 194496 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 18:06:08.156334 7f5c849f1700 7 osd.4 194496 consume_map version 194496 2014-08-13 18:06:08.156416 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] null 2014-08-13 18:06:08.156481 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.156525 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 2014-08-13 18:06:08.156559 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 18:06:08.156577 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.156584 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.156587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.156593 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.156610 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Started advmap 2014-08-13 18:06:08.156612 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.156627 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [4,286,375] newacting [4,375,10] 2014-08-13 18:06:08.156632 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.156645 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.156665 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary/Peering/WaitActingChange 0.748757 4 0.000267 2014-08-13 18:06:08.156688 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started/Primary 0.748790 0 0.000000 2014-08-13 18:06:08.156708 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] exit Started 0.937503 0 0.000000 2014-08-13 18:06:08.156723 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.156728 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] enter Reset 2014-08-13 18:06:08.156741 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.156745 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194495 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.156757 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.156777 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.156765 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194496 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: Reset advmap 2014-08-13 18:06:08.156791 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af8160 don't know epoch, doing nothing 2014-08-13 18:06:08.156782 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194496 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.156796 7f5c7bde3700 7 osd.4 194496 do_notify osd 351 on 1 PGs 2014-08-13 18:06:08.156799 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194496 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] acting_up_affected newup [4,286,375] newacting [4,375,10] 2014-08-13 18:06:08.156816 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194496 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.156829 7f5c7bde3700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55],24.30b=[4,375,10]} 2014-08-13 18:06:08.156833 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375] r=0 lpr=194496 pi=193640-194494/41 crt=194422'18660 lcod 0'0 mlcod 0'0 inactive m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.156876 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,286,375](4) acting [4,286,375](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,286,375](4) acting [4,286,375](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.156895 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] noting past interval(194495-194495 up [4,286,375](4) acting [4,286,375](4)) 2014-08-13 18:06:08.156902 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.156915 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] up [4,286,375] -> [4,286,375], acting [4,286,375] -> [4,375,10], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.156939 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] clear_primary_state 2014-08-13 18:06:08.156966 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] agent_stop 2014-08-13 18:06:08.156976 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 18:06:08.156986 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change 2014-08-13 18:06:08.157001 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.157007 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194496:30299 2014-08-13 18:06:08.157010 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.157026 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:08.157029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.157047 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.157047 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_copy_ops 2014-08-13 18:06:08.157067 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.157066 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_flush_ops 2014-08-13 18:06:08.157085 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:08.157086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.157102 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:08.157105 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.157119 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:08.157124 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2684 2014-08-13 18:06:08.157141 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.157137 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change_cleanup 2014-08-13 18:06:08.157165 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] on_change 2014-08-13 18:06:08.157159 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.746535 3 0.001468 2014-08-13 18:06:08.157187 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit NotTrimming 2014-08-13 18:06:08.157190 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.157206 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.958553 0 0.000000 2014-08-13 18:06:08.157206 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter NotTrimming 2014-08-13 18:06:08.157227 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] [4,286,375] -> [4,375,10], replicas changed 2014-08-13 18:06:08.157244 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] cancel_recovery 2014-08-13 18:06:08.157251 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.157260 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] clear_recovery_state 2014-08-13 18:06:08.157270 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194486 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.157280 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.157303 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map 2014-08-13 18:06:08.157310 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.157327 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.157324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] update_heartbeat_peers 4,10,286,301,375 -> 4,10,286,375 2014-08-13 18:06:08.157346 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.157342 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'479 -> 194215'479 2014-08-13 18:06:08.157349 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] take_waiters 2014-08-13 18:06:08.157368 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.157369 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Reset 0.000641 1 0.000717 2014-08-13 18:06:08.157392 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started 2014-08-13 18:06:08.157409 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Start 2014-08-13 18:06:08.157425 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: transitioning to Primary 2014-08-13 18:06:08.157446 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] exit Start 0.000036 0 0.000000 2014-08-13 18:06:08.157479 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary 2014-08-13 18:06:08.157496 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Peering 2014-08-13 18:06:08.157514 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.157531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.157550 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194495-194495 up [4,286,375](4) acting [4,286,375](4)) 2014-08-13 18:06:08.157570 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194492-194494 up [301,375,10](301) acting [301,375,10](301) maybe_went_rw) 2014-08-13 18:06:08.157591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior interval(194491-194491 up [301,375,10](301) acting [375,10,301](375)) 2014-08-13 18:06:08.157596 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 sending log((0'0,194215'479], crt=191727'415) 2014-08-13 18:06:08.157624 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:08.157610 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] PriorSet: build_prior final: probe 4,10,286,301,375 down blocked_by {} 2014-08-13 18:06:08.157629 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.157649 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194496:30300 2014-08-13 18:06:08.157667 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.10 2014-08-13 18:06:08.157671 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.58 22.362( DNE v 194215'479 lc 0'0 (0'0,194215'479] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 18:06:08.157688 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.286 2014-08-13 18:06:08.157696 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.362( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.157706 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.301 2014-08-13 18:06:08.157725 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: querying info from osd.375 2014-08-13 18:06:08.157743 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.157760 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.157874 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.157910 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru now 194496 (was 194495), currently 194495 2014-08-13 18:06:08.157898 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 sending log((0'0,194215'479], crt=191727'415) 2014-08-13 18:06:08.157916 7f5c7c7e4700 10 osd.4 194496 send_alive up_thru currently 194495 want 194496 2014-08-13 18:06:08.157920 7f5c7c7e4700 10 osd.4 194496 send_alive want 194496 2014-08-13 18:06:08.157922 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.157957 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:08.157962 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.10 on 1 PGs 2014-08-13 18:06:08.157949 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.362( DNE v 194215'479 lc 0'0 (0'0,194215'479] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 18:06:08.157986 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:08.157989 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.286 on 1 PGs 2014-08-13 18:06:08.158009 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing 2014-08-13 18:06:08.158013 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.301 on 1 PGs 2014-08-13 18:06:08.158033 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.158048 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.375 on 1 PGs 2014-08-13 18:06:08.158036 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.58 has 23 missing 2014-08-13 18:06:08.158053 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.343 has 23 missing 2014-08-13 18:06:08.158070 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.158073 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 is on osd.4 2014-08-13 18:06:08.158096 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 is on osd.4 2014-08-13 18:06:08.158113 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 is on osd.4 2014-08-13 18:06:08.158129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 is on osd.4 2014-08-13 18:06:08.158144 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 is on osd.4 2014-08-13 18:06:08.158160 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 is on osd.4 2014-08-13 18:06:08.158185 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.158202 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 is on osd.4 2014-08-13 18:06:08.158221 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 is on osd.4 2014-08-13 18:06:08.158237 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 is on osd.4 2014-08-13 18:06:08.158252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 is on osd.4 2014-08-13 18:06:08.158268 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 is on osd.4 2014-08-13 18:06:08.158284 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.158299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.158315 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 is on osd.4 2014-08-13 18:06:08.158331 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.158346 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 is on osd.4 2014-08-13 18:06:08.158362 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 is on osd.4 2014-08-13 18:06:08.158378 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 is on osd.4 2014-08-13 18:06:08.158393 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 is on osd.4 2014-08-13 18:06:08.158409 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 is on osd.4 2014-08-13 18:06:08.158424 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 is on osd.4 2014-08-13 18:06:08.158440 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 is on osd.4 2014-08-13 18:06:08.158455 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 18:06:08.158501 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate: adding 58 as a source 2014-08-13 18:06:08.158523 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 also missing on osd.58 2014-08-13 18:06:08.158541 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 also missing on osd.58 2014-08-13 18:06:08.158558 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 also missing on osd.58 2014-08-13 18:06:08.158575 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 also missing on osd.58 2014-08-13 18:06:08.158592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 also missing on osd.58 2014-08-13 18:06:08.158609 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 also missing on osd.58 2014-08-13 18:06:08.158626 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 also missing on osd.58 2014-08-13 18:06:08.158642 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 also missing on osd.58 2014-08-13 18:06:08.158659 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 also missing on osd.58 2014-08-13 18:06:08.158676 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 also missing on osd.58 2014-08-13 18:06:08.158693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 also missing on osd.58 2014-08-13 18:06:08.158710 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 also missing on osd.58 2014-08-13 18:06:08.158727 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 also missing on osd.58 2014-08-13 18:06:08.158743 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 also missing on osd.58 2014-08-13 18:06:08.158760 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 also missing on osd.58 2014-08-13 18:06:08.158777 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 also missing on osd.58 2014-08-13 18:06:08.158794 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 also missing on osd.58 2014-08-13 18:06:08.158811 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 also missing on osd.58 2014-08-13 18:06:08.158828 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 also missing on osd.58 2014-08-13 18:06:08.158844 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 also missing on osd.58 2014-08-13 18:06:08.158861 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 also missing on osd.58 2014-08-13 18:06:08.158878 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 also missing on osd.58 2014-08-13 18:06:08.158894 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 also missing on osd.58 2014-08-13 18:06:08.158890 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.158911 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 18:06:08.158961 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] activate: adding 343 as a source 2014-08-13 18:06:08.158979 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22 191727'361 also missing on osd.343 2014-08-13 18:06:08.158982 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 18:06:08.159007 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.158996 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22 191648'290 also missing on osd.343 2014-08-13 18:06:08.159015 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.159035 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.159014 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22 191727'358 also missing on osd.343 2014-08-13 18:06:08.159052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.159061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22 191727'370 also missing on osd.343 2014-08-13 18:06:08.159080 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22 191727'390 also missing on osd.343 2014-08-13 18:06:08.159097 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22 191648'273 also missing on osd.343 2014-08-13 18:06:08.159114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22 191727'414 also missing on osd.343 2014-08-13 18:06:08.159130 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22 191727'360 also missing on osd.343 2014-08-13 18:06:08.159147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22 191648'135 also missing on osd.343 2014-08-13 18:06:08.159150 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.159172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22 191727'366 also missing on osd.343 2014-08-13 18:06:08.159184 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.159190 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22 191727'347 also missing on osd.343 2014-08-13 18:06:08.159201 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.159221 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.159209 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22 191727'395 also missing on osd.343 2014-08-13 18:06:08.159234 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4423700 don't know epoch, doing nothing 2014-08-13 18:06:08.159237 7f5c7c7e4700 7 osd.4 194496 do_notify osd 9 on 1 PGs 2014-08-13 18:06:08.159227 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22 191727'398 also missing on osd.343 2014-08-13 18:06:08.159245 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22 191727'397 also missing on osd.343 2014-08-13 18:06:08.159262 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22 191648'4 also missing on osd.343 2014-08-13 18:06:08.159266 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.159280 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22 191727'411 also missing on osd.343 2014-08-13 18:06:08.159298 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22 191727'394 also missing on osd.343 2014-08-13 18:06:08.159315 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22 191648'80 also missing on osd.343 2014-08-13 18:06:08.159327 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.159333 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22 191648'162 also missing on osd.343 2014-08-13 18:06:08.159351 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22 191648'242 also missing on osd.343 2014-08-13 18:06:08.159367 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22 191648'198 also missing on osd.343 2014-08-13 18:06:08.159384 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22 191648'82 also missing on osd.343 2014-08-13 18:06:08.159401 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22 191648'224 also missing on osd.343 2014-08-13 18:06:08.159409 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 18:06:08.159433 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.159439 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.159456 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.159472 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.159417 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {1d361362/benchmark_data_storage0111-ib.pocstor_51285_object12518/head//22=191727'361,4e34362/benchmark_data_storage0111-ib.pocstor_40745_object20002/head//22=191648'290,64bb4362/benchmark_data_storage0111-ib.pocstor_51285_object10813/head//22=191727'358,55dd5362/benchmark_data_storage0111-ib.pocstor_51285_object17540/head//22=191727'370,bb926362/benchmark_data_storage0111-ib.pocstor_51285_object27025/head//22=191727'390,33dc6362/benchmark_data_storage0111-ib.pocstor_40745_object19089/head//22=191648'273,3b0e6362/benchmark_data_storage0111-ib.pocstor_51285_object38714/head//22=191727'414,64ee6362/benchmark_data_storage0111-ib.pocstor_51285_object11506/head//22=191727'360,696c7362/benchmark_data_storage0111-ib.pocstor_40745_object9882/head//22=191648'135,5ec28362/benchmark_data_storage0111-ib.pocstor_51285_object15581/head//22=191727'366,30ed8362/benchmark_data_storage0111-ib.pocstor_51285_object6280/head//22=191727'347,9958d362/benchmark_data_storage0111-ib.pocstor_51285_object29211/head//22=191727'395,335be362/benchmark_data_storage0111-ib.pocstor_51285_object29969/head//22=191727'398,b92cf362/benchmark_data_storage0111-ib.pocstor_51285_object29911/head//22=191727'397,910a1b62/benchmark_data_storage0111-ib.pocstor_40745_object280/head//22=191648'4,a6bb2b62/benchmark_data_storage0111-ib.pocstor_51285_object37514/head//22=191727'411,c3b67b62/benchmark_data_storage0111-ib.pocstor_51285_object29143/head//22=191727'394,79877b62/benchmark_data_storage0111-ib.pocstor_40745_object5690/head//22=191648'80,14db9b62/benchmark_data_storage0111-ib.pocstor_40745_object11468/head//22=191648'162,cb57cb62/benchmark_data_storage0111-ib.pocstor_40745_object17383/head//22=191648'242,5fa7cb62/benchmark_data_storage0111-ib.pocstor_40745_object13763/head//22=191648'198,6759eb62/benchmark_data_storage0111-ib.pocstor_40745_object5736/head//22=191648'82,2cadeb62/benchmark_data_storage0111-ib.pocstor_40745_object16212/head//22=191648'224} 2014-08-13 18:06:08.159500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.159516 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.159533 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.159534 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 58,61,343,349,362 2014-08-13 18:06:08.159552 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.159553 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.159569 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.159587 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.159573 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.159598 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:08.159601 7f5c7c7e4700 7 osd.4 194496 do_notify osd 58 on 1 PGs 2014-08-13 18:06:08.159591 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.159608 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.159627 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.159728 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.159827 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.159907 7f5c7bde3700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.160200 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 18:06:08.160230 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.160238 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.160256 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.160271 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.160289 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.160307 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.160324 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.160341 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2833 2014-08-13 18:06:08.160356 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.160374 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.750224 4 0.000330 2014-08-13 18:06:08.160394 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.160409 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.947880 0 0.000000 2014-08-13 18:06:08.160431 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.160447 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194486 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.160468 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.160484 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.160498 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'514 -> 194215'514 2014-08-13 18:06:08.160521 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 22.22c( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.160790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 sending log((0'0,194215'514], crt=191727'446) 2014-08-13 18:06:08.160816 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing 2014-08-13 18:06:08.160847 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.301 22.22c( DNE v 194215'514 lc 0'0 (0'0,194215'514] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(38) 2014-08-13 18:06:08.160870 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:08.160888 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 is up to date, queueing in pending_activators 2014-08-13 18:06:08.160956 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate peer osd.324 22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) uptodate 2014-08-13 18:06:08.161007 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.301 has 38 missing 2014-08-13 18:06:08.159990 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161023 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.161042 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 is on osd.4 2014-08-13 18:06:08.161049 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161059 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 is on osd.4 2014-08-13 18:06:08.161078 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.161075 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:08.161095 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.161100 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161113 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 is on osd.4 2014-08-13 18:06:08.161121 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161130 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 is on osd.4 2014-08-13 18:06:08.161141 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161149 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 is on osd.4 2014-08-13 18:06:08.161176 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.161192 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 is on osd.4 2014-08-13 18:06:08.161160 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] null 2014-08-13 18:06:08.161209 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 is on osd.4 2014-08-13 18:06:08.161223 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161227 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 is on osd.4 2014-08-13 18:06:08.161244 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 is on osd.4 2014-08-13 18:06:08.161244 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161240 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 18:06:08.161263 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 is on osd.4 2014-08-13 18:06:08.161286 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.161287 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 is on osd.4 2014-08-13 18:06:08.161296 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.161269 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] null 2014-08-13 18:06:08.161305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 is on osd.4 2014-08-13 18:06:08.161317 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.161323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 is on osd.4 2014-08-13 18:06:08.161326 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161337 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.161340 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 is on osd.4 2014-08-13 18:06:08.161350 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161360 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 is on osd.4 2014-08-13 18:06:08.161360 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.161373 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:08.161379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 is on osd.4 2014-08-13 18:06:08.161384 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.161396 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 is on osd.4 2014-08-13 18:06:08.161396 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] null 2014-08-13 18:06:08.161403 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.161416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 is on osd.4 2014-08-13 18:06:08.161422 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161426 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2736 2014-08-13 18:06:08.161436 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 is on osd.4 2014-08-13 18:06:08.161445 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161445 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.161456 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 is on osd.4 2014-08-13 18:06:08.161472 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161472 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.747532 3 0.000356 2014-08-13 18:06:08.161478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 is on osd.4 2014-08-13 18:06:08.161495 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161496 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.161497 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.161517 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161519 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.945456 0 0.000000 2014-08-13 18:06:08.161520 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 is on osd.4 2014-08-13 18:06:08.161539 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.161540 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 is on osd.4 2014-08-13 18:06:08.161546 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.161558 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 is on osd.4 2014-08-13 18:06:08.161562 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.161565 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194486 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.161577 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 is on osd.4 2014-08-13 18:06:08.161585 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:08.161589 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.161598 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 is on osd.4 2014-08-13 18:06:08.161607 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] null 2014-08-13 18:06:08.161609 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.161618 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 is on osd.4 2014-08-13 18:06:08.161636 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 is on osd.4 2014-08-13 18:06:08.161627 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'511 -> 194215'511 2014-08-13 18:06:08.161652 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 is on osd.4 2014-08-13 18:06:08.161669 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.161675 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.161685 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 is on osd.4 2014-08-13 18:06:08.161703 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 is on osd.4 2014-08-13 18:06:08.161718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 is on osd.4 2014-08-13 18:06:08.161734 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 18:06:08.161809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate: adding 301 as a source 2014-08-13 18:06:08.161829 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 also missing on osd.301 2014-08-13 18:06:08.161846 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 also missing on osd.301 2014-08-13 18:06:08.161864 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 also missing on osd.301 2014-08-13 18:06:08.161881 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 also missing on osd.301 2014-08-13 18:06:08.161898 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 also missing on osd.301 2014-08-13 18:06:08.161923 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 sending log((0'0,194215'511], crt=191727'439) 2014-08-13 18:06:08.161948 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4424360 don't know epoch, doing nothing 2014-08-13 18:06:08.161982 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.62 22.619( DNE v 194215'511 lc 0'0 (0'0,194215'511] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 18:06:08.162005 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 22.619( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.161630 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.162213 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162232 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] null 2014-08-13 18:06:08.162254 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162272 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.162290 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] null 2014-08-13 18:06:08.162310 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162309 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 sending log((0'0,194215'511], crt=191727'439) 2014-08-13 18:06:08.162337 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:08.162330 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.162349 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] null 2014-08-13 18:06:08.162371 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.162367 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate peer osd.65 22.619( DNE v 194215'511 lc 0'0 (0'0,194215'511] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 18:06:08.162391 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162409 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] null 2014-08-13 18:06:08.162461 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] null 2014-08-13 18:06:08.162465 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.62 has 29 missing 2014-08-13 18:06:08.162482 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.65 has 29 missing 2014-08-13 18:06:08.162484 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 is on osd.4 2014-08-13 18:06:08.162505 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] null 2014-08-13 18:06:08.162519 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 is on osd.4 2014-08-13 18:06:08.162526 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] null 2014-08-13 18:06:08.162537 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 is on osd.4 2014-08-13 18:06:08.162548 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] null 2014-08-13 18:06:08.162556 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 is on osd.4 2014-08-13 18:06:08.162570 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162591 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162573 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.161915 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 also missing on osd.301 2014-08-13 18:06:08.162609 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:08.162625 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 is on osd.4 2014-08-13 18:06:08.162631 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162629 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 also missing on osd.301 2014-08-13 18:06:08.162645 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 is on osd.4 2014-08-13 18:06:08.162652 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162652 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 also missing on osd.301 2014-08-13 18:06:08.162664 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 is on osd.4 2014-08-13 18:06:08.162675 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 also missing on osd.301 2014-08-13 18:06:08.162684 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 is on osd.4 2014-08-13 18:06:08.162694 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 also missing on osd.301 2014-08-13 18:06:08.162702 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 is on osd.4 2014-08-13 18:06:08.162714 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 also missing on osd.301 2014-08-13 18:06:08.162719 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.162732 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 also missing on osd.301 2014-08-13 18:06:08.162736 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 is on osd.4 2014-08-13 18:06:08.162750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 also missing on osd.301 2014-08-13 18:06:08.162753 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 is on osd.4 2014-08-13 18:06:08.162768 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 also missing on osd.301 2014-08-13 18:06:08.162770 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 is on osd.4 2014-08-13 18:06:08.162787 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 also missing on osd.301 2014-08-13 18:06:08.162787 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 is on osd.4 2014-08-13 18:06:08.162815 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 also missing on osd.301 2014-08-13 18:06:08.162824 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 is on osd.4 2014-08-13 18:06:08.162835 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 also missing on osd.301 2014-08-13 18:06:08.162687 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.162855 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 also missing on osd.301 2014-08-13 18:06:08.162875 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 also missing on osd.301 2014-08-13 18:06:08.162844 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 is on osd.4 2014-08-13 18:06:08.162892 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 also missing on osd.301 2014-08-13 18:06:08.162896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 is on osd.4 2014-08-13 18:06:08.162910 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 also missing on osd.301 2014-08-13 18:06:08.162928 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 also missing on osd.301 2014-08-13 18:06:08.162945 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 also missing on osd.301 2014-08-13 18:06:08.162963 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 also missing on osd.301 2014-08-13 18:06:08.162980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 also missing on osd.301 2014-08-13 18:06:08.162996 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 also missing on osd.301 2014-08-13 18:06:08.163013 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 also missing on osd.301 2014-08-13 18:06:08.163030 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 also missing on osd.301 2014-08-13 18:06:08.163059 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 also missing on osd.301 2014-08-13 18:06:08.162869 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] null 2014-08-13 18:06:08.163205 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:08.163233 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] null 2014-08-13 18:06:08.163263 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] null 2014-08-13 18:06:08.163286 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.163489 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163516 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163537 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163631 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.162914 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.163619 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.163655 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] null 2014-08-13 18:06:08.163076 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 also missing on osd.301 2014-08-13 18:06:08.163658 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.163659 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 is on osd.4 2014-08-13 18:06:08.163679 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] null 2014-08-13 18:06:08.163682 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 also missing on osd.301 2014-08-13 18:06:08.163692 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 is on osd.4 2014-08-13 18:06:08.163704 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] null 2014-08-13 18:06:08.163713 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 also missing on osd.301 2014-08-13 18:06:08.163712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 is on osd.4 2014-08-13 18:06:08.163728 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163735 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 also missing on osd.301 2014-08-13 18:06:08.163738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 is on osd.4 2014-08-13 18:06:08.163753 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.163754 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 also missing on osd.301 2014-08-13 18:06:08.163758 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 is on osd.4 2014-08-13 18:06:08.163775 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 also missing on osd.301 2014-08-13 18:06:08.163776 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163776 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 is on osd.4 2014-08-13 18:06:08.163796 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 also missing on osd.301 2014-08-13 18:06:08.163797 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 is on osd.4 2014-08-13 18:06:08.163800 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.163817 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 also missing on osd.301 2014-08-13 18:06:08.163818 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.163826 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] null 2014-08-13 18:06:08.163837 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 also missing on osd.301 2014-08-13 18:06:08.163843 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 is on osd.4 2014-08-13 18:06:08.163850 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.163862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 is on osd.4 2014-08-13 18:06:08.163872 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] null 2014-08-13 18:06:08.163893 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] null 2014-08-13 18:06:08.163916 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] null 2014-08-13 18:06:08.163856 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 18:06:08.163935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] activate: adding 324 as a source 2014-08-13 18:06:08.163935 7f5c849f1700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] null 2014-08-13 18:06:08.163882 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 18:06:08.163954 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22 191727'392 is on osd.324 2014-08-13 18:06:08.163960 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate: adding 62 as a source 2014-08-13 18:06:08.163976 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22 191727'401 is on osd.324 2014-08-13 18:06:08.163982 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 also missing on osd.62 2014-08-13 18:06:08.163994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22 191727'440 is on osd.324 2014-08-13 18:06:08.164001 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 also missing on osd.62 2014-08-13 18:06:08.164012 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22 191727'437 is on osd.324 2014-08-13 18:06:08.164020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 also missing on osd.62 2014-08-13 18:06:08.163962 7f5c849f1700 10 osd.4 194496 maybe_update_heartbeat_peers updating 2014-08-13 18:06:08.164040 7f5c849f1700 20 osd.4 194496 24.30b heartbeat_peers 4,10,286,375 2014-08-13 18:06:08.164031 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22 191727'399 is on osd.324 2014-08-13 18:06:08.164051 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22 191648'272 is on osd.324 2014-08-13 18:06:08.164068 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22 191727'424 is on osd.324 2014-08-13 18:06:08.164084 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22 191727'417 is on osd.324 2014-08-13 18:06:08.164108 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22 191727'371 is on osd.324 2014-08-13 18:06:08.164125 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22 191648'165 is on osd.324 2014-08-13 18:06:08.164142 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22 191727'379 is on osd.324 2014-08-13 18:06:08.164173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22 191727'421 is on osd.324 2014-08-13 18:06:08.164184 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.10 192.168.1.115:6816/60573 192.168.100.115:6834/60573 2014-08-13 18:06:08.164191 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22 191727'408 is on osd.324 2014-08-13 18:06:08.164208 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22 191648'309 is on osd.324 2014-08-13 18:06:08.164225 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22 191648'137 is on osd.324 2014-08-13 18:06:08.164246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22 191648'319 is on osd.324 2014-08-13 18:06:08.164263 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22 191648'4 is on osd.324 2014-08-13 18:06:08.164285 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22 191648'33 is on osd.324 2014-08-13 18:06:08.164293 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.375 192.168.1.114:6831/52879 192.168.100.114:6832/52879 2014-08-13 18:06:08.164310 7f5c849f1700 20 osd.4 194496 22.220 heartbeat_peers 2014-08-13 18:06:08.164303 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22 191648'108 is on osd.324 2014-08-13 18:06:08.164315 7f5c849f1700 20 osd.4 194496 22.362 heartbeat_peers 4,58,61,343,349,362 2014-08-13 18:06:08.164322 7f5c849f1700 20 osd.4 194496 22.28f heartbeat_peers 2014-08-13 18:06:08.164325 7f5c849f1700 20 osd.4 194496 22.14a heartbeat_peers 2014-08-13 18:06:08.164328 7f5c849f1700 20 osd.4 194496 22.22c heartbeat_peers 4,301,302,324,368 2014-08-13 18:06:08.164320 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22 191727'412 is on osd.324 2014-08-13 18:06:08.164334 7f5c849f1700 20 osd.4 194496 22.619 heartbeat_peers 4,44,48,62,65,314 2014-08-13 18:06:08.164342 7f5c849f1700 20 osd.4 194496 22.30f heartbeat_peers 2014-08-13 18:06:08.164345 7f5c849f1700 20 osd.4 194496 24.9b heartbeat_peers 4,291,350,365,367,378 2014-08-13 18:06:08.164338 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22 191727'434 is on osd.324 2014-08-13 18:06:08.164356 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22 191727'364 is on osd.324 2014-08-13 18:06:08.164377 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22 191648'79 is on osd.324 2014-08-13 18:06:08.164395 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22 191727'366 is on osd.324 2014-08-13 18:06:08.164417 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22 191727'431 is on osd.324 2014-08-13 18:06:08.164434 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22 191727'420 is on osd.324 2014-08-13 18:06:08.164435 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.291 192.168.1.101:6821/25633 192.168.100.101:6841/25633 2014-08-13 18:06:08.164450 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22 191727'389 is on osd.324 2014-08-13 18:06:08.164467 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22 191727'381 is on osd.324 2014-08-13 18:06:08.164488 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22 191727'382 is on osd.324 2014-08-13 18:06:08.164505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22 191727'432 is on osd.324 2014-08-13 18:06:08.164526 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22 191648'351 is on osd.324 2014-08-13 18:06:08.164543 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22 191727'409 is on osd.324 2014-08-13 18:06:08.164559 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22 191727'378 is on osd.324 2014-08-13 18:06:08.164576 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22 191727'367 is on osd.324 2014-08-13 18:06:08.164592 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22 191727'406 is on osd.324 2014-08-13 18:06:08.164609 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22 191648'48 is on osd.324 2014-08-13 18:06:08.164626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22 191727'386 is on osd.324 2014-08-13 18:06:08.164642 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22 191727'430 is on osd.324 2014-08-13 18:06:08.164687 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.350 192.168.1.114:6833/53185 192.168.100.114:6835/53185 2014-08-13 18:06:08.164659 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f943022c/benchmark_data_storage0111-ib.pocstor_51285_object10496/head//22=191727'392,e5c3022c/benchmark_data_storage0111-ib.pocstor_51285_object16208/head//22=191727'401,3afe022c/benchmark_data_storage0111-ib.pocstor_51285_object36162/head//22=191727'440,2311122c/benchmark_data_storage0111-ib.pocstor_51285_object34705/head//22=191727'437,fcbd222c/benchmark_data_storage0111-ib.pocstor_51285_object14385/head//22=191727'399,8cfe222c/benchmark_data_storage0111-ib.pocstor_40745_object17239/head//22=191648'272,98f9322c/benchmark_data_storage0111-ib.pocstor_51285_object28002/head//22=191727'424,1b0e422c/benchmark_data_storage0111-ib.pocstor_51285_object23617/head//22=191727'417,3c2e422c/benchmark_data_storage0111-ib.pocstor_51285_object2314/head//22=191727'371,4f31722c/benchmark_data_storage0111-ib.pocstor_40745_object10169/head//22=191648'165,98a7722c/benchmark_data_storage0111-ib.pocstor_51285_object4948/head//22=191727'379,765a822c/benchmark_data_storage0111-ib.pocstor_51285_object26499/head//22=191727'421,df0f822c/benchmark_data_storage0111-ib.pocstor_51285_object19174/head//22=191727'408,3331922c/benchmark_data_storage0111-ib.pocstor_40745_object19164/head//22=191648'309,7ad4922c/benchmark_data_storage0111-ib.pocstor_40745_object8600/head//22=191648'137,f50ca22c/benchmark_data_storage0111-ib.pocstor_40745_object19543/head//22=191648'319,a1d3b22c/benchmark_data_storage0111-ib.pocstor_40745_object84/head//22=191648'4,3e39f22c/benchmark_data_storage0111-ib.pocstor_40745_object1706/head//22=191648'33,a4ec1a2c/benchmark_data_storage0111-ib.pocstor_40745_object6328/head//22=191648'108,a00b2a2c/benchmark_data_storage0111-ib.pocstor_51285_object21427/head//22=191727'412,d1cc2a2c/benchmark_data_storage0111-ib.pocstor_51285_object32955/head//22=191727'434,2d303a2c/benchmark_data_storage0111-ib.pocstor_51285_object305/head//22=191727'364,300e4a2c/benchmark_data_storage0111-ib.pocstor_40745_object4480/head//22=191648'79,93a05a2c/benchmark_data_storage0111-ib.pocstor_51285_object1487/head//22=191727'366,37155a2c/benchmark_data_storage0111-ib.pocstor_51285_object31577/head//22=191727'431,13c75a2c/benchmark_data_storage0111-ib.pocstor_51285_object25894/head//22=191727'420,2bbe5a2c/benchmark_data_storage0111-ib.pocstor_51285_object9584/head//22=191727'389,f6509a2c/benchmark_data_storage0111-ib.pocstor_51285_object6029/head//22=191727'381,ef9a9a2c/benchmark_data_storage0111-ib.pocstor_51285_object6140/head//22=191727'382,a0baba2c/benchmark_data_storage0111-ib.pocstor_51285_object31975/head//22=191727'432,e1e2ca2c/benchmark_data_storage0111-ib.pocstor_40745_object22217/head//22=191648'351,ed1dca2c/benchmark_data_storage0111-ib.pocstor_51285_object19482/head//22=191727'409,1d7dca2c/benchmark_data_storage0111-ib.pocstor_51285_object4219/head//22=191727'378,9e8cea2c/benchmark_data_storage0111-ib.pocstor_51285_object1566/head//22=191727'367,fdfcea2c/benchmark_data_storage0111-ib.pocstor_51285_object18816/head//22=191727'406,cb1fa2c/benchmark_data_storage0111-ib.pocstor_40745_object2885/head//22=191648'48,ced2fa2c/benchmark_data_storage0111-ib.pocstor_51285_object8351/head//22=191727'386,3db9fa2c/benchmark_data_storage0111-ib.pocstor_51285_object31103/head//22=191727'430} 2014-08-13 18:06:08.164728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.164747 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.164765 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 301,302,324,368 2014-08-13 18:06:08.164786 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.365 192.168.1.102:6825/15623 192.168.100.102:6806/15623 2014-08-13 18:06:08.164786 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.164806 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.164823 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.164839 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.164940 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.164945 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.367 192.168.1.114:6841/55513 192.168.100.114:6847/55513 2014-08-13 18:06:08.164977 7f5c7c7e4700 20 osd.4 194496 Sending info 22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) to shard 324 2014-08-13 18:06:08.165044 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.378 192.168.1.102:6841/19195 192.168.100.102:6844/19195 2014-08-13 18:06:08.165060 7f5c849f1700 20 osd.4 194496 22.581 heartbeat_peers 2014-08-13 18:06:08.165064 7f5c849f1700 20 osd.4 194496 22.f3 heartbeat_peers 4,285,311,312,369 2014-08-13 18:06:08.165071 7f5c849f1700 20 osd.4 194496 22.116 heartbeat_peers 2014-08-13 18:06:08.165074 7f5c849f1700 20 osd.4 194496 24.3d heartbeat_peers 4,20,25,303,362,373 2014-08-13 18:06:08.165075 7f5c7c7e4700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4b2cba0 192.168.1.104:6814/65209 2014-08-13 18:06:08.165138 7f5c7c7e4700 10 osd.4 194496 note_peer_epoch osd.324 has 194496 2014-08-13 18:06:08.165350 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.3d=[4,20,303],24.9b=[4,291,350],24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.165450 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 2014-08-13 18:06:08.165486 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.165494 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.165513 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Started advmap 2014-08-13 18:06:08.165529 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [4,25,362] newacting [4,20,303] 2014-08-13 18:06:08.165545 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.165563 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary/Peering/WaitActingChange 0.753390 3 0.000284 2014-08-13 18:06:08.165592 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started/Primary 0.753441 0 0.000000 2014-08-13 18:06:08.165612 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] exit Started 0.902757 0 0.000000 2014-08-13 18:06:08.165631 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] enter Reset 2014-08-13 18:06:08.165646 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194495 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194496 2014-08-13 18:06:08.165663 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194496 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: Reset advmap 2014-08-13 18:06:08.165678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194496 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.165694 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194496 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] acting_up_affected newup [4,25,362] newacting [4,20,303] 2014-08-13 18:06:08.165709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194496 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.165725 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362] r=0 lpr=194496 pi=193635-194494/38 crt=194422'18731 lcod 0'0 mlcod 0'0 inactive m=57] set_last_peering_reset 194496 2014-08-13 18:06:08.165762 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,25,362](4) acting [4,25,362](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,25,362](4) acting [4,25,362](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.165780 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] noting past interval(194495-194495 up [4,25,362](4) acting [4,25,362](4)) 2014-08-13 18:06:08.165799 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] up [4,25,362] -> [4,25,362], acting [4,25,362] -> [4,20,303], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.165820 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] clear_primary_state 2014-08-13 18:06:08.165812 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.165844 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] agent_stop 2014-08-13 18:06:08.165846 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.165863 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change 2014-08-13 18:06:08.165883 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194496:29708 2014-08-13 18:06:08.165900 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:08.165919 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_copy_ops 2014-08-13 18:06:08.165935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_flush_ops 2014-08-13 18:06:08.165950 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:08.165965 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:08.165980 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:08.166001 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change_cleanup 2014-08-13 18:06:08.166017 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] on_change 2014-08-13 18:06:08.166036 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit NotTrimming 2014-08-13 18:06:08.166053 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter NotTrimming 2014-08-13 18:06:08.166069 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] [4,25,362] -> [4,20,303], replicas changed 2014-08-13 18:06:08.166086 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] cancel_recovery 2014-08-13 18:06:08.166101 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] clear_recovery_state 2014-08-13 18:06:08.166097 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.25 192.168.1.103:6801/36920 192.168.100.103:6802/36920 2014-08-13 18:06:08.166124 7f5c849f1700 20 osd.4 194496 22.4aa heartbeat_peers 2014-08-13 18:06:08.166129 7f5c849f1700 20 osd.4 194496 22.1d2 heartbeat_peers 4,58,61,330,331,349 2014-08-13 18:06:08.166120 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.166138 7f5c849f1700 20 osd.4 194496 24.1de heartbeat_peers 2014-08-13 18:06:08.166141 7f5c849f1700 20 osd.4 194496 2.2a heartbeat_peers 4,324,340,370 2014-08-13 18:06:08.166141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map 2014-08-13 18:06:08.166174 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] update_heartbeat_peers 4,20,25,303,362,373 -> 4,20,25,303,362 2014-08-13 18:06:08.166262 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 2014-08-13 18:06:08.166282 7f5c849f1700 20 osd.4 194496 22.113 heartbeat_peers 4,24,45,55,339,342 2014-08-13 18:06:08.166291 7f5c849f1700 20 osd.4 194496 24.39a heartbeat_peers 2014-08-13 18:06:08.166294 7f5c849f1700 20 osd.4 194496 24.33a heartbeat_peers 2014-08-13 18:06:08.166299 7f5c849f1700 20 osd.4 194496 1.2b heartbeat_peers 4,324,340,370 2014-08-13 18:06:08.166304 7f5c849f1700 20 osd.4 194496 24.34 heartbeat_peers 2014-08-13 18:06:08.166307 7f5c849f1700 20 osd.4 194496 22.99 heartbeat_peers 4,20,31,337,363,374 2014-08-13 18:06:08.166314 7f5c849f1700 20 osd.4 194496 22.84 heartbeat_peers 4,7,28,304 2014-08-13 18:06:08.166405 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 2014-08-13 18:06:08.166423 7f5c849f1700 20 osd.4 194496 22.4df heartbeat_peers 4,352,356,357,371,380 2014-08-13 18:06:08.166431 7f5c849f1700 20 osd.4 194496 22.39d heartbeat_peers 4,320,344,363,366,373 2014-08-13 18:06:08.166438 7f5c849f1700 20 osd.4 194496 22.2b9 heartbeat_peers 2014-08-13 18:06:08.166441 7f5c849f1700 20 osd.4 194496 24.2c heartbeat_peers 2014-08-13 18:06:08.166443 7f5c849f1700 20 osd.4 194496 24.3f7 heartbeat_peers 2014-08-13 18:06:08.166446 7f5c849f1700 20 osd.4 194496 22.775 heartbeat_peers 0,4,56,63,353,369 2014-08-13 18:06:08.166453 7f5c849f1700 20 osd.4 194496 22.630 heartbeat_peers 2014-08-13 18:06:08.166456 7f5c849f1700 20 osd.4 194496 24.e2 heartbeat_peers 2014-08-13 18:06:08.166459 7f5c849f1700 20 osd.4 194496 22.69f heartbeat_peers 2014-08-13 18:06:08.166462 7f5c849f1700 20 osd.4 194496 22.3e4 heartbeat_peers 4,45,303,313,318,342 2014-08-13 18:06:08.166469 7f5c849f1700 20 osd.4 194496 24.2a1 heartbeat_peers 2014-08-13 18:06:08.166471 7f5c849f1700 20 osd.4 194496 22.639 heartbeat_peers 2014-08-13 18:06:08.166474 7f5c849f1700 20 osd.4 194496 22.707 heartbeat_peers 4,15,21,303,310,342 2014-08-13 18:06:08.166481 7f5c849f1700 20 osd.4 194496 24.5e heartbeat_peers 2014-08-13 18:06:08.166484 7f5c849f1700 20 osd.4 194496 22.1f1 heartbeat_peers 4,5,11,47,54,368 2014-08-13 18:06:08.166494 7f5c849f1700 20 osd.4 194496 22.2d1 heartbeat_peers 2014-08-13 18:06:08.166500 7f5c849f1700 20 osd.4 194496 24.b6 heartbeat_peers 4,17,28,315,327,330 2014-08-13 18:06:08.166970 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.17 192.168.1.103:6812/37658 192.168.100.103:6818/37658 2014-08-13 18:06:08.164040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 also missing on osd.62 2014-08-13 18:06:08.167078 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 also missing on osd.62 2014-08-13 18:06:08.167107 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 also missing on osd.62 2014-08-13 18:06:08.167149 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 also missing on osd.62 2014-08-13 18:06:08.167167 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 also missing on osd.62 2014-08-13 18:06:08.167196 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 also missing on osd.62 2014-08-13 18:06:08.167214 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 also missing on osd.62 2014-08-13 18:06:08.167232 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 also missing on osd.62 2014-08-13 18:06:08.167250 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 also missing on osd.62 2014-08-13 18:06:08.167267 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 also missing on osd.62 2014-08-13 18:06:08.167284 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 also missing on osd.62 2014-08-13 18:06:08.167301 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 also missing on osd.62 2014-08-13 18:06:08.167318 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 also missing on osd.62 2014-08-13 18:06:08.167335 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 also missing on osd.62 2014-08-13 18:06:08.167352 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 also missing on osd.62 2014-08-13 18:06:08.167369 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 also missing on osd.62 2014-08-13 18:06:08.167386 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 also missing on osd.62 2014-08-13 18:06:08.167403 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 also missing on osd.62 2014-08-13 18:06:08.167420 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 also missing on osd.62 2014-08-13 18:06:08.167437 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 also missing on osd.62 2014-08-13 18:06:08.167454 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 also missing on osd.62 2014-08-13 18:06:08.167470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 also missing on osd.62 2014-08-13 18:06:08.167487 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 also missing on osd.62 2014-08-13 18:06:08.167504 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 also missing on osd.62 2014-08-13 18:06:08.167521 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 also missing on osd.62 2014-08-13 18:06:08.167538 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 also missing on osd.62 2014-08-13 18:06:08.169577 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.315 192.168.1.113:6801/14161 192.168.100.113:6801/14161 2014-08-13 18:06:08.169606 7f5c849f1700 20 osd.4 194496 24.3bf heartbeat_peers 2014-08-13 18:06:08.169610 7f5c849f1700 20 osd.4 194496 22.1fb heartbeat_peers 2014-08-13 18:06:08.169613 7f5c849f1700 20 osd.4 194496 24.350 heartbeat_peers 2014-08-13 18:06:08.169619 7f5c849f1700 20 osd.4 194496 24.1f7 heartbeat_peers 4,55,59,351,376 2014-08-13 18:06:08.167553 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 18:06:08.169897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] activate: adding 65 as a source 2014-08-13 18:06:08.169968 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22 191648'216 also missing on osd.65 2014-08-13 18:06:08.169990 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22 191727'372 also missing on osd.65 2014-08-13 18:06:08.170009 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22 191727'375 also missing on osd.65 2014-08-13 18:06:08.170027 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22 191727'438 also missing on osd.65 2014-08-13 18:06:08.170045 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22 191727'397 also missing on osd.65 2014-08-13 18:06:08.170076 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22 191727'366 also missing on osd.65 2014-08-13 18:06:08.170094 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22 191727'373 also missing on osd.65 2014-08-13 18:06:08.170120 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22 191648'298 also missing on osd.65 2014-08-13 18:06:08.170138 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22 191727'408 also missing on osd.65 2014-08-13 18:06:08.170169 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22 191648'155 also missing on osd.65 2014-08-13 18:06:08.170201 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22 191727'437 also missing on osd.65 2014-08-13 18:06:08.170252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22 191648'200 also missing on osd.65 2014-08-13 18:06:08.170362 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22 191648'361 also missing on osd.65 2014-08-13 18:06:08.170390 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22 191648'222 also missing on osd.65 2014-08-13 18:06:08.170417 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22 191648'29 also missing on osd.65 2014-08-13 18:06:08.170435 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22 191727'429 also missing on osd.65 2014-08-13 18:06:08.170513 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22 191727'374 also missing on osd.65 2014-08-13 18:06:08.170545 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22 191727'417 also missing on osd.65 2014-08-13 18:06:08.170613 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22 191727'420 also missing on osd.65 2014-08-13 18:06:08.170632 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22 191648'157 also missing on osd.65 2014-08-13 18:06:08.170649 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22 191648'215 also missing on osd.65 2014-08-13 18:06:08.170691 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22 191727'435 also missing on osd.65 2014-08-13 18:06:08.170710 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22 191727'377 also missing on osd.65 2014-08-13 18:06:08.170728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22 191727'427 also missing on osd.65 2014-08-13 18:06:08.170745 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22 191727'369 also missing on osd.65 2014-08-13 18:06:08.170787 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22 191648'35 also missing on osd.65 2014-08-13 18:06:08.170805 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22 191727'371 also missing on osd.65 2014-08-13 18:06:08.170833 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22 191648'295 also missing on osd.65 2014-08-13 18:06:08.170879 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22 191648'207 also missing on osd.65 2014-08-13 18:06:08.170895 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f2e90619/benchmark_data_storage0111-ib.pocstor_40745_object12402/head//22=191648'216,78521619/benchmark_data_storage0111-ib.pocstor_51285_object4442/head//22=191727'372,6c761619/benchmark_data_storage0111-ib.pocstor_51285_object6425/head//22=191727'375,43df1619/benchmark_data_storage0111-ib.pocstor_51285_object37178/head//22=191727'438,30b32619/benchmark_data_storage0111-ib.pocstor_51285_object17497/head//22=191727'397,1e9c2619/benchmark_data_storage0111-ib.pocstor_51285_object617/head//22=191727'366,170a4619/benchmark_data_storage0111-ib.pocstor_51285_object5944/head//22=191727'373,a6a46619/benchmark_data_storage0111-ib.pocstor_40745_object18353/head//22=191648'298,8bca6619/benchmark_data_storage0111-ib.pocstor_51285_object21275/head//22=191727'408,e25f7619/benchmark_data_storage0111-ib.pocstor_40745_object9104/head//22=191648'155,d9499619/benchmark_data_storage0111-ib.pocstor_51285_object36943/head//22=191727'437,ba4ba619/benchmark_data_storage0111-ib.pocstor_40745_object11708/head//22=191648'200,da71b619/benchmark_data_storage0111-ib.pocstor_40745_object22716/head//22=191648'361,e80cc619/benchmark_data_storage0111-ib.pocstor_40745_object12735/head//22=191648'222,b400e619/benchmark_data_storage0111-ib.pocstor_40745_object1796/head//22=191648'29,dad20e19/benchmark_data_storage0111-ib.pocstor_51285_object33391/head//22=191727'429,2daa0e19/benchmark_data_storage0111-ib.pocstor_51285_object6168/head//22=191727'374,737b0e19/benchmark_data_storage0111-ib.pocstor_51285_object26507/head//22=191727'417,846c0e19/benchmark_data_storage0111-ib.pocstor_51285_object28599/head//22=191727'420,e9217e19/benchmark_data_storage0111-ib.pocstor_40745_object9251/head//22=191648'157,66317e19/benchmark_data_storage0111-ib.pocstor_40745_object12400/head//22=191648'215,97a68e19/benchmark_data_storage0111-ib.pocstor_51285_object36749/head//22=191727'435,4c19ae19/benchmark_data_storage0111-ib.pocstor_51285_object7434/head//22=191727'377,6d41ce19/benchmark_data_storage0111-ib.pocstor_51285_object33053/head//22=191727'427,72cbce19/benchmark_data_storage0111-ib.pocstor_51285_object2412/head//22=191727'369,7a29de19/benchmark_data_storage0111-ib.pocstor_40745_object2101/head//22=191648'35,fc2dee19/benchmark_data_storage0111-ib.pocstor_51285_object4042/head//22=191727'371,d1f3fe19/benchmark_data_storage0111-ib.pocstor_40745_object18287/head//22=191648'295,4215fe19/benchmark_data_storage0111-ib.pocstor_40745_object12030/head//22=191648'207} 2014-08-13 18:06:08.208902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.208946 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.208968 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 44,48,62,65,314 2014-08-13 18:06:08.208987 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.209007 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.209026 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.209049 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.209184 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.209249 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.59 192.168.1.112:6823/5265 192.168.100.112:6804/5265 2014-08-13 18:06:08.211626 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.351 192.168.1.102:6831/16444 192.168.100.102:6826/16444 2014-08-13 18:06:08.211657 7f5c849f1700 20 osd.4 194496 24.136 heartbeat_peers 4,50,55,61,65,356 2014-08-13 18:06:08.211667 7f5c849f1700 20 osd.4 194496 22.4fe heartbeat_peers 2014-08-13 18:06:08.211670 7f5c849f1700 20 osd.4 194496 22.653 heartbeat_peers 2014-08-13 18:06:08.211673 7f5c849f1700 20 osd.4 194496 24.3b7 heartbeat_peers 2014-08-13 18:06:08.211676 7f5c849f1700 20 osd.4 194496 22.3b9 heartbeat_peers 2014-08-13 18:06:08.211679 7f5c849f1700 20 osd.4 194496 22.791 heartbeat_peers 2014-08-13 18:06:08.211681 7f5c849f1700 20 osd.4 194496 22.6bd heartbeat_peers 4,32,50,289 2014-08-13 18:06:08.211715 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 18:06:08.211759 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.211769 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.211791 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.211810 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.211867 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.211883 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.211911 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.211930 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.212433 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.212480 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.213630 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 2014-08-13 18:06:08.213656 7f5c849f1700 20 osd.4 194496 24.181 heartbeat_peers 2014-08-13 18:06:08.213661 7f5c849f1700 20 osd.4 194496 24.20e heartbeat_peers 2014-08-13 18:06:08.213664 7f5c849f1700 20 osd.4 194496 24.1e4 heartbeat_peers 2014-08-13 18:06:08.213667 7f5c849f1700 20 osd.4 194496 24.1e6 heartbeat_peers 2014-08-13 18:06:08.213670 7f5c849f1700 20 osd.4 194496 22.12a heartbeat_peers 2014-08-13 18:06:08.213676 7f5c849f1700 20 osd.4 194496 22.3af heartbeat_peers 4,44,61,339,362,369 2014-08-13 18:06:08.213686 7f5c849f1700 20 osd.4 194496 22.481 heartbeat_peers 4,287,303,306,309,365,381 2014-08-13 18:06:08.213695 7f5c849f1700 20 osd.4 194496 22.48e heartbeat_peers 4,289,327,341,376 2014-08-13 18:06:08.213702 7f5c849f1700 20 osd.4 194496 22.3ad heartbeat_peers 2014-08-13 18:06:08.213704 7f5c849f1700 20 osd.4 194496 24.206 heartbeat_peers 2014-08-13 18:06:08.213670 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 2014-08-13 18:06:08.213707 7f5c849f1700 20 osd.4 194496 24.1bb heartbeat_peers 4,331,332,337,348,379 2014-08-13 18:06:08.213725 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.213735 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.213761 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Started advmap 2014-08-13 18:06:08.213778 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,378,367] newacting [4,291,350] 2014-08-13 18:06:08.213799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.213807 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.332 192.168.1.110:6807/42024 192.168.100.110:6825/42024 2014-08-13 18:06:08.213819 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.800247 1 0.000237 2014-08-13 18:06:08.213842 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.800280 0 0.000000 2014-08-13 18:06:08.213862 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.992151 0 0.000000 2014-08-13 18:06:08.213894 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 18:06:08.213911 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194495 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194496 2014-08-13 18:06:08.213931 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194496 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 18:06:08.213949 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194496 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.213965 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194496 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,378,367] newacting [4,291,350] 2014-08-13 18:06:08.213962 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.348 192.168.1.110:6801/41204 192.168.100.110:6804/41204 2014-08-13 18:06:08.213981 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194496 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.213996 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367] r=0 lpr=194496 pi=193637-194494/35 crt=194422'17302 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194496 2014-08-13 18:06:08.214040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,378,367](4) acting [4,378,367](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,378,367](4) acting [4,378,367](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.214060 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194495/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] noting past interval(194495-194495 up [4,378,367](4) acting [4,378,367](4)) 2014-08-13 18:06:08.214083 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] up [4,378,367] -> [4,378,367], acting [4,378,367] -> [4,291,350], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.214089 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.379 192.168.1.108:6831/4861 192.168.100.108:6835/4861 2014-08-13 18:06:08.214107 7f5c849f1700 20 osd.4 194496 24.299 heartbeat_peers 2014-08-13 18:06:08.214112 7f5c849f1700 20 osd.4 194496 22.3d5 heartbeat_peers 4,10,286,343 2014-08-13 18:06:08.214104 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] clear_primary_state 2014-08-13 18:06:08.214121 7f5c849f1700 20 osd.4 194496 22.537 heartbeat_peers 2014-08-13 18:06:08.214124 7f5c849f1700 20 osd.4 194496 22.1b8 heartbeat_peers 4,7,340,347 2014-08-13 18:06:08.214130 7f5c849f1700 20 osd.4 194496 22.29a heartbeat_peers 2014-08-13 18:06:08.214133 7f5c849f1700 20 osd.4 194496 24.cd heartbeat_peers 2014-08-13 18:06:08.214137 7f5c849f1700 20 osd.4 194496 22.226 heartbeat_peers 2014-08-13 18:06:08.214140 7f5c849f1700 20 osd.4 194496 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 2014-08-13 18:06:08.214133 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] agent_stop 2014-08-13 18:06:08.214153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 18:06:08.214173 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194496:25291 2014-08-13 18:06:08.214206 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.214226 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 18:06:08.214247 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 18:06:08.214251 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.293 192.168.1.109:6817/33855 192.168.100.109:6844/33855 2014-08-13 18:06:08.214264 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.214280 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.214301 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.214321 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 18:06:08.214338 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 18:06:08.214363 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 18:06:08.214373 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.382 192.168.1.108:6801/30527 192.168.100.108:6801/30527 2014-08-13 18:06:08.214382 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 18:06:08.214399 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] [4,378,367] -> [4,291,350], replicas changed 2014-08-13 18:06:08.214416 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] cancel_recovery 2014-08-13 18:06:08.214431 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] clear_recovery_state 2014-08-13 18:06:08.214451 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.214472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map 2014-08-13 18:06:08.214493 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] update_heartbeat_peers 4,291,350,365,367,378 -> 4,291,350,367,378 2014-08-13 18:06:08.214966 7f5c849f1700 10 osd.4 194496 _add_heartbeat_peer: new peer osd.397 192.168.1.105:6809/41085 192.168.100.105:6802/41085 2014-08-13 18:06:08.214994 7f5c849f1700 20 osd.4 194496 24.2ec heartbeat_peers 2014-08-13 18:06:08.214999 7f5c849f1700 20 osd.4 194496 22.1a0 heartbeat_peers 2014-08-13 18:06:08.215002 7f5c849f1700 20 osd.4 194496 24.369 heartbeat_peers 2014-08-13 18:06:08.215010 7f5c849f1700 10 osd.4 194496 adding neighbor peer osd.3 2014-08-13 18:06:08.215013 7f5c849f1700 10 osd.4 194496 adding neighbor peer osd.5 2014-08-13 18:06:08.215025 7f5c849f1700 10 osd.4 194496 maybe_update_heartbeat_peers 91 peers, extras 3,5 2014-08-13 18:06:08.215037 7f5c849f1700 7 osd.4 194496 activate_map version 194496 2014-08-13 18:06:08.215051 7f5c849f1700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215055 7f5c849f1700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215052 7f5c758d8700 20 osd.4 194496 agent_entry empty queue 2014-08-13 18:06:08.215058 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.215070 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215081 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215084 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed1440 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215102 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215108 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215079 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] take_waiters 2014-08-13 18:06:08.215111 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215119 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215136 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.304 now has 194496 2014-08-13 18:06:08.215122 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Reset 0.049492 1 0.000647 2014-08-13 18:06:08.215149 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.215148 7f5c849f1700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215165 7f5c849f1700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215168 7f5c849f1700 20 osd.4 194496 _dispatch 0x51386c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215182 7f5c849f1700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215189 7f5c849f1700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215200 7f5c849f1700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215203 7f5c849f1700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215217 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.215219 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215225 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215227 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351dc0 pg_notify(24.30b(45) epoch 194496) v5 2014-08-13 18:06:08.215236 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.215219 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started 2014-08-13 18:06:08.215244 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] take_waiters 2014-08-13 18:06:08.215268 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Start 2014-08-13 18:06:08.215288 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.375 2014-08-13 18:06:08.215292 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351dc0 2014-08-13 18:06:08.215275 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.001381 1 0.000738 2014-08-13 18:06:08.215301 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started 2014-08-13 18:06:08.215287 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: transitioning to Primary 2014-08-13 18:06:08.215320 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Start 2014-08-13 18:06:08.215339 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary 2014-08-13 18:06:08.215356 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215359 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215336 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] exit Start 0.000067 0 0.000000 2014-08-13 18:06:08.215360 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000040 0 0.000000 2014-08-13 18:06:08.215370 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary 2014-08-13 18:06:08.215383 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary 2014-08-13 18:06:08.215410 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215413 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215416 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed4a40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215408 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering 2014-08-13 18:06:08.215424 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215430 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215406 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Peering 2014-08-13 18:06:08.215432 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215435 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215431 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.215465 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215450 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.215469 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215471 7f5c835ef700 20 osd.4 194496 _dispatch 0x5153600 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215443 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.215478 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215483 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215485 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215488 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215475 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194495-194495 up [4,378,367](4) acting [4,378,367](4)) 2014-08-13 18:06:08.215479 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.215507 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215510 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215512 7f5c835ef700 20 osd.4 194496 _dispatch 0x5150900 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215497 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194492-194494 up [365,350,291](365) acting [365,350,291](365) maybe_went_rw) 2014-08-13 18:06:08.215518 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215522 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215524 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215526 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215520 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194491-194491 up [365,350,291](365) acting [291,350](291)) 2014-08-13 18:06:08.215545 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215548 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215550 7f5c835ef700 20 osd.4 194496 _dispatch 0x5152d00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.215539 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior final: probe 4,291,350,365,367,378 down blocked_by {} 2014-08-13 18:06:08.215556 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.215560 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.215563 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.215565 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.215560 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.215503 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194495-194495 up [4,25,362](4) acting [4,25,362](4)) 2014-08-13 18:06:08.215583 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194496:25292 2014-08-13 18:06:08.215603 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.291 2014-08-13 18:06:08.215594 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194492-194494 up [20,373,303](20) acting [20,373,303](20) maybe_went_rw) 2014-08-13 18:06:08.215623 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.350 2014-08-13 18:06:08.215642 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.365 2014-08-13 18:06:08.215639 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior interval(194491-194491 up [20,373,303](20) acting [20,303,373](20)) 2014-08-13 18:06:08.215660 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.367 2014-08-13 18:06:08.215668 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] PriorSet: build_prior final: probe 4,20,25,303,362,373 down blocked_by {} 2014-08-13 18:06:08.215682 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.378 2014-08-13 18:06:08.215702 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.215694 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.215720 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.215729 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194496:29709 2014-08-13 18:06:08.215751 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.20 2014-08-13 18:06:08.215772 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.25 2014-08-13 18:06:08.215790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.303 2014-08-13 18:06:08.215810 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.362 2014-08-13 18:06:08.215835 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.215827 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: querying info from osd.373 2014-08-13 18:06:08.215850 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.215885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.215891 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 18:06:08.215919 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.215927 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.215945 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.215962 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.216043 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.216060 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.216075 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.216093 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.216081 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.216123 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 18:06:08.216146 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.216153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.216170 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.216197 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.216216 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.216233 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.216208 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 18:06:08.216250 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.216263 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.216268 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2863 2014-08-13 18:06:08.216272 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.216285 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.216303 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.804205 4 0.000364 2014-08-13 18:06:08.216323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.216292 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.216339 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.969845 0 0.000000 2014-08-13 18:06:08.216356 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.216363 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.216379 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.216401 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.216416 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.216430 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'506 -> 194215'506 2014-08-13 18:06:08.216433 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.216456 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 22.f3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.216452 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.216531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.216573 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.216602 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 18:06:08.216625 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.216633 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.216656 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.216709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.216728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.216740 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 sending log((0'0,194215'506], crt=191648'343) 2014-08-13 18:06:08.216751 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.216766 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.216770 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.216793 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:3023 2014-08-13 18:06:08.216800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.285 22.f3( DNE v 194215'506 lc 0'0 (0'0,194215'506] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 18:06:08.216822 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.216824 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) 2014-08-13 18:06:08.216841 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.804472 4 0.000397 2014-08-13 18:06:08.216844 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 is up to date, queueing in pending_activators 2014-08-13 18:06:08.216865 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.216885 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.941117 0 0.000000 2014-08-13 18:06:08.216904 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate peer osd.312 22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) uptodate 2014-08-13 18:06:08.216909 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.216947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.285 has 29 missing 2014-08-13 18:06:08.216956 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194486 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.216965 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 is on osd.4 2014-08-13 18:06:08.216985 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 is on osd.4 2014-08-13 18:06:08.216986 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.217004 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.217004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 is on osd.4 2014-08-13 18:06:08.217021 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'510 -> 194215'510 2014-08-13 18:06:08.217022 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 is on osd.4 2014-08-13 18:06:08.217040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 is on osd.4 2014-08-13 18:06:08.217049 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.217057 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 is on osd.4 2014-08-13 18:06:08.217074 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 is on osd.4 2014-08-13 18:06:08.217090 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.217106 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 is on osd.4 2014-08-13 18:06:08.217122 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.217137 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 is on osd.4 2014-08-13 18:06:08.217153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.217168 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 is on osd.4 2014-08-13 18:06:08.217184 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 is on osd.4 2014-08-13 18:06:08.217210 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.217226 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 is on osd.4 2014-08-13 18:06:08.217242 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 is on osd.4 2014-08-13 18:06:08.217258 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.217284 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 is on osd.4 2014-08-13 18:06:08.217300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 is on osd.4 2014-08-13 18:06:08.217315 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 is on osd.4 2014-08-13 18:06:08.217333 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.217324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 sending log((0'0,194215'510], crt=191648'372) 2014-08-13 18:06:08.217353 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:08.217350 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 is on osd.4 2014-08-13 18:06:08.217369 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.217386 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 is on osd.4 2014-08-13 18:06:08.217387 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.61 22.1d2( DNE v 194215'510 lc 0'0 (0'0,194215'510] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(28) 2014-08-13 18:06:08.217403 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 is on osd.4 2014-08-13 18:06:08.217411 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 22.1d2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.217420 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 is on osd.4 2014-08-13 18:06:08.217437 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 is on osd.4 2014-08-13 18:06:08.217453 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 is on osd.4 2014-08-13 18:06:08.217661 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 sending log((0'0,194215'510], crt=191648'372) 2014-08-13 18:06:08.217687 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4fa4fc0 don't know epoch, doing nothing 2014-08-13 18:06:08.217720 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate peer osd.349 22.1d2( DNE v 194215'510 lc 0'0 (0'0,194215'510] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(28) 2014-08-13 18:06:08.217831 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.61 has 28 missing 2014-08-13 18:06:08.217850 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.349 has 28 missing 2014-08-13 18:06:08.217868 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.217887 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 is on osd.4 2014-08-13 18:06:08.217904 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 is on osd.4 2014-08-13 18:06:08.217933 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 is on osd.4 2014-08-13 18:06:08.217955 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 is on osd.4 2014-08-13 18:06:08.217971 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 is on osd.4 2014-08-13 18:06:08.217987 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 is on osd.4 2014-08-13 18:06:08.218014 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 is on osd.4 2014-08-13 18:06:08.218032 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 is on osd.4 2014-08-13 18:06:08.218048 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 is on osd.4 2014-08-13 18:06:08.218064 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 is on osd.4 2014-08-13 18:06:08.218080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 is on osd.4 2014-08-13 18:06:08.218096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 is on osd.4 2014-08-13 18:06:08.218158 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 is on osd.4 2014-08-13 18:06:08.218179 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.218195 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 is on osd.4 2014-08-13 18:06:08.218211 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 is on osd.4 2014-08-13 18:06:08.218227 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.218257 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 is on osd.4 2014-08-13 18:06:08.218273 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 is on osd.4 2014-08-13 18:06:08.218289 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 is on osd.4 2014-08-13 18:06:08.218315 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 is on osd.4 2014-08-13 18:06:08.218332 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 is on osd.4 2014-08-13 18:06:08.218348 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 is on osd.4 2014-08-13 18:06:08.218364 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 is on osd.4 2014-08-13 18:06:08.218395 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 is on osd.4 2014-08-13 18:06:08.218412 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 is on osd.4 2014-08-13 18:06:08.218428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.218444 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 18:06:08.218526 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate: adding 61 as a source 2014-08-13 18:06:08.218547 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 also missing on osd.61 2014-08-13 18:06:08.218570 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 also missing on osd.61 2014-08-13 18:06:08.218588 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 also missing on osd.61 2014-08-13 18:06:08.218605 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 also missing on osd.61 2014-08-13 18:06:08.218634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 also missing on osd.61 2014-08-13 18:06:08.218655 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 also missing on osd.61 2014-08-13 18:06:08.218673 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 also missing on osd.61 2014-08-13 18:06:08.218702 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 also missing on osd.61 2014-08-13 18:06:08.218719 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 also missing on osd.61 2014-08-13 18:06:08.218752 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 also missing on osd.61 2014-08-13 18:06:08.218770 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 also missing on osd.61 2014-08-13 18:06:08.218787 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 also missing on osd.61 2014-08-13 18:06:08.218807 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 also missing on osd.61 2014-08-13 18:06:08.218825 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 also missing on osd.61 2014-08-13 18:06:08.218842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 also missing on osd.61 2014-08-13 18:06:08.218859 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 also missing on osd.61 2014-08-13 18:06:08.218888 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 also missing on osd.61 2014-08-13 18:06:08.218905 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 also missing on osd.61 2014-08-13 18:06:08.218923 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 also missing on osd.61 2014-08-13 18:06:08.218940 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 also missing on osd.61 2014-08-13 18:06:08.218957 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 also missing on osd.61 2014-08-13 18:06:08.218989 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 also missing on osd.61 2014-08-13 18:06:08.219008 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 also missing on osd.61 2014-08-13 18:06:08.219025 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 also missing on osd.61 2014-08-13 18:06:08.219046 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 also missing on osd.61 2014-08-13 18:06:08.219063 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 also missing on osd.61 2014-08-13 18:06:08.219080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 also missing on osd.61 2014-08-13 18:06:08.219097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 also missing on osd.61 2014-08-13 18:06:08.219113 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 18:06:08.219345 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] activate: adding 349 as a source 2014-08-13 18:06:08.219383 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22 191727'388 also missing on osd.349 2014-08-13 18:06:08.219407 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22 191648'265 also missing on osd.349 2014-08-13 18:06:08.219429 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22 191727'377 also missing on osd.349 2014-08-13 18:06:08.219446 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22 191648'208 also missing on osd.349 2014-08-13 18:06:08.219474 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22 191648'332 also missing on osd.349 2014-08-13 18:06:08.219492 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22 191648'140 also missing on osd.349 2014-08-13 18:06:08.219537 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22 191648'176 also missing on osd.349 2014-08-13 18:06:08.217468 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 18:06:08.219555 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22 191727'401 also missing on osd.349 2014-08-13 18:06:08.219579 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22 191648'297 also missing on osd.349 2014-08-13 18:06:08.219574 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate: adding 285 as a source 2014-08-13 18:06:08.219608 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 also missing on osd.285 2014-08-13 18:06:08.219632 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 also missing on osd.285 2014-08-13 18:06:08.219608 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22 191727'378 also missing on osd.349 2014-08-13 18:06:08.219652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 also missing on osd.285 2014-08-13 18:06:08.219660 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22 191727'422 also missing on osd.349 2014-08-13 18:06:08.219683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 also missing on osd.285 2014-08-13 18:06:08.219703 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 also missing on osd.285 2014-08-13 18:06:08.219720 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 also missing on osd.285 2014-08-13 18:06:08.219711 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22 191648'183 also missing on osd.349 2014-08-13 18:06:08.219737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 also missing on osd.285 2014-08-13 18:06:08.219755 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 also missing on osd.285 2014-08-13 18:06:08.219755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22 191727'435 also missing on osd.349 2014-08-13 18:06:08.219774 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 also missing on osd.285 2014-08-13 18:06:08.219780 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22 191648'55 also missing on osd.349 2014-08-13 18:06:08.219792 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 also missing on osd.285 2014-08-13 18:06:08.219810 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 also missing on osd.285 2014-08-13 18:06:08.219799 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22 191727'420 also missing on osd.349 2014-08-13 18:06:08.219827 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 also missing on osd.285 2014-08-13 18:06:08.219845 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 also missing on osd.285 2014-08-13 18:06:08.219833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22 191648'294 also missing on osd.349 2014-08-13 18:06:08.219863 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 also missing on osd.285 2014-08-13 18:06:08.219880 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 also missing on osd.285 2014-08-13 18:06:08.219869 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22 191727'430 also missing on osd.349 2014-08-13 18:06:08.219897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 also missing on osd.285 2014-08-13 18:06:08.219916 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 also missing on osd.285 2014-08-13 18:06:08.219907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22 191727'415 also missing on osd.349 2014-08-13 18:06:08.219934 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 also missing on osd.285 2014-08-13 18:06:08.219937 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22 191727'390 also missing on osd.349 2014-08-13 18:06:08.219952 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 also missing on osd.285 2014-08-13 18:06:08.219955 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22 191727'382 also missing on osd.349 2014-08-13 18:06:08.219970 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 also missing on osd.285 2014-08-13 18:06:08.219979 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22 191648'345 also missing on osd.349 2014-08-13 18:06:08.219989 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 also missing on osd.285 2014-08-13 18:06:08.219998 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22 191727'445 also missing on osd.349 2014-08-13 18:06:08.220008 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 also missing on osd.285 2014-08-13 18:06:08.220026 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 also missing on osd.285 2014-08-13 18:06:08.220025 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22 191648'82 also missing on osd.349 2014-08-13 18:06:08.220044 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 also missing on osd.285 2014-08-13 18:06:08.220050 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22 191727'383 also missing on osd.349 2014-08-13 18:06:08.220062 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 also missing on osd.285 2014-08-13 18:06:08.220069 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22 191648'119 also missing on osd.349 2014-08-13 18:06:08.220080 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 also missing on osd.285 2014-08-13 18:06:08.220087 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22 191648'246 also missing on osd.349 2014-08-13 18:06:08.220099 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 also missing on osd.285 2014-08-13 18:06:08.220116 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 also missing on osd.285 2014-08-13 18:06:08.220109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22 191648'231 also missing on osd.349 2014-08-13 18:06:08.220136 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 also missing on osd.285 2014-08-13 18:06:08.220137 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22 191727'392 also missing on osd.349 2014-08-13 18:06:08.220165 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {118601d2/benchmark_data_storage0111-ib.pocstor_51285_object7722/head//22=191727'388,a85b21d2/benchmark_data_storage0111-ib.pocstor_40745_object16469/head//22=191648'265,a6db61d2/benchmark_data_storage0111-ib.pocstor_51285_object1831/head//22=191727'377,db5f61d2/benchmark_data_storage0111-ib.pocstor_40745_object12898/head//22=191648'208,7d9381d2/benchmark_data_storage0111-ib.pocstor_40745_object20894/head//22=191648'332,f4f581d2/benchmark_data_storage0111-ib.pocstor_40745_object8765/head//22=191648'140,d91981d2/benchmark_data_storage0111-ib.pocstor_40745_object11132/head//22=191648'176,650791d2/benchmark_data_storage0111-ib.pocstor_51285_object15556/head//22=191727'401,7925a1d2/benchmark_data_storage0111-ib.pocstor_40745_object18254/head//22=191648'297,e44b1d2/benchmark_data_storage0111-ib.pocstor_51285_object1836/head//22=191727'378,42baf1d2/benchmark_data_storage0111-ib.pocstor_51285_object26717/head//22=191727'422,c39109d2/benchmark_data_storage0111-ib.pocstor_40745_object11612/head//22=191648'183,8e8d19d2/benchmark_data_storage0111-ib.pocstor_51285_object33451/head//22=191727'435,3d9029d2/benchmark_data_storage0111-ib.pocstor_40745_object3469/head//22=191648'55,957139d2/benchmark_data_storage0111-ib.pocstor_51285_object25854/head//22=191727'420,8af939d2/benchmark_data_storage0111-ib.pocstor_40745_object18081/head//22=191648'294,139e49d2/benchmark_data_storage0111-ib.pocstor_51285_object31401/head//22=191727'430,ded159d2/benchmark_data_storage0111-ib.pocstor_51285_object24054/head//22=191727'415,812c89d2/benchmark_data_storage0111-ib.pocstor_51285_object8993/head//22=191727'390,f790d9d2/benchmark_data_storage0111-ib.pocstor_51285_object5229/head//22=191727'382,d679d9d2/benchmark_data_storage0111-ib.pocstor_40745_object21463/head//22=191648'345,124ad9d2/benchmark_data_storage0111-ib.pocstor_51285_object37147/head//22=191727'445,cacfd9d2/benchmark_data_storage0111-ib.pocstor_40745_object4874/head//22=191648'82,b20fe9d2/benchmark_data_storage0111-ib.pocstor_51285_object5425/head//22=191727'383,79c2f9d2/benchmark_data_storage0111-ib.pocstor_40745_object7270/head//22=191648'119,f313f9d2/benchmark_data_storage0111-ib.pocstor_40745_object15321/head//22=191648'246,5e1ef9d2/benchmark_data_storage0111-ib.pocstor_40745_object14392/head//22=191648'231,48fff9d2/benchmark_data_storage0111-ib.pocstor_51285_object9726/head//22=191727'392} 2014-08-13 18:06:08.220227 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.220244 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.220267 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 58,61,330,331,349 2014-08-13 18:06:08.220285 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.220316 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.220335 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.220350 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.220465 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.220153 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 18:06:08.221339 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] activate: adding 312 as a source 2014-08-13 18:06:08.221371 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22 191727'409 is on osd.312 2014-08-13 18:06:08.221394 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22 191727'380 is on osd.312 2014-08-13 18:06:08.221413 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221418 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221420 7f5c835ef700 20 osd.4 194496 _dispatch 0x5152880 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.221413 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22 191727'344 is on osd.312 2014-08-13 18:06:08.221431 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.221437 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.221440 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221442 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221432 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22 191727'373 is on osd.312 2014-08-13 18:06:08.221463 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221451 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22 191727'377 is on osd.312 2014-08-13 18:06:08.221466 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221468 7f5c835ef700 20 osd.4 194496 _dispatch 0x5153f00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.221475 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.221479 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.221482 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22 191727'360 is on osd.312 2014-08-13 18:06:08.221485 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221488 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22 191727'347 is on osd.312 2014-08-13 18:06:08.221503 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221506 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221509 7f5c835ef700 20 osd.4 194496 _dispatch 0x5154c80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.221515 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.221519 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.221506 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22 191727'388 is on osd.312 2014-08-13 18:06:08.221522 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221524 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221525 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22 191727'357 is on osd.312 2014-08-13 18:06:08.221545 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221548 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221550 7f5c835ef700 20 osd.4 194496 _dispatch 0x51560c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.221544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22 191727'371 is on osd.312 2014-08-13 18:06:08.221556 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.221561 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.221563 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.221566 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.221562 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22 191727'351 is on osd.312 2014-08-13 18:06:08.221581 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22 191727'415 is on osd.312 2014-08-13 18:06:08.221598 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22 191648'159 is on osd.312 2014-08-13 18:06:08.221614 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22 191727'386 is on osd.312 2014-08-13 18:06:08.221631 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22 191727'397 is on osd.312 2014-08-13 18:06:08.221648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22 191727'405 is on osd.312 2014-08-13 18:06:08.221665 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22 191727'389 is on osd.312 2014-08-13 18:06:08.221684 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22 191727'414 is on osd.312 2014-08-13 18:06:08.221701 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22 191727'396 is on osd.312 2014-08-13 18:06:08.221718 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22 191648'332 is on osd.312 2014-08-13 18:06:08.221735 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22 191648'219 is on osd.312 2014-08-13 18:06:08.221752 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22 191727'398 is on osd.312 2014-08-13 18:06:08.221774 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22 191727'378 is on osd.312 2014-08-13 18:06:08.221796 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22 191727'392 is on osd.312 2014-08-13 18:06:08.221813 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22 191648'163 is on osd.312 2014-08-13 18:06:08.221830 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22 191648'326 is on osd.312 2014-08-13 18:06:08.221847 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22 191727'355 is on osd.312 2014-08-13 18:06:08.221864 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22 191727'394 is on osd.312 2014-08-13 18:06:08.221881 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22 191727'353 is on osd.312 2014-08-13 18:06:08.221897 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {9b150f3/benchmark_data_storage0111-ib.pocstor_51285_object35269/head//22=191727'409,a8f850f3/benchmark_data_storage0111-ib.pocstor_51285_object20874/head//22=191727'380,b8bc50f3/benchmark_data_storage0111-ib.pocstor_51285_object771/head//22=191727'344,3ef960f3/benchmark_data_storage0111-ib.pocstor_51285_object15185/head//22=191727'373,c61170f3/benchmark_data_storage0111-ib.pocstor_51285_object18665/head//22=191727'377,d2ab70f3/benchmark_data_storage0111-ib.pocstor_51285_object6317/head//22=191727'360,10c380f3/benchmark_data_storage0111-ib.pocstor_51285_object1538/head//22=191727'347,a281a0f3/benchmark_data_storage0111-ib.pocstor_51285_object23629/head//22=191727'388,42e3a0f3/benchmark_data_storage0111-ib.pocstor_51285_object5813/head//22=191727'357,5fb8a0f3/benchmark_data_storage0111-ib.pocstor_51285_object14780/head//22=191727'371,dc85b0f3/benchmark_data_storage0111-ib.pocstor_51285_object2721/head//22=191727'351,34fb0f3/benchmark_data_storage0111-ib.pocstor_51285_object37254/head//22=191727'415,35b8e0f3/benchmark_data_storage0111-ib.pocstor_40745_object10811/head//22=191648'159,8bf508f3/benchmark_data_storage0111-ib.pocstor_51285_object22621/head//22=191727'386,acfd08f3/benchmark_data_storage0111-ib.pocstor_51285_object27157/head//22=191727'397,f9e018f3/benchmark_data_storage0111-ib.pocstor_51285_object31674/head//22=191727'405,71d318f3/benchmark_data_storage0111-ib.pocstor_51285_object23747/head//22=191727'389,aaf028f3/benchmark_data_storage0111-ib.pocstor_51285_object37143/head//22=191727'414,964958f3/benchmark_data_storage0111-ib.pocstor_51285_object27045/head//22=191727'396,f44588f3/benchmark_data_storage0111-ib.pocstor_40745_object22196/head//22=191648'332,e72788f3/benchmark_data_storage0111-ib.pocstor_40745_object14985/head//22=191648'219,739198f3/benchmark_data_storage0111-ib.pocstor_51285_object27194/head//22=191727'398,aab7b8f3/benchmark_data_storage0111-ib.pocstor_51285_object20664/head//22=191727'378,5428b8f3/benchmark_data_storage0111-ib.pocstor_51285_object24357/head//22=191727'392,652ad8f3/benchmark_data_storage0111-ib.pocstor_40745_object11292/head//22=191648'163,1917e8f3/benchmark_data_storage0111-ib.pocstor_40745_object21657/head//22=191648'326,bebde8f3/benchmark_data_storage0111-ib.pocstor_51285_object5006/head//22=191727'355,21afe8f3/benchmark_data_storage0111-ib.pocstor_51285_object26650/head//22=191727'394,4ffdf8f3/benchmark_data_storage0111-ib.pocstor_51285_object3409/head//22=191727'353} 2014-08-13 18:06:08.221979 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.221995 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.222014 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 285,311,312,369 2014-08-13 18:06:08.222034 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222039 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222031 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.222041 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfe540 osd_map(194488..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222053 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194488,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222059 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222049 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.222062 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222065 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222067 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.222088 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222091 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222093 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc1440 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222083 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.222100 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222105 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222108 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222110 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222136 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222138 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222140 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc3840 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222146 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222150 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222152 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222154 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222187 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222188 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.222190 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222193 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee7500 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222199 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222203 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222205 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222207 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222228 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222230 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222232 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc2d00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222238 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222242 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222248 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222250 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222269 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222271 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222273 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed33c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222279 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222283 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222285 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222287 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222306 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222308 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222310 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc0b40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222316 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222320 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222322 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222324 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222340 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222343 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222345 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed2d00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222350 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222355 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222358 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222361 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222378 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222381 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222383 7f5c835ef700 20 osd.4 194496 _dispatch 0x47ffbc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222388 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222392 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222394 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222396 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222413 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222416 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222418 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee5100 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222423 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222427 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222429 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222431 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222447 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222450 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222452 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed18c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222457 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222461 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222463 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222465 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222482 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222485 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222486 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfe0c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222492 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222496 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222498 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222500 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222520 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222522 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222524 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfec00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222530 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222534 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222536 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222538 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222555 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222557 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222559 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee0d80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222565 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222568 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222571 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222573 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222591 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222593 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222595 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc5100 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222600 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222607 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222611 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222613 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222640 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222642 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222644 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfc140 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222650 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222654 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222656 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222658 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222680 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222682 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222684 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cf8480 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222690 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222694 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222696 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222698 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222715 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222717 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222719 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed60c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222724 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222728 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222730 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222732 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222750 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222752 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222762 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfbcc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222768 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222771 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222773 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222776 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222794 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222797 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222799 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee2d00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222804 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222808 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222810 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222812 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222831 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222834 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222836 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc57c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222841 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222845 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222847 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222849 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222866 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222868 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222870 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfd340 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222875 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222879 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222881 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222883 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222900 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222903 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222905 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee2880 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222911 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222915 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222917 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222920 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222889 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 2014-08-13 18:06:08.222931 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.222938 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222941 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222944 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfb3c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222916 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 18:06:08.222949 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222954 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222956 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222940 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Started advmap 2014-08-13 18:06:08.222955 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.222959 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222960 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,301] 2014-08-13 18:06:08.222977 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.222964 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.222980 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.222983 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed0480 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.222989 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.222978 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.222994 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.222985 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.223001 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223004 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223005 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.223003 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started/Stray 0.944270 4 0.000276 2014-08-13 18:06:08.223025 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223028 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223031 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc4380 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223037 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223041 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223043 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223028 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] exit Started 0.944367 0 0.000000 2014-08-13 18:06:08.223045 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223049 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] enter Reset 2014-08-13 18:06:08.223063 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223066 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223068 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfa1c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223074 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223079 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223081 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223065 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194495 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194496 2014-08-13 18:06:08.223084 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223075 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.223087 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194496 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: Reset advmap 2014-08-13 18:06:08.223104 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223107 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223095 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.223110 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc3f00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223117 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223121 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223106 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194496 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.223124 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223127 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.223137 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.223125 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194496 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] acting_up_affected newup [364,4,360] newacting [364,4,301] 2014-08-13 18:06:08.223146 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.223154 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223157 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223145 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194496 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.223159 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4425280 don't know epoch, doing nothing 2014-08-13 18:06:08.223160 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed1d40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223165 7f5c7bde3700 7 osd.4 194496 do_notify osd 54 on 1 PGs 2014-08-13 18:06:08.223167 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223183 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223185 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223188 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223165 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360] r=1 lpr=194496 pi=193125-194494/38 crt=194422'17074 lcod 0'0 inactive NOTIFY m=48] set_last_peering_reset 194496 2014-08-13 18:06:08.223208 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223210 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223212 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed2880 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223216 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:08.223219 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223221 7f5c7bde3700 7 osd.4 194496 do_notify osd 320 on 1 PGs 2014-08-13 18:06:08.223223 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223226 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223229 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223214 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [364,4,360](364) acting [364,4,360](364)): not rw, up_thru 194492 up_from 192966 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [364,4,360](364) acting [364,4,360](364)) : primary up 192966-194492 does not include interval 2014-08-13 18:06:08.223245 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:08.223247 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223250 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223249 7f5c7bde3700 7 osd.4 194496 do_notify osd 345 on 1 PGs 2014-08-13 18:06:08.223235 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194495/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] noting past interval(194495-194495 up [364,4,360](364) acting [364,4,360](364)) 2014-08-13 18:06:08.223255 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc7980 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223262 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223266 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223269 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223271 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223276 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing 2014-08-13 18:06:08.223256 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] up [364,4,360] -> [364,4,360], acting [364,4,360] -> [364,4,301], acting_primary 364 -> 364, up_primary 364 -> 364, role 1 -> 1 2014-08-13 18:06:08.223281 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.291 on 1 PGs 2014-08-13 18:06:08.223294 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223296 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223284 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:08.223299 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed6780 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223306 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing 2014-08-13 18:06:08.223306 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223311 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223310 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.350 on 1 PGs 2014-08-13 18:06:08.223315 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223303 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.223318 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223334 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4fa2260 don't know epoch, doing nothing 2014-08-13 18:06:08.223321 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:08.223338 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.365 on 1 PGs 2014-08-13 18:06:08.223349 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223352 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223354 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee6300 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223341 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_copy_ops 2014-08-13 18:06:08.223360 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.223360 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223364 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.367 on 1 PGs 2014-08-13 18:06:08.223366 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223369 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223371 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223361 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_flush_ops 2014-08-13 18:06:08.223386 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.223389 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223389 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.378 on 1 PGs 2014-08-13 18:06:08.223392 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change_cleanup 2014-08-13 18:06:08.223395 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee45c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223401 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223406 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223408 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223410 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223397 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] on_change 2014-08-13 18:06:08.223409 7f5c7bde3700 20 osd.4 194496 Sending info 22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) to shard 312 2014-08-13 18:06:08.223421 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4c5aec0 don't know epoch, doing nothing 2014-08-13 18:06:08.223429 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223432 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223418 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit NotTrimming 2014-08-13 18:06:08.223435 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee0480 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223441 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223446 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223448 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223418 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.223451 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223438 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter NotTrimming 2014-08-13 18:06:08.223454 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.223469 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223472 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] cancel_recovery 2014-08-13 18:06:08.223475 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee5a00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223468 7f5c7bde3700 10 osd.4 194496 send_pg_temp {24.b6=[4,17,315],24.136=[4,50,65],24.1bb=[4,332,337],24.1f7=[4,351,55]} 2014-08-13 18:06:08.223482 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223486 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223488 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223490 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223479 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] clear_recovery_state 2014-08-13 18:06:08.223511 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223514 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.223516 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cff080 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223521 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map 2014-08-13 18:06:08.223586 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 18:06:08.223612 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.223623 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] update_heartbeat_peers unchanged 2014-08-13 18:06:08.223619 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.223643 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223654 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223658 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223644 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters 2014-08-13 18:06:08.223661 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223653 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.223664 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Reset 0.000616 1 0.000581 2014-08-13 18:06:08.223674 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.223688 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223691 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223694 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc3180 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223701 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223705 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223692 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started 2014-08-13 18:06:08.223708 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223710 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223711 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Start 2014-08-13 18:06:08.223731 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223734 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223736 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cff740 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223723 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.223743 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223728 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: transitioning to Stray 2014-08-13 18:06:08.223748 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223751 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223753 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223742 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.223753 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Start 0.000042 0 0.000000 2014-08-13 18:06:08.223771 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223775 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223762 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.223777 7f5c835ef700 20 osd.4 194496 _dispatch 0x51557c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223781 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.223784 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223788 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223791 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223777 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/Stray 2014-08-13 18:06:08.223793 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223812 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223815 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223801 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.223817 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed45c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223824 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223800 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.223807 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 18:06:08.223829 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223832 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223834 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223833 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.223821 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.223832 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.223854 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223857 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223843 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.223860 7f5c835ef700 20 osd.4 194496 _dispatch 0x5156780 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223866 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223871 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223873 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223876 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223865 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.223893 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223896 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223884 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.223898 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cf8b40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223904 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223909 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223911 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223913 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.223930 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223920 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.223933 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223934 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.223938 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc21c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223944 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223948 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223950 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223953 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223939 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.223971 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223958 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2708 2014-08-13 18:06:08.223974 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223976 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed7500 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.223982 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.223986 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.223974 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.223989 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.223995 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.223985 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:08.223997 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.811080 6 0.000413 2014-08-13 18:06:08.224015 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224019 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224017 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.224021 7f5c835ef700 20 osd.4 194496 _dispatch 0x51518c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224028 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224032 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224035 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224019 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.224025 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.224039 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224040 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.914794 0 0.000000 2014-08-13 18:06:08.224045 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.224071 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224073 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224062 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.224076 7f5c835ef700 20 osd.4 194496 _dispatch 0x5151200 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224065 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.224083 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224089 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224091 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224093 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224082 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.224083 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.224111 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224114 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224103 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.224117 7f5c835ef700 20 osd.4 194496 _dispatch 0x5151f80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224108 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.224123 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224128 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224131 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224134 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224122 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.224126 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.224152 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224156 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224142 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:176856 2014-08-13 18:06:08.224159 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc0480 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224161 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.224166 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224184 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224186 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224189 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224143 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 18:06:08.224188 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.811591 3 0.000285 2014-08-13 18:06:08.224209 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224212 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224214 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfcc80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224221 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224210 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.224226 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224229 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224232 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224213 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.224227 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.932234 0 0.000000 2014-08-13 18:06:08.224250 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224253 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224255 7f5c835ef700 20 osd.4 194496 _dispatch 0x51545c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224261 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224252 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.224265 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224272 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224274 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224271 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.224292 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224294 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224297 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee4c80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224303 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224307 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224294 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.224309 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224311 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224311 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.224325 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'930 -> 194215'930 2014-08-13 18:06:08.224349 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224352 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224354 7f5c835ef700 20 osd.4 194496 _dispatch 0x5155340 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224360 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224363 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224366 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224368 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224385 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224387 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224389 7f5c835ef700 20 osd.4 194496 _dispatch 0x47fa1c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224394 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224398 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224400 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224402 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224421 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224424 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224407 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 2.2a( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.224426 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc2640 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224432 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224446 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224406 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] flushed 2014-08-13 18:06:08.224448 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224450 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224476 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224478 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224480 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc4a40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224486 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224490 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224492 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224494 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224511 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224514 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224517 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cf8fc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 sending log((0'0,194215'518], crt=191727'442) 2014-08-13 18:06:08.224523 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224527 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224526 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4494360 don't know epoch, doing nothing 2014-08-13 18:06:08.224529 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224532 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224555 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224557 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224559 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed7980 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224565 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224569 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224572 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224574 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224558 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.313 22.3e4( DNE v 194215'518 lc 0'0 (0'0,194215'518] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 18:06:08.224591 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224594 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224597 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cf9b00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224582 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 22.3e4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.224602 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224606 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224608 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224610 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224629 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224632 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224634 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfc5c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224639 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224643 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224645 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224647 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224671 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224674 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224675 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfda00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224681 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224685 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224687 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224689 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224706 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224708 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224710 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee0900 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224715 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224719 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224722 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224724 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224740 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224743 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224744 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed21c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224750 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224754 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224756 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224758 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224774 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224776 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224778 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed0b40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224784 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224788 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224790 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224792 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224809 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224811 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224814 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed5c40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224821 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224826 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224829 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224832 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224852 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224836 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 sending log((0'0,194215'518], crt=191727'442) 2014-08-13 18:06:08.224855 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224857 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed5580 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.224859 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3988dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.224863 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.224867 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.224870 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224872 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224891 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224894 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224896 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888380 pg_notify(24.30b(45) epoch 194496) v5 2014-08-13 18:06:08.224885 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.318 22.3e4( DNE v 194215'518 lc 0'0 (0'0,194215'518] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 18:06:08.224940 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.375 2014-08-13 18:06:08.224944 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888380 2014-08-13 18:06:08.224992 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.224995 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.224988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.313 has 32 missing 2014-08-13 18:06:08.225004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.318 has 32 missing 2014-08-13 18:06:08.225019 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 is on osd.4 2014-08-13 18:06:08.225043 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225046 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225048 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1f80 pg_notify(22.22c(1) epoch 194496) v5 2014-08-13 18:06:08.225037 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.225056 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 is on osd.4 2014-08-13 18:06:08.225073 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 is on osd.4 2014-08-13 18:06:08.225087 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.302 2014-08-13 18:06:08.225093 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1f80 2014-08-13 18:06:08.225091 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.225109 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.225131 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225134 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225125 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 is on osd.4 2014-08-13 18:06:08.225153 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225142 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 is on osd.4 2014-08-13 18:06:08.225156 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225158 7f5c835ef700 20 osd.4 194496 _dispatch 0x5425640 pg_query(24.e2 epoch 194496) v3 2014-08-13 18:06:08.225159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 is on osd.4 2014-08-13 18:06:08.225176 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.225205 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.346 epoch 194496 2014-08-13 18:06:08.225209 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5425640 2014-08-13 18:06:08.225200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 is on osd.4 2014-08-13 18:06:08.225220 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 is on osd.4 2014-08-13 18:06:08.225222 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_query query(info 0'0) from replica 346 2014-08-13 18:06:08.225238 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 is on osd.4 2014-08-13 18:06:08.225255 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 is on osd.4 2014-08-13 18:06:08.225270 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225273 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225271 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.225296 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225298 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225288 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 is on osd.4 2014-08-13 18:06:08.225301 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5080 pg_notify(22.1d2(30) epoch 194496) v5 2014-08-13 18:06:08.225306 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.225324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 is on osd.4 2014-08-13 18:06:08.225340 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.330 2014-08-13 18:06:08.225344 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5080 2014-08-13 18:06:08.225341 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 is on osd.4 2014-08-13 18:06:08.225358 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 is on osd.4 2014-08-13 18:06:08.225381 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225384 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225375 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 is on osd.4 2014-08-13 18:06:08.225392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 is on osd.4 2014-08-13 18:06:08.225409 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225412 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225408 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 is on osd.4 2014-08-13 18:06:08.225415 7f5c835ef700 20 osd.4 194496 _dispatch 0x588d780 pg_notify(24.9b(2) epoch 194496) v5 2014-08-13 18:06:08.225425 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.225442 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 is on osd.4 2014-08-13 18:06:08.225461 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.365 2014-08-13 18:06:08.225465 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588d780 2014-08-13 18:06:08.225460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 is on osd.4 2014-08-13 18:06:08.225477 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 is on osd.4 2014-08-13 18:06:08.225501 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225504 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225494 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 is on osd.4 2014-08-13 18:06:08.225523 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225511 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 is on osd.4 2014-08-13 18:06:08.225526 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225528 7f5c835ef700 20 osd.4 194496 _dispatch 0x5151b00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.225536 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.225541 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.225527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 is on osd.4 2014-08-13 18:06:08.225543 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225546 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 is on osd.4 2014-08-13 18:06:08.225565 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225568 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225570 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4440 pg_notify(22.39d(1) epoch 194496) v5 2014-08-13 18:06:08.225561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 is on osd.4 2014-08-13 18:06:08.225606 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.366 2014-08-13 18:06:08.225610 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4440 2014-08-13 18:06:08.225579 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 18:06:08.225644 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225648 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225646 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate: adding 313 as a source 2014-08-13 18:06:08.225666 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225669 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225671 7f5c835ef700 20 osd.4 194496 _dispatch 0x5153180 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.225678 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.225666 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 also missing on osd.313 2014-08-13 18:06:08.225683 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.225685 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225687 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225705 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 also missing on osd.313 2014-08-13 18:06:08.225708 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225710 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d080 pg_notify(22.775(36) epoch 194496) v5 2014-08-13 18:06:08.225711 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 also missing on osd.313 2014-08-13 18:06:08.225729 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 also missing on osd.313 2014-08-13 18:06:08.225749 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.63 2014-08-13 18:06:08.225753 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d080 2014-08-13 18:06:08.225747 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 also missing on osd.313 2014-08-13 18:06:08.225766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 also missing on osd.313 2014-08-13 18:06:08.225789 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225792 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225784 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 also missing on osd.313 2014-08-13 18:06:08.225802 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 also missing on osd.313 2014-08-13 18:06:08.225821 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225824 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225827 7f5c835ef700 20 osd.4 194496 _dispatch 0x588e040 pg_notify(22.362(34) epoch 194496) v5 2014-08-13 18:06:08.225819 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 also missing on osd.313 2014-08-13 18:06:08.225838 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 also missing on osd.313 2014-08-13 18:06:08.225864 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.349 2014-08-13 18:06:08.225867 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588e040 2014-08-13 18:06:08.225857 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 also missing on osd.313 2014-08-13 18:06:08.225875 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 also missing on osd.313 2014-08-13 18:06:08.225903 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225906 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225893 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 also missing on osd.313 2014-08-13 18:06:08.225911 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 also missing on osd.313 2014-08-13 18:06:08.225935 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.225938 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.225940 7f5c835ef700 20 osd.4 194496 _dispatch 0x588db00 pg_notify(22.3d9(54) epoch 194496) v5 2014-08-13 18:06:08.225929 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 also missing on osd.313 2014-08-13 18:06:08.225947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 also missing on osd.313 2014-08-13 18:06:08.225977 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.382 2014-08-13 18:06:08.225966 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 also missing on osd.313 2014-08-13 18:06:08.225980 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588db00 2014-08-13 18:06:08.225984 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 also missing on osd.313 2014-08-13 18:06:08.226002 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 also missing on osd.313 2014-08-13 18:06:08.226016 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226020 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 also missing on osd.313 2014-08-13 18:06:08.226037 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 also missing on osd.313 2014-08-13 18:06:08.226057 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226060 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226062 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b2c0 pg_notify(24.34(41) epoch 194496) v5 2014-08-13 18:06:08.226054 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 also missing on osd.313 2014-08-13 18:06:08.226072 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 also missing on osd.313 2014-08-13 18:06:08.226099 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.26 2014-08-13 18:06:08.226102 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b2c0 2014-08-13 18:06:08.226090 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 also missing on osd.313 2014-08-13 18:06:08.226108 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 also missing on osd.313 2014-08-13 18:06:08.226126 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 also missing on osd.313 2014-08-13 18:06:08.226139 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226142 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226144 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 also missing on osd.313 2014-08-13 18:06:08.226161 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 also missing on osd.313 2014-08-13 18:06:08.226181 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226184 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226186 7f5c835ef700 20 osd.4 194496 _dispatch 0x53571c0 pg_notify(22.3d5(2) epoch 194496) v5 2014-08-13 18:06:08.226178 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 also missing on osd.313 2014-08-13 18:06:08.226208 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 also missing on osd.313 2014-08-13 18:06:08.226227 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.10 2014-08-13 18:06:08.226230 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53571c0 2014-08-13 18:06:08.226228 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 also missing on osd.313 2014-08-13 18:06:08.226247 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 also missing on osd.313 2014-08-13 18:06:08.226280 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226283 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226307 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226310 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226312 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b100 pg_notify(22.84(2) epoch 194496) v5 2014-08-13 18:06:08.226263 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 18:06:08.226323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate: adding 318 as a source 2014-08-13 18:06:08.226349 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.28 2014-08-13 18:06:08.226352 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b100 2014-08-13 18:06:08.226343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22 191727'379 also missing on osd.318 2014-08-13 18:06:08.226361 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22 191727'437 also missing on osd.318 2014-08-13 18:06:08.226390 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226393 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226379 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22 191727'401 also missing on osd.318 2014-08-13 18:06:08.226398 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22 191727'439 also missing on osd.318 2014-08-13 18:06:08.226416 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226420 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226422 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b800 pg_notify(24.30b(2) epoch 194496) v5 2014-08-13 18:06:08.226415 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22 191727'399 also missing on osd.318 2014-08-13 18:06:08.226433 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22 191727'420 also missing on osd.318 2014-08-13 18:06:08.226459 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.301 2014-08-13 18:06:08.226462 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b800 2014-08-13 18:06:08.226452 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22 191727'380 also missing on osd.318 2014-08-13 18:06:08.226470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22 191727'393 also missing on osd.318 2014-08-13 18:06:08.226489 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22 191727'427 also missing on osd.318 2014-08-13 18:06:08.226503 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226506 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226506 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22 191727'414 also missing on osd.318 2014-08-13 18:06:08.226535 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226524 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22 191648'295 also missing on osd.318 2014-08-13 18:06:08.226538 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226541 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a43480 pg_info(1 pgs e194496:22.14a(2)) v4 2014-08-13 18:06:08.226541 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22 192674'450 also missing on osd.318 2014-08-13 18:06:08.226559 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22 191648'215 also missing on osd.318 2014-08-13 18:06:08.226586 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.14a(2)) v4 from osd.58 2014-08-13 18:06:08.226568 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 sending log((0'0,194215'930], crt=0'0) 2014-08-13 18:06:08.226578 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22 191648'118 also missing on osd.318 2014-08-13 18:06:08.226592 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a43480 2014-08-13 18:06:08.226599 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.226597 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22 191727'406 also missing on osd.318 2014-08-13 18:06:08.226616 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22 191727'419 also missing on osd.318 2014-08-13 18:06:08.226634 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226638 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22 191727'371 also missing on osd.318 2014-08-13 18:06:08.226634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 2.2a( DNE v 194215'930 (0'0,194215'930] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) uptodate 2014-08-13 18:06:08.226659 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226662 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226665 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351a40 pg_info(1 pgs e194496:22.28f(2)) v4 2014-08-13 18:06:08.226654 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22 191727'370 also missing on osd.318 2014-08-13 18:06:08.226660 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.226674 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22 191648'261 also missing on osd.318 2014-08-13 18:06:08.226678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.226693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22 191727'387 also missing on osd.318 2014-08-13 18:06:08.226696 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.226708 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.28f(2)) v4 from osd.9 2014-08-13 18:06:08.226717 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351a40 2014-08-13 18:06:08.226712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22 191648'269 also missing on osd.318 2014-08-13 18:06:08.226718 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.226731 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22 191648'268 also missing on osd.318 2014-08-13 18:06:08.226736 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.226754 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226758 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226751 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22 191727'402 also missing on osd.318 2014-08-13 18:06:08.226778 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226781 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226770 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22 191727'392 also missing on osd.318 2014-08-13 18:06:08.226783 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df39c0 pg_info(1 pgs e194496:22.29a(2)) v4 2014-08-13 18:06:08.226800 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.226787 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22 191727'442 also missing on osd.318 2014-08-13 18:06:08.226806 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22 191727'375 also missing on osd.318 2014-08-13 18:06:08.226822 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.29a(2)) v4 from osd.345 2014-08-13 18:06:08.226826 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df39c0 2014-08-13 18:06:08.226825 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22 191727'412 also missing on osd.318 2014-08-13 18:06:08.226843 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22 191648'20 also missing on osd.318 2014-08-13 18:06:08.226862 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226866 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226861 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22 191648'158 also missing on osd.318 2014-08-13 18:06:08.226884 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226887 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226889 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a45cc0 pg_info(1 pgs e194496:22.30f(2)) v4 2014-08-13 18:06:08.226879 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22 191648'13 also missing on osd.318 2014-08-13 18:06:08.226898 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22 191648'196 also missing on osd.318 2014-08-13 18:06:08.226926 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.30f(2)) v4 from osd.54 2014-08-13 18:06:08.226916 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] search_for_missing 75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22 191727'390 also missing on osd.318 2014-08-13 18:06:08.226931 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a45cc0 2014-08-13 18:06:08.226965 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226968 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226987 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.226990 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.226932 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {b35803e4/benchmark_data_storage0111-ib.pocstor_51285_object3617/head//22=191727'379,4b6133e4/benchmark_data_storage0111-ib.pocstor_51285_object37691/head//22=191727'437,fb7243e4/benchmark_data_storage0111-ib.pocstor_51285_object13470/head//22=191727'401,1d2053e4/benchmark_data_storage0111-ib.pocstor_51285_object38661/head//22=191727'439,c3d553e4/benchmark_data_storage0111-ib.pocstor_51285_object12239/head//22=191727'399,fda653e4/benchmark_data_storage0111-ib.pocstor_51285_object24194/head//22=191727'420,e89563e4/benchmark_data_storage0111-ib.pocstor_51285_object3870/head//22=191727'380,7c273e4/benchmark_data_storage0111-ib.pocstor_51285_object9116/head//22=191727'393,b1b673e4/benchmark_data_storage0111-ib.pocstor_51285_object31109/head//22=191727'427,520e83e4/benchmark_data_storage0111-ib.pocstor_51285_object21789/head//22=191727'414,1bc93e4/benchmark_data_storage0111-ib.pocstor_40745_object18816/head//22=191648'295,fc93e4/rb.0.43b76.238e1f29.0000000000ef/head//22=192674'450,3fb4a3e4/benchmark_data_storage0111-ib.pocstor_40745_object13102/head//22=191648'215,453ea3e4/benchmark_data_storage0111-ib.pocstor_40745_object7407/head//22=191648'118,9490b3e4/benchmark_data_storage0111-ib.pocstor_51285_object14900/head//22=191727'406,e128d3e4/benchmark_data_storage0111-ib.pocstor_51285_object23146/head//22=191727'419,6f620be4/benchmark_data_storage0111-ib.pocstor_51285_object849/head//22=191727'371,e7262be4/benchmark_data_storage0111-ib.pocstor_51285_object170/head//22=191727'370,f9f14be4/benchmark_data_storage0111-ib.pocstor_40745_object16499/head//22=191648'261,73a16be4/benchmark_data_storage0111-ib.pocstor_51285_object7583/head//22=191727'387,e86c6be4/benchmark_data_storage0111-ib.pocstor_40745_object16909/head//22=191648'269,26a28be4/benchmark_data_storage0111-ib.pocstor_40745_object16896/head//22=191648'268,8838be4/benchmark_data_storage0111-ib.pocstor_51285_object13495/head//22=191727'402,81a69be4/benchmark_data_storage0111-ib.pocstor_51285_object8511/head//22=191727'392,a607abe4/benchmark_data_storage0111-ib.pocstor_51285_object38711/head//22=191727'442,a2d7abe4/benchmark_data_storage0111-ib.pocstor_51285_object2334/head//22=191727'375,9d08bbe4/benchmark_data_storage0111-ib.pocstor_51285_object18557/head//22=191727'412,c289bbe4/benchmark_data_storage0111-ib.pocstor_40745_object1373/head//22=191648'20,e66fbbe4/benchmark_data_storage0111-ib.pocstor_40745_object9201/head//22=191648'158,ebcfcbe4/benchmark_data_storage0111-ib.pocstor_40745_object659/head//22=191648'13,1987ebe4/benchmark_data_storage0111-ib.pocstor_40745_object11827/head//22=191648'196,75a9fbe4/benchmark_data_storage0111-ib.pocstor_51285_object8184/head//22=191727'390} 2014-08-13 18:06:08.226992 7f5c835ef700 20 osd.4 194496 _dispatch 0x588eac0 pg_notify(24.3d(2),22.99(30) epoch 194496) v5 2014-08-13 18:06:08.226993 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.227009 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.227036 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.20 2014-08-13 18:06:08.227039 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588eac0 2014-08-13 18:06:08.227028 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 45,303,313,318,342 2014-08-13 18:06:08.227047 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.227065 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.227088 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227091 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227083 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.227099 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.227116 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227119 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227133 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5780 pg_info(1 pgs e194496:22.4aa(2)) v4 2014-08-13 18:06:08.227178 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4aa(2)) v4 from osd.321 2014-08-13 18:06:08.227183 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5780 2014-08-13 18:06:08.227214 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.227218 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227222 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227241 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227244 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227246 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889f80 pg_notify(24.34(54) epoch 194496) v5 2014-08-13 18:06:08.227280 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.30 2014-08-13 18:06:08.227283 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889f80 2014-08-13 18:06:08.227320 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227322 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227370 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227373 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227375 7f5c835ef700 20 osd.4 194496 _dispatch 0x21afda0 pg_query(24.1e6 epoch 194496) v3 2014-08-13 18:06:08.227409 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.285 epoch 194496 2014-08-13 18:06:08.227412 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21afda0 2014-08-13 18:06:08.227424 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_query query(info 0'0) from replica 285 2014-08-13 18:06:08.227468 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227471 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227489 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227492 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227494 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a68540 pg_notify(22.3af(1) epoch 194496) v5 2014-08-13 18:06:08.227528 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.61 2014-08-13 18:06:08.227531 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a68540 2014-08-13 18:06:08.227573 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227576 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227595 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227597 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227599 7f5c835ef700 20 osd.4 194496 _dispatch 0x21aa3a0 pg_query(24.1de epoch 194496) v3 2014-08-13 18:06:08.227632 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.364 epoch 194496 2014-08-13 18:06:08.227635 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21aa3a0 2014-08-13 18:06:08.227647 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_query query(info 0'0) from replica 364 2014-08-13 18:06:08.227694 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227696 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227715 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227717 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227720 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a42d80 pg_notify(22.6bd(28) epoch 194496) v5 2014-08-13 18:06:08.227755 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.289 2014-08-13 18:06:08.227758 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a42d80 2014-08-13 18:06:08.227791 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227794 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227825 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.227828 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.227830 7f5c835ef700 20 osd.4 194496 _dispatch 0x524f1c0 pg_notify(22.3e4(36) epoch 194496) v5 2014-08-13 18:06:08.227867 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.45 2014-08-13 18:06:08.227869 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524f1c0 2014-08-13 18:06:08.229634 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 18:06:08.229668 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.229678 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.229683 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.229657 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 2014-08-13 18:06:08.229676 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.229698 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.229695 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.229706 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Started advmap 2014-08-13 18:06:08.229723 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.229713 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.229731 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.229734 7f5c835ef700 20 osd.4 194496 _dispatch 0x54223a0 pg_query(24.2a1 epoch 194496) v3 2014-08-13 18:06:08.229729 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,4,395] 2014-08-13 18:06:08.229737 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.229748 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.229774 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.17 epoch 194496 2014-08-13 18:06:08.229758 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.229778 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x54223a0 2014-08-13 18:06:08.229771 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started/Stray 0.907726 4 0.000334 2014-08-13 18:06:08.229781 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.229795 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] exit Started 0.907837 0 0.000000 2014-08-13 18:06:08.229800 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2936 2014-08-13 18:06:08.229817 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.229816 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] enter Reset 2014-08-13 18:06:08.229834 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194495 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194496 2014-08-13 18:06:08.229836 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.816988 6 0.000404 2014-08-13 18:06:08.229853 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194496 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: Reset advmap 2014-08-13 18:06:08.229856 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.229870 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194496 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.229873 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.934375 0 0.000000 2014-08-13 18:06:08.229886 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194496 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] acting_up_affected newup [17,4,395] newacting [17,4,395] 2014-08-13 18:06:08.229897 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.229905 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194496 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.229915 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194486 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.229923 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395]/[17,395] r=-1 lpr=194496 pi=193188-194494/41 crt=194422'14353 lcod 0'0 remapped NOTIFY m=49] set_last_peering_reset 194496 2014-08-13 18:06:08.229937 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.229953 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.229958 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [17,4,395](17) acting [17,395](17)): not rw, up_thru 194492 up_from 192970 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [17,4,395](17) acting [17,395](17)) : primary up 192970-194492 does not include interval 2014-08-13 18:06:08.229969 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'528 -> 194215'528 2014-08-13 18:06:08.229977 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194495/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] noting past interval(194495-194495 up [17,4,395](17) acting [17,395](17)) 2014-08-13 18:06:08.229994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.229996 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] up [17,4,395] -> [17,4,395], acting [17,395] -> [17,4,395], acting_primary 17 -> 17, up_primary 17 -> 17, role -1 -> 1 2014-08-13 18:06:08.230017 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:08.230033 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.230047 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:08.230065 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_copy_ops 2014-08-13 18:06:08.230079 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_flush_ops 2014-08-13 18:06:08.230095 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change_cleanup 2014-08-13 18:06:08.230110 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_change 2014-08-13 18:06:08.230127 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit NotTrimming 2014-08-13 18:06:08.230144 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter NotTrimming 2014-08-13 18:06:08.230159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] on_role_change 2014-08-13 18:06:08.230174 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:08.230187 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] cancel_recovery 2014-08-13 18:06:08.230212 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] clear_recovery_state 2014-08-13 18:06:08.230229 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.230248 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:08.230245 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 sending log((0'0,194215'528], crt=191648'383) 2014-08-13 18:06:08.230275 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:08.230255 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.230290 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.230305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.45 22.113( DNE v 194215'528 lc 0'0 (0'0,194215'528] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(27) 2014-08-13 18:06:08.230329 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 22.113( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.230343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] update_heartbeat_peers unchanged 2014-08-13 18:06:08.230361 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:08.230377 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Reset 0.000562 1 0.000542 2014-08-13 18:06:08.230397 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started 2014-08-13 18:06:08.230413 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Start 2014-08-13 18:06:08.230427 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: transitioning to Stray 2014-08-13 18:06:08.230446 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Start 0.000034 0 0.000000 2014-08-13 18:06:08.230465 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/Stray 2014-08-13 18:06:08.230485 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.230501 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.230581 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 sending log((0'0,194215'528], crt=191648'383) 2014-08-13 18:06:08.230602 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.230605 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c594a0 don't know epoch, doing nothing 2014-08-13 18:06:08.230637 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate peer osd.339 22.113( DNE v 194215'528 lc 0'0 (0'0,194215'528] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(27) 2014-08-13 18:06:08.230709 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 18:06:08.230726 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.45 has 27 missing 2014-08-13 18:06:08.230739 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.230744 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.339 has 27 missing 2014-08-13 18:06:08.230746 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.230759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 is on osd.4 2014-08-13 18:06:08.230766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.230754 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(info 0'0) from replica 17 2014-08-13 18:06:08.230780 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 is on osd.4 2014-08-13 18:06:08.230784 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.230798 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 is on osd.4 2014-08-13 18:06:08.230815 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 is on osd.4 2014-08-13 18:06:08.230832 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.230837 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.230831 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 is on osd.4 2014-08-13 18:06:08.230848 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 is on osd.4 2014-08-13 18:06:08.230850 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.230866 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 is on osd.4 2014-08-13 18:06:08.230880 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.230883 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.230887 7f5c835ef700 20 osd.4 194496 _dispatch 0x588c440 pg_notify(22.619(1) epoch 194496) v5 2014-08-13 18:06:08.230868 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.230883 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 is on osd.4 2014-08-13 18:06:08.230896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.230921 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.230907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 is on osd.4 2014-08-13 18:06:08.230932 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.44 2014-08-13 18:06:08.230936 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588c440 2014-08-13 18:06:08.230926 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 is on osd.4 2014-08-13 18:06:08.230944 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 is on osd.4 2014-08-13 18:06:08.230946 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 18:06:08.230971 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.230962 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 is on osd.4 2014-08-13 18:06:08.230985 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.230989 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.230978 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.230980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 is on osd.4 2014-08-13 18:06:08.230996 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.231000 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 is on osd.4 2014-08-13 18:06:08.231020 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231024 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231013 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.231026 7f5c835ef700 20 osd.4 194496 _dispatch 0x5355400 pg_info(1 pgs e194496:22.2d1(2)) v4 2014-08-13 18:06:08.231019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.231033 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.231054 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.231070 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2d1(2)) v4 from osd.358 2014-08-13 18:06:08.231076 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5355400 2014-08-13 18:06:08.231037 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 is on osd.4 2014-08-13 18:06:08.231078 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.231087 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 is on osd.4 2014-08-13 18:06:08.231099 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2572 2014-08-13 18:06:08.231108 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 is on osd.4 2014-08-13 18:06:08.231124 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231127 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231118 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.231127 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 is on osd.4 2014-08-13 18:06:08.231137 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.816768 4 0.000340 2014-08-13 18:06:08.231156 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231159 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231146 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 is on osd.4 2014-08-13 18:06:08.231162 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357540 pg_info(1 pgs e194496:24.3bf(11)) v4 2014-08-13 18:06:08.231159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.231178 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.898183 0 0.000000 2014-08-13 18:06:08.231166 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.231215 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:24.3bf(11)) v4 from osd.19 2014-08-13 18:06:08.231220 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357540 2014-08-13 18:06:08.231207 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 is on osd.4 2014-08-13 18:06:08.231186 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.231211 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.231226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 is on osd.4 2014-08-13 18:06:08.231230 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.231233 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194486 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.231267 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231271 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231258 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.231247 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 is on osd.4 2014-08-13 18:06:08.231261 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] flushed 2014-08-13 18:06:08.231275 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.231278 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 is on osd.4 2014-08-13 18:06:08.231296 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231299 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231302 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3f00 pg_notify(24.136(36) epoch 194496) v5 2014-08-13 18:06:08.231293 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'483 -> 194215'483 2014-08-13 18:06:08.231297 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 is on osd.4 2014-08-13 18:06:08.231317 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 is on osd.4 2014-08-13 18:06:08.231317 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.231346 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.356 2014-08-13 18:06:08.231349 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3f00 2014-08-13 18:06:08.231337 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 18:06:08.231400 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231403 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231395 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate: adding 45 as a source 2014-08-13 18:06:08.231425 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 also missing on osd.45 2014-08-13 18:06:08.231439 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.231446 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.231448 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf4440 pg_notify(22.113(34) epoch 194496) v5 2014-08-13 18:06:08.231446 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 also missing on osd.45 2014-08-13 18:06:08.231465 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 also missing on osd.45 2014-08-13 18:06:08.231489 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.55 2014-08-13 18:06:08.231492 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf4440 2014-08-13 18:06:08.231484 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 also missing on osd.45 2014-08-13 18:06:08.231502 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 also missing on osd.45 2014-08-13 18:06:08.231519 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 also missing on osd.45 2014-08-13 18:06:08.231536 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 also missing on osd.45 2014-08-13 18:06:08.231553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 also missing on osd.45 2014-08-13 18:06:08.231551 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 sending log((0'0,194215'483], crt=191727'417) 2014-08-13 18:06:08.231579 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3a689a0 don't know epoch, doing nothing 2014-08-13 18:06:08.231572 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 also missing on osd.45 2014-08-13 18:06:08.231591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 also missing on osd.45 2014-08-13 18:06:08.231608 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 also missing on osd.45 2014-08-13 18:06:08.231614 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.21 22.707( DNE v 194215'483 lc 0'0 (0'0,194215'483] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(19) 2014-08-13 18:06:08.231626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 also missing on osd.45 2014-08-13 18:06:08.231639 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 22.707( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.231644 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 also missing on osd.45 2014-08-13 18:06:08.231663 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 also missing on osd.45 2014-08-13 18:06:08.231679 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 also missing on osd.45 2014-08-13 18:06:08.231696 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 also missing on osd.45 2014-08-13 18:06:08.231713 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 also missing on osd.45 2014-08-13 18:06:08.231730 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 also missing on osd.45 2014-08-13 18:06:08.231747 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 also missing on osd.45 2014-08-13 18:06:08.231763 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 also missing on osd.45 2014-08-13 18:06:08.231780 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 also missing on osd.45 2014-08-13 18:06:08.231797 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 also missing on osd.45 2014-08-13 18:06:08.231813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 also missing on osd.45 2014-08-13 18:06:08.231830 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 also missing on osd.45 2014-08-13 18:06:08.231839 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 sending log((0'0,194215'483], crt=191727'417) 2014-08-13 18:06:08.231848 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 also missing on osd.45 2014-08-13 18:06:08.231864 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4fa1fa0 don't know epoch, doing nothing 2014-08-13 18:06:08.231866 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 also missing on osd.45 2014-08-13 18:06:08.231885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 also missing on osd.45 2014-08-13 18:06:08.231892 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate peer osd.310 22.707( DNE v 194215'483 lc 0'0 (0'0,194215'483] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(19) 2014-08-13 18:06:08.231901 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 18:06:08.231948 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] activate: adding 339 as a source 2014-08-13 18:06:08.231965 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22 191648'77 also missing on osd.339 2014-08-13 18:06:08.231967 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.21 has 19 missing 2014-08-13 18:06:08.231983 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22 191727'424 also missing on osd.339 2014-08-13 18:06:08.231985 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.310 has 19 missing 2014-08-13 18:06:08.232001 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22 192674'470 also missing on osd.339 2014-08-13 18:06:08.232001 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.232020 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22 191727'426 also missing on osd.339 2014-08-13 18:06:08.232022 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 is on osd.4 2014-08-13 18:06:08.232039 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22 191727'453 also missing on osd.339 2014-08-13 18:06:08.232040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 is on osd.4 2014-08-13 18:06:08.232057 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22 191727'385 also missing on osd.339 2014-08-13 18:06:08.232058 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 is on osd.4 2014-08-13 18:06:08.232076 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22 191727'422 also missing on osd.339 2014-08-13 18:06:08.232077 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 is on osd.4 2014-08-13 18:06:08.232095 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 is on osd.4 2014-08-13 18:06:08.232096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22 191727'439 also missing on osd.339 2014-08-13 18:06:08.232114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 is on osd.4 2014-08-13 18:06:08.232115 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22 191727'431 also missing on osd.339 2014-08-13 18:06:08.232134 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22 191648'177 also missing on osd.339 2014-08-13 18:06:08.232135 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 is on osd.4 2014-08-13 18:06:08.232153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 is on osd.4 2014-08-13 18:06:08.232153 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22 191648'198 also missing on osd.339 2014-08-13 18:06:08.232174 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22 191648'168 also missing on osd.339 2014-08-13 18:06:08.232199 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22 191727'418 also missing on osd.339 2014-08-13 18:06:08.232216 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22 191648'330 also missing on osd.339 2014-08-13 18:06:08.232173 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 is on osd.4 2014-08-13 18:06:08.232233 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22 191727'406 also missing on osd.339 2014-08-13 18:06:08.232234 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 is on osd.4 2014-08-13 18:06:08.232251 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22 191727'436 also missing on osd.339 2014-08-13 18:06:08.232254 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 is on osd.4 2014-08-13 18:06:08.232269 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22 191727'462 also missing on osd.339 2014-08-13 18:06:08.232271 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.232287 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22 191727'443 also missing on osd.339 2014-08-13 18:06:08.232289 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 is on osd.4 2014-08-13 18:06:08.232306 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22 191727'404 also missing on osd.339 2014-08-13 18:06:08.232307 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 is on osd.4 2014-08-13 18:06:08.232325 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22 191648'105 also missing on osd.339 2014-08-13 18:06:08.232326 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 is on osd.4 2014-08-13 18:06:08.232345 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 is on osd.4 2014-08-13 18:06:08.232345 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22 191727'411 also missing on osd.339 2014-08-13 18:06:08.232365 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22 191648'103 also missing on osd.339 2014-08-13 18:06:08.232363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 is on osd.4 2014-08-13 18:06:08.232382 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22 191727'451 also missing on osd.339 2014-08-13 18:06:08.232382 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 is on osd.4 2014-08-13 18:06:08.232403 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22 191648'140 also missing on osd.339 2014-08-13 18:06:08.232423 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22 191727'400 also missing on osd.339 2014-08-13 18:06:08.232403 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 18:06:08.232439 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22 191727'417 also missing on osd.339 2014-08-13 18:06:08.232449 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate: adding 21 as a source 2014-08-13 18:06:08.232458 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22 191727'445 also missing on osd.339 2014-08-13 18:06:08.232469 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 also missing on osd.21 2014-08-13 18:06:08.232488 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 also missing on osd.21 2014-08-13 18:06:08.232506 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 also missing on osd.21 2014-08-13 18:06:08.232475 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {26484113/benchmark_data_storage0111-ib.pocstor_40745_object4383/head//22=191648'77,4f894113/benchmark_data_storage0111-ib.pocstor_51285_object22733/head//22=191727'424,b18c4113/rb.0.43b76.238e1f29.00000000030a/head//22=192674'470,8f435113/benchmark_data_storage0111-ib.pocstor_51285_object23831/head//22=191727'426,350e5113/benchmark_data_storage0111-ib.pocstor_51285_object34533/head//22=191727'453,5fce5113/benchmark_data_storage0111-ib.pocstor_51285_object526/head//22=191727'385,b3cf6113/benchmark_data_storage0111-ib.pocstor_51285_object22568/head//22=191727'422,77a57113/benchmark_data_storage0111-ib.pocstor_51285_object28886/head//22=191727'439,4de7113/benchmark_data_storage0111-ib.pocstor_51285_object26390/head//22=191727'431,54308113/benchmark_data_storage0111-ib.pocstor_40745_object10163/head//22=191648'177,50269113/benchmark_data_storage0111-ib.pocstor_40745_object11898/head//22=191648'198,8760a113/benchmark_data_storage0111-ib.pocstor_40745_object9438/head//22=191648'168,ae0cc113/benchmark_data_storage0111-ib.pocstor_51285_object21131/head//22=191727'418,ef64d113/benchmark_data_storage0111-ib.pocstor_40745_object19186/head//22=191648'330,71cfd113/benchmark_data_storage0111-ib.pocstor_51285_object13682/head//22=191727'406,d6d8e113/benchmark_data_storage0111-ib.pocstor_51285_object27707/head//22=191727'436,d1b50913/benchmark_data_storage0111-ib.pocstor_51285_object38323/head//22=191727'462,c39b0913/benchmark_data_storage0111-ib.pocstor_51285_object29729/head//22=191727'443,bf9a4913/benchmark_data_storage0111-ib.pocstor_51285_object12689/head//22=191727'404,59047913/benchmark_data_storage0111-ib.pocstor_40745_object5621/head//22=191648'105,23bc9913/benchmark_data_storage0111-ib.pocstor_51285_object15902/head//22=191727'411,9049a913/benchmark_data_storage0111-ib.pocstor_40745_object5573/head//22=191648'103,9adaa913/benchmark_data_storage0111-ib.pocstor_51285_object34067/head//22=191727'451,4a0db913/benchmark_data_storage0111-ib.pocstor_40745_object7671/head//22=191648'140,8192c913/benchmark_data_storage0111-ib.pocstor_51285_object9935/head//22=191727'400,de6cd913/benchmark_data_storage0111-ib.pocstor_51285_object19816/head//22=191727'417,6afed913/benchmark_data_storage0111-ib.pocstor_51285_object30638/head//22=191727'445} 2014-08-13 18:06:08.232523 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 also missing on osd.21 2014-08-13 18:06:08.232526 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.232542 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.232542 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 also missing on osd.21 2014-08-13 18:06:08.232563 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 also missing on osd.21 2014-08-13 18:06:08.232563 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 24,45,55,339,342 2014-08-13 18:06:08.232583 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.232583 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 also missing on osd.21 2014-08-13 18:06:08.232602 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.232603 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 also missing on osd.21 2014-08-13 18:06:08.232620 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.232622 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 also missing on osd.21 2014-08-13 18:06:08.232637 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.232641 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 also missing on osd.21 2014-08-13 18:06:08.232659 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 also missing on osd.21 2014-08-13 18:06:08.232677 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 also missing on osd.21 2014-08-13 18:06:08.232695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 also missing on osd.21 2014-08-13 18:06:08.232712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 also missing on osd.21 2014-08-13 18:06:08.232736 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.232728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 also missing on osd.21 2014-08-13 18:06:08.232746 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 also missing on osd.21 2014-08-13 18:06:08.232763 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 also missing on osd.21 2014-08-13 18:06:08.232780 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 also missing on osd.21 2014-08-13 18:06:08.232796 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 also missing on osd.21 2014-08-13 18:06:08.232824 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:08.232812 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 18:06:08.232851 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] activate: adding 310 as a source 2014-08-13 18:06:08.232865 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.232869 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22 191727'414 also missing on osd.310 2014-08-13 18:06:08.232872 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:08.232893 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.232898 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.232887 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22 191727'416 also missing on osd.310 2014-08-13 18:06:08.232892 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.232907 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22 191648'92 also missing on osd.310 2014-08-13 18:06:08.232912 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:08.232925 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22 191727'352 also missing on osd.310 2014-08-13 18:06:08.232949 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.232952 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.232954 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5780 pg_notify(22.99(30) epoch 194496) v5 2014-08-13 18:06:08.232943 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22 191727'374 also missing on osd.310 2014-08-13 18:06:08.232966 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22 191727'385 also missing on osd.310 2014-08-13 18:06:08.232984 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22 191648'85 also missing on osd.310 2014-08-13 18:06:08.232999 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.337 2014-08-13 18:06:08.233003 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5780 2014-08-13 18:06:08.233003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22 191648'320 also missing on osd.310 2014-08-13 18:06:08.233018 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:08.233021 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22 191648'282 also missing on osd.310 2014-08-13 18:06:08.233038 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194496 2014-08-13 18:06:08.233040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22 191727'373 also missing on osd.310 2014-08-13 18:06:08.233056 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.233071 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233059 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22 191727'351 also missing on osd.310 2014-08-13 18:06:08.233075 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233089 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.233078 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22 191727'391 also missing on osd.310 2014-08-13 18:06:08.233096 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22 191727'388 also missing on osd.310 2014-08-13 18:06:08.233113 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233117 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233120 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5400 pg_notify(22.4df(32) epoch 194496) v5 2014-08-13 18:06:08.233113 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22 191648'130 also missing on osd.310 2014-08-13 18:06:08.233117 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 2014-08-13 18:06:08.233133 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22 191648'116 also missing on osd.310 2014-08-13 18:06:08.233150 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.233161 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.371 2014-08-13 18:06:08.233165 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5400 2014-08-13 18:06:08.233152 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22 191727'379 also missing on osd.310 2014-08-13 18:06:08.233156 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Started advmap 2014-08-13 18:06:08.233173 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22 191648'285 also missing on osd.310 2014-08-13 18:06:08.233185 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323,4] 2014-08-13 18:06:08.233198 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22 191727'386 also missing on osd.310 2014-08-13 18:06:08.233163 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.233220 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233204 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.233224 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] search_for_missing dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22 191648'312 also missing on osd.310 2014-08-13 18:06:08.233222 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.233227 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started/Stray 0.922017 4 0.000288 2014-08-13 18:06:08.233261 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233264 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233267 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5b00 pg_notify(22.6bd(2) epoch 194496) v5 2014-08-13 18:06:08.233252 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] exit Started 0.922131 0 0.000000 2014-08-13 18:06:08.233239 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {f9fd1707/benchmark_data_storage0111-ib.pocstor_51285_object36299/head//22=191727'414,effb3707/benchmark_data_storage0111-ib.pocstor_51285_object37311/head//22=191727'416,e277707/benchmark_data_storage0111-ib.pocstor_40745_object6405/head//22=191648'92,92aa7707/benchmark_data_storage0111-ib.pocstor_51285_object596/head//22=191727'352,a0109707/benchmark_data_storage0111-ib.pocstor_51285_object11394/head//22=191727'374,66529707/benchmark_data_storage0111-ib.pocstor_51285_object18722/head//22=191727'385,136bb707/benchmark_data_storage0111-ib.pocstor_40745_object5716/head//22=191648'85,326eb707/benchmark_data_storage0111-ib.pocstor_40745_object20421/head//22=191648'320,4740f07/benchmark_data_storage0111-ib.pocstor_40745_object18037/head//22=191648'282,26e11f07/benchmark_data_storage0111-ib.pocstor_51285_object10861/head//22=191727'373,b0205f07/benchmark_data_storage0111-ib.pocstor_51285_object261/head//22=191727'351,e8a56f07/benchmark_data_storage0111-ib.pocstor_51285_object21351/head//22=191727'391,39c77f07/benchmark_data_storage0111-ib.pocstor_51285_object19493/head//22=191727'388,9f4a9f07/benchmark_data_storage0111-ib.pocstor_40745_object8995/head//22=191648'130,1519af07/benchmark_data_storage0111-ib.pocstor_40745_object8224/head//22=191648'116,d25caf07/benchmark_data_storage0111-ib.pocstor_51285_object14706/head//22=191727'379,e43cf07/benchmark_data_storage0111-ib.pocstor_40745_object18158/head//22=191648'285,5c91ff07/benchmark_data_storage0111-ib.pocstor_51285_object18846/head//22=191727'386,dee8ff07/benchmark_data_storage0111-ib.pocstor_40745_object20041/head//22=191648'312} 2014-08-13 18:06:08.233274 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] enter Reset 2014-08-13 18:06:08.233283 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.233292 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194495 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194496 2014-08-13 18:06:08.233300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.233317 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.32 2014-08-13 18:06:08.233321 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5b00 2014-08-13 18:06:08.233312 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194496 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: Reset advmap 2014-08-13 18:06:08.233321 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 15,21,303,310,342 2014-08-13 18:06:08.233331 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194496 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.233341 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.233364 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233350 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194496 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] acting_up_affected newup [304,323,4] newacting [304,323,4] 2014-08-13 18:06:08.233368 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233359 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.233388 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233391 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233378 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.233394 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1c00 pg_notify(22.3d5(2) epoch 194496) v5 2014-08-13 18:06:08.233396 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.233368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194496 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.233433 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.343 2014-08-13 18:06:08.233417 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4]/[304,323] r=-1 lpr=194496 pi=193250-194494/35 crt=194422'16034 lcod 0'0 remapped NOTIFY m=61] set_last_peering_reset 194496 2014-08-13 18:06:08.233436 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1c00 2014-08-13 18:06:08.233456 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [304,323,4](304) acting [304,323](304)): not rw, up_thru 194492 up_from 192965 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [304,323,4](304) acting [304,323](304)) : primary up 192965-194492 does not include interval 2014-08-13 18:06:08.233482 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233485 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233474 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194495/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] noting past interval(194495-194495 up [304,323,4](304) acting [304,323](304)) 2014-08-13 18:06:08.233500 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.233505 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.233508 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.233493 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] up [304,323,4] -> [304,323,4], acting [304,323] -> [304,323,4], acting_primary 304 -> 304, up_primary 304 -> 304, role -1 -> 2 2014-08-13 18:06:08.233510 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5cc0 pg_notify(22.707(38) epoch 194496) v5 2014-08-13 18:06:08.233513 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:08.233533 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.233548 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.15 2014-08-13 18:06:08.233551 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5cc0 2014-08-13 18:06:08.233549 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:08.233547 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 2014-08-13 18:06:08.235521 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.235507 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_copy_ops 2014-08-13 18:06:08.235532 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_flush_ops 2014-08-13 18:06:08.235532 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Started advmap 2014-08-13 18:06:08.234165 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.235549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change_cleanup 2014-08-13 18:06:08.235557 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,4,285] 2014-08-13 18:06:08.235566 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_change 2014-08-13 18:06:08.235576 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235589 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235577 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.235570 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.235586 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit NotTrimming 2014-08-13 18:06:08.235601 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started/Stray 0.900424 4 0.000271 2014-08-13 18:06:08.235607 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter NotTrimming 2014-08-13 18:06:08.235624 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] on_role_change 2014-08-13 18:06:08.235624 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] exit Started 0.900526 0 0.000000 2014-08-13 18:06:08.235645 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235648 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235651 7f5c835ef700 20 osd.4 194496 _dispatch 0x21abde0 pg_query(24.1e4 epoch 194496) v3 2014-08-13 18:06:08.235641 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:08.235647 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] enter Reset 2014-08-13 18:06:08.235658 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] cancel_recovery 2014-08-13 18:06:08.235666 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194495 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194496 2014-08-13 18:06:08.235675 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] clear_recovery_state 2014-08-13 18:06:08.235686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194496 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: Reset advmap 2014-08-13 18:06:08.235708 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.31 epoch 194496 2014-08-13 18:06:08.235696 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.235713 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21abde0 2014-08-13 18:06:08.235706 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194496 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.235718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map 2014-08-13 18:06:08.235726 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194496 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] acting_up_affected newup [289,4,285] newacting [289,4,285] 2014-08-13 18:06:08.235730 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_query query(info 0'0) from replica 31 2014-08-13 18:06:08.235744 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194496 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.235761 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285]/[289,285] r=-1 lpr=194496 pi=193125-194494/48 crt=194422'17034 lcod 0'0 remapped NOTIFY m=63] set_last_peering_reset 194496 2014-08-13 18:06:08.235786 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235790 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235795 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [289,4,285](289) acting [289,285](289)): not rw, up_thru 194492 up_from 192968 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [289,4,285](289) acting [289,285](289)) : primary up 192968-194492 does not include interval 2014-08-13 18:06:08.235801 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] update_heartbeat_peers unchanged 2014-08-13 18:06:08.235819 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235823 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235826 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354600 pg_info(1 pgs e194496:22.12a(2)) v4 2014-08-13 18:06:08.235813 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194495/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] noting past interval(194495-194495 up [289,4,285](289) acting [289,285](289)) 2014-08-13 18:06:08.235821 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters 2014-08-13 18:06:08.235835 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] up [289,4,285] -> [289,4,285], acting [289,285] -> [289,4,285], acting_primary 289 -> 289, up_primary 289 -> 289, role -1 -> 1 2014-08-13 18:06:08.235843 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Reset 0.002568 1 0.002561 2014-08-13 18:06:08.235859 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:08.235870 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.12a(2)) v4 from osd.323 2014-08-13 18:06:08.235876 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354600 2014-08-13 18:06:08.235866 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started 2014-08-13 18:06:08.235877 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.235885 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Start 2014-08-13 18:06:08.235894 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:08.235902 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: transitioning to Stray 2014-08-13 18:06:08.235921 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235925 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_copy_ops 2014-08-13 18:06:08.235922 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Start 0.000037 0 0.000000 2014-08-13 18:06:08.235947 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.235950 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.235930 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_flush_ops 2014-08-13 18:06:08.235952 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1500 pg_notify(22.84(28) epoch 194496) v5 2014-08-13 18:06:08.235944 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/Stray 2014-08-13 18:06:08.235955 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change_cleanup 2014-08-13 18:06:08.235973 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_change 2014-08-13 18:06:08.235992 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.304 2014-08-13 18:06:08.235997 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1500 2014-08-13 18:06:08.235980 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.235992 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit NotTrimming 2014-08-13 18:06:08.236001 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.236011 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter NotTrimming 2014-08-13 18:06:08.236030 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] on_role_change 2014-08-13 18:06:08.236046 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236051 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236046 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:08.236061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] cancel_recovery 2014-08-13 18:06:08.236075 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] clear_recovery_state 2014-08-13 18:06:08.236093 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236096 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236101 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.236099 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5240 pg_info(1 pgs e194496:22.3ad(2)) v4 2014-08-13 18:06:08.236097 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.236118 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map 2014-08-13 18:06:08.236142 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3ad(2)) v4 from osd.292 2014-08-13 18:06:08.236147 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5240 2014-08-13 18:06:08.236148 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:08.236184 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.236192 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.236205 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236210 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236209 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.236233 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236237 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.236222 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] update_heartbeat_peers unchanged 2014-08-13 18:06:08.236240 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a45e80 pg_notify(22.99(1),22.39d(34) epoch 194496) v5 2014-08-13 18:06:08.236245 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters 2014-08-13 18:06:08.236246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.236264 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Reset 0.000618 1 0.000587 2014-08-13 18:06:08.236289 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.363 2014-08-13 18:06:08.236292 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a45e80 2014-08-13 18:06:08.236283 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.236285 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started 2014-08-13 18:06:08.236302 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.236304 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Start 2014-08-13 18:06:08.236321 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: transitioning to Stray 2014-08-13 18:06:08.236322 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:177960 2014-08-13 18:06:08.236356 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236343 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.236360 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236342 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Start 0.000038 0 0.000000 2014-08-13 18:06:08.236363 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.823892 5 0.000357 2014-08-13 18:06:08.236372 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/Stray 2014-08-13 18:06:08.236384 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.236402 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236406 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236408 7f5c835ef700 20 osd.4 194496 _dispatch 0x588fa80 pg_notify(24.30b epoch 194496) v5 2014-08-13 18:06:08.236395 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.236402 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.922750 0 0.000000 2014-08-13 18:06:08.236416 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.236426 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.236449 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.286 2014-08-13 18:06:08.236452 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588fa80 2014-08-13 18:06:08.236445 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194486 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.236468 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.236488 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236492 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236484 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.236520 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'1672 -> 194215'1672 2014-08-13 18:06:08.236522 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236525 7f5c835ef700 20 osd.4 194496 _dispatch 0x21af9e0 pg_query(24.2ec epoch 194496) v3 2014-08-13 18:06:08.236533 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.236560 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.374 epoch 194496 2014-08-13 18:06:08.236563 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21af9e0 2014-08-13 18:06:08.236552 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] flushed 2014-08-13 18:06:08.236575 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_query query(info 0'0) from replica 374 2014-08-13 18:06:08.236580 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 18:06:08.236608 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.236623 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236626 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236618 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.236636 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.236652 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236656 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236659 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354b40 pg_info(1 pgs e194496:22.537(2)) v4 2014-08-13 18:06:08.236643 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 1.2b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.236655 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.236676 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.236701 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.537(2)) v4 from osd.51 2014-08-13 18:06:08.236706 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354b40 2014-08-13 18:06:08.236695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.236713 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.236732 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:3009 2014-08-13 18:06:08.236748 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236752 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236748 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.236778 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236765 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.821979 3 0.000310 2014-08-13 18:06:08.236781 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236783 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4b40 pg_notify(22.113(34) epoch 194496) v5 2014-08-13 18:06:08.236784 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.236801 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.896372 0 0.000000 2014-08-13 18:06:08.236821 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.24 2014-08-13 18:06:08.236824 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4b40 2014-08-13 18:06:08.236825 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.236842 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.236864 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236868 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236862 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.236879 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.236905 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.236893 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'493 -> 194215'493 2014-08-13 18:06:08.236908 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.236911 7f5c835ef700 20 osd.4 194496 _dispatch 0x21aa940 pg_query(24.cd epoch 194496) v3 2014-08-13 18:06:08.236918 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.236948 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.6 epoch 194496 2014-08-13 18:06:08.236951 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21aa940 2014-08-13 18:06:08.236963 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_query query(info 0'0) from replica 6 2014-08-13 18:06:08.237005 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237008 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237011 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] flushed 2014-08-13 18:06:08.237033 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237036 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237038 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a388c0 pg_notify(24.181(38) epoch 194496) v5 2014-08-13 18:06:08.237073 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.344 2014-08-13 18:06:08.237076 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a388c0 2014-08-13 18:06:08.237124 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237127 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237160 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237163 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237165 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4280 pg_notify(22.707(1) epoch 194496) v5 2014-08-13 18:06:08.237208 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.303 2014-08-13 18:06:08.237212 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4280 2014-08-13 18:06:08.237219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 sending log((0'0,194215'493], crt=191648'346) 2014-08-13 18:06:08.237250 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x449d280 don't know epoch, doing nothing 2014-08-13 18:06:08.237258 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237262 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237281 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237285 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237287 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4600 pg_info(1 pgs e194496:22.2b9(2)) v4 2014-08-13 18:06:08.237281 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.5 22.1f1( DNE v 194215'493 lc 0'0 (0'0,194215'493] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 18:06:08.237305 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 22.1f1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.237325 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2b9(2)) v4 from osd.328 2014-08-13 18:06:08.237330 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4600 2014-08-13 18:06:08.237369 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237372 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237390 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237392 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237394 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0e00 pg_info(1 pgs e194496:22.220(1)) v4 2014-08-13 18:06:08.237430 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.220(1)) v4 from osd.351 2014-08-13 18:06:08.237434 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0e00 2014-08-13 18:06:08.237478 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237481 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237516 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237519 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237521 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356200 pg_notify(22.3d9(50) epoch 194496) v5 2014-08-13 18:06:08.237556 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.338 2014-08-13 18:06:08.237559 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356200 2014-08-13 18:06:08.237559 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 sending log((0'0,194215'493], crt=191648'346) 2014-08-13 18:06:08.237584 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a5960 don't know epoch, doing nothing 2014-08-13 18:06:08.237606 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237610 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237610 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate peer osd.47 22.1f1( DNE v 194215'493 lc 0'0 (0'0,194215'493] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 18:06:08.237640 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237643 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237645 7f5c835ef700 20 osd.4 194496 _dispatch 0x588f000 pg_notify(22.1d2(1) epoch 194496) v5 2014-08-13 18:06:08.237680 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.331 2014-08-13 18:06:08.237683 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588f000 2014-08-13 18:06:08.237727 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237731 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237725 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.5 has 35 missing 2014-08-13 18:06:08.237750 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237753 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237741 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.47 has 35 missing 2014-08-13 18:06:08.237755 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4ec0 pg_info(1 pgs e194496:22.581(1)) v4 2014-08-13 18:06:08.237757 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 is on osd.4 2014-08-13 18:06:08.237794 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.581(1)) v4 from osd.320 2014-08-13 18:06:08.237783 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 is on osd.4 2014-08-13 18:06:08.237800 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4ec0 2014-08-13 18:06:08.237803 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 is on osd.4 2014-08-13 18:06:08.237821 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 is on osd.4 2014-08-13 18:06:08.237835 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237839 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237837 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 is on osd.4 2014-08-13 18:06:08.237866 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237854 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.237869 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237872 7f5c835ef700 20 osd.4 194496 _dispatch 0x588d240 pg_info(1 pgs e194496:22.69f(2)) v4 2014-08-13 18:06:08.237870 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 is on osd.4 2014-08-13 18:06:08.237888 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.237910 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.69f(2)) v4 from osd.334 2014-08-13 18:06:08.237914 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588d240 2014-08-13 18:06:08.237905 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 is on osd.4 2014-08-13 18:06:08.237922 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 is on osd.4 2014-08-13 18:06:08.237938 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 is on osd.4 2014-08-13 18:06:08.237954 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237958 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237954 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 is on osd.4 2014-08-13 18:06:08.237980 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.237983 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.237971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.237985 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6200 pg_info(1 pgs e194496:22.639(2)) v4 2014-08-13 18:06:08.237988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 is on osd.4 2014-08-13 18:06:08.238007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 is on osd.4 2014-08-13 18:06:08.238029 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.639(2)) v4 from osd.367 2014-08-13 18:06:08.238033 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6200 2014-08-13 18:06:08.238024 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 is on osd.4 2014-08-13 18:06:08.238041 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 is on osd.4 2014-08-13 18:06:08.238061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 is on osd.4 2014-08-13 18:06:08.238081 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238085 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238078 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 is on osd.4 2014-08-13 18:06:08.238105 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238094 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 is on osd.4 2014-08-13 18:06:08.238108 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238110 7f5c835ef700 20 osd.4 194496 _dispatch 0x588e580 pg_notify(24.136(36) epoch 194496) v5 2014-08-13 18:06:08.238110 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 is on osd.4 2014-08-13 18:06:08.238127 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 is on osd.4 2014-08-13 18:06:08.238148 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.65 2014-08-13 18:06:08.238151 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588e580 2014-08-13 18:06:08.238144 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 is on osd.4 2014-08-13 18:06:08.238161 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.238194 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238177 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.238198 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238201 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 is on osd.4 2014-08-13 18:06:08.238224 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238227 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238229 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6580 pg_notify(24.3d(38) epoch 194496) v5 2014-08-13 18:06:08.238218 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 is on osd.4 2014-08-13 18:06:08.238236 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 is on osd.4 2014-08-13 18:06:08.238252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 is on osd.4 2014-08-13 18:06:08.238273 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.373 2014-08-13 18:06:08.238276 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6580 2014-08-13 18:06:08.238270 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 is on osd.4 2014-08-13 18:06:08.238287 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 is on osd.4 2014-08-13 18:06:08.238304 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.238326 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238329 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238320 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.238337 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 is on osd.4 2014-08-13 18:06:08.238352 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 is on osd.4 2014-08-13 18:06:08.238366 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238370 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238373 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df55c0 pg_notify(22.3d9(21) epoch 194496) v5 2014-08-13 18:06:08.238409 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.397 2014-08-13 18:06:08.238412 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df55c0 2014-08-13 18:06:08.238369 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 18:06:08.238447 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238450 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238449 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate: adding 5 as a source 2014-08-13 18:06:08.238480 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238469 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 also missing on osd.5 2014-08-13 18:06:08.238483 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238485 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3b80 pg_notify(22.3d9(50) epoch 194496) v5 2014-08-13 18:06:08.238487 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 also missing on osd.5 2014-08-13 18:06:08.238505 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 also missing on osd.5 2014-08-13 18:06:08.238523 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.396 2014-08-13 18:06:08.238526 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3b80 2014-08-13 18:06:08.238523 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 also missing on osd.5 2014-08-13 18:06:08.238541 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 also missing on osd.5 2014-08-13 18:06:08.238559 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 also missing on osd.5 2014-08-13 18:06:08.238574 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238578 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238576 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 also missing on osd.5 2014-08-13 18:06:08.238594 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 also missing on osd.5 2014-08-13 18:06:08.238609 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238612 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238615 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf0c40 pg_notify(22.3d9(50) epoch 194496) v5 2014-08-13 18:06:08.238611 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 also missing on osd.5 2014-08-13 18:06:08.238630 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 also missing on osd.5 2014-08-13 18:06:08.238652 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.376 2014-08-13 18:06:08.238655 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf0c40 2014-08-13 18:06:08.238648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 also missing on osd.5 2014-08-13 18:06:08.238666 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 also missing on osd.5 2014-08-13 18:06:08.238692 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238695 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 also missing on osd.5 2014-08-13 18:06:08.238701 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 also missing on osd.5 2014-08-13 18:06:08.238718 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 also missing on osd.5 2014-08-13 18:06:08.238740 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238743 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238745 7f5c835ef700 20 osd.4 194496 _dispatch 0x53547c0 pg_info(1 pgs e194496:22.630(2)) v4 2014-08-13 18:06:08.238735 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 also missing on osd.5 2014-08-13 18:06:08.238754 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 also missing on osd.5 2014-08-13 18:06:08.238772 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 also missing on osd.5 2014-08-13 18:06:08.238785 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.630(2)) v4 from osd.294 2014-08-13 18:06:08.238789 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53547c0 2014-08-13 18:06:08.238790 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 also missing on osd.5 2014-08-13 18:06:08.238808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 also missing on osd.5 2014-08-13 18:06:08.238831 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238835 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238825 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 also missing on osd.5 2014-08-13 18:06:08.238843 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 also missing on osd.5 2014-08-13 18:06:08.238861 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238864 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238866 7f5c835ef700 20 osd.4 194496 _dispatch 0x53532c0 pg_notify(24.1bb(40) epoch 194496) v5 2014-08-13 18:06:08.238860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 also missing on osd.5 2014-08-13 18:06:08.238879 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 also missing on osd.5 2014-08-13 18:06:08.238904 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.379 2014-08-13 18:06:08.238908 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53532c0 2014-08-13 18:06:08.238897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 also missing on osd.5 2014-08-13 18:06:08.238916 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 also missing on osd.5 2014-08-13 18:06:08.238934 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 also missing on osd.5 2014-08-13 18:06:08.238948 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238952 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238952 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 also missing on osd.5 2014-08-13 18:06:08.238970 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 also missing on osd.5 2014-08-13 18:06:08.238986 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.238989 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.238992 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357a80 pg_notify(22.22c(30) epoch 194496) v5 2014-08-13 18:06:08.238987 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 also missing on osd.5 2014-08-13 18:06:08.239005 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 also missing on osd.5 2014-08-13 18:06:08.239034 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.324 2014-08-13 18:06:08.239037 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357a80 2014-08-13 18:06:08.239026 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 also missing on osd.5 2014-08-13 18:06:08.239044 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 also missing on osd.5 2014-08-13 18:06:08.239062 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 also missing on osd.5 2014-08-13 18:06:08.239092 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.239096 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.239080 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 also missing on osd.5 2014-08-13 18:06:08.239138 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.239142 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.239144 7f5c835ef700 20 osd.4 194496 _dispatch 0x524c600 pg_notify(22.775(36) epoch 194496) v5 2014-08-13 18:06:08.239180 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.0 2014-08-13 18:06:08.239128 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 18:06:08.239183 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524c600 2014-08-13 18:06:08.239191 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] activate: adding 47 as a source 2014-08-13 18:06:08.239223 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22 191727'417 also missing on osd.47 2014-08-13 18:06:08.239242 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.239246 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.239241 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22 191727'372 also missing on osd.47 2014-08-13 18:06:08.239259 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22 191648'8 also missing on osd.47 2014-08-13 18:06:08.239281 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.239284 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.239286 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d400 pg_notify(22.1f1(1) epoch 194496) v5 2014-08-13 18:06:08.239276 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22 191648'127 also missing on osd.47 2014-08-13 18:06:08.239294 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22 191648'80 also missing on osd.47 2014-08-13 18:06:08.239312 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22 191727'388 also missing on osd.47 2014-08-13 18:06:08.239329 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22 191727'378 also missing on osd.47 2014-08-13 18:06:08.239346 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22 191727'411 also missing on osd.47 2014-08-13 18:06:08.239360 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.11 2014-08-13 18:06:08.239363 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d400 2014-08-13 18:06:08.239364 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22 191648'206 also missing on osd.47 2014-08-13 18:06:08.239382 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22 191648'220 also missing on osd.47 2014-08-13 18:06:08.239399 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22 191727'395 also missing on osd.47 2014-08-13 18:06:08.239416 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22 191648'64 also missing on osd.47 2014-08-13 18:06:08.239433 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22 191727'414 also missing on osd.47 2014-08-13 18:06:08.239450 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22 191648'10 also missing on osd.47 2014-08-13 18:06:08.239466 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22 191648'200 also missing on osd.47 2014-08-13 18:06:08.239483 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22 191648'16 also missing on osd.47 2014-08-13 18:06:08.239500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22 191727'416 also missing on osd.47 2014-08-13 18:06:08.239517 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22 191727'354 also missing on osd.47 2014-08-13 18:06:08.239533 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22 191648'68 also missing on osd.47 2014-08-13 18:06:08.239550 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22 191727'404 also missing on osd.47 2014-08-13 18:06:08.239567 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22 191648'167 also missing on osd.47 2014-08-13 18:06:08.239583 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22 191727'353 also missing on osd.47 2014-08-13 18:06:08.239600 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22 191648'199 also missing on osd.47 2014-08-13 18:06:08.239616 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22 191727'406 also missing on osd.47 2014-08-13 18:06:08.239633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22 191727'399 also missing on osd.47 2014-08-13 18:06:08.239649 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22 191727'365 also missing on osd.47 2014-08-13 18:06:08.239666 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22 191727'366 also missing on osd.47 2014-08-13 18:06:08.239683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22 191727'379 also missing on osd.47 2014-08-13 18:06:08.239699 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22 191648'109 also missing on osd.47 2014-08-13 18:06:08.239716 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22 191648'25 also missing on osd.47 2014-08-13 18:06:08.239733 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22 191727'355 also missing on osd.47 2014-08-13 18:06:08.239749 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22 191727'371 also missing on osd.47 2014-08-13 18:06:08.239766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22 191727'415 also missing on osd.47 2014-08-13 18:06:08.239782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22 191648'345 also missing on osd.47 2014-08-13 18:06:08.239799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] search_for_missing cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22 191727'389 also missing on osd.47 2014-08-13 18:06:08.239814 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7ed511f1/benchmark_data_storage0111-ib.pocstor_51285_object35149/head//22=191727'417,e33b31f1/benchmark_data_storage0111-ib.pocstor_51285_object14724/head//22=191727'372,4cfc31f1/benchmark_data_storage0111-ib.pocstor_40745_object440/head//22=191648'8,d4d641f1/benchmark_data_storage0111-ib.pocstor_40745_object8470/head//22=191648'127,79b051f1/benchmark_data_storage0111-ib.pocstor_40745_object5478/head//22=191648'80,b03b51f1/benchmark_data_storage0111-ib.pocstor_51285_object19820/head//22=191727'388,e4a361f1/benchmark_data_storage0111-ib.pocstor_51285_object16927/head//22=191727'378,ea6291f1/benchmark_data_storage0111-ib.pocstor_51285_object32721/head//22=191727'411,c57a91f1/benchmark_data_storage0111-ib.pocstor_40745_object13679/head//22=191648'206,6f6d91f1/benchmark_data_storage0111-ib.pocstor_40745_object14804/head//22=191648'220,c2ad91f1/benchmark_data_storage0111-ib.pocstor_51285_object24930/head//22=191727'395,3986d1f1/benchmark_data_storage0111-ib.pocstor_40745_object4385/head//22=191648'64,d248d1f1/benchmark_data_storage0111-ib.pocstor_51285_object33476/head//22=191727'414,f93dd1f1/benchmark_data_storage0111-ib.pocstor_40745_object592/head//22=191648'10,187309f1/benchmark_data_storage0111-ib.pocstor_40745_object13424/head//22=191648'200,c49e09f1/benchmark_data_storage0111-ib.pocstor_40745_object1143/head//22=191648'16,3a7919f1/benchmark_data_storage0111-ib.pocstor_51285_object34626/head//22=191727'416,340d19f1/benchmark_data_storage0111-ib.pocstor_51285_object5870/head//22=191727'354,a74329f1/benchmark_data_storage0111-ib.pocstor_40745_object4724/head//22=191648'68,f0b729f1/benchmark_data_storage0111-ib.pocstor_51285_object30473/head//22=191727'404,bfa139f1/benchmark_data_storage0111-ib.pocstor_40745_object10845/head//22=191648'167,426d49f1/benchmark_data_storage0111-ib.pocstor_51285_object5299/head//22=191727'353,d06769f1/benchmark_data_storage0111-ib.pocstor_40745_object13342/head//22=191648'199,e55d69f1/benchmark_data_storage0111-ib.pocstor_51285_object31179/head//22=191727'406,6b0079f1/benchmark_data_storage0111-ib.pocstor_51285_object26004/head//22=191727'399,7ac879f1/benchmark_data_storage0111-ib.pocstor_51285_object10383/head//22=191727'365,d742a9f1/benchmark_data_storage0111-ib.pocstor_51285_object11575/head//22=191727'366,7431b9f1/benchmark_data_storage0111-ib.pocstor_51285_object18002/head//22=191727'379,a1d2c9f1/benchmark_data_storage0111-ib.pocstor_40745_object7622/head//22=191648'109,7516c9f1/benchmark_data_storage0111-ib.pocstor_40745_object1582/head//22=191648'25,ca32d9f1/benchmark_data_storage0111-ib.pocstor_51285_object6314/head//22=191727'355,a3a5e9f1/benchmark_data_storage0111-ib.pocstor_51285_object13815/head//22=191727'371,594ae9f1/benchmark_data_storage0111-ib.pocstor_51285_object33864/head//22=191727'415,7bd7f9f1/benchmark_data_storage0111-ib.pocstor_40745_object22730/head//22=191648'345,cad8f9f1/benchmark_data_storage0111-ib.pocstor_51285_object20073/head//22=191727'389} 2014-08-13 18:06:08.239875 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.239890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.239907 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 5,11,47,54,368 2014-08-13 18:06:08.239923 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.239939 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.239956 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.239971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.240067 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.240118 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 18:06:08.240140 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240144 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240143 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.240150 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.240167 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240169 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240172 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cd00 pg_notify(22.4df(1) epoch 194496) v5 2014-08-13 18:06:08.240167 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.240186 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.240216 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.352 2014-08-13 18:06:08.240220 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cd00 2014-08-13 18:06:08.240247 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.240267 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240271 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240264 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.240291 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240280 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.240294 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240297 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.240296 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352bc0 pg_notify(22.48e(34) epoch 194496) v5 2014-08-13 18:06:08.240332 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.327 2014-08-13 18:06:08.240336 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352bc0 2014-08-13 18:06:08.240322 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 2014-08-13 18:06:08.240352 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.240370 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240374 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240359 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.240379 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Started advmap 2014-08-13 18:06:08.240408 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240394 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,330,28] newacting [4,17,315] 2014-08-13 18:06:08.240411 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240414 7f5c835ef700 20 osd.4 194496 _dispatch 0x588f540 pg_notify(22.3af(36) epoch 194496) v5 2014-08-13 18:06:08.240412 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.240431 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary/Peering/WaitActingChange 0.825460 3 0.000252 2014-08-13 18:06:08.240451 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.339 2014-08-13 18:06:08.240454 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588f540 2014-08-13 18:06:08.240452 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started/Primary 0.825492 0 0.000000 2014-08-13 18:06:08.240473 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] exit Started 0.890297 0 0.000000 2014-08-13 18:06:08.240493 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240498 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240492 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] enter Reset 2014-08-13 18:06:08.240509 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194495 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194496 2014-08-13 18:06:08.240531 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240534 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240536 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353640 pg_notify(22.481(42) epoch 194496) v5 2014-08-13 18:06:08.240526 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194496 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: Reset advmap 2014-08-13 18:06:08.240509 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.240544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194496 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.240558 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.240560 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194496 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] acting_up_affected newup [4,330,28] newacting [4,17,315] 2014-08-13 18:06:08.240576 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.309 2014-08-13 18:06:08.240580 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353640 2014-08-13 18:06:08.240579 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194496 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.240596 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28] r=0 lpr=194496 pi=193748-194494/40 crt=194422'15454 lcod 0'0 mlcod 0'0 inactive m=59] set_last_peering_reset 194496 2014-08-13 18:06:08.240622 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240625 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,330,28](4) acting [4,330,28](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,330,28](4) acting [4,330,28](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.240653 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194495/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] noting past interval(194495-194495 up [4,330,28](4) acting [4,330,28](4)) 2014-08-13 18:06:08.240672 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240675 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240677 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356900 pg_notify(22.3d9(54) epoch 194496) v5 2014-08-13 18:06:08.240673 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] up [4,330,28] -> [4,330,28], acting [4,330,28] -> [4,17,315], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.240695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] clear_primary_state 2014-08-13 18:06:08.240715 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.293 2014-08-13 18:06:08.240718 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356900 2014-08-13 18:06:08.240717 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] agent_stop 2014-08-13 18:06:08.240737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 18:06:08.240759 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240763 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240756 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194496:22352 2014-08-13 18:06:08.240759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 sending log((0'0,194215'1672], crt=0'0) 2014-08-13 18:06:08.240774 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.240793 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.240800 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240803 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240806 7f5c835ef700 20 osd.4 194496 _dispatch 0x588fe00 pg_notify(22.113(1),22.3e4(36) epoch 194496) v5 2014-08-13 18:06:08.240793 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_copy_ops 2014-08-13 18:06:08.240812 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_flush_ops 2014-08-13 18:06:08.240830 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.240827 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 1.2b( DNE v 194215'1672 (0'0,194215'1672] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) uptodate 2014-08-13 18:06:08.240853 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.342 2014-08-13 18:06:08.240857 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588fe00 2014-08-13 18:06:08.240848 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.240853 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.240871 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.240873 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:08.240890 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.240892 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change_cleanup 2014-08-13 18:06:08.240909 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.240910 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] on_change 2014-08-13 18:06:08.240928 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240932 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240926 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.240930 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit NotTrimming 2014-08-13 18:06:08.240959 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.240962 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.240950 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter NotTrimming 2014-08-13 18:06:08.240965 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351880 pg_notify(22.619(36) epoch 194496) v5 2014-08-13 18:06:08.240968 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] [4,330,28] -> [4,17,315], replicas changed 2014-08-13 18:06:08.240991 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.240986 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] cancel_recovery 2014-08-13 18:06:08.241005 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.314 2014-08-13 18:06:08.241009 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351880 2014-08-13 18:06:08.241003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] clear_recovery_state 2014-08-13 18:06:08.241023 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.241044 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map 2014-08-13 18:06:08.241059 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241062 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241064 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] update_heartbeat_peers 4,17,28,315,327,330 -> 4,17,28,315,330 2014-08-13 18:06:08.241084 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.241099 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241102 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241088 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] take_waiters 2014-08-13 18:06:08.241107 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6900 pg_notify(22.4df(32) epoch 194496) v5 2014-08-13 18:06:08.241110 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Reset 0.000617 1 0.000685 2014-08-13 18:06:08.241131 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started 2014-08-13 18:06:08.241149 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Start 2014-08-13 18:06:08.241165 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: transitioning to Primary 2014-08-13 18:06:08.241194 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.380 2014-08-13 18:06:08.241198 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6900 2014-08-13 18:06:08.241185 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] exit Start 0.000035 0 0.000000 2014-08-13 18:06:08.241214 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary 2014-08-13 18:06:08.241251 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241255 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241241 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Peering 2014-08-13 18:06:08.241263 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.241280 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.241300 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241303 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241305 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0540 pg_notify(22.22c(28) epoch 194496) v5 2014-08-13 18:06:08.241299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194495-194495 up [4,330,28](4) acting [4,330,28](4)) 2014-08-13 18:06:08.241320 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194492-194494 up [327,17,315](327) acting [327,17,315](327) maybe_went_rw) 2014-08-13 18:06:08.241344 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.368 2014-08-13 18:06:08.241347 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0540 2014-08-13 18:06:08.241343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior interval(194491-194491 up [327,17,315](327) acting [17,315,327](17)) 2014-08-13 18:06:08.241363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] PriorSet: build_prior final: probe 4,17,28,315,327,330 down blocked_by {} 2014-08-13 18:06:08.241383 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241387 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241383 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.241411 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241414 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241416 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bf00 pg_notify(22.3af(36) epoch 194496) v5 2014-08-13 18:06:08.241404 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194496:22353 2014-08-13 18:06:08.241423 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.17 2014-08-13 18:06:08.241453 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.369 2014-08-13 18:06:08.241457 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bf00 2014-08-13 18:06:08.241445 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.28 2014-08-13 18:06:08.241464 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.315 2014-08-13 18:06:08.241482 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.327 2014-08-13 18:06:08.241505 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241509 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241499 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: querying info from osd.330 2014-08-13 18:06:08.241518 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.241534 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241538 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241540 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421c20 pg_query(24.3f7 epoch 194496) v3 2014-08-13 18:06:08.241535 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.241576 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.330 epoch 194496 2014-08-13 18:06:08.241580 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421c20 2014-08-13 18:06:08.241591 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_query query(info 0'0) from replica 330 2014-08-13 18:06:08.241651 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.241653 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241657 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241676 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241679 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241680 7f5c835ef700 20 osd.4 194496 _dispatch 0x588d5c0 pg_notify(22.481(42) epoch 194496) v5 2014-08-13 18:06:08.241718 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.365 2014-08-13 18:06:08.241721 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588d5c0 2014-08-13 18:06:08.241755 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241758 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241801 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241804 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241806 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889180 pg_notify(24.9b epoch 194496) v5 2014-08-13 18:06:08.241839 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.378 2014-08-13 18:06:08.241842 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889180 2014-08-13 18:06:08.241879 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241882 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241908 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241910 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.241912 7f5c835ef700 20 osd.4 194496 _dispatch 0x53539c0 pg_notify(22.481(47) epoch 194496) v5 2014-08-13 18:06:08.241947 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.306 2014-08-13 18:06:08.241950 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53539c0 2014-08-13 18:06:08.241988 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.241991 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.242028 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.242031 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.242033 7f5c835ef700 20 osd.4 194496 _dispatch 0x588de80 pg_info(1 pgs e194496:22.1d2) v4 2014-08-13 18:06:08.242067 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1d2) v4 from osd.349 2014-08-13 18:06:08.242072 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588de80 2014-08-13 18:06:08.242112 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.242115 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.242134 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.242137 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.242139 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357000 pg_notify(24.30b(41) epoch 194496) v5 2014-08-13 18:06:08.242194 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.10 2014-08-13 18:06:08.242198 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357000 2014-08-13 18:06:08.246402 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] handle_advance_map [26,30,340]/[4] -- 26/4 2014-08-13 18:06:08.246429 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 18:06:08.246896 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.249160 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.249163 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249168 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249160 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.249169 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: Started advmap 2014-08-13 18:06:08.249178 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.249188 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.249190 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.249228 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249232 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249217 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] acting_up_affected newup [26,30,340] newacting [4] 2014-08-13 18:06:08.249221 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 18:06:08.249234 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b640 pg_notify(24.30b(2) epoch 194496) v5 2014-08-13 18:06:08.249236 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.249255 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.925894 3 0.000256 2014-08-13 18:06:08.249260 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 18:06:08.249283 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.301 2014-08-13 18:06:08.249287 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b640 2014-08-13 18:06:08.249277 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] exit Started 0.925992 0 0.000000 2014-08-13 18:06:08.249281 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.249298 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] enter Reset 2014-08-13 18:06:08.249300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.249326 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.249314 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194488 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194496 2014-08-13 18:06:08.249339 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249342 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249332 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194496 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: Reset advmap 2014-08-13 18:06:08.249351 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 18:06:08.249370 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249359 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194496 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:08.249374 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249374 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.249377 7f5c835ef700 20 osd.4 194496 _dispatch 0x524eac0 pg_notify(22.1d2(30) epoch 194496) v5 2014-08-13 18:06:08.249376 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194496 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] acting_up_affected newup [26,30,340] newacting [4] 2014-08-13 18:06:08.249380 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.249393 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194496 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.249399 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.249421 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.58 2014-08-13 18:06:08.249411 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340] r=-1 lpr=194496 pi=193125-194487/53 crt=193305'14233 lcod 0'0 inactive NOTIFY] set_last_peering_reset 194496 2014-08-13 18:06:08.249425 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524eac0 2014-08-13 18:06:08.249419 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.249450 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(194488-194495 up [26,30,340](26) acting [26,30,340](26)): not rw, up_thru 194488 up_from 192966 last_epoch_clean 194258 generate_past_intervals interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) : primary up 192966-194488 includes interval 2014-08-13 18:06:08.249481 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249469 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194488/194488) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] noting past interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.249485 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249490 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] up [26,30,340] -> [26,30,340], acting [26,30,340] -> [4], acting_primary 26 -> 4, up_primary 26 -> 26, role -1 -> 0 2014-08-13 18:06:08.249500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.249520 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249524 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249510 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] clear_primary_state 2014-08-13 18:06:08.249527 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1880 pg_info(1 pgs e194496:22.116(1)) v4 2014-08-13 18:06:08.249519 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.249530 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] agent_stop 2014-08-13 18:06:08.249539 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.249559 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.249549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:08.249572 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.116(1)) v4 from osd.345 2014-08-13 18:06:08.249578 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1880 2014-08-13 18:06:08.249571 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] publish_stats_to_osd 194496:23844 2014-08-13 18:06:08.249589 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:08.249584 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 2014-08-13 18:06:08.249614 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.249609 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] cancel_copy_ops 2014-08-13 18:06:08.249631 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249635 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249620 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Started advmap 2014-08-13 18:06:08.249625 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] cancel_flush_ops 2014-08-13 18:06:08.249639 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364,4] 2014-08-13 18:06:08.249642 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:08.249662 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249665 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249668 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888700 pg_notify(22.1f1(34) epoch 194496) v5 2014-08-13 18:06:08.249657 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.249660 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:08.249677 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] requeue_ops 2014-08-13 18:06:08.249677 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started/Stray 0.894537 4 0.000213 2014-08-13 18:06:08.249709 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.54 2014-08-13 18:06:08.249696 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change_cleanup 2014-08-13 18:06:08.249713 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888700 2014-08-13 18:06:08.249702 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] exit Started 0.894607 0 0.000000 2014-08-13 18:06:08.249713 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] on_change 2014-08-13 18:06:08.249725 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] enter Reset 2014-08-13 18:06:08.249735 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] exit NotTrimming 2014-08-13 18:06:08.249744 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194495 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194496 2014-08-13 18:06:08.249766 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249753 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped NOTIFY] enter NotTrimming 2014-08-13 18:06:08.249770 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249763 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194496 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: Reset advmap 2014-08-13 18:06:08.249772 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] clear_stats 2014-08-13 18:06:08.249782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194496 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.249804 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249791 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] requeue_ops 2014-08-13 18:06:08.249807 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249810 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889dc0 pg_info(1 pgs e194496:22.791(2)) v4 2014-08-13 18:06:08.249801 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194496 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] acting_up_affected newup [356,364,4] newacting [356,364,4] 2014-08-13 18:06:08.249809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] on_role_change 2014-08-13 18:06:08.249822 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194496 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.249827 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] requeue_ops 2014-08-13 18:06:08.249852 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.791(2)) v4 from osd.30 2014-08-13 18:06:08.249841 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4]/[356,364] r=-1 lpr=194496 pi=193151-194494/35 crt=194422'17433 lcod 0'0 remapped NOTIFY m=51] set_last_peering_reset 194496 2014-08-13 18:06:08.249843 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] cancel_recovery 2014-08-13 18:06:08.249857 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889dc0 2014-08-13 18:06:08.249859 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] clear_recovery_state 2014-08-13 18:06:08.249879 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.249877 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [356,364,4](356) acting [356,364](356)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [356,364,4](356) acting [356,364](356)) : primary up 192969-194492 does not include interval 2014-08-13 18:06:08.249909 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249913 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249901 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] handle_activate_map 2014-08-13 18:06:08.249902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] noting past interval(194495-194495 up [356,364,4](356) acting [356,364](356)) 2014-08-13 18:06:08.249923 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] update_heartbeat_peers -> 4,26,30,340 2014-08-13 18:06:08.249922 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] up [356,364,4] -> [356,364,4], acting [356,364] -> [356,364,4], acting_primary 356 -> 356, up_primary 356 -> 356, role -1 -> 2 2014-08-13 18:06:08.249942 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.249956 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.249944 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:08.249945 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] take_waiters 2014-08-13 18:06:08.249960 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.249963 7f5c835ef700 20 osd.4 194496 _dispatch 0x4deabc0 pg_info(1 pgs e194496:22.f3) v4 2014-08-13 18:06:08.249962 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.249964 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Reset 0.000667 1 0.000723 2014-08-13 18:06:08.249980 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:08.249986 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started 2014-08-13 18:06:08.249997 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_copy_ops 2014-08-13 18:06:08.250004 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Start 2014-08-13 18:06:08.250014 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_flush_ops 2014-08-13 18:06:08.250021 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: transitioning to Primary 2014-08-13 18:06:08.250031 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change_cleanup 2014-08-13 18:06:08.250041 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] exit Start 0.000038 0 0.000000 2014-08-13 18:06:08.250050 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_change 2014-08-13 18:06:08.250074 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.f3) v4 from osd.285 2014-08-13 18:06:08.250064 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary 2014-08-13 18:06:08.250079 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4deabc0 2014-08-13 18:06:08.250069 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit NotTrimming 2014-08-13 18:06:08.250083 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Peering 2014-08-13 18:06:08.250087 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter NotTrimming 2014-08-13 18:06:08.250101 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.250105 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] on_role_change 2014-08-13 18:06:08.250119 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] _calc_past_interval_range: already have past intervals back to 194258 2014-08-13 18:06:08.250122 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:08.250138 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] cancel_recovery 2014-08-13 18:06:08.250137 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.250155 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] clear_recovery_state 2014-08-13 18:06:08.250166 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250171 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250159 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194486-194487 up [26,30,340](26) acting [4](4)) 2014-08-13 18:06:08.250172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.250187 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.250207 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250211 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250191 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map 2014-08-13 18:06:08.250214 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a68380 pg_notify(22.362(1) epoch 194496) v5 2014-08-13 18:06:08.250207 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194477-194478 up [26,30,340](26) acting [4](4)) 2014-08-13 18:06:08.250226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194473-194476 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.250253 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.61 2014-08-13 18:06:08.250256 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a68380 2014-08-13 18:06:08.250245 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior final: probe 4,26,30,340 down blocked_by {} 2014-08-13 18:06:08.250263 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.250273 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] update_heartbeat_peers unchanged 2014-08-13 18:06:08.250283 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194496:23845 2014-08-13 18:06:08.250296 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters 2014-08-13 18:06:08.250302 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.26 2014-08-13 18:06:08.250316 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250320 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250314 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Reset 0.000589 1 0.000571 2014-08-13 18:06:08.250326 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.30 2014-08-13 18:06:08.250342 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250345 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250348 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf4ec0 pg_notify(22.48e(1) epoch 194496) v5 2014-08-13 18:06:08.250337 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started 2014-08-13 18:06:08.250346 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: querying info from osd.340 2014-08-13 18:06:08.250357 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Start 2014-08-13 18:06:08.250368 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.250374 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: transitioning to Stray 2014-08-13 18:06:08.250390 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.289 2014-08-13 18:06:08.250394 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf4ec0 2014-08-13 18:06:08.250388 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.250395 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Start 0.000038 0 0.000000 2014-08-13 18:06:08.250416 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/Stray 2014-08-13 18:06:08.250439 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250442 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250442 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.250463 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250466 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250468 7f5c835ef700 20 osd.4 194496 _dispatch 0x21abc00 pg_query(24.369 epoch 194496) v3 2014-08-13 18:06:08.250460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.250505 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.45 epoch 194496 2014-08-13 18:06:08.250508 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21abc00 2014-08-13 18:06:08.250519 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.250520 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_query query(info 0'0) from replica 45 2014-08-13 18:06:08.250563 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.250578 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250582 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250569 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 18:06:08.250597 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.250602 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250605 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250608 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a43640 pg_notify(24.b6(44) epoch 194496) v5 2014-08-13 18:06:08.250604 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.250603 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 2014-08-13 18:06:08.250637 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.250626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.250647 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.17 2014-08-13 18:06:08.250652 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a43640 2014-08-13 18:06:08.250644 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.250644 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.250666 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.250666 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Started advmap 2014-08-13 18:06:08.250686 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.250702 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250706 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250700 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [4,59,351] newacting [4,351,55] 2014-08-13 18:06:08.250704 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.250719 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.250723 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2874 2014-08-13 18:06:08.250739 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250742 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250745 7f5c835ef700 20 osd.4 194496 _dispatch 0x5422ee0 pg_query(24.350,24.3b7 epoch 194496) v3 2014-08-13 18:06:08.250740 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.250742 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary/Peering/WaitActingChange 0.836644 1 0.000229 2014-08-13 18:06:08.250761 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.837160 4 0.000348 2014-08-13 18:06:08.250765 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started/Primary 0.836679 0 0.000000 2014-08-13 18:06:08.250787 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.356 epoch 194496 2014-08-13 18:06:08.250791 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5422ee0 2014-08-13 18:06:08.250783 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.250786 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] exit Started 0.916085 0 0.000000 2014-08-13 18:06:08.250800 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.925119 0 0.000000 2014-08-13 18:06:08.250808 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] enter Reset 2014-08-13 18:06:08.250804 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(info 0'0) from replica 356 2014-08-13 18:06:08.250825 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194495 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194496 2014-08-13 18:06:08.250826 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.250837 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 356 2014-08-13 18:06:08.250849 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194496 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: Reset advmap 2014-08-13 18:06:08.250851 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.250868 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194496 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.250875 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.250891 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250894 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250885 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194496 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] acting_up_affected newup [4,59,351] newacting [4,351,55] 2014-08-13 18:06:08.250893 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.250915 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.250919 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.250903 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194496 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.250910 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'504 -> 194215'504 2014-08-13 18:06:08.250923 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38540 pg_notify(24.1f7(2) epoch 194496) v5 2014-08-13 18:06:08.250924 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351] r=0 lpr=194496 pi=193678-194494/40 crt=194422'15456 lcod 0'0 mlcod 0'0 inactive m=62] set_last_peering_reset 194496 2014-08-13 18:06:08.250935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.250964 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.55 2014-08-13 18:06:08.250967 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38540 2014-08-13 18:06:08.250964 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,59,351](4) acting [4,59,351](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,59,351](4) acting [4,59,351](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.250984 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194495/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] noting past interval(194495-194495 up [4,59,351](4) acting [4,59,351](4)) 2014-08-13 18:06:08.251003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] up [4,59,351] -> [4,59,351], acting [4,59,351] -> [4,351,55], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.251024 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] clear_primary_state 2014-08-13 18:06:08.251058 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] agent_stop 2014-08-13 18:06:08.251076 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change 2014-08-13 18:06:08.251095 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] publish_stats_to_osd 194496:25640 2014-08-13 18:06:08.251112 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:08.251130 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_copy_ops 2014-08-13 18:06:08.251145 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_flush_ops 2014-08-13 18:06:08.251161 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:08.251176 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:08.251191 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:08.251196 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 sending log((0'0,194215'504], crt=191648'365) 2014-08-13 18:06:08.251217 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change_cleanup 2014-08-13 18:06:08.251225 7f5c7c7e4700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4857bc0 192.168.1.103:6831/41157 2014-08-13 18:06:08.251235 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] on_change 2014-08-13 18:06:08.251254 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit NotTrimming 2014-08-13 18:06:08.251267 7f5c7c7e4700 10 osd.4 194496 note_peer_epoch osd.31 has 194496 2014-08-13 18:06:08.251271 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter NotTrimming 2014-08-13 18:06:08.251288 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] [4,59,351] -> [4,351,55], replicas changed 2014-08-13 18:06:08.251305 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] cancel_recovery 2014-08-13 18:06:08.251320 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] clear_recovery_state 2014-08-13 18:06:08.251338 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.251358 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map 2014-08-13 18:06:08.251348 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] flushed 2014-08-13 18:06:08.251377 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] update_heartbeat_peers 4,55,59,351,376 -> 4,55,59,351 2014-08-13 18:06:08.251398 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.251401 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] take_waiters 2014-08-13 18:06:08.251418 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Reset 0.000609 1 0.000713 2014-08-13 18:06:08.251439 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started 2014-08-13 18:06:08.251415 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.31 22.99( DNE v 194215'504 lc 0'0 (0'0,194215'504] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 18:06:08.251457 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Start 2014-08-13 18:06:08.251460 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 22.99( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.251474 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: transitioning to Primary 2014-08-13 18:06:08.251493 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] exit Start 0.000037 0 0.000000 2014-08-13 18:06:08.251512 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary 2014-08-13 18:06:08.251529 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Peering 2014-08-13 18:06:08.251546 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.251562 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.251581 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194495-194495 up [4,59,351](4) acting [4,59,351](4)) 2014-08-13 18:06:08.251600 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194492-194494 up [55,351,376](55) acting [55,351,376](55) maybe_went_rw) 2014-08-13 18:06:08.251620 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior interval(194491-194491 up [55,351,376](55) acting [351,55,376](351)) 2014-08-13 18:06:08.251639 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] PriorSet: build_prior final: probe 4,55,59,351,376 down blocked_by {} 2014-08-13 18:06:08.251657 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.251676 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194496:25641 2014-08-13 18:06:08.251694 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.55 2014-08-13 18:06:08.251697 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 sending log((0'0,194215'504], crt=191648'365) 2014-08-13 18:06:08.251723 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:08.251713 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.59 2014-08-13 18:06:08.251731 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.351 2014-08-13 18:06:08.251749 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: querying info from osd.376 2014-08-13 18:06:08.251758 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate peer osd.374 22.99( DNE v 194215'504 lc 0'0 (0'0,194215'504] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(29) 2014-08-13 18:06:08.251768 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.251786 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.251855 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.31 has 29 missing 2014-08-13 18:06:08.251870 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.374 has 29 missing 2014-08-13 18:06:08.251893 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.251885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.251904 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.251920 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 is on osd.4 2014-08-13 18:06:08.251936 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 is on osd.4 2014-08-13 18:06:08.251964 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.251944 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 2014-08-13 18:06:08.251969 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.251954 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 is on osd.4 2014-08-13 18:06:08.251979 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.251976 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 is on osd.4 2014-08-13 18:06:08.251997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 is on osd.4 2014-08-13 18:06:08.252011 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252014 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252016 7f5c835ef700 20 osd.4 194496 _dispatch 0x588abc0 pg_info(1 pgs e194496:22.4fe(2)) v4 2014-08-13 18:06:08.252014 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 is on osd.4 2014-08-13 18:06:08.252033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 is on osd.4 2014-08-13 18:06:08.252058 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4fe(2)) v4 from osd.337 2014-08-13 18:06:08.252063 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588abc0 2014-08-13 18:06:08.251986 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.252051 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 is on osd.4 2014-08-13 18:06:08.252076 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Started advmap 2014-08-13 18:06:08.252081 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 is on osd.4 2014-08-13 18:06:08.252097 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [4,55,61] newacting [4,50,65] 2014-08-13 18:06:08.252100 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 is on osd.4 2014-08-13 18:06:08.252115 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.252118 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 is on osd.4 2014-08-13 18:06:08.252141 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252145 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252135 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 is on osd.4 2014-08-13 18:06:08.252134 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary/Peering/WaitActingChange 0.837579 4 0.000266 2014-08-13 18:06:08.252154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 is on osd.4 2014-08-13 18:06:08.252156 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started/Primary 0.837610 0 0.000000 2014-08-13 18:06:08.252174 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252177 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252180 7f5c835ef700 20 osd.4 194496 _dispatch 0x524c7c0 pg_info(1 pgs e194496:22.653(1)) v4 2014-08-13 18:06:08.252171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 is on osd.4 2014-08-13 18:06:08.252177 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] exit Started 0.915547 0 0.000000 2014-08-13 18:06:08.252201 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 is on osd.4 2014-08-13 18:06:08.252207 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] enter Reset 2014-08-13 18:06:08.252219 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.252237 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.653(1)) v4 from osd.371 2014-08-13 18:06:08.252243 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524c7c0 2014-08-13 18:06:08.252228 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194495 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194496 2014-08-13 18:06:08.252239 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 is on osd.4 2014-08-13 18:06:08.252249 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194496 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: Reset advmap 2014-08-13 18:06:08.252259 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 is on osd.4 2014-08-13 18:06:08.252268 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194496 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.252279 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 is on osd.4 2014-08-13 18:06:08.252286 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194496 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] acting_up_affected newup [4,55,61] newacting [4,50,65] 2014-08-13 18:06:08.252303 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252307 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252297 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.252305 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194496 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.252316 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.252334 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252338 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252324 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61] r=0 lpr=194496 pi=193747-194494/36 crt=194422'15470 lcod 0'0 mlcod 0'0 inactive m=51] set_last_peering_reset 194496 2014-08-13 18:06:08.252341 7f5c835ef700 20 osd.4 194496 _dispatch 0x5422940 pg_query(24.206 epoch 194496) v3 2014-08-13 18:06:08.252334 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.252355 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 is on osd.4 2014-08-13 18:06:08.252363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,55,61](4) acting [4,55,61](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194492 generate_past_intervals interval(194495-194495 up [4,55,61](4) acting [4,55,61](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.252383 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.343 epoch 194496 2014-08-13 18:06:08.252387 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5422940 2014-08-13 18:06:08.252374 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 is on osd.4 2014-08-13 18:06:08.252384 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194495/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] noting past interval(194495-194495 up [4,55,61](4) acting [4,55,61](4)) 2014-08-13 18:06:08.252402 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 is on osd.4 2014-08-13 18:06:08.252404 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_query query(info 0'0) from replica 343 2014-08-13 18:06:08.252407 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] up [4,55,61] -> [4,55,61], acting [4,55,61] -> [4,50,65], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.252421 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 is on osd.4 2014-08-13 18:06:08.252432 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] clear_primary_state 2014-08-13 18:06:08.252453 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252457 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252444 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 is on osd.4 2014-08-13 18:06:08.252456 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] agent_stop 2014-08-13 18:06:08.252487 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252491 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252493 7f5c835ef700 20 osd.4 194496 _dispatch 0x54212c0 pg_query(24.20e epoch 194496) v3 2014-08-13 18:06:08.252482 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change 2014-08-13 18:06:08.252502 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] publish_stats_to_osd 194496:22529 2014-08-13 18:06:08.252462 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 18:06:08.252521 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:08.252530 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.15 epoch 194496 2014-08-13 18:06:08.252540 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x54212c0 2014-08-13 18:06:08.252531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate: adding 31 as a source 2014-08-13 18:06:08.252541 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_copy_ops 2014-08-13 18:06:08.252553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 also missing on osd.31 2014-08-13 18:06:08.252552 7f5c835ef700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_query query(info 0'0) from replica 15 2014-08-13 18:06:08.252561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_flush_ops 2014-08-13 18:06:08.252575 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 also missing on osd.31 2014-08-13 18:06:08.252579 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:08.252603 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252607 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252597 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 also missing on osd.31 2014-08-13 18:06:08.252597 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:08.252616 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:08.252617 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 also missing on osd.31 2014-08-13 18:06:08.252635 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252639 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252642 7f5c835ef700 20 osd.4 194496 _dispatch 0x21af620 pg_query(24.33a epoch 194496) v3 2014-08-13 18:06:08.252634 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change_cleanup 2014-08-13 18:06:08.252636 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 also missing on osd.31 2014-08-13 18:06:08.252654 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] on_change 2014-08-13 18:06:08.252657 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 also missing on osd.31 2014-08-13 18:06:08.252682 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.304 epoch 194496 2014-08-13 18:06:08.252686 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21af620 2014-08-13 18:06:08.252674 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit NotTrimming 2014-08-13 18:06:08.252677 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 also missing on osd.31 2014-08-13 18:06:08.252694 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter NotTrimming 2014-08-13 18:06:08.252697 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 also missing on osd.31 2014-08-13 18:06:08.252699 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(info 0'0) from replica 304 2014-08-13 18:06:08.252712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] [4,55,61] -> [4,50,65], replicas changed 2014-08-13 18:06:08.252716 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 also missing on osd.31 2014-08-13 18:06:08.252745 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252732 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] cancel_recovery 2014-08-13 18:06:08.252748 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252736 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 also missing on osd.31 2014-08-13 18:06:08.252750 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] clear_recovery_state 2014-08-13 18:06:08.252755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 also missing on osd.31 2014-08-13 18:06:08.252778 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252781 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252783 7f5c835ef700 20 osd.4 194496 _dispatch 0x588f8c0 pg_notify(22.3d5(28) epoch 194496) v5 2014-08-13 18:06:08.252770 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.252773 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 also missing on osd.31 2014-08-13 18:06:08.252793 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 also missing on osd.31 2014-08-13 18:06:08.252792 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map 2014-08-13 18:06:08.252829 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.286 2014-08-13 18:06:08.252814 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 also missing on osd.31 2014-08-13 18:06:08.252834 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588f8c0 2014-08-13 18:06:08.252815 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] update_heartbeat_peers 4,50,55,61,65,356 -> 4,50,55,61,65 2014-08-13 18:06:08.252841 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.252837 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 also missing on osd.31 2014-08-13 18:06:08.252845 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] take_waiters 2014-08-13 18:06:08.252856 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 also missing on osd.31 2014-08-13 18:06:08.252867 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Reset 0.000659 1 0.000806 2014-08-13 18:06:08.252889 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252877 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 also missing on osd.31 2014-08-13 18:06:08.252893 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252889 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started 2014-08-13 18:06:08.252896 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 also missing on osd.31 2014-08-13 18:06:08.252908 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Start 2014-08-13 18:06:08.252925 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.252928 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.252914 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 also missing on osd.31 2014-08-13 18:06:08.252933 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df24c0 pg_notify(24.3d(38) epoch 194496) v5 2014-08-13 18:06:08.252926 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: transitioning to Primary 2014-08-13 18:06:08.252935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 also missing on osd.31 2014-08-13 18:06:08.252948 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] exit Start 0.000040 0 0.000000 2014-08-13 18:06:08.252957 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 also missing on osd.31 2014-08-13 18:06:08.252975 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.303 2014-08-13 18:06:08.252979 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df24c0 2014-08-13 18:06:08.252971 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary 2014-08-13 18:06:08.252976 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 also missing on osd.31 2014-08-13 18:06:08.252991 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Peering 2014-08-13 18:06:08.252995 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 also missing on osd.31 2014-08-13 18:06:08.253010 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.253015 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 also missing on osd.31 2014-08-13 18:06:08.253034 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253038 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.253033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 also missing on osd.31 2014-08-13 18:06:08.253065 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253049 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194495-194495 up [4,55,61](4) acting [4,55,61](4)) 2014-08-13 18:06:08.253068 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253060 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 also missing on osd.31 2014-08-13 18:06:08.253074 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3480 pg_info(1 pgs e194496:22.1a0(2)) v4 2014-08-13 18:06:08.253072 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194491-194494 up [50,65,356](50) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:08.253080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 also missing on osd.31 2014-08-13 18:06:08.253100 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 also missing on osd.31 2014-08-13 18:06:08.253097 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior interval(194489-194490 up [55,61](55) acting [50,65,356](50) maybe_went_rw) 2014-08-13 18:06:08.253117 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1a0(2)) v4 from osd.328 2014-08-13 18:06:08.253123 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3480 2014-08-13 18:06:08.253120 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 also missing on osd.31 2014-08-13 18:06:08.253119 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] PriorSet: build_prior final: probe 4,50,55,61,65,356 down blocked_by {} 2014-08-13 18:06:08.253140 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.253163 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194496:22530 2014-08-13 18:06:08.253177 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253183 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253139 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 18:06:08.253183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.50 2014-08-13 18:06:08.253216 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253206 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] activate: adding 374 as a source 2014-08-13 18:06:08.253222 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352680 pg_notify(24.1f7(44) epoch 194496) v5 2014-08-13 18:06:08.253211 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.55 2014-08-13 18:06:08.253227 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22 191727'420 also missing on osd.374 2014-08-13 18:06:08.253232 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.61 2014-08-13 18:06:08.253247 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22 191727'398 also missing on osd.374 2014-08-13 18:06:08.253264 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.351 2014-08-13 18:06:08.253252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.65 2014-08-13 18:06:08.253268 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352680 2014-08-13 18:06:08.253268 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22 191648'343 also missing on osd.374 2014-08-13 18:06:08.253273 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: querying info from osd.356 2014-08-13 18:06:08.253287 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22 191727'387 also missing on osd.374 2014-08-13 18:06:08.253293 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.253306 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22 191648'2 also missing on osd.374 2014-08-13 18:06:08.253322 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253326 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253313 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.253325 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22 191727'369 also missing on osd.374 2014-08-13 18:06:08.253346 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22 191727'432 also missing on osd.374 2014-08-13 18:06:08.253362 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253364 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253366 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5cc0 pg_notify(22.39d(34) epoch 194496) v5 2014-08-13 18:06:08.253366 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22 191648'67 also missing on osd.374 2014-08-13 18:06:08.253386 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22 191648'202 also missing on osd.374 2014-08-13 18:06:08.253406 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.320 2014-08-13 18:06:08.253410 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5cc0 2014-08-13 18:06:08.253404 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22 191727'395 also missing on osd.374 2014-08-13 18:06:08.253427 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.253423 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22 191727'428 also missing on osd.374 2014-08-13 18:06:08.253448 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253451 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253441 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22 191727'440 also missing on osd.374 2014-08-13 18:06:08.253459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22 191648'196 also missing on osd.374 2014-08-13 18:06:08.253475 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22 191727'375 also missing on osd.374 2014-08-13 18:06:08.253474 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 18:06:08.253501 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253502 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.253504 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253506 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf0e00 pg_notify(24.9b epoch 194496) v5 2014-08-13 18:06:08.253495 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22 191648'166 also missing on osd.374 2014-08-13 18:06:08.253508 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.253514 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22 191727'368 also missing on osd.374 2014-08-13 18:06:08.253527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.253532 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22 191648'172 also missing on osd.374 2014-08-13 18:06:08.253548 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.367 2014-08-13 18:06:08.253553 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf0e00 2014-08-13 18:06:08.253548 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.253551 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22 191727'406 also missing on osd.374 2014-08-13 18:06:08.253570 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22 191648'310 also missing on osd.374 2014-08-13 18:06:08.253595 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253599 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d194899/rb.0.43b76.238e1f29.0000000003de/head//22 192674'448 also missing on osd.374 2014-08-13 18:06:08.253605 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22 191648'116 also missing on osd.374 2014-08-13 18:06:08.253627 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253630 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253632 7f5c835ef700 20 osd.4 194496 _dispatch 0x588e3c0 pg_info(1 pgs e194496:22.619) v4 2014-08-13 18:06:08.253622 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22 191727'397 also missing on osd.374 2014-08-13 18:06:08.253640 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22 191727'415 also missing on osd.374 2014-08-13 18:06:08.253653 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.253659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22 191727'399 also missing on osd.374 2014-08-13 18:06:08.253672 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.253685 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.619) v4 from osd.65 2014-08-13 18:06:08.253690 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588e3c0 2014-08-13 18:06:08.253678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22 191727'434 also missing on osd.374 2014-08-13 18:06:08.253689 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.253711 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.253698 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22 191727'413 also missing on osd.374 2014-08-13 18:06:08.253717 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22 191648'54 also missing on osd.374 2014-08-13 18:06:08.253742 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253746 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253735 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22 191727'423 also missing on osd.374 2014-08-13 18:06:08.253737 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 18:06:08.253760 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.253753 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] search_for_missing ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22 191648'21 also missing on osd.374 2014-08-13 18:06:08.253773 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253777 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253780 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352a00 pg_info(1 pgs e194496:22.1fb(2)) v4 2014-08-13 18:06:08.253765 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.253791 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.253811 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.253824 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1fb(2)) v4 from osd.373 2014-08-13 18:06:08.253829 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352a00 2014-08-13 18:06:08.253773 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {e3a80099/benchmark_data_storage0111-ib.pocstor_51285_object23726/head//22=191727'420,2abf1099/benchmark_data_storage0111-ib.pocstor_51285_object15113/head//22=191727'398,b59b2099/benchmark_data_storage0111-ib.pocstor_40745_object21535/head//22=191648'343,c8204099/benchmark_data_storage0111-ib.pocstor_51285_object10913/head//22=191727'387,e34d5099/benchmark_data_storage0111-ib.pocstor_40745_object184/head//22=191648'2,2ed46099/benchmark_data_storage0111-ib.pocstor_51285_object1569/head//22=191727'369,4a8d6099/benchmark_data_storage0111-ib.pocstor_51285_object30597/head//22=191727'432,a947099/benchmark_data_storage0111-ib.pocstor_40745_object3714/head//22=191648'67,d5218099/benchmark_data_storage0111-ib.pocstor_40745_object11892/head//22=191648'202,a6439099/benchmark_data_storage0111-ib.pocstor_51285_object14435/head//22=191727'395,2937b099/benchmark_data_storage0111-ib.pocstor_51285_object29447/head//22=191727'428,b315e099/benchmark_data_storage0111-ib.pocstor_51285_object38336/head//22=191727'440,3cbbe099/benchmark_data_storage0111-ib.pocstor_40745_object11570/head//22=191648'196,ac02f099/benchmark_data_storage0111-ib.pocstor_51285_object4554/head//22=191727'375,b3f30899/benchmark_data_storage0111-ib.pocstor_40745_object9877/head//22=191648'166,74960899/benchmark_data_storage0111-ib.pocstor_51285_object753/head//22=191727'368,51f12899/benchmark_data_storage0111-ib.pocstor_40745_object10199/head//22=191648'172,63752899/benchmark_data_storage0111-ib.pocstor_51285_object19336/head//22=191727'406,63b92899/benchmark_data_storage0111-ib.pocstor_40745_object19421/head//22=191648'310,8d194899/rb.0.43b76.238e1f29.0000000003de/head//22=192674'448,c6aa6899/benchmark_data_storage0111-ib.pocstor_40745_object6615/head//22=191648'116,e2268899/benchmark_data_storage0111-ib.pocstor_51285_object14938/head//22=191727'397,9ea3a899/benchmark_data_storage0111-ib.pocstor_51285_object21335/head//22=191727'415,7c1eb899/benchmark_data_storage0111-ib.pocstor_51285_object15493/head//22=191727'399,f9d0d899/benchmark_data_storage0111-ib.pocstor_51285_object32206/head//22=191727'434,1e84d899/benchmark_data_storage0111-ib.pocstor_51285_object21148/head//22=191727'413,1f45e899/benchmark_data_storage0111-ib.pocstor_40745_object3012/head//22=191648'54,b658f899/benchmark_data_storage0111-ib.pocstor_51285_object25205/head//22=191727'423,ac8bf899/benchmark_data_storage0111-ib.pocstor_40745_object1461/head//22=191648'21} 2014-08-13 18:06:08.253835 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.253851 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:08.253879 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253869 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 20,31,337,363,374 2014-08-13 18:06:08.253883 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253885 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.253886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.253904 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.253907 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.253911 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a440c0 pg_info(1 pgs e194496:22.3b9(2)) v4 2014-08-13 18:06:08.253906 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.253908 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.253925 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.253945 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.253929 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.253954 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3b9(2)) v4 from osd.397 2014-08-13 18:06:08.253958 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a440c0 2014-08-13 18:06:08.253950 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.253974 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,32] -- 356/356 2014-08-13 18:06:08.254002 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.254009 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254013 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254008 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:08.254034 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254036 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254026 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,32] 2014-08-13 18:06:08.254039 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea4c0 pg_info(1 pgs e194496:24.299(11)) v4 2014-08-13 18:06:08.254053 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.254043 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.254061 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.909425 4 0.000194 2014-08-13 18:06:08.254080 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:24.299(11)) v4 from osd.396 2014-08-13 18:06:08.254084 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea4c0 2014-08-13 18:06:08.254082 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] exit Started 0.909487 0 0.000000 2014-08-13 18:06:08.254102 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 18:06:08.254118 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194495 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.254136 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254115 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 18:06:08.254140 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254145 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.254135 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194496 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:08.254164 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254168 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194496 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.254154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.254170 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5080 pg_notify(22.48e(34) epoch 194496) v5 2014-08-13 18:06:08.254171 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194496 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [356,4,334] newacting [356,4,32] 2014-08-13 18:06:08.254174 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.254200 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.254218 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.376 2014-08-13 18:06:08.254221 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5080 2014-08-13 18:06:08.254220 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.254188 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194496 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.254240 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.254244 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334] r=1 lpr=194496 pi=193150-194494/44 crt=194422'18845 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.254261 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254264 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254260 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.254279 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2908 2014-08-13 18:06:08.254298 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254302 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254286 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [356,4,334](356) acting [356,4,334](356)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [356,4,334](356) acting [356,4,334](356)) : primary up 192969-194492 does not include interval 2014-08-13 18:06:08.254304 7f5c835ef700 20 osd.4 194496 _dispatch 0x524f000 pg_notify(2.2a(1) epoch 194496) v5 2014-08-13 18:06:08.254296 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.254306 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194495/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] noting past interval(194495-194495 up [356,4,334](356) acting [356,4,334](356)) 2014-08-13 18:06:08.254317 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.848186 6 0.000468 2014-08-13 18:06:08.254345 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.324 2014-08-13 18:06:08.254349 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524f000 2014-08-13 18:06:08.254329 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] up [356,4,334] -> [356,4,334], acting [356,4,334] -> [356,4,32], acting_primary 356 -> 356, up_primary 356 -> 356, role 1 -> 1 2014-08-13 18:06:08.254342 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.254354 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:08.254361 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.921680 0 0.000000 2014-08-13 18:06:08.254374 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.254387 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.254405 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254392 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:08.254409 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254405 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.254413 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:08.254430 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254434 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254437 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352f40 pg_notify(24.b6(2) epoch 194496) v5 2014-08-13 18:06:08.254428 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.254432 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:08.254447 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.254450 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:08.254466 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'497 -> 194215'497 2014-08-13 18:06:08.254482 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.327 2014-08-13 18:06:08.254468 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:08.254486 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352f40 2014-08-13 18:06:08.254489 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:08.254492 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.254508 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:08.254513 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 is up to date, queueing in pending_activators 2014-08-13 18:06:08.254526 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:08.254550 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254554 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254542 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:08.254538 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.28 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.254575 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254559 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.254561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.254578 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254581 7f5c835ef700 20 osd.4 194496 _dispatch 0x588f380 pg_info(1 pgs e194496:22.113) v4 2014-08-13 18:06:08.254579 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 is up to date, queueing in pending_activators 2014-08-13 18:06:08.254592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 18:06:08.254602 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] activate peer osd.304 22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.254622 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.113) v4 from osd.339 2014-08-13 18:06:08.254628 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588f380 2014-08-13 18:06:08.254575 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.254625 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.254636 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.254641 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.254659 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.254678 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254682 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254675 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.254701 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254690 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.254705 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254707 7f5c835ef700 20 osd.4 194496 _dispatch 0x588fc40 pg_notify(22.707(38) epoch 194496) v5 2014-08-13 18:06:08.254708 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 18:06:08.254728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 18:06:08.254751 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.254750 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.342 2014-08-13 18:06:08.254756 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588fc40 2014-08-13 18:06:08.254749 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.000648 1 0.000576 2014-08-13 18:06:08.254772 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started 2014-08-13 18:06:08.254793 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Start 2014-08-13 18:06:08.254809 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254813 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254799 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 18:06:08.254810 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray 2014-08-13 18:06:08.254826 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.254830 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000038 0 0.000000 2014-08-13 18:06:08.254833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.254856 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254864 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254852 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.254867 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df2a00 pg_notify(22.1f1(34) epoch 194496) v5 2014-08-13 18:06:08.254851 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray 2014-08-13 18:06:08.254874 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.254879 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.254893 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.254908 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.368 2014-08-13 18:06:08.254912 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df2a00 2014-08-13 18:06:08.254900 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.254913 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.254935 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.254952 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2661 2014-08-13 18:06:08.254967 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.254971 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.254967 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.254985 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.845875 3 0.000317 2014-08-13 18:06:08.255006 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.255010 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.255012 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bd40 pg_notify(22.f3(26) epoch 194496) v5 2014-08-13 18:06:08.255015 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.255004 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.255022 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.027979 0 0.000000 2014-08-13 18:06:08.263895 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.369 2014-08-13 18:06:08.263900 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bd40 2014-08-13 18:06:08.263918 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.263952 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.263956 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.263951 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194486 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.255059 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 18:06:08.255416 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.263976 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.263988 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.263994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.264009 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'497 -> 194215'497 2014-08-13 18:06:08.264000 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.264007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.264036 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264040 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264043 7f5c835ef700 20 osd.4 194496 _dispatch 0x588d400 pg_notify(24.9b(2) epoch 194496) v5 2014-08-13 18:06:08.264035 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.264040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.264046 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] flushed 2014-08-13 18:06:08.264064 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.264091 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.365 2014-08-13 18:06:08.264094 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588d400 2014-08-13 18:06:08.264122 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.264138 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264142 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264139 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.264156 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.264180 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264185 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.264184 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264200 7f5c835ef700 20 osd.4 194496 _dispatch 0x2000fc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.264211 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.264217 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.264220 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264222 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264237 7f5c817ec700 10 osd.4 194496 note_peer_epoch osd.351 now has 194496 2014-08-13 18:06:08.264246 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 2014-08-13 18:06:08.264242 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 18:06:08.264262 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264265 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264267 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed57c0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.264267 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.264273 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.264278 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.264280 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264282 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264274 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.264302 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264292 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.264305 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264312 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a68c40 pg_notify(24.30b(41) epoch 194496) v5 2014-08-13 18:06:08.264312 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.264316 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 sending log((0'0,194215'497], crt=191727'433) 2014-08-13 18:06:08.264346 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.264353 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.10 2014-08-13 18:06:08.264357 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a68c40 2014-08-13 18:06:08.264394 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264398 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.356 22.4df( DNE v 194215'497 lc 0'0 (0'0,194215'497] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 18:06:08.264392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.264403 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 22.4df( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.264424 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264428 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264412 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.264430 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b480 pg_info(1 pgs e194496:22.22c) v4 2014-08-13 18:06:08.264430 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.264449 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.264466 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.22c) v4 from osd.301 2014-08-13 18:06:08.264471 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b480 2014-08-13 18:06:08.264480 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 18:06:08.264504 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.264509 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264513 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264511 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.264533 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264535 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264538 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357c40 pg_info(1 pgs e194496:22.362) v4 2014-08-13 18:06:08.264530 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.264548 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.264574 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.362) v4 from osd.58 2014-08-13 18:06:08.264566 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.264579 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357c40 2014-08-13 18:06:08.264587 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.264606 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.264621 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264625 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264609 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 sending log((0'0,194215'497], crt=191727'433) 2014-08-13 18:06:08.264633 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4afcd00 don't know epoch, doing nothing 2014-08-13 18:06:08.264625 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2709 2014-08-13 18:06:08.264647 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264650 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264652 7f5c835ef700 20 osd.4 194496 _dispatch 0x5151440 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.264659 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.264643 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.264664 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.264668 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264670 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264662 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate peer osd.357 22.4df( DNE v 194215'497 lc 0'0 (0'0,194215'497] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(35) 2014-08-13 18:06:08.264669 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.857321 6 0.000447 2014-08-13 18:06:08.264690 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264692 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264695 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421680 pg_query(24.5e epoch 194496) v3 2014-08-13 18:06:08.264693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.264710 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.911087 0 0.000000 2014-08-13 18:06:08.264732 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.289 epoch 194496 2014-08-13 18:06:08.264736 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421680 2014-08-13 18:06:08.264734 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.264752 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.264748 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_query query(info 0'0) from replica 289 2014-08-13 18:06:08.264775 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.264776 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.356 has 35 missing 2014-08-13 18:06:08.264793 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264797 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264794 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.264795 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.357 has 35 missing 2014-08-13 18:06:08.264823 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264827 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264812 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'504 -> 194215'504 2014-08-13 18:06:08.264813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 is on osd.4 2014-08-13 18:06:08.264829 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee3cc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.264836 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.264841 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.264844 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264847 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264839 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.264835 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 is on osd.4 2014-08-13 18:06:08.264866 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264869 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264871 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a432c0 pg_info(1 pgs e194496:22.113) v4 2014-08-13 18:06:08.264861 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 is up to date, queueing in pending_activators 2014-08-13 18:06:08.264862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 is on osd.4 2014-08-13 18:06:08.264882 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 is on osd.4 2014-08-13 18:06:08.264887 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.50 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.264909 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.113) v4 from osd.45 2014-08-13 18:06:08.264914 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a432c0 2014-08-13 18:06:08.264901 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 is on osd.4 2014-08-13 18:06:08.264908 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.264919 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 is on osd.4 2014-08-13 18:06:08.264931 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 is up to date, queueing in pending_activators 2014-08-13 18:06:08.264940 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 is on osd.4 2014-08-13 18:06:08.264955 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264959 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264957 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.264959 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.264979 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.264982 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264984 7f5c835ef700 20 osd.4 194496 _dispatch 0x2002400 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.264976 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 is on osd.4 2014-08-13 18:06:08.264992 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.264997 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.264983 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.265000 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265004 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.264994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 is on osd.4 2014-08-13 18:06:08.265002 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.265023 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265026 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265028 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38380 pg_notify(24.181(17) epoch 194496) v5 2014-08-13 18:06:08.265014 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 is on osd.4 2014-08-13 18:06:08.265021 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.265033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 is on osd.4 2014-08-13 18:06:08.265041 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.265053 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 is on osd.4 2014-08-13 18:06:08.265070 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.55 2014-08-13 18:06:08.265073 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38380 2014-08-13 18:06:08.265061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.265073 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 is on osd.4 2014-08-13 18:06:08.265092 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 is on osd.4 2014-08-13 18:06:08.265115 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265119 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 is on osd.4 2014-08-13 18:06:08.265127 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 is on osd.4 2014-08-13 18:06:08.265143 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.265152 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265154 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265143 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 is on osd.4 2014-08-13 18:06:08.265157 7f5c835ef700 20 osd.4 194496 _dispatch 0x588aa00 pg_notify(24.1bb(40) epoch 194496) v5 2014-08-13 18:06:08.265161 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 is on osd.4 2014-08-13 18:06:08.265180 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.265190 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.265205 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.337 2014-08-13 18:06:08.265209 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588aa00 2014-08-13 18:06:08.265208 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 is on osd.4 2014-08-13 18:06:08.265210 7f5c7bde3700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4493f40 192.168.1.103:6811/37658 2014-08-13 18:06:08.265226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 is on osd.4 2014-08-13 18:06:08.265249 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265251 7f5c7bde3700 10 osd.4 194496 note_peer_epoch osd.17 has 194496 2014-08-13 18:06:08.265253 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265256 7f5c7bde3700 7 osd.4 194496 do_notify osd 17 on 1 PGs 2014-08-13 18:06:08.265250 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 is on osd.4 2014-08-13 18:06:08.265273 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 don't know epoch, doing nothing 2014-08-13 18:06:08.265276 7f5c7bde3700 7 osd.4 194496 do_notify osd 19 on 1 PGs 2014-08-13 18:06:08.265268 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 is on osd.4 2014-08-13 18:06:08.265288 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265292 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265294 7f5c835ef700 20 osd.4 194496 _dispatch 0x524ee40 pg_info(1 pgs e194496:22.362) v4 2014-08-13 18:06:08.265296 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:08.265300 7f5c7bde3700 7 osd.4 194496 do_notify osd 30 on 1 PGs 2014-08-13 18:06:08.265286 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 is on osd.4 2014-08-13 18:06:08.265305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 is on osd.4 2014-08-13 18:06:08.265322 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:08.265326 7f5c7bde3700 7 osd.4 194496 do_notify osd 289 on 1 PGs 2014-08-13 18:06:08.265333 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.362) v4 from osd.343 2014-08-13 18:06:08.265323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 is on osd.4 2014-08-13 18:06:08.265338 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524ee40 2014-08-13 18:06:08.265349 7f5c7bde3700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4b12100 192.168.1.104:6808/62751 2014-08-13 18:06:08.265341 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 is on osd.4 2014-08-13 18:06:08.265376 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265376 7f5c7bde3700 10 osd.4 194496 note_peer_epoch osd.334 has 194496 2014-08-13 18:06:08.265379 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265358 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.265380 7f5c7bde3700 7 osd.4 194496 do_notify osd 334 on 1 PGs 2014-08-13 18:06:08.265395 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:08.265383 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 is on osd.4 2014-08-13 18:06:08.265398 7f5c7bde3700 7 osd.4 194496 do_notify osd 337 on 1 PGs 2014-08-13 18:06:08.265399 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265403 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265406 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a39180 pg_notify(22.3e4(1) epoch 194496) v5 2014-08-13 18:06:08.265401 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 is on osd.4 2014-08-13 18:06:08.265419 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.265422 7f5c7bde3700 7 osd.4 194496 do_notify osd 356 on 2 PGs 2014-08-13 18:06:08.265420 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.265443 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 don't know epoch, doing nothing 2014-08-13 18:06:08.265449 7f5c7bde3700 7 osd.4 194496 do_notify osd 358 on 1 PGs 2014-08-13 18:06:08.265450 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.303 2014-08-13 18:06:08.265438 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 is on osd.4 2014-08-13 18:06:08.265454 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a39180 2014-08-13 18:06:08.265456 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 is on osd.4 2014-08-13 18:06:08.265472 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.265475 7f5c7bde3700 7 osd.4 194496 do_notify osd 367 on 1 PGs 2014-08-13 18:06:08.265474 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 is on osd.4 2014-08-13 18:06:08.265493 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265497 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265499 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4851e40 don't know epoch, doing nothing 2014-08-13 18:06:08.265502 7f5c7bde3700 7 osd.4 194496 do_notify osd 371 on 1 PGs 2014-08-13 18:06:08.265517 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265520 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265522 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1880 pg_notify(24.181(17) epoch 194496) v5 2014-08-13 18:06:08.265524 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:08.265528 7f5c7bde3700 7 osd.4 194496 do_notify osd 373 on 1 PGs 2014-08-13 18:06:08.265546 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b10b00 don't know epoch, doing nothing 2014-08-13 18:06:08.265550 7f5c7bde3700 7 osd.4 194496 do_notify osd 397 on 1 PGs 2014-08-13 18:06:08.265492 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 18:06:08.265560 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.320 2014-08-13 18:06:08.265565 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1880 2014-08-13 18:06:08.265559 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate: adding 356 as a source 2014-08-13 18:06:08.265573 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 2014-08-13 18:06:08.265578 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.17 on 1 PGs 2014-08-13 18:06:08.265595 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:08.265580 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 also missing on osd.356 2014-08-13 18:06:08.265598 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.28 on 1 PGs 2014-08-13 18:06:08.265602 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265606 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265600 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 also missing on osd.356 2014-08-13 18:06:08.265619 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:08.265623 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.50 on 1 PGs 2014-08-13 18:06:08.265628 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265631 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265620 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 also missing on osd.356 2014-08-13 18:06:08.265634 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352840 pg_notify(22.3d9(47) epoch 194496) v5 2014-08-13 18:06:08.265643 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:08.265650 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.55 on 2 PGs 2014-08-13 18:06:08.265640 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 also missing on osd.356 2014-08-13 18:06:08.265670 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3b00c60 don't know epoch, doing nothing 2014-08-13 18:06:08.265659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 also missing on osd.356 2014-08-13 18:06:08.265674 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.59 on 1 PGs 2014-08-13 18:06:08.265676 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.373 2014-08-13 18:06:08.265680 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352840 2014-08-13 18:06:08.265678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 also missing on osd.356 2014-08-13 18:06:08.265695 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:08.265698 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.61 on 1 PGs 2014-08-13 18:06:08.265698 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 also missing on osd.356 2014-08-13 18:06:08.265716 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265719 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265719 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:08.265724 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.65 on 1 PGs 2014-08-13 18:06:08.265717 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 also missing on osd.356 2014-08-13 18:06:08.265745 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c51080 don't know epoch, doing nothing 2014-08-13 18:06:08.265735 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 also missing on osd.356 2014-08-13 18:06:08.265749 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.315 on 1 PGs 2014-08-13 18:06:08.265759 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265762 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265764 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a680 pg_notify(24.1f7(40) epoch 194496) v5 2014-08-13 18:06:08.265755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 also missing on osd.356 2014-08-13 18:06:08.265773 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4c58420 don't know epoch, doing nothing 2014-08-13 18:06:08.265776 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.327 on 1 PGs 2014-08-13 18:06:08.265774 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 also missing on osd.356 2014-08-13 18:06:08.265796 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.265800 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.330 on 1 PGs 2014-08-13 18:06:08.265794 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 also missing on osd.356 2014-08-13 18:06:08.265817 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.376 2014-08-13 18:06:08.265820 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a680 2014-08-13 18:06:08.265821 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 2014-08-13 18:06:08.265825 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.351 on 1 PGs 2014-08-13 18:06:08.265813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 also missing on osd.356 2014-08-13 18:06:08.265832 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 also missing on osd.356 2014-08-13 18:06:08.265845 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.265848 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.356 on 1 PGs 2014-08-13 18:06:08.265860 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265864 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265863 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:08.265879 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.376 on 1 PGs 2014-08-13 18:06:08.265899 7f5c7bde3700 20 osd.4 194496 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 50 2014-08-13 18:06:08.265909 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:08.265912 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265915 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.265917 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cb40 pg_notify(1.2b(1) epoch 194496) v5 2014-08-13 18:06:08.265921 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 2014-08-13 18:06:08.265850 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 also missing on osd.356 2014-08-13 18:06:08.265929 7f5c7bde3700 20 osd.4 194496 Sending info 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 289 2014-08-13 18:06:08.265939 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:08.265955 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.324 2014-08-13 18:06:08.265959 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cb40 2014-08-13 18:06:08.265940 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 also missing on osd.356 2014-08-13 18:06:08.265967 7f5c7bde3700 10 osd.4 194496 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 18:06:08.265965 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 also missing on osd.356 2014-08-13 18:06:08.266000 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.265988 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 also missing on osd.356 2014-08-13 18:06:08.266003 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266007 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 also missing on osd.356 2014-08-13 18:06:08.266026 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266030 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266033 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bb80 pg_notify(22.775(1) epoch 194496) v5 2014-08-13 18:06:08.266025 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 also missing on osd.356 2014-08-13 18:06:08.266046 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 also missing on osd.356 2014-08-13 18:06:08.266071 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.369 2014-08-13 18:06:08.266074 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bb80 2014-08-13 18:06:08.266064 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 also missing on osd.356 2014-08-13 18:06:08.266066 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] handle_advance_map [4,344,296]/[4,296,320] -- 4/4 2014-08-13 18:06:08.266084 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 also missing on osd.356 2014-08-13 18:06:08.266105 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.266110 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266113 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266104 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 also missing on osd.356 2014-08-13 18:06:08.266113 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Started advmap 2014-08-13 18:06:08.266132 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266136 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266124 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 also missing on osd.356 2014-08-13 18:06:08.266140 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4ec0 pg_info(1 pgs e194496:22.707) v4 2014-08-13 18:06:08.266135 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [4,296,320] 2014-08-13 18:06:08.266146 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 also missing on osd.356 2014-08-13 18:06:08.266158 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.266129 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.266181 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.707) v4 from osd.310 2014-08-13 18:06:08.266194 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4ec0 2014-08-13 18:06:08.266166 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 also missing on osd.356 2014-08-13 18:06:08.266186 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.266183 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started/Stray 0.910972 4 0.000194 2014-08-13 18:06:08.266201 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 also missing on osd.356 2014-08-13 18:06:08.266217 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] exit Started 0.911054 0 0.000000 2014-08-13 18:06:08.266234 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266238 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266223 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 also missing on osd.356 2014-08-13 18:06:08.266240 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] enter Reset 2014-08-13 18:06:08.266257 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266243 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 also missing on osd.356 2014-08-13 18:06:08.266261 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266263 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6580 pg_info(1 pgs e194496:22.1f1) v4 2014-08-13 18:06:08.266258 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194495 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194496 2014-08-13 18:06:08.266263 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 also missing on osd.356 2014-08-13 18:06:08.266283 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 also missing on osd.356 2014-08-13 18:06:08.266282 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194496 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: Reset advmap 2014-08-13 18:06:08.266308 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1f1) v4 from osd.47 2014-08-13 18:06:08.266312 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6580 2014-08-13 18:06:08.266303 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 also missing on osd.356 2014-08-13 18:06:08.266302 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194496 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:08.266325 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 also missing on osd.356 2014-08-13 18:06:08.266325 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194496 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] acting_up_affected newup [4,344,296] newacting [4,296,320] 2014-08-13 18:06:08.266352 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266356 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266346 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 also missing on osd.356 2014-08-13 18:06:08.266346 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194496 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.266374 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266377 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266380 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353640 pg_info(1 pgs e194496:22.619) v4 2014-08-13 18:06:08.266366 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[320,307,296] r=-1 lpr=194496 pi=193679-194494/41 crt=194422'19613 lcod 0'0 remapped NOTIFY m=47] set_last_peering_reset 194496 2014-08-13 18:06:08.266406 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,344,296](4) acting [320,307,296](320)): not rw, up_thru 194492 up_from 192979 last_epoch_clean 194466 generate_past_intervals interval(194495-194495 up [4,344,296](4) acting [320,307,296](320)) : primary up 192979-194492 does not include interval 2014-08-13 18:06:08.266365 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 18:06:08.266424 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.619) v4 from osd.62 2014-08-13 18:06:08.266429 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353640 2014-08-13 18:06:08.266428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] activate: adding 357 as a source 2014-08-13 18:06:08.266426 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194495/194492) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] noting past interval(194495-194495 up [4,344,296](4) acting [320,307,296](320)) 2014-08-13 18:06:08.266449 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22 191648'37 also missing on osd.357 2014-08-13 18:06:08.266466 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266469 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266450 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] up [4,344,296] -> [4,344,296], acting [320,307,296] -> [4,296,320], acting_primary 320 -> 4, up_primary 4 -> 4, role -1 -> 0 2014-08-13 18:06:08.266470 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22 191727'382 also missing on osd.357 2014-08-13 18:06:08.266487 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266491 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266476 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] clear_primary_state 2014-08-13 18:06:08.266496 7f5c835ef700 20 osd.4 194496 _dispatch 0x53547c0 pg_info(1 pgs e194496:22.3e4) v4 2014-08-13 18:06:08.266489 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22 191727'358 also missing on osd.357 2014-08-13 18:06:08.266500 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] agent_stop 2014-08-13 18:06:08.266510 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22 191727'377 also missing on osd.357 2014-08-13 18:06:08.266521 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:08.266540 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3e4) v4 from osd.313 2014-08-13 18:06:08.266545 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53547c0 2014-08-13 18:06:08.266533 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22 191727'412 also missing on osd.357 2014-08-13 18:06:08.266544 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] publish_stats_to_osd 194496:28885 2014-08-13 18:06:08.266555 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22 191727'425 also missing on osd.357 2014-08-13 18:06:08.266581 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266584 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266566 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:08.266575 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22 191648'7 also missing on osd.357 2014-08-13 18:06:08.266604 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266606 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266594 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22 191727'392 also missing on osd.357 2014-08-13 18:06:08.266609 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351880 pg_info(1 pgs e194496:22.1f1) v4 2014-08-13 18:06:08.266594 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] cancel_copy_ops 2014-08-13 18:06:08.266615 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22 191648'39 also missing on osd.357 2014-08-13 18:06:08.266617 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] cancel_flush_ops 2014-08-13 18:06:08.266634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22 191648'191 also missing on osd.357 2014-08-13 18:06:08.266636 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:08.266651 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1f1) v4 from osd.5 2014-08-13 18:06:08.266657 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351880 2014-08-13 18:06:08.266654 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22 191727'355 also missing on osd.357 2014-08-13 18:06:08.266655 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:08.266674 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22 191727'431 also missing on osd.357 2014-08-13 18:06:08.266675 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] requeue_ops 2014-08-13 18:06:08.266694 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266698 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266694 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22 191648'212 also missing on osd.357 2014-08-13 18:06:08.266694 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change_cleanup 2014-08-13 18:06:08.266718 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266721 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266723 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38fc0 pg_notify(22.481(42) epoch 194496) v5 2014-08-13 18:06:08.266713 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22 191648'319 also missing on osd.357 2014-08-13 18:06:08.266714 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] on_change 2014-08-13 18:06:08.266733 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22 191727'381 also missing on osd.357 2014-08-13 18:06:08.266737 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] exit NotTrimming 2014-08-13 18:06:08.266762 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.303 2014-08-13 18:06:08.266766 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38fc0 2014-08-13 18:06:08.266752 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22 191648'125 also missing on osd.357 2014-08-13 18:06:08.266758 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped NOTIFY m=47] enter NotTrimming 2014-08-13 18:06:08.266774 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22 191648'149 also missing on osd.357 2014-08-13 18:06:08.266776 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] clear_stats 2014-08-13 18:06:08.266805 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266794 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22 191727'413 also missing on osd.357 2014-08-13 18:06:08.266809 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266796 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops 2014-08-13 18:06:08.266814 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22 191727'360 also missing on osd.357 2014-08-13 18:06:08.266815 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] on_role_change 2014-08-13 18:06:08.266833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22 191727'398 also missing on osd.357 2014-08-13 18:06:08.266833 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops 2014-08-13 18:06:08.266859 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266853 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22 191648'140 also missing on osd.357 2014-08-13 18:06:08.266852 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] cancel_recovery 2014-08-13 18:06:08.266870 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266872 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a4c0 pg_info(1 pgs e194496:22.226(1)) v4 2014-08-13 18:06:08.266870 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] clear_recovery_state 2014-08-13 18:06:08.266872 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22 191727'378 also missing on osd.357 2014-08-13 18:06:08.266892 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22 191648'326 also missing on osd.357 2014-08-13 18:06:08.266893 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.266915 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.226(1)) v4 from osd.376 2014-08-13 18:06:08.266921 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a4c0 2014-08-13 18:06:08.266913 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22 191727'414 also missing on osd.357 2014-08-13 18:06:08.266916 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_activate_map 2014-08-13 18:06:08.266932 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22 191648'209 also missing on osd.357 2014-08-13 18:06:08.266938 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] update_heartbeat_peers -> 4,296,320,344 2014-08-13 18:06:08.266960 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.266961 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266965 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266951 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22 191648'101 also missing on osd.357 2014-08-13 18:06:08.266963 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] take_waiters 2014-08-13 18:06:08.266971 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22 191648'329 also missing on osd.357 2014-08-13 18:06:08.266987 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.266991 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.266994 7f5c835ef700 20 osd.4 194496 _dispatch 0x588cec0 pg_info(1 pgs e194496:22.22c) v4 2014-08-13 18:06:08.266985 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Reset 0.000745 1 0.000803 2014-08-13 18:06:08.266991 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22 191727'379 also missing on osd.357 2014-08-13 18:06:08.267010 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22 191727'415 also missing on osd.357 2014-08-13 18:06:08.267009 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started 2014-08-13 18:06:08.267033 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.22c) v4 from osd.324 2014-08-13 18:06:08.267038 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588cec0 2014-08-13 18:06:08.267030 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22 191648'318 also missing on osd.357 2014-08-13 18:06:08.267032 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Start 2014-08-13 18:06:08.267053 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22 191648'34 also missing on osd.357 2014-08-13 18:06:08.267061 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: transitioning to Primary 2014-08-13 18:06:08.267072 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22 191727'406 also missing on osd.357 2014-08-13 18:06:08.267084 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] exit Start 0.000052 0 0.000000 2014-08-13 18:06:08.267089 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22 191727'418 also missing on osd.357 2014-08-13 18:06:08.267109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22 191727'368 also missing on osd.357 2014-08-13 18:06:08.267108 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary 2014-08-13 18:06:08.267128 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22 191727'384 also missing on osd.357 2014-08-13 18:06:08.267202 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267240 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267195 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {6df614df/benchmark_data_storage0111-ib.pocstor_40745_object2194/head//22=191648'37,a02814df/benchmark_data_storage0111-ib.pocstor_51285_object13111/head//22=191727'382,e2d814df/benchmark_data_storage0111-ib.pocstor_51285_object3588/head//22=191727'358,fd9914df/benchmark_data_storage0111-ib.pocstor_51285_object9107/head//22=191727'377,10b934df/benchmark_data_storage0111-ib.pocstor_51285_object26109/head//22=191727'412,579144df/benchmark_data_storage0111-ib.pocstor_51285_object35529/head//22=191727'425,c5d054df/benchmark_data_storage0111-ib.pocstor_40745_object416/head//22=191648'7,51d064df/benchmark_data_storage0111-ib.pocstor_51285_object17351/head//22=191727'392,10a074df/benchmark_data_storage0111-ib.pocstor_40745_object2227/head//22=191648'39,88c274df/benchmark_data_storage0111-ib.pocstor_40745_object12481/head//22=191648'191,5b5284df/benchmark_data_storage0111-ib.pocstor_51285_object2975/head//22=191727'355,ef5984df/benchmark_data_storage0111-ib.pocstor_51285_object37977/head//22=191727'431,dca394df/benchmark_data_storage0111-ib.pocstor_40745_object13430/head//22=191648'212,2b6494df/benchmark_data_storage0111-ib.pocstor_40745_object21296/head//22=191648'319,2665b4df/benchmark_data_storage0111-ib.pocstor_51285_object11108/head//22=191727'381,d2bb4df/benchmark_data_storage0111-ib.pocstor_40745_object8183/head//22=191648'125,2f0eb4df/benchmark_data_storage0111-ib.pocstor_40745_object10004/head//22=191648'149,c8fc4df/benchmark_data_storage0111-ib.pocstor_51285_object26589/head//22=191727'413,60add4df/benchmark_data_storage0111-ib.pocstor_51285_object3806/head//22=191727'360,92151cdf/benchmark_data_storage0111-ib.pocstor_51285_object20350/head//22=191727'398,999c1cdf/benchmark_data_storage0111-ib.pocstor_40745_object9291/head//22=191648'140,55df2cdf/benchmark_data_storage0111-ib.pocstor_51285_object9202/head//22=191727'378,7c733cdf/benchmark_data_storage0111-ib.pocstor_40745_object21668/head//22=191648'326,36873cdf/benchmark_data_storage0111-ib.pocstor_51285_object26999/head//22=191727'414,78ca3cdf/benchmark_data_storage0111-ib.pocstor_40745_object13382/head//22=191648'209,3d6d5cdf/benchmark_data_storage0111-ib.pocstor_40745_object6389/head//22=191648'101,19ce6cdf/benchmark_data_storage0111-ib.pocstor_40745_object21821/head//22=191648'329,b90b9cdf/benchmark_data_storage0111-ib.pocstor_51285_object10100/head//22=191727'379,777eacdf/benchmark_data_storage0111-ib.pocstor_51285_object27588/head//22=191727'415,a580ccdf/benchmark_data_storage0111-ib.pocstor_40745_object21240/head//22=191648'318,cef5dcdf/benchmark_data_storage0111-ib.pocstor_40745_object1922/head//22=191648'34,a33fdcdf/benchmark_data_storage0111-ib.pocstor_51285_object23325/head//22=191727'406,d6aecdf/benchmark_data_storage0111-ib.pocstor_51285_object28170/head//22=191727'418,8d5becdf/benchmark_data_storage0111-ib.pocstor_51285_object7294/head//22=191727'368,4699fcdf/benchmark_data_storage0111-ib.pocstor_51285_object14089/head//22=191727'384} 2014-08-13 18:06:08.267263 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.267281 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267284 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267286 7f5c835ef700 20 osd.4 194496 _dispatch 0x588af40 pg_notify(24.b6 epoch 194496) v5 2014-08-13 18:06:08.267279 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.267299 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 352,356,357,371,380 2014-08-13 18:06:08.267317 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.267334 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.267358 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.28 2014-08-13 18:06:08.267363 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588af40 2014-08-13 18:06:08.267352 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.267370 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.267469 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.267470 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267476 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267526 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267530 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267522 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 18:06:08.267533 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a681c0 pg_notify(24.136 epoch 194496) v5 2014-08-13 18:06:08.267549 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.267557 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.267576 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.267600 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.61 2014-08-13 18:06:08.267603 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a681c0 2014-08-13 18:06:08.267593 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.267612 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.267630 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.267647 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.267665 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2616 2014-08-13 18:06:08.267681 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.267706 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267711 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267699 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.854438 3 0.000318 2014-08-13 18:06:08.267719 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.267734 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267743 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267745 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ae540 pg_query(24.350 epoch 194496) v3 2014-08-13 18:06:08.267734 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.021635 0 0.000000 2014-08-13 18:06:08.267758 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.267774 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.267796 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.267810 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.356 epoch 194496 2014-08-13 18:06:08.267815 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ae540 2014-08-13 18:06:08.267812 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.267827 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'480 -> 194215'480 2014-08-13 18:06:08.267828 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193151) from replica 356 2014-08-13 18:06:08.267851 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.267889 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267892 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267929 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.267933 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.267935 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a381c0 pg_notify(24.136 epoch 194496) v5 2014-08-13 18:06:08.267959 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.267994 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.55 2014-08-13 18:06:08.267998 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a381c0 2014-08-13 18:06:08.267993 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.267128 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Peering 2014-08-13 18:06:08.268061 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268064 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268063 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.268104 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268108 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268092 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:08.268126 7f5c835ef700 20 osd.4 194496 _dispatch 0x588e200 pg_notify(24.136(36) epoch 194496) v5 2014-08-13 18:06:08.268114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194495-194495 up [4,344,296](4) acting [320,307,296](320)) 2014-08-13 18:06:08.268135 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194492-194494 up [320,307,55](320) acting [320,307,296](320) maybe_went_rw) 2014-08-13 18:06:08.268156 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior interval(194491-194491 up [320,307,55](320) acting [296,320](296)) 2014-08-13 18:06:08.268185 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.65 2014-08-13 18:06:08.268188 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588e200 2014-08-13 18:06:08.268176 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] PriorSet: build_prior final: probe 4,296,307,320,344 down blocked_by {} 2014-08-13 18:06:08.268229 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268216 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.268233 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268239 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] publish_stats_to_osd 194496:28886 2014-08-13 18:06:08.268260 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268264 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268266 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a300 pg_notify(24.1f7(40) epoch 194496) v5 2014-08-13 18:06:08.268259 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.296 2014-08-13 18:06:08.268281 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.307 2014-08-13 18:06:08.268312 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.376 2014-08-13 18:06:08.268299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.320 2014-08-13 18:06:08.268316 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a300 2014-08-13 18:06:08.268318 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: querying info from osd.344 2014-08-13 18:06:08.268352 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268339 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.268355 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268356 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.268390 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268393 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268395 7f5c835ef700 20 osd.4 194496 _dispatch 0x54230c0 pg_query(24.5e epoch 194496) v3 2014-08-13 18:06:08.268462 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.289 epoch 194496 2014-08-13 18:06:08.268466 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x54230c0 2014-08-13 18:06:08.268479 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.268503 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192968) from replica 289 2014-08-13 18:06:08.268530 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,61] -- 15/15 2014-08-13 18:06:08.268563 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.268569 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268573 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268570 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:08.268622 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268627 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268630 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5080 pg_notify(24.136(36) epoch 194496) v5 2014-08-13 18:06:08.268588 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,61] 2014-08-13 18:06:08.268643 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.268692 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.356 2014-08-13 18:06:08.268696 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5080 2014-08-13 18:06:08.268684 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started/Stray 0.907374 4 0.000188 2014-08-13 18:06:08.268698 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 sending log((0'0,194215'480], crt=191727'416) 2014-08-13 18:06:08.268710 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] exit Started 0.907445 0 0.000000 2014-08-13 18:06:08.268728 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:08.268738 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.268732 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] enter Reset 2014-08-13 18:06:08.268748 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.268811 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.344 22.39d( DNE v 194215'480 lc 0'0 (0'0,194215'480] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 18:06:08.268839 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 22.39d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.269009 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269015 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269018 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38000 pg_notify(24.1f7(2) epoch 194496) v5 2014-08-13 18:06:08.269058 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.55 2014-08-13 18:06:08.269062 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38000 2014-08-13 18:06:08.269044 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 sending log((0'0,194215'480], crt=191727'416) 2014-08-13 18:06:08.269070 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:08.269103 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269106 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate peer osd.373 22.39d( DNE v 194215'480 lc 0'0 (0'0,194215'480] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(23) 2014-08-13 18:06:08.269127 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269130 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269132 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dead80 pg_notify(24.1f7(44) epoch 194496) v5 2014-08-13 18:06:08.269179 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.351 2014-08-13 18:06:08.269182 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dead80 2014-08-13 18:06:08.269186 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.344 has 23 missing 2014-08-13 18:06:08.269204 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.373 has 23 missing 2014-08-13 18:06:08.269219 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 is on osd.4 2014-08-13 18:06:08.269237 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 is on osd.4 2014-08-13 18:06:08.269253 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 is on osd.4 2014-08-13 18:06:08.269269 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 is on osd.4 2014-08-13 18:06:08.269285 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 is on osd.4 2014-08-13 18:06:08.269301 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 is on osd.4 2014-08-13 18:06:08.269316 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 is on osd.4 2014-08-13 18:06:08.269332 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 is on osd.4 2014-08-13 18:06:08.269347 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 is on osd.4 2014-08-13 18:06:08.269363 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 is on osd.4 2014-08-13 18:06:08.269379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 is on osd.4 2014-08-13 18:06:08.269396 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.269412 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 is on osd.4 2014-08-13 18:06:08.269428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 is on osd.4 2014-08-13 18:06:08.269443 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.269459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 is on osd.4 2014-08-13 18:06:08.269474 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 is on osd.4 2014-08-13 18:06:08.269490 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 is on osd.4 2014-08-13 18:06:08.269505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 is on osd.4 2014-08-13 18:06:08.269522 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 is on osd.4 2014-08-13 18:06:08.269537 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 is on osd.4 2014-08-13 18:06:08.269553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.269569 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 is on osd.4 2014-08-13 18:06:08.269584 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 18:06:08.269627 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate: adding 344 as a source 2014-08-13 18:06:08.269645 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 also missing on osd.344 2014-08-13 18:06:08.269254 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269665 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269663 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 also missing on osd.344 2014-08-13 18:06:08.269680 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 also missing on osd.344 2014-08-13 18:06:08.269697 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 also missing on osd.344 2014-08-13 18:06:08.269715 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269718 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269720 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352f40 pg_notify(24.b6(2) epoch 194496) v5 2014-08-13 18:06:08.269714 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 also missing on osd.344 2014-08-13 18:06:08.269732 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 also missing on osd.344 2014-08-13 18:06:08.269761 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.327 2014-08-13 18:06:08.269750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 also missing on osd.344 2014-08-13 18:06:08.269765 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352f40 2014-08-13 18:06:08.269769 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 also missing on osd.344 2014-08-13 18:06:08.269787 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 also missing on osd.344 2014-08-13 18:06:08.269805 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 also missing on osd.344 2014-08-13 18:06:08.269820 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269825 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269821 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 also missing on osd.344 2014-08-13 18:06:08.269839 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 also missing on osd.344 2014-08-13 18:06:08.269854 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269857 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269860 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed4ec0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.269856 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 also missing on osd.344 2014-08-13 18:06:08.269874 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 also missing on osd.344 2014-08-13 18:06:08.269872 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.269902 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.269891 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 also missing on osd.344 2014-08-13 18:06:08.269906 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.269909 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.269909 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 also missing on osd.344 2014-08-13 18:06:08.279239 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279251 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279254 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df47c0 pg_notify(24.b6 epoch 194496) v5 2014-08-13 18:06:08.269933 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 also missing on osd.344 2014-08-13 18:06:08.268750 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194495 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.279301 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.330 2014-08-13 18:06:08.279305 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df47c0 2014-08-13 18:06:08.279305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 also missing on osd.344 2014-08-13 18:06:08.279314 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194496 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: Reset advmap 2014-08-13 18:06:08.279351 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279364 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279348 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194496 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.279347 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 also missing on osd.344 2014-08-13 18:06:08.279369 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194496 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] acting_up_affected newup [15,4,52] newacting [15,4,61] 2014-08-13 18:06:08.279371 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 also missing on osd.344 2014-08-13 18:06:08.279393 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279396 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279399 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a43640 pg_info(1 pgs e194496:22.6bd) v4 2014-08-13 18:06:08.279386 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194496 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.279390 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 also missing on osd.344 2014-08-13 18:06:08.279405 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52] r=1 lpr=194496 pi=193264-194494/39 crt=194422'11806 lcod 0'0 inactive NOTIFY m=56] set_last_peering_reset 194496 2014-08-13 18:06:08.279410 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 also missing on osd.344 2014-08-13 18:06:08.279428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 also missing on osd.344 2014-08-13 18:06:08.279443 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.6bd) v4 from osd.289 2014-08-13 18:06:08.279448 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a43640 2014-08-13 18:06:08.279453 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [15,4,52](15) acting [15,4,52](15)): not rw, up_thru 194492 up_from 192969 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [15,4,52](15) acting [15,4,52](15)) : primary up 192969-194492 does not include interval 2014-08-13 18:06:08.279487 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279491 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279446 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 18:06:08.279474 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194495/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] noting past interval(194495-194495 up [15,4,52](15) acting [15,4,52](15)) 2014-08-13 18:06:08.279494 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] activate: adding 373 as a source 2014-08-13 18:06:08.279511 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279514 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279494 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] up [15,4,52] -> [15,4,52], acting [15,4,52] -> [15,4,61], acting_primary 15 -> 15, up_primary 15 -> 15, role 1 -> 1 2014-08-13 18:06:08.279516 7f5c835ef700 20 osd.4 194496 _dispatch 0x21a9860 pg_query(24.2a1 epoch 194496) v3 2014-08-13 18:06:08.279515 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22 191727'389 also missing on osd.373 2014-08-13 18:06:08.279520 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:08.279535 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22 191727'390 also missing on osd.373 2014-08-13 18:06:08.279539 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.279556 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.17 epoch 194496 2014-08-13 18:06:08.279560 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21a9860 2014-08-13 18:06:08.279555 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22 191727'363 also missing on osd.373 2014-08-13 18:06:08.279557 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:08.279576 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22 191727'374 also missing on osd.373 2014-08-13 18:06:08.279579 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_copy_ops 2014-08-13 18:06:08.279574 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193188) from replica 17 2014-08-13 18:06:08.279595 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22 191727'383 also missing on osd.373 2014-08-13 18:06:08.279597 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_flush_ops 2014-08-13 18:06:08.279622 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279625 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279615 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22 191648'26 also missing on osd.373 2014-08-13 18:06:08.279616 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change_cleanup 2014-08-13 18:06:08.279644 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279647 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279635 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22 191727'365 also missing on osd.373 2014-08-13 18:06:08.279650 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ac920 pg_query(24.3b7 epoch 194496) v3 2014-08-13 18:06:08.279635 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] on_change 2014-08-13 18:06:08.279655 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22 191727'403 also missing on osd.373 2014-08-13 18:06:08.279658 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit NotTrimming 2014-08-13 18:06:08.279674 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22 191727'359 also missing on osd.373 2014-08-13 18:06:08.279689 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.356 epoch 194496 2014-08-13 18:06:08.279678 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter NotTrimming 2014-08-13 18:06:08.279697 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ac920 2014-08-13 18:06:08.279696 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22 191727'368 also missing on osd.373 2014-08-13 18:06:08.279697 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] cancel_recovery 2014-08-13 18:06:08.279711 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194422'18845 ec=192688 les/c 194493/194493 194495/194496/192969) from replica 356 2014-08-13 18:06:08.279717 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22 191727'385 also missing on osd.373 2014-08-13 18:06:08.279716 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] clear_recovery_state 2014-08-13 18:06:08.279741 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22 191727'388 also missing on osd.373 2014-08-13 18:06:08.279758 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279744 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.279762 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279761 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22 191648'238 also missing on osd.373 2014-08-13 18:06:08.279766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 18:06:08.279783 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279785 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279788 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee1b00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.279780 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22 191727'362 also missing on osd.373 2014-08-13 18:06:08.279797 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.279802 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.279805 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279808 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279799 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22 191727'411 also missing on osd.373 2014-08-13 18:06:08.279826 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279829 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279817 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22 191727'348 also missing on osd.373 2014-08-13 18:06:08.279832 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc6540 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.279838 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.279842 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.279845 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279847 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279835 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22 191648'231 also missing on osd.373 2014-08-13 18:06:08.279864 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279867 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279853 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22 191727'393 also missing on osd.373 2014-08-13 18:06:08.279869 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc6e40 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.279876 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.279860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] update_heartbeat_peers unchanged 2014-08-13 18:06:08.279880 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.279883 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279886 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279872 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22 191648'196 also missing on osd.373 2014-08-13 18:06:08.279881 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 18:06:08.279892 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22 191648'122 also missing on osd.373 2014-08-13 18:06:08.279903 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Reset 0.011171 1 0.011196 2014-08-13 18:06:08.279923 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.279911 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22 191648'139 also missing on osd.373 2014-08-13 18:06:08.279927 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.279930 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d240 pg_info(1 pgs e194496:22.99) v4 2014-08-13 18:06:08.279927 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started 2014-08-13 18:06:08.279929 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22 191727'398 also missing on osd.373 2014-08-13 18:06:08.279947 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Start 2014-08-13 18:06:08.279949 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22 191727'354 also missing on osd.373 2014-08-13 18:06:08.279970 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.99) v4 from osd.374 2014-08-13 18:06:08.279975 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d240 2014-08-13 18:06:08.279967 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: transitioning to Stray 2014-08-13 18:06:08.279991 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Start 0.000044 0 0.000000 2014-08-13 18:06:08.279967 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {44ca039d/benchmark_data_storage0111-ib.pocstor_51285_object24633/head//22=191727'389,452f039d/benchmark_data_storage0111-ib.pocstor_51285_object25667/head//22=191727'390,9064339d/benchmark_data_storage0111-ib.pocstor_51285_object10097/head//22=191727'363,ef99339d/benchmark_data_storage0111-ib.pocstor_51285_object16548/head//22=191727'374,2cf3539d/benchmark_data_storage0111-ib.pocstor_51285_object22485/head//22=191727'383,e112739d/benchmark_data_storage0111-ib.pocstor_40745_object1298/head//22=191648'26,e742e39d/benchmark_data_storage0111-ib.pocstor_51285_object12884/head//22=191727'365,e392f39d/benchmark_data_storage0111-ib.pocstor_51285_object31329/head//22=191727'403,3c18f39d/benchmark_data_storage0111-ib.pocstor_51285_object7848/head//22=191727'359,e7282b9d/benchmark_data_storage0111-ib.pocstor_51285_object13707/head//22=191727'368,97624b9d/benchmark_data_storage0111-ib.pocstor_51285_object22600/head//22=191727'385,155c5b9d/benchmark_data_storage0111-ib.pocstor_51285_object24144/head//22=191727'388,9e3f6b9d/benchmark_data_storage0111-ib.pocstor_40745_object15586/head//22=191648'238,e6387b9d/benchmark_data_storage0111-ib.pocstor_51285_object9836/head//22=191727'362,aac97b9d/benchmark_data_storage0111-ib.pocstor_51285_object35463/head//22=191727'411,d4cd8b9d/benchmark_data_storage0111-ib.pocstor_51285_object459/head//22=191727'348,bb0f8b9d/benchmark_data_storage0111-ib.pocstor_40745_object15129/head//22=191648'231,e281ab9d/benchmark_data_storage0111-ib.pocstor_51285_object26105/head//22=191727'393,7fafbb9d/benchmark_data_storage0111-ib.pocstor_40745_object12773/head//22=191648'196,52d7cb9d/benchmark_data_storage0111-ib.pocstor_40745_object7732/head//22=191648'122,d8ecdb9d/benchmark_data_storage0111-ib.pocstor_40745_object8771/head//22=191648'139,d447eb9d/benchmark_data_storage0111-ib.pocstor_51285_object28265/head//22=191727'398,3b7ffb9d/benchmark_data_storage0111-ib.pocstor_51285_object4104/head//22=191727'354} 2014-08-13 18:06:08.280017 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280020 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280013 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/Stray 2014-08-13 18:06:08.280016 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.280041 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280043 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.280046 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cf9680 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280037 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.280054 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280059 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280062 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280064 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280053 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 320,344,363,366,373 2014-08-13 18:06:08.280057 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.280084 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280087 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280074 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.280090 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38700 pg_info(1 pgs e194496:22.39d) v4 2014-08-13 18:06:08.280093 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.280113 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.280129 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.39d) v4 from osd.344 2014-08-13 18:06:08.280133 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38700 2014-08-13 18:06:08.280132 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.280177 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.280254 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.280242 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,4] -- 31/31 2014-08-13 18:06:08.280276 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.280283 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Started advmap 2014-08-13 18:06:08.280313 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280317 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280302 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,4] 2014-08-13 18:06:08.280321 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.280313 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 18:06:08.280341 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280345 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280345 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.280347 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3b80 pg_info(1 pgs e194496:22.39d) v4 2014-08-13 18:06:08.280342 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started/Stray 0.917581 4 0.000201 2014-08-13 18:06:08.280354 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.280366 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] exit Started 0.917649 0 0.000000 2014-08-13 18:06:08.280389 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.39d) v4 from osd.373 2014-08-13 18:06:08.280378 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.280393 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3b80 2014-08-13 18:06:08.280389 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] enter Reset 2014-08-13 18:06:08.280401 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.280409 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194495 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194496 2014-08-13 18:06:08.280434 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280437 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280429 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194496 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: Reset advmap 2014-08-13 18:06:08.280455 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280458 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280446 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194496 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] _calc_past_interval_range: already have past intervals back to 194372 2014-08-13 18:06:08.280460 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee1200 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280468 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280473 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280475 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280463 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194496 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] acting_up_affected newup [31,338,4] newacting [31,338,4] 2014-08-13 18:06:08.280478 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280482 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194496 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.280483 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.280507 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280509 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280512 7f5c835ef700 20 osd.4 194496 _dispatch 0x5150480 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280500 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4]/[31,338,16] r=-1 lpr=194496 pi=193125-194494/47 crt=194422'18567 lcod 0'0 remapped NOTIFY m=46] set_last_peering_reset 194496 2014-08-13 18:06:08.280503 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.280518 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280523 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280526 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280528 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280521 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.280542 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.280547 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280549 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280552 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed3a80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280538 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [31,338,4](31) acting [31,338,16](31)): not rw, up_thru 194491 up_from 192978 last_epoch_clean 194372 generate_past_intervals interval(194495-194495 up [31,338,4](31) acting [31,338,16](31)) : primary up 192978-194491 does not include interval 2014-08-13 18:06:08.280557 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280562 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280564 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280566 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280556 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194495/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] noting past interval(194495-194495 up [31,338,4](31) acting [31,338,16](31)) 2014-08-13 18:06:08.280584 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280587 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280590 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf55c0 pg_notify(22.619(36) epoch 194496) v5 2014-08-13 18:06:08.280576 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] up [31,338,4] -> [31,338,4], acting [31,338,16] -> [31,338,4], acting_primary 31 -> 31, up_primary 31 -> 31, role -1 -> 2 2014-08-13 18:06:08.280575 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:08.280609 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.280599 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 18:06:08.280616 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.280616 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:08.280643 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.48 2014-08-13 18:06:08.280647 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf55c0 2014-08-13 18:06:08.280635 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 18:06:08.280638 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.280654 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_copy_ops 2014-08-13 18:06:08.280658 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:08.280671 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_flush_ops 2014-08-13 18:06:08.280686 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280690 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280688 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change_cleanup 2014-08-13 18:06:08.280704 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_change 2014-08-13 18:06:08.280722 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280726 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280727 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed6c00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280721 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit NotTrimming 2014-08-13 18:06:08.280735 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280739 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280743 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280746 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280739 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter NotTrimming 2014-08-13 18:06:08.280739 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:08.280768 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280771 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280759 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] on_role_change 2014-08-13 18:06:08.280774 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed3f00 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280761 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194496 2014-08-13 18:06:08.280781 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280786 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280789 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280775 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 18:06:08.280792 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280781 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.280804 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.280794 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] cancel_recovery 2014-08-13 18:06:08.280811 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280814 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280816 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee7740 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280822 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280812 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] clear_recovery_state 2014-08-13 18:06:08.280827 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280829 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280832 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280798 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] flushed 2014-08-13 18:06:08.280830 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.280829 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 2014-08-13 18:06:08.280851 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280854 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280856 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee45c0 osd_map(194488..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280860 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.280862 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194488,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280851 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map 2014-08-13 18:06:08.280866 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280869 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280871 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280866 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Started advmap 2014-08-13 18:06:08.280896 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280884 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,57] 2014-08-13 18:06:08.280899 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280902 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee5e80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.280908 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.280912 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.280915 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280901 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.280918 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280921 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started/Stray 1.017447 4 0.000303 2014-08-13 18:06:08.280937 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.280940 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.280942 7f5c835ef700 20 osd.4 194496 _dispatch 0x524f1c0 pg_notify(24.b6(44) epoch 194496) v5 2014-08-13 18:06:08.280942 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] exit Started 1.017544 0 0.000000 2014-08-13 18:06:08.280962 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] enter Reset 2014-08-13 18:06:08.280963 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] update_heartbeat_peers unchanged 2014-08-13 18:06:08.280980 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.17 2014-08-13 18:06:08.280985 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524f1c0 2014-08-13 18:06:08.280981 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194495 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194496 2014-08-13 18:06:08.280984 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] take_waiters 2014-08-13 18:06:08.281000 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194496 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: Reset advmap 2014-08-13 18:06:08.281003 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Reset 0.000614 1 0.000568 2014-08-13 18:06:08.281023 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281027 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281005 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.281019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194496 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.281026 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started 2014-08-13 18:06:08.281032 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.281037 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194496 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] acting_up_affected newup [330,4,288] newacting [330,4,57] 2014-08-13 18:06:08.281046 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Start 2014-08-13 18:06:08.281066 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281069 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281057 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194496 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.281072 7f5c835ef700 20 osd.4 194496 _dispatch 0x524f540 pg_info(1 pgs e194496:22.4df) v4 2014-08-13 18:06:08.281064 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: transitioning to Stray 2014-08-13 18:06:08.281075 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288] r=1 lpr=194496 pi=193227-194494/44 crt=194422'15537 lcod 0'0 inactive NOTIFY m=64] set_last_peering_reset 194496 2014-08-13 18:06:08.281086 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Start 0.000041 0 0.000000 2014-08-13 18:06:08.281112 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4df) v4 from osd.356 2014-08-13 18:06:08.281116 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524f540 2014-08-13 18:06:08.281108 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/Stray 2014-08-13 18:06:08.281116 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [330,4,288](330) acting [330,4,288](330)): not rw, up_thru 194492 up_from 192965 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [330,4,288](330) acting [330,4,288](330)) : primary up 192965-194492 does not include interval 2014-08-13 18:06:08.281129 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.281154 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281138 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194495/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] noting past interval(194495-194495 up [330,4,288](330) acting [330,4,288](330)) 2014-08-13 18:06:08.281149 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.281157 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281193 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281196 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281198 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a440c0 pg_notify(24.136(2),22.6bd(2) epoch 194496) v5 2014-08-13 18:06:08.281173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] up [330,4,288] -> [330,4,288], acting [330,4,288] -> [330,4,57], acting_primary 330 -> 330, up_primary 330 -> 330, role 1 -> 1 2014-08-13 18:06:08.281205 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:08.281224 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.281242 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.50 2014-08-13 18:06:08.281245 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a440c0 2014-08-13 18:06:08.281241 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:08.281271 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.281261 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_copy_ops 2014-08-13 18:06:08.281279 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_flush_ops 2014-08-13 18:06:08.281304 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281308 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281296 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change_cleanup 2014-08-13 18:06:08.281313 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] on_change 2014-08-13 18:06:08.281332 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281335 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281338 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee7500 osd_map(194488..194496 src has 185139..194496) v3 2014-08-13 18:06:08.281321 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 2014-08-13 18:06:08.281345 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194488,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.281333 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit NotTrimming 2014-08-13 18:06:08.281350 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.281353 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281355 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281355 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.281353 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter NotTrimming 2014-08-13 18:06:08.281377 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281380 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Started advmap 2014-08-13 18:06:08.281382 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351f80 pg_notify(22.362(34) epoch 194496) v5 2014-08-13 18:06:08.281373 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] cancel_recovery 2014-08-13 18:06:08.281383 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336,4] 2014-08-13 18:06:08.281391 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] clear_recovery_state 2014-08-13 18:06:08.281403 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.281424 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.362 2014-08-13 18:06:08.281413 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.281427 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351f80 2014-08-13 18:06:08.281425 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.917143 4 0.000182 2014-08-13 18:06:08.281440 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map 2014-08-13 18:06:08.281459 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] exit Started 0.917221 0 0.000000 2014-08-13 18:06:08.281483 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281486 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281480 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] enter Reset 2014-08-13 18:06:08.281496 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194496 2014-08-13 18:06:08.281520 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281523 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281525 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354440 pg_notify(24.1bb(2) epoch 194496) v5 2014-08-13 18:06:08.281513 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:08.281520 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] update_heartbeat_peers unchanged 2014-08-13 18:06:08.281531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.281541 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters 2014-08-13 18:06:08.281565 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.332 2014-08-13 18:06:08.281550 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [285,336,4] newacting [285,336,4] 2014-08-13 18:06:08.281569 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354440 2014-08-13 18:06:08.281562 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Reset 0.000600 1 0.000574 2014-08-13 18:06:08.281570 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.281588 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started 2014-08-13 18:06:08.281589 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4]/[285,336] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'17642 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194496 2014-08-13 18:06:08.281606 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281610 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281608 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Start 2014-08-13 18:06:08.281631 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281634 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281636 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356c80 pg_notify(22.1b8(28) epoch 194496) v5 2014-08-13 18:06:08.281627 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: transitioning to Stray 2014-08-13 18:06:08.281626 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [285,336,4](285) acting [285,336](285)): not rw, up_thru 194492 up_from 192975 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [285,336,4](285) acting [285,336](285)) : primary up 192975-194492 does not include interval 2014-08-13 18:06:08.281650 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194495/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] noting past interval(194495-194495 up [285,336,4](285) acting [285,336](285)) 2014-08-13 18:06:08.281653 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Start 0.000045 0 0.000000 2014-08-13 18:06:08.281674 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.347 2014-08-13 18:06:08.281678 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356c80 2014-08-13 18:06:08.281670 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] up [285,336,4] -> [285,336,4], acting [285,336] -> [285,336,4], acting_primary 285 -> 285, up_primary 285 -> 285, role -1 -> 2 2014-08-13 18:06:08.281676 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/Stray 2014-08-13 18:06:08.281693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:08.281715 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281700 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.281718 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281709 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.281720 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.281726 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.281742 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281745 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281748 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed0000 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.281755 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.281744 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:08.281759 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.281762 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281766 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281761 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:08.281756 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] flushed 2014-08-13 18:06:08.281779 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:08.281801 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281804 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281806 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5940 pg_notify(2.2a(2),1.2b(2) epoch 194496) v5 2014-08-13 18:06:08.281795 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:08.281814 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:08.281834 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.281831 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:08.281847 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.370 2014-08-13 18:06:08.281850 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5940 2014-08-13 18:06:08.281848 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 18:06:08.281863 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.281879 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:08.281900 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281903 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281884 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 18:06:08.281894 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:08.281912 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.281923 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.281926 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.281928 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1dc0 pg_notify(22.f3(26) epoch 194496) v5 2014-08-13 18:06:08.281920 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.281932 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 18:06:08.281941 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.281967 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.311 2014-08-13 18:06:08.281970 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1dc0 2014-08-13 18:06:08.281960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.281980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.282005 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282008 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.281997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.282016 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.282031 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282034 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282021 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 18:06:08.282037 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889500 pg_notify(22.f3(1) epoch 194496) v5 2014-08-13 18:06:08.282036 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2724 2014-08-13 18:06:08.282041 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 18:06:08.282054 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.282060 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000581 1 0.000569 2014-08-13 18:06:08.282077 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.312 2014-08-13 18:06:08.282081 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889500 2014-08-13 18:06:08.282074 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.872201 4 0.000379 2014-08-13 18:06:08.282081 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 18:06:08.282096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.282101 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 18:06:08.282116 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282119 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282113 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.015517 0 0.000000 2014-08-13 18:06:08.282118 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 18:06:08.282138 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282141 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282144 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351c00 pg_notify(24.9b(39) epoch 194496) v5 2014-08-13 18:06:08.282138 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000037 0 0.000000 2014-08-13 18:06:08.282140 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.282162 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 18:06:08.282162 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194486 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.282191 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.350 2014-08-13 18:06:08.282194 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351c00 2014-08-13 18:06:08.282185 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.282196 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.282214 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.282212 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.282232 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282236 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282232 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 18:06:08.282272 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282257 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.282275 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282278 7f5c835ef700 20 osd.4 194496 _dispatch 0x53540c0 pg_notify(22.481(49) epoch 194496) v5 2014-08-13 18:06:08.282313 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.381 2014-08-13 18:06:08.282316 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53540c0 2014-08-13 18:06:08.282319 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.282349 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282352 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282370 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 18:06:08.282391 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282398 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282398 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.282401 7f5c835ef700 20 osd.4 194496 _dispatch 0x53539c0 pg_notify(24.1f7 epoch 194496) v5 2014-08-13 18:06:08.282389 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] flushed 2014-08-13 18:06:08.282405 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.282425 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.282440 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.59 2014-08-13 18:06:08.282443 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53539c0 2014-08-13 18:06:08.282443 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.282477 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282480 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282498 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282501 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282509 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354b40 pg_notify(24.181(45) epoch 194496) v5 2014-08-13 18:06:08.282499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 sending log((0'0,194215'513], crt=191727'453) 2014-08-13 18:06:08.282524 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b17220 don't know epoch, doing nothing 2014-08-13 18:06:08.282526 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.282550 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.307 2014-08-13 18:06:08.282555 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354b40 2014-08-13 18:06:08.282544 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.282561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.282586 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.282567 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.56 22.775( DNE v 194215'513 lc 0'0 (0'0,194215'513] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 18:06:08.282592 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282595 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282592 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 22.775( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.282620 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282623 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282625 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1500 pg_notify(24.9b(39) epoch 194496) v5 2014-08-13 18:06:08.282611 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 18:06:08.282641 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.282661 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.291 2014-08-13 18:06:08.282648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.282665 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1500 2014-08-13 18:06:08.282666 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.282684 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.282699 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282703 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282703 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.282695 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 18:06:08.282733 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282721 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.282736 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282738 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353800 pg_notify(24.136(2) epoch 194496) v5 2014-08-13 18:06:08.282744 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.282775 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.50 2014-08-13 18:06:08.282763 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2686 2014-08-13 18:06:08.282779 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353800 2014-08-13 18:06:08.282780 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.282814 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282799 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.867150 6 0.000412 2014-08-13 18:06:08.282817 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282835 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282821 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.282838 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282820 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 sending log((0'0,194215'513], crt=191727'453) 2014-08-13 18:06:08.282841 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356ac0 pg_notify(24.181(38) epoch 194496) v5 2014-08-13 18:06:08.282847 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x50e1600 don't know epoch, doing nothing 2014-08-13 18:06:08.282840 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.909528 0 0.000000 2014-08-13 18:06:08.282879 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.296 2014-08-13 18:06:08.282882 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356ac0 2014-08-13 18:06:08.282873 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.282875 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate peer osd.353 22.775( DNE v 194215'513 lc 0'0 (0'0,194215'513] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(32) 2014-08-13 18:06:08.282891 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194486 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.282920 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282924 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282913 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.282930 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.282948 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.282951 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.282953 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea300 pg_info(1 pgs e194496:22.99) v4 2014-08-13 18:06:08.282945 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'539 -> 194215'539 2014-08-13 18:06:08.282970 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.282989 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.99) v4 from osd.31 2014-08-13 18:06:08.282994 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea300 2014-08-13 18:06:08.282981 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.56 has 32 missing 2014-08-13 18:06:08.282999 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.353 has 32 missing 2014-08-13 18:06:08.283028 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283015 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 is on osd.4 2014-08-13 18:06:08.283031 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283034 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 is on osd.4 2014-08-13 18:06:08.283049 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283052 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283055 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353b80 pg_info(1 pgs e194496:22.4df) v4 2014-08-13 18:06:08.283051 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 is on osd.4 2014-08-13 18:06:08.283069 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 is on osd.4 2014-08-13 18:06:08.283091 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4df) v4 from osd.357 2014-08-13 18:06:08.283096 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353b80 2014-08-13 18:06:08.283087 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.283104 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 is on osd.4 2014-08-13 18:06:08.283130 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283133 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283121 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 is on osd.4 2014-08-13 18:06:08.283150 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283139 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 is on osd.4 2014-08-13 18:06:08.283153 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283155 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1c00 pg_info(1 pgs e194496:22.f3) v4 2014-08-13 18:06:08.283156 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 is on osd.4 2014-08-13 18:06:08.283173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 is on osd.4 2014-08-13 18:06:08.283199 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.f3) v4 from osd.312 2014-08-13 18:06:08.283204 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1c00 2014-08-13 18:06:08.283198 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 is on osd.4 2014-08-13 18:06:08.283216 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 is on osd.4 2014-08-13 18:06:08.283241 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283244 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283233 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 is on osd.4 2014-08-13 18:06:08.283263 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283267 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283253 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 is on osd.4 2014-08-13 18:06:08.283269 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351a40 pg_notify(24.9b(39) epoch 194496) v5 2014-08-13 18:06:08.283258 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 sending log((0'0,194215'539], crt=191727'457) 2014-08-13 18:06:08.283270 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 is on osd.4 2014-08-13 18:06:08.283285 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x449a3c0 don't know epoch, doing nothing 2014-08-13 18:06:08.283292 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 is on osd.4 2014-08-13 18:06:08.283317 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.350 2014-08-13 18:06:08.283320 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351a40 2014-08-13 18:06:08.283310 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 is on osd.4 2014-08-13 18:06:08.283329 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 is on osd.4 2014-08-13 18:06:08.283324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.44 22.3af( DNE v 194215'539 lc 0'0 (0'0,194215'539] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(24) 2014-08-13 18:06:08.283347 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 is on osd.4 2014-08-13 18:06:08.283349 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 22.3af( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.283371 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283374 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283364 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 is on osd.4 2014-08-13 18:06:08.283382 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 is on osd.4 2014-08-13 18:06:08.283407 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283410 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283398 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 is on osd.4 2014-08-13 18:06:08.283412 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354600 pg_notify(24.9b(39) epoch 194496) v5 2014-08-13 18:06:08.283415 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 is on osd.4 2014-08-13 18:06:08.283432 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 is on osd.4 2014-08-13 18:06:08.283449 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.291 2014-08-13 18:06:08.283453 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354600 2014-08-13 18:06:08.283449 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 is on osd.4 2014-08-13 18:06:08.283466 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.283492 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283496 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283483 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 is on osd.4 2014-08-13 18:06:08.283500 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 is on osd.4 2014-08-13 18:06:08.283516 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 is on osd.4 2014-08-13 18:06:08.283534 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283537 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283532 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 is on osd.4 2014-08-13 18:06:08.283545 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357700 pg_info(1 pgs e194496:22.6bd) v4 2014-08-13 18:06:08.283549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 is on osd.4 2014-08-13 18:06:08.283583 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.6bd) v4 from osd.50 2014-08-13 18:06:08.283587 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357700 2014-08-13 18:06:08.283567 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 is on osd.4 2014-08-13 18:06:08.283623 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.283607 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 sending log((0'0,194215'539], crt=191727'457) 2014-08-13 18:06:08.283627 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.283631 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:08.283591 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 18:06:08.283652 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate: adding 56 as a source 2014-08-13 18:06:08.283656 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate peer osd.362 22.3af( DNE v 194215'539 lc 0'0 (0'0,194215'539] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(24) 2014-08-13 18:06:08.283671 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 also missing on osd.56 2014-08-13 18:06:08.283689 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 also missing on osd.56 2014-08-13 18:06:08.283706 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 also missing on osd.56 2014-08-13 18:06:08.283723 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 also missing on osd.56 2014-08-13 18:06:08.283740 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 also missing on osd.56 2014-08-13 18:06:08.283745 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.44 has 24 missing 2014-08-13 18:06:08.283758 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 also missing on osd.56 2014-08-13 18:06:08.283762 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.362 has 24 missing 2014-08-13 18:06:08.283776 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 also missing on osd.56 2014-08-13 18:06:08.283778 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 is on osd.4 2014-08-13 18:06:08.283795 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 also missing on osd.56 2014-08-13 18:06:08.283800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.283813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 also missing on osd.56 2014-08-13 18:06:08.283818 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 is on osd.4 2014-08-13 18:06:08.283831 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 also missing on osd.56 2014-08-13 18:06:08.283836 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 is on osd.4 2014-08-13 18:06:08.283849 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 also missing on osd.56 2014-08-13 18:06:08.283853 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 is on osd.4 2014-08-13 18:06:08.283867 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 also missing on osd.56 2014-08-13 18:06:08.283870 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.283886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 also missing on osd.56 2014-08-13 18:06:08.283888 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 is on osd.4 2014-08-13 18:06:08.283905 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 also missing on osd.56 2014-08-13 18:06:08.283906 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 is on osd.4 2014-08-13 18:06:08.283924 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 also missing on osd.56 2014-08-13 18:06:08.283924 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 is on osd.4 2014-08-13 18:06:08.283943 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 is on osd.4 2014-08-13 18:06:08.283943 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 also missing on osd.56 2014-08-13 18:06:08.283965 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 is on osd.4 2014-08-13 18:06:08.283967 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 also missing on osd.56 2014-08-13 18:06:08.283983 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.283986 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 also missing on osd.56 2014-08-13 18:06:08.284000 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 is on osd.4 2014-08-13 18:06:08.284004 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 also missing on osd.56 2014-08-13 18:06:08.284018 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 is on osd.4 2014-08-13 18:06:08.284023 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 also missing on osd.56 2014-08-13 18:06:08.284035 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 is on osd.4 2014-08-13 18:06:08.284041 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 also missing on osd.56 2014-08-13 18:06:08.284052 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 is on osd.4 2014-08-13 18:06:08.284060 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 also missing on osd.56 2014-08-13 18:06:08.284070 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.284080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 also missing on osd.56 2014-08-13 18:06:08.284089 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 is on osd.4 2014-08-13 18:06:08.284099 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 also missing on osd.56 2014-08-13 18:06:08.284107 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 is on osd.4 2014-08-13 18:06:08.284118 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 also missing on osd.56 2014-08-13 18:06:08.284124 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.284137 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 also missing on osd.56 2014-08-13 18:06:08.284142 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 is on osd.4 2014-08-13 18:06:08.284155 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 also missing on osd.56 2014-08-13 18:06:08.284159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 is on osd.4 2014-08-13 18:06:08.284176 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 is on osd.4 2014-08-13 18:06:08.284173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 also missing on osd.56 2014-08-13 18:06:08.284200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 is on osd.4 2014-08-13 18:06:08.284202 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 also missing on osd.56 2014-08-13 18:06:08.284221 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 also missing on osd.56 2014-08-13 18:06:08.284240 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 also missing on osd.56 2014-08-13 18:06:08.284219 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 18:06:08.284257 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 also missing on osd.56 2014-08-13 18:06:08.284269 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate: adding 44 as a source 2014-08-13 18:06:08.284289 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 also missing on osd.44 2014-08-13 18:06:08.284307 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 also missing on osd.44 2014-08-13 18:06:08.284274 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 18:06:08.284325 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 also missing on osd.44 2014-08-13 18:06:08.284333 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] activate: adding 353 as a source 2014-08-13 18:06:08.284344 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 also missing on osd.44 2014-08-13 18:06:08.284352 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22 191727'421 also missing on osd.353 2014-08-13 18:06:08.284363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 also missing on osd.44 2014-08-13 18:06:08.284370 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22 191727'384 also missing on osd.353 2014-08-13 18:06:08.284382 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 also missing on osd.44 2014-08-13 18:06:08.284388 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22 191727'370 also missing on osd.353 2014-08-13 18:06:08.284401 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 also missing on osd.44 2014-08-13 18:06:08.284407 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22 191648'138 also missing on osd.353 2014-08-13 18:06:08.284419 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 also missing on osd.44 2014-08-13 18:06:08.284425 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22 191727'392 also missing on osd.353 2014-08-13 18:06:08.284438 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 also missing on osd.44 2014-08-13 18:06:08.284443 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22 191727'410 also missing on osd.353 2014-08-13 18:06:08.284457 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 also missing on osd.44 2014-08-13 18:06:08.284462 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22 191727'388 also missing on osd.353 2014-08-13 18:06:08.284481 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22 191648'199 also missing on osd.353 2014-08-13 18:06:08.284475 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 also missing on osd.44 2014-08-13 18:06:08.284499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22 191727'389 also missing on osd.353 2014-08-13 18:06:08.284500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 also missing on osd.44 2014-08-13 18:06:08.284517 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22 191727'383 also missing on osd.353 2014-08-13 18:06:08.284519 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 also missing on osd.44 2014-08-13 18:06:08.284535 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22 191648'130 also missing on osd.353 2014-08-13 18:06:08.284538 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 also missing on osd.44 2014-08-13 18:06:08.284553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22 191727'433 also missing on osd.353 2014-08-13 18:06:08.284556 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 also missing on osd.44 2014-08-13 18:06:08.284572 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22 191727'442 also missing on osd.353 2014-08-13 18:06:08.284574 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 also missing on osd.44 2014-08-13 18:06:08.284590 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22 191727'402 also missing on osd.353 2014-08-13 18:06:08.284593 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 also missing on osd.44 2014-08-13 18:06:08.284608 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22 191727'441 also missing on osd.353 2014-08-13 18:06:08.284611 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 also missing on osd.44 2014-08-13 18:06:08.284626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22 191727'450 also missing on osd.353 2014-08-13 18:06:08.284630 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 also missing on osd.44 2014-08-13 18:06:08.284645 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22 191727'401 also missing on osd.353 2014-08-13 18:06:08.284648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 also missing on osd.44 2014-08-13 18:06:08.284663 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22 191727'434 also missing on osd.353 2014-08-13 18:06:08.284667 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 also missing on osd.44 2014-08-13 18:06:08.284682 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22 191648'59 also missing on osd.353 2014-08-13 18:06:08.284686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 also missing on osd.44 2014-08-13 18:06:08.284700 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22 191648'308 also missing on osd.353 2014-08-13 18:06:08.284704 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 also missing on osd.44 2014-08-13 18:06:08.284718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22 191727'429 also missing on osd.353 2014-08-13 18:06:08.284723 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 also missing on osd.44 2014-08-13 18:06:08.284736 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22 191648'107 also missing on osd.353 2014-08-13 18:06:08.284755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22 191727'445 also missing on osd.353 2014-08-13 18:06:08.284739 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 18:06:08.284772 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22 191648'323 also missing on osd.353 2014-08-13 18:06:08.284787 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] activate: adding 362 as a source 2014-08-13 18:06:08.284790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22 191648'281 also missing on osd.353 2014-08-13 18:06:08.292599 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.292605 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.292609 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc7080 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.292580 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22 191648'109 also missing on osd.362 2014-08-13 18:06:08.292620 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.292626 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.292629 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.292632 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.292598 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22 191727'399 also missing on osd.353 2014-08-13 18:06:08.292623 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22 191727'406 also missing on osd.362 2014-08-13 18:06:08.292657 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.292660 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.292644 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22 191727'391 also missing on osd.362 2014-08-13 18:06:08.292642 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22 191648'93 also missing on osd.353 2014-08-13 18:06:08.292663 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4ec0 pg_info(1 pgs e194496:22.775) v4 2014-08-13 18:06:08.292668 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22 191727'409 also missing on osd.362 2014-08-13 18:06:08.292669 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22 191648'275 also missing on osd.353 2014-08-13 18:06:08.292687 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22 191648'149 also missing on osd.362 2014-08-13 18:06:08.292713 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.775) v4 from osd.353 2014-08-13 18:06:08.292719 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4ec0 2014-08-13 18:06:08.292692 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22 191727'400 also missing on osd.353 2014-08-13 18:06:08.292718 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22 191727'437 also missing on osd.362 2014-08-13 18:06:08.292727 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing 69adf75/rb.0.43b76.238e1f29.000000000032/head//22 192674'461 also missing on osd.353 2014-08-13 18:06:08.292738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22 191648'165 also missing on osd.362 2014-08-13 18:06:08.292748 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22 191727'374 also missing on osd.353 2014-08-13 18:06:08.292759 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22 191727'402 also missing on osd.362 2014-08-13 18:06:08.292766 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] search_for_missing f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22 191648'55 also missing on osd.353 2014-08-13 18:06:08.292778 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22 191648'355 also missing on osd.362 2014-08-13 18:06:08.292797 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22 191727'386 also missing on osd.362 2014-08-13 18:06:08.292815 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22 191648'76 also missing on osd.362 2014-08-13 18:06:08.292783 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {70e10775/benchmark_data_storage0111-ib.pocstor_51285_object23919/head//22=191727'421,36c9775/benchmark_data_storage0111-ib.pocstor_51285_object7757/head//22=191727'384,379c9775/benchmark_data_storage0111-ib.pocstor_51285_object1990/head//22=191727'370,f210a775/benchmark_data_storage0111-ib.pocstor_40745_object7387/head//22=191648'138,a347c775/benchmark_data_storage0111-ib.pocstor_51285_object13335/head//22=191727'392,4dd8d775/benchmark_data_storage0111-ib.pocstor_51285_object19896/head//22=191727'410,fe3ad775/benchmark_data_storage0111-ib.pocstor_51285_object9744/head//22=191727'388,f86ad775/benchmark_data_storage0111-ib.pocstor_40745_object11403/head//22=191648'199,ef81e775/benchmark_data_storage0111-ib.pocstor_51285_object11769/head//22=191727'389,cd820f75/benchmark_data_storage0111-ib.pocstor_51285_object5893/head//22=191727'383,52160f75/benchmark_data_storage0111-ib.pocstor_40745_object6834/head//22=191648'130,1a9f0f75/benchmark_data_storage0111-ib.pocstor_51285_object29709/head//22=191727'433,9b812f75/benchmark_data_storage0111-ib.pocstor_51285_object34241/head//22=191727'442,6062f75/benchmark_data_storage0111-ib.pocstor_51285_object16646/head//22=191727'402,73b24f75/benchmark_data_storage0111-ib.pocstor_51285_object34099/head//22=191727'441,29554f75/benchmark_data_storage0111-ib.pocstor_51285_object36743/head//22=191727'450,d28a5f75/benchmark_data_storage0111-ib.pocstor_51285_object16476/head//22=191727'401,81b86f75/benchmark_data_storage0111-ib.pocstor_51285_object30316/head//22=191727'434,d4707f75/benchmark_data_storage0111-ib.pocstor_40745_object3103/head//22=191648'59,47978f75/benchmark_data_storage0111-ib.pocstor_40745_object18987/head//22=191648'308,fad39f75/benchmark_data_storage0111-ib.pocstor_51285_object28022/head//22=191727'429,e9479f75/benchmark_data_storage0111-ib.pocstor_40745_object5726/head//22=191648'107,3e5d9f75/benchmark_data_storage0111-ib.pocstor_51285_object34777/head//22=191727'445,9540af75/benchmark_data_storage0111-ib.pocstor_40745_object20240/head//22=191648'323,3f1af75/benchmark_data_storage0111-ib.pocstor_40745_object16441/head//22=191648'281,ff26bf75/benchmark_data_storage0111-ib.pocstor_51285_object16059/head//22=191727'399,c08cf75/benchmark_data_storage0111-ib.pocstor_40745_object4777/head//22=191648'93,c4aecf75/benchmark_data_storage0111-ib.pocstor_40745_object15850/head//22=191648'275,48df75/benchmark_data_storage0111-ib.pocstor_51285_object16310/head//22=191727'400,69adf75/rb.0.43b76.238e1f29.000000000032/head//22=192674'461,ec71ef75/benchmark_data_storage0111-ib.pocstor_51285_object3076/head//22=191727'374,f7baff75/benchmark_data_storage0111-ib.pocstor_40745_object3018/head//22=191648'55} 2014-08-13 18:06:08.292832 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22 191727'398 also missing on osd.362 2014-08-13 18:06:08.292844 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.292851 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22 191727'438 also missing on osd.362 2014-08-13 18:06:08.292861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.292871 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22 191727'428 also missing on osd.362 2014-08-13 18:06:08.292882 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 0,56,63,353,369 2014-08-13 18:06:08.292891 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22 191727'420 also missing on osd.362 2014-08-13 18:06:08.292901 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.292910 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22 191648'208 also missing on osd.362 2014-08-13 18:06:08.292920 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.292929 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22 191727'392 also missing on osd.362 2014-08-13 18:06:08.292940 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.292948 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22 191648'140 also missing on osd.362 2014-08-13 18:06:08.292958 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.292968 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22 191648'320 also missing on osd.362 2014-08-13 18:06:08.292986 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22 191727'397 also missing on osd.362 2014-08-13 18:06:08.293004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22 191727'403 also missing on osd.362 2014-08-13 18:06:08.293022 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22 191727'422 also missing on osd.362 2014-08-13 18:06:08.293039 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22 191727'413 also missing on osd.362 2014-08-13 18:06:08.293068 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.293056 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] search_for_missing b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22 192674'465 also missing on osd.362 2014-08-13 18:06:08.293071 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {54e303af/benchmark_data_storage0111-ib.pocstor_40745_object6337/head//22=191648'109,c2f303af/benchmark_data_storage0111-ib.pocstor_51285_object10505/head//22=191727'406,757c03af/benchmark_data_storage0111-ib.pocstor_51285_object3579/head//22=191727'391,ee9e03af/benchmark_data_storage0111-ib.pocstor_51285_object11864/head//22=191727'409,45ec23af/benchmark_data_storage0111-ib.pocstor_40745_object8899/head//22=191648'149,b46833af/benchmark_data_storage0111-ib.pocstor_51285_object24200/head//22=191727'437,f58643af/benchmark_data_storage0111-ib.pocstor_40745_object10090/head//22=191648'165,9bd43af/benchmark_data_storage0111-ib.pocstor_51285_object8853/head//22=191727'402,2fbd63af/benchmark_data_storage0111-ib.pocstor_40745_object20630/head//22=191648'355,798f93af/benchmark_data_storage0111-ib.pocstor_51285_object1875/head//22=191727'386,2404a3af/benchmark_data_storage0111-ib.pocstor_40745_object4305/head//22=191648'76,5140c3af/benchmark_data_storage0111-ib.pocstor_51285_object6916/head//22=191727'398,691c3af/benchmark_data_storage0111-ib.pocstor_51285_object24688/head//22=191727'438,6226c3af/benchmark_data_storage0111-ib.pocstor_51285_object20700/head//22=191727'428,794ac3af/benchmark_data_storage0111-ib.pocstor_51285_object16494/head//22=191727'420,82a2d3af/benchmark_data_storage0111-ib.pocstor_40745_object12744/head//22=191648'208,6b44e3af/benchmark_data_storage0111-ib.pocstor_51285_object3941/head//22=191727'392,8a752baf/benchmark_data_storage0111-ib.pocstor_40745_object8284/head//22=191648'140,b87b3baf/benchmark_data_storage0111-ib.pocstor_40745_object18768/head//22=191648'320,aef24baf/benchmark_data_storage0111-ib.pocstor_51285_object5492/head//22=191727'397,88935baf/benchmark_data_storage0111-ib.pocstor_51285_object8968/head//22=191727'403,4de75baf/benchmark_data_storage0111-ib.pocstor_51285_object18153/head//22=191727'422,f0c7bbaf/benchmark_data_storage0111-ib.pocstor_51285_object14381/head//22=191727'413,b05ffbaf/rb.0.43b76.238e1f29.00000000030e/head//22=192674'465} 2014-08-13 18:06:08.293120 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.293143 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.293147 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.293137 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.293130 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 18:06:08.293160 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.293158 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 44,61,339,362,369 2014-08-13 18:06:08.293168 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.293177 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.293199 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.293206 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.293218 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.293227 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.293244 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.293293 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.293312 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.293327 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.293345 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.293351 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.293371 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 2014-08-13 18:06:08.293405 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.293411 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Started advmap 2014-08-13 18:06:08.293407 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 18:06:08.293434 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.293428 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,288] 2014-08-13 18:06:08.293442 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.293446 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.293462 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.293466 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started/Stray 1.001068 4 0.000262 2014-08-13 18:06:08.293454 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.293479 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.293488 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] exit Started 1.001161 0 0.000000 2014-08-13 18:06:08.293498 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.293500 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.293511 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] enter Reset 2014-08-13 18:06:08.293522 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.293530 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194495 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194496 2014-08-13 18:06:08.293541 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.293549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194496 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: Reset advmap 2014-08-13 18:06:08.293561 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2685 2014-08-13 18:06:08.293566 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194496 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.293578 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.293582 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194496 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] acting_up_affected newup [346,4,304] newacting [346,4,288] 2014-08-13 18:06:08.293599 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194496 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.293597 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.871757 2 0.004910 2014-08-13 18:06:08.293615 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304] r=1 lpr=194496 pi=193125-194494/37 crt=194422'15014 lcod 0'0 inactive NOTIFY m=66] set_last_peering_reset 194496 2014-08-13 18:06:08.293618 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.293636 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.918738 0 0.000000 2014-08-13 18:06:08.293652 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [346,4,304](346) acting [346,4,304](346)): not rw, up_thru 194492 up_from 192955 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [346,4,304](346) acting [346,4,304](346)) : primary up 192955-194492 does not include interval 2014-08-13 18:06:08.293660 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.293672 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194495/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] noting past interval(194495-194495 up [346,4,304](346) acting [346,4,304](346)) 2014-08-13 18:06:08.293678 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194486 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.293693 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] up [346,4,304] -> [346,4,304], acting [346,4,304] -> [346,4,288], acting_primary 346 -> 346, up_primary 346 -> 346, role 1 -> 1 2014-08-13 18:06:08.293701 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.293714 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:08.293718 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.293732 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.293734 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'519 -> 194215'519 2014-08-13 18:06:08.293751 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 18:06:08.293758 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:08.293771 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_copy_ops 2014-08-13 18:06:08.293756 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.293780 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 is up to date, queueing in pending_activators 2014-08-13 18:06:08.293789 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_flush_ops 2014-08-13 18:06:08.293795 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.293807 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change_cleanup 2014-08-13 18:06:08.293824 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] on_change 2014-08-13 18:06:08.293843 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit NotTrimming 2014-08-13 18:06:08.293859 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter NotTrimming 2014-08-13 18:06:08.293876 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] cancel_recovery 2014-08-13 18:06:08.293891 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] clear_recovery_state 2014-08-13 18:06:08.293908 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.293912 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.287 22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) uptodate 2014-08-13 18:06:08.293929 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map 2014-08-13 18:06:08.293935 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) 2014-08-13 18:06:08.293956 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 is up to date, queueing in pending_activators 2014-08-13 18:06:08.294081 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] update_heartbeat_peers unchanged 2014-08-13 18:06:08.294106 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters 2014-08-13 18:06:08.294106 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate peer osd.309 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495) missing missing(32) 2014-08-13 18:06:08.294126 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Reset 0.000615 1 0.000516 2014-08-13 18:06:08.294147 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started 2014-08-13 18:06:08.294162 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.309 has 32 missing 2014-08-13 18:06:08.294164 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Start 2014-08-13 18:06:08.294180 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 is on osd.4 2014-08-13 18:06:08.294189 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: transitioning to Stray 2014-08-13 18:06:08.294208 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 is on osd.4 2014-08-13 18:06:08.294211 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Start 0.000048 0 0.000000 2014-08-13 18:06:08.294226 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 is on osd.4 2014-08-13 18:06:08.294232 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/Stray 2014-08-13 18:06:08.294243 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 is on osd.4 2014-08-13 18:06:08.294256 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.294261 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 is on osd.4 2014-08-13 18:06:08.294275 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.294279 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.294298 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 is on osd.4 2014-08-13 18:06:08.294315 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 is on osd.4 2014-08-13 18:06:08.294332 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 is on osd.4 2014-08-13 18:06:08.294347 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 is on osd.4 2014-08-13 18:06:08.294363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 is on osd.4 2014-08-13 18:06:08.294376 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.294379 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 is on osd.4 2014-08-13 18:06:08.294395 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 is on osd.4 2014-08-13 18:06:08.294407 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.294416 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:08.294420 7f5c7c7e4700 7 osd.4 194496 do_notify osd 20 on 1 PGs 2014-08-13 18:06:08.294410 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 is on osd.4 2014-08-13 18:06:08.294427 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 is on osd.4 2014-08-13 18:06:08.294443 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 is on osd.4 2014-08-13 18:06:08.294450 7f5c7c7e4700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4b2d6a0 192.168.1.101:6812/23402 2014-08-13 18:06:08.294459 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.294475 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 is on osd.4 2014-08-13 18:06:08.294491 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 is on osd.4 2014-08-13 18:06:08.294507 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 is on osd.4 2014-08-13 18:06:08.294522 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 is on osd.4 2014-08-13 18:06:08.294538 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 is on osd.4 2014-08-13 18:06:08.294553 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.294557 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.294559 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353100 pg_info(1 pgs e194496:22.3af) v4 2014-08-13 18:06:08.294554 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 is on osd.4 2014-08-13 18:06:08.294570 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 is on osd.4 2014-08-13 18:06:08.294587 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 is on osd.4 2014-08-13 18:06:08.294602 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3af) v4 from osd.362 2014-08-13 18:06:08.294600 7f5c7c7e4700 10 osd.4 194496 note_peer_epoch osd.294 has 194496 2014-08-13 18:06:08.294608 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353100 2014-08-13 18:06:08.294609 7f5c7c7e4700 7 osd.4 194496 do_notify osd 294 on 1 PGs 2014-08-13 18:06:08.294605 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 is on osd.4 2014-08-13 18:06:08.294623 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.294638 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.294642 7f5c7c7e4700 7 osd.4 194496 do_notify osd 304 on 1 PGs 2014-08-13 18:06:08.294655 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.294661 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.294663 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x44270c0 don't know epoch, doing nothing 2014-08-13 18:06:08.294667 7f5c7c7e4700 7 osd.4 194496 do_notify osd 321 on 1 PGs 2014-08-13 18:06:08.294687 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 2014-08-13 18:06:08.294690 7f5c7c7e4700 7 osd.4 194496 do_notify osd 324 on 1 PGs 2014-08-13 18:06:08.294709 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:08.294712 7f5c7c7e4700 7 osd.4 194496 do_notify osd 328 on 1 PGs 2014-08-13 18:06:08.294641 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 is on osd.4 2014-08-13 18:06:08.294829 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] flushed 2014-08-13 18:06:08.294840 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.294865 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.294873 7f5c7c7e4700 7 osd.4 194496 do_notify osd 330 on 1 PGs 2014-08-13 18:06:08.294868 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 is on osd.4 2014-08-13 18:06:08.294886 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 is on osd.4 2014-08-13 18:06:08.294901 7f5c7c7e4700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x4f21600 192.168.1.104:6820/2308 2014-08-13 18:06:08.294902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 is on osd.4 2014-08-13 18:06:08.294928 7f5c7c7e4700 10 osd.4 194496 note_peer_epoch osd.346 has 194496 2014-08-13 18:06:08.294932 7f5c7c7e4700 7 osd.4 194496 do_notify osd 346 on 1 PGs 2014-08-13 18:06:08.294925 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 18:06:08.294985 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate: adding 287 as a source 2014-08-13 18:06:08.295007 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:08.295013 7f5c7c7e4700 7 osd.4 194496 do_notify osd 364 on 1 PGs 2014-08-13 18:06:08.295001 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 is on osd.287 2014-08-13 18:06:08.295020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 is on osd.287 2014-08-13 18:06:08.295037 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:08.295040 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.20 on 1 PGs 2014-08-13 18:06:08.295037 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 is on osd.287 2014-08-13 18:06:08.295061 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing 2014-08-13 18:06:08.295064 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.25 on 1 PGs 2014-08-13 18:06:08.295055 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 is on osd.287 2014-08-13 18:06:08.295084 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing 2014-08-13 18:06:08.295088 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.26 on 1 PGs 2014-08-13 18:06:08.295073 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 is on osd.287 2014-08-13 18:06:08.295108 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:08.295097 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 is on osd.287 2014-08-13 18:06:08.295111 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.30 on 1 PGs 2014-08-13 18:06:08.295115 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 is on osd.287 2014-08-13 18:06:08.295138 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:08.295142 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.303 on 1 PGs 2014-08-13 18:06:08.295132 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 is on osd.287 2014-08-13 18:06:08.295151 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 is on osd.287 2014-08-13 18:06:08.295164 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.295167 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.340 on 1 PGs 2014-08-13 18:06:08.295168 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 is on osd.287 2014-08-13 18:06:08.295194 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:08.295198 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.362 on 1 PGs 2014-08-13 18:06:08.295186 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 is on osd.287 2014-08-13 18:06:08.295215 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:08.295219 7f5c7c7e4700 7 osd.4 194496 do_queries querying osd.373 on 1 PGs 2014-08-13 18:06:08.295213 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 is on osd.287 2014-08-13 18:06:08.295236 7f5c7c7e4700 20 osd.4 194496 Sending info 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 28 2014-08-13 18:06:08.295231 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 is on osd.287 2014-08-13 18:06:08.295246 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:08.295249 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 is on osd.287 2014-08-13 18:06:08.295270 7f5c7c7e4700 20 osd.4 194496 Sending info 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 304 2014-08-13 18:06:08.295279 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.295272 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 is on osd.287 2014-08-13 18:06:08.295290 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 is on osd.287 2014-08-13 18:06:08.295316 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 18:06:08.295308 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 is on osd.287 2014-08-13 18:06:08.295326 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 is on osd.287 2014-08-13 18:06:08.295343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 is on osd.287 2014-08-13 18:06:08.295361 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 is on osd.287 2014-08-13 18:06:08.295377 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 is on osd.287 2014-08-13 18:06:08.295394 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 is on osd.287 2014-08-13 18:06:08.295411 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 is on osd.287 2014-08-13 18:06:08.295414 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 2014-08-13 18:06:08.295429 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 is on osd.287 2014-08-13 18:06:08.295450 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.295447 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 is on osd.287 2014-08-13 18:06:08.295458 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Started advmap 2014-08-13 18:06:08.295465 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 is on osd.287 2014-08-13 18:06:08.295476 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,12] 2014-08-13 18:06:08.295483 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 is on osd.287 2014-08-13 18:06:08.295494 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.295502 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 is on osd.287 2014-08-13 18:06:08.295514 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started/Stray 0.912893 4 0.000223 2014-08-13 18:06:08.295521 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 is on osd.287 2014-08-13 18:06:08.295536 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] exit Started 0.912957 0 0.000000 2014-08-13 18:06:08.295540 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 is on osd.287 2014-08-13 18:06:08.295555 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] enter Reset 2014-08-13 18:06:08.295558 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 is on osd.287 2014-08-13 18:06:08.295571 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194495 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194496 2014-08-13 18:06:08.295576 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 is on osd.287 2014-08-13 18:06:08.295589 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194496 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: Reset advmap 2014-08-13 18:06:08.295605 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194496 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.295621 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194496 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] acting_up_affected newup [374,4,314] newacting [374,4,12] 2014-08-13 18:06:08.295595 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 18:06:08.295638 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194496 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.295652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] activate: adding 309 as a source 2014-08-13 18:06:08.295660 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314] r=1 lpr=194496 pi=192688-194494/53 crt=194422'16449 lcod 0'0 inactive NOTIFY m=52] set_last_peering_reset 194496 2014-08-13 18:06:08.295671 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22 191727'405 also missing on osd.309 2014-08-13 18:06:08.295690 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22 191648'214 also missing on osd.309 2014-08-13 18:06:08.295696 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [374,4,314](374) acting [374,4,314](374)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [374,4,314](374) acting [374,4,314](374)) : primary up 192967-194492 does not include interval 2014-08-13 18:06:08.295708 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22 191727'410 also missing on osd.309 2014-08-13 18:06:08.295716 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194495/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] noting past interval(194495-194495 up [374,4,314](374) acting [374,4,314](374)) 2014-08-13 18:06:08.295727 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22 191648'366 also missing on osd.309 2014-08-13 18:06:08.295737 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] up [374,4,314] -> [374,4,314], acting [374,4,314] -> [374,4,12], acting_primary 374 -> 374, up_primary 374 -> 374, role 1 -> 1 2014-08-13 18:06:08.295747 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22 191648'245 also missing on osd.309 2014-08-13 18:06:08.295759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:08.295766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22 191727'415 also missing on osd.309 2014-08-13 18:06:08.295777 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.295785 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22 191727'445 also missing on osd.309 2014-08-13 18:06:08.295795 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:08.295804 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22 191648'179 also missing on osd.309 2014-08-13 18:06:08.295814 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_copy_ops 2014-08-13 18:06:08.295824 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22 191648'128 also missing on osd.309 2014-08-13 18:06:08.295833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_flush_ops 2014-08-13 18:06:08.295843 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22 191727'447 also missing on osd.309 2014-08-13 18:06:08.295850 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change_cleanup 2014-08-13 18:06:08.295862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22 191648'355 also missing on osd.309 2014-08-13 18:06:08.295868 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] on_change 2014-08-13 18:06:08.295881 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22 191648'39 also missing on osd.309 2014-08-13 18:06:08.295888 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit NotTrimming 2014-08-13 18:06:08.295900 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22 191727'462 also missing on osd.309 2014-08-13 18:06:08.295906 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter NotTrimming 2014-08-13 18:06:08.295918 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22 191727'429 also missing on osd.309 2014-08-13 18:06:08.295925 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] cancel_recovery 2014-08-13 18:06:08.295937 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22 191727'428 also missing on osd.309 2014-08-13 18:06:08.295942 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] clear_recovery_state 2014-08-13 18:06:08.295956 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22 191727'394 also missing on osd.309 2014-08-13 18:06:08.295961 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.295974 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22 191727'398 also missing on osd.309 2014-08-13 18:06:08.295982 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map 2014-08-13 18:06:08.295992 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22 191648'160 also missing on osd.309 2014-08-13 18:06:08.296011 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22 191727'403 also missing on osd.309 2014-08-13 18:06:08.296028 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22 191727'455 also missing on osd.309 2014-08-13 18:06:08.296046 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22 191727'401 also missing on osd.309 2014-08-13 18:06:08.296061 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296065 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296068 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0fc0 pg_notify(24.3d(2) epoch 194496) v5 2014-08-13 18:06:08.296063 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22 191727'425 also missing on osd.309 2014-08-13 18:06:08.296082 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22 191648'385 also missing on osd.309 2014-08-13 18:06:08.296084 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] update_heartbeat_peers unchanged 2014-08-13 18:06:08.296113 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.20 2014-08-13 18:06:08.296101 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22 191648'32 also missing on osd.309 2014-08-13 18:06:08.296117 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0fc0 2014-08-13 18:06:08.296106 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters 2014-08-13 18:06:08.296122 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22 191727'448 also missing on osd.309 2014-08-13 18:06:08.296126 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Reset 0.000571 1 0.000523 2014-08-13 18:06:08.296142 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22 191727'457 also missing on osd.309 2014-08-13 18:06:08.296149 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started 2014-08-13 18:06:08.296164 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22 191727'411 also missing on osd.309 2014-08-13 18:06:08.296182 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296187 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296169 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Start 2014-08-13 18:06:08.296183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22 191727'433 also missing on osd.309 2014-08-13 18:06:08.296211 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296196 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: transitioning to Stray 2014-08-13 18:06:08.296215 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296217 7f5c835ef700 20 osd.4 194496 _dispatch 0x5425460 pg_query(24.e2 epoch 194496) v3 2014-08-13 18:06:08.296208 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22 191727'437 also missing on osd.309 2014-08-13 18:06:08.296221 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Start 0.000053 0 0.000000 2014-08-13 18:06:08.296228 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22 191727'439 also missing on osd.309 2014-08-13 18:06:08.296244 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/Stray 2014-08-13 18:06:08.296259 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.346 epoch 194496 2014-08-13 18:06:08.296248 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22 191727'430 also missing on osd.309 2014-08-13 18:06:08.296263 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5425460 2014-08-13 18:06:08.296267 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] search_for_missing dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22 191648'168 also missing on osd.309 2014-08-13 18:06:08.296267 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.296278 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194422'15014 ec=192688 les/c 194493/194493 194495/194496/192955) from replica 346 2014-08-13 18:06:08.296289 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.296327 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296331 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296287 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {5b1c0481/benchmark_data_storage0111-ib.pocstor_51285_object11275/head//22=191727'405,4dcd0481/benchmark_data_storage0111-ib.pocstor_40745_object12471/head//22=191648'214,49532481/benchmark_data_storage0111-ib.pocstor_51285_object13112/head//22=191727'410,fbc82481/benchmark_data_storage0111-ib.pocstor_40745_object21581/head//22=191648'366,3e094481/benchmark_data_storage0111-ib.pocstor_40745_object14321/head//22=191648'245,49f15481/benchmark_data_storage0111-ib.pocstor_51285_object14906/head//22=191727'415,45775481/benchmark_data_storage0111-ib.pocstor_51285_object31688/head//22=191727'445,a2f86481/benchmark_data_storage0111-ib.pocstor_40745_object10616/head//22=191648'179,c1a27481/benchmark_data_storage0111-ib.pocstor_40745_object7328/head//22=191648'128,976f7481/benchmark_data_storage0111-ib.pocstor_51285_object31919/head//22=191727'447,a45d8481/benchmark_data_storage0111-ib.pocstor_40745_object20930/head//22=191648'355,2179a481/benchmark_data_storage0111-ib.pocstor_40745_object2549/head//22=191648'39,925fa481/benchmark_data_storage0111-ib.pocstor_51285_object35834/head//22=191727'462,158fa481/benchmark_data_storage0111-ib.pocstor_51285_object24226/head//22=191727'429,b165b481/benchmark_data_storage0111-ib.pocstor_51285_object23754/head//22=191727'428,bdb7d481/benchmark_data_storage0111-ib.pocstor_51285_object5070/head//22=191727'394,f37df481/benchmark_data_storage0111-ib.pocstor_51285_object9015/head//22=191727'398,e45ff481/benchmark_data_storage0111-ib.pocstor_40745_object9611/head//22=191648'160,3afd0c81/benchmark_data_storage0111-ib.pocstor_51285_object10569/head//22=191727'403,21f41c81/benchmark_data_storage0111-ib.pocstor_51285_object33788/head//22=191727'455,36591c81/benchmark_data_storage0111-ib.pocstor_51285_object9585/head//22=191727'401,66822c81/benchmark_data_storage0111-ib.pocstor_51285_object21337/head//22=191727'425,67354c81/benchmark_data_storage0111-ib.pocstor_40745_object22846/head//22=191648'385,f86f5c81/benchmark_data_storage0111-ib.pocstor_40745_object2000/head//22=191648'32,f95b6c81/benchmark_data_storage0111-ib.pocstor_51285_object32223/head//22=191727'448,d16c8c81/benchmark_data_storage0111-ib.pocstor_51285_object34792/head//22=191727'457,61919c81/benchmark_data_storage0111-ib.pocstor_51285_object13179/head//22=191727'411,f5bcbc81/benchmark_data_storage0111-ib.pocstor_51285_object25071/head//22=191727'433,e1f0dc81/benchmark_data_storage0111-ib.pocstor_51285_object27293/head//22=191727'437,f367dc81/benchmark_data_storage0111-ib.pocstor_51285_object29699/head//22=191727'439,7d24ec81/benchmark_data_storage0111-ib.pocstor_51285_object24241/head//22=191727'430,dd48ec81/benchmark_data_storage0111-ib.pocstor_40745_object9897/head//22=191648'168} 2014-08-13 18:06:08.296351 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296353 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296356 7f5c835ef700 20 osd.4 194496 _dispatch 0x21aa580 pg_query(24.33a epoch 194496) v3 2014-08-13 18:06:08.296346 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.296362 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194304 2014-08-13 18:06:08.296393 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.304 epoch 194496 2014-08-13 18:06:08.296396 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21aa580 2014-08-13 18:06:08.296389 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 287,303,306,309,365,381 2014-08-13 18:06:08.296409 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.296407 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.296427 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.296408 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193250) from replica 304 2014-08-13 18:06:08.296443 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.296460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.296475 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296479 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296475 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 18:06:08.296503 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296505 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296504 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.296507 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3b80 pg_notify(24.3d epoch 194496) v5 2014-08-13 18:06:08.296511 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.296529 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.296544 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.362 2014-08-13 18:06:08.296548 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3b80 2014-08-13 18:06:08.296547 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.296575 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.296583 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296586 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296606 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296609 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296611 7f5c835ef700 20 osd.4 194496 _dispatch 0x524c7c0 pg_notify(24.3d(38) epoch 194496) v5 2014-08-13 18:06:08.296611 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.296630 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.296650 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.373 2014-08-13 18:06:08.296654 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524c7c0 2014-08-13 18:06:08.296637 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 18:06:08.296648 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.296668 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.296667 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.296674 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.296692 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.296695 7f5c7c7e4700 10 osd.4 pg_epoch: 194495 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 2014-08-13 18:06:08.296709 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.296725 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.296727 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.296732 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Started advmap 2014-08-13 18:06:08.296746 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.296750 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319,4] 2014-08-13 18:06:08.296765 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.296769 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.296783 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2691 2014-08-13 18:06:08.296788 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started/Stray 0.911461 4 0.000182 2014-08-13 18:06:08.296800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.296817 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] exit Started 0.911533 0 0.000000 2014-08-13 18:06:08.296818 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.880364 3 0.000335 2014-08-13 18:06:08.296837 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] enter Reset 2014-08-13 18:06:08.296839 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.296854 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194495 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194496 2014-08-13 18:06:08.296856 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.915251 0 0.000000 2014-08-13 18:06:08.296871 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194496 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: Reset advmap 2014-08-13 18:06:08.296880 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.296888 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194496 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.296897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.296906 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194496 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] acting_up_affected newup [45,319,4] newacting [45,319,4] 2014-08-13 18:06:08.296919 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.296924 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194496 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.296936 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.296941 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4]/[45,319] r=-1 lpr=194496 pi=193186-194494/35 crt=194422'13576 lcod 0'0 remapped NOTIFY m=50] set_last_peering_reset 194496 2014-08-13 18:06:08.296952 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'543 -> 194215'543 2014-08-13 18:06:08.296707 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.296975 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [45,319,4](45) acting [45,319](45)): not rw, up_thru 194492 up_from 192974 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [45,319,4](45) acting [45,319](45)) : primary up 192974-194492 does not include interval 2014-08-13 18:06:08.296978 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) 2014-08-13 18:06:08.296988 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.296980 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] flushed 2014-08-13 18:06:08.296997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194495/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] noting past interval(194495-194495 up [45,319,4](45) acting [45,319](45)) 2014-08-13 18:06:08.296999 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 is up to date, queueing in pending_activators 2014-08-13 18:06:08.297031 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297034 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] up [45,319,4] -> [45,319,4], acting [45,319] -> [45,319,4], acting_primary 45 -> 45, up_primary 45 -> 45, role -1 -> 2 2014-08-13 18:06:08.297037 7f5c835ef700 20 osd.4 194496 _dispatch 0x5154800 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.297046 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.297052 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.297041 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:08.297055 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297057 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297058 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.297078 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297081 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297083 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df24c0 pg_notify(24.34(41) epoch 194496) v5 2014-08-13 18:06:08.297073 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.297091 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_copy_ops 2014-08-13 18:06:08.297097 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.289 22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) uptodate 2014-08-13 18:06:08.297108 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_flush_ops 2014-08-13 18:06:08.297128 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.26 2014-08-13 18:06:08.297132 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df24c0 2014-08-13 18:06:08.297122 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 22.48e( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.297131 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change_cleanup 2014-08-13 18:06:08.297148 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_change 2014-08-13 18:06:08.297166 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit NotTrimming 2014-08-13 18:06:08.297188 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297192 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297191 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter NotTrimming 2014-08-13 18:06:08.297209 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] on_role_change 2014-08-13 18:06:08.297223 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.297241 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297244 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297247 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cd00 pg_notify(24.34(54) epoch 194496) v5 2014-08-13 18:06:08.297237 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] cancel_recovery 2014-08-13 18:06:08.297253 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] clear_recovery_state 2014-08-13 18:06:08.297271 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.297285 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.30 2014-08-13 18:06:08.297289 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cd00 2014-08-13 18:06:08.297291 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 18:06:08.297326 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297329 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297356 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297358 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297360 7f5c835ef700 20 osd.4 194496 _dispatch 0x21af260 pg_query(24.1de epoch 194496) v3 2014-08-13 18:06:08.297363 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] update_heartbeat_peers unchanged 2014-08-13 18:06:08.297381 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 18:06:08.297397 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.364 epoch 194496 2014-08-13 18:06:08.297401 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21af260 2014-08-13 18:06:08.297392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 sending log((0'0,194215'543], crt=191727'473) 2014-08-13 18:06:08.297385 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.297401 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Reset 0.000563 1 0.000558 2014-08-13 18:06:08.297422 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4ab4620 don't know epoch, doing nothing 2014-08-13 18:06:08.297415 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192966) from replica 364 2014-08-13 18:06:08.297426 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started 2014-08-13 18:06:08.297427 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.297445 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Start 2014-08-13 18:06:08.297464 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297468 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297463 7f5c7c7e4700 1 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: transitioning to Stray 2014-08-13 18:06:08.297464 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate peer osd.341 22.48e( DNE v 194215'543 lc 0'0 (0'0,194215'543] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(40) 2014-08-13 18:06:08.297483 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Start 0.000038 0 0.000000 2014-08-13 18:06:08.297488 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297513 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297503 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/Stray 2014-08-13 18:06:08.297516 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a38e00 pg_notify(24.3d(38) epoch 194496) v5 2014-08-13 18:06:08.297521 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.341 has 40 missing 2014-08-13 18:06:08.297523 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.297540 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 is on osd.4 2014-08-13 18:06:08.297542 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.297557 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.303 2014-08-13 18:06:08.297565 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a38e00 2014-08-13 18:06:08.297561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 is on osd.4 2014-08-13 18:06:08.297580 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 is on osd.4 2014-08-13 18:06:08.297603 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297606 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297597 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 is on osd.4 2014-08-13 18:06:08.297614 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 is on osd.4 2014-08-13 18:06:08.297642 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297630 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 is on osd.4 2014-08-13 18:06:08.297645 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297645 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297648 7f5c835ef700 20 osd.4 194496 _dispatch 0x524ee40 pg_notify(24.3d epoch 194496) v5 2014-08-13 18:06:08.297646 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 is on osd.4 2014-08-13 18:06:08.297664 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 is on osd.4 2014-08-13 18:06:08.297684 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.25 2014-08-13 18:06:08.297688 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524ee40 2014-08-13 18:06:08.297675 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.297685 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 is on osd.4 2014-08-13 18:06:08.297703 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 is on osd.4 2014-08-13 18:06:08.297705 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.297722 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297726 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297726 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297721 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 is on osd.4 2014-08-13 18:06:08.297745 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297747 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297734 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.297750 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1500 pg_info(1 pgs e194496:22.84) v4 2014-08-13 18:06:08.297739 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 is on osd.4 2014-08-13 18:06:08.297757 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.297776 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297758 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 is on osd.4 2014-08-13 18:06:08.297782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 is on osd.4 2014-08-13 18:06:08.297784 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 375 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.297800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 is on osd.4 2014-08-13 18:06:08.297817 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 is on osd.4 2014-08-13 18:06:08.297811 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.297834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 is on osd.4 2014-08-13 18:06:08.297850 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 is on osd.4 2014-08-13 18:06:08.297846 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,375 unchanged 2014-08-13 18:06:08.297869 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297867 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 is on osd.4 2014-08-13 18:06:08.297877 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.297884 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 is on osd.4 2014-08-13 18:06:08.297900 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.297902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 is on osd.4 2014-08-13 18:06:08.297919 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297920 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 is on osd.4 2014-08-13 18:06:08.297934 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.84) v4 from osd.28 2014-08-13 18:06:08.297926 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.297942 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1500 2014-08-13 18:06:08.297937 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 is on osd.4 2014-08-13 18:06:08.297947 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.297967 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.297955 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 is on osd.4 2014-08-13 18:06:08.297973 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 is on osd.4 2014-08-13 18:06:08.297988 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.297992 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.297974 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.297991 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 is on osd.4 2014-08-13 18:06:08.297998 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] requeue_ops 2014-08-13 18:06:08.298018 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298019 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.298022 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.298010 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 is on osd.4 2014-08-13 18:06:08.298025 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df5cc0 pg_info(1 pgs e194496:22.84) v4 2014-08-13 18:06:08.298029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 is on osd.4 2014-08-13 18:06:08.298025 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 375 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.298049 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 is on osd.4 2014-08-13 18:06:08.298064 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.84) v4 from osd.304 2014-08-13 18:06:08.298070 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df5cc0 2014-08-13 18:06:08.298052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.375 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.298077 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298067 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 is on osd.4 2014-08-13 18:06:08.298086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 is on osd.4 2014-08-13 18:06:08.298084 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 302 notify: (query_epoch:194496, epoch_sent:194496, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.298106 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.298110 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.298106 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 is on osd.4 2014-08-13 18:06:08.298109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 302, already have info from that osd, ignoring 2014-08-13 18:06:08.298130 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298132 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.298135 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.298137 7f5c835ef700 20 osd.4 194496 _dispatch 0x21afda0 pg_query(24.3f7 epoch 194496) v3 2014-08-13 18:06:08.298125 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 is on osd.4 2014-08-13 18:06:08.298136 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 346 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.298144 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 is on osd.4 2014-08-13 18:06:08.298164 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 is on osd.4 2014-08-13 18:06:08.298160 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info 2014-08-13 18:06:08.298189 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.330 epoch 194496 2014-08-13 18:06:08.298193 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21afda0 2014-08-13 18:06:08.298182 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 is on osd.4 2014-08-13 18:06:08.298208 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 is on osd.4 2014-08-13 18:06:08.298204 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193227) from replica 330 2014-08-13 18:06:08.298225 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 is on osd.4 2014-08-13 18:06:08.298249 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.298253 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.298257 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298248 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 is on osd.4 2014-08-13 18:06:08.298265 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 is on osd.4 2014-08-13 18:06:08.298266 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 330 notify: (query_epoch:194496, epoch_sent:194496, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.298290 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 330, already have info from that osd, ignoring 2014-08-13 18:06:08.298310 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298284 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] flushed 2014-08-13 18:06:08.298317 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 365 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.298282 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 18:06:08.298339 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.298351 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate: adding 289 as a source 2014-08-13 18:06:08.298368 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 is on osd.289 2014-08-13 18:06:08.298368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] osd.365 has stray content: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.298388 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 is on osd.289 2014-08-13 18:06:08.298394 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,367,378 -> 4,291,350,365,367,378 2014-08-13 18:06:08.298416 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.298421 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298406 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 is on osd.289 2014-08-13 18:06:08.298425 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 is on osd.289 2014-08-13 18:06:08.298428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 366 notify: (query_epoch:194496, epoch_sent:194496, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.298443 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 is on osd.289 2014-08-13 18:06:08.298449 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 366, already have info from that osd, ignoring 2014-08-13 18:06:08.298466 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298461 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 is on osd.289 2014-08-13 18:06:08.298473 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 63 notify: (query_epoch:194496, epoch_sent:194496, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.298479 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 is on osd.289 2014-08-13 18:06:08.298492 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 63, already have info from that osd, ignoring 2014-08-13 18:06:08.298510 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298497 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 is on osd.289 2014-08-13 18:06:08.298516 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 is on osd.289 2014-08-13 18:06:08.298516 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 26 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.298534 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 is on osd.289 2014-08-13 18:06:08.298539 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194478/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.298553 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 is on osd.289 2014-08-13 18:06:08.298570 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 is on osd.289 2014-08-13 18:06:08.298568 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 18:06:08.298588 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 is on osd.289 2014-08-13 18:06:08.298588 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last_epoch_started moved forward, rebuilding prior 2014-08-13 18:06:08.298607 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 is on osd.289 2014-08-13 18:06:08.298609 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.298625 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 is on osd.289 2014-08-13 18:06:08.298631 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194486-194487 up [26,30,340](26) acting [4](4)) 2014-08-13 18:06:08.298644 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 is on osd.289 2014-08-13 18:06:08.298649 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior interval(194479-194485 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.298662 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 is on osd.289 2014-08-13 18:06:08.298668 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] PriorSet: build_prior final: probe 4,26,30,340 down blocked_by {} 2014-08-13 18:06:08.298681 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 is on osd.289 2014-08-13 18:06:08.298688 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.298699 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 is on osd.289 2014-08-13 18:06:08.298708 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: have osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.298719 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 is on osd.289 2014-08-13 18:06:08.298733 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: already requested info from osd.30 2014-08-13 18:06:08.298737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 is on osd.289 2014-08-13 18:06:08.298750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: already requested info from osd.340 2014-08-13 18:06:08.298755 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 is on osd.289 2014-08-13 18:06:08.298774 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 is on osd.289 2014-08-13 18:06:08.298799 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298792 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 is on osd.289 2014-08-13 18:06:08.298809 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 is on osd.289 2014-08-13 18:06:08.298826 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 is on osd.289 2014-08-13 18:06:08.298823 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 28 notify: (query_epoch:194496, epoch_sent:194496, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.298844 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 is on osd.289 2014-08-13 18:06:08.298847 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 28, already have info from that osd, ignoring 2014-08-13 18:06:08.298869 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 is on osd.289 2014-08-13 18:06:08.298876 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 301 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.298884 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 is on osd.289 2014-08-13 18:06:08.298902 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 is on osd.289 2014-08-13 18:06:08.298898 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.298920 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 is on osd.289 2014-08-13 18:06:08.298938 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 is on osd.289 2014-08-13 18:06:08.298933 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] osd.301 has stray content: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.298956 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 is on osd.289 2014-08-13 18:06:08.298959 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,375 -> 4,10,286,301,375 2014-08-13 18:06:08.298979 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.298983 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.298975 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 is on osd.289 2014-08-13 18:06:08.298992 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 is on osd.289 2014-08-13 18:06:08.298990 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) 2014-08-13 18:06:08.299011 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 is on osd.289 2014-08-13 18:06:08.299019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: got info from osd.58 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) 2014-08-13 18:06:08.299029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 is on osd.289 2014-08-13 18:06:08.299044 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.089853 7 0.000375 2014-08-13 18:06:08.299048 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 is on osd.289 2014-08-13 18:06:08.299065 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.299066 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 is on osd.289 2014-08-13 18:06:08.299087 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 is on osd.289 2014-08-13 18:06:08.299093 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.299114 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.299132 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 18:06:08.299157 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.299105 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 18:06:08.299174 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] activate: adding 341 as a source 2014-08-13 18:06:08.299198 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22 191727'384 also missing on osd.341 2014-08-13 18:06:08.299218 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22 191727'385 also missing on osd.341 2014-08-13 18:06:08.299235 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22 191648'11 also missing on osd.341 2014-08-13 18:06:08.299251 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.299253 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22 191727'410 also missing on osd.341 2014-08-13 18:06:08.299278 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.299270 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22 191727'450 also missing on osd.341 2014-08-13 18:06:08.299286 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:08.299290 7f5c7c7e4700 7 osd.4 194496 do_notify osd 45 on 1 PGs 2014-08-13 18:06:08.299288 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22 191727'453 also missing on osd.341 2014-08-13 18:06:08.299317 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:08.299306 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22 191727'473 also missing on osd.341 2014-08-13 18:06:08.299321 7f5c7c7e4700 7 osd.4 194496 do_notify osd 328 on 1 PGs 2014-08-13 18:06:08.299325 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22 191727'428 also missing on osd.341 2014-08-13 18:06:08.299339 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f21600 already has epoch 194496 2014-08-13 18:06:08.299343 7f5c7c7e4700 7 osd.4 194496 do_notify osd 346 on 1 PGs 2014-08-13 18:06:08.299342 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22 191727'455 also missing on osd.341 2014-08-13 18:06:08.299363 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:08.299366 7f5c7c7e4700 7 osd.4 194496 do_notify osd 374 on 1 PGs 2014-08-13 18:06:08.299361 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22 191727'418 also missing on osd.341 2014-08-13 18:06:08.299378 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22 191727'398 also missing on osd.341 2014-08-13 18:06:08.299400 7f5c7c7e4700 10 osd.4 194496 send_pg_temp {24.1bb=[4,332,337]} 2014-08-13 18:06:08.299395 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22 191727'408 also missing on osd.341 2014-08-13 18:06:08.299413 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22 191648'67 also missing on osd.341 2014-08-13 18:06:08.299431 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22 191727'452 also missing on osd.341 2014-08-13 18:06:08.299448 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22 191648'125 also missing on osd.341 2014-08-13 18:06:08.299465 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22 191648'109 also missing on osd.341 2014-08-13 18:06:08.299466 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) 2014-08-13 18:06:08.299483 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22 191727'399 also missing on osd.341 2014-08-13 18:06:08.299491 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.9 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) 2014-08-13 18:06:08.299501 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22 191648'22 also missing on osd.341 2014-08-13 18:06:08.299517 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.097112 7 0.000391 2014-08-13 18:06:08.299522 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22 191727'403 also missing on osd.341 2014-08-13 18:06:08.299537 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.299540 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22 191648'265 also missing on osd.341 2014-08-13 18:06:08.299558 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.299558 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22 191648'8 also missing on osd.341 2014-08-13 18:06:08.299577 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194486 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.299579 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22 191648'51 also missing on osd.341 2014-08-13 18:06:08.299571 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'481 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.299597 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'495 -> 194215'495 2014-08-13 18:06:08.299600 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22 191727'406 also missing on osd.341 2014-08-13 18:06:08.299616 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:08.299618 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22 191648'273 also missing on osd.341 2014-08-13 18:06:08.299621 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.299636 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22 191727'380 also missing on osd.341 2014-08-13 18:06:08.299652 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] flushed 2014-08-13 18:06:08.299655 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22 191648'110 also missing on osd.341 2014-08-13 18:06:08.299674 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22 191648'159 also missing on osd.341 2014-08-13 18:06:08.299691 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22 191648'246 also missing on osd.341 2014-08-13 18:06:08.299709 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.299708 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22 191727'469 also missing on osd.341 2014-08-13 18:06:08.299726 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22 191648'1 also missing on osd.341 2014-08-13 18:06:08.299736 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) 2014-08-13 18:06:08.299743 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22 191648'371 also missing on osd.341 2014-08-13 18:06:08.299761 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22 191727'461 also missing on osd.341 2014-08-13 18:06:08.299760 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] state: got info from osd.54 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) 2014-08-13 18:06:08.299781 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22 191727'470 also missing on osd.341 2014-08-13 18:06:08.299783 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.080980 7 0.000336 2014-08-13 18:06:08.299799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22 191727'437 also missing on osd.341 2014-08-13 18:06:08.299803 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.299817 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22 191727'412 also missing on osd.341 2014-08-13 18:06:08.299823 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.299835 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22 191648'309 also missing on osd.341 2014-08-13 18:06:08.299841 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194486 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.299854 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22 191727'435 also missing on osd.341 2014-08-13 18:06:08.299858 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'473 -> 194215'473 2014-08-13 18:06:08.299872 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22 191727'458 also missing on osd.341 2014-08-13 18:06:08.299880 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.299891 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22 191727'427 also missing on osd.341 2014-08-13 18:06:08.299910 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] search_for_missing 94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22 191727'456 also missing on osd.341 2014-08-13 18:06:08.299965 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.299925 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {2954048e/benchmark_data_storage0111-ib.pocstor_51285_object3269/head//22=191727'384,4d7e048e/benchmark_data_storage0111-ib.pocstor_51285_object3367/head//22=191727'385,ced7148e/benchmark_data_storage0111-ib.pocstor_40745_object724/head//22=191648'11,df0b148e/benchmark_data_storage0111-ib.pocstor_51285_object11109/head//22=191727'410,6bcb148e/benchmark_data_storage0111-ib.pocstor_51285_object29671/head//22=191727'450,e49a348e/benchmark_data_storage0111-ib.pocstor_51285_object31892/head//22=191727'453,e58a448e/benchmark_data_storage0111-ib.pocstor_51285_object38456/head//22=191727'473,50ad448e/benchmark_data_storage0111-ib.pocstor_51285_object19100/head//22=191727'428,8be6648e/benchmark_data_storage0111-ib.pocstor_51285_object32198/head//22=191727'455,27ba748e/benchmark_data_storage0111-ib.pocstor_51285_object14259/head//22=191727'418,cdbe748e/benchmark_data_storage0111-ib.pocstor_51285_object7522/head//22=191727'398,fe10948e/benchmark_data_storage0111-ib.pocstor_51285_object10939/head//22=191727'408,714cb48e/benchmark_data_storage0111-ib.pocstor_40745_object3869/head//22=191648'67,c4aeb48e/benchmark_data_storage0111-ib.pocstor_51285_object30864/head//22=191727'452,f3bfb48e/benchmark_data_storage0111-ib.pocstor_40745_object7116/head//22=191648'125,1c41c48e/benchmark_data_storage0111-ib.pocstor_40745_object6496/head//22=191648'109,f8e6c48e/benchmark_data_storage0111-ib.pocstor_51285_object7570/head//22=191727'399,6891d48e/benchmark_data_storage0111-ib.pocstor_40745_object1180/head//22=191648'22,5107d48e/benchmark_data_storage0111-ib.pocstor_51285_object8444/head//22=191727'403,dc37d48e/benchmark_data_storage0111-ib.pocstor_40745_object15741/head//22=191648'265,8ba7d48e/benchmark_data_storage0111-ib.pocstor_40745_object593/head//22=191648'8,496ad48e/benchmark_data_storage0111-ib.pocstor_40745_object2701/head//22=191648'51,1538e48e/benchmark_data_storage0111-ib.pocstor_51285_object10454/head//22=191727'406,7b2bf48e/benchmark_data_storage0111-ib.pocstor_40745_object16605/head//22=191648'273,d5cbf48e/benchmark_data_storage0111-ib.pocstor_51285_object2401/head//22=191727'380,36c40c8e/benchmark_data_storage0111-ib.pocstor_40745_object6616/head//22=191648'110,fa660c8e/benchmark_data_storage0111-ib.pocstor_40745_object8615/head//22=191648'159,cf4a0c8e/benchmark_data_storage0111-ib.pocstor_40745_object14281/head//22=191648'246,d8942c8e/benchmark_data_storage0111-ib.pocstor_51285_object36637/head//22=191727'469,c4a24c8e/benchmark_data_storage0111-ib.pocstor_40745_object25/head//22=191648'1,57da4c8e/benchmark_data_storage0111-ib.pocstor_40745_object22736/head//22=191648'371,7e3e4c8e/benchmark_data_storage0111-ib.pocstor_51285_object35062/head//22=191727'461,4e455c8e/benchmark_data_storage0111-ib.pocstor_51285_object36837/head//22=191727'470,efe67c8e/benchmark_data_storage0111-ib.pocstor_51285_object23761/head//22=191727'437,970c7c8e/benchmark_data_storage0111-ib.pocstor_51285_object11411/head//22=191727'412,5fe9ac8e/benchmark_data_storage0111-ib.pocstor_40745_object18716/head//22=191648'309,bcfabc8e/benchmark_data_storage0111-ib.pocstor_51285_object22749/head//22=191727'435,837dc8e/benchmark_data_storage0111-ib.pocstor_51285_object33954/head//22=191727'458,8297ec8e/benchmark_data_storage0111-ib.pocstor_51285_object18339/head//22=191727'427,94d4fc8e/benchmark_data_storage0111-ib.pocstor_51285_object32240/head//22=191727'456} 2014-08-13 18:06:08.299988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.299990 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 20 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.300004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.300023 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 289,327,341,376 2014-08-13 18:06:08.300019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.300040 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.300051 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362 unchanged 2014-08-13 18:06:08.300057 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.300074 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300075 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.300081 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 20 notify: (query_epoch:194496, epoch_sent:194496, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.300091 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.300104 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 18:06:08.300122 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300112 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 crt=192888'463 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.300129 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) 2014-08-13 18:06:08.300150 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4423700 don't know epoch, doing nothing 2014-08-13 18:06:08.300150 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] state: got info from osd.321 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) 2014-08-13 18:06:08.300172 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.034485 7 0.000415 2014-08-13 18:06:08.300192 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] flushed 2014-08-13 18:06:08.300201 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.300221 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.300222 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.300240 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.300256 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'473 -> 194215'473 2014-08-13 18:06:08.300277 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.300275 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 18:06:08.300303 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.300310 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.300328 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.300344 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.300361 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300397 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 30 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.300390 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 crt=192888'431 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.300411 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.300437 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4425280 don't know epoch, doing nothing 2014-08-13 18:06:08.300422 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.300430 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.300447 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.300465 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.300453 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 18:06:08.300474 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 364 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.300491 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 2014-08-13 18:06:08.300498 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info 2014-08-13 18:06:08.300472 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] flushed 2014-08-13 18:06:08.300523 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.300529 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Started advmap 2014-08-13 18:06:08.300546 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45,4] 2014-08-13 18:06:08.300563 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.300581 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started/Stray 0.916147 4 0.000197 2014-08-13 18:06:08.300613 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300602 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] exit Started 0.916229 0 0.000000 2014-08-13 18:06:08.300623 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] enter Reset 2014-08-13 18:06:08.300622 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 289 notify: (query_epoch:194496, epoch_sent:194496, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.300640 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194495 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194496 2014-08-13 18:06:08.300647 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring 2014-08-13 18:06:08.300665 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300659 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194496 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: Reset advmap 2014-08-13 18:06:08.300675 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.300680 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.300682 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cb40 pg_info(1 pgs e194496:22.14a) v4 2014-08-13 18:06:08.300677 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194496 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.300672 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 45 notify: (query_epoch:194496, epoch_sent:194496, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.300697 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194496 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] acting_up_affected newup [343,45,4] newacting [343,45,4] 2014-08-13 18:06:08.300703 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 45, already have info from that osd, ignoring 2014-08-13 18:06:08.300722 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300716 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194496 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.300738 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.14a) v4 from osd.58 2014-08-13 18:06:08.300744 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cb40 2014-08-13 18:06:08.300735 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4]/[343,45] r=-1 lpr=194496 pi=193262-194494/35 crt=194422'14476 lcod 0'0 remapped NOTIFY m=41] set_last_peering_reset 194496 2014-08-13 18:06:08.300770 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [343,45,4](343) acting [343,45](343)): not rw, up_thru 194492 up_from 192942 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [343,45,4](343) acting [343,45](343)) : primary up 192942-194492 does not include interval 2014-08-13 18:06:08.300789 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.300793 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.300789 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194495/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] noting past interval(194495-194495 up [343,45,4](343) acting [343,45](343)) 2014-08-13 18:06:08.300808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] up [343,45,4] -> [343,45,4], acting [343,45] -> [343,45,4], acting_primary 343 -> 343, up_primary 343 -> 343, role -1 -> 2 2014-08-13 18:06:08.300827 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:08.300842 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.300857 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:08.300874 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_copy_ops 2014-08-13 18:06:08.300888 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_flush_ops 2014-08-13 18:06:08.300730 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.307237 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change_cleanup 2014-08-13 18:06:08.307250 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.307288 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.300944 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.307275 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_change 2014-08-13 18:06:08.307290 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307304 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307307 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ac560 pg_query(24.369 epoch 194496) v3 2014-08-13 18:06:08.307299 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit NotTrimming 2014-08-13 18:06:08.307298 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 17 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.307303 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.307319 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter NotTrimming 2014-08-13 18:06:08.307326 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info 2014-08-13 18:06:08.307338 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] on_role_change 2014-08-13 18:06:08.307345 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 crt=192888'451 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.307363 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.45 epoch 194496 2014-08-13 18:06:08.307367 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ac560 2014-08-13 18:06:08.307357 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:08.307376 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x44270c0 don't know epoch, doing nothing 2014-08-13 18:06:08.307374 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] cancel_recovery 2014-08-13 18:06:08.307391 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] clear_recovery_state 2014-08-13 18:06:08.307383 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193186) from replica 45 2014-08-13 18:06:08.307424 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.307408 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] flushed 2014-08-13 18:06:08.307410 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.307441 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307444 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307434 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 44 notify: (query_epoch:194496, epoch_sent:194496, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.307437 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map 2014-08-13 18:06:08.307472 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307475 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 44, already have info from that osd, ignoring 2014-08-13 18:06:08.307479 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.307477 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ab2a0 pg_query(24.2ec epoch 194496) v3 2014-08-13 18:06:08.307486 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) 2014-08-13 18:06:08.307514 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.374 epoch 194496 2014-08-13 18:06:08.307517 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ab2a0 2014-08-13 18:06:08.307507 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] state: got info from osd.358 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) 2014-08-13 18:06:08.307530 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.964394 7 0.000308 2014-08-13 18:06:08.307530 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193232) from replica 374 2014-08-13 18:06:08.307541 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] update_heartbeat_peers unchanged 2014-08-13 18:06:08.307555 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.307564 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters 2014-08-13 18:06:08.307578 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307581 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307582 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.307600 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307582 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Reset 0.006960 1 0.006905 2014-08-13 18:06:08.307603 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307606 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a440c0 pg_info(1 pgs e194496:22.14a) v4 2014-08-13 18:06:08.307604 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.307607 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started 2014-08-13 18:06:08.307623 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'526 -> 194215'526 2014-08-13 18:06:08.307626 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Start 2014-08-13 18:06:08.307645 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.14a) v4 from osd.58 2014-08-13 18:06:08.307650 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a440c0 2014-08-13 18:06:08.307643 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: transitioning to Stray 2014-08-13 18:06:08.307649 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.307663 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Start 0.000037 0 0.000000 2014-08-13 18:06:08.307689 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307692 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307686 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/Stray 2014-08-13 18:06:08.307710 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307713 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307715 7f5c835ef700 20 osd.4 194496 _dispatch 0x524f1c0 pg_info(1 pgs e194496:22.28f) v4 2014-08-13 18:06:08.307709 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.307745 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.307731 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.307755 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.28f) v4 from osd.9 2014-08-13 18:06:08.307760 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524f1c0 2014-08-13 18:06:08.307793 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307796 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307789 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194496 n=44 ec=192688 les/c 194492/194463 194495/194495/194468) 2014-08-13 18:06:08.307813 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307816 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307818 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d240 pg_info(1 pgs e194496:22.30f) v4 2014-08-13 18:06:08.307836 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.307815 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.19 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194496 n=44 ec=192688 les/c 194492/194463 194495/194495/194468) 2014-08-13 18:06:08.307855 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.30f) v4 from osd.54 2014-08-13 18:06:08.307859 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d240 2014-08-13 18:06:08.307843 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.955795 7 0.000247 2014-08-13 18:06:08.307866 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.307872 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.307895 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307898 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307887 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.307917 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.307903 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.307920 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307924 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.307908 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.307923 7f5c835ef700 20 osd.4 194496 _dispatch 0x588c280 pg_info(1 pgs e194496:22.3af) v4 2014-08-13 18:06:08.307928 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'13811 -> 194422'13811 2014-08-13 18:06:08.307961 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3af) v4 from osd.44 2014-08-13 18:06:08.307965 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588c280 2014-08-13 18:06:08.307954 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 2014-08-13 18:06:08.307989 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.308003 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308007 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.307997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] state: Activate Finished 2014-08-13 18:06:08.307998 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: verifying no want_acting [] targets didn't go down 2014-08-13 18:06:08.308031 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308034 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Started advmap 2014-08-13 18:06:08.308036 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee1f80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.308046 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.308051 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.308038 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] 2014-08-13 18:06:08.308054 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308056 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308068 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308055 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.308074 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308078 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308080 7f5c835ef700 20 osd.4 194496 _dispatch 0x5357e00 pg_info(1 pgs e194496:22.28f) v4 2014-08-13 18:06:08.308074 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary/Peering/WaitActingChange 0.891371 1 0.000221 2014-08-13 18:06:08.308094 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.308097 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started/Primary 0.891404 0 0.000000 2014-08-13 18:06:08.308118 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.28f) v4 from osd.9 2014-08-13 18:06:08.308125 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5357e00 2014-08-13 18:06:08.308118 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.308118 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] exit Started 0.922185 0 0.000000 2014-08-13 18:06:08.308141 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308142 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] enter Reset 2014-08-13 18:06:08.308148 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.308173 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308160 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194495 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194496 2014-08-13 18:06:08.308178 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308170 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] requeue_ops 2014-08-13 18:06:08.308200 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308204 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308180 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194496 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: Reset advmap 2014-08-13 18:06:08.308207 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308210 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889a40 pg_info(1 pgs e194496:22.30f) v4 2014-08-13 18:06:08.308188 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 crt=192888'500 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.308209 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194496 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.308208 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 356 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.308229 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 don't know epoch, doing nothing 2014-08-13 18:06:08.308230 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194496 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] acting_up_affected newup [4,348,331] newacting [4,332,337] 2014-08-13 18:06:08.308253 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.30f) v4 from osd.54 2014-08-13 18:06:08.308237 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.308258 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889a40 2014-08-13 18:06:08.308251 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194496 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.308264 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] flushed 2014-08-13 18:06:08.308271 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331] r=0 lpr=194496 pi=193644-194494/36 crt=194422'12517 lcod 0'0 mlcod 0'0 inactive m=48] set_last_peering_reset 194496 2014-08-13 18:06:08.308271 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] osd.356 has stray content: 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.308305 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308309 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308303 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65 -> 4,50,55,61,65,356 2014-08-13 18:06:08.308324 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.308330 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308331 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308334 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308316 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [4,348,331](4) acting [4,348,331](4)): not rw, up_thru 194486 up_from 194495 last_epoch_clean 194493 generate_past_intervals interval(194495-194495 up [4,348,331](4) acting [4,348,331](4)) : primary up 194495-194486 does not include interval 2014-08-13 18:06:08.308336 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0000 pg_notify(24.b6(44) epoch 194496) v5 2014-08-13 18:06:08.308337 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.308337 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194495/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] noting past interval(194495-194495 up [4,348,331](4) acting [4,348,331](4)) 2014-08-13 18:06:08.308374 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.315 2014-08-13 18:06:08.308378 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0000 2014-08-13 18:06:08.308362 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 18:06:08.308384 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308362 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] up [4,348,331] -> [4,348,331], acting [4,348,331] -> [4,332,337], acting_primary 4 -> 4, up_primary 4 -> 4, role 0 -> 0 2014-08-13 18:06:08.308388 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_primary_state 2014-08-13 18:06:08.308391 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 337 notify: (query_epoch:194496, epoch_sent:194496, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.308419 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308423 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308404 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] flushed 2014-08-13 18:06:08.308413 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 18:06:08.308433 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308418 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] agent_stop 2014-08-13 18:06:08.308440 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change 2014-08-13 18:06:08.308439 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 371 notify: (query_epoch:194496, epoch_sent:194496, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.308465 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308469 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308471 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353640 pg_notify(24.b6(44) epoch 194496) v5 2014-08-13 18:06:08.308462 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 371, already have info from that osd, ignoring 2014-08-13 18:06:08.308482 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308462 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194496:22153 2014-08-13 18:06:08.308487 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.308494 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 2014-08-13 18:06:08.308498 7f5c7c7e4700 7 osd.4 194496 do_notify osd 17 on 1 PGs 2014-08-13 18:06:08.308484 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:08.308513 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.315 2014-08-13 18:06:08.308516 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353640 2014-08-13 18:06:08.308508 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_copy_ops 2014-08-13 18:06:08.308534 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:08.308539 7f5c7c7e4700 7 osd.4 194496 do_notify osd 364 on 1 PGs 2014-08-13 18:06:08.308526 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_flush_ops 2014-08-13 18:06:08.308544 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:08.308559 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:08.308575 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:08.308590 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change_cleanup 2014-08-13 18:06:08.308606 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] on_change 2014-08-13 18:06:08.308615 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.308625 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit NotTrimming 2014-08-13 18:06:08.308641 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.308659 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308644 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter NotTrimming 2014-08-13 18:06:08.308665 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 32 notify: (query_epoch:194496, epoch_sent:194496, info:22.6bd( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.308673 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] [4,348,331] -> [4,332,337], replicas changed 2014-08-13 18:06:08.308685 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 32, already have info from that osd, ignoring 2014-08-13 18:06:08.308703 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] cancel_recovery 2014-08-13 18:06:08.308709 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] clear_recovery_state 2014-08-13 18:06:08.308709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 15 notify: (query_epoch:194496, epoch_sent:194496, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.308730 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.308745 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308732 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 15, already have info from that osd, ignoring 2014-08-13 18:06:08.308749 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308750 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308751 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map 2014-08-13 18:06:08.308757 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.308785 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308789 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308776 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.308792 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352f40 pg_info(1 pgs e194496:22.4aa) v4 2014-08-13 18:06:08.308795 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308772 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] update_heartbeat_peers 4,331,332,337,348,379 -> 4,331,332,337,348 2014-08-13 18:06:08.308802 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.308804 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 304 notify: (query_epoch:194496, epoch_sent:194496, info:22.84( v 194215'497 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.308809 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] take_waiters 2014-08-13 18:06:08.308834 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4aa) v4 from osd.321 2014-08-13 18:06:08.308824 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 304, already have info from that osd, ignoring 2014-08-13 18:06:08.308839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352f40 2014-08-13 18:06:08.308843 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308829 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Reset 0.000686 1 0.000754 2014-08-13 18:06:08.308851 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started 2014-08-13 18:06:08.308850 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 363 notify: (query_epoch:194496, epoch_sent:194496, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.308877 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308881 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308870 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Start 2014-08-13 18:06:08.308876 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 18:06:08.308896 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308900 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308904 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308888 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: transitioning to Primary 2014-08-13 18:06:08.308907 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0700 pg_info(1 pgs e194496:22.4aa) v4 2014-08-13 18:06:08.308886 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] _activate_committed 194496 telling primary 2014-08-13 18:06:08.308902 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 363 notify: (query_epoch:194496, epoch_sent:194496, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.308930 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 don't know epoch, doing nothing 2014-08-13 18:06:08.308913 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] exit Start 0.000043 0 0.000000 2014-08-13 18:06:08.308925 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 18:06:08.308946 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.308951 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4aa) v4 from osd.321 2014-08-13 18:06:08.308955 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0700 2014-08-13 18:06:08.308941 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary 2014-08-13 18:06:08.308962 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Peering 2014-08-13 18:06:08.308965 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed 2014-08-13 18:06:08.308993 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.308996 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.308974 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 286 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.308983 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetInfo 2014-08-13 18:06:08.309003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:08.309001 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.309026 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194495-194495 up [4,348,331](4) acting [4,348,331](4)) 2014-08-13 18:06:08.309033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 18:06:08.309056 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309046 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194492-194494 up [332,337,379](332) acting [332,337,379](332) maybe_went_rw) 2014-08-13 18:06:08.309064 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.309069 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior interval(194491-194491 up [332,337,379](332) acting [332,337](332)) 2014-08-13 18:06:08.309084 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] requeue_ops 2014-08-13 18:06:08.309102 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309088 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] PriorSet: build_prior final: probe 4,331,332,337,348,379 down blocked_by {} 2014-08-13 18:06:08.309108 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] up_thru 194495 < same_since 194496, must notify monitor 2014-08-13 18:06:08.309109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 374 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.309130 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194496:22154 2014-08-13 18:06:08.309131 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info 2014-08-13 18:06:08.309150 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.331 2014-08-13 18:06:08.309169 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.332 2014-08-13 18:06:08.309187 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.337 2014-08-13 18:06:08.309214 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.348 2014-08-13 18:06:08.309241 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309232 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: querying info from osd.379 2014-08-13 18:06:08.309251 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.309251 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 24 notify: (query_epoch:194496, epoch_sent:194496, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.309270 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.309275 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 24, already have info from that osd, ignoring 2014-08-13 18:06:08.309293 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309300 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.309320 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] requeue_ops 2014-08-13 18:06:08.309338 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309344 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.309362 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:08.309378 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.309382 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309392 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) 2014-08-13 18:06:08.309413 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] state: got info from osd.328 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) 2014-08-13 18:06:08.309435 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.060606 7 0.000400 2014-08-13 18:06:08.309435 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 18:06:08.309457 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.309470 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.309473 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.309478 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.309481 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351880 pg_info(1 pgs e194496:22.2d1) v4 2014-08-13 18:06:08.309477 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.309477 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: Started advmap 2014-08-13 18:06:08.309499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.309501 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.309529 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2d1) v4 from osd.358 2014-08-13 18:06:08.309520 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'506 -> 194215'506 2014-08-13 18:06:08.309535 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351880 2014-08-13 18:06:08.309522 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map 2014-08-13 18:06:08.309545 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.309575 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.309578 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.309562 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] take_waiters 2014-08-13 18:06:08.309584 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.309601 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.309619 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.309631 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.309646 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 18:06:08.309670 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.309659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) 2014-08-13 18:06:08.309676 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.309695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.309711 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.309727 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.309744 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.309761 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.309778 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2655 2014-08-13 18:06:08.309793 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.309810 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.901312 6 0.000475 2014-08-13 18:06:08.309829 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.309683 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] state: got info from osd.351 22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) 2014-08-13 18:06:08.309845 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.948616 0 0.000000 2014-08-13 18:06:08.309855 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.114086 7 0.000336 2014-08-13 18:06:08.309869 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.309879 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.309887 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194486 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.309901 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.309909 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.309919 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194486 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.309926 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.309938 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'521 -> 194215'521 2014-08-13 18:06:08.309942 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'466 -> 194215'466 2014-08-13 18:06:08.309963 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.309964 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.309985 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 is up to date, queueing in pending_activators 2014-08-13 18:06:08.310008 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.310008 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.286 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.310012 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.310031 7f5c835ef700 20 osd.4 194496 _dispatch 0x53547c0 pg_info(1 pgs e194496:24.3bf) v4 2014-08-13 18:06:08.310030 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.310053 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 is up to date, queueing in pending_activators 2014-08-13 18:06:08.310077 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:24.3bf) v4 from osd.19 2014-08-13 18:06:08.310081 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.310083 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53547c0 2014-08-13 18:06:08.310074 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] activate peer osd.343 22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.310095 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.310111 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.310126 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.310130 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.310113 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 331 notify: (query_epoch:194496, epoch_sent:194496, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.310131 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.310114 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'472 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.310152 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.310156 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.310141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 331, already have info from that osd, ignoring 2014-08-13 18:06:08.310162 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.310151 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.310164 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:08.310169 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.310158 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df1f80 pg_info(1 pgs e194496:22.2d1) v4 2014-08-13 18:06:08.310226 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2d1) v4 from osd.358 2014-08-13 18:06:08.310231 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df1f80 2014-08-13 18:06:08.310235 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.310218 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] flushed 2014-08-13 18:06:08.310267 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.310270 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.310276 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 18:06:08.310300 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.310306 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.310322 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.310339 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.310334 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) 2014-08-13 18:06:08.310368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] state: got info from osd.320 22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) 2014-08-13 18:06:08.310390 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.082398 7 0.000656 2014-08-13 18:06:08.310406 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.310409 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.310423 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.310430 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.310440 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.310459 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.310449 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194486 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.310467 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'527 -> 194215'527 2014-08-13 18:06:08.310488 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.310489 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 18:06:08.310512 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.310518 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.310534 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.310550 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.310566 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.310583 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.310582 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.310600 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.310617 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:3002 2014-08-13 18:06:08.310612 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) 2014-08-13 18:06:08.310633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.310635 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] state: got info from osd.334 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) 2014-08-13 18:06:08.310650 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.902801 6 0.000437 2014-08-13 18:06:08.310656 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.015967 7 0.000354 2014-08-13 18:06:08.310678 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.310686 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.310679 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 crt=192888'481 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.310695 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.946781 0 0.000000 2014-08-13 18:06:08.310718 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 2014-08-13 18:06:08.310706 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.310720 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.310726 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.310745 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'528 -> 194215'528 2014-08-13 18:06:08.310748 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] flushed 2014-08-13 18:06:08.310769 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.310850 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.310737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194486 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.310876 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) 2014-08-13 18:06:08.310898 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.367 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) 2014-08-13 18:06:08.310915 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.310918 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.985941 7 0.000310 2014-08-13 18:06:08.310936 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.310938 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.310952 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'479 -> 194215'479 2014-08-13 18:06:08.310957 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.310957 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 crt=192888'497 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.310976 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.310977 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.310996 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:08.310995 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'492 -> 194215'492 2014-08-13 18:06:08.311000 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 is up to date, queueing in pending_activators 2014-08-13 18:06:08.311018 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.311025 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.340 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.311028 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] flushed 2014-08-13 18:06:08.311047 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) 2014-08-13 18:06:08.311068 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 is up to date, queueing in pending_activators 2014-08-13 18:06:08.311100 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311087 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] activate peer osd.347 22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) uptodate 2014-08-13 18:06:08.311107 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-13 18:06:08.311122 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.311126 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 65 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.311139 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.311157 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.311153 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.311173 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.311202 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:08.311227 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311233 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.311239 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:08.311242 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.311243 7f5c7c7e4700 7 osd.4 194496 do_notify osd 374 on 1 PGs 2014-08-13 18:06:08.311262 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.311266 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.311268 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1dc0 pg_info(1 pgs e194496:22.2b9) v4 2014-08-13 18:06:08.311306 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 373 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.311306 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 18:06:08.311315 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.311338 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.311341 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.311334 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.311345 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.311365 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.311381 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2b9) v4 from osd.328 2014-08-13 18:06:08.311365 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] osd.373 has stray content: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.311387 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1dc0 2014-08-13 18:06:08.311385 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.311393 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362 -> 4,20,25,303,362,373 2014-08-13 18:06:08.311415 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.311420 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311437 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.311442 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.311426 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) 2014-08-13 18:06:08.311449 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] state: got info from osd.294 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) 2014-08-13 18:06:08.311471 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.032633 7 0.000774 2014-08-13 18:06:08.311477 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.311495 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.311495 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.311513 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.311515 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.311531 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.311534 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.311549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'517 -> 194215'517 2014-08-13 18:06:08.311555 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 2014-08-13 18:06:08.311571 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.311586 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.311592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Started advmap 2014-08-13 18:06:08.311611 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] 2014-08-13 18:06:08.311629 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, transitioning to Reset 2014-08-13 18:06:08.311654 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311647 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started/Stray 0.938958 4 0.000189 2014-08-13 18:06:08.311668 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] exit Started 0.939022 0 0.000000 2014-08-13 18:06:08.311687 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] enter Reset 2014-08-13 18:06:08.311681 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 324 notify: (query_epoch:194496, epoch_sent:194496, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.311704 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194495 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194496 2014-08-13 18:06:08.311711 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 18:06:08.311733 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311722 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: Reset advmap 2014-08-13 18:06:08.311740 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 0 notify: (query_epoch:194496, epoch_sent:194496, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.311740 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] _calc_past_interval_range: already have past intervals back to 194489 2014-08-13 18:06:08.311761 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-13 18:06:08.311764 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] acting_up_affected newup [6,374,4] newacting [6,374,4] 2014-08-13 18:06:08.311782 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] state: up or acting affected, calling start_peering_interval again 2014-08-13 18:06:08.311789 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 11 notify: (query_epoch:194496, epoch_sent:194496, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.311799 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4]/[6,374] r=-1 lpr=194496 pi=193125-194494/35 crt=194422'15992 lcod 0'0 remapped NOTIFY m=53] set_last_peering_reset 194496 2014-08-13 18:06:08.311811 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 11, already have info from that osd, ignoring 2014-08-13 18:06:08.311800 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'458 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.311831 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311841 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.311836 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] start_peering_interval: check_new_interval output: generate_past_intervals interval(194495-194495 up [6,374,4](6) acting [6,374](6)): not rw, up_thru 194492 up_from 192967 last_epoch_clean 194489 generate_past_intervals interval(194495-194495 up [6,374,4](6) acting [6,374](6)) : primary up 192967-194492 does not include interval 2014-08-13 18:06:08.311838 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 352 notify: (query_epoch:194496, epoch_sent:194496, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.311856 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194495/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] noting past interval(194495-194495 up [6,374,4](6) acting [6,374](6)) 2014-08-13 18:06:08.311859 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 352, already have info from that osd, ignoring 2014-08-13 18:06:08.311879 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311872 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] flushed 2014-08-13 18:06:08.311876 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] up [6,374,4] -> [6,374,4], acting [6,374] -> [6,374,4], acting_primary 6 -> 6, up_primary 6 -> 6, role -1 -> 2 2014-08-13 18:06:08.311885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.311898 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:08.311902 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 crt=192888'488 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.311909 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.311929 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.311929 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b12100 already has epoch 194496 2014-08-13 18:06:08.311918 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] publish_stats_to_osd -- not primary 2014-08-13 18:06:08.311935 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:08.311952 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_copy_ops 2014-08-13 18:06:08.311957 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] flushed 2014-08-13 18:06:08.311967 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_flush_ops 2014-08-13 18:06:08.311984 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change_cleanup 2014-08-13 18:06:08.312010 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.311999 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_change 2014-08-13 18:06:08.312013 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312015 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5940 pg_info(1 pgs e194496:22.2b9) v4 2014-08-13 18:06:08.312017 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit NotTrimming 2014-08-13 18:06:08.312034 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter NotTrimming 2014-08-13 18:06:08.312058 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.2b9) v4 from osd.328 2014-08-13 18:06:08.312050 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] on_role_change 2014-08-13 18:06:08.312063 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5940 2014-08-13 18:06:08.312066 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:08.312059 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 crt=192888'483 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.312082 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] cancel_recovery 2014-08-13 18:06:08.312109 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b2d6a0 already has epoch 194496 2014-08-13 18:06:08.312112 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312089 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 342 notify: (query_epoch:194496, epoch_sent:194496, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.312099 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] clear_recovery_state 2014-08-13 18:06:08.312124 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312123 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:08.312130 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.312146 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312147 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312155 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312158 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf0e00 pg_info(1 pgs e194496:22.220) v4 2014-08-13 18:06:08.312148 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] flushed 2014-08-13 18:06:08.312157 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map 2014-08-13 18:06:08.312164 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 342 notify: (query_epoch:194496, epoch_sent:194496, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.312208 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.220) v4 from osd.351 2014-08-13 18:06:08.312197 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:08.312213 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf0e00 2014-08-13 18:06:08.312215 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312223 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 314 notify: (query_epoch:194496, epoch_sent:194496, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.312251 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312254 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312242 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 314, already have info from that osd, ignoring 2014-08-13 18:06:08.312261 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312241 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194492 same_interval_since 194495 2014-08-13 18:06:08.312275 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312279 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312259 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] update_heartbeat_peers unchanged 2014-08-13 18:06:08.312281 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf5cc0 pg_info(1 pgs e194496:22.581) v4 2014-08-13 18:06:08.312268 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.312268 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 380 notify: (query_epoch:194496, epoch_sent:194496, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.312283 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters 2014-08-13 18:06:08.312294 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 380, already have info from that osd, ignoring 2014-08-13 18:06:08.312314 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312305 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Reset 0.000619 1 0.000565 2014-08-13 18:06:08.312322 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.581) v4 from osd.320 2014-08-13 18:06:08.312327 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf5cc0 2014-08-13 18:06:08.312321 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 368 notify: (query_epoch:194496, epoch_sent:194496, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.312329 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started 2014-08-13 18:06:08.312341 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 18:06:08.312348 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Start 2014-08-13 18:06:08.312372 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312375 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312365 7f5c7bde3700 1 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: transitioning to Stray 2014-08-13 18:06:08.312380 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312384 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Start 0.000036 0 0.000000 2014-08-13 18:06:08.312387 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 330 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.312404 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/Stray 2014-08-13 18:06:08.312410 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info 2014-08-13 18:06:08.312425 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.312441 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.312496 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 378 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.312534 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.312528 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.312556 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:08.312577 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312584 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 349 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.312585 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 18:06:08.312611 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.312605 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.349 activated and committed 2014-08-13 18:06:08.312623 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.312618 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: Started advmap 2014-08-13 18:06:08.312633 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312637 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312640 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354980 MRecoveryReserve REQUEST pgid: 22.220, query_epoch: 194496 v2 2014-08-13 18:06:08.312629 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 10 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.312636 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.312655 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map 2014-08-13 18:06:08.312653 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312691 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354980 2014-08-13 18:06:08.312683 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] update_heartbeat_peers 4,10,286,301,375 unchanged 2014-08-13 18:06:08.312717 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312721 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312706 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetInfo 0.155193 6 0.000654 2014-08-13 18:06:08.312730 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] take_waiters 2014-08-13 18:06:08.312731 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.312754 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312757 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312760 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea300 pg_info(1 pgs e194496:22.639) v4 2014-08-13 18:06:08.312749 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.312757 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.4 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312766 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.312786 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.312781 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312799 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.639) v4 from osd.367 2014-08-13 18:06:08.312805 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea300 2014-08-13 18:06:08.312804 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.312810 7f5c7bde3700 10 osd.4 pg_epoch: 194495 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 18:06:08.312835 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:08.312824 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.301 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312846 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312850 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312841 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-13 18:06:08.312847 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312870 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312873 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 194495, need_up_thru now false 2014-08-13 18:06:08.312876 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889880 MRecoveryReserve REQUEST pgid: 22.581, query_epoch: 194496 v2 2014-08-13 18:06:08.312878 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-13 18:06:08.312897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:08.312914 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889880 2014-08-13 18:06:08.312907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] calc_acting newest update on osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) shard 286 (up) backfill 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.375 (up) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) shard 10 (stray) accepted 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.312918 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-13 18:06:08.312938 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312942 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312906 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] flushed 2014-08-13 18:06:08.312932 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] actingbackfill is 4,10,286,375 2014-08-13 18:06:08.312938 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-13 18:06:08.312962 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.312965 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.312967 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1c00 pg_info(1 pgs e194496:22.69f) v4 2014-08-13 18:06:08.312955 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] choose_acting want [4,375,10] (== acting) backfill_targets 286 2014-08-13 18:06:08.312959 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 194496:2597 2014-08-13 18:06:08.312977 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-13 18:06:08.312981 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: leaving GetLog 2014-08-13 18:06:08.313004 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.69f) v4 from osd.334 2014-08-13 18:06:08.313009 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1c00 2014-08-13 18:06:08.312996 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.896577 2 0.000506 2014-08-13 18:06:08.313009 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetLog 0.000277 0 0.000000 2014-08-13 18:06:08.313017 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-13 18:06:08.313046 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313034 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.937018 0 0.000000 2014-08-13 18:06:08.313050 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313034 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.313057 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-13 18:06:08.313061 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.10 has no missing, identical log 2014-08-13 18:06:08.313074 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194486 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-13 18:06:08.313081 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: osd.375 has no missing, identical log 2014-08-13 18:06:08.313096 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:08.313100 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: still need up_thru update before going active 2014-08-13 18:06:08.313112 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-13 18:06:08.313123 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/GetMissing 0.000089 0 0.000000 2014-08-13 18:06:08.313129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 194215'511 -> 194215'511 2014-08-13 18:06:08.313153 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313157 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313159 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0540 pg_info(1 pgs e194496:22.639) v4 2014-08-13 18:06:08.313147 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.313169 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313153 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) 2014-08-13 18:06:08.313174 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 is up to date, queueing in pending_activators 2014-08-13 18:06:08.313204 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.639) v4 from osd.367 2014-08-13 18:06:08.313209 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0540 2014-08-13 18:06:08.313217 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.313248 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313251 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313247 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.313266 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313272 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313275 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313277 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf1500 pg_info(1 pgs e194496:22.630) v4 2014-08-13 18:06:08.313271 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 301 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.313295 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle_pg_notify from osd.301 2014-08-13 18:06:08.313315 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.630) v4 from osd.294 2014-08-13 18:06:08.313320 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf1500 2014-08-13 18:06:08.313299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.338 22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) uptodate 2014-08-13 18:06:08.313325 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) 2014-08-13 18:06:08.313314 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.301 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.313354 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313355 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313358 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313346 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 is up to date, queueing in pending_activators 2014-08-13 18:06:08.313377 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313381 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313383 7f5c835ef700 20 osd.4 194496 _dispatch 0x524c600 pg_info(1 pgs e194496:22.69f) v4 2014-08-13 18:06:08.313368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 58 notify: (query_epoch:194496, epoch_sent:194496, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.313371 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] flushed 2014-08-13 18:06:08.313390 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 58, already have info from that osd, ignoring 2014-08-13 18:06:08.313411 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313425 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.69f) v4 from osd.334 2014-08-13 18:06:08.313430 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524c600 2014-08-13 18:06:08.313419 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) 2014-08-13 18:06:08.313428 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate peer osd.376 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495) missing missing(28) 2014-08-13 18:06:08.313440 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) 2014-08-13 18:06:08.313465 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313471 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313465 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] exit Started/Stray 1.064364 7 0.000356 2014-08-13 18:06:08.313477 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.376 has 28 missing 2014-08-13 18:06:08.313485 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.313495 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.4 2014-08-13 18:06:08.313506 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.313515 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.4 2014-08-13 18:06:08.313525 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194486 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.313535 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.4 2014-08-13 18:06:08.313544 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'496 -> 194215'496 2014-08-13 18:06:08.313553 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.4 2014-08-13 18:06:08.313567 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.313571 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.4 2014-08-13 18:06:08.313588 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.4 2014-08-13 18:06:08.313605 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.4 2014-08-13 18:06:08.313621 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.4 2014-08-13 18:06:08.313637 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.4 2014-08-13 18:06:08.313654 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.4 2014-08-13 18:06:08.313678 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.313669 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.4 2014-08-13 18:06:08.313687 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.313691 7f5c7c7e4700 7 osd.4 194496 do_notify osd 330 on 1 PGs 2014-08-13 18:06:08.313686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.4 2014-08-13 18:06:08.313703 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.4 2014-08-13 18:06:08.313721 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.4 2014-08-13 18:06:08.313744 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313747 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313749 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a68000 pg_info(1 pgs e194496:22.1d2) v4 2014-08-13 18:06:08.313738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.4 2014-08-13 18:06:08.313754 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.4 2014-08-13 18:06:08.313759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 54 notify: (query_epoch:194496, epoch_sent:194496, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.313773 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.4 2014-08-13 18:06:08.313791 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1d2) v4 from osd.61 2014-08-13 18:06:08.313796 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a68000 2014-08-13 18:06:08.313789 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 54, already have info from that osd, ignoring 2014-08-13 18:06:08.313793 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.4 2014-08-13 18:06:08.313813 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.313811 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.4 2014-08-13 18:06:08.313820 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) 2014-08-13 18:06:08.313831 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.4 2014-08-13 18:06:08.313845 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313848 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.30 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) 2014-08-13 18:06:08.313848 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.4 2014-08-13 18:06:08.313869 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.313878 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313882 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6c80 pg_info(1 pgs e194496:22.630) v4 2014-08-13 18:06:08.313866 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.4 2014-08-13 18:06:08.313866 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.961597 7 0.000282 2014-08-13 18:06:08.313887 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.4 2014-08-13 18:06:08.313890 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.313905 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.4 2014-08-13 18:06:08.313920 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.630) v4 from osd.294 2014-08-13 18:06:08.313926 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6c80 2014-08-13 18:06:08.313911 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.323572 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.323577 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.313923 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.4 2014-08-13 18:06:08.323563 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194486 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.323610 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.323613 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.323601 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 18:06:08.323616 7f5c835ef700 20 osd.4 194496 _dispatch 0x2002400 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.323626 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.323632 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.323635 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.323637 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.323627 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.314024 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 crt=192888'464 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.323610 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.4 2014-08-13 18:06:08.323659 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.323662 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.323664 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3480 pg_notify(24.34(54) epoch 194496) v5 2014-08-13 18:06:08.323652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.4 2014-08-13 18:06:08.323675 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:08.323674 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.4 2014-08-13 18:06:08.323709 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.6 now has 194496 2014-08-13 18:06:08.323714 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.340 2014-08-13 18:06:08.323718 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3480 2014-08-13 18:06:08.323733 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.323692 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 18:06:08.323729 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] flushed 2014-08-13 18:06:08.323750 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 338 as a source 2014-08-13 18:06:08.323765 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.323768 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.323768 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 is on osd.338 2014-08-13 18:06:08.323787 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 is on osd.338 2014-08-13 18:06:08.323804 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 is on osd.338 2014-08-13 18:06:08.323821 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 is on osd.338 2014-08-13 18:06:08.323839 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.323838 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 is on osd.338 2014-08-13 18:06:08.323855 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 is on osd.338 2014-08-13 18:06:08.323872 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 is on osd.338 2014-08-13 18:06:08.323878 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 285 info: 22.f3( v 194215'506 lc 0'0 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.323900 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.323890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 is on osd.338 2014-08-13 18:06:08.323909 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 is on osd.338 2014-08-13 18:06:08.323907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.285 activated and committed 2014-08-13 18:06:08.323930 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.323927 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 is on osd.338 2014-08-13 18:06:08.323937 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 45 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.323945 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 is on osd.338 2014-08-13 18:06:08.323960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info 2014-08-13 18:06:08.323964 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 is on osd.338 2014-08-13 18:06:08.323982 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 is on osd.338 2014-08-13 18:06:08.323999 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 is on osd.338 2014-08-13 18:06:08.324016 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 is on osd.338 2014-08-13 18:06:08.324035 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324032 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 is on osd.338 2014-08-13 18:06:08.324050 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 is on osd.338 2014-08-13 18:06:08.324043 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 17 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.324067 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 is on osd.338 2014-08-13 18:06:08.324070 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.324085 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 is on osd.338 2014-08-13 18:06:08.324380 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 is on osd.338 2014-08-13 18:06:08.324381 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 unchanged 2014-08-13 18:06:08.324408 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324399 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 is on osd.338 2014-08-13 18:06:08.324417 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 is on osd.338 2014-08-13 18:06:08.324416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.324436 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 is on osd.338 2014-08-13 18:06:08.324439 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info 2014-08-13 18:06:08.324455 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 is on osd.338 2014-08-13 18:06:08.324472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 is on osd.338 2014-08-13 18:06:08.324490 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 is on osd.338 2014-08-13 18:06:08.324513 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324507 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 is on osd.338 2014-08-13 18:06:08.324524 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 is on osd.338 2014-08-13 18:06:08.324522 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.324544 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info 2014-08-13 18:06:08.324541 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 18:06:08.324592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] activate: adding 376 as a source 2014-08-13 18:06:08.324610 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22 191727'373 also missing on osd.376 2014-08-13 18:06:08.324627 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22 191648'48 also missing on osd.376 2014-08-13 18:06:08.324641 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324644 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22 191727'415 also missing on osd.376 2014-08-13 18:06:08.324649 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.324662 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22 191648'126 also missing on osd.376 2014-08-13 18:06:08.324673 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] requeue_ops 2014-08-13 18:06:08.324693 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22 191727'441 also missing on osd.376 2014-08-13 18:06:08.324667 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'492 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.324711 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:08.324701 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22 191727'370 also missing on osd.376 2014-08-13 18:06:08.324700 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.324720 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22 191727'404 also missing on osd.376 2014-08-13 18:06:08.324725 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.324738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22 191648'258 also missing on osd.376 2014-08-13 18:06:08.324742 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] flushed 2014-08-13 18:06:08.324757 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22 191727'422 also missing on osd.376 2014-08-13 18:06:08.324773 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.324757 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 18:06:08.324779 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.324781 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.324782 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf16c0 pg_info(1 pgs e194496:22.116) v4 2014-08-13 18:06:08.324778 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22 191648'8 also missing on osd.376 2014-08-13 18:06:08.324789 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) 2014-08-13 18:06:08.324799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22 191727'397 also missing on osd.376 2014-08-13 18:06:08.324811 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: got info from osd.337 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) 2014-08-13 18:06:08.324818 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22 191648'337 also missing on osd.376 2014-08-13 18:06:08.324832 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.116) v4 from osd.345 2014-08-13 18:06:08.324839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf16c0 2014-08-13 18:06:08.324837 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22 191727'392 also missing on osd.376 2014-08-13 18:06:08.324837 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.983712 7 0.000316 2014-08-13 18:06:08.324858 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22 191648'153 also missing on osd.376 2014-08-13 18:06:08.324864 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.324889 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.324876 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22 191648'320 also missing on osd.376 2014-08-13 18:06:08.324893 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.324886 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.324896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22 191727'384 also missing on osd.376 2014-08-13 18:06:08.324906 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194486 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.324915 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22 191727'371 also missing on osd.376 2014-08-13 18:06:08.324924 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'495 -> 194215'495 2014-08-13 18:06:08.324934 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22 191727'425 also missing on osd.376 2014-08-13 18:06:08.324949 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.324953 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22 191727'412 also missing on osd.376 2014-08-13 18:06:08.324971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22 191727'375 also missing on osd.376 2014-08-13 18:06:08.324989 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22 191648'189 also missing on osd.376 2014-08-13 18:06:08.325006 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22 191727'424 also missing on osd.376 2014-08-13 18:06:08.325023 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22 191648'304 also missing on osd.376 2014-08-13 18:06:08.325037 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.325039 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22 191727'411 also missing on osd.376 2014-08-13 18:06:08.325064 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22 191648'58 also missing on osd.376 2014-08-13 18:06:08.325067 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) 2014-08-13 18:06:08.325082 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22 191727'405 also missing on osd.376 2014-08-13 18:06:08.325092 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] state: got info from osd.371 22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) 2014-08-13 18:06:08.325102 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22 191648'198 also missing on osd.376 2014-08-13 18:06:08.325116 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.982165 7 0.000309 2014-08-13 18:06:08.325120 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] search_for_missing 7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22 191727'390 also missing on osd.376 2014-08-13 18:06:08.325135 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.325155 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.325136 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {7d8e03d9/benchmark_data_storage0111-ib.pocstor_51285_object5363/head//22=191727'373,68cc13d9/benchmark_data_storage0111-ib.pocstor_40745_object2770/head//22=191648'48,b0bd13d9/benchmark_data_storage0111-ib.pocstor_51285_object25869/head//22=191727'415,a80f13d9/benchmark_data_storage0111-ib.pocstor_40745_object7306/head//22=191648'126,d26343d9/benchmark_data_storage0111-ib.pocstor_51285_object38261/head//22=191727'441,98f543d9/benchmark_data_storage0111-ib.pocstor_51285_object2603/head//22=191727'370,a68b63d9/benchmark_data_storage0111-ib.pocstor_51285_object21165/head//22=191727'404,8fcd63d9/benchmark_data_storage0111-ib.pocstor_40745_object15055/head//22=191648'258,22c583d9/benchmark_data_storage0111-ib.pocstor_51285_object29893/head//22=191727'422,4b4783d9/benchmark_data_storage0111-ib.pocstor_40745_object821/head//22=191648'8,4ab8a3d9/benchmark_data_storage0111-ib.pocstor_51285_object17323/head//22=191727'397,b650b3d9/benchmark_data_storage0111-ib.pocstor_40745_object21173/head//22=191648'337,41b7b3d9/benchmark_data_storage0111-ib.pocstor_51285_object14176/head//22=191727'392,309bb3d9/benchmark_data_storage0111-ib.pocstor_40745_object8983/head//22=191648'153,e1dad3d9/benchmark_data_storage0111-ib.pocstor_40745_object19926/head//22=191648'320,7bc8e3d9/benchmark_data_storage0111-ib.pocstor_51285_object12053/head//22=191727'384,e17ee3d9/benchmark_data_storage0111-ib.pocstor_51285_object4311/head//22=191727'371,1ad31bd9/benchmark_data_storage0111-ib.pocstor_51285_object30532/head//22=191727'425,63313bd9/benchmark_data_storage0111-ib.pocstor_51285_object24639/head//22=191727'412,6f63bd9/benchmark_data_storage0111-ib.pocstor_51285_object7042/head//22=191727'375,15a15bd9/benchmark_data_storage0111-ib.pocstor_40745_object11164/head//22=191648'189,3b345bd9/benchmark_data_storage0111-ib.pocstor_51285_object30422/head//22=191727'424,4002bbd9/benchmark_data_storage0111-ib.pocstor_40745_object19106/head//22=191648'304,a084cbd9/benchmark_data_storage0111-ib.pocstor_51285_object24209/head//22=191727'411,bd09dbd9/benchmark_data_storage0111-ib.pocstor_40745_object3173/head//22=191648'58,38addbd9/benchmark_data_storage0111-ib.pocstor_51285_object21242/head//22=191727'405,a7deebd9/benchmark_data_storage0111-ib.pocstor_40745_object11653/head//22=191648'198,7479fbd9/benchmark_data_storage0111-ib.pocstor_51285_object13696/head//22=191727'390} 2014-08-13 18:06:08.325172 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194486 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.325188 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-13 18:06:08.325198 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'493 -> 194215'493 2014-08-13 18:06:08.325211 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 194389 2014-08-13 18:06:08.325220 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.325237 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 293,338,373,376,382,396,397 2014-08-13 18:06:08.325255 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.325273 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating 2014-08-13 18:06:08.325291 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 194495 while current is 194496 2014-08-13 18:06:08.325313 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.325307 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 NullEvt 2014-08-13 18:06:08.325339 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 304 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.325361 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info 2014-08-13 18:06:08.325432 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.325439 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.325447 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.325473 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.325483 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.325461 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 crt=192888'459 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.325488 7f5c7bde3700 7 osd.4 194496 do_notify osd 6 on 1 PGs 2014-08-13 18:06:08.325470 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.325498 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:08.325505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 18:06:08.325526 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.325677 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:08.325689 7f5c7bde3700 7 osd.4 194496 do_notify osd 15 on 1 PGs 2014-08-13 18:06:08.325671 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] flushed 2014-08-13 18:06:08.325720 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 2014-08-13 18:06:08.325725 7f5c7bde3700 7 osd.4 194496 do_notify osd 31 on 1 PGs 2014-08-13 18:06:08.325709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) 2014-08-13 18:06:08.325739 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] state: got info from osd.328 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) 2014-08-13 18:06:08.325761 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.941879 7 0.000270 2014-08-13 18:06:08.325781 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.325811 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.325801 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.325819 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.325822 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf4ec0 pg_info(1 pgs e194496:22.791) v4 2014-08-13 18:06:08.325824 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194486 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.325840 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'512 -> 194215'512 2014-08-13 18:06:08.325868 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.791) v4 from osd.30 2014-08-13 18:06:08.325873 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf4ec0 2014-08-13 18:06:08.325863 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.325887 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 crt=192888'459 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.325923 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.325926 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.325928 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4851e40 don't know epoch, doing nothing 2014-08-13 18:06:08.325926 7f5c7bde3700 10 osd.4 194496 send_incremental_map 194495 -> 194496 to 0x3cf1600 192.168.1.112:6832/6655 2014-08-13 18:06:08.325953 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.325971 7f5c7bde3700 10 osd.4 194496 note_peer_epoch osd.51 has 194496 2014-08-13 18:06:08.325975 7f5c7bde3700 7 osd.4 194496 do_notify osd 51 on 1 PGs 2014-08-13 18:06:08.325962 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] flushed 2014-08-13 18:06:08.326003 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] _activate_committed 194496 peer_activated now 4 last_epoch_started 194493 same_interval_since 194495 2014-08-13 18:06:08.326004 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 351 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.326025 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] flushed 2014-08-13 18:06:08.326032 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.326064 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 unchanged 2014-08-13 18:06:08.326085 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326077 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.326088 7f5c7bde3700 7 osd.4 194496 do_notify osd 285 on 1 PGs 2014-08-13 18:06:08.326092 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 320 notify: (query_epoch:194496, epoch_sent:194496, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.326120 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 don't know epoch, doing nothing 2014-08-13 18:06:08.326124 7f5c7bde3700 7 osd.4 194496 do_notify osd 292 on 1 PGs 2014-08-13 18:06:08.326114 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 320, already have info from that osd, ignoring 2014-08-13 18:06:08.326132 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326148 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a2520 don't know epoch, doing nothing 2014-08-13 18:06:08.326152 7f5c7bde3700 7 osd.4 194496 do_notify osd 323 on 1 PGs 2014-08-13 18:06:08.326139 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 367 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.326172 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.326176 7f5c7bde3700 7 osd.4 194496 do_notify osd 343 on 1 PGs 2014-08-13 18:06:08.326162 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.326207 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:08.326228 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326398 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:08.326408 7f5c7bde3700 7 osd.4 194496 do_notify osd 345 on 1 PGs 2014-08-13 18:06:08.326418 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.326423 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.326425 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353800 pg_info(1 pgs e194496:22.791) v4 2014-08-13 18:06:08.326398 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 65 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.326435 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:08.326438 7f5c7bde3700 7 osd.4 194496 do_notify osd 376 on 1 PGs 2014-08-13 18:06:08.326437 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.65 activated and committed 2014-08-13 18:06:08.326460 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326489 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing 2014-08-13 18:06:08.326495 7f5c7bde3700 7 osd.4 194496 do_notify osd 396 on 1 PGs 2014-08-13 18:06:08.326487 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) 2014-08-13 18:06:08.326520 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4854a40 don't know epoch, doing nothing 2014-08-13 18:06:08.326512 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: got info from osd.373 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) 2014-08-13 18:06:08.326530 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.296 on 1 PGs 2014-08-13 18:06:08.326535 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.973054 7 0.000285 2014-08-13 18:06:08.326552 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x43f3700 don't know epoch, doing nothing 2014-08-13 18:06:08.326556 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.307 on 1 PGs 2014-08-13 18:06:08.326560 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.326575 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:08.326578 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.320 on 1 PGs 2014-08-13 18:06:08.326581 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.326599 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194486 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.326616 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'522 -> 194215'522 2014-08-13 18:06:08.326626 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 crt=192888'478 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.326640 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.326678 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.791) v4 from osd.30 2014-08-13 18:06:08.326696 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353800 2014-08-13 18:06:08.326695 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4afdee0 don't know epoch, doing nothing 2014-08-13 18:06:08.326731 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326737 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] flushed 2014-08-13 18:06:08.326752 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.326756 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.326773 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing 2014-08-13 18:06:08.326778 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.331 on 1 PGs 2014-08-13 18:06:08.326762 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) 2014-08-13 18:06:08.326786 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.326789 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.326791 7f5c835ef700 20 osd.4 194496 _dispatch 0x21aaee0 pg_query(24.cd epoch 194496) v3 2014-08-13 18:06:08.326786 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: got info from osd.397 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) 2014-08-13 18:06:08.326803 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing 2014-08-13 18:06:08.326807 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.332 on 1 PGs 2014-08-13 18:06:08.326808 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.975803 7 0.000287 2014-08-13 18:06:08.326829 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:08.326833 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.337 on 1 PGs 2014-08-13 18:06:08.326832 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.6 epoch 194496 2014-08-13 18:06:08.326837 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21aaee0 2014-08-13 18:06:08.326828 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.326852 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:08.326855 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.344 on 1 PGs 2014-08-13 18:06:08.326849 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.326866 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.326882 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 2014-08-13 18:06:08.326905 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.326896 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194422'15992 ec=192688 les/c 194493/194489 194495/194496/192967) from replica 6 2014-08-13 18:06:08.326975 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x571b700 don't know epoch, doing nothing 2014-08-13 18:06:08.326983 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.348 on 1 PGs 2014-08-13 18:06:08.326988 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.326993 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.326996 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.327010 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2d120 don't know epoch, doing nothing 2014-08-13 18:06:08.327014 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.379 on 1 PGs 2014-08-13 18:06:08.327013 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.327022 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:08.327023 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.327026 7f5c7c7e4700 7 osd.4 194496 do_notify osd 45 on 1 PGs 2014-08-13 18:06:08.327039 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 286 2014-08-13 18:06:08.327174 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:08.327169 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.327030 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.327208 7f5c7bde3700 20 osd.4 194496 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) to shard 287 2014-08-13 18:06:08.327218 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a7bc0 don't know epoch, doing nothing 2014-08-13 18:06:08.327217 7f5c835ef700 20 osd.4 194496 _dispatch 0x4de9dc0 pg_info(1 pgs e194496:22.4fe) v4 2014-08-13 18:06:08.327222 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:08.327264 7f5c7bde3700 20 osd.4 194496 Sending info 22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) to shard 289 2014-08-13 18:06:08.327273 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:08.327257 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] flushed 2014-08-13 18:06:08.327287 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4fe) v4 from osd.337 2014-08-13 18:06:08.327294 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4de9dc0 2014-08-13 18:06:08.327308 7f5c7bde3700 20 osd.4 194496 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) to shard 309 2014-08-13 18:06:08.327319 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.327344 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.327354 7f5c7c7e4700 7 osd.4 194496 do_notify osd 304 on 1 PGs 2014-08-13 18:06:08.327380 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) to shard 338 2014-08-13 18:06:08.327391 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b1b5a0 don't know epoch, doing nothing 2014-08-13 18:06:08.327458 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.327618 7f5c7bde3700 20 osd.4 194496 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 340 2014-08-13 18:06:08.327629 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b10b00 don't know epoch, doing nothing 2014-08-13 18:06:08.327673 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] flushed 2014-08-13 18:06:08.327724 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.327752 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 343 2014-08-13 18:06:08.327764 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.327762 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.327773 7f5c7c7e4700 7 osd.4 194496 do_notify osd 356 on 2 PGs 2014-08-13 18:06:08.327907 7f5c7bde3700 20 osd.4 194496 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 347 2014-08-13 18:06:08.327924 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing 2014-08-13 18:06:08.327950 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) to shard 376 2014-08-13 18:06:08.327959 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:08.328173 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328189 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328181 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 349 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.328216 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328222 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4440 pg_info(1 pgs e194496:22.4fe) v4 2014-08-13 18:06:08.328217 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 349, already have info from that osd, ignoring 2014-08-13 18:06:08.328238 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.328290 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4fe) v4 from osd.337 2014-08-13 18:06:08.328295 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4440 2014-08-13 18:06:08.328317 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 309 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:08.328343 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 309, already have info from that osd, ignoring 2014-08-13 18:06:08.328361 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 365 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495)) 2014-08-13 18:06:08.328386 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 18:06:08.328403 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328417 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328421 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328408 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 306 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:08.328427 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 306, already have info from that osd, ignoring 2014-08-13 18:06:08.328443 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 343 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.328464 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328468 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328470 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ada00 pg_query(24.206 epoch 194496) v3 2014-08-13 18:06:08.328469 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info 2014-08-13 18:06:08.328508 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.343 epoch 194496 2014-08-13 18:06:08.328512 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ada00 2014-08-13 18:06:08.328556 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 15 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.328587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info 2014-08-13 18:06:08.328665 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328673 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194492/194463 194495/194495/193205) 2014-08-13 18:06:08.328694 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.396 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194492/194463 194495/194495/193205) 2014-08-13 18:06:08.328719 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.934997 7 0.000224 2014-08-13 18:06:08.328742 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.328726 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193262) from replica 343 2014-08-13 18:06:08.328764 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.328784 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.328799 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328803 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328802 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'15411 -> 194422'15411 2014-08-13 18:06:08.328829 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328831 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328833 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888c40 pg_notify(24.181(38) epoch 194496) v5 2014-08-13 18:06:08.328878 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] state: Activate Finished 2014-08-13 18:06:08.328895 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.296 2014-08-13 18:06:08.328899 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888c40 2014-08-13 18:06:08.328941 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.328944 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.328952 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.328984 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 324 notify: (query_epoch:194496, epoch_sent:194496, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.328247 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 382 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:08.329049 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 382, already have info from that osd, ignoring 2014-08-13 18:06:08.329066 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329073 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 10 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.329091 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 10, already have info from that osd, ignoring 2014-08-13 18:06:08.329107 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) 2014-08-13 18:06:08.329132 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) 2014-08-13 18:06:08.329153 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.957870 7 0.000312 2014-08-13 18:06:08.329182 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.329201 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.329220 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.329236 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'488 -> 194215'488 2014-08-13 18:06:08.329257 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.329266 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.329275 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.329278 7f5c835ef700 20 osd.4 194496 _dispatch 0x5420780 pg_query(24.20e epoch 194496) v3 2014-08-13 18:06:08.329336 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.15 epoch 194496 2014-08-13 18:06:08.329340 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329339 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5420780 2014-08-13 18:06:08.329367 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 285 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.329353 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193264) from replica 15 2014-08-13 18:06:08.329392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info 2014-08-13 18:06:08.329418 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.329421 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.329470 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329481 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.329503 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 18:06:08.329524 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 343 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.329551 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 343, already have info from that osd, ignoring 2014-08-13 18:06:08.329567 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.329589 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] _activate_committed 194496 telling primary 2014-08-13 18:06:08.329615 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing 2014-08-13 18:06:08.329647 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed 2014-08-13 18:06:08.329009 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 18:06:08.329694 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.329698 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.329704 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.329719 7f5c835ef700 20 osd.4 194496 _dispatch 0x5424b00 pg_query(24.1e4 epoch 194496) v3 2014-08-13 18:06:08.329702 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.329728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.329760 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.31 epoch 194496 2014-08-13 18:06:08.329764 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5424b00 2014-08-13 18:06:08.329759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] osd.327 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.329785 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 -> 4,17,28,315,327,330 2014-08-13 18:06:08.329804 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.329809 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.329816 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 339 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.329838 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.339 activated and committed 2014-08-13 18:06:08.329856 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.329863 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.329884 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.329901 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.329886 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.329907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 342 notify: (query_epoch:194496, epoch_sent:194496, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.329927 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing 2014-08-13 18:06:08.329927 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:08.329944 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.329574 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 31 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.329969 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info 2014-08-13 18:06:08.329981 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] flushed 2014-08-13 18:06:08.329997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 368 notify: (query_epoch:194496, epoch_sent:194496, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.330038 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 18:06:08.330066 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330086 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.330073 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.330096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:08.330114 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330100 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) 2014-08-13 18:06:08.330122 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.330105 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(log 194487'0 ec=192688 les/c 194492/194372 194495/194496/192978) from replica 31 2014-08-13 18:06:08.330124 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.323 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) 2014-08-13 18:06:08.330145 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.330163 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330147 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.958611 7 0.000284 2014-08-13 18:06:08.330170 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.330189 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330192 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330179 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.330201 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.330217 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330220 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330223 7f5c835ef700 20 osd.4 194496 _dispatch 0x588af40 pg_notify(24.1bb(2) epoch 194496) v5 2014-08-13 18:06:08.330219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.330231 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:08.330237 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'491 -> 194215'491 2014-08-13 18:06:08.330253 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330263 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.332 2014-08-13 18:06:08.330268 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588af40 2014-08-13 18:06:08.330261 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.330262 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 365 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.330288 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.330308 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330311 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330315 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330330 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330333 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330336 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0fc0 pg_info(1 pgs e194496:22.3d5) v4 2014-08-13 18:06:08.330322 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 10 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.330346 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle_pg_notify from osd.10 2014-08-13 18:06:08.330378 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.330379 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d5) v4 from osd.286 2014-08-13 18:06:08.330385 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0fc0 2014-08-13 18:06:08.330366 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.10 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.330405 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330420 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330424 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330413 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 301 info: 22.22c( v 194215'514 lc 0'0 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.330413 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) 2014-08-13 18:06:08.330444 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330447 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330436 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.301 activated and committed 2014-08-13 18:06:08.330450 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bb80 pg_notify(24.181(38) epoch 194496) v5 2014-08-13 18:06:08.330456 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330439 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: got info from osd.292 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) 2014-08-13 18:06:08.330462 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.947461 7 0.000262 2014-08-13 18:06:08.330464 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 289 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.330483 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.330486 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info 2014-08-13 18:06:08.330501 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.330518 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.330547 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.344 2014-08-13 18:06:08.330534 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 2014-08-13 18:06:08.330564 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.330550 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bb80 2014-08-13 18:06:08.330605 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330627 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330630 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330613 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 45 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.330643 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.330636 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.45 activated and committed 2014-08-13 18:06:08.330653 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.330668 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) 2014-08-13 18:06:08.330698 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330702 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330704 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351c00 pg_info(1 pgs e194496:22.1a0) v4 2014-08-13 18:06:08.330696 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.51 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) 2014-08-13 18:06:08.330673 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.330718 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.968160 7 0.000254 2014-08-13 18:06:08.330735 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:08.330737 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.330756 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.330743 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.330775 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c594a0 don't know epoch, doing nothing 2014-08-13 18:06:08.330773 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.330785 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1a0) v4 from osd.328 2014-08-13 18:06:08.330795 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351c00 2014-08-13 18:06:08.330791 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'501 -> 194215'501 2014-08-13 18:06:08.330813 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x49a2520 don't know epoch, doing nothing 2014-08-13 18:06:08.330807 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2937 2014-08-13 18:06:08.330817 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.330839 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.330842 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330845 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330867 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.330871 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.330873 7f5c835ef700 20 osd.4 194496 _dispatch 0x53571c0 pg_info(1 pgs e194496:22.3d9) v4 2014-08-13 18:06:08.330867 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] flushed 2014-08-13 18:06:08.330905 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.330930 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(info 0'0) 2014-08-13 18:06:08.330953 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info 2014-08-13 18:06:08.330963 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.330976 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:08.330979 7f5c7c7e4700 7 osd.4 194496 do_notify osd 15 on 1 PGs 2014-08-13 18:06:08.331035 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.331155 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d9) v4 from osd.338 2014-08-13 18:06:08.331163 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53571c0 2014-08-13 18:06:08.331187 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:08.331204 7f5c7c7e4700 7 osd.4 194496 do_notify osd 289 on 1 PGs 2014-08-13 18:06:08.331185 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 344 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.331215 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331220 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.331242 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331245 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331247 7f5c835ef700 20 osd.4 194496 _dispatch 0x588ee40 pg_notify(24.1bb epoch 194496) v5 2014-08-13 18:06:08.331254 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 unchanged 2014-08-13 18:06:08.331276 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.331284 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.331 2014-08-13 18:06:08.331287 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588ee40 2014-08-13 18:06:08.331320 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331329 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331349 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331351 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331353 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf40c0 pg_notify(24.181(17) epoch 194496) v5 2014-08-13 18:06:08.331356 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.331390 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.320 2014-08-13 18:06:08.331393 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf40c0 2014-08-13 18:06:08.331402 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.331407 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 don't know epoch, doing nothing 2014-08-13 18:06:08.331411 7f5c7c7e4700 7 osd.4 194496 do_notify osd 343 on 1 PGs 2014-08-13 18:06:08.331397 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 338 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:08.331429 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331433 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331423 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 338, already have info from that osd, ignoring 2014-08-13 18:06:08.331445 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.331447 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] flushed 2014-08-13 18:06:08.331466 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 397 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:08.331489 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 397, already have info from that osd, ignoring 2014-08-13 18:06:08.331517 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.331501 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.331511 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.331523 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 396 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:08.331539 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x3cf1600 already has epoch 194496 2014-08-13 18:06:08.331531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:08.331558 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 396, already have info from that osd, ignoring 2014-08-13 18:06:08.331564 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.331574 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] flushed 2014-08-13 18:06:08.331586 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.331614 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.331643 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.331667 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 2014-08-13 18:06:08.331686 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.331691 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331698 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 324 notify: (query_epoch:194496, epoch_sent:194496, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.331718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring 2014-08-13 18:06:08.331734 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331739 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.331762 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:08.331794 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331801 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.331822 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.331838 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331844 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 310 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.331862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.310 activated and committed 2014-08-13 18:06:08.331879 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 47 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.331903 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.47 activated and committed 2014-08-13 18:06:08.331920 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.331925 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 62 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.331943 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.62 activated and committed 2014-08-13 18:06:08.331966 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.331970 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.331960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.331973 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356c80 pg_info(1 pgs e194496:22.1fb) v4 2014-08-13 18:06:08.331975 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.332019 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4424360 don't know epoch, doing nothing 2014-08-13 18:06:08.332043 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1fb) v4 from osd.373 2014-08-13 18:06:08.332050 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356c80 2014-08-13 18:06:08.332093 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.332103 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.332113 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:08.332101 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:08.332142 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2737 2014-08-13 18:06:08.332133 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 18:06:08.332171 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332178 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 379 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.332176 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.332210 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.332241 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.332245 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 313 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.332266 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348 -> 4,331,332,337,348,379 2014-08-13 18:06:08.332287 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.332292 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332273 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.313 activated and committed 2014-08-13 18:06:08.332299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.332327 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.332326 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.332332 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.332319 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 327, already have info from that osd, ignoring 2014-08-13 18:06:08.332334 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354440 pg_info(1 pgs e194496:22.3b9) v4 2014-08-13 18:06:08.332340 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332336 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 5 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.332359 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.5 activated and committed 2014-08-13 18:06:08.332348 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 339 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.332387 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3b9) v4 from osd.397 2014-08-13 18:06:08.332377 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.332394 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354440 2014-08-13 18:06:08.332385 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 339, already have info from that osd, ignoring 2014-08-13 18:06:08.332394 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.332408 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332418 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.332430 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.332434 7f5c7bde3700 7 osd.4 194496 do_notify osd 6 on 1 PGs 2014-08-13 18:06:08.332434 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x449d280 don't know epoch, doing nothing 2014-08-13 18:06:08.332513 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x49a5960 don't know epoch, doing nothing 2014-08-13 18:06:08.332542 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3010 2014-08-13 18:06:08.332579 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.332650 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 2014-08-13 18:06:08.332661 7f5c7bde3700 7 osd.4 194496 do_notify osd 31 on 1 PGs 2014-08-13 18:06:08.332644 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:08.332692 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.332696 7f5c7bde3700 7 osd.4 194496 do_notify osd 285 on 1 PGs 2014-08-13 18:06:08.332692 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:08.332712 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.332730 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.332734 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.332722 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) 2014-08-13 18:06:08.332741 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.376 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) 2014-08-13 18:06:08.332775 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.332780 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.332764 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.958479 7 0.000290 2014-08-13 18:06:08.332782 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351f80 pg_info(1 pgs e194496:24.299) v4 2014-08-13 18:06:08.332769 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 293 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:08.332787 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-13 18:06:08.332798 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 293, already have info from that osd, ignoring 2014-08-13 18:06:08.332825 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332827 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:24.299) v4 from osd.396 2014-08-13 18:06:08.332834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.332854 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:08.332871 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332877 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.332812 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:08.332896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 18:06:08.332915 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332902 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-13 18:06:08.332921 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'485 -> 194215'485 2014-08-13 18:06:08.332921 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 289 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.332944 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring 2014-08-13 18:06:08.332946 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] state: Activate Finished 2014-08-13 18:06:08.332961 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.332968 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 286 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.332832 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351f80 2014-08-13 18:06:08.332986 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 286, already have info from that osd, ignoring 2014-08-13 18:06:08.333008 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.333015 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.333036 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333040 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333034 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 18:06:08.333052 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.333063 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.333063 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333066 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333069 7f5c835ef700 20 osd.4 194496 _dispatch 0x53563c0 pg_info(1 pgs e194496:22.3d9) v4 2014-08-13 18:06:08.333057 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.333077 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.58 activated and committed 2014-08-13 18:06:08.333095 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.333097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 324 info: 22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.333102 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.333126 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d9) v4 from osd.376 2014-08-13 18:06:08.333131 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53563c0 2014-08-13 18:06:08.333122 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.324 activated and committed 2014-08-13 18:06:08.333126 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.333141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.333157 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.333159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.55 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.333180 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing 2014-08-13 18:06:08.333195 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 -> 4,55,296,320,344 2014-08-13 18:06:08.333216 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.333221 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.333231 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333235 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333284 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333287 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333290 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352d80 pg_notify(24.1bb(40) epoch 194496) v5 2014-08-13 18:06:08.333332 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 2014-08-13 18:06:08.333355 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.379 2014-08-13 18:06:08.333359 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352d80 2014-08-13 18:06:08.333427 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333430 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333456 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2834 2014-08-13 18:06:08.333490 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.333489 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333493 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333496 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0e00 pg_info(1 pgs e194496:22.481) v4 2014-08-13 18:06:08.333499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 28 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.333523 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.333540 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.481) v4 from osd.309 2014-08-13 18:06:08.333545 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0e00 2014-08-13 18:06:08.333555 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 18:06:08.333546 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] _activate_committed 194496 telling primary 2014-08-13 18:06:08.333577 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.333582 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333585 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333588 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:08.333584 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.333607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.333618 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] flushed 2014-08-13 18:06:08.333635 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.333640 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.333642 7f5c835ef700 20 osd.4 194496 _dispatch 0x588c280 pg_notify(24.181(45) epoch 194496) v5 2014-08-13 18:06:08.333634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:08.333657 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.338834 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.307 2014-08-13 18:06:08.338839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588c280 2014-08-13 18:06:08.333664 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193151) 2014-08-13 18:06:08.333661 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 337 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.338889 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.338892 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.338859 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log request from 356 2014-08-13 18:06:08.338865 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.338929 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.338913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:08.338932 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.338934 7f5c835ef700 20 osd.4 194496 _dispatch 0x58896c0 pg_info(1 pgs e194496:22.29a) v4 2014-08-13 18:06:08.338937 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.338941 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info+missing+log since 194487'0 2014-08-13 18:06:08.338947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 343 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.338961 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending log((194422'17433,194422'17433], crt=194422'17433) missing(51) 2014-08-13 18:06:08.338979 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.29a) v4 from osd.345 2014-08-13 18:06:08.338984 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x58896c0 2014-08-13 18:06:08.338971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed 2014-08-13 18:06:08.338987 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.338991 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.339006 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.339022 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339023 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339027 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339046 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339049 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339051 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421c20 pg_query(24.1e6 epoch 194496) v3 2014-08-13 18:06:08.339097 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.285 epoch 194496 2014-08-13 18:06:08.339100 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421c20 2014-08-13 18:06:08.339112 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192975) from replica 285 2014-08-13 18:06:08.339176 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339179 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339200 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339202 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339204 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3f00 pg_info(1 pgs e194496:22.48e) v4 2014-08-13 18:06:08.339213 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing 2014-08-13 18:06:08.339212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.339244 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.48e) v4 from osd.289 2014-08-13 18:06:08.339249 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3f00 2014-08-13 18:06:08.339254 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.339243 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.339264 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339283 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2685 2014-08-13 18:06:08.339313 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.339291 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.339324 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339333 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339319 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.339355 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339358 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339361 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a840 pg_notify(24.1bb(40) epoch 194496) v5 2014-08-13 18:06:08.339352 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:08.339375 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339431 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.337 2014-08-13 18:06:08.339435 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a840 2014-08-13 18:06:08.339429 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 320 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.339436 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 65 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.339457 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.339460 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.65 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339485 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339495 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339499 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339490 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged 2014-08-13 18:06:08.339492 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.339513 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.339514 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339520 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 373 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:08.339537 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339547 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.339544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 18:06:08.339544 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339562 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339565 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.339564 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354d00 pg_info(1 pgs e194496:22.3d5) v4 2014-08-13 18:06:08.339573 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 289 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192968) 2014-08-13 18:06:08.339584 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 356 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.339593 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log request from 289 2014-08-13 18:06:08.339621 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d5) v4 from osd.343 2014-08-13 18:06:08.339626 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354d00 2014-08-13 18:06:08.339607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339631 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339637 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.339642 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info+missing+log since 194487'0 2014-08-13 18:06:08.339662 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending log((194422'17034,194422'17034], crt=194422'17034) missing(63) 2014-08-13 18:06:08.339658 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339682 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339681 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339683 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing 2014-08-13 18:06:08.339685 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339706 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339689 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 351 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.339709 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339711 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888380 pg_info(1 pgs e194496:22.12a) v4 2014-08-13 18:06:08.339713 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.339711 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339734 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339788 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.12a) v4 from osd.323 2014-08-13 18:06:08.339793 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888380 2014-08-13 18:06:08.339839 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339843 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339878 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339880 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339883 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc5e80 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.339893 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.339899 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.339884 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.339902 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339878 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 17 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193188) 2014-08-13 18:06:08.339909 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log request from 17 2014-08-13 18:06:08.339929 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.339909 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.339932 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.339934 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.339935 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889340 pg_info(1 pgs e194496:22.3ad) v4 2014-08-13 18:06:08.339941 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 330 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.339957 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info+missing+log since 194487'0 2014-08-13 18:06:08.339961 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.339974 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending log((194422'14353,194422'14353], crt=194422'14353) missing(49) 2014-08-13 18:06:08.339994 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 2014-08-13 18:06:08.339990 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 18:06:08.340008 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3ad) v4 from osd.292 2014-08-13 18:06:08.340012 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.340013 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889340 2014-08-13 18:06:08.340037 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340066 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340070 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340090 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340092 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340094 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b800 pg_info(1 pgs e194496:22.537) v4 2014-08-13 18:06:08.340133 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.537) v4 from osd.51 2014-08-13 18:06:08.340137 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b800 2014-08-13 18:06:08.340205 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340208 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340191 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 289 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.340200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194422'18845 ec=192688 les/c 194493/194493 194495/194496/192969) 2014-08-13 18:06:08.340226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed 2014-08-13 18:06:08.340226 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 2014-08-13 18:06:08.340245 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340247 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.340248 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340251 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356740 pg_info(1 pgs e194496:22.1b8) v4 2014-08-13 18:06:08.340255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 48 notify: (query_epoch:194496, epoch_sent:194496, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:08.340275 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 48, already have info from that osd, ignoring 2014-08-13 18:06:08.340274 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194422'18845 2014-08-13 18:06:08.340296 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.340310 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1b8) v4 from osd.347 2014-08-13 18:06:08.340297 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) 2014-08-13 18:06:08.340315 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356740 2014-08-13 18:06:08.340318 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.340302 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 17 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.340324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.17 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.340344 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340347 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.340370 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340373 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340402 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340405 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340407 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea4c0 pg_info(1 pgs e194496:22.1a0) v4 2014-08-13 18:06:08.340462 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1a0) v4 from osd.328 2014-08-13 18:06:08.340466 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea4c0 2014-08-13 18:06:08.340506 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340510 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340496 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 374 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.340502 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 356 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.340529 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.374 activated and committed 2014-08-13 18:06:08.340533 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340527 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.356 activated and committed 2014-08-13 18:06:08.340553 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340554 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.340553 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf55c0 pg_info(1 pgs e194496:22.1fb) v4 2014-08-13 18:06:08.340565 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 344 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.340571 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.340600 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1fb) v4 from osd.373 2014-08-13 18:06:08.340586 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.344 activated and committed 2014-08-13 18:06:08.340608 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340606 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf55c0 2014-08-13 18:06:08.340598 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340614 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.340633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.373 activated and committed 2014-08-13 18:06:08.340632 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged 2014-08-13 18:06:08.340651 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.340660 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.087649 8 0.000812 2014-08-13 18:06:08.340669 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.340691 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340694 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:08.340696 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340691 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.340718 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340721 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340724 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df55c0 pg_info(1 pgs e194496:22.3b9) v4 2014-08-13 18:06:08.340724 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing 2014-08-13 18:06:08.340718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340745 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2617 2014-08-13 18:06:08.340742 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340762 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3b9) v4 from osd.397 2014-08-13 18:06:08.340767 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df55c0 2014-08-13 18:06:08.340775 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340765 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.340782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.340786 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.340805 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops 2014-08-13 18:06:08.340824 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340827 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340806 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340828 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340836 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.340864 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.340872 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.340875 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3b80 MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 2014-08-13 18:06:08.340867 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.340885 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340897 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.340897 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:08.340941 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3b80 2014-08-13 18:06:08.340921 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] actingbackfill is 4,50,55,61,65 2014-08-13 18:06:08.340933 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 362 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.340947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] choose_acting want [4,50,65] (== acting) backfill_targets 55,61 2014-08-13 18:06:08.340958 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 362, already have info from that osd, ignoring 2014-08-13 18:06:08.340976 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.340980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: leaving GetLog 2014-08-13 18:06:08.341001 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341005 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 332 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.341006 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000315 0 0.000000 2014-08-13 18:06:08.341026 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341030 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341032 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b100 pg_info(1 pgs e194496:22.226) v4 2014-08-13 18:06:08.341034 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.341031 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.341055 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.50 has no missing, identical log 2014-08-13 18:06:08.341077 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.226) v4 from osd.376 2014-08-13 18:06:08.341082 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b100 2014-08-13 18:06:08.341065 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:08.341091 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.341080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.65 has no missing, identical log 2014-08-13 18:06:08.341098 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 307 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.341101 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: still need up_thru update before going active 2014-08-13 18:06:08.341121 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341125 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000091 0 0.000000 2014-08-13 18:06:08.341148 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.341147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.307 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341169 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341142 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341182 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 -> 4,55,296,307,320,344 2014-08-13 18:06:08.341206 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.341207 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341210 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341212 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.341212 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc7bc0 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.341214 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.341219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.341221 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.341240 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 50, already have info from that osd, ignoring 2014-08-13 18:06:08.341241 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.341259 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.341262 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341262 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341264 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.341266 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341291 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341294 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341278 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 296 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.341282 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 347 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.341300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 347, already have info from that osd, ignoring 2014-08-13 18:06:08.341326 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341297 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfc800 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.341333 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.341339 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.341342 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341345 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341331 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,307,320,344 unchanged 2014-08-13 18:06:08.341333 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.341355 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops 2014-08-13 18:06:08.341355 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.073293 6 0.001710 2014-08-13 18:06:08.341376 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341385 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341389 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341392 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d240 pg_info(1 pgs e194496:22.29a) v4 2014-08-13 18:06:08.341384 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.341385 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 370 notify: (query_epoch:194496, epoch_sent:194496, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.341407 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring 2014-08-13 18:06:08.341427 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341415 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.4 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341441 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.29a) v4 from osd.345 2014-08-13 18:06:08.341447 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d240 2014-08-13 18:06:08.341432 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 370 notify: (query_epoch:194496, epoch_sent:194496, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.341411 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] flushed 2014-08-13 18:06:08.341441 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341454 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring 2014-08-13 18:06:08.341477 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341468 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341493 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341496 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341482 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 311 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.341490 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341503 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 311, already have info from that osd, ignoring 2014-08-13 18:06:08.341517 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341520 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341521 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341523 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a440c0 pg_info(1 pgs e194496:22.12a) v4 2014-08-13 18:06:08.341511 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341528 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 312 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:08.341533 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341547 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 312, already have info from that osd, ignoring 2014-08-13 18:06:08.341566 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341569 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.12a) v4 from osd.323 2014-08-13 18:06:08.341574 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a440c0 2014-08-13 18:06:08.341572 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 350 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.341597 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.341633 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341637 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341629 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:08.341652 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341658 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341640 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting newest update on osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) shard 344 (up) backfill 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) osd.296 (up) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) shard 320 (stray) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:08.341661 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341664 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4ec0 pg_notify(22.481(49) epoch 194496) v5 2014-08-13 18:06:08.341659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 381 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:08.341663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] actingbackfill is 4,296,320,344 2014-08-13 18:06:08.341681 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 381, already have info from that osd, ignoring 2014-08-13 18:06:08.341698 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] choose_acting want [4,296,320] (== acting) backfill_targets 344 2014-08-13 18:06:08.341705 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.341708 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: leaving GetLog 2014-08-13 18:06:08.341738 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.287 2014-08-13 18:06:08.341742 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4ec0 2014-08-13 18:06:08.341729 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops 2014-08-13 18:06:08.341749 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.341733 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000350 0 0.000000 2014-08-13 18:06:08.341757 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.341760 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.341780 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.296 has no missing, identical log 2014-08-13 18:06:08.341800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.320 has no missing, identical log 2014-08-13 18:06:08.341806 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 59 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.341829 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341833 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341819 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.344 will fully backfill; can infer empty missing set 2014-08-13 18:06:08.341831 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.341840 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: still need up_thru update before going active 2014-08-13 18:06:08.341862 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000101 0 0.000000 2014-08-13 18:06:08.341862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged 2014-08-13 18:06:08.341885 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.341905 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.341942 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341947 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341887 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.090341 8 0.000658 2014-08-13 18:06:08.341950 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee5340 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.341959 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.341964 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.341967 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.341969 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.341959 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.341978 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 31 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.341987 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342011 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342005 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.31 activated and committed 2014-08-13 18:06:08.342015 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342010 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342031 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d080 pg_info(1 pgs e194496:22.3ad) v4 2014-08-13 18:06:08.342033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.342024 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.342052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342069 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.342074 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342094 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 2014-08-13 18:06:08.342111 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3ad) v4 from osd.292 2014-08-13 18:06:08.342116 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d080 2014-08-13 18:06:08.342131 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] actingbackfill is 4,55,59,351 2014-08-13 18:06:08.342138 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] flushed 2014-08-13 18:06:08.342173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] choose_acting want [4,351,55] (== acting) backfill_targets 59 2014-08-13 18:06:08.342186 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342192 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342215 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342218 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: leaving GetLog 2014-08-13 18:06:08.342232 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cd00 pg_info(1 pgs e194496:22.537) v4 2014-08-13 18:06:08.342241 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000282 0 0.000000 2014-08-13 18:06:08.342264 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:08.342264 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.342284 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.55 has no missing, identical log 2014-08-13 18:06:08.342302 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.537) v4 from osd.51 2014-08-13 18:06:08.342308 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cd00 2014-08-13 18:06:08.342304 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.351 has no missing, identical log 2014-08-13 18:06:08.342300 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2875 2014-08-13 18:06:08.342337 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: still need up_thru update before going active 2014-08-13 18:06:08.342345 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000082 0 0.000000 2014-08-13 18:06:08.342368 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342372 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342367 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.342386 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.342398 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342381 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 353 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.342401 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342404 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a140 MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 2014-08-13 18:06:08.342405 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.353 activated and committed 2014-08-13 18:06:08.342428 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 291 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.342460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.342483 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.342501 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342507 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.342530 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a140 2014-08-13 18:06:08.342519 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged 2014-08-13 18:06:08.342526 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.342544 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342544 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.127113 7 0.001501 2014-08-13 18:06:08.342559 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 362 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.342539 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] flushed 2014-08-13 18:06:08.342569 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342572 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.362 activated and committed 2014-08-13 18:06:08.342574 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.342592 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342591 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342599 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342601 7f5c835ef700 20 osd.4 194496 _dispatch 0x5355940 pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 2014-08-13 18:06:08.342602 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.342602 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342626 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops 2014-08-13 18:06:08.342626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342648 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342651 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342658 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 20 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.342674 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342681 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.342706 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342695 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.342712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 346 query_epoch 194496 query: query(log 194422'15014 ec=192688 les/c 194493/194493 194495/194496/192955) 2014-08-13 18:06:08.342715 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.342732 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 2014-08-13 18:06:08.342755 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.289 now has 194496 2014-08-13 18:06:08.342764 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 2014-08-13 18:06:08.342769 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.342790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,291,350,367,378 2014-08-13 18:06:08.342794 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194422'15014 2014-08-13 18:06:08.342814 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) 2014-08-13 18:06:08.342834 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.7 2014-08-13 18:06:08.342839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5355940 2014-08-13 18:06:08.342842 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 2014-08-13 18:06:08.342848 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4f21600 already has epoch 194496 2014-08-13 18:06:08.342880 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.342895 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342898 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,291,350] (== acting) backfill_targets 367,378 2014-08-13 18:06:08.342957 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342962 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.342945 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog 2014-08-13 18:06:08.342964 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee4800 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.342973 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.342988 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.342974 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000401 0 0.000000 2014-08-13 18:06:08.342992 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.342994 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343000 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.343018 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.291 has no missing, identical log 2014-08-13 18:06:08.343020 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343039 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343042 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea300 pg_notify(24.1bb epoch 194496) v5 2014-08-13 18:06:08.343038 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.350 has no missing, identical log 2014-08-13 18:06:08.343057 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active 2014-08-13 18:06:08.343061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 304 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193250) 2014-08-13 18:06:08.343092 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.348 2014-08-13 18:06:08.343080 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 2014-08-13 18:06:08.343100 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea300 2014-08-13 18:06:08.343093 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log request from 304 2014-08-13 18:06:08.343103 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.343125 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.343148 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info+missing+log since 194487'0 2014-08-13 18:06:08.343156 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:08.343166 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending log((194422'16034,194422'16034], crt=194422'16034) missing(61) 2014-08-13 18:06:08.343196 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.343190 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343183 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.50 2014-08-13 18:06:08.343232 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343265 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.50 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours 2014-08-13 18:06:08.343276 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.343268 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343302 7f5c835ef700 20 osd.4 194496 _dispatch 0x5425460 pg_query(24.3b7 epoch 194496) v3 2014-08-13 18:06:08.343340 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.356 epoch 194496 2014-08-13 18:06:08.343344 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5425460 2014-08-13 18:06:08.343338 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed 2014-08-13 18:06:08.343358 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192969) from replica 356 2014-08-13 18:06:08.343430 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343433 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343451 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 357 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.343485 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343466 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 362 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.343485 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.357 activated and committed 2014-08-13 18:06:08.343497 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.343504 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.343490 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343523 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353d40 pg_info(1 pgs e194496:22.481) v4 2014-08-13 18:06:08.343520 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.343544 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.343531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 18:06:08.343554 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343584 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.481) v4 from osd.287 2014-08-13 18:06:08.343589 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353d40 2014-08-13 18:06:08.343591 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4afcd00 don't know epoch, doing nothing 2014-08-13 18:06:08.343591 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 373 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.343616 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.373 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.343621 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2662 2014-08-13 18:06:08.343639 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343649 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.343646 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.343667 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops 2014-08-13 18:06:08.343685 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 312 info: 22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.343733 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.312 activated and committed 2014-08-13 18:06:08.343748 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343753 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.343765 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.343786 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.343724 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 26 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.343814 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c5aec0 don't know epoch, doing nothing 2014-08-13 18:06:08.343816 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343821 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343823 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bd40 pg_info(1 pgs e194496:22.48e) v4 2014-08-13 18:06:08.343802 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours 2014-08-13 18:06:08.343836 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343835 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2864 2014-08-13 18:06:08.343841 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 30 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.343862 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.343862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.30 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours 2014-08-13 18:06:08.343881 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.48e) v4 from osd.341 2014-08-13 18:06:08.343885 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.343886 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bd40 2014-08-13 18:06:08.343889 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 364 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192966) 2014-08-13 18:06:08.343925 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343912 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] log request from 364 2014-08-13 18:06:08.343929 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.343912 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 350 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.343941 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.350 2014-08-13 18:06:08.343954 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info+missing+log since 194487'0 2014-08-13 18:06:08.343973 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending log((194422'17074,194422'17074], crt=194422'17074) missing(48) 2014-08-13 18:06:08.343959 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.343992 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.343996 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.343995 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:08.344002 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.344005 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b480 pg_info(1 pgs e194496:22.3e4) v4 2014-08-13 18:06:08.344005 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 291 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.344026 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.344026 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.291 2014-08-13 18:06:08.344061 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3e4) v4 from osd.318 2014-08-13 18:06:08.344076 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b480 2014-08-13 18:06:08.344048 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.344083 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.344122 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.344126 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.344179 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.344189 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 50 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.344215 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.344219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.344221 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421a40 pg_query(24.e2 epoch 194496) v3 2014-08-13 18:06:08.344219 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.50 activated and committed 2014-08-13 18:06:08.344221 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.344239 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.344255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.344255 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.303 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.344278 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:08.344281 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.344285 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.346 epoch 194496 2014-08-13 18:06:08.344288 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421a40 2014-08-13 18:06:08.344325 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 2014-08-13 18:06:08.344301 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192955) from replica 346 2014-08-13 18:06:08.344327 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 25 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.344354 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2710 2014-08-13 18:06:08.344383 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.344373 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.344415 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.344420 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.344410 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged 2014-08-13 18:06:08.344433 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.344436 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.128992 9 0.050033 2014-08-13 18:06:08.344461 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.344463 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.344479 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.344486 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 28 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.344491 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.344505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.28 activated and committed 2014-08-13 18:06:08.344532 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.344515 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.344538 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 304 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.344537 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.344559 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.304 activated and committed 2014-08-13 18:06:08.344560 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.344576 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.344583 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.344592 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.344615 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:08.344603 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.344643 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.344664 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2909 2014-08-13 18:06:08.344663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.344690 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.344686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] actingbackfill is 4,20,25,303,362 2014-08-13 18:06:08.344705 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] choose_acting want [4,20,303] (== acting) backfill_targets 25,362 2014-08-13 18:06:08.344728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: leaving GetLog 2014-08-13 18:06:08.344752 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000290 0 0.000000 2014-08-13 18:06:08.344764 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 330 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193227) 2014-08-13 18:06:08.344776 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.344790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] log request from 330 2014-08-13 18:06:08.344795 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.20 has no missing, identical log 2014-08-13 18:06:08.344816 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.303 has no missing, identical log 2014-08-13 18:06:08.344834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: still need up_thru update before going active 2014-08-13 18:06:08.344841 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info+missing+log since 194487'0 2014-08-13 18:06:08.344856 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 2014-08-13 18:06:08.344861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending log((194422'15537,194422'15537], crt=194422'15537) missing(64) 2014-08-13 18:06:08.344883 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.344878 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.344899 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.344916 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345118 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345143 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops 2014-08-13 18:06:08.345162 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 374 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193232) 2014-08-13 18:06:08.345171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345186 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] log request from 374 2014-08-13 18:06:08.345199 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.345217 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345224 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345243 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.345260 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345244 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info+missing+log since 194487'0 2014-08-13 18:06:08.345263 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending log((194422'16449,194422'16449], crt=194422'16449) missing(52) 2014-08-13 18:06:08.345266 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345285 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:08.345287 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.345303 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345320 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.345309 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192935) 2014-08-13 18:06:08.345308 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] flushed 2014-08-13 18:06:08.345366 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 44 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.345554 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.44 activated and committed 2014-08-13 18:06:08.345553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345572 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.345580 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.345600 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345587 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.345612 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x449a3c0 don't know epoch, doing nothing 2014-08-13 18:06:08.345607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 45 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193186) 2014-08-13 18:06:08.345626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log request from 45 2014-08-13 18:06:08.345649 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:08.345667 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194487'0 2014-08-13 18:06:08.345674 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2687 2014-08-13 18:06:08.345701 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.345685 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending log((194422'13576,194422'13576], crt=194422'13576) missing(50) 2014-08-13 18:06:08.345779 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:08.345775 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.345809 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.345801 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.345819 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.345996 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 315 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.346021 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.346040 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346046 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.346029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346063 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged 2014-08-13 18:06:08.346086 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.104823 8 0.001041 2014-08-13 18:06:08.346094 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) 2014-08-13 18:06:08.346109 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.346147 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346133 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346155 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194489 194495/194495/192972) 2014-08-13 18:06:08.346183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.346204 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346233 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346225 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346245 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.346258 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194489 194495/194495/192936) 2014-08-13 18:06:08.346307 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.346323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,17,28,315,330 2014-08-13 18:06:08.346324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) 2014-08-13 18:06:08.346343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,17,315] (== acting) backfill_targets 28,330 2014-08-13 18:06:08.346371 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346365 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog 2014-08-13 18:06:08.346389 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000279 0 0.000000 2014-08-13 18:06:08.346386 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) 2014-08-13 18:06:08.346415 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.346441 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346434 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.17 has no missing, identical log 2014-08-13 18:06:08.346453 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.315 has no missing, identical log 2014-08-13 18:06:08.346458 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192971) 2014-08-13 18:06:08.346472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active 2014-08-13 18:06:08.346506 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346493 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000 2014-08-13 18:06:08.346515 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.346535 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346522 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) 2014-08-13 18:06:08.346570 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346562 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346588 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops 2014-08-13 18:06:08.346587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346608 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346610 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.346627 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346615 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 315 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.346634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346637 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.315 2014-08-13 18:06:08.346654 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.346672 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346657 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.346693 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346699 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops 2014-08-13 18:06:08.346716 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346702 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346722 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192968) 2014-08-13 18:06:08.346734 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops 2014-08-13 18:06:08.346754 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346773 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346761 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192966) 2014-08-13 18:06:08.346790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.346814 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] requeue_ops 2014-08-13 18:06:08.346834 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.346842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 61 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.346856 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.61 activated and committed 2014-08-13 18:06:08.346879 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.346881 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194496 n=44 ec=192688 les/c 194496/194463 194495/194495/194468) 2014-08-13 18:06:08.346894 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.346917 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:08.346937 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.346957 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4fa4fc0 don't know epoch, doing nothing 2014-08-13 18:06:08.346983 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3024 2014-08-13 18:06:08.347008 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.347002 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.347027 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.347045 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.347106 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) 2014-08-13 18:06:08.347114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) 2014-08-13 18:06:08.353507 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.347423 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed 2014-08-13 18:06:08.353526 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.353552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.353557 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353578 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.353596 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged 2014-08-13 18:06:08.353604 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.353611 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last maybe_went_rw interval was interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) 2014-08-13 18:06:08.353625 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.353641 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353637 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.103537 6 0.000793 2014-08-13 18:06:08.353648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.353660 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.353670 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.353688 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353686 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.4 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192949) 2014-08-13 18:06:08.353707 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353743 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353729 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353781 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.353802 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting newest update on osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) up[0] needs backfill, osd.4 selected as primary instead calc_acting primary is osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) shard 26 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) shard 30 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) shard 340 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:08.353805 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.353824 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353822 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] actingbackfill is 4,26,30,340 2014-08-13 18:06:08.353831 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.353840 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] choose_acting want [4] (== acting) backfill_targets 26,30,340 2014-08-13 18:06:08.353852 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.353870 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: leaving GetLog 2014-08-13 18:06:08.353876 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) 2014-08-13 18:06:08.353885 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000226 0 0.000000 2014-08-13 18:06:08.353908 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.353922 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.353928 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.26 will fully backfill; can infer empty missing set 2014-08-13 18:06:08.353949 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.353947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.30 will fully backfill; can infer empty missing set 2014-08-13 18:06:08.353964 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.340 will fully backfill; can infer empty missing set 2014-08-13 18:06:08.353980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: still need up_thru update before going active 2014-08-13 18:06:08.353988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) 2014-08-13 18:06:08.354000 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000092 0 0.000000 2014-08-13 18:06:08.354021 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.354039 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354040 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354075 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354087 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) 2014-08-13 18:06:08.354100 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.354121 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354140 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354129 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354149 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.354165 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) 2014-08-13 18:06:08.354185 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery 2014-08-13 18:06:08.354231 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354215 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.044314 3 0.000321 2014-08-13 18:06:08.354241 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:08.354255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354273 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354280 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.354298 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354291 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192976) 2014-08-13 18:06:08.354305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194489 194495/194495/192976) 2014-08-13 18:06:08.354295 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] flushed 2014-08-13 18:06:08.354344 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354354 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354389 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.354378 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery 2014-08-13 18:06:08.354399 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.043969 3 0.000160 2014-08-13 18:06:08.354418 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:08.354437 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354430 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) 2014-08-13 18:06:08.354444 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192946) 2014-08-13 18:06:08.354479 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354492 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354500 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(log 194422'15992 ec=192688 les/c 194493/194489 194495/194496/192967) 2014-08-13 18:06:08.354514 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) 2014-08-13 18:06:08.354521 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 2014-08-13 18:06:08.354564 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354566 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194422'15992 2014-08-13 18:06:08.354581 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) 2014-08-13 18:06:08.354585 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) 2014-08-13 18:06:08.354605 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.354612 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.354617 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.354619 7f5c835ef700 20 osd.4 194496 _dispatch 0x53540c0 pg_info(1 pgs e194496:22.653) v4 2014-08-13 18:06:08.354627 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354644 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354674 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.653) v4 from osd.371 2014-08-13 18:06:08.354679 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53540c0 2014-08-13 18:06:08.354668 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354692 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.354711 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/192943) 2014-08-13 18:06:08.354865 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354892 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.354897 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.354884 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) 2014-08-13 18:06:08.354827 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354915 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-13 18:06:08.354932 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354937 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.354950 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 286 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.354946 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.354974 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.286 activated and committed 2014-08-13 18:06:08.354993 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.354993 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355012 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.355000 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.355020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.355026 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.355040 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355057 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.355046 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355065 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.45 has 27 missing 2014-08-13 18:06:08.355082 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.339 has 27 missing 2014-08-13 18:06:08.355064 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.355097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.355104 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355122 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.355129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 343 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193262) 2014-08-13 18:06:08.355184 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log request from 343 2014-08-13 18:06:08.355228 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info+missing+log since 194487'0 2014-08-13 18:06:08.355252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending log((194422'14476,194422'14476], crt=194422'14476) missing(41) 2014-08-13 18:06:08.355277 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2938 2014-08-13 18:06:08.355298 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.355296 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.355317 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.355333 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.355340 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.355360 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355377 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.355384 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.355402 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355417 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.355423 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.355441 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355457 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.355463 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.355482 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.355496 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355516 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.355520 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.355511 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.62 has 29 missing 2014-08-13 18:06:08.355524 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6900 MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 2014-08-13 18:06:08.355527 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.65 has 29 missing 2014-08-13 18:06:08.355542 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.355567 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6900 2014-08-13 18:06:08.355608 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.355611 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.355711 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2738 2014-08-13 18:06:08.355728 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.355756 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.355750 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.355766 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.355773 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.355795 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.355771 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 296 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.355799 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.355802 7f5c835ef700 20 osd.4 194496 _dispatch 0x21af260 pg_query(24.cd epoch 194496) v3 2014-08-13 18:06:08.355793 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.355810 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.355825 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.5 has 35 missing 2014-08-13 18:06:08.355847 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.6 epoch 194496 2014-08-13 18:06:08.355851 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21af260 2014-08-13 18:06:08.355804 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.296 2014-08-13 18:06:08.355844 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.47 has 35 missing 2014-08-13 18:06:08.355861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.355865 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192967) from replica 6 2014-08-13 18:06:08.355860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.296 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours 2014-08-13 18:06:08.355901 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.355924 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.355928 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.355910 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 15 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193264) 2014-08-13 18:06:08.355959 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] log request from 15 2014-08-13 18:06:08.356026 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194487'0 2014-08-13 18:06:08.356059 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending log((194422'11806,194422'11806], crt=194422'11806) missing(56) 2014-08-13 18:06:08.356082 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4854620 don't know epoch, doing nothing 2014-08-13 18:06:08.356067 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3011 2014-08-13 18:06:08.356089 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.356112 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356104 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.356120 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.356297 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.356293 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.356321 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.356330 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops 2014-08-13 18:06:08.356337 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.356354 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356352 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] needs_recovery osd.301 has 38 missing 2014-08-13 18:06:08.356363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.356368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.356384 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.356402 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356409 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 31 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194492/194372 194495/194496/192978) 2014-08-13 18:06:08.356433 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log request from 31 2014-08-13 18:06:08.356471 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info+missing+log since 194487'0 2014-08-13 18:06:08.356487 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending log((194422'18567,194422'18567], crt=194422'18567) missing(46) 2014-08-13 18:06:08.356506 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 2014-08-13 18:06:08.356537 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356587 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2835 2014-08-13 18:06:08.356603 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.356617 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.356633 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.356693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 332 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.356700 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 309 info: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) 2014-08-13 18:06:08.356723 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.309 activated and committed 2014-08-13 18:06:08.356721 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.356743 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.356746 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356754 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.356750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.356770 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops 2014-08-13 18:06:08.356786 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.356794 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 285 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192975) 2014-08-13 18:06:08.356801 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 344 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.356813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log request from 285 2014-08-13 18:06:08.356826 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.344 2014-08-13 18:06:08.356845 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.344 info 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours 2014-08-13 18:06:08.356882 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.356867 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194487'0 2014-08-13 18:06:08.356885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending log((194422'17642,194422'17642], crt=194422'17642) missing(50) 2014-08-13 18:06:08.356905 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:08.356891 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194489 194495/194495/192949) 2014-08-13 18:06:08.356934 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.356948 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357110 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 289 info: 22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.357134 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed 2014-08-13 18:06:08.357152 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.357137 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 338 info: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) 2014-08-13 18:06:08.357159 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.357172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.338 activated and committed 2014-08-13 18:06:08.357193 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357177 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.357200 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.357200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 331 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.357218 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.58 has 23 missing 2014-08-13 18:06:08.357224 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.357235 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.343 has 23 missing 2014-08-13 18:06:08.357251 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.357257 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:08.357279 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357286 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 320 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.357306 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.320 2014-08-13 18:06:08.357324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.320 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours 2014-08-13 18:06:08.357359 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357366 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/192978) 2014-08-13 18:06:08.357391 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2686 2014-08-13 18:06:08.357407 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.357422 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.357437 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.357438 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357446 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 343 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.357465 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed 2014-08-13 18:06:08.357470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192958) 2014-08-13 18:06:08.357481 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.357495 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.357518 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:08.357526 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357553 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.357579 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2656 2014-08-13 18:06:08.357603 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.357625 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194496/194463 194495/194495/193205) 2014-08-13 18:06:08.357705 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357693 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194489 194495/194495/192978) 2014-08-13 18:06:08.357776 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.357738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) 2014-08-13 18:06:08.357788 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.376 activated and committed 2014-08-13 18:06:08.357806 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.357801 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192972) 2014-08-13 18:06:08.357822 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.357850 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b1b5a0 don't know epoch, doing nothing 2014-08-13 18:06:08.357854 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.357910 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing 2014-08-13 18:06:08.357936 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2598 2014-08-13 18:06:08.357964 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.357994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194489 194495/194495/192971) 2014-08-13 18:06:08.358050 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358045 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 379 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.358074 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.358098 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358069 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) 2014-08-13 18:06:08.358106 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 307 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:08.358136 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358128 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.307 2014-08-13 18:06:08.358155 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) 2014-08-13 18:06:08.358147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.307 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours 2014-08-13 18:06:08.358192 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358211 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192981) 2014-08-13 18:06:08.358252 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358236 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 287 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) 2014-08-13 18:06:08.358261 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 287, already have info from that osd, ignoring 2014-08-13 18:06:08.358280 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358273 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 337 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:08.358287 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) 2014-08-13 18:06:08.358297 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours 2014-08-13 18:06:08.358323 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358336 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358329 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 347 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.358350 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.347 activated and committed 2014-08-13 18:06:08.358368 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358356 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.358379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.358374 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) 2014-08-13 18:06:08.358395 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.358427 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.31 has 29 missing 2014-08-13 18:06:08.358433 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.374 has 29 missing 2014-08-13 18:06:08.358448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.358446 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) 2014-08-13 18:06:08.358494 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358510 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.358531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.358546 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.358561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.344 has 23 missing 2014-08-13 18:06:08.358577 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.373 has 23 missing 2014-08-13 18:06:08.358592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.358624 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2876 2014-08-13 18:06:08.358641 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.358657 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.358673 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358680 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) 2014-08-13 18:06:08.358725 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358736 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2618 2014-08-13 18:06:08.358744 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery 2014-08-13 18:06:08.358753 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.358769 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.358766 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.025954 3 0.000244 2014-08-13 18:06:08.358786 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358788 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:08.358793 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) 2014-08-13 18:06:08.358809 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358817 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.358839 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358841 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] requeue_ops 2014-08-13 18:06:08.358861 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358869 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 7 notify: (query_epoch:194496, epoch_sent:194496, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.358866 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestBackfillPrio: priority 1 2014-08-13 18:06:08.358890 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring 2014-08-13 18:06:08.358907 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358902 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] exit Started/ReplicaActive/RepNotRecovering 0.030137 3 0.000235 2014-08-13 18:06:08.358914 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.358928 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] enter Started/ReplicaActive/RepWaitBackfillReserved 2014-08-13 18:06:08.358950 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358937 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops 2014-08-13 18:06:08.358957 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.358957 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.358967 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192969) 2014-08-13 18:06:08.358982 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] requeue_ops 2014-08-13 18:06:08.359004 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.358989 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 2014-08-13 18:06:08.359013 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.359033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194487'0 2014-08-13 18:06:08.359052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) 2014-08-13 18:06:08.359074 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:08.359060 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) 2014-08-13 18:06:08.359108 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.359111 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.359281 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.359306 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.359322 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.359314 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.359337 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.356 has 35 missing 2014-08-13 18:06:08.359355 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.357 has 35 missing 2014-08-13 18:06:08.359357 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] requeue_ops 2014-08-13 18:06:08.359383 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.359371 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.359392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 7 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.359414 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring 2014-08-13 18:06:08.359433 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.359447 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 348 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-13 18:06:08.359470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.359502 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged 2014-08-13 18:06:08.359527 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.050544 9 0.000838 2014-08-13 18:06:08.359550 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog 2014-08-13 18:06:08.359580 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2663 2014-08-13 18:06:08.359576 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.359597 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.359599 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.359614 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.359631 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.359619 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.359638 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 287 info: 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) 2014-08-13 18:06:08.359641 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.359659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.287 activated and committed 2014-08-13 18:06:08.359663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:08.359677 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.359682 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.359693 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.359718 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x49a7bc0 don't know epoch, doing nothing 2014-08-13 18:06:08.359751 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing 2014-08-13 18:06:08.359743 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:08.359765 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] actingbackfill is 4,331,332,337,348 2014-08-13 18:06:08.359775 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2686 2014-08-13 18:06:08.359784 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] choose_acting want [4,332,337] (== acting) backfill_targets 331,348 2014-08-13 18:06:08.359803 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.359808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: leaving GetLog 2014-08-13 18:06:08.359831 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000281 0 0.000000 2014-08-13 18:06:08.359854 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing 2014-08-13 18:06:08.359861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.359873 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.332 has no missing, identical log 2014-08-13 18:06:08.359886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.359896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.337 has no missing, identical log 2014-08-13 18:06:08.359904 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.359917 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: still need up_thru update before going active 2014-08-13 18:06:08.359920 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] needs_recovery osd.285 has 29 missing 2014-08-13 18:06:08.359937 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.359939 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000086 0 0.000000 2014-08-13 18:06:08.359963 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru 2014-08-13 18:06:08.359982 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.360013 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.360039 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] requeue_ops 2014-08-13 18:06:08.360059 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.360068 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.360088 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.360103 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.360110 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2865 2014-08-13 18:06:08.360118 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.44 has 24 missing 2014-08-13 18:06:08.360128 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.360136 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.362 has 24 missing 2014-08-13 18:06:08.360145 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.360163 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.360154 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.360136 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] flushed 2014-08-13 18:06:08.360170 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 341 info: 22.48e( v 194215'543 lc 0'0 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.360191 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.341 activated and committed 2014-08-13 18:06:08.360215 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.360230 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.360251 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 2014-08-13 18:06:08.360284 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab4620 don't know epoch, doing nothing 2014-08-13 18:06:08.360307 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2692 2014-08-13 18:06:08.360333 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.360318 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2688 2014-08-13 18:06:08.360338 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.360353 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.360369 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.360405 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 318 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.360414 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.360429 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.318 activated and committed 2014-08-13 18:06:08.360439 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.360448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.360456 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.360464 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.360472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.61 has 28 missing 2014-08-13 18:06:08.360486 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4494360 don't know epoch, doing nothing 2014-08-13 18:06:08.360489 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.349 has 28 missing 2014-08-13 18:06:08.360514 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3988dc0 don't know epoch, doing nothing 2014-08-13 18:06:08.360504 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.360536 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2709 2014-08-13 18:06:08.360559 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.360623 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 346 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192955) 2014-08-13 18:06:08.360646 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 2014-08-13 18:06:08.360670 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3025 2014-08-13 18:06:08.360682 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194487'0 2014-08-13 18:06:08.360687 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.360702 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.360701 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) 2014-08-13 18:06:08.360720 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.360724 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f21600 already has epoch 194496 2014-08-13 18:06:08.360728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.360760 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.360750 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops 2014-08-13 18:06:08.360769 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.360936 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.360946 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RemoteRecoveryReserved 2014-08-13 18:06:08.360960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.360980 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 2014-08-13 18:06:08.360976 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.360992 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.361007 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.361012 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.006772 1 0.000066 2014-08-13 18:06:08.361022 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.361038 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepRecovering 2014-08-13 18:06:08.361057 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361054 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2711 2014-08-13 18:06:08.361070 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.361085 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.361101 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.361109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.361111 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] requeue_ops 2014-08-13 18:06:08.361131 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361127 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.361142 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.361139 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) 2014-08-13 18:06:08.361157 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.361172 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.361187 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.361210 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361215 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2910 2014-08-13 18:06:08.361231 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.361236 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361245 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.361261 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361268 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.361261 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.128658 9 0.003468 2014-08-13 18:06:08.361283 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361311 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361298 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361319 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.107412 9 0.003964 2014-08-13 18:06:08.361324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361342 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361345 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.152338 9 0.048258 2014-08-13 18:06:08.361363 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361365 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361384 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361370 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361389 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.081297 9 0.012971 2014-08-13 18:06:08.361390 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery 2014-08-13 18:06:08.361409 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361410 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.036256 3 0.000171 2014-08-13 18:06:08.361428 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361431 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:08.361435 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361451 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361453 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094119 9 0.013031 2014-08-13 18:06:08.361458 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361472 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361491 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361478 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.121539 9 0.003900 2014-08-13 18:06:08.361498 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.361504 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361524 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361517 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.361532 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.361530 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192967) 2014-08-13 18:06:08.361549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] needs_recovery osd.309 has 32 missing 2014-08-13 18:06:08.361552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 2014-08-13 18:06:08.361565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.361592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194487'0 2014-08-13 18:06:08.361609 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) 2014-08-13 18:06:08.361628 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 2014-08-13 18:06:08.361657 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361751 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2687 2014-08-13 18:06:08.361768 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.361782 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.361797 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361815 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361812 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361838 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.139790 9 0.006333 2014-08-13 18:06:08.361841 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.197036 9 0.005297 2014-08-13 18:06:08.361863 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361858 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361883 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361886 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.361890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.361893 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.361909 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.202336 9 0.003207 2014-08-13 18:06:08.361912 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.361927 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.361928 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.361948 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.361946 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] needs_recovery osd.341 has 40 missing 2014-08-13 18:06:08.361955 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.361962 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.361976 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.361992 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.362007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.362021 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.362036 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.362059 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2657 2014-08-13 18:06:08.362075 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.362090 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.362105 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362111 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.362129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.362143 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.362158 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] needs_recovery osd.376 has 28 missing 2014-08-13 18:06:08.362173 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.362188 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2693 2014-08-13 18:06:08.362213 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.362227 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.362243 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.362249 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.362267 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.362280 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.362294 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.313 has 32 missing 2014-08-13 18:06:08.362309 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.318 has 32 missing 2014-08-13 18:06:08.362323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.362348 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2599 2014-08-13 18:06:08.362364 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.362378 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.362394 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362400 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.362430 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt 2014-08-13 18:06:08.362453 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] requeue_ops 2014-08-13 18:06:08.362482 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362491 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.362501 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2710 2014-08-13 18:06:08.362510 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.069305 9 0.011024 2014-08-13 18:06:08.362519 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.362531 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.362534 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.362551 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.362551 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.362573 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.362585 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.097564 9 0.000853 2014-08-13 18:06:08.362595 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.142278 9 0.004108 2014-08-13 18:06:08.362607 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.362614 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.362633 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362631 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.362640 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 LocalRecoveryReserved 2014-08-13 18:06:08.362655 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000047 0 0.000000 2014-08-13 18:06:08.362660 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.001378 1 0.000051 2014-08-13 18:06:08.362675 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.362682 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved 2014-08-13 18:06:08.362692 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.362708 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.362728 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.362728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) 2014-08-13 18:06:08.362749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194488-194494/2 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) 2014-08-13 18:06:08.362767 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194491-194494/1 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) 2014-08-13 18:06:08.362785 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.362807 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:08.362799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.362822 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.052692 9 0.000759 2014-08-13 18:06:08.362838 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 2014-08-13 18:06:08.362842 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.362861 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.362860 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2712 2014-08-13 18:06:08.362882 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000041 0 0.000000 2014-08-13 18:06:08.362902 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.362924 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.362918 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.362932 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.362951 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) 2014-08-13 18:06:08.362971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194488-194494/2 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) 2014-08-13 18:06:08.362989 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194491-194494/1 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) 2014-08-13 18:06:08.363007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.363031 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:08.363026 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.363060 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing 2014-08-13 18:06:08.363051 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.108392 9 0.000793 2014-08-13 18:06:08.363072 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.363081 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2658 2014-08-13 18:06:08.363092 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.363111 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000039 0 0.000000 2014-08-13 18:06:08.363145 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.363132 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.363149 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.363163 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.363151 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.363189 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] purge_strays 32 2014-08-13 18:06:08.363181 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) 2014-08-13 18:06:08.363207 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.32 2014-08-13 18:06:08.363209 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194488-194494/2 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) 2014-08-13 18:06:08.363228 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194491-194494/1 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) 2014-08-13 18:06:08.363242 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4851a20 don't know epoch, doing nothing 2014-08-13 18:06:08.363246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.363265 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.363287 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,32,50,289 -> 4,50,289 2014-08-13 18:06:08.363292 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.038020 13 0.013009 2014-08-13 18:06:08.363308 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.363316 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2713 2014-08-13 18:06:08.363313 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.363341 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.363347 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.363410 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:08.363452 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 2014-08-13 18:06:08.363475 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2911 2014-08-13 18:06:08.363538 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.363591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.363596 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.363616 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.067189 12 0.003490 2014-08-13 18:06:08.363625 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] purge_strays 10 2014-08-13 18:06:08.363639 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.363642 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.10 2014-08-13 18:06:08.363659 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.363663 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:08.363703 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,10,286,343 -> 4,286,343 2014-08-13 18:06:08.363705 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.363724 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.363727 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.063671 9 0.003884 2014-08-13 18:06:08.363730 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2659 2014-08-13 18:06:08.363747 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.363766 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.363772 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.363789 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.136725 9 0.003701 2014-08-13 18:06:08.363807 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.363824 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.365279 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.365324 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 2014-08-13 18:06:08.365342 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 2014-08-13 18:06:08.365362 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing 2014-08-13 18:06:08.365415 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,28,304 -> 4,28,304 2014-08-13 18:06:08.365441 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.365448 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2912 2014-08-13 18:06:08.368911 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.368916 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.368918 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351dc0 pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 2014-08-13 18:06:08.368967 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.340 2014-08-13 18:06:08.368971 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351dc0 2014-08-13 18:06:08.369059 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.369064 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.369034 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) 2014-08-13 18:06:08.369071 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 340, already have info from that osd, ignoring 2014-08-13 18:06:08.369056 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:08.369092 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.369093 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.340 2014-08-13 18:06:08.369112 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.340 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours 2014-08-13 18:06:08.369151 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.369160 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 2014-08-13 18:06:08.374698 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.374703 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.374705 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee0000 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.374717 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.374722 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.374725 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.374727 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.374753 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.374755 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.374757 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a6a140 pg_info(1 pgs e194496:22.707) v4 2014-08-13 18:06:08.374800 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.707) v4 from osd.21 2014-08-13 18:06:08.374805 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a6a140 2014-08-13 18:06:08.374860 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.374863 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.374862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 21 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.374897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.21 activated and committed 2014-08-13 18:06:08.374931 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.374947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.374975 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3a689a0 don't know epoch, doing nothing 2014-08-13 18:06:08.375017 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4fa1fa0 don't know epoch, doing nothing 2014-08-13 18:06:08.375046 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2573 2014-08-13 18:06:08.375097 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.375204 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.375267 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.375283 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.375304 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.21 has 19 missing 2014-08-13 18:06:08.375333 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.310 has 19 missing 2014-08-13 18:06:08.375359 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.375534 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2574 2014-08-13 18:06:08.375554 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.375569 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.375606 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.375645 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.375668 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.142309 9 0.002936 2014-08-13 18:06:08.375690 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.375716 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.403233 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.403238 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.403241 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df2a00 pg_info(1 pgs e194496:2.2a) v4 2014-08-13 18:06:08.403285 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:2.2a) v4 from osd.340 2014-08-13 18:06:08.403290 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df2a00 2014-08-13 18:06:08.403353 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.403357 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.403383 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.403432 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 18:06:08.403451 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.403465 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.403488 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.403520 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:176857 2014-08-13 18:06:08.403549 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.403654 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.403684 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.403712 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.403727 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.403742 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.403755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.403780 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:176858 2014-08-13 18:06:08.403822 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.403837 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.403853 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.403864 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.403883 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.177187 7 0.003639 2014-08-13 18:06:08.403904 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.403922 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.403939 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000035 0 0.000000 2014-08-13 18:06:08.404006 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.404022 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.404097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.404140 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 2014-08-13 18:06:08.404143 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:08.404177 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194488-194494/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:08.404194 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194491-194494/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:08.404223 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 2014-08-13 18:06:08.404217 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.404240 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.404376 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:176859 2014-08-13 18:06:08.404480 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.404538 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.404543 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.404545 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bb80 pg_info(1 pgs e194496:22.1b8) v4 2014-08-13 18:06:08.404593 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1b8) v4 from osd.340 2014-08-13 18:06:08.404598 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bb80 2014-08-13 18:06:08.404650 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.404653 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.404652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) 2014-08-13 18:06:08.404695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 18:06:08.404679 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.404715 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.404715 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 2014-08-13 18:06:08.404731 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.404735 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 2014-08-13 18:06:08.404756 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 2014-08-13 18:06:08.404797 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.404796 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 2014-08-13 18:06:08.404840 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing 2014-08-13 18:06:08.404876 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 2014-08-13 18:06:08.404896 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.404903 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:176860 2014-08-13 18:06:08.404966 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing 2014-08-13 18:06:08.405004 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3003 2014-08-13 18:06:08.405042 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.405115 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.405141 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.405158 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.405183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.405198 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.405213 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.405248 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3004 2014-08-13 18:06:08.405265 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.405280 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.405301 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.405314 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.405333 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094194 9 0.001175 2014-08-13 18:06:08.405356 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.405374 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.405392 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000037 0 0.000000 2014-08-13 18:06:08.405411 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.405427 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.405441 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.405460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) 2014-08-13 18:06:08.405481 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194488-194494/2 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) 2014-08-13 18:06:08.405498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194491-194494/1 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) 2014-08-13 18:06:08.405516 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.405536 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.405569 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing 2014-08-13 18:06:08.405589 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:3005 2014-08-13 18:06:08.405663 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.405854 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.405876 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 2014-08-13 18:06:08.405892 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 2014-08-13 18:06:08.405911 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing 2014-08-13 18:06:08.405939 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,340,347 -> 4,340,347 2014-08-13 18:06:08.405959 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.405964 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:3006 2014-08-13 18:06:08.424816 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.424821 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.424824 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed4140 osd_map(194496..194496 src has 185139..194496) v3 2014-08-13 18:06:08.424835 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] 2014-08-13 18:06:08.424840 7f5c835ef700 10 osd.4 194496 no new maps here, dropping 2014-08-13 18:06:08.424843 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.424845 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.424869 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.424871 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.424873 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354440 pg_info(1 pgs e194496:22.775) v4 2014-08-13 18:06:08.424919 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.775) v4 from osd.56 2014-08-13 18:06:08.424924 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354440 2014-08-13 18:06:08.424980 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.424984 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.425021 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 56 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) 2014-08-13 18:06:08.425065 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.56 activated and committed 2014-08-13 18:06:08.425083 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.425098 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.425123 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b17220 don't know epoch, doing nothing 2014-08-13 18:06:08.425175 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x50e1600 don't know epoch, doing nothing 2014-08-13 18:06:08.425200 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2725 2014-08-13 18:06:08.425232 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.425381 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.425428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.425446 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.425462 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.56 has 32 missing 2014-08-13 18:06:08.425478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.353 has 32 missing 2014-08-13 18:06:08.425493 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:08.425688 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2726 2014-08-13 18:06:08.425708 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.425723 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.425740 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.425753 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery 2014-08-13 18:06:08.425772 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.132853 9 0.011511 2014-08-13 18:06:08.425794 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:08.425815 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:08.474482 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.474487 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.474490 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0700 pg_info(1 pgs e194496:1.2b) v4 2014-08-13 18:06:08.474539 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:1.2b) v4 from osd.340 2014-08-13 18:06:08.474544 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0700 2014-08-13 18:06:08.474605 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.474609 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.474633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) 2014-08-13 18:06:08.474677 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed 2014-08-13 18:06:08.474695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed 2014-08-13 18:06:08.474709 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info 2014-08-13 18:06:08.474733 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.474768 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:177961 2014-08-13 18:06:08.474799 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.474813 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated 2014-08-13 18:06:08.474833 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local 2014-08-13 18:06:08.474847 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops 2014-08-13 18:06:08.474862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.474877 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill 2014-08-13 18:06:08.474890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery 2014-08-13 18:06:08.474915 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:177962 2014-08-13 18:06:08.474931 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear 2014-08-13 18:06:08.474946 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop 2014-08-13 18:06:08.474960 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.474970 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered 2014-08-13 18:06:08.474989 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.234099 7 0.004925 2014-08-13 18:06:08.475010 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered 2014-08-13 18:06:08.475027 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered 2014-08-13 18:06:08.475045 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000034 0 0.000000 2014-08-13 18:06:08.475063 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean 2014-08-13 18:06:08.475078 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery 2014-08-13 18:06:08.475092 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state 2014-08-13 18:06:08.475109 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) 2014-08-13 18:06:08.475129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194488-194494/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) 2014-08-13 18:06:08.475145 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194491-194494/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) 2014-08-13 18:06:08.475184 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info 2014-08-13 18:06:08.475208 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:08.475234 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:177963 2014-08-13 18:06:08.475294 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:08.475583 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery 2014-08-13 18:06:08.475627 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 2014-08-13 18:06:08.475646 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 2014-08-13 18:06:08.475666 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 2014-08-13 18:06:08.475699 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 2014-08-13 18:06:08.475721 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing 2014-08-13 18:06:08.475743 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 2014-08-13 18:06:08.475769 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update 2014-08-13 18:06:08.475776 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:177964 2014-08-13 18:06:08.659890 7f5c817ec700 10 osd.4 194496 note_peer_epoch osd.15 now has 194496 2014-08-13 18:06:08.659901 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4854620 already has epoch 194496 2014-08-13 18:06:08.659963 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4854620 already has epoch 194496 2014-08-13 18:06:08.807831 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.343 now has 194496 2014-08-13 18:06:08.807844 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x48506e0 already has epoch 194496 2014-08-13 18:06:08.807882 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x48506e0 already has epoch 194496 2014-08-13 18:06:08.831842 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.371 now has 194496 2014-08-13 18:06:08.831852 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4851e40 already has epoch 194496 2014-08-13 18:06:08.831889 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4851e40 already has epoch 194496 2014-08-13 18:06:08.850521 7f5c817ec700 10 osd.4 194496 note_peer_epoch osd.292 now has 194496 2014-08-13 18:06:08.850530 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 already has epoch 194496 2014-08-13 18:06:08.850639 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 already has epoch 194496 2014-08-13 18:06:08.924532 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.924547 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.924549 7f5c835ef700 20 osd.4 194496 _dispatch 0x510ee00 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:08.924627 7f5c835ef700 10 osd.4 194496 handle_replica_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194496 2014-08-13 18:06:08.924639 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x510ee00 2014-08-13 18:06:08.924645 7f5c835ef700 20 osd.4 194496 _share_map_incoming osd.397 192.168.1.105:6808/41085 194496 2014-08-13 18:06:08.924654 7f5c835ef700 10 osd.4 194496 note_peer_epoch osd.397 now has 194496 2014-08-13 18:06:08.924663 7f5c835ef700 15 osd.4 194496 enqueue_op 0x20a50a0 prio 127 cost 0 latency 0.000416 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:08.924698 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.924702 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.924775 7f5c7c7e4700 10 osd.4 194496 dequeue_op 0x20a50a0 prio 127 cost 0 latency 0.000528 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] 2014-08-13 18:06:08.924842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_message: 0x20a50a0 2014-08-13 18:06:08.924875 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:08.924897 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] sub_op_scrub_reserve 2014-08-13 18:06:08.924929 7f5c7c7e4700 20 osd.4 194496 inc_scrubs_pending 0 -> 1 (max 1, active 0) 2014-08-13 18:06:08.924971 7f5c7c7e4700 10 osd.4 194496 dequeue_op 0x20a50a0 finish 2014-08-13 18:06:08.927442 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.927447 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.927450 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc0240 replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194496,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 2014-08-13 18:06:08.927464 7f5c835ef700 10 osd.4 194496 queueing MOSDRepScrub replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194496,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 2014-08-13 18:06:08.927477 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:08.927480 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:08.927538 7f5c7a9e1700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] replica_scrub 2014-08-13 18:06:08.927593 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map 2014-08-13 18:06:08.927611 7f5c7a9e1700 20 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] scrub_map_chunk [0//0//-1,e50d7bb9//0//-1) 2014-08-13 18:06:08.928474 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] _scan_list scanning 25 objects deeply 2014-08-13 18:06:08.990083 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map_chunk done. 2014-08-13 18:06:09.039174 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.358 now has 194496 2014-08-13 18:06:09.039197 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 already has epoch 194496 2014-08-13 18:06:09.039230 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 already has epoch 194496 2014-08-13 18:06:09.068538 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.367 now has 194496 2014-08-13 18:06:09.068548 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 already has epoch 194496 2014-08-13 18:06:09.068583 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 already has epoch 194496 2014-08-13 18:06:09.136612 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:09.136617 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:09.136620 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee4c80 replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194496,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 2014-08-13 18:06:09.136629 7f5c835ef700 10 osd.4 194496 queueing MOSDRepScrub replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194496,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 2014-08-13 18:06:09.136644 7f5c835ef700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:09.136646 7f5c835ef700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:09.136701 7f5c7a9e1700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] replica_scrub 2014-08-13 18:06:09.136739 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map 2014-08-13 18:06:09.136757 7f5c7a9e1700 20 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] scrub_map_chunk [e50d7bb9//0//-1,MAX) 2014-08-13 18:06:09.137525 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] _scan_list scanning 8 objects deeply 2014-08-13 18:06:09.186859 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.19 now has 194496 2014-08-13 18:06:09.186873 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 already has epoch 194496 2014-08-13 18:06:09.188116 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 already has epoch 194496 2014-08-13 18:06:09.188932 7f5c849f1700 10 osd.4 194496 do_waiters -- start 2014-08-13 18:06:09.188938 7f5c849f1700 10 osd.4 194496 do_waiters -- finish 2014-08-13 18:06:09.188941 7f5c849f1700 20 osd.4 194496 _dispatch 0x5138480 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.188983 7f5c849f1700 3 osd.4 194496 handle_osd_map epochs [194497,194497], i have 194496, src has [185139,194497] 2014-08-13 18:06:09.188991 7f5c849f1700 10 osd.4 194496 handle_osd_map got inc map for epoch 194497 2014-08-13 18:06:09.194252 7f5c849f1700 10 osd.4 194496 advance to epoch 194497 (<= newest 194497) 2014-08-13 18:06:09.194347 7f5c849f1700 7 osd.4 194497 advance_map epoch 194497 74 pgs 2014-08-13 18:06:09.194379 7f5c849f1700 10 osd.4 194497 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194497 [185139,194497] lci=[194495,194497]) 2014-08-13 18:06:09.194473 7f5c849f1700 0 osd.4 194497 crush map has features 2578087936000, adjusting msgr requires for mons 2014-08-13 18:06:09.194484 7f5c849f1700 7 osd.4 194497 consume_map version 194497 2014-08-13 18:06:09.309789 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.343 has 194497 2014-08-13 18:06:09.309800 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.309893 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.333911 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.371 has 194497 2014-08-13 18:06:09.333921 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4851e40 already has epoch 194497 2014-08-13 18:06:09.334027 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4851e40 already has epoch 194497 2014-08-13 18:06:09.361153 7f5c817ec700 10 osd.4 194497 note_peer_epoch osd.58 now has 194497 2014-08-13 18:06:09.361168 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x43f7900 already has epoch 194497 2014-08-13 18:06:09.361240 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x43f7900 already has epoch 194497 2014-08-13 18:06:09.485169 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.323 now has 194497 2014-08-13 18:06:09.485179 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x49a2520 already has epoch 194497 2014-08-13 18:06:09.485418 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x49a2520 already has epoch 194497 2014-08-13 18:06:09.485752 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map_chunk done. 2014-08-13 18:06:09.485923 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] null 2014-08-13 18:06:09.485984 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] null 2014-08-13 18:06:09.486010 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486045 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] null 2014-08-13 18:06:09.486030 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 2014-08-13 18:06:09.486073 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.486065 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] null 2014-08-13 18:06:09.486084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Peering advmap 2014-08-13 18:06:09.486087 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486075 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 2014-08-13 18:06:09.486119 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.486113 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.486130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.486136 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] null 2014-08-13 18:06:09.486139 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Started advmap 2014-08-13 18:06:09.486157 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] null 2014-08-13 18:06:09.486153 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.486185 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.486183 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] null 2014-08-13 18:06:09.486176 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.486203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.486216 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486221 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.486220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map 2014-08-13 18:06:09.486240 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.486240 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] null 2014-08-13 18:06:09.486245 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle ActMap primary 2014-08-13 18:06:09.486266 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.486267 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194497:30301 2014-08-13 18:06:09.486266 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] null 2014-08-13 18:06:09.486288 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] take_waiters 2014-08-13 18:06:09.486294 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] null 2014-08-13 18:06:09.486318 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486311 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 1.173163 5 0.000795 2014-08-13 18:06:09.486310 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 2014-08-13 18:06:09.486343 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.486340 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] null 2014-08-13 18:06:09.486340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Leaving Peering 2014-08-13 18:06:09.486351 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.486366 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.486364 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 1.328868 0 0.000000 2014-08-13 18:06:09.486372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.486389 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486393 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.486396 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active 2014-08-13 18:06:09.486412 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.486411 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:09.486431 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.486437 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null 2014-08-13 18:06:09.486417 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: In Active, about to call activate 2014-08-13 18:06:09.486458 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.486481 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.486484 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] null 2014-08-13 18:06:09.486508 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486503 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - snap_trimq [] 2014-08-13 18:06:09.486528 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.486525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - not complete, missing(56) 2014-08-13 18:06:09.486548 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486567 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486586 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] null 2014-08-13 18:06:09.486605 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null 2014-08-13 18:06:09.486626 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] null 2014-08-13 18:06:09.486647 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486666 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] null 2014-08-13 18:06:09.486685 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] null 2014-08-13 18:06:09.486692 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.486708 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.486708 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] null 2014-08-13 18:06:09.486730 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486749 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null 2014-08-13 18:06:09.486768 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] null 2014-08-13 18:06:09.486795 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486815 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null 2014-08-13 18:06:09.486833 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.486839 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2687 2014-08-13 18:06:09.486854 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] null 2014-08-13 18:06:09.486857 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.486874 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.486874 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] null 2014-08-13 18:06:09.486891 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.486911 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.486899 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null 2014-08-13 18:06:09.486922 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] null 2014-08-13 18:06:09.486935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 2014-08-13 18:06:09.486942 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null 2014-08-13 18:06:09.486958 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.486965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.486963 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] null 2014-08-13 18:06:09.486983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.486987 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] null 2014-08-13 18:06:09.487001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.487012 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] null 2014-08-13 18:06:09.487019 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.487034 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.487035 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] null 2014-08-13 18:06:09.487051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.487070 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.487057 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] null 2014-08-13 18:06:09.487082 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] null 2014-08-13 18:06:09.487099 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 2014-08-13 18:06:09.487104 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] null 2014-08-13 18:06:09.487120 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.487124 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.487126 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.487144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.487143 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] null 2014-08-13 18:06:09.487168 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] null 2014-08-13 18:06:09.487161 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.487187 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.487202 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.487189 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null 2014-08-13 18:06:09.487218 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.487237 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.487225 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:09.487246 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] null 2014-08-13 18:06:09.487259 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 2014-08-13 18:06:09.487266 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.487282 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.487287 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.487288 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.487307 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.487308 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.487326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,324) went down 2014-08-13 18:06:09.487330 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] null 2014-08-13 18:06:09.487344 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.487350 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null 2014-08-13 18:06:09.487361 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.487372 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] null 2014-08-13 18:06:09.487395 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null 2014-08-13 18:06:09.487416 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.487433 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] null 2014-08-13 18:06:09.487452 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] null 2014-08-13 18:06:09.487471 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] null 2014-08-13 18:06:09.487489 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null 2014-08-13 18:06:09.487515 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] null 2014-08-13 18:06:09.487534 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] null 2014-08-13 18:06:09.487553 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] null 2014-08-13 18:06:09.487573 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] null 2014-08-13 18:06:09.487591 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null 2014-08-13 18:06:09.487616 7f5c849f1700 10 osd.4 194497 maybe_update_heartbeat_peers updating 2014-08-13 18:06:09.487619 7f5c849f1700 20 osd.4 194497 24.30b heartbeat_peers 4,10,286,301,375 2014-08-13 18:06:09.487628 7f5c849f1700 20 osd.4 194497 22.220 heartbeat_peers 2014-08-13 18:06:09.487631 7f5c849f1700 20 osd.4 194497 22.362 heartbeat_peers 4,58,61,343,349,362 2014-08-13 18:06:09.487638 7f5c849f1700 20 osd.4 194497 22.28f heartbeat_peers 2014-08-13 18:06:09.487641 7f5c849f1700 20 osd.4 194497 22.14a heartbeat_peers 2014-08-13 18:06:09.487643 7f5c849f1700 20 osd.4 194497 22.22c heartbeat_peers 4,301,302,324,368 2014-08-13 18:06:09.487650 7f5c849f1700 20 osd.4 194497 22.619 heartbeat_peers 4,44,48,62,65,314 2014-08-13 18:06:09.487656 7f5c849f1700 20 osd.4 194497 22.30f heartbeat_peers 2014-08-13 18:06:09.487659 7f5c849f1700 20 osd.4 194497 24.9b heartbeat_peers 4,291,350,365,367,378 2014-08-13 18:06:09.487667 7f5c849f1700 20 osd.4 194497 22.581 heartbeat_peers 2014-08-13 18:06:09.487670 7f5c849f1700 20 osd.4 194497 22.f3 heartbeat_peers 4,285,311,312,369 2014-08-13 18:06:09.487675 7f5c849f1700 20 osd.4 194497 22.116 heartbeat_peers 2014-08-13 18:06:09.487677 7f5c849f1700 20 osd.4 194497 24.3d heartbeat_peers 4,20,25,303,362,373 2014-08-13 18:06:09.487685 7f5c849f1700 20 osd.4 194497 22.4aa heartbeat_peers 2014-08-13 18:06:09.487687 7f5c849f1700 20 osd.4 194497 22.1d2 heartbeat_peers 4,58,61,330,331,349 2014-08-13 18:06:09.487693 7f5c849f1700 20 osd.4 194497 24.1de heartbeat_peers 2014-08-13 18:06:09.487696 7f5c849f1700 20 osd.4 194497 2.2a heartbeat_peers 4,340 2014-08-13 18:06:09.487699 7f5c849f1700 20 osd.4 194497 22.113 heartbeat_peers 4,24,45,55,339,342 2014-08-13 18:06:09.487705 7f5c849f1700 20 osd.4 194497 24.39a heartbeat_peers 2014-08-13 18:06:09.487708 7f5c849f1700 20 osd.4 194497 24.33a heartbeat_peers 2014-08-13 18:06:09.487710 7f5c849f1700 20 osd.4 194497 1.2b heartbeat_peers 4,340 2014-08-13 18:06:09.487714 7f5c849f1700 20 osd.4 194497 24.34 heartbeat_peers 4,26,30,340 2014-08-13 18:06:09.487764 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.487779 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.487979 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2836 2014-08-13 18:06:09.487995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.488010 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.488025 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.488043 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.488282 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.26 192.168.1.115:6803/58522 192.168.100.115:6804/58522 2014-08-13 18:06:09.488329 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 2014-08-13 18:06:09.488375 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.488385 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.30 192.168.1.107:6824/55491 192.168.100.107:6841/55491 2014-08-13 18:06:09.488384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.488403 7f5c849f1700 20 osd.4 194497 22.99 heartbeat_peers 4,20,31,337,363,374 2014-08-13 18:06:09.488410 7f5c849f1700 20 osd.4 194497 22.84 heartbeat_peers 4,28,304 2014-08-13 18:06:09.488406 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.488415 7f5c849f1700 20 osd.4 194497 22.4df heartbeat_peers 4,352,356,357,371,380 2014-08-13 18:06:09.488430 7f5c849f1700 20 osd.4 194497 22.39d heartbeat_peers 4,320,344,363,366,373 2014-08-13 18:06:09.488437 7f5c849f1700 20 osd.4 194497 22.2b9 heartbeat_peers 2014-08-13 18:06:09.488439 7f5c849f1700 20 osd.4 194497 24.2c heartbeat_peers 2014-08-13 18:06:09.488442 7f5c849f1700 20 osd.4 194497 24.3f7 heartbeat_peers 2014-08-13 18:06:09.488429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.488445 7f5c849f1700 20 osd.4 194497 22.775 heartbeat_peers 0,4,56,63,353,369 2014-08-13 18:06:09.488451 7f5c849f1700 20 osd.4 194497 22.630 heartbeat_peers 2014-08-13 18:06:09.488455 7f5c849f1700 20 osd.4 194497 24.e2 heartbeat_peers 2014-08-13 18:06:09.488457 7f5c849f1700 20 osd.4 194497 22.69f heartbeat_peers 2014-08-13 18:06:09.488460 7f5c849f1700 20 osd.4 194497 22.3e4 heartbeat_peers 4,45,303,313,318,342 2014-08-13 18:06:09.488451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.488466 7f5c849f1700 20 osd.4 194497 24.2a1 heartbeat_peers 2014-08-13 18:06:09.488469 7f5c849f1700 20 osd.4 194497 22.639 heartbeat_peers 2014-08-13 18:06:09.488472 7f5c849f1700 20 osd.4 194497 22.707 heartbeat_peers 4,15,21,303,310,342 2014-08-13 18:06:09.488478 7f5c849f1700 20 osd.4 194497 24.5e heartbeat_peers 2014-08-13 18:06:09.488469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.488481 7f5c849f1700 20 osd.4 194497 22.1f1 heartbeat_peers 4,5,11,47,54,368 2014-08-13 18:06:09.488488 7f5c849f1700 20 osd.4 194497 22.2d1 heartbeat_peers 2014-08-13 18:06:09.488490 7f5c849f1700 20 osd.4 194497 24.b6 heartbeat_peers 4,17,28,315,327,330 2014-08-13 18:06:09.488497 7f5c849f1700 20 osd.4 194497 24.3bf heartbeat_peers 2014-08-13 18:06:09.488500 7f5c849f1700 20 osd.4 194497 22.1fb heartbeat_peers 2014-08-13 18:06:09.488502 7f5c849f1700 20 osd.4 194497 24.350 heartbeat_peers 2014-08-13 18:06:09.488505 7f5c849f1700 20 osd.4 194497 24.1f7 heartbeat_peers 4,55,59,351,376 2014-08-13 18:06:09.488511 7f5c849f1700 20 osd.4 194497 24.136 heartbeat_peers 4,50,55,61,65,356 2014-08-13 18:06:09.488518 7f5c849f1700 20 osd.4 194497 22.4fe heartbeat_peers 2014-08-13 18:06:09.488520 7f5c849f1700 20 osd.4 194497 22.653 heartbeat_peers 2014-08-13 18:06:09.488523 7f5c849f1700 20 osd.4 194497 24.3b7 heartbeat_peers 2014-08-13 18:06:09.488525 7f5c849f1700 20 osd.4 194497 22.3b9 heartbeat_peers 2014-08-13 18:06:09.488528 7f5c849f1700 20 osd.4 194497 22.791 heartbeat_peers 2014-08-13 18:06:09.488530 7f5c849f1700 20 osd.4 194497 22.6bd heartbeat_peers 4,50,289 2014-08-13 18:06:09.488535 7f5c849f1700 20 osd.4 194497 24.181 heartbeat_peers 4,55,296,307,320,344 2014-08-13 18:06:09.488617 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.296 192.168.1.109:6807/31286 192.168.100.109:6815/31286 2014-08-13 18:06:09.488797 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.488818 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.488824 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.307 192.168.1.109:6805/31016 192.168.100.109:6811/31016 2014-08-13 18:06:09.488845 7f5c849f1700 20 osd.4 194497 24.20e heartbeat_peers 2014-08-13 18:06:09.488849 7f5c849f1700 20 osd.4 194497 24.1e4 heartbeat_peers 2014-08-13 18:06:09.488851 7f5c849f1700 20 osd.4 194497 24.1e6 heartbeat_peers 2014-08-13 18:06:09.488854 7f5c849f1700 20 osd.4 194497 22.12a heartbeat_peers 2014-08-13 18:06:09.488856 7f5c849f1700 20 osd.4 194497 22.3af heartbeat_peers 4,44,61,339,362,369 2014-08-13 18:06:09.488862 7f5c849f1700 20 osd.4 194497 22.481 heartbeat_peers 4,287,303,306,309,365,381 2014-08-13 18:06:09.488869 7f5c849f1700 20 osd.4 194497 22.48e heartbeat_peers 4,289,327,341,376 2014-08-13 18:06:09.488874 7f5c849f1700 20 osd.4 194497 22.3ad heartbeat_peers 2014-08-13 18:06:09.488877 7f5c849f1700 20 osd.4 194497 24.206 heartbeat_peers 2014-08-13 18:06:09.488879 7f5c849f1700 20 osd.4 194497 24.1bb heartbeat_peers 4,331,332,337,348,379 2014-08-13 18:06:09.488887 7f5c849f1700 20 osd.4 194497 24.299 heartbeat_peers 2014-08-13 18:06:09.488889 7f5c849f1700 20 osd.4 194497 22.3d5 heartbeat_peers 4,286,343 2014-08-13 18:06:09.488893 7f5c849f1700 20 osd.4 194497 22.537 heartbeat_peers 2014-08-13 18:06:09.488896 7f5c849f1700 20 osd.4 194497 22.1b8 heartbeat_peers 4,340,347 2014-08-13 18:06:09.488900 7f5c849f1700 20 osd.4 194497 22.29a heartbeat_peers 2014-08-13 18:06:09.488902 7f5c849f1700 20 osd.4 194497 24.cd heartbeat_peers 2014-08-13 18:06:09.488905 7f5c849f1700 20 osd.4 194497 22.226 heartbeat_peers 2014-08-13 18:06:09.488907 7f5c849f1700 20 osd.4 194497 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 2014-08-13 18:06:09.488914 7f5c849f1700 20 osd.4 194497 24.2ec heartbeat_peers 2014-08-13 18:06:09.488916 7f5c849f1700 20 osd.4 194497 22.1a0 heartbeat_peers 2014-08-13 18:06:09.488919 7f5c849f1700 20 osd.4 194497 24.369 heartbeat_peers 2014-08-13 18:06:09.488922 7f5c849f1700 10 osd.4 194497 adding neighbor peer osd.3 2014-08-13 18:06:09.488925 7f5c849f1700 10 osd.4 194497 adding neighbor peer osd.5 2014-08-13 18:06:09.488936 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 2014-08-13 18:06:09.488978 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2739 2014-08-13 18:06:09.488995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.489012 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.489028 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.489048 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.489062 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 2014-08-13 18:06:09.489074 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 2014-08-13 18:06:09.489097 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.489104 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.489124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.489141 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.489158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.489172 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.489198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.489217 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.489246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 2014-08-13 18:06:09.489283 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.489291 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap 2014-08-13 18:06:09.489312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.489330 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap 2014-08-13 18:06:09.489349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.489368 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map 2014-08-13 18:06:09.489387 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary 2014-08-13 18:06:09.489412 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194497:25293 2014-08-13 18:06:09.489429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters 2014-08-13 18:06:09.489449 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 1.146346 5 0.000864 2014-08-13 18:06:09.489472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering 2014-08-13 18:06:09.489495 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.274086 0 0.000000 2014-08-13 18:06:09.489521 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active 2014-08-13 18:06:09.489539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate 2014-08-13 18:06:09.489535 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - starting recovery 2014-08-13 18:06:09.489564 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.489574 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] 2014-08-13 18:06:09.489582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] 2014-08-13 18:06:09.489599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) 2014-08-13 18:06:09.489635 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.489647 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 2014-08-13 18:06:09.489659 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 is up to date, queueing in pending_activators 2014-08-13 18:06:09.489746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.489771 7f5c849f1700 10 osd.4 194497 maybe_update_heartbeat_peers 92 peers, extras 3,5,7,32,370 2014-08-13 18:06:09.489780 7f5c849f1700 7 osd.4 194497 activate_map version 194497 2014-08-13 18:06:09.489793 7f5c849f1700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.489797 7f5c849f1700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489795 7f5c758d8700 20 osd.4 194497 agent_entry empty queue 2014-08-13 18:06:09.489812 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489823 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489826 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfbf00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.489841 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.489848 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.489851 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489853 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489854 7f5c7bde3700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 2014-08-13 18:06:09.489889 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489892 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489893 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee6300 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.489899 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.489904 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.489906 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489909 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489928 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489931 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489933 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5e80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.489938 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.489942 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.489944 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489946 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489970 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489973 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.489975 7f5c835ef700 20 osd.4 194497 _dispatch 0x5154380 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.489980 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.489984 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.489986 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.489989 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490008 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490010 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490012 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfd100 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490018 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490022 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490024 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490026 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490044 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490046 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490048 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee18c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490054 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490058 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490060 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490062 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490080 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490084 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee2640 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490089 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490093 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490095 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490097 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490115 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490117 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490119 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfb180 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490124 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490128 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490130 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490156 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490158 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490160 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4140 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490166 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490180 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490183 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490186 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490204 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490207 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490209 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0d80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490215 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490219 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490221 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490223 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490242 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490244 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490246 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf9f80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490252 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490256 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490258 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490260 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490278 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490281 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490283 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff2c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490288 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490293 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490295 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490297 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490314 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490317 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490319 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1b00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490324 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490329 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490331 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490333 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490351 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490354 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490356 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6300 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490362 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490366 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490368 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490370 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490388 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490390 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490392 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed2640 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490398 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490402 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490404 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490406 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490423 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490438 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490440 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6c00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490445 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490449 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490451 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490453 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490470 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490473 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490474 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed6540 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490480 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490484 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490486 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490488 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490506 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490509 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490510 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee4380 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490516 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490520 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490522 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490541 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490543 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490545 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee57c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490550 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490554 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490556 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490558 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490582 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490584 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490586 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfee40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490592 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490596 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490598 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490599 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490621 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490623 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490625 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc0fc0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490631 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490634 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490636 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490639 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490656 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490659 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490661 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff500 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490636 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 sending log((192718'15573,194422'18660], crt=194422'18660) 2014-08-13 18:06:09.490666 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490671 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490673 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490672 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:09.490675 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490693 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490675 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery 2014-08-13 18:06:09.490696 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490698 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4380 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490704 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490708 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490711 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490714 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490705 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] 2014-08-13 18:06:09.490732 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.490734 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490736 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf8900 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490742 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490746 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490749 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490751 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.490743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.490770 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490773 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 is up to date, queueing in pending_activators 2014-08-13 18:06:09.490776 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc1f80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490782 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490769 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 is up to date, queueing in pending_activators 2014-08-13 18:06:09.490786 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490794 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490814 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490816 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490818 7f5c835ef700 20 osd.4 194497 _dispatch 0x5151680 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490824 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490828 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490830 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490833 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490828 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.490850 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490856 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490858 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150fc0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490864 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490868 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490871 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490873 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.490893 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490896 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490898 7f5c835ef700 20 osd.4 194497 _dispatch 0x5151d40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490883 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.490904 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490909 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490912 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490914 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 is up to date, queueing in pending_activators 2014-08-13 18:06:09.490932 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490935 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490937 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0900 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490942 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490946 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490948 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490950 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490968 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490970 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490972 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfca40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.490977 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.490981 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.490983 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.490985 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491003 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491006 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490988 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.491008 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed7980 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491013 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491018 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491020 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491022 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.490903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery primary has 56 missing 2014-08-13 18:06:09.491012 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.491040 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491041 7f5c7c7e4700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 2014-08-13 18:06:09.491043 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491045 7f5c835ef700 20 osd.4 194497 _dispatch 0x5155100 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491051 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491055 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491057 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491059 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491076 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491079 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491080 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc2400 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491086 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491090 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491092 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491094 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491111 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491113 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491115 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4800 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491120 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491124 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491126 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491128 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491145 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491030 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.4 2014-08-13 18:06:09.491158 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491161 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf8d80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491167 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491178 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491181 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491183 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491201 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491204 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491206 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed7740 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491211 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491215 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491218 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491220 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491237 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491239 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491241 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf98c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491247 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491251 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491253 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491255 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491272 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491275 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491276 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150240 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491282 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491286 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491288 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491290 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491162 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.4 2014-08-13 18:06:09.491307 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491310 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491312 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfd7c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491318 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491322 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491324 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491326 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 sending log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 18:06:09.491303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.4 2014-08-13 18:06:09.491412 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4b11ce0 192.168.1.114:6840/55513 2014-08-13 18:06:09.491435 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491440 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491443 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee06c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491449 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.367 has 194497 2014-08-13 18:06:09.491454 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491461 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491464 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491466 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491489 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491492 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.491494 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1f80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491500 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491505 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491507 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491509 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491497 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.491525 7f5c7c7e4700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 2014-08-13 18:06:09.491529 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491531 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491533 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4c80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491423 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.4 2014-08-13 18:06:09.491539 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491544 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491546 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491549 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491567 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491569 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491571 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed0900 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491577 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491581 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491583 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491585 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491602 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491604 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491606 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5a00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491612 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491616 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491618 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491620 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491637 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491642 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed69c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491647 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491651 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491653 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491655 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491673 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491542 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.4 2014-08-13 18:06:09.491676 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491678 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed3cc0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491684 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491688 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491690 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491693 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491710 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491712 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491714 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5340 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491719 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491724 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491726 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491728 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491745 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491747 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491749 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1200 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491755 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491759 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491761 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491763 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491780 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491783 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491784 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4800 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491790 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491794 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491798 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491800 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491793 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 sending log((192834'15069,194422'17302], crt=194422'17302) 2014-08-13 18:06:09.491818 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.4 2014-08-13 18:06:09.491821 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491823 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff980 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491829 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491821 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing 2014-08-13 18:06:09.491833 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491836 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491838 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491857 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491860 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491863 7f5c835ef700 20 osd.4 194497 _dispatch 0x51506c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491869 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491873 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491875 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491878 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.491895 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491897 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491899 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152ac0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491905 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491909 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491911 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491913 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491933 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491936 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491938 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152640 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491825 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.4 2014-08-13 18:06:09.491944 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491949 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.491952 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491954 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491971 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.491974 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491989 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152f40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.491995 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.491999 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492001 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492003 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492020 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492022 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492024 7f5c835ef700 20 osd.4 194497 _dispatch 0x5153cc0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492029 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492033 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492035 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492037 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492055 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492057 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.491933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing 2014-08-13 18:06:09.491945 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.4 2014-08-13 18:06:09.492059 7f5c835ef700 20 osd.4 194497 _dispatch 0x5154a40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492065 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492071 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492074 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492076 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492097 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492099 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492101 7f5c835ef700 20 osd.4 194497 _dispatch 0x5155e80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492107 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492110 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492113 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492115 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492131 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492134 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492136 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cffbc0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492141 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492145 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492147 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492149 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492167 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492169 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492171 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee5a00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492176 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492189 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492192 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492194 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.4 2014-08-13 18:06:09.492065 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.4 2014-08-13 18:06:09.492212 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492216 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492218 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc1200 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492224 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492228 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492231 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492233 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492250 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492252 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492254 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee72c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492259 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492264 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492266 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492268 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492285 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492287 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492289 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc2ac0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492295 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492298 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492300 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492303 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492320 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.4 2014-08-13 18:06:09.492323 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492325 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed2ac0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492331 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492335 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492337 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492340 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492358 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492360 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492362 7f5c835ef700 20 osd.4 194497 _dispatch 0x5138d80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.4 2014-08-13 18:06:09.492367 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492372 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492374 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492376 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492394 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492398 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492399 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee5c40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492405 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492409 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492411 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492413 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492429 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492432 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492433 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1680 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492439 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492443 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492445 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.4 2014-08-13 18:06:09.492447 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492468 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492471 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492473 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfde80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492478 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492482 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492484 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492486 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.4 2014-08-13 18:06:09.492504 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492506 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492508 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfe9c0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492513 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492518 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492520 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492522 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492551 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492554 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492556 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0b40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492561 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492565 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492567 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.4 2014-08-13 18:06:09.492569 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492594 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492596 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492598 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4ec0 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492604 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492607 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492609 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492612 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492628 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492631 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492633 7f5c835ef700 20 osd.4 194497 _dispatch 0x2002d00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.492638 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.492642 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.492501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.4 2014-08-13 18:06:09.492644 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492647 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492678 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492680 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492682 7f5c835ef700 20 osd.4 194497 _dispatch 0x524ee40 pg_notify(22.3e4(36) epoch 194497) v5 2014-08-13 18:06:09.492576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.4 2014-08-13 18:06:09.492737 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.45 2014-08-13 18:06:09.492741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524ee40 2014-08-13 18:06:09.492648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.4 2014-08-13 18:06:09.492809 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492813 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492698 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.4 2014-08-13 18:06:09.492833 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492836 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492838 7f5c835ef700 20 osd.4 194497 _dispatch 0x4dea840 pg_info(1 pgs e194497:24.2a1(4)) v4 2014-08-13 18:06:09.492875 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2a1(4)) v4 from osd.17 2014-08-13 18:06:09.492767 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.4 2014-08-13 18:06:09.492881 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4dea840 2014-08-13 18:06:09.492923 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492927 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492821 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.4 2014-08-13 18:06:09.492946 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.492948 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.492951 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351880 pg_notify(22.619(1) epoch 194497) v5 2014-08-13 18:06:09.492990 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.44 2014-08-13 18:06:09.492993 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351880 2014-08-13 18:06:09.492883 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.4 2014-08-13 18:06:09.493031 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493034 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493053 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493055 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493057 7f5c835ef700 20 osd.4 194497 _dispatch 0x588abc0 pg_notify(24.136(36) epoch 194497) v5 2014-08-13 18:06:09.492943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.4 2014-08-13 18:06:09.493094 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.356 2014-08-13 18:06:09.493097 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588abc0 2014-08-13 18:06:09.493014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.4 2014-08-13 18:06:09.493140 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493143 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493162 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493165 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493167 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b800 pg_notify(22.113(34) epoch 194497) v5 2014-08-13 18:06:09.493070 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.4 2014-08-13 18:06:09.493211 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.55 2014-08-13 18:06:09.493215 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b800 2014-08-13 18:06:09.493252 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493255 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493273 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493275 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493277 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5cc0 pg_notify(24.136(2) epoch 194497) v5 2014-08-13 18:06:09.493315 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.50 2014-08-13 18:06:09.493199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.4 2014-08-13 18:06:09.493319 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5cc0 2014-08-13 18:06:09.493132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.4 2014-08-13 18:06:09.493368 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493372 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493390 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493392 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493394 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38380 pg_notify(22.99(30) epoch 194497) v5 2014-08-13 18:06:09.493429 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.337 2014-08-13 18:06:09.493432 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38380 2014-08-13 18:06:09.493323 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.4 2014-08-13 18:06:09.493468 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493472 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.4 2014-08-13 18:06:09.493490 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493493 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493495 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_notify(22.4df(32) epoch 194497) v5 2014-08-13 18:06:09.493530 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.371 2014-08-13 18:06:09.493534 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 2014-08-13 18:06:09.493450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.4 2014-08-13 18:06:09.493570 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493574 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493593 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493596 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493598 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df4440 pg_info(1 pgs e194497:24.206(4)) v4 2014-08-13 18:06:09.493488 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.4 2014-08-13 18:06:09.493634 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.206(4)) v4 from osd.343 2014-08-13 18:06:09.493639 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df4440 2014-08-13 18:06:09.493676 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493679 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493573 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.4 2014-08-13 18:06:09.493698 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493701 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493703 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888e00 pg_notify(24.181(38) epoch 194497) v5 2014-08-13 18:06:09.493739 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.296 2014-08-13 18:06:09.493742 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888e00 2014-08-13 18:06:09.493624 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.4 2014-08-13 18:06:09.493799 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493803 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493696 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.4 2014-08-13 18:06:09.493821 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493824 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493826 7f5c835ef700 20 osd.4 194497 _dispatch 0x524d240 pg_notify(22.707(38) epoch 194497) v5 2014-08-13 18:06:09.493862 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.15 2014-08-13 18:06:09.493865 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524d240 2014-08-13 18:06:09.493752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.4 2014-08-13 18:06:09.493918 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493921 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.4 2014-08-13 18:06:09.493942 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.493944 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493960 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354980 pg_info(1 pgs e194497:24.1e4(25)) v4 2014-08-13 18:06:09.493997 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e4(25)) v4 from osd.31 2014-08-13 18:06:09.494002 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354980 2014-08-13 18:06:09.494039 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494042 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.493943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.4 2014-08-13 18:06:09.494060 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494067 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494070 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352f40 pg_notify(22.362(34) epoch 194497) v5 2014-08-13 18:06:09.493892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.4 2014-08-13 18:06:09.494108 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.362 2014-08-13 18:06:09.494111 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352f40 2014-08-13 18:06:09.494147 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494150 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494169 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494172 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494173 7f5c835ef700 20 osd.4 194497 _dispatch 0x588ee40 pg_info(1 pgs e194497:24.33a(4)) v4 2014-08-13 18:06:09.494066 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.4 2014-08-13 18:06:09.494081 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.4 2014-08-13 18:06:09.494226 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.33a(4)) v4 from osd.304 2014-08-13 18:06:09.494230 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588ee40 2014-08-13 18:06:09.494268 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494272 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494291 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494295 7f5c835ef700 20 osd.4 194497 _dispatch 0x588cec0 pg_notify(24.1bb(2) epoch 194497) v5 2014-08-13 18:06:09.494193 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.4 2014-08-13 18:06:09.494332 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.332 2014-08-13 18:06:09.494336 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588cec0 2014-08-13 18:06:09.494370 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494373 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494393 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494396 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494397 7f5c835ef700 20 osd.4 194497 _dispatch 0x4a43100 pg_notify(22.99(1),22.39d(34) epoch 194497) v5 2014-08-13 18:06:09.494318 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.4 2014-08-13 18:06:09.494440 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.363 2014-08-13 18:06:09.494444 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4a43100 2014-08-13 18:06:09.494283 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.4 2014-08-13 18:06:09.494497 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494500 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494520 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494523 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494525 7f5c835ef700 20 osd.4 194497 _dispatch 0x53547c0 pg_info(1 pgs e194497:24.2ec(2)) v4 2014-08-13 18:06:09.494441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.4 2014-08-13 18:06:09.494562 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2ec(2)) v4 from osd.374 2014-08-13 18:06:09.494567 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x53547c0 2014-08-13 18:06:09.494605 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494609 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.4 2014-08-13 18:06:09.494627 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494629 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494632 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fe00 pg_notify(22.113(34) epoch 194497) v5 2014-08-13 18:06:09.494667 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.24 2014-08-13 18:06:09.494670 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fe00 2014-08-13 18:06:09.494563 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.4 2014-08-13 18:06:09.494710 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494713 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494731 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494735 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357700 pg_info(1 pgs e194497:24.cd(4)) v4 2014-08-13 18:06:09.494617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.4 2014-08-13 18:06:09.494775 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.cd(4)) v4 from osd.6 2014-08-13 18:06:09.494779 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357700 2014-08-13 18:06:09.494686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.4 2014-08-13 18:06:09.494821 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494824 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494843 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494845 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494847 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351c00 pg_notify(24.181(38) epoch 194497) v5 2014-08-13 18:06:09.494882 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.344 2014-08-13 18:06:09.494885 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351c00 2014-08-13 18:06:09.494752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.4 2014-08-13 18:06:09.494808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.4 2014-08-13 18:06:09.494940 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494944 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494962 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.494964 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494966 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38c40 pg_notify(22.707(1) epoch 194497) v5 2014-08-13 18:06:09.495001 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 2014-08-13 18:06:09.495004 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38c40 2014-08-13 18:06:09.494929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.4 2014-08-13 18:06:09.495050 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495054 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.494904 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.4 2014-08-13 18:06:09.495072 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495075 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495077 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357e00 pg_notify(24.1f7(44) epoch 194497) v5 2014-08-13 18:06:09.495127 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.351 2014-08-13 18:06:09.495130 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357e00 2014-08-13 18:06:09.495051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.4 2014-08-13 18:06:09.495184 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495188 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495208 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495211 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495213 7f5c835ef700 20 osd.4 194497 _dispatch 0x588ec80 pg_notify(22.1d2(1) epoch 194497) v5 2014-08-13 18:06:09.495248 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.331 2014-08-13 18:06:09.495251 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588ec80 2014-08-13 18:06:09.495285 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495288 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495174 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.4 2014-08-13 18:06:09.495307 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495310 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495312 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf0fc0 pg_notify(22.39d(34) epoch 194497) v5 2014-08-13 18:06:09.495060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.4 2014-08-13 18:06:09.495347 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.320 2014-08-13 18:06:09.495350 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf0fc0 2014-08-13 18:06:09.495392 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495395 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495419 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.4 2014-08-13 18:06:09.495421 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495423 7f5c835ef700 20 osd.4 194497 _dispatch 0x59e3700 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.495487 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 2014-08-13 18:06:09.495348 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.4 2014-08-13 18:06:09.495496 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x59e3700 2014-08-13 18:06:09.495500 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.367 192.168.1.114:6840/55513 194497 2014-08-13 18:06:09.495511 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a5a00 prio 127 cost 0 latency 0.006159 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.495524 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495527 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495424 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.4 2014-08-13 18:06:09.495544 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495547 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495550 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b640 pg_notify(24.136(36) epoch 194497) v5 2014-08-13 18:06:09.495585 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.65 2014-08-13 18:06:09.495588 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b640 2014-08-13 18:06:09.495637 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495500 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.4 2014-08-13 18:06:09.495640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495659 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495662 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495546 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.4 2014-08-13 18:06:09.495664 7f5c835ef700 20 osd.4 194497 _dispatch 0x5353800 pg_notify(24.3d(38) epoch 194497) v5 2014-08-13 18:06:09.495700 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.373 2014-08-13 18:06:09.495703 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5353800 2014-08-13 18:06:09.495739 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495741 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495759 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495762 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495763 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c0c0 pg_notify(22.3d9(21) epoch 194497) v5 2014-08-13 18:06:09.495643 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.4 2014-08-13 18:06:09.495669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.4 2014-08-13 18:06:09.495800 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.397 2014-08-13 18:06:09.495803 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c0c0 2014-08-13 18:06:09.495845 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495849 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495866 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495869 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495871 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356580 pg_notify(22.3d9(50) epoch 194497) v5 2014-08-13 18:06:09.495905 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.396 2014-08-13 18:06:09.495908 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356580 2014-08-13 18:06:09.495792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.4 2014-08-13 18:06:09.495776 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.4 2014-08-13 18:06:09.495944 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495947 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495985 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.495987 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.495990 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a696c0 pg_notify(24.1f7(40) epoch 194497) v5 2014-08-13 18:06:09.496025 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.376 2014-08-13 18:06:09.496028 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a696c0 2014-08-13 18:06:09.495914 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.4 2014-08-13 18:06:09.495919 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.4 2014-08-13 18:06:09.496066 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496069 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496086 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496089 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496091 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356040 pg_notify(24.1bb(40) epoch 194497) v5 2014-08-13 18:06:09.496125 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.379 2014-08-13 18:06:09.496128 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356040 2014-08-13 18:06:09.496037 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.4 2014-08-13 18:06:09.496164 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496167 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496046 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.4 2014-08-13 18:06:09.496216 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496219 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496221 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6ee40 pg_notify(22.775(36) epoch 194497) v5 2014-08-13 18:06:09.496256 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.0 2014-08-13 18:06:09.496259 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6ee40 2014-08-13 18:06:09.496159 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.4 2014-08-13 18:06:09.496295 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496298 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496192 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.4 2014-08-13 18:06:09.496348 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496351 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496353 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6bd40 pg_notify(22.1f1(1) epoch 194497) v5 2014-08-13 18:06:09.496388 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.11 2014-08-13 18:06:09.496391 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6bd40 2014-08-13 18:06:09.496286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.4 2014-08-13 18:06:09.496436 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496439 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.4 2014-08-13 18:06:09.496458 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496461 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496463 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e900 pg_notify(22.4df(1) epoch 194497) v5 2014-08-13 18:06:09.496498 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.352 2014-08-13 18:06:09.496501 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e900 2014-08-13 18:06:09.496409 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.4 2014-08-13 18:06:09.496538 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496541 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496558 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496561 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496563 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352a00 pg_notify(22.48e(34) epoch 194497) v5 2014-08-13 18:06:09.496451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.4 2014-08-13 18:06:09.496600 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.327 2014-08-13 18:06:09.496603 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352a00 2014-08-13 18:06:09.496531 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.4 2014-08-13 18:06:09.496649 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496652 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496670 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496673 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496675 7f5c835ef700 20 osd.4 194497 _dispatch 0x5889c00 pg_notify(22.3af(36) epoch 194497) v5 2014-08-13 18:06:09.496578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.4 2014-08-13 18:06:09.496713 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.339 2014-08-13 18:06:09.496716 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5889c00 2014-08-13 18:06:09.496752 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496755 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496653 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.4 2014-08-13 18:06:09.496775 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496778 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496780 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f380 pg_notify(22.f3(26) epoch 194497) v5 2014-08-13 18:06:09.496816 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.311 2014-08-13 18:06:09.496705 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.4 2014-08-13 18:06:09.496820 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f380 2014-08-13 18:06:09.496858 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496860 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496893 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496895 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.496778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.4 2014-08-13 18:06:09.496898 7f5c835ef700 20 osd.4 194497 _dispatch 0x588fa80 pg_notify(22.3d9(54) epoch 194497) v5 2014-08-13 18:06:09.496934 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.293 2014-08-13 18:06:09.496823 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.4 2014-08-13 18:06:09.496937 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588fa80 2014-08-13 18:06:09.496988 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.496992 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497010 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497012 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497014 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c980 pg_notify(22.113(1) epoch 194497) v5 2014-08-13 18:06:09.496902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.4 2014-08-13 18:06:09.497051 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.342 2014-08-13 18:06:09.496940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.4 2014-08-13 18:06:09.497057 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c980 2014-08-13 18:06:09.497092 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497095 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497114 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497117 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497119 7f5c835ef700 20 osd.4 194497 _dispatch 0x588cd00 pg_notify(22.619(36) epoch 194497) v5 2014-08-13 18:06:09.497025 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.4 2014-08-13 18:06:09.497155 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.314 2014-08-13 18:06:09.497158 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588cd00 2014-08-13 18:06:09.497061 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.4 2014-08-13 18:06:09.497211 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497233 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497235 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497237 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a39340 pg_notify(22.619(36) epoch 194497) v5 2014-08-13 18:06:09.497147 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.4 2014-08-13 18:06:09.497275 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.48 2014-08-13 18:06:09.497279 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a39340 2014-08-13 18:06:09.497186 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.4 2014-08-13 18:06:09.497314 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497317 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497341 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497344 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497346 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a3a680 pg_notify(22.4df(32) epoch 194497) v5 2014-08-13 18:06:09.497380 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.380 2014-08-13 18:06:09.497383 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a3a680 2014-08-13 18:06:09.497274 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.4 2014-08-13 18:06:09.497302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.4 2014-08-13 18:06:09.497417 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497420 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497449 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497452 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497453 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fa80 pg_notify(22.1f1(34) epoch 194497) v5 2014-08-13 18:06:09.497501 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.368 2014-08-13 18:06:09.497504 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fa80 2014-08-13 18:06:09.497397 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.4 2014-08-13 18:06:09.497420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.4 2014-08-13 18:06:09.497538 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497542 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497576 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.497579 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.497580 7f5c835ef700 20 osd.4 194497 _dispatch 0x5355e80 pg_notify(24.9b(39) epoch 194497) v5 2014-08-13 18:06:09.497615 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.350 2014-08-13 18:06:09.497618 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5355e80 2014-08-13 18:06:09.497520 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.4 2014-08-13 18:06:09.497540 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.4 2014-08-13 18:06:09.497643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.4 2014-08-13 18:06:09.497662 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.4 2014-08-13 18:06:09.497764 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.4 2014-08-13 18:06:09.497802 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.4 2014-08-13 18:06:09.497885 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.4 2014-08-13 18:06:09.497931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.4 2014-08-13 18:06:09.498006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.4 2014-08-13 18:06:09.498048 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.4 2014-08-13 18:06:09.498127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.4 2014-08-13 18:06:09.498190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.4 2014-08-13 18:06:09.498253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.4 2014-08-13 18:06:09.498338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.4 2014-08-13 18:06:09.498375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.4 2014-08-13 18:06:09.498461 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.4 2014-08-13 18:06:09.498496 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.4 2014-08-13 18:06:09.498620 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.4 2014-08-13 18:06:09.498609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.4 2014-08-13 18:06:09.498743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.4 2014-08-13 18:06:09.498748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.4 2014-08-13 18:06:09.498865 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.4 2014-08-13 18:06:09.498986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.4 2014-08-13 18:06:09.498880 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 18:06:09.499107 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.4 2014-08-13 18:06:09.499112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] activate: adding 10 as a source 2014-08-13 18:06:09.499233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.4 2014-08-13 18:06:09.499302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.10 2014-08-13 18:06:09.499353 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 18:06:09.499426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=55] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.10 2014-08-13 18:06:09.499549 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 291 as a source 2014-08-13 18:06:09.499563 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=54] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.10 2014-08-13 18:06:09.499670 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.291 2014-08-13 18:06:09.499692 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=53] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.10 2014-08-13 18:06:09.499792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.291 2014-08-13 18:06:09.499820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=52] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.10 2014-08-13 18:06:09.499921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.291 2014-08-13 18:06:09.499952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=51] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.10 2014-08-13 18:06:09.500052 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.291 2014-08-13 18:06:09.500086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=50] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.10 2014-08-13 18:06:09.500188 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.291 2014-08-13 18:06:09.500231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=49] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.10 2014-08-13 18:06:09.500326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.291 2014-08-13 18:06:09.500376 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=48] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.10 2014-08-13 18:06:09.500465 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.291 2014-08-13 18:06:09.500552 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=47] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.10 2014-08-13 18:06:09.500609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.291 2014-08-13 18:06:09.500754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.291 2014-08-13 18:06:09.500723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=46] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.10 2014-08-13 18:06:09.500902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.291 2014-08-13 18:06:09.500914 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=45] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.10 2014-08-13 18:06:09.501053 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.291 2014-08-13 18:06:09.501119 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=44] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.10 2014-08-13 18:06:09.501226 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.291 2014-08-13 18:06:09.501347 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=43] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.10 2014-08-13 18:06:09.501386 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.291 2014-08-13 18:06:09.501528 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=42] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.10 2014-08-13 18:06:09.501547 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.291 2014-08-13 18:06:09.501711 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.291 2014-08-13 18:06:09.501710 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=41] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.10 2014-08-13 18:06:09.501881 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.291 2014-08-13 18:06:09.501897 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=40] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.10 2014-08-13 18:06:09.502051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.291 2014-08-13 18:06:09.502104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=39] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.10 2014-08-13 18:06:09.502231 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.291 2014-08-13 18:06:09.502337 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=38] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.10 2014-08-13 18:06:09.502410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.291 2014-08-13 18:06:09.502514 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=37] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.10 2014-08-13 18:06:09.502589 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.291 2014-08-13 18:06:09.502690 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=36] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.10 2014-08-13 18:06:09.502771 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.291 2014-08-13 18:06:09.502869 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=35] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.10 2014-08-13 18:06:09.502955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.291 2014-08-13 18:06:09.503050 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=34] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.10 2014-08-13 18:06:09.503142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.291 2014-08-13 18:06:09.503275 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=33] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.10 2014-08-13 18:06:09.503338 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.291 2014-08-13 18:06:09.503492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=32] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.10 2014-08-13 18:06:09.503534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.291 2014-08-13 18:06:09.503731 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.291 2014-08-13 18:06:09.503709 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=31] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.10 2014-08-13 18:06:09.503931 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.291 2014-08-13 18:06:09.503967 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=30] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.10 2014-08-13 18:06:09.504135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.291 2014-08-13 18:06:09.504213 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=29] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.10 2014-08-13 18:06:09.504346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.291 2014-08-13 18:06:09.504555 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.291 2014-08-13 18:06:09.504477 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=28] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.10 2014-08-13 18:06:09.504768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.291 2014-08-13 18:06:09.504785 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=27] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.10 2014-08-13 18:06:09.504983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.291 2014-08-13 18:06:09.505002 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=26] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.10 2014-08-13 18:06:09.505213 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.291 2014-08-13 18:06:09.505221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=25] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.10 2014-08-13 18:06:09.505437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.291 2014-08-13 18:06:09.505475 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=24] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.10 2014-08-13 18:06:09.505662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.291 2014-08-13 18:06:09.505724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=23] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.10 2014-08-13 18:06:09.505891 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.291 2014-08-13 18:06:09.505984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=22] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.10 2014-08-13 18:06:09.506121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.291 2014-08-13 18:06:09.506280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=21] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.10 2014-08-13 18:06:09.506360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.291 2014-08-13 18:06:09.506548 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=20] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.10 2014-08-13 18:06:09.506597 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.291 2014-08-13 18:06:09.506841 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.291 2014-08-13 18:06:09.506820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=19] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.10 2014-08-13 18:06:09.507086 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.291 2014-08-13 18:06:09.507090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=18] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.10 2014-08-13 18:06:09.507340 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.291 2014-08-13 18:06:09.507399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=17] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.10 2014-08-13 18:06:09.507591 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.291 2014-08-13 18:06:09.507693 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=16] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.10 2014-08-13 18:06:09.507843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.291 2014-08-13 18:06:09.507973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=15] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.10 2014-08-13 18:06:09.508098 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.291 2014-08-13 18:06:09.508270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=14] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.10 2014-08-13 18:06:09.508360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.291 2014-08-13 18:06:09.508568 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=13] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.10 2014-08-13 18:06:09.508621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.291 2014-08-13 18:06:09.508886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.291 2014-08-13 18:06:09.508863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=12] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.10 2014-08-13 18:06:09.509154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.291 2014-08-13 18:06:09.509175 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=11] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.10 2014-08-13 18:06:09.509429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.291 2014-08-13 18:06:09.509479 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=10] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.10 2014-08-13 18:06:09.509703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.291 2014-08-13 18:06:09.509772 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=9] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.10 2014-08-13 18:06:09.509981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.291 2014-08-13 18:06:09.510084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=8] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.10 2014-08-13 18:06:09.510267 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.291 2014-08-13 18:06:09.510414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=7] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.10 2014-08-13 18:06:09.510554 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.291 2014-08-13 18:06:09.510690 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=6] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.10 2014-08-13 18:06:09.510840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.291 2014-08-13 18:06:09.510962 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=5] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.10 2014-08-13 18:06:09.511131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.291 2014-08-13 18:06:09.511253 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=4] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.10 2014-08-13 18:06:09.511429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.291 2014-08-13 18:06:09.511576 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=3] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.10 2014-08-13 18:06:09.511726 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.291 2014-08-13 18:06:09.511913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=2] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.10 2014-08-13 18:06:09.512030 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.291 2014-08-13 18:06:09.512264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=1] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.10 2014-08-13 18:06:09.512341 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 18:06:09.512599 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 18:06:09.512722 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 350 as a source 2014-08-13 18:06:09.512966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 286 as a source 2014-08-13 18:06:09.513030 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.350 2014-08-13 18:06:09.513360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.350 2014-08-13 18:06:09.513350 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.513673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.350 2014-08-13 18:06:09.513688 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.513984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.513983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.350 2014-08-13 18:06:09.514298 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.350 2014-08-13 18:06:09.514295 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.514620 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.514608 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.350 2014-08-13 18:06:09.514918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.514923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.350 2014-08-13 18:06:09.515235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.350 2014-08-13 18:06:09.515218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.515543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.515545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.350 2014-08-13 18:06:09.515839 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.515852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.350 2014-08-13 18:06:09.516131 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.516158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.350 2014-08-13 18:06:09.516465 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.516475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.350 2014-08-13 18:06:09.516760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.516782 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.350 2014-08-13 18:06:09.517051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.517090 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.350 2014-08-13 18:06:09.517390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.517400 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.350 2014-08-13 18:06:09.517691 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.517706 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.350 2014-08-13 18:06:09.518006 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.518012 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.350 2014-08-13 18:06:09.518322 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.350 2014-08-13 18:06:09.518313 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.518630 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.350 2014-08-13 18:06:09.518645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.518940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.350 2014-08-13 18:06:09.518939 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.519255 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.350 2014-08-13 18:06:09.519290 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.519564 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.350 2014-08-13 18:06:09.519596 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.519871 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.350 2014-08-13 18:06:09.519885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.520178 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.350 2014-08-13 18:06:09.520180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.520506 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.520491 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.350 2014-08-13 18:06:09.520803 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.520811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.350 2014-08-13 18:06:09.521119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.350 2014-08-13 18:06:09.521095 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.521439 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.350 2014-08-13 18:06:09.521440 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.521752 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.350 2014-08-13 18:06:09.521797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.522060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.350 2014-08-13 18:06:09.522088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.522382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.350 2014-08-13 18:06:09.522431 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.522691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.350 2014-08-13 18:06:09.522746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.522999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.350 2014-08-13 18:06:09.523042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.523312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.350 2014-08-13 18:06:09.523386 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.523621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.350 2014-08-13 18:06:09.523680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.523928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.350 2014-08-13 18:06:09.523987 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.524243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.350 2014-08-13 18:06:09.524326 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.524552 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.350 2014-08-13 18:06:09.524637 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.524859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.350 2014-08-13 18:06:09.524929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.525167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.350 2014-08-13 18:06:09.525227 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.525479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.350 2014-08-13 18:06:09.525553 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.525787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.350 2014-08-13 18:06:09.525846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.526095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.350 2014-08-13 18:06:09.526136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.526407 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.350 2014-08-13 18:06:09.526456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.526715 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.350 2014-08-13 18:06:09.526748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.527023 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.350 2014-08-13 18:06:09.527037 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.527336 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.350 2014-08-13 18:06:09.527341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.527644 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.527644 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.350 2014-08-13 18:06:09.527938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.527952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.350 2014-08-13 18:06:09.528262 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.350 2014-08-13 18:06:09.528258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.528570 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.350 2014-08-13 18:06:09.528608 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.528876 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.350 2014-08-13 18:06:09.528901 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.529183 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.350 2014-08-13 18:06:09.529205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.529495 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.350 2014-08-13 18:06:09.529523 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.529813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.529802 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.350 2014-08-13 18:06:09.530109 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.350 2014-08-13 18:06:09.530106 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.530422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.350 2014-08-13 18:06:09.530443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.286 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.530729 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.350 2014-08-13 18:06:09.530740 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 18:06:09.531038 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.350 2014-08-13 18:06:09.531101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 375 as a source 2014-08-13 18:06:09.531351 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 18:06:09.531414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.375 2014-08-13 18:06:09.531739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.375 2014-08-13 18:06:09.531733 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 367 as a source 2014-08-13 18:06:09.532034 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.375 2014-08-13 18:06:09.532039 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.532350 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.532344 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.375 2014-08-13 18:06:09.532659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.532672 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.375 2014-08-13 18:06:09.532967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.532984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.375 2014-08-13 18:06:09.533278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.533328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.375 2014-08-13 18:06:09.533586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.533640 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.375 2014-08-13 18:06:09.533892 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.533933 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.375 2014-08-13 18:06:09.534204 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.534230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.375 2014-08-13 18:06:09.534511 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.534561 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.375 2014-08-13 18:06:09.534819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.534855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.375 2014-08-13 18:06:09.535125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.535144 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.375 2014-08-13 18:06:09.535438 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.535461 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.375 2014-08-13 18:06:09.535752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.375 2014-08-13 18:06:09.535746 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.536045 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.375 2014-08-13 18:06:09.536051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.536360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.536349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.375 2014-08-13 18:06:09.536674 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.375 2014-08-13 18:06:09.536668 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.536971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.375 2014-08-13 18:06:09.536973 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.537276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.375 2014-08-13 18:06:09.537283 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.537587 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.375 2014-08-13 18:06:09.537587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.537883 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.375 2014-08-13 18:06:09.537894 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.538203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.538220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.375 2014-08-13 18:06:09.538510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.538518 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.375 2014-08-13 18:06:09.538818 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.538841 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.375 2014-08-13 18:06:09.539125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.539134 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.375 2014-08-13 18:06:09.539450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.375 2014-08-13 18:06:09.539437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.539749 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.375 2014-08-13 18:06:09.539753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.540042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.375 2014-08-13 18:06:09.540060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.540367 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.375 2014-08-13 18:06:09.540372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.540679 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.375 2014-08-13 18:06:09.540677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.540972 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.375 2014-08-13 18:06:09.540982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.541281 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.375 2014-08-13 18:06:09.541292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.541597 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.375 2014-08-13 18:06:09.541596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.541892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.375 2014-08-13 18:06:09.541902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.542211 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.542204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.375 2014-08-13 18:06:09.542525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.375 2014-08-13 18:06:09.542519 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.542820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.375 2014-08-13 18:06:09.542824 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.543113 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.375 2014-08-13 18:06:09.543128 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.543426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.375 2014-08-13 18:06:09.543437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.543727 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.375 2014-08-13 18:06:09.543741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.544045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.544039 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.375 2014-08-13 18:06:09.544356 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.544374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.375 2014-08-13 18:06:09.544667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.375 2014-08-13 18:06:09.544663 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.544961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.375 2014-08-13 18:06:09.544968 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.545265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.375 2014-08-13 18:06:09.545278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.545573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.375 2014-08-13 18:06:09.545582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.545866 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.375 2014-08-13 18:06:09.545886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.546164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.375 2014-08-13 18:06:09.546194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.546478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.375 2014-08-13 18:06:09.546499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.546783 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.375 2014-08-13 18:06:09.546804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.547088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.375 2014-08-13 18:06:09.547108 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.547416 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.547392 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.375 2014-08-13 18:06:09.547724 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.547743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.375 2014-08-13 18:06:09.548031 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.548165 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.375 2014-08-13 18:06:09.548342 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.548496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.375 2014-08-13 18:06:09.548650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.548790 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} 2014-08-13 18:06:09.548956 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.549156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound 2014-08-13 18:06:09.549270 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.549456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.549576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.549775 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound: built 10,286,301,375 2014-08-13 18:06:09.550068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Activate Finished 2014-08-13 18:06:09.549883 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.367 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.550371 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.550422 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 18:06:09.550681 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.550807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 378 as a source 2014-08-13 18:06:09.550973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.551376 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.551115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.551424 7f5c7bde3700 20 osd.4 194497 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 10 2014-08-13 18:06:09.551439 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:09.551521 7f5c7bde3700 20 osd.4 194497 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 375 2014-08-13 18:06:09.551596 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing 2014-08-13 18:06:09.551430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.551737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 2014-08-13 18:06:09.551779 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.551788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Peering advmap 2014-08-13 18:06:09.551808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.551831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Started advmap 2014-08-13 18:06:09.551868 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.551896 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map 2014-08-13 18:06:09.551916 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle ActMap primary 2014-08-13 18:06:09.551936 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194497:29710 2014-08-13 18:06:09.551954 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] take_waiters 2014-08-13 18:06:09.551973 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 1.207095 3 0.000660 2014-08-13 18:06:09.552001 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Leaving Peering 2014-08-13 18:06:09.552024 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 1.336617 0 0.000000 2014-08-13 18:06:09.551738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.552050 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active 2014-08-13 18:06:09.552068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: In Active, about to call activate 2014-08-13 18:06:09.552098 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.552115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - snap_trimq [] 2014-08-13 18:06:09.552132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - not complete, missing(57) 2014-08-13 18:06:09.551987 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.552044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.552329 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] flushed 2014-08-13 18:06:09.552354 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.552658 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.552965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.553276 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.553584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.553890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.554202 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.554490 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - starting recovery 2014-08-13 18:06:09.554527 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] 2014-08-13 18:06:09.554584 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.554611 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 is up to date, queueing in pending_activators 2014-08-13 18:06:09.554696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.554724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.554753 7f5c7bde3700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 2014-08-13 18:06:09.554509 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.554817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.555280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 sending log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 18:06:09.555317 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing 2014-08-13 18:06:09.555352 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.555380 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.555402 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 is up to date, queueing in pending_activators 2014-08-13 18:06:09.555123 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.555483 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.555522 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.555549 7f5c7bde3700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 2014-08-13 18:06:09.555433 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.556000 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 sending log((192803'16165,194422'18731], crt=194422'18731) 2014-08-13 18:06:09.556032 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:09.555740 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.556059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.556141 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery primary has 57 missing 2014-08-13 18:06:09.556044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.556282 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.4 2014-08-13 18:06:09.556416 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.4 2014-08-13 18:06:09.556353 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.556557 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.4 2014-08-13 18:06:09.556678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.4 2014-08-13 18:06:09.556800 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.4 2014-08-13 18:06:09.556660 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.556921 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.4 2014-08-13 18:06:09.557043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.4 2014-08-13 18:06:09.556967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.557179 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.4 2014-08-13 18:06:09.557299 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.4 2014-08-13 18:06:09.557425 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.4 2014-08-13 18:06:09.557278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.557573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.4 2014-08-13 18:06:09.557697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.4 2014-08-13 18:06:09.557583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.557818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.4 2014-08-13 18:06:09.557940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.4 2014-08-13 18:06:09.557889 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.558062 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.4 2014-08-13 18:06:09.558204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.4 2014-08-13 18:06:09.558332 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.4 2014-08-13 18:06:09.558200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.558468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.4 2014-08-13 18:06:09.558607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.4 2014-08-13 18:06:09.558508 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.558729 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.4 2014-08-13 18:06:09.558848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.4 2014-08-13 18:06:09.558969 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.4 2014-08-13 18:06:09.558815 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.559090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.4 2014-08-13 18:06:09.559222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.4 2014-08-13 18:06:09.559119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.559345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.4 2014-08-13 18:06:09.559481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.4 2014-08-13 18:06:09.559430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.559736 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.4 2014-08-13 18:06:09.559735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.559938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.4 2014-08-13 18:06:09.560059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.4 2014-08-13 18:06:09.560189 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.4 2014-08-13 18:06:09.560041 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.560319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.4 2014-08-13 18:06:09.560352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.560533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.4 2014-08-13 18:06:09.560758 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.4 2014-08-13 18:06:09.560659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.560881 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.4 2014-08-13 18:06:09.561003 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.4 2014-08-13 18:06:09.561124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.4 2014-08-13 18:06:09.560966 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.561257 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.4 2014-08-13 18:06:09.561384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.4 2014-08-13 18:06:09.561275 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.561532 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.4 2014-08-13 18:06:09.561655 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.4 2014-08-13 18:06:09.561582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.561797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.4 2014-08-13 18:06:09.561952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.4 2014-08-13 18:06:09.561888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.562097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.4 2014-08-13 18:06:09.562228 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.4 2014-08-13 18:06:09.562354 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.4 2014-08-13 18:06:09.562199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.562495 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.4 2014-08-13 18:06:09.562618 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.4 2014-08-13 18:06:09.562504 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.562754 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.4 2014-08-13 18:06:09.562893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.4 2014-08-13 18:06:09.562811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.563016 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.4 2014-08-13 18:06:09.563135 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.4 2014-08-13 18:06:09.563276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.4 2014-08-13 18:06:09.563117 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.563414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.4 2014-08-13 18:06:09.563545 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.4 2014-08-13 18:06:09.563426 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.563666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.4 2014-08-13 18:06:09.563787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.4 2014-08-13 18:06:09.563908 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.4 2014-08-13 18:06:09.563734 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.564026 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 18:06:09.564229 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] activate: adding 20 as a source 2014-08-13 18:06:09.564038 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.564358 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.20 2014-08-13 18:06:09.564500 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=56] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.20 2014-08-13 18:06:09.564362 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.564654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=55] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.20 2014-08-13 18:06:09.564788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=54] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.20 2014-08-13 18:06:09.564669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.564920 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=53] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.20 2014-08-13 18:06:09.564976 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.565056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=52] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.20 2014-08-13 18:06:09.565287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.565492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=51] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.20 2014-08-13 18:06:09.565639 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=50] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.20 2014-08-13 18:06:09.565594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.565784 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=49] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.20 2014-08-13 18:06:09.565929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=48] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.20 2014-08-13 18:06:09.565900 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.566078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=47] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.20 2014-08-13 18:06:09.566240 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=46] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.20 2014-08-13 18:06:09.566211 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.566403 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=45] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.20 2014-08-13 18:06:09.566583 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=44] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.20 2014-08-13 18:06:09.566518 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.566750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=43] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.20 2014-08-13 18:06:09.566919 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=42] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.20 2014-08-13 18:06:09.566824 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.567089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=41] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.20 2014-08-13 18:06:09.567130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.567270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=40] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.20 2014-08-13 18:06:09.567452 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=39] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.20 2014-08-13 18:06:09.567442 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.567648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=38] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.20 2014-08-13 18:06:09.567831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=37] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.20 2014-08-13 18:06:09.567749 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.568015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=36] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.20 2014-08-13 18:06:09.568056 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.568237 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=35] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.20 2014-08-13 18:06:09.568450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=34] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.20 2014-08-13 18:06:09.568366 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.568658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=33] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.20 2014-08-13 18:06:09.568671 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.568856 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=32] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.20 2014-08-13 18:06:09.569056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=31] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.20 2014-08-13 18:06:09.568983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.378 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.569269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=30] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.20 2014-08-13 18:06:09.569292 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} 2014-08-13 18:06:09.569501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=29] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.20 2014-08-13 18:06:09.569710 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=28] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.20 2014-08-13 18:06:09.569672 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound 2014-08-13 18:06:09.569921 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=27] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.20 2014-08-13 18:06:09.569977 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.570137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=26] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.20 2014-08-13 18:06:09.570290 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 291,350,365,367,378 2014-08-13 18:06:09.570363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=25] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.20 2014-08-13 18:06:09.570616 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=24] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.20 2014-08-13 18:06:09.570598 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished 2014-08-13 18:06:09.570843 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=23] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.20 2014-08-13 18:06:09.570906 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.571071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=22] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.20 2014-08-13 18:06:09.571218 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.571316 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=21] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.20 2014-08-13 18:06:09.571578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=20] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.20 2014-08-13 18:06:09.571525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.571920 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.571986 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 2014-08-13 18:06:09.572014 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.572032 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.572038 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.571816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=19] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.20 2014-08-13 18:06:09.572041 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.572069 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572072 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.572075 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a381c0 pg_notify(22.481(49) epoch 194497) v5 2014-08-13 18:06:09.572079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.572095 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.572112 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.572125 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.381 2014-08-13 18:06:09.572130 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.572129 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a381c0 2014-08-13 18:06:09.572154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 2014-08-13 18:06:09.572177 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.572174 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572188 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572183 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.572208 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.572227 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572230 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572232 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b9c0 pg_notify(22.3af(36) epoch 194497) v5 2014-08-13 18:06:09.572227 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,312) went down 2014-08-13 18:06:09.572247 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.572272 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 2014-08-13 18:06:09.572275 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b9c0 2014-08-13 18:06:09.572263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.572056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=18] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.20 2014-08-13 18:06:09.572349 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572353 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572371 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572374 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572376 7f5c835ef700 20 osd.4 194497 _dispatch 0x58881c0 pg_notify(24.30b(45) epoch 194497) v5 2014-08-13 18:06:09.572414 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.375 2014-08-13 18:06:09.572417 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x58881c0 2014-08-13 18:06:09.572459 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572462 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572501 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572504 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572506 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6af40 pg_notify(22.22c(1) epoch 194497) v5 2014-08-13 18:06:09.572542 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.302 2014-08-13 18:06:09.572545 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6af40 2014-08-13 18:06:09.572576 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.572589 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572593 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572591 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.572613 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572616 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=17] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.20 2014-08-13 18:06:09.572618 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df39c0 pg_info(1 pgs e194497:24.e2(2)) v4 2014-08-13 18:06:09.572655 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.e2(2)) v4 from osd.346 2014-08-13 18:06:09.572660 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df39c0 2014-08-13 18:06:09.572698 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572700 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572718 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572721 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572723 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5b00 pg_notify(22.1d2(30) epoch 194497) v5 2014-08-13 18:06:09.572760 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.330 2014-08-13 18:06:09.572763 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5b00 2014-08-13 18:06:09.572750 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2866 2014-08-13 18:06:09.572768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.572784 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.572809 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572812 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572800 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.572819 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.572831 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.572833 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.572835 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d080 pg_notify(24.9b(2) epoch 194497) v5 2014-08-13 18:06:09.572842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 2014-08-13 18:06:09.572865 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.572873 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.365 2014-08-13 18:06:09.572876 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d080 2014-08-13 18:06:09.572539 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.572872 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.572622 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=16] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.20 2014-08-13 18:06:09.572890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.572910 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.572926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.572940 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.572955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.572972 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.572980 7f5c7c7e4700 20 osd.4 194497 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 291 2014-08-13 18:06:09.572992 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing 2014-08-13 18:06:09.573033 7f5c7c7e4700 20 osd.4 194497 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 350 2014-08-13 18:06:09.573045 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing 2014-08-13 18:06:09.573141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 2014-08-13 18:06:09.573166 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.573173 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.572887 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] flushed 2014-08-13 18:06:09.572903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=15] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.20 2014-08-13 18:06:09.573198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.573217 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.573233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.573248 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.573263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.573281 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.573307 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573311 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 2014-08-13 18:06:09.573326 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.573336 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573338 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573341 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150d80 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.573332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.573351 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.573356 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.573359 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573361 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.573369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.573383 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573386 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573388 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a39880 pg_notify(22.39d(1) epoch 194497) v5 2014-08-13 18:06:09.573387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.573404 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.573434 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.366 2014-08-13 18:06:09.573437 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a39880 2014-08-13 18:06:09.573476 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573479 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573499 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573501 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573503 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356e40 pg_notify(22.481(47) epoch 194497) v5 2014-08-13 18:06:09.573218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=14] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.20 2014-08-13 18:06:09.573539 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.306 2014-08-13 18:06:09.573542 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356e40 2014-08-13 18:06:09.573577 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573580 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573622 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573624 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573626 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6aa00 pg_notify(22.775(36) epoch 194497) v5 2014-08-13 18:06:09.573662 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.63 2014-08-13 18:06:09.573665 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6aa00 2014-08-13 18:06:09.573709 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573713 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573731 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573735 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69180 pg_notify(22.362(34) epoch 194497) v5 2014-08-13 18:06:09.573737 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.573752 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.573773 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.349 2014-08-13 18:06:09.573776 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69180 2014-08-13 18:06:09.573818 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573821 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573839 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573843 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573846 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d940 pg_notify(22.3d9(54) epoch 194497) v5 2014-08-13 18:06:09.573543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=13] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.20 2014-08-13 18:06:09.573883 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.382 2014-08-13 18:06:09.573886 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d940 2014-08-13 18:06:09.573922 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573925 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573923 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2711 2014-08-13 18:06:09.573945 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.573947 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573949 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df1880 pg_notify(24.34(41) epoch 194497) v5 2014-08-13 18:06:09.573940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.573957 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.573974 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.573987 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.26 2014-08-13 18:06:09.573990 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df1880 2014-08-13 18:06:09.573997 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.574019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 2014-08-13 18:06:09.574043 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574047 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574049 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.574066 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574069 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574055 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:09.574071 7f5c835ef700 20 osd.4 194497 _dispatch 0x5889500 pg_notify(24.181(45) epoch 194497) v5 2014-08-13 18:06:09.574074 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.574091 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:09.574384 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.307 2014-08-13 18:06:09.574388 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5889500 2014-08-13 18:06:09.574425 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574429 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.573861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=12] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.20 2014-08-13 18:06:09.574445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:09.574478 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574481 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574468 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.574484 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68fc0 pg_notify(24.30b(41) epoch 194497) v5 2014-08-13 18:06:09.574486 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.574506 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.574522 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.10 2014-08-13 18:06:09.574525 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68fc0 2014-08-13 18:06:09.574532 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 2014-08-13 18:06:09.574555 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.574561 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574565 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574561 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.574590 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574579 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.574593 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574596 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f380 pg_notify(24.30b(2) epoch 194497) v5 2014-08-13 18:06:09.574596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.574613 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.574634 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.301 2014-08-13 18:06:09.574637 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f380 2014-08-13 18:06:09.574638 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.574655 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.574674 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.574681 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574685 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574703 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574705 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574707 7f5c835ef700 20 osd.4 194497 _dispatch 0x5353b80 pg_notify(22.1d2(30) epoch 194497) v5 2014-08-13 18:06:09.574695 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 2014-08-13 18:06:09.574717 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.574723 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.574745 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.58 2014-08-13 18:06:09.574748 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5353b80 2014-08-13 18:06:09.574741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.574759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.574466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=11] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.20 2014-08-13 18:06:09.574784 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574788 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574777 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.574795 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.574819 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574822 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574824 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888540 pg_notify(22.1f1(34) epoch 194497) v5 2014-08-13 18:06:09.574859 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.54 2014-08-13 18:06:09.574862 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888540 2014-08-13 18:06:09.574898 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574901 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574933 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.574936 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.574938 7f5c835ef700 20 osd.4 194497 _dispatch 0x588e900 pg_notify(24.3d(2),22.99(30) epoch 194497) v5 2014-08-13 18:06:09.574977 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.20 2014-08-13 18:06:09.574980 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588e900 2014-08-13 18:06:09.575003 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.575018 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.574777 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=10] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.20 2014-08-13 18:06:09.575125 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2575 2014-08-13 18:06:09.575150 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.575166 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.575181 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.575208 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.575231 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 2014-08-13 18:06:09.575259 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.575265 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Started advmap 2014-08-13 18:06:09.575282 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.575299 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map 2014-08-13 18:06:09.575410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters 2014-08-13 18:06:09.575430 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.575449 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.575466 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.575135 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=9] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.20 2014-08-13 18:06:09.575490 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 2014-08-13 18:06:09.575511 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.575517 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.575534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.575551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.575568 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.575583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.575471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=8] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.20 2014-08-13 18:06:09.575949 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.575963 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.575778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=7] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.20 2014-08-13 18:06:09.576147 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:3012 2014-08-13 18:06:09.576163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.576178 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.576202 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.576220 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.576243 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 2014-08-13 18:06:09.576264 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.576271 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.576287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.576304 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.576319 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.576055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=6] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.20 2014-08-13 18:06:09.576333 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.576349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.576366 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.576389 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 2014-08-13 18:06:09.576418 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.576425 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap 2014-08-13 18:06:09.576444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.576462 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap 2014-08-13 18:06:09.576480 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.576499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map 2014-08-13 18:06:09.576517 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary 2014-08-13 18:06:09.576535 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194497:22354 2014-08-13 18:06:09.576552 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters 2014-08-13 18:06:09.576571 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 1.230056 4 0.000661 2014-08-13 18:06:09.576593 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering 2014-08-13 18:06:09.576610 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.335370 0 0.000000 2014-08-13 18:06:09.576340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=5] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.20 2014-08-13 18:06:09.576648 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active 2014-08-13 18:06:09.576667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate 2014-08-13 18:06:09.576692 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.576709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] 2014-08-13 18:06:09.576725 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) 2014-08-13 18:06:09.576666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=4] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.20 2014-08-13 18:06:09.576995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=3] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.20 2014-08-13 18:06:09.577336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=2] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.20 2014-08-13 18:06:09.577805 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery 2014-08-13 18:06:09.577831 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] 2014-08-13 18:06:09.577866 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.577888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 is up to date, queueing in pending_activators 2014-08-13 18:06:09.577681 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=1] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.20 2014-08-13 18:06:09.577974 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.577999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.578027 7f5c7c7e4700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 2014-08-13 18:06:09.578329 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 sending log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 18:06:09.578359 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing 2014-08-13 18:06:09.577975 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 18:06:09.578392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.578420 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.578443 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 is up to date, queueing in pending_activators 2014-08-13 18:06:09.578514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.578537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.578563 7f5c7c7e4700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 2014-08-13 18:06:09.578391 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 25 as a source 2014-08-13 18:06:09.578921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 sending log((192834'13289,194422'15454], crt=194422'15454) 2014-08-13 18:06:09.578954 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:09.578983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.578746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.579056 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing 2014-08-13 18:06:09.579194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.4 2014-08-13 18:06:09.579106 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.579323 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.4 2014-08-13 18:06:09.579446 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.4 2014-08-13 18:06:09.579572 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.4 2014-08-13 18:06:09.579439 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.579696 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.4 2014-08-13 18:06:09.579822 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.4 2014-08-13 18:06:09.579946 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.4 2014-08-13 18:06:09.579771 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.580071 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.4 2014-08-13 18:06:09.580201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.4 2014-08-13 18:06:09.580327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.4 2014-08-13 18:06:09.580123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.580452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.4 2014-08-13 18:06:09.580578 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.4 2014-08-13 18:06:09.580456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.580703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.4 2014-08-13 18:06:09.580827 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.4 2014-08-13 18:06:09.580952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.4 2014-08-13 18:06:09.580764 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.581077 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.4 2014-08-13 18:06:09.581204 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.4 2014-08-13 18:06:09.581330 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.4 2014-08-13 18:06:09.581455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.4 2014-08-13 18:06:09.581189 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.581579 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.4 2014-08-13 18:06:09.581704 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.4 2014-08-13 18:06:09.581609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.581829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.4 2014-08-13 18:06:09.581951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.4 2014-08-13 18:06:09.582075 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.4 2014-08-13 18:06:09.581926 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.582207 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.4 2014-08-13 18:06:09.582332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.4 2014-08-13 18:06:09.582230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.582484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.4 2014-08-13 18:06:09.582610 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.4 2014-08-13 18:06:09.582556 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.582735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.4 2014-08-13 18:06:09.582857 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.4 2014-08-13 18:06:09.582983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.4 2014-08-13 18:06:09.582854 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.583107 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.4 2014-08-13 18:06:09.583238 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.4 2014-08-13 18:06:09.583163 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.583363 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.4 2014-08-13 18:06:09.583486 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.4 2014-08-13 18:06:09.583610 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.4 2014-08-13 18:06:09.583471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.583735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.4 2014-08-13 18:06:09.583859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.4 2014-08-13 18:06:09.583808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.583984 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.4 2014-08-13 18:06:09.584121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.4 2014-08-13 18:06:09.584255 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.4 2014-08-13 18:06:09.584112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.584380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.4 2014-08-13 18:06:09.584505 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.4 2014-08-13 18:06:09.584421 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.584630 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.4 2014-08-13 18:06:09.584752 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.4 2014-08-13 18:06:09.584876 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.4 2014-08-13 18:06:09.584737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.585001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.4 2014-08-13 18:06:09.585126 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.4 2014-08-13 18:06:09.585035 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.585256 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.4 2014-08-13 18:06:09.585381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.4 2014-08-13 18:06:09.585506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.4 2014-08-13 18:06:09.585339 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.585631 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.4 2014-08-13 18:06:09.585755 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.4 2014-08-13 18:06:09.585663 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.585879 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.4 2014-08-13 18:06:09.586002 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.4 2014-08-13 18:06:09.586127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.4 2014-08-13 18:06:09.585991 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.586256 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.4 2014-08-13 18:06:09.586381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.4 2014-08-13 18:06:09.586306 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.586506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.4 2014-08-13 18:06:09.586627 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 18:06:09.586619 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.586842 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 17 as a source 2014-08-13 18:06:09.586967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.17 2014-08-13 18:06:09.586917 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.587093 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.17 2014-08-13 18:06:09.587229 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.17 2014-08-13 18:06:09.587366 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.17 2014-08-13 18:06:09.587221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.587501 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.17 2014-08-13 18:06:09.587642 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.17 2014-08-13 18:06:09.587572 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.587784 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.17 2014-08-13 18:06:09.587928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.17 2014-08-13 18:06:09.587891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.588076 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.17 2014-08-13 18:06:09.588233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.17 2014-08-13 18:06:09.588196 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.588389 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.17 2014-08-13 18:06:09.588548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.17 2014-08-13 18:06:09.588508 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.588709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.17 2014-08-13 18:06:09.588898 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.17 2014-08-13 18:06:09.588813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.589067 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.17 2014-08-13 18:06:09.589244 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.17 2014-08-13 18:06:09.589109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.589417 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.17 2014-08-13 18:06:09.589434 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.589594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.17 2014-08-13 18:06:09.589771 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.17 2014-08-13 18:06:09.589733 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.589954 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.17 2014-08-13 18:06:09.590139 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.17 2014-08-13 18:06:09.590029 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.590332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.17 2014-08-13 18:06:09.590347 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.590524 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.17 2014-08-13 18:06:09.590720 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.17 2014-08-13 18:06:09.590676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.590918 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.17 2014-08-13 18:06:09.590975 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.591119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.17 2014-08-13 18:06:09.591328 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.17 2014-08-13 18:06:09.591279 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.591537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.17 2014-08-13 18:06:09.591744 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.17 2014-08-13 18:06:09.591711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.591954 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.17 2014-08-13 18:06:09.592166 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.17 2014-08-13 18:06:09.592063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.592387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.17 2014-08-13 18:06:09.592387 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.592611 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.17 2014-08-13 18:06:09.592715 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.592835 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.17 2014-08-13 18:06:09.593061 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.17 2014-08-13 18:06:09.593013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.593296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.17 2014-08-13 18:06:09.593318 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.593530 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.17 2014-08-13 18:06:09.593632 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.593766 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.17 2014-08-13 18:06:09.593929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.594003 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.17 2014-08-13 18:06:09.594251 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.17 2014-08-13 18:06:09.594230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.594498 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.17 2014-08-13 18:06:09.594578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.594747 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.17 2014-08-13 18:06:09.594877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.594999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.17 2014-08-13 18:06:09.595180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.595259 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.17 2014-08-13 18:06:09.595515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.17 2014-08-13 18:06:09.595505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.595776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.17 2014-08-13 18:06:09.595801 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.596040 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.17 2014-08-13 18:06:09.596096 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.596313 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.17 2014-08-13 18:06:09.596417 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.596584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.17 2014-08-13 18:06:09.596716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.25 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.596858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.17 2014-08-13 18:06:09.597012 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 18:06:09.597134 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.17 2014-08-13 18:06:09.597430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.17 2014-08-13 18:06:09.597423 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 303 as a source 2014-08-13 18:06:09.597717 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.17 2014-08-13 18:06:09.597734 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.303 2014-08-13 18:06:09.598005 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.17 2014-08-13 18:06:09.598060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.303 2014-08-13 18:06:09.598301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.17 2014-08-13 18:06:09.598373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.303 2014-08-13 18:06:09.598596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.17 2014-08-13 18:06:09.598686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.303 2014-08-13 18:06:09.598893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.17 2014-08-13 18:06:09.598985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.303 2014-08-13 18:06:09.599197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.17 2014-08-13 18:06:09.599289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.303 2014-08-13 18:06:09.599607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.303 2014-08-13 18:06:09.599499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.17 2014-08-13 18:06:09.599906 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.303 2014-08-13 18:06:09.599981 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 18:06:09.600211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.303 2014-08-13 18:06:09.600384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 28 as a source 2014-08-13 18:06:09.600529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.303 2014-08-13 18:06:09.600693 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.600827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.303 2014-08-13 18:06:09.601002 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.601125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.303 2014-08-13 18:06:09.601314 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.601489 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.303 2014-08-13 18:06:09.601622 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.601803 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.303 2014-08-13 18:06:09.601929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.602101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.303 2014-08-13 18:06:09.602242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.602411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.303 2014-08-13 18:06:09.602549 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.602728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.303 2014-08-13 18:06:09.602856 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.603097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.303 2014-08-13 18:06:09.603163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.603415 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.303 2014-08-13 18:06:09.603475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.603728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.303 2014-08-13 18:06:09.603783 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.604042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.303 2014-08-13 18:06:09.604090 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.604372 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.303 2014-08-13 18:06:09.604410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.604715 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.604711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.303 2014-08-13 18:06:09.605022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.605031 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.303 2014-08-13 18:06:09.605335 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.605345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.303 2014-08-13 18:06:09.605643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.605657 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.303 2014-08-13 18:06:09.605950 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.605955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.303 2014-08-13 18:06:09.606263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.606293 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.303 2014-08-13 18:06:09.606571 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.606614 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.303 2014-08-13 18:06:09.606878 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.606961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.303 2014-08-13 18:06:09.607194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.607269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.303 2014-08-13 18:06:09.607502 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.607592 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.303 2014-08-13 18:06:09.607809 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.607918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.303 2014-08-13 18:06:09.608115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.608249 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.303 2014-08-13 18:06:09.608426 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.608567 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.303 2014-08-13 18:06:09.608733 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.608865 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.303 2014-08-13 18:06:09.609040 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.609172 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.303 2014-08-13 18:06:09.609352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.609491 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.303 2014-08-13 18:06:09.609660 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.609791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.303 2014-08-13 18:06:09.609967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.610088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.303 2014-08-13 18:06:09.610278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.610399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.303 2014-08-13 18:06:09.610585 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.610717 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.303 2014-08-13 18:06:09.610892 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.611015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.303 2014-08-13 18:06:09.611203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.611319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.303 2014-08-13 18:06:09.611510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.611648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.303 2014-08-13 18:06:09.611817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.611966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.303 2014-08-13 18:06:09.612124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.612272 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.303 2014-08-13 18:06:09.612435 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.612578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.303 2014-08-13 18:06:09.612743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.612875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.303 2014-08-13 18:06:09.613050 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.613178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.303 2014-08-13 18:06:09.613361 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.613496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.303 2014-08-13 18:06:09.613669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.613795 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.303 2014-08-13 18:06:09.613975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.614105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.303 2014-08-13 18:06:09.614286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.614427 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.303 2014-08-13 18:06:09.614594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.614744 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.303 2014-08-13 18:06:09.614903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.615043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.303 2014-08-13 18:06:09.615231 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.615368 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.303 2014-08-13 18:06:09.615540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.615686 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 18:06:09.615847 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 362 as a source 2014-08-13 18:06:09.616154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616359 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616467 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616688 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616774 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.616985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617699 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.617904 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618233 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618317 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618553 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.28 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618850 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.618928 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 18:06:09.619167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.619311 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 315 as a source 2014-08-13 18:06:09.619514 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.619619 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.315 2014-08-13 18:06:09.619811 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.619928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.315 2014-08-13 18:06:09.620107 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.620242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.315 2014-08-13 18:06:09.620420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.620550 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.315 2014-08-13 18:06:09.620749 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.620858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.315 2014-08-13 18:06:09.621047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.621167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.315 2014-08-13 18:06:09.621353 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.621478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.315 2014-08-13 18:06:09.621666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.621786 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.315 2014-08-13 18:06:09.621962 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.622095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.315 2014-08-13 18:06:09.622264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.622407 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.315 2014-08-13 18:06:09.622579 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.622714 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.315 2014-08-13 18:06:09.622875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.623021 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.315 2014-08-13 18:06:09.623177 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.623334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.315 2014-08-13 18:06:09.623493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.623642 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.315 2014-08-13 18:06:09.623790 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.623949 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.315 2014-08-13 18:06:09.624086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.624262 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.315 2014-08-13 18:06:09.624393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.624570 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.315 2014-08-13 18:06:09.624706 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.624878 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.315 2014-08-13 18:06:09.625017 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.625190 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.315 2014-08-13 18:06:09.625329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.625499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.315 2014-08-13 18:06:09.625654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.625807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.315 2014-08-13 18:06:09.625960 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.626115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.315 2014-08-13 18:06:09.626266 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.626429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.315 2014-08-13 18:06:09.626580 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.626737 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.315 2014-08-13 18:06:09.626878 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.627045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.315 2014-08-13 18:06:09.627188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.627356 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.315 2014-08-13 18:06:09.627516 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.627664 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.315 2014-08-13 18:06:09.627814 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.627972 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.315 2014-08-13 18:06:09.628110 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.628284 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.315 2014-08-13 18:06:09.628433 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.628592 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.315 2014-08-13 18:06:09.628730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.628900 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.315 2014-08-13 18:06:09.629033 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.629212 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.315 2014-08-13 18:06:09.629341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.629521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.315 2014-08-13 18:06:09.629658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.629831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.315 2014-08-13 18:06:09.630064 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.630139 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.315 2014-08-13 18:06:09.630413 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.630452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.315 2014-08-13 18:06:09.630712 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.630760 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.315 2014-08-13 18:06:09.631008 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.631068 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.315 2014-08-13 18:06:09.631315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.631380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.315 2014-08-13 18:06:09.631631 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.631689 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.315 2014-08-13 18:06:09.631928 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.631996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.315 2014-08-13 18:06:09.632231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.632309 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.315 2014-08-13 18:06:09.632548 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.632617 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.315 2014-08-13 18:06:09.632846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.632925 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.315 2014-08-13 18:06:09.633141 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.633238 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.315 2014-08-13 18:06:09.633473 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.633546 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.315 2014-08-13 18:06:09.633772 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.362 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.633854 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.315 2014-08-13 18:06:09.634068 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} 2014-08-13 18:06:09.634162 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.315 2014-08-13 18:06:09.634457 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound 2014-08-13 18:06:09.634472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.315 2014-08-13 18:06:09.634761 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.634778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.315 2014-08-13 18:06:09.635060 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound: built 20,25,303,362,373 2014-08-13 18:06:09.635083 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.315 2014-08-13 18:06:09.635392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.315 2014-08-13 18:06:09.635374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Activate Finished 2014-08-13 18:06:09.635701 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.635700 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.315 2014-08-13 18:06:09.636002 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.636006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.315 2014-08-13 18:06:09.636316 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.315 2014-08-13 18:06:09.636310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.636727 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.636797 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 2014-08-13 18:06:09.636831 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.636840 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.636859 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.636915 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.636920 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.636909 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.636628 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.315 2014-08-13 18:06:09.636927 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.636950 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.636953 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.636956 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357540 pg_notify(24.9b(39) epoch 194497) v5 2014-08-13 18:06:09.636945 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.636963 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.636982 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.636998 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.291 2014-08-13 18:06:09.637002 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357540 2014-08-13 18:06:09.637008 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 2014-08-13 18:06:09.637031 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.637038 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.637054 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637058 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.637078 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637081 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637083 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354b40 pg_notify(24.34(54) epoch 194497) v5 2014-08-13 18:06:09.637074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.637093 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.637119 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.30 2014-08-13 18:06:09.637109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.637123 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354b40 2014-08-13 18:06:09.637161 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637164 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637212 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637217 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888700 pg_info(1 pgs e194497:24.1e6(4)) v4 2014-08-13 18:06:09.636934 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.315 2014-08-13 18:06:09.637255 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e6(4)) v4 from osd.285 2014-08-13 18:06:09.637261 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888700 2014-08-13 18:06:09.637304 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637307 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637326 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637329 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637331 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69340 pg_notify(22.3af(1) epoch 194497) v5 2014-08-13 18:06:09.637366 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.61 2014-08-13 18:06:09.637369 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69340 2014-08-13 18:06:09.637414 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637417 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637435 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637438 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637440 7f5c835ef700 20 osd.4 194497 _dispatch 0x4deeac0 pg_info(1 pgs e194497:24.1de(2)) v4 2014-08-13 18:06:09.637429 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.637446 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.637478 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1de(2)) v4 from osd.364 2014-08-13 18:06:09.637483 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4deeac0 2014-08-13 18:06:09.637521 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.315 2014-08-13 18:06:09.637560 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637563 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637566 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf0e00 pg_info(1 pgs e194497:24.5e(4)) v4 2014-08-13 18:06:09.637604 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.5e(4)) v4 from osd.289 2014-08-13 18:06:09.637275 7f5c87c27700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.637609 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf0e00 2014-08-13 18:06:09.637633 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:3026 2014-08-13 18:06:09.637666 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637670 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637663 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.637679 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.637698 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637700 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637702 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df0380 pg_notify(24.34(54) epoch 194497) v5 2014-08-13 18:06:09.637695 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.637715 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.637738 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.340 2014-08-13 18:06:09.637741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df0380 2014-08-13 18:06:09.637748 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 2014-08-13 18:06:09.637781 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.637792 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Started advmap 2014-08-13 18:06:09.637807 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.637848 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637851 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.315 2014-08-13 18:06:09.637854 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df6900 pg_info(1 pgs e194497:24.369(4)) v4 2014-08-13 18:06:09.637842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map 2014-08-13 18:06:09.637894 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.369(4)) v4 from osd.45 2014-08-13 18:06:09.637898 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df6900 2014-08-13 18:06:09.637613 7f5c87c27700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] flushed 2014-08-13 18:06:09.637941 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters 2014-08-13 18:06:09.637964 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637968 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637962 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.637988 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.637991 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.637993 7f5c835ef700 20 osd.4 194497 _dispatch 0x524c7c0 pg_notify(24.b6(44) epoch 194497) v5 2014-08-13 18:06:09.637981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.638018 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.638030 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.17 2014-08-13 18:06:09.638033 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524c7c0 2014-08-13 18:06:09.638042 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:09.638073 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.638079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.638099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.638123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.638140 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.638168 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.638183 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.638203 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:176861 2014-08-13 18:06:09.637858 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 18:06:09.638218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.638249 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.638270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.638300 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.638323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 2014-08-13 18:06:09.638350 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.638357 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.638373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.638407 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.638427 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.638447 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.638242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 330 as a source 2014-08-13 18:06:09.638782 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.638804 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.638548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.638978 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2939 2014-08-13 18:06:09.638999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.639015 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.639031 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.639054 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.639086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 2014-08-13 18:06:09.639132 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.638852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.639138 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap 2014-08-13 18:06:09.639183 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.639206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map 2014-08-13 18:06:09.639312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters 2014-08-13 18:06:09.639349 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194497 2014-08-13 18:06:09.639372 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.639401 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.639442 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 2014-08-13 18:06:09.639156 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.639471 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.639478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Started advmap 2014-08-13 18:06:09.639505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.639533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map 2014-08-13 18:06:09.639639 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters 2014-08-13 18:06:09.639660 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.639676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.639694 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.639715 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 2014-08-13 18:06:09.639749 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.639464 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.639763 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.639780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.639797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.639813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.639827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.639853 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.639870 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:177965 2014-08-13 18:06:09.639884 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.639897 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.639916 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.639933 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.639962 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_advance_map [26,30,340]/[4] -- 26/4 2014-08-13 18:06:09.640007 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.640014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Peering advmap 2014-08-13 18:06:09.640037 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.640054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Started advmap 2014-08-13 18:06:09.639769 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.640072 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.640108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map 2014-08-13 18:06:09.640126 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle ActMap primary 2014-08-13 18:06:09.640143 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194497:23846 2014-08-13 18:06:09.640174 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] take_waiters 2014-08-13 18:06:09.640193 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 1.286172 4 0.000655 2014-08-13 18:06:09.640218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Leaving Peering 2014-08-13 18:06:09.640235 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 1.390152 0 0.000000 2014-08-13 18:06:09.640263 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active 2014-08-13 18:06:09.640280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: In Active, about to call activate 2014-08-13 18:06:09.640306 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.640323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - snap_trimq [] 2014-08-13 18:06:09.640338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - no missing, moving last_complete 194215'14514 -> 194215'14514 2014-08-13 18:06:09.640075 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.640438 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:09.640462 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 is up to date, queueing in pending_activators 2014-08-13 18:06:09.640614 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate 2014-08-13 18:06:09.640648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:09.640382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.640686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 is up to date, queueing in pending_activators 2014-08-13 18:06:09.640777 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate 2014-08-13 18:06:09.640808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) 2014-08-13 18:06:09.640847 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 is up to date, queueing in pending_activators 2014-08-13 18:06:09.640945 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate 2014-08-13 18:06:09.640686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.640977 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] needs_recovery is recovered 2014-08-13 18:06:09.640995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Activate Finished 2014-08-13 18:06:09.641019 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.641050 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.641068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.641194 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.641251 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 2014-08-13 18:06:09.641281 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.640993 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.641288 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.641307 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.641365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.641385 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.641401 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.641303 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.641681 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] _activate_committed 194497 peer_activated now 4 last_epoch_started 194487 same_interval_since 194496 2014-08-13 18:06:09.641729 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] flushed 2014-08-13 18:06:09.641743 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.641761 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.641612 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.641917 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2877 2014-08-13 18:06:09.641935 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.641951 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.641966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.641985 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.642015 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 2014-08-13 18:06:09.642036 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.642041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.642058 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.642073 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.642089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.642104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.642117 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.642133 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2913 2014-08-13 18:06:09.642188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.642204 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.641919 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.642219 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.642255 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.642279 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 2014-08-13 18:06:09.642301 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.642307 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.642323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.642341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.642363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.642378 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.642229 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.642794 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.642815 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.642536 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.643006 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2664 2014-08-13 18:06:09.643028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.643044 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.643059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.643077 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.643101 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 2014-08-13 18:06:09.643122 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.643127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.642841 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.643144 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.643178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.643195 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.643211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.643144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.643499 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.643520 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.643648 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2619 2014-08-13 18:06:09.643667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.643683 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.643699 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.643726 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.643456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.643750 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 2014-08-13 18:06:09.643773 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.643781 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.643798 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.643815 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.643831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.643846 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.643861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.643878 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.644009 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 2014-08-13 18:06:09.644046 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.643761 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.644054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap 2014-08-13 18:06:09.644073 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.644092 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map 2014-08-13 18:06:09.644171 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters 2014-08-13 18:06:09.644195 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194497 2014-08-13 18:06:09.644211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.644230 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.644254 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 2014-08-13 18:06:09.644286 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.644292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Started advmap 2014-08-13 18:06:09.644310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.644328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map 2014-08-13 18:06:09.644067 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.644436 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters 2014-08-13 18:06:09.644483 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.644501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.644526 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.644553 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 2014-08-13 18:06:09.644578 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.644585 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.644617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.644635 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.644664 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.644375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.644680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.644682 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.645015 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.645032 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.645210 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2727 2014-08-13 18:06:09.645230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.645246 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.645262 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.645280 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.644990 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.645303 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 2014-08-13 18:06:09.645330 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.645340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.645357 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.645374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.645390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.645410 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.645426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.645459 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.645498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 2014-08-13 18:06:09.645528 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.645534 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Started advmap 2014-08-13 18:06:09.645552 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.645299 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.645575 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map 2014-08-13 18:06:09.645671 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters 2014-08-13 18:06:09.645710 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.645739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.645759 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.645769 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:09.645774 7f5c7bde3700 7 osd.4 194497 do_notify osd 20 on 1 PGs 2014-08-13 18:06:09.645810 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4ab69e0 192.168.1.101:6808/22782 2014-08-13 18:06:09.645858 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.304 has 194497 2014-08-13 18:06:09.645862 7f5c7bde3700 7 osd.4 194497 do_notify osd 304 on 1 PGs 2014-08-13 18:06:09.645877 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4b2cba0 192.168.1.104:6814/65209 2014-08-13 18:06:09.645604 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.645905 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.324 has 194497 2014-08-13 18:06:09.645921 7f5c7bde3700 7 osd.4 194497 do_notify osd 324 on 1 PGs 2014-08-13 18:06:09.645936 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:09.645939 7f5c7bde3700 7 osd.4 194497 do_notify osd 330 on 1 PGs 2014-08-13 18:06:09.645965 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4f21600 192.168.1.104:6820/2308 2014-08-13 18:06:09.645994 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.346 has 194497 2014-08-13 18:06:09.646001 7f5c7bde3700 7 osd.4 194497 do_notify osd 346 on 1 PGs 2014-08-13 18:06:09.646102 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:09.646109 7f5c7bde3700 7 osd.4 194497 do_notify osd 364 on 1 PGs 2014-08-13 18:06:09.645907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.646204 7f5c7bde3700 20 osd.4 194497 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 20 2014-08-13 18:06:09.646225 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:09.646302 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 26 2014-08-13 18:06:09.646313 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing 2014-08-13 18:06:09.646360 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 30 2014-08-13 18:06:09.646369 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:09.646403 7f5c7bde3700 20 osd.4 194497 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 303 2014-08-13 18:06:09.646414 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:09.646468 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 340 2014-08-13 18:06:09.646493 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:09.646216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.646650 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a5a00 prio 127 cost 0 latency 0.157299 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] 2014-08-13 18:06:09.646743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_message: 0x20a5a00 2014-08-13 18:06:09.646767 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.646521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.646788 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] sub_op_scrub_reserve 2014-08-13 18:06:09.646858 7f5c7bde3700 20 osd.4 194497 inc_scrubs_pending 1 + 0 active >= max 1 2014-08-13 18:06:09.646894 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a5a00 finish 2014-08-13 18:06:09.646991 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_advance_map [4,344,296]/[4,296,320] -- 4/4 2014-08-13 18:06:09.647034 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.647047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Peering advmap 2014-08-13 18:06:09.647068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.647085 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Started advmap 2014-08-13 18:06:09.647105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.646831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.647124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_activate_map 2014-08-13 18:06:09.647166 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle ActMap primary 2014-08-13 18:06:09.647188 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] publish_stats_to_osd 194497:28887 2014-08-13 18:06:09.647206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] take_waiters 2014-08-13 18:06:09.647234 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/WaitUpThru 1.305349 7 0.000991 2014-08-13 18:06:09.647256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Leaving Peering 2014-08-13 18:06:09.647273 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering 1.380145 0 0.000000 2014-08-13 18:06:09.647298 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active 2014-08-13 18:06:09.647316 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: In Active, about to call activate 2014-08-13 18:06:09.647338 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.647355 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - snap_trimq [] 2014-08-13 18:06:09.647371 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - not complete, missing(47) 2014-08-13 18:06:09.647142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.647452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.647758 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.648065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.648376 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.648796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - starting recovery 2014-08-13 18:06:09.648836 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] 2014-08-13 18:06:09.648876 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:09.648899 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 is up to date, queueing in pending_activators 2014-08-13 18:06:09.648683 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.648971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate 2014-08-13 18:06:09.648998 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:09.649019 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 is up to date, queueing in pending_activators 2014-08-13 18:06:09.649139 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate 2014-08-13 18:06:09.649187 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) 2014-08-13 18:06:09.649211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 is up to date, queueing in pending_activators 2014-08-13 18:06:09.648989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.649304 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate 2014-08-13 18:06:09.649375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] needs_recovery primary has 47 missing 2014-08-13 18:06:09.649302 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.649497 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 also missing on osd.4 2014-08-13 18:06:09.649625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 also missing on osd.4 2014-08-13 18:06:09.649731 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 also missing on osd.4 2014-08-13 18:06:09.649609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.649863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 also missing on osd.4 2014-08-13 18:06:09.649968 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 also missing on osd.4 2014-08-13 18:06:09.650087 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 also missing on osd.4 2014-08-13 18:06:09.649916 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.650203 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 also missing on osd.4 2014-08-13 18:06:09.650312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 also missing on osd.4 2014-08-13 18:06:09.650224 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.650420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 also missing on osd.4 2014-08-13 18:06:09.650543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 also missing on osd.4 2014-08-13 18:06:09.650647 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 also missing on osd.4 2014-08-13 18:06:09.650531 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.650750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 also missing on osd.4 2014-08-13 18:06:09.650851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 also missing on osd.4 2014-08-13 18:06:09.650973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 also missing on osd.4 2014-08-13 18:06:09.650837 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.651084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 also missing on osd.4 2014-08-13 18:06:09.651201 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 also missing on osd.4 2014-08-13 18:06:09.651305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 also missing on osd.4 2014-08-13 18:06:09.651143 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.651412 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 also missing on osd.4 2014-08-13 18:06:09.651529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 also missing on osd.4 2014-08-13 18:06:09.651649 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 also missing on osd.4 2014-08-13 18:06:09.651455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.651752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 also missing on osd.4 2014-08-13 18:06:09.651856 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 also missing on osd.4 2014-08-13 18:06:09.651959 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 also missing on osd.4 2014-08-13 18:06:09.651759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.652061 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 also missing on osd.4 2014-08-13 18:06:09.652173 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 also missing on osd.4 2014-08-13 18:06:09.652063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.652276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 also missing on osd.4 2014-08-13 18:06:09.652381 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 also missing on osd.4 2014-08-13 18:06:09.652493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 also missing on osd.4 2014-08-13 18:06:09.652376 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.652615 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 also missing on osd.4 2014-08-13 18:06:09.652716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 also missing on osd.4 2014-08-13 18:06:09.652819 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 also missing on osd.4 2014-08-13 18:06:09.652691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.652922 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 also missing on osd.4 2014-08-13 18:06:09.653022 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 also missing on osd.4 2014-08-13 18:06:09.653124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 also missing on osd.4 2014-08-13 18:06:09.652998 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.653234 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 also missing on osd.4 2014-08-13 18:06:09.653335 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 also missing on osd.4 2014-08-13 18:06:09.653442 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 also missing on osd.4 2014-08-13 18:06:09.653310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.653571 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 also missing on osd.4 2014-08-13 18:06:09.653676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 also missing on osd.4 2014-08-13 18:06:09.653778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 also missing on osd.4 2014-08-13 18:06:09.653617 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.653881 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 also missing on osd.4 2014-08-13 18:06:09.653984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 also missing on osd.4 2014-08-13 18:06:09.654086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 also missing on osd.4 2014-08-13 18:06:09.653924 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.654196 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 also missing on osd.4 2014-08-13 18:06:09.654300 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 also missing on osd.4 2014-08-13 18:06:09.654406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 also missing on osd.4 2014-08-13 18:06:09.654235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.654527 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 also missing on osd.4 2014-08-13 18:06:09.654641 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} 2014-08-13 18:06:09.654540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.654801 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] activate: adding 296 as a source 2014-08-13 18:06:09.654902 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 is on osd.296 2014-08-13 18:06:09.655006 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=46] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 is on osd.296 2014-08-13 18:06:09.654847 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.655115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=45] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 is on osd.296 2014-08-13 18:06:09.655241 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=44] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 is on osd.296 2014-08-13 18:06:09.655154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.655355 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=43] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 is on osd.296 2014-08-13 18:06:09.655483 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=42] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 is on osd.296 2014-08-13 18:06:09.655617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=41] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 is on osd.296 2014-08-13 18:06:09.655466 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.655757 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=40] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 is on osd.296 2014-08-13 18:06:09.655885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=39] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 is on osd.296 2014-08-13 18:06:09.655770 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.656013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=38] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 is on osd.296 2014-08-13 18:06:09.656154 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=37] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 is on osd.296 2014-08-13 18:06:09.656077 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.656292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=36] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 is on osd.296 2014-08-13 18:06:09.656434 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=35] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 is on osd.296 2014-08-13 18:06:09.656388 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.330 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.656593 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=34] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 is on osd.296 2014-08-13 18:06:09.656739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=33] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 is on osd.296 2014-08-13 18:06:09.656902 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=32] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 is on osd.296 2014-08-13 18:06:09.656695 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} 2014-08-13 18:06:09.657067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=31] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 is on osd.296 2014-08-13 18:06:09.657073 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound 2014-08-13 18:06:09.657235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=30] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 is on osd.296 2014-08-13 18:06:09.657393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=29] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 is on osd.296 2014-08-13 18:06:09.657384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.657578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=28] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 is on osd.296 2014-08-13 18:06:09.657741 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=27] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 is on osd.296 2014-08-13 18:06:09.657693 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 17,28,315,327,330 2014-08-13 18:06:09.657906 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=26] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 is on osd.296 2014-08-13 18:06:09.658074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=25] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 is on osd.296 2014-08-13 18:06:09.658001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished 2014-08-13 18:06:09.658252 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=24] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 is on osd.296 2014-08-13 18:06:09.658313 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.658432 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=23] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 is on osd.296 2014-08-13 18:06:09.658631 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=22] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 is on osd.296 2014-08-13 18:06:09.658621 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.658813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=21] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 is on osd.296 2014-08-13 18:06:09.658997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=20] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 is on osd.296 2014-08-13 18:06:09.658929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.659335 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.659190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=19] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 is on osd.296 2014-08-13 18:06:09.659411 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [4,47,378]/[19,65] -- 4/19 2014-08-13 18:06:09.659449 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.659457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap 2014-08-13 18:06:09.659478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.659500 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659505 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659497 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map 2014-08-13 18:06:09.659516 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters 2014-08-13 18:06:09.659536 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659539 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659541 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee1d40 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.659533 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.659551 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.659557 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.659560 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659562 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.659572 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.659583 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659586 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659597 7f5c835ef700 20 osd.4 194497 _dispatch 0x524c980 pg_info(1 pgs e194497:24.350(4)) v4 2014-08-13 18:06:09.659384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=18] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 is on osd.296 2014-08-13 18:06:09.659596 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 2014-08-13 18:06:09.659619 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.659625 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.659642 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.350(4)) v4 from osd.356 2014-08-13 18:06:09.659648 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524c980 2014-08-13 18:06:09.659644 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.659662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.659679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.659693 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.659705 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659709 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659708 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.659726 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.659731 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.659734 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.659736 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b640 pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 2014-08-13 18:06:09.659749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 2014-08-13 18:06:09.659775 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.659778 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.55 2014-08-13 18:06:09.659782 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b640 2014-08-13 18:06:09.659782 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Started advmap 2014-08-13 18:06:09.659616 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=17] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 is on osd.296 2014-08-13 18:06:09.659800 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.659817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map 2014-08-13 18:06:09.659895 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters 2014-08-13 18:06:09.659912 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.659928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.659945 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.659969 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 2014-08-13 18:06:09.659997 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.659815 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=16] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 is on osd.296 2014-08-13 18:06:09.660004 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Peering advmap 2014-08-13 18:06:09.660024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.660044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Started advmap 2014-08-13 18:06:09.660063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.660081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map 2014-08-13 18:06:09.660099 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: handle ActMap primary 2014-08-13 18:06:09.660118 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194497:25642 2014-08-13 18:06:09.660135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] take_waiters 2014-08-13 18:06:09.660153 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 1.317787 3 0.000655 2014-08-13 18:06:09.660175 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Leaving Peering 2014-08-13 18:06:09.660200 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 1.408671 0 0.000000 2014-08-13 18:06:09.660013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=15] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 is on osd.296 2014-08-13 18:06:09.660224 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active 2014-08-13 18:06:09.660243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: In Active, about to call activate 2014-08-13 18:06:09.660265 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.659928 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.660282 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - snap_trimq [] 2014-08-13 18:06:09.660300 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - not complete, missing(62) 2014-08-13 18:06:09.660221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=14] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 is on osd.296 2014-08-13 18:06:09.660290 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] flushed 2014-08-13 18:06:09.660433 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=13] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 is on osd.296 2014-08-13 18:06:09.660686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=12] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 is on osd.296 2014-08-13 18:06:09.660915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=11] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 is on osd.296 2014-08-13 18:06:09.661180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=10] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 is on osd.296 2014-08-13 18:06:09.661405 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=9] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 is on osd.296 2014-08-13 18:06:09.661754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - starting recovery 2014-08-13 18:06:09.661785 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] 2014-08-13 18:06:09.661626 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=8] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 is on osd.296 2014-08-13 18:06:09.661828 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.661851 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 is up to date, queueing in pending_activators 2014-08-13 18:06:09.661926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.661953 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.661983 7f5c7c7e4700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 2014-08-13 18:06:09.661850 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=7] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 is on osd.296 2014-08-13 18:06:09.662079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=6] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 is on osd.296 2014-08-13 18:06:09.662329 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 sending log((192718'12872,194422'15456], crt=194422'15456) 2014-08-13 18:06:09.662364 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3b00c60 don't know epoch, doing nothing 2014-08-13 18:06:09.662394 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.662422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.662444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 is up to date, queueing in pending_activators 2014-08-13 18:06:09.662535 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.662319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=5] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 is on osd.296 2014-08-13 18:06:09.662618 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery primary has 62 missing 2014-08-13 18:06:09.662569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=4] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 is on osd.296 2014-08-13 18:06:09.662754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.4 2014-08-13 18:06:09.662887 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.4 2014-08-13 18:06:09.662823 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=3] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 is on osd.296 2014-08-13 18:06:09.663021 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.4 2014-08-13 18:06:09.663152 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.4 2014-08-13 18:06:09.663078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=2] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 is on osd.296 2014-08-13 18:06:09.663290 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.4 2014-08-13 18:06:09.663421 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.4 2014-08-13 18:06:09.663328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=1] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 is on osd.296 2014-08-13 18:06:09.663551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.4 2014-08-13 18:06:09.663681 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.4 2014-08-13 18:06:09.663610 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} 2014-08-13 18:06:09.663811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.4 2014-08-13 18:06:09.663939 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.4 2014-08-13 18:06:09.663931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate: adding 320 as a source 2014-08-13 18:06:09.664068 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.4 2014-08-13 18:06:09.664200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.4 2014-08-13 18:06:09.664328 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.4 2014-08-13 18:06:09.664190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 is on osd.320 2014-08-13 18:06:09.664457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.4 2014-08-13 18:06:09.664587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.4 2014-08-13 18:06:09.664458 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 is on osd.320 2014-08-13 18:06:09.664716 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.4 2014-08-13 18:06:09.664728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 is on osd.320 2014-08-13 18:06:09.664846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.4 2014-08-13 18:06:09.664989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.4 2014-08-13 18:06:09.664981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 is on osd.320 2014-08-13 18:06:09.665121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.4 2014-08-13 18:06:09.665253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.4 2014-08-13 18:06:09.665384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.4 2014-08-13 18:06:09.665240 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 is on osd.320 2014-08-13 18:06:09.665514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.4 2014-08-13 18:06:09.665652 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.4 2014-08-13 18:06:09.665783 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.4 2014-08-13 18:06:09.665648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 is on osd.320 2014-08-13 18:06:09.665913 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.4 2014-08-13 18:06:09.665915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 is on osd.320 2014-08-13 18:06:09.666045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.4 2014-08-13 18:06:09.666172 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.4 2014-08-13 18:06:09.666172 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 is on osd.320 2014-08-13 18:06:09.666310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.4 2014-08-13 18:06:09.666438 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.4 2014-08-13 18:06:09.666568 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.4 2014-08-13 18:06:09.666426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 is on osd.320 2014-08-13 18:06:09.666699 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.4 2014-08-13 18:06:09.666701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 is on osd.320 2014-08-13 18:06:09.666830 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.4 2014-08-13 18:06:09.666957 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.4 2014-08-13 18:06:09.666951 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 is on osd.320 2014-08-13 18:06:09.667087 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.4 2014-08-13 18:06:09.667220 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.4 2014-08-13 18:06:09.667207 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 is on osd.320 2014-08-13 18:06:09.667350 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.4 2014-08-13 18:06:09.667478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.4 2014-08-13 18:06:09.667477 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 is on osd.320 2014-08-13 18:06:09.667609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.4 2014-08-13 18:06:09.667736 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.4 2014-08-13 18:06:09.667726 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 is on osd.320 2014-08-13 18:06:09.667864 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.4 2014-08-13 18:06:09.667991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.4 2014-08-13 18:06:09.667973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 is on osd.320 2014-08-13 18:06:09.668120 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.4 2014-08-13 18:06:09.668253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.4 2014-08-13 18:06:09.668227 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 is on osd.320 2014-08-13 18:06:09.668383 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.4 2014-08-13 18:06:09.668510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.4 2014-08-13 18:06:09.668482 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 is on osd.320 2014-08-13 18:06:09.668638 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.4 2014-08-13 18:06:09.668765 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.4 2014-08-13 18:06:09.668743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 is on osd.320 2014-08-13 18:06:09.668894 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.4 2014-08-13 18:06:09.669024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.4 2014-08-13 18:06:09.668991 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 is on osd.320 2014-08-13 18:06:09.669154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.4 2014-08-13 18:06:09.669289 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.4 2014-08-13 18:06:09.669243 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 is on osd.320 2014-08-13 18:06:09.669419 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.4 2014-08-13 18:06:09.669548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.4 2014-08-13 18:06:09.669506 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 is on osd.320 2014-08-13 18:06:09.669677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.4 2014-08-13 18:06:09.669804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.4 2014-08-13 18:06:09.669780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 is on osd.320 2014-08-13 18:06:09.669933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.4 2014-08-13 18:06:09.670063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.4 2014-08-13 18:06:09.670026 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 is on osd.320 2014-08-13 18:06:09.670197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.4 2014-08-13 18:06:09.670327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.4 2014-08-13 18:06:09.670280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 is on osd.320 2014-08-13 18:06:09.670456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.4 2014-08-13 18:06:09.670583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.4 2014-08-13 18:06:09.670712 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.4 2014-08-13 18:06:09.670557 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 is on osd.320 2014-08-13 18:06:09.670840 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 18:06:09.671042 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] activate: adding 55 as a source 2014-08-13 18:06:09.670907 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 is on osd.320 2014-08-13 18:06:09.671169 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.55 2014-08-13 18:06:09.671304 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=61] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.55 2014-08-13 18:06:09.671231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 is on osd.320 2014-08-13 18:06:09.671441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=60] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.55 2014-08-13 18:06:09.671581 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=59] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.55 2014-08-13 18:06:09.671504 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 is on osd.320 2014-08-13 18:06:09.671721 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=58] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.55 2014-08-13 18:06:09.671867 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=57] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.55 2014-08-13 18:06:09.671786 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 is on osd.320 2014-08-13 18:06:09.672016 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=56] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.55 2014-08-13 18:06:09.672168 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=55] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.55 2014-08-13 18:06:09.672063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 is on osd.320 2014-08-13 18:06:09.672327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=54] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.55 2014-08-13 18:06:09.672345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 is on osd.320 2014-08-13 18:06:09.672484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=53] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.55 2014-08-13 18:06:09.672643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=52] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.55 2014-08-13 18:06:09.672598 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 is on osd.320 2014-08-13 18:06:09.672819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=51] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.55 2014-08-13 18:06:09.672842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 is on osd.320 2014-08-13 18:06:09.672986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=50] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.55 2014-08-13 18:06:09.673156 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=49] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.55 2014-08-13 18:06:09.673087 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 is on osd.320 2014-08-13 18:06:09.673334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=48] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.55 2014-08-13 18:06:09.673341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 is on osd.320 2014-08-13 18:06:09.673512 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=47] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.55 2014-08-13 18:06:09.673609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 is on osd.320 2014-08-13 18:06:09.673691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=46] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.55 2014-08-13 18:06:09.673870 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=45] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.55 2014-08-13 18:06:09.673857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 is on osd.320 2014-08-13 18:06:09.674054 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=44] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.55 2014-08-13 18:06:09.674103 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 is on osd.320 2014-08-13 18:06:09.674247 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=43] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.55 2014-08-13 18:06:09.674360 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 is on osd.320 2014-08-13 18:06:09.674435 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=42] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.55 2014-08-13 18:06:09.674626 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=41] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.55 2014-08-13 18:06:09.674625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 is on osd.320 2014-08-13 18:06:09.674823 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=40] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.55 2014-08-13 18:06:09.674884 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 is on osd.320 2014-08-13 18:06:09.675022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=39] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.55 2014-08-13 18:06:09.675133 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 is on osd.320 2014-08-13 18:06:09.675230 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=38] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.55 2014-08-13 18:06:09.675437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=37] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.55 2014-08-13 18:06:09.675391 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 is on osd.320 2014-08-13 18:06:09.675646 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=36] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.55 2014-08-13 18:06:09.675653 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 is on osd.320 2014-08-13 18:06:09.675858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=35] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.55 2014-08-13 18:06:09.675901 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 is on osd.320 2014-08-13 18:06:09.676073 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=34] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.55 2014-08-13 18:06:09.676297 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=33] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.55 2014-08-13 18:06:09.676230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 is on osd.320 2014-08-13 18:06:09.676518 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=32] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.55 2014-08-13 18:06:09.676530 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 is on osd.320 2014-08-13 18:06:09.676743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=31] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.55 2014-08-13 18:06:09.676792 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} 2014-08-13 18:06:09.676973 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=30] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.55 2014-08-13 18:06:09.677096 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate: adding 344 as a source 2014-08-13 18:06:09.677210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=29] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.55 2014-08-13 18:06:09.677349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.677444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=28] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.55 2014-08-13 18:06:09.677629 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.677681 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=27] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.55 2014-08-13 18:06:09.677887 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.677922 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=26] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.55 2014-08-13 18:06:09.678137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.678163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=25] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.55 2014-08-13 18:06:09.678394 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.678411 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=24] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.55 2014-08-13 18:06:09.678656 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.678659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=23] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.55 2014-08-13 18:06:09.678903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.678909 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=22] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.55 2014-08-13 18:06:09.679156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.679162 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=21] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.55 2014-08-13 18:06:09.679406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.679421 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=20] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.55 2014-08-13 18:06:09.679668 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.679679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=19] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.55 2014-08-13 18:06:09.679915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.679940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=18] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.55 2014-08-13 18:06:09.680167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.680210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=17] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.55 2014-08-13 18:06:09.680416 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.680480 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=16] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.55 2014-08-13 18:06:09.680678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.680753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=15] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.55 2014-08-13 18:06:09.680924 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.681028 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=14] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.55 2014-08-13 18:06:09.681176 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.681317 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=13] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.55 2014-08-13 18:06:09.681435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.681600 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=12] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.55 2014-08-13 18:06:09.681689 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.681885 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=11] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.55 2014-08-13 18:06:09.681934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.682182 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.682174 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=10] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.55 2014-08-13 18:06:09.682442 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.682469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=9] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.55 2014-08-13 18:06:09.682696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.682763 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=8] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.55 2014-08-13 18:06:09.682941 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.683063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=7] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.55 2014-08-13 18:06:09.683242 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.683369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=6] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.55 2014-08-13 18:06:09.683517 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.683674 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=5] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.55 2014-08-13 18:06:09.683782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.683981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=4] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.55 2014-08-13 18:06:09.684043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.684295 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.684296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=3] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.55 2014-08-13 18:06:09.684561 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.684611 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=2] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.55 2014-08-13 18:06:09.684808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.684926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=1] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.55 2014-08-13 18:06:09.685053 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.685309 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.685250 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 18:06:09.685570 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.685649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 59 as a source 2014-08-13 18:06:09.685817 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.685971 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686062 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686298 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686622 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686620 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686880 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.686942 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687126 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687268 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687383 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687590 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.687908 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688234 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688392 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688554 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688898 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.688874 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.689142 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 also missing on osd.344 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.689199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.689409 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} 2014-08-13 18:06:09.689521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.689729 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] build_might_have_unfound 2014-08-13 18:06:09.689840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.689973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] _calc_past_interval_range: already have past intervals back to 194466 2014-08-13 18:06:09.690161 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.690231 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] build_might_have_unfound: built 55,296,307,320,344 2014-08-13 18:06:09.690502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Activate Finished 2014-08-13 18:06:09.690487 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.690759 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.690808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.691006 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.691131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.691258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.691622 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.691695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,61] -- 15/15 2014-08-13 18:06:09.691732 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.691740 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Started advmap 2014-08-13 18:06:09.691457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.691760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.691774 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.691779 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.691779 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 18:06:09.691810 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.691813 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.691815 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38700 pg_notify(24.1bb(40) epoch 194497) v5 2014-08-13 18:06:09.691857 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.337 2014-08-13 18:06:09.691861 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38700 2014-08-13 18:06:09.691891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 18:06:09.691911 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.691915 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.691916 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.691936 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.691939 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.691941 7f5c835ef700 20 osd.4 194497 _dispatch 0x524e900 pg_info(1 pgs e194497:24.20e(2)) v4 2014-08-13 18:06:09.691934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.691962 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.691980 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.20e(2)) v4 from osd.15 2014-08-13 18:06:09.691985 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524e900 2014-08-13 18:06:09.691988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,4] -- 31/31 2014-08-13 18:06:09.692021 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.692028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Started advmap 2014-08-13 18:06:09.692044 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692048 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692045 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.692067 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692070 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692072 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fe00 pg_info(1 pgs e194497:24.3d) v4 2014-08-13 18:06:09.692063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map 2014-08-13 18:06:09.691777 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.692107 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.362 2014-08-13 18:06:09.692112 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fe00 2014-08-13 18:06:09.692148 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692151 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692170 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692173 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692175 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4800 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.692164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] take_waiters 2014-08-13 18:06:09.692196 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.692202 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.692205 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692207 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692193 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.692210 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.692227 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692230 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692232 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38a80 pg_notify(24.3d(38) epoch 194497) v5 2014-08-13 18:06:09.692234 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.692269 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 2014-08-13 18:06:09.692273 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38a80 2014-08-13 18:06:09.692260 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 2014-08-13 18:06:09.692288 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.692294 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 18:06:09.692312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.692328 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692332 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 18:06:09.692351 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692353 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692355 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68c40 pg_notify(24.181(17) epoch 194497) v5 2014-08-13 18:06:09.692391 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.320 2014-08-13 18:06:09.692393 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68c40 2014-08-13 18:06:09.692394 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 18:06:09.692097 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.692419 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.692437 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.692454 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.692478 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 2014-08-13 18:06:09.692501 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.692507 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.692541 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.692569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.692311 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.692586 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.692613 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.692630 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.692661 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.692685 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 2014-08-13 18:06:09.692708 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.692714 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.692422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.692731 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.692755 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down 2014-08-13 18:06:09.692773 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.692788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.692606 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] flushed 2014-08-13 18:06:09.692965 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.692970 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.692998 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693000 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693002 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df3800 pg_info(1 pgs e194497:24.9b) v4 2014-08-13 18:06:09.693041 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.367 2014-08-13 18:06:09.693046 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df3800 2014-08-13 18:06:09.692741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.693093 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693096 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693086 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.693118 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693121 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693123 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf1880 pg_notify(22.3d9(47) epoch 194497) v5 2014-08-13 18:06:09.693118 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.693160 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.373 2014-08-13 18:06:09.693164 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf1880 2014-08-13 18:06:09.693247 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693250 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693276 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693279 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693280 7f5c835ef700 20 osd.4 194497 _dispatch 0x59e2c00 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.693284 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2689 2014-08-13 18:06:09.693308 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.693345 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 2014-08-13 18:06:09.693325 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.693353 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x59e2c00 2014-08-13 18:06:09.693358 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.397 192.168.1.105:6808/41085 194497 2014-08-13 18:06:09.693366 7f5c835ef700 10 osd.4 194497 note_peer_epoch osd.397 has 194497 2014-08-13 18:06:09.693356 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.693376 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.693372 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a7ad0 prio 127 cost 0 latency 0.205123 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.693059 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.693384 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693388 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693407 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693410 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693412 7f5c835ef700 20 osd.4 194497 _dispatch 0x588bd40 pg_notify(22.48e(34) epoch 194497) v5 2014-08-13 18:06:09.693401 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 2014-08-13 18:06:09.693429 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.693456 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.376 2014-08-13 18:06:09.693459 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588bd40 2014-08-13 18:06:09.693435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.693465 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.693484 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,287) went down 2014-08-13 18:06:09.693512 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693516 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693507 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.693536 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693538 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693540 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352680 pg_notify(24.b6(2) epoch 194497) v5 2014-08-13 18:06:09.693524 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.693578 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.327 2014-08-13 18:06:09.693581 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352680 2014-08-13 18:06:09.693617 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693619 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693637 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693642 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6c00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.693650 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.693655 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.693658 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693660 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693679 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693682 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693684 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c7c0 pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 2014-08-13 18:06:09.693383 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.693725 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.342 2014-08-13 18:06:09.693728 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c7c0 2014-08-13 18:06:09.693793 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693814 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693817 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693819 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f8c0 pg_notify(22.22c(28) epoch 194497) v5 2014-08-13 18:06:09.693853 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.368 2014-08-13 18:06:09.693856 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f8c0 2014-08-13 18:06:09.693909 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693913 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693930 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.693933 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693935 7f5c835ef700 20 osd.4 194497 _dispatch 0x588bf00 pg_info(1 pgs e194497:24.9b) v4 2014-08-13 18:06:09.693932 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.693953 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.693971 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.350 2014-08-13 18:06:09.693976 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588bf00 2014-08-13 18:06:09.694010 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694013 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.693703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.694031 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694034 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694036 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e3c0 pg_notify(22.f3(26) epoch 194497) v5 2014-08-13 18:06:09.694071 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 2014-08-13 18:06:09.694073 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e3c0 2014-08-13 18:06:09.694110 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694113 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694129 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694134 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356040 pg_info(1 pgs e194497:24.30b) v4 2014-08-13 18:06:09.694167 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.375 2014-08-13 18:06:09.694172 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356040 2014-08-13 18:06:09.694168 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2688 2014-08-13 18:06:09.694188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.694216 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694219 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694209 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.694239 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694242 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694244 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df4600 pg_info(1 pgs e194497:24.3f7(2)) v4 2014-08-13 18:06:09.694236 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.694263 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.694282 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3f7(2)) v4 from osd.330 2014-08-13 18:06:09.694287 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df4600 2014-08-13 18:06:09.694288 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 2014-08-13 18:06:09.694322 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.694323 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694327 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.694023 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.694345 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694349 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694352 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6cec0 pg_notify(22.481(42) epoch 194497) v5 2014-08-13 18:06:09.694345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.694371 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,289) went down 2014-08-13 18:06:09.694390 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.365 2014-08-13 18:06:09.694394 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6cec0 2014-08-13 18:06:09.694390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.694411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.694444 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694448 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694465 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694468 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694470 7f5c835ef700 20 osd.4 194497 _dispatch 0x4a43100 pg_info(1 pgs e194497:24.9b) v4 2014-08-13 18:06:09.694506 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.378 2014-08-13 18:06:09.694510 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4a43100 2014-08-13 18:06:09.694544 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694547 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694565 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694568 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694569 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_info(1 pgs e194497:24.34) v4 2014-08-13 18:06:09.694603 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.26 2014-08-13 18:06:09.694607 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 2014-08-13 18:06:09.694657 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694661 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.694680 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694684 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694686 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5240 pg_info(1 pgs e194497:24.30b) v4 2014-08-13 18:06:09.694720 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.10 2014-08-13 18:06:09.694724 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5240 2014-08-13 18:06:09.694756 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694759 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694776 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694779 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694781 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc3600 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.694788 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.694792 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.694795 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694797 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694814 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694816 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694818 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888fc0 pg_info(1 pgs e194497:24.3d) v4 2014-08-13 18:06:09.694852 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.20 2014-08-13 18:06:09.694857 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888fc0 2014-08-13 18:06:09.694889 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694891 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694908 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694911 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694914 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d5c0 pg_info(1 pgs e194497:24.9b) v4 2014-08-13 18:06:09.694898 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.694927 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.694949 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.291 2014-08-13 18:06:09.694953 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d5c0 2014-08-13 18:06:09.694983 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.694668 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.694986 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695004 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695006 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695008 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df3d40 pg_info(1 pgs e194497:24.34) v4 2014-08-13 18:06:09.695041 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.30 2014-08-13 18:06:09.695045 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df3d40 2014-08-13 18:06:09.695078 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695080 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695098 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695102 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888380 pg_notify(22.362(1) epoch 194497) v5 2014-08-13 18:06:09.695137 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.61 2014-08-13 18:06:09.695140 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888380 2014-08-13 18:06:09.695189 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695193 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695182 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2694 2014-08-13 18:06:09.695212 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695214 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695216 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df2140 pg_info(1 pgs e194497:24.34) v4 2014-08-13 18:06:09.695206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.695223 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.695252 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.340 2014-08-13 18:06:09.695256 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df2140 2014-08-13 18:06:09.695239 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.695275 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.695290 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.694988 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.695312 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695315 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695318 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352840 pg_notify(24.b6(44) epoch 194497) v5 2014-08-13 18:06:09.695305 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 2014-08-13 18:06:09.695330 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.695337 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.695355 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.315 2014-08-13 18:06:09.695358 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352840 2014-08-13 18:06:09.695366 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.695392 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695395 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695385 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.695406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.695431 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695421 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.695435 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695437 7f5c835ef700 20 osd.4 194497 _dispatch 0x524cec0 pg_info(1 pgs e194497:24.3b7(2)) v4 2014-08-13 18:06:09.695436 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.695470 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.695475 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3b7(2)) v4 from osd.356 2014-08-13 18:06:09.695479 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524cec0 2014-08-13 18:06:09.695498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 2014-08-13 18:06:09.695520 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695527 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.695541 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695544 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Started advmap 2014-08-13 18:06:09.695547 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68540 pg_info(1 pgs e194497:24.30b) v4 2014-08-13 18:06:09.695551 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.695583 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.286 2014-08-13 18:06:09.695587 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68540 2014-08-13 18:06:09.695586 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map 2014-08-13 18:06:09.695622 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695625 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.695647 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695649 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695651 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6db00 pg_notify(22.481(42) epoch 194497) v5 2014-08-13 18:06:09.695686 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 2014-08-13 18:06:09.695689 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6db00 2014-08-13 18:06:09.695701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters 2014-08-13 18:06:09.695730 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695726 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.695743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.695755 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695766 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.695768 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695770 7f5c835ef700 20 osd.4 194497 _dispatch 0x536ee00 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.695808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 2014-08-13 18:06:09.695828 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 2014-08-13 18:06:09.695836 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x536ee00 2014-08-13 18:06:09.695840 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.695840 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.367 192.168.1.114:6840/55513 194497 2014-08-13 18:06:09.695849 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a7e90 prio 127 cost 0 latency 0.047648 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.695848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Peering advmap 2014-08-13 18:06:09.695868 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695872 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695872 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.695890 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695893 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695896 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6bf00 pg_info(1 pgs e194497:22.3b9) v4 2014-08-13 18:06:09.695891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Started advmap 2014-08-13 18:06:09.695931 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:22.3b9) v4 from osd.397 2014-08-13 18:06:09.695936 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6bf00 2014-08-13 18:06:09.695929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.695633 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.695953 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map 2014-08-13 18:06:09.695973 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695977 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695975 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: handle ActMap primary 2014-08-13 18:06:09.695995 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.695997 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.695999 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351c00 pg_info(1 pgs e194497:24.3d) v4 2014-08-13 18:06:09.695994 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194497:22155 2014-08-13 18:06:09.696035 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.25 2014-08-13 18:06:09.696039 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351c00 2014-08-13 18:06:09.696021 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] take_waiters 2014-08-13 18:06:09.696058 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 1.336094 3 0.000640 2014-08-13 18:06:09.696082 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Leaving Peering 2014-08-13 18:06:09.696104 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 1.387143 0 0.000000 2014-08-13 18:06:09.696121 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696124 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696147 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696149 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696151 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e200 pg_notify(22.775(1) epoch 194497) v5 2014-08-13 18:06:09.696130 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active 2014-08-13 18:06:09.696175 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: In Active, about to call activate 2014-08-13 18:06:09.696196 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 2014-08-13 18:06:09.696199 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e200 2014-08-13 18:06:09.696203 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.696222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - snap_trimq [] 2014-08-13 18:06:09.696236 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696240 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696258 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696261 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696263 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f380 pg_info(1 pgs e194497:24.b6) v4 2014-08-13 18:06:09.696239 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - not complete, missing(48) 2014-08-13 18:06:09.695951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.696302 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.330 2014-08-13 18:06:09.696306 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f380 2014-08-13 18:06:09.696339 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696342 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696359 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696361 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696363 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf1dc0 pg_info(1 pgs e194497:24.b6) v4 2014-08-13 18:06:09.696396 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.28 2014-08-13 18:06:09.696400 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf1dc0 2014-08-13 18:06:09.696432 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696435 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696454 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696457 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696458 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d940 pg_notify(22.3e4(1) epoch 194497) v5 2014-08-13 18:06:09.696494 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 2014-08-13 18:06:09.696497 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d940 2014-08-13 18:06:09.696549 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696553 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696574 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696577 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696579 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df0a80 pg_info(1 pgs e194497:24.3d) v4 2014-08-13 18:06:09.696279 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.696614 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.303 2014-08-13 18:06:09.696618 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df0a80 2014-08-13 18:06:09.696650 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.696653 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.696599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.696917 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.697239 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.697557 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.698111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - starting recovery 2014-08-13 18:06:09.698153 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] 2014-08-13 18:06:09.697875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.698217 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.698253 7f5c7bde3700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 2014-08-13 18:06:09.698197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.698697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 sending log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 18:06:09.698731 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing 2014-08-13 18:06:09.698782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.698515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.698809 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.698838 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 is up to date, queueing in pending_activators 2014-08-13 18:06:09.698930 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.698956 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) 2014-08-13 18:06:09.698982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 is up to date, queueing in pending_activators 2014-08-13 18:06:09.699050 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.699079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.699431 7f5c7bde3700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 2014-08-13 18:06:09.698832 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.699442 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.699644 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 sending log((192718'9695,194422'12517], crt=194422'12517) 2014-08-13 18:06:09.699667 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x571b700 don't know epoch, doing nothing 2014-08-13 18:06:09.699687 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate 2014-08-13 18:06:09.699732 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery primary has 48 missing 2014-08-13 18:06:09.699626 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.699800 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.4 2014-08-13 18:06:09.699876 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.4 2014-08-13 18:06:09.699808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.699947 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.4 2014-08-13 18:06:09.700012 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.4 2014-08-13 18:06:09.700071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.4 2014-08-13 18:06:09.699991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.700136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.4 2014-08-13 18:06:09.700202 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.4 2014-08-13 18:06:09.700267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.4 2014-08-13 18:06:09.700175 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.700331 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.4 2014-08-13 18:06:09.700393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.4 2014-08-13 18:06:09.700466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.4 2014-08-13 18:06:09.700360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.700532 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.4 2014-08-13 18:06:09.700607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.4 2014-08-13 18:06:09.700542 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.700678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.4 2014-08-13 18:06:09.700748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.4 2014-08-13 18:06:09.700810 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.4 2014-08-13 18:06:09.700727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.700873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.4 2014-08-13 18:06:09.700933 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.4 2014-08-13 18:06:09.700994 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.4 2014-08-13 18:06:09.700910 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701058 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.4 2014-08-13 18:06:09.701119 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.4 2014-08-13 18:06:09.701094 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.4 2014-08-13 18:06:09.701328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.4 2014-08-13 18:06:09.701281 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701403 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.4 2014-08-13 18:06:09.701496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.4 2014-08-13 18:06:09.701578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.4 2014-08-13 18:06:09.701465 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701663 7f5c817ec700 10 osd.4 194497 note_peer_epoch osd.9 now has 194497 2014-08-13 18:06:09.701669 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4423700 already has epoch 194497 2014-08-13 18:06:09.701702 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4423700 already has epoch 194497 2014-08-13 18:06:09.701640 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.4 2014-08-13 18:06:09.701647 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.4 2014-08-13 18:06:09.701934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.4 2014-08-13 18:06:09.701832 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.701999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.4 2014-08-13 18:06:09.702060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.4 2014-08-13 18:06:09.702120 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.4 2014-08-13 18:06:09.702014 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.702191 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.4 2014-08-13 18:06:09.702256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.4 2014-08-13 18:06:09.702319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.4 2014-08-13 18:06:09.702198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.702379 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.4 2014-08-13 18:06:09.702448 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.4 2014-08-13 18:06:09.702381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.702518 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.4 2014-08-13 18:06:09.702577 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.4 2014-08-13 18:06:09.702637 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.4 2014-08-13 18:06:09.702565 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.702697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.4 2014-08-13 18:06:09.702756 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.4 2014-08-13 18:06:09.702834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.4 2014-08-13 18:06:09.702748 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.702895 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.4 2014-08-13 18:06:09.702955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.4 2014-08-13 18:06:09.703015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.4 2014-08-13 18:06:09.702932 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703076 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.4 2014-08-13 18:06:09.703136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.4 2014-08-13 18:06:09.703115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.59 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703221 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 18:06:09.703323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 331 as a source 2014-08-13 18:06:09.703395 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703463 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703319 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 18:06:09.703523 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703583 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703642 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 351 as a source 2014-08-13 18:06:09.703701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703819 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703732 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.351 2014-08-13 18:06:09.703879 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.703917 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.351 2014-08-13 18:06:09.704056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704114 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704101 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.351 2014-08-13 18:06:09.704285 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704346 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.351 2014-08-13 18:06:09.704411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704482 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704554 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.351 2014-08-13 18:06:09.704627 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704687 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704656 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.351 2014-08-13 18:06:09.704813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704935 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.704840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.351 2014-08-13 18:06:09.704994 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705128 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.351 2014-08-13 18:06:09.705193 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705255 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.351 2014-08-13 18:06:09.705384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705393 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.351 2014-08-13 18:06:09.705562 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705621 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705681 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705575 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.351 2014-08-13 18:06:09.705740 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705799 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705858 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705757 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.351 2014-08-13 18:06:09.705917 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705977 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.705941 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.351 2014-08-13 18:06:09.706101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.351 2014-08-13 18:06:09.706303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.331 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.706310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.351 2014-08-13 18:06:09.706447 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 18:06:09.706544 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 332 as a source 2014-08-13 18:06:09.706495 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.351 2014-08-13 18:06:09.706620 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.332 2014-08-13 18:06:09.706680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=47] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.332 2014-08-13 18:06:09.706745 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=46] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.332 2014-08-13 18:06:09.706679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.351 2014-08-13 18:06:09.706810 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=45] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.332 2014-08-13 18:06:09.706875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=44] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.332 2014-08-13 18:06:09.706944 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=43] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.332 2014-08-13 18:06:09.706863 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.351 2014-08-13 18:06:09.707017 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=42] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.332 2014-08-13 18:06:09.707089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=41] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.332 2014-08-13 18:06:09.707047 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.351 2014-08-13 18:06:09.707168 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=40] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.332 2014-08-13 18:06:09.707242 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=39] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.332 2014-08-13 18:06:09.707321 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=38] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.332 2014-08-13 18:06:09.707234 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.351 2014-08-13 18:06:09.707409 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=37] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.332 2014-08-13 18:06:09.707508 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=36] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.332 2014-08-13 18:06:09.707422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.351 2014-08-13 18:06:09.707591 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=35] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.332 2014-08-13 18:06:09.707604 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.351 2014-08-13 18:06:09.707699 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=34] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.332 2014-08-13 18:06:09.707796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=33] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.332 2014-08-13 18:06:09.707787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.351 2014-08-13 18:06:09.707885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=32] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.332 2014-08-13 18:06:09.707982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=31] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.332 2014-08-13 18:06:09.707971 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.351 2014-08-13 18:06:09.708074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=30] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.332 2014-08-13 18:06:09.708170 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=29] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.332 2014-08-13 18:06:09.708155 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.351 2014-08-13 18:06:09.708267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=28] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.332 2014-08-13 18:06:09.708369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=27] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.332 2014-08-13 18:06:09.708341 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.351 2014-08-13 18:06:09.708473 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=26] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.332 2014-08-13 18:06:09.708573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=25] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.332 2014-08-13 18:06:09.708525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.351 2014-08-13 18:06:09.708684 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=24] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.332 2014-08-13 18:06:09.708709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.351 2014-08-13 18:06:09.708789 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=23] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.332 2014-08-13 18:06:09.708892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=22] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.332 2014-08-13 18:06:09.708893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.351 2014-08-13 18:06:09.709001 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=21] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.332 2014-08-13 18:06:09.709109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=20] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.332 2014-08-13 18:06:09.709078 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.351 2014-08-13 18:06:09.709223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=19] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.332 2014-08-13 18:06:09.709265 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.351 2014-08-13 18:06:09.709338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=18] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.332 2014-08-13 18:06:09.709579 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.709583 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.709585 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee2400 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.709592 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.709596 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.709597 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.709599 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.709492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=17] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.332 2014-08-13 18:06:09.709612 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.709614 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.709615 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357c40 pg_info(1 pgs e194497:24.1f7) v4 2014-08-13 18:06:09.709450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.351 2014-08-13 18:06:09.709641 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.59 2014-08-13 18:06:09.709644 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357c40 2014-08-13 18:06:09.709607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=16] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.332 2014-08-13 18:06:09.709632 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.351 2014-08-13 18:06:09.709723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=15] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.332 2014-08-13 18:06:09.709842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=14] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.332 2014-08-13 18:06:09.709817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.351 2014-08-13 18:06:09.709963 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=13] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.332 2014-08-13 18:06:09.710001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.351 2014-08-13 18:06:09.710123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=12] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.332 2014-08-13 18:06:09.710187 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.351 2014-08-13 18:06:09.710303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=11] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.332 2014-08-13 18:06:09.710372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.351 2014-08-13 18:06:09.710470 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=10] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.332 2014-08-13 18:06:09.710600 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=9] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.332 2014-08-13 18:06:09.710555 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.351 2014-08-13 18:06:09.710730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=8] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.332 2014-08-13 18:06:09.710738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.351 2014-08-13 18:06:09.710861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=7] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.332 2014-08-13 18:06:09.710921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.351 2014-08-13 18:06:09.710993 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=6] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.332 2014-08-13 18:06:09.711127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=5] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.332 2014-08-13 18:06:09.711105 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.351 2014-08-13 18:06:09.711292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=4] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.332 2014-08-13 18:06:09.711296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.351 2014-08-13 18:06:09.711471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=3] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.332 2014-08-13 18:06:09.711479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.351 2014-08-13 18:06:09.711641 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=2] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.332 2014-08-13 18:06:09.711663 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.351 2014-08-13 18:06:09.711834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=1] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.332 2014-08-13 18:06:09.711846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.351 2014-08-13 18:06:09.711996 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 18:06:09.712029 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.351 2014-08-13 18:06:09.712181 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 337 as a source 2014-08-13 18:06:09.712216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.351 2014-08-13 18:06:09.712328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.337 2014-08-13 18:06:09.712401 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.351 2014-08-13 18:06:09.712491 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.337 2014-08-13 18:06:09.712585 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.351 2014-08-13 18:06:09.712638 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.337 2014-08-13 18:06:09.712782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.337 2014-08-13 18:06:09.712768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.351 2014-08-13 18:06:09.712926 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.337 2014-08-13 18:06:09.712952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.351 2014-08-13 18:06:09.713071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.337 2014-08-13 18:06:09.713135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.351 2014-08-13 18:06:09.713220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.337 2014-08-13 18:06:09.713322 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.351 2014-08-13 18:06:09.713368 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.337 2014-08-13 18:06:09.713526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.337 2014-08-13 18:06:09.713507 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.351 2014-08-13 18:06:09.713671 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.337 2014-08-13 18:06:09.713691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.351 2014-08-13 18:06:09.713816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.337 2014-08-13 18:06:09.713875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.351 2014-08-13 18:06:09.713961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.337 2014-08-13 18:06:09.714058 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.351 2014-08-13 18:06:09.714105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.337 2014-08-13 18:06:09.714252 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.337 2014-08-13 18:06:09.714245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.351 2014-08-13 18:06:09.714407 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.337 2014-08-13 18:06:09.714430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.351 2014-08-13 18:06:09.714562 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.337 2014-08-13 18:06:09.714614 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.351 2014-08-13 18:06:09.714708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.337 2014-08-13 18:06:09.714798 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.351 2014-08-13 18:06:09.714852 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.337 2014-08-13 18:06:09.714996 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.337 2014-08-13 18:06:09.714982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.351 2014-08-13 18:06:09.715140 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.337 2014-08-13 18:06:09.715165 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} 2014-08-13 18:06:09.715291 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.337 2014-08-13 18:06:09.715406 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound 2014-08-13 18:06:09.715443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.337 2014-08-13 18:06:09.715598 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.337 2014-08-13 18:06:09.715589 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.715746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.337 2014-08-13 18:06:09.715774 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound: built 55,59,351,376 2014-08-13 18:06:09.715905 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.337 2014-08-13 18:06:09.715957 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Activate Finished 2014-08-13 18:06:09.716051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.337 2014-08-13 18:06:09.716141 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.716200 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.337 2014-08-13 18:06:09.716354 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.337 2014-08-13 18:06:09.716329 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.716503 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.337 2014-08-13 18:06:09.716511 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.716748 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.716649 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.337 2014-08-13 18:06:09.716782 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 2014-08-13 18:06:09.716803 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.716803 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.716806 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.716808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Peering advmap 2014-08-13 18:06:09.716819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] adjust_need_up_thru now 194496, need_up_thru now false 2014-08-13 18:06:09.716829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Started advmap 2014-08-13 18:06:09.716840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.716852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map 2014-08-13 18:06:09.716863 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle ActMap primary 2014-08-13 18:06:09.716874 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194497:22531 2014-08-13 18:06:09.716884 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] take_waiters 2014-08-13 18:06:09.716895 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 1.375748 4 0.000737 2014-08-13 18:06:09.716907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Leaving Peering 2014-08-13 18:06:09.716917 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 1.463927 0 0.000000 2014-08-13 18:06:09.716931 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active 2014-08-13 18:06:09.716941 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: In Active, about to call activate 2014-08-13 18:06:09.716795 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.337 2014-08-13 18:06:09.716954 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - purged_snaps [] cached_removed_snaps [] 2014-08-13 18:06:09.716965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - snap_trimq [] 2014-08-13 18:06:09.716975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - not complete, missing(51) 2014-08-13 18:06:09.716957 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.337 2014-08-13 18:06:09.717104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.337 2014-08-13 18:06:09.717087 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.717254 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.337 2014-08-13 18:06:09.717308 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] flushed 2014-08-13 18:06:09.717399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.337 2014-08-13 18:06:09.717546 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.337 2014-08-13 18:06:09.717778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - starting recovery 2014-08-13 18:06:09.717796 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] 2014-08-13 18:06:09.717819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:09.717833 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 is up to date, queueing in pending_activators 2014-08-13 18:06:09.717694 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.337 2014-08-13 18:06:09.717875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate 2014-08-13 18:06:09.717890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.717907 7f5c7c7e4700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 2014-08-13 18:06:09.717855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.337 2014-08-13 18:06:09.718095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 sending log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 18:06:09.718114 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:09.718131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate 2014-08-13 18:06:09.718147 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-13 18:06:09.718014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.337 2014-08-13 18:06:09.718163 7f5c7c7e4700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 2014-08-13 18:06:09.718162 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.337 2014-08-13 18:06:09.718367 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 sending log((192834'12379,194422'15470], crt=194422'15470) 2014-08-13 18:06:09.718386 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x445b180 don't know epoch, doing nothing 2014-08-13 18:06:09.718402 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate 2014-08-13 18:06:09.718418 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) 2014-08-13 18:06:09.718430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 is up to date, queueing in pending_activators 2014-08-13 18:06:09.718321 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.337 2014-08-13 18:06:09.718479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate 2014-08-13 18:06:09.718519 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery primary has 51 missing 2014-08-13 18:06:09.718481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.337 2014-08-13 18:06:09.718586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.4 2014-08-13 18:06:09.718649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.4 2014-08-13 18:06:09.718713 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.4 2014-08-13 18:06:09.718645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.337 2014-08-13 18:06:09.718776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.4 2014-08-13 18:06:09.718840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.4 2014-08-13 18:06:09.718789 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.337 2014-08-13 18:06:09.718903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.4 2014-08-13 18:06:09.718966 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.4 2014-08-13 18:06:09.718934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.337 2014-08-13 18:06:09.719029 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.4 2014-08-13 18:06:09.719091 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.4 2014-08-13 18:06:09.719154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.4 2014-08-13 18:06:09.719078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.337 2014-08-13 18:06:09.719222 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.4 2014-08-13 18:06:09.719286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.4 2014-08-13 18:06:09.719226 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.337 2014-08-13 18:06:09.719349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.4 2014-08-13 18:06:09.719412 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.4 2014-08-13 18:06:09.719375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.337 2014-08-13 18:06:09.719475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.4 2014-08-13 18:06:09.719545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.4 2014-08-13 18:06:09.719609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.4 2014-08-13 18:06:09.719541 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 18:06:09.719673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.4 2014-08-13 18:06:09.719735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.4 2014-08-13 18:06:09.719798 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.4 2014-08-13 18:06:09.719861 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.4 2014-08-13 18:06:09.719730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 348 as a source 2014-08-13 18:06:09.719924 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.4 2014-08-13 18:06:09.719987 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.4 2014-08-13 18:06:09.720050 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.4 2014-08-13 18:06:09.719932 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720113 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.4 2014-08-13 18:06:09.720176 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.4 2014-08-13 18:06:09.720243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.4 2014-08-13 18:06:09.720136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720306 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.4 2014-08-13 18:06:09.720369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.4 2014-08-13 18:06:09.720336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.4 2014-08-13 18:06:09.720513 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.4 2014-08-13 18:06:09.720576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.4 2014-08-13 18:06:09.720499 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720639 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.4 2014-08-13 18:06:09.720702 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.4 2014-08-13 18:06:09.720658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720765 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.4 2014-08-13 18:06:09.720827 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.4 2014-08-13 18:06:09.720890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.4 2014-08-13 18:06:09.720811 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.720952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.4 2014-08-13 18:06:09.721016 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.4 2014-08-13 18:06:09.720956 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.4 2014-08-13 18:06:09.721142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.4 2014-08-13 18:06:09.721115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721209 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.4 2014-08-13 18:06:09.721271 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.4 2014-08-13 18:06:09.721334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.4 2014-08-13 18:06:09.721267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721397 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.4 2014-08-13 18:06:09.721460 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.4 2014-08-13 18:06:09.721420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721523 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.4 2014-08-13 18:06:09.721586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.4 2014-08-13 18:06:09.721564 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.4 2014-08-13 18:06:09.721714 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.4 2014-08-13 18:06:09.721778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.4 2014-08-13 18:06:09.721708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721840 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 18:06:09.721851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.721938 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] activate: adding 50 as a source 2014-08-13 18:06:09.722000 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.50 2014-08-13 18:06:09.722064 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=50] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.50 2014-08-13 18:06:09.721995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=49] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.50 2014-08-13 18:06:09.722203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=48] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.50 2014-08-13 18:06:09.722138 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722272 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=47] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.50 2014-08-13 18:06:09.722344 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=46] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.50 2014-08-13 18:06:09.722287 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722417 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=45] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.50 2014-08-13 18:06:09.722492 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=44] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.50 2014-08-13 18:06:09.722450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722569 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=43] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.50 2014-08-13 18:06:09.722647 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=42] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.50 2014-08-13 18:06:09.722604 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=41] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.50 2014-08-13 18:06:09.722809 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=40] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.50 2014-08-13 18:06:09.722750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=39] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.50 2014-08-13 18:06:09.722893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.722979 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=38] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.50 2014-08-13 18:06:09.723065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=37] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.50 2014-08-13 18:06:09.723038 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723155 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=36] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.50 2014-08-13 18:06:09.723186 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723249 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=35] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.50 2014-08-13 18:06:09.723340 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=34] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.50 2014-08-13 18:06:09.723333 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723434 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=33] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.50 2014-08-13 18:06:09.723530 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=32] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.50 2014-08-13 18:06:09.723493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723628 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=31] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.50 2014-08-13 18:06:09.723636 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=30] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.50 2014-08-13 18:06:09.723780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.723828 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=29] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.50 2014-08-13 18:06:09.723930 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=28] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.50 2014-08-13 18:06:09.723924 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=27] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.50 2014-08-13 18:06:09.724069 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=26] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.50 2014-08-13 18:06:09.724218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724270 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=25] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.50 2014-08-13 18:06:09.724379 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=24] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.50 2014-08-13 18:06:09.724373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724491 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=23] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.50 2014-08-13 18:06:09.724529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724605 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=22] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.50 2014-08-13 18:06:09.724674 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724720 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=21] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.50 2014-08-13 18:06:09.724837 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=20] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.50 2014-08-13 18:06:09.724818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.724955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=19] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.50 2014-08-13 18:06:09.724960 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725076 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=18] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.50 2014-08-13 18:06:09.725104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=17] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.50 2014-08-13 18:06:09.725255 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725325 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=16] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.50 2014-08-13 18:06:09.725418 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=15] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.50 2014-08-13 18:06:09.725563 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725581 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=14] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.50 2014-08-13 18:06:09.725710 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=13] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.50 2014-08-13 18:06:09.725708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725842 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=12] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.50 2014-08-13 18:06:09.725851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.725975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=11] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.50 2014-08-13 18:06:09.726016 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726109 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=10] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.50 2014-08-13 18:06:09.726171 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726248 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=9] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.50 2014-08-13 18:06:09.726326 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=8] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.50 2014-08-13 18:06:09.726487 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726526 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=7] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.50 2014-08-13 18:06:09.726641 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=6] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.50 2014-08-13 18:06:09.726791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=5] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.50 2014-08-13 18:06:09.726940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.726951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=4] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.50 2014-08-13 18:06:09.727084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.348 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.727096 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=3] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.50 2014-08-13 18:06:09.727245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=2] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.50 2014-08-13 18:06:09.727235 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} 2014-08-13 18:06:09.727395 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=1] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.50 2014-08-13 18:06:09.727429 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound 2014-08-13 18:06:09.727548 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 18:06:09.727604 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _calc_past_interval_range: already have past intervals back to 194493 2014-08-13 18:06:09.727737 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 55 as a source 2014-08-13 18:06:09.727761 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound: built 331,332,337,348,379 2014-08-13 18:06:09.727890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.727946 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Activate Finished 2014-08-13 18:06:09.728044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728122 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.728200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728278 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.728353 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728460 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.728506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728684 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.728725 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [396,368,4]/[396,363] -- 396/396 2014-08-13 18:06:09.728747 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.728753 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap 2014-08-13 18:06:09.728765 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.728776 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map 2014-08-13 18:06:09.728659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728804 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters 2014-08-13 18:06:09.728816 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.728827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.728842 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.728860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 2014-08-13 18:06:09.728873 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.728877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.728886 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.728896 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.728905 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.728913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.728922 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.728931 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2660 2014-08-13 18:06:09.728939 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.728947 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.728810 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.728955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.728965 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.728979 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 2014-08-13 18:06:09.728992 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.728996 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.729005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.729015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.729024 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.729032 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729040 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729051 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729067 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 2014-08-13 18:06:09.729078 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.729082 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.729091 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.729099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.728962 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.729117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.729126 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.728974 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 2014-08-13 18:06:09.729134 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:3007 2014-08-13 18:06:09.729143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.729158 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729176 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729189 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 2014-08-13 18:06:09.729202 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.729205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.729215 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.729224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.729233 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.729241 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729250 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729259 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729113 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729271 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 2014-08-13 18:06:09.729288 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.729143 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] flushed 2014-08-13 18:06:09.729292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Started advmap 2014-08-13 18:06:09.729302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.729311 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map 2014-08-13 18:06:09.729361 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters 2014-08-13 18:06:09.729371 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729379 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729390 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729268 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729408 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 2014-08-13 18:06:09.729421 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.729425 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.729435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.729445 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.729453 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.729463 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729472 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729482 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729494 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 2014-08-13 18:06:09.729506 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.729509 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap 2014-08-13 18:06:09.729529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap 2014-08-13 18:06:09.729540 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,338) went down 2014-08-13 18:06:09.729550 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map 2014-08-13 18:06:09.729420 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729559 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap 2014-08-13 18:06:09.729571 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729813 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] 2014-08-13 18:06:09.729837 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary 2014-08-13 18:06:09.729724 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.729957 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2600 2014-08-13 18:06:09.729970 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters 2014-08-13 18:06:09.729980 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.729989 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.729999 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.729875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730014 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 2014-08-13 18:06:09.730032 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.730036 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Started advmap 2014-08-13 18:06:09.730047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.730057 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map 2014-08-13 18:06:09.730120 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters 2014-08-13 18:06:09.730133 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.730143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.730160 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.730166 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4af98c0 192.168.1.107:6809/53141 2014-08-13 18:06:09.730026 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730189 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.6 has 194497 2014-08-13 18:06:09.730191 7f5c7bde3700 7 osd.4 194497 do_notify osd 6 on 1 PGs 2014-08-13 18:06:09.730200 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4854620 192.168.1.107:6812/53408 2014-08-13 18:06:09.730213 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.15 has 194497 2014-08-13 18:06:09.730215 7f5c7bde3700 7 osd.4 194497 do_notify osd 15 on 1 PGs 2014-08-13 18:06:09.730223 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4857bc0 192.168.1.103:6831/41157 2014-08-13 18:06:09.730236 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.31 has 194497 2014-08-13 18:06:09.730238 7f5c7bde3700 7 osd.4 194497 do_notify osd 31 on 1 PGs 2014-08-13 18:06:09.730294 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:09.730298 7f5c7bde3700 7 osd.4 194497 do_notify osd 285 on 1 PGs 2014-08-13 18:06:09.730177 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730333 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.730337 7f5c7bde3700 7 osd.4 194497 do_notify osd 343 on 1 PGs 2014-08-13 18:06:09.730350 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:09.730352 7f5c7bde3700 7 osd.4 194497 do_notify osd 374 on 1 PGs 2014-08-13 18:06:09.730370 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 296 2014-08-13 18:06:09.730394 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4854a40 don't know epoch, doing nothing 2014-08-13 18:06:09.730454 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 320 2014-08-13 18:06:09.730474 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:09.730333 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730497 7f5c7bde3700 20 osd.4 194497 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 332 2014-08-13 18:06:09.730503 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing 2014-08-13 18:06:09.730531 7f5c7bde3700 20 osd.4 194497 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 337 2014-08-13 18:06:09.730542 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:09.730569 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 344 2014-08-13 18:06:09.730577 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:09.730484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730699 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a7e90 prio 127 cost 0 latency 0.082497 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] 2014-08-13 18:06:09.730737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_message: 0x20a7e90 2014-08-13 18:06:09.730749 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.730764 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] sub_op_scrub_unreserve 2014-08-13 18:06:09.730789 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a7e90 finish 2014-08-13 18:06:09.730834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 2014-08-13 18:06:09.730637 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.730854 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.730866 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.730878 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.730888 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.730897 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.730922 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.730931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.730943 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.730959 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 2014-08-13 18:06:09.730981 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.730985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Started advmap 2014-08-13 18:06:09.730995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.730856 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map 2014-08-13 18:06:09.731046 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters 2014-08-13 18:06:09.731059 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.731068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.731078 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.731083 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 44 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.731097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 44, already have info from that osd, ignoring 2014-08-13 18:06:09.731107 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.731111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.731124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 18:06:09.731133 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.731137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 337 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.731011 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731155 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 18:06:09.731165 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.731169 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 371 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.731181 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 371, already have info from that osd, ignoring 2014-08-13 18:06:09.731191 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.731194 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) 2014-08-13 18:06:09.731205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: got info from osd.343 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) 2014-08-13 18:06:09.731219 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 1.423533 8 0.001105 2014-08-13 18:06:09.731232 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive 2014-08-13 18:06:09.731252 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.731264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.731273 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] activate - not complete, missing(41) 2014-08-13 18:06:09.731163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731318 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731624 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.731929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732152 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] state: Activate Finished 2014-08-13 18:06:09.732217 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.732081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 296 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:09.732489 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] _activate_committed 194497 telling primary 2014-08-13 18:06:09.732519 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.732536 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732538 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732388 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732540 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352a00 pg_info(1 pgs e194497:24.1bb) v4 2014-08-13 18:06:09.732539 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] flushed 2014-08-13 18:06:09.732567 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.332 2014-08-13 18:06:09.732570 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352a00 2014-08-13 18:06:09.732596 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732598 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 296, already have info from that osd, ignoring 2014-08-13 18:06:09.732623 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.732629 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 31 info: 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) 2014-08-13 18:06:09.732642 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: got info from osd.31 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) 2014-08-13 18:06:09.732659 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started/Stray 1.451552 8 0.000659 2014-08-13 18:06:09.732671 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive 2014-08-13 18:06:09.732540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732703 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.732716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.732726 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] activate - not complete, missing(46) 2014-08-13 18:06:09.732820 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732826 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732828 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df1880 pg_info(1 pgs e194497:24.181) v4 2014-08-13 18:06:09.732693 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.732862 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.344 2014-08-13 18:06:09.732865 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df1880 2014-08-13 18:06:09.732894 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732900 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732913 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732914 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732916 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356e40 pg_info(1 pgs e194497:24.181) v4 2014-08-13 18:06:09.732936 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.320 2014-08-13 18:06:09.732938 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356e40 2014-08-13 18:06:09.732958 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.732960 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733019 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733021 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.733022 7f5c835ef700 20 osd.4 194497 _dispatch 0x53563c0 pg_info(1 pgs e194497:24.1bb) v4 2014-08-13 18:06:09.733044 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.337 2014-08-13 18:06:09.733046 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x53563c0 2014-08-13 18:06:09.733072 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733074 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.732996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733267 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733270 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.733271 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b2c0 pg_info(1 pgs e194497:24.206) v4 2014-08-13 18:06:09.733148 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733303 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.206) v4 from osd.343 2014-08-13 18:06:09.733306 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b2c0 2014-08-13 18:06:09.733334 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733336 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.733302 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733475 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733476 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.733478 7f5c835ef700 20 osd.4 194497 _dispatch 0x5355e80 pg_info(1 pgs e194497:24.181) v4 2014-08-13 18:06:09.733478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] state: Activate Finished 2014-08-13 18:06:09.733499 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.296 2014-08-13 18:06:09.733502 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5355e80 2014-08-13 18:06:09.733523 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.733525 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.733555 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.733455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733596 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 362 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.733627 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 362, already have info from that osd, ignoring 2014-08-13 18:06:09.733647 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.733653 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) 2014-08-13 18:06:09.733664 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: got info from osd.304 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) 2014-08-13 18:06:09.733681 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 1.497738 8 0.000866 2014-08-13 18:06:09.733714 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive 2014-08-13 18:06:09.733727 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.733737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.733747 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] activate - not complete, missing(61) 2014-08-13 18:06:09.733606 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733851 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] _activate_committed 194497 telling primary 2014-08-13 18:06:09.733873 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4857bc0 already has epoch 194497 2014-08-13 18:06:09.733888 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] flushed 2014-08-13 18:06:09.733758 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.733908 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734061 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] state: Activate Finished 2014-08-13 18:06:09.734521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734714 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.734778 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.734780 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.734781 7f5c835ef700 20 osd.4 194497 _dispatch 0x588db00 pg_info(1 pgs e194497:24.1e4) v4 2014-08-13 18:06:09.734803 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e4) v4 from osd.31 2014-08-13 18:06:09.734806 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588db00 2014-08-13 18:06:09.734673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.734829 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.734831 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.734735 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 332 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.734825 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735037 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] _activate_committed 194497 telling primary 2014-08-13 18:06:09.734913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 332, already have info from that osd, ignoring 2014-08-13 18:06:09.735065 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735067 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.735068 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.735068 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4ab69e0 already has epoch 194497 2014-08-13 18:06:09.735069 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfbf00 osd_map(194497..194497 src has 185139..194497) v3 2014-08-13 18:06:09.735076 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] 2014-08-13 18:06:09.735079 7f5c835ef700 10 osd.4 194497 no new maps here, dropping 2014-08-13 18:06:09.735081 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.735098 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735100 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.735091 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] flushed 2014-08-13 18:06:09.735101 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357e00 pg_info(1 pgs e194497:24.1bb) v4 2014-08-13 18:06:09.735102 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 363 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.735124 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.348 2014-08-13 18:06:09.734978 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735127 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357e00 2014-08-13 18:06:09.735125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 18:06:09.735138 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.735143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 363 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.735153 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735155 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.735156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring 2014-08-13 18:06:09.735175 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.735179 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) 2014-08-13 18:06:09.735192 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: got info from osd.374 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) 2014-08-13 18:06:09.735206 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 1.438962 8 0.000623 2014-08-13 18:06:09.735221 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive 2014-08-13 18:06:09.735234 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.735246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.735256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] activate - not complete, missing(52) 2014-08-13 18:06:09.735129 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735283 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735434 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.55 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.735921 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735923 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.735740 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 18:06:09.735924 7f5c835ef700 20 osd.4 194497 _dispatch 0x588e040 pg_info(1 pgs e194497:24.33a) v4 2014-08-13 18:06:09.735946 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.33a) v4 from osd.304 2014-08-13 18:06:09.735949 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588e040 2014-08-13 18:06:09.735969 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.735971 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.736008 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] state: Activate Finished 2014-08-13 18:06:09.735926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 61 as a source 2014-08-13 18:06:09.736086 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.736105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 24 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.736122 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 24, already have info from that osd, ignoring 2014-08-13 18:06:09.736133 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.736137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) 2014-08-13 18:06:09.736148 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: got info from osd.6 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) 2014-08-13 18:06:09.736167 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 1.423764 9 0.000650 2014-08-13 18:06:09.736182 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive 2014-08-13 18:06:09.736193 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.736204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.736214 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] activate - not complete, missing(53) 2014-08-13 18:06:09.736079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736378 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] _activate_committed 194497 telling primary 2014-08-13 18:06:09.736402 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing 2014-08-13 18:06:09.736420 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] flushed 2014-08-13 18:06:09.736386 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736692 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736844 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.736996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737210 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.737215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.737218 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d780 pg_info(1 pgs e194497:24.2ec) v4 2014-08-13 18:06:09.737245 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2ec) v4 from osd.374 2014-08-13 18:06:09.737248 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d780 2014-08-13 18:06:09.737277 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.737279 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.737292 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.737293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.737295 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354980 pg_info(1 pgs e194497:24.1bb) v4 2014-08-13 18:06:09.737149 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737316 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.331 2014-08-13 18:06:09.737318 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354980 2014-08-13 18:06:09.737338 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.737340 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.737303 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] state: Activate Finished 2014-08-13 18:06:09.737761 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737933 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.737913 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.737950 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 344 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:09.738066 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738103 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 344, already have info from that osd, ignoring 2014-08-13 18:06:09.738232 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] _activate_committed 194497 telling primary 2014-08-13 18:06:09.738255 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738265 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4af98c0 already has epoch 194497 2014-08-13 18:06:09.738261 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 331 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.738274 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 331, already have info from that osd, ignoring 2014-08-13 18:06:09.738284 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738285 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] flushed 2014-08-13 18:06:09.738289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 320 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.738300 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 320, already have info from that osd, ignoring 2014-08-13 18:06:09.738320 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738324 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:09.738327 7f5c7bde3700 7 osd.4 194497 do_notify osd 45 on 1 PGs 2014-08-13 18:06:09.738223 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 373 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.738528 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738551 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 18:06:09.738724 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 397 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:09.738742 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 397, already have info from that osd, ignoring 2014-08-13 18:06:09.738753 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738757 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 396 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:09.738767 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 396, already have info from that osd, ignoring 2014-08-13 18:06:09.738776 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738680 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738779 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 379 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.738833 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.738929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 379, already have info from that osd, ignoring 2014-08-13 18:06:09.739074 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739079 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.739082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.739083 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69180 pg_info(1 pgs e194497:24.cd) v4 2014-08-13 18:06:09.739080 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 0 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739093 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-13 18:06:09.739103 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739114 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.cd) v4 from osd.6 2014-08-13 18:06:09.739107 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 352 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739117 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69180 2014-08-13 18:06:09.739118 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 352, already have info from that osd, ignoring 2014-08-13 18:06:09.739128 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.738986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 327 notify: (query_epoch:194497, epoch_sent:194497, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739146 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.739143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 327, already have info from that osd, ignoring 2014-08-13 18:06:09.739153 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739148 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.739157 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 339 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739174 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 339, already have info from that osd, ignoring 2014-08-13 18:06:09.739184 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 311 notify: (query_epoch:194497, epoch_sent:194497, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.739199 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 311, already have info from that osd, ignoring 2014-08-13 18:06:09.739209 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739213 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 293 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) 2014-08-13 18:06:09.739223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 293, already have info from that osd, ignoring 2014-08-13 18:06:09.739232 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739245 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:09.739255 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 314 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 314, already have info from that osd, ignoring 2014-08-13 18:06:09.739278 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739281 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 48 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739137 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739291 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring 2014-08-13 18:06:09.739301 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 380 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.739315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 380, already have info from that osd, ignoring 2014-08-13 18:06:09.739324 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739327 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.739445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739505 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops 2014-08-13 18:06:09.739674 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739598 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739679 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 350 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.739750 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.739871 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 350, already have info from that osd, ignoring 2014-08-13 18:06:09.740048 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.739903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 381 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:09.740067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 381, already have info from that osd, ignoring 2014-08-13 18:06:09.740076 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.740080 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.740090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:09.740104 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.740054 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 375 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.740210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740283 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 375, already have info from that osd, ignoring 2014-08-13 18:06:09.740451 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.740456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 302 notify: (query_epoch:194497, epoch_sent:194497, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.740468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 302, already have info from that osd, ignoring 2014-08-13 18:06:09.740478 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.740499 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) 2014-08-13 18:06:09.740363 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740512 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: got info from osd.346 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) 2014-08-13 18:06:09.740527 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 1.446295 9 0.000844 2014-08-13 18:06:09.740541 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive 2014-08-13 18:06:09.740558 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.740569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.740580 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] activate - not complete, missing(66) 2014-08-13 18:06:09.740514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740820 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.740972 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] state: Activate Finished 2014-08-13 18:06:09.741429 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.741280 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 330 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.741466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 330, already have info from that osd, ignoring 2014-08-13 18:06:09.741477 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.741432 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.741692 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] _activate_committed 194497 telling primary 2014-08-13 18:06:09.741720 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4f21600 already has epoch 194497 2014-08-13 18:06:09.741584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741740 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] flushed 2014-08-13 18:06:09.741661 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:09.741842 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.741735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.741848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 365 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.741888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742039 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742025 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 18:06:09.742206 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 366 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.742223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 366, already have info from that osd, ignoring 2014-08-13 18:06:09.742232 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742236 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 306 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:09.742246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 306, already have info from that osd, ignoring 2014-08-13 18:06:09.742255 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742259 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 63 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.742269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 63, already have info from that osd, ignoring 2014-08-13 18:06:09.742278 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742282 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 349 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.742292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 349, already have info from that osd, ignoring 2014-08-13 18:06:09.742302 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 382 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) 2014-08-13 18:06:09.742315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 382, already have info from that osd, ignoring 2014-08-13 18:06:09.742325 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 26 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:09.742194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 26, already have info from that osd, ignoring 2014-08-13 18:06:09.742354 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742359 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 307 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:09.742501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 307, already have info from that osd, ignoring 2014-08-13 18:06:09.742643 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742707 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.742712 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.742715 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df39c0 pg_info(1 pgs e194497:24.e2) v4 2014-08-13 18:06:09.742739 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.e2) v4 from osd.346 2014-08-13 18:06:09.742741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df39c0 2014-08-13 18:06:09.742767 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.742769 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.742650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742647 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 10 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.742803 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 10, already have info from that osd, ignoring 2014-08-13 18:06:09.742981 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.742965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.742985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 301 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.743118 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.743152 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 301, already have info from that osd, ignoring 2014-08-13 18:06:09.743324 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.743329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 58 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.743340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 58, already have info from that osd, ignoring 2014-08-13 18:06:09.743349 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.743274 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.743353 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 20 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.743427 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.743525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 18:06:09.743694 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.743700 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 20 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.743711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 20, already have info from that osd, ignoring 2014-08-13 18:06:09.743720 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.743580 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.743731 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.61 (past last_backfill 0//0//-1) 2014-08-13 18:06:09.743723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 291 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.743884 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 18:06:09.743900 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 291, already have info from that osd, ignoring 2014-08-13 18:06:09.744073 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.744078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 30 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:09.744090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 30, already have info from that osd, ignoring 2014-08-13 18:06:09.744100 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.744104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) 2014-08-13 18:06:09.744115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.285 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) 2014-08-13 18:06:09.744129 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.461966 8 0.000574 2014-08-13 18:06:09.744140 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive 2014-08-13 18:06:09.744152 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.744167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.744177 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] activate - not complete, missing(50) 2014-08-13 18:06:09.744072 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 65 as a source 2014-08-13 18:06:09.744228 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.65 2014-08-13 18:06:09.744382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.65 2014-08-13 18:06:09.744537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.65 2014-08-13 18:06:09.744690 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.65 2014-08-13 18:06:09.744857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] state: Activate Finished 2014-08-13 18:06:09.744919 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.744936 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 61 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.744951 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 18:06:09.744961 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.744843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.65 2014-08-13 18:06:09.744982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) 2014-08-13 18:06:09.745005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: got info from osd.364 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) 2014-08-13 18:06:09.745020 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 1.521242 8 0.000675 2014-08-13 18:06:09.745034 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive 2014-08-13 18:06:09.745046 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.745057 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.745067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] activate - not complete, missing(48) 2014-08-13 18:06:09.745000 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.65 2014-08-13 18:06:09.745145 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] _activate_committed 194497 telling primary 2014-08-13 18:06:09.745176 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing 2014-08-13 18:06:09.745194 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] flushed 2014-08-13 18:06:09.745153 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.65 2014-08-13 18:06:09.745309 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.65 2014-08-13 18:06:09.745462 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.65 2014-08-13 18:06:09.745616 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.65 2014-08-13 18:06:09.745792 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] state: Activate Finished 2014-08-13 18:06:09.745860 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.745877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 340 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) 2014-08-13 18:06:09.745893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 340, already have info from that osd, ignoring 2014-08-13 18:06:09.745905 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.745768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.65 2014-08-13 18:06:09.746063 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.746066 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.746067 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d080 pg_info(1 pgs e194497:24.1e6) v4 2014-08-13 18:06:09.745920 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.65 2014-08-13 18:06:09.745910 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.746091 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e6) v4 from osd.285 2014-08-13 18:06:09.746095 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d080 2014-08-13 18:06:09.746088 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] _activate_committed 194497 telling primary 2014-08-13 18:06:09.746119 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.746119 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing 2014-08-13 18:06:09.746121 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.746140 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] flushed 2014-08-13 18:06:09.746071 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.65 2014-08-13 18:06:09.746083 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops 2014-08-13 18:06:09.746260 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.746265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) 2014-08-13 18:06:09.746278 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.45 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) 2014-08-13 18:06:09.746291 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.448789 8 0.000563 2014-08-13 18:06:09.746302 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive 2014-08-13 18:06:09.746313 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.746323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.746332 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] activate - not complete, missing(50) 2014-08-13 18:06:09.746228 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.65 2014-08-13 18:06:09.746381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.65 2014-08-13 18:06:09.746534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.65 2014-08-13 18:06:09.746686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.65 2014-08-13 18:06:09.746839 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.65 2014-08-13 18:06:09.747099 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.747101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.747103 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_info(1 pgs e194497:24.1de) v4 2014-08-13 18:06:09.747117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] state: Activate Finished 2014-08-13 18:06:09.747125 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1de) v4 from osd.364 2014-08-13 18:06:09.747127 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 2014-08-13 18:06:09.746992 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.65 2014-08-13 18:06:09.747150 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.747152 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.747183 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.747198 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.747213 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] requeue_ops 2014-08-13 18:06:09.747224 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.747144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.65 2014-08-13 18:06:09.747228 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:09.747427 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] _activate_committed 194497 telling primary 2014-08-13 18:06:09.747301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.65 2014-08-13 18:06:09.747453 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4496720 don't know epoch, doing nothing 2014-08-13 18:06:09.747470 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] flushed 2014-08-13 18:06:09.747373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 18:06:09.747514 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.747453 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.65 2014-08-13 18:06:09.747519 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 337 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.747606 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.65 2014-08-13 18:06:09.747667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 337, already have info from that osd, ignoring 2014-08-13 18:06:09.747813 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.747818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 15 info: 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) 2014-08-13 18:06:09.747830 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: got info from osd.15 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) 2014-08-13 18:06:09.747844 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.467832 8 0.000669 2014-08-13 18:06:09.747857 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive 2014-08-13 18:06:09.747869 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.747880 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.747890 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] activate - not complete, missing(56) 2014-08-13 18:06:09.747759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.65 2014-08-13 18:06:09.747911 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.65 2014-08-13 18:06:09.748063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.65 2014-08-13 18:06:09.748219 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.65 2014-08-13 18:06:09.748441 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.748443 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.748444 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b800 pg_info(1 pgs e194497:24.369) v4 2014-08-13 18:06:09.748466 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.369) v4 from osd.45 2014-08-13 18:06:09.748469 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b800 2014-08-13 18:06:09.748490 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.748492 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.748372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.65 2014-08-13 18:06:09.748524 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.65 2014-08-13 18:06:09.748677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.65 2014-08-13 18:06:09.748829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.65 2014-08-13 18:06:09.748982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.65 2014-08-13 18:06:09.749243 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] state: Activate Finished 2014-08-13 18:06:09.749135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.65 2014-08-13 18:06:09.749310 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.749291 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.65 2014-08-13 18:06:09.749328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 362 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.749444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.65 2014-08-13 18:06:09.749576 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] _activate_committed 194497 telling primary 2014-08-13 18:06:09.749606 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 2014-08-13 18:06:09.749627 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] flushed 2014-08-13 18:06:09.749505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.362 activated and committed 2014-08-13 18:06:09.749676 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.749597 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.65 2014-08-13 18:06:09.749683 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.749750 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.65 2014-08-13 18:06:09.749855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:09.750025 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.749903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.65 2014-08-13 18:06:09.750030 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.750062 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.65 2014-08-13 18:06:09.750177 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops 2014-08-13 18:06:09.750319 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.750218 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.65 2014-08-13 18:06:09.750403 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.750406 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.750407 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b2c0 pg_info(1 pgs e194497:24.136) v4 2014-08-13 18:06:09.750429 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.55 2014-08-13 18:06:09.750431 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b2c0 2014-08-13 18:06:09.750324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 320 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) 2014-08-13 18:06:09.750372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.65 2014-08-13 18:06:09.750463 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 320, already have info from that osd, ignoring 2014-08-13 18:06:09.750604 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.750525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.65 2014-08-13 18:06:09.750609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 367 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.750678 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.65 2014-08-13 18:06:09.750787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.367 activated and committed 2014-08-13 18:06:09.750963 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.750969 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 373 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) 2014-08-13 18:06:09.750831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.65 2014-08-13 18:06:09.750981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 373, already have info from that osd, ignoring 2014-08-13 18:06:09.750991 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.750995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 376 notify: (query_epoch:194497, epoch_sent:194497, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.751005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 18:06:09.751014 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.751018 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 368 notify: (query_epoch:194497, epoch_sent:194497, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.751028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 18:06:09.751038 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.750983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.65 2014-08-13 18:06:09.751041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 350 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.751136 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.65 2014-08-13 18:06:09.751222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.350 activated and committed 2014-08-13 18:06:09.751399 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.751404 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.751415 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:09.751424 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.751292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.65 2014-08-13 18:06:09.751444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.65 2014-08-13 18:06:09.751428 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 375 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.751599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.65 2014-08-13 18:06:09.751603 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.375 activated and committed 2014-08-13 18:06:09.751770 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.751774 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) 2014-08-13 18:06:09.751787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: got info from osd.330 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) 2014-08-13 18:06:09.751801 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 1.470126 8 0.000649 2014-08-13 18:06:09.751813 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive 2014-08-13 18:06:09.751825 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.751835 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.751846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] activate - not complete, missing(64) 2014-08-13 18:06:09.751753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.65 2014-08-13 18:06:09.751906 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.65 2014-08-13 18:06:09.752059 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} 2014-08-13 18:06:09.752249 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound 2014-08-13 18:06:09.752401 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _calc_past_interval_range: already have past intervals back to 194492 2014-08-13 18:06:09.752602 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] state: Activate Finished 2014-08-13 18:06:09.752673 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.752554 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound: built 50,55,61,65,356 2014-08-13 18:06:09.752707 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 365 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495)) 2014-08-13 18:06:09.752724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 365, already have info from that osd, ignoring 2014-08-13 18:06:09.752734 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.752707 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Activate Finished 2014-08-13 18:06:09.752738 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 378 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.752902 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] _activate_committed 194497 telling primary 2014-08-13 18:06:09.752944 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing 2014-08-13 18:06:09.752967 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] flushed 2014-08-13 18:06:09.752861 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating 2014-08-13 18:06:09.752918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.378 activated and committed 2014-08-13 18:06:09.753094 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.753099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 26 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) 2014-08-13 18:06:09.753111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.26 activated and committed 2014-08-13 18:06:09.753128 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.753015 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 10 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.753168 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753371 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753428 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753432 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753418 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 2014-08-13 18:06:09.753436 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753446 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753447 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.753449 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c0c0 pg_info(1 pgs e194497:24.20e) v4 2014-08-13 18:06:09.753452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753306 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.10 activated and committed 2014-08-13 18:06:09.753463 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.753475 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.753477 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.20e) v4 from osd.15 2014-08-13 18:06:09.753473 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.753480 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c0c0 2014-08-13 18:06:09.753483 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753493 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753503 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753505 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753512 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753515 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753517 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753518 7f5c835ef700 20 osd.4 194497 _dispatch 0x588c280 pg_info(1 pgs e194497:24.136) v4 2014-08-13 18:06:09.753526 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 2014-08-13 18:06:09.753539 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753539 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.61 2014-08-13 18:06:09.753542 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588c280 2014-08-13 18:06:09.753543 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.753553 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753564 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753566 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753563 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.753577 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753579 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753573 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.753580 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df08c0 MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 2014-08-13 18:06:09.753582 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753592 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753602 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753603 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df08c0 2014-08-13 18:06:09.753616 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753617 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753619 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,32] -- 356/356 2014-08-13 18:06:09.753635 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753639 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Started advmap 2014-08-13 18:06:09.753480 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 20 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.753650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map 2014-08-13 18:06:09.753726 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] take_waiters 2014-08-13 18:06:09.753737 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753746 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753757 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753770 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 2014-08-13 18:06:09.753784 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.753797 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753636 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _activate_committed 194497 peer_activated now 4 last_epoch_started 194492 same_interval_since 194496 2014-08-13 18:06:09.753807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.753652 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.20 activated and committed 2014-08-13 18:06:09.753821 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.753816 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.753830 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753850 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753865 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 2014-08-13 18:06:09.753877 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753880 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap 2014-08-13 18:06:09.753889 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753898 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map 2014-08-13 18:06:09.753907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters 2014-08-13 18:06:09.753915 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.753923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.753933 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.753947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 2014-08-13 18:06:09.753959 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-13 18:06:09.753961 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.753963 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.753965 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357540 pg_info(1 pgs e194497:24.3f7) v4 2014-08-13 18:06:09.753812 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] flushed 2014-08-13 18:06:09.753963 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap 2014-08-13 18:06:09.753979 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap 2014-08-13 18:06:09.753987 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3f7) v4 from osd.330 2014-08-13 18:06:09.753990 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357540 2014-08-13 18:06:09.753989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down 2014-08-13 18:06:09.753827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 291 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.753998 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map 2014-08-13 18:06:09.754009 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap 2014-08-13 18:06:09.754017 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.754019 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.754018 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary 2014-08-13 18:06:09.754028 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2714 2014-08-13 18:06:09.754036 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] take_waiters 2014-08-13 18:06:09.754044 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 2014-08-13 18:06:09.754052 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt 2014-08-13 18:06:09.754062 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.754067 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4493f40 192.168.1.103:6811/37658 2014-08-13 18:06:09.754090 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.17 has 194497 2014-08-13 18:06:09.754092 7f5c7c7e4700 7 osd.4 194497 do_notify osd 17 on 1 PGs 2014-08-13 18:06:09.754003 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.291 activated and committed 2014-08-13 18:06:09.754237 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x445f7a0 192.168.1.113:6808/15284 2014-08-13 18:06:09.754264 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.289 has 194497 2014-08-13 18:06:09.754266 7f5c7c7e4700 7 osd.4 194497 do_notify osd 289 on 1 PGs 2014-08-13 18:06:09.754354 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:09.754362 7f5c7c7e4700 7 osd.4 194497 do_notify osd 356 on 2 PGs 2014-08-13 18:06:09.754381 7f5c7c7e4700 20 osd.4 194497 Sending info 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 17 2014-08-13 18:06:09.754388 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4493f40 already has epoch 194497 2014-08-13 18:06:09.754246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] all_activated_and_committed 2014-08-13 18:06:09.754422 7f5c7c7e4700 20 osd.4 194497 Sending info 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) to shard 50 2014-08-13 18:06:09.754428 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing 2014-08-13 18:06:09.754448 7f5c7c7e4700 20 osd.4 194497 Sending info 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 55 2014-08-13 18:06:09.754454 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing 2014-08-13 18:06:09.754475 7f5c7c7e4700 20 osd.4 194497 Sending info 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) to shard 65 2014-08-13 18:06:09.754480 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing 2014-08-13 18:06:09.754507 7f5c7c7e4700 20 osd.4 194497 Sending info 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 315 2014-08-13 18:06:09.754517 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c51080 don't know epoch, doing nothing 2014-08-13 18:06:09.754543 7f5c7c7e4700 20 osd.4 194497 Sending info 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 351 2014-08-13 18:06:09.754571 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4af8160 192.168.1.102:6830/16444 2014-08-13 18:06:09.754589 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.351 has 194497 2014-08-13 18:06:09.754420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] share_pg_info 2014-08-13 18:06:09.754598 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing 2014-08-13 18:06:09.754724 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing 2014-08-13 18:06:09.754741 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b11ce0 already has epoch 194497 2014-08-13 18:06:09.754756 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing 2014-08-13 18:06:09.754769 7f5c7c7e4700 10 osd.4 194497 dequeue_op 0x20a7ad0 prio 127 cost 0 latency 0.266520 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] 2014-08-13 18:06:09.754805 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_message: 0x20a7ad0 2014-08-13 18:06:09.754819 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 2014-08-13 18:06:09.754831 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] sub_op_scrub_unreserve 2014-08-13 18:06:09.754848 7f5c7c7e4700 20 osd.4 194497 dec_scrubs_pending 1 -> 0 (max 1, active 0) 2014-08-13 18:06:09.754850 7f5c7c7e4700 10 osd.4 194497 dequeue_op 0x20a7ad0 finish 2014-08-13 18:06:09.754877 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 45 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.754897 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 45, already have info from that osd, ignoring 2014-08-13 18:06:09.754908 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.754914 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) 2014-08-13 18:06:09.754926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: got info from osd.17 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) 2014-08-13 18:06:09.754770 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194497:25294 2014-08-13 18:06:09.754952 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.754943 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 1.524477 8 0.000592 2014-08-13 18:06:09.754956 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive 2014-08-13 18:06:09.754958 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 30 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) 2014-08-13 18:06:09.754969 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.754971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.30 activated and committed 2014-08-13 18:06:09.754982 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.754981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.754987 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 61 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) 2014-08-13 18:06:09.754991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] activate - not complete, missing(49) 2014-08-13 18:06:09.754999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring 2014-08-13 18:06:09.755009 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.755012 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 340 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) 2014-08-13 18:06:09.755023 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.340 activated and committed 2014-08-13 18:06:09.755033 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] all_activated_and_committed 2014-08-13 18:06:09.755042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] share_pg_info 2014-08-13 18:06:09.755054 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing 2014-08-13 18:06:09.755069 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing 2014-08-13 18:06:09.755084 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4499080 don't know epoch, doing nothing 2014-08-13 18:06:09.755097 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] publish_stats_to_osd 194497:23847 2014-08-13 18:06:09.755112 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.755203 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 286 info: 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.755376 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.286 activated and committed 2014-08-13 18:06:09.755580 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.755585 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.755587 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e900 MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 2014-08-13 18:06:09.755612 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e900 2014-08-13 18:06:09.755634 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.755636 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.755543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] all_activated_and_committed 2014-08-13 18:06:09.755785 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] state: Activate Finished 2014-08-13 18:06:09.755857 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.755709 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] share_pg_info 2014-08-13 18:06:09.755877 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing 2014-08-13 18:06:09.755895 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing 2014-08-13 18:06:09.755910 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing 2014-08-13 18:06:09.755886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 356 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:09.755925 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194497:30302 2014-08-13 18:06:09.756095 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.756100 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) 2014-08-13 18:06:09.756112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:09.756122 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.756129 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] _activate_committed 194497 telling primary 2014-08-13 18:06:09.756160 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4493f40 already has epoch 194497 2014-08-13 18:06:09.756188 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] flushed 2014-08-13 18:06:09.756053 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 356, already have info from that osd, ignoring 2014-08-13 18:06:09.756218 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.756125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 25 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.756225 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 50 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:09.756324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.25 activated and committed 2014-08-13 18:06:09.756496 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.756502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.756513 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring 2014-08-13 18:06:09.756523 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.756380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 50, already have info from that osd, ignoring 2014-08-13 18:06:09.756537 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.756540 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756542 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.756542 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f8c0 pg_info(1 pgs e194497:24.1f7) v4 2014-08-13 18:06:09.756547 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 15 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.756560 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 15, already have info from that osd, ignoring 2014-08-13 18:06:09.756568 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.55 2014-08-13 18:06:09.756570 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.756571 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f8c0 2014-08-13 18:06:09.756574 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.756586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:09.756595 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.756606 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.756608 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756626 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.756628 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756629 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f000 pg_info(1 pgs e194497:24.136) v4 2014-08-13 18:06:09.756654 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.50 2014-08-13 18:06:09.756656 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f000 2014-08-13 18:06:09.756680 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.756682 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 303 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.756599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 351 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.756788 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.756791 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756792 7f5c835ef700 20 osd.4 194497 _dispatch 0x588db00 pg_info(1 pgs e194497:24.1f7) v4 2014-08-13 18:06:09.756817 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.351 2014-08-13 18:06:09.756821 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588db00 2014-08-13 18:06:09.756695 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.303 activated and committed 2014-08-13 18:06:09.756792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 351, already have info from that osd, ignoring 2014-08-13 18:06:09.756975 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757010 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757012 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.756863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] all_activated_and_committed 2014-08-13 18:06:09.757040 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757042 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757043 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d240 pg_info(1 pgs e194497:24.b6) v4 2014-08-13 18:06:09.757067 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.17 2014-08-13 18:06:09.757070 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d240 2014-08-13 18:06:09.757101 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757103 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757119 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757120 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757121 7f5c835ef700 20 osd.4 194497 _dispatch 0x588fe00 pg_info(1 pgs e194497:24.2a1) v4 2014-08-13 18:06:09.756984 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 65 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) 2014-08-13 18:06:09.757143 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2a1) v4 from osd.17 2014-08-13 18:06:09.757145 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588fe00 2014-08-13 18:06:09.757178 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757179 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757195 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757196 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757029 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] share_pg_info 2014-08-13 18:06:09.757197 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6af40 pg_info(1 pgs e194497:24.b6) v4 2014-08-13 18:06:09.757205 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing 2014-08-13 18:06:09.757221 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing 2014-08-13 18:06:09.757225 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.315 2014-08-13 18:06:09.757228 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6af40 2014-08-13 18:06:09.757237 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing 2014-08-13 18:06:09.757253 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing 2014-08-13 18:06:09.757263 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757266 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757138 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 65, already have info from that osd, ignoring 2014-08-13 18:06:09.757295 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757420 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757422 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757423 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6aa00 pg_info(1 pgs e194497:24.136) v4 2014-08-13 18:06:09.757445 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.65 2014-08-13 18:06:09.757447 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6aa00 2014-08-13 18:06:09.757488 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.757490 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.757346 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194497:29711 2014-08-13 18:06:09.757527 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.757384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 376 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.757534 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.757593 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 376, already have info from that osd, ignoring 2014-08-13 18:06:09.757784 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757790 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 11 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.757804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 11, already have info from that osd, ignoring 2014-08-13 18:06:09.757814 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757685 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops 2014-08-13 18:06:09.757818 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 368 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.757828 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.757829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring 2014-08-13 18:06:09.757839 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.757843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 54 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.757848 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] requeue_ops 2014-08-13 18:06:09.757859 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.757854 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 54, already have info from that osd, ignoring 2014-08-13 18:06:09.757867 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.757871 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 289 info: 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) 2014-08-13 18:06:09.757884 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: got info from osd.289 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) 2014-08-13 18:06:09.757897 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 1.521525 8 0.000676 2014-08-13 18:06:09.757910 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive 2014-08-13 18:06:09.757923 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.757933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.757943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] activate - not complete, missing(63) 2014-08-13 18:06:09.757863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 332 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.758007 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.332 activated and committed 2014-08-13 18:06:09.758157 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.758167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 344 info: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194466 194495/194496/194496) 2014-08-13 18:06:09.758313 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.344 activated and committed 2014-08-13 18:06:09.758453 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.758458 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 320 info: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) 2014-08-13 18:06:09.758599 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.320 activated and committed 2014-08-13 18:06:09.758739 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.758743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 337 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.758949 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] state: Activate Finished 2014-08-13 18:06:09.759026 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.758888 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.337 activated and committed 2014-08-13 18:06:09.759031 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.759051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) 2014-08-13 18:06:09.759064 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: got info from osd.356 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) 2014-08-13 18:06:09.759079 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.504227 9 0.000755 2014-08-13 18:06:09.759094 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive 2014-08-13 18:06:09.759106 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.759117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.759127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] activate - not complete, missing(56) 2014-08-13 18:06:09.759057 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 17 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.759291 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] _activate_committed 194497 telling primary 2014-08-13 18:06:09.759320 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445f7a0 already has epoch 194497 2014-08-13 18:06:09.759338 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] flushed 2014-08-13 18:06:09.759259 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 17, already have info from that osd, ignoring 2014-08-13 18:06:09.759437 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.759445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) 2014-08-13 18:06:09.759457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: got info from osd.356 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) 2014-08-13 18:06:09.759470 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 1.509055 8 0.000603 2014-08-13 18:06:09.759481 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive 2014-08-13 18:06:09.759495 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive/RepNotRecovering 2014-08-13 18:06:09.759505 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: In ReplicaActive, about to call activate 2014-08-13 18:06:09.759515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] activate - not complete, missing(51) 2014-08-13 18:06:09.759869 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] state: Activate Finished 2014-08-13 18:06:09.759940 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.759961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) 2014-08-13 18:06:09.759999 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.760013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194497/194489 194495/194496/193262) 2014-08-13 18:06:09.760044 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.760159 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] state: Activate Finished 2014-08-13 18:06:09.760160 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] _activate_committed 194497 telling primary 2014-08-13 18:06:09.760196 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:09.760055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 296 info: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) 2014-08-13 18:06:09.760213 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.760215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.760216 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fa80 pg_info(1 pgs e194497:24.5e) v4 2014-08-13 18:06:09.760227 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.760216 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] flushed 2014-08-13 18:06:09.760239 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.5e) v4 from osd.289 2014-08-13 18:06:09.760241 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fa80 2014-08-13 18:06:09.760275 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.760277 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.760204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.296 activated and committed 2014-08-13 18:06:09.760343 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] all_activated_and_committed 2014-08-13 18:06:09.760290 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.760596 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] _activate_committed 194497 telling primary 2014-08-13 18:06:09.760630 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing 2014-08-13 18:06:09.760650 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] flushed 2014-08-13 18:06:09.760536 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] share_pg_info 2014-08-13 18:06:09.760678 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4854a40 don't know epoch, doing nothing 2014-08-13 18:06:09.760696 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing 2014-08-13 18:06:09.760711 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing 2014-08-13 18:06:09.760545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 55, already have info from that osd, ignoring 2014-08-13 18:06:09.760733 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.760726 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] publish_stats_to_osd 194497:28888 2014-08-13 18:06:09.760918 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.760928 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.760949 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] requeue_ops 2014-08-13 18:06:09.760974 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.760978 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 31 info: 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194497/194372 194495/194496/192978) 2014-08-13 18:06:09.760739 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.761006 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.761020 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.761034 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] requeue_ops 2014-08-13 18:06:09.761044 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.761063 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.761065 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.761066 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c980 pg_info(1 pgs e194497:24.3b7) v4 2014-08-13 18:06:09.761088 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3b7) v4 from osd.356 2014-08-13 18:06:09.761090 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c980 2014-08-13 18:06:09.761130 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.761132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.761138 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 2014-08-13 18:06:09.761170 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 2014-08-13 18:06:09.761049 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 348 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.761225 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.348 activated and committed 2014-08-13 18:06:09.761444 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.761452 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194497/194489 194495/194496/193250) 2014-08-13 18:06:09.761483 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.761498 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.761511 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] requeue_ops 2014-08-13 18:06:09.761522 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.760991 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops 2014-08-13 18:06:09.761526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194497/194493 194495/194496/193232) 2014-08-13 18:06:09.761535 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.761552 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.761677 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.761680 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.761681 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d780 pg_info(1 pgs e194497:24.350) v4 2014-08-13 18:06:09.761705 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.350) v4 from osd.356 2014-08-13 18:06:09.761565 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 331 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.761708 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d780 2014-08-13 18:06:09.761544 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 327 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.761749 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.761751 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.761713 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.331 activated and committed 2014-08-13 18:06:09.761727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 327, already have info from that osd, ignoring 2014-08-13 18:06:09.761905 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.761911 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.761923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:09.761933 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.761937 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.761948 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring 2014-08-13 18:06:09.761958 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.761857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] all_activated_and_committed 2014-08-13 18:06:09.762048 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.762050 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.762052 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f540 MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 2014-08-13 18:06:09.762073 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f540 2014-08-13 18:06:09.762099 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.762101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.761999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] share_pg_info 2014-08-13 18:06:09.762146 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing 2014-08-13 18:06:09.762174 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing 2014-08-13 18:06:09.761995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 315 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) 2014-08-13 18:06:09.762192 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing 2014-08-13 18:06:09.762207 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x571b700 don't know epoch, doing nothing 2014-08-13 18:06:09.762184 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 315, already have info from that osd, ignoring 2014-08-13 18:06:09.762362 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.762220 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194497:22156 2014-08-13 18:06:09.762369 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.762388 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] requeue_ops 2014-08-13 18:06:09.762398 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762402 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194497/194489 194495/194496/192967) 2014-08-13 18:06:09.762432 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762446 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.762459 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] requeue_ops 2014-08-13 18:06:09.762470 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762474 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194497/194493 194495/194496/192955) 2014-08-13 18:06:09.762500 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762513 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.762525 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] requeue_ops 2014-08-13 18:06:09.762535 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762368 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.762543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194497/194489 194495/194496/192975) 2014-08-13 18:06:09.762571 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762584 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.762596 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] requeue_ops 2014-08-13 18:06:09.762607 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762611 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194497/194493 194495/194496/192966) 2014-08-13 18:06:09.762640 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.762681 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] requeue_ops 2014-08-13 18:06:09.762692 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194497/194489 194495/194496/193188) 2014-08-13 18:06:09.762706 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.762709 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.762711 7f5c835ef700 20 osd.4 194497 _dispatch 0x524cd00 MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 2014-08-13 18:06:09.762545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.330 activated and committed 2014-08-13 18:06:09.762722 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.762723 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.762737 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524cd00 2014-08-13 18:06:09.762772 7f5c835ef700 10 osd.4 194497 do_waiters -- start 2014-08-13 18:06:09.762774 7f5c835ef700 10 osd.4 194497 do_waiters -- finish 2014-08-13 18:06:09.762739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated 2014-08-13 18:06:09.762738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 28 info: 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.762920 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] check_local 2014-08-13 18:06:09.762922 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.28 activated and committed 2014-08-13 18:06:09.763098 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.763103 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) 2014-08-13 18:06:09.763114 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring 2014-08-13 18:06:09.763124 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.763091 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] requeue_ops 2014-08-13 18:06:09.763127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 59 info: 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) 2014-08-13 18:06:09.763265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] needs_recovery primary has 57 missing 2014-08-13 18:06:09.763352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: peer osd.59 activated and committed 2014-08-13 18:06:09.763663 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.763537 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:09.763671 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.763996 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops 2014-08-13 18:06:09.764325 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764333 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.764355 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] requeue_ops 2014-08-13 18:06:09.764372 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764379 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194497/194489 194495/194496/193186) 2014-08-13 18:06:09.764122 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] publish_stats_to_osd 194497:29712 2014-08-13 18:06:09.764434 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764460 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.764485 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] requeue_ops 2014-08-13 18:06:09.764510 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.764525 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] requeue_ops 2014-08-13 18:06:09.764535 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764425 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 2014-08-13 18:06:09.764539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 55 info: 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194492 194495/194496/194495) 2014-08-13 18:06:09.764635 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 2014-08-13 18:06:09.764727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: peer osd.55 activated and committed 2014-08-13 18:06:09.764996 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.765006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 15 info: 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) 2014-08-13 18:06:09.765056 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.764913 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 2014-08-13 18:06:09.765079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 61 info: 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194492 194495/194496/194495) 2014-08-13 18:06:09.765189 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 18:06:09.765312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: peer osd.61 activated and committed 2014-08-13 18:06:09.765560 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.765565 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery 2014-08-13 18:06:09.765577 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 0.017708 3 0.001459 2014-08-13 18:06:09.765590 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:09.765604 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.765501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create target_size 1000 fpp 0.05 2014-08-13 18:06:09.765608 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.765839 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops 2014-08-13 18:06:09.766095 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.766103 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194497/194493 194495/194496/193227) 2014-08-13 18:06:09.766154 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.766071 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_apply_log 194422'18731 .. 194422'18731 2014-08-13 18:06:09.766167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated 2014-08-13 18:06:09.766341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_setup allocated new state, position a4b1d83d//0//24 2014-08-13 18:06:09.766402 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] check_local 2014-08-13 18:06:09.766539 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode: num_objects: 59 num_bytes: 243269718 num_objects_dirty: 58 num_objects_omap: 0 num_dirty: 58 num_user_objects: 58 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 18:06:09.766625 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] requeue_ops 2014-08-13 18:06:09.766827 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode dirty 0.059426 full 0.059426 2014-08-13 18:06:09.767064 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.767077 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.767110 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] requeue_ops 2014-08-13 18:06:09.767120 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.767124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.767135 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] requeue_ops 2014-08-13 18:06:09.767145 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.766916 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] needs_recovery primary has 59 missing 2014-08-13 18:06:09.767149 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 289 info: 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) 2014-08-13 18:06:09.767183 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.767197 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt 2014-08-13 18:06:09.767210 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] requeue_ops 2014-08-13 18:06:09.767220 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.767154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:09.767224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated 2014-08-13 18:06:09.767369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] check_local 2014-08-13 18:06:09.767587 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] requeue_ops 2014-08-13 18:06:09.767574 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] publish_stats_to_osd 194497:25295 2014-08-13 18:06:09.767818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] needs_recovery primary has 47 missing 2014-08-13 18:06:09.767872 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 2014-08-13 18:06:09.768048 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:09.768139 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 2014-08-13 18:06:09.768482 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] publish_stats_to_osd 194497:28889 2014-08-13 18:06:09.768437 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 2014-08-13 18:06:09.768728 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] on activate: bft=344 from 0//0//-1 2014-08-13 18:06:09.768737 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 18:06:09.768961 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] target shard 344 from 0//0//-1 2014-08-13 18:06:09.769034 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create target_size 1000 fpp 0.05 2014-08-13 18:06:09.769210 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 18:06:09.769426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_create target_size 1000 fpp 0.05 2014-08-13 18:06:09.769620 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_apply_log 194422'17302 .. 194422'17302 2014-08-13 18:06:09.769969 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_apply_log 194422'19613 .. 194422'19613 2014-08-13 18:06:09.769933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_setup allocated new state, position e6e6749b//0//24 2014-08-13 18:06:09.770223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_setup allocated new state, position 344c7181//0//24 2014-08-13 18:06:09.770465 7f5c7bde3700 0 log [WRN] : pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 18:06:09.770219 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode: num_objects: 61 num_bytes: 251658355 num_objects_dirty: 60 num_objects_omap: 0 num_dirty: 60 num_user_objects: 60 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 18:06:09.770468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode: num_objects: 42 num_bytes: 388397714 num_objects_dirty: 42 num_objects_omap: 0 num_dirty: 42 num_user_objects: 42 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 18:06:09.770493 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode dirty 0.061475 full 0.061475 2014-08-13 18:06:09.770720 7f5c7c7e4700 10 log is not dirty 2014-08-13 18:06:09.770740 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated 2014-08-13 18:06:09.770763 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] check_local 2014-08-13 18:06:09.770780 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] requeue_ops 2014-08-13 18:06:09.770796 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_recovery is recovered 2014-08-13 18:06:09.770813 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.26 has last_backfill 0//0//-1 2014-08-13 18:06:09.770829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.30 has last_backfill 0//0//-1 2014-08-13 18:06:09.770846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.340 has last_backfill 0//0//-1 2014-08-13 18:06:09.770638 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode dirty 0.043032 full 0.043032 2014-08-13 18:06:09.770862 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] activate queueing backfill 2014-08-13 18:06:09.770888 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] publish_stats_to_osd 194497:23848 2014-08-13 18:06:09.770907 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 2014-08-13 18:06:09.770924 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 2014-08-13 18:06:09.770940 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 2014-08-13 18:06:09.770956 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 2014-08-13 18:06:09.770978 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 18:06:09.771001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create target_size 1000 fpp 0.05 2014-08-13 18:06:09.770863 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode stats invalid (post-split), idle 2014-08-13 18:06:09.771109 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.771118 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) 2014-08-13 18:06:09.771178 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.771206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194497/194489 194495/194496/193151) 2014-08-13 18:06:09.771265 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.771278 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery 2014-08-13 18:06:09.771292 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] exit Started/ReplicaActive/RepNotRecovering 0.013370 3 0.001095 2014-08-13 18:06:09.771305 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:09.771318 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.771306 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_apply_log 194215'14514 .. 194215'14514 2014-08-13 18:06:09.771325 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_setup allocated new state, position 50f0e434//0//24 2014-08-13 18:06:09.771338 7f5c7c7e4700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 18:06:09.771341 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_choose_mode: num_objects: -1 num_bytes: -9223372036854775808 num_objects_dirty: -2 num_objects_omap: 0 num_dirty: 0 num_user_objects: 18446744073709551615 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 18:06:09.771322 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated 2014-08-13 18:06:09.771535 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] check_local 2014-08-13 18:06:09.771716 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] requeue_ops 2014-08-13 18:06:09.771952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] needs_recovery primary has 48 missing 2014-08-13 18:06:09.772212 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] activate not all replicas are up-to-date, queueing recovery 2014-08-13 18:06:09.772700 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] publish_stats_to_osd 194497:22157 2014-08-13 18:06:09.772953 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] on activate: bft=331,348 from 0//0//-1 2014-08-13 18:06:09.773211 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 331 from 0//0//-1 2014-08-13 18:06:09.773454 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 348 from 0//0//-1 2014-08-13 18:06:09.773690 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 2014-08-13 18:06:09.773948 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_create target_size 1000 fpp 0.05 2014-08-13 18:06:09.774504 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_apply_log 194422'12517 .. 194422'12517 2014-08-13 18:06:09.774773 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_setup allocated new state, position d8d705bb//0//24 2014-08-13 18:06:09.774994 7f5c7bde3700 0 log [WRN] : pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate 2014-08-13 18:06:09.774997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode: num_objects: 50 num_bytes: 205520955 num_objects_dirty: 54 num_objects_omap: 0 num_dirty: 54 num_user_objects: 50 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 2014-08-13 18:06:09.775204 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode dirty 0.055327 full 0.051229 2014-08-13 18:06:09.775443 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode stats invalid (post-split), idle 2014-08-13 18:06:09.775687 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.775715 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery 2014-08-13 18:06:09.775740 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 0.016634 3 0.000835 2014-08-13 18:06:09.775763 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved 2014-08-13 18:06:09.775784 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.775791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 DoRecovery 2014-08-13 18:06:09.776105 7f5c7c7e4700 -1 *** Caught signal (Floating point exception) ** in thread 7f5c7c7e4700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7f5c96352710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7f5c9634a9d1] 16: (clone()+0x6d) [0x7f5c952dab6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -10000> 2014-08-13 18:06:08.326825 7f5c835ef700 5 -- op tracker -- , seq: 496, time: 2014-08-13 18:06:08.326825, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194496) v3 -9999> 2014-08-13 18:06:08.326829 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing -9998> 2014-08-13 18:06:08.326833 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.337 on 1 PGs -9997> 2014-08-13 18:06:08.326832 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.6 epoch 194496 -9996> 2014-08-13 18:06:08.326837 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21aaee0 -9995> 2014-08-13 18:06:08.326828 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9994> 2014-08-13 18:06:08.326836 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6804/62197 -- pg_query(24.1bb epoch 194496) v3 -- ?+0 0x21ac920 con 0x3c55540 -9993> 2014-08-13 18:06:08.326842 7f5c835ef700 5 -- op tracker -- , seq: 496, time: 2014-08-13 18:06:08.326842, event: started, request: pg_query(24.cd epoch 194496) v3 -9992> 2014-08-13 18:06:08.326852 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing -9991> 2014-08-13 18:06:08.326855 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.344 on 1 PGs -9990> 2014-08-13 18:06:08.326849 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9989> 2014-08-13 18:06:08.326858 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6812/6757 -- pg_query(24.181 epoch 194496) v3 -- ?+0 0x21a9860 con 0x4aff220 -9988> 2014-08-13 18:06:08.326866 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194486 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9987> 2014-08-13 18:06:08.326882 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'518 -> 194215'518 -9986> 2014-08-13 18:06:08.326905 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] state: Activate Finished -9985> 2014-08-13 18:06:08.326896 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194422'15992 ec=192688 les/c 194493/194489 194495/194496/192967) from replica 6 -9984> 2014-08-13 18:06:08.326975 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x571b700 don't know epoch, doing nothing -9983> 2014-08-13 18:06:08.326964 7f5c835ef700 5 -- op tracker -- , seq: 496, time: 2014-08-13 18:06:08.326963, event: done, request: pg_query(24.cd epoch 194496) v3 -9982> 2014-08-13 18:06:08.326983 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.348 on 1 PGs -9981> 2014-08-13 18:06:08.326988 7f5c7c7e4700 10 log is not dirty -9980> 2014-08-13 18:06:08.326993 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9979> 2014-08-13 18:06:08.326996 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9978> 2014-08-13 18:06:08.326987 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.110:6800/41204 -- pg_query(24.1bb epoch 194496) v3 -- ?+0 0x21ae540 con 0x571b700 -9977> 2014-08-13 18:06:08.327010 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2d120 don't know epoch, doing nothing -9976> 2014-08-13 18:06:08.327014 7f5c7bde3700 7 osd.4 194496 do_queries querying osd.379 on 1 PGs -9975> 2014-08-13 18:06:08.327013 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -9974> 2014-08-13 18:06:08.327003 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 11 ==== pg_info(1 pgs e194496:22.4fe) v4 ==== 714+0+0 (4140894720 0 0) 0x4de9dc0 con 0x3c55540 -9973> 2014-08-13 18:06:08.327022 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing -9972> 2014-08-13 18:06:08.327023 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9971> 2014-08-13 18:06:08.327017 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6830/4861 -- pg_query(24.1bb epoch 194496) v3 -- ?+0 0x21af620 con 0x4b2d120 -9970> 2014-08-13 18:06:08.327026 7f5c7c7e4700 7 osd.4 194496 do_notify osd 45 on 1 PGs -9969> 2014-08-13 18:06:08.327030 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(36) epoch 194496) v5 -- ?+0 0x4a45e80 con 0x4496720 -9968> 2014-08-13 18:06:08.327039 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 286 -9967> 2014-08-13 18:06:08.327174 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing -9966> 2014-08-13 18:06:08.327187 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6820/19173 -- pg_info(1 pgs e194496:22.3d5(3)) v4 -- ?+0 0x4df0fc0 con 0x3cf2940 -9965> 2014-08-13 18:06:08.327169 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'492 lcod 0'0 inactive] _activate_committed 194496 telling primary -9964> 2014-08-13 18:06:08.327030 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9963> 2014-08-13 18:06:08.327208 7f5c7bde3700 20 osd.4 194496 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) to shard 287 -9962> 2014-08-13 18:06:08.327218 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a7bc0 don't know epoch, doing nothing -9961> 2014-08-13 18:06:08.327217 7f5c835ef700 20 osd.4 194496 _dispatch 0x4de9dc0 pg_info(1 pgs e194496:22.4fe) v4 -9960> 2014-08-13 18:06:08.327222 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing -9959> 2014-08-13 18:06:08.327228 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.326638, event: header_read, request: pg_info(1 pgs e194496:22.4fe) v4 -9958> 2014-08-13 18:06:08.327227 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194496:22.1fb) v4 -- ?+0 0x5356c80 -9957> 2014-08-13 18:06:08.327238 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.326641, event: throttled, request: pg_info(1 pgs e194496:22.4fe) v4 -9956> 2014-08-13 18:06:08.327246 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.326708, event: all_read, request: pg_info(1 pgs e194496:22.4fe) v4 -9955> 2014-08-13 18:06:08.327245 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.109:6814/33435 -- pg_info(1 pgs e194496:22.481(50)) v4 -- ?+0 0x4df4ec0 con 0x49a7bc0 -9954> 2014-08-13 18:06:08.327252 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.327022, event: dispatched, request: pg_info(1 pgs e194496:22.4fe) v4 -9953> 2014-08-13 18:06:08.327264 7f5c7bde3700 20 osd.4 194496 Sending info 22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) to shard 289 -9952> 2014-08-13 18:06:08.327273 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing -9951> 2014-08-13 18:06:08.327257 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] flushed -9950> 2014-08-13 18:06:08.327260 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.327259, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.4fe) v4 -9949> 2014-08-13 18:06:08.327287 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4fe) v4 from osd.337 -9948> 2014-08-13 18:06:08.327294 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4de9dc0 -9947> 2014-08-13 18:06:08.327290 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6808/15284 -- pg_info(1 pgs e194496:22.48e(35)) v4 -- ?+0 0x4df3f00 con 0x445f7a0 -9946> 2014-08-13 18:06:08.327299 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.327299, event: started, request: pg_info(1 pgs e194496:22.4fe) v4 -9945> 2014-08-13 18:06:08.327308 7f5c7bde3700 20 osd.4 194496 Sending info 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) to shard 309 -9944> 2014-08-13 18:06:08.327319 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing -9943> 2014-08-13 18:06:08.327344 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 -9942> 2014-08-13 18:06:08.327345 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6814/23703 -- pg_info(1 pgs e194496:22.481(50)) v4 -- ?+0 0x4df0e00 con 0x4c59ce0 -9941> 2014-08-13 18:06:08.327354 7f5c7c7e4700 7 osd.4 194496 do_notify osd 304 on 1 PGs -9940> 2014-08-13 18:06:08.327358 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(36) epoch 194496) v5 -- ?+0 0x4a42d80 con 0x4ab69e0 -9939> 2014-08-13 18:06:08.327380 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) to shard 338 -9938> 2014-08-13 18:06:08.327391 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b1b5a0 don't know epoch, doing nothing -9937> 2014-08-13 18:06:08.327421 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6814/25893 -- pg_info(1 pgs e194496:22.3d9(53)) v4 -- ?+0 0x53571c0 con 0x4b1b5a0 -9936> 2014-08-13 18:06:08.327458 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 crt=192888'478 lcod 0'0 inactive] _activate_committed 194496 telling primary -9935> 2014-08-13 18:06:08.327618 7f5c7bde3700 20 osd.4 194496 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 340 -9934> 2014-08-13 18:06:08.327629 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b10b00 don't know epoch, doing nothing -9933> 2014-08-13 18:06:08.327640 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.397 192.168.1.105:6808/41085 -- pg_info(1 pgs e194496:22.3b9) v4 -- ?+0 0x5354440 -9932> 2014-08-13 18:06:08.327673 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] flushed -9931> 2014-08-13 18:06:08.327724 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -9930> 2014-08-13 18:06:08.327733 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:22.1b8(3)) v4 -- ?+0 0x5351dc0 con 0x4499080 -9929> 2014-08-13 18:06:08.327752 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 343 -9928> 2014-08-13 18:06:08.327764 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -9927> 2014-08-13 18:06:08.327762 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -9926> 2014-08-13 18:06:08.327773 7f5c7c7e4700 7 osd.4 194496 do_notify osd 356 on 2 PGs -9925> 2014-08-13 18:06:08.327769 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6802/61960 -- pg_info(1 pgs e194496:22.3d5(3)) v4 -- ?+0 0x5354d00 con 0x48506e0 -9924> 2014-08-13 18:06:08.327777 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(36),24.3b7(45) epoch 194496) v5 -- ?+0 0x4a45cc0 con 0x3c527e0 -9923> 2014-08-13 18:06:08.327839 7f5c835ef700 5 -- op tracker -- , seq: 497, time: 2014-08-13 18:06:08.327838, event: done, request: pg_info(1 pgs e194496:22.4fe) v4 -9922> 2014-08-13 18:06:08.327907 7f5c7bde3700 20 osd.4 194496 Sending info 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) to shard 347 -9921> 2014-08-13 18:06:08.327924 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing -9920> 2014-08-13 18:06:08.327930 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6800/4640 -- pg_info(1 pgs e194496:22.1b8(3)) v4 -- ?+0 0x5356740 con 0x4afb020 -9919> 2014-08-13 18:06:08.327950 7f5c7bde3700 20 osd.4 194496 Sending info 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) to shard 376 -9918> 2014-08-13 18:06:08.327959 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing -9917> 2014-08-13 18:06:08.327984 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6838/6893 -- pg_info(1 pgs e194496:22.3d9(53)) v4 -- ?+0 0x53563c0 con 0x4b2bc80 -9916> 2014-08-13 18:06:08.328173 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9915> 2014-08-13 18:06:08.328189 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9914> 2014-08-13 18:06:08.328181 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 349 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -9913> 2014-08-13 18:06:08.328198 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 12 ==== pg_info(1 pgs e194496:22.4fe) v4 ==== 714+0+0 (1430283340 0 0) 0x4df4440 con 0x3c55540 -9912> 2014-08-13 18:06:08.328216 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9911> 2014-08-13 18:06:08.328219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9910> 2014-08-13 18:06:08.328222 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4440 pg_info(1 pgs e194496:22.4fe) v4 -9909> 2014-08-13 18:06:08.328217 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 349, already have info from that osd, ignoring -9908> 2014-08-13 18:06:08.328229 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.326888, event: header_read, request: pg_info(1 pgs e194496:22.4fe) v4 -9907> 2014-08-13 18:06:08.328238 7f5c7bde3700 10 log is not dirty -9906> 2014-08-13 18:06:08.328237 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.326892, event: throttled, request: pg_info(1 pgs e194496:22.4fe) v4 -9905> 2014-08-13 18:06:08.328244 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.326984, event: all_read, request: pg_info(1 pgs e194496:22.4fe) v4 -9904> 2014-08-13 18:06:08.328249 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.328215, event: dispatched, request: pg_info(1 pgs e194496:22.4fe) v4 -9903> 2014-08-13 18:06:08.328282 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.328281, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.4fe) v4 -9902> 2014-08-13 18:06:08.328290 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.4fe) v4 from osd.337 -9901> 2014-08-13 18:06:08.328295 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4440 -9900> 2014-08-13 18:06:08.328300 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.328300, event: started, request: pg_info(1 pgs e194496:22.4fe) v4 -9899> 2014-08-13 18:06:08.328317 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 309 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) -9898> 2014-08-13 18:06:08.328343 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 309, already have info from that osd, ignoring -9897> 2014-08-13 18:06:08.328361 7f5c7c7e4700 10 log is not dirty -9896> 2014-08-13 18:06:08.328368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 365 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495)) -9895> 2014-08-13 18:06:08.328386 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 365, already have info from that osd, ignoring -9894> 2014-08-13 18:06:08.328403 7f5c7c7e4700 10 log is not dirty -9893> 2014-08-13 18:06:08.328368 7f5c835ef700 5 -- op tracker -- , seq: 498, time: 2014-08-13 18:06:08.328368, event: done, request: pg_info(1 pgs e194496:22.4fe) v4 -9892> 2014-08-13 18:06:08.328417 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9891> 2014-08-13 18:06:08.328421 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9890> 2014-08-13 18:06:08.328408 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 306 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -9889> 2014-08-13 18:06:08.328427 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 306, already have info from that osd, ignoring -9888> 2014-08-13 18:06:08.328443 7f5c7c7e4700 10 log is not dirty -9887> 2014-08-13 18:06:08.328428 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.343 192.168.1.104:6802/61960 10 ==== pg_query(24.206 epoch 194496) v3 ==== 144+0+0 (85992332 0 0) 0x21ada00 con 0x48506e0 -9886> 2014-08-13 18:06:08.328448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 343 query_epoch 194496 query: query(info 0'0) -9885> 2014-08-13 18:06:08.328464 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9884> 2014-08-13 18:06:08.328468 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9883> 2014-08-13 18:06:08.328470 7f5c835ef700 20 osd.4 194496 _dispatch 0x21ada00 pg_query(24.206 epoch 194496) v3 -9882> 2014-08-13 18:06:08.328477 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.327604, event: header_read, request: pg_query(24.206 epoch 194496) v3 -9881> 2014-08-13 18:06:08.328469 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info -9880> 2014-08-13 18:06:08.328484 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.327607, event: throttled, request: pg_query(24.206 epoch 194496) v3 -9879> 2014-08-13 18:06:08.328490 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.327670, event: all_read, request: pg_query(24.206 epoch 194496) v3 -9878> 2014-08-13 18:06:08.328496 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.328463, event: dispatched, request: pg_query(24.206 epoch 194496) v3 -9877> 2014-08-13 18:06:08.328502 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.328502, event: waiting_for_osdmap, request: pg_query(24.206 epoch 194496) v3 -9876> 2014-08-13 18:06:08.328508 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.343 epoch 194496 -9875> 2014-08-13 18:06:08.328512 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21ada00 -9874> 2014-08-13 18:06:08.328524 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.328524, event: started, request: pg_query(24.206 epoch 194496) v3 -9873> 2014-08-13 18:06:08.328556 7f5c7c7e4700 10 log is not dirty -9872> 2014-08-13 18:06:08.328565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 15 query_epoch 194496 query: query(info 0'0) -9871> 2014-08-13 18:06:08.328587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info -9870> 2014-08-13 18:06:08.328665 7f5c7c7e4700 10 log is not dirty -9869> 2014-08-13 18:06:08.328673 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194492/194463 194495/194495/193205) -9868> 2014-08-13 18:06:08.328694 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: got info from osd.396 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194492/194463 194495/194495/193205) -9867> 2014-08-13 18:06:08.328719 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] exit Started/Stray 0.934997 7 0.000224 -9866> 2014-08-13 18:06:08.328742 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive -9865> 2014-08-13 18:06:08.328726 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193262) from replica 343 -9864> 2014-08-13 18:06:08.328764 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9863> 2014-08-13 18:06:08.328778 7f5c835ef700 5 -- op tracker -- , seq: 499, time: 2014-08-13 18:06:08.328778, event: done, request: pg_query(24.206 epoch 194496) v3 -9862> 2014-08-13 18:06:08.328784 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped NOTIFY] state: In ReplicaActive, about to call activate -9861> 2014-08-13 18:06:08.328799 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9860> 2014-08-13 18:06:08.328803 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9859> 2014-08-13 18:06:08.328802 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] activate - no missing, moving last_complete 194422'15411 -> 194422'15411 -9858> 2014-08-13 18:06:08.328811 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.296 192.168.1.109:6806/31286 3 ==== pg_notify(24.181(38) epoch 194496) v5 ==== 2882+0+0 (1577757805 0 0) 0x5888c40 con 0x4854a40 -9857> 2014-08-13 18:06:08.328829 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9856> 2014-08-13 18:06:08.328831 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9855> 2014-08-13 18:06:08.328833 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888c40 pg_notify(24.181(38) epoch 194496) v5 -9854> 2014-08-13 18:06:08.328839 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.327608, event: header_read, request: pg_notify(24.181(38) epoch 194496) v5 -9853> 2014-08-13 18:06:08.328846 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.327610, event: throttled, request: pg_notify(24.181(38) epoch 194496) v5 -9852> 2014-08-13 18:06:08.328852 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.327745, event: all_read, request: pg_notify(24.181(38) epoch 194496) v5 -9851> 2014-08-13 18:06:08.328880 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.328828, event: dispatched, request: pg_notify(24.181(38) epoch 194496) v5 -9850> 2014-08-13 18:06:08.328888 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.328888, event: waiting_for_osdmap, request: pg_notify(24.181(38) epoch 194496) v5 -9849> 2014-08-13 18:06:08.328878 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] state: Activate Finished -9848> 2014-08-13 18:06:08.328895 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.296 -9847> 2014-08-13 18:06:08.328899 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888c40 -9846> 2014-08-13 18:06:08.328905 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.328905, event: started, request: pg_notify(24.181(38) epoch 194496) v5 -9845> 2014-08-13 18:06:08.328930 7f5c835ef700 5 -- op tracker -- , seq: 500, time: 2014-08-13 18:06:08.328929, event: done, request: pg_notify(24.181(38) epoch 194496) v5 -9844> 2014-08-13 18:06:08.328941 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9843> 2014-08-13 18:06:08.328944 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9842> 2014-08-13 18:06:08.328952 7f5c7c7e4700 10 log is not dirty -9841> 2014-08-13 18:06:08.328984 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 324 notify: (query_epoch:194496, epoch_sent:194496, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) -9840> 2014-08-13 18:06:08.328999 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 8 ==== pg_query(24.20e epoch 194496) v3 ==== 144+0+0 (3005028388 0 0) 0x5420780 con 0x4854620 -9839> 2014-08-13 18:06:08.328247 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 382 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) -9838> 2014-08-13 18:06:08.329049 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 382, already have info from that osd, ignoring -9837> 2014-08-13 18:06:08.329066 7f5c7bde3700 10 log is not dirty -9836> 2014-08-13 18:06:08.329073 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 10 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 lc 0'0 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) -9835> 2014-08-13 18:06:08.329091 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 10, already have info from that osd, ignoring -9834> 2014-08-13 18:06:08.329107 7f5c7bde3700 10 log is not dirty -9833> 2014-08-13 18:06:08.329114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) -9832> 2014-08-13 18:06:08.329132 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: got info from osd.345 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) -9831> 2014-08-13 18:06:08.329153 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.957870 7 0.000312 -9830> 2014-08-13 18:06:08.329182 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9829> 2014-08-13 18:06:08.329201 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9828> 2014-08-13 18:06:08.329220 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194486 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9827> 2014-08-13 18:06:08.329236 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'488 -> 194215'488 -9826> 2014-08-13 18:06:08.329257 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] state: Activate Finished -9825> 2014-08-13 18:06:08.329266 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9824> 2014-08-13 18:06:08.329275 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9823> 2014-08-13 18:06:08.329278 7f5c835ef700 20 osd.4 194496 _dispatch 0x5420780 pg_query(24.20e epoch 194496) v3 -9822> 2014-08-13 18:06:08.329292 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.327814, event: header_read, request: pg_query(24.20e epoch 194496) v3 -9821> 2014-08-13 18:06:08.329302 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.327817, event: throttled, request: pg_query(24.20e epoch 194496) v3 -9820> 2014-08-13 18:06:08.329317 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.327875, event: all_read, request: pg_query(24.20e epoch 194496) v3 -9819> 2014-08-13 18:06:08.329323 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.329265, event: dispatched, request: pg_query(24.20e epoch 194496) v3 -9818> 2014-08-13 18:06:08.329330 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.329329, event: waiting_for_osdmap, request: pg_query(24.20e epoch 194496) v3 -9817> 2014-08-13 18:06:08.329336 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.15 epoch 194496 -9816> 2014-08-13 18:06:08.329340 7f5c7bde3700 10 log is not dirty -9815> 2014-08-13 18:06:08.329339 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5420780 -9814> 2014-08-13 18:06:08.329345 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.329345, event: started, request: pg_query(24.20e epoch 194496) v3 -9813> 2014-08-13 18:06:08.329367 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 285 query_epoch 194496 query: query(info 0'0) -9812> 2014-08-13 18:06:08.329353 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193264) from replica 15 -9811> 2014-08-13 18:06:08.329392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info -9810> 2014-08-13 18:06:08.329405 7f5c835ef700 5 -- op tracker -- , seq: 501, time: 2014-08-13 18:06:08.329404, event: done, request: pg_query(24.20e epoch 194496) v3 -9809> 2014-08-13 18:06:08.329418 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9808> 2014-08-13 18:06:08.329421 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9807> 2014-08-13 18:06:08.329470 7f5c7bde3700 10 log is not dirty -9806> 2014-08-13 18:06:08.329481 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -9805> 2014-08-13 18:06:08.329503 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring -9804> 2014-08-13 18:06:08.329524 7f5c7bde3700 10 log is not dirty -9803> 2014-08-13 18:06:08.329531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 343 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) -9802> 2014-08-13 18:06:08.329551 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 343, already have info from that osd, ignoring -9801> 2014-08-13 18:06:08.329567 7f5c7bde3700 10 log is not dirty -9800> 2014-08-13 18:06:08.329589 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 crt=0'0 lcod 0'0 remapped] _activate_committed 194496 telling primary -9799> 2014-08-13 18:06:08.329615 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b2c620 don't know epoch, doing nothing -9798> 2014-08-13 18:06:08.329619 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.396 192.168.1.105:6814/41806 -- pg_info(1 pgs e194496:24.299) v4 -- ?+0 0x5351f80 -9797> 2014-08-13 18:06:08.329647 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] flushed -9796> 2014-08-13 18:06:08.329009 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring -9795> 2014-08-13 18:06:08.329439 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.31 192.168.1.103:6831/41157 7 ==== pg_query(24.1e4 epoch 194496) v3 ==== 144+0+0 (3307604369 0 0) 0x5424b00 con 0x4857bc0 -9794> 2014-08-13 18:06:08.329694 7f5c7c7e4700 10 log is not dirty -9793> 2014-08-13 18:06:08.329698 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9792> 2014-08-13 18:06:08.329704 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9791> 2014-08-13 18:06:08.329719 7f5c835ef700 20 osd.4 194496 _dispatch 0x5424b00 pg_query(24.1e4 epoch 194496) v3 -9790> 2014-08-13 18:06:08.329702 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -9789> 2014-08-13 18:06:08.329726 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.327109, event: header_read, request: pg_query(24.1e4 epoch 194496) v3 -9788> 2014-08-13 18:06:08.329735 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.327112, event: throttled, request: pg_query(24.1e4 epoch 194496) v3 -9787> 2014-08-13 18:06:08.329741 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.327399, event: all_read, request: pg_query(24.1e4 epoch 194496) v3 -9786> 2014-08-13 18:06:08.329728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -9785> 2014-08-13 18:06:08.329747 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.329696, event: dispatched, request: pg_query(24.1e4 epoch 194496) v3 -9784> 2014-08-13 18:06:08.329754 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.329753, event: waiting_for_osdmap, request: pg_query(24.1e4 epoch 194496) v3 -9783> 2014-08-13 18:06:08.329760 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.31 epoch 194496 -9782> 2014-08-13 18:06:08.329764 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5424b00 -9781> 2014-08-13 18:06:08.329769 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.329769, event: started, request: pg_query(24.1e4 epoch 194496) v3 -9780> 2014-08-13 18:06:08.329759 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] osd.327 has stray content: 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -9779> 2014-08-13 18:06:08.329785 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,330 -> 4,17,28,315,327,330 -9778> 2014-08-13 18:06:08.329804 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update -9777> 2014-08-13 18:06:08.329809 7f5c7c7e4700 10 log is not dirty -9776> 2014-08-13 18:06:08.329816 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 339 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) -9775> 2014-08-13 18:06:08.329838 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.339 activated and committed -9774> 2014-08-13 18:06:08.329856 7f5c7c7e4700 10 log is not dirty -9773> 2014-08-13 18:06:08.329863 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9772> 2014-08-13 18:06:08.329884 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9771> 2014-08-13 18:06:08.329901 7f5c7c7e4700 10 log is not dirty -9770> 2014-08-13 18:06:08.329886 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 crt=192888'458 lcod 0'0 inactive] _activate_committed 194496 telling primary -9769> 2014-08-13 18:06:08.329907 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 342 notify: (query_epoch:194496, epoch_sent:194496, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) -9768> 2014-08-13 18:06:08.329927 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x44265c0 don't know epoch, doing nothing -9767> 2014-08-13 18:06:08.329927 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 342, already have info from that osd, ignoring -9766> 2014-08-13 18:06:08.329932 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.345 192.168.1.110:6820/46337 -- pg_info(1 pgs e194496:22.29a) v4 -- ?+0 0x524d240 -9765> 2014-08-13 18:06:08.329944 7f5c7c7e4700 10 log is not dirty -9764> 2014-08-13 18:06:08.329574 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 31 query_epoch 194496 query: query(info 0'0) -9763> 2014-08-13 18:06:08.329969 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info -9762> 2014-08-13 18:06:08.329981 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] flushed -9761> 2014-08-13 18:06:08.329997 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 368 notify: (query_epoch:194496, epoch_sent:194496, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -9760> 2014-08-13 18:06:08.330038 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 368, already have info from that osd, ignoring -9759> 2014-08-13 18:06:08.330066 7f5c7c7e4700 10 log is not dirty -9758> 2014-08-13 18:06:08.330086 7f5c7bde3700 10 log is not dirty -9757> 2014-08-13 18:06:08.330073 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -9756> 2014-08-13 18:06:08.330096 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -9755> 2014-08-13 18:06:08.330114 7f5c7c7e4700 10 log is not dirty -9754> 2014-08-13 18:06:08.330100 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) -9753> 2014-08-13 18:06:08.330122 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9752> 2014-08-13 18:06:08.330105 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_query query(log 194487'0 ec=192688 les/c 194492/194372 194495/194496/192978) from replica 31 -9751> 2014-08-13 18:06:08.330124 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: got info from osd.323 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) -9750> 2014-08-13 18:06:08.330145 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9749> 2014-08-13 18:06:08.330163 7f5c7c7e4700 10 log is not dirty -9748> 2014-08-13 18:06:08.330147 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.958611 7 0.000284 -9747> 2014-08-13 18:06:08.330172 7f5c835ef700 5 -- op tracker -- , seq: 502, time: 2014-08-13 18:06:08.330171, event: done, request: pg_query(24.1e4 epoch 194496) v3 -9746> 2014-08-13 18:06:08.330170 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9745> 2014-08-13 18:06:08.330189 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9744> 2014-08-13 18:06:08.330192 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9743> 2014-08-13 18:06:08.330179 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9742> 2014-08-13 18:06:08.330201 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9741> 2014-08-13 18:06:08.330200 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.332 192.168.1.110:6806/42024 6 ==== pg_notify(24.1bb(2) epoch 194496) v5 ==== 870+0+0 (794495379 0 0) 0x588af40 con 0x4ab2d60 -9740> 2014-08-13 18:06:08.330217 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9739> 2014-08-13 18:06:08.330220 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9738> 2014-08-13 18:06:08.330223 7f5c835ef700 20 osd.4 194496 _dispatch 0x588af40 pg_notify(24.1bb(2) epoch 194496) v5 -9737> 2014-08-13 18:06:08.330219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194486 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9736> 2014-08-13 18:06:08.330229 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.328019, event: header_read, request: pg_notify(24.1bb(2) epoch 194496) v5 -9735> 2014-08-13 18:06:08.330237 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.328021, event: throttled, request: pg_notify(24.1bb(2) epoch 194496) v5 -9734> 2014-08-13 18:06:08.330243 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.328097, event: all_read, request: pg_notify(24.1bb(2) epoch 194496) v5 -9733> 2014-08-13 18:06:08.330231 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] requeue_ops -9732> 2014-08-13 18:06:08.330237 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'491 -> 194215'491 -9731> 2014-08-13 18:06:08.330253 7f5c7c7e4700 10 log is not dirty -9730> 2014-08-13 18:06:08.330250 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.330217, event: dispatched, request: pg_notify(24.1bb(2) epoch 194496) v5 -9729> 2014-08-13 18:06:08.330257 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.330257, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194496) v5 -9728> 2014-08-13 18:06:08.330263 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.332 -9727> 2014-08-13 18:06:08.330268 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588af40 -9726> 2014-08-13 18:06:08.330261 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] state: Activate Finished -9725> 2014-08-13 18:06:08.330273 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.330273, event: started, request: pg_notify(24.1bb(2) epoch 194496) v5 -9724> 2014-08-13 18:06:08.330262 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 365 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -9723> 2014-08-13 18:06:08.330297 7f5c835ef700 5 -- op tracker -- , seq: 503, time: 2014-08-13 18:06:08.330297, event: done, request: pg_notify(24.1bb(2) epoch 194496) v5 -9722> 2014-08-13 18:06:08.330288 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.365 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9721> 2014-08-13 18:06:08.330308 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9720> 2014-08-13 18:06:08.330311 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9719> 2014-08-13 18:06:08.330315 7f5c7c7e4700 10 log is not dirty -9718> 2014-08-13 18:06:08.330317 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.286 192.168.1.113:6820/19173 8 ==== pg_info(1 pgs e194496:22.3d5) v4 ==== 714+0+0 (3836386094 0 0) 0x4df0fc0 con 0x3cf2940 -9717> 2014-08-13 18:06:08.330330 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9716> 2014-08-13 18:06:08.330333 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9715> 2014-08-13 18:06:08.330336 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0fc0 pg_info(1 pgs e194496:22.3d5) v4 -9714> 2014-08-13 18:06:08.330322 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 10 notify: (query_epoch:194496, epoch_sent:194496, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) -9713> 2014-08-13 18:06:08.330342 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.329220, event: header_read, request: pg_info(1 pgs e194496:22.3d5) v4 -9712> 2014-08-13 18:06:08.330349 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.329222, event: throttled, request: pg_info(1 pgs e194496:22.3d5) v4 -9711> 2014-08-13 18:06:08.330346 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle_pg_notify from osd.10 -9710> 2014-08-13 18:06:08.330360 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.329305, event: all_read, request: pg_info(1 pgs e194496:22.3d5) v4 -9709> 2014-08-13 18:06:08.330366 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.330330, event: dispatched, request: pg_info(1 pgs e194496:22.3d5) v4 -9708> 2014-08-13 18:06:08.330373 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.330373, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3d5) v4 -9707> 2014-08-13 18:06:08.330378 7f5c7bde3700 10 log is not dirty -9706> 2014-08-13 18:06:08.330379 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d5) v4 from osd.286 -9705> 2014-08-13 18:06:08.330385 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0fc0 -9704> 2014-08-13 18:06:08.330390 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.330390, event: started, request: pg_info(1 pgs e194496:22.3d5) v4 -9703> 2014-08-13 18:06:08.330366 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] got dup osd.10 info 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9702> 2014-08-13 18:06:08.330405 7f5c7c7e4700 10 log is not dirty -9701> 2014-08-13 18:06:08.330410 7f5c835ef700 5 -- op tracker -- , seq: 504, time: 2014-08-13 18:06:08.330410, event: done, request: pg_info(1 pgs e194496:22.3d5) v4 -9700> 2014-08-13 18:06:08.330420 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9699> 2014-08-13 18:06:08.330424 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9698> 2014-08-13 18:06:08.330413 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 301 info: 22.22c( v 194215'514 lc 0'0 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) -9697> 2014-08-13 18:06:08.330413 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) -9696> 2014-08-13 18:06:08.330429 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.344 192.168.1.106:6812/6757 6 ==== pg_notify(24.181(38) epoch 194496) v5 ==== 2882+0+0 (3931023722 0 0) 0x588bb80 con 0x4aff220 -9695> 2014-08-13 18:06:08.330444 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9694> 2014-08-13 18:06:08.330447 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9693> 2014-08-13 18:06:08.330436 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.301 activated and committed -9692> 2014-08-13 18:06:08.330450 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bb80 pg_notify(24.181(38) epoch 194496) v5 -9691> 2014-08-13 18:06:08.330456 7f5c7c7e4700 10 log is not dirty -9690> 2014-08-13 18:06:08.330439 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: got info from osd.292 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) -9689> 2014-08-13 18:06:08.330456 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.327991, event: header_read, request: pg_notify(24.181(38) epoch 194496) v5 -9688> 2014-08-13 18:06:08.330462 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.947461 7 0.000262 -9687> 2014-08-13 18:06:08.330464 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 289 query_epoch 194496 query: query(info 0'0) -9686> 2014-08-13 18:06:08.330464 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.327993, event: throttled, request: pg_notify(24.181(38) epoch 194496) v5 -9685> 2014-08-13 18:06:08.330483 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9684> 2014-08-13 18:06:08.330486 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info -9683> 2014-08-13 18:06:08.330501 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9682> 2014-08-13 18:06:08.330486 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.328127, event: all_read, request: pg_notify(24.181(38) epoch 194496) v5 -9681> 2014-08-13 18:06:08.330521 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.330443, event: dispatched, request: pg_notify(24.181(38) epoch 194496) v5 -9680> 2014-08-13 18:06:08.330518 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194486 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9679> 2014-08-13 18:06:08.330529 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.330529, event: waiting_for_osdmap, request: pg_notify(24.181(38) epoch 194496) v5 -9678> 2014-08-13 18:06:08.330547 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.344 -9677> 2014-08-13 18:06:08.330534 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'513 -> 194215'513 -9676> 2014-08-13 18:06:08.330564 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] state: Activate Finished -9675> 2014-08-13 18:06:08.330550 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bb80 -9674> 2014-08-13 18:06:08.330587 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.330587, event: started, request: pg_notify(24.181(38) epoch 194496) v5 -9673> 2014-08-13 18:06:08.330605 7f5c7c7e4700 10 log is not dirty -9672> 2014-08-13 18:06:08.330614 7f5c835ef700 5 -- op tracker -- , seq: 505, time: 2014-08-13 18:06:08.330614, event: done, request: pg_notify(24.181(38) epoch 194496) v5 -9671> 2014-08-13 18:06:08.330627 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9670> 2014-08-13 18:06:08.330630 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9669> 2014-08-13 18:06:08.330613 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 45 info: 22.113( v 194215'528 lc 0'0 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) -9668> 2014-08-13 18:06:08.330643 7f5c7bde3700 10 log is not dirty -9667> 2014-08-13 18:06:08.330636 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] state: peer osd.45 activated and committed -9666> 2014-08-13 18:06:08.330653 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194493/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9665> 2014-08-13 18:06:08.330668 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) -9664> 2014-08-13 18:06:08.330650 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.328 192.168.1.104:6812/64303 10 ==== pg_info(1 pgs e194496:22.1a0) v4 ==== 714+0+0 (1129115413 0 0) 0x5351c00 con 0x4afdee0 -9663> 2014-08-13 18:06:08.330698 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9662> 2014-08-13 18:06:08.330702 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9661> 2014-08-13 18:06:08.330704 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351c00 pg_info(1 pgs e194496:22.1a0) v4 -9660> 2014-08-13 18:06:08.330696 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.51 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) -9659> 2014-08-13 18:06:08.330673 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9658> 2014-08-13 18:06:08.330725 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.327391, event: header_read, request: pg_info(1 pgs e194496:22.1a0) v4 -9657> 2014-08-13 18:06:08.330718 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.968160 7 0.000254 -9656> 2014-08-13 18:06:08.330735 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing -9655> 2014-08-13 18:06:08.330734 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.327393, event: throttled, request: pg_info(1 pgs e194496:22.1a0) v4 -9654> 2014-08-13 18:06:08.330741 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.327467, event: all_read, request: pg_info(1 pgs e194496:22.1a0) v4 -9653> 2014-08-13 18:06:08.330737 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9652> 2014-08-13 18:06:08.330740 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194496:22.113) v4 -- ?+0 0x4a43480 -9651> 2014-08-13 18:06:08.330756 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9650> 2014-08-13 18:06:08.330747 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.330697, event: dispatched, request: pg_info(1 pgs e194496:22.1a0) v4 -9649> 2014-08-13 18:06:08.330743 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 crt=192888'463 lcod 0'0 inactive] _activate_committed 194496 telling primary -9648> 2014-08-13 18:06:08.330775 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c594a0 don't know epoch, doing nothing -9647> 2014-08-13 18:06:08.330778 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.330777, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1a0) v4 -9646> 2014-08-13 18:06:08.330773 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194486 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9645> 2014-08-13 18:06:08.330780 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.339 192.168.1.110:6818/45268 -- pg_info(1 pgs e194496:22.113) v4 -- ?+0 0x5889c00 -9644> 2014-08-13 18:06:08.330785 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1a0) v4 from osd.328 -9643> 2014-08-13 18:06:08.330795 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351c00 -9642> 2014-08-13 18:06:08.330800 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.330800, event: started, request: pg_info(1 pgs e194496:22.1a0) v4 -9641> 2014-08-13 18:06:08.330791 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'501 -> 194215'501 -9640> 2014-08-13 18:06:08.330813 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x49a2520 don't know epoch, doing nothing -9639> 2014-08-13 18:06:08.330807 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2937 -9638> 2014-08-13 18:06:08.330817 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] state: Activate Finished -9637> 2014-08-13 18:06:08.330829 7f5c835ef700 5 -- op tracker -- , seq: 506, time: 2014-08-13 18:06:08.330828, event: done, request: pg_info(1 pgs e194496:22.1a0) v4 -9636> 2014-08-13 18:06:08.330839 7f5c7c7e4700 10 log is not dirty -9635> 2014-08-13 18:06:08.330842 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9634> 2014-08-13 18:06:08.330821 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.323 192.168.1.106:6816/8118 -- pg_info(1 pgs e194496:22.12a) v4 -- ?+0 0x5888380 -9633> 2014-08-13 18:06:08.330845 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9632> 2014-08-13 18:06:08.330853 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.338 192.168.1.106:6814/25893 6 ==== pg_info(1 pgs e194496:22.3d9) v4 ==== 714+0+0 (1864472578 0 0) 0x53571c0 con 0x4b1b5a0 -9631> 2014-08-13 18:06:08.330867 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9630> 2014-08-13 18:06:08.330871 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9629> 2014-08-13 18:06:08.330873 7f5c835ef700 20 osd.4 194496 _dispatch 0x53571c0 pg_info(1 pgs e194496:22.3d9) v4 -9628> 2014-08-13 18:06:08.330879 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.329535, event: header_read, request: pg_info(1 pgs e194496:22.3d9) v4 -9627> 2014-08-13 18:06:08.330867 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] flushed -9626> 2014-08-13 18:06:08.330885 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.329536, event: throttled, request: pg_info(1 pgs e194496:22.3d9) v4 -9625> 2014-08-13 18:06:08.330905 7f5c7bde3700 10 log is not dirty -9624> 2014-08-13 18:06:08.330892 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.329619, event: all_read, request: pg_info(1 pgs e194496:22.3d9) v4 -9623> 2014-08-13 18:06:08.330927 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.330867, event: dispatched, request: pg_info(1 pgs e194496:22.3d9) v4 -9622> 2014-08-13 18:06:08.330930 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(info 0'0) -9621> 2014-08-13 18:06:08.330953 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info -9620> 2014-08-13 18:06:08.330963 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -9619> 2014-08-13 18:06:08.330976 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4854620 don't know epoch, doing nothing -9618> 2014-08-13 18:06:08.330979 7f5c7c7e4700 7 osd.4 194496 do_notify osd 15 on 1 PGs -9617> 2014-08-13 18:06:08.330983 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(40) epoch 194496) v5 -- ?+0 0x588fa80 con 0x4854620 -9616> 2014-08-13 18:06:08.331035 7f5c7bde3700 10 log is not dirty -9615> 2014-08-13 18:06:08.330952 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.330947, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3d9) v4 -9614> 2014-08-13 18:06:08.331155 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d9) v4 from osd.338 -9613> 2014-08-13 18:06:08.331163 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53571c0 -9612> 2014-08-13 18:06:08.331169 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.331169, event: started, request: pg_info(1 pgs e194496:22.3d9) v4 -9611> 2014-08-13 18:06:08.331195 7f5c835ef700 5 -- op tracker -- , seq: 507, time: 2014-08-13 18:06:08.331195, event: done, request: pg_info(1 pgs e194496:22.3d9) v4 -9610> 2014-08-13 18:06:08.331187 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing -9609> 2014-08-13 18:06:08.331204 7f5c7c7e4700 7 osd.4 194496 do_notify osd 289 on 1 PGs -9608> 2014-08-13 18:06:08.331185 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 344 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) -9607> 2014-08-13 18:06:08.331215 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9606> 2014-08-13 18:06:08.331219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9605> 2014-08-13 18:06:08.331209 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(49) epoch 194496) v5 -- ?+0 0x588c440 con 0x445f7a0 -9604> 2014-08-13 18:06:08.331226 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.331 192.168.1.104:6816/837 7 ==== pg_notify(24.1bb epoch 194496) v5 ==== 694+0+0 (660771902 0 0) 0x588ee40 con 0x4b12940 -9603> 2014-08-13 18:06:08.331220 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -9602> 2014-08-13 18:06:08.331242 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9601> 2014-08-13 18:06:08.331245 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9600> 2014-08-13 18:06:08.331247 7f5c835ef700 20 osd.4 194496 _dispatch 0x588ee40 pg_notify(24.1bb epoch 194496) v5 -9599> 2014-08-13 18:06:08.331252 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.327721, event: header_read, request: pg_notify(24.1bb epoch 194496) v5 -9598> 2014-08-13 18:06:08.331260 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.327722, event: throttled, request: pg_notify(24.1bb epoch 194496) v5 -9597> 2014-08-13 18:06:08.331265 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.327790, event: all_read, request: pg_notify(24.1bb epoch 194496) v5 -9596> 2014-08-13 18:06:08.331254 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 unchanged -9595> 2014-08-13 18:06:08.331271 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.331241, event: dispatched, request: pg_notify(24.1bb epoch 194496) v5 -9594> 2014-08-13 18:06:08.331276 7f5c7bde3700 10 log is not dirty -9593> 2014-08-13 18:06:08.331278 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.331278, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194496) v5 -9592> 2014-08-13 18:06:08.331284 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.331 -9591> 2014-08-13 18:06:08.331287 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588ee40 -9590> 2014-08-13 18:06:08.331291 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.331291, event: started, request: pg_notify(24.1bb epoch 194496) v5 -9589> 2014-08-13 18:06:08.331310 7f5c835ef700 5 -- op tracker -- , seq: 508, time: 2014-08-13 18:06:08.331310, event: done, request: pg_notify(24.1bb epoch 194496) v5 -9588> 2014-08-13 18:06:08.331320 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9587> 2014-08-13 18:06:08.331329 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9586> 2014-08-13 18:06:08.331335 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.320 192.168.1.106:6820/10050 12 ==== pg_notify(24.181(17) epoch 194496) v5 ==== 1715+0+0 (3046214134 0 0) 0x4bf40c0 con 0x4b12520 -9585> 2014-08-13 18:06:08.331349 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9584> 2014-08-13 18:06:08.331351 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9583> 2014-08-13 18:06:08.331353 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf40c0 pg_notify(24.181(17) epoch 194496) v5 -9582> 2014-08-13 18:06:08.331358 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.327530, event: header_read, request: pg_notify(24.181(17) epoch 194496) v5 -9581> 2014-08-13 18:06:08.331365 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.327531, event: throttled, request: pg_notify(24.181(17) epoch 194496) v5 -9580> 2014-08-13 18:06:08.331371 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.327770, event: all_read, request: pg_notify(24.181(17) epoch 194496) v5 -9579> 2014-08-13 18:06:08.331377 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.331348, event: dispatched, request: pg_notify(24.181(17) epoch 194496) v5 -9578> 2014-08-13 18:06:08.331356 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 crt=192888'477 lcod 0'0 inactive] _activate_committed 194496 telling primary -9577> 2014-08-13 18:06:08.331384 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.331383, event: waiting_for_osdmap, request: pg_notify(24.181(17) epoch 194496) v5 -9576> 2014-08-13 18:06:08.331390 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.320 -9575> 2014-08-13 18:06:08.331393 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf40c0 -9574> 2014-08-13 18:06:08.331397 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.331397, event: started, request: pg_notify(24.181(17) epoch 194496) v5 -9573> 2014-08-13 18:06:08.331402 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -9572> 2014-08-13 18:06:08.331407 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 don't know epoch, doing nothing -9571> 2014-08-13 18:06:08.331411 7f5c7c7e4700 7 osd.4 194496 do_notify osd 343 on 1 PGs -9570> 2014-08-13 18:06:08.331397 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 338 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) -9569> 2014-08-13 18:06:08.331419 7f5c835ef700 5 -- op tracker -- , seq: 509, time: 2014-08-13 18:06:08.331419, event: done, request: pg_notify(24.181(17) epoch 194496) v5 -9568> 2014-08-13 18:06:08.331412 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.292 192.168.1.113:6812/16056 -- pg_info(1 pgs e194496:22.3ad) v4 -- ?+0 0x5889340 -9567> 2014-08-13 18:06:08.331415 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(36) epoch 194496) v5 -- ?+0 0x5889f80 con 0x48506e0 -9566> 2014-08-13 18:06:08.331429 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9565> 2014-08-13 18:06:08.331433 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9564> 2014-08-13 18:06:08.331423 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 338, already have info from that osd, ignoring -9563> 2014-08-13 18:06:08.331445 7f5c7bde3700 10 log is not dirty -9562> 2014-08-13 18:06:08.331447 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] flushed -9561> 2014-08-13 18:06:08.331466 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 397 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) -9560> 2014-08-13 18:06:08.331489 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 397, already have info from that osd, ignoring -9559> 2014-08-13 18:06:08.331517 7f5c7bde3700 10 log is not dirty -9558> 2014-08-13 18:06:08.331501 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -9557> 2014-08-13 18:06:08.331511 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 crt=192888'461 lcod 0'0 inactive] _activate_committed 194496 telling primary -9556> 2014-08-13 18:06:08.331523 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 396 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) -9555> 2014-08-13 18:06:08.331539 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x3cf1600 already has epoch 194496 -9554> 2014-08-13 18:06:08.331531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring -9553> 2014-08-13 18:06:08.331543 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.51 192.168.1.112:6832/6655 -- pg_info(1 pgs e194496:22.537) v4 -- ?+0 0x588b800 -9552> 2014-08-13 18:06:08.331558 7f5c7c7e4700 10 log is not dirty -9551> 2014-08-13 18:06:08.331544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 396, already have info from that osd, ignoring -9550> 2014-08-13 18:06:08.331564 7f5c7bde3700 10 log is not dirty -9549> 2014-08-13 18:06:08.331574 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] flushed -9548> 2014-08-13 18:06:08.331586 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -9547> 2014-08-13 18:06:08.331614 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -9546> 2014-08-13 18:06:08.331643 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] osd.376 has stray content: 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -9545> 2014-08-13 18:06:08.331667 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351 -> 4,55,59,351,376 -9544> 2014-08-13 18:06:08.331686 7f5c7c7e4700 20 osd.4 194496 need_heartbeat_peer_update -9543> 2014-08-13 18:06:08.331691 7f5c7c7e4700 10 log is not dirty -9542> 2014-08-13 18:06:08.331698 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 324 notify: (query_epoch:194496, epoch_sent:194496, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) -9541> 2014-08-13 18:06:08.331718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 324, already have info from that osd, ignoring -9540> 2014-08-13 18:06:08.331734 7f5c7c7e4700 10 log is not dirty -9539> 2014-08-13 18:06:08.331739 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -9538> 2014-08-13 18:06:08.331762 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -9537> 2014-08-13 18:06:08.331794 7f5c7c7e4700 10 log is not dirty -9536> 2014-08-13 18:06:08.331801 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9535> 2014-08-13 18:06:08.331822 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9534> 2014-08-13 18:06:08.331838 7f5c7c7e4700 10 log is not dirty -9533> 2014-08-13 18:06:08.331844 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 310 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) -9532> 2014-08-13 18:06:08.331862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.310 activated and committed -9531> 2014-08-13 18:06:08.331879 7f5c7c7e4700 10 log is not dirty -9530> 2014-08-13 18:06:08.331886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 47 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) -9529> 2014-08-13 18:06:08.331903 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.47 activated and committed -9528> 2014-08-13 18:06:08.331920 7f5c7c7e4700 10 log is not dirty -9527> 2014-08-13 18:06:08.331925 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 62 info: 22.619( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) -9526> 2014-08-13 18:06:08.331943 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: peer osd.62 activated and committed -9525> 2014-08-13 18:06:08.331943 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.373 192.168.1.108:6840/7435 14 ==== pg_info(1 pgs e194496:22.1fb) v4 ==== 714+0+0 (3631941884 0 0) 0x5356c80 con 0x4b111e0 -9524> 2014-08-13 18:06:08.331966 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9523> 2014-08-13 18:06:08.331970 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9522> 2014-08-13 18:06:08.331960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194493/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9521> 2014-08-13 18:06:08.331973 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356c80 pg_info(1 pgs e194496:22.1fb) v4 -9520> 2014-08-13 18:06:08.331980 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.328166, event: header_read, request: pg_info(1 pgs e194496:22.1fb) v4 -9519> 2014-08-13 18:06:08.331988 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.328168, event: throttled, request: pg_info(1 pgs e194496:22.1fb) v4 -9518> 2014-08-13 18:06:08.331975 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9517> 2014-08-13 18:06:08.331994 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.328239, event: all_read, request: pg_info(1 pgs e194496:22.1fb) v4 -9516> 2014-08-13 18:06:08.332000 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.331965, event: dispatched, request: pg_info(1 pgs e194496:22.1fb) v4 -9515> 2014-08-13 18:06:08.332007 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.332007, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1fb) v4 -9514> 2014-08-13 18:06:08.332019 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4424360 don't know epoch, doing nothing -9513> 2014-08-13 18:06:08.332043 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1fb) v4 from osd.373 -9512> 2014-08-13 18:06:08.332050 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356c80 -9511> 2014-08-13 18:06:08.332056 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.332056, event: started, request: pg_info(1 pgs e194496:22.1fb) v4 -9510> 2014-08-13 18:06:08.332050 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.62 192.168.1.112:6824/5505 -- pg_info(1 pgs e194496:22.619) v4 -- ?+0 0x588eac0 -9509> 2014-08-13 18:06:08.332081 7f5c835ef700 5 -- op tracker -- , seq: 510, time: 2014-08-13 18:06:08.332081, event: done, request: pg_info(1 pgs e194496:22.1fb) v4 -9508> 2014-08-13 18:06:08.332093 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9507> 2014-08-13 18:06:08.332103 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9506> 2014-08-13 18:06:08.332113 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing -9505> 2014-08-13 18:06:08.332101 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) -9504> 2014-08-13 18:06:08.332118 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.65 192.168.1.111:6842/17345 -- pg_info(1 pgs e194496:22.619) v4 -- ?+0 0x588b640 -9503> 2014-08-13 18:06:08.332142 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2737 -9502> 2014-08-13 18:06:08.332133 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring -9501> 2014-08-13 18:06:08.332171 7f5c7bde3700 10 log is not dirty -9500> 2014-08-13 18:06:08.332178 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 379 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -9499> 2014-08-13 18:06:08.332176 7f5c7c7e4700 10 log is not dirty -9498> 2014-08-13 18:06:08.332210 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -9497> 2014-08-13 18:06:08.332241 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] osd.379 has stray content: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -9496> 2014-08-13 18:06:08.332245 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 313 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) -9495> 2014-08-13 18:06:08.332266 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348 -> 4,331,332,337,348,379 -9494> 2014-08-13 18:06:08.332287 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update -9493> 2014-08-13 18:06:08.332292 7f5c7bde3700 10 log is not dirty -9492> 2014-08-13 18:06:08.332273 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.313 activated and committed -9491> 2014-08-13 18:06:08.332299 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) -9490> 2014-08-13 18:06:08.332110 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 7 ==== pg_info(1 pgs e194496:22.3b9) v4 ==== 714+0+0 (1406044147 0 0) 0x5354440 con 0x4b10b00 -9489> 2014-08-13 18:06:08.332327 7f5c7c7e4700 10 log is not dirty -9488> 2014-08-13 18:06:08.332326 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9487> 2014-08-13 18:06:08.332332 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9486> 2014-08-13 18:06:08.332319 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 327, already have info from that osd, ignoring -9485> 2014-08-13 18:06:08.332334 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354440 pg_info(1 pgs e194496:22.3b9) v4 -9484> 2014-08-13 18:06:08.332340 7f5c7bde3700 10 log is not dirty -9483> 2014-08-13 18:06:08.332342 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.328579, event: header_read, request: pg_info(1 pgs e194496:22.3b9) v4 -9482> 2014-08-13 18:06:08.332336 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 5 info: 22.1f1( v 194215'493 lc 0'0 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) -9481> 2014-08-13 18:06:08.332351 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.328581, event: throttled, request: pg_info(1 pgs e194496:22.3b9) v4 -9480> 2014-08-13 18:06:08.332358 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.328672, event: all_read, request: pg_info(1 pgs e194496:22.3b9) v4 -9479> 2014-08-13 18:06:08.332364 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.332324, event: dispatched, request: pg_info(1 pgs e194496:22.3b9) v4 -9478> 2014-08-13 18:06:08.332359 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] state: peer osd.5 activated and committed -9477> 2014-08-13 18:06:08.332348 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 339 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -9476> 2014-08-13 18:06:08.332380 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.332380, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3b9) v4 -9475> 2014-08-13 18:06:08.332387 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3b9) v4 from osd.397 -9474> 2014-08-13 18:06:08.332377 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9473> 2014-08-13 18:06:08.332394 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354440 -9472> 2014-08-13 18:06:08.332399 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.332399, event: started, request: pg_info(1 pgs e194496:22.3b9) v4 -9471> 2014-08-13 18:06:08.332385 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 339, already have info from that osd, ignoring -9470> 2014-08-13 18:06:08.332394 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9469> 2014-08-13 18:06:08.332408 7f5c7bde3700 10 log is not dirty -9468> 2014-08-13 18:06:08.332418 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -9467> 2014-08-13 18:06:08.332430 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 -9466> 2014-08-13 18:06:08.332434 7f5c7bde3700 7 osd.4 194496 do_notify osd 6 on 1 PGs -9465> 2014-08-13 18:06:08.332434 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x449d280 don't know epoch, doing nothing -9464> 2014-08-13 18:06:08.332437 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(36) epoch 194496) v5 -- ?+0 0x5357700 con 0x4af98c0 -9463> 2014-08-13 18:06:08.332439 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.5 192.168.1.103:6827/40192 -- pg_info(1 pgs e194496:22.1f1) v4 -- ?+0 0x588de80 -9462> 2014-08-13 18:06:08.332513 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x49a5960 don't know epoch, doing nothing -9461> 2014-08-13 18:06:08.332519 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.47 192.168.1.111:6828/12102 -- pg_info(1 pgs e194496:22.1f1) v4 -- ?+0 0x5889180 -9460> 2014-08-13 18:06:08.332542 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3010 -9459> 2014-08-13 18:06:08.332579 7f5c7c7e4700 10 log is not dirty -9458> 2014-08-13 18:06:08.332650 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 -9457> 2014-08-13 18:06:08.332661 7f5c7bde3700 7 osd.4 194496 do_notify osd 31 on 1 PGs -9456> 2014-08-13 18:06:08.332665 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(48) epoch 194496) v5 -- ?+0 0x5354600 con 0x4857bc0 -9455> 2014-08-13 18:06:08.332644 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -9454> 2014-08-13 18:06:08.332692 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing -9453> 2014-08-13 18:06:08.332696 7f5c7bde3700 7 osd.4 194496 do_notify osd 285 on 1 PGs -9452> 2014-08-13 18:06:08.332699 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(36) epoch 194496) v5 -- ?+0 0x5351a40 con 0x445e5c0 -9451> 2014-08-13 18:06:08.332692 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 303, already have info from that osd, ignoring -9450> 2014-08-13 18:06:08.332712 7f5c7c7e4700 10 log is not dirty -9449> 2014-08-13 18:06:08.332705 7f5c835ef700 5 -- op tracker -- , seq: 511, time: 2014-08-13 18:06:08.332705, event: done, request: pg_info(1 pgs e194496:22.3b9) v4 -9448> 2014-08-13 18:06:08.332730 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9447> 2014-08-13 18:06:08.332734 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9446> 2014-08-13 18:06:08.332722 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) -9445> 2014-08-13 18:06:08.332741 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: got info from osd.376 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) -9444> 2014-08-13 18:06:08.332742 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.396 192.168.1.105:6814/41806 8 ==== pg_info(1 pgs e194496:24.299) v4 ==== 744+0+0 (2522468547 0 0) 0x5351f80 con 0x4b2c620 -9443> 2014-08-13 18:06:08.332775 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9442> 2014-08-13 18:06:08.332780 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9441> 2014-08-13 18:06:08.332764 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.958479 7 0.000290 -9440> 2014-08-13 18:06:08.332782 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351f80 pg_info(1 pgs e194496:24.299) v4 -9439> 2014-08-13 18:06:08.332769 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 293 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) -9438> 2014-08-13 18:06:08.332788 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.330592, event: header_read, request: pg_info(1 pgs e194496:24.299) v4 -9437> 2014-08-13 18:06:08.332787 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive -9436> 2014-08-13 18:06:08.332797 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.330594, event: throttled, request: pg_info(1 pgs e194496:24.299) v4 -9435> 2014-08-13 18:06:08.332807 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.330670, event: all_read, request: pg_info(1 pgs e194496:24.299) v4 -9434> 2014-08-13 18:06:08.332798 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 293, already have info from that osd, ignoring -9433> 2014-08-13 18:06:08.332813 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.332774, event: dispatched, request: pg_info(1 pgs e194496:24.299) v4 -9432> 2014-08-13 18:06:08.332821 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.332821, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:24.299) v4 -9431> 2014-08-13 18:06:08.332825 7f5c7bde3700 10 log is not dirty -9430> 2014-08-13 18:06:08.332827 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:24.299) v4 from osd.396 -9429> 2014-08-13 18:06:08.332834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 369 notify: (query_epoch:194496, epoch_sent:194496, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -9428> 2014-08-13 18:06:08.332854 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 369, already have info from that osd, ignoring -9427> 2014-08-13 18:06:08.332871 7f5c7bde3700 10 log is not dirty -9426> 2014-08-13 18:06:08.332877 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -9425> 2014-08-13 18:06:08.332812 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -9424> 2014-08-13 18:06:08.332896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 61, already have info from that osd, ignoring -9423> 2014-08-13 18:06:08.332915 7f5c7bde3700 10 log is not dirty -9422> 2014-08-13 18:06:08.332902 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194486 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -9421> 2014-08-13 18:06:08.332921 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] activate - no missing, moving last_complete 194215'485 -> 194215'485 -9420> 2014-08-13 18:06:08.332921 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 289 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) -9419> 2014-08-13 18:06:08.332944 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 289, already have info from that osd, ignoring -9418> 2014-08-13 18:06:08.332946 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] state: Activate Finished -9417> 2014-08-13 18:06:08.332961 7f5c7bde3700 10 log is not dirty -9416> 2014-08-13 18:06:08.332968 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 286 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d5( v 194215'466 (0'0,194215'466] local-les=194492 n=25 ec=183096 les/c 194492/194489 194495/194495/194495)) -9415> 2014-08-13 18:06:08.332832 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351f80 -9414> 2014-08-13 18:06:08.332990 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.332989, event: started, request: pg_info(1 pgs e194496:24.299) v4 -9413> 2014-08-13 18:06:08.332986 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 286, already have info from that osd, ignoring -9412> 2014-08-13 18:06:08.333008 7f5c7bde3700 10 log is not dirty -9411> 2014-08-13 18:06:08.333021 7f5c835ef700 5 -- op tracker -- , seq: 512, time: 2014-08-13 18:06:08.333021, event: done, request: pg_info(1 pgs e194496:24.299) v4 -9410> 2014-08-13 18:06:08.333015 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) -9409> 2014-08-13 18:06:08.333036 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9408> 2014-08-13 18:06:08.333040 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9407> 2014-08-13 18:06:08.333034 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 376, already have info from that osd, ignoring -9406> 2014-08-13 18:06:08.333052 7f5c7bde3700 10 log is not dirty -9405> 2014-08-13 18:06:08.333047 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 16 ==== pg_info(1 pgs e194496:22.3d9) v4 ==== 714+0+0 (2929729767 0 0) 0x53563c0 con 0x4b2bc80 -9404> 2014-08-13 18:06:08.333063 7f5c7c7e4700 10 log is not dirty -9403> 2014-08-13 18:06:08.333063 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9402> 2014-08-13 18:06:08.333066 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9401> 2014-08-13 18:06:08.333069 7f5c835ef700 20 osd.4 194496 _dispatch 0x53563c0 pg_info(1 pgs e194496:22.3d9) v4 -9400> 2014-08-13 18:06:08.333057 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) -9399> 2014-08-13 18:06:08.333075 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.330028, event: header_read, request: pg_info(1 pgs e194496:22.3d9) v4 -9398> 2014-08-13 18:06:08.333082 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.330030, event: throttled, request: pg_info(1 pgs e194496:22.3d9) v4 -9397> 2014-08-13 18:06:08.333077 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.58 activated and committed -9396> 2014-08-13 18:06:08.333088 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.330125, event: all_read, request: pg_info(1 pgs e194496:22.3d9) v4 -9395> 2014-08-13 18:06:08.333095 7f5c7bde3700 10 log is not dirty -9394> 2014-08-13 18:06:08.333095 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.333062, event: dispatched, request: pg_info(1 pgs e194496:22.3d9) v4 -9393> 2014-08-13 18:06:08.333097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 324 info: 22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) -9392> 2014-08-13 18:06:08.333102 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) -9391> 2014-08-13 18:06:08.333118 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.333118, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3d9) v4 -9390> 2014-08-13 18:06:08.333126 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d9) v4 from osd.376 -9389> 2014-08-13 18:06:08.333131 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53563c0 -9388> 2014-08-13 18:06:08.333122 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] state: peer osd.324 activated and committed -9387> 2014-08-13 18:06:08.333136 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.333136, event: started, request: pg_info(1 pgs e194496:22.3d9) v4 -9386> 2014-08-13 18:06:08.333126 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -9385> 2014-08-13 18:06:08.333141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194492/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9384> 2014-08-13 18:06:08.333157 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9383> 2014-08-13 18:06:08.333159 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.55 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -9382> 2014-08-13 18:06:08.333180 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f69e0 don't know epoch, doing nothing -9381> 2014-08-13 18:06:08.333184 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.301 192.168.1.113:6814/16727 -- pg_info(1 pgs e194496:22.22c) v4 -- ?+0 0x588f380 -9380> 2014-08-13 18:06:08.333195 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,296,320,344 -> 4,55,296,320,344 -9379> 2014-08-13 18:06:08.333216 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update -9378> 2014-08-13 18:06:08.333221 7f5c7bde3700 10 log is not dirty -9377> 2014-08-13 18:06:08.333217 7f5c835ef700 5 -- op tracker -- , seq: 513, time: 2014-08-13 18:06:08.333216, event: done, request: pg_info(1 pgs e194496:22.3d9) v4 -9376> 2014-08-13 18:06:08.333231 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9375> 2014-08-13 18:06:08.333235 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9374> 2014-08-13 18:06:08.333241 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.379 192.168.1.108:6830/4861 6 ==== pg_notify(24.1bb(40) epoch 194496) v5 ==== 3004+0+0 (2862996617 0 0) 0x5352d80 con 0x4b2d120 -9373> 2014-08-13 18:06:08.333284 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9372> 2014-08-13 18:06:08.333287 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9371> 2014-08-13 18:06:08.333290 7f5c835ef700 20 osd.4 194496 _dispatch 0x5352d80 pg_notify(24.1bb(40) epoch 194496) v5 -9370> 2014-08-13 18:06:08.333322 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.328092, event: header_read, request: pg_notify(24.1bb(40) epoch 194496) v5 -9369> 2014-08-13 18:06:08.333330 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.328095, event: throttled, request: pg_notify(24.1bb(40) epoch 194496) v5 -9368> 2014-08-13 18:06:08.333332 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 -9367> 2014-08-13 18:06:08.333336 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.328244, event: all_read, request: pg_notify(24.1bb(40) epoch 194496) v5 -9366> 2014-08-13 18:06:08.333343 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.333283, event: dispatched, request: pg_notify(24.1bb(40) epoch 194496) v5 -9365> 2014-08-13 18:06:08.333340 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.324 192.168.1.104:6814/65209 -- pg_info(1 pgs e194496:22.22c) v4 -- ?+0 0x588e3c0 -9364> 2014-08-13 18:06:08.333349 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.333349, event: waiting_for_osdmap, request: pg_notify(24.1bb(40) epoch 194496) v5 -9363> 2014-08-13 18:06:08.333355 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.379 -9362> 2014-08-13 18:06:08.333359 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5352d80 -9361> 2014-08-13 18:06:08.333365 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.333364, event: started, request: pg_notify(24.1bb(40) epoch 194496) v5 -9360> 2014-08-13 18:06:08.333412 7f5c835ef700 5 -- op tracker -- , seq: 514, time: 2014-08-13 18:06:08.333412, event: done, request: pg_notify(24.1bb(40) epoch 194496) v5 -9359> 2014-08-13 18:06:08.333427 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9358> 2014-08-13 18:06:08.333430 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9357> 2014-08-13 18:06:08.333456 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2834 -9356> 2014-08-13 18:06:08.333456 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.309 192.168.1.101:6814/23703 7 ==== pg_info(1 pgs e194496:22.481) v4 ==== 714+0+0 (1598751114 0 0) 0x4df0e00 con 0x4c59ce0 -9355> 2014-08-13 18:06:08.333490 7f5c7c7e4700 10 log is not dirty -9354> 2014-08-13 18:06:08.333489 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9353> 2014-08-13 18:06:08.333493 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9352> 2014-08-13 18:06:08.333496 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0e00 pg_info(1 pgs e194496:22.481) v4 -9351> 2014-08-13 18:06:08.333501 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.331730, event: header_read, request: pg_info(1 pgs e194496:22.481) v4 -9350> 2014-08-13 18:06:08.333499 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 28 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -9349> 2014-08-13 18:06:08.333509 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.331733, event: throttled, request: pg_info(1 pgs e194496:22.481) v4 -9348> 2014-08-13 18:06:08.333521 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.331804, event: all_read, request: pg_info(1 pgs e194496:22.481) v4 -9347> 2014-08-13 18:06:08.333527 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.333489, event: dispatched, request: pg_info(1 pgs e194496:22.481) v4 -9346> 2014-08-13 18:06:08.333534 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.333533, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.481) v4 -9345> 2014-08-13 18:06:08.333523 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9344> 2014-08-13 18:06:08.333540 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.481) v4 from osd.309 -9343> 2014-08-13 18:06:08.333545 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0e00 -9342> 2014-08-13 18:06:08.333550 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.333549, event: started, request: pg_info(1 pgs e194496:22.481) v4 -9341> 2014-08-13 18:06:08.333555 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged -9340> 2014-08-13 18:06:08.333546 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 crt=192888'461 lcod 0'0 inactive] _activate_committed 194496 telling primary -9339> 2014-08-13 18:06:08.333571 7f5c835ef700 5 -- op tracker -- , seq: 515, time: 2014-08-13 18:06:08.333570, event: done, request: pg_info(1 pgs e194496:22.481) v4 -9338> 2014-08-13 18:06:08.333577 7f5c7c7e4700 10 log is not dirty -9337> 2014-08-13 18:06:08.333582 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9336> 2014-08-13 18:06:08.333585 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9335> 2014-08-13 18:06:08.333588 7f5c88628700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing -9334> 2014-08-13 18:06:08.333584 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 61 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -9333> 2014-08-13 18:06:08.333592 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194496:22.226) v4 -- ?+0 0x588b100 -9332> 2014-08-13 18:06:08.333607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9331> 2014-08-13 18:06:08.333592 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.307 192.168.1.109:6804/31016 3 ==== pg_notify(24.181(45) epoch 194496) v5 ==== 3283+0+0 (500605647 0 0) 0x588c280 con 0x43f3700 -9330> 2014-08-13 18:06:08.333618 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] flushed -9329> 2014-08-13 18:06:08.333635 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9328> 2014-08-13 18:06:08.333640 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9327> 2014-08-13 18:06:08.333642 7f5c835ef700 20 osd.4 194496 _dispatch 0x588c280 pg_notify(24.181(45) epoch 194496) v5 -9326> 2014-08-13 18:06:08.333634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged -9325> 2014-08-13 18:06:08.333657 7f5c7c7e4700 10 log is not dirty -9324> 2014-08-13 18:06:08.333648 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.327835, event: header_read, request: pg_notify(24.181(45) epoch 194496) v5 -9323> 2014-08-13 18:06:08.333661 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.327837, event: throttled, request: pg_notify(24.181(45) epoch 194496) v5 -9322> 2014-08-13 18:06:08.333668 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.327982, event: all_read, request: pg_notify(24.181(45) epoch 194496) v5 -9321> 2014-08-13 18:06:08.333673 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.333634, event: dispatched, request: pg_notify(24.181(45) epoch 194496) v5 -9320> 2014-08-13 18:06:08.338814 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.338813, event: waiting_for_osdmap, request: pg_notify(24.181(45) epoch 194496) v5 -9319> 2014-08-13 18:06:08.338834 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.307 -9318> 2014-08-13 18:06:08.338839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588c280 -9317> 2014-08-13 18:06:08.333664 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193151) -9316> 2014-08-13 18:06:08.338845 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.338845, event: started, request: pg_notify(24.181(45) epoch 194496) v5 -9315> 2014-08-13 18:06:08.333661 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 337 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -9314> 2014-08-13 18:06:08.338873 7f5c835ef700 5 -- op tracker -- , seq: 516, time: 2014-08-13 18:06:08.338873, event: done, request: pg_notify(24.181(45) epoch 194496) v5 -9313> 2014-08-13 18:06:08.338889 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9312> 2014-08-13 18:06:08.338892 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9311> 2014-08-13 18:06:08.338859 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] log request from 356 -9310> 2014-08-13 18:06:08.338865 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -9309> 2014-08-13 18:06:08.338909 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.345 192.168.1.110:6820/46337 8 ==== pg_info(1 pgs e194496:22.29a) v4 ==== 714+0+0 (3352696858 0 0) 0x58896c0 con 0x44265c0 -9308> 2014-08-13 18:06:08.338929 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9307> 2014-08-13 18:06:08.338913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -9306> 2014-08-13 18:06:08.338932 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9305> 2014-08-13 18:06:08.338934 7f5c835ef700 20 osd.4 194496 _dispatch 0x58896c0 pg_info(1 pgs e194496:22.29a) v4 -9304> 2014-08-13 18:06:08.338937 7f5c7bde3700 10 log is not dirty -9303> 2014-08-13 18:06:08.338940 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.330787, event: header_read, request: pg_info(1 pgs e194496:22.29a) v4 -9302> 2014-08-13 18:06:08.338948 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.330789, event: throttled, request: pg_info(1 pgs e194496:22.29a) v4 -9301> 2014-08-13 18:06:08.338941 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending info+missing+log since 194487'0 -9300> 2014-08-13 18:06:08.338955 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.330866, event: all_read, request: pg_info(1 pgs e194496:22.29a) v4 -9299> 2014-08-13 18:06:08.338947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 343 info: 22.362( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) -9298> 2014-08-13 18:06:08.338961 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.338928, event: dispatched, request: pg_info(1 pgs e194496:22.29a) v4 -9297> 2014-08-13 18:06:08.338971 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.338971, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.29a) v4 -9296> 2014-08-13 18:06:08.338961 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] sending log((194422'17433,194422'17433], crt=194422'17433) missing(51) -9295> 2014-08-13 18:06:08.338979 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.29a) v4 from osd.345 -9294> 2014-08-13 18:06:08.338984 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x58896c0 -9293> 2014-08-13 18:06:08.338971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed -9292> 2014-08-13 18:06:08.338987 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -9291> 2014-08-13 18:06:08.338990 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.338990, event: started, request: pg_info(1 pgs e194496:22.29a) v4 -9290> 2014-08-13 18:06:08.338991 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194492/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9289> 2014-08-13 18:06:08.338991 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6832/5235 -- pg_log(24.350 epoch 194496 log log((194422'17433,194422'17433], crt=194422'17433) query_epoch 194496) v4 -- ?+0 0x4160600 con 0x3c527e0 -9288> 2014-08-13 18:06:08.339012 7f5c835ef700 5 -- op tracker -- , seq: 517, time: 2014-08-13 18:06:08.339012, event: done, request: pg_info(1 pgs e194496:22.29a) v4 -9287> 2014-08-13 18:06:08.339006 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9286> 2014-08-13 18:06:08.339022 7f5c7c7e4700 10 log is not dirty -9285> 2014-08-13 18:06:08.339023 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9284> 2014-08-13 18:06:08.339027 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9283> 2014-08-13 18:06:08.339033 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.285 192.168.1.101:6822/26295 7 ==== pg_query(24.1e6 epoch 194496) v3 ==== 144+0+0 (2129025724 0 0) 0x5421c20 con 0x445e5c0 -9282> 2014-08-13 18:06:08.339046 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9281> 2014-08-13 18:06:08.339049 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9280> 2014-08-13 18:06:08.339051 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421c20 pg_query(24.1e6 epoch 194496) v3 -9279> 2014-08-13 18:06:08.339056 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.327477, event: header_read, request: pg_query(24.1e6 epoch 194496) v3 -9278> 2014-08-13 18:06:08.339062 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.327480, event: throttled, request: pg_query(24.1e6 epoch 194496) v3 -9277> 2014-08-13 18:06:08.339068 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.327554, event: all_read, request: pg_query(24.1e6 epoch 194496) v3 -9276> 2014-08-13 18:06:08.339084 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.339045, event: dispatched, request: pg_query(24.1e6 epoch 194496) v3 -9275> 2014-08-13 18:06:08.339091 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.339091, event: waiting_for_osdmap, request: pg_query(24.1e6 epoch 194496) v3 -9274> 2014-08-13 18:06:08.339097 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.285 epoch 194496 -9273> 2014-08-13 18:06:08.339100 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421c20 -9272> 2014-08-13 18:06:08.339105 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.339105, event: started, request: pg_query(24.1e6 epoch 194496) v3 -9271> 2014-08-13 18:06:08.339112 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192975) from replica 285 -9270> 2014-08-13 18:06:08.339162 7f5c835ef700 5 -- op tracker -- , seq: 518, time: 2014-08-13 18:06:08.339161, event: done, request: pg_query(24.1e6 epoch 194496) v3 -9269> 2014-08-13 18:06:08.339176 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9268> 2014-08-13 18:06:08.339179 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9267> 2014-08-13 18:06:08.339186 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.289 192.168.1.113:6808/15284 13 ==== pg_info(1 pgs e194496:22.48e) v4 ==== 714+0+0 (1023951906 0 0) 0x4df3f00 con 0x445f7a0 -9266> 2014-08-13 18:06:08.339200 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9265> 2014-08-13 18:06:08.339202 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9264> 2014-08-13 18:06:08.339204 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3f00 pg_info(1 pgs e194496:22.48e) v4 -9263> 2014-08-13 18:06:08.339210 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.329186, event: header_read, request: pg_info(1 pgs e194496:22.48e) v4 -9262> 2014-08-13 18:06:08.339213 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x43f7900 don't know epoch, doing nothing -9261> 2014-08-13 18:06:08.339217 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.329189, event: throttled, request: pg_info(1 pgs e194496:22.48e) v4 -9260> 2014-08-13 18:06:08.339223 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.329277, event: all_read, request: pg_info(1 pgs e194496:22.48e) v4 -9259> 2014-08-13 18:06:08.339228 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.339199, event: dispatched, request: pg_info(1 pgs e194496:22.48e) v4 -9258> 2014-08-13 18:06:08.339220 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.58 192.168.1.111:6834/14316 -- pg_info(1 pgs e194496:22.362) v4 -- ?+0 0x5353b80 -9257> 2014-08-13 18:06:08.339212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9256> 2014-08-13 18:06:08.339235 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.339235, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.48e) v4 -9255> 2014-08-13 18:06:08.339244 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.48e) v4 from osd.289 -9254> 2014-08-13 18:06:08.339249 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3f00 -9253> 2014-08-13 18:06:08.339254 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -9252> 2014-08-13 18:06:08.339243 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] requeue_ops -9251> 2014-08-13 18:06:08.339254 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.339253, event: started, request: pg_info(1 pgs e194496:22.48e) v4 -9250> 2014-08-13 18:06:08.339264 7f5c7c7e4700 10 log is not dirty -9249> 2014-08-13 18:06:08.339259 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194496:22.362) v4 -- ?+0 0x5356ac0 -9248> 2014-08-13 18:06:08.339283 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2685 -9247> 2014-08-13 18:06:08.339313 7f5c7bde3700 10 log is not dirty -9246> 2014-08-13 18:06:08.339291 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -9245> 2014-08-13 18:06:08.339291 7f5c835ef700 5 -- op tracker -- , seq: 519, time: 2014-08-13 18:06:08.339291, event: done, request: pg_info(1 pgs e194496:22.48e) v4 -9244> 2014-08-13 18:06:08.339324 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9243> 2014-08-13 18:06:08.339333 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9242> 2014-08-13 18:06:08.339319 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9241> 2014-08-13 18:06:08.339340 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 13 ==== pg_notify(24.1bb(40) epoch 194496) v5 ==== 3004+0+0 (2862996617 0 0) 0x588a840 con 0x3c55540 -9240> 2014-08-13 18:06:08.339355 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9239> 2014-08-13 18:06:08.339358 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9238> 2014-08-13 18:06:08.339361 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a840 pg_notify(24.1bb(40) epoch 194496) v5 -9237> 2014-08-13 18:06:08.339352 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged -9236> 2014-08-13 18:06:08.339366 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.328357, event: header_read, request: pg_notify(24.1bb(40) epoch 194496) v5 -9235> 2014-08-13 18:06:08.339375 7f5c7c7e4700 10 log is not dirty -9234> 2014-08-13 18:06:08.339374 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.328360, event: throttled, request: pg_notify(24.1bb(40) epoch 194496) v5 -9233> 2014-08-13 18:06:08.339380 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.328486, event: all_read, request: pg_notify(24.1bb(40) epoch 194496) v5 -9232> 2014-08-13 18:06:08.339390 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.339355, event: dispatched, request: pg_notify(24.1bb(40) epoch 194496) v5 -9231> 2014-08-13 18:06:08.339424 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.339423, event: waiting_for_osdmap, request: pg_notify(24.1bb(40) epoch 194496) v5 -9230> 2014-08-13 18:06:08.339431 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.337 -9229> 2014-08-13 18:06:08.339435 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a840 -9228> 2014-08-13 18:06:08.339445 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.339444, event: started, request: pg_notify(24.1bb(40) epoch 194496) v5 -9227> 2014-08-13 18:06:08.339429 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 320 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -9226> 2014-08-13 18:06:08.339436 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 65 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -9225> 2014-08-13 18:06:08.339457 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -9224> 2014-08-13 18:06:08.339460 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.65 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours -9223> 2014-08-13 18:06:08.339480 7f5c835ef700 5 -- op tracker -- , seq: 520, time: 2014-08-13 18:06:08.339479, event: done, request: pg_notify(24.1bb(40) epoch 194496) v5 -9222> 2014-08-13 18:06:08.339485 7f5c7c7e4700 10 log is not dirty -9221> 2014-08-13 18:06:08.339495 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9220> 2014-08-13 18:06:08.339499 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9219> 2014-08-13 18:06:08.339490 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 unchanged -9218> 2014-08-13 18:06:08.339492 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 376 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -9217> 2014-08-13 18:06:08.339513 7f5c7bde3700 10 log is not dirty -9216> 2014-08-13 18:06:08.339514 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.376 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9215> 2014-08-13 18:06:08.339520 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 373 notify: (query_epoch:194496, epoch_sent:194496, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) -9214> 2014-08-13 18:06:08.339537 7f5c7c7e4700 10 log is not dirty -9213> 2014-08-13 18:06:08.339523 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.343 192.168.1.104:6802/61960 11 ==== pg_info(1 pgs e194496:22.3d5) v4 ==== 714+0+0 (1806870319 0 0) 0x5354d00 con 0x48506e0 -9212> 2014-08-13 18:06:08.339547 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -9211> 2014-08-13 18:06:08.339544 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 373, already have info from that osd, ignoring -9210> 2014-08-13 18:06:08.339544 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9209> 2014-08-13 18:06:08.339562 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9208> 2014-08-13 18:06:08.339565 7f5c7bde3700 10 log is not dirty -9207> 2014-08-13 18:06:08.339564 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354d00 pg_info(1 pgs e194496:22.3d5) v4 -9206> 2014-08-13 18:06:08.339573 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 289 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192968) -9205> 2014-08-13 18:06:08.339571 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.329531, event: header_read, request: pg_info(1 pgs e194496:22.3d5) v4 -9204> 2014-08-13 18:06:08.339593 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.329533, event: throttled, request: pg_info(1 pgs e194496:22.3d5) v4 -9203> 2014-08-13 18:06:08.339584 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 356 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -9202> 2014-08-13 18:06:08.339600 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.329615, event: all_read, request: pg_info(1 pgs e194496:22.3d5) v4 -9201> 2014-08-13 18:06:08.339593 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] log request from 289 -9200> 2014-08-13 18:06:08.339607 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.339543, event: dispatched, request: pg_info(1 pgs e194496:22.3d5) v4 -9199> 2014-08-13 18:06:08.339615 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.339614, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3d5) v4 -9198> 2014-08-13 18:06:08.339621 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3d5) v4 from osd.343 -9197> 2014-08-13 18:06:08.339626 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354d00 -9196> 2014-08-13 18:06:08.339607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.356 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours -9195> 2014-08-13 18:06:08.339631 7f5c7c7e4700 10 log is not dirty -9194> 2014-08-13 18:06:08.339630 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.339630, event: started, request: pg_info(1 pgs e194496:22.3d5) v4 -9193> 2014-08-13 18:06:08.339637 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 55 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -9192> 2014-08-13 18:06:08.339642 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending info+missing+log since 194487'0 -9191> 2014-08-13 18:06:08.339668 7f5c835ef700 5 -- op tracker -- , seq: 521, time: 2014-08-13 18:06:08.339667, event: done, request: pg_info(1 pgs e194496:22.3d5) v4 -9190> 2014-08-13 18:06:08.339662 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] sending log((194422'17034,194422'17034], crt=194422'17034) missing(63) -9189> 2014-08-13 18:06:08.339658 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.55 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9188> 2014-08-13 18:06:08.339682 7f5c7c7e4700 10 log is not dirty -9187> 2014-08-13 18:06:08.339681 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9186> 2014-08-13 18:06:08.339683 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 don't know epoch, doing nothing -9185> 2014-08-13 18:06:08.339685 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9184> 2014-08-13 18:06:08.339687 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6808/15284 -- pg_log(24.5e epoch 194496 log log((194422'17034,194422'17034], crt=194422'17034) query_epoch 194496) v4 -- ?+0 0x4164200 con 0x445f7a0 -9183> 2014-08-13 18:06:08.339691 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.323 192.168.1.106:6816/8118 5 ==== pg_info(1 pgs e194496:22.12a) v4 ==== 714+0+0 (291467519 0 0) 0x5888380 con 0x49a2520 -9182> 2014-08-13 18:06:08.339706 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9181> 2014-08-13 18:06:08.339689 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 351 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -9180> 2014-08-13 18:06:08.339709 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9179> 2014-08-13 18:06:08.339711 7f5c835ef700 20 osd.4 194496 _dispatch 0x5888380 pg_info(1 pgs e194496:22.12a) v4 -9178> 2014-08-13 18:06:08.339713 7f5c7bde3700 10 log is not dirty -9177> 2014-08-13 18:06:08.339716 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.331864, event: header_read, request: pg_info(1 pgs e194496:22.12a) v4 -9176> 2014-08-13 18:06:08.339711 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got dup osd.351 info 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9175> 2014-08-13 18:06:08.339734 7f5c7c7e4700 10 log is not dirty -9174> 2014-08-13 18:06:08.339723 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.331865, event: throttled, request: pg_info(1 pgs e194496:22.12a) v4 -9173> 2014-08-13 18:06:08.339746 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.331924, event: all_read, request: pg_info(1 pgs e194496:22.12a) v4 -9172> 2014-08-13 18:06:08.339752 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.339705, event: dispatched, request: pg_info(1 pgs e194496:22.12a) v4 -9171> 2014-08-13 18:06:08.339782 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.339781, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.12a) v4 -9170> 2014-08-13 18:06:08.339788 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.12a) v4 from osd.323 -9169> 2014-08-13 18:06:08.339793 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5888380 -9168> 2014-08-13 18:06:08.339798 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.339797, event: started, request: pg_info(1 pgs e194496:22.12a) v4 -9167> 2014-08-13 18:06:08.339818 7f5c835ef700 5 -- op tracker -- , seq: 522, time: 2014-08-13 18:06:08.339818, event: done, request: pg_info(1 pgs e194496:22.12a) v4 -9166> 2014-08-13 18:06:08.339839 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9165> 2014-08-13 18:06:08.339843 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9164> 2014-08-13 18:06:08.339861 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.287 192.168.1.109:6814/33435 4 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4bc5e80 con 0x49a7bc0 -9163> 2014-08-13 18:06:08.339878 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9162> 2014-08-13 18:06:08.339880 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9161> 2014-08-13 18:06:08.339883 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc5e80 osd_map(194496..194496 src has 185139..194496) v3 -9160> 2014-08-13 18:06:08.339893 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -9159> 2014-08-13 18:06:08.339899 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -9158> 2014-08-13 18:06:08.339884 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 327 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -9157> 2014-08-13 18:06:08.339902 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9156> 2014-08-13 18:06:08.339878 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 17 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193188) -9155> 2014-08-13 18:06:08.339909 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9154> 2014-08-13 18:06:08.339916 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.292 192.168.1.113:6812/16056 5 ==== pg_info(1 pgs e194496:22.3ad) v4 ==== 714+0+0 (3990303094 0 0) 0x5889340 con 0x4ab32e0 -9153> 2014-08-13 18:06:08.339913 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] log request from 17 -9152> 2014-08-13 18:06:08.339929 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9151> 2014-08-13 18:06:08.339909 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.327 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9150> 2014-08-13 18:06:08.339932 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9149> 2014-08-13 18:06:08.339934 7f5c7c7e4700 10 log is not dirty -9148> 2014-08-13 18:06:08.339935 7f5c835ef700 20 osd.4 194496 _dispatch 0x5889340 pg_info(1 pgs e194496:22.3ad) v4 -9147> 2014-08-13 18:06:08.339941 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.332176, event: header_read, request: pg_info(1 pgs e194496:22.3ad) v4 -9146> 2014-08-13 18:06:08.339941 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 330 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -9145> 2014-08-13 18:06:08.339948 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.332177, event: throttled, request: pg_info(1 pgs e194496:22.3ad) v4 -9144> 2014-08-13 18:06:08.339957 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending info+missing+log since 194487'0 -9143> 2014-08-13 18:06:08.339961 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9142> 2014-08-13 18:06:08.339974 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] sending log((194422'14353,194422'14353], crt=194422'14353) missing(49) -9141> 2014-08-13 18:06:08.339961 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.332273, event: all_read, request: pg_info(1 pgs e194496:22.3ad) v4 -9140> 2014-08-13 18:06:08.339994 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 -9139> 2014-08-13 18:06:08.339993 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.339928, event: dispatched, request: pg_info(1 pgs e194496:22.3ad) v4 -9138> 2014-08-13 18:06:08.340002 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.340002, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3ad) v4 -9137> 2014-08-13 18:06:08.339990 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged -9136> 2014-08-13 18:06:08.339998 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6811/37658 -- pg_log(24.2a1 epoch 194496 log log((194422'14353,194422'14353], crt=194422'14353) query_epoch 194496) v4 -- ?+0 0x4163600 con 0x4493f40 -9135> 2014-08-13 18:06:08.340008 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3ad) v4 from osd.292 -9134> 2014-08-13 18:06:08.340012 7f5c7c7e4700 10 log is not dirty -9133> 2014-08-13 18:06:08.340013 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5889340 -9132> 2014-08-13 18:06:08.340018 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.340018, event: started, request: pg_info(1 pgs e194496:22.3ad) v4 -9131> 2014-08-13 18:06:08.340037 7f5c7bde3700 10 log is not dirty -9130> 2014-08-13 18:06:08.340043 7f5c835ef700 5 -- op tracker -- , seq: 523, time: 2014-08-13 18:06:08.340043, event: done, request: pg_info(1 pgs e194496:22.3ad) v4 -9129> 2014-08-13 18:06:08.340066 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9128> 2014-08-13 18:06:08.340070 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9127> 2014-08-13 18:06:08.340076 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.51 192.168.1.112:6832/6655 5 ==== pg_info(1 pgs e194496:22.537) v4 ==== 714+0+0 (2921663850 0 0) 0x588b800 con 0x3cf1600 -9126> 2014-08-13 18:06:08.340090 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9125> 2014-08-13 18:06:08.340092 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9124> 2014-08-13 18:06:08.340094 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b800 pg_info(1 pgs e194496:22.537) v4 -9123> 2014-08-13 18:06:08.340099 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.332538, event: header_read, request: pg_info(1 pgs e194496:22.537) v4 -9122> 2014-08-13 18:06:08.340110 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.332540, event: throttled, request: pg_info(1 pgs e194496:22.537) v4 -9121> 2014-08-13 18:06:08.340116 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.332642, event: all_read, request: pg_info(1 pgs e194496:22.537) v4 -9120> 2014-08-13 18:06:08.340121 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.340089, event: dispatched, request: pg_info(1 pgs e194496:22.537) v4 -9119> 2014-08-13 18:06:08.340128 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.340127, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.537) v4 -9118> 2014-08-13 18:06:08.340133 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.537) v4 from osd.51 -9117> 2014-08-13 18:06:08.340137 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b800 -9116> 2014-08-13 18:06:08.340142 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.340142, event: started, request: pg_info(1 pgs e194496:22.537) v4 -9115> 2014-08-13 18:06:08.340193 7f5c835ef700 5 -- op tracker -- , seq: 524, time: 2014-08-13 18:06:08.340192, event: done, request: pg_info(1 pgs e194496:22.537) v4 -9114> 2014-08-13 18:06:08.340205 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9113> 2014-08-13 18:06:08.340208 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9112> 2014-08-13 18:06:08.340191 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 289 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) -9111> 2014-08-13 18:06:08.340200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194422'18845 ec=192688 les/c 194493/194493 194495/194496/192969) -9110> 2014-08-13 18:06:08.340214 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.347 192.168.1.106:6800/4640 6 ==== pg_info(1 pgs e194496:22.1b8) v4 ==== 714+0+0 (1521039773 0 0) 0x5356740 con 0x4afb020 -9109> 2014-08-13 18:06:08.340226 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed -9108> 2014-08-13 18:06:08.340226 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 -9107> 2014-08-13 18:06:08.340245 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9106> 2014-08-13 18:06:08.340247 7f5c7c7e4700 10 log is not dirty -9105> 2014-08-13 18:06:08.340248 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9104> 2014-08-13 18:06:08.340251 7f5c835ef700 20 osd.4 194496 _dispatch 0x5356740 pg_info(1 pgs e194496:22.1b8) v4 -9103> 2014-08-13 18:06:08.340256 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.330934, event: header_read, request: pg_info(1 pgs e194496:22.1b8) v4 -9102> 2014-08-13 18:06:08.340255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 48 notify: (query_epoch:194496, epoch_sent:194496, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) -9101> 2014-08-13 18:06:08.340270 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.330936, event: throttled, request: pg_info(1 pgs e194496:22.1b8) v4 -9100> 2014-08-13 18:06:08.340276 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.331005, event: all_read, request: pg_info(1 pgs e194496:22.1b8) v4 -9099> 2014-08-13 18:06:08.340283 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.340244, event: dispatched, request: pg_info(1 pgs e194496:22.1b8) v4 -9098> 2014-08-13 18:06:08.340275 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 48, already have info from that osd, ignoring -9097> 2014-08-13 18:06:08.340274 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194422'18845 -9096> 2014-08-13 18:06:08.340296 7f5c7c7e4700 10 log is not dirty -9095> 2014-08-13 18:06:08.340289 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.340289, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1b8) v4 -9094> 2014-08-13 18:06:08.340310 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1b8) v4 from osd.347 -9093> 2014-08-13 18:06:08.340297 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) -9092> 2014-08-13 18:06:08.340315 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5356740 -9091> 2014-08-13 18:06:08.340318 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -9090> 2014-08-13 18:06:08.340302 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 17 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -9089> 2014-08-13 18:06:08.340321 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.340321, event: started, request: pg_info(1 pgs e194496:22.1b8) v4 -9088> 2014-08-13 18:06:08.340322 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194496 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194496) v4 -- ?+0 0x4163000 con 0x3c527e0 -9087> 2014-08-13 18:06:08.340324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.17 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -9086> 2014-08-13 18:06:08.340344 7f5c7bde3700 10 log is not dirty -9085> 2014-08-13 18:06:08.340347 7f5c7c7e4700 10 log is not dirty -9084> 2014-08-13 18:06:08.340358 7f5c835ef700 5 -- op tracker -- , seq: 525, time: 2014-08-13 18:06:08.340358, event: done, request: pg_info(1 pgs e194496:22.1b8) v4 -9083> 2014-08-13 18:06:08.340370 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9082> 2014-08-13 18:06:08.340373 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9081> 2014-08-13 18:06:08.340379 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.328 192.168.1.104:6812/64303 11 ==== pg_info(1 pgs e194496:22.1a0) v4 ==== 714+0+0 (2862512127 0 0) 0x4dea4c0 con 0x4afdee0 -9080> 2014-08-13 18:06:08.340402 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9079> 2014-08-13 18:06:08.340405 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9078> 2014-08-13 18:06:08.340407 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea4c0 pg_info(1 pgs e194496:22.1a0) v4 -9077> 2014-08-13 18:06:08.340412 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.327948, event: header_read, request: pg_info(1 pgs e194496:22.1a0) v4 -9076> 2014-08-13 18:06:08.340426 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.327957, event: throttled, request: pg_info(1 pgs e194496:22.1a0) v4 -9075> 2014-08-13 18:06:08.340443 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.328100, event: all_read, request: pg_info(1 pgs e194496:22.1a0) v4 -9074> 2014-08-13 18:06:08.340450 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.340401, event: dispatched, request: pg_info(1 pgs e194496:22.1a0) v4 -9073> 2014-08-13 18:06:08.340456 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.340456, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1a0) v4 -9072> 2014-08-13 18:06:08.340462 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1a0) v4 from osd.328 -9071> 2014-08-13 18:06:08.340466 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea4c0 -9070> 2014-08-13 18:06:08.340471 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.340471, event: started, request: pg_info(1 pgs e194496:22.1a0) v4 -9069> 2014-08-13 18:06:08.340490 7f5c835ef700 5 -- op tracker -- , seq: 526, time: 2014-08-13 18:06:08.340490, event: done, request: pg_info(1 pgs e194496:22.1a0) v4 -9068> 2014-08-13 18:06:08.340506 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9067> 2014-08-13 18:06:08.340510 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9066> 2014-08-13 18:06:08.340496 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 374 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) -9065> 2014-08-13 18:06:08.340502 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 356 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) -9064> 2014-08-13 18:06:08.340516 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.373 192.168.1.108:6840/7435 15 ==== pg_info(1 pgs e194496:22.1fb) v4 ==== 714+0+0 (394481125 0 0) 0x4bf55c0 con 0x4b111e0 -9063> 2014-08-13 18:06:08.340529 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9062> 2014-08-13 18:06:08.340527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.374 activated and committed -9061> 2014-08-13 18:06:08.340533 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9060> 2014-08-13 18:06:08.340527 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.356 activated and committed -9059> 2014-08-13 18:06:08.340553 7f5c7bde3700 10 log is not dirty -9058> 2014-08-13 18:06:08.340554 7f5c7c7e4700 10 log is not dirty -9057> 2014-08-13 18:06:08.340553 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bf55c0 pg_info(1 pgs e194496:22.1fb) v4 -9056> 2014-08-13 18:06:08.340560 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.328703, event: header_read, request: pg_info(1 pgs e194496:22.1fb) v4 -9055> 2014-08-13 18:06:08.340573 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.328705, event: throttled, request: pg_info(1 pgs e194496:22.1fb) v4 -9054> 2014-08-13 18:06:08.340565 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 344 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) -9053> 2014-08-13 18:06:08.340580 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.328784, event: all_read, request: pg_info(1 pgs e194496:22.1fb) v4 -9052> 2014-08-13 18:06:08.340586 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.340529, event: dispatched, request: pg_info(1 pgs e194496:22.1fb) v4 -9051> 2014-08-13 18:06:08.340571 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -9050> 2014-08-13 18:06:08.340594 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.340593, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1fb) v4 -9049> 2014-08-13 18:06:08.340600 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1fb) v4 from osd.373 -9048> 2014-08-13 18:06:08.340586 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.344 activated and committed -9047> 2014-08-13 18:06:08.340608 7f5c7bde3700 10 log is not dirty -9046> 2014-08-13 18:06:08.340606 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4bf55c0 -9045> 2014-08-13 18:06:08.340615 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.340615, event: started, request: pg_info(1 pgs e194496:22.1fb) v4 -9044> 2014-08-13 18:06:08.340598 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -9043> 2014-08-13 18:06:08.340614 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.39d( v 194215'480 lc 0'0 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) -9042> 2014-08-13 18:06:08.340633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] state: peer osd.373 activated and committed -9041> 2014-08-13 18:06:08.340632 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] update_heartbeat_peers 4,50,55,61,65,356 unchanged -9040> 2014-08-13 18:06:08.340651 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -9039> 2014-08-13 18:06:08.340658 7f5c835ef700 5 -- op tracker -- , seq: 527, time: 2014-08-13 18:06:08.340658, event: done, request: pg_info(1 pgs e194496:22.1fb) v4 -9038> 2014-08-13 18:06:08.340660 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetInfo 0.087649 8 0.000812 -9037> 2014-08-13 18:06:08.340669 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] share_pg_info -9036> 2014-08-13 18:06:08.340691 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9035> 2014-08-13 18:06:08.340694 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing -9034> 2014-08-13 18:06:08.340696 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9033> 2014-08-13 18:06:08.340691 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetLog -9032> 2014-08-13 18:06:08.340698 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194496:22.39d) v4 -- ?+0 0x5351c00 -9031> 2014-08-13 18:06:08.340703 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 8 ==== pg_info(1 pgs e194496:22.3b9) v4 ==== 714+0+0 (1281546060 0 0) 0x4df55c0 con 0x4b10b00 -9030> 2014-08-13 18:06:08.340718 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9029> 2014-08-13 18:06:08.340721 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9028> 2014-08-13 18:06:08.340724 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df55c0 pg_info(1 pgs e194496:22.3b9) v4 -9027> 2014-08-13 18:06:08.340724 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b111e0 don't know epoch, doing nothing -9026> 2014-08-13 18:06:08.340729 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.329091, event: header_read, request: pg_info(1 pgs e194496:22.3b9) v4 -9025> 2014-08-13 18:06:08.340728 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.373 192.168.1.108:6840/7435 -- pg_info(1 pgs e194496:22.39d) v4 -- ?+0 0x5353800 -9024> 2014-08-13 18:06:08.340718 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.4 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -9023> 2014-08-13 18:06:08.340737 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.329093, event: throttled, request: pg_info(1 pgs e194496:22.3b9) v4 -9022> 2014-08-13 18:06:08.340743 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.329242, event: all_read, request: pg_info(1 pgs e194496:22.3b9) v4 -9021> 2014-08-13 18:06:08.340749 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.340717, event: dispatched, request: pg_info(1 pgs e194496:22.3b9) v4 -9020> 2014-08-13 18:06:08.340755 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.340755, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3b9) v4 -9019> 2014-08-13 18:06:08.340745 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2617 -9018> 2014-08-13 18:06:08.340742 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -9017> 2014-08-13 18:06:08.340762 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3b9) v4 from osd.397 -9016> 2014-08-13 18:06:08.340767 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df55c0 -9015> 2014-08-13 18:06:08.340775 7f5c7bde3700 10 log is not dirty -9014> 2014-08-13 18:06:08.340765 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9013> 2014-08-13 18:06:08.340777 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.340777, event: started, request: pg_info(1 pgs e194496:22.3b9) v4 -9012> 2014-08-13 18:06:08.340782 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9011> 2014-08-13 18:06:08.340786 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -9010> 2014-08-13 18:06:08.340811 7f5c835ef700 5 -- op tracker -- , seq: 528, time: 2014-08-13 18:06:08.340811, event: done, request: pg_info(1 pgs e194496:22.3b9) v4 -9009> 2014-08-13 18:06:08.340805 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] requeue_ops -9008> 2014-08-13 18:06:08.340824 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9007> 2014-08-13 18:06:08.340827 7f5c7bde3700 10 log is not dirty -9006> 2014-08-13 18:06:08.340806 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -9005> 2014-08-13 18:06:08.340828 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -9004> 2014-08-13 18:06:08.340833 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting osd.356 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -9003> 2014-08-13 18:06:08.340838 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.396 192.168.1.105:6814/41806 9 ==== MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 ==== 30+0+0 (617086093 0 0) 0x4df3b80 con 0x4b2c620 -9002> 2014-08-13 18:06:08.340836 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -9001> 2014-08-13 18:06:08.340864 7f5c835ef700 10 osd.4 194496 do_waiters -- start -9000> 2014-08-13 18:06:08.340872 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8999> 2014-08-13 18:06:08.340875 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df3b80 MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8998> 2014-08-13 18:06:08.340867 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8997> 2014-08-13 18:06:08.340885 7f5c7bde3700 10 log is not dirty -8996> 2014-08-13 18:06:08.340897 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8995> 2014-08-13 18:06:08.340881 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.331589, event: header_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8994> 2014-08-13 18:06:08.340907 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.331591, event: throttled, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8993> 2014-08-13 18:06:08.340897 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] calc_acting newest update on osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) shard 55 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 61 (up) backfill 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 50 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) shard 65 (stray) accepted 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -8992> 2014-08-13 18:06:08.340915 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.331644, event: all_read, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8991> 2014-08-13 18:06:08.340925 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.340863, event: dispatched, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8990> 2014-08-13 18:06:08.340933 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.340932, event: waiting_for_osdmap, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8989> 2014-08-13 18:06:08.340941 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df3b80 -8988> 2014-08-13 18:06:08.340921 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] actingbackfill is 4,50,55,61,65 -8987> 2014-08-13 18:06:08.340933 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 362 notify: (query_epoch:194496, epoch_sent:194496, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -8986> 2014-08-13 18:06:08.340947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] choose_acting want [4,50,65] (== acting) backfill_targets 55,61 -8985> 2014-08-13 18:06:08.340958 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 362, already have info from that osd, ignoring -8984> 2014-08-13 18:06:08.340968 7f5c835ef700 5 -- op tracker -- , seq: 529, time: 2014-08-13 18:06:08.340968, event: done, request: MBackfillReserve REQUEST pgid: 24.299, query_epoch: 194496, prio: 1 v3 -8983> 2014-08-13 18:06:08.340976 7f5c7bde3700 10 log is not dirty -8982> 2014-08-13 18:06:08.340980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: leaving GetLog -8981> 2014-08-13 18:06:08.341001 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8980> 2014-08-13 18:06:08.341005 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8979> 2014-08-13 18:06:08.341011 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 17 ==== pg_info(1 pgs e194496:22.226) v4 ==== 714+0+0 (1917955235 0 0) 0x588b100 con 0x4b2bc80 -8978> 2014-08-13 18:06:08.341004 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 332 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -8977> 2014-08-13 18:06:08.341006 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetLog 0.000315 0 0.000000 -8976> 2014-08-13 18:06:08.341026 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8975> 2014-08-13 18:06:08.341030 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8974> 2014-08-13 18:06:08.341032 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b100 pg_info(1 pgs e194496:22.226) v4 -8973> 2014-08-13 18:06:08.341038 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.335526, event: header_read, request: pg_info(1 pgs e194496:22.226) v4 -8972> 2014-08-13 18:06:08.341045 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.335529, event: throttled, request: pg_info(1 pgs e194496:22.226) v4 -8971> 2014-08-13 18:06:08.341034 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/GetMissing -8970> 2014-08-13 18:06:08.341031 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -8969> 2014-08-13 18:06:08.341051 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.335662, event: all_read, request: pg_info(1 pgs e194496:22.226) v4 -8968> 2014-08-13 18:06:08.341058 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.341026, event: dispatched, request: pg_info(1 pgs e194496:22.226) v4 -8967> 2014-08-13 18:06:08.341064 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.341064, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.226) v4 -8966> 2014-08-13 18:06:08.341055 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.50 has no missing, identical log -8965> 2014-08-13 18:06:08.341077 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.226) v4 from osd.376 -8964> 2014-08-13 18:06:08.341082 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b100 -8963> 2014-08-13 18:06:08.341065 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -8962> 2014-08-13 18:06:08.341091 7f5c7bde3700 10 log is not dirty -8961> 2014-08-13 18:06:08.341090 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.341089, event: started, request: pg_info(1 pgs e194496:22.226) v4 -8960> 2014-08-13 18:06:08.341080 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: osd.65 has no missing, identical log -8959> 2014-08-13 18:06:08.341098 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 307 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -8958> 2014-08-13 18:06:08.341101 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: still need up_thru update before going active -8957> 2014-08-13 18:06:08.341129 7f5c835ef700 5 -- op tracker -- , seq: 530, time: 2014-08-13 18:06:08.341129, event: done, request: pg_info(1 pgs e194496:22.226) v4 -8956> 2014-08-13 18:06:08.341121 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8955> 2014-08-13 18:06:08.341125 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/GetMissing 0.000091 0 0.000000 -8954> 2014-08-13 18:06:08.341148 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] enter Started/Primary/Peering/WaitUpThru -8953> 2014-08-13 18:06:08.341147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] osd.307 has stray content: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8952> 2014-08-13 18:06:08.341169 7f5c7c7e4700 10 log is not dirty -8951> 2014-08-13 18:06:08.341142 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8950> 2014-08-13 18:06:08.341182 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8949> 2014-08-13 18:06:08.341172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,320,344 -> 4,55,296,307,320,344 -8948> 2014-08-13 18:06:08.341190 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.7 192.168.1.115:6838/62713 6 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4bc7bc0 con 0x4c58f20 -8947> 2014-08-13 18:06:08.341206 7f5c7bde3700 20 osd.4 194496 need_heartbeat_peer_update -8946> 2014-08-13 18:06:08.341207 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8945> 2014-08-13 18:06:08.341210 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8944> 2014-08-13 18:06:08.341212 7f5c7bde3700 10 log is not dirty -8943> 2014-08-13 18:06:08.341212 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc7bc0 osd_map(194496..194496 src has 185139..194496) v3 -8942> 2014-08-13 18:06:08.341214 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:22.6bd( v 194215'504 (0'0,194215'504] local-les=194492 n=35 ec=183096 les/c 194492/194489 194495/194495/194495)) -8941> 2014-08-13 18:06:08.341219 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8940> 2014-08-13 18:06:08.341221 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -8939> 2014-08-13 18:06:08.341240 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 50, already have info from that osd, ignoring -8938> 2014-08-13 18:06:08.341241 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] requeue_ops -8937> 2014-08-13 18:06:08.341259 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -8936> 2014-08-13 18:06:08.341262 7f5c7c7e4700 10 log is not dirty -8935> 2014-08-13 18:06:08.341262 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8934> 2014-08-13 18:06:08.341264 7f5c7bde3700 10 log is not dirty -8933> 2014-08-13 18:06:08.341266 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8932> 2014-08-13 18:06:08.341273 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.348 192.168.1.110:6800/41204 3 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4cfc800 con 0x571b700 -8931> 2014-08-13 18:06:08.341291 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8930> 2014-08-13 18:06:08.341294 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8929> 2014-08-13 18:06:08.341278 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 296 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -8928> 2014-08-13 18:06:08.341282 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 347 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) -8927> 2014-08-13 18:06:08.341300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8926> 2014-08-13 18:06:08.341305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 347, already have info from that osd, ignoring -8925> 2014-08-13 18:06:08.341326 7f5c7c7e4700 10 log is not dirty -8924> 2014-08-13 18:06:08.341297 7f5c835ef700 20 osd.4 194496 _dispatch 0x4cfc800 osd_map(194496..194496 src has 185139..194496) v3 -8923> 2014-08-13 18:06:08.341333 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -8922> 2014-08-13 18:06:08.341339 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -8921> 2014-08-13 18:06:08.341342 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8920> 2014-08-13 18:06:08.341345 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8919> 2014-08-13 18:06:08.341331 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] update_heartbeat_peers 4,55,296,307,320,344 unchanged -8918> 2014-08-13 18:06:08.341333 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8917> 2014-08-13 18:06:08.341355 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] requeue_ops -8916> 2014-08-13 18:06:08.341355 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetInfo 0.073293 6 0.001710 -8915> 2014-08-13 18:06:08.341376 7f5c7c7e4700 10 log is not dirty -8914> 2014-08-13 18:06:08.341356 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.345 192.168.1.110:6820/46337 9 ==== pg_info(1 pgs e194496:22.29a) v4 ==== 714+0+0 (2365291454 0 0) 0x524d240 con 0x44265c0 -8913> 2014-08-13 18:06:08.341385 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8912> 2014-08-13 18:06:08.341389 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8911> 2014-08-13 18:06:08.341392 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d240 pg_info(1 pgs e194496:22.29a) v4 -8910> 2014-08-13 18:06:08.341384 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetLog -8909> 2014-08-13 18:06:08.341385 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 370 notify: (query_epoch:194496, epoch_sent:194496, info:2.2a( v 194215'930 (0'0,194215'930] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) -8908> 2014-08-13 18:06:08.341398 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.331008, event: header_read, request: pg_info(1 pgs e194496:22.29a) v4 -8907> 2014-08-13 18:06:08.341407 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.331009, event: throttled, request: pg_info(1 pgs e194496:22.29a) v4 -8906> 2014-08-13 18:06:08.341414 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.331111, event: all_read, request: pg_info(1 pgs e194496:22.29a) v4 -8905> 2014-08-13 18:06:08.341407 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring -8904> 2014-08-13 18:06:08.341427 7f5c7c7e4700 10 log is not dirty -8903> 2014-08-13 18:06:08.341425 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.341385, event: dispatched, request: pg_info(1 pgs e194496:22.29a) v4 -8902> 2014-08-13 18:06:08.341432 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.341432, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.29a) v4 -8901> 2014-08-13 18:06:08.341415 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.4 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8900> 2014-08-13 18:06:08.341441 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.29a) v4 from osd.345 -8899> 2014-08-13 18:06:08.341447 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d240 -8898> 2014-08-13 18:06:08.341432 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 370 notify: (query_epoch:194496, epoch_sent:194496, info:1.2b( v 194215'1672 (0'0,194215'1672] local-les=194492 n=0 ec=1 les/c 194492/194492 194495/194495/194495)) -8897> 2014-08-13 18:06:08.341411 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] flushed -8896> 2014-08-13 18:06:08.341452 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.341452, event: started, request: pg_info(1 pgs e194496:22.29a) v4 -8895> 2014-08-13 18:06:08.341441 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.55 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -8894> 2014-08-13 18:06:08.341454 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 370, already have info from that osd, ignoring -8893> 2014-08-13 18:06:08.341477 7f5c7c7e4700 10 log is not dirty -8892> 2014-08-13 18:06:08.341481 7f5c835ef700 5 -- op tracker -- , seq: 531, time: 2014-08-13 18:06:08.341481, event: done, request: pg_info(1 pgs e194496:22.29a) v4 -8891> 2014-08-13 18:06:08.341468 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8890> 2014-08-13 18:06:08.341493 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8889> 2014-08-13 18:06:08.341496 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8888> 2014-08-13 18:06:08.341482 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 311 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -8887> 2014-08-13 18:06:08.341490 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.307 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8886> 2014-08-13 18:06:08.341503 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.323 192.168.1.106:6816/8118 6 ==== pg_info(1 pgs e194496:22.12a) v4 ==== 714+0+0 (1100324420 0 0) 0x4a440c0 con 0x49a2520 -8885> 2014-08-13 18:06:08.341503 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 311, already have info from that osd, ignoring -8884> 2014-08-13 18:06:08.341517 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8883> 2014-08-13 18:06:08.341520 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8882> 2014-08-13 18:06:08.341521 7f5c7c7e4700 10 log is not dirty -8881> 2014-08-13 18:06:08.341523 7f5c835ef700 20 osd.4 194496 _dispatch 0x4a440c0 pg_info(1 pgs e194496:22.12a) v4 -8880> 2014-08-13 18:06:08.341511 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8879> 2014-08-13 18:06:08.341528 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.332048, event: header_read, request: pg_info(1 pgs e194496:22.12a) v4 -8878> 2014-08-13 18:06:08.341528 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 312 notify: (query_epoch:194496, epoch_sent:194496, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -8877> 2014-08-13 18:06:08.341536 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.332050, event: throttled, request: pg_info(1 pgs e194496:22.12a) v4 -8876> 2014-08-13 18:06:08.341547 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.332133, event: all_read, request: pg_info(1 pgs e194496:22.12a) v4 -8875> 2014-08-13 18:06:08.341533 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -8874> 2014-08-13 18:06:08.341554 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.341517, event: dispatched, request: pg_info(1 pgs e194496:22.12a) v4 -8873> 2014-08-13 18:06:08.341547 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 312, already have info from that osd, ignoring -8872> 2014-08-13 18:06:08.341566 7f5c7c7e4700 10 log is not dirty -8871> 2014-08-13 18:06:08.341561 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.341561, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.12a) v4 -8870> 2014-08-13 18:06:08.341569 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.12a) v4 from osd.323 -8869> 2014-08-13 18:06:08.341574 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4a440c0 -8868> 2014-08-13 18:06:08.341572 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 350 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -8867> 2014-08-13 18:06:08.341588 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.341588, event: started, request: pg_info(1 pgs e194496:22.12a) v4 -8866> 2014-08-13 18:06:08.341597 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8865> 2014-08-13 18:06:08.341611 7f5c835ef700 5 -- op tracker -- , seq: 532, time: 2014-08-13 18:06:08.341610, event: done, request: pg_info(1 pgs e194496:22.12a) v4 -8864> 2014-08-13 18:06:08.341633 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8863> 2014-08-13 18:06:08.341637 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8862> 2014-08-13 18:06:08.341629 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged -8861> 2014-08-13 18:06:08.341652 7f5c7c7e4700 10 log is not dirty -8860> 2014-08-13 18:06:08.341642 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.287 192.168.1.109:6814/33435 5 ==== pg_notify(22.481(49) epoch 194496) v5 ==== 3465+0+0 (3334857523 0 0) 0x4df4ec0 con 0x49a7bc0 -8859> 2014-08-13 18:06:08.341658 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8858> 2014-08-13 18:06:08.341640 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] calc_acting newest update on osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) shard 344 (up) backfill 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) osd.296 (up) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) shard 320 (stray) accepted 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -8857> 2014-08-13 18:06:08.341661 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8856> 2014-08-13 18:06:08.341664 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df4ec0 pg_notify(22.481(49) epoch 194496) v5 -8855> 2014-08-13 18:06:08.341659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 381 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -8854> 2014-08-13 18:06:08.341663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] actingbackfill is 4,296,320,344 -8853> 2014-08-13 18:06:08.341670 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.336543, event: header_read, request: pg_notify(22.481(49) epoch 194496) v5 -8852> 2014-08-13 18:06:08.341681 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 381, already have info from that osd, ignoring -8851> 2014-08-13 18:06:08.341698 7f5c7c7e4700 10 log is not dirty -8850> 2014-08-13 18:06:08.341683 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] choose_acting want [4,296,320] (== acting) backfill_targets 344 -8849> 2014-08-13 18:06:08.341698 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.336545, event: throttled, request: pg_notify(22.481(49) epoch 194496) v5 -8848> 2014-08-13 18:06:08.341705 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.336688, event: all_read, request: pg_notify(22.481(49) epoch 194496) v5 -8847> 2014-08-13 18:06:08.341705 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8846> 2014-08-13 18:06:08.341708 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: leaving GetLog -8845> 2014-08-13 18:06:08.341723 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.341657, event: dispatched, request: pg_notify(22.481(49) epoch 194496) v5 -8844> 2014-08-13 18:06:08.341732 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.341732, event: waiting_for_osdmap, request: pg_notify(22.481(49) epoch 194496) v5 -8843> 2014-08-13 18:06:08.341738 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.287 -8842> 2014-08-13 18:06:08.341742 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df4ec0 -8841> 2014-08-13 18:06:08.341729 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] requeue_ops -8840> 2014-08-13 18:06:08.341749 7f5c7c7e4700 10 log is not dirty -8839> 2014-08-13 18:06:08.341733 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetLog 0.000350 0 0.000000 -8838> 2014-08-13 18:06:08.341748 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.341748, event: started, request: pg_notify(22.481(49) epoch 194496) v5 -8837> 2014-08-13 18:06:08.341757 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8836> 2014-08-13 18:06:08.341760 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/GetMissing -8835> 2014-08-13 18:06:08.341780 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.296 has no missing, identical log -8834> 2014-08-13 18:06:08.341800 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.320 has no missing, identical log -8833> 2014-08-13 18:06:08.341814 7f5c835ef700 5 -- op tracker -- , seq: 533, time: 2014-08-13 18:06:08.341814, event: done, request: pg_notify(22.481(49) epoch 194496) v5 -8832> 2014-08-13 18:06:08.341806 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 59 notify: (query_epoch:194496, epoch_sent:194496, info:24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8831> 2014-08-13 18:06:08.341829 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8830> 2014-08-13 18:06:08.341833 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8829> 2014-08-13 18:06:08.341819 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: osd.344 will fully backfill; can infer empty missing set -8828> 2014-08-13 18:06:08.341831 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] got osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8827> 2014-08-13 18:06:08.341840 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: still need up_thru update before going active -8826> 2014-08-13 18:06:08.341862 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/GetMissing 0.000101 0 0.000000 -8825> 2014-08-13 18:06:08.341862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] update_heartbeat_peers 4,55,59,351,376 unchanged -8824> 2014-08-13 18:06:08.341885 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] enter Started/Primary/Peering/WaitUpThru -8823> 2014-08-13 18:06:08.341865 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.341 192.168.1.106:6818/9142 3 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4ee5340 con 0x4ab4620 -8822> 2014-08-13 18:06:08.341905 7f5c7bde3700 10 log is not dirty -8821> 2014-08-13 18:06:08.341942 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8820> 2014-08-13 18:06:08.341947 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8819> 2014-08-13 18:06:08.341887 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetInfo 0.090341 8 0.000658 -8818> 2014-08-13 18:06:08.341950 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee5340 osd_map(194496..194496 src has 185139..194496) v3 -8817> 2014-08-13 18:06:08.341959 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -8816> 2014-08-13 18:06:08.341964 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -8815> 2014-08-13 18:06:08.341967 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8814> 2014-08-13 18:06:08.341969 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8813> 2014-08-13 18:06:08.341959 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetLog -8812> 2014-08-13 18:06:08.341978 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 31 info: 22.99( v 194215'504 lc 0'0 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) -8811> 2014-08-13 18:06:08.341977 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.292 192.168.1.113:6812/16056 6 ==== pg_info(1 pgs e194496:22.3ad) v4 ==== 714+0+0 (109918133 0 0) 0x524d080 con 0x4ab32e0 -8810> 2014-08-13 18:06:08.341987 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.4 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -8809> 2014-08-13 18:06:08.342011 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8808> 2014-08-13 18:06:08.342005 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] state: peer osd.31 activated and committed -8807> 2014-08-13 18:06:08.342015 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8806> 2014-08-13 18:06:08.342010 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -8805> 2014-08-13 18:06:08.342031 7f5c835ef700 20 osd.4 194496 _dispatch 0x524d080 pg_info(1 pgs e194496:22.3ad) v4 -8804> 2014-08-13 18:06:08.342039 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.332464, event: header_read, request: pg_info(1 pgs e194496:22.3ad) v4 -8803> 2014-08-13 18:06:08.342033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8802> 2014-08-13 18:06:08.342047 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.332467, event: throttled, request: pg_info(1 pgs e194496:22.3ad) v4 -8801> 2014-08-13 18:06:08.342024 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8800> 2014-08-13 18:06:08.342052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -8799> 2014-08-13 18:06:08.342069 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8798> 2014-08-13 18:06:08.342054 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.332540, event: all_read, request: pg_info(1 pgs e194496:22.3ad) v4 -8797> 2014-08-13 18:06:08.342074 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting osd.376 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -8796> 2014-08-13 18:06:08.342094 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 -8795> 2014-08-13 18:06:08.342091 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.342010, event: dispatched, request: pg_info(1 pgs e194496:22.3ad) v4 -8794> 2014-08-13 18:06:08.342104 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.342104, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3ad) v4 -8793> 2014-08-13 18:06:08.342098 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194496:22.99) v4 -- ?+0 0x5354980 -8792> 2014-08-13 18:06:08.342111 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3ad) v4 from osd.292 -8791> 2014-08-13 18:06:08.342116 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524d080 -8790> 2014-08-13 18:06:08.342121 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.342121, event: started, request: pg_info(1 pgs e194496:22.3ad) v4 -8789> 2014-08-13 18:06:08.342131 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] calc_acting newest update on osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) shard 59 (up) backfill 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) osd.351 (up) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) shard 55 (stray) accepted 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -8788> 2014-08-13 18:06:08.342143 7f5c835ef700 5 -- op tracker -- , seq: 534, time: 2014-08-13 18:06:08.342143, event: done, request: pg_info(1 pgs e194496:22.3ad) v4 -8787> 2014-08-13 18:06:08.342154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] actingbackfill is 4,55,59,351 -8786> 2014-08-13 18:06:08.342138 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] flushed -8785> 2014-08-13 18:06:08.342173 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] choose_acting want [4,351,55] (== acting) backfill_targets 59 -8784> 2014-08-13 18:06:08.342186 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8783> 2014-08-13 18:06:08.342192 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8782> 2014-08-13 18:06:08.342199 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.51 192.168.1.112:6832/6655 6 ==== pg_info(1 pgs e194496:22.537) v4 ==== 714+0+0 (2543140109 0 0) 0x524cd00 con 0x3cf1600 -8781> 2014-08-13 18:06:08.342215 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8780> 2014-08-13 18:06:08.342218 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8779> 2014-08-13 18:06:08.342212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: leaving GetLog -8778> 2014-08-13 18:06:08.342232 7f5c835ef700 20 osd.4 194496 _dispatch 0x524cd00 pg_info(1 pgs e194496:22.537) v4 -8777> 2014-08-13 18:06:08.342249 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.332876, event: header_read, request: pg_info(1 pgs e194496:22.537) v4 -8776> 2014-08-13 18:06:08.342241 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetLog 0.000282 0 0.000000 -8775> 2014-08-13 18:06:08.342258 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.332877, event: throttled, request: pg_info(1 pgs e194496:22.537) v4 -8774> 2014-08-13 18:06:08.342264 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.332961, event: all_read, request: pg_info(1 pgs e194496:22.537) v4 -8773> 2014-08-13 18:06:08.342264 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing -8772> 2014-08-13 18:06:08.342270 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.342214, event: dispatched, request: pg_info(1 pgs e194496:22.537) v4 -8771> 2014-08-13 18:06:08.342264 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/GetMissing -8770> 2014-08-13 18:06:08.342273 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194496:22.99) v4 -- ?+0 0x53547c0 -8769> 2014-08-13 18:06:08.342282 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.342281, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.537) v4 -8768> 2014-08-13 18:06:08.342284 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.55 has no missing, identical log -8767> 2014-08-13 18:06:08.342302 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.537) v4 from osd.51 -8766> 2014-08-13 18:06:08.342308 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x524cd00 -8765> 2014-08-13 18:06:08.342304 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: osd.351 has no missing, identical log -8764> 2014-08-13 18:06:08.342300 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2875 -8763> 2014-08-13 18:06:08.342337 7f5c7bde3700 10 log is not dirty -8762> 2014-08-13 18:06:08.342319 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.342318, event: started, request: pg_info(1 pgs e194496:22.537) v4 -8761> 2014-08-13 18:06:08.342323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: still need up_thru update before going active -8760> 2014-08-13 18:06:08.342356 7f5c835ef700 5 -- op tracker -- , seq: 535, time: 2014-08-13 18:06:08.342356, event: done, request: pg_info(1 pgs e194496:22.537) v4 -8759> 2014-08-13 18:06:08.342345 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/GetMissing 0.000082 0 0.000000 -8758> 2014-08-13 18:06:08.342368 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8757> 2014-08-13 18:06:08.342372 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8756> 2014-08-13 18:06:08.342367 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] enter Started/Primary/Peering/WaitUpThru -8755> 2014-08-13 18:06:08.342386 7f5c7c7e4700 10 log is not dirty -8754> 2014-08-13 18:06:08.342382 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 18 ==== MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 ==== 26+0+0 (3428140277 0 0) 0x588a140 con 0x4b2bc80 -8753> 2014-08-13 18:06:08.342398 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8752> 2014-08-13 18:06:08.342381 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 353 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) -8751> 2014-08-13 18:06:08.342401 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8750> 2014-08-13 18:06:08.342404 7f5c835ef700 20 osd.4 194496 _dispatch 0x588a140 MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8749> 2014-08-13 18:06:08.342410 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.336304, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8748> 2014-08-13 18:06:08.342405 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.353 activated and committed -8747> 2014-08-13 18:06:08.342417 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.336306, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8746> 2014-08-13 18:06:08.342428 7f5c7bde3700 10 log is not dirty -8745> 2014-08-13 18:06:08.342416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 291 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -8744> 2014-08-13 18:06:08.342447 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.336386, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8743> 2014-08-13 18:06:08.342460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8742> 2014-08-13 18:06:08.342483 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8741> 2014-08-13 18:06:08.342501 7f5c7bde3700 10 log is not dirty -8740> 2014-08-13 18:06:08.342478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8739> 2014-08-13 18:06:08.342472 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.342398, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8738> 2014-08-13 18:06:08.342507 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8737> 2014-08-13 18:06:08.342520 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.342520, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8736> 2014-08-13 18:06:08.342530 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588a140 -8735> 2014-08-13 18:06:08.342519 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,291,350,365,367,378 unchanged -8734> 2014-08-13 18:06:08.342526 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8733> 2014-08-13 18:06:08.342544 7f5c7bde3700 10 log is not dirty -8732> 2014-08-13 18:06:08.342545 7f5c835ef700 5 -- op tracker -- , seq: 536, time: 2014-08-13 18:06:08.342545, event: done, request: MRecoveryReserve REQUEST pgid: 22.226, query_epoch: 194496 v2 -8731> 2014-08-13 18:06:08.342544 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.127113 7 0.001501 -8730> 2014-08-13 18:06:08.342559 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8729> 2014-08-13 18:06:08.342552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 362 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) -8728> 2014-08-13 18:06:08.342539 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] flushed -8727> 2014-08-13 18:06:08.342569 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8726> 2014-08-13 18:06:08.342572 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.362 activated and committed -8725> 2014-08-13 18:06:08.342575 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.7 192.168.1.115:6838/62713 7 ==== pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 ==== 1644+0+0 (1687547409 0 0) 0x5355940 con 0x4c58f20 -8724> 2014-08-13 18:06:08.342574 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -8723> 2014-08-13 18:06:08.342592 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8722> 2014-08-13 18:06:08.342591 7f5c7bde3700 10 log is not dirty -8721> 2014-08-13 18:06:08.342599 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8720> 2014-08-13 18:06:08.342601 7f5c835ef700 20 osd.4 194496 _dispatch 0x5355940 pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8719> 2014-08-13 18:06:08.342602 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8718> 2014-08-13 18:06:08.342602 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8717> 2014-08-13 18:06:08.342626 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] requeue_ops -8716> 2014-08-13 18:06:08.342626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8715> 2014-08-13 18:06:08.342648 7f5c7bde3700 10 log is not dirty -8714> 2014-08-13 18:06:08.342609 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.335722, event: header_read, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8713> 2014-08-13 18:06:08.342651 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8712> 2014-08-13 18:06:08.342658 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 20 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -8711> 2014-08-13 18:06:08.342674 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.365 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8710> 2014-08-13 18:06:08.342681 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.20 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8709> 2014-08-13 18:06:08.342706 7f5c7bde3700 10 log is not dirty -8708> 2014-08-13 18:06:08.342695 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8707> 2014-08-13 18:06:08.342694 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.342871) v2 ==== 47+0+0 (1716667214 0 0) 0x58888c0 con 0x4f88c60 -8706> 2014-08-13 18:06:08.342712 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 346 query_epoch 194496 query: query(log 194422'15014 ec=192688 les/c 194493/194493 194495/194496/192955) -8705> 2014-08-13 18:06:08.342715 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8704> 2014-08-13 18:06:08.342733 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.342871) v2 -- ?+0 0x4df1c00 con 0x4f88c60 -8703> 2014-08-13 18:06:08.342732 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 -8702> 2014-08-13 18:06:08.342755 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.289 now has 194496 -8701> 2014-08-13 18:06:08.342727 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.289 192.168.100.113:0/15284 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.342871) v2 ==== 47+0+0 (1716667214 0 0) 0x5888a80 con 0x4f20580 -8700> 2014-08-13 18:06:08.342764 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 -8699> 2014-08-13 18:06:08.342772 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.335724, event: throttled, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8698> 2014-08-13 18:06:08.342769 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) shard 378 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 367 (up) backfill 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 291 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) shard 350 (stray) accepted 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -8697> 2014-08-13 18:06:08.342787 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.335819, event: all_read, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8696> 2014-08-13 18:06:08.342790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,291,350,367,378 -8695> 2014-08-13 18:06:08.342794 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194422'15014 -8694> 2014-08-13 18:06:08.342783 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.113:0/15284 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.342871) v2 -- ?+0 0x4df5400 con 0x4f20580 -8693> 2014-08-13 18:06:08.342815 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.342592, event: dispatched, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8692> 2014-08-13 18:06:08.342826 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.342826, event: waiting_for_osdmap, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8691> 2014-08-13 18:06:08.342814 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) -8690> 2014-08-13 18:06:08.342834 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.7 -8689> 2014-08-13 18:06:08.342839 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5355940 -8688> 2014-08-13 18:06:08.342842 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 -8687> 2014-08-13 18:06:08.342844 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.342843, event: started, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8686> 2014-08-13 18:06:08.342848 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4f21600 already has epoch 194496 -8685> 2014-08-13 18:06:08.342853 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194496 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194496) v4 -- ?+0 0x4162a00 con 0x4f21600 -8684> 2014-08-13 18:06:08.342880 7f5c7bde3700 10 log is not dirty -8683> 2014-08-13 18:06:08.342882 7f5c835ef700 5 -- op tracker -- , seq: 537, time: 2014-08-13 18:06:08.342881, event: done, request: pg_notify(22.1b8(2),22.84(2) epoch 194496) v5 -8682> 2014-08-13 18:06:08.342895 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8681> 2014-08-13 18:06:08.342898 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8680> 2014-08-13 18:06:08.342809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,291,350] (== acting) backfill_targets 367,378 -8679> 2014-08-13 18:06:08.342921 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.318 192.168.1.110:6814/43857 3 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4ee4800 con 0x3988dc0 -8678> 2014-08-13 18:06:08.342957 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8677> 2014-08-13 18:06:08.342962 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8676> 2014-08-13 18:06:08.342945 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog -8675> 2014-08-13 18:06:08.342964 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee4800 osd_map(194496..194496 src has 185139..194496) v3 -8674> 2014-08-13 18:06:08.342973 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -8673> 2014-08-13 18:06:08.342988 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -8672> 2014-08-13 18:06:08.342974 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000401 0 0.000000 -8671> 2014-08-13 18:06:08.342992 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8670> 2014-08-13 18:06:08.342994 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8669> 2014-08-13 18:06:08.343000 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -8668> 2014-08-13 18:06:08.343005 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.348 192.168.1.110:6800/41204 4 ==== pg_notify(24.1bb epoch 194496) v5 ==== 694+0+0 (660771902 0 0) 0x4dea300 con 0x571b700 -8667> 2014-08-13 18:06:08.343018 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.291 has no missing, identical log -8666> 2014-08-13 18:06:08.343020 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8665> 2014-08-13 18:06:08.343039 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8664> 2014-08-13 18:06:08.343042 7f5c835ef700 20 osd.4 194496 _dispatch 0x4dea300 pg_notify(24.1bb epoch 194496) v5 -8663> 2014-08-13 18:06:08.343038 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.350 has no missing, identical log -8662> 2014-08-13 18:06:08.343051 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.335540, event: header_read, request: pg_notify(24.1bb epoch 194496) v5 -8661> 2014-08-13 18:06:08.343059 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.335541, event: throttled, request: pg_notify(24.1bb epoch 194496) v5 -8660> 2014-08-13 18:06:08.343057 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active -8659> 2014-08-13 18:06:08.343066 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.335624, event: all_read, request: pg_notify(24.1bb epoch 194496) v5 -8658> 2014-08-13 18:06:08.343078 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.343019, event: dispatched, request: pg_notify(24.1bb epoch 194496) v5 -8657> 2014-08-13 18:06:08.343061 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 304 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193250) -8656> 2014-08-13 18:06:08.343085 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.343085, event: waiting_for_osdmap, request: pg_notify(24.1bb epoch 194496) v5 -8655> 2014-08-13 18:06:08.343092 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.348 -8654> 2014-08-13 18:06:08.343080 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 -8653> 2014-08-13 18:06:08.343100 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4dea300 -8652> 2014-08-13 18:06:08.343093 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] log request from 304 -8651> 2014-08-13 18:06:08.343103 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -8650> 2014-08-13 18:06:08.343125 7f5c7c7e4700 10 log is not dirty -8649> 2014-08-13 18:06:08.343105 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.343105, event: started, request: pg_notify(24.1bb epoch 194496) v5 -8648> 2014-08-13 18:06:08.343148 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending info+missing+log since 194487'0 -8647> 2014-08-13 18:06:08.343156 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 50 notify: (query_epoch:194496, epoch_sent:194496, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -8646> 2014-08-13 18:06:08.343177 7f5c835ef700 5 -- op tracker -- , seq: 538, time: 2014-08-13 18:06:08.343177, event: done, request: pg_notify(24.1bb epoch 194496) v5 -8645> 2014-08-13 18:06:08.343166 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] sending log((194422'16034,194422'16034], crt=194422'16034) missing(61) -8644> 2014-08-13 18:06:08.343196 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 -8643> 2014-08-13 18:06:08.343190 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8642> 2014-08-13 18:06:08.343183 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle_pg_notify from osd.50 -8641> 2014-08-13 18:06:08.343200 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6808/22782 -- pg_log(24.33a epoch 194496 log log((194422'16034,194422'16034], crt=194422'16034) query_epoch 194496) v4 -- ?+0 0x4162400 con 0x4ab69e0 -8640> 2014-08-13 18:06:08.343232 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8639> 2014-08-13 18:06:08.343242 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 13 ==== pg_query(24.3b7 epoch 194496) v3 ==== 144+0+0 (542539684 0 0) 0x5425460 con 0x3c527e0 -8638> 2014-08-13 18:06:08.343265 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8637> 2014-08-13 18:06:08.343212 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] got dup osd.50 info 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495), identical to ours -8636> 2014-08-13 18:06:08.343276 7f5c7c7e4700 10 log is not dirty -8635> 2014-08-13 18:06:08.343268 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8634> 2014-08-13 18:06:08.343302 7f5c835ef700 20 osd.4 194496 _dispatch 0x5425460 pg_query(24.3b7 epoch 194496) v3 -8633> 2014-08-13 18:06:08.343309 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.341791, event: header_read, request: pg_query(24.3b7 epoch 194496) v3 -8632> 2014-08-13 18:06:08.343317 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.341793, event: throttled, request: pg_query(24.3b7 epoch 194496) v3 -8631> 2014-08-13 18:06:08.343323 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.341866, event: all_read, request: pg_query(24.3b7 epoch 194496) v3 -8630> 2014-08-13 18:06:08.343328 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.343257, event: dispatched, request: pg_query(24.3b7 epoch 194496) v3 -8629> 2014-08-13 18:06:08.343335 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.343334, event: waiting_for_osdmap, request: pg_query(24.3b7 epoch 194496) v3 -8628> 2014-08-13 18:06:08.343340 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.356 epoch 194496 -8627> 2014-08-13 18:06:08.343344 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5425460 -8626> 2014-08-13 18:06:08.343350 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.343350, event: started, request: pg_query(24.3b7 epoch 194496) v3 -8625> 2014-08-13 18:06:08.343338 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed -8624> 2014-08-13 18:06:08.343358 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192969) from replica 356 -8623> 2014-08-13 18:06:08.343418 7f5c835ef700 5 -- op tracker -- , seq: 539, time: 2014-08-13 18:06:08.343417, event: done, request: pg_query(24.3b7 epoch 194496) v3 -8622> 2014-08-13 18:06:08.343430 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8621> 2014-08-13 18:06:08.343433 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8620> 2014-08-13 18:06:08.343451 7f5c7bde3700 10 log is not dirty -8619> 2014-08-13 18:06:08.343459 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 357 info: 22.4df( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) -8618> 2014-08-13 18:06:08.343439 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.287 192.168.1.109:6814/33435 6 ==== pg_info(1 pgs e194496:22.481) v4 ==== 714+0+0 (856130319 0 0) 0x5353d40 con 0x49a7bc0 -8617> 2014-08-13 18:06:08.343485 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8616> 2014-08-13 18:06:08.343466 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 362 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8615> 2014-08-13 18:06:08.343485 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] state: peer osd.357 activated and committed -8614> 2014-08-13 18:06:08.343497 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8613> 2014-08-13 18:06:08.343504 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194493/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8612> 2014-08-13 18:06:08.343490 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8611> 2014-08-13 18:06:08.343523 7f5c835ef700 20 osd.4 194496 _dispatch 0x5353d40 pg_info(1 pgs e194496:22.481) v4 -8610> 2014-08-13 18:06:08.343520 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8609> 2014-08-13 18:06:08.343531 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.339823, event: header_read, request: pg_info(1 pgs e194496:22.481) v4 -8608> 2014-08-13 18:06:08.343540 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.339828, event: throttled, request: pg_info(1 pgs e194496:22.481) v4 -8607> 2014-08-13 18:06:08.343544 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -8606> 2014-08-13 18:06:08.343546 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.339922, event: all_read, request: pg_info(1 pgs e194496:22.481) v4 -8605> 2014-08-13 18:06:08.343531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged -8604> 2014-08-13 18:06:08.343554 7f5c7bde3700 10 log is not dirty -8603> 2014-08-13 18:06:08.343552 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.343485, event: dispatched, request: pg_info(1 pgs e194496:22.481) v4 -8602> 2014-08-13 18:06:08.343549 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194496:22.4df) v4 -- ?+0 0x588abc0 -8601> 2014-08-13 18:06:08.343559 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.343559, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.481) v4 -8600> 2014-08-13 18:06:08.343584 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.481) v4 from osd.287 -8599> 2014-08-13 18:06:08.343589 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5353d40 -8598> 2014-08-13 18:06:08.343591 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4afcd00 don't know epoch, doing nothing -8597> 2014-08-13 18:06:08.343597 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.357 192.168.1.102:6834/17224 -- pg_info(1 pgs e194496:22.4df) v4 -- ?+0 0x5889dc0 -8596> 2014-08-13 18:06:08.343591 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 373 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -8595> 2014-08-13 18:06:08.343616 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.373 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8594> 2014-08-13 18:06:08.343621 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2662 -8593> 2014-08-13 18:06:08.343639 7f5c7bde3700 10 log is not dirty -8592> 2014-08-13 18:06:08.343649 7f5c7c7e4700 10 log is not dirty -8591> 2014-08-13 18:06:08.343646 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8590> 2014-08-13 18:06:08.343667 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] requeue_ops -8589> 2014-08-13 18:06:08.343685 7f5c7bde3700 10 log is not dirty -8588> 2014-08-13 18:06:08.343594 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.343594, event: started, request: pg_info(1 pgs e194496:22.481) v4 -8587> 2014-08-13 18:06:08.343709 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 312 info: 22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) -8586> 2014-08-13 18:06:08.343724 7f5c835ef700 5 -- op tracker -- , seq: 540, time: 2014-08-13 18:06:08.343724, event: done, request: pg_info(1 pgs e194496:22.481) v4 -8585> 2014-08-13 18:06:08.343733 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] state: peer osd.312 activated and committed -8584> 2014-08-13 18:06:08.343748 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8583> 2014-08-13 18:06:08.343753 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8582> 2014-08-13 18:06:08.343750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8581> 2014-08-13 18:06:08.343765 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8580> 2014-08-13 18:06:08.343786 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing -8579> 2014-08-13 18:06:08.343724 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 26 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -8578> 2014-08-13 18:06:08.343789 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194496:22.f3) v4 -- ?+0 0x5888700 -8577> 2014-08-13 18:06:08.343760 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.341 192.168.1.106:6818/9142 4 ==== pg_info(1 pgs e194496:22.48e) v4 ==== 714+0+0 (2612803502 0 0) 0x588bd40 con 0x4ab4620 -8576> 2014-08-13 18:06:08.343814 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c5aec0 don't know epoch, doing nothing -8575> 2014-08-13 18:06:08.343816 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8574> 2014-08-13 18:06:08.343821 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8573> 2014-08-13 18:06:08.343823 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bd40 pg_info(1 pgs e194496:22.48e) v4 -8572> 2014-08-13 18:06:08.343818 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.312 192.168.1.113:6816/17521 -- pg_info(1 pgs e194496:22.f3) v4 -- ?+0 0x588a300 -8571> 2014-08-13 18:06:08.343802 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.26 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours -8570> 2014-08-13 18:06:08.343836 7f5c7bde3700 10 log is not dirty -8569> 2014-08-13 18:06:08.343829 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.341309, event: header_read, request: pg_info(1 pgs e194496:22.48e) v4 -8568> 2014-08-13 18:06:08.343846 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.341310, event: throttled, request: pg_info(1 pgs e194496:22.48e) v4 -8567> 2014-08-13 18:06:08.343835 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2864 -8566> 2014-08-13 18:06:08.343841 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 30 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -8565> 2014-08-13 18:06:08.343862 7f5c7c7e4700 10 log is not dirty -8564> 2014-08-13 18:06:08.343858 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.341369, event: all_read, request: pg_info(1 pgs e194496:22.48e) v4 -8563> 2014-08-13 18:06:08.343868 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.343815, event: dispatched, request: pg_info(1 pgs e194496:22.48e) v4 -8562> 2014-08-13 18:06:08.343875 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.343875, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.48e) v4 -8561> 2014-08-13 18:06:08.343862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.30 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours -8560> 2014-08-13 18:06:08.343881 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.48e) v4 from osd.341 -8559> 2014-08-13 18:06:08.343885 7f5c7bde3700 10 log is not dirty -8558> 2014-08-13 18:06:08.343886 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bd40 -8557> 2014-08-13 18:06:08.343892 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.343891, event: started, request: pg_info(1 pgs e194496:22.48e) v4 -8556> 2014-08-13 18:06:08.343889 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 364 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192966) -8555> 2014-08-13 18:06:08.343913 7f5c835ef700 5 -- op tracker -- , seq: 541, time: 2014-08-13 18:06:08.343913, event: done, request: pg_info(1 pgs e194496:22.48e) v4 -8554> 2014-08-13 18:06:08.343925 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8553> 2014-08-13 18:06:08.343912 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] log request from 364 -8552> 2014-08-13 18:06:08.343929 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8551> 2014-08-13 18:06:08.343912 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 350 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -8550> 2014-08-13 18:06:08.343941 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.350 -8549> 2014-08-13 18:06:08.343954 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending info+missing+log since 194487'0 -8548> 2014-08-13 18:06:08.343935 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.318 192.168.1.110:6814/43857 4 ==== pg_info(1 pgs e194496:22.3e4) v4 ==== 714+0+0 (3119571383 0 0) 0x588b480 con 0x3988dc0 -8547> 2014-08-13 18:06:08.343973 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] sending log((194422'17074,194422'17074], crt=194422'17074) missing(48) -8546> 2014-08-13 18:06:08.343959 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.350 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8545> 2014-08-13 18:06:08.343992 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8544> 2014-08-13 18:06:08.343996 7f5c7c7e4700 10 log is not dirty -8543> 2014-08-13 18:06:08.343995 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing -8542> 2014-08-13 18:06:08.344002 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8541> 2014-08-13 18:06:08.344005 7f5c835ef700 20 osd.4 194496 _dispatch 0x588b480 pg_info(1 pgs e194496:22.3e4) v4 -8540> 2014-08-13 18:06:08.343999 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6826/52317 -- pg_log(24.1de epoch 194496 log log((194422'17074,194422'17074], crt=194422'17074) query_epoch 194496) v4 -- ?+0 0x4160c00 con 0x445ad60 -8539> 2014-08-13 18:06:08.344005 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 291 notify: (query_epoch:194496, epoch_sent:194496, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -8538> 2014-08-13 18:06:08.344026 7f5c7bde3700 10 log is not dirty -8537> 2014-08-13 18:06:08.344011 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.342046, event: header_read, request: pg_info(1 pgs e194496:22.3e4) v4 -8536> 2014-08-13 18:06:08.344032 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.342047, event: throttled, request: pg_info(1 pgs e194496:22.3e4) v4 -8535> 2014-08-13 18:06:08.344038 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.342116, event: all_read, request: pg_info(1 pgs e194496:22.3e4) v4 -8534> 2014-08-13 18:06:08.344026 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.291 -8533> 2014-08-13 18:06:08.344047 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.343987, event: dispatched, request: pg_info(1 pgs e194496:22.3e4) v4 -8532> 2014-08-13 18:06:08.344054 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.344054, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.3e4) v4 -8531> 2014-08-13 18:06:08.344061 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.3e4) v4 from osd.318 -8530> 2014-08-13 18:06:08.344076 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588b480 -8529> 2014-08-13 18:06:08.344048 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.291 info 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8528> 2014-08-13 18:06:08.344083 7f5c7c7e4700 10 log is not dirty -8527> 2014-08-13 18:06:08.344081 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.344081, event: started, request: pg_info(1 pgs e194496:22.3e4) v4 -8526> 2014-08-13 18:06:08.344111 7f5c835ef700 5 -- op tracker -- , seq: 542, time: 2014-08-13 18:06:08.344111, event: done, request: pg_info(1 pgs e194496:22.3e4) v4 -8525> 2014-08-13 18:06:08.344122 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8524> 2014-08-13 18:06:08.344126 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8523> 2014-08-13 18:06:08.344179 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8522> 2014-08-13 18:06:08.344197 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.346 192.168.1.104:6820/2308 6 ==== pg_query(24.e2 epoch 194496) v3 ==== 144+0+0 (672034657 0 0) 0x5421a40 con 0x4f21600 -8521> 2014-08-13 18:06:08.344189 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 50 info: 22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) -8520> 2014-08-13 18:06:08.344215 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8519> 2014-08-13 18:06:08.344219 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8518> 2014-08-13 18:06:08.344221 7f5c835ef700 20 osd.4 194496 _dispatch 0x5421a40 pg_query(24.e2 epoch 194496) v3 -8517> 2014-08-13 18:06:08.344228 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344005, event: header_read, request: pg_query(24.e2 epoch 194496) v3 -8516> 2014-08-13 18:06:08.344219 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.50 activated and committed -8515> 2014-08-13 18:06:08.344221 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 303 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -8514> 2014-08-13 18:06:08.344239 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8513> 2014-08-13 18:06:08.344236 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344009, event: throttled, request: pg_query(24.e2 epoch 194496) v3 -8512> 2014-08-13 18:06:08.344264 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344083, event: all_read, request: pg_query(24.e2 epoch 194496) v3 -8511> 2014-08-13 18:06:08.344255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8510> 2014-08-13 18:06:08.344271 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344214, event: dispatched, request: pg_query(24.e2 epoch 194496) v3 -8509> 2014-08-13 18:06:08.344255 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got dup osd.303 info 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8508> 2014-08-13 18:06:08.344278 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing -8507> 2014-08-13 18:06:08.344281 7f5c7bde3700 10 log is not dirty -8506> 2014-08-13 18:06:08.344277 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344277, event: waiting_for_osdmap, request: pg_query(24.e2 epoch 194496) v3 -8505> 2014-08-13 18:06:08.344285 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.346 epoch 194496 -8504> 2014-08-13 18:06:08.344288 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5421a40 -8503> 2014-08-13 18:06:08.344282 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194496:22.6bd) v4 -- ?+0 0x588e200 -8502> 2014-08-13 18:06:08.344293 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344293, event: started, request: pg_query(24.e2 epoch 194496) v3 -8501> 2014-08-13 18:06:08.344325 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 -8500> 2014-08-13 18:06:08.344332 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194496:22.6bd) v4 -- ?+0 0x5356e40 -8499> 2014-08-13 18:06:08.344301 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_query query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192955) from replica 346 -8498> 2014-08-13 18:06:08.344355 7f5c835ef700 5 -- op tracker -- , seq: 543, time: 2014-08-13 18:06:08.344354, event: done, request: pg_query(24.e2 epoch 194496) v3 -8497> 2014-08-13 18:06:08.344327 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 25 notify: (query_epoch:194496, epoch_sent:194496, info:24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8496> 2014-08-13 18:06:08.344354 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2710 -8495> 2014-08-13 18:06:08.344383 7f5c7c7e4700 10 log is not dirty -8494> 2014-08-13 18:06:08.344373 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] got osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8493> 2014-08-13 18:06:08.344415 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8492> 2014-08-13 18:06:08.344420 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8491> 2014-08-13 18:06:08.344410 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] update_heartbeat_peers 4,20,25,303,362,373 unchanged -8490> 2014-08-13 18:06:08.344433 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8489> 2014-08-13 18:06:08.344436 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetInfo 0.128992 9 0.050033 -8488> 2014-08-13 18:06:08.344461 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8487> 2014-08-13 18:06:08.344463 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetLog -8486> 2014-08-13 18:06:08.344479 7f5c7c7e4700 10 log is not dirty -8485> 2014-08-13 18:06:08.344486 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 28 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) -8484> 2014-08-13 18:06:08.344491 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.4 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -8483> 2014-08-13 18:06:08.344505 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.28 activated and committed -8482> 2014-08-13 18:06:08.344532 7f5c7c7e4700 10 log is not dirty -8481> 2014-08-13 18:06:08.344515 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -8480> 2014-08-13 18:06:08.344538 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 304 info: 22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) -8479> 2014-08-13 18:06:08.344537 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8478> 2014-08-13 18:06:08.344559 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: peer osd.304 activated and committed -8477> 2014-08-13 18:06:08.344560 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -8476> 2014-08-13 18:06:08.344576 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8475> 2014-08-13 18:06:08.344583 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8474> 2014-08-13 18:06:08.344592 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8473> 2014-08-13 18:06:08.344615 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing -8472> 2014-08-13 18:06:08.344603 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting osd.373 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -8471> 2014-08-13 18:06:08.344618 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194496:22.84) v4 -- ?+0 0x53516c0 -8470> 2014-08-13 18:06:08.344643 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 -8469> 2014-08-13 18:06:08.344647 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194496:22.84) v4 -- ?+0 0x4df24c0 -8468> 2014-08-13 18:06:08.344664 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2909 -8467> 2014-08-13 18:06:08.344663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] calc_acting newest update on osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) shard 25 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 362 (up) backfill 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 20 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) shard 303 (stray) accepted 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -8466> 2014-08-13 18:06:08.344690 7f5c7c7e4700 10 log is not dirty -8465> 2014-08-13 18:06:08.344686 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] actingbackfill is 4,20,25,303,362 -8464> 2014-08-13 18:06:08.344705 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] choose_acting want [4,20,303] (== acting) backfill_targets 25,362 -8463> 2014-08-13 18:06:08.344728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: leaving GetLog -8462> 2014-08-13 18:06:08.344752 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetLog 0.000290 0 0.000000 -8461> 2014-08-13 18:06:08.344764 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 330 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193227) -8460> 2014-08-13 18:06:08.344776 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/GetMissing -8459> 2014-08-13 18:06:08.344790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] log request from 330 -8458> 2014-08-13 18:06:08.344795 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.20 has no missing, identical log -8457> 2014-08-13 18:06:08.344816 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: osd.303 has no missing, identical log -8456> 2014-08-13 18:06:08.344834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: still need up_thru update before going active -8455> 2014-08-13 18:06:08.344841 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending info+missing+log since 194487'0 -8454> 2014-08-13 18:06:08.344856 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000 -8453> 2014-08-13 18:06:08.344861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] sending log((194422'15537,194422'15537], crt=194422'15537) missing(64) -8452> 2014-08-13 18:06:08.344883 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing -8451> 2014-08-13 18:06:08.344878 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] enter Started/Primary/Peering/WaitUpThru -8450> 2014-08-13 18:06:08.344886 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6802/4876 -- pg_log(24.3f7 epoch 194496 log log((194422'15537,194422'15537], crt=194422'15537) query_epoch 194496) v4 -- ?+0 0x4160000 con 0x4f20dc0 -8449> 2014-08-13 18:06:08.344899 7f5c7bde3700 10 log is not dirty -8448> 2014-08-13 18:06:08.344916 7f5c7c7e4700 10 log is not dirty -8447> 2014-08-13 18:06:08.345118 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8446> 2014-08-13 18:06:08.345143 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] requeue_ops -8445> 2014-08-13 18:06:08.345162 7f5c7c7e4700 10 log is not dirty -8444> 2014-08-13 18:06:08.345147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 374 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193232) -8443> 2014-08-13 18:06:08.345171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8442> 2014-08-13 18:06:08.345186 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] log request from 374 -8441> 2014-08-13 18:06:08.345199 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops -8440> 2014-08-13 18:06:08.345217 7f5c7c7e4700 10 log is not dirty -8439> 2014-08-13 18:06:08.345224 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8438> 2014-08-13 18:06:08.345243 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops -8437> 2014-08-13 18:06:08.345260 7f5c7c7e4700 10 log is not dirty -8436> 2014-08-13 18:06:08.345244 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending info+missing+log since 194487'0 -8435> 2014-08-13 18:06:08.345263 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] sending log((194422'16449,194422'16449], crt=194422'16449) missing(52) -8434> 2014-08-13 18:06:08.345266 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8433> 2014-08-13 18:06:08.345285 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing -8432> 2014-08-13 18:06:08.345287 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] requeue_ops -8431> 2014-08-13 18:06:08.345303 7f5c7c7e4700 10 log is not dirty -8430> 2014-08-13 18:06:08.345290 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.102:6828/16129 -- pg_log(24.2ec epoch 194496 log log((194422'16449,194422'16449], crt=194422'16449) query_epoch 194496) v4 -- ?+0 0x58b9800 con 0x3cf4360 -8429> 2014-08-13 18:06:08.345320 7f5c7bde3700 10 log is not dirty -8428> 2014-08-13 18:06:08.345309 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192935) -8427> 2014-08-13 18:06:08.345308 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] flushed -8426> 2014-08-13 18:06:08.345366 7f5c7c7e4700 10 log is not dirty -8425> 2014-08-13 18:06:08.345527 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 44 info: 22.3af( v 194215'539 lc 0'0 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) -8424> 2014-08-13 18:06:08.345554 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] state: peer osd.44 activated and committed -8423> 2014-08-13 18:06:08.345553 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8422> 2014-08-13 18:06:08.345572 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194493/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8421> 2014-08-13 18:06:08.345580 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8420> 2014-08-13 18:06:08.345600 7f5c7c7e4700 10 log is not dirty -8419> 2014-08-13 18:06:08.345587 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8418> 2014-08-13 18:06:08.345612 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x449a3c0 don't know epoch, doing nothing -8417> 2014-08-13 18:06:08.345607 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 45 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193186) -8416> 2014-08-13 18:06:08.345620 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.44 192.168.1.111:6822/10782 -- pg_info(1 pgs e194496:22.3af) v4 -- ?+0 0x5351880 -8415> 2014-08-13 18:06:08.345626 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] log request from 45 -8414> 2014-08-13 18:06:08.345649 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing -8413> 2014-08-13 18:06:08.345653 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194496:22.3af) v4 -- ?+0 0x5352f40 -8412> 2014-08-13 18:06:08.345667 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194487'0 -8411> 2014-08-13 18:06:08.345674 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2687 -8410> 2014-08-13 18:06:08.345701 7f5c7bde3700 10 log is not dirty -8409> 2014-08-13 18:06:08.345685 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] sending log((194422'13576,194422'13576], crt=194422'13576) missing(50) -8408> 2014-08-13 18:06:08.345779 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4496720 don't know epoch, doing nothing -8407> 2014-08-13 18:06:08.345775 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8406> 2014-08-13 18:06:08.345785 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6828/6040 -- pg_log(24.369 epoch 194496 log log((194422'13576,194422'13576], crt=194422'13576) query_epoch 194496) v4 -- ?+0 0x4166000 con 0x4496720 -8405> 2014-08-13 18:06:08.345809 7f5c7c7e4700 10 log is not dirty -8404> 2014-08-13 18:06:08.345801 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] requeue_ops -8403> 2014-08-13 18:06:08.345819 7f5c7bde3700 10 log is not dirty -8402> 2014-08-13 18:06:08.345996 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8401> 2014-08-13 18:06:08.346003 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 315 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -8400> 2014-08-13 18:06:08.346021 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] requeue_ops -8399> 2014-08-13 18:06:08.346040 7f5c7c7e4700 10 log is not dirty -8398> 2014-08-13 18:06:08.346046 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8397> 2014-08-13 18:06:08.346029 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8396> 2014-08-13 18:06:08.346063 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] update_heartbeat_peers 4,17,28,315,327,330 unchanged -8395> 2014-08-13 18:06:08.346086 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetInfo 0.104823 8 0.001041 -8394> 2014-08-13 18:06:08.346094 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 58 info: 22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) -8393> 2014-08-13 18:06:08.346109 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetLog -8392> 2014-08-13 18:06:08.346147 7f5c7c7e4700 10 log is not dirty -8391> 2014-08-13 18:06:08.346133 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.4 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8390> 2014-08-13 18:06:08.346155 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8389> 2014-08-13 18:06:08.346171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194492/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194489 194495/194495/192972) -8388> 2014-08-13 18:06:08.346183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8387> 2014-08-13 18:06:08.346204 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8386> 2014-08-13 18:06:08.346233 7f5c7c7e4700 10 log is not dirty -8385> 2014-08-13 18:06:08.346225 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.327 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8384> 2014-08-13 18:06:08.346245 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8383> 2014-08-13 18:06:08.346258 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194492/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194489 194495/194495/192936) -8382> 2014-08-13 18:06:08.346307 7f5c7c7e4700 10 log is not dirty -8381> 2014-08-13 18:06:08.346300 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] calc_acting newest update on osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) shard 330 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 28 (up) backfill 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 17 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) shard 315 (stray) accepted 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -8380> 2014-08-13 18:06:08.346323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] actingbackfill is 4,17,28,315,330 -8379> 2014-08-13 18:06:08.346324 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194489 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 9 info: 22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) -8378> 2014-08-13 18:06:08.346343 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] choose_acting want [4,17,315] (== acting) backfill_targets 28,330 -8377> 2014-08-13 18:06:08.346371 7f5c7c7e4700 10 log is not dirty -8376> 2014-08-13 18:06:08.346365 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: leaving GetLog -8375> 2014-08-13 18:06:08.346389 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetLog 0.000279 0 0.000000 -8374> 2014-08-13 18:06:08.346386 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194489 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 54 info: 22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) -8373> 2014-08-13 18:06:08.346415 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/GetMissing -8372> 2014-08-13 18:06:08.346441 7f5c7c7e4700 10 log is not dirty -8371> 2014-08-13 18:06:08.346434 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.17 has no missing, identical log -8370> 2014-08-13 18:06:08.346453 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: osd.315 has no missing, identical log -8369> 2014-08-13 18:06:08.346458 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192971) -8368> 2014-08-13 18:06:08.346472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: still need up_thru update before going active -8367> 2014-08-13 18:06:08.346506 7f5c7c7e4700 10 log is not dirty -8366> 2014-08-13 18:06:08.346493 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/GetMissing 0.000079 0 0.000000 -8365> 2014-08-13 18:06:08.346515 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] enter Started/Primary/Peering/WaitUpThru -8364> 2014-08-13 18:06:08.346535 7f5c7bde3700 10 log is not dirty -8363> 2014-08-13 18:06:08.346522 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 321 info: 22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) -8362> 2014-08-13 18:06:08.346570 7f5c7c7e4700 10 log is not dirty -8361> 2014-08-13 18:06:08.346562 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8360> 2014-08-13 18:06:08.346588 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] requeue_ops -8359> 2014-08-13 18:06:08.346587 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8358> 2014-08-13 18:06:08.346608 7f5c7bde3700 10 log is not dirty -8357> 2014-08-13 18:06:08.346610 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] requeue_ops -8356> 2014-08-13 18:06:08.346627 7f5c7c7e4700 10 log is not dirty -8355> 2014-08-13 18:06:08.346615 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 315 notify: (query_epoch:194496, epoch_sent:194496, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -8354> 2014-08-13 18:06:08.346634 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8353> 2014-08-13 18:06:08.346637 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle_pg_notify from osd.315 -8352> 2014-08-13 18:06:08.346654 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8351> 2014-08-13 18:06:08.346672 7f5c7c7e4700 10 log is not dirty -8350> 2014-08-13 18:06:08.346657 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] got dup osd.315 info 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8349> 2014-08-13 18:06:08.346693 7f5c7bde3700 10 log is not dirty -8348> 2014-08-13 18:06:08.346678 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8347> 2014-08-13 18:06:08.346699 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] requeue_ops -8346> 2014-08-13 18:06:08.346716 7f5c7c7e4700 10 log is not dirty -8345> 2014-08-13 18:06:08.346702 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8344> 2014-08-13 18:06:08.346722 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192968) -8343> 2014-08-13 18:06:08.346734 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops -8342> 2014-08-13 18:06:08.346754 7f5c7bde3700 10 log is not dirty -8341> 2014-08-13 18:06:08.346773 7f5c7c7e4700 10 log is not dirty -8340> 2014-08-13 18:06:08.346761 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192966) -8339> 2014-08-13 18:06:08.346790 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8338> 2014-08-13 18:06:08.346814 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] requeue_ops -8337> 2014-08-13 18:06:08.346834 7f5c7c7e4700 10 log is not dirty -8336> 2014-08-13 18:06:08.346842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 61 info: 22.1d2( v 194215'510 lc 0'0 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) -8335> 2014-08-13 18:06:08.346856 7f5c7bde3700 10 log is not dirty -8334> 2014-08-13 18:06:08.346862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] state: peer osd.61 activated and committed -8333> 2014-08-13 18:06:08.346879 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194492/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8332> 2014-08-13 18:06:08.346881 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 19 info: 24.3bf( v 194422'13811 (192834'12014,194422'13811] local-les=194496 n=44 ec=192688 les/c 194496/194463 194495/194495/194468) -8331> 2014-08-13 18:06:08.346894 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8330> 2014-08-13 18:06:08.346917 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445b180 don't know epoch, doing nothing -8329> 2014-08-13 18:06:08.346921 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.61 192.168.1.111:6840/16566 -- pg_info(1 pgs e194496:22.1d2) v4 -- ?+0 0x5a69340 -8328> 2014-08-13 18:06:08.346937 7f5c7bde3700 10 log is not dirty -8327> 2014-08-13 18:06:08.346957 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4fa4fc0 don't know epoch, doing nothing -8326> 2014-08-13 18:06:08.346961 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.349 192.168.1.104:6800/61726 -- pg_info(1 pgs e194496:22.1d2) v4 -- ?+0 0x5a69180 -8325> 2014-08-13 18:06:08.346983 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3024 -8324> 2014-08-13 18:06:08.347008 7f5c7c7e4700 10 log is not dirty -8323> 2014-08-13 18:06:08.347002 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8322> 2014-08-13 18:06:08.347027 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] requeue_ops -8321> 2014-08-13 18:06:08.347045 7f5c7bde3700 10 log is not dirty -8320> 2014-08-13 18:06:08.347106 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 294 info: 22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) -8319> 2014-08-13 18:06:08.347114 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 358 info: 22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) -8318> 2014-08-13 18:06:08.353507 7f5c7c7e4700 10 log is not dirty -8317> 2014-08-13 18:06:08.347423 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] flushed -8316> 2014-08-13 18:06:08.353526 7f5c7bde3700 10 log is not dirty -8315> 2014-08-13 18:06:08.353531 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -8314> 2014-08-13 18:06:08.353552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8313> 2014-08-13 18:06:08.353557 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -8312> 2014-08-13 18:06:08.353578 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] requeue_ops -8311> 2014-08-13 18:06:08.353596 7f5c7bde3700 10 log is not dirty -8310> 2014-08-13 18:06:08.353591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] update_heartbeat_peers 4,26,30,340 unchanged -8309> 2014-08-13 18:06:08.353604 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8308> 2014-08-13 18:06:08.353611 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: last maybe_went_rw interval was interval(194488-194495 up [26,30,340](26) acting [26,30,340](26) maybe_went_rw) -8307> 2014-08-13 18:06:08.353625 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] requeue_ops -8306> 2014-08-13 18:06:08.353641 7f5c7bde3700 10 log is not dirty -8305> 2014-08-13 18:06:08.353637 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetInfo 0.103537 6 0.000793 -8304> 2014-08-13 18:06:08.353648 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8303> 2014-08-13 18:06:08.353660 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetLog -8302> 2014-08-13 18:06:08.353670 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8301> 2014-08-13 18:06:08.353688 7f5c7bde3700 10 log is not dirty -8300> 2014-08-13 18:06:08.353686 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.4 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) -8299> 2014-08-13 18:06:08.353695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192949) -8298> 2014-08-13 18:06:08.353707 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -8297> 2014-08-13 18:06:08.353743 7f5c7bde3700 10 log is not dirty -8296> 2014-08-13 18:06:08.353729 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -8295> 2014-08-13 18:06:08.353749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -8294> 2014-08-13 18:06:08.353781 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8293> 2014-08-13 18:06:08.353802 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] calc_acting newest update on osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) up[0] needs backfill, osd.4 selected as primary instead calc_acting primary is osd.4 with 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) shard 26 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) shard 30 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) shard 340 (up) backfill 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -8292> 2014-08-13 18:06:08.353805 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops -8291> 2014-08-13 18:06:08.353824 7f5c7bde3700 10 log is not dirty -8290> 2014-08-13 18:06:08.353822 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] actingbackfill is 4,26,30,340 -8289> 2014-08-13 18:06:08.353831 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8288> 2014-08-13 18:06:08.353840 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] choose_acting want [4] (== acting) backfill_targets 26,30,340 -8287> 2014-08-13 18:06:08.353852 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] requeue_ops -8286> 2014-08-13 18:06:08.353870 7f5c7bde3700 10 log is not dirty -8285> 2014-08-13 18:06:08.353862 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: leaving GetLog -8284> 2014-08-13 18:06:08.353876 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) -8283> 2014-08-13 18:06:08.353885 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetLog 0.000226 0 0.000000 -8282> 2014-08-13 18:06:08.353908 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/GetMissing -8281> 2014-08-13 18:06:08.353922 7f5c7bde3700 10 log is not dirty -8280> 2014-08-13 18:06:08.353928 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.26 will fully backfill; can infer empty missing set -8279> 2014-08-13 18:06:08.353949 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8278> 2014-08-13 18:06:08.353947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.30 will fully backfill; can infer empty missing set -8277> 2014-08-13 18:06:08.353964 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: osd.340 will fully backfill; can infer empty missing set -8276> 2014-08-13 18:06:08.353980 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: still need up_thru update before going active -8275> 2014-08-13 18:06:08.353988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194493/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 351 info: 22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) -8274> 2014-08-13 18:06:08.354000 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/GetMissing 0.000092 0 0.000000 -8273> 2014-08-13 18:06:08.354021 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] enter Started/Primary/Peering/WaitUpThru -8272> 2014-08-13 18:06:08.354039 7f5c7c7e4700 10 log is not dirty -8271> 2014-08-13 18:06:08.354040 7f5c7bde3700 10 log is not dirty -8270> 2014-08-13 18:06:08.354075 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8269> 2014-08-13 18:06:08.354087 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 320 info: 22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) -8268> 2014-08-13 18:06:08.354100 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] requeue_ops -8267> 2014-08-13 18:06:08.354121 7f5c7c7e4700 10 log is not dirty -8266> 2014-08-13 18:06:08.354140 7f5c7bde3700 10 log is not dirty -8265> 2014-08-13 18:06:08.354129 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8264> 2014-08-13 18:06:08.354149 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops -8263> 2014-08-13 18:06:08.354165 7f5c7c7e4700 10 log is not dirty -8262> 2014-08-13 18:06:08.354171 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194492/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) -8261> 2014-08-13 18:06:08.354185 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery -8260> 2014-08-13 18:06:08.354231 7f5c7c7e4700 10 log is not dirty -8259> 2014-08-13 18:06:08.354215 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.044314 3 0.000321 -8258> 2014-08-13 18:06:08.354241 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -8257> 2014-08-13 18:06:08.354255 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8256> 2014-08-13 18:06:08.354273 7f5c7bde3700 10 log is not dirty -8255> 2014-08-13 18:06:08.354280 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops -8254> 2014-08-13 18:06:08.354298 7f5c7c7e4700 10 log is not dirty -8253> 2014-08-13 18:06:08.354291 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192976) -8252> 2014-08-13 18:06:08.354305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194492/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194489 194495/194495/192976) -8251> 2014-08-13 18:06:08.354295 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] flushed -8250> 2014-08-13 18:06:08.354344 7f5c7bde3700 10 log is not dirty -8249> 2014-08-13 18:06:08.354354 7f5c7c7e4700 10 log is not dirty -8248> 2014-08-13 18:06:08.354389 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8247> 2014-08-13 18:06:08.354378 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery -8246> 2014-08-13 18:06:08.354399 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.043969 3 0.000160 -8245> 2014-08-13 18:06:08.354418 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -8244> 2014-08-13 18:06:08.354437 7f5c7bde3700 10 log is not dirty -8243> 2014-08-13 18:06:08.354430 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194489 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 30 info: 22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) -8242> 2014-08-13 18:06:08.354444 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192946) -8241> 2014-08-13 18:06:08.354479 7f5c7c7e4700 10 log is not dirty -8240> 2014-08-13 18:06:08.354492 7f5c7bde3700 10 log is not dirty -8239> 2014-08-13 18:06:08.354500 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(log 194422'15992 ec=192688 les/c 194493/194489 194495/194496/192967) -8238> 2014-08-13 18:06:08.354514 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 367 info: 22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) -8237> 2014-08-13 18:06:08.354521 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 -8236> 2014-08-13 18:06:08.354564 7f5c7bde3700 10 log is not dirty -8235> 2014-08-13 18:06:08.354566 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194422'15992 -8234> 2014-08-13 18:06:08.354581 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 334 info: 22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) -8233> 2014-08-13 18:06:08.354585 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) -8232> 2014-08-13 18:06:08.354605 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 -8231> 2014-08-13 18:06:08.354584 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.371 192.168.1.102:6832/16753 8 ==== pg_info(1 pgs e194496:22.653) v4 ==== 714+0+0 (180053725 0 0) 0x53540c0 con 0x4851e40 -8230> 2014-08-13 18:06:08.354612 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8229> 2014-08-13 18:06:08.354617 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8228> 2014-08-13 18:06:08.354619 7f5c835ef700 20 osd.4 194496 _dispatch 0x53540c0 pg_info(1 pgs e194496:22.653) v4 -8227> 2014-08-13 18:06:08.354609 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194496 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194496) v4 -- ?+0 0x4165a00 con 0x4af98c0 -8226> 2014-08-13 18:06:08.354627 7f5c7bde3700 10 log is not dirty -8225> 2014-08-13 18:06:08.354627 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354432, event: header_read, request: pg_info(1 pgs e194496:22.653) v4 -8224> 2014-08-13 18:06:08.354644 7f5c7c7e4700 10 log is not dirty -8223> 2014-08-13 18:06:08.354638 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354436, event: throttled, request: pg_info(1 pgs e194496:22.653) v4 -8222> 2014-08-13 18:06:08.354654 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354534, event: all_read, request: pg_info(1 pgs e194496:22.653) v4 -8221> 2014-08-13 18:06:08.354660 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354611, event: dispatched, request: pg_info(1 pgs e194496:22.653) v4 -8220> 2014-08-13 18:06:08.354667 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354667, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.653) v4 -8219> 2014-08-13 18:06:08.354674 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.653) v4 from osd.371 -8218> 2014-08-13 18:06:08.354679 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x53540c0 -8217> 2014-08-13 18:06:08.354668 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8216> 2014-08-13 18:06:08.354685 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354685, event: started, request: pg_info(1 pgs e194496:22.653) v4 -8215> 2014-08-13 18:06:08.354692 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] requeue_ops -8214> 2014-08-13 18:06:08.354711 7f5c7bde3700 10 log is not dirty -8213> 2014-08-13 18:06:08.354809 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194492/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/192943) -8212> 2014-08-13 18:06:08.354865 7f5c7c7e4700 10 log is not dirty -8211> 2014-08-13 18:06:08.354839 7f5c835ef700 5 -- op tracker -- , seq: 544, time: 2014-08-13 18:06:08.354838, event: done, request: pg_info(1 pgs e194496:22.653) v4 -8210> 2014-08-13 18:06:08.354892 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8209> 2014-08-13 18:06:08.354897 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8208> 2014-08-13 18:06:08.354884 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 337 info: 22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) -8207> 2014-08-13 18:06:08.354827 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8206> 2014-08-13 18:06:08.354915 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] requeue_ops -8205> 2014-08-13 18:06:08.354932 7f5c7c7e4700 10 log is not dirty -8204> 2014-08-13 18:06:08.354937 7f5c7bde3700 10 log is not dirty -8203> 2014-08-13 18:06:08.354950 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 286 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) -8202> 2014-08-13 18:06:08.354946 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8201> 2014-08-13 18:06:08.354974 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.286 activated and committed -8200> 2014-08-13 18:06:08.354993 7f5c7c7e4700 10 log is not dirty -8199> 2014-08-13 18:06:08.354993 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops -8198> 2014-08-13 18:06:08.355012 7f5c7bde3700 10 log is not dirty -8197> 2014-08-13 18:06:08.355000 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -8196> 2014-08-13 18:06:08.355020 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8195> 2014-08-13 18:06:08.355026 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] check_local -8194> 2014-08-13 18:06:08.355040 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] requeue_ops -8193> 2014-08-13 18:06:08.355057 7f5c7bde3700 10 log is not dirty -8192> 2014-08-13 18:06:08.355046 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] requeue_ops -8191> 2014-08-13 18:06:08.355065 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.45 has 27 missing -8190> 2014-08-13 18:06:08.355082 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] needs_recovery osd.339 has 27 missing -8189> 2014-08-13 18:06:08.355064 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8188> 2014-08-13 18:06:08.355097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -8187> 2014-08-13 18:06:08.355104 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] requeue_ops -8186> 2014-08-13 18:06:08.355122 7f5c7bde3700 10 log is not dirty -8185> 2014-08-13 18:06:08.355129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 343 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/193262) -8184> 2014-08-13 18:06:08.355184 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] log request from 343 -8183> 2014-08-13 18:06:08.355228 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending info+missing+log since 194487'0 -8182> 2014-08-13 18:06:08.355252 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] sending log((194422'14476,194422'14476], crt=194422'14476) missing(41) -8181> 2014-08-13 18:06:08.355277 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2938 -8180> 2014-08-13 18:06:08.355298 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -8179> 2014-08-13 18:06:08.355296 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] hit_set_clear -8178> 2014-08-13 18:06:08.355317 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] agent_stop -8177> 2014-08-13 18:06:08.355333 7f5c7c7e4700 10 log is not dirty -8176> 2014-08-13 18:06:08.355340 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8175> 2014-08-13 18:06:08.355360 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] requeue_ops -8174> 2014-08-13 18:06:08.355377 7f5c7c7e4700 10 log is not dirty -8173> 2014-08-13 18:06:08.355384 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8172> 2014-08-13 18:06:08.355402 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] requeue_ops -8171> 2014-08-13 18:06:08.355417 7f5c7c7e4700 10 log is not dirty -8170> 2014-08-13 18:06:08.355423 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8169> 2014-08-13 18:06:08.355441 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops -8168> 2014-08-13 18:06:08.355457 7f5c7c7e4700 10 log is not dirty -8167> 2014-08-13 18:06:08.355463 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -8166> 2014-08-13 18:06:08.355482 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] check_local -8165> 2014-08-13 18:06:08.355303 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6802/61960 -- pg_log(24.206 epoch 194496 log log((194422'14476,194422'14476], crt=194422'14476) query_epoch 194496) v4 -- ?+0 0x4167800 con 0x48506e0 -8164> 2014-08-13 18:06:08.355496 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] requeue_ops -8163> 2014-08-13 18:06:08.355492 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.371 192.168.1.102:6832/16753 9 ==== MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 ==== 26+0+0 (2690892917 0 0) 0x4df6900 con 0x4851e40 -8162> 2014-08-13 18:06:08.355516 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8161> 2014-08-13 18:06:08.355520 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8160> 2014-08-13 18:06:08.355511 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.62 has 29 missing -8159> 2014-08-13 18:06:08.355524 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df6900 MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8158> 2014-08-13 18:06:08.355532 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355344, event: header_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8157> 2014-08-13 18:06:08.355527 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] needs_recovery osd.65 has 29 missing -8156> 2014-08-13 18:06:08.355540 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355347, event: throttled, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8155> 2014-08-13 18:06:08.355547 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355436, event: all_read, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8154> 2014-08-13 18:06:08.355542 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -8153> 2014-08-13 18:06:08.355553 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355515, event: dispatched, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8152> 2014-08-13 18:06:08.355560 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355560, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8151> 2014-08-13 18:06:08.355567 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df6900 -8150> 2014-08-13 18:06:08.355584 7f5c835ef700 5 -- op tracker -- , seq: 545, time: 2014-08-13 18:06:08.355584, event: done, request: MRecoveryReserve REQUEST pgid: 22.653, query_epoch: 194496 v2 -8149> 2014-08-13 18:06:08.355608 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8148> 2014-08-13 18:06:08.355611 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8147> 2014-08-13 18:06:08.355711 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2738 -8146> 2014-08-13 18:06:08.355728 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] hit_set_clear -8145> 2014-08-13 18:06:08.355756 7f5c7bde3700 10 log is not dirty -8144> 2014-08-13 18:06:08.355750 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] agent_stop -8143> 2014-08-13 18:06:08.355766 7f5c7c7e4700 10 log is not dirty -8142> 2014-08-13 18:06:08.355773 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -8141> 2014-08-13 18:06:08.355774 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.6 192.168.1.107:6809/53141 6 ==== pg_query(24.cd epoch 194496) v3 ==== 144+0+0 (1475692056 0 0) 0x21af260 con 0x4af98c0 -8140> 2014-08-13 18:06:08.355795 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8139> 2014-08-13 18:06:08.355771 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 296 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -8138> 2014-08-13 18:06:08.355799 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8137> 2014-08-13 18:06:08.355802 7f5c835ef700 20 osd.4 194496 _dispatch 0x21af260 pg_query(24.cd epoch 194496) v3 -8136> 2014-08-13 18:06:08.355793 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] check_local -8135> 2014-08-13 18:06:08.355808 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355665, event: header_read, request: pg_query(24.cd epoch 194496) v3 -8134> 2014-08-13 18:06:08.355816 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355667, event: throttled, request: pg_query(24.cd epoch 194496) v3 -8133> 2014-08-13 18:06:08.355810 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] requeue_ops -8132> 2014-08-13 18:06:08.355822 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355738, event: all_read, request: pg_query(24.cd epoch 194496) v3 -8131> 2014-08-13 18:06:08.355828 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355794, event: dispatched, request: pg_query(24.cd epoch 194496) v3 -8130> 2014-08-13 18:06:08.355825 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.5 has 35 missing -8129> 2014-08-13 18:06:08.355835 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355835, event: waiting_for_osdmap, request: pg_query(24.cd epoch 194496) v3 -8128> 2014-08-13 18:06:08.355847 7f5c835ef700 7 osd.4 194496 handle_pg_query from osd.6 epoch 194496 -8127> 2014-08-13 18:06:08.355851 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x21af260 -8126> 2014-08-13 18:06:08.355804 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.296 -8125> 2014-08-13 18:06:08.355844 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] needs_recovery osd.47 has 35 missing -8124> 2014-08-13 18:06:08.355857 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355856, event: started, request: pg_query(24.cd epoch 194496) v3 -8123> 2014-08-13 18:06:08.355861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -8122> 2014-08-13 18:06:08.355865 7f5c835ef700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_query query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192967) from replica 6 -8121> 2014-08-13 18:06:08.355860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.296 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours -8120> 2014-08-13 18:06:08.355901 7f5c7bde3700 10 log is not dirty -8119> 2014-08-13 18:06:08.355903 7f5c835ef700 5 -- op tracker -- , seq: 546, time: 2014-08-13 18:06:08.355903, event: done, request: pg_query(24.cd epoch 194496) v3 -8118> 2014-08-13 18:06:08.355924 7f5c835ef700 10 osd.4 194496 do_waiters -- start -8117> 2014-08-13 18:06:08.355928 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -8116> 2014-08-13 18:06:08.355910 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 15 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/193264) -8115> 2014-08-13 18:06:08.355959 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] log request from 15 -8114> 2014-08-13 18:06:08.356026 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194487'0 -8113> 2014-08-13 18:06:08.356059 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] sending log((194422'11806,194422'11806], crt=194422'11806) missing(56) -8112> 2014-08-13 18:06:08.356082 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4854620 don't know epoch, doing nothing -8111> 2014-08-13 18:06:08.356067 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3011 -8110> 2014-08-13 18:06:08.356086 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6812/53408 -- pg_log(24.20e epoch 194496 log log((194422'11806,194422'11806], crt=194422'11806) query_epoch 194496) v4 -- ?+0 0x4167200 con 0x4854620 -8109> 2014-08-13 18:06:08.356089 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] hit_set_clear -8108> 2014-08-13 18:06:08.356112 7f5c7bde3700 10 log is not dirty -8107> 2014-08-13 18:06:08.356104 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] agent_stop -8106> 2014-08-13 18:06:08.356120 7f5c7c7e4700 10 log is not dirty -8105> 2014-08-13 18:06:08.356297 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -8104> 2014-08-13 18:06:08.356293 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8103> 2014-08-13 18:06:08.356321 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] check_local -8102> 2014-08-13 18:06:08.356330 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] requeue_ops -8101> 2014-08-13 18:06:08.356337 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] requeue_ops -8100> 2014-08-13 18:06:08.356354 7f5c7bde3700 10 log is not dirty -8099> 2014-08-13 18:06:08.356352 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] needs_recovery osd.301 has 38 missing -8098> 2014-08-13 18:06:08.356363 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8097> 2014-08-13 18:06:08.356368 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -8096> 2014-08-13 18:06:08.356384 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] requeue_ops -8095> 2014-08-13 18:06:08.356402 7f5c7bde3700 10 log is not dirty -8094> 2014-08-13 18:06:08.356409 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 31 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194492/194372 194495/194496/192978) -8093> 2014-08-13 18:06:08.356433 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] log request from 31 -8092> 2014-08-13 18:06:08.356471 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending info+missing+log since 194487'0 -8091> 2014-08-13 18:06:08.356487 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] sending log((194422'18567,194422'18567], crt=194422'18567) missing(46) -8090> 2014-08-13 18:06:08.356506 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4857bc0 already has epoch 194496 -8089> 2014-08-13 18:06:08.356510 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6831/41157 -- pg_log(24.1e4 epoch 194496 log log((194422'18567,194422'18567], crt=194422'18567) query_epoch 194496) v4 -- ?+0 0x4166c00 con 0x4857bc0 -8088> 2014-08-13 18:06:08.356537 7f5c7bde3700 10 log is not dirty -8087> 2014-08-13 18:06:08.356587 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2835 -8086> 2014-08-13 18:06:08.356603 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] hit_set_clear -8085> 2014-08-13 18:06:08.356617 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] agent_stop -8084> 2014-08-13 18:06:08.356633 7f5c7c7e4700 10 log is not dirty -8083> 2014-08-13 18:06:08.356693 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 332 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -8082> 2014-08-13 18:06:08.356700 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 309 info: 22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) -8081> 2014-08-13 18:06:08.356723 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.309 activated and committed -8080> 2014-08-13 18:06:08.356721 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.332 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8079> 2014-08-13 18:06:08.356743 7f5c7c7e4700 10 log is not dirty -8078> 2014-08-13 18:06:08.356746 7f5c7bde3700 10 log is not dirty -8077> 2014-08-13 18:06:08.356754 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -8076> 2014-08-13 18:06:08.356750 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -8075> 2014-08-13 18:06:08.356770 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] requeue_ops -8074> 2014-08-13 18:06:08.356786 7f5c7c7e4700 10 log is not dirty -8073> 2014-08-13 18:06:08.356794 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 285 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192975) -8072> 2014-08-13 18:06:08.356801 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 344 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) -8071> 2014-08-13 18:06:08.356813 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] log request from 285 -8070> 2014-08-13 18:06:08.356826 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.344 -8069> 2014-08-13 18:06:08.356845 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.344 info 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours -8068> 2014-08-13 18:06:08.356882 7f5c7bde3700 10 log is not dirty -8067> 2014-08-13 18:06:08.356867 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending info+missing+log since 194487'0 -8066> 2014-08-13 18:06:08.356885 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] sending log((194422'17642,194422'17642], crt=194422'17642) missing(50) -8065> 2014-08-13 18:06:08.356905 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing -8064> 2014-08-13 18:06:08.356891 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194492/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194489 194495/194495/192949) -8063> 2014-08-13 18:06:08.356908 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6822/26295 -- pg_log(24.1e6 epoch 194496 log log((194422'17642,194422'17642], crt=194422'17642) query_epoch 194496) v4 -- ?+0 0x4165400 con 0x445e5c0 -8062> 2014-08-13 18:06:08.356934 7f5c7c7e4700 10 log is not dirty -8061> 2014-08-13 18:06:08.356948 7f5c7bde3700 10 log is not dirty -8060> 2014-08-13 18:06:08.357110 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 289 info: 22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) -8059> 2014-08-13 18:06:08.357134 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.289 activated and committed -8058> 2014-08-13 18:06:08.357152 7f5c7c7e4700 10 log is not dirty -8057> 2014-08-13 18:06:08.357137 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 338 info: 22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) -8056> 2014-08-13 18:06:08.357159 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -8055> 2014-08-13 18:06:08.357172 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.338 activated and committed -8054> 2014-08-13 18:06:08.357193 7f5c7bde3700 10 log is not dirty -8053> 2014-08-13 18:06:08.357177 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] check_local -8052> 2014-08-13 18:06:08.357200 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] requeue_ops -8051> 2014-08-13 18:06:08.357200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 331 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -8050> 2014-08-13 18:06:08.357218 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.58 has 23 missing -8049> 2014-08-13 18:06:08.357224 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -8048> 2014-08-13 18:06:08.357235 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] needs_recovery osd.343 has 23 missing -8047> 2014-08-13 18:06:08.357251 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -8046> 2014-08-13 18:06:08.357257 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -8045> 2014-08-13 18:06:08.357279 7f5c7bde3700 10 log is not dirty -8044> 2014-08-13 18:06:08.357286 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 320 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -8043> 2014-08-13 18:06:08.357306 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.320 -8042> 2014-08-13 18:06:08.357324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.320 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours -8041> 2014-08-13 18:06:08.357359 7f5c7bde3700 10 log is not dirty -8040> 2014-08-13 18:06:08.357366 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194492/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/192978) -8039> 2014-08-13 18:06:08.357391 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2686 -8038> 2014-08-13 18:06:08.357407 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] hit_set_clear -8037> 2014-08-13 18:06:08.357422 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] agent_stop -8036> 2014-08-13 18:06:08.357437 7f5c7c7e4700 10 log is not dirty -8035> 2014-08-13 18:06:08.357438 7f5c7bde3700 10 log is not dirty -8034> 2014-08-13 18:06:08.357446 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 343 info: 22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) -8033> 2014-08-13 18:06:08.357465 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] state: peer osd.343 activated and committed -8032> 2014-08-13 18:06:08.357470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194492/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192958) -8031> 2014-08-13 18:06:08.357481 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194492/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8030> 2014-08-13 18:06:08.357495 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8029> 2014-08-13 18:06:08.357518 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing -8028> 2014-08-13 18:06:08.357526 7f5c7bde3700 10 log is not dirty -8027> 2014-08-13 18:06:08.357521 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194496:22.3d5) v4 -- ?+0 0x5a68540 -8026> 2014-08-13 18:06:08.357553 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -8025> 2014-08-13 18:06:08.357557 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194496:22.3d5) v4 -- ?+0 0x5a68c40 -8024> 2014-08-13 18:06:08.357579 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2656 -8023> 2014-08-13 18:06:08.357603 7f5c7c7e4700 10 log is not dirty -8022> 2014-08-13 18:06:08.357625 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 396 info: 24.299( v 194422'15411 (192834'13573,194422'15411] local-les=194496 n=45 ec=192688 les/c 194496/194463 194495/194495/193205) -8021> 2014-08-13 18:06:08.357705 7f5c7bde3700 10 log is not dirty -8020> 2014-08-13 18:06:08.357693 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194492/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194489 194495/194495/192978) -8019> 2014-08-13 18:06:08.357776 7f5c7c7e4700 10 log is not dirty -8018> 2014-08-13 18:06:08.357738 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) -8017> 2014-08-13 18:06:08.357788 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] state: peer osd.376 activated and committed -8016> 2014-08-13 18:06:08.357806 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194493/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -8015> 2014-08-13 18:06:08.357801 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194492/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192972) -8014> 2014-08-13 18:06:08.357822 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] share_pg_info -8013> 2014-08-13 18:06:08.357850 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b1b5a0 don't know epoch, doing nothing -8012> 2014-08-13 18:06:08.357854 7f5c7c7e4700 10 log is not dirty -8011> 2014-08-13 18:06:08.357855 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.338 192.168.1.106:6814/25893 -- pg_info(1 pgs e194496:22.3d9) v4 -- ?+0 0x5a69500 -8010> 2014-08-13 18:06:08.357910 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4b2bc80 don't know epoch, doing nothing -8009> 2014-08-13 18:06:08.357914 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.376 192.168.1.108:6838/6893 -- pg_info(1 pgs e194496:22.3d9) v4 -- ?+0 0x5a696c0 -8008> 2014-08-13 18:06:08.357936 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2598 -8007> 2014-08-13 18:06:08.357964 7f5c7bde3700 10 log is not dirty -8006> 2014-08-13 18:06:08.357994 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194492/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194489 194495/194495/192971) -8005> 2014-08-13 18:06:08.358050 7f5c7c7e4700 10 log is not dirty -8004> 2014-08-13 18:06:08.358045 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 379 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -8003> 2014-08-13 18:06:08.358074 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.379 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -8002> 2014-08-13 18:06:08.358098 7f5c7bde3700 10 log is not dirty -8001> 2014-08-13 18:06:08.358069 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194493/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 376 info: 22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) -8000> 2014-08-13 18:06:08.358106 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 307 notify: (query_epoch:194496, epoch_sent:194496, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -7999> 2014-08-13 18:06:08.358136 7f5c7c7e4700 10 log is not dirty -7998> 2014-08-13 18:06:08.358128 7f5c7bde3700 7 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle_pg_notify from osd.307 -7997> 2014-08-13 18:06:08.358155 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194489 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 323 info: 22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) -7996> 2014-08-13 18:06:08.358147 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] got dup osd.307 info 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496), identical to ours -7995> 2014-08-13 18:06:08.358192 7f5c7bde3700 10 log is not dirty -7994> 2014-08-13 18:06:08.358211 7f5c7c7e4700 10 log is not dirty -7993> 2014-08-13 18:06:08.358200 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194492/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192981) -7992> 2014-08-13 18:06:08.358252 7f5c7bde3700 10 log is not dirty -7991> 2014-08-13 18:06:08.358236 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 287 notify: (query_epoch:194496, epoch_sent:194496, info:22.481( v 194215'519 (0'0,194215'519] local-les=194489 n=32 ec=183096 les/c 194489/194304 194495/194495/194495)) -7990> 2014-08-13 18:06:08.358261 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 287, already have info from that osd, ignoring -7989> 2014-08-13 18:06:08.358280 7f5c7c7e4700 10 log is not dirty -7988> 2014-08-13 18:06:08.358273 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 337 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -7987> 2014-08-13 18:06:08.358287 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194489 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 292 info: 22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) -7986> 2014-08-13 18:06:08.358297 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got dup osd.337 info 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495), identical to ours -7985> 2014-08-13 18:06:08.358323 7f5c7bde3700 10 log is not dirty -7984> 2014-08-13 18:06:08.358336 7f5c7c7e4700 10 log is not dirty -7983> 2014-08-13 18:06:08.358329 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 347 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) -7982> 2014-08-13 18:06:08.358350 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.347 activated and committed -7981> 2014-08-13 18:06:08.358368 7f5c7bde3700 10 log is not dirty -7980> 2014-08-13 18:06:08.358356 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7979> 2014-08-13 18:06:08.358379 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] check_local -7978> 2014-08-13 18:06:08.358374 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194489 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 328 info: 22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) -7977> 2014-08-13 18:06:08.358395 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] requeue_ops -7976> 2014-08-13 18:06:08.358427 7f5c7bde3700 10 log is not dirty -7975> 2014-08-13 18:06:08.358416 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.31 has 29 missing -7974> 2014-08-13 18:06:08.358433 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] needs_recovery osd.374 has 29 missing -7973> 2014-08-13 18:06:08.358448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7972> 2014-08-13 18:06:08.358446 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 373 info: 22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) -7971> 2014-08-13 18:06:08.358494 7f5c7bde3700 10 log is not dirty -7970> 2014-08-13 18:06:08.358510 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7969> 2014-08-13 18:06:08.358531 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] check_local -7968> 2014-08-13 18:06:08.358546 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] requeue_ops -7967> 2014-08-13 18:06:08.358561 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.344 has 23 missing -7966> 2014-08-13 18:06:08.358577 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] needs_recovery osd.373 has 23 missing -7965> 2014-08-13 18:06:08.358592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7964> 2014-08-13 18:06:08.358624 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2876 -7963> 2014-08-13 18:06:08.358641 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] hit_set_clear -7962> 2014-08-13 18:06:08.358657 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] agent_stop -7961> 2014-08-13 18:06:08.358673 7f5c7c7e4700 10 log is not dirty -7960> 2014-08-13 18:06:08.358680 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194489 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 51 info: 22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) -7959> 2014-08-13 18:06:08.358725 7f5c7c7e4700 10 log is not dirty -7958> 2014-08-13 18:06:08.358736 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2618 -7957> 2014-08-13 18:06:08.358744 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery -7956> 2014-08-13 18:06:08.358753 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] hit_set_clear -7955> 2014-08-13 18:06:08.358769 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] agent_stop -7954> 2014-08-13 18:06:08.358766 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.025954 3 0.000244 -7953> 2014-08-13 18:06:08.358786 7f5c7bde3700 10 log is not dirty -7952> 2014-08-13 18:06:08.358788 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7951> 2014-08-13 18:06:08.358793 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194489 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) -7950> 2014-08-13 18:06:08.358809 7f5c7c7e4700 10 log is not dirty -7949> 2014-08-13 18:06:08.358817 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7948> 2014-08-13 18:06:08.358839 7f5c7bde3700 10 log is not dirty -7947> 2014-08-13 18:06:08.358841 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] requeue_ops -7946> 2014-08-13 18:06:08.358861 7f5c7c7e4700 10 log is not dirty -7945> 2014-08-13 18:06:08.358869 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 7 notify: (query_epoch:194496, epoch_sent:194496, info:22.84( v 194215'497 lc 0'0 (0'0,194215'497] local-les=194492 n=29 ec=183096 les/c 194492/194489 194495/194495/194495)) -7944> 2014-08-13 18:06:08.358866 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestBackfillPrio: priority 1 -7943> 2014-08-13 18:06:08.358890 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring -7942> 2014-08-13 18:06:08.358907 7f5c7c7e4700 10 log is not dirty -7941> 2014-08-13 18:06:08.358902 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] exit Started/ReplicaActive/RepNotRecovering 0.030137 3 0.000235 -7940> 2014-08-13 18:06:08.358914 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7939> 2014-08-13 18:06:08.358928 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] enter Started/ReplicaActive/RepWaitBackfillReserved -7938> 2014-08-13 18:06:08.358950 7f5c7bde3700 10 log is not dirty -7937> 2014-08-13 18:06:08.358937 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops -7936> 2014-08-13 18:06:08.358957 7f5c7c7e4700 10 log is not dirty -7935> 2014-08-13 18:06:08.358957 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7934> 2014-08-13 18:06:08.358967 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 356 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192969) -7933> 2014-08-13 18:06:08.358982 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] requeue_ops -7932> 2014-08-13 18:06:08.359004 7f5c7bde3700 10 log is not dirty -7931> 2014-08-13 18:06:08.358989 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] log request from 356 -7930> 2014-08-13 18:06:08.359013 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -7929> 2014-08-13 18:06:08.359033 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending info+missing+log since 194487'0 -7928> 2014-08-13 18:06:08.359052 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] sending log((194422'18845,194422'18845], crt=194422'18845) missing(56) -7927> 2014-08-13 18:06:08.359074 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -7926> 2014-08-13 18:06:08.359060 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194489 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 345 info: 22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) -7925> 2014-08-13 18:06:08.359078 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6832/5235 -- pg_log(24.3b7 epoch 194496 log log((194422'18845,194422'18845], crt=194422'18845) query_epoch 194496) v4 -- ?+0 0x4164e00 con 0x3c527e0 -7924> 2014-08-13 18:06:08.359108 7f5c7c7e4700 10 log is not dirty -7923> 2014-08-13 18:06:08.359111 7f5c7bde3700 10 log is not dirty -7922> 2014-08-13 18:06:08.359281 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7921> 2014-08-13 18:06:08.359306 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] check_local -7920> 2014-08-13 18:06:08.359322 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] requeue_ops -7919> 2014-08-13 18:06:08.359314 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7918> 2014-08-13 18:06:08.359337 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.356 has 35 missing -7917> 2014-08-13 18:06:08.359355 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] needs_recovery osd.357 has 35 missing -7916> 2014-08-13 18:06:08.359357 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] requeue_ops -7915> 2014-08-13 18:06:08.359383 7f5c7bde3700 10 log is not dirty -7914> 2014-08-13 18:06:08.359371 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7913> 2014-08-13 18:06:08.359392 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 7 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 lc 0'0 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) -7912> 2014-08-13 18:06:08.359414 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 7, already have info from that osd, ignoring -7911> 2014-08-13 18:06:08.359433 7f5c7bde3700 10 log is not dirty -7910> 2014-08-13 18:06:08.359447 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 348 notify: (query_epoch:194496, epoch_sent:194496, info:24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -7909> 2014-08-13 18:06:08.359470 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] got osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7908> 2014-08-13 18:06:08.359502 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] update_heartbeat_peers 4,331,332,337,348,379 unchanged -7907> 2014-08-13 18:06:08.359527 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetInfo 0.050544 9 0.000838 -7906> 2014-08-13 18:06:08.359550 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetLog -7905> 2014-08-13 18:06:08.359580 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2663 -7904> 2014-08-13 18:06:08.359576 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.4 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -7903> 2014-08-13 18:06:08.359597 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] hit_set_clear -7902> 2014-08-13 18:06:08.359599 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7901> 2014-08-13 18:06:08.359614 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] agent_stop -7900> 2014-08-13 18:06:08.359631 7f5c7c7e4700 10 log is not dirty -7899> 2014-08-13 18:06:08.359619 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -7898> 2014-08-13 18:06:08.359638 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 287 info: 22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) -7897> 2014-08-13 18:06:08.359641 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -7896> 2014-08-13 18:06:08.359659 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] state: peer osd.287 activated and committed -7895> 2014-08-13 18:06:08.359663 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -7894> 2014-08-13 18:06:08.359677 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194492/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7893> 2014-08-13 18:06:08.359682 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting osd.379 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -7892> 2014-08-13 18:06:08.359693 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7891> 2014-08-13 18:06:08.359718 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x49a7bc0 don't know epoch, doing nothing -7890> 2014-08-13 18:06:08.359722 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.287 192.168.1.109:6814/33435 -- pg_info(1 pgs e194496:22.481) v4 -- ?+0 0x5a68000 -7889> 2014-08-13 18:06:08.359751 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4c59ce0 don't know epoch, doing nothing -7888> 2014-08-13 18:06:08.359743 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] calc_acting newest update on osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) up_primary: 4) selected as primary calc_acting primary is osd.4 with 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) shard 348 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 331 (up) backfill 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) shard 332 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) shard 337 (stray) accepted 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -7887> 2014-08-13 18:06:08.359755 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.309 192.168.1.101:6814/23703 -- pg_info(1 pgs e194496:22.481) v4 -- ?+0 0x4bf40c0 -7886> 2014-08-13 18:06:08.359765 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] actingbackfill is 4,331,332,337,348 -7885> 2014-08-13 18:06:08.359775 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2686 -7884> 2014-08-13 18:06:08.359784 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] choose_acting want [4,332,337] (== acting) backfill_targets 331,348 -7883> 2014-08-13 18:06:08.359803 7f5c7c7e4700 10 log is not dirty -7882> 2014-08-13 18:06:08.359808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: leaving GetLog -7881> 2014-08-13 18:06:08.359831 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetLog 0.000281 0 0.000000 -7880> 2014-08-13 18:06:08.359854 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/GetMissing -7879> 2014-08-13 18:06:08.359861 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7878> 2014-08-13 18:06:08.359873 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.332 has no missing, identical log -7877> 2014-08-13 18:06:08.359886 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] check_local -7876> 2014-08-13 18:06:08.359896 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: osd.337 has no missing, identical log -7875> 2014-08-13 18:06:08.359904 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] requeue_ops -7874> 2014-08-13 18:06:08.359917 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: still need up_thru update before going active -7873> 2014-08-13 18:06:08.359920 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] needs_recovery osd.285 has 29 missing -7872> 2014-08-13 18:06:08.359937 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7871> 2014-08-13 18:06:08.359939 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/GetMissing 0.000086 0 0.000000 -7870> 2014-08-13 18:06:08.359963 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] enter Started/Primary/Peering/WaitUpThru -7869> 2014-08-13 18:06:08.359982 7f5c7bde3700 10 log is not dirty -7868> 2014-08-13 18:06:08.360013 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7867> 2014-08-13 18:06:08.360039 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] requeue_ops -7866> 2014-08-13 18:06:08.360059 7f5c7bde3700 10 log is not dirty -7865> 2014-08-13 18:06:08.360068 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7864> 2014-08-13 18:06:08.360088 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] check_local -7863> 2014-08-13 18:06:08.360103 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] requeue_ops -7862> 2014-08-13 18:06:08.360110 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2865 -7861> 2014-08-13 18:06:08.360118 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.44 has 24 missing -7860> 2014-08-13 18:06:08.360128 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] hit_set_clear -7859> 2014-08-13 18:06:08.360136 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] needs_recovery osd.362 has 24 missing -7858> 2014-08-13 18:06:08.360145 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] agent_stop -7857> 2014-08-13 18:06:08.360163 7f5c7c7e4700 10 log is not dirty -7856> 2014-08-13 18:06:08.360154 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7855> 2014-08-13 18:06:08.360136 7f5c88628700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] flushed -7854> 2014-08-13 18:06:08.360170 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 341 info: 22.48e( v 194215'543 lc 0'0 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) -7853> 2014-08-13 18:06:08.360191 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] state: peer osd.341 activated and committed -7852> 2014-08-13 18:06:08.360215 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194493/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7851> 2014-08-13 18:06:08.360230 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7850> 2014-08-13 18:06:08.360251 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 -7849> 2014-08-13 18:06:08.360255 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194496:22.48e) v4 -- ?+0 0x4bf4ec0 -7848> 2014-08-13 18:06:08.360284 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab4620 don't know epoch, doing nothing -7847> 2014-08-13 18:06:08.360288 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.341 192.168.1.106:6818/9142 -- pg_info(1 pgs e194496:22.48e) v4 -- ?+0 0x4bf16c0 -7846> 2014-08-13 18:06:08.360307 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2692 -7845> 2014-08-13 18:06:08.360333 7f5c7c7e4700 10 log is not dirty -7844> 2014-08-13 18:06:08.360318 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2688 -7843> 2014-08-13 18:06:08.360338 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] hit_set_clear -7842> 2014-08-13 18:06:08.360353 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] agent_stop -7841> 2014-08-13 18:06:08.360369 7f5c7bde3700 10 log is not dirty -7840> 2014-08-13 18:06:08.360405 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 318 info: 22.3e4( v 194215'518 lc 0'0 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) -7839> 2014-08-13 18:06:08.360414 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7838> 2014-08-13 18:06:08.360429 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] state: peer osd.318 activated and committed -7837> 2014-08-13 18:06:08.360439 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] check_local -7836> 2014-08-13 18:06:08.360448 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7835> 2014-08-13 18:06:08.360456 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] requeue_ops -7834> 2014-08-13 18:06:08.360464 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7833> 2014-08-13 18:06:08.360472 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.61 has 28 missing -7832> 2014-08-13 18:06:08.360486 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4494360 don't know epoch, doing nothing -7831> 2014-08-13 18:06:08.360490 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.313 192.168.1.109:6820/35445 -- pg_info(1 pgs e194496:22.3e4) v4 -- ?+0 0x4bf1500 -7830> 2014-08-13 18:06:08.360489 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] needs_recovery osd.349 has 28 missing -7829> 2014-08-13 18:06:08.360514 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3988dc0 don't know epoch, doing nothing -7828> 2014-08-13 18:06:08.360504 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7827> 2014-08-13 18:06:08.360519 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.318 192.168.1.110:6814/43857 -- pg_info(1 pgs e194496:22.3e4) v4 -- ?+0 0x4bf1c00 -7826> 2014-08-13 18:06:08.360536 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2709 -7825> 2014-08-13 18:06:08.360559 7f5c7c7e4700 10 log is not dirty -7824> 2014-08-13 18:06:08.360623 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 346 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194493 194495/194496/192955) -7823> 2014-08-13 18:06:08.360646 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] log request from 346 -7822> 2014-08-13 18:06:08.360670 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3025 -7821> 2014-08-13 18:06:08.360682 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending info+missing+log since 194487'0 -7820> 2014-08-13 18:06:08.360687 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] hit_set_clear -7819> 2014-08-13 18:06:08.360702 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] agent_stop -7818> 2014-08-13 18:06:08.360701 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] sending log((194422'15014,194422'15014], crt=194422'15014) missing(66) -7817> 2014-08-13 18:06:08.360720 7f5c7bde3700 10 log is not dirty -7816> 2014-08-13 18:06:08.360724 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4f21600 already has epoch 194496 -7815> 2014-08-13 18:06:08.360728 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7814> 2014-08-13 18:06:08.360727 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6820/2308 -- pg_log(24.e2 epoch 194496 log log((194422'15014,194422'15014], crt=194422'15014) query_epoch 194496) v4 -- ?+0 0x4164800 con 0x4f21600 -7813> 2014-08-13 18:06:08.360760 7f5c7c7e4700 10 log is not dirty -7812> 2014-08-13 18:06:08.360750 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] requeue_ops -7811> 2014-08-13 18:06:08.360769 7f5c7bde3700 10 log is not dirty -7810> 2014-08-13 18:06:08.360936 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7809> 2014-08-13 18:06:08.360946 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RemoteRecoveryReserved -7808> 2014-08-13 18:06:08.360960 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local -7807> 2014-08-13 18:06:08.360980 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af8160 already has epoch 194496 -7806> 2014-08-13 18:06:08.360976 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops -7805> 2014-08-13 18:06:08.360985 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.351 192.168.1.102:6830/16444 -- MRecoveryReserve GRANT pgid: 22.220, query_epoch: 194496 v2 -- ?+0 0x5357e00 -7804> 2014-08-13 18:06:08.360992 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7803> 2014-08-13 18:06:08.361007 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7802> 2014-08-13 18:06:08.361012 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] exit Started/ReplicaActive/RepWaitRecoveryReserved 0.006772 1 0.000066 -7801> 2014-08-13 18:06:08.361022 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7800> 2014-08-13 18:06:08.361038 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] enter Started/ReplicaActive/RepRecovering -7799> 2014-08-13 18:06:08.361057 7f5c7bde3700 10 log is not dirty -7798> 2014-08-13 18:06:08.361054 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2711 -7797> 2014-08-13 18:06:08.361070 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear -7796> 2014-08-13 18:06:08.361085 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop -7795> 2014-08-13 18:06:08.361101 7f5c7c7e4700 10 log is not dirty -7794> 2014-08-13 18:06:08.361086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7793> 2014-08-13 18:06:08.361109 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7792> 2014-08-13 18:06:08.361111 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] requeue_ops -7791> 2014-08-13 18:06:08.361131 7f5c7bde3700 10 log is not dirty -7790> 2014-08-13 18:06:08.361127 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] check_local -7789> 2014-08-13 18:06:08.361142 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] requeue_ops -7788> 2014-08-13 18:06:08.361139 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194493/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 371 info: 22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) -7787> 2014-08-13 18:06:08.361157 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7786> 2014-08-13 18:06:08.361172 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7785> 2014-08-13 18:06:08.361187 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7784> 2014-08-13 18:06:08.361210 7f5c7bde3700 10 log is not dirty -7783> 2014-08-13 18:06:08.361215 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2910 -7782> 2014-08-13 18:06:08.361231 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] hit_set_clear -7781> 2014-08-13 18:06:08.361236 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7780> 2014-08-13 18:06:08.361245 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] agent_stop -7779> 2014-08-13 18:06:08.361261 7f5c7c7e4700 10 log is not dirty -7778> 2014-08-13 18:06:08.361268 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -7777> 2014-08-13 18:06:08.361261 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.128658 9 0.003468 -7776> 2014-08-13 18:06:08.361283 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7775> 2014-08-13 18:06:08.361311 7f5c7bde3700 10 log is not dirty -7774> 2014-08-13 18:06:08.361298 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7773> 2014-08-13 18:06:08.361319 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.107412 9 0.003964 -7772> 2014-08-13 18:06:08.361324 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7771> 2014-08-13 18:06:08.361342 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7770> 2014-08-13 18:06:08.361345 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.152338 9 0.048258 -7769> 2014-08-13 18:06:08.361363 7f5c7c7e4700 10 log is not dirty -7768> 2014-08-13 18:06:08.361365 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7767> 2014-08-13 18:06:08.361384 7f5c7bde3700 10 log is not dirty -7766> 2014-08-13 18:06:08.361370 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7765> 2014-08-13 18:06:08.361389 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.081297 9 0.012971 -7764> 2014-08-13 18:06:08.361390 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 RequestRecovery -7763> 2014-08-13 18:06:08.361409 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7762> 2014-08-13 18:06:08.361410 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] exit Started/ReplicaActive/RepNotRecovering 0.036256 3 0.000171 -7761> 2014-08-13 18:06:08.361428 7f5c7c7e4700 10 log is not dirty -7760> 2014-08-13 18:06:08.361431 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] enter Started/ReplicaActive/RepWaitRecoveryReserved -7759> 2014-08-13 18:06:08.361435 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7758> 2014-08-13 18:06:08.361451 7f5c7bde3700 10 log is not dirty -7757> 2014-08-13 18:06:08.361453 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094119 9 0.013031 -7756> 2014-08-13 18:06:08.361458 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7755> 2014-08-13 18:06:08.361472 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7754> 2014-08-13 18:06:08.361491 7f5c7c7e4700 10 log is not dirty -7753> 2014-08-13 18:06:08.361478 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.121539 9 0.003900 -7752> 2014-08-13 18:06:08.361498 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7751> 2014-08-13 18:06:08.361504 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7750> 2014-08-13 18:06:08.361524 7f5c7bde3700 10 log is not dirty -7749> 2014-08-13 18:06:08.361517 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] check_local -7748> 2014-08-13 18:06:08.361532 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] requeue_ops -7747> 2014-08-13 18:06:08.361530 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MQuery from 6 query_epoch 194496 query: query(log 194487'0 ec=192688 les/c 194493/194489 194495/194496/192967) -7746> 2014-08-13 18:06:08.361549 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] needs_recovery osd.309 has 32 missing -7745> 2014-08-13 18:06:08.361552 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] log request from 6 -7744> 2014-08-13 18:06:08.361565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7743> 2014-08-13 18:06:08.361592 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending info+missing+log since 194487'0 -7742> 2014-08-13 18:06:08.361609 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] sending log((194422'15992,194422'15992], crt=194422'15992) missing(53) -7741> 2014-08-13 18:06:08.361628 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4af98c0 already has epoch 194496 -7740> 2014-08-13 18:06:08.361632 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6809/53141 -- pg_log(24.cd epoch 194496 log log((194422'15992,194422'15992], crt=194422'15992) query_epoch 194496) v4 -- ?+0 0x4166600 con 0x4af98c0 -7739> 2014-08-13 18:06:08.361657 7f5c7bde3700 10 log is not dirty -7738> 2014-08-13 18:06:08.361751 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2687 -7737> 2014-08-13 18:06:08.361768 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] hit_set_clear -7736> 2014-08-13 18:06:08.361782 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] agent_stop -7735> 2014-08-13 18:06:08.361797 7f5c7c7e4700 10 log is not dirty -7734> 2014-08-13 18:06:08.361815 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7733> 2014-08-13 18:06:08.361812 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7732> 2014-08-13 18:06:08.361838 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.139790 9 0.006333 -7731> 2014-08-13 18:06:08.361841 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.197036 9 0.005297 -7730> 2014-08-13 18:06:08.361863 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7729> 2014-08-13 18:06:08.361858 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7728> 2014-08-13 18:06:08.361883 7f5c7bde3700 10 log is not dirty -7727> 2014-08-13 18:06:08.361886 7f5c7c7e4700 10 log is not dirty -7726> 2014-08-13 18:06:08.361890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7725> 2014-08-13 18:06:08.361893 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7724> 2014-08-13 18:06:08.361909 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.202336 9 0.003207 -7723> 2014-08-13 18:06:08.361912 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] check_local -7722> 2014-08-13 18:06:08.361927 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] requeue_ops -7721> 2014-08-13 18:06:08.361928 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7720> 2014-08-13 18:06:08.361948 7f5c7bde3700 10 log is not dirty -7719> 2014-08-13 18:06:08.361946 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] needs_recovery osd.341 has 40 missing -7718> 2014-08-13 18:06:08.361955 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7717> 2014-08-13 18:06:08.361962 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7716> 2014-08-13 18:06:08.361976 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] check_local -7715> 2014-08-13 18:06:08.361992 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] requeue_ops -7714> 2014-08-13 18:06:08.362007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7713> 2014-08-13 18:06:08.362021 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7712> 2014-08-13 18:06:08.362036 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7711> 2014-08-13 18:06:08.362059 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2657 -7710> 2014-08-13 18:06:08.362075 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] hit_set_clear -7709> 2014-08-13 18:06:08.362090 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] agent_stop -7708> 2014-08-13 18:06:08.362105 7f5c7bde3700 10 log is not dirty -7707> 2014-08-13 18:06:08.362111 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7706> 2014-08-13 18:06:08.362129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] check_local -7705> 2014-08-13 18:06:08.362143 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] requeue_ops -7704> 2014-08-13 18:06:08.362158 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] needs_recovery osd.376 has 28 missing -7703> 2014-08-13 18:06:08.362173 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7702> 2014-08-13 18:06:08.362188 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2693 -7701> 2014-08-13 18:06:08.362213 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] hit_set_clear -7700> 2014-08-13 18:06:08.362227 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] agent_stop -7699> 2014-08-13 18:06:08.362243 7f5c7c7e4700 10 log is not dirty -7698> 2014-08-13 18:06:08.362249 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7697> 2014-08-13 18:06:08.362267 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] check_local -7696> 2014-08-13 18:06:08.362280 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] requeue_ops -7695> 2014-08-13 18:06:08.362294 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.313 has 32 missing -7694> 2014-08-13 18:06:08.362309 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery osd.318 has 32 missing -7693> 2014-08-13 18:06:08.362323 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7692> 2014-08-13 18:06:08.362348 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2599 -7691> 2014-08-13 18:06:08.362364 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] hit_set_clear -7690> 2014-08-13 18:06:08.362378 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] agent_stop -7689> 2014-08-13 18:06:08.362394 7f5c7bde3700 10 log is not dirty -7688> 2014-08-13 18:06:08.362400 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -7687> 2014-08-13 18:06:08.362430 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 FlushedEvt -7686> 2014-08-13 18:06:08.362453 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] requeue_ops -7685> 2014-08-13 18:06:08.362482 7f5c7bde3700 10 log is not dirty -7684> 2014-08-13 18:06:08.362491 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7683> 2014-08-13 18:06:08.362501 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2710 -7682> 2014-08-13 18:06:08.362510 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.069305 9 0.011024 -7681> 2014-08-13 18:06:08.362519 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] hit_set_clear -7680> 2014-08-13 18:06:08.362531 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7679> 2014-08-13 18:06:08.362534 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] agent_stop -7678> 2014-08-13 18:06:08.362551 7f5c7c7e4700 10 log is not dirty -7677> 2014-08-13 18:06:08.362551 7f5c7bde3700 10 log is not dirty -7676> 2014-08-13 18:06:08.362565 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7675> 2014-08-13 18:06:08.362573 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7674> 2014-08-13 18:06:08.362585 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.097564 9 0.000853 -7673> 2014-08-13 18:06:08.362595 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.142278 9 0.004108 -7672> 2014-08-13 18:06:08.362607 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7671> 2014-08-13 18:06:08.362614 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7670> 2014-08-13 18:06:08.362633 7f5c7bde3700 10 log is not dirty -7669> 2014-08-13 18:06:08.362631 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7668> 2014-08-13 18:06:08.362640 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 LocalRecoveryReserved -7667> 2014-08-13 18:06:08.362655 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000047 0 0.000000 -7666> 2014-08-13 18:06:08.362660 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.001378 1 0.000051 -7665> 2014-08-13 18:06:08.362675 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7664> 2014-08-13 18:06:08.362682 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] enter Started/Primary/Active/WaitRemoteRecoveryReserved -7663> 2014-08-13 18:06:08.362692 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] finish_recovery -7662> 2014-08-13 18:06:08.362706 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6828/6040 -- MRecoveryReserve REQUEST pgid: 22.113, query_epoch: 194496 v2 -- ?+0 0x5353100 con 0x4496720 -7661> 2014-08-13 18:06:08.362708 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194489 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7660> 2014-08-13 18:06:08.362728 7f5c7bde3700 10 log is not dirty -7659> 2014-08-13 18:06:08.362728 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194485-194494/3 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,50,289](4) acting [4,50,289](4) maybe_went_rw) -7658> 2014-08-13 18:06:08.362749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194488-194494/2 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [50,289](50) acting [50,289](50) maybe_went_rw) -7657> 2014-08-13 18:06:08.362767 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 pi=194491-194494/1 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [32,50,289](32) acting [32,50,289](32) maybe_went_rw) -7656> 2014-08-13 18:06:08.362785 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7655> 2014-08-13 18:06:08.362807 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing -7654> 2014-08-13 18:06:08.362799 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7653> 2014-08-13 18:06:08.362811 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.50 192.168.1.112:6842/9607 -- pg_info(1 pgs e194496:22.6bd) v4 -- ?+0 0x4bf5cc0 -7652> 2014-08-13 18:06:08.362822 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.052692 9 0.000759 -7651> 2014-08-13 18:06:08.362838 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x445f7a0 already has epoch 194496 -7650> 2014-08-13 18:06:08.362843 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194496:22.6bd) v4 -- ?+0 0x4bf0e00 -7649> 2014-08-13 18:06:08.362842 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7648> 2014-08-13 18:06:08.362861 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7647> 2014-08-13 18:06:08.362860 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2712 -7646> 2014-08-13 18:06:08.362882 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000041 0 0.000000 -7645> 2014-08-13 18:06:08.362902 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7644> 2014-08-13 18:06:08.362924 7f5c7c7e4700 10 log is not dirty -7643> 2014-08-13 18:06:08.362918 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] finish_recovery -7642> 2014-08-13 18:06:08.362932 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194489 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7641> 2014-08-13 18:06:08.362951 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194485-194494/3 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,343,286](4) acting [4,343,286](4) maybe_went_rw) -7640> 2014-08-13 18:06:08.362971 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194488-194494/2 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [343,286](343) acting [343,286](343) maybe_went_rw) -7639> 2014-08-13 18:06:08.362989 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 pi=194491-194494/1 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [10,343,286](10) acting [10,343,286](10) maybe_went_rw) -7638> 2014-08-13 18:06:08.363007 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7637> 2014-08-13 18:06:08.363031 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing -7636> 2014-08-13 18:06:08.363035 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194496:22.3d5) v4 -- ?+0 0x4df0fc0 -7635> 2014-08-13 18:06:08.363026 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7634> 2014-08-13 18:06:08.363060 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x48506e0 don't know epoch, doing nothing -7633> 2014-08-13 18:06:08.363051 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.108392 9 0.000793 -7632> 2014-08-13 18:06:08.363064 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194496:22.3d5) v4 -- ?+0 0x4df4440 -7631> 2014-08-13 18:06:08.363072 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7630> 2014-08-13 18:06:08.363081 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2658 -7629> 2014-08-13 18:06:08.363092 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7628> 2014-08-13 18:06:08.363111 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000039 0 0.000000 -7627> 2014-08-13 18:06:08.363145 7f5c7bde3700 10 log is not dirty -7626> 2014-08-13 18:06:08.363132 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7625> 2014-08-13 18:06:08.363149 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] finish_recovery -7624> 2014-08-13 18:06:08.363163 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194489 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7623> 2014-08-13 18:06:08.363151 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7622> 2014-08-13 18:06:08.363189 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] purge_strays 32 -7621> 2014-08-13 18:06:08.363181 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194485-194494/3 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,28,304](4) acting [4,28,304](4) maybe_went_rw) -7620> 2014-08-13 18:06:08.363207 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.32 -7619> 2014-08-13 18:06:08.363209 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194488-194494/2 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [28,304](28) acting [28,304](28) maybe_went_rw) -7618> 2014-08-13 18:06:08.363228 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 pi=194491-194494/1 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [28,7,304](28) acting [28,7,304](28) maybe_went_rw) -7617> 2014-08-13 18:06:08.363242 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4851a20 don't know epoch, doing nothing -7616> 2014-08-13 18:06:08.363246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7615> 2014-08-13 18:06:08.363249 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.32 192.168.1.115:6812/59915 -- osd pg remove(epoch 194496; pg22.6bd; ) v2 -- ?+0 0x5a69880 -7614> 2014-08-13 18:06:08.363265 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7613> 2014-08-13 18:06:08.363287 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,32,50,289 -> 4,50,289 -7612> 2014-08-13 18:06:08.363292 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.038020 13 0.013009 -7611> 2014-08-13 18:06:08.363308 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7610> 2014-08-13 18:06:08.363316 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2713 -7609> 2014-08-13 18:06:08.363313 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7608> 2014-08-13 18:06:08.363341 7f5c7bde3700 10 log is not dirty -7607> 2014-08-13 18:06:08.363347 7f5c7bde3700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -7606> 2014-08-13 18:06:08.363410 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing -7605> 2014-08-13 18:06:08.363420 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.28 192.168.1.103:6805/37391 -- pg_info(1 pgs e194496:22.84) v4 -- ?+0 0x4bf1dc0 -7604> 2014-08-13 18:06:08.363452 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4ab69e0 already has epoch 194496 -7603> 2014-08-13 18:06:08.363457 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194496:22.84) v4 -- ?+0 0x588ee40 -7602> 2014-08-13 18:06:08.363475 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2911 -7601> 2014-08-13 18:06:08.363538 7f5c7c7e4700 10 log is not dirty -7600> 2014-08-13 18:06:08.363591 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7599> 2014-08-13 18:06:08.363596 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7598> 2014-08-13 18:06:08.363616 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.067189 12 0.003490 -7597> 2014-08-13 18:06:08.363625 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] purge_strays 10 -7596> 2014-08-13 18:06:08.363639 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7595> 2014-08-13 18:06:08.363642 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.10 -7594> 2014-08-13 18:06:08.363659 7f5c7c7e4700 10 log is not dirty -7593> 2014-08-13 18:06:08.363663 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing -7592> 2014-08-13 18:06:08.363667 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.10 192.168.1.115:6815/60573 -- osd pg remove(epoch 194496; pg22.3d5; ) v2 -- ?+0 0x5a68fc0 -7591> 2014-08-13 18:06:08.363703 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,10,286,343 -> 4,286,343 -7590> 2014-08-13 18:06:08.363705 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7589> 2014-08-13 18:06:08.363724 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7588> 2014-08-13 18:06:08.363727 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.063671 9 0.003884 -7587> 2014-08-13 18:06:08.363730 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2659 -7586> 2014-08-13 18:06:08.363747 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7585> 2014-08-13 18:06:08.363766 7f5c7c7e4700 10 log is not dirty -7584> 2014-08-13 18:06:08.363772 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7583> 2014-08-13 18:06:08.363789 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.136725 9 0.003701 -7582> 2014-08-13 18:06:08.363807 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7581> 2014-08-13 18:06:08.363824 7f5c7c7e4700 10 log is not dirty -7580> 2014-08-13 18:06:08.365279 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7579> 2014-08-13 18:06:08.365324 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 -7578> 2014-08-13 18:06:08.365342 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 -7577> 2014-08-13 18:06:08.365362 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing -7576> 2014-08-13 18:06:08.365366 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194496; pg22.84; ) v2 -- ?+0 0x5a69dc0 -7575> 2014-08-13 18:06:08.365415 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,28,304 -> 4,28,304 -7574> 2014-08-13 18:06:08.365441 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7573> 2014-08-13 18:06:08.365448 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:2912 -7572> 2014-08-13 18:06:08.368879 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 8 ==== pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 ==== 4342+0+0 (2092083302 0 0) 0x5351dc0 con 0x4499080 -7571> 2014-08-13 18:06:08.368911 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7570> 2014-08-13 18:06:08.368916 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7569> 2014-08-13 18:06:08.368918 7f5c835ef700 20 osd.4 194496 _dispatch 0x5351dc0 pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7568> 2014-08-13 18:06:08.368926 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368654, event: header_read, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7567> 2014-08-13 18:06:08.368938 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368656, event: throttled, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7566> 2014-08-13 18:06:08.368945 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368824, event: all_read, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7565> 2014-08-13 18:06:08.368952 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368909, event: dispatched, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7564> 2014-08-13 18:06:08.368959 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368959, event: waiting_for_osdmap, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7563> 2014-08-13 18:06:08.368967 7f5c835ef700 7 osd.4 194496 handle_pg_notify from osd.340 -7562> 2014-08-13 18:06:08.368971 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5351dc0 -7561> 2014-08-13 18:06:08.368976 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.368976, event: started, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7560> 2014-08-13 18:06:08.369032 7f5c835ef700 5 -- op tracker -- , seq: 547, time: 2014-08-13 18:06:08.369031, event: done, request: pg_notify(22.1b8(2),24.34(54) epoch 194496) v5 -7559> 2014-08-13 18:06:08.369059 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7558> 2014-08-13 18:06:08.369064 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7557> 2014-08-13 18:06:08.369034 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:22.1b8( v 194215'479 (0'0,194215'479] local-les=194492 n=34 ec=183096 les/c 194492/194489 194495/194495/194495)) -7556> 2014-08-13 18:06:08.369071 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: Active: got notify from 340, already have info from that osd, ignoring -7555> 2014-08-13 18:06:08.369056 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MNotifyRec from 340 notify: (query_epoch:194496, epoch_sent:194496, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -7554> 2014-08-13 18:06:08.369092 7f5c7bde3700 10 log is not dirty -7553> 2014-08-13 18:06:08.369093 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle_pg_notify from osd.340 -7552> 2014-08-13 18:06:08.369112 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] got dup osd.340 info 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496), identical to ours -7551> 2014-08-13 18:06:08.369151 7f5c7c7e4700 10 log is not dirty -7550> 2014-08-13 18:06:08.369160 7f5c7c7e4700 10 osd.4 194496 queue_want_up_thru want 194496 <= queued 194496, currently 194495 -7549> 2014-08-13 18:06:08.374665 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.21 192.168.1.107:6821/54786 3 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4ee0000 con 0x3a689a0 -7548> 2014-08-13 18:06:08.374698 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7547> 2014-08-13 18:06:08.374703 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7546> 2014-08-13 18:06:08.374705 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee0000 osd_map(194496..194496 src has 185139..194496) v3 -7545> 2014-08-13 18:06:08.374717 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -7544> 2014-08-13 18:06:08.374722 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -7543> 2014-08-13 18:06:08.374725 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7542> 2014-08-13 18:06:08.374727 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7541> 2014-08-13 18:06:08.374738 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.21 192.168.1.107:6821/54786 4 ==== pg_info(1 pgs e194496:22.707) v4 ==== 714+0+0 (1703479997 0 0) 0x5a6a140 con 0x3a689a0 -7540> 2014-08-13 18:06:08.374753 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7539> 2014-08-13 18:06:08.374755 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7538> 2014-08-13 18:06:08.374757 7f5c835ef700 20 osd.4 194496 _dispatch 0x5a6a140 pg_info(1 pgs e194496:22.707) v4 -7537> 2014-08-13 18:06:08.374763 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374605, event: header_read, request: pg_info(1 pgs e194496:22.707) v4 -7536> 2014-08-13 18:06:08.374775 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374606, event: throttled, request: pg_info(1 pgs e194496:22.707) v4 -7535> 2014-08-13 18:06:08.374781 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374691, event: all_read, request: pg_info(1 pgs e194496:22.707) v4 -7534> 2014-08-13 18:06:08.374787 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374752, event: dispatched, request: pg_info(1 pgs e194496:22.707) v4 -7533> 2014-08-13 18:06:08.374794 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374794, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.707) v4 -7532> 2014-08-13 18:06:08.374800 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.707) v4 from osd.21 -7531> 2014-08-13 18:06:08.374805 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5a6a140 -7530> 2014-08-13 18:06:08.374811 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374811, event: started, request: pg_info(1 pgs e194496:22.707) v4 -7529> 2014-08-13 18:06:08.374844 7f5c835ef700 5 -- op tracker -- , seq: 548, time: 2014-08-13 18:06:08.374844, event: done, request: pg_info(1 pgs e194496:22.707) v4 -7528> 2014-08-13 18:06:08.374860 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7527> 2014-08-13 18:06:08.374863 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7526> 2014-08-13 18:06:08.374862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 21 info: 22.707( v 194215'483 lc 0'0 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) -7525> 2014-08-13 18:06:08.374897 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] state: peer osd.21 activated and committed -7524> 2014-08-13 18:06:08.374931 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194493/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7523> 2014-08-13 18:06:08.374947 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7522> 2014-08-13 18:06:08.374975 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x3a689a0 don't know epoch, doing nothing -7521> 2014-08-13 18:06:08.374979 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.21 192.168.1.107:6821/54786 -- pg_info(1 pgs e194496:22.707) v4 -- ?+0 0x4df3480 -7520> 2014-08-13 18:06:08.375017 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4fa1fa0 don't know epoch, doing nothing -7519> 2014-08-13 18:06:08.375021 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.310 192.168.1.109:6812/32413 -- pg_info(1 pgs e194496:22.707) v4 -- ?+0 0x4df1f80 -7518> 2014-08-13 18:06:08.375046 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2573 -7517> 2014-08-13 18:06:08.375097 7f5c7bde3700 10 log is not dirty -7516> 2014-08-13 18:06:08.375204 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7515> 2014-08-13 18:06:08.375267 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] check_local -7514> 2014-08-13 18:06:08.375283 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] requeue_ops -7513> 2014-08-13 18:06:08.375304 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.21 has 19 missing -7512> 2014-08-13 18:06:08.375333 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] needs_recovery osd.310 has 19 missing -7511> 2014-08-13 18:06:08.375359 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7510> 2014-08-13 18:06:08.375534 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2574 -7509> 2014-08-13 18:06:08.375554 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] hit_set_clear -7508> 2014-08-13 18:06:08.375569 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] agent_stop -7507> 2014-08-13 18:06:08.375606 7f5c7bde3700 10 log is not dirty -7506> 2014-08-13 18:06:08.375645 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7505> 2014-08-13 18:06:08.375668 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.142309 9 0.002936 -7504> 2014-08-13 18:06:08.375690 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7503> 2014-08-13 18:06:08.375716 7f5c7bde3700 10 log is not dirty -7502> 2014-08-13 18:06:08.403193 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 9 ==== pg_info(1 pgs e194496:2.2a) v4 ==== 706+0+0 (2516131942 0 0) 0x4df2a00 con 0x4499080 -7501> 2014-08-13 18:06:08.403233 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7500> 2014-08-13 18:06:08.403238 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7499> 2014-08-13 18:06:08.403241 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df2a00 pg_info(1 pgs e194496:2.2a) v4 -7498> 2014-08-13 18:06:08.403249 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.402996, event: header_read, request: pg_info(1 pgs e194496:2.2a) v4 -7497> 2014-08-13 18:06:08.403260 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403000, event: throttled, request: pg_info(1 pgs e194496:2.2a) v4 -7496> 2014-08-13 18:06:08.403266 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403098, event: all_read, request: pg_info(1 pgs e194496:2.2a) v4 -7495> 2014-08-13 18:06:08.403272 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403230, event: dispatched, request: pg_info(1 pgs e194496:2.2a) v4 -7494> 2014-08-13 18:06:08.403279 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403278, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:2.2a) v4 -7493> 2014-08-13 18:06:08.403285 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:2.2a) v4 from osd.340 -7492> 2014-08-13 18:06:08.403290 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df2a00 -7491> 2014-08-13 18:06:08.403296 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403295, event: started, request: pg_info(1 pgs e194496:2.2a) v4 -7490> 2014-08-13 18:06:08.403326 7f5c835ef700 5 -- op tracker -- , seq: 549, time: 2014-08-13 18:06:08.403326, event: done, request: pg_info(1 pgs e194496:2.2a) v4 -7489> 2014-08-13 18:06:08.403353 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7488> 2014-08-13 18:06:08.403357 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7487> 2014-08-13 18:06:08.403383 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) -7486> 2014-08-13 18:06:08.403432 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -7485> 2014-08-13 18:06:08.403451 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7484> 2014-08-13 18:06:08.403465 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7483> 2014-08-13 18:06:08.403488 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7482> 2014-08-13 18:06:08.403493 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:2.2a) v4 -- ?+0 0x588bb80 -7481> 2014-08-13 18:06:08.403520 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:176857 -7480> 2014-08-13 18:06:08.403549 7f5c7c7e4700 10 log is not dirty -7479> 2014-08-13 18:06:08.403654 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7478> 2014-08-13 18:06:08.403684 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local -7477> 2014-08-13 18:06:08.403712 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops -7476> 2014-08-13 18:06:08.403727 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7475> 2014-08-13 18:06:08.403742 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7474> 2014-08-13 18:06:08.403755 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7473> 2014-08-13 18:06:08.403780 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:176858 -7472> 2014-08-13 18:06:08.403822 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear -7471> 2014-08-13 18:06:08.403837 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop -7470> 2014-08-13 18:06:08.403853 7f5c7c7e4700 10 log is not dirty -7469> 2014-08-13 18:06:08.403864 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7468> 2014-08-13 18:06:08.403883 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.177187 7 0.003639 -7467> 2014-08-13 18:06:08.403904 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7466> 2014-08-13 18:06:08.403922 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7465> 2014-08-13 18:06:08.403939 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000035 0 0.000000 -7464> 2014-08-13 18:06:08.404006 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7463> 2014-08-13 18:06:08.404022 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery -7462> 2014-08-13 18:06:08.404071 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.17 192.168.100.103:0/37658 2 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.404027) v2 ==== 47+0+0 (706109619 0 0) 0x5a6a840 con 0x549d800 -7461> 2014-08-13 18:06:08.404110 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.404027) v2 -- ?+0 0x5888a80 con 0x549d800 -7460> 2014-08-13 18:06:08.404096 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.17 192.168.100.103:0/37658 2 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.404027) v2 ==== 47+0+0 (706109619 0 0) 0x5a6a4c0 con 0x549fbc0 -7459> 2014-08-13 18:06:08.404097 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7458> 2014-08-13 18:06:08.404140 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 -7457> 2014-08-13 18:06:08.404143 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) -7456> 2014-08-13 18:06:08.404177 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194488-194494/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) -7455> 2014-08-13 18:06:08.404185 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.103:0/37658 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.404027) v2 -- ?+0 0x58888c0 con 0x549fbc0 -7454> 2014-08-13 18:06:08.404194 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194491-194494/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) -7453> 2014-08-13 18:06:08.404223 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4493f40 already has epoch 194496 -7452> 2014-08-13 18:06:08.404217 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7451> 2014-08-13 18:06:08.404240 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7450> 2014-08-13 18:06:08.404244 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:2.2a) v4 -- ?+0 0x588af40 -7449> 2014-08-13 18:06:08.404376 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:176859 -7448> 2014-08-13 18:06:08.404480 7f5c7c7e4700 10 log is not dirty -7447> 2014-08-13 18:06:08.404512 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 10 ==== pg_info(1 pgs e194496:22.1b8) v4 ==== 714+0+0 (639922681 0 0) 0x588bb80 con 0x4499080 -7446> 2014-08-13 18:06:08.404538 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7445> 2014-08-13 18:06:08.404543 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7444> 2014-08-13 18:06:08.404545 7f5c835ef700 20 osd.4 194496 _dispatch 0x588bb80 pg_info(1 pgs e194496:22.1b8) v4 -7443> 2014-08-13 18:06:08.404553 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404329, event: header_read, request: pg_info(1 pgs e194496:22.1b8) v4 -7442> 2014-08-13 18:06:08.404567 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404331, event: throttled, request: pg_info(1 pgs e194496:22.1b8) v4 -7441> 2014-08-13 18:06:08.404573 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404421, event: all_read, request: pg_info(1 pgs e194496:22.1b8) v4 -7440> 2014-08-13 18:06:08.404579 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404537, event: dispatched, request: pg_info(1 pgs e194496:22.1b8) v4 -7439> 2014-08-13 18:06:08.404586 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404586, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.1b8) v4 -7438> 2014-08-13 18:06:08.404593 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.1b8) v4 from osd.340 -7437> 2014-08-13 18:06:08.404598 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x588bb80 -7436> 2014-08-13 18:06:08.404604 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404604, event: started, request: pg_info(1 pgs e194496:22.1b8) v4 -7435> 2014-08-13 18:06:08.404635 7f5c835ef700 5 -- op tracker -- , seq: 550, time: 2014-08-13 18:06:08.404635, event: done, request: pg_info(1 pgs e194496:22.1b8) v4 -7434> 2014-08-13 18:06:08.404650 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7433> 2014-08-13 18:06:08.404653 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7432> 2014-08-13 18:06:08.404652 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) -7431> 2014-08-13 18:06:08.404695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -7430> 2014-08-13 18:06:08.404679 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7429> 2014-08-13 18:06:08.404715 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194492/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7428> 2014-08-13 18:06:08.404715 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 -7427> 2014-08-13 18:06:08.404731 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7426> 2014-08-13 18:06:08.404735 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 -7425> 2014-08-13 18:06:08.404756 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 -7424> 2014-08-13 18:06:08.404760 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194496; pg2.2a; ) v2 -- ?+0 0x5a69c00 -7423> 2014-08-13 18:06:08.404797 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7422> 2014-08-13 18:06:08.404796 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 -7421> 2014-08-13 18:06:08.404804 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:22.1b8) v4 -- ?+0 0x4df0540 -7420> 2014-08-13 18:06:08.404840 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing -7419> 2014-08-13 18:06:08.404847 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194496; pg2.2a; ) v2 -- ?+0 0x5a69a40 -7418> 2014-08-13 18:06:08.404876 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 -7417> 2014-08-13 18:06:08.404896 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7416> 2014-08-13 18:06:08.404903 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:176860 -7415> 2014-08-13 18:06:08.404966 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing -7414> 2014-08-13 18:06:08.404976 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194496:22.1b8) v4 -- ?+0 0x4df6c80 -7413> 2014-08-13 18:06:08.405004 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:3003 -7412> 2014-08-13 18:06:08.405042 7f5c7bde3700 10 log is not dirty -7411> 2014-08-13 18:06:08.405115 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7410> 2014-08-13 18:06:08.405141 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] check_local -7409> 2014-08-13 18:06:08.405158 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] requeue_ops -7408> 2014-08-13 18:06:08.405183 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7407> 2014-08-13 18:06:08.405198 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7406> 2014-08-13 18:06:08.405213 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7405> 2014-08-13 18:06:08.405248 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:3004 -7404> 2014-08-13 18:06:08.405265 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] hit_set_clear -7403> 2014-08-13 18:06:08.405280 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] agent_stop -7402> 2014-08-13 18:06:08.405301 7f5c7bde3700 10 log is not dirty -7401> 2014-08-13 18:06:08.405314 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7400> 2014-08-13 18:06:08.405333 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.094194 9 0.001175 -7399> 2014-08-13 18:06:08.405356 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7398> 2014-08-13 18:06:08.405374 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7397> 2014-08-13 18:06:08.405392 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000037 0 0.000000 -7396> 2014-08-13 18:06:08.405411 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7395> 2014-08-13 18:06:08.405427 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] finish_recovery -7394> 2014-08-13 18:06:08.405441 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194489 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7393> 2014-08-13 18:06:08.405460 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194485-194494/3 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340,347](4) acting [4,340,347](4) maybe_went_rw) -7392> 2014-08-13 18:06:08.405481 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194488-194494/2 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340,347](340) acting [340,347](340) maybe_went_rw) -7391> 2014-08-13 18:06:08.405498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 pi=194491-194494/1 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [7,340,347](7) acting [7,340,347](7) maybe_went_rw) -7390> 2014-08-13 18:06:08.405516 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7389> 2014-08-13 18:06:08.405536 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7388> 2014-08-13 18:06:08.405540 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:22.1b8) v4 -- ?+0 0x4df0700 -7387> 2014-08-13 18:06:08.405569 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4afb020 don't know epoch, doing nothing -7386> 2014-08-13 18:06:08.405573 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.347 192.168.1.106:6800/4640 -- pg_info(1 pgs e194496:22.1b8) v4 -- ?+0 0x4df0000 -7385> 2014-08-13 18:06:08.405589 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:3005 -7384> 2014-08-13 18:06:08.405663 7f5c7bde3700 10 log is not dirty -7383> 2014-08-13 18:06:08.405854 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7382> 2014-08-13 18:06:08.405876 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] purge_strays 7 -7381> 2014-08-13 18:06:08.405892 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.7 -7380> 2014-08-13 18:06:08.405911 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c58f20 don't know epoch, doing nothing -7379> 2014-08-13 18:06:08.405914 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.7 192.168.1.115:6838/62713 -- osd pg remove(epoch 194496; pg22.1b8; ) v2 -- ?+0 0x4dea300 -7378> 2014-08-13 18:06:08.405939 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,7,340,347 -> 4,340,347 -7377> 2014-08-13 18:06:08.405959 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7376> 2014-08-13 18:06:08.405964 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:3006 -7375> 2014-08-13 18:06:08.424775 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.56 192.168.1.111:6826/11446 3 ==== osd_map(194496..194496 src has 185139..194496) v3 ==== 1194+0+0 (2779087241 0 0) 0x4ed4140 con 0x4b17220 -7374> 2014-08-13 18:06:08.424816 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7373> 2014-08-13 18:06:08.424821 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7372> 2014-08-13 18:06:08.424824 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ed4140 osd_map(194496..194496 src has 185139..194496) v3 -7371> 2014-08-13 18:06:08.424835 7f5c835ef700 3 osd.4 194496 handle_osd_map epochs [194496,194496], i have 194496, src has [185139,194496] -7370> 2014-08-13 18:06:08.424840 7f5c835ef700 10 osd.4 194496 no new maps here, dropping -7369> 2014-08-13 18:06:08.424843 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7368> 2014-08-13 18:06:08.424845 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7367> 2014-08-13 18:06:08.424855 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.56 192.168.1.111:6826/11446 4 ==== pg_info(1 pgs e194496:22.775) v4 ==== 714+0+0 (251694511 0 0) 0x5354440 con 0x4b17220 -7366> 2014-08-13 18:06:08.424869 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7365> 2014-08-13 18:06:08.424871 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7364> 2014-08-13 18:06:08.424873 7f5c835ef700 20 osd.4 194496 _dispatch 0x5354440 pg_info(1 pgs e194496:22.775) v4 -7363> 2014-08-13 18:06:08.424880 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424716, event: header_read, request: pg_info(1 pgs e194496:22.775) v4 -7362> 2014-08-13 18:06:08.424893 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424717, event: throttled, request: pg_info(1 pgs e194496:22.775) v4 -7361> 2014-08-13 18:06:08.424900 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424781, event: all_read, request: pg_info(1 pgs e194496:22.775) v4 -7360> 2014-08-13 18:06:08.424905 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424868, event: dispatched, request: pg_info(1 pgs e194496:22.775) v4 -7359> 2014-08-13 18:06:08.424912 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424912, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:22.775) v4 -7358> 2014-08-13 18:06:08.424919 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:22.775) v4 from osd.56 -7357> 2014-08-13 18:06:08.424924 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x5354440 -7356> 2014-08-13 18:06:08.424930 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424930, event: started, request: pg_info(1 pgs e194496:22.775) v4 -7355> 2014-08-13 18:06:08.424961 7f5c835ef700 5 -- op tracker -- , seq: 551, time: 2014-08-13 18:06:08.424961, event: done, request: pg_info(1 pgs e194496:22.775) v4 -7354> 2014-08-13 18:06:08.424980 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7353> 2014-08-13 18:06:08.424984 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7352> 2014-08-13 18:06:08.425021 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 56 info: 22.775( v 194215'513 lc 0'0 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) -7351> 2014-08-13 18:06:08.425065 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] state: peer osd.56 activated and committed -7350> 2014-08-13 18:06:08.425083 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194493/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7349> 2014-08-13 18:06:08.425098 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7348> 2014-08-13 18:06:08.425123 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x4b17220 don't know epoch, doing nothing -7347> 2014-08-13 18:06:08.425128 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.56 192.168.1.111:6826/11446 -- pg_info(1 pgs e194496:22.775) v4 -- ?+0 0x5888c40 -7346> 2014-08-13 18:06:08.425175 7f5c7c7e4700 20 osd.4 194496 _share_map_outgoing 0x50e1600 don't know epoch, doing nothing -7345> 2014-08-13 18:06:08.425180 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.353 192.168.1.108:6836/6227 -- pg_info(1 pgs e194496:22.775) v4 -- ?+0 0x5889880 -7344> 2014-08-13 18:06:08.425200 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:2725 -7343> 2014-08-13 18:06:08.425232 7f5c7c7e4700 10 log is not dirty -7342> 2014-08-13 18:06:08.425381 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7341> 2014-08-13 18:06:08.425428 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] check_local -7340> 2014-08-13 18:06:08.425446 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] requeue_ops -7339> 2014-08-13 18:06:08.425462 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.56 has 32 missing -7338> 2014-08-13 18:06:08.425478 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] needs_recovery osd.353 has 32 missing -7337> 2014-08-13 18:06:08.425493 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] activate not all replicas are up-to-date, queueing recovery -7336> 2014-08-13 18:06:08.425688 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:2726 -7335> 2014-08-13 18:06:08.425708 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] hit_set_clear -7334> 2014-08-13 18:06:08.425723 7f5c7c7e4700 20 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] agent_stop -7333> 2014-08-13 18:06:08.425740 7f5c7c7e4700 10 log is not dirty -7332> 2014-08-13 18:06:08.425753 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 DoRecovery -7331> 2014-08-13 18:06:08.425772 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.132853 9 0.011511 -7330> 2014-08-13 18:06:08.425794 7f5c7c7e4700 5 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/WaitLocalRecoveryReserved -7329> 2014-08-13 18:06:08.425815 7f5c7c7e4700 10 log is not dirty -7328> 2014-08-13 18:06:08.474448 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 11 ==== pg_info(1 pgs e194496:1.2b) v4 ==== 706+0+0 (3872950061 0 0) 0x4df0700 con 0x4499080 -7327> 2014-08-13 18:06:08.474482 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7326> 2014-08-13 18:06:08.474487 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7325> 2014-08-13 18:06:08.474490 7f5c835ef700 20 osd.4 194496 _dispatch 0x4df0700 pg_info(1 pgs e194496:1.2b) v4 -7324> 2014-08-13 18:06:08.474497 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474319, event: header_read, request: pg_info(1 pgs e194496:1.2b) v4 -7323> 2014-08-13 18:06:08.474508 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474322, event: throttled, request: pg_info(1 pgs e194496:1.2b) v4 -7322> 2014-08-13 18:06:08.474514 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474411, event: all_read, request: pg_info(1 pgs e194496:1.2b) v4 -7321> 2014-08-13 18:06:08.474520 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474481, event: dispatched, request: pg_info(1 pgs e194496:1.2b) v4 -7320> 2014-08-13 18:06:08.474532 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474532, event: waiting_for_osdmap, request: pg_info(1 pgs e194496:1.2b) v4 -7319> 2014-08-13 18:06:08.474539 7f5c835ef700 7 osd.4 194496 handle_pg_info pg_info(1 pgs e194496:1.2b) v4 from osd.340 -7318> 2014-08-13 18:06:08.474544 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x4df0700 -7317> 2014-08-13 18:06:08.474550 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474549, event: started, request: pg_info(1 pgs e194496:1.2b) v4 -7316> 2014-08-13 18:06:08.474590 7f5c835ef700 5 -- op tracker -- , seq: 552, time: 2014-08-13 18:06:08.474590, event: done, request: pg_info(1 pgs e194496:1.2b) v4 -7315> 2014-08-13 18:06:08.474605 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7314> 2014-08-13 18:06:08.474609 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7313> 2014-08-13 18:06:08.474633 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 MInfoRec from 340 info: 1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) -7312> 2014-08-13 18:06:08.474677 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: peer osd.340 activated and committed -7311> 2014-08-13 18:06:08.474695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194492/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed -7310> 2014-08-13 18:06:08.474709 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info -7309> 2014-08-13 18:06:08.474733 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7308> 2014-08-13 18:06:08.474737 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:1.2b) v4 -- ?+0 0x588fc40 -7307> 2014-08-13 18:06:08.474768 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 194496:177961 -7306> 2014-08-13 18:06:08.474799 7f5c7bde3700 10 log is not dirty -7305> 2014-08-13 18:06:08.474813 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasActivated -7304> 2014-08-13 18:06:08.474833 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local -7303> 2014-08-13 18:06:08.474847 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] requeue_ops -7302> 2014-08-13 18:06:08.474862 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7301> 2014-08-13 18:06:08.474877 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill -7300> 2014-08-13 18:06:08.474890 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery -7299> 2014-08-13 18:06:08.474915 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 194496:177962 -7298> 2014-08-13 18:06:08.474931 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear -7297> 2014-08-13 18:06:08.474946 7f5c7bde3700 20 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop -7296> 2014-08-13 18:06:08.474960 7f5c7bde3700 10 log is not dirty -7295> 2014-08-13 18:06:08.474970 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 194496 epoch_requested: 194496 AllReplicasRecovered -7294> 2014-08-13 18:06:08.474989 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.234099 7 0.004925 -7293> 2014-08-13 18:06:08.475010 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered -7292> 2014-08-13 18:06:08.475027 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered -7291> 2014-08-13 18:06:08.475045 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000034 0 0.000000 -7290> 2014-08-13 18:06:08.475063 7f5c7bde3700 5 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean -7289> 2014-08-13 18:06:08.475078 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery -7288> 2014-08-13 18:06:08.475092 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194492 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state -7287> 2014-08-13 18:06:08.475109 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194485-194494/3 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194485-194487 up [4,340](4) acting [4,340](4) maybe_went_rw) -7286> 2014-08-13 18:06:08.475129 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194488-194494/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194488-194490 up [340](340) acting [340](340) maybe_went_rw) -7285> 2014-08-13 18:06:08.475145 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 pi=194491-194494/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(194491-194494 up [324,370](324) acting [324,370](324) maybe_went_rw) -7284> 2014-08-13 18:06:08.475184 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info -7283> 2014-08-13 18:06:08.475208 7f5c7bde3700 20 osd.4 194496 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -7282> 2014-08-13 18:06:08.475212 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194496:1.2b) v4 -- ?+0 0x4bf5780 -7281> 2014-08-13 18:06:08.475234 7f5c7bde3700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:177963 -7280> 2014-08-13 18:06:08.475294 7f5c7bde3700 10 log is not dirty -7279> 2014-08-13 18:06:08.475583 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery -7278> 2014-08-13 18:06:08.475627 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 324,370 -7277> 2014-08-13 18:06:08.475646 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.324 -7276> 2014-08-13 18:06:08.475666 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4b2cba0 already has epoch 194496 -7275> 2014-08-13 18:06:08.475670 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.324 192.168.1.104:6814/65209 -- osd pg remove(epoch 194496; pg1.2b; ) v2 -- ?+0 0x4dea4c0 -7274> 2014-08-13 18:06:08.475699 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] sending PGRemove to osd.370 -7273> 2014-08-13 18:06:08.475721 7f5c87c27700 20 osd.4 194496 _share_map_outgoing 0x4c591e0 don't know epoch, doing nothing -7272> 2014-08-13 18:06:08.475725 7f5c87c27700 1 -- 192.168.1.115:6813/37708 --> osd.370 192.168.1.114:6842/56512 -- osd pg remove(epoch 194496; pg1.2b; ) v2 -- ?+0 0x4a440c0 -7271> 2014-08-13 18:06:08.475743 7f5c87c27700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] update_heartbeat_peers 4,324,340,370 -> 4,340 -7270> 2014-08-13 18:06:08.475769 7f5c87c27700 20 osd.4 194496 need_heartbeat_peer_update -7269> 2014-08-13 18:06:08.475776 7f5c87c27700 15 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194496:177964 -7268> 2014-08-13 18:06:08.659814 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.659613) v2 ==== 47+0+0 (1370925745 0 0) 0x4a45cc0 con 0x4c5dc20 -7267> 2014-08-13 18:06:08.659828 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.15 192.168.1.107:0/53408 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.659613) v2 ==== 47+0+0 (1370925745 0 0) 0x4a42d80 con 0x4c5d540 -7266> 2014-08-13 18:06:08.659867 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.659613) v2 -- ?+0 0x5a6a840 con 0x4c5dc20 -7265> 2014-08-13 18:06:08.659890 7f5c817ec700 10 osd.4 194496 note_peer_epoch osd.15 now has 194496 -7264> 2014-08-13 18:06:08.659901 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4854620 already has epoch 194496 -7263> 2014-08-13 18:06:08.659925 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.659613) v2 -- ?+0 0x5a6a4c0 con 0x4c5d540 -7262> 2014-08-13 18:06:08.659963 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4854620 already has epoch 194496 -7261> 2014-08-13 18:06:08.807751 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.343 192.168.1.104:0/61960 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.808279) v2 ==== 47+0+0 (2728568492 0 0) 0x4a45e80 con 0x4fa3b20 -7260> 2014-08-13 18:06:08.807803 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.808279) v2 -- ?+0 0x4a42d80 con 0x4fa3b20 -7259> 2014-08-13 18:06:08.807831 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.343 now has 194496 -7258> 2014-08-13 18:06:08.807815 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.343 192.168.1.104:0/61960 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.808279) v2 ==== 47+0+0 (2728568492 0 0) 0x4a43480 con 0x4fa2ec0 -7257> 2014-08-13 18:06:08.807844 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x48506e0 already has epoch 194496 -7256> 2014-08-13 18:06:08.807854 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.808279) v2 -- ?+0 0x4a45cc0 con 0x4fa2ec0 -7255> 2014-08-13 18:06:08.807882 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x48506e0 already has epoch 194496 -7254> 2014-08-13 18:06:08.831764 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.371 192.168.100.102:0/16753 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.831672) v2 ==== 47+0+0 (1478516253 0 0) 0x4def1c0 con 0x4c5d800 -7253> 2014-08-13 18:06:08.831817 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.831672) v2 -- ?+0 0x4a45e80 con 0x4c5d800 -7252> 2014-08-13 18:06:08.831842 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.371 now has 194496 -7251> 2014-08-13 18:06:08.831810 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.371 192.168.100.102:0/16753 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.831672) v2 ==== 47+0+0 (1478516253 0 0) 0x4dea140 con 0x4c5ce60 -7250> 2014-08-13 18:06:08.831852 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4851e40 already has epoch 194496 -7249> 2014-08-13 18:06:08.831860 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.831672) v2 -- ?+0 0x4a43480 con 0x4c5ce60 -7248> 2014-08-13 18:06:08.831889 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4851e40 already has epoch 194496 -7247> 2014-08-13 18:06:08.850445 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.292 192.168.1.113:0/16056 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.850390) v2 ==== 47+0+0 (2764020476 0 0) 0x4de9f80 con 0x4c5a680 -7246> 2014-08-13 18:06:08.850452 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.292 192.168.1.113:0/16056 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:08.850390) v2 ==== 47+0+0 (2764020476 0 0) 0x4deee40 con 0x4c5d3e0 -7245> 2014-08-13 18:06:08.850496 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.850390) v2 -- ?+0 0x4dea140 con 0x4c5a680 -7244> 2014-08-13 18:06:08.850521 7f5c817ec700 10 osd.4 194496 note_peer_epoch osd.292 now has 194496 -7243> 2014-08-13 18:06:08.850530 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 already has epoch 194496 -7242> 2014-08-13 18:06:08.850583 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.113:0/16056 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:08.850390) v2 -- ?+0 0x4def1c0 con 0x4c5d3e0 -7241> 2014-08-13 18:06:08.850639 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4ab32e0 already has epoch 194496 -7240> 2014-08-13 18:06:08.924479 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 9 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1199412992 0 0) 0x510ee00 con 0x4b10b00 -7239> 2014-08-13 18:06:08.924532 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7238> 2014-08-13 18:06:08.924547 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7237> 2014-08-13 18:06:08.924549 7f5c835ef700 20 osd.4 194496 _dispatch 0x510ee00 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7236> 2014-08-13 18:06:08.924562 7f5c835ef700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924247, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7235> 2014-08-13 18:06:08.924578 7f5c835ef700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924251, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7234> 2014-08-13 18:06:08.924593 7f5c835ef700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924396, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7233> 2014-08-13 18:06:08.924602 7f5c835ef700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924530, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7232> 2014-08-13 18:06:08.924614 7f5c835ef700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924613, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7231> 2014-08-13 18:06:08.924627 7f5c835ef700 10 osd.4 194496 handle_replica_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194496 -7230> 2014-08-13 18:06:08.924639 7f5c835ef700 15 osd.4 194496 require_same_or_newer_map 194496 (i am 194496) 0x510ee00 -7229> 2014-08-13 18:06:08.924645 7f5c835ef700 20 osd.4 194496 _share_map_incoming osd.397 192.168.1.105:6808/41085 194496 -7228> 2014-08-13 18:06:08.924654 7f5c835ef700 10 osd.4 194496 note_peer_epoch osd.397 now has 194496 -7227> 2014-08-13 18:06:08.924663 7f5c835ef700 15 osd.4 194496 enqueue_op 0x20a50a0 prio 127 cost 0 latency 0.000416 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7226> 2014-08-13 18:06:08.924698 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7225> 2014-08-13 18:06:08.924702 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7224> 2014-08-13 18:06:08.924775 7f5c7c7e4700 10 osd.4 194496 dequeue_op 0x20a50a0 prio 127 cost 0 latency 0.000528 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] -7223> 2014-08-13 18:06:08.924822 7f5c7c7e4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924822, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7222> 2014-08-13 18:06:08.924842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_message: 0x20a50a0 -7221> 2014-08-13 18:06:08.924875 7f5c7c7e4700 15 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7220> 2014-08-13 18:06:08.924897 7f5c7c7e4700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] sub_op_scrub_reserve -7219> 2014-08-13 18:06:08.924914 7f5c7c7e4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924914, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7218> 2014-08-13 18:06:08.924929 7f5c7c7e4700 20 osd.4 194496 inc_scrubs_pending 0 -> 1 (max 1, active 0) -7217> 2014-08-13 18:06:08.924940 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.105:6808/41085 -- osd_sub_op_reply(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x530b480 con 0x4b10b00 -7216> 2014-08-13 18:06:08.924971 7f5c7c7e4700 10 osd.4 194496 dequeue_op 0x20a50a0 finish -7215> 2014-08-13 18:06:08.924981 7f5c7c7e4700 5 -- op tracker -- , seq: 553, time: 2014-08-13 18:06:08.924981, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -7214> 2014-08-13 18:06:08.927396 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 10 ==== replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194496,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (2361942456 0 0) 0x4bc0240 con 0x4b10b00 -7213> 2014-08-13 18:06:08.927442 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7212> 2014-08-13 18:06:08.927447 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7211> 2014-08-13 18:06:08.927450 7f5c835ef700 20 osd.4 194496 _dispatch 0x4bc0240 replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194496,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 -7210> 2014-08-13 18:06:08.927464 7f5c835ef700 10 osd.4 194496 queueing MOSDRepScrub replica scrub(pg: 22.3b9,from:0'0,to:194215'518,epoch:194496,start:0//0//-1,end:e50d7bb9//0//-1,chunky:1,deep:1,version:5) v5 -7209> 2014-08-13 18:06:08.927477 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7208> 2014-08-13 18:06:08.927480 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7207> 2014-08-13 18:06:08.927538 7f5c7a9e1700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] replica_scrub -7206> 2014-08-13 18:06:08.927593 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map -7205> 2014-08-13 18:06:08.927611 7f5c7a9e1700 20 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] scrub_map_chunk [0//0//-1,e50d7bb9//0//-1) -7204> 2014-08-13 18:06:08.928474 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] _scan_list scanning 25 objects deeply -7203> 2014-08-13 18:06:08.990083 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map_chunk done. -7202> 2014-08-13 18:06:08.990248 7f5c7a9e1700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.105:6808/41085 -- osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+11549 0x5879600 con 0x4b10b00 -7201> 2014-08-13 18:06:09.039101 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.039139) v2 ==== 47+0+0 (1097321970 0 0) 0x4bf4280 con 0x549aaa0 -7200> 2014-08-13 18:06:09.039155 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.039139) v2 -- ?+0 0x4deee40 con 0x549aaa0 -7199> 2014-08-13 18:06:09.039174 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.358 now has 194496 -7198> 2014-08-13 18:06:09.039163 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.358 192.168.100.114:0/52078 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.039139) v2 ==== 47+0+0 (1097321970 0 0) 0x4bf1a40 con 0x5499a20 -7197> 2014-08-13 18:06:09.039197 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 already has epoch 194496 -7196> 2014-08-13 18:06:09.039207 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.114:0/52078 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.039139) v2 -- ?+0 0x4de9f80 con 0x5499a20 -7195> 2014-08-13 18:06:09.039230 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x3c514a0 already has epoch 194496 -7194> 2014-08-13 18:06:09.068460 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.367 192.168.100.114:0/55513 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.068371) v2 ==== 47+0+0 (1810093646 0 0) 0x5354600 con 0x53a0f20 -7193> 2014-08-13 18:06:09.068510 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.068371) v2 -- ?+0 0x4bf4280 con 0x53a0f20 -7192> 2014-08-13 18:06:09.068538 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.367 now has 194496 -7191> 2014-08-13 18:06:09.068513 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.367 192.168.100.114:0/55513 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.068371) v2 ==== 47+0+0 (1810093646 0 0) 0x5356ac0 con 0x53a02c0 -7190> 2014-08-13 18:06:09.068548 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 already has epoch 194496 -7189> 2014-08-13 18:06:09.068557 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.114:0/55513 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.068371) v2 -- ?+0 0x4bf1a40 con 0x53a02c0 -7188> 2014-08-13 18:06:09.068583 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x4b11ce0 already has epoch 194496 -7187> 2014-08-13 18:06:09.136565 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 11 ==== replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194496,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 ==== 126+0+0 (232189546 0 0) 0x4ee4c80 con 0x4b10b00 -7186> 2014-08-13 18:06:09.136612 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7185> 2014-08-13 18:06:09.136617 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7184> 2014-08-13 18:06:09.136620 7f5c835ef700 20 osd.4 194496 _dispatch 0x4ee4c80 replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194496,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -7183> 2014-08-13 18:06:09.136629 7f5c835ef700 10 osd.4 194496 queueing MOSDRepScrub replica scrub(pg: 22.3b9,from:0'0,to:194215'517,epoch:194496,start:e50d7bb9//0//-1,end:MAX,chunky:1,deep:1,version:5) v5 -7182> 2014-08-13 18:06:09.136644 7f5c835ef700 10 osd.4 194496 do_waiters -- start -7181> 2014-08-13 18:06:09.136646 7f5c835ef700 10 osd.4 194496 do_waiters -- finish -7180> 2014-08-13 18:06:09.136701 7f5c7a9e1700 7 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] replica_scrub -7179> 2014-08-13 18:06:09.136739 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map -7178> 2014-08-13 18:06:09.136757 7f5c7a9e1700 20 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] scrub_map_chunk [e50d7bb9//0//-1,MAX) -7177> 2014-08-13 18:06:09.137525 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] _scan_list scanning 8 objects deeply -7176> 2014-08-13 18:06:09.186747 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.19 192.168.1.115:0/61186 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.186510) v2 ==== 47+0+0 (939352235 0 0) 0x5351a40 con 0x549ca40 -7175> 2014-08-13 18:06:09.186799 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.115:0/61186 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.186510) v2 -- ?+0 0x5354600 con 0x549ca40 -7174> 2014-08-13 18:06:09.186859 7f5c803ea700 10 osd.4 194496 note_peer_epoch osd.19 now has 194496 -7173> 2014-08-13 18:06:09.186873 7f5c803ea700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 already has epoch 194496 -7172> 2014-08-13 18:06:09.186892 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.19 192.168.1.115:0/61186 1 ==== osd_ping(ping e194496 stamp 2014-08-13 18:06:09.186510) v2 ==== 47+0+0 (939352235 0 0) 0x5356e40 con 0x549c0a0 -7171> 2014-08-13 18:06:09.187057 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.115:0/61186 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.186510) v2 -- ?+0 0x5356ac0 con 0x549c0a0 -7170> 2014-08-13 18:06:09.188116 7f5c817ec700 20 osd.4 194496 _share_map_outgoing 0x3c53f40 already has epoch 194496 -7169> 2014-08-13 18:06:09.188891 7f5c849f1700 1 -- 192.168.100.115:6805/37708 <== mon.1 192.168.100.112:6789/0 18 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5138480 con 0x3bcfe80 -7168> 2014-08-13 18:06:09.188932 7f5c849f1700 10 osd.4 194496 do_waiters -- start -7167> 2014-08-13 18:06:09.188938 7f5c849f1700 10 osd.4 194496 do_waiters -- finish -7166> 2014-08-13 18:06:09.188941 7f5c849f1700 20 osd.4 194496 _dispatch 0x5138480 osd_map(194497..194497 src has 185139..194497) v3 -7165> 2014-08-13 18:06:09.188983 7f5c849f1700 3 osd.4 194496 handle_osd_map epochs [194497,194497], i have 194496, src has [185139,194497] -7164> 2014-08-13 18:06:09.188991 7f5c849f1700 10 osd.4 194496 handle_osd_map got inc map for epoch 194497 -7163> 2014-08-13 18:06:09.194252 7f5c849f1700 10 osd.4 194496 advance to epoch 194497 (<= newest 194497) -7162> 2014-08-13 18:06:09.194347 7f5c849f1700 7 osd.4 194497 advance_map epoch 194497 74 pgs -7161> 2014-08-13 18:06:09.194379 7f5c849f1700 10 osd.4 194497 write_superblock sb(009d3518-e60d-4f74-a26d-c08c1976263c osd.4 cd2a2e7d-33b9-4dc8-bfb2-0865b80d8237 e194497 [185139,194497] lci=[194495,194497]) -7160> 2014-08-13 18:06:09.194473 7f5c849f1700 0 osd.4 194497 crush map has features 2578087936000, adjusting msgr requires for mons -7159> 2014-08-13 18:06:09.194484 7f5c849f1700 7 osd.4 194497 consume_map version 194497 -7158> 2014-08-13 18:06:09.196596 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.45 192.168.1.112:6828/6040 11 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfbf00 con 0x4496720 -7157> 2014-08-13 18:06:09.309714 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.310103) v2 ==== 47+0+0 (2566481 0 0) 0x5a6fe00 con 0x4fa3b20 -7156> 2014-08-13 18:06:09.309764 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.310103) v2 -- ?+0 0x5351a40 con 0x4fa3b20 -7155> 2014-08-13 18:06:09.309789 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.343 has 194497 -7154> 2014-08-13 18:06:09.309800 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 -7153> 2014-08-13 18:06:09.309822 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.343 192.168.1.104:0/61960 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.310103) v2 ==== 47+0+0 (2566481 0 0) 0x5a39180 con 0x4fa2ec0 -7152> 2014-08-13 18:06:09.309868 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.104:0/61960 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.310103) v2 -- ?+0 0x5356e40 con 0x4fa2ec0 -7151> 2014-08-13 18:06:09.309893 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 -7150> 2014-08-13 18:06:09.333834 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.371 192.168.100.102:0/16753 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.333821) v2 ==== 47+0+0 (1668749129 0 0) 0x5a39c00 con 0x4c5d800 -7149> 2014-08-13 18:06:09.333843 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.371 192.168.100.102:0/16753 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.333821) v2 ==== 47+0+0 (1668749129 0 0) 0x5a39f80 con 0x4c5ce60 -7148> 2014-08-13 18:06:09.333886 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.333821) v2 -- ?+0 0x5a6fe00 con 0x4c5d800 -7147> 2014-08-13 18:06:09.333911 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.371 has 194497 -7146> 2014-08-13 18:06:09.333921 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4851e40 already has epoch 194497 -7145> 2014-08-13 18:06:09.333972 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.102:0/16753 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.333821) v2 -- ?+0 0x5a39180 con 0x4c5ce60 -7144> 2014-08-13 18:06:09.334027 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4851e40 already has epoch 194497 -7143> 2014-08-13 18:06:09.361077 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.360705) v2 ==== 47+0+0 (1164079812 0 0) 0x5a39180 con 0x4c5e1a0 -7142> 2014-08-13 18:06:09.361125 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.360705) v2 -- ?+0 0x5a39f80 con 0x4c5e1a0 -7141> 2014-08-13 18:06:09.361153 7f5c817ec700 10 osd.4 194497 note_peer_epoch osd.58 now has 194497 -7140> 2014-08-13 18:06:09.361168 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x43f7900 already has epoch 194497 -7139> 2014-08-13 18:06:09.361132 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.58 192.168.1.111:0/14316 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.360705) v2 ==== 47+0+0 (1164079812 0 0) 0x5a3a140 con 0x4c5d120 -7138> 2014-08-13 18:06:09.361205 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.111:0/14316 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.360705) v2 -- ?+0 0x5a39c00 con 0x4c5d120 -7137> 2014-08-13 18:06:09.361240 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x43f7900 already has epoch 194497 -7136> 2014-08-13 18:06:09.485084 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.484949) v2 ==== 47+0+0 (2312534689 0 0) 0x588af40 con 0x4f889a0 -7135> 2014-08-13 18:06:09.485145 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.484949) v2 -- ?+0 0x5a3a140 con 0x4f889a0 -7134> 2014-08-13 18:06:09.485169 7f5c803ea700 10 osd.4 194497 note_peer_epoch osd.323 now has 194497 -7133> 2014-08-13 18:06:09.485179 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x49a2520 already has epoch 194497 -7132> 2014-08-13 18:06:09.485087 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.323 192.168.100.106:0/8118 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.484949) v2 ==== 47+0+0 (2312534689 0 0) 0x588fc40 con 0x4f202c0 -7131> 2014-08-13 18:06:09.485391 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.106:0/8118 -- osd_ping(ping_reply e194496 stamp 2014-08-13 18:06:09.484949) v2 -- ?+0 0x5a39180 con 0x4f202c0 -7130> 2014-08-13 18:06:09.485418 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x49a2520 already has epoch 194497 -7129> 2014-08-13 18:06:09.485752 7f5c7a9e1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] build_scrub_map_chunk done. -7128> 2014-08-13 18:06:09.485841 7f5c7a9e1700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.105:6808/41085 -- osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -- ?+3730 0x59e2c00 con 0x4b10b00 -7127> 2014-08-13 18:06:09.485923 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] null -7126> 2014-08-13 18:06:09.485984 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] null -7125> 2014-08-13 18:06:09.486010 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7124> 2014-08-13 18:06:09.486045 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] null -7123> 2014-08-13 18:06:09.486030 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_advance_map [4,286,375]/[4,375,10] -- 4/4 -7122> 2014-08-13 18:06:09.486073 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7121> 2014-08-13 18:06:09.486065 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] null -7120> 2014-08-13 18:06:09.486084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Peering advmap -7119> 2014-08-13 18:06:09.486087 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7118> 2014-08-13 18:06:09.486075 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [351,4,18]/[351,4,18] -- 351/351 -7117> 2014-08-13 18:06:09.486119 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7116> 2014-08-13 18:06:09.486113 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7115> 2014-08-13 18:06:09.486109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] adjust_need_up_thru now 194496, need_up_thru now false -7114> 2014-08-13 18:06:09.486130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap -7113> 2014-08-13 18:06:09.486136 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] null -7112> 2014-08-13 18:06:09.486139 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Started advmap -7111> 2014-08-13 18:06:09.486157 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] null -7110> 2014-08-13 18:06:09.486153 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down -7109> 2014-08-13 18:06:09.486185 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map -7108> 2014-08-13 18:06:09.486183 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] null -7107> 2014-08-13 18:06:09.486176 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] check_recovery_sources no source osds () went down -7106> 2014-08-13 18:06:09.486203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters -7105> 2014-08-13 18:06:09.486216 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7104> 2014-08-13 18:06:09.486221 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -7103> 2014-08-13 18:06:09.486220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] handle_activate_map -7102> 2014-08-13 18:06:09.486240 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.220( v 194215'521 (0'0,194215'521] local-les=194496 n=36 ec=183096 les/c 194496/194493 194495/194495/192967) [351,4,18] r=1 lpr=194495 pi=192970-194494/36 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -7101> 2014-08-13 18:06:09.486240 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] null -7100> 2014-08-13 18:06:09.486245 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: handle ActMap primary -7099> 2014-08-13 18:06:09.486266 7f5c7c7e4700 10 log is not dirty -7098> 2014-08-13 18:06:09.486267 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] publish_stats_to_osd 194497:30301 -7097> 2014-08-13 18:06:09.486266 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] null -7096> 2014-08-13 18:06:09.486288 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] take_waiters -7095> 2014-08-13 18:06:09.486294 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] null -7094> 2014-08-13 18:06:09.486318 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7093> 2014-08-13 18:06:09.486311 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering/WaitUpThru 1.173163 5 0.000795 -7092> 2014-08-13 18:06:09.486310 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,58,343]/[4,58,343] -- 4/4 -7091> 2014-08-13 18:06:09.486343 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7090> 2014-08-13 18:06:09.486340 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] null -7089> 2014-08-13 18:06:09.486340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] state: Leaving Peering -7088> 2014-08-13 18:06:09.486351 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -7087> 2014-08-13 18:06:09.486366 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null -7086> 2014-08-13 18:06:09.486364 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped+peering m=56] exit Started/Primary/Peering 1.328868 0 0.000000 -7085> 2014-08-13 18:06:09.486372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -7084> 2014-08-13 18:06:09.486389 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7083> 2014-08-13 18:06:09.486393 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -7082> 2014-08-13 18:06:09.486396 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active -7081> 2014-08-13 18:06:09.486412 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -7080> 2014-08-13 18:06:09.486411 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] null -7079> 2014-08-13 18:06:09.486431 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -7078> 2014-08-13 18:06:09.486437 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] null -7077> 2014-08-13 18:06:09.486417 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: In Active, about to call activate -7076> 2014-08-13 18:06:09.486458 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null -7075> 2014-08-13 18:06:09.486481 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - purged_snaps [] cached_removed_snaps [] -7074> 2014-08-13 18:06:09.486484 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] null -7073> 2014-08-13 18:06:09.486508 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7072> 2014-08-13 18:06:09.486503 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - snap_trimq [] -7071> 2014-08-13 18:06:09.486528 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] null -7070> 2014-08-13 18:06:09.486525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - not complete, missing(56) -7069> 2014-08-13 18:06:09.486548 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7068> 2014-08-13 18:06:09.486567 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7067> 2014-08-13 18:06:09.486586 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] null -7066> 2014-08-13 18:06:09.486605 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] null -7065> 2014-08-13 18:06:09.486626 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] null -7064> 2014-08-13 18:06:09.486647 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7063> 2014-08-13 18:06:09.486666 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] null -7062> 2014-08-13 18:06:09.486685 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] null -7061> 2014-08-13 18:06:09.486692 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] -7060> 2014-08-13 18:06:09.486708 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -7059> 2014-08-13 18:06:09.486708 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] null -7058> 2014-08-13 18:06:09.486730 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7057> 2014-08-13 18:06:09.486749 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] null -7056> 2014-08-13 18:06:09.486768 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] null -7055> 2014-08-13 18:06:09.486795 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7054> 2014-08-13 18:06:09.486815 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] null -7053> 2014-08-13 18:06:09.486833 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7052> 2014-08-13 18:06:09.486839 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2687 -7051> 2014-08-13 18:06:09.486854 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] null -7050> 2014-08-13 18:06:09.486857 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -7049> 2014-08-13 18:06:09.486874 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -7048> 2014-08-13 18:06:09.486874 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] null -7047> 2014-08-13 18:06:09.486891 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -7046> 2014-08-13 18:06:09.486911 7f5c7c7e4700 10 log is not dirty -7045> 2014-08-13 18:06:09.486899 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null -7044> 2014-08-13 18:06:09.486922 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] null -7043> 2014-08-13 18:06:09.486935 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [9,292,4]/[9,292,4] -- 9/9 -7042> 2014-08-13 18:06:09.486942 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] null -7041> 2014-08-13 18:06:09.486958 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7040> 2014-08-13 18:06:09.486965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap -7039> 2014-08-13 18:06:09.486963 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] null -7038> 2014-08-13 18:06:09.486983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down -7037> 2014-08-13 18:06:09.486987 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] null -7036> 2014-08-13 18:06:09.487001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map -7035> 2014-08-13 18:06:09.487012 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] null -7034> 2014-08-13 18:06:09.487019 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters -7033> 2014-08-13 18:06:09.487034 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -7032> 2014-08-13 18:06:09.487035 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] null -7031> 2014-08-13 18:06:09.487051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.28f( v 194215'495 (0'0,194215'495] local-les=194496 n=42 ec=183096 les/c 194496/194496 194495/194495/192972) [9,292,4] r=2 lpr=194495 pi=192972-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -7030> 2014-08-13 18:06:09.487070 7f5c7c7e4700 10 log is not dirty -7029> 2014-08-13 18:06:09.487057 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] null -7028> 2014-08-13 18:06:09.487082 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] null -7027> 2014-08-13 18:06:09.487099 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_advance_map [58,28,4]/[58,28,4] -- 58/58 -7026> 2014-08-13 18:06:09.487104 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] null -7025> 2014-08-13 18:06:09.487120 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7024> 2014-08-13 18:06:09.487124 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] null -7023> 2014-08-13 18:06:09.487126 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] state: Started advmap -7022> 2014-08-13 18:06:09.487144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] check_recovery_sources no source osds () went down -7021> 2014-08-13 18:06:09.487143 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] null -7020> 2014-08-13 18:06:09.487168 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] null -7019> 2014-08-13 18:06:09.487161 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map -7018> 2014-08-13 18:06:09.487187 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] take_waiters -7017> 2014-08-13 18:06:09.487202 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -7016> 2014-08-13 18:06:09.487189 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] null -7015> 2014-08-13 18:06:09.487218 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.14a( v 194215'513 (0'0,194215'513] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192935) [58,28,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'481 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -7014> 2014-08-13 18:06:09.487237 7f5c7c7e4700 10 log is not dirty -7013> 2014-08-13 18:06:09.487225 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] null -7012> 2014-08-13 18:06:09.487246 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] null -7011> 2014-08-13 18:06:09.487259 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,301,324]/[4,301,324] -- 4/4 -7010> 2014-08-13 18:06:09.487266 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7009> 2014-08-13 18:06:09.487282 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -7008> 2014-08-13 18:06:09.487287 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7007> 2014-08-13 18:06:09.487288 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -7006> 2014-08-13 18:06:09.487307 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -7005> 2014-08-13 18:06:09.487308 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] null -7004> 2014-08-13 18:06:09.487326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,324) went down -7003> 2014-08-13 18:06:09.487330 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] null -7002> 2014-08-13 18:06:09.487344 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -7001> 2014-08-13 18:06:09.487350 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] null -7000> 2014-08-13 18:06:09.487361 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -6999> 2014-08-13 18:06:09.487372 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] null -6998> 2014-08-13 18:06:09.487395 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] null -6997> 2014-08-13 18:06:09.487416 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] null -6996> 2014-08-13 18:06:09.487433 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] null -6995> 2014-08-13 18:06:09.487452 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] null -6994> 2014-08-13 18:06:09.487471 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] null -6993> 2014-08-13 18:06:09.487489 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] null -6992> 2014-08-13 18:06:09.487515 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] null -6991> 2014-08-13 18:06:09.487534 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] null -6990> 2014-08-13 18:06:09.487553 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] null -6989> 2014-08-13 18:06:09.487573 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] null -6988> 2014-08-13 18:06:09.487591 7f5c849f1700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] null -6987> 2014-08-13 18:06:09.487616 7f5c849f1700 10 osd.4 194497 maybe_update_heartbeat_peers updating -6986> 2014-08-13 18:06:09.487619 7f5c849f1700 20 osd.4 194497 24.30b heartbeat_peers 4,10,286,301,375 -6985> 2014-08-13 18:06:09.487628 7f5c849f1700 20 osd.4 194497 22.220 heartbeat_peers -6984> 2014-08-13 18:06:09.487631 7f5c849f1700 20 osd.4 194497 22.362 heartbeat_peers 4,58,61,343,349,362 -6983> 2014-08-13 18:06:09.487638 7f5c849f1700 20 osd.4 194497 22.28f heartbeat_peers -6982> 2014-08-13 18:06:09.487641 7f5c849f1700 20 osd.4 194497 22.14a heartbeat_peers -6981> 2014-08-13 18:06:09.487643 7f5c849f1700 20 osd.4 194497 22.22c heartbeat_peers 4,301,302,324,368 -6980> 2014-08-13 18:06:09.487650 7f5c849f1700 20 osd.4 194497 22.619 heartbeat_peers 4,44,48,62,65,314 -6979> 2014-08-13 18:06:09.487656 7f5c849f1700 20 osd.4 194497 22.30f heartbeat_peers -6978> 2014-08-13 18:06:09.487659 7f5c849f1700 20 osd.4 194497 24.9b heartbeat_peers 4,291,350,365,367,378 -6977> 2014-08-13 18:06:09.487667 7f5c849f1700 20 osd.4 194497 22.581 heartbeat_peers -6976> 2014-08-13 18:06:09.487670 7f5c849f1700 20 osd.4 194497 22.f3 heartbeat_peers 4,285,311,312,369 -6975> 2014-08-13 18:06:09.487675 7f5c849f1700 20 osd.4 194497 22.116 heartbeat_peers -6974> 2014-08-13 18:06:09.487677 7f5c849f1700 20 osd.4 194497 24.3d heartbeat_peers 4,20,25,303,362,373 -6973> 2014-08-13 18:06:09.487685 7f5c849f1700 20 osd.4 194497 22.4aa heartbeat_peers -6972> 2014-08-13 18:06:09.487687 7f5c849f1700 20 osd.4 194497 22.1d2 heartbeat_peers 4,58,61,330,331,349 -6971> 2014-08-13 18:06:09.487693 7f5c849f1700 20 osd.4 194497 24.1de heartbeat_peers -6970> 2014-08-13 18:06:09.487696 7f5c849f1700 20 osd.4 194497 2.2a heartbeat_peers 4,340 -6969> 2014-08-13 18:06:09.487699 7f5c849f1700 20 osd.4 194497 22.113 heartbeat_peers 4,24,45,55,339,342 -6968> 2014-08-13 18:06:09.487705 7f5c849f1700 20 osd.4 194497 24.39a heartbeat_peers -6967> 2014-08-13 18:06:09.487708 7f5c849f1700 20 osd.4 194497 24.33a heartbeat_peers -6966> 2014-08-13 18:06:09.487710 7f5c849f1700 20 osd.4 194497 1.2b heartbeat_peers 4,340 -6965> 2014-08-13 18:06:09.487714 7f5c849f1700 20 osd.4 194497 24.34 heartbeat_peers 4,26,30,340 -6964> 2014-08-13 18:06:09.487764 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] -6963> 2014-08-13 18:06:09.487779 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -6962> 2014-08-13 18:06:09.487979 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2836 -6961> 2014-08-13 18:06:09.487995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -6960> 2014-08-13 18:06:09.488010 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -6959> 2014-08-13 18:06:09.488025 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -6958> 2014-08-13 18:06:09.488043 7f5c7c7e4700 10 log is not dirty -6957> 2014-08-13 18:06:09.488282 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.26 192.168.1.115:6803/58522 192.168.100.115:6804/58522 -6956> 2014-08-13 18:06:09.488329 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,65,62]/[4,65,62] -- 4/4 -6955> 2014-08-13 18:06:09.488375 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6954> 2014-08-13 18:06:09.488385 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.30 192.168.1.107:6824/55491 192.168.100.107:6841/55491 -6953> 2014-08-13 18:06:09.488384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -6952> 2014-08-13 18:06:09.488403 7f5c849f1700 20 osd.4 194497 22.99 heartbeat_peers 4,20,31,337,363,374 -6951> 2014-08-13 18:06:09.488410 7f5c849f1700 20 osd.4 194497 22.84 heartbeat_peers 4,28,304 -6950> 2014-08-13 18:06:09.488406 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -6949> 2014-08-13 18:06:09.488415 7f5c849f1700 20 osd.4 194497 22.4df heartbeat_peers 4,352,356,357,371,380 -6948> 2014-08-13 18:06:09.488430 7f5c849f1700 20 osd.4 194497 22.39d heartbeat_peers 4,320,344,363,366,373 -6947> 2014-08-13 18:06:09.488437 7f5c849f1700 20 osd.4 194497 22.2b9 heartbeat_peers -6946> 2014-08-13 18:06:09.488439 7f5c849f1700 20 osd.4 194497 24.2c heartbeat_peers -6945> 2014-08-13 18:06:09.488442 7f5c849f1700 20 osd.4 194497 24.3f7 heartbeat_peers -6944> 2014-08-13 18:06:09.488429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -6943> 2014-08-13 18:06:09.488445 7f5c849f1700 20 osd.4 194497 22.775 heartbeat_peers 0,4,56,63,353,369 -6942> 2014-08-13 18:06:09.488451 7f5c849f1700 20 osd.4 194497 22.630 heartbeat_peers -6941> 2014-08-13 18:06:09.488455 7f5c849f1700 20 osd.4 194497 24.e2 heartbeat_peers -6940> 2014-08-13 18:06:09.488457 7f5c849f1700 20 osd.4 194497 22.69f heartbeat_peers -6939> 2014-08-13 18:06:09.488460 7f5c849f1700 20 osd.4 194497 22.3e4 heartbeat_peers 4,45,303,313,318,342 -6938> 2014-08-13 18:06:09.488451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -6937> 2014-08-13 18:06:09.488466 7f5c849f1700 20 osd.4 194497 24.2a1 heartbeat_peers -6936> 2014-08-13 18:06:09.488469 7f5c849f1700 20 osd.4 194497 22.639 heartbeat_peers -6935> 2014-08-13 18:06:09.488472 7f5c849f1700 20 osd.4 194497 22.707 heartbeat_peers 4,15,21,303,310,342 -6934> 2014-08-13 18:06:09.488478 7f5c849f1700 20 osd.4 194497 24.5e heartbeat_peers -6933> 2014-08-13 18:06:09.488469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -6932> 2014-08-13 18:06:09.488481 7f5c849f1700 20 osd.4 194497 22.1f1 heartbeat_peers 4,5,11,47,54,368 -6931> 2014-08-13 18:06:09.488488 7f5c849f1700 20 osd.4 194497 22.2d1 heartbeat_peers -6930> 2014-08-13 18:06:09.488490 7f5c849f1700 20 osd.4 194497 24.b6 heartbeat_peers 4,17,28,315,327,330 -6929> 2014-08-13 18:06:09.488497 7f5c849f1700 20 osd.4 194497 24.3bf heartbeat_peers -6928> 2014-08-13 18:06:09.488500 7f5c849f1700 20 osd.4 194497 22.1fb heartbeat_peers -6927> 2014-08-13 18:06:09.488502 7f5c849f1700 20 osd.4 194497 24.350 heartbeat_peers -6926> 2014-08-13 18:06:09.488505 7f5c849f1700 20 osd.4 194497 24.1f7 heartbeat_peers 4,55,59,351,376 -6925> 2014-08-13 18:06:09.488511 7f5c849f1700 20 osd.4 194497 24.136 heartbeat_peers 4,50,55,61,65,356 -6924> 2014-08-13 18:06:09.488518 7f5c849f1700 20 osd.4 194497 22.4fe heartbeat_peers -6923> 2014-08-13 18:06:09.488520 7f5c849f1700 20 osd.4 194497 22.653 heartbeat_peers -6922> 2014-08-13 18:06:09.488523 7f5c849f1700 20 osd.4 194497 24.3b7 heartbeat_peers -6921> 2014-08-13 18:06:09.488525 7f5c849f1700 20 osd.4 194497 22.3b9 heartbeat_peers -6920> 2014-08-13 18:06:09.488528 7f5c849f1700 20 osd.4 194497 22.791 heartbeat_peers -6919> 2014-08-13 18:06:09.488530 7f5c849f1700 20 osd.4 194497 22.6bd heartbeat_peers 4,50,289 -6918> 2014-08-13 18:06:09.488535 7f5c849f1700 20 osd.4 194497 24.181 heartbeat_peers 4,55,296,307,320,344 -6917> 2014-08-13 18:06:09.488617 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.296 192.168.1.109:6807/31286 192.168.100.109:6815/31286 -6916> 2014-08-13 18:06:09.488797 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] -6915> 2014-08-13 18:06:09.488818 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -6914> 2014-08-13 18:06:09.488824 7f5c849f1700 10 osd.4 194497 _add_heartbeat_peer: new peer osd.307 192.168.1.109:6805/31016 192.168.100.109:6811/31016 -6913> 2014-08-13 18:06:09.488845 7f5c849f1700 20 osd.4 194497 24.20e heartbeat_peers -6912> 2014-08-13 18:06:09.488849 7f5c849f1700 20 osd.4 194497 24.1e4 heartbeat_peers -6911> 2014-08-13 18:06:09.488851 7f5c849f1700 20 osd.4 194497 24.1e6 heartbeat_peers -6910> 2014-08-13 18:06:09.488854 7f5c849f1700 20 osd.4 194497 22.12a heartbeat_peers -6909> 2014-08-13 18:06:09.488856 7f5c849f1700 20 osd.4 194497 22.3af heartbeat_peers 4,44,61,339,362,369 -6908> 2014-08-13 18:06:09.488862 7f5c849f1700 20 osd.4 194497 22.481 heartbeat_peers 4,287,303,306,309,365,381 -6907> 2014-08-13 18:06:09.488869 7f5c849f1700 20 osd.4 194497 22.48e heartbeat_peers 4,289,327,341,376 -6906> 2014-08-13 18:06:09.488874 7f5c849f1700 20 osd.4 194497 22.3ad heartbeat_peers -6905> 2014-08-13 18:06:09.488877 7f5c849f1700 20 osd.4 194497 24.206 heartbeat_peers -6904> 2014-08-13 18:06:09.488879 7f5c849f1700 20 osd.4 194497 24.1bb heartbeat_peers 4,331,332,337,348,379 -6903> 2014-08-13 18:06:09.488887 7f5c849f1700 20 osd.4 194497 24.299 heartbeat_peers -6902> 2014-08-13 18:06:09.488889 7f5c849f1700 20 osd.4 194497 22.3d5 heartbeat_peers 4,286,343 -6901> 2014-08-13 18:06:09.488893 7f5c849f1700 20 osd.4 194497 22.537 heartbeat_peers -6900> 2014-08-13 18:06:09.488896 7f5c849f1700 20 osd.4 194497 22.1b8 heartbeat_peers 4,340,347 -6899> 2014-08-13 18:06:09.488900 7f5c849f1700 20 osd.4 194497 22.29a heartbeat_peers -6898> 2014-08-13 18:06:09.488902 7f5c849f1700 20 osd.4 194497 24.cd heartbeat_peers -6897> 2014-08-13 18:06:09.488905 7f5c849f1700 20 osd.4 194497 22.226 heartbeat_peers -6896> 2014-08-13 18:06:09.488907 7f5c849f1700 20 osd.4 194497 22.3d9 heartbeat_peers 4,293,338,373,376,382,396,397 -6895> 2014-08-13 18:06:09.488914 7f5c849f1700 20 osd.4 194497 24.2ec heartbeat_peers -6894> 2014-08-13 18:06:09.488916 7f5c849f1700 20 osd.4 194497 22.1a0 heartbeat_peers -6893> 2014-08-13 18:06:09.488919 7f5c849f1700 20 osd.4 194497 24.369 heartbeat_peers -6892> 2014-08-13 18:06:09.488922 7f5c849f1700 10 osd.4 194497 adding neighbor peer osd.3 -6891> 2014-08-13 18:06:09.488925 7f5c849f1700 10 osd.4 194497 adding neighbor peer osd.5 -6890> 2014-08-13 18:06:09.488936 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.7 192.168.1.115:6839/62713 192.168.100.115:6846/62713 -6889> 2014-08-13 18:06:09.488951 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x53f8b00 -- 0x547d780 -6888> 2014-08-13 18:06:09.488978 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2739 -6887> 2014-08-13 18:06:09.488995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -6886> 2014-08-13 18:06:09.489016 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x53ffa60 -- 0x547d500 -6885> 2014-08-13 18:06:09.489012 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -6884> 2014-08-13 18:06:09.489028 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -6883> 2014-08-13 18:06:09.489048 7f5c7c7e4700 10 log is not dirty -6882> 2014-08-13 18:06:09.489062 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.32 192.168.1.115:6814/59915 192.168.100.115:6831/59915 -6881> 2014-08-13 18:06:09.489074 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x5908b00 -- 0x59f9180 -6880> 2014-08-13 18:06:09.489074 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_advance_map [54,331,4]/[54,331,4] -- 54/54 -6879> 2014-08-13 18:06:09.489097 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6878> 2014-08-13 18:06:09.489104 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] state: Started advmap -6877> 2014-08-13 18:06:09.489124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] check_recovery_sources no source osds () went down -6876> 2014-08-13 18:06:09.489141 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map -6875> 2014-08-13 18:06:09.489158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] take_waiters -6874> 2014-08-13 18:06:09.489172 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -6873> 2014-08-13 18:06:09.489198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.30f( v 194215'473 (0'0,194215'473] local-les=194496 n=27 ec=183096 les/c 194496/194496 194495/194495/192936) [54,331,4] r=2 lpr=194495 pi=192980-194494/27 luod=0'0 crt=192888'431 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -6872> 2014-08-13 18:06:09.489217 7f5c7c7e4700 10 log is not dirty -6871> 2014-08-13 18:06:09.489050 7f5c46aff700 2 -- 192.168.100.115:0/37708 >> 192.168.1.115:6839/62713 pipe(0x547d780 sd=502 :51294 s=4 pgs=144 cs=1 l=1 c=0x53f8b00).reader couldn't read tag, (0) Success -6870> 2014-08-13 18:06:09.489246 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,378,367]/[4,291,350] -- 4/4 -6869> 2014-08-13 18:06:09.489283 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -6868> 2014-08-13 18:06:09.489276 7f5c46aff700 2 -- 192.168.100.115:0/37708 >> 192.168.1.115:6839/62713 pipe(0x547d780 sd=502 :51294 s=4 pgs=144 cs=1 l=1 c=0x53f8b00).fault (0) Success -6867> 2014-08-13 18:06:09.489291 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap -6866> 2014-08-13 18:06:09.489312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194496, need_up_thru now false -6865> 2014-08-13 18:06:09.489330 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap -6864> 2014-08-13 18:06:09.489349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down -6863> 2014-08-13 18:06:09.489368 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map -6862> 2014-08-13 18:06:09.489093 7f5c45ff4700 2 -- 192.168.100.115:0/37708 >> 192.168.100.115:6846/62713 pipe(0x547d500 sd=506 :53307 s=4 pgs=144 cs=1 l=1 c=0x53ffa60).reader couldn't read tag, (0) Success -6861> 2014-08-13 18:06:09.489387 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary -6860> 2014-08-13 18:06:09.489390 7f5c45ff4700 2 -- 192.168.100.115:0/37708 >> 192.168.100.115:6846/62713 pipe(0x547d500 sd=506 :53307 s=4 pgs=144 cs=1 l=1 c=0x53ffa60).fault (0) Success -6859> 2014-08-13 18:06:09.489412 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194497:25293 -6858> 2014-08-13 18:06:09.489429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters -6857> 2014-08-13 18:06:09.489449 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 1.146346 5 0.000864 -6856> 2014-08-13 18:06:09.489454 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x5908840 -- 0x59f8f00 -6855> 2014-08-13 18:06:09.489472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering -6854> 2014-08-13 18:06:09.489495 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.274086 0 0.000000 -6853> 2014-08-13 18:06:09.489521 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -6852> 2014-08-13 18:06:09.489539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194487 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate -6851> 2014-08-13 18:06:09.489535 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate - starting recovery -6850> 2014-08-13 18:06:09.489564 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] -6849> 2014-08-13 18:06:09.489574 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] -6848> 2014-08-13 18:06:09.489582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] -6847> 2014-08-13 18:06:09.489599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) -6846> 2014-08-13 18:06:09.489635 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) -6845> 2014-08-13 18:06:09.489647 7f5c849f1700 20 osd.4 194497 removing heartbeat peer osd.370 192.168.1.114:6843/56512 192.168.100.114:6850/56512 -6844> 2014-08-13 18:06:09.489668 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x58eb180 -- 0x547dc80 -6843> 2014-08-13 18:06:09.489659 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 is up to date, queueing in pending_activators -6842> 2014-08-13 18:06:09.489728 7f5c849f1700 1 -- 192.168.100.115:0/37708 mark_down 0x58eaec0 -- 0x547da00 -6841> 2014-08-13 18:06:09.489656 7f5c44adf700 2 -- 192.168.100.115:0/37708 >> 192.168.100.115:6831/59915 pipe(0x59f8f00 sd=517 :46289 s=4 pgs=125 cs=1 l=1 c=0x5908840).reader couldn't read tag, (0) Success -6840> 2014-08-13 18:06:09.489742 7f5c44adf700 2 -- 192.168.100.115:0/37708 >> 192.168.100.115:6831/59915 pipe(0x59f8f00 sd=517 :46289 s=4 pgs=125 cs=1 l=1 c=0x5908840).fault (0) Success -6839> 2014-08-13 18:06:09.489700 7f5c463f8700 2 -- 192.168.100.115:0/37708 >> 192.168.1.114:6843/56512 pipe(0x547dc80 sd=503 :41072 s=4 pgs=102 cs=1 l=1 c=0x58eb180).reader couldn't read tag, (0) Success -6838> 2014-08-13 18:06:09.489746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.10 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6837> 2014-08-13 18:06:09.489771 7f5c849f1700 10 osd.4 194497 maybe_update_heartbeat_peers 92 peers, extras 3,5,7,32,370 -6836> 2014-08-13 18:06:09.489780 7f5c849f1700 7 osd.4 194497 activate_map version 194497 -6835> 2014-08-13 18:06:09.489772 7f5c463f8700 2 -- 192.168.100.115:0/37708 >> 192.168.1.114:6843/56512 pipe(0x547dc80 sd=503 :41072 s=4 pgs=102 cs=1 l=1 c=0x58eb180).fault (0) Success -6834> 2014-08-13 18:06:09.489793 7f5c849f1700 10 osd.4 194497 do_waiters -- start -6833> 2014-08-13 18:06:09.489778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -6832> 2014-08-13 18:06:09.489797 7f5c849f1700 10 osd.4 194497 do_waiters -- finish -6831> 2014-08-13 18:06:09.489795 7f5c758d8700 20 osd.4 194497 agent_entry empty queue -6830> 2014-08-13 18:06:09.489812 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6829> 2014-08-13 18:06:09.489823 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6828> 2014-08-13 18:06:09.489826 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfbf00 osd_map(194497..194497 src has 185139..194497) v3 -6827> 2014-08-13 18:06:09.489778 7f5c45bf0700 2 -- 192.168.100.115:0/37708 >> 192.168.100.114:6850/56512 pipe(0x547da00 sd=500 :33996 s=4 pgs=102 cs=1 l=1 c=0x58eaec0).reader couldn't read tag, (0) Success -6826> 2014-08-13 18:06:09.489841 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6825> 2014-08-13 18:06:09.489836 7f5c45bf0700 2 -- 192.168.100.115:0/37708 >> 192.168.100.114:6850/56512 pipe(0x547da00 sd=500 :33996 s=4 pgs=102 cs=1 l=1 c=0x58eaec0).fault (0) Success -6824> 2014-08-13 18:06:09.489848 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6823> 2014-08-13 18:06:09.489851 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6822> 2014-08-13 18:06:09.489853 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6821> 2014-08-13 18:06:09.489854 7f5c7bde3700 0 log [INF] : 24.30b restarting backfill on osd.286 from (0'0,0'0] MAX to 194422'18660 -6820> 2014-08-13 18:06:09.489869 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.17 192.168.1.103:6811/37658 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee6300 con 0x4493f40 -6819> 2014-08-13 18:06:09.489889 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6818> 2014-08-13 18:06:09.489892 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6817> 2014-08-13 18:06:09.489893 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee6300 osd_map(194497..194497 src has 185139..194497) v3 -6816> 2014-08-13 18:06:09.489899 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6815> 2014-08-13 18:06:09.489904 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6814> 2014-08-13 18:06:09.489906 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6813> 2014-08-13 18:06:09.489909 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6812> 2014-08-13 18:06:09.489915 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.44 192.168.1.111:6822/10782 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed5e80 con 0x449a3c0 -6811> 2014-08-13 18:06:09.489928 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6810> 2014-08-13 18:06:09.489931 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6809> 2014-08-13 18:06:09.489933 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5e80 osd_map(194497..194497 src has 185139..194497) v3 -6808> 2014-08-13 18:06:09.489938 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6807> 2014-08-13 18:06:09.489942 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6806> 2014-08-13 18:06:09.489944 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6805> 2014-08-13 18:06:09.489946 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6804> 2014-08-13 18:06:09.489958 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 14 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5154380 con 0x3c527e0 -6803> 2014-08-13 18:06:09.489970 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6802> 2014-08-13 18:06:09.489973 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6801> 2014-08-13 18:06:09.489975 7f5c835ef700 20 osd.4 194497 _dispatch 0x5154380 osd_map(194497..194497 src has 185139..194497) v3 -6800> 2014-08-13 18:06:09.489980 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6799> 2014-08-13 18:06:09.489984 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6798> 2014-08-13 18:06:09.489986 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6797> 2014-08-13 18:06:09.489989 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6796> 2014-08-13 18:06:09.489996 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.55 192.168.1.112:6834/6966 13 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfd100 con 0x3c523c0 -6795> 2014-08-13 18:06:09.490008 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6794> 2014-08-13 18:06:09.490010 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6793> 2014-08-13 18:06:09.490012 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfd100 osd_map(194497..194497 src has 185139..194497) v3 -6792> 2014-08-13 18:06:09.490018 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6791> 2014-08-13 18:06:09.490022 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6790> 2014-08-13 18:06:09.490024 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6789> 2014-08-13 18:06:09.490026 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6788> 2014-08-13 18:06:09.490032 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.50 192.168.1.112:6842/9607 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee18c0 con 0x3c56880 -6787> 2014-08-13 18:06:09.490044 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6786> 2014-08-13 18:06:09.490046 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6785> 2014-08-13 18:06:09.490048 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee18c0 osd_map(194497..194497 src has 185139..194497) v3 -6784> 2014-08-13 18:06:09.490054 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6783> 2014-08-13 18:06:09.490058 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6782> 2014-08-13 18:06:09.490060 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6781> 2014-08-13 18:06:09.490062 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6780> 2014-08-13 18:06:09.490067 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 14 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee2640 con 0x3c55540 -6779> 2014-08-13 18:06:09.490080 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6778> 2014-08-13 18:06:09.490082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6777> 2014-08-13 18:06:09.490084 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee2640 osd_map(194497..194497 src has 185139..194497) v3 -6776> 2014-08-13 18:06:09.490089 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6775> 2014-08-13 18:06:09.490093 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6774> 2014-08-13 18:06:09.490095 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6773> 2014-08-13 18:06:09.490097 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6772> 2014-08-13 18:06:09.490103 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.371 192.168.1.102:6832/16753 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfb180 con 0x4851e40 -6771> 2014-08-13 18:06:09.490115 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6770> 2014-08-13 18:06:09.490117 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6769> 2014-08-13 18:06:09.490119 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfb180 osd_map(194497..194497 src has 185139..194497) v3 -6768> 2014-08-13 18:06:09.490124 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6767> 2014-08-13 18:06:09.490128 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6766> 2014-08-13 18:06:09.490130 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6765> 2014-08-13 18:06:09.490132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6764> 2014-08-13 18:06:09.490143 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.343 192.168.1.104:6802/61960 12 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc4140 con 0x48506e0 -6763> 2014-08-13 18:06:09.490156 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6762> 2014-08-13 18:06:09.490158 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6761> 2014-08-13 18:06:09.490160 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4140 osd_map(194497..194497 src has 185139..194497) v3 -6760> 2014-08-13 18:06:09.490166 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6759> 2014-08-13 18:06:09.490180 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6758> 2014-08-13 18:06:09.490183 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6757> 2014-08-13 18:06:09.490186 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6756> 2014-08-13 18:06:09.490192 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.296 192.168.1.109:6806/31286 4 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee0d80 con 0x4854a40 -6755> 2014-08-13 18:06:09.490204 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6754> 2014-08-13 18:06:09.490207 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6753> 2014-08-13 18:06:09.490209 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0d80 osd_map(194497..194497 src has 185139..194497) v3 -6752> 2014-08-13 18:06:09.490215 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6751> 2014-08-13 18:06:09.490219 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6750> 2014-08-13 18:06:09.490221 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6749> 2014-08-13 18:06:09.490223 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6748> 2014-08-13 18:06:09.490230 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cf9f80 con 0x4854620 -6747> 2014-08-13 18:06:09.490242 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6746> 2014-08-13 18:06:09.490244 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6745> 2014-08-13 18:06:09.490246 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf9f80 osd_map(194497..194497 src has 185139..194497) v3 -6744> 2014-08-13 18:06:09.490252 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6743> 2014-08-13 18:06:09.490256 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6742> 2014-08-13 18:06:09.490258 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6741> 2014-08-13 18:06:09.490260 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6740> 2014-08-13 18:06:09.490267 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.31 192.168.1.103:6831/41157 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cff2c0 con 0x4857bc0 -6739> 2014-08-13 18:06:09.490278 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6738> 2014-08-13 18:06:09.490281 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6737> 2014-08-13 18:06:09.490283 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff2c0 osd_map(194497..194497 src has 185139..194497) v3 -6736> 2014-08-13 18:06:09.490288 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6735> 2014-08-13 18:06:09.490293 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6734> 2014-08-13 18:06:09.490295 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6733> 2014-08-13 18:06:09.490297 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6732> 2014-08-13 18:06:09.490303 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.323 192.168.1.106:6816/8118 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed1b00 con 0x49a2520 -6731> 2014-08-13 18:06:09.490314 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6730> 2014-08-13 18:06:09.490317 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6729> 2014-08-13 18:06:09.490319 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1b00 osd_map(194497..194497 src has 185139..194497) v3 -6728> 2014-08-13 18:06:09.490324 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6727> 2014-08-13 18:06:09.490329 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6726> 2014-08-13 18:06:09.490331 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6725> 2014-08-13 18:06:09.490333 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6724> 2014-08-13 18:06:09.490272 7f5c444d9700 2 -- 192.168.100.115:0/37708 >> 192.168.1.115:6814/59915 pipe(0x59f9180 sd=516 :37511 s=4 pgs=125 cs=1 l=1 c=0x5908b00).reader couldn't read tag, (0) Success -6723> 2014-08-13 18:06:09.490339 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.362 192.168.1.108:6842/8170 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc6300 con 0x49a53e0 -6722> 2014-08-13 18:06:09.490338 7f5c444d9700 2 -- 192.168.100.115:0/37708 >> 192.168.1.115:6814/59915 pipe(0x59f9180 sd=516 :37511 s=4 pgs=125 cs=1 l=1 c=0x5908b00).fault (0) Success -6721> 2014-08-13 18:06:09.490351 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6720> 2014-08-13 18:06:09.490354 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6719> 2014-08-13 18:06:09.490356 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6300 osd_map(194497..194497 src has 185139..194497) v3 -6718> 2014-08-13 18:06:09.490362 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6717> 2014-08-13 18:06:09.490366 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6716> 2014-08-13 18:06:09.490368 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6715> 2014-08-13 18:06:09.490370 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6714> 2014-08-13 18:06:09.490376 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.304 192.168.1.101:6808/22782 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed2640 con 0x4ab69e0 -6713> 2014-08-13 18:06:09.490388 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6712> 2014-08-13 18:06:09.490390 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6711> 2014-08-13 18:06:09.490392 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed2640 osd_map(194497..194497 src has 185139..194497) v3 -6710> 2014-08-13 18:06:09.490398 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6709> 2014-08-13 18:06:09.490402 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6708> 2014-08-13 18:06:09.490404 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6707> 2014-08-13 18:06:09.490406 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6706> 2014-08-13 18:06:09.490411 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.332 192.168.1.110:6806/42024 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc6c00 con 0x4ab2d60 -6705> 2014-08-13 18:06:09.490423 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6704> 2014-08-13 18:06:09.490438 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6703> 2014-08-13 18:06:09.490440 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6c00 osd_map(194497..194497 src has 185139..194497) v3 -6702> 2014-08-13 18:06:09.490445 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6701> 2014-08-13 18:06:09.490449 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6700> 2014-08-13 18:06:09.490451 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6699> 2014-08-13 18:06:09.490453 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6698> 2014-08-13 18:06:09.490459 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.363 192.168.1.102:6842/20017 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed6540 con 0x4ab7640 -6697> 2014-08-13 18:06:09.490470 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6696> 2014-08-13 18:06:09.490473 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6695> 2014-08-13 18:06:09.490474 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed6540 osd_map(194497..194497 src has 185139..194497) v3 -6694> 2014-08-13 18:06:09.490480 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6693> 2014-08-13 18:06:09.490484 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6692> 2014-08-13 18:06:09.490486 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6691> 2014-08-13 18:06:09.490488 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6690> 2014-08-13 18:06:09.490493 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.374 192.168.1.102:6828/16129 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee4380 con 0x3cf4360 -6689> 2014-08-13 18:06:09.490506 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6688> 2014-08-13 18:06:09.490509 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6687> 2014-08-13 18:06:09.490510 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee4380 osd_map(194497..194497 src has 185139..194497) v3 -6686> 2014-08-13 18:06:09.490516 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6685> 2014-08-13 18:06:09.490520 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6684> 2014-08-13 18:06:09.490522 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6683> 2014-08-13 18:06:09.490524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6682> 2014-08-13 18:06:09.490529 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.24 192.168.1.107:6800/52669 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee57c0 con 0x4af9ce0 -6681> 2014-08-13 18:06:09.490541 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6680> 2014-08-13 18:06:09.490543 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6679> 2014-08-13 18:06:09.490545 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee57c0 osd_map(194497..194497 src has 185139..194497) v3 -6678> 2014-08-13 18:06:09.490550 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6677> 2014-08-13 18:06:09.490554 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6676> 2014-08-13 18:06:09.490556 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6675> 2014-08-13 18:06:09.490558 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6674> 2014-08-13 18:06:09.490570 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.6 192.168.1.107:6809/53141 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfee40 con 0x4af98c0 -6673> 2014-08-13 18:06:09.490582 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6672> 2014-08-13 18:06:09.490584 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6671> 2014-08-13 18:06:09.490586 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfee40 osd_map(194497..194497 src has 185139..194497) v3 -6670> 2014-08-13 18:06:09.490592 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6669> 2014-08-13 18:06:09.490596 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6668> 2014-08-13 18:06:09.490598 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6667> 2014-08-13 18:06:09.490599 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6666> 2014-08-13 18:06:09.490605 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.344 192.168.1.106:6812/6757 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc0fc0 con 0x4aff220 -6665> 2014-08-13 18:06:09.490621 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6664> 2014-08-13 18:06:09.490623 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6663> 2014-08-13 18:06:09.490625 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc0fc0 osd_map(194497..194497 src has 185139..194497) v3 -6662> 2014-08-13 18:06:09.490631 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6661> 2014-08-13 18:06:09.490634 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6660> 2014-08-13 18:06:09.490636 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6659> 2014-08-13 18:06:09.490639 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6658> 2014-08-13 18:06:09.490644 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 16 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cff500 con 0x4af8580 -6657> 2014-08-13 18:06:09.490656 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6656> 2014-08-13 18:06:09.490659 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6655> 2014-08-13 18:06:09.490661 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff500 osd_map(194497..194497 src has 185139..194497) v3 -6654> 2014-08-13 18:06:09.490636 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 sending log((192718'15573,194422'18660], crt=194422'18660) -6653> 2014-08-13 18:06:09.490666 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6652> 2014-08-13 18:06:09.490671 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6651> 2014-08-13 18:06:09.490673 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6650> 2014-08-13 18:06:09.490672 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing -6649> 2014-08-13 18:06:09.490675 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6648> 2014-08-13 18:06:09.490681 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.351 192.168.1.102:6830/16444 11 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed4380 con 0x4af8160 -6647> 2014-08-13 18:06:09.490693 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6646> 2014-08-13 18:06:09.490677 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.286 192.168.1.113:6820/19173 -- pg_log(24.30b epoch 194497 log log((192718'15573,194422'18660], crt=194422'18660) query_epoch 194497) v4 -- ?+0 0x58b8600 -6645> 2014-08-13 18:06:09.490675 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery -6644> 2014-08-13 18:06:09.490696 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6643> 2014-08-13 18:06:09.490698 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4380 osd_map(194497..194497 src has 185139..194497) v3 -6642> 2014-08-13 18:06:09.490704 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6641> 2014-08-13 18:06:09.490708 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6640> 2014-08-13 18:06:09.490711 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6639> 2014-08-13 18:06:09.490714 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6638> 2014-08-13 18:06:09.490705 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] -6637> 2014-08-13 18:06:09.490719 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.331 192.168.1.104:6816/837 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cf8900 con 0x4b12940 -6636> 2014-08-13 18:06:09.490732 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6635> 2014-08-13 18:06:09.490708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.286 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6634> 2014-08-13 18:06:09.490734 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6633> 2014-08-13 18:06:09.490736 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf8900 osd_map(194497..194497 src has 185139..194497) v3 -6632> 2014-08-13 18:06:09.490742 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6631> 2014-08-13 18:06:09.490746 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6630> 2014-08-13 18:06:09.490749 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6629> 2014-08-13 18:06:09.490751 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6628> 2014-08-13 18:06:09.490737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) -6627> 2014-08-13 18:06:09.490743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -6626> 2014-08-13 18:06:09.490757 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.320 192.168.1.106:6820/10050 13 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc1f80 con 0x4b12520 -6625> 2014-08-13 18:06:09.490770 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6624> 2014-08-13 18:06:09.490773 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6623> 2014-08-13 18:06:09.490760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 is up to date, queueing in pending_activators -6622> 2014-08-13 18:06:09.490776 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc1f80 osd_map(194497..194497 src has 185139..194497) v3 -6621> 2014-08-13 18:06:09.490782 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6620> 2014-08-13 18:06:09.490769 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 is up to date, queueing in pending_activators -6619> 2014-08-13 18:06:09.490786 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6618> 2014-08-13 18:06:09.490794 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6617> 2014-08-13 18:06:09.490796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6616> 2014-08-13 18:06:09.490802 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.367 192.168.1.114:6840/55513 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5151680 con 0x4b11ce0 -6615> 2014-08-13 18:06:09.490814 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6614> 2014-08-13 18:06:09.490816 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6613> 2014-08-13 18:06:09.490818 7f5c835ef700 20 osd.4 194497 _dispatch 0x5151680 osd_map(194497..194497 src has 185139..194497) v3 -6612> 2014-08-13 18:06:09.490824 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6611> 2014-08-13 18:06:09.490828 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6610> 2014-08-13 18:06:09.490830 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6609> 2014-08-13 18:06:09.490833 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6608> 2014-08-13 18:06:09.490838 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.65 192.168.1.111:6842/17345 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5150fc0 con 0x4b11600 -6607> 2014-08-13 18:06:09.490828 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate peer osd.375 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6606> 2014-08-13 18:06:09.490850 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6605> 2014-08-13 18:06:09.490856 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6604> 2014-08-13 18:06:09.490858 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150fc0 osd_map(194497..194497 src has 185139..194497) v3 -6603> 2014-08-13 18:06:09.490864 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6602> 2014-08-13 18:06:09.490868 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6601> 2014-08-13 18:06:09.490871 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6600> 2014-08-13 18:06:09.490873 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6599> 2014-08-13 18:06:09.490858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.291 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6598> 2014-08-13 18:06:09.490880 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.373 192.168.1.108:6840/7435 16 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5151d40 con 0x4b111e0 -6597> 2014-08-13 18:06:09.490893 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6596> 2014-08-13 18:06:09.490896 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6595> 2014-08-13 18:06:09.490898 7f5c835ef700 20 osd.4 194497 _dispatch 0x5151d40 osd_map(194497..194497 src has 185139..194497) v3 -6594> 2014-08-13 18:06:09.490883 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) -6593> 2014-08-13 18:06:09.490904 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6592> 2014-08-13 18:06:09.490909 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6591> 2014-08-13 18:06:09.490912 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6590> 2014-08-13 18:06:09.490914 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6589> 2014-08-13 18:06:09.490907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 is up to date, queueing in pending_activators -6588> 2014-08-13 18:06:09.490920 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 12 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee0900 con 0x4b10b00 -6587> 2014-08-13 18:06:09.490932 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6586> 2014-08-13 18:06:09.490935 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6585> 2014-08-13 18:06:09.490937 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0900 osd_map(194497..194497 src has 185139..194497) v3 -6584> 2014-08-13 18:06:09.490942 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6583> 2014-08-13 18:06:09.490946 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6582> 2014-08-13 18:06:09.490948 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6581> 2014-08-13 18:06:09.490950 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6580> 2014-08-13 18:06:09.490956 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.396 192.168.1.105:6814/41806 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfca40 con 0x4b2c620 -6579> 2014-08-13 18:06:09.490968 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6578> 2014-08-13 18:06:09.490970 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6577> 2014-08-13 18:06:09.490972 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfca40 osd_map(194497..194497 src has 185139..194497) v3 -6576> 2014-08-13 18:06:09.490977 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6575> 2014-08-13 18:06:09.490981 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6574> 2014-08-13 18:06:09.490983 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6573> 2014-08-13 18:06:09.490985 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6572> 2014-08-13 18:06:09.490991 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 19 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed7980 con 0x4b2bc80 -6571> 2014-08-13 18:06:09.491003 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6570> 2014-08-13 18:06:09.491006 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6569> 2014-08-13 18:06:09.490988 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.350 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6568> 2014-08-13 18:06:09.491008 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed7980 osd_map(194497..194497 src has 185139..194497) v3 -6567> 2014-08-13 18:06:09.491013 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6566> 2014-08-13 18:06:09.491018 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6565> 2014-08-13 18:06:09.491020 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6564> 2014-08-13 18:06:09.491022 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6563> 2014-08-13 18:06:09.490903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery primary has 56 missing -6562> 2014-08-13 18:06:09.491012 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -6561> 2014-08-13 18:06:09.491028 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.379 192.168.1.108:6830/4861 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5155100 con 0x4b2d120 -6560> 2014-08-13 18:06:09.491040 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6559> 2014-08-13 18:06:09.491041 7f5c7c7e4700 0 log [INF] : 24.9b restarting backfill on osd.367 from (0'0,0'0] MAX to 194422'17302 -6558> 2014-08-13 18:06:09.491043 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6557> 2014-08-13 18:06:09.491045 7f5c835ef700 20 osd.4 194497 _dispatch 0x5155100 osd_map(194497..194497 src has 185139..194497) v3 -6556> 2014-08-13 18:06:09.491051 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6555> 2014-08-13 18:06:09.491055 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6554> 2014-08-13 18:06:09.491057 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6553> 2014-08-13 18:06:09.491059 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6552> 2014-08-13 18:06:09.491065 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.0 192.168.1.115:6810/59615 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc2400 con 0x4b2f4e0 -6551> 2014-08-13 18:06:09.491076 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6550> 2014-08-13 18:06:09.491079 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6549> 2014-08-13 18:06:09.491080 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc2400 osd_map(194497..194497 src has 185139..194497) v3 -6548> 2014-08-13 18:06:09.491086 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6547> 2014-08-13 18:06:09.491090 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6546> 2014-08-13 18:06:09.491092 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6545> 2014-08-13 18:06:09.491094 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6544> 2014-08-13 18:06:09.491099 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.11 192.168.1.103:6813/37929 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc4800 con 0x4b2f640 -6543> 2014-08-13 18:06:09.491111 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6542> 2014-08-13 18:06:09.491113 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6541> 2014-08-13 18:06:09.491115 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4800 osd_map(194497..194497 src has 185139..194497) v3 -6540> 2014-08-13 18:06:09.491120 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6539> 2014-08-13 18:06:09.491124 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6538> 2014-08-13 18:06:09.491126 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6537> 2014-08-13 18:06:09.491128 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6536> 2014-08-13 18:06:09.491133 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.352 192.168.1.108:6826/4273 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cf8d80 con 0x4b2fbc0 -6535> 2014-08-13 18:06:09.491145 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6534> 2014-08-13 18:06:09.491030 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.4 -6533> 2014-08-13 18:06:09.491158 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6532> 2014-08-13 18:06:09.491161 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf8d80 osd_map(194497..194497 src has 185139..194497) v3 -6531> 2014-08-13 18:06:09.491167 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6530> 2014-08-13 18:06:09.491178 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6529> 2014-08-13 18:06:09.491181 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6528> 2014-08-13 18:06:09.491183 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6527> 2014-08-13 18:06:09.491189 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.327 192.168.1.106:6804/5112 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed7740 con 0x4c58420 -6526> 2014-08-13 18:06:09.491201 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6525> 2014-08-13 18:06:09.491204 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6524> 2014-08-13 18:06:09.491206 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed7740 osd_map(194497..194497 src has 185139..194497) v3 -6523> 2014-08-13 18:06:09.491211 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6522> 2014-08-13 18:06:09.491215 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6521> 2014-08-13 18:06:09.491218 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6520> 2014-08-13 18:06:09.491220 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6519> 2014-08-13 18:06:09.491225 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.339 192.168.1.110:6818/45268 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cf98c0 con 0x4c594a0 -6518> 2014-08-13 18:06:09.491237 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6517> 2014-08-13 18:06:09.491239 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6516> 2014-08-13 18:06:09.491241 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cf98c0 osd_map(194497..194497 src has 185139..194497) v3 -6515> 2014-08-13 18:06:09.491247 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6514> 2014-08-13 18:06:09.491251 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6513> 2014-08-13 18:06:09.491253 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6512> 2014-08-13 18:06:09.491255 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6511> 2014-08-13 18:06:09.491260 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.311 192.168.1.101:6816/24270 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5150240 con 0x4c59a20 -6510> 2014-08-13 18:06:09.491272 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6509> 2014-08-13 18:06:09.491275 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6508> 2014-08-13 18:06:09.491276 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150240 osd_map(194497..194497 src has 185139..194497) v3 -6507> 2014-08-13 18:06:09.491282 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6506> 2014-08-13 18:06:09.491286 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6505> 2014-08-13 18:06:09.491288 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6504> 2014-08-13 18:06:09.491290 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6503> 2014-08-13 18:06:09.491162 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.4 -6502> 2014-08-13 18:06:09.491295 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.293 192.168.1.109:6816/33855 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfd7c0 con 0x4c59760 -6501> 2014-08-13 18:06:09.491307 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6500> 2014-08-13 18:06:09.491310 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6499> 2014-08-13 18:06:09.491312 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfd7c0 osd_map(194497..194497 src has 185139..194497) v3 -6498> 2014-08-13 18:06:09.491318 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6497> 2014-08-13 18:06:09.491322 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6496> 2014-08-13 18:06:09.491324 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6495> 2014-08-13 18:06:09.491326 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6494> 2014-08-13 18:06:09.491381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 sending log((192834'15069,194422'17302], crt=194422'17302) -6493> 2014-08-13 18:06:09.491303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.4 -6492> 2014-08-13 18:06:09.491412 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4b11ce0 192.168.1.114:6840/55513 -6491> 2014-08-13 18:06:09.491406 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.342 192.168.1.110:6802/41438 11 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee06c0 con 0x4c5b440 -6490> 2014-08-13 18:06:09.491435 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6489> 2014-08-13 18:06:09.491430 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6840/55513 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x47f9d40 con 0x4b11ce0 -6488> 2014-08-13 18:06:09.491440 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6487> 2014-08-13 18:06:09.491443 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee06c0 osd_map(194497..194497 src has 185139..194497) v3 -6486> 2014-08-13 18:06:09.491449 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.367 has 194497 -6485> 2014-08-13 18:06:09.491454 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6484> 2014-08-13 18:06:09.491461 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6483> 2014-08-13 18:06:09.491464 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6482> 2014-08-13 18:06:09.491454 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.367 192.168.1.114:6840/55513 -- pg_log(24.9b epoch 194497 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194497) v4 -- ?+0 0x4160600 -6481> 2014-08-13 18:06:09.491466 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6480> 2014-08-13 18:06:09.491475 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.314 192.168.1.101:6802/22303 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed1f80 con 0x4c5b180 -6479> 2014-08-13 18:06:09.491489 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6478> 2014-08-13 18:06:09.491492 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6477> 2014-08-13 18:06:09.491469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.367 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6476> 2014-08-13 18:06:09.491494 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1f80 osd_map(194497..194497 src has 185139..194497) v3 -6475> 2014-08-13 18:06:09.491500 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6474> 2014-08-13 18:06:09.491505 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6473> 2014-08-13 18:06:09.491507 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6472> 2014-08-13 18:06:09.491509 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6471> 2014-08-13 18:06:09.491497 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -6470> 2014-08-13 18:06:09.491525 7f5c7c7e4700 0 log [INF] : 24.9b restarting backfill on osd.378 from (0'0,0'0] MAX to 194422'17302 -6469> 2014-08-13 18:06:09.491516 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.48 192.168.1.112:6838/8300 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed4c80 con 0x4c5c620 -6468> 2014-08-13 18:06:09.491529 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6467> 2014-08-13 18:06:09.491531 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6466> 2014-08-13 18:06:09.491533 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4c80 osd_map(194497..194497 src has 185139..194497) v3 -6465> 2014-08-13 18:06:09.491423 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.4 -6464> 2014-08-13 18:06:09.491539 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6463> 2014-08-13 18:06:09.491544 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6462> 2014-08-13 18:06:09.491546 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6461> 2014-08-13 18:06:09.491549 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6460> 2014-08-13 18:06:09.491555 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.380 192.168.1.114:6822/51844 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed0900 con 0x4c5c0a0 -6459> 2014-08-13 18:06:09.491567 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6458> 2014-08-13 18:06:09.491569 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6457> 2014-08-13 18:06:09.491571 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed0900 osd_map(194497..194497 src has 185139..194497) v3 -6456> 2014-08-13 18:06:09.491577 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6455> 2014-08-13 18:06:09.491581 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6454> 2014-08-13 18:06:09.491583 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6453> 2014-08-13 18:06:09.491585 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6452> 2014-08-13 18:06:09.491590 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.368 192.168.1.102:6826/15861 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed5a00 con 0x4c5bde0 -6451> 2014-08-13 18:06:09.491602 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6450> 2014-08-13 18:06:09.491604 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6449> 2014-08-13 18:06:09.491606 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5a00 osd_map(194497..194497 src has 185139..194497) v3 -6448> 2014-08-13 18:06:09.491612 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6447> 2014-08-13 18:06:09.491616 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6446> 2014-08-13 18:06:09.491618 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6445> 2014-08-13 18:06:09.491620 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6444> 2014-08-13 18:06:09.491626 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.350 192.168.1.114:6832/53185 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed69c0 con 0x4c5ee00 -6443> 2014-08-13 18:06:09.491637 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6442> 2014-08-13 18:06:09.491640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6441> 2014-08-13 18:06:09.491642 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed69c0 osd_map(194497..194497 src has 185139..194497) v3 -6440> 2014-08-13 18:06:09.491647 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6439> 2014-08-13 18:06:09.491651 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6438> 2014-08-13 18:06:09.491653 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6437> 2014-08-13 18:06:09.491655 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6436> 2014-08-13 18:06:09.491662 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.381 192.168.1.102:6822/15390 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed3cc0 con 0x4c5eb40 -6435> 2014-08-13 18:06:09.491673 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6434> 2014-08-13 18:06:09.491542 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.4 -6433> 2014-08-13 18:06:09.491676 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6432> 2014-08-13 18:06:09.491678 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed3cc0 osd_map(194497..194497 src has 185139..194497) v3 -6431> 2014-08-13 18:06:09.491684 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6430> 2014-08-13 18:06:09.491688 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6429> 2014-08-13 18:06:09.491690 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6428> 2014-08-13 18:06:09.491693 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6427> 2014-08-13 18:06:09.491698 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.369 192.168.1.108:6824/4037 12 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed5340 con 0x4c5e880 -6426> 2014-08-13 18:06:09.491710 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6425> 2014-08-13 18:06:09.491712 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6424> 2014-08-13 18:06:09.491714 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed5340 osd_map(194497..194497 src has 185139..194497) v3 -6423> 2014-08-13 18:06:09.491719 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6422> 2014-08-13 18:06:09.491724 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6421> 2014-08-13 18:06:09.491726 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6420> 2014-08-13 18:06:09.491728 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6419> 2014-08-13 18:06:09.491734 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.375 192.168.1.114:6830/52879 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed1200 con 0x4c5e5c0 -6418> 2014-08-13 18:06:09.491745 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6417> 2014-08-13 18:06:09.491747 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6416> 2014-08-13 18:06:09.491749 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1200 osd_map(194497..194497 src has 185139..194497) v3 -6415> 2014-08-13 18:06:09.491755 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6414> 2014-08-13 18:06:09.491759 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6413> 2014-08-13 18:06:09.491761 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6412> 2014-08-13 18:06:09.491763 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6411> 2014-08-13 18:06:09.491768 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.302 192.168.1.113:6810/15731 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed4800 con 0x4f218c0 -6410> 2014-08-13 18:06:09.491780 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6409> 2014-08-13 18:06:09.491783 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6408> 2014-08-13 18:06:09.491784 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed4800 osd_map(194497..194497 src has 185139..194497) v3 -6407> 2014-08-13 18:06:09.491790 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6406> 2014-08-13 18:06:09.491794 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6405> 2014-08-13 18:06:09.491798 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6404> 2014-08-13 18:06:09.491800 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6403> 2014-08-13 18:06:09.491793 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 sending log((192834'15069,194422'17302], crt=194422'17302) -6402> 2014-08-13 18:06:09.491806 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.346 192.168.1.104:6820/2308 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cff980 con 0x4f21600 -6401> 2014-08-13 18:06:09.491818 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6400> 2014-08-13 18:06:09.491680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.4 -6399> 2014-08-13 18:06:09.491821 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6398> 2014-08-13 18:06:09.491823 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cff980 osd_map(194497..194497 src has 185139..194497) v3 -6397> 2014-08-13 18:06:09.491829 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6396> 2014-08-13 18:06:09.491821 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing -6395> 2014-08-13 18:06:09.491833 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6394> 2014-08-13 18:06:09.491836 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6393> 2014-08-13 18:06:09.491838 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6392> 2014-08-13 18:06:09.491835 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.378 192.168.1.102:6840/19195 -- pg_log(24.9b epoch 194497 log log((192834'15069,194422'17302], crt=194422'17302) query_epoch 194497) v4 -- ?+0 0x58bec00 -6391> 2014-08-13 18:06:09.491845 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 11 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x51506c0 con 0x4f20dc0 -6390> 2014-08-13 18:06:09.491857 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6389> 2014-08-13 18:06:09.491860 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6388> 2014-08-13 18:06:09.491863 7f5c835ef700 20 osd.4 194497 _dispatch 0x51506c0 osd_map(194497..194497 src has 185139..194497) v3 -6387> 2014-08-13 18:06:09.491869 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6386> 2014-08-13 18:06:09.491873 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6385> 2014-08-13 18:06:09.491875 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6384> 2014-08-13 18:06:09.491878 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6383> 2014-08-13 18:06:09.491859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.378 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -6382> 2014-08-13 18:06:09.491883 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.365 192.168.1.102:6824/15623 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5152ac0 con 0x4fa2260 -6381> 2014-08-13 18:06:09.491895 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6380> 2014-08-13 18:06:09.491897 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6379> 2014-08-13 18:06:09.491899 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152ac0 osd_map(194497..194497 src has 185139..194497) v3 -6378> 2014-08-13 18:06:09.491905 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6377> 2014-08-13 18:06:09.491909 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6376> 2014-08-13 18:06:09.491911 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6375> 2014-08-13 18:06:09.491913 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6374> 2014-08-13 18:06:09.491921 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.366 192.168.1.108:6828/4534 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5152640 con 0x4fa1a20 -6373> 2014-08-13 18:06:09.491933 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6372> 2014-08-13 18:06:09.491936 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6371> 2014-08-13 18:06:09.491938 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152640 osd_map(194497..194497 src has 185139..194497) v3 -6370> 2014-08-13 18:06:09.491825 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.4 -6369> 2014-08-13 18:06:09.491944 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6368> 2014-08-13 18:06:09.491949 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6367> 2014-08-13 18:06:09.491952 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6366> 2014-08-13 18:06:09.491954 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6365> 2014-08-13 18:06:09.491959 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.306 192.168.1.101:6806/6812 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5152f40 con 0x4fa5800 -6364> 2014-08-13 18:06:09.491971 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6363> 2014-08-13 18:06:09.491974 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6362> 2014-08-13 18:06:09.491989 7f5c835ef700 20 osd.4 194497 _dispatch 0x5152f40 osd_map(194497..194497 src has 185139..194497) v3 -6361> 2014-08-13 18:06:09.491995 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6360> 2014-08-13 18:06:09.491999 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6359> 2014-08-13 18:06:09.492001 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6358> 2014-08-13 18:06:09.492003 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6357> 2014-08-13 18:06:09.492008 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.63 192.168.1.111:6830/12680 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5153cc0 con 0x4fa5280 -6356> 2014-08-13 18:06:09.492020 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6355> 2014-08-13 18:06:09.492022 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6354> 2014-08-13 18:06:09.492024 7f5c835ef700 20 osd.4 194497 _dispatch 0x5153cc0 osd_map(194497..194497 src has 185139..194497) v3 -6353> 2014-08-13 18:06:09.492029 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6352> 2014-08-13 18:06:09.492033 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6351> 2014-08-13 18:06:09.492035 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6350> 2014-08-13 18:06:09.492037 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6349> 2014-08-13 18:06:09.492043 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.349 192.168.1.104:6800/61726 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5154a40 con 0x4fa4fc0 -6348> 2014-08-13 18:06:09.492055 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6347> 2014-08-13 18:06:09.492057 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6346> 2014-08-13 18:06:09.491933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing -6345> 2014-08-13 18:06:09.491945 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.4 -6344> 2014-08-13 18:06:09.492059 7f5c835ef700 20 osd.4 194497 _dispatch 0x5154a40 osd_map(194497..194497 src has 185139..194497) v3 -6343> 2014-08-13 18:06:09.492065 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6342> 2014-08-13 18:06:09.492071 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6341> 2014-08-13 18:06:09.492074 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6340> 2014-08-13 18:06:09.492076 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6339> 2014-08-13 18:06:09.492081 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.382 192.168.1.108:6800/30527 5 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5155e80 con 0x4fa4d00 -6338> 2014-08-13 18:06:09.492097 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6337> 2014-08-13 18:06:09.492099 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6336> 2014-08-13 18:06:09.492101 7f5c835ef700 20 osd.4 194497 _dispatch 0x5155e80 osd_map(194497..194497 src has 185139..194497) v3 -6335> 2014-08-13 18:06:09.492107 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6334> 2014-08-13 18:06:09.492110 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6333> 2014-08-13 18:06:09.492113 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6332> 2014-08-13 18:06:09.492115 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6331> 2014-08-13 18:06:09.492120 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.26 192.168.1.115:6802/58522 4 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cffbc0 con 0x3a7e9e0 -6330> 2014-08-13 18:06:09.492131 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6329> 2014-08-13 18:06:09.492134 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6328> 2014-08-13 18:06:09.492136 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cffbc0 osd_map(194497..194497 src has 185139..194497) v3 -6327> 2014-08-13 18:06:09.492141 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6326> 2014-08-13 18:06:09.492145 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6325> 2014-08-13 18:06:09.492147 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6324> 2014-08-13 18:06:09.492149 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6323> 2014-08-13 18:06:09.492155 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.307 192.168.1.109:6804/31016 4 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee5a00 con 0x43f3700 -6322> 2014-08-13 18:06:09.492167 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6321> 2014-08-13 18:06:09.492169 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6320> 2014-08-13 18:06:09.492171 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee5a00 osd_map(194497..194497 src has 185139..194497) v3 -6319> 2014-08-13 18:06:09.492176 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6318> 2014-08-13 18:06:09.492189 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6317> 2014-08-13 18:06:09.492192 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6316> 2014-08-13 18:06:09.492194 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6315> 2014-08-13 18:06:09.492065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.4 -6314> 2014-08-13 18:06:09.492065 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.4 -6313> 2014-08-13 18:06:09.492200 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.10 192.168.1.115:6815/60573 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc1200 con 0x43f0840 -6312> 2014-08-13 18:06:09.492212 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6311> 2014-08-13 18:06:09.492216 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6310> 2014-08-13 18:06:09.492218 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc1200 osd_map(194497..194497 src has 185139..194497) v3 -6309> 2014-08-13 18:06:09.492224 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6308> 2014-08-13 18:06:09.492228 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6307> 2014-08-13 18:06:09.492231 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6306> 2014-08-13 18:06:09.492233 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6305> 2014-08-13 18:06:09.492238 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.301 192.168.1.113:6814/16727 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee72c0 con 0x43f69e0 -6304> 2014-08-13 18:06:09.492250 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6303> 2014-08-13 18:06:09.492252 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6302> 2014-08-13 18:06:09.492254 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee72c0 osd_map(194497..194497 src has 185139..194497) v3 -6301> 2014-08-13 18:06:09.492259 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6300> 2014-08-13 18:06:09.492264 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6299> 2014-08-13 18:06:09.492266 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6298> 2014-08-13 18:06:09.492268 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6297> 2014-08-13 18:06:09.492273 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.58 192.168.1.111:6834/14316 12 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc2ac0 con 0x43f7900 -6296> 2014-08-13 18:06:09.492285 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6295> 2014-08-13 18:06:09.492287 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6294> 2014-08-13 18:06:09.492289 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc2ac0 osd_map(194497..194497 src has 185139..194497) v3 -6293> 2014-08-13 18:06:09.492295 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6292> 2014-08-13 18:06:09.492298 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6291> 2014-08-13 18:06:09.492300 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6290> 2014-08-13 18:06:09.492303 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6289> 2014-08-13 18:06:09.492308 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.54 192.168.1.111:6836/14923 10 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed2ac0 con 0x4425280 -6288> 2014-08-13 18:06:09.492320 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6287> 2014-08-13 18:06:09.492201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.4 -6286> 2014-08-13 18:06:09.492323 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6285> 2014-08-13 18:06:09.492325 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed2ac0 osd_map(194497..194497 src has 185139..194497) v3 -6284> 2014-08-13 18:06:09.492331 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6283> 2014-08-13 18:06:09.492335 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6282> 2014-08-13 18:06:09.492337 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6281> 2014-08-13 18:06:09.492340 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6280> 2014-08-13 18:06:09.492346 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.20 192.168.1.103:6815/38269 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5138d80 con 0x44261a0 -6279> 2014-08-13 18:06:09.492358 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6278> 2014-08-13 18:06:09.492360 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6277> 2014-08-13 18:06:09.492362 7f5c835ef700 20 osd.4 194497 _dispatch 0x5138d80 osd_map(194497..194497 src has 185139..194497) v3 -6276> 2014-08-13 18:06:09.492204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.4 -6275> 2014-08-13 18:06:09.492367 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6274> 2014-08-13 18:06:09.492372 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6273> 2014-08-13 18:06:09.492374 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6272> 2014-08-13 18:06:09.492376 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6271> 2014-08-13 18:06:09.492382 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.291 192.168.1.101:6820/25633 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee5c40 con 0x4424e60 -6270> 2014-08-13 18:06:09.492394 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6269> 2014-08-13 18:06:09.492398 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6268> 2014-08-13 18:06:09.492399 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee5c40 osd_map(194497..194497 src has 185139..194497) v3 -6267> 2014-08-13 18:06:09.492405 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6266> 2014-08-13 18:06:09.492409 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6265> 2014-08-13 18:06:09.492411 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6264> 2014-08-13 18:06:09.492413 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6263> 2014-08-13 18:06:09.492417 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.30 192.168.1.107:6823/55491 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ed1680 con 0x44586e0 -6262> 2014-08-13 18:06:09.492429 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6261> 2014-08-13 18:06:09.492432 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6260> 2014-08-13 18:06:09.492433 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ed1680 osd_map(194497..194497 src has 185139..194497) v3 -6259> 2014-08-13 18:06:09.492439 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6258> 2014-08-13 18:06:09.492443 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6257> 2014-08-13 18:06:09.492445 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6256> 2014-08-13 18:06:09.492326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.4 -6255> 2014-08-13 18:06:09.492447 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6254> 2014-08-13 18:06:09.492456 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.285 192.168.1.101:6822/26295 8 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfde80 con 0x445e5c0 -6253> 2014-08-13 18:06:09.492468 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6252> 2014-08-13 18:06:09.492471 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6251> 2014-08-13 18:06:09.492473 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfde80 osd_map(194497..194497 src has 185139..194497) v3 -6250> 2014-08-13 18:06:09.492478 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6249> 2014-08-13 18:06:09.492482 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6248> 2014-08-13 18:06:09.492484 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6247> 2014-08-13 18:06:09.492486 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6246> 2014-08-13 18:06:09.492374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.4 -6245> 2014-08-13 18:06:09.492491 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.61 192.168.1.111:6840/16566 13 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfe9c0 con 0x445b180 -6244> 2014-08-13 18:06:09.492504 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6243> 2014-08-13 18:06:09.492506 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6242> 2014-08-13 18:06:09.492508 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfe9c0 osd_map(194497..194497 src has 185139..194497) v3 -6241> 2014-08-13 18:06:09.492513 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6240> 2014-08-13 18:06:09.492518 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6239> 2014-08-13 18:06:09.492520 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6238> 2014-08-13 18:06:09.492522 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6237> 2014-08-13 18:06:09.492539 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.364 192.168.1.114:6826/52317 6 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee0b40 con 0x445ad60 -6236> 2014-08-13 18:06:09.492551 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6235> 2014-08-13 18:06:09.492554 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6234> 2014-08-13 18:06:09.492556 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee0b40 osd_map(194497..194497 src has 185139..194497) v3 -6233> 2014-08-13 18:06:09.492561 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6232> 2014-08-13 18:06:09.492565 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6231> 2014-08-13 18:06:09.492567 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6230> 2014-08-13 18:06:09.492451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.4 -6229> 2014-08-13 18:06:09.492569 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6228> 2014-08-13 18:06:09.492581 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.289 192.168.1.113:6808/15284 14 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc4ec0 con 0x445f7a0 -6227> 2014-08-13 18:06:09.492594 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6226> 2014-08-13 18:06:09.492596 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6225> 2014-08-13 18:06:09.492598 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4ec0 osd_map(194497..194497 src has 185139..194497) v3 -6224> 2014-08-13 18:06:09.492604 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6223> 2014-08-13 18:06:09.492607 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6222> 2014-08-13 18:06:09.492609 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6221> 2014-08-13 18:06:09.492612 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6220> 2014-08-13 18:06:09.492617 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 12 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x2002d00 con 0x4499080 -6219> 2014-08-13 18:06:09.492628 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6218> 2014-08-13 18:06:09.492631 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6217> 2014-08-13 18:06:09.492633 7f5c835ef700 20 osd.4 194497 _dispatch 0x2002d00 osd_map(194497..194497 src has 185139..194497) v3 -6216> 2014-08-13 18:06:09.492638 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -6215> 2014-08-13 18:06:09.492642 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -6214> 2014-08-13 18:06:09.492501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.4 -6213> 2014-08-13 18:06:09.492644 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6212> 2014-08-13 18:06:09.492647 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6211> 2014-08-13 18:06:09.492664 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.45 192.168.1.112:6828/6040 12 ==== pg_notify(22.3e4(36) epoch 194497) v5 ==== 2738+0+0 (1874354366 0 0) 0x524ee40 con 0x4496720 -6210> 2014-08-13 18:06:09.492678 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6209> 2014-08-13 18:06:09.492680 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6208> 2014-08-13 18:06:09.492682 7f5c835ef700 20 osd.4 194497 _dispatch 0x524ee40 pg_notify(22.3e4(36) epoch 194497) v5 -6207> 2014-08-13 18:06:09.492576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.4 -6206> 2014-08-13 18:06:09.492689 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.196521, event: header_read, request: pg_notify(22.3e4(36) epoch 194497) v5 -6205> 2014-08-13 18:06:09.492706 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.196523, event: throttled, request: pg_notify(22.3e4(36) epoch 194497) v5 -6204> 2014-08-13 18:06:09.492716 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.196712, event: all_read, request: pg_notify(22.3e4(36) epoch 194497) v5 -6203> 2014-08-13 18:06:09.492722 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.492677, event: dispatched, request: pg_notify(22.3e4(36) epoch 194497) v5 -6202> 2014-08-13 18:06:09.492730 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.492730, event: waiting_for_osdmap, request: pg_notify(22.3e4(36) epoch 194497) v5 -6201> 2014-08-13 18:06:09.492737 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.45 -6200> 2014-08-13 18:06:09.492741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524ee40 -6199> 2014-08-13 18:06:09.492747 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.492747, event: started, request: pg_notify(22.3e4(36) epoch 194497) v5 -6198> 2014-08-13 18:06:09.492648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.4 -6197> 2014-08-13 18:06:09.492776 7f5c835ef700 5 -- op tracker -- , seq: 554, time: 2014-08-13 18:06:09.492776, event: done, request: pg_notify(22.3e4(36) epoch 194497) v5 -6196> 2014-08-13 18:06:09.492809 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6195> 2014-08-13 18:06:09.492813 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6194> 2014-08-13 18:06:09.492698 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.4 -6193> 2014-08-13 18:06:09.492818 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.17 192.168.1.103:6811/37658 11 ==== pg_info(1 pgs e194497:24.2a1(4)) v4 ==== 968+0+0 (219976006 0 0) 0x4dea840 con 0x4493f40 -6192> 2014-08-13 18:06:09.492833 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6191> 2014-08-13 18:06:09.492836 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6190> 2014-08-13 18:06:09.492838 7f5c835ef700 20 osd.4 194497 _dispatch 0x4dea840 pg_info(1 pgs e194497:24.2a1(4)) v4 -6189> 2014-08-13 18:06:09.492844 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.196623, event: header_read, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6188> 2014-08-13 18:06:09.492851 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.196625, event: throttled, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6187> 2014-08-13 18:06:09.492856 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.196723, event: all_read, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6186> 2014-08-13 18:06:09.492862 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.492833, event: dispatched, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6185> 2014-08-13 18:06:09.492869 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.492869, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6184> 2014-08-13 18:06:09.492875 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2a1(4)) v4 from osd.17 -6183> 2014-08-13 18:06:09.492767 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.4 -6182> 2014-08-13 18:06:09.492881 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4dea840 -6181> 2014-08-13 18:06:09.492886 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.492886, event: started, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6180> 2014-08-13 18:06:09.492908 7f5c835ef700 5 -- op tracker -- , seq: 555, time: 2014-08-13 18:06:09.492907, event: done, request: pg_info(1 pgs e194497:24.2a1(4)) v4 -6179> 2014-08-13 18:06:09.492923 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6178> 2014-08-13 18:06:09.492927 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6177> 2014-08-13 18:06:09.492821 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.4 -6176> 2014-08-13 18:06:09.492932 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.44 192.168.1.111:6822/10782 9 ==== pg_notify(22.619(1) epoch 194497) v5 ==== 777+0+0 (3902026130 0 0) 0x5351880 con 0x449a3c0 -6175> 2014-08-13 18:06:09.492946 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6174> 2014-08-13 18:06:09.492948 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6173> 2014-08-13 18:06:09.492951 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351880 pg_notify(22.619(1) epoch 194497) v5 -6172> 2014-08-13 18:06:09.492956 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.310564, event: header_read, request: pg_notify(22.619(1) epoch 194497) v5 -6171> 2014-08-13 18:06:09.492966 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.310566, event: throttled, request: pg_notify(22.619(1) epoch 194497) v5 -6170> 2014-08-13 18:06:09.492972 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.310639, event: all_read, request: pg_notify(22.619(1) epoch 194497) v5 -6169> 2014-08-13 18:06:09.492978 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.492945, event: dispatched, request: pg_notify(22.619(1) epoch 194497) v5 -6168> 2014-08-13 18:06:09.492984 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.492984, event: waiting_for_osdmap, request: pg_notify(22.619(1) epoch 194497) v5 -6167> 2014-08-13 18:06:09.492990 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.44 -6166> 2014-08-13 18:06:09.492993 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351880 -6165> 2014-08-13 18:06:09.492997 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.492997, event: started, request: pg_notify(22.619(1) epoch 194497) v5 -6164> 2014-08-13 18:06:09.492883 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.4 -6163> 2014-08-13 18:06:09.493017 7f5c835ef700 5 -- op tracker -- , seq: 556, time: 2014-08-13 18:06:09.493017, event: done, request: pg_notify(22.619(1) epoch 194497) v5 -6162> 2014-08-13 18:06:09.493031 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6161> 2014-08-13 18:06:09.493034 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6160> 2014-08-13 18:06:09.493039 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 15 ==== pg_notify(24.136(36) epoch 194497) v5 ==== 2768+0+0 (2192713877 0 0) 0x588abc0 con 0x3c527e0 -6159> 2014-08-13 18:06:09.493053 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6158> 2014-08-13 18:06:09.493055 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6157> 2014-08-13 18:06:09.493057 7f5c835ef700 20 osd.4 194497 _dispatch 0x588abc0 pg_notify(24.136(36) epoch 194497) v5 -6156> 2014-08-13 18:06:09.492943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.4 -6155> 2014-08-13 18:06:09.493063 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.198411, event: header_read, request: pg_notify(24.136(36) epoch 194497) v5 -6154> 2014-08-13 18:06:09.493069 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.198416, event: throttled, request: pg_notify(24.136(36) epoch 194497) v5 -6153> 2014-08-13 18:06:09.493076 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.198565, event: all_read, request: pg_notify(24.136(36) epoch 194497) v5 -6152> 2014-08-13 18:06:09.493081 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.493052, event: dispatched, request: pg_notify(24.136(36) epoch 194497) v5 -6151> 2014-08-13 18:06:09.493088 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.493088, event: waiting_for_osdmap, request: pg_notify(24.136(36) epoch 194497) v5 -6150> 2014-08-13 18:06:09.493094 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.356 -6149> 2014-08-13 18:06:09.493097 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588abc0 -6148> 2014-08-13 18:06:09.493101 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.493101, event: started, request: pg_notify(24.136(36) epoch 194497) v5 -6147> 2014-08-13 18:06:09.493121 7f5c835ef700 5 -- op tracker -- , seq: 557, time: 2014-08-13 18:06:09.493121, event: done, request: pg_notify(24.136(36) epoch 194497) v5 -6146> 2014-08-13 18:06:09.493014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.4 -6145> 2014-08-13 18:06:09.493140 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6144> 2014-08-13 18:06:09.493143 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6143> 2014-08-13 18:06:09.493149 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.55 192.168.1.112:6834/6966 14 ==== pg_notify(22.113(34) epoch 194497) v5 ==== 2628+0+0 (2020322548 0 0) 0x5a6b800 con 0x3c523c0 -6142> 2014-08-13 18:06:09.493162 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6141> 2014-08-13 18:06:09.493165 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6140> 2014-08-13 18:06:09.493167 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b800 pg_notify(22.113(34) epoch 194497) v5 -6139> 2014-08-13 18:06:09.493171 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.224695, event: header_read, request: pg_notify(22.113(34) epoch 194497) v5 -6138> 2014-08-13 18:06:09.493186 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.224696, event: throttled, request: pg_notify(22.113(34) epoch 194497) v5 -6137> 2014-08-13 18:06:09.493070 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.4 -6136> 2014-08-13 18:06:09.493192 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.224863, event: all_read, request: pg_notify(22.113(34) epoch 194497) v5 -6135> 2014-08-13 18:06:09.493198 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.493161, event: dispatched, request: pg_notify(22.113(34) epoch 194497) v5 -6134> 2014-08-13 18:06:09.493205 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.493205, event: waiting_for_osdmap, request: pg_notify(22.113(34) epoch 194497) v5 -6133> 2014-08-13 18:06:09.493211 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.55 -6132> 2014-08-13 18:06:09.493215 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b800 -6131> 2014-08-13 18:06:09.493219 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.493219, event: started, request: pg_notify(22.113(34) epoch 194497) v5 -6130> 2014-08-13 18:06:09.493239 7f5c835ef700 5 -- op tracker -- , seq: 558, time: 2014-08-13 18:06:09.493239, event: done, request: pg_notify(22.113(34) epoch 194497) v5 -6129> 2014-08-13 18:06:09.493252 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6128> 2014-08-13 18:06:09.493255 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6127> 2014-08-13 18:06:09.493260 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.50 192.168.1.112:6842/9607 10 ==== pg_notify(24.136(2) epoch 194497) v5 ==== 870+0+0 (688238636 0 0) 0x4bf5cc0 con 0x3c56880 -6126> 2014-08-13 18:06:09.493273 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6125> 2014-08-13 18:06:09.493275 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6124> 2014-08-13 18:06:09.493277 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5cc0 pg_notify(24.136(2) epoch 194497) v5 -6123> 2014-08-13 18:06:09.493282 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.298793, event: header_read, request: pg_notify(24.136(2) epoch 194497) v5 -6122> 2014-08-13 18:06:09.493288 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.298794, event: throttled, request: pg_notify(24.136(2) epoch 194497) v5 -6121> 2014-08-13 18:06:09.493294 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.298855, event: all_read, request: pg_notify(24.136(2) epoch 194497) v5 -6120> 2014-08-13 18:06:09.493300 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.493272, event: dispatched, request: pg_notify(24.136(2) epoch 194497) v5 -6119> 2014-08-13 18:06:09.493309 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.493308, event: waiting_for_osdmap, request: pg_notify(24.136(2) epoch 194497) v5 -6118> 2014-08-13 18:06:09.493315 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.50 -6117> 2014-08-13 18:06:09.493199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.4 -6116> 2014-08-13 18:06:09.493319 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5cc0 -6115> 2014-08-13 18:06:09.493325 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.493325, event: started, request: pg_notify(24.136(2) epoch 194497) v5 -6114> 2014-08-13 18:06:09.493350 7f5c835ef700 5 -- op tracker -- , seq: 559, time: 2014-08-13 18:06:09.493350, event: done, request: pg_notify(24.136(2) epoch 194497) v5 -6113> 2014-08-13 18:06:09.493132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.4 -6112> 2014-08-13 18:06:09.493368 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6111> 2014-08-13 18:06:09.493372 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6110> 2014-08-13 18:06:09.493377 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 15 ==== pg_notify(22.99(30) epoch 194497) v5 ==== 2396+0+0 (3643774672 0 0) 0x5a38380 con 0x3c55540 -6109> 2014-08-13 18:06:09.493390 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6108> 2014-08-13 18:06:09.493392 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6107> 2014-08-13 18:06:09.493394 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38380 pg_notify(22.99(30) epoch 194497) v5 -6106> 2014-08-13 18:06:09.493400 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.312977, event: header_read, request: pg_notify(22.99(30) epoch 194497) v5 -6105> 2014-08-13 18:06:09.493406 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.312978, event: throttled, request: pg_notify(22.99(30) epoch 194497) v5 -6104> 2014-08-13 18:06:09.493412 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.313111, event: all_read, request: pg_notify(22.99(30) epoch 194497) v5 -6103> 2014-08-13 18:06:09.493417 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.493389, event: dispatched, request: pg_notify(22.99(30) epoch 194497) v5 -6102> 2014-08-13 18:06:09.493423 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.493423, event: waiting_for_osdmap, request: pg_notify(22.99(30) epoch 194497) v5 -6101> 2014-08-13 18:06:09.493429 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.337 -6100> 2014-08-13 18:06:09.493432 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38380 -6099> 2014-08-13 18:06:09.493436 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.493436, event: started, request: pg_notify(22.99(30) epoch 194497) v5 -6098> 2014-08-13 18:06:09.493323 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.4 -6097> 2014-08-13 18:06:09.493456 7f5c835ef700 5 -- op tracker -- , seq: 560, time: 2014-08-13 18:06:09.493456, event: done, request: pg_notify(22.99(30) epoch 194497) v5 -6096> 2014-08-13 18:06:09.493468 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6095> 2014-08-13 18:06:09.493472 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6094> 2014-08-13 18:06:09.493365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.4 -6093> 2014-08-13 18:06:09.493477 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.371 192.168.1.102:6832/16753 11 ==== pg_notify(22.4df(32) epoch 194497) v5 ==== 2510+0+0 (1160568909 0 0) 0x4df5940 con 0x4851e40 -6092> 2014-08-13 18:06:09.493490 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6091> 2014-08-13 18:06:09.493493 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6090> 2014-08-13 18:06:09.493495 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_notify(22.4df(32) epoch 194497) v5 -6089> 2014-08-13 18:06:09.493500 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.303948, event: header_read, request: pg_notify(22.4df(32) epoch 194497) v5 -6088> 2014-08-13 18:06:09.493507 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.303950, event: throttled, request: pg_notify(22.4df(32) epoch 194497) v5 -6087> 2014-08-13 18:06:09.493513 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.304069, event: all_read, request: pg_notify(22.4df(32) epoch 194497) v5 -6086> 2014-08-13 18:06:09.493519 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.493489, event: dispatched, request: pg_notify(22.4df(32) epoch 194497) v5 -6085> 2014-08-13 18:06:09.493525 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.493525, event: waiting_for_osdmap, request: pg_notify(22.4df(32) epoch 194497) v5 -6084> 2014-08-13 18:06:09.493530 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.371 -6083> 2014-08-13 18:06:09.493534 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 -6082> 2014-08-13 18:06:09.493538 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.493538, event: started, request: pg_notify(22.4df(32) epoch 194497) v5 -6081> 2014-08-13 18:06:09.493557 7f5c835ef700 5 -- op tracker -- , seq: 561, time: 2014-08-13 18:06:09.493557, event: done, request: pg_notify(22.4df(32) epoch 194497) v5 -6080> 2014-08-13 18:06:09.493450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.4 -6079> 2014-08-13 18:06:09.493570 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6078> 2014-08-13 18:06:09.493574 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6077> 2014-08-13 18:06:09.493579 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.343 192.168.1.104:6802/61960 13 ==== pg_info(1 pgs e194497:24.206(4)) v4 ==== 968+0+0 (986773228 0 0) 0x4df4440 con 0x48506e0 -6076> 2014-08-13 18:06:09.493593 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6075> 2014-08-13 18:06:09.493596 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6074> 2014-08-13 18:06:09.493598 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df4440 pg_info(1 pgs e194497:24.206(4)) v4 -6073> 2014-08-13 18:06:09.493603 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.209255, event: header_read, request: pg_info(1 pgs e194497:24.206(4)) v4 -6072> 2014-08-13 18:06:09.493609 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.209256, event: throttled, request: pg_info(1 pgs e194497:24.206(4)) v4 -6071> 2014-08-13 18:06:09.493615 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.209324, event: all_read, request: pg_info(1 pgs e194497:24.206(4)) v4 -6070> 2014-08-13 18:06:09.493488 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.4 -6069> 2014-08-13 18:06:09.493621 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.493593, event: dispatched, request: pg_info(1 pgs e194497:24.206(4)) v4 -6068> 2014-08-13 18:06:09.493628 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.493628, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.206(4)) v4 -6067> 2014-08-13 18:06:09.493634 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.206(4)) v4 from osd.343 -6066> 2014-08-13 18:06:09.493639 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df4440 -6065> 2014-08-13 18:06:09.493643 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.493643, event: started, request: pg_info(1 pgs e194497:24.206(4)) v4 -6064> 2014-08-13 18:06:09.493663 7f5c835ef700 5 -- op tracker -- , seq: 562, time: 2014-08-13 18:06:09.493663, event: done, request: pg_info(1 pgs e194497:24.206(4)) v4 -6063> 2014-08-13 18:06:09.493676 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6062> 2014-08-13 18:06:09.493679 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6061> 2014-08-13 18:06:09.493573 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.4 -6060> 2014-08-13 18:06:09.493684 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.296 192.168.1.109:6806/31286 5 ==== pg_notify(24.181(38) epoch 194497) v5 ==== 2882+0+0 (4153087757 0 0) 0x5888e00 con 0x4854a40 -6059> 2014-08-13 18:06:09.493698 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6058> 2014-08-13 18:06:09.493701 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6057> 2014-08-13 18:06:09.493703 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888e00 pg_notify(24.181(38) epoch 194497) v5 -6056> 2014-08-13 18:06:09.493708 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.296216, event: header_read, request: pg_notify(24.181(38) epoch 194497) v5 -6055> 2014-08-13 18:06:09.493716 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.296218, event: throttled, request: pg_notify(24.181(38) epoch 194497) v5 -6054> 2014-08-13 18:06:09.493721 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.296365, event: all_read, request: pg_notify(24.181(38) epoch 194497) v5 -6053> 2014-08-13 18:06:09.493727 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.493697, event: dispatched, request: pg_notify(24.181(38) epoch 194497) v5 -6052> 2014-08-13 18:06:09.493733 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.493733, event: waiting_for_osdmap, request: pg_notify(24.181(38) epoch 194497) v5 -6051> 2014-08-13 18:06:09.493739 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.296 -6050> 2014-08-13 18:06:09.493742 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888e00 -6049> 2014-08-13 18:06:09.493624 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.4 -6048> 2014-08-13 18:06:09.493746 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.493746, event: started, request: pg_notify(24.181(38) epoch 194497) v5 -6047> 2014-08-13 18:06:09.493767 7f5c835ef700 5 -- op tracker -- , seq: 563, time: 2014-08-13 18:06:09.493767, event: done, request: pg_notify(24.181(38) epoch 194497) v5 -6046> 2014-08-13 18:06:09.493799 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6045> 2014-08-13 18:06:09.493803 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6044> 2014-08-13 18:06:09.493696 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.4 -6043> 2014-08-13 18:06:09.493807 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 10 ==== pg_notify(22.707(38) epoch 194497) v5 ==== 2856+0+0 (428753354 0 0) 0x524d240 con 0x4854620 -6042> 2014-08-13 18:06:09.493821 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6041> 2014-08-13 18:06:09.493824 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6040> 2014-08-13 18:06:09.493826 7f5c835ef700 20 osd.4 194497 _dispatch 0x524d240 pg_notify(22.707(38) epoch 194497) v5 -6039> 2014-08-13 18:06:09.493832 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.198044, event: header_read, request: pg_notify(22.707(38) epoch 194497) v5 -6038> 2014-08-13 18:06:09.493839 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.198049, event: throttled, request: pg_notify(22.707(38) epoch 194497) v5 -6037> 2014-08-13 18:06:09.493845 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.198219, event: all_read, request: pg_notify(22.707(38) epoch 194497) v5 -6036> 2014-08-13 18:06:09.493850 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.493821, event: dispatched, request: pg_notify(22.707(38) epoch 194497) v5 -6035> 2014-08-13 18:06:09.493856 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.493856, event: waiting_for_osdmap, request: pg_notify(22.707(38) epoch 194497) v5 -6034> 2014-08-13 18:06:09.493862 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.15 -6033> 2014-08-13 18:06:09.493865 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524d240 -6032> 2014-08-13 18:06:09.493870 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.493869, event: started, request: pg_notify(22.707(38) epoch 194497) v5 -6031> 2014-08-13 18:06:09.493752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.4 -6030> 2014-08-13 18:06:09.493889 7f5c835ef700 5 -- op tracker -- , seq: 564, time: 2014-08-13 18:06:09.493888, event: done, request: pg_notify(22.707(38) epoch 194497) v5 -6029> 2014-08-13 18:06:09.493918 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6028> 2014-08-13 18:06:09.493921 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6027> 2014-08-13 18:06:09.493819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.4 -6026> 2014-08-13 18:06:09.493928 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.31 192.168.1.103:6831/41157 9 ==== pg_info(1 pgs e194497:24.1e4(25)) v4 ==== 2155+0+0 (2313966842 0 0) 0x5354980 con 0x4857bc0 -6025> 2014-08-13 18:06:09.493942 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6024> 2014-08-13 18:06:09.493944 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6023> 2014-08-13 18:06:09.493960 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354980 pg_info(1 pgs e194497:24.1e4(25)) v4 -6022> 2014-08-13 18:06:09.493966 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.198599, event: header_read, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6021> 2014-08-13 18:06:09.493973 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.198602, event: throttled, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6020> 2014-08-13 18:06:09.493979 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.198708, event: all_read, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6019> 2014-08-13 18:06:09.493985 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.493941, event: dispatched, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6018> 2014-08-13 18:06:09.493991 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.493991, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6017> 2014-08-13 18:06:09.493997 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e4(25)) v4 from osd.31 -6016> 2014-08-13 18:06:09.494002 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354980 -6015> 2014-08-13 18:06:09.494006 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.494006, event: started, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6014> 2014-08-13 18:06:09.494025 7f5c835ef700 5 -- op tracker -- , seq: 565, time: 2014-08-13 18:06:09.494025, event: done, request: pg_info(1 pgs e194497:24.1e4(25)) v4 -6013> 2014-08-13 18:06:09.494039 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6012> 2014-08-13 18:06:09.494042 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6011> 2014-08-13 18:06:09.494047 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.362 192.168.1.108:6842/8170 11 ==== pg_notify(22.362(34) epoch 194497) v5 ==== 2624+0+0 (251435578 0 0) 0x5352f40 con 0x49a53e0 -6010> 2014-08-13 18:06:09.493943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.4 -6009> 2014-08-13 18:06:09.494060 7f5c835ef700 10 osd.4 194497 do_waiters -- start -6008> 2014-08-13 18:06:09.494067 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -6007> 2014-08-13 18:06:09.494070 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352f40 pg_notify(22.362(34) epoch 194497) v5 -6006> 2014-08-13 18:06:09.493892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.4 -6005> 2014-08-13 18:06:09.494075 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.294002, event: header_read, request: pg_notify(22.362(34) epoch 194497) v5 -6004> 2014-08-13 18:06:09.494083 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.294003, event: throttled, request: pg_notify(22.362(34) epoch 194497) v5 -6003> 2014-08-13 18:06:09.494089 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.294125, event: all_read, request: pg_notify(22.362(34) epoch 194497) v5 -6002> 2014-08-13 18:06:09.494095 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.494060, event: dispatched, request: pg_notify(22.362(34) epoch 194497) v5 -6001> 2014-08-13 18:06:09.494102 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.494102, event: waiting_for_osdmap, request: pg_notify(22.362(34) epoch 194497) v5 -6000> 2014-08-13 18:06:09.494108 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.362 -5999> 2014-08-13 18:06:09.494111 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352f40 -5998> 2014-08-13 18:06:09.494115 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.494115, event: started, request: pg_notify(22.362(34) epoch 194497) v5 -5997> 2014-08-13 18:06:09.494135 7f5c835ef700 5 -- op tracker -- , seq: 566, time: 2014-08-13 18:06:09.494135, event: done, request: pg_notify(22.362(34) epoch 194497) v5 -5996> 2014-08-13 18:06:09.494147 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5995> 2014-08-13 18:06:09.494150 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5994> 2014-08-13 18:06:09.494156 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.304 192.168.1.101:6808/22782 11 ==== pg_info(1 pgs e194497:24.33a(4)) v4 ==== 968+0+0 (2835303997 0 0) 0x588ee40 con 0x4ab69e0 -5993> 2014-08-13 18:06:09.494169 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5992> 2014-08-13 18:06:09.494172 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5991> 2014-08-13 18:06:09.494173 7f5c835ef700 20 osd.4 194497 _dispatch 0x588ee40 pg_info(1 pgs e194497:24.33a(4)) v4 -5990> 2014-08-13 18:06:09.494066 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.4 -5989> 2014-08-13 18:06:09.494187 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.197226, event: header_read, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5988> 2014-08-13 18:06:09.494195 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.197227, event: throttled, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5987> 2014-08-13 18:06:09.494201 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.197330, event: all_read, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5986> 2014-08-13 18:06:09.494207 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.494169, event: dispatched, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5985> 2014-08-13 18:06:09.494219 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.494219, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5984> 2014-08-13 18:06:09.494081 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.4 -5983> 2014-08-13 18:06:09.494226 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.33a(4)) v4 from osd.304 -5982> 2014-08-13 18:06:09.494230 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588ee40 -5981> 2014-08-13 18:06:09.494235 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.494235, event: started, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5980> 2014-08-13 18:06:09.494255 7f5c835ef700 5 -- op tracker -- , seq: 567, time: 2014-08-13 18:06:09.494255, event: done, request: pg_info(1 pgs e194497:24.33a(4)) v4 -5979> 2014-08-13 18:06:09.494268 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5978> 2014-08-13 18:06:09.494272 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5977> 2014-08-13 18:06:09.494277 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.332 192.168.1.110:6806/42024 8 ==== pg_notify(24.1bb(2) epoch 194497) v5 ==== 870+0+0 (3395786314 0 0) 0x588cec0 con 0x4ab2d60 -5976> 2014-08-13 18:06:09.494291 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5975> 2014-08-13 18:06:09.494293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5974> 2014-08-13 18:06:09.494295 7f5c835ef700 20 osd.4 194497 _dispatch 0x588cec0 pg_notify(24.1bb(2) epoch 194497) v5 -5973> 2014-08-13 18:06:09.494300 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.400317, event: header_read, request: pg_notify(24.1bb(2) epoch 194497) v5 -5972> 2014-08-13 18:06:09.494307 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.400318, event: throttled, request: pg_notify(24.1bb(2) epoch 194497) v5 -5971> 2014-08-13 18:06:09.494193 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.4 -5970> 2014-08-13 18:06:09.494313 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.400398, event: all_read, request: pg_notify(24.1bb(2) epoch 194497) v5 -5969> 2014-08-13 18:06:09.494319 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.494290, event: dispatched, request: pg_notify(24.1bb(2) epoch 194497) v5 -5968> 2014-08-13 18:06:09.494326 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.494326, event: waiting_for_osdmap, request: pg_notify(24.1bb(2) epoch 194497) v5 -5967> 2014-08-13 18:06:09.494332 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.332 -5966> 2014-08-13 18:06:09.494336 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588cec0 -5965> 2014-08-13 18:06:09.494340 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.494340, event: started, request: pg_notify(24.1bb(2) epoch 194497) v5 -5964> 2014-08-13 18:06:09.494359 7f5c835ef700 5 -- op tracker -- , seq: 568, time: 2014-08-13 18:06:09.494359, event: done, request: pg_notify(24.1bb(2) epoch 194497) v5 -5963> 2014-08-13 18:06:09.494370 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5962> 2014-08-13 18:06:09.494373 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5961> 2014-08-13 18:06:09.494379 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.363 192.168.1.102:6842/20017 9 ==== pg_notify(22.99(1),22.39d(34) epoch 194497) v5 ==== 3389+0+0 (1010227695 0 0) 0x4a43100 con 0x4ab7640 -5960> 2014-08-13 18:06:09.494393 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5959> 2014-08-13 18:06:09.494396 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5958> 2014-08-13 18:06:09.494397 7f5c835ef700 20 osd.4 194497 _dispatch 0x4a43100 pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5957> 2014-08-13 18:06:09.494403 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.324595, event: header_read, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5956> 2014-08-13 18:06:09.494410 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.324602, event: throttled, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5955> 2014-08-13 18:06:09.494417 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.324724, event: all_read, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5954> 2014-08-13 18:06:09.494423 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.494393, event: dispatched, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5953> 2014-08-13 18:06:09.494318 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.4 -5952> 2014-08-13 18:06:09.494433 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.494433, event: waiting_for_osdmap, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5951> 2014-08-13 18:06:09.494440 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.363 -5950> 2014-08-13 18:06:09.494444 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4a43100 -5949> 2014-08-13 18:06:09.494448 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.494448, event: started, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5948> 2014-08-13 18:06:09.494480 7f5c835ef700 5 -- op tracker -- , seq: 569, time: 2014-08-13 18:06:09.494480, event: done, request: pg_notify(22.99(1),22.39d(34) epoch 194497) v5 -5947> 2014-08-13 18:06:09.494283 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.4 -5946> 2014-08-13 18:06:09.494497 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5945> 2014-08-13 18:06:09.494500 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5944> 2014-08-13 18:06:09.494506 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.374 192.168.1.102:6828/16129 9 ==== pg_info(1 pgs e194497:24.2ec(2)) v4 ==== 866+0+0 (2682968544 0 0) 0x53547c0 con 0x3cf4360 -5943> 2014-08-13 18:06:09.494520 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5942> 2014-08-13 18:06:09.494523 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5941> 2014-08-13 18:06:09.494525 7f5c835ef700 20 osd.4 194497 _dispatch 0x53547c0 pg_info(1 pgs e194497:24.2ec(2)) v4 -5940> 2014-08-13 18:06:09.494530 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.203290, event: header_read, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5939> 2014-08-13 18:06:09.494538 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.203291, event: throttled, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5938> 2014-08-13 18:06:09.494544 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.203358, event: all_read, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5937> 2014-08-13 18:06:09.494549 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.494520, event: dispatched, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5936> 2014-08-13 18:06:09.494441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.4 -5935> 2014-08-13 18:06:09.494556 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.494556, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5934> 2014-08-13 18:06:09.494562 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2ec(2)) v4 from osd.374 -5933> 2014-08-13 18:06:09.494567 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x53547c0 -5932> 2014-08-13 18:06:09.494572 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.494572, event: started, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5931> 2014-08-13 18:06:09.494592 7f5c835ef700 5 -- op tracker -- , seq: 570, time: 2014-08-13 18:06:09.494591, event: done, request: pg_info(1 pgs e194497:24.2ec(2)) v4 -5930> 2014-08-13 18:06:09.494605 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5929> 2014-08-13 18:06:09.494609 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5928> 2014-08-13 18:06:09.494493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.4 -5927> 2014-08-13 18:06:09.494614 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.24 192.168.1.107:6800/52669 7 ==== pg_notify(22.113(34) epoch 194497) v5 ==== 2628+0+0 (2020322548 0 0) 0x5a6fe00 con 0x4af9ce0 -5926> 2014-08-13 18:06:09.494627 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5925> 2014-08-13 18:06:09.494629 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5924> 2014-08-13 18:06:09.494632 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fe00 pg_notify(22.113(34) epoch 194497) v5 -5923> 2014-08-13 18:06:09.494637 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.346082, event: header_read, request: pg_notify(22.113(34) epoch 194497) v5 -5922> 2014-08-13 18:06:09.494643 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.346084, event: throttled, request: pg_notify(22.113(34) epoch 194497) v5 -5921> 2014-08-13 18:06:09.494649 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.346221, event: all_read, request: pg_notify(22.113(34) epoch 194497) v5 -5920> 2014-08-13 18:06:09.494655 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.494626, event: dispatched, request: pg_notify(22.113(34) epoch 194497) v5 -5919> 2014-08-13 18:06:09.494661 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.494661, event: waiting_for_osdmap, request: pg_notify(22.113(34) epoch 194497) v5 -5918> 2014-08-13 18:06:09.494667 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.24 -5917> 2014-08-13 18:06:09.494670 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fe00 -5916> 2014-08-13 18:06:09.494674 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.494674, event: started, request: pg_notify(22.113(34) epoch 194497) v5 -5915> 2014-08-13 18:06:09.494563 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.4 -5914> 2014-08-13 18:06:09.494694 7f5c835ef700 5 -- op tracker -- , seq: 571, time: 2014-08-13 18:06:09.494694, event: done, request: pg_notify(22.113(34) epoch 194497) v5 -5913> 2014-08-13 18:06:09.494710 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5912> 2014-08-13 18:06:09.494713 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5911> 2014-08-13 18:06:09.494718 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.6 192.168.1.107:6809/53141 8 ==== pg_info(1 pgs e194497:24.cd(4)) v4 ==== 968+0+0 (3126307362 0 0) 0x5357700 con 0x4af98c0 -5910> 2014-08-13 18:06:09.494731 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5909> 2014-08-13 18:06:09.494733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5908> 2014-08-13 18:06:09.494735 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357700 pg_info(1 pgs e194497:24.cd(4)) v4 -5907> 2014-08-13 18:06:09.494617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.4 -5906> 2014-08-13 18:06:09.494740 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.197804, event: header_read, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5905> 2014-08-13 18:06:09.494751 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.197806, event: throttled, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5904> 2014-08-13 18:06:09.494757 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.197876, event: all_read, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5903> 2014-08-13 18:06:09.494763 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.494730, event: dispatched, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5902> 2014-08-13 18:06:09.494769 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.494769, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5901> 2014-08-13 18:06:09.494775 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.cd(4)) v4 from osd.6 -5900> 2014-08-13 18:06:09.494779 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357700 -5899> 2014-08-13 18:06:09.494784 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.494783, event: started, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5898> 2014-08-13 18:06:09.494686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.4 -5897> 2014-08-13 18:06:09.494804 7f5c835ef700 5 -- op tracker -- , seq: 572, time: 2014-08-13 18:06:09.494804, event: done, request: pg_info(1 pgs e194497:24.cd(4)) v4 -5896> 2014-08-13 18:06:09.494821 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5895> 2014-08-13 18:06:09.494824 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5894> 2014-08-13 18:06:09.494830 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.344 192.168.1.106:6812/6757 8 ==== pg_notify(24.181(38) epoch 194497) v5 ==== 2882+0+0 (1137644042 0 0) 0x5351c00 con 0x4aff220 -5893> 2014-08-13 18:06:09.494843 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5892> 2014-08-13 18:06:09.494845 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5891> 2014-08-13 18:06:09.494847 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351c00 pg_notify(24.181(38) epoch 194497) v5 -5890> 2014-08-13 18:06:09.494852 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.284359, event: header_read, request: pg_notify(24.181(38) epoch 194497) v5 -5889> 2014-08-13 18:06:09.494859 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.284361, event: throttled, request: pg_notify(24.181(38) epoch 194497) v5 -5888> 2014-08-13 18:06:09.494865 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.284482, event: all_read, request: pg_notify(24.181(38) epoch 194497) v5 -5887> 2014-08-13 18:06:09.494870 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.494842, event: dispatched, request: pg_notify(24.181(38) epoch 194497) v5 -5886> 2014-08-13 18:06:09.494877 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.494876, event: waiting_for_osdmap, request: pg_notify(24.181(38) epoch 194497) v5 -5885> 2014-08-13 18:06:09.494882 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.344 -5884> 2014-08-13 18:06:09.494885 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351c00 -5883> 2014-08-13 18:06:09.494890 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.494889, event: started, request: pg_notify(24.181(38) epoch 194497) v5 -5882> 2014-08-13 18:06:09.494752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.4 -5881> 2014-08-13 18:06:09.494915 7f5c835ef700 5 -- op tracker -- , seq: 573, time: 2014-08-13 18:06:09.494914, event: done, request: pg_notify(24.181(38) epoch 194497) v5 -5880> 2014-08-13 18:06:09.494808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.4 -5879> 2014-08-13 18:06:09.494940 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5878> 2014-08-13 18:06:09.494944 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5877> 2014-08-13 18:06:09.494949 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 17 ==== pg_notify(22.707(1) epoch 194497) v5 ==== 777+0+0 (2895295546 0 0) 0x5a38c40 con 0x4af8580 -5876> 2014-08-13 18:06:09.494962 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5875> 2014-08-13 18:06:09.494964 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5874> 2014-08-13 18:06:09.494966 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38c40 pg_notify(22.707(1) epoch 194497) v5 -5873> 2014-08-13 18:06:09.494971 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.293447, event: header_read, request: pg_notify(22.707(1) epoch 194497) v5 -5872> 2014-08-13 18:06:09.494978 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.293449, event: throttled, request: pg_notify(22.707(1) epoch 194497) v5 -5871> 2014-08-13 18:06:09.494984 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.293517, event: all_read, request: pg_notify(22.707(1) epoch 194497) v5 -5870> 2014-08-13 18:06:09.494989 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.494961, event: dispatched, request: pg_notify(22.707(1) epoch 194497) v5 -5869> 2014-08-13 18:06:09.494995 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.494995, event: waiting_for_osdmap, request: pg_notify(22.707(1) epoch 194497) v5 -5868> 2014-08-13 18:06:09.495001 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 -5867> 2014-08-13 18:06:09.495004 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38c40 -5866> 2014-08-13 18:06:09.495008 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.495008, event: started, request: pg_notify(22.707(1) epoch 194497) v5 -5865> 2014-08-13 18:06:09.495027 7f5c835ef700 5 -- op tracker -- , seq: 574, time: 2014-08-13 18:06:09.495027, event: done, request: pg_notify(22.707(1) epoch 194497) v5 -5864> 2014-08-13 18:06:09.494929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.4 -5863> 2014-08-13 18:06:09.495050 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5862> 2014-08-13 18:06:09.495054 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5861> 2014-08-13 18:06:09.494904 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.4 -5860> 2014-08-13 18:06:09.495059 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.351 192.168.1.102:6830/16444 12 ==== pg_notify(24.1f7(44) epoch 194497) v5 ==== 3240+0+0 (1675219729 0 0) 0x5357e00 con 0x4af8160 -5859> 2014-08-13 18:06:09.495072 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5858> 2014-08-13 18:06:09.495075 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5857> 2014-08-13 18:06:09.495077 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357e00 pg_notify(24.1f7(44) epoch 194497) v5 -5856> 2014-08-13 18:06:09.495083 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.216779, event: header_read, request: pg_notify(24.1f7(44) epoch 194497) v5 -5855> 2014-08-13 18:06:09.495089 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.216781, event: throttled, request: pg_notify(24.1f7(44) epoch 194497) v5 -5854> 2014-08-13 18:06:09.495095 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.216918, event: all_read, request: pg_notify(24.1f7(44) epoch 194497) v5 -5853> 2014-08-13 18:06:09.495100 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.495072, event: dispatched, request: pg_notify(24.1f7(44) epoch 194497) v5 -5852> 2014-08-13 18:06:09.495107 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.495106, event: waiting_for_osdmap, request: pg_notify(24.1f7(44) epoch 194497) v5 -5851> 2014-08-13 18:06:09.495127 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.351 -5850> 2014-08-13 18:06:09.495130 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357e00 -5849> 2014-08-13 18:06:09.495135 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.495134, event: started, request: pg_notify(24.1f7(44) epoch 194497) v5 -5848> 2014-08-13 18:06:09.495154 7f5c835ef700 5 -- op tracker -- , seq: 575, time: 2014-08-13 18:06:09.495154, event: done, request: pg_notify(24.1f7(44) epoch 194497) v5 -5847> 2014-08-13 18:06:09.495051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.4 -5846> 2014-08-13 18:06:09.495184 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5845> 2014-08-13 18:06:09.495188 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5844> 2014-08-13 18:06:09.495194 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.331 192.168.1.104:6816/837 9 ==== pg_notify(22.1d2(1) epoch 194497) v5 ==== 777+0+0 (2799123809 0 0) 0x588ec80 con 0x4b12940 -5843> 2014-08-13 18:06:09.495208 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5842> 2014-08-13 18:06:09.495211 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5841> 2014-08-13 18:06:09.495213 7f5c835ef700 20 osd.4 194497 _dispatch 0x588ec80 pg_notify(22.1d2(1) epoch 194497) v5 -5840> 2014-08-13 18:06:09.495218 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.378927, event: header_read, request: pg_notify(22.1d2(1) epoch 194497) v5 -5839> 2014-08-13 18:06:09.495225 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.378928, event: throttled, request: pg_notify(22.1d2(1) epoch 194497) v5 -5838> 2014-08-13 18:06:09.495230 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.379005, event: all_read, request: pg_notify(22.1d2(1) epoch 194497) v5 -5837> 2014-08-13 18:06:09.495236 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.495208, event: dispatched, request: pg_notify(22.1d2(1) epoch 194497) v5 -5836> 2014-08-13 18:06:09.495242 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.495242, event: waiting_for_osdmap, request: pg_notify(22.1d2(1) epoch 194497) v5 -5835> 2014-08-13 18:06:09.495248 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.331 -5834> 2014-08-13 18:06:09.495251 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588ec80 -5833> 2014-08-13 18:06:09.495255 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.495255, event: started, request: pg_notify(22.1d2(1) epoch 194497) v5 -5832> 2014-08-13 18:06:09.495275 7f5c835ef700 5 -- op tracker -- , seq: 576, time: 2014-08-13 18:06:09.495274, event: done, request: pg_notify(22.1d2(1) epoch 194497) v5 -5831> 2014-08-13 18:06:09.495285 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5830> 2014-08-13 18:06:09.495288 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5829> 2014-08-13 18:06:09.495174 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.4 -5828> 2014-08-13 18:06:09.495294 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.320 192.168.1.106:6820/10050 14 ==== pg_notify(22.39d(34) epoch 194497) v5 ==== 2624+0+0 (751407494 0 0) 0x4bf0fc0 con 0x4b12520 -5827> 2014-08-13 18:06:09.495307 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5826> 2014-08-13 18:06:09.495310 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5825> 2014-08-13 18:06:09.495312 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf0fc0 pg_notify(22.39d(34) epoch 194497) v5 -5824> 2014-08-13 18:06:09.495317 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.217806, event: header_read, request: pg_notify(22.39d(34) epoch 194497) v5 -5823> 2014-08-13 18:06:09.495324 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.217810, event: throttled, request: pg_notify(22.39d(34) epoch 194497) v5 -5822> 2014-08-13 18:06:09.495329 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.217996, event: all_read, request: pg_notify(22.39d(34) epoch 194497) v5 -5821> 2014-08-13 18:06:09.495335 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.495307, event: dispatched, request: pg_notify(22.39d(34) epoch 194497) v5 -5820> 2014-08-13 18:06:09.495060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.4 -5819> 2014-08-13 18:06:09.495341 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.495341, event: waiting_for_osdmap, request: pg_notify(22.39d(34) epoch 194497) v5 -5818> 2014-08-13 18:06:09.495347 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.320 -5817> 2014-08-13 18:06:09.495350 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf0fc0 -5816> 2014-08-13 18:06:09.495355 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.495355, event: started, request: pg_notify(22.39d(34) epoch 194497) v5 -5815> 2014-08-13 18:06:09.495374 7f5c835ef700 5 -- op tracker -- , seq: 577, time: 2014-08-13 18:06:09.495374, event: done, request: pg_notify(22.39d(34) epoch 194497) v5 -5814> 2014-08-13 18:06:09.495392 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5813> 2014-08-13 18:06:09.495395 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5812> 2014-08-13 18:06:09.495400 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.367 192.168.1.114:6840/55513 10 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (1390037278 0 0) 0x59e3700 con 0x4b11ce0 -5811> 2014-08-13 18:06:09.495419 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5810> 2014-08-13 18:06:09.495301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.4 -5809> 2014-08-13 18:06:09.495421 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5808> 2014-08-13 18:06:09.495423 7f5c835ef700 20 osd.4 194497 _dispatch 0x59e3700 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5807> 2014-08-13 18:06:09.495433 7f5c835ef700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.489351, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5806> 2014-08-13 18:06:09.495444 7f5c835ef700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.489352, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5805> 2014-08-13 18:06:09.495458 7f5c835ef700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.489434, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5804> 2014-08-13 18:06:09.495467 7f5c835ef700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.495418, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5803> 2014-08-13 18:06:09.495477 7f5c835ef700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.495477, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5802> 2014-08-13 18:06:09.495487 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 -5801> 2014-08-13 18:06:09.495348 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.4 -5800> 2014-08-13 18:06:09.495496 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x59e3700 -5799> 2014-08-13 18:06:09.495500 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.367 192.168.1.114:6840/55513 194497 -5798> 2014-08-13 18:06:09.495511 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a5a00 prio 127 cost 0 latency 0.006159 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -5797> 2014-08-13 18:06:09.495524 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5796> 2014-08-13 18:06:09.495527 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5795> 2014-08-13 18:06:09.495532 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.65 192.168.1.111:6842/17345 10 ==== pg_notify(24.136(36) epoch 194497) v5 ==== 2768+0+0 (2192713877 0 0) 0x588b640 con 0x4b11600 -5794> 2014-08-13 18:06:09.495424 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.4 -5793> 2014-08-13 18:06:09.495544 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5792> 2014-08-13 18:06:09.495547 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5791> 2014-08-13 18:06:09.495550 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b640 pg_notify(24.136(36) epoch 194497) v5 -5790> 2014-08-13 18:06:09.495555 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.221135, event: header_read, request: pg_notify(24.136(36) epoch 194497) v5 -5789> 2014-08-13 18:06:09.495562 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.221136, event: throttled, request: pg_notify(24.136(36) epoch 194497) v5 -5788> 2014-08-13 18:06:09.495568 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.221281, event: all_read, request: pg_notify(24.136(36) epoch 194497) v5 -5787> 2014-08-13 18:06:09.495573 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.495544, event: dispatched, request: pg_notify(24.136(36) epoch 194497) v5 -5786> 2014-08-13 18:06:09.495580 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.495579, event: waiting_for_osdmap, request: pg_notify(24.136(36) epoch 194497) v5 -5785> 2014-08-13 18:06:09.495585 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.65 -5784> 2014-08-13 18:06:09.495588 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b640 -5783> 2014-08-13 18:06:09.495593 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.495593, event: started, request: pg_notify(24.136(36) epoch 194497) v5 -5782> 2014-08-13 18:06:09.495612 7f5c835ef700 5 -- op tracker -- , seq: 579, time: 2014-08-13 18:06:09.495612, event: done, request: pg_notify(24.136(36) epoch 194497) v5 -5781> 2014-08-13 18:06:09.495637 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5780> 2014-08-13 18:06:09.495500 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.4 -5779> 2014-08-13 18:06:09.495640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5778> 2014-08-13 18:06:09.495646 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.373 192.168.1.108:6840/7435 17 ==== pg_notify(24.3d(38) epoch 194497) v5 ==== 2890+0+0 (2519417228 0 0) 0x5353800 con 0x4b111e0 -5777> 2014-08-13 18:06:09.495659 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5776> 2014-08-13 18:06:09.495662 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5775> 2014-08-13 18:06:09.495546 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.4 -5774> 2014-08-13 18:06:09.495664 7f5c835ef700 20 osd.4 194497 _dispatch 0x5353800 pg_notify(24.3d(38) epoch 194497) v5 -5773> 2014-08-13 18:06:09.495669 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.244101, event: header_read, request: pg_notify(24.3d(38) epoch 194497) v5 -5772> 2014-08-13 18:06:09.495677 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.244103, event: throttled, request: pg_notify(24.3d(38) epoch 194497) v5 -5771> 2014-08-13 18:06:09.495682 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.244287, event: all_read, request: pg_notify(24.3d(38) epoch 194497) v5 -5770> 2014-08-13 18:06:09.495688 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.495658, event: dispatched, request: pg_notify(24.3d(38) epoch 194497) v5 -5769> 2014-08-13 18:06:09.495695 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.495694, event: waiting_for_osdmap, request: pg_notify(24.3d(38) epoch 194497) v5 -5768> 2014-08-13 18:06:09.495700 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.373 -5767> 2014-08-13 18:06:09.495703 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5353800 -5766> 2014-08-13 18:06:09.495707 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.495707, event: started, request: pg_notify(24.3d(38) epoch 194497) v5 -5765> 2014-08-13 18:06:09.495727 7f5c835ef700 5 -- op tracker -- , seq: 580, time: 2014-08-13 18:06:09.495727, event: done, request: pg_notify(24.3d(38) epoch 194497) v5 -5764> 2014-08-13 18:06:09.495739 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5763> 2014-08-13 18:06:09.495741 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5762> 2014-08-13 18:06:09.495746 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 13 ==== pg_notify(22.3d9(21) epoch 194497) v5 ==== 1905+0+0 (3181573404 0 0) 0x5a6c0c0 con 0x4b10b00 -5761> 2014-08-13 18:06:09.495759 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5760> 2014-08-13 18:06:09.495762 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5759> 2014-08-13 18:06:09.495763 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c0c0 pg_notify(22.3d9(21) epoch 194497) v5 -5758> 2014-08-13 18:06:09.495643 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.4 -5757> 2014-08-13 18:06:09.495768 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.229442, event: header_read, request: pg_notify(22.3d9(21) epoch 194497) v5 -5756> 2014-08-13 18:06:09.495775 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.229443, event: throttled, request: pg_notify(22.3d9(21) epoch 194497) v5 -5755> 2014-08-13 18:06:09.495781 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.229543, event: all_read, request: pg_notify(22.3d9(21) epoch 194497) v5 -5754> 2014-08-13 18:06:09.495669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.4 -5753> 2014-08-13 18:06:09.495787 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.495759, event: dispatched, request: pg_notify(22.3d9(21) epoch 194497) v5 -5752> 2014-08-13 18:06:09.495794 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.495794, event: waiting_for_osdmap, request: pg_notify(22.3d9(21) epoch 194497) v5 -5751> 2014-08-13 18:06:09.495800 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.397 -5750> 2014-08-13 18:06:09.495803 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c0c0 -5749> 2014-08-13 18:06:09.495809 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.495809, event: started, request: pg_notify(22.3d9(21) epoch 194497) v5 -5748> 2014-08-13 18:06:09.495828 7f5c835ef700 5 -- op tracker -- , seq: 581, time: 2014-08-13 18:06:09.495828, event: done, request: pg_notify(22.3d9(21) epoch 194497) v5 -5747> 2014-08-13 18:06:09.495845 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5746> 2014-08-13 18:06:09.495849 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5745> 2014-08-13 18:06:09.495853 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.396 192.168.1.105:6814/41806 11 ==== pg_notify(22.3d9(50) epoch 194497) v5 ==== 3448+0+0 (858632148 0 0) 0x5356580 con 0x4b2c620 -5744> 2014-08-13 18:06:09.495866 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5743> 2014-08-13 18:06:09.495869 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5742> 2014-08-13 18:06:09.495871 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356580 pg_notify(22.3d9(50) epoch 194497) v5 -5741> 2014-08-13 18:06:09.495875 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.303316, event: header_read, request: pg_notify(22.3d9(50) epoch 194497) v5 -5740> 2014-08-13 18:06:09.495882 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.303317, event: throttled, request: pg_notify(22.3d9(50) epoch 194497) v5 -5739> 2014-08-13 18:06:09.495888 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.303451, event: all_read, request: pg_notify(22.3d9(50) epoch 194497) v5 -5738> 2014-08-13 18:06:09.495893 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.495866, event: dispatched, request: pg_notify(22.3d9(50) epoch 194497) v5 -5737> 2014-08-13 18:06:09.495899 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.495899, event: waiting_for_osdmap, request: pg_notify(22.3d9(50) epoch 194497) v5 -5736> 2014-08-13 18:06:09.495905 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.396 -5735> 2014-08-13 18:06:09.495908 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356580 -5734> 2014-08-13 18:06:09.495792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.4 -5733> 2014-08-13 18:06:09.495776 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.4 -5732> 2014-08-13 18:06:09.495912 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.495912, event: started, request: pg_notify(22.3d9(50) epoch 194497) v5 -5731> 2014-08-13 18:06:09.495934 7f5c835ef700 5 -- op tracker -- , seq: 582, time: 2014-08-13 18:06:09.495934, event: done, request: pg_notify(22.3d9(50) epoch 194497) v5 -5730> 2014-08-13 18:06:09.495944 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5729> 2014-08-13 18:06:09.495947 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5728> 2014-08-13 18:06:09.495971 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 20 ==== pg_notify(24.1f7(40) epoch 194497) v5 ==== 3004+0+0 (3148095910 0 0) 0x5a696c0 con 0x4b2bc80 -5727> 2014-08-13 18:06:09.495985 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5726> 2014-08-13 18:06:09.495987 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5725> 2014-08-13 18:06:09.495990 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a696c0 pg_notify(24.1f7(40) epoch 194497) v5 -5724> 2014-08-13 18:06:09.495995 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.216457, event: header_read, request: pg_notify(24.1f7(40) epoch 194497) v5 -5723> 2014-08-13 18:06:09.496002 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.216458, event: throttled, request: pg_notify(24.1f7(40) epoch 194497) v5 -5722> 2014-08-13 18:06:09.496007 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.216610, event: all_read, request: pg_notify(24.1f7(40) epoch 194497) v5 -5721> 2014-08-13 18:06:09.496013 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.495984, event: dispatched, request: pg_notify(24.1f7(40) epoch 194497) v5 -5720> 2014-08-13 18:06:09.496019 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.496019, event: waiting_for_osdmap, request: pg_notify(24.1f7(40) epoch 194497) v5 -5719> 2014-08-13 18:06:09.496025 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.376 -5718> 2014-08-13 18:06:09.496028 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a696c0 -5717> 2014-08-13 18:06:09.495914 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.4 -5716> 2014-08-13 18:06:09.496032 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.496032, event: started, request: pg_notify(24.1f7(40) epoch 194497) v5 -5715> 2014-08-13 18:06:09.495919 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.4 -5714> 2014-08-13 18:06:09.496053 7f5c835ef700 5 -- op tracker -- , seq: 583, time: 2014-08-13 18:06:09.496052, event: done, request: pg_notify(24.1f7(40) epoch 194497) v5 -5713> 2014-08-13 18:06:09.496066 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5712> 2014-08-13 18:06:09.496069 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5711> 2014-08-13 18:06:09.496073 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.379 192.168.1.108:6830/4861 8 ==== pg_notify(24.1bb(40) epoch 194497) v5 ==== 3004+0+0 (2719403443 0 0) 0x5356040 con 0x4b2d120 -5710> 2014-08-13 18:06:09.496086 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5709> 2014-08-13 18:06:09.496089 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5708> 2014-08-13 18:06:09.496091 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356040 pg_notify(24.1bb(40) epoch 194497) v5 -5707> 2014-08-13 18:06:09.496095 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.309213, event: header_read, request: pg_notify(24.1bb(40) epoch 194497) v5 -5706> 2014-08-13 18:06:09.496102 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.309214, event: throttled, request: pg_notify(24.1bb(40) epoch 194497) v5 -5705> 2014-08-13 18:06:09.496108 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.309351, event: all_read, request: pg_notify(24.1bb(40) epoch 194497) v5 -5704> 2014-08-13 18:06:09.496113 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.496086, event: dispatched, request: pg_notify(24.1bb(40) epoch 194497) v5 -5703> 2014-08-13 18:06:09.496119 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.496119, event: waiting_for_osdmap, request: pg_notify(24.1bb(40) epoch 194497) v5 -5702> 2014-08-13 18:06:09.496125 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.379 -5701> 2014-08-13 18:06:09.496128 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356040 -5700> 2014-08-13 18:06:09.496132 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.496132, event: started, request: pg_notify(24.1bb(40) epoch 194497) v5 -5699> 2014-08-13 18:06:09.496037 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.4 -5698> 2014-08-13 18:06:09.496154 7f5c835ef700 5 -- op tracker -- , seq: 584, time: 2014-08-13 18:06:09.496153, event: done, request: pg_notify(24.1bb(40) epoch 194497) v5 -5697> 2014-08-13 18:06:09.496164 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5696> 2014-08-13 18:06:09.496167 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5695> 2014-08-13 18:06:09.496046 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.4 -5694> 2014-08-13 18:06:09.496200 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.0 192.168.1.115:6810/59615 7 ==== pg_notify(22.775(36) epoch 194497) v5 ==== 2738+0+0 (362354066 0 0) 0x5a6ee40 con 0x4b2f4e0 -5693> 2014-08-13 18:06:09.496216 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5692> 2014-08-13 18:06:09.496219 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5691> 2014-08-13 18:06:09.496221 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6ee40 pg_notify(22.775(36) epoch 194497) v5 -5690> 2014-08-13 18:06:09.496226 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.306410, event: header_read, request: pg_notify(22.775(36) epoch 194497) v5 -5689> 2014-08-13 18:06:09.496233 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.306412, event: throttled, request: pg_notify(22.775(36) epoch 194497) v5 -5688> 2014-08-13 18:06:09.496239 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.306535, event: all_read, request: pg_notify(22.775(36) epoch 194497) v5 -5687> 2014-08-13 18:06:09.496244 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.496215, event: dispatched, request: pg_notify(22.775(36) epoch 194497) v5 -5686> 2014-08-13 18:06:09.496250 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.496250, event: waiting_for_osdmap, request: pg_notify(22.775(36) epoch 194497) v5 -5685> 2014-08-13 18:06:09.496256 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.0 -5684> 2014-08-13 18:06:09.496259 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6ee40 -5683> 2014-08-13 18:06:09.496264 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.496263, event: started, request: pg_notify(22.775(36) epoch 194497) v5 -5682> 2014-08-13 18:06:09.496159 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.4 -5681> 2014-08-13 18:06:09.496284 7f5c835ef700 5 -- op tracker -- , seq: 585, time: 2014-08-13 18:06:09.496283, event: done, request: pg_notify(22.775(36) epoch 194497) v5 -5680> 2014-08-13 18:06:09.496295 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5679> 2014-08-13 18:06:09.496298 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5678> 2014-08-13 18:06:09.496192 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.4 -5677> 2014-08-13 18:06:09.496317 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.11 192.168.1.103:6813/37929 7 ==== pg_notify(22.1f1(1) epoch 194497) v5 ==== 777+0+0 (2081487949 0 0) 0x5a6bd40 con 0x4b2f640 -5676> 2014-08-13 18:06:09.496348 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5675> 2014-08-13 18:06:09.496351 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5674> 2014-08-13 18:06:09.496353 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6bd40 pg_notify(22.1f1(1) epoch 194497) v5 -5673> 2014-08-13 18:06:09.496358 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.224922, event: header_read, request: pg_notify(22.1f1(1) epoch 194497) v5 -5672> 2014-08-13 18:06:09.496365 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.224923, event: throttled, request: pg_notify(22.1f1(1) epoch 194497) v5 -5671> 2014-08-13 18:06:09.496371 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.224992, event: all_read, request: pg_notify(22.1f1(1) epoch 194497) v5 -5670> 2014-08-13 18:06:09.496376 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.496347, event: dispatched, request: pg_notify(22.1f1(1) epoch 194497) v5 -5669> 2014-08-13 18:06:09.496382 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.496382, event: waiting_for_osdmap, request: pg_notify(22.1f1(1) epoch 194497) v5 -5668> 2014-08-13 18:06:09.496388 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.11 -5667> 2014-08-13 18:06:09.496391 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6bd40 -5666> 2014-08-13 18:06:09.496396 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.496395, event: started, request: pg_notify(22.1f1(1) epoch 194497) v5 -5665> 2014-08-13 18:06:09.496286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.4 -5664> 2014-08-13 18:06:09.496416 7f5c835ef700 5 -- op tracker -- , seq: 586, time: 2014-08-13 18:06:09.496416, event: done, request: pg_notify(22.1f1(1) epoch 194497) v5 -5663> 2014-08-13 18:06:09.496436 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5662> 2014-08-13 18:06:09.496439 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5661> 2014-08-13 18:06:09.496310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.4 -5660> 2014-08-13 18:06:09.496445 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.352 192.168.1.108:6826/4273 7 ==== pg_notify(22.4df(1) epoch 194497) v5 ==== 777+0+0 (2787782931 0 0) 0x5a6e900 con 0x4b2fbc0 -5659> 2014-08-13 18:06:09.496458 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5658> 2014-08-13 18:06:09.496461 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5657> 2014-08-13 18:06:09.496463 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e900 pg_notify(22.4df(1) epoch 194497) v5 -5656> 2014-08-13 18:06:09.496468 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.304213, event: header_read, request: pg_notify(22.4df(1) epoch 194497) v5 -5655> 2014-08-13 18:06:09.496475 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.304214, event: throttled, request: pg_notify(22.4df(1) epoch 194497) v5 -5654> 2014-08-13 18:06:09.496481 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.304273, event: all_read, request: pg_notify(22.4df(1) epoch 194497) v5 -5653> 2014-08-13 18:06:09.496486 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.496458, event: dispatched, request: pg_notify(22.4df(1) epoch 194497) v5 -5652> 2014-08-13 18:06:09.496492 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.496492, event: waiting_for_osdmap, request: pg_notify(22.4df(1) epoch 194497) v5 -5651> 2014-08-13 18:06:09.496498 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.352 -5650> 2014-08-13 18:06:09.496501 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e900 -5649> 2014-08-13 18:06:09.496505 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.496505, event: started, request: pg_notify(22.4df(1) epoch 194497) v5 -5648> 2014-08-13 18:06:09.496409 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.4 -5647> 2014-08-13 18:06:09.496525 7f5c835ef700 5 -- op tracker -- , seq: 587, time: 2014-08-13 18:06:09.496524, event: done, request: pg_notify(22.4df(1) epoch 194497) v5 -5646> 2014-08-13 18:06:09.496538 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5645> 2014-08-13 18:06:09.496541 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5644> 2014-08-13 18:06:09.496546 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.327 192.168.1.106:6804/5112 11 ==== pg_notify(22.48e(34) epoch 194497) v5 ==== 2624+0+0 (2407140341 0 0) 0x5352a00 con 0x4c58420 -5643> 2014-08-13 18:06:09.496558 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5642> 2014-08-13 18:06:09.496561 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5641> 2014-08-13 18:06:09.496563 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352a00 pg_notify(22.48e(34) epoch 194497) v5 -5640> 2014-08-13 18:06:09.496568 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.229083, event: header_read, request: pg_notify(22.48e(34) epoch 194497) v5 -5639> 2014-08-13 18:06:09.496451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.4 -5638> 2014-08-13 18:06:09.496574 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.229085, event: throttled, request: pg_notify(22.48e(34) epoch 194497) v5 -5637> 2014-08-13 18:06:09.496581 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.229254, event: all_read, request: pg_notify(22.48e(34) epoch 194497) v5 -5636> 2014-08-13 18:06:09.496586 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.496558, event: dispatched, request: pg_notify(22.48e(34) epoch 194497) v5 -5635> 2014-08-13 18:06:09.496594 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.496594, event: waiting_for_osdmap, request: pg_notify(22.48e(34) epoch 194497) v5 -5634> 2014-08-13 18:06:09.496600 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.327 -5633> 2014-08-13 18:06:09.496603 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352a00 -5632> 2014-08-13 18:06:09.496607 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.496607, event: started, request: pg_notify(22.48e(34) epoch 194497) v5 -5631> 2014-08-13 18:06:09.496626 7f5c835ef700 5 -- op tracker -- , seq: 588, time: 2014-08-13 18:06:09.496626, event: done, request: pg_notify(22.48e(34) epoch 194497) v5 -5630> 2014-08-13 18:06:09.496531 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.4 -5629> 2014-08-13 18:06:09.496649 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5628> 2014-08-13 18:06:09.496652 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5627> 2014-08-13 18:06:09.496657 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.339 192.168.1.110:6818/45268 9 ==== pg_notify(22.3af(36) epoch 194497) v5 ==== 2738+0+0 (3958580385 0 0) 0x5889c00 con 0x4c594a0 -5626> 2014-08-13 18:06:09.496670 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5625> 2014-08-13 18:06:09.496673 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5624> 2014-08-13 18:06:09.496675 7f5c835ef700 20 osd.4 194497 _dispatch 0x5889c00 pg_notify(22.3af(36) epoch 194497) v5 -5623> 2014-08-13 18:06:09.496680 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.214425, event: header_read, request: pg_notify(22.3af(36) epoch 194497) v5 -5622> 2014-08-13 18:06:09.496686 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.214427, event: throttled, request: pg_notify(22.3af(36) epoch 194497) v5 -5621> 2014-08-13 18:06:09.496692 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.214561, event: all_read, request: pg_notify(22.3af(36) epoch 194497) v5 -5620> 2014-08-13 18:06:09.496578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.4 -5619> 2014-08-13 18:06:09.496697 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.496670, event: dispatched, request: pg_notify(22.3af(36) epoch 194497) v5 -5618> 2014-08-13 18:06:09.496706 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.496706, event: waiting_for_osdmap, request: pg_notify(22.3af(36) epoch 194497) v5 -5617> 2014-08-13 18:06:09.496713 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.339 -5616> 2014-08-13 18:06:09.496716 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5889c00 -5615> 2014-08-13 18:06:09.496720 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.496720, event: started, request: pg_notify(22.3af(36) epoch 194497) v5 -5614> 2014-08-13 18:06:09.496739 7f5c835ef700 5 -- op tracker -- , seq: 589, time: 2014-08-13 18:06:09.496739, event: done, request: pg_notify(22.3af(36) epoch 194497) v5 -5613> 2014-08-13 18:06:09.496752 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5612> 2014-08-13 18:06:09.496755 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5611> 2014-08-13 18:06:09.496760 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.311 192.168.1.101:6816/24270 7 ==== pg_notify(22.f3(26) epoch 194497) v5 ==== 2168+0+0 (2069233888 0 0) 0x5a6f380 con 0x4c59a20 -5610> 2014-08-13 18:06:09.496653 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.4 -5609> 2014-08-13 18:06:09.496775 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5608> 2014-08-13 18:06:09.496778 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5607> 2014-08-13 18:06:09.496780 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f380 pg_notify(22.f3(26) epoch 194497) v5 -5606> 2014-08-13 18:06:09.496786 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.307496, event: header_read, request: pg_notify(22.f3(26) epoch 194497) v5 -5605> 2014-08-13 18:06:09.496793 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.307498, event: throttled, request: pg_notify(22.f3(26) epoch 194497) v5 -5604> 2014-08-13 18:06:09.496799 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.307619, event: all_read, request: pg_notify(22.f3(26) epoch 194497) v5 -5603> 2014-08-13 18:06:09.496804 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.496774, event: dispatched, request: pg_notify(22.f3(26) epoch 194497) v5 -5602> 2014-08-13 18:06:09.496811 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.496810, event: waiting_for_osdmap, request: pg_notify(22.f3(26) epoch 194497) v5 -5601> 2014-08-13 18:06:09.496816 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.311 -5600> 2014-08-13 18:06:09.496705 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.4 -5599> 2014-08-13 18:06:09.496820 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f380 -5598> 2014-08-13 18:06:09.496824 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.496824, event: started, request: pg_notify(22.f3(26) epoch 194497) v5 -5597> 2014-08-13 18:06:09.496847 7f5c835ef700 5 -- op tracker -- , seq: 590, time: 2014-08-13 18:06:09.496847, event: done, request: pg_notify(22.f3(26) epoch 194497) v5 -5596> 2014-08-13 18:06:09.496858 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5595> 2014-08-13 18:06:09.496860 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5594> 2014-08-13 18:06:09.496879 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.293 192.168.1.109:6816/33855 7 ==== pg_notify(22.3d9(54) epoch 194497) v5 ==== 3660+0+0 (3480516652 0 0) 0x588fa80 con 0x4c59760 -5593> 2014-08-13 18:06:09.496893 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5592> 2014-08-13 18:06:09.496895 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5591> 2014-08-13 18:06:09.496778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.4 -5590> 2014-08-13 18:06:09.496898 7f5c835ef700 20 osd.4 194497 _dispatch 0x588fa80 pg_notify(22.3d9(54) epoch 194497) v5 -5589> 2014-08-13 18:06:09.496903 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.205284, event: header_read, request: pg_notify(22.3d9(54) epoch 194497) v5 -5588> 2014-08-13 18:06:09.496911 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.205285, event: throttled, request: pg_notify(22.3d9(54) epoch 194497) v5 -5587> 2014-08-13 18:06:09.496917 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.205427, event: all_read, request: pg_notify(22.3d9(54) epoch 194497) v5 -5586> 2014-08-13 18:06:09.496922 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.496892, event: dispatched, request: pg_notify(22.3d9(54) epoch 194497) v5 -5585> 2014-08-13 18:06:09.496929 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.496928, event: waiting_for_osdmap, request: pg_notify(22.3d9(54) epoch 194497) v5 -5584> 2014-08-13 18:06:09.496934 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.293 -5583> 2014-08-13 18:06:09.496823 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.4 -5582> 2014-08-13 18:06:09.496937 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588fa80 -5581> 2014-08-13 18:06:09.496942 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.496942, event: started, request: pg_notify(22.3d9(54) epoch 194497) v5 -5580> 2014-08-13 18:06:09.496962 7f5c835ef700 5 -- op tracker -- , seq: 591, time: 2014-08-13 18:06:09.496962, event: done, request: pg_notify(22.3d9(54) epoch 194497) v5 -5579> 2014-08-13 18:06:09.496988 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5578> 2014-08-13 18:06:09.496992 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5577> 2014-08-13 18:06:09.496997 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.342 192.168.1.110:6802/41438 12 ==== pg_notify(22.113(1) epoch 194497) v5 ==== 777+0+0 (312586343 0 0) 0x5a6c980 con 0x4c5b440 -5576> 2014-08-13 18:06:09.497010 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5575> 2014-08-13 18:06:09.497012 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5574> 2014-08-13 18:06:09.497014 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c980 pg_notify(22.113(1) epoch 194497) v5 -5573> 2014-08-13 18:06:09.496902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.4 -5572> 2014-08-13 18:06:09.497019 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.293892, event: header_read, request: pg_notify(22.113(1) epoch 194497) v5 -5571> 2014-08-13 18:06:09.497026 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.293894, event: throttled, request: pg_notify(22.113(1) epoch 194497) v5 -5570> 2014-08-13 18:06:09.497033 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.293965, event: all_read, request: pg_notify(22.113(1) epoch 194497) v5 -5569> 2014-08-13 18:06:09.497039 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.497009, event: dispatched, request: pg_notify(22.113(1) epoch 194497) v5 -5568> 2014-08-13 18:06:09.497045 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.497045, event: waiting_for_osdmap, request: pg_notify(22.113(1) epoch 194497) v5 -5567> 2014-08-13 18:06:09.497051 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.342 -5566> 2014-08-13 18:06:09.496940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.4 -5565> 2014-08-13 18:06:09.497057 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c980 -5564> 2014-08-13 18:06:09.497062 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.497061, event: started, request: pg_notify(22.113(1) epoch 194497) v5 -5563> 2014-08-13 18:06:09.497082 7f5c835ef700 5 -- op tracker -- , seq: 592, time: 2014-08-13 18:06:09.497082, event: done, request: pg_notify(22.113(1) epoch 194497) v5 -5562> 2014-08-13 18:06:09.497092 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5561> 2014-08-13 18:06:09.497095 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5560> 2014-08-13 18:06:09.497101 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.314 192.168.1.101:6802/22303 7 ==== pg_notify(22.619(36) epoch 194497) v5 ==== 2738+0+0 (1873143375 0 0) 0x588cd00 con 0x4c5b180 -5559> 2014-08-13 18:06:09.497114 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5558> 2014-08-13 18:06:09.497117 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5557> 2014-08-13 18:06:09.497119 7f5c835ef700 20 osd.4 194497 _dispatch 0x588cd00 pg_notify(22.619(36) epoch 194497) v5 -5556> 2014-08-13 18:06:09.497124 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.208605, event: header_read, request: pg_notify(22.619(36) epoch 194497) v5 -5555> 2014-08-13 18:06:09.497130 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.208608, event: throttled, request: pg_notify(22.619(36) epoch 194497) v5 -5554> 2014-08-13 18:06:09.497136 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.208741, event: all_read, request: pg_notify(22.619(36) epoch 194497) v5 -5553> 2014-08-13 18:06:09.497025 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.4 -5552> 2014-08-13 18:06:09.497142 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.497114, event: dispatched, request: pg_notify(22.619(36) epoch 194497) v5 -5551> 2014-08-13 18:06:09.497149 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.497148, event: waiting_for_osdmap, request: pg_notify(22.619(36) epoch 194497) v5 -5550> 2014-08-13 18:06:09.497155 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.314 -5549> 2014-08-13 18:06:09.497158 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588cd00 -5548> 2014-08-13 18:06:09.497163 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.497163, event: started, request: pg_notify(22.619(36) epoch 194497) v5 -5547> 2014-08-13 18:06:09.497061 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.4 -5546> 2014-08-13 18:06:09.497190 7f5c835ef700 5 -- op tracker -- , seq: 593, time: 2014-08-13 18:06:09.497189, event: done, request: pg_notify(22.619(36) epoch 194497) v5 -5545> 2014-08-13 18:06:09.497211 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5544> 2014-08-13 18:06:09.497215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5543> 2014-08-13 18:06:09.497220 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.48 192.168.1.112:6838/8300 7 ==== pg_notify(22.619(36) epoch 194497) v5 ==== 2738+0+0 (1873143375 0 0) 0x5a39340 con 0x4c5c620 -5542> 2014-08-13 18:06:09.497233 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5541> 2014-08-13 18:06:09.497235 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5540> 2014-08-13 18:06:09.497237 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a39340 pg_notify(22.619(36) epoch 194497) v5 -5539> 2014-08-13 18:06:09.497242 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.313363, event: header_read, request: pg_notify(22.619(36) epoch 194497) v5 -5538> 2014-08-13 18:06:09.497249 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.313366, event: throttled, request: pg_notify(22.619(36) epoch 194497) v5 -5537> 2014-08-13 18:06:09.497256 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.313481, event: all_read, request: pg_notify(22.619(36) epoch 194497) v5 -5536> 2014-08-13 18:06:09.497262 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.497233, event: dispatched, request: pg_notify(22.619(36) epoch 194497) v5 -5535> 2014-08-13 18:06:09.497147 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.4 -5534> 2014-08-13 18:06:09.497268 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.497268, event: waiting_for_osdmap, request: pg_notify(22.619(36) epoch 194497) v5 -5533> 2014-08-13 18:06:09.497275 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.48 -5532> 2014-08-13 18:06:09.497279 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a39340 -5531> 2014-08-13 18:06:09.497283 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.497283, event: started, request: pg_notify(22.619(36) epoch 194497) v5 -5530> 2014-08-13 18:06:09.497186 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.4 -5529> 2014-08-13 18:06:09.497303 7f5c835ef700 5 -- op tracker -- , seq: 594, time: 2014-08-13 18:06:09.497302, event: done, request: pg_notify(22.619(36) epoch 194497) v5 -5528> 2014-08-13 18:06:09.497314 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5527> 2014-08-13 18:06:09.497317 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5526> 2014-08-13 18:06:09.497328 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.380 192.168.1.114:6822/51844 7 ==== pg_notify(22.4df(32) epoch 194497) v5 ==== 2510+0+0 (1160568909 0 0) 0x5a3a680 con 0x4c5c0a0 -5525> 2014-08-13 18:06:09.497341 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5524> 2014-08-13 18:06:09.497344 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5523> 2014-08-13 18:06:09.497346 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a3a680 pg_notify(22.4df(32) epoch 194497) v5 -5522> 2014-08-13 18:06:09.497351 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.401202, event: header_read, request: pg_notify(22.4df(32) epoch 194497) v5 -5521> 2014-08-13 18:06:09.497357 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.401205, event: throttled, request: pg_notify(22.4df(32) epoch 194497) v5 -5520> 2014-08-13 18:06:09.497363 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.401334, event: all_read, request: pg_notify(22.4df(32) epoch 194497) v5 -5519> 2014-08-13 18:06:09.497368 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.497341, event: dispatched, request: pg_notify(22.4df(32) epoch 194497) v5 -5518> 2014-08-13 18:06:09.497374 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.497374, event: waiting_for_osdmap, request: pg_notify(22.4df(32) epoch 194497) v5 -5517> 2014-08-13 18:06:09.497380 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.380 -5516> 2014-08-13 18:06:09.497383 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a3a680 -5515> 2014-08-13 18:06:09.497388 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.497387, event: started, request: pg_notify(22.4df(32) epoch 194497) v5 -5514> 2014-08-13 18:06:09.497274 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.4 -5513> 2014-08-13 18:06:09.497407 7f5c835ef700 5 -- op tracker -- , seq: 595, time: 2014-08-13 18:06:09.497407, event: done, request: pg_notify(22.4df(32) epoch 194497) v5 -5512> 2014-08-13 18:06:09.497302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.4 -5511> 2014-08-13 18:06:09.497417 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5510> 2014-08-13 18:06:09.497420 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5509> 2014-08-13 18:06:09.497436 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.368 192.168.1.102:6826/15861 10 ==== pg_notify(22.1f1(34) epoch 194497) v5 ==== 2624+0+0 (581160723 0 0) 0x5a6fa80 con 0x4c5bde0 -5508> 2014-08-13 18:06:09.497449 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5507> 2014-08-13 18:06:09.497452 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5506> 2014-08-13 18:06:09.497453 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fa80 pg_notify(22.1f1(34) epoch 194497) v5 -5505> 2014-08-13 18:06:09.497458 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.308524, event: header_read, request: pg_notify(22.1f1(34) epoch 194497) v5 -5504> 2014-08-13 18:06:09.497466 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.308525, event: throttled, request: pg_notify(22.1f1(34) epoch 194497) v5 -5503> 2014-08-13 18:06:09.497472 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.308640, event: all_read, request: pg_notify(22.1f1(34) epoch 194497) v5 -5502> 2014-08-13 18:06:09.497477 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.497448, event: dispatched, request: pg_notify(22.1f1(34) epoch 194497) v5 -5501> 2014-08-13 18:06:09.497484 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.497483, event: waiting_for_osdmap, request: pg_notify(22.1f1(34) epoch 194497) v5 -5500> 2014-08-13 18:06:09.497501 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.368 -5499> 2014-08-13 18:06:09.497504 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fa80 -5498> 2014-08-13 18:06:09.497508 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.497508, event: started, request: pg_notify(22.1f1(34) epoch 194497) v5 -5497> 2014-08-13 18:06:09.497397 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.4 -5496> 2014-08-13 18:06:09.497527 7f5c835ef700 5 -- op tracker -- , seq: 596, time: 2014-08-13 18:06:09.497527, event: done, request: pg_notify(22.1f1(34) epoch 194497) v5 -5495> 2014-08-13 18:06:09.497420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.4 -5494> 2014-08-13 18:06:09.497538 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5493> 2014-08-13 18:06:09.497542 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5492> 2014-08-13 18:06:09.497562 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.350 192.168.1.114:6832/53185 8 ==== pg_notify(24.9b(39) epoch 194497) v5 ==== 2949+0+0 (3769220390 0 0) 0x5355e80 con 0x4c5ee00 -5491> 2014-08-13 18:06:09.497576 7f5c835ef700 10 osd.4 194497 do_waiters -- start -5490> 2014-08-13 18:06:09.497579 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -5489> 2014-08-13 18:06:09.497580 7f5c835ef700 20 osd.4 194497 _dispatch 0x5355e80 pg_notify(24.9b(39) epoch 194497) v5 -5488> 2014-08-13 18:06:09.497585 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.295523, event: header_read, request: pg_notify(24.9b(39) epoch 194497) v5 -5487> 2014-08-13 18:06:09.497592 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.295525, event: throttled, request: pg_notify(24.9b(39) epoch 194497) v5 -5486> 2014-08-13 18:06:09.497598 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.295687, event: all_read, request: pg_notify(24.9b(39) epoch 194497) v5 -5485> 2014-08-13 18:06:09.497603 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.497575, event: dispatched, request: pg_notify(24.9b(39) epoch 194497) v5 -5484> 2014-08-13 18:06:09.497609 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.497609, event: waiting_for_osdmap, request: pg_notify(24.9b(39) epoch 194497) v5 -5483> 2014-08-13 18:06:09.497615 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.350 -5482> 2014-08-13 18:06:09.497618 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5355e80 -5481> 2014-08-13 18:06:09.497622 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.497622, event: started, request: pg_notify(24.9b(39) epoch 194497) v5 -5480> 2014-08-13 18:06:09.497520 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.4 -5479> 2014-08-13 18:06:09.497540 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.4 -5478> 2014-08-13 18:06:09.497643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.4 -5477> 2014-08-13 18:06:09.497662 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.4 -5476> 2014-08-13 18:06:09.497764 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.4 -5475> 2014-08-13 18:06:09.497802 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.4 -5474> 2014-08-13 18:06:09.497885 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.4 -5473> 2014-08-13 18:06:09.497931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.4 -5472> 2014-08-13 18:06:09.498006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.4 -5471> 2014-08-13 18:06:09.498048 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.4 -5470> 2014-08-13 18:06:09.498127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.4 -5469> 2014-08-13 18:06:09.498190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.4 -5468> 2014-08-13 18:06:09.498253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.4 -5467> 2014-08-13 18:06:09.498338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.4 -5466> 2014-08-13 18:06:09.498375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.4 -5465> 2014-08-13 18:06:09.498461 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.4 -5464> 2014-08-13 18:06:09.498496 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.4 -5463> 2014-08-13 18:06:09.498620 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.4 -5462> 2014-08-13 18:06:09.498609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.4 -5461> 2014-08-13 18:06:09.498743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.4 -5460> 2014-08-13 18:06:09.498748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.4 -5459> 2014-08-13 18:06:09.498865 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.4 -5458> 2014-08-13 18:06:09.498986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.4 -5457> 2014-08-13 18:06:09.498880 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5456> 2014-08-13 18:06:09.499107 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.4 -5455> 2014-08-13 18:06:09.499112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] activate: adding 10 as a source -5454> 2014-08-13 18:06:09.499233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.4 -5453> 2014-08-13 18:06:09.499302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.10 -5452> 2014-08-13 18:06:09.499353 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5451> 2014-08-13 18:06:09.499426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=55] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.10 -5450> 2014-08-13 18:06:09.499549 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 291 as a source -5449> 2014-08-13 18:06:09.499563 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=54] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.10 -5448> 2014-08-13 18:06:09.499670 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.291 -5447> 2014-08-13 18:06:09.499692 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=53] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.10 -5446> 2014-08-13 18:06:09.499792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.291 -5445> 2014-08-13 18:06:09.499820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=52] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.10 -5444> 2014-08-13 18:06:09.499921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.291 -5443> 2014-08-13 18:06:09.499952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=51] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.10 -5442> 2014-08-13 18:06:09.500052 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.291 -5441> 2014-08-13 18:06:09.500086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=50] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.10 -5440> 2014-08-13 18:06:09.500188 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.291 -5439> 2014-08-13 18:06:09.500231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=49] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.10 -5438> 2014-08-13 18:06:09.500326 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.291 -5437> 2014-08-13 18:06:09.500376 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=48] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.10 -5436> 2014-08-13 18:06:09.500465 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.291 -5435> 2014-08-13 18:06:09.500552 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=47] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.10 -5434> 2014-08-13 18:06:09.500609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.291 -5433> 2014-08-13 18:06:09.500754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.291 -5432> 2014-08-13 18:06:09.500723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=46] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.10 -5431> 2014-08-13 18:06:09.500902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.291 -5430> 2014-08-13 18:06:09.500914 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=45] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.10 -5429> 2014-08-13 18:06:09.501053 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.291 -5428> 2014-08-13 18:06:09.501119 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=44] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.10 -5427> 2014-08-13 18:06:09.501226 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.291 -5426> 2014-08-13 18:06:09.501347 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=43] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.10 -5425> 2014-08-13 18:06:09.501386 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.291 -5424> 2014-08-13 18:06:09.501528 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=42] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.10 -5423> 2014-08-13 18:06:09.501547 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.291 -5422> 2014-08-13 18:06:09.501711 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.291 -5421> 2014-08-13 18:06:09.501710 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=41] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.10 -5420> 2014-08-13 18:06:09.501881 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.291 -5419> 2014-08-13 18:06:09.501897 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=40] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.10 -5418> 2014-08-13 18:06:09.502051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.291 -5417> 2014-08-13 18:06:09.502104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=39] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.10 -5416> 2014-08-13 18:06:09.502231 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.291 -5415> 2014-08-13 18:06:09.502337 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=38] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.10 -5414> 2014-08-13 18:06:09.502410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.291 -5413> 2014-08-13 18:06:09.502514 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=37] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.10 -5412> 2014-08-13 18:06:09.502589 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.291 -5411> 2014-08-13 18:06:09.502690 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=36] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.10 -5410> 2014-08-13 18:06:09.502771 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.291 -5409> 2014-08-13 18:06:09.502869 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=35] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.10 -5408> 2014-08-13 18:06:09.502955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.291 -5407> 2014-08-13 18:06:09.503050 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=34] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.10 -5406> 2014-08-13 18:06:09.503142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.291 -5405> 2014-08-13 18:06:09.503275 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=33] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.10 -5404> 2014-08-13 18:06:09.503338 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.291 -5403> 2014-08-13 18:06:09.503492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=32] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.10 -5402> 2014-08-13 18:06:09.503534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.291 -5401> 2014-08-13 18:06:09.503731 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.291 -5400> 2014-08-13 18:06:09.503709 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=31] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.10 -5399> 2014-08-13 18:06:09.503931 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.291 -5398> 2014-08-13 18:06:09.503967 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=30] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.10 -5397> 2014-08-13 18:06:09.504135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.291 -5396> 2014-08-13 18:06:09.504213 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=29] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.10 -5395> 2014-08-13 18:06:09.504346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.291 -5394> 2014-08-13 18:06:09.504555 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.291 -5393> 2014-08-13 18:06:09.504477 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=28] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.10 -5392> 2014-08-13 18:06:09.504768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.291 -5391> 2014-08-13 18:06:09.504785 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=27] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.10 -5390> 2014-08-13 18:06:09.504983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.291 -5389> 2014-08-13 18:06:09.505002 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=26] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.10 -5388> 2014-08-13 18:06:09.505213 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.291 -5387> 2014-08-13 18:06:09.505221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=25] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.10 -5386> 2014-08-13 18:06:09.505437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.291 -5385> 2014-08-13 18:06:09.505475 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=24] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.10 -5384> 2014-08-13 18:06:09.505662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.291 -5383> 2014-08-13 18:06:09.505724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=23] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.10 -5382> 2014-08-13 18:06:09.505891 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.291 -5381> 2014-08-13 18:06:09.505984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=22] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.10 -5380> 2014-08-13 18:06:09.506121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.291 -5379> 2014-08-13 18:06:09.506280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=21] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.10 -5378> 2014-08-13 18:06:09.506360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.291 -5377> 2014-08-13 18:06:09.506548 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=20] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.10 -5376> 2014-08-13 18:06:09.506597 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.291 -5375> 2014-08-13 18:06:09.506841 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.291 -5374> 2014-08-13 18:06:09.506820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=19] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.10 -5373> 2014-08-13 18:06:09.507086 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.291 -5372> 2014-08-13 18:06:09.507090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=18] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.10 -5371> 2014-08-13 18:06:09.507340 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.291 -5370> 2014-08-13 18:06:09.507399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=17] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.10 -5369> 2014-08-13 18:06:09.507591 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.291 -5368> 2014-08-13 18:06:09.507693 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=16] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.10 -5367> 2014-08-13 18:06:09.507843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.291 -5366> 2014-08-13 18:06:09.507973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=15] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.10 -5365> 2014-08-13 18:06:09.508098 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.291 -5364> 2014-08-13 18:06:09.508270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=14] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.10 -5363> 2014-08-13 18:06:09.508360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.291 -5362> 2014-08-13 18:06:09.508568 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=13] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.10 -5361> 2014-08-13 18:06:09.508621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.291 -5360> 2014-08-13 18:06:09.508886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.291 -5359> 2014-08-13 18:06:09.508863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=12] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.10 -5358> 2014-08-13 18:06:09.509154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.291 -5357> 2014-08-13 18:06:09.509175 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=11] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.10 -5356> 2014-08-13 18:06:09.509429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.291 -5355> 2014-08-13 18:06:09.509479 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=10] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.10 -5354> 2014-08-13 18:06:09.509703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.291 -5353> 2014-08-13 18:06:09.509772 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=9] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.10 -5352> 2014-08-13 18:06:09.509981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.291 -5351> 2014-08-13 18:06:09.510084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=8] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.10 -5350> 2014-08-13 18:06:09.510267 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.291 -5349> 2014-08-13 18:06:09.510414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=7] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.10 -5348> 2014-08-13 18:06:09.510554 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.291 -5347> 2014-08-13 18:06:09.510690 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=6] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.10 -5346> 2014-08-13 18:06:09.510840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.291 -5345> 2014-08-13 18:06:09.510962 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=5] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.10 -5344> 2014-08-13 18:06:09.511131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.291 -5343> 2014-08-13 18:06:09.511253 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=4] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.10 -5342> 2014-08-13 18:06:09.511429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.291 -5341> 2014-08-13 18:06:09.511576 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=3] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.10 -5340> 2014-08-13 18:06:09.511726 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.291 -5339> 2014-08-13 18:06:09.511913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=2] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.10 -5338> 2014-08-13 18:06:09.512030 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.291 -5337> 2014-08-13 18:06:09.512264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56 u=1] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.10 -5336> 2014-08-13 18:06:09.512341 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5335> 2014-08-13 18:06:09.512599 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5334> 2014-08-13 18:06:09.512722 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 350 as a source -5333> 2014-08-13 18:06:09.512966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 286 as a source -5332> 2014-08-13 18:06:09.513030 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 is on osd.350 -5331> 2014-08-13 18:06:09.513360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 is on osd.350 -5330> 2014-08-13 18:06:09.513350 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 also missing on osd.286 (past last_backfill 0//0//-1) -5329> 2014-08-13 18:06:09.513673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 is on osd.350 -5328> 2014-08-13 18:06:09.513688 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 also missing on osd.286 (past last_backfill 0//0//-1) -5327> 2014-08-13 18:06:09.513984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 also missing on osd.286 (past last_backfill 0//0//-1) -5326> 2014-08-13 18:06:09.513983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 is on osd.350 -5325> 2014-08-13 18:06:09.514298 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 is on osd.350 -5324> 2014-08-13 18:06:09.514295 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 also missing on osd.286 (past last_backfill 0//0//-1) -5323> 2014-08-13 18:06:09.514620 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 also missing on osd.286 (past last_backfill 0//0//-1) -5322> 2014-08-13 18:06:09.514608 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 is on osd.350 -5321> 2014-08-13 18:06:09.514918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 also missing on osd.286 (past last_backfill 0//0//-1) -5320> 2014-08-13 18:06:09.514923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 is on osd.350 -5319> 2014-08-13 18:06:09.515235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 is on osd.350 -5318> 2014-08-13 18:06:09.515218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 also missing on osd.286 (past last_backfill 0//0//-1) -5317> 2014-08-13 18:06:09.515543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 also missing on osd.286 (past last_backfill 0//0//-1) -5316> 2014-08-13 18:06:09.515545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 is on osd.350 -5315> 2014-08-13 18:06:09.515839 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 also missing on osd.286 (past last_backfill 0//0//-1) -5314> 2014-08-13 18:06:09.515852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 is on osd.350 -5313> 2014-08-13 18:06:09.516131 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 also missing on osd.286 (past last_backfill 0//0//-1) -5312> 2014-08-13 18:06:09.516158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 is on osd.350 -5311> 2014-08-13 18:06:09.516465 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 also missing on osd.286 (past last_backfill 0//0//-1) -5310> 2014-08-13 18:06:09.516475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 is on osd.350 -5309> 2014-08-13 18:06:09.516760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 also missing on osd.286 (past last_backfill 0//0//-1) -5308> 2014-08-13 18:06:09.516782 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 is on osd.350 -5307> 2014-08-13 18:06:09.517051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 also missing on osd.286 (past last_backfill 0//0//-1) -5306> 2014-08-13 18:06:09.517090 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 is on osd.350 -5305> 2014-08-13 18:06:09.517390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 also missing on osd.286 (past last_backfill 0//0//-1) -5304> 2014-08-13 18:06:09.517400 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 is on osd.350 -5303> 2014-08-13 18:06:09.517691 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 also missing on osd.286 (past last_backfill 0//0//-1) -5302> 2014-08-13 18:06:09.517706 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 is on osd.350 -5301> 2014-08-13 18:06:09.518006 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 also missing on osd.286 (past last_backfill 0//0//-1) -5300> 2014-08-13 18:06:09.518012 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 is on osd.350 -5299> 2014-08-13 18:06:09.518322 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 is on osd.350 -5298> 2014-08-13 18:06:09.518313 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 also missing on osd.286 (past last_backfill 0//0//-1) -5297> 2014-08-13 18:06:09.518630 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 is on osd.350 -5296> 2014-08-13 18:06:09.518645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 also missing on osd.286 (past last_backfill 0//0//-1) -5295> 2014-08-13 18:06:09.518940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 is on osd.350 -5294> 2014-08-13 18:06:09.518939 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 also missing on osd.286 (past last_backfill 0//0//-1) -5293> 2014-08-13 18:06:09.519255 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 is on osd.350 -5292> 2014-08-13 18:06:09.519290 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 also missing on osd.286 (past last_backfill 0//0//-1) -5291> 2014-08-13 18:06:09.519564 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 is on osd.350 -5290> 2014-08-13 18:06:09.519596 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 also missing on osd.286 (past last_backfill 0//0//-1) -5289> 2014-08-13 18:06:09.519871 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 is on osd.350 -5288> 2014-08-13 18:06:09.519885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 also missing on osd.286 (past last_backfill 0//0//-1) -5287> 2014-08-13 18:06:09.520178 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 is on osd.350 -5286> 2014-08-13 18:06:09.520180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 also missing on osd.286 (past last_backfill 0//0//-1) -5285> 2014-08-13 18:06:09.520506 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 also missing on osd.286 (past last_backfill 0//0//-1) -5284> 2014-08-13 18:06:09.520491 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 is on osd.350 -5283> 2014-08-13 18:06:09.520803 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 also missing on osd.286 (past last_backfill 0//0//-1) -5282> 2014-08-13 18:06:09.520811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 is on osd.350 -5281> 2014-08-13 18:06:09.521119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 is on osd.350 -5280> 2014-08-13 18:06:09.521095 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 also missing on osd.286 (past last_backfill 0//0//-1) -5279> 2014-08-13 18:06:09.521439 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 is on osd.350 -5278> 2014-08-13 18:06:09.521440 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 also missing on osd.286 (past last_backfill 0//0//-1) -5277> 2014-08-13 18:06:09.521752 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 is on osd.350 -5276> 2014-08-13 18:06:09.521797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 also missing on osd.286 (past last_backfill 0//0//-1) -5275> 2014-08-13 18:06:09.522060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 is on osd.350 -5274> 2014-08-13 18:06:09.522088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 also missing on osd.286 (past last_backfill 0//0//-1) -5273> 2014-08-13 18:06:09.522382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 is on osd.350 -5272> 2014-08-13 18:06:09.522431 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 also missing on osd.286 (past last_backfill 0//0//-1) -5271> 2014-08-13 18:06:09.522691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 is on osd.350 -5270> 2014-08-13 18:06:09.522746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 also missing on osd.286 (past last_backfill 0//0//-1) -5269> 2014-08-13 18:06:09.522999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 is on osd.350 -5268> 2014-08-13 18:06:09.523042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 also missing on osd.286 (past last_backfill 0//0//-1) -5267> 2014-08-13 18:06:09.523312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 is on osd.350 -5266> 2014-08-13 18:06:09.523386 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 also missing on osd.286 (past last_backfill 0//0//-1) -5265> 2014-08-13 18:06:09.523621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 is on osd.350 -5264> 2014-08-13 18:06:09.523680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 also missing on osd.286 (past last_backfill 0//0//-1) -5263> 2014-08-13 18:06:09.523928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 is on osd.350 -5262> 2014-08-13 18:06:09.523987 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 also missing on osd.286 (past last_backfill 0//0//-1) -5261> 2014-08-13 18:06:09.524243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 is on osd.350 -5260> 2014-08-13 18:06:09.524326 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 also missing on osd.286 (past last_backfill 0//0//-1) -5259> 2014-08-13 18:06:09.524552 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 is on osd.350 -5258> 2014-08-13 18:06:09.524637 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 also missing on osd.286 (past last_backfill 0//0//-1) -5257> 2014-08-13 18:06:09.524859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 is on osd.350 -5256> 2014-08-13 18:06:09.524929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 also missing on osd.286 (past last_backfill 0//0//-1) -5255> 2014-08-13 18:06:09.525167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 is on osd.350 -5254> 2014-08-13 18:06:09.525227 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 also missing on osd.286 (past last_backfill 0//0//-1) -5253> 2014-08-13 18:06:09.525479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 is on osd.350 -5252> 2014-08-13 18:06:09.525553 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 also missing on osd.286 (past last_backfill 0//0//-1) -5251> 2014-08-13 18:06:09.525787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 is on osd.350 -5250> 2014-08-13 18:06:09.525846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 also missing on osd.286 (past last_backfill 0//0//-1) -5249> 2014-08-13 18:06:09.526095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 is on osd.350 -5248> 2014-08-13 18:06:09.526136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 also missing on osd.286 (past last_backfill 0//0//-1) -5247> 2014-08-13 18:06:09.526407 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 is on osd.350 -5246> 2014-08-13 18:06:09.526456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 also missing on osd.286 (past last_backfill 0//0//-1) -5245> 2014-08-13 18:06:09.526715 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 is on osd.350 -5244> 2014-08-13 18:06:09.526748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 also missing on osd.286 (past last_backfill 0//0//-1) -5243> 2014-08-13 18:06:09.527023 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 is on osd.350 -5242> 2014-08-13 18:06:09.527037 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 also missing on osd.286 (past last_backfill 0//0//-1) -5241> 2014-08-13 18:06:09.527336 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 is on osd.350 -5240> 2014-08-13 18:06:09.527341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 also missing on osd.286 (past last_backfill 0//0//-1) -5239> 2014-08-13 18:06:09.527644 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 also missing on osd.286 (past last_backfill 0//0//-1) -5238> 2014-08-13 18:06:09.527644 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 is on osd.350 -5237> 2014-08-13 18:06:09.527938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 also missing on osd.286 (past last_backfill 0//0//-1) -5236> 2014-08-13 18:06:09.527952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 is on osd.350 -5235> 2014-08-13 18:06:09.528262 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 is on osd.350 -5234> 2014-08-13 18:06:09.528258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 also missing on osd.286 (past last_backfill 0//0//-1) -5233> 2014-08-13 18:06:09.528570 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 is on osd.350 -5232> 2014-08-13 18:06:09.528608 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 also missing on osd.286 (past last_backfill 0//0//-1) -5231> 2014-08-13 18:06:09.528876 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 is on osd.350 -5230> 2014-08-13 18:06:09.528901 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 also missing on osd.286 (past last_backfill 0//0//-1) -5229> 2014-08-13 18:06:09.529183 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 is on osd.350 -5228> 2014-08-13 18:06:09.529205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 also missing on osd.286 (past last_backfill 0//0//-1) -5227> 2014-08-13 18:06:09.529495 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 is on osd.350 -5226> 2014-08-13 18:06:09.529523 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 also missing on osd.286 (past last_backfill 0//0//-1) -5225> 2014-08-13 18:06:09.529813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 also missing on osd.286 (past last_backfill 0//0//-1) -5224> 2014-08-13 18:06:09.529802 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 is on osd.350 -5223> 2014-08-13 18:06:09.530109 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 is on osd.350 -5222> 2014-08-13 18:06:09.530106 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 also missing on osd.286 (past last_backfill 0//0//-1) -5221> 2014-08-13 18:06:09.530422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 is on osd.350 -5220> 2014-08-13 18:06:09.530443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 also missing on osd.286 (past last_backfill 0//0//-1) -5219> 2014-08-13 18:06:09.530729 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 is on osd.350 -5218> 2014-08-13 18:06:09.530740 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5217> 2014-08-13 18:06:09.531038 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 is on osd.350 -5216> 2014-08-13 18:06:09.531101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] activate: adding 375 as a source -5215> 2014-08-13 18:06:09.531351 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5214> 2014-08-13 18:06:09.531414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24 194416'18601 is on osd.375 -5213> 2014-08-13 18:06:09.531739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24 194370'18550 is on osd.375 -5212> 2014-08-13 18:06:09.531733 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 367 as a source -5211> 2014-08-13 18:06:09.532034 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24 194377'18588 is on osd.375 -5210> 2014-08-13 18:06:09.532039 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.367 (past last_backfill 0//0//-1) -5209> 2014-08-13 18:06:09.532350 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.367 (past last_backfill 0//0//-1) -5208> 2014-08-13 18:06:09.532344 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24 194416'18614 is on osd.375 -5207> 2014-08-13 18:06:09.532659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.367 (past last_backfill 0//0//-1) -5206> 2014-08-13 18:06:09.532672 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24 194377'18590 is on osd.375 -5205> 2014-08-13 18:06:09.532967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.367 (past last_backfill 0//0//-1) -5204> 2014-08-13 18:06:09.532984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24 194422'18646 is on osd.375 -5203> 2014-08-13 18:06:09.533278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.367 (past last_backfill 0//0//-1) -5202> 2014-08-13 18:06:09.533328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24 194370'18562 is on osd.375 -5201> 2014-08-13 18:06:09.533586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.367 (past last_backfill 0//0//-1) -5200> 2014-08-13 18:06:09.533640 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24 194416'18610 is on osd.375 -5199> 2014-08-13 18:06:09.533892 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.367 (past last_backfill 0//0//-1) -5198> 2014-08-13 18:06:09.533933 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24 194416'18620 is on osd.375 -5197> 2014-08-13 18:06:09.534204 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.367 (past last_backfill 0//0//-1) -5196> 2014-08-13 18:06:09.534230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24 194377'18586 is on osd.375 -5195> 2014-08-13 18:06:09.534511 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.367 (past last_backfill 0//0//-1) -5194> 2014-08-13 18:06:09.534561 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24 194416'18636 is on osd.375 -5193> 2014-08-13 18:06:09.534819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.367 (past last_backfill 0//0//-1) -5192> 2014-08-13 18:06:09.534855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24 194422'18654 is on osd.375 -5191> 2014-08-13 18:06:09.535125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.367 (past last_backfill 0//0//-1) -5190> 2014-08-13 18:06:09.535144 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24 194370'18576 is on osd.375 -5189> 2014-08-13 18:06:09.535438 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.367 (past last_backfill 0//0//-1) -5188> 2014-08-13 18:06:09.535461 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24 194370'18564 is on osd.375 -5187> 2014-08-13 18:06:09.535752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24 194370'18574 is on osd.375 -5186> 2014-08-13 18:06:09.535746 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.367 (past last_backfill 0//0//-1) -5185> 2014-08-13 18:06:09.536045 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24 194377'18580 is on osd.375 -5184> 2014-08-13 18:06:09.536051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.367 (past last_backfill 0//0//-1) -5183> 2014-08-13 18:06:09.536360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.367 (past last_backfill 0//0//-1) -5182> 2014-08-13 18:06:09.536349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24 194377'18584 is on osd.375 -5181> 2014-08-13 18:06:09.536674 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24 194396'18600 is on osd.375 -5180> 2014-08-13 18:06:09.536668 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.367 (past last_backfill 0//0//-1) -5179> 2014-08-13 18:06:09.536971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24 194370'18556 is on osd.375 -5178> 2014-08-13 18:06:09.536973 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.367 (past last_backfill 0//0//-1) -5177> 2014-08-13 18:06:09.537276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24 194370'18568 is on osd.375 -5176> 2014-08-13 18:06:09.537283 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.367 (past last_backfill 0//0//-1) -5175> 2014-08-13 18:06:09.537587 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24 194370'18554 is on osd.375 -5174> 2014-08-13 18:06:09.537587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.367 (past last_backfill 0//0//-1) -5173> 2014-08-13 18:06:09.537883 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24 194416'18608 is on osd.375 -5172> 2014-08-13 18:06:09.537894 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.367 (past last_backfill 0//0//-1) -5171> 2014-08-13 18:06:09.538203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.367 (past last_backfill 0//0//-1) -5170> 2014-08-13 18:06:09.538220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24 194422'18656 is on osd.375 -5169> 2014-08-13 18:06:09.538510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.367 (past last_backfill 0//0//-1) -5168> 2014-08-13 18:06:09.538518 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24 194422'18642 is on osd.375 -5167> 2014-08-13 18:06:09.538818 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.367 (past last_backfill 0//0//-1) -5166> 2014-08-13 18:06:09.538841 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24 194378'18592 is on osd.375 -5165> 2014-08-13 18:06:09.539125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.367 (past last_backfill 0//0//-1) -5164> 2014-08-13 18:06:09.539134 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24 194422'18658 is on osd.375 -5163> 2014-08-13 18:06:09.539450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24 194416'18618 is on osd.375 -5162> 2014-08-13 18:06:09.539437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.367 (past last_backfill 0//0//-1) -5161> 2014-08-13 18:06:09.539749 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24 194416'18632 is on osd.375 -5160> 2014-08-13 18:06:09.539753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.367 (past last_backfill 0//0//-1) -5159> 2014-08-13 18:06:09.540042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24 194416'18626 is on osd.375 -5158> 2014-08-13 18:06:09.540060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.367 (past last_backfill 0//0//-1) -5157> 2014-08-13 18:06:09.540367 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24 194422'18650 is on osd.375 -5156> 2014-08-13 18:06:09.540372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.367 (past last_backfill 0//0//-1) -5155> 2014-08-13 18:06:09.540679 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24 194416'18606 is on osd.375 -5154> 2014-08-13 18:06:09.540677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.367 (past last_backfill 0//0//-1) -5153> 2014-08-13 18:06:09.540972 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24 194416'18612 is on osd.375 -5152> 2014-08-13 18:06:09.540982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.367 (past last_backfill 0//0//-1) -5151> 2014-08-13 18:06:09.541281 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24 194380'18594 is on osd.375 -5150> 2014-08-13 18:06:09.541292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.367 (past last_backfill 0//0//-1) -5149> 2014-08-13 18:06:09.541597 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24 194416'18628 is on osd.375 -5148> 2014-08-13 18:06:09.541596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.367 (past last_backfill 0//0//-1) -5147> 2014-08-13 18:06:09.541892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24 194422'18638 is on osd.375 -5146> 2014-08-13 18:06:09.541902 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.367 (past last_backfill 0//0//-1) -5145> 2014-08-13 18:06:09.542211 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.367 (past last_backfill 0//0//-1) -5144> 2014-08-13 18:06:09.542204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24 194370'18560 is on osd.375 -5143> 2014-08-13 18:06:09.542525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24 194422'18652 is on osd.375 -5142> 2014-08-13 18:06:09.542519 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.367 (past last_backfill 0//0//-1) -5141> 2014-08-13 18:06:09.542820 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24 194370'18552 is on osd.375 -5140> 2014-08-13 18:06:09.542824 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.367 (past last_backfill 0//0//-1) -5139> 2014-08-13 18:06:09.543113 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24 194370'18570 is on osd.375 -5138> 2014-08-13 18:06:09.543128 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.367 (past last_backfill 0//0//-1) -5137> 2014-08-13 18:06:09.543426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24 194416'18634 is on osd.375 -5136> 2014-08-13 18:06:09.543437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.367 (past last_backfill 0//0//-1) -5135> 2014-08-13 18:06:09.543727 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24 194393'18598 is on osd.375 -5134> 2014-08-13 18:06:09.543741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.367 (past last_backfill 0//0//-1) -5133> 2014-08-13 18:06:09.544045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.367 (past last_backfill 0//0//-1) -5132> 2014-08-13 18:06:09.544039 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24 194370'18548 is on osd.375 -5131> 2014-08-13 18:06:09.544356 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.367 (past last_backfill 0//0//-1) -5130> 2014-08-13 18:06:09.544374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24 194422'18640 is on osd.375 -5129> 2014-08-13 18:06:09.544667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24 194416'18604 is on osd.375 -5128> 2014-08-13 18:06:09.544663 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.367 (past last_backfill 0//0//-1) -5127> 2014-08-13 18:06:09.544961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24 194377'18582 is on osd.375 -5126> 2014-08-13 18:06:09.544968 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.367 (past last_backfill 0//0//-1) -5125> 2014-08-13 18:06:09.545265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24 194416'18624 is on osd.375 -5124> 2014-08-13 18:06:09.545278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.367 (past last_backfill 0//0//-1) -5123> 2014-08-13 18:06:09.545573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24 194416'18622 is on osd.375 -5122> 2014-08-13 18:06:09.545582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.367 (past last_backfill 0//0//-1) -5121> 2014-08-13 18:06:09.545866 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24 194416'18630 is on osd.375 -5120> 2014-08-13 18:06:09.545886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.367 (past last_backfill 0//0//-1) -5119> 2014-08-13 18:06:09.546164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24 194370'18566 is on osd.375 -5118> 2014-08-13 18:06:09.546194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.367 (past last_backfill 0//0//-1) -5117> 2014-08-13 18:06:09.546478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24 194370'18558 is on osd.375 -5116> 2014-08-13 18:06:09.546499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.367 (past last_backfill 0//0//-1) -5115> 2014-08-13 18:06:09.546783 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24 194377'18578 is on osd.375 -5114> 2014-08-13 18:06:09.546804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.367 (past last_backfill 0//0//-1) -5113> 2014-08-13 18:06:09.547088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24 194422'18644 is on osd.375 -5112> 2014-08-13 18:06:09.547108 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.367 (past last_backfill 0//0//-1) -5111> 2014-08-13 18:06:09.547416 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.367 (past last_backfill 0//0//-1) -5110> 2014-08-13 18:06:09.547392 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24 194370'18572 is on osd.375 -5109> 2014-08-13 18:06:09.547724 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.367 (past last_backfill 0//0//-1) -5108> 2014-08-13 18:06:09.547743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24 194422'18660 is on osd.375 -5107> 2014-08-13 18:06:09.548031 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.367 (past last_backfill 0//0//-1) -5106> 2014-08-13 18:06:09.548165 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24 194422'18648 is on osd.375 -5105> 2014-08-13 18:06:09.548342 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.367 (past last_backfill 0//0//-1) -5104> 2014-08-13 18:06:09.548496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] search_for_missing 724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24 194416'18616 is on osd.375 -5103> 2014-08-13 18:06:09.548650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.367 (past last_backfill 0//0//-1) -5102> 2014-08-13 18:06:09.548790 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] needs_recovery_map missing {30b/hit_set_24.30b_archive_2014-08-12 20:49:27.261989_2014-08-13 15:28:34.591698/head/.ceph-internal/24=194416'18601,e105130b/benchmark_data_storage0111-ib.pocstor_12495_object1576/head//24=194370'18550,8ec9430b/benchmark_data_storage0111-ib.pocstor_25065_object2920/head//24=194377'18588,1882630b/benchmark_data_storage0111-ib.pocstor_19103_object6371/head//24=194416'18614,e981730b/benchmark_data_storage0111-ib.pocstor_25065_object3164/head//24=194377'18590,5d4fb30b/benchmark_data_storage0111-ib.pocstor_41231_object6765/head//24=194422'18646,3d36c30b/benchmark_data_storage0111-ib.pocstor_12495_object6326/head//24=194370'18562,a44fc30b/benchmark_data_storage0111-ib.pocstor_19103_object4624/head//24=194416'18610,570bd30b/benchmark_data_storage0111-ib.pocstor_19103_object10602/head//24=194416'18620,3d90e30b/benchmark_data_storage0111-ib.pocstor_25065_object2182/head//24=194377'18586,36c3e30b/benchmark_data_storage0111-ib.pocstor_19103_object18576/head//24=194416'18636,69dde30b/benchmark_data_storage0111-ib.pocstor_41231_object8451/head//24=194422'18654,aac5f30b/benchmark_data_storage0111-ib.pocstor_12495_object12131/head//24=194370'18576,3759f30b/benchmark_data_storage0111-ib.pocstor_12495_object6654/head//24=194370'18564,8779f30b/benchmark_data_storage0111-ib.pocstor_12495_object11696/head//24=194370'18574,eb78070b/benchmark_data_storage0111-ib.pocstor_25065_object922/head//24=194377'18580,a27a170b/benchmark_data_storage0111-ib.pocstor_25065_object2016/head//24=194377'18584,d02e270b/benchmark_data_storage0101-ib.pocstor_6819_object363/head//24=194396'18600,d674470b/benchmark_data_storage0111-ib.pocstor_12495_object4971/head//24=194370'18556,a483570b/benchmark_data_storage0111-ib.pocstor_12495_object7258/head//24=194370'18568,91f2870b/benchmark_data_storage0111-ib.pocstor_12495_object4880/head//24=194370'18554,ac6c870b/benchmark_data_storage0111-ib.pocstor_19103_object2554/head//24=194416'18608,e79ba70b/benchmark_data_storage0111-ib.pocstor_41231_object9528/head//24=194422'18656,bf59b70b/benchmark_data_storage0111-ib.pocstor_41231_object3715/head//24=194422'18642,8a33c70b/benchmark_data_storage0111-ib.pocstor_25065_object4861/head//24=194378'18592,c05dc70b/benchmark_data_storage0111-ib.pocstor_41231_object9684/head//24=194422'18658,f9bed70b/benchmark_data_storage0111-ib.pocstor_19103_object10468/head//24=194416'18618,8aced70b/benchmark_data_storage0111-ib.pocstor_19103_object17542/head//24=194416'18632,20e5f70b/benchmark_data_storage0111-ib.pocstor_19103_object15096/head//24=194416'18626,4f5df70b/benchmark_data_storage0111-ib.pocstor_41231_object7319/head//24=194422'18650,5f5ff70b/benchmark_data_storage0111-ib.pocstor_19103_object2111/head//24=194416'18606,58f50b0b/benchmark_data_storage0111-ib.pocstor_19103_object4658/head//24=194416'18612,31a15b0b/benchmark_data_storage0111-ib.pocstor_25065_object5506/head//24=194380'18594,ecba6b0b/benchmark_data_storage0111-ib.pocstor_19103_object15684/head//24=194416'18628,eeff6b0b/benchmark_data_storage0111-ib.pocstor_41231_object419/head//24=194422'18638,94258b0b/benchmark_data_storage0111-ib.pocstor_12495_object6023/head//24=194370'18560,eb9ab0b/benchmark_data_storage0111-ib.pocstor_41231_object8323/head//24=194422'18652,a650fb0b/benchmark_data_storage0111-ib.pocstor_12495_object2245/head//24=194370'18552,d016fb0b/benchmark_data_storage0111-ib.pocstor_12495_object8280/head//24=194370'18570,81041f0b/benchmark_data_storage0111-ib.pocstor_19103_object18362/head//24=194416'18634,ac602f0b/benchmark_data_storage0111-ib.pocstor_32482_object381/head//24=194393'18598,d9253f0b/benchmark_data_storage0111-ib.pocstor_12495_object422/head//24=194370'18548,b2644f0b/benchmark_data_storage0111-ib.pocstor_41231_object2117/head//24=194422'18640,c0354f0b/benchmark_data_storage0111-ib.pocstor_19103_object1019/head//24=194416'18604,f084f0b/benchmark_data_storage0111-ib.pocstor_25065_object1300/head//24=194377'18582,d9207f0b/benchmark_data_storage0111-ib.pocstor_19103_object14877/head//24=194416'18624,38117f0b/benchmark_data_storage0111-ib.pocstor_19103_object11349/head//24=194416'18622,98d67f0b/benchmark_data_storage0111-ib.pocstor_19103_object16895/head//24=194416'18630,eedb9f0b/benchmark_data_storage0111-ib.pocstor_12495_object7023/head//24=194370'18566,3b79af0b/benchmark_data_storage0111-ib.pocstor_12495_object5673/head//24=194370'18558,d62cbf0b/benchmark_data_storage0111-ib.pocstor_25065_object509/head//24=194377'18578,1ef9cf0b/benchmark_data_storage0111-ib.pocstor_41231_object5078/head//24=194422'18644,5afcef0b/benchmark_data_storage0111-ib.pocstor_12495_object11224/head//24=194370'18572,c225ff0b/benchmark_data_storage0111-ib.pocstor_41231_object9742/head//24=194422'18660,804cff0b/benchmark_data_storage0111-ib.pocstor_41231_object6937/head//24=194422'18648,724eff0b/benchmark_data_storage0111-ib.pocstor_19103_object9561/head//24=194416'18616} -5101> 2014-08-13 18:06:09.548956 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.367 (past last_backfill 0//0//-1) -5100> 2014-08-13 18:06:09.549156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound -5099> 2014-08-13 18:06:09.549270 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.367 (past last_backfill 0//0//-1) -5098> 2014-08-13 18:06:09.549456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _calc_past_interval_range: already have past intervals back to 194493 -5097> 2014-08-13 18:06:09.549576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.367 (past last_backfill 0//0//-1) -5096> 2014-08-13 18:06:09.549775 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] build_might_have_unfound: built 10,286,301,375 -5095> 2014-08-13 18:06:09.550068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Activate Finished -5094> 2014-08-13 18:06:09.549883 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.367 (past last_backfill 0//0//-1) -5093> 2014-08-13 18:06:09.550371 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] enter Started/Primary/Active/Activating -5092> 2014-08-13 18:06:09.550422 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -5091> 2014-08-13 18:06:09.550681 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -5090> 2014-08-13 18:06:09.550807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 378 as a source -5089> 2014-08-13 18:06:09.550973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -5088> 2014-08-13 18:06:09.551376 7f5c7bde3700 10 log is not dirty -5087> 2014-08-13 18:06:09.551115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24 194416'17245 also missing on osd.378 (past last_backfill 0//0//-1) -5086> 2014-08-13 18:06:09.551424 7f5c7bde3700 20 osd.4 194497 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 10 -5085> 2014-08-13 18:06:09.551439 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing -5084> 2014-08-13 18:06:09.551474 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.115:6815/60573 -- pg_info(1 pgs e194497:24.30b(42)) v4 -- ?+0 0x4bf5240 con 0x43f0840 -5083> 2014-08-13 18:06:09.551521 7f5c7bde3700 20 osd.4 194497 Sending info 24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 375 -5082> 2014-08-13 18:06:09.551596 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing -5081> 2014-08-13 18:06:09.551620 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6830/52879 -- pg_info(1 pgs e194497:24.30b(42)) v4 -- ?+0 0x5356040 con 0x4c5e5c0 -5080> 2014-08-13 18:06:09.551430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24 194416'17282 also missing on osd.378 (past last_backfill 0//0//-1) -5079> 2014-08-13 18:06:09.551737 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_advance_map [4,25,362]/[4,20,303] -- 4/4 -5078> 2014-08-13 18:06:09.551779 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -5077> 2014-08-13 18:06:09.551788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Peering advmap -5076> 2014-08-13 18:06:09.551808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] adjust_need_up_thru now 194496, need_up_thru now false -5075> 2014-08-13 18:06:09.551831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Started advmap -5074> 2014-08-13 18:06:09.551868 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] check_recovery_sources no source osds () went down -5073> 2014-08-13 18:06:09.551896 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] handle_activate_map -5072> 2014-08-13 18:06:09.551916 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: handle ActMap primary -5071> 2014-08-13 18:06:09.551936 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] publish_stats_to_osd 194497:29710 -5070> 2014-08-13 18:06:09.551954 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] take_waiters -5069> 2014-08-13 18:06:09.551973 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering/WaitUpThru 1.207095 3 0.000660 -5068> 2014-08-13 18:06:09.552001 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] state: Leaving Peering -5067> 2014-08-13 18:06:09.552024 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped+peering m=57] exit Started/Primary/Peering 1.336617 0 0.000000 -5066> 2014-08-13 18:06:09.551738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24 194377'17228 also missing on osd.378 (past last_backfill 0//0//-1) -5065> 2014-08-13 18:06:09.552050 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active -5064> 2014-08-13 18:06:09.552068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: In Active, about to call activate -5063> 2014-08-13 18:06:09.552098 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - purged_snaps [] cached_removed_snaps [] -5062> 2014-08-13 18:06:09.552115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - snap_trimq [] -5061> 2014-08-13 18:06:09.552132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - not complete, missing(57) -5060> 2014-08-13 18:06:09.551987 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -5059> 2014-08-13 18:06:09.552044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24 194383'17244 also missing on osd.378 (past last_backfill 0//0//-1) -5058> 2014-08-13 18:06:09.552329 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] flushed -5057> 2014-08-13 18:06:09.552354 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24 194416'17284 also missing on osd.378 (past last_backfill 0//0//-1) -5056> 2014-08-13 18:06:09.552658 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24 194377'17236 also missing on osd.378 (past last_backfill 0//0//-1) -5055> 2014-08-13 18:06:09.552965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24 194377'17234 also missing on osd.378 (past last_backfill 0//0//-1) -5054> 2014-08-13 18:06:09.553276 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24 194370'17214 also missing on osd.378 (past last_backfill 0//0//-1) -5053> 2014-08-13 18:06:09.553584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24 194416'17252 also missing on osd.378 (past last_backfill 0//0//-1) -5052> 2014-08-13 18:06:09.553890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24 194416'17260 also missing on osd.378 (past last_backfill 0//0//-1) -5051> 2014-08-13 18:06:09.554202 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24 194377'17230 also missing on osd.378 (past last_backfill 0//0//-1) -5050> 2014-08-13 18:06:09.554490 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate - starting recovery -5049> 2014-08-13 18:06:09.554527 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] -5048> 2014-08-13 18:06:09.554584 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -5047> 2014-08-13 18:06:09.554611 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 is up to date, queueing in pending_activators -5046> 2014-08-13 18:06:09.554696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.20 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -5045> 2014-08-13 18:06:09.554724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -5044> 2014-08-13 18:06:09.554753 7f5c7bde3700 0 log [INF] : 24.3d restarting backfill on osd.25 from (0'0,0'0] MAX to 194422'18731 -5043> 2014-08-13 18:06:09.554509 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24 194377'17218 also missing on osd.378 (past last_backfill 0//0//-1) -5042> 2014-08-13 18:06:09.554817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24 194416'17248 also missing on osd.378 (past last_backfill 0//0//-1) -5041> 2014-08-13 18:06:09.555280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 sending log((192803'16165,194422'18731], crt=194422'18731) -5040> 2014-08-13 18:06:09.555317 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing -5039> 2014-08-13 18:06:09.555326 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.25 192.168.1.103:6800/36920 -- pg_log(24.3d epoch 194497 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194497) v4 -- ?+0 0x4166c00 -5038> 2014-08-13 18:06:09.555352 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.25 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -5037> 2014-08-13 18:06:09.555380 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) -5036> 2014-08-13 18:06:09.555402 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 is up to date, queueing in pending_activators -5035> 2014-08-13 18:06:09.555123 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24 194370'17196 also missing on osd.378 (past last_backfill 0//0//-1) -5034> 2014-08-13 18:06:09.555483 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.303 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -5033> 2014-08-13 18:06:09.555522 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -5032> 2014-08-13 18:06:09.555549 7f5c7bde3700 0 log [INF] : 24.3d restarting backfill on osd.362 from (0'0,0'0] MAX to 194422'18731 -5031> 2014-08-13 18:06:09.555433 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24 194416'17270 also missing on osd.378 (past last_backfill 0//0//-1) -5030> 2014-08-13 18:06:09.556000 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 sending log((192803'16165,194422'18731], crt=194422'18731) -5029> 2014-08-13 18:06:09.556032 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing -5028> 2014-08-13 18:06:09.555740 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24 194380'17240 also missing on osd.378 (past last_backfill 0//0//-1) -5027> 2014-08-13 18:06:09.556036 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.362 192.168.1.108:6842/8170 -- pg_log(24.3d epoch 194497 log log((192803'16165,194422'18731], crt=194422'18731) query_epoch 194497) v4 -- ?+0 0x4163000 -5026> 2014-08-13 18:06:09.556059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate peer osd.362 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -5025> 2014-08-13 18:06:09.556141 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery primary has 57 missing -5024> 2014-08-13 18:06:09.556044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24 194416'17250 also missing on osd.378 (past last_backfill 0//0//-1) -5023> 2014-08-13 18:06:09.556282 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.4 -5022> 2014-08-13 18:06:09.556416 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.4 -5021> 2014-08-13 18:06:09.556353 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24 194416'17256 also missing on osd.378 (past last_backfill 0//0//-1) -5020> 2014-08-13 18:06:09.556557 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.4 -5019> 2014-08-13 18:06:09.556678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.4 -5018> 2014-08-13 18:06:09.556800 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.4 -5017> 2014-08-13 18:06:09.556660 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24 194370'17200 also missing on osd.378 (past last_backfill 0//0//-1) -5016> 2014-08-13 18:06:09.556921 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.4 -5015> 2014-08-13 18:06:09.557043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.4 -5014> 2014-08-13 18:06:09.556967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24 194422'17302 also missing on osd.378 (past last_backfill 0//0//-1) -5013> 2014-08-13 18:06:09.557179 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.4 -5012> 2014-08-13 18:06:09.557299 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.4 -5011> 2014-08-13 18:06:09.557425 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.4 -5010> 2014-08-13 18:06:09.557278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24 194422'17292 also missing on osd.378 (past last_backfill 0//0//-1) -5009> 2014-08-13 18:06:09.557573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.4 -5008> 2014-08-13 18:06:09.557697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.4 -5007> 2014-08-13 18:06:09.557583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24 194377'17232 also missing on osd.378 (past last_backfill 0//0//-1) -5006> 2014-08-13 18:06:09.557818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.4 -5005> 2014-08-13 18:06:09.557940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.4 -5004> 2014-08-13 18:06:09.557889 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24 194416'17278 also missing on osd.378 (past last_backfill 0//0//-1) -5003> 2014-08-13 18:06:09.558062 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.4 -5002> 2014-08-13 18:06:09.558204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.4 -5001> 2014-08-13 18:06:09.558332 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.4 -5000> 2014-08-13 18:06:09.558200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24 194370'17202 also missing on osd.378 (past last_backfill 0//0//-1) -4999> 2014-08-13 18:06:09.558468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.4 -4998> 2014-08-13 18:06:09.558607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.4 -4997> 2014-08-13 18:06:09.558508 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24 194422'17294 also missing on osd.378 (past last_backfill 0//0//-1) -4996> 2014-08-13 18:06:09.558729 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.4 -4995> 2014-08-13 18:06:09.558848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.4 -4994> 2014-08-13 18:06:09.558969 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.4 -4993> 2014-08-13 18:06:09.558815 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24 194370'17188 also missing on osd.378 (past last_backfill 0//0//-1) -4992> 2014-08-13 18:06:09.559090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.4 -4991> 2014-08-13 18:06:09.559222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.4 -4990> 2014-08-13 18:06:09.559119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24 194370'17208 also missing on osd.378 (past last_backfill 0//0//-1) -4989> 2014-08-13 18:06:09.559345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.4 -4988> 2014-08-13 18:06:09.559481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.4 -4987> 2014-08-13 18:06:09.559430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24 194377'17224 also missing on osd.378 (past last_backfill 0//0//-1) -4986> 2014-08-13 18:06:09.559736 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.4 -4985> 2014-08-13 18:06:09.559735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24 194377'17222 also missing on osd.378 (past last_backfill 0//0//-1) -4984> 2014-08-13 18:06:09.559938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.4 -4983> 2014-08-13 18:06:09.560059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.4 -4982> 2014-08-13 18:06:09.560189 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.4 -4981> 2014-08-13 18:06:09.560041 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24 194416'17272 also missing on osd.378 (past last_backfill 0//0//-1) -4980> 2014-08-13 18:06:09.560319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.4 -4979> 2014-08-13 18:06:09.560352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24 194370'17190 also missing on osd.378 (past last_backfill 0//0//-1) -4978> 2014-08-13 18:06:09.560533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.4 -4977> 2014-08-13 18:06:09.560758 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.4 -4976> 2014-08-13 18:06:09.560659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24 194416'17274 also missing on osd.378 (past last_backfill 0//0//-1) -4975> 2014-08-13 18:06:09.560881 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.4 -4974> 2014-08-13 18:06:09.561003 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.4 -4973> 2014-08-13 18:06:09.561124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.4 -4972> 2014-08-13 18:06:09.560966 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24 194416'17280 also missing on osd.378 (past last_backfill 0//0//-1) -4971> 2014-08-13 18:06:09.561257 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.4 -4970> 2014-08-13 18:06:09.561384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.4 -4969> 2014-08-13 18:06:09.561275 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24 194370'17216 also missing on osd.378 (past last_backfill 0//0//-1) -4968> 2014-08-13 18:06:09.561532 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.4 -4967> 2014-08-13 18:06:09.561655 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.4 -4966> 2014-08-13 18:06:09.561582 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24 194416'17262 also missing on osd.378 (past last_backfill 0//0//-1) -4965> 2014-08-13 18:06:09.561797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.4 -4964> 2014-08-13 18:06:09.561952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.4 -4963> 2014-08-13 18:06:09.561888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24 194378'17238 also missing on osd.378 (past last_backfill 0//0//-1) -4962> 2014-08-13 18:06:09.562097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.4 -4961> 2014-08-13 18:06:09.562228 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.4 -4960> 2014-08-13 18:06:09.562354 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.4 -4959> 2014-08-13 18:06:09.562199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24 194370'17192 also missing on osd.378 (past last_backfill 0//0//-1) -4958> 2014-08-13 18:06:09.562495 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.4 -4957> 2014-08-13 18:06:09.562618 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.4 -4956> 2014-08-13 18:06:09.562504 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24 194370'17194 also missing on osd.378 (past last_backfill 0//0//-1) -4955> 2014-08-13 18:06:09.562754 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.4 -4954> 2014-08-13 18:06:09.562893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.4 -4953> 2014-08-13 18:06:09.562811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24 194422'17300 also missing on osd.378 (past last_backfill 0//0//-1) -4952> 2014-08-13 18:06:09.563016 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.4 -4951> 2014-08-13 18:06:09.563135 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.4 -4950> 2014-08-13 18:06:09.563276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.4 -4949> 2014-08-13 18:06:09.563117 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24 194370'17204 also missing on osd.378 (past last_backfill 0//0//-1) -4948> 2014-08-13 18:06:09.563414 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.4 -4947> 2014-08-13 18:06:09.563545 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.4 -4946> 2014-08-13 18:06:09.563426 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24 194416'17264 also missing on osd.378 (past last_backfill 0//0//-1) -4945> 2014-08-13 18:06:09.563666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.4 -4944> 2014-08-13 18:06:09.563787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.4 -4943> 2014-08-13 18:06:09.563908 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.4 -4942> 2014-08-13 18:06:09.563734 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24 194377'17226 also missing on osd.378 (past last_backfill 0//0//-1) -4941> 2014-08-13 18:06:09.564026 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -4940> 2014-08-13 18:06:09.564229 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] activate: adding 20 as a source -4939> 2014-08-13 18:06:09.564038 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24 194370'17210 also missing on osd.378 (past last_backfill 0//0//-1) -4938> 2014-08-13 18:06:09.564358 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.20 -4937> 2014-08-13 18:06:09.564500 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=56] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.20 -4936> 2014-08-13 18:06:09.564362 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24 194422'17290 also missing on osd.378 (past last_backfill 0//0//-1) -4935> 2014-08-13 18:06:09.564654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=55] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.20 -4934> 2014-08-13 18:06:09.564788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=54] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.20 -4933> 2014-08-13 18:06:09.564669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24 194377'17220 also missing on osd.378 (past last_backfill 0//0//-1) -4932> 2014-08-13 18:06:09.564920 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=53] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.20 -4931> 2014-08-13 18:06:09.564976 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24 194370'17186 also missing on osd.378 (past last_backfill 0//0//-1) -4930> 2014-08-13 18:06:09.565056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=52] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.20 -4929> 2014-08-13 18:06:09.565287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24 194422'17288 also missing on osd.378 (past last_backfill 0//0//-1) -4928> 2014-08-13 18:06:09.565492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=51] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.20 -4927> 2014-08-13 18:06:09.565639 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=50] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.20 -4926> 2014-08-13 18:06:09.565594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24 194422'17298 also missing on osd.378 (past last_backfill 0//0//-1) -4925> 2014-08-13 18:06:09.565784 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=49] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.20 -4924> 2014-08-13 18:06:09.565929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=48] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.20 -4923> 2014-08-13 18:06:09.565900 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24 194416'17266 also missing on osd.378 (past last_backfill 0//0//-1) -4922> 2014-08-13 18:06:09.566078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=47] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.20 -4921> 2014-08-13 18:06:09.566240 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=46] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.20 -4920> 2014-08-13 18:06:09.566211 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24 194416'17258 also missing on osd.378 (past last_backfill 0//0//-1) -4919> 2014-08-13 18:06:09.566403 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=45] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.20 -4918> 2014-08-13 18:06:09.566583 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=44] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.20 -4917> 2014-08-13 18:06:09.566518 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24 194370'17198 also missing on osd.378 (past last_backfill 0//0//-1) -4916> 2014-08-13 18:06:09.566750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=43] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.20 -4915> 2014-08-13 18:06:09.566919 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=42] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.20 -4914> 2014-08-13 18:06:09.566824 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24 194370'17206 also missing on osd.378 (past last_backfill 0//0//-1) -4913> 2014-08-13 18:06:09.567089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=41] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.20 -4912> 2014-08-13 18:06:09.567130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24 194422'17296 also missing on osd.378 (past last_backfill 0//0//-1) -4911> 2014-08-13 18:06:09.567270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=40] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.20 -4910> 2014-08-13 18:06:09.567452 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=39] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.20 -4909> 2014-08-13 18:06:09.567442 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24 194416'17268 also missing on osd.378 (past last_backfill 0//0//-1) -4908> 2014-08-13 18:06:09.567648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=38] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.20 -4907> 2014-08-13 18:06:09.567831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=37] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.20 -4906> 2014-08-13 18:06:09.567749 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24 194422'17286 also missing on osd.378 (past last_backfill 0//0//-1) -4905> 2014-08-13 18:06:09.568015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=36] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.20 -4904> 2014-08-13 18:06:09.568056 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24 194416'17254 also missing on osd.378 (past last_backfill 0//0//-1) -4903> 2014-08-13 18:06:09.568237 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=35] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.20 -4902> 2014-08-13 18:06:09.568450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=34] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.20 -4901> 2014-08-13 18:06:09.568366 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24 194416'17276 also missing on osd.378 (past last_backfill 0//0//-1) -4900> 2014-08-13 18:06:09.568658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=33] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.20 -4899> 2014-08-13 18:06:09.568671 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24 194370'17212 also missing on osd.378 (past last_backfill 0//0//-1) -4898> 2014-08-13 18:06:09.568856 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=32] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.20 -4897> 2014-08-13 18:06:09.569056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=31] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.20 -4896> 2014-08-13 18:06:09.568983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24 194383'17242 also missing on osd.378 (past last_backfill 0//0//-1) -4895> 2014-08-13 18:06:09.569269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=30] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.20 -4894> 2014-08-13 18:06:09.569292 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {9b/hit_set_24.9b_archive_2014-08-12 20:37:08.991323_2014-08-13 15:28:33.799047/head/.ceph-internal/24=194416'17245,fe04009b/benchmark_data_storage0111-ib.pocstor_19103_object13356/head//24=194416'17282,8554009b/benchmark_data_storage0111-ib.pocstor_25065_object2635/head//24=194377'17228,1994009b/benchmark_data_storage0111-ib.pocstor_25065_object7345/head//24=194383'17244,5ad7109b/benchmark_data_storage0111-ib.pocstor_19103_object17770/head//24=194416'17284,3c38209b/benchmark_data_storage0111-ib.pocstor_25065_object4806/head//24=194377'17236,ef05309b/benchmark_data_storage0111-ib.pocstor_25065_object4435/head//24=194377'17234,d216509b/benchmark_data_storage0111-ib.pocstor_12495_object13880/head//24=194370'17214,a279609b/benchmark_data_storage0111-ib.pocstor_19103_object3486/head//24=194416'17252,7b74709b/benchmark_data_storage0111-ib.pocstor_19103_object7900/head//24=194416'17260,58da909b/benchmark_data_storage0111-ib.pocstor_25065_object3079/head//24=194377'17230,d74da09b/benchmark_data_storage0111-ib.pocstor_25065_object871/head//24=194377'17218,3448b09b/benchmark_data_storage0111-ib.pocstor_19103_object825/head//24=194416'17248,9e39b09b/benchmark_data_storage0111-ib.pocstor_12495_object8354/head//24=194370'17196,f37ab09b/benchmark_data_storage0111-ib.pocstor_19103_object11633/head//24=194416'17270,f8a8c09b/benchmark_data_storage0111-ib.pocstor_25065_object5342/head//24=194380'17240,5ee8c09b/benchmark_data_storage0111-ib.pocstor_19103_object2790/head//24=194416'17250,8c9fe09b/benchmark_data_storage0111-ib.pocstor_19103_object6844/head//24=194416'17256,38bfe09b/benchmark_data_storage0111-ib.pocstor_12495_object9227/head//24=194370'17200,a260149b/benchmark_data_storage0111-ib.pocstor_41231_object9916/head//24=194422'17302,564d249b/benchmark_data_storage0111-ib.pocstor_41231_object1426/head//24=194422'17292,22ae449b/benchmark_data_storage0111-ib.pocstor_25065_object4378/head//24=194377'17232,9d94549b/benchmark_data_storage0111-ib.pocstor_19103_object12801/head//24=194416'17278,efe7549b/benchmark_data_storage0111-ib.pocstor_12495_object9901/head//24=194370'17202,f8d9549b/benchmark_data_storage0111-ib.pocstor_41231_object3539/head//24=194422'17294,83aa549b/benchmark_data_storage0111-ib.pocstor_12495_object1198/head//24=194370'17188,ebfd649b/benchmark_data_storage0111-ib.pocstor_12495_object11193/head//24=194370'17208,6693849b/benchmark_data_storage0111-ib.pocstor_25065_object1486/head//24=194377'17224,6897849b/benchmark_data_storage0111-ib.pocstor_25065_object1352/head//24=194377'17222,a0ba849b/benchmark_data_storage0111-ib.pocstor_19103_object12183/head//24=194416'17272,b99ba49b/benchmark_data_storage0111-ib.pocstor_12495_object3589/head//24=194370'17190,322dc49b/benchmark_data_storage0111-ib.pocstor_19103_object12370/head//24=194416'17274,f8f9d49b/benchmark_data_storage0111-ib.pocstor_19103_object13195/head//24=194416'17280,b2eae49b/benchmark_data_storage0111-ib.pocstor_12495_object13993/head//24=194370'17216,6882189b/benchmark_data_storage0111-ib.pocstor_19103_object8309/head//24=194416'17262,2ba7389b/benchmark_data_storage0111-ib.pocstor_25065_object4865/head//24=194378'17238,c21b489b/benchmark_data_storage0111-ib.pocstor_12495_object4722/head//24=194370'17192,e817589b/benchmark_data_storage0111-ib.pocstor_12495_object6347/head//24=194370'17194,9dae589b/benchmark_data_storage0111-ib.pocstor_41231_object9579/head//24=194422'17300,61d6689b/benchmark_data_storage0111-ib.pocstor_12495_object10710/head//24=194370'17204,a4af789b/benchmark_data_storage0111-ib.pocstor_19103_object9361/head//24=194416'17264,b3a8889b/benchmark_data_storage0111-ib.pocstor_25065_object1995/head//24=194377'17226,9427989b/benchmark_data_storage0111-ib.pocstor_12495_object12614/head//24=194370'17210,56e5f89b/benchmark_data_storage0111-ib.pocstor_41231_object1038/head//24=194422'17290,b63ff89b/benchmark_data_storage0111-ib.pocstor_25065_object1272/head//24=194377'17220,76300c9b/benchmark_data_storage0111-ib.pocstor_12495_object156/head//24=194370'17186,4fe42c9b/benchmark_data_storage0111-ib.pocstor_41231_object912/head//24=194422'17288,18e2c9b/benchmark_data_storage0111-ib.pocstor_41231_object5452/head//24=194422'17298,55064c9b/benchmark_data_storage0111-ib.pocstor_19103_object9759/head//24=194416'17266,b8794c9b/benchmark_data_storage0111-ib.pocstor_19103_object7844/head//24=194416'17258,9b547c9b/benchmark_data_storage0111-ib.pocstor_12495_object8510/head//24=194370'17198,29de7c9b/benchmark_data_storage0111-ib.pocstor_12495_object11066/head//24=194370'17206,69f08c9b/benchmark_data_storage0111-ib.pocstor_41231_object5076/head//24=194422'17296,c65b8c9b/benchmark_data_storage0111-ib.pocstor_19103_object10261/head//24=194416'17268,5225ac9b/benchmark_data_storage0111-ib.pocstor_41231_object120/head//24=194422'17286,3e3cac9b/benchmark_data_storage0111-ib.pocstor_19103_object5999/head//24=194416'17254,d724cc9b/benchmark_data_storage0111-ib.pocstor_19103_object12774/head//24=194416'17276,ff13fc9b/benchmark_data_storage0111-ib.pocstor_12495_object13271/head//24=194370'17212,ab6fc9b/benchmark_data_storage0111-ib.pocstor_25065_object6318/head//24=194383'17242} -4893> 2014-08-13 18:06:09.569501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=29] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.20 -4892> 2014-08-13 18:06:09.569710 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=28] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.20 -4891> 2014-08-13 18:06:09.569672 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound -4890> 2014-08-13 18:06:09.569921 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=27] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.20 -4889> 2014-08-13 18:06:09.569977 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194493 -4888> 2014-08-13 18:06:09.570137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=26] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.20 -4887> 2014-08-13 18:06:09.570290 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 291,350,365,367,378 -4886> 2014-08-13 18:06:09.570363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=25] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.20 -4885> 2014-08-13 18:06:09.570616 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=24] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.20 -4884> 2014-08-13 18:06:09.570598 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished -4883> 2014-08-13 18:06:09.570843 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=23] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.20 -4882> 2014-08-13 18:06:09.570906 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -4881> 2014-08-13 18:06:09.571071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=22] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.20 -4880> 2014-08-13 18:06:09.571218 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4879> 2014-08-13 18:06:09.571316 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=21] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.20 -4878> 2014-08-13 18:06:09.571578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=20] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.20 -4877> 2014-08-13 18:06:09.571525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4876> 2014-08-13 18:06:09.571920 7f5c7c7e4700 10 log is not dirty -4875> 2014-08-13 18:06:09.571980 7f5c835ef700 5 -- op tracker -- , seq: 597, time: 2014-08-13 18:06:09.571979, event: done, request: pg_notify(24.9b(39) epoch 194497) v5 -4874> 2014-08-13 18:06:09.571986 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_advance_map [320,4,30]/[320,4,30] -- 320/320 -4873> 2014-08-13 18:06:09.572014 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4872> 2014-08-13 18:06:09.572032 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4871> 2014-08-13 18:06:09.572022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] state: Started advmap -4870> 2014-08-13 18:06:09.572038 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4869> 2014-08-13 18:06:09.571816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=19] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.20 -4868> 2014-08-13 18:06:09.572041 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] check_recovery_sources no source osds () went down -4867> 2014-08-13 18:06:09.572048 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.381 192.168.1.102:6822/15390 7 ==== pg_notify(22.481(49) epoch 194497) v5 ==== 3465+0+0 (4066386355 0 0) 0x5a381c0 con 0x4c5eb40 -4866> 2014-08-13 18:06:09.572069 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4865> 2014-08-13 18:06:09.572072 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4864> 2014-08-13 18:06:09.572060 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map -4863> 2014-08-13 18:06:09.572075 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a381c0 pg_notify(22.481(49) epoch 194497) v5 -4862> 2014-08-13 18:06:09.572082 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.312231, event: header_read, request: pg_notify(22.481(49) epoch 194497) v5 -4861> 2014-08-13 18:06:09.572079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] take_waiters -4860> 2014-08-13 18:06:09.572091 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.312235, event: throttled, request: pg_notify(22.481(49) epoch 194497) v5 -4859> 2014-08-13 18:06:09.572098 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.312399, event: all_read, request: pg_notify(22.481(49) epoch 194497) v5 -4858> 2014-08-13 18:06:09.572095 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4857> 2014-08-13 18:06:09.572104 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.572068, event: dispatched, request: pg_notify(22.481(49) epoch 194497) v5 -4856> 2014-08-13 18:06:09.572118 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.572118, event: waiting_for_osdmap, request: pg_notify(22.481(49) epoch 194497) v5 -4855> 2014-08-13 18:06:09.572112 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.581( v 194215'527 (0'0,194215'527] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/192979) [320,4,30] r=1 lpr=194495 pi=192979-194494/34 luod=0'0 crt=192888'497 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4854> 2014-08-13 18:06:09.572125 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.381 -4853> 2014-08-13 18:06:09.572130 7f5c7c7e4700 10 log is not dirty -4852> 2014-08-13 18:06:09.572129 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a381c0 -4851> 2014-08-13 18:06:09.572136 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.572135, event: started, request: pg_notify(22.481(49) epoch 194497) v5 -4850> 2014-08-13 18:06:09.572161 7f5c835ef700 5 -- op tracker -- , seq: 598, time: 2014-08-13 18:06:09.572161, event: done, request: pg_notify(22.481(49) epoch 194497) v5 -4849> 2014-08-13 18:06:09.572154 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,312,285]/[4,312,285] -- 4/4 -4848> 2014-08-13 18:06:09.572177 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4847> 2014-08-13 18:06:09.572174 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4846> 2014-08-13 18:06:09.572188 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4845> 2014-08-13 18:06:09.572183 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4844> 2014-08-13 18:06:09.572208 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4843> 2014-08-13 18:06:09.572210 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.369 192.168.1.108:6824/4037 13 ==== pg_notify(22.3af(36) epoch 194497) v5 ==== 2738+0+0 (3958580385 0 0) 0x588b9c0 con 0x4c5e880 -4842> 2014-08-13 18:06:09.572227 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4841> 2014-08-13 18:06:09.572230 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4840> 2014-08-13 18:06:09.572232 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b9c0 pg_notify(22.3af(36) epoch 194497) v5 -4839> 2014-08-13 18:06:09.572227 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,312) went down -4838> 2014-08-13 18:06:09.572238 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.296320, event: header_read, request: pg_notify(22.3af(36) epoch 194497) v5 -4837> 2014-08-13 18:06:09.572246 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.296321, event: throttled, request: pg_notify(22.3af(36) epoch 194497) v5 -4836> 2014-08-13 18:06:09.572253 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.296439, event: all_read, request: pg_notify(22.3af(36) epoch 194497) v5 -4835> 2014-08-13 18:06:09.572247 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4834> 2014-08-13 18:06:09.572259 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.572226, event: dispatched, request: pg_notify(22.3af(36) epoch 194497) v5 -4833> 2014-08-13 18:06:09.572266 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.572266, event: waiting_for_osdmap, request: pg_notify(22.3af(36) epoch 194497) v5 -4832> 2014-08-13 18:06:09.572272 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 -4831> 2014-08-13 18:06:09.572275 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b9c0 -4830> 2014-08-13 18:06:09.572263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4829> 2014-08-13 18:06:09.572281 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.572280, event: started, request: pg_notify(22.3af(36) epoch 194497) v5 -4828> 2014-08-13 18:06:09.572319 7f5c835ef700 5 -- op tracker -- , seq: 599, time: 2014-08-13 18:06:09.572318, event: done, request: pg_notify(22.3af(36) epoch 194497) v5 -4827> 2014-08-13 18:06:09.572056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=18] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.20 -4826> 2014-08-13 18:06:09.572349 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4825> 2014-08-13 18:06:09.572353 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4824> 2014-08-13 18:06:09.572357 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.375 192.168.1.114:6830/52879 8 ==== pg_notify(24.30b(45) epoch 194497) v5 ==== 3299+0+0 (1541005296 0 0) 0x58881c0 con 0x4c5e5c0 -4823> 2014-08-13 18:06:09.572371 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4822> 2014-08-13 18:06:09.572374 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4821> 2014-08-13 18:06:09.572376 7f5c835ef700 20 osd.4 194497 _dispatch 0x58881c0 pg_notify(24.30b(45) epoch 194497) v5 -4820> 2014-08-13 18:06:09.572381 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.310177, event: header_read, request: pg_notify(24.30b(45) epoch 194497) v5 -4819> 2014-08-13 18:06:09.572390 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.310180, event: throttled, request: pg_notify(24.30b(45) epoch 194497) v5 -4818> 2014-08-13 18:06:09.572396 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.310325, event: all_read, request: pg_notify(24.30b(45) epoch 194497) v5 -4817> 2014-08-13 18:06:09.572402 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.572371, event: dispatched, request: pg_notify(24.30b(45) epoch 194497) v5 -4816> 2014-08-13 18:06:09.572409 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.572408, event: waiting_for_osdmap, request: pg_notify(24.30b(45) epoch 194497) v5 -4815> 2014-08-13 18:06:09.572414 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.375 -4814> 2014-08-13 18:06:09.572417 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x58881c0 -4813> 2014-08-13 18:06:09.572427 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.572427, event: started, request: pg_notify(24.30b(45) epoch 194497) v5 -4812> 2014-08-13 18:06:09.572448 7f5c835ef700 5 -- op tracker -- , seq: 600, time: 2014-08-13 18:06:09.572448, event: done, request: pg_notify(24.30b(45) epoch 194497) v5 -4811> 2014-08-13 18:06:09.572459 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4810> 2014-08-13 18:06:09.572462 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4809> 2014-08-13 18:06:09.572487 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.302 192.168.1.113:6810/15731 7 ==== pg_notify(22.22c(1) epoch 194497) v5 ==== 777+0+0 (3911413569 0 0) 0x5a6af40 con 0x4f218c0 -4808> 2014-08-13 18:06:09.572501 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4807> 2014-08-13 18:06:09.572504 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4806> 2014-08-13 18:06:09.572506 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6af40 pg_notify(22.22c(1) epoch 194497) v5 -4805> 2014-08-13 18:06:09.572511 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.222867, event: header_read, request: pg_notify(22.22c(1) epoch 194497) v5 -4804> 2014-08-13 18:06:09.572518 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.222868, event: throttled, request: pg_notify(22.22c(1) epoch 194497) v5 -4803> 2014-08-13 18:06:09.572524 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.222935, event: all_read, request: pg_notify(22.22c(1) epoch 194497) v5 -4802> 2014-08-13 18:06:09.572530 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.572501, event: dispatched, request: pg_notify(22.22c(1) epoch 194497) v5 -4801> 2014-08-13 18:06:09.572536 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.572536, event: waiting_for_osdmap, request: pg_notify(22.22c(1) epoch 194497) v5 -4800> 2014-08-13 18:06:09.572542 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.302 -4799> 2014-08-13 18:06:09.572545 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6af40 -4798> 2014-08-13 18:06:09.572550 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.572550, event: started, request: pg_notify(22.22c(1) epoch 194497) v5 -4797> 2014-08-13 18:06:09.572569 7f5c835ef700 5 -- op tracker -- , seq: 601, time: 2014-08-13 18:06:09.572569, event: done, request: pg_notify(22.22c(1) epoch 194497) v5 -4796> 2014-08-13 18:06:09.572576 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] -4795> 2014-08-13 18:06:09.572589 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4794> 2014-08-13 18:06:09.572593 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4793> 2014-08-13 18:06:09.572591 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4792> 2014-08-13 18:06:09.572599 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.346 192.168.1.104:6820/2308 8 ==== pg_info(1 pgs e194497:24.e2(2)) v4 ==== 866+0+0 (3966519667 0 0) 0x4df39c0 con 0x4f21600 -4791> 2014-08-13 18:06:09.572613 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4790> 2014-08-13 18:06:09.572616 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4789> 2014-08-13 18:06:09.572336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=17] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.20 -4788> 2014-08-13 18:06:09.572618 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df39c0 pg_info(1 pgs e194497:24.e2(2)) v4 -4787> 2014-08-13 18:06:09.572623 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.199963, event: header_read, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4786> 2014-08-13 18:06:09.572631 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.199964, event: throttled, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4785> 2014-08-13 18:06:09.572637 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.200035, event: all_read, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4784> 2014-08-13 18:06:09.572643 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.572613, event: dispatched, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4783> 2014-08-13 18:06:09.572649 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.572649, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4782> 2014-08-13 18:06:09.572655 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.e2(2)) v4 from osd.346 -4781> 2014-08-13 18:06:09.572660 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df39c0 -4780> 2014-08-13 18:06:09.572664 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.572664, event: started, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4779> 2014-08-13 18:06:09.572685 7f5c835ef700 5 -- op tracker -- , seq: 602, time: 2014-08-13 18:06:09.572685, event: done, request: pg_info(1 pgs e194497:24.e2(2)) v4 -4778> 2014-08-13 18:06:09.572698 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4777> 2014-08-13 18:06:09.572700 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4776> 2014-08-13 18:06:09.572706 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 12 ==== pg_notify(22.1d2(30) epoch 194497) v5 ==== 2400+0+0 (3736757544 0 0) 0x4bf5b00 con 0x4f20dc0 -4775> 2014-08-13 18:06:09.572718 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4774> 2014-08-13 18:06:09.572721 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4773> 2014-08-13 18:06:09.572723 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5b00 pg_notify(22.1d2(30) epoch 194497) v5 -4772> 2014-08-13 18:06:09.572728 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.197874, event: header_read, request: pg_notify(22.1d2(30) epoch 194497) v5 -4771> 2014-08-13 18:06:09.572736 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.197877, event: throttled, request: pg_notify(22.1d2(30) epoch 194497) v5 -4770> 2014-08-13 18:06:09.572742 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.198045, event: all_read, request: pg_notify(22.1d2(30) epoch 194497) v5 -4769> 2014-08-13 18:06:09.572747 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.572718, event: dispatched, request: pg_notify(22.1d2(30) epoch 194497) v5 -4768> 2014-08-13 18:06:09.572754 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.572754, event: waiting_for_osdmap, request: pg_notify(22.1d2(30) epoch 194497) v5 -4767> 2014-08-13 18:06:09.572760 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.330 -4766> 2014-08-13 18:06:09.572763 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5b00 -4765> 2014-08-13 18:06:09.572750 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2866 -4764> 2014-08-13 18:06:09.572768 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.572767, event: started, request: pg_notify(22.1d2(30) epoch 194497) v5 -4763> 2014-08-13 18:06:09.572768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4762> 2014-08-13 18:06:09.572789 7f5c835ef700 5 -- op tracker -- , seq: 603, time: 2014-08-13 18:06:09.572788, event: done, request: pg_notify(22.1d2(30) epoch 194497) v5 -4761> 2014-08-13 18:06:09.572784 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4760> 2014-08-13 18:06:09.572809 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4759> 2014-08-13 18:06:09.572812 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4758> 2014-08-13 18:06:09.572800 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4757> 2014-08-13 18:06:09.572819 7f5c7c7e4700 10 log is not dirty -4756> 2014-08-13 18:06:09.572817 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.365 192.168.1.102:6824/15623 10 ==== pg_notify(24.9b(2) epoch 194497) v5 ==== 870+0+0 (2877738972 0 0) 0x5a6d080 con 0x4fa2260 -4755> 2014-08-13 18:06:09.572831 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4754> 2014-08-13 18:06:09.572833 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4753> 2014-08-13 18:06:09.572835 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d080 pg_notify(24.9b(2) epoch 194497) v5 -4752> 2014-08-13 18:06:09.572840 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.295032, event: header_read, request: pg_notify(24.9b(2) epoch 194497) v5 -4751> 2014-08-13 18:06:09.572847 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.295033, event: throttled, request: pg_notify(24.9b(2) epoch 194497) v5 -4750> 2014-08-13 18:06:09.572852 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.295136, event: all_read, request: pg_notify(24.9b(2) epoch 194497) v5 -4749> 2014-08-13 18:06:09.572842 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_advance_map [345,4,374]/[345,4,374] -- 345/345 -4748> 2014-08-13 18:06:09.572858 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.572830, event: dispatched, request: pg_notify(24.9b(2) epoch 194497) v5 -4747> 2014-08-13 18:06:09.572865 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4746> 2014-08-13 18:06:09.572865 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.572864, event: waiting_for_osdmap, request: pg_notify(24.9b(2) epoch 194497) v5 -4745> 2014-08-13 18:06:09.572873 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.365 -4744> 2014-08-13 18:06:09.572876 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d080 -4743> 2014-08-13 18:06:09.572539 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -4742> 2014-08-13 18:06:09.572881 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.572881, event: started, request: pg_notify(24.9b(2) epoch 194497) v5 -4741> 2014-08-13 18:06:09.572872 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] state: Started advmap -4740> 2014-08-13 18:06:09.572622 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=16] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.20 -4739> 2014-08-13 18:06:09.572890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] check_recovery_sources no source osds () went down -4738> 2014-08-13 18:06:09.572910 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map -4737> 2014-08-13 18:06:09.572926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] take_waiters -4736> 2014-08-13 18:06:09.572940 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4735> 2014-08-13 18:06:09.572955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.116( v 194215'496 (0'0,194215'496] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/192981) [345,4,374] r=1 lpr=194495 pi=192981-194494/28 luod=0'0 crt=192888'464 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4734> 2014-08-13 18:06:09.572972 7f5c7c7e4700 10 log is not dirty -4733> 2014-08-13 18:06:09.572980 7f5c7c7e4700 20 osd.4 194497 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 291 -4732> 2014-08-13 18:06:09.572992 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing -4731> 2014-08-13 18:06:09.573009 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6820/25633 -- pg_info(1 pgs e194497:24.9b(36)) v4 -- ?+0 0x588d5c0 con 0x4424e60 -4730> 2014-08-13 18:06:09.573033 7f5c7c7e4700 20 osd.4 194497 Sending info 24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 350 -4729> 2014-08-13 18:06:09.573045 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing -4728> 2014-08-13 18:06:09.573058 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6832/53185 -- pg_info(1 pgs e194497:24.9b(36)) v4 -- ?+0 0x588bf00 con 0x4c5ee00 -4727> 2014-08-13 18:06:09.573141 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_advance_map [334,289,4]/[334,289,4] -- 334/334 -4726> 2014-08-13 18:06:09.573166 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4725> 2014-08-13 18:06:09.573173 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] state: Started advmap -4724> 2014-08-13 18:06:09.572887 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] flushed -4723> 2014-08-13 18:06:09.572903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=15] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.20 -4722> 2014-08-13 18:06:09.573198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] check_recovery_sources no source osds () went down -4721> 2014-08-13 18:06:09.573217 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map -4720> 2014-08-13 18:06:09.573233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] take_waiters -4719> 2014-08-13 18:06:09.573248 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4718> 2014-08-13 18:06:09.573263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.69f( v 194215'528 (0'0,194215'528] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192946) [334,289,4] r=2 lpr=194495 pi=192974-194494/27 luod=0'0 crt=192888'488 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4717> 2014-08-13 18:06:09.573264 7f5c835ef700 5 -- op tracker -- , seq: 604, time: 2014-08-13 18:06:09.573264, event: done, request: pg_notify(24.9b(2) epoch 194497) v5 -4716> 2014-08-13 18:06:09.573281 7f5c7c7e4700 10 log is not dirty -4715> 2014-08-13 18:06:09.573307 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4714> 2014-08-13 18:06:09.573311 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4713> 2014-08-13 18:06:09.573305 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,313,318]/[4,313,318] -- 4/4 -4712> 2014-08-13 18:06:09.573326 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4711> 2014-08-13 18:06:09.573318 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.378 192.168.1.102:6840/19195 5 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x5150d80 con 0x4fa1ce0 -4710> 2014-08-13 18:06:09.573336 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4709> 2014-08-13 18:06:09.573338 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4708> 2014-08-13 18:06:09.573341 7f5c835ef700 20 osd.4 194497 _dispatch 0x5150d80 osd_map(194497..194497 src has 185139..194497) v3 -4707> 2014-08-13 18:06:09.573332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4706> 2014-08-13 18:06:09.573351 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -4705> 2014-08-13 18:06:09.573356 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -4704> 2014-08-13 18:06:09.573359 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4703> 2014-08-13 18:06:09.573361 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4702> 2014-08-13 18:06:09.573349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4701> 2014-08-13 18:06:09.573369 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.366 192.168.1.108:6828/4534 7 ==== pg_notify(22.39d(1) epoch 194497) v5 ==== 777+0+0 (1486107671 0 0) 0x5a39880 con 0x4fa1a20 -4700> 2014-08-13 18:06:09.573369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4699> 2014-08-13 18:06:09.573383 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4698> 2014-08-13 18:06:09.573386 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4697> 2014-08-13 18:06:09.573388 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a39880 pg_notify(22.39d(1) epoch 194497) v5 -4696> 2014-08-13 18:06:09.573387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4695> 2014-08-13 18:06:09.573394 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.315720, event: header_read, request: pg_notify(22.39d(1) epoch 194497) v5 -4694> 2014-08-13 18:06:09.573409 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.315722, event: throttled, request: pg_notify(22.39d(1) epoch 194497) v5 -4693> 2014-08-13 18:06:09.573404 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4692> 2014-08-13 18:06:09.573415 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.315783, event: all_read, request: pg_notify(22.39d(1) epoch 194497) v5 -4691> 2014-08-13 18:06:09.573422 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.573382, event: dispatched, request: pg_notify(22.39d(1) epoch 194497) v5 -4690> 2014-08-13 18:06:09.573428 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.573428, event: waiting_for_osdmap, request: pg_notify(22.39d(1) epoch 194497) v5 -4689> 2014-08-13 18:06:09.573434 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.366 -4688> 2014-08-13 18:06:09.573437 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a39880 -4687> 2014-08-13 18:06:09.573442 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.573442, event: started, request: pg_notify(22.39d(1) epoch 194497) v5 -4686> 2014-08-13 18:06:09.573465 7f5c835ef700 5 -- op tracker -- , seq: 605, time: 2014-08-13 18:06:09.573465, event: done, request: pg_notify(22.39d(1) epoch 194497) v5 -4685> 2014-08-13 18:06:09.573476 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4684> 2014-08-13 18:06:09.573479 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4683> 2014-08-13 18:06:09.573485 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.306 192.168.1.101:6806/6812 7 ==== pg_notify(22.481(47) epoch 194497) v5 ==== 3363+0+0 (1550049721 0 0) 0x5356e40 con 0x4fa5800 -4682> 2014-08-13 18:06:09.573499 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4681> 2014-08-13 18:06:09.573501 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4680> 2014-08-13 18:06:09.573503 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356e40 pg_notify(22.481(47) epoch 194497) v5 -4679> 2014-08-13 18:06:09.573509 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.311262, event: header_read, request: pg_notify(22.481(47) epoch 194497) v5 -4678> 2014-08-13 18:06:09.573515 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.311263, event: throttled, request: pg_notify(22.481(47) epoch 194497) v5 -4677> 2014-08-13 18:06:09.573521 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.311423, event: all_read, request: pg_notify(22.481(47) epoch 194497) v5 -4676> 2014-08-13 18:06:09.573526 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.573498, event: dispatched, request: pg_notify(22.481(47) epoch 194497) v5 -4675> 2014-08-13 18:06:09.573533 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.573533, event: waiting_for_osdmap, request: pg_notify(22.481(47) epoch 194497) v5 -4674> 2014-08-13 18:06:09.573218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=14] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.20 -4673> 2014-08-13 18:06:09.573539 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.306 -4672> 2014-08-13 18:06:09.573542 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356e40 -4671> 2014-08-13 18:06:09.573547 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.573547, event: started, request: pg_notify(22.481(47) epoch 194497) v5 -4670> 2014-08-13 18:06:09.573567 7f5c835ef700 5 -- op tracker -- , seq: 606, time: 2014-08-13 18:06:09.573566, event: done, request: pg_notify(22.481(47) epoch 194497) v5 -4669> 2014-08-13 18:06:09.573577 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4668> 2014-08-13 18:06:09.573580 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4667> 2014-08-13 18:06:09.573607 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.63 192.168.1.111:6830/12680 7 ==== pg_notify(22.775(36) epoch 194497) v5 ==== 2738+0+0 (362354066 0 0) 0x5a6aa00 con 0x4fa5280 -4666> 2014-08-13 18:06:09.573622 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4665> 2014-08-13 18:06:09.573624 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4664> 2014-08-13 18:06:09.573626 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6aa00 pg_notify(22.775(36) epoch 194497) v5 -4663> 2014-08-13 18:06:09.573632 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.218364, event: header_read, request: pg_notify(22.775(36) epoch 194497) v5 -4662> 2014-08-13 18:06:09.573639 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.218366, event: throttled, request: pg_notify(22.775(36) epoch 194497) v5 -4661> 2014-08-13 18:06:09.573644 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.218495, event: all_read, request: pg_notify(22.775(36) epoch 194497) v5 -4660> 2014-08-13 18:06:09.573650 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.573621, event: dispatched, request: pg_notify(22.775(36) epoch 194497) v5 -4659> 2014-08-13 18:06:09.573656 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.573656, event: waiting_for_osdmap, request: pg_notify(22.775(36) epoch 194497) v5 -4658> 2014-08-13 18:06:09.573662 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.63 -4657> 2014-08-13 18:06:09.573665 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6aa00 -4656> 2014-08-13 18:06:09.573669 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.573669, event: started, request: pg_notify(22.775(36) epoch 194497) v5 -4655> 2014-08-13 18:06:09.573689 7f5c835ef700 5 -- op tracker -- , seq: 607, time: 2014-08-13 18:06:09.573688, event: done, request: pg_notify(22.775(36) epoch 194497) v5 -4654> 2014-08-13 18:06:09.573709 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4653> 2014-08-13 18:06:09.573713 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4652> 2014-08-13 18:06:09.573717 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.349 192.168.1.104:6800/61726 9 ==== pg_notify(22.362(34) epoch 194497) v5 ==== 2624+0+0 (251435578 0 0) 0x5a69180 con 0x4fa4fc0 -4651> 2014-08-13 18:06:09.573731 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4650> 2014-08-13 18:06:09.573733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4649> 2014-08-13 18:06:09.573735 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69180 pg_notify(22.362(34) epoch 194497) v5 -4648> 2014-08-13 18:06:09.573740 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.295045, event: header_read, request: pg_notify(22.362(34) epoch 194497) v5 -4647> 2014-08-13 18:06:09.573737 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] -4646> 2014-08-13 18:06:09.573747 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.295047, event: throttled, request: pg_notify(22.362(34) epoch 194497) v5 -4645> 2014-08-13 18:06:09.573754 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.295166, event: all_read, request: pg_notify(22.362(34) epoch 194497) v5 -4644> 2014-08-13 18:06:09.573760 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.573730, event: dispatched, request: pg_notify(22.362(34) epoch 194497) v5 -4643> 2014-08-13 18:06:09.573752 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4642> 2014-08-13 18:06:09.573767 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.573766, event: waiting_for_osdmap, request: pg_notify(22.362(34) epoch 194497) v5 -4641> 2014-08-13 18:06:09.573773 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.349 -4640> 2014-08-13 18:06:09.573776 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69180 -4639> 2014-08-13 18:06:09.573780 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.573780, event: started, request: pg_notify(22.362(34) epoch 194497) v5 -4638> 2014-08-13 18:06:09.573799 7f5c835ef700 5 -- op tracker -- , seq: 608, time: 2014-08-13 18:06:09.573799, event: done, request: pg_notify(22.362(34) epoch 194497) v5 -4637> 2014-08-13 18:06:09.573818 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4636> 2014-08-13 18:06:09.573821 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4635> 2014-08-13 18:06:09.573825 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.382 192.168.1.108:6800/30527 6 ==== pg_notify(22.3d9(54) epoch 194497) v5 ==== 3660+0+0 (285260943 0 0) 0x588d940 con 0x4fa4d00 -4634> 2014-08-13 18:06:09.573839 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4633> 2014-08-13 18:06:09.573843 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4632> 2014-08-13 18:06:09.573846 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d940 pg_notify(22.3d9(54) epoch 194497) v5 -4631> 2014-08-13 18:06:09.573852 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.218090, event: header_read, request: pg_notify(22.3d9(54) epoch 194497) v5 -4630> 2014-08-13 18:06:09.573543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=13] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.20 -4629> 2014-08-13 18:06:09.573859 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.218091, event: throttled, request: pg_notify(22.3d9(54) epoch 194497) v5 -4628> 2014-08-13 18:06:09.573865 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.218269, event: all_read, request: pg_notify(22.3d9(54) epoch 194497) v5 -4627> 2014-08-13 18:06:09.573871 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.573839, event: dispatched, request: pg_notify(22.3d9(54) epoch 194497) v5 -4626> 2014-08-13 18:06:09.573877 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.573877, event: waiting_for_osdmap, request: pg_notify(22.3d9(54) epoch 194497) v5 -4625> 2014-08-13 18:06:09.573883 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.382 -4624> 2014-08-13 18:06:09.573886 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d940 -4623> 2014-08-13 18:06:09.573890 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.573890, event: started, request: pg_notify(22.3d9(54) epoch 194497) v5 -4622> 2014-08-13 18:06:09.573910 7f5c835ef700 5 -- op tracker -- , seq: 609, time: 2014-08-13 18:06:09.573909, event: done, request: pg_notify(22.3d9(54) epoch 194497) v5 -4621> 2014-08-13 18:06:09.573922 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4620> 2014-08-13 18:06:09.573925 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4619> 2014-08-13 18:06:09.573923 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2711 -4618> 2014-08-13 18:06:09.573931 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.26 192.168.1.115:6802/58522 5 ==== pg_notify(24.34(41) epoch 194497) v5 ==== 2843+0+0 (118683081 0 0) 0x4df1880 con 0x3a7e9e0 -4617> 2014-08-13 18:06:09.573945 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4616> 2014-08-13 18:06:09.573947 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4615> 2014-08-13 18:06:09.573949 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df1880 pg_notify(24.34(41) epoch 194497) v5 -4614> 2014-08-13 18:06:09.573940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4613> 2014-08-13 18:06:09.573954 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.234945, event: header_read, request: pg_notify(24.34(41) epoch 194497) v5 -4612> 2014-08-13 18:06:09.573962 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.234946, event: throttled, request: pg_notify(24.34(41) epoch 194497) v5 -4611> 2014-08-13 18:06:09.573957 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4610> 2014-08-13 18:06:09.573967 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.235083, event: all_read, request: pg_notify(24.34(41) epoch 194497) v5 -4609> 2014-08-13 18:06:09.573974 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.573944, event: dispatched, request: pg_notify(24.34(41) epoch 194497) v5 -4608> 2014-08-13 18:06:09.573981 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.573981, event: waiting_for_osdmap, request: pg_notify(24.34(41) epoch 194497) v5 -4607> 2014-08-13 18:06:09.573974 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4606> 2014-08-13 18:06:09.573987 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.26 -4605> 2014-08-13 18:06:09.573990 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df1880 -4604> 2014-08-13 18:06:09.573997 7f5c7c7e4700 10 log is not dirty -4603> 2014-08-13 18:06:09.573995 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.573995, event: started, request: pg_notify(24.34(41) epoch 194497) v5 -4602> 2014-08-13 18:06:09.574016 7f5c835ef700 5 -- op tracker -- , seq: 610, time: 2014-08-13 18:06:09.574016, event: done, request: pg_notify(24.34(41) epoch 194497) v5 -4601> 2014-08-13 18:06:09.574019 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [17,4,395]/[17,4,395] -- 17/17 -4600> 2014-08-13 18:06:09.574043 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4599> 2014-08-13 18:06:09.574047 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4598> 2014-08-13 18:06:09.574049 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4597> 2014-08-13 18:06:09.574052 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.307 192.168.1.109:6804/31016 5 ==== pg_notify(24.181(45) epoch 194497) v5 ==== 3283+0+0 (1088360767 0 0) 0x5889500 con 0x43f3700 -4596> 2014-08-13 18:06:09.574066 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4595> 2014-08-13 18:06:09.574069 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4594> 2014-08-13 18:06:09.574055 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: Started advmap -4593> 2014-08-13 18:06:09.574071 7f5c835ef700 20 osd.4 194497 _dispatch 0x5889500 pg_notify(24.181(45) epoch 194497) v5 -4592> 2014-08-13 18:06:09.574076 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.309827, event: header_read, request: pg_notify(24.181(45) epoch 194497) v5 -4591> 2014-08-13 18:06:09.574083 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.309828, event: throttled, request: pg_notify(24.181(45) epoch 194497) v5 -4590> 2014-08-13 18:06:09.574074 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down -4589> 2014-08-13 18:06:09.574089 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.310060, event: all_read, request: pg_notify(24.181(45) epoch 194497) v5 -4588> 2014-08-13 18:06:09.574095 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.574065, event: dispatched, request: pg_notify(24.181(45) epoch 194497) v5 -4587> 2014-08-13 18:06:09.574091 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map -4586> 2014-08-13 18:06:09.574373 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.574373, event: waiting_for_osdmap, request: pg_notify(24.181(45) epoch 194497) v5 -4585> 2014-08-13 18:06:09.574384 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.307 -4584> 2014-08-13 18:06:09.574388 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5889500 -4583> 2014-08-13 18:06:09.574393 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.574393, event: started, request: pg_notify(24.181(45) epoch 194497) v5 -4582> 2014-08-13 18:06:09.574414 7f5c835ef700 5 -- op tracker -- , seq: 611, time: 2014-08-13 18:06:09.574414, event: done, request: pg_notify(24.181(45) epoch 194497) v5 -4581> 2014-08-13 18:06:09.574425 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4580> 2014-08-13 18:06:09.574429 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4579> 2014-08-13 18:06:09.573861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=12] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.20 -4578> 2014-08-13 18:06:09.574445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] take_waiters -4577> 2014-08-13 18:06:09.574457 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.10 192.168.1.115:6815/60573 11 ==== pg_notify(24.30b(41) epoch 194497) v5 ==== 3063+0+0 (3301974287 0 0) 0x5a68fc0 con 0x43f0840 -4576> 2014-08-13 18:06:09.574478 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4575> 2014-08-13 18:06:09.574481 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4574> 2014-08-13 18:06:09.574468 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4573> 2014-08-13 18:06:09.574484 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68fc0 pg_notify(24.30b(41) epoch 194497) v5 -4572> 2014-08-13 18:06:09.574490 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.313228, event: header_read, request: pg_notify(24.30b(41) epoch 194497) v5 -4571> 2014-08-13 18:06:09.574486 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4570> 2014-08-13 18:06:09.574497 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.313230, event: throttled, request: pg_notify(24.30b(41) epoch 194497) v5 -4569> 2014-08-13 18:06:09.574506 7f5c7c7e4700 10 log is not dirty -4568> 2014-08-13 18:06:09.574504 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.313366, event: all_read, request: pg_notify(24.30b(41) epoch 194497) v5 -4567> 2014-08-13 18:06:09.574510 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.574478, event: dispatched, request: pg_notify(24.30b(41) epoch 194497) v5 -4566> 2014-08-13 18:06:09.574516 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.574516, event: waiting_for_osdmap, request: pg_notify(24.30b(41) epoch 194497) v5 -4565> 2014-08-13 18:06:09.574522 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.10 -4564> 2014-08-13 18:06:09.574525 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68fc0 -4563> 2014-08-13 18:06:09.574530 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.574530, event: started, request: pg_notify(24.30b(41) epoch 194497) v5 -4562> 2014-08-13 18:06:09.574532 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [367,44,4]/[367,44,4] -- 367/367 -4561> 2014-08-13 18:06:09.574550 7f5c835ef700 5 -- op tracker -- , seq: 612, time: 2014-08-13 18:06:09.574550, event: done, request: pg_notify(24.30b(41) epoch 194497) v5 -4560> 2014-08-13 18:06:09.574555 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4559> 2014-08-13 18:06:09.574561 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4558> 2014-08-13 18:06:09.574565 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4557> 2014-08-13 18:06:09.574561 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap -4556> 2014-08-13 18:06:09.574577 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.301 192.168.1.113:6814/16727 10 ==== pg_notify(24.30b(2) epoch 194497) v5 ==== 870+0+0 (844167236 0 0) 0x588f380 con 0x43f69e0 -4555> 2014-08-13 18:06:09.574590 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4554> 2014-08-13 18:06:09.574579 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down -4553> 2014-08-13 18:06:09.574593 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4552> 2014-08-13 18:06:09.574596 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f380 pg_notify(24.30b(2) epoch 194497) v5 -4551> 2014-08-13 18:06:09.574602 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.212889, event: header_read, request: pg_notify(24.30b(2) epoch 194497) v5 -4550> 2014-08-13 18:06:09.574596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map -4549> 2014-08-13 18:06:09.574609 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.212890, event: throttled, request: pg_notify(24.30b(2) epoch 194497) v5 -4548> 2014-08-13 18:06:09.574615 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.212963, event: all_read, request: pg_notify(24.30b(2) epoch 194497) v5 -4547> 2014-08-13 18:06:09.574621 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.574590, event: dispatched, request: pg_notify(24.30b(2) epoch 194497) v5 -4546> 2014-08-13 18:06:09.574613 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters -4545> 2014-08-13 18:06:09.574628 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.574627, event: waiting_for_osdmap, request: pg_notify(24.30b(2) epoch 194497) v5 -4544> 2014-08-13 18:06:09.574634 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.301 -4543> 2014-08-13 18:06:09.574637 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f380 -4542> 2014-08-13 18:06:09.574641 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.574641, event: started, request: pg_notify(24.30b(2) epoch 194497) v5 -4541> 2014-08-13 18:06:09.574638 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4540> 2014-08-13 18:06:09.574662 7f5c835ef700 5 -- op tracker -- , seq: 613, time: 2014-08-13 18:06:09.574662, event: done, request: pg_notify(24.30b(2) epoch 194497) v5 -4539> 2014-08-13 18:06:09.574655 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4538> 2014-08-13 18:06:09.574674 7f5c7c7e4700 10 log is not dirty -4537> 2014-08-13 18:06:09.574681 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4536> 2014-08-13 18:06:09.574685 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4535> 2014-08-13 18:06:09.574689 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.58 192.168.1.111:6834/14316 13 ==== pg_notify(22.1d2(30) epoch 194497) v5 ==== 2400+0+0 (3736757544 0 0) 0x5353b80 con 0x43f7900 -4534> 2014-08-13 18:06:09.574703 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4533> 2014-08-13 18:06:09.574705 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4532> 2014-08-13 18:06:09.574707 7f5c835ef700 20 osd.4 194497 _dispatch 0x5353b80 pg_notify(22.1d2(30) epoch 194497) v5 -4531> 2014-08-13 18:06:09.574695 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,310,21]/[4,310,21] -- 4/4 -4530> 2014-08-13 18:06:09.574713 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.323642, event: header_read, request: pg_notify(22.1d2(30) epoch 194497) v5 -4529> 2014-08-13 18:06:09.574717 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4528> 2014-08-13 18:06:09.574720 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.323644, event: throttled, request: pg_notify(22.1d2(30) epoch 194497) v5 -4527> 2014-08-13 18:06:09.574726 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.323750, event: all_read, request: pg_notify(22.1d2(30) epoch 194497) v5 -4526> 2014-08-13 18:06:09.574732 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.574702, event: dispatched, request: pg_notify(22.1d2(30) epoch 194497) v5 -4525> 2014-08-13 18:06:09.574723 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4524> 2014-08-13 18:06:09.574739 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.574738, event: waiting_for_osdmap, request: pg_notify(22.1d2(30) epoch 194497) v5 -4523> 2014-08-13 18:06:09.574745 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.58 -4522> 2014-08-13 18:06:09.574748 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5353b80 -4521> 2014-08-13 18:06:09.574741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4520> 2014-08-13 18:06:09.574753 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.574753, event: started, request: pg_notify(22.1d2(30) epoch 194497) v5 -4519> 2014-08-13 18:06:09.574759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4518> 2014-08-13 18:06:09.574466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=11] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.20 -4517> 2014-08-13 18:06:09.574773 7f5c835ef700 5 -- op tracker -- , seq: 614, time: 2014-08-13 18:06:09.574773, event: done, request: pg_notify(22.1d2(30) epoch 194497) v5 -4516> 2014-08-13 18:06:09.574784 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4515> 2014-08-13 18:06:09.574788 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4514> 2014-08-13 18:06:09.574777 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4513> 2014-08-13 18:06:09.574795 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4512> 2014-08-13 18:06:09.574805 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.54 192.168.1.111:6836/14923 11 ==== pg_notify(22.1f1(34) epoch 194497) v5 ==== 2624+0+0 (581160723 0 0) 0x5888540 con 0x4425280 -4511> 2014-08-13 18:06:09.574819 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4510> 2014-08-13 18:06:09.574822 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4509> 2014-08-13 18:06:09.574824 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888540 pg_notify(22.1f1(34) epoch 194497) v5 -4508> 2014-08-13 18:06:09.574829 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.313365, event: header_read, request: pg_notify(22.1f1(34) epoch 194497) v5 -4507> 2014-08-13 18:06:09.574836 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.313367, event: throttled, request: pg_notify(22.1f1(34) epoch 194497) v5 -4506> 2014-08-13 18:06:09.574841 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.313511, event: all_read, request: pg_notify(22.1f1(34) epoch 194497) v5 -4505> 2014-08-13 18:06:09.574847 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.574819, event: dispatched, request: pg_notify(22.1f1(34) epoch 194497) v5 -4504> 2014-08-13 18:06:09.574853 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.574853, event: waiting_for_osdmap, request: pg_notify(22.1f1(34) epoch 194497) v5 -4503> 2014-08-13 18:06:09.574859 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.54 -4502> 2014-08-13 18:06:09.574862 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888540 -4501> 2014-08-13 18:06:09.574866 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.574866, event: started, request: pg_notify(22.1f1(34) epoch 194497) v5 -4500> 2014-08-13 18:06:09.574885 7f5c835ef700 5 -- op tracker -- , seq: 615, time: 2014-08-13 18:06:09.574885, event: done, request: pg_notify(22.1f1(34) epoch 194497) v5 -4499> 2014-08-13 18:06:09.574898 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4498> 2014-08-13 18:06:09.574901 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4497> 2014-08-13 18:06:09.574919 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.20 192.168.1.103:6815/38269 10 ==== pg_notify(24.3d(2),22.99(30) epoch 194497) v5 ==== 3254+0+0 (1612875481 0 0) 0x588e900 con 0x44261a0 -4496> 2014-08-13 18:06:09.574933 7f5c835ef700 10 osd.4 194497 do_waiters -- start -4495> 2014-08-13 18:06:09.574936 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -4494> 2014-08-13 18:06:09.574938 7f5c835ef700 20 osd.4 194497 _dispatch 0x588e900 pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4493> 2014-08-13 18:06:09.574944 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.311594, event: header_read, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4492> 2014-08-13 18:06:09.574952 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.311595, event: throttled, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4491> 2014-08-13 18:06:09.574958 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.311716, event: all_read, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4490> 2014-08-13 18:06:09.574964 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.574933, event: dispatched, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4489> 2014-08-13 18:06:09.574971 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.574971, event: waiting_for_osdmap, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4488> 2014-08-13 18:06:09.574977 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.20 -4487> 2014-08-13 18:06:09.574980 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588e900 -4486> 2014-08-13 18:06:09.574984 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.574984, event: started, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -4485> 2014-08-13 18:06:09.575003 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] -4484> 2014-08-13 18:06:09.575018 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4483> 2014-08-13 18:06:09.574777 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=10] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.20 -4482> 2014-08-13 18:06:09.575125 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2575 -4481> 2014-08-13 18:06:09.575150 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4480> 2014-08-13 18:06:09.575166 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4479> 2014-08-13 18:06:09.575181 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4478> 2014-08-13 18:06:09.575208 7f5c7c7e4700 10 log is not dirty -4477> 2014-08-13 18:06:09.575231 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_advance_map [289,4,285]/[289,4,285] -- 289/289 -4476> 2014-08-13 18:06:09.575259 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4475> 2014-08-13 18:06:09.575265 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: Started advmap -4474> 2014-08-13 18:06:09.575282 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] check_recovery_sources no source osds () went down -4473> 2014-08-13 18:06:09.575299 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map -4472> 2014-08-13 18:06:09.575410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] take_waiters -4471> 2014-08-13 18:06:09.575430 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4470> 2014-08-13 18:06:09.575449 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4469> 2014-08-13 18:06:09.575466 7f5c7c7e4700 10 log is not dirty -4468> 2014-08-13 18:06:09.575135 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=9] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.20 -4467> 2014-08-13 18:06:09.575490 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,5,47]/[4,5,47] -- 4/4 -4466> 2014-08-13 18:06:09.575511 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4465> 2014-08-13 18:06:09.575517 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -4464> 2014-08-13 18:06:09.575534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -4463> 2014-08-13 18:06:09.575551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -4462> 2014-08-13 18:06:09.575568 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -4461> 2014-08-13 18:06:09.575583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -4460> 2014-08-13 18:06:09.575471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=8] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.20 -4459> 2014-08-13 18:06:09.575949 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] -4458> 2014-08-13 18:06:09.575963 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -4457> 2014-08-13 18:06:09.575778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=7] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.20 -4456> 2014-08-13 18:06:09.576147 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:3012 -4455> 2014-08-13 18:06:09.576163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -4454> 2014-08-13 18:06:09.576178 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4453> 2014-08-13 18:06:09.576202 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4452> 2014-08-13 18:06:09.576220 7f5c7c7e4700 10 log is not dirty -4451> 2014-08-13 18:06:09.576243 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_advance_map [358,374,4]/[358,374,4] -- 358/358 -4450> 2014-08-13 18:06:09.576264 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4449> 2014-08-13 18:06:09.576271 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] state: Started advmap -4448> 2014-08-13 18:06:09.576287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] check_recovery_sources no source osds () went down -4447> 2014-08-13 18:06:09.576304 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map -4446> 2014-08-13 18:06:09.576319 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] take_waiters -4445> 2014-08-13 18:06:09.576055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=6] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.20 -4444> 2014-08-13 18:06:09.576333 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -4443> 2014-08-13 18:06:09.576349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.2d1( v 194215'526 (0'0,194215'526] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192966) [358,374,4] r=2 lpr=194495 pi=192969-194494/27 luod=0'0 crt=192888'500 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -4442> 2014-08-13 18:06:09.576366 7f5c7c7e4700 10 log is not dirty -4441> 2014-08-13 18:06:09.576389 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_advance_map [4,330,28]/[4,17,315] -- 4/4 -4440> 2014-08-13 18:06:09.576418 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -4439> 2014-08-13 18:06:09.576425 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Peering advmap -4438> 2014-08-13 18:06:09.576444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] adjust_need_up_thru now 194496, need_up_thru now false -4437> 2014-08-13 18:06:09.576462 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Started advmap -4436> 2014-08-13 18:06:09.576480 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] check_recovery_sources no source osds () went down -4435> 2014-08-13 18:06:09.576499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] handle_activate_map -4434> 2014-08-13 18:06:09.576517 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: handle ActMap primary -4433> 2014-08-13 18:06:09.576535 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] publish_stats_to_osd 194497:22354 -4432> 2014-08-13 18:06:09.576552 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] take_waiters -4431> 2014-08-13 18:06:09.576571 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering/WaitUpThru 1.230056 4 0.000661 -4430> 2014-08-13 18:06:09.576593 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] state: Leaving Peering -4429> 2014-08-13 18:06:09.576610 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped+peering m=59] exit Started/Primary/Peering 1.335370 0 0.000000 -4428> 2014-08-13 18:06:09.576340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=5] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.20 -4427> 2014-08-13 18:06:09.576648 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active -4426> 2014-08-13 18:06:09.576667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194487 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: In Active, about to call activate -4425> 2014-08-13 18:06:09.576692 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - purged_snaps [] cached_removed_snaps [] -4424> 2014-08-13 18:06:09.576709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - snap_trimq [] -4423> 2014-08-13 18:06:09.576725 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - not complete, missing(59) -4422> 2014-08-13 18:06:09.576666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=4] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.20 -4421> 2014-08-13 18:06:09.576995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=3] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.20 -4420> 2014-08-13 18:06:09.577336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=2] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.20 -4419> 2014-08-13 18:06:09.577805 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate - starting recovery -4418> 2014-08-13 18:06:09.577831 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] -4417> 2014-08-13 18:06:09.577866 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -4416> 2014-08-13 18:06:09.577888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 is up to date, queueing in pending_activators -4415> 2014-08-13 18:06:09.577681 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57 u=1] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.20 -4414> 2014-08-13 18:06:09.577974 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.17 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -4413> 2014-08-13 18:06:09.577999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -4412> 2014-08-13 18:06:09.578027 7f5c7c7e4700 0 log [INF] : 24.b6 restarting backfill on osd.28 from (0'0,0'0] MAX to 194422'15454 -4411> 2014-08-13 18:06:09.578329 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 sending log((192834'13289,194422'15454], crt=194422'15454) -4410> 2014-08-13 18:06:09.578359 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x43f35a0 don't know epoch, doing nothing -4409> 2014-08-13 18:06:09.578363 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.28 192.168.1.103:6805/37391 -- pg_log(24.b6 epoch 194497 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194497) v4 -- ?+0 0x58b9800 -4408> 2014-08-13 18:06:09.577975 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -4407> 2014-08-13 18:06:09.578392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.28 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -4406> 2014-08-13 18:06:09.578420 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) -4405> 2014-08-13 18:06:09.578443 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 is up to date, queueing in pending_activators -4404> 2014-08-13 18:06:09.578514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.315 24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -4403> 2014-08-13 18:06:09.578537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -4402> 2014-08-13 18:06:09.578563 7f5c7c7e4700 0 log [INF] : 24.b6 restarting backfill on osd.330 from (0'0,0'0] MAX to 194422'15454 -4401> 2014-08-13 18:06:09.578391 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 25 as a source -4400> 2014-08-13 18:06:09.578921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 sending log((192834'13289,194422'15454], crt=194422'15454) -4399> 2014-08-13 18:06:09.578954 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing -4398> 2014-08-13 18:06:09.578958 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.330 192.168.1.106:6802/4876 -- pg_log(24.b6 epoch 194497 log log((192834'13289,194422'15454], crt=194422'15454) query_epoch 194497) v4 -- ?+0 0x58bf200 -4397> 2014-08-13 18:06:09.578983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate peer osd.330 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -4396> 2014-08-13 18:06:09.578746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.25 (past last_backfill 0//0//-1) -4395> 2014-08-13 18:06:09.579056 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery primary has 59 missing -4394> 2014-08-13 18:06:09.579194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.4 -4393> 2014-08-13 18:06:09.579106 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.25 (past last_backfill 0//0//-1) -4392> 2014-08-13 18:06:09.579323 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.4 -4391> 2014-08-13 18:06:09.579446 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.4 -4390> 2014-08-13 18:06:09.579572 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.4 -4389> 2014-08-13 18:06:09.579439 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.25 (past last_backfill 0//0//-1) -4388> 2014-08-13 18:06:09.579696 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.4 -4387> 2014-08-13 18:06:09.579822 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.4 -4386> 2014-08-13 18:06:09.579946 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.4 -4385> 2014-08-13 18:06:09.579771 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.25 (past last_backfill 0//0//-1) -4384> 2014-08-13 18:06:09.580071 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.4 -4383> 2014-08-13 18:06:09.580201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.4 -4382> 2014-08-13 18:06:09.580327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.4 -4381> 2014-08-13 18:06:09.580123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.25 (past last_backfill 0//0//-1) -4380> 2014-08-13 18:06:09.580452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.4 -4379> 2014-08-13 18:06:09.580578 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.4 -4378> 2014-08-13 18:06:09.580456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.25 (past last_backfill 0//0//-1) -4377> 2014-08-13 18:06:09.580703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.4 -4376> 2014-08-13 18:06:09.580827 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.4 -4375> 2014-08-13 18:06:09.580952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.4 -4374> 2014-08-13 18:06:09.580764 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.25 (past last_backfill 0//0//-1) -4373> 2014-08-13 18:06:09.581077 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.4 -4372> 2014-08-13 18:06:09.581204 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.4 -4371> 2014-08-13 18:06:09.581330 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.4 -4370> 2014-08-13 18:06:09.581455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.4 -4369> 2014-08-13 18:06:09.581189 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.25 (past last_backfill 0//0//-1) -4368> 2014-08-13 18:06:09.581579 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.4 -4367> 2014-08-13 18:06:09.581704 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.4 -4366> 2014-08-13 18:06:09.581609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.25 (past last_backfill 0//0//-1) -4365> 2014-08-13 18:06:09.581829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.4 -4364> 2014-08-13 18:06:09.581951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.4 -4363> 2014-08-13 18:06:09.582075 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.4 -4362> 2014-08-13 18:06:09.581926 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.25 (past last_backfill 0//0//-1) -4361> 2014-08-13 18:06:09.582207 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.4 -4360> 2014-08-13 18:06:09.582332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.4 -4359> 2014-08-13 18:06:09.582230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.25 (past last_backfill 0//0//-1) -4358> 2014-08-13 18:06:09.582484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.4 -4357> 2014-08-13 18:06:09.582610 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.4 -4356> 2014-08-13 18:06:09.582556 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.25 (past last_backfill 0//0//-1) -4355> 2014-08-13 18:06:09.582735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.4 -4354> 2014-08-13 18:06:09.582857 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.4 -4353> 2014-08-13 18:06:09.582983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.4 -4352> 2014-08-13 18:06:09.582854 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.25 (past last_backfill 0//0//-1) -4351> 2014-08-13 18:06:09.583107 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.4 -4350> 2014-08-13 18:06:09.583238 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.4 -4349> 2014-08-13 18:06:09.583163 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.25 (past last_backfill 0//0//-1) -4348> 2014-08-13 18:06:09.583363 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.4 -4347> 2014-08-13 18:06:09.583486 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.4 -4346> 2014-08-13 18:06:09.583610 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.4 -4345> 2014-08-13 18:06:09.583471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.25 (past last_backfill 0//0//-1) -4344> 2014-08-13 18:06:09.583735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.4 -4343> 2014-08-13 18:06:09.583859 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.4 -4342> 2014-08-13 18:06:09.583808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.25 (past last_backfill 0//0//-1) -4341> 2014-08-13 18:06:09.583984 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.4 -4340> 2014-08-13 18:06:09.584121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.4 -4339> 2014-08-13 18:06:09.584255 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.4 -4338> 2014-08-13 18:06:09.584112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.25 (past last_backfill 0//0//-1) -4337> 2014-08-13 18:06:09.584380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.4 -4336> 2014-08-13 18:06:09.584505 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.4 -4335> 2014-08-13 18:06:09.584421 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.25 (past last_backfill 0//0//-1) -4334> 2014-08-13 18:06:09.584630 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.4 -4333> 2014-08-13 18:06:09.584752 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.4 -4332> 2014-08-13 18:06:09.584876 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.4 -4331> 2014-08-13 18:06:09.584737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.25 (past last_backfill 0//0//-1) -4330> 2014-08-13 18:06:09.585001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.4 -4329> 2014-08-13 18:06:09.585126 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.4 -4328> 2014-08-13 18:06:09.585035 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.25 (past last_backfill 0//0//-1) -4327> 2014-08-13 18:06:09.585256 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.4 -4326> 2014-08-13 18:06:09.585381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.4 -4325> 2014-08-13 18:06:09.585506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.4 -4324> 2014-08-13 18:06:09.585339 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.25 (past last_backfill 0//0//-1) -4323> 2014-08-13 18:06:09.585631 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.4 -4322> 2014-08-13 18:06:09.585755 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.4 -4321> 2014-08-13 18:06:09.585663 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.25 (past last_backfill 0//0//-1) -4320> 2014-08-13 18:06:09.585879 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.4 -4319> 2014-08-13 18:06:09.586002 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.4 -4318> 2014-08-13 18:06:09.586127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.4 -4317> 2014-08-13 18:06:09.585991 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.25 (past last_backfill 0//0//-1) -4316> 2014-08-13 18:06:09.586256 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.4 -4315> 2014-08-13 18:06:09.586381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.4 -4314> 2014-08-13 18:06:09.586306 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.25 (past last_backfill 0//0//-1) -4313> 2014-08-13 18:06:09.586506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.4 -4312> 2014-08-13 18:06:09.586627 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -4311> 2014-08-13 18:06:09.586619 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.25 (past last_backfill 0//0//-1) -4310> 2014-08-13 18:06:09.586842 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] activate: adding 17 as a source -4309> 2014-08-13 18:06:09.586967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.17 -4308> 2014-08-13 18:06:09.586917 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.25 (past last_backfill 0//0//-1) -4307> 2014-08-13 18:06:09.587093 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=58] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.17 -4306> 2014-08-13 18:06:09.587229 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=57] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.17 -4305> 2014-08-13 18:06:09.587366 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=56] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.17 -4304> 2014-08-13 18:06:09.587221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.25 (past last_backfill 0//0//-1) -4303> 2014-08-13 18:06:09.587501 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=55] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.17 -4302> 2014-08-13 18:06:09.587642 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=54] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.17 -4301> 2014-08-13 18:06:09.587572 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.25 (past last_backfill 0//0//-1) -4300> 2014-08-13 18:06:09.587784 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=53] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.17 -4299> 2014-08-13 18:06:09.587928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=52] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.17 -4298> 2014-08-13 18:06:09.587891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.25 (past last_backfill 0//0//-1) -4297> 2014-08-13 18:06:09.588076 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=51] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.17 -4296> 2014-08-13 18:06:09.588233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=50] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.17 -4295> 2014-08-13 18:06:09.588196 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.25 (past last_backfill 0//0//-1) -4294> 2014-08-13 18:06:09.588389 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=49] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.17 -4293> 2014-08-13 18:06:09.588548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=48] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.17 -4292> 2014-08-13 18:06:09.588508 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.25 (past last_backfill 0//0//-1) -4291> 2014-08-13 18:06:09.588709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=47] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.17 -4290> 2014-08-13 18:06:09.588898 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=46] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.17 -4289> 2014-08-13 18:06:09.588813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.25 (past last_backfill 0//0//-1) -4288> 2014-08-13 18:06:09.589067 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=45] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.17 -4287> 2014-08-13 18:06:09.589244 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=44] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.17 -4286> 2014-08-13 18:06:09.589109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.25 (past last_backfill 0//0//-1) -4285> 2014-08-13 18:06:09.589417 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=43] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.17 -4284> 2014-08-13 18:06:09.589434 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.25 (past last_backfill 0//0//-1) -4283> 2014-08-13 18:06:09.589594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=42] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.17 -4282> 2014-08-13 18:06:09.589771 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=41] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.17 -4281> 2014-08-13 18:06:09.589733 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.25 (past last_backfill 0//0//-1) -4280> 2014-08-13 18:06:09.589954 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=40] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.17 -4279> 2014-08-13 18:06:09.590139 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=39] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.17 -4278> 2014-08-13 18:06:09.590029 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.25 (past last_backfill 0//0//-1) -4277> 2014-08-13 18:06:09.590332 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=38] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.17 -4276> 2014-08-13 18:06:09.590347 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.25 (past last_backfill 0//0//-1) -4275> 2014-08-13 18:06:09.590524 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=37] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.17 -4274> 2014-08-13 18:06:09.590720 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=36] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.17 -4273> 2014-08-13 18:06:09.590676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.25 (past last_backfill 0//0//-1) -4272> 2014-08-13 18:06:09.590918 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=35] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.17 -4271> 2014-08-13 18:06:09.590975 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.25 (past last_backfill 0//0//-1) -4270> 2014-08-13 18:06:09.591119 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=34] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.17 -4269> 2014-08-13 18:06:09.591328 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=33] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.17 -4268> 2014-08-13 18:06:09.591279 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.25 (past last_backfill 0//0//-1) -4267> 2014-08-13 18:06:09.591537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=32] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.17 -4266> 2014-08-13 18:06:09.591744 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=31] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.17 -4265> 2014-08-13 18:06:09.591711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.25 (past last_backfill 0//0//-1) -4264> 2014-08-13 18:06:09.591954 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=30] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.17 -4263> 2014-08-13 18:06:09.592166 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=29] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.17 -4262> 2014-08-13 18:06:09.592063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.25 (past last_backfill 0//0//-1) -4261> 2014-08-13 18:06:09.592387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=28] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.17 -4260> 2014-08-13 18:06:09.592387 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.25 (past last_backfill 0//0//-1) -4259> 2014-08-13 18:06:09.592611 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=27] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.17 -4258> 2014-08-13 18:06:09.592715 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.25 (past last_backfill 0//0//-1) -4257> 2014-08-13 18:06:09.592835 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=26] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.17 -4256> 2014-08-13 18:06:09.593061 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=25] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.17 -4255> 2014-08-13 18:06:09.593013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.25 (past last_backfill 0//0//-1) -4254> 2014-08-13 18:06:09.593296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=24] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.17 -4253> 2014-08-13 18:06:09.593318 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.25 (past last_backfill 0//0//-1) -4252> 2014-08-13 18:06:09.593530 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=23] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.17 -4251> 2014-08-13 18:06:09.593632 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.25 (past last_backfill 0//0//-1) -4250> 2014-08-13 18:06:09.593766 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=22] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.17 -4249> 2014-08-13 18:06:09.593929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.25 (past last_backfill 0//0//-1) -4248> 2014-08-13 18:06:09.594003 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=21] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.17 -4247> 2014-08-13 18:06:09.594251 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=20] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.17 -4246> 2014-08-13 18:06:09.594230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.25 (past last_backfill 0//0//-1) -4245> 2014-08-13 18:06:09.594498 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=19] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.17 -4244> 2014-08-13 18:06:09.594578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.25 (past last_backfill 0//0//-1) -4243> 2014-08-13 18:06:09.594747 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=18] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.17 -4242> 2014-08-13 18:06:09.594877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.25 (past last_backfill 0//0//-1) -4241> 2014-08-13 18:06:09.594999 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=17] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.17 -4240> 2014-08-13 18:06:09.595180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.25 (past last_backfill 0//0//-1) -4239> 2014-08-13 18:06:09.595259 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=16] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.17 -4238> 2014-08-13 18:06:09.595515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=15] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.17 -4237> 2014-08-13 18:06:09.595505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.25 (past last_backfill 0//0//-1) -4236> 2014-08-13 18:06:09.595776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=14] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.17 -4235> 2014-08-13 18:06:09.595801 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.25 (past last_backfill 0//0//-1) -4234> 2014-08-13 18:06:09.596040 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=13] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.17 -4233> 2014-08-13 18:06:09.596096 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.25 (past last_backfill 0//0//-1) -4232> 2014-08-13 18:06:09.596313 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=12] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.17 -4231> 2014-08-13 18:06:09.596417 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.25 (past last_backfill 0//0//-1) -4230> 2014-08-13 18:06:09.596584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=11] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.17 -4229> 2014-08-13 18:06:09.596716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.25 (past last_backfill 0//0//-1) -4228> 2014-08-13 18:06:09.596858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=10] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.17 -4227> 2014-08-13 18:06:09.597012 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -4226> 2014-08-13 18:06:09.597134 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=9] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.17 -4225> 2014-08-13 18:06:09.597430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=8] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.17 -4224> 2014-08-13 18:06:09.597423 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 303 as a source -4223> 2014-08-13 18:06:09.597717 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=7] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.17 -4222> 2014-08-13 18:06:09.597734 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 is on osd.303 -4221> 2014-08-13 18:06:09.598005 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=6] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.17 -4220> 2014-08-13 18:06:09.598060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 is on osd.303 -4219> 2014-08-13 18:06:09.598301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=5] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.17 -4218> 2014-08-13 18:06:09.598373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 is on osd.303 -4217> 2014-08-13 18:06:09.598596 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=4] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.17 -4216> 2014-08-13 18:06:09.598686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 is on osd.303 -4215> 2014-08-13 18:06:09.598893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=3] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.17 -4214> 2014-08-13 18:06:09.598985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 is on osd.303 -4213> 2014-08-13 18:06:09.599197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=2] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.17 -4212> 2014-08-13 18:06:09.599289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 is on osd.303 -4211> 2014-08-13 18:06:09.599607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 is on osd.303 -4210> 2014-08-13 18:06:09.599499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59 u=1] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.17 -4209> 2014-08-13 18:06:09.599906 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 is on osd.303 -4208> 2014-08-13 18:06:09.599981 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -4207> 2014-08-13 18:06:09.600211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 is on osd.303 -4206> 2014-08-13 18:06:09.600384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 28 as a source -4205> 2014-08-13 18:06:09.600529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 is on osd.303 -4204> 2014-08-13 18:06:09.600693 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.28 (past last_backfill 0//0//-1) -4203> 2014-08-13 18:06:09.600827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 is on osd.303 -4202> 2014-08-13 18:06:09.601002 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.28 (past last_backfill 0//0//-1) -4201> 2014-08-13 18:06:09.601125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 is on osd.303 -4200> 2014-08-13 18:06:09.601314 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.28 (past last_backfill 0//0//-1) -4199> 2014-08-13 18:06:09.601489 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 is on osd.303 -4198> 2014-08-13 18:06:09.601622 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.28 (past last_backfill 0//0//-1) -4197> 2014-08-13 18:06:09.601803 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 is on osd.303 -4196> 2014-08-13 18:06:09.601929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.28 (past last_backfill 0//0//-1) -4195> 2014-08-13 18:06:09.602101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 is on osd.303 -4194> 2014-08-13 18:06:09.602242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.28 (past last_backfill 0//0//-1) -4193> 2014-08-13 18:06:09.602411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 is on osd.303 -4192> 2014-08-13 18:06:09.602549 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.28 (past last_backfill 0//0//-1) -4191> 2014-08-13 18:06:09.602728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 is on osd.303 -4190> 2014-08-13 18:06:09.602856 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.28 (past last_backfill 0//0//-1) -4189> 2014-08-13 18:06:09.603097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 is on osd.303 -4188> 2014-08-13 18:06:09.603163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.28 (past last_backfill 0//0//-1) -4187> 2014-08-13 18:06:09.603415 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 is on osd.303 -4186> 2014-08-13 18:06:09.603475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.28 (past last_backfill 0//0//-1) -4185> 2014-08-13 18:06:09.603728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 is on osd.303 -4184> 2014-08-13 18:06:09.603783 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.28 (past last_backfill 0//0//-1) -4183> 2014-08-13 18:06:09.604042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 is on osd.303 -4182> 2014-08-13 18:06:09.604090 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.28 (past last_backfill 0//0//-1) -4181> 2014-08-13 18:06:09.604372 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 is on osd.303 -4180> 2014-08-13 18:06:09.604410 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.28 (past last_backfill 0//0//-1) -4179> 2014-08-13 18:06:09.604715 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.28 (past last_backfill 0//0//-1) -4178> 2014-08-13 18:06:09.604711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 is on osd.303 -4177> 2014-08-13 18:06:09.605022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.28 (past last_backfill 0//0//-1) -4176> 2014-08-13 18:06:09.605031 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 is on osd.303 -4175> 2014-08-13 18:06:09.605335 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.28 (past last_backfill 0//0//-1) -4174> 2014-08-13 18:06:09.605345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 is on osd.303 -4173> 2014-08-13 18:06:09.605643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.28 (past last_backfill 0//0//-1) -4172> 2014-08-13 18:06:09.605657 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 is on osd.303 -4171> 2014-08-13 18:06:09.605950 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.28 (past last_backfill 0//0//-1) -4170> 2014-08-13 18:06:09.605955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 is on osd.303 -4169> 2014-08-13 18:06:09.606263 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.28 (past last_backfill 0//0//-1) -4168> 2014-08-13 18:06:09.606293 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 is on osd.303 -4167> 2014-08-13 18:06:09.606571 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.28 (past last_backfill 0//0//-1) -4166> 2014-08-13 18:06:09.606614 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 is on osd.303 -4165> 2014-08-13 18:06:09.606878 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.28 (past last_backfill 0//0//-1) -4164> 2014-08-13 18:06:09.606961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 is on osd.303 -4163> 2014-08-13 18:06:09.607194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.28 (past last_backfill 0//0//-1) -4162> 2014-08-13 18:06:09.607269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 is on osd.303 -4161> 2014-08-13 18:06:09.607502 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.28 (past last_backfill 0//0//-1) -4160> 2014-08-13 18:06:09.607592 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 is on osd.303 -4159> 2014-08-13 18:06:09.607809 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.28 (past last_backfill 0//0//-1) -4158> 2014-08-13 18:06:09.607918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 is on osd.303 -4157> 2014-08-13 18:06:09.608115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.28 (past last_backfill 0//0//-1) -4156> 2014-08-13 18:06:09.608249 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 is on osd.303 -4155> 2014-08-13 18:06:09.608426 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.28 (past last_backfill 0//0//-1) -4154> 2014-08-13 18:06:09.608567 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 is on osd.303 -4153> 2014-08-13 18:06:09.608733 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.28 (past last_backfill 0//0//-1) -4152> 2014-08-13 18:06:09.608865 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 is on osd.303 -4151> 2014-08-13 18:06:09.609040 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.28 (past last_backfill 0//0//-1) -4150> 2014-08-13 18:06:09.609172 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 is on osd.303 -4149> 2014-08-13 18:06:09.609352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.28 (past last_backfill 0//0//-1) -4148> 2014-08-13 18:06:09.609491 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 is on osd.303 -4147> 2014-08-13 18:06:09.609660 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.28 (past last_backfill 0//0//-1) -4146> 2014-08-13 18:06:09.609791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 is on osd.303 -4145> 2014-08-13 18:06:09.609967 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.28 (past last_backfill 0//0//-1) -4144> 2014-08-13 18:06:09.610088 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 is on osd.303 -4143> 2014-08-13 18:06:09.610278 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.28 (past last_backfill 0//0//-1) -4142> 2014-08-13 18:06:09.610399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 is on osd.303 -4141> 2014-08-13 18:06:09.610585 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.28 (past last_backfill 0//0//-1) -4140> 2014-08-13 18:06:09.610717 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 is on osd.303 -4139> 2014-08-13 18:06:09.610892 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.28 (past last_backfill 0//0//-1) -4138> 2014-08-13 18:06:09.611015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 is on osd.303 -4137> 2014-08-13 18:06:09.611203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.28 (past last_backfill 0//0//-1) -4136> 2014-08-13 18:06:09.611319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 is on osd.303 -4135> 2014-08-13 18:06:09.611510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.28 (past last_backfill 0//0//-1) -4134> 2014-08-13 18:06:09.611648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 is on osd.303 -4133> 2014-08-13 18:06:09.611817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.28 (past last_backfill 0//0//-1) -4132> 2014-08-13 18:06:09.611966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 is on osd.303 -4131> 2014-08-13 18:06:09.612124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.28 (past last_backfill 0//0//-1) -4130> 2014-08-13 18:06:09.612272 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 is on osd.303 -4129> 2014-08-13 18:06:09.612435 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.28 (past last_backfill 0//0//-1) -4128> 2014-08-13 18:06:09.612578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 is on osd.303 -4127> 2014-08-13 18:06:09.612743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.28 (past last_backfill 0//0//-1) -4126> 2014-08-13 18:06:09.612875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 is on osd.303 -4125> 2014-08-13 18:06:09.613050 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.28 (past last_backfill 0//0//-1) -4124> 2014-08-13 18:06:09.613178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 is on osd.303 -4123> 2014-08-13 18:06:09.613361 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.28 (past last_backfill 0//0//-1) -4122> 2014-08-13 18:06:09.613496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 is on osd.303 -4121> 2014-08-13 18:06:09.613669 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.28 (past last_backfill 0//0//-1) -4120> 2014-08-13 18:06:09.613795 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 is on osd.303 -4119> 2014-08-13 18:06:09.613975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.28 (past last_backfill 0//0//-1) -4118> 2014-08-13 18:06:09.614105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 is on osd.303 -4117> 2014-08-13 18:06:09.614286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.28 (past last_backfill 0//0//-1) -4116> 2014-08-13 18:06:09.614427 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 is on osd.303 -4115> 2014-08-13 18:06:09.614594 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.28 (past last_backfill 0//0//-1) -4114> 2014-08-13 18:06:09.614744 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 is on osd.303 -4113> 2014-08-13 18:06:09.614903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.28 (past last_backfill 0//0//-1) -4112> 2014-08-13 18:06:09.615043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 is on osd.303 -4111> 2014-08-13 18:06:09.615231 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.28 (past last_backfill 0//0//-1) -4110> 2014-08-13 18:06:09.615368 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 is on osd.303 -4109> 2014-08-13 18:06:09.615540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.28 (past last_backfill 0//0//-1) -4108> 2014-08-13 18:06:09.615686 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -4107> 2014-08-13 18:06:09.615847 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.28 (past last_backfill 0//0//-1) -4106> 2014-08-13 18:06:09.616054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] activate: adding 362 as a source -4105> 2014-08-13 18:06:09.616154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.28 (past last_backfill 0//0//-1) -4104> 2014-08-13 18:06:09.616359 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24 194416'18672 also missing on osd.362 (past last_backfill 0//0//-1) -4103> 2014-08-13 18:06:09.616467 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.28 (past last_backfill 0//0//-1) -4102> 2014-08-13 18:06:09.616688 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24 194377'18655 also missing on osd.362 (past last_backfill 0//0//-1) -4101> 2014-08-13 18:06:09.616774 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.28 (past last_backfill 0//0//-1) -4100> 2014-08-13 18:06:09.616985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24 194377'18643 also missing on osd.362 (past last_backfill 0//0//-1) -4099> 2014-08-13 18:06:09.617081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.28 (past last_backfill 0//0//-1) -4098> 2014-08-13 18:06:09.617289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24 194416'18677 also missing on osd.362 (past last_backfill 0//0//-1) -4097> 2014-08-13 18:06:09.617392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.28 (past last_backfill 0//0//-1) -4096> 2014-08-13 18:06:09.617607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24 194370'18635 also missing on osd.362 (past last_backfill 0//0//-1) -4095> 2014-08-13 18:06:09.617699 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.28 (past last_backfill 0//0//-1) -4094> 2014-08-13 18:06:09.617904 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24 194370'18621 also missing on osd.362 (past last_backfill 0//0//-1) -4093> 2014-08-13 18:06:09.618006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.28 (past last_backfill 0//0//-1) -4092> 2014-08-13 18:06:09.618233 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24 194416'18705 also missing on osd.362 (past last_backfill 0//0//-1) -4091> 2014-08-13 18:06:09.618317 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.28 (past last_backfill 0//0//-1) -4090> 2014-08-13 18:06:09.618553 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24 194422'18723 also missing on osd.362 (past last_backfill 0//0//-1) -4089> 2014-08-13 18:06:09.618621 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.28 (past last_backfill 0//0//-1) -4088> 2014-08-13 18:06:09.618850 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24 194370'18639 also missing on osd.362 (past last_backfill 0//0//-1) -4087> 2014-08-13 18:06:09.618928 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -4086> 2014-08-13 18:06:09.619167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24 194416'18697 also missing on osd.362 (past last_backfill 0//0//-1) -4085> 2014-08-13 18:06:09.619311 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 315 as a source -4084> 2014-08-13 18:06:09.619514 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24 194393'18671 also missing on osd.362 (past last_backfill 0//0//-1) -4083> 2014-08-13 18:06:09.619619 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 is on osd.315 -4082> 2014-08-13 18:06:09.619811 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24 194370'18631 also missing on osd.362 (past last_backfill 0//0//-1) -4081> 2014-08-13 18:06:09.619928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 is on osd.315 -4080> 2014-08-13 18:06:09.620107 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24 194416'18707 also missing on osd.362 (past last_backfill 0//0//-1) -4079> 2014-08-13 18:06:09.620242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 is on osd.315 -4078> 2014-08-13 18:06:09.620420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24 194393'18669 also missing on osd.362 (past last_backfill 0//0//-1) -4077> 2014-08-13 18:06:09.620550 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 is on osd.315 -4076> 2014-08-13 18:06:09.620749 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24 194416'18699 also missing on osd.362 (past last_backfill 0//0//-1) -4075> 2014-08-13 18:06:09.620858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 is on osd.315 -4074> 2014-08-13 18:06:09.621047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24 194422'18725 also missing on osd.362 (past last_backfill 0//0//-1) -4073> 2014-08-13 18:06:09.621167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 is on osd.315 -4072> 2014-08-13 18:06:09.621353 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24 194370'18627 also missing on osd.362 (past last_backfill 0//0//-1) -4071> 2014-08-13 18:06:09.621478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 is on osd.315 -4070> 2014-08-13 18:06:09.621666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24 194370'18619 also missing on osd.362 (past last_backfill 0//0//-1) -4069> 2014-08-13 18:06:09.621786 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 is on osd.315 -4068> 2014-08-13 18:06:09.621962 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24 194370'18625 also missing on osd.362 (past last_backfill 0//0//-1) -4067> 2014-08-13 18:06:09.622095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 is on osd.315 -4066> 2014-08-13 18:06:09.622264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24 194370'18633 also missing on osd.362 (past last_backfill 0//0//-1) -4065> 2014-08-13 18:06:09.622407 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 is on osd.315 -4064> 2014-08-13 18:06:09.622579 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24 194383'18663 also missing on osd.362 (past last_backfill 0//0//-1) -4063> 2014-08-13 18:06:09.622714 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 is on osd.315 -4062> 2014-08-13 18:06:09.622875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24 194377'18645 also missing on osd.362 (past last_backfill 0//0//-1) -4061> 2014-08-13 18:06:09.623021 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 is on osd.315 -4060> 2014-08-13 18:06:09.623177 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24 194370'18617 also missing on osd.362 (past last_backfill 0//0//-1) -4059> 2014-08-13 18:06:09.623334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 is on osd.315 -4058> 2014-08-13 18:06:09.623493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24 194370'18629 also missing on osd.362 (past last_backfill 0//0//-1) -4057> 2014-08-13 18:06:09.623642 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 is on osd.315 -4056> 2014-08-13 18:06:09.623790 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24 194377'18649 also missing on osd.362 (past last_backfill 0//0//-1) -4055> 2014-08-13 18:06:09.623949 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 is on osd.315 -4054> 2014-08-13 18:06:09.624086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24 194422'18727 also missing on osd.362 (past last_backfill 0//0//-1) -4053> 2014-08-13 18:06:09.624262 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 is on osd.315 -4052> 2014-08-13 18:06:09.624393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24 194416'18713 also missing on osd.362 (past last_backfill 0//0//-1) -4051> 2014-08-13 18:06:09.624570 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 is on osd.315 -4050> 2014-08-13 18:06:09.624706 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24 194416'18691 also missing on osd.362 (past last_backfill 0//0//-1) -4049> 2014-08-13 18:06:09.624878 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 is on osd.315 -4048> 2014-08-13 18:06:09.625017 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24 194416'18675 also missing on osd.362 (past last_backfill 0//0//-1) -4047> 2014-08-13 18:06:09.625190 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 is on osd.315 -4046> 2014-08-13 18:06:09.625329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24 194370'18623 also missing on osd.362 (past last_backfill 0//0//-1) -4045> 2014-08-13 18:06:09.625499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 is on osd.315 -4044> 2014-08-13 18:06:09.625654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24 194416'18679 also missing on osd.362 (past last_backfill 0//0//-1) -4043> 2014-08-13 18:06:09.625807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 is on osd.315 -4042> 2014-08-13 18:06:09.625960 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24 194377'18657 also missing on osd.362 (past last_backfill 0//0//-1) -4041> 2014-08-13 18:06:09.626115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 is on osd.315 -4040> 2014-08-13 18:06:09.626266 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24 194377'18659 also missing on osd.362 (past last_backfill 0//0//-1) -4039> 2014-08-13 18:06:09.626429 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 is on osd.315 -4038> 2014-08-13 18:06:09.626580 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24 194393'18667 also missing on osd.362 (past last_backfill 0//0//-1) -4037> 2014-08-13 18:06:09.626737 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 is on osd.315 -4036> 2014-08-13 18:06:09.626878 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24 194416'18709 also missing on osd.362 (past last_backfill 0//0//-1) -4035> 2014-08-13 18:06:09.627045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 is on osd.315 -4034> 2014-08-13 18:06:09.627188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24 194422'18721 also missing on osd.362 (past last_backfill 0//0//-1) -4033> 2014-08-13 18:06:09.627356 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 is on osd.315 -4032> 2014-08-13 18:06:09.627516 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24 194416'18701 also missing on osd.362 (past last_backfill 0//0//-1) -4031> 2014-08-13 18:06:09.627664 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 is on osd.315 -4030> 2014-08-13 18:06:09.627814 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24 194416'18695 also missing on osd.362 (past last_backfill 0//0//-1) -4029> 2014-08-13 18:06:09.627972 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 is on osd.315 -4028> 2014-08-13 18:06:09.628110 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24 194416'18687 also missing on osd.362 (past last_backfill 0//0//-1) -4027> 2014-08-13 18:06:09.628284 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 is on osd.315 -4026> 2014-08-13 18:06:09.628433 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24 194422'18729 also missing on osd.362 (past last_backfill 0//0//-1) -4025> 2014-08-13 18:06:09.628592 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 is on osd.315 -4024> 2014-08-13 18:06:09.628730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24 194422'18717 also missing on osd.362 (past last_backfill 0//0//-1) -4023> 2014-08-13 18:06:09.628900 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 is on osd.315 -4022> 2014-08-13 18:06:09.629033 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24 194370'18637 also missing on osd.362 (past last_backfill 0//0//-1) -4021> 2014-08-13 18:06:09.629212 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 is on osd.315 -4020> 2014-08-13 18:06:09.629341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24 194422'18719 also missing on osd.362 (past last_backfill 0//0//-1) -4019> 2014-08-13 18:06:09.629521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 is on osd.315 -4018> 2014-08-13 18:06:09.629658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24 194416'18715 also missing on osd.362 (past last_backfill 0//0//-1) -4017> 2014-08-13 18:06:09.629831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 is on osd.315 -4016> 2014-08-13 18:06:09.630064 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24 194383'18661 also missing on osd.362 (past last_backfill 0//0//-1) -4015> 2014-08-13 18:06:09.630139 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 is on osd.315 -4014> 2014-08-13 18:06:09.630413 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24 194377'18647 also missing on osd.362 (past last_backfill 0//0//-1) -4013> 2014-08-13 18:06:09.630452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 is on osd.315 -4012> 2014-08-13 18:06:09.630712 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24 194416'18685 also missing on osd.362 (past last_backfill 0//0//-1) -4011> 2014-08-13 18:06:09.630760 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 is on osd.315 -4010> 2014-08-13 18:06:09.631008 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24 194422'18731 also missing on osd.362 (past last_backfill 0//0//-1) -4009> 2014-08-13 18:06:09.631068 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 is on osd.315 -4008> 2014-08-13 18:06:09.631315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24 194416'18703 also missing on osd.362 (past last_backfill 0//0//-1) -4007> 2014-08-13 18:06:09.631380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 is on osd.315 -4006> 2014-08-13 18:06:09.631631 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24 194370'18641 also missing on osd.362 (past last_backfill 0//0//-1) -4005> 2014-08-13 18:06:09.631689 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 is on osd.315 -4004> 2014-08-13 18:06:09.631928 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24 194377'18653 also missing on osd.362 (past last_backfill 0//0//-1) -4003> 2014-08-13 18:06:09.631996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 is on osd.315 -4002> 2014-08-13 18:06:09.632231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24 194416'18683 also missing on osd.362 (past last_backfill 0//0//-1) -4001> 2014-08-13 18:06:09.632309 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 is on osd.315 -4000> 2014-08-13 18:06:09.632548 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24 194416'18711 also missing on osd.362 (past last_backfill 0//0//-1) -3999> 2014-08-13 18:06:09.632617 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 is on osd.315 -3998> 2014-08-13 18:06:09.632846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24 194416'18689 also missing on osd.362 (past last_backfill 0//0//-1) -3997> 2014-08-13 18:06:09.632925 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 is on osd.315 -3996> 2014-08-13 18:06:09.633141 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24 194377'18651 also missing on osd.362 (past last_backfill 0//0//-1) -3995> 2014-08-13 18:06:09.633238 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 is on osd.315 -3994> 2014-08-13 18:06:09.633473 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24 194416'18693 also missing on osd.362 (past last_backfill 0//0//-1) -3993> 2014-08-13 18:06:09.633546 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 is on osd.315 -3992> 2014-08-13 18:06:09.633772 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] search_for_missing 6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24 194416'18681 also missing on osd.362 (past last_backfill 0//0//-1) -3991> 2014-08-13 18:06:09.633854 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 is on osd.315 -3990> 2014-08-13 18:06:09.634068 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] needs_recovery_map missing {3d/hit_set_24.3d_archive_2014-08-12 20:47:27.684572_2014-08-13 15:28:30.875840/head/.ceph-internal/24=194416'18672,66bc003d/benchmark_data_storage0111-ib.pocstor_25065_object3091/head//24=194377'18655,7931603d/benchmark_data_storage0111-ib.pocstor_25065_object203/head//24=194377'18643,8959603d/benchmark_data_storage0111-ib.pocstor_19103_object305/head//24=194416'18677,d1c703d/benchmark_data_storage0111-ib.pocstor_12495_object10940/head//24=194370'18635,744c803d/benchmark_data_storage0111-ib.pocstor_12495_object4126/head//24=194370'18621,47d803d/benchmark_data_storage0111-ib.pocstor_19103_object13659/head//24=194416'18705,43fa03d/benchmark_data_storage0111-ib.pocstor_41231_object3489/head//24=194422'18723,3b9b03d/benchmark_data_storage0111-ib.pocstor_12495_object13047/head//24=194370'18639,3cf8c03d/benchmark_data_storage0111-ib.pocstor_19103_object10737/head//24=194416'18697,d9d3d03d/benchmark_data_storage0111-ib.pocstor_32482_object269/head//24=194393'18671,97a5d03d/benchmark_data_storage0111-ib.pocstor_12495_object10151/head//24=194370'18631,5cdfe03d/benchmark_data_storage0111-ib.pocstor_19103_object14807/head//24=194416'18707,3c2f03d/benchmark_data_storage0111-ib.pocstor_24170_object515/head//24=194393'18669,ac99043d/benchmark_data_storage0111-ib.pocstor_19103_object10907/head//24=194416'18699,e98f043d/benchmark_data_storage0111-ib.pocstor_41231_object6105/head//24=194422'18725,cadd143d/benchmark_data_storage0111-ib.pocstor_12495_object9309/head//24=194370'18627,cfaf143d/benchmark_data_storage0111-ib.pocstor_12495_object3487/head//24=194370'18619,fe4a243d/benchmark_data_storage0111-ib.pocstor_12495_object8647/head//24=194370'18625,fb8b243d/benchmark_data_storage0111-ib.pocstor_12495_object10287/head//24=194370'18633,abd243d/benchmark_data_storage0111-ib.pocstor_25065_object7103/head//24=194383'18663,a61e343d/benchmark_data_storage0111-ib.pocstor_25065_object895/head//24=194377'18645,8089643d/benchmark_data_storage0111-ib.pocstor_12495_object1615/head//24=194370'18617,f0e9643d/benchmark_data_storage0111-ib.pocstor_12495_object9658/head//24=194370'18629,74d2743d/benchmark_data_storage0111-ib.pocstor_25065_object1452/head//24=194377'18649,60c5843d/benchmark_data_storage0111-ib.pocstor_41231_object6421/head//24=194422'18727,aa12a43d/benchmark_data_storage0111-ib.pocstor_19103_object17120/head//24=194416'18713,897ca43d/benchmark_data_storage0111-ib.pocstor_19103_object8428/head//24=194416'18691,96a5b43d/benchmark_data_storage0111-ib.pocstor_19103_object178/head//24=194416'18675,9915e43d/benchmark_data_storage0111-ib.pocstor_12495_object6727/head//24=194370'18623,d8ce43d/benchmark_data_storage0111-ib.pocstor_19103_object2220/head//24=194416'18679,2874f43d/benchmark_data_storage0111-ib.pocstor_25065_object4035/head//24=194377'18657,8e2f383d/benchmark_data_storage0111-ib.pocstor_25065_object4735/head//24=194377'18659,91d0683d/benchmark_data_storage0111-ib.pocstor_24170_object408/head//24=194393'18667,ab87683d/benchmark_data_storage0111-ib.pocstor_19103_object16002/head//24=194416'18709,f34b683d/benchmark_data_storage0111-ib.pocstor_41231_object2338/head//24=194422'18721,3d7b683d/benchmark_data_storage0111-ib.pocstor_19103_object12159/head//24=194416'18701,ee5a983d/benchmark_data_storage0111-ib.pocstor_19103_object10652/head//24=194416'18695,edde983d/benchmark_data_storage0111-ib.pocstor_19103_object7487/head//24=194416'18687,846bc83d/benchmark_data_storage0111-ib.pocstor_41231_object8961/head//24=194422'18729,7f8ce83d/benchmark_data_storage0111-ib.pocstor_41231_object33/head//24=194422'18717,610de83d/benchmark_data_storage0111-ib.pocstor_12495_object12967/head//24=194370'18637,50e9f83d/benchmark_data_storage0111-ib.pocstor_41231_object1367/head//24=194422'18719,601bf83d/benchmark_data_storage0111-ib.pocstor_19103_object17751/head//24=194416'18715,a91cf83d/benchmark_data_storage0111-ib.pocstor_25065_object6393/head//24=194383'18661,cccff83d/benchmark_data_storage0111-ib.pocstor_25065_object1191/head//24=194377'18647,b1df1c3d/benchmark_data_storage0111-ib.pocstor_19103_object6648/head//24=194416'18685,95e3c3d/benchmark_data_storage0111-ib.pocstor_41231_object10576/head//24=194422'18731,741a5c3d/benchmark_data_storage0111-ib.pocstor_19103_object13181/head//24=194416'18703,5dd46c3d/benchmark_data_storage0111-ib.pocstor_12495_object13313/head//24=194370'18641,cc1c6c3d/benchmark_data_storage0111-ib.pocstor_25065_object2904/head//24=194377'18653,7670bc3d/benchmark_data_storage0111-ib.pocstor_19103_object5787/head//24=194416'18683,2bd5bc3d/benchmark_data_storage0111-ib.pocstor_19103_object16549/head//24=194416'18711,c875cc3d/benchmark_data_storage0111-ib.pocstor_19103_object8289/head//24=194416'18689,46f5cc3d/benchmark_data_storage0111-ib.pocstor_25065_object2240/head//24=194377'18651,22c0dc3d/benchmark_data_storage0111-ib.pocstor_19103_object9867/head//24=194416'18693,6080fc3d/benchmark_data_storage0111-ib.pocstor_19103_object5295/head//24=194416'18681} -3989> 2014-08-13 18:06:09.634162 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 is on osd.315 -3988> 2014-08-13 18:06:09.634457 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound -3987> 2014-08-13 18:06:09.634472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 is on osd.315 -3986> 2014-08-13 18:06:09.634761 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _calc_past_interval_range: already have past intervals back to 194493 -3985> 2014-08-13 18:06:09.634778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 is on osd.315 -3984> 2014-08-13 18:06:09.635060 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] build_might_have_unfound: built 20,25,303,362,373 -3983> 2014-08-13 18:06:09.635083 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 is on osd.315 -3982> 2014-08-13 18:06:09.635392 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 is on osd.315 -3981> 2014-08-13 18:06:09.635374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Activate Finished -3980> 2014-08-13 18:06:09.635701 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] enter Started/Primary/Active/Activating -3979> 2014-08-13 18:06:09.635700 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 is on osd.315 -3978> 2014-08-13 18:06:09.636002 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3977> 2014-08-13 18:06:09.636006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 is on osd.315 -3976> 2014-08-13 18:06:09.636316 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 is on osd.315 -3975> 2014-08-13 18:06:09.636310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3974> 2014-08-13 18:06:09.636727 7f5c7bde3700 10 log is not dirty -3973> 2014-08-13 18:06:09.636797 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_advance_map [321,379,4]/[321,379,4] -- 321/321 -3972> 2014-08-13 18:06:09.636831 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3971> 2014-08-13 18:06:09.636840 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] state: Started advmap -3970> 2014-08-13 18:06:09.636861 7f5c835ef700 5 -- op tracker -- , seq: 616, time: 2014-08-13 18:06:09.636860, event: done, request: pg_notify(24.3d(2),22.99(30) epoch 194497) v5 -3969> 2014-08-13 18:06:09.636859 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] check_recovery_sources no source osds () went down -3968> 2014-08-13 18:06:09.636915 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3967> 2014-08-13 18:06:09.636920 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3966> 2014-08-13 18:06:09.636909 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map -3965> 2014-08-13 18:06:09.636628 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 is on osd.315 -3964> 2014-08-13 18:06:09.636927 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] take_waiters -3963> 2014-08-13 18:06:09.636929 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.291 192.168.1.101:6820/25633 8 ==== pg_notify(24.9b(39) epoch 194497) v5 ==== 2949+0+0 (2477562341 0 0) 0x5357540 con 0x4424e60 -3962> 2014-08-13 18:06:09.636950 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3961> 2014-08-13 18:06:09.636953 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3960> 2014-08-13 18:06:09.636956 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357540 pg_notify(24.9b(39) epoch 194497) v5 -3959> 2014-08-13 18:06:09.636945 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3958> 2014-08-13 18:06:09.636963 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.294483, event: header_read, request: pg_notify(24.9b(39) epoch 194497) v5 -3957> 2014-08-13 18:06:09.636971 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.294485, event: throttled, request: pg_notify(24.9b(39) epoch 194497) v5 -3956> 2014-08-13 18:06:09.636963 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4aa( v 194215'473 (0'0,194215'473] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/192971) [321,379,4] r=2 lpr=194495 pi=192971-194494/27 luod=0'0 crt=192888'451 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3955> 2014-08-13 18:06:09.636978 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.294621, event: all_read, request: pg_notify(24.9b(39) epoch 194497) v5 -3954> 2014-08-13 18:06:09.636982 7f5c7bde3700 10 log is not dirty -3953> 2014-08-13 18:06:09.636984 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.636950, event: dispatched, request: pg_notify(24.9b(39) epoch 194497) v5 -3952> 2014-08-13 18:06:09.636991 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.636991, event: waiting_for_osdmap, request: pg_notify(24.9b(39) epoch 194497) v5 -3951> 2014-08-13 18:06:09.636998 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.291 -3950> 2014-08-13 18:06:09.637002 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357540 -3949> 2014-08-13 18:06:09.637007 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.637007, event: started, request: pg_notify(24.9b(39) epoch 194497) v5 -3948> 2014-08-13 18:06:09.637008 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,349,61]/[4,349,61] -- 4/4 -3947> 2014-08-13 18:06:09.637031 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3946> 2014-08-13 18:06:09.637029 7f5c835ef700 5 -- op tracker -- , seq: 617, time: 2014-08-13 18:06:09.637029, event: done, request: pg_notify(24.9b(39) epoch 194497) v5 -3945> 2014-08-13 18:06:09.637038 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3944> 2014-08-13 18:06:09.637054 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3943> 2014-08-13 18:06:09.637058 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3942> 2014-08-13 18:06:09.637055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3941> 2014-08-13 18:06:09.637064 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.30 192.168.1.107:6823/55491 10 ==== pg_notify(24.34(54) epoch 194497) v5 ==== 3526+0+0 (2502057381 0 0) 0x5354b40 con 0x44586e0 -3940> 2014-08-13 18:06:09.637078 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3939> 2014-08-13 18:06:09.637081 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3938> 2014-08-13 18:06:09.637083 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354b40 pg_notify(24.34(54) epoch 194497) v5 -3937> 2014-08-13 18:06:09.637074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3936> 2014-08-13 18:06:09.637088 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.404624, event: header_read, request: pg_notify(24.34(54) epoch 194497) v5 -3935> 2014-08-13 18:06:09.637095 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.404626, event: throttled, request: pg_notify(24.34(54) epoch 194497) v5 -3934> 2014-08-13 18:06:09.637101 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.404777, event: all_read, request: pg_notify(24.34(54) epoch 194497) v5 -3933> 2014-08-13 18:06:09.637093 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3932> 2014-08-13 18:06:09.637106 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.637078, event: dispatched, request: pg_notify(24.34(54) epoch 194497) v5 -3931> 2014-08-13 18:06:09.637113 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.637113, event: waiting_for_osdmap, request: pg_notify(24.34(54) epoch 194497) v5 -3930> 2014-08-13 18:06:09.637119 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.30 -3929> 2014-08-13 18:06:09.637109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3928> 2014-08-13 18:06:09.637123 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354b40 -3927> 2014-08-13 18:06:09.637129 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.637128, event: started, request: pg_notify(24.34(54) epoch 194497) v5 -3926> 2014-08-13 18:06:09.637150 7f5c835ef700 5 -- op tracker -- , seq: 618, time: 2014-08-13 18:06:09.637149, event: done, request: pg_notify(24.34(54) epoch 194497) v5 -3925> 2014-08-13 18:06:09.637161 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3924> 2014-08-13 18:06:09.637164 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3923> 2014-08-13 18:06:09.637196 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.285 192.168.1.101:6822/26295 9 ==== pg_info(1 pgs e194497:24.1e6(4)) v4 ==== 968+0+0 (2313122121 0 0) 0x5888700 con 0x445e5c0 -3922> 2014-08-13 18:06:09.637212 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3921> 2014-08-13 18:06:09.637215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3920> 2014-08-13 18:06:09.637217 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888700 pg_info(1 pgs e194497:24.1e6(4)) v4 -3919> 2014-08-13 18:06:09.637222 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.199689, event: header_read, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3918> 2014-08-13 18:06:09.637230 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.199691, event: throttled, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3917> 2014-08-13 18:06:09.637236 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.199782, event: all_read, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3916> 2014-08-13 18:06:09.636934 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 is on osd.315 -3915> 2014-08-13 18:06:09.637242 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.637211, event: dispatched, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3914> 2014-08-13 18:06:09.637249 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.637249, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3913> 2014-08-13 18:06:09.637255 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e6(4)) v4 from osd.285 -3912> 2014-08-13 18:06:09.637261 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888700 -3911> 2014-08-13 18:06:09.637266 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.637265, event: started, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3910> 2014-08-13 18:06:09.637288 7f5c835ef700 5 -- op tracker -- , seq: 619, time: 2014-08-13 18:06:09.637287, event: done, request: pg_info(1 pgs e194497:24.1e6(4)) v4 -3909> 2014-08-13 18:06:09.637304 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3908> 2014-08-13 18:06:09.637307 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3907> 2014-08-13 18:06:09.637313 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.61 192.168.1.111:6840/16566 14 ==== pg_notify(22.3af(1) epoch 194497) v5 ==== 777+0+0 (1233671410 0 0) 0x5a69340 con 0x445b180 -3906> 2014-08-13 18:06:09.637326 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3905> 2014-08-13 18:06:09.637329 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3904> 2014-08-13 18:06:09.637331 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69340 pg_notify(22.3af(1) epoch 194497) v5 -3903> 2014-08-13 18:06:09.637336 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.209998, event: header_read, request: pg_notify(22.3af(1) epoch 194497) v5 -3902> 2014-08-13 18:06:09.637343 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.210000, event: throttled, request: pg_notify(22.3af(1) epoch 194497) v5 -3901> 2014-08-13 18:06:09.637349 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.210059, event: all_read, request: pg_notify(22.3af(1) epoch 194497) v5 -3900> 2014-08-13 18:06:09.637354 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.637326, event: dispatched, request: pg_notify(22.3af(1) epoch 194497) v5 -3899> 2014-08-13 18:06:09.637361 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.637361, event: waiting_for_osdmap, request: pg_notify(22.3af(1) epoch 194497) v5 -3898> 2014-08-13 18:06:09.637366 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.61 -3897> 2014-08-13 18:06:09.637369 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69340 -3896> 2014-08-13 18:06:09.637374 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.637374, event: started, request: pg_notify(22.3af(1) epoch 194497) v5 -3895> 2014-08-13 18:06:09.637393 7f5c835ef700 5 -- op tracker -- , seq: 620, time: 2014-08-13 18:06:09.637393, event: done, request: pg_notify(22.3af(1) epoch 194497) v5 -3894> 2014-08-13 18:06:09.637414 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3893> 2014-08-13 18:06:09.637417 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3892> 2014-08-13 18:06:09.637422 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.364 192.168.1.114:6826/52317 7 ==== pg_info(1 pgs e194497:24.1de(2)) v4 ==== 866+0+0 (745639744 0 0) 0x4deeac0 con 0x445ad60 -3891> 2014-08-13 18:06:09.637435 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3890> 2014-08-13 18:06:09.637438 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3889> 2014-08-13 18:06:09.637440 7f5c835ef700 20 osd.4 194497 _dispatch 0x4deeac0 pg_info(1 pgs e194497:24.1de(2)) v4 -3888> 2014-08-13 18:06:09.637429 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] -3887> 2014-08-13 18:06:09.637446 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.203598, event: header_read, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3886> 2014-08-13 18:06:09.637453 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.203600, event: throttled, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3885> 2014-08-13 18:06:09.637446 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3884> 2014-08-13 18:06:09.637460 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.203694, event: all_read, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3883> 2014-08-13 18:06:09.637466 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.637435, event: dispatched, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3882> 2014-08-13 18:06:09.637472 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.637472, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3881> 2014-08-13 18:06:09.637478 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1de(2)) v4 from osd.364 -3880> 2014-08-13 18:06:09.637483 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4deeac0 -3879> 2014-08-13 18:06:09.637487 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.637487, event: started, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3878> 2014-08-13 18:06:09.637508 7f5c835ef700 5 -- op tracker -- , seq: 621, time: 2014-08-13 18:06:09.637508, event: done, request: pg_info(1 pgs e194497:24.1de(2)) v4 -3877> 2014-08-13 18:06:09.637521 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3876> 2014-08-13 18:06:09.637524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3875> 2014-08-13 18:06:09.637245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 is on osd.315 -3874> 2014-08-13 18:06:09.637546 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.289 192.168.1.113:6808/15284 15 ==== pg_info(1 pgs e194497:24.5e(4)) v4 ==== 968+0+0 (3465183483 0 0) 0x4bf0e00 con 0x445f7a0 -3873> 2014-08-13 18:06:09.637560 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3872> 2014-08-13 18:06:09.637563 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3871> 2014-08-13 18:06:09.637566 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf0e00 pg_info(1 pgs e194497:24.5e(4)) v4 -3870> 2014-08-13 18:06:09.637571 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.197981, event: header_read, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3869> 2014-08-13 18:06:09.637581 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.197983, event: throttled, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3868> 2014-08-13 18:06:09.637587 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.198059, event: all_read, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3867> 2014-08-13 18:06:09.637592 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.637560, event: dispatched, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3866> 2014-08-13 18:06:09.637599 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.637598, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3865> 2014-08-13 18:06:09.637604 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.5e(4)) v4 from osd.289 -3864> 2014-08-13 18:06:09.637275 7f5c87c27700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -3863> 2014-08-13 18:06:09.637609 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf0e00 -3862> 2014-08-13 18:06:09.637613 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.637613, event: started, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3861> 2014-08-13 18:06:09.637640 7f5c835ef700 5 -- op tracker -- , seq: 622, time: 2014-08-13 18:06:09.637639, event: done, request: pg_info(1 pgs e194497:24.5e(4)) v4 -3860> 2014-08-13 18:06:09.637633 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:3026 -3859> 2014-08-13 18:06:09.637666 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3858> 2014-08-13 18:06:09.637670 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3857> 2014-08-13 18:06:09.637663 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3856> 2014-08-13 18:06:09.637679 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3855> 2014-08-13 18:06:09.637683 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 13 ==== pg_notify(24.34(54) epoch 194497) v5 ==== 3526+0+0 (4172521127 0 0) 0x4df0380 con 0x4499080 -3854> 2014-08-13 18:06:09.637698 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3853> 2014-08-13 18:06:09.637700 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3852> 2014-08-13 18:06:09.637702 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df0380 pg_notify(24.34(54) epoch 194497) v5 -3851> 2014-08-13 18:06:09.637695 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3850> 2014-08-13 18:06:09.637708 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.438424, event: header_read, request: pg_notify(24.34(54) epoch 194497) v5 -3849> 2014-08-13 18:06:09.637715 7f5c7bde3700 10 log is not dirty -3848> 2014-08-13 18:06:09.637715 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.438427, event: throttled, request: pg_notify(24.34(54) epoch 194497) v5 -3847> 2014-08-13 18:06:09.637721 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.438602, event: all_read, request: pg_notify(24.34(54) epoch 194497) v5 -3846> 2014-08-13 18:06:09.637726 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.637697, event: dispatched, request: pg_notify(24.34(54) epoch 194497) v5 -3845> 2014-08-13 18:06:09.637733 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.637732, event: waiting_for_osdmap, request: pg_notify(24.34(54) epoch 194497) v5 -3844> 2014-08-13 18:06:09.637738 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.340 -3843> 2014-08-13 18:06:09.637741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df0380 -3842> 2014-08-13 18:06:09.637746 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.637745, event: started, request: pg_notify(24.34(54) epoch 194497) v5 -3841> 2014-08-13 18:06:09.637748 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_advance_map [364,4,360]/[364,4,301] -- 364/364 -3840> 2014-08-13 18:06:09.637772 7f5c835ef700 5 -- op tracker -- , seq: 623, time: 2014-08-13 18:06:09.637771, event: done, request: pg_notify(24.34(54) epoch 194497) v5 -3839> 2014-08-13 18:06:09.637781 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3838> 2014-08-13 18:06:09.637792 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3837> 2014-08-13 18:06:09.637796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3836> 2014-08-13 18:06:09.637788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: Started advmap -3835> 2014-08-13 18:06:09.637807 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] check_recovery_sources no source osds () went down -3834> 2014-08-13 18:06:09.637832 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.45 192.168.1.112:6828/6040 13 ==== pg_info(1 pgs e194497:24.369(4)) v4 ==== 968+0+0 (1424897791 0 0) 0x4df6900 con 0x4496720 -3833> 2014-08-13 18:06:09.637848 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3832> 2014-08-13 18:06:09.637851 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3831> 2014-08-13 18:06:09.637551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 is on osd.315 -3830> 2014-08-13 18:06:09.637854 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df6900 pg_info(1 pgs e194497:24.369(4)) v4 -3829> 2014-08-13 18:06:09.637842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map -3828> 2014-08-13 18:06:09.637860 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.198983, event: header_read, request: pg_info(1 pgs e194497:24.369(4)) v4 -3827> 2014-08-13 18:06:09.637868 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.198985, event: throttled, request: pg_info(1 pgs e194497:24.369(4)) v4 -3826> 2014-08-13 18:06:09.637874 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.199077, event: all_read, request: pg_info(1 pgs e194497:24.369(4)) v4 -3825> 2014-08-13 18:06:09.637879 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.637847, event: dispatched, request: pg_info(1 pgs e194497:24.369(4)) v4 -3824> 2014-08-13 18:06:09.637886 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.637886, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.369(4)) v4 -3823> 2014-08-13 18:06:09.637894 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.369(4)) v4 from osd.45 -3822> 2014-08-13 18:06:09.637898 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df6900 -3821> 2014-08-13 18:06:09.637903 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.637903, event: started, request: pg_info(1 pgs e194497:24.369(4)) v4 -3820> 2014-08-13 18:06:09.637613 7f5c87c27700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] flushed -3819> 2014-08-13 18:06:09.637929 7f5c835ef700 5 -- op tracker -- , seq: 624, time: 2014-08-13 18:06:09.637929, event: done, request: pg_info(1 pgs e194497:24.369(4)) v4 -3818> 2014-08-13 18:06:09.637941 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] take_waiters -3817> 2014-08-13 18:06:09.637964 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3816> 2014-08-13 18:06:09.637968 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3815> 2014-08-13 18:06:09.637962 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3814> 2014-08-13 18:06:09.637973 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.17 192.168.1.103:6811/37658 12 ==== pg_notify(24.b6(44) epoch 194497) v5 ==== 3240+0+0 (40169421 0 0) 0x524c7c0 con 0x4493f40 -3813> 2014-08-13 18:06:09.637988 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3812> 2014-08-13 18:06:09.637991 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3811> 2014-08-13 18:06:09.637993 7f5c835ef700 20 osd.4 194497 _dispatch 0x524c7c0 pg_notify(24.b6(44) epoch 194497) v5 -3810> 2014-08-13 18:06:09.637999 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.197955, event: header_read, request: pg_notify(24.b6(44) epoch 194497) v5 -3809> 2014-08-13 18:06:09.638006 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.197957, event: throttled, request: pg_notify(24.b6(44) epoch 194497) v5 -3808> 2014-08-13 18:06:09.637981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3807> 2014-08-13 18:06:09.638012 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.198089, event: all_read, request: pg_notify(24.b6(44) epoch 194497) v5 -3806> 2014-08-13 18:06:09.638018 7f5c7bde3700 10 log is not dirty -3805> 2014-08-13 18:06:09.638018 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.637988, event: dispatched, request: pg_notify(24.b6(44) epoch 194497) v5 -3804> 2014-08-13 18:06:09.638024 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.638024, event: waiting_for_osdmap, request: pg_notify(24.b6(44) epoch 194497) v5 -3803> 2014-08-13 18:06:09.638030 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.17 -3802> 2014-08-13 18:06:09.638033 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524c7c0 -3801> 2014-08-13 18:06:09.638038 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.638037, event: started, request: pg_notify(24.b6(44) epoch 194497) v5 -3800> 2014-08-13 18:06:09.638042 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 -3799> 2014-08-13 18:06:09.638073 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3798> 2014-08-13 18:06:09.638079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -3797> 2014-08-13 18:06:09.638099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -3796> 2014-08-13 18:06:09.638123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -3795> 2014-08-13 18:06:09.638140 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -3794> 2014-08-13 18:06:09.638168 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -3793> 2014-08-13 18:06:09.638183 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -3792> 2014-08-13 18:06:09.638203 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:176861 -3791> 2014-08-13 18:06:09.637858 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -3790> 2014-08-13 18:06:09.638218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters -3789> 2014-08-13 18:06:09.638249 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3788> 2014-08-13 18:06:09.638270 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[2.2a( v 194215'930 (0'0,194215'930] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3787> 2014-08-13 18:06:09.638300 7f5c7bde3700 10 log is not dirty -3786> 2014-08-13 18:06:09.638323 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,339,45]/[4,339,45] -- 4/4 -3785> 2014-08-13 18:06:09.638350 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3784> 2014-08-13 18:06:09.638357 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3783> 2014-08-13 18:06:09.638373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3782> 2014-08-13 18:06:09.638407 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3781> 2014-08-13 18:06:09.638427 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3780> 2014-08-13 18:06:09.638447 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3779> 2014-08-13 18:06:09.638242 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] activate: adding 330 as a source -3778> 2014-08-13 18:06:09.638782 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] -3777> 2014-08-13 18:06:09.638804 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3776> 2014-08-13 18:06:09.638548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24 194416'15393 also missing on osd.330 (past last_backfill 0//0//-1) -3775> 2014-08-13 18:06:09.638978 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2939 -3774> 2014-08-13 18:06:09.638999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3773> 2014-08-13 18:06:09.639015 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3772> 2014-08-13 18:06:09.639031 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3771> 2014-08-13 18:06:09.639054 7f5c7bde3700 10 log is not dirty -3770> 2014-08-13 18:06:09.639086 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_advance_map [324,13,395]/[324,13,395] -- 324/324 -3769> 2014-08-13 18:06:09.639132 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3768> 2014-08-13 18:06:09.638852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24 194377'15380 also missing on osd.330 (past last_backfill 0//0//-1) -3767> 2014-08-13 18:06:09.639138 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] state: Started advmap -3766> 2014-08-13 18:06:09.639183 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] check_recovery_sources no source osds () went down -3765> 2014-08-13 18:06:09.639206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map -3764> 2014-08-13 18:06:09.639312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] take_waiters -3763> 2014-08-13 18:06:09.639349 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194497 -3762> 2014-08-13 18:06:09.639372 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.39a( v 194422'13151 lc 194370'13054 (192718'10533,194422'13151] local-les=194453 n=50 ec=192688 les/c 194449/194351 194459/194460/194460) [324,13,395] r=-1 lpr=194470 pi=193304-194459/56 crt=194422'13151 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3761> 2014-08-13 18:06:09.639401 7f5c7bde3700 10 log is not dirty -3760> 2014-08-13 18:06:09.639442 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_advance_map [304,323,4]/[304,323,4] -- 304/304 -3759> 2014-08-13 18:06:09.639156 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24 194416'15408 also missing on osd.330 (past last_backfill 0//0//-1) -3758> 2014-08-13 18:06:09.639471 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3757> 2014-08-13 18:06:09.639478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: Started advmap -3756> 2014-08-13 18:06:09.639505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] check_recovery_sources no source osds () went down -3755> 2014-08-13 18:06:09.639533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map -3754> 2014-08-13 18:06:09.639639 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] take_waiters -3753> 2014-08-13 18:06:09.639660 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3752> 2014-08-13 18:06:09.639676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3751> 2014-08-13 18:06:09.639694 7f5c7bde3700 10 log is not dirty -3750> 2014-08-13 18:06:09.639715 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340]/[4,340] -- 4/4 -3749> 2014-08-13 18:06:09.639749 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3748> 2014-08-13 18:06:09.639464 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24 194422'15452 also missing on osd.330 (past last_backfill 0//0//-1) -3747> 2014-08-13 18:06:09.639763 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -3746> 2014-08-13 18:06:09.639780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -3745> 2014-08-13 18:06:09.639797 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -3744> 2014-08-13 18:06:09.639813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -3743> 2014-08-13 18:06:09.639827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -3742> 2014-08-13 18:06:09.639853 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -3741> 2014-08-13 18:06:09.639870 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:177965 -3740> 2014-08-13 18:06:09.639884 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] take_waiters -3739> 2014-08-13 18:06:09.639897 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3738> 2014-08-13 18:06:09.639916 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[1.2b( v 194215'1672 (0'0,194215'1672] local-les=194496 n=0 ec=1 les/c 194496/194496 194495/194495/194495) [4,340] r=0 lpr=194495 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3737> 2014-08-13 18:06:09.639933 7f5c7bde3700 10 log is not dirty -3736> 2014-08-13 18:06:09.639962 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_advance_map [26,30,340]/[4] -- 26/4 -3735> 2014-08-13 18:06:09.640007 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3734> 2014-08-13 18:06:09.640014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Peering advmap -3733> 2014-08-13 18:06:09.640037 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] adjust_need_up_thru now 194496, need_up_thru now false -3732> 2014-08-13 18:06:09.640054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Started advmap -3731> 2014-08-13 18:06:09.639769 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24 194370'15364 also missing on osd.330 (past last_backfill 0//0//-1) -3730> 2014-08-13 18:06:09.640072 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] check_recovery_sources no source osds () went down -3729> 2014-08-13 18:06:09.640108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] handle_activate_map -3728> 2014-08-13 18:06:09.640126 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: handle ActMap primary -3727> 2014-08-13 18:06:09.640143 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] publish_stats_to_osd 194497:23846 -3726> 2014-08-13 18:06:09.640174 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] take_waiters -3725> 2014-08-13 18:06:09.640193 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering/WaitUpThru 1.286172 4 0.000655 -3724> 2014-08-13 18:06:09.640218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] state: Leaving Peering -3723> 2014-08-13 18:06:09.640235 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped+peering] exit Started/Primary/Peering 1.390152 0 0.000000 -3722> 2014-08-13 18:06:09.640263 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] enter Started/Primary/Active -3721> 2014-08-13 18:06:09.640280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194487 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] state: In Active, about to call activate -3720> 2014-08-13 18:06:09.640306 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - purged_snaps [] cached_removed_snaps [] -3719> 2014-08-13 18:06:09.640323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - snap_trimq [] -3718> 2014-08-13 18:06:09.640338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate - no missing, moving last_complete 194215'14514 -> 194215'14514 -3717> 2014-08-13 18:06:09.640075 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24 194422'15444 also missing on osd.330 (past last_backfill 0//0//-1) -3716> 2014-08-13 18:06:09.640438 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -3715> 2014-08-13 18:06:09.640462 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 is up to date, queueing in pending_activators -3714> 2014-08-13 18:06:09.640614 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.26 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate -3713> 2014-08-13 18:06:09.640648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -3712> 2014-08-13 18:06:09.640382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24 194416'15432 also missing on osd.330 (past last_backfill 0//0//-1) -3711> 2014-08-13 18:06:09.640686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 is up to date, queueing in pending_activators -3710> 2014-08-13 18:06:09.640777 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.30 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate -3709> 2014-08-13 18:06:09.640808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) -3708> 2014-08-13 18:06:09.640847 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 is up to date, queueing in pending_activators -3707> 2014-08-13 18:06:09.640945 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] activate peer osd.340 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496) uptodate -3706> 2014-08-13 18:06:09.640686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24 194416'15438 also missing on osd.330 (past last_backfill 0//0//-1) -3705> 2014-08-13 18:06:09.640977 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 remapped] needs_recovery is recovered -3704> 2014-08-13 18:06:09.640995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Activate Finished -3703> 2014-08-13 18:06:09.641019 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] enter Started/Primary/Active/Activating -3702> 2014-08-13 18:06:09.641050 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3701> 2014-08-13 18:06:09.641068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3700> 2014-08-13 18:06:09.641194 7f5c7bde3700 10 log is not dirty -3699> 2014-08-13 18:06:09.641251 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,374,31]/[4,374,31] -- 4/4 -3698> 2014-08-13 18:06:09.641281 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3697> 2014-08-13 18:06:09.640993 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24 194370'15358 also missing on osd.330 (past last_backfill 0//0//-1) -3696> 2014-08-13 18:06:09.641288 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3695> 2014-08-13 18:06:09.641307 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3694> 2014-08-13 18:06:09.641365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3693> 2014-08-13 18:06:09.641385 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3692> 2014-08-13 18:06:09.641401 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3691> 2014-08-13 18:06:09.641303 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24 194370'15356 also missing on osd.330 (past last_backfill 0//0//-1) -3690> 2014-08-13 18:06:09.641681 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] _activate_committed 194497 peer_activated now 4 last_epoch_started 194487 same_interval_since 194496 -3689> 2014-08-13 18:06:09.641729 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] flushed -3688> 2014-08-13 18:06:09.641743 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] -3687> 2014-08-13 18:06:09.641761 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3686> 2014-08-13 18:06:09.641612 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24 194416'15440 also missing on osd.330 (past last_backfill 0//0//-1) -3685> 2014-08-13 18:06:09.641917 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2877 -3684> 2014-08-13 18:06:09.641935 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3683> 2014-08-13 18:06:09.641951 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3682> 2014-08-13 18:06:09.641966 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3681> 2014-08-13 18:06:09.641985 7f5c7bde3700 10 log is not dirty -3680> 2014-08-13 18:06:09.642015 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,28,304]/[4,28,304] -- 4/4 -3679> 2014-08-13 18:06:09.642036 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3678> 2014-08-13 18:06:09.642041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -3677> 2014-08-13 18:06:09.642058 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -3676> 2014-08-13 18:06:09.642073 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -3675> 2014-08-13 18:06:09.642089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -3674> 2014-08-13 18:06:09.642104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -3673> 2014-08-13 18:06:09.642117 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -3672> 2014-08-13 18:06:09.642133 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2913 -3671> 2014-08-13 18:06:09.642188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] take_waiters -3670> 2014-08-13 18:06:09.642204 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3669> 2014-08-13 18:06:09.641919 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24 194416'15414 also missing on osd.330 (past last_backfill 0//0//-1) -3668> 2014-08-13 18:06:09.642219 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.84( v 194215'497 (0'0,194215'497] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/194495) [4,28,304] r=0 lpr=194495 crt=191648'337 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3667> 2014-08-13 18:06:09.642255 7f5c7bde3700 10 log is not dirty -3666> 2014-08-13 18:06:09.642279 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,356,357]/[4,356,357] -- 4/4 -3665> 2014-08-13 18:06:09.642301 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3664> 2014-08-13 18:06:09.642307 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3663> 2014-08-13 18:06:09.642323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3662> 2014-08-13 18:06:09.642341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3661> 2014-08-13 18:06:09.642363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3660> 2014-08-13 18:06:09.642378 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3659> 2014-08-13 18:06:09.642229 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24 194416'15426 also missing on osd.330 (past last_backfill 0//0//-1) -3658> 2014-08-13 18:06:09.642794 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] -3657> 2014-08-13 18:06:09.642815 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3656> 2014-08-13 18:06:09.642536 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24 194416'15430 also missing on osd.330 (past last_backfill 0//0//-1) -3655> 2014-08-13 18:06:09.643006 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2664 -3654> 2014-08-13 18:06:09.643028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3653> 2014-08-13 18:06:09.643044 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3652> 2014-08-13 18:06:09.643059 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3651> 2014-08-13 18:06:09.643077 7f5c7bde3700 10 log is not dirty -3650> 2014-08-13 18:06:09.643101 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,373,344]/[4,373,344] -- 4/4 -3649> 2014-08-13 18:06:09.643122 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3648> 2014-08-13 18:06:09.643127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3647> 2014-08-13 18:06:09.642841 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24 194416'15398 also missing on osd.330 (past last_backfill 0//0//-1) -3646> 2014-08-13 18:06:09.643144 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3645> 2014-08-13 18:06:09.643178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3644> 2014-08-13 18:06:09.643195 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3643> 2014-08-13 18:06:09.643211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3642> 2014-08-13 18:06:09.643144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24 194416'15410 also missing on osd.330 (past last_backfill 0//0//-1) -3641> 2014-08-13 18:06:09.643499 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] -3640> 2014-08-13 18:06:09.643520 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3639> 2014-08-13 18:06:09.643648 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2619 -3638> 2014-08-13 18:06:09.643667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3637> 2014-08-13 18:06:09.643683 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3636> 2014-08-13 18:06:09.643699 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3635> 2014-08-13 18:06:09.643726 7f5c7bde3700 10 log is not dirty -3634> 2014-08-13 18:06:09.643456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24 194416'15434 also missing on osd.330 (past last_backfill 0//0//-1) -3633> 2014-08-13 18:06:09.643750 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_advance_map [328,364,4]/[328,364,4] -- 328/328 -3632> 2014-08-13 18:06:09.643773 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3631> 2014-08-13 18:06:09.643781 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] state: Started advmap -3630> 2014-08-13 18:06:09.643798 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] check_recovery_sources no source osds () went down -3629> 2014-08-13 18:06:09.643815 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map -3628> 2014-08-13 18:06:09.643831 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] take_waiters -3627> 2014-08-13 18:06:09.643846 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3626> 2014-08-13 18:06:09.643861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.2b9( v 194215'506 (0'0,194215'506] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192949) [328,364,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'472 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3625> 2014-08-13 18:06:09.643878 7f5c7bde3700 10 log is not dirty -3624> 2014-08-13 18:06:09.644009 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_advance_map [20,26,374]/[20,374] -- 20/20 -3623> 2014-08-13 18:06:09.644046 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3622> 2014-08-13 18:06:09.643761 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24 194416'15402 also missing on osd.330 (past last_backfill 0//0//-1) -3621> 2014-08-13 18:06:09.644054 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] state: Started advmap -3620> 2014-08-13 18:06:09.644073 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] check_recovery_sources no source osds () went down -3619> 2014-08-13 18:06:09.644092 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map -3618> 2014-08-13 18:06:09.644171 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] take_waiters -3617> 2014-08-13 18:06:09.644195 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194470 while current is 194497 -3616> 2014-08-13 18:06:09.644211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2c( v 194422'15445 lc 194215'15333 (192718'12964,194422'15445] local-les=194455 n=56 ec=192688 les/c 194455/194442 194467/194467/192971) [20,26,374]/[20,374] r=-1 lpr=194470 pi=193125-194466/32 crt=194422'15445 lcod 0'0 inactive NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3615> 2014-08-13 18:06:09.644230 7f5c7bde3700 10 log is not dirty -3614> 2014-08-13 18:06:09.644254 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_advance_map [330,4,288]/[330,4,57] -- 330/330 -3613> 2014-08-13 18:06:09.644286 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3612> 2014-08-13 18:06:09.644292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: Started advmap -3611> 2014-08-13 18:06:09.644310 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] check_recovery_sources no source osds () went down -3610> 2014-08-13 18:06:09.644328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map -3609> 2014-08-13 18:06:09.644067 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24 194370'15346 also missing on osd.330 (past last_backfill 0//0//-1) -3608> 2014-08-13 18:06:09.644436 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] take_waiters -3607> 2014-08-13 18:06:09.644483 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3606> 2014-08-13 18:06:09.644501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3605> 2014-08-13 18:06:09.644526 7f5c7bde3700 10 log is not dirty -3604> 2014-08-13 18:06:09.644553 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,353,56]/[4,353,56] -- 4/4 -3603> 2014-08-13 18:06:09.644578 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3602> 2014-08-13 18:06:09.644585 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -3601> 2014-08-13 18:06:09.644617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -3600> 2014-08-13 18:06:09.644635 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -3599> 2014-08-13 18:06:09.644664 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -3598> 2014-08-13 18:06:09.644375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24 194393'15388 also missing on osd.330 (past last_backfill 0//0//-1) -3597> 2014-08-13 18:06:09.644680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -3596> 2014-08-13 18:06:09.644682 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24 194422'15448 also missing on osd.330 (past last_backfill 0//0//-1) -3595> 2014-08-13 18:06:09.645015 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] -3594> 2014-08-13 18:06:09.645032 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -3593> 2014-08-13 18:06:09.645210 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2727 -3592> 2014-08-13 18:06:09.645230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -3591> 2014-08-13 18:06:09.645246 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3590> 2014-08-13 18:06:09.645262 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3589> 2014-08-13 18:06:09.645280 7f5c7bde3700 10 log is not dirty -3588> 2014-08-13 18:06:09.644990 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24 194370'15348 also missing on osd.330 (past last_backfill 0//0//-1) -3587> 2014-08-13 18:06:09.645303 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_advance_map [294,327,4]/[294,327,4] -- 294/294 -3586> 2014-08-13 18:06:09.645330 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3585> 2014-08-13 18:06:09.645340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] state: Started advmap -3584> 2014-08-13 18:06:09.645357 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] check_recovery_sources no source osds () went down -3583> 2014-08-13 18:06:09.645374 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map -3582> 2014-08-13 18:06:09.645390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] take_waiters -3581> 2014-08-13 18:06:09.645410 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3580> 2014-08-13 18:06:09.645426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.630( v 194215'517 (0'0,194215'517] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192968) [294,327,4] r=2 lpr=194495 pi=192977-194494/27 luod=0'0 crt=192888'483 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3579> 2014-08-13 18:06:09.645459 7f5c7bde3700 10 log is not dirty -3578> 2014-08-13 18:06:09.645498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_advance_map [346,4,304]/[346,4,288] -- 346/346 -3577> 2014-08-13 18:06:09.645528 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3576> 2014-08-13 18:06:09.645534 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: Started advmap -3575> 2014-08-13 18:06:09.645552 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] check_recovery_sources no source osds () went down -3574> 2014-08-13 18:06:09.645299 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24 194370'15366 also missing on osd.330 (past last_backfill 0//0//-1) -3573> 2014-08-13 18:06:09.645575 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map -3572> 2014-08-13 18:06:09.645671 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] take_waiters -3571> 2014-08-13 18:06:09.645710 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3570> 2014-08-13 18:06:09.645739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3569> 2014-08-13 18:06:09.645759 7f5c7bde3700 10 log is not dirty -3568> 2014-08-13 18:06:09.645769 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing -3567> 2014-08-13 18:06:09.645774 7f5c7bde3700 7 osd.4 194497 do_notify osd 20 on 1 PGs -3566> 2014-08-13 18:06:09.645777 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6815/38269 -- pg_notify(24.2c(32) epoch 194497) v5 -- ?+0 0x5356580 con 0x44261a0 -3565> 2014-08-13 18:06:09.645810 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4ab69e0 192.168.1.101:6808/22782 -3564> 2014-08-13 18:06:09.645837 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6808/22782 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4ee7980 con 0x4ab69e0 -3563> 2014-08-13 18:06:09.645858 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.304 has 194497 -3562> 2014-08-13 18:06:09.645862 7f5c7bde3700 7 osd.4 194497 do_notify osd 304 on 1 PGs -3561> 2014-08-13 18:06:09.645865 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6808/22782 -- pg_notify(24.33a(36) epoch 194497) v5 -- ?+0 0x5351880 con 0x4ab69e0 -3560> 2014-08-13 18:06:09.645877 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4b2cba0 192.168.1.104:6814/65209 -3559> 2014-08-13 18:06:09.645885 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6814/65209 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x47fa1c0 con 0x4b2cba0 -3558> 2014-08-13 18:06:09.645604 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24 194416'15424 also missing on osd.330 (past last_backfill 0//0//-1) -3557> 2014-08-13 18:06:09.645905 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.324 has 194497 -3556> 2014-08-13 18:06:09.645921 7f5c7bde3700 7 osd.4 194497 do_notify osd 324 on 1 PGs -3555> 2014-08-13 18:06:09.645924 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6814/65209 -- pg_notify(24.39a(56) epoch 194497) v5 -- ?+0 0x5354440 con 0x4b2cba0 -3554> 2014-08-13 18:06:09.645936 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing -3553> 2014-08-13 18:06:09.645939 7f5c7bde3700 7 osd.4 194497 do_notify osd 330 on 1 PGs -3552> 2014-08-13 18:06:09.645941 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6802/4876 -- pg_notify(24.3f7(45) epoch 194497) v5 -- ?+0 0x5888e00 con 0x4f20dc0 -3551> 2014-08-13 18:06:09.645965 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4f21600 192.168.1.104:6820/2308 -3550> 2014-08-13 18:06:09.645977 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6820/2308 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4cff2c0 con 0x4f21600 -3549> 2014-08-13 18:06:09.645994 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.346 has 194497 -3548> 2014-08-13 18:06:09.646001 7f5c7bde3700 7 osd.4 194497 do_notify osd 346 on 1 PGs -3547> 2014-08-13 18:06:09.646004 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6820/2308 -- pg_notify(24.e2(38) epoch 194497) v5 -- ?+0 0x588ec80 con 0x4f21600 -3546> 2014-08-13 18:06:09.646102 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing -3545> 2014-08-13 18:06:09.646109 7f5c7bde3700 7 osd.4 194497 do_notify osd 364 on 1 PGs -3544> 2014-08-13 18:06:09.646112 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6826/52317 -- pg_notify(24.1de(39) epoch 194497) v5 -- ?+0 0x588cec0 con 0x445ad60 -3543> 2014-08-13 18:06:09.645907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24 194416'15406 also missing on osd.330 (past last_backfill 0//0//-1) -3542> 2014-08-13 18:06:09.646204 7f5c7bde3700 20 osd.4 194497 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 20 -3541> 2014-08-13 18:06:09.646225 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing -3540> 2014-08-13 18:06:09.646257 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6815/38269 -- pg_info(1 pgs e194497:24.3d(39)) v4 -- ?+0 0x5888fc0 con 0x44261a0 -3539> 2014-08-13 18:06:09.646302 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 26 -3538> 2014-08-13 18:06:09.646313 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing -3537> 2014-08-13 18:06:09.646342 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.115:6802/58522 -- pg_info(1 pgs e194497:24.34(54)) v4 -- ?+0 0x4df5940 con 0x3a7e9e0 -3536> 2014-08-13 18:06:09.646360 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 30 -3535> 2014-08-13 18:06:09.646369 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing -3534> 2014-08-13 18:06:09.646387 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6823/55491 -- pg_info(1 pgs e194497:24.34(54)) v4 -- ?+0 0x4df3d40 con 0x44586e0 -3533> 2014-08-13 18:06:09.646403 7f5c7bde3700 20 osd.4 194497 Sending info 24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 303 -3532> 2014-08-13 18:06:09.646414 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing -3531> 2014-08-13 18:06:09.646440 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.109:6802/30768 -- pg_info(1 pgs e194497:24.3d(39)) v4 -- ?+0 0x4df0a80 con 0x4af8580 -3530> 2014-08-13 18:06:09.646468 7f5c7bde3700 20 osd.4 194497 Sending info 24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) to shard 340 -3529> 2014-08-13 18:06:09.646493 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -3528> 2014-08-13 18:06:09.646216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24 194416'15416 also missing on osd.330 (past last_backfill 0//0//-1) -3527> 2014-08-13 18:06:09.646535 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6806/62463 -- pg_info(1 pgs e194497:24.34(54)) v4 -- ?+0 0x4df2140 con 0x4499080 -3526> 2014-08-13 18:06:09.646650 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a5a00 prio 127 cost 0 latency 0.157299 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] -3525> 2014-08-13 18:06:09.646695 7f5c7bde3700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.646695, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3524> 2014-08-13 18:06:09.646743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_message: 0x20a5a00 -3523> 2014-08-13 18:06:09.646767 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3522> 2014-08-13 18:06:09.646521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24 194370'15344 also missing on osd.330 (past last_backfill 0//0//-1) -3521> 2014-08-13 18:06:09.646788 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] sub_op_scrub_reserve -3520> 2014-08-13 18:06:09.646837 7f5c7bde3700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.646837, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3519> 2014-08-13 18:06:09.646858 7f5c7bde3700 20 osd.4 194497 inc_scrubs_pending 1 + 0 active >= max 1 -3518> 2014-08-13 18:06:09.646869 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.114:6840/55513 -- osd_sub_op_reply(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x59fbe80 con 0x4b11ce0 -3517> 2014-08-13 18:06:09.646894 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a5a00 finish -3516> 2014-08-13 18:06:09.646900 7f5c7bde3700 5 -- op tracker -- , seq: 578, time: 2014-08-13 18:06:09.646900, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -3515> 2014-08-13 18:06:09.646991 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_advance_map [4,344,296]/[4,296,320] -- 4/4 -3514> 2014-08-13 18:06:09.647034 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3513> 2014-08-13 18:06:09.647047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Peering advmap -3512> 2014-08-13 18:06:09.647068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] adjust_need_up_thru now 194496, need_up_thru now false -3511> 2014-08-13 18:06:09.647085 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Started advmap -3510> 2014-08-13 18:06:09.647105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] check_recovery_sources no source osds () went down -3509> 2014-08-13 18:06:09.646831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24 194416'15404 also missing on osd.330 (past last_backfill 0//0//-1) -3508> 2014-08-13 18:06:09.647124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] handle_activate_map -3507> 2014-08-13 18:06:09.647166 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: handle ActMap primary -3506> 2014-08-13 18:06:09.647188 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] publish_stats_to_osd 194497:28887 -3505> 2014-08-13 18:06:09.647206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] take_waiters -3504> 2014-08-13 18:06:09.647234 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering/WaitUpThru 1.305349 7 0.000991 -3503> 2014-08-13 18:06:09.647256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] state: Leaving Peering -3502> 2014-08-13 18:06:09.647273 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped+peering m=47] exit Started/Primary/Peering 1.380145 0 0.000000 -3501> 2014-08-13 18:06:09.647298 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active -3500> 2014-08-13 18:06:09.647316 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194487 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: In Active, about to call activate -3499> 2014-08-13 18:06:09.647338 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - purged_snaps [] cached_removed_snaps [] -3498> 2014-08-13 18:06:09.647355 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - snap_trimq [] -3497> 2014-08-13 18:06:09.647371 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - not complete, missing(47) -3496> 2014-08-13 18:06:09.647142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24 194416'15436 also missing on osd.330 (past last_backfill 0//0//-1) -3495> 2014-08-13 18:06:09.647452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24 194422'15446 also missing on osd.330 (past last_backfill 0//0//-1) -3494> 2014-08-13 18:06:09.647758 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24 194370'15350 also missing on osd.330 (past last_backfill 0//0//-1) -3493> 2014-08-13 18:06:09.648065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24 194416'15420 also missing on osd.330 (past last_backfill 0//0//-1) -3492> 2014-08-13 18:06:09.648376 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24 194370'15354 also missing on osd.330 (past last_backfill 0//0//-1) -3491> 2014-08-13 18:06:09.648796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate - starting recovery -3490> 2014-08-13 18:06:09.648836 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] -3489> 2014-08-13 18:06:09.648876 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -3488> 2014-08-13 18:06:09.648899 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 is up to date, queueing in pending_activators -3487> 2014-08-13 18:06:09.648683 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24 194370'15336 also missing on osd.330 (past last_backfill 0//0//-1) -3486> 2014-08-13 18:06:09.648971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.296 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate -3485> 2014-08-13 18:06:09.648998 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) -3484> 2014-08-13 18:06:09.649019 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 is up to date, queueing in pending_activators -3483> 2014-08-13 18:06:09.649139 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.320 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate -3482> 2014-08-13 18:06:09.649187 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) -3481> 2014-08-13 18:06:09.649211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 is up to date, queueing in pending_activators -3480> 2014-08-13 18:06:09.648989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24 194370'15342 also missing on osd.330 (past last_backfill 0//0//-1) -3479> 2014-08-13 18:06:09.649304 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate peer osd.344 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496) uptodate -3478> 2014-08-13 18:06:09.649375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] needs_recovery primary has 47 missing -3477> 2014-08-13 18:06:09.649302 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24 194370'15368 also missing on osd.330 (past last_backfill 0//0//-1) -3476> 2014-08-13 18:06:09.649497 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 also missing on osd.4 -3475> 2014-08-13 18:06:09.649625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 also missing on osd.4 -3474> 2014-08-13 18:06:09.649731 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 also missing on osd.4 -3473> 2014-08-13 18:06:09.649609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24 194370'15338 also missing on osd.330 (past last_backfill 0//0//-1) -3472> 2014-08-13 18:06:09.649863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 also missing on osd.4 -3471> 2014-08-13 18:06:09.649968 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 also missing on osd.4 -3470> 2014-08-13 18:06:09.650087 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 also missing on osd.4 -3469> 2014-08-13 18:06:09.649916 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24 194393'15390 also missing on osd.330 (past last_backfill 0//0//-1) -3468> 2014-08-13 18:06:09.650203 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 also missing on osd.4 -3467> 2014-08-13 18:06:09.650312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 also missing on osd.4 -3466> 2014-08-13 18:06:09.650224 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24 194370'15362 also missing on osd.330 (past last_backfill 0//0//-1) -3465> 2014-08-13 18:06:09.650420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 also missing on osd.4 -3464> 2014-08-13 18:06:09.650543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 also missing on osd.4 -3463> 2014-08-13 18:06:09.650647 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 also missing on osd.4 -3462> 2014-08-13 18:06:09.650531 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24 194377'15374 also missing on osd.330 (past last_backfill 0//0//-1) -3461> 2014-08-13 18:06:09.650750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 also missing on osd.4 -3460> 2014-08-13 18:06:09.650851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 also missing on osd.4 -3459> 2014-08-13 18:06:09.650973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 also missing on osd.4 -3458> 2014-08-13 18:06:09.650837 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24 194370'15352 also missing on osd.330 (past last_backfill 0//0//-1) -3457> 2014-08-13 18:06:09.651084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 also missing on osd.4 -3456> 2014-08-13 18:06:09.651201 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 also missing on osd.4 -3455> 2014-08-13 18:06:09.651305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 also missing on osd.4 -3454> 2014-08-13 18:06:09.651143 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24 194416'15412 also missing on osd.330 (past last_backfill 0//0//-1) -3453> 2014-08-13 18:06:09.651412 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 also missing on osd.4 -3452> 2014-08-13 18:06:09.651529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 also missing on osd.4 -3451> 2014-08-13 18:06:09.651649 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 also missing on osd.4 -3450> 2014-08-13 18:06:09.651455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24 194416'15418 also missing on osd.330 (past last_backfill 0//0//-1) -3449> 2014-08-13 18:06:09.651752 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 also missing on osd.4 -3448> 2014-08-13 18:06:09.651856 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 also missing on osd.4 -3447> 2014-08-13 18:06:09.651959 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 also missing on osd.4 -3446> 2014-08-13 18:06:09.651759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24 194377'15376 also missing on osd.330 (past last_backfill 0//0//-1) -3445> 2014-08-13 18:06:09.652061 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 also missing on osd.4 -3444> 2014-08-13 18:06:09.652173 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 also missing on osd.4 -3443> 2014-08-13 18:06:09.652063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24 194416'15422 also missing on osd.330 (past last_backfill 0//0//-1) -3442> 2014-08-13 18:06:09.652276 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 also missing on osd.4 -3441> 2014-08-13 18:06:09.652381 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 also missing on osd.4 -3440> 2014-08-13 18:06:09.652493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 also missing on osd.4 -3439> 2014-08-13 18:06:09.652376 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24 194416'15428 also missing on osd.330 (past last_backfill 0//0//-1) -3438> 2014-08-13 18:06:09.652615 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 also missing on osd.4 -3437> 2014-08-13 18:06:09.652716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 also missing on osd.4 -3436> 2014-08-13 18:06:09.652819 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 also missing on osd.4 -3435> 2014-08-13 18:06:09.652691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24 194370'15340 also missing on osd.330 (past last_backfill 0//0//-1) -3434> 2014-08-13 18:06:09.652922 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 also missing on osd.4 -3433> 2014-08-13 18:06:09.653022 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 also missing on osd.4 -3432> 2014-08-13 18:06:09.653124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 also missing on osd.4 -3431> 2014-08-13 18:06:09.652998 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24 194370'15360 also missing on osd.330 (past last_backfill 0//0//-1) -3430> 2014-08-13 18:06:09.653234 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 also missing on osd.4 -3429> 2014-08-13 18:06:09.653335 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 also missing on osd.4 -3428> 2014-08-13 18:06:09.653442 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 also missing on osd.4 -3427> 2014-08-13 18:06:09.653310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24 194422'15454 also missing on osd.330 (past last_backfill 0//0//-1) -3426> 2014-08-13 18:06:09.653571 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 also missing on osd.4 -3425> 2014-08-13 18:06:09.653676 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 also missing on osd.4 -3424> 2014-08-13 18:06:09.653778 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 also missing on osd.4 -3423> 2014-08-13 18:06:09.653617 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24 194396'15392 also missing on osd.330 (past last_backfill 0//0//-1) -3422> 2014-08-13 18:06:09.653881 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 also missing on osd.4 -3421> 2014-08-13 18:06:09.653984 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 also missing on osd.4 -3420> 2014-08-13 18:06:09.654086 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 also missing on osd.4 -3419> 2014-08-13 18:06:09.653924 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24 194377'15372 also missing on osd.330 (past last_backfill 0//0//-1) -3418> 2014-08-13 18:06:09.654196 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 also missing on osd.4 -3417> 2014-08-13 18:06:09.654300 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 also missing on osd.4 -3416> 2014-08-13 18:06:09.654406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 also missing on osd.4 -3415> 2014-08-13 18:06:09.654235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24 194380'15382 also missing on osd.330 (past last_backfill 0//0//-1) -3414> 2014-08-13 18:06:09.654527 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 also missing on osd.4 -3413> 2014-08-13 18:06:09.654641 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} -3412> 2014-08-13 18:06:09.654540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24 194416'15400 also missing on osd.330 (past last_backfill 0//0//-1) -3411> 2014-08-13 18:06:09.654801 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] activate: adding 296 as a source -3410> 2014-08-13 18:06:09.654902 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 is on osd.296 -3409> 2014-08-13 18:06:09.655006 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=46] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 is on osd.296 -3408> 2014-08-13 18:06:09.654847 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24 194416'15396 also missing on osd.330 (past last_backfill 0//0//-1) -3407> 2014-08-13 18:06:09.655115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=45] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 is on osd.296 -3406> 2014-08-13 18:06:09.655241 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=44] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 is on osd.296 -3405> 2014-08-13 18:06:09.655154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24 194377'15378 also missing on osd.330 (past last_backfill 0//0//-1) -3404> 2014-08-13 18:06:09.655355 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=43] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 is on osd.296 -3403> 2014-08-13 18:06:09.655483 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=42] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 is on osd.296 -3402> 2014-08-13 18:06:09.655617 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=41] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 is on osd.296 -3401> 2014-08-13 18:06:09.655466 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24 194422'15450 also missing on osd.330 (past last_backfill 0//0//-1) -3400> 2014-08-13 18:06:09.655757 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=40] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 is on osd.296 -3399> 2014-08-13 18:06:09.655885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=39] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 is on osd.296 -3398> 2014-08-13 18:06:09.655770 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24 194422'15442 also missing on osd.330 (past last_backfill 0//0//-1) -3397> 2014-08-13 18:06:09.656013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=38] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 is on osd.296 -3396> 2014-08-13 18:06:09.656154 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=37] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 is on osd.296 -3395> 2014-08-13 18:06:09.656077 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24 194377'15370 also missing on osd.330 (past last_backfill 0//0//-1) -3394> 2014-08-13 18:06:09.656292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=36] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 is on osd.296 -3393> 2014-08-13 18:06:09.656434 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=35] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 is on osd.296 -3392> 2014-08-13 18:06:09.656388 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] search_for_missing 9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24 194383'15384 also missing on osd.330 (past last_backfill 0//0//-1) -3391> 2014-08-13 18:06:09.656593 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=34] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 is on osd.296 -3390> 2014-08-13 18:06:09.656739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=33] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 is on osd.296 -3389> 2014-08-13 18:06:09.656902 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=32] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 is on osd.296 -3388> 2014-08-13 18:06:09.656695 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] needs_recovery_map missing {b6/hit_set_24.b6_archive_2014-08-12 20:52:32.941779_2014-08-13 15:28:32.208147/head/.ceph-internal/24=194416'15393,229100b6/benchmark_data_storage0111-ib.pocstor_25065_object4804/head//24=194377'15380,25a610b6/benchmark_data_storage0111-ib.pocstor_19103_object5798/head//24=194416'15408,c95130b6/benchmark_data_storage0111-ib.pocstor_41231_object9754/head//24=194422'15452,e66130b6/benchmark_data_storage0111-ib.pocstor_12495_object13234/head//24=194370'15364,b3b940b6/benchmark_data_storage0111-ib.pocstor_41231_object3735/head//24=194422'15444,ff3a40b6/benchmark_data_storage0111-ib.pocstor_19103_object15046/head//24=194416'15432,9ee860b6/benchmark_data_storage0111-ib.pocstor_19103_object17917/head//24=194416'15438,49af60b6/benchmark_data_storage0111-ib.pocstor_12495_object8493/head//24=194370'15358,1b4570b6/benchmark_data_storage0111-ib.pocstor_12495_object7493/head//24=194370'15356,1c1a0b6/benchmark_data_storage0111-ib.pocstor_19103_object17929/head//24=194416'15440,3d4ea0b6/benchmark_data_storage0111-ib.pocstor_19103_object8385/head//24=194416'15414,5041b0b6/benchmark_data_storage0111-ib.pocstor_19103_object11892/head//24=194416'15426,d7eec0b6/benchmark_data_storage0111-ib.pocstor_19103_object14109/head//24=194416'15430,85e4d0b6/benchmark_data_storage0111-ib.pocstor_19103_object1865/head//24=194416'15398,bb46d0b6/benchmark_data_storage0111-ib.pocstor_19103_object7493/head//24=194416'15410,e096f0b6/benchmark_data_storage0111-ib.pocstor_19103_object15103/head//24=194416'15434,4f7604b6/benchmark_data_storage0111-ib.pocstor_19103_object2905/head//24=194416'15402,b1a804b6/benchmark_data_storage0111-ib.pocstor_12495_object3743/head//24=194370'15346,7a7514b6/benchmark_data_storage0111-ib.pocstor_48304_object104/head//24=194393'15388,bf5024b6/benchmark_data_storage0111-ib.pocstor_41231_object7726/head//24=194422'15448,414724b6/benchmark_data_storage0111-ib.pocstor_12495_object4080/head//24=194370'15348,b18a24b6/benchmark_data_storage0111-ib.pocstor_12495_object13532/head//24=194370'15366,a00464b6/benchmark_data_storage0111-ib.pocstor_19103_object11583/head//24=194416'15424,512564b6/benchmark_data_storage0111-ib.pocstor_19103_object5478/head//24=194416'15406,3662b4b6/benchmark_data_storage0111-ib.pocstor_19103_object9271/head//24=194416'15416,37efb4b6/benchmark_data_storage0111-ib.pocstor_12495_object2248/head//24=194370'15344,d21dc4b6/benchmark_data_storage0111-ib.pocstor_19103_object3966/head//24=194416'15404,4f8d4b6/benchmark_data_storage0111-ib.pocstor_19103_object15456/head//24=194416'15436,be5ee4b6/benchmark_data_storage0111-ib.pocstor_41231_object3921/head//24=194422'15446,785f4b6/benchmark_data_storage0111-ib.pocstor_12495_object4081/head//24=194370'15350,20b608b6/benchmark_data_storage0111-ib.pocstor_19103_object10580/head//24=194416'15420,9f0618b6/benchmark_data_storage0111-ib.pocstor_12495_object6519/head//24=194370'15354,31eb18b6/benchmark_data_storage0111-ib.pocstor_12495_object409/head//24=194370'15336,5b0a58b6/benchmark_data_storage0111-ib.pocstor_12495_object1351/head//24=194370'15342,fc278b6/benchmark_data_storage0111-ib.pocstor_12495_object13801/head//24=194370'15368,fd9198b6/benchmark_data_storage0111-ib.pocstor_12495_object1143/head//24=194370'15338,912398b6/benchmark_data_storage0111-ib.pocstor_48304_object162/head//24=194393'15390,dbcc98b6/benchmark_data_storage0111-ib.pocstor_12495_object10873/head//24=194370'15362,2adaa8b6/benchmark_data_storage0111-ib.pocstor_25065_object1279/head//24=194377'15374,c863c8b6/benchmark_data_storage0111-ib.pocstor_12495_object4121/head//24=194370'15352,aefe8b6/benchmark_data_storage0111-ib.pocstor_19103_object8010/head//24=194416'15412,ddf1f8b6/benchmark_data_storage0111-ib.pocstor_19103_object9390/head//24=194416'15418,5742f8b6/benchmark_data_storage0111-ib.pocstor_25065_object2436/head//24=194377'15376,4a510cb6/benchmark_data_storage0111-ib.pocstor_19103_object11053/head//24=194416'15422,21e30cb6/benchmark_data_storage0111-ib.pocstor_19103_object13728/head//24=194416'15428,e1741cb6/benchmark_data_storage0111-ib.pocstor_12495_object1337/head//24=194370'15340,b1002cb6/benchmark_data_storage0111-ib.pocstor_12495_object10497/head//24=194370'15360,5f002cb6/benchmark_data_storage0111-ib.pocstor_41231_object10573/head//24=194422'15454,95403cb6/benchmark_data_storage0101-ib.pocstor_6819_object277/head//24=194396'15392,a5d43cb6/benchmark_data_storage0111-ib.pocstor_25065_object984/head//24=194377'15372,3b553cb6/benchmark_data_storage0111-ib.pocstor_25065_object5317/head//24=194380'15382,c3483cb6/benchmark_data_storage0111-ib.pocstor_19103_object2409/head//24=194416'15400,2df4cb6/benchmark_data_storage0111-ib.pocstor_19103_object455/head//24=194416'15396,56e15cb6/benchmark_data_storage0111-ib.pocstor_25065_object3943/head//24=194377'15378,888a5cb6/benchmark_data_storage0111-ib.pocstor_41231_object9598/head//24=194422'15450,92308cb6/benchmark_data_storage0111-ib.pocstor_41231_object2866/head//24=194422'15442,cbf09cb6/benchmark_data_storage0111-ib.pocstor_25065_object591/head//24=194377'15370,9c0acb6/benchmark_data_storage0111-ib.pocstor_25065_object5831/head//24=194383'15384} -3387> 2014-08-13 18:06:09.657067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=31] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 is on osd.296 -3386> 2014-08-13 18:06:09.657073 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound -3385> 2014-08-13 18:06:09.657235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=30] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 is on osd.296 -3384> 2014-08-13 18:06:09.657393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=29] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 is on osd.296 -3383> 2014-08-13 18:06:09.657384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _calc_past_interval_range: already have past intervals back to 194493 -3382> 2014-08-13 18:06:09.657578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=28] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 is on osd.296 -3381> 2014-08-13 18:06:09.657741 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=27] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 is on osd.296 -3380> 2014-08-13 18:06:09.657693 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] build_might_have_unfound: built 17,28,315,327,330 -3379> 2014-08-13 18:06:09.657906 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=26] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 is on osd.296 -3378> 2014-08-13 18:06:09.658074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=25] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 is on osd.296 -3377> 2014-08-13 18:06:09.658001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Activate Finished -3376> 2014-08-13 18:06:09.658252 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=24] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 is on osd.296 -3375> 2014-08-13 18:06:09.658313 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] enter Started/Primary/Active/Activating -3374> 2014-08-13 18:06:09.658432 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=23] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 is on osd.296 -3373> 2014-08-13 18:06:09.658631 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=22] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 is on osd.296 -3372> 2014-08-13 18:06:09.658621 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3371> 2014-08-13 18:06:09.658813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=21] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 is on osd.296 -3370> 2014-08-13 18:06:09.658997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=20] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 is on osd.296 -3369> 2014-08-13 18:06:09.658929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3368> 2014-08-13 18:06:09.659335 7f5c7c7e4700 10 log is not dirty -3367> 2014-08-13 18:06:09.659190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=19] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 is on osd.296 -3366> 2014-08-13 18:06:09.659411 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [4,47,378]/[19,65] -- 4/19 -3365> 2014-08-13 18:06:09.659449 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3364> 2014-08-13 18:06:09.659450 7f5c835ef700 5 -- op tracker -- , seq: 625, time: 2014-08-13 18:06:09.659449, event: done, request: pg_notify(24.b6(44) epoch 194497) v5 -3363> 2014-08-13 18:06:09.659457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap -3362> 2014-08-13 18:06:09.659478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down -3361> 2014-08-13 18:06:09.659500 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3360> 2014-08-13 18:06:09.659505 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3359> 2014-08-13 18:06:09.659497 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map -3358> 2014-08-13 18:06:09.659516 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters -3357> 2014-08-13 18:06:09.659514 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.315 192.168.1.113:6800/14161 7 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee1d40 con 0x3c51080 -3356> 2014-08-13 18:06:09.659536 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3355> 2014-08-13 18:06:09.659539 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3354> 2014-08-13 18:06:09.659541 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee1d40 osd_map(194497..194497 src has 185139..194497) v3 -3353> 2014-08-13 18:06:09.659533 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3352> 2014-08-13 18:06:09.659551 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -3351> 2014-08-13 18:06:09.659557 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -3350> 2014-08-13 18:06:09.659560 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3349> 2014-08-13 18:06:09.659562 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3348> 2014-08-13 18:06:09.659551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3bf( v 194422'13811 (192834'12014,194422'13811] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/194468) [4,47,378]/[19,65] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3347> 2014-08-13 18:06:09.659572 7f5c7c7e4700 10 log is not dirty -3346> 2014-08-13 18:06:09.659570 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 16 ==== pg_info(1 pgs e194497:24.350(4)) v4 ==== 968+0+0 (810898773 0 0) 0x524c980 con 0x3c527e0 -3345> 2014-08-13 18:06:09.659583 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3344> 2014-08-13 18:06:09.659586 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3343> 2014-08-13 18:06:09.659597 7f5c835ef700 20 osd.4 194497 _dispatch 0x524c980 pg_info(1 pgs e194497:24.350(4)) v4 -3342> 2014-08-13 18:06:09.659604 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.198765, event: header_read, request: pg_info(1 pgs e194497:24.350(4)) v4 -3341> 2014-08-13 18:06:09.659384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=18] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 is on osd.296 -3340> 2014-08-13 18:06:09.659596 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [373,360,4]/[373,360,4] -- 373/373 -3339> 2014-08-13 18:06:09.659613 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.198768, event: throttled, request: pg_info(1 pgs e194497:24.350(4)) v4 -3338> 2014-08-13 18:06:09.659619 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3337> 2014-08-13 18:06:09.659620 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.198852, event: all_read, request: pg_info(1 pgs e194497:24.350(4)) v4 -3336> 2014-08-13 18:06:09.659627 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.659583, event: dispatched, request: pg_info(1 pgs e194497:24.350(4)) v4 -3335> 2014-08-13 18:06:09.659635 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.659635, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.350(4)) v4 -3334> 2014-08-13 18:06:09.659625 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap -3333> 2014-08-13 18:06:09.659642 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.350(4)) v4 from osd.356 -3332> 2014-08-13 18:06:09.659648 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524c980 -3331> 2014-08-13 18:06:09.659644 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down -3330> 2014-08-13 18:06:09.659654 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.659654, event: started, request: pg_info(1 pgs e194497:24.350(4)) v4 -3329> 2014-08-13 18:06:09.659662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map -3328> 2014-08-13 18:06:09.659681 7f5c835ef700 5 -- op tracker -- , seq: 626, time: 2014-08-13 18:06:09.659681, event: done, request: pg_info(1 pgs e194497:24.350(4)) v4 -3327> 2014-08-13 18:06:09.659679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters -3326> 2014-08-13 18:06:09.659693 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3325> 2014-08-13 18:06:09.659705 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3324> 2014-08-13 18:06:09.659709 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3323> 2014-08-13 18:06:09.659708 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1fb( v 194215'522 (0'0,194215'522] local-les=194496 n=29 ec=183096 les/c 194496/194496 194495/194495/192978) [373,360,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3322> 2014-08-13 18:06:09.659726 7f5c7c7e4700 10 log is not dirty -3321> 2014-08-13 18:06:09.659715 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.55 192.168.1.112:6834/6966 15 ==== pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 ==== 2573+0+0 (2494703945 0 0) 0x5a6b640 con 0x3c523c0 -3320> 2014-08-13 18:06:09.659731 7f5c835ef700 10 osd.4 194497 do_waiters -- start -3319> 2014-08-13 18:06:09.659734 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -3318> 2014-08-13 18:06:09.659736 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b640 pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3317> 2014-08-13 18:06:09.659742 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.226111, event: header_read, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3316> 2014-08-13 18:06:09.659750 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.226116, event: throttled, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3315> 2014-08-13 18:06:09.659757 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.226278, event: all_read, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3314> 2014-08-13 18:06:09.659749 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_advance_map [356,364,4]/[356,364,4] -- 356/356 -3313> 2014-08-13 18:06:09.659763 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.659731, event: dispatched, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3312> 2014-08-13 18:06:09.659771 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.659771, event: waiting_for_osdmap, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3311> 2014-08-13 18:06:09.659775 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3310> 2014-08-13 18:06:09.659778 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.55 -3309> 2014-08-13 18:06:09.659782 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b640 -3308> 2014-08-13 18:06:09.659788 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.659787, event: started, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -3307> 2014-08-13 18:06:09.659782 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: Started advmap -3306> 2014-08-13 18:06:09.659616 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=17] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 is on osd.296 -3305> 2014-08-13 18:06:09.659800 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] check_recovery_sources no source osds () went down -3304> 2014-08-13 18:06:09.659817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map -3303> 2014-08-13 18:06:09.659895 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] take_waiters -3302> 2014-08-13 18:06:09.659912 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -3301> 2014-08-13 18:06:09.659928 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -3300> 2014-08-13 18:06:09.659945 7f5c7c7e4700 10 log is not dirty -3299> 2014-08-13 18:06:09.659969 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_advance_map [4,59,351]/[4,351,55] -- 4/4 -3298> 2014-08-13 18:06:09.659997 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3297> 2014-08-13 18:06:09.659815 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=16] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 is on osd.296 -3296> 2014-08-13 18:06:09.660004 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Peering advmap -3295> 2014-08-13 18:06:09.660024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] adjust_need_up_thru now 194496, need_up_thru now false -3294> 2014-08-13 18:06:09.660044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Started advmap -3293> 2014-08-13 18:06:09.660063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] check_recovery_sources no source osds () went down -3292> 2014-08-13 18:06:09.660081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] handle_activate_map -3291> 2014-08-13 18:06:09.660099 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: handle ActMap primary -3290> 2014-08-13 18:06:09.660118 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] publish_stats_to_osd 194497:25642 -3289> 2014-08-13 18:06:09.660135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] take_waiters -3288> 2014-08-13 18:06:09.660153 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering/WaitUpThru 1.317787 3 0.000655 -3287> 2014-08-13 18:06:09.660175 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] state: Leaving Peering -3286> 2014-08-13 18:06:09.660200 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped+peering m=62] exit Started/Primary/Peering 1.408671 0 0.000000 -3285> 2014-08-13 18:06:09.660013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=15] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 is on osd.296 -3284> 2014-08-13 18:06:09.660224 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active -3283> 2014-08-13 18:06:09.660243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194487 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: In Active, about to call activate -3282> 2014-08-13 18:06:09.660265 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - purged_snaps [] cached_removed_snaps [] -3281> 2014-08-13 18:06:09.659928 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -3280> 2014-08-13 18:06:09.660282 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - snap_trimq [] -3279> 2014-08-13 18:06:09.660300 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - not complete, missing(62) -3278> 2014-08-13 18:06:09.660221 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=14] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 is on osd.296 -3277> 2014-08-13 18:06:09.660290 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] flushed -3276> 2014-08-13 18:06:09.660433 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=13] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 is on osd.296 -3275> 2014-08-13 18:06:09.660686 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=12] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 is on osd.296 -3274> 2014-08-13 18:06:09.660915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=11] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 is on osd.296 -3273> 2014-08-13 18:06:09.661180 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=10] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 is on osd.296 -3272> 2014-08-13 18:06:09.661405 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=9] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 is on osd.296 -3271> 2014-08-13 18:06:09.661754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate - starting recovery -3270> 2014-08-13 18:06:09.661785 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] -3269> 2014-08-13 18:06:09.661626 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=8] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 is on osd.296 -3268> 2014-08-13 18:06:09.661828 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -3267> 2014-08-13 18:06:09.661851 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 is up to date, queueing in pending_activators -3266> 2014-08-13 18:06:09.661926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.55 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -3265> 2014-08-13 18:06:09.661953 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -3264> 2014-08-13 18:06:09.661983 7f5c7c7e4700 0 log [INF] : 24.1f7 restarting backfill on osd.59 from (0'0,0'0] MAX to 194422'15456 -3263> 2014-08-13 18:06:09.661850 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=7] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 is on osd.296 -3262> 2014-08-13 18:06:09.662079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=6] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 is on osd.296 -3261> 2014-08-13 18:06:09.662329 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 sending log((192718'12872,194422'15456], crt=194422'15456) -3260> 2014-08-13 18:06:09.662364 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3b00c60 don't know epoch, doing nothing -3259> 2014-08-13 18:06:09.662369 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.59 192.168.1.112:6822/5265 -- pg_log(24.1f7 epoch 194497 log log((192718'12872,194422'15456], crt=194422'15456) query_epoch 194497) v4 -- ?+0 0x58bc200 -3258> 2014-08-13 18:06:09.662394 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.59 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -3257> 2014-08-13 18:06:09.662422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) -3256> 2014-08-13 18:06:09.662444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 is up to date, queueing in pending_activators -3255> 2014-08-13 18:06:09.662535 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate peer osd.351 24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -3254> 2014-08-13 18:06:09.662319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=5] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 is on osd.296 -3253> 2014-08-13 18:06:09.662618 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery primary has 62 missing -3252> 2014-08-13 18:06:09.662569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=4] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 is on osd.296 -3251> 2014-08-13 18:06:09.662754 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.4 -3250> 2014-08-13 18:06:09.662887 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.4 -3249> 2014-08-13 18:06:09.662823 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=3] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 is on osd.296 -3248> 2014-08-13 18:06:09.663021 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.4 -3247> 2014-08-13 18:06:09.663152 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.4 -3246> 2014-08-13 18:06:09.663078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=2] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 is on osd.296 -3245> 2014-08-13 18:06:09.663290 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.4 -3244> 2014-08-13 18:06:09.663421 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.4 -3243> 2014-08-13 18:06:09.663328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47 u=1] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 is on osd.296 -3242> 2014-08-13 18:06:09.663551 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.4 -3241> 2014-08-13 18:06:09.663681 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.4 -3240> 2014-08-13 18:06:09.663610 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} -3239> 2014-08-13 18:06:09.663811 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.4 -3238> 2014-08-13 18:06:09.663939 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.4 -3237> 2014-08-13 18:06:09.663931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate: adding 320 as a source -3236> 2014-08-13 18:06:09.664068 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.4 -3235> 2014-08-13 18:06:09.664200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.4 -3234> 2014-08-13 18:06:09.664328 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.4 -3233> 2014-08-13 18:06:09.664190 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 is on osd.320 -3232> 2014-08-13 18:06:09.664457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.4 -3231> 2014-08-13 18:06:09.664587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.4 -3230> 2014-08-13 18:06:09.664458 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 is on osd.320 -3229> 2014-08-13 18:06:09.664716 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.4 -3228> 2014-08-13 18:06:09.664728 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 is on osd.320 -3227> 2014-08-13 18:06:09.664846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.4 -3226> 2014-08-13 18:06:09.664989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.4 -3225> 2014-08-13 18:06:09.664981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 is on osd.320 -3224> 2014-08-13 18:06:09.665121 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.4 -3223> 2014-08-13 18:06:09.665253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.4 -3222> 2014-08-13 18:06:09.665384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.4 -3221> 2014-08-13 18:06:09.665240 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 is on osd.320 -3220> 2014-08-13 18:06:09.665514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.4 -3219> 2014-08-13 18:06:09.665652 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.4 -3218> 2014-08-13 18:06:09.665783 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.4 -3217> 2014-08-13 18:06:09.665648 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 is on osd.320 -3216> 2014-08-13 18:06:09.665913 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.4 -3215> 2014-08-13 18:06:09.665915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 is on osd.320 -3214> 2014-08-13 18:06:09.666045 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.4 -3213> 2014-08-13 18:06:09.666172 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.4 -3212> 2014-08-13 18:06:09.666172 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 is on osd.320 -3211> 2014-08-13 18:06:09.666310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.4 -3210> 2014-08-13 18:06:09.666438 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.4 -3209> 2014-08-13 18:06:09.666568 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.4 -3208> 2014-08-13 18:06:09.666426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 is on osd.320 -3207> 2014-08-13 18:06:09.666699 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.4 -3206> 2014-08-13 18:06:09.666701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 is on osd.320 -3205> 2014-08-13 18:06:09.666830 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.4 -3204> 2014-08-13 18:06:09.666957 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.4 -3203> 2014-08-13 18:06:09.666951 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 is on osd.320 -3202> 2014-08-13 18:06:09.667087 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.4 -3201> 2014-08-13 18:06:09.667220 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.4 -3200> 2014-08-13 18:06:09.667207 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 is on osd.320 -3199> 2014-08-13 18:06:09.667350 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.4 -3198> 2014-08-13 18:06:09.667478 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.4 -3197> 2014-08-13 18:06:09.667477 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 is on osd.320 -3196> 2014-08-13 18:06:09.667609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.4 -3195> 2014-08-13 18:06:09.667736 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.4 -3194> 2014-08-13 18:06:09.667726 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 is on osd.320 -3193> 2014-08-13 18:06:09.667864 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.4 -3192> 2014-08-13 18:06:09.667991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.4 -3191> 2014-08-13 18:06:09.667973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 is on osd.320 -3190> 2014-08-13 18:06:09.668120 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.4 -3189> 2014-08-13 18:06:09.668253 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.4 -3188> 2014-08-13 18:06:09.668227 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 is on osd.320 -3187> 2014-08-13 18:06:09.668383 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.4 -3186> 2014-08-13 18:06:09.668510 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.4 -3185> 2014-08-13 18:06:09.668482 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 is on osd.320 -3184> 2014-08-13 18:06:09.668638 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.4 -3183> 2014-08-13 18:06:09.668765 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.4 -3182> 2014-08-13 18:06:09.668743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 is on osd.320 -3181> 2014-08-13 18:06:09.668894 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.4 -3180> 2014-08-13 18:06:09.669024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.4 -3179> 2014-08-13 18:06:09.668991 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 is on osd.320 -3178> 2014-08-13 18:06:09.669154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.4 -3177> 2014-08-13 18:06:09.669289 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.4 -3176> 2014-08-13 18:06:09.669243 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 is on osd.320 -3175> 2014-08-13 18:06:09.669419 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.4 -3174> 2014-08-13 18:06:09.669548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.4 -3173> 2014-08-13 18:06:09.669506 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 is on osd.320 -3172> 2014-08-13 18:06:09.669677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.4 -3171> 2014-08-13 18:06:09.669804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.4 -3170> 2014-08-13 18:06:09.669780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 is on osd.320 -3169> 2014-08-13 18:06:09.669933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.4 -3168> 2014-08-13 18:06:09.670063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.4 -3167> 2014-08-13 18:06:09.670026 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 is on osd.320 -3166> 2014-08-13 18:06:09.670197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.4 -3165> 2014-08-13 18:06:09.670327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.4 -3164> 2014-08-13 18:06:09.670280 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 is on osd.320 -3163> 2014-08-13 18:06:09.670456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.4 -3162> 2014-08-13 18:06:09.670583 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.4 -3161> 2014-08-13 18:06:09.670712 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.4 -3160> 2014-08-13 18:06:09.670557 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 is on osd.320 -3159> 2014-08-13 18:06:09.670840 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -3158> 2014-08-13 18:06:09.671042 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] activate: adding 55 as a source -3157> 2014-08-13 18:06:09.670907 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 is on osd.320 -3156> 2014-08-13 18:06:09.671169 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.55 -3155> 2014-08-13 18:06:09.671304 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=61] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.55 -3154> 2014-08-13 18:06:09.671231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 is on osd.320 -3153> 2014-08-13 18:06:09.671441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=60] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.55 -3152> 2014-08-13 18:06:09.671581 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=59] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.55 -3151> 2014-08-13 18:06:09.671504 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 is on osd.320 -3150> 2014-08-13 18:06:09.671721 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=58] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.55 -3149> 2014-08-13 18:06:09.671867 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=57] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.55 -3148> 2014-08-13 18:06:09.671786 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 is on osd.320 -3147> 2014-08-13 18:06:09.672016 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=56] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.55 -3146> 2014-08-13 18:06:09.672168 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=55] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.55 -3145> 2014-08-13 18:06:09.672063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 is on osd.320 -3144> 2014-08-13 18:06:09.672327 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=54] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.55 -3143> 2014-08-13 18:06:09.672345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 is on osd.320 -3142> 2014-08-13 18:06:09.672484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=53] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.55 -3141> 2014-08-13 18:06:09.672643 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=52] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.55 -3140> 2014-08-13 18:06:09.672598 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 is on osd.320 -3139> 2014-08-13 18:06:09.672819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=51] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.55 -3138> 2014-08-13 18:06:09.672842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 is on osd.320 -3137> 2014-08-13 18:06:09.672986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=50] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.55 -3136> 2014-08-13 18:06:09.673156 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=49] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.55 -3135> 2014-08-13 18:06:09.673087 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 is on osd.320 -3134> 2014-08-13 18:06:09.673334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=48] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.55 -3133> 2014-08-13 18:06:09.673341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 is on osd.320 -3132> 2014-08-13 18:06:09.673512 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=47] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.55 -3131> 2014-08-13 18:06:09.673609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 is on osd.320 -3130> 2014-08-13 18:06:09.673691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=46] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.55 -3129> 2014-08-13 18:06:09.673870 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=45] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.55 -3128> 2014-08-13 18:06:09.673857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 is on osd.320 -3127> 2014-08-13 18:06:09.674054 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=44] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.55 -3126> 2014-08-13 18:06:09.674103 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 is on osd.320 -3125> 2014-08-13 18:06:09.674247 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=43] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.55 -3124> 2014-08-13 18:06:09.674360 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 is on osd.320 -3123> 2014-08-13 18:06:09.674435 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=42] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.55 -3122> 2014-08-13 18:06:09.674626 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=41] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.55 -3121> 2014-08-13 18:06:09.674625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 is on osd.320 -3120> 2014-08-13 18:06:09.674823 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=40] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.55 -3119> 2014-08-13 18:06:09.674884 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 is on osd.320 -3118> 2014-08-13 18:06:09.675022 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=39] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.55 -3117> 2014-08-13 18:06:09.675133 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 is on osd.320 -3116> 2014-08-13 18:06:09.675230 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=38] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.55 -3115> 2014-08-13 18:06:09.675437 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=37] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.55 -3114> 2014-08-13 18:06:09.675391 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 is on osd.320 -3113> 2014-08-13 18:06:09.675646 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=36] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.55 -3112> 2014-08-13 18:06:09.675653 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 is on osd.320 -3111> 2014-08-13 18:06:09.675858 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=35] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.55 -3110> 2014-08-13 18:06:09.675901 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 is on osd.320 -3109> 2014-08-13 18:06:09.676073 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=34] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.55 -3108> 2014-08-13 18:06:09.676297 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=33] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.55 -3107> 2014-08-13 18:06:09.676230 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 is on osd.320 -3106> 2014-08-13 18:06:09.676518 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=32] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.55 -3105> 2014-08-13 18:06:09.676530 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 is on osd.320 -3104> 2014-08-13 18:06:09.676743 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=31] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.55 -3103> 2014-08-13 18:06:09.676792 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} -3102> 2014-08-13 18:06:09.676973 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=30] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.55 -3101> 2014-08-13 18:06:09.677096 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] activate: adding 344 as a source -3100> 2014-08-13 18:06:09.677210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=29] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.55 -3099> 2014-08-13 18:06:09.677349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24 194416'19599 also missing on osd.344 (past last_backfill 0//0//-1) -3098> 2014-08-13 18:06:09.677444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=28] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.55 -3097> 2014-08-13 18:06:09.677629 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24 194422'19611 also missing on osd.344 (past last_backfill 0//0//-1) -3096> 2014-08-13 18:06:09.677681 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=27] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.55 -3095> 2014-08-13 18:06:09.677887 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24 194416'19563 also missing on osd.344 (past last_backfill 0//0//-1) -3094> 2014-08-13 18:06:09.677922 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=26] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.55 -3093> 2014-08-13 18:06:09.678137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24 194370'19527 also missing on osd.344 (past last_backfill 0//0//-1) -3092> 2014-08-13 18:06:09.678163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=25] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.55 -3091> 2014-08-13 18:06:09.678394 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24 194370'19537 also missing on osd.344 (past last_backfill 0//0//-1) -3090> 2014-08-13 18:06:09.678411 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=24] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.55 -3089> 2014-08-13 18:06:09.678656 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24 194416'19571 also missing on osd.344 (past last_backfill 0//0//-1) -3088> 2014-08-13 18:06:09.678659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=23] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.55 -3087> 2014-08-13 18:06:09.678903 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24 194416'19597 also missing on osd.344 (past last_backfill 0//0//-1) -3086> 2014-08-13 18:06:09.678909 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=22] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.55 -3085> 2014-08-13 18:06:09.679156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24 194422'19605 also missing on osd.344 (past last_backfill 0//0//-1) -3084> 2014-08-13 18:06:09.679162 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=21] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.55 -3083> 2014-08-13 18:06:09.679406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24 194370'19529 also missing on osd.344 (past last_backfill 0//0//-1) -3082> 2014-08-13 18:06:09.679421 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=20] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.55 -3081> 2014-08-13 18:06:09.679668 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24 194416'19569 also missing on osd.344 (past last_backfill 0//0//-1) -3080> 2014-08-13 18:06:09.679679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=19] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.55 -3079> 2014-08-13 18:06:09.679915 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24 194370'19523 also missing on osd.344 (past last_backfill 0//0//-1) -3078> 2014-08-13 18:06:09.679940 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=18] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.55 -3077> 2014-08-13 18:06:09.680167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24 194416'19587 also missing on osd.344 (past last_backfill 0//0//-1) -3076> 2014-08-13 18:06:09.680210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=17] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.55 -3075> 2014-08-13 18:06:09.680416 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24 194416'19579 also missing on osd.344 (past last_backfill 0//0//-1) -3074> 2014-08-13 18:06:09.680480 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=16] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.55 -3073> 2014-08-13 18:06:09.680678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24 194416'19573 also missing on osd.344 (past last_backfill 0//0//-1) -3072> 2014-08-13 18:06:09.680753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=15] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.55 -3071> 2014-08-13 18:06:09.680924 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24 194416'19583 also missing on osd.344 (past last_backfill 0//0//-1) -3070> 2014-08-13 18:06:09.681028 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=14] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.55 -3069> 2014-08-13 18:06:09.681176 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24 194416'19591 also missing on osd.344 (past last_backfill 0//0//-1) -3068> 2014-08-13 18:06:09.681317 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=13] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.55 -3067> 2014-08-13 18:06:09.681435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24 194370'19541 also missing on osd.344 (past last_backfill 0//0//-1) -3066> 2014-08-13 18:06:09.681600 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=12] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.55 -3065> 2014-08-13 18:06:09.681689 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24 194383'19555 also missing on osd.344 (past last_backfill 0//0//-1) -3064> 2014-08-13 18:06:09.681885 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=11] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.55 -3063> 2014-08-13 18:06:09.681934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24 194416'19595 also missing on osd.344 (past last_backfill 0//0//-1) -3062> 2014-08-13 18:06:09.682182 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24 194383'19559 also missing on osd.344 (past last_backfill 0//0//-1) -3061> 2014-08-13 18:06:09.682174 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=10] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.55 -3060> 2014-08-13 18:06:09.682442 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24 194416'19585 also missing on osd.344 (past last_backfill 0//0//-1) -3059> 2014-08-13 18:06:09.682469 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=9] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.55 -3058> 2014-08-13 18:06:09.682696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24 194377'19547 also missing on osd.344 (past last_backfill 0//0//-1) -3057> 2014-08-13 18:06:09.682763 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=8] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.55 -3056> 2014-08-13 18:06:09.682941 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24 194422'19613 also missing on osd.344 (past last_backfill 0//0//-1) -3055> 2014-08-13 18:06:09.683063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=7] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.55 -3054> 2014-08-13 18:06:09.683242 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24 194416'19567 also missing on osd.344 (past last_backfill 0//0//-1) -3053> 2014-08-13 18:06:09.683369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=6] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.55 -3052> 2014-08-13 18:06:09.683517 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24 194422'19603 also missing on osd.344 (past last_backfill 0//0//-1) -3051> 2014-08-13 18:06:09.683674 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=5] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.55 -3050> 2014-08-13 18:06:09.683782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24 194370'19521 also missing on osd.344 (past last_backfill 0//0//-1) -3049> 2014-08-13 18:06:09.683981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=4] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.55 -3048> 2014-08-13 18:06:09.684043 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24 194422'19607 also missing on osd.344 (past last_backfill 0//0//-1) -3047> 2014-08-13 18:06:09.684295 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24 194416'19575 also missing on osd.344 (past last_backfill 0//0//-1) -3046> 2014-08-13 18:06:09.684296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=3] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.55 -3045> 2014-08-13 18:06:09.684561 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24 194416'19593 also missing on osd.344 (past last_backfill 0//0//-1) -3044> 2014-08-13 18:06:09.684611 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=2] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.55 -3043> 2014-08-13 18:06:09.684808 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24 194370'19539 also missing on osd.344 (past last_backfill 0//0//-1) -3042> 2014-08-13 18:06:09.684926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62 u=1] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.55 -3041> 2014-08-13 18:06:09.685053 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24 194370'19525 also missing on osd.344 (past last_backfill 0//0//-1) -3040> 2014-08-13 18:06:09.685309 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24 194422'19609 also missing on osd.344 (past last_backfill 0//0//-1) -3039> 2014-08-13 18:06:09.685250 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -3038> 2014-08-13 18:06:09.685570 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24 194377'19553 also missing on osd.344 (past last_backfill 0//0//-1) -3037> 2014-08-13 18:06:09.685649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 59 as a source -3036> 2014-08-13 18:06:09.685817 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24 194422'19601 also missing on osd.344 (past last_backfill 0//0//-1) -3035> 2014-08-13 18:06:09.685971 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 also missing on osd.59 (past last_backfill 0//0//-1) -3034> 2014-08-13 18:06:09.686062 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24 194370'19531 also missing on osd.344 (past last_backfill 0//0//-1) -3033> 2014-08-13 18:06:09.686298 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 also missing on osd.59 (past last_backfill 0//0//-1) -3032> 2014-08-13 18:06:09.686349 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24 194377'19549 also missing on osd.344 (past last_backfill 0//0//-1) -3031> 2014-08-13 18:06:09.686622 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24 194383'19557 also missing on osd.344 (past last_backfill 0//0//-1) -3030> 2014-08-13 18:06:09.686620 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 also missing on osd.59 (past last_backfill 0//0//-1) -3029> 2014-08-13 18:06:09.686880 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24 194393'19561 also missing on osd.344 (past last_backfill 0//0//-1) -3028> 2014-08-13 18:06:09.686942 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 also missing on osd.59 (past last_backfill 0//0//-1) -3027> 2014-08-13 18:06:09.687126 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24 194370'19533 also missing on osd.344 (past last_backfill 0//0//-1) -3026> 2014-08-13 18:06:09.687268 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 also missing on osd.59 (past last_backfill 0//0//-1) -3025> 2014-08-13 18:06:09.687383 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24 194416'19589 also missing on osd.344 (past last_backfill 0//0//-1) -3024> 2014-08-13 18:06:09.687645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24 194377'19551 also missing on osd.344 (past last_backfill 0//0//-1) -3023> 2014-08-13 18:06:09.687590 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 also missing on osd.59 (past last_backfill 0//0//-1) -3022> 2014-08-13 18:06:09.687891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24 194416'19565 also missing on osd.344 (past last_backfill 0//0//-1) -3021> 2014-08-13 18:06:09.687908 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 also missing on osd.59 (past last_backfill 0//0//-1) -3020> 2014-08-13 18:06:09.688136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24 194370'19543 also missing on osd.344 (past last_backfill 0//0//-1) -3019> 2014-08-13 18:06:09.688234 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 also missing on osd.59 (past last_backfill 0//0//-1) -3018> 2014-08-13 18:06:09.688392 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24 194416'19577 also missing on osd.344 (past last_backfill 0//0//-1) -3017> 2014-08-13 18:06:09.688554 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 also missing on osd.59 (past last_backfill 0//0//-1) -3016> 2014-08-13 18:06:09.688654 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24 194370'19535 also missing on osd.344 (past last_backfill 0//0//-1) -3015> 2014-08-13 18:06:09.688898 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24 194416'19581 also missing on osd.344 (past last_backfill 0//0//-1) -3014> 2014-08-13 18:06:09.688874 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 also missing on osd.59 (past last_backfill 0//0//-1) -3013> 2014-08-13 18:06:09.689142 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] search_for_missing 2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24 194377'19545 also missing on osd.344 (past last_backfill 0//0//-1) -3012> 2014-08-13 18:06:09.689199 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 also missing on osd.59 (past last_backfill 0//0//-1) -3011> 2014-08-13 18:06:09.689409 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] needs_recovery_map missing {87970181/benchmark_data_storage0111-ib.pocstor_19103_object18341/head//24=194416'19599,ac051181/benchmark_data_storage0111-ib.pocstor_41231_object8034/head//24=194422'19611,cdf5181/benchmark_data_storage0111-ib.pocstor_19103_object306/head//24=194416'19563,9f97181/benchmark_data_storage0111-ib.pocstor_12495_object2606/head//24=194370'19527,b4d9c181/benchmark_data_storage0111-ib.pocstor_12495_object4276/head//24=194370'19537,2e3cd181/benchmark_data_storage0111-ib.pocstor_19103_object2127/head//24=194416'19571,5251e181/benchmark_data_storage0111-ib.pocstor_19103_object16964/head//24=194416'19597,68d06581/benchmark_data_storage0111-ib.pocstor_41231_object5145/head//24=194422'19605,6cc66581/benchmark_data_storage0111-ib.pocstor_12495_object2706/head//24=194370'19529,34b89581/benchmark_data_storage0111-ib.pocstor_19103_object1606/head//24=194416'19569,1a0a9581/benchmark_data_storage0111-ib.pocstor_12495_object1664/head//24=194370'19523,5814a581/benchmark_data_storage0111-ib.pocstor_19103_object12436/head//24=194416'19587,26cca581/benchmark_data_storage0111-ib.pocstor_19103_object10174/head//24=194416'19579,c27d581/benchmark_data_storage0111-ib.pocstor_19103_object4615/head//24=194416'19573,b5d9d581/benchmark_data_storage0111-ib.pocstor_19103_object12161/head//24=194416'19583,7b28e581/benchmark_data_storage0111-ib.pocstor_19103_object13215/head//24=194416'19591,15682981/benchmark_data_storage0111-ib.pocstor_12495_object5622/head//24=194370'19541,d20c2981/benchmark_data_storage0111-ib.pocstor_25065_object5952/head//24=194383'19555,a8793981/benchmark_data_storage0111-ib.pocstor_19103_object16567/head//24=194416'19595,a1d3981/benchmark_data_storage0111-ib.pocstor_25065_object6669/head//24=194383'19559,53f05981/benchmark_data_storage0111-ib.pocstor_19103_object12333/head//24=194416'19585,a6965981/benchmark_data_storage0111-ib.pocstor_25065_object217/head//24=194377'19547,dd406981/benchmark_data_storage0111-ib.pocstor_41231_object8462/head//24=194422'19613,13e07981/benchmark_data_storage0111-ib.pocstor_19103_object923/head//24=194416'19567,89b27981/benchmark_data_storage0111-ib.pocstor_41231_object4420/head//24=194422'19603,c5f97981/benchmark_data_storage0111-ib.pocstor_12495_object941/head//24=194370'19521,c4789981/benchmark_data_storage0111-ib.pocstor_41231_object6380/head//24=194422'19607,ded89981/benchmark_data_storage0111-ib.pocstor_19103_object6547/head//24=194416'19575,3efa9981/benchmark_data_storage0111-ib.pocstor_19103_object13812/head//24=194416'19593,d00e9981/benchmark_data_storage0111-ib.pocstor_12495_object4332/head//24=194370'19539,b05a981/benchmark_data_storage0111-ib.pocstor_12495_object2151/head//24=194370'19525,58cdb981/benchmark_data_storage0111-ib.pocstor_41231_object7747/head//24=194422'19609,1a1dc981/benchmark_data_storage0111-ib.pocstor_25065_object2478/head//24=194377'19553,de9cd981/benchmark_data_storage0111-ib.pocstor_41231_object2785/head//24=194422'19601,22a5e981/benchmark_data_storage0111-ib.pocstor_12495_object2786/head//24=194370'19531,74481d81/benchmark_data_storage0111-ib.pocstor_25065_object462/head//24=194377'19549,98f22d81/benchmark_data_storage0111-ib.pocstor_25065_object6193/head//24=194383'19557,98592d81/benchmark_data_storage0111-ib.pocstor_48304_object63/head//24=194393'19561,61792d81/benchmark_data_storage0111-ib.pocstor_12495_object3455/head//24=194370'19533,9fa24d81/benchmark_data_storage0111-ib.pocstor_19103_object12986/head//24=194416'19589,9c244d81/benchmark_data_storage0111-ib.pocstor_25065_object574/head//24=194377'19551,127e4d81/benchmark_data_storage0111-ib.pocstor_19103_object449/head//24=194416'19565,51457d81/benchmark_data_storage0111-ib.pocstor_12495_object11553/head//24=194370'19543,b1c7d81/benchmark_data_storage0111-ib.pocstor_19103_object7870/head//24=194416'19577,e7ff7d81/benchmark_data_storage0111-ib.pocstor_12495_object3853/head//24=194370'19535,866fad81/benchmark_data_storage0111-ib.pocstor_19103_object11423/head//24=194416'19581,2c3dbd81/benchmark_data_storage0111-ib.pocstor_25065_object169/head//24=194377'19545} -3010> 2014-08-13 18:06:09.689521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 also missing on osd.59 (past last_backfill 0//0//-1) -3009> 2014-08-13 18:06:09.689729 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] build_might_have_unfound -3008> 2014-08-13 18:06:09.689840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 also missing on osd.59 (past last_backfill 0//0//-1) -3007> 2014-08-13 18:06:09.689973 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] _calc_past_interval_range: already have past intervals back to 194466 -3006> 2014-08-13 18:06:09.690161 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 also missing on osd.59 (past last_backfill 0//0//-1) -3005> 2014-08-13 18:06:09.690231 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] build_might_have_unfound: built 55,296,307,320,344 -3004> 2014-08-13 18:06:09.690502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Activate Finished -3003> 2014-08-13 18:06:09.690487 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 also missing on osd.59 (past last_backfill 0//0//-1) -3002> 2014-08-13 18:06:09.690759 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] enter Started/Primary/Active/Activating -3001> 2014-08-13 18:06:09.690808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 also missing on osd.59 (past last_backfill 0//0//-1) -3000> 2014-08-13 18:06:09.691006 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2999> 2014-08-13 18:06:09.691131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 also missing on osd.59 (past last_backfill 0//0//-1) -2998> 2014-08-13 18:06:09.691258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2997> 2014-08-13 18:06:09.691622 7f5c7bde3700 10 log is not dirty -2996> 2014-08-13 18:06:09.691695 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [15,4,52]/[15,4,61] -- 15/15 -2995> 2014-08-13 18:06:09.691732 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2994> 2014-08-13 18:06:09.691729 7f5c835ef700 5 -- op tracker -- , seq: 627, time: 2014-08-13 18:06:09.691728, event: done, request: pg_notify(24.1f7(2),24.181(17) epoch 194497) v5 -2993> 2014-08-13 18:06:09.691740 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: Started advmap -2992> 2014-08-13 18:06:09.691457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 also missing on osd.59 (past last_backfill 0//0//-1) -2991> 2014-08-13 18:06:09.691760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down -2990> 2014-08-13 18:06:09.691774 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2989> 2014-08-13 18:06:09.691779 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2988> 2014-08-13 18:06:09.691779 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map -2987> 2014-08-13 18:06:09.691789 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 16 ==== pg_notify(24.1bb(40) epoch 194497) v5 ==== 3004+0+0 (2719403443 0 0) 0x5a38700 con 0x3c55540 -2986> 2014-08-13 18:06:09.691810 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2985> 2014-08-13 18:06:09.691813 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2984> 2014-08-13 18:06:09.691815 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38700 pg_notify(24.1bb(40) epoch 194497) v5 -2983> 2014-08-13 18:06:09.691821 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.314107, event: header_read, request: pg_notify(24.1bb(40) epoch 194497) v5 -2982> 2014-08-13 18:06:09.691829 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.314109, event: throttled, request: pg_notify(24.1bb(40) epoch 194497) v5 -2981> 2014-08-13 18:06:09.691835 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.314275, event: all_read, request: pg_notify(24.1bb(40) epoch 194497) v5 -2980> 2014-08-13 18:06:09.691844 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.691809, event: dispatched, request: pg_notify(24.1bb(40) epoch 194497) v5 -2979> 2014-08-13 18:06:09.691850 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.691850, event: waiting_for_osdmap, request: pg_notify(24.1bb(40) epoch 194497) v5 -2978> 2014-08-13 18:06:09.691857 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.337 -2977> 2014-08-13 18:06:09.691861 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38700 -2976> 2014-08-13 18:06:09.691866 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.691866, event: started, request: pg_notify(24.1bb(40) epoch 194497) v5 -2975> 2014-08-13 18:06:09.691896 7f5c835ef700 5 -- op tracker -- , seq: 628, time: 2014-08-13 18:06:09.691896, event: done, request: pg_notify(24.1bb(40) epoch 194497) v5 -2974> 2014-08-13 18:06:09.691891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] take_waiters -2973> 2014-08-13 18:06:09.691911 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2972> 2014-08-13 18:06:09.691915 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2971> 2014-08-13 18:06:09.691916 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2970> 2014-08-13 18:06:09.691920 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 11 ==== pg_info(1 pgs e194497:24.20e(2)) v4 ==== 866+0+0 (2412025547 0 0) 0x524e900 con 0x4854620 -2969> 2014-08-13 18:06:09.691936 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2968> 2014-08-13 18:06:09.691939 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2967> 2014-08-13 18:06:09.691941 7f5c835ef700 20 osd.4 194497 _dispatch 0x524e900 pg_info(1 pgs e194497:24.20e(2)) v4 -2966> 2014-08-13 18:06:09.691934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2965> 2014-08-13 18:06:09.691947 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.203597, event: header_read, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2964> 2014-08-13 18:06:09.691955 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.203601, event: throttled, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2963> 2014-08-13 18:06:09.691962 7f5c7bde3700 10 log is not dirty -2962> 2014-08-13 18:06:09.691961 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.203680, event: all_read, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2961> 2014-08-13 18:06:09.691967 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.691935, event: dispatched, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2960> 2014-08-13 18:06:09.691973 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.691973, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2959> 2014-08-13 18:06:09.691980 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.20e(2)) v4 from osd.15 -2958> 2014-08-13 18:06:09.691985 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524e900 -2957> 2014-08-13 18:06:09.691989 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.691989, event: started, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2956> 2014-08-13 18:06:09.691988 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_advance_map [31,338,4]/[31,338,4] -- 31/31 -2955> 2014-08-13 18:06:09.692012 7f5c835ef700 5 -- op tracker -- , seq: 629, time: 2014-08-13 18:06:09.692012, event: done, request: pg_info(1 pgs e194497:24.20e(2)) v4 -2954> 2014-08-13 18:06:09.692021 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2953> 2014-08-13 18:06:09.692028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: Started advmap -2952> 2014-08-13 18:06:09.692044 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2951> 2014-08-13 18:06:09.692048 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2950> 2014-08-13 18:06:09.692045 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] check_recovery_sources no source osds () went down -2949> 2014-08-13 18:06:09.692054 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.362 192.168.1.108:6842/8170 12 ==== pg_info(1 pgs e194497:24.3d) v4 ==== 724+0+0 (2559288265 0 0) 0x5a6fe00 con 0x49a53e0 -2948> 2014-08-13 18:06:09.692067 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2947> 2014-08-13 18:06:09.692070 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2946> 2014-08-13 18:06:09.692072 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fe00 pg_info(1 pgs e194497:24.3d) v4 -2945> 2014-08-13 18:06:09.692063 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map -2944> 2014-08-13 18:06:09.692077 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.605026, event: header_read, request: pg_info(1 pgs e194497:24.3d) v4 -2943> 2014-08-13 18:06:09.692084 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.605029, event: throttled, request: pg_info(1 pgs e194497:24.3d) v4 -2942> 2014-08-13 18:06:09.692089 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.605124, event: all_read, request: pg_info(1 pgs e194497:24.3d) v4 -2941> 2014-08-13 18:06:09.691777 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 also missing on osd.59 (past last_backfill 0//0//-1) -2940> 2014-08-13 18:06:09.692095 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.692066, event: dispatched, request: pg_info(1 pgs e194497:24.3d) v4 -2939> 2014-08-13 18:06:09.692102 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.692102, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3d) v4 -2938> 2014-08-13 18:06:09.692107 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.362 -2937> 2014-08-13 18:06:09.692112 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fe00 -2936> 2014-08-13 18:06:09.692116 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.692116, event: started, request: pg_info(1 pgs e194497:24.3d) v4 -2935> 2014-08-13 18:06:09.692137 7f5c835ef700 5 -- op tracker -- , seq: 630, time: 2014-08-13 18:06:09.692137, event: done, request: pg_info(1 pgs e194497:24.3d) v4 -2934> 2014-08-13 18:06:09.692148 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2933> 2014-08-13 18:06:09.692151 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2932> 2014-08-13 18:06:09.692156 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.286 192.168.1.113:6820/19173 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc4800 con 0x3cf2940 -2931> 2014-08-13 18:06:09.692170 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2930> 2014-08-13 18:06:09.692173 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2929> 2014-08-13 18:06:09.692175 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc4800 osd_map(194497..194497 src has 185139..194497) v3 -2928> 2014-08-13 18:06:09.692164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] take_waiters -2927> 2014-08-13 18:06:09.692196 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -2926> 2014-08-13 18:06:09.692202 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -2925> 2014-08-13 18:06:09.692205 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2924> 2014-08-13 18:06:09.692207 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2923> 2014-08-13 18:06:09.692193 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2922> 2014-08-13 18:06:09.692210 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2921> 2014-08-13 18:06:09.692213 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 18 ==== pg_notify(24.3d(38) epoch 194497) v5 ==== 2890+0+0 (2519417228 0 0) 0x5a38a80 con 0x4af8580 -2920> 2014-08-13 18:06:09.692227 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2919> 2014-08-13 18:06:09.692230 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2918> 2014-08-13 18:06:09.692232 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a38a80 pg_notify(24.3d(38) epoch 194497) v5 -2917> 2014-08-13 18:06:09.692234 7f5c7bde3700 10 log is not dirty -2916> 2014-08-13 18:06:09.692237 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.294186, event: header_read, request: pg_notify(24.3d(38) epoch 194497) v5 -2915> 2014-08-13 18:06:09.692245 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.294189, event: throttled, request: pg_notify(24.3d(38) epoch 194497) v5 -2914> 2014-08-13 18:06:09.692251 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.294352, event: all_read, request: pg_notify(24.3d(38) epoch 194497) v5 -2913> 2014-08-13 18:06:09.692256 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.692227, event: dispatched, request: pg_notify(24.3d(38) epoch 194497) v5 -2912> 2014-08-13 18:06:09.692263 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.692263, event: waiting_for_osdmap, request: pg_notify(24.3d(38) epoch 194497) v5 -2911> 2014-08-13 18:06:09.692269 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 -2910> 2014-08-13 18:06:09.692273 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a38a80 -2909> 2014-08-13 18:06:09.692260 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [285,336,4]/[285,336,4] -- 285/285 -2908> 2014-08-13 18:06:09.692277 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.692277, event: started, request: pg_notify(24.3d(38) epoch 194497) v5 -2907> 2014-08-13 18:06:09.692288 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2906> 2014-08-13 18:06:09.692294 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: Started advmap -2905> 2014-08-13 18:06:09.692307 7f5c835ef700 5 -- op tracker -- , seq: 631, time: 2014-08-13 18:06:09.692307, event: done, request: pg_notify(24.3d(38) epoch 194497) v5 -2904> 2014-08-13 18:06:09.692312 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down -2903> 2014-08-13 18:06:09.692328 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2902> 2014-08-13 18:06:09.692332 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2901> 2014-08-13 18:06:09.692329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map -2900> 2014-08-13 18:06:09.692337 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.320 192.168.1.106:6820/10050 15 ==== pg_notify(24.181(17) epoch 194497) v5 ==== 1715+0+0 (3148320142 0 0) 0x5a68c40 con 0x4b12520 -2899> 2014-08-13 18:06:09.692351 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2898> 2014-08-13 18:06:09.692353 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2897> 2014-08-13 18:06:09.692355 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68c40 pg_notify(24.181(17) epoch 194497) v5 -2896> 2014-08-13 18:06:09.692360 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.218052, event: header_read, request: pg_notify(24.181(17) epoch 194497) v5 -2895> 2014-08-13 18:06:09.692367 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.218054, event: throttled, request: pg_notify(24.181(17) epoch 194497) v5 -2894> 2014-08-13 18:06:09.692373 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.218238, event: all_read, request: pg_notify(24.181(17) epoch 194497) v5 -2893> 2014-08-13 18:06:09.692378 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.692350, event: dispatched, request: pg_notify(24.181(17) epoch 194497) v5 -2892> 2014-08-13 18:06:09.692385 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.692385, event: waiting_for_osdmap, request: pg_notify(24.181(17) epoch 194497) v5 -2891> 2014-08-13 18:06:09.692391 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.320 -2890> 2014-08-13 18:06:09.692393 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68c40 -2889> 2014-08-13 18:06:09.692398 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.692398, event: started, request: pg_notify(24.181(17) epoch 194497) v5 -2888> 2014-08-13 18:06:09.692394 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] take_waiters -2887> 2014-08-13 18:06:09.692097 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 also missing on osd.59 (past last_backfill 0//0//-1) -2886> 2014-08-13 18:06:09.692419 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2885> 2014-08-13 18:06:09.692437 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2884> 2014-08-13 18:06:09.692454 7f5c7bde3700 10 log is not dirty -2883> 2014-08-13 18:06:09.692478 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_advance_map [323,359,4]/[323,359,4] -- 323/323 -2882> 2014-08-13 18:06:09.692501 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2881> 2014-08-13 18:06:09.692507 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] state: Started advmap -2880> 2014-08-13 18:06:09.692541 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] check_recovery_sources no source osds () went down -2879> 2014-08-13 18:06:09.692569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map -2878> 2014-08-13 18:06:09.692311 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -2877> 2014-08-13 18:06:09.692586 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] take_waiters -2876> 2014-08-13 18:06:09.692613 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2875> 2014-08-13 18:06:09.692630 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.12a( v 194215'491 (0'0,194215'491] local-les=194496 n=41 ec=183096 les/c 194496/194496 194495/194495/192978) [323,359,4] r=2 lpr=194495 pi=192978-194494/27 luod=0'0 crt=192888'463 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2874> 2014-08-13 18:06:09.692661 7f5c7bde3700 10 log is not dirty -2873> 2014-08-13 18:06:09.692685 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,44,362]/[4,44,362] -- 4/4 -2872> 2014-08-13 18:06:09.692708 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2871> 2014-08-13 18:06:09.692714 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2870> 2014-08-13 18:06:09.692422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 also missing on osd.59 (past last_backfill 0//0//-1) -2869> 2014-08-13 18:06:09.692731 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2868> 2014-08-13 18:06:09.692755 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4) went down -2867> 2014-08-13 18:06:09.692773 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2866> 2014-08-13 18:06:09.692788 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2865> 2014-08-13 18:06:09.692606 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] flushed -2864> 2014-08-13 18:06:09.692921 7f5c835ef700 5 -- op tracker -- , seq: 632, time: 2014-08-13 18:06:09.692921, event: done, request: pg_notify(24.181(17) epoch 194497) v5 -2863> 2014-08-13 18:06:09.692965 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2862> 2014-08-13 18:06:09.692970 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2861> 2014-08-13 18:06:09.692978 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.367 192.168.1.114:6840/55513 11 ==== pg_info(1 pgs e194497:24.9b) v4 ==== 724+0+0 (1807575895 0 0) 0x4df3800 con 0x4b11ce0 -2860> 2014-08-13 18:06:09.692998 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2859> 2014-08-13 18:06:09.693000 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2858> 2014-08-13 18:06:09.693002 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df3800 pg_info(1 pgs e194497:24.9b) v4 -2857> 2014-08-13 18:06:09.693009 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.524014, event: header_read, request: pg_info(1 pgs e194497:24.9b) v4 -2856> 2014-08-13 18:06:09.693016 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.524017, event: throttled, request: pg_info(1 pgs e194497:24.9b) v4 -2855> 2014-08-13 18:06:09.693022 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.524109, event: all_read, request: pg_info(1 pgs e194497:24.9b) v4 -2854> 2014-08-13 18:06:09.693028 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.692997, event: dispatched, request: pg_info(1 pgs e194497:24.9b) v4 -2853> 2014-08-13 18:06:09.693034 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.693034, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.9b) v4 -2852> 2014-08-13 18:06:09.693041 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.367 -2851> 2014-08-13 18:06:09.693046 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df3800 -2850> 2014-08-13 18:06:09.693051 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.693051, event: started, request: pg_info(1 pgs e194497:24.9b) v4 -2849> 2014-08-13 18:06:09.692741 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 also missing on osd.59 (past last_backfill 0//0//-1) -2848> 2014-08-13 18:06:09.693081 7f5c835ef700 5 -- op tracker -- , seq: 633, time: 2014-08-13 18:06:09.693081, event: done, request: pg_info(1 pgs e194497:24.9b) v4 -2847> 2014-08-13 18:06:09.693093 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2846> 2014-08-13 18:06:09.693096 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2845> 2014-08-13 18:06:09.693086 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] -2844> 2014-08-13 18:06:09.693103 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.373 192.168.1.108:6840/7435 18 ==== pg_notify(22.3d9(47) epoch 194497) v5 ==== 3303+0+0 (2574027154 0 0) 0x4bf1880 con 0x4b111e0 -2843> 2014-08-13 18:06:09.693118 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2842> 2014-08-13 18:06:09.693121 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2841> 2014-08-13 18:06:09.693123 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf1880 pg_notify(22.3d9(47) epoch 194497) v5 -2840> 2014-08-13 18:06:09.693118 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2839> 2014-08-13 18:06:09.693129 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.244992, event: header_read, request: pg_notify(22.3d9(47) epoch 194497) v5 -2838> 2014-08-13 18:06:09.693136 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.244994, event: throttled, request: pg_notify(22.3d9(47) epoch 194497) v5 -2837> 2014-08-13 18:06:09.693142 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.245135, event: all_read, request: pg_notify(22.3d9(47) epoch 194497) v5 -2836> 2014-08-13 18:06:09.693147 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.693117, event: dispatched, request: pg_notify(22.3d9(47) epoch 194497) v5 -2835> 2014-08-13 18:06:09.693154 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.693154, event: waiting_for_osdmap, request: pg_notify(22.3d9(47) epoch 194497) v5 -2834> 2014-08-13 18:06:09.693160 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.373 -2833> 2014-08-13 18:06:09.693164 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf1880 -2832> 2014-08-13 18:06:09.693168 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.693168, event: started, request: pg_notify(22.3d9(47) epoch 194497) v5 -2831> 2014-08-13 18:06:09.693202 7f5c835ef700 5 -- op tracker -- , seq: 634, time: 2014-08-13 18:06:09.693202, event: done, request: pg_notify(22.3d9(47) epoch 194497) v5 -2830> 2014-08-13 18:06:09.693247 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2829> 2014-08-13 18:06:09.693250 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2828> 2014-08-13 18:06:09.693255 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 14 ==== osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3829533981 0 0) 0x59e2c00 con 0x4b10b00 -2827> 2014-08-13 18:06:09.693276 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2826> 2014-08-13 18:06:09.693279 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2825> 2014-08-13 18:06:09.693280 7f5c835ef700 20 osd.4 194497 _dispatch 0x59e2c00 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2824> 2014-08-13 18:06:09.693290 7f5c835ef700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.488248, event: header_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2823> 2014-08-13 18:06:09.693284 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2689 -2822> 2014-08-13 18:06:09.693302 7f5c835ef700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.488253, event: throttled, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2821> 2014-08-13 18:06:09.693314 7f5c835ef700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.488386, event: all_read, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2820> 2014-08-13 18:06:09.693308 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2819> 2014-08-13 18:06:09.693324 7f5c835ef700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.693275, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2818> 2014-08-13 18:06:09.693334 7f5c835ef700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.693334, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2817> 2014-08-13 18:06:09.693345 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 -2816> 2014-08-13 18:06:09.693325 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2815> 2014-08-13 18:06:09.693353 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x59e2c00 -2814> 2014-08-13 18:06:09.693358 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.397 192.168.1.105:6808/41085 194497 -2813> 2014-08-13 18:06:09.693366 7f5c835ef700 10 osd.4 194497 note_peer_epoch osd.397 has 194497 -2812> 2014-08-13 18:06:09.693356 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2811> 2014-08-13 18:06:09.693376 7f5c7bde3700 10 log is not dirty -2810> 2014-08-13 18:06:09.693372 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a7ad0 prio 127 cost 0 latency 0.205123 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2809> 2014-08-13 18:06:09.693059 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 also missing on osd.59 (past last_backfill 0//0//-1) -2808> 2014-08-13 18:06:09.693384 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2807> 2014-08-13 18:06:09.693388 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2806> 2014-08-13 18:06:09.693394 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.376 192.168.1.108:6838/6893 21 ==== pg_notify(22.48e(34) epoch 194497) v5 ==== 2624+0+0 (2407140341 0 0) 0x588bd40 con 0x4b2bc80 -2805> 2014-08-13 18:06:09.693407 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2804> 2014-08-13 18:06:09.693410 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2803> 2014-08-13 18:06:09.693412 7f5c835ef700 20 osd.4 194497 _dispatch 0x588bd40 pg_notify(22.48e(34) epoch 194497) v5 -2802> 2014-08-13 18:06:09.693417 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.216694, event: header_read, request: pg_notify(22.48e(34) epoch 194497) v5 -2801> 2014-08-13 18:06:09.693401 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,287,309]/[4,287,309] -- 4/4 -2800> 2014-08-13 18:06:09.693429 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2799> 2014-08-13 18:06:09.693424 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.216696, event: throttled, request: pg_notify(22.48e(34) epoch 194497) v5 -2798> 2014-08-13 18:06:09.693438 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.216850, event: all_read, request: pg_notify(22.48e(34) epoch 194497) v5 -2797> 2014-08-13 18:06:09.693443 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.693406, event: dispatched, request: pg_notify(22.48e(34) epoch 194497) v5 -2796> 2014-08-13 18:06:09.693450 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.693450, event: waiting_for_osdmap, request: pg_notify(22.48e(34) epoch 194497) v5 -2795> 2014-08-13 18:06:09.693456 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.376 -2794> 2014-08-13 18:06:09.693459 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588bd40 -2793> 2014-08-13 18:06:09.693435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2792> 2014-08-13 18:06:09.693464 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.693463, event: started, request: pg_notify(22.48e(34) epoch 194497) v5 -2791> 2014-08-13 18:06:09.693465 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2790> 2014-08-13 18:06:09.693486 7f5c835ef700 5 -- op tracker -- , seq: 636, time: 2014-08-13 18:06:09.693486, event: done, request: pg_notify(22.48e(34) epoch 194497) v5 -2789> 2014-08-13 18:06:09.693484 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,287) went down -2788> 2014-08-13 18:06:09.693512 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2787> 2014-08-13 18:06:09.693516 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2786> 2014-08-13 18:06:09.693507 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2785> 2014-08-13 18:06:09.693521 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.327 192.168.1.106:6804/5112 12 ==== pg_notify(24.b6(2) epoch 194497) v5 ==== 870+0+0 (3636328315 0 0) 0x5352680 con 0x4c58420 -2784> 2014-08-13 18:06:09.693536 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2783> 2014-08-13 18:06:09.693538 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2782> 2014-08-13 18:06:09.693540 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352680 pg_notify(24.b6(2) epoch 194497) v5 -2781> 2014-08-13 18:06:09.693545 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.230063, event: header_read, request: pg_notify(24.b6(2) epoch 194497) v5 -2780> 2014-08-13 18:06:09.693524 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2779> 2014-08-13 18:06:09.693552 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.230066, event: throttled, request: pg_notify(24.b6(2) epoch 194497) v5 -2778> 2014-08-13 18:06:09.693558 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.230183, event: all_read, request: pg_notify(24.b6(2) epoch 194497) v5 -2777> 2014-08-13 18:06:09.693563 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.693535, event: dispatched, request: pg_notify(24.b6(2) epoch 194497) v5 -2776> 2014-08-13 18:06:09.693570 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.693570, event: waiting_for_osdmap, request: pg_notify(24.b6(2) epoch 194497) v5 -2775> 2014-08-13 18:06:09.693578 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.327 -2774> 2014-08-13 18:06:09.693581 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352680 -2773> 2014-08-13 18:06:09.693585 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.693585, event: started, request: pg_notify(24.b6(2) epoch 194497) v5 -2772> 2014-08-13 18:06:09.693605 7f5c835ef700 5 -- op tracker -- , seq: 637, time: 2014-08-13 18:06:09.693604, event: done, request: pg_notify(24.b6(2) epoch 194497) v5 -2771> 2014-08-13 18:06:09.693617 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2770> 2014-08-13 18:06:09.693619 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2769> 2014-08-13 18:06:09.693624 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.25 192.168.1.103:6800/36920 5 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc6c00 con 0x4c58160 -2768> 2014-08-13 18:06:09.693637 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2767> 2014-08-13 18:06:09.693640 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2766> 2014-08-13 18:06:09.693642 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc6c00 osd_map(194497..194497 src has 185139..194497) v3 -2765> 2014-08-13 18:06:09.693650 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -2764> 2014-08-13 18:06:09.693655 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -2763> 2014-08-13 18:06:09.693658 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2762> 2014-08-13 18:06:09.693660 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2761> 2014-08-13 18:06:09.693666 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.342 192.168.1.110:6802/41438 13 ==== pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 ==== 5582+0+0 (1268055395 0 0) 0x5a6c7c0 con 0x4c5b440 -2760> 2014-08-13 18:06:09.693679 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2759> 2014-08-13 18:06:09.693682 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2758> 2014-08-13 18:06:09.693684 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c7c0 pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2757> 2014-08-13 18:06:09.693690 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.295071, event: header_read, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2756> 2014-08-13 18:06:09.693383 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 also missing on osd.59 (past last_backfill 0//0//-1) -2755> 2014-08-13 18:06:09.693697 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.295073, event: throttled, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2754> 2014-08-13 18:06:09.693704 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.295307, event: all_read, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2753> 2014-08-13 18:06:09.693711 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.693679, event: dispatched, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2752> 2014-08-13 18:06:09.693719 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.693718, event: waiting_for_osdmap, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2751> 2014-08-13 18:06:09.693725 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.342 -2750> 2014-08-13 18:06:09.693728 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c7c0 -2749> 2014-08-13 18:06:09.693733 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.693732, event: started, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2748> 2014-08-13 18:06:09.693771 7f5c835ef700 5 -- op tracker -- , seq: 638, time: 2014-08-13 18:06:09.693771, event: done, request: pg_notify(22.3e4(36),22.707(38) epoch 194497) v5 -2747> 2014-08-13 18:06:09.693793 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2746> 2014-08-13 18:06:09.693796 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2745> 2014-08-13 18:06:09.693801 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.368 192.168.1.102:6826/15861 11 ==== pg_notify(22.22c(28) epoch 194497) v5 ==== 2282+0+0 (2757487545 0 0) 0x5a6f8c0 con 0x4c5bde0 -2744> 2014-08-13 18:06:09.693814 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2743> 2014-08-13 18:06:09.693817 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2742> 2014-08-13 18:06:09.693819 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f8c0 pg_notify(22.22c(28) epoch 194497) v5 -2741> 2014-08-13 18:06:09.693824 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.308656, event: header_read, request: pg_notify(22.22c(28) epoch 194497) v5 -2740> 2014-08-13 18:06:09.693830 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.308657, event: throttled, request: pg_notify(22.22c(28) epoch 194497) v5 -2739> 2014-08-13 18:06:09.693836 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.308751, event: all_read, request: pg_notify(22.22c(28) epoch 194497) v5 -2738> 2014-08-13 18:06:09.693841 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.693814, event: dispatched, request: pg_notify(22.22c(28) epoch 194497) v5 -2737> 2014-08-13 18:06:09.693847 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.693847, event: waiting_for_osdmap, request: pg_notify(22.22c(28) epoch 194497) v5 -2736> 2014-08-13 18:06:09.693853 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.368 -2735> 2014-08-13 18:06:09.693856 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f8c0 -2734> 2014-08-13 18:06:09.693860 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.693860, event: started, request: pg_notify(22.22c(28) epoch 194497) v5 -2733> 2014-08-13 18:06:09.693880 7f5c835ef700 5 -- op tracker -- , seq: 639, time: 2014-08-13 18:06:09.693879, event: done, request: pg_notify(22.22c(28) epoch 194497) v5 -2732> 2014-08-13 18:06:09.693909 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2731> 2014-08-13 18:06:09.693913 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2730> 2014-08-13 18:06:09.693917 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.350 192.168.1.114:6832/53185 9 ==== pg_info(1 pgs e194497:24.9b) v4 ==== 748+0+0 (1763869706 0 0) 0x588bf00 con 0x4c5ee00 -2729> 2014-08-13 18:06:09.693930 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2728> 2014-08-13 18:06:09.693933 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2727> 2014-08-13 18:06:09.693935 7f5c835ef700 20 osd.4 194497 _dispatch 0x588bf00 pg_info(1 pgs e194497:24.9b) v4 -2726> 2014-08-13 18:06:09.693940 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.574921, event: header_read, request: pg_info(1 pgs e194497:24.9b) v4 -2725> 2014-08-13 18:06:09.693947 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.574924, event: throttled, request: pg_info(1 pgs e194497:24.9b) v4 -2724> 2014-08-13 18:06:09.693932 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] -2723> 2014-08-13 18:06:09.693953 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.575014, event: all_read, request: pg_info(1 pgs e194497:24.9b) v4 -2722> 2014-08-13 18:06:09.693958 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.693930, event: dispatched, request: pg_info(1 pgs e194497:24.9b) v4 -2721> 2014-08-13 18:06:09.693965 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.693964, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.9b) v4 -2720> 2014-08-13 18:06:09.693953 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2719> 2014-08-13 18:06:09.693971 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.350 -2718> 2014-08-13 18:06:09.693976 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588bf00 -2717> 2014-08-13 18:06:09.693980 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.693980, event: started, request: pg_info(1 pgs e194497:24.9b) v4 -2716> 2014-08-13 18:06:09.694000 7f5c835ef700 5 -- op tracker -- , seq: 640, time: 2014-08-13 18:06:09.693999, event: done, request: pg_info(1 pgs e194497:24.9b) v4 -2715> 2014-08-13 18:06:09.694010 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2714> 2014-08-13 18:06:09.694013 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2713> 2014-08-13 18:06:09.693703 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 also missing on osd.59 (past last_backfill 0//0//-1) -2712> 2014-08-13 18:06:09.694018 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.369 192.168.1.108:6824/4037 14 ==== pg_notify(22.f3(26) epoch 194497) v5 ==== 2168+0+0 (2069233888 0 0) 0x5a6e3c0 con 0x4c5e880 -2711> 2014-08-13 18:06:09.694031 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2710> 2014-08-13 18:06:09.694034 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2709> 2014-08-13 18:06:09.694036 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e3c0 pg_notify(22.f3(26) epoch 194497) v5 -2708> 2014-08-13 18:06:09.694042 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.297786, event: header_read, request: pg_notify(22.f3(26) epoch 194497) v5 -2707> 2014-08-13 18:06:09.694048 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.297789, event: throttled, request: pg_notify(22.f3(26) epoch 194497) v5 -2706> 2014-08-13 18:06:09.694054 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.297913, event: all_read, request: pg_notify(22.f3(26) epoch 194497) v5 -2705> 2014-08-13 18:06:09.694059 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.694030, event: dispatched, request: pg_notify(22.f3(26) epoch 194497) v5 -2704> 2014-08-13 18:06:09.694065 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.694065, event: waiting_for_osdmap, request: pg_notify(22.f3(26) epoch 194497) v5 -2703> 2014-08-13 18:06:09.694071 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 -2702> 2014-08-13 18:06:09.694073 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e3c0 -2701> 2014-08-13 18:06:09.694078 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.694077, event: started, request: pg_notify(22.f3(26) epoch 194497) v5 -2700> 2014-08-13 18:06:09.694097 7f5c835ef700 5 -- op tracker -- , seq: 641, time: 2014-08-13 18:06:09.694097, event: done, request: pg_notify(22.f3(26) epoch 194497) v5 -2699> 2014-08-13 18:06:09.694110 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2698> 2014-08-13 18:06:09.694113 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2697> 2014-08-13 18:06:09.694117 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.375 192.168.1.114:6830/52879 9 ==== pg_info(1 pgs e194497:24.30b) v4 ==== 748+0+0 (2629198101 0 0) 0x5356040 con 0x4c5e5c0 -2696> 2014-08-13 18:06:09.694129 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2695> 2014-08-13 18:06:09.694132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2694> 2014-08-13 18:06:09.694134 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356040 pg_info(1 pgs e194497:24.30b) v4 -2693> 2014-08-13 18:06:09.694138 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.553673, event: header_read, request: pg_info(1 pgs e194497:24.30b) v4 -2692> 2014-08-13 18:06:09.694145 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.553674, event: throttled, request: pg_info(1 pgs e194497:24.30b) v4 -2691> 2014-08-13 18:06:09.694150 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.553759, event: all_read, request: pg_info(1 pgs e194497:24.30b) v4 -2690> 2014-08-13 18:06:09.694156 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.694129, event: dispatched, request: pg_info(1 pgs e194497:24.30b) v4 -2689> 2014-08-13 18:06:09.694162 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.694162, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.30b) v4 -2688> 2014-08-13 18:06:09.694167 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.375 -2687> 2014-08-13 18:06:09.694172 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356040 -2686> 2014-08-13 18:06:09.694168 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2688 -2685> 2014-08-13 18:06:09.694176 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.694176, event: started, request: pg_info(1 pgs e194497:24.30b) v4 -2684> 2014-08-13 18:06:09.694188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2683> 2014-08-13 18:06:09.694205 7f5c835ef700 5 -- op tracker -- , seq: 642, time: 2014-08-13 18:06:09.694205, event: done, request: pg_info(1 pgs e194497:24.30b) v4 -2682> 2014-08-13 18:06:09.694216 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2681> 2014-08-13 18:06:09.694219 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2680> 2014-08-13 18:06:09.694209 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2679> 2014-08-13 18:06:09.694225 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 13 ==== pg_info(1 pgs e194497:24.3f7(2)) v4 ==== 866+0+0 (3856497074 0 0) 0x4df4600 con 0x4f20dc0 -2678> 2014-08-13 18:06:09.694239 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2677> 2014-08-13 18:06:09.694242 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2676> 2014-08-13 18:06:09.694244 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df4600 pg_info(1 pgs e194497:24.3f7(2)) v4 -2675> 2014-08-13 18:06:09.694249 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.201825, event: header_read, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2674> 2014-08-13 18:06:09.694236 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2673> 2014-08-13 18:06:09.694258 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.201828, event: throttled, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2672> 2014-08-13 18:06:09.694263 7f5c7bde3700 10 log is not dirty -2671> 2014-08-13 18:06:09.694264 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.201908, event: all_read, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2670> 2014-08-13 18:06:09.694270 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.694238, event: dispatched, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2669> 2014-08-13 18:06:09.694277 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.694276, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2668> 2014-08-13 18:06:09.694282 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3f7(2)) v4 from osd.330 -2667> 2014-08-13 18:06:09.694287 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df4600 -2666> 2014-08-13 18:06:09.694291 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.694291, event: started, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2665> 2014-08-13 18:06:09.694310 7f5c835ef700 5 -- op tracker -- , seq: 643, time: 2014-08-13 18:06:09.694310, event: done, request: pg_info(1 pgs e194497:24.3f7(2)) v4 -2664> 2014-08-13 18:06:09.694288 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,289,341]/[4,289,341] -- 4/4 -2663> 2014-08-13 18:06:09.694322 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2662> 2014-08-13 18:06:09.694323 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2661> 2014-08-13 18:06:09.694327 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2660> 2014-08-13 18:06:09.694328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -2659> 2014-08-13 18:06:09.694332 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.365 192.168.1.102:6824/15623 11 ==== pg_notify(22.481(42) epoch 194497) v5 ==== 3080+0+0 (2077509107 0 0) 0x5a6cec0 con 0x4fa2260 -2658> 2014-08-13 18:06:09.694023 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 also missing on osd.59 (past last_backfill 0//0//-1) -2657> 2014-08-13 18:06:09.694345 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2656> 2014-08-13 18:06:09.694349 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2655> 2014-08-13 18:06:09.694352 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6cec0 pg_notify(22.481(42) epoch 194497) v5 -2654> 2014-08-13 18:06:09.694358 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.296913, event: header_read, request: pg_notify(22.481(42) epoch 194497) v5 -2653> 2014-08-13 18:06:09.694345 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -2652> 2014-08-13 18:06:09.694365 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.296915, event: throttled, request: pg_notify(22.481(42) epoch 194497) v5 -2651> 2014-08-13 18:06:09.694371 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.297087, event: all_read, request: pg_notify(22.481(42) epoch 194497) v5 -2650> 2014-08-13 18:06:09.694377 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.694345, event: dispatched, request: pg_notify(22.481(42) epoch 194497) v5 -2649> 2014-08-13 18:06:09.694371 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,289) went down -2648> 2014-08-13 18:06:09.694383 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.694383, event: waiting_for_osdmap, request: pg_notify(22.481(42) epoch 194497) v5 -2647> 2014-08-13 18:06:09.694390 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.365 -2646> 2014-08-13 18:06:09.694394 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6cec0 -2645> 2014-08-13 18:06:09.694398 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.694398, event: started, request: pg_notify(22.481(42) epoch 194497) v5 -2644> 2014-08-13 18:06:09.694390 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -2643> 2014-08-13 18:06:09.694419 7f5c835ef700 5 -- op tracker -- , seq: 644, time: 2014-08-13 18:06:09.694419, event: done, request: pg_notify(22.481(42) epoch 194497) v5 -2642> 2014-08-13 18:06:09.694411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -2641> 2014-08-13 18:06:09.694444 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2640> 2014-08-13 18:06:09.694448 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2639> 2014-08-13 18:06:09.694453 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.378 192.168.1.102:6840/19195 6 ==== pg_info(1 pgs e194497:24.9b) v4 ==== 724+0+0 (1807575895 0 0) 0x4a43100 con 0x4fa1ce0 -2638> 2014-08-13 18:06:09.694465 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2637> 2014-08-13 18:06:09.694468 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2636> 2014-08-13 18:06:09.694470 7f5c835ef700 20 osd.4 194497 _dispatch 0x4a43100 pg_info(1 pgs e194497:24.9b) v4 -2635> 2014-08-13 18:06:09.694475 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.521771, event: header_read, request: pg_info(1 pgs e194497:24.9b) v4 -2634> 2014-08-13 18:06:09.694482 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.521773, event: throttled, request: pg_info(1 pgs e194497:24.9b) v4 -2633> 2014-08-13 18:06:09.694487 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.521840, event: all_read, request: pg_info(1 pgs e194497:24.9b) v4 -2632> 2014-08-13 18:06:09.694494 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.694465, event: dispatched, request: pg_info(1 pgs e194497:24.9b) v4 -2631> 2014-08-13 18:06:09.694500 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.694500, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.9b) v4 -2630> 2014-08-13 18:06:09.694506 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.378 -2629> 2014-08-13 18:06:09.694510 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4a43100 -2628> 2014-08-13 18:06:09.694514 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.694514, event: started, request: pg_info(1 pgs e194497:24.9b) v4 -2627> 2014-08-13 18:06:09.694534 7f5c835ef700 5 -- op tracker -- , seq: 645, time: 2014-08-13 18:06:09.694533, event: done, request: pg_info(1 pgs e194497:24.9b) v4 -2626> 2014-08-13 18:06:09.694544 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2625> 2014-08-13 18:06:09.694547 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2624> 2014-08-13 18:06:09.694552 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.26 192.168.1.115:6802/58522 6 ==== pg_info(1 pgs e194497:24.34) v4 ==== 740+0+0 (2553470028 0 0) 0x4df5940 con 0x3a7e9e0 -2623> 2014-08-13 18:06:09.694565 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2622> 2014-08-13 18:06:09.694568 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2621> 2014-08-13 18:06:09.694569 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_info(1 pgs e194497:24.34) v4 -2620> 2014-08-13 18:06:09.694574 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.648739, event: header_read, request: pg_info(1 pgs e194497:24.34) v4 -2619> 2014-08-13 18:06:09.694581 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.648742, event: throttled, request: pg_info(1 pgs e194497:24.34) v4 -2618> 2014-08-13 18:06:09.694586 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.648826, event: all_read, request: pg_info(1 pgs e194497:24.34) v4 -2617> 2014-08-13 18:06:09.694591 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.694564, event: dispatched, request: pg_info(1 pgs e194497:24.34) v4 -2616> 2014-08-13 18:06:09.694597 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.694597, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.34) v4 -2615> 2014-08-13 18:06:09.694603 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.26 -2614> 2014-08-13 18:06:09.694607 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 -2613> 2014-08-13 18:06:09.694622 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.694622, event: started, request: pg_info(1 pgs e194497:24.34) v4 -2612> 2014-08-13 18:06:09.694646 7f5c835ef700 5 -- op tracker -- , seq: 646, time: 2014-08-13 18:06:09.694645, event: done, request: pg_info(1 pgs e194497:24.34) v4 -2611> 2014-08-13 18:06:09.694657 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2610> 2014-08-13 18:06:09.694661 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2609> 2014-08-13 18:06:09.694346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 also missing on osd.59 (past last_backfill 0//0//-1) -2608> 2014-08-13 18:06:09.694666 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.10 192.168.1.115:6815/60573 12 ==== pg_info(1 pgs e194497:24.30b) v4 ==== 748+0+0 (1112630046 0 0) 0x4bf5240 con 0x43f0840 -2607> 2014-08-13 18:06:09.694680 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2606> 2014-08-13 18:06:09.694684 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2605> 2014-08-13 18:06:09.694686 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf5240 pg_info(1 pgs e194497:24.30b) v4 -2604> 2014-08-13 18:06:09.694691 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.554539, event: header_read, request: pg_info(1 pgs e194497:24.30b) v4 -2603> 2014-08-13 18:06:09.694697 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.554541, event: throttled, request: pg_info(1 pgs e194497:24.30b) v4 -2602> 2014-08-13 18:06:09.694703 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.554628, event: all_read, request: pg_info(1 pgs e194497:24.30b) v4 -2601> 2014-08-13 18:06:09.694708 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.694679, event: dispatched, request: pg_info(1 pgs e194497:24.30b) v4 -2600> 2014-08-13 18:06:09.694714 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.694714, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.30b) v4 -2599> 2014-08-13 18:06:09.694720 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.10 -2598> 2014-08-13 18:06:09.694724 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf5240 -2597> 2014-08-13 18:06:09.694728 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.694728, event: started, request: pg_info(1 pgs e194497:24.30b) v4 -2596> 2014-08-13 18:06:09.694746 7f5c835ef700 5 -- op tracker -- , seq: 647, time: 2014-08-13 18:06:09.694746, event: done, request: pg_info(1 pgs e194497:24.30b) v4 -2595> 2014-08-13 18:06:09.694756 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2594> 2014-08-13 18:06:09.694759 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2593> 2014-08-13 18:06:09.694764 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.28 192.168.1.103:6805/37391 9 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4bc3600 con 0x43f35a0 -2592> 2014-08-13 18:06:09.694776 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2591> 2014-08-13 18:06:09.694779 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2590> 2014-08-13 18:06:09.694781 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bc3600 osd_map(194497..194497 src has 185139..194497) v3 -2589> 2014-08-13 18:06:09.694788 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -2588> 2014-08-13 18:06:09.694792 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -2587> 2014-08-13 18:06:09.694795 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2586> 2014-08-13 18:06:09.694797 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2585> 2014-08-13 18:06:09.694802 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.20 192.168.1.103:6815/38269 11 ==== pg_info(1 pgs e194497:24.3d) v4 ==== 748+0+0 (3182156585 0 0) 0x5888fc0 con 0x44261a0 -2584> 2014-08-13 18:06:09.694814 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2583> 2014-08-13 18:06:09.694816 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2582> 2014-08-13 18:06:09.694818 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888fc0 pg_info(1 pgs e194497:24.3d) v4 -2581> 2014-08-13 18:06:09.694823 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.649187, event: header_read, request: pg_info(1 pgs e194497:24.3d) v4 -2580> 2014-08-13 18:06:09.694830 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.649189, event: throttled, request: pg_info(1 pgs e194497:24.3d) v4 -2579> 2014-08-13 18:06:09.694835 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.649256, event: all_read, request: pg_info(1 pgs e194497:24.3d) v4 -2578> 2014-08-13 18:06:09.694841 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.694813, event: dispatched, request: pg_info(1 pgs e194497:24.3d) v4 -2577> 2014-08-13 18:06:09.694847 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.694847, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3d) v4 -2576> 2014-08-13 18:06:09.694852 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.20 -2575> 2014-08-13 18:06:09.694857 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888fc0 -2574> 2014-08-13 18:06:09.694861 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.694861, event: started, request: pg_info(1 pgs e194497:24.3d) v4 -2573> 2014-08-13 18:06:09.694879 7f5c835ef700 5 -- op tracker -- , seq: 648, time: 2014-08-13 18:06:09.694879, event: done, request: pg_info(1 pgs e194497:24.3d) v4 -2572> 2014-08-13 18:06:09.694889 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2571> 2014-08-13 18:06:09.694891 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2570> 2014-08-13 18:06:09.694897 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.291 192.168.1.101:6820/25633 9 ==== pg_info(1 pgs e194497:24.9b) v4 ==== 748+0+0 (2738754650 0 0) 0x588d5c0 con 0x4424e60 -2569> 2014-08-13 18:06:09.694908 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2568> 2014-08-13 18:06:09.694911 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2567> 2014-08-13 18:06:09.694914 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d5c0 pg_info(1 pgs e194497:24.9b) v4 -2566> 2014-08-13 18:06:09.694919 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.575666, event: header_read, request: pg_info(1 pgs e194497:24.9b) v4 -2565> 2014-08-13 18:06:09.694898 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] -2564> 2014-08-13 18:06:09.694925 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.575668, event: throttled, request: pg_info(1 pgs e194497:24.9b) v4 -2563> 2014-08-13 18:06:09.694931 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.575743, event: all_read, request: pg_info(1 pgs e194497:24.9b) v4 -2562> 2014-08-13 18:06:09.694936 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.694908, event: dispatched, request: pg_info(1 pgs e194497:24.9b) v4 -2561> 2014-08-13 18:06:09.694927 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -2560> 2014-08-13 18:06:09.694943 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.694942, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.9b) v4 -2559> 2014-08-13 18:06:09.694949 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.9b) v4 from osd.291 -2558> 2014-08-13 18:06:09.694953 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d5c0 -2557> 2014-08-13 18:06:09.694957 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.694957, event: started, request: pg_info(1 pgs e194497:24.9b) v4 -2556> 2014-08-13 18:06:09.694974 7f5c835ef700 5 -- op tracker -- , seq: 649, time: 2014-08-13 18:06:09.694974, event: done, request: pg_info(1 pgs e194497:24.9b) v4 -2555> 2014-08-13 18:06:09.694983 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2554> 2014-08-13 18:06:09.694668 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 also missing on osd.59 (past last_backfill 0//0//-1) -2553> 2014-08-13 18:06:09.694986 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2552> 2014-08-13 18:06:09.694991 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.30 192.168.1.107:6823/55491 11 ==== pg_info(1 pgs e194497:24.34) v4 ==== 740+0+0 (963828384 0 0) 0x4df3d40 con 0x44586e0 -2551> 2014-08-13 18:06:09.695004 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2550> 2014-08-13 18:06:09.695006 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2549> 2014-08-13 18:06:09.695008 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df3d40 pg_info(1 pgs e194497:24.34) v4 -2548> 2014-08-13 18:06:09.695013 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.650019, event: header_read, request: pg_info(1 pgs e194497:24.34) v4 -2547> 2014-08-13 18:06:09.695019 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.650021, event: throttled, request: pg_info(1 pgs e194497:24.34) v4 -2546> 2014-08-13 18:06:09.695024 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.650085, event: all_read, request: pg_info(1 pgs e194497:24.34) v4 -2545> 2014-08-13 18:06:09.695030 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.695003, event: dispatched, request: pg_info(1 pgs e194497:24.34) v4 -2544> 2014-08-13 18:06:09.695036 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.695036, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.34) v4 -2543> 2014-08-13 18:06:09.695041 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.30 -2542> 2014-08-13 18:06:09.695045 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df3d40 -2541> 2014-08-13 18:06:09.695049 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.695049, event: started, request: pg_info(1 pgs e194497:24.34) v4 -2540> 2014-08-13 18:06:09.695068 7f5c835ef700 5 -- op tracker -- , seq: 650, time: 2014-08-13 18:06:09.695068, event: done, request: pg_info(1 pgs e194497:24.34) v4 -2539> 2014-08-13 18:06:09.695078 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2538> 2014-08-13 18:06:09.695080 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2537> 2014-08-13 18:06:09.695085 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.61 192.168.1.111:6840/16566 15 ==== pg_notify(22.362(1) epoch 194497) v5 ==== 777+0+0 (3329296148 0 0) 0x5888380 con 0x445b180 -2536> 2014-08-13 18:06:09.695098 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2535> 2014-08-13 18:06:09.695101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2534> 2014-08-13 18:06:09.695102 7f5c835ef700 20 osd.4 194497 _dispatch 0x5888380 pg_notify(22.362(1) epoch 194497) v5 -2533> 2014-08-13 18:06:09.695107 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.210954, event: header_read, request: pg_notify(22.362(1) epoch 194497) v5 -2532> 2014-08-13 18:06:09.695114 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.210956, event: throttled, request: pg_notify(22.362(1) epoch 194497) v5 -2531> 2014-08-13 18:06:09.695119 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.211043, event: all_read, request: pg_notify(22.362(1) epoch 194497) v5 -2530> 2014-08-13 18:06:09.695125 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.695098, event: dispatched, request: pg_notify(22.362(1) epoch 194497) v5 -2529> 2014-08-13 18:06:09.695131 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.695131, event: waiting_for_osdmap, request: pg_notify(22.362(1) epoch 194497) v5 -2528> 2014-08-13 18:06:09.695137 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.61 -2527> 2014-08-13 18:06:09.695140 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5888380 -2526> 2014-08-13 18:06:09.695144 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.695144, event: started, request: pg_notify(22.362(1) epoch 194497) v5 -2525> 2014-08-13 18:06:09.695164 7f5c835ef700 5 -- op tracker -- , seq: 651, time: 2014-08-13 18:06:09.695163, event: done, request: pg_notify(22.362(1) epoch 194497) v5 -2524> 2014-08-13 18:06:09.695189 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2523> 2014-08-13 18:06:09.695193 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2522> 2014-08-13 18:06:09.695182 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2694 -2521> 2014-08-13 18:06:09.695198 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.340 192.168.1.104:6806/62463 14 ==== pg_info(1 pgs e194497:24.34) v4 ==== 740+0+0 (288974402 0 0) 0x4df2140 con 0x4499080 -2520> 2014-08-13 18:06:09.695212 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2519> 2014-08-13 18:06:09.695214 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2518> 2014-08-13 18:06:09.695216 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df2140 pg_info(1 pgs e194497:24.34) v4 -2517> 2014-08-13 18:06:09.695206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -2516> 2014-08-13 18:06:09.695222 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.648777, event: header_read, request: pg_info(1 pgs e194497:24.34) v4 -2515> 2014-08-13 18:06:09.695228 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.648779, event: throttled, request: pg_info(1 pgs e194497:24.34) v4 -2514> 2014-08-13 18:06:09.695223 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2513> 2014-08-13 18:06:09.695234 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.648864, event: all_read, request: pg_info(1 pgs e194497:24.34) v4 -2512> 2014-08-13 18:06:09.695240 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.695211, event: dispatched, request: pg_info(1 pgs e194497:24.34) v4 -2511> 2014-08-13 18:06:09.695246 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.695246, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.34) v4 -2510> 2014-08-13 18:06:09.695252 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.34) v4 from osd.340 -2509> 2014-08-13 18:06:09.695256 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df2140 -2508> 2014-08-13 18:06:09.695261 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.695261, event: started, request: pg_info(1 pgs e194497:24.34) v4 -2507> 2014-08-13 18:06:09.695239 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2506> 2014-08-13 18:06:09.695275 7f5c7bde3700 10 log is not dirty -2505> 2014-08-13 18:06:09.695280 7f5c835ef700 5 -- op tracker -- , seq: 652, time: 2014-08-13 18:06:09.695280, event: done, request: pg_info(1 pgs e194497:24.34) v4 -2504> 2014-08-13 18:06:09.695290 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2503> 2014-08-13 18:06:09.695293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2502> 2014-08-13 18:06:09.695298 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.315 192.168.1.113:6800/14161 8 ==== pg_notify(24.b6(44) epoch 194497) v5 ==== 3240+0+0 (2293337751 0 0) 0x5352840 con 0x3c51080 -2501> 2014-08-13 18:06:09.694988 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 also missing on osd.59 (past last_backfill 0//0//-1) -2500> 2014-08-13 18:06:09.695312 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2499> 2014-08-13 18:06:09.695315 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2498> 2014-08-13 18:06:09.695318 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352840 pg_notify(24.b6(44) epoch 194497) v5 -2497> 2014-08-13 18:06:09.695305 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_advance_map [292,345,4]/[292,345,4] -- 292/292 -2496> 2014-08-13 18:06:09.695323 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.504787, event: header_read, request: pg_notify(24.b6(44) epoch 194497) v5 -2495> 2014-08-13 18:06:09.695330 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2494> 2014-08-13 18:06:09.695330 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.504789, event: throttled, request: pg_notify(24.b6(44) epoch 194497) v5 -2493> 2014-08-13 18:06:09.695336 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.504927, event: all_read, request: pg_notify(24.b6(44) epoch 194497) v5 -2492> 2014-08-13 18:06:09.695343 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.695311, event: dispatched, request: pg_notify(24.b6(44) epoch 194497) v5 -2491> 2014-08-13 18:06:09.695349 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.695349, event: waiting_for_osdmap, request: pg_notify(24.b6(44) epoch 194497) v5 -2490> 2014-08-13 18:06:09.695337 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] state: Started advmap -2489> 2014-08-13 18:06:09.695355 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.315 -2488> 2014-08-13 18:06:09.695358 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352840 -2487> 2014-08-13 18:06:09.695362 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.695362, event: started, request: pg_notify(24.b6(44) epoch 194497) v5 -2486> 2014-08-13 18:06:09.695366 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] check_recovery_sources no source osds () went down -2485> 2014-08-13 18:06:09.695381 7f5c835ef700 5 -- op tracker -- , seq: 653, time: 2014-08-13 18:06:09.695381, event: done, request: pg_notify(24.b6(44) epoch 194497) v5 -2484> 2014-08-13 18:06:09.695392 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2483> 2014-08-13 18:06:09.695395 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2482> 2014-08-13 18:06:09.695385 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map -2481> 2014-08-13 18:06:09.695406 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] take_waiters -2480> 2014-08-13 18:06:09.695417 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 17 ==== pg_info(1 pgs e194497:24.3b7(2)) v4 ==== 866+0+0 (3030285740 0 0) 0x524cec0 con 0x3c527e0 -2479> 2014-08-13 18:06:09.695431 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2478> 2014-08-13 18:06:09.695421 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2477> 2014-08-13 18:06:09.695435 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2476> 2014-08-13 18:06:09.695437 7f5c835ef700 20 osd.4 194497 _dispatch 0x524cec0 pg_info(1 pgs e194497:24.3b7(2)) v4 -2475> 2014-08-13 18:06:09.695443 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.202717, event: header_read, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2474> 2014-08-13 18:06:09.695450 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.202721, event: throttled, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2473> 2014-08-13 18:06:09.695456 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.202803, event: all_read, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2472> 2014-08-13 18:06:09.695436 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3ad( v 194215'513 (0'0,194215'513] local-les=194496 n=37 ec=183096 les/c 194496/194496 194495/194495/192972) [292,345,4] r=2 lpr=194495 pi=192975-194494/27 luod=0'0 crt=192888'477 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2471> 2014-08-13 18:06:09.695462 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.695431, event: dispatched, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2470> 2014-08-13 18:06:09.695470 7f5c7bde3700 10 log is not dirty -2469> 2014-08-13 18:06:09.695469 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.695469, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2468> 2014-08-13 18:06:09.695475 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3b7(2)) v4 from osd.356 -2467> 2014-08-13 18:06:09.695479 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524cec0 -2466> 2014-08-13 18:06:09.695484 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.695484, event: started, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2465> 2014-08-13 18:06:09.695503 7f5c835ef700 5 -- op tracker -- , seq: 654, time: 2014-08-13 18:06:09.695503, event: done, request: pg_info(1 pgs e194497:24.3b7(2)) v4 -2464> 2014-08-13 18:06:09.695498 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_advance_map [343,45,4]/[343,45,4] -- 343/343 -2463> 2014-08-13 18:06:09.695520 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2462> 2014-08-13 18:06:09.695524 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2461> 2014-08-13 18:06:09.695527 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2460> 2014-08-13 18:06:09.695528 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.286 192.168.1.113:6820/19173 10 ==== pg_info(1 pgs e194497:24.30b) v4 ==== 724+0+0 (2851252639 0 0) 0x5a68540 con 0x3cf2940 -2459> 2014-08-13 18:06:09.695541 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2458> 2014-08-13 18:06:09.695544 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2457> 2014-08-13 18:06:09.695533 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: Started advmap -2456> 2014-08-13 18:06:09.695547 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a68540 pg_info(1 pgs e194497:24.30b) v4 -2455> 2014-08-13 18:06:09.695552 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.538503, event: header_read, request: pg_info(1 pgs e194497:24.30b) v4 -2454> 2014-08-13 18:06:09.695559 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.538505, event: throttled, request: pg_info(1 pgs e194497:24.30b) v4 -2453> 2014-08-13 18:06:09.695565 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.538571, event: all_read, request: pg_info(1 pgs e194497:24.30b) v4 -2452> 2014-08-13 18:06:09.695570 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.695541, event: dispatched, request: pg_info(1 pgs e194497:24.30b) v4 -2451> 2014-08-13 18:06:09.695577 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.695577, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.30b) v4 -2450> 2014-08-13 18:06:09.695551 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] check_recovery_sources no source osds () went down -2449> 2014-08-13 18:06:09.695583 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.30b) v4 from osd.286 -2448> 2014-08-13 18:06:09.695587 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a68540 -2447> 2014-08-13 18:06:09.695592 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.695592, event: started, request: pg_info(1 pgs e194497:24.30b) v4 -2446> 2014-08-13 18:06:09.695586 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map -2445> 2014-08-13 18:06:09.695612 7f5c835ef700 5 -- op tracker -- , seq: 655, time: 2014-08-13 18:06:09.695612, event: done, request: pg_info(1 pgs e194497:24.30b) v4 -2444> 2014-08-13 18:06:09.695622 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2443> 2014-08-13 18:06:09.695625 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2442> 2014-08-13 18:06:09.695312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 also missing on osd.59 (past last_backfill 0//0//-1) -2441> 2014-08-13 18:06:09.695630 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 19 ==== pg_notify(22.481(42) epoch 194497) v5 ==== 3080+0+0 (4018193186 0 0) 0x5a6db00 con 0x4af8580 -2440> 2014-08-13 18:06:09.695647 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2439> 2014-08-13 18:06:09.695649 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2438> 2014-08-13 18:06:09.695651 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6db00 pg_notify(22.481(42) epoch 194497) v5 -2437> 2014-08-13 18:06:09.695656 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.295540, event: header_read, request: pg_notify(22.481(42) epoch 194497) v5 -2436> 2014-08-13 18:06:09.695663 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.295541, event: throttled, request: pg_notify(22.481(42) epoch 194497) v5 -2435> 2014-08-13 18:06:09.695669 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.295673, event: all_read, request: pg_notify(22.481(42) epoch 194497) v5 -2434> 2014-08-13 18:06:09.695674 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.695646, event: dispatched, request: pg_notify(22.481(42) epoch 194497) v5 -2433> 2014-08-13 18:06:09.695681 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.695680, event: waiting_for_osdmap, request: pg_notify(22.481(42) epoch 194497) v5 -2432> 2014-08-13 18:06:09.695686 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 -2431> 2014-08-13 18:06:09.695689 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6db00 -2430> 2014-08-13 18:06:09.695694 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.695693, event: started, request: pg_notify(22.481(42) epoch 194497) v5 -2429> 2014-08-13 18:06:09.695713 7f5c835ef700 5 -- op tracker -- , seq: 656, time: 2014-08-13 18:06:09.695713, event: done, request: pg_notify(22.481(42) epoch 194497) v5 -2428> 2014-08-13 18:06:09.695701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] take_waiters -2427> 2014-08-13 18:06:09.695730 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2426> 2014-08-13 18:06:09.695733 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2425> 2014-08-13 18:06:09.695726 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -2424> 2014-08-13 18:06:09.695738 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.367 192.168.1.114:6840/55513 12 ==== osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 ==== 1145+0+0 (3280706235 0 0) 0x536ee00 con 0x4b11ce0 -2423> 2014-08-13 18:06:09.695743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -2422> 2014-08-13 18:06:09.695755 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2421> 2014-08-13 18:06:09.695766 7f5c7bde3700 10 log is not dirty -2420> 2014-08-13 18:06:09.695768 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2419> 2014-08-13 18:06:09.695770 7f5c835ef700 20 osd.4 194497 _dispatch 0x536ee00 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2418> 2014-08-13 18:06:09.695779 7f5c835ef700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.648201, event: header_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2417> 2014-08-13 18:06:09.695789 7f5c835ef700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.648203, event: throttled, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2416> 2014-08-13 18:06:09.695798 7f5c835ef700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.648315, event: all_read, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2415> 2014-08-13 18:06:09.695807 7f5c835ef700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.695755, event: dispatched, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2414> 2014-08-13 18:06:09.695818 7f5c835ef700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.695818, event: waiting_for_osdmap, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2413> 2014-08-13 18:06:09.695808 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_advance_map [4,348,331]/[4,332,337] -- 4/4 -2412> 2014-08-13 18:06:09.695828 7f5c835ef700 10 osd.4 194497 handle_replica_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 epoch 194497 -2411> 2014-08-13 18:06:09.695836 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x536ee00 -2410> 2014-08-13 18:06:09.695840 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -2409> 2014-08-13 18:06:09.695840 7f5c835ef700 20 osd.4 194497 _share_map_incoming osd.367 192.168.1.114:6840/55513 194497 -2408> 2014-08-13 18:06:09.695849 7f5c835ef700 15 osd.4 194497 enqueue_op 0x20a7e90 prio 127 cost 0 latency 0.047648 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -2407> 2014-08-13 18:06:09.695848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Peering advmap -2406> 2014-08-13 18:06:09.695868 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2405> 2014-08-13 18:06:09.695872 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2404> 2014-08-13 18:06:09.695877 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.397 192.168.1.105:6808/41085 15 ==== pg_info(1 pgs e194497:22.3b9) v4 ==== 714+0+0 (1234108852 0 0) 0x5a6bf00 con 0x4b10b00 -2403> 2014-08-13 18:06:09.695872 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] adjust_need_up_thru now 194496, need_up_thru now false -2402> 2014-08-13 18:06:09.695890 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2401> 2014-08-13 18:06:09.695893 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2400> 2014-08-13 18:06:09.695896 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6bf00 pg_info(1 pgs e194497:22.3b9) v4 -2399> 2014-08-13 18:06:09.695901 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.488504, event: header_read, request: pg_info(1 pgs e194497:22.3b9) v4 -2398> 2014-08-13 18:06:09.695907 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.488508, event: throttled, request: pg_info(1 pgs e194497:22.3b9) v4 -2397> 2014-08-13 18:06:09.695913 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.488616, event: all_read, request: pg_info(1 pgs e194497:22.3b9) v4 -2396> 2014-08-13 18:06:09.695918 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.695889, event: dispatched, request: pg_info(1 pgs e194497:22.3b9) v4 -2395> 2014-08-13 18:06:09.695891 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Started advmap -2394> 2014-08-13 18:06:09.695925 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.695925, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:22.3b9) v4 -2393> 2014-08-13 18:06:09.695931 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:22.3b9) v4 from osd.397 -2392> 2014-08-13 18:06:09.695936 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6bf00 -2391> 2014-08-13 18:06:09.695940 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.695940, event: started, request: pg_info(1 pgs e194497:22.3b9) v4 -2390> 2014-08-13 18:06:09.695929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] check_recovery_sources no source osds () went down -2389> 2014-08-13 18:06:09.695633 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 also missing on osd.59 (past last_backfill 0//0//-1) -2388> 2014-08-13 18:06:09.695961 7f5c835ef700 5 -- op tracker -- , seq: 658, time: 2014-08-13 18:06:09.695961, event: done, request: pg_info(1 pgs e194497:22.3b9) v4 -2387> 2014-08-13 18:06:09.695953 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] handle_activate_map -2386> 2014-08-13 18:06:09.695973 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2385> 2014-08-13 18:06:09.695977 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2384> 2014-08-13 18:06:09.695975 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: handle ActMap primary -2383> 2014-08-13 18:06:09.695982 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.25 192.168.1.103:6800/36920 6 ==== pg_info(1 pgs e194497:24.3d) v4 ==== 724+0+0 (2559288265 0 0) 0x5351c00 con 0x4c58160 -2382> 2014-08-13 18:06:09.695995 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2381> 2014-08-13 18:06:09.695997 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2380> 2014-08-13 18:06:09.695999 7f5c835ef700 20 osd.4 194497 _dispatch 0x5351c00 pg_info(1 pgs e194497:24.3d) v4 -2379> 2014-08-13 18:06:09.696005 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.604760, event: header_read, request: pg_info(1 pgs e194497:24.3d) v4 -2378> 2014-08-13 18:06:09.696011 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.604761, event: throttled, request: pg_info(1 pgs e194497:24.3d) v4 -2377> 2014-08-13 18:06:09.695994 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] publish_stats_to_osd 194497:22155 -2376> 2014-08-13 18:06:09.696018 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.604827, event: all_read, request: pg_info(1 pgs e194497:24.3d) v4 -2375> 2014-08-13 18:06:09.696024 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.695994, event: dispatched, request: pg_info(1 pgs e194497:24.3d) v4 -2374> 2014-08-13 18:06:09.696030 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.696030, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3d) v4 -2373> 2014-08-13 18:06:09.696035 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.25 -2372> 2014-08-13 18:06:09.696039 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5351c00 -2371> 2014-08-13 18:06:09.696044 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.696043, event: started, request: pg_info(1 pgs e194497:24.3d) v4 -2370> 2014-08-13 18:06:09.696021 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] take_waiters -2369> 2014-08-13 18:06:09.696058 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering/WaitUpThru 1.336094 3 0.000640 -2368> 2014-08-13 18:06:09.696082 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] state: Leaving Peering -2367> 2014-08-13 18:06:09.696110 7f5c835ef700 5 -- op tracker -- , seq: 659, time: 2014-08-13 18:06:09.696109, event: done, request: pg_info(1 pgs e194497:24.3d) v4 -2366> 2014-08-13 18:06:09.696104 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped+peering m=48] exit Started/Primary/Peering 1.387143 0 0.000000 -2365> 2014-08-13 18:06:09.696121 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2364> 2014-08-13 18:06:09.696124 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2363> 2014-08-13 18:06:09.696133 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.369 192.168.1.108:6824/4037 15 ==== pg_notify(22.775(1) epoch 194497) v5 ==== 777+0+0 (524519124 0 0) 0x5a6e200 con 0x4c5e880 -2362> 2014-08-13 18:06:09.696147 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2361> 2014-08-13 18:06:09.696149 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2360> 2014-08-13 18:06:09.696151 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e200 pg_notify(22.775(1) epoch 194497) v5 -2359> 2014-08-13 18:06:09.696156 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.298991, event: header_read, request: pg_notify(22.775(1) epoch 194497) v5 -2358> 2014-08-13 18:06:09.696163 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.298993, event: throttled, request: pg_notify(22.775(1) epoch 194497) v5 -2357> 2014-08-13 18:06:09.696130 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active -2356> 2014-08-13 18:06:09.696169 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.299069, event: all_read, request: pg_notify(22.775(1) epoch 194497) v5 -2355> 2014-08-13 18:06:09.696175 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.696146, event: dispatched, request: pg_notify(22.775(1) epoch 194497) v5 -2354> 2014-08-13 18:06:09.696190 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.696189, event: waiting_for_osdmap, request: pg_notify(22.775(1) epoch 194497) v5 -2353> 2014-08-13 18:06:09.696175 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194487 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: In Active, about to call activate -2352> 2014-08-13 18:06:09.696196 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.369 -2351> 2014-08-13 18:06:09.696199 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e200 -2350> 2014-08-13 18:06:09.696204 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.696203, event: started, request: pg_notify(22.775(1) epoch 194497) v5 -2349> 2014-08-13 18:06:09.696203 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - purged_snaps [] cached_removed_snaps [] -2348> 2014-08-13 18:06:09.696226 7f5c835ef700 5 -- op tracker -- , seq: 660, time: 2014-08-13 18:06:09.696225, event: done, request: pg_notify(22.775(1) epoch 194497) v5 -2347> 2014-08-13 18:06:09.696222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - snap_trimq [] -2346> 2014-08-13 18:06:09.696236 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2345> 2014-08-13 18:06:09.696240 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2344> 2014-08-13 18:06:09.696246 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 14 ==== pg_info(1 pgs e194497:24.b6) v4 ==== 724+0+0 (2967772655 0 0) 0x5a6f380 con 0x4f20dc0 -2343> 2014-08-13 18:06:09.696258 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2342> 2014-08-13 18:06:09.696261 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2341> 2014-08-13 18:06:09.696263 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f380 pg_info(1 pgs e194497:24.b6) v4 -2340> 2014-08-13 18:06:09.696239 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - not complete, missing(48) -2339> 2014-08-13 18:06:09.696268 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.606891, event: header_read, request: pg_info(1 pgs e194497:24.b6) v4 -2338> 2014-08-13 18:06:09.695951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 also missing on osd.59 (past last_backfill 0//0//-1) -2337> 2014-08-13 18:06:09.696278 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.606894, event: throttled, request: pg_info(1 pgs e194497:24.b6) v4 -2336> 2014-08-13 18:06:09.696285 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.606965, event: all_read, request: pg_info(1 pgs e194497:24.b6) v4 -2335> 2014-08-13 18:06:09.696290 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.696258, event: dispatched, request: pg_info(1 pgs e194497:24.b6) v4 -2334> 2014-08-13 18:06:09.696296 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.696296, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.b6) v4 -2333> 2014-08-13 18:06:09.696302 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.330 -2332> 2014-08-13 18:06:09.696306 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f380 -2331> 2014-08-13 18:06:09.696310 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.696310, event: started, request: pg_info(1 pgs e194497:24.b6) v4 -2330> 2014-08-13 18:06:09.696329 7f5c835ef700 5 -- op tracker -- , seq: 661, time: 2014-08-13 18:06:09.696329, event: done, request: pg_info(1 pgs e194497:24.b6) v4 -2329> 2014-08-13 18:06:09.696339 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2328> 2014-08-13 18:06:09.696342 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2327> 2014-08-13 18:06:09.696347 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.28 192.168.1.103:6805/37391 10 ==== pg_info(1 pgs e194497:24.b6) v4 ==== 724+0+0 (2967772655 0 0) 0x4bf1dc0 con 0x43f35a0 -2326> 2014-08-13 18:06:09.696359 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2325> 2014-08-13 18:06:09.696361 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2324> 2014-08-13 18:06:09.696363 7f5c835ef700 20 osd.4 194497 _dispatch 0x4bf1dc0 pg_info(1 pgs e194497:24.b6) v4 -2323> 2014-08-13 18:06:09.696368 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.606302, event: header_read, request: pg_info(1 pgs e194497:24.b6) v4 -2322> 2014-08-13 18:06:09.696374 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.606303, event: throttled, request: pg_info(1 pgs e194497:24.b6) v4 -2321> 2014-08-13 18:06:09.696379 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.606366, event: all_read, request: pg_info(1 pgs e194497:24.b6) v4 -2320> 2014-08-13 18:06:09.696385 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.696358, event: dispatched, request: pg_info(1 pgs e194497:24.b6) v4 -2319> 2014-08-13 18:06:09.696391 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.696390, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.b6) v4 -2318> 2014-08-13 18:06:09.696396 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.28 -2317> 2014-08-13 18:06:09.696400 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4bf1dc0 -2316> 2014-08-13 18:06:09.696404 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.696404, event: started, request: pg_info(1 pgs e194497:24.b6) v4 -2315> 2014-08-13 18:06:09.696422 7f5c835ef700 5 -- op tracker -- , seq: 662, time: 2014-08-13 18:06:09.696422, event: done, request: pg_info(1 pgs e194497:24.b6) v4 -2314> 2014-08-13 18:06:09.696432 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2313> 2014-08-13 18:06:09.696435 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2312> 2014-08-13 18:06:09.696442 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 20 ==== pg_notify(22.3e4(1) epoch 194497) v5 ==== 777+0+0 (2635583822 0 0) 0x5a6d940 con 0x4af8580 -2311> 2014-08-13 18:06:09.696454 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2310> 2014-08-13 18:06:09.696457 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2309> 2014-08-13 18:06:09.696458 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d940 pg_notify(22.3e4(1) epoch 194497) v5 -2308> 2014-08-13 18:06:09.696463 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.296395, event: header_read, request: pg_notify(22.3e4(1) epoch 194497) v5 -2307> 2014-08-13 18:06:09.696470 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.296396, event: throttled, request: pg_notify(22.3e4(1) epoch 194497) v5 -2306> 2014-08-13 18:06:09.696477 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.296457, event: all_read, request: pg_notify(22.3e4(1) epoch 194497) v5 -2305> 2014-08-13 18:06:09.696483 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.696453, event: dispatched, request: pg_notify(22.3e4(1) epoch 194497) v5 -2304> 2014-08-13 18:06:09.696489 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.696488, event: waiting_for_osdmap, request: pg_notify(22.3e4(1) epoch 194497) v5 -2303> 2014-08-13 18:06:09.696494 7f5c835ef700 7 osd.4 194497 handle_pg_notify from osd.303 -2302> 2014-08-13 18:06:09.696497 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d940 -2301> 2014-08-13 18:06:09.696501 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.696501, event: started, request: pg_notify(22.3e4(1) epoch 194497) v5 -2300> 2014-08-13 18:06:09.696521 7f5c835ef700 5 -- op tracker -- , seq: 663, time: 2014-08-13 18:06:09.696521, event: done, request: pg_notify(22.3e4(1) epoch 194497) v5 -2299> 2014-08-13 18:06:09.696549 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2298> 2014-08-13 18:06:09.696553 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2297> 2014-08-13 18:06:09.696560 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.303 192.168.1.109:6802/30768 21 ==== pg_info(1 pgs e194497:24.3d) v4 ==== 748+0+0 (835867987 0 0) 0x4df0a80 con 0x4af8580 -2296> 2014-08-13 18:06:09.696574 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2295> 2014-08-13 18:06:09.696577 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2294> 2014-08-13 18:06:09.696579 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df0a80 pg_info(1 pgs e194497:24.3d) v4 -2293> 2014-08-13 18:06:09.696584 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.650969, event: header_read, request: pg_info(1 pgs e194497:24.3d) v4 -2292> 2014-08-13 18:06:09.696590 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.650971, event: throttled, request: pg_info(1 pgs e194497:24.3d) v4 -2291> 2014-08-13 18:06:09.696279 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 also missing on osd.59 (past last_backfill 0//0//-1) -2290> 2014-08-13 18:06:09.696596 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.651039, event: all_read, request: pg_info(1 pgs e194497:24.3d) v4 -2289> 2014-08-13 18:06:09.696602 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.696573, event: dispatched, request: pg_info(1 pgs e194497:24.3d) v4 -2288> 2014-08-13 18:06:09.696608 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.696608, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3d) v4 -2287> 2014-08-13 18:06:09.696614 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3d) v4 from osd.303 -2286> 2014-08-13 18:06:09.696618 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df0a80 -2285> 2014-08-13 18:06:09.696623 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.696622, event: started, request: pg_info(1 pgs e194497:24.3d) v4 -2284> 2014-08-13 18:06:09.696641 7f5c835ef700 5 -- op tracker -- , seq: 664, time: 2014-08-13 18:06:09.696641, event: done, request: pg_info(1 pgs e194497:24.3d) v4 -2283> 2014-08-13 18:06:09.696650 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2282> 2014-08-13 18:06:09.696653 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2281> 2014-08-13 18:06:09.696599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 also missing on osd.59 (past last_backfill 0//0//-1) -2280> 2014-08-13 18:06:09.696917 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 also missing on osd.59 (past last_backfill 0//0//-1) -2279> 2014-08-13 18:06:09.697239 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 also missing on osd.59 (past last_backfill 0//0//-1) -2278> 2014-08-13 18:06:09.697557 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 also missing on osd.59 (past last_backfill 0//0//-1) -2277> 2014-08-13 18:06:09.698111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate - starting recovery -2276> 2014-08-13 18:06:09.698153 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] -2275> 2014-08-13 18:06:09.697875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 also missing on osd.59 (past last_backfill 0//0//-1) -2274> 2014-08-13 18:06:09.698217 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -2273> 2014-08-13 18:06:09.698253 7f5c7bde3700 0 log [INF] : 24.1bb restarting backfill on osd.331 from (0'0,0'0] MAX to 194422'12517 -2272> 2014-08-13 18:06:09.698197 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 also missing on osd.59 (past last_backfill 0//0//-1) -2271> 2014-08-13 18:06:09.698697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 sending log((192718'9695,194422'12517], crt=194422'12517) -2270> 2014-08-13 18:06:09.698731 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing -2269> 2014-08-13 18:06:09.698735 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.331 192.168.1.104:6816/837 -- pg_log(24.1bb epoch 194497 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194497) v4 -- ?+0 0x4166600 -2268> 2014-08-13 18:06:09.698782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.331 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -2267> 2014-08-13 18:06:09.698515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 also missing on osd.59 (past last_backfill 0//0//-1) -2266> 2014-08-13 18:06:09.698809 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -2265> 2014-08-13 18:06:09.698838 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 is up to date, queueing in pending_activators -2264> 2014-08-13 18:06:09.698930 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.332 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -2263> 2014-08-13 18:06:09.698956 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) -2262> 2014-08-13 18:06:09.698982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 is up to date, queueing in pending_activators -2261> 2014-08-13 18:06:09.699050 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.337 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -2260> 2014-08-13 18:06:09.699079 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -2259> 2014-08-13 18:06:09.699431 7f5c7bde3700 0 log [INF] : 24.1bb restarting backfill on osd.348 from (0'0,0'0] MAX to 194422'12517 -2258> 2014-08-13 18:06:09.698832 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 also missing on osd.59 (past last_backfill 0//0//-1) -2257> 2014-08-13 18:06:09.699442 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 also missing on osd.59 (past last_backfill 0//0//-1) -2256> 2014-08-13 18:06:09.699644 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 sending log((192718'9695,194422'12517], crt=194422'12517) -2255> 2014-08-13 18:06:09.699667 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x571b700 don't know epoch, doing nothing -2254> 2014-08-13 18:06:09.699669 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.348 192.168.1.110:6800/41204 -- pg_log(24.1bb epoch 194497 log log((192718'9695,194422'12517], crt=194422'12517) query_epoch 194497) v4 -- ?+0 0x4165a00 -2253> 2014-08-13 18:06:09.699687 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate peer osd.348 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194493 194495/194496/194495) uptodate -2252> 2014-08-13 18:06:09.699732 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery primary has 48 missing -2251> 2014-08-13 18:06:09.699626 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 also missing on osd.59 (past last_backfill 0//0//-1) -2250> 2014-08-13 18:06:09.699800 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.4 -2249> 2014-08-13 18:06:09.699876 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.4 -2248> 2014-08-13 18:06:09.699808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 also missing on osd.59 (past last_backfill 0//0//-1) -2247> 2014-08-13 18:06:09.699947 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.4 -2246> 2014-08-13 18:06:09.700012 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.4 -2245> 2014-08-13 18:06:09.700071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.4 -2244> 2014-08-13 18:06:09.699991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 also missing on osd.59 (past last_backfill 0//0//-1) -2243> 2014-08-13 18:06:09.700136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.4 -2242> 2014-08-13 18:06:09.700202 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.4 -2241> 2014-08-13 18:06:09.700267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.4 -2240> 2014-08-13 18:06:09.700175 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 also missing on osd.59 (past last_backfill 0//0//-1) -2239> 2014-08-13 18:06:09.700331 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.4 -2238> 2014-08-13 18:06:09.700393 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.4 -2237> 2014-08-13 18:06:09.700466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.4 -2236> 2014-08-13 18:06:09.700360 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 also missing on osd.59 (past last_backfill 0//0//-1) -2235> 2014-08-13 18:06:09.700532 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.4 -2234> 2014-08-13 18:06:09.700607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.4 -2233> 2014-08-13 18:06:09.700542 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 also missing on osd.59 (past last_backfill 0//0//-1) -2232> 2014-08-13 18:06:09.700678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.4 -2231> 2014-08-13 18:06:09.700748 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.4 -2230> 2014-08-13 18:06:09.700810 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.4 -2229> 2014-08-13 18:06:09.700727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 also missing on osd.59 (past last_backfill 0//0//-1) -2228> 2014-08-13 18:06:09.700873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.4 -2227> 2014-08-13 18:06:09.700933 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.4 -2226> 2014-08-13 18:06:09.700994 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.4 -2225> 2014-08-13 18:06:09.700910 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 also missing on osd.59 (past last_backfill 0//0//-1) -2224> 2014-08-13 18:06:09.701058 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.4 -2223> 2014-08-13 18:06:09.701119 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.4 -2222> 2014-08-13 18:06:09.701094 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 also missing on osd.59 (past last_backfill 0//0//-1) -2221> 2014-08-13 18:06:09.701231 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.4 -2220> 2014-08-13 18:06:09.701328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.4 -2219> 2014-08-13 18:06:09.701281 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 also missing on osd.59 (past last_backfill 0//0//-1) -2218> 2014-08-13 18:06:09.701403 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.4 -2217> 2014-08-13 18:06:09.701496 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.4 -2216> 2014-08-13 18:06:09.701578 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.4 -2215> 2014-08-13 18:06:09.701622 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.9 192.168.100.107:0/53997 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.701657) v2 ==== 47+0+0 (3270073331 0 0) 0x524ee40 con 0x4f20840 -2214> 2014-08-13 18:06:09.701465 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 also missing on osd.59 (past last_backfill 0//0//-1) -2213> 2014-08-13 18:06:09.701626 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.9 192.168.100.107:0/53997 1 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.701657) v2 ==== 47+0+0 (3270073331 0 0) 0x524d240 con 0x4c5f220 -2212> 2014-08-13 18:06:09.701648 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194497 stamp 2014-08-13 18:06:09.701657) v2 -- ?+0 0x588fc40 con 0x4f20840 -2211> 2014-08-13 18:06:09.701663 7f5c817ec700 10 osd.4 194497 note_peer_epoch osd.9 now has 194497 -2210> 2014-08-13 18:06:09.701669 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4423700 already has epoch 194497 -2209> 2014-08-13 18:06:09.701681 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.100.107:0/53997 -- osd_ping(ping_reply e194497 stamp 2014-08-13 18:06:09.701657) v2 -- ?+0 0x588af40 con 0x4c5f220 -2208> 2014-08-13 18:06:09.701702 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4423700 already has epoch 194497 -2207> 2014-08-13 18:06:09.701640 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.4 -2206> 2014-08-13 18:06:09.701647 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 also missing on osd.59 (past last_backfill 0//0//-1) -2205> 2014-08-13 18:06:09.701796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.4 -2204> 2014-08-13 18:06:09.701934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.4 -2203> 2014-08-13 18:06:09.701832 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 also missing on osd.59 (past last_backfill 0//0//-1) -2202> 2014-08-13 18:06:09.701999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.4 -2201> 2014-08-13 18:06:09.702060 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.4 -2200> 2014-08-13 18:06:09.702120 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.4 -2199> 2014-08-13 18:06:09.702014 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 also missing on osd.59 (past last_backfill 0//0//-1) -2198> 2014-08-13 18:06:09.702191 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.4 -2197> 2014-08-13 18:06:09.702256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.4 -2196> 2014-08-13 18:06:09.702319 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.4 -2195> 2014-08-13 18:06:09.702198 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 also missing on osd.59 (past last_backfill 0//0//-1) -2194> 2014-08-13 18:06:09.702379 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.4 -2193> 2014-08-13 18:06:09.702448 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.4 -2192> 2014-08-13 18:06:09.702381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 also missing on osd.59 (past last_backfill 0//0//-1) -2191> 2014-08-13 18:06:09.702518 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.4 -2190> 2014-08-13 18:06:09.702577 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.4 -2189> 2014-08-13 18:06:09.702637 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.4 -2188> 2014-08-13 18:06:09.702565 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 also missing on osd.59 (past last_backfill 0//0//-1) -2187> 2014-08-13 18:06:09.702697 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.4 -2186> 2014-08-13 18:06:09.702756 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.4 -2185> 2014-08-13 18:06:09.702834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.4 -2184> 2014-08-13 18:06:09.702748 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 also missing on osd.59 (past last_backfill 0//0//-1) -2183> 2014-08-13 18:06:09.702895 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.4 -2182> 2014-08-13 18:06:09.702955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.4 -2181> 2014-08-13 18:06:09.703015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.4 -2180> 2014-08-13 18:06:09.702932 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 also missing on osd.59 (past last_backfill 0//0//-1) -2179> 2014-08-13 18:06:09.703076 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.4 -2178> 2014-08-13 18:06:09.703136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.4 -2177> 2014-08-13 18:06:09.703115 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 also missing on osd.59 (past last_backfill 0//0//-1) -2176> 2014-08-13 18:06:09.703221 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -2175> 2014-08-13 18:06:09.703323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 331 as a source -2174> 2014-08-13 18:06:09.703395 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.331 (past last_backfill 0//0//-1) -2173> 2014-08-13 18:06:09.703463 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.331 (past last_backfill 0//0//-1) -2172> 2014-08-13 18:06:09.703319 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -2171> 2014-08-13 18:06:09.703523 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.331 (past last_backfill 0//0//-1) -2170> 2014-08-13 18:06:09.703583 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.331 (past last_backfill 0//0//-1) -2169> 2014-08-13 18:06:09.703642 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.331 (past last_backfill 0//0//-1) -2168> 2014-08-13 18:06:09.703548 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] activate: adding 351 as a source -2167> 2014-08-13 18:06:09.703701 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.331 (past last_backfill 0//0//-1) -2166> 2014-08-13 18:06:09.703760 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.331 (past last_backfill 0//0//-1) -2165> 2014-08-13 18:06:09.703819 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.331 (past last_backfill 0//0//-1) -2164> 2014-08-13 18:06:09.703732 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24 194416'15395 is on osd.351 -2163> 2014-08-13 18:06:09.703879 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.331 (past last_backfill 0//0//-1) -2162> 2014-08-13 18:06:09.703938 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.331 (past last_backfill 0//0//-1) -2161> 2014-08-13 18:06:09.703997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.331 (past last_backfill 0//0//-1) -2160> 2014-08-13 18:06:09.703917 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24 194416'15402 is on osd.351 -2159> 2014-08-13 18:06:09.704056 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.331 (past last_backfill 0//0//-1) -2158> 2014-08-13 18:06:09.704114 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.331 (past last_backfill 0//0//-1) -2157> 2014-08-13 18:06:09.704178 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.331 (past last_backfill 0//0//-1) -2156> 2014-08-13 18:06:09.704101 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24 194377'15364 is on osd.351 -2155> 2014-08-13 18:06:09.704285 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.331 (past last_backfill 0//0//-1) -2154> 2014-08-13 18:06:09.704346 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.331 (past last_backfill 0//0//-1) -2153> 2014-08-13 18:06:09.704287 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24 194416'15422 is on osd.351 -2152> 2014-08-13 18:06:09.704411 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.331 (past last_backfill 0//0//-1) -2151> 2014-08-13 18:06:09.704482 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.331 (past last_backfill 0//0//-1) -2150> 2014-08-13 18:06:09.704554 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.331 (past last_backfill 0//0//-1) -2149> 2014-08-13 18:06:09.704472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24 194370'15330 is on osd.351 -2148> 2014-08-13 18:06:09.704627 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.331 (past last_backfill 0//0//-1) -2147> 2014-08-13 18:06:09.704687 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.331 (past last_backfill 0//0//-1) -2146> 2014-08-13 18:06:09.704746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.331 (past last_backfill 0//0//-1) -2145> 2014-08-13 18:06:09.704656 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24 194370'15356 is on osd.351 -2144> 2014-08-13 18:06:09.704813 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.331 (past last_backfill 0//0//-1) -2143> 2014-08-13 18:06:09.704873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.331 (past last_backfill 0//0//-1) -2142> 2014-08-13 18:06:09.704935 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.331 (past last_backfill 0//0//-1) -2141> 2014-08-13 18:06:09.704840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24 194416'15412 is on osd.351 -2140> 2014-08-13 18:06:09.704994 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.331 (past last_backfill 0//0//-1) -2139> 2014-08-13 18:06:09.705068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.331 (past last_backfill 0//0//-1) -2138> 2014-08-13 18:06:09.705128 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.331 (past last_backfill 0//0//-1) -2137> 2014-08-13 18:06:09.705024 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24 194370'15348 is on osd.351 -2136> 2014-08-13 18:06:09.705193 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.331 (past last_backfill 0//0//-1) -2135> 2014-08-13 18:06:09.705255 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.331 (past last_backfill 0//0//-1) -2134> 2014-08-13 18:06:09.705324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.331 (past last_backfill 0//0//-1) -2133> 2014-08-13 18:06:09.705210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24 194377'15368 is on osd.351 -2132> 2014-08-13 18:06:09.705384 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.331 (past last_backfill 0//0//-1) -2131> 2014-08-13 18:06:09.705443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.331 (past last_backfill 0//0//-1) -2130> 2014-08-13 18:06:09.705502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.331 (past last_backfill 0//0//-1) -2129> 2014-08-13 18:06:09.705393 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24 194416'15430 is on osd.351 -2128> 2014-08-13 18:06:09.705562 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.331 (past last_backfill 0//0//-1) -2127> 2014-08-13 18:06:09.705621 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.331 (past last_backfill 0//0//-1) -2126> 2014-08-13 18:06:09.705681 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.331 (past last_backfill 0//0//-1) -2125> 2014-08-13 18:06:09.705575 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24 194422'15438 is on osd.351 -2124> 2014-08-13 18:06:09.705740 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.331 (past last_backfill 0//0//-1) -2123> 2014-08-13 18:06:09.705799 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.331 (past last_backfill 0//0//-1) -2122> 2014-08-13 18:06:09.705858 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.331 (past last_backfill 0//0//-1) -2121> 2014-08-13 18:06:09.705757 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24 194422'15452 is on osd.351 -2120> 2014-08-13 18:06:09.705917 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.331 (past last_backfill 0//0//-1) -2119> 2014-08-13 18:06:09.705977 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.331 (past last_backfill 0//0//-1) -2118> 2014-08-13 18:06:09.706041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.331 (past last_backfill 0//0//-1) -2117> 2014-08-13 18:06:09.705941 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24 194393'15380 is on osd.351 -2116> 2014-08-13 18:06:09.706101 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.331 (past last_backfill 0//0//-1) -2115> 2014-08-13 18:06:09.706164 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.331 (past last_backfill 0//0//-1) -2114> 2014-08-13 18:06:09.706224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.331 (past last_backfill 0//0//-1) -2113> 2014-08-13 18:06:09.706125 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24 194370'15336 is on osd.351 -2112> 2014-08-13 18:06:09.706303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.331 (past last_backfill 0//0//-1) -2111> 2014-08-13 18:06:09.706369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.331 (past last_backfill 0//0//-1) -2110> 2014-08-13 18:06:09.706310 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24 194416'15406 is on osd.351 -2109> 2014-08-13 18:06:09.706447 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -2108> 2014-08-13 18:06:09.706544 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] activate: adding 332 as a source -2107> 2014-08-13 18:06:09.706495 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24 194370'15334 is on osd.351 -2106> 2014-08-13 18:06:09.706620 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.332 -2105> 2014-08-13 18:06:09.706680 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=47] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.332 -2104> 2014-08-13 18:06:09.706745 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=46] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.332 -2103> 2014-08-13 18:06:09.706679 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24 194377'15370 is on osd.351 -2102> 2014-08-13 18:06:09.706810 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=45] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.332 -2101> 2014-08-13 18:06:09.706875 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=44] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.332 -2100> 2014-08-13 18:06:09.706944 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=43] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.332 -2099> 2014-08-13 18:06:09.706863 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24 194422'15442 is on osd.351 -2098> 2014-08-13 18:06:09.707017 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=42] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.332 -2097> 2014-08-13 18:06:09.707089 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=41] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.332 -2096> 2014-08-13 18:06:09.707047 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24 194393'15386 is on osd.351 -2095> 2014-08-13 18:06:09.707168 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=40] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.332 -2094> 2014-08-13 18:06:09.707242 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=39] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.332 -2093> 2014-08-13 18:06:09.707321 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=38] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.332 -2092> 2014-08-13 18:06:09.707234 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24 194416'15418 is on osd.351 -2091> 2014-08-13 18:06:09.707409 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=37] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.332 -2090> 2014-08-13 18:06:09.707508 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=36] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.332 -2089> 2014-08-13 18:06:09.707422 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24 194377'15366 is on osd.351 -2088> 2014-08-13 18:06:09.707591 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=35] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.332 -2087> 2014-08-13 18:06:09.707604 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24 194422'15444 is on osd.351 -2086> 2014-08-13 18:06:09.707699 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=34] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.332 -2085> 2014-08-13 18:06:09.707796 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=33] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.332 -2084> 2014-08-13 18:06:09.707787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24 194370'15338 is on osd.351 -2083> 2014-08-13 18:06:09.707885 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=32] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.332 -2082> 2014-08-13 18:06:09.707982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=31] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.332 -2081> 2014-08-13 18:06:09.707971 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24 194383'15374 is on osd.351 -2080> 2014-08-13 18:06:09.708074 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=30] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.332 -2079> 2014-08-13 18:06:09.708170 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=29] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.332 -2078> 2014-08-13 18:06:09.708155 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24 194416'15404 is on osd.351 -2077> 2014-08-13 18:06:09.708267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=28] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.332 -2076> 2014-08-13 18:06:09.708369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=27] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.332 -2075> 2014-08-13 18:06:09.708341 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24 194422'15450 is on osd.351 -2074> 2014-08-13 18:06:09.708473 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=26] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.332 -2073> 2014-08-13 18:06:09.708573 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=25] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.332 -2072> 2014-08-13 18:06:09.708525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24 194370'15346 is on osd.351 -2071> 2014-08-13 18:06:09.708684 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=24] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.332 -2070> 2014-08-13 18:06:09.708709 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24 194416'15400 is on osd.351 -2069> 2014-08-13 18:06:09.708789 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=23] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.332 -2068> 2014-08-13 18:06:09.708892 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=22] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.332 -2067> 2014-08-13 18:06:09.708893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24 194393'15378 is on osd.351 -2066> 2014-08-13 18:06:09.709001 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=21] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.332 -2065> 2014-08-13 18:06:09.709109 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=20] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.332 -2064> 2014-08-13 18:06:09.709078 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24 194422'15446 is on osd.351 -2063> 2014-08-13 18:06:09.709223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=19] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.332 -2062> 2014-08-13 18:06:09.709265 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24 194416'15398 is on osd.351 -2061> 2014-08-13 18:06:09.709338 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=18] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.332 -2060> 2014-08-13 18:06:09.709556 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.59 192.168.1.112:6822/5265 5 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4ee2400 con 0x3b00c60 -2059> 2014-08-13 18:06:09.709579 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2058> 2014-08-13 18:06:09.709583 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2057> 2014-08-13 18:06:09.709585 7f5c835ef700 20 osd.4 194497 _dispatch 0x4ee2400 osd_map(194497..194497 src has 185139..194497) v3 -2056> 2014-08-13 18:06:09.709592 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -2055> 2014-08-13 18:06:09.709596 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -2054> 2014-08-13 18:06:09.709597 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2053> 2014-08-13 18:06:09.709599 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2052> 2014-08-13 18:06:09.709492 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=17] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.332 -2051> 2014-08-13 18:06:09.709604 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.59 192.168.1.112:6822/5265 6 ==== pg_info(1 pgs e194497:24.1f7) v4 ==== 724+0+0 (3473166398 0 0) 0x5357c40 con 0x3b00c60 -2050> 2014-08-13 18:06:09.709612 7f5c835ef700 10 osd.4 194497 do_waiters -- start -2049> 2014-08-13 18:06:09.709614 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -2048> 2014-08-13 18:06:09.709615 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357c40 pg_info(1 pgs e194497:24.1f7) v4 -2047> 2014-08-13 18:06:09.709619 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709516, event: header_read, request: pg_info(1 pgs e194497:24.1f7) v4 -2046> 2014-08-13 18:06:09.709625 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709517, event: throttled, request: pg_info(1 pgs e194497:24.1f7) v4 -2045> 2014-08-13 18:06:09.709450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24 194416'15416 is on osd.351 -2044> 2014-08-13 18:06:09.709629 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709554, event: all_read, request: pg_info(1 pgs e194497:24.1f7) v4 -2043> 2014-08-13 18:06:09.709632 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709612, event: dispatched, request: pg_info(1 pgs e194497:24.1f7) v4 -2042> 2014-08-13 18:06:09.709637 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709637, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1f7) v4 -2041> 2014-08-13 18:06:09.709641 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.59 -2040> 2014-08-13 18:06:09.709644 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357c40 -2039> 2014-08-13 18:06:09.709647 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.709647, event: started, request: pg_info(1 pgs e194497:24.1f7) v4 -2038> 2014-08-13 18:06:09.709607 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=16] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.332 -2037> 2014-08-13 18:06:09.709632 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24 194416'15428 is on osd.351 -2036> 2014-08-13 18:06:09.709723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=15] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.332 -2035> 2014-08-13 18:06:09.709842 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=14] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.332 -2034> 2014-08-13 18:06:09.709817 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24 194396'15394 is on osd.351 -2033> 2014-08-13 18:06:09.709963 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=13] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.332 -2032> 2014-08-13 18:06:09.710001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24 194370'15344 is on osd.351 -2031> 2014-08-13 18:06:09.710123 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=12] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.332 -2030> 2014-08-13 18:06:09.710187 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24 194370'15360 is on osd.351 -2029> 2014-08-13 18:06:09.710303 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=11] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.332 -2028> 2014-08-13 18:06:09.710372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24 194416'15414 is on osd.351 -2027> 2014-08-13 18:06:09.710470 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=10] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.332 -2026> 2014-08-13 18:06:09.710600 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=9] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.332 -2025> 2014-08-13 18:06:09.710555 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24 194370'15332 is on osd.351 -2024> 2014-08-13 18:06:09.710730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=8] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.332 -2023> 2014-08-13 18:06:09.710738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24 194370'15350 is on osd.351 -2022> 2014-08-13 18:06:09.710861 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=7] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.332 -2021> 2014-08-13 18:06:09.710921 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24 194377'15362 is on osd.351 -2020> 2014-08-13 18:06:09.710993 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=6] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.332 -2019> 2014-08-13 18:06:09.711127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=5] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.332 -2018> 2014-08-13 18:06:09.711105 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24 194370'15342 is on osd.351 -2017> 2014-08-13 18:06:09.711292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=4] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.332 -2016> 2014-08-13 18:06:09.711296 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24 194416'15410 is on osd.351 -2015> 2014-08-13 18:06:09.711471 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=3] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.332 -2014> 2014-08-13 18:06:09.711479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24 194396'15390 is on osd.351 -2013> 2014-08-13 18:06:09.711641 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=2] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.332 -2012> 2014-08-13 18:06:09.711663 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24 194422'15434 is on osd.351 -2011> 2014-08-13 18:06:09.711834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48 u=1] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.332 -2010> 2014-08-13 18:06:09.711846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24 194422'15456 is on osd.351 -2009> 2014-08-13 18:06:09.711996 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -2008> 2014-08-13 18:06:09.712029 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24 194422'15440 is on osd.351 -2007> 2014-08-13 18:06:09.712181 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 337 as a source -2006> 2014-08-13 18:06:09.712216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24 194370'15352 is on osd.351 -2005> 2014-08-13 18:06:09.712328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 is on osd.337 -2004> 2014-08-13 18:06:09.712401 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24 194393'15384 is on osd.351 -2003> 2014-08-13 18:06:09.712491 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 is on osd.337 -2002> 2014-08-13 18:06:09.712585 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24 194416'15420 is on osd.351 -2001> 2014-08-13 18:06:09.712638 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 is on osd.337 -2000> 2014-08-13 18:06:09.712782 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 is on osd.337 -1999> 2014-08-13 18:06:09.712768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24 194393'15382 is on osd.351 -1998> 2014-08-13 18:06:09.712926 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 is on osd.337 -1997> 2014-08-13 18:06:09.712952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24 194422'15436 is on osd.351 -1996> 2014-08-13 18:06:09.713071 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 is on osd.337 -1995> 2014-08-13 18:06:09.713135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24 194416'15426 is on osd.351 -1994> 2014-08-13 18:06:09.713220 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 is on osd.337 -1993> 2014-08-13 18:06:09.713322 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24 194378'15372 is on osd.351 -1992> 2014-08-13 18:06:09.713368 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 is on osd.337 -1991> 2014-08-13 18:06:09.713526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 is on osd.337 -1990> 2014-08-13 18:06:09.713507 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24 194370'15340 is on osd.351 -1989> 2014-08-13 18:06:09.713671 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 is on osd.337 -1988> 2014-08-13 18:06:09.713691 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24 194416'15408 is on osd.351 -1987> 2014-08-13 18:06:09.713816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 is on osd.337 -1986> 2014-08-13 18:06:09.713875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24 194396'15392 is on osd.351 -1985> 2014-08-13 18:06:09.713961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 is on osd.337 -1984> 2014-08-13 18:06:09.714058 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24 194416'15432 is on osd.351 -1983> 2014-08-13 18:06:09.714105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 is on osd.337 -1982> 2014-08-13 18:06:09.714252 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 is on osd.337 -1981> 2014-08-13 18:06:09.714245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24 194422'15448 is on osd.351 -1980> 2014-08-13 18:06:09.714407 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 is on osd.337 -1979> 2014-08-13 18:06:09.714430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24 194416'15424 is on osd.351 -1978> 2014-08-13 18:06:09.714562 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 is on osd.337 -1977> 2014-08-13 18:06:09.714614 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24 194422'15454 is on osd.351 -1976> 2014-08-13 18:06:09.714708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 is on osd.337 -1975> 2014-08-13 18:06:09.714798 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24 194370'15358 is on osd.351 -1974> 2014-08-13 18:06:09.714852 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 is on osd.337 -1973> 2014-08-13 18:06:09.714996 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 is on osd.337 -1972> 2014-08-13 18:06:09.714982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] search_for_missing 8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24 194370'15354 is on osd.351 -1971> 2014-08-13 18:06:09.715140 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 is on osd.337 -1970> 2014-08-13 18:06:09.715165 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] needs_recovery_map missing {1f7/hit_set_24.1f7_archive_2014-08-12 20:53:00.360473_2014-08-13 15:28:31.199671/head/.ceph-internal/24=194416'15395,7f611f7/benchmark_data_storage0111-ib.pocstor_19103_object2895/head//24=194416'15402,702221f7/benchmark_data_storage0111-ib.pocstor_25065_object1924/head//24=194377'15364,7f5131f7/benchmark_data_storage0111-ib.pocstor_19103_object9160/head//24=194416'15422,4ba441f7/benchmark_data_storage0111-ib.pocstor_12495_object461/head//24=194370'15330,e9cd41f7/benchmark_data_storage0111-ib.pocstor_12495_object11898/head//24=194370'15356,5dd491f7/benchmark_data_storage0111-ib.pocstor_19103_object7814/head//24=194416'15412,3081a1f7/benchmark_data_storage0111-ib.pocstor_12495_object5512/head//24=194370'15348,a641f1f7/benchmark_data_storage0111-ib.pocstor_25065_object4539/head//24=194377'15368,b36ff1f7/benchmark_data_storage0111-ib.pocstor_19103_object17157/head//24=194416'15430,296905f7/benchmark_data_storage0111-ib.pocstor_41231_object3062/head//24=194422'15438,ae2655f7/benchmark_data_storage0111-ib.pocstor_41231_object6822/head//24=194422'15452,36a55f7/benchmark_data_storage0111-ib.pocstor_24170_object363/head//24=194393'15380,683065f7/benchmark_data_storage0111-ib.pocstor_12495_object1863/head//24=194370'15336,f0395f7/benchmark_data_storage0111-ib.pocstor_19103_object4207/head//24=194416'15406,de36d5f7/benchmark_data_storage0111-ib.pocstor_12495_object841/head//24=194370'15334,d813e5f7/benchmark_data_storage0111-ib.pocstor_25065_object4582/head//24=194377'15370,aa9fe5f7/benchmark_data_storage0111-ib.pocstor_41231_object4265/head//24=194422'15442,9a7c09f7/benchmark_data_storage0111-ib.pocstor_32482_object158/head//24=194393'15386,f6a119f7/benchmark_data_storage0111-ib.pocstor_19103_object8492/head//24=194416'15418,60c039f7/benchmark_data_storage0111-ib.pocstor_25065_object2215/head//24=194377'15366,6c2139f7/benchmark_data_storage0111-ib.pocstor_41231_object5840/head//24=194422'15444,9e1659f7/benchmark_data_storage0111-ib.pocstor_12495_object2314/head//24=194370'15338,1ffd59f7/benchmark_data_storage0111-ib.pocstor_25065_object6121/head//24=194383'15374,3a6f69f7/benchmark_data_storage0111-ib.pocstor_19103_object3814/head//24=194416'15404,98b979f7/benchmark_data_storage0111-ib.pocstor_41231_object6416/head//24=194422'15450,48f989f7/benchmark_data_storage0111-ib.pocstor_12495_object4486/head//24=194370'15346,2d4099f7/benchmark_data_storage0111-ib.pocstor_19103_object1079/head//24=194416'15400,1ea2b9f7/benchmark_data_storage0111-ib.pocstor_24170_object104/head//24=194393'15378,56db9f7/benchmark_data_storage0111-ib.pocstor_41231_object5917/head//24=194422'15446,806c9f7/benchmark_data_storage0111-ib.pocstor_19103_object245/head//24=194416'15398,4a1ac9f7/benchmark_data_storage0111-ib.pocstor_19103_object8435/head//24=194416'15416,d2ebd9f7/benchmark_data_storage0111-ib.pocstor_19103_object17083/head//24=194416'15428,da1dd9f7/benchmark_data_storage0101-ib.pocstor_6819_object405/head//24=194396'15394,fdb6e9f7/benchmark_data_storage0111-ib.pocstor_12495_object4209/head//24=194370'15344,5adae9f7/benchmark_data_storage0111-ib.pocstor_12495_object13463/head//24=194370'15360,46c0df7/benchmark_data_storage0111-ib.pocstor_19103_object8186/head//24=194416'15414,2ef11df7/benchmark_data_storage0111-ib.pocstor_12495_object826/head//24=194370'15332,1af82df7/benchmark_data_storage0111-ib.pocstor_12495_object6877/head//24=194370'15350,de983df7/benchmark_data_storage0111-ib.pocstor_25065_object1184/head//24=194377'15362,3e8b3df7/benchmark_data_storage0111-ib.pocstor_12495_object2959/head//24=194370'15342,ccbf3df7/benchmark_data_storage0111-ib.pocstor_19103_object7531/head//24=194416'15410,a9164df7/benchmark_data_storage0101-ib.pocstor_6819_object308/head//24=194396'15390,12f64df7/benchmark_data_storage0111-ib.pocstor_41231_object174/head//24=194422'15434,110c4df7/benchmark_data_storage0111-ib.pocstor_41231_object10308/head//24=194422'15456,b2e46df7/benchmark_data_storage0111-ib.pocstor_41231_object3527/head//24=194422'15440,af6f7df7/benchmark_data_storage0111-ib.pocstor_12495_object10255/head//24=194370'15352,93fa8df7/benchmark_data_storage0111-ib.pocstor_32482_object145/head//24=194393'15384,6aa2adf7/benchmark_data_storage0111-ib.pocstor_19103_object8934/head//24=194416'15420,e568adf7/benchmark_data_storage0111-ib.pocstor_24170_object543/head//24=194393'15382,e889adf7/benchmark_data_storage0111-ib.pocstor_41231_object187/head//24=194422'15436,5473bdf7/benchmark_data_storage0111-ib.pocstor_19103_object14821/head//24=194416'15426,2658bdf7/benchmark_data_storage0111-ib.pocstor_25065_object4813/head//24=194378'15372,6c88bdf7/benchmark_data_storage0111-ib.pocstor_12495_object2318/head//24=194370'15340,853bbdf7/benchmark_data_storage0111-ib.pocstor_19103_object6598/head//24=194416'15408,26f9cdf7/benchmark_data_storage0101-ib.pocstor_6819_object340/head//24=194396'15392,9648ddf7/benchmark_data_storage0111-ib.pocstor_19103_object17791/head//24=194416'15432,be5fedf7/benchmark_data_storage0111-ib.pocstor_41231_object6279/head//24=194422'15448,9b75fdf7/benchmark_data_storage0111-ib.pocstor_19103_object11512/head//24=194416'15424,831afdf7/benchmark_data_storage0111-ib.pocstor_41231_object8844/head//24=194422'15454,fd8afdf7/benchmark_data_storage0111-ib.pocstor_12495_object13450/head//24=194370'15358,8fffdf7/benchmark_data_storage0111-ib.pocstor_12495_object11248/head//24=194370'15354} -1969> 2014-08-13 18:06:09.715291 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 is on osd.337 -1968> 2014-08-13 18:06:09.715406 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound -1967> 2014-08-13 18:06:09.715443 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 is on osd.337 -1966> 2014-08-13 18:06:09.715598 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 is on osd.337 -1965> 2014-08-13 18:06:09.715589 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _calc_past_interval_range: already have past intervals back to 194493 -1964> 2014-08-13 18:06:09.715746 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 is on osd.337 -1963> 2014-08-13 18:06:09.715774 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] build_might_have_unfound: built 55,59,351,376 -1962> 2014-08-13 18:06:09.715905 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 is on osd.337 -1961> 2014-08-13 18:06:09.715957 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Activate Finished -1960> 2014-08-13 18:06:09.716051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 is on osd.337 -1959> 2014-08-13 18:06:09.716141 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] enter Started/Primary/Active/Activating -1958> 2014-08-13 18:06:09.716200 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 is on osd.337 -1957> 2014-08-13 18:06:09.716354 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 is on osd.337 -1956> 2014-08-13 18:06:09.716329 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1955> 2014-08-13 18:06:09.716503 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 is on osd.337 -1954> 2014-08-13 18:06:09.716511 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1953> 2014-08-13 18:06:09.716748 7f5c7c7e4700 10 log is not dirty -1952> 2014-08-13 18:06:09.716649 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 is on osd.337 -1951> 2014-08-13 18:06:09.716780 7f5c835ef700 5 -- op tracker -- , seq: 665, time: 2014-08-13 18:06:09.716780, event: done, request: pg_info(1 pgs e194497:24.1f7) v4 -1950> 2014-08-13 18:06:09.716782 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_advance_map [4,55,61]/[4,50,65] -- 4/4 -1949> 2014-08-13 18:06:09.716803 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1948> 2014-08-13 18:06:09.716803 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1947> 2014-08-13 18:06:09.716806 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1946> 2014-08-13 18:06:09.716808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Peering advmap -1945> 2014-08-13 18:06:09.716819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] adjust_need_up_thru now 194496, need_up_thru now false -1944> 2014-08-13 18:06:09.716829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Started advmap -1943> 2014-08-13 18:06:09.716840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] check_recovery_sources no source osds () went down -1942> 2014-08-13 18:06:09.716852 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] handle_activate_map -1941> 2014-08-13 18:06:09.716863 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: handle ActMap primary -1940> 2014-08-13 18:06:09.716874 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] publish_stats_to_osd 194497:22531 -1939> 2014-08-13 18:06:09.716884 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] take_waiters -1938> 2014-08-13 18:06:09.716895 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering/WaitUpThru 1.375748 4 0.000737 -1937> 2014-08-13 18:06:09.716907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] state: Leaving Peering -1936> 2014-08-13 18:06:09.716917 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped+peering m=51] exit Started/Primary/Peering 1.463927 0 0.000000 -1935> 2014-08-13 18:06:09.716931 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active -1934> 2014-08-13 18:06:09.716941 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194487 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: In Active, about to call activate -1933> 2014-08-13 18:06:09.716795 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 is on osd.337 -1932> 2014-08-13 18:06:09.716954 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - purged_snaps [] cached_removed_snaps [] -1931> 2014-08-13 18:06:09.716965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - snap_trimq [] -1930> 2014-08-13 18:06:09.716975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - not complete, missing(51) -1929> 2014-08-13 18:06:09.716957 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 is on osd.337 -1928> 2014-08-13 18:06:09.717104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 is on osd.337 -1927> 2014-08-13 18:06:09.717087 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -1926> 2014-08-13 18:06:09.717254 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 is on osd.337 -1925> 2014-08-13 18:06:09.717308 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] flushed -1924> 2014-08-13 18:06:09.717399 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 is on osd.337 -1923> 2014-08-13 18:06:09.717546 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 is on osd.337 -1922> 2014-08-13 18:06:09.717778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate - starting recovery -1921> 2014-08-13 18:06:09.717796 7f5c7c7e4700 10 osd.4 194497 queue_for_recovery queued pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] -1920> 2014-08-13 18:06:09.717819 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -1919> 2014-08-13 18:06:09.717833 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 is up to date, queueing in pending_activators -1918> 2014-08-13 18:06:09.717694 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 is on osd.337 -1917> 2014-08-13 18:06:09.717875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.50 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate -1916> 2014-08-13 18:06:09.717890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1915> 2014-08-13 18:06:09.717907 7f5c7c7e4700 0 log [INF] : 24.136 restarting backfill on osd.55 from (0'0,0'0] MAX to 194422'15470 -1914> 2014-08-13 18:06:09.717855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 is on osd.337 -1913> 2014-08-13 18:06:09.718095 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 sending log((192834'12379,194422'15470], crt=194422'15470) -1912> 2014-08-13 18:06:09.718114 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing -1911> 2014-08-13 18:06:09.718116 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.55 192.168.1.112:6834/6966 -- pg_log(24.136 epoch 194497 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194497) v4 -- ?+0 0x58bf800 -1910> 2014-08-13 18:06:09.718131 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.55 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate -1909> 2014-08-13 18:06:09.718147 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1908> 2014-08-13 18:06:09.718014 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 is on osd.337 -1907> 2014-08-13 18:06:09.718163 7f5c7c7e4700 0 log [INF] : 24.136 restarting backfill on osd.61 from (0'0,0'0] MAX to 194422'15470 -1906> 2014-08-13 18:06:09.718162 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 is on osd.337 -1905> 2014-08-13 18:06:09.718367 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 sending log((192834'12379,194422'15470], crt=194422'15470) -1904> 2014-08-13 18:06:09.718386 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x445b180 don't know epoch, doing nothing -1903> 2014-08-13 18:06:09.718389 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> osd.61 192.168.1.111:6840/16566 -- pg_log(24.136 epoch 194497 log log((192834'12379,194422'15470], crt=194422'15470) query_epoch 194497) v4 -- ?+0 0x58bec00 -1902> 2014-08-13 18:06:09.718402 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.61 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate -1901> 2014-08-13 18:06:09.718418 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) -1900> 2014-08-13 18:06:09.718430 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 is up to date, queueing in pending_activators -1899> 2014-08-13 18:06:09.718321 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 is on osd.337 -1898> 2014-08-13 18:06:09.718479 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate peer osd.65 24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) uptodate -1897> 2014-08-13 18:06:09.718519 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery primary has 51 missing -1896> 2014-08-13 18:06:09.718481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 is on osd.337 -1895> 2014-08-13 18:06:09.718586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.4 -1894> 2014-08-13 18:06:09.718649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.4 -1893> 2014-08-13 18:06:09.718713 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.4 -1892> 2014-08-13 18:06:09.718645 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 is on osd.337 -1891> 2014-08-13 18:06:09.718776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.4 -1890> 2014-08-13 18:06:09.718840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.4 -1889> 2014-08-13 18:06:09.718789 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 is on osd.337 -1888> 2014-08-13 18:06:09.718903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.4 -1887> 2014-08-13 18:06:09.718966 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.4 -1886> 2014-08-13 18:06:09.718934 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 is on osd.337 -1885> 2014-08-13 18:06:09.719029 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.4 -1884> 2014-08-13 18:06:09.719091 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.4 -1883> 2014-08-13 18:06:09.719154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.4 -1882> 2014-08-13 18:06:09.719078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 is on osd.337 -1881> 2014-08-13 18:06:09.719222 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.4 -1880> 2014-08-13 18:06:09.719286 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.4 -1879> 2014-08-13 18:06:09.719226 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 is on osd.337 -1878> 2014-08-13 18:06:09.719349 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.4 -1877> 2014-08-13 18:06:09.719412 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.4 -1876> 2014-08-13 18:06:09.719375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 is on osd.337 -1875> 2014-08-13 18:06:09.719475 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.4 -1874> 2014-08-13 18:06:09.719545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.4 -1873> 2014-08-13 18:06:09.719609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.4 -1872> 2014-08-13 18:06:09.719541 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1871> 2014-08-13 18:06:09.719673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.4 -1870> 2014-08-13 18:06:09.719735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.4 -1869> 2014-08-13 18:06:09.719798 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.4 -1868> 2014-08-13 18:06:09.719861 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.4 -1867> 2014-08-13 18:06:09.719730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] activate: adding 348 as a source -1866> 2014-08-13 18:06:09.719924 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.4 -1865> 2014-08-13 18:06:09.719987 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.4 -1864> 2014-08-13 18:06:09.720050 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.4 -1863> 2014-08-13 18:06:09.719932 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24 194416'12466 also missing on osd.348 (past last_backfill 0//0//-1) -1862> 2014-08-13 18:06:09.720113 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.4 -1861> 2014-08-13 18:06:09.720176 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.4 -1860> 2014-08-13 18:06:09.720243 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.4 -1859> 2014-08-13 18:06:09.720136 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24 194397'12465 also missing on osd.348 (past last_backfill 0//0//-1) -1858> 2014-08-13 18:06:09.720306 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.4 -1857> 2014-08-13 18:06:09.720369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.4 -1856> 2014-08-13 18:06:09.720336 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24 194416'12491 also missing on osd.348 (past last_backfill 0//0//-1) -1855> 2014-08-13 18:06:09.720451 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.4 -1854> 2014-08-13 18:06:09.720513 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.4 -1853> 2014-08-13 18:06:09.720576 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.4 -1852> 2014-08-13 18:06:09.720499 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24 194416'12489 also missing on osd.348 (past last_backfill 0//0//-1) -1851> 2014-08-13 18:06:09.720639 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.4 -1850> 2014-08-13 18:06:09.720702 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.4 -1849> 2014-08-13 18:06:09.720658 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24 194370'12439 also missing on osd.348 (past last_backfill 0//0//-1) -1848> 2014-08-13 18:06:09.720765 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.4 -1847> 2014-08-13 18:06:09.720827 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.4 -1846> 2014-08-13 18:06:09.720890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.4 -1845> 2014-08-13 18:06:09.720811 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24 194377'12461 also missing on osd.348 (past last_backfill 0//0//-1) -1844> 2014-08-13 18:06:09.720952 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.4 -1843> 2014-08-13 18:06:09.721016 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.4 -1842> 2014-08-13 18:06:09.720956 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24 194377'12457 also missing on osd.348 (past last_backfill 0//0//-1) -1841> 2014-08-13 18:06:09.721079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.4 -1840> 2014-08-13 18:06:09.721142 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.4 -1839> 2014-08-13 18:06:09.721115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24 194416'12493 also missing on osd.348 (past last_backfill 0//0//-1) -1838> 2014-08-13 18:06:09.721209 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.4 -1837> 2014-08-13 18:06:09.721271 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.4 -1836> 2014-08-13 18:06:09.721334 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.4 -1835> 2014-08-13 18:06:09.721267 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24 194377'12455 also missing on osd.348 (past last_backfill 0//0//-1) -1834> 2014-08-13 18:06:09.721397 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.4 -1833> 2014-08-13 18:06:09.721460 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.4 -1832> 2014-08-13 18:06:09.721420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24 194416'12475 also missing on osd.348 (past last_backfill 0//0//-1) -1831> 2014-08-13 18:06:09.721523 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.4 -1830> 2014-08-13 18:06:09.721586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.4 -1829> 2014-08-13 18:06:09.721564 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24 194370'12451 also missing on osd.348 (past last_backfill 0//0//-1) -1828> 2014-08-13 18:06:09.721649 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.4 -1827> 2014-08-13 18:06:09.721714 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.4 -1826> 2014-08-13 18:06:09.721778 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.4 -1825> 2014-08-13 18:06:09.721708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24 194370'12429 also missing on osd.348 (past last_backfill 0//0//-1) -1824> 2014-08-13 18:06:09.721840 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -1823> 2014-08-13 18:06:09.721851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24 194416'12481 also missing on osd.348 (past last_backfill 0//0//-1) -1822> 2014-08-13 18:06:09.721938 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] activate: adding 50 as a source -1821> 2014-08-13 18:06:09.722000 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.50 -1820> 2014-08-13 18:06:09.722064 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=50] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.50 -1819> 2014-08-13 18:06:09.721995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24 194370'12421 also missing on osd.348 (past last_backfill 0//0//-1) -1818> 2014-08-13 18:06:09.722130 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=49] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.50 -1817> 2014-08-13 18:06:09.722203 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=48] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.50 -1816> 2014-08-13 18:06:09.722138 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24 194377'12459 also missing on osd.348 (past last_backfill 0//0//-1) -1815> 2014-08-13 18:06:09.722272 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=47] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.50 -1814> 2014-08-13 18:06:09.722344 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=46] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.50 -1813> 2014-08-13 18:06:09.722287 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24 194416'12485 also missing on osd.348 (past last_backfill 0//0//-1) -1812> 2014-08-13 18:06:09.722417 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=45] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.50 -1811> 2014-08-13 18:06:09.722492 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=44] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.50 -1810> 2014-08-13 18:06:09.722450 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24 194422'12515 also missing on osd.348 (past last_backfill 0//0//-1) -1809> 2014-08-13 18:06:09.722569 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=43] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.50 -1808> 2014-08-13 18:06:09.722647 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=42] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.50 -1807> 2014-08-13 18:06:09.722604 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24 194370'12445 also missing on osd.348 (past last_backfill 0//0//-1) -1806> 2014-08-13 18:06:09.722727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=41] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.50 -1805> 2014-08-13 18:06:09.722809 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=40] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.50 -1804> 2014-08-13 18:06:09.722750 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24 194370'12427 also missing on osd.348 (past last_backfill 0//0//-1) -1803> 2014-08-13 18:06:09.722893 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=39] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.50 -1802> 2014-08-13 18:06:09.722893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24 194422'12513 also missing on osd.348 (past last_backfill 0//0//-1) -1801> 2014-08-13 18:06:09.722979 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=38] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.50 -1800> 2014-08-13 18:06:09.723065 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=37] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.50 -1799> 2014-08-13 18:06:09.723038 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24 194422'12505 also missing on osd.348 (past last_backfill 0//0//-1) -1798> 2014-08-13 18:06:09.723155 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=36] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.50 -1797> 2014-08-13 18:06:09.723186 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24 194416'12495 also missing on osd.348 (past last_backfill 0//0//-1) -1796> 2014-08-13 18:06:09.723249 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=35] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.50 -1795> 2014-08-13 18:06:09.723340 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=34] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.50 -1794> 2014-08-13 18:06:09.723333 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24 194422'12509 also missing on osd.348 (past last_backfill 0//0//-1) -1793> 2014-08-13 18:06:09.723434 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=33] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.50 -1792> 2014-08-13 18:06:09.723530 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=32] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.50 -1791> 2014-08-13 18:06:09.723493 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24 194422'12503 also missing on osd.348 (past last_backfill 0//0//-1) -1790> 2014-08-13 18:06:09.723628 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=31] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.50 -1789> 2014-08-13 18:06:09.723636 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24 194370'12447 also missing on osd.348 (past last_backfill 0//0//-1) -1788> 2014-08-13 18:06:09.723727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=30] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.50 -1787> 2014-08-13 18:06:09.723780 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24 194416'12497 also missing on osd.348 (past last_backfill 0//0//-1) -1786> 2014-08-13 18:06:09.723828 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=29] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.50 -1785> 2014-08-13 18:06:09.723930 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=28] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.50 -1784> 2014-08-13 18:06:09.723924 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24 194416'12479 also missing on osd.348 (past last_backfill 0//0//-1) -1783> 2014-08-13 18:06:09.724051 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=27] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.50 -1782> 2014-08-13 18:06:09.724069 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24 194370'12435 also missing on osd.348 (past last_backfill 0//0//-1) -1781> 2014-08-13 18:06:09.724158 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=26] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.50 -1780> 2014-08-13 18:06:09.724218 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24 194422'12507 also missing on osd.348 (past last_backfill 0//0//-1) -1779> 2014-08-13 18:06:09.724270 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=25] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.50 -1778> 2014-08-13 18:06:09.724379 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=24] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.50 -1777> 2014-08-13 18:06:09.724373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24 194370'12433 also missing on osd.348 (past last_backfill 0//0//-1) -1776> 2014-08-13 18:06:09.724491 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=23] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.50 -1775> 2014-08-13 18:06:09.724529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24 194370'12437 also missing on osd.348 (past last_backfill 0//0//-1) -1774> 2014-08-13 18:06:09.724605 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=22] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.50 -1773> 2014-08-13 18:06:09.724674 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24 194370'12441 also missing on osd.348 (past last_backfill 0//0//-1) -1772> 2014-08-13 18:06:09.724720 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=21] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.50 -1771> 2014-08-13 18:06:09.724837 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=20] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.50 -1770> 2014-08-13 18:06:09.724818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24 194416'12477 also missing on osd.348 (past last_backfill 0//0//-1) -1769> 2014-08-13 18:06:09.724955 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=19] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.50 -1768> 2014-08-13 18:06:09.724960 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24 194370'12425 also missing on osd.348 (past last_backfill 0//0//-1) -1767> 2014-08-13 18:06:09.725076 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=18] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.50 -1766> 2014-08-13 18:06:09.725104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24 194416'12487 also missing on osd.348 (past last_backfill 0//0//-1) -1765> 2014-08-13 18:06:09.725201 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=17] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.50 -1764> 2014-08-13 18:06:09.725255 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24 194422'12517 also missing on osd.348 (past last_backfill 0//0//-1) -1763> 2014-08-13 18:06:09.725325 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=16] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.50 -1762> 2014-08-13 18:06:09.725418 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24 194416'12471 also missing on osd.348 (past last_backfill 0//0//-1) -1761> 2014-08-13 18:06:09.725450 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=15] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.50 -1760> 2014-08-13 18:06:09.725563 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24 194370'12443 also missing on osd.348 (past last_backfill 0//0//-1) -1759> 2014-08-13 18:06:09.725581 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=14] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.50 -1758> 2014-08-13 18:06:09.725710 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=13] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.50 -1757> 2014-08-13 18:06:09.725708 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24 194370'12431 also missing on osd.348 (past last_backfill 0//0//-1) -1756> 2014-08-13 18:06:09.725842 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=12] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.50 -1755> 2014-08-13 18:06:09.725851 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24 194422'12501 also missing on osd.348 (past last_backfill 0//0//-1) -1754> 2014-08-13 18:06:09.725975 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=11] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.50 -1753> 2014-08-13 18:06:09.726016 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24 194370'12423 also missing on osd.348 (past last_backfill 0//0//-1) -1752> 2014-08-13 18:06:09.726109 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=10] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.50 -1751> 2014-08-13 18:06:09.726171 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24 194416'12473 also missing on osd.348 (past last_backfill 0//0//-1) -1750> 2014-08-13 18:06:09.726248 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=9] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.50 -1749> 2014-08-13 18:06:09.726326 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24 194422'12511 also missing on osd.348 (past last_backfill 0//0//-1) -1748> 2014-08-13 18:06:09.726387 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=8] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.50 -1747> 2014-08-13 18:06:09.726487 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24 194416'12469 also missing on osd.348 (past last_backfill 0//0//-1) -1746> 2014-08-13 18:06:09.726526 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=7] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.50 -1745> 2014-08-13 18:06:09.726641 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24 194416'12499 also missing on osd.348 (past last_backfill 0//0//-1) -1744> 2014-08-13 18:06:09.726667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=6] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.50 -1743> 2014-08-13 18:06:09.726791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24 194377'12453 also missing on osd.348 (past last_backfill 0//0//-1) -1742> 2014-08-13 18:06:09.726808 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=5] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.50 -1741> 2014-08-13 18:06:09.726940 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24 194416'12483 also missing on osd.348 (past last_backfill 0//0//-1) -1740> 2014-08-13 18:06:09.726951 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=4] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.50 -1739> 2014-08-13 18:06:09.727084 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] search_for_missing 802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24 194370'12449 also missing on osd.348 (past last_backfill 0//0//-1) -1738> 2014-08-13 18:06:09.727096 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=3] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.50 -1737> 2014-08-13 18:06:09.727245 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=2] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.50 -1736> 2014-08-13 18:06:09.727235 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] needs_recovery_map missing {1bb/hit_set_24.1bb_archive_2014-08-12 20:53:01.910321_2014-08-13 15:28:36.883152/head/.ceph-internal/24=194416'12466,b3a901bb/benchmark_data_storage0101-ib.pocstor_6819_object564/head//24=194397'12465,848a21bb/benchmark_data_storage0111-ib.pocstor_19103_object14396/head//24=194416'12491,bffd21bb/benchmark_data_storage0111-ib.pocstor_19103_object13268/head//24=194416'12489,f17131bb/benchmark_data_storage0111-ib.pocstor_12495_object8307/head//24=194370'12439,ada131bb/benchmark_data_storage0111-ib.pocstor_25065_object3519/head//24=194377'12461,b02181bb/benchmark_data_storage0111-ib.pocstor_25065_object2385/head//24=194377'12457,263ac1bb/benchmark_data_storage0111-ib.pocstor_19103_object14843/head//24=194416'12493,ecc315bb/benchmark_data_storage0111-ib.pocstor_25065_object1330/head//24=194377'12455,310025bb/benchmark_data_storage0111-ib.pocstor_19103_object3114/head//24=194416'12475,68d425bb/benchmark_data_storage0111-ib.pocstor_12495_object13106/head//24=194370'12451,3ffc25bb/benchmark_data_storage0111-ib.pocstor_12495_object2704/head//24=194370'12429,779435bb/benchmark_data_storage0111-ib.pocstor_19103_object8806/head//24=194416'12481,4d1535bb/benchmark_data_storage0111-ib.pocstor_12495_object845/head//24=194370'12421,c66465bb/benchmark_data_storage0111-ib.pocstor_25065_object3057/head//24=194377'12459,f39385bb/benchmark_data_storage0111-ib.pocstor_19103_object12369/head//24=194416'12485,44c95bb/benchmark_data_storage0111-ib.pocstor_41231_object9944/head//24=194422'12515,eeff95bb/benchmark_data_storage0111-ib.pocstor_12495_object10603/head//24=194370'12445,5979e5bb/benchmark_data_storage0111-ib.pocstor_12495_object1181/head//24=194370'12427,f27ae5bb/benchmark_data_storage0111-ib.pocstor_41231_object7081/head//24=194422'12513,3aa109bb/benchmark_data_storage0111-ib.pocstor_41231_object4639/head//24=194422'12505,699419bb/benchmark_data_storage0111-ib.pocstor_19103_object15444/head//24=194416'12495,559c19bb/benchmark_data_storage0111-ib.pocstor_41231_object6203/head//24=194422'12509,adb339bb/benchmark_data_storage0111-ib.pocstor_41231_object1547/head//24=194422'12503,99dd39bb/benchmark_data_storage0111-ib.pocstor_12495_object12052/head//24=194370'12447,cdac69bb/benchmark_data_storage0111-ib.pocstor_19103_object15604/head//24=194416'12497,d47379bb/benchmark_data_storage0111-ib.pocstor_19103_object8463/head//24=194416'12479,325489bb/benchmark_data_storage0111-ib.pocstor_12495_object7086/head//24=194370'12435,edb989bb/benchmark_data_storage0111-ib.pocstor_41231_object5456/head//24=194422'12507,435a89bb/benchmark_data_storage0111-ib.pocstor_12495_object6281/head//24=194370'12433,436aa9bb/benchmark_data_storage0111-ib.pocstor_12495_object7454/head//24=194370'12437,1fb4d9bb/benchmark_data_storage0111-ib.pocstor_12495_object8813/head//24=194370'12441,6886e9bb/benchmark_data_storage0111-ib.pocstor_19103_object7218/head//24=194416'12477,7ea9e9bb/benchmark_data_storage0111-ib.pocstor_12495_object1026/head//24=194370'12425,8fd8f9bb/benchmark_data_storage0111-ib.pocstor_19103_object12894/head//24=194416'12487,a5f20dbb/benchmark_data_storage0111-ib.pocstor_41231_object10434/head//24=194422'12517,64ae0dbb/benchmark_data_storage0111-ib.pocstor_19103_object2828/head//24=194416'12471,16af0dbb/benchmark_data_storage0111-ib.pocstor_12495_object10460/head//24=194370'12443,aca3dbb/benchmark_data_storage0111-ib.pocstor_12495_object3584/head//24=194370'12431,61414dbb/benchmark_data_storage0111-ib.pocstor_41231_object1097/head//24=194422'12501,fe3a4dbb/benchmark_data_storage0111-ib.pocstor_12495_object892/head//24=194370'12423,24df4dbb/benchmark_data_storage0111-ib.pocstor_19103_object3087/head//24=194416'12473,8a245dbb/benchmark_data_storage0111-ib.pocstor_41231_object6343/head//24=194422'12511,5b3b6dbb/benchmark_data_storage0111-ib.pocstor_19103_object1626/head//24=194416'12469,c1ffadbb/benchmark_data_storage0111-ib.pocstor_19103_object17909/head//24=194416'12499,efb5bdbb/benchmark_data_storage0111-ib.pocstor_25065_object750/head//24=194377'12453,663dcdbb/benchmark_data_storage0111-ib.pocstor_19103_object11268/head//24=194416'12483,802cedbb/benchmark_data_storage0111-ib.pocstor_12495_object12466/head//24=194370'12449} -1735> 2014-08-13 18:06:09.727395 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51 u=1] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.50 -1734> 2014-08-13 18:06:09.727429 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound -1733> 2014-08-13 18:06:09.727548 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -1732> 2014-08-13 18:06:09.727604 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _calc_past_interval_range: already have past intervals back to 194493 -1731> 2014-08-13 18:06:09.727737 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 55 as a source -1730> 2014-08-13 18:06:09.727761 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] build_might_have_unfound: built 331,332,337,348,379 -1729> 2014-08-13 18:06:09.727890 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.55 (past last_backfill 0//0//-1) -1728> 2014-08-13 18:06:09.727946 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Activate Finished -1727> 2014-08-13 18:06:09.728044 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.55 (past last_backfill 0//0//-1) -1726> 2014-08-13 18:06:09.728122 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] enter Started/Primary/Active/Activating -1725> 2014-08-13 18:06:09.728200 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.55 (past last_backfill 0//0//-1) -1724> 2014-08-13 18:06:09.728278 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1723> 2014-08-13 18:06:09.728353 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.55 (past last_backfill 0//0//-1) -1722> 2014-08-13 18:06:09.728460 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1721> 2014-08-13 18:06:09.728506 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.55 (past last_backfill 0//0//-1) -1720> 2014-08-13 18:06:09.728684 7f5c7bde3700 10 log is not dirty -1719> 2014-08-13 18:06:09.728725 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_advance_map [396,368,4]/[396,363] -- 396/396 -1718> 2014-08-13 18:06:09.728747 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1717> 2014-08-13 18:06:09.728753 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] state: Started advmap -1716> 2014-08-13 18:06:09.728765 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] check_recovery_sources no source osds () went down -1715> 2014-08-13 18:06:09.728776 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map -1714> 2014-08-13 18:06:09.728659 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.55 (past last_backfill 0//0//-1) -1713> 2014-08-13 18:06:09.728804 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] take_waiters -1712> 2014-08-13 18:06:09.728816 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1711> 2014-08-13 18:06:09.728827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.299( v 194422'15411 (192834'13573,194422'15411] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194496/194463 194495/194495/193205) [396,368,4]/[396,363] r=-1 lpr=194495 pi=194462-194494/11 luod=0'0 crt=0'0 lcod 0'0 active+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1710> 2014-08-13 18:06:09.728842 7f5c7bde3700 10 log is not dirty -1709> 2014-08-13 18:06:09.728860 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,343,286]/[4,343,286] -- 4/4 -1708> 2014-08-13 18:06:09.728873 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1707> 2014-08-13 18:06:09.728877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -1706> 2014-08-13 18:06:09.728886 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -1705> 2014-08-13 18:06:09.728896 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -1704> 2014-08-13 18:06:09.728905 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -1703> 2014-08-13 18:06:09.728913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -1702> 2014-08-13 18:06:09.728922 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -1701> 2014-08-13 18:06:09.728931 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2660 -1700> 2014-08-13 18:06:09.728939 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] take_waiters -1699> 2014-08-13 18:06:09.728947 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1698> 2014-08-13 18:06:09.728810 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.55 (past last_backfill 0//0//-1) -1697> 2014-08-13 18:06:09.728955 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d5( v 194215'466 (0'0,194215'466] local-les=194496 n=25 ec=183096 les/c 194496/194496 194495/194495/194495) [4,343,286] r=0 lpr=194495 crt=191727'426 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1696> 2014-08-13 18:06:09.728965 7f5c7bde3700 10 log is not dirty -1695> 2014-08-13 18:06:09.728979 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [51,4,308]/[51,4,308] -- 51/51 -1694> 2014-08-13 18:06:09.728992 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1693> 2014-08-13 18:06:09.728996 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap -1692> 2014-08-13 18:06:09.729005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down -1691> 2014-08-13 18:06:09.729015 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map -1690> 2014-08-13 18:06:09.729024 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters -1689> 2014-08-13 18:06:09.729032 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1688> 2014-08-13 18:06:09.729040 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.537( v 194215'501 (0'0,194215'501] local-les=194496 n=28 ec=183096 les/c 194496/194496 194495/194495/192971) [51,4,308] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1687> 2014-08-13 18:06:09.729051 7f5c7bde3700 10 log is not dirty -1686> 2014-08-13 18:06:09.729067 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,340,347]/[4,340,347] -- 4/4 -1685> 2014-08-13 18:06:09.729078 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1684> 2014-08-13 18:06:09.729082 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -1683> 2014-08-13 18:06:09.729091 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -1682> 2014-08-13 18:06:09.729099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -1681> 2014-08-13 18:06:09.728962 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.55 (past last_backfill 0//0//-1) -1680> 2014-08-13 18:06:09.729108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -1679> 2014-08-13 18:06:09.729117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -1678> 2014-08-13 18:06:09.729126 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -1677> 2014-08-13 18:06:09.728974 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] _activate_committed 194497 peer_activated now 4 last_epoch_started 194493 same_interval_since 194496 -1676> 2014-08-13 18:06:09.729134 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:3007 -1675> 2014-08-13 18:06:09.729143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] take_waiters -1674> 2014-08-13 18:06:09.729158 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1673> 2014-08-13 18:06:09.729167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1b8( v 194215'479 (0'0,194215'479] local-les=194496 n=34 ec=183096 les/c 194496/194496 194495/194495/194495) [4,340,347] r=0 lpr=194495 crt=191648'362 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1672> 2014-08-13 18:06:09.729176 7f5c7bde3700 10 log is not dirty -1671> 2014-08-13 18:06:09.729189 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_advance_map [345,24,4]/[345,24,4] -- 345/345 -1670> 2014-08-13 18:06:09.729202 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1669> 2014-08-13 18:06:09.729205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] state: Started advmap -1668> 2014-08-13 18:06:09.729215 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] check_recovery_sources no source osds () went down -1667> 2014-08-13 18:06:09.729224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map -1666> 2014-08-13 18:06:09.729233 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] take_waiters -1665> 2014-08-13 18:06:09.729241 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1664> 2014-08-13 18:06:09.729250 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.29a( v 194215'488 (0'0,194215'488] local-les=194496 n=31 ec=183096 les/c 194496/194496 194495/194495/192981) [345,24,4] r=2 lpr=194495 pi=192984-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1663> 2014-08-13 18:06:09.729259 7f5c7bde3700 10 log is not dirty -1662> 2014-08-13 18:06:09.729113 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.55 (past last_backfill 0//0//-1) -1661> 2014-08-13 18:06:09.729271 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_advance_map [6,374,4]/[6,374,4] -- 6/6 -1660> 2014-08-13 18:06:09.729288 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1659> 2014-08-13 18:06:09.729143 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] flushed -1658> 2014-08-13 18:06:09.729292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: Started advmap -1657> 2014-08-13 18:06:09.729302 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] check_recovery_sources no source osds () went down -1656> 2014-08-13 18:06:09.729311 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map -1655> 2014-08-13 18:06:09.729361 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] take_waiters -1654> 2014-08-13 18:06:09.729371 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1653> 2014-08-13 18:06:09.729379 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1652> 2014-08-13 18:06:09.729390 7f5c7bde3700 10 log is not dirty -1651> 2014-08-13 18:06:09.729268 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.55 (past last_backfill 0//0//-1) -1650> 2014-08-13 18:06:09.729408 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_advance_map [376,4,303]/[376,4,303] -- 376/376 -1649> 2014-08-13 18:06:09.729421 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1648> 2014-08-13 18:06:09.729425 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] state: Started advmap -1647> 2014-08-13 18:06:09.729435 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] check_recovery_sources no source osds () went down -1646> 2014-08-13 18:06:09.729445 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map -1645> 2014-08-13 18:06:09.729453 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] take_waiters -1644> 2014-08-13 18:06:09.729463 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1643> 2014-08-13 18:06:09.729472 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.226( v 194215'485 (0'0,194215'485] local-les=194496 n=33 ec=183096 les/c 194496/194493 194495/194495/192974) [376,4,303] r=1 lpr=194495 pi=192975-194494/34 luod=0'0 crt=192888'461 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1642> 2014-08-13 18:06:09.729482 7f5c7bde3700 10 log is not dirty -1641> 2014-08-13 18:06:09.729494 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_advance_map [4,338,376]/[4,338,376] -- 4/4 -1640> 2014-08-13 18:06:09.729506 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1639> 2014-08-13 18:06:09.729509 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active advmap -1638> 2014-08-13 18:06:09.729529 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Started advmap -1637> 2014-08-13 18:06:09.729540 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] check_recovery_sources no source osds (4,338) went down -1636> 2014-08-13 18:06:09.729550 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map -1635> 2014-08-13 18:06:09.729420 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.55 (past last_backfill 0//0//-1) -1634> 2014-08-13 18:06:09.729559 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: handling ActMap -1633> 2014-08-13 18:06:09.729571 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.55 (past last_backfill 0//0//-1) -1632> 2014-08-13 18:06:09.729813 7f5c7bde3700 10 osd.4 194497 queue_for_recovery queued pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] -1631> 2014-08-13 18:06:09.729837 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: handle ActMap primary -1630> 2014-08-13 18:06:09.729724 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.55 (past last_backfill 0//0//-1) -1629> 2014-08-13 18:06:09.729957 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] publish_stats_to_osd 194497:2600 -1628> 2014-08-13 18:06:09.729970 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] take_waiters -1627> 2014-08-13 18:06:09.729980 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1626> 2014-08-13 18:06:09.729989 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1625> 2014-08-13 18:06:09.729999 7f5c7bde3700 10 log is not dirty -1624> 2014-08-13 18:06:09.729875 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.55 (past last_backfill 0//0//-1) -1623> 2014-08-13 18:06:09.730014 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_advance_map [374,4,314]/[374,4,12] -- 374/374 -1622> 2014-08-13 18:06:09.730032 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1621> 2014-08-13 18:06:09.730036 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: Started advmap -1620> 2014-08-13 18:06:09.730047 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] check_recovery_sources no source osds () went down -1619> 2014-08-13 18:06:09.730057 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map -1618> 2014-08-13 18:06:09.730120 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] take_waiters -1617> 2014-08-13 18:06:09.730133 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1616> 2014-08-13 18:06:09.730143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1615> 2014-08-13 18:06:09.730160 7f5c7bde3700 10 log is not dirty -1614> 2014-08-13 18:06:09.730166 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4af98c0 192.168.1.107:6809/53141 -1613> 2014-08-13 18:06:09.730026 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.55 (past last_backfill 0//0//-1) -1612> 2014-08-13 18:06:09.730176 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6809/53141 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4cf9f80 con 0x4af98c0 -1611> 2014-08-13 18:06:09.730189 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.6 has 194497 -1610> 2014-08-13 18:06:09.730191 7f5c7bde3700 7 osd.4 194497 do_notify osd 6 on 1 PGs -1609> 2014-08-13 18:06:09.730193 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6809/53141 -- pg_notify(24.cd(36) epoch 194497) v5 -- ?+0 0x4df6ac0 con 0x4af98c0 -1608> 2014-08-13 18:06:09.730200 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4854620 192.168.1.107:6812/53408 -1607> 2014-08-13 18:06:09.730205 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6812/53408 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4cfb180 con 0x4854620 -1606> 2014-08-13 18:06:09.730213 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.15 has 194497 -1605> 2014-08-13 18:06:09.730215 7f5c7bde3700 7 osd.4 194497 do_notify osd 15 on 1 PGs -1604> 2014-08-13 18:06:09.730217 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.107:6812/53408 -- pg_notify(24.20e(40) epoch 194497) v5 -- ?+0 0x4df08c0 con 0x4854620 -1603> 2014-08-13 18:06:09.730223 7f5c7bde3700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4857bc0 192.168.1.103:6831/41157 -1602> 2014-08-13 18:06:09.730227 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6831/41157 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4cfd100 con 0x4857bc0 -1601> 2014-08-13 18:06:09.730236 7f5c7bde3700 10 osd.4 194497 note_peer_epoch osd.31 has 194497 -1600> 2014-08-13 18:06:09.730238 7f5c7bde3700 7 osd.4 194497 do_notify osd 31 on 1 PGs -1599> 2014-08-13 18:06:09.730240 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6831/41157 -- pg_notify(24.1e4(48) epoch 194497) v5 -- ?+0 0x4df1a40 con 0x4857bc0 -1598> 2014-08-13 18:06:09.730294 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing -1597> 2014-08-13 18:06:09.730298 7f5c7bde3700 7 osd.4 194497 do_notify osd 285 on 1 PGs -1596> 2014-08-13 18:06:09.730299 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.101:6822/26295 -- pg_notify(24.1e6(36) epoch 194497) v5 -- ?+0 0x4df16c0 con 0x445e5c0 -1595> 2014-08-13 18:06:09.730177 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.55 (past last_backfill 0//0//-1) -1594> 2014-08-13 18:06:09.730333 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 -1593> 2014-08-13 18:06:09.730337 7f5c7bde3700 7 osd.4 194497 do_notify osd 343 on 1 PGs -1592> 2014-08-13 18:06:09.730338 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6802/61960 -- pg_notify(24.206(36) epoch 194497) v5 -- ?+0 0x4df2300 con 0x48506e0 -1591> 2014-08-13 18:06:09.730350 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing -1590> 2014-08-13 18:06:09.730352 7f5c7bde3700 7 osd.4 194497 do_notify osd 374 on 1 PGs -1589> 2014-08-13 18:06:09.730354 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.102:6828/16129 -- pg_notify(24.2ec(54) epoch 194497) v5 -- ?+0 0x4df0fc0 con 0x3cf4360 -1588> 2014-08-13 18:06:09.730370 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 296 -1587> 2014-08-13 18:06:09.730394 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4854a40 don't know epoch, doing nothing -1586> 2014-08-13 18:06:09.730437 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.109:6806/31286 -- pg_info(1 pgs e194497:24.181(42)) v4 -- ?+0 0x5355e80 con 0x4854a40 -1585> 2014-08-13 18:06:09.730454 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 320 -1584> 2014-08-13 18:06:09.730474 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing -1583> 2014-08-13 18:06:09.730333 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.55 (past last_backfill 0//0//-1) -1582> 2014-08-13 18:06:09.730485 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6820/10050 -- pg_info(1 pgs e194497:24.181(42)) v4 -- ?+0 0x5356e40 con 0x4b12520 -1581> 2014-08-13 18:06:09.730497 7f5c7bde3700 20 osd.4 194497 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 332 -1580> 2014-08-13 18:06:09.730503 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing -1579> 2014-08-13 18:06:09.730519 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.110:6806/42024 -- pg_info(1 pgs e194497:24.1bb(37)) v4 -- ?+0 0x5352a00 con 0x4ab2d60 -1578> 2014-08-13 18:06:09.730531 7f5c7bde3700 20 osd.4 194497 Sending info 24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 337 -1577> 2014-08-13 18:06:09.730542 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing -1576> 2014-08-13 18:06:09.730558 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.104:6804/62197 -- pg_info(1 pgs e194497:24.1bb(37)) v4 -- ?+0 0x53563c0 con 0x3c55540 -1575> 2014-08-13 18:06:09.730569 7f5c7bde3700 20 osd.4 194497 Sending info 24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) to shard 344 -1574> 2014-08-13 18:06:09.730577 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing -1573> 2014-08-13 18:06:09.730605 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.106:6812/6757 -- pg_info(1 pgs e194497:24.181(42)) v4 -- ?+0 0x4df1880 con 0x4aff220 -1572> 2014-08-13 18:06:09.730484 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.55 (past last_backfill 0//0//-1) -1571> 2014-08-13 18:06:09.730699 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a7e90 prio 127 cost 0 latency 0.082497 osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] -1570> 2014-08-13 18:06:09.730725 7f5c7bde3700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.730725, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1569> 2014-08-13 18:06:09.730737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] handle_message: 0x20a7e90 -1568> 2014-08-13 18:06:09.730749 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1567> 2014-08-13 18:06:09.730764 7f5c7bde3700 7 osd.4 pg_epoch: 194497 pg[22.639( v 194215'492 (0'0,194215'492] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192976) [367,44,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'458 lcod 0'0 active] sub_op_scrub_unreserve -1566> 2014-08-13 18:06:09.730781 7f5c7bde3700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.730781, event: started, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1565> 2014-08-13 18:06:09.730789 7f5c7bde3700 10 osd.4 194497 dequeue_op 0x20a7e90 finish -1564> 2014-08-13 18:06:09.730792 7f5c7bde3700 5 -- op tracker -- , seq: 657, time: 2014-08-13 18:06:09.730792, event: done, request: osd_sub_op(unknown.0.0:0 22.639 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -1563> 2014-08-13 18:06:09.730834 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [328,4,20]/[328,4,20] -- 328/328 -1562> 2014-08-13 18:06:09.730637 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.55 (past last_backfill 0//0//-1) -1561> 2014-08-13 18:06:09.730854 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1560> 2014-08-13 18:06:09.730866 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap -1559> 2014-08-13 18:06:09.730878 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down -1558> 2014-08-13 18:06:09.730888 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map -1557> 2014-08-13 18:06:09.730897 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters -1556> 2014-08-13 18:06:09.730922 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1555> 2014-08-13 18:06:09.730931 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1a0( v 194215'512 (0'0,194215'512] local-les=194496 n=30 ec=183096 les/c 194496/194496 194495/194495/192949) [328,4,20] r=1 lpr=194495 pi=192975-194494/28 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1554> 2014-08-13 18:06:09.730943 7f5c7bde3700 10 log is not dirty -1553> 2014-08-13 18:06:09.730959 7f5c7bde3700 10 osd.4 pg_epoch: 194496 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_advance_map [45,319,4]/[45,319,4] -- 45/45 -1552> 2014-08-13 18:06:09.730981 7f5c7bde3700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1551> 2014-08-13 18:06:09.730985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: Started advmap -1550> 2014-08-13 18:06:09.730995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] check_recovery_sources no source osds () went down -1549> 2014-08-13 18:06:09.730856 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.55 (past last_backfill 0//0//-1) -1548> 2014-08-13 18:06:09.731005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map -1547> 2014-08-13 18:06:09.731046 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] take_waiters -1546> 2014-08-13 18:06:09.731059 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -1545> 2014-08-13 18:06:09.731068 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -1544> 2014-08-13 18:06:09.731078 7f5c7bde3700 10 log is not dirty -1543> 2014-08-13 18:06:09.731083 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 44 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) -1542> 2014-08-13 18:06:09.731097 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 44, already have info from that osd, ignoring -1541> 2014-08-13 18:06:09.731107 7f5c7bde3700 10 log is not dirty -1540> 2014-08-13 18:06:09.731111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) -1539> 2014-08-13 18:06:09.731124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 55, already have info from that osd, ignoring -1538> 2014-08-13 18:06:09.731133 7f5c7bde3700 10 log is not dirty -1537> 2014-08-13 18:06:09.731137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 337 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -1536> 2014-08-13 18:06:09.731011 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.55 (past last_backfill 0//0//-1) -1535> 2014-08-13 18:06:09.731155 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 337, already have info from that osd, ignoring -1534> 2014-08-13 18:06:09.731165 7f5c7bde3700 10 log is not dirty -1533> 2014-08-13 18:06:09.731169 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 371 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -1532> 2014-08-13 18:06:09.731181 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 371, already have info from that osd, ignoring -1531> 2014-08-13 18:06:09.731191 7f5c7bde3700 10 log is not dirty -1530> 2014-08-13 18:06:09.731194 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) -1529> 2014-08-13 18:06:09.731205 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: got info from osd.343 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) -1528> 2014-08-13 18:06:09.731219 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] exit Started/Stray 1.423533 8 0.001105 -1527> 2014-08-13 18:06:09.731232 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive -1526> 2014-08-13 18:06:09.731252 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] enter Started/ReplicaActive/RepNotRecovering -1525> 2014-08-13 18:06:09.731264 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194487 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive NOTIFY m=41] state: In ReplicaActive, about to call activate -1524> 2014-08-13 18:06:09.731273 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] activate - not complete, missing(41) -1523> 2014-08-13 18:06:09.731163 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.55 (past last_backfill 0//0//-1) -1522> 2014-08-13 18:06:09.731318 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.55 (past last_backfill 0//0//-1) -1521> 2014-08-13 18:06:09.731472 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.55 (past last_backfill 0//0//-1) -1520> 2014-08-13 18:06:09.731624 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.55 (past last_backfill 0//0//-1) -1519> 2014-08-13 18:06:09.731776 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.55 (past last_backfill 0//0//-1) -1518> 2014-08-13 18:06:09.731929 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.55 (past last_backfill 0//0//-1) -1517> 2014-08-13 18:06:09.732152 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] state: Activate Finished -1516> 2014-08-13 18:06:09.732217 7f5c7bde3700 10 log is not dirty -1515> 2014-08-13 18:06:09.732081 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.55 (past last_backfill 0//0//-1) -1514> 2014-08-13 18:06:09.732235 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.55 (past last_backfill 0//0//-1) -1513> 2014-08-13 18:06:09.732235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 296 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -1512> 2014-08-13 18:06:09.732489 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 crt=194422'14476 lcod 0'0 inactive m=41] _activate_committed 194497 telling primary -1511> 2014-08-13 18:06:09.732519 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 -1510> 2014-08-13 18:06:09.732522 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.343 192.168.1.104:6802/61960 -- pg_info(1 pgs e194497:24.206) v4 -- ?+0 0x588b2c0 -1509> 2014-08-13 18:06:09.732521 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.332 192.168.1.110:6806/42024 9 ==== pg_info(1 pgs e194497:24.1bb) v4 ==== 748+0+0 (4096746971 0 0) 0x5352a00 con 0x4ab2d60 -1508> 2014-08-13 18:06:09.732536 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1507> 2014-08-13 18:06:09.732538 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1506> 2014-08-13 18:06:09.732388 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.55 (past last_backfill 0//0//-1) -1505> 2014-08-13 18:06:09.732540 7f5c835ef700 20 osd.4 194497 _dispatch 0x5352a00 pg_info(1 pgs e194497:24.1bb) v4 -1504> 2014-08-13 18:06:09.732545 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732452, event: header_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1503> 2014-08-13 18:06:09.732539 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] flushed -1502> 2014-08-13 18:06:09.732551 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732453, event: throttled, request: pg_info(1 pgs e194497:24.1bb) v4 -1501> 2014-08-13 18:06:09.732555 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732503, event: all_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1500> 2014-08-13 18:06:09.732558 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732535, event: dispatched, request: pg_info(1 pgs e194497:24.1bb) v4 -1499> 2014-08-13 18:06:09.732563 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732563, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1bb) v4 -1498> 2014-08-13 18:06:09.732567 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.332 -1497> 2014-08-13 18:06:09.732570 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5352a00 -1496> 2014-08-13 18:06:09.732573 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732573, event: started, request: pg_info(1 pgs e194497:24.1bb) v4 -1495> 2014-08-13 18:06:09.732588 7f5c835ef700 5 -- op tracker -- , seq: 666, time: 2014-08-13 18:06:09.732588, event: done, request: pg_info(1 pgs e194497:24.1bb) v4 -1494> 2014-08-13 18:06:09.732596 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1493> 2014-08-13 18:06:09.732598 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1492> 2014-08-13 18:06:09.732451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 296, already have info from that osd, ignoring -1491> 2014-08-13 18:06:09.732623 7f5c7bde3700 10 log is not dirty -1490> 2014-08-13 18:06:09.732629 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 31 info: 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) -1489> 2014-08-13 18:06:09.732642 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: got info from osd.31 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) -1488> 2014-08-13 18:06:09.732659 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] exit Started/Stray 1.451552 8 0.000659 -1487> 2014-08-13 18:06:09.732671 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive -1486> 2014-08-13 18:06:09.732540 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.55 (past last_backfill 0//0//-1) -1485> 2014-08-13 18:06:09.732703 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] enter Started/ReplicaActive/RepNotRecovering -1484> 2014-08-13 18:06:09.732716 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194487 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive NOTIFY m=46] state: In ReplicaActive, about to call activate -1483> 2014-08-13 18:06:09.732726 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] activate - not complete, missing(46) -1482> 2014-08-13 18:06:09.732798 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.344 192.168.1.106:6812/6757 9 ==== pg_info(1 pgs e194497:24.181) v4 ==== 748+0+0 (3029468415 0 0) 0x4df1880 con 0x4aff220 -1481> 2014-08-13 18:06:09.732820 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1480> 2014-08-13 18:06:09.732826 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1479> 2014-08-13 18:06:09.732828 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df1880 pg_info(1 pgs e194497:24.181) v4 -1478> 2014-08-13 18:06:09.732832 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732714, event: header_read, request: pg_info(1 pgs e194497:24.181) v4 -1477> 2014-08-13 18:06:09.732693 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.55 (past last_backfill 0//0//-1) -1476> 2014-08-13 18:06:09.732839 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732715, event: throttled, request: pg_info(1 pgs e194497:24.181) v4 -1475> 2014-08-13 18:06:09.732844 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732765, event: all_read, request: pg_info(1 pgs e194497:24.181) v4 -1474> 2014-08-13 18:06:09.732853 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732819, event: dispatched, request: pg_info(1 pgs e194497:24.181) v4 -1473> 2014-08-13 18:06:09.732858 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732858, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.181) v4 -1472> 2014-08-13 18:06:09.732862 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.344 -1471> 2014-08-13 18:06:09.732865 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df1880 -1470> 2014-08-13 18:06:09.732869 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732869, event: started, request: pg_info(1 pgs e194497:24.181) v4 -1469> 2014-08-13 18:06:09.732885 7f5c835ef700 5 -- op tracker -- , seq: 667, time: 2014-08-13 18:06:09.732885, event: done, request: pg_info(1 pgs e194497:24.181) v4 -1468> 2014-08-13 18:06:09.732894 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1467> 2014-08-13 18:06:09.732900 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1466> 2014-08-13 18:06:09.732905 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.320 192.168.1.106:6820/10050 16 ==== pg_info(1 pgs e194497:24.181) v4 ==== 748+0+0 (3161807814 0 0) 0x5356e40 con 0x4b12520 -1465> 2014-08-13 18:06:09.732913 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1464> 2014-08-13 18:06:09.732914 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1463> 2014-08-13 18:06:09.732916 7f5c835ef700 20 osd.4 194497 _dispatch 0x5356e40 pg_info(1 pgs e194497:24.181) v4 -1462> 2014-08-13 18:06:09.732919 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732723, event: header_read, request: pg_info(1 pgs e194497:24.181) v4 -1461> 2014-08-13 18:06:09.732923 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732724, event: throttled, request: pg_info(1 pgs e194497:24.181) v4 -1460> 2014-08-13 18:06:09.732926 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732768, event: all_read, request: pg_info(1 pgs e194497:24.181) v4 -1459> 2014-08-13 18:06:09.732929 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732913, event: dispatched, request: pg_info(1 pgs e194497:24.181) v4 -1458> 2014-08-13 18:06:09.732933 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732932, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.181) v4 -1457> 2014-08-13 18:06:09.732936 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.320 -1456> 2014-08-13 18:06:09.732938 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5356e40 -1455> 2014-08-13 18:06:09.732941 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732941, event: started, request: pg_info(1 pgs e194497:24.181) v4 -1454> 2014-08-13 18:06:09.732952 7f5c835ef700 5 -- op tracker -- , seq: 668, time: 2014-08-13 18:06:09.732952, event: done, request: pg_info(1 pgs e194497:24.181) v4 -1453> 2014-08-13 18:06:09.732958 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1452> 2014-08-13 18:06:09.732960 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1451> 2014-08-13 18:06:09.732843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.55 (past last_backfill 0//0//-1) -1450> 2014-08-13 18:06:09.733004 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.337 192.168.1.104:6804/62197 17 ==== pg_info(1 pgs e194497:24.1bb) v4 ==== 748+0+0 (1903430705 0 0) 0x53563c0 con 0x3c55540 -1449> 2014-08-13 18:06:09.733019 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1448> 2014-08-13 18:06:09.733021 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1447> 2014-08-13 18:06:09.733022 7f5c835ef700 20 osd.4 194497 _dispatch 0x53563c0 pg_info(1 pgs e194497:24.1bb) v4 -1446> 2014-08-13 18:06:09.733026 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.732936, event: header_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1445> 2014-08-13 18:06:09.733030 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.732937, event: throttled, request: pg_info(1 pgs e194497:24.1bb) v4 -1444> 2014-08-13 18:06:09.733034 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.732978, event: all_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1443> 2014-08-13 18:06:09.733037 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.733018, event: dispatched, request: pg_info(1 pgs e194497:24.1bb) v4 -1442> 2014-08-13 18:06:09.733040 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.733040, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1bb) v4 -1441> 2014-08-13 18:06:09.733044 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.337 -1440> 2014-08-13 18:06:09.733046 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x53563c0 -1439> 2014-08-13 18:06:09.733049 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.733049, event: started, request: pg_info(1 pgs e194497:24.1bb) v4 -1438> 2014-08-13 18:06:09.733065 7f5c835ef700 5 -- op tracker -- , seq: 669, time: 2014-08-13 18:06:09.733065, event: done, request: pg_info(1 pgs e194497:24.1bb) v4 -1437> 2014-08-13 18:06:09.733072 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1436> 2014-08-13 18:06:09.733074 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1435> 2014-08-13 18:06:09.732996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.55 (past last_backfill 0//0//-1) -1434> 2014-08-13 18:06:09.733251 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.343 192.168.1.104:6802/61960 14 ==== pg_info(1 pgs e194497:24.206) v4 ==== 748+0+0 (3995010539 0 0) 0x588b2c0 con 0x48506e0 -1433> 2014-08-13 18:06:09.733267 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1432> 2014-08-13 18:06:09.733270 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1431> 2014-08-13 18:06:09.733271 7f5c835ef700 20 osd.4 194497 _dispatch 0x588b2c0 pg_info(1 pgs e194497:24.206) v4 -1430> 2014-08-13 18:06:09.733276 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733178, event: header_read, request: pg_info(1 pgs e194497:24.206) v4 -1429> 2014-08-13 18:06:09.733281 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733179, event: throttled, request: pg_info(1 pgs e194497:24.206) v4 -1428> 2014-08-13 18:06:09.733291 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733215, event: all_read, request: pg_info(1 pgs e194497:24.206) v4 -1427> 2014-08-13 18:06:09.733295 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733267, event: dispatched, request: pg_info(1 pgs e194497:24.206) v4 -1426> 2014-08-13 18:06:09.733148 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.55 (past last_backfill 0//0//-1) -1425> 2014-08-13 18:06:09.733299 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733299, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.206) v4 -1424> 2014-08-13 18:06:09.733303 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.206) v4 from osd.343 -1423> 2014-08-13 18:06:09.733306 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588b2c0 -1422> 2014-08-13 18:06:09.733310 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733310, event: started, request: pg_info(1 pgs e194497:24.206) v4 -1421> 2014-08-13 18:06:09.733324 7f5c835ef700 5 -- op tracker -- , seq: 670, time: 2014-08-13 18:06:09.733324, event: done, request: pg_info(1 pgs e194497:24.206) v4 -1420> 2014-08-13 18:06:09.733334 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1419> 2014-08-13 18:06:09.733336 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1418> 2014-08-13 18:06:09.733302 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.55 (past last_backfill 0//0//-1) -1417> 2014-08-13 18:06:09.733465 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.296 192.168.1.109:6806/31286 6 ==== pg_info(1 pgs e194497:24.181) v4 ==== 748+0+0 (2320439669 0 0) 0x5355e80 con 0x4854a40 -1416> 2014-08-13 18:06:09.733475 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1415> 2014-08-13 18:06:09.733476 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1414> 2014-08-13 18:06:09.733478 7f5c835ef700 20 osd.4 194497 _dispatch 0x5355e80 pg_info(1 pgs e194497:24.181) v4 -1413> 2014-08-13 18:06:09.733481 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733416, event: header_read, request: pg_info(1 pgs e194497:24.181) v4 -1412> 2014-08-13 18:06:09.733485 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733418, event: throttled, request: pg_info(1 pgs e194497:24.181) v4 -1411> 2014-08-13 18:06:09.733488 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733454, event: all_read, request: pg_info(1 pgs e194497:24.181) v4 -1410> 2014-08-13 18:06:09.733492 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733474, event: dispatched, request: pg_info(1 pgs e194497:24.181) v4 -1409> 2014-08-13 18:06:09.733496 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733495, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.181) v4 -1408> 2014-08-13 18:06:09.733478 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] state: Activate Finished -1407> 2014-08-13 18:06:09.733499 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.181) v4 from osd.296 -1406> 2014-08-13 18:06:09.733502 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5355e80 -1405> 2014-08-13 18:06:09.733505 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733505, event: started, request: pg_info(1 pgs e194497:24.181) v4 -1404> 2014-08-13 18:06:09.733516 7f5c835ef700 5 -- op tracker -- , seq: 671, time: 2014-08-13 18:06:09.733516, event: done, request: pg_info(1 pgs e194497:24.181) v4 -1403> 2014-08-13 18:06:09.733523 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1402> 2014-08-13 18:06:09.733525 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1401> 2014-08-13 18:06:09.733555 7f5c7bde3700 10 log is not dirty -1400> 2014-08-13 18:06:09.733455 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.55 (past last_backfill 0//0//-1) -1399> 2014-08-13 18:06:09.733596 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 362 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -1398> 2014-08-13 18:06:09.733627 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 362, already have info from that osd, ignoring -1397> 2014-08-13 18:06:09.733647 7f5c7bde3700 10 log is not dirty -1396> 2014-08-13 18:06:09.733653 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) -1395> 2014-08-13 18:06:09.733664 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: got info from osd.304 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) -1394> 2014-08-13 18:06:09.733681 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] exit Started/Stray 1.497738 8 0.000866 -1393> 2014-08-13 18:06:09.733714 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive -1392> 2014-08-13 18:06:09.733727 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] enter Started/ReplicaActive/RepNotRecovering -1391> 2014-08-13 18:06:09.733737 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194487 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive NOTIFY m=61] state: In ReplicaActive, about to call activate -1390> 2014-08-13 18:06:09.733747 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] activate - not complete, missing(61) -1389> 2014-08-13 18:06:09.733606 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.55 (past last_backfill 0//0//-1) -1388> 2014-08-13 18:06:09.733851 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 crt=194422'18567 lcod 0'0 inactive m=46] _activate_committed 194497 telling primary -1387> 2014-08-13 18:06:09.733873 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4857bc0 already has epoch 194497 -1386> 2014-08-13 18:06:09.733876 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.31 192.168.1.103:6831/41157 -- pg_info(1 pgs e194497:24.1e4) v4 -- ?+0 0x588db00 -1385> 2014-08-13 18:06:09.733888 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] flushed -1384> 2014-08-13 18:06:09.733758 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.55 (past last_backfill 0//0//-1) -1383> 2014-08-13 18:06:09.733908 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.55 (past last_backfill 0//0//-1) -1382> 2014-08-13 18:06:09.734061 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.55 (past last_backfill 0//0//-1) -1381> 2014-08-13 18:06:09.734216 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.55 (past last_backfill 0//0//-1) -1380> 2014-08-13 18:06:09.734369 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.55 (past last_backfill 0//0//-1) -1379> 2014-08-13 18:06:09.734625 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] state: Activate Finished -1378> 2014-08-13 18:06:09.734521 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.55 (past last_backfill 0//0//-1) -1377> 2014-08-13 18:06:09.734714 7f5c7bde3700 10 log is not dirty -1376> 2014-08-13 18:06:09.734766 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.31 192.168.1.103:6831/41157 10 ==== pg_info(1 pgs e194497:24.1e4) v4 ==== 748+0+0 (564141691 0 0) 0x588db00 con 0x4857bc0 -1375> 2014-08-13 18:06:09.734778 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1374> 2014-08-13 18:06:09.734780 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1373> 2014-08-13 18:06:09.734781 7f5c835ef700 20 osd.4 194497 _dispatch 0x588db00 pg_info(1 pgs e194497:24.1e4) v4 -1372> 2014-08-13 18:06:09.734785 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734698, event: header_read, request: pg_info(1 pgs e194497:24.1e4) v4 -1371> 2014-08-13 18:06:09.734789 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734700, event: throttled, request: pg_info(1 pgs e194497:24.1e4) v4 -1370> 2014-08-13 18:06:09.734793 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734753, event: all_read, request: pg_info(1 pgs e194497:24.1e4) v4 -1369> 2014-08-13 18:06:09.734796 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734777, event: dispatched, request: pg_info(1 pgs e194497:24.1e4) v4 -1368> 2014-08-13 18:06:09.734800 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734800, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1e4) v4 -1367> 2014-08-13 18:06:09.734803 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e4) v4 from osd.31 -1366> 2014-08-13 18:06:09.734806 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588db00 -1365> 2014-08-13 18:06:09.734809 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734809, event: started, request: pg_info(1 pgs e194497:24.1e4) v4 -1364> 2014-08-13 18:06:09.734673 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.55 (past last_backfill 0//0//-1) -1363> 2014-08-13 18:06:09.734822 7f5c835ef700 5 -- op tracker -- , seq: 672, time: 2014-08-13 18:06:09.734822, event: done, request: pg_info(1 pgs e194497:24.1e4) v4 -1362> 2014-08-13 18:06:09.734829 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1361> 2014-08-13 18:06:09.734831 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1360> 2014-08-13 18:06:09.734735 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 332 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -1359> 2014-08-13 18:06:09.734825 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.55 (past last_backfill 0//0//-1) -1358> 2014-08-13 18:06:09.735037 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 crt=194422'16034 lcod 0'0 inactive m=61] _activate_committed 194497 telling primary -1357> 2014-08-13 18:06:09.735044 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.348 192.168.1.110:6800/41204 5 ==== osd_map(194497..194497 src has 185139..194497) v3 ==== 326+0+0 (752165786 0 0) 0x4cfbf00 con 0x571b700 -1356> 2014-08-13 18:06:09.734913 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 332, already have info from that osd, ignoring -1355> 2014-08-13 18:06:09.735065 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1354> 2014-08-13 18:06:09.735067 7f5c7bde3700 10 log is not dirty -1353> 2014-08-13 18:06:09.735068 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1352> 2014-08-13 18:06:09.735068 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4ab69e0 already has epoch 194497 -1351> 2014-08-13 18:06:09.735069 7f5c835ef700 20 osd.4 194497 _dispatch 0x4cfbf00 osd_map(194497..194497 src has 185139..194497) v3 -1350> 2014-08-13 18:06:09.735076 7f5c835ef700 3 osd.4 194497 handle_osd_map epochs [194497,194497], i have 194497, src has [185139,194497] -1349> 2014-08-13 18:06:09.735071 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.304 192.168.1.101:6808/22782 -- pg_info(1 pgs e194497:24.33a) v4 -- ?+0 0x588e040 -1348> 2014-08-13 18:06:09.735079 7f5c835ef700 10 osd.4 194497 no new maps here, dropping -1347> 2014-08-13 18:06:09.735081 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1346> 2014-08-13 18:06:09.735082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1345> 2014-08-13 18:06:09.735090 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.348 192.168.1.110:6800/41204 6 ==== pg_info(1 pgs e194497:24.1bb) v4 ==== 724+0+0 (3301543155 0 0) 0x5357e00 con 0x571b700 -1344> 2014-08-13 18:06:09.735098 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1343> 2014-08-13 18:06:09.735100 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1342> 2014-08-13 18:06:09.735091 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] flushed -1341> 2014-08-13 18:06:09.735101 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357e00 pg_info(1 pgs e194497:24.1bb) v4 -1340> 2014-08-13 18:06:09.735105 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.734892, event: header_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1339> 2014-08-13 18:06:09.735110 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.734893, event: throttled, request: pg_info(1 pgs e194497:24.1bb) v4 -1338> 2014-08-13 18:06:09.735113 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.734934, event: all_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1337> 2014-08-13 18:06:09.735117 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.735098, event: dispatched, request: pg_info(1 pgs e194497:24.1bb) v4 -1336> 2014-08-13 18:06:09.735102 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 363 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -1335> 2014-08-13 18:06:09.735121 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.735121, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1bb) v4 -1334> 2014-08-13 18:06:09.735124 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.348 -1333> 2014-08-13 18:06:09.734978 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.55 (past last_backfill 0//0//-1) -1332> 2014-08-13 18:06:09.735127 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357e00 -1331> 2014-08-13 18:06:09.735130 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.735130, event: started, request: pg_info(1 pgs e194497:24.1bb) v4 -1330> 2014-08-13 18:06:09.735125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring -1329> 2014-08-13 18:06:09.735138 7f5c7bde3700 10 log is not dirty -1328> 2014-08-13 18:06:09.735146 7f5c835ef700 5 -- op tracker -- , seq: 673, time: 2014-08-13 18:06:09.735146, event: done, request: pg_info(1 pgs e194497:24.1bb) v4 -1327> 2014-08-13 18:06:09.735143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 363 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) -1326> 2014-08-13 18:06:09.735153 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1325> 2014-08-13 18:06:09.735155 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1324> 2014-08-13 18:06:09.735156 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 363, already have info from that osd, ignoring -1323> 2014-08-13 18:06:09.735175 7f5c7bde3700 10 log is not dirty -1322> 2014-08-13 18:06:09.735179 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) -1321> 2014-08-13 18:06:09.735192 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: got info from osd.374 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) -1320> 2014-08-13 18:06:09.735206 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] exit Started/Stray 1.438962 8 0.000623 -1319> 2014-08-13 18:06:09.735221 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive -1318> 2014-08-13 18:06:09.735234 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] enter Started/ReplicaActive/RepNotRecovering -1317> 2014-08-13 18:06:09.735246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194487 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped NOTIFY m=52] state: In ReplicaActive, about to call activate -1316> 2014-08-13 18:06:09.735256 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] activate - not complete, missing(52) -1315> 2014-08-13 18:06:09.735129 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.55 (past last_backfill 0//0//-1) -1314> 2014-08-13 18:06:09.735283 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.55 (past last_backfill 0//0//-1) -1313> 2014-08-13 18:06:09.735434 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.55 (past last_backfill 0//0//-1) -1312> 2014-08-13 18:06:09.735587 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.55 (past last_backfill 0//0//-1) -1311> 2014-08-13 18:06:09.735908 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.304 192.168.1.101:6808/22782 12 ==== pg_info(1 pgs e194497:24.33a) v4 ==== 748+0+0 (1107939381 0 0) 0x588e040 con 0x4ab69e0 -1310> 2014-08-13 18:06:09.735921 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1309> 2014-08-13 18:06:09.735923 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1308> 2014-08-13 18:06:09.735740 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -1307> 2014-08-13 18:06:09.735924 7f5c835ef700 20 osd.4 194497 _dispatch 0x588e040 pg_info(1 pgs e194497:24.33a) v4 -1306> 2014-08-13 18:06:09.735928 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735834, event: header_read, request: pg_info(1 pgs e194497:24.33a) v4 -1305> 2014-08-13 18:06:09.735932 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735835, event: throttled, request: pg_info(1 pgs e194497:24.33a) v4 -1304> 2014-08-13 18:06:09.735936 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735896, event: all_read, request: pg_info(1 pgs e194497:24.33a) v4 -1303> 2014-08-13 18:06:09.735939 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735920, event: dispatched, request: pg_info(1 pgs e194497:24.33a) v4 -1302> 2014-08-13 18:06:09.735943 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735943, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.33a) v4 -1301> 2014-08-13 18:06:09.735946 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.33a) v4 from osd.304 -1300> 2014-08-13 18:06:09.735949 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588e040 -1299> 2014-08-13 18:06:09.735951 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735951, event: started, request: pg_info(1 pgs e194497:24.33a) v4 -1298> 2014-08-13 18:06:09.735963 7f5c835ef700 5 -- op tracker -- , seq: 674, time: 2014-08-13 18:06:09.735963, event: done, request: pg_info(1 pgs e194497:24.33a) v4 -1297> 2014-08-13 18:06:09.735969 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1296> 2014-08-13 18:06:09.735971 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1295> 2014-08-13 18:06:09.736008 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] state: Activate Finished -1294> 2014-08-13 18:06:09.735926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 61 as a source -1293> 2014-08-13 18:06:09.736086 7f5c7bde3700 10 log is not dirty -1292> 2014-08-13 18:06:09.736105 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 24 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) -1291> 2014-08-13 18:06:09.736122 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 24, already have info from that osd, ignoring -1290> 2014-08-13 18:06:09.736133 7f5c7bde3700 10 log is not dirty -1289> 2014-08-13 18:06:09.736137 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) -1288> 2014-08-13 18:06:09.736148 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: got info from osd.6 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) -1287> 2014-08-13 18:06:09.736167 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] exit Started/Stray 1.423764 9 0.000650 -1286> 2014-08-13 18:06:09.736182 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive -1285> 2014-08-13 18:06:09.736193 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] enter Started/ReplicaActive/RepNotRecovering -1284> 2014-08-13 18:06:09.736204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive NOTIFY m=53] state: In ReplicaActive, about to call activate -1283> 2014-08-13 18:06:09.736214 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] activate - not complete, missing(53) -1282> 2014-08-13 18:06:09.736079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 also missing on osd.61 (past last_backfill 0//0//-1) -1281> 2014-08-13 18:06:09.736233 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 also missing on osd.61 (past last_backfill 0//0//-1) -1280> 2014-08-13 18:06:09.736378 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 crt=194422'16449 lcod 0'0 remapped m=52] _activate_committed 194497 telling primary -1279> 2014-08-13 18:06:09.736402 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3cf4360 don't know epoch, doing nothing -1278> 2014-08-13 18:06:09.736405 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.374 192.168.1.102:6828/16129 -- pg_info(1 pgs e194497:24.2ec) v4 -- ?+0 0x588d780 -1277> 2014-08-13 18:06:09.736420 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] flushed -1276> 2014-08-13 18:06:09.736386 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 also missing on osd.61 (past last_backfill 0//0//-1) -1275> 2014-08-13 18:06:09.736539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 also missing on osd.61 (past last_backfill 0//0//-1) -1274> 2014-08-13 18:06:09.736692 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 also missing on osd.61 (past last_backfill 0//0//-1) -1273> 2014-08-13 18:06:09.736844 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 also missing on osd.61 (past last_backfill 0//0//-1) -1272> 2014-08-13 18:06:09.736996 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 also missing on osd.61 (past last_backfill 0//0//-1) -1271> 2014-08-13 18:06:09.737188 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.374 192.168.1.102:6828/16129 10 ==== pg_info(1 pgs e194497:24.2ec) v4 ==== 748+0+0 (1314106506 0 0) 0x588d780 con 0x3cf4360 -1270> 2014-08-13 18:06:09.737210 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1269> 2014-08-13 18:06:09.737215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1268> 2014-08-13 18:06:09.737218 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d780 pg_info(1 pgs e194497:24.2ec) v4 -1267> 2014-08-13 18:06:09.737223 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737092, event: header_read, request: pg_info(1 pgs e194497:24.2ec) v4 -1266> 2014-08-13 18:06:09.737230 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737093, event: throttled, request: pg_info(1 pgs e194497:24.2ec) v4 -1265> 2014-08-13 18:06:09.737233 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737140, event: all_read, request: pg_info(1 pgs e194497:24.2ec) v4 -1264> 2014-08-13 18:06:09.737237 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737209, event: dispatched, request: pg_info(1 pgs e194497:24.2ec) v4 -1263> 2014-08-13 18:06:09.737241 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737241, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.2ec) v4 -1262> 2014-08-13 18:06:09.737245 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2ec) v4 from osd.374 -1261> 2014-08-13 18:06:09.737248 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d780 -1260> 2014-08-13 18:06:09.737251 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737251, event: started, request: pg_info(1 pgs e194497:24.2ec) v4 -1259> 2014-08-13 18:06:09.737267 7f5c835ef700 5 -- op tracker -- , seq: 675, time: 2014-08-13 18:06:09.737266, event: done, request: pg_info(1 pgs e194497:24.2ec) v4 -1258> 2014-08-13 18:06:09.737277 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1257> 2014-08-13 18:06:09.737279 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1256> 2014-08-13 18:06:09.737284 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.331 192.168.1.104:6816/837 10 ==== pg_info(1 pgs e194497:24.1bb) v4 ==== 724+0+0 (3301543155 0 0) 0x5354980 con 0x4b12940 -1255> 2014-08-13 18:06:09.737292 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1254> 2014-08-13 18:06:09.737293 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1253> 2014-08-13 18:06:09.737295 7f5c835ef700 20 osd.4 194497 _dispatch 0x5354980 pg_info(1 pgs e194497:24.1bb) v4 -1252> 2014-08-13 18:06:09.737298 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737101, event: header_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1251> 2014-08-13 18:06:09.737149 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 also missing on osd.61 (past last_backfill 0//0//-1) -1250> 2014-08-13 18:06:09.737301 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737103, event: throttled, request: pg_info(1 pgs e194497:24.1bb) v4 -1249> 2014-08-13 18:06:09.737305 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737159, event: all_read, request: pg_info(1 pgs e194497:24.1bb) v4 -1248> 2014-08-13 18:06:09.737309 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737292, event: dispatched, request: pg_info(1 pgs e194497:24.1bb) v4 -1247> 2014-08-13 18:06:09.737312 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737312, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1bb) v4 -1246> 2014-08-13 18:06:09.737316 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1bb) v4 from osd.331 -1245> 2014-08-13 18:06:09.737318 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5354980 -1244> 2014-08-13 18:06:09.737321 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737321, event: started, request: pg_info(1 pgs e194497:24.1bb) v4 -1243> 2014-08-13 18:06:09.737333 7f5c835ef700 5 -- op tracker -- , seq: 676, time: 2014-08-13 18:06:09.737332, event: done, request: pg_info(1 pgs e194497:24.1bb) v4 -1242> 2014-08-13 18:06:09.737338 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1241> 2014-08-13 18:06:09.737340 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1240> 2014-08-13 18:06:09.737303 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 also missing on osd.61 (past last_backfill 0//0//-1) -1239> 2014-08-13 18:06:09.737456 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 also missing on osd.61 (past last_backfill 0//0//-1) -1238> 2014-08-13 18:06:09.737609 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 also missing on osd.61 (past last_backfill 0//0//-1) -1237> 2014-08-13 18:06:09.737873 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] state: Activate Finished -1236> 2014-08-13 18:06:09.737761 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 also missing on osd.61 (past last_backfill 0//0//-1) -1235> 2014-08-13 18:06:09.737933 7f5c7bde3700 10 log is not dirty -1234> 2014-08-13 18:06:09.737913 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 also missing on osd.61 (past last_backfill 0//0//-1) -1233> 2014-08-13 18:06:09.737950 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 344 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) -1232> 2014-08-13 18:06:09.738066 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 also missing on osd.61 (past last_backfill 0//0//-1) -1231> 2014-08-13 18:06:09.738103 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 344, already have info from that osd, ignoring -1230> 2014-08-13 18:06:09.738232 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'15992 lcod 0'0 inactive m=53] _activate_committed 194497 telling primary -1229> 2014-08-13 18:06:09.738255 7f5c7bde3700 10 log is not dirty -1228> 2014-08-13 18:06:09.738265 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4af98c0 already has epoch 194497 -1227> 2014-08-13 18:06:09.738261 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 331 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) -1226> 2014-08-13 18:06:09.738268 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.6 192.168.1.107:6809/53141 -- pg_info(1 pgs e194497:24.cd) v4 -- ?+0 0x5a69180 -1225> 2014-08-13 18:06:09.738274 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 331, already have info from that osd, ignoring -1224> 2014-08-13 18:06:09.738284 7f5c7bde3700 10 log is not dirty -1223> 2014-08-13 18:06:09.738285 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] flushed -1222> 2014-08-13 18:06:09.738289 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 320 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) -1221> 2014-08-13 18:06:09.738300 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 320, already have info from that osd, ignoring -1220> 2014-08-13 18:06:09.738320 7f5c7bde3700 10 log is not dirty -1219> 2014-08-13 18:06:09.738324 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4496720 don't know epoch, doing nothing -1218> 2014-08-13 18:06:09.738327 7f5c7bde3700 7 osd.4 194497 do_notify osd 45 on 1 PGs -1217> 2014-08-13 18:06:09.738329 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6828/6040 -- pg_notify(24.369(36) epoch 194497) v5 -- ?+0 0x4df0380 con 0x4496720 -1216> 2014-08-13 18:06:09.738223 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 also missing on osd.61 (past last_backfill 0//0//-1) -1215> 2014-08-13 18:06:09.738375 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 also missing on osd.61 (past last_backfill 0//0//-1) -1214> 2014-08-13 18:06:09.738365 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 373 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -1213> 2014-08-13 18:06:09.738528 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 also missing on osd.61 (past last_backfill 0//0//-1) -1212> 2014-08-13 18:06:09.738551 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 373, already have info from that osd, ignoring -1211> 2014-08-13 18:06:09.738724 7f5c7bde3700 10 log is not dirty -1210> 2014-08-13 18:06:09.738730 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 397 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) -1209> 2014-08-13 18:06:09.738742 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 397, already have info from that osd, ignoring -1208> 2014-08-13 18:06:09.738753 7f5c7bde3700 10 log is not dirty -1207> 2014-08-13 18:06:09.738757 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 396 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 lc 0'0 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) -1206> 2014-08-13 18:06:09.738767 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 396, already have info from that osd, ignoring -1205> 2014-08-13 18:06:09.738776 7f5c7bde3700 10 log is not dirty -1204> 2014-08-13 18:06:09.738680 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 also missing on osd.61 (past last_backfill 0//0//-1) -1203> 2014-08-13 18:06:09.738779 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 379 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -1202> 2014-08-13 18:06:09.738833 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 also missing on osd.61 (past last_backfill 0//0//-1) -1201> 2014-08-13 18:06:09.738929 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 379, already have info from that osd, ignoring -1200> 2014-08-13 18:06:09.739074 7f5c7bde3700 10 log is not dirty -1199> 2014-08-13 18:06:09.739060 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.6 192.168.1.107:6809/53141 9 ==== pg_info(1 pgs e194497:24.cd) v4 ==== 748+0+0 (3175862379 0 0) 0x5a69180 con 0x4af98c0 -1198> 2014-08-13 18:06:09.739079 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1197> 2014-08-13 18:06:09.739082 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1196> 2014-08-13 18:06:09.739083 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a69180 pg_info(1 pgs e194497:24.cd) v4 -1195> 2014-08-13 18:06:09.739080 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 0 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -1194> 2014-08-13 18:06:09.739087 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.738961, event: header_read, request: pg_info(1 pgs e194497:24.cd) v4 -1193> 2014-08-13 18:06:09.739093 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 0, already have info from that osd, ignoring -1192> 2014-08-13 18:06:09.739098 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.738962, event: throttled, request: pg_info(1 pgs e194497:24.cd) v4 -1191> 2014-08-13 18:06:09.739103 7f5c7bde3700 10 log is not dirty -1190> 2014-08-13 18:06:09.739102 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.739014, event: all_read, request: pg_info(1 pgs e194497:24.cd) v4 -1189> 2014-08-13 18:06:09.739106 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.739078, event: dispatched, request: pg_info(1 pgs e194497:24.cd) v4 -1188> 2014-08-13 18:06:09.739110 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.739110, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.cd) v4 -1187> 2014-08-13 18:06:09.739114 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.cd) v4 from osd.6 -1186> 2014-08-13 18:06:09.739107 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 352 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -1185> 2014-08-13 18:06:09.739117 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a69180 -1184> 2014-08-13 18:06:09.739121 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.739121, event: started, request: pg_info(1 pgs e194497:24.cd) v4 -1183> 2014-08-13 18:06:09.739118 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 352, already have info from that osd, ignoring -1182> 2014-08-13 18:06:09.739128 7f5c7bde3700 10 log is not dirty -1181> 2014-08-13 18:06:09.738986 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 also missing on osd.61 (past last_backfill 0//0//-1) -1180> 2014-08-13 18:06:09.739136 7f5c835ef700 5 -- op tracker -- , seq: 677, time: 2014-08-13 18:06:09.739136, event: done, request: pg_info(1 pgs e194497:24.cd) v4 -1179> 2014-08-13 18:06:09.739132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 327 notify: (query_epoch:194497, epoch_sent:194497, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) -1178> 2014-08-13 18:06:09.739146 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1177> 2014-08-13 18:06:09.739143 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 327, already have info from that osd, ignoring -1176> 2014-08-13 18:06:09.739153 7f5c7bde3700 10 log is not dirty -1175> 2014-08-13 18:06:09.739148 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1174> 2014-08-13 18:06:09.739157 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 339 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -1173> 2014-08-13 18:06:09.739174 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 339, already have info from that osd, ignoring -1172> 2014-08-13 18:06:09.739184 7f5c7bde3700 10 log is not dirty -1171> 2014-08-13 18:06:09.739188 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 311 notify: (query_epoch:194497, epoch_sent:194497, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -1170> 2014-08-13 18:06:09.739199 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 311, already have info from that osd, ignoring -1169> 2014-08-13 18:06:09.739209 7f5c7bde3700 10 log is not dirty -1168> 2014-08-13 18:06:09.739213 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 293 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194490 n=28 ec=183096 les/c 194490/194389 194495/194495/194495)) -1167> 2014-08-13 18:06:09.739223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 293, already have info from that osd, ignoring -1166> 2014-08-13 18:06:09.739232 7f5c7bde3700 10 log is not dirty -1165> 2014-08-13 18:06:09.739235 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.113( v 194215'528 (0'0,194215'528] local-les=194493 n=27 ec=183096 les/c 194493/194493 194495/194495/194495)) -1164> 2014-08-13 18:06:09.739245 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.113( v 194215'528 (0'0,194215'528] local-les=194496 n=27 ec=183096 les/c 194496/194493 194495/194495/194495) [4,339,45] r=0 lpr=194495 pi=193306-194494/33 crt=191648'383 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring -1163> 2014-08-13 18:06:09.739255 7f5c7bde3700 10 log is not dirty -1162> 2014-08-13 18:06:09.739258 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 314 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) -1161> 2014-08-13 18:06:09.739269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 314, already have info from that osd, ignoring -1160> 2014-08-13 18:06:09.739278 7f5c7bde3700 10 log is not dirty -1159> 2014-08-13 18:06:09.739281 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 48 notify: (query_epoch:194497, epoch_sent:194497, info:22.619( v 194215'511 (0'0,194215'511] local-les=194493 n=29 ec=183096 les/c 194493/194493 194495/194495/194495)) -1158> 2014-08-13 18:06:09.739137 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 also missing on osd.61 (past last_backfill 0//0//-1) -1157> 2014-08-13 18:06:09.739291 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.619( v 194215'511 (0'0,194215'511] local-les=194496 n=29 ec=183096 les/c 194496/194493 194495/194495/194495) [4,65,62] r=0 lpr=194495 pi=193306-194494/37 crt=191727'439 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 48, already have info from that osd, ignoring -1156> 2014-08-13 18:06:09.739301 7f5c7bde3700 10 log is not dirty -1155> 2014-08-13 18:06:09.739305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 380 notify: (query_epoch:194497, epoch_sent:194497, info:22.4df( v 194215'497 (0'0,194215'497] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -1154> 2014-08-13 18:06:09.739315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.4df( v 194215'497 (0'0,194215'497] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,356,357] r=0 lpr=194495 pi=193306-194494/33 crt=191727'433 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 380, already have info from that osd, ignoring -1153> 2014-08-13 18:06:09.739324 7f5c7bde3700 10 log is not dirty -1152> 2014-08-13 18:06:09.739292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 also missing on osd.61 (past last_backfill 0//0//-1) -1151> 2014-08-13 18:06:09.739327 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -1150> 2014-08-13 18:06:09.739445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 also missing on osd.61 (past last_backfill 0//0//-1) -1149> 2014-08-13 18:06:09.739505 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] requeue_ops -1148> 2014-08-13 18:06:09.739674 7f5c7bde3700 10 log is not dirty -1147> 2014-08-13 18:06:09.739598 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 also missing on osd.61 (past last_backfill 0//0//-1) -1146> 2014-08-13 18:06:09.739679 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 350 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -1145> 2014-08-13 18:06:09.739750 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 also missing on osd.61 (past last_backfill 0//0//-1) -1144> 2014-08-13 18:06:09.739871 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 350, already have info from that osd, ignoring -1143> 2014-08-13 18:06:09.740048 7f5c7bde3700 10 log is not dirty -1142> 2014-08-13 18:06:09.739903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 also missing on osd.61 (past last_backfill 0//0//-1) -1141> 2014-08-13 18:06:09.740055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 381 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -1140> 2014-08-13 18:06:09.740067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 381, already have info from that osd, ignoring -1139> 2014-08-13 18:06:09.740076 7f5c7bde3700 10 log is not dirty -1138> 2014-08-13 18:06:09.740080 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -1137> 2014-08-13 18:06:09.740090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -1136> 2014-08-13 18:06:09.740104 7f5c7bde3700 10 log is not dirty -1135> 2014-08-13 18:06:09.740054 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 also missing on osd.61 (past last_backfill 0//0//-1) -1134> 2014-08-13 18:06:09.740108 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 375 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) -1133> 2014-08-13 18:06:09.740210 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 also missing on osd.61 (past last_backfill 0//0//-1) -1132> 2014-08-13 18:06:09.740283 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 375, already have info from that osd, ignoring -1131> 2014-08-13 18:06:09.740451 7f5c7bde3700 10 log is not dirty -1130> 2014-08-13 18:06:09.740456 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 302 notify: (query_epoch:194497, epoch_sent:194497, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) -1129> 2014-08-13 18:06:09.740468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 302, already have info from that osd, ignoring -1128> 2014-08-13 18:06:09.740478 7f5c7bde3700 10 log is not dirty -1127> 2014-08-13 18:06:09.740499 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) -1126> 2014-08-13 18:06:09.740363 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 also missing on osd.61 (past last_backfill 0//0//-1) -1125> 2014-08-13 18:06:09.740512 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: got info from osd.346 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) -1124> 2014-08-13 18:06:09.740527 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] exit Started/Stray 1.446295 9 0.000844 -1123> 2014-08-13 18:06:09.740541 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive -1122> 2014-08-13 18:06:09.740558 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] enter Started/ReplicaActive/RepNotRecovering -1121> 2014-08-13 18:06:09.740569 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194487 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped NOTIFY m=66] state: In ReplicaActive, about to call activate -1120> 2014-08-13 18:06:09.740580 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] activate - not complete, missing(66) -1119> 2014-08-13 18:06:09.740514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 also missing on osd.61 (past last_backfill 0//0//-1) -1118> 2014-08-13 18:06:09.740667 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 also missing on osd.61 (past last_backfill 0//0//-1) -1117> 2014-08-13 18:06:09.740820 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 also missing on osd.61 (past last_backfill 0//0//-1) -1116> 2014-08-13 18:06:09.740972 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 also missing on osd.61 (past last_backfill 0//0//-1) -1115> 2014-08-13 18:06:09.741124 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 also missing on osd.61 (past last_backfill 0//0//-1) -1114> 2014-08-13 18:06:09.741363 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] state: Activate Finished -1113> 2014-08-13 18:06:09.741429 7f5c7bde3700 10 log is not dirty -1112> 2014-08-13 18:06:09.741280 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 also missing on osd.61 (past last_backfill 0//0//-1) -1111> 2014-08-13 18:06:09.741451 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 330 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) -1110> 2014-08-13 18:06:09.741466 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 330, already have info from that osd, ignoring -1109> 2014-08-13 18:06:09.741477 7f5c7bde3700 10 log is not dirty -1108> 2014-08-13 18:06:09.741432 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 also missing on osd.61 (past last_backfill 0//0//-1) -1107> 2014-08-13 18:06:09.741481 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -1106> 2014-08-13 18:06:09.741692 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 crt=194422'15014 lcod 0'0 remapped m=66] _activate_committed 194497 telling primary -1105> 2014-08-13 18:06:09.741720 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4f21600 already has epoch 194497 -1104> 2014-08-13 18:06:09.741723 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.346 192.168.1.104:6820/2308 -- pg_info(1 pgs e194497:24.e2) v4 -- ?+0 0x5a68fc0 -1103> 2014-08-13 18:06:09.741584 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 also missing on osd.61 (past last_backfill 0//0//-1) -1102> 2014-08-13 18:06:09.741740 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] flushed -1101> 2014-08-13 18:06:09.741661 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops -1100> 2014-08-13 18:06:09.741842 7f5c7bde3700 10 log is not dirty -1099> 2014-08-13 18:06:09.741735 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 also missing on osd.61 (past last_backfill 0//0//-1) -1098> 2014-08-13 18:06:09.741848 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 365 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -1097> 2014-08-13 18:06:09.741888 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 also missing on osd.61 (past last_backfill 0//0//-1) -1096> 2014-08-13 18:06:09.742039 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 also missing on osd.61 (past last_backfill 0//0//-1) -1095> 2014-08-13 18:06:09.742025 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 365, already have info from that osd, ignoring -1094> 2014-08-13 18:06:09.742206 7f5c7bde3700 10 log is not dirty -1093> 2014-08-13 18:06:09.742211 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 366 notify: (query_epoch:194497, epoch_sent:194497, info:22.39d( v 194215'480 (0'0,194215'480] local-les=194493 n=23 ec=183096 les/c 194493/194493 194495/194495/194495)) -1092> 2014-08-13 18:06:09.742223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.39d( v 194215'480 (0'0,194215'480] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/194495) [4,373,344] r=0 lpr=194495 pi=193306-194494/35 crt=191727'416 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 366, already have info from that osd, ignoring -1091> 2014-08-13 18:06:09.742232 7f5c7bde3700 10 log is not dirty -1090> 2014-08-13 18:06:09.742236 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 306 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -1089> 2014-08-13 18:06:09.742246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 306, already have info from that osd, ignoring -1088> 2014-08-13 18:06:09.742255 7f5c7bde3700 10 log is not dirty -1087> 2014-08-13 18:06:09.742259 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 63 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -1086> 2014-08-13 18:06:09.742269 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 63, already have info from that osd, ignoring -1085> 2014-08-13 18:06:09.742278 7f5c7bde3700 10 log is not dirty -1084> 2014-08-13 18:06:09.742282 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 349 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -1083> 2014-08-13 18:06:09.742292 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 349, already have info from that osd, ignoring -1082> 2014-08-13 18:06:09.742302 7f5c7bde3700 10 log is not dirty -1081> 2014-08-13 18:06:09.742305 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 382 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 lc 191648'126 (0'0,194215'511] local-les=194412 n=28 ec=183096 les/c 194412/194389 194495/194495/194495)) -1080> 2014-08-13 18:06:09.742315 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 382, already have info from that osd, ignoring -1079> 2014-08-13 18:06:09.742325 7f5c7bde3700 10 log is not dirty -1078> 2014-08-13 18:06:09.742328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 26 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -1077> 2014-08-13 18:06:09.742194 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 also missing on osd.61 (past last_backfill 0//0//-1) -1076> 2014-08-13 18:06:09.742341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 26, already have info from that osd, ignoring -1075> 2014-08-13 18:06:09.742354 7f5c7bde3700 10 log is not dirty -1074> 2014-08-13 18:06:09.742346 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 also missing on osd.61 (past last_backfill 0//0//-1) -1073> 2014-08-13 18:06:09.742359 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 307 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -1072> 2014-08-13 18:06:09.742501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 307, already have info from that osd, ignoring -1071> 2014-08-13 18:06:09.742643 7f5c7bde3700 10 log is not dirty -1070> 2014-08-13 18:06:09.742499 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 also missing on osd.61 (past last_backfill 0//0//-1) -1069> 2014-08-13 18:06:09.742689 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.346 192.168.1.104:6820/2308 9 ==== pg_info(1 pgs e194497:24.e2) v4 ==== 748+0+0 (41218022 0 0) 0x4df39c0 con 0x4f21600 -1068> 2014-08-13 18:06:09.742707 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1067> 2014-08-13 18:06:09.742712 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1066> 2014-08-13 18:06:09.742715 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df39c0 pg_info(1 pgs e194497:24.e2) v4 -1065> 2014-08-13 18:06:09.742719 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742619, event: header_read, request: pg_info(1 pgs e194497:24.e2) v4 -1064> 2014-08-13 18:06:09.742725 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742621, event: throttled, request: pg_info(1 pgs e194497:24.e2) v4 -1063> 2014-08-13 18:06:09.742728 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742675, event: all_read, request: pg_info(1 pgs e194497:24.e2) v4 -1062> 2014-08-13 18:06:09.742731 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742707, event: dispatched, request: pg_info(1 pgs e194497:24.e2) v4 -1061> 2014-08-13 18:06:09.742735 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742735, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.e2) v4 -1060> 2014-08-13 18:06:09.742739 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.e2) v4 from osd.346 -1059> 2014-08-13 18:06:09.742741 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df39c0 -1058> 2014-08-13 18:06:09.742745 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742744, event: started, request: pg_info(1 pgs e194497:24.e2) v4 -1057> 2014-08-13 18:06:09.742760 7f5c835ef700 5 -- op tracker -- , seq: 678, time: 2014-08-13 18:06:09.742760, event: done, request: pg_info(1 pgs e194497:24.e2) v4 -1056> 2014-08-13 18:06:09.742767 7f5c835ef700 10 osd.4 194497 do_waiters -- start -1055> 2014-08-13 18:06:09.742769 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -1054> 2014-08-13 18:06:09.742650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 also missing on osd.61 (past last_backfill 0//0//-1) -1053> 2014-08-13 18:06:09.742647 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 10 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) -1052> 2014-08-13 18:06:09.742803 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 also missing on osd.61 (past last_backfill 0//0//-1) -1051> 2014-08-13 18:06:09.742816 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 10, already have info from that osd, ignoring -1050> 2014-08-13 18:06:09.742981 7f5c7bde3700 10 log is not dirty -1049> 2014-08-13 18:06:09.742965 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 also missing on osd.61 (past last_backfill 0//0//-1) -1048> 2014-08-13 18:06:09.742985 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 301 notify: (query_epoch:194497, epoch_sent:194497, info:24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194493 n=56 ec=192688 les/c 194493/194493 194495/194496/194495)) -1047> 2014-08-13 18:06:09.743118 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 also missing on osd.61 (past last_backfill 0//0//-1) -1046> 2014-08-13 18:06:09.743152 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: Active: got notify from 301, already have info from that osd, ignoring -1045> 2014-08-13 18:06:09.743324 7f5c7bde3700 10 log is not dirty -1044> 2014-08-13 18:06:09.743329 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 58 notify: (query_epoch:194497, epoch_sent:194497, info:22.1d2( v 194215'510 (0'0,194215'510] local-les=194492 n=28 ec=183096 les/c 194492/194492 194495/194495/194495)) -1043> 2014-08-13 18:06:09.743340 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.1d2( v 194215'510 (0'0,194215'510] local-les=194496 n=28 ec=183096 les/c 194496/194492 194495/194495/194495) [4,349,61] r=0 lpr=194495 pi=193306-194494/29 crt=191648'372 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 58, already have info from that osd, ignoring -1042> 2014-08-13 18:06:09.743349 7f5c7bde3700 10 log is not dirty -1041> 2014-08-13 18:06:09.743274 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 also missing on osd.61 (past last_backfill 0//0//-1) -1040> 2014-08-13 18:06:09.743353 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 20 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -1039> 2014-08-13 18:06:09.743427 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 also missing on osd.61 (past last_backfill 0//0//-1) -1038> 2014-08-13 18:06:09.743525 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 20, already have info from that osd, ignoring -1037> 2014-08-13 18:06:09.743694 7f5c7bde3700 10 log is not dirty -1036> 2014-08-13 18:06:09.743700 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 20 notify: (query_epoch:194497, epoch_sent:194497, info:22.99( v 194215'504 (0'0,194215'504] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -1035> 2014-08-13 18:06:09.743711 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.99( v 194215'504 (0'0,194215'504] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,374,31] r=0 lpr=194495 pi=193306-194494/31 crt=191648'365 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 20, already have info from that osd, ignoring -1034> 2014-08-13 18:06:09.743720 7f5c7bde3700 10 log is not dirty -1033> 2014-08-13 18:06:09.743580 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 also missing on osd.61 (past last_backfill 0//0//-1) -1032> 2014-08-13 18:06:09.743731 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 also missing on osd.61 (past last_backfill 0//0//-1) -1031> 2014-08-13 18:06:09.743723 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 291 notify: (query_epoch:194497, epoch_sent:194497, info:24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194493 n=61 ec=192688 les/c 194493/194493 194495/194496/194495)) -1030> 2014-08-13 18:06:09.743884 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -1029> 2014-08-13 18:06:09.743900 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 291, already have info from that osd, ignoring -1028> 2014-08-13 18:06:09.744073 7f5c7bde3700 10 log is not dirty -1027> 2014-08-13 18:06:09.744078 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 30 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -1026> 2014-08-13 18:06:09.744090 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 30, already have info from that osd, ignoring -1025> 2014-08-13 18:06:09.744100 7f5c7bde3700 10 log is not dirty -1024> 2014-08-13 18:06:09.744104 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) -1023> 2014-08-13 18:06:09.744115 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.285 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) -1022> 2014-08-13 18:06:09.744129 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.461966 8 0.000574 -1021> 2014-08-13 18:06:09.744140 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -1020> 2014-08-13 18:06:09.744152 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -1019> 2014-08-13 18:06:09.744167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194487 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate -1018> 2014-08-13 18:06:09.744177 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] activate - not complete, missing(50) -1017> 2014-08-13 18:06:09.744072 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] activate: adding 65 as a source -1016> 2014-08-13 18:06:09.744228 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24 194416'15415 is on osd.65 -1015> 2014-08-13 18:06:09.744382 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24 194370'15388 is on osd.65 -1014> 2014-08-13 18:06:09.744537 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24 194416'15438 is on osd.65 -1013> 2014-08-13 18:06:09.744690 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24 194422'15448 is on osd.65 -1012> 2014-08-13 18:06:09.744857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] state: Activate Finished -1011> 2014-08-13 18:06:09.744919 7f5c7bde3700 10 log is not dirty -1010> 2014-08-13 18:06:09.744936 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 61 notify: (query_epoch:194497, epoch_sent:194497, info:22.3af( v 194215'539 (0'0,194215'539] local-les=194493 n=24 ec=183096 les/c 194493/194493 194495/194495/194495)) -1009> 2014-08-13 18:06:09.744951 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3af( v 194215'539 (0'0,194215'539] local-les=194496 n=24 ec=183096 les/c 194496/194493 194495/194495/194495) [4,44,362] r=0 lpr=194495 pi=193306-194494/37 crt=191727'457 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring -1008> 2014-08-13 18:06:09.744961 7f5c7bde3700 10 log is not dirty -1007> 2014-08-13 18:06:09.744843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24 194422'15466 is on osd.65 -1006> 2014-08-13 18:06:09.744982 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) -1005> 2014-08-13 18:06:09.745005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: got info from osd.364 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) -1004> 2014-08-13 18:06:09.745020 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] exit Started/Stray 1.521242 8 0.000675 -1003> 2014-08-13 18:06:09.745034 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive -1002> 2014-08-13 18:06:09.745046 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] enter Started/ReplicaActive/RepNotRecovering -1001> 2014-08-13 18:06:09.745057 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194487 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped NOTIFY m=48] state: In ReplicaActive, about to call activate -1000> 2014-08-13 18:06:09.745067 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] activate - not complete, missing(48) -999> 2014-08-13 18:06:09.745000 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24 194370'15376 is on osd.65 -998> 2014-08-13 18:06:09.745145 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 crt=194422'17642 lcod 0'0 inactive m=50] _activate_committed 194497 telling primary -997> 2014-08-13 18:06:09.745176 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445e5c0 don't know epoch, doing nothing -996> 2014-08-13 18:06:09.745179 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.285 192.168.1.101:6822/26295 -- pg_info(1 pgs e194497:24.1e6) v4 -- ?+0 0x5a6d080 -995> 2014-08-13 18:06:09.745194 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] flushed -994> 2014-08-13 18:06:09.745153 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24 194422'15468 is on osd.65 -993> 2014-08-13 18:06:09.745309 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24 194370'15382 is on osd.65 -992> 2014-08-13 18:06:09.745462 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24 194380'15406 is on osd.65 -991> 2014-08-13 18:06:09.745616 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24 194422'15462 is on osd.65 -990> 2014-08-13 18:06:09.745792 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] state: Activate Finished -989> 2014-08-13 18:06:09.745860 7f5c7bde3700 10 log is not dirty -988> 2014-08-13 18:06:09.745877 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 340 notify: (query_epoch:194497, epoch_sent:194497, info:24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194487/194258 194459/194496/194496)) -987> 2014-08-13 18:06:09.745893 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: Active: got notify from 340, already have info from that osd, ignoring -986> 2014-08-13 18:06:09.745905 7f5c7bde3700 10 log is not dirty -985> 2014-08-13 18:06:09.745768 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24 194370'15384 is on osd.65 -984> 2014-08-13 18:06:09.746045 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.285 192.168.1.101:6822/26295 10 ==== pg_info(1 pgs e194497:24.1e6) v4 ==== 748+0+0 (2894265652 0 0) 0x5a6d080 con 0x445e5c0 -983> 2014-08-13 18:06:09.746063 7f5c835ef700 10 osd.4 194497 do_waiters -- start -982> 2014-08-13 18:06:09.746066 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -981> 2014-08-13 18:06:09.746067 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d080 pg_info(1 pgs e194497:24.1e6) v4 -980> 2014-08-13 18:06:09.745920 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24 194416'15424 is on osd.65 -979> 2014-08-13 18:06:09.746071 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.745971, event: header_read, request: pg_info(1 pgs e194497:24.1e6) v4 -978> 2014-08-13 18:06:09.746076 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.745972, event: throttled, request: pg_info(1 pgs e194497:24.1e6) v4 -977> 2014-08-13 18:06:09.745910 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -976> 2014-08-13 18:06:09.746080 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.746030, event: all_read, request: pg_info(1 pgs e194497:24.1e6) v4 -975> 2014-08-13 18:06:09.746083 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.746062, event: dispatched, request: pg_info(1 pgs e194497:24.1e6) v4 -974> 2014-08-13 18:06:09.746088 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.746087, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1e6) v4 -973> 2014-08-13 18:06:09.746091 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1e6) v4 from osd.285 -972> 2014-08-13 18:06:09.746095 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d080 -971> 2014-08-13 18:06:09.746098 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.746097, event: started, request: pg_info(1 pgs e194497:24.1e6) v4 -970> 2014-08-13 18:06:09.746088 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 crt=194422'17074 lcod 0'0 remapped m=48] _activate_committed 194497 telling primary -969> 2014-08-13 18:06:09.746112 7f5c835ef700 5 -- op tracker -- , seq: 679, time: 2014-08-13 18:06:09.746111, event: done, request: pg_info(1 pgs e194497:24.1e6) v4 -968> 2014-08-13 18:06:09.746119 7f5c835ef700 10 osd.4 194497 do_waiters -- start -967> 2014-08-13 18:06:09.746119 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445ad60 don't know epoch, doing nothing -966> 2014-08-13 18:06:09.746121 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -965> 2014-08-13 18:06:09.746122 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.364 192.168.1.114:6826/52317 -- pg_info(1 pgs e194497:24.1de) v4 -- ?+0 0x5a6bd40 -964> 2014-08-13 18:06:09.746140 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] flushed -963> 2014-08-13 18:06:09.746071 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24 194416'15444 is on osd.65 -962> 2014-08-13 18:06:09.746083 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] requeue_ops -961> 2014-08-13 18:06:09.746260 7f5c7bde3700 10 log is not dirty -960> 2014-08-13 18:06:09.746265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) -959> 2014-08-13 18:06:09.746278 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: got info from osd.45 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) -958> 2014-08-13 18:06:09.746291 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] exit Started/Stray 1.448789 8 0.000563 -957> 2014-08-13 18:06:09.746302 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive -956> 2014-08-13 18:06:09.746313 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] enter Started/ReplicaActive/RepNotRecovering -955> 2014-08-13 18:06:09.746323 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194487 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive NOTIFY m=50] state: In ReplicaActive, about to call activate -954> 2014-08-13 18:06:09.746332 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] activate - not complete, missing(50) -953> 2014-08-13 18:06:09.746228 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24 194370'15368 is on osd.65 -952> 2014-08-13 18:06:09.746381 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24 194370'15380 is on osd.65 -951> 2014-08-13 18:06:09.746534 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24 194370'15374 is on osd.65 -950> 2014-08-13 18:06:09.746686 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24 194422'15454 is on osd.65 -949> 2014-08-13 18:06:09.746839 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24 194370'15390 is on osd.65 -948> 2014-08-13 18:06:09.747086 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.364 192.168.1.114:6826/52317 8 ==== pg_info(1 pgs e194497:24.1de) v4 ==== 748+0+0 (1377488550 0 0) 0x4df5940 con 0x445ad60 -947> 2014-08-13 18:06:09.747099 7f5c835ef700 10 osd.4 194497 do_waiters -- start -946> 2014-08-13 18:06:09.747101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -945> 2014-08-13 18:06:09.747103 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df5940 pg_info(1 pgs e194497:24.1de) v4 -944> 2014-08-13 18:06:09.747106 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747023, event: header_read, request: pg_info(1 pgs e194497:24.1de) v4 -943> 2014-08-13 18:06:09.747110 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747024, event: throttled, request: pg_info(1 pgs e194497:24.1de) v4 -942> 2014-08-13 18:06:09.747114 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747074, event: all_read, request: pg_info(1 pgs e194497:24.1de) v4 -941> 2014-08-13 18:06:09.747117 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747099, event: dispatched, request: pg_info(1 pgs e194497:24.1de) v4 -940> 2014-08-13 18:06:09.747121 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747121, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1de) v4 -939> 2014-08-13 18:06:09.747117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] state: Activate Finished -938> 2014-08-13 18:06:09.747125 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1de) v4 from osd.364 -937> 2014-08-13 18:06:09.747127 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df5940 -936> 2014-08-13 18:06:09.747130 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747130, event: started, request: pg_info(1 pgs e194497:24.1de) v4 -935> 2014-08-13 18:06:09.746992 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24 194370'15370 is on osd.65 -934> 2014-08-13 18:06:09.747143 7f5c835ef700 5 -- op tracker -- , seq: 680, time: 2014-08-13 18:06:09.747143, event: done, request: pg_info(1 pgs e194497:24.1de) v4 -933> 2014-08-13 18:06:09.747150 7f5c835ef700 10 osd.4 194497 do_waiters -- start -932> 2014-08-13 18:06:09.747152 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -931> 2014-08-13 18:06:09.747183 7f5c7bde3700 10 log is not dirty -930> 2014-08-13 18:06:09.747198 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -929> 2014-08-13 18:06:09.747213 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] requeue_ops -928> 2014-08-13 18:06:09.747224 7f5c7bde3700 10 log is not dirty -927> 2014-08-13 18:06:09.747144 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24 194416'15420 is on osd.65 -926> 2014-08-13 18:06:09.747228 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194493/194466 194495/194496/194496)) -925> 2014-08-13 18:06:09.747427 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 crt=194422'13576 lcod 0'0 inactive m=50] _activate_committed 194497 telling primary -924> 2014-08-13 18:06:09.747301 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24 194416'15436 is on osd.65 -923> 2014-08-13 18:06:09.747453 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4496720 don't know epoch, doing nothing -922> 2014-08-13 18:06:09.747456 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.45 192.168.1.112:6828/6040 -- pg_info(1 pgs e194497:24.369) v4 -- ?+0 0x5a6b800 -921> 2014-08-13 18:06:09.747470 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] flushed -920> 2014-08-13 18:06:09.747373 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 55, already have info from that osd, ignoring -919> 2014-08-13 18:06:09.747514 7f5c7bde3700 10 log is not dirty -918> 2014-08-13 18:06:09.747453 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24 194377'15398 is on osd.65 -917> 2014-08-13 18:06:09.747519 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 337 notify: (query_epoch:194497, epoch_sent:194497, info:24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194493 n=50 ec=192688 les/c 194493/194493 194495/194496/194495)) -916> 2014-08-13 18:06:09.747606 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24 194416'15432 is on osd.65 -915> 2014-08-13 18:06:09.747667 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: Active: got notify from 337, already have info from that osd, ignoring -914> 2014-08-13 18:06:09.747813 7f5c7bde3700 10 log is not dirty -913> 2014-08-13 18:06:09.747818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 15 info: 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) -912> 2014-08-13 18:06:09.747830 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: got info from osd.15 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) -911> 2014-08-13 18:06:09.747844 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.467832 8 0.000669 -910> 2014-08-13 18:06:09.747857 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive -909> 2014-08-13 18:06:09.747869 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering -908> 2014-08-13 18:06:09.747880 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194487 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped NOTIFY m=56] state: In ReplicaActive, about to call activate -907> 2014-08-13 18:06:09.747890 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] activate - not complete, missing(56) -906> 2014-08-13 18:06:09.747759 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24 194416'15426 is on osd.65 -905> 2014-08-13 18:06:09.747911 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24 194393'15414 is on osd.65 -904> 2014-08-13 18:06:09.748063 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24 194416'15422 is on osd.65 -903> 2014-08-13 18:06:09.748219 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24 194377'15402 is on osd.65 -902> 2014-08-13 18:06:09.748426 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.45 192.168.1.112:6828/6040 14 ==== pg_info(1 pgs e194497:24.369) v4 ==== 748+0+0 (679363464 0 0) 0x5a6b800 con 0x4496720 -901> 2014-08-13 18:06:09.748441 7f5c835ef700 10 osd.4 194497 do_waiters -- start -900> 2014-08-13 18:06:09.748443 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -899> 2014-08-13 18:06:09.748444 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b800 pg_info(1 pgs e194497:24.369) v4 -898> 2014-08-13 18:06:09.748448 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748357, event: header_read, request: pg_info(1 pgs e194497:24.369) v4 -897> 2014-08-13 18:06:09.748452 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748358, event: throttled, request: pg_info(1 pgs e194497:24.369) v4 -896> 2014-08-13 18:06:09.748455 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748414, event: all_read, request: pg_info(1 pgs e194497:24.369) v4 -895> 2014-08-13 18:06:09.748459 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748440, event: dispatched, request: pg_info(1 pgs e194497:24.369) v4 -894> 2014-08-13 18:06:09.748463 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748462, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.369) v4 -893> 2014-08-13 18:06:09.748466 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.369) v4 from osd.45 -892> 2014-08-13 18:06:09.748469 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b800 -891> 2014-08-13 18:06:09.748471 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748471, event: started, request: pg_info(1 pgs e194497:24.369) v4 -890> 2014-08-13 18:06:09.748484 7f5c835ef700 5 -- op tracker -- , seq: 681, time: 2014-08-13 18:06:09.748483, event: done, request: pg_info(1 pgs e194497:24.369) v4 -889> 2014-08-13 18:06:09.748490 7f5c835ef700 10 osd.4 194497 do_waiters -- start -888> 2014-08-13 18:06:09.748492 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -887> 2014-08-13 18:06:09.748372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24 194370'15372 is on osd.65 -886> 2014-08-13 18:06:09.748524 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24 194416'15430 is on osd.65 -885> 2014-08-13 18:06:09.748677 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24 194377'15400 is on osd.65 -884> 2014-08-13 18:06:09.748829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24 194380'15408 is on osd.65 -883> 2014-08-13 18:06:09.748982 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24 194380'15410 is on osd.65 -882> 2014-08-13 18:06:09.749243 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] state: Activate Finished -881> 2014-08-13 18:06:09.749135 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24 194416'15446 is on osd.65 -880> 2014-08-13 18:06:09.749310 7f5c7bde3700 10 log is not dirty -879> 2014-08-13 18:06:09.749291 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24 194422'15460 is on osd.65 -878> 2014-08-13 18:06:09.749328 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 362 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -877> 2014-08-13 18:06:09.749444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24 194377'15396 is on osd.65 -876> 2014-08-13 18:06:09.749576 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 crt=194422'11806 lcod 0'0 remapped m=56] _activate_committed 194497 telling primary -875> 2014-08-13 18:06:09.749606 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 -874> 2014-08-13 18:06:09.749609 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.15 192.168.1.107:6812/53408 -- pg_info(1 pgs e194497:24.20e) v4 -- ?+0 0x5a6c0c0 -873> 2014-08-13 18:06:09.749627 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] flushed -872> 2014-08-13 18:06:09.749505 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.362 activated and committed -871> 2014-08-13 18:06:09.749676 7f5c7bde3700 10 log is not dirty -870> 2014-08-13 18:06:09.749597 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24 194422'15452 is on osd.65 -869> 2014-08-13 18:06:09.749683 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194493 n=59 ec=192688 les/c 194493/194493 194495/194496/194495)) -868> 2014-08-13 18:06:09.749750 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24 194422'15464 is on osd.65 -867> 2014-08-13 18:06:09.749855 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: Active: got notify from 303, already have info from that osd, ignoring -866> 2014-08-13 18:06:09.750025 7f5c7bde3700 10 log is not dirty -865> 2014-08-13 18:06:09.749903 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24 194416'15442 is on osd.65 -864> 2014-08-13 18:06:09.750030 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -863> 2014-08-13 18:06:09.750062 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24 194370'15386 is on osd.65 -862> 2014-08-13 18:06:09.750177 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] requeue_ops -861> 2014-08-13 18:06:09.750319 7f5c7bde3700 10 log is not dirty -860> 2014-08-13 18:06:09.750218 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24 194377'15394 is on osd.65 -859> 2014-08-13 18:06:09.750387 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.55 192.168.1.112:6834/6966 16 ==== pg_info(1 pgs e194497:24.136) v4 ==== 724+0+0 (3582957228 0 0) 0x5a6b2c0 con 0x3c523c0 -858> 2014-08-13 18:06:09.750403 7f5c835ef700 10 osd.4 194497 do_waiters -- start -857> 2014-08-13 18:06:09.750406 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -856> 2014-08-13 18:06:09.750407 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6b2c0 pg_info(1 pgs e194497:24.136) v4 -855> 2014-08-13 18:06:09.750410 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750319, event: header_read, request: pg_info(1 pgs e194497:24.136) v4 -854> 2014-08-13 18:06:09.750415 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750320, event: throttled, request: pg_info(1 pgs e194497:24.136) v4 -853> 2014-08-13 18:06:09.750419 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750368, event: all_read, request: pg_info(1 pgs e194497:24.136) v4 -852> 2014-08-13 18:06:09.750422 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750402, event: dispatched, request: pg_info(1 pgs e194497:24.136) v4 -851> 2014-08-13 18:06:09.750425 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750425, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.136) v4 -850> 2014-08-13 18:06:09.750429 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.55 -849> 2014-08-13 18:06:09.750431 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6b2c0 -848> 2014-08-13 18:06:09.750434 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.750434, event: started, request: pg_info(1 pgs e194497:24.136) v4 -847> 2014-08-13 18:06:09.750324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 320 notify: (query_epoch:194497, epoch_sent:194497, info:24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194493 n=42 ec=192688 les/c 194493/194466 194495/194496/194496)) -846> 2014-08-13 18:06:09.750372 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24 194370'15378 is on osd.65 -845> 2014-08-13 18:06:09.750463 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: Active: got notify from 320, already have info from that osd, ignoring -844> 2014-08-13 18:06:09.750604 7f5c7bde3700 10 log is not dirty -843> 2014-08-13 18:06:09.750525 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24 194422'15470 is on osd.65 -842> 2014-08-13 18:06:09.750609 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 367 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -841> 2014-08-13 18:06:09.750678 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24 194416'15440 is on osd.65 -840> 2014-08-13 18:06:09.750787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.367 activated and committed -839> 2014-08-13 18:06:09.750963 7f5c7bde3700 10 log is not dirty -838> 2014-08-13 18:06:09.750969 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 373 notify: (query_epoch:194497, epoch_sent:194497, info:22.3d9( v 194215'511 (0'0,194215'511] local-les=194493 n=28 ec=183096 les/c 194493/194389 194495/194495/194495)) -837> 2014-08-13 18:06:09.750831 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24 194416'15434 is on osd.65 -836> 2014-08-13 18:06:09.750981 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3d9( v 194215'511 (0'0,194215'511] local-les=194496 n=28 ec=183096 les/c 194496/194389 194495/194495/194495) [4,338,376] r=0 lpr=194495 pi=193306-194494/53 crt=191727'441 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 373, already have info from that osd, ignoring -835> 2014-08-13 18:06:09.750991 7f5c7bde3700 10 log is not dirty -834> 2014-08-13 18:06:09.750995 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 376 notify: (query_epoch:194497, epoch_sent:194497, info:22.48e( v 194215'543 (0'0,194215'543] local-les=194493 n=40 ec=183096 les/c 194493/194493 194495/194495/194495)) -833> 2014-08-13 18:06:09.751005 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.48e( v 194215'543 (0'0,194215'543] local-les=194496 n=40 ec=183096 les/c 194496/194493 194495/194495/194495) [4,289,341] r=0 lpr=194495 pi=192978-194494/35 crt=191727'473 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 376, already have info from that osd, ignoring -832> 2014-08-13 18:06:09.751014 7f5c7bde3700 10 log is not dirty -831> 2014-08-13 18:06:09.751018 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 368 notify: (query_epoch:194497, epoch_sent:194497, info:22.22c( v 194215'514 (0'0,194215'514] local-les=194492 n=38 ec=183096 les/c 194492/194492 194495/194495/194495)) -830> 2014-08-13 18:06:09.751028 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.22c( v 194215'514 (0'0,194215'514] local-les=194496 n=38 ec=183096 les/c 194496/194492 194495/194495/194495) [4,301,324] r=0 lpr=194495 pi=193306-194494/29 crt=191727'446 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring -829> 2014-08-13 18:06:09.751038 7f5c7bde3700 10 log is not dirty -828> 2014-08-13 18:06:09.750983 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24 194422'15450 is on osd.65 -827> 2014-08-13 18:06:09.751041 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 350 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) -826> 2014-08-13 18:06:09.751136 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24 194370'15392 is on osd.65 -825> 2014-08-13 18:06:09.751222 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.350 activated and committed -824> 2014-08-13 18:06:09.751399 7f5c7bde3700 10 log is not dirty -823> 2014-08-13 18:06:09.751404 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.f3( v 194215'506 (0'0,194215'506] local-les=194492 n=29 ec=183096 les/c 194492/194492 194495/194495/194495)) -822> 2014-08-13 18:06:09.751415 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.f3( v 194215'506 (0'0,194215'506] local-les=194496 n=29 ec=183096 les/c 194496/194492 194495/194495/194495) [4,312,285] r=0 lpr=194495 pi=193306-194494/27 crt=191648'343 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -821> 2014-08-13 18:06:09.751424 7f5c7bde3700 10 log is not dirty -820> 2014-08-13 18:06:09.751292 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24 194416'15428 is on osd.65 -819> 2014-08-13 18:06:09.751444 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24 194422'15458 is on osd.65 -818> 2014-08-13 18:06:09.751428 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 375 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) -817> 2014-08-13 18:06:09.751599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24 194378'15404 is on osd.65 -816> 2014-08-13 18:06:09.751603 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.375 activated and committed -815> 2014-08-13 18:06:09.751770 7f5c7bde3700 10 log is not dirty -814> 2014-08-13 18:06:09.751774 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) -813> 2014-08-13 18:06:09.751787 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: got info from osd.330 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) -812> 2014-08-13 18:06:09.751801 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] exit Started/Stray 1.470126 8 0.000649 -811> 2014-08-13 18:06:09.751813 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive -810> 2014-08-13 18:06:09.751825 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] enter Started/ReplicaActive/RepNotRecovering -809> 2014-08-13 18:06:09.751835 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194487 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped NOTIFY m=64] state: In ReplicaActive, about to call activate -808> 2014-08-13 18:06:09.751846 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] activate - not complete, missing(64) -807> 2014-08-13 18:06:09.751753 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing 7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24 194422'15456 is on osd.65 -806> 2014-08-13 18:06:09.751906 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] search_for_missing f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24 194416'15418 is on osd.65 -805> 2014-08-13 18:06:09.752059 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] needs_recovery_map missing {136/hit_set_24.136_archive_2014-08-12 20:49:27.349351_2014-08-13 15:28:40.333934/head/.ceph-internal/24=194416'15415,3e841136/benchmark_data_storage0111-ib.pocstor_12495_object10343/head//24=194370'15388,9dcf1136/benchmark_data_storage0111-ib.pocstor_19103_object11990/head//24=194416'15438,5d702136/benchmark_data_storage0111-ib.pocstor_41231_object955/head//24=194422'15448,e0732136/benchmark_data_storage0111-ib.pocstor_41231_object8537/head//24=194422'15466,8ebb5136/benchmark_data_storage0111-ib.pocstor_12495_object5412/head//24=194370'15376,11118136/benchmark_data_storage0111-ib.pocstor_41231_object8748/head//24=194422'15468,ecb28136/benchmark_data_storage0111-ib.pocstor_12495_object6811/head//24=194370'15382,d91a136/benchmark_data_storage0111-ib.pocstor_25065_object5070/head//24=194380'15406,1bd6a136/benchmark_data_storage0111-ib.pocstor_41231_object4196/head//24=194422'15462,26bae136/benchmark_data_storage0111-ib.pocstor_12495_object8482/head//24=194370'15384,5e64f136/benchmark_data_storage0111-ib.pocstor_19103_object6542/head//24=194416'15424,6a4e0536/benchmark_data_storage0111-ib.pocstor_19103_object15418/head//24=194416'15444,d8681536/benchmark_data_storage0111-ib.pocstor_12495_object141/head//24=194370'15368,2d9a4536/benchmark_data_storage0111-ib.pocstor_12495_object5472/head//24=194370'15380,488f5536/benchmark_data_storage0111-ib.pocstor_12495_object2479/head//24=194370'15374,69166536/benchmark_data_storage0111-ib.pocstor_41231_object2679/head//24=194422'15454,cc9b8536/benchmark_data_storage0111-ib.pocstor_12495_object10563/head//24=194370'15390,e8a2a536/benchmark_data_storage0111-ib.pocstor_12495_object1004/head//24=194370'15370,aa15b536/benchmark_data_storage0111-ib.pocstor_19103_object4580/head//24=194416'15420,b828b536/benchmark_data_storage0111-ib.pocstor_19103_object11581/head//24=194416'15436,2fd8b536/benchmark_data_storage0111-ib.pocstor_25065_object2396/head//24=194377'15398,4845c536/benchmark_data_storage0111-ib.pocstor_19103_object10206/head//24=194416'15432,8b3ac536/benchmark_data_storage0111-ib.pocstor_19103_object7232/head//24=194416'15426,1801f536/benchmark_data_storage0111-ib.pocstor_32482_object396/head//24=194393'15414,870ef536/benchmark_data_storage0111-ib.pocstor_19103_object6063/head//24=194416'15422,4f430936/benchmark_data_storage0111-ib.pocstor_25065_object4728/head//24=194377'15402,f33e4936/benchmark_data_storage0111-ib.pocstor_12495_object2452/head//24=194370'15372,af105936/benchmark_data_storage0111-ib.pocstor_19103_object9253/head//24=194416'15430,2c306936/benchmark_data_storage0111-ib.pocstor_25065_object4205/head//24=194377'15400,444d6936/benchmark_data_storage0111-ib.pocstor_25065_object5074/head//24=194380'15408,1e207936/benchmark_data_storage0111-ib.pocstor_25065_object5112/head//24=194380'15410,a1b68936/benchmark_data_storage0111-ib.pocstor_19103_object16907/head//24=194416'15446,e0a29936/benchmark_data_storage0111-ib.pocstor_41231_object4191/head//24=194422'15460,5a52a936/benchmark_data_storage0111-ib.pocstor_25065_object779/head//24=194377'15396,50b5b936/benchmark_data_storage0111-ib.pocstor_41231_object2139/head//24=194422'15452,1433c936/benchmark_data_storage0111-ib.pocstor_41231_object7425/head//24=194422'15464,513bc936/benchmark_data_storage0111-ib.pocstor_19103_object15310/head//24=194416'15442,8182e936/benchmark_data_storage0111-ib.pocstor_12495_object9653/head//24=194370'15386,e0a2e936/benchmark_data_storage0111-ib.pocstor_25065_object756/head//24=194377'15394,3453e936/benchmark_data_storage0111-ib.pocstor_12495_object5428/head//24=194370'15378,95b5e936/benchmark_data_storage0111-ib.pocstor_41231_object9526/head//24=194422'15470,fb692d36/benchmark_data_storage0111-ib.pocstor_19103_object13586/head//24=194416'15440,19cb3d36/benchmark_data_storage0111-ib.pocstor_19103_object11324/head//24=194416'15434,88d64d36/benchmark_data_storage0111-ib.pocstor_41231_object1111/head//24=194422'15450,c3b05d36/benchmark_data_storage0111-ib.pocstor_12495_object11653/head//24=194370'15392,897a6d36/benchmark_data_storage0111-ib.pocstor_19103_object8299/head//24=194416'15428,94027d36/benchmark_data_storage0111-ib.pocstor_41231_object3789/head//24=194422'15458,d1ef9d36/benchmark_data_storage0111-ib.pocstor_25065_object4859/head//24=194378'15404,7394ad36/benchmark_data_storage0111-ib.pocstor_41231_object3709/head//24=194422'15456,f8ecad36/benchmark_data_storage0111-ib.pocstor_19103_object2540/head//24=194416'15418} -804> 2014-08-13 18:06:09.752249 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound -803> 2014-08-13 18:06:09.752401 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _calc_past_interval_range: already have past intervals back to 194492 -802> 2014-08-13 18:06:09.752602 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] state: Activate Finished -801> 2014-08-13 18:06:09.752673 7f5c7bde3700 10 log is not dirty -800> 2014-08-13 18:06:09.752554 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] build_might_have_unfound: built 50,55,61,65,356 -799> 2014-08-13 18:06:09.752707 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 365 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 lc 0'0 (0'0,194215'519] local-les=194369 n=32 ec=183096 les/c 194369/194304 194495/194495/194495)) -798> 2014-08-13 18:06:09.752724 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 365, already have info from that osd, ignoring -797> 2014-08-13 18:06:09.752734 7f5c7bde3700 10 log is not dirty -796> 2014-08-13 18:06:09.752707 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Activate Finished -795> 2014-08-13 18:06:09.752738 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 378 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -794> 2014-08-13 18:06:09.752902 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 crt=194422'15537 lcod 0'0 remapped m=64] _activate_committed 194497 telling primary -793> 2014-08-13 18:06:09.752944 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4f20dc0 don't know epoch, doing nothing -792> 2014-08-13 18:06:09.752947 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.330 192.168.1.106:6802/4876 -- pg_info(1 pgs e194497:24.3f7) v4 -- ?+0 0x5357540 -791> 2014-08-13 18:06:09.752967 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] flushed -790> 2014-08-13 18:06:09.752861 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] enter Started/Primary/Active/Activating -789> 2014-08-13 18:06:09.752918 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.378 activated and committed -788> 2014-08-13 18:06:09.753094 7f5c7bde3700 10 log is not dirty -787> 2014-08-13 18:06:09.753099 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 26 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) -786> 2014-08-13 18:06:09.753111 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.26 activated and committed -785> 2014-08-13 18:06:09.753128 7f5c7bde3700 10 log is not dirty -784> 2014-08-13 18:06:09.753015 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -783> 2014-08-13 18:06:09.753132 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 10 info: 24.30b( v 194422'18660 (192718'15659,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) -782> 2014-08-13 18:06:09.753168 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -781> 2014-08-13 18:06:09.753371 7f5c7c7e4700 10 log is not dirty -780> 2014-08-13 18:06:09.753414 7f5c835ef700 5 -- op tracker -- , seq: 682, time: 2014-08-13 18:06:09.753414, event: done, request: pg_info(1 pgs e194497:24.136) v4 -779> 2014-08-13 18:06:09.753428 7f5c835ef700 10 osd.4 194497 do_waiters -- start -778> 2014-08-13 18:06:09.753432 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -777> 2014-08-13 18:06:09.753418 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [337,20,4]/[337,20,4] -- 337/337 -776> 2014-08-13 18:06:09.753436 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -775> 2014-08-13 18:06:09.753437 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 12 ==== pg_info(1 pgs e194497:24.20e) v4 ==== 748+0+0 (3321811378 0 0) 0x5a6c0c0 con 0x4854620 -774> 2014-08-13 18:06:09.753446 7f5c835ef700 10 osd.4 194497 do_waiters -- start -773> 2014-08-13 18:06:09.753447 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -772> 2014-08-13 18:06:09.753441 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap -771> 2014-08-13 18:06:09.753449 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c0c0 pg_info(1 pgs e194497:24.20e) v4 -770> 2014-08-13 18:06:09.753452 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.750339, event: header_read, request: pg_info(1 pgs e194497:24.20e) v4 -769> 2014-08-13 18:06:09.753457 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.750341, event: throttled, request: pg_info(1 pgs e194497:24.20e) v4 -768> 2014-08-13 18:06:09.753452 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down -767> 2014-08-13 18:06:09.753460 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.750389, event: all_read, request: pg_info(1 pgs e194497:24.20e) v4 -766> 2014-08-13 18:06:09.753463 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.753445, event: dispatched, request: pg_info(1 pgs e194497:24.20e) v4 -765> 2014-08-13 18:06:09.753306 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.10 activated and committed -764> 2014-08-13 18:06:09.753463 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map -763> 2014-08-13 18:06:09.753475 7f5c7bde3700 10 log is not dirty -762> 2014-08-13 18:06:09.753467 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.753467, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.20e) v4 -761> 2014-08-13 18:06:09.753477 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.20e) v4 from osd.15 -760> 2014-08-13 18:06:09.753473 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters -759> 2014-08-13 18:06:09.753480 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c0c0 -758> 2014-08-13 18:06:09.753484 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.753483, event: started, request: pg_info(1 pgs e194497:24.20e) v4 -757> 2014-08-13 18:06:09.753483 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -756> 2014-08-13 18:06:09.753497 7f5c835ef700 5 -- op tracker -- , seq: 683, time: 2014-08-13 18:06:09.753497, event: done, request: pg_info(1 pgs e194497:24.20e) v4 -755> 2014-08-13 18:06:09.753493 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.4fe( v 194215'495 (0'0,194215'495] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/192943) [337,20,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -754> 2014-08-13 18:06:09.753503 7f5c835ef700 10 osd.4 194497 do_waiters -- start -753> 2014-08-13 18:06:09.753505 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -752> 2014-08-13 18:06:09.753512 7f5c7c7e4700 10 log is not dirty -751> 2014-08-13 18:06:09.753508 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.61 192.168.1.111:6840/16566 16 ==== pg_info(1 pgs e194497:24.136) v4 ==== 724+0+0 (3582957228 0 0) 0x588c280 con 0x445b180 -750> 2014-08-13 18:06:09.753515 7f5c835ef700 10 osd.4 194497 do_waiters -- start -749> 2014-08-13 18:06:09.753517 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -748> 2014-08-13 18:06:09.753518 7f5c835ef700 20 osd.4 194497 _dispatch 0x588c280 pg_info(1 pgs e194497:24.136) v4 -747> 2014-08-13 18:06:09.753521 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753039, event: header_read, request: pg_info(1 pgs e194497:24.136) v4 -746> 2014-08-13 18:06:09.753525 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753040, event: throttled, request: pg_info(1 pgs e194497:24.136) v4 -745> 2014-08-13 18:06:09.753528 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753085, event: all_read, request: pg_info(1 pgs e194497:24.136) v4 -744> 2014-08-13 18:06:09.753532 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753515, event: dispatched, request: pg_info(1 pgs e194497:24.136) v4 -743> 2014-08-13 18:06:09.753526 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_advance_map [371,4,21]/[371,4,21] -- 371/371 -742> 2014-08-13 18:06:09.753535 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753535, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.136) v4 -741> 2014-08-13 18:06:09.753539 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -740> 2014-08-13 18:06:09.753539 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.61 -739> 2014-08-13 18:06:09.753542 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588c280 -738> 2014-08-13 18:06:09.753544 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753544, event: started, request: pg_info(1 pgs e194497:24.136) v4 -737> 2014-08-13 18:06:09.753543 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] state: Started advmap -736> 2014-08-13 18:06:09.753558 7f5c835ef700 5 -- op tracker -- , seq: 684, time: 2014-08-13 18:06:09.753557, event: done, request: pg_info(1 pgs e194497:24.136) v4 -735> 2014-08-13 18:06:09.753553 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] check_recovery_sources no source osds () went down -734> 2014-08-13 18:06:09.753564 7f5c835ef700 10 osd.4 194497 do_waiters -- start -733> 2014-08-13 18:06:09.753566 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -732> 2014-08-13 18:06:09.753563 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map -731> 2014-08-13 18:06:09.753569 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.15 192.168.1.107:6812/53408 13 ==== MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 ==== 26+0+0 (2175379851 0 0) 0x4df08c0 con 0x4854620 -730> 2014-08-13 18:06:09.753577 7f5c835ef700 10 osd.4 194497 do_waiters -- start -729> 2014-08-13 18:06:09.753579 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -728> 2014-08-13 18:06:09.753573 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] take_waiters -727> 2014-08-13 18:06:09.753580 7f5c835ef700 20 osd.4 194497 _dispatch 0x4df08c0 MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -726> 2014-08-13 18:06:09.753583 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.751927, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -725> 2014-08-13 18:06:09.753587 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.751929, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -724> 2014-08-13 18:06:09.753582 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -723> 2014-08-13 18:06:09.753591 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.751966, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -722> 2014-08-13 18:06:09.753594 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.753577, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -721> 2014-08-13 18:06:09.753592 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.653( v 194215'493 (0'0,194215'493] local-les=194496 n=23 ec=183096 les/c 194496/194493 194495/194495/192970) [371,4,21] r=1 lpr=194495 pi=192977-194494/34 luod=0'0 crt=192888'459 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -720> 2014-08-13 18:06:09.753598 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.753598, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -719> 2014-08-13 18:06:09.753602 7f5c7c7e4700 10 log is not dirty -718> 2014-08-13 18:06:09.753603 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x4df08c0 -717> 2014-08-13 18:06:09.753610 7f5c835ef700 5 -- op tracker -- , seq: 685, time: 2014-08-13 18:06:09.753610, event: done, request: MRecoveryReserve REQUEST pgid: 24.20e, query_epoch: 194497 v2 -716> 2014-08-13 18:06:09.753616 7f5c835ef700 10 osd.4 194497 do_waiters -- start -715> 2014-08-13 18:06:09.753617 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -714> 2014-08-13 18:06:09.753619 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_advance_map [356,4,334]/[356,4,32] -- 356/356 -713> 2014-08-13 18:06:09.753635 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -712> 2014-08-13 18:06:09.753639 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: Started advmap -711> 2014-08-13 18:06:09.753480 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 20 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) -710> 2014-08-13 18:06:09.753650 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] check_recovery_sources no source osds () went down -709> 2014-08-13 18:06:09.753662 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map -708> 2014-08-13 18:06:09.753726 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] take_waiters -707> 2014-08-13 18:06:09.753737 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -706> 2014-08-13 18:06:09.753746 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -705> 2014-08-13 18:06:09.753757 7f5c7c7e4700 10 log is not dirty -704> 2014-08-13 18:06:09.753770 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_advance_map [397,343,4]/[397,343,4] -- 397/397 -703> 2014-08-13 18:06:09.753784 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -702> 2014-08-13 18:06:09.753787 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] state: Started advmap -701> 2014-08-13 18:06:09.753797 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] check_recovery_sources no source osds () went down -700> 2014-08-13 18:06:09.753636 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] _activate_committed 194497 peer_activated now 4 last_epoch_started 194492 same_interval_since 194496 -699> 2014-08-13 18:06:09.753807 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map -698> 2014-08-13 18:06:09.753652 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.20 activated and committed -697> 2014-08-13 18:06:09.753821 7f5c7bde3700 10 log is not dirty -696> 2014-08-13 18:06:09.753816 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] take_waiters -695> 2014-08-13 18:06:09.753830 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -694> 2014-08-13 18:06:09.753840 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -693> 2014-08-13 18:06:09.753850 7f5c7c7e4700 10 log is not dirty -692> 2014-08-13 18:06:09.753865 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_advance_map [30,335,4]/[30,335,4] -- 30/30 -691> 2014-08-13 18:06:09.753877 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -690> 2014-08-13 18:06:09.753880 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] state: Started advmap -689> 2014-08-13 18:06:09.753889 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] check_recovery_sources no source osds () went down -688> 2014-08-13 18:06:09.753898 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map -687> 2014-08-13 18:06:09.753907 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] take_waiters -686> 2014-08-13 18:06:09.753915 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -685> 2014-08-13 18:06:09.753923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.791( v 194215'518 (0'0,194215'518] local-les=194496 n=26 ec=183096 les/c 194496/194496 194495/194495/192976) [30,335,4] r=2 lpr=194495 pi=192976-194494/27 luod=0'0 crt=192888'492 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -684> 2014-08-13 18:06:09.753933 7f5c7c7e4700 10 log is not dirty -683> 2014-08-13 18:06:09.753947 7f5c7c7e4700 10 osd.4 pg_epoch: 194496 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [4,50,289]/[4,50,289] -- 4/4 -682> 2014-08-13 18:06:09.753951 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 15 ==== pg_info(1 pgs e194497:24.3f7) v4 ==== 748+0+0 (1625479620 0 0) 0x5357540 con 0x4f20dc0 -681> 2014-08-13 18:06:09.753959 7f5c7c7e4700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -680> 2014-08-13 18:06:09.753961 7f5c835ef700 10 osd.4 194497 do_waiters -- start -679> 2014-08-13 18:06:09.753963 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -678> 2014-08-13 18:06:09.753965 7f5c835ef700 20 osd.4 194497 _dispatch 0x5357540 pg_info(1 pgs e194497:24.3f7) v4 -677> 2014-08-13 18:06:09.753812 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] flushed -676> 2014-08-13 18:06:09.753968 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753855, event: header_read, request: pg_info(1 pgs e194497:24.3f7) v4 -675> 2014-08-13 18:06:09.753973 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753857, event: throttled, request: pg_info(1 pgs e194497:24.3f7) v4 -674> 2014-08-13 18:06:09.753963 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active advmap -673> 2014-08-13 18:06:09.753976 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753909, event: all_read, request: pg_info(1 pgs e194497:24.3f7) v4 -672> 2014-08-13 18:06:09.753980 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753961, event: dispatched, request: pg_info(1 pgs e194497:24.3f7) v4 -671> 2014-08-13 18:06:09.753984 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753983, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3f7) v4 -670> 2014-08-13 18:06:09.753979 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Started advmap -669> 2014-08-13 18:06:09.753987 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3f7) v4 from osd.330 -668> 2014-08-13 18:06:09.753990 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5357540 -667> 2014-08-13 18:06:09.753993 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.753993, event: started, request: pg_info(1 pgs e194497:24.3f7) v4 -666> 2014-08-13 18:06:09.753989 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] check_recovery_sources no source osds () went down -665> 2014-08-13 18:06:09.753827 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 291 info: 24.9b( v 194422'17302 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) -664> 2014-08-13 18:06:09.753998 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map -663> 2014-08-13 18:06:09.754005 7f5c835ef700 5 -- op tracker -- , seq: 686, time: 2014-08-13 18:06:09.754005, event: done, request: pg_info(1 pgs e194497:24.3f7) v4 -662> 2014-08-13 18:06:09.754009 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: Active: handling ActMap -661> 2014-08-13 18:06:09.754017 7f5c835ef700 10 osd.4 194497 do_waiters -- start -660> 2014-08-13 18:06:09.754019 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -659> 2014-08-13 18:06:09.754018 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] state: handle ActMap primary -658> 2014-08-13 18:06:09.754028 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 194497:2714 -657> 2014-08-13 18:06:09.754036 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] take_waiters -656> 2014-08-13 18:06:09.754044 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_activate_map: Not dirtying info: last_persisted is 194496 while current is 194497 -655> 2014-08-13 18:06:09.754052 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.6bd( v 194215'504 (0'0,194215'504] local-les=194496 n=35 ec=183096 les/c 194496/194496 194495/194495/194495) [4,50,289] r=0 lpr=194495 crt=191727'442 lcod 0'0 mlcod 0'0 active+clean] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 NullEvt -654> 2014-08-13 18:06:09.754062 7f5c7c7e4700 10 log is not dirty -653> 2014-08-13 18:06:09.754067 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4493f40 192.168.1.103:6811/37658 -652> 2014-08-13 18:06:09.754077 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6811/37658 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x2003180 con 0x4493f40 -651> 2014-08-13 18:06:09.754090 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.17 has 194497 -650> 2014-08-13 18:06:09.754092 7f5c7c7e4700 7 osd.4 194497 do_notify osd 17 on 1 PGs -649> 2014-08-13 18:06:09.754094 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6811/37658 -- pg_notify(24.2a1(42) epoch 194497) v5 -- ?+0 0x588fe00 con 0x4493f40 -648> 2014-08-13 18:06:09.754003 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.291 activated and committed -647> 2014-08-13 18:06:09.754237 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x445f7a0 192.168.1.113:6808/15284 -646> 2014-08-13 18:06:09.754254 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6808/15284 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x4bc4140 con 0x445f7a0 -645> 2014-08-13 18:06:09.754264 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.289 has 194497 -644> 2014-08-13 18:06:09.754266 7f5c7c7e4700 7 osd.4 194497 do_notify osd 289 on 1 PGs -643> 2014-08-13 18:06:09.754268 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6808/15284 -- pg_notify(24.5e(49) epoch 194497) v5 -- ?+0 0x588f540 con 0x445f7a0 -642> 2014-08-13 18:06:09.754354 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -641> 2014-08-13 18:06:09.754362 7f5c7c7e4700 7 osd.4 194497 do_notify osd 356 on 2 PGs -640> 2014-08-13 18:06:09.754365 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.108:6832/5235 -- pg_notify(24.350(36),24.3b7(45) epoch 194497) v5 -- ?+0 0x588e580 con 0x3c527e0 -639> 2014-08-13 18:06:09.754381 7f5c7c7e4700 20 osd.4 194497 Sending info 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 17 -638> 2014-08-13 18:06:09.754388 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4493f40 already has epoch 194497 -637> 2014-08-13 18:06:09.754411 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.103:6811/37658 -- pg_info(1 pgs e194497:24.b6(41)) v4 -- ?+0 0x588d240 con 0x4493f40 -636> 2014-08-13 18:06:09.754246 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194493/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] all_activated_and_committed -635> 2014-08-13 18:06:09.754422 7f5c7c7e4700 20 osd.4 194497 Sending info 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) to shard 50 -634> 2014-08-13 18:06:09.754428 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c56880 don't know epoch, doing nothing -633> 2014-08-13 18:06:09.754437 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6842/9607 -- pg_info(1 pgs e194497:24.136(37)) v4 -- ?+0 0x588f000 con 0x3c56880 -632> 2014-08-13 18:06:09.754448 7f5c7c7e4700 20 osd.4 194497 Sending info 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 55 -631> 2014-08-13 18:06:09.754454 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c523c0 don't know epoch, doing nothing -630> 2014-08-13 18:06:09.754466 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.112:6834/6966 -- pg_info(1 pgs e194497:24.1f7(41)) v4 -- ?+0 0x5a6f8c0 con 0x3c523c0 -629> 2014-08-13 18:06:09.754475 7f5c7c7e4700 20 osd.4 194497 Sending info 24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) to shard 65 -628> 2014-08-13 18:06:09.754480 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x4b11600 don't know epoch, doing nothing -627> 2014-08-13 18:06:09.754497 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.111:6842/17345 -- pg_info(1 pgs e194497:24.136(37)) v4 -- ?+0 0x5a6aa00 con 0x4b11600 -626> 2014-08-13 18:06:09.754507 7f5c7c7e4700 20 osd.4 194497 Sending info 24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 315 -625> 2014-08-13 18:06:09.754517 7f5c7c7e4700 20 osd.4 194497 _share_map_outgoing 0x3c51080 don't know epoch, doing nothing -624> 2014-08-13 18:06:09.754534 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.113:6800/14161 -- pg_info(1 pgs e194497:24.b6(41)) v4 -- ?+0 0x5a6af40 con 0x3c51080 -623> 2014-08-13 18:06:09.754543 7f5c7c7e4700 20 osd.4 194497 Sending info 24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) to shard 351 -622> 2014-08-13 18:06:09.754571 7f5c7c7e4700 10 osd.4 194497 send_incremental_map 194496 -> 194497 to 0x4af8160 192.168.1.102:6830/16444 -621> 2014-08-13 18:06:09.754578 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.102:6830/16444 -- osd_map(194497..194497 src has 185139..194497) v3 -- ?+0 0x5151680 con 0x4af8160 -620> 2014-08-13 18:06:09.754589 7f5c7c7e4700 10 osd.4 194497 note_peer_epoch osd.351 has 194497 -619> 2014-08-13 18:06:09.754420 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] share_pg_info -618> 2014-08-13 18:06:09.754598 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4424e60 don't know epoch, doing nothing -617> 2014-08-13 18:06:09.754601 7f5c7c7e4700 1 -- 192.168.1.115:6813/37708 --> 192.168.1.102:6830/16444 -- pg_info(1 pgs e194497:24.1f7(41)) v4 -- ?+0 0x588db00 con 0x4af8160 -616> 2014-08-13 18:06:09.754601 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.291 192.168.1.101:6820/25633 -- pg_info(1 pgs e194497:24.9b) v4 -- ?+0 0x588ee40 -615> 2014-08-13 18:06:09.754724 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5ee00 don't know epoch, doing nothing -614> 2014-08-13 18:06:09.754727 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.350 192.168.1.114:6832/53185 -- pg_info(1 pgs e194497:24.9b) v4 -- ?+0 0x588abc0 -613> 2014-08-13 18:06:09.754741 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b11ce0 already has epoch 194497 -612> 2014-08-13 18:06:09.754743 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.367 192.168.1.114:6840/55513 -- pg_info(1 pgs e194497:24.9b) v4 -- ?+0 0x588fa80 -611> 2014-08-13 18:06:09.754756 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4fa1ce0 don't know epoch, doing nothing -610> 2014-08-13 18:06:09.754758 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.378 192.168.1.102:6840/19195 -- pg_info(1 pgs e194497:24.9b) v4 -- ?+0 0x588cd00 -609> 2014-08-13 18:06:09.754769 7f5c7c7e4700 10 osd.4 194497 dequeue_op 0x20a7ad0 prio 127 cost 0 latency 0.266520 osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 pg pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] -608> 2014-08-13 18:06:09.754794 7f5c7c7e4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.754794, event: reached_pg, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -607> 2014-08-13 18:06:09.754805 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_message: 0x20a7ad0 -606> 2014-08-13 18:06:09.754819 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] do_sub_op osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -605> 2014-08-13 18:06:09.754831 7f5c7c7e4700 7 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] sub_op_scrub_unreserve -604> 2014-08-13 18:06:09.754841 7f5c7c7e4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.754840, event: started, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -603> 2014-08-13 18:06:09.754848 7f5c7c7e4700 20 osd.4 194497 dec_scrubs_pending 1 -> 0 (max 1, active 0) -602> 2014-08-13 18:06:09.754850 7f5c7c7e4700 10 osd.4 194497 dequeue_op 0x20a7ad0 finish -601> 2014-08-13 18:06:09.754853 7f5c7c7e4700 5 -- op tracker -- , seq: 635, time: 2014-08-13 18:06:09.754853, event: done, request: osd_sub_op(unknown.0.0:0 22.3b9 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v10 -600> 2014-08-13 18:06:09.754877 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 45 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -599> 2014-08-13 18:06:09.754897 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 45, already have info from that osd, ignoring -598> 2014-08-13 18:06:09.754908 7f5c7c7e4700 10 log is not dirty -597> 2014-08-13 18:06:09.754914 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) -596> 2014-08-13 18:06:09.754926 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: got info from osd.17 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) -595> 2014-08-13 18:06:09.754770 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] publish_stats_to_osd 194497:25294 -594> 2014-08-13 18:06:09.754952 7f5c7bde3700 10 log is not dirty -593> 2014-08-13 18:06:09.754943 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] exit Started/Stray 1.524477 8 0.000592 -592> 2014-08-13 18:06:09.754956 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive -591> 2014-08-13 18:06:09.754958 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 30 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) -590> 2014-08-13 18:06:09.754969 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] enter Started/ReplicaActive/RepNotRecovering -589> 2014-08-13 18:06:09.754971 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.30 activated and committed -588> 2014-08-13 18:06:09.754982 7f5c7bde3700 10 log is not dirty -587> 2014-08-13 18:06:09.754981 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194487 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive NOTIFY m=49] state: In ReplicaActive, about to call activate -586> 2014-08-13 18:06:09.754987 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 61 notify: (query_epoch:194497, epoch_sent:194497, info:22.362( v 194215'479 (0'0,194215'479] local-les=194492 n=23 ec=183096 les/c 194492/194492 194495/194495/194495)) -585> 2014-08-13 18:06:09.754991 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] activate - not complete, missing(49) -584> 2014-08-13 18:06:09.754999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.362( v 194215'479 (0'0,194215'479] local-les=194496 n=23 ec=183096 les/c 194496/194492 194495/194495/194495) [4,58,343] r=0 lpr=194495 pi=193306-194494/35 crt=191727'415 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 61, already have info from that osd, ignoring -583> 2014-08-13 18:06:09.755009 7f5c7bde3700 10 log is not dirty -582> 2014-08-13 18:06:09.755012 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 340 info: 24.34( v 194215'14514 (192718'11510,194215'14514] lb 0//0//-1 local-les=194258 n=0 ec=192688 les/c 194497/194258 194459/194496/194496) -581> 2014-08-13 18:06:09.755023 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] state: peer osd.340 activated and committed -580> 2014-08-13 18:06:09.755033 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194487/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] all_activated_and_committed -579> 2014-08-13 18:06:09.755042 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] share_pg_info -578> 2014-08-13 18:06:09.755054 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3a7e9e0 don't know epoch, doing nothing -577> 2014-08-13 18:06:09.755056 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.26 192.168.1.115:6802/58522 -- pg_info(1 pgs e194497:24.34) v4 -- ?+0 0x5889c00 -576> 2014-08-13 18:06:09.755069 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44586e0 don't know epoch, doing nothing -575> 2014-08-13 18:06:09.755071 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.30 192.168.1.107:6823/55491 -- pg_info(1 pgs e194497:24.34) v4 -- ?+0 0x5888540 -574> 2014-08-13 18:06:09.755084 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4499080 don't know epoch, doing nothing -573> 2014-08-13 18:06:09.755086 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.340 192.168.1.104:6806/62463 -- pg_info(1 pgs e194497:24.34) v4 -- ?+0 0x5889500 -572> 2014-08-13 18:06:09.755097 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] publish_stats_to_osd 194497:23847 -571> 2014-08-13 18:06:09.755112 7f5c7bde3700 10 log is not dirty -570> 2014-08-13 18:06:09.755203 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 286 info: 24.30b( v 194422'18660 (192718'15573,194422'18660] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -569> 2014-08-13 18:06:09.755376 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] state: peer osd.286 activated and committed -568> 2014-08-13 18:06:09.755562 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.330 192.168.1.106:6802/4876 16 ==== MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 ==== 26+0+0 (772834934 0 0) 0x5a6e900 con 0x4f20dc0 -567> 2014-08-13 18:06:09.755580 7f5c835ef700 10 osd.4 194497 do_waiters -- start -566> 2014-08-13 18:06:09.755585 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -565> 2014-08-13 18:06:09.755587 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6e900 MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -564> 2014-08-13 18:06:09.755591 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755481, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -563> 2014-08-13 18:06:09.755597 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755484, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -562> 2014-08-13 18:06:09.755600 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755531, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -561> 2014-08-13 18:06:09.755604 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755579, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -560> 2014-08-13 18:06:09.755608 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755607, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -559> 2014-08-13 18:06:09.755612 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6e900 -558> 2014-08-13 18:06:09.755621 7f5c835ef700 5 -- op tracker -- , seq: 687, time: 2014-08-13 18:06:09.755621, event: done, request: MRecoveryReserve REQUEST pgid: 24.3f7, query_epoch: 194497 v2 -557> 2014-08-13 18:06:09.755634 7f5c835ef700 10 osd.4 194497 do_waiters -- start -556> 2014-08-13 18:06:09.755636 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -555> 2014-08-13 18:06:09.755543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] all_activated_and_committed -554> 2014-08-13 18:06:09.755785 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] state: Activate Finished -553> 2014-08-13 18:06:09.755857 7f5c7c7e4700 10 log is not dirty -552> 2014-08-13 18:06:09.755709 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] share_pg_info -551> 2014-08-13 18:06:09.755877 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x43f0840 don't know epoch, doing nothing -550> 2014-08-13 18:06:09.755879 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.10 192.168.1.115:6815/60573 -- pg_info(1 pgs e194497:24.30b) v4 -- ?+0 0x588b9c0 -549> 2014-08-13 18:06:09.755895 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3cf2940 don't know epoch, doing nothing -548> 2014-08-13 18:06:09.755897 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.286 192.168.1.113:6820/19173 -- pg_info(1 pgs e194497:24.30b) v4 -- ?+0 0x588b640 -547> 2014-08-13 18:06:09.755910 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c5e5c0 don't know epoch, doing nothing -546> 2014-08-13 18:06:09.755912 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.375 192.168.1.114:6830/52879 -- pg_info(1 pgs e194497:24.30b) v4 -- ?+0 0x58881c0 -545> 2014-08-13 18:06:09.755886 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 356 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -544> 2014-08-13 18:06:09.755925 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.30b( v 194422'18660 lc 194370'18547 (192718'15573,194422'18660] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/194495) [4,286,375]/[4,375,10] r=0 lpr=194496 pi=193640-194495/42 bft=286 crt=194422'18660 lcod 0'0 mlcod 0'0 remapped m=56] publish_stats_to_osd 194497:30302 -543> 2014-08-13 18:06:09.756095 7f5c7bde3700 10 log is not dirty -542> 2014-08-13 18:06:09.756100 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.481( v 194215'519 (0'0,194215'519] local-les=194492 n=32 ec=183096 les/c 194492/194304 194495/194495/194495)) -541> 2014-08-13 18:06:09.756112 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.481( v 194215'519 (0'0,194215'519] local-les=194496 n=32 ec=183096 les/c 194496/194304 194495/194495/194495) [4,287,309] r=0 lpr=194495 pi=193306-194494/50 crt=191727'463 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring -540> 2014-08-13 18:06:09.756122 7f5c7bde3700 10 log is not dirty -539> 2014-08-13 18:06:09.756129 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 crt=194422'14353 lcod 0'0 inactive m=49] _activate_committed 194497 telling primary -538> 2014-08-13 18:06:09.756160 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x4493f40 already has epoch 194497 -537> 2014-08-13 18:06:09.756163 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.17 192.168.1.103:6811/37658 -- pg_info(1 pgs e194497:24.2a1) v4 -- ?+0 0x5357700 -536> 2014-08-13 18:06:09.756188 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] flushed -535> 2014-08-13 18:06:09.756053 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 356, already have info from that osd, ignoring -534> 2014-08-13 18:06:09.756218 7f5c7c7e4700 10 log is not dirty -533> 2014-08-13 18:06:09.756125 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 25 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -532> 2014-08-13 18:06:09.756225 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 50 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -531> 2014-08-13 18:06:09.756324 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.25 activated and committed -530> 2014-08-13 18:06:09.756496 7f5c7bde3700 10 log is not dirty -529> 2014-08-13 18:06:09.756502 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 369 notify: (query_epoch:194497, epoch_sent:194497, info:22.775( v 194215'513 (0'0,194215'513] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -528> 2014-08-13 18:06:09.756513 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.775( v 194215'513 (0'0,194215'513] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,353,56] r=0 lpr=194495 pi=193306-194494/37 crt=191727'453 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 369, already have info from that osd, ignoring -527> 2014-08-13 18:06:09.756523 7f5c7bde3700 10 log is not dirty -526> 2014-08-13 18:06:09.756380 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 50, already have info from that osd, ignoring -525> 2014-08-13 18:06:09.756519 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.55 192.168.1.112:6834/6966 17 ==== pg_info(1 pgs e194497:24.1f7) v4 ==== 748+0+0 (1047934475 0 0) 0x5a6f8c0 con 0x3c523c0 -524> 2014-08-13 18:06:09.756537 7f5c835ef700 10 osd.4 194497 do_waiters -- start -523> 2014-08-13 18:06:09.756540 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -522> 2014-08-13 18:06:09.756542 7f5c7c7e4700 10 log is not dirty -521> 2014-08-13 18:06:09.756542 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6f8c0 pg_info(1 pgs e194497:24.1f7) v4 -520> 2014-08-13 18:06:09.756546 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756414, event: header_read, request: pg_info(1 pgs e194497:24.1f7) v4 -519> 2014-08-13 18:06:09.756553 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756415, event: throttled, request: pg_info(1 pgs e194497:24.1f7) v4 -518> 2014-08-13 18:06:09.756547 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 15 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) -517> 2014-08-13 18:06:09.756556 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756463, event: all_read, request: pg_info(1 pgs e194497:24.1f7) v4 -516> 2014-08-13 18:06:09.756560 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756536, event: dispatched, request: pg_info(1 pgs e194497:24.1f7) v4 -515> 2014-08-13 18:06:09.756564 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756564, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1f7) v4 -514> 2014-08-13 18:06:09.756560 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 15, already have info from that osd, ignoring -513> 2014-08-13 18:06:09.756568 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.55 -512> 2014-08-13 18:06:09.756570 7f5c7c7e4700 10 log is not dirty -511> 2014-08-13 18:06:09.756571 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6f8c0 -510> 2014-08-13 18:06:09.756575 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756574, event: started, request: pg_info(1 pgs e194497:24.1f7) v4 -509> 2014-08-13 18:06:09.756574 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) -508> 2014-08-13 18:06:09.756586 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring -507> 2014-08-13 18:06:09.756595 7f5c7c7e4700 10 log is not dirty -506> 2014-08-13 18:06:09.756592 7f5c835ef700 5 -- op tracker -- , seq: 688, time: 2014-08-13 18:06:09.756592, event: done, request: pg_info(1 pgs e194497:24.1f7) v4 -505> 2014-08-13 18:06:09.756606 7f5c835ef700 10 osd.4 194497 do_waiters -- start -504> 2014-08-13 18:06:09.756608 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -503> 2014-08-13 18:06:09.756618 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.50 192.168.1.112:6842/9607 11 ==== pg_info(1 pgs e194497:24.136) v4 ==== 748+0+0 (41070967 0 0) 0x588f000 con 0x3c56880 -502> 2014-08-13 18:06:09.756626 7f5c835ef700 10 osd.4 194497 do_waiters -- start -501> 2014-08-13 18:06:09.756628 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -500> 2014-08-13 18:06:09.756629 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f000 pg_info(1 pgs e194497:24.136) v4 -499> 2014-08-13 18:06:09.756632 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756486, event: header_read, request: pg_info(1 pgs e194497:24.136) v4 -498> 2014-08-13 18:06:09.756636 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756487, event: throttled, request: pg_info(1 pgs e194497:24.136) v4 -497> 2014-08-13 18:06:09.756644 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756534, event: all_read, request: pg_info(1 pgs e194497:24.136) v4 -496> 2014-08-13 18:06:09.756647 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756626, event: dispatched, request: pg_info(1 pgs e194497:24.136) v4 -495> 2014-08-13 18:06:09.756651 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756650, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.136) v4 -494> 2014-08-13 18:06:09.756654 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.50 -493> 2014-08-13 18:06:09.756656 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f000 -492> 2014-08-13 18:06:09.756659 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756659, event: started, request: pg_info(1 pgs e194497:24.136) v4 -491> 2014-08-13 18:06:09.756670 7f5c835ef700 5 -- op tracker -- , seq: 689, time: 2014-08-13 18:06:09.756670, event: done, request: pg_info(1 pgs e194497:24.136) v4 -490> 2014-08-13 18:06:09.756680 7f5c835ef700 10 osd.4 194497 do_waiters -- start -489> 2014-08-13 18:06:09.756682 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -488> 2014-08-13 18:06:09.756526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 303 info: 24.3d( v 194422'18731 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) -487> 2014-08-13 18:06:09.756770 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.351 192.168.1.102:6830/16444 13 ==== pg_info(1 pgs e194497:24.1f7) v4 ==== 748+0+0 (3766943654 0 0) 0x588db00 con 0x4af8160 -486> 2014-08-13 18:06:09.756599 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 351 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -485> 2014-08-13 18:06:09.756788 7f5c835ef700 10 osd.4 194497 do_waiters -- start -484> 2014-08-13 18:06:09.756791 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -483> 2014-08-13 18:06:09.756792 7f5c835ef700 20 osd.4 194497 _dispatch 0x588db00 pg_info(1 pgs e194497:24.1f7) v4 -482> 2014-08-13 18:06:09.756797 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756679, event: header_read, request: pg_info(1 pgs e194497:24.1f7) v4 -481> 2014-08-13 18:06:09.756803 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756680, event: throttled, request: pg_info(1 pgs e194497:24.1f7) v4 -480> 2014-08-13 18:06:09.756806 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756729, event: all_read, request: pg_info(1 pgs e194497:24.1f7) v4 -479> 2014-08-13 18:06:09.756810 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756787, event: dispatched, request: pg_info(1 pgs e194497:24.1f7) v4 -478> 2014-08-13 18:06:09.756814 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756814, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.1f7) v4 -477> 2014-08-13 18:06:09.756817 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.1f7) v4 from osd.351 -476> 2014-08-13 18:06:09.756821 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588db00 -475> 2014-08-13 18:06:09.756824 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756824, event: started, request: pg_info(1 pgs e194497:24.1f7) v4 -474> 2014-08-13 18:06:09.756695 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] state: peer osd.303 activated and committed -473> 2014-08-13 18:06:09.756792 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 351, already have info from that osd, ignoring -472> 2014-08-13 18:06:09.756975 7f5c7c7e4700 10 log is not dirty -471> 2014-08-13 18:06:09.756991 7f5c835ef700 5 -- op tracker -- , seq: 690, time: 2014-08-13 18:06:09.756991, event: done, request: pg_info(1 pgs e194497:24.1f7) v4 -470> 2014-08-13 18:06:09.757010 7f5c835ef700 10 osd.4 194497 do_waiters -- start -469> 2014-08-13 18:06:09.757012 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -468> 2014-08-13 18:06:09.756863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] all_activated_and_committed -467> 2014-08-13 18:06:09.757031 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.17 192.168.1.103:6811/37658 13 ==== pg_info(1 pgs e194497:24.b6) v4 ==== 748+0+0 (1981628245 0 0) 0x588d240 con 0x4493f40 -466> 2014-08-13 18:06:09.757040 7f5c835ef700 10 osd.4 194497 do_waiters -- start -465> 2014-08-13 18:06:09.757042 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -464> 2014-08-13 18:06:09.757043 7f5c835ef700 20 osd.4 194497 _dispatch 0x588d240 pg_info(1 pgs e194497:24.b6) v4 -463> 2014-08-13 18:06:09.757049 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.756821, event: header_read, request: pg_info(1 pgs e194497:24.b6) v4 -462> 2014-08-13 18:06:09.757054 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.756823, event: throttled, request: pg_info(1 pgs e194497:24.b6) v4 -461> 2014-08-13 18:06:09.757057 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.756871, event: all_read, request: pg_info(1 pgs e194497:24.b6) v4 -460> 2014-08-13 18:06:09.757060 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.757040, event: dispatched, request: pg_info(1 pgs e194497:24.b6) v4 -459> 2014-08-13 18:06:09.757064 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.757063, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.b6) v4 -458> 2014-08-13 18:06:09.757067 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.17 -457> 2014-08-13 18:06:09.757070 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588d240 -456> 2014-08-13 18:06:09.757072 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.757072, event: started, request: pg_info(1 pgs e194497:24.b6) v4 -455> 2014-08-13 18:06:09.757085 7f5c835ef700 5 -- op tracker -- , seq: 691, time: 2014-08-13 18:06:09.757085, event: done, request: pg_info(1 pgs e194497:24.b6) v4 -454> 2014-08-13 18:06:09.757101 7f5c835ef700 10 osd.4 194497 do_waiters -- start -453> 2014-08-13 18:06:09.757103 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -452> 2014-08-13 18:06:09.757111 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.17 192.168.1.103:6811/37658 14 ==== pg_info(1 pgs e194497:24.2a1) v4 ==== 748+0+0 (2247930935 0 0) 0x588fe00 con 0x4493f40 -451> 2014-08-13 18:06:09.757119 7f5c835ef700 10 osd.4 194497 do_waiters -- start -450> 2014-08-13 18:06:09.757120 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -449> 2014-08-13 18:06:09.757121 7f5c835ef700 20 osd.4 194497 _dispatch 0x588fe00 pg_info(1 pgs e194497:24.2a1) v4 -448> 2014-08-13 18:06:09.757124 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.756900, event: header_read, request: pg_info(1 pgs e194497:24.2a1) v4 -447> 2014-08-13 18:06:09.757128 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.756902, event: throttled, request: pg_info(1 pgs e194497:24.2a1) v4 -446> 2014-08-13 18:06:09.757132 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.756937, event: all_read, request: pg_info(1 pgs e194497:24.2a1) v4 -445> 2014-08-13 18:06:09.756984 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 65 notify: (query_epoch:194497, epoch_sent:194497, info:24.136( v 194422'15470 (192834'12481,194422'15470] local-les=194492 n=52 ec=192688 les/c 194492/194492 194495/194496/194495)) -444> 2014-08-13 18:06:09.757135 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.757118, event: dispatched, request: pg_info(1 pgs e194497:24.2a1) v4 -443> 2014-08-13 18:06:09.757139 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.757138, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.2a1) v4 -442> 2014-08-13 18:06:09.757143 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.2a1) v4 from osd.17 -441> 2014-08-13 18:06:09.757145 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588fe00 -440> 2014-08-13 18:06:09.757148 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.757148, event: started, request: pg_info(1 pgs e194497:24.2a1) v4 -439> 2014-08-13 18:06:09.757162 7f5c835ef700 5 -- op tracker -- , seq: 692, time: 2014-08-13 18:06:09.757162, event: done, request: pg_info(1 pgs e194497:24.2a1) v4 -438> 2014-08-13 18:06:09.757178 7f5c835ef700 10 osd.4 194497 do_waiters -- start -437> 2014-08-13 18:06:09.757179 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -436> 2014-08-13 18:06:09.757187 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.315 192.168.1.113:6800/14161 9 ==== pg_info(1 pgs e194497:24.b6) v4 ==== 748+0+0 (1438571010 0 0) 0x5a6af40 con 0x3c51080 -435> 2014-08-13 18:06:09.757195 7f5c835ef700 10 osd.4 194497 do_waiters -- start -434> 2014-08-13 18:06:09.757196 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -433> 2014-08-13 18:06:09.757029 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] share_pg_info -432> 2014-08-13 18:06:09.757197 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6af40 pg_info(1 pgs e194497:24.b6) v4 -431> 2014-08-13 18:06:09.757205 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x44261a0 don't know epoch, doing nothing -430> 2014-08-13 18:06:09.757205 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757021, event: header_read, request: pg_info(1 pgs e194497:24.b6) v4 -429> 2014-08-13 18:06:09.757207 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.20 192.168.1.103:6815/38269 -- pg_info(1 pgs e194497:24.3d) v4 -- ?+0 0x5a6e200 -428> 2014-08-13 18:06:09.757211 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757022, event: throttled, request: pg_info(1 pgs e194497:24.b6) v4 -427> 2014-08-13 18:06:09.757215 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757067, event: all_read, request: pg_info(1 pgs e194497:24.b6) v4 -426> 2014-08-13 18:06:09.757218 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757194, event: dispatched, request: pg_info(1 pgs e194497:24.b6) v4 -425> 2014-08-13 18:06:09.757221 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4c58160 don't know epoch, doing nothing -424> 2014-08-13 18:06:09.757221 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757221, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.b6) v4 -423> 2014-08-13 18:06:09.757225 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.b6) v4 from osd.315 -422> 2014-08-13 18:06:09.757224 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.25 192.168.1.103:6800/36920 -- pg_info(1 pgs e194497:24.3d) v4 -- ?+0 0x5a6bf00 -421> 2014-08-13 18:06:09.757228 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6af40 -420> 2014-08-13 18:06:09.757231 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757231, event: started, request: pg_info(1 pgs e194497:24.b6) v4 -419> 2014-08-13 18:06:09.757237 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4af8580 don't know epoch, doing nothing -418> 2014-08-13 18:06:09.757239 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.303 192.168.1.109:6802/30768 -- pg_info(1 pgs e194497:24.3d) v4 -- ?+0 0x5a6e3c0 -417> 2014-08-13 18:06:09.757245 7f5c835ef700 5 -- op tracker -- , seq: 693, time: 2014-08-13 18:06:09.757245, event: done, request: pg_info(1 pgs e194497:24.b6) v4 -416> 2014-08-13 18:06:09.757253 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x49a53e0 don't know epoch, doing nothing -415> 2014-08-13 18:06:09.757255 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.362 192.168.1.108:6842/8170 -- pg_info(1 pgs e194497:24.3d) v4 -- ?+0 0x5a68540 -414> 2014-08-13 18:06:09.757263 7f5c835ef700 10 osd.4 194497 do_waiters -- start -413> 2014-08-13 18:06:09.757266 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -412> 2014-08-13 18:06:09.757138 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: Active: got notify from 65, already have info from that osd, ignoring -411> 2014-08-13 18:06:09.757295 7f5c7c7e4700 10 log is not dirty -410> 2014-08-13 18:06:09.757410 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.65 192.168.1.111:6842/17345 11 ==== pg_info(1 pgs e194497:24.136) v4 ==== 748+0+0 (1125041187 0 0) 0x5a6aa00 con 0x4b11600 -409> 2014-08-13 18:06:09.757420 7f5c835ef700 10 osd.4 194497 do_waiters -- start -408> 2014-08-13 18:06:09.757422 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -407> 2014-08-13 18:06:09.757423 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6aa00 pg_info(1 pgs e194497:24.136) v4 -406> 2014-08-13 18:06:09.757426 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757337, event: header_read, request: pg_info(1 pgs e194497:24.136) v4 -405> 2014-08-13 18:06:09.757431 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757338, event: throttled, request: pg_info(1 pgs e194497:24.136) v4 -404> 2014-08-13 18:06:09.757434 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757382, event: all_read, request: pg_info(1 pgs e194497:24.136) v4 -403> 2014-08-13 18:06:09.757437 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757420, event: dispatched, request: pg_info(1 pgs e194497:24.136) v4 -402> 2014-08-13 18:06:09.757441 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757441, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.136) v4 -401> 2014-08-13 18:06:09.757445 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.136) v4 from osd.65 -400> 2014-08-13 18:06:09.757447 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6aa00 -399> 2014-08-13 18:06:09.757450 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757450, event: started, request: pg_info(1 pgs e194497:24.136) v4 -398> 2014-08-13 18:06:09.757465 7f5c835ef700 5 -- op tracker -- , seq: 694, time: 2014-08-13 18:06:09.757465, event: done, request: pg_info(1 pgs e194497:24.136) v4 -397> 2014-08-13 18:06:09.757488 7f5c835ef700 10 osd.4 194497 do_waiters -- start -396> 2014-08-13 18:06:09.757490 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -395> 2014-08-13 18:06:09.757346 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] publish_stats_to_osd 194497:29711 -394> 2014-08-13 18:06:09.757527 7f5c7bde3700 10 log is not dirty -393> 2014-08-13 18:06:09.757384 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 376 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -392> 2014-08-13 18:06:09.757534 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -391> 2014-08-13 18:06:09.757593 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 376, already have info from that osd, ignoring -390> 2014-08-13 18:06:09.757784 7f5c7c7e4700 10 log is not dirty -389> 2014-08-13 18:06:09.757790 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 11 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -388> 2014-08-13 18:06:09.757804 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 11, already have info from that osd, ignoring -387> 2014-08-13 18:06:09.757814 7f5c7c7e4700 10 log is not dirty -386> 2014-08-13 18:06:09.757685 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] requeue_ops -385> 2014-08-13 18:06:09.757818 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 368 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -384> 2014-08-13 18:06:09.757828 7f5c7bde3700 10 log is not dirty -383> 2014-08-13 18:06:09.757829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 368, already have info from that osd, ignoring -382> 2014-08-13 18:06:09.757839 7f5c7c7e4700 10 log is not dirty -381> 2014-08-13 18:06:09.757834 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -380> 2014-08-13 18:06:09.757843 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 54 notify: (query_epoch:194497, epoch_sent:194497, info:22.1f1( v 194215'493 (0'0,194215'493] local-les=194493 n=35 ec=183096 les/c 194493/194493 194495/194495/194495)) -379> 2014-08-13 18:06:09.757848 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] requeue_ops -378> 2014-08-13 18:06:09.757859 7f5c7bde3700 10 log is not dirty -377> 2014-08-13 18:06:09.757854 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.1f1( v 194215'493 (0'0,194215'493] local-les=194496 n=35 ec=183096 les/c 194496/194493 194495/194495/194495) [4,5,47] r=0 lpr=194495 pi=193306-194494/35 crt=191648'346 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 54, already have info from that osd, ignoring -376> 2014-08-13 18:06:09.757867 7f5c7c7e4700 10 log is not dirty -375> 2014-08-13 18:06:09.757871 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 289 info: 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) -374> 2014-08-13 18:06:09.757884 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: got info from osd.289 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) -373> 2014-08-13 18:06:09.757897 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] exit Started/Stray 1.521525 8 0.000676 -372> 2014-08-13 18:06:09.757910 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive -371> 2014-08-13 18:06:09.757923 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] enter Started/ReplicaActive/RepNotRecovering -370> 2014-08-13 18:06:09.757933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194487 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive NOTIFY m=63] state: In ReplicaActive, about to call activate -369> 2014-08-13 18:06:09.757943 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] activate - not complete, missing(63) -368> 2014-08-13 18:06:09.757863 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 332 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) -367> 2014-08-13 18:06:09.758007 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.332 activated and committed -366> 2014-08-13 18:06:09.758157 7f5c7bde3700 10 log is not dirty -365> 2014-08-13 18:06:09.758167 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 344 info: 24.181( v 194422'19613 (192834'17288,194422'19613] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194466 194495/194496/194496) -364> 2014-08-13 18:06:09.758313 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.344 activated and committed -363> 2014-08-13 18:06:09.758453 7f5c7bde3700 10 log is not dirty -362> 2014-08-13 18:06:09.758458 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 320 info: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) -361> 2014-08-13 18:06:09.758599 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.320 activated and committed -360> 2014-08-13 18:06:09.758739 7f5c7bde3700 10 log is not dirty -359> 2014-08-13 18:06:09.758743 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 337 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) -358> 2014-08-13 18:06:09.758949 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] state: Activate Finished -357> 2014-08-13 18:06:09.759026 7f5c7c7e4700 10 log is not dirty -356> 2014-08-13 18:06:09.758888 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.337 activated and committed -355> 2014-08-13 18:06:09.759031 7f5c7bde3700 10 log is not dirty -354> 2014-08-13 18:06:09.759051 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) -353> 2014-08-13 18:06:09.759064 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: got info from osd.356 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) -352> 2014-08-13 18:06:09.759079 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] exit Started/Stray 1.504227 9 0.000755 -351> 2014-08-13 18:06:09.759094 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive -350> 2014-08-13 18:06:09.759106 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] enter Started/ReplicaActive/RepNotRecovering -349> 2014-08-13 18:06:09.759117 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194487 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped NOTIFY m=56] state: In ReplicaActive, about to call activate -348> 2014-08-13 18:06:09.759127 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] activate - not complete, missing(56) -347> 2014-08-13 18:06:09.759057 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 17 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -346> 2014-08-13 18:06:09.759291 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 crt=194422'17034 lcod 0'0 inactive m=63] _activate_committed 194497 telling primary -345> 2014-08-13 18:06:09.759320 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x445f7a0 already has epoch 194497 -344> 2014-08-13 18:06:09.759322 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.289 192.168.1.113:6808/15284 -- pg_info(1 pgs e194497:24.5e) v4 -- ?+0 0x5a6fa80 -343> 2014-08-13 18:06:09.759338 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] flushed -342> 2014-08-13 18:06:09.759259 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 17, already have info from that osd, ignoring -341> 2014-08-13 18:06:09.759437 7f5c7c7e4700 10 log is not dirty -340> 2014-08-13 18:06:09.759445 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) -339> 2014-08-13 18:06:09.759457 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: got info from osd.356 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) -338> 2014-08-13 18:06:09.759470 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] exit Started/Stray 1.509055 8 0.000603 -337> 2014-08-13 18:06:09.759481 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive -336> 2014-08-13 18:06:09.759495 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] enter Started/ReplicaActive/RepNotRecovering -335> 2014-08-13 18:06:09.759505 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194487 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive NOTIFY m=51] state: In ReplicaActive, about to call activate -334> 2014-08-13 18:06:09.759515 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] activate - not complete, missing(51) -333> 2014-08-13 18:06:09.759869 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] state: Activate Finished -332> 2014-08-13 18:06:09.759940 7f5c7bde3700 10 log is not dirty -331> 2014-08-13 18:06:09.759961 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) [397,343,4] r=2 lpr=194495 pi=192965-194494/27 luod=0'0 crt=192888'478 lcod 0'0 active] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 397 info: 22.3b9( v 194215'518 (0'0,194215'518] local-les=194496 n=33 ec=183096 les/c 194496/194496 194495/194495/192958) -330> 2014-08-13 18:06:09.759999 7f5c7bde3700 10 log is not dirty -329> 2014-08-13 18:06:09.760013 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.206( v 194422'14476 lc 194370'14395 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194493/194489 194495/194496/193262) [343,45,4] r=2 lpr=194496 pi=193262-194495/36 luod=0'0 crt=194422'14476 lcod 0'0 active m=41] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 343 info: 24.206( v 194422'14476 (192718'11573,194422'14476] local-les=194497 n=43 ec=192688 les/c 194497/194489 194495/194496/193262) -328> 2014-08-13 18:06:09.760044 7f5c7bde3700 10 log is not dirty -327> 2014-08-13 18:06:09.760159 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] state: Activate Finished -326> 2014-08-13 18:06:09.760160 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 crt=194422'18845 lcod 0'0 remapped m=56] _activate_committed 194497 telling primary -325> 2014-08-13 18:06:09.760196 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -324> 2014-08-13 18:06:09.760055 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 296 info: 24.181( v 194422'19613 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) -323> 2014-08-13 18:06:09.760199 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194497:24.3b7) v4 -- ?+0 0x5a6c980 -322> 2014-08-13 18:06:09.760198 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.289 192.168.1.113:6808/15284 16 ==== pg_info(1 pgs e194497:24.5e) v4 ==== 748+0+0 (1654334740 0 0) 0x5a6fa80 con 0x445f7a0 -321> 2014-08-13 18:06:09.760213 7f5c835ef700 10 osd.4 194497 do_waiters -- start -320> 2014-08-13 18:06:09.760215 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -319> 2014-08-13 18:06:09.760216 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6fa80 pg_info(1 pgs e194497:24.5e) v4 -318> 2014-08-13 18:06:09.760220 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760121, event: header_read, request: pg_info(1 pgs e194497:24.5e) v4 -317> 2014-08-13 18:06:09.760224 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760122, event: throttled, request: pg_info(1 pgs e194497:24.5e) v4 -316> 2014-08-13 18:06:09.760227 7f5c7c7e4700 10 log is not dirty -315> 2014-08-13 18:06:09.760216 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] flushed -314> 2014-08-13 18:06:09.760228 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760185, event: all_read, request: pg_info(1 pgs e194497:24.5e) v4 -313> 2014-08-13 18:06:09.760231 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760212, event: dispatched, request: pg_info(1 pgs e194497:24.5e) v4 -312> 2014-08-13 18:06:09.760235 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760235, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.5e) v4 -311> 2014-08-13 18:06:09.760239 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.5e) v4 from osd.289 -310> 2014-08-13 18:06:09.760241 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6fa80 -309> 2014-08-13 18:06:09.760244 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760243, event: started, request: pg_info(1 pgs e194497:24.5e) v4 -308> 2014-08-13 18:06:09.760256 7f5c835ef700 5 -- op tracker -- , seq: 695, time: 2014-08-13 18:06:09.760255, event: done, request: pg_info(1 pgs e194497:24.5e) v4 -307> 2014-08-13 18:06:09.760275 7f5c835ef700 10 osd.4 194497 do_waiters -- start -306> 2014-08-13 18:06:09.760277 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -305> 2014-08-13 18:06:09.760204 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] state: peer osd.296 activated and committed -304> 2014-08-13 18:06:09.760343 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194493/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] all_activated_and_committed -303> 2014-08-13 18:06:09.760290 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 55 notify: (query_epoch:194497, epoch_sent:194497, info:24.1f7( v 194422'15456 (192718'12872,194422'15456] local-les=194493 n=63 ec=192688 les/c 194493/194493 194495/194496/194495)) -302> 2014-08-13 18:06:09.760596 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 crt=194422'17433 lcod 0'0 inactive m=51] _activate_committed 194497 telling primary -301> 2014-08-13 18:06:09.760630 7f5c88628700 20 osd.4 194497 _share_map_outgoing 0x3c527e0 don't know epoch, doing nothing -300> 2014-08-13 18:06:09.760634 7f5c88628700 1 -- 192.168.1.115:6813/37708 --> osd.356 192.168.1.108:6832/5235 -- pg_info(1 pgs e194497:24.350) v4 -- ?+0 0x5a6d780 -299> 2014-08-13 18:06:09.760650 7f5c88628700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] flushed -298> 2014-08-13 18:06:09.760536 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] share_pg_info -297> 2014-08-13 18:06:09.760678 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4854a40 don't know epoch, doing nothing -296> 2014-08-13 18:06:09.760681 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.296 192.168.1.109:6806/31286 -- pg_info(1 pgs e194497:24.181) v4 -- ?+0 0x5a68fc0 -295> 2014-08-13 18:06:09.760696 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12520 don't know epoch, doing nothing -294> 2014-08-13 18:06:09.760698 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.320 192.168.1.106:6820/10050 -- pg_info(1 pgs e194497:24.181) v4 -- ?+0 0x5a6bd40 -293> 2014-08-13 18:06:09.760711 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4aff220 don't know epoch, doing nothing -292> 2014-08-13 18:06:09.760713 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.344 192.168.1.106:6812/6757 -- pg_info(1 pgs e194497:24.181) v4 -- ?+0 0x5a6b480 -291> 2014-08-13 18:06:09.760545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: Active: got notify from 55, already have info from that osd, ignoring -290> 2014-08-13 18:06:09.760733 7f5c7c7e4700 10 log is not dirty -289> 2014-08-13 18:06:09.760726 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] publish_stats_to_osd 194497:28888 -288> 2014-08-13 18:06:09.760918 7f5c7bde3700 10 log is not dirty -287> 2014-08-13 18:06:09.760928 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -286> 2014-08-13 18:06:09.760949 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] requeue_ops -285> 2014-08-13 18:06:09.760974 7f5c7bde3700 10 log is not dirty -284> 2014-08-13 18:06:09.760978 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e4( v 194422'18567 lc 194370'18476 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194492/194372 194495/194496/192978) [31,338,4] r=2 lpr=194496 pi=193125-194495/48 luod=0'0 crt=194422'18567 lcod 0'0 active m=46] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 31 info: 24.1e4( v 194422'18567 (192834'16484,194422'18567] local-les=194497 n=50 ec=192688 les/c 194497/194372 194495/194496/192978) -283> 2014-08-13 18:06:09.760739 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -282> 2014-08-13 18:06:09.761006 7f5c7bde3700 10 log is not dirty -281> 2014-08-13 18:06:09.761020 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -280> 2014-08-13 18:06:09.761034 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] requeue_ops -279> 2014-08-13 18:06:09.761044 7f5c7bde3700 10 log is not dirty -278> 2014-08-13 18:06:09.761052 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 18 ==== pg_info(1 pgs e194497:24.3b7) v4 ==== 748+0+0 (2409056284 0 0) 0x5a6c980 con 0x3c527e0 -277> 2014-08-13 18:06:09.761063 7f5c835ef700 10 osd.4 194497 do_waiters -- start -276> 2014-08-13 18:06:09.761065 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -275> 2014-08-13 18:06:09.761066 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6c980 pg_info(1 pgs e194497:24.3b7) v4 -274> 2014-08-13 18:06:09.761070 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.760997, event: header_read, request: pg_info(1 pgs e194497:24.3b7) v4 -273> 2014-08-13 18:06:09.761074 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.760999, event: throttled, request: pg_info(1 pgs e194497:24.3b7) v4 -272> 2014-08-13 18:06:09.761077 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.761039, event: all_read, request: pg_info(1 pgs e194497:24.3b7) v4 -271> 2014-08-13 18:06:09.761081 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.761063, event: dispatched, request: pg_info(1 pgs e194497:24.3b7) v4 -270> 2014-08-13 18:06:09.761084 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.761084, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.3b7) v4 -269> 2014-08-13 18:06:09.761088 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.3b7) v4 from osd.356 -268> 2014-08-13 18:06:09.761090 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6c980 -267> 2014-08-13 18:06:09.761093 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.761093, event: started, request: pg_info(1 pgs e194497:24.3b7) v4 -266> 2014-08-13 18:06:09.761105 7f5c835ef700 5 -- op tracker -- , seq: 696, time: 2014-08-13 18:06:09.761105, event: done, request: pg_info(1 pgs e194497:24.3b7) v4 -265> 2014-08-13 18:06:09.761102 7f5c803ea700 1 -- 192.168.1.115:6817/37708 <== osd.15 192.168.1.107:0/53408 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.761167) v2 ==== 47+0+0 (2313391166 0 0) 0x5a68e00 con 0x4c5d540 -264> 2014-08-13 18:06:09.761123 7f5c803ea700 1 -- 192.168.1.115:6817/37708 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194497 stamp 2014-08-13 18:06:09.761167) v2 -- ?+0 0x524d240 con 0x4c5d540 -263> 2014-08-13 18:06:09.761130 7f5c835ef700 10 osd.4 194497 do_waiters -- start -262> 2014-08-13 18:06:09.761132 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -261> 2014-08-13 18:06:09.761118 7f5c817ec700 1 -- 192.168.100.115:6806/37708 <== osd.15 192.168.1.107:0/53408 2 ==== osd_ping(ping e194497 stamp 2014-08-13 18:06:09.761167) v2 ==== 47+0+0 (2313391166 0 0) 0x5a681c0 con 0x4c5dc20 -260> 2014-08-13 18:06:09.761138 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 -259> 2014-08-13 18:06:09.761149 7f5c817ec700 1 -- 192.168.100.115:6806/37708 --> 192.168.1.107:0/53408 -- osd_ping(ping_reply e194497 stamp 2014-08-13 18:06:09.761167) v2 -- ?+0 0x524ee40 con 0x4c5dc20 -258> 2014-08-13 18:06:09.761170 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4854620 already has epoch 194497 -257> 2014-08-13 18:06:09.761049 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 348 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -256> 2014-08-13 18:06:09.761225 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.348 activated and committed -255> 2014-08-13 18:06:09.761444 7f5c7bde3700 10 log is not dirty -254> 2014-08-13 18:06:09.761452 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.33a( v 194422'16034 lc 194215'15912 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194493/194489 194495/194496/193250) [304,323,4] r=2 lpr=194496 pi=193250-194495/36 luod=0'0 crt=194422'16034 lcod 0'0 active m=61] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 304 info: 24.33a( v 194422'16034 (192834'14161,194422'16034] local-les=194497 n=60 ec=192688 les/c 194497/194489 194495/194496/193250) -253> 2014-08-13 18:06:09.761483 7f5c7bde3700 10 log is not dirty -252> 2014-08-13 18:06:09.761498 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -251> 2014-08-13 18:06:09.761511 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] requeue_ops -250> 2014-08-13 18:06:09.761522 7f5c7bde3700 10 log is not dirty -249> 2014-08-13 18:06:09.760991 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] requeue_ops -248> 2014-08-13 18:06:09.761526 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2ec( v 194422'16449 lc 194370'16342 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194493/194493 194495/194496/193232) [374,4,314]/[374,4,12] r=1 lpr=194496 pi=192688-194495/54 luod=0'0 crt=194422'16449 lcod 0'0 active+remapped m=52] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 374 info: 24.2ec( v 194422'16449 (192834'14495,194422'16449] local-les=194497 n=52 ec=192688 les/c 194497/194493 194495/194496/193232) -247> 2014-08-13 18:06:09.761535 7f5c7c7e4700 10 log is not dirty -246> 2014-08-13 18:06:09.761552 7f5c7bde3700 10 log is not dirty -245> 2014-08-13 18:06:09.761663 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 19 ==== pg_info(1 pgs e194497:24.350) v4 ==== 748+0+0 (1110271851 0 0) 0x5a6d780 con 0x3c527e0 -244> 2014-08-13 18:06:09.761677 7f5c835ef700 10 osd.4 194497 do_waiters -- start -243> 2014-08-13 18:06:09.761680 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -242> 2014-08-13 18:06:09.761681 7f5c835ef700 20 osd.4 194497 _dispatch 0x5a6d780 pg_info(1 pgs e194497:24.350) v4 -241> 2014-08-13 18:06:09.761685 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761601, event: header_read, request: pg_info(1 pgs e194497:24.350) v4 -240> 2014-08-13 18:06:09.761690 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761603, event: throttled, request: pg_info(1 pgs e194497:24.350) v4 -239> 2014-08-13 18:06:09.761694 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761649, event: all_read, request: pg_info(1 pgs e194497:24.350) v4 -238> 2014-08-13 18:06:09.761698 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761677, event: dispatched, request: pg_info(1 pgs e194497:24.350) v4 -237> 2014-08-13 18:06:09.761701 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761701, event: waiting_for_osdmap, request: pg_info(1 pgs e194497:24.350) v4 -236> 2014-08-13 18:06:09.761705 7f5c835ef700 7 osd.4 194497 handle_pg_info pg_info(1 pgs e194497:24.350) v4 from osd.356 -235> 2014-08-13 18:06:09.761565 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 331 info: 24.1bb( v 194422'12517 (192718'9695,194422'12517] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -234> 2014-08-13 18:06:09.761708 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x5a6d780 -233> 2014-08-13 18:06:09.761713 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761713, event: started, request: pg_info(1 pgs e194497:24.350) v4 -232> 2014-08-13 18:06:09.761544 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 327 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -231> 2014-08-13 18:06:09.761727 7f5c835ef700 5 -- op tracker -- , seq: 697, time: 2014-08-13 18:06:09.761727, event: done, request: pg_info(1 pgs e194497:24.350) v4 -230> 2014-08-13 18:06:09.761749 7f5c835ef700 10 osd.4 194497 do_waiters -- start -229> 2014-08-13 18:06:09.761751 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -228> 2014-08-13 18:06:09.761713 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] state: peer osd.331 activated and committed -227> 2014-08-13 18:06:09.761727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 327, already have info from that osd, ignoring -226> 2014-08-13 18:06:09.761905 7f5c7c7e4700 10 log is not dirty -225> 2014-08-13 18:06:09.761911 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -224> 2014-08-13 18:06:09.761923 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring -223> 2014-08-13 18:06:09.761933 7f5c7c7e4700 10 log is not dirty -222> 2014-08-13 18:06:09.761937 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 342 notify: (query_epoch:194497, epoch_sent:194497, info:22.707( v 194215'483 (0'0,194215'483] local-les=194493 n=19 ec=183096 les/c 194493/194493 194495/194495/194495)) -221> 2014-08-13 18:06:09.761948 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.707( v 194215'483 (0'0,194215'483] local-les=194496 n=19 ec=183096 les/c 194496/194493 194495/194495/194495) [4,310,21] r=0 lpr=194495 pi=193306-194494/37 crt=191727'417 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 342, already have info from that osd, ignoring -220> 2014-08-13 18:06:09.761958 7f5c7c7e4700 10 log is not dirty -219> 2014-08-13 18:06:09.761857 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194493/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] all_activated_and_committed -218> 2014-08-13 18:06:09.762034 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.289 192.168.1.113:6808/15284 17 ==== MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 ==== 26+0+0 (842568772 0 0) 0x588f540 con 0x445f7a0 -217> 2014-08-13 18:06:09.762048 7f5c835ef700 10 osd.4 194497 do_waiters -- start -216> 2014-08-13 18:06:09.762050 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -215> 2014-08-13 18:06:09.762052 7f5c835ef700 20 osd.4 194497 _dispatch 0x588f540 MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -214> 2014-08-13 18:06:09.762055 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.761977, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -213> 2014-08-13 18:06:09.762060 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.761978, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -212> 2014-08-13 18:06:09.762063 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.762021, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -211> 2014-08-13 18:06:09.762066 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.762048, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -210> 2014-08-13 18:06:09.762070 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.762070, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -209> 2014-08-13 18:06:09.762073 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x588f540 -208> 2014-08-13 18:06:09.762081 7f5c835ef700 5 -- op tracker -- , seq: 698, time: 2014-08-13 18:06:09.762081, event: done, request: MRecoveryReserve REQUEST pgid: 24.5e, query_epoch: 194497 v2 -207> 2014-08-13 18:06:09.762099 7f5c835ef700 10 osd.4 194497 do_waiters -- start -206> 2014-08-13 18:06:09.762101 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -205> 2014-08-13 18:06:09.761999 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] share_pg_info -204> 2014-08-13 18:06:09.762146 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4b12940 don't know epoch, doing nothing -203> 2014-08-13 18:06:09.762149 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.331 192.168.1.104:6816/837 -- pg_info(1 pgs e194497:24.1bb) v4 -- ?+0 0x5a6b100 -202> 2014-08-13 18:06:09.762174 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x4ab2d60 don't know epoch, doing nothing -201> 2014-08-13 18:06:09.761995 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 315 notify: (query_epoch:194497, epoch_sent:194497, info:24.b6( v 194422'15454 (192834'13289,194422'15454] local-les=194493 n=62 ec=192688 les/c 194493/194493 194495/194496/194495)) -200> 2014-08-13 18:06:09.762178 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.332 192.168.1.110:6806/42024 -- pg_info(1 pgs e194497:24.1bb) v4 -- ?+0 0x5354980 -199> 2014-08-13 18:06:09.762192 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x3c55540 don't know epoch, doing nothing -198> 2014-08-13 18:06:09.762195 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.337 192.168.1.104:6804/62197 -- pg_info(1 pgs e194497:24.1bb) v4 -- ?+0 0x5357e00 -197> 2014-08-13 18:06:09.762207 7f5c7bde3700 20 osd.4 194497 _share_map_outgoing 0x571b700 don't know epoch, doing nothing -196> 2014-08-13 18:06:09.762209 7f5c7bde3700 1 -- 192.168.1.115:6813/37708 --> osd.348 192.168.1.110:6800/41204 -- pg_info(1 pgs e194497:24.1bb) v4 -- ?+0 0x5355e80 -195> 2014-08-13 18:06:09.762184 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: Active: got notify from 315, already have info from that osd, ignoring -194> 2014-08-13 18:06:09.762362 7f5c7c7e4700 10 log is not dirty -193> 2014-08-13 18:06:09.762220 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] publish_stats_to_osd 194497:22156 -192> 2014-08-13 18:06:09.762369 7f5c7bde3700 10 log is not dirty -191> 2014-08-13 18:06:09.762375 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -190> 2014-08-13 18:06:09.762388 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] requeue_ops -189> 2014-08-13 18:06:09.762398 7f5c7bde3700 10 log is not dirty -188> 2014-08-13 18:06:09.762402 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.cd( v 194422'15992 lc 194215'15886 (192718'12885,194422'15992] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/192967) [6,374,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'15992 lcod 0'0 active m=53] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 6 info: 24.cd( v 194422'15992 (192718'12889,194422'15992] local-les=194497 n=53 ec=192688 les/c 194497/194489 194495/194496/192967) -187> 2014-08-13 18:06:09.762432 7f5c7bde3700 10 log is not dirty -186> 2014-08-13 18:06:09.762446 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -185> 2014-08-13 18:06:09.762459 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] requeue_ops -184> 2014-08-13 18:06:09.762470 7f5c7bde3700 10 log is not dirty -183> 2014-08-13 18:06:09.762474 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.e2( v 194422'15014 lc 194370'14879 (192834'11882,194422'15014] local-les=194497 n=69 ec=192688 les/c 194493/194493 194495/194496/192955) [346,4,304]/[346,4,288] r=1 lpr=194496 pi=193125-194495/38 luod=0'0 crt=194422'15014 lcod 0'0 active+remapped m=66] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 346 info: 24.e2( v 194422'15014 (192834'12106,194422'15014] local-les=194497 n=69 ec=192688 les/c 194497/194493 194495/194496/192955) -182> 2014-08-13 18:06:09.762500 7f5c7bde3700 10 log is not dirty -181> 2014-08-13 18:06:09.762513 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -180> 2014-08-13 18:06:09.762525 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] requeue_ops -179> 2014-08-13 18:06:09.762535 7f5c7bde3700 10 log is not dirty -178> 2014-08-13 18:06:09.762368 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -177> 2014-08-13 18:06:09.762543 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1e6( v 194422'17642 lc 194370'17543 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194493/194489 194495/194496/192975) [285,336,4] r=2 lpr=194496 pi=193125-194495/36 luod=0'0 crt=194422'17642 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 285 info: 24.1e6( v 194422'17642 (192718'14692,194422'17642] local-les=194497 n=48 ec=192688 les/c 194497/194489 194495/194496/192975) -176> 2014-08-13 18:06:09.762571 7f5c7bde3700 10 log is not dirty -175> 2014-08-13 18:06:09.762584 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -174> 2014-08-13 18:06:09.762596 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] requeue_ops -173> 2014-08-13 18:06:09.762607 7f5c7bde3700 10 log is not dirty -172> 2014-08-13 18:06:09.762611 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1de( v 194422'17074 lc 194370'16977 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194493/194493 194495/194496/192966) [364,4,360]/[364,4,301] r=1 lpr=194496 pi=193125-194495/39 luod=0'0 crt=194422'17074 lcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 364 info: 24.1de( v 194422'17074 (192834'15290,194422'17074] local-les=194497 n=49 ec=192688 les/c 194497/194493 194495/194496/192966) -171> 2014-08-13 18:06:09.762640 7f5c7bde3700 10 log is not dirty -170> 2014-08-13 18:06:09.762666 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -169> 2014-08-13 18:06:09.762681 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] requeue_ops -168> 2014-08-13 18:06:09.762692 7f5c7bde3700 10 log is not dirty -167> 2014-08-13 18:06:09.762691 7f5c835ef700 1 -- 192.168.1.115:6813/37708 <== osd.356 192.168.1.108:6832/5235 20 ==== MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 ==== 26+0+0 (2422586993 0 0) 0x524cd00 con 0x3c527e0 -166> 2014-08-13 18:06:09.762696 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.2a1( v 194422'14353 lc 194370'14256 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194493/194489 194495/194496/193188) [17,4,395] r=1 lpr=194496 pi=193188-194495/42 luod=0'0 crt=194422'14353 lcod 0'0 active m=49] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 17 info: 24.2a1( v 194422'14353 (192718'11272,194422'14353] local-les=194497 n=52 ec=192688 les/c 194497/194489 194495/194496/193188) -165> 2014-08-13 18:06:09.762706 7f5c835ef700 10 osd.4 194497 do_waiters -- start -164> 2014-08-13 18:06:09.762709 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -163> 2014-08-13 18:06:09.762711 7f5c835ef700 20 osd.4 194497 _dispatch 0x524cd00 MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -162> 2014-08-13 18:06:09.762545 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.330 activated and committed -161> 2014-08-13 18:06:09.762715 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762646, event: header_read, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -160> 2014-08-13 18:06:09.762722 7f5c7c7e4700 10 log is not dirty -159> 2014-08-13 18:06:09.762721 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762647, event: throttled, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -158> 2014-08-13 18:06:09.762723 7f5c7bde3700 10 log is not dirty -157> 2014-08-13 18:06:09.762725 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762677, event: all_read, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -156> 2014-08-13 18:06:09.762729 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762705, event: dispatched, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -155> 2014-08-13 18:06:09.762733 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762732, event: waiting_for_osdmap, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -154> 2014-08-13 18:06:09.762737 7f5c835ef700 15 osd.4 194497 require_same_or_newer_map 194497 (i am 194497) 0x524cd00 -153> 2014-08-13 18:06:09.762747 7f5c835ef700 5 -- op tracker -- , seq: 699, time: 2014-08-13 18:06:09.762747, event: done, request: MRecoveryReserve REQUEST pgid: 24.3b7, query_epoch: 194497 v2 -152> 2014-08-13 18:06:09.762772 7f5c835ef700 10 osd.4 194497 do_waiters -- start -151> 2014-08-13 18:06:09.762774 7f5c835ef700 10 osd.4 194497 do_waiters -- finish -150> 2014-08-13 18:06:09.762739 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated -149> 2014-08-13 18:06:09.762738 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 28 info: 24.b6( v 194422'15454 (192834'13289,194422'15454] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -148> 2014-08-13 18:06:09.762920 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] check_local -147> 2014-08-13 18:06:09.762922 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.b6( v 194422'15454 lc 194370'15335 (192834'13289,194422'15454] local-les=194497 n=62 ec=192688 les/c 194493/194493 194495/194496/194495) [4,330,28]/[4,17,315] r=0 lpr=194496 pi=193748-194495/41 bft=28,330 crt=194422'15454 lcod 0'0 mlcod 0'0 remapped m=59] state: peer osd.28 activated and committed -146> 2014-08-13 18:06:09.763098 7f5c7c7e4700 10 log is not dirty -145> 2014-08-13 18:06:09.763103 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MNotifyRec from 303 notify: (query_epoch:194497, epoch_sent:194497, info:22.3e4( v 194215'518 (0'0,194215'518] local-les=194493 n=32 ec=183096 les/c 194493/194493 194495/194495/194495)) -144> 2014-08-13 18:06:09.763114 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[22.3e4( v 194215'518 (0'0,194215'518] local-les=194496 n=32 ec=183096 les/c 194496/194493 194495/194495/194495) [4,313,318] r=0 lpr=194495 pi=193306-194494/37 crt=191727'442 lcod 0'0 mlcod 0'0 active+recovery_wait] state: Active: got notify from 303, already have info from that osd, ignoring -143> 2014-08-13 18:06:09.763124 7f5c7c7e4700 10 log is not dirty -142> 2014-08-13 18:06:09.763091 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] requeue_ops -141> 2014-08-13 18:06:09.763127 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 59 info: 24.1f7( v 194422'15456 (192718'12872,194422'15456] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194493 194495/194496/194495) -140> 2014-08-13 18:06:09.763265 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] needs_recovery primary has 57 missing -139> 2014-08-13 18:06:09.763352 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] state: peer osd.59 activated and committed -138> 2014-08-13 18:06:09.763663 7f5c7c7e4700 10 log is not dirty -137> 2014-08-13 18:06:09.763537 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] activate not all replicas are up-to-date, queueing recovery -136> 2014-08-13 18:06:09.763671 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -135> 2014-08-13 18:06:09.763996 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.1f7( v 194422'15456 lc 194370'15329 (192718'12872,194422'15456] local-les=194497 n=63 ec=192688 les/c 194493/194493 194495/194496/194495) [4,59,351]/[4,351,55] r=0 lpr=194496 pi=193678-194495/41 bft=59 crt=194422'15456 lcod 0'0 mlcod 0'0 remapped m=62] requeue_ops -134> 2014-08-13 18:06:09.764325 7f5c7c7e4700 10 log is not dirty -133> 2014-08-13 18:06:09.764333 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -132> 2014-08-13 18:06:09.764355 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] requeue_ops -131> 2014-08-13 18:06:09.764372 7f5c7c7e4700 10 log is not dirty -130> 2014-08-13 18:06:09.764379 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.369( v 194422'13576 lc 194370'13477 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194493/194489 194495/194496/193186) [45,319,4] r=2 lpr=194496 pi=193186-194495/36 luod=0'0 crt=194422'13576 lcod 0'0 active m=50] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 45 info: 24.369( v 194422'13576 (192834'11428,194422'13576] local-les=194497 n=51 ec=192688 les/c 194497/194489 194495/194496/193186) -129> 2014-08-13 18:06:09.764122 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] publish_stats_to_osd 194497:29712 -128> 2014-08-13 18:06:09.764434 7f5c7c7e4700 10 log is not dirty -127> 2014-08-13 18:06:09.764460 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -126> 2014-08-13 18:06:09.764485 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] requeue_ops -125> 2014-08-13 18:06:09.764510 7f5c7c7e4700 10 log is not dirty -124> 2014-08-13 18:06:09.764514 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -123> 2014-08-13 18:06:09.764525 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] requeue_ops -122> 2014-08-13 18:06:09.764535 7f5c7c7e4700 10 log is not dirty -121> 2014-08-13 18:06:09.764425 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] on activate: bft=25,362 from 0//0//-1 -120> 2014-08-13 18:06:09.764539 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 55 info: 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194492 194495/194496/194495) -119> 2014-08-13 18:06:09.764635 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 25 from 0//0//-1 -118> 2014-08-13 18:06:09.764727 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: peer osd.55 activated and committed -117> 2014-08-13 18:06:09.764996 7f5c7c7e4700 10 log is not dirty -116> 2014-08-13 18:06:09.765006 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194493/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 15 info: 24.20e( v 194422'11806 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) -115> 2014-08-13 18:06:09.765056 7f5c7c7e4700 10 log is not dirty -114> 2014-08-13 18:06:09.764913 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] target shard 362 from 0//0//-1 -113> 2014-08-13 18:06:09.765079 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 61 info: 24.136( v 194422'15470 (192834'12379,194422'15470] lb 0//0//-1 local-les=0 n=0 ec=192688 les/c 194497/194492 194495/194496/194495) -112> 2014-08-13 18:06:09.765189 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -111> 2014-08-13 18:06:09.765312 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] state: peer osd.61 activated and committed -110> 2014-08-13 18:06:09.765560 7f5c7c7e4700 10 log is not dirty -109> 2014-08-13 18:06:09.765565 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery -108> 2014-08-13 18:06:09.765577 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 0.017708 3 0.001459 -107> 2014-08-13 18:06:09.765590 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.20e( v 194422'11806 lc 194370'11691 (192718'8900,194422'11806] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/193264) [15,4,52]/[15,4,61] r=1 lpr=194496 pi=193264-194495/40 luod=0'0 crt=194422'11806 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved -106> 2014-08-13 18:06:09.765604 7f5c7c7e4700 10 log is not dirty -105> 2014-08-13 18:06:09.765501 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_create target_size 1000 fpp 0.05 -104> 2014-08-13 18:06:09.765608 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -103> 2014-08-13 18:06:09.765839 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.136( v 194422'15470 lc 194370'15367 (192834'12379,194422'15470] local-les=194497 n=52 ec=192688 les/c 194492/194492 194495/194496/194495) [4,55,61]/[4,50,65] r=0 lpr=194496 pi=193747-194495/37 bft=55,61 crt=194422'15470 lcod 0'0 mlcod 0'0 remapped m=51] requeue_ops -102> 2014-08-13 18:06:09.766095 7f5c7c7e4700 10 log is not dirty -101> 2014-08-13 18:06:09.766103 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.3f7( v 194422'15537 lc 194370'15408 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194493/194493 194495/194496/193227) [330,4,288]/[330,4,57] r=1 lpr=194496 pi=193227-194495/45 luod=0'0 crt=194422'15537 lcod 0'0 active+remapped m=64] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 330 info: 24.3f7( v 194422'15537 (192718'12872,194422'15537] local-les=194497 n=66 ec=192688 les/c 194497/194493 194495/194496/193227) -100> 2014-08-13 18:06:09.766154 7f5c7c7e4700 10 log is not dirty -99> 2014-08-13 18:06:09.766071 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] hit_set_apply_log 194422'18731 .. 194422'18731 -98> 2014-08-13 18:06:09.766167 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 remapped m=59] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated -97> 2014-08-13 18:06:09.766341 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_setup allocated new state, position a4b1d83d//0//24 -96> 2014-08-13 18:06:09.766402 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] check_local -95> 2014-08-13 18:06:09.766539 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode: num_objects: 59 num_bytes: 243269718 num_objects_dirty: 58 num_objects_omap: 0 num_dirty: 58 num_user_objects: 58 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -94> 2014-08-13 18:06:09.766625 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] requeue_ops -93> 2014-08-13 18:06:09.766827 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] agent_choose_mode dirty 0.059426 full 0.059426 -92> 2014-08-13 18:06:09.767064 7f5c7bde3700 10 log is not dirty -91> 2014-08-13 18:06:09.767077 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -90> 2014-08-13 18:06:09.767110 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] requeue_ops -89> 2014-08-13 18:06:09.767120 7f5c7bde3700 10 log is not dirty -88> 2014-08-13 18:06:09.767124 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -87> 2014-08-13 18:06:09.767135 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] requeue_ops -86> 2014-08-13 18:06:09.767145 7f5c7bde3700 10 log is not dirty -85> 2014-08-13 18:06:09.766916 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] needs_recovery primary has 59 missing -84> 2014-08-13 18:06:09.767149 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194493/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 289 info: 24.5e( v 194422'17034 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) -83> 2014-08-13 18:06:09.767183 7f5c7bde3700 10 log is not dirty -82> 2014-08-13 18:06:09.767197 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 FlushedEvt -81> 2014-08-13 18:06:09.767210 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] requeue_ops -80> 2014-08-13 18:06:09.767220 7f5c7bde3700 10 log is not dirty -79> 2014-08-13 18:06:09.767154 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] activate not all replicas are up-to-date, queueing recovery -78> 2014-08-13 18:06:09.767224 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated -77> 2014-08-13 18:06:09.767369 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] check_local -76> 2014-08-13 18:06:09.767587 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] requeue_ops -75> 2014-08-13 18:06:09.767574 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] publish_stats_to_osd 194497:25295 -74> 2014-08-13 18:06:09.767818 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] needs_recovery primary has 47 missing -73> 2014-08-13 18:06:09.767872 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] on activate: bft=367,378 from 0//0//-1 -72> 2014-08-13 18:06:09.768048 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] activate not all replicas are up-to-date, queueing recovery -71> 2014-08-13 18:06:09.768139 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 367 from 0//0//-1 -70> 2014-08-13 18:06:09.768482 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] publish_stats_to_osd 194497:28889 -69> 2014-08-13 18:06:09.768437 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] target shard 378 from 0//0//-1 -68> 2014-08-13 18:06:09.768728 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] on activate: bft=344 from 0//0//-1 -67> 2014-08-13 18:06:09.768737 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -66> 2014-08-13 18:06:09.768961 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] target shard 344 from 0//0//-1 -65> 2014-08-13 18:06:09.769034 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_create target_size 1000 fpp 0.05 -64> 2014-08-13 18:06:09.769210 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -63> 2014-08-13 18:06:09.769426 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_create target_size 1000 fpp 0.05 -62> 2014-08-13 18:06:09.769620 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] hit_set_apply_log 194422'17302 .. 194422'17302 -61> 2014-08-13 18:06:09.769969 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] hit_set_apply_log 194422'19613 .. 194422'19613 -60> 2014-08-13 18:06:09.769933 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_setup allocated new state, position e6e6749b//0//24 -59> 2014-08-13 18:06:09.770223 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_setup allocated new state, position 344c7181//0//24 -58> 2014-08-13 18:06:09.770465 7f5c7bde3700 0 log [WRN] : pg 24.181 has invalid (post-split) stats; must scrub before tier agent can activate -57> 2014-08-13 18:06:09.770219 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode: num_objects: 61 num_bytes: 251658355 num_objects_dirty: 60 num_objects_omap: 0 num_dirty: 60 num_user_objects: 60 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -56> 2014-08-13 18:06:09.770468 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode: num_objects: 42 num_bytes: 388397714 num_objects_dirty: 42 num_objects_omap: 0 num_dirty: 42 num_user_objects: 42 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -55> 2014-08-13 18:06:09.770493 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.9b( v 194422'17302 lc 194370'17185 (192834'15069,194422'17302] local-les=194497 n=61 ec=192688 les/c 194497/194493 194495/194496/194495) [4,378,367]/[4,291,350] r=0 lpr=194496 pi=193637-194495/36 bft=367,378 crt=194422'17302 lcod 0'0 mlcod 0'0 active+remapped m=59] agent_choose_mode dirty 0.061475 full 0.061475 -54> 2014-08-13 18:06:09.770720 7f5c7c7e4700 10 log is not dirty -53> 2014-08-13 18:06:09.770740 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 degraded+remapped] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated -52> 2014-08-13 18:06:09.770763 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] check_local -51> 2014-08-13 18:06:09.770780 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] requeue_ops -50> 2014-08-13 18:06:09.770796 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_recovery is recovered -49> 2014-08-13 18:06:09.770813 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.26 has last_backfill 0//0//-1 -48> 2014-08-13 18:06:09.770829 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.30 has last_backfill 0//0//-1 -47> 2014-08-13 18:06:09.770846 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] needs_backfill osd.340 has last_backfill 0//0//-1 -46> 2014-08-13 18:06:09.770638 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode dirty 0.043032 full 0.043032 -45> 2014-08-13 18:06:09.770862 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] activate queueing backfill -44> 2014-08-13 18:06:09.770888 7f5c7c7e4700 15 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] publish_stats_to_osd 194497:23848 -43> 2014-08-13 18:06:09.770907 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] on activate: bft=26,30,340 from 0//0//-1 -42> 2014-08-13 18:06:09.770924 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 26 from 0//0//-1 -41> 2014-08-13 18:06:09.770940 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 30 from 0//0//-1 -40> 2014-08-13 18:06:09.770956 7f5c7c7e4700 5 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] target shard 340 from 0//0//-1 -39> 2014-08-13 18:06:09.770978 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -38> 2014-08-13 18:06:09.771001 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_create target_size 1000 fpp 0.05 -37> 2014-08-13 18:06:09.770863 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] agent_choose_mode stats invalid (post-split), idle -36> 2014-08-13 18:06:09.771109 7f5c7bde3700 10 log is not dirty -35> 2014-08-13 18:06:09.771118 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194493/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.3b7( v 194422'18845 (192834'17056,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) -34> 2014-08-13 18:06:09.771178 7f5c7bde3700 10 log is not dirty -33> 2014-08-13 18:06:09.771206 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.350( v 194422'17433 lc 194370'17332 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194493/194489 194495/194496/193151) [356,364,4] r=2 lpr=194496 pi=193151-194495/36 luod=0'0 crt=194422'17433 lcod 0'0 active m=51] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 MInfoRec from 356 info: 24.350( v 194422'17433 (192834'15725,194422'17433] local-les=194497 n=53 ec=192688 les/c 194497/194489 194495/194496/193151) -32> 2014-08-13 18:06:09.771265 7f5c7bde3700 10 log is not dirty -31> 2014-08-13 18:06:09.771278 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery -30> 2014-08-13 18:06:09.771292 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] exit Started/ReplicaActive/RepNotRecovering 0.013370 3 0.001095 -29> 2014-08-13 18:06:09.771305 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.5e( v 194422'17034 lc 194215'16908 (192718'14372,194422'17034] local-les=194497 n=63 ec=192688 les/c 194497/194489 194495/194496/192968) [289,4,285] r=1 lpr=194496 pi=193125-194495/49 luod=0'0 crt=194422'17034 lcod 0'0 active m=63] enter Started/ReplicaActive/RepWaitRecoveryReserved -28> 2014-08-13 18:06:09.771318 7f5c7bde3700 10 log is not dirty -27> 2014-08-13 18:06:09.771306 7f5c7c7e4700 20 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] hit_set_apply_log 194215'14514 .. 194215'14514 -26> 2014-08-13 18:06:09.771325 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_setup allocated new state, position 50f0e434//0//24 -25> 2014-08-13 18:06:09.771338 7f5c7c7e4700 0 log [WRN] : pg 24.34 has invalid (post-split) stats; must scrub before tier agent can activate -24> 2014-08-13 18:06:09.771341 7f5c7c7e4700 10 osd.4 pg_epoch: 194497 pg[24.34( v 194215'14514 (192718'11510,194215'14514] local-les=194497 n=-1 ec=192688 les/c 194497/194258 194459/194496/194496) [26,30,340]/[4] r=0 lpr=194496 pi=193125-194495/54 bft=26,30,340 crt=193305'14233 lcod 0'0 mlcod 0'0 active+degraded+remapped] agent_choose_mode: num_objects: -1 num_bytes: -9223372036854775808 num_objects_dirty: -2 num_objects_omap: 0 num_dirty: 0 num_user_objects: 18446744073709551615 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -23> 2014-08-13 18:06:09.771322 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 AllReplicasActivated -22> 2014-08-13 18:06:09.771535 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] check_local -21> 2014-08-13 18:06:09.771716 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] requeue_ops -20> 2014-08-13 18:06:09.771952 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] needs_recovery primary has 48 missing -19> 2014-08-13 18:06:09.772212 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] activate not all replicas are up-to-date, queueing recovery -18> 2014-08-13 18:06:09.772700 7f5c7bde3700 15 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] publish_stats_to_osd 194497:22157 -17> 2014-08-13 18:06:09.772953 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] on activate: bft=331,348 from 0//0//-1 -16> 2014-08-13 18:06:09.773211 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 331 from 0//0//-1 -15> 2014-08-13 18:06:09.773454 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] target shard 348 from 0//0//-1 -14> 2014-08-13 18:06:09.773690 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_create bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} -13> 2014-08-13 18:06:09.773948 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_create target_size 1000 fpp 0.05 -12> 2014-08-13 18:06:09.774504 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] hit_set_apply_log 194422'12517 .. 194422'12517 -11> 2014-08-13 18:06:09.774773 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_setup allocated new state, position d8d705bb//0//24 -10> 2014-08-13 18:06:09.774994 7f5c7bde3700 0 log [WRN] : pg 24.1bb has invalid (post-split) stats; must scrub before tier agent can activate -9> 2014-08-13 18:06:09.774997 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode: num_objects: 50 num_bytes: 205520955 num_objects_dirty: 54 num_objects_omap: 0 num_dirty: 54 num_user_objects: 50 pool.info.target_max_bytes: 900000000000000000 pool.info.target_max_objects: 1000000 -8> 2014-08-13 18:06:09.775204 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode dirty 0.055327 full 0.051229 -7> 2014-08-13 18:06:09.775443 7f5c7bde3700 20 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] agent_choose_mode stats invalid (post-split), idle -6> 2014-08-13 18:06:09.775687 7f5c7bde3700 10 log is not dirty -5> 2014-08-13 18:06:09.775715 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 RequestRecovery -4> 2014-08-13 18:06:09.775740 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] exit Started/ReplicaActive/RepNotRecovering 0.016634 3 0.000835 -3> 2014-08-13 18:06:09.775763 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3b7( v 194422'18845 lc 194217'18730 (192718'15726,194422'18845] local-les=194497 n=56 ec=192688 les/c 194497/194493 194495/194496/192969) [356,4,334]/[356,4,32] r=1 lpr=194496 pi=193150-194495/45 luod=0'0 crt=194422'18845 lcod 0'0 active+remapped m=56] enter Started/ReplicaActive/RepWaitRecoveryReserved -2> 2014-08-13 18:06:09.775784 7f5c7bde3700 10 log is not dirty -1> 2014-08-13 18:06:09.775791 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 DoRecovery 0> 2014-08-13 18:06:09.776105 7f5c7c7e4700 -1 *** Caught signal (Floating point exception) ** in thread 7f5c7c7e4700 ceph version 0.80.5 (38b73c67d375a2552d8ed67843c8a65c2c0feba6) 1: /usr/bin/ceph-osd() [0x9ae061] 2: (()+0xf710) [0x7f5c96352710] 3: (ReplicatedPG::agent_choose_mode(bool)+0xa6e) [0x84f57e] 4: (ReplicatedPG::agent_setup()+0x1d2) [0x866d72] 5: (ReplicatedPG::on_activate()+0x457) [0x8677a7] 6: (PG::RecoveryState::Active::react(PG::AllReplicasActivated const&)+0x73) [0x7dd173] 7: (boost::statechart::detail::reaction_result boost::statechart::simple_state::local_react_impl_non_empty::local_react_impl, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, boost::statechart::custom_reaction, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::statechart::simple_state >(boost::statechart::simple_state&, boost::statechart::event_base const&, void const*)+0x36c) [0x844fec] 8: (boost::statechart::simple_state::react_impl(boost::statechart::event_base const&, void const*)+0x26) [0x845066] 9: (boost::statechart::state_machine, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&)+0x4b) [0x83b4db] 10: (PG::handle_peering_event(std::tr1::shared_ptr, PG::RecoveryCtx*)+0x32f) [0x7f756f] 11: (OSD::process_peering_events(std::list > const&, ThreadPool::TPHandle&)+0x2f2) [0x6475b2] 12: (OSD::PeeringWQ::_process(std::list > const&, ThreadPool::TPHandle&)+0x16) [0x69e686] 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0x551) [0xab8561] 14: (ThreadPool::WorkThread::entry()+0x10) [0xabb5a0] 15: (()+0x79d1) [0x7f5c9634a9d1] 16: (clone()+0x6d) [0x7f5c952dab6d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 20/20 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.4.log --- end dump of recent events --- 2014-08-13 18:06:09.776096 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] exit Started/Primary/Active/Activating 0.140395 11 0.090274 2014-08-13 18:06:09.776392 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.3d( v 194422'18731 lc 194370'18616 (192803'16165,194422'18731] local-les=194497 n=59 ec=192688 les/c 194497/194493 194495/194496/194495) [4,25,362]/[4,20,303] r=0 lpr=194496 pi=193635-194495/39 bft=25,362 crt=194422'18731 lcod 0'0 mlcod 0'0 active+remapped m=57] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:09.776673 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.776678 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 DoRecovery 2014-08-13 18:06:09.776912 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] exit Started/Primary/Active/Activating 0.086152 12 0.049493 2014-08-13 18:06:09.777141 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.181( v 194422'19613 lc 194370'19520 (192834'17288,194422'19613] local-les=194497 n=42 ec=192688 les/c 194497/194466 194495/194496/194496) [4,344,296]/[4,296,320] r=0 lpr=194496 pi=193679-194495/42 bft=344 crt=194422'19613 lcod 0'0 mlcod 0'0 active+remapped m=47] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:09.777378 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.777386 7f5c7bde3700 10 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] handle_peering_event: epoch_sent: 194497 epoch_requested: 194497 DoRecovery 2014-08-13 18:06:09.777638 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] exit Started/Primary/Active/Activating 0.049515 11 0.038214 2014-08-13 18:06:09.777887 7f5c7bde3700 5 osd.4 pg_epoch: 194497 pg[24.1bb( v 194422'12517 lc 194370'12420 (192718'9695,194422'12517] local-les=194497 n=50 ec=192688 les/c 194497/194493 194495/194496/194495) [4,348,331]/[4,332,337] r=0 lpr=194496 pi=193644-194495/37 bft=331,348 crt=194422'12517 lcod 0'0 mlcod 0'0 active+remapped m=48] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-13 18:06:09.778123 7f5c7bde3700 10 log is not dirty 2014-08-13 18:06:09.811544 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.811617 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x48506e0 already has epoch 194497 2014-08-13 18:06:09.896879 7f5c803ea700 20 osd.4 194497 _share_map_outgoing 0x4b10b00 already has epoch 194497 2014-08-13 18:06:09.896943 7f5c817ec700 20 osd.4 194497 _share_map_outgoing 0x4b10b00 already has epoch 194497